2024-09-21T03:23:08.709 INFO:root:teuthology version: 1.2.2.dev2+g3752d38 2024-09-21T03:23:08.737 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-09-21T03:23:08.832 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/pdonnell-2024-09-20_23:48:22-fs-wip-pdonnell-testing-20240920.212106-debug-distro-default-smithi/7913471 branch: wip-pdonnell-testing-20240920.212106-debug description: fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v1} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} ms_mode/secure wsync/yes} objectstore-ec/bluestore-bitmap omap_limit/10 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/random export-check n/5 replication/always} standby-replay tasks/{0-subvolume/{with-namespace-isolated-and-quota} 1-check-counter 2-scrub/yes 3-snaps/yes 4-flush/no 5-quiesce/no 6-workunit/suites/iozone}} email: pdonnell@redhat.com first_in_suite: false job_id: '7913471' kernel: client: sha1: distro kdb: 1 sha1: distro ktype: distro last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi name: pdonnell-2024-09-20_23:48:22-fs-wip-pdonnell-testing-20240920.212106-debug-distro-default-smithi no_nested_subset: false openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: centos os_version: 9.stream overrides: admin_socket: branch: wip-pdonnell-testing-20240920.212106-debug ceph: cephfs: max_mds: 5 session_timeout: 300 standby_replay: true cluster-conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 rados mon op timeout: 900 rados osd op timeout: 900 mds: debug mds: 20 debug mds balancer: 20 debug ms: 2 mds bal fragment size max: 10000 mds bal merge size: 5 mds bal split bits: 3 mds bal split size: 100 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: client mount timeout: 30 debug cephsqlite: 20 debug client: 20 debug mgr: 20 debug ms: 1 mgr/snap_schedule/allow_m_granularity: true mgr/snap_schedule/dump_on_update: true mgr/snap_schedule/log_level: debug mon warn on pool no app: false mon: mon op complaint time: 120 osd: osd op complaint time: 180 conf: global: ms die on skipped message: false mds: mds_bal_replicate_threshold: 1 mds_export_ephemeral_random_max: 0.1 mgr: cephsqlite lock renewal timeout: 900000 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon down mkfs grace: 300 osd: bdev async discard: true bdev enable discard: true bluestore allocator: bitmap bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug ms: 1 debug osd: 20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd shutdown pgref assert: true osd_max_omap_entries_per_request: 10 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - FS_DEGRADED - fs.*is degraded - filesystem is degraded - FS_INLINE_DATA_DEPRECATED - FS_WITH_FAILED_MDS - MDS_ALL_DOWN - filesystem is offline - MDS_DAMAGE - MDS_DEGRADED - MDS_FAILED - MDS_INSUFFICIENT_STANDBY - insufficient standby MDS daemons available - MDS_UP_LESS_THAN_MAX - online, but wants - filesystem is online with fewer MDS than max_mds - POOL_APP_NOT_ENABLED - do not have an application enabled - overall HEALTH_ - Replacing daemon - deprecated feature inline_data - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - PG_AVAILABILITY - PG_DEGRADED - Reduced data availability - Degraded data redundancy - slow metadata IO - SLOW_OPS - slow request mgr-modules: - snap_schedule sha1: 1843221895adccfb3b692060251dd6a0183bc410 subvols: create: 2 subvol_options: --namespace-isolated --size 25000000000 ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} osd: bdev async discard: true bdev enable discard: true bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs ceph-fuse: client.0: mount_subvol_num: 0 check-counter: counters: mds: - mds.exported - mds.imported - min: 3 name: mds_cache.dir_update - min: 3 name: mds_cache.dir_update_receipt - mds.root_rsnaps - mds_server.req_mksnap_latency.avgcount - mds_server.req_rmsnap_latency.avgcount dry_run: true install: ceph: flavor: default sha1: 1843221895adccfb3b692060251dd6a0183bc410 kclient: client.0: mount_subvol_num: 1 mntopts: - ms_mode=secure - wsync syntax: v1 selinux: allowlist: - scontext=system_u:system_r:logrotate_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-pdonnell-testing-20240920.212106-debug sha1: 1843221895adccfb3b692060251dd6a0183bc410 owner: scheduled_pdonnell@teuthology priority: 90 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - mgr.x - mds.a - mds.d - mds.g - mds.j - osd.0 - osd.3 - osd.6 - osd.9 - client.0 - - mon.b - mgr.y - mds.b - mds.e - mds.h - mds.k - osd.1 - osd.4 - osd.7 - osd.10 - - mon.c - mgr.z - mds.c - mds.f - mds.i - mds.l - osd.2 - osd.5 - osd.8 - osd.11 seed: 7864 sha1: 1843221895adccfb3b692060251dd6a0183bc410 sleep_before_teardown: 0 subset: 67/512 suite: fs suite_branch: wip-pdonnell-testing-20240920.212106-debug suite_path: /home/teuthworker/src/git.ceph.com_ceph-c_1843221895adccfb3b692060251dd6a0183bc410/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph-ci.git suite_sha1: 1843221895adccfb3b692060251dd6a0183bc410 syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi039.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBE6rLuY/X0JVqFFFpOJtZxIAzm5SSOA8sHRVjmvrzfJclGq6nMWnODKioFHdJC6VqUkI40WGvZ+R+7+Aw6Ux6Kg= smithi150.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBFO9bKRhgfp8hPwPdADAoG5VoX5oQiardE2NlftBR2jcBiCnpL1+xHqy3aq6FYt2kf5dnvBR6UGaACDi9LqCCfY= smithi161.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBOD130bkAshU+oUV3upBEA0sdb+J3gwqlrWHojRcamjslHlJagE7lu+jBi3OFlYnwhXUUMnmQ9JheKdNqR3h7zQ= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - cephadm: roleless: false - cephadm.shell: mon.a: - ceph orch status - ceph orch ps - ceph orch ls - ceph orch host ls - ceph orch device ls - cephadm.shell: mon.a: - ceph fs dump - ceph osd dump - fs.ready: timeout: 300 - kclient: null - exec: mon.a: - ceph fs set cephfs balance_automate false - ceph fs subvolumegroup pin cephfs qa random 0.10 - check-counter: {} - fwd_scrub: scrub_timeout: 900 sleep_between_iterations: 1 - exec: mon.a: - ceph fs snap-schedule add --fs=cephfs --path=/ --snap_schedule=1m - ceph fs snap-schedule retention add --fs=cephfs --path=/ --retention-spec-or-period=6m3h - ceph fs snap-schedule status --fs=cephfs --path=/ - ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true - date +%s > START_TIME - full_sequential_finally: - exec: mon.a: - date +%s > END_TIME - START_TIME=$(cat START_TIME); END_TIME=$(cat END_TIME); DIFF_TIME=$((600-(END_TIME-START_TIME))); if [ "$DIFF_TIME" -gt 0 ]; then sleep "$DIFF_TIME"; fi - ceph fs snap-schedule status --fs=cephfs --path=/ - ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true - workunit: clients: all: - suites/iozone.sh teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return yaml.ktype == 'upstream' end - local function is_kdistro() return yaml.ktype == 'distro' end - local function is_fuse() return false end - local function syntax_version() return yaml.overrides.kclient.syntax end teuthology_branch: main teuthology_sha1: 3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a timestamp: 2024-09-20_23:48:22 tube: smithi user: pdonnell verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2902 2024-09-21T03:23:08.832 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph-c_1843221895adccfb3b692060251dd6a0183bc410/qa; will attempt to use it 2024-09-21T03:23:08.833 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph-c_1843221895adccfb3b692060251dd6a0183bc410/qa/tasks 2024-09-21T03:23:08.833 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-09-21T03:23:08.835 INFO:teuthology.task.internal:Checking packages... 2024-09-21T03:23:08.855 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash '1843221895adccfb3b692060251dd6a0183bc410' 2024-09-21T03:23:08.855 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-09-21T03:23:08.855 INFO:teuthology.packaging:ref: None 2024-09-21T03:23:08.855 INFO:teuthology.packaging:tag: None 2024-09-21T03:23:08.855 INFO:teuthology.packaging:branch: wip-pdonnell-testing-20240920.212106-debug 2024-09-21T03:23:08.855 INFO:teuthology.packaging:sha1: 1843221895adccfb3b692060251dd6a0183bc410 2024-09-21T03:23:08.855 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=wip-pdonnell-testing-20240920.212106-debug 2024-09-21T03:23:08.995 INFO:teuthology.task.internal:Found packages for ceph version 19.3.0-5120.g18432218 2024-09-21T03:23:08.997 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-09-21T03:23:09.022 INFO:teuthology.task.internal:no buildpackages task found 2024-09-21T03:23:09.022 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-09-21T03:23:09.074 INFO:teuthology.task.internal:Saving configuration 2024-09-21T03:23:09.099 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-09-21T03:23:09.108 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-09-21T03:23:09.132 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi039.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/pdonnell-2024-09-20_23:48:22-fs-wip-pdonnell-testing-20240920.212106-debug-distro-default-smithi/7913471', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-21 03:17:11.389396', 'locked_by': 'scheduled_pdonnell@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBE6rLuY/X0JVqFFFpOJtZxIAzm5SSOA8sHRVjmvrzfJclGq6nMWnODKioFHdJC6VqUkI40WGvZ+R+7+Aw6Ux6Kg='} 2024-09-21T03:23:09.149 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi150.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/pdonnell-2024-09-20_23:48:22-fs-wip-pdonnell-testing-20240920.212106-debug-distro-default-smithi/7913471', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-21 03:17:11.390926', 'locked_by': 'scheduled_pdonnell@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBFO9bKRhgfp8hPwPdADAoG5VoX5oQiardE2NlftBR2jcBiCnpL1+xHqy3aq6FYt2kf5dnvBR6UGaACDi9LqCCfY='} 2024-09-21T03:23:09.168 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi161.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/pdonnell-2024-09-20_23:48:22-fs-wip-pdonnell-testing-20240920.212106-debug-distro-default-smithi/7913471', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-21 03:17:11.387788', 'locked_by': 'scheduled_pdonnell@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBOD130bkAshU+oUV3upBEA0sdb+J3gwqlrWHojRcamjslHlJagE7lu+jBi3OFlYnwhXUUMnmQ9JheKdNqR3h7zQ='} 2024-09-21T03:23:09.168 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-09-21T03:23:09.176 INFO:teuthology.task.internal:roles: ubuntu@smithi039.front.sepia.ceph.com - ['mon.a', 'mgr.x', 'mds.a', 'mds.d', 'mds.g', 'mds.j', 'osd.0', 'osd.3', 'osd.6', 'osd.9', 'client.0'] 2024-09-21T03:23:09.176 INFO:teuthology.task.internal:roles: ubuntu@smithi150.front.sepia.ceph.com - ['mon.b', 'mgr.y', 'mds.b', 'mds.e', 'mds.h', 'mds.k', 'osd.1', 'osd.4', 'osd.7', 'osd.10'] 2024-09-21T03:23:09.176 INFO:teuthology.task.internal:roles: ubuntu@smithi161.front.sepia.ceph.com - ['mon.c', 'mgr.z', 'mds.c', 'mds.f', 'mds.i', 'mds.l', 'osd.2', 'osd.5', 'osd.8', 'osd.11'] 2024-09-21T03:23:09.176 INFO:teuthology.run_tasks:Running task console_log... 2024-09-21T03:23:09.275 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f3421dfb5b0>, signals=[15]) 2024-09-21T03:23:09.275 INFO:teuthology.run_tasks:Running task internal.connect... 2024-09-21T03:23:09.282 INFO:teuthology.task.internal:Opening connections... 2024-09-21T03:23:09.282 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi039.front.sepia.ceph.com 2024-09-21T03:23:09.283 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi039.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-21T03:23:09.363 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi150.front.sepia.ceph.com 2024-09-21T03:23:09.364 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi150.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-21T03:23:09.443 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi161.front.sepia.ceph.com 2024-09-21T03:23:09.444 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi161.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-21T03:23:09.522 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-09-21T03:23:09.528 DEBUG:teuthology.orchestra.run.smithi039:> uname -m 2024-09-21T03:23:09.546 INFO:teuthology.orchestra.run.smithi039.stdout:x86_64 2024-09-21T03:23:09.546 DEBUG:teuthology.orchestra.run.smithi039:> cat /etc/os-release 2024-09-21T03:23:09.603 INFO:teuthology.orchestra.run.smithi039.stdout:NAME="CentOS Stream" 2024-09-21T03:23:09.603 INFO:teuthology.orchestra.run.smithi039.stdout:VERSION="9" 2024-09-21T03:23:09.603 INFO:teuthology.orchestra.run.smithi039.stdout:ID="centos" 2024-09-21T03:23:09.603 INFO:teuthology.orchestra.run.smithi039.stdout:ID_LIKE="rhel fedora" 2024-09-21T03:23:09.603 INFO:teuthology.orchestra.run.smithi039.stdout:VERSION_ID="9" 2024-09-21T03:23:09.603 INFO:teuthology.orchestra.run.smithi039.stdout:PLATFORM_ID="platform:el9" 2024-09-21T03:23:09.604 INFO:teuthology.orchestra.run.smithi039.stdout:PRETTY_NAME="CentOS Stream 9" 2024-09-21T03:23:09.604 INFO:teuthology.orchestra.run.smithi039.stdout:ANSI_COLOR="0;31" 2024-09-21T03:23:09.604 INFO:teuthology.orchestra.run.smithi039.stdout:LOGO="fedora-logo-icon" 2024-09-21T03:23:09.604 INFO:teuthology.orchestra.run.smithi039.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-09-21T03:23:09.604 INFO:teuthology.orchestra.run.smithi039.stdout:HOME_URL="https://centos.org/" 2024-09-21T03:23:09.604 INFO:teuthology.orchestra.run.smithi039.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2024-09-21T03:23:09.604 INFO:teuthology.orchestra.run.smithi039.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-09-21T03:23:09.604 INFO:teuthology.orchestra.run.smithi039.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-09-21T03:23:09.605 INFO:teuthology.lock.ops:Updating smithi039.front.sepia.ceph.com on lock server 2024-09-21T03:23:09.627 DEBUG:teuthology.orchestra.run.smithi150:> uname -m 2024-09-21T03:23:09.646 INFO:teuthology.orchestra.run.smithi150.stdout:x86_64 2024-09-21T03:23:09.646 DEBUG:teuthology.orchestra.run.smithi150:> cat /etc/os-release 2024-09-21T03:23:09.705 INFO:teuthology.orchestra.run.smithi150.stdout:NAME="CentOS Stream" 2024-09-21T03:23:09.705 INFO:teuthology.orchestra.run.smithi150.stdout:VERSION="9" 2024-09-21T03:23:09.705 INFO:teuthology.orchestra.run.smithi150.stdout:ID="centos" 2024-09-21T03:23:09.706 INFO:teuthology.orchestra.run.smithi150.stdout:ID_LIKE="rhel fedora" 2024-09-21T03:23:09.706 INFO:teuthology.orchestra.run.smithi150.stdout:VERSION_ID="9" 2024-09-21T03:23:09.706 INFO:teuthology.orchestra.run.smithi150.stdout:PLATFORM_ID="platform:el9" 2024-09-21T03:23:09.706 INFO:teuthology.orchestra.run.smithi150.stdout:PRETTY_NAME="CentOS Stream 9" 2024-09-21T03:23:09.706 INFO:teuthology.orchestra.run.smithi150.stdout:ANSI_COLOR="0;31" 2024-09-21T03:23:09.706 INFO:teuthology.orchestra.run.smithi150.stdout:LOGO="fedora-logo-icon" 2024-09-21T03:23:09.707 INFO:teuthology.orchestra.run.smithi150.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-09-21T03:23:09.707 INFO:teuthology.orchestra.run.smithi150.stdout:HOME_URL="https://centos.org/" 2024-09-21T03:23:09.707 INFO:teuthology.orchestra.run.smithi150.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2024-09-21T03:23:09.707 INFO:teuthology.orchestra.run.smithi150.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-09-21T03:23:09.707 INFO:teuthology.orchestra.run.smithi150.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-09-21T03:23:09.708 INFO:teuthology.lock.ops:Updating smithi150.front.sepia.ceph.com on lock server 2024-09-21T03:23:09.735 DEBUG:teuthology.orchestra.run.smithi161:> uname -m 2024-09-21T03:23:09.754 INFO:teuthology.orchestra.run.smithi161.stdout:x86_64 2024-09-21T03:23:09.754 DEBUG:teuthology.orchestra.run.smithi161:> cat /etc/os-release 2024-09-21T03:23:09.812 INFO:teuthology.orchestra.run.smithi161.stdout:NAME="CentOS Stream" 2024-09-21T03:23:09.812 INFO:teuthology.orchestra.run.smithi161.stdout:VERSION="9" 2024-09-21T03:23:09.812 INFO:teuthology.orchestra.run.smithi161.stdout:ID="centos" 2024-09-21T03:23:09.812 INFO:teuthology.orchestra.run.smithi161.stdout:ID_LIKE="rhel fedora" 2024-09-21T03:23:09.812 INFO:teuthology.orchestra.run.smithi161.stdout:VERSION_ID="9" 2024-09-21T03:23:09.812 INFO:teuthology.orchestra.run.smithi161.stdout:PLATFORM_ID="platform:el9" 2024-09-21T03:23:09.812 INFO:teuthology.orchestra.run.smithi161.stdout:PRETTY_NAME="CentOS Stream 9" 2024-09-21T03:23:09.812 INFO:teuthology.orchestra.run.smithi161.stdout:ANSI_COLOR="0;31" 2024-09-21T03:23:09.812 INFO:teuthology.orchestra.run.smithi161.stdout:LOGO="fedora-logo-icon" 2024-09-21T03:23:09.812 INFO:teuthology.orchestra.run.smithi161.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-09-21T03:23:09.812 INFO:teuthology.orchestra.run.smithi161.stdout:HOME_URL="https://centos.org/" 2024-09-21T03:23:09.812 INFO:teuthology.orchestra.run.smithi161.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2024-09-21T03:23:09.812 INFO:teuthology.orchestra.run.smithi161.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-09-21T03:23:09.813 INFO:teuthology.orchestra.run.smithi161.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-09-21T03:23:09.813 INFO:teuthology.lock.ops:Updating smithi161.front.sepia.ceph.com on lock server 2024-09-21T03:23:09.835 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-09-21T03:23:09.847 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-09-21T03:23:09.867 INFO:teuthology.task.internal:Checking for old test directory... 2024-09-21T03:23:09.867 DEBUG:teuthology.orchestra.run.smithi039:> test '!' -e /home/ubuntu/cephtest 2024-09-21T03:23:09.870 DEBUG:teuthology.orchestra.run.smithi150:> test '!' -e /home/ubuntu/cephtest 2024-09-21T03:23:09.872 DEBUG:teuthology.orchestra.run.smithi161:> test '!' -e /home/ubuntu/cephtest 2024-09-21T03:23:09.889 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-09-21T03:23:09.897 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-09-21T03:23:09.898 DEBUG:teuthology.orchestra.run.smithi039:> test -z $(ls -A /var/lib/ceph) 2024-09-21T03:23:09.927 DEBUG:teuthology.orchestra.run.smithi150:> test -z $(ls -A /var/lib/ceph) 2024-09-21T03:23:09.929 DEBUG:teuthology.orchestra.run.smithi161:> test -z $(ls -A /var/lib/ceph) 2024-09-21T03:23:09.960 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-09-21T03:23:10.032 INFO:teuthology.run_tasks:Running task kernel... 2024-09-21T03:23:10.047 INFO:teuthology.task.kernel:normalize config orig: {'client': {'sha1': 'distro'}, 'kdb': 1, 'sha1': 'distro'} 2024-09-21T03:23:10.048 INFO:teuthology.task.kernel:normalize config final: {'client.0': {'sha1': 'distro'}} 2024-09-21T03:23:10.048 INFO:teuthology.task.kernel:config {'client.0': {'sha1': 'distro'}}, timeout 300 2024-09-21T03:23:10.048 DEBUG:teuthology.orchestra.run.smithi039:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-21T03:23:10.064 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T03:23:10.065 DEBUG:teuthology.orchestra.run.smithi039:> uname -r 2024-09-21T03:23:10.123 INFO:teuthology.orchestra.run.smithi039.stdout:5.14.0-503.el9.x86_64 2024-09-21T03:23:10.123 INFO:teuthology.task.kernel:Running kernel on smithi039: 5.14.0-503.el9.x86_64 2024-09-21T03:23:10.123 DEBUG:teuthology.orchestra.run.smithi039:> sudo yum install -y kernel 2024-09-21T03:23:12.451 INFO:teuthology.orchestra.run.smithi039.stdout:CentOS Stream 9 - BaseOS 14 MB/s | 8.3 MB 00:00 2024-09-21T03:23:14.251 INFO:teuthology.orchestra.run.smithi039.stdout:CentOS Stream 9 - AppStream 31 MB/s | 20 MB 00:00 2024-09-21T03:23:21.143 INFO:teuthology.orchestra.run.smithi039.stdout:CentOS Stream 9 - CRB 2.2 MB/s | 6.5 MB 00:03 2024-09-21T03:23:22.727 INFO:teuthology.orchestra.run.smithi039.stdout:CentOS Stream 9 - Extras packages 61 kB/s | 19 kB 00:00 2024-09-21T03:23:23.864 INFO:teuthology.orchestra.run.smithi039.stdout:Extra Packages for Enterprise Linux 22 MB/s | 23 MB 00:01 2024-09-21T03:23:29.521 INFO:teuthology.orchestra.run.smithi039.stdout:lab-extras 48 kB/s | 1.7 kB 00:00 2024-09-21T03:23:31.511 INFO:teuthology.orchestra.run.smithi039.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-09-21T03:23:31.511 INFO:teuthology.orchestra.run.smithi039.stdout:Package kernel-5.14.0-503.el9.x86_64 is already installed. 2024-09-21T03:23:31.611 INFO:teuthology.orchestra.run.smithi039.stdout:Dependencies resolved. 2024-09-21T03:23:31.616 INFO:teuthology.orchestra.run.smithi039.stdout:================================================================================ 2024-09-21T03:23:31.616 INFO:teuthology.orchestra.run.smithi039.stdout: Package Architecture Version Repository Size 2024-09-21T03:23:31.616 INFO:teuthology.orchestra.run.smithi039.stdout:================================================================================ 2024-09-21T03:23:31.616 INFO:teuthology.orchestra.run.smithi039.stdout:Installing: 2024-09-21T03:23:31.616 INFO:teuthology.orchestra.run.smithi039.stdout: kernel x86_64 5.14.0-508.el9 baseos 36 k 2024-09-21T03:23:31.616 INFO:teuthology.orchestra.run.smithi039.stdout:Installing dependencies: 2024-09-21T03:23:31.616 INFO:teuthology.orchestra.run.smithi039.stdout: kernel-core x86_64 5.14.0-508.el9 baseos 16 M 2024-09-21T03:23:31.616 INFO:teuthology.orchestra.run.smithi039.stdout: kernel-modules x86_64 5.14.0-508.el9 baseos 35 M 2024-09-21T03:23:31.616 INFO:teuthology.orchestra.run.smithi039.stdout: kernel-modules-core x86_64 5.14.0-508.el9 baseos 29 M 2024-09-21T03:23:31.617 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T03:23:31.617 INFO:teuthology.orchestra.run.smithi039.stdout:Transaction Summary 2024-09-21T03:23:31.617 INFO:teuthology.orchestra.run.smithi039.stdout:================================================================================ 2024-09-21T03:23:31.617 INFO:teuthology.orchestra.run.smithi039.stdout:Install 4 Packages 2024-09-21T03:23:31.617 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T03:23:31.617 INFO:teuthology.orchestra.run.smithi039.stdout:Total download size: 79 M 2024-09-21T03:23:31.617 INFO:teuthology.orchestra.run.smithi039.stdout:Installed size: 126 M 2024-09-21T03:23:31.617 INFO:teuthology.orchestra.run.smithi039.stdout:Downloading Packages: 2024-09-21T03:23:32.057 INFO:teuthology.orchestra.run.smithi039.stdout:(1/4): kernel-5.14.0-508.el9.x86_64.rpm 122 kB/s | 36 kB 00:00 2024-09-21T03:23:38.447 INFO:teuthology.orchestra.run.smithi039.stdout:(2/4): kernel-core-5.14.0-508.el9.x86_64.rpm 2.3 MB/s | 16 MB 00:06 2024-09-21T03:23:38.763 INFO:teuthology.orchestra.run.smithi039.stdout:(3/4): kernel-modules-5.14.0-508.el9.x86_64.rpm 4.9 MB/s | 35 MB 00:07 2024-09-21T03:23:41.680 INFO:teuthology.orchestra.run.smithi039.stdout:(4/4): kernel-modules-core-5.14.0-508.el9.x86_6 3.0 MB/s | 29 MB 00:09 2024-09-21T03:23:41.681 INFO:teuthology.orchestra.run.smithi039.stdout:-------------------------------------------------------------------------------- 2024-09-21T03:23:41.681 INFO:teuthology.orchestra.run.smithi039.stdout:Total 7.8 MB/s | 79 MB 00:10 2024-09-21T03:23:42.387 INFO:teuthology.orchestra.run.smithi039.stdout:Running transaction check 2024-09-21T03:23:42.780 INFO:teuthology.orchestra.run.smithi039.stdout:Transaction check succeeded. 2024-09-21T03:23:42.780 INFO:teuthology.orchestra.run.smithi039.stdout:Running transaction test 2024-09-21T03:23:43.781 INFO:teuthology.orchestra.run.smithi039.stdout:Transaction test succeeded. 2024-09-21T03:23:43.781 INFO:teuthology.orchestra.run.smithi039.stdout:Running transaction 2024-09-21T03:23:46.327 INFO:teuthology.orchestra.run.smithi039.stdout: Preparing : 1/1 2024-09-21T03:23:47.008 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : kernel-modules-core-5.14.0-508.el9.x86_64 1/4 2024-09-21T03:23:47.097 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : kernel-core-5.14.0-508.el9.x86_64 2/4 2024-09-21T03:23:49.843 INFO:teuthology.orchestra.run.smithi039.stdout: Running scriptlet: kernel-core-5.14.0-508.el9.x86_64 2/4 2024-09-21T03:23:49.977 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : kernel-modules-5.14.0-508.el9.x86_64 3/4 2024-09-21T03:23:55.110 INFO:teuthology.orchestra.run.smithi039.stdout: Running scriptlet: kernel-modules-5.14.0-508.el9.x86_64 3/4 2024-09-21T03:23:55.178 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : kernel-5.14.0-508.el9.x86_64 4/4 2024-09-21T03:24:00.329 INFO:teuthology.orchestra.run.smithi039.stdout: Running scriptlet: kernel-modules-core-5.14.0-508.el9.x86_64 4/4 2024-09-21T03:24:39.303 INFO:teuthology.orchestra.run.smithi039.stdout: Running scriptlet: kernel-core-5.14.0-508.el9.x86_64 4/4 2024-09-21T03:24:39.344 INFO:teuthology.orchestra.run.smithi039.stdout: Running scriptlet: kernel-modules-5.14.0-508.el9.x86_64 4/4 2024-09-21T03:24:39.636 INFO:teuthology.orchestra.run.smithi039.stdout: Running scriptlet: kernel-5.14.0-508.el9.x86_64 4/4 2024-09-21T03:24:39.637 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : kernel-5.14.0-508.el9.x86_64 1/4 2024-09-21T03:24:39.637 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : kernel-core-5.14.0-508.el9.x86_64 2/4 2024-09-21T03:24:39.637 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : kernel-modules-5.14.0-508.el9.x86_64 3/4 2024-09-21T03:24:40.441 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : kernel-modules-core-5.14.0-508.el9.x86_64 4/4 2024-09-21T03:24:40.441 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T03:24:40.441 INFO:teuthology.orchestra.run.smithi039.stdout:Installed: 2024-09-21T03:24:40.441 INFO:teuthology.orchestra.run.smithi039.stdout: kernel-5.14.0-508.el9.x86_64 2024-09-21T03:24:40.441 INFO:teuthology.orchestra.run.smithi039.stdout: kernel-core-5.14.0-508.el9.x86_64 2024-09-21T03:24:40.441 INFO:teuthology.orchestra.run.smithi039.stdout: kernel-modules-5.14.0-508.el9.x86_64 2024-09-21T03:24:40.441 INFO:teuthology.orchestra.run.smithi039.stdout: kernel-modules-core-5.14.0-508.el9.x86_64 2024-09-21T03:24:40.441 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T03:24:40.441 INFO:teuthology.orchestra.run.smithi039.stdout:Complete! 2024-09-21T03:24:40.803 DEBUG:teuthology.orchestra.run.smithi039:> rpm -q kernel | sort -rV | head -n 1 2024-09-21T03:24:40.894 INFO:teuthology.orchestra.run.smithi039.stdout:kernel-5.14.0-508.el9.x86_64 2024-09-21T03:24:40.894 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-508.el9.x86_64 2024-09-21T03:24:40.894 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-503.el9.x86_64 Expected: 5.14.0-508.el9.x86_64 2024-09-21T03:24:40.894 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-09-21T03:24:40.894 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-09-21T03:24:40.894 INFO:teuthology.task.kernel:Installing distro kernel on client.0... 2024-09-21T03:24:40.894 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi039.front.sepia.ceph.com, path=None, version=distro) 2024-09-21T03:24:40.895 DEBUG:teuthology.orchestra.run.smithi039:> rpm -q kernel | sort -rV | head -n 1 2024-09-21T03:24:40.952 INFO:teuthology.orchestra.run.smithi039.stdout:kernel-5.14.0-508.el9.x86_64 2024-09-21T03:24:40.953 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-508.el9.x86_64 2024-09-21T03:24:40.953 DEBUG:teuthology.orchestra.run.smithi039:> sudo rpm -qi grub2-tools 2024-09-21T03:24:40.984 INFO:teuthology.orchestra.run.smithi039.stdout:Name : grub2-tools 2024-09-21T03:24:40.984 INFO:teuthology.orchestra.run.smithi039.stdout:Epoch : 1 2024-09-21T03:24:40.984 INFO:teuthology.orchestra.run.smithi039.stdout:Version : 2.06 2024-09-21T03:24:40.984 INFO:teuthology.orchestra.run.smithi039.stdout:Release : 82.el9 2024-09-21T03:24:40.984 INFO:teuthology.orchestra.run.smithi039.stdout:Architecture: x86_64 2024-09-21T03:24:40.984 INFO:teuthology.orchestra.run.smithi039.stdout:Install Date: Wed 28 Aug 2024 09:06:27 PM UTC 2024-09-21T03:24:40.984 INFO:teuthology.orchestra.run.smithi039.stdout:Group : Unspecified 2024-09-21T03:24:40.984 INFO:teuthology.orchestra.run.smithi039.stdout:Size : 8274670 2024-09-21T03:24:40.984 INFO:teuthology.orchestra.run.smithi039.stdout:License : GPLv3+ 2024-09-21T03:24:40.985 INFO:teuthology.orchestra.run.smithi039.stdout:Signature : RSA/SHA256, Mon 01 Jul 2024 07:46:42 AM UTC, Key ID 05b555b38483c65d 2024-09-21T03:24:40.985 INFO:teuthology.orchestra.run.smithi039.stdout:Source RPM : grub2-2.06-82.el9.src.rpm 2024-09-21T03:24:40.985 INFO:teuthology.orchestra.run.smithi039.stdout:Build Date : Fri 28 Jun 2024 09:24:44 AM UTC 2024-09-21T03:24:40.985 INFO:teuthology.orchestra.run.smithi039.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-09-21T03:24:40.985 INFO:teuthology.orchestra.run.smithi039.stdout:Packager : builder@centos.org 2024-09-21T03:24:40.985 INFO:teuthology.orchestra.run.smithi039.stdout:Vendor : CentOS 2024-09-21T03:24:40.985 INFO:teuthology.orchestra.run.smithi039.stdout:URL : http://www.gnu.org/software/grub/ 2024-09-21T03:24:40.985 INFO:teuthology.orchestra.run.smithi039.stdout:Summary : Support tools for GRUB. 2024-09-21T03:24:40.985 INFO:teuthology.orchestra.run.smithi039.stdout:Description : 2024-09-21T03:24:40.986 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T03:24:40.986 INFO:teuthology.orchestra.run.smithi039.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-09-21T03:24:40.986 INFO:teuthology.orchestra.run.smithi039.stdout:customizable bootloader with modular architecture. It supports a rich 2024-09-21T03:24:40.986 INFO:teuthology.orchestra.run.smithi039.stdout:variety of kernel formats, file systems, computer architectures and 2024-09-21T03:24:40.986 INFO:teuthology.orchestra.run.smithi039.stdout:hardware devices. 2024-09-21T03:24:40.986 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T03:24:40.986 INFO:teuthology.orchestra.run.smithi039.stdout:This subpackage provides tools for support of all platforms. 2024-09-21T03:24:40.987 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-09-21T03:24:40.987 INFO:teuthology.task.kernel:Updating grub on smithi039 to boot 5.14.0-508.el9.x86_64 2024-09-21T03:24:40.987 DEBUG:teuthology.orchestra.run.smithi039:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-09-21T03:24:41.659 INFO:teuthology.orchestra.run.smithi039.stderr:Generating grub configuration file ... 2024-09-21T03:24:42.976 INFO:teuthology.orchestra.run.smithi039.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-09-21T03:24:43.024 INFO:teuthology.orchestra.run.smithi039.stderr:done 2024-09-21T03:24:43.025 DEBUG:teuthology.orchestra.run.smithi039:> mktemp 2024-09-21T03:24:43.041 INFO:teuthology.orchestra.run.smithi039.stdout:/tmp/tmp.02aCsbadfv 2024-09-21T03:24:43.041 DEBUG:teuthology.orchestra.run.smithi039:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.02aCsbadfv 2024-09-21T03:24:43.108 DEBUG:teuthology.orchestra.run.smithi039:> sudo chmod 0666 /tmp/tmp.02aCsbadfv 2024-09-21T03:24:43.255 DEBUG:teuthology.orchestra.remote:smithi039:/tmp/tmp.02aCsbadfv is 6KB 2024-09-21T03:24:43.303 DEBUG:teuthology.orchestra.run.smithi039:> rm -fr /tmp/tmp.02aCsbadfv 2024-09-21T03:24:43.318 DEBUG:teuthology.orchestra.run.smithi039:> sudo /bin/ls /boot/loader/entries || true 2024-09-21T03:24:43.380 INFO:teuthology.orchestra.run.smithi039.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-0-rescue.conf 2024-09-21T03:24:43.380 INFO:teuthology.orchestra.run.smithi039.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-239.el9.x86_64.conf 2024-09-21T03:24:43.380 INFO:teuthology.orchestra.run.smithi039.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-503.el9.x86_64.conf 2024-09-21T03:24:43.380 INFO:teuthology.orchestra.run.smithi039.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-508.el9.x86_64.conf 2024-09-21T03:24:43.382 DEBUG:teuthology.orchestra.run.smithi039:> sudo grub2-set-default fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-508.el9.x86_64 2024-09-21T03:24:43.532 DEBUG:teuthology.orchestra.run.smithi039:> sudo shutdown -r now 2024-09-21T03:25:13.535 INFO:teuthology.task.kernel:Checking client client.0 for new kernel version... 2024-09-21T03:25:13.535 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi039.front.sepia.ceph.com' 2024-09-21T03:25:13.536 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi039.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-21T03:25:48.143 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.39 2024-09-21T03:25:57.144 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi039.front.sepia.ceph.com' 2024-09-21T03:25:57.144 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi039.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-21T03:26:00.206 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.39 2024-09-21T03:26:12.207 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi039.front.sepia.ceph.com' 2024-09-21T03:26:12.207 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi039.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-21T03:27:12.210 DEBUG:teuthology.orchestra.remote:timed out 2024-09-21T03:27:27.212 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi039.front.sepia.ceph.com' 2024-09-21T03:27:27.212 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi039.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-21T03:27:27.483 DEBUG:teuthology.orchestra.run.smithi039:> true 2024-09-21T03:27:27.825 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi039.front.sepia.ceph.com' 2024-09-21T03:27:27.825 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "5.14.0-508.el9.x86_64"... 2024-09-21T03:27:27.825 DEBUG:teuthology.orchestra.run.smithi039:> uname -r 2024-09-21T03:27:27.841 INFO:teuthology.orchestra.run.smithi039.stdout:5.14.0-508.el9.x86_64 2024-09-21T03:27:27.842 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-508.el9.x86_64 vs 5.14.0-508.el9.x86_64 2024-09-21T03:27:27.842 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-09-21T03:27:27.842 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-09-21T03:27:28.843 DEBUG:teuthology.parallel:result is None 2024-09-21T03:27:28.843 INFO:teuthology.run_tasks:Running task internal.base... 2024-09-21T03:27:28.851 INFO:teuthology.task.internal:Creating test directory... 2024-09-21T03:27:28.851 DEBUG:teuthology.orchestra.run.smithi039:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-21T03:27:28.853 DEBUG:teuthology.orchestra.run.smithi150:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-21T03:27:28.856 DEBUG:teuthology.orchestra.run.smithi161:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-21T03:27:28.874 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-09-21T03:27:28.881 INFO:teuthology.run_tasks:Running task internal.archive... 2024-09-21T03:27:28.911 INFO:teuthology.task.internal:Creating archive directory... 2024-09-21T03:27:28.911 DEBUG:teuthology.orchestra.run.smithi039:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-21T03:27:28.913 DEBUG:teuthology.orchestra.run.smithi150:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-21T03:27:28.916 DEBUG:teuthology.orchestra.run.smithi161:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-21T03:27:28.950 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-09-21T03:27:28.956 INFO:teuthology.task.internal:Enabling coredump saving... 2024-09-21T03:27:28.957 DEBUG:teuthology.orchestra.run.smithi150:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-21T03:27:28.995 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T03:27:28.996 DEBUG:teuthology.orchestra.run.smithi161:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-21T03:27:29.012 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T03:27:29.013 DEBUG:teuthology.orchestra.run.smithi039:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-21T03:27:29.016 DEBUG:teuthology.orchestra.run.smithi150:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-21T03:27:29.039 DEBUG:teuthology.orchestra.run.smithi161:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-21T03:27:29.071 INFO:teuthology.orchestra.run.smithi150.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-21T03:27:29.086 INFO:teuthology.orchestra.run.smithi150.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-21T03:27:29.087 INFO:teuthology.orchestra.run.smithi161.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-21T03:27:29.101 INFO:teuthology.orchestra.run.smithi161.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-21T03:27:29.115 INFO:teuthology.orchestra.run.smithi039.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-21T03:27:29.133 INFO:teuthology.orchestra.run.smithi039.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-21T03:27:29.135 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-09-21T03:27:29.142 INFO:teuthology.task.internal:Configuring sudo... 2024-09-21T03:27:29.142 DEBUG:teuthology.orchestra.run.smithi039:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-21T03:27:29.144 DEBUG:teuthology.orchestra.run.smithi150:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-21T03:27:29.147 DEBUG:teuthology.orchestra.run.smithi161:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-21T03:27:29.174 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-09-21T03:27:29.183 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-09-21T03:27:29.183 DEBUG:teuthology.orchestra.run.smithi039:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-21T03:27:29.214 DEBUG:teuthology.orchestra.run.smithi150:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-21T03:27:29.218 DEBUG:teuthology.orchestra.run.smithi161:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-21T03:27:29.234 DEBUG:teuthology.orchestra.run.smithi039:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-21T03:27:29.309 DEBUG:teuthology.orchestra.run.smithi039:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-21T03:27:29.366 DEBUG:teuthology.orchestra.run.smithi039:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-21T03:27:29.451 DEBUG:teuthology.orchestra.run.smithi039:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-21T03:27:29.494 DEBUG:teuthology.orchestra.run.smithi039:> set -ex 2024-09-21T03:27:29.494 DEBUG:teuthology.orchestra.run.smithi039:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-21T03:27:29.561 DEBUG:teuthology.orchestra.run.smithi150:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-21T03:27:29.605 DEBUG:teuthology.orchestra.run.smithi150:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-21T03:27:29.662 DEBUG:teuthology.orchestra.run.smithi150:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-21T03:27:29.746 DEBUG:teuthology.orchestra.run.smithi150:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-21T03:27:29.789 DEBUG:teuthology.orchestra.run.smithi150:> set -ex 2024-09-21T03:27:29.789 DEBUG:teuthology.orchestra.run.smithi150:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-21T03:27:29.855 DEBUG:teuthology.orchestra.run.smithi161:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-21T03:27:29.900 DEBUG:teuthology.orchestra.run.smithi161:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-21T03:27:29.954 DEBUG:teuthology.orchestra.run.smithi161:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-21T03:27:30.036 DEBUG:teuthology.orchestra.run.smithi161:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-21T03:27:30.084 DEBUG:teuthology.orchestra.run.smithi161:> set -ex 2024-09-21T03:27:30.084 DEBUG:teuthology.orchestra.run.smithi161:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-21T03:27:30.152 DEBUG:teuthology.orchestra.run.smithi039:> sudo service rsyslog restart 2024-09-21T03:27:30.155 DEBUG:teuthology.orchestra.run.smithi150:> sudo service rsyslog restart 2024-09-21T03:27:30.157 DEBUG:teuthology.orchestra.run.smithi161:> sudo service rsyslog restart 2024-09-21T03:27:30.197 INFO:teuthology.orchestra.run.smithi150.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-21T03:27:30.205 INFO:teuthology.orchestra.run.smithi039.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-21T03:27:30.228 INFO:teuthology.orchestra.run.smithi161.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-21T03:27:30.621 INFO:teuthology.run_tasks:Running task internal.timer... 2024-09-21T03:27:30.629 INFO:teuthology.task.internal:Starting timer... 2024-09-21T03:27:30.629 INFO:teuthology.run_tasks:Running task pcp... 2024-09-21T03:27:30.673 INFO:teuthology.run_tasks:Running task selinux... 2024-09-21T03:27:30.705 DEBUG:teuthology.task:Applying overrides for task selinux: {'allowlist': ['scontext=system_u:system_r:logrotate_t:s0']} 2024-09-21T03:27:30.705 DEBUG:teuthology.orchestra.run.smithi039:> sudo service auditd rotate 2024-09-21T03:27:30.790 INFO:teuthology.orchestra.run.smithi039.stdout:Rotating logs: 2024-09-21T03:27:30.791 DEBUG:teuthology.orchestra.run.smithi150:> sudo service auditd rotate 2024-09-21T03:27:30.834 INFO:teuthology.orchestra.run.smithi150.stdout:Rotating logs: 2024-09-21T03:27:30.836 DEBUG:teuthology.orchestra.run.smithi161:> sudo service auditd rotate 2024-09-21T03:27:30.888 INFO:teuthology.orchestra.run.smithi161.stdout:Rotating logs: 2024-09-21T03:27:30.890 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-09-21T03:27:30.891 DEBUG:teuthology.orchestra.run.smithi039:> /usr/sbin/getenforce 2024-09-21T03:27:30.926 INFO:teuthology.orchestra.run.smithi039.stdout:Permissive 2024-09-21T03:27:30.926 DEBUG:teuthology.orchestra.run.smithi150:> /usr/sbin/getenforce 2024-09-21T03:27:30.955 INFO:teuthology.orchestra.run.smithi150.stdout:Permissive 2024-09-21T03:27:30.955 DEBUG:teuthology.orchestra.run.smithi161:> /usr/sbin/getenforce 2024-09-21T03:27:30.982 INFO:teuthology.orchestra.run.smithi161.stdout:Permissive 2024-09-21T03:27:30.982 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi039.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi150.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi161.front.sepia.ceph.com': 'permissive'} 2024-09-21T03:27:30.982 DEBUG:teuthology.orchestra.run.smithi039:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 2024-09-21T03:27:31.012 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T03:27:31.012 DEBUG:teuthology.orchestra.run.smithi150:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 2024-09-21T03:27:31.042 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T03:27:31.042 DEBUG:teuthology.orchestra.run.smithi161:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 2024-09-21T03:27:31.071 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T03:27:31.071 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-09-21T03:27:31.072 DEBUG:teuthology.orchestra.run.smithi039:> sudo /usr/sbin/setenforce permissive 2024-09-21T03:27:31.107 DEBUG:teuthology.orchestra.run.smithi150:> sudo /usr/sbin/setenforce permissive 2024-09-21T03:27:31.152 DEBUG:teuthology.orchestra.run.smithi161:> sudo /usr/sbin/setenforce permissive 2024-09-21T03:27:31.194 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-09-21T03:27:31.203 DEBUG:teuthology.repo_utils:Setting repo remote to https://git.ceph.com/ceph-cm-ansible.git 2024-09-21T03:27:31.206 INFO:teuthology.repo_utils:Fetching git.ceph.com_ceph-cm-ansible_main from origin 2024-09-21T03:27:32.203 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-09-21T03:27:32.213 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-09-21T03:27:32.214 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi039.front.sepia.ceph.com,smithi150.front.sepia.ceph.com,smithi161.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-09-21T03:31:32.442 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi039.front.sepia.ceph.com'), Remote(name='ubuntu@smithi150.front.sepia.ceph.com'), Remote(name='ubuntu@smithi161.front.sepia.ceph.com')] 2024-09-21T03:31:32.443 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi039.front.sepia.ceph.com' 2024-09-21T03:31:32.443 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi039.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-21T03:31:32.523 DEBUG:teuthology.orchestra.run.smithi039:> true 2024-09-21T03:31:32.603 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi039.front.sepia.ceph.com' 2024-09-21T03:31:32.603 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi150.front.sepia.ceph.com' 2024-09-21T03:31:32.604 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi150.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-21T03:31:32.679 DEBUG:teuthology.orchestra.run.smithi150:> true 2024-09-21T03:31:32.754 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi150.front.sepia.ceph.com' 2024-09-21T03:31:32.755 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi161.front.sepia.ceph.com' 2024-09-21T03:31:32.755 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi161.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-21T03:31:32.831 DEBUG:teuthology.orchestra.run.smithi161:> true 2024-09-21T03:31:32.919 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi161.front.sepia.ceph.com' 2024-09-21T03:31:32.920 INFO:teuthology.run_tasks:Running task clock... 2024-09-21T03:31:32.936 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-09-21T03:31:32.937 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-21T03:31:32.937 DEBUG:teuthology.orchestra.run.smithi039:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-21T03:31:32.939 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-21T03:31:32.940 DEBUG:teuthology.orchestra.run.smithi150:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-21T03:31:32.942 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-21T03:31:32.942 DEBUG:teuthology.orchestra.run.smithi161:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-21T03:31:32.970 INFO:teuthology.orchestra.run.smithi039.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-09-21T03:31:32.971 INFO:teuthology.orchestra.run.smithi150.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-09-21T03:31:32.986 INFO:teuthology.orchestra.run.smithi150.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-09-21T03:31:32.986 INFO:teuthology.orchestra.run.smithi039.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-09-21T03:31:32.991 INFO:teuthology.orchestra.run.smithi161.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-09-21T03:31:33.006 INFO:teuthology.orchestra.run.smithi161.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-09-21T03:31:33.011 INFO:teuthology.orchestra.run.smithi150.stderr:sudo: ntpd: command not found 2024-09-21T03:31:33.013 INFO:teuthology.orchestra.run.smithi039.stderr:sudo: ntpd: command not found 2024-09-21T03:31:33.023 INFO:teuthology.orchestra.run.smithi150.stdout:506 Cannot talk to daemon 2024-09-21T03:31:33.025 INFO:teuthology.orchestra.run.smithi039.stdout:506 Cannot talk to daemon 2024-09-21T03:31:33.031 INFO:teuthology.orchestra.run.smithi161.stderr:sudo: ntpd: command not found 2024-09-21T03:31:33.038 INFO:teuthology.orchestra.run.smithi150.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-09-21T03:31:33.040 INFO:teuthology.orchestra.run.smithi039.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-09-21T03:31:33.043 INFO:teuthology.orchestra.run.smithi161.stdout:506 Cannot talk to daemon 2024-09-21T03:31:33.052 INFO:teuthology.orchestra.run.smithi150.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-09-21T03:31:33.055 INFO:teuthology.orchestra.run.smithi039.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-09-21T03:31:33.057 INFO:teuthology.orchestra.run.smithi161.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-09-21T03:31:33.071 INFO:teuthology.orchestra.run.smithi161.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-09-21T03:31:33.113 INFO:teuthology.orchestra.run.smithi150.stderr:bash: line 1: ntpq: command not found 2024-09-21T03:31:33.117 INFO:teuthology.orchestra.run.smithi150.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-21T03:31:33.117 INFO:teuthology.orchestra.run.smithi150.stdout:=============================================================================== 2024-09-21T03:31:33.117 INFO:teuthology.orchestra.run.smithi150.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-21T03:31:33.117 INFO:teuthology.orchestra.run.smithi150.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-21T03:31:33.117 INFO:teuthology.orchestra.run.smithi150.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-21T03:31:33.117 INFO:teuthology.orchestra.run.smithi150.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-21T03:31:33.123 INFO:teuthology.orchestra.run.smithi039.stderr:bash: line 1: ntpq: command not found 2024-09-21T03:31:33.127 INFO:teuthology.orchestra.run.smithi039.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-21T03:31:33.128 INFO:teuthology.orchestra.run.smithi039.stdout:=============================================================================== 2024-09-21T03:31:33.128 INFO:teuthology.orchestra.run.smithi039.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-21T03:31:33.128 INFO:teuthology.orchestra.run.smithi039.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-21T03:31:33.128 INFO:teuthology.orchestra.run.smithi039.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-21T03:31:33.128 INFO:teuthology.orchestra.run.smithi039.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-21T03:31:33.129 INFO:teuthology.orchestra.run.smithi161.stderr:bash: line 1: ntpq: command not found 2024-09-21T03:31:33.133 INFO:teuthology.orchestra.run.smithi161.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-21T03:31:33.133 INFO:teuthology.orchestra.run.smithi161.stdout:=============================================================================== 2024-09-21T03:31:33.133 INFO:teuthology.orchestra.run.smithi161.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-21T03:31:33.133 INFO:teuthology.orchestra.run.smithi161.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-21T03:31:33.133 INFO:teuthology.orchestra.run.smithi161.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-21T03:31:33.133 INFO:teuthology.orchestra.run.smithi161.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-21T03:31:33.134 INFO:teuthology.run_tasks:Running task install... 2024-09-21T03:31:33.142 DEBUG:teuthology.task.install:project ceph 2024-09-21T03:31:33.142 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '1843221895adccfb3b692060251dd6a0183bc410'}} 2024-09-21T03:31:33.142 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '1843221895adccfb3b692060251dd6a0183bc410'} 2024-09-21T03:31:33.142 INFO:teuthology.task.install:Using flavor: default 2024-09-21T03:31:33.147 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-09-21T03:31:33.147 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']} 2024-09-21T03:31:33.147 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '1843221895adccfb3b692060251dd6a0183bc410', 'tag': None, 'wait_for_package': False} 2024-09-21T03:31:33.148 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=1843221895adccfb3b692060251dd6a0183bc410 2024-09-21T03:31:33.149 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '1843221895adccfb3b692060251dd6a0183bc410', 'tag': None, 'wait_for_package': False} 2024-09-21T03:31:33.149 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=1843221895adccfb3b692060251dd6a0183bc410 2024-09-21T03:31:33.150 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '1843221895adccfb3b692060251dd6a0183bc410', 'tag': None, 'wait_for_package': False} 2024-09-21T03:31:33.151 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=1843221895adccfb3b692060251dd6a0183bc410 2024-09-21T03:31:33.301 INFO:teuthology.task.install.rpm:Pulling from https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/centos/9/flavors/default/ 2024-09-21T03:31:33.301 INFO:teuthology.task.install.rpm:Package version is 19.3.0-5120.g18432218 2024-09-21T03:31:33.304 INFO:teuthology.task.install.rpm:Pulling from https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/centos/9/flavors/default/ 2024-09-21T03:31:33.304 INFO:teuthology.task.install.rpm:Package version is 19.3.0-5120.g18432218 2024-09-21T03:31:33.309 INFO:teuthology.task.install.rpm:Pulling from https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/centos/9/flavors/default/ 2024-09-21T03:31:33.309 INFO:teuthology.task.install.rpm:Package version is 19.3.0-5120.g18432218 2024-09-21T03:31:33.409 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-09-21T03:31:33.409 DEBUG:teuthology.orchestra.run.smithi039:> set -ex 2024-09-21T03:31:33.409 DEBUG:teuthology.orchestra.run.smithi039:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-09-21T03:31:33.415 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-09-21T03:31:33.415 DEBUG:teuthology.orchestra.run.smithi161:> set -ex 2024-09-21T03:31:33.415 DEBUG:teuthology.orchestra.run.smithi161:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-09-21T03:31:33.422 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-09-21T03:31:33.422 DEBUG:teuthology.orchestra.run.smithi150:> set -ex 2024-09-21T03:31:33.422 DEBUG:teuthology.orchestra.run.smithi150:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-09-21T03:31:33.436 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-09-21T03:31:33.436 DEBUG:teuthology.orchestra.run.smithi039:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/1843221895adccfb3b692060251dd6a0183bc410/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-09-21T03:31:33.440 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-09-21T03:31:33.440 DEBUG:teuthology.orchestra.run.smithi161:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/1843221895adccfb3b692060251dd6a0183bc410/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-09-21T03:31:33.447 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-09-21T03:31:33.448 DEBUG:teuthology.orchestra.run.smithi150:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/1843221895adccfb3b692060251dd6a0183bc410/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-09-21T03:31:33.503 DEBUG:teuthology.orchestra.run.smithi039:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-09-21T03:31:33.507 DEBUG:teuthology.orchestra.run.smithi161:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-09-21T03:31:33.513 DEBUG:teuthology.orchestra.run.smithi150:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-09-21T03:31:33.580 DEBUG:teuthology.orchestra.run.smithi039:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-09-21T03:31:33.582 DEBUG:teuthology.orchestra.run.smithi161:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-09-21T03:31:33.588 DEBUG:teuthology.orchestra.run.smithi150:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-09-21T03:31:33.645 INFO:teuthology.orchestra.run.smithi039.stdout:check_obsoletes = 1 2024-09-21T03:31:33.646 INFO:teuthology.orchestra.run.smithi161.stdout:check_obsoletes = 1 2024-09-21T03:31:33.647 DEBUG:teuthology.orchestra.run.smithi039:> sudo yum clean all 2024-09-21T03:31:33.648 DEBUG:teuthology.orchestra.run.smithi161:> sudo yum clean all 2024-09-21T03:31:33.653 INFO:teuthology.orchestra.run.smithi150.stdout:check_obsoletes = 1 2024-09-21T03:31:33.655 DEBUG:teuthology.orchestra.run.smithi150:> sudo yum clean all 2024-09-21T03:31:33.873 INFO:teuthology.orchestra.run.smithi161.stdout:45 files removed 2024-09-21T03:31:33.881 INFO:teuthology.orchestra.run.smithi150.stdout:45 files removed 2024-09-21T03:31:33.883 INFO:teuthology.orchestra.run.smithi039.stdout:45 files removed 2024-09-21T03:31:33.900 DEBUG:teuthology.orchestra.run.smithi161:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-09-21T03:31:33.908 DEBUG:teuthology.orchestra.run.smithi150:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-09-21T03:31:33.911 DEBUG:teuthology.orchestra.run.smithi039:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-09-21T03:31:34.435 INFO:teuthology.orchestra.run.smithi161.stdout:ceph packages for x86_64 326 kB/s | 84 kB 00:00 2024-09-21T03:31:34.458 INFO:teuthology.orchestra.run.smithi150.stdout:ceph packages for x86_64 307 kB/s | 84 kB 00:00 2024-09-21T03:31:34.488 INFO:teuthology.orchestra.run.smithi039.stdout:ceph packages for x86_64 305 kB/s | 84 kB 00:00 2024-09-21T03:31:34.711 INFO:teuthology.orchestra.run.smithi161.stdout:ceph noarch packages 104 kB/s | 24 kB 00:00 2024-09-21T03:31:34.743 INFO:teuthology.orchestra.run.smithi150.stdout:ceph noarch packages 100 kB/s | 24 kB 00:00 2024-09-21T03:31:34.789 INFO:teuthology.orchestra.run.smithi039.stdout:ceph noarch packages 95 kB/s | 24 kB 00:00 2024-09-21T03:31:34.970 INFO:teuthology.orchestra.run.smithi161.stdout:ceph source packages 8.5 kB/s | 1.9 kB 00:00 2024-09-21T03:31:35.018 INFO:teuthology.orchestra.run.smithi150.stdout:ceph source packages 8.0 kB/s | 1.9 kB 00:00 2024-09-21T03:31:35.073 INFO:teuthology.orchestra.run.smithi039.stdout:ceph source packages 7.8 kB/s | 1.9 kB 00:00 2024-09-21T03:31:36.436 INFO:teuthology.orchestra.run.smithi161.stdout:CentOS Stream 9 - BaseOS 5.8 MB/s | 8.3 MB 00:01 2024-09-21T03:31:37.873 INFO:teuthology.orchestra.run.smithi039.stdout:CentOS Stream 9 - BaseOS 3.0 MB/s | 8.3 MB 00:02 2024-09-21T03:31:38.268 INFO:teuthology.orchestra.run.smithi150.stdout:CentOS Stream 9 - BaseOS 2.6 MB/s | 8.3 MB 00:03 2024-09-21T03:31:44.903 INFO:teuthology.orchestra.run.smithi161.stdout:CentOS Stream 9 - AppStream 2.8 MB/s | 20 MB 00:07 2024-09-21T03:31:46.507 INFO:teuthology.orchestra.run.smithi039.stdout:CentOS Stream 9 - AppStream 2.7 MB/s | 20 MB 00:07 2024-09-21T03:31:46.619 INFO:teuthology.orchestra.run.smithi150.stdout:CentOS Stream 9 - AppStream 2.8 MB/s | 20 MB 00:07 2024-09-21T03:31:50.320 INFO:teuthology.orchestra.run.smithi161.stdout:CentOS Stream 9 - CRB 4.0 MB/s | 6.5 MB 00:01 2024-09-21T03:31:50.958 INFO:teuthology.orchestra.run.smithi039.stdout:CentOS Stream 9 - CRB 15 MB/s | 6.5 MB 00:00 2024-09-21T03:31:51.344 INFO:teuthology.orchestra.run.smithi150.stdout:CentOS Stream 9 - CRB 7.1 MB/s | 6.5 MB 00:00 2024-09-21T03:31:51.880 INFO:teuthology.orchestra.run.smithi161.stdout:CentOS Stream 9 - Extras packages 56 kB/s | 19 kB 00:00 2024-09-21T03:31:52.433 INFO:teuthology.orchestra.run.smithi039.stdout:CentOS Stream 9 - Extras packages 95 kB/s | 19 kB 00:00 2024-09-21T03:31:53.075 INFO:teuthology.orchestra.run.smithi161.stdout:Extra Packages for Enterprise Linux 21 MB/s | 23 MB 00:01 2024-09-21T03:31:53.103 INFO:teuthology.orchestra.run.smithi150.stdout:CentOS Stream 9 - Extras packages 36 kB/s | 19 kB 00:00 2024-09-21T03:31:53.879 INFO:teuthology.orchestra.run.smithi039.stdout:Extra Packages for Enterprise Linux 17 MB/s | 23 MB 00:01 2024-09-21T03:31:54.299 INFO:teuthology.orchestra.run.smithi150.stdout:Extra Packages for Enterprise Linux 21 MB/s | 23 MB 00:01 2024-09-21T03:31:58.481 INFO:teuthology.orchestra.run.smithi161.stdout:lab-extras 44 kB/s | 1.7 kB 00:00 2024-09-21T03:31:59.644 INFO:teuthology.orchestra.run.smithi039.stdout:lab-extras 29 kB/s | 1.7 kB 00:00 2024-09-21T03:31:59.664 INFO:teuthology.orchestra.run.smithi150.stdout:lab-extras 33 kB/s | 1.7 kB 00:00 2024-09-21T03:32:00.608 INFO:teuthology.orchestra.run.smithi161.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-21T03:32:00.608 INFO:teuthology.orchestra.run.smithi161.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-21T03:32:00.620 INFO:teuthology.orchestra.run.smithi161.stdout:Package NetworkManager-1:1.48.8-1.el9.x86_64 is already installed. 2024-09-21T03:32:00.621 INFO:teuthology.orchestra.run.smithi161.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-09-21T03:32:00.621 INFO:teuthology.orchestra.run.smithi161.stdout:Package util-linux-2.37.4-20.el9.x86_64 is already installed. 2024-09-21T03:32:00.622 INFO:teuthology.orchestra.run.smithi161.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-09-21T03:32:00.624 INFO:teuthology.orchestra.run.smithi161.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-09-21T03:32:00.626 INFO:teuthology.orchestra.run.smithi161.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-09-21T03:32:00.627 INFO:teuthology.orchestra.run.smithi161.stdout:Package xfsprogs-6.4.0-4.el9.x86_64 is already installed. 2024-09-21T03:32:00.690 INFO:teuthology.orchestra.run.smithi161.stdout:Dependencies resolved. 2024-09-21T03:32:00.700 INFO:teuthology.orchestra.run.smithi161.stdout:======================================================================================= 2024-09-21T03:32:00.700 INFO:teuthology.orchestra.run.smithi161.stdout: Package Arch Version Repository Size 2024-09-21T03:32:00.701 INFO:teuthology.orchestra.run.smithi161.stdout:======================================================================================= 2024-09-21T03:32:00.701 INFO:teuthology.orchestra.run.smithi161.stdout:Installing: 2024-09-21T03:32:00.701 INFO:teuthology.orchestra.run.smithi161.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-09-21T03:32:00.701 INFO:teuthology.orchestra.run.smithi161.stdout: ceph x86_64 2:19.3.0-5120.g18432218.el9 ceph 6.6 k 2024-09-21T03:32:00.701 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-base x86_64 2:19.3.0-5120.g18432218.el9 ceph 5.5 M 2024-09-21T03:32:00.701 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-fuse x86_64 2:19.3.0-5120.g18432218.el9 ceph 873 k 2024-09-21T03:32:00.701 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-immutable-object-cache x86_64 2:19.3.0-5120.g18432218.el9 ceph 148 k 2024-09-21T03:32:00.701 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-mgr x86_64 2:19.3.0-5120.g18432218.el9 ceph 1.5 M 2024-09-21T03:32:00.701 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-mgr-cephadm noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 154 k 2024-09-21T03:32:00.701 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-mgr-dashboard noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 14 M 2024-09-21T03:32:00.701 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-mgr-diskprediction-local noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 7.4 M 2024-09-21T03:32:00.701 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-mgr-rook noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 49 k 2024-09-21T03:32:00.701 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-radosgw x86_64 2:19.3.0-5120.g18432218.el9 ceph 10 M 2024-09-21T03:32:00.702 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-test x86_64 2:19.3.0-5120.g18432218.el9 ceph 40 M 2024-09-21T03:32:00.702 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-volume noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 285 k 2024-09-21T03:32:00.702 INFO:teuthology.orchestra.run.smithi161.stdout: cephadm noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 784 k 2024-09-21T03:32:00.702 INFO:teuthology.orchestra.run.smithi161.stdout: cephfs-mirror x86_64 2:19.3.0-5120.g18432218.el9 ceph 226 k 2024-09-21T03:32:00.702 INFO:teuthology.orchestra.run.smithi161.stdout: cephfs-shell x86_64 2:19.3.0-5120.g18432218.el9 ceph 27 k 2024-09-21T03:32:00.702 INFO:teuthology.orchestra.run.smithi161.stdout: cephfs-top noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 18 k 2024-09-21T03:32:00.702 INFO:teuthology.orchestra.run.smithi161.stdout: elfutils-libelf-devel x86_64 0.191-4.el9 appstream 22 k 2024-09-21T03:32:00.702 INFO:teuthology.orchestra.run.smithi161.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-09-21T03:32:00.702 INFO:teuthology.orchestra.run.smithi161.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-09-21T03:32:00.702 INFO:teuthology.orchestra.run.smithi161.stdout: libcephfs-devel x86_64 2:19.3.0-5120.g18432218.el9 ceph 32 k 2024-09-21T03:32:00.702 INFO:teuthology.orchestra.run.smithi161.stdout: libcephfs2 x86_64 2:19.3.0-5120.g18432218.el9 ceph 766 k 2024-09-21T03:32:00.703 INFO:teuthology.orchestra.run.smithi161.stdout: librados-devel x86_64 2:19.3.0-5120.g18432218.el9 ceph 127 k 2024-09-21T03:32:00.703 INFO:teuthology.orchestra.run.smithi161.stdout: libtool x86_64 2.4.6-46.el9 appstream 581 k 2024-09-21T03:32:00.703 INFO:teuthology.orchestra.run.smithi161.stdout: libuuid-devel x86_64 2.37.4-20.el9 appstream 24 k 2024-09-21T03:32:00.703 INFO:teuthology.orchestra.run.smithi161.stdout: openssl-devel x86_64 1:3.2.2-6.el9 appstream 4.4 M 2024-09-21T03:32:00.703 INFO:teuthology.orchestra.run.smithi161.stdout: postgresql x86_64 13.16-1.el9 appstream 1.6 M 2024-09-21T03:32:00.703 INFO:teuthology.orchestra.run.smithi161.stdout: postgresql-contrib x86_64 13.16-1.el9 appstream 910 k 2024-09-21T03:32:00.703 INFO:teuthology.orchestra.run.smithi161.stdout: postgresql-server x86_64 13.16-1.el9 appstream 5.8 M 2024-09-21T03:32:00.703 INFO:teuthology.orchestra.run.smithi161.stdout: python3-cephfs x86_64 2:19.3.0-5120.g18432218.el9 ceph 161 k 2024-09-21T03:32:00.703 INFO:teuthology.orchestra.run.smithi161.stdout: python3-rados x86_64 2:19.3.0-5120.g18432218.el9 ceph 321 k 2024-09-21T03:32:00.703 INFO:teuthology.orchestra.run.smithi161.stdout: python3-rbd x86_64 2:19.3.0-5120.g18432218.el9 ceph 307 k 2024-09-21T03:32:00.703 INFO:teuthology.orchestra.run.smithi161.stdout: python3-rgw x86_64 2:19.3.0-5120.g18432218.el9 ceph 99 k 2024-09-21T03:32:00.703 INFO:teuthology.orchestra.run.smithi161.stdout: rbd-fuse x86_64 2:19.3.0-5120.g18432218.el9 ceph 86 k 2024-09-21T03:32:00.703 INFO:teuthology.orchestra.run.smithi161.stdout: rbd-mirror x86_64 2:19.3.0-5120.g18432218.el9 ceph 3.1 M 2024-09-21T03:32:00.704 INFO:teuthology.orchestra.run.smithi161.stdout: rbd-nbd x86_64 2:19.3.0-5120.g18432218.el9 ceph 170 k 2024-09-21T03:32:00.704 INFO:teuthology.orchestra.run.smithi161.stdout: xfsprogs-devel x86_64 6.4.0-4.el9 appstream 96 k 2024-09-21T03:32:00.704 INFO:teuthology.orchestra.run.smithi161.stdout:Upgrading: 2024-09-21T03:32:00.704 INFO:teuthology.orchestra.run.smithi161.stdout: NetworkManager x86_64 1:1.51.0-1.el9 baseos 2.3 M 2024-09-21T03:32:00.704 INFO:teuthology.orchestra.run.smithi161.stdout: NetworkManager-libnm x86_64 1:1.51.0-1.el9 baseos 1.8 M 2024-09-21T03:32:00.704 INFO:teuthology.orchestra.run.smithi161.stdout: NetworkManager-team x86_64 1:1.51.0-1.el9 baseos 41 k 2024-09-21T03:32:00.704 INFO:teuthology.orchestra.run.smithi161.stdout: NetworkManager-tui x86_64 1:1.51.0-1.el9 baseos 249 k 2024-09-21T03:32:00.704 INFO:teuthology.orchestra.run.smithi161.stdout: librados2 x86_64 2:19.3.0-5120.g18432218.el9 ceph 3.4 M 2024-09-21T03:32:00.704 INFO:teuthology.orchestra.run.smithi161.stdout: librbd1 x86_64 2:19.3.0-5120.g18432218.el9 ceph 3.2 M 2024-09-21T03:32:00.704 INFO:teuthology.orchestra.run.smithi161.stdout: openssl x86_64 1:3.2.2-6.el9 baseos 1.4 M 2024-09-21T03:32:00.704 INFO:teuthology.orchestra.run.smithi161.stdout: openssl-libs x86_64 1:3.2.2-6.el9 baseos 2.4 M 2024-09-21T03:32:00.704 INFO:teuthology.orchestra.run.smithi161.stdout:Installing dependencies: 2024-09-21T03:32:00.704 INFO:teuthology.orchestra.run.smithi161.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-09-21T03:32:00.704 INFO:teuthology.orchestra.run.smithi161.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-09-21T03:32:00.705 INFO:teuthology.orchestra.run.smithi161.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-09-21T03:32:00.705 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-common x86_64 2:19.3.0-5120.g18432218.el9 ceph 20 M 2024-09-21T03:32:00.705 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-grafana-dashboards noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 34 k 2024-09-21T03:32:00.705 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-mds x86_64 2:19.3.0-5120.g18432218.el9 ceph 2.3 M 2024-09-21T03:32:00.705 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-mgr-modules-core noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 292 k 2024-09-21T03:32:00.705 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-mon x86_64 2:19.3.0-5120.g18432218.el9 ceph 5.0 M 2024-09-21T03:32:00.705 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-osd x86_64 2:19.3.0-5120.g18432218.el9 ceph 18 M 2024-09-21T03:32:00.705 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-prometheus-alerts noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 16 k 2024-09-21T03:32:00.705 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-selinux x86_64 2:19.3.0-5120.g18432218.el9 ceph 25 k 2024-09-21T03:32:00.705 INFO:teuthology.orchestra.run.smithi161.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-09-21T03:32:00.705 INFO:teuthology.orchestra.run.smithi161.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-09-21T03:32:00.705 INFO:teuthology.orchestra.run.smithi161.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-09-21T03:32:00.705 INFO:teuthology.orchestra.run.smithi161.stdout: fuse x86_64 2.9.9-16.el9 baseos 80 k 2024-09-21T03:32:00.705 INFO:teuthology.orchestra.run.smithi161.stdout: gperftools-libs x86_64 2.9.1-3.el9 epel 308 k 2024-09-21T03:32:00.706 INFO:teuthology.orchestra.run.smithi161.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-09-21T03:32:00.706 INFO:teuthology.orchestra.run.smithi161.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-09-21T03:32:00.706 INFO:teuthology.orchestra.run.smithi161.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-09-21T03:32:00.706 INFO:teuthology.orchestra.run.smithi161.stdout: libcephsqlite x86_64 2:19.3.0-5120.g18432218.el9 ceph 158 k 2024-09-21T03:32:00.706 INFO:teuthology.orchestra.run.smithi161.stdout: libgfortran x86_64 11.5.0-2.el9 baseos 801 k 2024-09-21T03:32:00.706 INFO:teuthology.orchestra.run.smithi161.stdout: libnbd x86_64 1.20.2-2.el9 appstream 171 k 2024-09-21T03:32:00.706 INFO:teuthology.orchestra.run.smithi161.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-09-21T03:32:00.706 INFO:teuthology.orchestra.run.smithi161.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-09-21T03:32:00.706 INFO:teuthology.orchestra.run.smithi161.stdout: libquadmath x86_64 11.5.0-2.el9 baseos 191 k 2024-09-21T03:32:00.706 INFO:teuthology.orchestra.run.smithi161.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-09-21T03:32:00.706 INFO:teuthology.orchestra.run.smithi161.stdout: libradosstriper1 x86_64 2:19.3.0-5120.g18432218.el9 ceph 507 k 2024-09-21T03:32:00.707 INFO:teuthology.orchestra.run.smithi161.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-09-21T03:32:00.707 INFO:teuthology.orchestra.run.smithi161.stdout: librgw2 x86_64 2:19.3.0-5120.g18432218.el9 ceph 5.3 M 2024-09-21T03:32:00.707 INFO:teuthology.orchestra.run.smithi161.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-09-21T03:32:00.707 INFO:teuthology.orchestra.run.smithi161.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-09-21T03:32:00.707 INFO:teuthology.orchestra.run.smithi161.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-09-21T03:32:00.707 INFO:teuthology.orchestra.run.smithi161.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-09-21T03:32:00.707 INFO:teuthology.orchestra.run.smithi161.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-09-21T03:32:00.707 INFO:teuthology.orchestra.run.smithi161.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2024-09-21T03:32:00.707 INFO:teuthology.orchestra.run.smithi161.stdout: openblas x86_64 0.3.26-2.el9 appstream 38 k 2024-09-21T03:32:00.707 INFO:teuthology.orchestra.run.smithi161.stdout: openblas-openmp x86_64 0.3.26-2.el9 appstream 4.9 M 2024-09-21T03:32:00.707 INFO:teuthology.orchestra.run.smithi161.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-09-21T03:32:00.707 INFO:teuthology.orchestra.run.smithi161.stdout: postgresql-private-libs x86_64 13.16-1.el9 appstream 137 k 2024-09-21T03:32:00.707 INFO:teuthology.orchestra.run.smithi161.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-09-21T03:32:00.708 INFO:teuthology.orchestra.run.smithi161.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-09-21T03:32:00.708 INFO:teuthology.orchestra.run.smithi161.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-09-21T03:32:00.708 INFO:teuthology.orchestra.run.smithi161.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-09-21T03:32:00.708 INFO:teuthology.orchestra.run.smithi161.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-09-21T03:32:00.708 INFO:teuthology.orchestra.run.smithi161.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-09-21T03:32:00.708 INFO:teuthology.orchestra.run.smithi161.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-09-21T03:32:00.708 INFO:teuthology.orchestra.run.smithi161.stdout: python3-ceph-argparse x86_64 2:19.3.0-5120.g18432218.el9 ceph 45 k 2024-09-21T03:32:00.708 INFO:teuthology.orchestra.run.smithi161.stdout: python3-ceph-common x86_64 2:19.3.0-5120.g18432218.el9 ceph 148 k 2024-09-21T03:32:00.708 INFO:teuthology.orchestra.run.smithi161.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2024-09-21T03:32:00.708 INFO:teuthology.orchestra.run.smithi161.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-09-21T03:32:00.708 INFO:teuthology.orchestra.run.smithi161.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-09-21T03:32:00.708 INFO:teuthology.orchestra.run.smithi161.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-09-21T03:32:00.708 INFO:teuthology.orchestra.run.smithi161.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-09-21T03:32:00.708 INFO:teuthology.orchestra.run.smithi161.stdout: python3-colorama noarch 0.4.6-3.el9 appstream 61 k 2024-09-21T03:32:00.709 INFO:teuthology.orchestra.run.smithi161.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-09-21T03:32:00.709 INFO:teuthology.orchestra.run.smithi161.stdout: python3-devel x86_64 3.9.19-8.el9 appstream 245 k 2024-09-21T03:32:00.709 INFO:teuthology.orchestra.run.smithi161.stdout: python3-google-auth noarch 1:2.34.0-1.el9 epel 220 k 2024-09-21T03:32:00.709 INFO:teuthology.orchestra.run.smithi161.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-09-21T03:32:00.709 INFO:teuthology.orchestra.run.smithi161.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-09-21T03:32:00.709 INFO:teuthology.orchestra.run.smithi161.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-09-21T03:32:00.709 INFO:teuthology.orchestra.run.smithi161.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-09-21T03:32:00.709 INFO:teuthology.orchestra.run.smithi161.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-09-21T03:32:00.709 INFO:teuthology.orchestra.run.smithi161.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-09-21T03:32:00.709 INFO:teuthology.orchestra.run.smithi161.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-09-21T03:32:00.709 INFO:teuthology.orchestra.run.smithi161.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-09-21T03:32:00.710 INFO:teuthology.orchestra.run.smithi161.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-09-21T03:32:00.710 INFO:teuthology.orchestra.run.smithi161.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-09-21T03:32:00.710 INFO:teuthology.orchestra.run.smithi161.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-09-21T03:32:00.710 INFO:teuthology.orchestra.run.smithi161.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-09-21T03:32:00.710 INFO:teuthology.orchestra.run.smithi161.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-09-21T03:32:00.710 INFO:teuthology.orchestra.run.smithi161.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-09-21T03:32:00.710 INFO:teuthology.orchestra.run.smithi161.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-09-21T03:32:00.710 INFO:teuthology.orchestra.run.smithi161.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-09-21T03:32:00.710 INFO:teuthology.orchestra.run.smithi161.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-09-21T03:32:00.710 INFO:teuthology.orchestra.run.smithi161.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-09-21T03:32:00.710 INFO:teuthology.orchestra.run.smithi161.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-09-21T03:32:00.710 INFO:teuthology.orchestra.run.smithi161.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-09-21T03:32:00.710 INFO:teuthology.orchestra.run.smithi161.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-09-21T03:32:00.711 INFO:teuthology.orchestra.run.smithi161.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-09-21T03:32:00.711 INFO:teuthology.orchestra.run.smithi161.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-09-21T03:32:00.711 INFO:teuthology.orchestra.run.smithi161.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-09-21T03:32:00.711 INFO:teuthology.orchestra.run.smithi161.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-09-21T03:32:00.711 INFO:teuthology.orchestra.run.smithi161.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-09-21T03:32:00.711 INFO:teuthology.orchestra.run.smithi161.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-09-21T03:32:00.711 INFO:teuthology.orchestra.run.smithi161.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-09-21T03:32:00.711 INFO:teuthology.orchestra.run.smithi161.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-09-21T03:32:00.711 INFO:teuthology.orchestra.run.smithi161.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-09-21T03:32:00.711 INFO:teuthology.orchestra.run.smithi161.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-09-21T03:32:00.711 INFO:teuthology.orchestra.run.smithi161.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-09-21T03:32:00.712 INFO:teuthology.orchestra.run.smithi161.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-09-21T03:32:00.712 INFO:teuthology.orchestra.run.smithi161.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-09-21T03:32:00.712 INFO:teuthology.orchestra.run.smithi161.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-09-21T03:32:00.712 INFO:teuthology.orchestra.run.smithi161.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-09-21T03:32:00.712 INFO:teuthology.orchestra.run.smithi161.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-09-21T03:32:00.712 INFO:teuthology.orchestra.run.smithi161.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-09-21T03:32:00.712 INFO:teuthology.orchestra.run.smithi161.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-09-21T03:32:00.712 INFO:teuthology.orchestra.run.smithi161.stdout: python3-typing-extensions noarch 4.12.2-2.el9 epel 74 k 2024-09-21T03:32:00.712 INFO:teuthology.orchestra.run.smithi161.stdout: python3-urllib3 noarch 1.26.5-6.el9 baseos 215 k 2024-09-21T03:32:00.712 INFO:teuthology.orchestra.run.smithi161.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-09-21T03:32:00.712 INFO:teuthology.orchestra.run.smithi161.stdout: python3-webob noarch 1.8.8-2.el9 epel 230 k 2024-09-21T03:32:00.712 INFO:teuthology.orchestra.run.smithi161.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-09-21T03:32:00.712 INFO:teuthology.orchestra.run.smithi161.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-09-21T03:32:00.712 INFO:teuthology.orchestra.run.smithi161.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-09-21T03:32:00.713 INFO:teuthology.orchestra.run.smithi161.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-09-21T03:32:00.713 INFO:teuthology.orchestra.run.smithi161.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-09-21T03:32:00.713 INFO:teuthology.orchestra.run.smithi161.stdout: re2 x86_64 1:20211101-20.el9 epel 191 k 2024-09-21T03:32:00.713 INFO:teuthology.orchestra.run.smithi161.stdout: socat x86_64 1.7.4.1-6.el9 appstream 304 k 2024-09-21T03:32:00.713 INFO:teuthology.orchestra.run.smithi161.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-09-21T03:32:00.713 INFO:teuthology.orchestra.run.smithi161.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-09-21T03:32:00.713 INFO:teuthology.orchestra.run.smithi161.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-09-21T03:32:00.713 INFO:teuthology.orchestra.run.smithi161.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-09-21T03:32:00.713 INFO:teuthology.orchestra.run.smithi161.stdout:Installing weak dependencies: 2024-09-21T03:32:00.713 INFO:teuthology.orchestra.run.smithi161.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-09-21T03:32:00.713 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-09-21T03:32:00.713 INFO:teuthology.orchestra.run.smithi161.stdout:Transaction Summary 2024-09-21T03:32:00.713 INFO:teuthology.orchestra.run.smithi161.stdout:======================================================================================= 2024-09-21T03:32:00.713 INFO:teuthology.orchestra.run.smithi161.stdout:Install 149 Packages 2024-09-21T03:32:00.713 INFO:teuthology.orchestra.run.smithi161.stdout:Upgrade 8 Packages 2024-09-21T03:32:00.713 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-09-21T03:32:00.714 INFO:teuthology.orchestra.run.smithi161.stdout:Total download size: 228 M 2024-09-21T03:32:00.714 INFO:teuthology.orchestra.run.smithi161.stdout:Downloading Packages: 2024-09-21T03:32:01.632 INFO:teuthology.orchestra.run.smithi161.stdout:(1/157): ceph-19.3.0-5120.g18432218.el9.x86_64. 36 kB/s | 6.6 kB 00:00 2024-09-21T03:32:01.793 INFO:teuthology.orchestra.run.smithi150.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-21T03:32:01.794 INFO:teuthology.orchestra.run.smithi150.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-21T03:32:01.807 INFO:teuthology.orchestra.run.smithi150.stdout:Package NetworkManager-1:1.48.8-1.el9.x86_64 is already installed. 2024-09-21T03:32:01.808 INFO:teuthology.orchestra.run.smithi150.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-09-21T03:32:01.809 INFO:teuthology.orchestra.run.smithi150.stdout:Package util-linux-2.37.4-20.el9.x86_64 is already installed. 2024-09-21T03:32:01.810 INFO:teuthology.orchestra.run.smithi150.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-09-21T03:32:01.812 INFO:teuthology.orchestra.run.smithi150.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-09-21T03:32:01.815 INFO:teuthology.orchestra.run.smithi150.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-09-21T03:32:01.816 INFO:teuthology.orchestra.run.smithi150.stdout:Package xfsprogs-6.4.0-4.el9.x86_64 is already installed. 2024-09-21T03:32:01.879 INFO:teuthology.orchestra.run.smithi150.stdout:Dependencies resolved. 2024-09-21T03:32:01.903 INFO:teuthology.orchestra.run.smithi150.stdout:======================================================================================= 2024-09-21T03:32:01.903 INFO:teuthology.orchestra.run.smithi150.stdout: Package Arch Version Repository Size 2024-09-21T03:32:01.903 INFO:teuthology.orchestra.run.smithi150.stdout:======================================================================================= 2024-09-21T03:32:01.903 INFO:teuthology.orchestra.run.smithi150.stdout:Installing: 2024-09-21T03:32:01.903 INFO:teuthology.orchestra.run.smithi150.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-09-21T03:32:01.903 INFO:teuthology.orchestra.run.smithi150.stdout: ceph x86_64 2:19.3.0-5120.g18432218.el9 ceph 6.6 k 2024-09-21T03:32:01.903 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-base x86_64 2:19.3.0-5120.g18432218.el9 ceph 5.5 M 2024-09-21T03:32:01.903 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-fuse x86_64 2:19.3.0-5120.g18432218.el9 ceph 873 k 2024-09-21T03:32:01.903 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-immutable-object-cache x86_64 2:19.3.0-5120.g18432218.el9 ceph 148 k 2024-09-21T03:32:01.903 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-mgr x86_64 2:19.3.0-5120.g18432218.el9 ceph 1.5 M 2024-09-21T03:32:01.903 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-mgr-cephadm noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 154 k 2024-09-21T03:32:01.903 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-mgr-dashboard noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 14 M 2024-09-21T03:32:01.904 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-mgr-diskprediction-local noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 7.4 M 2024-09-21T03:32:01.904 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-mgr-rook noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 49 k 2024-09-21T03:32:01.904 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-radosgw x86_64 2:19.3.0-5120.g18432218.el9 ceph 10 M 2024-09-21T03:32:01.904 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-test x86_64 2:19.3.0-5120.g18432218.el9 ceph 40 M 2024-09-21T03:32:01.904 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-volume noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 285 k 2024-09-21T03:32:01.904 INFO:teuthology.orchestra.run.smithi150.stdout: cephadm noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 784 k 2024-09-21T03:32:01.904 INFO:teuthology.orchestra.run.smithi150.stdout: cephfs-mirror x86_64 2:19.3.0-5120.g18432218.el9 ceph 226 k 2024-09-21T03:32:01.904 INFO:teuthology.orchestra.run.smithi150.stdout: cephfs-shell x86_64 2:19.3.0-5120.g18432218.el9 ceph 27 k 2024-09-21T03:32:01.904 INFO:teuthology.orchestra.run.smithi150.stdout: cephfs-top noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 18 k 2024-09-21T03:32:01.904 INFO:teuthology.orchestra.run.smithi150.stdout: elfutils-libelf-devel x86_64 0.191-4.el9 appstream 22 k 2024-09-21T03:32:01.904 INFO:teuthology.orchestra.run.smithi150.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-09-21T03:32:01.904 INFO:teuthology.orchestra.run.smithi150.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-09-21T03:32:01.904 INFO:teuthology.orchestra.run.smithi150.stdout: libcephfs-devel x86_64 2:19.3.0-5120.g18432218.el9 ceph 32 k 2024-09-21T03:32:01.904 INFO:teuthology.orchestra.run.smithi150.stdout: libcephfs2 x86_64 2:19.3.0-5120.g18432218.el9 ceph 766 k 2024-09-21T03:32:01.904 INFO:teuthology.orchestra.run.smithi150.stdout: librados-devel x86_64 2:19.3.0-5120.g18432218.el9 ceph 127 k 2024-09-21T03:32:01.905 INFO:teuthology.orchestra.run.smithi150.stdout: libtool x86_64 2.4.6-46.el9 appstream 581 k 2024-09-21T03:32:01.905 INFO:teuthology.orchestra.run.smithi150.stdout: libuuid-devel x86_64 2.37.4-20.el9 appstream 24 k 2024-09-21T03:32:01.905 INFO:teuthology.orchestra.run.smithi150.stdout: openssl-devel x86_64 1:3.2.2-6.el9 appstream 4.4 M 2024-09-21T03:32:01.905 INFO:teuthology.orchestra.run.smithi150.stdout: postgresql x86_64 13.16-1.el9 appstream 1.6 M 2024-09-21T03:32:01.905 INFO:teuthology.orchestra.run.smithi150.stdout: postgresql-contrib x86_64 13.16-1.el9 appstream 910 k 2024-09-21T03:32:01.905 INFO:teuthology.orchestra.run.smithi150.stdout: postgresql-server x86_64 13.16-1.el9 appstream 5.8 M 2024-09-21T03:32:01.905 INFO:teuthology.orchestra.run.smithi150.stdout: python3-cephfs x86_64 2:19.3.0-5120.g18432218.el9 ceph 161 k 2024-09-21T03:32:01.905 INFO:teuthology.orchestra.run.smithi150.stdout: python3-rados x86_64 2:19.3.0-5120.g18432218.el9 ceph 321 k 2024-09-21T03:32:01.905 INFO:teuthology.orchestra.run.smithi150.stdout: python3-rbd x86_64 2:19.3.0-5120.g18432218.el9 ceph 307 k 2024-09-21T03:32:01.905 INFO:teuthology.orchestra.run.smithi150.stdout: python3-rgw x86_64 2:19.3.0-5120.g18432218.el9 ceph 99 k 2024-09-21T03:32:01.905 INFO:teuthology.orchestra.run.smithi150.stdout: rbd-fuse x86_64 2:19.3.0-5120.g18432218.el9 ceph 86 k 2024-09-21T03:32:01.905 INFO:teuthology.orchestra.run.smithi150.stdout: rbd-mirror x86_64 2:19.3.0-5120.g18432218.el9 ceph 3.1 M 2024-09-21T03:32:01.905 INFO:teuthology.orchestra.run.smithi150.stdout: rbd-nbd x86_64 2:19.3.0-5120.g18432218.el9 ceph 170 k 2024-09-21T03:32:01.905 INFO:teuthology.orchestra.run.smithi150.stdout: xfsprogs-devel x86_64 6.4.0-4.el9 appstream 96 k 2024-09-21T03:32:01.905 INFO:teuthology.orchestra.run.smithi150.stdout:Upgrading: 2024-09-21T03:32:01.905 INFO:teuthology.orchestra.run.smithi150.stdout: NetworkManager x86_64 1:1.51.0-1.el9 baseos 2.3 M 2024-09-21T03:32:01.906 INFO:teuthology.orchestra.run.smithi150.stdout: NetworkManager-libnm x86_64 1:1.51.0-1.el9 baseos 1.8 M 2024-09-21T03:32:01.906 INFO:teuthology.orchestra.run.smithi150.stdout: NetworkManager-team x86_64 1:1.51.0-1.el9 baseos 41 k 2024-09-21T03:32:01.906 INFO:teuthology.orchestra.run.smithi150.stdout: NetworkManager-tui x86_64 1:1.51.0-1.el9 baseos 249 k 2024-09-21T03:32:01.906 INFO:teuthology.orchestra.run.smithi150.stdout: librados2 x86_64 2:19.3.0-5120.g18432218.el9 ceph 3.4 M 2024-09-21T03:32:01.906 INFO:teuthology.orchestra.run.smithi150.stdout: librbd1 x86_64 2:19.3.0-5120.g18432218.el9 ceph 3.2 M 2024-09-21T03:32:01.906 INFO:teuthology.orchestra.run.smithi150.stdout: openssl x86_64 1:3.2.2-6.el9 baseos 1.4 M 2024-09-21T03:32:01.906 INFO:teuthology.orchestra.run.smithi150.stdout: openssl-libs x86_64 1:3.2.2-6.el9 baseos 2.4 M 2024-09-21T03:32:01.906 INFO:teuthology.orchestra.run.smithi150.stdout:Installing dependencies: 2024-09-21T03:32:01.906 INFO:teuthology.orchestra.run.smithi150.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-09-21T03:32:01.906 INFO:teuthology.orchestra.run.smithi150.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-09-21T03:32:01.906 INFO:teuthology.orchestra.run.smithi150.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-09-21T03:32:01.906 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-common x86_64 2:19.3.0-5120.g18432218.el9 ceph 20 M 2024-09-21T03:32:01.906 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-grafana-dashboards noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 34 k 2024-09-21T03:32:01.906 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-mds x86_64 2:19.3.0-5120.g18432218.el9 ceph 2.3 M 2024-09-21T03:32:01.906 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-mgr-modules-core noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 292 k 2024-09-21T03:32:01.907 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-mon x86_64 2:19.3.0-5120.g18432218.el9 ceph 5.0 M 2024-09-21T03:32:01.907 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-osd x86_64 2:19.3.0-5120.g18432218.el9 ceph 18 M 2024-09-21T03:32:01.907 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-prometheus-alerts noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 16 k 2024-09-21T03:32:01.907 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-selinux x86_64 2:19.3.0-5120.g18432218.el9 ceph 25 k 2024-09-21T03:32:01.907 INFO:teuthology.orchestra.run.smithi150.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-09-21T03:32:01.907 INFO:teuthology.orchestra.run.smithi150.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-09-21T03:32:01.907 INFO:teuthology.orchestra.run.smithi150.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-09-21T03:32:01.907 INFO:teuthology.orchestra.run.smithi150.stdout: fuse x86_64 2.9.9-16.el9 baseos 80 k 2024-09-21T03:32:01.907 INFO:teuthology.orchestra.run.smithi150.stdout: gperftools-libs x86_64 2.9.1-3.el9 epel 308 k 2024-09-21T03:32:01.907 INFO:teuthology.orchestra.run.smithi150.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-09-21T03:32:01.907 INFO:teuthology.orchestra.run.smithi150.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-09-21T03:32:01.907 INFO:teuthology.orchestra.run.smithi150.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-09-21T03:32:01.907 INFO:teuthology.orchestra.run.smithi150.stdout: libcephsqlite x86_64 2:19.3.0-5120.g18432218.el9 ceph 158 k 2024-09-21T03:32:01.907 INFO:teuthology.orchestra.run.smithi150.stdout: libgfortran x86_64 11.5.0-2.el9 baseos 801 k 2024-09-21T03:32:01.908 INFO:teuthology.orchestra.run.smithi150.stdout: libnbd x86_64 1.20.2-2.el9 appstream 171 k 2024-09-21T03:32:01.908 INFO:teuthology.orchestra.run.smithi150.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-09-21T03:32:01.908 INFO:teuthology.orchestra.run.smithi150.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-09-21T03:32:01.908 INFO:teuthology.orchestra.run.smithi150.stdout: libquadmath x86_64 11.5.0-2.el9 baseos 191 k 2024-09-21T03:32:01.908 INFO:teuthology.orchestra.run.smithi150.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-09-21T03:32:01.908 INFO:teuthology.orchestra.run.smithi150.stdout: libradosstriper1 x86_64 2:19.3.0-5120.g18432218.el9 ceph 507 k 2024-09-21T03:32:01.908 INFO:teuthology.orchestra.run.smithi150.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-09-21T03:32:01.908 INFO:teuthology.orchestra.run.smithi150.stdout: librgw2 x86_64 2:19.3.0-5120.g18432218.el9 ceph 5.3 M 2024-09-21T03:32:01.908 INFO:teuthology.orchestra.run.smithi150.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-09-21T03:32:01.908 INFO:teuthology.orchestra.run.smithi150.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-09-21T03:32:01.908 INFO:teuthology.orchestra.run.smithi150.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-09-21T03:32:01.908 INFO:teuthology.orchestra.run.smithi150.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-09-21T03:32:01.908 INFO:teuthology.orchestra.run.smithi150.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-09-21T03:32:01.908 INFO:teuthology.orchestra.run.smithi150.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2024-09-21T03:32:01.908 INFO:teuthology.orchestra.run.smithi150.stdout: openblas x86_64 0.3.26-2.el9 appstream 38 k 2024-09-21T03:32:01.908 INFO:teuthology.orchestra.run.smithi150.stdout: openblas-openmp x86_64 0.3.26-2.el9 appstream 4.9 M 2024-09-21T03:32:01.909 INFO:teuthology.orchestra.run.smithi150.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-09-21T03:32:01.909 INFO:teuthology.orchestra.run.smithi150.stdout: postgresql-private-libs x86_64 13.16-1.el9 appstream 137 k 2024-09-21T03:32:01.909 INFO:teuthology.orchestra.run.smithi150.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-09-21T03:32:01.909 INFO:teuthology.orchestra.run.smithi150.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-09-21T03:32:01.909 INFO:teuthology.orchestra.run.smithi150.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-09-21T03:32:01.909 INFO:teuthology.orchestra.run.smithi150.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-09-21T03:32:01.909 INFO:teuthology.orchestra.run.smithi150.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-09-21T03:32:01.909 INFO:teuthology.orchestra.run.smithi150.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-09-21T03:32:01.909 INFO:teuthology.orchestra.run.smithi150.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-09-21T03:32:01.909 INFO:teuthology.orchestra.run.smithi150.stdout: python3-ceph-argparse x86_64 2:19.3.0-5120.g18432218.el9 ceph 45 k 2024-09-21T03:32:01.909 INFO:teuthology.orchestra.run.smithi150.stdout: python3-ceph-common x86_64 2:19.3.0-5120.g18432218.el9 ceph 148 k 2024-09-21T03:32:01.909 INFO:teuthology.orchestra.run.smithi150.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2024-09-21T03:32:01.909 INFO:teuthology.orchestra.run.smithi150.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-09-21T03:32:01.909 INFO:teuthology.orchestra.run.smithi150.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-09-21T03:32:01.909 INFO:teuthology.orchestra.run.smithi150.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-09-21T03:32:01.910 INFO:teuthology.orchestra.run.smithi150.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-09-21T03:32:01.910 INFO:teuthology.orchestra.run.smithi150.stdout: python3-colorama noarch 0.4.6-3.el9 appstream 61 k 2024-09-21T03:32:01.910 INFO:teuthology.orchestra.run.smithi150.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-09-21T03:32:01.910 INFO:teuthology.orchestra.run.smithi150.stdout: python3-devel x86_64 3.9.19-8.el9 appstream 245 k 2024-09-21T03:32:01.910 INFO:teuthology.orchestra.run.smithi150.stdout: python3-google-auth noarch 1:2.34.0-1.el9 epel 220 k 2024-09-21T03:32:01.910 INFO:teuthology.orchestra.run.smithi150.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-09-21T03:32:01.910 INFO:teuthology.orchestra.run.smithi150.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-09-21T03:32:01.910 INFO:teuthology.orchestra.run.smithi150.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-09-21T03:32:01.910 INFO:teuthology.orchestra.run.smithi150.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-09-21T03:32:01.910 INFO:teuthology.orchestra.run.smithi150.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-09-21T03:32:01.910 INFO:teuthology.orchestra.run.smithi150.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-09-21T03:32:01.910 INFO:teuthology.orchestra.run.smithi150.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-09-21T03:32:01.910 INFO:teuthology.orchestra.run.smithi150.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-09-21T03:32:01.911 INFO:teuthology.orchestra.run.smithi150.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-09-21T03:32:01.911 INFO:teuthology.orchestra.run.smithi150.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-09-21T03:32:01.911 INFO:teuthology.orchestra.run.smithi150.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-09-21T03:32:01.911 INFO:teuthology.orchestra.run.smithi150.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-09-21T03:32:01.911 INFO:teuthology.orchestra.run.smithi150.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-09-21T03:32:01.911 INFO:teuthology.orchestra.run.smithi150.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-09-21T03:32:01.911 INFO:teuthology.orchestra.run.smithi150.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-09-21T03:32:01.911 INFO:teuthology.orchestra.run.smithi150.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-09-21T03:32:01.911 INFO:teuthology.orchestra.run.smithi150.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-09-21T03:32:01.911 INFO:teuthology.orchestra.run.smithi150.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-09-21T03:32:01.911 INFO:teuthology.orchestra.run.smithi150.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-09-21T03:32:01.911 INFO:teuthology.orchestra.run.smithi150.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-09-21T03:32:01.911 INFO:teuthology.orchestra.run.smithi150.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-09-21T03:32:01.911 INFO:teuthology.orchestra.run.smithi150.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-09-21T03:32:01.912 INFO:teuthology.orchestra.run.smithi150.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-09-21T03:32:01.912 INFO:teuthology.orchestra.run.smithi150.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-09-21T03:32:01.912 INFO:teuthology.orchestra.run.smithi150.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-09-21T03:32:01.912 INFO:teuthology.orchestra.run.smithi150.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-09-21T03:32:01.912 INFO:teuthology.orchestra.run.smithi150.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-09-21T03:32:01.912 INFO:teuthology.orchestra.run.smithi150.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-09-21T03:32:01.912 INFO:teuthology.orchestra.run.smithi150.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-09-21T03:32:01.912 INFO:teuthology.orchestra.run.smithi150.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-09-21T03:32:01.912 INFO:teuthology.orchestra.run.smithi150.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-09-21T03:32:01.912 INFO:teuthology.orchestra.run.smithi150.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-09-21T03:32:01.912 INFO:teuthology.orchestra.run.smithi150.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-09-21T03:32:01.912 INFO:teuthology.orchestra.run.smithi150.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-09-21T03:32:01.912 INFO:teuthology.orchestra.run.smithi150.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-09-21T03:32:01.912 INFO:teuthology.orchestra.run.smithi150.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-09-21T03:32:01.912 INFO:teuthology.orchestra.run.smithi150.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-09-21T03:32:01.912 INFO:teuthology.orchestra.run.smithi150.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-09-21T03:32:01.913 INFO:teuthology.orchestra.run.smithi150.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-09-21T03:32:01.913 INFO:teuthology.orchestra.run.smithi150.stdout: python3-typing-extensions noarch 4.12.2-2.el9 epel 74 k 2024-09-21T03:32:01.913 INFO:teuthology.orchestra.run.smithi150.stdout: python3-urllib3 noarch 1.26.5-6.el9 baseos 215 k 2024-09-21T03:32:01.913 INFO:teuthology.orchestra.run.smithi150.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-09-21T03:32:01.913 INFO:teuthology.orchestra.run.smithi150.stdout: python3-webob noarch 1.8.8-2.el9 epel 230 k 2024-09-21T03:32:01.913 INFO:teuthology.orchestra.run.smithi150.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-09-21T03:32:01.913 INFO:teuthology.orchestra.run.smithi150.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-09-21T03:32:01.913 INFO:teuthology.orchestra.run.smithi150.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-09-21T03:32:01.913 INFO:teuthology.orchestra.run.smithi150.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-09-21T03:32:01.913 INFO:teuthology.orchestra.run.smithi150.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-09-21T03:32:01.913 INFO:teuthology.orchestra.run.smithi150.stdout: re2 x86_64 1:20211101-20.el9 epel 191 k 2024-09-21T03:32:01.913 INFO:teuthology.orchestra.run.smithi150.stdout: socat x86_64 1.7.4.1-6.el9 appstream 304 k 2024-09-21T03:32:01.913 INFO:teuthology.orchestra.run.smithi150.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-09-21T03:32:01.913 INFO:teuthology.orchestra.run.smithi150.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-09-21T03:32:01.913 INFO:teuthology.orchestra.run.smithi150.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-09-21T03:32:01.913 INFO:teuthology.orchestra.run.smithi150.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-09-21T03:32:01.914 INFO:teuthology.orchestra.run.smithi150.stdout:Installing weak dependencies: 2024-09-21T03:32:01.914 INFO:teuthology.orchestra.run.smithi150.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-09-21T03:32:01.914 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-09-21T03:32:01.914 INFO:teuthology.orchestra.run.smithi150.stdout:Transaction Summary 2024-09-21T03:32:01.914 INFO:teuthology.orchestra.run.smithi150.stdout:======================================================================================= 2024-09-21T03:32:01.914 INFO:teuthology.orchestra.run.smithi150.stdout:Install 149 Packages 2024-09-21T03:32:01.914 INFO:teuthology.orchestra.run.smithi150.stdout:Upgrade 8 Packages 2024-09-21T03:32:01.914 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-09-21T03:32:01.914 INFO:teuthology.orchestra.run.smithi150.stdout:Total download size: 228 M 2024-09-21T03:32:01.914 INFO:teuthology.orchestra.run.smithi150.stdout:Downloading Packages: 2024-09-21T03:32:01.938 INFO:teuthology.orchestra.run.smithi039.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-21T03:32:01.939 INFO:teuthology.orchestra.run.smithi039.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-21T03:32:01.950 INFO:teuthology.orchestra.run.smithi039.stdout:Package NetworkManager-1:1.48.8-1.el9.x86_64 is already installed. 2024-09-21T03:32:01.951 INFO:teuthology.orchestra.run.smithi039.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-09-21T03:32:01.952 INFO:teuthology.orchestra.run.smithi039.stdout:Package util-linux-2.37.4-20.el9.x86_64 is already installed. 2024-09-21T03:32:01.953 INFO:teuthology.orchestra.run.smithi039.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-09-21T03:32:01.955 INFO:teuthology.orchestra.run.smithi039.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-09-21T03:32:01.957 INFO:teuthology.orchestra.run.smithi039.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-09-21T03:32:01.958 INFO:teuthology.orchestra.run.smithi039.stdout:Package xfsprogs-6.4.0-4.el9.x86_64 is already installed. 2024-09-21T03:32:02.025 INFO:teuthology.orchestra.run.smithi039.stdout:Dependencies resolved. 2024-09-21T03:32:02.032 INFO:teuthology.orchestra.run.smithi161.stdout:(2/157): ceph-fuse-19.3.0-5120.g18432218.el9.x8 2.1 MB/s | 873 kB 00:00 2024-09-21T03:32:02.044 INFO:teuthology.orchestra.run.smithi039.stdout:======================================================================================= 2024-09-21T03:32:02.044 INFO:teuthology.orchestra.run.smithi039.stdout: Package Arch Version Repository Size 2024-09-21T03:32:02.044 INFO:teuthology.orchestra.run.smithi039.stdout:======================================================================================= 2024-09-21T03:32:02.044 INFO:teuthology.orchestra.run.smithi039.stdout:Installing: 2024-09-21T03:32:02.044 INFO:teuthology.orchestra.run.smithi039.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-09-21T03:32:02.044 INFO:teuthology.orchestra.run.smithi039.stdout: ceph x86_64 2:19.3.0-5120.g18432218.el9 ceph 6.6 k 2024-09-21T03:32:02.044 INFO:teuthology.orchestra.run.smithi039.stdout: ceph-base x86_64 2:19.3.0-5120.g18432218.el9 ceph 5.5 M 2024-09-21T03:32:02.044 INFO:teuthology.orchestra.run.smithi039.stdout: ceph-fuse x86_64 2:19.3.0-5120.g18432218.el9 ceph 873 k 2024-09-21T03:32:02.044 INFO:teuthology.orchestra.run.smithi039.stdout: ceph-immutable-object-cache x86_64 2:19.3.0-5120.g18432218.el9 ceph 148 k 2024-09-21T03:32:02.044 INFO:teuthology.orchestra.run.smithi039.stdout: ceph-mgr x86_64 2:19.3.0-5120.g18432218.el9 ceph 1.5 M 2024-09-21T03:32:02.044 INFO:teuthology.orchestra.run.smithi039.stdout: ceph-mgr-cephadm noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 154 k 2024-09-21T03:32:02.045 INFO:teuthology.orchestra.run.smithi039.stdout: ceph-mgr-dashboard noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 14 M 2024-09-21T03:32:02.045 INFO:teuthology.orchestra.run.smithi039.stdout: ceph-mgr-diskprediction-local noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 7.4 M 2024-09-21T03:32:02.045 INFO:teuthology.orchestra.run.smithi039.stdout: ceph-mgr-rook noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 49 k 2024-09-21T03:32:02.045 INFO:teuthology.orchestra.run.smithi039.stdout: ceph-radosgw x86_64 2:19.3.0-5120.g18432218.el9 ceph 10 M 2024-09-21T03:32:02.045 INFO:teuthology.orchestra.run.smithi039.stdout: ceph-test x86_64 2:19.3.0-5120.g18432218.el9 ceph 40 M 2024-09-21T03:32:02.045 INFO:teuthology.orchestra.run.smithi039.stdout: ceph-volume noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 285 k 2024-09-21T03:32:02.045 INFO:teuthology.orchestra.run.smithi039.stdout: cephadm noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 784 k 2024-09-21T03:32:02.045 INFO:teuthology.orchestra.run.smithi039.stdout: cephfs-mirror x86_64 2:19.3.0-5120.g18432218.el9 ceph 226 k 2024-09-21T03:32:02.045 INFO:teuthology.orchestra.run.smithi039.stdout: cephfs-shell x86_64 2:19.3.0-5120.g18432218.el9 ceph 27 k 2024-09-21T03:32:02.045 INFO:teuthology.orchestra.run.smithi039.stdout: cephfs-top noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 18 k 2024-09-21T03:32:02.045 INFO:teuthology.orchestra.run.smithi039.stdout: elfutils-libelf-devel x86_64 0.191-4.el9 appstream 22 k 2024-09-21T03:32:02.045 INFO:teuthology.orchestra.run.smithi039.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-09-21T03:32:02.045 INFO:teuthology.orchestra.run.smithi039.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-09-21T03:32:02.045 INFO:teuthology.orchestra.run.smithi039.stdout: libcephfs-devel x86_64 2:19.3.0-5120.g18432218.el9 ceph 32 k 2024-09-21T03:32:02.045 INFO:teuthology.orchestra.run.smithi039.stdout: libcephfs2 x86_64 2:19.3.0-5120.g18432218.el9 ceph 766 k 2024-09-21T03:32:02.046 INFO:teuthology.orchestra.run.smithi039.stdout: librados-devel x86_64 2:19.3.0-5120.g18432218.el9 ceph 127 k 2024-09-21T03:32:02.046 INFO:teuthology.orchestra.run.smithi039.stdout: libtool x86_64 2.4.6-46.el9 appstream 581 k 2024-09-21T03:32:02.046 INFO:teuthology.orchestra.run.smithi039.stdout: libuuid-devel x86_64 2.37.4-20.el9 appstream 24 k 2024-09-21T03:32:02.046 INFO:teuthology.orchestra.run.smithi039.stdout: openssl-devel x86_64 1:3.2.2-6.el9 appstream 4.4 M 2024-09-21T03:32:02.046 INFO:teuthology.orchestra.run.smithi039.stdout: postgresql x86_64 13.16-1.el9 appstream 1.6 M 2024-09-21T03:32:02.046 INFO:teuthology.orchestra.run.smithi039.stdout: postgresql-contrib x86_64 13.16-1.el9 appstream 910 k 2024-09-21T03:32:02.046 INFO:teuthology.orchestra.run.smithi039.stdout: postgresql-server x86_64 13.16-1.el9 appstream 5.8 M 2024-09-21T03:32:02.046 INFO:teuthology.orchestra.run.smithi039.stdout: python3-cephfs x86_64 2:19.3.0-5120.g18432218.el9 ceph 161 k 2024-09-21T03:32:02.046 INFO:teuthology.orchestra.run.smithi039.stdout: python3-rados x86_64 2:19.3.0-5120.g18432218.el9 ceph 321 k 2024-09-21T03:32:02.046 INFO:teuthology.orchestra.run.smithi039.stdout: python3-rbd x86_64 2:19.3.0-5120.g18432218.el9 ceph 307 k 2024-09-21T03:32:02.046 INFO:teuthology.orchestra.run.smithi039.stdout: python3-rgw x86_64 2:19.3.0-5120.g18432218.el9 ceph 99 k 2024-09-21T03:32:02.046 INFO:teuthology.orchestra.run.smithi039.stdout: rbd-fuse x86_64 2:19.3.0-5120.g18432218.el9 ceph 86 k 2024-09-21T03:32:02.046 INFO:teuthology.orchestra.run.smithi039.stdout: rbd-mirror x86_64 2:19.3.0-5120.g18432218.el9 ceph 3.1 M 2024-09-21T03:32:02.046 INFO:teuthology.orchestra.run.smithi039.stdout: rbd-nbd x86_64 2:19.3.0-5120.g18432218.el9 ceph 170 k 2024-09-21T03:32:02.046 INFO:teuthology.orchestra.run.smithi039.stdout: xfsprogs-devel x86_64 6.4.0-4.el9 appstream 96 k 2024-09-21T03:32:02.047 INFO:teuthology.orchestra.run.smithi039.stdout:Upgrading: 2024-09-21T03:32:02.047 INFO:teuthology.orchestra.run.smithi039.stdout: NetworkManager x86_64 1:1.51.0-1.el9 baseos 2.3 M 2024-09-21T03:32:02.047 INFO:teuthology.orchestra.run.smithi039.stdout: NetworkManager-libnm x86_64 1:1.51.0-1.el9 baseos 1.8 M 2024-09-21T03:32:02.047 INFO:teuthology.orchestra.run.smithi039.stdout: NetworkManager-team x86_64 1:1.51.0-1.el9 baseos 41 k 2024-09-21T03:32:02.047 INFO:teuthology.orchestra.run.smithi039.stdout: NetworkManager-tui x86_64 1:1.51.0-1.el9 baseos 249 k 2024-09-21T03:32:02.047 INFO:teuthology.orchestra.run.smithi039.stdout: librados2 x86_64 2:19.3.0-5120.g18432218.el9 ceph 3.4 M 2024-09-21T03:32:02.047 INFO:teuthology.orchestra.run.smithi039.stdout: librbd1 x86_64 2:19.3.0-5120.g18432218.el9 ceph 3.2 M 2024-09-21T03:32:02.047 INFO:teuthology.orchestra.run.smithi039.stdout: openssl x86_64 1:3.2.2-6.el9 baseos 1.4 M 2024-09-21T03:32:02.047 INFO:teuthology.orchestra.run.smithi039.stdout: openssl-libs x86_64 1:3.2.2-6.el9 baseos 2.4 M 2024-09-21T03:32:02.047 INFO:teuthology.orchestra.run.smithi039.stdout:Installing dependencies: 2024-09-21T03:32:02.047 INFO:teuthology.orchestra.run.smithi039.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-09-21T03:32:02.047 INFO:teuthology.orchestra.run.smithi039.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-09-21T03:32:02.047 INFO:teuthology.orchestra.run.smithi039.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-09-21T03:32:02.047 INFO:teuthology.orchestra.run.smithi039.stdout: ceph-common x86_64 2:19.3.0-5120.g18432218.el9 ceph 20 M 2024-09-21T03:32:02.047 INFO:teuthology.orchestra.run.smithi039.stdout: ceph-grafana-dashboards noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 34 k 2024-09-21T03:32:02.047 INFO:teuthology.orchestra.run.smithi039.stdout: ceph-mds x86_64 2:19.3.0-5120.g18432218.el9 ceph 2.3 M 2024-09-21T03:32:02.048 INFO:teuthology.orchestra.run.smithi039.stdout: ceph-mgr-modules-core noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 292 k 2024-09-21T03:32:02.048 INFO:teuthology.orchestra.run.smithi039.stdout: ceph-mon x86_64 2:19.3.0-5120.g18432218.el9 ceph 5.0 M 2024-09-21T03:32:02.048 INFO:teuthology.orchestra.run.smithi039.stdout: ceph-osd x86_64 2:19.3.0-5120.g18432218.el9 ceph 18 M 2024-09-21T03:32:02.048 INFO:teuthology.orchestra.run.smithi039.stdout: ceph-prometheus-alerts noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 16 k 2024-09-21T03:32:02.048 INFO:teuthology.orchestra.run.smithi039.stdout: ceph-selinux x86_64 2:19.3.0-5120.g18432218.el9 ceph 25 k 2024-09-21T03:32:02.048 INFO:teuthology.orchestra.run.smithi039.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-09-21T03:32:02.048 INFO:teuthology.orchestra.run.smithi039.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-09-21T03:32:02.048 INFO:teuthology.orchestra.run.smithi039.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-09-21T03:32:02.048 INFO:teuthology.orchestra.run.smithi039.stdout: fuse x86_64 2.9.9-16.el9 baseos 80 k 2024-09-21T03:32:02.048 INFO:teuthology.orchestra.run.smithi039.stdout: gperftools-libs x86_64 2.9.1-3.el9 epel 308 k 2024-09-21T03:32:02.048 INFO:teuthology.orchestra.run.smithi039.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-09-21T03:32:02.048 INFO:teuthology.orchestra.run.smithi039.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-09-21T03:32:02.048 INFO:teuthology.orchestra.run.smithi039.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-09-21T03:32:02.048 INFO:teuthology.orchestra.run.smithi039.stdout: libcephsqlite x86_64 2:19.3.0-5120.g18432218.el9 ceph 158 k 2024-09-21T03:32:02.048 INFO:teuthology.orchestra.run.smithi039.stdout: libgfortran x86_64 11.5.0-2.el9 baseos 801 k 2024-09-21T03:32:02.049 INFO:teuthology.orchestra.run.smithi039.stdout: libnbd x86_64 1.20.2-2.el9 appstream 171 k 2024-09-21T03:32:02.049 INFO:teuthology.orchestra.run.smithi039.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-09-21T03:32:02.049 INFO:teuthology.orchestra.run.smithi039.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-09-21T03:32:02.049 INFO:teuthology.orchestra.run.smithi039.stdout: libquadmath x86_64 11.5.0-2.el9 baseos 191 k 2024-09-21T03:32:02.049 INFO:teuthology.orchestra.run.smithi039.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-09-21T03:32:02.049 INFO:teuthology.orchestra.run.smithi039.stdout: libradosstriper1 x86_64 2:19.3.0-5120.g18432218.el9 ceph 507 k 2024-09-21T03:32:02.049 INFO:teuthology.orchestra.run.smithi039.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-09-21T03:32:02.049 INFO:teuthology.orchestra.run.smithi039.stdout: librgw2 x86_64 2:19.3.0-5120.g18432218.el9 ceph 5.3 M 2024-09-21T03:32:02.049 INFO:teuthology.orchestra.run.smithi039.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-09-21T03:32:02.049 INFO:teuthology.orchestra.run.smithi039.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-09-21T03:32:02.049 INFO:teuthology.orchestra.run.smithi039.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-09-21T03:32:02.049 INFO:teuthology.orchestra.run.smithi039.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-09-21T03:32:02.049 INFO:teuthology.orchestra.run.smithi039.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-09-21T03:32:02.049 INFO:teuthology.orchestra.run.smithi039.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2024-09-21T03:32:02.049 INFO:teuthology.orchestra.run.smithi039.stdout: openblas x86_64 0.3.26-2.el9 appstream 38 k 2024-09-21T03:32:02.050 INFO:teuthology.orchestra.run.smithi039.stdout: openblas-openmp x86_64 0.3.26-2.el9 appstream 4.9 M 2024-09-21T03:32:02.050 INFO:teuthology.orchestra.run.smithi039.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-09-21T03:32:02.050 INFO:teuthology.orchestra.run.smithi039.stdout: postgresql-private-libs x86_64 13.16-1.el9 appstream 137 k 2024-09-21T03:32:02.050 INFO:teuthology.orchestra.run.smithi039.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-09-21T03:32:02.050 INFO:teuthology.orchestra.run.smithi039.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-09-21T03:32:02.050 INFO:teuthology.orchestra.run.smithi039.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-09-21T03:32:02.050 INFO:teuthology.orchestra.run.smithi039.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-09-21T03:32:02.050 INFO:teuthology.orchestra.run.smithi039.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-09-21T03:32:02.050 INFO:teuthology.orchestra.run.smithi039.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-09-21T03:32:02.050 INFO:teuthology.orchestra.run.smithi039.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-09-21T03:32:02.050 INFO:teuthology.orchestra.run.smithi039.stdout: python3-ceph-argparse x86_64 2:19.3.0-5120.g18432218.el9 ceph 45 k 2024-09-21T03:32:02.050 INFO:teuthology.orchestra.run.smithi039.stdout: python3-ceph-common x86_64 2:19.3.0-5120.g18432218.el9 ceph 148 k 2024-09-21T03:32:02.050 INFO:teuthology.orchestra.run.smithi039.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2024-09-21T03:32:02.051 INFO:teuthology.orchestra.run.smithi039.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-09-21T03:32:02.051 INFO:teuthology.orchestra.run.smithi039.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-09-21T03:32:02.051 INFO:teuthology.orchestra.run.smithi039.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-09-21T03:32:02.051 INFO:teuthology.orchestra.run.smithi039.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-09-21T03:32:02.051 INFO:teuthology.orchestra.run.smithi039.stdout: python3-colorama noarch 0.4.6-3.el9 appstream 61 k 2024-09-21T03:32:02.051 INFO:teuthology.orchestra.run.smithi039.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-09-21T03:32:02.051 INFO:teuthology.orchestra.run.smithi039.stdout: python3-devel x86_64 3.9.19-8.el9 appstream 245 k 2024-09-21T03:32:02.051 INFO:teuthology.orchestra.run.smithi039.stdout: python3-google-auth noarch 1:2.34.0-1.el9 epel 220 k 2024-09-21T03:32:02.051 INFO:teuthology.orchestra.run.smithi039.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-09-21T03:32:02.051 INFO:teuthology.orchestra.run.smithi039.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-09-21T03:32:02.051 INFO:teuthology.orchestra.run.smithi039.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-09-21T03:32:02.051 INFO:teuthology.orchestra.run.smithi039.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-09-21T03:32:02.051 INFO:teuthology.orchestra.run.smithi039.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-09-21T03:32:02.051 INFO:teuthology.orchestra.run.smithi039.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-09-21T03:32:02.052 INFO:teuthology.orchestra.run.smithi039.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-09-21T03:32:02.052 INFO:teuthology.orchestra.run.smithi039.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-09-21T03:32:02.052 INFO:teuthology.orchestra.run.smithi039.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-09-21T03:32:02.052 INFO:teuthology.orchestra.run.smithi039.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-09-21T03:32:02.052 INFO:teuthology.orchestra.run.smithi039.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-09-21T03:32:02.052 INFO:teuthology.orchestra.run.smithi039.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-09-21T03:32:02.052 INFO:teuthology.orchestra.run.smithi039.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-09-21T03:32:02.052 INFO:teuthology.orchestra.run.smithi039.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-09-21T03:32:02.052 INFO:teuthology.orchestra.run.smithi039.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-09-21T03:32:02.052 INFO:teuthology.orchestra.run.smithi039.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-09-21T03:32:02.052 INFO:teuthology.orchestra.run.smithi039.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-09-21T03:32:02.052 INFO:teuthology.orchestra.run.smithi039.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-09-21T03:32:02.052 INFO:teuthology.orchestra.run.smithi039.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-09-21T03:32:02.052 INFO:teuthology.orchestra.run.smithi039.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-09-21T03:32:02.052 INFO:teuthology.orchestra.run.smithi039.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-09-21T03:32:02.052 INFO:teuthology.orchestra.run.smithi039.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-09-21T03:32:02.053 INFO:teuthology.orchestra.run.smithi039.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-09-21T03:32:02.053 INFO:teuthology.orchestra.run.smithi039.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-09-21T03:32:02.053 INFO:teuthology.orchestra.run.smithi039.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-09-21T03:32:02.053 INFO:teuthology.orchestra.run.smithi039.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-09-21T03:32:02.053 INFO:teuthology.orchestra.run.smithi039.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-09-21T03:32:02.053 INFO:teuthology.orchestra.run.smithi039.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-09-21T03:32:02.053 INFO:teuthology.orchestra.run.smithi039.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-09-21T03:32:02.053 INFO:teuthology.orchestra.run.smithi039.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-09-21T03:32:02.053 INFO:teuthology.orchestra.run.smithi039.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-09-21T03:32:02.053 INFO:teuthology.orchestra.run.smithi039.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-09-21T03:32:02.053 INFO:teuthology.orchestra.run.smithi039.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-09-21T03:32:02.053 INFO:teuthology.orchestra.run.smithi039.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-09-21T03:32:02.053 INFO:teuthology.orchestra.run.smithi039.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-09-21T03:32:02.053 INFO:teuthology.orchestra.run.smithi039.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-09-21T03:32:02.053 INFO:teuthology.orchestra.run.smithi039.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-09-21T03:32:02.053 INFO:teuthology.orchestra.run.smithi039.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-09-21T03:32:02.054 INFO:teuthology.orchestra.run.smithi039.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-09-21T03:32:02.054 INFO:teuthology.orchestra.run.smithi039.stdout: python3-typing-extensions noarch 4.12.2-2.el9 epel 74 k 2024-09-21T03:32:02.054 INFO:teuthology.orchestra.run.smithi039.stdout: python3-urllib3 noarch 1.26.5-6.el9 baseos 215 k 2024-09-21T03:32:02.054 INFO:teuthology.orchestra.run.smithi039.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-09-21T03:32:02.054 INFO:teuthology.orchestra.run.smithi039.stdout: python3-webob noarch 1.8.8-2.el9 epel 230 k 2024-09-21T03:32:02.054 INFO:teuthology.orchestra.run.smithi039.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-09-21T03:32:02.054 INFO:teuthology.orchestra.run.smithi039.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-09-21T03:32:02.054 INFO:teuthology.orchestra.run.smithi039.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-09-21T03:32:02.054 INFO:teuthology.orchestra.run.smithi039.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-09-21T03:32:02.054 INFO:teuthology.orchestra.run.smithi039.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-09-21T03:32:02.054 INFO:teuthology.orchestra.run.smithi039.stdout: re2 x86_64 1:20211101-20.el9 epel 191 k 2024-09-21T03:32:02.054 INFO:teuthology.orchestra.run.smithi039.stdout: socat x86_64 1.7.4.1-6.el9 appstream 304 k 2024-09-21T03:32:02.054 INFO:teuthology.orchestra.run.smithi039.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-09-21T03:32:02.054 INFO:teuthology.orchestra.run.smithi039.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-09-21T03:32:02.055 INFO:teuthology.orchestra.run.smithi039.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-09-21T03:32:02.055 INFO:teuthology.orchestra.run.smithi039.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-09-21T03:32:02.055 INFO:teuthology.orchestra.run.smithi039.stdout:Installing weak dependencies: 2024-09-21T03:32:02.055 INFO:teuthology.orchestra.run.smithi039.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-09-21T03:32:02.055 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T03:32:02.055 INFO:teuthology.orchestra.run.smithi039.stdout:Transaction Summary 2024-09-21T03:32:02.055 INFO:teuthology.orchestra.run.smithi039.stdout:======================================================================================= 2024-09-21T03:32:02.055 INFO:teuthology.orchestra.run.smithi039.stdout:Install 149 Packages 2024-09-21T03:32:02.055 INFO:teuthology.orchestra.run.smithi039.stdout:Upgrade 8 Packages 2024-09-21T03:32:02.055 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T03:32:02.055 INFO:teuthology.orchestra.run.smithi039.stdout:Total download size: 228 M 2024-09-21T03:32:02.056 INFO:teuthology.orchestra.run.smithi039.stdout:Downloading Packages: 2024-09-21T03:32:02.099 INFO:teuthology.orchestra.run.smithi161.stdout:(3/157): ceph-immutable-object-cache-19.3.0-512 2.2 MB/s | 148 kB 00:00 2024-09-21T03:32:02.507 INFO:teuthology.orchestra.run.smithi150.stdout:(1/157): ceph-19.3.0-5120.g18432218.el9.x86_64. 25 kB/s | 6.6 kB 00:00 2024-09-21T03:32:02.637 INFO:teuthology.orchestra.run.smithi039.stdout:(1/157): ceph-19.3.0-5120.g18432218.el9.x86_64. 37 kB/s | 6.6 kB 00:00 2024-09-21T03:32:02.940 INFO:teuthology.orchestra.run.smithi150.stdout:(2/157): ceph-fuse-19.3.0-5120.g18432218.el9.x8 2.0 MB/s | 873 kB 00:00 2024-09-21T03:32:03.082 INFO:teuthology.orchestra.run.smithi150.stdout:(3/157): ceph-immutable-object-cache-19.3.0-512 1.0 MB/s | 148 kB 00:00 2024-09-21T03:32:03.175 INFO:teuthology.orchestra.run.smithi161.stdout:(4/157): ceph-mds-19.3.0-5120.g18432218.el9.x86 2.1 MB/s | 2.3 MB 00:01 2024-09-21T03:32:03.187 INFO:teuthology.orchestra.run.smithi039.stdout:(2/157): ceph-fuse-19.3.0-5120.g18432218.el9.x8 1.6 MB/s | 873 kB 00:00 2024-09-21T03:32:03.329 INFO:teuthology.orchestra.run.smithi039.stdout:(3/157): ceph-immutable-object-cache-19.3.0-512 1.0 MB/s | 148 kB 00:00 2024-09-21T03:32:03.700 INFO:teuthology.orchestra.run.smithi161.stdout:(5/157): ceph-base-19.3.0-5120.g18432218.el9.x8 2.5 MB/s | 5.5 MB 00:02 2024-09-21T03:32:03.916 INFO:teuthology.orchestra.run.smithi150.stdout:(4/157): ceph-mds-19.3.0-5120.g18432218.el9.x86 2.7 MB/s | 2.3 MB 00:00 2024-09-21T03:32:04.033 INFO:teuthology.orchestra.run.smithi161.stdout:(6/157): ceph-mgr-19.3.0-5120.g18432218.el9.x86 1.8 MB/s | 1.5 MB 00:00 2024-09-21T03:32:04.283 INFO:teuthology.orchestra.run.smithi150.stdout:(5/157): ceph-mgr-19.3.0-5120.g18432218.el9.x86 4.1 MB/s | 1.5 MB 00:00 2024-09-21T03:32:04.908 INFO:teuthology.orchestra.run.smithi150.stdout:(6/157): ceph-base-19.3.0-5120.g18432218.el9.x8 2.1 MB/s | 5.5 MB 00:02 2024-09-21T03:32:05.042 INFO:teuthology.orchestra.run.smithi161.stdout:(7/157): ceph-common-19.3.0-5120.g18432218.el9. 5.6 MB/s | 20 MB 00:03 2024-09-21T03:32:05.135 INFO:teuthology.orchestra.run.smithi039.stdout:(4/157): ceph-base-19.3.0-5120.g18432218.el9.x8 2.1 MB/s | 5.5 MB 00:02 2024-09-21T03:32:05.135 INFO:teuthology.orchestra.run.smithi150.stdout:(7/157): ceph-mon-19.3.0-5120.g18432218.el9.x86 6.0 MB/s | 5.0 MB 00:00 2024-09-21T03:32:05.571 INFO:teuthology.orchestra.run.smithi039.stdout:(5/157): ceph-mds-19.3.0-5120.g18432218.el9.x86 1.0 MB/s | 2.3 MB 00:02 2024-09-21T03:32:06.063 INFO:teuthology.orchestra.run.smithi039.stdout:(6/157): ceph-mgr-19.3.0-5120.g18432218.el9.x86 1.6 MB/s | 1.5 MB 00:00 2024-09-21T03:32:06.426 INFO:teuthology.orchestra.run.smithi161.stdout:(8/157): ceph-mon-19.3.0-5120.g18432218.el9.x86 1.8 MB/s | 5.0 MB 00:02 2024-09-21T03:32:06.484 INFO:teuthology.orchestra.run.smithi161.stdout:(9/157): ceph-selinux-19.3.0-5120.g18432218.el9 434 kB/s | 25 kB 00:00 2024-09-21T03:32:06.609 INFO:teuthology.orchestra.run.smithi161.stdout:(10/157): ceph-radosgw-19.3.0-5120.g18432218.el 6.7 MB/s | 10 MB 00:01 2024-09-21T03:32:06.617 INFO:teuthology.orchestra.run.smithi150.stdout:(8/157): ceph-radosgw-19.3.0-5120.g18432218.el9 6.9 MB/s | 10 MB 00:01 2024-09-21T03:32:06.668 INFO:teuthology.orchestra.run.smithi161.stdout:(11/157): cephfs-mirror-19.3.0-5120.g18432218.e 3.8 MB/s | 226 kB 00:00 2024-09-21T03:32:06.675 INFO:teuthology.orchestra.run.smithi150.stdout:(9/157): ceph-selinux-19.3.0-5120.g18432218.el9 434 kB/s | 25 kB 00:00 2024-09-21T03:32:06.735 INFO:teuthology.orchestra.run.smithi161.stdout:(12/157): cephfs-shell-19.3.0-5120.g18432218.el 404 kB/s | 27 kB 00:00 2024-09-21T03:32:06.802 INFO:teuthology.orchestra.run.smithi161.stdout:(13/157): libcephfs-devel-19.3.0-5120.g18432218 477 kB/s | 32 kB 00:00 2024-09-21T03:32:06.969 INFO:teuthology.orchestra.run.smithi161.stdout:(14/157): libcephfs2-19.3.0-5120.g18432218.el9. 4.5 MB/s | 766 kB 00:00 2024-09-21T03:32:07.119 INFO:teuthology.orchestra.run.smithi161.stdout:(15/157): libcephsqlite-19.3.0-5120.g18432218.e 1.0 MB/s | 158 kB 00:00 2024-09-21T03:32:07.194 INFO:teuthology.orchestra.run.smithi161.stdout:(16/157): librados-devel-19.3.0-5120.g18432218. 1.6 MB/s | 127 kB 00:00 2024-09-21T03:32:07.311 INFO:teuthology.orchestra.run.smithi161.stdout:(17/157): libradosstriper1-19.3.0-5120.g1843221 4.2 MB/s | 507 kB 00:00 2024-09-21T03:32:08.087 INFO:teuthology.orchestra.run.smithi161.stdout:(18/157): librgw2-19.3.0-5120.g18432218.el9.x86 6.8 MB/s | 5.3 MB 00:00 2024-09-21T03:32:08.145 INFO:teuthology.orchestra.run.smithi161.stdout:(19/157): python3-ceph-argparse-19.3.0-5120.g18 780 kB/s | 45 kB 00:00 2024-09-21T03:32:08.204 INFO:teuthology.orchestra.run.smithi161.stdout:(20/157): python3-ceph-common-19.3.0-5120.g1843 2.5 MB/s | 148 kB 00:00 2024-09-21T03:32:08.271 INFO:teuthology.orchestra.run.smithi161.stdout:(21/157): python3-cephfs-19.3.0-5120.g18432218. 2.4 MB/s | 161 kB 00:00 2024-09-21T03:32:08.354 INFO:teuthology.orchestra.run.smithi161.stdout:(22/157): python3-rados-19.3.0-5120.g18432218.e 3.8 MB/s | 321 kB 00:00 2024-09-21T03:32:08.438 INFO:teuthology.orchestra.run.smithi161.stdout:(23/157): python3-rbd-19.3.0-5120.g18432218.el9 3.6 MB/s | 307 kB 00:00 2024-09-21T03:32:08.446 INFO:teuthology.orchestra.run.smithi039.stdout:(7/157): ceph-mon-19.3.0-5120.g18432218.el9.x86 1.7 MB/s | 5.0 MB 00:02 2024-09-21T03:32:08.505 INFO:teuthology.orchestra.run.smithi161.stdout:(24/157): python3-rgw-19.3.0-5120.g18432218.el9 1.5 MB/s | 99 kB 00:00 2024-09-21T03:32:08.572 INFO:teuthology.orchestra.run.smithi161.stdout:(25/157): rbd-fuse-19.3.0-5120.g18432218.el9.x8 1.3 MB/s | 86 kB 00:00 2024-09-21T03:32:08.997 INFO:teuthology.orchestra.run.smithi161.stdout:(26/157): rbd-mirror-19.3.0-5120.g18432218.el9. 7.3 MB/s | 3.1 MB 00:00 2024-09-21T03:32:09.147 INFO:teuthology.orchestra.run.smithi161.stdout:(27/157): rbd-nbd-19.3.0-5120.g18432218.el9.x86 1.1 MB/s | 170 kB 00:00 2024-09-21T03:32:09.167 INFO:teuthology.orchestra.run.smithi150.stdout:(10/157): ceph-common-19.3.0-5120.g18432218.el9 2.9 MB/s | 20 MB 00:06 2024-09-21T03:32:09.206 INFO:teuthology.orchestra.run.smithi161.stdout:(28/157): ceph-grafana-dashboards-19.3.0-5120.g 589 kB/s | 34 kB 00:00 2024-09-21T03:32:09.273 INFO:teuthology.orchestra.run.smithi161.stdout:(29/157): ceph-mgr-cephadm-19.3.0-5120.g1843221 2.3 MB/s | 154 kB 00:00 2024-09-21T03:32:09.284 INFO:teuthology.orchestra.run.smithi150.stdout:(11/157): cephfs-mirror-19.3.0-5120.g18432218.e 1.9 MB/s | 226 kB 00:00 2024-09-21T03:32:09.351 INFO:teuthology.orchestra.run.smithi150.stdout:(12/157): cephfs-shell-19.3.0-5120.g18432218.el 405 kB/s | 27 kB 00:00 2024-09-21T03:32:09.426 INFO:teuthology.orchestra.run.smithi150.stdout:(13/157): libcephfs-devel-19.3.0-5120.g18432218 424 kB/s | 32 kB 00:00 2024-09-21T03:32:09.677 INFO:teuthology.orchestra.run.smithi150.stdout:(14/157): libcephfs2-19.3.0-5120.g18432218.el9. 3.0 MB/s | 766 kB 00:00 2024-09-21T03:32:09.802 INFO:teuthology.orchestra.run.smithi150.stdout:(15/157): libcephsqlite-19.3.0-5120.g18432218.e 1.2 MB/s | 158 kB 00:00 2024-09-21T03:32:09.886 INFO:teuthology.orchestra.run.smithi150.stdout:(16/157): librados-devel-19.3.0-5120.g18432218. 1.5 MB/s | 127 kB 00:00 2024-09-21T03:32:10.144 INFO:teuthology.orchestra.run.smithi150.stdout:(17/157): ceph-osd-19.3.0-5120.g18432218.el9.x8 3.4 MB/s | 18 MB 00:05 2024-09-21T03:32:10.178 INFO:teuthology.orchestra.run.smithi150.stdout:(18/157): libradosstriper1-19.3.0-5120.g1843221 1.7 MB/s | 507 kB 00:00 2024-09-21T03:32:10.253 INFO:teuthology.orchestra.run.smithi150.stdout:(19/157): python3-ceph-argparse-19.3.0-5120.g18 599 kB/s | 45 kB 00:00 2024-09-21T03:32:10.329 INFO:teuthology.orchestra.run.smithi150.stdout:(20/157): python3-ceph-common-19.3.0-5120.g1843 1.9 MB/s | 148 kB 00:00 2024-09-21T03:32:10.348 INFO:teuthology.orchestra.run.smithi161.stdout:(30/157): ceph-osd-19.3.0-5120.g18432218.el9.x8 2.8 MB/s | 18 MB 00:06 2024-09-21T03:32:10.404 INFO:teuthology.orchestra.run.smithi150.stdout:(21/157): python3-cephfs-19.3.0-5120.g18432218. 2.1 MB/s | 161 kB 00:00 2024-09-21T03:32:10.537 INFO:teuthology.orchestra.run.smithi150.stdout:(22/157): python3-rados-19.3.0-5120.g18432218.e 2.4 MB/s | 321 kB 00:00 2024-09-21T03:32:10.671 INFO:teuthology.orchestra.run.smithi150.stdout:(23/157): python3-rbd-19.3.0-5120.g18432218.el9 2.2 MB/s | 307 kB 00:00 2024-09-21T03:32:10.763 INFO:teuthology.orchestra.run.smithi150.stdout:(24/157): python3-rgw-19.3.0-5120.g18432218.el9 1.1 MB/s | 99 kB 00:00 2024-09-21T03:32:10.805 INFO:teuthology.orchestra.run.smithi039.stdout:(8/157): ceph-common-19.3.0-5120.g18432218.el9. 2.4 MB/s | 20 MB 00:08 2024-09-21T03:32:10.838 INFO:teuthology.orchestra.run.smithi150.stdout:(25/157): rbd-fuse-19.3.0-5120.g18432218.el9.x8 1.1 MB/s | 86 kB 00:00 2024-09-21T03:32:10.880 INFO:teuthology.orchestra.run.smithi039.stdout:(9/157): ceph-selinux-19.3.0-5120.g18432218.el9 337 kB/s | 25 kB 00:00 2024-09-21T03:32:11.299 INFO:teuthology.orchestra.run.smithi161.stdout:(31/157): ceph-mgr-dashboard-19.3.0-5120.g18432 7.0 MB/s | 14 MB 00:02 2024-09-21T03:32:11.382 INFO:teuthology.orchestra.run.smithi161.stdout:(32/157): ceph-mgr-modules-core-19.3.0-5120.g18 3.4 MB/s | 292 kB 00:00 2024-09-21T03:32:11.457 INFO:teuthology.orchestra.run.smithi161.stdout:(33/157): ceph-mgr-rook-19.3.0-5120.g18432218.e 659 kB/s | 49 kB 00:00 2024-09-21T03:32:11.533 INFO:teuthology.orchestra.run.smithi161.stdout:(34/157): ceph-prometheus-alerts-19.3.0-5120.g1 219 kB/s | 16 kB 00:00 2024-09-21T03:32:11.608 INFO:teuthology.orchestra.run.smithi161.stdout:(35/157): ceph-volume-19.3.0-5120.g18432218.el9 3.7 MB/s | 285 kB 00:00 2024-09-21T03:32:11.672 INFO:teuthology.orchestra.run.smithi150.stdout:(26/157): ceph-test-19.3.0-5120.g18432218.el9.x 8.0 MB/s | 40 MB 00:04 2024-09-21T03:32:11.725 INFO:teuthology.orchestra.run.smithi161.stdout:(36/157): cephadm-19.3.0-5120.g18432218.el9.noa 6.6 MB/s | 784 kB 00:00 2024-09-21T03:32:11.747 INFO:teuthology.orchestra.run.smithi150.stdout:(27/157): librgw2-19.3.0-5120.g18432218.el9.x86 3.3 MB/s | 5.3 MB 00:01 2024-09-21T03:32:11.772 INFO:teuthology.orchestra.run.smithi150.stdout:(28/157): rbd-nbd-19.3.0-5120.g18432218.el9.x86 1.7 MB/s | 170 kB 00:00 2024-09-21T03:32:11.783 INFO:teuthology.orchestra.run.smithi161.stdout:(37/157): cephfs-top-19.3.0-5120.g18432218.el9. 316 kB/s | 18 kB 00:00 2024-09-21T03:32:11.806 INFO:teuthology.orchestra.run.smithi150.stdout:(29/157): ceph-grafana-dashboards-19.3.0-5120.g 587 kB/s | 34 kB 00:00 2024-09-21T03:32:11.832 INFO:teuthology.orchestra.run.smithi150.stdout:(30/157): ceph-mgr-cephadm-19.3.0-5120.g1843221 2.6 MB/s | 154 kB 00:00 2024-09-21T03:32:11.884 INFO:teuthology.orchestra.run.smithi161.stdout:(38/157): fuse-2.9.9-16.el9.x86_64.rpm 798 kB/s | 80 kB 00:00 2024-09-21T03:32:11.890 INFO:teuthology.orchestra.run.smithi150.stdout:(31/157): rbd-mirror-19.3.0-5120.g18432218.el9. 3.0 MB/s | 3.1 MB 00:01 2024-09-21T03:32:11.984 INFO:teuthology.orchestra.run.smithi161.stdout:(39/157): libgfortran-11.5.0-2.el9.x86_64.rpm 7.8 MB/s | 801 kB 00:00 2024-09-21T03:32:12.005 INFO:teuthology.orchestra.run.smithi039.stdout:(10/157): ceph-osd-19.3.0-5120.g18432218.el9.x8 3.0 MB/s | 18 MB 00:05 2024-09-21T03:32:12.007 INFO:teuthology.orchestra.run.smithi150.stdout:(32/157): ceph-mgr-modules-core-19.3.0-5120.g18 2.4 MB/s | 292 kB 00:00 2024-09-21T03:32:12.026 INFO:teuthology.orchestra.run.smithi161.stdout:(40/157): libquadmath-11.5.0-2.el9.x86_64.rpm 4.5 MB/s | 191 kB 00:00 2024-09-21T03:32:12.076 INFO:teuthology.orchestra.run.smithi161.stdout:(41/157): python3-cffi-1.14.5-5.el9.x86_64.rpm 5.0 MB/s | 253 kB 00:00 2024-09-21T03:32:12.099 INFO:teuthology.orchestra.run.smithi150.stdout:(33/157): ceph-mgr-rook-19.3.0-5120.g18432218.e 538 kB/s | 49 kB 00:00 2024-09-21T03:32:12.122 INFO:teuthology.orchestra.run.smithi039.stdout:(11/157): cephfs-mirror-19.3.0-5120.g18432218.e 1.9 MB/s | 226 kB 00:00 2024-09-21T03:32:12.166 INFO:teuthology.orchestra.run.smithi150.stdout:(34/157): ceph-prometheus-alerts-19.3.0-5120.g1 247 kB/s | 16 kB 00:00 2024-09-21T03:32:12.172 INFO:teuthology.orchestra.run.smithi039.stdout:(12/157): cephfs-shell-19.3.0-5120.g18432218.el 542 kB/s | 27 kB 00:00 2024-09-21T03:32:12.177 INFO:teuthology.orchestra.run.smithi161.stdout:(42/157): python3-cryptography-36.0.1-4.el9.x86 12 MB/s | 1.2 MB 00:00 2024-09-21T03:32:12.219 INFO:teuthology.orchestra.run.smithi161.stdout:(43/157): python3-ply-3.11-14.el9.noarch.rpm 2.5 MB/s | 106 kB 00:00 2024-09-21T03:32:12.231 INFO:teuthology.orchestra.run.smithi039.stdout:(13/157): libcephfs-devel-19.3.0-5120.g18432218 543 kB/s | 32 kB 00:00 2024-09-21T03:32:12.277 INFO:teuthology.orchestra.run.smithi161.stdout:(44/157): python3-pycparser-2.20-6.el9.noarch.r 2.3 MB/s | 135 kB 00:00 2024-09-21T03:32:12.291 INFO:teuthology.orchestra.run.smithi150.stdout:(35/157): ceph-volume-19.3.0-5120.g18432218.el9 2.2 MB/s | 285 kB 00:00 2024-09-21T03:32:12.369 INFO:teuthology.orchestra.run.smithi161.stdout:(45/157): ceph-mgr-diskprediction-local-19.3.0- 3.6 MB/s | 7.4 MB 00:02 2024-09-21T03:32:12.395 INFO:teuthology.orchestra.run.smithi161.stdout:(46/157): python3-requests-2.25.1-8.el9.noarch. 1.0 MB/s | 125 kB 00:00 2024-09-21T03:32:12.473 INFO:teuthology.orchestra.run.smithi039.stdout:(14/157): ceph-radosgw-19.3.0-5120.g18432218.el 2.6 MB/s | 10 MB 00:04 2024-09-21T03:32:12.500 INFO:teuthology.orchestra.run.smithi150.stdout:(36/157): cephadm-19.3.0-5120.g18432218.el9.noa 3.7 MB/s | 784 kB 00:00 2024-09-21T03:32:12.503 INFO:teuthology.orchestra.run.smithi161.stdout:(47/157): python3-urllib3-1.26.5-6.el9.noarch.r 1.6 MB/s | 215 kB 00:00 2024-09-21T03:32:12.506 INFO:teuthology.orchestra.run.smithi039.stdout:(15/157): libcephfs2-19.3.0-5120.g18432218.el9. 2.7 MB/s | 766 kB 00:00 2024-09-21T03:32:12.532 INFO:teuthology.orchestra.run.smithi039.stdout:(16/157): libcephsqlite-19.3.0-5120.g18432218.e 2.7 MB/s | 158 kB 00:00 2024-09-21T03:32:12.550 INFO:teuthology.orchestra.run.smithi150.stdout:(37/157): cephfs-top-19.3.0-5120.g18432218.el9. 370 kB/s | 18 kB 00:00 2024-09-21T03:32:12.573 INFO:teuthology.orchestra.run.smithi039.stdout:(17/157): librados-devel-19.3.0-5120.g18432218. 1.9 MB/s | 127 kB 00:00 2024-09-21T03:32:12.692 INFO:teuthology.orchestra.run.smithi150.stdout:(38/157): ceph-mgr-diskprediction-local-19.3.0- 8.6 MB/s | 7.4 MB 00:00 2024-09-21T03:32:12.724 INFO:teuthology.orchestra.run.smithi039.stdout:(18/157): libradosstriper1-19.3.0-5120.g1843221 2.6 MB/s | 507 kB 00:00 2024-09-21T03:32:12.782 INFO:teuthology.orchestra.run.smithi039.stdout:(19/157): python3-ceph-argparse-19.3.0-5120.g18 774 kB/s | 45 kB 00:00 2024-09-21T03:32:12.841 INFO:teuthology.orchestra.run.smithi039.stdout:(20/157): python3-ceph-common-19.3.0-5120.g1843 2.5 MB/s | 148 kB 00:00 2024-09-21T03:32:12.908 INFO:teuthology.orchestra.run.smithi039.stdout:(21/157): python3-cephfs-19.3.0-5120.g18432218. 2.4 MB/s | 161 kB 00:00 2024-09-21T03:32:12.984 INFO:teuthology.orchestra.run.smithi150.stdout:(39/157): fuse-2.9.9-16.el9.x86_64.rpm 184 kB/s | 80 kB 00:00 2024-09-21T03:32:13.000 INFO:teuthology.orchestra.run.smithi039.stdout:(22/157): python3-rados-19.3.0-5120.g18432218.e 3.4 MB/s | 321 kB 00:00 2024-09-21T03:32:13.100 INFO:teuthology.orchestra.run.smithi039.stdout:(23/157): python3-rbd-19.3.0-5120.g18432218.el9 3.0 MB/s | 307 kB 00:00 2024-09-21T03:32:13.159 INFO:teuthology.orchestra.run.smithi039.stdout:(24/157): python3-rgw-19.3.0-5120.g18432218.el9 1.7 MB/s | 99 kB 00:00 2024-09-21T03:32:13.184 INFO:teuthology.orchestra.run.smithi150.stdout:(40/157): libquadmath-11.5.0-2.el9.x86_64.rpm 956 kB/s | 191 kB 00:00 2024-09-21T03:32:13.209 INFO:teuthology.orchestra.run.smithi039.stdout:(25/157): rbd-fuse-19.3.0-5120.g18432218.el9.x8 1.7 MB/s | 86 kB 00:00 2024-09-21T03:32:13.229 INFO:teuthology.orchestra.run.smithi161.stdout:(48/157): automake-1.16.2-8.el9.noarch.rpm 827 kB/s | 689 kB 00:00 2024-09-21T03:32:13.346 INFO:teuthology.orchestra.run.smithi161.stdout:(49/157): boost-program-options-1.75.0-8.el9.x8 917 kB/s | 107 kB 00:00 2024-09-21T03:32:13.385 INFO:teuthology.orchestra.run.smithi150.stdout:(41/157): python3-cffi-1.14.5-5.el9.x86_64.rpm 1.2 MB/s | 253 kB 00:00 2024-09-21T03:32:13.454 INFO:teuthology.orchestra.run.smithi161.stdout:(50/157): elfutils-libelf-devel-0.191-4.el9.x86 200 kB/s | 22 kB 00:00 2024-09-21T03:32:13.493 INFO:teuthology.orchestra.run.smithi039.stdout:(26/157): librgw2-19.3.0-5120.g18432218.el9.x86 5.7 MB/s | 5.3 MB 00:00 2024-09-21T03:32:13.496 INFO:teuthology.orchestra.run.smithi161.stdout:(51/157): bison-3.7.4-5.el9.x86_64.rpm 950 kB/s | 943 kB 00:00 2024-09-21T03:32:13.510 INFO:teuthology.orchestra.run.smithi150.stdout:(42/157): libgfortran-11.5.0-2.el9.x86_64.rpm 981 kB/s | 801 kB 00:00 2024-09-21T03:32:13.543 INFO:teuthology.orchestra.run.smithi039.stdout:(27/157): rbd-nbd-19.3.0-5120.g18432218.el9.x86 3.3 MB/s | 170 kB 00:00 2024-09-21T03:32:13.571 INFO:teuthology.orchestra.run.smithi161.stdout:(52/157): flex-2.6.4-9.el9.x86_64.rpm 2.6 MB/s | 313 kB 00:00 2024-09-21T03:32:13.594 INFO:teuthology.orchestra.run.smithi039.stdout:(28/157): ceph-grafana-dashboards-19.3.0-5120.g 687 kB/s | 34 kB 00:00 2024-09-21T03:32:13.597 INFO:teuthology.orchestra.run.smithi161.stdout:(53/157): flexiblas-3.0.4-8.el9.x86_64.rpm 326 kB/s | 33 kB 00:00 2024-09-21T03:32:13.627 INFO:teuthology.orchestra.run.smithi150.stdout:(43/157): python3-ply-3.11-14.el9.noarch.rpm 914 kB/s | 106 kB 00:00 2024-09-21T03:32:13.644 INFO:teuthology.orchestra.run.smithi039.stdout:(29/157): ceph-mgr-cephadm-19.3.0-5120.g1843221 3.0 MB/s | 154 kB 00:00 2024-09-21T03:32:13.705 INFO:teuthology.orchestra.run.smithi161.stdout:(54/157): flexiblas-openblas-openmp-3.0.4-8.el9 168 kB/s | 18 kB 00:00 2024-09-21T03:32:13.752 INFO:teuthology.orchestra.run.smithi150.stdout:(44/157): python3-pycparser-2.20-6.el9.noarch.r 1.1 MB/s | 135 kB 00:00 2024-09-21T03:32:13.786 INFO:teuthology.orchestra.run.smithi039.stdout:(30/157): rbd-mirror-19.3.0-5120.g18432218.el9. 5.4 MB/s | 3.1 MB 00:00 2024-09-21T03:32:13.794 INFO:teuthology.orchestra.run.smithi150.stdout:(45/157): python3-cryptography-36.0.1-4.el9.x86 3.0 MB/s | 1.2 MB 00:00 2024-09-21T03:32:13.814 INFO:teuthology.orchestra.run.smithi161.stdout:(55/157): libaio-devel-0.3.111-13.el9.x86_64.rp 103 kB/s | 11 kB 00:00 2024-09-21T03:32:13.930 INFO:teuthology.orchestra.run.smithi161.stdout:(56/157): libnbd-1.20.2-2.el9.x86_64.rpm 1.4 MB/s | 171 kB 00:00 2024-09-21T03:32:13.936 INFO:teuthology.orchestra.run.smithi150.stdout:(46/157): ceph-mgr-dashboard-19.3.0-5120.g18432 6.7 MB/s | 14 MB 00:02 2024-09-21T03:32:13.961 INFO:teuthology.orchestra.run.smithi150.stdout:(47/157): python3-requests-2.25.1-8.el9.noarch. 598 kB/s | 125 kB 00:00 2024-09-21T03:32:13.987 INFO:teuthology.orchestra.run.smithi150.stdout:(48/157): python3-urllib3-1.26.5-6.el9.noarch.r 1.1 MB/s | 215 kB 00:00 2024-09-21T03:32:14.048 INFO:teuthology.orchestra.run.smithi161.stdout:(57/157): libpmemobj-1.12.1-1.el9.x86_64.rpm 1.3 MB/s | 160 kB 00:00 2024-09-21T03:32:14.162 INFO:teuthology.orchestra.run.smithi150.stdout:(49/157): boost-program-options-1.75.0-8.el9.x8 610 kB/s | 107 kB 00:00 2024-09-21T03:32:14.165 INFO:teuthology.orchestra.run.smithi161.stdout:(58/157): flexiblas-netlib-3.0.4-8.el9.x86_64.r 5.0 MB/s | 3.0 MB 00:00 2024-09-21T03:32:14.181 INFO:teuthology.orchestra.run.smithi161.stdout:(59/157): librabbitmq-0.11.0-7.el9.x86_64.rpm 339 kB/s | 45 kB 00:00 2024-09-21T03:32:14.221 INFO:teuthology.orchestra.run.smithi150.stdout:(50/157): automake-1.16.2-8.el9.noarch.rpm 2.4 MB/s | 689 kB 00:00 2024-09-21T03:32:14.246 INFO:teuthology.orchestra.run.smithi150.stdout:(51/157): elfutils-libelf-devel-0.191-4.el9.x86 260 kB/s | 22 kB 00:00 2024-09-21T03:32:14.282 INFO:teuthology.orchestra.run.smithi161.stdout:(60/157): librdkafka-1.6.1-102.el9.x86_64.rpm 5.5 MB/s | 662 kB 00:00 2024-09-21T03:32:14.288 INFO:teuthology.orchestra.run.smithi150.stdout:(52/157): bison-3.7.4-5.el9.x86_64.rpm 2.8 MB/s | 943 kB 00:00 2024-09-21T03:32:14.313 INFO:teuthology.orchestra.run.smithi150.stdout:(53/157): flexiblas-3.0.4-8.el9.x86_64.rpm 489 kB/s | 33 kB 00:00 2024-09-21T03:32:14.338 INFO:teuthology.orchestra.run.smithi150.stdout:(54/157): flex-2.6.4-9.el9.x86_64.rpm 3.3 MB/s | 313 kB 00:00 2024-09-21T03:32:14.364 INFO:teuthology.orchestra.run.smithi150.stdout:(55/157): flexiblas-openblas-openmp-3.0.4-8.el9 364 kB/s | 18 kB 00:00 2024-09-21T03:32:14.382 INFO:teuthology.orchestra.run.smithi161.stdout:(61/157): libuuid-devel-2.37.4-20.el9.x86_64.rp 245 kB/s | 24 kB 00:00 2024-09-21T03:32:14.389 INFO:teuthology.orchestra.run.smithi150.stdout:(56/157): libaio-devel-0.3.111-13.el9.x86_64.rp 221 kB/s | 11 kB 00:00 2024-09-21T03:32:14.416 INFO:teuthology.orchestra.run.smithi161.stdout:(62/157): libtool-2.4.6-46.el9.x86_64.rpm 2.4 MB/s | 581 kB 00:00 2024-09-21T03:32:14.448 INFO:teuthology.orchestra.run.smithi150.stdout:(57/157): libpmemobj-1.12.1-1.el9.x86_64.rpm 2.7 MB/s | 160 kB 00:00 2024-09-21T03:32:14.473 INFO:teuthology.orchestra.run.smithi150.stdout:(58/157): libnbd-1.20.2-2.el9.x86_64.rpm 1.5 MB/s | 171 kB 00:00 2024-09-21T03:32:14.490 INFO:teuthology.orchestra.run.smithi150.stdout:(59/157): librabbitmq-0.11.0-7.el9.x86_64.rpm 1.1 MB/s | 45 kB 00:00 2024-09-21T03:32:14.491 INFO:teuthology.orchestra.run.smithi161.stdout:(63/157): libzstd-devel-1.5.1-2.el9.x86_64.rpm 438 kB/s | 47 kB 00:00 2024-09-21T03:32:14.533 INFO:teuthology.orchestra.run.smithi161.stdout:(64/157): lttng-ust-2.12.0-6.el9.x86_64.rpm 2.4 MB/s | 292 kB 00:00 2024-09-21T03:32:14.556 INFO:teuthology.orchestra.run.smithi150.stdout:(60/157): libtool-2.4.6-46.el9.x86_64.rpm 8.5 MB/s | 581 kB 00:00 2024-09-21T03:32:14.600 INFO:teuthology.orchestra.run.smithi161.stdout:(65/157): lua-5.4.4-4.el9.x86_64.rpm 1.7 MB/s | 188 kB 00:00 2024-09-21T03:32:14.607 INFO:teuthology.orchestra.run.smithi150.stdout:(61/157): libuuid-devel-2.37.4-20.el9.x86_64.rp 489 kB/s | 24 kB 00:00 2024-09-21T03:32:14.640 INFO:teuthology.orchestra.run.smithi150.stdout:(62/157): librdkafka-1.6.1-102.el9.x86_64.rpm 3.9 MB/s | 662 kB 00:00 2024-09-21T03:32:14.642 INFO:teuthology.orchestra.run.smithi161.stdout:(66/157): openblas-0.3.26-2.el9.x86_64.rpm 354 kB/s | 38 kB 00:00 2024-09-21T03:32:14.657 INFO:teuthology.orchestra.run.smithi150.stdout:(63/157): libzstd-devel-1.5.1-2.el9.x86_64.rpm 948 kB/s | 47 kB 00:00 2024-09-21T03:32:14.699 INFO:teuthology.orchestra.run.smithi150.stdout:(64/157): lttng-ust-2.12.0-6.el9.x86_64.rpm 4.9 MB/s | 292 kB 00:00 2024-09-21T03:32:14.750 INFO:teuthology.orchestra.run.smithi150.stdout:(65/157): openblas-0.3.26-2.el9.x86_64.rpm 769 kB/s | 38 kB 00:00 2024-09-21T03:32:14.775 INFO:teuthology.orchestra.run.smithi150.stdout:(66/157): lua-5.4.4-4.el9.x86_64.rpm 1.6 MB/s | 188 kB 00:00 2024-09-21T03:32:14.844 INFO:teuthology.orchestra.run.smithi039.stdout:(31/157): ceph-mgr-diskprediction-local-19.3.0- 7.0 MB/s | 7.4 MB 00:01 2024-09-21T03:32:14.920 INFO:teuthology.orchestra.run.smithi039.stdout:(32/157): ceph-mgr-modules-core-19.3.0-5120.g18 3.8 MB/s | 292 kB 00:00 2024-09-21T03:32:14.970 INFO:teuthology.orchestra.run.smithi039.stdout:(33/157): ceph-mgr-rook-19.3.0-5120.g18432218.e 990 kB/s | 49 kB 00:00 2024-09-21T03:32:15.012 INFO:teuthology.orchestra.run.smithi039.stdout:(34/157): ceph-prometheus-alerts-19.3.0-5120.g1 394 kB/s | 16 kB 00:00 2024-09-21T03:32:15.087 INFO:teuthology.orchestra.run.smithi039.stdout:(35/157): ceph-volume-19.3.0-5120.g18432218.el9 3.7 MB/s | 285 kB 00:00 2024-09-21T03:32:15.126 INFO:teuthology.orchestra.run.smithi161.stdout:(67/157): openblas-openmp-0.3.26-2.el9.x86_64.r 9.4 MB/s | 4.9 MB 00:00 2024-09-21T03:32:15.221 INFO:teuthology.orchestra.run.smithi039.stdout:(36/157): cephadm-19.3.0-5120.g18432218.el9.noa 5.7 MB/s | 784 kB 00:00 2024-09-21T03:32:15.263 INFO:teuthology.orchestra.run.smithi039.stdout:(37/157): cephfs-top-19.3.0-5120.g18432218.el9. 442 kB/s | 18 kB 00:00 2024-09-21T03:32:15.413 INFO:teuthology.orchestra.run.smithi039.stdout:(38/157): ceph-mgr-dashboard-19.3.0-5120.g18432 8.0 MB/s | 14 MB 00:01 2024-09-21T03:32:15.467 INFO:teuthology.orchestra.run.smithi150.stdout:(67/157): flexiblas-netlib-3.0.4-8.el9.x86_64.r 2.5 MB/s | 3.0 MB 00:01 2024-09-21T03:32:15.530 INFO:teuthology.orchestra.run.smithi039.stdout:(39/157): fuse-2.9.9-16.el9.x86_64.rpm 299 kB/s | 80 kB 00:00 2024-09-21T03:32:15.572 INFO:teuthology.orchestra.run.smithi039.stdout:(40/157): libgfortran-11.5.0-2.el9.x86_64.rpm 4.9 MB/s | 801 kB 00:00 2024-09-21T03:32:15.614 INFO:teuthology.orchestra.run.smithi039.stdout:(41/157): libquadmath-11.5.0-2.el9.x86_64.rpm 2.2 MB/s | 191 kB 00:00 2024-09-21T03:32:15.656 INFO:teuthology.orchestra.run.smithi039.stdout:(42/157): python3-cffi-1.14.5-5.el9.x86_64.rpm 3.0 MB/s | 253 kB 00:00 2024-09-21T03:32:15.715 INFO:teuthology.orchestra.run.smithi039.stdout:(43/157): python3-cryptography-36.0.1-4.el9.x86 12 MB/s | 1.2 MB 00:00 2024-09-21T03:32:15.740 INFO:teuthology.orchestra.run.smithi039.stdout:(44/157): python3-ply-3.11-14.el9.noarch.rpm 1.2 MB/s | 106 kB 00:00 2024-09-21T03:32:15.790 INFO:teuthology.orchestra.run.smithi039.stdout:(45/157): python3-pycparser-2.20-6.el9.noarch.r 1.8 MB/s | 135 kB 00:00 2024-09-21T03:32:15.824 INFO:teuthology.orchestra.run.smithi039.stdout:(46/157): python3-requests-2.25.1-8.el9.noarch. 1.5 MB/s | 125 kB 00:00 2024-09-21T03:32:15.834 INFO:teuthology.orchestra.run.smithi161.stdout:(68/157): ceph-test-19.3.0-5120.g18432218.el9.x 4.3 MB/s | 40 MB 00:09 2024-09-21T03:32:15.866 INFO:teuthology.orchestra.run.smithi039.stdout:(47/157): python3-urllib3-1.26.5-6.el9.noarch.r 2.8 MB/s | 215 kB 00:00 2024-09-21T03:32:16.016 INFO:teuthology.orchestra.run.smithi039.stdout:(48/157): automake-1.16.2-8.el9.noarch.rpm 3.5 MB/s | 689 kB 00:00 2024-09-21T03:32:16.074 INFO:teuthology.orchestra.run.smithi039.stdout:(49/157): bison-3.7.4-5.el9.x86_64.rpm 4.4 MB/s | 943 kB 00:00 2024-09-21T03:32:16.100 INFO:teuthology.orchestra.run.smithi039.stdout:(50/157): boost-program-options-1.75.0-8.el9.x8 1.2 MB/s | 107 kB 00:00 2024-09-21T03:32:16.125 INFO:teuthology.orchestra.run.smithi039.stdout:(51/157): elfutils-libelf-devel-0.191-4.el9.x86 432 kB/s | 22 kB 00:00 2024-09-21T03:32:16.151 INFO:teuthology.orchestra.run.smithi039.stdout:(52/157): flex-2.6.4-9.el9.x86_64.rpm 6.1 MB/s | 313 kB 00:00 2024-09-21T03:32:16.176 INFO:teuthology.orchestra.run.smithi039.stdout:(53/157): flexiblas-3.0.4-8.el9.x86_64.rpm 649 kB/s | 33 kB 00:00 2024-09-21T03:32:16.218 INFO:teuthology.orchestra.run.smithi039.stdout:(54/157): flexiblas-openblas-openmp-3.0.4-8.el9 440 kB/s | 18 kB 00:00 2024-09-21T03:32:16.259 INFO:teuthology.orchestra.run.smithi150.stdout:(68/157): postgresql-13.16-1.el9.x86_64.rpm 2.0 MB/s | 1.6 MB 00:00 2024-09-21T03:32:16.285 INFO:teuthology.orchestra.run.smithi039.stdout:(55/157): flexiblas-netlib-3.0.4-8.el9.x86_64.r 22 MB/s | 3.0 MB 00:00 2024-09-21T03:32:16.310 INFO:teuthology.orchestra.run.smithi039.stdout:(56/157): libaio-devel-0.3.111-13.el9.x86_64.rp 121 kB/s | 11 kB 00:00 2024-09-21T03:32:16.335 INFO:teuthology.orchestra.run.smithi039.stdout:(57/157): libnbd-1.20.2-2.el9.x86_64.rpm 3.3 MB/s | 171 kB 00:00 2024-09-21T03:32:16.361 INFO:teuthology.orchestra.run.smithi039.stdout:(58/157): libpmemobj-1.12.1-1.el9.x86_64.rpm 3.1 MB/s | 160 kB 00:00 2024-09-21T03:32:16.386 INFO:teuthology.orchestra.run.smithi039.stdout:(59/157): librabbitmq-0.11.0-7.el9.x86_64.rpm 899 kB/s | 45 kB 00:00 2024-09-21T03:32:16.420 INFO:teuthology.orchestra.run.smithi039.stdout:(60/157): librdkafka-1.6.1-102.el9.x86_64.rpm 11 MB/s | 662 kB 00:00 2024-09-21T03:32:16.462 INFO:teuthology.orchestra.run.smithi039.stdout:(61/157): libtool-2.4.6-46.el9.x86_64.rpm 7.5 MB/s | 581 kB 00:00 2024-09-21T03:32:16.487 INFO:teuthology.orchestra.run.smithi039.stdout:(62/157): libuuid-devel-2.37.4-20.el9.x86_64.rp 366 kB/s | 24 kB 00:00 2024-09-21T03:32:16.512 INFO:teuthology.orchestra.run.smithi039.stdout:(63/157): libzstd-devel-1.5.1-2.el9.x86_64.rpm 948 kB/s | 47 kB 00:00 2024-09-21T03:32:16.546 INFO:teuthology.orchestra.run.smithi039.stdout:(64/157): lttng-ust-2.12.0-6.el9.x86_64.rpm 4.9 MB/s | 292 kB 00:00 2024-09-21T03:32:16.550 INFO:teuthology.orchestra.run.smithi150.stdout:(69/157): openssl-devel-3.2.2-6.el9.x86_64.rpm 2.5 MB/s | 4.4 MB 00:01 2024-09-21T03:32:16.571 INFO:teuthology.orchestra.run.smithi039.stdout:(65/157): lua-5.4.4-4.el9.x86_64.rpm 3.1 MB/s | 188 kB 00:00 2024-09-21T03:32:16.676 INFO:teuthology.orchestra.run.smithi150.stdout:(70/157): postgresql-private-libs-13.16-1.el9.x 1.1 MB/s | 137 kB 00:00 2024-09-21T03:32:16.693 INFO:teuthology.orchestra.run.smithi161.stdout:(69/157): openssl-devel-3.2.2-6.el9.x86_64.rpm 2.2 MB/s | 4.4 MB 00:02 2024-09-21T03:32:16.718 INFO:teuthology.orchestra.run.smithi150.stdout:(71/157): postgresql-contrib-13.16-1.el9.x86_64 1.9 MB/s | 910 kB 00:00 2024-09-21T03:32:16.743 INFO:teuthology.orchestra.run.smithi161.stdout:(70/157): postgresql-contrib-13.16-1.el9.x86_64 1.0 MB/s | 910 kB 00:00 2024-09-21T03:32:16.876 INFO:teuthology.orchestra.run.smithi150.stdout:(72/157): openblas-openmp-0.3.26-2.el9.x86_64.r 2.3 MB/s | 4.9 MB 00:02 2024-09-21T03:32:16.955 INFO:teuthology.orchestra.run.smithi039.stdout:(66/157): ceph-test-19.3.0-5120.g18432218.el9.x 6.6 MB/s | 40 MB 00:06 2024-09-21T03:32:16.968 INFO:teuthology.orchestra.run.smithi150.stdout:(73/157): python3-attrs-20.3.0-7.el9.noarch.rpm 953 kB/s | 87 kB 00:00 2024-09-21T03:32:16.976 INFO:teuthology.orchestra.run.smithi161.stdout:(71/157): postgresql-private-libs-13.16-1.el9.x 485 kB/s | 137 kB 00:00 2024-09-21T03:32:16.980 INFO:teuthology.orchestra.run.smithi039.stdout:(67/157): openblas-0.3.26-2.el9.x86_64.rpm 89 kB/s | 38 kB 00:00 2024-09-21T03:32:17.072 INFO:teuthology.orchestra.run.smithi039.stdout:(68/157): openblas-openmp-0.3.26-2.el9.x86_64.r 9.9 MB/s | 4.9 MB 00:00 2024-09-21T03:32:17.147 INFO:teuthology.orchestra.run.smithi039.stdout:(69/157): openssl-devel-3.2.2-6.el9.x86_64.rpm 23 MB/s | 4.4 MB 00:00 2024-09-21T03:32:17.189 INFO:teuthology.orchestra.run.smithi039.stdout:(70/157): postgresql-contrib-13.16-1.el9.x86_64 7.6 MB/s | 910 kB 00:00 2024-09-21T03:32:17.214 INFO:teuthology.orchestra.run.smithi039.stdout:(71/157): postgresql-private-libs-13.16-1.el9.x 2.0 MB/s | 137 kB 00:00 2024-09-21T03:32:17.244 INFO:teuthology.orchestra.run.smithi150.stdout:(74/157): protobuf-3.14.0-13.el9.x86_64.rpm 1.9 MB/s | 1.0 MB 00:00 2024-09-21T03:32:17.265 INFO:teuthology.orchestra.run.smithi039.stdout:(72/157): protobuf-3.14.0-13.el9.x86_64.rpm 20 MB/s | 1.0 MB 00:00 2024-09-21T03:32:17.294 INFO:teuthology.orchestra.run.smithi150.stdout:(75/157): python3-colorama-0.4.6-3.el9.noarch.r 1.2 MB/s | 61 kB 00:00 2024-09-21T03:32:17.307 INFO:teuthology.orchestra.run.smithi039.stdout:(73/157): postgresql-13.16-1.el9.x86_64.rpm 4.9 MB/s | 1.6 MB 00:00 2024-09-21T03:32:17.340 INFO:teuthology.orchestra.run.smithi039.stdout:(74/157): python3-attrs-20.3.0-7.el9.noarch.rpm 1.1 MB/s | 87 kB 00:00 2024-09-21T03:32:17.424 INFO:teuthology.orchestra.run.smithi039.stdout:(75/157): postgresql-server-13.16-1.el9.x86_64. 25 MB/s | 5.8 MB 00:00 2024-09-21T03:32:17.436 INFO:teuthology.orchestra.run.smithi150.stdout:(76/157): python3-devel-3.9.19-8.el9.x86_64.rpm 1.7 MB/s | 245 kB 00:00 2024-09-21T03:32:17.441 INFO:teuthology.orchestra.run.smithi039.stdout:(76/157): python3-colorama-0.4.6-3.el9.noarch.r 610 kB/s | 61 kB 00:00 2024-09-21T03:32:17.475 INFO:teuthology.orchestra.run.smithi039.stdout:(77/157): python3-devel-3.9.19-8.el9.x86_64.rpm 4.8 MB/s | 245 kB 00:00 2024-09-21T03:32:17.500 INFO:teuthology.orchestra.run.smithi039.stdout:(78/157): python3-jinja2-2.11.3-6.el9.noarch.rp 4.1 MB/s | 248 kB 00:00 2024-09-21T03:32:17.525 INFO:teuthology.orchestra.run.smithi039.stdout:(79/157): python3-jsonpatch-1.21-16.el9.noarch. 511 kB/s | 26 kB 00:00 2024-09-21T03:32:17.535 INFO:teuthology.orchestra.run.smithi161.stdout:(72/157): postgresql-13.16-1.el9.x86_64.rpm 674 kB/s | 1.6 MB 00:02 2024-09-21T03:32:17.550 INFO:teuthology.orchestra.run.smithi039.stdout:(80/157): python3-jsonpointer-2.0-4.el9.noarch. 383 kB/s | 19 kB 00:00 2024-09-21T03:32:17.584 INFO:teuthology.orchestra.run.smithi039.stdout:(81/157): python3-mako-1.1.4-6.el9.noarch.rpm 2.9 MB/s | 172 kB 00:00 2024-09-21T03:32:17.586 INFO:teuthology.orchestra.run.smithi150.stdout:(77/157): python3-jinja2-2.11.3-6.el9.noarch.rp 1.6 MB/s | 248 kB 00:00 2024-09-21T03:32:17.609 INFO:teuthology.orchestra.run.smithi039.stdout:(82/157): python3-markupsafe-1.1.1-12.el9.x86_6 590 kB/s | 35 kB 00:00 2024-09-21T03:32:17.637 INFO:teuthology.orchestra.run.smithi150.stdout:(78/157): python3-jsonpatch-1.21-16.el9.noarch. 514 kB/s | 26 kB 00:00 2024-09-21T03:32:17.652 INFO:teuthology.orchestra.run.smithi039.stdout:(83/157): python3-numpy-f2py-1.20.1-5.el9.x86_6 5.9 MB/s | 250 kB 00:00 2024-09-21T03:32:17.687 INFO:teuthology.orchestra.run.smithi150.stdout:(79/157): python3-jsonpointer-2.0-4.el9.noarch. 384 kB/s | 19 kB 00:00 2024-09-21T03:32:17.735 INFO:teuthology.orchestra.run.smithi161.stdout:(73/157): python3-attrs-20.3.0-7.el9.noarch.rpm 437 kB/s | 87 kB 00:00 2024-09-21T03:32:17.744 INFO:teuthology.orchestra.run.smithi039.stdout:(84/157): python3-babel-2.9.1-2.el9.noarch.rpm 14 MB/s | 6.0 MB 00:00 2024-09-21T03:32:17.769 INFO:teuthology.orchestra.run.smithi039.stdout:(85/157): python3-oauthlib-3.1.1-5.el9.noarch.r 1.9 MB/s | 222 kB 00:00 2024-09-21T03:32:17.802 INFO:teuthology.orchestra.run.smithi039.stdout:(86/157): python3-packaging-20.9-5.el9.noarch.r 1.3 MB/s | 77 kB 00:00 2024-09-21T03:32:17.804 INFO:teuthology.orchestra.run.smithi150.stdout:(80/157): python3-mako-1.1.4-6.el9.noarch.rpm 1.4 MB/s | 172 kB 00:00 2024-09-21T03:32:17.828 INFO:teuthology.orchestra.run.smithi039.stdout:(87/157): python3-prettytable-0.7.2-27.el9.noar 721 kB/s | 42 kB 00:00 2024-09-21T03:32:17.853 INFO:teuthology.orchestra.run.smithi039.stdout:(88/157): python3-protobuf-3.14.0-13.el9.noarch 5.2 MB/s | 269 kB 00:00 2024-09-21T03:32:17.854 INFO:teuthology.orchestra.run.smithi150.stdout:(81/157): python3-markupsafe-1.1.1-12.el9.x86_6 697 kB/s | 35 kB 00:00 2024-09-21T03:32:17.920 INFO:teuthology.orchestra.run.smithi039.stdout:(89/157): python3-numpy-1.20.1-5.el9.x86_64.rpm 15 MB/s | 5.1 MB 00:00 2024-09-21T03:32:17.937 INFO:teuthology.orchestra.run.smithi150.stdout:(82/157): python3-babel-2.9.1-2.el9.noarch.rpm 6.2 MB/s | 6.0 MB 00:00 2024-09-21T03:32:17.945 INFO:teuthology.orchestra.run.smithi039.stdout:(90/157): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.3 MB/s | 159 kB 00:00 2024-09-21T03:32:17.979 INFO:teuthology.orchestra.run.smithi039.stdout:(91/157): python3-pyasn1-modules-0.4.8-6.el9.no 2.2 MB/s | 279 kB 00:00 2024-09-21T03:32:18.004 INFO:teuthology.orchestra.run.smithi039.stdout:(92/157): python3-pytz-2021.1-5.el9.noarch.rpm 872 kB/s | 51 kB 00:00 2024-09-21T03:32:18.005 INFO:teuthology.orchestra.run.smithi150.stdout:(83/157): python3-numpy-f2py-1.20.1-5.el9.x86_6 3.7 MB/s | 250 kB 00:00 2024-09-21T03:32:18.029 INFO:teuthology.orchestra.run.smithi039.stdout:(93/157): python3-requests-oauthlib-1.3.0-12.el 642 kB/s | 54 kB 00:00 2024-09-21T03:32:18.071 INFO:teuthology.orchestra.run.smithi039.stdout:(94/157): python3-toml-0.10.2-6.el9.noarch.rpm 1.0 MB/s | 42 kB 00:00 2024-09-21T03:32:18.071 INFO:teuthology.orchestra.run.smithi150.stdout:(84/157): python3-oauthlib-3.1.1-5.el9.noarch.r 3.3 MB/s | 222 kB 00:00 2024-09-21T03:32:18.096 INFO:teuthology.orchestra.run.smithi039.stdout:(95/157): python3-wcwidth-0.2.5-8.el9.noarch.rp 664 kB/s | 44 kB 00:00 2024-09-21T03:32:18.130 INFO:teuthology.orchestra.run.smithi150.stdout:(85/157): python3-packaging-20.9-5.el9.noarch.r 1.3 MB/s | 77 kB 00:00 2024-09-21T03:32:18.138 INFO:teuthology.orchestra.run.smithi039.stdout:(96/157): qatlib-service-24.02.0-1.el9.x86_64.r 865 kB/s | 36 kB 00:00 2024-09-21T03:32:18.163 INFO:teuthology.orchestra.run.smithi039.stdout:(97/157): qatlib-24.02.0-1.el9.x86_64.rpm 3.2 MB/s | 221 kB 00:00 2024-09-21T03:32:18.180 INFO:teuthology.orchestra.run.smithi150.stdout:(86/157): python3-prettytable-0.7.2-27.el9.noar 847 kB/s | 42 kB 00:00 2024-09-21T03:32:18.188 INFO:teuthology.orchestra.run.smithi039.stdout:(98/157): qatzip-libs-1.2.0-1.el9.x86_64.rpm 948 kB/s | 47 kB 00:00 2024-09-21T03:32:18.222 INFO:teuthology.orchestra.run.smithi039.stdout:(99/157): socat-1.7.4.1-6.el9.x86_64.rpm 5.1 MB/s | 304 kB 00:00 2024-09-21T03:32:18.239 INFO:teuthology.orchestra.run.smithi150.stdout:(87/157): python3-protobuf-3.14.0-13.el9.noarch 4.5 MB/s | 269 kB 00:00 2024-09-21T03:32:18.247 INFO:teuthology.orchestra.run.smithi039.stdout:(100/157): uuid-1.6.2-55.el9.x86_64.rpm 987 kB/s | 58 kB 00:00 2024-09-21T03:32:18.273 INFO:teuthology.orchestra.run.smithi039.stdout:(101/157): xfsprogs-devel-6.4.0-4.el9.x86_64.rp 1.9 MB/s | 96 kB 00:00 2024-09-21T03:32:18.298 INFO:teuthology.orchestra.run.smithi039.stdout:(102/157): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.2 MB/s | 64 kB 00:00 2024-09-21T03:32:18.306 INFO:teuthology.orchestra.run.smithi150.stdout:(88/157): python3-pyasn1-0.4.8-6.el9.noarch.rpm 2.3 MB/s | 159 kB 00:00 2024-09-21T03:32:18.348 INFO:teuthology.orchestra.run.smithi039.stdout:(103/157): zlib-devel-1.2.11-41.el9.x86_64.rpm 596 kB/s | 45 kB 00:00 2024-09-21T03:32:18.365 INFO:teuthology.orchestra.run.smithi150.stdout:(89/157): python3-pyasn1-modules-0.4.8-6.el9.no 4.7 MB/s | 279 kB 00:00 2024-09-21T03:32:18.415 INFO:teuthology.orchestra.run.smithi150.stdout:(90/157): python3-pytz-2021.1-5.el9.noarch.rpm 1.0 MB/s | 51 kB 00:00 2024-09-21T03:32:18.465 INFO:teuthology.orchestra.run.smithi150.stdout:(91/157): python3-requests-oauthlib-1.3.0-12.el 1.0 MB/s | 54 kB 00:00 2024-09-21T03:32:18.540 INFO:teuthology.orchestra.run.smithi039.stdout:(104/157): python3-scipy-1.6.2-8.el9.x86_64.rpm 29 MB/s | 16 MB 00:00 2024-09-21T03:32:18.560 INFO:teuthology.orchestra.run.smithi161.stdout:(74/157): protobuf-3.14.0-13.el9.x86_64.rpm 652 kB/s | 1.0 MB 00:01 2024-09-21T03:32:18.565 INFO:teuthology.orchestra.run.smithi039.stdout:(105/157): lua-devel-5.4.4-4.el9.x86_64.rpm 83 kB/s | 22 kB 00:00 2024-09-21T03:32:18.657 INFO:teuthology.orchestra.run.smithi039.stdout:(106/157): gperftools-libs-2.9.1-3.el9.x86_64.r 3.3 MB/s | 308 kB 00:00 2024-09-21T03:32:18.683 INFO:teuthology.orchestra.run.smithi039.stdout:(107/157): abseil-cpp-20211102.0-3.el9.x86_64.r 3.8 MB/s | 548 kB 00:00 2024-09-21T03:32:18.694 INFO:teuthology.orchestra.run.smithi161.stdout:(75/157): python3-colorama-0.4.6-3.el9.noarch.r 459 kB/s | 61 kB 00:00 2024-09-21T03:32:18.708 INFO:teuthology.orchestra.run.smithi039.stdout:(108/157): grpc-data-1.46.7-10.el9.noarch.rpm 387 kB/s | 19 kB 00:00 2024-09-21T03:32:18.733 INFO:teuthology.orchestra.run.smithi039.stdout:(109/157): libarrow-doc-9.0.0-11.el9.noarch.rpm 1.0 MB/s | 26 kB 00:00 2024-09-21T03:32:18.758 INFO:teuthology.orchestra.run.smithi039.stdout:(110/157): liboath-2.6.7-2.el9.x86_64.rpm 1.9 MB/s | 49 kB 00:00 2024-09-21T03:32:18.784 INFO:teuthology.orchestra.run.smithi039.stdout:(111/157): libunwind-1.6.2-1.el9.x86_64.rpm 2.6 MB/s | 67 kB 00:00 2024-09-21T03:32:18.818 INFO:teuthology.orchestra.run.smithi039.stdout:(112/157): protobuf-compiler-3.14.0-13.el9.x86_ 1.8 MB/s | 863 kB 00:00 2024-09-21T03:32:18.824 INFO:teuthology.orchestra.run.smithi150.stdout:(92/157): python3-scipy-1.6.2-8.el9.x86_64.rpm 45 MB/s | 16 MB 00:00 2024-09-21T03:32:18.843 INFO:teuthology.orchestra.run.smithi039.stdout:(113/157): luarocks-3.9.2-5.el9.noarch.rpm 2.5 MB/s | 151 kB 00:00 2024-09-21T03:32:18.874 INFO:teuthology.orchestra.run.smithi150.stdout:(93/157): python3-toml-0.10.2-6.el9.noarch.rpm 836 kB/s | 42 kB 00:00 2024-09-21T03:32:18.910 INFO:teuthology.orchestra.run.smithi039.stdout:(114/157): python3-asyncssh-2.13.2-5.el9.noarch 9.2 MB/s | 548 kB 00:00 2024-09-21T03:32:18.924 INFO:teuthology.orchestra.run.smithi150.stdout:(94/157): python3-wcwidth-0.2.5-8.el9.noarch.rp 885 kB/s | 44 kB 00:00 2024-09-21T03:32:18.944 INFO:teuthology.orchestra.run.smithi039.stdout:(115/157): python3-bcrypt-3.2.2-1.el9.x86_64.rp 1.3 MB/s | 43 kB 00:00 2024-09-21T03:32:18.960 INFO:teuthology.orchestra.run.smithi039.stdout:(116/157): python3-cachetools-4.2.4-1.el9.noarc 1.9 MB/s | 32 kB 00:00 2024-09-21T03:32:18.977 INFO:teuthology.orchestra.run.smithi039.stdout:(117/157): python3-certifi-2023.05.07-4.el9.noa 851 kB/s | 14 kB 00:00 2024-09-21T03:32:18.983 INFO:teuthology.orchestra.run.smithi150.stdout:(95/157): qatlib-24.02.0-1.el9.x86_64.rpm 3.7 MB/s | 221 kB 00:00 2024-09-21T03:32:19.020 INFO:teuthology.orchestra.run.smithi039.stdout:(118/157): parquet-libs-9.0.0-11.el9.x86_64.rpm 4.1 MB/s | 839 kB 00:00 2024-09-21T03:32:19.033 INFO:teuthology.orchestra.run.smithi150.stdout:(96/157): qatlib-service-24.02.0-1.el9.x86_64.r 719 kB/s | 36 kB 00:00 2024-09-21T03:32:19.044 INFO:teuthology.orchestra.run.smithi039.stdout:(119/157): python3-cheroot-8.6.0-4.el9.noarch.r 2.5 MB/s | 172 kB 00:00 2024-09-21T03:32:19.069 INFO:teuthology.orchestra.run.smithi161.stdout:(76/157): python3-devel-3.9.19-8.el9.x86_64.rpm 652 kB/s | 245 kB 00:00 2024-09-21T03:32:19.083 INFO:teuthology.orchestra.run.smithi150.stdout:(97/157): qatzip-libs-1.2.0-1.el9.x86_64.rpm 947 kB/s | 47 kB 00:00 2024-09-21T03:32:19.087 INFO:teuthology.orchestra.run.smithi039.stdout:(120/157): python3-cherrypy-18.6.1-2.el9.noarch 5.2 MB/s | 358 kB 00:00 2024-09-21T03:32:19.112 INFO:teuthology.orchestra.run.smithi039.stdout:(121/157): python3-cmd2-2.4.2-2.el9.noarch.rpm 4.4 MB/s | 303 kB 00:00 2024-09-21T03:32:19.142 INFO:teuthology.orchestra.run.smithi150.stdout:(98/157): socat-1.7.4.1-6.el9.x86_64.rpm 5.1 MB/s | 304 kB 00:00 2024-09-21T03:32:19.145 INFO:teuthology.orchestra.run.smithi039.stdout:(122/157): python3-google-auth-2.34.0-1.el9.noa 3.7 MB/s | 220 kB 00:00 2024-09-21T03:32:19.179 INFO:teuthology.orchestra.run.smithi039.stdout:(123/157): python3-grpcio-tools-1.46.7-10.el9.x 4.3 MB/s | 144 kB 00:00 2024-09-21T03:32:19.184 INFO:teuthology.orchestra.run.smithi150.stdout:(99/157): uuid-1.6.2-55.el9.x86_64.rpm 1.4 MB/s | 58 kB 00:00 2024-09-21T03:32:19.204 INFO:teuthology.orchestra.run.smithi039.stdout:(124/157): python3-jaraco-8.2.1-3.el9.noarch.rp 427 kB/s | 11 kB 00:00 2024-09-21T03:32:19.234 INFO:teuthology.orchestra.run.smithi150.stdout:(100/157): xfsprogs-devel-6.4.0-4.el9.x86_64.rp 1.9 MB/s | 96 kB 00:00 2024-09-21T03:32:19.276 INFO:teuthology.orchestra.run.smithi150.stdout:(101/157): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.5 MB/s | 64 kB 00:00 2024-09-21T03:32:19.288 INFO:teuthology.orchestra.run.smithi039.stdout:(125/157): libarrow-9.0.0-11.el9.x86_64.rpm 7.3 MB/s | 4.4 MB 00:00 2024-09-21T03:32:19.314 INFO:teuthology.orchestra.run.smithi039.stdout:(126/157): python3-jaraco-classes-3.2.1-5.el9.n 163 kB/s | 18 kB 00:00 2024-09-21T03:32:19.327 INFO:teuthology.orchestra.run.smithi150.stdout:(102/157): zlib-devel-1.2.11-41.el9.x86_64.rpm 893 kB/s | 45 kB 00:00 2024-09-21T03:32:19.338 INFO:teuthology.orchestra.run.smithi039.stdout:(127/157): python3-jaraco-collections-3.0.0-8.e 461 kB/s | 23 kB 00:00 2024-09-21T03:32:19.364 INFO:teuthology.orchestra.run.smithi039.stdout:(128/157): python3-jaraco-functools-3.5.0-2.el9 387 kB/s | 19 kB 00:00 2024-09-21T03:32:19.389 INFO:teuthology.orchestra.run.smithi039.stdout:(129/157): python3-jaraco-text-3.2.0-6.el9.noar 393 kB/s | 20 kB 00:00 2024-09-21T03:32:19.402 INFO:teuthology.orchestra.run.smithi150.stdout:(103/157): lua-devel-5.4.4-4.el9.x86_64.rpm 297 kB/s | 22 kB 00:00 2024-09-21T03:32:19.414 INFO:teuthology.orchestra.run.smithi039.stdout:(130/157): python3-logutils-0.3.5-21.el9.noarch 1.8 MB/s | 46 kB 00:00 2024-09-21T03:32:19.445 INFO:teuthology.orchestra.run.smithi161.stdout:(77/157): python3-jinja2-2.11.3-6.el9.noarch.rp 662 kB/s | 248 kB 00:00 2024-09-21T03:32:19.448 INFO:teuthology.orchestra.run.smithi039.stdout:(131/157): python3-more-itertools-8.12.0-2.el9. 2.3 MB/s | 79 kB 00:00 2024-09-21T03:32:19.507 INFO:teuthology.orchestra.run.smithi039.stdout:(132/157): python3-grpcio-1.46.7-10.el9.x86_64. 5.2 MB/s | 2.0 MB 00:00 2024-09-21T03:32:19.511 INFO:teuthology.orchestra.run.smithi150.stdout:(104/157): protobuf-compiler-3.14.0-13.el9.x86_ 7.8 MB/s | 863 kB 00:00 2024-09-21T03:32:19.532 INFO:teuthology.orchestra.run.smithi039.stdout:(133/157): python3-natsort-7.1.1-5.el9.noarch.r 689 kB/s | 58 kB 00:00 2024-09-21T03:32:19.553 INFO:teuthology.orchestra.run.smithi161.stdout:(78/157): python3-jsonpatch-1.21-16.el9.noarch. 237 kB/s | 26 kB 00:00 2024-09-21T03:32:19.557 INFO:teuthology.orchestra.run.smithi039.stdout:(134/157): python3-portend-3.1.0-2.el9.noarch.r 664 kB/s | 16 kB 00:00 2024-09-21T03:32:19.591 INFO:teuthology.orchestra.run.smithi039.stdout:(135/157): python3-pecan-1.4.2-3.el9.noarch.rpm 3.2 MB/s | 272 kB 00:00 2024-09-21T03:32:19.611 INFO:teuthology.orchestra.run.smithi150.stdout:(105/157): abseil-cpp-20211102.0-3.el9.x86_64.r 5.4 MB/s | 548 kB 00:00 2024-09-21T03:32:19.616 INFO:teuthology.orchestra.run.smithi039.stdout:(136/157): python3-pyperclip-1.8.2-2.el9.noarch 1.1 MB/s | 28 kB 00:00 2024-09-21T03:32:19.645 INFO:teuthology.orchestra.run.smithi150.stdout:(106/157): gperftools-libs-2.9.1-3.el9.x86_64.r 9.1 MB/s | 308 kB 00:00 2024-09-21T03:32:19.658 INFO:teuthology.orchestra.run.smithi039.stdout:(137/157): python3-kubernetes-26.1.0-2.el9.noar 3.5 MB/s | 1.0 MB 00:00 2024-09-21T03:32:19.662 INFO:teuthology.orchestra.run.smithi161.stdout:(79/157): python3-jsonpointer-2.0-4.el9.noarch. 177 kB/s | 19 kB 00:00 2024-09-21T03:32:19.670 INFO:teuthology.orchestra.run.smithi150.stdout:(107/157): grpc-data-1.46.7-10.el9.noarch.rpm 780 kB/s | 19 kB 00:00 2024-09-21T03:32:19.683 INFO:teuthology.orchestra.run.smithi039.stdout:(138/157): python3-pyOpenSSL-21.0.0-1.el9.noarc 715 kB/s | 90 kB 00:00 2024-09-21T03:32:19.709 INFO:teuthology.orchestra.run.smithi039.stdout:(139/157): python3-repoze-lru-0.7-16.el9.noarch 334 kB/s | 31 kB 00:00 2024-09-21T03:32:19.784 INFO:teuthology.orchestra.run.smithi039.stdout:(140/157): python3-rsa-4.9-2.el9.noarch.rpm 586 kB/s | 59 kB 00:00 2024-09-21T03:32:19.801 INFO:teuthology.orchestra.run.smithi039.stdout:(141/157): python3-tempora-5.0.0-2.el9.noarch.r 387 kB/s | 36 kB 00:00 2024-09-21T03:32:19.827 INFO:teuthology.orchestra.run.smithi039.stdout:(142/157): python3-routes-2.5.1-5.el9.noarch.rp 1.1 MB/s | 188 kB 00:00 2024-09-21T03:32:19.852 INFO:teuthology.orchestra.run.smithi039.stdout:(143/157): python3-typing-extensions-4.12.2-2.e 1.1 MB/s | 74 kB 00:00 2024-09-21T03:32:19.878 INFO:teuthology.orchestra.run.smithi039.stdout:(144/157): python3-websocket-client-1.2.3-2.el9 1.7 MB/s | 90 kB 00:00 2024-09-21T03:32:19.920 INFO:teuthology.orchestra.run.smithi039.stdout:(145/157): python3-webob-1.8.8-2.el9.noarch.rpm 1.9 MB/s | 230 kB 00:00 2024-09-21T03:32:19.945 INFO:teuthology.orchestra.run.smithi039.stdout:(146/157): python3-zc-lockfile-2.0-10.el9.noarc 299 kB/s | 20 kB 00:00 2024-09-21T03:32:19.945 INFO:teuthology.orchestra.run.smithi161.stdout:(80/157): python3-mako-1.1.4-6.el9.noarch.rpm 608 kB/s | 172 kB 00:00 2024-09-21T03:32:19.945 INFO:teuthology.orchestra.run.smithi150.stdout:(108/157): python3-numpy-1.20.1-5.el9.x86_64.rp 2.4 MB/s | 5.1 MB 00:02 2024-09-21T03:32:19.970 INFO:teuthology.orchestra.run.smithi039.stdout:(147/157): re2-20211101-20.el9.x86_64.rpm 3.7 MB/s | 191 kB 00:00 2024-09-21T03:32:20.004 INFO:teuthology.orchestra.run.smithi039.stdout:(148/157): python3-werkzeug-2.0.3-3.el9.1.noarc 2.8 MB/s | 427 kB 00:00 2024-09-21T03:32:20.037 INFO:teuthology.orchestra.run.smithi150.stdout:(109/157): postgresql-server-13.16-1.el9.x86_64 1.7 MB/s | 5.8 MB 00:03 2024-09-21T03:32:20.045 INFO:teuthology.orchestra.run.smithi161.stdout:(81/157): python3-markupsafe-1.1.1-12.el9.x86_6 348 kB/s | 35 kB 00:00 2024-09-21T03:32:20.062 INFO:teuthology.orchestra.run.smithi039.stdout:(149/157): thrift-0.15.0-2.el9.x86_64.rpm 14 MB/s | 1.6 MB 00:00 2024-09-21T03:32:20.062 INFO:teuthology.orchestra.run.smithi150.stdout:(110/157): libarrow-doc-9.0.0-11.el9.noarch.rpm 223 kB/s | 26 kB 00:00 2024-09-21T03:32:20.105 INFO:teuthology.orchestra.run.smithi150.stdout:(111/157): libunwind-1.6.2-1.el9.x86_64.rpm 1.9 MB/s | 67 kB 00:00 2024-09-21T03:32:20.154 INFO:teuthology.orchestra.run.smithi039.stdout:(150/157): NetworkManager-1.51.0-1.el9.x86_64.r 25 MB/s | 2.3 MB 00:00 2024-09-21T03:32:20.163 INFO:teuthology.orchestra.run.smithi150.stdout:(112/157): libarrow-9.0.0-11.el9.x86_64.rpm 9.0 MB/s | 4.4 MB 00:00 2024-09-21T03:32:20.188 INFO:teuthology.orchestra.run.smithi150.stdout:(113/157): liboath-2.6.7-2.el9.x86_64.rpm 323 kB/s | 49 kB 00:00 2024-09-21T03:32:20.214 INFO:teuthology.orchestra.run.smithi150.stdout:(114/157): luarocks-3.9.2-5.el9.noarch.rpm 1.4 MB/s | 151 kB 00:00 2024-09-21T03:32:20.239 INFO:teuthology.orchestra.run.smithi150.stdout:(115/157): python3-bcrypt-3.2.2-1.el9.x86_64.rp 1.7 MB/s | 43 kB 00:00 2024-09-21T03:32:20.246 INFO:teuthology.orchestra.run.smithi039.stdout:(151/157): NetworkManager-libnm-1.51.0-1.el9.x8 20 MB/s | 1.8 MB 00:00 2024-09-21T03:32:20.264 INFO:teuthology.orchestra.run.smithi150.stdout:(116/157): python3-cachetools-4.2.4-1.el9.noarc 1.3 MB/s | 32 kB 00:00 2024-09-21T03:32:20.289 INFO:teuthology.orchestra.run.smithi150.stdout:(117/157): python3-certifi-2023.05.07-4.el9.noa 567 kB/s | 14 kB 00:00 2024-09-21T03:32:20.313 INFO:teuthology.orchestra.run.smithi039.stdout:(152/157): NetworkManager-team-1.51.0-1.el9.x86 616 kB/s | 41 kB 00:00 2024-09-21T03:32:20.315 INFO:teuthology.orchestra.run.smithi150.stdout:(118/157): parquet-libs-9.0.0-11.el9.x86_64.rpm 5.4 MB/s | 839 kB 00:00 2024-09-21T03:32:20.351 INFO:teuthology.orchestra.run.smithi150.stdout:(119/157): python3-asyncssh-2.13.2-5.el9.noarch 3.4 MB/s | 548 kB 00:00 2024-09-21T03:32:20.473 INFO:teuthology.orchestra.run.smithi161.stdout:(82/157): postgresql-server-13.16-1.el9.x86_64. 1.6 MB/s | 5.8 MB 00:03 2024-09-21T03:32:20.473 INFO:teuthology.orchestra.run.smithi039.stdout:(153/157): NetworkManager-tui-1.51.0-1.el9.x86_ 2.7 MB/s | 249 kB 00:00 2024-09-21T03:32:20.473 INFO:teuthology.orchestra.run.smithi039.stdout:(154/157): librbd1-19.3.0-5120.g18432218.el9.x8 7.0 MB/s | 3.2 MB 00:00 2024-09-21T03:32:20.474 INFO:teuthology.orchestra.run.smithi150.stdout:(120/157): python3-cheroot-8.6.0-4.el9.noarch.r 2.0 MB/s | 172 kB 00:00 2024-09-21T03:32:20.474 INFO:teuthology.orchestra.run.smithi150.stdout:(121/157): python3-cherrypy-18.6.1-2.el9.noarch 3.8 MB/s | 358 kB 00:00 2024-09-21T03:32:20.474 INFO:teuthology.orchestra.run.smithi150.stdout:(122/157): python3-google-auth-2.34.0-1.el9.noa 3.2 MB/s | 220 kB 00:00 2024-09-21T03:32:20.474 INFO:teuthology.orchestra.run.smithi150.stdout:(123/157): python3-cmd2-2.4.2-2.el9.noarch.rpm 2.5 MB/s | 303 kB 00:00 2024-09-21T03:32:20.491 INFO:teuthology.orchestra.run.smithi150.stdout:(124/157): python3-grpcio-tools-1.46.7-10.el9.x 2.8 MB/s | 144 kB 00:00 2024-09-21T03:32:20.506 INFO:teuthology.orchestra.run.smithi039.stdout:(155/157): openssl-3.2.2-6.el9.x86_64.rpm 14 MB/s | 1.4 MB 00:00 2024-09-21T03:32:20.508 INFO:teuthology.orchestra.run.smithi150.stdout:(125/157): python3-jaraco-8.2.1-3.el9.noarch.rp 255 kB/s | 11 kB 00:00 2024-09-21T03:32:20.525 INFO:teuthology.orchestra.run.smithi150.stdout:(126/157): python3-jaraco-classes-3.2.1-5.el9.n 528 kB/s | 18 kB 00:00 2024-09-21T03:32:20.542 INFO:teuthology.orchestra.run.smithi150.stdout:(127/157): python3-jaraco-collections-3.0.0-8.e 693 kB/s | 23 kB 00:00 2024-09-21T03:32:20.559 INFO:teuthology.orchestra.run.smithi150.stdout:(128/157): python3-jaraco-functools-3.5.0-2.el9 579 kB/s | 19 kB 00:00 2024-09-21T03:32:20.564 INFO:teuthology.orchestra.run.smithi039.stdout:(156/157): librados2-19.3.0-5120.g18432218.el9. 5.7 MB/s | 3.4 MB 00:00 2024-09-21T03:32:20.576 INFO:teuthology.orchestra.run.smithi150.stdout:(129/157): python3-jaraco-text-3.2.0-6.el9.noar 586 kB/s | 20 kB 00:00 2024-09-21T03:32:20.593 INFO:teuthology.orchestra.run.smithi150.stdout:(130/157): python3-logutils-0.3.5-21.el9.noarch 2.7 MB/s | 46 kB 00:00 2024-09-21T03:32:20.623 INFO:teuthology.orchestra.run.smithi039.stdout:(157/157): openssl-libs-3.2.2-6.el9.x86_64.rpm 14 MB/s | 2.4 MB 00:00 2024-09-21T03:32:20.627 INFO:teuthology.orchestra.run.smithi150.stdout:(131/157): python3-more-itertools-8.12.0-2.el9. 2.3 MB/s | 79 kB 00:00 2024-09-21T03:32:20.629 INFO:teuthology.orchestra.run.smithi039.stdout:-------------------------------------------------------------------------------- 2024-09-21T03:32:20.629 INFO:teuthology.orchestra.run.smithi039.stdout:Total 12 MB/s | 228 MB 00:18 2024-09-21T03:32:20.660 INFO:teuthology.orchestra.run.smithi150.stdout:(132/157): python3-kubernetes-26.1.0-2.el9.noar 10 MB/s | 1.0 MB 00:00 2024-09-21T03:32:20.686 INFO:teuthology.orchestra.run.smithi150.stdout:(133/157): python3-natsort-7.1.1-5.el9.noarch.r 985 kB/s | 58 kB 00:00 2024-09-21T03:32:20.711 INFO:teuthology.orchestra.run.smithi150.stdout:(134/157): python3-pecan-1.4.2-3.el9.noarch.rpm 5.3 MB/s | 272 kB 00:00 2024-09-21T03:32:20.736 INFO:teuthology.orchestra.run.smithi150.stdout:(135/157): python3-portend-3.1.0-2.el9.noarch.r 326 kB/s | 16 kB 00:00 2024-09-21T03:32:20.746 INFO:teuthology.orchestra.run.smithi161.stdout:(83/157): python3-numpy-f2py-1.20.1-5.el9.x86_6 654 kB/s | 250 kB 00:00 2024-09-21T03:32:20.762 INFO:teuthology.orchestra.run.smithi150.stdout:(136/157): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.7 MB/s | 90 kB 00:00 2024-09-21T03:32:20.787 INFO:teuthology.orchestra.run.smithi150.stdout:(137/157): python3-pyperclip-1.8.2-2.el9.noarch 564 kB/s | 28 kB 00:00 2024-09-21T03:32:20.812 INFO:teuthology.orchestra.run.smithi150.stdout:(138/157): python3-repoze-lru-0.7-16.el9.noarch 614 kB/s | 31 kB 00:00 2024-09-21T03:32:20.863 INFO:teuthology.orchestra.run.smithi150.stdout:(139/157): python3-grpcio-1.46.7-10.el9.x86_64. 4.5 MB/s | 2.0 MB 00:00 2024-09-21T03:32:20.888 INFO:teuthology.orchestra.run.smithi150.stdout:(140/157): python3-rsa-4.9-2.el9.noarch.rpm 786 kB/s | 59 kB 00:00 2024-09-21T03:32:20.913 INFO:teuthology.orchestra.run.smithi150.stdout:(141/157): python3-tempora-5.0.0-2.el9.noarch.r 714 kB/s | 36 kB 00:00 2024-09-21T03:32:20.947 INFO:teuthology.orchestra.run.smithi150.stdout:(142/157): python3-routes-2.5.1-5.el9.noarch.rp 1.2 MB/s | 188 kB 00:00 2024-09-21T03:32:20.972 INFO:teuthology.orchestra.run.smithi150.stdout:(143/157): python3-typing-extensions-4.12.2-2.e 891 kB/s | 74 kB 00:00 2024-09-21T03:32:20.997 INFO:teuthology.orchestra.run.smithi150.stdout:(144/157): python3-websocket-client-1.2.3-2.el9 1.7 MB/s | 90 kB 00:00 2024-09-21T03:32:21.022 INFO:teuthology.orchestra.run.smithi150.stdout:(145/157): python3-webob-1.8.8-2.el9.noarch.rpm 2.1 MB/s | 230 kB 00:00 2024-09-21T03:32:21.048 INFO:teuthology.orchestra.run.smithi150.stdout:(146/157): python3-zc-lockfile-2.0-10.el9.noarc 398 kB/s | 20 kB 00:00 2024-09-21T03:32:21.073 INFO:teuthology.orchestra.run.smithi150.stdout:(147/157): re2-20211101-20.el9.x86_64.rpm 3.7 MB/s | 191 kB 00:00 2024-09-21T03:32:21.107 INFO:teuthology.orchestra.run.smithi150.stdout:(148/157): python3-werkzeug-2.0.3-3.el9.1.noarc 3.1 MB/s | 427 kB 00:00 2024-09-21T03:32:21.121 INFO:teuthology.orchestra.run.smithi161.stdout:(84/157): python3-oauthlib-3.1.1-5.el9.noarch.r 592 kB/s | 222 kB 00:00 2024-09-21T03:32:21.182 INFO:teuthology.orchestra.run.smithi150.stdout:(149/157): thrift-0.15.0-2.el9.x86_64.rpm 12 MB/s | 1.6 MB 00:00 2024-09-21T03:32:21.321 INFO:teuthology.orchestra.run.smithi161.stdout:(85/157): python3-packaging-20.9-5.el9.noarch.r 386 kB/s | 77 kB 00:00 2024-09-21T03:32:21.424 INFO:teuthology.orchestra.run.smithi150.stdout:(150/157): librados2-19.3.0-5120.g18432218.el9. 9.7 MB/s | 3.4 MB 00:00 2024-09-21T03:32:21.438 INFO:teuthology.orchestra.run.smithi161.stdout:(86/157): python3-prettytable-0.7.2-27.el9.noar 362 kB/s | 42 kB 00:00 2024-09-21T03:32:21.507 INFO:teuthology.orchestra.run.smithi150.stdout:(151/157): librbd1-19.3.0-5120.g18432218.el9.x8 7.9 MB/s | 3.2 MB 00:00 2024-09-21T03:32:21.883 INFO:teuthology.orchestra.run.smithi150.stdout:(152/157): NetworkManager-team-1.51.0-1.el9.x86 109 kB/s | 41 kB 00:00 2024-09-21T03:32:21.913 INFO:teuthology.orchestra.run.smithi161.stdout:(87/157): python3-protobuf-3.14.0-13.el9.noarch 567 kB/s | 269 kB 00:00 2024-09-21T03:32:21.918 INFO:teuthology.orchestra.run.smithi039.stdout:Running transaction check 2024-09-21T03:32:22.090 INFO:teuthology.orchestra.run.smithi039.stdout:Transaction check succeeded. 2024-09-21T03:32:22.090 INFO:teuthology.orchestra.run.smithi039.stdout:Running transaction test 2024-09-21T03:32:22.175 INFO:teuthology.orchestra.run.smithi150.stdout:(153/157): NetworkManager-tui-1.51.0-1.el9.x86_ 855 kB/s | 249 kB 00:00 2024-09-21T03:32:22.197 INFO:teuthology.orchestra.run.smithi161.stdout:(88/157): python3-pyasn1-0.4.8-6.el9.noarch.rpm 560 kB/s | 159 kB 00:00 2024-09-21T03:32:22.400 INFO:teuthology.orchestra.run.smithi150.stdout:(154/157): NetworkManager-1.51.0-1.el9.x86_64.r 1.9 MB/s | 2.3 MB 00:01 2024-09-21T03:32:22.517 INFO:teuthology.orchestra.run.smithi150.stdout:(155/157): NetworkManager-libnm-1.51.0-1.el9.x8 1.7 MB/s | 1.8 MB 00:01 2024-09-21T03:32:22.656 INFO:teuthology.orchestra.run.smithi161.stdout:(89/157): python3-pyasn1-modules-0.4.8-6.el9.no 608 kB/s | 279 kB 00:00 2024-09-21T03:32:22.750 INFO:teuthology.orchestra.run.smithi150.stdout:(156/157): openssl-libs-3.2.2-6.el9.x86_64.rpm 6.8 MB/s | 2.4 MB 00:00 2024-09-21T03:32:22.764 INFO:teuthology.orchestra.run.smithi161.stdout:(90/157): python3-pytz-2021.1-5.el9.noarch.rpm 472 kB/s | 51 kB 00:00 2024-09-21T03:32:22.867 INFO:teuthology.orchestra.run.smithi150.stdout:(157/157): openssl-3.2.2-6.el9.x86_64.rpm 2.0 MB/s | 1.4 MB 00:00 2024-09-21T03:32:22.873 INFO:teuthology.orchestra.run.smithi161.stdout:(91/157): python3-requests-oauthlib-1.3.0-12.el 495 kB/s | 54 kB 00:00 2024-09-21T03:32:22.874 INFO:teuthology.orchestra.run.smithi150.stdout:-------------------------------------------------------------------------------- 2024-09-21T03:32:22.874 INFO:teuthology.orchestra.run.smithi150.stdout:Total 11 MB/s | 228 MB 00:20 2024-09-21T03:32:24.130 INFO:teuthology.orchestra.run.smithi150.stdout:Running transaction check 2024-09-21T03:32:24.284 INFO:teuthology.orchestra.run.smithi150.stdout:Transaction check succeeded. 2024-09-21T03:32:24.284 INFO:teuthology.orchestra.run.smithi150.stdout:Running transaction test 2024-09-21T03:32:24.430 INFO:teuthology.orchestra.run.smithi039.stdout:Transaction test succeeded. 2024-09-21T03:32:24.431 INFO:teuthology.orchestra.run.smithi039.stdout:Running transaction 2024-09-21T03:32:26.101 INFO:teuthology.orchestra.run.smithi150.stdout:Transaction test succeeded. 2024-09-21T03:32:26.101 INFO:teuthology.orchestra.run.smithi150.stdout:Running transaction 2024-09-21T03:32:26.406 INFO:teuthology.orchestra.run.smithi161.stdout:(92/157): python3-babel-2.9.1-2.el9.noarch.rpm 704 kB/s | 6.0 MB 00:08 2024-09-21T03:32:26.506 INFO:teuthology.orchestra.run.smithi161.stdout:(93/157): python3-toml-0.10.2-6.el9.noarch.rpm 418 kB/s | 42 kB 00:00 2024-09-21T03:32:26.607 INFO:teuthology.orchestra.run.smithi161.stdout:(94/157): python3-wcwidth-0.2.5-8.el9.noarch.rp 441 kB/s | 44 kB 00:00 2024-09-21T03:32:26.982 INFO:teuthology.orchestra.run.smithi161.stdout:(95/157): qatlib-24.02.0-1.el9.x86_64.rpm 589 kB/s | 221 kB 00:00 2024-09-21T03:32:27.082 INFO:teuthology.orchestra.run.smithi161.stdout:(96/157): qatlib-service-24.02.0-1.el9.x86_64.r 358 kB/s | 36 kB 00:00 2024-09-21T03:32:27.183 INFO:teuthology.orchestra.run.smithi161.stdout:(97/157): qatzip-libs-1.2.0-1.el9.x86_64.rpm 473 kB/s | 47 kB 00:00 2024-09-21T03:32:27.741 INFO:teuthology.orchestra.run.smithi161.stdout:(98/157): python3-numpy-1.20.1-5.el9.x86_64.rpm 678 kB/s | 5.1 MB 00:07 2024-09-21T03:32:27.766 INFO:teuthology.orchestra.run.smithi161.stdout:(99/157): socat-1.7.4.1-6.el9.x86_64.rpm 521 kB/s | 304 kB 00:00 2024-09-21T03:32:27.850 INFO:teuthology.orchestra.run.smithi161.stdout:(100/157): uuid-1.6.2-55.el9.x86_64.rpm 533 kB/s | 58 kB 00:00 2024-09-21T03:32:27.959 INFO:teuthology.orchestra.run.smithi161.stdout:(101/157): xmlstarlet-1.6.1-20.el9.x86_64.rpm 588 kB/s | 64 kB 00:00 2024-09-21T03:32:27.984 INFO:teuthology.orchestra.run.smithi161.stdout:(102/157): xfsprogs-devel-6.4.0-4.el9.x86_64.rp 441 kB/s | 96 kB 00:00 2024-09-21T03:32:28.067 INFO:teuthology.orchestra.run.smithi161.stdout:(103/157): lua-devel-5.4.4-4.el9.x86_64.rpm 269 kB/s | 22 kB 00:00 2024-09-21T03:32:28.093 INFO:teuthology.orchestra.run.smithi161.stdout:(104/157): zlib-devel-1.2.11-41.el9.x86_64.rpm 335 kB/s | 45 kB 00:00 2024-09-21T03:32:28.151 INFO:teuthology.orchestra.run.smithi161.stdout:(105/157): abseil-cpp-20211102.0-3.el9.x86_64.r 9.2 MB/s | 548 kB 00:00 2024-09-21T03:32:28.193 INFO:teuthology.orchestra.run.smithi161.stdout:(106/157): gperftools-libs-2.9.1-3.el9.x86_64.r 7.2 MB/s | 308 kB 00:00 2024-09-21T03:32:28.218 INFO:teuthology.orchestra.run.smithi161.stdout:(107/157): grpc-data-1.46.7-10.el9.noarch.rpm 782 kB/s | 19 kB 00:00 2024-09-21T03:32:28.252 INFO:teuthology.orchestra.run.smithi161.stdout:(108/157): protobuf-compiler-3.14.0-13.el9.x86_ 4.6 MB/s | 863 kB 00:00 2024-09-21T03:32:28.302 INFO:teuthology.orchestra.run.smithi161.stdout:(109/157): libarrow-doc-9.0.0-11.el9.noarch.rpm 522 kB/s | 26 kB 00:00 2024-09-21T03:32:28.328 INFO:teuthology.orchestra.run.smithi161.stdout:(110/157): liboath-2.6.7-2.el9.x86_64.rpm 1.9 MB/s | 49 kB 00:00 2024-09-21T03:32:28.361 INFO:teuthology.orchestra.run.smithi161.stdout:(111/157): libunwind-1.6.2-1.el9.x86_64.rpm 2.0 MB/s | 67 kB 00:00 2024-09-21T03:32:28.395 INFO:teuthology.orchestra.run.smithi161.stdout:(112/157): luarocks-3.9.2-5.el9.noarch.rpm 4.5 MB/s | 151 kB 00:00 2024-09-21T03:32:28.470 INFO:teuthology.orchestra.run.smithi161.stdout:(113/157): parquet-libs-9.0.0-11.el9.x86_64.rpm 11 MB/s | 839 kB 00:00 2024-09-21T03:32:28.520 INFO:teuthology.orchestra.run.smithi161.stdout:(114/157): python3-asyncssh-2.13.2-5.el9.noarch 11 MB/s | 548 kB 00:00 2024-09-21T03:32:28.587 INFO:teuthology.orchestra.run.smithi161.stdout:(115/157): libarrow-9.0.0-11.el9.x86_64.rpm 12 MB/s | 4.4 MB 00:00 2024-09-21T03:32:28.613 INFO:teuthology.orchestra.run.smithi161.stdout:(116/157): python3-bcrypt-3.2.2-1.el9.x86_64.rp 472 kB/s | 43 kB 00:00 2024-09-21T03:32:28.638 INFO:teuthology.orchestra.run.smithi161.stdout:(117/157): python3-cachetools-4.2.4-1.el9.noarc 641 kB/s | 32 kB 00:00 2024-09-21T03:32:28.655 INFO:teuthology.orchestra.run.smithi161.stdout:(118/157): python3-certifi-2023.05.07-4.el9.noa 338 kB/s | 14 kB 00:00 2024-09-21T03:32:28.680 INFO:teuthology.orchestra.run.smithi161.stdout:(119/157): python3-cheroot-8.6.0-4.el9.noarch.r 4.0 MB/s | 172 kB 00:00 2024-09-21T03:32:28.705 INFO:teuthology.orchestra.run.smithi161.stdout:(120/157): python3-cherrypy-18.6.1-2.el9.noarch 7.0 MB/s | 358 kB 00:00 2024-09-21T03:32:28.731 INFO:teuthology.orchestra.run.smithi161.stdout:(121/157): python3-cmd2-2.4.2-2.el9.noarch.rpm 5.9 MB/s | 303 kB 00:00 2024-09-21T03:32:28.756 INFO:teuthology.orchestra.run.smithi161.stdout:(122/157): python3-google-auth-2.34.0-1.el9.noa 4.3 MB/s | 220 kB 00:00 2024-09-21T03:32:28.790 INFO:teuthology.orchestra.run.smithi161.stdout:(123/157): python3-grpcio-tools-1.46.7-10.el9.x 4.3 MB/s | 144 kB 00:00 2024-09-21T03:32:28.806 INFO:teuthology.orchestra.run.smithi161.stdout:(124/157): python3-jaraco-8.2.1-3.el9.noarch.rp 644 kB/s | 11 kB 00:00 2024-09-21T03:32:28.832 INFO:teuthology.orchestra.run.smithi161.stdout:(125/157): python3-jaraco-classes-3.2.1-5.el9.n 706 kB/s | 18 kB 00:00 2024-09-21T03:32:28.849 INFO:teuthology.orchestra.run.smithi161.stdout:(126/157): python3-jaraco-collections-3.0.0-8.e 1.4 MB/s | 23 kB 00:00 2024-09-21T03:32:28.866 INFO:teuthology.orchestra.run.smithi161.stdout:(127/157): python3-jaraco-functools-3.5.0-2.el9 1.1 MB/s | 19 kB 00:00 2024-09-21T03:32:28.883 INFO:teuthology.orchestra.run.smithi161.stdout:(128/157): python3-jaraco-text-3.2.0-6.el9.noar 1.2 MB/s | 20 kB 00:00 2024-09-21T03:32:28.953 INFO:teuthology.orchestra.run.smithi161.stdout:(129/157): python3-grpcio-1.46.7-10.el9.x86_64. 9.2 MB/s | 2.0 MB 00:00 2024-09-21T03:32:28.975 INFO:teuthology.orchestra.run.smithi161.stdout:(130/157): python3-logutils-0.3.5-21.el9.noarch 2.1 MB/s | 46 kB 00:00 2024-09-21T03:32:29.000 INFO:teuthology.orchestra.run.smithi161.stdout:(131/157): python3-more-itertools-8.12.0-2.el9. 3.1 MB/s | 79 kB 00:00 2024-09-21T03:32:29.017 INFO:teuthology.orchestra.run.smithi161.stdout:(132/157): python3-natsort-7.1.1-5.el9.noarch.r 3.4 MB/s | 58 kB 00:00 2024-09-21T03:32:29.059 INFO:teuthology.orchestra.run.smithi161.stdout:(133/157): python3-kubernetes-26.1.0-2.el9.noar 5.8 MB/s | 1.0 MB 00:00 2024-09-21T03:32:29.084 INFO:teuthology.orchestra.run.smithi161.stdout:(134/157): python3-pecan-1.4.2-3.el9.noarch.rpm 4.0 MB/s | 272 kB 00:00 2024-09-21T03:32:29.101 INFO:teuthology.orchestra.run.smithi161.stdout:(135/157): python3-portend-3.1.0-2.el9.noarch.r 391 kB/s | 16 kB 00:00 2024-09-21T03:32:29.127 INFO:teuthology.orchestra.run.smithi161.stdout:(136/157): python3-pyperclip-1.8.2-2.el9.noarch 1.1 MB/s | 28 kB 00:00 2024-09-21T03:32:29.152 INFO:teuthology.orchestra.run.smithi161.stdout:(137/157): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.3 MB/s | 90 kB 00:00 2024-09-21T03:32:29.177 INFO:teuthology.orchestra.run.smithi161.stdout:(138/157): python3-repoze-lru-0.7-16.el9.noarch 611 kB/s | 31 kB 00:00 2024-09-21T03:32:29.203 INFO:teuthology.orchestra.run.smithi161.stdout:(139/157): python3-rsa-4.9-2.el9.noarch.rpm 2.3 MB/s | 59 kB 00:00 2024-09-21T03:32:29.228 INFO:teuthology.orchestra.run.smithi161.stdout:(140/157): python3-routes-2.5.1-5.el9.noarch.rp 2.4 MB/s | 188 kB 00:00 2024-09-21T03:32:29.253 INFO:teuthology.orchestra.run.smithi161.stdout:(141/157): python3-tempora-5.0.0-2.el9.noarch.r 712 kB/s | 36 kB 00:00 2024-09-21T03:32:29.278 INFO:teuthology.orchestra.run.smithi161.stdout:(142/157): python3-typing-extensions-4.12.2-2.e 1.4 MB/s | 74 kB 00:00 2024-09-21T03:32:29.303 INFO:teuthology.orchestra.run.smithi161.stdout:(143/157): python3-websocket-client-1.2.3-2.el9 3.5 MB/s | 90 kB 00:00 2024-09-21T03:32:29.337 INFO:teuthology.orchestra.run.smithi161.stdout:(144/157): python3-webob-1.8.8-2.el9.noarch.rpm 2.7 MB/s | 230 kB 00:00 2024-09-21T03:32:29.362 INFO:teuthology.orchestra.run.smithi161.stdout:(145/157): python3-zc-lockfile-2.0-10.el9.noarc 803 kB/s | 20 kB 00:00 2024-09-21T03:32:29.388 INFO:teuthology.orchestra.run.smithi161.stdout:(146/157): python3-werkzeug-2.0.3-3.el9.1.noarc 5.0 MB/s | 427 kB 00:00 2024-09-21T03:32:29.421 INFO:teuthology.orchestra.run.smithi161.stdout:(147/157): re2-20211101-20.el9.x86_64.rpm 3.2 MB/s | 191 kB 00:00 2024-09-21T03:32:29.480 INFO:teuthology.orchestra.run.smithi161.stdout:(148/157): thrift-0.15.0-2.el9.x86_64.rpm 17 MB/s | 1.6 MB 00:00 2024-09-21T03:32:29.856 INFO:teuthology.orchestra.run.smithi161.stdout:(149/157): librbd1-19.3.0-5120.g18432218.el9.x8 8.4 MB/s | 3.2 MB 00:00 2024-09-21T03:32:29.914 INFO:teuthology.orchestra.run.smithi161.stdout:(150/157): librados2-19.3.0-5120.g18432218.el9. 6.9 MB/s | 3.4 MB 00:00 2024-09-21T03:32:30.022 INFO:teuthology.orchestra.run.smithi161.stdout:(151/157): NetworkManager-1.51.0-1.el9.x86_64.r 14 MB/s | 2.3 MB 00:00 2024-09-21T03:32:30.064 INFO:teuthology.orchestra.run.smithi161.stdout:(152/157): NetworkManager-team-1.51.0-1.el9.x86 980 kB/s | 41 kB 00:00 2024-09-21T03:32:30.123 INFO:teuthology.orchestra.run.smithi161.stdout:(153/157): NetworkManager-libnm-1.51.0-1.el9.x8 8.7 MB/s | 1.8 MB 00:00 2024-09-21T03:32:30.148 INFO:teuthology.orchestra.run.smithi161.stdout:(154/157): NetworkManager-tui-1.51.0-1.el9.x86_ 2.9 MB/s | 249 kB 00:00 2024-09-21T03:32:30.198 INFO:teuthology.orchestra.run.smithi161.stdout:(155/157): openssl-3.2.2-6.el9.x86_64.rpm 18 MB/s | 1.4 MB 00:00 2024-09-21T03:32:30.257 INFO:teuthology.orchestra.run.smithi161.stdout:(156/157): openssl-libs-3.2.2-6.el9.x86_64.rpm 22 MB/s | 2.4 MB 00:00 2024-09-21T03:32:31.180 INFO:teuthology.orchestra.run.smithi039.stdout: Preparing : 1/1 2024-09-21T03:32:31.327 INFO:teuthology.orchestra.run.smithi039.stdout: Upgrading : openssl-libs-1:3.2.2-6.el9.x86_64 1/165 2024-09-21T03:32:31.369 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/165 2024-09-21T03:32:31.429 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/165 2024-09-21T03:32:31.469 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/165 2024-09-21T03:32:31.495 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/165 2024-09-21T03:32:31.553 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/165 2024-09-21T03:32:31.921 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/165 2024-09-21T03:32:32.029 INFO:teuthology.orchestra.run.smithi039.stdout: Upgrading : librados2-2:19.3.0-5120.g18432218.el9.x86_64 8/165 2024-09-21T03:32:32.160 INFO:teuthology.orchestra.run.smithi039.stdout: Running scriptlet: librados2-2:19.3.0-5120.g18432218.el9.x86_64 8/165 2024-09-21T03:32:32.188 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : libcephfs2-2:19.3.0-5120.g18432218.el9.x86_64 9/165 2024-09-21T03:32:32.281 INFO:teuthology.orchestra.run.smithi039.stdout: Running scriptlet: libcephfs2-2:19.3.0-5120.g18432218.el9.x86_64 9/165 2024-09-21T03:32:32.309 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-rados-2:19.3.0-5120.g18432218.el9.x86_64 10/165 2024-09-21T03:32:32.340 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : postgresql-private-libs-13.16-1.el9.x86_64 11/165 2024-09-21T03:32:32.354 INFO:teuthology.orchestra.run.smithi150.stdout: Preparing : 1/1 2024-09-21T03:32:32.463 INFO:teuthology.orchestra.run.smithi150.stdout: Upgrading : openssl-libs-1:3.2.2-6.el9.x86_64 1/165 2024-09-21T03:32:32.503 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/165 2024-09-21T03:32:32.525 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : libnbd-1.20.2-2.el9.x86_64 12/165 2024-09-21T03:32:32.556 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/165 2024-09-21T03:32:32.583 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : postgresql-13.16-1.el9.x86_64 13/165 2024-09-21T03:32:32.589 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/165 2024-09-21T03:32:32.610 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : libcephsqlite-2:19.3.0-5120.g18432218.el9.x86_64 14/165 2024-09-21T03:32:32.614 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/165 2024-09-21T03:32:32.657 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/165 2024-09-21T03:32:32.726 INFO:teuthology.orchestra.run.smithi039.stdout: Running scriptlet: libcephsqlite-2:19.3.0-5120.g18432218.el9.x86_64 14/165 2024-09-21T03:32:32.753 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : libradosstriper1-2:19.3.0-5120.g18432218.el9.x86 15/165 2024-09-21T03:32:32.816 INFO:teuthology.orchestra.run.smithi039.stdout: Running scriptlet: libradosstriper1-2:19.3.0-5120.g18432218.el9.x86 15/165 2024-09-21T03:32:32.845 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 16/165 2024-09-21T03:32:32.868 INFO:teuthology.orchestra.run.smithi039.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 16/165 2024-09-21T03:32:33.015 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/165 2024-09-21T03:32:33.058 INFO:teuthology.orchestra.run.smithi039.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 17/165 2024-09-21T03:32:33.224 INFO:teuthology.orchestra.run.smithi150.stdout: Upgrading : librados2-2:19.3.0-5120.g18432218.el9.x86_64 8/165 2024-09-21T03:32:33.232 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 17/165 2024-09-21T03:32:33.353 INFO:teuthology.orchestra.run.smithi039.stdout: Upgrading : NetworkManager-libnm-1:1.51.0-1.el9.x86_64 18/165 2024-09-21T03:32:33.469 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: librados2-2:19.3.0-5120.g18432218.el9.x86_64 8/165 2024-09-21T03:32:33.484 INFO:teuthology.orchestra.run.smithi039.stdout: Running scriptlet: NetworkManager-1:1.51.0-1.el9.x86_64 19/165 2024-09-21T03:32:33.491 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : libcephfs2-2:19.3.0-5120.g18432218.el9.x86_64 9/165 2024-09-21T03:32:33.567 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: libcephfs2-2:19.3.0-5120.g18432218.el9.x86_64 9/165 2024-09-21T03:32:33.571 INFO:teuthology.orchestra.run.smithi039.stdout: Upgrading : NetworkManager-1:1.51.0-1.el9.x86_64 19/165 2024-09-21T03:32:33.596 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-rados-2:19.3.0-5120.g18432218.el9.x86_64 10/165 2024-09-21T03:32:33.634 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : postgresql-private-libs-13.16-1.el9.x86_64 11/165 2024-09-21T03:32:33.796 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : libnbd-1.20.2-2.el9.x86_64 12/165 2024-09-21T03:32:33.802 INFO:teuthology.orchestra.run.smithi039.stdout: Running scriptlet: NetworkManager-1:1.51.0-1.el9.x86_64 19/165 2024-09-21T03:32:33.844 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : postgresql-13.16-1.el9.x86_64 13/165 2024-09-21T03:32:33.871 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : libcephsqlite-2:19.3.0-5120.g18432218.el9.x86_64 14/165 2024-09-21T03:32:33.914 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : re2-1:20211101-20.el9.x86_64 20/165 2024-09-21T03:32:33.944 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 21/165 2024-09-21T03:32:33.978 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: libcephsqlite-2:19.3.0-5120.g18432218.el9.x86_64 14/165 2024-09-21T03:32:33.985 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : liboath-2.6.7-2.el9.x86_64 22/165 2024-09-21T03:32:34.006 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : libradosstriper1-2:19.3.0-5120.g18432218.el9.x86 15/165 2024-09-21T03:32:34.068 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: libradosstriper1-2:19.3.0-5120.g18432218.el9.x86 15/165 2024-09-21T03:32:34.069 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 23/165 2024-09-21T03:32:34.090 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 16/165 2024-09-21T03:32:34.110 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 16/165 2024-09-21T03:32:34.129 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 24/165 2024-09-21T03:32:34.177 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-packaging-20.9-5.el9.noarch 25/165 2024-09-21T03:32:34.221 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 26/165 2024-09-21T03:32:34.253 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 27/165 2024-09-21T03:32:34.275 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 17/165 2024-09-21T03:32:34.280 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : lua-5.4.4-4.el9.x86_64 28/165 2024-09-21T03:32:34.352 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 29/165 2024-09-21T03:32:34.413 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-urllib3-1.26.5-6.el9.noarch 30/165 2024-09-21T03:32:34.449 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 17/165 2024-09-21T03:32:34.450 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 31/165 2024-09-21T03:32:34.497 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : libquadmath-11.5.0-2.el9.x86_64 32/165 2024-09-21T03:32:34.562 INFO:teuthology.orchestra.run.smithi150.stdout: Upgrading : NetworkManager-libnm-1:1.51.0-1.el9.x86_64 18/165 2024-09-21T03:32:34.578 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : libgfortran-11.5.0-2.el9.x86_64 33/165 2024-09-21T03:32:34.621 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-ceph-common-2:19.3.0-5120.g18432218.el9. 34/165 2024-09-21T03:32:34.660 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: NetworkManager-1:1.51.0-1.el9.x86_64 19/165 2024-09-21T03:32:34.665 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-ceph-argparse-2:19.3.0-5120.g18432218.el 35/165 2024-09-21T03:32:34.693 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-cephfs-2:19.3.0-5120.g18432218.el9.x86_6 36/165 2024-09-21T03:32:34.743 INFO:teuthology.orchestra.run.smithi150.stdout: Upgrading : NetworkManager-1:1.51.0-1.el9.x86_64 19/165 2024-09-21T03:32:34.766 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 37/165 2024-09-21T03:32:34.808 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : luarocks-3.9.2-5.el9.noarch 38/165 2024-09-21T03:32:34.894 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 39/165 2024-09-21T03:32:34.965 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: NetworkManager-1:1.51.0-1.el9.x86_64 19/165 2024-09-21T03:32:35.049 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 40/165 2024-09-21T03:32:35.073 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : re2-1:20211101-20.el9.x86_64 20/165 2024-09-21T03:32:35.103 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 41/165 2024-09-21T03:32:35.105 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 21/165 2024-09-21T03:32:35.139 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : liboath-2.6.7-2.el9.x86_64 22/165 2024-09-21T03:32:35.204 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 23/165 2024-09-21T03:32:35.258 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 24/165 2024-09-21T03:32:35.297 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-packaging-20.9-5.el9.noarch 25/165 2024-09-21T03:32:35.341 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 26/165 2024-09-21T03:32:35.373 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 27/165 2024-09-21T03:32:35.401 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : lua-5.4.4-4.el9.x86_64 28/165 2024-09-21T03:32:35.469 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 29/165 2024-09-21T03:32:35.523 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-urllib3-1.26.5-6.el9.noarch 30/165 2024-09-21T03:32:35.554 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 31/165 2024-09-21T03:32:35.589 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : libquadmath-11.5.0-2.el9.x86_64 32/165 2024-09-21T03:32:35.675 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-rsa-4.9-2.el9.noarch 42/165 2024-09-21T03:32:35.692 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : libgfortran-11.5.0-2.el9.x86_64 33/165 2024-09-21T03:32:35.734 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-ceph-common-2:19.3.0-5120.g18432218.el9. 34/165 2024-09-21T03:32:35.779 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 43/165 2024-09-21T03:32:35.780 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-ceph-argparse-2:19.3.0-5120.g18432218.el 35/165 2024-09-21T03:32:35.801 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 44/165 2024-09-21T03:32:35.806 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-cephfs-2:19.3.0-5120.g18432218.el9.x86_6 36/165 2024-09-21T03:32:35.850 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 45/165 2024-09-21T03:32:35.874 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 37/165 2024-09-21T03:32:35.883 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : librados-devel-2:19.3.0-5120.g18432218.el9.x86_6 46/165 2024-09-21T03:32:35.920 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : luarocks-3.9.2-5.el9.noarch 38/165 2024-09-21T03:32:35.925 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 47/165 2024-09-21T03:32:35.996 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 39/165 2024-09-21T03:32:36.085 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : socat-1.7.4.1-6.el9.x86_64 48/165 2024-09-21T03:32:36.126 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 40/165 2024-09-21T03:32:36.144 INFO:teuthology.orchestra.run.smithi039.stdout: Upgrading : openssl-1:3.2.2-6.el9.x86_64 49/165 2024-09-21T03:32:36.180 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 41/165 2024-09-21T03:32:36.248 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 50/165 2024-09-21T03:32:36.310 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 51/165 2024-09-21T03:32:36.384 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 52/165 2024-09-21T03:32:36.421 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-webob-1.8.8-2.el9.noarch 53/165 2024-09-21T03:32:36.457 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-typing-extensions-4.12.2-2.el9.noarch 54/165 2024-09-21T03:32:36.505 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 55/165 2024-09-21T03:32:36.548 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 56/165 2024-09-21T03:32:36.592 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 57/165 2024-09-21T03:32:36.700 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-rsa-4.9-2.el9.noarch 42/165 2024-09-21T03:32:36.714 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 58/165 2024-09-21T03:32:36.786 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 43/165 2024-09-21T03:32:36.814 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 44/165 2024-09-21T03:32:36.857 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 45/165 2024-09-21T03:32:36.870 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 59/165 2024-09-21T03:32:36.904 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : librados-devel-2:19.3.0-5120.g18432218.el9.x86_6 46/165 2024-09-21T03:32:36.913 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 60/165 2024-09-21T03:32:36.946 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 47/165 2024-09-21T03:32:36.955 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 61/165 2024-09-21T03:32:37.064 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 62/165 2024-09-21T03:32:37.097 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-google-auth-1:2.34.0-1.el9.noarch 63/165 2024-09-21T03:32:37.114 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : socat-1.7.4.1-6.el9.x86_64 48/165 2024-09-21T03:32:37.153 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 64/165 2024-09-21T03:32:37.190 INFO:teuthology.orchestra.run.smithi150.stdout: Upgrading : openssl-1:3.2.2-6.el9.x86_64 49/165 2024-09-21T03:32:37.212 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : gperftools-libs-2.9.1-3.el9.x86_64 65/165 2024-09-21T03:32:37.236 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 66/165 2024-09-21T03:32:37.281 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 50/165 2024-09-21T03:32:37.333 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 67/165 2024-09-21T03:32:37.346 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 51/165 2024-09-21T03:32:37.426 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 52/165 2024-09-21T03:32:37.471 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 68/165 2024-09-21T03:32:37.484 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-webob-1.8.8-2.el9.noarch 53/165 2024-09-21T03:32:37.511 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 69/165 2024-09-21T03:32:37.519 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-typing-extensions-4.12.2-2.el9.noarch 54/165 2024-09-21T03:32:37.547 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 70/165 2024-09-21T03:32:37.566 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 55/165 2024-09-21T03:32:37.582 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 71/165 2024-09-21T03:32:37.611 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 56/165 2024-09-21T03:32:37.621 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : uuid-1.6.2-55.el9.x86_64 72/165 2024-09-21T03:32:37.654 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 57/165 2024-09-21T03:32:37.669 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 73/165 2024-09-21T03:32:37.699 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 74/165 2024-09-21T03:32:37.705 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 58/165 2024-09-21T03:32:37.742 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 75/165 2024-09-21T03:32:37.784 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 76/165 2024-09-21T03:32:37.826 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 77/165 2024-09-21T03:32:37.840 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 59/165 2024-09-21T03:32:37.860 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 78/165 2024-09-21T03:32:37.881 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 60/165 2024-09-21T03:32:37.930 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 61/165 2024-09-21T03:32:37.951 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 79/165 2024-09-21T03:32:37.999 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 80/165 2024-09-21T03:32:38.029 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 81/165 2024-09-21T03:32:38.033 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 62/165 2024-09-21T03:32:38.073 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-google-auth-1:2.34.0-1.el9.noarch 63/165 2024-09-21T03:32:38.113 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 64/165 2024-09-21T03:32:38.138 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : gperftools-libs-2.9.1-3.el9.x86_64 65/165 2024-09-21T03:32:38.140 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 82/165 2024-09-21T03:32:38.162 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 66/165 2024-09-21T03:32:38.199 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 83/165 2024-09-21T03:32:38.237 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 67/165 2024-09-21T03:32:38.365 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 68/165 2024-09-21T03:32:38.403 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 69/165 2024-09-21T03:32:38.440 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 70/165 2024-09-21T03:32:38.482 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 71/165 2024-09-21T03:32:38.521 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : uuid-1.6.2-55.el9.x86_64 72/165 2024-09-21T03:32:38.553 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 73/165 2024-09-21T03:32:38.583 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 74/165 2024-09-21T03:32:38.634 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 75/165 2024-09-21T03:32:38.667 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 76/165 2024-09-21T03:32:38.717 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 77/165 2024-09-21T03:32:38.752 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 78/165 2024-09-21T03:32:38.838 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 79/165 2024-09-21T03:32:38.890 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 80/165 2024-09-21T03:32:38.921 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 81/165 2024-09-21T03:32:38.994 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 84/165 2024-09-21T03:32:39.036 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 85/165 2024-09-21T03:32:39.067 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 82/165 2024-09-21T03:32:39.070 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 86/165 2024-09-21T03:32:39.122 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 83/165 2024-09-21T03:32:39.177 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 87/165 2024-09-21T03:32:39.452 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-devel-3.9.19-8.el9.x86_64 88/165 2024-09-21T03:32:39.614 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-colorama-0.4.6-3.el9.noarch 89/165 2024-09-21T03:32:39.705 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 90/165 2024-09-21T03:32:39.739 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 91/165 2024-09-21T03:32:39.855 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 84/165 2024-09-21T03:32:39.903 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 85/165 2024-09-21T03:32:39.932 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : openblas-0.3.26-2.el9.x86_64 92/165 2024-09-21T03:32:39.937 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 86/165 2024-09-21T03:32:39.961 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : openblas-openmp-0.3.26-2.el9.x86_64 93/165 2024-09-21T03:32:40.034 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 87/165 2024-09-21T03:32:40.051 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 94/165 2024-09-21T03:32:40.091 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-devel-3.9.19-8.el9.x86_64 88/165 2024-09-21T03:32:40.136 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-colorama-0.4.6-3.el9.noarch 89/165 2024-09-21T03:32:40.232 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 90/165 2024-09-21T03:32:40.305 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 91/165 2024-09-21T03:32:40.490 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : openblas-0.3.26-2.el9.x86_64 92/165 2024-09-21T03:32:40.528 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : openblas-openmp-0.3.26-2.el9.x86_64 93/165 2024-09-21T03:32:40.606 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 94/165 2024-09-21T03:32:40.772 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 95/165 2024-09-21T03:32:40.899 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 96/165 2024-09-21T03:32:41.252 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 95/165 2024-09-21T03:32:41.369 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 96/165 2024-09-21T03:32:42.286 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 97/165 2024-09-21T03:32:42.382 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 98/165 2024-09-21T03:32:42.410 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 99/165 2024-09-21T03:32:42.445 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : libuuid-devel-2.37.4-20.el9.x86_64 100/165 2024-09-21T03:32:42.585 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 97/165 2024-09-21T03:32:42.675 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 98/165 2024-09-21T03:32:42.703 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 99/165 2024-09-21T03:32:42.736 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : libuuid-devel-2.37.4-20.el9.x86_64 100/165 2024-09-21T03:32:42.789 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 101/165 2024-09-21T03:32:42.833 INFO:teuthology.orchestra.run.smithi039.stdout: Upgrading : librbd1-2:19.3.0-5120.g18432218.el9.x86_64 102/165 2024-09-21T03:32:42.934 INFO:teuthology.orchestra.run.smithi039.stdout: Running scriptlet: librbd1-2:19.3.0-5120.g18432218.el9.x86_64 102/165 2024-09-21T03:32:42.962 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-rbd-2:19.3.0-5120.g18432218.el9.x86_64 103/165 2024-09-21T03:32:42.999 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 104/165 2024-09-21T03:32:43.061 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 101/165 2024-09-21T03:32:43.116 INFO:teuthology.orchestra.run.smithi150.stdout: Upgrading : librbd1-2:19.3.0-5120.g18432218.el9.x86_64 102/165 2024-09-21T03:32:43.216 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: librbd1-2:19.3.0-5120.g18432218.el9.x86_64 102/165 2024-09-21T03:32:43.254 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-rbd-2:19.3.0-5120.g18432218.el9.x86_64 103/165 2024-09-21T03:32:43.290 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 104/165 2024-09-21T03:32:43.544 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 105/165 2024-09-21T03:32:43.577 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : librgw2-2:19.3.0-5120.g18432218.el9.x86_64 106/165 2024-09-21T03:32:43.632 INFO:teuthology.orchestra.run.smithi039.stdout: Running scriptlet: librgw2-2:19.3.0-5120.g18432218.el9.x86_64 106/165 2024-09-21T03:32:43.662 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-rgw-2:19.3.0-5120.g18432218.el9.x86_64 107/165 2024-09-21T03:32:43.792 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 105/165 2024-09-21T03:32:43.818 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : librgw2-2:19.3.0-5120.g18432218.el9.x86_64 106/165 2024-09-21T03:32:43.889 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: librgw2-2:19.3.0-5120.g18432218.el9.x86_64 106/165 2024-09-21T03:32:43.920 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-rgw-2:19.3.0-5120.g18432218.el9.x86_64 107/165 2024-09-21T03:32:45.968 INFO:teuthology.orchestra.run.smithi039.stdout: Running scriptlet: ceph-common-2:19.3.0-5120.g18432218.el9.x86_64 108/165 2024-09-21T03:32:46.008 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : ceph-common-2:19.3.0-5120.g18432218.el9.x86_64 108/165 2024-09-21T03:32:46.192 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: ceph-common-2:19.3.0-5120.g18432218.el9.x86_64 108/165 2024-09-21T03:32:46.233 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : ceph-common-2:19.3.0-5120.g18432218.el9.x86_64 108/165 2024-09-21T03:32:46.273 INFO:teuthology.orchestra.run.smithi161.stdout:(157/157): python3-scipy-1.6.2-8.el9.x86_64.rpm 711 kB/s | 16 MB 00:23 2024-09-21T03:32:46.278 INFO:teuthology.orchestra.run.smithi161.stdout:-------------------------------------------------------------------------------- 2024-09-21T03:32:46.278 INFO:teuthology.orchestra.run.smithi161.stdout:Total 5.0 MB/s | 228 MB 00:45 2024-09-21T03:32:46.634 INFO:teuthology.orchestra.run.smithi039.stdout: Running scriptlet: ceph-common-2:19.3.0-5120.g18432218.el9.x86_64 108/165 2024-09-21T03:32:46.678 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : ceph-base-2:19.3.0-5120.g18432218.el9.x86_64 109/165 2024-09-21T03:32:46.781 INFO:teuthology.orchestra.run.smithi039.stdout: Running scriptlet: ceph-base-2:19.3.0-5120.g18432218.el9.x86_64 109/165 2024-09-21T03:32:46.781 INFO:teuthology.orchestra.run.smithi039.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-09-21T03:32:46.781 INFO:teuthology.orchestra.run.smithi039.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-09-21T03:32:46.781 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T03:32:46.804 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: ceph-common-2:19.3.0-5120.g18432218.el9.x86_64 108/165 2024-09-21T03:32:46.821 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : ceph-selinux-2:19.3.0-5120.g18432218.el9.x86_64 110/165 2024-09-21T03:32:46.845 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : ceph-base-2:19.3.0-5120.g18432218.el9.x86_64 109/165 2024-09-21T03:32:46.920 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: ceph-base-2:19.3.0-5120.g18432218.el9.x86_64 109/165 2024-09-21T03:32:46.920 INFO:teuthology.orchestra.run.smithi150.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-09-21T03:32:46.920 INFO:teuthology.orchestra.run.smithi150.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-09-21T03:32:46.920 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-09-21T03:32:46.954 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : ceph-selinux-2:19.3.0-5120.g18432218.el9.x86_64 110/165 2024-09-21T03:32:47.525 INFO:teuthology.orchestra.run.smithi161.stdout:Running transaction check 2024-09-21T03:32:47.677 INFO:teuthology.orchestra.run.smithi161.stdout:Transaction check succeeded. 2024-09-21T03:32:47.678 INFO:teuthology.orchestra.run.smithi161.stdout:Running transaction test 2024-09-21T03:32:49.493 INFO:teuthology.orchestra.run.smithi161.stdout:Transaction test succeeded. 2024-09-21T03:32:49.493 INFO:teuthology.orchestra.run.smithi161.stdout:Running transaction 2024-09-21T03:32:55.758 INFO:teuthology.orchestra.run.smithi161.stdout: Preparing : 1/1 2024-09-21T03:32:55.876 INFO:teuthology.orchestra.run.smithi161.stdout: Upgrading : openssl-libs-1:3.2.2-6.el9.x86_64 1/165 2024-09-21T03:32:55.914 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/165 2024-09-21T03:32:55.967 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/165 2024-09-21T03:32:56.001 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/165 2024-09-21T03:32:56.027 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/165 2024-09-21T03:32:56.070 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/165 2024-09-21T03:32:56.409 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/165 2024-09-21T03:32:56.536 INFO:teuthology.orchestra.run.smithi161.stdout: Upgrading : librados2-2:19.3.0-5120.g18432218.el9.x86_64 8/165 2024-09-21T03:32:56.654 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: librados2-2:19.3.0-5120.g18432218.el9.x86_64 8/165 2024-09-21T03:32:56.688 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : libcephfs2-2:19.3.0-5120.g18432218.el9.x86_64 9/165 2024-09-21T03:32:56.762 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: libcephfs2-2:19.3.0-5120.g18432218.el9.x86_64 9/165 2024-09-21T03:32:56.791 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-rados-2:19.3.0-5120.g18432218.el9.x86_64 10/165 2024-09-21T03:32:56.821 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : postgresql-private-libs-13.16-1.el9.x86_64 11/165 2024-09-21T03:32:56.976 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : libnbd-1.20.2-2.el9.x86_64 12/165 2024-09-21T03:32:57.031 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : postgresql-13.16-1.el9.x86_64 13/165 2024-09-21T03:32:57.059 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : libcephsqlite-2:19.3.0-5120.g18432218.el9.x86_64 14/165 2024-09-21T03:32:57.165 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: libcephsqlite-2:19.3.0-5120.g18432218.el9.x86_64 14/165 2024-09-21T03:32:57.193 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : libradosstriper1-2:19.3.0-5120.g18432218.el9.x86 15/165 2024-09-21T03:32:57.239 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: libradosstriper1-2:19.3.0-5120.g18432218.el9.x86 15/165 2024-09-21T03:32:57.261 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 16/165 2024-09-21T03:32:57.287 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 16/165 2024-09-21T03:32:57.482 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 17/165 2024-09-21T03:32:57.638 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 17/165 2024-09-21T03:32:57.779 INFO:teuthology.orchestra.run.smithi161.stdout: Upgrading : NetworkManager-libnm-1:1.51.0-1.el9.x86_64 18/165 2024-09-21T03:32:57.883 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: NetworkManager-1:1.51.0-1.el9.x86_64 19/165 2024-09-21T03:32:57.938 INFO:teuthology.orchestra.run.smithi161.stdout: Upgrading : NetworkManager-1:1.51.0-1.el9.x86_64 19/165 2024-09-21T03:32:58.153 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: NetworkManager-1:1.51.0-1.el9.x86_64 19/165 2024-09-21T03:32:58.260 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : re2-1:20211101-20.el9.x86_64 20/165 2024-09-21T03:32:58.284 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 21/165 2024-09-21T03:32:58.327 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : liboath-2.6.7-2.el9.x86_64 22/165 2024-09-21T03:32:58.392 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 23/165 2024-09-21T03:32:58.436 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 24/165 2024-09-21T03:32:58.468 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-packaging-20.9-5.el9.noarch 25/165 2024-09-21T03:32:58.511 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 26/165 2024-09-21T03:32:58.569 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 27/165 2024-09-21T03:32:58.597 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : lua-5.4.4-4.el9.x86_64 28/165 2024-09-21T03:32:58.645 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: ceph-selinux-2:19.3.0-5120.g18432218.el9.x86_64 110/165 2024-09-21T03:32:58.645 INFO:teuthology.orchestra.run.smithi150.stdout:skipping the directory /sys 2024-09-21T03:32:58.645 INFO:teuthology.orchestra.run.smithi150.stdout:skipping the directory /proc 2024-09-21T03:32:58.645 INFO:teuthology.orchestra.run.smithi150.stdout:skipping the directory /mnt 2024-09-21T03:32:58.645 INFO:teuthology.orchestra.run.smithi150.stdout:skipping the directory /var/tmp 2024-09-21T03:32:58.645 INFO:teuthology.orchestra.run.smithi150.stdout:skipping the directory /home 2024-09-21T03:32:58.645 INFO:teuthology.orchestra.run.smithi150.stdout:skipping the directory /root 2024-09-21T03:32:58.646 INFO:teuthology.orchestra.run.smithi150.stdout:skipping the directory /tmp 2024-09-21T03:32:58.646 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-09-21T03:32:58.665 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 29/165 2024-09-21T03:32:58.727 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-urllib3-1.26.5-6.el9.noarch 30/165 2024-09-21T03:32:58.775 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 31/165 2024-09-21T03:32:58.810 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : libquadmath-11.5.0-2.el9.x86_64 32/165 2024-09-21T03:32:58.824 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : ceph-volume-2:19.3.0-5120.g18432218.el9.noarch 111/165 2024-09-21T03:32:58.840 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: ceph-volume-2:19.3.0-5120.g18432218.el9.noarch 111/165 2024-09-21T03:32:58.840 INFO:teuthology.orchestra.run.smithi150.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T03:32:58.840 INFO:teuthology.orchestra.run.smithi150.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-09-21T03:32:58.840 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-09-21T03:32:58.891 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : libgfortran-11.5.0-2.el9.x86_64 33/165 2024-09-21T03:32:58.929 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-ceph-common-2:19.3.0-5120.g18432218.el9. 34/165 2024-09-21T03:32:58.973 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-ceph-argparse-2:19.3.0-5120.g18432218.el 35/165 2024-09-21T03:32:59.001 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-cephfs-2:19.3.0-5120.g18432218.el9.x86_6 36/165 2024-09-21T03:32:59.071 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 37/165 2024-09-21T03:32:59.116 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : luarocks-3.9.2-5.el9.noarch 38/165 2024-09-21T03:32:59.190 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 39/165 2024-09-21T03:32:59.219 INFO:teuthology.orchestra.run.smithi039.stdout: Running scriptlet: ceph-selinux-2:19.3.0-5120.g18432218.el9.x86_64 110/165 2024-09-21T03:32:59.219 INFO:teuthology.orchestra.run.smithi039.stdout:skipping the directory /sys 2024-09-21T03:32:59.219 INFO:teuthology.orchestra.run.smithi039.stdout:skipping the directory /proc 2024-09-21T03:32:59.220 INFO:teuthology.orchestra.run.smithi039.stdout:skipping the directory /mnt 2024-09-21T03:32:59.220 INFO:teuthology.orchestra.run.smithi039.stdout:skipping the directory /var/tmp 2024-09-21T03:32:59.220 INFO:teuthology.orchestra.run.smithi039.stdout:skipping the directory /home 2024-09-21T03:32:59.220 INFO:teuthology.orchestra.run.smithi039.stdout:skipping the directory /root 2024-09-21T03:32:59.220 INFO:teuthology.orchestra.run.smithi039.stdout:skipping the directory /tmp 2024-09-21T03:32:59.220 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T03:32:59.340 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 40/165 2024-09-21T03:32:59.393 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 41/165 2024-09-21T03:32:59.408 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : ceph-volume-2:19.3.0-5120.g18432218.el9.noarch 111/165 2024-09-21T03:32:59.425 INFO:teuthology.orchestra.run.smithi039.stdout: Running scriptlet: ceph-volume-2:19.3.0-5120.g18432218.el9.noarch 111/165 2024-09-21T03:32:59.425 INFO:teuthology.orchestra.run.smithi039.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T03:32:59.425 INFO:teuthology.orchestra.run.smithi039.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-09-21T03:32:59.425 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T03:32:59.911 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-rsa-4.9-2.el9.noarch 42/165 2024-09-21T03:33:00.005 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 43/165 2024-09-21T03:33:00.034 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 44/165 2024-09-21T03:33:00.077 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 45/165 2024-09-21T03:33:00.115 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : librados-devel-2:19.3.0-5120.g18432218.el9.x86_6 46/165 2024-09-21T03:33:00.157 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 47/165 2024-09-21T03:33:00.324 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : socat-1.7.4.1-6.el9.x86_64 48/165 2024-09-21T03:33:00.385 INFO:teuthology.orchestra.run.smithi161.stdout: Upgrading : openssl-1:3.2.2-6.el9.x86_64 49/165 2024-09-21T03:33:00.435 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : ceph-osd-2:19.3.0-5120.g18432218.el9.x86_64 112/165 2024-09-21T03:33:00.462 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: ceph-osd-2:19.3.0-5120.g18432218.el9.x86_64 112/165 2024-09-21T03:33:00.462 INFO:teuthology.orchestra.run.smithi150.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T03:33:00.462 INFO:teuthology.orchestra.run.smithi150.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-09-21T03:33:00.463 INFO:teuthology.orchestra.run.smithi150.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-21T03:33:00.463 INFO:teuthology.orchestra.run.smithi150.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-21T03:33:00.463 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-09-21T03:33:00.476 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 50/165 2024-09-21T03:33:00.540 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 51/165 2024-09-21T03:33:00.612 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 52/165 2024-09-21T03:33:00.645 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-webob-1.8.8-2.el9.noarch 53/165 2024-09-21T03:33:00.694 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : ceph-mds-2:19.3.0-5120.g18432218.el9.x86_64 113/165 2024-09-21T03:33:00.698 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-typing-extensions-4.12.2-2.el9.noarch 54/165 2024-09-21T03:33:00.718 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: ceph-mds-2:19.3.0-5120.g18432218.el9.x86_64 113/165 2024-09-21T03:33:00.718 INFO:teuthology.orchestra.run.smithi150.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T03:33:00.718 INFO:teuthology.orchestra.run.smithi150.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-09-21T03:33:00.718 INFO:teuthology.orchestra.run.smithi150.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-21T03:33:00.718 INFO:teuthology.orchestra.run.smithi150.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-21T03:33:00.718 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-09-21T03:33:00.744 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 55/165 2024-09-21T03:33:00.789 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 56/165 2024-09-21T03:33:00.840 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 57/165 2024-09-21T03:33:00.882 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 58/165 2024-09-21T03:33:01.019 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 59/165 2024-09-21T03:33:01.059 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 60/165 2024-09-21T03:33:01.077 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : ceph-osd-2:19.3.0-5120.g18432218.el9.x86_64 112/165 2024-09-21T03:33:01.101 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 61/165 2024-09-21T03:33:01.106 INFO:teuthology.orchestra.run.smithi039.stdout: Running scriptlet: ceph-osd-2:19.3.0-5120.g18432218.el9.x86_64 112/165 2024-09-21T03:33:01.106 INFO:teuthology.orchestra.run.smithi039.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T03:33:01.106 INFO:teuthology.orchestra.run.smithi039.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-09-21T03:33:01.106 INFO:teuthology.orchestra.run.smithi039.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-21T03:33:01.106 INFO:teuthology.orchestra.run.smithi039.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-21T03:33:01.107 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T03:33:01.186 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : ceph-mon-2:19.3.0-5120.g18432218.el9.x86_64 114/165 2024-09-21T03:33:01.203 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 62/165 2024-09-21T03:33:01.210 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: ceph-mon-2:19.3.0-5120.g18432218.el9.x86_64 114/165 2024-09-21T03:33:01.210 INFO:teuthology.orchestra.run.smithi150.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T03:33:01.210 INFO:teuthology.orchestra.run.smithi150.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-09-21T03:33:01.210 INFO:teuthology.orchestra.run.smithi150.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-21T03:33:01.210 INFO:teuthology.orchestra.run.smithi150.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-21T03:33:01.210 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-09-21T03:33:01.234 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-google-auth-1:2.34.0-1.el9.noarch 63/165 2024-09-21T03:33:01.282 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 64/165 2024-09-21T03:33:01.307 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : gperftools-libs-2.9.1-3.el9.x86_64 65/165 2024-09-21T03:33:01.361 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : automake-1.16.2-8.el9.noarch 115/165 2024-09-21T03:33:01.362 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 66/165 2024-09-21T03:33:01.362 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : ceph-mds-2:19.3.0-5120.g18432218.el9.x86_64 113/165 2024-09-21T03:33:01.369 INFO:teuthology.orchestra.run.smithi039.stdout: Running scriptlet: ceph-mds-2:19.3.0-5120.g18432218.el9.x86_64 113/165 2024-09-21T03:33:01.370 INFO:teuthology.orchestra.run.smithi039.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T03:33:01.370 INFO:teuthology.orchestra.run.smithi039.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-09-21T03:33:01.370 INFO:teuthology.orchestra.run.smithi039.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-21T03:33:01.370 INFO:teuthology.orchestra.run.smithi039.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-21T03:33:01.370 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T03:33:01.373 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-ply-3.11-14.el9.noarch 116/165 2024-09-21T03:33:01.417 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 67/165 2024-09-21T03:33:01.445 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 117/165 2024-09-21T03:33:01.551 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 68/165 2024-09-21T03:33:01.591 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 69/165 2024-09-21T03:33:01.619 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 70/165 2024-09-21T03:33:01.632 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 118/165 2024-09-21T03:33:01.653 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 71/165 2024-09-21T03:33:01.678 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 119/165 2024-09-21T03:33:01.693 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : uuid-1.6.2-55.el9.x86_64 72/165 2024-09-21T03:33:01.724 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 73/165 2024-09-21T03:33:01.747 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 120/165 2024-09-21T03:33:01.753 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 74/165 2024-09-21T03:33:01.788 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 75/165 2024-09-21T03:33:01.830 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 76/165 2024-09-21T03:33:01.845 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 121/165 2024-09-21T03:33:01.862 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : ceph-mon-2:19.3.0-5120.g18432218.el9.x86_64 114/165 2024-09-21T03:33:01.880 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 77/165 2024-09-21T03:33:01.887 INFO:teuthology.orchestra.run.smithi039.stdout: Running scriptlet: ceph-mon-2:19.3.0-5120.g18432218.el9.x86_64 114/165 2024-09-21T03:33:01.887 INFO:teuthology.orchestra.run.smithi039.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T03:33:01.887 INFO:teuthology.orchestra.run.smithi039.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-09-21T03:33:01.887 INFO:teuthology.orchestra.run.smithi039.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-21T03:33:01.887 INFO:teuthology.orchestra.run.smithi039.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-21T03:33:01.887 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T03:33:01.914 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 78/165 2024-09-21T03:33:01.993 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 122/165 2024-09-21T03:33:02.002 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 79/165 2024-09-21T03:33:02.002 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : automake-1.16.2-8.el9.noarch 115/165 2024-09-21T03:33:02.037 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 123/165 2024-09-21T03:33:02.053 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 80/165 2024-09-21T03:33:02.061 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-ply-3.11-14.el9.noarch 116/165 2024-09-21T03:33:02.069 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 124/165 2024-09-21T03:33:02.084 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 81/165 2024-09-21T03:33:02.096 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : fuse-2.9.9-16.el9.x86_64 125/165 2024-09-21T03:33:02.125 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 117/165 2024-09-21T03:33:02.187 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 82/165 2024-09-21T03:33:02.243 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 83/165 2024-09-21T03:33:02.332 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 118/165 2024-09-21T03:33:02.389 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 119/165 2024-09-21T03:33:02.471 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 120/165 2024-09-21T03:33:02.570 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 121/165 2024-09-21T03:33:02.573 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: cephadm-2:19.3.0-5120.g18432218.el9.noarch 126/165 2024-09-21T03:33:02.604 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : cephadm-2:19.3.0-5120.g18432218.el9.noarch 126/165 2024-09-21T03:33:02.628 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : ceph-prometheus-alerts-2:19.3.0-5120.g18432218.e 127/165 2024-09-21T03:33:02.693 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : ceph-grafana-dashboards-2:19.3.0-5120.g18432218. 128/165 2024-09-21T03:33:02.719 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : ceph-mgr-cephadm-2:19.3.0-5120.g18432218.el9.noa 129/165 2024-09-21T03:33:02.723 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 122/165 2024-09-21T03:33:02.764 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 123/165 2024-09-21T03:33:02.796 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 124/165 2024-09-21T03:33:02.822 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : fuse-2.9.9-16.el9.x86_64 125/165 2024-09-21T03:33:02.979 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 84/165 2024-09-21T03:33:03.024 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 85/165 2024-09-21T03:33:03.057 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 86/165 2024-09-21T03:33:03.158 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 87/165 2024-09-21T03:33:03.204 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-devel-3.9.19-8.el9.x86_64 88/165 2024-09-21T03:33:03.257 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-colorama-0.4.6-3.el9.noarch 89/165 2024-09-21T03:33:03.293 INFO:teuthology.orchestra.run.smithi039.stdout: Running scriptlet: cephadm-2:19.3.0-5120.g18432218.el9.noarch 126/165 2024-09-21T03:33:03.322 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : cephadm-2:19.3.0-5120.g18432218.el9.noarch 126/165 2024-09-21T03:33:03.344 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 90/165 2024-09-21T03:33:03.354 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : ceph-prometheus-alerts-2:19.3.0-5120.g18432218.e 127/165 2024-09-21T03:33:03.385 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 91/165 2024-09-21T03:33:03.425 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : ceph-grafana-dashboards-2:19.3.0-5120.g18432218. 128/165 2024-09-21T03:33:03.453 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : ceph-mgr-cephadm-2:19.3.0-5120.g18432218.el9.noa 129/165 2024-09-21T03:33:03.568 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : openblas-0.3.26-2.el9.x86_64 92/165 2024-09-21T03:33:03.599 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : openblas-openmp-0.3.26-2.el9.x86_64 93/165 2024-09-21T03:33:03.676 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 94/165 2024-09-21T03:33:04.322 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 95/165 2024-09-21T03:33:04.449 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 96/165 2024-09-21T03:33:05.673 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 97/165 2024-09-21T03:33:05.752 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 98/165 2024-09-21T03:33:05.780 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 99/165 2024-09-21T03:33:05.813 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : libuuid-devel-2.37.4-20.el9.x86_64 100/165 2024-09-21T03:33:05.922 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.3.0-5120.g18432218.el9.noa 129/165 2024-09-21T03:33:05.983 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : ceph-mgr-dashboard-2:19.3.0-5120.g18432218.el9.n 130/165 2024-09-21T03:33:06.138 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 101/165 2024-09-21T03:33:06.184 INFO:teuthology.orchestra.run.smithi161.stdout: Upgrading : librbd1-2:19.3.0-5120.g18432218.el9.x86_64 102/165 2024-09-21T03:33:06.266 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: librbd1-2:19.3.0-5120.g18432218.el9.x86_64 102/165 2024-09-21T03:33:06.297 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-rbd-2:19.3.0-5120.g18432218.el9.x86_64 103/165 2024-09-21T03:33:06.333 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 104/165 2024-09-21T03:33:06.831 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 105/165 2024-09-21T03:33:06.854 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : librgw2-2:19.3.0-5120.g18432218.el9.x86_64 106/165 2024-09-21T03:33:06.915 INFO:teuthology.orchestra.run.smithi039.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.3.0-5120.g18432218.el9.noa 129/165 2024-09-21T03:33:06.925 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: librgw2-2:19.3.0-5120.g18432218.el9.x86_64 106/165 2024-09-21T03:33:06.946 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.3.0-5120.g18432218.el9.n 130/165 2024-09-21T03:33:06.955 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-rgw-2:19.3.0-5120.g18432218.el9.x86_64 107/165 2024-09-21T03:33:06.975 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : ceph-mgr-diskprediction-local-2:19.3.0-5120.g184 131/165 2024-09-21T03:33:06.975 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : ceph-mgr-dashboard-2:19.3.0-5120.g18432218.el9.n 130/165 2024-09-21T03:33:07.089 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.3.0-5120.g184 131/165 2024-09-21T03:33:07.251 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : ceph-mgr-modules-core-2:19.3.0-5120.g18432218.el 132/165 2024-09-21T03:33:07.285 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : ceph-mgr-2:19.3.0-5120.g18432218.el9.x86_64 133/165 2024-09-21T03:33:07.310 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: ceph-mgr-2:19.3.0-5120.g18432218.el9.x86_64 133/165 2024-09-21T03:33:07.310 INFO:teuthology.orchestra.run.smithi150.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T03:33:07.310 INFO:teuthology.orchestra.run.smithi150.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-09-21T03:33:07.310 INFO:teuthology.orchestra.run.smithi150.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-21T03:33:07.310 INFO:teuthology.orchestra.run.smithi150.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-21T03:33:07.310 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-09-21T03:33:07.360 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : ceph-mgr-rook-2:19.3.0-5120.g18432218.el9.noarch 134/165 2024-09-21T03:33:07.372 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: ceph-mgr-rook-2:19.3.0-5120.g18432218.el9.noarch 134/165 2024-09-21T03:33:07.473 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : ceph-2:19.3.0-5120.g18432218.el9.x86_64 135/165 2024-09-21T03:33:07.517 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : ceph-fuse-2:19.3.0-5120.g18432218.el9.x86_64 136/165 2024-09-21T03:33:08.005 INFO:teuthology.orchestra.run.smithi039.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.3.0-5120.g18432218.el9.n 130/165 2024-09-21T03:33:08.034 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : ceph-mgr-diskprediction-local-2:19.3.0-5120.g184 131/165 2024-09-21T03:33:08.161 INFO:teuthology.orchestra.run.smithi039.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.3.0-5120.g184 131/165 2024-09-21T03:33:08.334 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : ceph-mgr-modules-core-2:19.3.0-5120.g18432218.el 132/165 2024-09-21T03:33:08.369 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : ceph-mgr-2:19.3.0-5120.g18432218.el9.x86_64 133/165 2024-09-21T03:33:08.395 INFO:teuthology.orchestra.run.smithi039.stdout: Running scriptlet: ceph-mgr-2:19.3.0-5120.g18432218.el9.x86_64 133/165 2024-09-21T03:33:08.395 INFO:teuthology.orchestra.run.smithi039.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T03:33:08.395 INFO:teuthology.orchestra.run.smithi039.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-09-21T03:33:08.396 INFO:teuthology.orchestra.run.smithi039.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-21T03:33:08.396 INFO:teuthology.orchestra.run.smithi039.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-21T03:33:08.396 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T03:33:08.445 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : ceph-mgr-rook-2:19.3.0-5120.g18432218.el9.noarch 134/165 2024-09-21T03:33:08.457 INFO:teuthology.orchestra.run.smithi039.stdout: Running scriptlet: ceph-mgr-rook-2:19.3.0-5120.g18432218.el9.noarch 134/165 2024-09-21T03:33:08.480 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : libtool-2.4.6-46.el9.x86_64 137/165 2024-09-21T03:33:08.513 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : ceph-radosgw-2:19.3.0-5120.g18432218.el9.x86_64 138/165 2024-09-21T03:33:08.537 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: ceph-radosgw-2:19.3.0-5120.g18432218.el9.x86_64 138/165 2024-09-21T03:33:08.538 INFO:teuthology.orchestra.run.smithi150.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T03:33:08.538 INFO:teuthology.orchestra.run.smithi150.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-09-21T03:33:08.538 INFO:teuthology.orchestra.run.smithi150.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-21T03:33:08.538 INFO:teuthology.orchestra.run.smithi150.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-21T03:33:08.538 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-09-21T03:33:08.561 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : ceph-2:19.3.0-5120.g18432218.el9.x86_64 135/165 2024-09-21T03:33:08.581 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : ceph-immutable-object-cache-2:19.3.0-5120.g18432 139/165 2024-09-21T03:33:08.604 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.3.0-5120.g18432 139/165 2024-09-21T03:33:08.604 INFO:teuthology.orchestra.run.smithi150.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T03:33:08.604 INFO:teuthology.orchestra.run.smithi150.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-09-21T03:33:08.604 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-09-21T03:33:08.616 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : ceph-fuse-2:19.3.0-5120.g18432218.el9.x86_64 136/165 2024-09-21T03:33:08.656 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : cephfs-mirror-2:19.3.0-5120.g18432218.el9.x86_64 140/165 2024-09-21T03:33:08.680 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: cephfs-mirror-2:19.3.0-5120.g18432218.el9.x86_64 140/165 2024-09-21T03:33:08.680 INFO:teuthology.orchestra.run.smithi150.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T03:33:08.680 INFO:teuthology.orchestra.run.smithi150.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-09-21T03:33:08.680 INFO:teuthology.orchestra.run.smithi150.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-21T03:33:08.680 INFO:teuthology.orchestra.run.smithi150.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-21T03:33:08.680 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-09-21T03:33:09.007 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : rbd-mirror-2:19.3.0-5120.g18432218.el9.x86_64 141/165 2024-09-21T03:33:09.031 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: rbd-mirror-2:19.3.0-5120.g18432218.el9.x86_64 141/165 2024-09-21T03:33:09.031 INFO:teuthology.orchestra.run.smithi150.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T03:33:09.031 INFO:teuthology.orchestra.run.smithi150.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-09-21T03:33:09.031 INFO:teuthology.orchestra.run.smithi150.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-21T03:33:09.031 INFO:teuthology.orchestra.run.smithi150.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-21T03:33:09.031 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-09-21T03:33:09.297 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: ceph-common-2:19.3.0-5120.g18432218.el9.x86_64 108/165 2024-09-21T03:33:09.336 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : ceph-common-2:19.3.0-5120.g18432218.el9.x86_64 108/165 2024-09-21T03:33:09.635 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : libtool-2.4.6-46.el9.x86_64 137/165 2024-09-21T03:33:09.664 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : ceph-radosgw-2:19.3.0-5120.g18432218.el9.x86_64 138/165 2024-09-21T03:33:09.690 INFO:teuthology.orchestra.run.smithi039.stdout: Running scriptlet: ceph-radosgw-2:19.3.0-5120.g18432218.el9.x86_64 138/165 2024-09-21T03:33:09.690 INFO:teuthology.orchestra.run.smithi039.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T03:33:09.690 INFO:teuthology.orchestra.run.smithi039.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-09-21T03:33:09.690 INFO:teuthology.orchestra.run.smithi039.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-21T03:33:09.690 INFO:teuthology.orchestra.run.smithi039.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-21T03:33:09.691 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T03:33:09.732 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : ceph-immutable-object-cache-2:19.3.0-5120.g18432 139/165 2024-09-21T03:33:09.756 INFO:teuthology.orchestra.run.smithi039.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.3.0-5120.g18432 139/165 2024-09-21T03:33:09.756 INFO:teuthology.orchestra.run.smithi039.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T03:33:09.756 INFO:teuthology.orchestra.run.smithi039.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-09-21T03:33:09.757 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T03:33:09.807 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : cephfs-mirror-2:19.3.0-5120.g18432218.el9.x86_64 140/165 2024-09-21T03:33:09.832 INFO:teuthology.orchestra.run.smithi039.stdout: Running scriptlet: cephfs-mirror-2:19.3.0-5120.g18432218.el9.x86_64 140/165 2024-09-21T03:33:09.832 INFO:teuthology.orchestra.run.smithi039.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T03:33:09.832 INFO:teuthology.orchestra.run.smithi039.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-09-21T03:33:09.832 INFO:teuthology.orchestra.run.smithi039.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-21T03:33:09.833 INFO:teuthology.orchestra.run.smithi039.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-21T03:33:09.833 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T03:33:09.923 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: ceph-common-2:19.3.0-5120.g18432218.el9.x86_64 108/165 2024-09-21T03:33:09.964 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : ceph-base-2:19.3.0-5120.g18432218.el9.x86_64 109/165 2024-09-21T03:33:10.047 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: ceph-base-2:19.3.0-5120.g18432218.el9.x86_64 109/165 2024-09-21T03:33:10.047 INFO:teuthology.orchestra.run.smithi161.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-09-21T03:33:10.047 INFO:teuthology.orchestra.run.smithi161.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-09-21T03:33:10.047 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-09-21T03:33:10.074 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : ceph-selinux-2:19.3.0-5120.g18432218.el9.x86_64 110/165 2024-09-21T03:33:10.175 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : rbd-mirror-2:19.3.0-5120.g18432218.el9.x86_64 141/165 2024-09-21T03:33:10.200 INFO:teuthology.orchestra.run.smithi039.stdout: Running scriptlet: rbd-mirror-2:19.3.0-5120.g18432218.el9.x86_64 141/165 2024-09-21T03:33:10.200 INFO:teuthology.orchestra.run.smithi039.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T03:33:10.200 INFO:teuthology.orchestra.run.smithi039.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-09-21T03:33:10.200 INFO:teuthology.orchestra.run.smithi039.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-21T03:33:10.200 INFO:teuthology.orchestra.run.smithi039.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-21T03:33:10.200 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T03:33:13.028 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : ceph-test-2:19.3.0-5120.g18432218.el9.x86_64 142/165 2024-09-21T03:33:13.070 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : rbd-fuse-2:19.3.0-5120.g18432218.el9.x86_64 143/165 2024-09-21T03:33:13.110 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : rbd-nbd-2:19.3.0-5120.g18432218.el9.x86_64 144/165 2024-09-21T03:33:13.142 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : xfsprogs-devel-6.4.0-4.el9.x86_64 145/165 2024-09-21T03:33:13.169 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : elfutils-libelf-devel-0.191-4.el9.x86_64 146/165 2024-09-21T03:33:13.355 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : cephfs-shell-2:19.3.0-5120.g18432218.el9.x86_64 147/165 2024-09-21T03:33:13.399 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : postgresql-contrib-13.16-1.el9.x86_64 148/165 2024-09-21T03:33:13.429 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : libcephfs-devel-2:19.3.0-5120.g18432218.el9.x86_ 149/165 2024-09-21T03:33:13.459 INFO:teuthology.orchestra.run.smithi150.stdout: Upgrading : NetworkManager-team-1:1.51.0-1.el9.x86_64 150/165 2024-09-21T03:33:13.530 INFO:teuthology.orchestra.run.smithi150.stdout: Upgrading : NetworkManager-tui-1:1.51.0-1.el9.x86_64 151/165 2024-09-21T03:33:14.142 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 152/165 2024-09-21T03:33:14.178 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : postgresql-server-13.16-1.el9.x86_64 152/165 2024-09-21T03:33:14.198 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 152/165 2024-09-21T03:33:14.463 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : ceph-test-2:19.3.0-5120.g18432218.el9.x86_64 142/165 2024-09-21T03:33:14.506 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : rbd-fuse-2:19.3.0-5120.g18432218.el9.x86_64 143/165 2024-09-21T03:33:14.671 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : rbd-nbd-2:19.3.0-5120.g18432218.el9.x86_64 144/165 2024-09-21T03:33:14.701 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : xfsprogs-devel-6.4.0-4.el9.x86_64 145/165 2024-09-21T03:33:14.729 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : elfutils-libelf-devel-0.191-4.el9.x86_64 146/165 2024-09-21T03:33:14.942 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : cephfs-shell-2:19.3.0-5120.g18432218.el9.x86_64 147/165 2024-09-21T03:33:14.983 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : postgresql-contrib-13.16-1.el9.x86_64 148/165 2024-09-21T03:33:15.036 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : libcephfs-devel-2:19.3.0-5120.g18432218.el9.x86_ 149/165 2024-09-21T03:33:15.038 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : cephfs-top-2:19.3.0-5120.g18432218.el9.noarch 153/165 2024-09-21T03:33:15.068 INFO:teuthology.orchestra.run.smithi039.stdout: Upgrading : NetworkManager-team-1:1.51.0-1.el9.x86_64 150/165 2024-09-21T03:33:15.114 INFO:teuthology.orchestra.run.smithi039.stdout: Upgrading : NetworkManager-tui-1:1.51.0-1.el9.x86_64 151/165 2024-09-21T03:33:15.133 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : openssl-devel-1:3.2.2-6.el9.x86_64 154/165 2024-09-21T03:33:15.173 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 155/165 2024-09-21T03:33:15.255 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : flex-2.6.4-9.el9.x86_64 156/165 2024-09-21T03:33:15.294 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : bison-3.7.4-5.el9.x86_64 157/165 2024-09-21T03:33:15.319 INFO:teuthology.orchestra.run.smithi150.stdout: Cleanup : NetworkManager-tui-1:1.48.8-1.el9.x86_64 158/165 2024-09-21T03:33:15.345 INFO:teuthology.orchestra.run.smithi150.stdout: Cleanup : openssl-1:3.2.2-4.el9.x86_64 159/165 2024-09-21T03:33:15.345 INFO:teuthology.orchestra.run.smithi150.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 160/165 2024-09-21T03:33:15.429 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 160/165 2024-09-21T03:33:15.430 INFO:teuthology.orchestra.run.smithi150.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 161/165 2024-09-21T03:33:15.514 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 161/165 2024-09-21T03:33:15.538 INFO:teuthology.orchestra.run.smithi150.stdout: Cleanup : NetworkManager-team-1:1.48.8-1.el9.x86_64 162/165 2024-09-21T03:33:15.544 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 163/165 2024-09-21T03:33:15.544 INFO:teuthology.orchestra.run.smithi150.stdout: Cleanup : NetworkManager-1:1.48.8-1.el9.x86_64 163/165 2024-09-21T03:33:15.724 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 163/165 2024-09-21T03:33:15.724 INFO:teuthology.orchestra.run.smithi150.stdout: Cleanup : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 164/165 2024-09-21T03:33:15.752 INFO:teuthology.orchestra.run.smithi039.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 152/165 2024-09-21T03:33:15.808 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: NetworkManager-libnm-1:1.48.8-1.el9.x86_64 164/165 2024-09-21T03:33:15.845 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : postgresql-server-13.16-1.el9.x86_64 152/165 2024-09-21T03:33:15.867 INFO:teuthology.orchestra.run.smithi039.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 152/165 2024-09-21T03:33:15.871 INFO:teuthology.orchestra.run.smithi150.stdout: Cleanup : openssl-libs-1:3.2.2-4.el9.x86_64 165/165 2024-09-21T03:33:16.790 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : cephfs-top-2:19.3.0-5120.g18432218.el9.noarch 153/165 2024-09-21T03:33:16.875 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : openssl-devel-1:3.2.2-6.el9.x86_64 154/165 2024-09-21T03:33:16.927 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 155/165 2024-09-21T03:33:17.027 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : flex-2.6.4-9.el9.x86_64 156/165 2024-09-21T03:33:17.061 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : bison-3.7.4-5.el9.x86_64 157/165 2024-09-21T03:33:17.087 INFO:teuthology.orchestra.run.smithi039.stdout: Cleanup : NetworkManager-tui-1:1.48.8-1.el9.x86_64 158/165 2024-09-21T03:33:17.113 INFO:teuthology.orchestra.run.smithi039.stdout: Cleanup : openssl-1:3.2.2-4.el9.x86_64 159/165 2024-09-21T03:33:17.113 INFO:teuthology.orchestra.run.smithi039.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 160/165 2024-09-21T03:33:17.205 INFO:teuthology.orchestra.run.smithi039.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 160/165 2024-09-21T03:33:17.205 INFO:teuthology.orchestra.run.smithi039.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 161/165 2024-09-21T03:33:17.289 INFO:teuthology.orchestra.run.smithi039.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 161/165 2024-09-21T03:33:17.314 INFO:teuthology.orchestra.run.smithi039.stdout: Cleanup : NetworkManager-team-1:1.48.8-1.el9.x86_64 162/165 2024-09-21T03:33:17.320 INFO:teuthology.orchestra.run.smithi039.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 163/165 2024-09-21T03:33:17.320 INFO:teuthology.orchestra.run.smithi039.stdout: Cleanup : NetworkManager-1:1.48.8-1.el9.x86_64 163/165 2024-09-21T03:33:17.508 INFO:teuthology.orchestra.run.smithi039.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 163/165 2024-09-21T03:33:17.508 INFO:teuthology.orchestra.run.smithi039.stdout: Cleanup : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 164/165 2024-09-21T03:33:17.592 INFO:teuthology.orchestra.run.smithi039.stdout: Running scriptlet: NetworkManager-libnm-1:1.48.8-1.el9.x86_64 164/165 2024-09-21T03:33:17.660 INFO:teuthology.orchestra.run.smithi039.stdout: Cleanup : openssl-libs-1:3.2.2-4.el9.x86_64 165/165 2024-09-21T03:33:21.853 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: ceph-selinux-2:19.3.0-5120.g18432218.el9.x86_64 110/165 2024-09-21T03:33:21.853 INFO:teuthology.orchestra.run.smithi161.stdout:skipping the directory /sys 2024-09-21T03:33:21.853 INFO:teuthology.orchestra.run.smithi161.stdout:skipping the directory /proc 2024-09-21T03:33:21.853 INFO:teuthology.orchestra.run.smithi161.stdout:skipping the directory /mnt 2024-09-21T03:33:21.853 INFO:teuthology.orchestra.run.smithi161.stdout:skipping the directory /var/tmp 2024-09-21T03:33:21.853 INFO:teuthology.orchestra.run.smithi161.stdout:skipping the directory /home 2024-09-21T03:33:21.853 INFO:teuthology.orchestra.run.smithi161.stdout:skipping the directory /root 2024-09-21T03:33:21.853 INFO:teuthology.orchestra.run.smithi161.stdout:skipping the directory /tmp 2024-09-21T03:33:21.853 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-09-21T03:33:22.025 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : ceph-volume-2:19.3.0-5120.g18432218.el9.noarch 111/165 2024-09-21T03:33:22.043 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: ceph-volume-2:19.3.0-5120.g18432218.el9.noarch 111/165 2024-09-21T03:33:22.043 INFO:teuthology.orchestra.run.smithi161.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T03:33:22.043 INFO:teuthology.orchestra.run.smithi161.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-09-21T03:33:22.043 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-09-21T03:33:22.200 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: openssl-libs-1:3.2.2-4.el9.x86_64 165/165 2024-09-21T03:33:22.200 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : ceph-2:19.3.0-5120.g18432218.el9.x86_64 1/165 2024-09-21T03:33:22.200 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : ceph-base-2:19.3.0-5120.g18432218.el9.x86_64 2/165 2024-09-21T03:33:22.200 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : ceph-common-2:19.3.0-5120.g18432218.el9.x86_64 3/165 2024-09-21T03:33:22.200 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : ceph-fuse-2:19.3.0-5120.g18432218.el9.x86_64 4/165 2024-09-21T03:33:22.200 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : ceph-immutable-object-cache-2:19.3.0-5120.g18432 5/165 2024-09-21T03:33:22.200 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : ceph-mds-2:19.3.0-5120.g18432218.el9.x86_64 6/165 2024-09-21T03:33:22.200 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : ceph-mgr-2:19.3.0-5120.g18432218.el9.x86_64 7/165 2024-09-21T03:33:22.200 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : ceph-mon-2:19.3.0-5120.g18432218.el9.x86_64 8/165 2024-09-21T03:33:22.200 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : ceph-osd-2:19.3.0-5120.g18432218.el9.x86_64 9/165 2024-09-21T03:33:22.200 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : ceph-radosgw-2:19.3.0-5120.g18432218.el9.x86_64 10/165 2024-09-21T03:33:22.201 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : ceph-selinux-2:19.3.0-5120.g18432218.el9.x86_64 11/165 2024-09-21T03:33:22.201 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : ceph-test-2:19.3.0-5120.g18432218.el9.x86_64 12/165 2024-09-21T03:33:22.201 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : cephfs-mirror-2:19.3.0-5120.g18432218.el9.x86_64 13/165 2024-09-21T03:33:22.201 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : cephfs-shell-2:19.3.0-5120.g18432218.el9.x86_64 14/165 2024-09-21T03:33:22.201 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : libcephfs-devel-2:19.3.0-5120.g18432218.el9.x86_ 15/165 2024-09-21T03:33:22.201 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : libcephfs2-2:19.3.0-5120.g18432218.el9.x86_64 16/165 2024-09-21T03:33:22.201 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : libcephsqlite-2:19.3.0-5120.g18432218.el9.x86_64 17/165 2024-09-21T03:33:22.201 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : librados-devel-2:19.3.0-5120.g18432218.el9.x86_6 18/165 2024-09-21T03:33:22.201 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : libradosstriper1-2:19.3.0-5120.g18432218.el9.x86 19/165 2024-09-21T03:33:22.201 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : librgw2-2:19.3.0-5120.g18432218.el9.x86_64 20/165 2024-09-21T03:33:22.201 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-ceph-argparse-2:19.3.0-5120.g18432218.el 21/165 2024-09-21T03:33:22.201 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-ceph-common-2:19.3.0-5120.g18432218.el9. 22/165 2024-09-21T03:33:22.201 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-cephfs-2:19.3.0-5120.g18432218.el9.x86_6 23/165 2024-09-21T03:33:22.202 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-rados-2:19.3.0-5120.g18432218.el9.x86_64 24/165 2024-09-21T03:33:22.202 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-rbd-2:19.3.0-5120.g18432218.el9.x86_64 25/165 2024-09-21T03:33:22.202 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-rgw-2:19.3.0-5120.g18432218.el9.x86_64 26/165 2024-09-21T03:33:22.202 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : rbd-fuse-2:19.3.0-5120.g18432218.el9.x86_64 27/165 2024-09-21T03:33:22.202 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : rbd-mirror-2:19.3.0-5120.g18432218.el9.x86_64 28/165 2024-09-21T03:33:22.202 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : rbd-nbd-2:19.3.0-5120.g18432218.el9.x86_64 29/165 2024-09-21T03:33:22.202 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : ceph-grafana-dashboards-2:19.3.0-5120.g18432218. 30/165 2024-09-21T03:33:22.202 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : ceph-mgr-cephadm-2:19.3.0-5120.g18432218.el9.noa 31/165 2024-09-21T03:33:22.202 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : ceph-mgr-dashboard-2:19.3.0-5120.g18432218.el9.n 32/165 2024-09-21T03:33:22.202 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.3.0-5120.g184 33/165 2024-09-21T03:33:22.202 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : ceph-mgr-modules-core-2:19.3.0-5120.g18432218.el 34/165 2024-09-21T03:33:22.202 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : ceph-mgr-rook-2:19.3.0-5120.g18432218.el9.noarch 35/165 2024-09-21T03:33:22.202 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : ceph-prometheus-alerts-2:19.3.0-5120.g18432218.e 36/165 2024-09-21T03:33:22.203 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : ceph-volume-2:19.3.0-5120.g18432218.el9.noarch 37/165 2024-09-21T03:33:22.203 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : cephadm-2:19.3.0-5120.g18432218.el9.noarch 38/165 2024-09-21T03:33:22.203 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : cephfs-top-2:19.3.0-5120.g18432218.el9.noarch 39/165 2024-09-21T03:33:22.203 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : fuse-2.9.9-16.el9.x86_64 40/165 2024-09-21T03:33:22.203 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : libgfortran-11.5.0-2.el9.x86_64 41/165 2024-09-21T03:33:22.203 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : libquadmath-11.5.0-2.el9.x86_64 42/165 2024-09-21T03:33:22.203 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/165 2024-09-21T03:33:22.203 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/165 2024-09-21T03:33:22.203 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/165 2024-09-21T03:33:22.203 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/165 2024-09-21T03:33:22.203 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/165 2024-09-21T03:33:22.203 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-urllib3-1.26.5-6.el9.noarch 48/165 2024-09-21T03:33:22.204 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/165 2024-09-21T03:33:22.204 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/165 2024-09-21T03:33:22.204 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/165 2024-09-21T03:33:22.204 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : elfutils-libelf-devel-0.191-4.el9.x86_64 52/165 2024-09-21T03:33:22.204 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/165 2024-09-21T03:33:22.204 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/165 2024-09-21T03:33:22.204 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/165 2024-09-21T03:33:22.204 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/165 2024-09-21T03:33:22.204 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/165 2024-09-21T03:33:22.204 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : libnbd-1.20.2-2.el9.x86_64 58/165 2024-09-21T03:33:22.204 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 59/165 2024-09-21T03:33:22.204 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 60/165 2024-09-21T03:33:22.204 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 61/165 2024-09-21T03:33:22.205 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : libtool-2.4.6-46.el9.x86_64 62/165 2024-09-21T03:33:22.205 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : libuuid-devel-2.37.4-20.el9.x86_64 63/165 2024-09-21T03:33:22.205 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 64/165 2024-09-21T03:33:22.205 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 65/165 2024-09-21T03:33:22.205 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : lua-5.4.4-4.el9.x86_64 66/165 2024-09-21T03:33:22.205 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : openblas-0.3.26-2.el9.x86_64 67/165 2024-09-21T03:33:22.205 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : openblas-openmp-0.3.26-2.el9.x86_64 68/165 2024-09-21T03:33:22.205 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : openssl-devel-1:3.2.2-6.el9.x86_64 69/165 2024-09-21T03:33:22.205 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : postgresql-13.16-1.el9.x86_64 70/165 2024-09-21T03:33:22.205 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : postgresql-contrib-13.16-1.el9.x86_64 71/165 2024-09-21T03:33:22.205 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : postgresql-private-libs-13.16-1.el9.x86_64 72/165 2024-09-21T03:33:22.205 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : postgresql-server-13.16-1.el9.x86_64 73/165 2024-09-21T03:33:22.205 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 74/165 2024-09-21T03:33:22.206 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 75/165 2024-09-21T03:33:22.206 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 76/165 2024-09-21T03:33:22.206 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-colorama-0.4.6-3.el9.noarch 77/165 2024-09-21T03:33:22.206 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-devel-3.9.19-8.el9.x86_64 78/165 2024-09-21T03:33:22.206 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 79/165 2024-09-21T03:33:22.206 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 80/165 2024-09-21T03:33:22.206 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 81/165 2024-09-21T03:33:22.206 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 82/165 2024-09-21T03:33:22.206 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 83/165 2024-09-21T03:33:22.206 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 84/165 2024-09-21T03:33:22.206 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 85/165 2024-09-21T03:33:22.206 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 86/165 2024-09-21T03:33:22.206 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 87/165 2024-09-21T03:33:22.206 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 88/165 2024-09-21T03:33:22.207 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 89/165 2024-09-21T03:33:22.207 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 90/165 2024-09-21T03:33:22.207 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 91/165 2024-09-21T03:33:22.207 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 92/165 2024-09-21T03:33:22.207 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 93/165 2024-09-21T03:33:22.207 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 94/165 2024-09-21T03:33:22.207 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 95/165 2024-09-21T03:33:22.207 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 96/165 2024-09-21T03:33:22.207 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 97/165 2024-09-21T03:33:22.207 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 98/165 2024-09-21T03:33:22.207 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 99/165 2024-09-21T03:33:22.207 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : socat-1.7.4.1-6.el9.x86_64 100/165 2024-09-21T03:33:22.207 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 101/165 2024-09-21T03:33:22.207 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : xfsprogs-devel-6.4.0-4.el9.x86_64 102/165 2024-09-21T03:33:22.208 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 103/165 2024-09-21T03:33:22.208 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 104/165 2024-09-21T03:33:22.208 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 105/165 2024-09-21T03:33:22.208 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 106/165 2024-09-21T03:33:22.208 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 107/165 2024-09-21T03:33:22.208 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : gperftools-libs-2.9.1-3.el9.x86_64 108/165 2024-09-21T03:33:22.208 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 109/165 2024-09-21T03:33:22.208 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 110/165 2024-09-21T03:33:22.208 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 111/165 2024-09-21T03:33:22.208 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 112/165 2024-09-21T03:33:22.208 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 113/165 2024-09-21T03:33:22.208 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 114/165 2024-09-21T03:33:22.208 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 115/165 2024-09-21T03:33:22.209 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 116/165 2024-09-21T03:33:22.209 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 117/165 2024-09-21T03:33:22.209 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 118/165 2024-09-21T03:33:22.209 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 119/165 2024-09-21T03:33:22.209 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 120/165 2024-09-21T03:33:22.209 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 121/165 2024-09-21T03:33:22.209 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 122/165 2024-09-21T03:33:22.209 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-google-auth-1:2.34.0-1.el9.noarch 123/165 2024-09-21T03:33:22.209 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 124/165 2024-09-21T03:33:22.209 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 125/165 2024-09-21T03:33:22.209 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 126/165 2024-09-21T03:33:22.209 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 127/165 2024-09-21T03:33:22.209 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 128/165 2024-09-21T03:33:22.209 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 129/165 2024-09-21T03:33:22.210 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 130/165 2024-09-21T03:33:22.210 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 131/165 2024-09-21T03:33:22.210 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 132/165 2024-09-21T03:33:22.210 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 133/165 2024-09-21T03:33:22.210 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 134/165 2024-09-21T03:33:22.210 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 135/165 2024-09-21T03:33:22.210 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 136/165 2024-09-21T03:33:22.210 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 137/165 2024-09-21T03:33:22.210 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 138/165 2024-09-21T03:33:22.210 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 139/165 2024-09-21T03:33:22.210 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 140/165 2024-09-21T03:33:22.210 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 141/165 2024-09-21T03:33:22.210 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 142/165 2024-09-21T03:33:22.210 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-typing-extensions-4.12.2-2.el9.noarch 143/165 2024-09-21T03:33:22.211 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-webob-1.8.8-2.el9.noarch 144/165 2024-09-21T03:33:22.211 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 145/165 2024-09-21T03:33:22.211 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 146/165 2024-09-21T03:33:22.211 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 147/165 2024-09-21T03:33:22.211 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : re2-1:20211101-20.el9.x86_64 148/165 2024-09-21T03:33:22.211 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 149/165 2024-09-21T03:33:22.211 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : librados2-2:19.3.0-5120.g18432218.el9.x86_64 150/165 2024-09-21T03:33:22.211 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 151/165 2024-09-21T03:33:22.211 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : librbd1-2:19.3.0-5120.g18432218.el9.x86_64 152/165 2024-09-21T03:33:22.211 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 153/165 2024-09-21T03:33:22.211 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : NetworkManager-1:1.51.0-1.el9.x86_64 154/165 2024-09-21T03:33:22.211 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : NetworkManager-1:1.48.8-1.el9.x86_64 155/165 2024-09-21T03:33:22.211 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : NetworkManager-libnm-1:1.51.0-1.el9.x86_64 156/165 2024-09-21T03:33:22.212 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 157/165 2024-09-21T03:33:22.212 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : NetworkManager-team-1:1.51.0-1.el9.x86_64 158/165 2024-09-21T03:33:22.212 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : NetworkManager-team-1:1.48.8-1.el9.x86_64 159/165 2024-09-21T03:33:22.212 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : NetworkManager-tui-1:1.51.0-1.el9.x86_64 160/165 2024-09-21T03:33:22.212 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : NetworkManager-tui-1:1.48.8-1.el9.x86_64 161/165 2024-09-21T03:33:22.212 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : openssl-1:3.2.2-6.el9.x86_64 162/165 2024-09-21T03:33:22.212 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : openssl-1:3.2.2-4.el9.x86_64 163/165 2024-09-21T03:33:22.212 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : openssl-libs-1:3.2.2-6.el9.x86_64 164/165 2024-09-21T03:33:23.636 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : ceph-osd-2:19.3.0-5120.g18432218.el9.x86_64 112/165 2024-09-21T03:33:23.664 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: ceph-osd-2:19.3.0-5120.g18432218.el9.x86_64 112/165 2024-09-21T03:33:23.664 INFO:teuthology.orchestra.run.smithi161.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T03:33:23.664 INFO:teuthology.orchestra.run.smithi161.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-09-21T03:33:23.664 INFO:teuthology.orchestra.run.smithi161.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-21T03:33:23.664 INFO:teuthology.orchestra.run.smithi161.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-21T03:33:23.664 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-09-21T03:33:23.887 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : ceph-mds-2:19.3.0-5120.g18432218.el9.x86_64 113/165 2024-09-21T03:33:23.911 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: ceph-mds-2:19.3.0-5120.g18432218.el9.x86_64 113/165 2024-09-21T03:33:23.911 INFO:teuthology.orchestra.run.smithi161.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T03:33:23.911 INFO:teuthology.orchestra.run.smithi161.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-09-21T03:33:23.911 INFO:teuthology.orchestra.run.smithi161.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-21T03:33:23.912 INFO:teuthology.orchestra.run.smithi161.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-21T03:33:23.912 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-09-21T03:33:24.058 INFO:teuthology.orchestra.run.smithi039.stdout: Running scriptlet: openssl-libs-1:3.2.2-4.el9.x86_64 165/165 2024-09-21T03:33:24.058 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : ceph-2:19.3.0-5120.g18432218.el9.x86_64 1/165 2024-09-21T03:33:24.058 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : ceph-base-2:19.3.0-5120.g18432218.el9.x86_64 2/165 2024-09-21T03:33:24.058 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : ceph-common-2:19.3.0-5120.g18432218.el9.x86_64 3/165 2024-09-21T03:33:24.058 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : ceph-fuse-2:19.3.0-5120.g18432218.el9.x86_64 4/165 2024-09-21T03:33:24.058 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : ceph-immutable-object-cache-2:19.3.0-5120.g18432 5/165 2024-09-21T03:33:24.058 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : ceph-mds-2:19.3.0-5120.g18432218.el9.x86_64 6/165 2024-09-21T03:33:24.058 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : ceph-mgr-2:19.3.0-5120.g18432218.el9.x86_64 7/165 2024-09-21T03:33:24.058 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : ceph-mon-2:19.3.0-5120.g18432218.el9.x86_64 8/165 2024-09-21T03:33:24.058 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : ceph-osd-2:19.3.0-5120.g18432218.el9.x86_64 9/165 2024-09-21T03:33:24.059 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : ceph-radosgw-2:19.3.0-5120.g18432218.el9.x86_64 10/165 2024-09-21T03:33:24.059 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : ceph-selinux-2:19.3.0-5120.g18432218.el9.x86_64 11/165 2024-09-21T03:33:24.059 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : ceph-test-2:19.3.0-5120.g18432218.el9.x86_64 12/165 2024-09-21T03:33:24.059 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : cephfs-mirror-2:19.3.0-5120.g18432218.el9.x86_64 13/165 2024-09-21T03:33:24.059 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : cephfs-shell-2:19.3.0-5120.g18432218.el9.x86_64 14/165 2024-09-21T03:33:24.059 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : libcephfs-devel-2:19.3.0-5120.g18432218.el9.x86_ 15/165 2024-09-21T03:33:24.059 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : libcephfs2-2:19.3.0-5120.g18432218.el9.x86_64 16/165 2024-09-21T03:33:24.059 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : libcephsqlite-2:19.3.0-5120.g18432218.el9.x86_64 17/165 2024-09-21T03:33:24.059 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : librados-devel-2:19.3.0-5120.g18432218.el9.x86_6 18/165 2024-09-21T03:33:24.059 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : libradosstriper1-2:19.3.0-5120.g18432218.el9.x86 19/165 2024-09-21T03:33:24.059 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : librgw2-2:19.3.0-5120.g18432218.el9.x86_64 20/165 2024-09-21T03:33:24.059 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-ceph-argparse-2:19.3.0-5120.g18432218.el 21/165 2024-09-21T03:33:24.059 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-ceph-common-2:19.3.0-5120.g18432218.el9. 22/165 2024-09-21T03:33:24.059 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-cephfs-2:19.3.0-5120.g18432218.el9.x86_6 23/165 2024-09-21T03:33:24.060 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-rados-2:19.3.0-5120.g18432218.el9.x86_64 24/165 2024-09-21T03:33:24.060 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-rbd-2:19.3.0-5120.g18432218.el9.x86_64 25/165 2024-09-21T03:33:24.060 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-rgw-2:19.3.0-5120.g18432218.el9.x86_64 26/165 2024-09-21T03:33:24.060 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : rbd-fuse-2:19.3.0-5120.g18432218.el9.x86_64 27/165 2024-09-21T03:33:24.060 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : rbd-mirror-2:19.3.0-5120.g18432218.el9.x86_64 28/165 2024-09-21T03:33:24.060 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : rbd-nbd-2:19.3.0-5120.g18432218.el9.x86_64 29/165 2024-09-21T03:33:24.060 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : ceph-grafana-dashboards-2:19.3.0-5120.g18432218. 30/165 2024-09-21T03:33:24.060 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : ceph-mgr-cephadm-2:19.3.0-5120.g18432218.el9.noa 31/165 2024-09-21T03:33:24.060 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : ceph-mgr-dashboard-2:19.3.0-5120.g18432218.el9.n 32/165 2024-09-21T03:33:24.060 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.3.0-5120.g184 33/165 2024-09-21T03:33:24.060 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : ceph-mgr-modules-core-2:19.3.0-5120.g18432218.el 34/165 2024-09-21T03:33:24.060 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : ceph-mgr-rook-2:19.3.0-5120.g18432218.el9.noarch 35/165 2024-09-21T03:33:24.060 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : ceph-prometheus-alerts-2:19.3.0-5120.g18432218.e 36/165 2024-09-21T03:33:24.060 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : ceph-volume-2:19.3.0-5120.g18432218.el9.noarch 37/165 2024-09-21T03:33:24.061 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : cephadm-2:19.3.0-5120.g18432218.el9.noarch 38/165 2024-09-21T03:33:24.061 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : cephfs-top-2:19.3.0-5120.g18432218.el9.noarch 39/165 2024-09-21T03:33:24.061 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : fuse-2.9.9-16.el9.x86_64 40/165 2024-09-21T03:33:24.061 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : libgfortran-11.5.0-2.el9.x86_64 41/165 2024-09-21T03:33:24.061 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : libquadmath-11.5.0-2.el9.x86_64 42/165 2024-09-21T03:33:24.061 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/165 2024-09-21T03:33:24.061 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/165 2024-09-21T03:33:24.061 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/165 2024-09-21T03:33:24.061 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/165 2024-09-21T03:33:24.061 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/165 2024-09-21T03:33:24.061 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-urllib3-1.26.5-6.el9.noarch 48/165 2024-09-21T03:33:24.061 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/165 2024-09-21T03:33:24.061 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/165 2024-09-21T03:33:24.061 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/165 2024-09-21T03:33:24.062 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : elfutils-libelf-devel-0.191-4.el9.x86_64 52/165 2024-09-21T03:33:24.062 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/165 2024-09-21T03:33:24.062 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/165 2024-09-21T03:33:24.062 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/165 2024-09-21T03:33:24.062 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/165 2024-09-21T03:33:24.062 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/165 2024-09-21T03:33:24.062 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : libnbd-1.20.2-2.el9.x86_64 58/165 2024-09-21T03:33:24.062 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 59/165 2024-09-21T03:33:24.062 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 60/165 2024-09-21T03:33:24.062 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 61/165 2024-09-21T03:33:24.062 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : libtool-2.4.6-46.el9.x86_64 62/165 2024-09-21T03:33:24.062 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : libuuid-devel-2.37.4-20.el9.x86_64 63/165 2024-09-21T03:33:24.062 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 64/165 2024-09-21T03:33:24.062 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 65/165 2024-09-21T03:33:24.062 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : lua-5.4.4-4.el9.x86_64 66/165 2024-09-21T03:33:24.063 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : openblas-0.3.26-2.el9.x86_64 67/165 2024-09-21T03:33:24.063 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : openblas-openmp-0.3.26-2.el9.x86_64 68/165 2024-09-21T03:33:24.063 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : openssl-devel-1:3.2.2-6.el9.x86_64 69/165 2024-09-21T03:33:24.063 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : postgresql-13.16-1.el9.x86_64 70/165 2024-09-21T03:33:24.063 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : postgresql-contrib-13.16-1.el9.x86_64 71/165 2024-09-21T03:33:24.063 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : postgresql-private-libs-13.16-1.el9.x86_64 72/165 2024-09-21T03:33:24.063 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : postgresql-server-13.16-1.el9.x86_64 73/165 2024-09-21T03:33:24.063 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 74/165 2024-09-21T03:33:24.063 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 75/165 2024-09-21T03:33:24.063 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 76/165 2024-09-21T03:33:24.063 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-colorama-0.4.6-3.el9.noarch 77/165 2024-09-21T03:33:24.063 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-devel-3.9.19-8.el9.x86_64 78/165 2024-09-21T03:33:24.063 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 79/165 2024-09-21T03:33:24.063 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 80/165 2024-09-21T03:33:24.064 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 81/165 2024-09-21T03:33:24.064 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 82/165 2024-09-21T03:33:24.064 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 83/165 2024-09-21T03:33:24.064 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 84/165 2024-09-21T03:33:24.064 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 85/165 2024-09-21T03:33:24.064 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 86/165 2024-09-21T03:33:24.064 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 87/165 2024-09-21T03:33:24.064 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 88/165 2024-09-21T03:33:24.064 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 89/165 2024-09-21T03:33:24.064 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 90/165 2024-09-21T03:33:24.064 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 91/165 2024-09-21T03:33:24.064 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 92/165 2024-09-21T03:33:24.064 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 93/165 2024-09-21T03:33:24.064 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 94/165 2024-09-21T03:33:24.065 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 95/165 2024-09-21T03:33:24.065 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 96/165 2024-09-21T03:33:24.065 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 97/165 2024-09-21T03:33:24.065 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 98/165 2024-09-21T03:33:24.065 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 99/165 2024-09-21T03:33:24.065 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : socat-1.7.4.1-6.el9.x86_64 100/165 2024-09-21T03:33:24.065 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 101/165 2024-09-21T03:33:24.065 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : xfsprogs-devel-6.4.0-4.el9.x86_64 102/165 2024-09-21T03:33:24.065 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 103/165 2024-09-21T03:33:24.065 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 104/165 2024-09-21T03:33:24.065 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 105/165 2024-09-21T03:33:24.065 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 106/165 2024-09-21T03:33:24.065 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 107/165 2024-09-21T03:33:24.066 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : gperftools-libs-2.9.1-3.el9.x86_64 108/165 2024-09-21T03:33:24.066 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 109/165 2024-09-21T03:33:24.066 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 110/165 2024-09-21T03:33:24.066 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 111/165 2024-09-21T03:33:24.066 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 112/165 2024-09-21T03:33:24.066 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 113/165 2024-09-21T03:33:24.066 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 114/165 2024-09-21T03:33:24.066 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 115/165 2024-09-21T03:33:24.066 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 116/165 2024-09-21T03:33:24.066 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 117/165 2024-09-21T03:33:24.066 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 118/165 2024-09-21T03:33:24.066 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 119/165 2024-09-21T03:33:24.066 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 120/165 2024-09-21T03:33:24.066 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 121/165 2024-09-21T03:33:24.067 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 122/165 2024-09-21T03:33:24.067 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-google-auth-1:2.34.0-1.el9.noarch 123/165 2024-09-21T03:33:24.067 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 124/165 2024-09-21T03:33:24.067 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 125/165 2024-09-21T03:33:24.067 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 126/165 2024-09-21T03:33:24.067 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 127/165 2024-09-21T03:33:24.067 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 128/165 2024-09-21T03:33:24.067 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 129/165 2024-09-21T03:33:24.067 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 130/165 2024-09-21T03:33:24.067 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 131/165 2024-09-21T03:33:24.067 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 132/165 2024-09-21T03:33:24.067 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 133/165 2024-09-21T03:33:24.067 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 134/165 2024-09-21T03:33:24.067 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 135/165 2024-09-21T03:33:24.067 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 136/165 2024-09-21T03:33:24.068 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 137/165 2024-09-21T03:33:24.068 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 138/165 2024-09-21T03:33:24.068 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 139/165 2024-09-21T03:33:24.068 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 140/165 2024-09-21T03:33:24.068 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 141/165 2024-09-21T03:33:24.068 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 142/165 2024-09-21T03:33:24.068 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-typing-extensions-4.12.2-2.el9.noarch 143/165 2024-09-21T03:33:24.068 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-webob-1.8.8-2.el9.noarch 144/165 2024-09-21T03:33:24.068 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 145/165 2024-09-21T03:33:24.068 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 146/165 2024-09-21T03:33:24.068 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 147/165 2024-09-21T03:33:24.068 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : re2-1:20211101-20.el9.x86_64 148/165 2024-09-21T03:33:24.068 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 149/165 2024-09-21T03:33:24.068 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : librados2-2:19.3.0-5120.g18432218.el9.x86_64 150/165 2024-09-21T03:33:24.068 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 151/165 2024-09-21T03:33:24.069 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : librbd1-2:19.3.0-5120.g18432218.el9.x86_64 152/165 2024-09-21T03:33:24.069 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 153/165 2024-09-21T03:33:24.069 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : NetworkManager-1:1.51.0-1.el9.x86_64 154/165 2024-09-21T03:33:24.069 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : NetworkManager-1:1.48.8-1.el9.x86_64 155/165 2024-09-21T03:33:24.069 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : NetworkManager-libnm-1:1.51.0-1.el9.x86_64 156/165 2024-09-21T03:33:24.069 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 157/165 2024-09-21T03:33:24.069 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : NetworkManager-team-1:1.51.0-1.el9.x86_64 158/165 2024-09-21T03:33:24.069 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : NetworkManager-team-1:1.48.8-1.el9.x86_64 159/165 2024-09-21T03:33:24.069 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : NetworkManager-tui-1:1.51.0-1.el9.x86_64 160/165 2024-09-21T03:33:24.069 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : NetworkManager-tui-1:1.48.8-1.el9.x86_64 161/165 2024-09-21T03:33:24.069 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : openssl-1:3.2.2-6.el9.x86_64 162/165 2024-09-21T03:33:24.069 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : openssl-1:3.2.2-4.el9.x86_64 163/165 2024-09-21T03:33:24.069 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : openssl-libs-1:3.2.2-6.el9.x86_64 164/165 2024-09-21T03:33:24.371 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : ceph-mon-2:19.3.0-5120.g18432218.el9.x86_64 114/165 2024-09-21T03:33:24.395 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: ceph-mon-2:19.3.0-5120.g18432218.el9.x86_64 114/165 2024-09-21T03:33:24.395 INFO:teuthology.orchestra.run.smithi161.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T03:33:24.395 INFO:teuthology.orchestra.run.smithi161.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-09-21T03:33:24.395 INFO:teuthology.orchestra.run.smithi161.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-21T03:33:24.395 INFO:teuthology.orchestra.run.smithi161.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-21T03:33:24.395 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-09-21T03:33:24.509 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : automake-1.16.2-8.el9.noarch 115/165 2024-09-21T03:33:24.566 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-ply-3.11-14.el9.noarch 116/165 2024-09-21T03:33:24.621 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 117/165 2024-09-21T03:33:24.816 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 118/165 2024-09-21T03:33:24.863 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 119/165 2024-09-21T03:33:24.940 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 120/165 2024-09-21T03:33:25.000 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : openssl-libs-1:3.2.2-4.el9.x86_64 165/165 2024-09-21T03:33:25.000 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-09-21T03:33:25.000 INFO:teuthology.orchestra.run.smithi150.stdout:Upgraded: 2024-09-21T03:33:25.000 INFO:teuthology.orchestra.run.smithi150.stdout: NetworkManager-1:1.51.0-1.el9.x86_64 2024-09-21T03:33:25.000 INFO:teuthology.orchestra.run.smithi150.stdout: NetworkManager-libnm-1:1.51.0-1.el9.x86_64 2024-09-21T03:33:25.000 INFO:teuthology.orchestra.run.smithi150.stdout: NetworkManager-team-1:1.51.0-1.el9.x86_64 2024-09-21T03:33:25.000 INFO:teuthology.orchestra.run.smithi150.stdout: NetworkManager-tui-1:1.51.0-1.el9.x86_64 2024-09-21T03:33:25.000 INFO:teuthology.orchestra.run.smithi150.stdout: librados2-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:33:25.000 INFO:teuthology.orchestra.run.smithi150.stdout: librbd1-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:33:25.000 INFO:teuthology.orchestra.run.smithi150.stdout: openssl-1:3.2.2-6.el9.x86_64 2024-09-21T03:33:25.000 INFO:teuthology.orchestra.run.smithi150.stdout: openssl-libs-1:3.2.2-6.el9.x86_64 2024-09-21T03:33:25.001 INFO:teuthology.orchestra.run.smithi150.stdout:Installed: 2024-09-21T03:33:25.001 INFO:teuthology.orchestra.run.smithi150.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-09-21T03:33:25.001 INFO:teuthology.orchestra.run.smithi150.stdout: automake-1.16.2-8.el9.noarch 2024-09-21T03:33:25.001 INFO:teuthology.orchestra.run.smithi150.stdout: bison-3.7.4-5.el9.x86_64 2024-09-21T03:33:25.001 INFO:teuthology.orchestra.run.smithi150.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-09-21T03:33:25.001 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:33:25.001 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-base-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:33:25.001 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-common-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:33:25.001 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-fuse-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:33:25.001 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-grafana-dashboards-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T03:33:25.002 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-immutable-object-cache-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:33:25.002 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-mds-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:33:25.002 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-mgr-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:33:25.002 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-mgr-cephadm-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T03:33:25.002 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-mgr-dashboard-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T03:33:25.002 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-mgr-diskprediction-local-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T03:33:25.002 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-mgr-modules-core-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T03:33:25.002 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-mgr-rook-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T03:33:25.002 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-mon-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:33:25.002 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-osd-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:33:25.002 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-prometheus-alerts-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T03:33:25.002 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-radosgw-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:33:25.002 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-selinux-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:33:25.003 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-test-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:33:25.003 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-volume-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T03:33:25.003 INFO:teuthology.orchestra.run.smithi150.stdout: cephadm-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T03:33:25.003 INFO:teuthology.orchestra.run.smithi150.stdout: cephfs-mirror-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:33:25.003 INFO:teuthology.orchestra.run.smithi150.stdout: cephfs-shell-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:33:25.003 INFO:teuthology.orchestra.run.smithi150.stdout: cephfs-top-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T03:33:25.003 INFO:teuthology.orchestra.run.smithi150.stdout: elfutils-libelf-devel-0.191-4.el9.x86_64 2024-09-21T03:33:25.003 INFO:teuthology.orchestra.run.smithi150.stdout: flex-2.6.4-9.el9.x86_64 2024-09-21T03:33:25.003 INFO:teuthology.orchestra.run.smithi150.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-09-21T03:33:25.004 INFO:teuthology.orchestra.run.smithi150.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-09-21T03:33:25.004 INFO:teuthology.orchestra.run.smithi150.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-09-21T03:33:25.004 INFO:teuthology.orchestra.run.smithi150.stdout: fuse-2.9.9-16.el9.x86_64 2024-09-21T03:33:25.004 INFO:teuthology.orchestra.run.smithi150.stdout: gperftools-libs-2.9.1-3.el9.x86_64 2024-09-21T03:33:25.004 INFO:teuthology.orchestra.run.smithi150.stdout: grpc-data-1.46.7-10.el9.noarch 2024-09-21T03:33:25.004 INFO:teuthology.orchestra.run.smithi150.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-09-21T03:33:25.004 INFO:teuthology.orchestra.run.smithi150.stdout: libarrow-9.0.0-11.el9.x86_64 2024-09-21T03:33:25.004 INFO:teuthology.orchestra.run.smithi150.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-09-21T03:33:25.004 INFO:teuthology.orchestra.run.smithi150.stdout: libcephfs-devel-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:33:25.004 INFO:teuthology.orchestra.run.smithi150.stdout: libcephfs2-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:33:25.004 INFO:teuthology.orchestra.run.smithi150.stdout: libcephsqlite-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:33:25.004 INFO:teuthology.orchestra.run.smithi150.stdout: libgfortran-11.5.0-2.el9.x86_64 2024-09-21T03:33:25.004 INFO:teuthology.orchestra.run.smithi150.stdout: libnbd-1.20.2-2.el9.x86_64 2024-09-21T03:33:25.004 INFO:teuthology.orchestra.run.smithi150.stdout: liboath-2.6.7-2.el9.x86_64 2024-09-21T03:33:25.004 INFO:teuthology.orchestra.run.smithi150.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-09-21T03:33:25.004 INFO:teuthology.orchestra.run.smithi150.stdout: libquadmath-11.5.0-2.el9.x86_64 2024-09-21T03:33:25.005 INFO:teuthology.orchestra.run.smithi150.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-09-21T03:33:25.005 INFO:teuthology.orchestra.run.smithi150.stdout: librados-devel-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:33:25.005 INFO:teuthology.orchestra.run.smithi150.stdout: libradosstriper1-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:33:25.005 INFO:teuthology.orchestra.run.smithi150.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-09-21T03:33:25.005 INFO:teuthology.orchestra.run.smithi150.stdout: librgw2-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:33:25.005 INFO:teuthology.orchestra.run.smithi150.stdout: libtool-2.4.6-46.el9.x86_64 2024-09-21T03:33:25.005 INFO:teuthology.orchestra.run.smithi150.stdout: libunwind-1.6.2-1.el9.x86_64 2024-09-21T03:33:25.005 INFO:teuthology.orchestra.run.smithi150.stdout: libuuid-devel-2.37.4-20.el9.x86_64 2024-09-21T03:33:25.005 INFO:teuthology.orchestra.run.smithi150.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-09-21T03:33:25.005 INFO:teuthology.orchestra.run.smithi150.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-09-21T03:33:25.005 INFO:teuthology.orchestra.run.smithi150.stdout: lua-5.4.4-4.el9.x86_64 2024-09-21T03:33:25.005 INFO:teuthology.orchestra.run.smithi150.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-09-21T03:33:25.005 INFO:teuthology.orchestra.run.smithi150.stdout: luarocks-3.9.2-5.el9.noarch 2024-09-21T03:33:25.005 INFO:teuthology.orchestra.run.smithi150.stdout: openblas-0.3.26-2.el9.x86_64 2024-09-21T03:33:25.005 INFO:teuthology.orchestra.run.smithi150.stdout: openblas-openmp-0.3.26-2.el9.x86_64 2024-09-21T03:33:25.005 INFO:teuthology.orchestra.run.smithi150.stdout: openssl-devel-1:3.2.2-6.el9.x86_64 2024-09-21T03:33:25.006 INFO:teuthology.orchestra.run.smithi150.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-09-21T03:33:25.006 INFO:teuthology.orchestra.run.smithi150.stdout: postgresql-13.16-1.el9.x86_64 2024-09-21T03:33:25.006 INFO:teuthology.orchestra.run.smithi150.stdout: postgresql-contrib-13.16-1.el9.x86_64 2024-09-21T03:33:25.006 INFO:teuthology.orchestra.run.smithi150.stdout: postgresql-private-libs-13.16-1.el9.x86_64 2024-09-21T03:33:25.006 INFO:teuthology.orchestra.run.smithi150.stdout: postgresql-server-13.16-1.el9.x86_64 2024-09-21T03:33:25.006 INFO:teuthology.orchestra.run.smithi150.stdout: protobuf-3.14.0-13.el9.x86_64 2024-09-21T03:33:25.006 INFO:teuthology.orchestra.run.smithi150.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-09-21T03:33:25.006 INFO:teuthology.orchestra.run.smithi150.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-09-21T03:33:25.006 INFO:teuthology.orchestra.run.smithi150.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-09-21T03:33:25.006 INFO:teuthology.orchestra.run.smithi150.stdout: python3-babel-2.9.1-2.el9.noarch 2024-09-21T03:33:25.006 INFO:teuthology.orchestra.run.smithi150.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-09-21T03:33:25.006 INFO:teuthology.orchestra.run.smithi150.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-09-21T03:33:25.006 INFO:teuthology.orchestra.run.smithi150.stdout: python3-ceph-argparse-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:33:25.006 INFO:teuthology.orchestra.run.smithi150.stdout: python3-ceph-common-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:33:25.006 INFO:teuthology.orchestra.run.smithi150.stdout: python3-cephfs-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:33:25.007 INFO:teuthology.orchestra.run.smithi150.stdout: python3-certifi-2023.05.07-4.el9.noarch 2024-09-21T03:33:25.007 INFO:teuthology.orchestra.run.smithi150.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-09-21T03:33:25.007 INFO:teuthology.orchestra.run.smithi150.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-09-21T03:33:25.007 INFO:teuthology.orchestra.run.smithi150.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-09-21T03:33:25.007 INFO:teuthology.orchestra.run.smithi150.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-09-21T03:33:25.007 INFO:teuthology.orchestra.run.smithi150.stdout: python3-colorama-0.4.6-3.el9.noarch 2024-09-21T03:33:25.007 INFO:teuthology.orchestra.run.smithi150.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-09-21T03:33:25.007 INFO:teuthology.orchestra.run.smithi150.stdout: python3-devel-3.9.19-8.el9.x86_64 2024-09-21T03:33:25.007 INFO:teuthology.orchestra.run.smithi150.stdout: python3-google-auth-1:2.34.0-1.el9.noarch 2024-09-21T03:33:25.007 INFO:teuthology.orchestra.run.smithi150.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-09-21T03:33:25.007 INFO:teuthology.orchestra.run.smithi150.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-09-21T03:33:25.007 INFO:teuthology.orchestra.run.smithi150.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-09-21T03:33:25.007 INFO:teuthology.orchestra.run.smithi150.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-09-21T03:33:25.008 INFO:teuthology.orchestra.run.smithi150.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-09-21T03:33:25.008 INFO:teuthology.orchestra.run.smithi150.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-09-21T03:33:25.008 INFO:teuthology.orchestra.run.smithi150.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-09-21T03:33:25.008 INFO:teuthology.orchestra.run.smithi150.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-09-21T03:33:25.008 INFO:teuthology.orchestra.run.smithi150.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-09-21T03:33:25.008 INFO:teuthology.orchestra.run.smithi150.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-09-21T03:33:25.008 INFO:teuthology.orchestra.run.smithi150.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-09-21T03:33:25.008 INFO:teuthology.orchestra.run.smithi150.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-09-21T03:33:25.008 INFO:teuthology.orchestra.run.smithi150.stdout: python3-mako-1.1.4-6.el9.noarch 2024-09-21T03:33:25.008 INFO:teuthology.orchestra.run.smithi150.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-09-21T03:33:25.008 INFO:teuthology.orchestra.run.smithi150.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-09-21T03:33:25.008 INFO:teuthology.orchestra.run.smithi150.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-09-21T03:33:25.008 INFO:teuthology.orchestra.run.smithi150.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-09-21T03:33:25.008 INFO:teuthology.orchestra.run.smithi150.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-09-21T03:33:25.009 INFO:teuthology.orchestra.run.smithi150.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-09-21T03:33:25.009 INFO:teuthology.orchestra.run.smithi150.stdout: python3-packaging-20.9-5.el9.noarch 2024-09-21T03:33:25.009 INFO:teuthology.orchestra.run.smithi150.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-09-21T03:33:25.009 INFO:teuthology.orchestra.run.smithi150.stdout: python3-ply-3.11-14.el9.noarch 2024-09-21T03:33:25.009 INFO:teuthology.orchestra.run.smithi150.stdout: python3-portend-3.1.0-2.el9.noarch 2024-09-21T03:33:25.009 INFO:teuthology.orchestra.run.smithi150.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-09-21T03:33:25.009 INFO:teuthology.orchestra.run.smithi150.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-09-21T03:33:25.009 INFO:teuthology.orchestra.run.smithi150.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-09-21T03:33:25.009 INFO:teuthology.orchestra.run.smithi150.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-09-21T03:33:25.009 INFO:teuthology.orchestra.run.smithi150.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-09-21T03:33:25.009 INFO:teuthology.orchestra.run.smithi150.stdout: python3-pycparser-2.20-6.el9.noarch 2024-09-21T03:33:25.009 INFO:teuthology.orchestra.run.smithi150.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-09-21T03:33:25.009 INFO:teuthology.orchestra.run.smithi150.stdout: python3-pytz-2021.1-5.el9.noarch 2024-09-21T03:33:25.009 INFO:teuthology.orchestra.run.smithi150.stdout: python3-rados-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:33:25.009 INFO:teuthology.orchestra.run.smithi150.stdout: python3-rbd-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:33:25.010 INFO:teuthology.orchestra.run.smithi150.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-09-21T03:33:25.010 INFO:teuthology.orchestra.run.smithi150.stdout: python3-requests-2.25.1-8.el9.noarch 2024-09-21T03:33:25.010 INFO:teuthology.orchestra.run.smithi150.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-09-21T03:33:25.010 INFO:teuthology.orchestra.run.smithi150.stdout: python3-rgw-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:33:25.010 INFO:teuthology.orchestra.run.smithi150.stdout: python3-routes-2.5.1-5.el9.noarch 2024-09-21T03:33:25.010 INFO:teuthology.orchestra.run.smithi150.stdout: python3-rsa-4.9-2.el9.noarch 2024-09-21T03:33:25.010 INFO:teuthology.orchestra.run.smithi150.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-09-21T03:33:25.010 INFO:teuthology.orchestra.run.smithi150.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-09-21T03:33:25.010 INFO:teuthology.orchestra.run.smithi150.stdout: python3-toml-0.10.2-6.el9.noarch 2024-09-21T03:33:25.010 INFO:teuthology.orchestra.run.smithi150.stdout: python3-typing-extensions-4.12.2-2.el9.noarch 2024-09-21T03:33:25.010 INFO:teuthology.orchestra.run.smithi150.stdout: python3-urllib3-1.26.5-6.el9.noarch 2024-09-21T03:33:25.010 INFO:teuthology.orchestra.run.smithi150.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-09-21T03:33:25.010 INFO:teuthology.orchestra.run.smithi150.stdout: python3-webob-1.8.8-2.el9.noarch 2024-09-21T03:33:25.010 INFO:teuthology.orchestra.run.smithi150.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-09-21T03:33:25.010 INFO:teuthology.orchestra.run.smithi150.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-09-21T03:33:25.011 INFO:teuthology.orchestra.run.smithi150.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-09-21T03:33:25.011 INFO:teuthology.orchestra.run.smithi150.stdout: qatlib-24.02.0-1.el9.x86_64 2024-09-21T03:33:25.011 INFO:teuthology.orchestra.run.smithi150.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-09-21T03:33:25.011 INFO:teuthology.orchestra.run.smithi150.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-09-21T03:33:25.011 INFO:teuthology.orchestra.run.smithi150.stdout: rbd-fuse-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:33:25.011 INFO:teuthology.orchestra.run.smithi150.stdout: rbd-mirror-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:33:25.011 INFO:teuthology.orchestra.run.smithi150.stdout: rbd-nbd-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:33:25.011 INFO:teuthology.orchestra.run.smithi150.stdout: re2-1:20211101-20.el9.x86_64 2024-09-21T03:33:25.011 INFO:teuthology.orchestra.run.smithi150.stdout: socat-1.7.4.1-6.el9.x86_64 2024-09-21T03:33:25.011 INFO:teuthology.orchestra.run.smithi150.stdout: thrift-0.15.0-2.el9.x86_64 2024-09-21T03:33:25.011 INFO:teuthology.orchestra.run.smithi150.stdout: uuid-1.6.2-55.el9.x86_64 2024-09-21T03:33:25.011 INFO:teuthology.orchestra.run.smithi150.stdout: xfsprogs-devel-6.4.0-4.el9.x86_64 2024-09-21T03:33:25.011 INFO:teuthology.orchestra.run.smithi150.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-09-21T03:33:25.012 INFO:teuthology.orchestra.run.smithi150.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-09-21T03:33:25.012 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-09-21T03:33:25.012 INFO:teuthology.orchestra.run.smithi150.stdout:Complete! 2024-09-21T03:33:25.029 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 121/165 2024-09-21T03:33:25.169 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 122/165 2024-09-21T03:33:25.221 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 123/165 2024-09-21T03:33:25.253 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 124/165 2024-09-21T03:33:25.281 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : fuse-2.9.9-16.el9.x86_64 125/165 2024-09-21T03:33:25.486 DEBUG:teuthology.parallel:result is None 2024-09-21T03:33:25.749 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: cephadm-2:19.3.0-5120.g18432218.el9.noarch 126/165 2024-09-21T03:33:25.788 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : cephadm-2:19.3.0-5120.g18432218.el9.noarch 126/165 2024-09-21T03:33:25.829 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : ceph-prometheus-alerts-2:19.3.0-5120.g18432218.e 127/165 2024-09-21T03:33:25.894 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : ceph-grafana-dashboards-2:19.3.0-5120.g18432218. 128/165 2024-09-21T03:33:25.929 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : ceph-mgr-cephadm-2:19.3.0-5120.g18432218.el9.noa 129/165 2024-09-21T03:33:26.513 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : openssl-libs-1:3.2.2-4.el9.x86_64 165/165 2024-09-21T03:33:26.514 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T03:33:26.514 INFO:teuthology.orchestra.run.smithi039.stdout:Upgraded: 2024-09-21T03:33:26.514 INFO:teuthology.orchestra.run.smithi039.stdout: NetworkManager-1:1.51.0-1.el9.x86_64 2024-09-21T03:33:26.514 INFO:teuthology.orchestra.run.smithi039.stdout: NetworkManager-libnm-1:1.51.0-1.el9.x86_64 2024-09-21T03:33:26.514 INFO:teuthology.orchestra.run.smithi039.stdout: NetworkManager-team-1:1.51.0-1.el9.x86_64 2024-09-21T03:33:26.514 INFO:teuthology.orchestra.run.smithi039.stdout: NetworkManager-tui-1:1.51.0-1.el9.x86_64 2024-09-21T03:33:26.514 INFO:teuthology.orchestra.run.smithi039.stdout: librados2-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:33:26.514 INFO:teuthology.orchestra.run.smithi039.stdout: librbd1-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:33:26.514 INFO:teuthology.orchestra.run.smithi039.stdout: openssl-1:3.2.2-6.el9.x86_64 2024-09-21T03:33:26.514 INFO:teuthology.orchestra.run.smithi039.stdout: openssl-libs-1:3.2.2-6.el9.x86_64 2024-09-21T03:33:26.515 INFO:teuthology.orchestra.run.smithi039.stdout:Installed: 2024-09-21T03:33:26.515 INFO:teuthology.orchestra.run.smithi039.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-09-21T03:33:26.515 INFO:teuthology.orchestra.run.smithi039.stdout: automake-1.16.2-8.el9.noarch 2024-09-21T03:33:26.515 INFO:teuthology.orchestra.run.smithi039.stdout: bison-3.7.4-5.el9.x86_64 2024-09-21T03:33:26.515 INFO:teuthology.orchestra.run.smithi039.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-09-21T03:33:26.515 INFO:teuthology.orchestra.run.smithi039.stdout: ceph-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:33:26.515 INFO:teuthology.orchestra.run.smithi039.stdout: ceph-base-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:33:26.515 INFO:teuthology.orchestra.run.smithi039.stdout: ceph-common-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:33:26.515 INFO:teuthology.orchestra.run.smithi039.stdout: ceph-fuse-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:33:26.515 INFO:teuthology.orchestra.run.smithi039.stdout: ceph-grafana-dashboards-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T03:33:26.515 INFO:teuthology.orchestra.run.smithi039.stdout: ceph-immutable-object-cache-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:33:26.515 INFO:teuthology.orchestra.run.smithi039.stdout: ceph-mds-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:33:26.515 INFO:teuthology.orchestra.run.smithi039.stdout: ceph-mgr-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:33:26.515 INFO:teuthology.orchestra.run.smithi039.stdout: ceph-mgr-cephadm-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T03:33:26.516 INFO:teuthology.orchestra.run.smithi039.stdout: ceph-mgr-dashboard-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T03:33:26.516 INFO:teuthology.orchestra.run.smithi039.stdout: ceph-mgr-diskprediction-local-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T03:33:26.516 INFO:teuthology.orchestra.run.smithi039.stdout: ceph-mgr-modules-core-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T03:33:26.516 INFO:teuthology.orchestra.run.smithi039.stdout: ceph-mgr-rook-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T03:33:26.516 INFO:teuthology.orchestra.run.smithi039.stdout: ceph-mon-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:33:26.516 INFO:teuthology.orchestra.run.smithi039.stdout: ceph-osd-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:33:26.516 INFO:teuthology.orchestra.run.smithi039.stdout: ceph-prometheus-alerts-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T03:33:26.516 INFO:teuthology.orchestra.run.smithi039.stdout: ceph-radosgw-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:33:26.516 INFO:teuthology.orchestra.run.smithi039.stdout: ceph-selinux-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:33:26.516 INFO:teuthology.orchestra.run.smithi039.stdout: ceph-test-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:33:26.516 INFO:teuthology.orchestra.run.smithi039.stdout: ceph-volume-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T03:33:26.516 INFO:teuthology.orchestra.run.smithi039.stdout: cephadm-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T03:33:26.516 INFO:teuthology.orchestra.run.smithi039.stdout: cephfs-mirror-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:33:26.516 INFO:teuthology.orchestra.run.smithi039.stdout: cephfs-shell-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:33:26.517 INFO:teuthology.orchestra.run.smithi039.stdout: cephfs-top-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T03:33:26.517 INFO:teuthology.orchestra.run.smithi039.stdout: elfutils-libelf-devel-0.191-4.el9.x86_64 2024-09-21T03:33:26.517 INFO:teuthology.orchestra.run.smithi039.stdout: flex-2.6.4-9.el9.x86_64 2024-09-21T03:33:26.517 INFO:teuthology.orchestra.run.smithi039.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-09-21T03:33:26.517 INFO:teuthology.orchestra.run.smithi039.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-09-21T03:33:26.517 INFO:teuthology.orchestra.run.smithi039.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-09-21T03:33:26.517 INFO:teuthology.orchestra.run.smithi039.stdout: fuse-2.9.9-16.el9.x86_64 2024-09-21T03:33:26.517 INFO:teuthology.orchestra.run.smithi039.stdout: gperftools-libs-2.9.1-3.el9.x86_64 2024-09-21T03:33:26.517 INFO:teuthology.orchestra.run.smithi039.stdout: grpc-data-1.46.7-10.el9.noarch 2024-09-21T03:33:26.517 INFO:teuthology.orchestra.run.smithi039.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-09-21T03:33:26.517 INFO:teuthology.orchestra.run.smithi039.stdout: libarrow-9.0.0-11.el9.x86_64 2024-09-21T03:33:26.517 INFO:teuthology.orchestra.run.smithi039.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-09-21T03:33:26.517 INFO:teuthology.orchestra.run.smithi039.stdout: libcephfs-devel-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:33:26.517 INFO:teuthology.orchestra.run.smithi039.stdout: libcephfs2-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:33:26.518 INFO:teuthology.orchestra.run.smithi039.stdout: libcephsqlite-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:33:26.518 INFO:teuthology.orchestra.run.smithi039.stdout: libgfortran-11.5.0-2.el9.x86_64 2024-09-21T03:33:26.518 INFO:teuthology.orchestra.run.smithi039.stdout: libnbd-1.20.2-2.el9.x86_64 2024-09-21T03:33:26.518 INFO:teuthology.orchestra.run.smithi039.stdout: liboath-2.6.7-2.el9.x86_64 2024-09-21T03:33:26.518 INFO:teuthology.orchestra.run.smithi039.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-09-21T03:33:26.518 INFO:teuthology.orchestra.run.smithi039.stdout: libquadmath-11.5.0-2.el9.x86_64 2024-09-21T03:33:26.518 INFO:teuthology.orchestra.run.smithi039.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-09-21T03:33:26.518 INFO:teuthology.orchestra.run.smithi039.stdout: librados-devel-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:33:26.518 INFO:teuthology.orchestra.run.smithi039.stdout: libradosstriper1-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:33:26.518 INFO:teuthology.orchestra.run.smithi039.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-09-21T03:33:26.518 INFO:teuthology.orchestra.run.smithi039.stdout: librgw2-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:33:26.518 INFO:teuthology.orchestra.run.smithi039.stdout: libtool-2.4.6-46.el9.x86_64 2024-09-21T03:33:26.518 INFO:teuthology.orchestra.run.smithi039.stdout: libunwind-1.6.2-1.el9.x86_64 2024-09-21T03:33:26.518 INFO:teuthology.orchestra.run.smithi039.stdout: libuuid-devel-2.37.4-20.el9.x86_64 2024-09-21T03:33:26.519 INFO:teuthology.orchestra.run.smithi039.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-09-21T03:33:26.519 INFO:teuthology.orchestra.run.smithi039.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-09-21T03:33:26.519 INFO:teuthology.orchestra.run.smithi039.stdout: lua-5.4.4-4.el9.x86_64 2024-09-21T03:33:26.519 INFO:teuthology.orchestra.run.smithi039.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-09-21T03:33:26.519 INFO:teuthology.orchestra.run.smithi039.stdout: luarocks-3.9.2-5.el9.noarch 2024-09-21T03:33:26.519 INFO:teuthology.orchestra.run.smithi039.stdout: openblas-0.3.26-2.el9.x86_64 2024-09-21T03:33:26.519 INFO:teuthology.orchestra.run.smithi039.stdout: openblas-openmp-0.3.26-2.el9.x86_64 2024-09-21T03:33:26.519 INFO:teuthology.orchestra.run.smithi039.stdout: openssl-devel-1:3.2.2-6.el9.x86_64 2024-09-21T03:33:26.519 INFO:teuthology.orchestra.run.smithi039.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-09-21T03:33:26.519 INFO:teuthology.orchestra.run.smithi039.stdout: postgresql-13.16-1.el9.x86_64 2024-09-21T03:33:26.519 INFO:teuthology.orchestra.run.smithi039.stdout: postgresql-contrib-13.16-1.el9.x86_64 2024-09-21T03:33:26.519 INFO:teuthology.orchestra.run.smithi039.stdout: postgresql-private-libs-13.16-1.el9.x86_64 2024-09-21T03:33:26.519 INFO:teuthology.orchestra.run.smithi039.stdout: postgresql-server-13.16-1.el9.x86_64 2024-09-21T03:33:26.519 INFO:teuthology.orchestra.run.smithi039.stdout: protobuf-3.14.0-13.el9.x86_64 2024-09-21T03:33:26.520 INFO:teuthology.orchestra.run.smithi039.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-09-21T03:33:26.520 INFO:teuthology.orchestra.run.smithi039.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-09-21T03:33:26.520 INFO:teuthology.orchestra.run.smithi039.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-09-21T03:33:26.520 INFO:teuthology.orchestra.run.smithi039.stdout: python3-babel-2.9.1-2.el9.noarch 2024-09-21T03:33:26.520 INFO:teuthology.orchestra.run.smithi039.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-09-21T03:33:26.520 INFO:teuthology.orchestra.run.smithi039.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-09-21T03:33:26.520 INFO:teuthology.orchestra.run.smithi039.stdout: python3-ceph-argparse-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:33:26.520 INFO:teuthology.orchestra.run.smithi039.stdout: python3-ceph-common-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:33:26.520 INFO:teuthology.orchestra.run.smithi039.stdout: python3-cephfs-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:33:26.520 INFO:teuthology.orchestra.run.smithi039.stdout: python3-certifi-2023.05.07-4.el9.noarch 2024-09-21T03:33:26.520 INFO:teuthology.orchestra.run.smithi039.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-09-21T03:33:26.520 INFO:teuthology.orchestra.run.smithi039.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-09-21T03:33:26.520 INFO:teuthology.orchestra.run.smithi039.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-09-21T03:33:26.520 INFO:teuthology.orchestra.run.smithi039.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-09-21T03:33:26.521 INFO:teuthology.orchestra.run.smithi039.stdout: python3-colorama-0.4.6-3.el9.noarch 2024-09-21T03:33:26.521 INFO:teuthology.orchestra.run.smithi039.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-09-21T03:33:26.521 INFO:teuthology.orchestra.run.smithi039.stdout: python3-devel-3.9.19-8.el9.x86_64 2024-09-21T03:33:26.521 INFO:teuthology.orchestra.run.smithi039.stdout: python3-google-auth-1:2.34.0-1.el9.noarch 2024-09-21T03:33:26.521 INFO:teuthology.orchestra.run.smithi039.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-09-21T03:33:26.521 INFO:teuthology.orchestra.run.smithi039.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-09-21T03:33:26.521 INFO:teuthology.orchestra.run.smithi039.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-09-21T03:33:26.521 INFO:teuthology.orchestra.run.smithi039.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-09-21T03:33:26.521 INFO:teuthology.orchestra.run.smithi039.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-09-21T03:33:26.521 INFO:teuthology.orchestra.run.smithi039.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-09-21T03:33:26.521 INFO:teuthology.orchestra.run.smithi039.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-09-21T03:33:26.521 INFO:teuthology.orchestra.run.smithi039.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-09-21T03:33:26.521 INFO:teuthology.orchestra.run.smithi039.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-09-21T03:33:26.521 INFO:teuthology.orchestra.run.smithi039.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-09-21T03:33:26.522 INFO:teuthology.orchestra.run.smithi039.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-09-21T03:33:26.522 INFO:teuthology.orchestra.run.smithi039.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-09-21T03:33:26.522 INFO:teuthology.orchestra.run.smithi039.stdout: python3-mako-1.1.4-6.el9.noarch 2024-09-21T03:33:26.522 INFO:teuthology.orchestra.run.smithi039.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-09-21T03:33:26.522 INFO:teuthology.orchestra.run.smithi039.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-09-21T03:33:26.522 INFO:teuthology.orchestra.run.smithi039.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-09-21T03:33:26.522 INFO:teuthology.orchestra.run.smithi039.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-09-21T03:33:26.522 INFO:teuthology.orchestra.run.smithi039.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-09-21T03:33:26.522 INFO:teuthology.orchestra.run.smithi039.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-09-21T03:33:26.522 INFO:teuthology.orchestra.run.smithi039.stdout: python3-packaging-20.9-5.el9.noarch 2024-09-21T03:33:26.522 INFO:teuthology.orchestra.run.smithi039.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-09-21T03:33:26.522 INFO:teuthology.orchestra.run.smithi039.stdout: python3-ply-3.11-14.el9.noarch 2024-09-21T03:33:26.522 INFO:teuthology.orchestra.run.smithi039.stdout: python3-portend-3.1.0-2.el9.noarch 2024-09-21T03:33:26.522 INFO:teuthology.orchestra.run.smithi039.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-09-21T03:33:26.523 INFO:teuthology.orchestra.run.smithi039.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-09-21T03:33:26.523 INFO:teuthology.orchestra.run.smithi039.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-09-21T03:33:26.523 INFO:teuthology.orchestra.run.smithi039.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-09-21T03:33:26.523 INFO:teuthology.orchestra.run.smithi039.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-09-21T03:33:26.523 INFO:teuthology.orchestra.run.smithi039.stdout: python3-pycparser-2.20-6.el9.noarch 2024-09-21T03:33:26.523 INFO:teuthology.orchestra.run.smithi039.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-09-21T03:33:26.523 INFO:teuthology.orchestra.run.smithi039.stdout: python3-pytz-2021.1-5.el9.noarch 2024-09-21T03:33:26.523 INFO:teuthology.orchestra.run.smithi039.stdout: python3-rados-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:33:26.523 INFO:teuthology.orchestra.run.smithi039.stdout: python3-rbd-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:33:26.523 INFO:teuthology.orchestra.run.smithi039.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-09-21T03:33:26.523 INFO:teuthology.orchestra.run.smithi039.stdout: python3-requests-2.25.1-8.el9.noarch 2024-09-21T03:33:26.523 INFO:teuthology.orchestra.run.smithi039.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-09-21T03:33:26.523 INFO:teuthology.orchestra.run.smithi039.stdout: python3-rgw-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:33:26.523 INFO:teuthology.orchestra.run.smithi039.stdout: python3-routes-2.5.1-5.el9.noarch 2024-09-21T03:33:26.524 INFO:teuthology.orchestra.run.smithi039.stdout: python3-rsa-4.9-2.el9.noarch 2024-09-21T03:33:26.524 INFO:teuthology.orchestra.run.smithi039.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-09-21T03:33:26.524 INFO:teuthology.orchestra.run.smithi039.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-09-21T03:33:26.524 INFO:teuthology.orchestra.run.smithi039.stdout: python3-toml-0.10.2-6.el9.noarch 2024-09-21T03:33:26.524 INFO:teuthology.orchestra.run.smithi039.stdout: python3-typing-extensions-4.12.2-2.el9.noarch 2024-09-21T03:33:26.524 INFO:teuthology.orchestra.run.smithi039.stdout: python3-urllib3-1.26.5-6.el9.noarch 2024-09-21T03:33:26.524 INFO:teuthology.orchestra.run.smithi039.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-09-21T03:33:26.524 INFO:teuthology.orchestra.run.smithi039.stdout: python3-webob-1.8.8-2.el9.noarch 2024-09-21T03:33:26.524 INFO:teuthology.orchestra.run.smithi039.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-09-21T03:33:26.524 INFO:teuthology.orchestra.run.smithi039.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-09-21T03:33:26.524 INFO:teuthology.orchestra.run.smithi039.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-09-21T03:33:26.524 INFO:teuthology.orchestra.run.smithi039.stdout: qatlib-24.02.0-1.el9.x86_64 2024-09-21T03:33:26.524 INFO:teuthology.orchestra.run.smithi039.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-09-21T03:33:26.524 INFO:teuthology.orchestra.run.smithi039.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-09-21T03:33:26.524 INFO:teuthology.orchestra.run.smithi039.stdout: rbd-fuse-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:33:26.525 INFO:teuthology.orchestra.run.smithi039.stdout: rbd-mirror-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:33:26.525 INFO:teuthology.orchestra.run.smithi039.stdout: rbd-nbd-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:33:26.525 INFO:teuthology.orchestra.run.smithi039.stdout: re2-1:20211101-20.el9.x86_64 2024-09-21T03:33:26.525 INFO:teuthology.orchestra.run.smithi039.stdout: socat-1.7.4.1-6.el9.x86_64 2024-09-21T03:33:26.525 INFO:teuthology.orchestra.run.smithi039.stdout: thrift-0.15.0-2.el9.x86_64 2024-09-21T03:33:26.525 INFO:teuthology.orchestra.run.smithi039.stdout: uuid-1.6.2-55.el9.x86_64 2024-09-21T03:33:26.525 INFO:teuthology.orchestra.run.smithi039.stdout: xfsprogs-devel-6.4.0-4.el9.x86_64 2024-09-21T03:33:26.525 INFO:teuthology.orchestra.run.smithi039.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-09-21T03:33:26.525 INFO:teuthology.orchestra.run.smithi039.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-09-21T03:33:26.525 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T03:33:26.526 INFO:teuthology.orchestra.run.smithi039.stdout:Complete! 2024-09-21T03:33:26.988 DEBUG:teuthology.parallel:result is None 2024-09-21T03:33:29.125 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.3.0-5120.g18432218.el9.noa 129/165 2024-09-21T03:33:29.192 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : ceph-mgr-dashboard-2:19.3.0-5120.g18432218.el9.n 130/165 2024-09-21T03:33:30.162 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.3.0-5120.g18432218.el9.n 130/165 2024-09-21T03:33:30.192 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : ceph-mgr-diskprediction-local-2:19.3.0-5120.g184 131/165 2024-09-21T03:33:30.306 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.3.0-5120.g184 131/165 2024-09-21T03:33:30.476 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : ceph-mgr-modules-core-2:19.3.0-5120.g18432218.el 132/165 2024-09-21T03:33:30.502 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : ceph-mgr-2:19.3.0-5120.g18432218.el9.x86_64 133/165 2024-09-21T03:33:30.528 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: ceph-mgr-2:19.3.0-5120.g18432218.el9.x86_64 133/165 2024-09-21T03:33:30.528 INFO:teuthology.orchestra.run.smithi161.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T03:33:30.528 INFO:teuthology.orchestra.run.smithi161.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-09-21T03:33:30.528 INFO:teuthology.orchestra.run.smithi161.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-21T03:33:30.528 INFO:teuthology.orchestra.run.smithi161.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-21T03:33:30.528 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-09-21T03:33:30.578 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : ceph-mgr-rook-2:19.3.0-5120.g18432218.el9.noarch 134/165 2024-09-21T03:33:30.590 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: ceph-mgr-rook-2:19.3.0-5120.g18432218.el9.noarch 134/165 2024-09-21T03:33:30.688 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : ceph-2:19.3.0-5120.g18432218.el9.x86_64 135/165 2024-09-21T03:33:30.742 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : ceph-fuse-2:19.3.0-5120.g18432218.el9.x86_64 136/165 2024-09-21T03:33:31.710 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : libtool-2.4.6-46.el9.x86_64 137/165 2024-09-21T03:33:31.747 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : ceph-radosgw-2:19.3.0-5120.g18432218.el9.x86_64 138/165 2024-09-21T03:33:31.772 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: ceph-radosgw-2:19.3.0-5120.g18432218.el9.x86_64 138/165 2024-09-21T03:33:31.772 INFO:teuthology.orchestra.run.smithi161.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T03:33:31.772 INFO:teuthology.orchestra.run.smithi161.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-09-21T03:33:31.772 INFO:teuthology.orchestra.run.smithi161.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-21T03:33:31.772 INFO:teuthology.orchestra.run.smithi161.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-21T03:33:31.772 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-09-21T03:33:31.815 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : ceph-immutable-object-cache-2:19.3.0-5120.g18432 139/165 2024-09-21T03:33:31.838 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.3.0-5120.g18432 139/165 2024-09-21T03:33:31.838 INFO:teuthology.orchestra.run.smithi161.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T03:33:31.838 INFO:teuthology.orchestra.run.smithi161.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-09-21T03:33:31.839 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-09-21T03:33:31.890 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : cephfs-mirror-2:19.3.0-5120.g18432218.el9.x86_64 140/165 2024-09-21T03:33:31.914 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: cephfs-mirror-2:19.3.0-5120.g18432218.el9.x86_64 140/165 2024-09-21T03:33:31.915 INFO:teuthology.orchestra.run.smithi161.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T03:33:31.915 INFO:teuthology.orchestra.run.smithi161.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-09-21T03:33:31.915 INFO:teuthology.orchestra.run.smithi161.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-21T03:33:31.915 INFO:teuthology.orchestra.run.smithi161.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-21T03:33:31.915 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-09-21T03:33:32.233 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : rbd-mirror-2:19.3.0-5120.g18432218.el9.x86_64 141/165 2024-09-21T03:33:32.256 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: rbd-mirror-2:19.3.0-5120.g18432218.el9.x86_64 141/165 2024-09-21T03:33:32.257 INFO:teuthology.orchestra.run.smithi161.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T03:33:32.257 INFO:teuthology.orchestra.run.smithi161.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-09-21T03:33:32.257 INFO:teuthology.orchestra.run.smithi161.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-21T03:33:32.257 INFO:teuthology.orchestra.run.smithi161.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-21T03:33:32.257 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-09-21T03:33:36.262 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : ceph-test-2:19.3.0-5120.g18432218.el9.x86_64 142/165 2024-09-21T03:33:36.303 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : rbd-fuse-2:19.3.0-5120.g18432218.el9.x86_64 143/165 2024-09-21T03:33:36.344 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : rbd-nbd-2:19.3.0-5120.g18432218.el9.x86_64 144/165 2024-09-21T03:33:36.376 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : xfsprogs-devel-6.4.0-4.el9.x86_64 145/165 2024-09-21T03:33:36.403 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : elfutils-libelf-devel-0.191-4.el9.x86_64 146/165 2024-09-21T03:33:36.585 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : cephfs-shell-2:19.3.0-5120.g18432218.el9.x86_64 147/165 2024-09-21T03:33:36.625 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : postgresql-contrib-13.16-1.el9.x86_64 148/165 2024-09-21T03:33:36.655 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : libcephfs-devel-2:19.3.0-5120.g18432218.el9.x86_ 149/165 2024-09-21T03:33:36.685 INFO:teuthology.orchestra.run.smithi161.stdout: Upgrading : NetworkManager-team-1:1.51.0-1.el9.x86_64 150/165 2024-09-21T03:33:36.739 INFO:teuthology.orchestra.run.smithi161.stdout: Upgrading : NetworkManager-tui-1:1.51.0-1.el9.x86_64 151/165 2024-09-21T03:33:37.312 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 152/165 2024-09-21T03:33:37.354 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : postgresql-server-13.16-1.el9.x86_64 152/165 2024-09-21T03:33:37.375 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 152/165 2024-09-21T03:33:38.251 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : cephfs-top-2:19.3.0-5120.g18432218.el9.noarch 153/165 2024-09-21T03:33:38.342 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : openssl-devel-1:3.2.2-6.el9.x86_64 154/165 2024-09-21T03:33:38.383 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 155/165 2024-09-21T03:33:38.481 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : flex-2.6.4-9.el9.x86_64 156/165 2024-09-21T03:33:38.520 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : bison-3.7.4-5.el9.x86_64 157/165 2024-09-21T03:33:38.545 INFO:teuthology.orchestra.run.smithi161.stdout: Cleanup : NetworkManager-tui-1:1.48.8-1.el9.x86_64 158/165 2024-09-21T03:33:38.571 INFO:teuthology.orchestra.run.smithi161.stdout: Cleanup : openssl-1:3.2.2-4.el9.x86_64 159/165 2024-09-21T03:33:38.571 INFO:teuthology.orchestra.run.smithi161.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 160/165 2024-09-21T03:33:38.647 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 160/165 2024-09-21T03:33:38.647 INFO:teuthology.orchestra.run.smithi161.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 161/165 2024-09-21T03:33:38.714 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 161/165 2024-09-21T03:33:38.739 INFO:teuthology.orchestra.run.smithi161.stdout: Cleanup : NetworkManager-team-1:1.48.8-1.el9.x86_64 162/165 2024-09-21T03:33:38.745 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 163/165 2024-09-21T03:33:38.745 INFO:teuthology.orchestra.run.smithi161.stdout: Cleanup : NetworkManager-1:1.48.8-1.el9.x86_64 163/165 2024-09-21T03:33:38.933 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 163/165 2024-09-21T03:33:38.933 INFO:teuthology.orchestra.run.smithi161.stdout: Cleanup : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 164/165 2024-09-21T03:33:39.009 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: NetworkManager-libnm-1:1.48.8-1.el9.x86_64 164/165 2024-09-21T03:33:39.072 INFO:teuthology.orchestra.run.smithi161.stdout: Cleanup : openssl-libs-1:3.2.2-4.el9.x86_64 165/165 2024-09-21T03:33:45.075 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: openssl-libs-1:3.2.2-4.el9.x86_64 165/165 2024-09-21T03:33:45.075 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : ceph-2:19.3.0-5120.g18432218.el9.x86_64 1/165 2024-09-21T03:33:45.075 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : ceph-base-2:19.3.0-5120.g18432218.el9.x86_64 2/165 2024-09-21T03:33:45.076 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : ceph-common-2:19.3.0-5120.g18432218.el9.x86_64 3/165 2024-09-21T03:33:45.076 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : ceph-fuse-2:19.3.0-5120.g18432218.el9.x86_64 4/165 2024-09-21T03:33:45.076 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : ceph-immutable-object-cache-2:19.3.0-5120.g18432 5/165 2024-09-21T03:33:45.076 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : ceph-mds-2:19.3.0-5120.g18432218.el9.x86_64 6/165 2024-09-21T03:33:45.076 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : ceph-mgr-2:19.3.0-5120.g18432218.el9.x86_64 7/165 2024-09-21T03:33:45.076 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : ceph-mon-2:19.3.0-5120.g18432218.el9.x86_64 8/165 2024-09-21T03:33:45.076 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : ceph-osd-2:19.3.0-5120.g18432218.el9.x86_64 9/165 2024-09-21T03:33:45.076 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : ceph-radosgw-2:19.3.0-5120.g18432218.el9.x86_64 10/165 2024-09-21T03:33:45.076 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : ceph-selinux-2:19.3.0-5120.g18432218.el9.x86_64 11/165 2024-09-21T03:33:45.076 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : ceph-test-2:19.3.0-5120.g18432218.el9.x86_64 12/165 2024-09-21T03:33:45.076 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : cephfs-mirror-2:19.3.0-5120.g18432218.el9.x86_64 13/165 2024-09-21T03:33:45.076 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : cephfs-shell-2:19.3.0-5120.g18432218.el9.x86_64 14/165 2024-09-21T03:33:45.076 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : libcephfs-devel-2:19.3.0-5120.g18432218.el9.x86_ 15/165 2024-09-21T03:33:45.076 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : libcephfs2-2:19.3.0-5120.g18432218.el9.x86_64 16/165 2024-09-21T03:33:45.077 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : libcephsqlite-2:19.3.0-5120.g18432218.el9.x86_64 17/165 2024-09-21T03:33:45.077 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : librados-devel-2:19.3.0-5120.g18432218.el9.x86_6 18/165 2024-09-21T03:33:45.077 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : libradosstriper1-2:19.3.0-5120.g18432218.el9.x86 19/165 2024-09-21T03:33:45.077 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : librgw2-2:19.3.0-5120.g18432218.el9.x86_64 20/165 2024-09-21T03:33:45.077 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-ceph-argparse-2:19.3.0-5120.g18432218.el 21/165 2024-09-21T03:33:45.077 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-ceph-common-2:19.3.0-5120.g18432218.el9. 22/165 2024-09-21T03:33:45.077 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-cephfs-2:19.3.0-5120.g18432218.el9.x86_6 23/165 2024-09-21T03:33:45.077 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-rados-2:19.3.0-5120.g18432218.el9.x86_64 24/165 2024-09-21T03:33:45.077 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-rbd-2:19.3.0-5120.g18432218.el9.x86_64 25/165 2024-09-21T03:33:45.077 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-rgw-2:19.3.0-5120.g18432218.el9.x86_64 26/165 2024-09-21T03:33:45.077 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : rbd-fuse-2:19.3.0-5120.g18432218.el9.x86_64 27/165 2024-09-21T03:33:45.077 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : rbd-mirror-2:19.3.0-5120.g18432218.el9.x86_64 28/165 2024-09-21T03:33:45.077 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : rbd-nbd-2:19.3.0-5120.g18432218.el9.x86_64 29/165 2024-09-21T03:33:45.078 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : ceph-grafana-dashboards-2:19.3.0-5120.g18432218. 30/165 2024-09-21T03:33:45.078 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : ceph-mgr-cephadm-2:19.3.0-5120.g18432218.el9.noa 31/165 2024-09-21T03:33:45.078 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : ceph-mgr-dashboard-2:19.3.0-5120.g18432218.el9.n 32/165 2024-09-21T03:33:45.078 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.3.0-5120.g184 33/165 2024-09-21T03:33:45.078 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : ceph-mgr-modules-core-2:19.3.0-5120.g18432218.el 34/165 2024-09-21T03:33:45.078 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : ceph-mgr-rook-2:19.3.0-5120.g18432218.el9.noarch 35/165 2024-09-21T03:33:45.078 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : ceph-prometheus-alerts-2:19.3.0-5120.g18432218.e 36/165 2024-09-21T03:33:45.078 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : ceph-volume-2:19.3.0-5120.g18432218.el9.noarch 37/165 2024-09-21T03:33:45.078 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : cephadm-2:19.3.0-5120.g18432218.el9.noarch 38/165 2024-09-21T03:33:45.078 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : cephfs-top-2:19.3.0-5120.g18432218.el9.noarch 39/165 2024-09-21T03:33:45.078 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : fuse-2.9.9-16.el9.x86_64 40/165 2024-09-21T03:33:45.078 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : libgfortran-11.5.0-2.el9.x86_64 41/165 2024-09-21T03:33:45.078 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : libquadmath-11.5.0-2.el9.x86_64 42/165 2024-09-21T03:33:45.079 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/165 2024-09-21T03:33:45.079 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/165 2024-09-21T03:33:45.079 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/165 2024-09-21T03:33:45.079 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/165 2024-09-21T03:33:45.079 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/165 2024-09-21T03:33:45.079 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-urllib3-1.26.5-6.el9.noarch 48/165 2024-09-21T03:33:45.079 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/165 2024-09-21T03:33:45.079 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/165 2024-09-21T03:33:45.079 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/165 2024-09-21T03:33:45.079 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : elfutils-libelf-devel-0.191-4.el9.x86_64 52/165 2024-09-21T03:33:45.079 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/165 2024-09-21T03:33:45.079 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/165 2024-09-21T03:33:45.079 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/165 2024-09-21T03:33:45.079 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/165 2024-09-21T03:33:45.079 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/165 2024-09-21T03:33:45.080 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : libnbd-1.20.2-2.el9.x86_64 58/165 2024-09-21T03:33:45.080 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 59/165 2024-09-21T03:33:45.080 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 60/165 2024-09-21T03:33:45.080 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 61/165 2024-09-21T03:33:45.080 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : libtool-2.4.6-46.el9.x86_64 62/165 2024-09-21T03:33:45.080 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : libuuid-devel-2.37.4-20.el9.x86_64 63/165 2024-09-21T03:33:45.080 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 64/165 2024-09-21T03:33:45.080 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 65/165 2024-09-21T03:33:45.080 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : lua-5.4.4-4.el9.x86_64 66/165 2024-09-21T03:33:45.080 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : openblas-0.3.26-2.el9.x86_64 67/165 2024-09-21T03:33:45.080 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : openblas-openmp-0.3.26-2.el9.x86_64 68/165 2024-09-21T03:33:45.080 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : openssl-devel-1:3.2.2-6.el9.x86_64 69/165 2024-09-21T03:33:45.080 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : postgresql-13.16-1.el9.x86_64 70/165 2024-09-21T03:33:45.080 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : postgresql-contrib-13.16-1.el9.x86_64 71/165 2024-09-21T03:33:45.081 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : postgresql-private-libs-13.16-1.el9.x86_64 72/165 2024-09-21T03:33:45.081 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : postgresql-server-13.16-1.el9.x86_64 73/165 2024-09-21T03:33:45.081 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 74/165 2024-09-21T03:33:45.081 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 75/165 2024-09-21T03:33:45.081 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 76/165 2024-09-21T03:33:45.081 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-colorama-0.4.6-3.el9.noarch 77/165 2024-09-21T03:33:45.081 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-devel-3.9.19-8.el9.x86_64 78/165 2024-09-21T03:33:45.081 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 79/165 2024-09-21T03:33:45.081 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 80/165 2024-09-21T03:33:45.081 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 81/165 2024-09-21T03:33:45.081 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 82/165 2024-09-21T03:33:45.081 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 83/165 2024-09-21T03:33:45.081 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 84/165 2024-09-21T03:33:45.081 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 85/165 2024-09-21T03:33:45.082 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 86/165 2024-09-21T03:33:45.082 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 87/165 2024-09-21T03:33:45.082 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 88/165 2024-09-21T03:33:45.082 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 89/165 2024-09-21T03:33:45.082 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 90/165 2024-09-21T03:33:45.082 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 91/165 2024-09-21T03:33:45.082 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 92/165 2024-09-21T03:33:45.082 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 93/165 2024-09-21T03:33:45.082 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 94/165 2024-09-21T03:33:45.082 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 95/165 2024-09-21T03:33:45.082 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 96/165 2024-09-21T03:33:45.082 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 97/165 2024-09-21T03:33:45.082 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 98/165 2024-09-21T03:33:45.082 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 99/165 2024-09-21T03:33:45.083 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : socat-1.7.4.1-6.el9.x86_64 100/165 2024-09-21T03:33:45.083 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 101/165 2024-09-21T03:33:45.083 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : xfsprogs-devel-6.4.0-4.el9.x86_64 102/165 2024-09-21T03:33:45.083 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 103/165 2024-09-21T03:33:45.083 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 104/165 2024-09-21T03:33:45.083 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 105/165 2024-09-21T03:33:45.083 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 106/165 2024-09-21T03:33:45.083 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 107/165 2024-09-21T03:33:45.083 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : gperftools-libs-2.9.1-3.el9.x86_64 108/165 2024-09-21T03:33:45.083 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 109/165 2024-09-21T03:33:45.083 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 110/165 2024-09-21T03:33:45.083 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 111/165 2024-09-21T03:33:45.083 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 112/165 2024-09-21T03:33:45.083 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 113/165 2024-09-21T03:33:45.084 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 114/165 2024-09-21T03:33:45.084 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 115/165 2024-09-21T03:33:45.084 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 116/165 2024-09-21T03:33:45.084 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 117/165 2024-09-21T03:33:45.084 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 118/165 2024-09-21T03:33:45.084 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 119/165 2024-09-21T03:33:45.084 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 120/165 2024-09-21T03:33:45.084 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 121/165 2024-09-21T03:33:45.084 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 122/165 2024-09-21T03:33:45.084 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-google-auth-1:2.34.0-1.el9.noarch 123/165 2024-09-21T03:33:45.084 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 124/165 2024-09-21T03:33:45.084 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 125/165 2024-09-21T03:33:45.084 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 126/165 2024-09-21T03:33:45.084 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 127/165 2024-09-21T03:33:45.084 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 128/165 2024-09-21T03:33:45.085 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 129/165 2024-09-21T03:33:45.085 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 130/165 2024-09-21T03:33:45.085 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 131/165 2024-09-21T03:33:45.085 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 132/165 2024-09-21T03:33:45.085 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 133/165 2024-09-21T03:33:45.085 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 134/165 2024-09-21T03:33:45.085 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 135/165 2024-09-21T03:33:45.085 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 136/165 2024-09-21T03:33:45.085 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 137/165 2024-09-21T03:33:45.085 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 138/165 2024-09-21T03:33:45.085 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 139/165 2024-09-21T03:33:45.085 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 140/165 2024-09-21T03:33:45.085 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 141/165 2024-09-21T03:33:45.085 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 142/165 2024-09-21T03:33:45.085 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-typing-extensions-4.12.2-2.el9.noarch 143/165 2024-09-21T03:33:45.086 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-webob-1.8.8-2.el9.noarch 144/165 2024-09-21T03:33:45.086 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 145/165 2024-09-21T03:33:45.086 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 146/165 2024-09-21T03:33:45.086 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 147/165 2024-09-21T03:33:45.086 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : re2-1:20211101-20.el9.x86_64 148/165 2024-09-21T03:33:45.086 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 149/165 2024-09-21T03:33:45.086 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : librados2-2:19.3.0-5120.g18432218.el9.x86_64 150/165 2024-09-21T03:33:45.086 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 151/165 2024-09-21T03:33:45.086 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : librbd1-2:19.3.0-5120.g18432218.el9.x86_64 152/165 2024-09-21T03:33:45.086 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 153/165 2024-09-21T03:33:45.086 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : NetworkManager-1:1.51.0-1.el9.x86_64 154/165 2024-09-21T03:33:45.086 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : NetworkManager-1:1.48.8-1.el9.x86_64 155/165 2024-09-21T03:33:45.086 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : NetworkManager-libnm-1:1.51.0-1.el9.x86_64 156/165 2024-09-21T03:33:45.086 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 157/165 2024-09-21T03:33:45.087 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : NetworkManager-team-1:1.51.0-1.el9.x86_64 158/165 2024-09-21T03:33:45.087 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : NetworkManager-team-1:1.48.8-1.el9.x86_64 159/165 2024-09-21T03:33:45.087 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : NetworkManager-tui-1:1.51.0-1.el9.x86_64 160/165 2024-09-21T03:33:45.087 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : NetworkManager-tui-1:1.48.8-1.el9.x86_64 161/165 2024-09-21T03:33:45.087 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : openssl-1:3.2.2-6.el9.x86_64 162/165 2024-09-21T03:33:45.087 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : openssl-1:3.2.2-4.el9.x86_64 163/165 2024-09-21T03:33:45.087 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : openssl-libs-1:3.2.2-6.el9.x86_64 164/165 2024-09-21T03:33:47.910 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : openssl-libs-1:3.2.2-4.el9.x86_64 165/165 2024-09-21T03:33:47.911 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-09-21T03:33:47.911 INFO:teuthology.orchestra.run.smithi161.stdout:Upgraded: 2024-09-21T03:33:47.911 INFO:teuthology.orchestra.run.smithi161.stdout: NetworkManager-1:1.51.0-1.el9.x86_64 2024-09-21T03:33:47.911 INFO:teuthology.orchestra.run.smithi161.stdout: NetworkManager-libnm-1:1.51.0-1.el9.x86_64 2024-09-21T03:33:47.911 INFO:teuthology.orchestra.run.smithi161.stdout: NetworkManager-team-1:1.51.0-1.el9.x86_64 2024-09-21T03:33:47.911 INFO:teuthology.orchestra.run.smithi161.stdout: NetworkManager-tui-1:1.51.0-1.el9.x86_64 2024-09-21T03:33:47.911 INFO:teuthology.orchestra.run.smithi161.stdout: librados2-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:33:47.911 INFO:teuthology.orchestra.run.smithi161.stdout: librbd1-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:33:47.911 INFO:teuthology.orchestra.run.smithi161.stdout: openssl-1:3.2.2-6.el9.x86_64 2024-09-21T03:33:47.911 INFO:teuthology.orchestra.run.smithi161.stdout: openssl-libs-1:3.2.2-6.el9.x86_64 2024-09-21T03:33:47.911 INFO:teuthology.orchestra.run.smithi161.stdout:Installed: 2024-09-21T03:33:47.911 INFO:teuthology.orchestra.run.smithi161.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-09-21T03:33:47.911 INFO:teuthology.orchestra.run.smithi161.stdout: automake-1.16.2-8.el9.noarch 2024-09-21T03:33:47.912 INFO:teuthology.orchestra.run.smithi161.stdout: bison-3.7.4-5.el9.x86_64 2024-09-21T03:33:47.912 INFO:teuthology.orchestra.run.smithi161.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-09-21T03:33:47.912 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:33:47.912 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-base-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:33:47.912 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-common-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:33:47.912 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-fuse-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:33:47.912 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-grafana-dashboards-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T03:33:47.912 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-immutable-object-cache-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:33:47.912 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-mds-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:33:47.912 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-mgr-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:33:47.912 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-mgr-cephadm-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T03:33:47.912 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-mgr-dashboard-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T03:33:47.912 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-mgr-diskprediction-local-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T03:33:47.912 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-mgr-modules-core-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T03:33:47.912 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-mgr-rook-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T03:33:47.913 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-mon-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:33:47.913 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-osd-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:33:47.913 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-prometheus-alerts-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T03:33:47.913 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-radosgw-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:33:47.913 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-selinux-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:33:47.913 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-test-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:33:47.913 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-volume-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T03:33:47.913 INFO:teuthology.orchestra.run.smithi161.stdout: cephadm-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T03:33:47.913 INFO:teuthology.orchestra.run.smithi161.stdout: cephfs-mirror-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:33:47.913 INFO:teuthology.orchestra.run.smithi161.stdout: cephfs-shell-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:33:47.913 INFO:teuthology.orchestra.run.smithi161.stdout: cephfs-top-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T03:33:47.913 INFO:teuthology.orchestra.run.smithi161.stdout: elfutils-libelf-devel-0.191-4.el9.x86_64 2024-09-21T03:33:47.913 INFO:teuthology.orchestra.run.smithi161.stdout: flex-2.6.4-9.el9.x86_64 2024-09-21T03:33:47.913 INFO:teuthology.orchestra.run.smithi161.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-09-21T03:33:47.913 INFO:teuthology.orchestra.run.smithi161.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-09-21T03:33:47.914 INFO:teuthology.orchestra.run.smithi161.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-09-21T03:33:47.914 INFO:teuthology.orchestra.run.smithi161.stdout: fuse-2.9.9-16.el9.x86_64 2024-09-21T03:33:47.914 INFO:teuthology.orchestra.run.smithi161.stdout: gperftools-libs-2.9.1-3.el9.x86_64 2024-09-21T03:33:47.914 INFO:teuthology.orchestra.run.smithi161.stdout: grpc-data-1.46.7-10.el9.noarch 2024-09-21T03:33:47.914 INFO:teuthology.orchestra.run.smithi161.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-09-21T03:33:47.914 INFO:teuthology.orchestra.run.smithi161.stdout: libarrow-9.0.0-11.el9.x86_64 2024-09-21T03:33:47.914 INFO:teuthology.orchestra.run.smithi161.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-09-21T03:33:47.914 INFO:teuthology.orchestra.run.smithi161.stdout: libcephfs-devel-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:33:47.914 INFO:teuthology.orchestra.run.smithi161.stdout: libcephfs2-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:33:47.914 INFO:teuthology.orchestra.run.smithi161.stdout: libcephsqlite-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:33:47.914 INFO:teuthology.orchestra.run.smithi161.stdout: libgfortran-11.5.0-2.el9.x86_64 2024-09-21T03:33:47.914 INFO:teuthology.orchestra.run.smithi161.stdout: libnbd-1.20.2-2.el9.x86_64 2024-09-21T03:33:47.914 INFO:teuthology.orchestra.run.smithi161.stdout: liboath-2.6.7-2.el9.x86_64 2024-09-21T03:33:47.915 INFO:teuthology.orchestra.run.smithi161.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-09-21T03:33:47.915 INFO:teuthology.orchestra.run.smithi161.stdout: libquadmath-11.5.0-2.el9.x86_64 2024-09-21T03:33:47.915 INFO:teuthology.orchestra.run.smithi161.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-09-21T03:33:47.915 INFO:teuthology.orchestra.run.smithi161.stdout: librados-devel-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:33:47.915 INFO:teuthology.orchestra.run.smithi161.stdout: libradosstriper1-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:33:47.915 INFO:teuthology.orchestra.run.smithi161.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-09-21T03:33:47.915 INFO:teuthology.orchestra.run.smithi161.stdout: librgw2-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:33:47.915 INFO:teuthology.orchestra.run.smithi161.stdout: libtool-2.4.6-46.el9.x86_64 2024-09-21T03:33:47.915 INFO:teuthology.orchestra.run.smithi161.stdout: libunwind-1.6.2-1.el9.x86_64 2024-09-21T03:33:47.915 INFO:teuthology.orchestra.run.smithi161.stdout: libuuid-devel-2.37.4-20.el9.x86_64 2024-09-21T03:33:47.915 INFO:teuthology.orchestra.run.smithi161.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-09-21T03:33:47.915 INFO:teuthology.orchestra.run.smithi161.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-09-21T03:33:47.915 INFO:teuthology.orchestra.run.smithi161.stdout: lua-5.4.4-4.el9.x86_64 2024-09-21T03:33:47.915 INFO:teuthology.orchestra.run.smithi161.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-09-21T03:33:47.916 INFO:teuthology.orchestra.run.smithi161.stdout: luarocks-3.9.2-5.el9.noarch 2024-09-21T03:33:47.916 INFO:teuthology.orchestra.run.smithi161.stdout: openblas-0.3.26-2.el9.x86_64 2024-09-21T03:33:47.916 INFO:teuthology.orchestra.run.smithi161.stdout: openblas-openmp-0.3.26-2.el9.x86_64 2024-09-21T03:33:47.916 INFO:teuthology.orchestra.run.smithi161.stdout: openssl-devel-1:3.2.2-6.el9.x86_64 2024-09-21T03:33:47.916 INFO:teuthology.orchestra.run.smithi161.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-09-21T03:33:47.916 INFO:teuthology.orchestra.run.smithi161.stdout: postgresql-13.16-1.el9.x86_64 2024-09-21T03:33:47.916 INFO:teuthology.orchestra.run.smithi161.stdout: postgresql-contrib-13.16-1.el9.x86_64 2024-09-21T03:33:47.916 INFO:teuthology.orchestra.run.smithi161.stdout: postgresql-private-libs-13.16-1.el9.x86_64 2024-09-21T03:33:47.916 INFO:teuthology.orchestra.run.smithi161.stdout: postgresql-server-13.16-1.el9.x86_64 2024-09-21T03:33:47.916 INFO:teuthology.orchestra.run.smithi161.stdout: protobuf-3.14.0-13.el9.x86_64 2024-09-21T03:33:47.916 INFO:teuthology.orchestra.run.smithi161.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-09-21T03:33:47.916 INFO:teuthology.orchestra.run.smithi161.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-09-21T03:33:47.916 INFO:teuthology.orchestra.run.smithi161.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-09-21T03:33:47.916 INFO:teuthology.orchestra.run.smithi161.stdout: python3-babel-2.9.1-2.el9.noarch 2024-09-21T03:33:47.916 INFO:teuthology.orchestra.run.smithi161.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-09-21T03:33:47.916 INFO:teuthology.orchestra.run.smithi161.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-09-21T03:33:47.917 INFO:teuthology.orchestra.run.smithi161.stdout: python3-ceph-argparse-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:33:47.917 INFO:teuthology.orchestra.run.smithi161.stdout: python3-ceph-common-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:33:47.917 INFO:teuthology.orchestra.run.smithi161.stdout: python3-cephfs-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:33:47.917 INFO:teuthology.orchestra.run.smithi161.stdout: python3-certifi-2023.05.07-4.el9.noarch 2024-09-21T03:33:47.917 INFO:teuthology.orchestra.run.smithi161.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-09-21T03:33:47.917 INFO:teuthology.orchestra.run.smithi161.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-09-21T03:33:47.917 INFO:teuthology.orchestra.run.smithi161.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-09-21T03:33:47.917 INFO:teuthology.orchestra.run.smithi161.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-09-21T03:33:47.917 INFO:teuthology.orchestra.run.smithi161.stdout: python3-colorama-0.4.6-3.el9.noarch 2024-09-21T03:33:47.917 INFO:teuthology.orchestra.run.smithi161.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-09-21T03:33:47.917 INFO:teuthology.orchestra.run.smithi161.stdout: python3-devel-3.9.19-8.el9.x86_64 2024-09-21T03:33:47.917 INFO:teuthology.orchestra.run.smithi161.stdout: python3-google-auth-1:2.34.0-1.el9.noarch 2024-09-21T03:33:47.917 INFO:teuthology.orchestra.run.smithi161.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-09-21T03:33:47.917 INFO:teuthology.orchestra.run.smithi161.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-09-21T03:33:47.917 INFO:teuthology.orchestra.run.smithi161.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-09-21T03:33:47.917 INFO:teuthology.orchestra.run.smithi161.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-09-21T03:33:47.917 INFO:teuthology.orchestra.run.smithi161.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-09-21T03:33:47.918 INFO:teuthology.orchestra.run.smithi161.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-09-21T03:33:47.918 INFO:teuthology.orchestra.run.smithi161.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-09-21T03:33:47.918 INFO:teuthology.orchestra.run.smithi161.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-09-21T03:33:47.918 INFO:teuthology.orchestra.run.smithi161.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-09-21T03:33:47.918 INFO:teuthology.orchestra.run.smithi161.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-09-21T03:33:47.918 INFO:teuthology.orchestra.run.smithi161.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-09-21T03:33:47.918 INFO:teuthology.orchestra.run.smithi161.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-09-21T03:33:47.918 INFO:teuthology.orchestra.run.smithi161.stdout: python3-mako-1.1.4-6.el9.noarch 2024-09-21T03:33:47.918 INFO:teuthology.orchestra.run.smithi161.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-09-21T03:33:47.918 INFO:teuthology.orchestra.run.smithi161.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-09-21T03:33:47.918 INFO:teuthology.orchestra.run.smithi161.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-09-21T03:33:47.918 INFO:teuthology.orchestra.run.smithi161.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-09-21T03:33:47.918 INFO:teuthology.orchestra.run.smithi161.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-09-21T03:33:47.918 INFO:teuthology.orchestra.run.smithi161.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-09-21T03:33:47.919 INFO:teuthology.orchestra.run.smithi161.stdout: python3-packaging-20.9-5.el9.noarch 2024-09-21T03:33:47.919 INFO:teuthology.orchestra.run.smithi161.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-09-21T03:33:47.919 INFO:teuthology.orchestra.run.smithi161.stdout: python3-ply-3.11-14.el9.noarch 2024-09-21T03:33:47.919 INFO:teuthology.orchestra.run.smithi161.stdout: python3-portend-3.1.0-2.el9.noarch 2024-09-21T03:33:47.919 INFO:teuthology.orchestra.run.smithi161.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-09-21T03:33:47.919 INFO:teuthology.orchestra.run.smithi161.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-09-21T03:33:47.919 INFO:teuthology.orchestra.run.smithi161.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-09-21T03:33:47.919 INFO:teuthology.orchestra.run.smithi161.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-09-21T03:33:47.919 INFO:teuthology.orchestra.run.smithi161.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-09-21T03:33:47.919 INFO:teuthology.orchestra.run.smithi161.stdout: python3-pycparser-2.20-6.el9.noarch 2024-09-21T03:33:47.919 INFO:teuthology.orchestra.run.smithi161.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-09-21T03:33:47.919 INFO:teuthology.orchestra.run.smithi161.stdout: python3-pytz-2021.1-5.el9.noarch 2024-09-21T03:33:47.919 INFO:teuthology.orchestra.run.smithi161.stdout: python3-rados-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:33:47.919 INFO:teuthology.orchestra.run.smithi161.stdout: python3-rbd-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:33:47.920 INFO:teuthology.orchestra.run.smithi161.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-09-21T03:33:47.920 INFO:teuthology.orchestra.run.smithi161.stdout: python3-requests-2.25.1-8.el9.noarch 2024-09-21T03:33:47.920 INFO:teuthology.orchestra.run.smithi161.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-09-21T03:33:47.920 INFO:teuthology.orchestra.run.smithi161.stdout: python3-rgw-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:33:47.920 INFO:teuthology.orchestra.run.smithi161.stdout: python3-routes-2.5.1-5.el9.noarch 2024-09-21T03:33:47.920 INFO:teuthology.orchestra.run.smithi161.stdout: python3-rsa-4.9-2.el9.noarch 2024-09-21T03:33:47.920 INFO:teuthology.orchestra.run.smithi161.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-09-21T03:33:47.920 INFO:teuthology.orchestra.run.smithi161.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-09-21T03:33:47.920 INFO:teuthology.orchestra.run.smithi161.stdout: python3-toml-0.10.2-6.el9.noarch 2024-09-21T03:33:47.920 INFO:teuthology.orchestra.run.smithi161.stdout: python3-typing-extensions-4.12.2-2.el9.noarch 2024-09-21T03:33:47.920 INFO:teuthology.orchestra.run.smithi161.stdout: python3-urllib3-1.26.5-6.el9.noarch 2024-09-21T03:33:47.920 INFO:teuthology.orchestra.run.smithi161.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-09-21T03:33:47.920 INFO:teuthology.orchestra.run.smithi161.stdout: python3-webob-1.8.8-2.el9.noarch 2024-09-21T03:33:47.920 INFO:teuthology.orchestra.run.smithi161.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-09-21T03:33:47.920 INFO:teuthology.orchestra.run.smithi161.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-09-21T03:33:47.921 INFO:teuthology.orchestra.run.smithi161.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-09-21T03:33:47.921 INFO:teuthology.orchestra.run.smithi161.stdout: qatlib-24.02.0-1.el9.x86_64 2024-09-21T03:33:47.921 INFO:teuthology.orchestra.run.smithi161.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-09-21T03:33:47.921 INFO:teuthology.orchestra.run.smithi161.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-09-21T03:33:47.921 INFO:teuthology.orchestra.run.smithi161.stdout: rbd-fuse-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:33:47.921 INFO:teuthology.orchestra.run.smithi161.stdout: rbd-mirror-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:33:47.921 INFO:teuthology.orchestra.run.smithi161.stdout: rbd-nbd-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:33:47.921 INFO:teuthology.orchestra.run.smithi161.stdout: re2-1:20211101-20.el9.x86_64 2024-09-21T03:33:47.921 INFO:teuthology.orchestra.run.smithi161.stdout: socat-1.7.4.1-6.el9.x86_64 2024-09-21T03:33:47.921 INFO:teuthology.orchestra.run.smithi161.stdout: thrift-0.15.0-2.el9.x86_64 2024-09-21T03:33:47.921 INFO:teuthology.orchestra.run.smithi161.stdout: uuid-1.6.2-55.el9.x86_64 2024-09-21T03:33:47.921 INFO:teuthology.orchestra.run.smithi161.stdout: xfsprogs-devel-6.4.0-4.el9.x86_64 2024-09-21T03:33:47.921 INFO:teuthology.orchestra.run.smithi161.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-09-21T03:33:47.922 INFO:teuthology.orchestra.run.smithi161.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-09-21T03:33:47.922 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-09-21T03:33:47.922 INFO:teuthology.orchestra.run.smithi161.stdout:Complete! 2024-09-21T03:33:48.360 DEBUG:teuthology.parallel:result is None 2024-09-21T03:33:48.361 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=1843221895adccfb3b692060251dd6a0183bc410 2024-09-21T03:33:48.506 DEBUG:teuthology.orchestra.run.smithi039:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-09-21T03:33:48.534 INFO:teuthology.orchestra.run.smithi039.stdout:19.3.0-5120.g18432218.el9 2024-09-21T03:33:48.535 INFO:teuthology.packaging:The installed version of ceph is 19.3.0-5120.g18432218.el9 2024-09-21T03:33:48.535 INFO:teuthology.task.install:The correct ceph version 19.3.0-5120.g18432218 is installed. 2024-09-21T03:33:48.537 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=1843221895adccfb3b692060251dd6a0183bc410 2024-09-21T03:33:48.679 DEBUG:teuthology.orchestra.run.smithi150:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-09-21T03:33:48.706 INFO:teuthology.orchestra.run.smithi150.stdout:19.3.0-5120.g18432218.el9 2024-09-21T03:33:48.706 INFO:teuthology.packaging:The installed version of ceph is 19.3.0-5120.g18432218.el9 2024-09-21T03:33:48.706 INFO:teuthology.task.install:The correct ceph version 19.3.0-5120.g18432218 is installed. 2024-09-21T03:33:48.708 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=1843221895adccfb3b692060251dd6a0183bc410 2024-09-21T03:33:48.847 DEBUG:teuthology.orchestra.run.smithi161:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-09-21T03:33:48.870 INFO:teuthology.orchestra.run.smithi161.stdout:19.3.0-5120.g18432218.el9 2024-09-21T03:33:48.871 INFO:teuthology.packaging:The installed version of ceph is 19.3.0-5120.g18432218.el9 2024-09-21T03:33:48.871 INFO:teuthology.task.install:The correct ceph version 19.3.0-5120.g18432218 is installed. 2024-09-21T03:33:48.873 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-09-21T03:33:48.873 DEBUG:teuthology.orchestra.run.smithi039:> set -ex 2024-09-21T03:33:48.873 DEBUG:teuthology.orchestra.run.smithi039:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-21T03:33:48.907 DEBUG:teuthology.orchestra.run.smithi150:> set -ex 2024-09-21T03:33:48.907 DEBUG:teuthology.orchestra.run.smithi150:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-21T03:33:48.940 DEBUG:teuthology.orchestra.run.smithi161:> set -ex 2024-09-21T03:33:48.940 DEBUG:teuthology.orchestra.run.smithi161:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-21T03:33:48.968 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-09-21T03:33:48.968 DEBUG:teuthology.orchestra.run.smithi039:> set -ex 2024-09-21T03:33:48.968 DEBUG:teuthology.orchestra.run.smithi039:> sudo dd of=/usr/bin/daemon-helper 2024-09-21T03:33:48.999 DEBUG:teuthology.orchestra.run.smithi039:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-21T03:33:49.067 DEBUG:teuthology.orchestra.run.smithi150:> set -ex 2024-09-21T03:33:49.067 DEBUG:teuthology.orchestra.run.smithi150:> sudo dd of=/usr/bin/daemon-helper 2024-09-21T03:33:49.096 DEBUG:teuthology.orchestra.run.smithi150:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-21T03:33:49.163 DEBUG:teuthology.orchestra.run.smithi161:> set -ex 2024-09-21T03:33:49.163 DEBUG:teuthology.orchestra.run.smithi161:> sudo dd of=/usr/bin/daemon-helper 2024-09-21T03:33:49.189 DEBUG:teuthology.orchestra.run.smithi161:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-21T03:33:49.254 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-09-21T03:33:49.254 DEBUG:teuthology.orchestra.run.smithi039:> set -ex 2024-09-21T03:33:49.254 DEBUG:teuthology.orchestra.run.smithi039:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-21T03:33:49.280 DEBUG:teuthology.orchestra.run.smithi039:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-21T03:33:49.345 DEBUG:teuthology.orchestra.run.smithi150:> set -ex 2024-09-21T03:33:49.346 DEBUG:teuthology.orchestra.run.smithi150:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-21T03:33:49.374 DEBUG:teuthology.orchestra.run.smithi150:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-21T03:33:49.441 DEBUG:teuthology.orchestra.run.smithi161:> set -ex 2024-09-21T03:33:49.441 DEBUG:teuthology.orchestra.run.smithi161:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-21T03:33:49.466 DEBUG:teuthology.orchestra.run.smithi161:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-21T03:33:49.531 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-09-21T03:33:49.531 DEBUG:teuthology.orchestra.run.smithi039:> set -ex 2024-09-21T03:33:49.531 DEBUG:teuthology.orchestra.run.smithi039:> sudo dd of=/usr/bin/stdin-killer 2024-09-21T03:33:49.558 DEBUG:teuthology.orchestra.run.smithi039:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-21T03:33:49.624 DEBUG:teuthology.orchestra.run.smithi150:> set -ex 2024-09-21T03:33:49.624 DEBUG:teuthology.orchestra.run.smithi150:> sudo dd of=/usr/bin/stdin-killer 2024-09-21T03:33:49.649 DEBUG:teuthology.orchestra.run.smithi150:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-21T03:33:49.714 DEBUG:teuthology.orchestra.run.smithi161:> set -ex 2024-09-21T03:33:49.714 DEBUG:teuthology.orchestra.run.smithi161:> sudo dd of=/usr/bin/stdin-killer 2024-09-21T03:33:49.739 DEBUG:teuthology.orchestra.run.smithi161:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-21T03:33:49.804 INFO:teuthology.run_tasks:Running task cephadm... 2024-09-21T03:33:49.896 INFO:tasks.cephadm:Config: {'roleless': False, 'cephfs': {'max_mds': 5, 'session_timeout': 300, 'standby_replay': True}, 'cluster-conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 2, 'mds bal fragment size max': 10000, 'mds bal merge size': 5, 'mds bal split bits': 3, 'mds bal split size': 100, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'client mount timeout': 30, 'debug cephsqlite': 20, 'debug client': 20, 'debug mgr': 20, 'debug ms': 1, 'mgr/snap_schedule/allow_m_granularity': True, 'mgr/snap_schedule/dump_on_update': True, 'mgr/snap_schedule/log_level': 'debug', 'mon warn on pool no app': False}, 'mon': {'mon op complaint time': 120}, 'osd': {'osd op complaint time': 180}}, 'conf': {'global': {'ms die on skipped message': False}, 'mds': {'mds_bal_replicate_threshold': 1, 'mds_export_ephemeral_random_max': 0.1}, 'mgr': {'cephsqlite lock renewal timeout': 900000, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd shutdown pgref assert': True, 'osd_max_omap_entries_per_request': 10}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'fs.*is degraded', 'filesystem is degraded', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'filesystem is offline', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'insufficient standby MDS daemons available', 'MDS_UP_LESS_THAN_MAX', 'online, but wants', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'do not have an application enabled', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability', 'Degraded data redundancy', 'slow metadata IO', 'SLOW_OPS', 'slow request'], 'mgr-modules': ['snap_schedule'], 'sha1': '1843221895adccfb3b692060251dd6a0183bc410', 'subvols': {'create': 2, 'subvol_options': '--namespace-isolated --size 25000000000'}} 2024-09-21T03:33:49.896 INFO:tasks.cephadm:Cluster image is quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 2024-09-21T03:33:49.897 INFO:tasks.cephadm:Cluster fsid is 51116d5c-77ca-11ef-baf5-efdc52797490 2024-09-21T03:33:49.897 INFO:tasks.cephadm:Choosing monitor IPs and ports... 2024-09-21T03:33:49.897 INFO:tasks.cephadm:Monitor IPs: {'mon.a': '172.21.15.39', 'mon.b': '172.21.15.150', 'mon.c': '172.21.15.161'} 2024-09-21T03:33:49.897 INFO:tasks.cephadm:First mon is mon.a on smithi039 2024-09-21T03:33:49.897 INFO:tasks.cephadm:First mgr is x 2024-09-21T03:33:49.898 INFO:tasks.cephadm:Normalizing hostnames... 2024-09-21T03:33:49.898 DEBUG:teuthology.orchestra.run.smithi039:> sudo hostname $(hostname -s) 2024-09-21T03:33:49.924 DEBUG:teuthology.orchestra.run.smithi150:> sudo hostname $(hostname -s) 2024-09-21T03:33:49.950 DEBUG:teuthology.orchestra.run.smithi161:> sudo hostname $(hostname -s) 2024-09-21T03:33:49.974 INFO:tasks.cephadm:Downloading "compiled" cephadm from cachra 2024-09-21T03:33:49.975 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=1843221895adccfb3b692060251dd6a0183bc410 2024-09-21T03:33:50.115 INFO:tasks.cephadm:builder_project result: [{'status': 'ready', 'sha1': '1843221895adccfb3b692060251dd6a0183bc410', 'extra': {'build_url': 'https://jenkins.ceph.com/job/ceph-dev-new-build/ARCH=arm64,AVAILABLE_ARCH=arm64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic/83190/', 'root_build_cause': 'SCMTRIGGER', 'version': '19.3.0-5120-g18432218', 'node_name': '172.21.4.75+confusa13', 'job_name': 'ceph-dev-new-build/ARCH=arm64,AVAILABLE_ARCH=arm64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic', 'package_manager_version': '19.3.0-5120.g18432218'}, 'url': 'https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/centos/9/flavors/default/', 'distro_codename': None, 'modified': '2024-09-21 00:04:17.484514', 'distro_version': '9', 'project': 'ceph', 'flavor': 'default', 'ref': 'wip-pdonnell-testing-20240920.212106-debug', 'chacra_url': 'https://4.chacra.ceph.com/repos/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/centos/9/flavors/default/', 'archs': ['x86_64', 'source', 'arm64'], 'distro': 'centos'}, {'status': 'ready', 'sha1': '1843221895adccfb3b692060251dd6a0183bc410', 'extra': {'build_url': 'https://jenkins.ceph.com/job/ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic/83190/', 'root_build_cause': 'SCMTRIGGER', 'version': '19.3.0-5120-g18432218', 'node_name': '172.21.2.22+braggi22', 'job_name': 'ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic', 'package_manager_version': '19.3.0-5120.g18432218'}, 'url': 'https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/centos/9/flavors/default/', 'distro_codename': None, 'modified': '2024-09-20 22:54:16.499090', 'distro_version': '9', 'project': 'ceph', 'flavor': 'default', 'ref': 'wip-pdonnell-testing-20240920.212106-debug', 'chacra_url': 'https://3.chacra.ceph.com/repos/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/centos/9/flavors/default/', 'archs': ['source', 'x86_64'], 'distro': 'centos'}] 2024-09-21T03:33:50.258 INFO:tasks.util.chacra:got chacra host 4.chacra.ceph.com, ref wip-pdonnell-testing-20240920.212106-debug, sha1 1843221895adccfb3b692060251dd6a0183bc410 from https://shaman.ceph.com/api/search/?project=ceph&distros=centos%2F9%2Fx86_64&flavor=default&sha1=1843221895adccfb3b692060251dd6a0183bc410 2024-09-21T03:33:50.260 INFO:tasks.cephadm:Discovered cachra url: https://4.chacra.ceph.com/binaries/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/centos/9/x86_64/flavors/default/cephadm 2024-09-21T03:33:50.260 INFO:tasks.cephadm:Downloading cephadm from url: https://4.chacra.ceph.com/binaries/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/centos/9/x86_64/flavors/default/cephadm 2024-09-21T03:33:50.260 DEBUG:teuthology.orchestra.run.smithi039:> curl --silent -L https://4.chacra.ceph.com/binaries/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-09-21T03:33:50.793 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r--. 1 ubuntu ubuntu 804283 Sep 21 03:33 /home/ubuntu/cephtest/cephadm 2024-09-21T03:33:50.793 DEBUG:teuthology.orchestra.run.smithi150:> curl --silent -L https://4.chacra.ceph.com/binaries/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-09-21T03:33:51.339 INFO:teuthology.orchestra.run.smithi150.stdout:-rw-r--r--. 1 ubuntu ubuntu 804283 Sep 21 03:33 /home/ubuntu/cephtest/cephadm 2024-09-21T03:33:51.339 DEBUG:teuthology.orchestra.run.smithi161:> curl --silent -L https://4.chacra.ceph.com/binaries/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-09-21T03:33:51.860 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r--. 1 ubuntu ubuntu 804283 Sep 21 03:33 /home/ubuntu/cephtest/cephadm 2024-09-21T03:33:51.861 DEBUG:teuthology.orchestra.run.smithi039:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-09-21T03:33:51.881 DEBUG:teuthology.orchestra.run.smithi150:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-09-21T03:33:51.901 DEBUG:teuthology.orchestra.run.smithi161:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-09-21T03:33:51.935 INFO:tasks.cephadm:Pulling image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 on all hosts... 2024-09-21T03:33:51.936 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 pull 2024-09-21T03:33:51.938 DEBUG:teuthology.orchestra.run.smithi150:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 pull 2024-09-21T03:33:51.944 DEBUG:teuthology.orchestra.run.smithi161:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 pull 2024-09-21T03:33:53.628 INFO:teuthology.orchestra.run.smithi150.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410... 2024-09-21T03:33:53.633 INFO:teuthology.orchestra.run.smithi039.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410... 2024-09-21T03:33:53.648 INFO:teuthology.orchestra.run.smithi161.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410... 2024-09-21T03:34:44.707 INFO:teuthology.orchestra.run.smithi161.stdout:{ 2024-09-21T03:34:44.707 INFO:teuthology.orchestra.run.smithi161.stdout: "ceph_version": "ceph version 19.3.0-5120-g18432218 (1843221895adccfb3b692060251dd6a0183bc410) squid (dev)", 2024-09-21T03:34:44.707 INFO:teuthology.orchestra.run.smithi161.stdout: "image_id": "ddce307e1f3263fb077b4f3902e15660050631dad4d2823a20ee3f72bf18d977", 2024-09-21T03:34:44.707 INFO:teuthology.orchestra.run.smithi161.stdout: "repo_digests": [ 2024-09-21T03:34:44.708 INFO:teuthology.orchestra.run.smithi161.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262" 2024-09-21T03:34:44.708 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-21T03:34:44.708 INFO:teuthology.orchestra.run.smithi161.stdout:} 2024-09-21T03:35:08.873 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-09-21T03:35:08.874 INFO:teuthology.orchestra.run.smithi039.stdout: "ceph_version": "ceph version 19.3.0-5120-g18432218 (1843221895adccfb3b692060251dd6a0183bc410) squid (dev)", 2024-09-21T03:35:08.874 INFO:teuthology.orchestra.run.smithi039.stdout: "image_id": "ddce307e1f3263fb077b4f3902e15660050631dad4d2823a20ee3f72bf18d977", 2024-09-21T03:35:08.874 INFO:teuthology.orchestra.run.smithi039.stdout: "repo_digests": [ 2024-09-21T03:35:08.874 INFO:teuthology.orchestra.run.smithi039.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262" 2024-09-21T03:35:08.874 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-21T03:35:08.874 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-09-21T03:35:12.382 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-21T03:35:12.382 INFO:teuthology.orchestra.run.smithi150.stdout: "ceph_version": "ceph version 19.3.0-5120-g18432218 (1843221895adccfb3b692060251dd6a0183bc410) squid (dev)", 2024-09-21T03:35:12.383 INFO:teuthology.orchestra.run.smithi150.stdout: "image_id": "ddce307e1f3263fb077b4f3902e15660050631dad4d2823a20ee3f72bf18d977", 2024-09-21T03:35:12.383 INFO:teuthology.orchestra.run.smithi150.stdout: "repo_digests": [ 2024-09-21T03:35:12.383 INFO:teuthology.orchestra.run.smithi150.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262" 2024-09-21T03:35:12.383 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T03:35:12.383 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-21T03:35:12.407 DEBUG:teuthology.orchestra.run.smithi039:> sudo mkdir -p /etc/ceph 2024-09-21T03:35:12.441 DEBUG:teuthology.orchestra.run.smithi150:> sudo mkdir -p /etc/ceph 2024-09-21T03:35:12.468 DEBUG:teuthology.orchestra.run.smithi161:> sudo mkdir -p /etc/ceph 2024-09-21T03:35:12.502 DEBUG:teuthology.orchestra.run.smithi039:> sudo chmod 777 /etc/ceph 2024-09-21T03:35:12.533 DEBUG:teuthology.orchestra.run.smithi150:> sudo chmod 777 /etc/ceph 2024-09-21T03:35:12.557 DEBUG:teuthology.orchestra.run.smithi161:> sudo chmod 777 /etc/ceph 2024-09-21T03:35:12.586 INFO:tasks.cephadm:Writing seed config... 2024-09-21T03:35:12.587 INFO:tasks.cephadm: override: [global] ms die on skipped message = False 2024-09-21T03:35:12.587 INFO:tasks.cephadm: override: [mds] mds_bal_replicate_threshold = 1 2024-09-21T03:35:12.588 INFO:tasks.cephadm: override: [mds] mds_export_ephemeral_random_max = 0.1 2024-09-21T03:35:12.588 INFO:tasks.cephadm: override: [mgr] cephsqlite lock renewal timeout = 900000 2024-09-21T03:35:12.588 INFO:tasks.cephadm: override: [mgr] debug mgr = 20 2024-09-21T03:35:12.588 INFO:tasks.cephadm: override: [mgr] debug ms = 1 2024-09-21T03:35:12.588 INFO:tasks.cephadm: override: [mon] debug mon = 20 2024-09-21T03:35:12.588 INFO:tasks.cephadm: override: [mon] debug ms = 1 2024-09-21T03:35:12.588 INFO:tasks.cephadm: override: [mon] debug paxos = 20 2024-09-21T03:35:12.588 INFO:tasks.cephadm: override: [mon] mon down mkfs grace = 300 2024-09-21T03:35:12.588 INFO:tasks.cephadm: override: [osd] bdev async discard = True 2024-09-21T03:35:12.588 INFO:tasks.cephadm: override: [osd] bdev enable discard = True 2024-09-21T03:35:12.588 INFO:tasks.cephadm: override: [osd] bluestore allocator = bitmap 2024-09-21T03:35:12.588 INFO:tasks.cephadm: override: [osd] bluestore block size = 96636764160 2024-09-21T03:35:12.588 INFO:tasks.cephadm: override: [osd] bluestore fsck on mount = True 2024-09-21T03:35:12.589 INFO:tasks.cephadm: override: [osd] debug bluefs = 1/20 2024-09-21T03:35:12.589 INFO:tasks.cephadm: override: [osd] debug bluestore = 1/20 2024-09-21T03:35:12.589 INFO:tasks.cephadm: override: [osd] debug ms = 1 2024-09-21T03:35:12.589 INFO:tasks.cephadm: override: [osd] debug osd = 20 2024-09-21T03:35:12.589 INFO:tasks.cephadm: override: [osd] debug rocksdb = 4/10 2024-09-21T03:35:12.589 INFO:tasks.cephadm: override: [osd] mon osd backfillfull_ratio = 0.85 2024-09-21T03:35:12.589 INFO:tasks.cephadm: override: [osd] mon osd full ratio = 0.9 2024-09-21T03:35:12.589 INFO:tasks.cephadm: override: [osd] mon osd nearfull ratio = 0.8 2024-09-21T03:35:12.589 INFO:tasks.cephadm: override: [osd] osd failsafe full ratio = 0.95 2024-09-21T03:35:12.589 INFO:tasks.cephadm: override: [osd] osd objectstore = bluestore 2024-09-21T03:35:12.589 INFO:tasks.cephadm: override: [osd] osd shutdown pgref assert = True 2024-09-21T03:35:12.589 INFO:tasks.cephadm: override: [osd] osd_max_omap_entries_per_request = 10 2024-09-21T03:35:12.590 DEBUG:teuthology.orchestra.run.smithi039:> set -ex 2024-09-21T03:35:12.590 DEBUG:teuthology.orchestra.run.smithi039:> dd of=/home/ubuntu/cephtest/seed.ceph.conf 2024-09-21T03:35:12.609 DEBUG:tasks.cephadm:Final config: [global] # make logging friendly to teuthology log_to_file = true log_to_stderr = false log to journald = false mon cluster log to file = true mon cluster log file level = debug mon clock drift allowed = 1.000 # replicate across OSDs, not hosts osd crush chooseleaf type = 0 #osd pool default size = 2 osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd # enable some debugging auth debug = true ms die on old message = true ms die on bug = true debug asserts on shutdown = true # adjust warnings mon max pg per osd = 10000# >= luminous mon pg warn max object skew = 0 mon osd allow primary affinity = true mon osd allow pg remap = true mon warn on legacy crush tunables = false mon warn on crush straw calc version zero = false mon warn on no sortbitwise = false mon warn on osd down out interval zero = false mon warn on too few osds = false mon_warn_on_pool_pg_num_not_power_of_two = false # disable pg_autoscaler by default for new pools osd_pool_default_pg_autoscale_mode = off # tests delete pools mon allow pool delete = true fsid = 51116d5c-77ca-11ef-baf5-efdc52797490 ms die on skipped message = False [osd] osd scrub load threshold = 5.0 osd scrub max interval = 600 osd mclock profile = high_recovery_ops osd recover clone overlap = true osd recovery max chunk = 1048576 osd deep scrub update digest min age = 30 osd map max advance = 10 osd memory target autotune = true # debugging osd debug shutdown = true osd debug op order = true osd debug verify stray on activate = true osd debug pg log writeout = true osd debug verify cached snaps = true osd debug verify missing on start = true osd debug misdirected ops = true osd op queue = debug_random osd op queue cut off = debug_random osd shutdown pgref assert = True bdev debug aio = true osd sloppy crc = true bdev async discard = True bdev enable discard = True bluestore allocator = bitmap bluestore block size = 96636764160 bluestore fsck on mount = True debug bluefs = 1/20 debug bluestore = 1/20 debug ms = 1 debug osd = 20 debug rocksdb = 4/10 mon osd backfillfull_ratio = 0.85 mon osd full ratio = 0.9 mon osd nearfull ratio = 0.8 osd failsafe full ratio = 0.95 osd objectstore = bluestore osd_max_omap_entries_per_request = 10 [mgr] mon reweight min pgs per osd = 4 mon reweight min bytes per osd = 10 mgr/telemetry/nag = false cephsqlite lock renewal timeout = 900000 debug mgr = 20 debug ms = 1 [mon] mon data avail warn = 5 mon mgr mkfs grace = 240 mon reweight min pgs per osd = 4 mon osd reporter subtree level = osd mon osd prime pg temp = true mon reweight min bytes per osd = 10 # rotate auth tickets quickly to exercise renewal paths auth mon ticket ttl = 660# 11m auth service ticket ttl = 240# 4m # don't complain about global id reclaim mon_warn_on_insecure_global_id_reclaim = false mon_warn_on_insecure_global_id_reclaim_allowed = false debug mon = 20 debug ms = 1 debug paxos = 20 mon down mkfs grace = 300 [client.rgw] rgw cache enabled = true rgw enable ops log = true rgw enable usage log = true [mds] mds_bal_replicate_threshold = 1 mds_export_ephemeral_random_max = 0.1 2024-09-21T03:35:12.609 DEBUG:teuthology.orchestra.run.smithi039:mon.a> sudo journalctl -f -n 0 -u ceph-51116d5c-77ca-11ef-baf5-efdc52797490@mon.a.service 2024-09-21T03:35:12.652 DEBUG:teuthology.orchestra.run.smithi039:mgr.x> sudo journalctl -f -n 0 -u ceph-51116d5c-77ca-11ef-baf5-efdc52797490@mgr.x.service 2024-09-21T03:35:12.695 INFO:tasks.cephadm:Bootstrapping... 2024-09-21T03:35:12.695 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 -v bootstrap --fsid 51116d5c-77ca-11ef-baf5-efdc52797490 --config /home/ubuntu/cephtest/seed.ceph.conf --output-config /etc/ceph/ceph.conf --output-keyring /etc/ceph/ceph.client.admin.keyring --output-pub-ssh-key /home/ubuntu/cephtest/ceph.pub --mon-id a --mgr-id x --orphan-initial-daemons --skip-monitoring-stack --mon-ip 172.21.15.39 --skip-admin-label && sudo chmod +r /etc/ceph/ceph.client.admin.keyring 2024-09-21T03:35:12.909 INFO:teuthology.orchestra.run.smithi039.stdout:-------------------------------------------------------------------------------- 2024-09-21T03:35:12.909 INFO:teuthology.orchestra.run.smithi039.stdout:cephadm ['--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410', '-v', 'bootstrap', '--fsid', '51116d5c-77ca-11ef-baf5-efdc52797490', '--config', '/home/ubuntu/cephtest/seed.ceph.conf', '--output-config', '/etc/ceph/ceph.conf', '--output-keyring', '/etc/ceph/ceph.client.admin.keyring', '--output-pub-ssh-key', '/home/ubuntu/cephtest/ceph.pub', '--mon-id', 'a', '--mgr-id', 'x', '--orphan-initial-daemons', '--skip-monitoring-stack', '--mon-ip', '172.21.15.39', '--skip-admin-label'] 2024-09-21T03:35:12.909 INFO:teuthology.orchestra.run.smithi039.stderr:Specifying an fsid for your cluster offers no advantages and may increase the likelihood of fsid conflicts. 2024-09-21T03:35:12.909 INFO:teuthology.orchestra.run.smithi039.stdout:Verifying podman|docker is present... 2024-09-21T03:35:12.933 INFO:teuthology.orchestra.run.smithi039.stdout:/bin/podman: stdout 5.2.2 2024-09-21T03:35:12.933 INFO:teuthology.orchestra.run.smithi039.stdout:Verifying lvm2 is present... 2024-09-21T03:35:12.933 INFO:teuthology.orchestra.run.smithi039.stdout:Verifying time synchronization is in place... 2024-09-21T03:35:12.940 INFO:teuthology.orchestra.run.smithi039.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-09-21T03:35:12.941 INFO:teuthology.orchestra.run.smithi039.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-09-21T03:35:12.947 INFO:teuthology.orchestra.run.smithi039.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-09-21T03:35:12.947 INFO:teuthology.orchestra.run.smithi039.stdout:systemctl: stdout inactive 2024-09-21T03:35:12.954 INFO:teuthology.orchestra.run.smithi039.stdout:systemctl: stdout enabled 2024-09-21T03:35:12.961 INFO:teuthology.orchestra.run.smithi039.stdout:systemctl: stdout active 2024-09-21T03:35:12.961 INFO:teuthology.orchestra.run.smithi039.stdout:Unit chronyd.service is enabled and running 2024-09-21T03:35:12.961 INFO:teuthology.orchestra.run.smithi039.stdout:Repeating the final host check... 2024-09-21T03:35:12.989 INFO:teuthology.orchestra.run.smithi039.stdout:/bin/podman: stdout 5.2.2 2024-09-21T03:35:12.990 INFO:teuthology.orchestra.run.smithi039.stdout:podman (/bin/podman) version 5.2.2 is present 2024-09-21T03:35:12.990 INFO:teuthology.orchestra.run.smithi039.stdout:systemctl is present 2024-09-21T03:35:12.990 INFO:teuthology.orchestra.run.smithi039.stdout:lvcreate is present 2024-09-21T03:35:12.996 INFO:teuthology.orchestra.run.smithi039.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-09-21T03:35:12.996 INFO:teuthology.orchestra.run.smithi039.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-09-21T03:35:13.003 INFO:teuthology.orchestra.run.smithi039.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-09-21T03:35:13.003 INFO:teuthology.orchestra.run.smithi039.stdout:systemctl: stdout inactive 2024-09-21T03:35:13.010 INFO:teuthology.orchestra.run.smithi039.stdout:systemctl: stdout enabled 2024-09-21T03:35:13.016 INFO:teuthology.orchestra.run.smithi039.stdout:systemctl: stdout active 2024-09-21T03:35:13.017 INFO:teuthology.orchestra.run.smithi039.stdout:Unit chronyd.service is enabled and running 2024-09-21T03:35:13.017 INFO:teuthology.orchestra.run.smithi039.stdout:Host looks OK 2024-09-21T03:35:13.017 INFO:teuthology.orchestra.run.smithi039.stdout:Cluster fsid: 51116d5c-77ca-11ef-baf5-efdc52797490 2024-09-21T03:35:13.017 INFO:teuthology.orchestra.run.smithi039.stdout:Acquiring lock 140050490343040 on /run/cephadm/51116d5c-77ca-11ef-baf5-efdc52797490.lock 2024-09-21T03:35:13.017 INFO:teuthology.orchestra.run.smithi039.stdout:Lock 140050490343040 acquired on /run/cephadm/51116d5c-77ca-11ef-baf5-efdc52797490.lock 2024-09-21T03:35:13.017 INFO:teuthology.orchestra.run.smithi039.stdout:Verifying IP 172.21.15.39 port 3300 ... 2024-09-21T03:35:13.018 INFO:teuthology.orchestra.run.smithi039.stdout:Verifying IP 172.21.15.39 port 6789 ... 2024-09-21T03:35:13.018 INFO:teuthology.orchestra.run.smithi039.stdout:Base mon IP(s) is [172.21.15.39:3300, 172.21.15.39:6789], mon addrv is [v2:172.21.15.39:3300,v1:172.21.15.39:6789] 2024-09-21T03:35:13.021 INFO:teuthology.orchestra.run.smithi039.stdout:/sbin/ip: stdout default via 172.21.15.254 dev ens1f0 proto dhcp src 172.21.15.39 metric 100 2024-09-21T03:35:13.021 INFO:teuthology.orchestra.run.smithi039.stdout:/sbin/ip: stdout 172.21.0.0/20 dev ens1f0 proto kernel scope link src 172.21.15.39 metric 100 2024-09-21T03:35:13.023 INFO:teuthology.orchestra.run.smithi039.stdout:/sbin/ip: stdout ::1 dev lo proto kernel metric 256 pref medium 2024-09-21T03:35:13.024 INFO:teuthology.orchestra.run.smithi039.stdout:/sbin/ip: stdout fe80::/64 dev ens1f0 proto kernel metric 256 pref medium 2024-09-21T03:35:13.024 INFO:teuthology.orchestra.run.smithi039.stdout:/sbin/ip: stdout default via fe80::327c:5e00:6487:71e0 dev ens1f0 proto ra metric 1024 expires 1796sec hoplimit 64 pref medium 2024-09-21T03:35:13.026 INFO:teuthology.orchestra.run.smithi039.stdout:/sbin/ip: stdout 1: lo: mtu 65536 state UNKNOWN qlen 1000 2024-09-21T03:35:13.026 INFO:teuthology.orchestra.run.smithi039.stdout:/sbin/ip: stdout inet6 ::1/128 scope host 2024-09-21T03:35:13.026 INFO:teuthology.orchestra.run.smithi039.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-09-21T03:35:13.026 INFO:teuthology.orchestra.run.smithi039.stdout:/sbin/ip: stdout 4: ens1f0: mtu 1500 state UP qlen 1000 2024-09-21T03:35:13.026 INFO:teuthology.orchestra.run.smithi039.stdout:/sbin/ip: stdout inet6 fe80::ec4:7aff:febd:1370/64 scope link 2024-09-21T03:35:13.026 INFO:teuthology.orchestra.run.smithi039.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-09-21T03:35:13.027 INFO:teuthology.orchestra.run.smithi039.stdout:Mon IP `172.21.15.39` is in CIDR network `172.21.0.0/20` 2024-09-21T03:35:13.027 INFO:teuthology.orchestra.run.smithi039.stdout:Mon IP `172.21.15.39` is in CIDR network `172.21.0.0/20` 2024-09-21T03:35:13.027 INFO:teuthology.orchestra.run.smithi039.stdout:Inferred mon public CIDR from local network configuration ['172.21.0.0/20', '172.21.0.0/20'] 2024-09-21T03:35:13.027 INFO:teuthology.orchestra.run.smithi039.stdout:Internal network (--cluster-network) has not been provided, OSD replication will default to the public_network 2024-09-21T03:35:13.028 INFO:teuthology.orchestra.run.smithi039.stdout:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410... 2024-09-21T03:35:14.185 INFO:teuthology.orchestra.run.smithi039.stdout:/bin/podman: stdout ddce307e1f3263fb077b4f3902e15660050631dad4d2823a20ee3f72bf18d977 2024-09-21T03:35:14.185 INFO:teuthology.orchestra.run.smithi039.stdout:/bin/podman: stderr Trying to pull quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410... 2024-09-21T03:35:14.185 INFO:teuthology.orchestra.run.smithi039.stdout:/bin/podman: stderr Getting image source signatures 2024-09-21T03:35:14.185 INFO:teuthology.orchestra.run.smithi039.stdout:/bin/podman: stderr Copying blob sha256:8425bc7408d91b180e7fdc62f23e7ae6603757f4f36e9969a4eaf9fd9416b764 2024-09-21T03:35:14.185 INFO:teuthology.orchestra.run.smithi039.stdout:/bin/podman: stderr Copying blob sha256:e8b54c863393b7a8216a71737771b73b16a8e43ec7acf927c7faa175c255e551 2024-09-21T03:35:14.186 INFO:teuthology.orchestra.run.smithi039.stdout:/bin/podman: stderr Copying config sha256:ddce307e1f3263fb077b4f3902e15660050631dad4d2823a20ee3f72bf18d977 2024-09-21T03:35:14.186 INFO:teuthology.orchestra.run.smithi039.stdout:/bin/podman: stderr Writing manifest to image destination 2024-09-21T03:35:15.165 INFO:teuthology.orchestra.run.smithi039.stdout:ceph: stdout ceph version 19.3.0-5120-g18432218 (1843221895adccfb3b692060251dd6a0183bc410) squid (dev) 2024-09-21T03:35:15.165 INFO:teuthology.orchestra.run.smithi039.stdout:Ceph version: ceph version 19.3.0-5120-g18432218 (1843221895adccfb3b692060251dd6a0183bc410) squid (dev) 2024-09-21T03:35:15.166 INFO:teuthology.orchestra.run.smithi039.stdout:Extracting ceph user uid/gid from container image... 2024-09-21T03:35:16.063 INFO:teuthology.orchestra.run.smithi039.stdout:stat: stdout 167 167 2024-09-21T03:35:16.063 INFO:teuthology.orchestra.run.smithi039.stdout:Creating initial keys... 2024-09-21T03:35:16.918 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph-authtool: stdout AQD0Pu5moCplFxAA8q1nN0iL+fOKwQ9+XOck6A== 2024-09-21T03:35:17.849 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph-authtool: stdout AQD1Pu5mEIC7EBAAagYEwj8UPaTdFu+x2MrwpQ== 2024-09-21T03:35:18.773 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph-authtool: stdout AQD2Pu5m2wa/CxAABQwgs3ng+gXhk0ptCwfK7Q== 2024-09-21T03:35:18.774 INFO:teuthology.orchestra.run.smithi039.stdout:Creating initial monmap... 2024-09-21T03:35:27.528 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: monmap file /tmp/monmap 2024-09-21T03:35:27.528 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/monmaptool: stdout setting min_mon_release = quincy 2024-09-21T03:35:27.528 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: set fsid to 51116d5c-77ca-11ef-baf5-efdc52797490 2024-09-21T03:35:27.528 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-09-21T03:35:27.529 INFO:teuthology.orchestra.run.smithi039.stdout:monmaptool for a [v2:172.21.15.39:3300,v1:172.21.15.39:6789] on /usr/bin/monmaptool: monmap file /tmp/monmap 2024-09-21T03:35:27.529 INFO:teuthology.orchestra.run.smithi039.stdout:setting min_mon_release = quincy 2024-09-21T03:35:27.529 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/monmaptool: set fsid to 51116d5c-77ca-11ef-baf5-efdc52797490 2024-09-21T03:35:27.529 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-09-21T03:35:27.529 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T03:35:27.529 INFO:teuthology.orchestra.run.smithi039.stdout:Creating mon... 2024-09-21T03:35:32.016 INFO:teuthology.orchestra.run.smithi039.stdout:create mon.a on 2024-09-21T03:35:32.225 INFO:teuthology.orchestra.run.smithi039.stdout:systemctl: stderr Removed "/etc/systemd/system/multi-user.target.wants/ceph.target". 2024-09-21T03:35:32.421 INFO:teuthology.orchestra.run.smithi039.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /etc/systemd/system/ceph.target. 2024-09-21T03:35:32.614 INFO:teuthology.orchestra.run.smithi039.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph-51116d5c-77ca-11ef-baf5-efdc52797490.target → /etc/systemd/system/ceph-51116d5c-77ca-11ef-baf5-efdc52797490.target. 2024-09-21T03:35:32.614 INFO:teuthology.orchestra.run.smithi039.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph.target.wants/ceph-51116d5c-77ca-11ef-baf5-efdc52797490.target → /etc/systemd/system/ceph-51116d5c-77ca-11ef-baf5-efdc52797490.target. 2024-09-21T03:35:32.915 INFO:teuthology.orchestra.run.smithi039.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-51116d5c-77ca-11ef-baf5-efdc52797490@mon.a 2024-09-21T03:35:32.916 INFO:teuthology.orchestra.run.smithi039.stdout:systemctl: stderr Failed to reset failed state of unit ceph-51116d5c-77ca-11ef-baf5-efdc52797490@mon.a.service: Unit ceph-51116d5c-77ca-11ef-baf5-efdc52797490@mon.a.service not loaded. 2024-09-21T03:35:33.093 INFO:teuthology.orchestra.run.smithi039.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-51116d5c-77ca-11ef-baf5-efdc52797490.target.wants/ceph-51116d5c-77ca-11ef-baf5-efdc52797490@mon.a.service → /etc/systemd/system/ceph-51116d5c-77ca-11ef-baf5-efdc52797490@.service. 2024-09-21T03:35:33.584 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:35:33 smithi039 podman[30832]: 2024-09-21 03:35:33.579200388 +0000 UTC m=+0.366066663 container start 80e551051dd9506bf0d3a21cb948fee0d48cb38ffb89f2d723410ba373f29346 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410, name=ceph-51116d5c-77ca-11ef-baf5-efdc52797490-mon-a, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_BRANCH=HEAD, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, ceph=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0) 2024-09-21T03:35:33.690 INFO:teuthology.orchestra.run.smithi039.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-09-21T03:35:33.691 INFO:teuthology.orchestra.run.smithi039.stdout:systemctl: stdout disabled 2024-09-21T03:35:33.698 INFO:teuthology.orchestra.run.smithi039.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-09-21T03:35:33.698 INFO:teuthology.orchestra.run.smithi039.stdout:systemctl: stdout inactive 2024-09-21T03:35:33.698 INFO:teuthology.orchestra.run.smithi039.stdout:firewalld.service is not enabled 2024-09-21T03:35:33.698 INFO:teuthology.orchestra.run.smithi039.stdout:Not possible to enable service . firewalld.service is not available 2024-09-21T03:35:33.698 INFO:teuthology.orchestra.run.smithi039.stdout:Waiting for mon to start... 2024-09-21T03:35:33.698 INFO:teuthology.orchestra.run.smithi039.stdout:Waiting for mon... 2024-09-21T03:35:33.912 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:35:33 smithi039 ceph-mon[30850]: mkfs 51116d5c-77ca-11ef-baf5-efdc52797490 2024-09-21T03:35:33.913 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:35:33 smithi039 ceph-mon[30850]: mon.a is new leader, mons a in quorum (ranks 0) 2024-09-21T03:35:33.913 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:35:33 smithi039 bash[30832]: 80e551051dd9506bf0d3a21cb948fee0d48cb38ffb89f2d723410ba373f29346 2024-09-21T03:35:33.913 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:35:33 smithi039 systemd[1]: Started Ceph mon.a for 51116d5c-77ca-11ef-baf5-efdc52797490. 2024-09-21T03:35:34.734 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:35:34 smithi039 ceph-mon[30850]: mon.a is new leader, mons a in quorum (ranks 0) 2024-09-21T03:35:34.734 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:35:34 smithi039 ceph-mon[30850]: monmap epoch 1 2024-09-21T03:35:34.734 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:35:34 smithi039 ceph-mon[30850]: fsid 51116d5c-77ca-11ef-baf5-efdc52797490 2024-09-21T03:35:34.734 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:35:34 smithi039 ceph-mon[30850]: last_changed 2024-09-21T03:35:21.020316+0000 2024-09-21T03:35:34.734 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:35:34 smithi039 ceph-mon[30850]: created 2024-09-21T03:35:21.020316+0000 2024-09-21T03:35:34.734 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:35:34 smithi039 ceph-mon[30850]: min_mon_release 19 (squid) 2024-09-21T03:35:34.734 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:35:34 smithi039 ceph-mon[30850]: election_strategy: 1 2024-09-21T03:35:34.734 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:35:34 smithi039 ceph-mon[30850]: 0: [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] mon.a 2024-09-21T03:35:34.734 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:35:34 smithi039 ceph-mon[30850]: fsmap 2024-09-21T03:35:34.734 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:35:34 smithi039 ceph-mon[30850]: osdmap e1: 0 total, 0 up, 0 in 2024-09-21T03:35:34.734 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:35:34 smithi039 ceph-mon[30850]: mgrmap e1: no daemons active 2024-09-21T03:35:34.735 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:35:34 smithi039 ceph-mon[30850]: from='client.? 172.21.15.39:0/1486331635' entity='client.admin' cmd={"prefix": "status"} : dispatch 2024-09-21T03:35:34.736 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout cluster: 2024-09-21T03:35:34.737 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout id: 51116d5c-77ca-11ef-baf5-efdc52797490 2024-09-21T03:35:34.737 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout health: HEALTH_OK 2024-09-21T03:35:34.737 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout 2024-09-21T03:35:34.737 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout services: 2024-09-21T03:35:34.737 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout mon: 1 daemons, quorum a (age 0.551661s) 2024-09-21T03:35:34.738 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout mgr: no daemons active 2024-09-21T03:35:34.738 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout osd: 0 osds: 0 up, 0 in 2024-09-21T03:35:34.738 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout 2024-09-21T03:35:34.738 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout data: 2024-09-21T03:35:34.738 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout pools: 0 pools, 0 pgs 2024-09-21T03:35:34.738 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout objects: 0 objects, 0 B 2024-09-21T03:35:34.738 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout usage: 0 B used, 0 B / 0 B avail 2024-09-21T03:35:34.738 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout pgs: 2024-09-21T03:35:34.738 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout 2024-09-21T03:35:34.738 INFO:teuthology.orchestra.run.smithi039.stdout:mon is available 2024-09-21T03:35:34.738 INFO:teuthology.orchestra.run.smithi039.stdout:Assimilating anything we can from ceph.conf... 2024-09-21T03:35:35.833 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:35:35 smithi039 ceph-mon[30850]: from='client.? 172.21.15.39:0/2710672063' entity='client.admin' cmd={"prefix": "config assimilate-conf"} : dispatch 2024-09-21T03:35:35.833 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:35:35 smithi039 ceph-mon[30850]: from='client.? 172.21.15.39:0/2710672063' entity='client.admin' cmd='[{"prefix": "config assimilate-conf"}]': finished 2024-09-21T03:35:36.103 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout 2024-09-21T03:35:36.103 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout [global] 2024-09-21T03:35:36.104 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout fsid = 51116d5c-77ca-11ef-baf5-efdc52797490 2024-09-21T03:35:36.104 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout mon_cluster_log_file_level = debug 2024-09-21T03:35:36.104 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout mon_host = [v2:172.21.15.39:3300,v1:172.21.15.39:6789] 2024-09-21T03:35:36.104 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-09-21T03:35:36.104 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-09-21T03:35:36.104 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-09-21T03:35:36.104 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-09-21T03:35:36.104 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout 2024-09-21T03:35:36.104 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout [mgr] 2024-09-21T03:35:36.104 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-09-21T03:35:36.104 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout 2024-09-21T03:35:36.104 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout [osd] 2024-09-21T03:35:36.104 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout bdev_async_discard = True 2024-09-21T03:35:36.104 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-09-21T03:35:36.105 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-09-21T03:35:36.105 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-09-21T03:35:36.105 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-09-21T03:35:36.105 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-09-21T03:35:36.105 INFO:teuthology.orchestra.run.smithi039.stdout:Generating new minimal ceph.conf... 2024-09-21T03:35:36.833 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:35:36 smithi039 ceph-mon[30850]: from='client.? 172.21.15.39:0/4229882339' entity='client.admin' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:35:37.940 INFO:teuthology.orchestra.run.smithi039.stdout:Restarting the monitor... 2024-09-21T03:35:38.223 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:35:37 smithi039 systemd[1]: Stopping Ceph mon.a for 51116d5c-77ca-11ef-baf5-efdc52797490... 2024-09-21T03:35:38.565 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:35:38 smithi039 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-mon-a[30846]: 2024-09-21T03:35:38.221+0000 7f2b8231c640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-09-21T03:35:38.565 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:35:38 smithi039 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-mon-a[30846]: 2024-09-21T03:35:38.221+0000 7f2b8231c640 -1 mon.a@0(leader) e1 *** Got Signal Terminated *** 2024-09-21T03:35:38.565 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:35:38 smithi039 podman[31092]: 2024-09-21 03:35:38.283450822 +0000 UTC m=+0.167311212 container died 80e551051dd9506bf0d3a21cb948fee0d48cb38ffb89f2d723410ba373f29346 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410, name=ceph-51116d5c-77ca-11ef-baf5-efdc52797490-mon-a, maintainer=Guillaume Abrioux , ceph=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.schema-version=1.0) 2024-09-21T03:35:38.833 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:35:38 smithi039 podman[31092]: 2024-09-21 03:35:38.671781775 +0000 UTC m=+0.555642152 container cleanup 80e551051dd9506bf0d3a21cb948fee0d48cb38ffb89f2d723410ba373f29346 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410, name=ceph-51116d5c-77ca-11ef-baf5-efdc52797490-mon-a, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, ceph=True, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0) 2024-09-21T03:35:38.834 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:35:38 smithi039 bash[31092]: ceph-51116d5c-77ca-11ef-baf5-efdc52797490-mon-a 2024-09-21T03:35:39.300 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:35:38 smithi039 podman[31104]: 2024-09-21 03:35:38.956374483 +0000 UTC m=+0.671747600 container remove 80e551051dd9506bf0d3a21cb948fee0d48cb38ffb89f2d723410ba373f29346 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410, name=ceph-51116d5c-77ca-11ef-baf5-efdc52797490-mon-a, ceph=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.build-date=20240731) 2024-09-21T03:35:39.300 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:35:39 smithi039 systemd[1]: ceph-51116d5c-77ca-11ef-baf5-efdc52797490@mon.a.service: Deactivated successfully. 2024-09-21T03:35:39.300 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:35:39 smithi039 systemd[1]: Stopped Ceph mon.a for 51116d5c-77ca-11ef-baf5-efdc52797490. 2024-09-21T03:35:39.300 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:35:39 smithi039 systemd[1]: Starting Ceph mon.a for 51116d5c-77ca-11ef-baf5-efdc52797490... 2024-09-21T03:35:39.554 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:35:39 smithi039 podman[31192]: 2024-09-21 03:35:39.298843715 +0000 UTC m=+0.103742640 container create 861e490fc665c0095d7e9950a2d6ccd4b7d704a1d4b24d58cda56cf79cda489f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410, name=ceph-51116d5c-77ca-11ef-baf5-efdc52797490-mon-a, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD) 2024-09-21T03:35:39.554 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:35:39 smithi039 podman[31192]: 2024-09-21 03:35:39.211102318 +0000 UTC m=+0.016001255 image pull ddce307e1f3263fb077b4f3902e15660050631dad4d2823a20ee3f72bf18d977 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 2024-09-21T03:35:39.554 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:35:39 smithi039 podman[31192]: 2024-09-21 03:35:39.517858895 +0000 UTC m=+0.322757833 container init 861e490fc665c0095d7e9950a2d6ccd4b7d704a1d4b24d58cda56cf79cda489f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410, name=ceph-51116d5c-77ca-11ef-baf5-efdc52797490-mon-a, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, ceph=True, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, org.label-schema.build-date=20240731) 2024-09-21T03:35:39.554 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:35:39 smithi039 podman[31192]: 2024-09-21 03:35:39.521063007 +0000 UTC m=+0.325961935 container start 861e490fc665c0095d7e9950a2d6ccd4b7d704a1d4b24d58cda56cf79cda489f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410, name=ceph-51116d5c-77ca-11ef-baf5-efdc52797490-mon-a, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, ceph=True, io.buildah.version=1.37.2, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23) 2024-09-21T03:35:39.581 INFO:teuthology.orchestra.run.smithi039.stdout:Setting public_network to 172.21.0.0/20 in mon config section 2024-09-21T03:35:39.806 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:35:39 smithi039 ceph-mon[31211]: set uid:gid to 167:167 (ceph:ceph) 2024-09-21T03:35:39.806 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:35:39 smithi039 ceph-mon[31211]: ceph version 19.3.0-5120-g18432218 (1843221895adccfb3b692060251dd6a0183bc410) squid (dev), process ceph-mon, pid 2 2024-09-21T03:35:39.806 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:35:39 smithi039 ceph-mon[31211]: pidfile_write: ignore empty --pid-file 2024-09-21T03:35:39.806 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:35:39 smithi039 ceph-mon[31211]: load: jerasure load: lrc 2024-09-21T03:35:39.806 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:35:39 smithi039 ceph-mon[31211]: rocksdb: RocksDB version: 7.9.2 2024-09-21T03:35:39.806 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:35:39 smithi039 ceph-mon[31211]: rocksdb: Git sha 0 2024-09-21T03:35:39.807 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:35:39 smithi039 ceph-mon[31211]: rocksdb: Compile date 2024-09-20 22:29:49 2024-09-21T03:35:39.807 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:35:39 smithi039 ceph-mon[31211]: rocksdb: DB SUMMARY 2024-09-21T03:35:39.807 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:35:39 smithi039 ceph-mon[31211]: rocksdb: DB Session ID: OZGA2F5ORY09AFE06L52 2024-09-21T03:35:39.807 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:35:39 smithi039 ceph-mon[31211]: rocksdb: CURRENT file: CURRENT 2024-09-21T03:35:39.807 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:35:39 smithi039 ceph-mon[31211]: rocksdb: IDENTITY file: IDENTITY 2024-09-21T03:35:39.807 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:35:39 smithi039 ceph-mon[31211]: rocksdb: MANIFEST file: MANIFEST-000010 size: 179 Bytes 2024-09-21T03:35:39.807 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:35:39 smithi039 ceph-mon[31211]: rocksdb: SST files in /var/lib/ceph/mon/ceph-a/store.db dir, Total Num: 1, files: 000008.sst 2024-09-21T03:35:39.807 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:35:39 smithi039 ceph-mon[31211]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-a/store.db: 000009.log size: 95750 ; 2024-09-21T03:35:39.807 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:35:39 smithi039 ceph-mon[31211]: rocksdb: Options.error_if_exists: 0 2024-09-21T03:35:39.807 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:35:39 smithi039 ceph-mon[31211]: rocksdb: Options.create_if_missing: 0 2024-09-21T03:35:39.807 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:35:39 smithi039 ceph-mon[31211]: rocksdb: Options.paranoid_checks: 1 2024-09-21T03:35:39.807 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:35:39 smithi039 ceph-mon[31211]: rocksdb: Options.flush_verify_memtable_count: 1 2024-09-21T03:35:39.807 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:35:39 smithi039 ceph-mon[31211]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-09-21T03:35:39.807 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:35:39 smithi039 ceph-mon[31211]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-09-21T03:35:39.808 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:35:39 smithi039 ceph-mon[31211]: rocksdb: Options.env: 0x56507cbcad60 2024-09-21T03:35:39.808 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:35:39 smithi039 ceph-mon[31211]: rocksdb: Options.fs: PosixFileSystem 2024-09-21T03:35:39.808 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:35:39 smithi039 ceph-mon[31211]: rocksdb: Options.info_log: 0x5650816e5360 2024-09-21T03:35:39.808 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:35:39 smithi039 ceph-mon[31211]: rocksdb: Options.max_file_opening_threads: 16 2024-09-21T03:35:39.808 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:35:39 smithi039 ceph-mon[31211]: rocksdb: Options.statistics: (nil) 2024-09-21T03:35:39.808 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:35:39 smithi039 ceph-mon[31211]: rocksdb: Options.use_fsync: 0 2024-09-21T03:35:39.808 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:35:39 smithi039 ceph-mon[31211]: rocksdb: Options.max_log_file_size: 0 2024-09-21T03:35:39.808 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:35:39 smithi039 ceph-mon[31211]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-09-21T03:35:39.808 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:35:39 smithi039 ceph-mon[31211]: rocksdb: Options.log_file_time_to_roll: 0 2024-09-21T03:35:39.808 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:35:39 smithi039 ceph-mon[31211]: rocksdb: Options.keep_log_file_num: 1000 2024-09-21T03:35:39.808 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:35:39 smithi039 ceph-mon[31211]: rocksdb: Options.recycle_log_file_num: 0 2024-09-21T03:35:39.809 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:35:39 smithi039 ceph-mon[31211]: rocksdb: Options.allow_fallocate: 1 2024-09-21T03:35:39.809 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:35:39 smithi039 ceph-mon[31211]: rocksdb: Options.allow_mmap_reads: 0 2024-09-21T03:35:39.809 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:35:39 smithi039 ceph-mon[31211]: rocksdb: Options.allow_mmap_writes: 0 2024-09-21T03:35:39.809 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:35:39 smithi039 ceph-mon[31211]: rocksdb: Options.use_direct_reads: 0 2024-09-21T03:35:39.809 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:35:39 smithi039 ceph-mon[31211]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-09-21T03:35:39.809 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:35:39 smithi039 ceph-mon[31211]: rocksdb: Options.create_missing_column_families: 0 2024-09-21T03:35:39.809 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:35:39 smithi039 ceph-mon[31211]: rocksdb: Options.db_log_dir: 2024-09-21T03:35:39.809 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:35:39 smithi039 ceph-mon[31211]: rocksdb: Options.wal_dir: 2024-09-21T03:35:39.809 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:35:39 smithi039 ceph-mon[31211]: rocksdb: Options.table_cache_numshardbits: 6 2024-09-21T03:35:39.811 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:35:39 smithi039 ceph-mon[31211]: rocksdb: Options.WAL_ttl_seconds: 0 2024-09-21T03:35:39.811 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:35:39 smithi039 ceph-mon[31211]: rocksdb: Options.WAL_size_limit_MB: 0 2024-09-21T03:35:39.811 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:35:39 smithi039 ceph-mon[31211]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-09-21T03:35:39.811 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:35:39 smithi039 ceph-mon[31211]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-09-21T03:35:39.811 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:35:39 smithi039 ceph-mon[31211]: rocksdb: Options.is_fd_close_on_exec: 1 2024-09-21T03:35:39.811 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:35:39 smithi039 ceph-mon[31211]: rocksdb: Options.advise_random_on_open: 1 2024-09-21T03:35:39.811 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:35:39 smithi039 ceph-mon[31211]: rocksdb: Options.db_write_buffer_size: 0 2024-09-21T03:35:39.811 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:35:39 smithi039 ceph-mon[31211]: rocksdb: Options.write_buffer_manager: 0x5650816e9d60 2024-09-21T03:35:39.811 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:35:39 smithi039 ceph-mon[31211]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-09-21T03:35:39.812 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:35:39 smithi039 ceph-mon[31211]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-09-21T03:35:39.812 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:35:39 smithi039 ceph-mon[31211]: rocksdb: Options.use_adaptive_mutex: 0 2024-09-21T03:35:39.812 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:35:39 smithi039 ceph-mon[31211]: rocksdb: Options.rate_limiter: (nil) 2024-09-21T03:35:39.812 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:35:39 smithi039 ceph-mon[31211]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-09-21T03:35:39.812 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:35:39 smithi039 ceph-mon[31211]: rocksdb: Options.wal_recovery_mode: 2 2024-09-21T03:35:39.812 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:35:39 smithi039 ceph-mon[31211]: rocksdb: Options.enable_thread_tracking: 0 2024-09-21T03:35:39.812 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:35:39 smithi039 ceph-mon[31211]: rocksdb: Options.enable_pipelined_write: 0 2024-09-21T03:35:39.812 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:35:39 smithi039 ceph-mon[31211]: rocksdb: Options.unordered_write: 0 2024-09-21T03:35:39.812 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:35:39 smithi039 ceph-mon[31211]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-09-21T03:35:39.812 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:35:39 smithi039 ceph-mon[31211]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-09-21T03:35:39.812 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:35:39 smithi039 ceph-mon[31211]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-09-21T03:35:39.812 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:35:39 smithi039 ceph-mon[31211]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-09-21T03:35:39.812 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:35:39 smithi039 ceph-mon[31211]: rocksdb: Options.row_cache: None 2024-09-21T03:35:39.813 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:35:39 smithi039 ceph-mon[31211]: rocksdb: Options.wal_filter: None 2024-09-21T03:35:39.813 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:35:39 smithi039 ceph-mon[31211]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-09-21T03:35:39.813 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:35:39 smithi039 ceph-mon[31211]: rocksdb: Options.allow_ingest_behind: 0 2024-09-21T03:35:39.813 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:35:39 smithi039 ceph-mon[31211]: rocksdb: Options.two_write_queues: 0 2024-09-21T03:35:39.813 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:35:39 smithi039 ceph-mon[31211]: rocksdb: Options.manual_wal_flush: 0 2024-09-21T03:35:39.813 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:35:39 smithi039 ceph-mon[31211]: rocksdb: Options.wal_compression: 0 2024-09-21T03:35:39.813 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:35:39 smithi039 ceph-mon[31211]: rocksdb: Options.atomic_flush: 0 2024-09-21T03:35:39.813 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:35:39 smithi039 ceph-mon[31211]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-09-21T03:35:39.813 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:35:39 smithi039 ceph-mon[31211]: rocksdb: Options.persist_stats_to_disk: 0 2024-09-21T03:35:39.813 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:35:39 smithi039 ceph-mon[31211]: rocksdb: Options.write_dbid_to_manifest: 0 2024-09-21T03:35:39.814 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:35:39 smithi039 ceph-mon[31211]: rocksdb: Options.log_readahead_size: 0 2024-09-21T03:35:39.814 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:35:39 smithi039 ceph-mon[31211]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-09-21T03:35:39.814 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:35:39 smithi039 ceph-mon[31211]: rocksdb: Options.best_efforts_recovery: 0 2024-09-21T03:35:39.814 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:35:39 smithi039 ceph-mon[31211]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-09-21T03:35:39.814 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:35:39 smithi039 ceph-mon[31211]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-09-21T03:35:39.814 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:35:39 smithi039 ceph-mon[31211]: rocksdb: Options.allow_data_in_errors: 0 2024-09-21T03:35:39.814 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:35:39 smithi039 ceph-mon[31211]: rocksdb: Options.db_host_id: __hostname__ 2024-09-21T03:35:39.814 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:35:39 smithi039 ceph-mon[31211]: rocksdb: Options.enforce_single_del_contracts: true 2024-09-21T03:35:39.814 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:35:39 smithi039 ceph-mon[31211]: rocksdb: Options.max_background_jobs: 2 2024-09-21T03:35:39.814 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:35:39 smithi039 ceph-mon[31211]: rocksdb: Options.max_background_compactions: -1 2024-09-21T03:35:39.814 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:35:39 smithi039 ceph-mon[31211]: rocksdb: Options.max_subcompactions: 1 2024-09-21T03:35:39.815 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:35:39 smithi039 ceph-mon[31211]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-09-21T03:35:39.815 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:35:39 smithi039 ceph-mon[31211]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-09-21T03:35:39.815 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:35:39 smithi039 ceph-mon[31211]: rocksdb: Options.delayed_write_rate : 16777216 2024-09-21T03:35:39.815 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:35:39 smithi039 ceph-mon[31211]: rocksdb: Options.max_total_wal_size: 0 2024-09-21T03:35:39.815 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:35:39 smithi039 ceph-mon[31211]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-09-21T03:35:39.815 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:35:39 smithi039 ceph-mon[31211]: rocksdb: Options.stats_dump_period_sec: 600 2024-09-21T03:35:39.815 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:35:39 smithi039 ceph-mon[31211]: rocksdb: Options.stats_persist_period_sec: 600 2024-09-21T03:35:39.815 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:35:39 smithi039 ceph-mon[31211]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-09-21T03:35:39.815 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:35:39 smithi039 ceph-mon[31211]: rocksdb: Options.max_open_files: -1 2024-09-21T03:35:39.815 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:35:39 smithi039 ceph-mon[31211]: rocksdb: Options.bytes_per_sync: 0 2024-09-21T03:35:39.816 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:35:39 smithi039 ceph-mon[31211]: rocksdb: Options.wal_bytes_per_sync: 0 2024-09-21T03:35:39.816 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:35:39 smithi039 ceph-mon[31211]: rocksdb: Options.strict_bytes_per_sync: 0 2024-09-21T03:35:39.816 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:35:39 smithi039 ceph-mon[31211]: rocksdb: Options.compaction_readahead_size: 0 2024-09-21T03:35:39.816 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:35:39 smithi039 ceph-mon[31211]: rocksdb: Options.max_background_flushes: -1 2024-09-21T03:35:39.816 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:35:39 smithi039 ceph-mon[31211]: rocksdb: Compression algorithms supported: 2024-09-21T03:35:39.816 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:35:39 smithi039 ceph-mon[31211]: rocksdb: kZSTD supported: 0 2024-09-21T03:35:39.816 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:35:39 smithi039 ceph-mon[31211]: rocksdb: kXpressCompression supported: 0 2024-09-21T03:35:39.816 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:35:39 smithi039 ceph-mon[31211]: rocksdb: kBZip2Compression supported: 0 2024-09-21T03:35:39.816 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:35:39 smithi039 ceph-mon[31211]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-09-21T03:35:39.816 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:35:39 smithi039 ceph-mon[31211]: rocksdb: kLZ4Compression supported: 1 2024-09-21T03:35:39.817 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:35:39 smithi039 ceph-mon[31211]: rocksdb: kZlibCompression supported: 1 2024-09-21T03:35:39.817 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:35:39 smithi039 ceph-mon[31211]: rocksdb: kLZ4HCCompression supported: 1 2024-09-21T03:35:39.817 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:35:39 smithi039 ceph-mon[31211]: rocksdb: kSnappyCompression supported: 1 2024-09-21T03:35:39.817 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:35:39 smithi039 ceph-mon[31211]: rocksdb: Fast CRC32 supported: Supported on x86 2024-09-21T03:35:39.817 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:35:39 smithi039 ceph-mon[31211]: rocksdb: DMutex implementation: pthread_mutex_t 2024-09-21T03:35:39.817 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:35:39 smithi039 ceph-mon[31211]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 2024-09-21T03:35:39.817 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:35:39 smithi039 ceph-mon[31211]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-09-21T03:35:39.818 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:35:39 smithi039 ceph-mon[31211]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-09-21T03:35:39.818 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:35:39 smithi039 ceph-mon[31211]: rocksdb: Options.merge_operator: 2024-09-21T03:35:39.818 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:35:39 smithi039 ceph-mon[31211]: rocksdb: Options.compaction_filter: None 2024-09-21T03:35:39.818 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:35:39 smithi039 ceph-mon[31211]: rocksdb: Options.compaction_filter_factory: None 2024-09-21T03:35:39.818 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:35:39 smithi039 ceph-mon[31211]: rocksdb: Options.sst_partitioner_factory: None 2024-09-21T03:35:39.818 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:35:39 smithi039 ceph-mon[31211]: rocksdb: Options.memtable_factory: SkipListFactory 2024-09-21T03:35:39.818 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:35:39 smithi039 ceph-mon[31211]: rocksdb: Options.table_factory: BlockBasedTable 2024-09-21T03:35:39.818 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:35:39 smithi039 ceph-mon[31211]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x5650816e4fa0) 2024-09-21T03:35:39.818 INFO:journalctl@ceph.mon.a.smithi039.stdout: cache_index_and_filter_blocks: 1 2024-09-21T03:35:39.818 INFO:journalctl@ceph.mon.a.smithi039.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-09-21T03:35:39.818 INFO:journalctl@ceph.mon.a.smithi039.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-09-21T03:35:39.818 INFO:journalctl@ceph.mon.a.smithi039.stdout: pin_top_level_index_and_filter: 1 2024-09-21T03:35:39.818 INFO:journalctl@ceph.mon.a.smithi039.stdout: index_type: 0 2024-09-21T03:35:39.818 INFO:journalctl@ceph.mon.a.smithi039.stdout: data_block_index_type: 0 2024-09-21T03:35:39.819 INFO:journalctl@ceph.mon.a.smithi039.stdout: index_shortening: 1 2024-09-21T03:35:39.819 INFO:journalctl@ceph.mon.a.smithi039.stdout: data_block_hash_table_util_ratio: 0.750000 2024-09-21T03:35:39.819 INFO:journalctl@ceph.mon.a.smithi039.stdout: checksum: 4 2024-09-21T03:35:39.819 INFO:journalctl@ceph.mon.a.smithi039.stdout: no_block_cache: 0 2024-09-21T03:35:39.819 INFO:journalctl@ceph.mon.a.smithi039.stdout: block_cache: 0x56507d6a9610 2024-09-21T03:35:39.819 INFO:journalctl@ceph.mon.a.smithi039.stdout: block_cache_name: BinnedLRUCache 2024-09-21T03:35:39.819 INFO:journalctl@ceph.mon.a.smithi039.stdout: block_cache_options: 2024-09-21T03:35:39.819 INFO:journalctl@ceph.mon.a.smithi039.stdout: capacity : 536870912 2024-09-21T03:35:39.819 INFO:journalctl@ceph.mon.a.smithi039.stdout: num_shard_bits : 4 2024-09-21T03:35:39.819 INFO:journalctl@ceph.mon.a.smithi039.stdout: strict_capacity_limit : 0 2024-09-21T03:35:39.819 INFO:journalctl@ceph.mon.a.smithi039.stdout: high_pri_pool_ratio: 0.000 2024-09-21T03:35:39.819 INFO:journalctl@ceph.mon.a.smithi039.stdout: block_cache_compressed: (nil) 2024-09-21T03:35:39.819 INFO:journalctl@ceph.mon.a.smithi039.stdout: persistent_cache: (nil) 2024-09-21T03:35:39.819 INFO:journalctl@ceph.mon.a.smithi039.stdout: block_size: 4096 2024-09-21T03:35:39.819 INFO:journalctl@ceph.mon.a.smithi039.stdout: block_size_deviation: 10 2024-09-21T03:35:39.820 INFO:journalctl@ceph.mon.a.smithi039.stdout: block_restart_interval: 16 2024-09-21T03:35:39.820 INFO:journalctl@ceph.mon.a.smithi039.stdout: index_block_restart_interval: 1 2024-09-21T03:35:39.820 INFO:journalctl@ceph.mon.a.smithi039.stdout: metadata_block_size: 4096 2024-09-21T03:35:39.820 INFO:journalctl@ceph.mon.a.smithi039.stdout: partition_filters: 0 2024-09-21T03:35:39.820 INFO:journalctl@ceph.mon.a.smithi039.stdout: use_delta_encoding: 1 2024-09-21T03:35:39.820 INFO:journalctl@ceph.mon.a.smithi039.stdout: filter_policy: bloomfilter 2024-09-21T03:35:39.820 INFO:journalctl@ceph.mon.a.smithi039.stdout: whole_key_filtering: 1 2024-09-21T03:35:39.820 INFO:journalctl@ceph.mon.a.smithi039.stdout: verify_compression: 0 2024-09-21T03:35:39.820 INFO:journalctl@ceph.mon.a.smithi039.stdout: read_amp_bytes_per_bit: 0 2024-09-21T03:35:39.820 INFO:journalctl@ceph.mon.a.smithi039.stdout: format_version: 5 2024-09-21T03:35:39.820 INFO:journalctl@ceph.mon.a.smithi039.stdout: enable_index_compression: 1 2024-09-21T03:35:39.820 INFO:journalctl@ceph.mon.a.smithi039.stdout: block_align: 0 2024-09-21T03:35:39.820 INFO:journalctl@ceph.mon.a.smithi039.stdout: max_auto_readahead_size: 262144 2024-09-21T03:35:39.820 INFO:journalctl@ceph.mon.a.smithi039.stdout: prepopulate_block_cache: 0 2024-09-21T03:35:39.821 INFO:journalctl@ceph.mon.a.smithi039.stdout: initial_auto_readahead_size: 8192 2024-09-21T03:35:39.821 INFO:journalctl@ceph.mon.a.smithi039.stdout: num_file_reads_for_auto_readahead: 2 2024-09-21T03:35:39.821 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:35:39 smithi039 ceph-mon[31211]: rocksdb: Options.write_buffer_size: 33554432 2024-09-21T03:35:39.821 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:35:39 smithi039 ceph-mon[31211]: rocksdb: Options.max_write_buffer_number: 2 2024-09-21T03:35:39.821 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:35:39 smithi039 ceph-mon[31211]: rocksdb: Options.compression: NoCompression 2024-09-21T03:35:39.821 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:35:39 smithi039 ceph-mon[31211]: rocksdb: Options.bottommost_compression: Disabled 2024-09-21T03:35:39.821 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:35:39 smithi039 ceph-mon[31211]: rocksdb: Options.prefix_extractor: nullptr 2024-09-21T03:35:39.821 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:35:39 smithi039 ceph-mon[31211]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-09-21T03:35:39.821 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:35:39 smithi039 ceph-mon[31211]: rocksdb: Options.num_levels: 7 2024-09-21T03:35:39.821 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:35:39 smithi039 ceph-mon[31211]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-09-21T03:35:39.821 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:35:39 smithi039 ceph-mon[31211]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-09-21T03:35:39.821 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:35:39 smithi039 ceph-mon[31211]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-09-21T03:35:39.821 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:35:39 smithi039 ceph-mon[31211]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-09-21T03:35:39.821 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:35:39 smithi039 ceph-mon[31211]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-09-21T03:35:39.821 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:35:39 smithi039 ceph-mon[31211]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-09-21T03:35:39.822 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:35:39 smithi039 ceph-mon[31211]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-09-21T03:35:39.822 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:35:39 smithi039 ceph-mon[31211]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-09-21T03:35:39.822 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:35:39 smithi039 ceph-mon[31211]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-09-21T03:35:39.822 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:35:39 smithi039 ceph-mon[31211]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-09-21T03:35:39.822 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:35:39 smithi039 ceph-mon[31211]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-09-21T03:35:39.822 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:35:39 smithi039 ceph-mon[31211]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-09-21T03:35:39.822 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:35:39 smithi039 ceph-mon[31211]: rocksdb: Options.compression_opts.window_bits: -14 2024-09-21T03:35:39.822 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:35:39 smithi039 ceph-mon[31211]: rocksdb: Options.compression_opts.level: 32767 2024-09-21T03:35:39.822 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:35:39 smithi039 ceph-mon[31211]: rocksdb: Options.compression_opts.strategy: 0 2024-09-21T03:35:39.822 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:35:39 smithi039 ceph-mon[31211]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-09-21T03:35:39.822 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:35:39 smithi039 ceph-mon[31211]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-09-21T03:35:39.822 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:35:39 smithi039 ceph-mon[31211]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-09-21T03:35:39.822 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:35:39 smithi039 ceph-mon[31211]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-09-21T03:35:39.822 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:35:39 smithi039 ceph-mon[31211]: rocksdb: Options.compression_opts.enabled: false 2024-09-21T03:35:39.823 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:35:39 smithi039 ceph-mon[31211]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-09-21T03:35:39.823 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:35:39 smithi039 ceph-mon[31211]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-09-21T03:35:39.823 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:35:39 smithi039 ceph-mon[31211]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-09-21T03:35:39.823 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:35:39 smithi039 ceph-mon[31211]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-09-21T03:35:39.823 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:35:39 smithi039 ceph-mon[31211]: rocksdb: Options.target_file_size_base: 67108864 2024-09-21T03:35:39.823 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:35:39 smithi039 ceph-mon[31211]: rocksdb: Options.target_file_size_multiplier: 1 2024-09-21T03:35:39.823 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:35:39 smithi039 ceph-mon[31211]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-09-21T03:35:39.823 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:35:39 smithi039 ceph-mon[31211]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-09-21T03:35:39.823 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:35:39 smithi039 ceph-mon[31211]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-09-21T03:35:39.823 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:35:39 smithi039 ceph-mon[31211]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-09-21T03:35:39.823 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:35:39 smithi039 ceph-mon[31211]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-09-21T03:35:39.823 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:35:39 smithi039 ceph-mon[31211]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-09-21T03:35:39.823 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:35:39 smithi039 ceph-mon[31211]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-09-21T03:35:39.823 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:35:39 smithi039 ceph-mon[31211]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-09-21T03:35:39.823 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:35:39 smithi039 ceph-mon[31211]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-09-21T03:35:39.824 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:35:39 smithi039 ceph-mon[31211]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-09-21T03:35:39.824 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:35:39 smithi039 ceph-mon[31211]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-09-21T03:35:39.824 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:35:39 smithi039 ceph-mon[31211]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-09-21T03:35:39.824 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:35:39 smithi039 ceph-mon[31211]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-09-21T03:35:39.824 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:35:39 smithi039 ceph-mon[31211]: rocksdb: Options.arena_block_size: 1048576 2024-09-21T03:35:39.824 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:35:39 smithi039 ceph-mon[31211]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-09-21T03:35:39.824 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:35:39 smithi039 ceph-mon[31211]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-09-21T03:35:39.824 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:35:39 smithi039 ceph-mon[31211]: rocksdb: Options.disable_auto_compactions: 0 2024-09-21T03:35:39.824 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:35:39 smithi039 ceph-mon[31211]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-09-21T03:35:39.824 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:35:39 smithi039 ceph-mon[31211]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-09-21T03:35:39.824 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:35:39 smithi039 ceph-mon[31211]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-09-21T03:35:39.824 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:35:39 smithi039 ceph-mon[31211]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-09-21T03:35:39.824 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:35:39 smithi039 ceph-mon[31211]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-09-21T03:35:39.824 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:35:39 smithi039 ceph-mon[31211]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-09-21T03:35:39.824 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:35:39 smithi039 ceph-mon[31211]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-09-21T03:35:39.825 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:35:39 smithi039 ceph-mon[31211]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-09-21T03:35:39.825 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:35:39 smithi039 ceph-mon[31211]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-09-21T03:35:39.825 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:35:39 smithi039 ceph-mon[31211]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-09-21T03:35:39.825 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:35:39 smithi039 ceph-mon[31211]: rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2024-09-21T03:35:39.825 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:35:39 smithi039 ceph-mon[31211]: rocksdb: Options.inplace_update_support: 0 2024-09-21T03:35:39.825 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:35:39 smithi039 ceph-mon[31211]: rocksdb: Options.inplace_update_num_locks: 10000 2024-09-21T03:35:39.825 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:35:39 smithi039 ceph-mon[31211]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-09-21T03:35:39.825 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:35:39 smithi039 ceph-mon[31211]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-09-21T03:35:39.825 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:35:39 smithi039 ceph-mon[31211]: rocksdb: Options.memtable_huge_page_size: 0 2024-09-21T03:35:39.825 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:35:39 smithi039 ceph-mon[31211]: rocksdb: Options.bloom_locality: 0 2024-09-21T03:35:39.825 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:35:39 smithi039 ceph-mon[31211]: rocksdb: Options.max_successive_merges: 0 2024-09-21T03:35:39.825 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:35:39 smithi039 ceph-mon[31211]: rocksdb: Options.optimize_filters_for_hits: 0 2024-09-21T03:35:39.825 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:35:39 smithi039 ceph-mon[31211]: rocksdb: Options.paranoid_file_checks: 0 2024-09-21T03:35:39.825 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:35:39 smithi039 ceph-mon[31211]: rocksdb: Options.force_consistency_checks: 1 2024-09-21T03:35:39.826 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:35:39 smithi039 ceph-mon[31211]: rocksdb: Options.report_bg_io_stats: 0 2024-09-21T03:35:39.826 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:35:39 smithi039 ceph-mon[31211]: rocksdb: Options.ttl: 2592000 2024-09-21T03:35:39.826 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:35:39 smithi039 ceph-mon[31211]: rocksdb: Options.periodic_compaction_seconds: 0 2024-09-21T03:35:39.826 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:35:39 smithi039 ceph-mon[31211]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-09-21T03:35:39.826 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:35:39 smithi039 ceph-mon[31211]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-09-21T03:35:39.826 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:35:39 smithi039 ceph-mon[31211]: rocksdb: Options.enable_blob_files: false 2024-09-21T03:35:39.826 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:35:39 smithi039 ceph-mon[31211]: rocksdb: Options.min_blob_size: 0 2024-09-21T03:35:39.826 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:35:39 smithi039 ceph-mon[31211]: rocksdb: Options.blob_file_size: 268435456 2024-09-21T03:35:39.826 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:35:39 smithi039 ceph-mon[31211]: rocksdb: Options.blob_compression_type: NoCompression 2024-09-21T03:35:39.826 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:35:39 smithi039 ceph-mon[31211]: rocksdb: Options.enable_blob_garbage_collection: false 2024-09-21T03:35:39.826 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:35:39 smithi039 ceph-mon[31211]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-09-21T03:35:39.826 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:35:39 smithi039 ceph-mon[31211]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-09-21T03:35:39.826 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:35:39 smithi039 ceph-mon[31211]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-09-21T03:35:39.827 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:35:39 smithi039 ceph-mon[31211]: rocksdb: Options.blob_file_starting_level: 0 2024-09-21T03:35:39.827 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:35:39 smithi039 ceph-mon[31211]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-09-21T03:35:39.827 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:35:39 smithi039 ceph-mon[31211]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 succeeded,manifest_file_number is 10, next_file_number is 12, last_sequence is 5, log_number is 5,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 5 2024-09-21T03:35:39.827 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:35:39 smithi039 ceph-mon[31211]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 5 2024-09-21T03:35:39.827 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:35:39 smithi039 ceph-mon[31211]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: b86f6228-8055-43a6-b82e-ea82d9cfe183 2024-09-21T03:35:39.827 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:35:39 smithi039 ceph-mon[31211]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726889739558214, "job": 1, "event": "recovery_started", "wal_files": [9]} 2024-09-21T03:35:39.827 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:35:39 smithi039 ceph-mon[31211]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #9 mode 2 2024-09-21T03:35:39.827 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:35:39 smithi039 ceph-mon[31211]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726889739559467, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 13, "file_size": 91873, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 8, "largest_seqno": 287, "table_properties": {"data_size": 89920, "index_size": 273, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 645, "raw_key_size": 11841, "raw_average_key_size": 48, "raw_value_size": 83226, "raw_average_value_size": 338, "num_data_blocks": 12, "num_entries": 246, "num_filter_entries": 246, "num_deletions": 3, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[CompactOnDeletionCollector]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1726889739, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "b86f6228-8055-43a6-b82e-ea82d9cfe183", "db_session_id": "OZGA2F5ORY09AFE06L52", "orig_file_number": 13, "seqno_to_time_mapping": "N/A"}} 2024-09-21T03:35:39.827 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:35:39 smithi039 ceph-mon[31211]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726889739559583, "job": 1, "event": "recovery_finished"} 2024-09-21T03:35:39.827 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:35:39 smithi039 ceph-mon[31211]: rocksdb: [db/version_set.cc:5047] Creating manifest 15 2024-09-21T03:35:39.827 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:35:39 smithi039 ceph-mon[31211]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-a/store.db/000009.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-09-21T03:35:39.827 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:35:39 smithi039 ceph-mon[31211]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x565081708a80 2024-09-21T03:35:39.827 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:35:39 smithi039 ceph-mon[31211]: rocksdb: DB pointer 0x5650824ae000 2024-09-21T03:35:39.828 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:35:39 smithi039 ceph-mon[31211]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-09-21T03:35:39.828 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:35:39 smithi039 ceph-mon[31211]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-09-21T03:35:39.828 INFO:journalctl@ceph.mon.a.smithi039.stdout: ** DB Stats ** 2024-09-21T03:35:39.828 INFO:journalctl@ceph.mon.a.smithi039.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-21T03:35:39.828 INFO:journalctl@ceph.mon.a.smithi039.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-09-21T03:35:39.828 INFO:journalctl@ceph.mon.a.smithi039.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-09-21T03:35:39.828 INFO:journalctl@ceph.mon.a.smithi039.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-09-21T03:35:39.828 INFO:journalctl@ceph.mon.a.smithi039.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-09-21T03:35:39.828 INFO:journalctl@ceph.mon.a.smithi039.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-09-21T03:35:39.828 INFO:journalctl@ceph.mon.a.smithi039.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-09-21T03:35:39.828 INFO:journalctl@ceph.mon.a.smithi039.stdout: 2024-09-21T03:35:39.828 INFO:journalctl@ceph.mon.a.smithi039.stdout: ** Compaction Stats [default] ** 2024-09-21T03:35:39.828 INFO:journalctl@ceph.mon.a.smithi039.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-09-21T03:35:39.829 INFO:journalctl@ceph.mon.a.smithi039.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-09-21T03:35:39.829 INFO:journalctl@ceph.mon.a.smithi039.stdout: L0 2/0 91.58 KB 0.5 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 112.6 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-09-21T03:35:39.829 INFO:journalctl@ceph.mon.a.smithi039.stdout: Sum 2/0 91.58 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 112.6 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-09-21T03:35:39.829 INFO:journalctl@ceph.mon.a.smithi039.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 112.6 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-09-21T03:35:39.829 INFO:journalctl@ceph.mon.a.smithi039.stdout: 2024-09-21T03:35:39.829 INFO:journalctl@ceph.mon.a.smithi039.stdout: ** Compaction Stats [default] ** 2024-09-21T03:35:39.829 INFO:journalctl@ceph.mon.a.smithi039.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-09-21T03:35:39.829 INFO:journalctl@ceph.mon.a.smithi039.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-09-21T03:35:39.829 INFO:journalctl@ceph.mon.a.smithi039.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 112.6 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-09-21T03:35:39.829 INFO:journalctl@ceph.mon.a.smithi039.stdout: 2024-09-21T03:35:39.829 INFO:journalctl@ceph.mon.a.smithi039.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-09-21T03:35:39.829 INFO:journalctl@ceph.mon.a.smithi039.stdout: 2024-09-21T03:35:39.829 INFO:journalctl@ceph.mon.a.smithi039.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-21T03:35:39.829 INFO:journalctl@ceph.mon.a.smithi039.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-09-21T03:35:39.830 INFO:journalctl@ceph.mon.a.smithi039.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-09-21T03:35:39.830 INFO:journalctl@ceph.mon.a.smithi039.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-09-21T03:35:39.830 INFO:journalctl@ceph.mon.a.smithi039.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-09-21T03:35:39.830 INFO:journalctl@ceph.mon.a.smithi039.stdout: AddFile(Keys): cumulative 0, interval 0 2024-09-21T03:35:39.830 INFO:journalctl@ceph.mon.a.smithi039.stdout: Cumulative compaction: 0.00 GB write, 13.61 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-21T03:35:39.830 INFO:journalctl@ceph.mon.a.smithi039.stdout: Interval compaction: 0.00 GB write, 13.61 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-21T03:35:39.830 INFO:journalctl@ceph.mon.a.smithi039.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-09-21T03:35:39.830 INFO:journalctl@ceph.mon.a.smithi039.stdout: Block cache BinnedLRUCache@0x56507d6a9610#2 capacity: 512.00 MB usage: 1.31 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 1e-05 secs_since: 0 2024-09-21T03:35:39.830 INFO:journalctl@ceph.mon.a.smithi039.stdout: Block cache entry stats(count,size,portion): FilterBlock(2,0.84 KB,0.000160933%) IndexBlock(2,0.47 KB,8.9407e-05%) Misc(1,0.00 KB,0%) 2024-09-21T03:35:39.830 INFO:journalctl@ceph.mon.a.smithi039.stdout: 2024-09-21T03:35:39.830 INFO:journalctl@ceph.mon.a.smithi039.stdout: ** File Read Latency Histogram By Level [default] ** 2024-09-21T03:35:39.830 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:35:39 smithi039 ceph-mon[31211]: starting mon.a rank 0 at public addrs [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] at bind addrs [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] mon_data /var/lib/ceph/mon/ceph-a fsid 51116d5c-77ca-11ef-baf5-efdc52797490 2024-09-21T03:35:39.830 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:35:39 smithi039 ceph-mon[31211]: mon.a@-1(???) e1 preinit fsid 51116d5c-77ca-11ef-baf5-efdc52797490 2024-09-21T03:35:39.831 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:35:39 smithi039 ceph-mon[31211]: mon.a@-1(???).mds e1 new map 2024-09-21T03:35:39.831 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:35:39 smithi039 ceph-mon[31211]: mon.a@-1(???).mds e1 print_map 2024-09-21T03:35:39.831 INFO:journalctl@ceph.mon.a.smithi039.stdout: e1 2024-09-21T03:35:39.831 INFO:journalctl@ceph.mon.a.smithi039.stdout: btime 2024-09-21T03:35:33:631109+0000 2024-09-21T03:35:39.831 INFO:journalctl@ceph.mon.a.smithi039.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-09-21T03:35:39.831 INFO:journalctl@ceph.mon.a.smithi039.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-21T03:35:39.831 INFO:journalctl@ceph.mon.a.smithi039.stdout: legacy client fscid: -1 2024-09-21T03:35:39.831 INFO:journalctl@ceph.mon.a.smithi039.stdout: 2024-09-21T03:35:39.831 INFO:journalctl@ceph.mon.a.smithi039.stdout: No filesystems configured 2024-09-21T03:35:39.831 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:35:39 smithi039 ceph-mon[31211]: mon.a@-1(???).osd e1 crush map has features 3314932999778484224, adjusting msgr requires 2024-09-21T03:35:39.831 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:35:39 smithi039 ceph-mon[31211]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-09-21T03:35:39.831 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:35:39 smithi039 ceph-mon[31211]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-09-21T03:35:39.831 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:35:39 smithi039 ceph-mon[31211]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-09-21T03:35:39.831 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:35:39 smithi039 ceph-mon[31211]: mon.a@-1(???).paxosservice(auth 1..2) refresh upgraded, format 0 -> 3 2024-09-21T03:35:39.832 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:35:39 smithi039 bash[31192]: 861e490fc665c0095d7e9950a2d6ccd4b7d704a1d4b24d58cda56cf79cda489f 2024-09-21T03:35:39.832 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:35:39 smithi039 systemd[1]: Started Ceph mon.a for 51116d5c-77ca-11ef-baf5-efdc52797490. 2024-09-21T03:35:39.832 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:35:39 smithi039 ceph-mon[31211]: mon.a is new leader, mons a in quorum (ranks 0) 2024-09-21T03:35:39.832 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:35:39 smithi039 ceph-mon[31211]: monmap epoch 1 2024-09-21T03:35:39.832 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:35:39 smithi039 ceph-mon[31211]: fsid 51116d5c-77ca-11ef-baf5-efdc52797490 2024-09-21T03:35:39.832 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:35:39 smithi039 ceph-mon[31211]: last_changed 2024-09-21T03:35:21.020316+0000 2024-09-21T03:35:39.832 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:35:39 smithi039 ceph-mon[31211]: created 2024-09-21T03:35:21.020316+0000 2024-09-21T03:35:39.832 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:35:39 smithi039 ceph-mon[31211]: min_mon_release 19 (squid) 2024-09-21T03:35:39.832 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:35:39 smithi039 ceph-mon[31211]: election_strategy: 1 2024-09-21T03:35:39.832 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:35:39 smithi039 ceph-mon[31211]: 0: [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] mon.a 2024-09-21T03:35:39.832 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:35:39 smithi039 ceph-mon[31211]: fsmap 2024-09-21T03:35:39.832 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:35:39 smithi039 ceph-mon[31211]: osdmap e1: 0 total, 0 up, 0 in 2024-09-21T03:35:39.832 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:35:39 smithi039 ceph-mon[31211]: mgrmap e1: no daemons active 2024-09-21T03:35:40.663 INFO:teuthology.orchestra.run.smithi039.stdout:Wrote config to /etc/ceph/ceph.conf 2024-09-21T03:35:40.684 INFO:teuthology.orchestra.run.smithi039.stdout:Wrote keyring to /etc/ceph/ceph.client.admin.keyring 2024-09-21T03:35:40.684 INFO:teuthology.orchestra.run.smithi039.stdout:Creating mgr... 2024-09-21T03:35:40.684 INFO:teuthology.orchestra.run.smithi039.stdout:Verifying port 0.0.0.0:9283 ... 2024-09-21T03:35:40.685 INFO:teuthology.orchestra.run.smithi039.stdout:Verifying port 0.0.0.0:8765 ... 2024-09-21T03:35:40.951 INFO:teuthology.orchestra.run.smithi039.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-51116d5c-77ca-11ef-baf5-efdc52797490@mgr.x 2024-09-21T03:35:40.951 INFO:teuthology.orchestra.run.smithi039.stdout:systemctl: stderr Failed to reset failed state of unit ceph-51116d5c-77ca-11ef-baf5-efdc52797490@mgr.x.service: Unit ceph-51116d5c-77ca-11ef-baf5-efdc52797490@mgr.x.service not loaded. 2024-09-21T03:35:41.104 INFO:teuthology.orchestra.run.smithi039.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-51116d5c-77ca-11ef-baf5-efdc52797490.target.wants/ceph-51116d5c-77ca-11ef-baf5-efdc52797490@mgr.x.service → /etc/systemd/system/ceph-51116d5c-77ca-11ef-baf5-efdc52797490@.service. 2024-09-21T03:35:41.333 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:35:41 smithi039 ceph-mon[31211]: from='client.? 172.21.15.39:0/1308034817' entity='client.admin' 2024-09-21T03:35:41.606 INFO:journalctl@ceph.mgr.x.smithi039.stdout:Sep 21 03:35:41 smithi039 podman[31423]: 2024-09-21 03:35:41.549381228 +0000 UTC m=+0.319972213 container start 622e116e4f4c4ff4ba97137a440df23968df53c17062d2444f01c9d50b66a68a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410, name=ceph-51116d5c-77ca-11ef-baf5-efdc52797490-mgr-x, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, ceph=True, maintainer=Guillaume Abrioux , GIT_CLEAN=True, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-21T03:35:41.607 INFO:journalctl@ceph.mgr.x.smithi039.stdout:Sep 21 03:35:41 smithi039 bash[31423]: 622e116e4f4c4ff4ba97137a440df23968df53c17062d2444f01c9d50b66a68a 2024-09-21T03:35:41.621 INFO:teuthology.orchestra.run.smithi039.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-09-21T03:35:41.621 INFO:teuthology.orchestra.run.smithi039.stdout:systemctl: stdout disabled 2024-09-21T03:35:41.630 INFO:teuthology.orchestra.run.smithi039.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-09-21T03:35:41.630 INFO:teuthology.orchestra.run.smithi039.stdout:systemctl: stdout inactive 2024-09-21T03:35:41.630 INFO:teuthology.orchestra.run.smithi039.stdout:firewalld.service is not enabled 2024-09-21T03:35:41.630 INFO:teuthology.orchestra.run.smithi039.stdout:Not possible to enable service . firewalld.service is not available 2024-09-21T03:35:41.642 INFO:teuthology.orchestra.run.smithi039.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-09-21T03:35:41.642 INFO:teuthology.orchestra.run.smithi039.stdout:systemctl: stdout disabled 2024-09-21T03:35:41.651 INFO:teuthology.orchestra.run.smithi039.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-09-21T03:35:41.651 INFO:teuthology.orchestra.run.smithi039.stdout:systemctl: stdout inactive 2024-09-21T03:35:41.651 INFO:teuthology.orchestra.run.smithi039.stdout:firewalld.service is not enabled 2024-09-21T03:35:41.651 INFO:teuthology.orchestra.run.smithi039.stdout:Not possible to open ports <[9283, 8765]>. firewalld.service is not available 2024-09-21T03:35:41.651 INFO:teuthology.orchestra.run.smithi039.stdout:Waiting for mgr to start... 2024-09-21T03:35:41.651 INFO:teuthology.orchestra.run.smithi039.stdout:Waiting for mgr... 2024-09-21T03:35:41.875 INFO:journalctl@ceph.mgr.x.smithi039.stdout:Sep 21 03:35:41 smithi039 systemd[1]: Started Ceph mgr.x for 51116d5c-77ca-11ef-baf5-efdc52797490. 2024-09-21T03:35:42.583 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:35:42 smithi039 ceph-mon[31211]: from='client.? 172.21.15.39:0/3414778131' entity='client.admin' cmd={"prefix": "status", "format": "json-pretty"} : dispatch 2024-09-21T03:35:42.737 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout 2024-09-21T03:35:42.737 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout { 2024-09-21T03:35:42.738 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "fsid": "51116d5c-77ca-11ef-baf5-efdc52797490", 2024-09-21T03:35:42.738 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "health": { 2024-09-21T03:35:42.738 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-09-21T03:35:42.738 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-09-21T03:35:42.738 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-09-21T03:35:42.738 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout }, 2024-09-21T03:35:42.738 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-09-21T03:35:42.738 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-09-21T03:35:42.738 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout 0 2024-09-21T03:35:42.738 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout ], 2024-09-21T03:35:42.738 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-09-21T03:35:42.739 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "a" 2024-09-21T03:35:42.739 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout ], 2024-09-21T03:35:42.739 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "quorum_age": 2, 2024-09-21T03:35:42.739 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "monmap": { 2024-09-21T03:35:42.739 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-21T03:35:42.739 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-09-21T03:35:42.739 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-09-21T03:35:42.739 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout }, 2024-09-21T03:35:42.739 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-09-21T03:35:42.739 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-21T03:35:42.739 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-09-21T03:35:42.739 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-09-21T03:35:42.740 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-09-21T03:35:42.740 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-09-21T03:35:42.740 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-09-21T03:35:42.740 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-09-21T03:35:42.740 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout }, 2024-09-21T03:35:42.740 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-09-21T03:35:42.740 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-09-21T03:35:42.740 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-09-21T03:35:42.740 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-09-21T03:35:42.740 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-09-21T03:35:42.740 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-09-21T03:35:42.740 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-09-21T03:35:42.741 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-09-21T03:35:42.741 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-09-21T03:35:42.741 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout }, 2024-09-21T03:35:42.741 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-09-21T03:35:42.741 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-21T03:35:42.741 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "btime": "2024-09-21T03:35:33:631109+0000", 2024-09-21T03:35:42.741 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-09-21T03:35:42.741 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-09-21T03:35:42.741 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout }, 2024-09-21T03:35:42.741 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-09-21T03:35:42.741 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "available": false, 2024-09-21T03:35:42.741 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-09-21T03:35:42.741 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "modules": [ 2024-09-21T03:35:42.742 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "iostat", 2024-09-21T03:35:42.742 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "nfs", 2024-09-21T03:35:42.742 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "restful" 2024-09-21T03:35:42.742 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout ], 2024-09-21T03:35:42.742 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-21T03:35:42.742 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout }, 2024-09-21T03:35:42.742 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-09-21T03:35:42.742 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-21T03:35:42.742 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "modified": "2024-09-21T03:35:33.631962+0000", 2024-09-21T03:35:42.742 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-21T03:35:42.743 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout }, 2024-09-21T03:35:42.743 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-09-21T03:35:42.743 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout } 2024-09-21T03:35:42.743 INFO:teuthology.orchestra.run.smithi039.stdout:mgr not available, waiting (1/15)... 2024-09-21T03:35:45.569 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:35:45 smithi039 ceph-mon[31211]: from='client.? 172.21.15.39:0/782096386' entity='client.admin' cmd={"prefix": "status", "format": "json-pretty"} : dispatch 2024-09-21T03:35:45.936 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout 2024-09-21T03:35:45.936 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout { 2024-09-21T03:35:45.937 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "fsid": "51116d5c-77ca-11ef-baf5-efdc52797490", 2024-09-21T03:35:45.937 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "health": { 2024-09-21T03:35:45.937 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-09-21T03:35:45.937 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-09-21T03:35:45.937 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-09-21T03:35:45.937 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout }, 2024-09-21T03:35:45.937 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-09-21T03:35:45.937 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-09-21T03:35:45.937 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout 0 2024-09-21T03:35:45.937 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout ], 2024-09-21T03:35:45.937 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-09-21T03:35:45.937 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "a" 2024-09-21T03:35:45.937 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout ], 2024-09-21T03:35:45.938 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "quorum_age": 5, 2024-09-21T03:35:45.938 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "monmap": { 2024-09-21T03:35:45.938 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-21T03:35:45.938 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-09-21T03:35:45.938 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-09-21T03:35:45.938 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout }, 2024-09-21T03:35:45.938 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-09-21T03:35:45.938 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-21T03:35:45.938 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-09-21T03:35:45.938 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-09-21T03:35:45.938 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-09-21T03:35:45.938 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-09-21T03:35:45.938 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-09-21T03:35:45.938 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-09-21T03:35:45.939 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout }, 2024-09-21T03:35:45.939 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-09-21T03:35:45.939 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-09-21T03:35:45.939 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-09-21T03:35:45.939 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-09-21T03:35:45.939 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-09-21T03:35:45.939 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-09-21T03:35:45.939 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-09-21T03:35:45.939 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-09-21T03:35:45.939 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-09-21T03:35:45.939 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout }, 2024-09-21T03:35:45.939 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-09-21T03:35:45.939 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-21T03:35:45.939 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "btime": "2024-09-21T03:35:33:631109+0000", 2024-09-21T03:35:45.940 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-09-21T03:35:45.940 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-09-21T03:35:45.940 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout }, 2024-09-21T03:35:45.940 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-09-21T03:35:45.940 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "available": false, 2024-09-21T03:35:45.940 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-09-21T03:35:45.940 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "modules": [ 2024-09-21T03:35:45.940 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "iostat", 2024-09-21T03:35:45.940 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "nfs", 2024-09-21T03:35:45.940 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "restful" 2024-09-21T03:35:45.940 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout ], 2024-09-21T03:35:45.940 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-21T03:35:45.940 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout }, 2024-09-21T03:35:45.940 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-09-21T03:35:45.940 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-21T03:35:45.941 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "modified": "2024-09-21T03:35:33.631962+0000", 2024-09-21T03:35:45.941 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-21T03:35:45.941 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout }, 2024-09-21T03:35:45.941 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-09-21T03:35:45.941 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout } 2024-09-21T03:35:45.941 INFO:teuthology.orchestra.run.smithi039.stdout:mgr not available, waiting (2/15)... 2024-09-21T03:35:47.333 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:35:46 smithi039 ceph-mon[31211]: Activating manager daemon x 2024-09-21T03:35:47.334 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:35:46 smithi039 ceph-mon[31211]: mgrmap e2: x(active, starting, since 0.00357634s) 2024-09-21T03:35:47.334 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:35:46 smithi039 ceph-mon[31211]: from='mgr.14100 172.21.15.39:0/2587722394' entity='mgr.x' cmd={"prefix": "mds metadata"} : dispatch 2024-09-21T03:35:47.334 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:35:46 smithi039 ceph-mon[31211]: from='mgr.14100 172.21.15.39:0/2587722394' entity='mgr.x' cmd={"prefix": "osd metadata"} : dispatch 2024-09-21T03:35:47.334 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:35:46 smithi039 ceph-mon[31211]: from='mgr.14100 172.21.15.39:0/2587722394' entity='mgr.x' cmd={"prefix": "mon metadata"} : dispatch 2024-09-21T03:35:47.334 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:35:46 smithi039 ceph-mon[31211]: from='mgr.14100 172.21.15.39:0/2587722394' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-21T03:35:47.334 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:35:46 smithi039 ceph-mon[31211]: from='mgr.14100 172.21.15.39:0/2587722394' entity='mgr.x' cmd={"prefix": "mgr metadata", "who": "x", "id": "x"} : dispatch 2024-09-21T03:35:47.334 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:35:46 smithi039 ceph-mon[31211]: Manager daemon x is now available 2024-09-21T03:35:47.334 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:35:46 smithi039 ceph-mon[31211]: from='mgr.14100 172.21.15.39:0/2587722394' entity='mgr.x' cmd={"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"} : dispatch 2024-09-21T03:35:47.334 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:35:46 smithi039 ceph-mon[31211]: from='mgr.14100 172.21.15.39:0/2587722394' entity='mgr.x' cmd={"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"} : dispatch 2024-09-21T03:35:47.334 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:35:46 smithi039 ceph-mon[31211]: from='mgr.14100 172.21.15.39:0/2587722394' entity='mgr.x' 2024-09-21T03:35:47.334 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:35:46 smithi039 ceph-mon[31211]: from='mgr.14100 172.21.15.39:0/2587722394' entity='mgr.x' 2024-09-21T03:35:47.335 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:35:46 smithi039 ceph-mon[31211]: from='mgr.14100 172.21.15.39:0/2587722394' entity='mgr.x' 2024-09-21T03:35:48.934 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:35:48 smithi039 ceph-mon[31211]: mgrmap e3: x(active, since 1.00857s) 2024-09-21T03:35:48.934 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:35:48 smithi039 ceph-mon[31211]: from='client.? 172.21.15.39:0/2390920551' entity='client.admin' cmd={"prefix": "status", "format": "json-pretty"} : dispatch 2024-09-21T03:35:49.247 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout 2024-09-21T03:35:49.248 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout { 2024-09-21T03:35:49.248 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "fsid": "51116d5c-77ca-11ef-baf5-efdc52797490", 2024-09-21T03:35:49.248 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "health": { 2024-09-21T03:35:49.248 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-09-21T03:35:49.248 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-09-21T03:35:49.248 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-09-21T03:35:49.248 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout }, 2024-09-21T03:35:49.248 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-09-21T03:35:49.248 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-09-21T03:35:49.248 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout 0 2024-09-21T03:35:49.248 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout ], 2024-09-21T03:35:49.248 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-09-21T03:35:49.248 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "a" 2024-09-21T03:35:49.249 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout ], 2024-09-21T03:35:49.249 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "quorum_age": 9, 2024-09-21T03:35:49.249 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "monmap": { 2024-09-21T03:35:49.249 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-21T03:35:49.249 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-09-21T03:35:49.249 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-09-21T03:35:49.249 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout }, 2024-09-21T03:35:49.249 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-09-21T03:35:49.249 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-21T03:35:49.249 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-09-21T03:35:49.249 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-09-21T03:35:49.249 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-09-21T03:35:49.252 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-09-21T03:35:49.253 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-09-21T03:35:49.253 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-09-21T03:35:49.253 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout }, 2024-09-21T03:35:49.253 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-09-21T03:35:49.253 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-09-21T03:35:49.253 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-09-21T03:35:49.253 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-09-21T03:35:49.253 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-09-21T03:35:49.253 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-09-21T03:35:49.253 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-09-21T03:35:49.253 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-09-21T03:35:49.253 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-09-21T03:35:49.253 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout }, 2024-09-21T03:35:49.254 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-09-21T03:35:49.254 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-21T03:35:49.254 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "btime": "2024-09-21T03:35:33:631109+0000", 2024-09-21T03:35:49.254 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-09-21T03:35:49.254 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-09-21T03:35:49.254 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout }, 2024-09-21T03:35:49.254 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-09-21T03:35:49.254 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "available": true, 2024-09-21T03:35:49.254 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-09-21T03:35:49.254 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "modules": [ 2024-09-21T03:35:49.254 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "iostat", 2024-09-21T03:35:49.254 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "nfs", 2024-09-21T03:35:49.254 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "restful" 2024-09-21T03:35:49.254 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout ], 2024-09-21T03:35:49.255 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-21T03:35:49.255 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout }, 2024-09-21T03:35:49.255 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-09-21T03:35:49.255 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-21T03:35:49.255 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "modified": "2024-09-21T03:35:33.631962+0000", 2024-09-21T03:35:49.255 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-21T03:35:49.255 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout }, 2024-09-21T03:35:49.255 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-09-21T03:35:49.255 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout } 2024-09-21T03:35:49.255 INFO:teuthology.orchestra.run.smithi039.stdout:mgr is available 2024-09-21T03:35:49.880 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:35:49 smithi039 ceph-mon[31211]: mgrmap e4: x(active, since 2s) 2024-09-21T03:35:50.488 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout 2024-09-21T03:35:50.488 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout [global] 2024-09-21T03:35:50.489 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout fsid = 51116d5c-77ca-11ef-baf5-efdc52797490 2024-09-21T03:35:50.489 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout mon_cluster_log_file_level = debug 2024-09-21T03:35:50.489 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout mon_host = [v2:172.21.15.39:3300,v1:172.21.15.39:6789] 2024-09-21T03:35:50.489 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-09-21T03:35:50.489 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-09-21T03:35:50.489 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-09-21T03:35:50.489 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-09-21T03:35:50.489 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout 2024-09-21T03:35:50.489 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout [mgr] 2024-09-21T03:35:50.489 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-09-21T03:35:50.489 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout 2024-09-21T03:35:50.489 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout [osd] 2024-09-21T03:35:50.489 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout bdev_async_discard = True 2024-09-21T03:35:50.490 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-09-21T03:35:50.490 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-09-21T03:35:50.490 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-09-21T03:35:50.490 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-09-21T03:35:50.490 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-09-21T03:35:50.490 INFO:teuthology.orchestra.run.smithi039.stdout:Enabling cephadm module... 2024-09-21T03:35:51.333 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:35:50 smithi039 ceph-mon[31211]: from='client.? 172.21.15.39:0/3266822361' entity='client.admin' cmd={"prefix": "config assimilate-conf"} : dispatch 2024-09-21T03:35:52.149 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:35:51 smithi039 ceph-mon[31211]: from='client.? 172.21.15.39:0/2859619296' entity='client.admin' cmd={"prefix": "mgr module enable", "module": "cephadm"} : dispatch 2024-09-21T03:35:52.149 INFO:journalctl@ceph.mgr.x.smithi039.stdout:Sep 21 03:35:51 smithi039 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-mgr-x[31438]: ignoring --setuser ceph since I am not root 2024-09-21T03:35:52.149 INFO:journalctl@ceph.mgr.x.smithi039.stdout:Sep 21 03:35:51 smithi039 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-mgr-x[31438]: ignoring --setgroup ceph since I am not root 2024-09-21T03:35:53.075 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:35:52 smithi039 ceph-mon[31211]: from='client.? 172.21.15.39:0/2859619296' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "cephadm"}]': finished 2024-09-21T03:35:53.434 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:35:52 smithi039 ceph-mon[31211]: mgrmap e5: x(active, since 5s) 2024-09-21T03:35:53.692 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout { 2024-09-21T03:35:53.694 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "epoch": 5, 2024-09-21T03:35:53.694 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "available": true, 2024-09-21T03:35:53.694 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-09-21T03:35:53.694 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-09-21T03:35:53.694 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout } 2024-09-21T03:35:53.694 INFO:teuthology.orchestra.run.smithi039.stdout:Waiting for the mgr to restart... 2024-09-21T03:35:53.694 INFO:teuthology.orchestra.run.smithi039.stdout:Waiting for mgr epoch 5... 2024-09-21T03:35:54.026 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:35:53 smithi039 ceph-mon[31211]: from='client.? 172.21.15.39:0/524193500' entity='client.admin' cmd={"prefix": "mgr stat"} : dispatch 2024-09-21T03:35:57.583 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:35:57 smithi039 ceph-mon[31211]: Active manager daemon x restarted 2024-09-21T03:35:57.584 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:35:57 smithi039 ceph-mon[31211]: Activating manager daemon x 2024-09-21T03:35:57.584 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:35:57 smithi039 ceph-mon[31211]: osdmap e2: 0 total, 0 up, 0 in 2024-09-21T03:35:57.584 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:35:57 smithi039 ceph-mon[31211]: mgrmap e6: x(active, starting, since 0.00875908s) 2024-09-21T03:35:57.584 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:35:57 smithi039 ceph-mon[31211]: from='mgr.14118 172.21.15.39:0/2203062369' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-21T03:35:57.584 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:35:57 smithi039 ceph-mon[31211]: from='mgr.14118 172.21.15.39:0/2203062369' entity='mgr.x' cmd={"prefix": "mgr metadata", "who": "x", "id": "x"} : dispatch 2024-09-21T03:35:57.584 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:35:57 smithi039 ceph-mon[31211]: from='mgr.14118 172.21.15.39:0/2203062369' entity='mgr.x' cmd={"prefix": "mds metadata"} : dispatch 2024-09-21T03:35:57.584 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:35:57 smithi039 ceph-mon[31211]: from='mgr.14118 172.21.15.39:0/2203062369' entity='mgr.x' cmd={"prefix": "osd metadata"} : dispatch 2024-09-21T03:35:57.584 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:35:57 smithi039 ceph-mon[31211]: from='mgr.14118 172.21.15.39:0/2203062369' entity='mgr.x' cmd={"prefix": "mon metadata"} : dispatch 2024-09-21T03:35:57.584 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:35:57 smithi039 ceph-mon[31211]: Manager daemon x is now available 2024-09-21T03:35:58.782 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout { 2024-09-21T03:35:58.782 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 7, 2024-09-21T03:35:58.782 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "initialized": true 2024-09-21T03:35:58.783 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout } 2024-09-21T03:35:58.783 INFO:teuthology.orchestra.run.smithi039.stdout:mgr epoch 5 is available 2024-09-21T03:35:58.783 INFO:teuthology.orchestra.run.smithi039.stdout:Verifying orchestrator module is enabled... 2024-09-21T03:35:59.575 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:35:59 smithi039 ceph-mon[31211]: mgrmap e7: x(active, since 1.01521s) 2024-09-21T03:35:59.575 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:35:59 smithi039 ceph-mon[31211]: from='mgr.14118 172.21.15.39:0/2203062369' entity='mgr.x' 2024-09-21T03:35:59.575 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:35:59 smithi039 ceph-mon[31211]: from='mgr.14118 172.21.15.39:0/2203062369' entity='mgr.x' 2024-09-21T03:35:59.575 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:35:59 smithi039 ceph-mon[31211]: Found migration_current of "None". Setting to last migration. 2024-09-21T03:35:59.575 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:35:59 smithi039 ceph-mon[31211]: from='mgr.14118 172.21.15.39:0/2203062369' entity='mgr.x' 2024-09-21T03:35:59.575 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:35:59 smithi039 ceph-mon[31211]: from='mgr.14118 172.21.15.39:0/2203062369' entity='mgr.x' 2024-09-21T03:35:59.575 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:35:59 smithi039 ceph-mon[31211]: from='mgr.14118 172.21.15.39:0/2203062369' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:35:59.575 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:35:59 smithi039 ceph-mon[31211]: from='mgr.14118 172.21.15.39:0/2203062369' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:35:59.575 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:35:59 smithi039 ceph-mon[31211]: from='mgr.14118 172.21.15.39:0/2203062369' entity='mgr.x' cmd={"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"} : dispatch 2024-09-21T03:35:59.576 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:35:59 smithi039 ceph-mon[31211]: from='mgr.14118 172.21.15.39:0/2203062369' entity='mgr.x' cmd={"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"} : dispatch 2024-09-21T03:35:59.834 INFO:journalctl@ceph.mgr.x.smithi039.stdout:Sep 21 03:35:59 smithi039 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-mgr-x[31438]: 2024-09-21T03:35:59.572+0000 7fc7ce6c8640 -1 log_channel(cephadm) log [ERR] : [21/Sep/2024:03:35:59] ENGINE Error in HTTPServer.serve 2024-09-21T03:35:59.834 INFO:journalctl@ceph.mgr.x.smithi039.stdout:Sep 21 03:35:59 smithi039 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-mgr-x[31438]: Traceback (most recent call last): 2024-09-21T03:35:59.834 INFO:journalctl@ceph.mgr.x.smithi039.stdout:Sep 21 03:35:59 smithi039 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-mgr-x[31438]: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-21T03:35:59.834 INFO:journalctl@ceph.mgr.x.smithi039.stdout:Sep 21 03:35:59 smithi039 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-mgr-x[31438]: self._connections.run(self.expiration_interval) 2024-09-21T03:35:59.834 INFO:journalctl@ceph.mgr.x.smithi039.stdout:Sep 21 03:35:59 smithi039 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-mgr-x[31438]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-21T03:35:59.834 INFO:journalctl@ceph.mgr.x.smithi039.stdout:Sep 21 03:35:59 smithi039 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-mgr-x[31438]: self._run(expiration_interval) 2024-09-21T03:35:59.834 INFO:journalctl@ceph.mgr.x.smithi039.stdout:Sep 21 03:35:59 smithi039 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-mgr-x[31438]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-21T03:35:59.835 INFO:journalctl@ceph.mgr.x.smithi039.stdout:Sep 21 03:35:59 smithi039 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-mgr-x[31438]: new_conn = self._from_server_socket(self.server.socket) 2024-09-21T03:35:59.835 INFO:journalctl@ceph.mgr.x.smithi039.stdout:Sep 21 03:35:59 smithi039 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-mgr-x[31438]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-21T03:35:59.835 INFO:journalctl@ceph.mgr.x.smithi039.stdout:Sep 21 03:35:59 smithi039 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-mgr-x[31438]: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-21T03:35:59.835 INFO:journalctl@ceph.mgr.x.smithi039.stdout:Sep 21 03:35:59 smithi039 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-mgr-x[31438]: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-21T03:35:59.835 INFO:journalctl@ceph.mgr.x.smithi039.stdout:Sep 21 03:35:59 smithi039 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-mgr-x[31438]: s = self.context.wrap_socket( 2024-09-21T03:35:59.835 INFO:journalctl@ceph.mgr.x.smithi039.stdout:Sep 21 03:35:59 smithi039 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-mgr-x[31438]: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-21T03:35:59.835 INFO:journalctl@ceph.mgr.x.smithi039.stdout:Sep 21 03:35:59 smithi039 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-mgr-x[31438]: return self.sslsocket_class._create( 2024-09-21T03:35:59.835 INFO:journalctl@ceph.mgr.x.smithi039.stdout:Sep 21 03:35:59 smithi039 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-mgr-x[31438]: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-21T03:35:59.835 INFO:journalctl@ceph.mgr.x.smithi039.stdout:Sep 21 03:35:59 smithi039 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-mgr-x[31438]: self.do_handshake() 2024-09-21T03:35:59.835 INFO:journalctl@ceph.mgr.x.smithi039.stdout:Sep 21 03:35:59 smithi039 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-mgr-x[31438]: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-21T03:35:59.835 INFO:journalctl@ceph.mgr.x.smithi039.stdout:Sep 21 03:35:59 smithi039 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-mgr-x[31438]: self._sslobj.do_handshake() 2024-09-21T03:35:59.835 INFO:journalctl@ceph.mgr.x.smithi039.stdout:Sep 21 03:35:59 smithi039 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-mgr-x[31438]: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-21T03:35:59.835 INFO:journalctl@ceph.mgr.x.smithi039.stdout:Sep 21 03:35:59 smithi039 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-mgr-x[31438]: 2024-09-21T03:36:00.584 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:36:00 smithi039 ceph-mon[31211]: mgrmap e8: x(active, since 2s) 2024-09-21T03:36:00.584 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:36:00 smithi039 ceph-mon[31211]: [21/Sep/2024:03:35:59] ENGINE Bus STARTING 2024-09-21T03:36:00.584 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:36:00 smithi039 ceph-mon[31211]: from='client.? 172.21.15.39:0/1435626300' entity='client.admin' cmd={"prefix": "mgr module enable", "module": "orchestrator"} : dispatch 2024-09-21T03:36:00.584 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:36:00 smithi039 ceph-mon[31211]: [21/Sep/2024:03:35:59] ENGINE Serving on https://172.21.15.39:7150 2024-09-21T03:36:00.584 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:36:00 smithi039 ceph-mon[31211]: [21/Sep/2024:03:35:59] ENGINE Error in HTTPServer.serve 2024-09-21T03:36:00.584 INFO:journalctl@ceph.mon.a.smithi039.stdout: Traceback (most recent call last): 2024-09-21T03:36:00.584 INFO:journalctl@ceph.mon.a.smithi039.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-21T03:36:00.584 INFO:journalctl@ceph.mon.a.smithi039.stdout: self._connections.run(self.expiration_interval) 2024-09-21T03:36:00.584 INFO:journalctl@ceph.mon.a.smithi039.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-21T03:36:00.584 INFO:journalctl@ceph.mon.a.smithi039.stdout: self._run(expiration_interval) 2024-09-21T03:36:00.585 INFO:journalctl@ceph.mon.a.smithi039.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-21T03:36:00.585 INFO:journalctl@ceph.mon.a.smithi039.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-09-21T03:36:00.585 INFO:journalctl@ceph.mon.a.smithi039.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-21T03:36:00.585 INFO:journalctl@ceph.mon.a.smithi039.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-21T03:36:00.585 INFO:journalctl@ceph.mon.a.smithi039.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-21T03:36:00.585 INFO:journalctl@ceph.mon.a.smithi039.stdout: s = self.context.wrap_socket( 2024-09-21T03:36:00.585 INFO:journalctl@ceph.mon.a.smithi039.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-21T03:36:00.585 INFO:journalctl@ceph.mon.a.smithi039.stdout: return self.sslsocket_class._create( 2024-09-21T03:36:00.585 INFO:journalctl@ceph.mon.a.smithi039.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-21T03:36:00.585 INFO:journalctl@ceph.mon.a.smithi039.stdout: self.do_handshake() 2024-09-21T03:36:00.585 INFO:journalctl@ceph.mon.a.smithi039.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-21T03:36:00.586 INFO:journalctl@ceph.mon.a.smithi039.stdout: self._sslobj.do_handshake() 2024-09-21T03:36:00.586 INFO:journalctl@ceph.mon.a.smithi039.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-21T03:36:00.586 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:36:00 smithi039 ceph-mon[31211]: [21/Sep/2024:03:35:59] ENGINE Serving on http://172.21.15.39:8765 2024-09-21T03:36:00.586 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:36:00 smithi039 ceph-mon[31211]: [21/Sep/2024:03:35:59] ENGINE Bus STARTED 2024-09-21T03:36:00.586 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:36:00 smithi039 ceph-mon[31211]: from='mgr.14118 172.21.15.39:0/2203062369' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:36:00.890 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr module 'orchestrator' is already enabled (always-on) 2024-09-21T03:36:00.890 INFO:teuthology.orchestra.run.smithi039.stdout:Setting orchestrator backend to cephadm... 2024-09-21T03:36:01.539 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:36:01 smithi039 ceph-mon[31211]: from='client.? 172.21.15.39:0/1435626300' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "orchestrator"}]': finished 2024-09-21T03:36:01.540 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:36:01 smithi039 ceph-mon[31211]: mgrmap e9: x(active, since 3s) 2024-09-21T03:36:02.699 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:36:02 smithi039 ceph-mon[31211]: from='client.14132 -' entity='client.admin' cmd=[{"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:36:02.699 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:36:02 smithi039 ceph-mon[31211]: from='mgr.14118 172.21.15.39:0/2203062369' entity='mgr.x' 2024-09-21T03:36:02.699 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:36:02 smithi039 ceph-mon[31211]: from='mgr.14118 172.21.15.39:0/2203062369' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:36:03.336 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout value unchanged 2024-09-21T03:36:03.337 INFO:teuthology.orchestra.run.smithi039.stdout:Generating ssh key... 2024-09-21T03:36:03.653 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:36:03 smithi039 ceph-mon[31211]: from='client.14134 -' entity='client.admin' cmd=[{"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:36:04.572 INFO:journalctl@ceph.mgr.x.smithi039.stdout:Sep 21 03:36:04 smithi039 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-mgr-x[31438]: Generating public/private rsa key pair. 2024-09-21T03:36:04.573 INFO:journalctl@ceph.mgr.x.smithi039.stdout:Sep 21 03:36:04 smithi039 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-mgr-x[31438]: Your identification has been saved in /tmp/tmpfg98qyg7/key 2024-09-21T03:36:04.573 INFO:journalctl@ceph.mgr.x.smithi039.stdout:Sep 21 03:36:04 smithi039 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-mgr-x[31438]: Your public key has been saved in /tmp/tmpfg98qyg7/key.pub 2024-09-21T03:36:04.573 INFO:journalctl@ceph.mgr.x.smithi039.stdout:Sep 21 03:36:04 smithi039 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-mgr-x[31438]: The key fingerprint is: 2024-09-21T03:36:04.573 INFO:journalctl@ceph.mgr.x.smithi039.stdout:Sep 21 03:36:04 smithi039 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-mgr-x[31438]: SHA256:ns4h08AuITULdyNL6pyA/8yy9rNfl5nakQCYBS6k6A0 ceph-51116d5c-77ca-11ef-baf5-efdc52797490 2024-09-21T03:36:04.573 INFO:journalctl@ceph.mgr.x.smithi039.stdout:Sep 21 03:36:04 smithi039 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-mgr-x[31438]: The key's randomart image is: 2024-09-21T03:36:04.573 INFO:journalctl@ceph.mgr.x.smithi039.stdout:Sep 21 03:36:04 smithi039 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-mgr-x[31438]: +---[RSA 3072]----+ 2024-09-21T03:36:04.573 INFO:journalctl@ceph.mgr.x.smithi039.stdout:Sep 21 03:36:04 smithi039 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-mgr-x[31438]: | . ... | 2024-09-21T03:36:04.573 INFO:journalctl@ceph.mgr.x.smithi039.stdout:Sep 21 03:36:04 smithi039 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-mgr-x[31438]: |.o . + | 2024-09-21T03:36:04.573 INFO:journalctl@ceph.mgr.x.smithi039.stdout:Sep 21 03:36:04 smithi039 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-mgr-x[31438]: |oEo X + | 2024-09-21T03:36:04.573 INFO:journalctl@ceph.mgr.x.smithi039.stdout:Sep 21 03:36:04 smithi039 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-mgr-x[31438]: |o oB B o | 2024-09-21T03:36:04.573 INFO:journalctl@ceph.mgr.x.smithi039.stdout:Sep 21 03:36:04 smithi039 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-mgr-x[31438]: |o.o.+ o S | 2024-09-21T03:36:04.573 INFO:journalctl@ceph.mgr.x.smithi039.stdout:Sep 21 03:36:04 smithi039 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-mgr-x[31438]: | = o o + o = | 2024-09-21T03:36:04.573 INFO:journalctl@ceph.mgr.x.smithi039.stdout:Sep 21 03:36:04 smithi039 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-mgr-x[31438]: | = . + * B | 2024-09-21T03:36:04.573 INFO:journalctl@ceph.mgr.x.smithi039.stdout:Sep 21 03:36:04 smithi039 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-mgr-x[31438]: | o+.. * = . | 2024-09-21T03:36:04.574 INFO:journalctl@ceph.mgr.x.smithi039.stdout:Sep 21 03:36:04 smithi039 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-mgr-x[31438]: | ..+*+. + . | 2024-09-21T03:36:04.574 INFO:journalctl@ceph.mgr.x.smithi039.stdout:Sep 21 03:36:04 smithi039 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-mgr-x[31438]: +----[SHA256]-----+ 2024-09-21T03:36:05.454 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:36:05 smithi039 ceph-mon[31211]: from='client.14136 -' entity='client.admin' cmd=[{"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:36:05.454 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:36:05 smithi039 ceph-mon[31211]: Generating ssh key... 2024-09-21T03:36:05.455 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:36:05 smithi039 ceph-mon[31211]: from='mgr.14118 172.21.15.39:0/2203062369' entity='mgr.x' 2024-09-21T03:36:05.455 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:36:05 smithi039 ceph-mon[31211]: from='mgr.14118 172.21.15.39:0/2203062369' entity='mgr.x' 2024-09-21T03:36:06.015 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCdXyV6VsXd0Yn4HDH1fa4yXMDTjUd9Ae7U+4BBYwRjrbJ4WGOsw4tmCk8fIKruyIWwSwsvgh3+43Ay2Mrz0cqW6meMFojSme8pjU3Qg1E3Ugb/kk8lI2hS+Hv3sIWzaXvToAEXeliGygWkdmru1I9sd7KU3HfqdmBl+FDHC5OfOE+siobc+PsLhx0EurpOM7ChYK50KSCEN5IsQU9alrzSWnFb/6mUF4a5SRTzFNznVClrig2gK46DYoktqCrcCpByFCHBuKZULwcMVQM+qZQcTP6P9EMvOSYUvzuAYb0bRlfZaLDQ/AMO8QI11tq1ipmOPeTGMQ2aZSaG9SYs9iaHeYLUc+DCqEsUomIFAr3q4QOZ9lXTF3pjo0NesWui7QL5CMzCZR/UC6OBBEpg0moqEE69NG/BGScaA3yuh9y/fMAQKubHETK4IDnIJlVpurzagAPfOZREYSwFQM/BBnEVeb3DHrJV9/Quwn1h4s8t42K3TnohbALTj2emxy/w3+s= ceph-51116d5c-77ca-11ef-baf5-efdc52797490 2024-09-21T03:36:06.015 INFO:teuthology.orchestra.run.smithi039.stdout:Wrote public SSH key to /home/ubuntu/cephtest/ceph.pub 2024-09-21T03:36:06.015 INFO:teuthology.orchestra.run.smithi039.stdout:Adding key to root@localhost authorized_keys... 2024-09-21T03:36:06.022 INFO:teuthology.orchestra.run.smithi039.stdout:Adding host smithi039... 2024-09-21T03:36:06.244 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:36:06 smithi039 ceph-mon[31211]: from='client.14138 -' entity='client.admin' cmd=[{"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:36:07.343 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:36:07 smithi039 ceph-mon[31211]: from='client.14140 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi039", "addr": "172.21.15.39", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:36:08.480 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:36:08 smithi039 ceph-mon[31211]: Deploying cephadm binary to smithi039 2024-09-21T03:36:11.072 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout Added host 'smithi039' with addr '172.21.15.39' 2024-09-21T03:36:11.072 INFO:teuthology.orchestra.run.smithi039.stdout:Deploying unmanaged mon service... 2024-09-21T03:36:11.712 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:36:11 smithi039 ceph-mon[31211]: from='mgr.14118 172.21.15.39:0/2203062369' entity='mgr.x' 2024-09-21T03:36:11.712 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:36:11 smithi039 ceph-mon[31211]: Added host smithi039 2024-09-21T03:36:11.712 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:36:11 smithi039 ceph-mon[31211]: from='mgr.14118 172.21.15.39:0/2203062369' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:36:12.340 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout Scheduled mon update... 2024-09-21T03:36:12.340 INFO:teuthology.orchestra.run.smithi039.stdout:Deploying unmanaged mgr service... 2024-09-21T03:36:12.916 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:36:12 smithi039 ceph-mon[31211]: from='client.14142 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:36:12.916 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:36:12 smithi039 ceph-mon[31211]: Saving service mon spec with placement count:5 2024-09-21T03:36:12.916 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:36:12 smithi039 ceph-mon[31211]: from='mgr.14118 172.21.15.39:0/2203062369' entity='mgr.x' 2024-09-21T03:36:13.623 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout Scheduled mgr update... 2024-09-21T03:36:14.333 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:36:13 smithi039 ceph-mon[31211]: from='client.14144 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:36:14.333 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:36:13 smithi039 ceph-mon[31211]: Saving service mgr spec with placement count:2 2024-09-21T03:36:14.334 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:36:13 smithi039 ceph-mon[31211]: from='mgr.14118 172.21.15.39:0/2203062369' entity='mgr.x' 2024-09-21T03:36:15.826 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:36:15 smithi039 ceph-mon[31211]: from='client.? 172.21.15.39:0/617711862' entity='client.admin' 2024-09-21T03:36:15.826 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:36:15 smithi039 ceph-mon[31211]: from='mgr.14118 172.21.15.39:0/2203062369' entity='mgr.x' 2024-09-21T03:36:15.826 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:36:15 smithi039 ceph-mon[31211]: from='mgr.14118 172.21.15.39:0/2203062369' entity='mgr.x' 2024-09-21T03:36:17.324 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:36:17 smithi039 ceph-mon[31211]: from='client.? 172.21.15.39:0/1066243919' entity='client.admin' 2024-09-21T03:36:17.324 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:36:17 smithi039 ceph-mon[31211]: from='mgr.14118 172.21.15.39:0/2203062369' entity='mgr.x' 2024-09-21T03:36:17.326 INFO:teuthology.orchestra.run.smithi039.stdout:Enabling the dashboard module... 2024-09-21T03:36:19.084 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:36:18 smithi039 ceph-mon[31211]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:36:19.084 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:36:18 smithi039 ceph-mon[31211]: from='mgr.14118 172.21.15.39:0/2203062369' entity='mgr.x' 2024-09-21T03:36:19.084 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:36:18 smithi039 ceph-mon[31211]: from='client.? 172.21.15.39:0/4268545213' entity='client.admin' cmd={"prefix": "mgr module enable", "module": "dashboard"} : dispatch 2024-09-21T03:36:19.084 INFO:journalctl@ceph.mgr.x.smithi039.stdout:Sep 21 03:36:18 smithi039 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-mgr-x[31438]: ignoring --setuser ceph since I am not root 2024-09-21T03:36:19.084 INFO:journalctl@ceph.mgr.x.smithi039.stdout:Sep 21 03:36:18 smithi039 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-mgr-x[31438]: ignoring --setgroup ceph since I am not root 2024-09-21T03:36:19.992 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:36:19 smithi039 ceph-mon[31211]: from='client.? 172.21.15.39:0/4268545213' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "dashboard"}]': finished 2024-09-21T03:36:19.992 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:36:19 smithi039 ceph-mon[31211]: mgrmap e10: x(active, since 21s) 2024-09-21T03:36:21.013 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout { 2024-09-21T03:36:21.013 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "epoch": 10, 2024-09-21T03:36:21.013 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "available": true, 2024-09-21T03:36:21.013 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-09-21T03:36:21.014 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-09-21T03:36:21.014 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout } 2024-09-21T03:36:21.014 INFO:teuthology.orchestra.run.smithi039.stdout:Waiting for the mgr to restart... 2024-09-21T03:36:21.014 INFO:teuthology.orchestra.run.smithi039.stdout:Waiting for mgr epoch 10... 2024-09-21T03:36:21.083 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:36:20 smithi039 ceph-mon[31211]: from='client.? 172.21.15.39:0/2408744551' entity='client.admin' cmd={"prefix": "mgr stat"} : dispatch 2024-09-21T03:36:24.583 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:36:24 smithi039 ceph-mon[31211]: Active manager daemon x restarted 2024-09-21T03:36:24.584 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:36:24 smithi039 ceph-mon[31211]: Activating manager daemon x 2024-09-21T03:36:24.584 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:36:24 smithi039 ceph-mon[31211]: osdmap e3: 0 total, 0 up, 0 in 2024-09-21T03:36:24.584 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:36:24 smithi039 ceph-mon[31211]: mgrmap e11: x(active, starting, since 0.0090338s) 2024-09-21T03:36:24.584 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:36:24 smithi039 ceph-mon[31211]: from='mgr.14152 172.21.15.39:0/3595829223' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-21T03:36:24.584 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:36:24 smithi039 ceph-mon[31211]: from='mgr.14152 172.21.15.39:0/3595829223' entity='mgr.x' cmd={"prefix": "mgr metadata", "who": "x", "id": "x"} : dispatch 2024-09-21T03:36:24.584 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:36:24 smithi039 ceph-mon[31211]: from='mgr.14152 172.21.15.39:0/3595829223' entity='mgr.x' cmd={"prefix": "mds metadata"} : dispatch 2024-09-21T03:36:24.584 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:36:24 smithi039 ceph-mon[31211]: from='mgr.14152 172.21.15.39:0/3595829223' entity='mgr.x' cmd={"prefix": "osd metadata"} : dispatch 2024-09-21T03:36:24.584 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:36:24 smithi039 ceph-mon[31211]: from='mgr.14152 172.21.15.39:0/3595829223' entity='mgr.x' cmd={"prefix": "mon metadata"} : dispatch 2024-09-21T03:36:24.584 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:36:24 smithi039 ceph-mon[31211]: Manager daemon x is now available 2024-09-21T03:36:25.408 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:36:25 smithi039 ceph-mon[31211]: from='mgr.14152 172.21.15.39:0/3595829223' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:36:25.408 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:36:25 smithi039 ceph-mon[31211]: from='mgr.14152 172.21.15.39:0/3595829223' entity='mgr.x' cmd={"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"} : dispatch 2024-09-21T03:36:25.408 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:36:25 smithi039 ceph-mon[31211]: from='mgr.14152 172.21.15.39:0/3595829223' entity='mgr.x' cmd={"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"} : dispatch 2024-09-21T03:36:25.408 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:36:25 smithi039 ceph-mon[31211]: mgrmap e12: x(active, since 1.0079s) 2024-09-21T03:36:25.685 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout { 2024-09-21T03:36:25.686 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 12, 2024-09-21T03:36:25.686 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "initialized": true 2024-09-21T03:36:25.686 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout } 2024-09-21T03:36:25.686 INFO:teuthology.orchestra.run.smithi039.stdout:mgr epoch 10 is available 2024-09-21T03:36:25.687 INFO:teuthology.orchestra.run.smithi039.stdout:Using certmgr to generate dashboard self-signed certificate... 2024-09-21T03:36:26.012 INFO:journalctl@ceph.mgr.x.smithi039.stdout:Sep 21 03:36:25 smithi039 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-mgr-x[31438]: 2024-09-21T03:36:25.711+0000 7f0f2b021640 -1 log_channel(cephadm) log [ERR] : [21/Sep/2024:03:36:25] ENGINE Error in HTTPServer.serve 2024-09-21T03:36:26.012 INFO:journalctl@ceph.mgr.x.smithi039.stdout:Sep 21 03:36:25 smithi039 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-mgr-x[31438]: Traceback (most recent call last): 2024-09-21T03:36:26.012 INFO:journalctl@ceph.mgr.x.smithi039.stdout:Sep 21 03:36:25 smithi039 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-mgr-x[31438]: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-21T03:36:26.012 INFO:journalctl@ceph.mgr.x.smithi039.stdout:Sep 21 03:36:25 smithi039 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-mgr-x[31438]: self._connections.run(self.expiration_interval) 2024-09-21T03:36:26.012 INFO:journalctl@ceph.mgr.x.smithi039.stdout:Sep 21 03:36:25 smithi039 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-mgr-x[31438]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-21T03:36:26.012 INFO:journalctl@ceph.mgr.x.smithi039.stdout:Sep 21 03:36:25 smithi039 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-mgr-x[31438]: self._run(expiration_interval) 2024-09-21T03:36:26.012 INFO:journalctl@ceph.mgr.x.smithi039.stdout:Sep 21 03:36:25 smithi039 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-mgr-x[31438]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-21T03:36:26.012 INFO:journalctl@ceph.mgr.x.smithi039.stdout:Sep 21 03:36:25 smithi039 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-mgr-x[31438]: new_conn = self._from_server_socket(self.server.socket) 2024-09-21T03:36:26.012 INFO:journalctl@ceph.mgr.x.smithi039.stdout:Sep 21 03:36:25 smithi039 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-mgr-x[31438]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-21T03:36:26.012 INFO:journalctl@ceph.mgr.x.smithi039.stdout:Sep 21 03:36:25 smithi039 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-mgr-x[31438]: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-21T03:36:26.012 INFO:journalctl@ceph.mgr.x.smithi039.stdout:Sep 21 03:36:25 smithi039 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-mgr-x[31438]: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-21T03:36:26.013 INFO:journalctl@ceph.mgr.x.smithi039.stdout:Sep 21 03:36:25 smithi039 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-mgr-x[31438]: s = self.context.wrap_socket( 2024-09-21T03:36:26.013 INFO:journalctl@ceph.mgr.x.smithi039.stdout:Sep 21 03:36:25 smithi039 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-mgr-x[31438]: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-21T03:36:26.013 INFO:journalctl@ceph.mgr.x.smithi039.stdout:Sep 21 03:36:25 smithi039 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-mgr-x[31438]: return self.sslsocket_class._create( 2024-09-21T03:36:26.013 INFO:journalctl@ceph.mgr.x.smithi039.stdout:Sep 21 03:36:25 smithi039 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-mgr-x[31438]: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-21T03:36:26.013 INFO:journalctl@ceph.mgr.x.smithi039.stdout:Sep 21 03:36:25 smithi039 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-mgr-x[31438]: self.do_handshake() 2024-09-21T03:36:26.013 INFO:journalctl@ceph.mgr.x.smithi039.stdout:Sep 21 03:36:25 smithi039 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-mgr-x[31438]: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-21T03:36:26.013 INFO:journalctl@ceph.mgr.x.smithi039.stdout:Sep 21 03:36:25 smithi039 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-mgr-x[31438]: self._sslobj.do_handshake() 2024-09-21T03:36:26.013 INFO:journalctl@ceph.mgr.x.smithi039.stdout:Sep 21 03:36:25 smithi039 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-mgr-x[31438]: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-21T03:36:26.013 INFO:journalctl@ceph.mgr.x.smithi039.stdout:Sep 21 03:36:25 smithi039 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-mgr-x[31438]: 2024-09-21T03:36:26.334 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:36:26 smithi039 ceph-mon[31211]: [21/Sep/2024:03:36:25] ENGINE Bus STARTING 2024-09-21T03:36:26.334 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:36:26 smithi039 ceph-mon[31211]: [21/Sep/2024:03:36:25] ENGINE Serving on http://172.21.15.39:8765 2024-09-21T03:36:26.334 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:36:26 smithi039 ceph-mon[31211]: [21/Sep/2024:03:36:25] ENGINE Serving on https://172.21.15.39:7150 2024-09-21T03:36:26.334 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:36:26 smithi039 ceph-mon[31211]: [21/Sep/2024:03:36:25] ENGINE Bus STARTED 2024-09-21T03:36:26.334 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:36:26 smithi039 ceph-mon[31211]: [21/Sep/2024:03:36:25] ENGINE Error in HTTPServer.serve 2024-09-21T03:36:26.334 INFO:journalctl@ceph.mon.a.smithi039.stdout: Traceback (most recent call last): 2024-09-21T03:36:26.334 INFO:journalctl@ceph.mon.a.smithi039.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-21T03:36:26.334 INFO:journalctl@ceph.mon.a.smithi039.stdout: self._connections.run(self.expiration_interval) 2024-09-21T03:36:26.334 INFO:journalctl@ceph.mon.a.smithi039.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-21T03:36:26.334 INFO:journalctl@ceph.mon.a.smithi039.stdout: self._run(expiration_interval) 2024-09-21T03:36:26.334 INFO:journalctl@ceph.mon.a.smithi039.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-21T03:36:26.335 INFO:journalctl@ceph.mon.a.smithi039.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-09-21T03:36:26.335 INFO:journalctl@ceph.mon.a.smithi039.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-21T03:36:26.335 INFO:journalctl@ceph.mon.a.smithi039.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-21T03:36:26.335 INFO:journalctl@ceph.mon.a.smithi039.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-21T03:36:26.335 INFO:journalctl@ceph.mon.a.smithi039.stdout: s = self.context.wrap_socket( 2024-09-21T03:36:26.335 INFO:journalctl@ceph.mon.a.smithi039.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-21T03:36:26.335 INFO:journalctl@ceph.mon.a.smithi039.stdout: return self.sslsocket_class._create( 2024-09-21T03:36:26.335 INFO:journalctl@ceph.mon.a.smithi039.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-21T03:36:26.335 INFO:journalctl@ceph.mon.a.smithi039.stdout: self.do_handshake() 2024-09-21T03:36:26.335 INFO:journalctl@ceph.mon.a.smithi039.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-21T03:36:26.335 INFO:journalctl@ceph.mon.a.smithi039.stdout: self._sslobj.do_handshake() 2024-09-21T03:36:26.335 INFO:journalctl@ceph.mon.a.smithi039.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-21T03:36:27.583 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:36:27 smithi039 ceph-mon[31211]: mgrmap e13: x(active, since 2s) 2024-09-21T03:36:27.584 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:36:27 smithi039 ceph-mon[31211]: from='client.14164 -' entity='client.admin' cmd=[{"prefix": "orch certmgr generate-certificates", "module_name": "dashboard", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:36:27.584 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:36:27 smithi039 ceph-mon[31211]: from='mgr.14152 172.21.15.39:0/3595829223' entity='mgr.x' 2024-09-21T03:36:28.261 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout {"cert": "-----BEGIN CERTIFICATE-----\nMIIE+zCCAuOgAwIBAgIUZu4wmdcS7mQaVQhdqdowqUlGWr0wDQYJKoZIhvcNAQEL\nBQAwFzEVMBMGA1UEAwwMY2VwaGFkbS1yb290MB4XDTI0MDkyMTAzMzYyN1oXDTM0\nMDkyMjAzMzYyN1owFzEVMBMGA1UEAwwMMTcyLjIxLjE1LjM5MIICIjANBgkqhkiG\n9w0BAQEFAAOCAg8AMIICCgKCAgEAsfrIy4yfhDBwUppBRAk3duHbx+t2MHn1Ge6f\nQzIICaRwijEwMZ6X3Yq0mRwcO6ImRKfCzxVKPgj0IB42t8tq08YYncNSnQebXOP7\nQ5yaxiqc3+si5eQB8UEEbu/bjp61lAFzweOiZxaT1NjMRQJ/DMTUlV5Xj+MEpcUd\nU7g16CJcADHfKG0q+E6tnXzPxTv+TFOuiC2yyALBNq8yfemvjuhqOkYc3yJ9/Ely\nGKCbgn9a0hXGwv+x/ks1auaY10sohgfxyQrbEaZm1QEKsFhJm73J1itvLMX4I4Mi\nNXgKzv8dlYOuQY3ZjqA2fP8kmhbaqkheWpfqYOKooLbdsRG8/ns/Yu9TJP6BGPkT\nnyso5K6Vxi4dUyNBDBvEbGArLNX2ijrmGUFKCss7U7fC1WdPWihC0JcsxJD2VRNS\nWaw/oG+zKZmQ0tK0gBmcCsACdUJqUxSwSe3e75PKrHBbijM5T7T/1Zv42oVkxZV0\nG54l6rI+Ubv6G0s3IIyxloIVSVVvlcpFkyaRm5PS+0u8aKBvj9vBystWWvLLCKiX\n+a/TlmPx6yq9YPaDAk+pUYFr/EzskUk9el/iUXamfZsMGgsbPrO+Ek5pgd3k2FFc\neNKBpIhSmicnfakuV4WWTtzur+K4SnBjvr9zSquKHq0obVWPjAWlyu7wZ1tarEPr\ndaA9RkECAwEAAaM/MD0wLQYDVR0RBCYwJIIJc21pdGhpMDM5ghFkYXNoYm9hcmRf\nc2VydmVyc4cErBUPJzAMBgNVHRMBAf8EAjAAMA0GCSqGSIb3DQEBCwUAA4ICAQBl\nzZ6SDggzWH/VU2Bse8PWmWveL5VN41denfqctq/CdbJgNLCXi7lNsBed5QuLdysm\nd6TL1Od/r1iHyyt4z27Ed2fRmGhPzU5MN3d68HXF0cvqu5y4AyB06MpQFBhZOkIn\nKZvDioV0uBJnealkFz9MEd85DyUkKsYRBLAOi57XU0ZNAZziGemE9xnQO0laFA4x\n2qzORXbcAG30jv3ET2/+gC1MG7HLO+9yC3aEbZmCFStckknRUkXzhF26jjEBJOFa\n5UkGDc/05cdnmMwh0dQxDpoYTUgw1+QSVGRXPvkM2+kQQlpDLuYoQmCIt8B83Jhf\nSfqXRp2tITTEb7q/iyuQp/lC4LCFxg/GWUHi5WBn0cdpxzgBgnW0EKAl8QaCVkyI\nm3/c21m0CJXlAxEnxtbZoIqO5+lMeLsjoG3xw4WE/opOv89X8+rPOTO3TY88ewVr\nARknoFCEagIuQ2E2LUqjt2CcQWiZ0bzOVm9A9c6CGrcbXBBxMn7Vwj9vY9dWAZla\nW3BweSeDFVbpfCRIgwXelF40o29P+dy0jokU4jojwCes6W4Gs+hBAHjtgKhf7njb\nAu/rwDNZO0yACcKBNoiY/iQAYZdf2yvzkCYYUfXHW+wma4aUSjf/Fs4IHtORbhL3\nkoV+F2RzRjyXNJhuWkdXKaJP74GN4p42imYD+Q65jg==\n-----END CERTIFICATE-----\n", "key": "-----BEGIN RSA PRIVATE KEY-----\nMIIJKAIBAAKCAgEAsfrIy4yfhDBwUppBRAk3duHbx+t2MHn1Ge6fQzIICaRwijEw\nMZ6X3Yq0mRwcO6ImRKfCzxVKPgj0IB42t8tq08YYncNSnQebXOP7Q5yaxiqc3+si\n5eQB8UEEbu/bjp61lAFzweOiZxaT1NjMRQJ/DMTUlV5Xj+MEpcUdU7g16CJcADHf\nKG0q+E6tnXzPxTv+TFOuiC2yyALBNq8yfemvjuhqOkYc3yJ9/ElyGKCbgn9a0hXG\nwv+x/ks1auaY10sohgfxyQrbEaZm1QEKsFhJm73J1itvLMX4I4MiNXgKzv8dlYOu\nQY3ZjqA2fP8kmhbaqkheWpfqYOKooLbdsRG8/ns/Yu9TJP6BGPkTnyso5K6Vxi4d\nUyNBDBvEbGArLNX2ijrmGUFKCss7U7fC1WdPWihC0JcsxJD2VRNSWaw/oG+zKZmQ\n0tK0gBmcCsACdUJqUxSwSe3e75PKrHBbijM5T7T/1Zv42oVkxZV0G54l6rI+Ubv6\nG0s3IIyxloIVSVVvlcpFkyaRm5PS+0u8aKBvj9vBystWWvLLCKiX+a/TlmPx6yq9\nYPaDAk+pUYFr/EzskUk9el/iUXamfZsMGgsbPrO+Ek5pgd3k2FFceNKBpIhSmicn\nfakuV4WWTtzur+K4SnBjvr9zSquKHq0obVWPjAWlyu7wZ1tarEPrdaA9RkECAwEA\nAQKCAgBSWODfwPwrSsglh+aLpqIJeL3aNZkVG9BqQ6xfiD4MzA8f+Vs9kg3c/KjL\nsKRleQQwvtgVPVoA8t6VH7Wce1s9icno8K8AJb1eEMO4SDSaXKwBtIcMi97UM4ts\nEsForwj/Zl6bRonlHZ6fnAGDKoEDNOzRAXoNAnIritSjYDdc5MzhcDG6wCviAHE2\nzUFKlhP2V7+IlXBDN3cTKKN5e451VhaUSM1h9lEJUigeuezJx8gLLPUuUuUVbdHU\n2MW90zH4NaPKq6GpfG7Wvekmp2rDSuAgni8Nrbk+IRXKXHTPylzzSR5uiUWwRnN/\niQNz4gAhSmORkflpA2YXGhTfsP/ajwN8E81AgFPtxplMGZCaBTuGcaA1TfaRJ46B\nI5wpcLvfl6nqTYO8Wm1q6OTeKnAtJRRgsCCT+9g4ZJO3M2zVTazCaxZyYbvfwQBQ\n31XASJqB3xuNxSnB4DdUfwNlD3YnLuskifzonwAYuws1CbHq8G7kg+7xyfqYWVWp\n2SVu+AGV8EEYHzHQ7QAPwt1Sw7O4Uzd9SgCWwcLcHK8mcVQJ4NGvpmafCOJX026p\n9uFh7yniZGNuugVECuMOkXARi75HL+z+vtJZXpzLuzESZ0V9rLE+37kM4TSdH263\nkUKdtt7CCdUPoMDKfsPy9nCjXd/uQwgN5S4tN0E9Fb8rmz2PUwKCAQEA4DXSAnd/\n1SRts5HcoC44vJnc4d0/9+zsKLs6/4lLDVYH45NyBxAFMgVxwjN0N74yRE3+kr9s\n2qfczv4TDhQhjZ3AYYwHE2SUCUPW1H4km8vCXhh4KyKBxd0dbxXOZZtKbPWh8Xm+\nx3YMu7W3T8GYWnHNjJYhLPQKJVZIPen0OhBI387czOa7cK7ShiW0YtyWUEzX14G7\nD8XS53WIiB9cbq2uEugY+lhXBk8Yxrmf2TI5SnyoOz/1hIb3t+IaiMZmvMQtB1Uq\ngXLgBKvT+c7bp6K+6iyOTzo+BoESCI3x7N/QvOA7hmPZ4HO0U8ehcJyMtHNan9Tb\n/bLgCPbRsYDBywKCAQEAyzbtY97FRJxp8Dtl2DXNgG1BcPhRWxbfd9g/5uaJXHu/\nVqFo+x45zlVGBa/xMDBaQKJves+Wg0yJ5Y90uc1a0hNzmJYHkM7urYlI6C5VuMs2\nzIGiIUwAxvcfTGDjdqGxIjbgRfrKisB0bkUvb3dO0LKvzQoBwD4HE6tmHkBUAoij\n1neaopP9rNgPVcDekkY+hFQh+B/otrZ1bY8sQWLwhYL62yvCUE1csTi1rMA5tMzz\nYODm1P/rVDiGrC0o83ngp5fiYiZX6yyKHISx/0K/eJpzHtLv+VY0KocXCEos3LQv\njbT/w96eiwv/5LapzqzYyRvFYO75ApkzsKFcHMFmowKCAQBTX3ASfsFITt2EMq8p\n2UY7y7M5Qf6xgKZolwbfUvxXnzAbzg5O1gQ4j5G3t052FAG5KI+LnicqP9V6GzyW\ntaGseYa6P0m6SmJJ97IBHk8ap/BE8TBuScyb+Rci4ab4XYCVd9maY70vipQ9kNrz\ni3fN2henOYRopO7I54bBjyfEF54hq1ir0PjOdBPQ0mB7iq9SeOCiQJmAyf0dGLWG\neLL0bR0X5rqzDPp/XBZIf2oTZ/up2STts2DmwRGIjUl0puI5O3HLEj5WGJN46TRP\n5j4D3h7EEYFxsKSz4ncEO1RE0w1ZzsATvzmbnyrlJIzBxVUFHQgL+yk687MfYjij\nms91AoIBAH1DuIWNdrBacnshQiBK2ghGUMagaxXpPYsUhB+DED0wErwF9z6EXUGx\nEt7Cm7wGsjuRZYKgn7kD3yTgjyVRTqCcWO8/3B4JUNsr+wMUPMcSjQ73Acfgn8IF\n5wcf2lbwSM48YXcBpwRsqdinURw2wSCoZPHRL08fbSFU0Dfpk1+UdtnnFC0PaJm4\n/YrwvsIQBs1vRxk6IisuwaftT2xm3riOCrA7gyYY6gBcYrDANVIb3BJl0CSQNLHP\nvkQ3+jxLbo4gc+3okN1osSHvMgGVVY20Ny5S/JYH61QjvbDgeFL7xyxk3BwTCEvR\n88U6S+YYafFd6wdbh6dwZIqGjM808F0CggEBAIYN/i0sXtSQ8JqFJefdk5U35hSN\nFEuctQBESatlfCjde9iNuD1CbuUb9u7XHHBTAR7oraQ/fiZKW7ce6qn+3kDxGo8A\nDlrGUs+9Qtvt5zanqM726yw02ALfrhuWvrg9TQC5I6a7egg25X1jITdC33hSfTDD\n9y6PSN/faXaD9dlGkJlFoieaKKkgd+UT6fpAyewkKDMInylHsCsIBIenHeukO6fU\n7Chpx508t7wqgEIgvDFMRXbBQXvCy2aJlJixvETCrYyI5v2nNYaUBBMvnNPtmSwG\nwJw2KGjwk2H6SU/BnC2UFtjda7NOFTeYsateFeVj0IjbkpNcE+oVpXWLgxw=\n-----END RSA PRIVATE KEY-----\n"} 2024-09-21T03:36:28.583 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:36:28 smithi039 ceph-mon[31211]: from='mgr.14152 172.21.15.39:0/3595829223' entity='mgr.x' 2024-09-21T03:36:29.747 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout SSL certificate updated 2024-09-21T03:36:30.189 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:36:30 smithi039 ceph-mon[31211]: from='client.14166 -' entity='client.admin' cmd=[{"prefix": "dashboard set-ssl-certificate", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:36:30.189 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:36:30 smithi039 ceph-mon[31211]: from='mgr.14152 172.21.15.39:0/3595829223' entity='mgr.x' 2024-09-21T03:36:31.231 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout SSL certificate key updated 2024-09-21T03:36:31.232 INFO:teuthology.orchestra.run.smithi039.stdout:Creating initial admin user... 2024-09-21T03:36:31.556 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:36:31 smithi039 ceph-mon[31211]: from='client.14168 -' entity='client.admin' cmd=[{"prefix": "dashboard set-ssl-certificate-key", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:36:31.556 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:36:31 smithi039 ceph-mon[31211]: from='mgr.14152 172.21.15.39:0/3595829223' entity='mgr.x' 2024-09-21T03:36:32.729 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout {"username": "admin", "password": "$2b$12$w9uXoc3KIZRO0IrWnR6opuLcQBOxCOHFy9I1tM36uh/v8L2LfB6ty", "roles": ["administrator"], "name": null, "email": null, "lastUpdate": 1726889792, "enabled": true, "pwdExpirationDate": null, "pwdUpdateRequired": true} 2024-09-21T03:36:32.729 INFO:teuthology.orchestra.run.smithi039.stdout:Fetching dashboard port number... 2024-09-21T03:36:33.312 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:36:33 smithi039 ceph-mon[31211]: from='client.14170 -' entity='client.admin' cmd=[{"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:36:33.312 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:36:33 smithi039 ceph-mon[31211]: from='mgr.14152 172.21.15.39:0/3595829223' entity='mgr.x' 2024-09-21T03:36:33.852 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout 8443 2024-09-21T03:36:33.862 INFO:teuthology.orchestra.run.smithi039.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-09-21T03:36:33.862 INFO:teuthology.orchestra.run.smithi039.stdout:systemctl: stdout disabled 2024-09-21T03:36:33.870 INFO:teuthology.orchestra.run.smithi039.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-09-21T03:36:33.870 INFO:teuthology.orchestra.run.smithi039.stdout:systemctl: stdout inactive 2024-09-21T03:36:33.870 INFO:teuthology.orchestra.run.smithi039.stdout:firewalld.service is not enabled 2024-09-21T03:36:33.870 INFO:teuthology.orchestra.run.smithi039.stdout:Not possible to open ports <[8443]>. firewalld.service is not available 2024-09-21T03:36:33.872 INFO:teuthology.orchestra.run.smithi039.stdout:Ceph Dashboard is now available at: 2024-09-21T03:36:33.872 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T03:36:33.872 INFO:teuthology.orchestra.run.smithi039.stdout: URL: https://smithi039.front.sepia.ceph.com:8443/ 2024-09-21T03:36:33.873 INFO:teuthology.orchestra.run.smithi039.stdout: User: admin 2024-09-21T03:36:33.873 INFO:teuthology.orchestra.run.smithi039.stdout: Password: 69d4yj86hj 2024-09-21T03:36:33.873 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T03:36:33.873 INFO:teuthology.orchestra.run.smithi039.stdout:Saving cluster configuration to /var/lib/ceph/51116d5c-77ca-11ef-baf5-efdc52797490/config directory 2024-09-21T03:36:34.583 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:36:34 smithi039 ceph-mon[31211]: from='client.? 172.21.15.39:0/4103346254' entity='client.admin' cmd={"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"} : dispatch 2024-09-21T03:36:35.233 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr set mgr/dashboard/cluster/status 2024-09-21T03:36:35.233 INFO:teuthology.orchestra.run.smithi039.stdout:You can access the Ceph CLI as following in case of multi-cluster or non-default config: 2024-09-21T03:36:35.233 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T03:36:35.233 INFO:teuthology.orchestra.run.smithi039.stdout: sudo /home/ubuntu/cephtest/cephadm shell --fsid 51116d5c-77ca-11ef-baf5-efdc52797490 -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring 2024-09-21T03:36:35.234 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T03:36:35.234 INFO:teuthology.orchestra.run.smithi039.stdout:Or, if you are only running a single cluster on this host: 2024-09-21T03:36:35.234 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T03:36:35.234 INFO:teuthology.orchestra.run.smithi039.stdout: sudo /home/ubuntu/cephtest/cephadm shell 2024-09-21T03:36:35.234 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T03:36:35.234 INFO:teuthology.orchestra.run.smithi039.stdout:Please consider enabling telemetry to help improve Ceph: 2024-09-21T03:36:35.234 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T03:36:35.234 INFO:teuthology.orchestra.run.smithi039.stdout: ceph telemetry on 2024-09-21T03:36:35.234 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T03:36:35.234 INFO:teuthology.orchestra.run.smithi039.stdout:For more information see: 2024-09-21T03:36:35.234 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T03:36:35.234 INFO:teuthology.orchestra.run.smithi039.stdout: https://docs.ceph.com/en/latest/mgr/telemetry/ 2024-09-21T03:36:35.234 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T03:36:35.235 INFO:teuthology.orchestra.run.smithi039.stdout:Bootstrap complete. 2024-09-21T03:36:35.272 INFO:tasks.cephadm:Fetching config... 2024-09-21T03:36:35.272 DEBUG:teuthology.orchestra.run.smithi039:> set -ex 2024-09-21T03:36:35.272 DEBUG:teuthology.orchestra.run.smithi039:> dd if=/etc/ceph/ceph.conf of=/dev/stdout 2024-09-21T03:36:35.288 INFO:tasks.cephadm:Fetching client.admin keyring... 2024-09-21T03:36:35.288 DEBUG:teuthology.orchestra.run.smithi039:> set -ex 2024-09-21T03:36:35.288 DEBUG:teuthology.orchestra.run.smithi039:> dd if=/etc/ceph/ceph.client.admin.keyring of=/dev/stdout 2024-09-21T03:36:35.345 INFO:tasks.cephadm:Fetching mon keyring... 2024-09-21T03:36:35.345 DEBUG:teuthology.orchestra.run.smithi039:> set -ex 2024-09-21T03:36:35.345 DEBUG:teuthology.orchestra.run.smithi039:> sudo dd if=/var/lib/ceph/51116d5c-77ca-11ef-baf5-efdc52797490/mon.a/keyring of=/dev/stdout 2024-09-21T03:36:35.411 INFO:tasks.cephadm:Fetching pub ssh key... 2024-09-21T03:36:35.411 DEBUG:teuthology.orchestra.run.smithi039:> set -ex 2024-09-21T03:36:35.411 DEBUG:teuthology.orchestra.run.smithi039:> dd if=/home/ubuntu/cephtest/ceph.pub of=/dev/stdout 2024-09-21T03:36:35.467 INFO:tasks.cephadm:Installing pub ssh key for root users... 2024-09-21T03:36:35.467 DEBUG:teuthology.orchestra.run.smithi039:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCdXyV6VsXd0Yn4HDH1fa4yXMDTjUd9Ae7U+4BBYwRjrbJ4WGOsw4tmCk8fIKruyIWwSwsvgh3+43Ay2Mrz0cqW6meMFojSme8pjU3Qg1E3Ugb/kk8lI2hS+Hv3sIWzaXvToAEXeliGygWkdmru1I9sd7KU3HfqdmBl+FDHC5OfOE+siobc+PsLhx0EurpOM7ChYK50KSCEN5IsQU9alrzSWnFb/6mUF4a5SRTzFNznVClrig2gK46DYoktqCrcCpByFCHBuKZULwcMVQM+qZQcTP6P9EMvOSYUvzuAYb0bRlfZaLDQ/AMO8QI11tq1ipmOPeTGMQ2aZSaG9SYs9iaHeYLUc+DCqEsUomIFAr3q4QOZ9lXTF3pjo0NesWui7QL5CMzCZR/UC6OBBEpg0moqEE69NG/BGScaA3yuh9y/fMAQKubHETK4IDnIJlVpurzagAPfOZREYSwFQM/BBnEVeb3DHrJV9/Quwn1h4s8t42K3TnohbALTj2emxy/w3+s= ceph-51116d5c-77ca-11ef-baf5-efdc52797490' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-09-21T03:36:35.542 INFO:teuthology.orchestra.run.smithi039.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCdXyV6VsXd0Yn4HDH1fa4yXMDTjUd9Ae7U+4BBYwRjrbJ4WGOsw4tmCk8fIKruyIWwSwsvgh3+43Ay2Mrz0cqW6meMFojSme8pjU3Qg1E3Ugb/kk8lI2hS+Hv3sIWzaXvToAEXeliGygWkdmru1I9sd7KU3HfqdmBl+FDHC5OfOE+siobc+PsLhx0EurpOM7ChYK50KSCEN5IsQU9alrzSWnFb/6mUF4a5SRTzFNznVClrig2gK46DYoktqCrcCpByFCHBuKZULwcMVQM+qZQcTP6P9EMvOSYUvzuAYb0bRlfZaLDQ/AMO8QI11tq1ipmOPeTGMQ2aZSaG9SYs9iaHeYLUc+DCqEsUomIFAr3q4QOZ9lXTF3pjo0NesWui7QL5CMzCZR/UC6OBBEpg0moqEE69NG/BGScaA3yuh9y/fMAQKubHETK4IDnIJlVpurzagAPfOZREYSwFQM/BBnEVeb3DHrJV9/Quwn1h4s8t42K3TnohbALTj2emxy/w3+s= ceph-51116d5c-77ca-11ef-baf5-efdc52797490 2024-09-21T03:36:35.554 DEBUG:teuthology.orchestra.run.smithi150:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCdXyV6VsXd0Yn4HDH1fa4yXMDTjUd9Ae7U+4BBYwRjrbJ4WGOsw4tmCk8fIKruyIWwSwsvgh3+43Ay2Mrz0cqW6meMFojSme8pjU3Qg1E3Ugb/kk8lI2hS+Hv3sIWzaXvToAEXeliGygWkdmru1I9sd7KU3HfqdmBl+FDHC5OfOE+siobc+PsLhx0EurpOM7ChYK50KSCEN5IsQU9alrzSWnFb/6mUF4a5SRTzFNznVClrig2gK46DYoktqCrcCpByFCHBuKZULwcMVQM+qZQcTP6P9EMvOSYUvzuAYb0bRlfZaLDQ/AMO8QI11tq1ipmOPeTGMQ2aZSaG9SYs9iaHeYLUc+DCqEsUomIFAr3q4QOZ9lXTF3pjo0NesWui7QL5CMzCZR/UC6OBBEpg0moqEE69NG/BGScaA3yuh9y/fMAQKubHETK4IDnIJlVpurzagAPfOZREYSwFQM/BBnEVeb3DHrJV9/Quwn1h4s8t42K3TnohbALTj2emxy/w3+s= ceph-51116d5c-77ca-11ef-baf5-efdc52797490' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-09-21T03:36:35.607 INFO:teuthology.orchestra.run.smithi150.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCdXyV6VsXd0Yn4HDH1fa4yXMDTjUd9Ae7U+4BBYwRjrbJ4WGOsw4tmCk8fIKruyIWwSwsvgh3+43Ay2Mrz0cqW6meMFojSme8pjU3Qg1E3Ugb/kk8lI2hS+Hv3sIWzaXvToAEXeliGygWkdmru1I9sd7KU3HfqdmBl+FDHC5OfOE+siobc+PsLhx0EurpOM7ChYK50KSCEN5IsQU9alrzSWnFb/6mUF4a5SRTzFNznVClrig2gK46DYoktqCrcCpByFCHBuKZULwcMVQM+qZQcTP6P9EMvOSYUvzuAYb0bRlfZaLDQ/AMO8QI11tq1ipmOPeTGMQ2aZSaG9SYs9iaHeYLUc+DCqEsUomIFAr3q4QOZ9lXTF3pjo0NesWui7QL5CMzCZR/UC6OBBEpg0moqEE69NG/BGScaA3yuh9y/fMAQKubHETK4IDnIJlVpurzagAPfOZREYSwFQM/BBnEVeb3DHrJV9/Quwn1h4s8t42K3TnohbALTj2emxy/w3+s= ceph-51116d5c-77ca-11ef-baf5-efdc52797490 2024-09-21T03:36:35.621 DEBUG:teuthology.orchestra.run.smithi161:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCdXyV6VsXd0Yn4HDH1fa4yXMDTjUd9Ae7U+4BBYwRjrbJ4WGOsw4tmCk8fIKruyIWwSwsvgh3+43Ay2Mrz0cqW6meMFojSme8pjU3Qg1E3Ugb/kk8lI2hS+Hv3sIWzaXvToAEXeliGygWkdmru1I9sd7KU3HfqdmBl+FDHC5OfOE+siobc+PsLhx0EurpOM7ChYK50KSCEN5IsQU9alrzSWnFb/6mUF4a5SRTzFNznVClrig2gK46DYoktqCrcCpByFCHBuKZULwcMVQM+qZQcTP6P9EMvOSYUvzuAYb0bRlfZaLDQ/AMO8QI11tq1ipmOPeTGMQ2aZSaG9SYs9iaHeYLUc+DCqEsUomIFAr3q4QOZ9lXTF3pjo0NesWui7QL5CMzCZR/UC6OBBEpg0moqEE69NG/BGScaA3yuh9y/fMAQKubHETK4IDnIJlVpurzagAPfOZREYSwFQM/BBnEVeb3DHrJV9/Quwn1h4s8t42K3TnohbALTj2emxy/w3+s= ceph-51116d5c-77ca-11ef-baf5-efdc52797490' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-09-21T03:36:35.679 INFO:teuthology.orchestra.run.smithi161.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCdXyV6VsXd0Yn4HDH1fa4yXMDTjUd9Ae7U+4BBYwRjrbJ4WGOsw4tmCk8fIKruyIWwSwsvgh3+43Ay2Mrz0cqW6meMFojSme8pjU3Qg1E3Ugb/kk8lI2hS+Hv3sIWzaXvToAEXeliGygWkdmru1I9sd7KU3HfqdmBl+FDHC5OfOE+siobc+PsLhx0EurpOM7ChYK50KSCEN5IsQU9alrzSWnFb/6mUF4a5SRTzFNznVClrig2gK46DYoktqCrcCpByFCHBuKZULwcMVQM+qZQcTP6P9EMvOSYUvzuAYb0bRlfZaLDQ/AMO8QI11tq1ipmOPeTGMQ2aZSaG9SYs9iaHeYLUc+DCqEsUomIFAr3q4QOZ9lXTF3pjo0NesWui7QL5CMzCZR/UC6OBBEpg0moqEE69NG/BGScaA3yuh9y/fMAQKubHETK4IDnIJlVpurzagAPfOZREYSwFQM/BBnEVeb3DHrJV9/Quwn1h4s8t42K3TnohbALTj2emxy/w3+s= ceph-51116d5c-77ca-11ef-baf5-efdc52797490 2024-09-21T03:36:35.694 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 51116d5c-77ca-11ef-baf5-efdc52797490 -- ceph config set mgr mgr/cephadm/allow_ptrace true 2024-09-21T03:36:35.800 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:36:35 smithi039 ceph-mon[31211]: from='client.? 172.21.15.39:0/4027733766' entity='client.admin' 2024-09-21T03:36:35.948 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/51116d5c-77ca-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:36:37.738 INFO:tasks.cephadm:Distributing conf and client.admin keyring to all hosts + 0755 2024-09-21T03:36:37.739 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 51116d5c-77ca-11ef-baf5-efdc52797490 -- ceph orch client-keyring set client.admin '*' --mode 0755 2024-09-21T03:36:37.833 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:36:37 smithi039 ceph-mon[31211]: mgrmap e14: x(active, since 12s) 2024-09-21T03:36:37.833 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:36:37 smithi039 ceph-mon[31211]: from='client.? 172.21.15.39:0/4277314191' entity='client.admin' 2024-09-21T03:36:38.314 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/51116d5c-77ca-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:36:40.088 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi150 2024-09-21T03:36:40.088 DEBUG:teuthology.orchestra.run.smithi150:> set -ex 2024-09-21T03:36:40.088 DEBUG:teuthology.orchestra.run.smithi150:> dd of=/etc/ceph/ceph.conf 2024-09-21T03:36:40.107 DEBUG:teuthology.orchestra.run.smithi150:> set -ex 2024-09-21T03:36:40.108 DEBUG:teuthology.orchestra.run.smithi150:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-09-21T03:36:40.166 INFO:tasks.cephadm:Adding host smithi150 to orchestrator... 2024-09-21T03:36:40.166 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 51116d5c-77ca-11ef-baf5-efdc52797490 -- ceph orch host add smithi150 2024-09-21T03:36:40.333 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:36:40 smithi039 ceph-mon[31211]: from='client.14178 -' entity='client.admin' cmd=[{"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:36:40.334 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:36:40 smithi039 ceph-mon[31211]: from='mgr.14152 172.21.15.39:0/3595829223' entity='mgr.x' 2024-09-21T03:36:40.334 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:36:40 smithi039 ceph-mon[31211]: from='mgr.14152 172.21.15.39:0/3595829223' entity='mgr.x' 2024-09-21T03:36:40.334 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:36:40 smithi039 ceph-mon[31211]: from='mgr.14152 172.21.15.39:0/3595829223' entity='mgr.x' 2024-09-21T03:36:40.334 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:36:40 smithi039 ceph-mon[31211]: from='mgr.14152 172.21.15.39:0/3595829223' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd/host:smithi039", "name": "osd_memory_target"} : dispatch 2024-09-21T03:36:40.334 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:36:40 smithi039 ceph-mon[31211]: from='mgr.14152 172.21.15.39:0/3595829223' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:36:40.334 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:36:40 smithi039 ceph-mon[31211]: from='mgr.14152 172.21.15.39:0/3595829223' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:36:40.334 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:36:40 smithi039 ceph-mon[31211]: Updating smithi039:/etc/ceph/ceph.conf 2024-09-21T03:36:40.334 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:36:40 smithi039 ceph-mon[31211]: Updating smithi039:/var/lib/ceph/51116d5c-77ca-11ef-baf5-efdc52797490/config/ceph.conf 2024-09-21T03:36:40.334 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:36:40 smithi039 ceph-mon[31211]: Updating smithi039:/etc/ceph/ceph.client.admin.keyring 2024-09-21T03:36:40.335 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:36:40 smithi039 ceph-mon[31211]: Updating smithi039:/var/lib/ceph/51116d5c-77ca-11ef-baf5-efdc52797490/config/ceph.client.admin.keyring 2024-09-21T03:36:40.335 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:36:40 smithi039 ceph-mon[31211]: from='mgr.14152 172.21.15.39:0/3595829223' entity='mgr.x' 2024-09-21T03:36:40.335 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:36:40 smithi039 ceph-mon[31211]: from='mgr.14152 172.21.15.39:0/3595829223' entity='mgr.x' 2024-09-21T03:36:40.335 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:36:40 smithi039 ceph-mon[31211]: from='mgr.14152 172.21.15.39:0/3595829223' entity='mgr.x' 2024-09-21T03:36:40.335 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:36:40 smithi039 ceph-mon[31211]: from='mgr.14152 172.21.15.39:0/3595829223' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:36:40.335 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:36:40 smithi039 ceph-mon[31211]: from='mgr.14152 172.21.15.39:0/3595829223' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:36:40.335 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:36:40 smithi039 ceph-mon[31211]: from='mgr.14152 172.21.15.39:0/3595829223' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:36:40.335 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:36:40 smithi039 ceph-mon[31211]: from='mgr.14152 172.21.15.39:0/3595829223' entity='mgr.x' 2024-09-21T03:36:40.418 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/51116d5c-77ca-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:36:42.334 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:36:41 smithi039 ceph-mon[31211]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi150", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:36:43.333 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:36:42 smithi039 ceph-mon[31211]: Deploying cephadm binary to smithi150 2024-09-21T03:36:44.808 INFO:teuthology.orchestra.run.smithi039.stdout:Added host 'smithi150' with addr '172.21.15.150' 2024-09-21T03:36:45.354 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 51116d5c-77ca-11ef-baf5-efdc52797490 -- ceph orch host ls --format=json 2024-09-21T03:36:45.603 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/51116d5c-77ca-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:36:46.003 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:36:45 smithi039 ceph-mon[31211]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:36:46.003 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:36:45 smithi039 ceph-mon[31211]: from='mgr.14152 172.21.15.39:0/3595829223' entity='mgr.x' 2024-09-21T03:36:46.003 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:36:45 smithi039 ceph-mon[31211]: Added host smithi150 2024-09-21T03:36:46.003 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:36:45 smithi039 ceph-mon[31211]: from='mgr.14152 172.21.15.39:0/3595829223' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:36:46.003 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:36:45 smithi039 ceph-mon[31211]: from='mgr.14152 172.21.15.39:0/3595829223' entity='mgr.x' 2024-09-21T03:36:46.003 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:36:45 smithi039 ceph-mon[31211]: from='mgr.14152 172.21.15.39:0/3595829223' entity='mgr.x' 2024-09-21T03:36:46.304 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T03:36:46.304 INFO:teuthology.orchestra.run.smithi039.stdout:[{"addr": "172.21.15.39", "hostname": "smithi039", "labels": [], "status": ""}, {"addr": "172.21.15.150", "hostname": "smithi150", "labels": [], "status": ""}] 2024-09-21T03:36:47.004 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi161 2024-09-21T03:36:47.004 DEBUG:teuthology.orchestra.run.smithi161:> set -ex 2024-09-21T03:36:47.004 DEBUG:teuthology.orchestra.run.smithi161:> dd of=/etc/ceph/ceph.conf 2024-09-21T03:36:47.023 DEBUG:teuthology.orchestra.run.smithi161:> set -ex 2024-09-21T03:36:47.023 DEBUG:teuthology.orchestra.run.smithi161:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-09-21T03:36:47.081 INFO:tasks.cephadm:Adding host smithi161 to orchestrator... 2024-09-21T03:36:47.082 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 51116d5c-77ca-11ef-baf5-efdc52797490 -- ceph orch host add smithi161 2024-09-21T03:36:47.333 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:36:47 smithi039 ceph-mon[31211]: from='mgr.14152 172.21.15.39:0/3595829223' entity='mgr.x' 2024-09-21T03:36:47.334 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:36:47 smithi039 ceph-mon[31211]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:36:47.334 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:36:47 smithi039 ceph-mon[31211]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T03:36:47.344 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/51116d5c-77ca-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:36:49.583 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:36:49 smithi039 ceph-mon[31211]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi161", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:36:49.583 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:36:49 smithi039 ceph-mon[31211]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:36:49.584 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:36:49 smithi039 ceph-mon[31211]: from='mgr.14152 172.21.15.39:0/3595829223' entity='mgr.x' 2024-09-21T03:36:49.584 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:36:49 smithi039 ceph-mon[31211]: from='mgr.14152 172.21.15.39:0/3595829223' entity='mgr.x' 2024-09-21T03:36:49.584 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:36:49 smithi039 ceph-mon[31211]: from='mgr.14152 172.21.15.39:0/3595829223' entity='mgr.x' 2024-09-21T03:36:49.584 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:36:49 smithi039 ceph-mon[31211]: from='mgr.14152 172.21.15.39:0/3595829223' entity='mgr.x' 2024-09-21T03:36:49.584 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:36:49 smithi039 ceph-mon[31211]: from='mgr.14152 172.21.15.39:0/3595829223' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd/host:smithi150", "name": "osd_memory_target"} : dispatch 2024-09-21T03:36:49.584 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:36:49 smithi039 ceph-mon[31211]: from='mgr.14152 172.21.15.39:0/3595829223' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:36:49.584 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:36:49 smithi039 ceph-mon[31211]: from='mgr.14152 172.21.15.39:0/3595829223' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:36:50.833 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:36:50 smithi039 ceph-mon[31211]: Updating smithi150:/etc/ceph/ceph.conf 2024-09-21T03:36:50.834 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:36:50 smithi039 ceph-mon[31211]: Updating smithi150:/var/lib/ceph/51116d5c-77ca-11ef-baf5-efdc52797490/config/ceph.conf 2024-09-21T03:36:50.834 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:36:50 smithi039 ceph-mon[31211]: Deploying cephadm binary to smithi161 2024-09-21T03:36:50.834 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:36:50 smithi039 ceph-mon[31211]: Updating smithi150:/etc/ceph/ceph.client.admin.keyring 2024-09-21T03:36:50.834 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:36:50 smithi039 ceph-mon[31211]: Updating smithi150:/var/lib/ceph/51116d5c-77ca-11ef-baf5-efdc52797490/config/ceph.client.admin.keyring 2024-09-21T03:36:50.834 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:36:50 smithi039 ceph-mon[31211]: from='mgr.14152 172.21.15.39:0/3595829223' entity='mgr.x' 2024-09-21T03:36:50.834 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:36:50 smithi039 ceph-mon[31211]: from='mgr.14152 172.21.15.39:0/3595829223' entity='mgr.x' 2024-09-21T03:36:50.834 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:36:50 smithi039 ceph-mon[31211]: from='mgr.14152 172.21.15.39:0/3595829223' entity='mgr.x' 2024-09-21T03:36:51.812 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:36:51 smithi039 ceph-mon[31211]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:36:51.812 INFO:teuthology.orchestra.run.smithi039.stdout:Added host 'smithi161' with addr '172.21.15.161' 2024-09-21T03:36:52.416 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 51116d5c-77ca-11ef-baf5-efdc52797490 -- ceph orch host ls --format=json 2024-09-21T03:36:52.664 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/51116d5c-77ca-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:36:53.083 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:36:52 smithi039 ceph-mon[31211]: from='mgr.14152 172.21.15.39:0/3595829223' entity='mgr.x' 2024-09-21T03:36:53.084 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:36:52 smithi039 ceph-mon[31211]: Added host smithi161 2024-09-21T03:36:53.084 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:36:52 smithi039 ceph-mon[31211]: from='mgr.14152 172.21.15.39:0/3595829223' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:36:53.084 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:36:52 smithi039 ceph-mon[31211]: from='mgr.14152 172.21.15.39:0/3595829223' entity='mgr.x' 2024-09-21T03:36:53.084 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:36:52 smithi039 ceph-mon[31211]: from='mgr.14152 172.21.15.39:0/3595829223' entity='mgr.x' 2024-09-21T03:36:53.314 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T03:36:53.314 INFO:teuthology.orchestra.run.smithi039.stdout:[{"addr": "172.21.15.39", "hostname": "smithi039", "labels": [], "status": ""}, {"addr": "172.21.15.150", "hostname": "smithi150", "labels": [], "status": ""}, {"addr": "172.21.15.161", "hostname": "smithi161", "labels": [], "status": ""}] 2024-09-21T03:36:53.859 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:36:53 smithi039 ceph-mon[31211]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:36:53.860 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:36:53 smithi039 ceph-mon[31211]: from='mgr.14152 172.21.15.39:0/3595829223' entity='mgr.x' 2024-09-21T03:36:53.888 INFO:tasks.cephadm:Setting crush tunables to default 2024-09-21T03:36:53.888 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 51116d5c-77ca-11ef-baf5-efdc52797490 -- ceph osd crush tunables default 2024-09-21T03:36:54.135 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/51116d5c-77ca-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:36:54.815 INFO:teuthology.orchestra.run.smithi039.stderr:adjusted tunables profile to default 2024-09-21T03:36:55.085 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:36:54 smithi039 ceph-mon[31211]: from='client.14186 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T03:36:55.086 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:36:54 smithi039 ceph-mon[31211]: from='client.? 172.21.15.39:0/1846763711' entity='client.admin' cmd={"prefix": "osd crush tunables", "profile": "default"} : dispatch 2024-09-21T03:36:55.411 INFO:tasks.cephadm:Adding mon.a on smithi039 2024-09-21T03:36:55.411 INFO:tasks.cephadm:Adding mon.b on smithi150 2024-09-21T03:36:55.411 INFO:tasks.cephadm:Adding mon.c on smithi161 2024-09-21T03:36:55.411 DEBUG:teuthology.orchestra.run.smithi161:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 51116d5c-77ca-11ef-baf5-efdc52797490 -- ceph orch apply mon '3;smithi039:172.21.15.39=a;smithi150:172.21.15.150=b;smithi161:172.21.15.161=c' 2024-09-21T03:36:55.842 INFO:teuthology.orchestra.run.smithi161.stderr:Inferring config /etc/ceph/ceph.conf 2024-09-21T03:36:55.888 INFO:teuthology.orchestra.run.smithi161.stderr:Inferring config /etc/ceph/ceph.conf 2024-09-21T03:36:56.083 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:36:55 smithi039 ceph-mon[31211]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:36:56.083 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:36:55 smithi039 ceph-mon[31211]: from='client.? 172.21.15.39:0/1846763711' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-09-21T03:36:56.083 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:36:55 smithi039 ceph-mon[31211]: osdmap e4: 0 total, 0 up, 0 in 2024-09-21T03:36:56.084 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:36:55 smithi039 ceph-mon[31211]: from='mgr.14152 172.21.15.39:0/3595829223' entity='mgr.x' 2024-09-21T03:36:56.573 INFO:teuthology.orchestra.run.smithi161.stdout:Scheduled mon update... 2024-09-21T03:36:57.084 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:36:56 smithi039 ceph-mon[31211]: from='mgr.14152 172.21.15.39:0/3595829223' entity='mgr.x' 2024-09-21T03:36:57.084 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:36:56 smithi039 ceph-mon[31211]: from='mgr.14152 172.21.15.39:0/3595829223' entity='mgr.x' 2024-09-21T03:36:57.084 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:36:56 smithi039 ceph-mon[31211]: from='mgr.14152 172.21.15.39:0/3595829223' entity='mgr.x' 2024-09-21T03:36:57.084 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:36:56 smithi039 ceph-mon[31211]: from='mgr.14152 172.21.15.39:0/3595829223' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd/host:smithi161", "name": "osd_memory_target"} : dispatch 2024-09-21T03:36:57.084 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:36:56 smithi039 ceph-mon[31211]: from='mgr.14152 172.21.15.39:0/3595829223' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:36:57.084 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:36:56 smithi039 ceph-mon[31211]: from='mgr.14152 172.21.15.39:0/3595829223' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:36:57.084 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:36:56 smithi039 ceph-mon[31211]: Updating smithi161:/etc/ceph/ceph.conf 2024-09-21T03:36:57.085 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:36:56 smithi039 ceph-mon[31211]: from='mgr.14152 172.21.15.39:0/3595829223' entity='mgr.x' 2024-09-21T03:36:57.085 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:36:56 smithi039 ceph-mon[31211]: from='mgr.14152 172.21.15.39:0/3595829223' entity='mgr.x' 2024-09-21T03:36:57.085 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:36:56 smithi039 ceph-mon[31211]: from='mgr.14152 172.21.15.39:0/3595829223' entity='mgr.x' 2024-09-21T03:36:57.085 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:36:56 smithi039 ceph-mon[31211]: from='mgr.14152 172.21.15.39:0/3595829223' entity='mgr.x' 2024-09-21T03:36:57.085 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:36:56 smithi039 ceph-mon[31211]: from='mgr.14152 172.21.15.39:0/3595829223' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:36:57.085 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:36:56 smithi039 ceph-mon[31211]: from='mgr.14152 172.21.15.39:0/3595829223' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:36:57.085 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:36:56 smithi039 ceph-mon[31211]: from='mgr.14152 172.21.15.39:0/3595829223' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:36:57.085 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:36:56 smithi039 ceph-mon[31211]: from='mgr.14152 172.21.15.39:0/3595829223' entity='mgr.x' 2024-09-21T03:36:57.085 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:36:56 smithi039 ceph-mon[31211]: from='mgr.14152 172.21.15.39:0/3595829223' entity='mgr.x' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-09-21T03:36:57.085 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:36:56 smithi039 ceph-mon[31211]: from='mgr.14152 172.21.15.39:0/3595829223' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:36:57.353 DEBUG:teuthology.orchestra.run.smithi150:mon.b> sudo journalctl -f -n 0 -u ceph-51116d5c-77ca-11ef-baf5-efdc52797490@mon.b.service 2024-09-21T03:36:57.356 DEBUG:teuthology.orchestra.run.smithi161:mon.c> sudo journalctl -f -n 0 -u ceph-51116d5c-77ca-11ef-baf5-efdc52797490@mon.c.service 2024-09-21T03:36:57.358 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-09-21T03:36:57.358 DEBUG:teuthology.orchestra.run.smithi161:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 51116d5c-77ca-11ef-baf5-efdc52797490 -- ceph mon dump -f json 2024-09-21T03:36:58.047 INFO:teuthology.orchestra.run.smithi161.stderr:Inferring config /var/lib/ceph/51116d5c-77ca-11ef-baf5-efdc52797490/config/ceph.conf 2024-09-21T03:36:58.333 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:36:57 smithi039 ceph-mon[31211]: Updating smithi161:/var/lib/ceph/51116d5c-77ca-11ef-baf5-efdc52797490/config/ceph.conf 2024-09-21T03:36:58.334 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:36:57 smithi039 ceph-mon[31211]: Updating smithi161:/etc/ceph/ceph.client.admin.keyring 2024-09-21T03:36:58.334 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:36:57 smithi039 ceph-mon[31211]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:36:58.334 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:36:57 smithi039 ceph-mon[31211]: Updating smithi161:/var/lib/ceph/51116d5c-77ca-11ef-baf5-efdc52797490/config/ceph.client.admin.keyring 2024-09-21T03:36:58.334 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:36:57 smithi039 ceph-mon[31211]: from='client.14190 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi039:172.21.15.39=a;smithi150:172.21.15.150=b;smithi161:172.21.15.161=c", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:36:58.334 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:36:57 smithi039 ceph-mon[31211]: Saving service mon spec with placement smithi039:172.21.15.39=a;smithi150:172.21.15.150=b;smithi161:172.21.15.161=c;count:3 2024-09-21T03:36:58.334 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:36:57 smithi039 ceph-mon[31211]: Deploying daemon mon.c on smithi161 2024-09-21T03:37:00.090 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-09-21T03:37:00.091 INFO:teuthology.orchestra.run.smithi161.stdout:{"epoch":1,"fsid":"51116d5c-77ca-11ef-baf5-efdc52797490","modified":"2024-09-21T03:35:21.020316Z","created":"2024-09-21T03:35:21.020316Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:3300","nonce":0},{"type":"v1","addr":"172.21.15.39:6789","nonce":0}]},"addr":"172.21.15.39:6789/0","public_addr":"172.21.15.39:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-09-21T03:37:00.091 INFO:teuthology.orchestra.run.smithi161.stderr:dumped monmap epoch 1 2024-09-21T03:37:00.334 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:36:59 smithi039 ceph-mon[31211]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:37:01.334 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:37:00 smithi039 ceph-mon[31211]: from='client.? 172.21.15.161:0/261893970' entity='client.admin' cmd={"prefix": "mon dump", "format": "json"} : dispatch 2024-09-21T03:37:01.467 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:37:01 smithi161 ceph-mon[34533]: mon.c@-1(synchronizing).paxosservice(auth 1..3) refresh upgraded, format 0 -> 3 2024-09-21T03:37:01.838 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-09-21T03:37:01.838 DEBUG:teuthology.orchestra.run.smithi161:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 51116d5c-77ca-11ef-baf5-efdc52797490 -- ceph mon dump -f json 2024-09-21T03:37:02.087 INFO:teuthology.orchestra.run.smithi161.stderr:Inferring config /var/lib/ceph/51116d5c-77ca-11ef-baf5-efdc52797490/mon.c/config 2024-09-21T03:37:06.718 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:37:06 smithi161 ceph-mon[34533]: from='mgr.14152 172.21.15.39:0/3595829223' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-21T03:37:06.719 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:37:06 smithi161 ceph-mon[34533]: from='mgr.14152 172.21.15.39:0/3595829223' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T03:37:06.719 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:37:06 smithi161 ceph-mon[34533]: mon.a calling monitor election 2024-09-21T03:37:06.719 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:37:06 smithi161 ceph-mon[34533]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:37:06.719 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:37:06 smithi161 ceph-mon[34533]: from='mgr.14152 172.21.15.39:0/3595829223' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T03:37:06.719 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:37:06 smithi161 ceph-mon[34533]: from='mgr.14152 172.21.15.39:0/3595829223' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T03:37:06.719 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:37:06 smithi161 ceph-mon[34533]: mon.c calling monitor election 2024-09-21T03:37:06.719 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:37:06 smithi161 ceph-mon[34533]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:37:06.719 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:37:06 smithi161 ceph-mon[34533]: from='mgr.14152 172.21.15.39:0/3595829223' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T03:37:06.719 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:37:06 smithi161 ceph-mon[34533]: from='mgr.14152 172.21.15.39:0/3595829223' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T03:37:06.719 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:37:06 smithi161 ceph-mon[34533]: from='mgr.14152 172.21.15.39:0/3595829223' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T03:37:06.719 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:37:06 smithi161 ceph-mon[34533]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-09-21T03:37:06.719 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:37:06 smithi161 ceph-mon[34533]: monmap epoch 2 2024-09-21T03:37:06.719 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:37:06 smithi161 ceph-mon[34533]: fsid 51116d5c-77ca-11ef-baf5-efdc52797490 2024-09-21T03:37:06.720 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:37:06 smithi161 ceph-mon[34533]: last_changed 2024-09-21T03:37:01.289853+0000 2024-09-21T03:37:06.720 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:37:06 smithi161 ceph-mon[34533]: created 2024-09-21T03:35:21.020316+0000 2024-09-21T03:37:06.720 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:37:06 smithi161 ceph-mon[34533]: min_mon_release 19 (squid) 2024-09-21T03:37:06.720 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:37:06 smithi161 ceph-mon[34533]: election_strategy: 1 2024-09-21T03:37:06.720 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:37:06 smithi161 ceph-mon[34533]: 0: [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] mon.a 2024-09-21T03:37:06.720 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:37:06 smithi161 ceph-mon[34533]: 1: [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] mon.c 2024-09-21T03:37:06.720 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:37:06 smithi161 ceph-mon[34533]: fsmap 2024-09-21T03:37:06.720 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:37:06 smithi161 ceph-mon[34533]: osdmap e4: 0 total, 0 up, 0 in 2024-09-21T03:37:06.720 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:37:06 smithi161 ceph-mon[34533]: mgrmap e14: x(active, since 42s) 2024-09-21T03:37:06.720 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:37:06 smithi161 ceph-mon[34533]: overall HEALTH_OK 2024-09-21T03:37:06.720 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:37:06 smithi161 ceph-mon[34533]: from='mgr.14152 172.21.15.39:0/3595829223' entity='mgr.x' 2024-09-21T03:37:06.720 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:37:06 smithi161 ceph-mon[34533]: from='mgr.14152 172.21.15.39:0/3595829223' entity='mgr.x' 2024-09-21T03:37:06.720 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:37:06 smithi161 ceph-mon[34533]: from='mgr.14152 172.21.15.39:0/3595829223' entity='mgr.x' 2024-09-21T03:37:06.720 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:37:06 smithi161 ceph-mon[34533]: from='mgr.14152 172.21.15.39:0/3595829223' entity='mgr.x' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-09-21T03:37:06.720 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:37:06 smithi161 ceph-mon[34533]: from='mgr.14152 172.21.15.39:0/3595829223' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:37:06.834 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:37:06 smithi039 ceph-mon[31211]: from='mgr.14152 172.21.15.39:0/3595829223' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-21T03:37:06.834 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:37:06 smithi039 ceph-mon[31211]: from='mgr.14152 172.21.15.39:0/3595829223' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T03:37:06.834 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:37:06 smithi039 ceph-mon[31211]: mon.a calling monitor election 2024-09-21T03:37:06.834 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:37:06 smithi039 ceph-mon[31211]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:37:06.834 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:37:06 smithi039 ceph-mon[31211]: from='mgr.14152 172.21.15.39:0/3595829223' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T03:37:06.834 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:37:06 smithi039 ceph-mon[31211]: from='mgr.14152 172.21.15.39:0/3595829223' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T03:37:06.834 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:37:06 smithi039 ceph-mon[31211]: mon.c calling monitor election 2024-09-21T03:37:06.834 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:37:06 smithi039 ceph-mon[31211]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:37:06.834 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:37:06 smithi039 ceph-mon[31211]: from='mgr.14152 172.21.15.39:0/3595829223' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T03:37:06.834 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:37:06 smithi039 ceph-mon[31211]: from='mgr.14152 172.21.15.39:0/3595829223' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T03:37:06.834 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:37:06 smithi039 ceph-mon[31211]: from='mgr.14152 172.21.15.39:0/3595829223' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T03:37:06.834 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:37:06 smithi039 ceph-mon[31211]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-09-21T03:37:06.835 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:37:06 smithi039 ceph-mon[31211]: monmap epoch 2 2024-09-21T03:37:06.835 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:37:06 smithi039 ceph-mon[31211]: fsid 51116d5c-77ca-11ef-baf5-efdc52797490 2024-09-21T03:37:06.835 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:37:06 smithi039 ceph-mon[31211]: last_changed 2024-09-21T03:37:01.289853+0000 2024-09-21T03:37:06.835 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:37:06 smithi039 ceph-mon[31211]: created 2024-09-21T03:35:21.020316+0000 2024-09-21T03:37:06.835 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:37:06 smithi039 ceph-mon[31211]: min_mon_release 19 (squid) 2024-09-21T03:37:06.835 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:37:06 smithi039 ceph-mon[31211]: election_strategy: 1 2024-09-21T03:37:06.835 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:37:06 smithi039 ceph-mon[31211]: 0: [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] mon.a 2024-09-21T03:37:06.835 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:37:06 smithi039 ceph-mon[31211]: 1: [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] mon.c 2024-09-21T03:37:06.835 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:37:06 smithi039 ceph-mon[31211]: fsmap 2024-09-21T03:37:06.835 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:37:06 smithi039 ceph-mon[31211]: osdmap e4: 0 total, 0 up, 0 in 2024-09-21T03:37:06.835 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:37:06 smithi039 ceph-mon[31211]: mgrmap e14: x(active, since 42s) 2024-09-21T03:37:06.835 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:37:06 smithi039 ceph-mon[31211]: overall HEALTH_OK 2024-09-21T03:37:06.835 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:37:06 smithi039 ceph-mon[31211]: from='mgr.14152 172.21.15.39:0/3595829223' entity='mgr.x' 2024-09-21T03:37:06.835 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:37:06 smithi039 ceph-mon[31211]: from='mgr.14152 172.21.15.39:0/3595829223' entity='mgr.x' 2024-09-21T03:37:06.836 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:37:06 smithi039 ceph-mon[31211]: from='mgr.14152 172.21.15.39:0/3595829223' entity='mgr.x' 2024-09-21T03:37:06.836 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:37:06 smithi039 ceph-mon[31211]: from='mgr.14152 172.21.15.39:0/3595829223' entity='mgr.x' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-09-21T03:37:06.836 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:37:06 smithi039 ceph-mon[31211]: from='mgr.14152 172.21.15.39:0/3595829223' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:37:07.189 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-09-21T03:37:07.189 INFO:teuthology.orchestra.run.smithi161.stdout:{"epoch":2,"fsid":"51116d5c-77ca-11ef-baf5-efdc52797490","modified":"2024-09-21T03:37:01.289853Z","created":"2024-09-21T03:35:21.020316Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:3300","nonce":0},{"type":"v1","addr":"172.21.15.39:6789","nonce":0}]},"addr":"172.21.15.39:6789/0","public_addr":"172.21.15.39:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:3300","nonce":0},{"type":"v1","addr":"172.21.15.161:6789","nonce":0}]},"addr":"172.21.15.161:6789/0","public_addr":"172.21.15.161:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-09-21T03:37:07.189 INFO:teuthology.orchestra.run.smithi161.stderr:dumped monmap epoch 2 2024-09-21T03:37:07.467 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:37:07 smithi161 ceph-mon[34533]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:37:07.468 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:37:07 smithi161 ceph-mon[34533]: Deploying daemon mon.b on smithi150 2024-09-21T03:37:07.468 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:37:07 smithi161 ceph-mon[34533]: from='client.? 172.21.15.161:0/1614888602' entity='client.admin' cmd={"prefix": "mon dump", "format": "json"} : dispatch 2024-09-21T03:37:07.468 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:37:07 smithi161 ceph-mon[34533]: from='mgr.14152 172.21.15.39:0/3595829223' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T03:37:07.833 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:37:07 smithi039 ceph-mon[31211]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:37:07.833 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:37:07 smithi039 ceph-mon[31211]: Deploying daemon mon.b on smithi150 2024-09-21T03:37:07.833 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:37:07 smithi039 ceph-mon[31211]: from='client.? 172.21.15.161:0/1614888602' entity='client.admin' cmd={"prefix": "mon dump", "format": "json"} : dispatch 2024-09-21T03:37:07.834 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:37:07 smithi039 ceph-mon[31211]: from='mgr.14152 172.21.15.39:0/3595829223' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T03:37:08.831 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-09-21T03:37:08.831 DEBUG:teuthology.orchestra.run.smithi161:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 51116d5c-77ca-11ef-baf5-efdc52797490 -- ceph mon dump -f json 2024-09-21T03:37:09.085 INFO:teuthology.orchestra.run.smithi161.stderr:Inferring config /var/lib/ceph/51116d5c-77ca-11ef-baf5-efdc52797490/mon.c/config 2024-09-21T03:37:09.467 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:37:09 smithi161 ceph-mon[34533]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:37:09.637 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:37:09 smithi039 ceph-mon[31211]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:37:09.763 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-09-21T03:37:09.764 INFO:teuthology.orchestra.run.smithi161.stdout:{"epoch":2,"fsid":"51116d5c-77ca-11ef-baf5-efdc52797490","modified":"2024-09-21T03:37:01.289853Z","created":"2024-09-21T03:35:21.020316Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:3300","nonce":0},{"type":"v1","addr":"172.21.15.39:6789","nonce":0}]},"addr":"172.21.15.39:6789/0","public_addr":"172.21.15.39:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:3300","nonce":0},{"type":"v1","addr":"172.21.15.161:6789","nonce":0}]},"addr":"172.21.15.161:6789/0","public_addr":"172.21.15.161:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-09-21T03:37:09.764 INFO:teuthology.orchestra.run.smithi161.stderr:dumped monmap epoch 2 2024-09-21T03:37:10.108 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:37:10 smithi150 ceph-mon[34289]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-09-21T03:37:10.108 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:37:10 smithi150 ceph-mon[34289]: rocksdb: Options.persist_stats_to_disk: 0 2024-09-21T03:37:10.109 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:37:10 smithi150 ceph-mon[34289]: rocksdb: Options.write_dbid_to_manifest: 0 2024-09-21T03:37:10.109 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:37:10 smithi150 ceph-mon[34289]: rocksdb: Options.log_readahead_size: 0 2024-09-21T03:37:10.109 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:37:10 smithi150 ceph-mon[34289]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-09-21T03:37:10.109 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:37:10 smithi150 ceph-mon[34289]: rocksdb: Options.best_efforts_recovery: 0 2024-09-21T03:37:10.109 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:37:10 smithi150 ceph-mon[34289]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-09-21T03:37:10.109 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:37:10 smithi150 ceph-mon[34289]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-09-21T03:37:10.109 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:37:10 smithi150 ceph-mon[34289]: rocksdb: Options.allow_data_in_errors: 0 2024-09-21T03:37:10.109 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:37:10 smithi150 ceph-mon[34289]: rocksdb: Options.db_host_id: __hostname__ 2024-09-21T03:37:10.109 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:37:10 smithi150 ceph-mon[34289]: rocksdb: Options.enforce_single_del_contracts: true 2024-09-21T03:37:10.109 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:37:10 smithi150 ceph-mon[34289]: rocksdb: Options.max_background_jobs: 2 2024-09-21T03:37:10.475 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:37:10 smithi150 ceph-mon[34289]: rocksdb: Options.max_background_compactions: -1 2024-09-21T03:37:10.475 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:37:10 smithi150 ceph-mon[34289]: rocksdb: Options.max_subcompactions: 1 2024-09-21T03:37:10.475 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:37:10 smithi150 ceph-mon[34289]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-09-21T03:37:10.475 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:37:10 smithi150 ceph-mon[34289]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-09-21T03:37:10.475 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:37:10 smithi150 ceph-mon[34289]: rocksdb: Options.delayed_write_rate : 16777216 2024-09-21T03:37:10.476 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:37:10 smithi150 ceph-mon[34289]: rocksdb: Options.max_total_wal_size: 0 2024-09-21T03:37:10.476 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:37:10 smithi150 ceph-mon[34289]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-09-21T03:37:10.476 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:37:10 smithi150 ceph-mon[34289]: rocksdb: Options.stats_dump_period_sec: 600 2024-09-21T03:37:10.476 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:37:10 smithi150 ceph-mon[34289]: rocksdb: Options.stats_persist_period_sec: 600 2024-09-21T03:37:10.476 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:37:10 smithi150 ceph-mon[34289]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-09-21T03:37:10.476 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:37:10 smithi150 ceph-mon[34289]: rocksdb: Options.max_open_files: -1 2024-09-21T03:37:10.476 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:37:10 smithi150 ceph-mon[34289]: rocksdb: Options.bytes_per_sync: 0 2024-09-21T03:37:10.476 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:37:10 smithi150 ceph-mon[34289]: rocksdb: Options.wal_bytes_per_sync: 0 2024-09-21T03:37:10.476 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:37:10 smithi150 ceph-mon[34289]: rocksdb: Options.strict_bytes_per_sync: 0 2024-09-21T03:37:10.476 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:37:10 smithi150 ceph-mon[34289]: rocksdb: Options.compaction_readahead_size: 0 2024-09-21T03:37:10.476 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:37:10 smithi150 ceph-mon[34289]: rocksdb: Options.max_background_flushes: -1 2024-09-21T03:37:10.476 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:37:10 smithi150 ceph-mon[34289]: rocksdb: Compression algorithms supported: 2024-09-21T03:37:10.477 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:37:10 smithi150 ceph-mon[34289]: rocksdb: kZSTD supported: 0 2024-09-21T03:37:10.477 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:37:10 smithi150 ceph-mon[34289]: rocksdb: kXpressCompression supported: 0 2024-09-21T03:37:10.477 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:37:10 smithi150 ceph-mon[34289]: rocksdb: kBZip2Compression supported: 0 2024-09-21T03:37:10.477 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:37:10 smithi150 ceph-mon[34289]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-09-21T03:37:10.477 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:37:10 smithi150 ceph-mon[34289]: rocksdb: kLZ4Compression supported: 1 2024-09-21T03:37:10.477 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:37:10 smithi150 ceph-mon[34289]: rocksdb: kZlibCompression supported: 1 2024-09-21T03:37:10.477 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:37:10 smithi150 ceph-mon[34289]: rocksdb: kLZ4HCCompression supported: 1 2024-09-21T03:37:10.477 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:37:10 smithi150 ceph-mon[34289]: rocksdb: kSnappyCompression supported: 1 2024-09-21T03:37:10.477 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:37:10 smithi150 ceph-mon[34289]: rocksdb: Fast CRC32 supported: Supported on x86 2024-09-21T03:37:10.477 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:37:10 smithi150 ceph-mon[34289]: rocksdb: DMutex implementation: pthread_mutex_t 2024-09-21T03:37:10.477 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:37:10 smithi150 ceph-mon[34289]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-b/store.db/MANIFEST-000005 2024-09-21T03:37:10.478 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:37:10 smithi150 ceph-mon[34289]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-09-21T03:37:10.478 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:37:10 smithi150 ceph-mon[34289]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-09-21T03:37:10.478 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:37:10 smithi150 ceph-mon[34289]: rocksdb: Options.merge_operator: 2024-09-21T03:37:10.478 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:37:10 smithi150 ceph-mon[34289]: rocksdb: Options.compaction_filter: None 2024-09-21T03:37:10.478 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:37:10 smithi150 ceph-mon[34289]: rocksdb: Options.compaction_filter_factory: None 2024-09-21T03:37:10.478 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:37:10 smithi150 ceph-mon[34289]: rocksdb: Options.sst_partitioner_factory: None 2024-09-21T03:37:10.478 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:37:10 smithi150 ceph-mon[34289]: rocksdb: Options.memtable_factory: SkipListFactory 2024-09-21T03:37:10.478 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:37:10 smithi150 ceph-mon[34289]: rocksdb: Options.table_factory: BlockBasedTable 2024-09-21T03:37:10.478 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:37:10 smithi150 ceph-mon[34289]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x559a98c12fa0) 2024-09-21T03:37:10.478 INFO:journalctl@ceph.mon.b.smithi150.stdout: cache_index_and_filter_blocks: 1 2024-09-21T03:37:10.478 INFO:journalctl@ceph.mon.b.smithi150.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-09-21T03:37:10.478 INFO:journalctl@ceph.mon.b.smithi150.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-09-21T03:37:10.479 INFO:journalctl@ceph.mon.b.smithi150.stdout: pin_top_level_index_and_filter: 1 2024-09-21T03:37:10.479 INFO:journalctl@ceph.mon.b.smithi150.stdout: index_type: 0 2024-09-21T03:37:10.479 INFO:journalctl@ceph.mon.b.smithi150.stdout: data_block_index_type: 0 2024-09-21T03:37:10.479 INFO:journalctl@ceph.mon.b.smithi150.stdout: index_shortening: 1 2024-09-21T03:37:10.479 INFO:journalctl@ceph.mon.b.smithi150.stdout: data_block_hash_table_util_ratio: 0.750000 2024-09-21T03:37:10.479 INFO:journalctl@ceph.mon.b.smithi150.stdout: checksum: 4 2024-09-21T03:37:10.479 INFO:journalctl@ceph.mon.b.smithi150.stdout: no_block_cache: 0 2024-09-21T03:37:10.479 INFO:journalctl@ceph.mon.b.smithi150.stdout: block_cache: 0x559a94bd7610 2024-09-21T03:37:10.479 INFO:journalctl@ceph.mon.b.smithi150.stdout: block_cache_name: BinnedLRUCache 2024-09-21T03:37:10.479 INFO:journalctl@ceph.mon.b.smithi150.stdout: block_cache_options: 2024-09-21T03:37:10.479 INFO:journalctl@ceph.mon.b.smithi150.stdout: capacity : 536870912 2024-09-21T03:37:10.479 INFO:journalctl@ceph.mon.b.smithi150.stdout: num_shard_bits : 4 2024-09-21T03:37:10.480 INFO:journalctl@ceph.mon.b.smithi150.stdout: strict_capacity_limit : 0 2024-09-21T03:37:10.480 INFO:journalctl@ceph.mon.b.smithi150.stdout: high_pri_pool_ratio: 0.000 2024-09-21T03:37:10.480 INFO:journalctl@ceph.mon.b.smithi150.stdout: block_cache_compressed: (nil) 2024-09-21T03:37:10.480 INFO:journalctl@ceph.mon.b.smithi150.stdout: persistent_cache: (nil) 2024-09-21T03:37:10.480 INFO:journalctl@ceph.mon.b.smithi150.stdout: block_size: 4096 2024-09-21T03:37:10.480 INFO:journalctl@ceph.mon.b.smithi150.stdout: block_size_deviation: 10 2024-09-21T03:37:10.480 INFO:journalctl@ceph.mon.b.smithi150.stdout: block_restart_interval: 16 2024-09-21T03:37:10.480 INFO:journalctl@ceph.mon.b.smithi150.stdout: index_block_restart_interval: 1 2024-09-21T03:37:10.480 INFO:journalctl@ceph.mon.b.smithi150.stdout: metadata_block_size: 4096 2024-09-21T03:37:10.480 INFO:journalctl@ceph.mon.b.smithi150.stdout: partition_filters: 0 2024-09-21T03:37:10.480 INFO:journalctl@ceph.mon.b.smithi150.stdout: use_delta_encoding: 1 2024-09-21T03:37:10.480 INFO:journalctl@ceph.mon.b.smithi150.stdout: filter_policy: bloomfilter 2024-09-21T03:37:10.480 INFO:journalctl@ceph.mon.b.smithi150.stdout: whole_key_filtering: 1 2024-09-21T03:37:10.481 INFO:journalctl@ceph.mon.b.smithi150.stdout: verify_compression: 0 2024-09-21T03:37:10.481 INFO:journalctl@ceph.mon.b.smithi150.stdout: read_amp_bytes_per_bit: 0 2024-09-21T03:37:10.481 INFO:journalctl@ceph.mon.b.smithi150.stdout: format_version: 5 2024-09-21T03:37:10.481 INFO:journalctl@ceph.mon.b.smithi150.stdout: enable_index_compression: 1 2024-09-21T03:37:10.481 INFO:journalctl@ceph.mon.b.smithi150.stdout: block_align: 0 2024-09-21T03:37:10.481 INFO:journalctl@ceph.mon.b.smithi150.stdout: max_auto_readahead_size: 262144 2024-09-21T03:37:10.481 INFO:journalctl@ceph.mon.b.smithi150.stdout: prepopulate_block_cache: 0 2024-09-21T03:37:10.481 INFO:journalctl@ceph.mon.b.smithi150.stdout: initial_auto_readahead_size: 8192 2024-09-21T03:37:10.481 INFO:journalctl@ceph.mon.b.smithi150.stdout: num_file_reads_for_auto_readahead: 2 2024-09-21T03:37:10.481 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:37:10 smithi150 ceph-mon[34289]: rocksdb: Options.write_buffer_size: 33554432 2024-09-21T03:37:10.481 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:37:10 smithi150 ceph-mon[34289]: rocksdb: Options.max_write_buffer_number: 2 2024-09-21T03:37:10.482 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:37:10 smithi150 ceph-mon[34289]: rocksdb: Options.compression: NoCompression 2024-09-21T03:37:10.482 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:37:10 smithi150 ceph-mon[34289]: rocksdb: Options.bottommost_compression: Disabled 2024-09-21T03:37:10.482 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:37:10 smithi150 ceph-mon[34289]: rocksdb: Options.prefix_extractor: nullptr 2024-09-21T03:37:10.482 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:37:10 smithi150 ceph-mon[34289]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-09-21T03:37:10.482 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:37:10 smithi150 ceph-mon[34289]: rocksdb: Options.num_levels: 7 2024-09-21T03:37:10.482 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:37:10 smithi150 ceph-mon[34289]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-09-21T03:37:10.482 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:37:10 smithi150 ceph-mon[34289]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-09-21T03:37:10.482 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:37:10 smithi150 ceph-mon[34289]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-09-21T03:37:10.482 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:37:10 smithi150 ceph-mon[34289]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-09-21T03:37:10.482 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:37:10 smithi150 ceph-mon[34289]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-09-21T03:37:10.482 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:37:10 smithi150 ceph-mon[34289]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-09-21T03:37:10.482 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:37:10 smithi150 ceph-mon[34289]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-09-21T03:37:10.483 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:37:10 smithi150 ceph-mon[34289]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-09-21T03:37:10.483 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:37:10 smithi150 ceph-mon[34289]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-09-21T03:37:10.484 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:37:10 smithi150 ceph-mon[34289]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-09-21T03:37:10.484 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:37:10 smithi150 ceph-mon[34289]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-09-21T03:37:10.484 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:37:10 smithi150 ceph-mon[34289]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-09-21T03:37:10.484 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:37:10 smithi150 ceph-mon[34289]: rocksdb: Options.compression_opts.window_bits: -14 2024-09-21T03:37:10.484 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:37:10 smithi150 ceph-mon[34289]: rocksdb: Options.compression_opts.level: 32767 2024-09-21T03:37:10.484 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:37:10 smithi150 ceph-mon[34289]: rocksdb: Options.compression_opts.strategy: 0 2024-09-21T03:37:10.484 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:37:10 smithi150 ceph-mon[34289]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-09-21T03:37:10.484 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:37:10 smithi150 ceph-mon[34289]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-09-21T03:37:10.485 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:37:10 smithi150 ceph-mon[34289]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-09-21T03:37:10.485 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:37:10 smithi150 ceph-mon[34289]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-09-21T03:37:10.485 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:37:10 smithi150 ceph-mon[34289]: rocksdb: Options.compression_opts.enabled: false 2024-09-21T03:37:10.485 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:37:10 smithi150 ceph-mon[34289]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-09-21T03:37:10.485 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:37:10 smithi150 ceph-mon[34289]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-09-21T03:37:10.485 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:37:10 smithi150 ceph-mon[34289]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-09-21T03:37:10.485 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:37:10 smithi150 ceph-mon[34289]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-09-21T03:37:10.485 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:37:10 smithi150 ceph-mon[34289]: rocksdb: Options.target_file_size_base: 67108864 2024-09-21T03:37:10.485 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:37:10 smithi150 ceph-mon[34289]: rocksdb: Options.target_file_size_multiplier: 1 2024-09-21T03:37:10.485 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:37:10 smithi150 ceph-mon[34289]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-09-21T03:37:10.485 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:37:10 smithi150 ceph-mon[34289]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-09-21T03:37:10.485 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:37:10 smithi150 ceph-mon[34289]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-09-21T03:37:10.486 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:37:10 smithi150 ceph-mon[34289]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-09-21T03:37:10.486 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:37:10 smithi150 ceph-mon[34289]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-09-21T03:37:10.486 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:37:10 smithi150 ceph-mon[34289]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-09-21T03:37:10.486 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:37:10 smithi150 ceph-mon[34289]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-09-21T03:37:10.486 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:37:10 smithi150 ceph-mon[34289]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-09-21T03:37:10.486 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:37:10 smithi150 ceph-mon[34289]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-09-21T03:37:10.486 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:37:10 smithi150 ceph-mon[34289]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-09-21T03:37:10.486 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:37:10 smithi150 ceph-mon[34289]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-09-21T03:37:10.486 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:37:10 smithi150 ceph-mon[34289]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-09-21T03:37:10.486 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:37:10 smithi150 ceph-mon[34289]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-09-21T03:37:10.486 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:37:10 smithi150 ceph-mon[34289]: rocksdb: Options.arena_block_size: 1048576 2024-09-21T03:37:10.487 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:37:10 smithi150 ceph-mon[34289]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-09-21T03:37:10.487 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:37:10 smithi150 ceph-mon[34289]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-09-21T03:37:10.487 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:37:10 smithi150 ceph-mon[34289]: rocksdb: Options.disable_auto_compactions: 0 2024-09-21T03:37:10.487 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:37:10 smithi150 ceph-mon[34289]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-09-21T03:37:10.487 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:37:10 smithi150 ceph-mon[34289]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-09-21T03:37:10.487 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:37:10 smithi150 ceph-mon[34289]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-09-21T03:37:10.487 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:37:10 smithi150 ceph-mon[34289]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-09-21T03:37:10.487 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:37:10 smithi150 ceph-mon[34289]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-09-21T03:37:10.487 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:37:10 smithi150 ceph-mon[34289]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-09-21T03:37:10.487 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:37:10 smithi150 ceph-mon[34289]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-09-21T03:37:10.487 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:37:10 smithi150 ceph-mon[34289]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-09-21T03:37:10.487 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:37:10 smithi150 ceph-mon[34289]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-09-21T03:37:10.488 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:37:10 smithi150 ceph-mon[34289]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-09-21T03:37:10.488 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:37:10 smithi150 ceph-mon[34289]: rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2024-09-21T03:37:10.488 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:37:10 smithi150 ceph-mon[34289]: rocksdb: Options.inplace_update_support: 0 2024-09-21T03:37:10.488 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:37:10 smithi150 ceph-mon[34289]: rocksdb: Options.inplace_update_num_locks: 10000 2024-09-21T03:37:10.488 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:37:10 smithi150 ceph-mon[34289]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-09-21T03:37:10.488 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:37:10 smithi150 ceph-mon[34289]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-09-21T03:37:10.488 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:37:10 smithi150 ceph-mon[34289]: rocksdb: Options.memtable_huge_page_size: 0 2024-09-21T03:37:10.488 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:37:10 smithi150 ceph-mon[34289]: rocksdb: Options.bloom_locality: 0 2024-09-21T03:37:10.488 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:37:10 smithi150 ceph-mon[34289]: rocksdb: Options.max_successive_merges: 0 2024-09-21T03:37:10.488 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:37:10 smithi150 ceph-mon[34289]: rocksdb: Options.optimize_filters_for_hits: 0 2024-09-21T03:37:10.488 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:37:10 smithi150 ceph-mon[34289]: rocksdb: Options.paranoid_file_checks: 0 2024-09-21T03:37:10.488 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:37:10 smithi150 ceph-mon[34289]: rocksdb: Options.force_consistency_checks: 1 2024-09-21T03:37:10.489 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:37:10 smithi150 ceph-mon[34289]: rocksdb: Options.report_bg_io_stats: 0 2024-09-21T03:37:10.489 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:37:10 smithi150 ceph-mon[34289]: rocksdb: Options.ttl: 2592000 2024-09-21T03:37:10.489 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:37:10 smithi150 ceph-mon[34289]: rocksdb: Options.periodic_compaction_seconds: 0 2024-09-21T03:37:10.489 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:37:10 smithi150 ceph-mon[34289]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-09-21T03:37:10.489 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:37:10 smithi150 ceph-mon[34289]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-09-21T03:37:10.489 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:37:10 smithi150 ceph-mon[34289]: rocksdb: Options.enable_blob_files: false 2024-09-21T03:37:10.489 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:37:10 smithi150 ceph-mon[34289]: rocksdb: Options.min_blob_size: 0 2024-09-21T03:37:10.489 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:37:10 smithi150 ceph-mon[34289]: rocksdb: Options.blob_file_size: 268435456 2024-09-21T03:37:10.489 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:37:10 smithi150 ceph-mon[34289]: rocksdb: Options.blob_compression_type: NoCompression 2024-09-21T03:37:10.489 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:37:10 smithi150 ceph-mon[34289]: rocksdb: Options.enable_blob_garbage_collection: false 2024-09-21T03:37:10.489 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:37:10 smithi150 ceph-mon[34289]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-09-21T03:37:10.489 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:37:10 smithi150 ceph-mon[34289]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-09-21T03:37:10.490 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:37:10 smithi150 ceph-mon[34289]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-09-21T03:37:10.490 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:37:10 smithi150 ceph-mon[34289]: rocksdb: Options.blob_file_starting_level: 0 2024-09-21T03:37:10.490 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:37:10 smithi150 ceph-mon[34289]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-09-21T03:37:10.490 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:37:10 smithi150 ceph-mon[34289]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-b/store.db/MANIFEST-000005 succeeded,manifest_file_number is 5, next_file_number is 7, last_sequence is 0, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2024-09-21T03:37:10.490 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:37:10 smithi150 ceph-mon[34289]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 0 2024-09-21T03:37:10.490 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:37:10 smithi150 ceph-mon[34289]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 5234a69c-650e-4d29-bc61-a776b3f45d01 2024-09-21T03:37:10.490 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:37:10 smithi150 ceph-mon[34289]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726889830106844, "job": 1, "event": "recovery_started", "wal_files": [4]} 2024-09-21T03:37:10.490 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:37:10 smithi150 ceph-mon[34289]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #4 mode 2 2024-09-21T03:37:10.490 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:37:10 smithi150 ceph-mon[34289]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726889830107303, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 8, "file_size": 1773, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 1, "largest_seqno": 5, "table_properties": {"data_size": 648, "index_size": 31, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 69, "raw_key_size": 115, "raw_average_key_size": 23, "raw_value_size": 526, "raw_average_value_size": 105, "num_data_blocks": 1, "num_entries": 5, "num_filter_entries": 5, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[CompactOnDeletionCollector]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1726889830, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "5234a69c-650e-4d29-bc61-a776b3f45d01", "db_session_id": "D453BJP0JD1XXMITUUS0", "orig_file_number": 8, "seqno_to_time_mapping": "N/A"}} 2024-09-21T03:37:10.490 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:37:10 smithi150 ceph-mon[34289]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726889830107415, "job": 1, "event": "recovery_finished"} 2024-09-21T03:37:10.490 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:37:10 smithi150 ceph-mon[34289]: rocksdb: [db/version_set.cc:5047] Creating manifest 10 2024-09-21T03:37:10.491 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:37:10 smithi150 ceph-mon[34289]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-b/store.db/000004.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-09-21T03:37:10.491 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:37:10 smithi150 ceph-mon[34289]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x559a98c36a80 2024-09-21T03:37:10.491 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:37:10 smithi150 ceph-mon[34289]: rocksdb: DB pointer 0x559a999dc000 2024-09-21T03:37:10.491 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:37:10 smithi150 ceph-mon[34289]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-09-21T03:37:10.491 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:37:10 smithi150 ceph-mon[34289]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-09-21T03:37:10.491 INFO:journalctl@ceph.mon.b.smithi150.stdout: ** DB Stats ** 2024-09-21T03:37:10.491 INFO:journalctl@ceph.mon.b.smithi150.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-21T03:37:10.491 INFO:journalctl@ceph.mon.b.smithi150.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-09-21T03:37:10.491 INFO:journalctl@ceph.mon.b.smithi150.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-09-21T03:37:10.491 INFO:journalctl@ceph.mon.b.smithi150.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-09-21T03:37:10.491 INFO:journalctl@ceph.mon.b.smithi150.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-09-21T03:37:10.491 INFO:journalctl@ceph.mon.b.smithi150.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-09-21T03:37:10.492 INFO:journalctl@ceph.mon.b.smithi150.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-09-21T03:37:10.492 INFO:journalctl@ceph.mon.b.smithi150.stdout: 2024-09-21T03:37:10.492 INFO:journalctl@ceph.mon.b.smithi150.stdout: ** Compaction Stats [default] ** 2024-09-21T03:37:10.492 INFO:journalctl@ceph.mon.b.smithi150.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-09-21T03:37:10.492 INFO:journalctl@ceph.mon.b.smithi150.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-09-21T03:37:10.492 INFO:journalctl@ceph.mon.b.smithi150.stdout: L0 1/0 1.73 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.8 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-09-21T03:37:10.492 INFO:journalctl@ceph.mon.b.smithi150.stdout: Sum 1/0 1.73 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.8 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-09-21T03:37:10.492 INFO:journalctl@ceph.mon.b.smithi150.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.8 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-09-21T03:37:10.492 INFO:journalctl@ceph.mon.b.smithi150.stdout: 2024-09-21T03:37:10.492 INFO:journalctl@ceph.mon.b.smithi150.stdout: ** Compaction Stats [default] ** 2024-09-21T03:37:10.492 INFO:journalctl@ceph.mon.b.smithi150.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-09-21T03:37:10.492 INFO:journalctl@ceph.mon.b.smithi150.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-09-21T03:37:10.493 INFO:journalctl@ceph.mon.b.smithi150.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 3.8 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-09-21T03:37:10.493 INFO:journalctl@ceph.mon.b.smithi150.stdout: 2024-09-21T03:37:10.493 INFO:journalctl@ceph.mon.b.smithi150.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-09-21T03:37:10.493 INFO:journalctl@ceph.mon.b.smithi150.stdout: 2024-09-21T03:37:10.493 INFO:journalctl@ceph.mon.b.smithi150.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-21T03:37:10.493 INFO:journalctl@ceph.mon.b.smithi150.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-09-21T03:37:10.493 INFO:journalctl@ceph.mon.b.smithi150.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-09-21T03:37:10.493 INFO:journalctl@ceph.mon.b.smithi150.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-09-21T03:37:10.493 INFO:journalctl@ceph.mon.b.smithi150.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-09-21T03:37:10.493 INFO:journalctl@ceph.mon.b.smithi150.stdout: AddFile(Keys): cumulative 0, interval 0 2024-09-21T03:37:10.493 INFO:journalctl@ceph.mon.b.smithi150.stdout: Cumulative compaction: 0.00 GB write, 0.59 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-21T03:37:10.493 INFO:journalctl@ceph.mon.b.smithi150.stdout: Interval compaction: 0.00 GB write, 0.59 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-21T03:37:10.494 INFO:journalctl@ceph.mon.b.smithi150.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-09-21T03:37:10.494 INFO:journalctl@ceph.mon.b.smithi150.stdout: Block cache BinnedLRUCache@0x559a94bd7610#2 capacity: 512.00 MB usage: 0.23 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 9e-06 secs_since: 0 2024-09-21T03:37:10.494 INFO:journalctl@ceph.mon.b.smithi150.stdout: Block cache entry stats(count,size,portion): FilterBlock(1,0.12 KB,2.23517e-05%) IndexBlock(1,0.11 KB,2.08616e-05%) Misc(1,0.00 KB,0%) 2024-09-21T03:37:10.494 INFO:journalctl@ceph.mon.b.smithi150.stdout: 2024-09-21T03:37:10.494 INFO:journalctl@ceph.mon.b.smithi150.stdout: ** File Read Latency Histogram By Level [default] ** 2024-09-21T03:37:10.494 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:37:10 smithi150 ceph-mon[34289]: mon.b does not exist in monmap, will attempt to join an existing cluster 2024-09-21T03:37:10.494 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:37:10 smithi150 ceph-mon[34289]: using public_addr v2:172.21.15.150:0/0 -> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] 2024-09-21T03:37:10.494 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:37:10 smithi150 ceph-mon[34289]: starting mon.b rank -1 at public addrs [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] at bind addrs [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] mon_data /var/lib/ceph/mon/ceph-b fsid 51116d5c-77ca-11ef-baf5-efdc52797490 2024-09-21T03:37:10.494 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:37:10 smithi150 ceph-mon[34289]: mon.b@-1(???) e0 preinit fsid 51116d5c-77ca-11ef-baf5-efdc52797490 2024-09-21T03:37:10.494 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:37:10 smithi150 ceph-mon[34289]: mon.b@-1(synchronizing).mds e1 new map 2024-09-21T03:37:10.494 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:37:10 smithi150 ceph-mon[34289]: mon.b@-1(synchronizing).mds e1 print_map 2024-09-21T03:37:10.494 INFO:journalctl@ceph.mon.b.smithi150.stdout: e1 2024-09-21T03:37:10.495 INFO:journalctl@ceph.mon.b.smithi150.stdout: btime 2024-09-21T03:35:33:631109+0000 2024-09-21T03:37:10.495 INFO:journalctl@ceph.mon.b.smithi150.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-09-21T03:37:10.495 INFO:journalctl@ceph.mon.b.smithi150.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-21T03:37:10.495 INFO:journalctl@ceph.mon.b.smithi150.stdout: legacy client fscid: -1 2024-09-21T03:37:10.495 INFO:journalctl@ceph.mon.b.smithi150.stdout: 2024-09-21T03:37:10.495 INFO:journalctl@ceph.mon.b.smithi150.stdout: No filesystems configured 2024-09-21T03:37:10.495 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:37:10 smithi150 ceph-mon[34289]: mon.b@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2024-09-21T03:37:10.495 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:37:10 smithi150 ceph-mon[34289]: mon.b@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2024-09-21T03:37:10.495 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:37:10 smithi150 ceph-mon[34289]: mon.b@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2024-09-21T03:37:10.495 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:37:10 smithi150 ceph-mon[34289]: mon.b@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2024-09-21T03:37:10.495 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:37:10 smithi150 ceph-mon[34289]: mon.b@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2024-09-21T03:37:10.495 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:37:10 smithi150 ceph-mon[34289]: mon.b@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2024-09-21T03:37:10.496 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:37:10 smithi150 ceph-mon[34289]: mon.b@-1(synchronizing).osd e4 crush map has features 3314932999778484224, adjusting msgr requires 2024-09-21T03:37:10.496 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:37:10 smithi150 ceph-mon[34289]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-09-21T03:37:10.496 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:37:10 smithi150 ceph-mon[34289]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-09-21T03:37:10.496 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:37:10 smithi150 ceph-mon[34289]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-09-21T03:37:10.496 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:37:10 smithi150 ceph-mon[34289]: Updating smithi150:/etc/ceph/ceph.conf 2024-09-21T03:37:10.496 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:37:10 smithi150 ceph-mon[34289]: Updating smithi150:/var/lib/ceph/51116d5c-77ca-11ef-baf5-efdc52797490/config/ceph.conf 2024-09-21T03:37:10.496 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:37:10 smithi150 ceph-mon[34289]: Deploying cephadm binary to smithi161 2024-09-21T03:37:10.496 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:37:10 smithi150 ceph-mon[34289]: Updating smithi150:/etc/ceph/ceph.client.admin.keyring 2024-09-21T03:37:10.496 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:37:10 smithi150 ceph-mon[34289]: Updating smithi150:/var/lib/ceph/51116d5c-77ca-11ef-baf5-efdc52797490/config/ceph.client.admin.keyring 2024-09-21T03:37:10.496 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:37:10 smithi150 ceph-mon[34289]: from='mgr.14152 172.21.15.39:0/3595829223' entity='mgr.x' 2024-09-21T03:37:10.496 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:37:10 smithi150 ceph-mon[34289]: from='mgr.14152 172.21.15.39:0/3595829223' entity='mgr.x' 2024-09-21T03:37:10.496 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:37:10 smithi150 ceph-mon[34289]: from='mgr.14152 172.21.15.39:0/3595829223' entity='mgr.x' 2024-09-21T03:37:10.496 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:37:10 smithi150 ceph-mon[34289]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:37:10.497 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:37:10 smithi150 ceph-mon[34289]: from='mgr.14152 172.21.15.39:0/3595829223' entity='mgr.x' 2024-09-21T03:37:10.497 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:37:10 smithi150 ceph-mon[34289]: Added host smithi161 2024-09-21T03:37:10.497 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:37:10 smithi150 ceph-mon[34289]: from='mgr.14152 172.21.15.39:0/3595829223' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:37:10.497 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:37:10 smithi150 ceph-mon[34289]: from='mgr.14152 172.21.15.39:0/3595829223' entity='mgr.x' 2024-09-21T03:37:10.497 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:37:10 smithi150 ceph-mon[34289]: from='mgr.14152 172.21.15.39:0/3595829223' entity='mgr.x' 2024-09-21T03:37:10.497 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:37:10 smithi150 ceph-mon[34289]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:37:10.497 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:37:10 smithi150 ceph-mon[34289]: from='mgr.14152 172.21.15.39:0/3595829223' entity='mgr.x' 2024-09-21T03:37:10.497 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:37:10 smithi150 ceph-mon[34289]: from='client.14186 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T03:37:10.497 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:37:10 smithi150 ceph-mon[34289]: from='client.? 172.21.15.39:0/1846763711' entity='client.admin' cmd={"prefix": "osd crush tunables", "profile": "default"} : dispatch 2024-09-21T03:37:10.497 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:37:10 smithi150 ceph-mon[34289]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:37:10.497 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:37:10 smithi150 ceph-mon[34289]: from='client.? 172.21.15.39:0/1846763711' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-09-21T03:37:10.497 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:37:10 smithi150 ceph-mon[34289]: osdmap e4: 0 total, 0 up, 0 in 2024-09-21T03:37:10.498 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:37:10 smithi150 ceph-mon[34289]: from='mgr.14152 172.21.15.39:0/3595829223' entity='mgr.x' 2024-09-21T03:37:10.498 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:37:10 smithi150 ceph-mon[34289]: from='mgr.14152 172.21.15.39:0/3595829223' entity='mgr.x' 2024-09-21T03:37:10.498 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:37:10 smithi150 ceph-mon[34289]: from='mgr.14152 172.21.15.39:0/3595829223' entity='mgr.x' 2024-09-21T03:37:10.498 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:37:10 smithi150 ceph-mon[34289]: from='mgr.14152 172.21.15.39:0/3595829223' entity='mgr.x' 2024-09-21T03:37:10.498 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:37:10 smithi150 ceph-mon[34289]: from='mgr.14152 172.21.15.39:0/3595829223' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd/host:smithi161", "name": "osd_memory_target"} : dispatch 2024-09-21T03:37:10.498 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:37:10 smithi150 ceph-mon[34289]: from='mgr.14152 172.21.15.39:0/3595829223' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:37:10.498 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:37:10 smithi150 ceph-mon[34289]: from='mgr.14152 172.21.15.39:0/3595829223' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:37:10.498 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:37:10 smithi150 ceph-mon[34289]: Updating smithi161:/etc/ceph/ceph.conf 2024-09-21T03:37:10.498 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:37:10 smithi150 ceph-mon[34289]: from='mgr.14152 172.21.15.39:0/3595829223' entity='mgr.x' 2024-09-21T03:37:10.498 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:37:10 smithi150 ceph-mon[34289]: from='mgr.14152 172.21.15.39:0/3595829223' entity='mgr.x' 2024-09-21T03:37:10.498 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:37:10 smithi150 ceph-mon[34289]: from='mgr.14152 172.21.15.39:0/3595829223' entity='mgr.x' 2024-09-21T03:37:10.499 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:37:10 smithi150 ceph-mon[34289]: from='mgr.14152 172.21.15.39:0/3595829223' entity='mgr.x' 2024-09-21T03:37:10.499 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:37:10 smithi150 ceph-mon[34289]: from='mgr.14152 172.21.15.39:0/3595829223' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:37:10.499 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:37:10 smithi150 ceph-mon[34289]: from='mgr.14152 172.21.15.39:0/3595829223' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:37:10.499 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:37:10 smithi150 ceph-mon[34289]: from='mgr.14152 172.21.15.39:0/3595829223' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:37:10.499 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:37:10 smithi150 ceph-mon[34289]: from='mgr.14152 172.21.15.39:0/3595829223' entity='mgr.x' 2024-09-21T03:37:10.499 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:37:10 smithi150 ceph-mon[34289]: from='mgr.14152 172.21.15.39:0/3595829223' entity='mgr.x' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-09-21T03:37:10.499 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:37:10 smithi150 ceph-mon[34289]: from='mgr.14152 172.21.15.39:0/3595829223' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:37:10.499 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:37:10 smithi150 ceph-mon[34289]: Updating smithi161:/var/lib/ceph/51116d5c-77ca-11ef-baf5-efdc52797490/config/ceph.conf 2024-09-21T03:37:10.499 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:37:10 smithi150 ceph-mon[34289]: Updating smithi161:/etc/ceph/ceph.client.admin.keyring 2024-09-21T03:37:10.499 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:37:10 smithi150 ceph-mon[34289]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:37:10.499 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:37:10 smithi150 ceph-mon[34289]: Updating smithi161:/var/lib/ceph/51116d5c-77ca-11ef-baf5-efdc52797490/config/ceph.client.admin.keyring 2024-09-21T03:37:10.499 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:37:10 smithi150 ceph-mon[34289]: from='client.14190 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi039:172.21.15.39=a;smithi150:172.21.15.150=b;smithi161:172.21.15.161=c", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:37:10.499 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:37:10 smithi150 ceph-mon[34289]: Saving service mon spec with placement smithi039:172.21.15.39=a;smithi150:172.21.15.150=b;smithi161:172.21.15.161=c;count:3 2024-09-21T03:37:10.500 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:37:10 smithi150 ceph-mon[34289]: Deploying daemon mon.c on smithi161 2024-09-21T03:37:10.500 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:37:10 smithi150 ceph-mon[34289]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:37:10.500 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:37:10 smithi150 ceph-mon[34289]: from='client.? 172.21.15.161:0/261893970' entity='client.admin' cmd={"prefix": "mon dump", "format": "json"} : dispatch 2024-09-21T03:37:10.500 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:37:10 smithi150 ceph-mon[34289]: from='mgr.14152 172.21.15.39:0/3595829223' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-21T03:37:10.500 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:37:10 smithi150 ceph-mon[34289]: from='mgr.14152 172.21.15.39:0/3595829223' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T03:37:10.500 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:37:10 smithi150 ceph-mon[34289]: mon.a calling monitor election 2024-09-21T03:37:10.500 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:37:10 smithi150 ceph-mon[34289]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:37:10.500 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:37:10 smithi150 ceph-mon[34289]: from='mgr.14152 172.21.15.39:0/3595829223' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T03:37:10.500 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:37:10 smithi150 ceph-mon[34289]: from='mgr.14152 172.21.15.39:0/3595829223' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T03:37:10.500 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:37:10 smithi150 ceph-mon[34289]: mon.c calling monitor election 2024-09-21T03:37:10.500 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:37:10 smithi150 ceph-mon[34289]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:37:10.501 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:37:10 smithi150 ceph-mon[34289]: from='mgr.14152 172.21.15.39:0/3595829223' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T03:37:10.501 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:37:10 smithi150 ceph-mon[34289]: from='mgr.14152 172.21.15.39:0/3595829223' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T03:37:10.501 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:37:10 smithi150 ceph-mon[34289]: from='mgr.14152 172.21.15.39:0/3595829223' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T03:37:10.501 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:37:10 smithi150 ceph-mon[34289]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-09-21T03:37:10.501 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:37:10 smithi150 ceph-mon[34289]: monmap epoch 2 2024-09-21T03:37:10.501 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:37:10 smithi150 ceph-mon[34289]: fsid 51116d5c-77ca-11ef-baf5-efdc52797490 2024-09-21T03:37:10.501 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:37:10 smithi150 ceph-mon[34289]: last_changed 2024-09-21T03:37:01.289853+0000 2024-09-21T03:37:10.501 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:37:10 smithi150 ceph-mon[34289]: created 2024-09-21T03:35:21.020316+0000 2024-09-21T03:37:10.501 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:37:10 smithi150 ceph-mon[34289]: min_mon_release 19 (squid) 2024-09-21T03:37:10.501 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:37:10 smithi150 ceph-mon[34289]: election_strategy: 1 2024-09-21T03:37:10.501 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:37:10 smithi150 ceph-mon[34289]: 0: [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] mon.a 2024-09-21T03:37:10.501 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:37:10 smithi150 ceph-mon[34289]: 1: [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] mon.c 2024-09-21T03:37:10.501 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:37:10 smithi150 ceph-mon[34289]: fsmap 2024-09-21T03:37:10.502 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:37:10 smithi150 ceph-mon[34289]: osdmap e4: 0 total, 0 up, 0 in 2024-09-21T03:37:10.502 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:37:10 smithi150 ceph-mon[34289]: mgrmap e14: x(active, since 42s) 2024-09-21T03:37:10.502 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:37:10 smithi150 ceph-mon[34289]: overall HEALTH_OK 2024-09-21T03:37:10.502 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:37:10 smithi150 ceph-mon[34289]: from='mgr.14152 172.21.15.39:0/3595829223' entity='mgr.x' 2024-09-21T03:37:10.502 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:37:10 smithi150 ceph-mon[34289]: from='mgr.14152 172.21.15.39:0/3595829223' entity='mgr.x' 2024-09-21T03:37:10.502 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:37:10 smithi150 ceph-mon[34289]: from='mgr.14152 172.21.15.39:0/3595829223' entity='mgr.x' 2024-09-21T03:37:10.502 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:37:10 smithi150 ceph-mon[34289]: from='mgr.14152 172.21.15.39:0/3595829223' entity='mgr.x' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-09-21T03:37:10.502 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:37:10 smithi150 ceph-mon[34289]: from='mgr.14152 172.21.15.39:0/3595829223' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:37:10.502 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:37:10 smithi150 ceph-mon[34289]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:37:10.502 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:37:10 smithi150 ceph-mon[34289]: Deploying daemon mon.b on smithi150 2024-09-21T03:37:10.502 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:37:10 smithi150 ceph-mon[34289]: from='client.? 172.21.15.161:0/1614888602' entity='client.admin' cmd={"prefix": "mon dump", "format": "json"} : dispatch 2024-09-21T03:37:10.502 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:37:10 smithi150 ceph-mon[34289]: from='mgr.14152 172.21.15.39:0/3595829223' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T03:37:10.503 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:37:10 smithi150 ceph-mon[34289]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:37:10.503 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:37:10 smithi150 ceph-mon[34289]: mon.b@-1(synchronizing).paxosservice(auth 1..3) refresh upgraded, format 0 -> 3 2024-09-21T03:37:10.503 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:37:10 smithi150 bash[34271]: 6cc60c3531e0d4f3e3fb1862c1d4ebe3d84e01caaf4b7ef4c0a953a703ab7ffb 2024-09-21T03:37:10.503 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:37:10 smithi150 systemd[1]: Started Ceph mon.b for 51116d5c-77ca-11ef-baf5-efdc52797490. 2024-09-21T03:37:11.337 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-09-21T03:37:11.338 DEBUG:teuthology.orchestra.run.smithi161:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 51116d5c-77ca-11ef-baf5-efdc52797490 -- ceph mon dump -f json 2024-09-21T03:37:11.589 INFO:teuthology.orchestra.run.smithi161.stderr:Inferring config /var/lib/ceph/51116d5c-77ca-11ef-baf5-efdc52797490/mon.c/config 2024-09-21T03:37:15.468 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:37:15 smithi161 ceph-mon[34533]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:37:15.468 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:37:15 smithi161 ceph-mon[34533]: from='mgr.14152 172.21.15.39:0/3595829223' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-21T03:37:15.468 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:37:15 smithi161 ceph-mon[34533]: mon.c calling monitor election 2024-09-21T03:37:15.469 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:37:15 smithi161 ceph-mon[34533]: mon.a calling monitor election 2024-09-21T03:37:15.469 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:37:15 smithi161 ceph-mon[34533]: from='mgr.14152 172.21.15.39:0/3595829223' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T03:37:15.469 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:37:15 smithi161 ceph-mon[34533]: from='mgr.14152 172.21.15.39:0/3595829223' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T03:37:15.469 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:37:15 smithi161 ceph-mon[34533]: from='mgr.14152 172.21.15.39:0/3595829223' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T03:37:15.469 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:37:15 smithi161 ceph-mon[34533]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:37:15.469 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:37:15 smithi161 ceph-mon[34533]: from='mgr.14152 172.21.15.39:0/3595829223' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T03:37:15.469 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:37:15 smithi161 ceph-mon[34533]: from='mgr.14152 172.21.15.39:0/3595829223' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T03:37:15.469 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:37:15 smithi161 ceph-mon[34533]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:37:15.469 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:37:15 smithi161 ceph-mon[34533]: from='mgr.14152 172.21.15.39:0/3595829223' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T03:37:15.469 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:37:15 smithi161 ceph-mon[34533]: from='mgr.14152 172.21.15.39:0/3595829223' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T03:37:15.469 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:37:15 smithi161 ceph-mon[34533]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-09-21T03:37:15.469 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:37:15 smithi161 ceph-mon[34533]: monmap epoch 3 2024-09-21T03:37:15.469 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:37:15 smithi161 ceph-mon[34533]: fsid 51116d5c-77ca-11ef-baf5-efdc52797490 2024-09-21T03:37:15.469 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:37:15 smithi161 ceph-mon[34533]: last_changed 2024-09-21T03:37:10.134590+0000 2024-09-21T03:37:15.470 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:37:15 smithi161 ceph-mon[34533]: created 2024-09-21T03:35:21.020316+0000 2024-09-21T03:37:15.470 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:37:15 smithi161 ceph-mon[34533]: min_mon_release 19 (squid) 2024-09-21T03:37:15.470 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:37:15 smithi161 ceph-mon[34533]: election_strategy: 1 2024-09-21T03:37:15.470 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:37:15 smithi161 ceph-mon[34533]: 0: [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] mon.a 2024-09-21T03:37:15.470 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:37:15 smithi161 ceph-mon[34533]: 1: [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] mon.c 2024-09-21T03:37:15.470 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:37:15 smithi161 ceph-mon[34533]: 2: [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] mon.b 2024-09-21T03:37:15.470 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:37:15 smithi161 ceph-mon[34533]: fsmap 2024-09-21T03:37:15.470 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:37:15 smithi161 ceph-mon[34533]: osdmap e4: 0 total, 0 up, 0 in 2024-09-21T03:37:15.470 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:37:15 smithi161 ceph-mon[34533]: mgrmap e14: x(active, since 51s) 2024-09-21T03:37:15.470 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:37:15 smithi161 ceph-mon[34533]: overall HEALTH_OK 2024-09-21T03:37:15.470 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:37:15 smithi161 ceph-mon[34533]: from='mgr.14152 172.21.15.39:0/3595829223' entity='mgr.x' 2024-09-21T03:37:15.470 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:37:15 smithi161 ceph-mon[34533]: from='mgr.14152 172.21.15.39:0/3595829223' entity='mgr.x' 2024-09-21T03:37:15.470 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:37:15 smithi161 ceph-mon[34533]: from='mgr.14152 172.21.15.39:0/3595829223' entity='mgr.x' 2024-09-21T03:37:15.470 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:37:15 smithi161 ceph-mon[34533]: from='mgr.14152 172.21.15.39:0/3595829223' entity='mgr.x' 2024-09-21T03:37:15.526 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-09-21T03:37:15.526 INFO:teuthology.orchestra.run.smithi161.stdout:{"epoch":3,"fsid":"51116d5c-77ca-11ef-baf5-efdc52797490","modified":"2024-09-21T03:37:10.134590Z","created":"2024-09-21T03:35:21.020316Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:3300","nonce":0},{"type":"v1","addr":"172.21.15.39:6789","nonce":0}]},"addr":"172.21.15.39:6789/0","public_addr":"172.21.15.39:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:3300","nonce":0},{"type":"v1","addr":"172.21.15.161:6789","nonce":0}]},"addr":"172.21.15.161:6789/0","public_addr":"172.21.15.161:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:3300","nonce":0},{"type":"v1","addr":"172.21.15.150:6789","nonce":0}]},"addr":"172.21.15.150:6789/0","public_addr":"172.21.15.150:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-09-21T03:37:15.526 INFO:teuthology.orchestra.run.smithi161.stderr:dumped monmap epoch 3 2024-09-21T03:37:15.584 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:37:15 smithi039 ceph-mon[31211]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:37:15.584 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:37:15 smithi039 ceph-mon[31211]: from='mgr.14152 172.21.15.39:0/3595829223' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-21T03:37:15.584 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:37:15 smithi039 ceph-mon[31211]: mon.c calling monitor election 2024-09-21T03:37:15.584 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:37:15 smithi039 ceph-mon[31211]: mon.a calling monitor election 2024-09-21T03:37:15.584 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:37:15 smithi039 ceph-mon[31211]: from='mgr.14152 172.21.15.39:0/3595829223' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T03:37:15.584 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:37:15 smithi039 ceph-mon[31211]: from='mgr.14152 172.21.15.39:0/3595829223' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T03:37:15.584 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:37:15 smithi039 ceph-mon[31211]: from='mgr.14152 172.21.15.39:0/3595829223' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T03:37:15.584 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:37:15 smithi039 ceph-mon[31211]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:37:15.584 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:37:15 smithi039 ceph-mon[31211]: from='mgr.14152 172.21.15.39:0/3595829223' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T03:37:15.584 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:37:15 smithi039 ceph-mon[31211]: from='mgr.14152 172.21.15.39:0/3595829223' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T03:37:15.584 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:37:15 smithi039 ceph-mon[31211]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:37:15.584 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:37:15 smithi039 ceph-mon[31211]: from='mgr.14152 172.21.15.39:0/3595829223' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T03:37:15.584 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:37:15 smithi039 ceph-mon[31211]: from='mgr.14152 172.21.15.39:0/3595829223' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T03:37:15.585 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:37:15 smithi039 ceph-mon[31211]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-09-21T03:37:15.585 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:37:15 smithi039 ceph-mon[31211]: monmap epoch 3 2024-09-21T03:37:15.585 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:37:15 smithi039 ceph-mon[31211]: fsid 51116d5c-77ca-11ef-baf5-efdc52797490 2024-09-21T03:37:15.585 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:37:15 smithi039 ceph-mon[31211]: last_changed 2024-09-21T03:37:10.134590+0000 2024-09-21T03:37:15.585 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:37:15 smithi039 ceph-mon[31211]: created 2024-09-21T03:35:21.020316+0000 2024-09-21T03:37:15.585 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:37:15 smithi039 ceph-mon[31211]: min_mon_release 19 (squid) 2024-09-21T03:37:15.585 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:37:15 smithi039 ceph-mon[31211]: election_strategy: 1 2024-09-21T03:37:15.585 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:37:15 smithi039 ceph-mon[31211]: 0: [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] mon.a 2024-09-21T03:37:15.585 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:37:15 smithi039 ceph-mon[31211]: 1: [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] mon.c 2024-09-21T03:37:15.585 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:37:15 smithi039 ceph-mon[31211]: 2: [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] mon.b 2024-09-21T03:37:15.585 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:37:15 smithi039 ceph-mon[31211]: fsmap 2024-09-21T03:37:15.585 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:37:15 smithi039 ceph-mon[31211]: osdmap e4: 0 total, 0 up, 0 in 2024-09-21T03:37:15.585 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:37:15 smithi039 ceph-mon[31211]: mgrmap e14: x(active, since 51s) 2024-09-21T03:37:15.585 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:37:15 smithi039 ceph-mon[31211]: overall HEALTH_OK 2024-09-21T03:37:15.586 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:37:15 smithi039 ceph-mon[31211]: from='mgr.14152 172.21.15.39:0/3595829223' entity='mgr.x' 2024-09-21T03:37:15.586 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:37:15 smithi039 ceph-mon[31211]: from='mgr.14152 172.21.15.39:0/3595829223' entity='mgr.x' 2024-09-21T03:37:15.586 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:37:15 smithi039 ceph-mon[31211]: from='mgr.14152 172.21.15.39:0/3595829223' entity='mgr.x' 2024-09-21T03:37:15.586 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:37:15 smithi039 ceph-mon[31211]: from='mgr.14152 172.21.15.39:0/3595829223' entity='mgr.x' 2024-09-21T03:37:16.099 INFO:tasks.cephadm:Generating final ceph.conf file... 2024-09-21T03:37:16.100 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 51116d5c-77ca-11ef-baf5-efdc52797490 -- ceph config generate-minimal-conf 2024-09-21T03:37:16.324 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:37:16 smithi161 ceph-mon[34533]: from='mgr.14152 172.21.15.39:0/3595829223' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:37:16.324 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:37:16 smithi161 ceph-mon[34533]: from='client.? 172.21.15.161:0/380404219' entity='client.admin' cmd={"prefix": "mon dump", "format": "json"} : dispatch 2024-09-21T03:37:16.324 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:37:16 smithi161 ceph-mon[34533]: from='mgr.14152 172.21.15.39:0/3595829223' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T03:37:16.365 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/51116d5c-77ca-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:37:16.482 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:37:16 smithi039 ceph-mon[31211]: from='mgr.14152 172.21.15.39:0/3595829223' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:37:16.482 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:37:16 smithi039 ceph-mon[31211]: from='client.? 172.21.15.161:0/380404219' entity='client.admin' cmd={"prefix": "mon dump", "format": "json"} : dispatch 2024-09-21T03:37:16.482 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:37:16 smithi039 ceph-mon[31211]: from='mgr.14152 172.21.15.39:0/3595829223' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T03:37:16.965 INFO:teuthology.orchestra.run.smithi039.stdout:# minimal ceph.conf for 51116d5c-77ca-11ef-baf5-efdc52797490 2024-09-21T03:37:16.965 INFO:teuthology.orchestra.run.smithi039.stdout:[global] 2024-09-21T03:37:16.965 INFO:teuthology.orchestra.run.smithi039.stdout: fsid = 51116d5c-77ca-11ef-baf5-efdc52797490 2024-09-21T03:37:16.965 INFO:teuthology.orchestra.run.smithi039.stdout: mon_host = [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] 2024-09-21T03:37:17.475 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:37:17 smithi150 ceph-mon[34289]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:37:17.475 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:37:17 smithi150 ceph-mon[34289]: from='mgr.14152 172.21.15.39:0/3595829223' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-21T03:37:17.475 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:37:17 smithi150 ceph-mon[34289]: mon.c calling monitor election 2024-09-21T03:37:17.475 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:37:17 smithi150 ceph-mon[34289]: mon.a calling monitor election 2024-09-21T03:37:17.475 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:37:17 smithi150 ceph-mon[34289]: from='mgr.14152 172.21.15.39:0/3595829223' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T03:37:17.475 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:37:17 smithi150 ceph-mon[34289]: from='mgr.14152 172.21.15.39:0/3595829223' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T03:37:17.475 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:37:17 smithi150 ceph-mon[34289]: from='mgr.14152 172.21.15.39:0/3595829223' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T03:37:17.475 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:37:17 smithi150 ceph-mon[34289]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:37:17.475 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:37:17 smithi150 ceph-mon[34289]: from='mgr.14152 172.21.15.39:0/3595829223' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T03:37:17.476 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:37:17 smithi150 ceph-mon[34289]: from='mgr.14152 172.21.15.39:0/3595829223' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T03:37:17.476 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:37:17 smithi150 ceph-mon[34289]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:37:17.476 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:37:17 smithi150 ceph-mon[34289]: from='mgr.14152 172.21.15.39:0/3595829223' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T03:37:17.476 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:37:17 smithi150 ceph-mon[34289]: from='mgr.14152 172.21.15.39:0/3595829223' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T03:37:17.476 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:37:17 smithi150 ceph-mon[34289]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-09-21T03:37:17.476 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:37:17 smithi150 ceph-mon[34289]: monmap epoch 3 2024-09-21T03:37:17.476 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:37:17 smithi150 ceph-mon[34289]: fsid 51116d5c-77ca-11ef-baf5-efdc52797490 2024-09-21T03:37:17.476 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:37:17 smithi150 ceph-mon[34289]: last_changed 2024-09-21T03:37:10.134590+0000 2024-09-21T03:37:17.476 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:37:17 smithi150 ceph-mon[34289]: created 2024-09-21T03:35:21.020316+0000 2024-09-21T03:37:17.476 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:37:17 smithi150 ceph-mon[34289]: min_mon_release 19 (squid) 2024-09-21T03:37:17.476 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:37:17 smithi150 ceph-mon[34289]: election_strategy: 1 2024-09-21T03:37:17.476 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:37:17 smithi150 ceph-mon[34289]: 0: [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] mon.a 2024-09-21T03:37:17.476 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:37:17 smithi150 ceph-mon[34289]: 1: [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] mon.c 2024-09-21T03:37:17.476 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:37:17 smithi150 ceph-mon[34289]: 2: [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] mon.b 2024-09-21T03:37:17.477 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:37:17 smithi150 ceph-mon[34289]: fsmap 2024-09-21T03:37:17.477 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:37:17 smithi150 ceph-mon[34289]: osdmap e4: 0 total, 0 up, 0 in 2024-09-21T03:37:17.477 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:37:17 smithi150 ceph-mon[34289]: mgrmap e14: x(active, since 51s) 2024-09-21T03:37:17.477 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:37:17 smithi150 ceph-mon[34289]: overall HEALTH_OK 2024-09-21T03:37:17.477 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:37:17 smithi150 ceph-mon[34289]: from='mgr.14152 172.21.15.39:0/3595829223' entity='mgr.x' 2024-09-21T03:37:17.477 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:37:17 smithi150 ceph-mon[34289]: from='mgr.14152 172.21.15.39:0/3595829223' entity='mgr.x' 2024-09-21T03:37:17.477 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:37:17 smithi150 ceph-mon[34289]: from='mgr.14152 172.21.15.39:0/3595829223' entity='mgr.x' 2024-09-21T03:37:17.477 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:37:17 smithi150 ceph-mon[34289]: from='mgr.14152 172.21.15.39:0/3595829223' entity='mgr.x' 2024-09-21T03:37:17.477 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:37:17 smithi150 ceph-mon[34289]: from='mgr.14152 172.21.15.39:0/3595829223' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:37:17.477 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:37:17 smithi150 ceph-mon[34289]: from='client.? 172.21.15.161:0/380404219' entity='client.admin' cmd={"prefix": "mon dump", "format": "json"} : dispatch 2024-09-21T03:37:17.477 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:37:17 smithi150 ceph-mon[34289]: from='mgr.14152 172.21.15.39:0/3595829223' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T03:37:17.582 INFO:tasks.cephadm:Distributing (final) config and client.admin keyring... 2024-09-21T03:37:17.582 DEBUG:teuthology.orchestra.run.smithi039:> set -ex 2024-09-21T03:37:17.582 DEBUG:teuthology.orchestra.run.smithi039:> sudo dd of=/etc/ceph/ceph.conf 2024-09-21T03:37:17.611 DEBUG:teuthology.orchestra.run.smithi039:> set -ex 2024-09-21T03:37:17.612 DEBUG:teuthology.orchestra.run.smithi039:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-09-21T03:37:17.677 DEBUG:teuthology.orchestra.run.smithi150:> set -ex 2024-09-21T03:37:17.677 DEBUG:teuthology.orchestra.run.smithi150:> sudo dd of=/etc/ceph/ceph.conf 2024-09-21T03:37:17.705 DEBUG:teuthology.orchestra.run.smithi150:> set -ex 2024-09-21T03:37:17.706 DEBUG:teuthology.orchestra.run.smithi150:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-09-21T03:37:17.771 DEBUG:teuthology.orchestra.run.smithi161:> set -ex 2024-09-21T03:37:17.771 DEBUG:teuthology.orchestra.run.smithi161:> sudo dd of=/etc/ceph/ceph.conf 2024-09-21T03:37:17.801 DEBUG:teuthology.orchestra.run.smithi161:> set -ex 2024-09-21T03:37:17.801 DEBUG:teuthology.orchestra.run.smithi161:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-09-21T03:37:17.869 INFO:tasks.cephadm:enabling module snap_schedule 2024-09-21T03:37:17.869 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 51116d5c-77ca-11ef-baf5-efdc52797490 -- sudo ceph --cluster ceph mgr module enable snap_schedule 2024-09-21T03:37:18.334 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:37:18 smithi039 ceph-mon[31211]: mon.b calling monitor election 2024-09-21T03:37:18.334 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:37:18 smithi039 ceph-mon[31211]: Updating smithi161:/var/lib/ceph/51116d5c-77ca-11ef-baf5-efdc52797490/config/ceph.conf 2024-09-21T03:37:18.334 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:37:18 smithi039 ceph-mon[31211]: Updating smithi039:/var/lib/ceph/51116d5c-77ca-11ef-baf5-efdc52797490/config/ceph.conf 2024-09-21T03:37:18.334 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:37:18 smithi039 ceph-mon[31211]: Updating smithi150:/var/lib/ceph/51116d5c-77ca-11ef-baf5-efdc52797490/config/ceph.conf 2024-09-21T03:37:18.334 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:37:18 smithi039 ceph-mon[31211]: mon.b calling monitor election 2024-09-21T03:37:18.334 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:37:18 smithi039 ceph-mon[31211]: mon.c calling monitor election 2024-09-21T03:37:18.335 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:37:18 smithi039 ceph-mon[31211]: mon.a calling monitor election 2024-09-21T03:37:18.335 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:37:18 smithi039 ceph-mon[31211]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-09-21T03:37:18.335 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:37:18 smithi039 ceph-mon[31211]: monmap epoch 3 2024-09-21T03:37:18.335 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:37:18 smithi039 ceph-mon[31211]: fsid 51116d5c-77ca-11ef-baf5-efdc52797490 2024-09-21T03:37:18.335 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:37:18 smithi039 ceph-mon[31211]: last_changed 2024-09-21T03:37:10.134590+0000 2024-09-21T03:37:18.335 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:37:18 smithi039 ceph-mon[31211]: created 2024-09-21T03:35:21.020316+0000 2024-09-21T03:37:18.335 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:37:18 smithi039 ceph-mon[31211]: min_mon_release 19 (squid) 2024-09-21T03:37:18.335 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:37:18 smithi039 ceph-mon[31211]: election_strategy: 1 2024-09-21T03:37:18.335 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:37:18 smithi039 ceph-mon[31211]: 0: [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] mon.a 2024-09-21T03:37:18.335 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:37:18 smithi039 ceph-mon[31211]: 1: [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] mon.c 2024-09-21T03:37:18.335 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:37:18 smithi039 ceph-mon[31211]: 2: [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] mon.b 2024-09-21T03:37:18.335 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:37:18 smithi039 ceph-mon[31211]: fsmap 2024-09-21T03:37:18.335 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:37:18 smithi039 ceph-mon[31211]: osdmap e4: 0 total, 0 up, 0 in 2024-09-21T03:37:18.335 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:37:18 smithi039 ceph-mon[31211]: mgrmap e14: x(active, since 53s) 2024-09-21T03:37:18.336 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:37:18 smithi039 ceph-mon[31211]: overall HEALTH_OK 2024-09-21T03:37:18.336 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:37:18 smithi039 ceph-mon[31211]: from='mgr.14152 172.21.15.39:0/3595829223' entity='mgr.x' 2024-09-21T03:37:18.336 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:37:18 smithi039 ceph-mon[31211]: from='mgr.14152 172.21.15.39:0/3595829223' entity='mgr.x' 2024-09-21T03:37:18.336 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:37:18 smithi039 ceph-mon[31211]: from='mgr.14152 172.21.15.39:0/3595829223' entity='mgr.x' 2024-09-21T03:37:18.336 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:37:18 smithi039 ceph-mon[31211]: from='mgr.14152 172.21.15.39:0/3595829223' entity='mgr.x' 2024-09-21T03:37:18.336 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:37:18 smithi039 ceph-mon[31211]: Reconfiguring mon.a (unknown last config time)... 2024-09-21T03:37:18.336 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:37:18 smithi039 ceph-mon[31211]: from='mgr.14152 172.21.15.39:0/3595829223' entity='mgr.x' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-09-21T03:37:18.336 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:37:18 smithi039 ceph-mon[31211]: from='mgr.14152 172.21.15.39:0/3595829223' entity='mgr.x' cmd={"prefix": "config get", "who": "mon", "key": "public_network"} : dispatch 2024-09-21T03:37:18.336 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:37:18 smithi039 ceph-mon[31211]: from='mgr.14152 172.21.15.39:0/3595829223' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:37:18.336 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:37:18 smithi039 ceph-mon[31211]: Reconfiguring daemon mon.a on smithi039 2024-09-21T03:37:18.399 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:37:18 smithi161 ceph-mon[34533]: mon.b calling monitor election 2024-09-21T03:37:18.399 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:37:18 smithi161 ceph-mon[34533]: Updating smithi161:/var/lib/ceph/51116d5c-77ca-11ef-baf5-efdc52797490/config/ceph.conf 2024-09-21T03:37:18.399 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:37:18 smithi161 ceph-mon[34533]: Updating smithi039:/var/lib/ceph/51116d5c-77ca-11ef-baf5-efdc52797490/config/ceph.conf 2024-09-21T03:37:18.399 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:37:18 smithi161 ceph-mon[34533]: Updating smithi150:/var/lib/ceph/51116d5c-77ca-11ef-baf5-efdc52797490/config/ceph.conf 2024-09-21T03:37:18.399 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:37:18 smithi161 ceph-mon[34533]: mon.b calling monitor election 2024-09-21T03:37:18.399 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:37:18 smithi161 ceph-mon[34533]: mon.c calling monitor election 2024-09-21T03:37:18.399 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:37:18 smithi161 ceph-mon[34533]: mon.a calling monitor election 2024-09-21T03:37:18.399 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:37:18 smithi161 ceph-mon[34533]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-09-21T03:37:18.399 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:37:18 smithi161 ceph-mon[34533]: monmap epoch 3 2024-09-21T03:37:18.399 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:37:18 smithi161 ceph-mon[34533]: fsid 51116d5c-77ca-11ef-baf5-efdc52797490 2024-09-21T03:37:18.400 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:37:18 smithi161 ceph-mon[34533]: last_changed 2024-09-21T03:37:10.134590+0000 2024-09-21T03:37:18.400 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:37:18 smithi161 ceph-mon[34533]: created 2024-09-21T03:35:21.020316+0000 2024-09-21T03:37:18.400 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:37:18 smithi161 ceph-mon[34533]: min_mon_release 19 (squid) 2024-09-21T03:37:18.400 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:37:18 smithi161 ceph-mon[34533]: election_strategy: 1 2024-09-21T03:37:18.400 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:37:18 smithi161 ceph-mon[34533]: 0: [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] mon.a 2024-09-21T03:37:18.400 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:37:18 smithi161 ceph-mon[34533]: 1: [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] mon.c 2024-09-21T03:37:18.400 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:37:18 smithi161 ceph-mon[34533]: 2: [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] mon.b 2024-09-21T03:37:18.400 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:37:18 smithi161 ceph-mon[34533]: fsmap 2024-09-21T03:37:18.400 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:37:18 smithi161 ceph-mon[34533]: osdmap e4: 0 total, 0 up, 0 in 2024-09-21T03:37:18.400 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:37:18 smithi161 ceph-mon[34533]: mgrmap e14: x(active, since 53s) 2024-09-21T03:37:18.400 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:37:18 smithi161 ceph-mon[34533]: overall HEALTH_OK 2024-09-21T03:37:18.400 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:37:18 smithi161 ceph-mon[34533]: from='mgr.14152 172.21.15.39:0/3595829223' entity='mgr.x' 2024-09-21T03:37:18.400 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:37:18 smithi161 ceph-mon[34533]: from='mgr.14152 172.21.15.39:0/3595829223' entity='mgr.x' 2024-09-21T03:37:18.400 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:37:18 smithi161 ceph-mon[34533]: from='mgr.14152 172.21.15.39:0/3595829223' entity='mgr.x' 2024-09-21T03:37:18.400 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:37:18 smithi161 ceph-mon[34533]: from='mgr.14152 172.21.15.39:0/3595829223' entity='mgr.x' 2024-09-21T03:37:18.401 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:37:18 smithi161 ceph-mon[34533]: Reconfiguring mon.a (unknown last config time)... 2024-09-21T03:37:18.401 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:37:18 smithi161 ceph-mon[34533]: from='mgr.14152 172.21.15.39:0/3595829223' entity='mgr.x' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-09-21T03:37:18.401 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:37:18 smithi161 ceph-mon[34533]: from='mgr.14152 172.21.15.39:0/3595829223' entity='mgr.x' cmd={"prefix": "config get", "who": "mon", "key": "public_network"} : dispatch 2024-09-21T03:37:18.401 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:37:18 smithi161 ceph-mon[34533]: from='mgr.14152 172.21.15.39:0/3595829223' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:37:18.401 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:37:18 smithi161 ceph-mon[34533]: Reconfiguring daemon mon.a on smithi039 2024-09-21T03:37:18.475 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:37:18 smithi150 ceph-mon[34289]: mon.b calling monitor election 2024-09-21T03:37:18.475 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:37:18 smithi150 ceph-mon[34289]: Updating smithi161:/var/lib/ceph/51116d5c-77ca-11ef-baf5-efdc52797490/config/ceph.conf 2024-09-21T03:37:18.475 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:37:18 smithi150 ceph-mon[34289]: Updating smithi039:/var/lib/ceph/51116d5c-77ca-11ef-baf5-efdc52797490/config/ceph.conf 2024-09-21T03:37:18.475 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:37:18 smithi150 ceph-mon[34289]: Updating smithi150:/var/lib/ceph/51116d5c-77ca-11ef-baf5-efdc52797490/config/ceph.conf 2024-09-21T03:37:18.475 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:37:18 smithi150 ceph-mon[34289]: mon.b calling monitor election 2024-09-21T03:37:18.475 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:37:18 smithi150 ceph-mon[34289]: mon.c calling monitor election 2024-09-21T03:37:18.475 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:37:18 smithi150 ceph-mon[34289]: mon.a calling monitor election 2024-09-21T03:37:18.475 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:37:18 smithi150 ceph-mon[34289]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-09-21T03:37:18.475 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:37:18 smithi150 ceph-mon[34289]: monmap epoch 3 2024-09-21T03:37:18.475 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:37:18 smithi150 ceph-mon[34289]: fsid 51116d5c-77ca-11ef-baf5-efdc52797490 2024-09-21T03:37:18.475 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:37:18 smithi150 ceph-mon[34289]: last_changed 2024-09-21T03:37:10.134590+0000 2024-09-21T03:37:18.475 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:37:18 smithi150 ceph-mon[34289]: created 2024-09-21T03:35:21.020316+0000 2024-09-21T03:37:18.475 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:37:18 smithi150 ceph-mon[34289]: min_mon_release 19 (squid) 2024-09-21T03:37:18.476 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:37:18 smithi150 ceph-mon[34289]: election_strategy: 1 2024-09-21T03:37:18.476 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:37:18 smithi150 ceph-mon[34289]: 0: [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] mon.a 2024-09-21T03:37:18.476 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:37:18 smithi150 ceph-mon[34289]: 1: [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] mon.c 2024-09-21T03:37:18.476 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:37:18 smithi150 ceph-mon[34289]: 2: [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] mon.b 2024-09-21T03:37:18.476 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:37:18 smithi150 ceph-mon[34289]: fsmap 2024-09-21T03:37:18.476 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:37:18 smithi150 ceph-mon[34289]: osdmap e4: 0 total, 0 up, 0 in 2024-09-21T03:37:18.476 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:37:18 smithi150 ceph-mon[34289]: mgrmap e14: x(active, since 53s) 2024-09-21T03:37:18.476 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:37:18 smithi150 ceph-mon[34289]: overall HEALTH_OK 2024-09-21T03:37:18.476 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:37:18 smithi150 ceph-mon[34289]: from='mgr.14152 172.21.15.39:0/3595829223' entity='mgr.x' 2024-09-21T03:37:18.476 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:37:18 smithi150 ceph-mon[34289]: from='mgr.14152 172.21.15.39:0/3595829223' entity='mgr.x' 2024-09-21T03:37:18.476 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:37:18 smithi150 ceph-mon[34289]: from='mgr.14152 172.21.15.39:0/3595829223' entity='mgr.x' 2024-09-21T03:37:18.476 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:37:18 smithi150 ceph-mon[34289]: from='mgr.14152 172.21.15.39:0/3595829223' entity='mgr.x' 2024-09-21T03:37:18.476 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:37:18 smithi150 ceph-mon[34289]: Reconfiguring mon.a (unknown last config time)... 2024-09-21T03:37:18.476 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:37:18 smithi150 ceph-mon[34289]: from='mgr.14152 172.21.15.39:0/3595829223' entity='mgr.x' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-09-21T03:37:18.476 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:37:18 smithi150 ceph-mon[34289]: from='mgr.14152 172.21.15.39:0/3595829223' entity='mgr.x' cmd={"prefix": "config get", "who": "mon", "key": "public_network"} : dispatch 2024-09-21T03:37:18.476 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:37:18 smithi150 ceph-mon[34289]: from='mgr.14152 172.21.15.39:0/3595829223' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:37:18.476 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:37:18 smithi150 ceph-mon[34289]: Reconfiguring daemon mon.a on smithi039 2024-09-21T03:37:18.755 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/51116d5c-77ca-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:37:19.127 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:37:19 smithi039 ceph-mon[31211]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:37:19.127 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:37:19 smithi039 ceph-mon[31211]: from='mgr.14152 172.21.15.39:0/3595829223' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T03:37:19.127 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:37:19 smithi039 ceph-mon[31211]: from='mgr.14152 172.21.15.39:0/3595829223' entity='mgr.x' 2024-09-21T03:37:19.127 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:37:19 smithi039 ceph-mon[31211]: from='mgr.14152 172.21.15.39:0/3595829223' entity='mgr.x' 2024-09-21T03:37:19.127 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:37:19 smithi039 ceph-mon[31211]: Reconfiguring mon.b (monmap changed)... 2024-09-21T03:37:19.127 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:37:19 smithi039 ceph-mon[31211]: from='mgr.14152 172.21.15.39:0/3595829223' entity='mgr.x' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-09-21T03:37:19.127 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:37:19 smithi039 ceph-mon[31211]: from='mgr.14152 172.21.15.39:0/3595829223' entity='mgr.x' cmd={"prefix": "config get", "who": "mon", "key": "public_network"} : dispatch 2024-09-21T03:37:19.127 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:37:19 smithi039 ceph-mon[31211]: from='mgr.14152 172.21.15.39:0/3595829223' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:37:19.128 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:37:19 smithi039 ceph-mon[31211]: Reconfiguring daemon mon.b on smithi150 2024-09-21T03:37:19.283 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:37:19 smithi150 ceph-mon[34289]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:37:19.283 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:37:19 smithi150 ceph-mon[34289]: from='mgr.14152 172.21.15.39:0/3595829223' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T03:37:19.283 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:37:19 smithi150 ceph-mon[34289]: from='mgr.14152 172.21.15.39:0/3595829223' entity='mgr.x' 2024-09-21T03:37:19.283 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:37:19 smithi150 ceph-mon[34289]: from='mgr.14152 172.21.15.39:0/3595829223' entity='mgr.x' 2024-09-21T03:37:19.283 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:37:19 smithi150 ceph-mon[34289]: Reconfiguring mon.b (monmap changed)... 2024-09-21T03:37:19.283 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:37:19 smithi150 ceph-mon[34289]: from='mgr.14152 172.21.15.39:0/3595829223' entity='mgr.x' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-09-21T03:37:19.283 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:37:19 smithi150 ceph-mon[34289]: from='mgr.14152 172.21.15.39:0/3595829223' entity='mgr.x' cmd={"prefix": "config get", "who": "mon", "key": "public_network"} : dispatch 2024-09-21T03:37:19.283 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:37:19 smithi150 ceph-mon[34289]: from='mgr.14152 172.21.15.39:0/3595829223' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:37:19.283 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:37:19 smithi150 ceph-mon[34289]: Reconfiguring daemon mon.b on smithi150 2024-09-21T03:37:19.468 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:37:19 smithi161 ceph-mon[34533]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:37:19.468 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:37:19 smithi161 ceph-mon[34533]: from='mgr.14152 172.21.15.39:0/3595829223' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T03:37:19.468 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:37:19 smithi161 ceph-mon[34533]: from='mgr.14152 172.21.15.39:0/3595829223' entity='mgr.x' 2024-09-21T03:37:19.468 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:37:19 smithi161 ceph-mon[34533]: from='mgr.14152 172.21.15.39:0/3595829223' entity='mgr.x' 2024-09-21T03:37:19.468 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:37:19 smithi161 ceph-mon[34533]: Reconfiguring mon.b (monmap changed)... 2024-09-21T03:37:19.468 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:37:19 smithi161 ceph-mon[34533]: from='mgr.14152 172.21.15.39:0/3595829223' entity='mgr.x' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-09-21T03:37:19.468 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:37:19 smithi161 ceph-mon[34533]: from='mgr.14152 172.21.15.39:0/3595829223' entity='mgr.x' cmd={"prefix": "config get", "who": "mon", "key": "public_network"} : dispatch 2024-09-21T03:37:19.468 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:37:19 smithi161 ceph-mon[34533]: from='mgr.14152 172.21.15.39:0/3595829223' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:37:19.468 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:37:19 smithi161 ceph-mon[34533]: Reconfiguring daemon mon.b on smithi150 2024-09-21T03:37:20.162 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:37:20 smithi150 ceph-mon[34289]: from='client.? 172.21.15.39:0/1673749288' entity='client.admin' cmd={"prefix": "mgr module enable", "module": "snap_schedule"} : dispatch 2024-09-21T03:37:20.162 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:37:20 smithi150 ceph-mon[34289]: from='mgr.14152 172.21.15.39:0/3595829223' entity='mgr.x' 2024-09-21T03:37:20.162 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:37:20 smithi150 ceph-mon[34289]: from='mgr.14152 172.21.15.39:0/3595829223' entity='mgr.x' 2024-09-21T03:37:20.163 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:37:20 smithi150 ceph-mon[34289]: from='mgr.14152 172.21.15.39:0/3595829223' entity='mgr.x' 2024-09-21T03:37:20.163 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:37:20 smithi150 ceph-mon[34289]: from='mgr.14152 172.21.15.39:0/3595829223' entity='mgr.x' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-09-21T03:37:20.163 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:37:20 smithi150 ceph-mon[34289]: from='mgr.14152 172.21.15.39:0/3595829223' entity='mgr.x' cmd={"prefix": "config get", "who": "mon", "key": "public_network"} : dispatch 2024-09-21T03:37:20.163 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:37:20 smithi150 ceph-mon[34289]: from='mgr.14152 172.21.15.39:0/3595829223' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:37:20.467 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:37:20 smithi161 ceph-mon[34533]: from='client.? 172.21.15.39:0/1673749288' entity='client.admin' cmd={"prefix": "mgr module enable", "module": "snap_schedule"} : dispatch 2024-09-21T03:37:20.467 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:37:20 smithi161 ceph-mon[34533]: from='mgr.14152 172.21.15.39:0/3595829223' entity='mgr.x' 2024-09-21T03:37:20.468 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:37:20 smithi161 ceph-mon[34533]: from='mgr.14152 172.21.15.39:0/3595829223' entity='mgr.x' 2024-09-21T03:37:20.468 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:37:20 smithi161 ceph-mon[34533]: from='mgr.14152 172.21.15.39:0/3595829223' entity='mgr.x' 2024-09-21T03:37:20.468 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:37:20 smithi161 ceph-mon[34533]: from='mgr.14152 172.21.15.39:0/3595829223' entity='mgr.x' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-09-21T03:37:20.468 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:37:20 smithi161 ceph-mon[34533]: from='mgr.14152 172.21.15.39:0/3595829223' entity='mgr.x' cmd={"prefix": "config get", "who": "mon", "key": "public_network"} : dispatch 2024-09-21T03:37:20.468 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:37:20 smithi161 ceph-mon[34533]: from='mgr.14152 172.21.15.39:0/3595829223' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:37:20.493 INFO:journalctl@ceph.mgr.x.smithi039.stdout:Sep 21 03:37:20 smithi039 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-mgr-x[31438]: ignoring --setuser ceph since I am not root 2024-09-21T03:37:20.493 INFO:journalctl@ceph.mgr.x.smithi039.stdout:Sep 21 03:37:20 smithi039 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-mgr-x[31438]: ignoring --setgroup ceph since I am not root 2024-09-21T03:37:20.493 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:37:20 smithi039 ceph-mon[31211]: from='client.? 172.21.15.39:0/1673749288' entity='client.admin' cmd={"prefix": "mgr module enable", "module": "snap_schedule"} : dispatch 2024-09-21T03:37:20.493 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:37:20 smithi039 ceph-mon[31211]: from='mgr.14152 172.21.15.39:0/3595829223' entity='mgr.x' 2024-09-21T03:37:20.493 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:37:20 smithi039 ceph-mon[31211]: from='mgr.14152 172.21.15.39:0/3595829223' entity='mgr.x' 2024-09-21T03:37:20.493 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:37:20 smithi039 ceph-mon[31211]: from='mgr.14152 172.21.15.39:0/3595829223' entity='mgr.x' 2024-09-21T03:37:20.493 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:37:20 smithi039 ceph-mon[31211]: from='mgr.14152 172.21.15.39:0/3595829223' entity='mgr.x' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-09-21T03:37:20.493 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:37:20 smithi039 ceph-mon[31211]: from='mgr.14152 172.21.15.39:0/3595829223' entity='mgr.x' cmd={"prefix": "config get", "who": "mon", "key": "public_network"} : dispatch 2024-09-21T03:37:20.494 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:37:20 smithi039 ceph-mon[31211]: from='mgr.14152 172.21.15.39:0/3595829223' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:37:20.858 INFO:tasks.cephadm:Adding mgr.x on smithi039 2024-09-21T03:37:20.858 INFO:tasks.cephadm:Adding mgr.y on smithi150 2024-09-21T03:37:20.859 INFO:tasks.cephadm:Adding mgr.z on smithi161 2024-09-21T03:37:20.859 DEBUG:teuthology.orchestra.run.smithi161:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 51116d5c-77ca-11ef-baf5-efdc52797490 -- ceph orch apply mgr '3;smithi039=x;smithi150=y;smithi161=z' 2024-09-21T03:37:21.218 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:37:21 smithi161 ceph-mon[34533]: from='client.? 172.21.15.39:0/1673749288' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2024-09-21T03:37:21.218 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:37:21 smithi161 ceph-mon[34533]: mgrmap e15: x(active, since 56s) 2024-09-21T03:37:21.433 INFO:teuthology.orchestra.run.smithi161.stderr:Inferring config /var/lib/ceph/51116d5c-77ca-11ef-baf5-efdc52797490/mon.c/config 2024-09-21T03:37:21.474 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:37:21 smithi150 ceph-mon[34289]: from='client.? 172.21.15.39:0/1673749288' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2024-09-21T03:37:21.474 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:37:21 smithi150 ceph-mon[34289]: mgrmap e15: x(active, since 56s) 2024-09-21T03:37:21.583 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:37:21 smithi039 ceph-mon[31211]: from='client.? 172.21.15.39:0/1673749288' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2024-09-21T03:37:21.583 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:37:21 smithi039 ceph-mon[31211]: mgrmap e15: x(active, since 56s) 2024-09-21T03:37:25.718 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:37:25 smithi161 ceph-mon[34533]: Active manager daemon x restarted 2024-09-21T03:37:25.718 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:37:25 smithi161 ceph-mon[34533]: Activating manager daemon x 2024-09-21T03:37:25.718 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:37:25 smithi161 ceph-mon[34533]: osdmap e5: 0 total, 0 up, 0 in 2024-09-21T03:37:25.718 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:37:25 smithi161 ceph-mon[34533]: mgrmap e16: x(active, starting, since 0.0068634s) 2024-09-21T03:37:25.718 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:37:25 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-21T03:37:25.718 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:37:25 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T03:37:25.718 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:37:25 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T03:37:25.718 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:37:25 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "mgr metadata", "who": "x", "id": "x"} : dispatch 2024-09-21T03:37:25.718 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:37:25 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "mds metadata"} : dispatch 2024-09-21T03:37:25.718 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:37:25 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "osd metadata"} : dispatch 2024-09-21T03:37:25.718 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:37:25 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "mon metadata"} : dispatch 2024-09-21T03:37:25.719 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:37:25 smithi161 ceph-mon[34533]: Manager daemon x is now available 2024-09-21T03:37:25.724 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:37:25 smithi150 ceph-mon[34289]: Active manager daemon x restarted 2024-09-21T03:37:25.724 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:37:25 smithi150 ceph-mon[34289]: Activating manager daemon x 2024-09-21T03:37:25.725 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:37:25 smithi150 ceph-mon[34289]: osdmap e5: 0 total, 0 up, 0 in 2024-09-21T03:37:25.725 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:37:25 smithi150 ceph-mon[34289]: mgrmap e16: x(active, starting, since 0.0068634s) 2024-09-21T03:37:25.725 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:37:25 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-21T03:37:25.725 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:37:25 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T03:37:25.725 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:37:25 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T03:37:25.725 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:37:25 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "mgr metadata", "who": "x", "id": "x"} : dispatch 2024-09-21T03:37:25.725 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:37:25 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "mds metadata"} : dispatch 2024-09-21T03:37:25.725 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:37:25 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "osd metadata"} : dispatch 2024-09-21T03:37:25.725 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:37:25 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "mon metadata"} : dispatch 2024-09-21T03:37:25.725 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:37:25 smithi150 ceph-mon[34289]: Manager daemon x is now available 2024-09-21T03:37:25.833 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:37:25 smithi039 ceph-mon[31211]: Active manager daemon x restarted 2024-09-21T03:37:25.834 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:37:25 smithi039 ceph-mon[31211]: Activating manager daemon x 2024-09-21T03:37:25.834 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:37:25 smithi039 ceph-mon[31211]: osdmap e5: 0 total, 0 up, 0 in 2024-09-21T03:37:25.834 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:37:25 smithi039 ceph-mon[31211]: mgrmap e16: x(active, starting, since 0.0068634s) 2024-09-21T03:37:25.834 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:37:25 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-21T03:37:25.834 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:37:25 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T03:37:25.834 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:37:25 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T03:37:25.834 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:37:25 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "mgr metadata", "who": "x", "id": "x"} : dispatch 2024-09-21T03:37:25.834 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:37:25 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "mds metadata"} : dispatch 2024-09-21T03:37:25.834 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:37:25 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "osd metadata"} : dispatch 2024-09-21T03:37:25.834 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:37:25 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "mon metadata"} : dispatch 2024-09-21T03:37:25.834 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:37:25 smithi039 ceph-mon[31211]: Manager daemon x is now available 2024-09-21T03:37:26.410 INFO:teuthology.orchestra.run.smithi161.stdout:Scheduled mgr update... 2024-09-21T03:37:26.718 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:37:26 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:37:26.718 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:37:26 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"} : dispatch 2024-09-21T03:37:26.718 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:37:26 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"} : dispatch 2024-09-21T03:37:26.718 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:37:26 smithi161 ceph-mon[34533]: mgrmap e17: x(active, since 1.01s) 2024-09-21T03:37:26.718 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:37:26 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:37:26.724 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:37:26 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:37:26.724 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:37:26 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"} : dispatch 2024-09-21T03:37:26.725 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:37:26 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"} : dispatch 2024-09-21T03:37:26.725 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:37:26 smithi150 ceph-mon[34289]: mgrmap e17: x(active, since 1.01s) 2024-09-21T03:37:26.725 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:37:26 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:37:26.759 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:37:26 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:37:26.759 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:37:26 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"} : dispatch 2024-09-21T03:37:26.759 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:37:26 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"} : dispatch 2024-09-21T03:37:26.759 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:37:26 smithi039 ceph-mon[31211]: mgrmap e17: x(active, since 1.01s) 2024-09-21T03:37:26.759 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:37:26 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:37:27.043 DEBUG:teuthology.orchestra.run.smithi150:mgr.y> sudo journalctl -f -n 0 -u ceph-51116d5c-77ca-11ef-baf5-efdc52797490@mgr.y.service 2024-09-21T03:37:27.045 DEBUG:teuthology.orchestra.run.smithi161:mgr.z> sudo journalctl -f -n 0 -u ceph-51116d5c-77ca-11ef-baf5-efdc52797490@mgr.z.service 2024-09-21T03:37:27.048 INFO:tasks.cephadm:setting config [client] client_mount_timeout = 600 2024-09-21T03:37:27.048 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 51116d5c-77ca-11ef-baf5-efdc52797490 -- ceph config set client client_mount_timeout 600 2024-09-21T03:37:27.050 INFO:tasks.cephadm:setting config [client] debug_client = 20 2024-09-21T03:37:27.050 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 51116d5c-77ca-11ef-baf5-efdc52797490 -- ceph config set client debug_client 20 2024-09-21T03:37:27.093 INFO:tasks.cephadm:setting config [client] debug_ms = 1 2024-09-21T03:37:27.093 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 51116d5c-77ca-11ef-baf5-efdc52797490 -- ceph config set client debug_ms 1 2024-09-21T03:37:27.136 INFO:tasks.cephadm:setting config [client] rados_mon_op_timeout = 900 2024-09-21T03:37:27.136 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 51116d5c-77ca-11ef-baf5-efdc52797490 -- ceph config set client rados_mon_op_timeout 900 2024-09-21T03:37:27.179 INFO:tasks.cephadm:setting config [client] rados_osd_op_timeout = 900 2024-09-21T03:37:27.179 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 51116d5c-77ca-11ef-baf5-efdc52797490 -- ceph config set client rados_osd_op_timeout 900 2024-09-21T03:37:27.222 INFO:tasks.cephadm:setting config [mds] debug_mds = 20 2024-09-21T03:37:27.223 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 51116d5c-77ca-11ef-baf5-efdc52797490 -- ceph config set mds debug_mds 20 2024-09-21T03:37:27.265 INFO:tasks.cephadm:setting config [mds] debug_mds_balancer = 20 2024-09-21T03:37:27.265 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 51116d5c-77ca-11ef-baf5-efdc52797490 -- ceph config set mds debug_mds_balancer 20 2024-09-21T03:37:27.309 INFO:tasks.cephadm:setting config [mds] debug_ms = 2 2024-09-21T03:37:27.309 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 51116d5c-77ca-11ef-baf5-efdc52797490 -- ceph config set mds debug_ms 2 2024-09-21T03:37:27.337 INFO:journalctl@ceph.mgr.x.smithi039.stdout:Sep 21 03:37:27 smithi039 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-mgr-x[31438]: 2024-09-21T03:37:27.229+0000 7fe42164c640 -1 log_channel(cephadm) log [ERR] : [21/Sep/2024:03:37:27] ENGINE Error in HTTPServer.serve 2024-09-21T03:37:27.337 INFO:journalctl@ceph.mgr.x.smithi039.stdout:Sep 21 03:37:27 smithi039 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-mgr-x[31438]: Traceback (most recent call last): 2024-09-21T03:37:27.338 INFO:journalctl@ceph.mgr.x.smithi039.stdout:Sep 21 03:37:27 smithi039 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-mgr-x[31438]: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-21T03:37:27.338 INFO:journalctl@ceph.mgr.x.smithi039.stdout:Sep 21 03:37:27 smithi039 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-mgr-x[31438]: self._connections.run(self.expiration_interval) 2024-09-21T03:37:27.338 INFO:journalctl@ceph.mgr.x.smithi039.stdout:Sep 21 03:37:27 smithi039 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-mgr-x[31438]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-21T03:37:27.338 INFO:journalctl@ceph.mgr.x.smithi039.stdout:Sep 21 03:37:27 smithi039 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-mgr-x[31438]: self._run(expiration_interval) 2024-09-21T03:37:27.338 INFO:journalctl@ceph.mgr.x.smithi039.stdout:Sep 21 03:37:27 smithi039 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-mgr-x[31438]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-21T03:37:27.338 INFO:journalctl@ceph.mgr.x.smithi039.stdout:Sep 21 03:37:27 smithi039 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-mgr-x[31438]: new_conn = self._from_server_socket(self.server.socket) 2024-09-21T03:37:27.338 INFO:journalctl@ceph.mgr.x.smithi039.stdout:Sep 21 03:37:27 smithi039 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-mgr-x[31438]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-21T03:37:27.338 INFO:journalctl@ceph.mgr.x.smithi039.stdout:Sep 21 03:37:27 smithi039 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-mgr-x[31438]: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-21T03:37:27.338 INFO:journalctl@ceph.mgr.x.smithi039.stdout:Sep 21 03:37:27 smithi039 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-mgr-x[31438]: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-21T03:37:27.338 INFO:journalctl@ceph.mgr.x.smithi039.stdout:Sep 21 03:37:27 smithi039 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-mgr-x[31438]: s = self.context.wrap_socket( 2024-09-21T03:37:27.338 INFO:journalctl@ceph.mgr.x.smithi039.stdout:Sep 21 03:37:27 smithi039 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-mgr-x[31438]: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-21T03:37:27.338 INFO:journalctl@ceph.mgr.x.smithi039.stdout:Sep 21 03:37:27 smithi039 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-mgr-x[31438]: return self.sslsocket_class._create( 2024-09-21T03:37:27.338 INFO:journalctl@ceph.mgr.x.smithi039.stdout:Sep 21 03:37:27 smithi039 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-mgr-x[31438]: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-21T03:37:27.338 INFO:journalctl@ceph.mgr.x.smithi039.stdout:Sep 21 03:37:27 smithi039 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-mgr-x[31438]: self.do_handshake() 2024-09-21T03:37:27.339 INFO:journalctl@ceph.mgr.x.smithi039.stdout:Sep 21 03:37:27 smithi039 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-mgr-x[31438]: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-21T03:37:27.339 INFO:journalctl@ceph.mgr.x.smithi039.stdout:Sep 21 03:37:27 smithi039 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-mgr-x[31438]: self._sslobj.do_handshake() 2024-09-21T03:37:27.339 INFO:journalctl@ceph.mgr.x.smithi039.stdout:Sep 21 03:37:27 smithi039 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-mgr-x[31438]: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-21T03:37:27.339 INFO:journalctl@ceph.mgr.x.smithi039.stdout:Sep 21 03:37:27 smithi039 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-mgr-x[31438]: 2024-09-21T03:37:27.339 INFO:tasks.cephadm:setting config [mds] mds_bal_fragment_size_max = 10000 2024-09-21T03:37:27.339 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 51116d5c-77ca-11ef-baf5-efdc52797490 -- ceph config set mds mds_bal_fragment_size_max 10000 2024-09-21T03:37:27.387 INFO:tasks.cephadm:setting config [mds] mds_bal_merge_size = 5 2024-09-21T03:37:27.388 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 51116d5c-77ca-11ef-baf5-efdc52797490 -- ceph config set mds mds_bal_merge_size 5 2024-09-21T03:37:27.433 INFO:tasks.cephadm:setting config [mds] mds_bal_split_bits = 3 2024-09-21T03:37:27.433 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 51116d5c-77ca-11ef-baf5-efdc52797490 -- ceph config set mds mds_bal_split_bits 3 2024-09-21T03:37:27.438 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/51116d5c-77ca-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:37:27.445 INFO:tasks.cephadm:setting config [mds] mds_bal_split_size = 100 2024-09-21T03:37:27.446 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 51116d5c-77ca-11ef-baf5-efdc52797490 -- ceph config set mds mds_bal_split_size 100 2024-09-21T03:37:27.499 INFO:tasks.cephadm:setting config [mds] mds_debug_frag = True 2024-09-21T03:37:27.501 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 51116d5c-77ca-11ef-baf5-efdc52797490 -- ceph config set mds mds_debug_frag True 2024-09-21T03:37:27.509 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/51116d5c-77ca-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:37:27.518 INFO:tasks.cephadm:setting config [mds] mds_debug_scatterstat = True 2024-09-21T03:37:27.519 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 51116d5c-77ca-11ef-baf5-efdc52797490 -- ceph config set mds mds_debug_scatterstat True 2024-09-21T03:37:27.570 INFO:tasks.cephadm:setting config [mds] mds_op_complaint_time = 180 2024-09-21T03:37:27.570 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 51116d5c-77ca-11ef-baf5-efdc52797490 -- ceph config set mds mds_op_complaint_time 180 2024-09-21T03:37:27.624 INFO:tasks.cephadm:setting config [mds] mds_verify_scatter = True 2024-09-21T03:37:27.625 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 51116d5c-77ca-11ef-baf5-efdc52797490 -- ceph config set mds mds_verify_scatter True 2024-09-21T03:37:27.681 INFO:tasks.cephadm:setting config [mds] osd_op_complaint_time = 180 2024-09-21T03:37:27.681 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 51116d5c-77ca-11ef-baf5-efdc52797490 -- ceph config set mds osd_op_complaint_time 180 2024-09-21T03:37:27.727 INFO:tasks.cephadm:setting config [mds] rados_mon_op_timeout = 900 2024-09-21T03:37:27.727 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 51116d5c-77ca-11ef-baf5-efdc52797490 -- ceph config set mds rados_mon_op_timeout 900 2024-09-21T03:37:27.780 INFO:tasks.cephadm:setting config [mds] rados_osd_op_timeout = 900 2024-09-21T03:37:27.781 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 51116d5c-77ca-11ef-baf5-efdc52797490 -- ceph config set mds rados_osd_op_timeout 900 2024-09-21T03:37:27.835 INFO:tasks.cephadm:setting config [mgr] client_mount_timeout = 30 2024-09-21T03:37:27.835 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 51116d5c-77ca-11ef-baf5-efdc52797490 -- ceph config set mgr client_mount_timeout 30 2024-09-21T03:37:27.911 INFO:tasks.cephadm:setting config [mgr] debug_cephsqlite = 20 2024-09-21T03:37:27.911 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 51116d5c-77ca-11ef-baf5-efdc52797490 -- ceph config set mgr debug_cephsqlite 20 2024-09-21T03:37:27.970 INFO:tasks.cephadm:setting config [mgr] debug_client = 20 2024-09-21T03:37:27.970 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 51116d5c-77ca-11ef-baf5-efdc52797490 -- ceph config set mgr debug_client 20 2024-09-21T03:37:28.024 INFO:tasks.cephadm:setting config [mgr] debug_mgr = 20 2024-09-21T03:37:28.024 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 51116d5c-77ca-11ef-baf5-efdc52797490 -- ceph config set mgr debug_mgr 20 2024-09-21T03:37:28.071 INFO:tasks.cephadm:setting config [mgr] debug_ms = 1 2024-09-21T03:37:28.071 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 51116d5c-77ca-11ef-baf5-efdc52797490 -- ceph config set mgr debug_ms 1 2024-09-21T03:37:28.136 INFO:tasks.cephadm:setting config [mgr] mgr/snap_schedule/allow_m_granularity = True 2024-09-21T03:37:28.137 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 51116d5c-77ca-11ef-baf5-efdc52797490 -- ceph config set mgr mgr/snap_schedule/allow_m_granularity True 2024-09-21T03:37:28.185 INFO:tasks.cephadm:setting config [mgr] mgr/snap_schedule/dump_on_update = True 2024-09-21T03:37:28.185 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 51116d5c-77ca-11ef-baf5-efdc52797490 -- ceph config set mgr mgr/snap_schedule/dump_on_update True 2024-09-21T03:37:28.249 INFO:tasks.cephadm:setting config [mgr] mgr/snap_schedule/log_level = debug 2024-09-21T03:37:28.249 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 51116d5c-77ca-11ef-baf5-efdc52797490 -- ceph config set mgr mgr/snap_schedule/log_level debug 2024-09-21T03:37:28.303 INFO:tasks.cephadm:setting config [mgr] mon_warn_on_pool_no_app = False 2024-09-21T03:37:28.303 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 51116d5c-77ca-11ef-baf5-efdc52797490 -- ceph config set mgr mon_warn_on_pool_no_app False 2024-09-21T03:37:28.354 INFO:tasks.cephadm:setting config [mon] mon_op_complaint_time = 120 2024-09-21T03:37:28.355 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 51116d5c-77ca-11ef-baf5-efdc52797490 -- ceph config set mon mon_op_complaint_time 120 2024-09-21T03:37:28.412 INFO:tasks.cephadm:setting config [osd] osd_op_complaint_time = 180 2024-09-21T03:37:28.412 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 51116d5c-77ca-11ef-baf5-efdc52797490 -- ceph config set osd osd_op_complaint_time 180 2024-09-21T03:37:28.413 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:37:28 smithi039 ceph-mon[31211]: Saving service mgr spec with placement smithi039=x;smithi150=y;smithi161=z;count:3 2024-09-21T03:37:28.413 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:37:28 smithi039 ceph-mon[31211]: [21/Sep/2024:03:37:26] ENGINE Bus STARTING 2024-09-21T03:37:28.414 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:37:28 smithi039 ceph-mon[31211]: [21/Sep/2024:03:37:27] ENGINE Serving on http://172.21.15.39:8765 2024-09-21T03:37:28.414 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:37:28 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:37:28.414 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:37:28 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:37:28.414 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:37:28 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:37:28.414 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:37:28 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:37:28.414 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:37:28 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:37:28.414 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:37:28 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:37:28.415 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:37:28 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:37:28.415 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:37:28 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:37:28.415 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:37:28 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd/host:smithi150", "name": "osd_memory_target"} : dispatch 2024-09-21T03:37:28.466 DEBUG:tasks.cephadm:set 30 configs 2024-09-21T03:37:28.466 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '51116d5c-77ca-11ef-baf5-efdc52797490', '--', 'ceph', 'config', 'set', 'client', 'client_mount_timeout', '600'], check_status=True, hostname='smithi039') 2024-09-21T03:37:28.467 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:37:28 smithi161 ceph-mon[34533]: Saving service mgr spec with placement smithi039=x;smithi150=y;smithi161=z;count:3 2024-09-21T03:37:28.468 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:37:28 smithi161 ceph-mon[34533]: [21/Sep/2024:03:37:26] ENGINE Bus STARTING 2024-09-21T03:37:28.468 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:37:28 smithi161 ceph-mon[34533]: [21/Sep/2024:03:37:27] ENGINE Serving on http://172.21.15.39:8765 2024-09-21T03:37:28.468 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:37:28 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:37:28.468 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:37:28 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:37:28.468 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:37:28 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:37:28.468 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:37:28 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:37:28.468 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:37:28 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:37:28.469 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:37:28 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:37:28.469 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:37:28 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:37:28.469 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:37:28 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:37:28.469 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:37:28 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd/host:smithi150", "name": "osd_memory_target"} : dispatch 2024-09-21T03:37:28.474 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:37:28 smithi150 ceph-mon[34289]: Saving service mgr spec with placement smithi039=x;smithi150=y;smithi161=z;count:3 2024-09-21T03:37:28.474 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:37:28 smithi150 ceph-mon[34289]: [21/Sep/2024:03:37:26] ENGINE Bus STARTING 2024-09-21T03:37:28.475 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:37:28 smithi150 ceph-mon[34289]: [21/Sep/2024:03:37:27] ENGINE Serving on http://172.21.15.39:8765 2024-09-21T03:37:28.475 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:37:28 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:37:28.475 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:37:28 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:37:28.475 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:37:28 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:37:28.475 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:37:28 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:37:28.475 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:37:28 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:37:28.475 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:37:28 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:37:28.475 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:37:28 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:37:28.475 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:37:28 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:37:28.476 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:37:28 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd/host:smithi150", "name": "osd_memory_target"} : dispatch 2024-09-21T03:37:29.331 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:37:29 smithi039 ceph-mon[31211]: [21/Sep/2024:03:37:27] ENGINE Serving on https://172.21.15.39:7150 2024-09-21T03:37:29.332 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:37:29 smithi039 ceph-mon[31211]: [21/Sep/2024:03:37:27] ENGINE Bus STARTED 2024-09-21T03:37:29.332 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:37:29 smithi039 ceph-mon[31211]: [21/Sep/2024:03:37:27] ENGINE Error in HTTPServer.serve 2024-09-21T03:37:29.332 INFO:journalctl@ceph.mon.a.smithi039.stdout: Traceback (most recent call last): 2024-09-21T03:37:29.332 INFO:journalctl@ceph.mon.a.smithi039.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-21T03:37:29.332 INFO:journalctl@ceph.mon.a.smithi039.stdout: self._connections.run(self.expiration_interval) 2024-09-21T03:37:29.332 INFO:journalctl@ceph.mon.a.smithi039.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-21T03:37:29.332 INFO:journalctl@ceph.mon.a.smithi039.stdout: self._run(expiration_interval) 2024-09-21T03:37:29.332 INFO:journalctl@ceph.mon.a.smithi039.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-21T03:37:29.332 INFO:journalctl@ceph.mon.a.smithi039.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-09-21T03:37:29.332 INFO:journalctl@ceph.mon.a.smithi039.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-21T03:37:29.332 INFO:journalctl@ceph.mon.a.smithi039.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-21T03:37:29.332 INFO:journalctl@ceph.mon.a.smithi039.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-21T03:37:29.333 INFO:journalctl@ceph.mon.a.smithi039.stdout: s = self.context.wrap_socket( 2024-09-21T03:37:29.333 INFO:journalctl@ceph.mon.a.smithi039.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-21T03:37:29.333 INFO:journalctl@ceph.mon.a.smithi039.stdout: return self.sslsocket_class._create( 2024-09-21T03:37:29.333 INFO:journalctl@ceph.mon.a.smithi039.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-21T03:37:29.333 INFO:journalctl@ceph.mon.a.smithi039.stdout: self.do_handshake() 2024-09-21T03:37:29.333 INFO:journalctl@ceph.mon.a.smithi039.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-21T03:37:29.333 INFO:journalctl@ceph.mon.a.smithi039.stdout: self._sslobj.do_handshake() 2024-09-21T03:37:29.333 INFO:journalctl@ceph.mon.a.smithi039.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-21T03:37:29.333 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:37:29 smithi039 ceph-mon[31211]: mgrmap e18: x(active, since 2s) 2024-09-21T03:37:29.333 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:37:29 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:37:29.333 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:37:29 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:37:29.333 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:37:29 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd/host:smithi161", "name": "osd_memory_target"} : dispatch 2024-09-21T03:37:29.333 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:37:29 smithi039 ceph-mon[31211]: from='client.? 172.21.15.39:0/1022558908' entity='client.admin' 2024-09-21T03:37:29.333 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:37:29 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:37:29.333 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:37:29 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:37:29.334 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:37:29 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd/host:smithi039", "name": "osd_memory_target"} : dispatch 2024-09-21T03:37:29.334 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:37:29 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:37:29.334 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:37:29 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:37:29.468 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:37:29 smithi161 ceph-mon[34533]: [21/Sep/2024:03:37:27] ENGINE Serving on https://172.21.15.39:7150 2024-09-21T03:37:29.468 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:37:29 smithi161 ceph-mon[34533]: [21/Sep/2024:03:37:27] ENGINE Bus STARTED 2024-09-21T03:37:29.468 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:37:29 smithi161 ceph-mon[34533]: [21/Sep/2024:03:37:27] ENGINE Error in HTTPServer.serve 2024-09-21T03:37:29.468 INFO:journalctl@ceph.mon.c.smithi161.stdout: Traceback (most recent call last): 2024-09-21T03:37:29.468 INFO:journalctl@ceph.mon.c.smithi161.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-21T03:37:29.468 INFO:journalctl@ceph.mon.c.smithi161.stdout: self._connections.run(self.expiration_interval) 2024-09-21T03:37:29.468 INFO:journalctl@ceph.mon.c.smithi161.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-21T03:37:29.468 INFO:journalctl@ceph.mon.c.smithi161.stdout: self._run(expiration_interval) 2024-09-21T03:37:29.468 INFO:journalctl@ceph.mon.c.smithi161.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-21T03:37:29.469 INFO:journalctl@ceph.mon.c.smithi161.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-09-21T03:37:29.469 INFO:journalctl@ceph.mon.c.smithi161.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-21T03:37:29.469 INFO:journalctl@ceph.mon.c.smithi161.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-21T03:37:29.469 INFO:journalctl@ceph.mon.c.smithi161.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-21T03:37:29.469 INFO:journalctl@ceph.mon.c.smithi161.stdout: s = self.context.wrap_socket( 2024-09-21T03:37:29.469 INFO:journalctl@ceph.mon.c.smithi161.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-21T03:37:29.469 INFO:journalctl@ceph.mon.c.smithi161.stdout: return self.sslsocket_class._create( 2024-09-21T03:37:29.469 INFO:journalctl@ceph.mon.c.smithi161.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-21T03:37:29.469 INFO:journalctl@ceph.mon.c.smithi161.stdout: self.do_handshake() 2024-09-21T03:37:29.469 INFO:journalctl@ceph.mon.c.smithi161.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-21T03:37:29.469 INFO:journalctl@ceph.mon.c.smithi161.stdout: self._sslobj.do_handshake() 2024-09-21T03:37:29.469 INFO:journalctl@ceph.mon.c.smithi161.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-21T03:37:29.469 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:37:29 smithi161 ceph-mon[34533]: mgrmap e18: x(active, since 2s) 2024-09-21T03:37:29.470 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:37:29 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:37:29.470 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:37:29 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:37:29.470 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:37:29 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd/host:smithi161", "name": "osd_memory_target"} : dispatch 2024-09-21T03:37:29.470 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:37:29 smithi161 ceph-mon[34533]: from='client.? 172.21.15.39:0/1022558908' entity='client.admin' 2024-09-21T03:37:29.470 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:37:29 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:37:29.470 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:37:29 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:37:29.470 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:37:29 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd/host:smithi039", "name": "osd_memory_target"} : dispatch 2024-09-21T03:37:29.470 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:37:29 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:37:29.470 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:37:29 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:37:29.474 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:37:29 smithi150 ceph-mon[34289]: [21/Sep/2024:03:37:27] ENGINE Serving on https://172.21.15.39:7150 2024-09-21T03:37:29.474 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:37:29 smithi150 ceph-mon[34289]: [21/Sep/2024:03:37:27] ENGINE Bus STARTED 2024-09-21T03:37:29.474 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:37:29 smithi150 ceph-mon[34289]: [21/Sep/2024:03:37:27] ENGINE Error in HTTPServer.serve 2024-09-21T03:37:29.475 INFO:journalctl@ceph.mon.b.smithi150.stdout: Traceback (most recent call last): 2024-09-21T03:37:29.475 INFO:journalctl@ceph.mon.b.smithi150.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-21T03:37:29.475 INFO:journalctl@ceph.mon.b.smithi150.stdout: self._connections.run(self.expiration_interval) 2024-09-21T03:37:29.475 INFO:journalctl@ceph.mon.b.smithi150.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-21T03:37:29.475 INFO:journalctl@ceph.mon.b.smithi150.stdout: self._run(expiration_interval) 2024-09-21T03:37:29.475 INFO:journalctl@ceph.mon.b.smithi150.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-21T03:37:29.475 INFO:journalctl@ceph.mon.b.smithi150.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-09-21T03:37:29.475 INFO:journalctl@ceph.mon.b.smithi150.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-21T03:37:29.475 INFO:journalctl@ceph.mon.b.smithi150.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-21T03:37:29.475 INFO:journalctl@ceph.mon.b.smithi150.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-21T03:37:29.475 INFO:journalctl@ceph.mon.b.smithi150.stdout: s = self.context.wrap_socket( 2024-09-21T03:37:29.475 INFO:journalctl@ceph.mon.b.smithi150.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-21T03:37:29.475 INFO:journalctl@ceph.mon.b.smithi150.stdout: return self.sslsocket_class._create( 2024-09-21T03:37:29.475 INFO:journalctl@ceph.mon.b.smithi150.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-21T03:37:29.475 INFO:journalctl@ceph.mon.b.smithi150.stdout: self.do_handshake() 2024-09-21T03:37:29.476 INFO:journalctl@ceph.mon.b.smithi150.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-21T03:37:29.476 INFO:journalctl@ceph.mon.b.smithi150.stdout: self._sslobj.do_handshake() 2024-09-21T03:37:29.476 INFO:journalctl@ceph.mon.b.smithi150.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-21T03:37:29.476 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:37:29 smithi150 ceph-mon[34289]: mgrmap e18: x(active, since 2s) 2024-09-21T03:37:29.476 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:37:29 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:37:29.476 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:37:29 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:37:29.476 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:37:29 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd/host:smithi161", "name": "osd_memory_target"} : dispatch 2024-09-21T03:37:29.476 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:37:29 smithi150 ceph-mon[34289]: from='client.? 172.21.15.39:0/1022558908' entity='client.admin' 2024-09-21T03:37:29.476 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:37:29 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:37:29.476 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:37:29 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:37:29.476 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:37:29 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd/host:smithi039", "name": "osd_memory_target"} : dispatch 2024-09-21T03:37:29.476 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:37:29 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:37:29.476 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:37:29 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:37:30.334 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:37:30 smithi039 ceph-mon[31211]: Updating smithi039:/etc/ceph/ceph.conf 2024-09-21T03:37:30.334 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:37:30 smithi039 ceph-mon[31211]: Updating smithi150:/etc/ceph/ceph.conf 2024-09-21T03:37:30.334 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:37:30 smithi039 ceph-mon[31211]: Updating smithi161:/etc/ceph/ceph.conf 2024-09-21T03:37:30.334 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:37:30 smithi039 ceph-mon[31211]: from='client.? 172.21.15.39:0/3263596123' entity='client.admin' 2024-09-21T03:37:30.334 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:37:30 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:37:30.334 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:37:30 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:37:30.334 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:37:30 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:37:30.334 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:37:30 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:37:30.334 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:37:30 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:37:30.335 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:37:30 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:37:30.335 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:37:30 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:37:30.335 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:37:30 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]} : dispatch 2024-09-21T03:37:30.335 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:37:30 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-21T03:37:30.335 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:37:30 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "mgr services"} : dispatch 2024-09-21T03:37:30.335 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:37:30 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:37:30.389 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '51116d5c-77ca-11ef-baf5-efdc52797490', '--', 'ceph', 'config', 'set', 'client', 'debug_client', '20'], check_status=True, hostname='smithi039') 2024-09-21T03:37:30.389 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '51116d5c-77ca-11ef-baf5-efdc52797490', '--', 'ceph', 'config', 'set', 'client', 'debug_ms', '1'], check_status=True, hostname='smithi039') 2024-09-21T03:37:30.418 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:37:30 smithi161 ceph-mon[34533]: Updating smithi039:/etc/ceph/ceph.conf 2024-09-21T03:37:30.419 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:37:30 smithi161 ceph-mon[34533]: Updating smithi150:/etc/ceph/ceph.conf 2024-09-21T03:37:30.419 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:37:30 smithi161 ceph-mon[34533]: Updating smithi161:/etc/ceph/ceph.conf 2024-09-21T03:37:30.419 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:37:30 smithi161 ceph-mon[34533]: from='client.? 172.21.15.39:0/3263596123' entity='client.admin' 2024-09-21T03:37:30.419 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:37:30 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:37:30.419 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:37:30 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:37:30.419 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:37:30 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:37:30.419 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:37:30 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:37:30.419 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:37:30 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:37:30.419 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:37:30 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:37:30.419 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:37:30 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:37:30.419 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:37:30 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]} : dispatch 2024-09-21T03:37:30.419 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:37:30 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-21T03:37:30.419 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:37:30 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "mgr services"} : dispatch 2024-09-21T03:37:30.420 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:37:30 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:37:30.473 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/51116d5c-77ca-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:37:30.474 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:37:30 smithi150 ceph-mon[34289]: Updating smithi039:/etc/ceph/ceph.conf 2024-09-21T03:37:30.475 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:37:30 smithi150 ceph-mon[34289]: Updating smithi150:/etc/ceph/ceph.conf 2024-09-21T03:37:30.475 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:37:30 smithi150 ceph-mon[34289]: Updating smithi161:/etc/ceph/ceph.conf 2024-09-21T03:37:30.475 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:37:30 smithi150 ceph-mon[34289]: from='client.? 172.21.15.39:0/3263596123' entity='client.admin' 2024-09-21T03:37:30.475 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:37:30 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:37:30.475 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:37:30 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:37:30.475 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:37:30 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:37:30.475 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:37:30 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:37:30.475 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:37:30 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:37:30.475 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:37:30 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:37:30.475 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:37:30 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:37:30.475 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:37:30 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]} : dispatch 2024-09-21T03:37:30.475 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:37:30 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-21T03:37:30.475 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:37:30 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "mgr services"} : dispatch 2024-09-21T03:37:30.476 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:37:30 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:37:30.476 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/51116d5c-77ca-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:37:30.502 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/51116d5c-77ca-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:37:31.302 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:37:31 smithi039 ceph-mon[31211]: Updating smithi039:/var/lib/ceph/51116d5c-77ca-11ef-baf5-efdc52797490/config/ceph.conf 2024-09-21T03:37:31.303 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:37:31 smithi039 ceph-mon[31211]: Updating smithi161:/var/lib/ceph/51116d5c-77ca-11ef-baf5-efdc52797490/config/ceph.conf 2024-09-21T03:37:31.303 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:37:31 smithi039 ceph-mon[31211]: Updating smithi150:/var/lib/ceph/51116d5c-77ca-11ef-baf5-efdc52797490/config/ceph.conf 2024-09-21T03:37:31.303 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:37:31 smithi039 ceph-mon[31211]: Updating smithi039:/etc/ceph/ceph.client.admin.keyring 2024-09-21T03:37:31.303 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:37:31 smithi039 ceph-mon[31211]: Updating smithi161:/etc/ceph/ceph.client.admin.keyring 2024-09-21T03:37:31.303 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:37:31 smithi039 ceph-mon[31211]: Updating smithi150:/etc/ceph/ceph.client.admin.keyring 2024-09-21T03:37:31.303 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:37:31 smithi039 ceph-mon[31211]: Updating smithi039:/var/lib/ceph/51116d5c-77ca-11ef-baf5-efdc52797490/config/ceph.client.admin.keyring 2024-09-21T03:37:31.303 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:37:31 smithi039 ceph-mon[31211]: Updating smithi161:/var/lib/ceph/51116d5c-77ca-11ef-baf5-efdc52797490/config/ceph.client.admin.keyring 2024-09-21T03:37:31.303 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:37:31 smithi039 ceph-mon[31211]: Updating smithi150:/var/lib/ceph/51116d5c-77ca-11ef-baf5-efdc52797490/config/ceph.client.admin.keyring 2024-09-21T03:37:31.303 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:37:31 smithi039 ceph-mon[31211]: Deploying daemon mgr.z on smithi161 2024-09-21T03:37:31.375 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:31.373+0000 7f82cab2a640 1 -- 172.21.15.39:0/3867846079 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f82b403fba0 msgr2=0x7f82b40420b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:37:31.375 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:31.373+0000 7f82cab2a640 1 --2- 172.21.15.39:0/3867846079 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f82b403fba0 0x7f82b40420b0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f82b8005e10 tx=0x7f82b8005da0 comp rx=0 tx=0).stop 2024-09-21T03:37:31.375 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:31.373+0000 7f82cab2a640 1 -- 172.21.15.39:0/3867846079 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f82c4108d20 msgr2=0x7f82c419e9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:37:31.375 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:31.373+0000 7f82cab2a640 1 --2- 172.21.15.39:0/3867846079 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f82c4108d20 0x7f82c419e9d0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f82bc003ba0 tx=0x7f82bc003bd0 comp rx=0 tx=0).stop 2024-09-21T03:37:31.375 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:31.373+0000 7f82cab2a640 1 -- 172.21.15.39:0/3867846079 shutdown_connections 2024-09-21T03:37:31.375 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:31.373+0000 7f82cab2a640 1 --2- 172.21.15.39:0/3867846079 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f82b403fba0 0x7f82b40420b0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:31.375 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:31.373+0000 7f82cab2a640 1 --2- 172.21.15.39:0/3867846079 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f82c4108d20 0x7f82c419e9d0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:31.376 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:31.373+0000 7f82cab2a640 1 --2- 172.21.15.39:0/3867846079 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f82c41083d0 0x7f82c4199730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:31.376 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:31.373+0000 7f82cab2a640 1 --2- 172.21.15.39:0/3867846079 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f82c4102ae0 0x7f82c4199190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:31.376 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:31.373+0000 7f82cab2a640 1 -- 172.21.15.39:0/3867846079 >> 172.21.15.39:0/3867846079 conn(0x7f82c40797b0 msgr2=0x7f82c41065c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:37:31.376 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:31.373+0000 7f82cab2a640 1 -- 172.21.15.39:0/3867846079 shutdown_connections 2024-09-21T03:37:31.376 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:31.373+0000 7f82cab2a640 1 -- 172.21.15.39:0/3867846079 wait complete. 2024-09-21T03:37:31.468 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:37:31 smithi161 ceph-mon[34533]: Updating smithi039:/var/lib/ceph/51116d5c-77ca-11ef-baf5-efdc52797490/config/ceph.conf 2024-09-21T03:37:31.468 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:37:31 smithi161 ceph-mon[34533]: Updating smithi161:/var/lib/ceph/51116d5c-77ca-11ef-baf5-efdc52797490/config/ceph.conf 2024-09-21T03:37:31.468 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:37:31 smithi161 ceph-mon[34533]: Updating smithi150:/var/lib/ceph/51116d5c-77ca-11ef-baf5-efdc52797490/config/ceph.conf 2024-09-21T03:37:31.468 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:37:31 smithi161 ceph-mon[34533]: Updating smithi039:/etc/ceph/ceph.client.admin.keyring 2024-09-21T03:37:31.468 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:37:31 smithi161 ceph-mon[34533]: Updating smithi161:/etc/ceph/ceph.client.admin.keyring 2024-09-21T03:37:31.468 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:37:31 smithi161 ceph-mon[34533]: Updating smithi150:/etc/ceph/ceph.client.admin.keyring 2024-09-21T03:37:31.468 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:37:31 smithi161 ceph-mon[34533]: Updating smithi039:/var/lib/ceph/51116d5c-77ca-11ef-baf5-efdc52797490/config/ceph.client.admin.keyring 2024-09-21T03:37:31.468 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:37:31 smithi161 ceph-mon[34533]: Updating smithi161:/var/lib/ceph/51116d5c-77ca-11ef-baf5-efdc52797490/config/ceph.client.admin.keyring 2024-09-21T03:37:31.468 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:37:31 smithi161 ceph-mon[34533]: Updating smithi150:/var/lib/ceph/51116d5c-77ca-11ef-baf5-efdc52797490/config/ceph.client.admin.keyring 2024-09-21T03:37:31.468 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:37:31 smithi161 ceph-mon[34533]: Deploying daemon mgr.z on smithi161 2024-09-21T03:37:31.474 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:37:31 smithi150 ceph-mon[34289]: Updating smithi039:/var/lib/ceph/51116d5c-77ca-11ef-baf5-efdc52797490/config/ceph.conf 2024-09-21T03:37:31.475 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:37:31 smithi150 ceph-mon[34289]: Updating smithi161:/var/lib/ceph/51116d5c-77ca-11ef-baf5-efdc52797490/config/ceph.conf 2024-09-21T03:37:31.475 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:37:31 smithi150 ceph-mon[34289]: Updating smithi150:/var/lib/ceph/51116d5c-77ca-11ef-baf5-efdc52797490/config/ceph.conf 2024-09-21T03:37:31.475 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:37:31 smithi150 ceph-mon[34289]: Updating smithi039:/etc/ceph/ceph.client.admin.keyring 2024-09-21T03:37:31.475 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:37:31 smithi150 ceph-mon[34289]: Updating smithi161:/etc/ceph/ceph.client.admin.keyring 2024-09-21T03:37:31.475 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:37:31 smithi150 ceph-mon[34289]: Updating smithi150:/etc/ceph/ceph.client.admin.keyring 2024-09-21T03:37:31.475 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:37:31 smithi150 ceph-mon[34289]: Updating smithi039:/var/lib/ceph/51116d5c-77ca-11ef-baf5-efdc52797490/config/ceph.client.admin.keyring 2024-09-21T03:37:31.475 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:37:31 smithi150 ceph-mon[34289]: Updating smithi161:/var/lib/ceph/51116d5c-77ca-11ef-baf5-efdc52797490/config/ceph.client.admin.keyring 2024-09-21T03:37:31.475 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:37:31 smithi150 ceph-mon[34289]: Updating smithi150:/var/lib/ceph/51116d5c-77ca-11ef-baf5-efdc52797490/config/ceph.client.admin.keyring 2024-09-21T03:37:31.475 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:37:31 smithi150 ceph-mon[34289]: Deploying daemon mgr.z on smithi161 2024-09-21T03:37:31.761 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:31.759+0000 7f5582e59640 1 -- 172.21.15.39:0/3495492830 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f557c108cf0 msgr2=0x7f557c10f520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:37:31.762 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:31.759+0000 7f5582e59640 1 --2- 172.21.15.39:0/3495492830 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f557c108cf0 0x7f557c10f520 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f5574009900 tx=0x7f557402a410 comp rx=0 tx=0).stop 2024-09-21T03:37:31.762 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:31.760+0000 7f5582e59640 1 -- 172.21.15.39:0/3495492830 shutdown_connections 2024-09-21T03:37:31.762 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:31.760+0000 7f5582e59640 1 --2- 172.21.15.39:0/3495492830 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f557c108cf0 0x7f557c10f520 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:31.762 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:31.760+0000 7f5582e59640 1 --2- 172.21.15.39:0/3495492830 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f557c1083a0 0x7f557c1087b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:31.762 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:31.760+0000 7f5582e59640 1 --2- 172.21.15.39:0/3495492830 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f557c102a60 0x7f557c102e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:31.762 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:31.760+0000 7f5582e59640 1 -- 172.21.15.39:0/3495492830 >> 172.21.15.39:0/3495492830 conn(0x7f557c079730 msgr2=0x7f557c079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:37:31.762 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:31.760+0000 7f5582e59640 1 -- 172.21.15.39:0/3495492830 shutdown_connections 2024-09-21T03:37:31.762 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:31.760+0000 7f5582e59640 1 -- 172.21.15.39:0/3495492830 wait complete. 2024-09-21T03:37:31.763 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:31.760+0000 7f5582e59640 1 Processor -- start 2024-09-21T03:37:31.764 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:31.760+0000 7f5582e59640 1 -- start start 2024-09-21T03:37:31.764 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:31.761+0000 7f5582e59640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f557c102a60 0x7f557c19a320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:37:31.764 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:31.761+0000 7f5580bf5640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f557c102a60 0x7f557c19a320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:37:31.764 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:31.761+0000 7f5582e59640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f557c1083a0 0x7f557c19a8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:37:31.764 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:31.761+0000 7f5580bf5640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f557c102a60 0x7f557c19a320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:49748/0 (socket says 172.21.15.39:49748) 2024-09-21T03:37:31.764 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:31.761+0000 7f5582e59640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f557c108cf0 0x7f557c19fb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:37:31.764 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:31.761+0000 7f5582e59640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f557c112720 con 0x7f557c102a60 2024-09-21T03:37:31.764 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:31.761+0000 7f5582e59640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f557c1125a0 con 0x7f557c108cf0 2024-09-21T03:37:31.764 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:31.761+0000 7f5582e59640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f557c1128a0 con 0x7f557c1083a0 2024-09-21T03:37:31.764 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:31.761+0000 7f5580bf5640 1 -- 172.21.15.39:0/2046322898 learned_addr learned my addr 172.21.15.39:0/2046322898 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:37:31.764 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:31.761+0000 7f557bfff640 1 --2- 172.21.15.39:0/2046322898 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f557c1083a0 0x7f557c19a8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:37:31.764 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:31.761+0000 7f55813f6640 1 --2- 172.21.15.39:0/2046322898 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f557c108cf0 0x7f557c19fb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:37:31.764 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:31.762+0000 7f5580bf5640 1 -- 172.21.15.39:0/2046322898 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f557c1083a0 msgr2=0x7f557c19a8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:37:31.765 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:31.762+0000 7f5580bf5640 1 --2- 172.21.15.39:0/2046322898 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f557c1083a0 0x7f557c19a8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:31.765 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:31.762+0000 7f5580bf5640 1 -- 172.21.15.39:0/2046322898 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f557c108cf0 msgr2=0x7f557c19fb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:37:31.765 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:31.762+0000 7f5580bf5640 1 --2- 172.21.15.39:0/2046322898 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f557c108cf0 0x7f557c19fb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:31.765 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:31.762+0000 7f5580bf5640 1 -- 172.21.15.39:0/2046322898 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f557c1a0290 con 0x7f557c102a60 2024-09-21T03:37:31.765 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:31.762+0000 7f557bfff640 1 --2- 172.21.15.39:0/2046322898 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f557c1083a0 0x7f557c19a8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:37:31.765 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:31.762+0000 7f5580bf5640 1 --2- 172.21.15.39:0/2046322898 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f557c102a60 0x7f557c19a320 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f556400d950 tx=0x7f556400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:37:31.765 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:31.762+0000 7f5579ffb640 1 -- 172.21.15.39:0/2046322898 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5564017030 con 0x7f557c102a60 2024-09-21T03:37:31.765 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:31.762+0000 7f5579ffb640 1 -- 172.21.15.39:0/2046322898 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(24 keys) ==== 1039+0+0 (secure 0 0 0) 0x7f5564004020 con 0x7f557c102a60 2024-09-21T03:37:31.765 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:31.762+0000 7f5582e59640 1 -- 172.21.15.39:0/2046322898 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f557c1a0580 con 0x7f557c102a60 2024-09-21T03:37:31.766 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:31.762+0000 7f5579ffb640 1 -- 172.21.15.39:0/2046322898 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5564005580 con 0x7f557c102a60 2024-09-21T03:37:31.766 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:31.763+0000 7f5582e59640 1 -- 172.21.15.39:0/2046322898 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f557c1a6ce0 con 0x7f557c102a60 2024-09-21T03:37:31.766 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:31.763+0000 7f5579ffb640 1 -- 172.21.15.39:0/2046322898 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 18) ==== 51915+0+0 (secure 0 0 0) 0x7f55640047a0 con 0x7f557c102a60 2024-09-21T03:37:31.766 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:31.764+0000 7f5582e59640 1 -- 172.21.15.39:0/2046322898 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f557c102e70 con 0x7f557c102a60 2024-09-21T03:37:31.767 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:31.764+0000 7f5579ffb640 1 --2- 172.21.15.39:0/2046322898 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f556803fb70 0x7f5568042080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:37:31.767 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:31.764+0000 7f5579ffb640 1 -- 172.21.15.39:0/2046322898 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1370+0+0 (secure 0 0 0) 0x7f5564053fc0 con 0x7f557c102a60 2024-09-21T03:37:31.767 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:31.764+0000 7f557bfff640 1 --2- 172.21.15.39:0/2046322898 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f556803fb70 0x7f5568042080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:37:31.767 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:31.764+0000 7f557bfff640 1 --2- 172.21.15.39:0/2046322898 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f556803fb70 0x7f5568042080 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f556c005c40 tx=0x7f556c005bd0 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:37:31.770 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:31.769+0000 7f5579ffb640 1 -- 172.21.15.39:0/2046322898 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f556401ad40 con 0x7f557c102a60 2024-09-21T03:37:31.961 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:31.959+0000 7f7f1f9ba640 1 -- 172.21.15.39:0/3292429375 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7f181083c0 msgr2=0x7f7f181087d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:37:31.961 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:31.959+0000 7f7f1f9ba640 1 --2- 172.21.15.39:0/3292429375 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7f181083c0 0x7f7f181087d0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f7f08009900 tx=0x7f7f0802a410 comp rx=0 tx=0).stop 2024-09-21T03:37:31.961 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:31.959+0000 7f7f1f9ba640 1 -- 172.21.15.39:0/3292429375 shutdown_connections 2024-09-21T03:37:31.961 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:31.959+0000 7f7f1f9ba640 1 --2- 172.21.15.39:0/3292429375 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7f18108d10 0x7f7f1810f540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:31.961 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:31.959+0000 7f7f1f9ba640 1 --2- 172.21.15.39:0/3292429375 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7f181083c0 0x7f7f181087d0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:31.961 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:31.959+0000 7f7f1f9ba640 1 --2- 172.21.15.39:0/3292429375 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7f18102ad0 0x7f7f18102ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:31.961 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:31.959+0000 7f7f1f9ba640 1 -- 172.21.15.39:0/3292429375 >> 172.21.15.39:0/3292429375 conn(0x7f7f180796d0 msgr2=0x7f7f18079ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:37:31.961 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:31.959+0000 7f7f1f9ba640 1 -- 172.21.15.39:0/3292429375 shutdown_connections 2024-09-21T03:37:31.962 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:31.959+0000 7f7f1f9ba640 1 -- 172.21.15.39:0/3292429375 wait complete. 2024-09-21T03:37:31.963 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:31.960+0000 7f7f1f9ba640 1 Processor -- start 2024-09-21T03:37:31.964 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:31.960+0000 7f7f1f9ba640 1 -- start start 2024-09-21T03:37:31.964 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:31.961+0000 7f7f1f9ba640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7f18102ad0 0x7f7f1819b100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:37:31.964 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:31.961+0000 7f7f1f9ba640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7f181083c0 0x7f7f1819b640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:37:31.964 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:31.961+0000 7f7f1f9ba640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7f18108d10 0x7f7f1819bb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:37:31.964 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:31.961+0000 7f7f1f9ba640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f7f18112790 con 0x7f7f181083c0 2024-09-21T03:37:31.964 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:31.961+0000 7f7f1f9ba640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f7f18112610 con 0x7f7f18102ad0 2024-09-21T03:37:31.964 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:31.961+0000 7f7f1f9ba640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f7f18112910 con 0x7f7f18108d10 2024-09-21T03:37:31.964 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:31.961+0000 7f7f1d756640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7f18102ad0 0x7f7f1819b100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:37:31.964 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:31.961+0000 7f7f1df57640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7f18108d10 0x7f7f1819bb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:37:31.964 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:31.961+0000 7f7f1df57640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7f18108d10 0x7f7f1819bb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.39:33312/0 (socket says 172.21.15.39:33312) 2024-09-21T03:37:31.964 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:31.961+0000 7f7f1d756640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7f18102ad0 0x7f7f1819b100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.39:56764/0 (socket says 172.21.15.39:56764) 2024-09-21T03:37:31.964 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:31.961+0000 7f7f1df57640 1 -- 172.21.15.39:0/2411178255 learned_addr learned my addr 172.21.15.39:0/2411178255 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:37:31.964 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:31.961+0000 7f7f1cf55640 1 --2- 172.21.15.39:0/2411178255 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7f181083c0 0x7f7f1819b640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:37:31.965 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:31.961+0000 7f7f1df57640 1 -- 172.21.15.39:0/2411178255 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7f18102ad0 msgr2=0x7f7f1819b100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:37:31.965 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:31.961+0000 7f7f1df57640 1 --2- 172.21.15.39:0/2411178255 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7f18102ad0 0x7f7f1819b100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:31.965 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:31.961+0000 7f7f1df57640 1 -- 172.21.15.39:0/2411178255 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7f181083c0 msgr2=0x7f7f1819b640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:37:31.965 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:31.961+0000 7f7f1df57640 1 --2- 172.21.15.39:0/2411178255 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7f181083c0 0x7f7f1819b640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:31.965 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:31.961+0000 7f7f1df57640 1 -- 172.21.15.39:0/2411178255 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7f181a6410 con 0x7f7f18108d10 2024-09-21T03:37:31.965 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:31.961+0000 7f7f1d756640 1 --2- 172.21.15.39:0/2411178255 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7f18102ad0 0x7f7f1819b100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:37:31.965 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:31.962+0000 7f7f1df57640 1 --2- 172.21.15.39:0/2411178255 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7f18108d10 0x7f7f1819bb80 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f7f1000eb10 tx=0x7f7f1000c2d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:37:31.965 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:31.962+0000 7f7f0e7fc640 1 -- 172.21.15.39:0/2411178255 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f10004b20 con 0x7f7f18108d10 2024-09-21T03:37:31.965 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:31.962+0000 7f7f0e7fc640 1 -- 172.21.15.39:0/2411178255 <== mon.1 v2:172.21.15.161:3300/0 2 ==== config(24 keys) ==== 1039+0+0 (secure 0 0 0) 0x7f7f10004cc0 con 0x7f7f18108d10 2024-09-21T03:37:31.965 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:31.962+0000 7f7f1f9ba640 1 -- 172.21.15.39:0/2411178255 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7f181a6700 con 0x7f7f18108d10 2024-09-21T03:37:31.965 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:31.962+0000 7f7f0e7fc640 1 -- 172.21.15.39:0/2411178255 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f10010690 con 0x7f7f18108d10 2024-09-21T03:37:31.965 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:31.962+0000 7f7f1f9ba640 1 -- 172.21.15.39:0/2411178255 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7f181a6c10 con 0x7f7f18108d10 2024-09-21T03:37:31.966 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:31.963+0000 7f7f1f9ba640 1 -- 172.21.15.39:0/2411178255 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7f18103d00 con 0x7f7f18108d10 2024-09-21T03:37:31.966 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:31.963+0000 7f7f0e7fc640 1 -- 172.21.15.39:0/2411178255 <== mon.1 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 18) ==== 51915+0+0 (secure 0 0 0) 0x7f7f10010830 con 0x7f7f18108d10 2024-09-21T03:37:31.966 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:31.963+0000 7f7f0e7fc640 1 --2- 172.21.15.39:0/2411178255 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f7f0403fb70 0x7f7f04042080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:37:31.966 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:31.963+0000 7f7f0e7fc640 1 -- 172.21.15.39:0/2411178255 <== mon.1 v2:172.21.15.161:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1370+0+0 (secure 0 0 0) 0x7f7f10054be0 con 0x7f7f18108d10 2024-09-21T03:37:31.967 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:31.963+0000 7f7f1d756640 1 --2- 172.21.15.39:0/2411178255 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f7f0403fb70 0x7f7f04042080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:37:31.967 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:31.964+0000 7f7f1d756640 1 --2- 172.21.15.39:0/2411178255 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f7f0403fb70 0x7f7f04042080 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f7f1806c700 tx=0x7f7f14005f90 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:37:31.967 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:31.965+0000 7f5582e59640 1 -- 172.21.15.39:0/2046322898 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command([{prefix=config set, name=rados_osd_op_timeout}] v 0) -- 0x7f557c1a6f10 con 0x7f557c102a60 2024-09-21T03:37:31.970 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:31.969+0000 7f7f0e7fc640 1 -- 172.21.15.39:0/2411178255 <== mon.1 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f7f100059f0 con 0x7f7f18108d10 2024-09-21T03:37:31.972 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:31.971+0000 7f5579ffb640 1 -- 172.21.15.39:0/2046322898 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{prefix=config set, name=rados_osd_op_timeout}]=0 v13)=0 v13) ==== 123+0+0 (secure 0 0 0) 0x7f5564003b10 con 0x7f557c102a60 2024-09-21T03:37:31.973 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:31.972+0000 7f7f0e7fc640 1 -- 172.21.15.39:0/2411178255 <== mon.1 v2:172.21.15.161:3300/0 7 ==== config(25 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f7f1005a080 con 0x7f7f18108d10 2024-09-21T03:37:31.974 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:31.972+0000 7f5579ffb640 1 -- 172.21.15.39:0/2046322898 <== mon.0 v2:172.21.15.39:3300/0 8 ==== config(25 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f55640041d0 con 0x7f557c102a60 2024-09-21T03:37:31.976 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:31.974+0000 7f5582e59640 1 -- 172.21.15.39:0/2046322898 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f556803fb70 msgr2=0x7f5568042080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:37:31.976 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:31.974+0000 7f5582e59640 1 --2- 172.21.15.39:0/2046322898 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f556803fb70 0x7f5568042080 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f556c005c40 tx=0x7f556c005bd0 comp rx=0 tx=0).stop 2024-09-21T03:37:31.976 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:31.974+0000 7f5582e59640 1 -- 172.21.15.39:0/2046322898 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f557c102a60 msgr2=0x7f557c19a320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:37:31.976 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:31.974+0000 7f5582e59640 1 --2- 172.21.15.39:0/2046322898 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f557c102a60 0x7f557c19a320 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f556400d950 tx=0x7f556400de10 comp rx=0 tx=0).stop 2024-09-21T03:37:31.976 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:31.974+0000 7f5582e59640 1 -- 172.21.15.39:0/2046322898 shutdown_connections 2024-09-21T03:37:31.976 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:31.974+0000 7f5582e59640 1 --2- 172.21.15.39:0/2046322898 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f556803fb70 0x7f5568042080 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:31.976 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:31.974+0000 7f5582e59640 1 --2- 172.21.15.39:0/2046322898 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f557c108cf0 0x7f557c19fb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:31.976 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:31.974+0000 7f5582e59640 1 --2- 172.21.15.39:0/2046322898 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f557c1083a0 0x7f557c19a8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:31.976 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:31.974+0000 7f5582e59640 1 --2- 172.21.15.39:0/2046322898 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f557c102a60 0x7f557c19a320 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:31.976 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:31.974+0000 7f5582e59640 1 -- 172.21.15.39:0/2046322898 >> 172.21.15.39:0/2046322898 conn(0x7f557c079730 msgr2=0x7f557c101c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:37:31.976 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:31.974+0000 7f5582e59640 1 -- 172.21.15.39:0/2046322898 shutdown_connections 2024-09-21T03:37:31.976 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:31.974+0000 7f5582e59640 1 -- 172.21.15.39:0/2046322898 wait complete. 2024-09-21T03:37:32.165 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:32.164+0000 7f7f1f9ba640 1 -- 172.21.15.39:0/2411178255 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command([{prefix=config set, name=rados_mon_op_timeout}] v 0) -- 0x7f7f1810ab00 con 0x7f7f18108d10 2024-09-21T03:37:32.170 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:32.169+0000 7f7f0e7fc640 1 -- 172.21.15.39:0/2411178255 <== mon.1 v2:172.21.15.161:3300/0 8 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7f1001f6e0 con 0x7f7f18108d10 2024-09-21T03:37:32.171 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:32.169+0000 7f7f0e7fc640 1 -- 172.21.15.39:0/2411178255 <== mon.1 v2:172.21.15.161:3300/0 9 ==== mon_command_ack([{prefix=config set, name=rados_mon_op_timeout}]=0 v14)=0 v14) ==== 123+0+0 (secure 0 0 0) 0x7f7f10019d50 con 0x7f7f18108d10 2024-09-21T03:37:32.173 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:32.171+0000 7f7f1f9ba640 1 -- 172.21.15.39:0/2411178255 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f7f0403fb70 msgr2=0x7f7f04042080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:37:32.173 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:32.171+0000 7f7f1f9ba640 1 --2- 172.21.15.39:0/2411178255 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f7f0403fb70 0x7f7f04042080 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f7f1806c700 tx=0x7f7f14005f90 comp rx=0 tx=0).stop 2024-09-21T03:37:32.173 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:32.171+0000 7f7f1f9ba640 1 -- 172.21.15.39:0/2411178255 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7f18108d10 msgr2=0x7f7f1819bb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:37:32.173 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:32.171+0000 7f7f1f9ba640 1 --2- 172.21.15.39:0/2411178255 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7f18108d10 0x7f7f1819bb80 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f7f1000eb10 tx=0x7f7f1000c2d0 comp rx=0 tx=0).stop 2024-09-21T03:37:32.173 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:32.171+0000 7f7f1f9ba640 1 -- 172.21.15.39:0/2411178255 shutdown_connections 2024-09-21T03:37:32.173 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:32.171+0000 7f7f1f9ba640 1 --2- 172.21.15.39:0/2411178255 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f7f0403fb70 0x7f7f04042080 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:32.173 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:32.171+0000 7f7f1f9ba640 1 --2- 172.21.15.39:0/2411178255 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7f18108d10 0x7f7f1819bb80 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:32.174 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:32.171+0000 7f7f1f9ba640 1 --2- 172.21.15.39:0/2411178255 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7f181083c0 0x7f7f1819b640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:32.174 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:32.171+0000 7f7f1f9ba640 1 --2- 172.21.15.39:0/2411178255 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7f18102ad0 0x7f7f1819b100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:32.174 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:32.171+0000 7f7f1f9ba640 1 -- 172.21.15.39:0/2411178255 >> 172.21.15.39:0/2411178255 conn(0x7f7f180796d0 msgr2=0x7f7f181065b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:37:32.174 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:32.171+0000 7f7f1f9ba640 1 -- 172.21.15.39:0/2411178255 shutdown_connections 2024-09-21T03:37:32.174 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:32.171+0000 7f7f1f9ba640 1 -- 172.21.15.39:0/2411178255 wait complete. 2024-09-21T03:37:32.217 INFO:journalctl@ceph.mgr.z.smithi161.stdout:Sep 21 03:37:31 smithi161 systemd[1]: Started Ceph mgr.z for 51116d5c-77ca-11ef-baf5-efdc52797490. 2024-09-21T03:37:32.420 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/51116d5c-77ca-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:37:32.420 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/51116d5c-77ca-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:37:32.433 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/51116d5c-77ca-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:37:32.433 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/51116d5c-77ca-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:37:32.433 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/51116d5c-77ca-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:37:32.434 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/51116d5c-77ca-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:37:32.434 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/51116d5c-77ca-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:37:32.434 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/51116d5c-77ca-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:37:32.434 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/51116d5c-77ca-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:37:32.435 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/51116d5c-77ca-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:37:32.435 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/51116d5c-77ca-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:37:32.436 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/51116d5c-77ca-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:37:32.437 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/51116d5c-77ca-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:37:32.540 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:37:32 smithi039 ceph-mon[31211]: from='client.? 172.21.15.39:0/3867846079' entity='client.admin' 2024-09-21T03:37:32.540 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:37:32 smithi039 ceph-mon[31211]: from='client.? 172.21.15.39:0/2046322898' entity='client.admin' 2024-09-21T03:37:32.540 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:37:32 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:37:32.540 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:37:32 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:37:32.540 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:37:32 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:37:32.540 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:37:32 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]} : dispatch 2024-09-21T03:37:32.540 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:37:32 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-21T03:37:32.540 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:37:32 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "mgr services"} : dispatch 2024-09-21T03:37:32.541 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:37:32 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:37:32.541 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:37:32 smithi039 ceph-mon[31211]: Deploying daemon mgr.y on smithi150 2024-09-21T03:37:32.541 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:37:32 smithi039 ceph-mon[31211]: from='client.? ' entity='client.admin' 2024-09-21T03:37:32.692 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:37:32 smithi150 ceph-mon[34289]: from='client.? 172.21.15.39:0/3867846079' entity='client.admin' 2024-09-21T03:37:32.692 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:37:32 smithi150 ceph-mon[34289]: from='client.? 172.21.15.39:0/2046322898' entity='client.admin' 2024-09-21T03:37:32.692 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:37:32 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:37:32.692 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:37:32 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:37:32.692 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:37:32 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:37:32.693 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:37:32 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]} : dispatch 2024-09-21T03:37:32.693 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:37:32 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-21T03:37:32.693 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:37:32 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "mgr services"} : dispatch 2024-09-21T03:37:32.693 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:37:32 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:37:32.693 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:37:32 smithi150 ceph-mon[34289]: Deploying daemon mgr.y on smithi150 2024-09-21T03:37:32.693 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:37:32 smithi150 ceph-mon[34289]: from='client.? ' entity='client.admin' 2024-09-21T03:37:32.717 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:37:32 smithi161 ceph-mon[34533]: from='client.? 172.21.15.39:0/3867846079' entity='client.admin' 2024-09-21T03:37:32.718 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:37:32 smithi161 ceph-mon[34533]: from='client.? 172.21.15.39:0/2046322898' entity='client.admin' 2024-09-21T03:37:32.718 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:37:32 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:37:32.718 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:37:32 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:37:32.718 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:37:32 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:37:32.718 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:37:32 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]} : dispatch 2024-09-21T03:37:32.718 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:37:32 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-21T03:37:32.718 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:37:32 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "mgr services"} : dispatch 2024-09-21T03:37:32.718 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:37:32 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:37:32.718 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:37:32 smithi161 ceph-mon[34533]: Deploying daemon mgr.y on smithi150 2024-09-21T03:37:32.718 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:37:32 smithi161 ceph-mon[34533]: from='client.? ' entity='client.admin' 2024-09-21T03:37:32.864 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/51116d5c-77ca-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:37:32.865 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/51116d5c-77ca-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:37:32.865 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/51116d5c-77ca-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:37:32.865 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/51116d5c-77ca-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:37:32.868 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/51116d5c-77ca-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:37:32.869 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/51116d5c-77ca-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:37:32.873 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/51116d5c-77ca-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:37:32.876 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '51116d5c-77ca-11ef-baf5-efdc52797490', '--', 'ceph', 'config', 'set', 'client', 'rados_mon_op_timeout', '900'], check_status=True, hostname='smithi039') 2024-09-21T03:37:32.890 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/51116d5c-77ca-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:37:33.918 INFO:journalctl@ceph.mgr.y.smithi150.stdout:Sep 21 03:37:33 smithi150 systemd[1]: Starting Ceph mgr.y for 51116d5c-77ca-11ef-baf5-efdc52797490... 2024-09-21T03:37:34.213 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:34.208+0000 7fdfcf9b7640 1 -- 172.21.15.39:0/2822018191 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdfc8108cf0 msgr2=0x7fdfc810f520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:37:34.213 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:34.208+0000 7fdfcf9b7640 1 --2- 172.21.15.39:0/2822018191 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdfc8108cf0 0x7fdfc810f520 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fdfb8009ca0 tx=0x7fdfb802a8c0 comp rx=0 tx=0).stop 2024-09-21T03:37:34.213 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:34.208+0000 7fdfcf9b7640 1 -- 172.21.15.39:0/2822018191 shutdown_connections 2024-09-21T03:37:34.213 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:34.208+0000 7fdfcf9b7640 1 --2- 172.21.15.39:0/2822018191 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdfc8108cf0 0x7fdfc810f520 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:34.213 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:34.208+0000 7fdfcf9b7640 1 --2- 172.21.15.39:0/2822018191 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdfc81083a0 0x7fdfc81087b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:34.213 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:34.208+0000 7fdfcf9b7640 1 --2- 172.21.15.39:0/2822018191 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdfc8102a60 0x7fdfc8102e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:34.213 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:34.208+0000 7fdfcf9b7640 1 -- 172.21.15.39:0/2822018191 >> 172.21.15.39:0/2822018191 conn(0x7fdfc8079730 msgr2=0x7fdfc8079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:37:34.213 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:34.208+0000 7fdfcf9b7640 1 -- 172.21.15.39:0/2822018191 shutdown_connections 2024-09-21T03:37:34.213 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:34.208+0000 7fdfcf9b7640 1 -- 172.21.15.39:0/2822018191 wait complete. 2024-09-21T03:37:34.213 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:34.209+0000 7fdfcf9b7640 1 Processor -- start 2024-09-21T03:37:34.214 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:34.209+0000 7fdfcf9b7640 1 -- start start 2024-09-21T03:37:34.214 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:34.209+0000 7fdfcf9b7640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdfc8102a60 0x7fdfc80ff670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:37:34.214 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:34.209+0000 7fdfcf9b7640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdfc81083a0 0x7fdfc80ffbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:37:34.214 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:34.209+0000 7fdfcf9b7640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdfc8108cf0 0x7fdfc81000f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:37:34.214 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:34.209+0000 7fdfcf9b7640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fdfc81129b0 con 0x7fdfc8108cf0 2024-09-21T03:37:34.214 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:34.209+0000 7fdfcf9b7640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fdfc8112830 con 0x7fdfc81083a0 2024-09-21T03:37:34.214 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:34.209+0000 7fdfcf9b7640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fdfc8112b30 con 0x7fdfc8102a60 2024-09-21T03:37:34.214 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:34.209+0000 7fdfcdf54640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdfc8108cf0 0x7fdfc81000f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:37:34.214 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:34.209+0000 7fdfcd753640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdfc8102a60 0x7fdfc80ff670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:37:34.214 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:34.209+0000 7fdfcdf54640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdfc8108cf0 0x7fdfc81000f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:49770/0 (socket says 172.21.15.39:49770) 2024-09-21T03:37:34.214 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:34.209+0000 7fdfcd753640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdfc8102a60 0x7fdfc80ff670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.39:33340/0 (socket says 172.21.15.39:33340) 2024-09-21T03:37:34.214 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:34.209+0000 7fdfcdf54640 1 -- 172.21.15.39:0/3067419537 learned_addr learned my addr 172.21.15.39:0/3067419537 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:37:34.214 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:34.209+0000 7fdfccf52640 1 --2- 172.21.15.39:0/3067419537 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdfc81083a0 0x7fdfc80ffbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:37:34.215 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:34.210+0000 7fdfcdf54640 1 -- 172.21.15.39:0/3067419537 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdfc8102a60 msgr2=0x7fdfc80ff670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:37:34.215 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:34.210+0000 7fdfcdf54640 1 --2- 172.21.15.39:0/3067419537 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdfc8102a60 0x7fdfc80ff670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:34.215 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:34.210+0000 7fdfcdf54640 1 -- 172.21.15.39:0/3067419537 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdfc81083a0 msgr2=0x7fdfc80ffbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:37:34.215 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:34.210+0000 7fdfcdf54640 1 --2- 172.21.15.39:0/3067419537 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdfc81083a0 0x7fdfc80ffbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:34.215 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:34.210+0000 7fdfcdf54640 1 -- 172.21.15.39:0/3067419537 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdfc81a86f0 con 0x7fdfc8108cf0 2024-09-21T03:37:34.215 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:34.210+0000 7fdfcdf54640 1 --2- 172.21.15.39:0/3067419537 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdfc8108cf0 0x7fdfc81000f0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fdfb8009430 tx=0x7fdfb80319f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:37:34.215 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:34.210+0000 7fdfc67fc640 1 -- 172.21.15.39:0/3067419537 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfb8003c30 con 0x7fdfc8108cf0 2024-09-21T03:37:34.215 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:34.210+0000 7fdfc67fc640 1 -- 172.21.15.39:0/3067419537 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdfb8003dd0 con 0x7fdfc8108cf0 2024-09-21T03:37:34.215 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:34.210+0000 7fdfcf9b7640 1 -- 172.21.15.39:0/3067419537 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdfc81a8980 con 0x7fdfc8108cf0 2024-09-21T03:37:34.215 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:34.210+0000 7fdfc67fc640 1 -- 172.21.15.39:0/3067419537 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfb802faa0 con 0x7fdfc8108cf0 2024-09-21T03:37:34.215 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:34.210+0000 7fdfcf9b7640 1 -- 172.21.15.39:0/3067419537 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdfc81a8ec0 con 0x7fdfc8108cf0 2024-09-21T03:37:34.216 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:34.211+0000 7fdfc67fc640 1 -- 172.21.15.39:0/3067419537 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 18) ==== 51915+0+0 (secure 0 0 0) 0x7fdfb802fc40 con 0x7fdfc8108cf0 2024-09-21T03:37:34.216 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:34.211+0000 7fdfcf9b7640 1 -- 172.21.15.39:0/3067419537 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdfc8102e70 con 0x7fdfc8108cf0 2024-09-21T03:37:34.216 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:34.211+0000 7fdfc67fc640 1 --2- 172.21.15.39:0/3067419537 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7fdfb003fb70 0x7fdfb0042080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:37:34.216 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:34.211+0000 7fdfc67fc640 1 -- 172.21.15.39:0/3067419537 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1370+0+0 (secure 0 0 0) 0x7fdfb8075a10 con 0x7fdfc8108cf0 2024-09-21T03:37:34.217 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:34.212+0000 7fdfcd753640 1 --2- 172.21.15.39:0/3067419537 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7fdfb003fb70 0x7fdfb0042080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:37:34.217 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:34.212+0000 7fdfcd753640 1 --2- 172.21.15.39:0/3067419537 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7fdfb003fb70 0x7fdfb0042080 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fdfc81010c0 tx=0x7fdfc000b390 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:37:34.219 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:34.216+0000 7fdfc67fc640 1 -- 172.21.15.39:0/3067419537 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fdfb804e4a0 con 0x7fdfc8108cf0 2024-09-21T03:37:34.224 INFO:journalctl@ceph.mgr.y.smithi150.stdout:Sep 21 03:37:33 smithi150 podman[35793]: 2024-09-21 03:37:33.818756376 +0000 UTC m=+0.016279662 image pull ddce307e1f3263fb077b4f3902e15660050631dad4d2823a20ee3f72bf18d977 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262 2024-09-21T03:37:34.224 INFO:journalctl@ceph.mgr.y.smithi150.stdout:Sep 21 03:37:33 smithi150 podman[35793]: 2024-09-21 03:37:33.930029735 +0000 UTC m=+0.127553027 container create 5ebc4c7778bc6649a379adde9140ae07418d4fa84e63658822d1ed1bba8a4b9b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-51116d5c-77ca-11ef-baf5-efdc52797490-mgr-y, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, ceph=True, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=) 2024-09-21T03:37:34.224 INFO:journalctl@ceph.mgr.y.smithi150.stdout:Sep 21 03:37:34 smithi150 podman[35793]: 2024-09-21 03:37:34.123694968 +0000 UTC m=+0.321218264 container init 5ebc4c7778bc6649a379adde9140ae07418d4fa84e63658822d1ed1bba8a4b9b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-51116d5c-77ca-11ef-baf5-efdc52797490-mgr-y, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, ceph=True, GIT_BRANCH=HEAD) 2024-09-21T03:37:34.225 INFO:journalctl@ceph.mgr.y.smithi150.stdout:Sep 21 03:37:34 smithi150 podman[35793]: 2024-09-21 03:37:34.126511995 +0000 UTC m=+0.324035289 container start 5ebc4c7778bc6649a379adde9140ae07418d4fa84e63658822d1ed1bba8a4b9b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-51116d5c-77ca-11ef-baf5-efdc52797490-mgr-y, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, ceph=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.build-date=20240731, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS) 2024-09-21T03:37:34.225 INFO:journalctl@ceph.mgr.y.smithi150.stdout:Sep 21 03:37:34 smithi150 ceph-mgr[35812]: -- 172.21.15.150:0/2468164914 <== mon.2 v2:172.21.15.150:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55ca1e80d0e0 con 0x55ca235b0480 2024-09-21T03:37:34.414 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:34.411+0000 7fdfcf9b7640 1 -- 172.21.15.39:0/3067419537 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command([{prefix=config set, name=osd_op_complaint_time}] v 0) -- 0x7fdfc802e640 con 0x7fdfc8108cf0 2024-09-21T03:37:34.417 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:34.415+0000 7fdfc67fc640 1 -- 172.21.15.39:0/3067419537 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{prefix=config set, name=osd_op_complaint_time}]=0 v15)=0 v15) ==== 121+0+0 (secure 0 0 0) 0x7fdfb8041d40 con 0x7fdfc8108cf0 2024-09-21T03:37:34.420 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:34.417+0000 7fdfcf9b7640 1 -- 172.21.15.39:0/3067419537 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7fdfb003fb70 msgr2=0x7fdfb0042080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:37:34.420 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:34.417+0000 7fdfcf9b7640 1 --2- 172.21.15.39:0/3067419537 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7fdfb003fb70 0x7fdfb0042080 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fdfc81010c0 tx=0x7fdfc000b390 comp rx=0 tx=0).stop 2024-09-21T03:37:34.420 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:34.417+0000 7fdfcf9b7640 1 -- 172.21.15.39:0/3067419537 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdfc8108cf0 msgr2=0x7fdfc81000f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:37:34.420 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:34.417+0000 7fdfcf9b7640 1 --2- 172.21.15.39:0/3067419537 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdfc8108cf0 0x7fdfc81000f0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fdfb8009430 tx=0x7fdfb80319f0 comp rx=0 tx=0).stop 2024-09-21T03:37:34.420 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:34.417+0000 7fdfcf9b7640 1 -- 172.21.15.39:0/3067419537 shutdown_connections 2024-09-21T03:37:34.420 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:34.417+0000 7fdfcf9b7640 1 --2- 172.21.15.39:0/3067419537 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7fdfb003fb70 0x7fdfb0042080 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:34.420 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:34.417+0000 7fdfcf9b7640 1 --2- 172.21.15.39:0/3067419537 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdfc8108cf0 0x7fdfc81000f0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:34.420 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:34.417+0000 7fdfcf9b7640 1 --2- 172.21.15.39:0/3067419537 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdfc81083a0 0x7fdfc80ffbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:34.420 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:34.417+0000 7fdfcf9b7640 1 --2- 172.21.15.39:0/3067419537 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdfc8102a60 0x7fdfc80ff670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:34.420 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:34.417+0000 7fdfcf9b7640 1 -- 172.21.15.39:0/3067419537 >> 172.21.15.39:0/3067419537 conn(0x7fdfc8079730 msgr2=0x7fdfc806b140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:37:34.420 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:34.417+0000 7fdfcf9b7640 1 -- 172.21.15.39:0/3067419537 shutdown_connections 2024-09-21T03:37:34.421 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:34.417+0000 7fdfcf9b7640 1 -- 172.21.15.39:0/3067419537 wait complete. 2024-09-21T03:37:34.724 INFO:journalctl@ceph.mgr.y.smithi150.stdout:Sep 21 03:37:34 smithi150 bash[35793]: 5ebc4c7778bc6649a379adde9140ae07418d4fa84e63658822d1ed1bba8a4b9b 2024-09-21T03:37:34.724 INFO:journalctl@ceph.mgr.y.smithi150.stdout:Sep 21 03:37:34 smithi150 systemd[1]: Started Ceph mgr.y for 51116d5c-77ca-11ef-baf5-efdc52797490. 2024-09-21T03:37:34.894 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:34.891+0000 7f5b291d2640 1 -- 172.21.15.39:0/3716622671 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5b24105310 msgr2=0x7f5b2410d860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:37:34.894 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:34.891+0000 7f5b291d2640 1 --2- 172.21.15.39:0/3716622671 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5b24105310 0x7f5b2410d860 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f5b18009900 tx=0x7f5b1802a410 comp rx=0 tx=0).stop 2024-09-21T03:37:34.894 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:34.891+0000 7f5b291d2640 1 -- 172.21.15.39:0/3716622671 shutdown_connections 2024-09-21T03:37:34.894 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:34.891+0000 7f5b291d2640 1 --2- 172.21.15.39:0/3716622671 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5b2410de20 0x7f5b24110310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:34.894 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:34.891+0000 7f5b291d2640 1 --2- 172.21.15.39:0/3716622671 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5b24105310 0x7f5b2410d860 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:34.894 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:34.891+0000 7f5b291d2640 1 --2- 172.21.15.39:0/3716622671 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5b241049c0 0x7f5b24104dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:34.894 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:34.891+0000 7f5b291d2640 1 -- 172.21.15.39:0/3716622671 >> 172.21.15.39:0/3716622671 conn(0x7f5b24079710 msgr2=0x7f5b24079b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:37:34.895 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:34.892+0000 7f5b291d2640 1 -- 172.21.15.39:0/3716622671 shutdown_connections 2024-09-21T03:37:34.895 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:34.892+0000 7f5b291d2640 1 -- 172.21.15.39:0/3716622671 wait complete. 2024-09-21T03:37:34.895 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:34.892+0000 7f5b291d2640 1 Processor -- start 2024-09-21T03:37:34.896 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:34.893+0000 7f5b291d2640 1 -- start start 2024-09-21T03:37:34.896 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:34.893+0000 7f5b291d2640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5b241049c0 0x7f5b2419a4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:37:34.896 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:34.893+0000 7f5b291d2640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5b24105310 0x7f5b2419aa90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:37:34.896 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:34.893+0000 7f5b291d2640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5b2410de20 0x7f5b2419fd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:37:34.896 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:34.893+0000 7f5b291d2640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f5b24113ac0 con 0x7f5b2410de20 2024-09-21T03:37:34.896 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:34.893+0000 7f5b291d2640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f5b24113940 con 0x7f5b24105310 2024-09-21T03:37:34.896 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:34.893+0000 7f5b291d2640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f5b24113c40 con 0x7f5b241049c0 2024-09-21T03:37:34.896 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:34.893+0000 7f5b2359e640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5b2410de20 0x7f5b2419fd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:37:34.896 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:34.893+0000 7f5b22d9d640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5b241049c0 0x7f5b2419a4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:37:34.896 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:34.893+0000 7f5b2259c640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5b24105310 0x7f5b2419aa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:37:34.896 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:34.893+0000 7f5b22d9d640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5b241049c0 0x7f5b2419a4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.39:33358/0 (socket says 172.21.15.39:33358) 2024-09-21T03:37:34.896 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:34.893+0000 7f5b2359e640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5b2410de20 0x7f5b2419fd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:49798/0 (socket says 172.21.15.39:49798) 2024-09-21T03:37:34.896 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:34.893+0000 7f5b22d9d640 1 -- 172.21.15.39:0/3717414697 learned_addr learned my addr 172.21.15.39:0/3717414697 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:37:34.897 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:34.893+0000 7f5b22d9d640 1 -- 172.21.15.39:0/3717414697 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5b24105310 msgr2=0x7f5b2419aa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:37:34.897 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:34.893+0000 7f5b22d9d640 1 --2- 172.21.15.39:0/3717414697 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5b24105310 0x7f5b2419aa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:34.897 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:34.893+0000 7f5b22d9d640 1 -- 172.21.15.39:0/3717414697 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5b2410de20 msgr2=0x7f5b2419fd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:37:34.897 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:34.893+0000 7f5b22d9d640 1 --2- 172.21.15.39:0/3717414697 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5b2410de20 0x7f5b2419fd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:34.898 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:34.894+0000 7f5b22d9d640 1 -- 172.21.15.39:0/3717414697 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5b241a0460 con 0x7f5b241049c0 2024-09-21T03:37:34.898 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:34.894+0000 7f5b22d9d640 1 --2- 172.21.15.39:0/3717414697 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5b241049c0 0x7f5b2419a4f0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f5b1400b4a0 tx=0x7f5b1400b960 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:37:34.898 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:34.894+0000 7f5b0bfff640 1 -- 172.21.15.39:0/3717414697 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b14014070 con 0x7f5b241049c0 2024-09-21T03:37:34.898 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:34.894+0000 7f5b0bfff640 1 -- 172.21.15.39:0/3717414697 <== mon.1 v2:172.21.15.161:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5b14010ac0 con 0x7f5b241049c0 2024-09-21T03:37:34.898 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:34.894+0000 7f5b291d2640 1 -- 172.21.15.39:0/3717414697 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5b241a0750 con 0x7f5b241049c0 2024-09-21T03:37:34.898 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:34.894+0000 7f5b0bfff640 1 -- 172.21.15.39:0/3717414697 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b14017c30 con 0x7f5b241049c0 2024-09-21T03:37:34.898 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:34.894+0000 7f5b291d2640 1 -- 172.21.15.39:0/3717414697 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5b241a6eb0 con 0x7f5b241049c0 2024-09-21T03:37:34.899 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:34.895+0000 7f5b291d2640 1 -- 172.21.15.39:0/3717414697 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5b2406c280 con 0x7f5b241049c0 2024-09-21T03:37:34.899 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:34.895+0000 7f5b0bfff640 1 -- 172.21.15.39:0/3717414697 <== mon.1 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 18) ==== 51915+0+0 (secure 0 0 0) 0x7f5b14017070 con 0x7f5b241049c0 2024-09-21T03:37:34.899 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:34.896+0000 7f5b0bfff640 1 --2- 172.21.15.39:0/3717414697 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f5b0c03fb70 0x7f5b0c042080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:37:34.899 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:34.896+0000 7f5b0bfff640 1 -- 172.21.15.39:0/3717414697 <== mon.1 v2:172.21.15.161:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1370+0+0 (secure 0 0 0) 0x7f5b14053b70 con 0x7f5b241049c0 2024-09-21T03:37:34.899 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:34.896+0000 7f5b2259c640 1 --2- 172.21.15.39:0/3717414697 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f5b0c03fb70 0x7f5b0c042080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:37:34.899 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:34.896+0000 7f5b2259c640 1 --2- 172.21.15.39:0/3717414697 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f5b0c03fb70 0x7f5b0c042080 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f5b18009230 tx=0x7f5b18031400 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:37:34.903 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:34.901+0000 7f5b0bfff640 1 -- 172.21.15.39:0/3717414697 <== mon.1 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f5b1400c230 con 0x7f5b241049c0 2024-09-21T03:37:35.094 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:35.092+0000 7f5b291d2640 1 -- 172.21.15.39:0/3717414697 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command([{prefix=config set, name=debug_mds_balancer}] v 0) -- 0x7f5b24108990 con 0x7f5b241049c0 2024-09-21T03:37:35.100 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:35.097+0000 7f5b0bfff640 1 -- 172.21.15.39:0/3717414697 <== mon.1 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{prefix=config set, name=debug_mds_balancer}]=0 v16)=0 v16) ==== 117+0+0 (secure 0 0 0) 0x7f5b14017dd0 con 0x7f5b241049c0 2024-09-21T03:37:35.102 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:35.099+0000 7f5b291d2640 1 -- 172.21.15.39:0/3717414697 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f5b0c03fb70 msgr2=0x7f5b0c042080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:37:35.103 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:35.099+0000 7f5b291d2640 1 --2- 172.21.15.39:0/3717414697 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f5b0c03fb70 0x7f5b0c042080 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f5b18009230 tx=0x7f5b18031400 comp rx=0 tx=0).stop 2024-09-21T03:37:35.103 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:35.100+0000 7f5b291d2640 1 -- 172.21.15.39:0/3717414697 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5b241049c0 msgr2=0x7f5b2419a4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:37:35.103 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:35.100+0000 7f5b291d2640 1 --2- 172.21.15.39:0/3717414697 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5b241049c0 0x7f5b2419a4f0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f5b1400b4a0 tx=0x7f5b1400b960 comp rx=0 tx=0).stop 2024-09-21T03:37:35.103 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:35.100+0000 7f5b291d2640 1 -- 172.21.15.39:0/3717414697 shutdown_connections 2024-09-21T03:37:35.103 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:35.100+0000 7f5b291d2640 1 --2- 172.21.15.39:0/3717414697 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f5b0c03fb70 0x7f5b0c042080 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:35.103 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:35.100+0000 7f5b291d2640 1 --2- 172.21.15.39:0/3717414697 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5b2410de20 0x7f5b2419fd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:35.103 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:35.100+0000 7f5b291d2640 1 --2- 172.21.15.39:0/3717414697 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5b24105310 0x7f5b2419aa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:35.103 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:35.100+0000 7f5b291d2640 1 --2- 172.21.15.39:0/3717414697 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5b241049c0 0x7f5b2419a4f0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:35.103 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:35.100+0000 7f5b291d2640 1 -- 172.21.15.39:0/3717414697 >> 172.21.15.39:0/3717414697 conn(0x7f5b24079710 msgr2=0x7f5b24100830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:37:35.103 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:35.100+0000 7f5b291d2640 1 -- 172.21.15.39:0/3717414697 shutdown_connections 2024-09-21T03:37:35.103 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:35.100+0000 7f5b291d2640 1 -- 172.21.15.39:0/3717414697 wait complete. 2024-09-21T03:37:35.583 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:37:35 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:37:35.584 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:37:35 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:37:35.584 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:37:35 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:37:35.584 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:37:35 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:37:35.584 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:37:35 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:37:35.584 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:37:35 smithi039 ceph-mon[31211]: from='client.? 172.21.15.39:0/3067419537' entity='client.admin' 2024-09-21T03:37:35.584 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:37:35 smithi039 ceph-mon[31211]: from='client.? ' entity='client.admin' 2024-09-21T03:37:35.654 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:35.615+0000 7fd199dfe640 1 -- 172.21.15.39:0/1152308564 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd194105300 msgr2=0x7fd19410d850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:37:35.654 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:35.615+0000 7fd199dfe640 1 --2- 172.21.15.39:0/1152308564 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd194105300 0x7fd19410d850 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fd188009900 tx=0x7fd18802a410 comp rx=0 tx=0).stop 2024-09-21T03:37:35.654 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:35.615+0000 7fd199dfe640 1 -- 172.21.15.39:0/1152308564 shutdown_connections 2024-09-21T03:37:35.654 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:35.615+0000 7fd199dfe640 1 --2- 172.21.15.39:0/1152308564 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd19410de10 0x7fd194110300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:35.655 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:35.615+0000 7fd199dfe640 1 --2- 172.21.15.39:0/1152308564 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd194105300 0x7fd19410d850 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:35.655 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:35.615+0000 7fd199dfe640 1 --2- 172.21.15.39:0/1152308564 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd1941049b0 0x7fd194104dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:35.655 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:35.615+0000 7fd199dfe640 1 -- 172.21.15.39:0/1152308564 >> 172.21.15.39:0/1152308564 conn(0x7fd194079730 msgr2=0x7fd194079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:37:35.655 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:35.615+0000 7fd199dfe640 1 -- 172.21.15.39:0/1152308564 shutdown_connections 2024-09-21T03:37:35.655 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:35.615+0000 7fd199dfe640 1 -- 172.21.15.39:0/1152308564 wait complete. 2024-09-21T03:37:35.655 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:35.616+0000 7fd199dfe640 1 Processor -- start 2024-09-21T03:37:35.655 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:35.616+0000 7fd199dfe640 1 -- start start 2024-09-21T03:37:35.655 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:35.617+0000 7fd199dfe640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd1941049b0 0x7fd19419c670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:37:35.655 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:35.617+0000 7fd199dfe640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd194105300 0x7fd19419cc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:37:35.655 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:35.617+0000 7fd199dfe640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd19410de10 0x7fd1941a1eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:37:35.655 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:35.617+0000 7fd199dfe640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fd194113ac0 con 0x7fd1941049b0 2024-09-21T03:37:35.655 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:35.617+0000 7fd199dfe640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fd194113940 con 0x7fd194105300 2024-09-21T03:37:35.656 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:35.617+0000 7fd199dfe640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fd194113c40 con 0x7fd19410de10 2024-09-21T03:37:35.656 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:35.617+0000 7fd1937fe640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd1941049b0 0x7fd19419c670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:37:35.656 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:35.617+0000 7fd192ffd640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd194105300 0x7fd19419cc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:37:35.656 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:35.617+0000 7fd1937fe640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd1941049b0 0x7fd19419c670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:49828/0 (socket says 172.21.15.39:49828) 2024-09-21T03:37:35.656 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:35.617+0000 7fd192ffd640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd194105300 0x7fd19419cc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.39:56848/0 (socket says 172.21.15.39:56848) 2024-09-21T03:37:35.656 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:35.617+0000 7fd1937fe640 1 -- 172.21.15.39:0/982812054 learned_addr learned my addr 172.21.15.39:0/982812054 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:37:35.656 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:35.617+0000 7fd193fff640 1 --2- 172.21.15.39:0/982812054 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd19410de10 0x7fd1941a1eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:37:35.656 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:35.617+0000 7fd1937fe640 1 -- 172.21.15.39:0/982812054 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd19410de10 msgr2=0x7fd1941a1eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:37:35.656 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:35.617+0000 7fd1937fe640 1 --2- 172.21.15.39:0/982812054 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd19410de10 0x7fd1941a1eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:35.656 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:35.617+0000 7fd1937fe640 1 -- 172.21.15.39:0/982812054 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd194105300 msgr2=0x7fd19419cc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:37:35.656 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:35.617+0000 7fd1937fe640 1 --2- 172.21.15.39:0/982812054 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd194105300 0x7fd19419cc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:35.656 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:35.617+0000 7fd1937fe640 1 -- 172.21.15.39:0/982812054 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd1941a25e0 con 0x7fd1941049b0 2024-09-21T03:37:35.656 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:35.618+0000 7fd1937fe640 1 --2- 172.21.15.39:0/982812054 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd1941049b0 0x7fd19419c670 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fd18400d950 tx=0x7fd18400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:37:35.657 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:35.618+0000 7fd190ff9640 1 -- 172.21.15.39:0/982812054 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd184017020 con 0x7fd1941049b0 2024-09-21T03:37:35.657 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:35.618+0000 7fd190ff9640 1 -- 172.21.15.39:0/982812054 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd184009c30 con 0x7fd1941049b0 2024-09-21T03:37:35.657 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:35.618+0000 7fd190ff9640 1 -- 172.21.15.39:0/982812054 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd18400f6b0 con 0x7fd1941049b0 2024-09-21T03:37:35.657 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:35.618+0000 7fd199dfe640 1 -- 172.21.15.39:0/982812054 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd1941a28d0 con 0x7fd1941049b0 2024-09-21T03:37:35.657 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:35.618+0000 7fd199dfe640 1 -- 172.21.15.39:0/982812054 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd1941a9030 con 0x7fd1941049b0 2024-09-21T03:37:35.657 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:35.619+0000 7fd190ff9640 1 -- 172.21.15.39:0/982812054 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 18) ==== 51915+0+0 (secure 0 0 0) 0x7fd18401a020 con 0x7fd1941049b0 2024-09-21T03:37:35.657 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:35.619+0000 7fd199dfe640 1 -- 172.21.15.39:0/982812054 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd194104dc0 con 0x7fd1941049b0 2024-09-21T03:37:35.657 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:35.619+0000 7fd190ff9640 1 --2- 172.21.15.39:0/982812054 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7fd17c03fb70 0x7fd17c042080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:37:35.657 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:35.619+0000 7fd190ff9640 1 -- 172.21.15.39:0/982812054 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1370+0+0 (secure 0 0 0) 0x7fd184055140 con 0x7fd1941049b0 2024-09-21T03:37:35.657 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:35.620+0000 7fd192ffd640 1 --2- 172.21.15.39:0/982812054 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7fd17c03fb70 0x7fd17c042080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:37:35.657 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:35.620+0000 7fd192ffd640 1 --2- 172.21.15.39:0/982812054 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7fd17c03fb70 0x7fd17c042080 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fd18802aa00 tx=0x7fd188005a20 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:37:35.657 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:35.624+0000 7fd190ff9640 1 -- 172.21.15.39:0/982812054 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fd184009de0 con 0x7fd1941049b0 2024-09-21T03:37:35.717 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:37:35 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:37:35.717 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:37:35 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:37:35.718 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:37:35 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:37:35.718 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:37:35 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:37:35.718 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:37:35 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:37:35.718 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:37:35 smithi161 ceph-mon[34533]: from='client.? 172.21.15.39:0/3067419537' entity='client.admin' 2024-09-21T03:37:35.718 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:37:35 smithi161 ceph-mon[34533]: from='client.? ' entity='client.admin' 2024-09-21T03:37:35.724 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:37:35 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:37:35.724 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:37:35 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:37:35.724 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:37:35 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:37:35.724 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:37:35 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:37:35.725 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:37:35 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:37:35.725 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:37:35 smithi150 ceph-mon[34289]: from='client.? 172.21.15.39:0/3067419537' entity='client.admin' 2024-09-21T03:37:35.725 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:37:35 smithi150 ceph-mon[34289]: from='client.? ' entity='client.admin' 2024-09-21T03:37:35.832 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:35.829+0000 7fd199dfe640 1 -- 172.21.15.39:0/982812054 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command([{prefix=config set, name=mds_verify_scatter}] v 0) -- 0x7fd1941a9410 con 0x7fd1941049b0 2024-09-21T03:37:35.836 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:35.834+0000 7fd190ff9640 1 -- 172.21.15.39:0/982812054 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_verify_scatter}]=0 v18)=0 v18) ==== 119+0+0 (secure 0 0 0) 0x7fd18401c410 con 0x7fd1941049b0 2024-09-21T03:37:35.839 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:35.836+0000 7fd199dfe640 1 -- 172.21.15.39:0/982812054 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7fd17c03fb70 msgr2=0x7fd17c042080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:37:35.839 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:35.836+0000 7fd199dfe640 1 --2- 172.21.15.39:0/982812054 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7fd17c03fb70 0x7fd17c042080 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fd18802aa00 tx=0x7fd188005a20 comp rx=0 tx=0).stop 2024-09-21T03:37:35.839 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:35.836+0000 7fd199dfe640 1 -- 172.21.15.39:0/982812054 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd1941049b0 msgr2=0x7fd19419c670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:37:35.839 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:35.836+0000 7fd199dfe640 1 --2- 172.21.15.39:0/982812054 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd1941049b0 0x7fd19419c670 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fd18400d950 tx=0x7fd18400de10 comp rx=0 tx=0).stop 2024-09-21T03:37:35.839 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:35.836+0000 7fd199dfe640 1 -- 172.21.15.39:0/982812054 shutdown_connections 2024-09-21T03:37:35.839 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:35.836+0000 7fd199dfe640 1 --2- 172.21.15.39:0/982812054 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7fd17c03fb70 0x7fd17c042080 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:35.839 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:35.836+0000 7fd199dfe640 1 --2- 172.21.15.39:0/982812054 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd19410de10 0x7fd1941a1eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:35.839 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:35.836+0000 7fd199dfe640 1 --2- 172.21.15.39:0/982812054 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd194105300 0x7fd19419cc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:35.839 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:35.836+0000 7fd199dfe640 1 --2- 172.21.15.39:0/982812054 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd1941049b0 0x7fd19419c670 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:35.839 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:35.836+0000 7fd199dfe640 1 -- 172.21.15.39:0/982812054 >> 172.21.15.39:0/982812054 conn(0x7fd194079730 msgr2=0x7fd1941036a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:37:35.839 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:35.836+0000 7fd199dfe640 1 -- 172.21.15.39:0/982812054 shutdown_connections 2024-09-21T03:37:35.839 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:35.836+0000 7fd199dfe640 1 -- 172.21.15.39:0/982812054 wait complete. 2024-09-21T03:37:36.562 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:37:36 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:37:36.562 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:37:36 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:37:36.562 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:37:36 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:37:36.562 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:37:36 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:37:36.562 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:37:36 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:37:36.562 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:37:36 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:37:36.563 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:37:36 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:37:36.563 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:37:36 smithi039 ceph-mon[31211]: Reconfiguring mgr.x (unknown last config time)... 2024-09-21T03:37:36.563 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:37:36 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]} : dispatch 2024-09-21T03:37:36.563 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:37:36 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "mgr services"} : dispatch 2024-09-21T03:37:36.563 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:37:36 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:37:36.563 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:37:36 smithi039 ceph-mon[31211]: Reconfiguring daemon mgr.x on smithi039 2024-09-21T03:37:36.563 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:37:36 smithi039 ceph-mon[31211]: from='client.? 172.21.15.39:0/1684594876' entity='client.admin' 2024-09-21T03:37:36.563 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:37:36 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:37:36.563 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:37:36 smithi039 ceph-mon[31211]: from='client.? 172.21.15.39:0/982812054' entity='client.admin' 2024-09-21T03:37:36.619 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:37:36 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:37:36.619 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:37:36 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:37:36.620 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:37:36 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:37:36.620 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:37:36 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:37:36.620 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:37:36 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:37:36.620 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:37:36 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:37:36.620 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:37:36 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:37:36.620 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:37:36 smithi161 ceph-mon[34533]: Reconfiguring mgr.x (unknown last config time)... 2024-09-21T03:37:36.620 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:37:36 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]} : dispatch 2024-09-21T03:37:36.620 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:37:36 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "mgr services"} : dispatch 2024-09-21T03:37:36.620 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:37:36 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:37:36.620 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:37:36 smithi161 ceph-mon[34533]: Reconfiguring daemon mgr.x on smithi039 2024-09-21T03:37:36.620 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:37:36 smithi161 ceph-mon[34533]: from='client.? 172.21.15.39:0/1684594876' entity='client.admin' 2024-09-21T03:37:36.620 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:37:36 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:37:36.620 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:37:36 smithi161 ceph-mon[34533]: from='client.? 172.21.15.39:0/982812054' entity='client.admin' 2024-09-21T03:37:36.626 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:37:36 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:37:36.626 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:37:36 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:37:36.626 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:37:36 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:37:36.626 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:37:36 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:37:36.626 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:37:36 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:37:36.626 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:37:36 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:37:36.626 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:37:36 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:37:36.627 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:37:36 smithi150 ceph-mon[34289]: Reconfiguring mgr.x (unknown last config time)... 2024-09-21T03:37:36.627 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:37:36 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]} : dispatch 2024-09-21T03:37:36.627 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:37:36 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "mgr services"} : dispatch 2024-09-21T03:37:36.627 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:37:36 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:37:36.627 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:37:36 smithi150 ceph-mon[34289]: Reconfiguring daemon mgr.x on smithi039 2024-09-21T03:37:36.627 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:37:36 smithi150 ceph-mon[34289]: from='client.? 172.21.15.39:0/1684594876' entity='client.admin' 2024-09-21T03:37:36.627 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:37:36 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:37:36.627 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:37:36 smithi150 ceph-mon[34289]: from='client.? 172.21.15.39:0/982812054' entity='client.admin' 2024-09-21T03:37:36.662 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:36.654+0000 7fab3fd7f640 1 -- 172.21.15.39:0/3866133202 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fab38102ae0 msgr2=0x7fab38102ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:37:36.662 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:36.654+0000 7fab3fd7f640 1 --2- 172.21.15.39:0/3866133202 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fab38102ae0 0x7fab38102ef0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fab34009900 tx=0x7fab3402a410 comp rx=0 tx=0).stop 2024-09-21T03:37:36.662 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:36.654+0000 7fab3fd7f640 1 -- 172.21.15.39:0/3866133202 shutdown_connections 2024-09-21T03:37:36.662 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:36.654+0000 7fab3fd7f640 1 --2- 172.21.15.39:0/3866133202 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fab38108d20 0x7fab3810f550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:36.662 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:36.654+0000 7fab3fd7f640 1 --2- 172.21.15.39:0/3866133202 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fab381083d0 0x7fab381087e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:36.662 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:36.654+0000 7fab3fd7f640 1 --2- 172.21.15.39:0/3866133202 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fab38102ae0 0x7fab38102ef0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:36.662 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:36.654+0000 7fab3fd7f640 1 -- 172.21.15.39:0/3866133202 >> 172.21.15.39:0/3866133202 conn(0x7fab38079760 msgr2=0x7fab38079b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:37:36.662 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:36.654+0000 7fab3fd7f640 1 -- 172.21.15.39:0/3866133202 shutdown_connections 2024-09-21T03:37:36.662 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:36.654+0000 7fab3fd7f640 1 -- 172.21.15.39:0/3866133202 wait complete. 2024-09-21T03:37:36.662 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:36.655+0000 7fab3fd7f640 1 Processor -- start 2024-09-21T03:37:36.663 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:36.655+0000 7fab3fd7f640 1 -- start start 2024-09-21T03:37:36.663 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:36.655+0000 7fab3fd7f640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fab38102ae0 0x7fab3819a470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:37:36.663 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:36.655+0000 7fab3fd7f640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fab381083d0 0x7fab3819aa10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:37:36.663 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:36.655+0000 7fab3fd7f640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fab38108d20 0x7fab3819fcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:37:36.663 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:36.655+0000 7fab3fd7f640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fab38112990 con 0x7fab381083d0 2024-09-21T03:37:36.663 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:36.655+0000 7fab3fd7f640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fab38112810 con 0x7fab38108d20 2024-09-21T03:37:36.663 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:36.655+0000 7fab3fd7f640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fab38112b10 con 0x7fab38102ae0 2024-09-21T03:37:36.663 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:36.655+0000 7fab3d31a640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fab381083d0 0x7fab3819aa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:37:36.663 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:36.655+0000 7fab3d31a640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fab381083d0 0x7fab3819aa10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:49846/0 (socket says 172.21.15.39:49846) 2024-09-21T03:37:36.663 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:36.655+0000 7fab3d31a640 1 -- 172.21.15.39:0/1554142917 learned_addr learned my addr 172.21.15.39:0/1554142917 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:37:36.663 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:36.655+0000 7fab3db1b640 1 --2- 172.21.15.39:0/1554142917 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fab38102ae0 0x7fab3819a470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:37:36.663 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:36.655+0000 7fab3e31c640 1 --2- 172.21.15.39:0/1554142917 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fab38108d20 0x7fab3819fcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:37:36.663 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:36.656+0000 7fab3d31a640 1 -- 172.21.15.39:0/1554142917 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fab38102ae0 msgr2=0x7fab3819a470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:37:36.663 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:36.656+0000 7fab3d31a640 1 --2- 172.21.15.39:0/1554142917 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fab38102ae0 0x7fab3819a470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:36.664 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:36.656+0000 7fab3d31a640 1 -- 172.21.15.39:0/1554142917 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fab38108d20 msgr2=0x7fab3819fcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:37:36.664 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:36.656+0000 7fab3d31a640 1 --2- 172.21.15.39:0/1554142917 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fab38108d20 0x7fab3819fcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:36.664 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:36.656+0000 7fab3d31a640 1 -- 172.21.15.39:0/1554142917 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fab381a03e0 con 0x7fab381083d0 2024-09-21T03:37:36.664 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:36.656+0000 7fab3db1b640 1 --2- 172.21.15.39:0/1554142917 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fab38102ae0 0x7fab3819a470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:37:36.664 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:36.656+0000 7fab3d31a640 1 --2- 172.21.15.39:0/1554142917 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fab381083d0 0x7fab3819aa10 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fab2800e9f0 tx=0x7fab2800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:37:36.664 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:36.656+0000 7fab2effd640 1 -- 172.21.15.39:0/1554142917 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab2800ccc0 con 0x7fab381083d0 2024-09-21T03:37:36.664 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:36.656+0000 7fab2effd640 1 -- 172.21.15.39:0/1554142917 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fab280054d0 con 0x7fab381083d0 2024-09-21T03:37:36.664 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:36.656+0000 7fab3fd7f640 1 -- 172.21.15.39:0/1554142917 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fab381a06d0 con 0x7fab381083d0 2024-09-21T03:37:36.664 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:36.656+0000 7fab2effd640 1 -- 172.21.15.39:0/1554142917 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab28010430 con 0x7fab381083d0 2024-09-21T03:37:36.664 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:36.656+0000 7fab3fd7f640 1 -- 172.21.15.39:0/1554142917 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fab381a6e30 con 0x7fab381083d0 2024-09-21T03:37:36.664 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:36.657+0000 7fab3fd7f640 1 -- 172.21.15.39:0/1554142917 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fab38103d10 con 0x7fab381083d0 2024-09-21T03:37:36.664 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:36.657+0000 7fab2effd640 1 -- 172.21.15.39:0/1554142917 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 18) ==== 51915+0+0 (secure 0 0 0) 0x7fab280105d0 con 0x7fab381083d0 2024-09-21T03:37:36.664 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:36.658+0000 7fab2effd640 1 --2- 172.21.15.39:0/1554142917 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7fab2403fb70 0x7fab24042080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:37:36.664 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:36.658+0000 7fab2effd640 1 -- 172.21.15.39:0/1554142917 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1370+0+0 (secure 0 0 0) 0x7fab280549b0 con 0x7fab381083d0 2024-09-21T03:37:36.664 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:36.658+0000 7fab3db1b640 1 --2- 172.21.15.39:0/1554142917 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7fab2403fb70 0x7fab24042080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:37:36.665 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:36.658+0000 7fab3db1b640 1 --2- 172.21.15.39:0/1554142917 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7fab2403fb70 0x7fab24042080 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fab34031440 tx=0x7fab34032ed0 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:37:36.665 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:36.663+0000 7fab2effd640 1 -- 172.21.15.39:0/1554142917 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fab2800f3c0 con 0x7fab381083d0 2024-09-21T03:37:36.867 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:36.864+0000 7fab3fd7f640 1 -- 172.21.15.39:0/1554142917 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command([{prefix=config set, name=mon_warn_on_pool_no_app}] v 0) -- 0x7fab381a7150 con 0x7fab381083d0 2024-09-21T03:37:36.871 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:36.868+0000 7fab2effd640 1 -- 172.21.15.39:0/1554142917 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_warn_on_pool_no_app}]=0 v19)=0 v19) ==== 125+0+0 (secure 0 0 0) 0x7fab28020e10 con 0x7fab381083d0 2024-09-21T03:37:36.873 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:36.871+0000 7fab3fd7f640 1 -- 172.21.15.39:0/1554142917 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7fab2403fb70 msgr2=0x7fab24042080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:37:36.873 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:36.871+0000 7fab3fd7f640 1 --2- 172.21.15.39:0/1554142917 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7fab2403fb70 0x7fab24042080 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fab34031440 tx=0x7fab34032ed0 comp rx=0 tx=0).stop 2024-09-21T03:37:36.874 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:36.871+0000 7fab3fd7f640 1 -- 172.21.15.39:0/1554142917 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fab381083d0 msgr2=0x7fab3819aa10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:37:36.874 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:36.871+0000 7fab3fd7f640 1 --2- 172.21.15.39:0/1554142917 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fab381083d0 0x7fab3819aa10 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fab2800e9f0 tx=0x7fab2800eeb0 comp rx=0 tx=0).stop 2024-09-21T03:37:36.874 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:36.871+0000 7fab3fd7f640 1 -- 172.21.15.39:0/1554142917 shutdown_connections 2024-09-21T03:37:36.874 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:36.871+0000 7fab3fd7f640 1 --2- 172.21.15.39:0/1554142917 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7fab2403fb70 0x7fab24042080 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:36.874 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:36.871+0000 7fab3fd7f640 1 --2- 172.21.15.39:0/1554142917 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fab38108d20 0x7fab3819fcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:36.874 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:36.871+0000 7fab3fd7f640 1 --2- 172.21.15.39:0/1554142917 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fab381083d0 0x7fab3819aa10 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:36.874 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:36.871+0000 7fab3fd7f640 1 --2- 172.21.15.39:0/1554142917 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fab38102ae0 0x7fab3819a470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:36.874 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:36.871+0000 7fab3fd7f640 1 -- 172.21.15.39:0/1554142917 >> 172.21.15.39:0/1554142917 conn(0x7fab38079760 msgr2=0x7fab381067a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:37:36.874 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:36.871+0000 7fab3fd7f640 1 -- 172.21.15.39:0/1554142917 shutdown_connections 2024-09-21T03:37:36.874 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:36.871+0000 7fab3fd7f640 1 -- 172.21.15.39:0/1554142917 wait complete. 2024-09-21T03:37:37.765 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:37.762+0000 7fbe965a8640 1 -- 172.21.15.39:0/1481428309 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbe90107bd0 msgr2=0x7fbe90107fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:37:37.765 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:37.762+0000 7fbe965a8640 1 --2- 172.21.15.39:0/1481428309 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbe90107bd0 0x7fbe90107fe0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fbe78009900 tx=0x7fbe7802a410 comp rx=0 tx=0).stop 2024-09-21T03:37:37.765 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:37.762+0000 7fbe965a8640 1 -- 172.21.15.39:0/1481428309 shutdown_connections 2024-09-21T03:37:37.765 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:37.762+0000 7fbe965a8640 1 --2- 172.21.15.39:0/1481428309 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbe90108520 0x7fbe9010ed50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:37.765 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:37.762+0000 7fbe965a8640 1 --2- 172.21.15.39:0/1481428309 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbe90107bd0 0x7fbe90107fe0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:37.765 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:37.762+0000 7fbe965a8640 1 --2- 172.21.15.39:0/1481428309 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbe90102290 0x7fbe901026a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:37.765 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:37.762+0000 7fbe965a8640 1 -- 172.21.15.39:0/1481428309 >> 172.21.15.39:0/1481428309 conn(0x7fbe90079ea0 msgr2=0x7fbe9007a2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:37:37.765 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:37.762+0000 7fbe965a8640 1 -- 172.21.15.39:0/1481428309 shutdown_connections 2024-09-21T03:37:37.765 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:37.762+0000 7fbe965a8640 1 -- 172.21.15.39:0/1481428309 wait complete. 2024-09-21T03:37:37.768 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:37.763+0000 7fbe965a8640 1 Processor -- start 2024-09-21T03:37:37.768 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:37.763+0000 7fbe965a8640 1 -- start start 2024-09-21T03:37:37.768 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:37.763+0000 7fbe965a8640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbe90102290 0x7fbe9019c7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:37:37.768 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:37.763+0000 7fbe965a8640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbe90107bd0 0x7fbe9019cd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:37:37.768 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:37.763+0000 7fbe965a8640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbe90108520 0x7fbe901a1fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:37:37.768 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:37.763+0000 7fbe965a8640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fbe901121e0 con 0x7fbe90107bd0 2024-09-21T03:37:37.768 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:37.763+0000 7fbe965a8640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fbe90112060 con 0x7fbe90102290 2024-09-21T03:37:37.768 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:37.763+0000 7fbe965a8640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fbe90112360 con 0x7fbe90108520 2024-09-21T03:37:37.768 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:37.763+0000 7fbe87fff640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbe90107bd0 0x7fbe9019cd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:37:37.768 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:37.763+0000 7fbe8ffff640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbe90102290 0x7fbe9019c7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:37:37.768 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:37.763+0000 7fbe87fff640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbe90107bd0 0x7fbe9019cd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:46586/0 (socket says 172.21.15.39:46586) 2024-09-21T03:37:37.769 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:37.763+0000 7fbe87fff640 1 -- 172.21.15.39:0/3409249567 learned_addr learned my addr 172.21.15.39:0/3409249567 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:37:37.769 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:37.763+0000 7fbe94b45640 1 --2- 172.21.15.39:0/3409249567 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbe90108520 0x7fbe901a1fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:37:37.769 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:37.763+0000 7fbe87fff640 1 -- 172.21.15.39:0/3409249567 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbe90108520 msgr2=0x7fbe901a1fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:37:37.769 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:37.763+0000 7fbe87fff640 1 --2- 172.21.15.39:0/3409249567 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbe90108520 0x7fbe901a1fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:37.769 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:37.763+0000 7fbe87fff640 1 -- 172.21.15.39:0/3409249567 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbe90102290 msgr2=0x7fbe9019c7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:37:37.769 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:37.763+0000 7fbe87fff640 1 --2- 172.21.15.39:0/3409249567 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbe90102290 0x7fbe9019c7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:37.769 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:37.764+0000 7fbe87fff640 1 -- 172.21.15.39:0/3409249567 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbe901a2710 con 0x7fbe90107bd0 2024-09-21T03:37:37.769 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:37.764+0000 7fbe87fff640 1 --2- 172.21.15.39:0/3409249567 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbe90107bd0 0x7fbe9019cd40 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fbe78005ab0 tx=0x7fbe78003cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:37:37.769 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:37.764+0000 7fbe8dffb640 1 -- 172.21.15.39:0/3409249567 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe7803f070 con 0x7fbe90107bd0 2024-09-21T03:37:37.769 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:37.764+0000 7fbe8dffb640 1 -- 172.21.15.39:0/3409249567 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbe78031d60 con 0x7fbe90107bd0 2024-09-21T03:37:37.769 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:37.764+0000 7fbe965a8640 1 -- 172.21.15.39:0/3409249567 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbe901a29a0 con 0x7fbe90107bd0 2024-09-21T03:37:37.769 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:37.764+0000 7fbe8dffb640 1 -- 172.21.15.39:0/3409249567 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe78035030 con 0x7fbe90107bd0 2024-09-21T03:37:37.769 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:37.764+0000 7fbe965a8640 1 -- 172.21.15.39:0/3409249567 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbe901a9160 con 0x7fbe90107bd0 2024-09-21T03:37:37.769 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:37.765+0000 7fbe8dffb640 1 -- 172.21.15.39:0/3409249567 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 18) ==== 51915+0+0 (secure 0 0 0) 0x7fbe78042070 con 0x7fbe90107bd0 2024-09-21T03:37:37.769 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:37.765+0000 7fbe965a8640 1 -- 172.21.15.39:0/3409249567 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbe901026a0 con 0x7fbe90107bd0 2024-09-21T03:37:37.770 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:37.765+0000 7fbe8dffb640 1 --2- 172.21.15.39:0/3409249567 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7fbe7c03fb70 0x7fbe7c042080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:37:37.770 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:37.765+0000 7fbe8dffb640 1 -- 172.21.15.39:0/3409249567 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1370+0+0 (secure 0 0 0) 0x7fbe78074d90 con 0x7fbe90107bd0 2024-09-21T03:37:37.770 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:37.765+0000 7fbe8ffff640 1 --2- 172.21.15.39:0/3409249567 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7fbe7c03fb70 0x7fbe7c042080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:37:37.770 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:37.766+0000 7fbe8ffff640 1 --2- 172.21.15.39:0/3409249567 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7fbe7c03fb70 0x7fbe7c042080 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fbe80005d00 tx=0x7fbe8000b410 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:37:37.773 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:37.770+0000 7fbe8dffb640 1 -- 172.21.15.39:0/3409249567 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fbe78042360 con 0x7fbe90107bd0 2024-09-21T03:37:37.853 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:37.084+0000 7fbe6f6e2640 1 -- 172.21.15.39:0/2013583902 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbe68109e50 msgr2=0x7fbe68110680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:37:37.854 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:37.084+0000 7fbe6f6e2640 1 --2- 172.21.15.39:0/2013583902 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbe68109e50 0x7fbe68110680 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fbe60009ca0 tx=0x7fbe6002a8c0 comp rx=0 tx=0).stop 2024-09-21T03:37:37.854 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:37.084+0000 7fbe6f6e2640 1 -- 172.21.15.39:0/2013583902 shutdown_connections 2024-09-21T03:37:37.854 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:37.084+0000 7fbe6f6e2640 1 --2- 172.21.15.39:0/2013583902 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbe68109e50 0x7fbe68110680 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:37.854 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:37.084+0000 7fbe6f6e2640 1 --2- 172.21.15.39:0/2013583902 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbe68109500 0x7fbe68109910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:37.854 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:37.084+0000 7fbe6f6e2640 1 --2- 172.21.15.39:0/2013583902 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbe68101fc0 0x7fbe681023d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:37.854 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:37.084+0000 7fbe6f6e2640 1 -- 172.21.15.39:0/2013583902 >> 172.21.15.39:0/2013583902 conn(0x7fbe68079f90 msgr2=0x7fbe6807a3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:37:37.854 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:37.084+0000 7fbe6f6e2640 1 -- 172.21.15.39:0/2013583902 shutdown_connections 2024-09-21T03:37:37.854 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:37.084+0000 7fbe6f6e2640 1 -- 172.21.15.39:0/2013583902 wait complete. 2024-09-21T03:37:37.854 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:37.085+0000 7fbe6f6e2640 1 Processor -- start 2024-09-21T03:37:37.854 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:37.085+0000 7fbe6f6e2640 1 -- start start 2024-09-21T03:37:37.854 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:37.085+0000 7fbe6f6e2640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbe68101fc0 0x7fbe6819a4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:37:37.854 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:37.085+0000 7fbe6f6e2640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbe68109500 0x7fbe6819aa40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:37:37.854 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:37.085+0000 7fbe6f6e2640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbe68109e50 0x7fbe6819fce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:37:37.855 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:37.085+0000 7fbe6f6e2640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fbe68113b10 con 0x7fbe68109500 2024-09-21T03:37:37.855 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:37.085+0000 7fbe6f6e2640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fbe68113990 con 0x7fbe68109e50 2024-09-21T03:37:37.855 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:37.085+0000 7fbe6f6e2640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fbe68113c90 con 0x7fbe68101fc0 2024-09-21T03:37:37.855 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:37.085+0000 7fbe6d47e640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbe68101fc0 0x7fbe6819a4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:37:37.855 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:37.085+0000 7fbe6cc7d640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbe68109500 0x7fbe6819aa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:37:37.855 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:37.085+0000 7fbe6d47e640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbe68101fc0 0x7fbe6819a4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.39:33426/0 (socket says 172.21.15.39:33426) 2024-09-21T03:37:37.855 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:37.085+0000 7fbe6d47e640 1 -- 172.21.15.39:0/1825647635 learned_addr learned my addr 172.21.15.39:0/1825647635 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:37:37.855 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:37.085+0000 7fbe6dc7f640 1 --2- 172.21.15.39:0/1825647635 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbe68109e50 0x7fbe6819fce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:37:37.855 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:37.086+0000 7fbe6cc7d640 1 -- 172.21.15.39:0/1825647635 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbe68101fc0 msgr2=0x7fbe6819a4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:37:37.855 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:37.086+0000 7fbe6cc7d640 1 --2- 172.21.15.39:0/1825647635 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbe68101fc0 0x7fbe6819a4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:37.855 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:37.086+0000 7fbe6cc7d640 1 -- 172.21.15.39:0/1825647635 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbe68109e50 msgr2=0x7fbe6819fce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:37:37.855 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:37.086+0000 7fbe6cc7d640 1 --2- 172.21.15.39:0/1825647635 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbe68109e50 0x7fbe6819fce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:37.855 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:37.086+0000 7fbe6cc7d640 1 -- 172.21.15.39:0/1825647635 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbe681a0410 con 0x7fbe68109500 2024-09-21T03:37:37.855 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:37.086+0000 7fbe6d47e640 1 --2- 172.21.15.39:0/1825647635 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbe68101fc0 0x7fbe6819a4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:37:37.856 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:37.086+0000 7fbe6cc7d640 1 --2- 172.21.15.39:0/1825647635 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbe68109500 0x7fbe6819aa40 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fbe68075a60 tx=0x7fbe5800ee50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:37:37.856 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:37.086+0000 7fbe5e7fc640 1 -- 172.21.15.39:0/1825647635 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe5800cc40 con 0x7fbe68109500 2024-09-21T03:37:37.856 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:37.086+0000 7fbe5e7fc640 1 -- 172.21.15.39:0/1825647635 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbe5800cde0 con 0x7fbe68109500 2024-09-21T03:37:37.856 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:37.086+0000 7fbe6f6e2640 1 -- 172.21.15.39:0/1825647635 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbe681a0700 con 0x7fbe68109500 2024-09-21T03:37:37.856 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:37.086+0000 7fbe5e7fc640 1 -- 172.21.15.39:0/1825647635 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe58004a00 con 0x7fbe68109500 2024-09-21T03:37:37.856 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:37.086+0000 7fbe6f6e2640 1 -- 172.21.15.39:0/1825647635 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbe681a6e60 con 0x7fbe68109500 2024-09-21T03:37:37.856 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:37.087+0000 7fbe5e7fc640 1 -- 172.21.15.39:0/1825647635 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 18) ==== 51915+0+0 (secure 0 0 0) 0x7fbe58004ba0 con 0x7fbe68109500 2024-09-21T03:37:37.856 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:37.087+0000 7fbe6f6e2640 1 -- 172.21.15.39:0/1825647635 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbe681023d0 con 0x7fbe68109500 2024-09-21T03:37:37.856 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:37.087+0000 7fbe5e7fc640 1 --2- 172.21.15.39:0/1825647635 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7fbe5403fb70 0x7fbe54042080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:37:37.856 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:37.087+0000 7fbe5e7fc640 1 -- 172.21.15.39:0/1825647635 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1370+0+0 (secure 0 0 0) 0x7fbe58054650 con 0x7fbe68109500 2024-09-21T03:37:37.856 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:37.087+0000 7fbe6d47e640 1 --2- 172.21.15.39:0/1825647635 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7fbe5403fb70 0x7fbe54042080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:37:37.856 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:37.088+0000 7fbe6d47e640 1 --2- 172.21.15.39:0/1825647635 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7fbe5403fb70 0x7fbe54042080 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fbe64005fd0 tx=0x7fbe6400b4a0 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:37:37.856 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:37.092+0000 7fbe5e7fc640 1 -- 172.21.15.39:0/1825647635 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fbe58013190 con 0x7fbe68109500 2024-09-21T03:37:37.856 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:37.284+0000 7fbe6f6e2640 1 -- 172.21.15.39:0/1825647635 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command([{prefix=config set, name=debug_cephsqlite}] v 0) -- 0x7fbe6802e640 con 0x7fbe68109500 2024-09-21T03:37:37.856 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:37.289+0000 7fbe5e7fc640 1 -- 172.21.15.39:0/1825647635 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{prefix=config set, name=debug_cephsqlite}]=0 v20)=0 v20) ==== 115+0+0 (secure 0 0 0) 0x7fbe58020c70 con 0x7fbe68109500 2024-09-21T03:37:37.857 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:37.292+0000 7fbe6f6e2640 1 -- 172.21.15.39:0/1825647635 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7fbe5403fb70 msgr2=0x7fbe54042080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:37:37.857 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:37.292+0000 7fbe6f6e2640 1 --2- 172.21.15.39:0/1825647635 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7fbe5403fb70 0x7fbe54042080 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fbe64005fd0 tx=0x7fbe6400b4a0 comp rx=0 tx=0).stop 2024-09-21T03:37:37.857 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:37.292+0000 7fbe6f6e2640 1 -- 172.21.15.39:0/1825647635 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbe68109500 msgr2=0x7fbe6819aa40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:37:37.857 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:37.292+0000 7fbe6f6e2640 1 --2- 172.21.15.39:0/1825647635 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbe68109500 0x7fbe6819aa40 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fbe68075a60 tx=0x7fbe5800ee50 comp rx=0 tx=0).stop 2024-09-21T03:37:37.857 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:37.292+0000 7fbe6f6e2640 1 -- 172.21.15.39:0/1825647635 shutdown_connections 2024-09-21T03:37:37.857 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:37.292+0000 7fbe6f6e2640 1 --2- 172.21.15.39:0/1825647635 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7fbe5403fb70 0x7fbe54042080 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:37.857 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:37.292+0000 7fbe6f6e2640 1 --2- 172.21.15.39:0/1825647635 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbe68109e50 0x7fbe6819fce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:37.857 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:37.292+0000 7fbe6f6e2640 1 --2- 172.21.15.39:0/1825647635 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbe68109500 0x7fbe6819aa40 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:37.857 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:37.292+0000 7fbe6f6e2640 1 --2- 172.21.15.39:0/1825647635 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbe68101fc0 0x7fbe6819a4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:37.857 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:37.292+0000 7fbe6f6e2640 1 -- 172.21.15.39:0/1825647635 >> 172.21.15.39:0/1825647635 conn(0x7fbe68079f90 msgr2=0x7fbe68107510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:37:37.857 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:37.292+0000 7fbe6f6e2640 1 -- 172.21.15.39:0/1825647635 shutdown_connections 2024-09-21T03:37:37.857 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:37.292+0000 7fbe6f6e2640 1 -- 172.21.15.39:0/1825647635 wait complete. 2024-09-21T03:37:37.879 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:37.877+0000 7fbe8dffb640 1 -- 172.21.15.39:0/3409249567 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mgrmap(e 19) ==== 103194+0+0 (secure 0 0 0) 0x7fbe7807b080 con 0x7fbe90107bd0 2024-09-21T03:37:37.895 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:37:37 smithi150 ceph-mon[34289]: from='client.? 172.21.15.39:0/1554142917' entity='client.admin' 2024-09-21T03:37:37.895 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:37:37 smithi150 ceph-mon[34289]: Standby manager daemon z started 2024-09-21T03:37:37.895 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:37:37 smithi150 ceph-mon[34289]: from='mgr.? 172.21.15.161:0/3017326223' entity='mgr.z' cmd={"prefix": "config-key get", "key": "mgr/dashboard/z/crt"} : dispatch 2024-09-21T03:37:37.895 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:37:37 smithi150 ceph-mon[34289]: from='mgr.? 172.21.15.161:0/3017326223' entity='mgr.z' cmd={"prefix": "config-key get", "key": "mgr/dashboard/crt"} : dispatch 2024-09-21T03:37:37.895 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:37:37 smithi150 ceph-mon[34289]: from='mgr.? 172.21.15.161:0/3017326223' entity='mgr.z' cmd={"prefix": "config-key get", "key": "mgr/dashboard/z/key"} : dispatch 2024-09-21T03:37:37.895 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:37:37 smithi150 ceph-mon[34289]: from='mgr.? 172.21.15.161:0/3017326223' entity='mgr.z' cmd={"prefix": "config-key get", "key": "mgr/dashboard/key"} : dispatch 2024-09-21T03:37:37.895 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:37:37 smithi150 ceph-mon[34289]: from='client.? 172.21.15.39:0/1825647635' entity='client.admin' 2024-09-21T03:37:37.965 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:37.963+0000 7fbe965a8640 1 -- 172.21.15.39:0/3409249567 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/log_level}] v 0) -- 0x7fbe901a9330 con 0x7fbe90107bd0 2024-09-21T03:37:37.966 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:37:37 smithi039 ceph-mon[31211]: from='client.? 172.21.15.39:0/1554142917' entity='client.admin' 2024-09-21T03:37:37.966 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:37:37 smithi039 ceph-mon[31211]: Standby manager daemon z started 2024-09-21T03:37:37.966 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:37:37 smithi039 ceph-mon[31211]: from='mgr.? 172.21.15.161:0/3017326223' entity='mgr.z' cmd={"prefix": "config-key get", "key": "mgr/dashboard/z/crt"} : dispatch 2024-09-21T03:37:37.966 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:37:37 smithi039 ceph-mon[31211]: from='mgr.? 172.21.15.161:0/3017326223' entity='mgr.z' cmd={"prefix": "config-key get", "key": "mgr/dashboard/crt"} : dispatch 2024-09-21T03:37:37.966 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:37:37 smithi039 ceph-mon[31211]: from='mgr.? 172.21.15.161:0/3017326223' entity='mgr.z' cmd={"prefix": "config-key get", "key": "mgr/dashboard/z/key"} : dispatch 2024-09-21T03:37:37.966 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:37:37 smithi039 ceph-mon[31211]: from='mgr.? 172.21.15.161:0/3017326223' entity='mgr.z' cmd={"prefix": "config-key get", "key": "mgr/dashboard/key"} : dispatch 2024-09-21T03:37:37.966 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:37:37 smithi039 ceph-mon[31211]: from='client.? 172.21.15.39:0/1825647635' entity='client.admin' 2024-09-21T03:37:37.967 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:37:37 smithi161 ceph-mon[34533]: from='client.? 172.21.15.39:0/1554142917' entity='client.admin' 2024-09-21T03:37:37.968 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:37:37 smithi161 ceph-mon[34533]: Standby manager daemon z started 2024-09-21T03:37:37.968 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:37:37 smithi161 ceph-mon[34533]: from='mgr.? 172.21.15.161:0/3017326223' entity='mgr.z' cmd={"prefix": "config-key get", "key": "mgr/dashboard/z/crt"} : dispatch 2024-09-21T03:37:37.968 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:37:37 smithi161 ceph-mon[34533]: from='mgr.? 172.21.15.161:0/3017326223' entity='mgr.z' cmd={"prefix": "config-key get", "key": "mgr/dashboard/crt"} : dispatch 2024-09-21T03:37:37.968 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:37:37 smithi161 ceph-mon[34533]: from='mgr.? 172.21.15.161:0/3017326223' entity='mgr.z' cmd={"prefix": "config-key get", "key": "mgr/dashboard/z/key"} : dispatch 2024-09-21T03:37:37.968 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:37:37 smithi161 ceph-mon[34533]: from='mgr.? 172.21.15.161:0/3017326223' entity='mgr.z' cmd={"prefix": "config-key get", "key": "mgr/dashboard/key"} : dispatch 2024-09-21T03:37:37.968 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:37:37 smithi161 ceph-mon[34533]: from='client.? 172.21.15.39:0/1825647635' entity='client.admin' 2024-09-21T03:37:37.970 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:37.967+0000 7fbe8dffb640 1 -- 172.21.15.39:0/3409249567 <== mon.0 v2:172.21.15.39:3300/0 8 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/log_level}]=0 v21)=0 v21) ==== 129+0+0 (secure 0 0 0) 0x7fbe7805a800 con 0x7fbe90107bd0 2024-09-21T03:37:37.972 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:37.970+0000 7fbe965a8640 1 -- 172.21.15.39:0/3409249567 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7fbe7c03fb70 msgr2=0x7fbe7c042080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:37:37.973 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:37.970+0000 7fbe965a8640 1 --2- 172.21.15.39:0/3409249567 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7fbe7c03fb70 0x7fbe7c042080 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fbe80005d00 tx=0x7fbe8000b410 comp rx=0 tx=0).stop 2024-09-21T03:37:37.973 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:37.970+0000 7fbe965a8640 1 -- 172.21.15.39:0/3409249567 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbe90107bd0 msgr2=0x7fbe9019cd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:37:37.973 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:37.970+0000 7fbe965a8640 1 --2- 172.21.15.39:0/3409249567 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbe90107bd0 0x7fbe9019cd40 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fbe78005ab0 tx=0x7fbe78003cd0 comp rx=0 tx=0).stop 2024-09-21T03:37:37.973 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:37.970+0000 7fbe965a8640 1 -- 172.21.15.39:0/3409249567 shutdown_connections 2024-09-21T03:37:37.973 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:37.970+0000 7fbe965a8640 1 --2- 172.21.15.39:0/3409249567 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7fbe7c03fb70 0x7fbe7c042080 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:37.973 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:37.970+0000 7fbe965a8640 1 --2- 172.21.15.39:0/3409249567 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbe90108520 0x7fbe901a1fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:37.973 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:37.970+0000 7fbe965a8640 1 --2- 172.21.15.39:0/3409249567 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbe90107bd0 0x7fbe9019cd40 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:37.973 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:37.970+0000 7fbe965a8640 1 --2- 172.21.15.39:0/3409249567 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbe90102290 0x7fbe9019c7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:37.973 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:37.970+0000 7fbe965a8640 1 -- 172.21.15.39:0/3409249567 >> 172.21.15.39:0/3409249567 conn(0x7fbe90079ea0 msgr2=0x7fbe901063a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:37:37.973 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:37.970+0000 7fbe965a8640 1 -- 172.21.15.39:0/3409249567 shutdown_connections 2024-09-21T03:37:37.973 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:37.970+0000 7fbe965a8640 1 -- 172.21.15.39:0/3409249567 wait complete. 2024-09-21T03:37:38.362 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:38.359+0000 7f34bdc56640 1 -- 172.21.15.39:0/3492786242 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f34b81031b0 msgr2=0x7f34b81035c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:37:38.362 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:38.359+0000 7f34bdc56640 1 --2- 172.21.15.39:0/3492786242 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f34b81031b0 0x7f34b81035c0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f34a8009900 tx=0x7f34a802a410 comp rx=0 tx=0).stop 2024-09-21T03:37:38.362 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:38.359+0000 7f34bdc56640 1 -- 172.21.15.39:0/3492786242 shutdown_connections 2024-09-21T03:37:38.362 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:38.359+0000 7f34bdc56640 1 --2- 172.21.15.39:0/3492786242 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f34b8105b10 0x7f34b810f5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:38.362 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:38.359+0000 7f34bdc56640 1 --2- 172.21.15.39:0/3492786242 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f34b81051c0 0x7f34b81055d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:38.362 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:38.359+0000 7f34bdc56640 1 --2- 172.21.15.39:0/3492786242 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f34b81031b0 0x7f34b81035c0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:38.362 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:38.359+0000 7f34bdc56640 1 -- 172.21.15.39:0/3492786242 >> 172.21.15.39:0/3492786242 conn(0x7f34b8079e40 msgr2=0x7f34b807a250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:37:38.362 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:38.359+0000 7f34bdc56640 1 -- 172.21.15.39:0/3492786242 shutdown_connections 2024-09-21T03:37:38.363 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:38.359+0000 7f34bdc56640 1 -- 172.21.15.39:0/3492786242 wait complete. 2024-09-21T03:37:38.364 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:38.360+0000 7f34bdc56640 1 Processor -- start 2024-09-21T03:37:38.364 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:38.360+0000 7f34bdc56640 1 -- start start 2024-09-21T03:37:38.365 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:38.360+0000 7f34bdc56640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f34b81031b0 0x7f34b819a540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:37:38.365 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:38.360+0000 7f34bdc56640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f34b81051c0 0x7f34b819aae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:37:38.365 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:38.360+0000 7f34bdc56640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f34b8105b10 0x7f34b819fd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:37:38.365 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:38.360+0000 7f34bdc56640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f34b8113b10 con 0x7f34b81051c0 2024-09-21T03:37:38.365 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:38.360+0000 7f34bdc56640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f34b8113990 con 0x7f34b8105b10 2024-09-21T03:37:38.365 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:38.360+0000 7f34bdc56640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f34b8113c90 con 0x7f34b81031b0 2024-09-21T03:37:38.365 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:38.360+0000 7f34b6ffd640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f34b81051c0 0x7f34b819aae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:37:38.365 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:38.360+0000 7f34b6ffd640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f34b81051c0 0x7f34b819aae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:46600/0 (socket says 172.21.15.39:46600) 2024-09-21T03:37:38.365 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:38.360+0000 7f34b6ffd640 1 -- 172.21.15.39:0/1026439920 learned_addr learned my addr 172.21.15.39:0/1026439920 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:37:38.365 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:38.360+0000 7f34b7fff640 1 --2- 172.21.15.39:0/1026439920 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f34b8105b10 0x7f34b819fd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:37:38.365 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:38.360+0000 7f34b77fe640 1 --2- 172.21.15.39:0/1026439920 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f34b81031b0 0x7f34b819a540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:37:38.365 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:38.361+0000 7f34b6ffd640 1 -- 172.21.15.39:0/1026439920 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f34b81031b0 msgr2=0x7f34b819a540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:37:38.366 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:38.361+0000 7f34b6ffd640 1 --2- 172.21.15.39:0/1026439920 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f34b81031b0 0x7f34b819a540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:38.366 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:38.361+0000 7f34b6ffd640 1 -- 172.21.15.39:0/1026439920 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f34b8105b10 msgr2=0x7f34b819fd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:37:38.366 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:38.361+0000 7f34b6ffd640 1 --2- 172.21.15.39:0/1026439920 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f34b8105b10 0x7f34b819fd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:38.366 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:38.361+0000 7f34b6ffd640 1 -- 172.21.15.39:0/1026439920 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f34b81a04b0 con 0x7f34b81051c0 2024-09-21T03:37:38.366 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:38.361+0000 7f34b6ffd640 1 --2- 172.21.15.39:0/1026439920 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f34b81051c0 0x7f34b819aae0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f34a000e9f0 tx=0x7f34a000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:37:38.366 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:38.361+0000 7f34b4ff9640 1 -- 172.21.15.39:0/1026439920 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34a000ccc0 con 0x7f34b81051c0 2024-09-21T03:37:38.366 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:38.361+0000 7f34b4ff9640 1 -- 172.21.15.39:0/1026439920 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f34a00108e0 con 0x7f34b81051c0 2024-09-21T03:37:38.366 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:38.361+0000 7f34bdc56640 1 -- 172.21.15.39:0/1026439920 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f34b81a07a0 con 0x7f34b81051c0 2024-09-21T03:37:38.366 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:38.361+0000 7f34b4ff9640 1 -- 172.21.15.39:0/1026439920 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34a0018860 con 0x7f34b81051c0 2024-09-21T03:37:38.366 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:38.361+0000 7f34bdc56640 1 -- 172.21.15.39:0/1026439920 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f34b81a6f00 con 0x7f34b81051c0 2024-09-21T03:37:38.367 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:38.362+0000 7f34bdc56640 1 -- 172.21.15.39:0/1026439920 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f34b81035c0 con 0x7f34b81051c0 2024-09-21T03:37:38.367 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:38.363+0000 7f34b4ff9640 1 -- 172.21.15.39:0/1026439920 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 19) ==== 103194+0+0 (secure 0 0 0) 0x7f34a0010430 con 0x7f34b81051c0 2024-09-21T03:37:38.367 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:38.364+0000 7f34b4ff9640 1 --2- 172.21.15.39:0/1026439920 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f34a407b020 0x7f34a407d530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:37:38.367 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:38.364+0000 7f34b4ff9640 1 -- 172.21.15.39:0/1026439920 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1370+0+0 (secure 0 0 0) 0x7f34a009d0c0 con 0x7f34b81051c0 2024-09-21T03:37:38.367 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:38.364+0000 7f34b77fe640 1 --2- 172.21.15.39:0/1026439920 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f34a407b020 0x7f34a407d530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:37:38.368 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:38.364+0000 7f34b77fe640 1 --2- 172.21.15.39:0/1026439920 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f34a407b020 0x7f34a407d530 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f34a8004770 tx=0x7f34a80058d0 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:37:38.371 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:38.369+0000 7f34b4ff9640 1 -- 172.21.15.39:0/1026439920 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f34a0069ad0 con 0x7f34b81051c0 2024-09-21T03:37:38.569 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:38.566+0000 7f34bdc56640 1 -- 172.21.15.39:0/1026439920 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}] v 0) -- 0x7f34b810ac30 con 0x7f34b81051c0 2024-09-21T03:37:38.573 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:38.570+0000 7f34b4ff9640 1 -- 172.21.15.39:0/1026439920 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}]=0 v22)=0 v22) ==== 138+0+0 (secure 0 0 0) 0x7f34a006e980 con 0x7f34b81051c0 2024-09-21T03:37:38.575 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:38.573+0000 7f34bdc56640 1 -- 172.21.15.39:0/1026439920 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f34a407b020 msgr2=0x7f34a407d530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:37:38.576 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:38.573+0000 7f34bdc56640 1 --2- 172.21.15.39:0/1026439920 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f34a407b020 0x7f34a407d530 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f34a8004770 tx=0x7f34a80058d0 comp rx=0 tx=0).stop 2024-09-21T03:37:38.576 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:38.573+0000 7f34bdc56640 1 -- 172.21.15.39:0/1026439920 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f34b81051c0 msgr2=0x7f34b819aae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:37:38.576 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:38.573+0000 7f34bdc56640 1 --2- 172.21.15.39:0/1026439920 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f34b81051c0 0x7f34b819aae0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f34a000e9f0 tx=0x7f34a000eeb0 comp rx=0 tx=0).stop 2024-09-21T03:37:38.576 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:38.573+0000 7f34bdc56640 1 -- 172.21.15.39:0/1026439920 shutdown_connections 2024-09-21T03:37:38.576 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:38.573+0000 7f34bdc56640 1 --2- 172.21.15.39:0/1026439920 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f34a407b020 0x7f34a407d530 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:38.576 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:38.573+0000 7f34bdc56640 1 --2- 172.21.15.39:0/1026439920 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f34b8105b10 0x7f34b819fd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:38.576 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:38.573+0000 7f34bdc56640 1 --2- 172.21.15.39:0/1026439920 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f34b81051c0 0x7f34b819aae0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:38.576 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:38.573+0000 7f34bdc56640 1 --2- 172.21.15.39:0/1026439920 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f34b81031b0 0x7f34b819a540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:38.576 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:38.573+0000 7f34bdc56640 1 -- 172.21.15.39:0/1026439920 >> 172.21.15.39:0/1026439920 conn(0x7f34b8079e40 msgr2=0x7f34b8106490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:37:38.576 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:38.573+0000 7f34bdc56640 1 -- 172.21.15.39:0/1026439920 shutdown_connections 2024-09-21T03:37:38.576 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:38.573+0000 7f34bdc56640 1 -- 172.21.15.39:0/1026439920 wait complete. 2024-09-21T03:37:38.864 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:38.862+0000 7f85c5636640 1 -- 172.21.15.39:0/1744947619 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f85c01048c0 msgr2=0x7f85c0104cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:37:38.865 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:38.862+0000 7f85c5636640 1 --2- 172.21.15.39:0/1744947619 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f85c01048c0 0x7f85c0104cd0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f85b0009900 tx=0x7f85b002a410 comp rx=0 tx=0).stop 2024-09-21T03:37:38.865 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:38.862+0000 7f85c5636640 1 -- 172.21.15.39:0/1744947619 shutdown_connections 2024-09-21T03:37:38.865 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:38.862+0000 7f85c5636640 1 --2- 172.21.15.39:0/1744947619 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f85c010de70 0x7f85c0110340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:38.865 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:38.862+0000 7f85c5636640 1 --2- 172.21.15.39:0/1744947619 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f85c01052a0 0x7f85c010d930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:38.865 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:38.862+0000 7f85c5636640 1 --2- 172.21.15.39:0/1744947619 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f85c01048c0 0x7f85c0104cd0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:38.865 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:38.862+0000 7f85c5636640 1 -- 172.21.15.39:0/1744947619 >> 172.21.15.39:0/1744947619 conn(0x7f85c0079790 msgr2=0x7f85c0079ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:37:38.865 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:38.862+0000 7f85c5636640 1 -- 172.21.15.39:0/1744947619 shutdown_connections 2024-09-21T03:37:38.865 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:38.862+0000 7f85c5636640 1 -- 172.21.15.39:0/1744947619 wait complete. 2024-09-21T03:37:38.867 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:38.863+0000 7f85c5636640 1 Processor -- start 2024-09-21T03:37:38.867 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:38.863+0000 7f85c5636640 1 -- start start 2024-09-21T03:37:38.867 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:38.863+0000 7f85c5636640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f85c01048c0 0x7f85c019c670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:37:38.867 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:38.863+0000 7f85c5636640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f85c01052a0 0x7f85c019cc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:37:38.867 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:38.863+0000 7f85c5636640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f85c010de70 0x7f85c01a1eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:37:38.867 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:38.863+0000 7f85c5636640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f85c0113ab0 con 0x7f85c01052a0 2024-09-21T03:37:38.867 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:38.863+0000 7f85c5636640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f85c0113930 con 0x7f85c010de70 2024-09-21T03:37:38.867 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:38.863+0000 7f85c5636640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f85c0113c30 con 0x7f85c01048c0 2024-09-21T03:37:38.867 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:38.863+0000 7f85b7fff640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f85c01052a0 0x7f85c019cc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:37:38.867 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:38.863+0000 7f85b7fff640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f85c01052a0 0x7f85c019cc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:46618/0 (socket says 172.21.15.39:46618) 2024-09-21T03:37:38.867 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:38.863+0000 7f85b7fff640 1 -- 172.21.15.39:0/3313852442 learned_addr learned my addr 172.21.15.39:0/3313852442 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:37:38.867 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:38.863+0000 7f85bf7fe640 1 --2- 172.21.15.39:0/3313852442 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f85c010de70 0x7f85c01a1eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:37:38.868 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:38.863+0000 7f85beffd640 1 --2- 172.21.15.39:0/3313852442 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f85c01048c0 0x7f85c019c670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:37:38.868 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:38.863+0000 7f85b7fff640 1 -- 172.21.15.39:0/3313852442 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f85c01048c0 msgr2=0x7f85c019c670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:37:38.868 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:38.864+0000 7f85b7fff640 1 --2- 172.21.15.39:0/3313852442 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f85c01048c0 0x7f85c019c670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:38.868 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:38.864+0000 7f85b7fff640 1 -- 172.21.15.39:0/3313852442 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f85c010de70 msgr2=0x7f85c01a1eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:37:38.868 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:38.864+0000 7f85b7fff640 1 --2- 172.21.15.39:0/3313852442 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f85c010de70 0x7f85c01a1eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:38.868 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:38.864+0000 7f85b7fff640 1 -- 172.21.15.39:0/3313852442 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f85c01a25e0 con 0x7f85c01052a0 2024-09-21T03:37:38.868 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:38.864+0000 7f85b7fff640 1 --2- 172.21.15.39:0/3313852442 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f85c01052a0 0x7f85c019cc10 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f85a800d950 tx=0x7f85a800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:37:38.868 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:38.864+0000 7f85bcff9640 1 -- 172.21.15.39:0/3313852442 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85a8017070 con 0x7f85c01052a0 2024-09-21T03:37:38.868 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:38.864+0000 7f85bcff9640 1 -- 172.21.15.39:0/3313852442 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f85a8009c30 con 0x7f85c01052a0 2024-09-21T03:37:38.868 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:38.864+0000 7f85bcff9640 1 -- 172.21.15.39:0/3313852442 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85a80047a0 con 0x7f85c01052a0 2024-09-21T03:37:38.868 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:38.864+0000 7f85c5636640 1 -- 172.21.15.39:0/3313852442 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f85c01a28d0 con 0x7f85c01052a0 2024-09-21T03:37:38.868 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:38.864+0000 7f85c5636640 1 -- 172.21.15.39:0/3313852442 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f85c01a9030 con 0x7f85c01052a0 2024-09-21T03:37:38.869 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:38.865+0000 7f85c5636640 1 -- 172.21.15.39:0/3313852442 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f85c0104cd0 con 0x7f85c01052a0 2024-09-21T03:37:38.869 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:38.866+0000 7f85bcff9640 1 -- 172.21.15.39:0/3313852442 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 19) ==== 103194+0+0 (secure 0 0 0) 0x7f85a8004020 con 0x7f85c01052a0 2024-09-21T03:37:38.869 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:38.867+0000 7f85bcff9640 1 --2- 172.21.15.39:0/3313852442 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f85ac07b020 0x7f85ac07d530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:37:38.869 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:38.867+0000 7f85beffd640 1 --2- 172.21.15.39:0/3313852442 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f85ac07b020 0x7f85ac07d530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:37:38.869 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:38.867+0000 7f85bcff9640 1 -- 172.21.15.39:0/3313852442 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1370+0+0 (secure 0 0 0) 0x7f85a809ca10 con 0x7f85c01052a0 2024-09-21T03:37:38.870 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:38.867+0000 7f85beffd640 1 --2- 172.21.15.39:0/3313852442 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f85ac07b020 0x7f85ac07d530 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f85b0031440 tx=0x7f85b0009560 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:37:38.874 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:38.871+0000 7f85bcff9640 1 -- 172.21.15.39:0/3313852442 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f85a8020020 con 0x7f85c01052a0 2024-09-21T03:37:38.964 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/51116d5c-77ca-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:37:38.964 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/51116d5c-77ca-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:37:38.965 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/51116d5c-77ca-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:37:38.965 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/51116d5c-77ca-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:37:39.071 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:39.063+0000 7f5161450640 1 -- 172.21.15.39:0/1580954859 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f515c1083d0 msgr2=0x7f515c1087e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:37:39.071 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:39.063+0000 7f5161450640 1 --2- 172.21.15.39:0/1580954859 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f515c1083d0 0x7f515c1087e0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f5144009900 tx=0x7f514402a410 comp rx=0 tx=0).stop 2024-09-21T03:37:39.071 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:39.064+0000 7f5161450640 1 -- 172.21.15.39:0/1580954859 shutdown_connections 2024-09-21T03:37:39.071 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:39.064+0000 7f5161450640 1 --2- 172.21.15.39:0/1580954859 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f515c108d20 0x7f515c10f550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:39.071 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:39.064+0000 7f5161450640 1 --2- 172.21.15.39:0/1580954859 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f515c1083d0 0x7f515c1087e0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:39.072 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:39.064+0000 7f5161450640 1 --2- 172.21.15.39:0/1580954859 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f515c102ae0 0x7f515c102ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:39.072 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:39.064+0000 7f5161450640 1 -- 172.21.15.39:0/1580954859 >> 172.21.15.39:0/1580954859 conn(0x7f515c0797b0 msgr2=0x7f515c079bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:37:39.072 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:39.064+0000 7f5161450640 1 -- 172.21.15.39:0/1580954859 shutdown_connections 2024-09-21T03:37:39.072 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:39.064+0000 7f5161450640 1 -- 172.21.15.39:0/1580954859 wait complete. 2024-09-21T03:37:39.072 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:39.064+0000 7f5161450640 1 Processor -- start 2024-09-21T03:37:39.072 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:39.064+0000 7f5161450640 1 -- start start 2024-09-21T03:37:39.072 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:39.065+0000 7f5161450640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f515c102ae0 0x7f515c0ff670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:37:39.072 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:39.065+0000 7f5161450640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f515c1083d0 0x7f515c0ffbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:37:39.072 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:39.065+0000 7f5161450640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f515c108d20 0x7f515c1000f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:37:39.072 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:39.065+0000 7f5161450640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f515c1129e0 con 0x7f515c108d20 2024-09-21T03:37:39.072 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:39.065+0000 7f5161450640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f515c112860 con 0x7f515c102ae0 2024-09-21T03:37:39.072 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:39.065+0000 7f5161450640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f515c112b60 con 0x7f515c1083d0 2024-09-21T03:37:39.072 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:39.065+0000 7f515affd640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f515c102ae0 0x7f515c0ff670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:37:39.072 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:39.065+0000 7f515b7fe640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f515c108d20 0x7f515c1000f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:37:39.072 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:39.065+0000 7f515affd640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f515c102ae0 0x7f515c0ff670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.39:54628/0 (socket says 172.21.15.39:54628) 2024-09-21T03:37:39.072 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:39.065+0000 7f515affd640 1 -- 172.21.15.39:0/2885535905 learned_addr learned my addr 172.21.15.39:0/2885535905 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:37:39.073 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:39.065+0000 7f515b7fe640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f515c108d20 0x7f515c1000f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:46636/0 (socket says 172.21.15.39:46636) 2024-09-21T03:37:39.073 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:39.065+0000 7f51527fc640 1 --2- 172.21.15.39:0/2885535905 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f515c1083d0 0x7f515c0ffbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:37:39.073 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:39.065+0000 7f515b7fe640 1 -- 172.21.15.39:0/2885535905 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f515c1083d0 msgr2=0x7f515c0ffbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:37:39.073 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:39.065+0000 7f515b7fe640 1 --2- 172.21.15.39:0/2885535905 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f515c1083d0 0x7f515c0ffbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:39.073 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:39.065+0000 7f515b7fe640 1 -- 172.21.15.39:0/2885535905 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f515c102ae0 msgr2=0x7f515c0ff670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:37:39.073 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:39.065+0000 7f515b7fe640 1 --2- 172.21.15.39:0/2885535905 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f515c102ae0 0x7f515c0ff670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:39.073 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:39.065+0000 7f515b7fe640 1 -- 172.21.15.39:0/2885535905 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f515c1a86b0 con 0x7f515c108d20 2024-09-21T03:37:39.073 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:39.065+0000 7f515affd640 1 --2- 172.21.15.39:0/2885535905 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f515c102ae0 0x7f515c0ff670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:37:39.073 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:39.066+0000 7f515b7fe640 1 --2- 172.21.15.39:0/2885535905 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f515c108d20 0x7f515c1000f0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f515400e940 tx=0x7f515400ee00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:37:39.073 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:39.066+0000 7f5158ff9640 1 -- 172.21.15.39:0/2885535905 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5154004d20 con 0x7f515c108d20 2024-09-21T03:37:39.073 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:39.066+0000 7f5158ff9640 1 -- 172.21.15.39:0/2885535905 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5154005020 con 0x7f515c108d20 2024-09-21T03:37:39.073 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:39.066+0000 7f5158ff9640 1 -- 172.21.15.39:0/2885535905 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51540116d0 con 0x7f515c108d20 2024-09-21T03:37:39.073 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:39.066+0000 7f5161450640 1 -- 172.21.15.39:0/2885535905 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f515c1a89a0 con 0x7f515c108d20 2024-09-21T03:37:39.073 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:39.066+0000 7f5161450640 1 -- 172.21.15.39:0/2885535905 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f515c1a8eb0 con 0x7f515c108d20 2024-09-21T03:37:39.073 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:39.067+0000 7f5161450640 1 -- 172.21.15.39:0/2885535905 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f515c102ef0 con 0x7f515c108d20 2024-09-21T03:37:39.074 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:39.068+0000 7f5158ff9640 1 -- 172.21.15.39:0/2885535905 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 19) ==== 103194+0+0 (secure 0 0 0) 0x7f5154005460 con 0x7f515c108d20 2024-09-21T03:37:39.074 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:39.068+0000 7f5158ff9640 1 --2- 172.21.15.39:0/2885535905 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f514807b020 0x7f514807d530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:37:39.074 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:39.069+0000 7f5158ff9640 1 -- 172.21.15.39:0/2885535905 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1370+0+0 (secure 0 0 0) 0x7f515409cea0 con 0x7f515c108d20 2024-09-21T03:37:39.074 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:39.069+0000 7f515affd640 1 --2- 172.21.15.39:0/2885535905 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f514807b020 0x7f514807d530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:37:39.074 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:39.069+0000 7f515affd640 1 --2- 172.21.15.39:0/2885535905 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f514807b020 0x7f514807d530 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f515c1010c0 tx=0x7f514c00a380 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:37:39.077 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:39.074+0000 7f5158ff9640 1 -- 172.21.15.39:0/2885535905 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f5154016030 con 0x7f515c108d20 2024-09-21T03:37:39.083 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:39.080+0000 7f85c5636640 1 -- 172.21.15.39:0/3313852442 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command([{prefix=config set, name=mds_debug_scatterstat}] v 0) -- 0x7f85c01a9200 con 0x7f85c01052a0 2024-09-21T03:37:39.088 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:39.086+0000 7f85bcff9640 1 -- 172.21.15.39:0/3313852442 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_debug_scatterstat}]=0 v23)=0 v23) ==== 122+0+0 (secure 0 0 0) 0x7f85a8009de0 con 0x7f85c01052a0 2024-09-21T03:37:39.092 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:39.089+0000 7f85c5636640 1 -- 172.21.15.39:0/3313852442 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f85ac07b020 msgr2=0x7f85ac07d530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:37:39.092 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:39.089+0000 7f85c5636640 1 --2- 172.21.15.39:0/3313852442 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f85ac07b020 0x7f85ac07d530 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f85b0031440 tx=0x7f85b0009560 comp rx=0 tx=0).stop 2024-09-21T03:37:39.092 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:39.089+0000 7f85c5636640 1 -- 172.21.15.39:0/3313852442 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f85c01052a0 msgr2=0x7f85c019cc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:37:39.092 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:39.089+0000 7f85c5636640 1 --2- 172.21.15.39:0/3313852442 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f85c01052a0 0x7f85c019cc10 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f85a800d950 tx=0x7f85a800de10 comp rx=0 tx=0).stop 2024-09-21T03:37:39.092 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:39.089+0000 7f85c5636640 1 -- 172.21.15.39:0/3313852442 shutdown_connections 2024-09-21T03:37:39.092 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:39.089+0000 7f85c5636640 1 --2- 172.21.15.39:0/3313852442 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f85ac07b020 0x7f85ac07d530 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:39.092 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:39.089+0000 7f85c5636640 1 --2- 172.21.15.39:0/3313852442 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f85c010de70 0x7f85c01a1eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:39.092 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:39.089+0000 7f85c5636640 1 --2- 172.21.15.39:0/3313852442 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f85c01052a0 0x7f85c019cc10 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:39.092 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:39.089+0000 7f85c5636640 1 --2- 172.21.15.39:0/3313852442 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f85c01048c0 0x7f85c019c670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:39.092 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:39.089+0000 7f85c5636640 1 -- 172.21.15.39:0/3313852442 >> 172.21.15.39:0/3313852442 conn(0x7f85c0079790 msgr2=0x7f85c01005f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:37:39.092 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:39.089+0000 7f85c5636640 1 -- 172.21.15.39:0/3313852442 shutdown_connections 2024-09-21T03:37:39.092 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:39.089+0000 7f85c5636640 1 -- 172.21.15.39:0/3313852442 wait complete. 2024-09-21T03:37:39.173 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:37:38 smithi039 ceph-mon[31211]: mgrmap e19: x(active, since 12s), standbys: z 2024-09-21T03:37:39.173 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:37:38 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "mgr metadata", "who": "z", "id": "z"} : dispatch 2024-09-21T03:37:39.173 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:37:38 smithi039 ceph-mon[31211]: from='client.? 172.21.15.39:0/3409249567' entity='client.admin' 2024-09-21T03:37:39.173 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:37:38 smithi039 ceph-mon[31211]: from='client.? 172.21.15.39:0/1026439920' entity='client.admin' 2024-09-21T03:37:39.217 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:37:38 smithi161 ceph-mon[34533]: mgrmap e19: x(active, since 12s), standbys: z 2024-09-21T03:37:39.217 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:37:38 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "mgr metadata", "who": "z", "id": "z"} : dispatch 2024-09-21T03:37:39.218 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:37:38 smithi161 ceph-mon[34533]: from='client.? 172.21.15.39:0/3409249567' entity='client.admin' 2024-09-21T03:37:39.218 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:37:38 smithi161 ceph-mon[34533]: from='client.? 172.21.15.39:0/1026439920' entity='client.admin' 2024-09-21T03:37:39.224 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:37:38 smithi150 ceph-mon[34289]: mgrmap e19: x(active, since 12s), standbys: z 2024-09-21T03:37:39.224 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:37:38 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "mgr metadata", "who": "z", "id": "z"} : dispatch 2024-09-21T03:37:39.225 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:37:38 smithi150 ceph-mon[34289]: from='client.? 172.21.15.39:0/3409249567' entity='client.admin' 2024-09-21T03:37:39.225 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:37:38 smithi150 ceph-mon[34289]: from='client.? 172.21.15.39:0/1026439920' entity='client.admin' 2024-09-21T03:37:39.282 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:39.279+0000 7f5161450640 1 -- 172.21.15.39:0/2885535905 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command([{prefix=config set, name=mds_bal_split_bits}] v 0) -- 0x7f515c10bb80 con 0x7f515c108d20 2024-09-21T03:37:39.286 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:39.283+0000 7f5158ff9640 1 -- 172.21.15.39:0/2885535905 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_bal_split_bits}]=0 v24)=0 v24) ==== 116+0+0 (secure 0 0 0) 0x7f51540698b0 con 0x7f515c108d20 2024-09-21T03:37:39.288 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:39.286+0000 7f5161450640 1 -- 172.21.15.39:0/2885535905 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f514807b020 msgr2=0x7f514807d530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:37:39.288 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:39.286+0000 7f5161450640 1 --2- 172.21.15.39:0/2885535905 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f514807b020 0x7f514807d530 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f515c1010c0 tx=0x7f514c00a380 comp rx=0 tx=0).stop 2024-09-21T03:37:39.288 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:39.286+0000 7f5161450640 1 -- 172.21.15.39:0/2885535905 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f515c108d20 msgr2=0x7f515c1000f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:37:39.288 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:39.286+0000 7f5161450640 1 --2- 172.21.15.39:0/2885535905 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f515c108d20 0x7f515c1000f0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f515400e940 tx=0x7f515400ee00 comp rx=0 tx=0).stop 2024-09-21T03:37:39.288 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:39.286+0000 7f5161450640 1 -- 172.21.15.39:0/2885535905 shutdown_connections 2024-09-21T03:37:39.289 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:39.286+0000 7f5161450640 1 --2- 172.21.15.39:0/2885535905 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f514807b020 0x7f514807d530 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:39.289 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:39.286+0000 7f5161450640 1 --2- 172.21.15.39:0/2885535905 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f515c108d20 0x7f515c1000f0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:39.289 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:39.286+0000 7f5161450640 1 --2- 172.21.15.39:0/2885535905 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f515c1083d0 0x7f515c0ffbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:39.289 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:39.286+0000 7f5161450640 1 --2- 172.21.15.39:0/2885535905 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f515c102ae0 0x7f515c0ff670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:39.289 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:39.286+0000 7f5161450640 1 -- 172.21.15.39:0/2885535905 >> 172.21.15.39:0/2885535905 conn(0x7f515c0797b0 msgr2=0x7f515c101b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:37:39.289 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:39.286+0000 7f5161450640 1 -- 172.21.15.39:0/2885535905 shutdown_connections 2024-09-21T03:37:39.289 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:39.286+0000 7f5161450640 1 -- 172.21.15.39:0/2885535905 wait complete. 2024-09-21T03:37:40.468 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:37:40 smithi161 ceph-mon[34533]: from='client.? 172.21.15.39:0/3313852442' entity='client.admin' 2024-09-21T03:37:40.468 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:37:40 smithi161 ceph-mon[34533]: Standby manager daemon y started 2024-09-21T03:37:40.468 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:37:40 smithi161 ceph-mon[34533]: from='mgr.? 172.21.15.150:0/1823180593' entity='mgr.y' cmd={"prefix": "config-key get", "key": "mgr/dashboard/y/crt"} : dispatch 2024-09-21T03:37:40.468 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:37:40 smithi161 ceph-mon[34533]: from='mgr.? 172.21.15.150:0/1823180593' entity='mgr.y' cmd={"prefix": "config-key get", "key": "mgr/dashboard/crt"} : dispatch 2024-09-21T03:37:40.468 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:37:40 smithi161 ceph-mon[34533]: from='mgr.? 172.21.15.150:0/1823180593' entity='mgr.y' cmd={"prefix": "config-key get", "key": "mgr/dashboard/y/key"} : dispatch 2024-09-21T03:37:40.468 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:37:40 smithi161 ceph-mon[34533]: from='mgr.? 172.21.15.150:0/1823180593' entity='mgr.y' cmd={"prefix": "config-key get", "key": "mgr/dashboard/key"} : dispatch 2024-09-21T03:37:40.468 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:37:40 smithi161 ceph-mon[34533]: from='client.? 172.21.15.39:0/2885535905' entity='client.admin' 2024-09-21T03:37:40.474 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:37:40 smithi150 ceph-mon[34289]: from='client.? 172.21.15.39:0/3313852442' entity='client.admin' 2024-09-21T03:37:40.474 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:37:40 smithi150 ceph-mon[34289]: Standby manager daemon y started 2024-09-21T03:37:40.474 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:37:40 smithi150 ceph-mon[34289]: from='mgr.? 172.21.15.150:0/1823180593' entity='mgr.y' cmd={"prefix": "config-key get", "key": "mgr/dashboard/y/crt"} : dispatch 2024-09-21T03:37:40.474 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:37:40 smithi150 ceph-mon[34289]: from='mgr.? 172.21.15.150:0/1823180593' entity='mgr.y' cmd={"prefix": "config-key get", "key": "mgr/dashboard/crt"} : dispatch 2024-09-21T03:37:40.474 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:37:40 smithi150 ceph-mon[34289]: from='mgr.? 172.21.15.150:0/1823180593' entity='mgr.y' cmd={"prefix": "config-key get", "key": "mgr/dashboard/y/key"} : dispatch 2024-09-21T03:37:40.475 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:37:40 smithi150 ceph-mon[34289]: from='mgr.? 172.21.15.150:0/1823180593' entity='mgr.y' cmd={"prefix": "config-key get", "key": "mgr/dashboard/key"} : dispatch 2024-09-21T03:37:40.475 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:37:40 smithi150 ceph-mon[34289]: from='client.? 172.21.15.39:0/2885535905' entity='client.admin' 2024-09-21T03:37:40.513 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:37:40 smithi039 ceph-mon[31211]: from='client.? 172.21.15.39:0/3313852442' entity='client.admin' 2024-09-21T03:37:40.513 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:37:40 smithi039 ceph-mon[31211]: Standby manager daemon y started 2024-09-21T03:37:40.513 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:37:40 smithi039 ceph-mon[31211]: from='mgr.? 172.21.15.150:0/1823180593' entity='mgr.y' cmd={"prefix": "config-key get", "key": "mgr/dashboard/y/crt"} : dispatch 2024-09-21T03:37:40.513 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:37:40 smithi039 ceph-mon[31211]: from='mgr.? 172.21.15.150:0/1823180593' entity='mgr.y' cmd={"prefix": "config-key get", "key": "mgr/dashboard/crt"} : dispatch 2024-09-21T03:37:40.513 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:37:40 smithi039 ceph-mon[31211]: from='mgr.? 172.21.15.150:0/1823180593' entity='mgr.y' cmd={"prefix": "config-key get", "key": "mgr/dashboard/y/key"} : dispatch 2024-09-21T03:37:40.513 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:37:40 smithi039 ceph-mon[31211]: from='mgr.? 172.21.15.150:0/1823180593' entity='mgr.y' cmd={"prefix": "config-key get", "key": "mgr/dashboard/key"} : dispatch 2024-09-21T03:37:40.513 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:37:40 smithi039 ceph-mon[31211]: from='client.? 172.21.15.39:0/2885535905' entity='client.admin' 2024-09-21T03:37:41.352 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:41.349+0000 7f688656e640 1 -- 172.21.15.39:0/713764479 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6880109e00 msgr2=0x7f6880110630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:37:41.352 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:41.349+0000 7f688656e640 1 --2- 172.21.15.39:0/713764479 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6880109e00 0x7f6880110630 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f6878009ca0 tx=0x7f687802a8c0 comp rx=0 tx=0).stop 2024-09-21T03:37:41.352 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:41.349+0000 7f688656e640 1 -- 172.21.15.39:0/713764479 shutdown_connections 2024-09-21T03:37:41.352 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:41.349+0000 7f688656e640 1 --2- 172.21.15.39:0/713764479 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6880109e00 0x7f6880110630 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:41.352 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:41.349+0000 7f688656e640 1 --2- 172.21.15.39:0/713764479 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f68801094b0 0x7f68801098c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:41.352 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:41.349+0000 7f688656e640 1 --2- 172.21.15.39:0/713764479 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6880103bc0 0x7f6880103fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:41.352 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:41.349+0000 7f688656e640 1 -- 172.21.15.39:0/713764479 >> 172.21.15.39:0/713764479 conn(0x7f688007b910 msgr2=0x7f688007bd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:37:41.352 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:41.349+0000 7f688656e640 1 -- 172.21.15.39:0/713764479 shutdown_connections 2024-09-21T03:37:41.353 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:41.349+0000 7f688656e640 1 -- 172.21.15.39:0/713764479 wait complete. 2024-09-21T03:37:41.353 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:37:41 smithi039 ceph-mon[31211]: mgrmap e20: x(active, since 14s), standbys: z, y 2024-09-21T03:37:41.353 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:37:41 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "mgr metadata", "who": "y", "id": "y"} : dispatch 2024-09-21T03:37:41.354 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:41.350+0000 7f688656e640 1 Processor -- start 2024-09-21T03:37:41.354 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:41.350+0000 7f688656e640 1 -- start start 2024-09-21T03:37:41.354 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:41.350+0000 7f688656e640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6880103bc0 0x7f688019a410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:37:41.355 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:41.350+0000 7f688656e640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f68801094b0 0x7f688019a9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:37:41.355 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:41.350+0000 7f688656e640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6880109e00 0x7f688019fc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:37:41.355 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:41.351+0000 7f688656e640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f6880113ac0 con 0x7f6880109e00 2024-09-21T03:37:41.355 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:41.351+0000 7f688656e640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f6880113940 con 0x7f6880103bc0 2024-09-21T03:37:41.355 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:41.351+0000 7f688656e640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f6880113c40 con 0x7f68801094b0 2024-09-21T03:37:41.355 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:41.351+0000 7f6884b0b640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6880109e00 0x7f688019fc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:37:41.355 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:41.351+0000 7f6884b0b640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6880109e00 0x7f688019fc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:46664/0 (socket says 172.21.15.39:46664) 2024-09-21T03:37:41.355 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:41.351+0000 7f6884b0b640 1 -- 172.21.15.39:0/651322856 learned_addr learned my addr 172.21.15.39:0/651322856 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:37:41.355 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:41.351+0000 7f687ffff640 1 --2- 172.21.15.39:0/651322856 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6880103bc0 0x7f688019a410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:37:41.355 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:41.351+0000 7f6884b0b640 1 -- 172.21.15.39:0/651322856 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f68801094b0 msgr2=0x7f688019a9b0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-21T03:37:41.355 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:41.351+0000 7f6884b0b640 1 --2- 172.21.15.39:0/651322856 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f68801094b0 0x7f688019a9b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:41.355 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:41.351+0000 7f6884b0b640 1 -- 172.21.15.39:0/651322856 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6880103bc0 msgr2=0x7f688019a410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:37:41.355 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:41.351+0000 7f6884b0b640 1 --2- 172.21.15.39:0/651322856 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6880103bc0 0x7f688019a410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:41.355 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:41.351+0000 7f6884b0b640 1 -- 172.21.15.39:0/651322856 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f68801a0380 con 0x7f6880109e00 2024-09-21T03:37:41.356 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:41.351+0000 7f6884b0b640 1 --2- 172.21.15.39:0/651322856 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6880109e00 0x7f688019fc50 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f68780098d0 tx=0x7f68780317b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:37:41.356 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:41.352+0000 7f687d7fa640 1 -- 172.21.15.39:0/651322856 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6878003b70 con 0x7f6880109e00 2024-09-21T03:37:41.356 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:41.352+0000 7f687d7fa640 1 -- 172.21.15.39:0/651322856 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6878003d10 con 0x7f6880109e00 2024-09-21T03:37:41.356 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:41.352+0000 7f688656e640 1 -- 172.21.15.39:0/651322856 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f68801a0610 con 0x7f6880109e00 2024-09-21T03:37:41.356 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:41.352+0000 7f687d7fa640 1 -- 172.21.15.39:0/651322856 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f687802fa20 con 0x7f6880109e00 2024-09-21T03:37:41.356 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:41.352+0000 7f688656e640 1 -- 172.21.15.39:0/651322856 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f68801a6dd0 con 0x7f6880109e00 2024-09-21T03:37:41.356 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:41.353+0000 7f688656e640 1 -- 172.21.15.39:0/651322856 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6880103fd0 con 0x7f6880109e00 2024-09-21T03:37:41.357 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:41.354+0000 7f687d7fa640 1 -- 172.21.15.39:0/651322856 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 20) ==== 154473+0+0 (secure 0 0 0) 0x7f687802fc00 con 0x7f6880109e00 2024-09-21T03:37:41.357 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:41.355+0000 7f687d7fa640 1 --2- 172.21.15.39:0/651322856 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f68680b6550 0x7f68680b8a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:37:41.358 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:41.355+0000 7f687ffff640 1 --2- 172.21.15.39:0/651322856 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f68680b6550 0x7f68680b8a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:37:41.358 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:41.355+0000 7f687d7fa640 1 -- 172.21.15.39:0/651322856 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1370+0+0 (secure 0 0 0) 0x7f6878106bb0 con 0x7f6880109e00 2024-09-21T03:37:41.358 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:41.356+0000 7f687ffff640 1 --2- 172.21.15.39:0/651322856 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f68680b6550 0x7f68680b8a60 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f6870009bd0 tx=0x7f6870005f90 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:37:41.362 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:41.360+0000 7f687d7fa640 1 -- 172.21.15.39:0/651322856 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f68780cf040 con 0x7f6880109e00 2024-09-21T03:37:41.467 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:37:41 smithi161 ceph-mon[34533]: mgrmap e20: x(active, since 14s), standbys: z, y 2024-09-21T03:37:41.468 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:37:41 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "mgr metadata", "who": "y", "id": "y"} : dispatch 2024-09-21T03:37:41.474 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:37:41 smithi150 ceph-mon[34289]: mgrmap e20: x(active, since 14s), standbys: z, y 2024-09-21T03:37:41.474 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:37:41 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "mgr metadata", "who": "y", "id": "y"} : dispatch 2024-09-21T03:37:41.553 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:41.550+0000 7f688656e640 1 -- 172.21.15.39:0/651322856 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command([{prefix=config set, name=mds_op_complaint_time}] v 0) -- 0x7f688010bbf0 con 0x7f6880109e00 2024-09-21T03:37:41.558 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:41.556+0000 7f687d7fa640 1 -- 172.21.15.39:0/651322856 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_op_complaint_time}]=0 v25)=0 v25) ==== 121+0+0 (secure 0 0 0) 0x7f68780d3420 con 0x7f6880109e00 2024-09-21T03:37:41.561 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:41.558+0000 7f688656e640 1 -- 172.21.15.39:0/651322856 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f68680b6550 msgr2=0x7f68680b8a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:37:41.561 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:41.558+0000 7f688656e640 1 --2- 172.21.15.39:0/651322856 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f68680b6550 0x7f68680b8a60 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f6870009bd0 tx=0x7f6870005f90 comp rx=0 tx=0).stop 2024-09-21T03:37:41.561 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:41.558+0000 7f688656e640 1 -- 172.21.15.39:0/651322856 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6880109e00 msgr2=0x7f688019fc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:37:41.561 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:41.558+0000 7f688656e640 1 --2- 172.21.15.39:0/651322856 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6880109e00 0x7f688019fc50 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f68780098d0 tx=0x7f68780317b0 comp rx=0 tx=0).stop 2024-09-21T03:37:41.561 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:41.558+0000 7f688656e640 1 -- 172.21.15.39:0/651322856 shutdown_connections 2024-09-21T03:37:41.561 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:41.558+0000 7f688656e640 1 --2- 172.21.15.39:0/651322856 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f68680b6550 0x7f68680b8a60 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:41.561 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:41.558+0000 7f688656e640 1 --2- 172.21.15.39:0/651322856 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6880109e00 0x7f688019fc50 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:41.561 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:41.558+0000 7f688656e640 1 --2- 172.21.15.39:0/651322856 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f68801094b0 0x7f688019a9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:41.561 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:41.558+0000 7f688656e640 1 --2- 172.21.15.39:0/651322856 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6880103bc0 0x7f688019a410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:41.562 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:41.558+0000 7f688656e640 1 -- 172.21.15.39:0/651322856 >> 172.21.15.39:0/651322856 conn(0x7f688007b910 msgr2=0x7f6880105480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:37:41.562 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:41.558+0000 7f688656e640 1 -- 172.21.15.39:0/651322856 shutdown_connections 2024-09-21T03:37:41.562 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:41.558+0000 7f688656e640 1 -- 172.21.15.39:0/651322856 wait complete. 2024-09-21T03:37:42.687 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:37:42 smithi039 ceph-mon[31211]: from='client.? 172.21.15.39:0/651322856' entity='client.admin' 2024-09-21T03:37:42.842 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:42.840+0000 7f257bcdf640 1 -- 172.21.15.39:0/2162065502 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f25741049b0 msgr2=0x7f2574104dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:37:42.842 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:42.840+0000 7f257bcdf640 1 --2- 172.21.15.39:0/2162065502 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f25741049b0 0x7f2574104dc0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f2568009900 tx=0x7f256802a410 comp rx=0 tx=0).stop 2024-09-21T03:37:42.842 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:42.840+0000 7f257bcdf640 1 -- 172.21.15.39:0/2162065502 shutdown_connections 2024-09-21T03:37:42.843 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:42.840+0000 7f257bcdf640 1 --2- 172.21.15.39:0/2162065502 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f257410de10 0x7f2574110300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:42.843 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:42.840+0000 7f257bcdf640 1 --2- 172.21.15.39:0/2162065502 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2574105300 0x7f257410d850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:42.843 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:42.840+0000 7f257bcdf640 1 --2- 172.21.15.39:0/2162065502 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f25741049b0 0x7f2574104dc0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:42.843 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:42.840+0000 7f257bcdf640 1 -- 172.21.15.39:0/2162065502 >> 172.21.15.39:0/2162065502 conn(0x7f2574079730 msgr2=0x7f2574079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:37:42.843 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:42.840+0000 7f257bcdf640 1 -- 172.21.15.39:0/2162065502 shutdown_connections 2024-09-21T03:37:42.843 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:42.840+0000 7f257bcdf640 1 -- 172.21.15.39:0/2162065502 wait complete. 2024-09-21T03:37:42.844 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:42.840+0000 7f257bcdf640 1 Processor -- start 2024-09-21T03:37:42.844 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:42.841+0000 7f257bcdf640 1 -- start start 2024-09-21T03:37:42.845 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:42.841+0000 7f257bcdf640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f25741049b0 0x7f257419a4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:37:42.845 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:42.841+0000 7f257bcdf640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2574105300 0x7f257419aa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:37:42.845 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:42.841+0000 7f257bcdf640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f257410de10 0x7f257419fd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:37:42.845 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:42.841+0000 7f257bcdf640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f2574113a70 con 0x7f2574105300 2024-09-21T03:37:42.845 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:42.841+0000 7f257bcdf640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f25741138f0 con 0x7f25741049b0 2024-09-21T03:37:42.845 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:42.841+0000 7f257bcdf640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f2574113bf0 con 0x7f257410de10 2024-09-21T03:37:42.845 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:42.841+0000 7f2579a7b640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f25741049b0 0x7f257419a4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:37:42.845 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:42.841+0000 7f257927a640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2574105300 0x7f257419aa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:37:42.845 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:42.841+0000 7f2579a7b640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f25741049b0 0x7f257419a4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.39:54666/0 (socket says 172.21.15.39:54666) 2024-09-21T03:37:42.845 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:42.841+0000 7f257927a640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2574105300 0x7f257419aa60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:46688/0 (socket says 172.21.15.39:46688) 2024-09-21T03:37:42.845 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:42.841+0000 7f2579a7b640 1 -- 172.21.15.39:0/2053507619 learned_addr learned my addr 172.21.15.39:0/2053507619 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:37:42.845 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:42.841+0000 7f257a27c640 1 --2- 172.21.15.39:0/2053507619 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f257410de10 0x7f257419fd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:37:42.845 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:42.841+0000 7f257927a640 1 -- 172.21.15.39:0/2053507619 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f257410de10 msgr2=0x7f257419fd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:37:42.846 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:42.841+0000 7f257927a640 1 --2- 172.21.15.39:0/2053507619 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f257410de10 0x7f257419fd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:42.846 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:42.842+0000 7f257927a640 1 -- 172.21.15.39:0/2053507619 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f25741049b0 msgr2=0x7f257419a4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:37:42.846 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:42.842+0000 7f257927a640 1 --2- 172.21.15.39:0/2053507619 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f25741049b0 0x7f257419a4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:42.846 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:42.842+0000 7f257927a640 1 -- 172.21.15.39:0/2053507619 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f25741a0430 con 0x7f2574105300 2024-09-21T03:37:42.846 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:42.842+0000 7f2579a7b640 1 --2- 172.21.15.39:0/2053507619 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f25741049b0 0x7f257419a4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:37:42.846 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:42.842+0000 7f257927a640 1 --2- 172.21.15.39:0/2053507619 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2574105300 0x7f257419aa60 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f257406ad30 tx=0x7f257000ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:37:42.846 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:42.842+0000 7f2566ffd640 1 -- 172.21.15.39:0/2053507619 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f257000cd50 con 0x7f2574105300 2024-09-21T03:37:42.846 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:42.842+0000 7f2566ffd640 1 -- 172.21.15.39:0/2053507619 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2570011920 con 0x7f2574105300 2024-09-21T03:37:42.846 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:42.842+0000 7f257bcdf640 1 -- 172.21.15.39:0/2053507619 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f25741a0720 con 0x7f2574105300 2024-09-21T03:37:42.846 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:42.842+0000 7f2566ffd640 1 -- 172.21.15.39:0/2053507619 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f257001a9a0 con 0x7f2574105300 2024-09-21T03:37:42.846 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:42.842+0000 7f257bcdf640 1 -- 172.21.15.39:0/2053507619 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f25741a6e80 con 0x7f2574105300 2024-09-21T03:37:42.847 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:42.844+0000 7f257bcdf640 1 -- 172.21.15.39:0/2053507619 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2574104dc0 con 0x7f2574105300 2024-09-21T03:37:42.847 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:42.845+0000 7f2566ffd640 1 -- 172.21.15.39:0/2053507619 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 20) ==== 154473+0+0 (secure 0 0 0) 0x7f2570011470 con 0x7f2574105300 2024-09-21T03:37:42.848 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:42.846+0000 7f2566ffd640 1 --2- 172.21.15.39:0/2053507619 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f25600b6550 0x7f25600b8a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:37:42.849 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:42.846+0000 7f2579a7b640 1 --2- 172.21.15.39:0/2053507619 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f25600b6550 0x7f25600b8a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:37:42.849 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:42.846+0000 7f2566ffd640 1 -- 172.21.15.39:0/2053507619 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1370+0+0 (secure 0 0 0) 0x7f25700e5a60 con 0x7f2574105300 2024-09-21T03:37:42.849 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:42.846+0000 7f2579a7b640 1 --2- 172.21.15.39:0/2053507619 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f25600b6550 0x7f25600b8a60 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f25680096c0 tx=0x7f2568032ed0 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:37:42.853 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:42.850+0000 7f2566ffd640 1 -- 172.21.15.39:0/2053507619 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f2570016030 con 0x7f2574105300 2024-09-21T03:37:42.967 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:37:42 smithi161 ceph-mon[34533]: from='client.? 172.21.15.39:0/651322856' entity='client.admin' 2024-09-21T03:37:42.974 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:37:42 smithi150 ceph-mon[34289]: from='client.? 172.21.15.39:0/651322856' entity='client.admin' 2024-09-21T03:37:43.043 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:43.040+0000 7fbe43bf6640 1 -- 172.21.15.39:0/3960435044 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbe3c109e10 msgr2=0x7fbe3c110640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:37:43.043 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:43.040+0000 7fbe43bf6640 1 --2- 172.21.15.39:0/3960435044 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbe3c109e10 0x7fbe3c110640 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fbe34009ca0 tx=0x7fbe3402a8c0 comp rx=0 tx=0).stop 2024-09-21T03:37:43.043 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:43.040+0000 7fbe43bf6640 1 -- 172.21.15.39:0/3960435044 shutdown_connections 2024-09-21T03:37:43.043 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:43.040+0000 7fbe43bf6640 1 --2- 172.21.15.39:0/3960435044 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbe3c109e10 0x7fbe3c110640 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:43.043 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:43.040+0000 7fbe43bf6640 1 --2- 172.21.15.39:0/3960435044 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbe3c1094c0 0x7fbe3c1098d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:43.043 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:43.040+0000 7fbe43bf6640 1 --2- 172.21.15.39:0/3960435044 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbe3c103b80 0x7fbe3c103f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:43.043 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:43.040+0000 7fbe43bf6640 1 -- 172.21.15.39:0/3960435044 >> 172.21.15.39:0/3960435044 conn(0x7fbe3c0ff610 msgr2=0x7fbe3c101a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:37:43.044 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:43.041+0000 7fbe43bf6640 1 -- 172.21.15.39:0/3960435044 shutdown_connections 2024-09-21T03:37:43.044 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:43.041+0000 7fbe43bf6640 1 -- 172.21.15.39:0/3960435044 wait complete. 2024-09-21T03:37:43.044 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:43.042+0000 7fbe43bf6640 1 Processor -- start 2024-09-21T03:37:43.044 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:43.042+0000 7fbe43bf6640 1 -- start start 2024-09-21T03:37:43.044 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:43.042+0000 7f257bcdf640 1 -- 172.21.15.39:0/2053507619 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command([{prefix=config set, name=rados_mon_op_timeout}] v 0) -- 0x7f2574108f40 con 0x7f2574105300 2024-09-21T03:37:43.046 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:43.042+0000 7fbe43bf6640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbe3c103b80 0x7fbe3c19a420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:37:43.046 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:43.042+0000 7fbe43bf6640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbe3c1094c0 0x7fbe3c19a9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:37:43.046 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:43.042+0000 7fbe43bf6640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbe3c109e10 0x7fbe3c19fc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:37:43.046 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:43.042+0000 7fbe43bf6640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fbe3c113ad0 con 0x7fbe3c1094c0 2024-09-21T03:37:43.046 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:43.042+0000 7fbe43bf6640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fbe3c113950 con 0x7fbe3c109e10 2024-09-21T03:37:43.046 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:43.042+0000 7fbe43bf6640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fbe3c113c50 con 0x7fbe3c103b80 2024-09-21T03:37:43.046 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:43.042+0000 7fbe41191640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbe3c1094c0 0x7fbe3c19a9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:37:43.046 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:43.042+0000 7fbe41191640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbe3c1094c0 0x7fbe3c19a9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:46694/0 (socket says 172.21.15.39:46694) 2024-09-21T03:37:43.046 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:43.042+0000 7fbe41191640 1 -- 172.21.15.39:0/2453418576 learned_addr learned my addr 172.21.15.39:0/2453418576 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:37:43.046 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:43.042+0000 7fbe41992640 1 --2- 172.21.15.39:0/2453418576 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbe3c103b80 0x7fbe3c19a420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:37:43.046 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:43.042+0000 7fbe42193640 1 --2- 172.21.15.39:0/2453418576 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbe3c109e10 0x7fbe3c19fc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:37:43.046 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:43.042+0000 7fbe41191640 1 -- 172.21.15.39:0/2453418576 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbe3c103b80 msgr2=0x7fbe3c19a420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:37:43.046 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:43.043+0000 7fbe41191640 1 --2- 172.21.15.39:0/2453418576 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbe3c103b80 0x7fbe3c19a420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:43.046 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:43.043+0000 7fbe41191640 1 -- 172.21.15.39:0/2453418576 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbe3c109e10 msgr2=0x7fbe3c19fc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:37:43.047 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:43.043+0000 7fbe41191640 1 --2- 172.21.15.39:0/2453418576 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbe3c109e10 0x7fbe3c19fc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:43.047 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:43.043+0000 7fbe41191640 1 -- 172.21.15.39:0/2453418576 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbe3c1a0390 con 0x7fbe3c1094c0 2024-09-21T03:37:43.047 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:43.043+0000 7fbe41191640 1 --2- 172.21.15.39:0/2453418576 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbe3c1094c0 0x7fbe3c19a9c0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fbe2c00d950 tx=0x7fbe2c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:37:43.047 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:43.043+0000 7fbe32ffd640 1 -- 172.21.15.39:0/2453418576 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe2c017070 con 0x7fbe3c1094c0 2024-09-21T03:37:43.047 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:43.043+0000 7fbe32ffd640 1 -- 172.21.15.39:0/2453418576 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbe2c009c30 con 0x7fbe3c1094c0 2024-09-21T03:37:43.047 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:43.043+0000 7fbe32ffd640 1 -- 172.21.15.39:0/2453418576 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe2c0047a0 con 0x7fbe3c1094c0 2024-09-21T03:37:43.047 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:43.043+0000 7fbe43bf6640 1 -- 172.21.15.39:0/2453418576 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbe3c1a0680 con 0x7fbe3c1094c0 2024-09-21T03:37:43.047 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:43.043+0000 7fbe43bf6640 1 -- 172.21.15.39:0/2453418576 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbe3c1a6de0 con 0x7fbe3c1094c0 2024-09-21T03:37:43.047 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:43.044+0000 7fbe43bf6640 1 -- 172.21.15.39:0/2453418576 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbe3c103f90 con 0x7fbe3c1094c0 2024-09-21T03:37:43.049 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:43.046+0000 7fbe32ffd640 1 -- 172.21.15.39:0/2453418576 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 20) ==== 154473+0+0 (secure 0 0 0) 0x7fbe2c004020 con 0x7fbe3c1094c0 2024-09-21T03:37:43.050 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:43.047+0000 7fbe32ffd640 1 --2- 172.21.15.39:0/2453418576 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7fbe280b6550 0x7fbe280b8a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:37:43.050 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:43.048+0000 7fbe41992640 1 --2- 172.21.15.39:0/2453418576 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7fbe280b6550 0x7fbe280b8a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:37:43.050 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:43.048+0000 7fbe32ffd640 1 -- 172.21.15.39:0/2453418576 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1370+0+0 (secure 0 0 0) 0x7fbe2c0e5740 con 0x7fbe3c1094c0 2024-09-21T03:37:43.050 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:43.048+0000 7f2566ffd640 1 -- 172.21.15.39:0/2053507619 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{prefix=config set, name=rados_mon_op_timeout}]=0 v26)=0 v26) ==== 120+0+0 (secure 0 0 0) 0x7f25700adef0 con 0x7f2574105300 2024-09-21T03:37:43.051 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:43.048+0000 7fbe41992640 1 --2- 172.21.15.39:0/2453418576 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7fbe280b6550 0x7fbe280b8a60 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fbe38009bd0 tx=0x7fbe38005f90 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:37:43.054 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:43.051+0000 7f257bcdf640 1 -- 172.21.15.39:0/2053507619 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f25600b6550 msgr2=0x7f25600b8a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:37:43.054 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:43.051+0000 7f257bcdf640 1 --2- 172.21.15.39:0/2053507619 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f25600b6550 0x7f25600b8a60 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f25680096c0 tx=0x7f2568032ed0 comp rx=0 tx=0).stop 2024-09-21T03:37:43.054 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:43.051+0000 7f257bcdf640 1 -- 172.21.15.39:0/2053507619 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2574105300 msgr2=0x7f257419aa60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:37:43.054 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:43.051+0000 7f257bcdf640 1 --2- 172.21.15.39:0/2053507619 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2574105300 0x7f257419aa60 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f257406ad30 tx=0x7f257000ee80 comp rx=0 tx=0).stop 2024-09-21T03:37:43.054 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:43.051+0000 7f257bcdf640 1 -- 172.21.15.39:0/2053507619 shutdown_connections 2024-09-21T03:37:43.054 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:43.051+0000 7f257bcdf640 1 --2- 172.21.15.39:0/2053507619 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f25600b6550 0x7f25600b8a60 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:43.054 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:43.051+0000 7f257bcdf640 1 --2- 172.21.15.39:0/2053507619 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f257410de10 0x7f257419fd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:43.054 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:43.051+0000 7f257bcdf640 1 --2- 172.21.15.39:0/2053507619 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2574105300 0x7f257419aa60 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:43.054 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:43.051+0000 7f257bcdf640 1 --2- 172.21.15.39:0/2053507619 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f25741049b0 0x7f257419a4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:43.054 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:43.051+0000 7f257bcdf640 1 -- 172.21.15.39:0/2053507619 >> 172.21.15.39:0/2053507619 conn(0x7f2574079730 msgr2=0x7f25741007d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:37:43.054 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:43.051+0000 7f257bcdf640 1 -- 172.21.15.39:0/2053507619 shutdown_connections 2024-09-21T03:37:43.054 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:43.051+0000 7f257bcdf640 1 -- 172.21.15.39:0/2053507619 wait complete. 2024-09-21T03:37:43.058 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:43.055+0000 7fbe32ffd640 1 -- 172.21.15.39:0/2453418576 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fbe2c020020 con 0x7fbe3c1094c0 2024-09-21T03:37:43.251 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:43.249+0000 7fbe43bf6640 1 -- 172.21.15.39:0/2453418576 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command([{prefix=config set, name=mon_op_complaint_time}] v 0) -- 0x7fbe3c10bc60 con 0x7fbe3c1094c0 2024-09-21T03:37:43.257 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:43.254+0000 7fbe32ffd640 1 -- 172.21.15.39:0/2453418576 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_op_complaint_time}]=0 v27)=0 v27) ==== 121+0+0 (secure 0 0 0) 0x7fbe2c0adc50 con 0x7fbe3c1094c0 2024-09-21T03:37:43.260 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:43.257+0000 7fbe43bf6640 1 -- 172.21.15.39:0/2453418576 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7fbe280b6550 msgr2=0x7fbe280b8a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:37:43.260 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:43.257+0000 7fbe43bf6640 1 --2- 172.21.15.39:0/2453418576 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7fbe280b6550 0x7fbe280b8a60 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fbe38009bd0 tx=0x7fbe38005f90 comp rx=0 tx=0).stop 2024-09-21T03:37:43.260 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:43.257+0000 7fbe43bf6640 1 -- 172.21.15.39:0/2453418576 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbe3c1094c0 msgr2=0x7fbe3c19a9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:37:43.260 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:43.257+0000 7fbe43bf6640 1 --2- 172.21.15.39:0/2453418576 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbe3c1094c0 0x7fbe3c19a9c0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fbe2c00d950 tx=0x7fbe2c00de10 comp rx=0 tx=0).stop 2024-09-21T03:37:43.260 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:43.257+0000 7fbe43bf6640 1 -- 172.21.15.39:0/2453418576 shutdown_connections 2024-09-21T03:37:43.260 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:43.257+0000 7fbe43bf6640 1 --2- 172.21.15.39:0/2453418576 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7fbe280b6550 0x7fbe280b8a60 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:43.260 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:43.257+0000 7fbe43bf6640 1 --2- 172.21.15.39:0/2453418576 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbe3c109e10 0x7fbe3c19fc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:43.260 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:43.257+0000 7fbe43bf6640 1 --2- 172.21.15.39:0/2453418576 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbe3c1094c0 0x7fbe3c19a9c0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:43.260 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:43.257+0000 7fbe43bf6640 1 --2- 172.21.15.39:0/2453418576 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbe3c103b80 0x7fbe3c19a420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:43.260 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:43.257+0000 7fbe43bf6640 1 -- 172.21.15.39:0/2453418576 >> 172.21.15.39:0/2453418576 conn(0x7fbe3c0ff610 msgr2=0x7fbe3c106a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:37:43.260 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:43.257+0000 7fbe43bf6640 1 -- 172.21.15.39:0/2453418576 shutdown_connections 2024-09-21T03:37:43.261 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:43.257+0000 7fbe43bf6640 1 -- 172.21.15.39:0/2453418576 wait complete. 2024-09-21T03:37:43.838 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:43.836+0000 7f227bb31640 1 -- 172.21.15.39:0/1800736286 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2274103b80 msgr2=0x7f2274103f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:37:43.839 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:43.836+0000 7f227bb31640 1 --2- 172.21.15.39:0/1800736286 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2274103b80 0x7f2274103f90 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f2264009ca0 tx=0x7f226402a8c0 comp rx=0 tx=0).stop 2024-09-21T03:37:43.839 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:43.836+0000 7f227bb31640 1 -- 172.21.15.39:0/1800736286 shutdown_connections 2024-09-21T03:37:43.839 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:43.836+0000 7f227bb31640 1 --2- 172.21.15.39:0/1800736286 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2274109e10 0x7f2274110640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:43.839 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:43.836+0000 7f227bb31640 1 --2- 172.21.15.39:0/1800736286 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f22741094c0 0x7f22741098d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:43.839 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:43.836+0000 7f227bb31640 1 --2- 172.21.15.39:0/1800736286 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2274103b80 0x7f2274103f90 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:43.839 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:43.836+0000 7f227bb31640 1 -- 172.21.15.39:0/1800736286 >> 172.21.15.39:0/1800736286 conn(0x7f22740ff610 msgr2=0x7f2274101a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:37:43.839 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:43.836+0000 7f227bb31640 1 -- 172.21.15.39:0/1800736286 shutdown_connections 2024-09-21T03:37:43.839 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:43.836+0000 7f227bb31640 1 -- 172.21.15.39:0/1800736286 wait complete. 2024-09-21T03:37:43.841 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:43.837+0000 7f227bb31640 1 Processor -- start 2024-09-21T03:37:43.841 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:43.837+0000 7f227bb31640 1 -- start start 2024-09-21T03:37:43.841 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:43.837+0000 7f227bb31640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2274103b80 0x7f227419a4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:37:43.841 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:43.837+0000 7f227bb31640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f22741094c0 0x7f227419aa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:37:43.841 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:43.837+0000 7f227bb31640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2274109e10 0x7f227419fd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:37:43.841 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:43.837+0000 7f227bb31640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f2274113ad0 con 0x7f2274109e10 2024-09-21T03:37:43.841 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:43.837+0000 7f227bb31640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f2274113950 con 0x7f2274103b80 2024-09-21T03:37:43.841 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:43.837+0000 7f227bb31640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f2274113c50 con 0x7f22741094c0 2024-09-21T03:37:43.841 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:43.837+0000 7f227a0ce640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2274109e10 0x7f227419fd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:37:43.841 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:43.837+0000 7f227a0ce640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2274109e10 0x7f227419fd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:46704/0 (socket says 172.21.15.39:46704) 2024-09-21T03:37:43.841 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:43.837+0000 7f227a0ce640 1 -- 172.21.15.39:0/2180263451 learned_addr learned my addr 172.21.15.39:0/2180263451 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:37:43.841 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:43.837+0000 7f22790cc640 1 --2- 172.21.15.39:0/2180263451 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f22741094c0 0x7f227419aa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:37:43.842 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:43.837+0000 7f22798cd640 1 --2- 172.21.15.39:0/2180263451 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2274103b80 0x7f227419a4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:37:43.842 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:43.838+0000 7f227a0ce640 1 -- 172.21.15.39:0/2180263451 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f22741094c0 msgr2=0x7f227419aa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:37:43.842 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:43.838+0000 7f227a0ce640 1 --2- 172.21.15.39:0/2180263451 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f22741094c0 0x7f227419aa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:43.842 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:43.838+0000 7f227a0ce640 1 -- 172.21.15.39:0/2180263451 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2274103b80 msgr2=0x7f227419a4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:37:43.842 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:43.838+0000 7f227a0ce640 1 --2- 172.21.15.39:0/2180263451 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2274103b80 0x7f227419a4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:43.842 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:43.838+0000 7f227a0ce640 1 -- 172.21.15.39:0/2180263451 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f22741a0390 con 0x7f2274109e10 2024-09-21T03:37:43.842 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:43.838+0000 7f22790cc640 1 --2- 172.21.15.39:0/2180263451 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f22741094c0 0x7f227419aa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:37:43.842 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:43.838+0000 7f227a0ce640 1 --2- 172.21.15.39:0/2180263451 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2274109e10 0x7f227419fd20 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f226c00d750 tx=0x7f226c00db50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:37:43.842 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:43.838+0000 7f2272ffd640 1 -- 172.21.15.39:0/2180263451 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f226c017070 con 0x7f2274109e10 2024-09-21T03:37:43.842 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:43.838+0000 7f2272ffd640 1 -- 172.21.15.39:0/2180263451 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f226c0097c0 con 0x7f2274109e10 2024-09-21T03:37:43.842 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:43.838+0000 7f227bb31640 1 -- 172.21.15.39:0/2180263451 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f22741a0680 con 0x7f2274109e10 2024-09-21T03:37:43.842 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:43.838+0000 7f2272ffd640 1 -- 172.21.15.39:0/2180263451 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f226c004ba0 con 0x7f2274109e10 2024-09-21T03:37:43.842 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:43.838+0000 7f227bb31640 1 -- 172.21.15.39:0/2180263451 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f22741a6ea0 con 0x7f2274109e10 2024-09-21T03:37:43.843 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:43.840+0000 7f227bb31640 1 -- 172.21.15.39:0/2180263451 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2274103f90 con 0x7f2274109e10 2024-09-21T03:37:43.843 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:43.841+0000 7f2272ffd640 1 -- 172.21.15.39:0/2180263451 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 20) ==== 154473+0+0 (secure 0 0 0) 0x7f226c005220 con 0x7f2274109e10 2024-09-21T03:37:43.844 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:43.842+0000 7f2272ffd640 1 --2- 172.21.15.39:0/2180263451 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f225c0b6580 0x7f225c0b8a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:37:43.845 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:43.842+0000 7f2272ffd640 1 -- 172.21.15.39:0/2180263451 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1370+0+0 (secure 0 0 0) 0x7f226c0e5050 con 0x7f2274109e10 2024-09-21T03:37:43.845 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:43.843+0000 7f22798cd640 1 --2- 172.21.15.39:0/2180263451 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f225c0b6580 0x7f225c0b8a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:37:43.845 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:43.843+0000 7f22798cd640 1 --2- 172.21.15.39:0/2180263451 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f225c0b6580 0x7f225c0b8a90 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f2264005e30 tx=0x7f2264009360 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:37:43.849 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:43.847+0000 7f2272ffd640 1 -- 172.21.15.39:0/2180263451 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f226c0ad4e0 con 0x7f2274109e10 2024-09-21T03:37:44.040 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:44.037+0000 7f227bb31640 1 -- 172.21.15.39:0/2180263451 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command([{prefix=config set, name=debug_mds}] v 0) -- 0x7f227410bbe0 con 0x7f2274109e10 2024-09-21T03:37:44.044 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:44.042+0000 7f2272ffd640 1 -- 172.21.15.39:0/2180263451 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{prefix=config set, name=debug_mds}]=0 v28)=0 v28) ==== 108+0+0 (secure 0 0 0) 0x7f226c0b18c0 con 0x7f2274109e10 2024-09-21T03:37:44.047 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:44.044+0000 7f227bb31640 1 -- 172.21.15.39:0/2180263451 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f225c0b6580 msgr2=0x7f225c0b8a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:37:44.047 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:44.044+0000 7f227bb31640 1 --2- 172.21.15.39:0/2180263451 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f225c0b6580 0x7f225c0b8a90 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f2264005e30 tx=0x7f2264009360 comp rx=0 tx=0).stop 2024-09-21T03:37:44.047 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:44.044+0000 7f227bb31640 1 -- 172.21.15.39:0/2180263451 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2274109e10 msgr2=0x7f227419fd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:37:44.047 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:44.044+0000 7f227bb31640 1 --2- 172.21.15.39:0/2180263451 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2274109e10 0x7f227419fd20 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f226c00d750 tx=0x7f226c00db50 comp rx=0 tx=0).stop 2024-09-21T03:37:44.047 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:44.044+0000 7f227bb31640 1 -- 172.21.15.39:0/2180263451 shutdown_connections 2024-09-21T03:37:44.047 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:44.044+0000 7f227bb31640 1 --2- 172.21.15.39:0/2180263451 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f225c0b6580 0x7f225c0b8a90 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:44.047 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:44.044+0000 7f227bb31640 1 --2- 172.21.15.39:0/2180263451 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2274109e10 0x7f227419fd20 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:44.047 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:44.044+0000 7f227bb31640 1 --2- 172.21.15.39:0/2180263451 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f22741094c0 0x7f227419aa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:44.047 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:44.044+0000 7f227bb31640 1 --2- 172.21.15.39:0/2180263451 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2274103b80 0x7f227419a4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:44.047 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:44.044+0000 7f227bb31640 1 -- 172.21.15.39:0/2180263451 >> 172.21.15.39:0/2180263451 conn(0x7f22740ff610 msgr2=0x7f2274101a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:37:44.047 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:44.044+0000 7f227bb31640 1 -- 172.21.15.39:0/2180263451 shutdown_connections 2024-09-21T03:37:44.048 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:44.044+0000 7f227bb31640 1 -- 172.21.15.39:0/2180263451 wait complete. 2024-09-21T03:37:44.333 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:37:44 smithi039 ceph-mon[31211]: from='client.? 172.21.15.39:0/2053507619' entity='client.admin' 2024-09-21T03:37:44.334 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:37:44 smithi039 ceph-mon[31211]: from='client.? 172.21.15.39:0/2453418576' entity='client.admin' 2024-09-21T03:37:44.334 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:37:44 smithi039 ceph-mon[31211]: from='client.? 172.21.15.39:0/2180263451' entity='client.admin' 2024-09-21T03:37:44.467 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:37:44 smithi161 ceph-mon[34533]: from='client.? 172.21.15.39:0/2053507619' entity='client.admin' 2024-09-21T03:37:44.467 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:37:44 smithi161 ceph-mon[34533]: from='client.? 172.21.15.39:0/2453418576' entity='client.admin' 2024-09-21T03:37:44.467 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:37:44 smithi161 ceph-mon[34533]: from='client.? 172.21.15.39:0/2180263451' entity='client.admin' 2024-09-21T03:37:44.474 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:37:44 smithi150 ceph-mon[34289]: from='client.? 172.21.15.39:0/2053507619' entity='client.admin' 2024-09-21T03:37:44.474 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:37:44 smithi150 ceph-mon[34289]: from='client.? 172.21.15.39:0/2453418576' entity='client.admin' 2024-09-21T03:37:44.474 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:37:44 smithi150 ceph-mon[34289]: from='client.? 172.21.15.39:0/2180263451' entity='client.admin' 2024-09-21T03:37:45.195 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:45.193+0000 7f93c2c2c640 1 -- 172.21.15.39:0/1202127846 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f93bc109df0 msgr2=0x7f93bc110620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:37:45.196 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:45.193+0000 7f93c2c2c640 1 --2- 172.21.15.39:0/1202127846 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f93bc109df0 0x7f93bc110620 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f93b4009ca0 tx=0x7f93b402a8c0 comp rx=0 tx=0).stop 2024-09-21T03:37:45.196 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:45.193+0000 7f93c2c2c640 1 -- 172.21.15.39:0/1202127846 shutdown_connections 2024-09-21T03:37:45.196 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:45.193+0000 7f93c2c2c640 1 --2- 172.21.15.39:0/1202127846 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f93bc109df0 0x7f93bc110620 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:45.196 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:45.193+0000 7f93c2c2c640 1 --2- 172.21.15.39:0/1202127846 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f93bc1094a0 0x7f93bc1098b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:45.196 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:45.193+0000 7f93c2c2c640 1 --2- 172.21.15.39:0/1202127846 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f93bc103b60 0x7f93bc103f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:45.196 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:45.193+0000 7f93c2c2c640 1 -- 172.21.15.39:0/1202127846 >> 172.21.15.39:0/1202127846 conn(0x7f93bc07b850 msgr2=0x7f93bc07bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:37:45.196 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:45.193+0000 7f93c2c2c640 1 -- 172.21.15.39:0/1202127846 shutdown_connections 2024-09-21T03:37:45.196 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:45.193+0000 7f93c2c2c640 1 -- 172.21.15.39:0/1202127846 wait complete. 2024-09-21T03:37:45.198 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:45.194+0000 7f93c2c2c640 1 Processor -- start 2024-09-21T03:37:45.198 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:45.194+0000 7f93c2c2c640 1 -- start start 2024-09-21T03:37:45.198 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:45.194+0000 7f93c2c2c640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f93bc103b60 0x7f93bc19a460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:37:45.198 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:45.194+0000 7f93c2c2c640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f93bc1094a0 0x7f93bc19aa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:37:45.198 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:45.194+0000 7f93c2c2c640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f93bc109df0 0x7f93bc19fca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:37:45.198 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:45.194+0000 7f93c2c2c640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f93bc113ab0 con 0x7f93bc103b60 2024-09-21T03:37:45.198 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:45.194+0000 7f93c2c2c640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f93bc113930 con 0x7f93bc1094a0 2024-09-21T03:37:45.198 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:45.194+0000 7f93c2c2c640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f93bc113c30 con 0x7f93bc109df0 2024-09-21T03:37:45.199 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:45.194+0000 7f93c09c8640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f93bc103b60 0x7f93bc19a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:37:45.199 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:45.194+0000 7f93c09c8640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f93bc103b60 0x7f93bc19a460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:46718/0 (socket says 172.21.15.39:46718) 2024-09-21T03:37:45.199 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:45.194+0000 7f93c09c8640 1 -- 172.21.15.39:0/2000080130 learned_addr learned my addr 172.21.15.39:0/2000080130 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:37:45.199 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:45.194+0000 7f93bbfff640 1 --2- 172.21.15.39:0/2000080130 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f93bc1094a0 0x7f93bc19aa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:37:45.199 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:45.194+0000 7f93c11c9640 1 --2- 172.21.15.39:0/2000080130 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f93bc109df0 0x7f93bc19fca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:37:45.199 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:45.194+0000 7f93c09c8640 1 -- 172.21.15.39:0/2000080130 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f93bc109df0 msgr2=0x7f93bc19fca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:37:45.199 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:45.194+0000 7f93c09c8640 1 --2- 172.21.15.39:0/2000080130 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f93bc109df0 0x7f93bc19fca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:45.199 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:45.194+0000 7f93c09c8640 1 -- 172.21.15.39:0/2000080130 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f93bc1094a0 msgr2=0x7f93bc19aa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:37:45.199 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:45.194+0000 7f93c09c8640 1 --2- 172.21.15.39:0/2000080130 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f93bc1094a0 0x7f93bc19aa00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:45.199 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:45.194+0000 7f93c09c8640 1 -- 172.21.15.39:0/2000080130 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f93bc1a03d0 con 0x7f93bc103b60 2024-09-21T03:37:45.199 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:45.195+0000 7f93c11c9640 1 --2- 172.21.15.39:0/2000080130 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f93bc109df0 0x7f93bc19fca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:37:45.199 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:45.195+0000 7f93c09c8640 1 --2- 172.21.15.39:0/2000080130 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f93bc103b60 0x7f93bc19a460 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f93ac00e9f0 tx=0x7f93ac00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:37:45.199 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:45.195+0000 7f93b9ffb640 1 -- 172.21.15.39:0/2000080130 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93ac00cd80 con 0x7f93bc103b60 2024-09-21T03:37:45.199 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:45.195+0000 7f93b9ffb640 1 -- 172.21.15.39:0/2000080130 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f93ac0108e0 con 0x7f93bc103b60 2024-09-21T03:37:45.200 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:45.195+0000 7f93c2c2c640 1 -- 172.21.15.39:0/2000080130 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f93bc1a06c0 con 0x7f93bc103b60 2024-09-21T03:37:45.200 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:45.195+0000 7f93b9ffb640 1 -- 172.21.15.39:0/2000080130 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93ac018980 con 0x7f93bc103b60 2024-09-21T03:37:45.200 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:45.195+0000 7f93c2c2c640 1 -- 172.21.15.39:0/2000080130 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f93bc1a6e20 con 0x7f93bc103b60 2024-09-21T03:37:45.200 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:45.196+0000 7f93c2c2c640 1 -- 172.21.15.39:0/2000080130 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f93bc103f70 con 0x7f93bc103b60 2024-09-21T03:37:45.200 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:45.198+0000 7f93b9ffb640 1 -- 172.21.15.39:0/2000080130 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 20) ==== 154473+0+0 (secure 0 0 0) 0x7f93ac010430 con 0x7f93bc103b60 2024-09-21T03:37:45.202 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:45.199+0000 7f93b9ffb640 1 --2- 172.21.15.39:0/2000080130 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f93a40b6550 0x7f93a40b8a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:37:45.202 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:45.199+0000 7f93bbfff640 1 --2- 172.21.15.39:0/2000080130 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f93a40b6550 0x7f93a40b8a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:37:45.202 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:45.199+0000 7f93b9ffb640 1 -- 172.21.15.39:0/2000080130 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1370+0+0 (secure 0 0 0) 0x7f93ac0e5f30 con 0x7f93bc103b60 2024-09-21T03:37:45.202 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:45.200+0000 7f93bbfff640 1 --2- 172.21.15.39:0/2000080130 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f93a40b6550 0x7f93a40b8a60 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f93bc109040 tx=0x7f93b0005f90 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:37:45.205 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:45.203+0000 7f93b9ffb640 1 -- 172.21.15.39:0/2000080130 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f93ac0ae3a0 con 0x7f93bc103b60 2024-09-21T03:37:45.395 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:45.393+0000 7f93c2c2c640 1 -- 172.21.15.39:0/2000080130 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command([{prefix=config set, name=debug_client}] v 0) -- 0x7f93bc02e640 con 0x7f93bc103b60 2024-09-21T03:37:45.401 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:45.398+0000 7f93b9ffb640 1 -- 172.21.15.39:0/2000080130 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{prefix=config set, name=debug_client}]=0 v29)=0 v29) ==== 111+0+0 (secure 0 0 0) 0x7f93ac0b2780 con 0x7f93bc103b60 2024-09-21T03:37:45.404 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:45.401+0000 7f93c2c2c640 1 -- 172.21.15.39:0/2000080130 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f93a40b6550 msgr2=0x7f93a40b8a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:37:45.404 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:45.401+0000 7f93c2c2c640 1 --2- 172.21.15.39:0/2000080130 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f93a40b6550 0x7f93a40b8a60 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f93bc109040 tx=0x7f93b0005f90 comp rx=0 tx=0).stop 2024-09-21T03:37:45.404 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:45.401+0000 7f93c2c2c640 1 -- 172.21.15.39:0/2000080130 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f93bc103b60 msgr2=0x7f93bc19a460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:37:45.405 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:45.401+0000 7f93c2c2c640 1 --2- 172.21.15.39:0/2000080130 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f93bc103b60 0x7f93bc19a460 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f93ac00e9f0 tx=0x7f93ac00eeb0 comp rx=0 tx=0).stop 2024-09-21T03:37:45.405 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:45.402+0000 7f93c2c2c640 1 -- 172.21.15.39:0/2000080130 shutdown_connections 2024-09-21T03:37:45.405 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:45.402+0000 7f93c2c2c640 1 --2- 172.21.15.39:0/2000080130 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f93a40b6550 0x7f93a40b8a60 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:45.405 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:45.402+0000 7f93c2c2c640 1 --2- 172.21.15.39:0/2000080130 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f93bc109df0 0x7f93bc19fca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:45.405 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:45.402+0000 7f93c2c2c640 1 --2- 172.21.15.39:0/2000080130 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f93bc1094a0 0x7f93bc19aa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:45.405 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:45.402+0000 7f93c2c2c640 1 --2- 172.21.15.39:0/2000080130 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f93bc103b60 0x7f93bc19a460 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:45.405 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:45.402+0000 7f93c2c2c640 1 -- 172.21.15.39:0/2000080130 >> 172.21.15.39:0/2000080130 conn(0x7f93bc07b850 msgr2=0x7f93bc1074f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:37:45.405 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:45.402+0000 7f93c2c2c640 1 -- 172.21.15.39:0/2000080130 shutdown_connections 2024-09-21T03:37:45.405 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:45.402+0000 7f93c2c2c640 1 -- 172.21.15.39:0/2000080130 wait complete. 2024-09-21T03:37:46.584 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:46.581+0000 7f0df50a6640 1 -- 172.21.15.39:0/3653463230 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0df0101fc0 msgr2=0x7f0df01023d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:37:46.584 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:46.581+0000 7f0df50a6640 1 --2- 172.21.15.39:0/3653463230 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0df0101fc0 0x7f0df01023d0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f0de0009900 tx=0x7f0de002a410 comp rx=0 tx=0).stop 2024-09-21T03:37:46.584 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:46.581+0000 7f0df50a6640 1 -- 172.21.15.39:0/3653463230 shutdown_connections 2024-09-21T03:37:46.584 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:46.581+0000 7f0df50a6640 1 --2- 172.21.15.39:0/3653463230 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0df0109e50 0x7f0df0110680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:46.584 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:46.581+0000 7f0df50a6640 1 --2- 172.21.15.39:0/3653463230 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0df0109500 0x7f0df0109910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:46.584 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:46.581+0000 7f0df50a6640 1 --2- 172.21.15.39:0/3653463230 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0df0101fc0 0x7f0df01023d0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:46.584 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:46.581+0000 7f0df50a6640 1 -- 172.21.15.39:0/3653463230 >> 172.21.15.39:0/3653463230 conn(0x7f0df0079f90 msgr2=0x7f0df007a3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:37:46.585 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:46.581+0000 7f0df50a6640 1 -- 172.21.15.39:0/3653463230 shutdown_connections 2024-09-21T03:37:46.585 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:46.582+0000 7f0df50a6640 1 -- 172.21.15.39:0/3653463230 wait complete. 2024-09-21T03:37:46.586 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:46.582+0000 7f0df50a6640 1 Processor -- start 2024-09-21T03:37:46.586 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:46.583+0000 7f0df50a6640 1 -- start start 2024-09-21T03:37:46.586 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:46.583+0000 7f0df50a6640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0df0101fc0 0x7f0df0102f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:37:46.586 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:46.583+0000 7f0df50a6640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0df0109500 0x7f0df01034a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:37:46.586 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:46.583+0000 7f0df50a6640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0df0109e50 0x7f0df01039e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:37:46.586 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:46.583+0000 7f0df50a6640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f0df0113b10 con 0x7f0df0101fc0 2024-09-21T03:37:46.586 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:46.583+0000 7f0df50a6640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f0df0113990 con 0x7f0df0109500 2024-09-21T03:37:46.586 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:46.583+0000 7f0df50a6640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f0df0113c90 con 0x7f0df0109e50 2024-09-21T03:37:46.586 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:46.583+0000 7f0deed9d640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0df0101fc0 0x7f0df0102f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:37:46.586 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:46.583+0000 7f0deed9d640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0df0101fc0 0x7f0df0102f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:46742/0 (socket says 172.21.15.39:46742) 2024-09-21T03:37:46.586 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:46.583+0000 7f0deed9d640 1 -- 172.21.15.39:0/2836629176 learned_addr learned my addr 172.21.15.39:0/2836629176 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:37:46.587 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:46.583+0000 7f0dee59c640 1 --2- 172.21.15.39:0/2836629176 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0df0109500 0x7f0df01034a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:37:46.587 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:46.583+0000 7f0def59e640 1 --2- 172.21.15.39:0/2836629176 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0df0109e50 0x7f0df01039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:37:46.587 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:46.583+0000 7f0deed9d640 1 -- 172.21.15.39:0/2836629176 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0df0109e50 msgr2=0x7f0df01039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:37:46.587 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:46.583+0000 7f0deed9d640 1 --2- 172.21.15.39:0/2836629176 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0df0109e50 0x7f0df01039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:46.587 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:46.583+0000 7f0deed9d640 1 -- 172.21.15.39:0/2836629176 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0df0109500 msgr2=0x7f0df01034a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:37:46.587 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:46.583+0000 7f0deed9d640 1 --2- 172.21.15.39:0/2836629176 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0df0109500 0x7f0df01034a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:46.587 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:46.583+0000 7f0deed9d640 1 -- 172.21.15.39:0/2836629176 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0df01a87c0 con 0x7f0df0101fc0 2024-09-21T03:37:46.587 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:46.584+0000 7f0deed9d640 1 --2- 172.21.15.39:0/2836629176 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0df0101fc0 0x7f0df0102f60 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f0de0005940 tx=0x7f0de0004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:37:46.588 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:46.584+0000 7f0dd7fff640 1 -- 172.21.15.39:0/2836629176 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0de003f030 con 0x7f0df0101fc0 2024-09-21T03:37:46.588 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:46.584+0000 7f0df50a6640 1 -- 172.21.15.39:0/2836629176 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0df01a8a50 con 0x7f0df0101fc0 2024-09-21T03:37:46.588 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:46.584+0000 7f0dd7fff640 1 -- 172.21.15.39:0/2836629176 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0de0003aa0 con 0x7f0df0101fc0 2024-09-21T03:37:46.588 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:46.584+0000 7f0dd7fff640 1 -- 172.21.15.39:0/2836629176 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0de0035070 con 0x7f0df0101fc0 2024-09-21T03:37:46.588 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:46.584+0000 7f0df50a6640 1 -- 172.21.15.39:0/2836629176 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0df01a8f80 con 0x7f0df0101fc0 2024-09-21T03:37:46.588 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:46.585+0000 7f0df50a6640 1 -- 172.21.15.39:0/2836629176 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0df01023d0 con 0x7f0df0101fc0 2024-09-21T03:37:46.589 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:46.586+0000 7f0dd7fff640 1 -- 172.21.15.39:0/2836629176 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 20) ==== 154473+0+0 (secure 0 0 0) 0x7f0de0042030 con 0x7f0df0101fc0 2024-09-21T03:37:46.590 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:46.587+0000 7f0dd7fff640 1 --2- 172.21.15.39:0/2836629176 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f0dd80b64d0 0x7f0dd80b89e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:37:46.590 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:46.587+0000 7f0dee59c640 1 --2- 172.21.15.39:0/2836629176 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f0dd80b64d0 0x7f0dd80b89e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:37:46.590 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:46.587+0000 7f0dd7fff640 1 -- 172.21.15.39:0/2836629176 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1370+0+0 (secure 0 0 0) 0x7f0de0105de0 con 0x7f0df0101fc0 2024-09-21T03:37:46.590 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:46.588+0000 7f0dee59c640 1 --2- 172.21.15.39:0/2836629176 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f0dd80b64d0 0x7f0dd80b89e0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f0df0109070 tx=0x7f0de4005f90 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:37:46.594 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:46.591+0000 7f0dd7fff640 1 -- 172.21.15.39:0/2836629176 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f0de00d2650 con 0x7f0df0101fc0 2024-09-21T03:37:46.717 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:37:46 smithi161 ceph-mon[34533]: from='client.? 172.21.15.39:0/2000080130' entity='client.admin' 2024-09-21T03:37:46.717 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:37:46 smithi161 ceph-mon[34533]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:37:46.724 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:37:46 smithi150 ceph-mon[34289]: from='client.? 172.21.15.39:0/2000080130' entity='client.admin' 2024-09-21T03:37:46.724 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:37:46 smithi150 ceph-mon[34289]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:37:46.754 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:37:46 smithi039 ceph-mon[31211]: from='client.? 172.21.15.39:0/2000080130' entity='client.admin' 2024-09-21T03:37:46.754 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:37:46 smithi039 ceph-mon[31211]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:37:46.785 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:46.783+0000 7f0df50a6640 1 -- 172.21.15.39:0/2836629176 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command([{prefix=config set, name=client_mount_timeout}] v 0) -- 0x7f0df002e640 con 0x7f0df0101fc0 2024-09-21T03:37:46.791 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:46.788+0000 7f0dd7fff640 1 -- 172.21.15.39:0/2836629176 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{prefix=config set, name=client_mount_timeout}]=0 v30)=0 v30) ==== 119+0+0 (secure 0 0 0) 0x7f0de00d7590 con 0x7f0df0101fc0 2024-09-21T03:37:46.794 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:46.791+0000 7f0df50a6640 1 -- 172.21.15.39:0/2836629176 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f0dd80b64d0 msgr2=0x7f0dd80b89e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:37:46.794 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:46.791+0000 7f0df50a6640 1 --2- 172.21.15.39:0/2836629176 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f0dd80b64d0 0x7f0dd80b89e0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f0df0109070 tx=0x7f0de4005f90 comp rx=0 tx=0).stop 2024-09-21T03:37:46.794 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:46.791+0000 7f0df50a6640 1 -- 172.21.15.39:0/2836629176 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0df0101fc0 msgr2=0x7f0df0102f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:37:46.794 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:46.791+0000 7f0df50a6640 1 --2- 172.21.15.39:0/2836629176 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0df0101fc0 0x7f0df0102f60 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f0de0005940 tx=0x7f0de0004270 comp rx=0 tx=0).stop 2024-09-21T03:37:46.795 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:46.792+0000 7f0df50a6640 1 -- 172.21.15.39:0/2836629176 shutdown_connections 2024-09-21T03:37:46.795 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:46.792+0000 7f0df50a6640 1 --2- 172.21.15.39:0/2836629176 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f0dd80b64d0 0x7f0dd80b89e0 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:46.795 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:46.792+0000 7f0df50a6640 1 --2- 172.21.15.39:0/2836629176 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0df0109e50 0x7f0df01039e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:46.795 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:46.792+0000 7f0df50a6640 1 --2- 172.21.15.39:0/2836629176 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0df0109500 0x7f0df01034a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:46.795 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:46.792+0000 7f0df50a6640 1 --2- 172.21.15.39:0/2836629176 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0df0101fc0 0x7f0df0102f60 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:46.795 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:46.792+0000 7f0df50a6640 1 -- 172.21.15.39:0/2836629176 >> 172.21.15.39:0/2836629176 conn(0x7f0df0079f90 msgr2=0x7f0df0107590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:37:46.795 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:46.792+0000 7f0df50a6640 1 -- 172.21.15.39:0/2836629176 shutdown_connections 2024-09-21T03:37:46.795 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:46.792+0000 7f0df50a6640 1 -- 172.21.15.39:0/2836629176 wait complete. 2024-09-21T03:37:47.967 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:37:47 smithi161 ceph-mon[34533]: from='client.? 172.21.15.39:0/2836629176' entity='client.admin' 2024-09-21T03:37:47.967 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:37:47 smithi161 ceph-mon[34533]: from='client.? 172.21.15.39:0/681662804' entity='client.admin' 2024-09-21T03:37:48.062 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:37:47 smithi039 ceph-mon[31211]: from='client.? 172.21.15.39:0/2836629176' entity='client.admin' 2024-09-21T03:37:48.063 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:37:47 smithi039 ceph-mon[31211]: from='client.? 172.21.15.39:0/681662804' entity='client.admin' 2024-09-21T03:37:48.076 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:48.073+0000 7f666dd49640 1 -- 172.21.15.39:0/3020562576 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6668108cf0 msgr2=0x7f666810f520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:37:48.076 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:48.073+0000 7f666dd49640 1 --2- 172.21.15.39:0/3020562576 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6668108cf0 0x7f666810f520 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f6660009ca0 tx=0x7f666002a8c0 comp rx=0 tx=0).stop 2024-09-21T03:37:48.076 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:48.073+0000 7f666dd49640 1 -- 172.21.15.39:0/3020562576 shutdown_connections 2024-09-21T03:37:48.076 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:48.073+0000 7f666dd49640 1 --2- 172.21.15.39:0/3020562576 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6668108cf0 0x7f666810f520 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:48.076 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:48.073+0000 7f666dd49640 1 --2- 172.21.15.39:0/3020562576 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f66681083a0 0x7f66681087b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:48.076 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:48.073+0000 7f666dd49640 1 --2- 172.21.15.39:0/3020562576 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6668102a60 0x7f6668102e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:48.076 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:48.073+0000 7f666dd49640 1 -- 172.21.15.39:0/3020562576 >> 172.21.15.39:0/3020562576 conn(0x7f6668079730 msgr2=0x7f6668079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:37:48.076 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:48.073+0000 7f666dd49640 1 -- 172.21.15.39:0/3020562576 shutdown_connections 2024-09-21T03:37:48.076 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:48.073+0000 7f666dd49640 1 -- 172.21.15.39:0/3020562576 wait complete. 2024-09-21T03:37:48.078 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:48.074+0000 7f666dd49640 1 Processor -- start 2024-09-21T03:37:48.078 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:48.074+0000 7f666dd49640 1 -- start start 2024-09-21T03:37:48.078 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:48.074+0000 7f666dd49640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6668102a60 0x7f6668199350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:37:48.078 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:48.074+0000 7f666dd49640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f66681083a0 0x7f66681998f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:37:48.078 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:48.074+0000 7f666dd49640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6668108cf0 0x7f666819eb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:37:48.079 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:48.074+0000 7f666dd49640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f6668112960 con 0x7f66681083a0 2024-09-21T03:37:48.079 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:48.074+0000 7f666dd49640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f66681127e0 con 0x7f6668102a60 2024-09-21T03:37:48.079 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:48.074+0000 7f666dd49640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f6668112ae0 con 0x7f6668108cf0 2024-09-21T03:37:48.079 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:48.074+0000 7f6666ffd640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f66681083a0 0x7f66681998f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:37:48.079 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:48.074+0000 7f6666ffd640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f66681083a0 0x7f66681998f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:34788/0 (socket says 172.21.15.39:34788) 2024-09-21T03:37:48.079 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:48.074+0000 7f6666ffd640 1 -- 172.21.15.39:0/390677902 learned_addr learned my addr 172.21.15.39:0/390677902 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:37:48.079 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:48.074+0000 7f66677fe640 1 --2- 172.21.15.39:0/390677902 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6668102a60 0x7f6668199350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:37:48.079 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:48.075+0000 7f6667fff640 1 --2- 172.21.15.39:0/390677902 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6668108cf0 0x7f666819eb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:37:48.079 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:48.075+0000 7f6666ffd640 1 -- 172.21.15.39:0/390677902 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6668108cf0 msgr2=0x7f666819eb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:37:48.079 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:48.075+0000 7f6666ffd640 1 --2- 172.21.15.39:0/390677902 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6668108cf0 0x7f666819eb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:48.079 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:48.075+0000 7f6666ffd640 1 -- 172.21.15.39:0/390677902 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6668102a60 msgr2=0x7f6668199350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:37:48.079 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:48.075+0000 7f6666ffd640 1 --2- 172.21.15.39:0/390677902 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6668102a60 0x7f6668199350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:48.079 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:48.075+0000 7f6666ffd640 1 -- 172.21.15.39:0/390677902 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f666819f2c0 con 0x7f66681083a0 2024-09-21T03:37:48.080 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:48.075+0000 7f66677fe640 1 --2- 172.21.15.39:0/390677902 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6668102a60 0x7f6668199350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:37:48.080 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:48.075+0000 7f6666ffd640 1 --2- 172.21.15.39:0/390677902 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f66681083a0 0x7f66681998f0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f665c00d770 tx=0x7f665c00dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:37:48.080 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:48.075+0000 7f6664ff9640 1 -- 172.21.15.39:0/390677902 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f665c017070 con 0x7f66681083a0 2024-09-21T03:37:48.080 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:48.075+0000 7f6664ff9640 1 -- 172.21.15.39:0/390677902 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f665c009c30 con 0x7f66681083a0 2024-09-21T03:37:48.080 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:48.075+0000 7f6664ff9640 1 -- 172.21.15.39:0/390677902 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f665c00f620 con 0x7f66681083a0 2024-09-21T03:37:48.080 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:48.075+0000 7f666dd49640 1 -- 172.21.15.39:0/390677902 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f666819f5b0 con 0x7f66681083a0 2024-09-21T03:37:48.080 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:48.075+0000 7f666dd49640 1 -- 172.21.15.39:0/390677902 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f66681a5d10 con 0x7f66681083a0 2024-09-21T03:37:48.080 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:48.076+0000 7f666dd49640 1 -- 172.21.15.39:0/390677902 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6668102e70 con 0x7f66681083a0 2024-09-21T03:37:48.080 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:48.078+0000 7f6664ff9640 1 -- 172.21.15.39:0/390677902 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 20) ==== 154473+0+0 (secure 0 0 0) 0x7f665c01a070 con 0x7f66681083a0 2024-09-21T03:37:48.081 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:48.079+0000 7f6664ff9640 1 --2- 172.21.15.39:0/390677902 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f66500b64d0 0x7f66500b89e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:37:48.081 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:48.079+0000 7f66677fe640 1 --2- 172.21.15.39:0/390677902 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f66500b64d0 0x7f66500b89e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:37:48.081 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:48.079+0000 7f6664ff9640 1 -- 172.21.15.39:0/390677902 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1370+0+0 (secure 0 0 0) 0x7f665c0e5120 con 0x7f66681083a0 2024-09-21T03:37:48.082 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:48.079+0000 7f66677fe640 1 --2- 172.21.15.39:0/390677902 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f66500b64d0 0x7f66500b89e0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f665800bfd0 tx=0x7f665800b390 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:37:48.084 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:48.082+0000 7f6664ff9640 1 -- 172.21.15.39:0/390677902 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f665c009de0 con 0x7f66681083a0 2024-09-21T03:37:48.224 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:37:47 smithi150 ceph-mon[34289]: from='client.? 172.21.15.39:0/2836629176' entity='client.admin' 2024-09-21T03:37:48.225 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:37:47 smithi150 ceph-mon[34289]: from='client.? 172.21.15.39:0/681662804' entity='client.admin' 2024-09-21T03:37:48.276 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:48.274+0000 7f666dd49640 1 -- 172.21.15.39:0/390677902 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command([{prefix=config set, name=mds_bal_split_size}] v 0) -- 0x7f666819a480 con 0x7f66681083a0 2024-09-21T03:37:48.282 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:48.279+0000 7f6664ff9640 1 -- 172.21.15.39:0/390677902 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_bal_split_size}]=0 v32)=0 v32) ==== 118+0+0 (secure 0 0 0) 0x7f666819a480 con 0x7f66681083a0 2024-09-21T03:37:48.285 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:48.282+0000 7f666dd49640 1 -- 172.21.15.39:0/390677902 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f66500b64d0 msgr2=0x7f66500b89e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:37:48.285 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:48.282+0000 7f666dd49640 1 --2- 172.21.15.39:0/390677902 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f66500b64d0 0x7f66500b89e0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f665800bfd0 tx=0x7f665800b390 comp rx=0 tx=0).stop 2024-09-21T03:37:48.285 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:48.282+0000 7f666dd49640 1 -- 172.21.15.39:0/390677902 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f66681083a0 msgr2=0x7f66681998f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:37:48.285 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:48.282+0000 7f666dd49640 1 --2- 172.21.15.39:0/390677902 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f66681083a0 0x7f66681998f0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f665c00d770 tx=0x7f665c00dc30 comp rx=0 tx=0).stop 2024-09-21T03:37:48.285 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:48.282+0000 7f666dd49640 1 -- 172.21.15.39:0/390677902 shutdown_connections 2024-09-21T03:37:48.285 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:48.282+0000 7f666dd49640 1 --2- 172.21.15.39:0/390677902 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f66500b64d0 0x7f66500b89e0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:48.285 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:48.282+0000 7f666dd49640 1 --2- 172.21.15.39:0/390677902 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6668108cf0 0x7f666819eb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:48.285 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:48.282+0000 7f666dd49640 1 --2- 172.21.15.39:0/390677902 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f66681083a0 0x7f66681998f0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:48.285 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:48.282+0000 7f666dd49640 1 --2- 172.21.15.39:0/390677902 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6668102a60 0x7f6668199350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:48.285 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:48.282+0000 7f666dd49640 1 -- 172.21.15.39:0/390677902 >> 172.21.15.39:0/390677902 conn(0x7f6668079730 msgr2=0x7f6668104410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:37:48.285 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:48.282+0000 7f666dd49640 1 -- 172.21.15.39:0/390677902 shutdown_connections 2024-09-21T03:37:48.285 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:48.282+0000 7f666dd49640 1 -- 172.21.15.39:0/390677902 wait complete. 2024-09-21T03:37:49.083 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:37:48 smithi039 ceph-mon[31211]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:37:49.084 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:37:48 smithi039 ceph-mon[31211]: from='client.? 172.21.15.39:0/390677902' entity='client.admin' 2024-09-21T03:37:49.218 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:37:48 smithi161 ceph-mon[34533]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:37:49.218 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:37:48 smithi161 ceph-mon[34533]: from='client.? 172.21.15.39:0/390677902' entity='client.admin' 2024-09-21T03:37:49.224 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:37:48 smithi150 ceph-mon[34289]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:37:49.224 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:37:48 smithi150 ceph-mon[34289]: from='client.? 172.21.15.39:0/390677902' entity='client.admin' 2024-09-21T03:37:50.034 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:35.480+0000 7f35acb2e640 1 -- 172.21.15.39:0/4286507973 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f35a8102a70 msgr2=0x7f35a8102e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:37:50.034 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:35.480+0000 7f35acb2e640 1 --2- 172.21.15.39:0/4286507973 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f35a8102a70 0x7f35a8102e80 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f3598009900 tx=0x7f359802a410 comp rx=0 tx=0).stop 2024-09-21T03:37:50.034 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:35.480+0000 7f35acb2e640 1 -- 172.21.15.39:0/4286507973 shutdown_connections 2024-09-21T03:37:50.035 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:35.480+0000 7f35acb2e640 1 --2- 172.21.15.39:0/4286507973 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f35a8108d00 0x7f35a810f530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:50.035 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:35.480+0000 7f35acb2e640 1 --2- 172.21.15.39:0/4286507973 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f35a81083b0 0x7f35a81087c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:50.035 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:35.480+0000 7f35acb2e640 1 --2- 172.21.15.39:0/4286507973 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f35a8102a70 0x7f35a8102e80 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:50.035 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:35.480+0000 7f35acb2e640 1 -- 172.21.15.39:0/4286507973 >> 172.21.15.39:0/4286507973 conn(0x7f35a8079730 msgr2=0x7f35a8079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:37:50.035 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:35.480+0000 7f35acb2e640 1 -- 172.21.15.39:0/4286507973 shutdown_connections 2024-09-21T03:37:50.035 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:35.480+0000 7f35acb2e640 1 -- 172.21.15.39:0/4286507973 wait complete. 2024-09-21T03:37:50.035 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:35.481+0000 7f35acb2e640 1 Processor -- start 2024-09-21T03:37:50.035 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:35.481+0000 7f35acb2e640 1 -- start start 2024-09-21T03:37:50.035 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:35.481+0000 7f35acb2e640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f35a8102a70 0x7f35a8104260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:37:50.035 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:35.481+0000 7f35acb2e640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f35a81083b0 0x7f35a81047a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:37:50.035 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:35.481+0000 7f35acb2e640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f35a8108d00 0x7f35a8104ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:37:50.036 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:35.481+0000 7f35acb2e640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f35a8112970 con 0x7f35a8102a70 2024-09-21T03:37:50.036 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:35.481+0000 7f35acb2e640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f35a81127f0 con 0x7f35a81083b0 2024-09-21T03:37:50.036 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:35.481+0000 7f35acb2e640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f35a8112af0 con 0x7f35a8108d00 2024-09-21T03:37:50.036 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:35.481+0000 7f35a659c640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f35a8102a70 0x7f35a8104260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:37:50.036 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:35.481+0000 7f35a659c640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f35a8102a70 0x7f35a8104260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:49816/0 (socket says 172.21.15.39:49816) 2024-09-21T03:37:50.036 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:35.481+0000 7f35a659c640 1 -- 172.21.15.39:0/1684594876 learned_addr learned my addr 172.21.15.39:0/1684594876 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:37:50.036 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:35.481+0000 7f35a5d9b640 1 --2- 172.21.15.39:0/1684594876 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f35a81083b0 0x7f35a81047a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:37:50.036 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:35.481+0000 7f35a6d9d640 1 --2- 172.21.15.39:0/1684594876 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f35a8108d00 0x7f35a8104ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:37:50.036 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:35.481+0000 7f35a659c640 1 -- 172.21.15.39:0/1684594876 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f35a8108d00 msgr2=0x7f35a8104ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:37:50.036 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:35.482+0000 7f35a659c640 1 --2- 172.21.15.39:0/1684594876 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f35a8108d00 0x7f35a8104ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:50.036 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:35.482+0000 7f35a659c640 1 -- 172.21.15.39:0/1684594876 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f35a81083b0 msgr2=0x7f35a81047a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:37:50.036 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:35.482+0000 7f35a659c640 1 --2- 172.21.15.39:0/1684594876 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f35a81083b0 0x7f35a81047a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:50.036 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:35.482+0000 7f35a659c640 1 -- 172.21.15.39:0/1684594876 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f35a81a8730 con 0x7f35a8102a70 2024-09-21T03:37:50.036 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:35.482+0000 7f35a5d9b640 1 --2- 172.21.15.39:0/1684594876 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f35a81083b0 0x7f35a81047a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:37:50.037 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:35.482+0000 7f35a6d9d640 1 --2- 172.21.15.39:0/1684594876 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f35a8108d00 0x7f35a8104ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:37:50.037 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:35.482+0000 7f35a659c640 1 --2- 172.21.15.39:0/1684594876 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f35a8102a70 0x7f35a8104260 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f3598005940 tx=0x7f35980040c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:37:50.037 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:35.482+0000 7f35977fe640 1 -- 172.21.15.39:0/1684594876 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f359803f030 con 0x7f35a8102a70 2024-09-21T03:37:50.037 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:35.482+0000 7f35acb2e640 1 -- 172.21.15.39:0/1684594876 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f35a81a89c0 con 0x7f35a8102a70 2024-09-21T03:37:50.037 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:35.482+0000 7f35977fe640 1 -- 172.21.15.39:0/1684594876 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f35980044c0 con 0x7f35a8102a70 2024-09-21T03:37:50.037 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:35.482+0000 7f35977fe640 1 -- 172.21.15.39:0/1684594876 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3598035070 con 0x7f35a8102a70 2024-09-21T03:37:50.037 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:35.482+0000 7f35acb2e640 1 -- 172.21.15.39:0/1684594876 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f35a81a8ea0 con 0x7f35a8102a70 2024-09-21T03:37:50.037 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:35.483+0000 7f35acb2e640 1 -- 172.21.15.39:0/1684594876 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f35a8103cf0 con 0x7f35a8102a70 2024-09-21T03:37:50.037 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:35.483+0000 7f35977fe640 1 -- 172.21.15.39:0/1684594876 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 18) ==== 51915+0+0 (secure 0 0 0) 0x7f3598042030 con 0x7f35a8102a70 2024-09-21T03:37:50.037 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:35.484+0000 7f35977fe640 1 --2- 172.21.15.39:0/1684594876 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f359003fb70 0x7f3590042080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:37:50.037 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:35.484+0000 7f35977fe640 1 -- 172.21.15.39:0/1684594876 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1370+0+0 (secure 0 0 0) 0x7f35980746b0 con 0x7f35a8102a70 2024-09-21T03:37:50.037 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:35.484+0000 7f35a5d9b640 1 --2- 172.21.15.39:0/1684594876 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f359003fb70 0x7f3590042080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:37:50.037 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:35.484+0000 7f35a5d9b640 1 --2- 172.21.15.39:0/1684594876 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f359003fb70 0x7f3590042080 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f35a8100af0 tx=0x7f359c005f90 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:37:50.038 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:35.489+0000 7f35977fe640 1 -- 172.21.15.39:0/1684594876 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f35980303a0 con 0x7f35a8102a70 2024-09-21T03:37:50.038 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:35.686+0000 7f35acb2e640 1 -- 172.21.15.39:0/1684594876 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command([{prefix=config set, name=mds_bal_fragment_size_max}] v 0) -- 0x7f35a81a9210 con 0x7f35a8102a70 2024-09-21T03:37:50.038 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:35.692+0000 7f35977fe640 1 -- 172.21.15.39:0/1684594876 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_bal_fragment_size_max}]=0 v17)=0 v17) ==== 127+0+0 (secure 0 0 0) 0x7f359802f300 con 0x7f35a8102a70 2024-09-21T03:37:50.038 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:35.694+0000 7f35acb2e640 1 -- 172.21.15.39:0/1684594876 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f359003fb70 msgr2=0x7f3590042080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:37:50.038 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:35.694+0000 7f35acb2e640 1 --2- 172.21.15.39:0/1684594876 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f359003fb70 0x7f3590042080 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f35a8100af0 tx=0x7f359c005f90 comp rx=0 tx=0).stop 2024-09-21T03:37:50.038 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:35.694+0000 7f35acb2e640 1 -- 172.21.15.39:0/1684594876 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f35a8102a70 msgr2=0x7f35a8104260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:37:50.038 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:35.694+0000 7f35acb2e640 1 --2- 172.21.15.39:0/1684594876 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f35a8102a70 0x7f35a8104260 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f3598005940 tx=0x7f35980040c0 comp rx=0 tx=0).stop 2024-09-21T03:37:50.038 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:35.694+0000 7f35acb2e640 1 -- 172.21.15.39:0/1684594876 shutdown_connections 2024-09-21T03:37:50.038 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:35.694+0000 7f35acb2e640 1 --2- 172.21.15.39:0/1684594876 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f359003fb70 0x7f3590042080 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:50.038 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:35.694+0000 7f35acb2e640 1 --2- 172.21.15.39:0/1684594876 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f35a8108d00 0x7f35a8104ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:50.038 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:35.694+0000 7f35acb2e640 1 --2- 172.21.15.39:0/1684594876 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f35a81083b0 0x7f35a81047a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:50.038 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:35.694+0000 7f35acb2e640 1 --2- 172.21.15.39:0/1684594876 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f35a8102a70 0x7f35a8104260 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:50.039 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:35.694+0000 7f35acb2e640 1 -- 172.21.15.39:0/1684594876 >> 172.21.15.39:0/1684594876 conn(0x7f35a8079730 msgr2=0x7f35a8101ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:37:50.039 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:35.695+0000 7f35acb2e640 1 -- 172.21.15.39:0/1684594876 shutdown_connections 2024-09-21T03:37:50.039 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:35.695+0000 7f35acb2e640 1 -- 172.21.15.39:0/1684594876 wait complete. 2024-09-21T03:37:50.436 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:47.912+0000 7fbb0c93b640 1 -- 172.21.15.39:0/422982927 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbb08108d10 msgr2=0x7fbb0810f540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:37:50.436 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:47.912+0000 7fbb0c93b640 1 --2- 172.21.15.39:0/422982927 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbb08108d10 0x7fbb0810f540 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fbb00009ca0 tx=0x7fbb0002a8c0 comp rx=0 tx=0).stop 2024-09-21T03:37:50.436 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:47.912+0000 7fbb0c93b640 1 -- 172.21.15.39:0/422982927 shutdown_connections 2024-09-21T03:37:50.436 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:47.912+0000 7fbb0c93b640 1 --2- 172.21.15.39:0/422982927 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbb08108d10 0x7fbb0810f540 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:50.436 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:47.912+0000 7fbb0c93b640 1 --2- 172.21.15.39:0/422982927 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbb081083c0 0x7fbb081087d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:50.436 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:47.912+0000 7fbb0c93b640 1 --2- 172.21.15.39:0/422982927 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbb08102a80 0x7fbb08102e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:50.436 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:47.912+0000 7fbb0c93b640 1 -- 172.21.15.39:0/422982927 >> 172.21.15.39:0/422982927 conn(0x7fbb08079740 msgr2=0x7fbb08079b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:37:50.436 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:47.912+0000 7fbb0c93b640 1 -- 172.21.15.39:0/422982927 shutdown_connections 2024-09-21T03:37:50.436 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:47.912+0000 7fbb0c93b640 1 -- 172.21.15.39:0/422982927 wait complete. 2024-09-21T03:37:50.437 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:47.913+0000 7fbb0c93b640 1 Processor -- start 2024-09-21T03:37:50.437 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:47.913+0000 7fbb0c93b640 1 -- start start 2024-09-21T03:37:50.437 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:47.913+0000 7fbb0c93b640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbb08102a80 0x7fbb0819b1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:37:50.437 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:47.914+0000 7fbb0c93b640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbb081083c0 0x7fbb0819b700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:37:50.437 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:47.914+0000 7fbb0c93b640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbb08108d10 0x7fbb0819bc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:37:50.437 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:47.914+0000 7fbb0c93b640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fbb081129d0 con 0x7fbb08102a80 2024-09-21T03:37:50.437 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:47.914+0000 7fbb0c93b640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fbb08112850 con 0x7fbb08108d10 2024-09-21T03:37:50.437 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:47.914+0000 7fbb0c93b640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fbb08112b50 con 0x7fbb081083c0 2024-09-21T03:37:50.437 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:47.914+0000 7fbb0659c640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbb08102a80 0x7fbb0819b1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:37:50.437 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:47.914+0000 7fbb0659c640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbb08102a80 0x7fbb0819b1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:34776/0 (socket says 172.21.15.39:34776) 2024-09-21T03:37:50.437 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:47.914+0000 7fbb0659c640 1 -- 172.21.15.39:0/2229915500 learned_addr learned my addr 172.21.15.39:0/2229915500 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:37:50.437 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:47.914+0000 7fbb05d9b640 1 --2- 172.21.15.39:0/2229915500 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbb081083c0 0x7fbb0819b700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:37:50.437 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:47.914+0000 7fbb0659c640 1 -- 172.21.15.39:0/2229915500 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbb081083c0 msgr2=0x7fbb0819b700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:37:50.438 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:47.914+0000 7fbb06d9d640 1 --2- 172.21.15.39:0/2229915500 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbb08108d10 0x7fbb0819bc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:37:50.438 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:47.914+0000 7fbb0659c640 1 --2- 172.21.15.39:0/2229915500 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbb081083c0 0x7fbb0819b700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:50.438 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:47.914+0000 7fbb0659c640 1 -- 172.21.15.39:0/2229915500 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbb08108d10 msgr2=0x7fbb0819bc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:37:50.438 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:47.914+0000 7fbb0659c640 1 --2- 172.21.15.39:0/2229915500 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbb08108d10 0x7fbb0819bc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:50.438 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:47.914+0000 7fbb0659c640 1 -- 172.21.15.39:0/2229915500 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbb081a6440 con 0x7fbb08102a80 2024-09-21T03:37:50.438 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:47.914+0000 7fbb0659c640 1 --2- 172.21.15.39:0/2229915500 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbb08102a80 0x7fbb0819b1c0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fbaf800c920 tx=0x7fbaf800cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:37:50.438 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:47.914+0000 7fbaff7fe640 1 -- 172.21.15.39:0/2229915500 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbaf8007c70 con 0x7fbb08102a80 2024-09-21T03:37:50.438 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:47.914+0000 7fbaff7fe640 1 -- 172.21.15.39:0/2229915500 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbaf8007e10 con 0x7fbb08102a80 2024-09-21T03:37:50.438 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:47.914+0000 7fbb0c93b640 1 -- 172.21.15.39:0/2229915500 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbb081a6730 con 0x7fbb08102a80 2024-09-21T03:37:50.438 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:47.914+0000 7fbaff7fe640 1 -- 172.21.15.39:0/2229915500 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbaf8018820 con 0x7fbb08102a80 2024-09-21T03:37:50.438 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:47.914+0000 7fbb0c93b640 1 -- 172.21.15.39:0/2229915500 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbb081a6cc0 con 0x7fbb08102a80 2024-09-21T03:37:50.438 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:47.915+0000 7fbb0c93b640 1 -- 172.21.15.39:0/2229915500 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbb08103d00 con 0x7fbb08102a80 2024-09-21T03:37:50.438 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:47.917+0000 7fbaff7fe640 1 -- 172.21.15.39:0/2229915500 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 20) ==== 154473+0+0 (secure 0 0 0) 0x7fbaf800f450 con 0x7fbb08102a80 2024-09-21T03:37:50.439 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:47.919+0000 7fbaff7fe640 1 --2- 172.21.15.39:0/2229915500 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7fbaf40b6550 0x7fbaf40b8a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:37:50.439 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:47.919+0000 7fbb05d9b640 1 --2- 172.21.15.39:0/2229915500 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7fbaf40b6550 0x7fbaf40b8a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:37:50.439 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:47.919+0000 7fbaff7fe640 1 -- 172.21.15.39:0/2229915500 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1370+0+0 (secure 0 0 0) 0x7fbaf80e5b50 con 0x7fbb08102a80 2024-09-21T03:37:50.439 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:47.919+0000 7fbb05d9b640 1 --2- 172.21.15.39:0/2229915500 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7fbaf40b6550 0x7fbaf40b8a60 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fbb08105f20 tx=0x7fbaf000b590 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:37:50.439 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:47.922+0000 7fbaff7fe640 1 -- 172.21.15.39:0/2229915500 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fbaf8014030 con 0x7fbb08102a80 2024-09-21T03:37:50.439 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:48.113+0000 7fbb0c93b640 1 -- 172.21.15.39:0/2229915500 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command([{prefix=config set, name=debug_ms}] v 0) -- 0x7fbb0806bfb0 con 0x7fbb08102a80 2024-09-21T03:37:50.439 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:48.114+0000 7fbaff7fe640 1 -- 172.21.15.39:0/2229915500 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{prefix=config set, name=debug_ms}]=0 v31)=0 v31) ==== 106+0+0 (secure 0 0 0) 0x7fbaf80ae0f0 con 0x7fbb08102a80 2024-09-21T03:37:50.439 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:48.116+0000 7fbb0c93b640 1 -- 172.21.15.39:0/2229915500 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7fbaf40b6550 msgr2=0x7fbaf40b8a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:37:50.439 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:48.116+0000 7fbb0c93b640 1 --2- 172.21.15.39:0/2229915500 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7fbaf40b6550 0x7fbaf40b8a60 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fbb08105f20 tx=0x7fbaf000b590 comp rx=0 tx=0).stop 2024-09-21T03:37:50.439 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:48.116+0000 7fbb0c93b640 1 -- 172.21.15.39:0/2229915500 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbb08102a80 msgr2=0x7fbb0819b1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:37:50.439 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:48.116+0000 7fbb0c93b640 1 --2- 172.21.15.39:0/2229915500 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbb08102a80 0x7fbb0819b1c0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fbaf800c920 tx=0x7fbaf800cde0 comp rx=0 tx=0).stop 2024-09-21T03:37:50.439 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:48.116+0000 7fbb0c93b640 1 -- 172.21.15.39:0/2229915500 shutdown_connections 2024-09-21T03:37:50.439 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:48.116+0000 7fbb0c93b640 1 --2- 172.21.15.39:0/2229915500 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7fbaf40b6550 0x7fbaf40b8a60 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:50.440 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:48.116+0000 7fbb0c93b640 1 --2- 172.21.15.39:0/2229915500 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbb08108d10 0x7fbb0819bc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:50.440 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:48.116+0000 7fbb0c93b640 1 --2- 172.21.15.39:0/2229915500 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbb081083c0 0x7fbb0819b700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:50.440 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:48.116+0000 7fbb0c93b640 1 --2- 172.21.15.39:0/2229915500 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbb08102a80 0x7fbb0819b1c0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:50.440 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:48.116+0000 7fbb0c93b640 1 -- 172.21.15.39:0/2229915500 >> 172.21.15.39:0/2229915500 conn(0x7fbb08079740 msgr2=0x7fbb08104330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:37:50.440 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:48.116+0000 7fbb0c93b640 1 -- 172.21.15.39:0/2229915500 shutdown_connections 2024-09-21T03:37:50.440 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:48.116+0000 7fbb0c93b640 1 -- 172.21.15.39:0/2229915500 wait complete. 2024-09-21T03:37:50.563 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '51116d5c-77ca-11ef-baf5-efdc52797490', '--', 'ceph', 'config', 'set', 'client', 'rados_osd_op_timeout', '900'], check_status=True, hostname='smithi039') 2024-09-21T03:37:50.563 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '51116d5c-77ca-11ef-baf5-efdc52797490', '--', 'ceph', 'config', 'set', 'mds', 'debug_mds', '20'], check_status=True, hostname='smithi039') 2024-09-21T03:37:50.563 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '51116d5c-77ca-11ef-baf5-efdc52797490', '--', 'ceph', 'config', 'set', 'mds', 'debug_mds_balancer', '20'], check_status=True, hostname='smithi039') 2024-09-21T03:37:50.563 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '51116d5c-77ca-11ef-baf5-efdc52797490', '--', 'ceph', 'config', 'set', 'mds', 'debug_ms', '2'], check_status=True, hostname='smithi039') 2024-09-21T03:37:50.854 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:47.416+0000 7fe1f0d9e640 1 -- 172.21.15.39:0/1834850432 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe1ec102a80 msgr2=0x7fe1ec102e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:37:50.854 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:47.416+0000 7fe1f0d9e640 1 --2- 172.21.15.39:0/1834850432 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe1ec102a80 0x7fe1ec102e90 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fe1dc009900 tx=0x7fe1dc02a410 comp rx=0 tx=0).stop 2024-09-21T03:37:50.854 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:47.416+0000 7fe1f0d9e640 1 -- 172.21.15.39:0/1834850432 shutdown_connections 2024-09-21T03:37:50.854 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:47.416+0000 7fe1f0d9e640 1 --2- 172.21.15.39:0/1834850432 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe1ec108d10 0x7fe1ec10f540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:50.854 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:47.416+0000 7fe1f0d9e640 1 --2- 172.21.15.39:0/1834850432 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe1ec1083c0 0x7fe1ec1087d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:50.854 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:47.416+0000 7fe1f0d9e640 1 --2- 172.21.15.39:0/1834850432 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe1ec102a80 0x7fe1ec102e90 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:50.854 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:47.416+0000 7fe1f0d9e640 1 -- 172.21.15.39:0/1834850432 >> 172.21.15.39:0/1834850432 conn(0x7fe1ec079740 msgr2=0x7fe1ec079b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:37:50.855 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:47.416+0000 7fe1f0d9e640 1 -- 172.21.15.39:0/1834850432 shutdown_connections 2024-09-21T03:37:50.855 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:47.416+0000 7fe1f0d9e640 1 -- 172.21.15.39:0/1834850432 wait complete. 2024-09-21T03:37:50.855 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:47.417+0000 7fe1f0d9e640 1 Processor -- start 2024-09-21T03:37:50.855 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:47.417+0000 7fe1f0d9e640 1 -- start start 2024-09-21T03:37:50.855 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:47.417+0000 7fe1f0d9e640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe1ec102a80 0x7fe1ec19a450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:37:50.855 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:47.417+0000 7fe1f0d9e640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe1ec1083c0 0x7fe1ec19a9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:37:50.855 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:47.417+0000 7fe1f0d9e640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe1ec108d10 0x7fe1ec19fc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:37:50.855 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:47.417+0000 7fe1f0d9e640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fe1ec113aa0 con 0x7fe1ec108d10 2024-09-21T03:37:50.855 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:47.417+0000 7fe1f0d9e640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fe1ec113920 con 0x7fe1ec102a80 2024-09-21T03:37:50.855 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:47.417+0000 7fe1f0d9e640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fe1ec113c20 con 0x7fe1ec1083c0 2024-09-21T03:37:50.855 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:47.417+0000 7fe1ead9d640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe1ec108d10 0x7fe1ec19fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:37:50.855 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:47.417+0000 7fe1ead9d640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe1ec108d10 0x7fe1ec19fc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:34764/0 (socket says 172.21.15.39:34764) 2024-09-21T03:37:50.855 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:47.417+0000 7fe1ead9d640 1 -- 172.21.15.39:0/681662804 learned_addr learned my addr 172.21.15.39:0/681662804 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:37:50.855 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:47.417+0000 7fe1ea59c640 1 --2- 172.21.15.39:0/681662804 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe1ec102a80 0x7fe1ec19a450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:37:50.855 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:47.418+0000 7fe1e9d9b640 1 --2- 172.21.15.39:0/681662804 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe1ec1083c0 0x7fe1ec19a9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:37:50.856 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:47.418+0000 7fe1ead9d640 1 -- 172.21.15.39:0/681662804 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe1ec1083c0 msgr2=0x7fe1ec19a9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:37:50.856 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:47.418+0000 7fe1ead9d640 1 --2- 172.21.15.39:0/681662804 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe1ec1083c0 0x7fe1ec19a9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:50.856 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:47.418+0000 7fe1ead9d640 1 -- 172.21.15.39:0/681662804 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe1ec102a80 msgr2=0x7fe1ec19a450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:37:50.856 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:47.418+0000 7fe1ead9d640 1 --2- 172.21.15.39:0/681662804 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe1ec102a80 0x7fe1ec19a450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:50.856 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:47.418+0000 7fe1ead9d640 1 -- 172.21.15.39:0/681662804 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe1ec1a0300 con 0x7fe1ec108d10 2024-09-21T03:37:50.856 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:47.418+0000 7fe1ead9d640 1 --2- 172.21.15.39:0/681662804 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe1ec108d10 0x7fe1ec19fc90 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fe1e400eaa0 tx=0x7fe1e400ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:37:50.856 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:47.418+0000 7fe1db7fe640 1 -- 172.21.15.39:0/681662804 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1e400cdb0 con 0x7fe1ec108d10 2024-09-21T03:37:50.856 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:47.418+0000 7fe1db7fe640 1 -- 172.21.15.39:0/681662804 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe1e4005020 con 0x7fe1ec108d10 2024-09-21T03:37:50.856 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:47.418+0000 7fe1db7fe640 1 -- 172.21.15.39:0/681662804 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1e40116d0 con 0x7fe1ec108d10 2024-09-21T03:37:50.856 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:47.418+0000 7fe1f0d9e640 1 -- 172.21.15.39:0/681662804 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe1ec1a05f0 con 0x7fe1ec108d10 2024-09-21T03:37:50.856 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:47.418+0000 7fe1f0d9e640 1 -- 172.21.15.39:0/681662804 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe1ec1a6e10 con 0x7fe1ec108d10 2024-09-21T03:37:50.856 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:47.419+0000 7fe1f0d9e640 1 -- 172.21.15.39:0/681662804 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe1ec103d00 con 0x7fe1ec108d10 2024-09-21T03:37:50.856 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:47.421+0000 7fe1db7fe640 1 -- 172.21.15.39:0/681662804 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 20) ==== 154473+0+0 (secure 0 0 0) 0x7fe1e40051d0 con 0x7fe1ec108d10 2024-09-21T03:37:50.857 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:47.421+0000 7fe1db7fe640 1 --2- 172.21.15.39:0/681662804 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7fe1d40b6500 0x7fe1d40b8a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:37:50.857 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:47.422+0000 7fe1ea59c640 1 --2- 172.21.15.39:0/681662804 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7fe1d40b6500 0x7fe1d40b8a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:37:50.857 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:47.422+0000 7fe1db7fe640 1 -- 172.21.15.39:0/681662804 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1370+0+0 (secure 0 0 0) 0x7fe1e40e58c0 con 0x7fe1ec108d10 2024-09-21T03:37:50.857 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:47.422+0000 7fe1ea59c640 1 --2- 172.21.15.39:0/681662804 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7fe1d40b6500 0x7fe1d40b8a10 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fe1dc031440 tx=0x7fe1dc009560 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:37:50.857 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:47.426+0000 7fe1db7fe640 1 -- 172.21.15.39:0/681662804 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fe1e4016030 con 0x7fe1ec108d10 2024-09-21T03:37:50.857 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:47.637+0000 7fe1f0d9e640 1 -- 172.21.15.39:0/681662804 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command([{prefix=config set, name=mds_debug_frag}] v 0) -- 0x7fe1ec02e640 con 0x7fe1ec108d10 2024-09-21T03:37:50.857 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:47.643+0000 7fe1db7fe640 1 -- 172.21.15.39:0/681662804 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_debug_frag}]=0 v31)=0 v31) ==== 115+0+0 (secure 0 0 0) 0x7fe1e40add50 con 0x7fe1ec108d10 2024-09-21T03:37:50.857 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:47.645+0000 7fe1f0d9e640 1 -- 172.21.15.39:0/681662804 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7fe1d40b6500 msgr2=0x7fe1d40b8a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:37:50.857 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:47.645+0000 7fe1f0d9e640 1 --2- 172.21.15.39:0/681662804 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7fe1d40b6500 0x7fe1d40b8a10 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fe1dc031440 tx=0x7fe1dc009560 comp rx=0 tx=0).stop 2024-09-21T03:37:50.857 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:47.645+0000 7fe1f0d9e640 1 -- 172.21.15.39:0/681662804 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe1ec108d10 msgr2=0x7fe1ec19fc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:37:50.857 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:47.645+0000 7fe1f0d9e640 1 --2- 172.21.15.39:0/681662804 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe1ec108d10 0x7fe1ec19fc90 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fe1e400eaa0 tx=0x7fe1e400ef60 comp rx=0 tx=0).stop 2024-09-21T03:37:50.857 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:47.645+0000 7fe1f0d9e640 1 -- 172.21.15.39:0/681662804 shutdown_connections 2024-09-21T03:37:50.857 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:47.645+0000 7fe1f0d9e640 1 --2- 172.21.15.39:0/681662804 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7fe1d40b6500 0x7fe1d40b8a10 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:50.857 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:47.645+0000 7fe1f0d9e640 1 --2- 172.21.15.39:0/681662804 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe1ec108d10 0x7fe1ec19fc90 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:50.858 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:47.645+0000 7fe1f0d9e640 1 --2- 172.21.15.39:0/681662804 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe1ec1083c0 0x7fe1ec19a9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:50.858 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:47.645+0000 7fe1f0d9e640 1 --2- 172.21.15.39:0/681662804 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe1ec102a80 0x7fe1ec19a450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:50.858 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:47.645+0000 7fe1f0d9e640 1 -- 172.21.15.39:0/681662804 >> 172.21.15.39:0/681662804 conn(0x7fe1ec079740 msgr2=0x7fe1ec1013f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:37:50.858 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:47.645+0000 7fe1f0d9e640 1 -- 172.21.15.39:0/681662804 shutdown_connections 2024-09-21T03:37:50.858 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:47.645+0000 7fe1f0d9e640 1 -- 172.21.15.39:0/681662804 wait complete. 2024-09-21T03:37:51.083 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:37:50 smithi039 ceph-mon[31211]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:37:51.218 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:37:50 smithi161 ceph-mon[34533]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:37:51.224 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:37:50 smithi150 ceph-mon[34289]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:37:52.943 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:37:52 smithi039 ceph-mon[31211]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:37:52.956 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:52.953+0000 7f1fce451640 1 -- 172.21.15.39:0/3810598249 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1fc8109df0 msgr2=0x7f1fc8110620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:37:52.956 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:52.953+0000 7f1fce451640 1 --2- 172.21.15.39:0/3810598249 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1fc8109df0 0x7f1fc8110620 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f1fc0009ca0 tx=0x7f1fc002a8c0 comp rx=0 tx=0).stop 2024-09-21T03:37:52.956 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:52.953+0000 7f1fce451640 1 -- 172.21.15.39:0/3810598249 shutdown_connections 2024-09-21T03:37:52.956 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:52.953+0000 7f1fce451640 1 --2- 172.21.15.39:0/3810598249 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1fc8109df0 0x7f1fc8110620 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:52.956 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:52.953+0000 7f1fce451640 1 --2- 172.21.15.39:0/3810598249 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1fc81094a0 0x7f1fc81098b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:52.957 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:52.953+0000 7f1fce451640 1 --2- 172.21.15.39:0/3810598249 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1fc8103b60 0x7f1fc8103f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:52.957 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:52.953+0000 7f1fce451640 1 -- 172.21.15.39:0/3810598249 >> 172.21.15.39:0/3810598249 conn(0x7f1fc80ff5d0 msgr2=0x7f1fc8101a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:37:52.957 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:52.953+0000 7f1fce451640 1 -- 172.21.15.39:0/3810598249 shutdown_connections 2024-09-21T03:37:52.957 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:52.953+0000 7f1fce451640 1 -- 172.21.15.39:0/3810598249 wait complete. 2024-09-21T03:37:52.958 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:52.954+0000 7f1fce451640 1 Processor -- start 2024-09-21T03:37:52.958 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:52.954+0000 7f1fce451640 1 -- start start 2024-09-21T03:37:52.959 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:52.955+0000 7f1fce451640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1fc8103b60 0x7f1fc8075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:37:52.959 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:52.955+0000 7f1fce451640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1fc81094a0 0x7f1fc8075c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:37:52.959 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:52.955+0000 7f1fce451640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1fc8109df0 0x7f1fc8076190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:37:52.959 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:52.955+0000 7f1fce451640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f1fc8113ab0 con 0x7f1fc81094a0 2024-09-21T03:37:52.959 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:52.955+0000 7f1fce451640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f1fc8113930 con 0x7f1fc8109df0 2024-09-21T03:37:52.959 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:52.955+0000 7f1fce451640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f1fc8113c30 con 0x7f1fc8103b60 2024-09-21T03:37:52.959 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:52.955+0000 7f1fc77fe640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1fc81094a0 0x7f1fc8075c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:37:52.959 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:52.955+0000 7f1fc77fe640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1fc81094a0 0x7f1fc8075c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:34814/0 (socket says 172.21.15.39:34814) 2024-09-21T03:37:52.959 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:52.955+0000 7f1fc77fe640 1 -- 172.21.15.39:0/2438298538 learned_addr learned my addr 172.21.15.39:0/2438298538 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:37:52.959 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:52.955+0000 7f1fcc9ee640 1 --2- 172.21.15.39:0/2438298538 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1fc8109df0 0x7f1fc8076190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:37:52.959 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:52.955+0000 7f1fc7fff640 1 --2- 172.21.15.39:0/2438298538 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1fc8103b60 0x7f1fc8075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:37:52.959 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:52.955+0000 7f1fc77fe640 1 -- 172.21.15.39:0/2438298538 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1fc8103b60 msgr2=0x7f1fc8075710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:37:52.959 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:52.955+0000 7f1fc77fe640 1 --2- 172.21.15.39:0/2438298538 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1fc8103b60 0x7f1fc8075710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:52.960 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:52.955+0000 7f1fc77fe640 1 -- 172.21.15.39:0/2438298538 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1fc8109df0 msgr2=0x7f1fc8076190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:37:52.960 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:52.955+0000 7f1fc77fe640 1 --2- 172.21.15.39:0/2438298538 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1fc8109df0 0x7f1fc8076190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:52.960 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:52.955+0000 7f1fc77fe640 1 -- 172.21.15.39:0/2438298538 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1fc81a8700 con 0x7f1fc81094a0 2024-09-21T03:37:52.960 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:52.955+0000 7f1fc77fe640 1 --2- 172.21.15.39:0/2438298538 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1fc81094a0 0x7f1fc8075c50 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f1fbc00c970 tx=0x7f1fbc00ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:37:52.960 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:52.956+0000 7f1fc57fa640 1 -- 172.21.15.39:0/2438298538 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fbc007bb0 con 0x7f1fc81094a0 2024-09-21T03:37:52.960 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:52.956+0000 7f1fc57fa640 1 -- 172.21.15.39:0/2438298538 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1fbc007d50 con 0x7f1fc81094a0 2024-09-21T03:37:52.960 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:52.956+0000 7f1fce451640 1 -- 172.21.15.39:0/2438298538 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1fc81a89f0 con 0x7f1fc81094a0 2024-09-21T03:37:52.960 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:52.956+0000 7f1fc57fa640 1 -- 172.21.15.39:0/2438298538 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fbc00f450 con 0x7f1fc81094a0 2024-09-21T03:37:52.960 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:52.956+0000 7f1fce451640 1 -- 172.21.15.39:0/2438298538 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1fc81a8f30 con 0x7f1fc81094a0 2024-09-21T03:37:52.961 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:52.957+0000 7f1fce451640 1 -- 172.21.15.39:0/2438298538 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1fc8104de0 con 0x7f1fc81094a0 2024-09-21T03:37:52.961 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:52.958+0000 7f1fc57fa640 1 -- 172.21.15.39:0/2438298538 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 20) ==== 154473+0+0 (secure 0 0 0) 0x7f1fbc004eb0 con 0x7f1fc81094a0 2024-09-21T03:37:52.962 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:52.959+0000 7f1fc57fa640 1 --2- 172.21.15.39:0/2438298538 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f1fb00b64d0 0x7f1fb00b89e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:37:52.962 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:52.960+0000 7f1fc7fff640 1 --2- 172.21.15.39:0/2438298538 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f1fb00b64d0 0x7f1fb00b89e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:37:52.962 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:52.960+0000 7f1fc57fa640 1 -- 172.21.15.39:0/2438298538 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1370+0+0 (secure 0 0 0) 0x7f1fbc0e5750 con 0x7f1fc81094a0 2024-09-21T03:37:52.962 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:52.960+0000 7f1fc7fff640 1 --2- 172.21.15.39:0/2438298538 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f1fb00b64d0 0x7f1fb00b89e0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f1fc8077160 tx=0x7f1fb800b410 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:37:52.965 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:52.963+0000 7f1fc57fa640 1 -- 172.21.15.39:0/2438298538 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f1fbc014030 con 0x7f1fc81094a0 2024-09-21T03:37:53.161 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:53.159+0000 7f1fce451640 1 -- 172.21.15.39:0/2438298538 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command([{prefix=config set, name=mds_bal_merge_size}] v 0) -- 0x7f1fc8103f70 con 0x7f1fc81094a0 2024-09-21T03:37:53.165 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:53.163+0000 7f1fc57fa640 1 -- 172.21.15.39:0/2438298538 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_bal_merge_size}]=0 v34)=0 v34) ==== 116+0+0 (secure 0 0 0) 0x7f1fbc0b1f60 con 0x7f1fc81094a0 2024-09-21T03:37:53.168 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:53.165+0000 7f1fce451640 1 -- 172.21.15.39:0/2438298538 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f1fb00b64d0 msgr2=0x7f1fb00b89e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:37:53.168 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:53.165+0000 7f1fce451640 1 --2- 172.21.15.39:0/2438298538 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f1fb00b64d0 0x7f1fb00b89e0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f1fc8077160 tx=0x7f1fb800b410 comp rx=0 tx=0).stop 2024-09-21T03:37:53.168 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:53.165+0000 7f1fce451640 1 -- 172.21.15.39:0/2438298538 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1fc81094a0 msgr2=0x7f1fc8075c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:37:53.168 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:53.165+0000 7f1fce451640 1 --2- 172.21.15.39:0/2438298538 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1fc81094a0 0x7f1fc8075c50 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f1fbc00c970 tx=0x7f1fbc00ce30 comp rx=0 tx=0).stop 2024-09-21T03:37:53.168 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:53.165+0000 7f1fce451640 1 -- 172.21.15.39:0/2438298538 shutdown_connections 2024-09-21T03:37:53.168 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:53.165+0000 7f1fce451640 1 --2- 172.21.15.39:0/2438298538 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f1fb00b64d0 0x7f1fb00b89e0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:53.168 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:53.165+0000 7f1fce451640 1 --2- 172.21.15.39:0/2438298538 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1fc8109df0 0x7f1fc8076190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:53.168 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:53.165+0000 7f1fce451640 1 --2- 172.21.15.39:0/2438298538 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1fc81094a0 0x7f1fc8075c50 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:53.168 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:53.165+0000 7f1fce451640 1 --2- 172.21.15.39:0/2438298538 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1fc8103b60 0x7f1fc8075710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:53.169 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:53.165+0000 7f1fce451640 1 -- 172.21.15.39:0/2438298538 >> 172.21.15.39:0/2438298538 conn(0x7f1fc80ff5d0 msgr2=0x7f1fc81019f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:37:53.169 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:53.165+0000 7f1fce451640 1 -- 172.21.15.39:0/2438298538 shutdown_connections 2024-09-21T03:37:53.169 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:53.165+0000 7f1fce451640 1 -- 172.21.15.39:0/2438298538 wait complete. 2024-09-21T03:37:53.217 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:37:52 smithi161 ceph-mon[34533]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:37:53.224 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:37:52 smithi150 ceph-mon[34289]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:37:54.066 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:52.851+0000 7f0e7415b640 1 -- 172.21.15.39:0/3840510767 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0e6c104a30 msgr2=0x7f0e6c104e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:37:54.067 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:52.851+0000 7f0e7415b640 1 --2- 172.21.15.39:0/3840510767 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0e6c104a30 0x7f0e6c104e40 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f0e68009900 tx=0x7f0e6802a410 comp rx=0 tx=0).stop 2024-09-21T03:37:54.067 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:52.851+0000 7f0e7415b640 1 -- 172.21.15.39:0/3840510767 shutdown_connections 2024-09-21T03:37:54.067 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:52.851+0000 7f0e7415b640 1 --2- 172.21.15.39:0/3840510767 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0e6c10de60 0x7f0e6c110330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:54.067 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:52.851+0000 7f0e7415b640 1 --2- 172.21.15.39:0/3840510767 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0e6c105380 0x7f0e6c10d8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:54.067 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:52.851+0000 7f0e7415b640 1 --2- 172.21.15.39:0/3840510767 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0e6c104a30 0x7f0e6c104e40 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:54.067 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:52.851+0000 7f0e7415b640 1 -- 172.21.15.39:0/3840510767 >> 172.21.15.39:0/3840510767 conn(0x7f0e6c0797b0 msgr2=0x7f0e6c079bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:37:54.067 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:52.851+0000 7f0e7415b640 1 -- 172.21.15.39:0/3840510767 shutdown_connections 2024-09-21T03:37:54.067 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:52.851+0000 7f0e7415b640 1 -- 172.21.15.39:0/3840510767 wait complete. 2024-09-21T03:37:54.067 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:52.852+0000 7f0e7415b640 1 Processor -- start 2024-09-21T03:37:54.067 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:52.852+0000 7f0e7415b640 1 -- start start 2024-09-21T03:37:54.067 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:52.852+0000 7f0e7415b640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0e6c104a30 0x7f0e6c102770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:37:54.068 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:52.852+0000 7f0e7415b640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0e6c105380 0x7f0e6c102cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:37:54.068 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:52.852+0000 7f0e7415b640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0e6c10de60 0x7f0e6c1031f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:37:54.068 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:52.852+0000 7f0e7415b640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f0e6c113aa0 con 0x7f0e6c104a30 2024-09-21T03:37:54.068 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:52.852+0000 7f0e7415b640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f0e6c113920 con 0x7f0e6c105380 2024-09-21T03:37:54.068 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:52.852+0000 7f0e7415b640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f0e6c113c20 con 0x7f0e6c10de60 2024-09-21T03:37:54.068 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:52.852+0000 7f0e71ef7640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0e6c104a30 0x7f0e6c102770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:37:54.068 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:52.852+0000 7f0e71ef7640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0e6c104a30 0x7f0e6c102770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:34794/0 (socket says 172.21.15.39:34794) 2024-09-21T03:37:54.068 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:52.852+0000 7f0e71ef7640 1 -- 172.21.15.39:0/961399667 learned_addr learned my addr 172.21.15.39:0/961399667 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:37:54.068 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:52.852+0000 7f0e726f8640 1 --2- 172.21.15.39:0/961399667 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0e6c10de60 0x7f0e6c1031f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:37:54.068 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:52.853+0000 7f0e716f6640 1 --2- 172.21.15.39:0/961399667 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0e6c105380 0x7f0e6c102cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:37:54.068 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:52.853+0000 7f0e71ef7640 1 -- 172.21.15.39:0/961399667 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0e6c10de60 msgr2=0x7f0e6c1031f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:37:54.068 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:52.853+0000 7f0e71ef7640 1 --2- 172.21.15.39:0/961399667 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0e6c10de60 0x7f0e6c1031f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:54.068 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:52.853+0000 7f0e71ef7640 1 -- 172.21.15.39:0/961399667 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0e6c105380 msgr2=0x7f0e6c102cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:37:54.069 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:52.853+0000 7f0e71ef7640 1 --2- 172.21.15.39:0/961399667 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0e6c105380 0x7f0e6c102cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:54.069 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:52.853+0000 7f0e71ef7640 1 -- 172.21.15.39:0/961399667 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0e6c1a8790 con 0x7f0e6c104a30 2024-09-21T03:37:54.069 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:52.853+0000 7f0e726f8640 1 --2- 172.21.15.39:0/961399667 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0e6c10de60 0x7f0e6c1031f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:37:54.069 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:52.853+0000 7f0e71ef7640 1 --2- 172.21.15.39:0/961399667 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0e6c104a30 0x7f0e6c102770 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f0e68005940 tx=0x7f0e680042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:37:54.069 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:52.853+0000 7f0e62ffd640 1 -- 172.21.15.39:0/961399667 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e6803f030 con 0x7f0e6c104a30 2024-09-21T03:37:54.069 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:52.853+0000 7f0e62ffd640 1 -- 172.21.15.39:0/961399667 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0e68003cb0 con 0x7f0e6c104a30 2024-09-21T03:37:54.069 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:52.853+0000 7f0e7415b640 1 -- 172.21.15.39:0/961399667 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0e6c1a8a20 con 0x7f0e6c104a30 2024-09-21T03:37:54.069 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:52.853+0000 7f0e62ffd640 1 -- 172.21.15.39:0/961399667 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e68035070 con 0x7f0e6c104a30 2024-09-21T03:37:54.069 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:52.853+0000 7f0e7415b640 1 -- 172.21.15.39:0/961399667 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0e6c1a8f50 con 0x7f0e6c104a30 2024-09-21T03:37:54.069 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:52.854+0000 7f0e7415b640 1 -- 172.21.15.39:0/961399667 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0e6c104e40 con 0x7f0e6c104a30 2024-09-21T03:37:54.069 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:52.856+0000 7f0e62ffd640 1 -- 172.21.15.39:0/961399667 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 20) ==== 154473+0+0 (secure 0 0 0) 0x7f0e68042030 con 0x7f0e6c104a30 2024-09-21T03:37:54.069 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:52.856+0000 7f0e62ffd640 1 --2- 172.21.15.39:0/961399667 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f0e580b6550 0x7f0e580b8a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:37:54.069 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:52.857+0000 7f0e716f6640 1 --2- 172.21.15.39:0/961399667 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f0e580b6550 0x7f0e580b8a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:37:54.069 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:52.857+0000 7f0e62ffd640 1 -- 172.21.15.39:0/961399667 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1370+0+0 (secure 0 0 0) 0x7f0e68106270 con 0x7f0e6c104a30 2024-09-21T03:37:54.070 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:52.857+0000 7f0e716f6640 1 --2- 172.21.15.39:0/961399667 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f0e580b6550 0x7f0e580b8a60 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f0e5c005fd0 tx=0x7f0e5c005850 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:37:54.070 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:52.860+0000 7f0e62ffd640 1 -- 172.21.15.39:0/961399667 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f0e680ce700 con 0x7f0e6c104a30 2024-09-21T03:37:54.070 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:53.054+0000 7f0e7415b640 1 -- 172.21.15.39:0/961399667 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command([{prefix=config set, name=rados_osd_op_timeout}] v 0) -- 0x7f0e6c103a70 con 0x7f0e6c104a30 2024-09-21T03:37:54.070 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:53.062+0000 7f0e62ffd640 1 -- 172.21.15.39:0/961399667 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{prefix=config set, name=rados_osd_op_timeout}]=0 v33)=0 v33) ==== 120+0+0 (secure 0 0 0) 0x7f0e680d2ae0 con 0x7f0e6c104a30 2024-09-21T03:37:54.070 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:53.065+0000 7f0e7415b640 1 -- 172.21.15.39:0/961399667 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f0e580b6550 msgr2=0x7f0e580b8a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:37:54.070 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:53.065+0000 7f0e7415b640 1 --2- 172.21.15.39:0/961399667 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f0e580b6550 0x7f0e580b8a60 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f0e5c005fd0 tx=0x7f0e5c005850 comp rx=0 tx=0).stop 2024-09-21T03:37:54.070 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:53.065+0000 7f0e7415b640 1 -- 172.21.15.39:0/961399667 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0e6c104a30 msgr2=0x7f0e6c102770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:37:54.070 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:53.065+0000 7f0e7415b640 1 --2- 172.21.15.39:0/961399667 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0e6c104a30 0x7f0e6c102770 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f0e68005940 tx=0x7f0e680042f0 comp rx=0 tx=0).stop 2024-09-21T03:37:54.070 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:53.065+0000 7f0e7415b640 1 -- 172.21.15.39:0/961399667 shutdown_connections 2024-09-21T03:37:54.070 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:53.065+0000 7f0e7415b640 1 --2- 172.21.15.39:0/961399667 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f0e580b6550 0x7f0e580b8a60 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:54.070 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:53.065+0000 7f0e7415b640 1 --2- 172.21.15.39:0/961399667 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0e6c10de60 0x7f0e6c1031f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:54.070 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:53.065+0000 7f0e7415b640 1 --2- 172.21.15.39:0/961399667 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0e6c105380 0x7f0e6c102cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:54.070 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:53.065+0000 7f0e7415b640 1 --2- 172.21.15.39:0/961399667 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0e6c104a30 0x7f0e6c102770 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:54.070 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:53.065+0000 7f0e7415b640 1 -- 172.21.15.39:0/961399667 >> 172.21.15.39:0/961399667 conn(0x7f0e6c0797b0 msgr2=0x7f0e6c100890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:37:54.071 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:53.065+0000 7f0e7415b640 1 -- 172.21.15.39:0/961399667 shutdown_connections 2024-09-21T03:37:54.071 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:53.065+0000 7f0e7415b640 1 -- 172.21.15.39:0/961399667 wait complete. 2024-09-21T03:37:54.333 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:37:54 smithi039 ceph-mon[31211]: from='client.? 172.21.15.39:0/961399667' entity='client.admin' 2024-09-21T03:37:54.333 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:37:54 smithi039 ceph-mon[31211]: from='client.? 172.21.15.39:0/2438298538' entity='client.admin' 2024-09-21T03:37:54.467 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:37:54 smithi161 ceph-mon[34533]: from='client.? 172.21.15.39:0/961399667' entity='client.admin' 2024-09-21T03:37:54.468 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:37:54 smithi161 ceph-mon[34533]: from='client.? 172.21.15.39:0/2438298538' entity='client.admin' 2024-09-21T03:37:54.474 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:37:54 smithi150 ceph-mon[34289]: from='client.? 172.21.15.39:0/961399667' entity='client.admin' 2024-09-21T03:37:54.474 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:37:54 smithi150 ceph-mon[34289]: from='client.? 172.21.15.39:0/2438298538' entity='client.admin' 2024-09-21T03:37:55.333 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:37:55 smithi039 ceph-mon[31211]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:37:55.333 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:37:55 smithi039 ceph-mon[31211]: from='client.? 172.21.15.39:0/3268487474' entity='client.admin' 2024-09-21T03:37:55.467 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:37:55 smithi161 ceph-mon[34533]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:37:55.468 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:37:55 smithi161 ceph-mon[34533]: from='client.? 172.21.15.39:0/3268487474' entity='client.admin' 2024-09-21T03:37:55.474 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:37:55 smithi150 ceph-mon[34289]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:37:55.474 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:37:55 smithi150 ceph-mon[34289]: from='client.? 172.21.15.39:0/3268487474' entity='client.admin' 2024-09-21T03:37:55.615 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:55.612+0000 7fad1be22640 1 -- 172.21.15.39:0/2929979897 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fad141083f0 msgr2=0x7fad14108800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:37:55.615 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:55.612+0000 7fad1be22640 1 --2- 172.21.15.39:0/2929979897 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fad141083f0 0x7fad14108800 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fad04009900 tx=0x7fad0402a410 comp rx=0 tx=0).stop 2024-09-21T03:37:55.615 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:55.612+0000 7fad1be22640 1 -- 172.21.15.39:0/2929979897 shutdown_connections 2024-09-21T03:37:55.615 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:55.612+0000 7fad1be22640 1 --2- 172.21.15.39:0/2929979897 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fad14108d40 0x7fad1410f570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:55.615 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:55.612+0000 7fad1be22640 1 --2- 172.21.15.39:0/2929979897 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fad141083f0 0x7fad14108800 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:55.616 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:55.612+0000 7fad1be22640 1 --2- 172.21.15.39:0/2929979897 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fad14102ab0 0x7fad14102ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:55.616 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:55.612+0000 7fad1be22640 1 -- 172.21.15.39:0/2929979897 >> 172.21.15.39:0/2929979897 conn(0x7fad140796b0 msgr2=0x7fad14079ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:37:55.616 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:55.613+0000 7fad1be22640 1 -- 172.21.15.39:0/2929979897 shutdown_connections 2024-09-21T03:37:55.616 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:55.613+0000 7fad1be22640 1 -- 172.21.15.39:0/2929979897 wait complete. 2024-09-21T03:37:55.617 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:55.613+0000 7fad1be22640 1 Processor -- start 2024-09-21T03:37:55.617 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:55.613+0000 7fad1be22640 1 -- start start 2024-09-21T03:37:55.618 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:55.614+0000 7fad1be22640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fad14102ab0 0x7fad14110000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:37:55.618 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:55.614+0000 7fad1be22640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fad141083f0 0x7fad14110540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:37:55.618 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:55.614+0000 7fad1be22640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fad14108d40 0x7fad14110a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:37:55.618 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:55.614+0000 7fad1be22640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fad14113ad0 con 0x7fad141083f0 2024-09-21T03:37:55.618 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:55.614+0000 7fad1be22640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fad14113950 con 0x7fad14108d40 2024-09-21T03:37:55.618 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:55.614+0000 7fad1be22640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fad14113c50 con 0x7fad14102ab0 2024-09-21T03:37:55.618 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:55.614+0000 7fad193bd640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fad141083f0 0x7fad14110540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:37:55.618 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:55.614+0000 7fad193bd640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fad141083f0 0x7fad14110540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:34852/0 (socket says 172.21.15.39:34852) 2024-09-21T03:37:55.618 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:55.614+0000 7fad193bd640 1 -- 172.21.15.39:0/3358714540 learned_addr learned my addr 172.21.15.39:0/3358714540 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:37:55.618 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:55.614+0000 7fad19bbe640 1 --2- 172.21.15.39:0/3358714540 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fad14102ab0 0x7fad14110000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:37:55.618 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:55.614+0000 7fad193bd640 1 -- 172.21.15.39:0/3358714540 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fad14102ab0 msgr2=0x7fad14110000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:37:55.618 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:55.614+0000 7fad193bd640 1 --2- 172.21.15.39:0/3358714540 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fad14102ab0 0x7fad14110000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:55.619 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:55.614+0000 7fad193bd640 1 -- 172.21.15.39:0/3358714540 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fad14108d40 msgr2=0x7fad14110a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:37:55.619 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:55.614+0000 7fad193bd640 1 --2- 172.21.15.39:0/3358714540 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fad14108d40 0x7fad14110a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:55.619 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:55.614+0000 7fad193bd640 1 -- 172.21.15.39:0/3358714540 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fad141a8670 con 0x7fad141083f0 2024-09-21T03:37:55.619 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:55.614+0000 7fad193bd640 1 --2- 172.21.15.39:0/3358714540 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fad141083f0 0x7fad14110540 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fad04032fd0 tx=0x7fad04004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:37:55.619 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:55.614+0000 7fad0affd640 1 -- 172.21.15.39:0/3358714540 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad0403f030 con 0x7fad141083f0 2024-09-21T03:37:55.619 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:55.614+0000 7fad0affd640 1 -- 172.21.15.39:0/3358714540 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fad04004490 con 0x7fad141083f0 2024-09-21T03:37:55.619 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:55.614+0000 7fad0affd640 1 -- 172.21.15.39:0/3358714540 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad04035070 con 0x7fad141083f0 2024-09-21T03:37:55.619 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:55.614+0000 7fad1be22640 1 -- 172.21.15.39:0/3358714540 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fad141a8900 con 0x7fad141083f0 2024-09-21T03:37:55.619 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:55.615+0000 7fad1be22640 1 -- 172.21.15.39:0/3358714540 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fad141a8e30 con 0x7fad141083f0 2024-09-21T03:37:55.620 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:55.616+0000 7fad1be22640 1 -- 172.21.15.39:0/3358714540 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fad14102ec0 con 0x7fad141083f0 2024-09-21T03:37:55.620 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:55.617+0000 7fad0affd640 1 -- 172.21.15.39:0/3358714540 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 20) ==== 154473+0+0 (secure 0 0 0) 0x7fad04042030 con 0x7fad141083f0 2024-09-21T03:37:55.621 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:55.618+0000 7fad0affd640 1 --2- 172.21.15.39:0/3358714540 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7fad000b6550 0x7fad000b8a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:37:55.621 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:55.618+0000 7fad19bbe640 1 --2- 172.21.15.39:0/3358714540 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7fad000b6550 0x7fad000b8a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:37:55.621 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:55.618+0000 7fad0affd640 1 -- 172.21.15.39:0/3358714540 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1370+0+0 (secure 0 0 0) 0x7fad04105f70 con 0x7fad141083f0 2024-09-21T03:37:55.621 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:55.619+0000 7fad19bbe640 1 --2- 172.21.15.39:0/3358714540 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7fad000b6550 0x7fad000b8a60 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fad14111a50 tx=0x7fad1000a2d0 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:37:55.624 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:55.621+0000 7fad0affd640 1 -- 172.21.15.39:0/3358714540 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fad040ce400 con 0x7fad141083f0 2024-09-21T03:37:55.814 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:55.811+0000 7fad1be22640 1 -- 172.21.15.39:0/3358714540 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command([{prefix=config set, name=debug_mgr}] v 0) -- 0x7fad1402e640 con 0x7fad141083f0 2024-09-21T03:37:55.814 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:55.812+0000 7fad0affd640 1 -- 172.21.15.39:0/3358714540 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{prefix=config set, name=debug_mgr}]=0 v35)=0 v35) ==== 108+0+0 (secure 0 0 0) 0x7fad040d27e0 con 0x7fad141083f0 2024-09-21T03:37:55.817 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:55.814+0000 7fad1be22640 1 -- 172.21.15.39:0/3358714540 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7fad000b6550 msgr2=0x7fad000b8a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:37:55.817 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:55.814+0000 7fad1be22640 1 --2- 172.21.15.39:0/3358714540 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7fad000b6550 0x7fad000b8a60 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fad14111a50 tx=0x7fad1000a2d0 comp rx=0 tx=0).stop 2024-09-21T03:37:55.817 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:55.814+0000 7fad1be22640 1 -- 172.21.15.39:0/3358714540 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fad141083f0 msgr2=0x7fad14110540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:37:55.817 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:55.814+0000 7fad1be22640 1 --2- 172.21.15.39:0/3358714540 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fad141083f0 0x7fad14110540 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fad04032fd0 tx=0x7fad04004270 comp rx=0 tx=0).stop 2024-09-21T03:37:55.817 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:55.814+0000 7fad1be22640 1 -- 172.21.15.39:0/3358714540 shutdown_connections 2024-09-21T03:37:55.817 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:55.814+0000 7fad1be22640 1 --2- 172.21.15.39:0/3358714540 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7fad000b6550 0x7fad000b8a60 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:55.817 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:55.814+0000 7fad1be22640 1 --2- 172.21.15.39:0/3358714540 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fad14108d40 0x7fad14110a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:55.817 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:55.814+0000 7fad1be22640 1 --2- 172.21.15.39:0/3358714540 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fad141083f0 0x7fad14110540 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:55.817 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:55.814+0000 7fad1be22640 1 --2- 172.21.15.39:0/3358714540 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fad14102ab0 0x7fad14110000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:55.817 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:55.814+0000 7fad1be22640 1 -- 172.21.15.39:0/3358714540 >> 172.21.15.39:0/3358714540 conn(0x7fad140796b0 msgr2=0x7fad141052f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:37:55.818 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:55.814+0000 7fad1be22640 1 -- 172.21.15.39:0/3358714540 shutdown_connections 2024-09-21T03:37:55.818 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:55.814+0000 7fad1be22640 1 -- 172.21.15.39:0/3358714540 wait complete. 2024-09-21T03:37:57.467 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:37:57 smithi161 ceph-mon[34533]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:37:57.474 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:37:57 smithi150 ceph-mon[34289]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:37:57.583 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:37:57 smithi039 ceph-mon[31211]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:37:57.772 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:57.768+0000 7f39d2318640 1 -- 172.21.15.39:0/2852166633 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f39cc105290 msgr2=0x7f39cc10d8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:37:57.772 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:57.768+0000 7f39d2318640 1 --2- 172.21.15.39:0/2852166633 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f39cc105290 0x7f39cc10d8d0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f39c0009900 tx=0x7f39c002a410 comp rx=0 tx=0).stop 2024-09-21T03:37:57.772 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:57.768+0000 7f39d2318640 1 -- 172.21.15.39:0/2852166633 shutdown_connections 2024-09-21T03:37:57.772 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:57.768+0000 7f39d2318640 1 --2- 172.21.15.39:0/2852166633 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f39cc10de10 0x7f39cc1102f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:57.772 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:57.768+0000 7f39d2318640 1 --2- 172.21.15.39:0/2852166633 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f39cc105290 0x7f39cc10d8d0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:57.772 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:57.768+0000 7f39d2318640 1 --2- 172.21.15.39:0/2852166633 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f39cc104860 0x7f39cc104c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:57.772 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:57.768+0000 7f39d2318640 1 -- 172.21.15.39:0/2852166633 >> 172.21.15.39:0/2852166633 conn(0x7f39cc079730 msgr2=0x7f39cc079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:37:57.773 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:57.768+0000 7f39d2318640 1 -- 172.21.15.39:0/2852166633 shutdown_connections 2024-09-21T03:37:57.773 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:57.768+0000 7f39d2318640 1 -- 172.21.15.39:0/2852166633 wait complete. 2024-09-21T03:37:57.773 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:57.769+0000 7f39d2318640 1 Processor -- start 2024-09-21T03:37:57.773 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:57.769+0000 7f39d2318640 1 -- start start 2024-09-21T03:37:57.773 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:57.769+0000 7f39d2318640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f39cc104860 0x7f39cc19a530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:37:57.773 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:57.769+0000 7f39d2318640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f39cc105290 0x7f39cc19aad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:37:57.773 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:57.769+0000 7f39d2318640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f39cc10de10 0x7f39cc19fd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:37:57.773 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:57.769+0000 7f39d2318640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f39cc113a60 con 0x7f39cc105290 2024-09-21T03:37:57.773 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:57.769+0000 7f39d2318640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f39cc1138e0 con 0x7f39cc10de10 2024-09-21T03:37:57.773 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:57.769+0000 7f39d2318640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f39cc113be0 con 0x7f39cc104860 2024-09-21T03:37:57.773 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:57.769+0000 7f39cb7fe640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f39cc105290 0x7f39cc19aad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:37:57.773 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:57.769+0000 7f39cb7fe640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f39cc105290 0x7f39cc19aad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:43196/0 (socket says 172.21.15.39:43196) 2024-09-21T03:37:57.773 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:57.769+0000 7f39cb7fe640 1 -- 172.21.15.39:0/2417711972 learned_addr learned my addr 172.21.15.39:0/2417711972 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:37:57.773 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:57.770+0000 7f39d08b5640 1 --2- 172.21.15.39:0/2417711972 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f39cc10de10 0x7f39cc19fd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:37:57.773 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:57.770+0000 7f39cbfff640 1 --2- 172.21.15.39:0/2417711972 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f39cc104860 0x7f39cc19a530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:37:57.774 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:57.770+0000 7f39cb7fe640 1 -- 172.21.15.39:0/2417711972 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f39cc104860 msgr2=0x7f39cc19a530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:37:57.774 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:57.770+0000 7f39cb7fe640 1 --2- 172.21.15.39:0/2417711972 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f39cc104860 0x7f39cc19a530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:57.774 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:57.770+0000 7f39cb7fe640 1 -- 172.21.15.39:0/2417711972 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f39cc10de10 msgr2=0x7f39cc19fd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:37:57.774 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:57.770+0000 7f39cb7fe640 1 --2- 172.21.15.39:0/2417711972 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f39cc10de10 0x7f39cc19fd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:57.774 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:57.770+0000 7f39cb7fe640 1 -- 172.21.15.39:0/2417711972 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f39cc1a03e0 con 0x7f39cc105290 2024-09-21T03:37:57.774 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:57.770+0000 7f39cb7fe640 1 --2- 172.21.15.39:0/2417711972 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f39cc105290 0x7f39cc19aad0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f39c002a930 tx=0x7f39c0004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:37:57.774 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:57.770+0000 7f39c97fa640 1 -- 172.21.15.39:0/2417711972 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39c003f030 con 0x7f39cc105290 2024-09-21T03:37:57.774 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:57.770+0000 7f39d2318640 1 -- 172.21.15.39:0/2417711972 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f39cc1a0670 con 0x7f39cc105290 2024-09-21T03:37:57.774 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:57.770+0000 7f39c97fa640 1 -- 172.21.15.39:0/2417711972 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f39c0003aa0 con 0x7f39cc105290 2024-09-21T03:37:57.774 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:57.770+0000 7f39c97fa640 1 -- 172.21.15.39:0/2417711972 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39c0035070 con 0x7f39cc105290 2024-09-21T03:37:57.774 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:57.770+0000 7f39d2318640 1 -- 172.21.15.39:0/2417711972 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f39cc1a0ba0 con 0x7f39cc105290 2024-09-21T03:37:57.775 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:57.771+0000 7f39d2318640 1 -- 172.21.15.39:0/2417711972 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f39cc104c70 con 0x7f39cc105290 2024-09-21T03:37:57.775 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:57.774+0000 7f39c97fa640 1 -- 172.21.15.39:0/2417711972 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 20) ==== 154473+0+0 (secure 0 0 0) 0x7f39c0042030 con 0x7f39cc105290 2024-09-21T03:37:57.776 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:57.775+0000 7f39c97fa640 1 --2- 172.21.15.39:0/2417711972 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f39b40b6500 0x7f39b40b8a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:37:57.776 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:57.775+0000 7f39cbfff640 1 --2- 172.21.15.39:0/2417711972 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f39b40b6500 0x7f39b40b8a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:37:57.776 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:57.775+0000 7f39c97fa640 1 -- 172.21.15.39:0/2417711972 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1370+0+0 (secure 0 0 0) 0x7f39c0106180 con 0x7f39cc105290 2024-09-21T03:37:57.777 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:57.775+0000 7f39cbfff640 1 --2- 172.21.15.39:0/2417711972 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f39b40b6500 0x7f39b40b8a10 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f39bc00bbe0 tx=0x7f39bc00b390 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:37:57.780 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:57.778+0000 7f39c97fa640 1 -- 172.21.15.39:0/2417711972 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f39cc1a0ba0 con 0x7f39cc105290 2024-09-21T03:37:57.963 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:54.828+0000 7ff342dee640 1 -- 172.21.15.39:0/1811077798 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff33c1083d0 msgr2=0x7ff33c1087e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:37:57.964 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:54.828+0000 7ff342dee640 1 --2- 172.21.15.39:0/1811077798 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff33c1083d0 0x7ff33c1087e0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7ff330009900 tx=0x7ff33002a410 comp rx=0 tx=0).stop 2024-09-21T03:37:57.964 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:54.828+0000 7ff342dee640 1 -- 172.21.15.39:0/1811077798 shutdown_connections 2024-09-21T03:37:57.964 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:54.828+0000 7ff342dee640 1 --2- 172.21.15.39:0/1811077798 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff33c108d20 0x7ff33c10f550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:57.964 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:54.828+0000 7ff342dee640 1 --2- 172.21.15.39:0/1811077798 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff33c1083d0 0x7ff33c1087e0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:57.964 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:54.828+0000 7ff342dee640 1 --2- 172.21.15.39:0/1811077798 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff33c102ae0 0x7ff33c102ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:57.964 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:54.828+0000 7ff342dee640 1 -- 172.21.15.39:0/1811077798 >> 172.21.15.39:0/1811077798 conn(0x7ff33c079760 msgr2=0x7ff33c079b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:37:57.964 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:54.828+0000 7ff342dee640 1 -- 172.21.15.39:0/1811077798 shutdown_connections 2024-09-21T03:37:57.964 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:54.828+0000 7ff342dee640 1 -- 172.21.15.39:0/1811077798 wait complete. 2024-09-21T03:37:57.964 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:54.829+0000 7ff342dee640 1 Processor -- start 2024-09-21T03:37:57.964 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:54.829+0000 7ff342dee640 1 -- start start 2024-09-21T03:37:57.964 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:54.829+0000 7ff342dee640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff33c102ae0 0x7ff33c1993a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:37:57.964 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:54.829+0000 7ff342dee640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff33c1083d0 0x7ff33c199940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:37:57.964 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:54.829+0000 7ff342dee640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff33c108d20 0x7ff33c19ebe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:37:57.964 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:54.829+0000 7ff342dee640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7ff33c112990 con 0x7ff33c102ae0 2024-09-21T03:37:57.965 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:54.829+0000 7ff342dee640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7ff33c112810 con 0x7ff33c1083d0 2024-09-21T03:37:57.965 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:54.829+0000 7ff342dee640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7ff33c112b10 con 0x7ff33c108d20 2024-09-21T03:37:57.965 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:54.829+0000 7ff340b8a640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff33c102ae0 0x7ff33c1993a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:37:57.965 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:54.829+0000 7ff340b8a640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff33c102ae0 0x7ff33c1993a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:34834/0 (socket says 172.21.15.39:34834) 2024-09-21T03:37:57.965 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:54.829+0000 7ff340b8a640 1 -- 172.21.15.39:0/3268487474 learned_addr learned my addr 172.21.15.39:0/3268487474 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:37:57.965 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:54.829+0000 7ff33bfff640 1 --2- 172.21.15.39:0/3268487474 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff33c1083d0 0x7ff33c199940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:37:57.965 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:54.829+0000 7ff34138b640 1 --2- 172.21.15.39:0/3268487474 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff33c108d20 0x7ff33c19ebe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:37:57.965 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:54.830+0000 7ff340b8a640 1 -- 172.21.15.39:0/3268487474 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff33c108d20 msgr2=0x7ff33c19ebe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:37:57.965 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:54.830+0000 7ff340b8a640 1 --2- 172.21.15.39:0/3268487474 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff33c108d20 0x7ff33c19ebe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:57.965 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:54.830+0000 7ff340b8a640 1 -- 172.21.15.39:0/3268487474 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff33c1083d0 msgr2=0x7ff33c199940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:37:57.965 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:54.830+0000 7ff340b8a640 1 --2- 172.21.15.39:0/3268487474 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff33c1083d0 0x7ff33c199940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:57.965 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:54.830+0000 7ff340b8a640 1 -- 172.21.15.39:0/3268487474 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff33c19f310 con 0x7ff33c102ae0 2024-09-21T03:37:57.965 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:54.830+0000 7ff34138b640 1 --2- 172.21.15.39:0/3268487474 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff33c108d20 0x7ff33c19ebe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:37:57.965 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:54.830+0000 7ff340b8a640 1 --2- 172.21.15.39:0/3268487474 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff33c102ae0 0x7ff33c1993a0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7ff32c00d950 tx=0x7ff32c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:37:57.966 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:54.830+0000 7ff339ffb640 1 -- 172.21.15.39:0/3268487474 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff32c017070 con 0x7ff33c102ae0 2024-09-21T03:37:57.966 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:54.830+0000 7ff339ffb640 1 -- 172.21.15.39:0/3268487474 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff32c009c30 con 0x7ff33c102ae0 2024-09-21T03:37:57.966 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:54.830+0000 7ff339ffb640 1 -- 172.21.15.39:0/3268487474 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff32c00f620 con 0x7ff33c102ae0 2024-09-21T03:37:57.966 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:54.830+0000 7ff342dee640 1 -- 172.21.15.39:0/3268487474 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff33c19f600 con 0x7ff33c102ae0 2024-09-21T03:37:57.966 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:54.830+0000 7ff342dee640 1 -- 172.21.15.39:0/3268487474 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff33c1a5d60 con 0x7ff33c102ae0 2024-09-21T03:37:57.966 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:54.831+0000 7ff342dee640 1 -- 172.21.15.39:0/3268487474 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff33c103d10 con 0x7ff33c102ae0 2024-09-21T03:37:57.966 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:54.833+0000 7ff339ffb640 1 -- 172.21.15.39:0/3268487474 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 20) ==== 154473+0+0 (secure 0 0 0) 0x7ff32c010460 con 0x7ff33c102ae0 2024-09-21T03:37:57.966 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:54.833+0000 7ff339ffb640 1 --2- 172.21.15.39:0/3268487474 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7ff3240b6550 0x7ff3240b8a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:37:57.966 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:54.834+0000 7ff33bfff640 1 --2- 172.21.15.39:0/3268487474 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7ff3240b6550 0x7ff3240b8a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:37:57.966 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:54.834+0000 7ff339ffb640 1 -- 172.21.15.39:0/3268487474 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1370+0+0 (secure 0 0 0) 0x7ff32c0e53c0 con 0x7ff33c102ae0 2024-09-21T03:37:57.966 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:54.834+0000 7ff33bfff640 1 --2- 172.21.15.39:0/3268487474 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7ff3240b6550 0x7ff3240b8a60 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7ff330000c00 tx=0x7ff330031400 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:37:57.966 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:54.838+0000 7ff339ffb640 1 -- 172.21.15.39:0/3268487474 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7ff32c0ad850 con 0x7ff33c102ae0 2024-09-21T03:37:57.966 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:55.027+0000 7ff342dee640 1 -- 172.21.15.39:0/3268487474 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/dump_on_update}] v 0) -- 0x7ff33c102ef0 con 0x7ff33c102ae0 2024-09-21T03:37:57.966 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:55.035+0000 7ff339ffb640 1 -- 172.21.15.39:0/3268487474 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/dump_on_update}]=0 v35)=0 v35) ==== 133+0+0 (secure 0 0 0) 0x7ff32c009de0 con 0x7ff33c102ae0 2024-09-21T03:37:57.967 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:55.037+0000 7ff342dee640 1 -- 172.21.15.39:0/3268487474 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7ff3240b6550 msgr2=0x7ff3240b8a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:37:57.967 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:55.037+0000 7ff342dee640 1 --2- 172.21.15.39:0/3268487474 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7ff3240b6550 0x7ff3240b8a60 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7ff330000c00 tx=0x7ff330031400 comp rx=0 tx=0).stop 2024-09-21T03:37:57.967 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:55.037+0000 7ff342dee640 1 -- 172.21.15.39:0/3268487474 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff33c102ae0 msgr2=0x7ff33c1993a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:37:57.967 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:55.037+0000 7ff342dee640 1 --2- 172.21.15.39:0/3268487474 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff33c102ae0 0x7ff33c1993a0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7ff32c00d950 tx=0x7ff32c00de10 comp rx=0 tx=0).stop 2024-09-21T03:37:57.967 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:55.037+0000 7ff342dee640 1 -- 172.21.15.39:0/3268487474 shutdown_connections 2024-09-21T03:37:57.967 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:55.037+0000 7ff342dee640 1 --2- 172.21.15.39:0/3268487474 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7ff3240b6550 0x7ff3240b8a60 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:57.967 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:55.037+0000 7ff342dee640 1 --2- 172.21.15.39:0/3268487474 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff33c108d20 0x7ff33c19ebe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:57.967 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:55.037+0000 7ff342dee640 1 --2- 172.21.15.39:0/3268487474 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff33c1083d0 0x7ff33c199940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:57.967 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:55.037+0000 7ff342dee640 1 --2- 172.21.15.39:0/3268487474 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff33c102ae0 0x7ff33c1993a0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:57.967 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:55.037+0000 7ff342dee640 1 -- 172.21.15.39:0/3268487474 >> 172.21.15.39:0/3268487474 conn(0x7ff33c079760 msgr2=0x7ff33c104400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:37:57.967 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:55.037+0000 7ff342dee640 1 -- 172.21.15.39:0/3268487474 shutdown_connections 2024-09-21T03:37:57.967 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:55.037+0000 7ff342dee640 1 -- 172.21.15.39:0/3268487474 wait complete. 2024-09-21T03:37:57.983 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:57.981+0000 7f39d2318640 1 -- 172.21.15.39:0/2417711972 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command([{prefix=config set, name=osd_op_complaint_time}] v 0) -- 0x7f39cc19b660 con 0x7f39cc105290 2024-09-21T03:37:57.991 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:57.990+0000 7f39c97fa640 1 -- 172.21.15.39:0/2417711972 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{prefix=config set, name=osd_op_complaint_time}]=0 v36)=0 v36) ==== 121+0+0 (secure 0 0 0) 0x7f39c00d29f0 con 0x7f39cc105290 2024-09-21T03:37:57.995 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:57.994+0000 7f39d2318640 1 -- 172.21.15.39:0/2417711972 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f39b40b6500 msgr2=0x7f39b40b8a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:37:57.996 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:57.994+0000 7f39d2318640 1 --2- 172.21.15.39:0/2417711972 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f39b40b6500 0x7f39b40b8a10 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f39bc00bbe0 tx=0x7f39bc00b390 comp rx=0 tx=0).stop 2024-09-21T03:37:57.996 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:57.994+0000 7f39d2318640 1 -- 172.21.15.39:0/2417711972 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f39cc105290 msgr2=0x7f39cc19aad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:37:57.996 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:57.994+0000 7f39d2318640 1 --2- 172.21.15.39:0/2417711972 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f39cc105290 0x7f39cc19aad0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f39c002a930 tx=0x7f39c0004270 comp rx=0 tx=0).stop 2024-09-21T03:37:57.996 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:57.994+0000 7f39d2318640 1 -- 172.21.15.39:0/2417711972 shutdown_connections 2024-09-21T03:37:57.996 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:57.994+0000 7f39d2318640 1 --2- 172.21.15.39:0/2417711972 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f39b40b6500 0x7f39b40b8a10 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:57.996 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:57.994+0000 7f39d2318640 1 --2- 172.21.15.39:0/2417711972 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f39cc10de10 0x7f39cc19fd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:57.996 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:57.994+0000 7f39d2318640 1 --2- 172.21.15.39:0/2417711972 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f39cc105290 0x7f39cc19aad0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:57.996 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:57.994+0000 7f39d2318640 1 --2- 172.21.15.39:0/2417711972 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f39cc104860 0x7f39cc19a530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:57.996 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:57.994+0000 7f39d2318640 1 -- 172.21.15.39:0/2417711972 >> 172.21.15.39:0/2417711972 conn(0x7f39cc079730 msgr2=0x7f39cc100950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:37:57.996 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:57.994+0000 7f39d2318640 1 -- 172.21.15.39:0/2417711972 shutdown_connections 2024-09-21T03:37:57.996 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:57.994+0000 7f39d2318640 1 -- 172.21.15.39:0/2417711972 wait complete. 2024-09-21T03:37:59.084 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:37:58 smithi039 ceph-mon[31211]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:37:59.084 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:37:58 smithi039 ceph-mon[31211]: from='client.? 172.21.15.39:0/2417711972' entity='client.admin' 2024-09-21T03:37:59.084 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:37:58 smithi039 ceph-mon[31211]: from='client.? 172.21.15.39:0/2057827469' entity='client.admin' 2024-09-21T03:37:59.468 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:37:58 smithi161 ceph-mon[34533]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:37:59.468 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:37:58 smithi161 ceph-mon[34533]: from='client.? 172.21.15.39:0/2417711972' entity='client.admin' 2024-09-21T03:37:59.468 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:37:58 smithi161 ceph-mon[34533]: from='client.? 172.21.15.39:0/2057827469' entity='client.admin' 2024-09-21T03:37:59.474 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:37:58 smithi150 ceph-mon[34289]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:37:59.474 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:37:58 smithi150 ceph-mon[34289]: from='client.? 172.21.15.39:0/2417711972' entity='client.admin' 2024-09-21T03:37:59.474 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:37:58 smithi150 ceph-mon[34289]: from='client.? 172.21.15.39:0/2057827469' entity='client.admin' 2024-09-21T03:38:00.492 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:57.957+0000 7f652d38d640 1 -- 172.21.15.39:0/3711553337 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6528102ae0 msgr2=0x7f6528102ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:38:00.492 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:57.957+0000 7f652d38d640 1 --2- 172.21.15.39:0/3711553337 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6528102ae0 0x7f6528102ef0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f6518009900 tx=0x7f651802a410 comp rx=0 tx=0).stop 2024-09-21T03:38:00.492 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:57.957+0000 7f652d38d640 1 -- 172.21.15.39:0/3711553337 shutdown_connections 2024-09-21T03:38:00.492 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:57.957+0000 7f652d38d640 1 --2- 172.21.15.39:0/3711553337 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6528108d20 0x7f652810f550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:00.492 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:57.957+0000 7f652d38d640 1 --2- 172.21.15.39:0/3711553337 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f65281083d0 0x7f65281087e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:00.493 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:57.957+0000 7f652d38d640 1 --2- 172.21.15.39:0/3711553337 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6528102ae0 0x7f6528102ef0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:00.493 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:57.957+0000 7f652d38d640 1 -- 172.21.15.39:0/3711553337 >> 172.21.15.39:0/3711553337 conn(0x7f65280797b0 msgr2=0x7f6528079bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:38:00.493 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:57.957+0000 7f652d38d640 1 -- 172.21.15.39:0/3711553337 shutdown_connections 2024-09-21T03:38:00.493 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:57.957+0000 7f652d38d640 1 -- 172.21.15.39:0/3711553337 wait complete. 2024-09-21T03:38:00.493 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:57.958+0000 7f652d38d640 1 Processor -- start 2024-09-21T03:38:00.493 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:57.958+0000 7f652d38d640 1 -- start start 2024-09-21T03:38:00.493 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:57.958+0000 7f652d38d640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6528102ae0 0x7f652819a490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:38:00.493 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:57.959+0000 7f652d38d640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f65281083d0 0x7f652819aa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:38:00.493 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:57.959+0000 7f652d38d640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6528108d20 0x7f652819fcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:38:00.493 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:57.959+0000 7f652d38d640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f6528113b00 con 0x7f6528102ae0 2024-09-21T03:38:00.493 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:57.959+0000 7f652d38d640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f6528113980 con 0x7f6528108d20 2024-09-21T03:38:00.493 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:57.959+0000 7f652d38d640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f6528113c80 con 0x7f65281083d0 2024-09-21T03:38:00.493 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:57.959+0000 7f6526ffd640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6528102ae0 0x7f652819a490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:38:00.494 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:57.959+0000 7f6526ffd640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6528102ae0 0x7f652819a490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:43214/0 (socket says 172.21.15.39:43214) 2024-09-21T03:38:00.494 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:57.959+0000 7f6526ffd640 1 -- 172.21.15.39:0/2057827469 learned_addr learned my addr 172.21.15.39:0/2057827469 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:38:00.494 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:57.959+0000 7f65277fe640 1 --2- 172.21.15.39:0/2057827469 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6528108d20 0x7f652819fcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:38:00.494 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:57.959+0000 7f65267fc640 1 --2- 172.21.15.39:0/2057827469 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f65281083d0 0x7f652819aa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:38:00.494 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:57.959+0000 7f6526ffd640 1 -- 172.21.15.39:0/2057827469 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f65281083d0 msgr2=0x7f652819aa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:38:00.494 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:57.959+0000 7f6526ffd640 1 --2- 172.21.15.39:0/2057827469 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f65281083d0 0x7f652819aa30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:00.494 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:57.959+0000 7f6526ffd640 1 -- 172.21.15.39:0/2057827469 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6528108d20 msgr2=0x7f652819fcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:38:00.494 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:57.959+0000 7f6526ffd640 1 --2- 172.21.15.39:0/2057827469 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6528108d20 0x7f652819fcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:00.494 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:57.959+0000 7f6526ffd640 1 -- 172.21.15.39:0/2057827469 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f65281a0400 con 0x7f6528102ae0 2024-09-21T03:38:00.494 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:57.959+0000 7f65267fc640 1 --2- 172.21.15.39:0/2057827469 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f65281083d0 0x7f652819aa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:38:00.494 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:57.959+0000 7f6526ffd640 1 --2- 172.21.15.39:0/2057827469 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6528102ae0 0x7f652819a490 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f6518005940 tx=0x7f65180042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:38:00.494 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:57.959+0000 7f650ffff640 1 -- 172.21.15.39:0/2057827469 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f651803f030 con 0x7f6528102ae0 2024-09-21T03:38:00.494 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:57.960+0000 7f650ffff640 1 -- 172.21.15.39:0/2057827469 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6518003cb0 con 0x7f6528102ae0 2024-09-21T03:38:00.495 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:57.960+0000 7f652d38d640 1 -- 172.21.15.39:0/2057827469 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f65281a0690 con 0x7f6528102ae0 2024-09-21T03:38:00.495 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:57.960+0000 7f650ffff640 1 -- 172.21.15.39:0/2057827469 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6518035070 con 0x7f6528102ae0 2024-09-21T03:38:00.495 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:57.960+0000 7f652d38d640 1 -- 172.21.15.39:0/2057827469 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f65281a6e50 con 0x7f6528102ae0 2024-09-21T03:38:00.495 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:57.961+0000 7f652d38d640 1 -- 172.21.15.39:0/2057827469 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6528102ef0 con 0x7f6528102ae0 2024-09-21T03:38:00.495 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:57.962+0000 7f650ffff640 1 -- 172.21.15.39:0/2057827469 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 20) ==== 154473+0+0 (secure 0 0 0) 0x7f6518042030 con 0x7f6528102ae0 2024-09-21T03:38:00.495 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:57.963+0000 7f650ffff640 1 --2- 172.21.15.39:0/2057827469 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f65100b64d0 0x7f65100b89e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:38:00.495 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:57.963+0000 7f65267fc640 1 --2- 172.21.15.39:0/2057827469 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f65100b64d0 0x7f65100b89e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:38:00.495 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:57.963+0000 7f650ffff640 1 -- 172.21.15.39:0/2057827469 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1370+0+0 (secure 0 0 0) 0x7f6518106020 con 0x7f6528102ae0 2024-09-21T03:38:00.495 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:57.964+0000 7f65267fc640 1 --2- 172.21.15.39:0/2057827469 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f65100b64d0 0x7f65100b89e0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f651c0058c0 tx=0x7f651c005850 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:38:00.495 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:57.972+0000 7f650ffff640 1 -- 172.21.15.39:0/2057827469 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f65180ce4b0 con 0x7f6528102ae0 2024-09-21T03:38:00.495 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:58.176+0000 7f652d38d640 1 -- 172.21.15.39:0/2057827469 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command([{prefix=config set, name=debug_ms}] v 0) -- 0x7f652810aaf0 con 0x7f6528102ae0 2024-09-21T03:38:00.495 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:58.184+0000 7f650ffff640 1 -- 172.21.15.39:0/2057827469 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{prefix=config set, name=debug_ms}]=0 v37)=0 v37) ==== 106+0+0 (secure 0 0 0) 0x7f65180d2890 con 0x7f6528102ae0 2024-09-21T03:38:00.495 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:58.187+0000 7f652d38d640 1 -- 172.21.15.39:0/2057827469 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f65100b64d0 msgr2=0x7f65100b89e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:38:00.496 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:58.187+0000 7f652d38d640 1 --2- 172.21.15.39:0/2057827469 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f65100b64d0 0x7f65100b89e0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f651c0058c0 tx=0x7f651c005850 comp rx=0 tx=0).stop 2024-09-21T03:38:00.496 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:58.187+0000 7f652d38d640 1 -- 172.21.15.39:0/2057827469 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6528102ae0 msgr2=0x7f652819a490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:38:00.496 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:58.187+0000 7f652d38d640 1 --2- 172.21.15.39:0/2057827469 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6528102ae0 0x7f652819a490 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f6518005940 tx=0x7f65180042f0 comp rx=0 tx=0).stop 2024-09-21T03:38:00.496 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:58.187+0000 7f652d38d640 1 -- 172.21.15.39:0/2057827469 shutdown_connections 2024-09-21T03:38:00.496 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:58.187+0000 7f652d38d640 1 --2- 172.21.15.39:0/2057827469 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f65100b64d0 0x7f65100b89e0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:00.496 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:58.187+0000 7f652d38d640 1 --2- 172.21.15.39:0/2057827469 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6528108d20 0x7f652819fcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:00.496 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:58.187+0000 7f652d38d640 1 --2- 172.21.15.39:0/2057827469 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f65281083d0 0x7f652819aa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:00.496 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:58.187+0000 7f652d38d640 1 --2- 172.21.15.39:0/2057827469 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6528102ae0 0x7f652819a490 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:00.496 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:58.187+0000 7f652d38d640 1 -- 172.21.15.39:0/2057827469 >> 172.21.15.39:0/2057827469 conn(0x7f65280797b0 msgr2=0x7f6528106480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:38:00.496 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:58.187+0000 7f652d38d640 1 -- 172.21.15.39:0/2057827469 shutdown_connections 2024-09-21T03:38:00.496 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:37:58.187+0000 7f652d38d640 1 -- 172.21.15.39:0/2057827469 wait complete. 2024-09-21T03:38:00.987 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '51116d5c-77ca-11ef-baf5-efdc52797490', '--', 'ceph', 'config', 'set', 'mds', 'mds_bal_fragment_size_max', '10000'], check_status=True, hostname='smithi039') 2024-09-21T03:38:00.987 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '51116d5c-77ca-11ef-baf5-efdc52797490', '--', 'ceph', 'config', 'set', 'mds', 'mds_bal_merge_size', '5'], check_status=True, hostname='smithi039') 2024-09-21T03:38:00.987 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '51116d5c-77ca-11ef-baf5-efdc52797490', '--', 'ceph', 'config', 'set', 'mds', 'mds_bal_split_bits', '3'], check_status=True, hostname='smithi039') 2024-09-21T03:38:00.987 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '51116d5c-77ca-11ef-baf5-efdc52797490', '--', 'ceph', 'config', 'set', 'mds', 'mds_bal_split_size', '100'], check_status=True, hostname='smithi039') 2024-09-21T03:38:00.987 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '51116d5c-77ca-11ef-baf5-efdc52797490', '--', 'ceph', 'config', 'set', 'mds', 'mds_debug_frag', 'True'], check_status=True, hostname='smithi039') 2024-09-21T03:38:00.987 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '51116d5c-77ca-11ef-baf5-efdc52797490', '--', 'ceph', 'config', 'set', 'mds', 'mds_debug_scatterstat', 'True'], check_status=True, hostname='smithi039') 2024-09-21T03:38:00.987 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '51116d5c-77ca-11ef-baf5-efdc52797490', '--', 'ceph', 'config', 'set', 'mds', 'mds_op_complaint_time', '180'], check_status=True, hostname='smithi039') 2024-09-21T03:38:00.987 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '51116d5c-77ca-11ef-baf5-efdc52797490', '--', 'ceph', 'config', 'set', 'mds', 'mds_verify_scatter', 'True'], check_status=True, hostname='smithi039') 2024-09-21T03:38:00.988 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '51116d5c-77ca-11ef-baf5-efdc52797490', '--', 'ceph', 'config', 'set', 'mds', 'osd_op_complaint_time', '180'], check_status=True, hostname='smithi039') 2024-09-21T03:38:00.988 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '51116d5c-77ca-11ef-baf5-efdc52797490', '--', 'ceph', 'config', 'set', 'mds', 'rados_mon_op_timeout', '900'], check_status=True, hostname='smithi039') 2024-09-21T03:38:00.988 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '51116d5c-77ca-11ef-baf5-efdc52797490', '--', 'ceph', 'config', 'set', 'mds', 'rados_osd_op_timeout', '900'], check_status=True, hostname='smithi039') 2024-09-21T03:38:01.083 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:38:00 smithi039 ceph-mon[31211]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:38:01.467 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:38:00 smithi161 ceph-mon[34533]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:38:01.474 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:38:00 smithi150 ceph-mon[34289]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:38:01.833 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '51116d5c-77ca-11ef-baf5-efdc52797490', '--', 'ceph', 'config', 'set', 'mgr', 'client_mount_timeout', '30'], check_status=True, hostname='smithi039') 2024-09-21T03:38:01.833 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '51116d5c-77ca-11ef-baf5-efdc52797490', '--', 'ceph', 'config', 'set', 'mgr', 'debug_cephsqlite', '20'], check_status=True, hostname='smithi039') 2024-09-21T03:38:01.833 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '51116d5c-77ca-11ef-baf5-efdc52797490', '--', 'ceph', 'config', 'set', 'mgr', 'debug_client', '20'], check_status=True, hostname='smithi039') 2024-09-21T03:38:01.833 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '51116d5c-77ca-11ef-baf5-efdc52797490', '--', 'ceph', 'config', 'set', 'mgr', 'debug_mgr', '20'], check_status=True, hostname='smithi039') 2024-09-21T03:38:01.833 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '51116d5c-77ca-11ef-baf5-efdc52797490', '--', 'ceph', 'config', 'set', 'mgr', 'debug_ms', '1'], check_status=True, hostname='smithi039') 2024-09-21T03:38:01.833 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '51116d5c-77ca-11ef-baf5-efdc52797490', '--', 'ceph', 'config', 'set', 'mgr', 'mgr/snap_schedule/allow_m_granularity', 'True'], check_status=True, hostname='smithi039') 2024-09-21T03:38:01.833 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '51116d5c-77ca-11ef-baf5-efdc52797490', '--', 'ceph', 'config', 'set', 'mgr', 'mgr/snap_schedule/dump_on_update', 'True'], check_status=True, hostname='smithi039') 2024-09-21T03:38:01.833 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '51116d5c-77ca-11ef-baf5-efdc52797490', '--', 'ceph', 'config', 'set', 'mgr', 'mgr/snap_schedule/log_level', 'debug'], check_status=True, hostname='smithi039') 2024-09-21T03:38:01.834 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '51116d5c-77ca-11ef-baf5-efdc52797490', '--', 'ceph', 'config', 'set', 'mgr', 'mon_warn_on_pool_no_app', 'False'], check_status=True, hostname='smithi039') 2024-09-21T03:38:01.834 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '51116d5c-77ca-11ef-baf5-efdc52797490', '--', 'ceph', 'config', 'set', 'mon', 'mon_op_complaint_time', '120'], check_status=True, hostname='smithi039') 2024-09-21T03:38:01.834 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '51116d5c-77ca-11ef-baf5-efdc52797490', '--', 'ceph', 'config', 'set', 'osd', 'osd_op_complaint_time', '180'], check_status=True, hostname='smithi039') 2024-09-21T03:38:01.834 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 51116d5c-77ca-11ef-baf5-efdc52797490 -- ceph config dump 2024-09-21T03:38:02.092 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/51116d5c-77ca-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:38:02.577 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:38:02.563+0000 7f3b2055d640 1 -- 172.21.15.39:0/2832424139 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3b18102ae0 msgr2=0x7f3b18102ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:38:02.577 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:38:02.563+0000 7f3b2055d640 1 --2- 172.21.15.39:0/2832424139 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3b18102ae0 0x7f3b18102ef0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f3b14009900 tx=0x7f3b1402a410 comp rx=0 tx=0).stop 2024-09-21T03:38:02.577 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:38:02.563+0000 7f3b2055d640 1 -- 172.21.15.39:0/2832424139 shutdown_connections 2024-09-21T03:38:02.578 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:38:02.563+0000 7f3b2055d640 1 --2- 172.21.15.39:0/2832424139 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3b18108d20 0x7f3b1810f550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:02.578 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:38:02.563+0000 7f3b2055d640 1 --2- 172.21.15.39:0/2832424139 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3b181083d0 0x7f3b181087e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:02.578 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:38:02.563+0000 7f3b2055d640 1 --2- 172.21.15.39:0/2832424139 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3b18102ae0 0x7f3b18102ef0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:02.578 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:38:02.563+0000 7f3b2055d640 1 -- 172.21.15.39:0/2832424139 >> 172.21.15.39:0/2832424139 conn(0x7f3b18079760 msgr2=0x7f3b18079b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:38:02.578 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:38:02.563+0000 7f3b2055d640 1 -- 172.21.15.39:0/2832424139 shutdown_connections 2024-09-21T03:38:02.578 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:38:02.563+0000 7f3b2055d640 1 -- 172.21.15.39:0/2832424139 wait complete. 2024-09-21T03:38:02.578 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:38:02.564+0000 7f3b2055d640 1 Processor -- start 2024-09-21T03:38:02.578 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:38:02.564+0000 7f3b2055d640 1 -- start start 2024-09-21T03:38:02.578 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:38:02.564+0000 7f3b2055d640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3b18102ae0 0x7f3b181993d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:38:02.578 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:38:02.564+0000 7f3b2055d640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3b181083d0 0x7f3b18199970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:38:02.578 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:38:02.564+0000 7f3b2055d640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3b18108d20 0x7f3b1819ec10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:38:02.578 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:38:02.564+0000 7f3b2055d640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f3b181129e0 con 0x7f3b18108d20 2024-09-21T03:38:02.578 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:38:02.564+0000 7f3b2055d640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f3b18112860 con 0x7f3b181083d0 2024-09-21T03:38:02.578 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:38:02.564+0000 7f3b2055d640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f3b18112b60 con 0x7f3b18102ae0 2024-09-21T03:38:02.579 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:38:02.564+0000 7f3b1eafa640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3b18108d20 0x7f3b1819ec10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:38:02.579 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:38:02.564+0000 7f3b1eafa640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3b18108d20 0x7f3b1819ec10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:43238/0 (socket says 172.21.15.39:43238) 2024-09-21T03:38:02.579 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:38:02.564+0000 7f3b1eafa640 1 -- 172.21.15.39:0/3502275107 learned_addr learned my addr 172.21.15.39:0/3502275107 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:38:02.579 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:38:02.564+0000 7f3b1e2f9640 1 --2- 172.21.15.39:0/3502275107 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3b18102ae0 0x7f3b181993d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:38:02.579 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:38:02.564+0000 7f3b1daf8640 1 --2- 172.21.15.39:0/3502275107 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3b181083d0 0x7f3b18199970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:38:02.579 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:38:02.565+0000 7f3b1eafa640 1 -- 172.21.15.39:0/3502275107 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3b18102ae0 msgr2=0x7f3b181993d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:38:02.579 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:38:02.565+0000 7f3b1eafa640 1 --2- 172.21.15.39:0/3502275107 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3b18102ae0 0x7f3b181993d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:02.579 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:38:02.565+0000 7f3b1eafa640 1 -- 172.21.15.39:0/3502275107 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3b181083d0 msgr2=0x7f3b18199970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:38:02.579 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:38:02.565+0000 7f3b1eafa640 1 --2- 172.21.15.39:0/3502275107 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3b181083d0 0x7f3b18199970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:02.579 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:38:02.565+0000 7f3b1eafa640 1 -- 172.21.15.39:0/3502275107 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3b1819f340 con 0x7f3b18108d20 2024-09-21T03:38:02.579 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:38:02.565+0000 7f3b1eafa640 1 --2- 172.21.15.39:0/3502275107 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3b18108d20 0x7f3b1819ec10 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f3b1000eb10 tx=0x7f3b1000c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:38:02.579 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:38:02.565+0000 7f3b0b7fe640 1 -- 172.21.15.39:0/3502275107 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b10004b20 con 0x7f3b18108d20 2024-09-21T03:38:02.579 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:38:02.565+0000 7f3b0b7fe640 1 -- 172.21.15.39:0/3502275107 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3b10004cc0 con 0x7f3b18108d20 2024-09-21T03:38:02.579 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:38:02.565+0000 7f3b2055d640 1 -- 172.21.15.39:0/3502275107 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3b1819f630 con 0x7f3b18108d20 2024-09-21T03:38:02.579 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:38:02.565+0000 7f3b0b7fe640 1 -- 172.21.15.39:0/3502275107 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b10010670 con 0x7f3b18108d20 2024-09-21T03:38:02.580 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:38:02.565+0000 7f3b2055d640 1 -- 172.21.15.39:0/3502275107 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3b181a5d90 con 0x7f3b18108d20 2024-09-21T03:38:02.580 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:38:02.566+0000 7f3b2055d640 1 -- 172.21.15.39:0/3502275107 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3b18103d10 con 0x7f3b18108d20 2024-09-21T03:38:02.580 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:38:02.568+0000 7f3b0b7fe640 1 -- 172.21.15.39:0/3502275107 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 20) ==== 154473+0+0 (secure 0 0 0) 0x7f3b10005000 con 0x7f3b18108d20 2024-09-21T03:38:02.580 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:38:02.569+0000 7f3b0b7fe640 1 --2- 172.21.15.39:0/3502275107 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f3b040b6550 0x7f3b040b8a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:38:02.580 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:38:02.569+0000 7f3b1e2f9640 1 --2- 172.21.15.39:0/3502275107 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f3b040b6550 0x7f3b040b8a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:38:02.580 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:38:02.569+0000 7f3b0b7fe640 1 -- 172.21.15.39:0/3502275107 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1370+0+0 (secure 0 0 0) 0x7f3b100e5d90 con 0x7f3b18108d20 2024-09-21T03:38:02.580 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:38:02.569+0000 7f3b1e2f9640 1 --2- 172.21.15.39:0/3502275107 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f3b040b6550 0x7f3b040b8a60 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f3b14005ab0 tx=0x7f3b14005a20 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:38:02.580 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:38:02.572+0000 7f3b0b7fe640 1 -- 172.21.15.39:0/3502275107 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f3b10056710 con 0x7f3b18108d20 2024-09-21T03:38:02.764 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:38:02.762+0000 7f3b2055d640 1 -- 172.21.15.39:0/3502275107 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "config dump"} v 0) -- 0x7f3b18102ef0 con 0x7f3b18108d20 2024-09-21T03:38:02.764 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:38:02 smithi039 ceph-mon[31211]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:38:02.764 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:38:02 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:38:02.765 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:38:02 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:38:02.765 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:38:02 smithi039 ceph-mon[31211]: Reconfiguring mon.c (monmap changed)... 2024-09-21T03:38:02.765 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:38:02 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-09-21T03:38:02.765 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:38:02 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "config get", "who": "mon", "key": "public_network"} : dispatch 2024-09-21T03:38:02.765 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:38:02 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:38:02.765 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:38:02 smithi039 ceph-mon[31211]: Reconfiguring daemon mon.c on smithi161 2024-09-21T03:38:02.765 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:38:02.763+0000 7f3b0b7fe640 1 -- 172.21.15.39:0/3502275107 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "config dump"}]=0 v37) ==== 59+0+22533 (secure 0 0 0) 0x7f3b100ae220 con 0x7f3b18108d20 2024-09-21T03:38:02.766 INFO:teuthology.orchestra.run.smithi039.stdout:WHO MASK LEVEL OPTION VALUE RO 2024-09-21T03:38:02.766 INFO:teuthology.orchestra.run.smithi039.stdout:global dev auth_debug true 2024-09-21T03:38:02.766 INFO:teuthology.orchestra.run.smithi039.stdout:global basic container_image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262 * 2024-09-21T03:38:02.766 INFO:teuthology.orchestra.run.smithi039.stdout:global dev debug_asserts_on_shutdown true 2024-09-21T03:38:02.766 INFO:teuthology.orchestra.run.smithi039.stdout:global basic log_to_file true 2024-09-21T03:38:02.766 INFO:teuthology.orchestra.run.smithi039.stdout:global basic log_to_journald false 2024-09-21T03:38:02.766 INFO:teuthology.orchestra.run.smithi039.stdout:global basic log_to_stderr false 2024-09-21T03:38:02.767 INFO:teuthology.orchestra.run.smithi039.stdout:global advanced mon_allow_pool_delete true 2024-09-21T03:38:02.767 INFO:teuthology.orchestra.run.smithi039.stdout:global advanced mon_clock_drift_allowed 1.000000 2024-09-21T03:38:02.767 INFO:teuthology.orchestra.run.smithi039.stdout:global advanced mon_cluster_log_to_file true 2024-09-21T03:38:02.767 INFO:teuthology.orchestra.run.smithi039.stdout:global advanced mon_max_pg_per_osd 10000 2024-09-21T03:38:02.767 INFO:teuthology.orchestra.run.smithi039.stdout:global advanced mon_pg_warn_max_object_skew 0.000000 2024-09-21T03:38:02.767 INFO:teuthology.orchestra.run.smithi039.stdout:global advanced mon_warn_on_crush_straw_calc_version_zero false 2024-09-21T03:38:02.767 INFO:teuthology.orchestra.run.smithi039.stdout:global advanced mon_warn_on_legacy_crush_tunables false 2024-09-21T03:38:02.767 INFO:teuthology.orchestra.run.smithi039.stdout:global advanced mon_warn_on_osd_down_out_interval_zero false 2024-09-21T03:38:02.767 INFO:teuthology.orchestra.run.smithi039.stdout:global dev mon_warn_on_pool_pg_num_not_power_of_two false 2024-09-21T03:38:02.767 INFO:teuthology.orchestra.run.smithi039.stdout:global advanced mon_warn_on_too_few_osds false 2024-09-21T03:38:02.767 INFO:teuthology.orchestra.run.smithi039.stdout:global dev ms_die_on_bug true 2024-09-21T03:38:02.767 INFO:teuthology.orchestra.run.smithi039.stdout:global dev ms_die_on_old_message true 2024-09-21T03:38:02.767 INFO:teuthology.orchestra.run.smithi039.stdout:global dev ms_die_on_skipped_message false 2024-09-21T03:38:02.767 INFO:teuthology.orchestra.run.smithi039.stdout:global advanced osd_pool_default_erasure_code_profile plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-09-21T03:38:02.768 INFO:teuthology.orchestra.run.smithi039.stdout:global advanced osd_pool_default_pg_autoscale_mode off 2024-09-21T03:38:02.768 INFO:teuthology.orchestra.run.smithi039.stdout:mon advanced auth_allow_insecure_global_id_reclaim false 2024-09-21T03:38:02.768 INFO:teuthology.orchestra.run.smithi039.stdout:mon advanced auth_mon_ticket_ttl 660.000000 2024-09-21T03:38:02.768 INFO:teuthology.orchestra.run.smithi039.stdout:mon advanced auth_service_ticket_ttl 240.000000 2024-09-21T03:38:02.768 INFO:teuthology.orchestra.run.smithi039.stdout:mon advanced debug_mon 20/20 2024-09-21T03:38:02.768 INFO:teuthology.orchestra.run.smithi039.stdout:mon advanced debug_ms 1/1 2024-09-21T03:38:02.768 INFO:teuthology.orchestra.run.smithi039.stdout:mon advanced debug_paxos 20/20 2024-09-21T03:38:02.768 INFO:teuthology.orchestra.run.smithi039.stdout:mon advanced mon_data_avail_warn 5 2024-09-21T03:38:02.768 INFO:teuthology.orchestra.run.smithi039.stdout:mon advanced mon_down_mkfs_grace 300 2024-09-21T03:38:02.768 INFO:teuthology.orchestra.run.smithi039.stdout:mon advanced mon_mgr_mkfs_grace 240 2024-09-21T03:38:02.768 INFO:teuthology.orchestra.run.smithi039.stdout:mon advanced mon_op_complaint_time 120 2024-09-21T03:38:02.769 INFO:teuthology.orchestra.run.smithi039.stdout:mon dev mon_osd_prime_pg_temp true 2024-09-21T03:38:02.769 INFO:teuthology.orchestra.run.smithi039.stdout:mon advanced mon_osd_reporter_subtree_level osd 2024-09-21T03:38:02.769 INFO:teuthology.orchestra.run.smithi039.stdout:mon advanced mon_reweight_min_bytes_per_osd 10 2024-09-21T03:38:02.769 INFO:teuthology.orchestra.run.smithi039.stdout:mon advanced mon_reweight_min_pgs_per_osd 4 2024-09-21T03:38:02.769 INFO:teuthology.orchestra.run.smithi039.stdout:mon advanced mon_warn_on_insecure_global_id_reclaim false 2024-09-21T03:38:02.769 INFO:teuthology.orchestra.run.smithi039.stdout:mon advanced mon_warn_on_insecure_global_id_reclaim_allowed false 2024-09-21T03:38:02.769 INFO:teuthology.orchestra.run.smithi039.stdout:mon advanced public_network 172.21.0.0/20 * 2024-09-21T03:38:02.769 INFO:teuthology.orchestra.run.smithi039.stdout:mgr advanced cephsqlite_lock_renewal_timeout 900000 2024-09-21T03:38:02.769 INFO:teuthology.orchestra.run.smithi039.stdout:mgr advanced client_mount_timeout 30 2024-09-21T03:38:02.769 INFO:teuthology.orchestra.run.smithi039.stdout:mgr advanced debug_cephsqlite 20/20 2024-09-21T03:38:02.769 INFO:teuthology.orchestra.run.smithi039.stdout:mgr advanced debug_client 20/20 2024-09-21T03:38:02.769 INFO:teuthology.orchestra.run.smithi039.stdout:mgr advanced debug_mgr 20/20 2024-09-21T03:38:02.769 INFO:teuthology.orchestra.run.smithi039.stdout:mgr advanced debug_ms 1/1 2024-09-21T03:38:02.769 INFO:teuthology.orchestra.run.smithi039.stdout:mgr advanced mgr/cephadm/allow_ptrace true * 2024-09-21T03:38:02.769 INFO:teuthology.orchestra.run.smithi039.stdout:mgr advanced mgr/cephadm/container_init True * 2024-09-21T03:38:02.770 INFO:teuthology.orchestra.run.smithi039.stdout:mgr advanced mgr/cephadm/migration_current 7 * 2024-09-21T03:38:02.770 INFO:teuthology.orchestra.run.smithi039.stdout:mgr advanced mgr/dashboard/ssl_server_port 8443 * 2024-09-21T03:38:02.770 INFO:teuthology.orchestra.run.smithi039.stdout:mgr advanced mgr/orchestrator/orchestrator cephadm 2024-09-21T03:38:02.770 INFO:teuthology.orchestra.run.smithi039.stdout:mgr advanced mgr/snap_schedule/allow_m_granularity true 2024-09-21T03:38:02.770 INFO:teuthology.orchestra.run.smithi039.stdout:mgr advanced mgr/snap_schedule/dump_on_update true 2024-09-21T03:38:02.770 INFO:teuthology.orchestra.run.smithi039.stdout:mgr advanced mgr/snap_schedule/log_level debug 2024-09-21T03:38:02.770 INFO:teuthology.orchestra.run.smithi039.stdout:mgr advanced mon_reweight_min_bytes_per_osd 10 2024-09-21T03:38:02.770 INFO:teuthology.orchestra.run.smithi039.stdout:mgr advanced mon_reweight_min_pgs_per_osd 4 2024-09-21T03:38:02.770 INFO:teuthology.orchestra.run.smithi039.stdout:mgr dev mon_warn_on_pool_no_app false 2024-09-21T03:38:02.770 INFO:teuthology.orchestra.run.smithi039.stdout:osd dev bdev_debug_aio true 2024-09-21T03:38:02.770 INFO:teuthology.orchestra.run.smithi039.stdout:osd advanced bdev_enable_discard true 2024-09-21T03:38:02.770 INFO:teuthology.orchestra.run.smithi039.stdout:osd advanced bluestore_allocator bitmap * 2024-09-21T03:38:02.770 INFO:teuthology.orchestra.run.smithi039.stdout:osd dev bluestore_block_size 96636764160 * 2024-09-21T03:38:02.771 INFO:teuthology.orchestra.run.smithi039.stdout:osd dev bluestore_fsck_on_mount true 2024-09-21T03:38:02.771 INFO:teuthology.orchestra.run.smithi039.stdout:osd advanced debug_bluefs 1/20 2024-09-21T03:38:02.771 INFO:teuthology.orchestra.run.smithi039.stdout:osd advanced debug_bluestore 1/20 2024-09-21T03:38:02.771 INFO:teuthology.orchestra.run.smithi039.stdout:osd advanced debug_ms 1/1 2024-09-21T03:38:02.771 INFO:teuthology.orchestra.run.smithi039.stdout:osd advanced debug_osd 20/20 2024-09-21T03:38:02.771 INFO:teuthology.orchestra.run.smithi039.stdout:osd advanced debug_rocksdb 4/10 2024-09-21T03:38:02.771 INFO:teuthology.orchestra.run.smithi039.stdout:osd dev osd_debug_misdirected_ops true 2024-09-21T03:38:02.771 INFO:teuthology.orchestra.run.smithi039.stdout:osd dev osd_debug_op_order true 2024-09-21T03:38:02.771 INFO:teuthology.orchestra.run.smithi039.stdout:osd dev osd_debug_pg_log_writeout true 2024-09-21T03:38:02.771 INFO:teuthology.orchestra.run.smithi039.stdout:osd dev osd_debug_shutdown true 2024-09-21T03:38:02.771 INFO:teuthology.orchestra.run.smithi039.stdout:osd dev osd_debug_verify_cached_snaps true 2024-09-21T03:38:02.771 INFO:teuthology.orchestra.run.smithi039.stdout:osd dev osd_debug_verify_missing_on_start true 2024-09-21T03:38:02.771 INFO:teuthology.orchestra.run.smithi039.stdout:osd dev osd_debug_verify_stray_on_activate true 2024-09-21T03:38:02.771 INFO:teuthology.orchestra.run.smithi039.stdout:osd advanced osd_deep_scrub_update_digest_min_age 30 2024-09-21T03:38:02.771 INFO:teuthology.orchestra.run.smithi039.stdout:osd advanced osd_failsafe_full_ratio 0.950000 2024-09-21T03:38:02.772 INFO:teuthology.orchestra.run.smithi039.stdout:osd advanced osd_max_omap_entries_per_request 10 2024-09-21T03:38:02.772 INFO:teuthology.orchestra.run.smithi039.stdout:osd advanced osd_mclock_profile high_recovery_ops 2024-09-21T03:38:02.772 INFO:teuthology.orchestra.run.smithi039.stdout:osd advanced osd_memory_target_autotune true 2024-09-21T03:38:02.772 INFO:teuthology.orchestra.run.smithi039.stdout:osd advanced osd_objectstore bluestore * 2024-09-21T03:38:02.772 INFO:teuthology.orchestra.run.smithi039.stdout:osd advanced osd_op_complaint_time 180.000000 2024-09-21T03:38:02.772 INFO:teuthology.orchestra.run.smithi039.stdout:osd advanced osd_op_queue debug_random * 2024-09-21T03:38:02.772 INFO:teuthology.orchestra.run.smithi039.stdout:osd advanced osd_op_queue_cut_off debug_random * 2024-09-21T03:38:02.772 INFO:teuthology.orchestra.run.smithi039.stdout:osd advanced osd_recover_clone_overlap true 2024-09-21T03:38:02.772 INFO:teuthology.orchestra.run.smithi039.stdout:osd advanced osd_recovery_max_chunk 1048576 2024-09-21T03:38:02.772 INFO:teuthology.orchestra.run.smithi039.stdout:osd advanced osd_scrub_load_threshold 5.000000 2024-09-21T03:38:02.772 INFO:teuthology.orchestra.run.smithi039.stdout:osd advanced osd_scrub_max_interval 600.000000 2024-09-21T03:38:02.772 INFO:teuthology.orchestra.run.smithi039.stdout:osd advanced osd_shutdown_pgref_assert true 2024-09-21T03:38:02.772 INFO:teuthology.orchestra.run.smithi039.stdout:mds advanced debug_mds 20/20 2024-09-21T03:38:02.772 INFO:teuthology.orchestra.run.smithi039.stdout:mds advanced debug_mds_balancer 20/20 2024-09-21T03:38:02.772 INFO:teuthology.orchestra.run.smithi039.stdout:mds advanced debug_ms 2/2 2024-09-21T03:38:02.773 INFO:teuthology.orchestra.run.smithi039.stdout:mds advanced mds_bal_fragment_size_max 10000 2024-09-21T03:38:02.773 INFO:teuthology.orchestra.run.smithi039.stdout:mds advanced mds_bal_merge_size 5 2024-09-21T03:38:02.773 INFO:teuthology.orchestra.run.smithi039.stdout:mds advanced mds_bal_replicate_threshold 1.000000 2024-09-21T03:38:02.773 INFO:teuthology.orchestra.run.smithi039.stdout:mds advanced mds_bal_split_bits 3 2024-09-21T03:38:02.773 INFO:teuthology.orchestra.run.smithi039.stdout:mds advanced mds_bal_split_size 100 2024-09-21T03:38:02.773 INFO:teuthology.orchestra.run.smithi039.stdout:mds dev mds_debug_frag true 2024-09-21T03:38:02.773 INFO:teuthology.orchestra.run.smithi039.stdout:mds dev mds_debug_scatterstat true 2024-09-21T03:38:02.773 INFO:teuthology.orchestra.run.smithi039.stdout:mds advanced mds_export_ephemeral_random_max 0.100000 2024-09-21T03:38:02.773 INFO:teuthology.orchestra.run.smithi039.stdout:mds advanced mds_op_complaint_time 180.000000 2024-09-21T03:38:02.773 INFO:teuthology.orchestra.run.smithi039.stdout:mds dev mds_verify_scatter true 2024-09-21T03:38:02.773 INFO:teuthology.orchestra.run.smithi039.stdout:mds advanced osd_op_complaint_time 180.000000 2024-09-21T03:38:02.773 INFO:teuthology.orchestra.run.smithi039.stdout:mds advanced rados_mon_op_timeout 900 2024-09-21T03:38:02.773 INFO:teuthology.orchestra.run.smithi039.stdout:mds advanced rados_osd_op_timeout 900 2024-09-21T03:38:02.773 INFO:teuthology.orchestra.run.smithi039.stdout:client advanced client_mount_timeout 600 2024-09-21T03:38:02.774 INFO:teuthology.orchestra.run.smithi039.stdout:client advanced debug_client 20/20 2024-09-21T03:38:02.774 INFO:teuthology.orchestra.run.smithi039.stdout:client advanced debug_ms 1/1 2024-09-21T03:38:02.774 INFO:teuthology.orchestra.run.smithi039.stdout:client advanced rados_mon_op_timeout 900 2024-09-21T03:38:02.774 INFO:teuthology.orchestra.run.smithi039.stdout:client advanced rados_osd_op_timeout 900 2024-09-21T03:38:02.774 INFO:teuthology.orchestra.run.smithi039.stdout:client.rgw advanced rgw_cache_enabled true 2024-09-21T03:38:02.774 INFO:teuthology.orchestra.run.smithi039.stdout:client.rgw advanced rgw_enable_ops_log true 2024-09-21T03:38:02.774 INFO:teuthology.orchestra.run.smithi039.stdout:client.rgw advanced rgw_enable_usage_log true 2024-09-21T03:38:02.775 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:38:02.767+0000 7f3b2055d640 1 -- 172.21.15.39:0/3502275107 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f3b040b6550 msgr2=0x7f3b040b8a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:38:02.775 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:38:02.767+0000 7f3b2055d640 1 --2- 172.21.15.39:0/3502275107 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f3b040b6550 0x7f3b040b8a60 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f3b14005ab0 tx=0x7f3b14005a20 comp rx=0 tx=0).stop 2024-09-21T03:38:02.775 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:38:02.767+0000 7f3b2055d640 1 -- 172.21.15.39:0/3502275107 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3b18108d20 msgr2=0x7f3b1819ec10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:38:02.775 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:38:02.767+0000 7f3b2055d640 1 --2- 172.21.15.39:0/3502275107 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3b18108d20 0x7f3b1819ec10 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f3b1000eb10 tx=0x7f3b1000c2d0 comp rx=0 tx=0).stop 2024-09-21T03:38:02.775 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:38:02.767+0000 7f3b2055d640 1 -- 172.21.15.39:0/3502275107 shutdown_connections 2024-09-21T03:38:02.775 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:38:02.767+0000 7f3b2055d640 1 --2- 172.21.15.39:0/3502275107 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f3b040b6550 0x7f3b040b8a60 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:02.775 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:38:02.767+0000 7f3b2055d640 1 --2- 172.21.15.39:0/3502275107 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3b18108d20 0x7f3b1819ec10 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:02.775 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:38:02.767+0000 7f3b2055d640 1 --2- 172.21.15.39:0/3502275107 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3b181083d0 0x7f3b18199970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:02.775 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:38:02.767+0000 7f3b2055d640 1 --2- 172.21.15.39:0/3502275107 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3b18102ae0 0x7f3b181993d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:02.775 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:38:02.767+0000 7f3b2055d640 1 -- 172.21.15.39:0/3502275107 >> 172.21.15.39:0/3502275107 conn(0x7f3b18079760 msgr2=0x7f3b18101410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:38:02.775 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:38:02.767+0000 7f3b2055d640 1 -- 172.21.15.39:0/3502275107 shutdown_connections 2024-09-21T03:38:02.775 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:38:02.767+0000 7f3b2055d640 1 -- 172.21.15.39:0/3502275107 wait complete. 2024-09-21T03:38:02.832 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:38:02 smithi161 ceph-mon[34533]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:38:02.832 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:38:02 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:38:02.832 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:38:02 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:38:02.832 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:38:02 smithi161 ceph-mon[34533]: Reconfiguring mon.c (monmap changed)... 2024-09-21T03:38:02.832 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:38:02 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-09-21T03:38:02.832 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:38:02 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "config get", "who": "mon", "key": "public_network"} : dispatch 2024-09-21T03:38:02.833 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:38:02 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:38:02.833 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:38:02 smithi161 ceph-mon[34533]: Reconfiguring daemon mon.c on smithi161 2024-09-21T03:38:02.974 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:38:02 smithi150 ceph-mon[34289]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:38:02.974 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:38:02 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:38:02.975 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:38:02 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:38:02.975 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:38:02 smithi150 ceph-mon[34289]: Reconfiguring mon.c (monmap changed)... 2024-09-21T03:38:02.975 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:38:02 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-09-21T03:38:02.975 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:38:02 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "config get", "who": "mon", "key": "public_network"} : dispatch 2024-09-21T03:38:02.975 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:38:02 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:38:02.975 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:38:02 smithi150 ceph-mon[34289]: Reconfiguring daemon mon.c on smithi161 2024-09-21T03:38:03.394 INFO:tasks.cephadm:Deploying OSDs... 2024-09-21T03:38:03.394 DEBUG:teuthology.orchestra.run.smithi039:> set -ex 2024-09-21T03:38:03.394 DEBUG:teuthology.orchestra.run.smithi039:> dd if=/scratch_devs of=/dev/stdout 2024-09-21T03:38:03.411 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-21T03:38:03.412 DEBUG:teuthology.orchestra.run.smithi039:> stat /dev/vg_nvme/lv_1 2024-09-21T03:38:03.467 INFO:teuthology.orchestra.run.smithi039.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-21T03:38:03.468 INFO:teuthology.orchestra.run.smithi039.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-21T03:38:03.468 INFO:teuthology.orchestra.run.smithi039.stdout:Device: 5h/5d Inode: 666 Links: 1 2024-09-21T03:38:03.468 INFO:teuthology.orchestra.run.smithi039.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-21T03:38:03.468 INFO:teuthology.orchestra.run.smithi039.stdout:Context: system_u:object_r:device_t:s0 2024-09-21T03:38:03.468 INFO:teuthology.orchestra.run.smithi039.stdout:Access: 2024-09-21 03:36:38.587062972 +0000 2024-09-21T03:38:03.468 INFO:teuthology.orchestra.run.smithi039.stdout:Modify: 2024-09-21 03:31:05.584123463 +0000 2024-09-21T03:38:03.468 INFO:teuthology.orchestra.run.smithi039.stdout:Change: 2024-09-21 03:31:05.584123463 +0000 2024-09-21T03:38:03.468 INFO:teuthology.orchestra.run.smithi039.stdout: Birth: 2024-09-21 03:31:05.584123463 +0000 2024-09-21T03:38:03.468 DEBUG:teuthology.orchestra.run.smithi039:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-21T03:38:03.536 INFO:teuthology.orchestra.run.smithi039.stderr:1+0 records in 2024-09-21T03:38:03.536 INFO:teuthology.orchestra.run.smithi039.stderr:1+0 records out 2024-09-21T03:38:03.536 INFO:teuthology.orchestra.run.smithi039.stderr:512 bytes copied, 0.000308865 s, 1.7 MB/s 2024-09-21T03:38:03.537 DEBUG:teuthology.orchestra.run.smithi039:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-21T03:38:03.594 DEBUG:teuthology.orchestra.run.smithi039:> stat /dev/vg_nvme/lv_2 2024-09-21T03:38:03.650 INFO:teuthology.orchestra.run.smithi039.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-21T03:38:03.650 INFO:teuthology.orchestra.run.smithi039.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-21T03:38:03.650 INFO:teuthology.orchestra.run.smithi039.stdout:Device: 5h/5d Inode: 685 Links: 1 2024-09-21T03:38:03.650 INFO:teuthology.orchestra.run.smithi039.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-21T03:38:03.650 INFO:teuthology.orchestra.run.smithi039.stdout:Context: system_u:object_r:device_t:s0 2024-09-21T03:38:03.650 INFO:teuthology.orchestra.run.smithi039.stdout:Access: 2024-09-21 03:36:38.625062222 +0000 2024-09-21T03:38:03.650 INFO:teuthology.orchestra.run.smithi039.stdout:Modify: 2024-09-21 03:31:05.910116999 +0000 2024-09-21T03:38:03.650 INFO:teuthology.orchestra.run.smithi039.stdout:Change: 2024-09-21 03:31:05.910116999 +0000 2024-09-21T03:38:03.651 INFO:teuthology.orchestra.run.smithi039.stdout: Birth: 2024-09-21 03:31:05.910116999 +0000 2024-09-21T03:38:03.651 DEBUG:teuthology.orchestra.run.smithi039:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-21T03:38:03.713 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:38:03 smithi039 ceph-mon[31211]: from='client.? 172.21.15.39:0/3502275107' entity='client.admin' cmd={"prefix": "config dump"} : dispatch 2024-09-21T03:38:03.713 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:38:03 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:38:03.713 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:38:03 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:38:03.713 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:38:03 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:38:03.713 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:38:03 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:38:03.713 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:38:03 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:38:03.713 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:38:03 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:38:03.715 INFO:teuthology.orchestra.run.smithi039.stderr:1+0 records in 2024-09-21T03:38:03.715 INFO:teuthology.orchestra.run.smithi039.stderr:1+0 records out 2024-09-21T03:38:03.715 INFO:teuthology.orchestra.run.smithi039.stderr:512 bytes copied, 0.000282547 s, 1.8 MB/s 2024-09-21T03:38:03.717 DEBUG:teuthology.orchestra.run.smithi039:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-21T03:38:03.774 DEBUG:teuthology.orchestra.run.smithi039:> stat /dev/vg_nvme/lv_3 2024-09-21T03:38:03.830 INFO:teuthology.orchestra.run.smithi039.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-21T03:38:03.830 INFO:teuthology.orchestra.run.smithi039.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-21T03:38:03.830 INFO:teuthology.orchestra.run.smithi039.stdout:Device: 5h/5d Inode: 703 Links: 1 2024-09-21T03:38:03.831 INFO:teuthology.orchestra.run.smithi039.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-21T03:38:03.831 INFO:teuthology.orchestra.run.smithi039.stdout:Context: system_u:object_r:device_t:s0 2024-09-21T03:38:03.831 INFO:teuthology.orchestra.run.smithi039.stdout:Access: 2024-09-21 03:36:38.654061650 +0000 2024-09-21T03:38:03.831 INFO:teuthology.orchestra.run.smithi039.stdout:Modify: 2024-09-21 03:31:06.239110475 +0000 2024-09-21T03:38:03.831 INFO:teuthology.orchestra.run.smithi039.stdout:Change: 2024-09-21 03:31:06.239110475 +0000 2024-09-21T03:38:03.831 INFO:teuthology.orchestra.run.smithi039.stdout: Birth: 2024-09-21 03:31:06.239110475 +0000 2024-09-21T03:38:03.831 DEBUG:teuthology.orchestra.run.smithi039:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-21T03:38:03.894 INFO:teuthology.orchestra.run.smithi039.stderr:1+0 records in 2024-09-21T03:38:03.894 INFO:teuthology.orchestra.run.smithi039.stderr:1+0 records out 2024-09-21T03:38:03.894 INFO:teuthology.orchestra.run.smithi039.stderr:512 bytes copied, 0.000284431 s, 1.8 MB/s 2024-09-21T03:38:03.896 DEBUG:teuthology.orchestra.run.smithi039:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-21T03:38:03.953 DEBUG:teuthology.orchestra.run.smithi039:> stat /dev/vg_nvme/lv_4 2024-09-21T03:38:03.968 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:38:03 smithi161 ceph-mon[34533]: from='client.? 172.21.15.39:0/3502275107' entity='client.admin' cmd={"prefix": "config dump"} : dispatch 2024-09-21T03:38:03.968 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:38:03 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:38:03.968 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:38:03 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:38:03.968 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:38:03 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:38:03.968 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:38:03 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:38:03.968 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:38:03 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:38:03.968 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:38:03 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:38:03.974 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:38:03 smithi150 ceph-mon[34289]: from='client.? 172.21.15.39:0/3502275107' entity='client.admin' cmd={"prefix": "config dump"} : dispatch 2024-09-21T03:38:03.974 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:38:03 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:38:03.974 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:38:03 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:38:03.974 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:38:03 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:38:03.975 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:38:03 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:38:03.975 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:38:03 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:38:03.975 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:38:03 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:38:04.009 INFO:teuthology.orchestra.run.smithi039.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-21T03:38:04.009 INFO:teuthology.orchestra.run.smithi039.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-21T03:38:04.009 INFO:teuthology.orchestra.run.smithi039.stdout:Device: 5h/5d Inode: 722 Links: 1 2024-09-21T03:38:04.009 INFO:teuthology.orchestra.run.smithi039.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-21T03:38:04.010 INFO:teuthology.orchestra.run.smithi039.stdout:Context: system_u:object_r:device_t:s0 2024-09-21T03:38:04.010 INFO:teuthology.orchestra.run.smithi039.stdout:Access: 2024-09-21 03:36:38.681061117 +0000 2024-09-21T03:38:04.010 INFO:teuthology.orchestra.run.smithi039.stdout:Modify: 2024-09-21 03:31:06.565104011 +0000 2024-09-21T03:38:04.010 INFO:teuthology.orchestra.run.smithi039.stdout:Change: 2024-09-21 03:31:06.565104011 +0000 2024-09-21T03:38:04.010 INFO:teuthology.orchestra.run.smithi039.stdout: Birth: 2024-09-21 03:31:06.565104011 +0000 2024-09-21T03:38:04.010 DEBUG:teuthology.orchestra.run.smithi039:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-21T03:38:04.073 INFO:teuthology.orchestra.run.smithi039.stderr:1+0 records in 2024-09-21T03:38:04.074 INFO:teuthology.orchestra.run.smithi039.stderr:1+0 records out 2024-09-21T03:38:04.074 INFO:teuthology.orchestra.run.smithi039.stderr:512 bytes copied, 0.000264808 s, 1.9 MB/s 2024-09-21T03:38:04.075 DEBUG:teuthology.orchestra.run.smithi039:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-21T03:38:04.132 DEBUG:teuthology.orchestra.run.smithi150:> set -ex 2024-09-21T03:38:04.132 DEBUG:teuthology.orchestra.run.smithi150:> dd if=/scratch_devs of=/dev/stdout 2024-09-21T03:38:04.152 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-21T03:38:04.152 DEBUG:teuthology.orchestra.run.smithi150:> stat /dev/vg_nvme/lv_1 2024-09-21T03:38:04.210 INFO:teuthology.orchestra.run.smithi150.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-21T03:38:04.211 INFO:teuthology.orchestra.run.smithi150.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-21T03:38:04.211 INFO:teuthology.orchestra.run.smithi150.stdout:Device: 5h/5d Inode: 665 Links: 1 2024-09-21T03:38:04.211 INFO:teuthology.orchestra.run.smithi150.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-21T03:38:04.211 INFO:teuthology.orchestra.run.smithi150.stdout:Context: system_u:object_r:device_t:s0 2024-09-21T03:38:04.211 INFO:teuthology.orchestra.run.smithi150.stdout:Access: 2024-09-21 03:37:33.652284884 +0000 2024-09-21T03:38:04.211 INFO:teuthology.orchestra.run.smithi150.stdout:Modify: 2024-09-21 03:31:09.839046564 +0000 2024-09-21T03:38:04.211 INFO:teuthology.orchestra.run.smithi150.stdout:Change: 2024-09-21 03:31:09.839046564 +0000 2024-09-21T03:38:04.211 INFO:teuthology.orchestra.run.smithi150.stdout: Birth: 2024-09-21 03:31:09.839046564 +0000 2024-09-21T03:38:04.211 DEBUG:teuthology.orchestra.run.smithi150:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-21T03:38:04.281 INFO:teuthology.orchestra.run.smithi150.stderr:1+0 records in 2024-09-21T03:38:04.281 INFO:teuthology.orchestra.run.smithi150.stderr:1+0 records out 2024-09-21T03:38:04.281 INFO:teuthology.orchestra.run.smithi150.stderr:512 bytes copied, 0.000299086 s, 1.7 MB/s 2024-09-21T03:38:04.282 DEBUG:teuthology.orchestra.run.smithi150:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-21T03:38:04.341 DEBUG:teuthology.orchestra.run.smithi150:> stat /dev/vg_nvme/lv_2 2024-09-21T03:38:04.397 INFO:teuthology.orchestra.run.smithi150.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-21T03:38:04.397 INFO:teuthology.orchestra.run.smithi150.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-21T03:38:04.397 INFO:teuthology.orchestra.run.smithi150.stdout:Device: 5h/5d Inode: 685 Links: 1 2024-09-21T03:38:04.397 INFO:teuthology.orchestra.run.smithi150.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-21T03:38:04.397 INFO:teuthology.orchestra.run.smithi150.stdout:Context: system_u:object_r:device_t:s0 2024-09-21T03:38:04.398 INFO:teuthology.orchestra.run.smithi150.stdout:Access: 2024-09-21 03:37:33.652284884 +0000 2024-09-21T03:38:04.398 INFO:teuthology.orchestra.run.smithi150.stdout:Modify: 2024-09-21 03:31:10.180039521 +0000 2024-09-21T03:38:04.398 INFO:teuthology.orchestra.run.smithi150.stdout:Change: 2024-09-21 03:31:10.180039521 +0000 2024-09-21T03:38:04.398 INFO:teuthology.orchestra.run.smithi150.stdout: Birth: 2024-09-21 03:31:10.180039521 +0000 2024-09-21T03:38:04.398 DEBUG:teuthology.orchestra.run.smithi150:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-21T03:38:04.463 INFO:teuthology.orchestra.run.smithi150.stderr:1+0 records in 2024-09-21T03:38:04.463 INFO:teuthology.orchestra.run.smithi150.stderr:1+0 records out 2024-09-21T03:38:04.463 INFO:teuthology.orchestra.run.smithi150.stderr:512 bytes copied, 0.000232 s, 2.2 MB/s 2024-09-21T03:38:04.464 DEBUG:teuthology.orchestra.run.smithi150:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-21T03:38:04.521 DEBUG:teuthology.orchestra.run.smithi150:> stat /dev/vg_nvme/lv_3 2024-09-21T03:38:04.578 INFO:teuthology.orchestra.run.smithi150.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-21T03:38:04.578 INFO:teuthology.orchestra.run.smithi150.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-21T03:38:04.578 INFO:teuthology.orchestra.run.smithi150.stdout:Device: 5h/5d Inode: 704 Links: 1 2024-09-21T03:38:04.578 INFO:teuthology.orchestra.run.smithi150.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-21T03:38:04.578 INFO:teuthology.orchestra.run.smithi150.stdout:Context: system_u:object_r:device_t:s0 2024-09-21T03:38:04.578 INFO:teuthology.orchestra.run.smithi150.stdout:Access: 2024-09-21 03:37:33.653284863 +0000 2024-09-21T03:38:04.578 INFO:teuthology.orchestra.run.smithi150.stdout:Modify: 2024-09-21 03:31:10.516032581 +0000 2024-09-21T03:38:04.578 INFO:teuthology.orchestra.run.smithi150.stdout:Change: 2024-09-21 03:31:10.516032581 +0000 2024-09-21T03:38:04.578 INFO:teuthology.orchestra.run.smithi150.stdout: Birth: 2024-09-21 03:31:10.516032581 +0000 2024-09-21T03:38:04.578 DEBUG:teuthology.orchestra.run.smithi150:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-21T03:38:04.642 INFO:teuthology.orchestra.run.smithi150.stderr:1+0 records in 2024-09-21T03:38:04.642 INFO:teuthology.orchestra.run.smithi150.stderr:1+0 records out 2024-09-21T03:38:04.642 INFO:teuthology.orchestra.run.smithi150.stderr:512 bytes copied, 0.000194967 s, 2.6 MB/s 2024-09-21T03:38:04.643 DEBUG:teuthology.orchestra.run.smithi150:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-21T03:38:04.700 DEBUG:teuthology.orchestra.run.smithi150:> stat /dev/vg_nvme/lv_4 2024-09-21T03:38:04.755 INFO:teuthology.orchestra.run.smithi150.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-21T03:38:04.755 INFO:teuthology.orchestra.run.smithi150.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-21T03:38:04.755 INFO:teuthology.orchestra.run.smithi150.stdout:Device: 5h/5d Inode: 721 Links: 1 2024-09-21T03:38:04.755 INFO:teuthology.orchestra.run.smithi150.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-21T03:38:04.755 INFO:teuthology.orchestra.run.smithi150.stdout:Context: system_u:object_r:device_t:s0 2024-09-21T03:38:04.755 INFO:teuthology.orchestra.run.smithi150.stdout:Access: 2024-09-21 03:37:33.653284863 +0000 2024-09-21T03:38:04.755 INFO:teuthology.orchestra.run.smithi150.stdout:Modify: 2024-09-21 03:31:10.851025661 +0000 2024-09-21T03:38:04.755 INFO:teuthology.orchestra.run.smithi150.stdout:Change: 2024-09-21 03:31:10.851025661 +0000 2024-09-21T03:38:04.756 INFO:teuthology.orchestra.run.smithi150.stdout: Birth: 2024-09-21 03:31:10.851025661 +0000 2024-09-21T03:38:04.756 DEBUG:teuthology.orchestra.run.smithi150:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-21T03:38:04.819 INFO:teuthology.orchestra.run.smithi150.stderr:1+0 records in 2024-09-21T03:38:04.819 INFO:teuthology.orchestra.run.smithi150.stderr:1+0 records out 2024-09-21T03:38:04.819 INFO:teuthology.orchestra.run.smithi150.stderr:512 bytes copied, 0.000270232 s, 1.9 MB/s 2024-09-21T03:38:04.821 DEBUG:teuthology.orchestra.run.smithi150:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-21T03:38:04.877 DEBUG:teuthology.orchestra.run.smithi161:> set -ex 2024-09-21T03:38:04.877 DEBUG:teuthology.orchestra.run.smithi161:> dd if=/scratch_devs of=/dev/stdout 2024-09-21T03:38:04.897 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-21T03:38:04.897 DEBUG:teuthology.orchestra.run.smithi161:> stat /dev/vg_nvme/lv_1 2024-09-21T03:38:04.956 INFO:teuthology.orchestra.run.smithi161.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-21T03:38:04.956 INFO:teuthology.orchestra.run.smithi161.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-21T03:38:04.956 INFO:teuthology.orchestra.run.smithi161.stdout:Device: 5h/5d Inode: 667 Links: 1 2024-09-21T03:38:04.956 INFO:teuthology.orchestra.run.smithi161.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-21T03:38:04.956 INFO:teuthology.orchestra.run.smithi161.stdout:Context: system_u:object_r:device_t:s0 2024-09-21T03:38:04.956 INFO:teuthology.orchestra.run.smithi161.stdout:Access: 2024-09-21 03:37:31.427355638 +0000 2024-09-21T03:38:04.957 INFO:teuthology.orchestra.run.smithi161.stdout:Modify: 2024-09-21 03:31:14.597777708 +0000 2024-09-21T03:38:04.957 INFO:teuthology.orchestra.run.smithi161.stdout:Change: 2024-09-21 03:31:14.597777708 +0000 2024-09-21T03:38:04.957 INFO:teuthology.orchestra.run.smithi161.stdout: Birth: 2024-09-21 03:31:14.597777708 +0000 2024-09-21T03:38:04.957 DEBUG:teuthology.orchestra.run.smithi161:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-21T03:38:04.974 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:38:04 smithi150 ceph-mon[34289]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:38:05.027 INFO:teuthology.orchestra.run.smithi161.stderr:1+0 records in 2024-09-21T03:38:05.027 INFO:teuthology.orchestra.run.smithi161.stderr:1+0 records out 2024-09-21T03:38:05.027 INFO:teuthology.orchestra.run.smithi161.stderr:512 bytes copied, 0.000306363 s, 1.7 MB/s 2024-09-21T03:38:05.028 DEBUG:teuthology.orchestra.run.smithi161:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-21T03:38:05.083 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:38:04 smithi039 ceph-mon[31211]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:38:05.087 DEBUG:teuthology.orchestra.run.smithi161:> stat /dev/vg_nvme/lv_2 2024-09-21T03:38:05.145 INFO:teuthology.orchestra.run.smithi161.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-21T03:38:05.145 INFO:teuthology.orchestra.run.smithi161.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-21T03:38:05.145 INFO:teuthology.orchestra.run.smithi161.stdout:Device: 5h/5d Inode: 683 Links: 1 2024-09-21T03:38:05.145 INFO:teuthology.orchestra.run.smithi161.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-21T03:38:05.145 INFO:teuthology.orchestra.run.smithi161.stdout:Context: system_u:object_r:device_t:s0 2024-09-21T03:38:05.145 INFO:teuthology.orchestra.run.smithi161.stdout:Access: 2024-09-21 03:37:31.428355617 +0000 2024-09-21T03:38:05.145 INFO:teuthology.orchestra.run.smithi161.stdout:Modify: 2024-09-21 03:31:14.922770720 +0000 2024-09-21T03:38:05.145 INFO:teuthology.orchestra.run.smithi161.stdout:Change: 2024-09-21 03:31:14.922770720 +0000 2024-09-21T03:38:05.145 INFO:teuthology.orchestra.run.smithi161.stdout: Birth: 2024-09-21 03:31:14.922770720 +0000 2024-09-21T03:38:05.146 DEBUG:teuthology.orchestra.run.smithi161:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-21T03:38:05.209 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:38:04 smithi161 ceph-mon[34533]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:38:05.211 INFO:teuthology.orchestra.run.smithi161.stderr:1+0 records in 2024-09-21T03:38:05.211 INFO:teuthology.orchestra.run.smithi161.stderr:1+0 records out 2024-09-21T03:38:05.211 INFO:teuthology.orchestra.run.smithi161.stderr:512 bytes copied, 0.00024883 s, 2.1 MB/s 2024-09-21T03:38:05.212 DEBUG:teuthology.orchestra.run.smithi161:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-21T03:38:05.270 DEBUG:teuthology.orchestra.run.smithi161:> stat /dev/vg_nvme/lv_3 2024-09-21T03:38:05.326 INFO:teuthology.orchestra.run.smithi161.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-21T03:38:05.326 INFO:teuthology.orchestra.run.smithi161.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-21T03:38:05.326 INFO:teuthology.orchestra.run.smithi161.stdout:Device: 5h/5d Inode: 703 Links: 1 2024-09-21T03:38:05.326 INFO:teuthology.orchestra.run.smithi161.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-21T03:38:05.326 INFO:teuthology.orchestra.run.smithi161.stdout:Context: system_u:object_r:device_t:s0 2024-09-21T03:38:05.326 INFO:teuthology.orchestra.run.smithi161.stdout:Access: 2024-09-21 03:37:31.428355617 +0000 2024-09-21T03:38:05.326 INFO:teuthology.orchestra.run.smithi161.stdout:Modify: 2024-09-21 03:31:15.234764012 +0000 2024-09-21T03:38:05.327 INFO:teuthology.orchestra.run.smithi161.stdout:Change: 2024-09-21 03:31:15.234764012 +0000 2024-09-21T03:38:05.327 INFO:teuthology.orchestra.run.smithi161.stdout: Birth: 2024-09-21 03:31:15.234764012 +0000 2024-09-21T03:38:05.327 DEBUG:teuthology.orchestra.run.smithi161:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-21T03:38:05.391 INFO:teuthology.orchestra.run.smithi161.stderr:1+0 records in 2024-09-21T03:38:05.391 INFO:teuthology.orchestra.run.smithi161.stderr:1+0 records out 2024-09-21T03:38:05.391 INFO:teuthology.orchestra.run.smithi161.stderr:512 bytes copied, 0.000232365 s, 2.2 MB/s 2024-09-21T03:38:05.392 DEBUG:teuthology.orchestra.run.smithi161:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-21T03:38:05.449 DEBUG:teuthology.orchestra.run.smithi161:> stat /dev/vg_nvme/lv_4 2024-09-21T03:38:05.505 INFO:teuthology.orchestra.run.smithi161.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-21T03:38:05.505 INFO:teuthology.orchestra.run.smithi161.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-21T03:38:05.505 INFO:teuthology.orchestra.run.smithi161.stdout:Device: 5h/5d Inode: 721 Links: 1 2024-09-21T03:38:05.505 INFO:teuthology.orchestra.run.smithi161.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-21T03:38:05.505 INFO:teuthology.orchestra.run.smithi161.stdout:Context: system_u:object_r:device_t:s0 2024-09-21T03:38:05.505 INFO:teuthology.orchestra.run.smithi161.stdout:Access: 2024-09-21 03:37:31.429355596 +0000 2024-09-21T03:38:05.505 INFO:teuthology.orchestra.run.smithi161.stdout:Modify: 2024-09-21 03:31:15.559757023 +0000 2024-09-21T03:38:05.505 INFO:teuthology.orchestra.run.smithi161.stdout:Change: 2024-09-21 03:31:15.559757023 +0000 2024-09-21T03:38:05.505 INFO:teuthology.orchestra.run.smithi161.stdout: Birth: 2024-09-21 03:31:15.559757023 +0000 2024-09-21T03:38:05.506 DEBUG:teuthology.orchestra.run.smithi161:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-21T03:38:05.569 INFO:teuthology.orchestra.run.smithi161.stderr:1+0 records in 2024-09-21T03:38:05.569 INFO:teuthology.orchestra.run.smithi161.stderr:1+0 records out 2024-09-21T03:38:05.569 INFO:teuthology.orchestra.run.smithi161.stderr:512 bytes copied, 0.000205524 s, 2.5 MB/s 2024-09-21T03:38:05.570 DEBUG:teuthology.orchestra.run.smithi161:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-21T03:38:05.626 INFO:tasks.cephadm:Deploying osd.0 on smithi039 with /dev/vg_nvme/lv_4... 2024-09-21T03:38:05.627 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 51116d5c-77ca-11ef-baf5-efdc52797490 -- lvm zap /dev/vg_nvme/lv_4 2024-09-21T03:38:05.889 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/51116d5c-77ca-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:38:07.080 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:38:06 smithi039 ceph-mon[31211]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:38:07.218 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:38:06 smithi161 ceph-mon[34533]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:38:07.224 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:38:06 smithi150 ceph-mon[34289]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:38:08.364 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T03:38:08.392 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 51116d5c-77ca-11ef-baf5-efdc52797490 -- ceph orch daemon add osd smithi039:vg_nvme/lv_4 2024-09-21T03:38:08.643 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/51116d5c-77ca-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:38:09.022 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:38:08 smithi039 ceph-mon[31211]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:38:09.088 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:38:08 smithi161 ceph-mon[34533]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:38:09.121 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:38:09.119+0000 7f46e38fa640 1 -- 172.21.15.39:0/2098352732 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f46dc103b80 msgr2=0x7f46dc103f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:38:09.121 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:38:09.119+0000 7f46e38fa640 1 --2- 172.21.15.39:0/2098352732 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f46dc103b80 0x7f46dc103f90 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f46d0009900 tx=0x7f46d002a410 comp rx=0 tx=0).stop 2024-09-21T03:38:09.121 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:38:09.119+0000 7f46e38fa640 1 -- 172.21.15.39:0/2098352732 shutdown_connections 2024-09-21T03:38:09.121 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:38:09.119+0000 7f46e38fa640 1 --2- 172.21.15.39:0/2098352732 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f46dc109e10 0x7f46dc110640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:09.121 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:38:09.119+0000 7f46e38fa640 1 --2- 172.21.15.39:0/2098352732 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f46dc1094c0 0x7f46dc1098d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:09.122 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:38:09.119+0000 7f46e38fa640 1 --2- 172.21.15.39:0/2098352732 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f46dc103b80 0x7f46dc103f90 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:09.122 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:38:09.119+0000 7f46e38fa640 1 -- 172.21.15.39:0/2098352732 >> 172.21.15.39:0/2098352732 conn(0x7f46dc0ff610 msgr2=0x7f46dc101a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:38:09.122 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:38:09.119+0000 7f46e38fa640 1 -- 172.21.15.39:0/2098352732 shutdown_connections 2024-09-21T03:38:09.122 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:38:09.119+0000 7f46e38fa640 1 -- 172.21.15.39:0/2098352732 wait complete. 2024-09-21T03:38:09.124 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:38:09.120+0000 7f46e38fa640 1 Processor -- start 2024-09-21T03:38:09.124 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:38:09.120+0000 7f46e38fa640 1 -- start start 2024-09-21T03:38:09.124 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:38:09.121+0000 7f46e38fa640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f46dc103b80 0x7f46dc19a4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:38:09.124 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:38:09.121+0000 7f46e38fa640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f46dc1094c0 0x7f46dc19aa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:38:09.124 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:38:09.121+0000 7f46e38fa640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f46dc109e10 0x7f46dc19fd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:38:09.124 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:38:09.121+0000 7f46e38fa640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f46dc113ad0 con 0x7f46dc109e10 2024-09-21T03:38:09.124 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:38:09.121+0000 7f46e38fa640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f46dc113950 con 0x7f46dc1094c0 2024-09-21T03:38:09.124 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:38:09.121+0000 7f46e38fa640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f46dc113c50 con 0x7f46dc103b80 2024-09-21T03:38:09.124 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:38:09.121+0000 7f46e1e97640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f46dc109e10 0x7f46dc19fd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:38:09.124 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:38:09.121+0000 7f46e1e97640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f46dc109e10 0x7f46dc19fd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:39162/0 (socket says 172.21.15.39:39162) 2024-09-21T03:38:09.125 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:38:09.121+0000 7f46e1e97640 1 -- 172.21.15.39:0/1353940274 learned_addr learned my addr 172.21.15.39:0/1353940274 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:38:09.125 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:38:09.121+0000 7f46e0e95640 1 --2- 172.21.15.39:0/1353940274 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f46dc1094c0 0x7f46dc19aa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:38:09.125 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:38:09.121+0000 7f46e1696640 1 --2- 172.21.15.39:0/1353940274 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f46dc103b80 0x7f46dc19a4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:38:09.125 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:38:09.121+0000 7f46e1e97640 1 -- 172.21.15.39:0/1353940274 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f46dc103b80 msgr2=0x7f46dc19a4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:38:09.125 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:38:09.121+0000 7f46e1e97640 1 --2- 172.21.15.39:0/1353940274 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f46dc103b80 0x7f46dc19a4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:09.125 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:38:09.121+0000 7f46e1e97640 1 -- 172.21.15.39:0/1353940274 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f46dc1094c0 msgr2=0x7f46dc19aa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:38:09.125 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:38:09.121+0000 7f46e1e97640 1 --2- 172.21.15.39:0/1353940274 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f46dc1094c0 0x7f46dc19aa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:09.125 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:38:09.121+0000 7f46e1e97640 1 -- 172.21.15.39:0/1353940274 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f46dc1a0450 con 0x7f46dc109e10 2024-09-21T03:38:09.125 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:38:09.121+0000 7f46e1696640 1 --2- 172.21.15.39:0/1353940274 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f46dc103b80 0x7f46dc19a4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:38:09.125 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:38:09.121+0000 7f46e1e97640 1 --2- 172.21.15.39:0/1353940274 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f46dc109e10 0x7f46dc19fd20 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f46d400cad0 tx=0x7f46d400cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:38:09.125 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:38:09.122+0000 7f46ce7fc640 1 -- 172.21.15.39:0/1353940274 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46d400ae20 con 0x7f46dc109e10 2024-09-21T03:38:09.125 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:38:09.122+0000 7f46e38fa640 1 -- 172.21.15.39:0/1353940274 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f46dc1a0740 con 0x7f46dc109e10 2024-09-21T03:38:09.125 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:38:09.122+0000 7f46ce7fc640 1 -- 172.21.15.39:0/1353940274 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f46d4005020 con 0x7f46dc109e10 2024-09-21T03:38:09.125 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:38:09.122+0000 7f46ce7fc640 1 -- 172.21.15.39:0/1353940274 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46d400f6b0 con 0x7f46dc109e10 2024-09-21T03:38:09.126 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:38:09.122+0000 7f46e38fa640 1 -- 172.21.15.39:0/1353940274 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f46dc1a6ea0 con 0x7f46dc109e10 2024-09-21T03:38:09.126 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:38:09.123+0000 7f46e38fa640 1 -- 172.21.15.39:0/1353940274 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f46dc103f90 con 0x7f46dc109e10 2024-09-21T03:38:09.126 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:38:09.125+0000 7f46ce7fc640 1 -- 172.21.15.39:0/1353940274 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 20) ==== 154473+0+0 (secure 0 0 0) 0x7f46d40051d0 con 0x7f46dc109e10 2024-09-21T03:38:09.127 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:38:09.125+0000 7f46ce7fc640 1 --2- 172.21.15.39:0/1353940274 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f46c80b64d0 0x7f46c80b89e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:38:09.127 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:38:09.125+0000 7f46e1696640 1 --2- 172.21.15.39:0/1353940274 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f46c80b64d0 0x7f46c80b89e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:38:09.127 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:38:09.125+0000 7f46ce7fc640 1 -- 172.21.15.39:0/1353940274 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1370+0+0 (secure 0 0 0) 0x7f46d40e5b30 con 0x7f46dc109e10 2024-09-21T03:38:09.127 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:38:09.126+0000 7f46e1696640 1 --2- 172.21.15.39:0/1353940274 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f46c80b64d0 0x7f46c80b89e0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f46d0031440 tx=0x7f46d0009680 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:38:09.134 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:38:09.133+0000 7f46ce7fc640 1 -- 172.21.15.39:0/1353940274 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f46d4014030 con 0x7f46dc109e10 2024-09-21T03:38:09.224 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:38:08 smithi150 ceph-mon[34289]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:38:09.336 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:38:09.334+0000 7f46e38fa640 1 -- 172.21.15.39:0/1353940274 --> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi039:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7f46dc063800 con 0x7f46c80b64d0 2024-09-21T03:38:09.989 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:38:09 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T03:38:09.989 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:38:09 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T03:38:09.989 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:38:09 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:38:10.217 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:38:09 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T03:38:10.218 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:38:09 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T03:38:10.218 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:38:09 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:38:10.224 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:38:09 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T03:38:10.224 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:38:09 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T03:38:10.225 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:38:09 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:38:11.057 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:38:10 smithi039 ceph-mon[31211]: from='client.14451 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi039:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:38:11.058 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:38:10 smithi039 ceph-mon[31211]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:38:11.217 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:38:10 smithi161 ceph-mon[34533]: from='client.14451 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi039:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:38:11.218 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:38:10 smithi161 ceph-mon[34533]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:38:11.224 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:38:10 smithi150 ceph-mon[34289]: from='client.14451 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi039:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:38:11.224 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:38:10 smithi150 ceph-mon[34289]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:38:12.083 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:38:11 smithi039 ceph-mon[31211]: from='client.? 172.21.15.39:0/2854892245' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "1ea7148f-2ace-4c10-86fe-b8019de80c1f"} : dispatch 2024-09-21T03:38:12.084 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:38:11 smithi039 ceph-mon[31211]: from='client.? 172.21.15.39:0/2854892245' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "1ea7148f-2ace-4c10-86fe-b8019de80c1f"}]': finished 2024-09-21T03:38:12.217 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:38:11 smithi161 ceph-mon[34533]: from='client.? 172.21.15.39:0/2854892245' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "1ea7148f-2ace-4c10-86fe-b8019de80c1f"} : dispatch 2024-09-21T03:38:12.218 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:38:11 smithi161 ceph-mon[34533]: from='client.? 172.21.15.39:0/2854892245' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "1ea7148f-2ace-4c10-86fe-b8019de80c1f"}]': finished 2024-09-21T03:38:12.224 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:38:11 smithi150 ceph-mon[34289]: from='client.? 172.21.15.39:0/2854892245' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "1ea7148f-2ace-4c10-86fe-b8019de80c1f"} : dispatch 2024-09-21T03:38:12.224 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:38:11 smithi150 ceph-mon[34289]: from='client.? 172.21.15.39:0/2854892245' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "1ea7148f-2ace-4c10-86fe-b8019de80c1f"}]': finished 2024-09-21T03:38:13.217 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:38:12 smithi161 ceph-mon[34533]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:38:13.217 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:38:12 smithi161 ceph-mon[34533]: osdmap e6: 1 total, 0 up, 1 in 2024-09-21T03:38:13.218 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:38:12 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-21T03:38:13.218 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:38:12 smithi161 ceph-mon[34533]: from='client.? 172.21.15.39:0/1829696176' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T03:38:13.224 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:38:12 smithi150 ceph-mon[34289]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:38:13.224 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:38:12 smithi150 ceph-mon[34289]: osdmap e6: 1 total, 0 up, 1 in 2024-09-21T03:38:13.224 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:38:12 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-21T03:38:13.224 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:38:12 smithi150 ceph-mon[34289]: from='client.? 172.21.15.39:0/1829696176' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T03:38:13.334 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:38:12 smithi039 ceph-mon[31211]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:38:13.334 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:38:12 smithi039 ceph-mon[31211]: osdmap e6: 1 total, 0 up, 1 in 2024-09-21T03:38:13.334 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:38:12 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-21T03:38:13.334 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:38:12 smithi039 ceph-mon[31211]: from='client.? 172.21.15.39:0/1829696176' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T03:38:15.333 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:38:15 smithi039 ceph-mon[31211]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:38:15.467 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:38:15 smithi161 ceph-mon[34533]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:38:15.474 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:38:15 smithi150 ceph-mon[34289]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:38:17.333 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:38:17 smithi039 ceph-mon[31211]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:38:17.467 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:38:17 smithi161 ceph-mon[34533]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:38:17.474 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:38:17 smithi150 ceph-mon[34289]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:38:19.333 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:38:19 smithi039 ceph-mon[31211]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:38:19.467 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:38:19 smithi161 ceph-mon[34533]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:38:19.474 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:38:19 smithi150 ceph-mon[34289]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:38:21.333 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:38:21 smithi039 ceph-mon[31211]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:38:21.468 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:38:21 smithi161 ceph-mon[34533]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:38:21.474 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:38:21 smithi150 ceph-mon[34289]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:38:22.333 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:38:22 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.0"} : dispatch 2024-09-21T03:38:22.334 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:38:22 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:38:22.467 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:38:22 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.0"} : dispatch 2024-09-21T03:38:22.467 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:38:22 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:38:22.474 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:38:22 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.0"} : dispatch 2024-09-21T03:38:22.474 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:38:22 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:38:23.140 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:38:23 smithi039 ceph-mon[31211]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:38:23.140 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:38:23 smithi039 ceph-mon[31211]: Deploying daemon osd.0 on smithi039 2024-09-21T03:38:23.467 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:38:23 smithi161 ceph-mon[34533]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:38:23.468 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:38:23 smithi161 ceph-mon[34533]: Deploying daemon osd.0 on smithi039 2024-09-21T03:38:23.474 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:38:23 smithi150 ceph-mon[34289]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:38:23.474 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:38:23 smithi150 ceph-mon[34289]: Deploying daemon osd.0 on smithi039 2024-09-21T03:38:25.346 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:38:25 smithi039 ceph-mon[31211]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:38:25.467 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:38:25 smithi161 ceph-mon[34533]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:38:25.474 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:38:25 smithi150 ceph-mon[34289]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:38:27.247 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:38:27 smithi039 ceph-mon[31211]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:38:27.467 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:38:27 smithi161 ceph-mon[34533]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:38:27.474 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:38:27 smithi150 ceph-mon[34289]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:38:29.154 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:38:29 smithi039 ceph-mon[31211]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:38:29.154 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:38:29 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:38:29.154 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:38:29 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:38:29.154 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:38:29 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:38:29.468 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:38:29 smithi161 ceph-mon[34533]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:38:29.468 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:38:29 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:38:29.468 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:38:29 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:38:29.468 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:38:29 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:38:29.474 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:38:29 smithi150 ceph-mon[34289]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:38:29.474 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:38:29 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:38:29.474 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:38:29 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:38:29.474 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:38:29 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:38:31.333 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:38:31 smithi039 ceph-mon[31211]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:38:31.467 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:38:31 smithi161 ceph-mon[34533]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:38:31.474 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:38:31 smithi150 ceph-mon[34289]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:38:32.584 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:38:32 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:38:32.584 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:38:32 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:38:32.584 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:38:32 smithi039 ceph-mon[31211]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:38:32.584 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:38:32 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:38:32.584 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:38:32 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:38:32.584 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:38:32 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:38:32.717 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:38:32 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:38:32.718 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:38:32 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:38:32.718 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:38:32 smithi161 ceph-mon[34533]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:38:32.718 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:38:32 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:38:32.718 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:38:32 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:38:32.718 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:38:32 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:38:32.724 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:38:32 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:38:32.724 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:38:32 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:38:32.724 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:38:32 smithi150 ceph-mon[34289]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:38:32.724 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:38:32 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:38:32.724 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:38:32 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:38:32.725 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:38:32 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:38:32.927 INFO:teuthology.orchestra.run.smithi039.stdout:Created osd(s) 0 on host 'smithi039' 2024-09-21T03:38:32.928 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:38:32.924+0000 7f46ce7fc640 1 -- 172.21.15.39:0/1353940274 <== mgr.14223 v2:172.21.15.39:6800/1696381982 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f46dc063800 con 0x7f46c80b64d0 2024-09-21T03:38:32.930 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:38:32.927+0000 7f46e38fa640 1 -- 172.21.15.39:0/1353940274 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f46c80b64d0 msgr2=0x7f46c80b89e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:38:32.930 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:38:32.927+0000 7f46e38fa640 1 --2- 172.21.15.39:0/1353940274 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f46c80b64d0 0x7f46c80b89e0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f46d0031440 tx=0x7f46d0009680 comp rx=0 tx=0).stop 2024-09-21T03:38:32.930 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:38:32.927+0000 7f46e38fa640 1 -- 172.21.15.39:0/1353940274 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f46dc109e10 msgr2=0x7f46dc19fd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:38:32.930 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:38:32.927+0000 7f46e38fa640 1 --2- 172.21.15.39:0/1353940274 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f46dc109e10 0x7f46dc19fd20 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f46d400cad0 tx=0x7f46d400cf90 comp rx=0 tx=0).stop 2024-09-21T03:38:32.930 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:38:32.927+0000 7f46e38fa640 1 -- 172.21.15.39:0/1353940274 shutdown_connections 2024-09-21T03:38:32.930 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:38:32.927+0000 7f46e38fa640 1 --2- 172.21.15.39:0/1353940274 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f46c80b64d0 0x7f46c80b89e0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:32.930 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:38:32.927+0000 7f46e38fa640 1 --2- 172.21.15.39:0/1353940274 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f46dc109e10 0x7f46dc19fd20 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:32.931 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:38:32.927+0000 7f46e38fa640 1 --2- 172.21.15.39:0/1353940274 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f46dc1094c0 0x7f46dc19aa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:32.931 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:38:32.927+0000 7f46e38fa640 1 --2- 172.21.15.39:0/1353940274 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f46dc103b80 0x7f46dc19a4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:32.931 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:38:32.927+0000 7f46e38fa640 1 -- 172.21.15.39:0/1353940274 >> 172.21.15.39:0/1353940274 conn(0x7f46dc0ff610 msgr2=0x7f46dc101a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:38:32.931 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:38:32.927+0000 7f46e38fa640 1 -- 172.21.15.39:0/1353940274 shutdown_connections 2024-09-21T03:38:32.931 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:38:32.927+0000 7f46e38fa640 1 -- 172.21.15.39:0/1353940274 wait complete. 2024-09-21T03:38:33.583 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:38:33 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:38:33.584 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:38:33 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:38:33.584 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:38:33 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:38:33.660 DEBUG:teuthology.orchestra.run.smithi039:osd.0> sudo journalctl -f -n 0 -u ceph-51116d5c-77ca-11ef-baf5-efdc52797490@osd.0.service 2024-09-21T03:38:33.662 INFO:tasks.cephadm:Deploying osd.1 on smithi150 with /dev/vg_nvme/lv_4... 2024-09-21T03:38:33.663 DEBUG:teuthology.orchestra.run.smithi150:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 51116d5c-77ca-11ef-baf5-efdc52797490 -- lvm zap /dev/vg_nvme/lv_4 2024-09-21T03:38:33.690 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:38:33 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:38:33.690 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:38:33 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:38:33.690 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:38:33 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:38:33.717 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:38:33 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:38:33.717 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:38:33 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:38:33.718 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:38:33 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:38:33.914 INFO:teuthology.orchestra.run.smithi150.stderr:Inferring config /var/lib/ceph/51116d5c-77ca-11ef-baf5-efdc52797490/mon.b/config 2024-09-21T03:38:34.328 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:38:34 smithi150 ceph-mon[34289]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:38:34.536 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:38:34 smithi039 ceph-mon[31211]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:38:34.717 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:38:34 smithi161 ceph-mon[34533]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:38:34.788 INFO:journalctl@ceph.osd.0.smithi039.stdout:Sep 21 03:38:34 smithi039 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-0[47798]: 2024-09-21T03:38:34.533+0000 7f6ca0c92640 -1 osd.0 0 log_to_monitors true 2024-09-21T03:38:35.334 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:38:35 smithi039 ceph-mon[31211]: from='osd.0 [v2:172.21.15.39:6802/870188791,v1:172.21.15.39:6803/870188791]' entity='osd.0' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]} : dispatch 2024-09-21T03:38:35.401 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:38:35 smithi150 ceph-mon[34289]: from='osd.0 [v2:172.21.15.39:6802/870188791,v1:172.21.15.39:6803/870188791]' entity='osd.0' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]} : dispatch 2024-09-21T03:38:35.717 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:38:35 smithi161 ceph-mon[34533]: from='osd.0 [v2:172.21.15.39:6802/870188791,v1:172.21.15.39:6803/870188791]' entity='osd.0' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]} : dispatch 2024-09-21T03:38:36.266 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-09-21T03:38:36.294 DEBUG:teuthology.orchestra.run.smithi150:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 51116d5c-77ca-11ef-baf5-efdc52797490 -- ceph orch daemon add osd smithi150:vg_nvme/lv_4 2024-09-21T03:38:36.333 INFO:journalctl@ceph.osd.0.smithi039.stdout:Sep 21 03:38:36 smithi039 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-0[47798]: 2024-09-21T03:38:36.297+0000 7f6c9cc14640 -1 osd.0 0 waiting for initial osdmap 2024-09-21T03:38:36.334 INFO:journalctl@ceph.osd.0.smithi039.stdout:Sep 21 03:38:36 smithi039 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-0[47798]: 2024-09-21T03:38:36.303+0000 7f6c98a12640 -1 osd.0 8 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-21T03:38:36.334 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:38:36 smithi039 ceph-mon[31211]: from='osd.0 [v2:172.21.15.39:6802/870188791,v1:172.21.15.39:6803/870188791]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-09-21T03:38:36.334 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:38:36 smithi039 ceph-mon[31211]: osdmap e7: 1 total, 0 up, 1 in 2024-09-21T03:38:36.334 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:38:36 smithi039 ceph-mon[31211]: from='osd.0 [v2:172.21.15.39:6802/870188791,v1:172.21.15.39:6803/870188791]' entity='osd.0' cmd={"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi039", "root=default"]} : dispatch 2024-09-21T03:38:36.334 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:38:36 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-21T03:38:36.334 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:38:36 smithi039 ceph-mon[31211]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:38:36.572 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:38:36 smithi150 ceph-mon[34289]: from='osd.0 [v2:172.21.15.39:6802/870188791,v1:172.21.15.39:6803/870188791]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-09-21T03:38:36.572 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:38:36 smithi150 ceph-mon[34289]: osdmap e7: 1 total, 0 up, 1 in 2024-09-21T03:38:36.572 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:38:36 smithi150 ceph-mon[34289]: from='osd.0 [v2:172.21.15.39:6802/870188791,v1:172.21.15.39:6803/870188791]' entity='osd.0' cmd={"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi039", "root=default"]} : dispatch 2024-09-21T03:38:36.573 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:38:36 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-21T03:38:36.573 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:38:36 smithi150 ceph-mon[34289]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:38:36.585 INFO:teuthology.orchestra.run.smithi150.stderr:Inferring config /var/lib/ceph/51116d5c-77ca-11ef-baf5-efdc52797490/mon.b/config 2024-09-21T03:38:36.619 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:38:36 smithi161 ceph-mon[34533]: from='osd.0 [v2:172.21.15.39:6802/870188791,v1:172.21.15.39:6803/870188791]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-09-21T03:38:36.619 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:38:36 smithi161 ceph-mon[34533]: osdmap e7: 1 total, 0 up, 1 in 2024-09-21T03:38:36.619 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:38:36 smithi161 ceph-mon[34533]: from='osd.0 [v2:172.21.15.39:6802/870188791,v1:172.21.15.39:6803/870188791]' entity='osd.0' cmd={"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi039", "root=default"]} : dispatch 2024-09-21T03:38:36.619 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:38:36 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-21T03:38:36.619 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:38:36 smithi161 ceph-mon[34533]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:38:37.055 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T03:38:37.053+0000 7fe59d9dc640 1 -- 172.21.15.150:0/3689207764 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe598102ae0 msgr2=0x7fe598102ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:38:37.056 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T03:38:37.053+0000 7fe59d9dc640 1 --2- 172.21.15.150:0/3689207764 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe598102ae0 0x7fe598102ef0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fe590009900 tx=0x7fe59002a410 comp rx=0 tx=0).stop 2024-09-21T03:38:37.056 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T03:38:37.053+0000 7fe59d9dc640 1 -- 172.21.15.150:0/3689207764 shutdown_connections 2024-09-21T03:38:37.056 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T03:38:37.053+0000 7fe59d9dc640 1 --2- 172.21.15.150:0/3689207764 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe598108d20 0x7fe59810f550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:37.056 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T03:38:37.053+0000 7fe59d9dc640 1 --2- 172.21.15.150:0/3689207764 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe5981083d0 0x7fe5981087e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:37.056 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T03:38:37.053+0000 7fe59d9dc640 1 --2- 172.21.15.150:0/3689207764 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe598102ae0 0x7fe598102ef0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:37.056 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T03:38:37.053+0000 7fe59d9dc640 1 -- 172.21.15.150:0/3689207764 >> 172.21.15.150:0/3689207764 conn(0x7fe598079790 msgr2=0x7fe598079ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:38:37.056 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T03:38:37.053+0000 7fe59d9dc640 1 -- 172.21.15.150:0/3689207764 shutdown_connections 2024-09-21T03:38:37.056 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T03:38:37.053+0000 7fe59d9dc640 1 -- 172.21.15.150:0/3689207764 wait complete. 2024-09-21T03:38:37.058 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T03:38:37.054+0000 7fe59d9dc640 1 Processor -- start 2024-09-21T03:38:37.058 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T03:38:37.054+0000 7fe59d9dc640 1 -- start start 2024-09-21T03:38:37.058 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T03:38:37.055+0000 7fe59d9dc640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe598102ae0 0x7fe59819a5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:38:37.058 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T03:38:37.055+0000 7fe59d9dc640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe5981083d0 0x7fe59819ab80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:38:37.058 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T03:38:37.055+0000 7fe59d9dc640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe598108d20 0x7fe59819fe20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:38:37.058 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T03:38:37.055+0000 7fe59d9dc640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fe598113b30 con 0x7fe598108d20 2024-09-21T03:38:37.059 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T03:38:37.055+0000 7fe59d9dc640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fe5981139b0 con 0x7fe598102ae0 2024-09-21T03:38:37.059 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T03:38:37.055+0000 7fe59d9dc640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fe598113cb0 con 0x7fe5981083d0 2024-09-21T03:38:37.059 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T03:38:37.055+0000 7fe596ffd640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe598102ae0 0x7fe59819a5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:38:37.059 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T03:38:37.055+0000 7fe596ffd640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe598102ae0 0x7fe59819a5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:56148/0 (socket says 172.21.15.150:56148) 2024-09-21T03:38:37.059 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T03:38:37.055+0000 7fe596ffd640 1 -- 172.21.15.150:0/1825317219 learned_addr learned my addr 172.21.15.150:0/1825317219 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-09-21T03:38:37.059 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T03:38:37.055+0000 7fe5967fc640 1 --2- 172.21.15.150:0/1825317219 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe5981083d0 0x7fe59819ab80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:38:37.059 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T03:38:37.055+0000 7fe5977fe640 1 --2- 172.21.15.150:0/1825317219 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe598108d20 0x7fe59819fe20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:38:37.059 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T03:38:37.055+0000 7fe596ffd640 1 -- 172.21.15.150:0/1825317219 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe5981083d0 msgr2=0x7fe59819ab80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:38:37.059 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T03:38:37.055+0000 7fe596ffd640 1 --2- 172.21.15.150:0/1825317219 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe5981083d0 0x7fe59819ab80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:37.059 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T03:38:37.055+0000 7fe596ffd640 1 -- 172.21.15.150:0/1825317219 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe598108d20 msgr2=0x7fe59819fe20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:38:37.059 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T03:38:37.055+0000 7fe596ffd640 1 --2- 172.21.15.150:0/1825317219 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe598108d20 0x7fe59819fe20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:37.059 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T03:38:37.055+0000 7fe596ffd640 1 -- 172.21.15.150:0/1825317219 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe5981a0550 con 0x7fe598102ae0 2024-09-21T03:38:37.059 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T03:38:37.056+0000 7fe596ffd640 1 --2- 172.21.15.150:0/1825317219 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe598102ae0 0x7fe59819a5e0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fe590005ab0 tx=0x7fe590003cd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:38:37.060 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T03:38:37.056+0000 7fe59c9da640 1 -- 172.21.15.150:0/1825317219 <== mon.2 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe59003f070 con 0x7fe598102ae0 2024-09-21T03:38:37.060 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T03:38:37.056+0000 7fe59c9da640 1 -- 172.21.15.150:0/1825317219 <== mon.2 v2:172.21.15.150:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe590031d60 con 0x7fe598102ae0 2024-09-21T03:38:37.060 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T03:38:37.056+0000 7fe59d9dc640 1 -- 172.21.15.150:0/1825317219 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe5981a07e0 con 0x7fe598102ae0 2024-09-21T03:38:37.060 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T03:38:37.056+0000 7fe59c9da640 1 -- 172.21.15.150:0/1825317219 <== mon.2 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe590035030 con 0x7fe598102ae0 2024-09-21T03:38:37.060 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T03:38:37.056+0000 7fe59d9dc640 1 -- 172.21.15.150:0/1825317219 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe5981a6fa0 con 0x7fe598102ae0 2024-09-21T03:38:37.060 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T03:38:37.057+0000 7fe59d9dc640 1 -- 172.21.15.150:0/1825317219 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe598103d10 con 0x7fe598102ae0 2024-09-21T03:38:37.060 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T03:38:37.059+0000 7fe59c9da640 1 -- 172.21.15.150:0/1825317219 <== mon.2 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 20) ==== 154473+0+0 (secure 0 0 0) 0x7fe590042070 con 0x7fe598102ae0 2024-09-21T03:38:37.061 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T03:38:37.059+0000 7fe59c9da640 1 --2- 172.21.15.150:0/1825317219 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7fe5800b6550 0x7fe5800b8a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:38:37.061 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T03:38:37.060+0000 7fe59c9da640 1 -- 172.21.15.150:0/1825317219 <== mon.2 v2:172.21.15.150:3300/0 5 ==== osd_map(8..8 src has 1..8) ==== 1702+0+0 (secure 0 0 0) 0x7fe590106700 con 0x7fe598102ae0 2024-09-21T03:38:37.061 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T03:38:37.060+0000 7fe5967fc640 1 --2- 172.21.15.150:0/1825317219 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7fe5800b6550 0x7fe5800b8a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:38:37.062 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T03:38:37.060+0000 7fe5967fc640 1 --2- 172.21.15.150:0/1825317219 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7fe5800b6550 0x7fe5800b8a60 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fe598079cb0 tx=0x7fe58c009680 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:38:37.069 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T03:38:37.067+0000 7fe59c9da640 1 -- 172.21.15.150:0/1825317219 <== mon.2 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fe5900cea40 con 0x7fe598102ae0 2024-09-21T03:38:37.263 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T03:38:37.260+0000 7fe59d9dc640 1 -- 172.21.15.150:0/1825317219 --> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi150:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7fe598063800 con 0x7fe5800b6550 2024-09-21T03:38:37.579 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:38:37 smithi150 ceph-mon[34289]: from='osd.0 [v2:172.21.15.39:6802/870188791,v1:172.21.15.39:6803/870188791]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi039", "root=default"]}]': finished 2024-09-21T03:38:37.579 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:38:37 smithi150 ceph-mon[34289]: osdmap e8: 1 total, 0 up, 1 in 2024-09-21T03:38:37.579 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:38:37 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-21T03:38:37.580 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:38:37 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-21T03:38:37.580 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:38:37 smithi150 ceph-mon[34289]: Detected new or changed devices on smithi039 2024-09-21T03:38:37.580 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:38:37 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:38:37.580 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:38:37 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:38:37.580 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:38:37 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-21T03:38:37.580 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:38:37 smithi150 ceph-mon[34289]: Adjusting osd_memory_target on smithi039 to 17088M 2024-09-21T03:38:37.580 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:38:37 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:38:37.580 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:38:37 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:38:37.580 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:38:37 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:38:37.580 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:38:37 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:38:37.580 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:38:37 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T03:38:37.580 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:38:37 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T03:38:37.580 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:38:37 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:38:37.584 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:38:37 smithi039 ceph-mon[31211]: from='osd.0 [v2:172.21.15.39:6802/870188791,v1:172.21.15.39:6803/870188791]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi039", "root=default"]}]': finished 2024-09-21T03:38:37.584 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:38:37 smithi039 ceph-mon[31211]: osdmap e8: 1 total, 0 up, 1 in 2024-09-21T03:38:37.584 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:38:37 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-21T03:38:37.584 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:38:37 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-21T03:38:37.584 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:38:37 smithi039 ceph-mon[31211]: Detected new or changed devices on smithi039 2024-09-21T03:38:37.584 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:38:37 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:38:37.584 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:38:37 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:38:37.584 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:38:37 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-21T03:38:37.584 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:38:37 smithi039 ceph-mon[31211]: Adjusting osd_memory_target on smithi039 to 17088M 2024-09-21T03:38:37.584 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:38:37 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:38:37.584 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:38:37 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:38:37.584 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:38:37 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:38:37.584 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:38:37 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:38:37.585 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:38:37 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T03:38:37.585 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:38:37 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T03:38:37.585 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:38:37 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:38:37.718 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:38:37 smithi161 ceph-mon[34533]: from='osd.0 [v2:172.21.15.39:6802/870188791,v1:172.21.15.39:6803/870188791]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi039", "root=default"]}]': finished 2024-09-21T03:38:37.718 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:38:37 smithi161 ceph-mon[34533]: osdmap e8: 1 total, 0 up, 1 in 2024-09-21T03:38:37.718 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:38:37 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-21T03:38:37.718 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:38:37 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-21T03:38:37.718 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:38:37 smithi161 ceph-mon[34533]: Detected new or changed devices on smithi039 2024-09-21T03:38:37.718 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:38:37 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:38:37.718 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:38:37 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:38:37.718 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:38:37 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-21T03:38:37.718 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:38:37 smithi161 ceph-mon[34533]: Adjusting osd_memory_target on smithi039 to 17088M 2024-09-21T03:38:37.719 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:38:37 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:38:37.719 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:38:37 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:38:37.719 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:38:37 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:38:37.719 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:38:37 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:38:37.719 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:38:37 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T03:38:37.719 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:38:37 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T03:38:37.719 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:38:37 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:38:38.573 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:38:38 smithi150 ceph-mon[34289]: purged_snaps scrub starts 2024-09-21T03:38:38.574 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:38:38 smithi150 ceph-mon[34289]: purged_snaps scrub ok 2024-09-21T03:38:38.574 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:38:38 smithi150 ceph-mon[34289]: from='client.24308 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi150:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:38:38.574 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:38:38 smithi150 ceph-mon[34289]: osd.0 [v2:172.21.15.39:6802/870188791,v1:172.21.15.39:6803/870188791] boot 2024-09-21T03:38:38.574 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:38:38 smithi150 ceph-mon[34289]: osdmap e9: 1 total, 1 up, 1 in 2024-09-21T03:38:38.574 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:38:38 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-21T03:38:38.574 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:38:38 smithi150 ceph-mon[34289]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:38:38.584 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:38:38 smithi039 ceph-mon[31211]: purged_snaps scrub starts 2024-09-21T03:38:38.584 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:38:38 smithi039 ceph-mon[31211]: purged_snaps scrub ok 2024-09-21T03:38:38.584 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:38:38 smithi039 ceph-mon[31211]: from='client.24308 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi150:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:38:38.584 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:38:38 smithi039 ceph-mon[31211]: osd.0 [v2:172.21.15.39:6802/870188791,v1:172.21.15.39:6803/870188791] boot 2024-09-21T03:38:38.584 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:38:38 smithi039 ceph-mon[31211]: osdmap e9: 1 total, 1 up, 1 in 2024-09-21T03:38:38.584 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:38:38 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-21T03:38:38.584 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:38:38 smithi039 ceph-mon[31211]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:38:38.718 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:38:38 smithi161 ceph-mon[34533]: purged_snaps scrub starts 2024-09-21T03:38:38.718 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:38:38 smithi161 ceph-mon[34533]: purged_snaps scrub ok 2024-09-21T03:38:38.718 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:38:38 smithi161 ceph-mon[34533]: from='client.24308 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi150:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:38:38.718 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:38:38 smithi161 ceph-mon[34533]: osd.0 [v2:172.21.15.39:6802/870188791,v1:172.21.15.39:6803/870188791] boot 2024-09-21T03:38:38.718 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:38:38 smithi161 ceph-mon[34533]: osdmap e9: 1 total, 1 up, 1 in 2024-09-21T03:38:38.718 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:38:38 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-21T03:38:38.718 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:38:38 smithi161 ceph-mon[34533]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:38:39.474 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:38:39 smithi150 ceph-mon[34289]: osdmap e10: 1 total, 1 up, 1 in 2024-09-21T03:38:39.583 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:38:39 smithi039 ceph-mon[31211]: osdmap e10: 1 total, 1 up, 1 in 2024-09-21T03:38:39.717 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:38:39 smithi161 ceph-mon[34533]: osdmap e10: 1 total, 1 up, 1 in 2024-09-21T03:38:40.474 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:38:40 smithi150 ceph-mon[34289]: pgmap v35: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-21T03:38:40.474 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:38:40 smithi150 ceph-mon[34289]: from='client.? 172.21.15.150:0/1910356315' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "7cb5db90-28e8-4a30-bedb-c039687d48b0"} : dispatch 2024-09-21T03:38:40.475 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:38:40 smithi150 ceph-mon[34289]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "7cb5db90-28e8-4a30-bedb-c039687d48b0"} : dispatch 2024-09-21T03:38:40.475 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:38:40 smithi150 ceph-mon[34289]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "7cb5db90-28e8-4a30-bedb-c039687d48b0"}]': finished 2024-09-21T03:38:40.475 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:38:40 smithi150 ceph-mon[34289]: osdmap e11: 2 total, 1 up, 2 in 2024-09-21T03:38:40.475 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:38:40 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-21T03:38:40.475 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:38:40 smithi150 ceph-mon[34289]: from='client.? 172.21.15.150:0/1564289643' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T03:38:40.583 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:38:40 smithi039 ceph-mon[31211]: pgmap v35: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-21T03:38:40.584 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:38:40 smithi039 ceph-mon[31211]: from='client.? 172.21.15.150:0/1910356315' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "7cb5db90-28e8-4a30-bedb-c039687d48b0"} : dispatch 2024-09-21T03:38:40.584 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:38:40 smithi039 ceph-mon[31211]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "7cb5db90-28e8-4a30-bedb-c039687d48b0"} : dispatch 2024-09-21T03:38:40.584 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:38:40 smithi039 ceph-mon[31211]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "7cb5db90-28e8-4a30-bedb-c039687d48b0"}]': finished 2024-09-21T03:38:40.584 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:38:40 smithi039 ceph-mon[31211]: osdmap e11: 2 total, 1 up, 2 in 2024-09-21T03:38:40.584 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:38:40 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-21T03:38:40.584 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:38:40 smithi039 ceph-mon[31211]: from='client.? 172.21.15.150:0/1564289643' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T03:38:40.718 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:38:40 smithi161 ceph-mon[34533]: pgmap v35: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-21T03:38:40.718 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:38:40 smithi161 ceph-mon[34533]: from='client.? 172.21.15.150:0/1910356315' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "7cb5db90-28e8-4a30-bedb-c039687d48b0"} : dispatch 2024-09-21T03:38:40.718 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:38:40 smithi161 ceph-mon[34533]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "7cb5db90-28e8-4a30-bedb-c039687d48b0"} : dispatch 2024-09-21T03:38:40.718 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:38:40 smithi161 ceph-mon[34533]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "7cb5db90-28e8-4a30-bedb-c039687d48b0"}]': finished 2024-09-21T03:38:40.718 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:38:40 smithi161 ceph-mon[34533]: osdmap e11: 2 total, 1 up, 2 in 2024-09-21T03:38:40.718 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:38:40 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-21T03:38:40.718 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:38:40 smithi161 ceph-mon[34533]: from='client.? 172.21.15.150:0/1564289643' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T03:38:42.724 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:38:42 smithi150 ceph-mon[34289]: pgmap v37: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-21T03:38:42.833 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:38:42 smithi039 ceph-mon[31211]: pgmap v37: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-21T03:38:42.967 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:38:42 smithi161 ceph-mon[34533]: pgmap v37: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-21T03:38:44.724 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:38:44 smithi150 ceph-mon[34289]: pgmap v38: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-21T03:38:44.833 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:38:44 smithi039 ceph-mon[31211]: pgmap v38: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-21T03:38:44.967 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:38:44 smithi161 ceph-mon[34533]: pgmap v38: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-21T03:38:46.821 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:38:46 smithi150 ceph-mon[34289]: pgmap v39: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-21T03:38:46.834 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:38:46 smithi039 ceph-mon[31211]: pgmap v39: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-21T03:38:46.967 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:38:46 smithi161 ceph-mon[34533]: pgmap v39: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-21T03:38:48.724 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:38:48 smithi150 ceph-mon[34289]: pgmap v40: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-21T03:38:48.725 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:38:48 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.1"} : dispatch 2024-09-21T03:38:48.725 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:38:48 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:38:48.725 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:38:48 smithi150 ceph-mon[34289]: Deploying daemon osd.1 on smithi150 2024-09-21T03:38:48.834 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:38:48 smithi039 ceph-mon[31211]: pgmap v40: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-21T03:38:48.834 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:38:48 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.1"} : dispatch 2024-09-21T03:38:48.834 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:38:48 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:38:48.834 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:38:48 smithi039 ceph-mon[31211]: Deploying daemon osd.1 on smithi150 2024-09-21T03:38:48.967 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:38:48 smithi161 ceph-mon[34533]: pgmap v40: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-21T03:38:48.968 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:38:48 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.1"} : dispatch 2024-09-21T03:38:48.968 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:38:48 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:38:48.968 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:38:48 smithi161 ceph-mon[34533]: Deploying daemon osd.1 on smithi150 2024-09-21T03:38:50.583 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:38:50 smithi039 ceph-mon[31211]: pgmap v41: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-21T03:38:50.656 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:38:50 smithi150 ceph-mon[34289]: pgmap v41: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-21T03:38:50.967 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:38:50 smithi161 ceph-mon[34533]: pgmap v41: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-21T03:38:52.833 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:38:52 smithi039 ceph-mon[31211]: pgmap v42: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-21T03:38:52.931 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:38:52 smithi150 ceph-mon[34289]: pgmap v42: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-21T03:38:52.967 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:38:52 smithi161 ceph-mon[34533]: pgmap v42: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-21T03:38:54.718 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:38:54 smithi161 ceph-mon[34533]: pgmap v43: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-21T03:38:54.718 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:38:54 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:38:54.718 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:38:54 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:38:54.718 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:38:54 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:38:54.790 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:38:54 smithi150 ceph-mon[34289]: pgmap v43: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-21T03:38:54.790 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:38:54 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:38:54.791 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:38:54 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:38:54.791 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:38:54 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:38:54.833 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:38:54 smithi039 ceph-mon[31211]: pgmap v43: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-21T03:38:54.834 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:38:54 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:38:54.834 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:38:54 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:38:54.834 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:38:54 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:38:56.724 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:38:56 smithi150 ceph-mon[34289]: pgmap v44: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-21T03:38:56.833 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:38:56 smithi039 ceph-mon[31211]: pgmap v44: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-21T03:38:56.967 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:38:56 smithi161 ceph-mon[34533]: pgmap v44: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-21T03:38:58.218 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:38:57 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:38:58.218 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:38:57 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:38:58.218 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:38:57 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:38:58.218 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:38:57 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:38:58.218 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:38:57 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:38:58.224 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:38:57 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:38:58.224 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:38:57 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:38:58.224 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:38:57 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:38:58.224 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:38:57 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:38:58.225 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:38:57 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:38:58.333 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:38:57 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:38:58.334 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:38:57 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:38:58.334 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:38:57 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:38:58.334 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:38:57 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:38:58.334 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:38:57 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:38:58.575 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T03:38:58.573+0000 7fe59c9da640 1 -- 172.21.15.150:0/1825317219 <== mgr.14223 v2:172.21.15.39:6800/1696381982 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fe598063800 con 0x7fe5800b6550 2024-09-21T03:38:58.575 INFO:teuthology.orchestra.run.smithi150.stdout:Created osd(s) 1 on host 'smithi150' 2024-09-21T03:38:58.578 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T03:38:58.576+0000 7fe59d9dc640 1 -- 172.21.15.150:0/1825317219 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7fe5800b6550 msgr2=0x7fe5800b8a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:38:58.579 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T03:38:58.576+0000 7fe59d9dc640 1 --2- 172.21.15.150:0/1825317219 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7fe5800b6550 0x7fe5800b8a60 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fe598079cb0 tx=0x7fe58c009680 comp rx=0 tx=0).stop 2024-09-21T03:38:58.579 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T03:38:58.576+0000 7fe59d9dc640 1 -- 172.21.15.150:0/1825317219 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe598102ae0 msgr2=0x7fe59819a5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:38:58.579 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T03:38:58.576+0000 7fe59d9dc640 1 --2- 172.21.15.150:0/1825317219 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe598102ae0 0x7fe59819a5e0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fe590005ab0 tx=0x7fe590003cd0 comp rx=0 tx=0).stop 2024-09-21T03:38:58.579 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T03:38:58.577+0000 7fe59d9dc640 1 -- 172.21.15.150:0/1825317219 shutdown_connections 2024-09-21T03:38:58.579 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T03:38:58.577+0000 7fe59d9dc640 1 --2- 172.21.15.150:0/1825317219 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7fe5800b6550 0x7fe5800b8a60 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:58.579 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T03:38:58.577+0000 7fe59d9dc640 1 --2- 172.21.15.150:0/1825317219 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe598108d20 0x7fe59819fe20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:58.579 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T03:38:58.577+0000 7fe59d9dc640 1 --2- 172.21.15.150:0/1825317219 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe5981083d0 0x7fe59819ab80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:58.579 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T03:38:58.577+0000 7fe59d9dc640 1 --2- 172.21.15.150:0/1825317219 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe598102ae0 0x7fe59819a5e0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:58.579 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T03:38:58.577+0000 7fe59d9dc640 1 -- 172.21.15.150:0/1825317219 >> 172.21.15.150:0/1825317219 conn(0x7fe598079790 msgr2=0x7fe598106520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:38:58.579 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T03:38:58.577+0000 7fe59d9dc640 1 -- 172.21.15.150:0/1825317219 shutdown_connections 2024-09-21T03:38:58.579 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T03:38:58.577+0000 7fe59d9dc640 1 -- 172.21.15.150:0/1825317219 wait complete. 2024-09-21T03:38:59.197 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:38:58 smithi150 ceph-mon[34289]: pgmap v45: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-21T03:38:59.198 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:38:58 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:38:59.198 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:38:58 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:38:59.198 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:38:58 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:38:59.230 DEBUG:teuthology.orchestra.run.smithi150:osd.1> sudo journalctl -f -n 0 -u ceph-51116d5c-77ca-11ef-baf5-efdc52797490@osd.1.service 2024-09-21T03:38:59.233 INFO:tasks.cephadm:Deploying osd.2 on smithi161 with /dev/vg_nvme/lv_4... 2024-09-21T03:38:59.233 DEBUG:teuthology.orchestra.run.smithi161:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 51116d5c-77ca-11ef-baf5-efdc52797490 -- lvm zap /dev/vg_nvme/lv_4 2024-09-21T03:38:59.260 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:38:58 smithi161 ceph-mon[34533]: pgmap v45: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-21T03:38:59.260 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:38:58 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:38:59.260 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:38:58 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:38:59.260 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:38:58 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:38:59.333 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:38:58 smithi039 ceph-mon[31211]: pgmap v45: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-21T03:38:59.334 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:38:58 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:38:59.334 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:38:58 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:38:59.334 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:38:58 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:38:59.484 INFO:teuthology.orchestra.run.smithi161.stderr:Inferring config /var/lib/ceph/51116d5c-77ca-11ef-baf5-efdc52797490/mon.c/config 2024-09-21T03:39:00.634 INFO:journalctl@ceph.osd.1.smithi150.stdout:Sep 21 03:39:00 smithi150 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-1[38911]: 2024-09-21T03:39:00.451+0000 7fea1c452640 -1 osd.1 0 log_to_monitors true 2024-09-21T03:39:01.218 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:39:00 smithi161 ceph-mon[34533]: pgmap v46: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-21T03:39:01.218 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:39:00 smithi161 ceph-mon[34533]: from='osd.1 [v2:172.21.15.150:6800/172427832,v1:172.21.15.150:6801/172427832]' entity='osd.1' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]} : dispatch 2024-09-21T03:39:01.218 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:39:00 smithi161 ceph-mon[34533]: from='osd.1 ' entity='osd.1' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]} : dispatch 2024-09-21T03:39:01.334 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:39:00 smithi039 ceph-mon[31211]: pgmap v46: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-21T03:39:01.334 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:39:00 smithi039 ceph-mon[31211]: from='osd.1 [v2:172.21.15.150:6800/172427832,v1:172.21.15.150:6801/172427832]' entity='osd.1' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]} : dispatch 2024-09-21T03:39:01.334 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:39:00 smithi039 ceph-mon[31211]: from='osd.1 ' entity='osd.1' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]} : dispatch 2024-09-21T03:39:01.343 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:39:00 smithi150 ceph-mon[34289]: pgmap v46: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-21T03:39:01.344 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:39:00 smithi150 ceph-mon[34289]: from='osd.1 [v2:172.21.15.150:6800/172427832,v1:172.21.15.150:6801/172427832]' entity='osd.1' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]} : dispatch 2024-09-21T03:39:01.344 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:39:00 smithi150 ceph-mon[34289]: from='osd.1 ' entity='osd.1' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]} : dispatch 2024-09-21T03:39:01.954 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-09-21T03:39:01.990 DEBUG:teuthology.orchestra.run.smithi161:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 51116d5c-77ca-11ef-baf5-efdc52797490 -- ceph orch daemon add osd smithi161:vg_nvme/lv_4 2024-09-21T03:39:02.218 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:39:01 smithi161 ceph-mon[34533]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-09-21T03:39:02.218 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:39:01 smithi161 ceph-mon[34533]: from='osd.1 [v2:172.21.15.150:6800/172427832,v1:172.21.15.150:6801/172427832]' entity='osd.1' cmd={"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi150", "root=default"]} : dispatch 2024-09-21T03:39:02.218 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:39:01 smithi161 ceph-mon[34533]: osdmap e12: 2 total, 1 up, 2 in 2024-09-21T03:39:02.218 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:39:01 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-21T03:39:02.218 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:39:01 smithi161 ceph-mon[34533]: from='osd.1 ' entity='osd.1' cmd={"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi150", "root=default"]} : dispatch 2024-09-21T03:39:02.218 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:39:01 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:39:02.218 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:39:01 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:39:02.218 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:39:01 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-09-21T03:39:02.218 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:39:01 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:39:02.218 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:39:01 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:39:02.219 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:39:01 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:39:02.219 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:39:01 smithi161 ceph-mon[34533]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi150", "root=default"]}]': finished 2024-09-21T03:39:02.224 INFO:journalctl@ceph.osd.1.smithi150.stdout:Sep 21 03:39:01 smithi150 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-1[38911]: 2024-09-21T03:39:01.992+0000 7fea183d4640 -1 osd.1 0 waiting for initial osdmap 2024-09-21T03:39:02.225 INFO:journalctl@ceph.osd.1.smithi150.stdout:Sep 21 03:39:02 smithi150 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-1[38911]: 2024-09-21T03:39:02.005+0000 7fea141d2640 -1 osd.1 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-21T03:39:02.225 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:39:02 smithi150 ceph-mon[34289]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-09-21T03:39:02.225 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:39:02 smithi150 ceph-mon[34289]: from='osd.1 [v2:172.21.15.150:6800/172427832,v1:172.21.15.150:6801/172427832]' entity='osd.1' cmd={"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi150", "root=default"]} : dispatch 2024-09-21T03:39:02.225 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:39:02 smithi150 ceph-mon[34289]: osdmap e12: 2 total, 1 up, 2 in 2024-09-21T03:39:02.225 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:39:02 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-21T03:39:02.225 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:39:02 smithi150 ceph-mon[34289]: from='osd.1 ' entity='osd.1' cmd={"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi150", "root=default"]} : dispatch 2024-09-21T03:39:02.225 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:39:02 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:39:02.225 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:39:02 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:39:02.225 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:39:02 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-09-21T03:39:02.225 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:39:02 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:39:02.225 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:39:02 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:39:02.225 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:39:02 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:39:02.226 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:39:02 smithi150 ceph-mon[34289]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi150", "root=default"]}]': finished 2024-09-21T03:39:02.281 INFO:teuthology.orchestra.run.smithi161.stderr:Inferring config /var/lib/ceph/51116d5c-77ca-11ef-baf5-efdc52797490/mon.c/config 2024-09-21T03:39:02.334 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:39:01 smithi039 ceph-mon[31211]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-09-21T03:39:02.334 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:39:01 smithi039 ceph-mon[31211]: from='osd.1 [v2:172.21.15.150:6800/172427832,v1:172.21.15.150:6801/172427832]' entity='osd.1' cmd={"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi150", "root=default"]} : dispatch 2024-09-21T03:39:02.334 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:39:01 smithi039 ceph-mon[31211]: osdmap e12: 2 total, 1 up, 2 in 2024-09-21T03:39:02.334 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:39:01 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-21T03:39:02.334 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:39:01 smithi039 ceph-mon[31211]: from='osd.1 ' entity='osd.1' cmd={"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi150", "root=default"]} : dispatch 2024-09-21T03:39:02.334 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:39:01 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:39:02.334 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:39:01 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:39:02.334 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:39:01 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-09-21T03:39:02.334 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:39:01 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:39:02.334 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:39:01 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:39:02.334 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:39:01 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:39:02.334 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:39:01 smithi039 ceph-mon[31211]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi150", "root=default"]}]': finished 2024-09-21T03:39:02.758 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-21T03:39:02.753+0000 7f4a3f506640 1 -- 172.21.15.161:0/3864673280 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4a381083c0 msgr2=0x7f4a381087d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:02.758 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-21T03:39:02.753+0000 7f4a3f506640 1 --2- 172.21.15.161:0/3864673280 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4a381083c0 0x7f4a381087d0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f4a28009900 tx=0x7f4a2802a410 comp rx=0 tx=0).stop 2024-09-21T03:39:02.758 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-21T03:39:02.753+0000 7f4a3f506640 1 -- 172.21.15.161:0/3864673280 shutdown_connections 2024-09-21T03:39:02.758 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-21T03:39:02.753+0000 7f4a3f506640 1 --2- 172.21.15.161:0/3864673280 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4a38108d10 0x7f4a3810f540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:02.758 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-21T03:39:02.753+0000 7f4a3f506640 1 --2- 172.21.15.161:0/3864673280 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4a381083c0 0x7f4a381087d0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:02.758 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-21T03:39:02.753+0000 7f4a3f506640 1 --2- 172.21.15.161:0/3864673280 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4a38102a80 0x7f4a38102e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:02.758 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-21T03:39:02.753+0000 7f4a3f506640 1 -- 172.21.15.161:0/3864673280 >> 172.21.15.161:0/3864673280 conn(0x7f4a38079740 msgr2=0x7f4a38079b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:39:02.758 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-21T03:39:02.753+0000 7f4a3f506640 1 -- 172.21.15.161:0/3864673280 shutdown_connections 2024-09-21T03:39:02.758 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-21T03:39:02.753+0000 7f4a3f506640 1 -- 172.21.15.161:0/3864673280 wait complete. 2024-09-21T03:39:02.758 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-21T03:39:02.754+0000 7f4a3f506640 1 Processor -- start 2024-09-21T03:39:02.758 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-21T03:39:02.754+0000 7f4a3f506640 1 -- start start 2024-09-21T03:39:02.759 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-21T03:39:02.754+0000 7f4a3f506640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4a38102a80 0x7f4a3810ffd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:02.759 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-21T03:39:02.754+0000 7f4a3f506640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4a381083c0 0x7f4a38110510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:02.759 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-21T03:39:02.754+0000 7f4a3f506640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4a38108d10 0x7f4a38110a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:02.759 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-21T03:39:02.754+0000 7f4a3f506640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f4a38113ad0 con 0x7f4a38102a80 2024-09-21T03:39:02.759 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-21T03:39:02.754+0000 7f4a3f506640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f4a38113950 con 0x7f4a381083c0 2024-09-21T03:39:02.759 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-21T03:39:02.754+0000 7f4a3f506640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f4a38113c50 con 0x7f4a38108d10 2024-09-21T03:39:02.759 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-21T03:39:02.754+0000 7f4a3daa3640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4a38108d10 0x7f4a38110a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:02.759 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-21T03:39:02.754+0000 7f4a3daa3640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4a38108d10 0x7f4a38110a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:57066/0 (socket says 172.21.15.161:57066) 2024-09-21T03:39:02.759 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-21T03:39:02.754+0000 7f4a3daa3640 1 -- 172.21.15.161:0/3535015558 learned_addr learned my addr 172.21.15.161:0/3535015558 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-09-21T03:39:02.759 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-21T03:39:02.755+0000 7f4a3d2a2640 1 --2- 172.21.15.161:0/3535015558 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4a38102a80 0x7f4a3810ffd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:02.759 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-21T03:39:02.755+0000 7f4a3caa1640 1 --2- 172.21.15.161:0/3535015558 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4a381083c0 0x7f4a38110510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:02.759 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-21T03:39:02.755+0000 7f4a3daa3640 1 -- 172.21.15.161:0/3535015558 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4a381083c0 msgr2=0x7f4a38110510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:02.759 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-21T03:39:02.755+0000 7f4a3daa3640 1 --2- 172.21.15.161:0/3535015558 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4a381083c0 0x7f4a38110510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:02.760 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-21T03:39:02.755+0000 7f4a3daa3640 1 -- 172.21.15.161:0/3535015558 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4a38102a80 msgr2=0x7f4a3810ffd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:02.760 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-21T03:39:02.755+0000 7f4a3daa3640 1 --2- 172.21.15.161:0/3535015558 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4a38102a80 0x7f4a3810ffd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:02.760 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-21T03:39:02.755+0000 7f4a3daa3640 1 -- 172.21.15.161:0/3535015558 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4a381a8780 con 0x7f4a38108d10 2024-09-21T03:39:02.760 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-21T03:39:02.755+0000 7f4a3daa3640 1 --2- 172.21.15.161:0/3535015558 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4a38108d10 0x7f4a38110a50 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f4a3000d750 tx=0x7f4a3000dc10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:39:02.760 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-21T03:39:02.755+0000 7f4a2e7fc640 1 -- 172.21.15.161:0/3535015558 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a30017070 con 0x7f4a38108d10 2024-09-21T03:39:02.760 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-21T03:39:02.755+0000 7f4a2e7fc640 1 -- 172.21.15.161:0/3535015558 <== mon.1 v2:172.21.15.161:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4a300054d0 con 0x7f4a38108d10 2024-09-21T03:39:02.760 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-21T03:39:02.755+0000 7f4a3f506640 1 -- 172.21.15.161:0/3535015558 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4a381a8a70 con 0x7f4a38108d10 2024-09-21T03:39:02.760 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-21T03:39:02.755+0000 7f4a2e7fc640 1 -- 172.21.15.161:0/3535015558 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a30004c60 con 0x7f4a38108d10 2024-09-21T03:39:02.760 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-21T03:39:02.755+0000 7f4a3f506640 1 -- 172.21.15.161:0/3535015558 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4a381a8f80 con 0x7f4a38108d10 2024-09-21T03:39:02.761 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-21T03:39:02.756+0000 7f4a3f506640 1 -- 172.21.15.161:0/3535015558 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4a38102e90 con 0x7f4a38108d10 2024-09-21T03:39:02.761 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-21T03:39:02.758+0000 7f4a2e7fc640 1 -- 172.21.15.161:0/3535015558 <== mon.1 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 20) ==== 154473+0+0 (secure 0 0 0) 0x7f4a30009bd0 con 0x7f4a38108d10 2024-09-21T03:39:02.761 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-21T03:39:02.758+0000 7f4a2e7fc640 1 --2- 172.21.15.161:0/3535015558 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f4a240b6550 0x7f4a240b8a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:02.761 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-21T03:39:02.759+0000 7f4a2e7fc640 1 -- 172.21.15.161:0/3535015558 <== mon.1 v2:172.21.15.161:3300/0 5 ==== osd_map(13..13 src has 1..13) ==== 2244+0+0 (secure 0 0 0) 0x7f4a300e5740 con 0x7f4a38108d10 2024-09-21T03:39:02.761 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-21T03:39:02.759+0000 7f4a3d2a2640 1 --2- 172.21.15.161:0/3535015558 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f4a240b6550 0x7f4a240b8a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:02.761 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-21T03:39:02.759+0000 7f4a3d2a2640 1 --2- 172.21.15.161:0/3535015558 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f4a240b6550 0x7f4a240b8a60 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f4a38111a20 tx=0x7f4a34009680 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:39:02.763 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-21T03:39:02.762+0000 7f4a2e7fc640 1 -- 172.21.15.161:0/3535015558 <== mon.1 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f4a300b1cc0 con 0x7f4a38108d10 2024-09-21T03:39:02.947 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-21T03:39:02.945+0000 7f4a3f506640 1 -- 172.21.15.161:0/3535015558 --> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi161:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7f4a38063800 con 0x7f4a240b6550 2024-09-21T03:39:03.218 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:39:03 smithi161 ceph-mon[34533]: pgmap v48: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-21T03:39:03.218 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:39:03 smithi161 ceph-mon[34533]: Detected new or changed devices on smithi150 2024-09-21T03:39:03.218 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:39:03 smithi161 ceph-mon[34533]: Adjusting osd_memory_target on smithi150 to 17088M 2024-09-21T03:39:03.218 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:39:03 smithi161 ceph-mon[34533]: osdmap e13: 2 total, 1 up, 2 in 2024-09-21T03:39:03.218 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:39:03 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-21T03:39:03.218 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:39:03 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:39:03.218 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:39:03 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-21T03:39:03.218 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:39:03 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T03:39:03.218 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:39:03 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T03:39:03.218 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:39:03 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:39:03.219 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:39:03 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-21T03:39:03.333 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:39:03 smithi039 ceph-mon[31211]: pgmap v48: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-21T03:39:03.334 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:39:03 smithi039 ceph-mon[31211]: Detected new or changed devices on smithi150 2024-09-21T03:39:03.334 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:39:03 smithi039 ceph-mon[31211]: Adjusting osd_memory_target on smithi150 to 17088M 2024-09-21T03:39:03.334 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:39:03 smithi039 ceph-mon[31211]: osdmap e13: 2 total, 1 up, 2 in 2024-09-21T03:39:03.334 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:39:03 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-21T03:39:03.334 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:39:03 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:39:03.334 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:39:03 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-21T03:39:03.334 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:39:03 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T03:39:03.334 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:39:03 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T03:39:03.334 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:39:03 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:39:03.334 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:39:03 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-21T03:39:03.474 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:39:03 smithi150 ceph-mon[34289]: pgmap v48: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-21T03:39:03.474 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:39:03 smithi150 ceph-mon[34289]: Detected new or changed devices on smithi150 2024-09-21T03:39:03.475 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:39:03 smithi150 ceph-mon[34289]: Adjusting osd_memory_target on smithi150 to 17088M 2024-09-21T03:39:03.475 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:39:03 smithi150 ceph-mon[34289]: osdmap e13: 2 total, 1 up, 2 in 2024-09-21T03:39:03.475 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:39:03 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-21T03:39:03.475 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:39:03 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:39:03.475 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:39:03 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-21T03:39:03.475 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:39:03 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T03:39:03.475 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:39:03 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T03:39:03.475 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:39:03 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:39:03.475 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:39:03 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-21T03:39:04.300 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:39:04 smithi161 ceph-mon[34533]: purged_snaps scrub starts 2024-09-21T03:39:04.300 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:39:04 smithi161 ceph-mon[34533]: purged_snaps scrub ok 2024-09-21T03:39:04.300 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:39:04 smithi161 ceph-mon[34533]: from='client.24334 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi161:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:39:04.300 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:39:04 smithi161 ceph-mon[34533]: osd.1 [v2:172.21.15.150:6800/172427832,v1:172.21.15.150:6801/172427832] boot 2024-09-21T03:39:04.300 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:39:04 smithi161 ceph-mon[34533]: osdmap e14: 2 total, 2 up, 2 in 2024-09-21T03:39:04.301 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:39:04 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-21T03:39:04.333 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:39:04 smithi039 ceph-mon[31211]: purged_snaps scrub starts 2024-09-21T03:39:04.334 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:39:04 smithi039 ceph-mon[31211]: purged_snaps scrub ok 2024-09-21T03:39:04.334 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:39:04 smithi039 ceph-mon[31211]: from='client.24334 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi161:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:39:04.334 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:39:04 smithi039 ceph-mon[31211]: osd.1 [v2:172.21.15.150:6800/172427832,v1:172.21.15.150:6801/172427832] boot 2024-09-21T03:39:04.334 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:39:04 smithi039 ceph-mon[31211]: osdmap e14: 2 total, 2 up, 2 in 2024-09-21T03:39:04.334 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:39:04 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-21T03:39:04.474 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:39:04 smithi150 ceph-mon[34289]: purged_snaps scrub starts 2024-09-21T03:39:04.475 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:39:04 smithi150 ceph-mon[34289]: purged_snaps scrub ok 2024-09-21T03:39:04.475 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:39:04 smithi150 ceph-mon[34289]: from='client.24334 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi161:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:39:04.475 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:39:04 smithi150 ceph-mon[34289]: osd.1 [v2:172.21.15.150:6800/172427832,v1:172.21.15.150:6801/172427832] boot 2024-09-21T03:39:04.475 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:39:04 smithi150 ceph-mon[34289]: osdmap e14: 2 total, 2 up, 2 in 2024-09-21T03:39:04.475 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:39:04 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-21T03:39:05.217 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:39:05 smithi161 ceph-mon[34533]: pgmap v51: 0 pgs: ; 0 B data, 27 MiB used, 89 GiB / 89 GiB avail 2024-09-21T03:39:05.218 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:39:05 smithi161 ceph-mon[34533]: osdmap e15: 2 total, 2 up, 2 in 2024-09-21T03:39:05.333 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:39:05 smithi039 ceph-mon[31211]: pgmap v51: 0 pgs: ; 0 B data, 27 MiB used, 89 GiB / 89 GiB avail 2024-09-21T03:39:05.333 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:39:05 smithi039 ceph-mon[31211]: osdmap e15: 2 total, 2 up, 2 in 2024-09-21T03:39:05.474 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:39:05 smithi150 ceph-mon[34289]: pgmap v51: 0 pgs: ; 0 B data, 27 MiB used, 89 GiB / 89 GiB avail 2024-09-21T03:39:05.474 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:39:05 smithi150 ceph-mon[34289]: osdmap e15: 2 total, 2 up, 2 in 2024-09-21T03:39:06.217 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:39:06 smithi161 ceph-mon[34533]: from='client.? 172.21.15.161:0/3271467123' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "b25c6ab3-fc95-4751-bd63-6e3fc23e8bd7"} : dispatch 2024-09-21T03:39:06.218 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:39:06 smithi161 ceph-mon[34533]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "b25c6ab3-fc95-4751-bd63-6e3fc23e8bd7"} : dispatch 2024-09-21T03:39:06.218 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:39:06 smithi161 ceph-mon[34533]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "b25c6ab3-fc95-4751-bd63-6e3fc23e8bd7"}]': finished 2024-09-21T03:39:06.218 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:39:06 smithi161 ceph-mon[34533]: osdmap e16: 3 total, 2 up, 3 in 2024-09-21T03:39:06.218 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:39:06 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-21T03:39:06.218 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:39:06 smithi161 ceph-mon[34533]: from='client.? 172.21.15.161:0/4248437316' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T03:39:06.333 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:39:06 smithi039 ceph-mon[31211]: from='client.? 172.21.15.161:0/3271467123' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "b25c6ab3-fc95-4751-bd63-6e3fc23e8bd7"} : dispatch 2024-09-21T03:39:06.334 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:39:06 smithi039 ceph-mon[31211]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "b25c6ab3-fc95-4751-bd63-6e3fc23e8bd7"} : dispatch 2024-09-21T03:39:06.334 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:39:06 smithi039 ceph-mon[31211]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "b25c6ab3-fc95-4751-bd63-6e3fc23e8bd7"}]': finished 2024-09-21T03:39:06.334 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:39:06 smithi039 ceph-mon[31211]: osdmap e16: 3 total, 2 up, 3 in 2024-09-21T03:39:06.334 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:39:06 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-21T03:39:06.334 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:39:06 smithi039 ceph-mon[31211]: from='client.? 172.21.15.161:0/4248437316' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T03:39:06.474 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:39:06 smithi150 ceph-mon[34289]: from='client.? 172.21.15.161:0/3271467123' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "b25c6ab3-fc95-4751-bd63-6e3fc23e8bd7"} : dispatch 2024-09-21T03:39:06.475 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:39:06 smithi150 ceph-mon[34289]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "b25c6ab3-fc95-4751-bd63-6e3fc23e8bd7"} : dispatch 2024-09-21T03:39:06.475 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:39:06 smithi150 ceph-mon[34289]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "b25c6ab3-fc95-4751-bd63-6e3fc23e8bd7"}]': finished 2024-09-21T03:39:06.475 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:39:06 smithi150 ceph-mon[34289]: osdmap e16: 3 total, 2 up, 3 in 2024-09-21T03:39:06.475 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:39:06 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-21T03:39:06.475 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:39:06 smithi150 ceph-mon[34289]: from='client.? 172.21.15.161:0/4248437316' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T03:39:07.333 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:39:07 smithi039 ceph-mon[31211]: pgmap v54: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-21T03:39:07.375 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:39:07 smithi150 ceph-mon[34289]: pgmap v54: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-21T03:39:07.467 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:39:07 smithi161 ceph-mon[34533]: pgmap v54: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-21T03:39:09.333 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:39:09 smithi039 ceph-mon[31211]: pgmap v55: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-21T03:39:09.467 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:39:09 smithi161 ceph-mon[34533]: pgmap v55: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-21T03:39:09.474 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:39:09 smithi150 ceph-mon[34289]: pgmap v55: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-21T03:39:11.333 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:39:11 smithi039 ceph-mon[31211]: pgmap v56: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-21T03:39:11.431 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:39:11 smithi161 ceph-mon[34533]: pgmap v56: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-21T03:39:11.474 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:39:11 smithi150 ceph-mon[34289]: pgmap v56: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-21T03:39:13.333 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:39:13 smithi039 ceph-mon[31211]: pgmap v57: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-21T03:39:13.468 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:39:13 smithi161 ceph-mon[34533]: pgmap v57: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-21T03:39:13.474 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:39:13 smithi150 ceph-mon[34289]: pgmap v57: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-21T03:39:14.300 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:39:14 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.2"} : dispatch 2024-09-21T03:39:14.300 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:39:14 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:39:14.333 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:39:14 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.2"} : dispatch 2024-09-21T03:39:14.334 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:39:14 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:39:14.474 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:39:14 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.2"} : dispatch 2024-09-21T03:39:14.475 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:39:14 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:39:15.333 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:39:15 smithi039 ceph-mon[31211]: pgmap v58: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-21T03:39:15.334 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:39:15 smithi039 ceph-mon[31211]: Deploying daemon osd.2 on smithi161 2024-09-21T03:39:15.406 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:39:15 smithi161 ceph-mon[34533]: pgmap v58: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-21T03:39:15.406 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:39:15 smithi161 ceph-mon[34533]: Deploying daemon osd.2 on smithi161 2024-09-21T03:39:15.474 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:39:15 smithi150 ceph-mon[34289]: pgmap v58: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-21T03:39:15.474 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:39:15 smithi150 ceph-mon[34289]: Deploying daemon osd.2 on smithi161 2024-09-21T03:39:17.164 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:39:17 smithi161 ceph-mon[34533]: pgmap v59: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-21T03:39:17.333 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:39:17 smithi039 ceph-mon[31211]: pgmap v59: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-21T03:39:17.474 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:39:17 smithi150 ceph-mon[34289]: pgmap v59: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-21T03:39:19.209 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:39:19 smithi161 ceph-mon[34533]: pgmap v60: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-21T03:39:19.333 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:39:19 smithi039 ceph-mon[31211]: pgmap v60: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-21T03:39:19.474 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:39:19 smithi150 ceph-mon[34289]: pgmap v60: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-21T03:39:21.334 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:39:21 smithi039 ceph-mon[31211]: pgmap v61: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-21T03:39:21.334 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:39:21 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:39:21.334 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:39:21 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:39:21.334 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:39:21 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:39:21.460 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:39:21 smithi161 ceph-mon[34533]: pgmap v61: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-21T03:39:21.460 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:39:21 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:39:21.461 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:39:21 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:39:21.461 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:39:21 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:39:21.474 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:39:21 smithi150 ceph-mon[34289]: pgmap v61: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-21T03:39:21.474 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:39:21 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:39:21.475 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:39:21 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:39:21.475 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:39:21 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:39:23.334 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:39:23 smithi039 ceph-mon[31211]: pgmap v62: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-21T03:39:23.334 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:39:23 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:39:23.334 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:39:23 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:39:23.334 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:39:23 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:39:23.334 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:39:23 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:39:23.334 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:39:23 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:39:23.467 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:39:23 smithi161 ceph-mon[34533]: pgmap v62: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-21T03:39:23.468 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:39:23 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:39:23.468 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:39:23 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:39:23.468 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:39:23 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:39:23.468 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:39:23 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:39:23.468 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:39:23 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:39:23.474 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:39:23 smithi150 ceph-mon[34289]: pgmap v62: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-21T03:39:23.475 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:39:23 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:39:23.475 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:39:23 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:39:23.475 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:39:23 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:39:23.475 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:39:23 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:39:23.475 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:39:23 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:39:24.491 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-21T03:39:24.488+0000 7f4a2e7fc640 1 -- 172.21.15.161:0/3535015558 <== mgr.14223 v2:172.21.15.39:6800/1696381982 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f4a38063800 con 0x7f4a240b6550 2024-09-21T03:39:24.492 INFO:teuthology.orchestra.run.smithi161.stdout:Created osd(s) 2 on host 'smithi161' 2024-09-21T03:39:24.494 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-21T03:39:24.491+0000 7f4a3f506640 1 -- 172.21.15.161:0/3535015558 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f4a240b6550 msgr2=0x7f4a240b8a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:24.494 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-21T03:39:24.491+0000 7f4a3f506640 1 --2- 172.21.15.161:0/3535015558 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f4a240b6550 0x7f4a240b8a60 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f4a38111a20 tx=0x7f4a34009680 comp rx=0 tx=0).stop 2024-09-21T03:39:24.494 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-21T03:39:24.491+0000 7f4a3f506640 1 -- 172.21.15.161:0/3535015558 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4a38108d10 msgr2=0x7f4a38110a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:24.494 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-21T03:39:24.491+0000 7f4a3f506640 1 --2- 172.21.15.161:0/3535015558 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4a38108d10 0x7f4a38110a50 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f4a3000d750 tx=0x7f4a3000dc10 comp rx=0 tx=0).stop 2024-09-21T03:39:24.494 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-21T03:39:24.491+0000 7f4a3f506640 1 -- 172.21.15.161:0/3535015558 shutdown_connections 2024-09-21T03:39:24.494 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-21T03:39:24.491+0000 7f4a3f506640 1 --2- 172.21.15.161:0/3535015558 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f4a240b6550 0x7f4a240b8a60 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:24.494 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-21T03:39:24.491+0000 7f4a3f506640 1 --2- 172.21.15.161:0/3535015558 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4a38108d10 0x7f4a38110a50 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:24.494 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-21T03:39:24.491+0000 7f4a3f506640 1 --2- 172.21.15.161:0/3535015558 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4a381083c0 0x7f4a38110510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:24.494 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-21T03:39:24.491+0000 7f4a3f506640 1 --2- 172.21.15.161:0/3535015558 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4a38102a80 0x7f4a3810ffd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:24.494 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-21T03:39:24.491+0000 7f4a3f506640 1 -- 172.21.15.161:0/3535015558 >> 172.21.15.161:0/3535015558 conn(0x7f4a38079740 msgr2=0x7f4a381043d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:39:24.495 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-21T03:39:24.491+0000 7f4a3f506640 1 -- 172.21.15.161:0/3535015558 shutdown_connections 2024-09-21T03:39:24.495 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-21T03:39:24.491+0000 7f4a3f506640 1 -- 172.21.15.161:0/3535015558 wait complete. 2024-09-21T03:39:25.111 DEBUG:teuthology.orchestra.run.smithi161:osd.2> sudo journalctl -f -n 0 -u ceph-51116d5c-77ca-11ef-baf5-efdc52797490@osd.2.service 2024-09-21T03:39:25.114 INFO:tasks.cephadm:Deploying osd.3 on smithi039 with /dev/vg_nvme/lv_3... 2024-09-21T03:39:25.114 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 51116d5c-77ca-11ef-baf5-efdc52797490 -- lvm zap /dev/vg_nvme/lv_3 2024-09-21T03:39:25.333 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:39:25 smithi039 ceph-mon[31211]: pgmap v63: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-21T03:39:25.334 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:39:25 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:39:25.334 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:39:25 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:39:25.334 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:39:25 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:39:25.351 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:39:25 smithi161 ceph-mon[34533]: pgmap v63: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-21T03:39:25.351 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:39:25 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:39:25.351 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:39:25 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:39:25.351 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:39:25 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:39:25.384 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/51116d5c-77ca-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:39:25.474 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:39:25 smithi150 ceph-mon[34289]: pgmap v63: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-21T03:39:25.474 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:39:25 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:39:25.475 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:39:25 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:39:25.475 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:39:25 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:39:26.425 INFO:journalctl@ceph.osd.2.smithi161.stdout:Sep 21 03:39:26 smithi161 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-2[39689]: 2024-09-21T03:39:26.090+0000 7fe819cf8640 -1 osd.2 0 log_to_monitors true 2024-09-21T03:39:27.333 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:39:27 smithi039 ceph-mon[31211]: pgmap v64: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-21T03:39:27.334 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:39:27 smithi039 ceph-mon[31211]: from='osd.2 [v2:172.21.15.161:6800/2727876133,v1:172.21.15.161:6801/2727876133]' entity='osd.2' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]} : dispatch 2024-09-21T03:39:27.334 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:39:27 smithi039 ceph-mon[31211]: from='osd.2 ' entity='osd.2' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]} : dispatch 2024-09-21T03:39:27.467 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:39:27 smithi161 ceph-mon[34533]: pgmap v64: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-21T03:39:27.467 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:39:27 smithi161 ceph-mon[34533]: from='osd.2 [v2:172.21.15.161:6800/2727876133,v1:172.21.15.161:6801/2727876133]' entity='osd.2' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]} : dispatch 2024-09-21T03:39:27.467 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:39:27 smithi161 ceph-mon[34533]: from='osd.2 ' entity='osd.2' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]} : dispatch 2024-09-21T03:39:27.474 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:39:27 smithi150 ceph-mon[34289]: pgmap v64: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-21T03:39:27.474 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:39:27 smithi150 ceph-mon[34289]: from='osd.2 [v2:172.21.15.161:6800/2727876133,v1:172.21.15.161:6801/2727876133]' entity='osd.2' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]} : dispatch 2024-09-21T03:39:27.474 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:39:27 smithi150 ceph-mon[34289]: from='osd.2 ' entity='osd.2' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]} : dispatch 2024-09-21T03:39:28.086 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T03:39:28.116 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 51116d5c-77ca-11ef-baf5-efdc52797490 -- ceph orch daemon add osd smithi039:vg_nvme/lv_3 2024-09-21T03:39:28.218 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:39:28 smithi161 ceph-mon[34533]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-09-21T03:39:28.218 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:39:28 smithi161 ceph-mon[34533]: from='osd.2 [v2:172.21.15.161:6800/2727876133,v1:172.21.15.161:6801/2727876133]' entity='osd.2' cmd={"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi161", "root=default"]} : dispatch 2024-09-21T03:39:28.218 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:39:28 smithi161 ceph-mon[34533]: osdmap e17: 3 total, 2 up, 3 in 2024-09-21T03:39:28.218 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:39:28 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-21T03:39:28.219 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:39:28 smithi161 ceph-mon[34533]: from='osd.2 ' entity='osd.2' cmd={"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi161", "root=default"]} : dispatch 2024-09-21T03:39:28.219 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:39:28 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:39:28.219 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:39:28 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:39:28.219 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:39:28 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-09-21T03:39:28.219 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:39:28 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:39:28.219 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:39:28 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:39:28.220 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:39:28 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:39:28.220 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:39:28 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:39:28.220 INFO:journalctl@ceph.osd.2.smithi161.stdout:Sep 21 03:39:28 smithi161 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-2[39689]: 2024-09-21T03:39:28.103+0000 7fe815c7a640 -1 osd.2 0 waiting for initial osdmap 2024-09-21T03:39:28.220 INFO:journalctl@ceph.osd.2.smithi161.stdout:Sep 21 03:39:28 smithi161 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-2[39689]: 2024-09-21T03:39:28.108+0000 7fe811a78640 -1 osd.2 18 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-21T03:39:28.334 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:39:28 smithi039 ceph-mon[31211]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-09-21T03:39:28.334 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:39:28 smithi039 ceph-mon[31211]: from='osd.2 [v2:172.21.15.161:6800/2727876133,v1:172.21.15.161:6801/2727876133]' entity='osd.2' cmd={"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi161", "root=default"]} : dispatch 2024-09-21T03:39:28.334 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:39:28 smithi039 ceph-mon[31211]: osdmap e17: 3 total, 2 up, 3 in 2024-09-21T03:39:28.334 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:39:28 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-21T03:39:28.334 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:39:28 smithi039 ceph-mon[31211]: from='osd.2 ' entity='osd.2' cmd={"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi161", "root=default"]} : dispatch 2024-09-21T03:39:28.334 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:39:28 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:39:28.334 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:39:28 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:39:28.334 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:39:28 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-09-21T03:39:28.335 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:39:28 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:39:28.335 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:39:28 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:39:28.335 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:39:28 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:39:28.335 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:39:28 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:39:28.373 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/51116d5c-77ca-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:39:28.474 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:39:28 smithi150 ceph-mon[34289]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-09-21T03:39:28.475 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:39:28 smithi150 ceph-mon[34289]: from='osd.2 [v2:172.21.15.161:6800/2727876133,v1:172.21.15.161:6801/2727876133]' entity='osd.2' cmd={"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi161", "root=default"]} : dispatch 2024-09-21T03:39:28.475 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:39:28 smithi150 ceph-mon[34289]: osdmap e17: 3 total, 2 up, 3 in 2024-09-21T03:39:28.475 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:39:28 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-21T03:39:28.475 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:39:28 smithi150 ceph-mon[34289]: from='osd.2 ' entity='osd.2' cmd={"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi161", "root=default"]} : dispatch 2024-09-21T03:39:28.475 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:39:28 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:39:28.475 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:39:28 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:39:28.475 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:39:28 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-09-21T03:39:28.475 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:39:28 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:39:28.475 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:39:28 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:39:28.475 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:39:28 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:39:28.475 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:39:28 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:39:28.852 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:39:28.829+0000 7fc8b4470640 1 -- 172.21.15.39:0/1119385385 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc8ac1083d0 msgr2=0x7fc8ac1087e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:28.852 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:39:28.829+0000 7fc8b4470640 1 --2- 172.21.15.39:0/1119385385 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc8ac1083d0 0x7fc8ac1087e0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fc89c009900 tx=0x7fc89c02a410 comp rx=0 tx=0).stop 2024-09-21T03:39:28.852 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:39:28.830+0000 7fc8b4470640 1 -- 172.21.15.39:0/1119385385 shutdown_connections 2024-09-21T03:39:28.853 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:39:28.830+0000 7fc8b4470640 1 --2- 172.21.15.39:0/1119385385 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc8ac108d20 0x7fc8ac10f550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:28.853 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:39:28.830+0000 7fc8b4470640 1 --2- 172.21.15.39:0/1119385385 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc8ac1083d0 0x7fc8ac1087e0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:28.853 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:39:28.830+0000 7fc8b4470640 1 --2- 172.21.15.39:0/1119385385 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc8ac102ae0 0x7fc8ac102ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:28.853 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:39:28.830+0000 7fc8b4470640 1 -- 172.21.15.39:0/1119385385 >> 172.21.15.39:0/1119385385 conn(0x7fc8ac079760 msgr2=0x7fc8ac079b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:39:28.853 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:39:28.830+0000 7fc8b4470640 1 -- 172.21.15.39:0/1119385385 shutdown_connections 2024-09-21T03:39:28.853 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:39:28.830+0000 7fc8b4470640 1 -- 172.21.15.39:0/1119385385 wait complete. 2024-09-21T03:39:28.853 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:39:28.830+0000 7fc8b4470640 1 Processor -- start 2024-09-21T03:39:28.853 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:39:28.830+0000 7fc8b4470640 1 -- start start 2024-09-21T03:39:28.853 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:39:28.831+0000 7fc8b4470640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc8ac102ae0 0x7fc8ac19a470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:28.853 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:39:28.831+0000 7fc8b4470640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc8ac1083d0 0x7fc8ac19aa10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:28.853 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:39:28.831+0000 7fc8b4470640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc8ac108d20 0x7fc8ac19fcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:28.853 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:39:28.831+0000 7fc8b4470640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fc8ac112990 con 0x7fc8ac102ae0 2024-09-21T03:39:28.854 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:39:28.831+0000 7fc8b4470640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fc8ac112810 con 0x7fc8ac108d20 2024-09-21T03:39:28.854 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:39:28.831+0000 7fc8b4470640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fc8ac112b10 con 0x7fc8ac1083d0 2024-09-21T03:39:28.854 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:39:28.831+0000 7fc8b220c640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc8ac102ae0 0x7fc8ac19a470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:28.854 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:39:28.831+0000 7fc8b220c640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc8ac102ae0 0x7fc8ac19a470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:50762/0 (socket says 172.21.15.39:50762) 2024-09-21T03:39:28.854 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:39:28.831+0000 7fc8b220c640 1 -- 172.21.15.39:0/3731748904 learned_addr learned my addr 172.21.15.39:0/3731748904 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:39:28.854 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:39:28.831+0000 7fc8b1a0b640 1 --2- 172.21.15.39:0/3731748904 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc8ac1083d0 0x7fc8ac19aa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:28.854 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:39:28.831+0000 7fc8b2a0d640 1 --2- 172.21.15.39:0/3731748904 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc8ac108d20 0x7fc8ac19fcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:28.854 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:39:28.831+0000 7fc8b220c640 1 -- 172.21.15.39:0/3731748904 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc8ac1083d0 msgr2=0x7fc8ac19aa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:28.854 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:39:28.831+0000 7fc8b220c640 1 --2- 172.21.15.39:0/3731748904 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc8ac1083d0 0x7fc8ac19aa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:28.854 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:39:28.831+0000 7fc8b220c640 1 -- 172.21.15.39:0/3731748904 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc8ac108d20 msgr2=0x7fc8ac19fcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:28.854 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:39:28.831+0000 7fc8b220c640 1 --2- 172.21.15.39:0/3731748904 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc8ac108d20 0x7fc8ac19fcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:28.854 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:39:28.831+0000 7fc8b220c640 1 -- 172.21.15.39:0/3731748904 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc8ac1a03e0 con 0x7fc8ac102ae0 2024-09-21T03:39:28.854 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:39:28.831+0000 7fc8b2a0d640 1 --2- 172.21.15.39:0/3731748904 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc8ac108d20 0x7fc8ac19fcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:39:28.854 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:39:28.831+0000 7fc8b220c640 1 --2- 172.21.15.39:0/3731748904 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc8ac102ae0 0x7fc8ac19a470 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fc8a800c890 tx=0x7fc8a800cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:39:28.855 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:39:28.831+0000 7fc8a37fe640 1 -- 172.21.15.39:0/3731748904 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8a8004220 con 0x7fc8ac102ae0 2024-09-21T03:39:28.855 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:39:28.832+0000 7fc8b4470640 1 -- 172.21.15.39:0/3731748904 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc8ac1a06d0 con 0x7fc8ac102ae0 2024-09-21T03:39:28.855 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:39:28.832+0000 7fc8a37fe640 1 -- 172.21.15.39:0/3731748904 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc8a80043c0 con 0x7fc8ac102ae0 2024-09-21T03:39:28.855 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:39:28.832+0000 7fc8a37fe640 1 -- 172.21.15.39:0/3731748904 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8a8004a00 con 0x7fc8ac102ae0 2024-09-21T03:39:28.855 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:39:28.832+0000 7fc8b4470640 1 -- 172.21.15.39:0/3731748904 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc8ac1a6e30 con 0x7fc8ac102ae0 2024-09-21T03:39:28.855 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:39:28.833+0000 7fc8b4470640 1 -- 172.21.15.39:0/3731748904 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc8ac103d10 con 0x7fc8ac102ae0 2024-09-21T03:39:28.855 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:39:28.834+0000 7fc8a37fe640 1 -- 172.21.15.39:0/3731748904 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 20) ==== 154473+0+0 (secure 0 0 0) 0x7fc8a80079b0 con 0x7fc8ac102ae0 2024-09-21T03:39:28.855 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:39:28.835+0000 7fc8a37fe640 1 --2- 172.21.15.39:0/3731748904 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7fc8980b64d0 0x7fc8980b89e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:28.855 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:39:28.835+0000 7fc8b1a0b640 1 --2- 172.21.15.39:0/3731748904 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7fc8980b64d0 0x7fc8980b89e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:28.855 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:39:28.835+0000 7fc8a37fe640 1 -- 172.21.15.39:0/3731748904 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(18..18 src has 1..18) ==== 2786+0+0 (secure 0 0 0) 0x7fc8a80e5e30 con 0x7fc8ac102ae0 2024-09-21T03:39:28.855 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:39:28.836+0000 7fc8b1a0b640 1 --2- 172.21.15.39:0/3731748904 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7fc8980b64d0 0x7fc8980b89e0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fc89c000c00 tx=0x7fc89c031400 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:39:28.855 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:39:28.838+0000 7fc8a37fe640 1 -- 172.21.15.39:0/3731748904 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fc8a8014030 con 0x7fc8ac102ae0 2024-09-21T03:39:29.042 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:39:29.039+0000 7fc8b4470640 1 -- 172.21.15.39:0/3731748904 --> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi039:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7fc8ac063800 con 0x7fc8980b64d0 2024-09-21T03:39:29.333 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:39:29 smithi039 ceph-mon[31211]: purged_snaps scrub starts 2024-09-21T03:39:29.334 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:39:29 smithi039 ceph-mon[31211]: purged_snaps scrub ok 2024-09-21T03:39:29.334 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:39:29 smithi039 ceph-mon[31211]: pgmap v66: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-21T03:39:29.334 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:39:29 smithi039 ceph-mon[31211]: Detected new or changed devices on smithi161 2024-09-21T03:39:29.334 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:39:29 smithi039 ceph-mon[31211]: Adjusting osd_memory_target on smithi161 to 17088M 2024-09-21T03:39:29.334 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:39:29 smithi039 ceph-mon[31211]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi161", "root=default"]}]': finished 2024-09-21T03:39:29.334 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:39:29 smithi039 ceph-mon[31211]: osdmap e18: 3 total, 2 up, 3 in 2024-09-21T03:39:29.334 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:39:29 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-21T03:39:29.334 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:39:29 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T03:39:29.334 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:39:29 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T03:39:29.334 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:39:29 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:39:29.334 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:39:29 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-21T03:39:29.468 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:39:29 smithi161 ceph-mon[34533]: purged_snaps scrub starts 2024-09-21T03:39:29.468 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:39:29 smithi161 ceph-mon[34533]: purged_snaps scrub ok 2024-09-21T03:39:29.468 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:39:29 smithi161 ceph-mon[34533]: pgmap v66: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-21T03:39:29.468 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:39:29 smithi161 ceph-mon[34533]: Detected new or changed devices on smithi161 2024-09-21T03:39:29.468 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:39:29 smithi161 ceph-mon[34533]: Adjusting osd_memory_target on smithi161 to 17088M 2024-09-21T03:39:29.468 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:39:29 smithi161 ceph-mon[34533]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi161", "root=default"]}]': finished 2024-09-21T03:39:29.468 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:39:29 smithi161 ceph-mon[34533]: osdmap e18: 3 total, 2 up, 3 in 2024-09-21T03:39:29.468 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:39:29 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-21T03:39:29.468 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:39:29 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T03:39:29.468 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:39:29 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T03:39:29.468 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:39:29 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:39:29.469 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:39:29 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-21T03:39:29.474 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:39:29 smithi150 ceph-mon[34289]: purged_snaps scrub starts 2024-09-21T03:39:29.474 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:39:29 smithi150 ceph-mon[34289]: purged_snaps scrub ok 2024-09-21T03:39:29.475 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:39:29 smithi150 ceph-mon[34289]: pgmap v66: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-21T03:39:29.475 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:39:29 smithi150 ceph-mon[34289]: Detected new or changed devices on smithi161 2024-09-21T03:39:29.475 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:39:29 smithi150 ceph-mon[34289]: Adjusting osd_memory_target on smithi161 to 17088M 2024-09-21T03:39:29.475 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:39:29 smithi150 ceph-mon[34289]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi161", "root=default"]}]': finished 2024-09-21T03:39:29.475 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:39:29 smithi150 ceph-mon[34289]: osdmap e18: 3 total, 2 up, 3 in 2024-09-21T03:39:29.475 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:39:29 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-21T03:39:29.475 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:39:29 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T03:39:29.475 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:39:29 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T03:39:29.475 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:39:29 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:39:29.475 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:39:29 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-21T03:39:30.459 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:39:30 smithi039 ceph-mon[31211]: from='client.14526 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi039:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:39:30.459 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:39:30 smithi039 ceph-mon[31211]: osd.2 [v2:172.21.15.161:6800/2727876133,v1:172.21.15.161:6801/2727876133] boot 2024-09-21T03:39:30.459 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:39:30 smithi039 ceph-mon[31211]: osdmap e19: 3 total, 3 up, 3 in 2024-09-21T03:39:30.459 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:39:30 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-21T03:39:30.459 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:39:30 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true} : dispatch 2024-09-21T03:39:30.467 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:39:30 smithi161 ceph-mon[34533]: from='client.14526 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi039:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:39:30.467 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:39:30 smithi161 ceph-mon[34533]: osd.2 [v2:172.21.15.161:6800/2727876133,v1:172.21.15.161:6801/2727876133] boot 2024-09-21T03:39:30.468 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:39:30 smithi161 ceph-mon[34533]: osdmap e19: 3 total, 3 up, 3 in 2024-09-21T03:39:30.468 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:39:30 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-21T03:39:30.468 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:39:30 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true} : dispatch 2024-09-21T03:39:30.474 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:39:30 smithi150 ceph-mon[34289]: from='client.14526 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi039:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:39:30.474 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:39:30 smithi150 ceph-mon[34289]: osd.2 [v2:172.21.15.161:6800/2727876133,v1:172.21.15.161:6801/2727876133] boot 2024-09-21T03:39:30.474 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:39:30 smithi150 ceph-mon[34289]: osdmap e19: 3 total, 3 up, 3 in 2024-09-21T03:39:30.474 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:39:30 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-21T03:39:30.475 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:39:30 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true} : dispatch 2024-09-21T03:39:31.415 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:39:31 smithi039 ceph-mon[31211]: pgmap v69: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-21T03:39:31.416 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:39:31 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-09-21T03:39:31.416 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:39:31 smithi039 ceph-mon[31211]: osdmap e20: 3 total, 3 up, 3 in 2024-09-21T03:39:31.416 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:39:31 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true} : dispatch 2024-09-21T03:39:31.416 INFO:journalctl@ceph.osd.0.smithi039.stdout:Sep 21 03:39:31 smithi039 sudo[51909]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-21T03:39:31.416 INFO:journalctl@ceph.osd.0.smithi039.stdout:Sep 21 03:39:31 smithi039 sudo[51909]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-21T03:39:31.416 INFO:journalctl@ceph.osd.0.smithi039.stdout:Sep 21 03:39:31 smithi039 sudo[51909]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-21T03:39:31.416 INFO:journalctl@ceph.osd.0.smithi039.stdout:Sep 21 03:39:31 smithi039 sudo[51909]: pam_unix(sudo:session): session closed for user root 2024-09-21T03:39:31.416 INFO:journalctl@ceph.osd.0.smithi039.stdout:Sep 21 03:39:31 smithi039 sudo[51912]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-21T03:39:31.416 INFO:journalctl@ceph.osd.0.smithi039.stdout:Sep 21 03:39:31 smithi039 sudo[51912]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-21T03:39:31.416 INFO:journalctl@ceph.osd.0.smithi039.stdout:Sep 21 03:39:31 smithi039 sudo[51912]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-21T03:39:31.416 INFO:journalctl@ceph.osd.0.smithi039.stdout:Sep 21 03:39:31 smithi039 sudo[51912]: pam_unix(sudo:session): session closed for user root 2024-09-21T03:39:31.459 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:39:31 smithi161 ceph-mon[34533]: pgmap v69: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-21T03:39:31.459 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:39:31 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-09-21T03:39:31.459 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:39:31 smithi161 ceph-mon[34533]: osdmap e20: 3 total, 3 up, 3 in 2024-09-21T03:39:31.459 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:39:31 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true} : dispatch 2024-09-21T03:39:31.459 INFO:journalctl@ceph.osd.2.smithi161.stdout:Sep 21 03:39:31 smithi161 sudo[43312]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-21T03:39:31.459 INFO:journalctl@ceph.osd.2.smithi161.stdout:Sep 21 03:39:31 smithi161 sudo[43312]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-21T03:39:31.459 INFO:journalctl@ceph.osd.2.smithi161.stdout:Sep 21 03:39:31 smithi161 sudo[43312]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-21T03:39:31.459 INFO:journalctl@ceph.osd.2.smithi161.stdout:Sep 21 03:39:31 smithi161 sudo[43312]: pam_unix(sudo:session): session closed for user root 2024-09-21T03:39:31.460 INFO:journalctl@ceph.osd.2.smithi161.stdout:Sep 21 03:39:31 smithi161 sudo[43315]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-21T03:39:31.460 INFO:journalctl@ceph.osd.2.smithi161.stdout:Sep 21 03:39:31 smithi161 sudo[43315]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-21T03:39:31.460 INFO:journalctl@ceph.osd.2.smithi161.stdout:Sep 21 03:39:31 smithi161 sudo[43315]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-21T03:39:31.460 INFO:journalctl@ceph.osd.2.smithi161.stdout:Sep 21 03:39:31 smithi161 sudo[43315]: pam_unix(sudo:session): session closed for user root 2024-09-21T03:39:31.474 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:39:31 smithi150 ceph-mon[34289]: pgmap v69: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-21T03:39:31.475 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:39:31 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-09-21T03:39:31.475 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:39:31 smithi150 ceph-mon[34289]: osdmap e20: 3 total, 3 up, 3 in 2024-09-21T03:39:31.475 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:39:31 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true} : dispatch 2024-09-21T03:39:31.475 INFO:journalctl@ceph.osd.1.smithi150.stdout:Sep 21 03:39:31 smithi150 sudo[42718]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-21T03:39:31.475 INFO:journalctl@ceph.osd.1.smithi150.stdout:Sep 21 03:39:31 smithi150 sudo[42718]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-21T03:39:31.475 INFO:journalctl@ceph.osd.1.smithi150.stdout:Sep 21 03:39:31 smithi150 sudo[42718]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-21T03:39:31.475 INFO:journalctl@ceph.osd.1.smithi150.stdout:Sep 21 03:39:31 smithi150 sudo[42718]: pam_unix(sudo:session): session closed for user root 2024-09-21T03:39:31.475 INFO:journalctl@ceph.osd.1.smithi150.stdout:Sep 21 03:39:31 smithi150 sudo[42721]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-21T03:39:31.476 INFO:journalctl@ceph.osd.1.smithi150.stdout:Sep 21 03:39:31 smithi150 sudo[42721]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-21T03:39:31.476 INFO:journalctl@ceph.osd.1.smithi150.stdout:Sep 21 03:39:31 smithi150 sudo[42721]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-21T03:39:31.476 INFO:journalctl@ceph.osd.1.smithi150.stdout:Sep 21 03:39:31 smithi150 sudo[42721]: pam_unix(sudo:session): session closed for user root 2024-09-21T03:39:31.718 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:39:31 smithi161 sudo[43318]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-21T03:39:31.718 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:39:31 smithi161 sudo[43318]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-21T03:39:31.718 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:39:31 smithi161 sudo[43318]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-21T03:39:31.718 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:39:31 smithi161 sudo[43318]: pam_unix(sudo:session): session closed for user root 2024-09-21T03:39:31.718 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:39:31 smithi161 sudo[43321]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-21T03:39:31.718 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:39:31 smithi161 sudo[43321]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-21T03:39:31.718 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:39:31 smithi161 sudo[43321]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-21T03:39:31.718 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:39:31 smithi161 sudo[43321]: pam_unix(sudo:session): session closed for user root 2024-09-21T03:39:31.809 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:39:31 smithi039 sudo[51935]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-21T03:39:31.809 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:39:31 smithi039 sudo[51935]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-21T03:39:31.810 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:39:31 smithi039 sudo[51935]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-21T03:39:31.810 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:39:31 smithi039 sudo[51935]: pam_unix(sudo:session): session closed for user root 2024-09-21T03:39:31.810 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:39:31 smithi039 sudo[51938]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-21T03:39:31.810 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:39:31 smithi039 sudo[51938]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-21T03:39:31.810 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:39:31 smithi039 sudo[51938]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-21T03:39:31.810 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:39:31 smithi039 sudo[51938]: pam_unix(sudo:session): session closed for user root 2024-09-21T03:39:31.974 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:39:31 smithi150 sudo[42724]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-21T03:39:31.975 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:39:31 smithi150 sudo[42724]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-21T03:39:31.975 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:39:31 smithi150 sudo[42724]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-21T03:39:31.975 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:39:31 smithi150 sudo[42724]: pam_unix(sudo:session): session closed for user root 2024-09-21T03:39:31.975 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:39:31 smithi150 sudo[42727]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-21T03:39:31.975 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:39:31 smithi150 sudo[42727]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-21T03:39:31.975 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:39:31 smithi150 sudo[42727]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-21T03:39:31.975 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:39:31 smithi150 sudo[42727]: pam_unix(sudo:session): session closed for user root 2024-09-21T03:39:32.419 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:39:32.416+0000 7fc8a37fe640 1 -- 172.21.15.39:0/3731748904 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mgrmap(e 21) ==== 154529+0+0 (secure 0 0 0) 0x7fc8a80a98a0 con 0x7fc8ac102ae0 2024-09-21T03:39:32.419 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:39:32 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-09-21T03:39:32.419 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:39:32 smithi039 ceph-mon[31211]: osdmap e21: 3 total, 3 up, 3 in 2024-09-21T03:39:32.419 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:39:32 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:39:32.419 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:39:32 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:39:32.420 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:39:32 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:39:32.420 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:39:32 smithi039 ceph-mon[31211]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-21T03:39:32.420 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:39:32 smithi039 ceph-mon[31211]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-21T03:39:32.420 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:39:32 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-21T03:39:32.420 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:39:32 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T03:39:32.420 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:39:32 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T03:39:32.420 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:39:32 smithi039 ceph-mon[31211]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-21T03:39:32.420 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:39:32 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-21T03:39:32.420 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:39:32 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T03:39:32.420 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:39:32 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T03:39:32.420 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:39:32 smithi039 ceph-mon[31211]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-21T03:39:32.420 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:39:32 smithi039 ceph-mon[31211]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-21T03:39:32.420 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:39:32 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-21T03:39:32.420 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:39:32 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T03:39:32.420 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:39:32 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T03:39:32.421 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:39:32 smithi039 ceph-mon[31211]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-21T03:39:32.421 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:39:32 smithi039 ceph-mon[31211]: from='client.? 172.21.15.39:0/1912125109' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "14417ae1-4536-45ba-9a5c-c3b447d1a72d"} : dispatch 2024-09-21T03:39:32.421 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:39:32 smithi039 ceph-mon[31211]: from='client.? 172.21.15.39:0/1912125109' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "14417ae1-4536-45ba-9a5c-c3b447d1a72d"}]': finished 2024-09-21T03:39:32.421 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:39:32 smithi039 ceph-mon[31211]: osdmap e22: 4 total, 3 up, 4 in 2024-09-21T03:39:32.421 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:39:32 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-21T03:39:32.468 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:39:32 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-09-21T03:39:32.468 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:39:32 smithi161 ceph-mon[34533]: osdmap e21: 3 total, 3 up, 3 in 2024-09-21T03:39:32.468 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:39:32 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:39:32.468 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:39:32 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:39:32.468 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:39:32 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:39:32.468 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:39:32 smithi161 ceph-mon[34533]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-21T03:39:32.468 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:39:32 smithi161 ceph-mon[34533]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-21T03:39:32.468 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:39:32 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-21T03:39:32.468 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:39:32 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T03:39:32.468 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:39:32 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T03:39:32.469 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:39:32 smithi161 ceph-mon[34533]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-21T03:39:32.469 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:39:32 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-21T03:39:32.469 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:39:32 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T03:39:32.469 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:39:32 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T03:39:32.469 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:39:32 smithi161 ceph-mon[34533]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-21T03:39:32.469 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:39:32 smithi161 ceph-mon[34533]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-21T03:39:32.469 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:39:32 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-21T03:39:32.469 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:39:32 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T03:39:32.469 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:39:32 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T03:39:32.469 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:39:32 smithi161 ceph-mon[34533]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-21T03:39:32.469 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:39:32 smithi161 ceph-mon[34533]: from='client.? 172.21.15.39:0/1912125109' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "14417ae1-4536-45ba-9a5c-c3b447d1a72d"} : dispatch 2024-09-21T03:39:32.469 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:39:32 smithi161 ceph-mon[34533]: from='client.? 172.21.15.39:0/1912125109' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "14417ae1-4536-45ba-9a5c-c3b447d1a72d"}]': finished 2024-09-21T03:39:32.469 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:39:32 smithi161 ceph-mon[34533]: osdmap e22: 4 total, 3 up, 4 in 2024-09-21T03:39:32.469 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:39:32 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-21T03:39:32.474 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:39:32 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-09-21T03:39:32.475 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:39:32 smithi150 ceph-mon[34289]: osdmap e21: 3 total, 3 up, 3 in 2024-09-21T03:39:32.475 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:39:32 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:39:32.475 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:39:32 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:39:32.475 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:39:32 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:39:32.475 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:39:32 smithi150 ceph-mon[34289]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-21T03:39:32.475 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:39:32 smithi150 ceph-mon[34289]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-21T03:39:32.475 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:39:32 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-21T03:39:32.475 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:39:32 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T03:39:32.475 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:39:32 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T03:39:32.475 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:39:32 smithi150 ceph-mon[34289]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-21T03:39:32.475 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:39:32 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-21T03:39:32.475 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:39:32 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T03:39:32.475 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:39:32 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T03:39:32.475 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:39:32 smithi150 ceph-mon[34289]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-21T03:39:32.476 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:39:32 smithi150 ceph-mon[34289]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-21T03:39:32.476 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:39:32 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-21T03:39:32.476 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:39:32 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T03:39:32.476 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:39:32 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T03:39:32.476 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:39:32 smithi150 ceph-mon[34289]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-21T03:39:32.476 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:39:32 smithi150 ceph-mon[34289]: from='client.? 172.21.15.39:0/1912125109' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "14417ae1-4536-45ba-9a5c-c3b447d1a72d"} : dispatch 2024-09-21T03:39:32.476 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:39:32 smithi150 ceph-mon[34289]: from='client.? 172.21.15.39:0/1912125109' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "14417ae1-4536-45ba-9a5c-c3b447d1a72d"}]': finished 2024-09-21T03:39:32.476 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:39:32 smithi150 ceph-mon[34289]: osdmap e22: 4 total, 3 up, 4 in 2024-09-21T03:39:32.476 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:39:32 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-21T03:39:33.718 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:39:33 smithi161 ceph-mon[34533]: pgmap v72: 1 pgs: 1 unknown; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2024-09-21T03:39:33.718 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:39:33 smithi161 ceph-mon[34533]: from='client.? 172.21.15.39:0/4241393658' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T03:39:33.718 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:39:33 smithi161 ceph-mon[34533]: mgrmap e21: x(active, since 2m), standbys: z, y 2024-09-21T03:39:33.724 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:39:33 smithi150 ceph-mon[34289]: pgmap v72: 1 pgs: 1 unknown; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2024-09-21T03:39:33.724 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:39:33 smithi150 ceph-mon[34289]: from='client.? 172.21.15.39:0/4241393658' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T03:39:33.724 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:39:33 smithi150 ceph-mon[34289]: mgrmap e21: x(active, since 2m), standbys: z, y 2024-09-21T03:39:33.834 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:39:33 smithi039 ceph-mon[31211]: pgmap v72: 1 pgs: 1 unknown; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2024-09-21T03:39:33.834 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:39:33 smithi039 ceph-mon[31211]: from='client.? 172.21.15.39:0/4241393658' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T03:39:33.834 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:39:33 smithi039 ceph-mon[31211]: mgrmap e21: x(active, since 2m), standbys: z, y 2024-09-21T03:39:34.717 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:39:34 smithi161 ceph-mon[34533]: pgmap v74: 1 pgs: 1 unknown; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2024-09-21T03:39:34.724 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:39:34 smithi150 ceph-mon[34289]: pgmap v74: 1 pgs: 1 unknown; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2024-09-21T03:39:34.833 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:39:34 smithi039 ceph-mon[31211]: pgmap v74: 1 pgs: 1 unknown; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2024-09-21T03:39:36.786 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:39:36 smithi161 ceph-mon[34533]: pgmap v75: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-09-21T03:39:36.833 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:39:36 smithi039 ceph-mon[31211]: pgmap v75: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-09-21T03:39:36.974 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:39:36 smithi150 ceph-mon[34289]: pgmap v75: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-09-21T03:39:38.711 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:39:38 smithi039 ceph-mon[31211]: pgmap v76: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-09-21T03:39:38.777 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:39:38 smithi150 ceph-mon[34289]: pgmap v76: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-09-21T03:39:38.968 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:39:38 smithi161 ceph-mon[34533]: pgmap v76: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-09-21T03:39:40.660 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:39:40 smithi039 ceph-mon[31211]: pgmap v77: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-09-21T03:39:40.967 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:39:40 smithi161 ceph-mon[34533]: pgmap v77: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-09-21T03:39:40.974 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:39:40 smithi150 ceph-mon[34289]: pgmap v77: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-09-21T03:39:41.834 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:39:41 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.3"} : dispatch 2024-09-21T03:39:41.834 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:39:41 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:39:41.967 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:39:41 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.3"} : dispatch 2024-09-21T03:39:41.968 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:39:41 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:39:41.974 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:39:41 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.3"} : dispatch 2024-09-21T03:39:41.974 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:39:41 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:39:42.823 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:39:42 smithi039 ceph-mon[31211]: pgmap v78: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-09-21T03:39:42.823 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:39:42 smithi039 ceph-mon[31211]: Deploying daemon osd.3 on smithi039 2024-09-21T03:39:42.967 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:39:42 smithi161 ceph-mon[34533]: pgmap v78: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-09-21T03:39:42.968 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:39:42 smithi161 ceph-mon[34533]: Deploying daemon osd.3 on smithi039 2024-09-21T03:39:42.974 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:39:42 smithi150 ceph-mon[34289]: pgmap v78: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-09-21T03:39:42.974 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:39:42 smithi150 ceph-mon[34289]: Deploying daemon osd.3 on smithi039 2024-09-21T03:39:44.828 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:39:44 smithi039 ceph-mon[31211]: pgmap v79: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-09-21T03:39:44.970 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:39:44 smithi161 ceph-mon[34533]: pgmap v79: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-09-21T03:39:44.974 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:39:44 smithi150 ceph-mon[34289]: pgmap v79: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-09-21T03:39:46.834 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:39:46 smithi039 ceph-mon[31211]: pgmap v80: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-09-21T03:39:46.968 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:39:46 smithi161 ceph-mon[34533]: pgmap v80: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-09-21T03:39:46.974 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:39:46 smithi150 ceph-mon[34289]: pgmap v80: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-09-21T03:39:48.833 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:39:48 smithi039 ceph-mon[31211]: pgmap v81: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-09-21T03:39:48.834 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:39:48 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:39:48.834 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:39:48 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:39:48.834 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:39:48 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:39:48.967 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:39:48 smithi161 ceph-mon[34533]: pgmap v81: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-09-21T03:39:48.968 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:39:48 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:39:48.968 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:39:48 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:39:48.968 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:39:48 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:39:48.974 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:39:48 smithi150 ceph-mon[34289]: pgmap v81: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-09-21T03:39:48.974 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:39:48 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:39:48.974 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:39:48 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:39:48.974 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:39:48 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:39:50.690 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:39:50 smithi039 ceph-mon[31211]: pgmap v82: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-09-21T03:39:50.967 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:39:50 smithi161 ceph-mon[34533]: pgmap v82: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-09-21T03:39:50.974 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:39:50 smithi150 ceph-mon[34289]: pgmap v82: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-09-21T03:39:51.833 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:39:51 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:39:51.834 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:39:51 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:39:51.834 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:39:51 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:39:51.834 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:39:51 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:39:51.834 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:39:51 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:39:51.968 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:39:51 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:39:51.968 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:39:51 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:39:51.968 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:39:51 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:39:51.968 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:39:51 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:39:51.968 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:39:51 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:39:51.974 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:39:51 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:39:51.974 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:39:51 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:39:51.974 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:39:51 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:39:51.974 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:39:51 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:39:51.975 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:39:51 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:39:52.302 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:39:52.301+0000 7fc8a37fe640 1 -- 172.21.15.39:0/3731748904 <== mgr.14223 v2:172.21.15.39:6800/1696381982 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fc8ac063800 con 0x7fc8980b64d0 2024-09-21T03:39:52.303 INFO:teuthology.orchestra.run.smithi039.stdout:Created osd(s) 3 on host 'smithi039' 2024-09-21T03:39:52.305 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:39:52.303+0000 7fc8b4470640 1 -- 172.21.15.39:0/3731748904 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7fc8980b64d0 msgr2=0x7fc8980b89e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:52.305 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:39:52.303+0000 7fc8b4470640 1 --2- 172.21.15.39:0/3731748904 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7fc8980b64d0 0x7fc8980b89e0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fc89c000c00 tx=0x7fc89c031400 comp rx=0 tx=0).stop 2024-09-21T03:39:52.305 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:39:52.303+0000 7fc8b4470640 1 -- 172.21.15.39:0/3731748904 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc8ac102ae0 msgr2=0x7fc8ac19a470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:52.305 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:39:52.303+0000 7fc8b4470640 1 --2- 172.21.15.39:0/3731748904 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc8ac102ae0 0x7fc8ac19a470 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fc8a800c890 tx=0x7fc8a800cd50 comp rx=0 tx=0).stop 2024-09-21T03:39:52.305 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:39:52.303+0000 7fc8b4470640 1 -- 172.21.15.39:0/3731748904 shutdown_connections 2024-09-21T03:39:52.305 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:39:52.303+0000 7fc8b4470640 1 --2- 172.21.15.39:0/3731748904 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7fc8980b64d0 0x7fc8980b89e0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:52.306 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:39:52.303+0000 7fc8b4470640 1 --2- 172.21.15.39:0/3731748904 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc8ac108d20 0x7fc8ac19fcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:52.306 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:39:52.303+0000 7fc8b4470640 1 --2- 172.21.15.39:0/3731748904 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc8ac1083d0 0x7fc8ac19aa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:52.306 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:39:52.303+0000 7fc8b4470640 1 --2- 172.21.15.39:0/3731748904 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc8ac102ae0 0x7fc8ac19a470 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:52.306 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:39:52.303+0000 7fc8b4470640 1 -- 172.21.15.39:0/3731748904 >> 172.21.15.39:0/3731748904 conn(0x7fc8ac079760 msgr2=0x7fc8ac101b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:39:52.306 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:39:52.303+0000 7fc8b4470640 1 -- 172.21.15.39:0/3731748904 shutdown_connections 2024-09-21T03:39:52.306 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:39:52.303+0000 7fc8b4470640 1 -- 172.21.15.39:0/3731748904 wait complete. 2024-09-21T03:39:52.957 DEBUG:teuthology.orchestra.run.smithi039:osd.3> sudo journalctl -f -n 0 -u ceph-51116d5c-77ca-11ef-baf5-efdc52797490@osd.3.service 2024-09-21T03:39:52.960 INFO:tasks.cephadm:Deploying osd.4 on smithi150 with /dev/vg_nvme/lv_3... 2024-09-21T03:39:52.960 DEBUG:teuthology.orchestra.run.smithi150:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 51116d5c-77ca-11ef-baf5-efdc52797490 -- lvm zap /dev/vg_nvme/lv_3 2024-09-21T03:39:53.218 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:39:52 smithi161 ceph-mon[34533]: pgmap v83: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-09-21T03:39:53.218 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:39:52 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:39:53.218 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:39:52 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:39:53.218 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:39:52 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:39:53.224 INFO:teuthology.orchestra.run.smithi150.stderr:Inferring config /var/lib/ceph/51116d5c-77ca-11ef-baf5-efdc52797490/mon.b/config 2024-09-21T03:39:53.224 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:39:52 smithi150 ceph-mon[34289]: pgmap v83: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-09-21T03:39:53.225 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:39:52 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:39:53.225 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:39:52 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:39:53.225 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:39:52 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:39:53.243 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:39:52 smithi039 ceph-mon[31211]: pgmap v83: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-09-21T03:39:53.243 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:39:52 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:39:53.243 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:39:52 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:39:53.244 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:39:52 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:39:54.383 INFO:journalctl@ceph.osd.3.smithi039.stdout:Sep 21 03:39:54 smithi039 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-3[53927]: 2024-09-21T03:39:54.125+0000 7f7964c7f640 -1 osd.3 0 log_to_monitors true 2024-09-21T03:39:55.217 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:39:54 smithi161 ceph-mon[34533]: pgmap v84: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-09-21T03:39:55.218 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:39:54 smithi161 ceph-mon[34533]: from='osd.3 [v2:172.21.15.39:6810/295394468,v1:172.21.15.39:6811/295394468]' entity='osd.3' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]} : dispatch 2024-09-21T03:39:55.225 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:39:54 smithi150 ceph-mon[34289]: pgmap v84: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-09-21T03:39:55.225 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:39:54 smithi150 ceph-mon[34289]: from='osd.3 [v2:172.21.15.39:6810/295394468,v1:172.21.15.39:6811/295394468]' entity='osd.3' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]} : dispatch 2024-09-21T03:39:55.333 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:39:54 smithi039 ceph-mon[31211]: pgmap v84: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-09-21T03:39:55.334 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:39:54 smithi039 ceph-mon[31211]: from='osd.3 [v2:172.21.15.39:6810/295394468,v1:172.21.15.39:6811/295394468]' entity='osd.3' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]} : dispatch 2024-09-21T03:39:55.801 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-09-21T03:39:55.831 DEBUG:teuthology.orchestra.run.smithi150:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 51116d5c-77ca-11ef-baf5-efdc52797490 -- ceph orch daemon add osd smithi150:vg_nvme/lv_3 2024-09-21T03:39:56.081 INFO:teuthology.orchestra.run.smithi150.stderr:Inferring config /var/lib/ceph/51116d5c-77ca-11ef-baf5-efdc52797490/mon.b/config 2024-09-21T03:39:56.083 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:39:55 smithi039 ceph-mon[31211]: from='osd.3 [v2:172.21.15.39:6810/295394468,v1:172.21.15.39:6811/295394468]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-09-21T03:39:56.083 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:39:55 smithi039 ceph-mon[31211]: osdmap e23: 4 total, 3 up, 4 in 2024-09-21T03:39:56.084 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:39:55 smithi039 ceph-mon[31211]: from='osd.3 [v2:172.21.15.39:6810/295394468,v1:172.21.15.39:6811/295394468]' entity='osd.3' cmd={"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi039", "root=default"]} : dispatch 2024-09-21T03:39:56.084 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:39:55 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-21T03:39:56.084 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:39:55 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:39:56.084 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:39:55 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:39:56.084 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:39:55 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-21T03:39:56.084 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:39:55 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"} : dispatch 2024-09-21T03:39:56.084 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:39:55 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:39:56.084 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:39:55 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:39:56.084 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:39:55 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:39:56.084 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:39:55 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:39:56.201 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:39:55 smithi150 ceph-mon[34289]: from='osd.3 [v2:172.21.15.39:6810/295394468,v1:172.21.15.39:6811/295394468]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-09-21T03:39:56.202 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:39:55 smithi150 ceph-mon[34289]: osdmap e23: 4 total, 3 up, 4 in 2024-09-21T03:39:56.202 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:39:55 smithi150 ceph-mon[34289]: from='osd.3 [v2:172.21.15.39:6810/295394468,v1:172.21.15.39:6811/295394468]' entity='osd.3' cmd={"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi039", "root=default"]} : dispatch 2024-09-21T03:39:56.202 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:39:55 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-21T03:39:56.202 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:39:55 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:39:56.202 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:39:55 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:39:56.202 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:39:55 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-21T03:39:56.202 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:39:55 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"} : dispatch 2024-09-21T03:39:56.202 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:39:55 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:39:56.202 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:39:55 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:39:56.202 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:39:55 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:39:56.202 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:39:55 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:39:56.218 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:39:55 smithi161 ceph-mon[34533]: from='osd.3 [v2:172.21.15.39:6810/295394468,v1:172.21.15.39:6811/295394468]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-09-21T03:39:56.218 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:39:55 smithi161 ceph-mon[34533]: osdmap e23: 4 total, 3 up, 4 in 2024-09-21T03:39:56.218 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:39:55 smithi161 ceph-mon[34533]: from='osd.3 [v2:172.21.15.39:6810/295394468,v1:172.21.15.39:6811/295394468]' entity='osd.3' cmd={"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi039", "root=default"]} : dispatch 2024-09-21T03:39:56.218 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:39:55 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-21T03:39:56.218 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:39:55 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:39:56.218 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:39:55 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:39:56.218 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:39:55 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-21T03:39:56.218 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:39:55 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"} : dispatch 2024-09-21T03:39:56.218 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:39:55 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:39:56.219 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:39:55 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:39:56.219 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:39:55 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:39:56.219 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:39:55 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:39:56.571 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T03:39:56.568+0000 7febf3a9d640 1 -- 172.21.15.150:0/3358278923 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7febec1083a0 msgr2=0x7febec1087b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:56.571 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T03:39:56.568+0000 7febf3a9d640 1 --2- 172.21.15.150:0/3358278923 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7febec1083a0 0x7febec1087b0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7febdc009900 tx=0x7febdc02a410 comp rx=0 tx=0).stop 2024-09-21T03:39:56.571 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T03:39:56.568+0000 7febf3a9d640 1 -- 172.21.15.150:0/3358278923 shutdown_connections 2024-09-21T03:39:56.571 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T03:39:56.568+0000 7febf3a9d640 1 --2- 172.21.15.150:0/3358278923 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7febec108cf0 0x7febec10f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:56.571 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T03:39:56.568+0000 7febf3a9d640 1 --2- 172.21.15.150:0/3358278923 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7febec1083a0 0x7febec1087b0 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:56.571 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T03:39:56.568+0000 7febf3a9d640 1 --2- 172.21.15.150:0/3358278923 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7febec102a60 0x7febec102e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:56.571 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T03:39:56.568+0000 7febf3a9d640 1 -- 172.21.15.150:0/3358278923 >> 172.21.15.150:0/3358278923 conn(0x7febec079730 msgr2=0x7febec079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:39:56.572 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T03:39:56.568+0000 7febf3a9d640 1 -- 172.21.15.150:0/3358278923 shutdown_connections 2024-09-21T03:39:56.572 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T03:39:56.568+0000 7febf3a9d640 1 -- 172.21.15.150:0/3358278923 wait complete. 2024-09-21T03:39:56.572 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T03:39:56.569+0000 7febf3a9d640 1 Processor -- start 2024-09-21T03:39:56.572 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T03:39:56.569+0000 7febf3a9d640 1 -- start start 2024-09-21T03:39:56.572 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T03:39:56.569+0000 7febf3a9d640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7febec102a60 0x7febec19a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:56.572 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T03:39:56.569+0000 7febf3a9d640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7febec1083a0 0x7febec19aa20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:56.572 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T03:39:56.569+0000 7febf3a9d640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7febec108cf0 0x7febec19fcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:56.572 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T03:39:56.569+0000 7febf3a9d640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7febec112960 con 0x7febec1083a0 2024-09-21T03:39:56.572 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T03:39:56.569+0000 7febf3a9d640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7febec1127e0 con 0x7febec108cf0 2024-09-21T03:39:56.572 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T03:39:56.569+0000 7febf3a9d640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7febec112ae0 con 0x7febec102a60 2024-09-21T03:39:56.572 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T03:39:56.569+0000 7febf1038640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7febec1083a0 0x7febec19aa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:56.572 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T03:39:56.569+0000 7febf203a640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7febec108cf0 0x7febec19fcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:56.572 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T03:39:56.569+0000 7febf1839640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7febec102a60 0x7febec19a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:56.573 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T03:39:56.569+0000 7febf1038640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7febec1083a0 0x7febec19aa20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.150:48394/0 (socket says 172.21.15.150:48394) 2024-09-21T03:39:56.573 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T03:39:56.569+0000 7febf1038640 1 -- 172.21.15.150:0/644512146 learned_addr learned my addr 172.21.15.150:0/644512146 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-09-21T03:39:56.574 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T03:39:56.570+0000 7febf203a640 1 -- 172.21.15.150:0/644512146 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7febec102a60 msgr2=0x7febec19a480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:56.574 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T03:39:56.570+0000 7febf203a640 1 --2- 172.21.15.150:0/644512146 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7febec102a60 0x7febec19a480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:56.574 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T03:39:56.570+0000 7febf203a640 1 -- 172.21.15.150:0/644512146 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7febec1083a0 msgr2=0x7febec19aa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:56.574 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T03:39:56.570+0000 7febf203a640 1 --2- 172.21.15.150:0/644512146 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7febec1083a0 0x7febec19aa20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:56.574 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T03:39:56.570+0000 7febf203a640 1 -- 172.21.15.150:0/644512146 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7febec1a03f0 con 0x7febec108cf0 2024-09-21T03:39:56.574 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T03:39:56.570+0000 7febf1839640 1 --2- 172.21.15.150:0/644512146 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7febec102a60 0x7febec19a480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:39:56.574 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T03:39:56.570+0000 7febf203a640 1 --2- 172.21.15.150:0/644512146 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7febec108cf0 0x7febec19fcc0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7febe400da10 tx=0x7febe400ded0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:39:56.574 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T03:39:56.570+0000 7febe2ffd640 1 -- 172.21.15.150:0/644512146 <== mon.2 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febe400bd70 con 0x7febec108cf0 2024-09-21T03:39:56.574 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T03:39:56.570+0000 7febe2ffd640 1 -- 172.21.15.150:0/644512146 <== mon.2 v2:172.21.15.150:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7febe4003e20 con 0x7febec108cf0 2024-09-21T03:39:56.574 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T03:39:56.570+0000 7febf3a9d640 1 -- 172.21.15.150:0/644512146 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7febec1a06e0 con 0x7febec108cf0 2024-09-21T03:39:56.575 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T03:39:56.570+0000 7febe2ffd640 1 -- 172.21.15.150:0/644512146 <== mon.2 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febe40106c0 con 0x7febec108cf0 2024-09-21T03:39:56.575 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T03:39:56.570+0000 7febf3a9d640 1 -- 172.21.15.150:0/644512146 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7febec1a6e40 con 0x7febec108cf0 2024-09-21T03:39:56.575 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T03:39:56.571+0000 7febf3a9d640 1 -- 172.21.15.150:0/644512146 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7febec102e70 con 0x7febec108cf0 2024-09-21T03:39:56.575 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T03:39:56.573+0000 7febe2ffd640 1 -- 172.21.15.150:0/644512146 <== mon.2 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 21) ==== 154529+0+0 (secure 0 0 0) 0x7febe4004270 con 0x7febec108cf0 2024-09-21T03:39:56.575 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T03:39:56.574+0000 7febe2ffd640 1 --2- 172.21.15.150:0/644512146 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7febd80b65e0 0x7febd80b8af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:56.576 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T03:39:56.574+0000 7febf1839640 1 --2- 172.21.15.150:0/644512146 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7febd80b65e0 0x7febd80b8af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:56.576 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T03:39:56.574+0000 7febe2ffd640 1 -- 172.21.15.150:0/644512146 <== mon.2 v2:172.21.15.150:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 3604+0+0 (secure 0 0 0) 0x7febe40e6440 con 0x7febec108cf0 2024-09-21T03:39:56.576 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T03:39:56.574+0000 7febf1839640 1 --2- 172.21.15.150:0/644512146 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7febd80b65e0 0x7febd80b8af0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7febe80058c0 tx=0x7febe800a250 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:39:56.580 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T03:39:56.578+0000 7febe2ffd640 1 -- 172.21.15.150:0/644512146 <== mon.2 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7febe40adf30 con 0x7febec108cf0 2024-09-21T03:39:56.766 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T03:39:56.764+0000 7febf3a9d640 1 -- 172.21.15.150:0/644512146 --> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi150:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7febec063800 con 0x7febd80b65e0 2024-09-21T03:39:57.118 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:39:56 smithi150 ceph-mon[34289]: pgmap v86: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-09-21T03:39:57.119 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:39:56 smithi150 ceph-mon[34289]: Detected new or changed devices on smithi039 2024-09-21T03:39:57.119 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:39:56 smithi150 ceph-mon[34289]: Adjusting osd_memory_target on smithi039 to 8544M 2024-09-21T03:39:57.119 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:39:56 smithi150 ceph-mon[34289]: from='osd.3 [v2:172.21.15.39:6810/295394468,v1:172.21.15.39:6811/295394468]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi039", "root=default"]}]': finished 2024-09-21T03:39:57.119 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:39:56 smithi150 ceph-mon[34289]: osdmap e24: 4 total, 3 up, 4 in 2024-09-21T03:39:57.119 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:39:56 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-21T03:39:57.119 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:39:56 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-21T03:39:57.119 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:39:56 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T03:39:57.119 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:39:56 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T03:39:57.119 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:39:56 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:39:57.119 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:39:56 smithi150 ceph-mon[34289]: from='osd.3 [v2:172.21.15.39:6810/295394468,v1:172.21.15.39:6811/295394468]' entity='osd.3' 2024-09-21T03:39:57.119 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:39:56 smithi150 ceph-mon[34289]: osdmap e25: 4 total, 3 up, 4 in 2024-09-21T03:39:57.120 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:39:56 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-21T03:39:57.120 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:39:56 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-21T03:39:57.218 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:39:56 smithi161 ceph-mon[34533]: pgmap v86: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-09-21T03:39:57.218 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:39:56 smithi161 ceph-mon[34533]: Detected new or changed devices on smithi039 2024-09-21T03:39:57.218 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:39:56 smithi161 ceph-mon[34533]: Adjusting osd_memory_target on smithi039 to 8544M 2024-09-21T03:39:57.218 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:39:56 smithi161 ceph-mon[34533]: from='osd.3 [v2:172.21.15.39:6810/295394468,v1:172.21.15.39:6811/295394468]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi039", "root=default"]}]': finished 2024-09-21T03:39:57.218 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:39:56 smithi161 ceph-mon[34533]: osdmap e24: 4 total, 3 up, 4 in 2024-09-21T03:39:57.218 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:39:56 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-21T03:39:57.218 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:39:56 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-21T03:39:57.219 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:39:56 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T03:39:57.219 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:39:56 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T03:39:57.219 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:39:56 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:39:57.219 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:39:56 smithi161 ceph-mon[34533]: from='osd.3 [v2:172.21.15.39:6810/295394468,v1:172.21.15.39:6811/295394468]' entity='osd.3' 2024-09-21T03:39:57.219 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:39:56 smithi161 ceph-mon[34533]: osdmap e25: 4 total, 3 up, 4 in 2024-09-21T03:39:57.219 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:39:56 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-21T03:39:57.219 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:39:56 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-21T03:39:57.334 INFO:journalctl@ceph.osd.3.smithi039.stdout:Sep 21 03:39:56 smithi039 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-3[53927]: 2024-09-21T03:39:56.910+0000 7f7960c01640 -1 osd.3 0 waiting for initial osdmap 2024-09-21T03:39:57.334 INFO:journalctl@ceph.osd.3.smithi039.stdout:Sep 21 03:39:56 smithi039 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-3[53927]: 2024-09-21T03:39:56.950+0000 7f79589f7640 -1 osd.3 24 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-21T03:39:57.334 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:39:56 smithi039 ceph-mon[31211]: pgmap v86: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-09-21T03:39:57.334 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:39:56 smithi039 ceph-mon[31211]: Detected new or changed devices on smithi039 2024-09-21T03:39:57.334 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:39:56 smithi039 ceph-mon[31211]: Adjusting osd_memory_target on smithi039 to 8544M 2024-09-21T03:39:57.334 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:39:56 smithi039 ceph-mon[31211]: from='osd.3 [v2:172.21.15.39:6810/295394468,v1:172.21.15.39:6811/295394468]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi039", "root=default"]}]': finished 2024-09-21T03:39:57.334 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:39:56 smithi039 ceph-mon[31211]: osdmap e24: 4 total, 3 up, 4 in 2024-09-21T03:39:57.335 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:39:56 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-21T03:39:57.335 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:39:56 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-21T03:39:57.335 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:39:56 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T03:39:57.335 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:39:56 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T03:39:57.335 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:39:56 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:39:57.335 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:39:56 smithi039 ceph-mon[31211]: from='osd.3 [v2:172.21.15.39:6810/295394468,v1:172.21.15.39:6811/295394468]' entity='osd.3' 2024-09-21T03:39:57.335 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:39:56 smithi039 ceph-mon[31211]: osdmap e25: 4 total, 3 up, 4 in 2024-09-21T03:39:57.335 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:39:56 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-21T03:39:57.335 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:39:56 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-21T03:39:58.132 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:39:57 smithi150 ceph-mon[34289]: purged_snaps scrub starts 2024-09-21T03:39:58.132 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:39:57 smithi150 ceph-mon[34289]: purged_snaps scrub ok 2024-09-21T03:39:58.132 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:39:57 smithi150 ceph-mon[34289]: from='client.24383 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi150:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:39:58.132 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:39:57 smithi150 ceph-mon[34289]: osd.3 [v2:172.21.15.39:6810/295394468,v1:172.21.15.39:6811/295394468] boot 2024-09-21T03:39:58.132 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:39:57 smithi150 ceph-mon[34289]: osdmap e26: 4 total, 4 up, 4 in 2024-09-21T03:39:58.133 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:39:57 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-21T03:39:58.217 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:39:57 smithi161 ceph-mon[34533]: purged_snaps scrub starts 2024-09-21T03:39:58.218 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:39:57 smithi161 ceph-mon[34533]: purged_snaps scrub ok 2024-09-21T03:39:58.218 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:39:57 smithi161 ceph-mon[34533]: from='client.24383 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi150:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:39:58.218 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:39:57 smithi161 ceph-mon[34533]: osd.3 [v2:172.21.15.39:6810/295394468,v1:172.21.15.39:6811/295394468] boot 2024-09-21T03:39:58.218 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:39:57 smithi161 ceph-mon[34533]: osdmap e26: 4 total, 4 up, 4 in 2024-09-21T03:39:58.218 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:39:57 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-21T03:39:58.334 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:39:57 smithi039 ceph-mon[31211]: purged_snaps scrub starts 2024-09-21T03:39:58.334 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:39:57 smithi039 ceph-mon[31211]: purged_snaps scrub ok 2024-09-21T03:39:58.334 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:39:57 smithi039 ceph-mon[31211]: from='client.24383 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi150:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:39:58.334 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:39:57 smithi039 ceph-mon[31211]: osd.3 [v2:172.21.15.39:6810/295394468,v1:172.21.15.39:6811/295394468] boot 2024-09-21T03:39:58.334 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:39:57 smithi039 ceph-mon[31211]: osdmap e26: 4 total, 4 up, 4 in 2024-09-21T03:39:58.334 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:39:57 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-21T03:39:59.217 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:39:58 smithi161 ceph-mon[34533]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-09-21T03:39:59.218 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:39:58 smithi161 ceph-mon[34533]: osdmap e27: 4 total, 4 up, 4 in 2024-09-21T03:39:59.224 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:39:58 smithi150 ceph-mon[34289]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-09-21T03:39:59.224 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:39:58 smithi150 ceph-mon[34289]: osdmap e27: 4 total, 4 up, 4 in 2024-09-21T03:39:59.333 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:39:58 smithi039 ceph-mon[31211]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-09-21T03:39:59.334 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:39:58 smithi039 ceph-mon[31211]: osdmap e27: 4 total, 4 up, 4 in 2024-09-21T03:40:00.218 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:39:59 smithi161 ceph-mon[34533]: from='client.? 172.21.15.150:0/1655635834' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "33c732ef-20c2-43c4-abe6-045b687fc521"} : dispatch 2024-09-21T03:40:00.218 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:39:59 smithi161 ceph-mon[34533]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "33c732ef-20c2-43c4-abe6-045b687fc521"} : dispatch 2024-09-21T03:40:00.218 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:39:59 smithi161 ceph-mon[34533]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "33c732ef-20c2-43c4-abe6-045b687fc521"}]': finished 2024-09-21T03:40:00.218 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:39:59 smithi161 ceph-mon[34533]: osdmap e28: 5 total, 4 up, 5 in 2024-09-21T03:40:00.218 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:39:59 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-21T03:40:00.218 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:39:59 smithi161 ceph-mon[34533]: from='client.? 172.21.15.150:0/1783388188' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T03:40:00.224 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:39:59 smithi150 ceph-mon[34289]: from='client.? 172.21.15.150:0/1655635834' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "33c732ef-20c2-43c4-abe6-045b687fc521"} : dispatch 2024-09-21T03:40:00.224 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:39:59 smithi150 ceph-mon[34289]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "33c732ef-20c2-43c4-abe6-045b687fc521"} : dispatch 2024-09-21T03:40:00.224 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:39:59 smithi150 ceph-mon[34289]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "33c732ef-20c2-43c4-abe6-045b687fc521"}]': finished 2024-09-21T03:40:00.225 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:39:59 smithi150 ceph-mon[34289]: osdmap e28: 5 total, 4 up, 5 in 2024-09-21T03:40:00.225 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:39:59 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-21T03:40:00.225 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:39:59 smithi150 ceph-mon[34289]: from='client.? 172.21.15.150:0/1783388188' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T03:40:00.334 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:39:59 smithi039 ceph-mon[31211]: from='client.? 172.21.15.150:0/1655635834' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "33c732ef-20c2-43c4-abe6-045b687fc521"} : dispatch 2024-09-21T03:40:00.334 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:39:59 smithi039 ceph-mon[31211]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "33c732ef-20c2-43c4-abe6-045b687fc521"} : dispatch 2024-09-21T03:40:00.334 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:39:59 smithi039 ceph-mon[31211]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "33c732ef-20c2-43c4-abe6-045b687fc521"}]': finished 2024-09-21T03:40:00.334 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:39:59 smithi039 ceph-mon[31211]: osdmap e28: 5 total, 4 up, 5 in 2024-09-21T03:40:00.334 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:39:59 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-21T03:40:00.334 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:39:59 smithi039 ceph-mon[31211]: from='client.? 172.21.15.150:0/1783388188' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T03:40:01.218 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:40:00 smithi161 ceph-mon[34533]: pgmap v93: 1 pgs: 1 peering; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-09-21T03:40:01.218 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:40:00 smithi161 ceph-mon[34533]: overall HEALTH_OK 2024-09-21T03:40:01.224 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:40:00 smithi150 ceph-mon[34289]: pgmap v93: 1 pgs: 1 peering; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-09-21T03:40:01.224 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:40:00 smithi150 ceph-mon[34289]: overall HEALTH_OK 2024-09-21T03:40:01.333 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:40:00 smithi039 ceph-mon[31211]: pgmap v93: 1 pgs: 1 peering; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-09-21T03:40:01.334 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:40:00 smithi039 ceph-mon[31211]: overall HEALTH_OK 2024-09-21T03:40:03.224 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:40:02 smithi150 ceph-mon[34289]: pgmap v94: 1 pgs: 1 peering; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-09-21T03:40:03.333 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:40:02 smithi039 ceph-mon[31211]: pgmap v94: 1 pgs: 1 peering; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-09-21T03:40:03.467 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:40:02 smithi161 ceph-mon[34533]: pgmap v94: 1 pgs: 1 peering; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-09-21T03:40:05.224 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:40:04 smithi150 ceph-mon[34289]: pgmap v95: 1 pgs: 1 peering; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-09-21T03:40:05.333 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:40:04 smithi039 ceph-mon[31211]: pgmap v95: 1 pgs: 1 peering; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-09-21T03:40:05.467 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:40:04 smithi161 ceph-mon[34533]: pgmap v95: 1 pgs: 1 peering; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-09-21T03:40:07.142 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:40:06 smithi150 ceph-mon[34289]: pgmap v96: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail; 77 KiB/s, 0 objects/s recovering 2024-09-21T03:40:07.270 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:40:06 smithi161 ceph-mon[34533]: pgmap v96: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail; 77 KiB/s, 0 objects/s recovering 2024-09-21T03:40:07.333 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:40:06 smithi039 ceph-mon[31211]: pgmap v96: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail; 77 KiB/s, 0 objects/s recovering 2024-09-21T03:40:08.224 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:40:07 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.4"} : dispatch 2024-09-21T03:40:08.224 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:40:07 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:40:08.334 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:40:07 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.4"} : dispatch 2024-09-21T03:40:08.334 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:40:07 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:40:08.470 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:40:07 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.4"} : dispatch 2024-09-21T03:40:08.470 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:40:07 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:40:09.273 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:40:08 smithi150 ceph-mon[34289]: pgmap v97: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail; 68 KiB/s, 0 objects/s recovering 2024-09-21T03:40:09.274 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:40:08 smithi150 ceph-mon[34289]: Deploying daemon osd.4 on smithi150 2024-09-21T03:40:09.333 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:40:08 smithi039 ceph-mon[31211]: pgmap v97: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail; 68 KiB/s, 0 objects/s recovering 2024-09-21T03:40:09.334 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:40:08 smithi039 ceph-mon[31211]: Deploying daemon osd.4 on smithi150 2024-09-21T03:40:09.468 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:40:08 smithi161 ceph-mon[34533]: pgmap v97: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail; 68 KiB/s, 0 objects/s recovering 2024-09-21T03:40:09.468 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:40:08 smithi161 ceph-mon[34533]: Deploying daemon osd.4 on smithi150 2024-09-21T03:40:11.252 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:40:11 smithi150 ceph-mon[34289]: pgmap v98: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail; 56 KiB/s, 0 objects/s recovering 2024-09-21T03:40:11.334 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:40:11 smithi039 ceph-mon[31211]: pgmap v98: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail; 56 KiB/s, 0 objects/s recovering 2024-09-21T03:40:11.467 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:40:11 smithi161 ceph-mon[34533]: pgmap v98: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail; 56 KiB/s, 0 objects/s recovering 2024-09-21T03:40:13.224 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:40:13 smithi150 ceph-mon[34289]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-21T03:40:13.333 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:40:13 smithi039 ceph-mon[31211]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-21T03:40:13.468 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:40:13 smithi161 ceph-mon[34533]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-21T03:40:15.225 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:40:15 smithi150 ceph-mon[34289]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-21T03:40:15.225 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:40:15 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:40:15.225 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:40:15 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:40:15.225 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:40:15 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:40:15.333 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:40:15 smithi039 ceph-mon[31211]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-21T03:40:15.333 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:40:15 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:40:15.334 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:40:15 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:40:15.334 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:40:15 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:40:15.467 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:40:15 smithi161 ceph-mon[34533]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-21T03:40:15.468 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:40:15 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:40:15.468 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:40:15 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:40:15.468 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:40:15 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:40:17.334 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:40:17 smithi039 ceph-mon[31211]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-21T03:40:17.334 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:40:17 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:40:17.334 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:40:17 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:40:17.334 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:40:17 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:40:17.334 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:40:17 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:40:17.334 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:40:17 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:40:17.467 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:40:17 smithi161 ceph-mon[34533]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-21T03:40:17.468 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:40:17 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:40:17.468 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:40:17 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:40:17.468 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:40:17 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:40:17.468 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:40:17 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:40:17.468 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:40:17 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:40:17.474 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:40:17 smithi150 ceph-mon[34289]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-21T03:40:17.474 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:40:17 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:40:17.474 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:40:17 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:40:17.474 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:40:17 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:40:17.474 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:40:17 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:40:17.475 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:40:17 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:40:18.187 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T03:40:18.184+0000 7febe2ffd640 1 -- 172.21.15.150:0/644512146 <== mgr.14223 v2:172.21.15.39:6800/1696381982 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7febec063800 con 0x7febd80b65e0 2024-09-21T03:40:18.187 INFO:teuthology.orchestra.run.smithi150.stdout:Created osd(s) 4 on host 'smithi150' 2024-09-21T03:40:18.189 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T03:40:18.186+0000 7febf3a9d640 1 -- 172.21.15.150:0/644512146 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7febd80b65e0 msgr2=0x7febd80b8af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:18.189 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T03:40:18.186+0000 7febf3a9d640 1 --2- 172.21.15.150:0/644512146 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7febd80b65e0 0x7febd80b8af0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7febe80058c0 tx=0x7febe800a250 comp rx=0 tx=0).stop 2024-09-21T03:40:18.189 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T03:40:18.186+0000 7febf3a9d640 1 -- 172.21.15.150:0/644512146 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7febec108cf0 msgr2=0x7febec19fcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:18.190 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T03:40:18.186+0000 7febf3a9d640 1 --2- 172.21.15.150:0/644512146 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7febec108cf0 0x7febec19fcc0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7febe400da10 tx=0x7febe400ded0 comp rx=0 tx=0).stop 2024-09-21T03:40:18.190 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T03:40:18.186+0000 7febf3a9d640 1 -- 172.21.15.150:0/644512146 shutdown_connections 2024-09-21T03:40:18.190 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T03:40:18.186+0000 7febf3a9d640 1 --2- 172.21.15.150:0/644512146 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7febd80b65e0 0x7febd80b8af0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:18.190 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T03:40:18.186+0000 7febf3a9d640 1 --2- 172.21.15.150:0/644512146 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7febec108cf0 0x7febec19fcc0 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:18.190 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T03:40:18.186+0000 7febf3a9d640 1 --2- 172.21.15.150:0/644512146 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7febec1083a0 0x7febec19aa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:18.190 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T03:40:18.186+0000 7febf3a9d640 1 --2- 172.21.15.150:0/644512146 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7febec102a60 0x7febec19a480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:18.190 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T03:40:18.186+0000 7febf3a9d640 1 -- 172.21.15.150:0/644512146 >> 172.21.15.150:0/644512146 conn(0x7febec079730 msgr2=0x7febec06b2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:40:18.190 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T03:40:18.187+0000 7febf3a9d640 1 -- 172.21.15.150:0/644512146 shutdown_connections 2024-09-21T03:40:18.190 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T03:40:18.187+0000 7febf3a9d640 1 -- 172.21.15.150:0/644512146 wait complete. 2024-09-21T03:40:18.801 DEBUG:teuthology.orchestra.run.smithi150:osd.4> sudo journalctl -f -n 0 -u ceph-51116d5c-77ca-11ef-baf5-efdc52797490@osd.4.service 2024-09-21T03:40:18.803 INFO:tasks.cephadm:Deploying osd.5 on smithi161 with /dev/vg_nvme/lv_3... 2024-09-21T03:40:18.803 DEBUG:teuthology.orchestra.run.smithi161:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 51116d5c-77ca-11ef-baf5-efdc52797490 -- lvm zap /dev/vg_nvme/lv_3 2024-09-21T03:40:19.064 INFO:teuthology.orchestra.run.smithi161.stderr:Inferring config /var/lib/ceph/51116d5c-77ca-11ef-baf5-efdc52797490/mon.c/config 2024-09-21T03:40:19.185 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:40:19 smithi161 ceph-mon[34533]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-09-21T03:40:19.185 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:40:19 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:40:19.185 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:40:19 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:40:19.185 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:40:19 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:40:19.318 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:40:19 smithi150 ceph-mon[34289]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-09-21T03:40:19.318 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:40:19 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:40:19.318 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:40:19 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:40:19.318 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:40:19 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:40:19.333 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:40:19 smithi039 ceph-mon[31211]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-09-21T03:40:19.334 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:40:19 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:40:19.334 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:40:19 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:40:19.334 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:40:19 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:40:20.725 INFO:journalctl@ceph.osd.4.smithi150.stdout:Sep 21 03:40:20 smithi150 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-4[45362]: 2024-09-21T03:40:20.264+0000 7f6b1d0c9640 -1 osd.4 0 log_to_monitors true 2024-09-21T03:40:21.218 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:40:21 smithi161 ceph-mon[34533]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-09-21T03:40:21.218 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:40:21 smithi161 ceph-mon[34533]: from='osd.4 [v2:172.21.15.150:6808/3097721553,v1:172.21.15.150:6809/3097721553]' entity='osd.4' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]} : dispatch 2024-09-21T03:40:21.218 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:40:21 smithi161 ceph-mon[34533]: from='osd.4 ' entity='osd.4' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]} : dispatch 2024-09-21T03:40:21.224 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:40:21 smithi150 ceph-mon[34289]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-09-21T03:40:21.224 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:40:21 smithi150 ceph-mon[34289]: from='osd.4 [v2:172.21.15.150:6808/3097721553,v1:172.21.15.150:6809/3097721553]' entity='osd.4' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]} : dispatch 2024-09-21T03:40:21.224 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:40:21 smithi150 ceph-mon[34289]: from='osd.4 ' entity='osd.4' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]} : dispatch 2024-09-21T03:40:21.333 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:40:21 smithi039 ceph-mon[31211]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-09-21T03:40:21.334 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:40:21 smithi039 ceph-mon[31211]: from='osd.4 [v2:172.21.15.150:6808/3097721553,v1:172.21.15.150:6809/3097721553]' entity='osd.4' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]} : dispatch 2024-09-21T03:40:21.334 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:40:21 smithi039 ceph-mon[31211]: from='osd.4 ' entity='osd.4' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]} : dispatch 2024-09-21T03:40:21.634 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-09-21T03:40:21.665 DEBUG:teuthology.orchestra.run.smithi161:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 51116d5c-77ca-11ef-baf5-efdc52797490 -- ceph orch daemon add osd smithi161:vg_nvme/lv_3 2024-09-21T03:40:21.917 INFO:teuthology.orchestra.run.smithi161.stderr:Inferring config /var/lib/ceph/51116d5c-77ca-11ef-baf5-efdc52797490/mon.c/config 2024-09-21T03:40:22.301 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:40:22 smithi161 ceph-mon[34533]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-09-21T03:40:22.301 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:40:22 smithi161 ceph-mon[34533]: from='osd.4 [v2:172.21.15.150:6808/3097721553,v1:172.21.15.150:6809/3097721553]' entity='osd.4' cmd={"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi150", "root=default"]} : dispatch 2024-09-21T03:40:22.301 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:40:22 smithi161 ceph-mon[34533]: osdmap e29: 5 total, 4 up, 5 in 2024-09-21T03:40:22.301 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:40:22 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-21T03:40:22.301 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:40:22 smithi161 ceph-mon[34533]: from='osd.4 ' entity='osd.4' cmd={"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi150", "root=default"]} : dispatch 2024-09-21T03:40:22.301 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:40:22 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:40:22.301 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:40:22 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:40:22.302 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:40:22 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-09-21T03:40:22.302 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:40:22 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-09-21T03:40:22.302 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:40:22 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:40:22.302 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:40:22 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:40:22.302 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:40:22 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:40:22.302 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:40:22 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:40:22.302 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:40:22 smithi161 ceph-mon[34533]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi150", "root=default"]}]': finished 2024-09-21T03:40:22.334 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:40:22 smithi039 ceph-mon[31211]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-09-21T03:40:22.334 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:40:22 smithi039 ceph-mon[31211]: from='osd.4 [v2:172.21.15.150:6808/3097721553,v1:172.21.15.150:6809/3097721553]' entity='osd.4' cmd={"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi150", "root=default"]} : dispatch 2024-09-21T03:40:22.334 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:40:22 smithi039 ceph-mon[31211]: osdmap e29: 5 total, 4 up, 5 in 2024-09-21T03:40:22.334 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:40:22 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-21T03:40:22.334 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:40:22 smithi039 ceph-mon[31211]: from='osd.4 ' entity='osd.4' cmd={"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi150", "root=default"]} : dispatch 2024-09-21T03:40:22.334 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:40:22 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:40:22.334 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:40:22 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:40:22.334 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:40:22 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-09-21T03:40:22.334 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:40:22 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-09-21T03:40:22.334 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:40:22 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:40:22.334 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:40:22 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:40:22.335 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:40:22 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:40:22.335 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:40:22 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:40:22.335 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:40:22 smithi039 ceph-mon[31211]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi150", "root=default"]}]': finished 2024-09-21T03:40:22.475 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:40:22 smithi150 ceph-mon[34289]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-09-21T03:40:22.475 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:40:22 smithi150 ceph-mon[34289]: from='osd.4 [v2:172.21.15.150:6808/3097721553,v1:172.21.15.150:6809/3097721553]' entity='osd.4' cmd={"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi150", "root=default"]} : dispatch 2024-09-21T03:40:22.475 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:40:22 smithi150 ceph-mon[34289]: osdmap e29: 5 total, 4 up, 5 in 2024-09-21T03:40:22.475 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:40:22 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-21T03:40:22.475 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:40:22 smithi150 ceph-mon[34289]: from='osd.4 ' entity='osd.4' cmd={"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi150", "root=default"]} : dispatch 2024-09-21T03:40:22.475 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:40:22 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:40:22.475 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:40:22 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:40:22.475 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:40:22 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-09-21T03:40:22.475 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:40:22 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-09-21T03:40:22.475 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:40:22 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:40:22.475 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:40:22 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:40:22.475 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:40:22 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:40:22.476 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:40:22 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:40:22.476 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:40:22 smithi150 ceph-mon[34289]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi150", "root=default"]}]': finished 2024-09-21T03:40:22.555 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-21T03:40:22.551+0000 7f4d1a211640 1 -- 172.21.15.161:0/2850506497 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4d14109df0 msgr2=0x7f4d14110620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:22.555 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-21T03:40:22.551+0000 7f4d1a211640 1 --2- 172.21.15.161:0/2850506497 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4d14109df0 0x7f4d14110620 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f4d0c009ca0 tx=0x7f4d0c02a8c0 comp rx=0 tx=0).stop 2024-09-21T03:40:22.555 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-21T03:40:22.551+0000 7f4d1a211640 1 -- 172.21.15.161:0/2850506497 shutdown_connections 2024-09-21T03:40:22.555 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-21T03:40:22.551+0000 7f4d1a211640 1 --2- 172.21.15.161:0/2850506497 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4d14109df0 0x7f4d14110620 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:22.555 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-21T03:40:22.551+0000 7f4d1a211640 1 --2- 172.21.15.161:0/2850506497 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4d141094a0 0x7f4d141098b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:22.555 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-21T03:40:22.551+0000 7f4d1a211640 1 --2- 172.21.15.161:0/2850506497 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4d14103b60 0x7f4d14103f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:22.555 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-21T03:40:22.551+0000 7f4d1a211640 1 -- 172.21.15.161:0/2850506497 >> 172.21.15.161:0/2850506497 conn(0x7f4d1407b850 msgr2=0x7f4d1407bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:40:22.555 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-21T03:40:22.551+0000 7f4d1a211640 1 -- 172.21.15.161:0/2850506497 shutdown_connections 2024-09-21T03:40:22.555 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-21T03:40:22.551+0000 7f4d1a211640 1 -- 172.21.15.161:0/2850506497 wait complete. 2024-09-21T03:40:22.555 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-21T03:40:22.551+0000 7f4d1a211640 1 Processor -- start 2024-09-21T03:40:22.556 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-21T03:40:22.552+0000 7f4d1a211640 1 -- start start 2024-09-21T03:40:22.556 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-21T03:40:22.552+0000 7f4d1a211640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4d14103b60 0x7f4d1419a420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:22.556 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-21T03:40:22.552+0000 7f4d1a211640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4d141094a0 0x7f4d1419a9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:22.556 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-21T03:40:22.552+0000 7f4d1a211640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4d14109df0 0x7f4d1419fc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:22.556 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-21T03:40:22.552+0000 7f4d1a211640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f4d14113ab0 con 0x7f4d141094a0 2024-09-21T03:40:22.556 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-21T03:40:22.552+0000 7f4d1a211640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f4d14113930 con 0x7f4d14103b60 2024-09-21T03:40:22.556 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-21T03:40:22.552+0000 7f4d1a211640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f4d14113c30 con 0x7f4d14109df0 2024-09-21T03:40:22.556 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-21T03:40:22.552+0000 7f4d137fe640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4d14103b60 0x7f4d1419a420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:22.556 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-21T03:40:22.552+0000 7f4d13fff640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4d14109df0 0x7f4d1419fc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:22.556 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-21T03:40:22.552+0000 7f4d137fe640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4d14103b60 0x7f4d1419a420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.161:39018/0 (socket says 172.21.15.161:39018) 2024-09-21T03:40:22.556 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-21T03:40:22.552+0000 7f4d137fe640 1 -- 172.21.15.161:0/133786579 learned_addr learned my addr 172.21.15.161:0/133786579 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-09-21T03:40:22.556 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-21T03:40:22.552+0000 7f4d0bfff640 1 --2- 172.21.15.161:0/133786579 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4d141094a0 0x7f4d1419a9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:22.556 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-21T03:40:22.552+0000 7f4d137fe640 1 -- 172.21.15.161:0/133786579 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4d14109df0 msgr2=0x7f4d1419fc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:22.556 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-21T03:40:22.552+0000 7f4d137fe640 1 --2- 172.21.15.161:0/133786579 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4d14109df0 0x7f4d1419fc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:22.557 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-21T03:40:22.552+0000 7f4d137fe640 1 -- 172.21.15.161:0/133786579 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4d141094a0 msgr2=0x7f4d1419a9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:22.557 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-21T03:40:22.552+0000 7f4d137fe640 1 --2- 172.21.15.161:0/133786579 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4d141094a0 0x7f4d1419a9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:22.557 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-21T03:40:22.552+0000 7f4d137fe640 1 -- 172.21.15.161:0/133786579 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4d141a0390 con 0x7f4d14103b60 2024-09-21T03:40:22.557 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-21T03:40:22.552+0000 7f4d13fff640 1 --2- 172.21.15.161:0/133786579 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4d14109df0 0x7f4d1419fc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:40:22.557 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-21T03:40:22.553+0000 7f4d137fe640 1 --2- 172.21.15.161:0/133786579 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4d14103b60 0x7f4d1419a420 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f4d0400d950 tx=0x7f4d0400de10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:40:22.557 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-21T03:40:22.553+0000 7f4d117fa640 1 -- 172.21.15.161:0/133786579 <== mon.2 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d04017030 con 0x7f4d14103b60 2024-09-21T03:40:22.557 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-21T03:40:22.553+0000 7f4d117fa640 1 -- 172.21.15.161:0/133786579 <== mon.2 v2:172.21.15.150:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4d04004020 con 0x7f4d14103b60 2024-09-21T03:40:22.557 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-21T03:40:22.553+0000 7f4d117fa640 1 -- 172.21.15.161:0/133786579 <== mon.2 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d040047a0 con 0x7f4d14103b60 2024-09-21T03:40:22.557 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-21T03:40:22.553+0000 7f4d1a211640 1 -- 172.21.15.161:0/133786579 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4d141a0680 con 0x7f4d14103b60 2024-09-21T03:40:22.557 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-21T03:40:22.553+0000 7f4d1a211640 1 -- 172.21.15.161:0/133786579 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4d140755f0 con 0x7f4d14103b60 2024-09-21T03:40:22.558 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-21T03:40:22.554+0000 7f4d1a211640 1 -- 172.21.15.161:0/133786579 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4d14103f70 con 0x7f4d14103b60 2024-09-21T03:40:22.558 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-21T03:40:22.556+0000 7f4d117fa640 1 -- 172.21.15.161:0/133786579 <== mon.2 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 21) ==== 154529+0+0 (secure 0 0 0) 0x7f4d04009a30 con 0x7f4d14103b60 2024-09-21T03:40:22.558 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-21T03:40:22.556+0000 7f4d117fa640 1 --2- 172.21.15.161:0/133786579 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f4d000b65e0 0x7f4d000b8af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:22.559 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-21T03:40:22.556+0000 7f4d117fa640 1 -- 172.21.15.161:0/133786579 <== mon.2 v2:172.21.15.150:3300/0 5 ==== osd_map(30..30 src has 1..30) ==== 4036+0+0 (secure 0 0 0) 0x7f4d040e63a0 con 0x7f4d14103b60 2024-09-21T03:40:22.559 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-21T03:40:22.557+0000 7f4d0bfff640 1 --2- 172.21.15.161:0/133786579 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f4d000b65e0 0x7f4d000b8af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:22.559 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-21T03:40:22.557+0000 7f4d0bfff640 1 --2- 172.21.15.161:0/133786579 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f4d000b65e0 0x7f4d000b8af0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f4d14108330 tx=0x7f4cfc00b410 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:40:22.563 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-21T03:40:22.562+0000 7f4d117fa640 1 -- 172.21.15.161:0/133786579 <== mon.2 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f4d040adcc0 con 0x7f4d14103b60 2024-09-21T03:40:22.752 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-21T03:40:22.750+0000 7f4d1a211640 1 -- 172.21.15.161:0/133786579 --> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi161:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7f4d14063800 con 0x7f4d000b65e0 2024-09-21T03:40:23.224 INFO:journalctl@ceph.osd.4.smithi150.stdout:Sep 21 03:40:22 smithi150 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-4[45362]: 2024-09-21T03:40:22.965+0000 7f6b1904b640 -1 osd.4 0 waiting for initial osdmap 2024-09-21T03:40:23.224 INFO:journalctl@ceph.osd.4.smithi150.stdout:Sep 21 03:40:23 smithi150 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-4[45362]: 2024-09-21T03:40:23.065+0000 7f6b10e41640 -1 osd.4 30 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-21T03:40:23.225 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:40:23 smithi150 ceph-mon[34289]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-09-21T03:40:23.225 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:40:23 smithi150 ceph-mon[34289]: Detected new or changed devices on smithi150 2024-09-21T03:40:23.225 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:40:23 smithi150 ceph-mon[34289]: Adjusting osd_memory_target on smithi150 to 8544M 2024-09-21T03:40:23.225 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:40:23 smithi150 ceph-mon[34289]: osdmap e30: 5 total, 4 up, 5 in 2024-09-21T03:40:23.225 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:40:23 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-21T03:40:23.225 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:40:23 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T03:40:23.225 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:40:23 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T03:40:23.225 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:40:23 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:40:23.225 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:40:23 smithi150 ceph-mon[34289]: from='osd.4 ' entity='osd.4' 2024-09-21T03:40:23.225 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:40:23 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-21T03:40:23.334 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:40:23 smithi039 ceph-mon[31211]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-09-21T03:40:23.334 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:40:23 smithi039 ceph-mon[31211]: Detected new or changed devices on smithi150 2024-09-21T03:40:23.334 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:40:23 smithi039 ceph-mon[31211]: Adjusting osd_memory_target on smithi150 to 8544M 2024-09-21T03:40:23.334 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:40:23 smithi039 ceph-mon[31211]: osdmap e30: 5 total, 4 up, 5 in 2024-09-21T03:40:23.334 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:40:23 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-21T03:40:23.334 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:40:23 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T03:40:23.334 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:40:23 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T03:40:23.334 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:40:23 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:40:23.334 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:40:23 smithi039 ceph-mon[31211]: from='osd.4 ' entity='osd.4' 2024-09-21T03:40:23.334 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:40:23 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-21T03:40:23.386 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:40:23 smithi161 ceph-mon[34533]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-09-21T03:40:23.387 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:40:23 smithi161 ceph-mon[34533]: Detected new or changed devices on smithi150 2024-09-21T03:40:23.387 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:40:23 smithi161 ceph-mon[34533]: Adjusting osd_memory_target on smithi150 to 8544M 2024-09-21T03:40:23.387 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:40:23 smithi161 ceph-mon[34533]: osdmap e30: 5 total, 4 up, 5 in 2024-09-21T03:40:23.387 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:40:23 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-21T03:40:23.387 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:40:23 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T03:40:23.387 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:40:23 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T03:40:23.387 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:40:23 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:40:23.387 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:40:23 smithi161 ceph-mon[34533]: from='osd.4 ' entity='osd.4' 2024-09-21T03:40:23.387 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:40:23 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-21T03:40:24.377 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:40:24 smithi161 ceph-mon[34533]: purged_snaps scrub starts 2024-09-21T03:40:24.377 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:40:24 smithi161 ceph-mon[34533]: purged_snaps scrub ok 2024-09-21T03:40:24.377 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:40:24 smithi161 ceph-mon[34533]: from='client.24410 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi161:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:40:24.377 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:40:24 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-21T03:40:24.474 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:40:24 smithi150 ceph-mon[34289]: purged_snaps scrub starts 2024-09-21T03:40:24.474 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:40:24 smithi150 ceph-mon[34289]: purged_snaps scrub ok 2024-09-21T03:40:24.475 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:40:24 smithi150 ceph-mon[34289]: from='client.24410 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi161:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:40:24.475 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:40:24 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-21T03:40:24.584 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:40:24 smithi039 ceph-mon[31211]: purged_snaps scrub starts 2024-09-21T03:40:24.584 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:40:24 smithi039 ceph-mon[31211]: purged_snaps scrub ok 2024-09-21T03:40:24.584 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:40:24 smithi039 ceph-mon[31211]: from='client.24410 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi161:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:40:24.584 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:40:24 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-21T03:40:25.361 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:40:25 smithi161 ceph-mon[34533]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-09-21T03:40:25.362 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:40:25 smithi161 ceph-mon[34533]: osd.4 [v2:172.21.15.150:6808/3097721553,v1:172.21.15.150:6809/3097721553] boot 2024-09-21T03:40:25.362 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:40:25 smithi161 ceph-mon[34533]: osdmap e31: 5 total, 5 up, 5 in 2024-09-21T03:40:25.362 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:40:25 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-21T03:40:25.476 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:40:25 smithi150 ceph-mon[34289]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-09-21T03:40:25.477 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:40:25 smithi150 ceph-mon[34289]: osd.4 [v2:172.21.15.150:6808/3097721553,v1:172.21.15.150:6809/3097721553] boot 2024-09-21T03:40:25.477 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:40:25 smithi150 ceph-mon[34289]: osdmap e31: 5 total, 5 up, 5 in 2024-09-21T03:40:25.477 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:40:25 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-21T03:40:25.586 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:40:25 smithi039 ceph-mon[31211]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-09-21T03:40:25.586 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:40:25 smithi039 ceph-mon[31211]: osd.4 [v2:172.21.15.150:6808/3097721553,v1:172.21.15.150:6809/3097721553] boot 2024-09-21T03:40:25.586 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:40:25 smithi039 ceph-mon[31211]: osdmap e31: 5 total, 5 up, 5 in 2024-09-21T03:40:25.586 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:40:25 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-21T03:40:26.469 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:40:26 smithi161 ceph-mon[34533]: osdmap e32: 5 total, 5 up, 5 in 2024-09-21T03:40:26.469 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:40:26 smithi161 ceph-mon[34533]: from='client.? 172.21.15.161:0/2464274126' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "74673722-abd6-400a-bb50-ec3dfba0c5e9"} : dispatch 2024-09-21T03:40:26.469 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:40:26 smithi161 ceph-mon[34533]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "74673722-abd6-400a-bb50-ec3dfba0c5e9"} : dispatch 2024-09-21T03:40:26.469 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:40:26 smithi161 ceph-mon[34533]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "74673722-abd6-400a-bb50-ec3dfba0c5e9"}]': finished 2024-09-21T03:40:26.469 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:40:26 smithi161 ceph-mon[34533]: osdmap e33: 6 total, 5 up, 6 in 2024-09-21T03:40:26.469 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:40:26 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-21T03:40:26.469 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:40:26 smithi161 ceph-mon[34533]: from='client.? 172.21.15.161:0/1291974554' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T03:40:26.474 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:40:26 smithi150 ceph-mon[34289]: osdmap e32: 5 total, 5 up, 5 in 2024-09-21T03:40:26.474 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:40:26 smithi150 ceph-mon[34289]: from='client.? 172.21.15.161:0/2464274126' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "74673722-abd6-400a-bb50-ec3dfba0c5e9"} : dispatch 2024-09-21T03:40:26.475 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:40:26 smithi150 ceph-mon[34289]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "74673722-abd6-400a-bb50-ec3dfba0c5e9"} : dispatch 2024-09-21T03:40:26.475 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:40:26 smithi150 ceph-mon[34289]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "74673722-abd6-400a-bb50-ec3dfba0c5e9"}]': finished 2024-09-21T03:40:26.475 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:40:26 smithi150 ceph-mon[34289]: osdmap e33: 6 total, 5 up, 6 in 2024-09-21T03:40:26.475 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:40:26 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-21T03:40:26.475 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:40:26 smithi150 ceph-mon[34289]: from='client.? 172.21.15.161:0/1291974554' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T03:40:26.584 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:40:26 smithi039 ceph-mon[31211]: osdmap e32: 5 total, 5 up, 5 in 2024-09-21T03:40:26.584 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:40:26 smithi039 ceph-mon[31211]: from='client.? 172.21.15.161:0/2464274126' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "74673722-abd6-400a-bb50-ec3dfba0c5e9"} : dispatch 2024-09-21T03:40:26.584 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:40:26 smithi039 ceph-mon[31211]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "74673722-abd6-400a-bb50-ec3dfba0c5e9"} : dispatch 2024-09-21T03:40:26.584 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:40:26 smithi039 ceph-mon[31211]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "74673722-abd6-400a-bb50-ec3dfba0c5e9"}]': finished 2024-09-21T03:40:26.584 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:40:26 smithi039 ceph-mon[31211]: osdmap e33: 6 total, 5 up, 6 in 2024-09-21T03:40:26.584 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:40:26 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-21T03:40:26.584 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:40:26 smithi039 ceph-mon[31211]: from='client.? 172.21.15.161:0/1291974554' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T03:40:27.470 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:40:27 smithi161 ceph-mon[34533]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-09-21T03:40:27.474 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:40:27 smithi150 ceph-mon[34289]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-09-21T03:40:27.583 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:40:27 smithi039 ceph-mon[31211]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-09-21T03:40:29.467 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:40:29 smithi161 ceph-mon[34533]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-09-21T03:40:29.474 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:40:29 smithi150 ceph-mon[34289]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-09-21T03:40:29.583 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:40:29 smithi039 ceph-mon[31211]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-09-21T03:40:31.468 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:40:31 smithi161 ceph-mon[34533]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-09-21T03:40:31.474 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:40:31 smithi150 ceph-mon[34289]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-09-21T03:40:31.584 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:40:31 smithi039 ceph-mon[31211]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-09-21T03:40:33.461 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:40:33 smithi161 ceph-mon[34533]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-09-21T03:40:33.474 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:40:33 smithi150 ceph-mon[34289]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-09-21T03:40:33.584 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:40:33 smithi039 ceph-mon[31211]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-09-21T03:40:34.467 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:40:34 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.5"} : dispatch 2024-09-21T03:40:34.468 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:40:34 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:40:34.474 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:40:34 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.5"} : dispatch 2024-09-21T03:40:34.474 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:40:34 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:40:34.583 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:40:34 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.5"} : dispatch 2024-09-21T03:40:34.583 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:40:34 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:40:35.467 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:40:35 smithi161 ceph-mon[34533]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-09-21T03:40:35.468 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:40:35 smithi161 ceph-mon[34533]: Deploying daemon osd.5 on smithi161 2024-09-21T03:40:35.474 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:40:35 smithi150 ceph-mon[34289]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-09-21T03:40:35.474 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:40:35 smithi150 ceph-mon[34289]: Deploying daemon osd.5 on smithi161 2024-09-21T03:40:35.583 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:40:35 smithi039 ceph-mon[31211]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-09-21T03:40:35.583 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:40:35 smithi039 ceph-mon[31211]: Deploying daemon osd.5 on smithi161 2024-09-21T03:40:37.468 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:40:37 smithi161 ceph-mon[34533]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-09-21T03:40:37.474 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:40:37 smithi150 ceph-mon[34289]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-09-21T03:40:37.583 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:40:37 smithi039 ceph-mon[31211]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-09-21T03:40:39.468 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:40:39 smithi161 ceph-mon[34533]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-09-21T03:40:39.474 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:40:39 smithi150 ceph-mon[34289]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-09-21T03:40:39.583 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:40:39 smithi039 ceph-mon[31211]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-09-21T03:40:40.468 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:40:40 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:40:40.468 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:40:40 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:40:40.468 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:40:40 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:40:40.474 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:40:40 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:40:40.474 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:40:40 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:40:40.474 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:40:40 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:40:40.583 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:40:40 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:40:40.584 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:40:40 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:40:40.584 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:40:40 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:40:41.468 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:40:41 smithi161 ceph-mon[34533]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-09-21T03:40:41.478 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:40:41 smithi150 ceph-mon[34289]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-09-21T03:40:41.585 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:40:41 smithi039 ceph-mon[31211]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-09-21T03:40:43.468 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:40:43 smithi161 ceph-mon[34533]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-09-21T03:40:43.468 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:40:43 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:40:43.468 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:40:43 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:40:43.468 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:40:43 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:40:43.468 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:40:43 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:40:43.468 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:40:43 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:40:43.474 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:40:43 smithi150 ceph-mon[34289]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-09-21T03:40:43.474 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:40:43 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:40:43.475 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:40:43 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:40:43.475 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:40:43 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:40:43.475 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:40:43 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:40:43.475 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:40:43 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:40:43.583 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:40:43 smithi039 ceph-mon[31211]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-09-21T03:40:43.584 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:40:43 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:40:43.584 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:40:43 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:40:43.584 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:40:43 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:40:43.584 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:40:43 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:40:43.584 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:40:43 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:40:44.160 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-21T03:40:44.157+0000 7f4d117fa640 1 -- 172.21.15.161:0/133786579 <== mgr.14223 v2:172.21.15.39:6800/1696381982 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f4d14063800 con 0x7f4d000b65e0 2024-09-21T03:40:44.160 INFO:teuthology.orchestra.run.smithi161.stdout:Created osd(s) 5 on host 'smithi161' 2024-09-21T03:40:44.162 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-21T03:40:44.160+0000 7f4d1a211640 1 -- 172.21.15.161:0/133786579 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f4d000b65e0 msgr2=0x7f4d000b8af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:44.162 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-21T03:40:44.160+0000 7f4d1a211640 1 --2- 172.21.15.161:0/133786579 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f4d000b65e0 0x7f4d000b8af0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f4d14108330 tx=0x7f4cfc00b410 comp rx=0 tx=0).stop 2024-09-21T03:40:44.162 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-21T03:40:44.160+0000 7f4d1a211640 1 -- 172.21.15.161:0/133786579 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4d14103b60 msgr2=0x7f4d1419a420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:44.162 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-21T03:40:44.160+0000 7f4d1a211640 1 --2- 172.21.15.161:0/133786579 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4d14103b60 0x7f4d1419a420 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f4d0400d950 tx=0x7f4d0400de10 comp rx=0 tx=0).stop 2024-09-21T03:40:44.162 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-21T03:40:44.160+0000 7f4d1a211640 1 -- 172.21.15.161:0/133786579 shutdown_connections 2024-09-21T03:40:44.162 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-21T03:40:44.160+0000 7f4d1a211640 1 --2- 172.21.15.161:0/133786579 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f4d000b65e0 0x7f4d000b8af0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:44.162 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-21T03:40:44.160+0000 7f4d1a211640 1 --2- 172.21.15.161:0/133786579 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4d14109df0 0x7f4d1419fc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:44.162 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-21T03:40:44.160+0000 7f4d1a211640 1 --2- 172.21.15.161:0/133786579 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4d141094a0 0x7f4d1419a9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:44.162 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-21T03:40:44.160+0000 7f4d1a211640 1 --2- 172.21.15.161:0/133786579 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4d14103b60 0x7f4d1419a420 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:44.163 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-21T03:40:44.160+0000 7f4d1a211640 1 -- 172.21.15.161:0/133786579 >> 172.21.15.161:0/133786579 conn(0x7f4d1407b850 msgr2=0x7f4d14105430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:40:44.163 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-21T03:40:44.160+0000 7f4d1a211640 1 -- 172.21.15.161:0/133786579 shutdown_connections 2024-09-21T03:40:44.163 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-21T03:40:44.160+0000 7f4d1a211640 1 -- 172.21.15.161:0/133786579 wait complete. 2024-09-21T03:40:44.468 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:40:44 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:40:44.468 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:40:44 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:40:44.468 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:40:44 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:40:44.474 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:40:44 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:40:44.474 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:40:44 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:40:44.474 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:40:44 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:40:44.584 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:40:44 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:40:44.584 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:40:44 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:40:44.584 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:40:44 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:40:44.783 DEBUG:teuthology.orchestra.run.smithi161:osd.5> sudo journalctl -f -n 0 -u ceph-51116d5c-77ca-11ef-baf5-efdc52797490@osd.5.service 2024-09-21T03:40:44.786 INFO:tasks.cephadm:Deploying osd.6 on smithi039 with /dev/vg_nvme/lv_2... 2024-09-21T03:40:44.786 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 51116d5c-77ca-11ef-baf5-efdc52797490 -- lvm zap /dev/vg_nvme/lv_2 2024-09-21T03:40:45.055 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/51116d5c-77ca-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:40:45.446 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:40:45 smithi039 ceph-mon[31211]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-09-21T03:40:45.467 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:40:45 smithi161 ceph-mon[34533]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-09-21T03:40:45.474 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:40:45 smithi150 ceph-mon[34289]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-09-21T03:40:46.396 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:40:46 smithi161 ceph-mon[34533]: from='osd.5 [v2:172.21.15.161:6808/742751023,v1:172.21.15.161:6809/742751023]' entity='osd.5' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]} : dispatch 2024-09-21T03:40:46.396 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:40:46 smithi161 ceph-mon[34533]: from='osd.5 ' entity='osd.5' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]} : dispatch 2024-09-21T03:40:46.396 INFO:journalctl@ceph.osd.5.smithi161.stdout:Sep 21 03:40:46 smithi161 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-5[45969]: 2024-09-21T03:40:46.147+0000 7fab308f1640 -1 osd.5 0 log_to_monitors true 2024-09-21T03:40:46.583 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:40:46 smithi039 ceph-mon[31211]: from='osd.5 [v2:172.21.15.161:6808/742751023,v1:172.21.15.161:6809/742751023]' entity='osd.5' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]} : dispatch 2024-09-21T03:40:46.584 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:40:46 smithi039 ceph-mon[31211]: from='osd.5 ' entity='osd.5' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]} : dispatch 2024-09-21T03:40:46.724 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:40:46 smithi150 ceph-mon[34289]: from='osd.5 [v2:172.21.15.161:6808/742751023,v1:172.21.15.161:6809/742751023]' entity='osd.5' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]} : dispatch 2024-09-21T03:40:46.725 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:40:46 smithi150 ceph-mon[34289]: from='osd.5 ' entity='osd.5' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]} : dispatch 2024-09-21T03:40:47.441 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:40:47 smithi039 ceph-mon[31211]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-09-21T03:40:47.442 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:40:47 smithi039 ceph-mon[31211]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-09-21T03:40:47.442 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:40:47 smithi039 ceph-mon[31211]: from='osd.5 [v2:172.21.15.161:6808/742751023,v1:172.21.15.161:6809/742751023]' entity='osd.5' cmd={"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi161", "root=default"]} : dispatch 2024-09-21T03:40:47.442 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:40:47 smithi039 ceph-mon[31211]: osdmap e34: 6 total, 5 up, 6 in 2024-09-21T03:40:47.442 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:40:47 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-21T03:40:47.442 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:40:47 smithi039 ceph-mon[31211]: from='osd.5 ' entity='osd.5' cmd={"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi161", "root=default"]} : dispatch 2024-09-21T03:40:47.451 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:40:47 smithi161 ceph-mon[34533]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-09-21T03:40:47.451 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:40:47 smithi161 ceph-mon[34533]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-09-21T03:40:47.451 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:40:47 smithi161 ceph-mon[34533]: from='osd.5 [v2:172.21.15.161:6808/742751023,v1:172.21.15.161:6809/742751023]' entity='osd.5' cmd={"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi161", "root=default"]} : dispatch 2024-09-21T03:40:47.451 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:40:47 smithi161 ceph-mon[34533]: osdmap e34: 6 total, 5 up, 6 in 2024-09-21T03:40:47.451 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:40:47 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-21T03:40:47.451 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:40:47 smithi161 ceph-mon[34533]: from='osd.5 ' entity='osd.5' cmd={"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi161", "root=default"]} : dispatch 2024-09-21T03:40:47.724 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:40:47 smithi150 ceph-mon[34289]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-09-21T03:40:47.725 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:40:47 smithi150 ceph-mon[34289]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-09-21T03:40:47.725 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:40:47 smithi150 ceph-mon[34289]: from='osd.5 [v2:172.21.15.161:6808/742751023,v1:172.21.15.161:6809/742751023]' entity='osd.5' cmd={"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi161", "root=default"]} : dispatch 2024-09-21T03:40:47.725 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:40:47 smithi150 ceph-mon[34289]: osdmap e34: 6 total, 5 up, 6 in 2024-09-21T03:40:47.725 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:40:47 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-21T03:40:47.725 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:40:47 smithi150 ceph-mon[34289]: from='osd.5 ' entity='osd.5' cmd={"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi161", "root=default"]} : dispatch 2024-09-21T03:40:47.771 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T03:40:47.799 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 51116d5c-77ca-11ef-baf5-efdc52797490 -- ceph orch daemon add osd smithi039:vg_nvme/lv_2 2024-09-21T03:40:48.077 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/51116d5c-77ca-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:40:48.467 INFO:journalctl@ceph.osd.5.smithi161.stdout:Sep 21 03:40:48 smithi161 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-5[45969]: 2024-09-21T03:40:48.166+0000 7fab2c873640 -1 osd.5 0 waiting for initial osdmap 2024-09-21T03:40:48.468 INFO:journalctl@ceph.osd.5.smithi161.stdout:Sep 21 03:40:48 smithi161 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-5[45969]: 2024-09-21T03:40:48.328+0000 7fab24669640 -1 osd.5 35 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-21T03:40:48.468 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:40:48 smithi161 ceph-mon[34533]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi161", "root=default"]}]': finished 2024-09-21T03:40:48.468 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:40:48 smithi161 ceph-mon[34533]: osdmap e35: 6 total, 5 up, 6 in 2024-09-21T03:40:48.468 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:40:48 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-21T03:40:48.468 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:40:48 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-21T03:40:48.468 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:40:48 smithi161 ceph-mon[34533]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-09-21T03:40:48.468 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:40:48 smithi161 ceph-mon[34533]: Detected new or changed devices on smithi161 2024-09-21T03:40:48.468 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:40:48 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:40:48.468 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:40:48 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:40:48.469 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:40:48 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-09-21T03:40:48.469 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:40:48 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-09-21T03:40:48.469 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:40:48 smithi161 ceph-mon[34533]: Adjusting osd_memory_target on smithi161 to 8544M 2024-09-21T03:40:48.469 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:40:48 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:40:48.469 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:40:48 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:40:48.469 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:40:48 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:40:48.469 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:40:48 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:40:48.469 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:40:48 smithi161 ceph-mon[34533]: from='osd.5 ' entity='osd.5' 2024-09-21T03:40:48.469 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:40:48 smithi161 ceph-mon[34533]: osdmap e36: 6 total, 5 up, 6 in 2024-09-21T03:40:48.469 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:40:48 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-21T03:40:48.469 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:40:48 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-21T03:40:48.471 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:40:48 smithi039 ceph-mon[31211]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi161", "root=default"]}]': finished 2024-09-21T03:40:48.471 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:40:48 smithi039 ceph-mon[31211]: osdmap e35: 6 total, 5 up, 6 in 2024-09-21T03:40:48.471 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:40:48 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-21T03:40:48.471 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:40:48 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-21T03:40:48.471 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:40:48 smithi039 ceph-mon[31211]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-09-21T03:40:48.472 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:40:48 smithi039 ceph-mon[31211]: Detected new or changed devices on smithi161 2024-09-21T03:40:48.472 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:40:48 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:40:48.472 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:40:48 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:40:48.472 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:40:48 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-09-21T03:40:48.472 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:40:48 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-09-21T03:40:48.472 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:40:48 smithi039 ceph-mon[31211]: Adjusting osd_memory_target on smithi161 to 8544M 2024-09-21T03:40:48.472 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:40:48 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:40:48.472 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:40:48 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:40:48.472 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:40:48 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:40:48.472 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:40:48 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:40:48.472 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:40:48 smithi039 ceph-mon[31211]: from='osd.5 ' entity='osd.5' 2024-09-21T03:40:48.472 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:40:48 smithi039 ceph-mon[31211]: osdmap e36: 6 total, 5 up, 6 in 2024-09-21T03:40:48.472 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:40:48 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-21T03:40:48.472 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:40:48 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-21T03:40:48.614 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:40:48.566+0000 7fd545bb2640 1 -- 172.21.15.39:0/1191244028 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd540105380 msgr2=0x7fd54010d8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:48.614 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:40:48.566+0000 7fd545bb2640 1 --2- 172.21.15.39:0/1191244028 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd540105380 0x7fd54010d8d0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fd528009900 tx=0x7fd52802a3f0 comp rx=0 tx=0).stop 2024-09-21T03:40:48.614 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:40:48.566+0000 7fd545bb2640 1 -- 172.21.15.39:0/1191244028 shutdown_connections 2024-09-21T03:40:48.614 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:40:48.566+0000 7fd545bb2640 1 --2- 172.21.15.39:0/1191244028 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd54010de60 0x7fd540110330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:48.614 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:40:48.566+0000 7fd545bb2640 1 --2- 172.21.15.39:0/1191244028 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd540105380 0x7fd54010d8d0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:48.614 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:40:48.566+0000 7fd545bb2640 1 --2- 172.21.15.39:0/1191244028 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd540104a30 0x7fd540104e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:48.614 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:40:48.566+0000 7fd545bb2640 1 -- 172.21.15.39:0/1191244028 >> 172.21.15.39:0/1191244028 conn(0x7fd540079760 msgr2=0x7fd540079b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:40:48.614 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:40:48.566+0000 7fd545bb2640 1 -- 172.21.15.39:0/1191244028 shutdown_connections 2024-09-21T03:40:48.614 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:40:48.566+0000 7fd545bb2640 1 -- 172.21.15.39:0/1191244028 wait complete. 2024-09-21T03:40:48.614 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:40:48.567+0000 7fd545bb2640 1 Processor -- start 2024-09-21T03:40:48.615 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:40:48.567+0000 7fd545bb2640 1 -- start start 2024-09-21T03:40:48.615 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:40:48.567+0000 7fd545bb2640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd540104a30 0x7fd54019a4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:48.615 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:40:48.568+0000 7fd545bb2640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd540105380 0x7fd54019aa70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:48.615 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:40:48.568+0000 7fd545bb2640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd54010de60 0x7fd54019fd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:48.615 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:40:48.568+0000 7fd545bb2640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fd540113aa0 con 0x7fd54010de60 2024-09-21T03:40:48.615 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:40:48.568+0000 7fd545bb2640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fd540113920 con 0x7fd540104a30 2024-09-21T03:40:48.615 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:40:48.568+0000 7fd545bb2640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fd540113c20 con 0x7fd540105380 2024-09-21T03:40:48.615 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:40:48.568+0000 7fd53ffff640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd54010de60 0x7fd54019fd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:48.615 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:40:48.568+0000 7fd53ffff640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd54010de60 0x7fd54019fd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:47392/0 (socket says 172.21.15.39:47392) 2024-09-21T03:40:48.615 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:40:48.568+0000 7fd53ffff640 1 -- 172.21.15.39:0/2841192207 learned_addr learned my addr 172.21.15.39:0/2841192207 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:40:48.615 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:40:48.568+0000 7fd53f7fe640 1 --2- 172.21.15.39:0/2841192207 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd540104a30 0x7fd54019a4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:48.615 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:40:48.568+0000 7fd536ffd640 1 --2- 172.21.15.39:0/2841192207 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd540105380 0x7fd54019aa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:48.615 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:40:48.568+0000 7fd53ffff640 1 -- 172.21.15.39:0/2841192207 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd540105380 msgr2=0x7fd54019aa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:48.615 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:40:48.568+0000 7fd53ffff640 1 --2- 172.21.15.39:0/2841192207 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd540105380 0x7fd54019aa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:48.616 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:40:48.568+0000 7fd53ffff640 1 -- 172.21.15.39:0/2841192207 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd540104a30 msgr2=0x7fd54019a4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:48.616 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:40:48.568+0000 7fd53ffff640 1 --2- 172.21.15.39:0/2841192207 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd540104a30 0x7fd54019a4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:48.616 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:40:48.568+0000 7fd53ffff640 1 -- 172.21.15.39:0/2841192207 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd5401a0440 con 0x7fd54010de60 2024-09-21T03:40:48.616 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:40:48.568+0000 7fd536ffd640 1 --2- 172.21.15.39:0/2841192207 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd540105380 0x7fd54019aa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:40:48.616 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:40:48.568+0000 7fd53ffff640 1 --2- 172.21.15.39:0/2841192207 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd54010de60 0x7fd54019fd10 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fd53800cad0 tx=0x7fd53800cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:40:48.616 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:40:48.568+0000 7fd53d7fa640 1 -- 172.21.15.39:0/2841192207 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd53800ae20 con 0x7fd54010de60 2024-09-21T03:40:48.616 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:40:48.568+0000 7fd545bb2640 1 -- 172.21.15.39:0/2841192207 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd5401a0730 con 0x7fd54010de60 2024-09-21T03:40:48.616 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:40:48.568+0000 7fd53d7fa640 1 -- 172.21.15.39:0/2841192207 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd538005020 con 0x7fd54010de60 2024-09-21T03:40:48.616 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:40:48.568+0000 7fd53d7fa640 1 -- 172.21.15.39:0/2841192207 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd53800f6b0 con 0x7fd54010de60 2024-09-21T03:40:48.616 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:40:48.569+0000 7fd545bb2640 1 -- 172.21.15.39:0/2841192207 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd5401a6e90 con 0x7fd54010de60 2024-09-21T03:40:48.616 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:40:48.570+0000 7fd545bb2640 1 -- 172.21.15.39:0/2841192207 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd54006c1f0 con 0x7fd54010de60 2024-09-21T03:40:48.616 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:40:48.571+0000 7fd53d7fa640 1 -- 172.21.15.39:0/2841192207 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 21) ==== 154529+0+0 (secure 0 0 0) 0x7fd5380051d0 con 0x7fd54010de60 2024-09-21T03:40:48.616 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:40:48.572+0000 7fd53d7fa640 1 --2- 172.21.15.39:0/2841192207 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7fd52c0b6560 0x7fd52c0b8a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:48.616 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:40:48.572+0000 7fd53f7fe640 1 --2- 172.21.15.39:0/2841192207 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7fd52c0b6560 0x7fd52c0b8a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:48.617 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:40:48.572+0000 7fd53d7fa640 1 -- 172.21.15.39:0/2841192207 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 4501+0+0 (secure 0 0 0) 0x7fd5380e6940 con 0x7fd54010de60 2024-09-21T03:40:48.617 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:40:48.572+0000 7fd53f7fe640 1 --2- 172.21.15.39:0/2841192207 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7fd52c0b6560 0x7fd52c0b8a70 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fd530005fd0 tx=0x7fd530005850 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:40:48.617 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:40:48.575+0000 7fd53d7fa640 1 -- 172.21.15.39:0/2841192207 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fd538014030 con 0x7fd54010de60 2024-09-21T03:40:48.725 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:40:48 smithi150 ceph-mon[34289]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi161", "root=default"]}]': finished 2024-09-21T03:40:48.725 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:40:48 smithi150 ceph-mon[34289]: osdmap e35: 6 total, 5 up, 6 in 2024-09-21T03:40:48.725 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:40:48 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-21T03:40:48.725 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:40:48 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-21T03:40:48.725 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:40:48 smithi150 ceph-mon[34289]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-09-21T03:40:48.725 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:40:48 smithi150 ceph-mon[34289]: Detected new or changed devices on smithi161 2024-09-21T03:40:48.725 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:40:48 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:40:48.725 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:40:48 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:40:48.725 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:40:48 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-09-21T03:40:48.725 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:40:48 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-09-21T03:40:48.725 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:40:48 smithi150 ceph-mon[34289]: Adjusting osd_memory_target on smithi161 to 8544M 2024-09-21T03:40:48.726 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:40:48 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:40:48.726 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:40:48 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:40:48.726 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:40:48 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:40:48.726 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:40:48 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:40:48.726 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:40:48 smithi150 ceph-mon[34289]: from='osd.5 ' entity='osd.5' 2024-09-21T03:40:48.726 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:40:48 smithi150 ceph-mon[34289]: osdmap e36: 6 total, 5 up, 6 in 2024-09-21T03:40:48.726 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:40:48 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-21T03:40:48.726 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:40:48 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-21T03:40:48.777 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:40:48.775+0000 7fd545bb2640 1 -- 172.21.15.39:0/2841192207 --> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi039:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7fd540063800 con 0x7fd52c0b6560 2024-09-21T03:40:49.483 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:40:49 smithi039 ceph-mon[31211]: purged_snaps scrub starts 2024-09-21T03:40:49.483 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:40:49 smithi039 ceph-mon[31211]: purged_snaps scrub ok 2024-09-21T03:40:49.483 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:40:49 smithi039 ceph-mon[31211]: from='client.14610 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi039:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:40:49.484 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:40:49 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T03:40:49.484 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:40:49 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T03:40:49.484 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:40:49 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:40:49.484 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:40:49 smithi039 ceph-mon[31211]: osd.5 [v2:172.21.15.161:6808/742751023,v1:172.21.15.161:6809/742751023] boot 2024-09-21T03:40:49.484 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:40:49 smithi039 ceph-mon[31211]: osdmap e37: 6 total, 6 up, 6 in 2024-09-21T03:40:49.484 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:40:49 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-21T03:40:49.577 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:40:49 smithi161 ceph-mon[34533]: purged_snaps scrub starts 2024-09-21T03:40:49.577 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:40:49 smithi161 ceph-mon[34533]: purged_snaps scrub ok 2024-09-21T03:40:49.577 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:40:49 smithi161 ceph-mon[34533]: from='client.14610 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi039:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:40:49.577 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:40:49 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T03:40:49.577 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:40:49 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T03:40:49.577 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:40:49 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:40:49.577 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:40:49 smithi161 ceph-mon[34533]: osd.5 [v2:172.21.15.161:6808/742751023,v1:172.21.15.161:6809/742751023] boot 2024-09-21T03:40:49.577 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:40:49 smithi161 ceph-mon[34533]: osdmap e37: 6 total, 6 up, 6 in 2024-09-21T03:40:49.577 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:40:49 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-21T03:40:49.725 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:40:49 smithi150 ceph-mon[34289]: purged_snaps scrub starts 2024-09-21T03:40:49.725 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:40:49 smithi150 ceph-mon[34289]: purged_snaps scrub ok 2024-09-21T03:40:49.725 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:40:49 smithi150 ceph-mon[34289]: from='client.14610 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi039:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:40:49.725 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:40:49 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T03:40:49.725 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:40:49 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T03:40:49.725 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:40:49 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:40:49.725 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:40:49 smithi150 ceph-mon[34289]: osd.5 [v2:172.21.15.161:6808/742751023,v1:172.21.15.161:6809/742751023] boot 2024-09-21T03:40:49.725 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:40:49 smithi150 ceph-mon[34289]: osdmap e37: 6 total, 6 up, 6 in 2024-09-21T03:40:49.725 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:40:49 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-21T03:40:50.834 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:40:50 smithi039 ceph-mon[31211]: pgmap v127: 1 pgs: 1 remapped+peering; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-09-21T03:40:50.834 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:40:50 smithi039 ceph-mon[31211]: osdmap e38: 6 total, 6 up, 6 in 2024-09-21T03:40:50.968 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:40:50 smithi161 ceph-mon[34533]: pgmap v127: 1 pgs: 1 remapped+peering; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-09-21T03:40:50.968 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:40:50 smithi161 ceph-mon[34533]: osdmap e38: 6 total, 6 up, 6 in 2024-09-21T03:40:50.974 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:40:50 smithi150 ceph-mon[34289]: pgmap v127: 1 pgs: 1 remapped+peering; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-09-21T03:40:50.974 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:40:50 smithi150 ceph-mon[34289]: osdmap e38: 6 total, 6 up, 6 in 2024-09-21T03:40:51.834 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:40:51 smithi039 ceph-mon[31211]: osdmap e39: 6 total, 6 up, 6 in 2024-09-21T03:40:51.834 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:40:51 smithi039 ceph-mon[31211]: from='client.? 172.21.15.39:0/1620251178' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "06548677-74c4-4efa-8054-df09b957b649"} : dispatch 2024-09-21T03:40:51.834 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:40:51 smithi039 ceph-mon[31211]: from='client.? 172.21.15.39:0/1620251178' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "06548677-74c4-4efa-8054-df09b957b649"}]': finished 2024-09-21T03:40:51.834 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:40:51 smithi039 ceph-mon[31211]: osdmap e40: 7 total, 6 up, 7 in 2024-09-21T03:40:51.834 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:40:51 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-21T03:40:51.968 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:40:51 smithi161 ceph-mon[34533]: osdmap e39: 6 total, 6 up, 6 in 2024-09-21T03:40:51.968 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:40:51 smithi161 ceph-mon[34533]: from='client.? 172.21.15.39:0/1620251178' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "06548677-74c4-4efa-8054-df09b957b649"} : dispatch 2024-09-21T03:40:51.968 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:40:51 smithi161 ceph-mon[34533]: from='client.? 172.21.15.39:0/1620251178' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "06548677-74c4-4efa-8054-df09b957b649"}]': finished 2024-09-21T03:40:51.968 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:40:51 smithi161 ceph-mon[34533]: osdmap e40: 7 total, 6 up, 7 in 2024-09-21T03:40:51.968 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:40:51 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-21T03:40:51.974 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:40:51 smithi150 ceph-mon[34289]: osdmap e39: 6 total, 6 up, 6 in 2024-09-21T03:40:51.974 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:40:51 smithi150 ceph-mon[34289]: from='client.? 172.21.15.39:0/1620251178' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "06548677-74c4-4efa-8054-df09b957b649"} : dispatch 2024-09-21T03:40:51.974 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:40:51 smithi150 ceph-mon[34289]: from='client.? 172.21.15.39:0/1620251178' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "06548677-74c4-4efa-8054-df09b957b649"}]': finished 2024-09-21T03:40:51.974 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:40:51 smithi150 ceph-mon[34289]: osdmap e40: 7 total, 6 up, 7 in 2024-09-21T03:40:51.974 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:40:51 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-21T03:40:53.083 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:40:52 smithi039 ceph-mon[31211]: pgmap v131: 1 pgs: 1 remapped+peering; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-09-21T03:40:53.084 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:40:52 smithi039 ceph-mon[31211]: from='client.? 172.21.15.39:0/140522316' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T03:40:53.217 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:40:52 smithi161 ceph-mon[34533]: pgmap v131: 1 pgs: 1 remapped+peering; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-09-21T03:40:53.217 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:40:52 smithi161 ceph-mon[34533]: from='client.? 172.21.15.39:0/140522316' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T03:40:53.224 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:40:52 smithi150 ceph-mon[34289]: pgmap v131: 1 pgs: 1 remapped+peering; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-09-21T03:40:53.224 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:40:52 smithi150 ceph-mon[34289]: from='client.? 172.21.15.39:0/140522316' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T03:40:55.083 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:40:54 smithi039 ceph-mon[31211]: pgmap v132: 1 pgs: 1 remapped+peering; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-09-21T03:40:55.217 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:40:54 smithi161 ceph-mon[34533]: pgmap v132: 1 pgs: 1 remapped+peering; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-09-21T03:40:55.224 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:40:54 smithi150 ceph-mon[34289]: pgmap v132: 1 pgs: 1 remapped+peering; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-09-21T03:40:57.084 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:40:56 smithi039 ceph-mon[31211]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-21T03:40:57.217 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:40:56 smithi161 ceph-mon[34533]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-21T03:40:57.224 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:40:56 smithi150 ceph-mon[34289]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-21T03:40:59.083 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:40:58 smithi039 ceph-mon[31211]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail; 74 KiB/s, 0 objects/s recovering 2024-09-21T03:40:59.217 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:40:58 smithi161 ceph-mon[34533]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail; 74 KiB/s, 0 objects/s recovering 2024-09-21T03:40:59.224 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:40:58 smithi150 ceph-mon[34289]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail; 74 KiB/s, 0 objects/s recovering 2024-09-21T03:41:01.068 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:41:00 smithi039 ceph-mon[31211]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail; 65 KiB/s, 0 objects/s recovering 2024-09-21T03:41:01.218 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:41:00 smithi161 ceph-mon[34533]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail; 65 KiB/s, 0 objects/s recovering 2024-09-21T03:41:01.224 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:41:00 smithi150 ceph-mon[34289]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail; 65 KiB/s, 0 objects/s recovering 2024-09-21T03:41:02.084 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:41:01 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.6"} : dispatch 2024-09-21T03:41:02.084 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:41:01 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:41:02.217 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:41:01 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.6"} : dispatch 2024-09-21T03:41:02.217 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:41:01 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:41:02.224 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:41:01 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.6"} : dispatch 2024-09-21T03:41:02.224 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:41:01 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:41:03.179 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:41:02 smithi039 ceph-mon[31211]: Deploying daemon osd.6 on smithi039 2024-09-21T03:41:03.180 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:41:02 smithi039 ceph-mon[31211]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail; 57 KiB/s, 0 objects/s recovering 2024-09-21T03:41:03.217 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:41:02 smithi161 ceph-mon[34533]: Deploying daemon osd.6 on smithi039 2024-09-21T03:41:03.217 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:41:02 smithi161 ceph-mon[34533]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail; 57 KiB/s, 0 objects/s recovering 2024-09-21T03:41:03.224 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:41:02 smithi150 ceph-mon[34289]: Deploying daemon osd.6 on smithi039 2024-09-21T03:41:03.224 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:41:02 smithi150 ceph-mon[34289]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail; 57 KiB/s, 0 objects/s recovering 2024-09-21T03:41:05.084 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:41:04 smithi039 ceph-mon[31211]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-21T03:41:05.217 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:41:04 smithi161 ceph-mon[34533]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-21T03:41:05.224 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:41:04 smithi150 ceph-mon[34289]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-21T03:41:07.062 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:41:06 smithi039 ceph-mon[31211]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-21T03:41:07.217 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:41:06 smithi161 ceph-mon[34533]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-21T03:41:07.224 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:41:06 smithi150 ceph-mon[34289]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-21T03:41:07.909 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:41:07 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:41:07.909 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:41:07 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:41:07.909 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:41:07 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:41:08.141 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:41:07 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:41:08.141 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:41:07 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:41:08.141 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:41:07 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:41:08.224 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:41:07 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:41:08.225 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:41:07 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:41:08.225 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:41:07 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:41:09.126 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:41:08 smithi150 ceph-mon[34289]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-09-21T03:41:09.144 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:41:08 smithi039 ceph-mon[31211]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-09-21T03:41:09.218 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:41:08 smithi161 ceph-mon[34533]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-09-21T03:41:11.084 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:41:10 smithi039 ceph-mon[31211]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-09-21T03:41:11.084 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:41:10 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:41:11.084 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:41:10 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:41:11.084 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:41:10 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:41:11.084 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:41:10 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:41:11.084 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:41:10 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:41:11.217 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:41:10 smithi161 ceph-mon[34533]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-09-21T03:41:11.218 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:41:10 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:41:11.218 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:41:10 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:41:11.218 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:41:10 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:41:11.218 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:41:10 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:41:11.218 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:41:10 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:41:11.224 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:41:10 smithi150 ceph-mon[34289]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-09-21T03:41:11.224 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:41:10 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:41:11.224 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:41:10 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:41:11.224 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:41:10 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:41:11.225 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:41:10 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:41:11.225 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:41:10 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:41:11.576 INFO:teuthology.orchestra.run.smithi039.stdout:Created osd(s) 6 on host 'smithi039' 2024-09-21T03:41:11.576 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:41:11.573+0000 7fd53d7fa640 1 -- 172.21.15.39:0/2841192207 <== mgr.14223 v2:172.21.15.39:6800/1696381982 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fd540063800 con 0x7fd52c0b6560 2024-09-21T03:41:11.579 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:41:11.576+0000 7fd545bb2640 1 -- 172.21.15.39:0/2841192207 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7fd52c0b6560 msgr2=0x7fd52c0b8a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:11.579 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:41:11.576+0000 7fd545bb2640 1 --2- 172.21.15.39:0/2841192207 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7fd52c0b6560 0x7fd52c0b8a70 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fd530005fd0 tx=0x7fd530005850 comp rx=0 tx=0).stop 2024-09-21T03:41:11.579 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:41:11.576+0000 7fd545bb2640 1 -- 172.21.15.39:0/2841192207 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd54010de60 msgr2=0x7fd54019fd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:11.579 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:41:11.576+0000 7fd545bb2640 1 --2- 172.21.15.39:0/2841192207 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd54010de60 0x7fd54019fd10 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fd53800cad0 tx=0x7fd53800cf90 comp rx=0 tx=0).stop 2024-09-21T03:41:11.579 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:41:11.576+0000 7fd545bb2640 1 -- 172.21.15.39:0/2841192207 shutdown_connections 2024-09-21T03:41:11.579 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:41:11.576+0000 7fd545bb2640 1 --2- 172.21.15.39:0/2841192207 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7fd52c0b6560 0x7fd52c0b8a70 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:11.579 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:41:11.576+0000 7fd545bb2640 1 --2- 172.21.15.39:0/2841192207 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd54010de60 0x7fd54019fd10 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:11.579 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:41:11.576+0000 7fd545bb2640 1 --2- 172.21.15.39:0/2841192207 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd540105380 0x7fd54019aa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:11.579 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:41:11.576+0000 7fd545bb2640 1 --2- 172.21.15.39:0/2841192207 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd540104a30 0x7fd54019a4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:11.579 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:41:11.576+0000 7fd545bb2640 1 -- 172.21.15.39:0/2841192207 >> 172.21.15.39:0/2841192207 conn(0x7fd540079760 msgr2=0x7fd5401008f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:41:11.580 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:41:11.576+0000 7fd545bb2640 1 -- 172.21.15.39:0/2841192207 shutdown_connections 2024-09-21T03:41:11.580 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:41:11.576+0000 7fd545bb2640 1 -- 172.21.15.39:0/2841192207 wait complete. 2024-09-21T03:41:12.148 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:41:11 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:41:12.162 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:41:11 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:41:12.162 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:41:11 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:41:12.182 DEBUG:teuthology.orchestra.run.smithi039:osd.6> sudo journalctl -f -n 0 -u ceph-51116d5c-77ca-11ef-baf5-efdc52797490@osd.6.service 2024-09-21T03:41:12.184 INFO:tasks.cephadm:Deploying osd.7 on smithi150 with /dev/vg_nvme/lv_2... 2024-09-21T03:41:12.184 DEBUG:teuthology.orchestra.run.smithi150:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 51116d5c-77ca-11ef-baf5-efdc52797490 -- lvm zap /dev/vg_nvme/lv_2 2024-09-21T03:41:12.212 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:41:11 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:41:12.212 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:41:11 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:41:12.213 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:41:11 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:41:12.217 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:41:11 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:41:12.217 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:41:11 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:41:12.217 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:41:11 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:41:12.437 INFO:teuthology.orchestra.run.smithi150.stderr:Inferring config /var/lib/ceph/51116d5c-77ca-11ef-baf5-efdc52797490/mon.b/config 2024-09-21T03:41:13.084 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:41:12 smithi039 ceph-mon[31211]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-09-21T03:41:13.217 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:41:12 smithi161 ceph-mon[34533]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-09-21T03:41:13.224 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:41:12 smithi150 ceph-mon[34289]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-09-21T03:41:13.753 INFO:journalctl@ceph.osd.6.smithi039.stdout:Sep 21 03:41:13 smithi039 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-6[60520]: 2024-09-21T03:41:13.470+0000 7f98af913640 -1 osd.6 0 log_to_monitors true 2024-09-21T03:41:14.084 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:41:13 smithi039 ceph-mon[31211]: from='osd.6 [v2:172.21.15.39:6818/1072700290,v1:172.21.15.39:6819/1072700290]' entity='osd.6' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]} : dispatch 2024-09-21T03:41:14.140 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:41:13 smithi150 ceph-mon[34289]: from='osd.6 [v2:172.21.15.39:6818/1072700290,v1:172.21.15.39:6819/1072700290]' entity='osd.6' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]} : dispatch 2024-09-21T03:41:14.217 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:41:13 smithi161 ceph-mon[34533]: from='osd.6 [v2:172.21.15.39:6818/1072700290,v1:172.21.15.39:6819/1072700290]' entity='osd.6' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]} : dispatch 2024-09-21T03:41:14.835 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-09-21T03:41:14.866 DEBUG:teuthology.orchestra.run.smithi150:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 51116d5c-77ca-11ef-baf5-efdc52797490 -- ceph orch daemon add osd smithi150:vg_nvme/lv_2 2024-09-21T03:41:15.083 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:41:14 smithi039 ceph-mon[31211]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-09-21T03:41:15.084 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:41:14 smithi039 ceph-mon[31211]: from='osd.6 [v2:172.21.15.39:6818/1072700290,v1:172.21.15.39:6819/1072700290]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-09-21T03:41:15.084 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:41:14 smithi039 ceph-mon[31211]: osdmap e41: 7 total, 6 up, 7 in 2024-09-21T03:41:15.084 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:41:14 smithi039 ceph-mon[31211]: from='osd.6 [v2:172.21.15.39:6818/1072700290,v1:172.21.15.39:6819/1072700290]' entity='osd.6' cmd={"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi039", "root=default"]} : dispatch 2024-09-21T03:41:15.084 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:41:14 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-21T03:41:15.084 INFO:journalctl@ceph.osd.6.smithi039.stdout:Sep 21 03:41:14 smithi039 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-6[60520]: 2024-09-21T03:41:14.907+0000 7f98ab895640 -1 osd.6 0 waiting for initial osdmap 2024-09-21T03:41:15.084 INFO:journalctl@ceph.osd.6.smithi039.stdout:Sep 21 03:41:14 smithi039 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-6[60520]: 2024-09-21T03:41:14.918+0000 7f98a7693640 -1 osd.6 42 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-21T03:41:15.159 INFO:teuthology.orchestra.run.smithi150.stderr:Inferring config /var/lib/ceph/51116d5c-77ca-11ef-baf5-efdc52797490/mon.b/config 2024-09-21T03:41:15.217 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:41:14 smithi161 ceph-mon[34533]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-09-21T03:41:15.218 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:41:14 smithi161 ceph-mon[34533]: from='osd.6 [v2:172.21.15.39:6818/1072700290,v1:172.21.15.39:6819/1072700290]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-09-21T03:41:15.218 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:41:14 smithi161 ceph-mon[34533]: osdmap e41: 7 total, 6 up, 7 in 2024-09-21T03:41:15.218 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:41:14 smithi161 ceph-mon[34533]: from='osd.6 [v2:172.21.15.39:6818/1072700290,v1:172.21.15.39:6819/1072700290]' entity='osd.6' cmd={"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi039", "root=default"]} : dispatch 2024-09-21T03:41:15.218 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:41:14 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-21T03:41:15.224 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:41:14 smithi150 ceph-mon[34289]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-09-21T03:41:15.224 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:41:14 smithi150 ceph-mon[34289]: from='osd.6 [v2:172.21.15.39:6818/1072700290,v1:172.21.15.39:6819/1072700290]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-09-21T03:41:15.224 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:41:14 smithi150 ceph-mon[34289]: osdmap e41: 7 total, 6 up, 7 in 2024-09-21T03:41:15.224 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:41:14 smithi150 ceph-mon[34289]: from='osd.6 [v2:172.21.15.39:6818/1072700290,v1:172.21.15.39:6819/1072700290]' entity='osd.6' cmd={"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi039", "root=default"]} : dispatch 2024-09-21T03:41:15.224 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:41:14 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-21T03:41:15.663 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T03:41:15.661+0000 7f077907a640 1 -- 172.21.15.150:0/291335174 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0774109df0 msgr2=0x7f0774110620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:15.664 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T03:41:15.661+0000 7f077907a640 1 --2- 172.21.15.150:0/291335174 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0774109df0 0x7f0774110620 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f076c009ca0 tx=0x7f076c02a8c0 comp rx=0 tx=0).stop 2024-09-21T03:41:15.664 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T03:41:15.661+0000 7f077907a640 1 -- 172.21.15.150:0/291335174 shutdown_connections 2024-09-21T03:41:15.664 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T03:41:15.661+0000 7f077907a640 1 --2- 172.21.15.150:0/291335174 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0774109df0 0x7f0774110620 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:15.664 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T03:41:15.661+0000 7f077907a640 1 --2- 172.21.15.150:0/291335174 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f07741094a0 0x7f07741098b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:15.664 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T03:41:15.661+0000 7f077907a640 1 --2- 172.21.15.150:0/291335174 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0774103b60 0x7f0774103f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:15.664 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T03:41:15.661+0000 7f077907a640 1 -- 172.21.15.150:0/291335174 >> 172.21.15.150:0/291335174 conn(0x7f077407b850 msgr2=0x7f077407bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:41:15.664 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T03:41:15.661+0000 7f077907a640 1 -- 172.21.15.150:0/291335174 shutdown_connections 2024-09-21T03:41:15.664 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T03:41:15.661+0000 7f077907a640 1 -- 172.21.15.150:0/291335174 wait complete. 2024-09-21T03:41:15.664 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T03:41:15.662+0000 7f077907a640 1 Processor -- start 2024-09-21T03:41:15.664 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T03:41:15.662+0000 7f077907a640 1 -- start start 2024-09-21T03:41:15.666 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T03:41:15.662+0000 7f077907a640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0774103b60 0x7f077419a4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:15.666 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T03:41:15.662+0000 7f077907a640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f07741094a0 0x7f077419aa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:15.666 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T03:41:15.662+0000 7f077907a640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0774109df0 0x7f077419fd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:15.666 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T03:41:15.662+0000 7f077907a640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f0774113ab0 con 0x7f07741094a0 2024-09-21T03:41:15.666 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T03:41:15.662+0000 7f077907a640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f0774113930 con 0x7f0774103b60 2024-09-21T03:41:15.667 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T03:41:15.662+0000 7f077907a640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f0774113c30 con 0x7f0774109df0 2024-09-21T03:41:15.667 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T03:41:15.662+0000 7f0772d9d640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0774103b60 0x7f077419a4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:15.667 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T03:41:15.662+0000 7f0772d9d640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0774103b60 0x7f077419a4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:42738/0 (socket says 172.21.15.150:42738) 2024-09-21T03:41:15.667 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T03:41:15.662+0000 7f0772d9d640 1 -- 172.21.15.150:0/320270229 learned_addr learned my addr 172.21.15.150:0/320270229 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-09-21T03:41:15.667 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T03:41:15.662+0000 7f076bfff640 1 --2- 172.21.15.150:0/320270229 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f07741094a0 0x7f077419aa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:15.667 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T03:41:15.662+0000 7f077359e640 1 --2- 172.21.15.150:0/320270229 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0774109df0 0x7f077419fd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:15.667 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T03:41:15.663+0000 7f0772d9d640 1 -- 172.21.15.150:0/320270229 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0774109df0 msgr2=0x7f077419fd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:15.667 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T03:41:15.663+0000 7f0772d9d640 1 --2- 172.21.15.150:0/320270229 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0774109df0 0x7f077419fd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:15.667 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T03:41:15.663+0000 7f0772d9d640 1 -- 172.21.15.150:0/320270229 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f07741094a0 msgr2=0x7f077419aa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:15.667 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T03:41:15.663+0000 7f0772d9d640 1 --2- 172.21.15.150:0/320270229 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f07741094a0 0x7f077419aa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:15.667 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T03:41:15.663+0000 7f0772d9d640 1 -- 172.21.15.150:0/320270229 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f07741a0450 con 0x7f0774103b60 2024-09-21T03:41:15.667 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T03:41:15.663+0000 7f0772d9d640 1 --2- 172.21.15.150:0/320270229 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0774103b60 0x7f077419a4e0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f076400e9f0 tx=0x7f076400edf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:41:15.668 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T03:41:15.663+0000 7f0770d99640 1 -- 172.21.15.150:0/320270229 <== mon.2 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f076400cd80 con 0x7f0774103b60 2024-09-21T03:41:15.668 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T03:41:15.663+0000 7f077907a640 1 -- 172.21.15.150:0/320270229 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f07741a0740 con 0x7f0774103b60 2024-09-21T03:41:15.668 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T03:41:15.663+0000 7f0770d99640 1 -- 172.21.15.150:0/320270229 <== mon.2 v2:172.21.15.150:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f07640054d0 con 0x7f0774103b60 2024-09-21T03:41:15.668 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T03:41:15.663+0000 7f0770d99640 1 -- 172.21.15.150:0/320270229 <== mon.2 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0764010690 con 0x7f0774103b60 2024-09-21T03:41:15.668 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T03:41:15.663+0000 7f077907a640 1 -- 172.21.15.150:0/320270229 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f07741a6ea0 con 0x7f0774103b60 2024-09-21T03:41:15.668 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T03:41:15.664+0000 7f077907a640 1 -- 172.21.15.150:0/320270229 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0774103f70 con 0x7f0774103b60 2024-09-21T03:41:15.669 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T03:41:15.666+0000 7f0770d99640 1 -- 172.21.15.150:0/320270229 <== mon.2 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 21) ==== 154529+0+0 (secure 0 0 0) 0x7f0764004e30 con 0x7f0774103b60 2024-09-21T03:41:15.669 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T03:41:15.666+0000 7f0770d99640 1 --2- 172.21.15.150:0/320270229 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f07600b6560 0x7f07600b8a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:15.669 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T03:41:15.666+0000 7f0770d99640 1 -- 172.21.15.150:0/320270229 <== mon.2 v2:172.21.15.150:3300/0 5 ==== osd_map(42..42 src has 1..42) ==== 4900+0+0 (secure 0 0 0) 0x7f07640e6bd0 con 0x7f0774103b60 2024-09-21T03:41:15.669 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T03:41:15.666+0000 7f076bfff640 1 --2- 172.21.15.150:0/320270229 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f07600b6560 0x7f07600b8a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:15.669 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T03:41:15.667+0000 7f076bfff640 1 --2- 172.21.15.150:0/320270229 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f07600b6560 0x7f07600b8a70 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f07741074b0 tx=0x7f075c00b390 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:41:15.671 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T03:41:15.669+0000 7f0770d99640 1 -- 172.21.15.150:0/320270229 <== mon.2 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f07640ae1b0 con 0x7f0774103b60 2024-09-21T03:41:15.860 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T03:41:15.858+0000 7f077907a640 1 -- 172.21.15.150:0/320270229 --> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi150:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7f0774063800 con 0x7f07600b6560 2024-09-21T03:41:16.218 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:41:15 smithi161 ceph-mon[34533]: from='osd.6 [v2:172.21.15.39:6818/1072700290,v1:172.21.15.39:6819/1072700290]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi039", "root=default"]}]': finished 2024-09-21T03:41:16.218 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:41:15 smithi161 ceph-mon[34533]: osdmap e42: 7 total, 6 up, 7 in 2024-09-21T03:41:16.218 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:41:15 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-21T03:41:16.218 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:41:15 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-21T03:41:16.218 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:41:15 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:41:16.218 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:41:15 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:41:16.218 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:41:15 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-21T03:41:16.218 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:41:15 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"} : dispatch 2024-09-21T03:41:16.218 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:41:15 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"} : dispatch 2024-09-21T03:41:16.218 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:41:15 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:41:16.218 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:41:15 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:41:16.219 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:41:15 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:41:16.219 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:41:15 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:41:16.219 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:41:15 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T03:41:16.219 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:41:15 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T03:41:16.219 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:41:15 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:41:16.224 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:41:15 smithi150 ceph-mon[34289]: from='osd.6 [v2:172.21.15.39:6818/1072700290,v1:172.21.15.39:6819/1072700290]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi039", "root=default"]}]': finished 2024-09-21T03:41:16.225 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:41:15 smithi150 ceph-mon[34289]: osdmap e42: 7 total, 6 up, 7 in 2024-09-21T03:41:16.225 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:41:15 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-21T03:41:16.225 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:41:15 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-21T03:41:16.225 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:41:15 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:41:16.225 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:41:15 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:41:16.225 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:41:15 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-21T03:41:16.225 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:41:15 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"} : dispatch 2024-09-21T03:41:16.225 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:41:15 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"} : dispatch 2024-09-21T03:41:16.225 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:41:15 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:41:16.225 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:41:15 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:41:16.225 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:41:15 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:41:16.225 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:41:15 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:41:16.225 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:41:15 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T03:41:16.226 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:41:15 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T03:41:16.226 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:41:15 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:41:16.334 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:41:15 smithi039 ceph-mon[31211]: from='osd.6 [v2:172.21.15.39:6818/1072700290,v1:172.21.15.39:6819/1072700290]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi039", "root=default"]}]': finished 2024-09-21T03:41:16.334 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:41:15 smithi039 ceph-mon[31211]: osdmap e42: 7 total, 6 up, 7 in 2024-09-21T03:41:16.334 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:41:15 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-21T03:41:16.334 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:41:15 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-21T03:41:16.334 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:41:15 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:41:16.334 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:41:15 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:41:16.334 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:41:15 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-21T03:41:16.334 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:41:15 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"} : dispatch 2024-09-21T03:41:16.335 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:41:15 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"} : dispatch 2024-09-21T03:41:16.335 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:41:15 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:41:16.335 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:41:15 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:41:16.335 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:41:15 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:41:16.335 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:41:15 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:41:16.335 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:41:15 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T03:41:16.335 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:41:15 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T03:41:16.335 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:41:15 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:41:17.218 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:41:16 smithi161 ceph-mon[34533]: purged_snaps scrub starts 2024-09-21T03:41:17.218 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:41:16 smithi161 ceph-mon[34533]: purged_snaps scrub ok 2024-09-21T03:41:17.218 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:41:16 smithi161 ceph-mon[34533]: Detected new or changed devices on smithi039 2024-09-21T03:41:17.218 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:41:16 smithi161 ceph-mon[34533]: Adjusting osd_memory_target on smithi039 to 5696M 2024-09-21T03:41:17.218 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:41:16 smithi161 ceph-mon[34533]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-09-21T03:41:17.218 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:41:16 smithi161 ceph-mon[34533]: from='client.24449 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi150:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:41:17.218 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:41:16 smithi161 ceph-mon[34533]: osd.6 [v2:172.21.15.39:6818/1072700290,v1:172.21.15.39:6819/1072700290] boot 2024-09-21T03:41:17.218 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:41:16 smithi161 ceph-mon[34533]: osdmap e43: 7 total, 7 up, 7 in 2024-09-21T03:41:17.218 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:41:16 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-21T03:41:17.225 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:41:16 smithi150 ceph-mon[34289]: purged_snaps scrub starts 2024-09-21T03:41:17.225 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:41:16 smithi150 ceph-mon[34289]: purged_snaps scrub ok 2024-09-21T03:41:17.225 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:41:16 smithi150 ceph-mon[34289]: Detected new or changed devices on smithi039 2024-09-21T03:41:17.225 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:41:16 smithi150 ceph-mon[34289]: Adjusting osd_memory_target on smithi039 to 5696M 2024-09-21T03:41:17.225 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:41:16 smithi150 ceph-mon[34289]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-09-21T03:41:17.225 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:41:16 smithi150 ceph-mon[34289]: from='client.24449 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi150:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:41:17.225 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:41:16 smithi150 ceph-mon[34289]: osd.6 [v2:172.21.15.39:6818/1072700290,v1:172.21.15.39:6819/1072700290] boot 2024-09-21T03:41:17.225 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:41:16 smithi150 ceph-mon[34289]: osdmap e43: 7 total, 7 up, 7 in 2024-09-21T03:41:17.225 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:41:16 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-21T03:41:17.333 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:41:16 smithi039 ceph-mon[31211]: purged_snaps scrub starts 2024-09-21T03:41:17.334 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:41:16 smithi039 ceph-mon[31211]: purged_snaps scrub ok 2024-09-21T03:41:17.334 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:41:16 smithi039 ceph-mon[31211]: Detected new or changed devices on smithi039 2024-09-21T03:41:17.334 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:41:16 smithi039 ceph-mon[31211]: Adjusting osd_memory_target on smithi039 to 5696M 2024-09-21T03:41:17.334 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:41:16 smithi039 ceph-mon[31211]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-09-21T03:41:17.334 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:41:16 smithi039 ceph-mon[31211]: from='client.24449 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi150:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:41:17.334 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:41:16 smithi039 ceph-mon[31211]: osd.6 [v2:172.21.15.39:6818/1072700290,v1:172.21.15.39:6819/1072700290] boot 2024-09-21T03:41:17.334 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:41:16 smithi039 ceph-mon[31211]: osdmap e43: 7 total, 7 up, 7 in 2024-09-21T03:41:17.334 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:41:16 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-21T03:41:18.718 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:41:18 smithi161 ceph-mon[34533]: osdmap e44: 7 total, 7 up, 7 in 2024-09-21T03:41:18.718 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:41:18 smithi161 ceph-mon[34533]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 201 MiB used, 626 GiB / 626 GiB avail 2024-09-21T03:41:18.718 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:41:18 smithi161 ceph-mon[34533]: from='client.? 172.21.15.150:0/2273693064' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "838735ca-c693-4f58-9426-b8ee857dbe3a"} : dispatch 2024-09-21T03:41:18.718 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:41:18 smithi161 ceph-mon[34533]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "838735ca-c693-4f58-9426-b8ee857dbe3a"} : dispatch 2024-09-21T03:41:18.718 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:41:18 smithi161 ceph-mon[34533]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "838735ca-c693-4f58-9426-b8ee857dbe3a"}]': finished 2024-09-21T03:41:18.718 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:41:18 smithi161 ceph-mon[34533]: osdmap e45: 8 total, 7 up, 8 in 2024-09-21T03:41:18.718 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:41:18 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-09-21T03:41:18.724 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:41:18 smithi150 ceph-mon[34289]: osdmap e44: 7 total, 7 up, 7 in 2024-09-21T03:41:18.724 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:41:18 smithi150 ceph-mon[34289]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 201 MiB used, 626 GiB / 626 GiB avail 2024-09-21T03:41:18.725 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:41:18 smithi150 ceph-mon[34289]: from='client.? 172.21.15.150:0/2273693064' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "838735ca-c693-4f58-9426-b8ee857dbe3a"} : dispatch 2024-09-21T03:41:18.725 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:41:18 smithi150 ceph-mon[34289]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "838735ca-c693-4f58-9426-b8ee857dbe3a"} : dispatch 2024-09-21T03:41:18.725 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:41:18 smithi150 ceph-mon[34289]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "838735ca-c693-4f58-9426-b8ee857dbe3a"}]': finished 2024-09-21T03:41:18.725 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:41:18 smithi150 ceph-mon[34289]: osdmap e45: 8 total, 7 up, 8 in 2024-09-21T03:41:18.725 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:41:18 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-09-21T03:41:18.834 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:41:18 smithi039 ceph-mon[31211]: osdmap e44: 7 total, 7 up, 7 in 2024-09-21T03:41:18.834 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:41:18 smithi039 ceph-mon[31211]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 201 MiB used, 626 GiB / 626 GiB avail 2024-09-21T03:41:18.834 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:41:18 smithi039 ceph-mon[31211]: from='client.? 172.21.15.150:0/2273693064' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "838735ca-c693-4f58-9426-b8ee857dbe3a"} : dispatch 2024-09-21T03:41:18.834 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:41:18 smithi039 ceph-mon[31211]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "838735ca-c693-4f58-9426-b8ee857dbe3a"} : dispatch 2024-09-21T03:41:18.834 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:41:18 smithi039 ceph-mon[31211]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "838735ca-c693-4f58-9426-b8ee857dbe3a"}]': finished 2024-09-21T03:41:18.834 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:41:18 smithi039 ceph-mon[31211]: osdmap e45: 8 total, 7 up, 8 in 2024-09-21T03:41:18.834 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:41:18 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-09-21T03:41:19.717 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:41:19 smithi161 ceph-mon[34533]: from='client.? 172.21.15.150:0/3763730963' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T03:41:19.724 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:41:19 smithi150 ceph-mon[34289]: from='client.? 172.21.15.150:0/3763730963' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T03:41:19.833 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:41:19 smithi039 ceph-mon[31211]: from='client.? 172.21.15.150:0/3763730963' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T03:41:20.718 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:41:20 smithi161 ceph-mon[34533]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 201 MiB used, 626 GiB / 626 GiB avail 2024-09-21T03:41:20.724 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:41:20 smithi150 ceph-mon[34289]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 201 MiB used, 626 GiB / 626 GiB avail 2024-09-21T03:41:20.833 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:41:20 smithi039 ceph-mon[31211]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 201 MiB used, 626 GiB / 626 GiB avail 2024-09-21T03:41:22.833 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:41:22 smithi039 ceph-mon[31211]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 202 MiB used, 626 GiB / 626 GiB avail 2024-09-21T03:41:22.967 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:41:22 smithi161 ceph-mon[34533]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 202 MiB used, 626 GiB / 626 GiB avail 2024-09-21T03:41:22.974 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:41:22 smithi150 ceph-mon[34289]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 202 MiB used, 626 GiB / 626 GiB avail 2024-09-21T03:41:24.701 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:41:24 smithi150 ceph-mon[34289]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 202 MiB used, 626 GiB / 626 GiB avail 2024-09-21T03:41:24.833 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:41:24 smithi039 ceph-mon[31211]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 202 MiB used, 626 GiB / 626 GiB avail 2024-09-21T03:41:24.967 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:41:24 smithi161 ceph-mon[34533]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 202 MiB used, 626 GiB / 626 GiB avail 2024-09-21T03:41:26.814 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:41:26 smithi150 ceph-mon[34289]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 202 MiB used, 626 GiB / 626 GiB avail 2024-09-21T03:41:26.833 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:41:26 smithi039 ceph-mon[31211]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 202 MiB used, 626 GiB / 626 GiB avail 2024-09-21T03:41:26.967 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:41:26 smithi161 ceph-mon[34533]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 202 MiB used, 626 GiB / 626 GiB avail 2024-09-21T03:41:27.834 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:41:27 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.7"} : dispatch 2024-09-21T03:41:27.834 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:41:27 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:41:27.834 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:41:27 smithi039 ceph-mon[31211]: Deploying daemon osd.7 on smithi150 2024-09-21T03:41:27.967 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:41:27 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.7"} : dispatch 2024-09-21T03:41:27.967 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:41:27 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:41:27.967 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:41:27 smithi161 ceph-mon[34533]: Deploying daemon osd.7 on smithi150 2024-09-21T03:41:27.974 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:41:27 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.7"} : dispatch 2024-09-21T03:41:27.974 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:41:27 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:41:27.974 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:41:27 smithi150 ceph-mon[34289]: Deploying daemon osd.7 on smithi150 2024-09-21T03:41:28.780 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:41:28 smithi150 ceph-mon[34289]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 202 MiB used, 626 GiB / 626 GiB avail 2024-09-21T03:41:28.833 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:41:28 smithi039 ceph-mon[31211]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 202 MiB used, 626 GiB / 626 GiB avail 2024-09-21T03:41:28.968 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:41:28 smithi161 ceph-mon[34533]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 202 MiB used, 626 GiB / 626 GiB avail 2024-09-21T03:41:30.725 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:41:30 smithi150 ceph-mon[34289]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 202 MiB used, 626 GiB / 626 GiB avail 2024-09-21T03:41:30.833 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:41:30 smithi039 ceph-mon[31211]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 202 MiB used, 626 GiB / 626 GiB avail 2024-09-21T03:41:30.967 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:41:30 smithi161 ceph-mon[34533]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 202 MiB used, 626 GiB / 626 GiB avail 2024-09-21T03:41:32.764 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:41:32 smithi150 ceph-mon[34289]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 202 MiB used, 626 GiB / 626 GiB avail 2024-09-21T03:41:32.833 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:41:32 smithi039 ceph-mon[31211]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 202 MiB used, 626 GiB / 626 GiB avail 2024-09-21T03:41:32.967 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:41:32 smithi161 ceph-mon[34533]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 202 MiB used, 626 GiB / 626 GiB avail 2024-09-21T03:41:33.833 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:41:33 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:41:33.834 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:41:33 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:41:33.834 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:41:33 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:41:33.968 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:41:33 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:41:33.968 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:41:33 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:41:33.968 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:41:33 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:41:33.974 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:41:33 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:41:33.974 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:41:33 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:41:33.974 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:41:33 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:41:35.131 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:41:34 smithi150 ceph-mon[34289]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 202 MiB used, 626 GiB / 626 GiB avail 2024-09-21T03:41:35.217 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:41:34 smithi161 ceph-mon[34533]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 202 MiB used, 626 GiB / 626 GiB avail 2024-09-21T03:41:35.333 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:41:34 smithi039 ceph-mon[31211]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 202 MiB used, 626 GiB / 626 GiB avail 2024-09-21T03:41:36.588 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:41:36 smithi150 ceph-mon[34289]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 202 MiB used, 626 GiB / 626 GiB avail 2024-09-21T03:41:36.588 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:41:36 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:41:36.588 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:41:36 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:41:36.588 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:41:36 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:41:36.588 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:41:36 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:41:36.588 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:41:36 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:41:36.636 INFO:teuthology.orchestra.run.smithi150.stdout:Created osd(s) 7 on host 'smithi150' 2024-09-21T03:41:36.636 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T03:41:36.633+0000 7f0770d99640 1 -- 172.21.15.150:0/320270229 <== mgr.14223 v2:172.21.15.39:6800/1696381982 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f0774063800 con 0x7f07600b6560 2024-09-21T03:41:36.639 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T03:41:36.636+0000 7f077907a640 1 -- 172.21.15.150:0/320270229 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f07600b6560 msgr2=0x7f07600b8a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:36.639 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T03:41:36.636+0000 7f077907a640 1 --2- 172.21.15.150:0/320270229 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f07600b6560 0x7f07600b8a70 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f07741074b0 tx=0x7f075c00b390 comp rx=0 tx=0).stop 2024-09-21T03:41:36.639 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T03:41:36.636+0000 7f077907a640 1 -- 172.21.15.150:0/320270229 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0774103b60 msgr2=0x7f077419a4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:36.639 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T03:41:36.636+0000 7f077907a640 1 --2- 172.21.15.150:0/320270229 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0774103b60 0x7f077419a4e0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f076400e9f0 tx=0x7f076400edf0 comp rx=0 tx=0).stop 2024-09-21T03:41:36.639 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T03:41:36.636+0000 7f077907a640 1 -- 172.21.15.150:0/320270229 shutdown_connections 2024-09-21T03:41:36.639 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T03:41:36.636+0000 7f077907a640 1 --2- 172.21.15.150:0/320270229 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f07600b6560 0x7f07600b8a70 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:36.639 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T03:41:36.636+0000 7f077907a640 1 --2- 172.21.15.150:0/320270229 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0774109df0 0x7f077419fd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:36.639 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T03:41:36.636+0000 7f077907a640 1 --2- 172.21.15.150:0/320270229 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f07741094a0 0x7f077419aa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:36.639 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T03:41:36.636+0000 7f077907a640 1 --2- 172.21.15.150:0/320270229 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0774103b60 0x7f077419a4e0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:36.639 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T03:41:36.636+0000 7f077907a640 1 -- 172.21.15.150:0/320270229 >> 172.21.15.150:0/320270229 conn(0x7f077407b850 msgr2=0x7f0774105430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:41:36.639 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T03:41:36.637+0000 7f077907a640 1 -- 172.21.15.150:0/320270229 shutdown_connections 2024-09-21T03:41:36.639 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T03:41:36.637+0000 7f077907a640 1 -- 172.21.15.150:0/320270229 wait complete. 2024-09-21T03:41:36.834 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:41:36 smithi039 ceph-mon[31211]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 202 MiB used, 626 GiB / 626 GiB avail 2024-09-21T03:41:36.834 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:41:36 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:41:36.834 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:41:36 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:41:36.834 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:41:36 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:41:36.834 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:41:36 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:41:36.834 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:41:36 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:41:36.967 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:41:36 smithi161 ceph-mon[34533]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 202 MiB used, 626 GiB / 626 GiB avail 2024-09-21T03:41:36.968 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:41:36 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:41:36.968 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:41:36 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:41:36.968 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:41:36 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:41:36.968 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:41:36 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:41:36.968 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:41:36 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:41:37.285 DEBUG:teuthology.orchestra.run.smithi150:osd.7> sudo journalctl -f -n 0 -u ceph-51116d5c-77ca-11ef-baf5-efdc52797490@osd.7.service 2024-09-21T03:41:37.287 INFO:tasks.cephadm:Deploying osd.8 on smithi161 with /dev/vg_nvme/lv_2... 2024-09-21T03:41:37.287 DEBUG:teuthology.orchestra.run.smithi161:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 51116d5c-77ca-11ef-baf5-efdc52797490 -- lvm zap /dev/vg_nvme/lv_2 2024-09-21T03:41:37.545 INFO:teuthology.orchestra.run.smithi161.stderr:Inferring config /var/lib/ceph/51116d5c-77ca-11ef-baf5-efdc52797490/mon.c/config 2024-09-21T03:41:37.624 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:41:37 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:41:37.624 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:41:37 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:41:37.624 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:41:37 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:41:37.666 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:41:37 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:41:37.666 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:41:37 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:41:37.666 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:41:37 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:41:37.833 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:41:37 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:41:37.833 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:41:37 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:41:37.834 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:41:37 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:41:38.785 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:41:38 smithi150 ceph-mon[34289]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 202 MiB used, 626 GiB / 626 GiB avail 2024-09-21T03:41:38.941 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:41:38 smithi161 ceph-mon[34533]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 202 MiB used, 626 GiB / 626 GiB avail 2024-09-21T03:41:39.083 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:41:38 smithi039 ceph-mon[31211]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 202 MiB used, 626 GiB / 626 GiB avail 2024-09-21T03:41:39.225 INFO:journalctl@ceph.osd.7.smithi150.stdout:Sep 21 03:41:38 smithi150 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-7[51981]: 2024-09-21T03:41:38.963+0000 7f2e17857640 -1 osd.7 0 log_to_monitors true 2024-09-21T03:41:39.780 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:41:39 smithi150 ceph-mon[34289]: from='osd.7 [v2:172.21.15.150:6816/2162619103,v1:172.21.15.150:6817/2162619103]' entity='osd.7' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]} : dispatch 2024-09-21T03:41:39.781 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:41:39 smithi150 ceph-mon[34289]: from='osd.7 ' entity='osd.7' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]} : dispatch 2024-09-21T03:41:39.897 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:41:39 smithi161 ceph-mon[34533]: from='osd.7 [v2:172.21.15.150:6816/2162619103,v1:172.21.15.150:6817/2162619103]' entity='osd.7' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]} : dispatch 2024-09-21T03:41:39.897 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:41:39 smithi161 ceph-mon[34533]: from='osd.7 ' entity='osd.7' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]} : dispatch 2024-09-21T03:41:39.985 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:41:39 smithi039 ceph-mon[31211]: from='osd.7 [v2:172.21.15.150:6816/2162619103,v1:172.21.15.150:6817/2162619103]' entity='osd.7' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]} : dispatch 2024-09-21T03:41:39.985 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:41:39 smithi039 ceph-mon[31211]: from='osd.7 ' entity='osd.7' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]} : dispatch 2024-09-21T03:41:40.131 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-09-21T03:41:40.159 DEBUG:teuthology.orchestra.run.smithi161:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 51116d5c-77ca-11ef-baf5-efdc52797490 -- ceph orch daemon add osd smithi161:vg_nvme/lv_2 2024-09-21T03:41:40.416 INFO:teuthology.orchestra.run.smithi161.stderr:Inferring config /var/lib/ceph/51116d5c-77ca-11ef-baf5-efdc52797490/mon.c/config 2024-09-21T03:41:40.834 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:41:40 smithi039 ceph-mon[31211]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 202 MiB used, 626 GiB / 626 GiB avail 2024-09-21T03:41:40.834 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:41:40 smithi039 ceph-mon[31211]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-09-21T03:41:40.834 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:41:40 smithi039 ceph-mon[31211]: osdmap e46: 8 total, 7 up, 8 in 2024-09-21T03:41:40.834 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:41:40 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-09-21T03:41:40.834 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:41:40 smithi039 ceph-mon[31211]: from='osd.7 [v2:172.21.15.150:6816/2162619103,v1:172.21.15.150:6817/2162619103]' entity='osd.7' cmd={"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi150", "root=default"]} : dispatch 2024-09-21T03:41:40.834 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:41:40 smithi039 ceph-mon[31211]: from='osd.7 ' entity='osd.7' cmd={"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi150", "root=default"]} : dispatch 2024-09-21T03:41:40.834 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:41:40 smithi039 ceph-mon[31211]: Detected new or changed devices on smithi150 2024-09-21T03:41:40.834 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:41:40 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:41:40.835 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:41:40 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:41:40.835 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:41:40 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-09-21T03:41:40.835 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:41:40 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-09-21T03:41:40.835 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:41:40 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"} : dispatch 2024-09-21T03:41:40.835 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:41:40 smithi039 ceph-mon[31211]: Adjusting osd_memory_target on smithi150 to 5696M 2024-09-21T03:41:40.835 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:41:40 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:41:40.835 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:41:40 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:41:40.835 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:41:40 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:41:40.835 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:41:40 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:41:40.856 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:41:40 smithi161 ceph-mon[34533]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 202 MiB used, 626 GiB / 626 GiB avail 2024-09-21T03:41:40.856 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:41:40 smithi161 ceph-mon[34533]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-09-21T03:41:40.857 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:41:40 smithi161 ceph-mon[34533]: osdmap e46: 8 total, 7 up, 8 in 2024-09-21T03:41:40.857 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:41:40 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-09-21T03:41:40.857 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:41:40 smithi161 ceph-mon[34533]: from='osd.7 [v2:172.21.15.150:6816/2162619103,v1:172.21.15.150:6817/2162619103]' entity='osd.7' cmd={"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi150", "root=default"]} : dispatch 2024-09-21T03:41:40.857 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:41:40 smithi161 ceph-mon[34533]: from='osd.7 ' entity='osd.7' cmd={"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi150", "root=default"]} : dispatch 2024-09-21T03:41:40.857 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:41:40 smithi161 ceph-mon[34533]: Detected new or changed devices on smithi150 2024-09-21T03:41:40.857 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:41:40 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:41:40.857 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:41:40 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:41:40.857 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:41:40 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-09-21T03:41:40.857 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:41:40 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-09-21T03:41:40.857 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:41:40 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"} : dispatch 2024-09-21T03:41:40.857 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:41:40 smithi161 ceph-mon[34533]: Adjusting osd_memory_target on smithi150 to 5696M 2024-09-21T03:41:40.857 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:41:40 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:41:40.857 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:41:40 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:41:40.857 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:41:40 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:41:40.857 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:41:40 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:41:40.867 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-21T03:41:40.865+0000 7f827c38c640 1 -- 172.21.15.161:0/3721234468 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8274103bb0 msgr2=0x7f8274103fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:40.867 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-21T03:41:40.865+0000 7f827c38c640 1 --2- 172.21.15.161:0/3721234468 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8274103bb0 0x7f8274103fc0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f8270009900 tx=0x7f827002a410 comp rx=0 tx=0).stop 2024-09-21T03:41:40.867 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-21T03:41:40.865+0000 7f827c38c640 1 -- 172.21.15.161:0/3721234468 shutdown_connections 2024-09-21T03:41:40.867 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-21T03:41:40.865+0000 7f827c38c640 1 --2- 172.21.15.161:0/3721234468 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8274109e40 0x7f8274110670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:40.867 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-21T03:41:40.865+0000 7f827c38c640 1 --2- 172.21.15.161:0/3721234468 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f82741094f0 0x7f8274109900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:40.867 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-21T03:41:40.865+0000 7f827c38c640 1 --2- 172.21.15.161:0/3721234468 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8274103bb0 0x7f8274103fc0 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:40.867 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-21T03:41:40.865+0000 7f827c38c640 1 -- 172.21.15.161:0/3721234468 >> 172.21.15.161:0/3721234468 conn(0x7f82740ff600 msgr2=0x7f8274101a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:41:40.867 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-21T03:41:40.865+0000 7f827c38c640 1 -- 172.21.15.161:0/3721234468 shutdown_connections 2024-09-21T03:41:40.867 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-21T03:41:40.865+0000 7f827c38c640 1 -- 172.21.15.161:0/3721234468 wait complete. 2024-09-21T03:41:40.869 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-21T03:41:40.866+0000 7f827c38c640 1 Processor -- start 2024-09-21T03:41:40.869 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-21T03:41:40.866+0000 7f827c38c640 1 -- start start 2024-09-21T03:41:40.869 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-21T03:41:40.866+0000 7f827c38c640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8274103bb0 0x7f827419a490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:40.870 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-21T03:41:40.866+0000 7f827c38c640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f82741094f0 0x7f827419aa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:40.870 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-21T03:41:40.866+0000 7f827c38c640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8274109e40 0x7f827419fcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:40.870 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-21T03:41:40.866+0000 7f827c38c640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f8274113b00 con 0x7f8274109e40 2024-09-21T03:41:40.870 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-21T03:41:40.866+0000 7f827c38c640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f8274113980 con 0x7f82741094f0 2024-09-21T03:41:40.870 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-21T03:41:40.866+0000 7f827c38c640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f8274113c80 con 0x7f8274103bb0 2024-09-21T03:41:40.870 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-21T03:41:40.866+0000 7f827a128640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8274103bb0 0x7f827419a490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:40.870 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-21T03:41:40.866+0000 7f827a128640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8274103bb0 0x7f827419a490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:52930/0 (socket says 172.21.15.161:52930) 2024-09-21T03:41:40.870 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-21T03:41:40.866+0000 7f827a128640 1 -- 172.21.15.161:0/143275618 learned_addr learned my addr 172.21.15.161:0/143275618 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-09-21T03:41:40.870 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-21T03:41:40.866+0000 7f8279927640 1 --2- 172.21.15.161:0/143275618 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f82741094f0 0x7f827419aa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:40.870 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-21T03:41:40.866+0000 7f827a929640 1 --2- 172.21.15.161:0/143275618 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8274109e40 0x7f827419fcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:40.870 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-21T03:41:40.866+0000 7f827a128640 1 -- 172.21.15.161:0/143275618 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f82741094f0 msgr2=0x7f827419aa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:40.870 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-21T03:41:40.866+0000 7f827a128640 1 --2- 172.21.15.161:0/143275618 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f82741094f0 0x7f827419aa30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:40.870 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-21T03:41:40.866+0000 7f827a128640 1 -- 172.21.15.161:0/143275618 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8274109e40 msgr2=0x7f827419fcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:40.871 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-21T03:41:40.866+0000 7f827a128640 1 --2- 172.21.15.161:0/143275618 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8274109e40 0x7f827419fcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:40.871 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-21T03:41:40.867+0000 7f827a128640 1 -- 172.21.15.161:0/143275618 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f82741a0400 con 0x7f8274103bb0 2024-09-21T03:41:40.871 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-21T03:41:40.867+0000 7f8279927640 1 --2- 172.21.15.161:0/143275618 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f82741094f0 0x7f827419aa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:41:40.871 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-21T03:41:40.867+0000 7f827a128640 1 --2- 172.21.15.161:0/143275618 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8274103bb0 0x7f827419a490 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f8270005940 tx=0x7f82700040c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:41:40.871 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-21T03:41:40.867+0000 7f826b7fe640 1 -- 172.21.15.161:0/143275618 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f827003f030 con 0x7f8274103bb0 2024-09-21T03:41:40.871 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-21T03:41:40.867+0000 7f826b7fe640 1 -- 172.21.15.161:0/143275618 <== mon.1 v2:172.21.15.161:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f82700044c0 con 0x7f8274103bb0 2024-09-21T03:41:40.871 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-21T03:41:40.867+0000 7f826b7fe640 1 -- 172.21.15.161:0/143275618 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8270035070 con 0x7f8274103bb0 2024-09-21T03:41:40.871 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-21T03:41:40.867+0000 7f827c38c640 1 -- 172.21.15.161:0/143275618 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f82741a0690 con 0x7f8274103bb0 2024-09-21T03:41:40.871 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-21T03:41:40.867+0000 7f827c38c640 1 -- 172.21.15.161:0/143275618 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f82741a6e50 con 0x7f8274103bb0 2024-09-21T03:41:40.871 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-21T03:41:40.868+0000 7f827c38c640 1 -- 172.21.15.161:0/143275618 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8274104e30 con 0x7f8274103bb0 2024-09-21T03:41:40.872 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-21T03:41:40.870+0000 7f826b7fe640 1 -- 172.21.15.161:0/143275618 <== mon.1 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 21) ==== 154529+0+0 (secure 0 0 0) 0x7f8270042030 con 0x7f8274103bb0 2024-09-21T03:41:40.872 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-21T03:41:40.870+0000 7f826b7fe640 1 --2- 172.21.15.161:0/143275618 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f82600b6560 0x7f82600b8a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:40.872 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-21T03:41:40.871+0000 7f8279927640 1 --2- 172.21.15.161:0/143275618 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f82600b6560 0x7f82600b8a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:40.872 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-21T03:41:40.871+0000 7f826b7fe640 1 -- 172.21.15.161:0/143275618 <== mon.1 v2:172.21.15.161:3300/0 5 ==== osd_map(47..47 src has 1..47) ==== 5332+0+0 (secure 0 0 0) 0x7f8270106d90 con 0x7f8274103bb0 2024-09-21T03:41:40.873 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-21T03:41:40.871+0000 7f8279927640 1 --2- 172.21.15.161:0/143275618 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f82600b6560 0x7f82600b8a70 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f8274108fe0 tx=0x7f8264005f90 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:41:40.876 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-21T03:41:40.874+0000 7f826b7fe640 1 -- 172.21.15.161:0/143275618 <== mon.1 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f82700d25a0 con 0x7f8274103bb0 2024-09-21T03:41:40.974 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:41:40 smithi150 ceph-mon[34289]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 202 MiB used, 626 GiB / 626 GiB avail 2024-09-21T03:41:40.975 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:41:40 smithi150 ceph-mon[34289]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-09-21T03:41:40.975 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:41:40 smithi150 ceph-mon[34289]: osdmap e46: 8 total, 7 up, 8 in 2024-09-21T03:41:40.975 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:41:40 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-09-21T03:41:40.975 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:41:40 smithi150 ceph-mon[34289]: from='osd.7 [v2:172.21.15.150:6816/2162619103,v1:172.21.15.150:6817/2162619103]' entity='osd.7' cmd={"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi150", "root=default"]} : dispatch 2024-09-21T03:41:40.975 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:41:40 smithi150 ceph-mon[34289]: from='osd.7 ' entity='osd.7' cmd={"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi150", "root=default"]} : dispatch 2024-09-21T03:41:40.975 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:41:40 smithi150 ceph-mon[34289]: Detected new or changed devices on smithi150 2024-09-21T03:41:40.975 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:41:40 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:41:40.975 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:41:40 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:41:40.975 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:41:40 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-09-21T03:41:40.975 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:41:40 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-09-21T03:41:40.975 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:41:40 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"} : dispatch 2024-09-21T03:41:40.975 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:41:40 smithi150 ceph-mon[34289]: Adjusting osd_memory_target on smithi150 to 5696M 2024-09-21T03:41:40.975 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:41:40 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:41:40.976 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:41:40 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:41:40.976 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:41:40 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:41:40.976 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:41:40 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:41:41.063 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-21T03:41:41.061+0000 7f827c38c640 1 -- 172.21.15.161:0/143275618 --> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi161:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7f8274063800 con 0x7f82600b6560 2024-09-21T03:41:41.717 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:41:41 smithi161 ceph-mon[34533]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi150", "root=default"]}]': finished 2024-09-21T03:41:41.718 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:41:41 smithi161 ceph-mon[34533]: osdmap e47: 8 total, 7 up, 8 in 2024-09-21T03:41:41.718 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:41:41 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-09-21T03:41:41.718 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:41:41 smithi161 ceph-mon[34533]: from='client.24472 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi161:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:41:41.718 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:41:41 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T03:41:41.718 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:41:41 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T03:41:41.718 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:41:41 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:41:41.718 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:41:41 smithi161 ceph-mon[34533]: from='osd.7 ' entity='osd.7' 2024-09-21T03:41:41.718 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:41:41 smithi161 ceph-mon[34533]: osdmap e48: 8 total, 7 up, 8 in 2024-09-21T03:41:41.718 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:41:41 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-09-21T03:41:41.834 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:41:41 smithi039 ceph-mon[31211]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi150", "root=default"]}]': finished 2024-09-21T03:41:41.834 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:41:41 smithi039 ceph-mon[31211]: osdmap e47: 8 total, 7 up, 8 in 2024-09-21T03:41:41.834 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:41:41 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-09-21T03:41:41.834 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:41:41 smithi039 ceph-mon[31211]: from='client.24472 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi161:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:41:41.834 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:41:41 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T03:41:41.834 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:41:41 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T03:41:41.834 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:41:41 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:41:41.834 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:41:41 smithi039 ceph-mon[31211]: from='osd.7 ' entity='osd.7' 2024-09-21T03:41:41.834 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:41:41 smithi039 ceph-mon[31211]: osdmap e48: 8 total, 7 up, 8 in 2024-09-21T03:41:41.834 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:41:41 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-09-21T03:41:41.975 INFO:journalctl@ceph.osd.7.smithi150.stdout:Sep 21 03:41:41 smithi150 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-7[51981]: 2024-09-21T03:41:41.562+0000 7f2e13fee640 -1 osd.7 0 waiting for initial osdmap 2024-09-21T03:41:41.975 INFO:journalctl@ceph.osd.7.smithi150.stdout:Sep 21 03:41:41 smithi150 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-7[51981]: 2024-09-21T03:41:41.717+0000 7f2e0b5cf640 -1 osd.7 48 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-21T03:41:41.975 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:41:41 smithi150 ceph-mon[34289]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi150", "root=default"]}]': finished 2024-09-21T03:41:41.975 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:41:41 smithi150 ceph-mon[34289]: osdmap e47: 8 total, 7 up, 8 in 2024-09-21T03:41:41.975 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:41:41 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-09-21T03:41:41.975 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:41:41 smithi150 ceph-mon[34289]: from='client.24472 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi161:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:41:41.975 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:41:41 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T03:41:41.975 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:41:41 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T03:41:41.975 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:41:41 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:41:41.975 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:41:41 smithi150 ceph-mon[34289]: from='osd.7 ' entity='osd.7' 2024-09-21T03:41:41.976 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:41:41 smithi150 ceph-mon[34289]: osdmap e48: 8 total, 7 up, 8 in 2024-09-21T03:41:41.976 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:41:41 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-09-21T03:41:42.967 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:41:42 smithi161 ceph-mon[34533]: purged_snaps scrub starts 2024-09-21T03:41:42.968 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:41:42 smithi161 ceph-mon[34533]: purged_snaps scrub ok 2024-09-21T03:41:42.968 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:41:42 smithi161 ceph-mon[34533]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 202 MiB used, 626 GiB / 626 GiB avail 2024-09-21T03:41:42.968 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:41:42 smithi161 ceph-mon[34533]: osd.7 [v2:172.21.15.150:6816/2162619103,v1:172.21.15.150:6817/2162619103] boot 2024-09-21T03:41:42.968 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:41:42 smithi161 ceph-mon[34533]: osdmap e49: 8 total, 8 up, 8 in 2024-09-21T03:41:42.968 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:41:42 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-09-21T03:41:42.974 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:41:42 smithi150 ceph-mon[34289]: purged_snaps scrub starts 2024-09-21T03:41:42.975 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:41:42 smithi150 ceph-mon[34289]: purged_snaps scrub ok 2024-09-21T03:41:42.975 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:41:42 smithi150 ceph-mon[34289]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 202 MiB used, 626 GiB / 626 GiB avail 2024-09-21T03:41:42.975 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:41:42 smithi150 ceph-mon[34289]: osd.7 [v2:172.21.15.150:6816/2162619103,v1:172.21.15.150:6817/2162619103] boot 2024-09-21T03:41:42.975 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:41:42 smithi150 ceph-mon[34289]: osdmap e49: 8 total, 8 up, 8 in 2024-09-21T03:41:42.975 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:41:42 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-09-21T03:41:43.083 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:41:42 smithi039 ceph-mon[31211]: purged_snaps scrub starts 2024-09-21T03:41:43.084 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:41:42 smithi039 ceph-mon[31211]: purged_snaps scrub ok 2024-09-21T03:41:43.084 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:41:42 smithi039 ceph-mon[31211]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 202 MiB used, 626 GiB / 626 GiB avail 2024-09-21T03:41:43.084 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:41:42 smithi039 ceph-mon[31211]: osd.7 [v2:172.21.15.150:6816/2162619103,v1:172.21.15.150:6817/2162619103] boot 2024-09-21T03:41:43.084 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:41:42 smithi039 ceph-mon[31211]: osdmap e49: 8 total, 8 up, 8 in 2024-09-21T03:41:43.084 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:41:42 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-09-21T03:41:43.967 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:41:43 smithi161 ceph-mon[34533]: from='client.? 172.21.15.161:0/4059013117' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "4973b357-b4c2-4c00-b7f7-dd25012a0f72"} : dispatch 2024-09-21T03:41:43.968 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:41:43 smithi161 ceph-mon[34533]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "4973b357-b4c2-4c00-b7f7-dd25012a0f72"} : dispatch 2024-09-21T03:41:43.968 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:41:43 smithi161 ceph-mon[34533]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "4973b357-b4c2-4c00-b7f7-dd25012a0f72"}]': finished 2024-09-21T03:41:43.968 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:41:43 smithi161 ceph-mon[34533]: osdmap e50: 9 total, 8 up, 9 in 2024-09-21T03:41:43.968 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:41:43 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2024-09-21T03:41:43.974 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:41:43 smithi150 ceph-mon[34289]: from='client.? 172.21.15.161:0/4059013117' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "4973b357-b4c2-4c00-b7f7-dd25012a0f72"} : dispatch 2024-09-21T03:41:43.974 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:41:43 smithi150 ceph-mon[34289]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "4973b357-b4c2-4c00-b7f7-dd25012a0f72"} : dispatch 2024-09-21T03:41:43.974 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:41:43 smithi150 ceph-mon[34289]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "4973b357-b4c2-4c00-b7f7-dd25012a0f72"}]': finished 2024-09-21T03:41:43.974 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:41:43 smithi150 ceph-mon[34289]: osdmap e50: 9 total, 8 up, 9 in 2024-09-21T03:41:43.974 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:41:43 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2024-09-21T03:41:44.083 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:41:43 smithi039 ceph-mon[31211]: from='client.? 172.21.15.161:0/4059013117' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "4973b357-b4c2-4c00-b7f7-dd25012a0f72"} : dispatch 2024-09-21T03:41:44.083 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:41:43 smithi039 ceph-mon[31211]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "4973b357-b4c2-4c00-b7f7-dd25012a0f72"} : dispatch 2024-09-21T03:41:44.084 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:41:43 smithi039 ceph-mon[31211]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "4973b357-b4c2-4c00-b7f7-dd25012a0f72"}]': finished 2024-09-21T03:41:44.084 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:41:43 smithi039 ceph-mon[31211]: osdmap e50: 9 total, 8 up, 9 in 2024-09-21T03:41:44.084 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:41:43 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2024-09-21T03:41:44.974 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:41:44 smithi150 ceph-mon[34289]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 202 MiB used, 626 GiB / 626 GiB avail 2024-09-21T03:41:44.975 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:41:44 smithi150 ceph-mon[34289]: from='client.? 172.21.15.161:0/3261508598' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T03:41:44.975 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:41:44 smithi150 ceph-mon[34289]: osdmap e51: 9 total, 8 up, 9 in 2024-09-21T03:41:44.975 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:41:44 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2024-09-21T03:41:45.083 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:41:44 smithi039 ceph-mon[31211]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 202 MiB used, 626 GiB / 626 GiB avail 2024-09-21T03:41:45.084 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:41:44 smithi039 ceph-mon[31211]: from='client.? 172.21.15.161:0/3261508598' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T03:41:45.084 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:41:44 smithi039 ceph-mon[31211]: osdmap e51: 9 total, 8 up, 9 in 2024-09-21T03:41:45.084 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:41:44 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2024-09-21T03:41:45.217 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:41:44 smithi161 ceph-mon[34533]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 202 MiB used, 626 GiB / 626 GiB avail 2024-09-21T03:41:45.217 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:41:44 smithi161 ceph-mon[34533]: from='client.? 172.21.15.161:0/3261508598' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T03:41:45.217 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:41:44 smithi161 ceph-mon[34533]: osdmap e51: 9 total, 8 up, 9 in 2024-09-21T03:41:45.218 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:41:44 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2024-09-21T03:41:46.968 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:41:46 smithi161 ceph-mon[34533]: pgmap v169: 1 pgs: 1 peering; 0 B data, 241 MiB used, 715 GiB / 715 GiB avail 2024-09-21T03:41:46.974 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:41:46 smithi150 ceph-mon[34289]: pgmap v169: 1 pgs: 1 peering; 0 B data, 241 MiB used, 715 GiB / 715 GiB avail 2024-09-21T03:41:47.083 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:41:46 smithi039 ceph-mon[31211]: pgmap v169: 1 pgs: 1 peering; 0 B data, 241 MiB used, 715 GiB / 715 GiB avail 2024-09-21T03:41:48.975 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:41:48 smithi150 ceph-mon[34289]: pgmap v170: 1 pgs: 1 peering; 0 B data, 241 MiB used, 715 GiB / 715 GiB avail 2024-09-21T03:41:49.083 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:41:48 smithi039 ceph-mon[31211]: pgmap v170: 1 pgs: 1 peering; 0 B data, 241 MiB used, 715 GiB / 715 GiB avail 2024-09-21T03:41:49.087 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:41:48 smithi161 ceph-mon[34533]: pgmap v170: 1 pgs: 1 peering; 0 B data, 241 MiB used, 715 GiB / 715 GiB avail 2024-09-21T03:41:50.811 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:41:50 smithi161 ceph-mon[34533]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail 2024-09-21T03:41:51.083 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:41:50 smithi039 ceph-mon[31211]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail 2024-09-21T03:41:51.224 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:41:50 smithi150 ceph-mon[34289]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail 2024-09-21T03:41:53.084 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:41:52 smithi039 ceph-mon[31211]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail 2024-09-21T03:41:53.084 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:41:52 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.8"} : dispatch 2024-09-21T03:41:53.084 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:41:52 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:41:53.084 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:41:52 smithi039 ceph-mon[31211]: Deploying daemon osd.8 on smithi161 2024-09-21T03:41:53.218 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:41:52 smithi161 ceph-mon[34533]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail 2024-09-21T03:41:53.218 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:41:52 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.8"} : dispatch 2024-09-21T03:41:53.218 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:41:52 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:41:53.218 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:41:52 smithi161 ceph-mon[34533]: Deploying daemon osd.8 on smithi161 2024-09-21T03:41:53.224 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:41:52 smithi150 ceph-mon[34289]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail 2024-09-21T03:41:53.224 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:41:52 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.8"} : dispatch 2024-09-21T03:41:53.224 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:41:52 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:41:53.225 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:41:52 smithi150 ceph-mon[34289]: Deploying daemon osd.8 on smithi161 2024-09-21T03:41:55.083 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:41:54 smithi039 ceph-mon[31211]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail 2024-09-21T03:41:55.094 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:41:54 smithi161 ceph-mon[34533]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail 2024-09-21T03:41:55.224 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:41:54 smithi150 ceph-mon[34289]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail 2024-09-21T03:41:56.968 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:41:56 smithi161 ceph-mon[34533]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail 2024-09-21T03:41:57.083 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:41:56 smithi039 ceph-mon[31211]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail 2024-09-21T03:41:57.224 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:41:56 smithi150 ceph-mon[34289]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail 2024-09-21T03:41:58.964 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:41:58 smithi161 ceph-mon[34533]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail 2024-09-21T03:41:58.965 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:41:58 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:41:58.965 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:41:58 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:41:58.965 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:41:58 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:41:59.083 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:41:58 smithi039 ceph-mon[31211]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail 2024-09-21T03:41:59.084 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:41:58 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:41:59.084 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:41:58 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:41:59.084 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:41:58 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:41:59.224 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:41:58 smithi150 ceph-mon[34289]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail 2024-09-21T03:41:59.224 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:41:58 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:41:59.224 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:41:58 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:41:59.225 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:41:58 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:42:00.850 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:42:00 smithi161 ceph-mon[34533]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail 2024-09-21T03:42:01.083 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:42:00 smithi039 ceph-mon[31211]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail 2024-09-21T03:42:01.224 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:42:00 smithi150 ceph-mon[34289]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail 2024-09-21T03:42:02.084 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:42:01 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:42:02.084 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:42:01 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:42:02.084 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:42:01 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:42:02.084 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:42:01 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:42:02.084 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:42:01 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:42:02.087 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:42:01 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:42:02.087 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:42:01 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:42:02.087 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:42:01 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:42:02.088 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:42:01 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:42:02.088 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:42:01 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:42:02.132 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-21T03:42:02.130+0000 7f826b7fe640 1 -- 172.21.15.161:0/143275618 <== mgr.14223 v2:172.21.15.39:6800/1696381982 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f8274063800 con 0x7f82600b6560 2024-09-21T03:42:02.132 INFO:teuthology.orchestra.run.smithi161.stdout:Created osd(s) 8 on host 'smithi161' 2024-09-21T03:42:02.134 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-21T03:42:02.132+0000 7f827c38c640 1 -- 172.21.15.161:0/143275618 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f82600b6560 msgr2=0x7f82600b8a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:02.134 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-21T03:42:02.132+0000 7f827c38c640 1 --2- 172.21.15.161:0/143275618 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f82600b6560 0x7f82600b8a70 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f8274108fe0 tx=0x7f8264005f90 comp rx=0 tx=0).stop 2024-09-21T03:42:02.134 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-21T03:42:02.132+0000 7f827c38c640 1 -- 172.21.15.161:0/143275618 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8274103bb0 msgr2=0x7f827419a490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:02.134 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-21T03:42:02.132+0000 7f827c38c640 1 --2- 172.21.15.161:0/143275618 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8274103bb0 0x7f827419a490 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f8270005940 tx=0x7f82700040c0 comp rx=0 tx=0).stop 2024-09-21T03:42:02.134 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-21T03:42:02.132+0000 7f827c38c640 1 -- 172.21.15.161:0/143275618 shutdown_connections 2024-09-21T03:42:02.135 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-21T03:42:02.132+0000 7f827c38c640 1 --2- 172.21.15.161:0/143275618 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f82600b6560 0x7f82600b8a70 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:02.135 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-21T03:42:02.132+0000 7f827c38c640 1 --2- 172.21.15.161:0/143275618 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8274109e40 0x7f827419fcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:02.135 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-21T03:42:02.132+0000 7f827c38c640 1 --2- 172.21.15.161:0/143275618 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f82741094f0 0x7f827419aa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:02.135 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-21T03:42:02.132+0000 7f827c38c640 1 --2- 172.21.15.161:0/143275618 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8274103bb0 0x7f827419a490 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:02.135 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-21T03:42:02.132+0000 7f827c38c640 1 -- 172.21.15.161:0/143275618 >> 172.21.15.161:0/143275618 conn(0x7f82740ff600 msgr2=0x7f8274101a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:42:02.135 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-21T03:42:02.132+0000 7f827c38c640 1 -- 172.21.15.161:0/143275618 shutdown_connections 2024-09-21T03:42:02.135 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-21T03:42:02.132+0000 7f827c38c640 1 -- 172.21.15.161:0/143275618 wait complete. 2024-09-21T03:42:02.224 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:42:01 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:42:02.224 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:42:01 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:42:02.225 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:42:01 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:42:02.225 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:42:01 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:42:02.225 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:42:01 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:42:02.776 DEBUG:teuthology.orchestra.run.smithi161:osd.8> sudo journalctl -f -n 0 -u ceph-51116d5c-77ca-11ef-baf5-efdc52797490@osd.8.service 2024-09-21T03:42:02.778 INFO:tasks.cephadm:Deploying osd.9 on smithi039 with /dev/vg_nvme/lv_1... 2024-09-21T03:42:02.778 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 51116d5c-77ca-11ef-baf5-efdc52797490 -- lvm zap /dev/vg_nvme/lv_1 2024-09-21T03:42:03.050 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/51116d5c-77ca-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:42:03.083 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:42:02 smithi039 ceph-mon[31211]: pgmap v177: 1 pgs: 1 active+clean; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail 2024-09-21T03:42:03.083 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:42:02 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:42:03.083 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:42:02 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:42:03.083 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:42:02 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:42:03.105 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:42:02 smithi161 ceph-mon[34533]: pgmap v177: 1 pgs: 1 active+clean; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail 2024-09-21T03:42:03.105 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:42:02 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:42:03.105 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:42:02 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:42:03.105 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:42:02 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:42:03.224 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:42:02 smithi150 ceph-mon[34289]: pgmap v177: 1 pgs: 1 active+clean; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail 2024-09-21T03:42:03.224 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:42:02 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:42:03.225 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:42:02 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:42:03.225 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:42:02 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:42:04.837 INFO:journalctl@ceph.osd.8.smithi161.stdout:Sep 21 03:42:04 smithi161 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-8[52151]: 2024-09-21T03:42:04.560+0000 7f7ea0b73640 -1 osd.8 0 log_to_monitors true 2024-09-21T03:42:05.069 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:42:04 smithi039 ceph-mon[31211]: pgmap v178: 1 pgs: 1 active+clean; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail 2024-09-21T03:42:05.069 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:42:04 smithi039 ceph-mon[31211]: from='osd.8 [v2:172.21.15.161:6816/52966633,v1:172.21.15.161:6817/52966633]' entity='osd.8' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]} : dispatch 2024-09-21T03:42:05.069 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:42:04 smithi039 ceph-mon[31211]: from='osd.8 ' entity='osd.8' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]} : dispatch 2024-09-21T03:42:05.217 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:42:04 smithi161 ceph-mon[34533]: pgmap v178: 1 pgs: 1 active+clean; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail 2024-09-21T03:42:05.218 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:42:04 smithi161 ceph-mon[34533]: from='osd.8 [v2:172.21.15.161:6816/52966633,v1:172.21.15.161:6817/52966633]' entity='osd.8' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]} : dispatch 2024-09-21T03:42:05.218 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:42:04 smithi161 ceph-mon[34533]: from='osd.8 ' entity='osd.8' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]} : dispatch 2024-09-21T03:42:05.225 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:42:04 smithi150 ceph-mon[34289]: pgmap v178: 1 pgs: 1 active+clean; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail 2024-09-21T03:42:05.225 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:42:04 smithi150 ceph-mon[34289]: from='osd.8 [v2:172.21.15.161:6816/52966633,v1:172.21.15.161:6817/52966633]' entity='osd.8' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]} : dispatch 2024-09-21T03:42:05.225 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:42:04 smithi150 ceph-mon[34289]: from='osd.8 ' entity='osd.8' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]} : dispatch 2024-09-21T03:42:05.968 INFO:journalctl@ceph.osd.8.smithi161.stdout:Sep 21 03:42:05 smithi161 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-8[52151]: 2024-09-21T03:42:05.834+0000 7f7e9caf5640 -1 osd.8 0 waiting for initial osdmap 2024-09-21T03:42:05.968 INFO:journalctl@ceph.osd.8.smithi161.stdout:Sep 21 03:42:05 smithi161 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-8[52151]: 2024-09-21T03:42:05.850+0000 7f7e988f3640 -1 osd.8 53 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-21T03:42:05.968 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:42:05 smithi161 ceph-mon[34533]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-09-21T03:42:05.968 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:42:05 smithi161 ceph-mon[34533]: from='osd.8 [v2:172.21.15.161:6816/52966633,v1:172.21.15.161:6817/52966633]' entity='osd.8' cmd={"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi161", "root=default"]} : dispatch 2024-09-21T03:42:05.968 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:42:05 smithi161 ceph-mon[34533]: osdmap e52: 9 total, 8 up, 9 in 2024-09-21T03:42:05.968 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:42:05 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2024-09-21T03:42:05.969 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:42:05 smithi161 ceph-mon[34533]: from='osd.8 ' entity='osd.8' cmd={"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi161", "root=default"]} : dispatch 2024-09-21T03:42:05.969 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:42:05 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:42:05.969 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:42:05 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:42:05.969 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:42:05 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-09-21T03:42:05.969 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:42:05 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-09-21T03:42:05.969 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:42:05 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"} : dispatch 2024-09-21T03:42:05.969 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:42:05 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:42:05.969 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:42:05 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:42:05.969 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:42:05 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:42:05.969 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:42:05 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:42:05.969 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:42:05 smithi161 ceph-mon[34533]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi161", "root=default"]}]': finished 2024-09-21T03:42:05.969 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:42:05 smithi161 ceph-mon[34533]: osdmap e53: 9 total, 8 up, 9 in 2024-09-21T03:42:05.969 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:42:05 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2024-09-21T03:42:05.982 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T03:42:06.013 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 51116d5c-77ca-11ef-baf5-efdc52797490 -- ceph orch daemon add osd smithi039:vg_nvme/lv_1 2024-09-21T03:42:06.225 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:42:05 smithi150 ceph-mon[34289]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-09-21T03:42:06.225 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:42:05 smithi150 ceph-mon[34289]: from='osd.8 [v2:172.21.15.161:6816/52966633,v1:172.21.15.161:6817/52966633]' entity='osd.8' cmd={"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi161", "root=default"]} : dispatch 2024-09-21T03:42:06.225 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:42:05 smithi150 ceph-mon[34289]: osdmap e52: 9 total, 8 up, 9 in 2024-09-21T03:42:06.225 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:42:05 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2024-09-21T03:42:06.225 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:42:05 smithi150 ceph-mon[34289]: from='osd.8 ' entity='osd.8' cmd={"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi161", "root=default"]} : dispatch 2024-09-21T03:42:06.225 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:42:05 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:42:06.225 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:42:05 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:42:06.225 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:42:05 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-09-21T03:42:06.225 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:42:05 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-09-21T03:42:06.225 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:42:05 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"} : dispatch 2024-09-21T03:42:06.226 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:42:05 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:42:06.226 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:42:05 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:42:06.226 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:42:05 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:42:06.226 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:42:05 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:42:06.226 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:42:05 smithi150 ceph-mon[34289]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi161", "root=default"]}]': finished 2024-09-21T03:42:06.226 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:42:05 smithi150 ceph-mon[34289]: osdmap e53: 9 total, 8 up, 9 in 2024-09-21T03:42:06.226 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:42:05 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2024-09-21T03:42:06.269 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/51116d5c-77ca-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:42:06.333 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:42:05 smithi039 ceph-mon[31211]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-09-21T03:42:06.334 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:42:05 smithi039 ceph-mon[31211]: from='osd.8 [v2:172.21.15.161:6816/52966633,v1:172.21.15.161:6817/52966633]' entity='osd.8' cmd={"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi161", "root=default"]} : dispatch 2024-09-21T03:42:06.334 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:42:05 smithi039 ceph-mon[31211]: osdmap e52: 9 total, 8 up, 9 in 2024-09-21T03:42:06.334 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:42:05 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2024-09-21T03:42:06.334 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:42:05 smithi039 ceph-mon[31211]: from='osd.8 ' entity='osd.8' cmd={"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi161", "root=default"]} : dispatch 2024-09-21T03:42:06.334 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:42:05 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:42:06.334 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:42:05 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:42:06.334 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:42:05 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-09-21T03:42:06.334 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:42:05 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-09-21T03:42:06.334 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:42:05 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"} : dispatch 2024-09-21T03:42:06.334 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:42:05 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:42:06.334 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:42:05 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:42:06.335 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:42:05 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:42:06.335 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:42:05 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:42:06.335 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:42:05 smithi039 ceph-mon[31211]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi161", "root=default"]}]': finished 2024-09-21T03:42:06.335 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:42:05 smithi039 ceph-mon[31211]: osdmap e53: 9 total, 8 up, 9 in 2024-09-21T03:42:06.335 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:42:05 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2024-09-21T03:42:06.744 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:42:06.739+0000 7fcef874f640 1 -- 172.21.15.39:0/630541060 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcef0103b60 msgr2=0x7fcef0103f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:06.744 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:42:06.739+0000 7fcef874f640 1 --2- 172.21.15.39:0/630541060 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcef0103b60 0x7fcef0103f70 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fceec009900 tx=0x7fceec02a410 comp rx=0 tx=0).stop 2024-09-21T03:42:06.745 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:42:06.740+0000 7fcef874f640 1 -- 172.21.15.39:0/630541060 shutdown_connections 2024-09-21T03:42:06.745 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:42:06.740+0000 7fcef874f640 1 --2- 172.21.15.39:0/630541060 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcef0109df0 0x7fcef0110620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:06.745 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:42:06.740+0000 7fcef874f640 1 --2- 172.21.15.39:0/630541060 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcef01094a0 0x7fcef01098b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:06.745 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:42:06.740+0000 7fcef874f640 1 --2- 172.21.15.39:0/630541060 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcef0103b60 0x7fcef0103f70 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:06.745 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:42:06.740+0000 7fcef874f640 1 -- 172.21.15.39:0/630541060 >> 172.21.15.39:0/630541060 conn(0x7fcef007b850 msgr2=0x7fcef007bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:42:06.745 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:42:06.740+0000 7fcef874f640 1 -- 172.21.15.39:0/630541060 shutdown_connections 2024-09-21T03:42:06.745 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:42:06.740+0000 7fcef874f640 1 -- 172.21.15.39:0/630541060 wait complete. 2024-09-21T03:42:06.745 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:42:06.740+0000 7fcef874f640 1 Processor -- start 2024-09-21T03:42:06.745 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:42:06.740+0000 7fcef874f640 1 -- start start 2024-09-21T03:42:06.745 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:42:06.741+0000 7fcef874f640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcef0103b60 0x7fcef019a4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:06.745 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:42:06.741+0000 7fcef874f640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcef01094a0 0x7fcef019aa40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:06.745 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:42:06.741+0000 7fcef874f640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcef0109df0 0x7fcef019fce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:06.745 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:42:06.741+0000 7fcef874f640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fcef0113ab0 con 0x7fcef01094a0 2024-09-21T03:42:06.745 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:42:06.741+0000 7fcef874f640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fcef0113930 con 0x7fcef0109df0 2024-09-21T03:42:06.745 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:42:06.741+0000 7fcef874f640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fcef0113c30 con 0x7fcef0103b60 2024-09-21T03:42:06.746 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:42:06.741+0000 7fcef5cea640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcef01094a0 0x7fcef019aa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:06.746 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:42:06.741+0000 7fcef5cea640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcef01094a0 0x7fcef019aa40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:58994/0 (socket says 172.21.15.39:58994) 2024-09-21T03:42:06.746 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:42:06.741+0000 7fcef5cea640 1 -- 172.21.15.39:0/2619980029 learned_addr learned my addr 172.21.15.39:0/2619980029 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:42:06.746 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:42:06.741+0000 7fcef64eb640 1 --2- 172.21.15.39:0/2619980029 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcef0103b60 0x7fcef019a4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:06.746 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:42:06.741+0000 7fcef6cec640 1 --2- 172.21.15.39:0/2619980029 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcef0109df0 0x7fcef019fce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:06.746 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:42:06.741+0000 7fcef5cea640 1 -- 172.21.15.39:0/2619980029 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcef0103b60 msgr2=0x7fcef019a4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:06.746 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:42:06.741+0000 7fcef5cea640 1 --2- 172.21.15.39:0/2619980029 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcef0103b60 0x7fcef019a4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:06.746 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:42:06.741+0000 7fcef5cea640 1 -- 172.21.15.39:0/2619980029 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcef0109df0 msgr2=0x7fcef019fce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:06.746 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:42:06.741+0000 7fcef5cea640 1 --2- 172.21.15.39:0/2619980029 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcef0109df0 0x7fcef019fce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:06.746 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:42:06.741+0000 7fcef5cea640 1 -- 172.21.15.39:0/2619980029 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcef01a0410 con 0x7fcef01094a0 2024-09-21T03:42:06.746 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:42:06.741+0000 7fcef5cea640 1 --2- 172.21.15.39:0/2619980029 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcef01094a0 0x7fcef019aa40 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fcee000d740 tx=0x7fcee000dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:42:06.746 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:42:06.742+0000 7fcee77fe640 1 -- 172.21.15.39:0/2619980029 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcee0017070 con 0x7fcef01094a0 2024-09-21T03:42:06.746 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:42:06.742+0000 7fcef874f640 1 -- 172.21.15.39:0/2619980029 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcef01a0700 con 0x7fcef01094a0 2024-09-21T03:42:06.746 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:42:06.742+0000 7fcee77fe640 1 -- 172.21.15.39:0/2619980029 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcee000de50 con 0x7fcef01094a0 2024-09-21T03:42:06.747 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:42:06.742+0000 7fcee77fe640 1 -- 172.21.15.39:0/2619980029 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcee000f620 con 0x7fcef01094a0 2024-09-21T03:42:06.747 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:42:06.742+0000 7fcef874f640 1 -- 172.21.15.39:0/2619980029 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcef01a6e60 con 0x7fcef01094a0 2024-09-21T03:42:06.747 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:42:06.743+0000 7fcef874f640 1 -- 172.21.15.39:0/2619980029 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcef0103f70 con 0x7fcef01094a0 2024-09-21T03:42:06.747 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:42:06.745+0000 7fcee77fe640 1 -- 172.21.15.39:0/2619980029 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 21) ==== 154529+0+0 (secure 0 0 0) 0x7fcee001a070 con 0x7fcef01094a0 2024-09-21T03:42:06.748 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:42:06.746+0000 7fcee77fe640 1 --2- 172.21.15.39:0/2619980029 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7fcedc0b6560 0x7fcedc0b8a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:06.748 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:42:06.746+0000 7fcef64eb640 1 --2- 172.21.15.39:0/2619980029 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7fcedc0b6560 0x7fcedc0b8a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:06.749 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:42:06.746+0000 7fcee77fe640 1 -- 172.21.15.39:0/2619980029 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(53..53 src has 1..53) ==== 5764+0+0 (secure 0 0 0) 0x7fcee00e6260 con 0x7fcef01094a0 2024-09-21T03:42:06.749 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:42:06.746+0000 7fcef64eb640 1 --2- 172.21.15.39:0/2619980029 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7fcedc0b6560 0x7fcedc0b8a70 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fceec031440 tx=0x7fceec009560 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:42:06.751 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:42:06.749+0000 7fcee77fe640 1 -- 172.21.15.39:0/2619980029 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fcee00b18c0 con 0x7fcef01094a0 2024-09-21T03:42:06.951 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:42:06.948+0000 7fcef874f640 1 -- 172.21.15.39:0/2619980029 --> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi039:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7fcef0063800 con 0x7fcedc0b6560 2024-09-21T03:42:06.951 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:42:06 smithi039 ceph-mon[31211]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail 2024-09-21T03:42:06.951 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:42:06 smithi039 ceph-mon[31211]: Detected new or changed devices on smithi161 2024-09-21T03:42:06.951 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:42:06 smithi039 ceph-mon[31211]: Adjusting osd_memory_target on smithi161 to 5696M 2024-09-21T03:42:06.951 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:42:06 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2024-09-21T03:42:06.951 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:42:06 smithi039 ceph-mon[31211]: osd.8 [v2:172.21.15.161:6816/52966633,v1:172.21.15.161:6817/52966633] boot 2024-09-21T03:42:06.951 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:42:06 smithi039 ceph-mon[31211]: osdmap e54: 9 total, 9 up, 9 in 2024-09-21T03:42:06.952 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:42:06 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2024-09-21T03:42:07.218 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:42:06 smithi161 ceph-mon[34533]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail 2024-09-21T03:42:07.218 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:42:06 smithi161 ceph-mon[34533]: Detected new or changed devices on smithi161 2024-09-21T03:42:07.218 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:42:06 smithi161 ceph-mon[34533]: Adjusting osd_memory_target on smithi161 to 5696M 2024-09-21T03:42:07.218 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:42:06 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2024-09-21T03:42:07.218 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:42:06 smithi161 ceph-mon[34533]: osd.8 [v2:172.21.15.161:6816/52966633,v1:172.21.15.161:6817/52966633] boot 2024-09-21T03:42:07.218 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:42:06 smithi161 ceph-mon[34533]: osdmap e54: 9 total, 9 up, 9 in 2024-09-21T03:42:07.218 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:42:06 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2024-09-21T03:42:07.224 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:42:06 smithi150 ceph-mon[34289]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail 2024-09-21T03:42:07.224 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:42:06 smithi150 ceph-mon[34289]: Detected new or changed devices on smithi161 2024-09-21T03:42:07.224 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:42:06 smithi150 ceph-mon[34289]: Adjusting osd_memory_target on smithi161 to 5696M 2024-09-21T03:42:07.224 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:42:06 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2024-09-21T03:42:07.224 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:42:06 smithi150 ceph-mon[34289]: osd.8 [v2:172.21.15.161:6816/52966633,v1:172.21.15.161:6817/52966633] boot 2024-09-21T03:42:07.225 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:42:06 smithi150 ceph-mon[34289]: osdmap e54: 9 total, 9 up, 9 in 2024-09-21T03:42:07.225 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:42:06 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2024-09-21T03:42:08.068 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:42:07 smithi039 ceph-mon[31211]: purged_snaps scrub starts 2024-09-21T03:42:08.068 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:42:07 smithi039 ceph-mon[31211]: purged_snaps scrub ok 2024-09-21T03:42:08.069 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:42:07 smithi039 ceph-mon[31211]: from='client.14673 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi039:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:42:08.069 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:42:07 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T03:42:08.069 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:42:07 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T03:42:08.069 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:42:07 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:42:08.069 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:42:07 smithi039 ceph-mon[31211]: osdmap e55: 9 total, 9 up, 9 in 2024-09-21T03:42:08.119 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:42:07 smithi161 ceph-mon[34533]: purged_snaps scrub starts 2024-09-21T03:42:08.119 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:42:07 smithi161 ceph-mon[34533]: purged_snaps scrub ok 2024-09-21T03:42:08.119 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:42:07 smithi161 ceph-mon[34533]: from='client.14673 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi039:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:42:08.119 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:42:07 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T03:42:08.119 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:42:07 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T03:42:08.119 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:42:07 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:42:08.119 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:42:07 smithi161 ceph-mon[34533]: osdmap e55: 9 total, 9 up, 9 in 2024-09-21T03:42:08.224 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:42:07 smithi150 ceph-mon[34289]: purged_snaps scrub starts 2024-09-21T03:42:08.224 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:42:07 smithi150 ceph-mon[34289]: purged_snaps scrub ok 2024-09-21T03:42:08.225 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:42:07 smithi150 ceph-mon[34289]: from='client.14673 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi039:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:42:08.225 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:42:07 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T03:42:08.225 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:42:07 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T03:42:08.225 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:42:07 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:42:08.225 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:42:07 smithi150 ceph-mon[34289]: osdmap e55: 9 total, 9 up, 9 in 2024-09-21T03:42:08.888 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:42:08 smithi039 ceph-mon[31211]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail 2024-09-21T03:42:08.888 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:42:08 smithi039 ceph-mon[31211]: osdmap e56: 9 total, 9 up, 9 in 2024-09-21T03:42:09.126 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:42:08 smithi150 ceph-mon[34289]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail 2024-09-21T03:42:09.126 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:42:08 smithi150 ceph-mon[34289]: osdmap e56: 9 total, 9 up, 9 in 2024-09-21T03:42:09.217 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:42:08 smithi161 ceph-mon[34533]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail 2024-09-21T03:42:09.218 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:42:08 smithi161 ceph-mon[34533]: osdmap e56: 9 total, 9 up, 9 in 2024-09-21T03:42:10.084 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:42:09 smithi039 ceph-mon[31211]: from='client.? 172.21.15.39:0/3226144818' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "6028613c-ebfd-42ea-9949-3f0d5f6cf025"} : dispatch 2024-09-21T03:42:10.084 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:42:09 smithi039 ceph-mon[31211]: from='client.? 172.21.15.39:0/3226144818' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "6028613c-ebfd-42ea-9949-3f0d5f6cf025"}]': finished 2024-09-21T03:42:10.084 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:42:09 smithi039 ceph-mon[31211]: osdmap e57: 10 total, 9 up, 10 in 2024-09-21T03:42:10.084 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:42:09 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2024-09-21T03:42:10.217 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:42:09 smithi161 ceph-mon[34533]: from='client.? 172.21.15.39:0/3226144818' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "6028613c-ebfd-42ea-9949-3f0d5f6cf025"} : dispatch 2024-09-21T03:42:10.218 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:42:09 smithi161 ceph-mon[34533]: from='client.? 172.21.15.39:0/3226144818' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "6028613c-ebfd-42ea-9949-3f0d5f6cf025"}]': finished 2024-09-21T03:42:10.218 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:42:09 smithi161 ceph-mon[34533]: osdmap e57: 10 total, 9 up, 10 in 2024-09-21T03:42:10.218 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:42:09 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2024-09-21T03:42:10.224 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:42:09 smithi150 ceph-mon[34289]: from='client.? 172.21.15.39:0/3226144818' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "6028613c-ebfd-42ea-9949-3f0d5f6cf025"} : dispatch 2024-09-21T03:42:10.224 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:42:09 smithi150 ceph-mon[34289]: from='client.? 172.21.15.39:0/3226144818' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "6028613c-ebfd-42ea-9949-3f0d5f6cf025"}]': finished 2024-09-21T03:42:10.225 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:42:09 smithi150 ceph-mon[34289]: osdmap e57: 10 total, 9 up, 10 in 2024-09-21T03:42:10.225 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:42:09 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2024-09-21T03:42:11.467 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:42:11 smithi161 ceph-mon[34533]: pgmap v186: 1 pgs: 1 peering; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail 2024-09-21T03:42:11.468 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:42:11 smithi161 ceph-mon[34533]: from='client.? 172.21.15.39:0/4165717616' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T03:42:11.474 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:42:11 smithi150 ceph-mon[34289]: pgmap v186: 1 pgs: 1 peering; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail 2024-09-21T03:42:11.474 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:42:11 smithi150 ceph-mon[34289]: from='client.? 172.21.15.39:0/4165717616' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T03:42:11.583 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:42:11 smithi039 ceph-mon[31211]: pgmap v186: 1 pgs: 1 peering; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail 2024-09-21T03:42:11.584 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:42:11 smithi039 ceph-mon[31211]: from='client.? 172.21.15.39:0/4165717616' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T03:42:13.467 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:42:13 smithi161 ceph-mon[34533]: pgmap v188: 1 pgs: 1 peering; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail 2024-09-21T03:42:13.474 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:42:13 smithi150 ceph-mon[34289]: pgmap v188: 1 pgs: 1 peering; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail 2024-09-21T03:42:13.583 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:42:13 smithi039 ceph-mon[31211]: pgmap v188: 1 pgs: 1 peering; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail 2024-09-21T03:42:15.420 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:42:15 smithi039 ceph-mon[31211]: pgmap v189: 1 pgs: 1 peering; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail 2024-09-21T03:42:15.467 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:42:15 smithi161 ceph-mon[34533]: pgmap v189: 1 pgs: 1 peering; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail 2024-09-21T03:42:15.474 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:42:15 smithi150 ceph-mon[34289]: pgmap v189: 1 pgs: 1 peering; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail 2024-09-21T03:42:17.281 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:42:17 smithi039 ceph-mon[31211]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail; 76 KiB/s, 0 objects/s recovering 2024-09-21T03:42:17.467 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:42:17 smithi161 ceph-mon[34533]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail; 76 KiB/s, 0 objects/s recovering 2024-09-21T03:42:17.474 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:42:17 smithi150 ceph-mon[34289]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail; 76 KiB/s, 0 objects/s recovering 2024-09-21T03:42:19.334 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:42:19 smithi039 ceph-mon[31211]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail; 67 KiB/s, 0 objects/s recovering 2024-09-21T03:42:19.425 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:42:19 smithi161 ceph-mon[34533]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail; 67 KiB/s, 0 objects/s recovering 2024-09-21T03:42:19.474 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:42:19 smithi150 ceph-mon[34289]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail; 67 KiB/s, 0 objects/s recovering 2024-09-21T03:42:20.157 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:42:20 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.9"} : dispatch 2024-09-21T03:42:20.157 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:42:20 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:42:20.468 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:42:20 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.9"} : dispatch 2024-09-21T03:42:20.468 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:42:20 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:42:20.474 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:42:20 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.9"} : dispatch 2024-09-21T03:42:20.474 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:42:20 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:42:21.190 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:42:21 smithi039 ceph-mon[31211]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-21T03:42:21.190 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:42:21 smithi039 ceph-mon[31211]: Deploying daemon osd.9 on smithi039 2024-09-21T03:42:21.467 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:42:21 smithi161 ceph-mon[34533]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-21T03:42:21.468 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:42:21 smithi161 ceph-mon[34533]: Deploying daemon osd.9 on smithi039 2024-09-21T03:42:21.474 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:42:21 smithi150 ceph-mon[34289]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-21T03:42:21.474 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:42:21 smithi150 ceph-mon[34289]: Deploying daemon osd.9 on smithi039 2024-09-21T03:42:23.278 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:42:23 smithi039 ceph-mon[31211]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-21T03:42:23.467 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:42:23 smithi161 ceph-mon[34533]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-21T03:42:23.474 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:42:23 smithi150 ceph-mon[34289]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-21T03:42:25.272 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:42:25 smithi039 ceph-mon[31211]: pgmap v194: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-21T03:42:25.467 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:42:25 smithi161 ceph-mon[34533]: pgmap v194: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-21T03:42:25.474 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:42:25 smithi150 ceph-mon[34289]: pgmap v194: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-21T03:42:26.296 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:42:26 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:42:26.296 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:42:26 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:42:26.296 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:42:26 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:42:26.468 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:42:26 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:42:26.468 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:42:26 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:42:26.468 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:42:26 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:42:26.474 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:42:26 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:42:26.474 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:42:26 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:42:26.475 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:42:26 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:42:27.285 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:42:27 smithi039 ceph-mon[31211]: pgmap v195: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-21T03:42:27.467 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:42:27 smithi161 ceph-mon[34533]: pgmap v195: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-21T03:42:27.474 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:42:27 smithi150 ceph-mon[34289]: pgmap v195: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-21T03:42:28.545 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:42:28.543+0000 7fcee77fe640 1 -- 172.21.15.39:0/2619980029 <== mgr.14223 v2:172.21.15.39:6800/1696381982 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fcef0063800 con 0x7fcedc0b6560 2024-09-21T03:42:28.545 INFO:teuthology.orchestra.run.smithi039.stdout:Created osd(s) 9 on host 'smithi039' 2024-09-21T03:42:28.547 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:42:28.545+0000 7fcef874f640 1 -- 172.21.15.39:0/2619980029 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7fcedc0b6560 msgr2=0x7fcedc0b8a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:28.547 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:42:28.545+0000 7fcef874f640 1 --2- 172.21.15.39:0/2619980029 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7fcedc0b6560 0x7fcedc0b8a70 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fceec031440 tx=0x7fceec009560 comp rx=0 tx=0).stop 2024-09-21T03:42:28.547 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:42:28.545+0000 7fcef874f640 1 -- 172.21.15.39:0/2619980029 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcef01094a0 msgr2=0x7fcef019aa40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:28.547 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:42:28.545+0000 7fcef874f640 1 --2- 172.21.15.39:0/2619980029 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcef01094a0 0x7fcef019aa40 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fcee000d740 tx=0x7fcee000dc00 comp rx=0 tx=0).stop 2024-09-21T03:42:28.547 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:42:28.545+0000 7fcef874f640 1 -- 172.21.15.39:0/2619980029 shutdown_connections 2024-09-21T03:42:28.547 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:42:28.545+0000 7fcef874f640 1 --2- 172.21.15.39:0/2619980029 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7fcedc0b6560 0x7fcedc0b8a70 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:28.547 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:42:28.545+0000 7fcef874f640 1 --2- 172.21.15.39:0/2619980029 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcef0109df0 0x7fcef019fce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:28.547 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:42:28.545+0000 7fcef874f640 1 --2- 172.21.15.39:0/2619980029 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcef01094a0 0x7fcef019aa40 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:28.548 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:42:28.545+0000 7fcef874f640 1 --2- 172.21.15.39:0/2619980029 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcef0103b60 0x7fcef019a4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:28.548 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:42:28.545+0000 7fcef874f640 1 -- 172.21.15.39:0/2619980029 >> 172.21.15.39:0/2619980029 conn(0x7fcef007b850 msgr2=0x7fcef0107530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:42:28.548 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:42:28.545+0000 7fcef874f640 1 -- 172.21.15.39:0/2619980029 shutdown_connections 2024-09-21T03:42:28.548 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:42:28.545+0000 7fcef874f640 1 -- 172.21.15.39:0/2619980029 wait complete. 2024-09-21T03:42:29.468 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:42:29 smithi161 ceph-mon[34533]: pgmap v196: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail 2024-09-21T03:42:29.468 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:42:29 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:42:29.468 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:42:29 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:42:29.474 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:42:29 smithi150 ceph-mon[34289]: pgmap v196: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail 2024-09-21T03:42:29.474 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:42:29 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:42:29.474 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:42:29 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:42:29.583 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:42:29 smithi039 ceph-mon[31211]: pgmap v196: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail 2024-09-21T03:42:29.584 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:42:29 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:42:29.584 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:42:29 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:42:29.620 DEBUG:teuthology.orchestra.run.smithi039:osd.9> sudo journalctl -f -n 0 -u ceph-51116d5c-77ca-11ef-baf5-efdc52797490@osd.9.service 2024-09-21T03:42:29.622 INFO:tasks.cephadm:Deploying osd.10 on smithi150 with /dev/vg_nvme/lv_1... 2024-09-21T03:42:29.622 DEBUG:teuthology.orchestra.run.smithi150:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 51116d5c-77ca-11ef-baf5-efdc52797490 -- lvm zap /dev/vg_nvme/lv_1 2024-09-21T03:42:29.880 INFO:teuthology.orchestra.run.smithi150.stderr:Inferring config /var/lib/ceph/51116d5c-77ca-11ef-baf5-efdc52797490/mon.b/config 2024-09-21T03:42:31.197 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:42:30 smithi150 ceph-mon[34289]: pgmap v197: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail 2024-09-21T03:42:31.198 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:42:30 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:42:31.198 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:42:30 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:42:31.217 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:42:30 smithi161 ceph-mon[34533]: pgmap v197: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail 2024-09-21T03:42:31.218 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:42:30 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:42:31.218 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:42:30 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:42:31.334 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:42:30 smithi039 ceph-mon[31211]: pgmap v197: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail 2024-09-21T03:42:31.334 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:42:30 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:42:31.334 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:42:30 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:42:31.834 INFO:journalctl@ceph.osd.9.smithi039.stdout:Sep 21 03:42:31 smithi039 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-9[67006]: 2024-09-21T03:42:31.646+0000 7f4a1a341640 -1 osd.9 0 log_to_monitors true 2024-09-21T03:42:32.093 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:42:31 smithi039 ceph-mon[31211]: from='osd.9 [v2:172.21.15.39:6826/2001303253,v1:172.21.15.39:6827/2001303253]' entity='osd.9' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]} : dispatch 2024-09-21T03:42:32.094 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:42:31 smithi150 ceph-mon[34289]: from='osd.9 [v2:172.21.15.39:6826/2001303253,v1:172.21.15.39:6827/2001303253]' entity='osd.9' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]} : dispatch 2024-09-21T03:42:32.217 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:42:31 smithi161 ceph-mon[34533]: from='osd.9 [v2:172.21.15.39:6826/2001303253,v1:172.21.15.39:6827/2001303253]' entity='osd.9' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]} : dispatch 2024-09-21T03:42:32.481 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-09-21T03:42:32.508 DEBUG:teuthology.orchestra.run.smithi150:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 51116d5c-77ca-11ef-baf5-efdc52797490 -- ceph orch daemon add osd smithi150:vg_nvme/lv_1 2024-09-21T03:42:32.765 INFO:teuthology.orchestra.run.smithi150.stderr:Inferring config /var/lib/ceph/51116d5c-77ca-11ef-baf5-efdc52797490/mon.b/config 2024-09-21T03:42:33.013 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:42:32 smithi039 ceph-mon[31211]: pgmap v198: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail 2024-09-21T03:42:33.013 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:42:32 smithi039 ceph-mon[31211]: from='osd.9 [v2:172.21.15.39:6826/2001303253,v1:172.21.15.39:6827/2001303253]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-09-21T03:42:33.013 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:42:32 smithi039 ceph-mon[31211]: osdmap e58: 10 total, 9 up, 10 in 2024-09-21T03:42:33.014 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:42:32 smithi039 ceph-mon[31211]: from='osd.9 [v2:172.21.15.39:6826/2001303253,v1:172.21.15.39:6827/2001303253]' entity='osd.9' cmd={"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi039", "root=default"]} : dispatch 2024-09-21T03:42:33.014 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:42:32 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2024-09-21T03:42:33.014 INFO:journalctl@ceph.osd.9.smithi039.stdout:Sep 21 03:42:32 smithi039 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-9[67006]: 2024-09-21T03:42:32.863+0000 7f4a162c3640 -1 osd.9 0 waiting for initial osdmap 2024-09-21T03:42:33.014 INFO:journalctl@ceph.osd.9.smithi039.stdout:Sep 21 03:42:32 smithi039 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-9[67006]: 2024-09-21T03:42:32.881+0000 7f4a120c1640 -1 osd.9 59 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-21T03:42:33.062 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:42:32 smithi150 ceph-mon[34289]: pgmap v198: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail 2024-09-21T03:42:33.062 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:42:32 smithi150 ceph-mon[34289]: from='osd.9 [v2:172.21.15.39:6826/2001303253,v1:172.21.15.39:6827/2001303253]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-09-21T03:42:33.062 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:42:32 smithi150 ceph-mon[34289]: osdmap e58: 10 total, 9 up, 10 in 2024-09-21T03:42:33.062 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:42:32 smithi150 ceph-mon[34289]: from='osd.9 [v2:172.21.15.39:6826/2001303253,v1:172.21.15.39:6827/2001303253]' entity='osd.9' cmd={"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi039", "root=default"]} : dispatch 2024-09-21T03:42:33.062 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:42:32 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2024-09-21T03:42:33.217 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:42:32 smithi161 ceph-mon[34533]: pgmap v198: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail 2024-09-21T03:42:33.218 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:42:32 smithi161 ceph-mon[34533]: from='osd.9 [v2:172.21.15.39:6826/2001303253,v1:172.21.15.39:6827/2001303253]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-09-21T03:42:33.218 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:42:32 smithi161 ceph-mon[34533]: osdmap e58: 10 total, 9 up, 10 in 2024-09-21T03:42:33.218 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:42:32 smithi161 ceph-mon[34533]: from='osd.9 [v2:172.21.15.39:6826/2001303253,v1:172.21.15.39:6827/2001303253]' entity='osd.9' cmd={"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi039", "root=default"]} : dispatch 2024-09-21T03:42:33.218 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:42:32 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2024-09-21T03:42:33.298 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T03:42:33.285+0000 7f03125f7640 1 -- 172.21.15.150:0/103252362 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f030c108d30 msgr2=0x7f030c10f560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:33.298 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T03:42:33.285+0000 7f03125f7640 1 --2- 172.21.15.150:0/103252362 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f030c108d30 0x7f030c10f560 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f0304009ca0 tx=0x7f030402a8c0 comp rx=0 tx=0).stop 2024-09-21T03:42:33.298 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T03:42:33.285+0000 7f03125f7640 1 -- 172.21.15.150:0/103252362 shutdown_connections 2024-09-21T03:42:33.299 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T03:42:33.285+0000 7f03125f7640 1 --2- 172.21.15.150:0/103252362 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f030c108d30 0x7f030c10f560 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:33.299 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T03:42:33.285+0000 7f03125f7640 1 --2- 172.21.15.150:0/103252362 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f030c1083e0 0x7f030c1087f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:33.299 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T03:42:33.285+0000 7f03125f7640 1 --2- 172.21.15.150:0/103252362 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f030c102aa0 0x7f030c102eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:33.299 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T03:42:33.285+0000 7f03125f7640 1 -- 172.21.15.150:0/103252362 >> 172.21.15.150:0/103252362 conn(0x7f030c0797e0 msgr2=0x7f030c079bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:42:33.299 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T03:42:33.285+0000 7f03125f7640 1 -- 172.21.15.150:0/103252362 shutdown_connections 2024-09-21T03:42:33.299 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T03:42:33.285+0000 7f03125f7640 1 -- 172.21.15.150:0/103252362 wait complete. 2024-09-21T03:42:33.299 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T03:42:33.286+0000 7f03125f7640 1 Processor -- start 2024-09-21T03:42:33.299 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T03:42:33.286+0000 7f03125f7640 1 -- start start 2024-09-21T03:42:33.299 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T03:42:33.286+0000 7f03125f7640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f030c102aa0 0x7f030c19b310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:33.299 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T03:42:33.286+0000 7f03125f7640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f030c1083e0 0x7f030c19b850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:33.299 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T03:42:33.286+0000 7f03125f7640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f030c108d30 0x7f030c19bd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:33.299 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T03:42:33.286+0000 7f03125f7640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f030c112a80 con 0x7f030c108d30 2024-09-21T03:42:33.299 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T03:42:33.286+0000 7f03125f7640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f030c112900 con 0x7f030c1083e0 2024-09-21T03:42:33.300 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T03:42:33.286+0000 7f03125f7640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f030c112c00 con 0x7f030c102aa0 2024-09-21T03:42:33.300 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T03:42:33.286+0000 7f0303fff640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f030c1083e0 0x7f030c19b850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:33.300 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T03:42:33.286+0000 7f0303fff640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f030c1083e0 0x7f030c19b850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:54436/0 (socket says 172.21.15.150:54436) 2024-09-21T03:42:33.300 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T03:42:33.286+0000 7f0303fff640 1 -- 172.21.15.150:0/3699633880 learned_addr learned my addr 172.21.15.150:0/3699633880 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-09-21T03:42:33.300 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T03:42:33.286+0000 7f030bfff640 1 --2- 172.21.15.150:0/3699633880 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f030c102aa0 0x7f030c19b310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:33.300 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T03:42:33.287+0000 7f0310b94640 1 --2- 172.21.15.150:0/3699633880 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f030c108d30 0x7f030c19bd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:33.300 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T03:42:33.287+0000 7f0303fff640 1 -- 172.21.15.150:0/3699633880 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f030c102aa0 msgr2=0x7f030c19b310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:33.300 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T03:42:33.287+0000 7f0303fff640 1 --2- 172.21.15.150:0/3699633880 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f030c102aa0 0x7f030c19b310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:33.300 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T03:42:33.287+0000 7f0303fff640 1 -- 172.21.15.150:0/3699633880 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f030c108d30 msgr2=0x7f030c19bd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:33.300 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T03:42:33.287+0000 7f0303fff640 1 --2- 172.21.15.150:0/3699633880 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f030c108d30 0x7f030c19bd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:33.300 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T03:42:33.287+0000 7f0303fff640 1 -- 172.21.15.150:0/3699633880 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f030c1a6620 con 0x7f030c1083e0 2024-09-21T03:42:33.300 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T03:42:33.287+0000 7f030bfff640 1 --2- 172.21.15.150:0/3699633880 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f030c102aa0 0x7f030c19b310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:42:33.300 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T03:42:33.287+0000 7f0303fff640 1 --2- 172.21.15.150:0/3699633880 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f030c1083e0 0x7f030c19b850 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f02f400b450 tx=0x7f02f400b910 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:42:33.300 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T03:42:33.287+0000 7f0309ffb640 1 -- 172.21.15.150:0/3699633880 <== mon.2 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02f4015020 con 0x7f030c1083e0 2024-09-21T03:42:33.300 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T03:42:33.287+0000 7f03125f7640 1 -- 172.21.15.150:0/3699633880 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f030c1a6910 con 0x7f030c1083e0 2024-09-21T03:42:33.301 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T03:42:33.287+0000 7f0309ffb640 1 -- 172.21.15.150:0/3699633880 <== mon.2 v2:172.21.15.150:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f02f4011d40 con 0x7f030c1083e0 2024-09-21T03:42:33.301 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T03:42:33.287+0000 7f0309ffb640 1 -- 172.21.15.150:0/3699633880 <== mon.2 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02f400f340 con 0x7f030c1083e0 2024-09-21T03:42:33.301 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T03:42:33.287+0000 7f03125f7640 1 -- 172.21.15.150:0/3699633880 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f030c1a6e80 con 0x7f030c1083e0 2024-09-21T03:42:33.301 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T03:42:33.289+0000 7f03125f7640 1 -- 172.21.15.150:0/3699633880 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f030c102f30 con 0x7f030c1083e0 2024-09-21T03:42:33.301 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T03:42:33.290+0000 7f0309ffb640 1 -- 172.21.15.150:0/3699633880 <== mon.2 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 21) ==== 154529+0+0 (secure 0 0 0) 0x7f02f4018020 con 0x7f030c1083e0 2024-09-21T03:42:33.301 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T03:42:33.290+0000 7f0309ffb640 1 --2- 172.21.15.150:0/3699633880 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f02f80b6560 0x7f02f80b8a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:33.301 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T03:42:33.291+0000 7f030bfff640 1 --2- 172.21.15.150:0/3699633880 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f02f80b6560 0x7f02f80b8a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:33.301 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T03:42:33.291+0000 7f0309ffb640 1 -- 172.21.15.150:0/3699633880 <== mon.2 v2:172.21.15.150:3300/0 5 ==== osd_map(59..59 src has 1..59) ==== 6196+0+0 (secure 0 0 0) 0x7f02f40e65f0 con 0x7f030c1083e0 2024-09-21T03:42:33.301 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T03:42:33.291+0000 7f030bfff640 1 --2- 172.21.15.150:0/3699633880 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f02f80b6560 0x7f02f80b8a70 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f030c06c3e0 tx=0x7f02fc008040 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:42:33.301 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T03:42:33.295+0000 7f0309ffb640 1 -- 172.21.15.150:0/3699633880 <== mon.2 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f02f401f020 con 0x7f030c1083e0 2024-09-21T03:42:33.487 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T03:42:33.485+0000 7f03125f7640 1 -- 172.21.15.150:0/3699633880 --> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi150:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7f030c063800 con 0x7f02f80b6560 2024-09-21T03:42:34.118 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:42:33 smithi150 ceph-mon[34289]: from='osd.9 [v2:172.21.15.39:6826/2001303253,v1:172.21.15.39:6827/2001303253]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi039", "root=default"]}]': finished 2024-09-21T03:42:34.118 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:42:33 smithi150 ceph-mon[34289]: osdmap e59: 10 total, 9 up, 10 in 2024-09-21T03:42:34.118 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:42:33 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2024-09-21T03:42:34.118 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:42:33 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2024-09-21T03:42:34.118 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:42:33 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:42:34.118 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:42:33 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:42:34.118 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:42:33 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-21T03:42:34.118 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:42:33 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"} : dispatch 2024-09-21T03:42:34.118 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:42:33 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"} : dispatch 2024-09-21T03:42:34.118 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:42:33 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"} : dispatch 2024-09-21T03:42:34.118 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:42:33 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:42:34.118 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:42:33 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:42:34.119 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:42:33 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:42:34.119 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:42:33 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:42:34.119 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:42:33 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T03:42:34.119 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:42:33 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T03:42:34.119 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:42:33 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:42:34.119 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:42:33 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:42:34.119 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:42:33 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:42:34.119 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:42:33 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:42:34.119 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:42:33 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:42:34.218 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:42:33 smithi161 ceph-mon[34533]: from='osd.9 [v2:172.21.15.39:6826/2001303253,v1:172.21.15.39:6827/2001303253]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi039", "root=default"]}]': finished 2024-09-21T03:42:34.218 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:42:33 smithi161 ceph-mon[34533]: osdmap e59: 10 total, 9 up, 10 in 2024-09-21T03:42:34.218 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:42:33 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2024-09-21T03:42:34.218 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:42:33 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2024-09-21T03:42:34.218 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:42:33 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:42:34.218 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:42:33 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:42:34.218 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:42:33 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-21T03:42:34.218 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:42:33 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"} : dispatch 2024-09-21T03:42:34.219 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:42:33 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"} : dispatch 2024-09-21T03:42:34.219 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:42:33 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"} : dispatch 2024-09-21T03:42:34.219 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:42:33 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:42:34.219 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:42:33 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:42:34.219 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:42:33 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:42:34.219 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:42:33 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:42:34.219 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:42:33 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T03:42:34.219 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:42:33 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T03:42:34.219 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:42:33 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:42:34.219 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:42:33 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:42:34.219 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:42:33 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:42:34.219 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:42:33 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:42:34.219 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:42:33 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:42:34.334 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:42:33 smithi039 ceph-mon[31211]: from='osd.9 [v2:172.21.15.39:6826/2001303253,v1:172.21.15.39:6827/2001303253]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi039", "root=default"]}]': finished 2024-09-21T03:42:34.334 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:42:33 smithi039 ceph-mon[31211]: osdmap e59: 10 total, 9 up, 10 in 2024-09-21T03:42:34.334 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:42:33 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2024-09-21T03:42:34.334 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:42:33 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2024-09-21T03:42:34.334 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:42:33 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:42:34.334 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:42:33 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:42:34.334 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:42:33 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-21T03:42:34.334 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:42:33 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"} : dispatch 2024-09-21T03:42:34.334 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:42:33 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"} : dispatch 2024-09-21T03:42:34.334 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:42:33 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"} : dispatch 2024-09-21T03:42:34.334 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:42:33 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:42:34.335 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:42:33 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:42:34.335 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:42:33 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:42:34.335 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:42:33 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:42:34.335 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:42:33 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T03:42:34.335 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:42:33 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T03:42:34.335 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:42:33 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:42:34.335 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:42:33 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:42:34.335 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:42:33 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:42:34.335 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:42:33 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:42:34.335 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:42:33 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:42:35.123 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:42:34 smithi150 ceph-mon[34289]: purged_snaps scrub starts 2024-09-21T03:42:35.123 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:42:34 smithi150 ceph-mon[34289]: purged_snaps scrub ok 2024-09-21T03:42:35.124 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:42:34 smithi150 ceph-mon[34289]: Detected new or changed devices on smithi039 2024-09-21T03:42:35.124 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:42:34 smithi150 ceph-mon[34289]: pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail 2024-09-21T03:42:35.124 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:42:34 smithi150 ceph-mon[34289]: Adjusting osd_memory_target on smithi039 to 4272M 2024-09-21T03:42:35.124 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:42:34 smithi150 ceph-mon[34289]: from='client.24524 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi150:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:42:35.124 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:42:34 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2024-09-21T03:42:35.124 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:42:34 smithi150 ceph-mon[34289]: osd.9 [v2:172.21.15.39:6826/2001303253,v1:172.21.15.39:6827/2001303253] boot 2024-09-21T03:42:35.124 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:42:34 smithi150 ceph-mon[34289]: osdmap e60: 10 total, 10 up, 10 in 2024-09-21T03:42:35.124 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:42:34 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2024-09-21T03:42:35.218 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:42:34 smithi161 ceph-mon[34533]: purged_snaps scrub starts 2024-09-21T03:42:35.218 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:42:34 smithi161 ceph-mon[34533]: purged_snaps scrub ok 2024-09-21T03:42:35.218 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:42:34 smithi161 ceph-mon[34533]: Detected new or changed devices on smithi039 2024-09-21T03:42:35.218 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:42:34 smithi161 ceph-mon[34533]: pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail 2024-09-21T03:42:35.218 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:42:34 smithi161 ceph-mon[34533]: Adjusting osd_memory_target on smithi039 to 4272M 2024-09-21T03:42:35.218 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:42:34 smithi161 ceph-mon[34533]: from='client.24524 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi150:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:42:35.218 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:42:34 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2024-09-21T03:42:35.218 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:42:34 smithi161 ceph-mon[34533]: osd.9 [v2:172.21.15.39:6826/2001303253,v1:172.21.15.39:6827/2001303253] boot 2024-09-21T03:42:35.219 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:42:34 smithi161 ceph-mon[34533]: osdmap e60: 10 total, 10 up, 10 in 2024-09-21T03:42:35.219 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:42:34 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2024-09-21T03:42:35.334 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:42:34 smithi039 ceph-mon[31211]: purged_snaps scrub starts 2024-09-21T03:42:35.334 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:42:34 smithi039 ceph-mon[31211]: purged_snaps scrub ok 2024-09-21T03:42:35.334 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:42:34 smithi039 ceph-mon[31211]: Detected new or changed devices on smithi039 2024-09-21T03:42:35.334 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:42:34 smithi039 ceph-mon[31211]: pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail 2024-09-21T03:42:35.334 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:42:34 smithi039 ceph-mon[31211]: Adjusting osd_memory_target on smithi039 to 4272M 2024-09-21T03:42:35.334 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:42:34 smithi039 ceph-mon[31211]: from='client.24524 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi150:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:42:35.334 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:42:34 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2024-09-21T03:42:35.334 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:42:34 smithi039 ceph-mon[31211]: osd.9 [v2:172.21.15.39:6826/2001303253,v1:172.21.15.39:6827/2001303253] boot 2024-09-21T03:42:35.334 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:42:34 smithi039 ceph-mon[31211]: osdmap e60: 10 total, 10 up, 10 in 2024-09-21T03:42:35.334 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:42:34 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2024-09-21T03:42:36.725 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:42:36 smithi150 ceph-mon[34289]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 296 MiB used, 894 GiB / 894 GiB avail 2024-09-21T03:42:36.725 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:42:36 smithi150 ceph-mon[34289]: osdmap e61: 10 total, 10 up, 10 in 2024-09-21T03:42:36.725 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:42:36 smithi150 ceph-mon[34289]: from='client.? 172.21.15.150:0/158923596' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "1c71c765-2a6c-4ffe-9fe1-c8adae3d1c32"} : dispatch 2024-09-21T03:42:36.725 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:42:36 smithi150 ceph-mon[34289]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "1c71c765-2a6c-4ffe-9fe1-c8adae3d1c32"} : dispatch 2024-09-21T03:42:36.725 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:42:36 smithi150 ceph-mon[34289]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "1c71c765-2a6c-4ffe-9fe1-c8adae3d1c32"}]': finished 2024-09-21T03:42:36.725 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:42:36 smithi150 ceph-mon[34289]: osdmap e62: 11 total, 10 up, 11 in 2024-09-21T03:42:36.725 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:42:36 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 10} : dispatch 2024-09-21T03:42:36.725 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:42:36 smithi150 ceph-mon[34289]: from='client.? 172.21.15.150:0/2662442382' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T03:42:36.834 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:42:36 smithi039 ceph-mon[31211]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 296 MiB used, 894 GiB / 894 GiB avail 2024-09-21T03:42:36.834 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:42:36 smithi039 ceph-mon[31211]: osdmap e61: 10 total, 10 up, 10 in 2024-09-21T03:42:36.834 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:42:36 smithi039 ceph-mon[31211]: from='client.? 172.21.15.150:0/158923596' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "1c71c765-2a6c-4ffe-9fe1-c8adae3d1c32"} : dispatch 2024-09-21T03:42:36.834 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:42:36 smithi039 ceph-mon[31211]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "1c71c765-2a6c-4ffe-9fe1-c8adae3d1c32"} : dispatch 2024-09-21T03:42:36.834 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:42:36 smithi039 ceph-mon[31211]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "1c71c765-2a6c-4ffe-9fe1-c8adae3d1c32"}]': finished 2024-09-21T03:42:36.834 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:42:36 smithi039 ceph-mon[31211]: osdmap e62: 11 total, 10 up, 11 in 2024-09-21T03:42:36.834 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:42:36 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 10} : dispatch 2024-09-21T03:42:36.834 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:42:36 smithi039 ceph-mon[31211]: from='client.? 172.21.15.150:0/2662442382' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T03:42:36.968 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:42:36 smithi161 ceph-mon[34533]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 296 MiB used, 894 GiB / 894 GiB avail 2024-09-21T03:42:36.968 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:42:36 smithi161 ceph-mon[34533]: osdmap e61: 10 total, 10 up, 10 in 2024-09-21T03:42:36.968 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:42:36 smithi161 ceph-mon[34533]: from='client.? 172.21.15.150:0/158923596' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "1c71c765-2a6c-4ffe-9fe1-c8adae3d1c32"} : dispatch 2024-09-21T03:42:36.968 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:42:36 smithi161 ceph-mon[34533]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "1c71c765-2a6c-4ffe-9fe1-c8adae3d1c32"} : dispatch 2024-09-21T03:42:36.968 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:42:36 smithi161 ceph-mon[34533]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "1c71c765-2a6c-4ffe-9fe1-c8adae3d1c32"}]': finished 2024-09-21T03:42:36.968 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:42:36 smithi161 ceph-mon[34533]: osdmap e62: 11 total, 10 up, 11 in 2024-09-21T03:42:36.968 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:42:36 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 10} : dispatch 2024-09-21T03:42:36.968 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:42:36 smithi161 ceph-mon[34533]: from='client.? 172.21.15.150:0/2662442382' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T03:42:38.771 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:42:38 smithi161 ceph-mon[34533]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 296 MiB used, 894 GiB / 894 GiB avail 2024-09-21T03:42:38.833 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:42:38 smithi039 ceph-mon[31211]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 296 MiB used, 894 GiB / 894 GiB avail 2024-09-21T03:42:38.974 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:42:38 smithi150 ceph-mon[34289]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 296 MiB used, 894 GiB / 894 GiB avail 2024-09-21T03:42:40.776 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:42:40 smithi150 ceph-mon[34289]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 296 MiB used, 894 GiB / 894 GiB avail 2024-09-21T03:42:40.833 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:42:40 smithi039 ceph-mon[31211]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 296 MiB used, 894 GiB / 894 GiB avail 2024-09-21T03:42:40.968 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:42:40 smithi161 ceph-mon[34533]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 296 MiB used, 894 GiB / 894 GiB avail 2024-09-21T03:42:42.589 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:42:42 smithi150 ceph-mon[34289]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 296 MiB used, 894 GiB / 894 GiB avail 2024-09-21T03:42:42.834 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:42:42 smithi039 ceph-mon[31211]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 296 MiB used, 894 GiB / 894 GiB avail 2024-09-21T03:42:42.967 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:42:42 smithi161 ceph-mon[34533]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 296 MiB used, 894 GiB / 894 GiB avail 2024-09-21T03:42:44.602 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:42:44 smithi150 ceph-mon[34289]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 296 MiB used, 894 GiB / 894 GiB avail 2024-09-21T03:42:44.833 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:42:44 smithi039 ceph-mon[31211]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 296 MiB used, 894 GiB / 894 GiB avail 2024-09-21T03:42:44.967 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:42:44 smithi161 ceph-mon[34533]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 296 MiB used, 894 GiB / 894 GiB avail 2024-09-21T03:42:45.725 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:42:45 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.10"} : dispatch 2024-09-21T03:42:45.725 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:42:45 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:42:45.725 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:42:45 smithi150 ceph-mon[34289]: Deploying daemon osd.10 on smithi150 2024-09-21T03:42:45.833 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:42:45 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.10"} : dispatch 2024-09-21T03:42:45.833 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:42:45 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:42:45.834 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:42:45 smithi039 ceph-mon[31211]: Deploying daemon osd.10 on smithi150 2024-09-21T03:42:45.967 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:42:45 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.10"} : dispatch 2024-09-21T03:42:45.968 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:42:45 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:42:45.968 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:42:45 smithi161 ceph-mon[34533]: Deploying daemon osd.10 on smithi150 2024-09-21T03:42:46.603 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:42:46 smithi150 ceph-mon[34289]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 296 MiB used, 894 GiB / 894 GiB avail 2024-09-21T03:42:46.834 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:42:46 smithi039 ceph-mon[31211]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 296 MiB used, 894 GiB / 894 GiB avail 2024-09-21T03:42:46.968 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:42:46 smithi161 ceph-mon[34533]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 296 MiB used, 894 GiB / 894 GiB avail 2024-09-21T03:42:48.725 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:42:48 smithi150 ceph-mon[34289]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 296 MiB used, 894 GiB / 894 GiB avail 2024-09-21T03:42:48.833 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:42:48 smithi039 ceph-mon[31211]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 296 MiB used, 894 GiB / 894 GiB avail 2024-09-21T03:42:48.967 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:42:48 smithi161 ceph-mon[34533]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 296 MiB used, 894 GiB / 894 GiB avail 2024-09-21T03:42:50.725 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:42:50 smithi150 ceph-mon[34289]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 296 MiB used, 894 GiB / 894 GiB avail 2024-09-21T03:42:50.725 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:42:50 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:42:50.725 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:42:50 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:42:50.725 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:42:50 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:42:50.833 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:42:50 smithi039 ceph-mon[31211]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 296 MiB used, 894 GiB / 894 GiB avail 2024-09-21T03:42:50.834 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:42:50 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:42:50.834 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:42:50 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:42:50.834 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:42:50 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:42:50.968 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:42:50 smithi161 ceph-mon[34533]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 296 MiB used, 894 GiB / 894 GiB avail 2024-09-21T03:42:50.968 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:42:50 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:42:50.968 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:42:50 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:42:50.968 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:42:50 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:42:52.834 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:42:52 smithi039 ceph-mon[31211]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 296 MiB used, 894 GiB / 894 GiB avail 2024-09-21T03:42:52.941 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:42:52 smithi150 ceph-mon[34289]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 296 MiB used, 894 GiB / 894 GiB avail 2024-09-21T03:42:52.967 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:42:52 smithi161 ceph-mon[34533]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 296 MiB used, 894 GiB / 894 GiB avail 2024-09-21T03:42:54.033 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T03:42:54.029+0000 7f0309ffb640 1 -- 172.21.15.150:0/3699633880 <== mgr.14223 v2:172.21.15.39:6800/1696381982 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7f030c063800 con 0x7f02f80b6560 2024-09-21T03:42:54.033 INFO:teuthology.orchestra.run.smithi150.stdout:Created osd(s) 10 on host 'smithi150' 2024-09-21T03:42:54.034 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T03:42:54.032+0000 7f03125f7640 1 -- 172.21.15.150:0/3699633880 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f02f80b6560 msgr2=0x7f02f80b8a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:54.034 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T03:42:54.032+0000 7f03125f7640 1 --2- 172.21.15.150:0/3699633880 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f02f80b6560 0x7f02f80b8a70 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f030c06c3e0 tx=0x7f02fc008040 comp rx=0 tx=0).stop 2024-09-21T03:42:54.034 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T03:42:54.032+0000 7f03125f7640 1 -- 172.21.15.150:0/3699633880 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f030c1083e0 msgr2=0x7f030c19b850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:54.034 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T03:42:54.032+0000 7f03125f7640 1 --2- 172.21.15.150:0/3699633880 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f030c1083e0 0x7f030c19b850 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f02f400b450 tx=0x7f02f400b910 comp rx=0 tx=0).stop 2024-09-21T03:42:54.034 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T03:42:54.032+0000 7f03125f7640 1 -- 172.21.15.150:0/3699633880 shutdown_connections 2024-09-21T03:42:54.034 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T03:42:54.032+0000 7f03125f7640 1 --2- 172.21.15.150:0/3699633880 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f02f80b6560 0x7f02f80b8a70 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:54.035 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T03:42:54.032+0000 7f03125f7640 1 --2- 172.21.15.150:0/3699633880 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f030c108d30 0x7f030c19bd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:54.035 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T03:42:54.032+0000 7f03125f7640 1 --2- 172.21.15.150:0/3699633880 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f030c1083e0 0x7f030c19b850 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:54.035 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T03:42:54.032+0000 7f03125f7640 1 --2- 172.21.15.150:0/3699633880 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f030c102aa0 0x7f030c19b310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:54.035 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T03:42:54.032+0000 7f03125f7640 1 -- 172.21.15.150:0/3699633880 >> 172.21.15.150:0/3699633880 conn(0x7f030c0797e0 msgr2=0x7f030c1052a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:42:54.035 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T03:42:54.032+0000 7f03125f7640 1 -- 172.21.15.150:0/3699633880 shutdown_connections 2024-09-21T03:42:54.035 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T03:42:54.032+0000 7f03125f7640 1 -- 172.21.15.150:0/3699633880 wait complete. 2024-09-21T03:42:54.584 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:42:54 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:42:54.584 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:42:54 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:42:54.584 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:42:54 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:42:54.584 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:42:54 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:42:54.584 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:42:54 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:42:54.584 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:42:54 smithi039 ceph-mon[31211]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 296 MiB used, 894 GiB / 894 GiB avail 2024-09-21T03:42:54.584 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:42:54 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:42:54.584 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:42:54 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:42:54.584 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:42:54 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:42:54.682 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:42:54 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:42:54.682 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:42:54 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:42:54.683 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:42:54 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:42:54.683 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:42:54 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:42:54.683 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:42:54 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:42:54.683 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:42:54 smithi150 ceph-mon[34289]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 296 MiB used, 894 GiB / 894 GiB avail 2024-09-21T03:42:54.683 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:42:54 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:42:54.683 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:42:54 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:42:54.683 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:42:54 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:42:54.709 DEBUG:teuthology.orchestra.run.smithi150:osd.10> sudo journalctl -f -n 0 -u ceph-51116d5c-77ca-11ef-baf5-efdc52797490@osd.10.service 2024-09-21T03:42:54.711 INFO:tasks.cephadm:Deploying osd.11 on smithi161 with /dev/vg_nvme/lv_1... 2024-09-21T03:42:54.711 DEBUG:teuthology.orchestra.run.smithi161:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 51116d5c-77ca-11ef-baf5-efdc52797490 -- lvm zap /dev/vg_nvme/lv_1 2024-09-21T03:42:54.718 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:42:54 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:42:54.718 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:42:54 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:42:54.718 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:42:54 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:42:54.718 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:42:54 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:42:54.718 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:42:54 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:42:54.718 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:42:54 smithi161 ceph-mon[34533]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 296 MiB used, 894 GiB / 894 GiB avail 2024-09-21T03:42:54.718 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:42:54 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:42:54.718 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:42:54 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:42:54.718 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:42:54 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:42:54.969 INFO:teuthology.orchestra.run.smithi161.stderr:Inferring config /var/lib/ceph/51116d5c-77ca-11ef-baf5-efdc52797490/mon.c/config 2024-09-21T03:42:56.474 INFO:journalctl@ceph.osd.10.smithi150.stdout:Sep 21 03:42:56 smithi150 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-10[58206]: 2024-09-21T03:42:56.455+0000 7f3408b5e640 -1 osd.10 0 log_to_monitors true 2024-09-21T03:42:56.718 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:42:56 smithi161 ceph-mon[34533]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 296 MiB used, 894 GiB / 894 GiB avail 2024-09-21T03:42:56.718 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:42:56 smithi161 ceph-mon[34533]: from='osd.10 [v2:172.21.15.150:6824/2484627938,v1:172.21.15.150:6825/2484627938]' entity='osd.10' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]} : dispatch 2024-09-21T03:42:56.718 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:42:56 smithi161 ceph-mon[34533]: from='osd.10 ' entity='osd.10' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]} : dispatch 2024-09-21T03:42:56.833 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:42:56 smithi039 ceph-mon[31211]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 296 MiB used, 894 GiB / 894 GiB avail 2024-09-21T03:42:56.834 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:42:56 smithi039 ceph-mon[31211]: from='osd.10 [v2:172.21.15.150:6824/2484627938,v1:172.21.15.150:6825/2484627938]' entity='osd.10' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]} : dispatch 2024-09-21T03:42:56.834 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:42:56 smithi039 ceph-mon[31211]: from='osd.10 ' entity='osd.10' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]} : dispatch 2024-09-21T03:42:56.942 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:42:56 smithi150 ceph-mon[34289]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 296 MiB used, 894 GiB / 894 GiB avail 2024-09-21T03:42:56.942 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:42:56 smithi150 ceph-mon[34289]: from='osd.10 [v2:172.21.15.150:6824/2484627938,v1:172.21.15.150:6825/2484627938]' entity='osd.10' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]} : dispatch 2024-09-21T03:42:56.942 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:42:56 smithi150 ceph-mon[34289]: from='osd.10 ' entity='osd.10' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]} : dispatch 2024-09-21T03:42:57.510 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-09-21T03:42:57.537 DEBUG:teuthology.orchestra.run.smithi161:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 51116d5c-77ca-11ef-baf5-efdc52797490 -- ceph orch daemon add osd smithi161:vg_nvme/lv_1 2024-09-21T03:42:57.600 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:42:57 smithi161 ceph-mon[34533]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-09-21T03:42:57.601 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:42:57 smithi161 ceph-mon[34533]: osdmap e63: 11 total, 10 up, 11 in 2024-09-21T03:42:57.601 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:42:57 smithi161 ceph-mon[34533]: from='osd.10 [v2:172.21.15.150:6824/2484627938,v1:172.21.15.150:6825/2484627938]' entity='osd.10' cmd={"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi150", "root=default"]} : dispatch 2024-09-21T03:42:57.601 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:42:57 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 10} : dispatch 2024-09-21T03:42:57.601 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:42:57 smithi161 ceph-mon[34533]: from='osd.10 ' entity='osd.10' cmd={"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi150", "root=default"]} : dispatch 2024-09-21T03:42:57.601 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:42:57 smithi161 ceph-mon[34533]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi150", "root=default"]}]': finished 2024-09-21T03:42:57.638 INFO:journalctl@ceph.osd.10.smithi150.stdout:Sep 21 03:42:57 smithi150 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-10[58206]: 2024-09-21T03:42:57.535+0000 7f3404ae0640 -1 osd.10 0 waiting for initial osdmap 2024-09-21T03:42:57.638 INFO:journalctl@ceph.osd.10.smithi150.stdout:Sep 21 03:42:57 smithi150 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-10[58206]: 2024-09-21T03:42:57.550+0000 7f34008de640 -1 osd.10 64 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-21T03:42:57.638 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:42:57 smithi150 ceph-mon[34289]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-09-21T03:42:57.638 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:42:57 smithi150 ceph-mon[34289]: osdmap e63: 11 total, 10 up, 11 in 2024-09-21T03:42:57.638 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:42:57 smithi150 ceph-mon[34289]: from='osd.10 [v2:172.21.15.150:6824/2484627938,v1:172.21.15.150:6825/2484627938]' entity='osd.10' cmd={"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi150", "root=default"]} : dispatch 2024-09-21T03:42:57.638 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:42:57 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 10} : dispatch 2024-09-21T03:42:57.638 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:42:57 smithi150 ceph-mon[34289]: from='osd.10 ' entity='osd.10' cmd={"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi150", "root=default"]} : dispatch 2024-09-21T03:42:57.639 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:42:57 smithi150 ceph-mon[34289]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi150", "root=default"]}]': finished 2024-09-21T03:42:57.833 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:42:57 smithi039 ceph-mon[31211]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-09-21T03:42:57.834 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:42:57 smithi039 ceph-mon[31211]: osdmap e63: 11 total, 10 up, 11 in 2024-09-21T03:42:57.834 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:42:57 smithi039 ceph-mon[31211]: from='osd.10 [v2:172.21.15.150:6824/2484627938,v1:172.21.15.150:6825/2484627938]' entity='osd.10' cmd={"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi150", "root=default"]} : dispatch 2024-09-21T03:42:57.834 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:42:57 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 10} : dispatch 2024-09-21T03:42:57.834 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:42:57 smithi039 ceph-mon[31211]: from='osd.10 ' entity='osd.10' cmd={"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi150", "root=default"]} : dispatch 2024-09-21T03:42:57.834 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:42:57 smithi039 ceph-mon[31211]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi150", "root=default"]}]': finished 2024-09-21T03:42:57.836 INFO:teuthology.orchestra.run.smithi161.stderr:Inferring config /var/lib/ceph/51116d5c-77ca-11ef-baf5-efdc52797490/mon.c/config 2024-09-21T03:42:58.325 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-21T03:42:58.322+0000 7f27e4afc640 1 -- 172.21.15.161:0/1695367564 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f27e0109ec0 msgr2=0x7f27e01106f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:58.325 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-21T03:42:58.322+0000 7f27e4afc640 1 --2- 172.21.15.161:0/1695367564 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f27e0109ec0 0x7f27e01106f0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f27d0009900 tx=0x7f27d002a410 comp rx=0 tx=0).stop 2024-09-21T03:42:58.325 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-21T03:42:58.322+0000 7f27e4afc640 1 -- 172.21.15.161:0/1695367564 shutdown_connections 2024-09-21T03:42:58.325 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-21T03:42:58.322+0000 7f27e4afc640 1 --2- 172.21.15.161:0/1695367564 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f27e0109ec0 0x7f27e01106f0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:58.325 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-21T03:42:58.322+0000 7f27e4afc640 1 --2- 172.21.15.161:0/1695367564 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f27e0109570 0x7f27e0109980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:58.325 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-21T03:42:58.322+0000 7f27e4afc640 1 --2- 172.21.15.161:0/1695367564 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f27e0103c30 0x7f27e0104040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:58.325 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-21T03:42:58.322+0000 7f27e4afc640 1 -- 172.21.15.161:0/1695367564 >> 172.21.15.161:0/1695367564 conn(0x7f27e007b8d0 msgr2=0x7f27e007bce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:42:58.325 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-21T03:42:58.322+0000 7f27e4afc640 1 -- 172.21.15.161:0/1695367564 shutdown_connections 2024-09-21T03:42:58.325 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-21T03:42:58.322+0000 7f27e4afc640 1 -- 172.21.15.161:0/1695367564 wait complete. 2024-09-21T03:42:58.327 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-21T03:42:58.323+0000 7f27e4afc640 1 Processor -- start 2024-09-21T03:42:58.327 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-21T03:42:58.323+0000 7f27e4afc640 1 -- start start 2024-09-21T03:42:58.327 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-21T03:42:58.323+0000 7f27e4afc640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f27e0103c30 0x7f27e0075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:58.328 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-21T03:42:58.323+0000 7f27e4afc640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f27e0109570 0x7f27e0075c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:58.328 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-21T03:42:58.323+0000 7f27e4afc640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f27e0109ec0 0x7f27e0076190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:58.328 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-21T03:42:58.323+0000 7f27e4afc640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f27e0113ae0 con 0x7f27e0109ec0 2024-09-21T03:42:58.328 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-21T03:42:58.323+0000 7f27e4afc640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f27e0113960 con 0x7f27e0103c30 2024-09-21T03:42:58.328 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-21T03:42:58.323+0000 7f27e4afc640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f27e0113c60 con 0x7f27e0109570 2024-09-21T03:42:58.328 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-21T03:42:58.324+0000 7f27de59c640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f27e0103c30 0x7f27e0075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:58.328 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-21T03:42:58.324+0000 7f27ddd9b640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f27e0109570 0x7f27e0075c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:58.328 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-21T03:42:58.324+0000 7f27de59c640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f27e0103c30 0x7f27e0075710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.161:41816/0 (socket says 172.21.15.161:41816) 2024-09-21T03:42:58.328 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-21T03:42:58.324+0000 7f27ddd9b640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f27e0109570 0x7f27e0075c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:50164/0 (socket says 172.21.15.161:50164) 2024-09-21T03:42:58.328 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-21T03:42:58.324+0000 7f27de59c640 1 -- 172.21.15.161:0/217215358 learned_addr learned my addr 172.21.15.161:0/217215358 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-09-21T03:42:58.328 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-21T03:42:58.324+0000 7f27ded9d640 1 --2- 172.21.15.161:0/217215358 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f27e0109ec0 0x7f27e0076190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:58.328 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-21T03:42:58.324+0000 7f27de59c640 1 -- 172.21.15.161:0/217215358 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f27e0109570 msgr2=0x7f27e0075c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:58.328 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-21T03:42:58.324+0000 7f27de59c640 1 --2- 172.21.15.161:0/217215358 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f27e0109570 0x7f27e0075c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:58.328 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-21T03:42:58.324+0000 7f27de59c640 1 -- 172.21.15.161:0/217215358 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f27e0109ec0 msgr2=0x7f27e0076190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:58.329 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-21T03:42:58.324+0000 7f27de59c640 1 --2- 172.21.15.161:0/217215358 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f27e0109ec0 0x7f27e0076190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:58.329 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-21T03:42:58.324+0000 7f27de59c640 1 -- 172.21.15.161:0/217215358 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f27e01a8730 con 0x7f27e0103c30 2024-09-21T03:42:58.329 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-21T03:42:58.324+0000 7f27ddd9b640 1 --2- 172.21.15.161:0/217215358 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f27e0109570 0x7f27e0075c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:42:58.329 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-21T03:42:58.324+0000 7f27ded9d640 1 --2- 172.21.15.161:0/217215358 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f27e0109ec0 0x7f27e0076190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:42:58.329 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-21T03:42:58.324+0000 7f27de59c640 1 --2- 172.21.15.161:0/217215358 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f27e0103c30 0x7f27e0075710 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f27d800ea80 tx=0x7f27d800ef40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:42:58.329 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-21T03:42:58.324+0000 7f27d77fe640 1 -- 172.21.15.161:0/217215358 <== mon.2 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27d800ce40 con 0x7f27e0103c30 2024-09-21T03:42:58.329 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-21T03:42:58.324+0000 7f27e4afc640 1 -- 172.21.15.161:0/217215358 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f27e01a8a20 con 0x7f27e0103c30 2024-09-21T03:42:58.329 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-21T03:42:58.324+0000 7f27d77fe640 1 -- 172.21.15.161:0/217215358 <== mon.2 v2:172.21.15.150:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f27d8005020 con 0x7f27e0103c30 2024-09-21T03:42:58.329 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-21T03:42:58.325+0000 7f27d77fe640 1 -- 172.21.15.161:0/217215358 <== mon.2 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27d80116d0 con 0x7f27e0103c30 2024-09-21T03:42:58.329 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-21T03:42:58.325+0000 7f27e4afc640 1 -- 172.21.15.161:0/217215358 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f27e01a8f60 con 0x7f27e0103c30 2024-09-21T03:42:58.329 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-21T03:42:58.325+0000 7f27e4afc640 1 -- 172.21.15.161:0/217215358 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f27e0104040 con 0x7f27e0103c30 2024-09-21T03:42:58.330 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-21T03:42:58.327+0000 7f27d77fe640 1 -- 172.21.15.161:0/217215358 <== mon.2 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 21) ==== 154529+0+0 (secure 0 0 0) 0x7f27d80054d0 con 0x7f27e0103c30 2024-09-21T03:42:58.330 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-21T03:42:58.328+0000 7f27d77fe640 1 --2- 172.21.15.161:0/217215358 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f27cc0b6560 0x7f27cc0b8a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:58.330 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-21T03:42:58.328+0000 7f27d77fe640 1 -- 172.21.15.161:0/217215358 <== mon.2 v2:172.21.15.150:3300/0 5 ==== osd_map(64..64 src has 1..64) ==== 6629+0+0 (secure 0 0 0) 0x7f27d80e73b0 con 0x7f27e0103c30 2024-09-21T03:42:58.330 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-21T03:42:58.328+0000 7f27ddd9b640 1 --2- 172.21.15.161:0/217215358 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f27cc0b6560 0x7f27cc0b8a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:58.330 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-21T03:42:58.329+0000 7f27ddd9b640 1 --2- 172.21.15.161:0/217215358 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f27cc0b6560 0x7f27cc0b8a70 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f27e01071c0 tx=0x7f27c800a2d0 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:42:58.332 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-21T03:42:58.331+0000 7f27d77fe640 1 -- 172.21.15.161:0/217215358 <== mon.2 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f27d8016030 con 0x7f27e0103c30 2024-09-21T03:42:58.531 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-21T03:42:58.529+0000 7f27e4afc640 1 -- 172.21.15.161:0/217215358 --> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi161:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7f27e0063800 con 0x7f27cc0b6560 2024-09-21T03:42:58.827 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:42:58 smithi161 ceph-mon[34533]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 296 MiB used, 894 GiB / 894 GiB avail 2024-09-21T03:42:58.827 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:42:58 smithi161 ceph-mon[34533]: osdmap e64: 11 total, 10 up, 11 in 2024-09-21T03:42:58.827 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:42:58 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 10} : dispatch 2024-09-21T03:42:58.827 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:42:58 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 10} : dispatch 2024-09-21T03:42:58.827 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:42:58 smithi161 ceph-mon[34533]: Detected new or changed devices on smithi150 2024-09-21T03:42:58.827 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:42:58 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:42:58.828 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:42:58 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:42:58.828 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:42:58 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-09-21T03:42:58.828 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:42:58 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-09-21T03:42:58.828 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:42:58 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"} : dispatch 2024-09-21T03:42:58.828 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:42:58 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"} : dispatch 2024-09-21T03:42:58.828 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:42:58 smithi161 ceph-mon[34533]: Adjusting osd_memory_target on smithi150 to 4272M 2024-09-21T03:42:58.828 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:42:58 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:42:58.828 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:42:58 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:42:58.828 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:42:58 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:42:58.828 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:42:58 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:42:58.828 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:42:58 smithi161 ceph-mon[34533]: osd.10 [v2:172.21.15.150:6824/2484627938,v1:172.21.15.150:6825/2484627938] boot 2024-09-21T03:42:58.828 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:42:58 smithi161 ceph-mon[34533]: osdmap e65: 11 total, 11 up, 11 in 2024-09-21T03:42:58.828 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:42:58 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 10} : dispatch 2024-09-21T03:42:58.828 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:42:58 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T03:42:58.828 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:42:58 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T03:42:58.834 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:42:58 smithi039 ceph-mon[31211]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 296 MiB used, 894 GiB / 894 GiB avail 2024-09-21T03:42:58.834 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:42:58 smithi039 ceph-mon[31211]: osdmap e64: 11 total, 10 up, 11 in 2024-09-21T03:42:58.834 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:42:58 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 10} : dispatch 2024-09-21T03:42:58.834 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:42:58 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 10} : dispatch 2024-09-21T03:42:58.834 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:42:58 smithi039 ceph-mon[31211]: Detected new or changed devices on smithi150 2024-09-21T03:42:58.834 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:42:58 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:42:58.834 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:42:58 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:42:58.834 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:42:58 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-09-21T03:42:58.834 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:42:58 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-09-21T03:42:58.834 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:42:58 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"} : dispatch 2024-09-21T03:42:58.834 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:42:58 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"} : dispatch 2024-09-21T03:42:58.835 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:42:58 smithi039 ceph-mon[31211]: Adjusting osd_memory_target on smithi150 to 4272M 2024-09-21T03:42:58.835 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:42:58 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:42:58.835 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:42:58 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:42:58.835 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:42:58 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:42:58.835 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:42:58 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:42:58.835 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:42:58 smithi039 ceph-mon[31211]: osd.10 [v2:172.21.15.150:6824/2484627938,v1:172.21.15.150:6825/2484627938] boot 2024-09-21T03:42:58.835 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:42:58 smithi039 ceph-mon[31211]: osdmap e65: 11 total, 11 up, 11 in 2024-09-21T03:42:58.835 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:42:58 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 10} : dispatch 2024-09-21T03:42:58.835 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:42:58 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T03:42:58.835 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:42:58 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T03:42:58.975 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:42:58 smithi150 ceph-mon[34289]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 296 MiB used, 894 GiB / 894 GiB avail 2024-09-21T03:42:58.975 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:42:58 smithi150 ceph-mon[34289]: osdmap e64: 11 total, 10 up, 11 in 2024-09-21T03:42:58.975 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:42:58 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 10} : dispatch 2024-09-21T03:42:58.975 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:42:58 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 10} : dispatch 2024-09-21T03:42:58.975 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:42:58 smithi150 ceph-mon[34289]: Detected new or changed devices on smithi150 2024-09-21T03:42:58.975 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:42:58 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:42:58.975 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:42:58 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:42:58.975 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:42:58 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-09-21T03:42:58.975 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:42:58 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-09-21T03:42:58.975 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:42:58 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"} : dispatch 2024-09-21T03:42:58.976 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:42:58 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"} : dispatch 2024-09-21T03:42:58.976 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:42:58 smithi150 ceph-mon[34289]: Adjusting osd_memory_target on smithi150 to 4272M 2024-09-21T03:42:58.976 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:42:58 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:42:58.976 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:42:58 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:42:58.976 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:42:58 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:42:58.976 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:42:58 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:42:58.976 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:42:58 smithi150 ceph-mon[34289]: osd.10 [v2:172.21.15.150:6824/2484627938,v1:172.21.15.150:6825/2484627938] boot 2024-09-21T03:42:58.976 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:42:58 smithi150 ceph-mon[34289]: osdmap e65: 11 total, 11 up, 11 in 2024-09-21T03:42:58.976 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:42:58 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 10} : dispatch 2024-09-21T03:42:58.976 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:42:58 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T03:42:58.976 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:42:58 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T03:42:59.834 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:42:59 smithi039 ceph-mon[31211]: purged_snaps scrub starts 2024-09-21T03:42:59.834 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:42:59 smithi039 ceph-mon[31211]: purged_snaps scrub ok 2024-09-21T03:42:59.834 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:42:59 smithi039 ceph-mon[31211]: from='client.24551 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi161:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:42:59.834 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:42:59 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:42:59.834 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:42:59 smithi039 ceph-mon[31211]: osdmap e66: 11 total, 11 up, 11 in 2024-09-21T03:42:59.859 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:42:59 smithi161 ceph-mon[34533]: purged_snaps scrub starts 2024-09-21T03:42:59.859 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:42:59 smithi161 ceph-mon[34533]: purged_snaps scrub ok 2024-09-21T03:42:59.859 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:42:59 smithi161 ceph-mon[34533]: from='client.24551 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi161:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:42:59.859 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:42:59 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:42:59.859 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:42:59 smithi161 ceph-mon[34533]: osdmap e66: 11 total, 11 up, 11 in 2024-09-21T03:42:59.974 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:42:59 smithi150 ceph-mon[34289]: purged_snaps scrub starts 2024-09-21T03:42:59.974 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:42:59 smithi150 ceph-mon[34289]: purged_snaps scrub ok 2024-09-21T03:42:59.975 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:42:59 smithi150 ceph-mon[34289]: from='client.24551 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi161:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:42:59.975 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:42:59 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:42:59.975 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:42:59 smithi150 ceph-mon[34289]: osdmap e66: 11 total, 11 up, 11 in 2024-09-21T03:43:00.718 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:43:00 smithi161 ceph-mon[34533]: pgmap v220: 1 pgs: 1 remapped+peering; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail 2024-09-21T03:43:00.718 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:43:00 smithi161 ceph-mon[34533]: osdmap e67: 11 total, 11 up, 11 in 2024-09-21T03:43:00.834 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:43:00 smithi039 ceph-mon[31211]: pgmap v220: 1 pgs: 1 remapped+peering; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail 2024-09-21T03:43:00.834 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:43:00 smithi039 ceph-mon[31211]: osdmap e67: 11 total, 11 up, 11 in 2024-09-21T03:43:00.974 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:43:00 smithi150 ceph-mon[34289]: pgmap v220: 1 pgs: 1 remapped+peering; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail 2024-09-21T03:43:00.975 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:43:00 smithi150 ceph-mon[34289]: osdmap e67: 11 total, 11 up, 11 in 2024-09-21T03:43:01.718 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:43:01 smithi161 ceph-mon[34533]: from='client.? 172.21.15.161:0/3534358040' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "f382f522-62db-46da-997e-2b28f84d1612"} : dispatch 2024-09-21T03:43:01.718 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:43:01 smithi161 ceph-mon[34533]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "f382f522-62db-46da-997e-2b28f84d1612"} : dispatch 2024-09-21T03:43:01.718 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:43:01 smithi161 ceph-mon[34533]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f382f522-62db-46da-997e-2b28f84d1612"}]': finished 2024-09-21T03:43:01.718 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:43:01 smithi161 ceph-mon[34533]: osdmap e68: 12 total, 11 up, 12 in 2024-09-21T03:43:01.718 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:43:01 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2024-09-21T03:43:01.718 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:43:01 smithi161 ceph-mon[34533]: from='client.? 172.21.15.161:0/595597676' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T03:43:01.834 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:43:01 smithi039 ceph-mon[31211]: from='client.? 172.21.15.161:0/3534358040' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "f382f522-62db-46da-997e-2b28f84d1612"} : dispatch 2024-09-21T03:43:01.834 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:43:01 smithi039 ceph-mon[31211]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "f382f522-62db-46da-997e-2b28f84d1612"} : dispatch 2024-09-21T03:43:01.834 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:43:01 smithi039 ceph-mon[31211]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f382f522-62db-46da-997e-2b28f84d1612"}]': finished 2024-09-21T03:43:01.834 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:43:01 smithi039 ceph-mon[31211]: osdmap e68: 12 total, 11 up, 12 in 2024-09-21T03:43:01.834 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:43:01 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2024-09-21T03:43:01.834 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:43:01 smithi039 ceph-mon[31211]: from='client.? 172.21.15.161:0/595597676' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T03:43:01.975 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:43:01 smithi150 ceph-mon[34289]: from='client.? 172.21.15.161:0/3534358040' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "f382f522-62db-46da-997e-2b28f84d1612"} : dispatch 2024-09-21T03:43:01.975 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:43:01 smithi150 ceph-mon[34289]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "f382f522-62db-46da-997e-2b28f84d1612"} : dispatch 2024-09-21T03:43:01.975 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:43:01 smithi150 ceph-mon[34289]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f382f522-62db-46da-997e-2b28f84d1612"}]': finished 2024-09-21T03:43:01.975 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:43:01 smithi150 ceph-mon[34289]: osdmap e68: 12 total, 11 up, 12 in 2024-09-21T03:43:01.975 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:43:01 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2024-09-21T03:43:01.975 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:43:01 smithi150 ceph-mon[34289]: from='client.? 172.21.15.161:0/595597676' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T03:43:03.217 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:43:02 smithi161 ceph-mon[34533]: pgmap v224: 1 pgs: 1 remapped+peering; 577 KiB data, 324 MiB used, 983 GiB / 983 GiB avail 2024-09-21T03:43:03.224 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:43:02 smithi150 ceph-mon[34289]: pgmap v224: 1 pgs: 1 remapped+peering; 577 KiB data, 324 MiB used, 983 GiB / 983 GiB avail 2024-09-21T03:43:03.333 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:43:02 smithi039 ceph-mon[31211]: pgmap v224: 1 pgs: 1 remapped+peering; 577 KiB data, 324 MiB used, 983 GiB / 983 GiB avail 2024-09-21T03:43:05.217 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:43:04 smithi161 ceph-mon[34533]: pgmap v225: 1 pgs: 1 remapped+peering; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail 2024-09-21T03:43:05.224 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:43:04 smithi150 ceph-mon[34289]: pgmap v225: 1 pgs: 1 remapped+peering; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail 2024-09-21T03:43:05.334 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:43:04 smithi039 ceph-mon[31211]: pgmap v225: 1 pgs: 1 remapped+peering; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail 2024-09-21T03:43:07.185 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:43:06 smithi161 ceph-mon[34533]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 324 MiB used, 983 GiB / 983 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-21T03:43:07.224 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:43:06 smithi150 ceph-mon[34289]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 324 MiB used, 983 GiB / 983 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-21T03:43:07.333 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:43:06 smithi039 ceph-mon[31211]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 324 MiB used, 983 GiB / 983 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-21T03:43:09.218 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:43:08 smithi161 ceph-mon[34533]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 324 MiB used, 983 GiB / 983 GiB avail; 73 KiB/s, 0 objects/s recovering 2024-09-21T03:43:09.224 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:43:08 smithi150 ceph-mon[34289]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 324 MiB used, 983 GiB / 983 GiB avail; 73 KiB/s, 0 objects/s recovering 2024-09-21T03:43:09.333 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:43:08 smithi039 ceph-mon[31211]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 324 MiB used, 983 GiB / 983 GiB avail; 73 KiB/s, 0 objects/s recovering 2024-09-21T03:43:10.076 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:43:09 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.11"} : dispatch 2024-09-21T03:43:10.076 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:43:09 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:43:10.224 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:43:09 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.11"} : dispatch 2024-09-21T03:43:10.224 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:43:09 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:43:10.234 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:43:09 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.11"} : dispatch 2024-09-21T03:43:10.234 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:43:09 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:43:11.217 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:43:10 smithi161 ceph-mon[34533]: Deploying daemon osd.11 on smithi161 2024-09-21T03:43:11.218 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:43:10 smithi161 ceph-mon[34533]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 324 MiB used, 983 GiB / 983 GiB avail; 65 KiB/s, 0 objects/s recovering 2024-09-21T03:43:11.224 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:43:10 smithi150 ceph-mon[34289]: Deploying daemon osd.11 on smithi161 2024-09-21T03:43:11.224 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:43:10 smithi150 ceph-mon[34289]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 324 MiB used, 983 GiB / 983 GiB avail; 65 KiB/s, 0 objects/s recovering 2024-09-21T03:43:11.334 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:43:10 smithi039 ceph-mon[31211]: Deploying daemon osd.11 on smithi161 2024-09-21T03:43:11.334 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:43:10 smithi039 ceph-mon[31211]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 324 MiB used, 983 GiB / 983 GiB avail; 65 KiB/s, 0 objects/s recovering 2024-09-21T03:43:13.192 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:43:12 smithi161 ceph-mon[34533]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 324 MiB used, 983 GiB / 983 GiB avail; 55 KiB/s, 0 objects/s recovering 2024-09-21T03:43:13.224 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:43:12 smithi150 ceph-mon[34289]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 324 MiB used, 983 GiB / 983 GiB avail; 55 KiB/s, 0 objects/s recovering 2024-09-21T03:43:13.333 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:43:12 smithi039 ceph-mon[31211]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 324 MiB used, 983 GiB / 983 GiB avail; 55 KiB/s, 0 objects/s recovering 2024-09-21T03:43:15.120 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:43:14 smithi161 ceph-mon[34533]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 324 MiB used, 983 GiB / 983 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-21T03:43:15.224 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:43:14 smithi150 ceph-mon[34289]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 324 MiB used, 983 GiB / 983 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-21T03:43:15.333 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:43:14 smithi039 ceph-mon[31211]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 324 MiB used, 983 GiB / 983 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-21T03:43:16.111 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:43:15 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:43:16.111 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:43:15 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:43:16.111 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:43:15 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:43:16.225 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:43:15 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:43:16.225 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:43:15 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:43:16.225 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:43:15 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:43:16.333 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:43:15 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:43:16.334 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:43:15 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:43:16.334 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:43:15 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:43:17.217 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:43:16 smithi161 ceph-mon[34533]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 324 MiB used, 983 GiB / 983 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-21T03:43:17.224 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:43:16 smithi150 ceph-mon[34289]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 324 MiB used, 983 GiB / 983 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-21T03:43:17.333 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:43:16 smithi039 ceph-mon[31211]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 324 MiB used, 983 GiB / 983 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-21T03:43:18.939 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-21T03:43:18.937+0000 7f27d77fe640 1 -- 172.21.15.161:0/217215358 <== mgr.14223 v2:172.21.15.39:6800/1696381982 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7f27e0063800 con 0x7f27cc0b6560 2024-09-21T03:43:18.940 INFO:teuthology.orchestra.run.smithi161.stdout:Created osd(s) 11 on host 'smithi161' 2024-09-21T03:43:18.942 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-21T03:43:18.940+0000 7f27e4afc640 1 -- 172.21.15.161:0/217215358 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f27cc0b6560 msgr2=0x7f27cc0b8a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:18.942 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-21T03:43:18.940+0000 7f27e4afc640 1 --2- 172.21.15.161:0/217215358 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f27cc0b6560 0x7f27cc0b8a70 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f27e01071c0 tx=0x7f27c800a2d0 comp rx=0 tx=0).stop 2024-09-21T03:43:18.942 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-21T03:43:18.940+0000 7f27e4afc640 1 -- 172.21.15.161:0/217215358 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f27e0103c30 msgr2=0x7f27e0075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:18.942 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-21T03:43:18.940+0000 7f27e4afc640 1 --2- 172.21.15.161:0/217215358 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f27e0103c30 0x7f27e0075710 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f27d800ea80 tx=0x7f27d800ef40 comp rx=0 tx=0).stop 2024-09-21T03:43:18.942 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-21T03:43:18.940+0000 7f27e4afc640 1 -- 172.21.15.161:0/217215358 shutdown_connections 2024-09-21T03:43:18.943 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-21T03:43:18.940+0000 7f27e4afc640 1 --2- 172.21.15.161:0/217215358 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f27cc0b6560 0x7f27cc0b8a70 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:18.943 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-21T03:43:18.940+0000 7f27e4afc640 1 --2- 172.21.15.161:0/217215358 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f27e0109ec0 0x7f27e0076190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:18.943 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-21T03:43:18.940+0000 7f27e4afc640 1 --2- 172.21.15.161:0/217215358 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f27e0109570 0x7f27e0075c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:18.943 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-21T03:43:18.940+0000 7f27e4afc640 1 --2- 172.21.15.161:0/217215358 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f27e0103c30 0x7f27e0075710 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:18.943 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-21T03:43:18.940+0000 7f27e4afc640 1 -- 172.21.15.161:0/217215358 >> 172.21.15.161:0/217215358 conn(0x7f27e007b8d0 msgr2=0x7f27e0105500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:43:18.943 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-21T03:43:18.940+0000 7f27e4afc640 1 -- 172.21.15.161:0/217215358 shutdown_connections 2024-09-21T03:43:18.943 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-21T03:43:18.940+0000 7f27e4afc640 1 -- 172.21.15.161:0/217215358 wait complete. 2024-09-21T03:43:19.467 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:43:19 smithi161 ceph-mon[34533]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 324 MiB used, 983 GiB / 983 GiB avail 2024-09-21T03:43:19.468 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:43:19 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:43:19.468 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:43:19 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:43:19.468 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:43:19 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:43:19.468 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:43:19 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:43:19.468 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:43:19 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:43:19.468 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:43:19 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:43:19.468 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:43:19 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:43:19.468 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:43:19 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:43:19.474 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:43:19 smithi150 ceph-mon[34289]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 324 MiB used, 983 GiB / 983 GiB avail 2024-09-21T03:43:19.474 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:43:19 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:43:19.475 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:43:19 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:43:19.475 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:43:19 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:43:19.475 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:43:19 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:43:19.475 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:43:19 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:43:19.475 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:43:19 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:43:19.475 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:43:19 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:43:19.475 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:43:19 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:43:19.584 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:43:19 smithi039 ceph-mon[31211]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 324 MiB used, 983 GiB / 983 GiB avail 2024-09-21T03:43:19.584 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:43:19 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:43:19.584 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:43:19 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:43:19.584 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:43:19 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:43:19.584 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:43:19 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:43:19.584 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:43:19 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:43:19.584 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:43:19 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:43:19.584 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:43:19 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:43:19.584 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:43:19 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:43:19.630 DEBUG:teuthology.orchestra.run.smithi161:osd.11> sudo journalctl -f -n 0 -u ceph-51116d5c-77ca-11ef-baf5-efdc52797490@osd.11.service 2024-09-21T03:43:19.673 INFO:tasks.cephadm:Waiting for 12 OSDs to come up... 2024-09-21T03:43:19.673 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 51116d5c-77ca-11ef-baf5-efdc52797490 -- ceph osd stat -f json 2024-09-21T03:43:19.943 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/51116d5c-77ca-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:43:20.511 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:20.507+0000 7f8a5d2f4640 1 -- 172.21.15.39:0/215158653 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8a58102230 msgr2=0x7f8a58102640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:20.511 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:20.507+0000 7f8a5d2f4640 1 --2- 172.21.15.39:0/215158653 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8a58102230 0x7f8a58102640 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f8a48009900 tx=0x7f8a4802a410 comp rx=0 tx=0).stop 2024-09-21T03:43:20.511 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:20.507+0000 7f8a5d2f4640 1 -- 172.21.15.39:0/215158653 shutdown_connections 2024-09-21T03:43:20.511 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:20.507+0000 7f8a5d2f4640 1 --2- 172.21.15.39:0/215158653 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8a581084c0 0x7f8a5810ecf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:20.511 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:20.507+0000 7f8a5d2f4640 1 --2- 172.21.15.39:0/215158653 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8a58107b70 0x7f8a58107f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:20.511 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:20.507+0000 7f8a5d2f4640 1 --2- 172.21.15.39:0/215158653 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8a58102230 0x7f8a58102640 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:20.511 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:20.507+0000 7f8a5d2f4640 1 -- 172.21.15.39:0/215158653 >> 172.21.15.39:0/215158653 conn(0x7f8a58079e40 msgr2=0x7f8a5807a250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:43:20.511 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:20.507+0000 7f8a5d2f4640 1 -- 172.21.15.39:0/215158653 shutdown_connections 2024-09-21T03:43:20.511 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:20.507+0000 7f8a5d2f4640 1 -- 172.21.15.39:0/215158653 wait complete. 2024-09-21T03:43:20.511 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:20.507+0000 7f8a5d2f4640 1 Processor -- start 2024-09-21T03:43:20.511 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:20.508+0000 7f8a5d2f4640 1 -- start start 2024-09-21T03:43:20.512 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:20.508+0000 7f8a5d2f4640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8a58102230 0x7f8a580fee70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:20.512 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:20.508+0000 7f8a5d2f4640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8a58107b70 0x7f8a580ff3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:20.512 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:20.508+0000 7f8a5d2f4640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8a581084c0 0x7f8a580ff8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:20.512 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:20.508+0000 7f8a5d2f4640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f8a581129f0 con 0x7f8a581084c0 2024-09-21T03:43:20.512 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:20.508+0000 7f8a5d2f4640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f8a58112870 con 0x7f8a58102230 2024-09-21T03:43:20.512 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:20.508+0000 7f8a5d2f4640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f8a58112b70 con 0x7f8a58107b70 2024-09-21T03:43:20.512 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:20.508+0000 7f8a577fe640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8a581084c0 0x7f8a580ff8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:20.512 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:20.508+0000 7f8a567fc640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8a58107b70 0x7f8a580ff3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:20.512 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:20.508+0000 7f8a577fe640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8a581084c0 0x7f8a580ff8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:37848/0 (socket says 172.21.15.39:37848) 2024-09-21T03:43:20.512 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:20.508+0000 7f8a567fc640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8a58107b70 0x7f8a580ff3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.39:33222/0 (socket says 172.21.15.39:33222) 2024-09-21T03:43:20.512 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:20.508+0000 7f8a577fe640 1 -- 172.21.15.39:0/2518192962 learned_addr learned my addr 172.21.15.39:0/2518192962 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:43:20.512 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:20.508+0000 7f8a56ffd640 1 --2- 172.21.15.39:0/2518192962 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8a58102230 0x7f8a580fee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:20.512 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:20.508+0000 7f8a577fe640 1 -- 172.21.15.39:0/2518192962 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8a58107b70 msgr2=0x7f8a580ff3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:20.512 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:20.508+0000 7f8a577fe640 1 --2- 172.21.15.39:0/2518192962 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8a58107b70 0x7f8a580ff3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:20.513 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:20.508+0000 7f8a577fe640 1 -- 172.21.15.39:0/2518192962 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8a58102230 msgr2=0x7f8a580fee70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:20.513 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:20.508+0000 7f8a577fe640 1 --2- 172.21.15.39:0/2518192962 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8a58102230 0x7f8a580fee70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:20.513 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:20.509+0000 7f8a577fe640 1 -- 172.21.15.39:0/2518192962 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8a581a8830 con 0x7f8a581084c0 2024-09-21T03:43:20.513 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:20.509+0000 7f8a577fe640 1 --2- 172.21.15.39:0/2518192962 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8a581084c0 0x7f8a580ff8f0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f8a50009fd0 tx=0x7f8a5000b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:43:20.513 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:20.509+0000 7f8a3ffff640 1 -- 172.21.15.39:0/2518192962 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a50017070 con 0x7f8a581084c0 2024-09-21T03:43:20.513 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:20.509+0000 7f8a5d2f4640 1 -- 172.21.15.39:0/2518192962 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8a581a8b20 con 0x7f8a581084c0 2024-09-21T03:43:20.513 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:20.509+0000 7f8a3ffff640 1 -- 172.21.15.39:0/2518192962 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8a50003e20 con 0x7f8a581084c0 2024-09-21T03:43:20.513 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:20.509+0000 7f8a3ffff640 1 -- 172.21.15.39:0/2518192962 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a50010460 con 0x7f8a581084c0 2024-09-21T03:43:20.513 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:20.509+0000 7f8a5d2f4640 1 -- 172.21.15.39:0/2518192962 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8a581a9010 con 0x7f8a581084c0 2024-09-21T03:43:20.514 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:20.510+0000 7f8a5d2f4640 1 -- 172.21.15.39:0/2518192962 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8a58102640 con 0x7f8a581084c0 2024-09-21T03:43:20.514 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:20.512+0000 7f8a3ffff640 1 -- 172.21.15.39:0/2518192962 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 21) ==== 154529+0+0 (secure 0 0 0) 0x7f8a50003970 con 0x7f8a581084c0 2024-09-21T03:43:20.514 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:20.513+0000 7f8a3ffff640 1 --2- 172.21.15.39:0/2518192962 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f8a400b6560 0x7f8a400b8a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:20.514 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:20.513+0000 7f8a56ffd640 1 --2- 172.21.15.39:0/2518192962 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f8a400b6560 0x7f8a400b8a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:20.514 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:20.513+0000 7f8a3ffff640 1 -- 172.21.15.39:0/2518192962 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(68..68 src has 1..68) ==== 7024+0+0 (secure 0 0 0) 0x7f8a500e6a90 con 0x7f8a581084c0 2024-09-21T03:43:20.515 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:20.513+0000 7f8a56ffd640 1 --2- 172.21.15.39:0/2518192962 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f8a400b6560 0x7f8a400b8a70 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f8a48000c00 tx=0x7f8a48031400 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:43:20.517 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:20.516+0000 7f8a3ffff640 1 -- 172.21.15.39:0/2518192962 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f8a500b1c10 con 0x7f8a581084c0 2024-09-21T03:43:20.709 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:20.707+0000 7f8a5d2f4640 1 -- 172.21.15.39:0/2518192962 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) -- 0x7f8a581a93d0 con 0x7f8a581084c0 2024-09-21T03:43:20.709 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:20.708+0000 7f8a3ffff640 1 -- 172.21.15.39:0/2518192962 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v68) ==== 74+0+133 (secure 0 0 0) 0x7f8a500b6b50 con 0x7f8a581084c0 2024-09-21T03:43:20.709 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T03:43:20.712 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:20.710+0000 7f8a5d2f4640 1 -- 172.21.15.39:0/2518192962 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f8a400b6560 msgr2=0x7f8a400b8a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:20.712 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:20.710+0000 7f8a5d2f4640 1 --2- 172.21.15.39:0/2518192962 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f8a400b6560 0x7f8a400b8a70 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f8a48000c00 tx=0x7f8a48031400 comp rx=0 tx=0).stop 2024-09-21T03:43:20.712 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:20.710+0000 7f8a5d2f4640 1 -- 172.21.15.39:0/2518192962 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8a581084c0 msgr2=0x7f8a580ff8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:20.712 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:20.710+0000 7f8a5d2f4640 1 --2- 172.21.15.39:0/2518192962 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8a581084c0 0x7f8a580ff8f0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f8a50009fd0 tx=0x7f8a5000b260 comp rx=0 tx=0).stop 2024-09-21T03:43:20.712 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:20.710+0000 7f8a5d2f4640 1 -- 172.21.15.39:0/2518192962 shutdown_connections 2024-09-21T03:43:20.712 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:20.710+0000 7f8a5d2f4640 1 --2- 172.21.15.39:0/2518192962 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f8a400b6560 0x7f8a400b8a70 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:20.712 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:20.710+0000 7f8a5d2f4640 1 --2- 172.21.15.39:0/2518192962 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8a581084c0 0x7f8a580ff8f0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:20.712 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:20.710+0000 7f8a5d2f4640 1 --2- 172.21.15.39:0/2518192962 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8a58107b70 0x7f8a580ff3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:20.712 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:20.710+0000 7f8a5d2f4640 1 --2- 172.21.15.39:0/2518192962 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8a58102230 0x7f8a580fee70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:20.713 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:20.710+0000 7f8a5d2f4640 1 -- 172.21.15.39:0/2518192962 >> 172.21.15.39:0/2518192962 conn(0x7f8a58079e40 msgr2=0x7f8a581011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:43:20.713 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:20.710+0000 7f8a5d2f4640 1 -- 172.21.15.39:0/2518192962 shutdown_connections 2024-09-21T03:43:20.713 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:20.710+0000 7f8a5d2f4640 1 -- 172.21.15.39:0/2518192962 wait complete. 2024-09-21T03:43:21.416 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":68,"num_osds":12,"num_up_osds":11,"osd_up_since":1726890178,"num_in_osds":12,"osd_in_since":1726890180,"num_remapped_pgs":0} 2024-09-21T03:43:21.467 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:43:21 smithi161 ceph-mon[34533]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 324 MiB used, 983 GiB / 983 GiB avail 2024-09-21T03:43:21.468 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:43:21 smithi161 ceph-mon[34533]: from='client.? 172.21.15.39:0/2518192962' entity='client.admin' cmd={"prefix": "osd stat", "format": "json"} : dispatch 2024-09-21T03:43:21.468 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 21 03:43:21 smithi161 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-11[58829]: 2024-09-21T03:43:21.226+0000 7f53959c7640 -1 osd.11 0 log_to_monitors true 2024-09-21T03:43:21.475 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:43:21 smithi150 ceph-mon[34289]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 324 MiB used, 983 GiB / 983 GiB avail 2024-09-21T03:43:21.475 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:43:21 smithi150 ceph-mon[34289]: from='client.? 172.21.15.39:0/2518192962' entity='client.admin' cmd={"prefix": "osd stat", "format": "json"} : dispatch 2024-09-21T03:43:21.583 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:43:21 smithi039 ceph-mon[31211]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 324 MiB used, 983 GiB / 983 GiB avail 2024-09-21T03:43:21.583 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:43:21 smithi039 ceph-mon[31211]: from='client.? 172.21.15.39:0/2518192962' entity='client.admin' cmd={"prefix": "osd stat", "format": "json"} : dispatch 2024-09-21T03:43:22.417 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 51116d5c-77ca-11ef-baf5-efdc52797490 -- ceph osd stat -f json 2024-09-21T03:43:22.467 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:43:22 smithi161 ceph-mon[34533]: from='osd.11 [v2:172.21.15.161:6824/2178709707,v1:172.21.15.161:6825/2178709707]' entity='osd.11' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]} : dispatch 2024-09-21T03:43:22.468 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:43:22 smithi161 ceph-mon[34533]: from='osd.11 ' entity='osd.11' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]} : dispatch 2024-09-21T03:43:22.474 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:43:22 smithi150 ceph-mon[34289]: from='osd.11 [v2:172.21.15.161:6824/2178709707,v1:172.21.15.161:6825/2178709707]' entity='osd.11' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]} : dispatch 2024-09-21T03:43:22.474 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:43:22 smithi150 ceph-mon[34289]: from='osd.11 ' entity='osd.11' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]} : dispatch 2024-09-21T03:43:22.583 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:43:22 smithi039 ceph-mon[31211]: from='osd.11 [v2:172.21.15.161:6824/2178709707,v1:172.21.15.161:6825/2178709707]' entity='osd.11' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]} : dispatch 2024-09-21T03:43:22.584 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:43:22 smithi039 ceph-mon[31211]: from='osd.11 ' entity='osd.11' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]} : dispatch 2024-09-21T03:43:22.694 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/51116d5c-77ca-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:43:23.226 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:23.224+0000 7f273c4e1640 1 -- 172.21.15.39:0/3201745615 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f27341083c0 msgr2=0x7f27341087d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:23.227 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:23.224+0000 7f273c4e1640 1 --2- 172.21.15.39:0/3201745615 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f27341083c0 0x7f27341087d0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f2724009900 tx=0x7f272402a410 comp rx=0 tx=0).stop 2024-09-21T03:43:23.227 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:23.224+0000 7f273c4e1640 1 -- 172.21.15.39:0/3201745615 shutdown_connections 2024-09-21T03:43:23.227 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:23.224+0000 7f273c4e1640 1 --2- 172.21.15.39:0/3201745615 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2734108d10 0x7f273410f540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:23.227 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:23.224+0000 7f273c4e1640 1 --2- 172.21.15.39:0/3201745615 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f27341083c0 0x7f27341087d0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:23.227 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:23.224+0000 7f273c4e1640 1 --2- 172.21.15.39:0/3201745615 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2734102a80 0x7f2734102e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:23.227 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:23.224+0000 7f273c4e1640 1 -- 172.21.15.39:0/3201745615 >> 172.21.15.39:0/3201745615 conn(0x7f2734079740 msgr2=0x7f2734079b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:43:23.227 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:23.224+0000 7f273c4e1640 1 -- 172.21.15.39:0/3201745615 shutdown_connections 2024-09-21T03:43:23.227 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:23.224+0000 7f273c4e1640 1 -- 172.21.15.39:0/3201745615 wait complete. 2024-09-21T03:43:23.227 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:23.225+0000 7f273c4e1640 1 Processor -- start 2024-09-21T03:43:23.228 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:23.225+0000 7f273c4e1640 1 -- start start 2024-09-21T03:43:23.229 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:23.225+0000 7f273c4e1640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2734102a80 0x7f27340ff690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:23.229 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:23.225+0000 7f273c4e1640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f27341083c0 0x7f27340ffbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:23.229 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:23.225+0000 7f273c4e1640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2734108d10 0x7f2734100110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:23.229 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:23.225+0000 7f273c4e1640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f27341129d0 con 0x7f2734108d10 2024-09-21T03:43:23.230 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:23.225+0000 7f273c4e1640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f2734112850 con 0x7f27341083c0 2024-09-21T03:43:23.230 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:23.225+0000 7f273c4e1640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f2734112b50 con 0x7f2734102a80 2024-09-21T03:43:23.230 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:23.225+0000 7f273aa7e640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2734108d10 0x7f2734100110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:23.230 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:23.225+0000 7f273aa7e640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2734108d10 0x7f2734100110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:37862/0 (socket says 172.21.15.39:37862) 2024-09-21T03:43:23.230 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:23.225+0000 7f273aa7e640 1 -- 172.21.15.39:0/4217158029 learned_addr learned my addr 172.21.15.39:0/4217158029 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:43:23.230 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:23.225+0000 7f273a27d640 1 --2- 172.21.15.39:0/4217158029 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2734102a80 0x7f27340ff690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:23.230 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:23.226+0000 7f2739a7c640 1 --2- 172.21.15.39:0/4217158029 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f27341083c0 0x7f27340ffbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:23.230 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:23.226+0000 7f273aa7e640 1 -- 172.21.15.39:0/4217158029 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2734102a80 msgr2=0x7f27340ff690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:23.230 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:23.226+0000 7f273aa7e640 1 --2- 172.21.15.39:0/4217158029 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2734102a80 0x7f27340ff690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:23.230 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:23.226+0000 7f273aa7e640 1 -- 172.21.15.39:0/4217158029 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f27341083c0 msgr2=0x7f27340ffbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:23.230 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:23.226+0000 7f273aa7e640 1 --2- 172.21.15.39:0/4217158029 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f27341083c0 0x7f27340ffbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:23.230 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:23.226+0000 7f273aa7e640 1 -- 172.21.15.39:0/4217158029 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f27341a8750 con 0x7f2734108d10 2024-09-21T03:43:23.231 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:23.226+0000 7f2739a7c640 1 --2- 172.21.15.39:0/4217158029 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f27341083c0 0x7f27340ffbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:43:23.231 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:23.226+0000 7f273aa7e640 1 --2- 172.21.15.39:0/4217158029 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2734108d10 0x7f2734100110 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f272c00eaa0 tx=0x7f272c00ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:43:23.231 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:23.226+0000 7f272b7fe640 1 -- 172.21.15.39:0/4217158029 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f272c00cdb0 con 0x7f2734108d10 2024-09-21T03:43:23.231 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:23.226+0000 7f272b7fe640 1 -- 172.21.15.39:0/4217158029 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f272c005020 con 0x7f2734108d10 2024-09-21T03:43:23.231 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:23.226+0000 7f273c4e1640 1 -- 172.21.15.39:0/4217158029 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f27341a8a40 con 0x7f2734108d10 2024-09-21T03:43:23.231 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:23.226+0000 7f272b7fe640 1 -- 172.21.15.39:0/4217158029 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f272c0116d0 con 0x7f2734108d10 2024-09-21T03:43:23.231 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:23.226+0000 7f273c4e1640 1 -- 172.21.15.39:0/4217158029 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f27341a8f30 con 0x7f2734108d10 2024-09-21T03:43:23.231 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:23.227+0000 7f273c4e1640 1 -- 172.21.15.39:0/4217158029 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2734103d00 con 0x7f2734108d10 2024-09-21T03:43:23.232 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:23.229+0000 7f272b7fe640 1 -- 172.21.15.39:0/4217158029 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 21) ==== 154529+0+0 (secure 0 0 0) 0x7f272c0051d0 con 0x7f2734108d10 2024-09-21T03:43:23.232 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:23.230+0000 7f272b7fe640 1 --2- 172.21.15.39:0/4217158029 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f27200b6560 0x7f27200b8a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:23.232 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:23.230+0000 7f273a27d640 1 --2- 172.21.15.39:0/4217158029 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f27200b6560 0x7f27200b8a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:23.232 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:23.230+0000 7f272b7fe640 1 -- 172.21.15.39:0/4217158029 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(69..69 src has 1..69) ==== 7046+0+0 (secure 0 0 0) 0x7f272c0e7440 con 0x7f2734108d10 2024-09-21T03:43:23.232 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:23.230+0000 7f273a27d640 1 --2- 172.21.15.39:0/4217158029 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f27200b6560 0x7f27200b8a70 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f27341010e0 tx=0x7f273000b390 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:43:23.234 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:23.233+0000 7f272b7fe640 1 -- 172.21.15.39:0/4217158029 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f272c016030 con 0x7f2734108d10 2024-09-21T03:43:23.428 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:23.425+0000 7f273c4e1640 1 -- 172.21.15.39:0/4217158029 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) -- 0x7f2734102e90 con 0x7f2734108d10 2024-09-21T03:43:23.428 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:23.425+0000 7f272b7fe640 1 -- 172.21.15.39:0/4217158029 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v70) ==== 74+0+133 (secure 0 0 0) 0x7f2734102e90 con 0x7f2734108d10 2024-09-21T03:43:23.428 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:43:23 smithi039 ceph-mon[31211]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 324 MiB used, 983 GiB / 983 GiB avail 2024-09-21T03:43:23.428 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:43:23 smithi039 ceph-mon[31211]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-09-21T03:43:23.428 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:43:23 smithi039 ceph-mon[31211]: osdmap e69: 12 total, 11 up, 12 in 2024-09-21T03:43:23.429 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:43:23 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2024-09-21T03:43:23.429 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:43:23 smithi039 ceph-mon[31211]: from='osd.11 [v2:172.21.15.161:6824/2178709707,v1:172.21.15.161:6825/2178709707]' entity='osd.11' cmd={"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi161", "root=default"]} : dispatch 2024-09-21T03:43:23.429 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:43:23 smithi039 ceph-mon[31211]: from='osd.11 ' entity='osd.11' cmd={"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi161", "root=default"]} : dispatch 2024-09-21T03:43:23.429 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:43:23 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:43:23.429 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:43:23 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:43:23.429 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:43:23 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-09-21T03:43:23.429 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:43:23 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-09-21T03:43:23.429 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:43:23 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"} : dispatch 2024-09-21T03:43:23.429 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:43:23 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"} : dispatch 2024-09-21T03:43:23.429 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:43:23 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:43:23.429 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:43:23 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:43:23.429 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:43:23 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:43:23.429 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:43:23 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:43:23.430 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T03:43:23.431 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:23.428+0000 7f273c4e1640 1 -- 172.21.15.39:0/4217158029 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f27200b6560 msgr2=0x7f27200b8a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:23.431 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:23.428+0000 7f273c4e1640 1 --2- 172.21.15.39:0/4217158029 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f27200b6560 0x7f27200b8a70 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f27341010e0 tx=0x7f273000b390 comp rx=0 tx=0).stop 2024-09-21T03:43:23.431 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:23.428+0000 7f273c4e1640 1 -- 172.21.15.39:0/4217158029 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2734108d10 msgr2=0x7f2734100110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:23.431 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:23.428+0000 7f273c4e1640 1 --2- 172.21.15.39:0/4217158029 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2734108d10 0x7f2734100110 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f272c00eaa0 tx=0x7f272c00ef60 comp rx=0 tx=0).stop 2024-09-21T03:43:23.431 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:23.428+0000 7f273c4e1640 1 -- 172.21.15.39:0/4217158029 shutdown_connections 2024-09-21T03:43:23.432 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:23.428+0000 7f273c4e1640 1 --2- 172.21.15.39:0/4217158029 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f27200b6560 0x7f27200b8a70 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:23.432 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:23.428+0000 7f273c4e1640 1 --2- 172.21.15.39:0/4217158029 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2734108d10 0x7f2734100110 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:23.432 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:23.428+0000 7f273c4e1640 1 --2- 172.21.15.39:0/4217158029 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f27341083c0 0x7f27340ffbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:23.432 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:23.428+0000 7f273c4e1640 1 --2- 172.21.15.39:0/4217158029 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2734102a80 0x7f27340ff690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:23.432 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:23.428+0000 7f273c4e1640 1 -- 172.21.15.39:0/4217158029 >> 172.21.15.39:0/4217158029 conn(0x7f2734079740 msgr2=0x7f273406bed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:43:23.432 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:23.428+0000 7f273c4e1640 1 -- 172.21.15.39:0/4217158029 shutdown_connections 2024-09-21T03:43:23.432 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:23.428+0000 7f273c4e1640 1 -- 172.21.15.39:0/4217158029 wait complete. 2024-09-21T03:43:23.718 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 21 03:43:23 smithi161 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-11[58829]: 2024-09-21T03:43:23.241+0000 7f5391949640 -1 osd.11 0 waiting for initial osdmap 2024-09-21T03:43:23.718 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 21 03:43:23 smithi161 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-11[58829]: 2024-09-21T03:43:23.261+0000 7f538d747640 -1 osd.11 70 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-21T03:43:23.718 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:43:23 smithi161 ceph-mon[34533]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 324 MiB used, 983 GiB / 983 GiB avail 2024-09-21T03:43:23.718 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:43:23 smithi161 ceph-mon[34533]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-09-21T03:43:23.718 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:43:23 smithi161 ceph-mon[34533]: osdmap e69: 12 total, 11 up, 12 in 2024-09-21T03:43:23.718 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:43:23 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2024-09-21T03:43:23.718 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:43:23 smithi161 ceph-mon[34533]: from='osd.11 [v2:172.21.15.161:6824/2178709707,v1:172.21.15.161:6825/2178709707]' entity='osd.11' cmd={"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi161", "root=default"]} : dispatch 2024-09-21T03:43:23.719 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:43:23 smithi161 ceph-mon[34533]: from='osd.11 ' entity='osd.11' cmd={"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi161", "root=default"]} : dispatch 2024-09-21T03:43:23.719 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:43:23 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:43:23.719 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:43:23 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:43:23.719 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:43:23 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-09-21T03:43:23.719 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:43:23 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-09-21T03:43:23.719 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:43:23 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"} : dispatch 2024-09-21T03:43:23.719 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:43:23 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"} : dispatch 2024-09-21T03:43:23.719 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:43:23 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:43:23.719 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:43:23 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:43:23.719 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:43:23 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:43:23.719 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:43:23 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:43:23.724 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:43:23 smithi150 ceph-mon[34289]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 324 MiB used, 983 GiB / 983 GiB avail 2024-09-21T03:43:23.725 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:43:23 smithi150 ceph-mon[34289]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-09-21T03:43:23.725 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:43:23 smithi150 ceph-mon[34289]: osdmap e69: 12 total, 11 up, 12 in 2024-09-21T03:43:23.725 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:43:23 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2024-09-21T03:43:23.725 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:43:23 smithi150 ceph-mon[34289]: from='osd.11 [v2:172.21.15.161:6824/2178709707,v1:172.21.15.161:6825/2178709707]' entity='osd.11' cmd={"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi161", "root=default"]} : dispatch 2024-09-21T03:43:23.725 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:43:23 smithi150 ceph-mon[34289]: from='osd.11 ' entity='osd.11' cmd={"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi161", "root=default"]} : dispatch 2024-09-21T03:43:23.725 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:43:23 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:43:23.725 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:43:23 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:43:23.725 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:43:23 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-09-21T03:43:23.725 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:43:23 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-09-21T03:43:23.725 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:43:23 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"} : dispatch 2024-09-21T03:43:23.725 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:43:23 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"} : dispatch 2024-09-21T03:43:23.725 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:43:23 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:43:23.725 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:43:23 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:43:23.726 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:43:23 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:43:23.726 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:43:23 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:43:24.334 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":70,"num_osds":12,"num_up_osds":11,"osd_up_since":1726890178,"num_in_osds":12,"osd_in_since":1726890180,"num_remapped_pgs":0} 2024-09-21T03:43:24.583 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:43:24 smithi039 ceph-mon[31211]: purged_snaps scrub starts 2024-09-21T03:43:24.584 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:43:24 smithi039 ceph-mon[31211]: purged_snaps scrub ok 2024-09-21T03:43:24.584 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:43:24 smithi039 ceph-mon[31211]: Detected new or changed devices on smithi161 2024-09-21T03:43:24.584 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:43:24 smithi039 ceph-mon[31211]: Adjusting osd_memory_target on smithi161 to 4272M 2024-09-21T03:43:24.584 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:43:24 smithi039 ceph-mon[31211]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi161", "root=default"]}]': finished 2024-09-21T03:43:24.584 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:43:24 smithi039 ceph-mon[31211]: osdmap e70: 12 total, 11 up, 12 in 2024-09-21T03:43:24.584 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:43:24 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2024-09-21T03:43:24.584 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:43:24 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2024-09-21T03:43:24.584 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:43:24 smithi039 ceph-mon[31211]: from='client.? 172.21.15.39:0/4217158029' entity='client.admin' cmd={"prefix": "osd stat", "format": "json"} : dispatch 2024-09-21T03:43:24.718 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:43:24 smithi161 ceph-mon[34533]: purged_snaps scrub starts 2024-09-21T03:43:24.718 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:43:24 smithi161 ceph-mon[34533]: purged_snaps scrub ok 2024-09-21T03:43:24.718 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:43:24 smithi161 ceph-mon[34533]: Detected new or changed devices on smithi161 2024-09-21T03:43:24.718 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:43:24 smithi161 ceph-mon[34533]: Adjusting osd_memory_target on smithi161 to 4272M 2024-09-21T03:43:24.718 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:43:24 smithi161 ceph-mon[34533]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi161", "root=default"]}]': finished 2024-09-21T03:43:24.718 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:43:24 smithi161 ceph-mon[34533]: osdmap e70: 12 total, 11 up, 12 in 2024-09-21T03:43:24.718 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:43:24 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2024-09-21T03:43:24.718 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:43:24 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2024-09-21T03:43:24.718 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:43:24 smithi161 ceph-mon[34533]: from='client.? 172.21.15.39:0/4217158029' entity='client.admin' cmd={"prefix": "osd stat", "format": "json"} : dispatch 2024-09-21T03:43:24.724 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:43:24 smithi150 ceph-mon[34289]: purged_snaps scrub starts 2024-09-21T03:43:24.725 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:43:24 smithi150 ceph-mon[34289]: purged_snaps scrub ok 2024-09-21T03:43:24.725 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:43:24 smithi150 ceph-mon[34289]: Detected new or changed devices on smithi161 2024-09-21T03:43:24.725 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:43:24 smithi150 ceph-mon[34289]: Adjusting osd_memory_target on smithi161 to 4272M 2024-09-21T03:43:24.725 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:43:24 smithi150 ceph-mon[34289]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi161", "root=default"]}]': finished 2024-09-21T03:43:24.725 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:43:24 smithi150 ceph-mon[34289]: osdmap e70: 12 total, 11 up, 12 in 2024-09-21T03:43:24.725 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:43:24 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2024-09-21T03:43:24.725 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:43:24 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2024-09-21T03:43:24.725 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:43:24 smithi150 ceph-mon[34289]: from='client.? 172.21.15.39:0/4217158029' entity='client.admin' cmd={"prefix": "osd stat", "format": "json"} : dispatch 2024-09-21T03:43:25.335 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 51116d5c-77ca-11ef-baf5-efdc52797490 -- ceph osd stat -f json 2024-09-21T03:43:25.584 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:43:25 smithi039 ceph-mon[31211]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 324 MiB used, 983 GiB / 983 GiB avail 2024-09-21T03:43:25.584 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:43:25 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2024-09-21T03:43:25.584 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:43:25 smithi039 ceph-mon[31211]: osd.11 [v2:172.21.15.161:6824/2178709707,v1:172.21.15.161:6825/2178709707] boot 2024-09-21T03:43:25.584 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:43:25 smithi039 ceph-mon[31211]: osdmap e71: 12 total, 12 up, 12 in 2024-09-21T03:43:25.584 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:43:25 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2024-09-21T03:43:25.604 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/51116d5c-77ca-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:43:25.717 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:43:25 smithi161 ceph-mon[34533]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 324 MiB used, 983 GiB / 983 GiB avail 2024-09-21T03:43:25.718 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:43:25 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2024-09-21T03:43:25.718 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:43:25 smithi161 ceph-mon[34533]: osd.11 [v2:172.21.15.161:6824/2178709707,v1:172.21.15.161:6825/2178709707] boot 2024-09-21T03:43:25.718 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:43:25 smithi161 ceph-mon[34533]: osdmap e71: 12 total, 12 up, 12 in 2024-09-21T03:43:25.718 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:43:25 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2024-09-21T03:43:25.724 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:43:25 smithi150 ceph-mon[34289]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 324 MiB used, 983 GiB / 983 GiB avail 2024-09-21T03:43:25.724 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:43:25 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2024-09-21T03:43:25.724 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:43:25 smithi150 ceph-mon[34289]: osd.11 [v2:172.21.15.161:6824/2178709707,v1:172.21.15.161:6825/2178709707] boot 2024-09-21T03:43:25.724 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:43:25 smithi150 ceph-mon[34289]: osdmap e71: 12 total, 12 up, 12 in 2024-09-21T03:43:25.725 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:43:25 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2024-09-21T03:43:26.081 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:26.070+0000 7f2b40c8b640 1 -- 172.21.15.39:0/1592568378 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2b3c109df0 msgr2=0x7f2b3c110620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:26.081 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:26.070+0000 7f2b40c8b640 1 --2- 172.21.15.39:0/1592568378 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2b3c109df0 0x7f2b3c110620 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f2b34009ca0 tx=0x7f2b3402a8c0 comp rx=0 tx=0).stop 2024-09-21T03:43:26.081 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:26.070+0000 7f2b40c8b640 1 -- 172.21.15.39:0/1592568378 shutdown_connections 2024-09-21T03:43:26.081 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:26.070+0000 7f2b40c8b640 1 --2- 172.21.15.39:0/1592568378 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2b3c109df0 0x7f2b3c110620 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:26.081 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:26.070+0000 7f2b40c8b640 1 --2- 172.21.15.39:0/1592568378 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2b3c1094a0 0x7f2b3c1098b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:26.081 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:26.070+0000 7f2b40c8b640 1 --2- 172.21.15.39:0/1592568378 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2b3c103b60 0x7f2b3c103f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:26.081 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:26.070+0000 7f2b40c8b640 1 -- 172.21.15.39:0/1592568378 >> 172.21.15.39:0/1592568378 conn(0x7f2b3c07b850 msgr2=0x7f2b3c07bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:43:26.082 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:26.070+0000 7f2b40c8b640 1 -- 172.21.15.39:0/1592568378 shutdown_connections 2024-09-21T03:43:26.082 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:26.070+0000 7f2b40c8b640 1 -- 172.21.15.39:0/1592568378 wait complete. 2024-09-21T03:43:26.082 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:26.071+0000 7f2b40c8b640 1 Processor -- start 2024-09-21T03:43:26.082 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:26.071+0000 7f2b40c8b640 1 -- start start 2024-09-21T03:43:26.082 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:26.071+0000 7f2b40c8b640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2b3c103b60 0x7f2b3c19a400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:26.082 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:26.071+0000 7f2b40c8b640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2b3c1094a0 0x7f2b3c19a9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:26.082 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:26.071+0000 7f2b40c8b640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2b3c109df0 0x7f2b3c19fc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:26.082 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:26.071+0000 7f2b40c8b640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f2b3c113ab0 con 0x7f2b3c1094a0 2024-09-21T03:43:26.082 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:26.071+0000 7f2b40c8b640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f2b3c113930 con 0x7f2b3c103b60 2024-09-21T03:43:26.082 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:26.071+0000 7f2b40c8b640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f2b3c113c30 con 0x7f2b3c109df0 2024-09-21T03:43:26.082 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:26.071+0000 7f2b39d9b640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2b3c1094a0 0x7f2b3c19a9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:26.082 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:26.071+0000 7f2b39d9b640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2b3c1094a0 0x7f2b3c19a9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:37872/0 (socket says 172.21.15.39:37872) 2024-09-21T03:43:26.082 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:26.071+0000 7f2b39d9b640 1 -- 172.21.15.39:0/4167054222 learned_addr learned my addr 172.21.15.39:0/4167054222 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:43:26.083 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:26.071+0000 7f2b3ad9d640 1 --2- 172.21.15.39:0/4167054222 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2b3c109df0 0x7f2b3c19fc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:26.083 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:26.072+0000 7f2b3a59c640 1 --2- 172.21.15.39:0/4167054222 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2b3c103b60 0x7f2b3c19a400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:26.083 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:26.072+0000 7f2b39d9b640 1 -- 172.21.15.39:0/4167054222 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2b3c109df0 msgr2=0x7f2b3c19fc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:26.083 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:26.072+0000 7f2b39d9b640 1 --2- 172.21.15.39:0/4167054222 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2b3c109df0 0x7f2b3c19fc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:26.083 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:26.072+0000 7f2b39d9b640 1 -- 172.21.15.39:0/4167054222 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2b3c103b60 msgr2=0x7f2b3c19a400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:26.083 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:26.072+0000 7f2b39d9b640 1 --2- 172.21.15.39:0/4167054222 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2b3c103b60 0x7f2b3c19a400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:26.083 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:26.072+0000 7f2b39d9b640 1 -- 172.21.15.39:0/4167054222 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2b3c1a02b0 con 0x7f2b3c1094a0 2024-09-21T03:43:26.083 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:26.072+0000 7f2b3ad9d640 1 --2- 172.21.15.39:0/4167054222 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2b3c109df0 0x7f2b3c19fc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:43:26.083 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:26.072+0000 7f2b39d9b640 1 --2- 172.21.15.39:0/4167054222 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2b3c1094a0 0x7f2b3c19a9a0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f2b3000c970 tx=0x7f2b3000ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:43:26.083 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:26.072+0000 7f2b2b7fe640 1 -- 172.21.15.39:0/4167054222 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b30007bb0 con 0x7f2b3c1094a0 2024-09-21T03:43:26.083 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:26.072+0000 7f2b40c8b640 1 -- 172.21.15.39:0/4167054222 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2b3c1a05a0 con 0x7f2b3c1094a0 2024-09-21T03:43:26.083 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:26.072+0000 7f2b2b7fe640 1 -- 172.21.15.39:0/4167054222 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2b30007d50 con 0x7f2b3c1094a0 2024-09-21T03:43:26.083 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:26.072+0000 7f2b2b7fe640 1 -- 172.21.15.39:0/4167054222 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b30004e10 con 0x7f2b3c1094a0 2024-09-21T03:43:26.083 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:26.072+0000 7f2b40c8b640 1 -- 172.21.15.39:0/4167054222 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2b3c1a6dc0 con 0x7f2b3c1094a0 2024-09-21T03:43:26.083 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:26.073+0000 7f2b40c8b640 1 -- 172.21.15.39:0/4167054222 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2b3c103f70 con 0x7f2b3c1094a0 2024-09-21T03:43:26.084 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:26.075+0000 7f2b2b7fe640 1 -- 172.21.15.39:0/4167054222 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 21) ==== 154529+0+0 (secure 0 0 0) 0x7f2b30004fb0 con 0x7f2b3c1094a0 2024-09-21T03:43:26.084 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:26.076+0000 7f2b2b7fe640 1 --2- 172.21.15.39:0/4167054222 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f2b240b6590 0x7f2b240b8aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:26.084 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:26.076+0000 7f2b3a59c640 1 --2- 172.21.15.39:0/4167054222 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f2b240b6590 0x7f2b240b8aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:26.084 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:26.076+0000 7f2b2b7fe640 1 -- 172.21.15.39:0/4167054222 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 7342+0+0 (secure 0 0 0) 0x7f2b300e71a0 con 0x7f2b3c1094a0 2024-09-21T03:43:26.084 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:26.076+0000 7f2b3a59c640 1 --2- 172.21.15.39:0/4167054222 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f2b240b6590 0x7f2b240b8aa0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f2b2c005e10 tx=0x7f2b2c005da0 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:43:26.084 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:26.079+0000 7f2b2b7fe640 1 -- 172.21.15.39:0/4167054222 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f2b30014030 con 0x7f2b3c1094a0 2024-09-21T03:43:26.269 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:26.266+0000 7f2b40c8b640 1 -- 172.21.15.39:0/4167054222 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) -- 0x7f2b3c10b650 con 0x7f2b3c1094a0 2024-09-21T03:43:26.272 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:26.270+0000 7f2b2b7fe640 1 -- 172.21.15.39:0/4167054222 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v72) ==== 74+0+133 (secure 0 0 0) 0x7f2b300ade00 con 0x7f2b3c1094a0 2024-09-21T03:43:26.272 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T03:43:26.275 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:26.272+0000 7f2b40c8b640 1 -- 172.21.15.39:0/4167054222 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f2b240b6590 msgr2=0x7f2b240b8aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:26.275 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:26.272+0000 7f2b40c8b640 1 --2- 172.21.15.39:0/4167054222 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f2b240b6590 0x7f2b240b8aa0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f2b2c005e10 tx=0x7f2b2c005da0 comp rx=0 tx=0).stop 2024-09-21T03:43:26.275 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:26.272+0000 7f2b40c8b640 1 -- 172.21.15.39:0/4167054222 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2b3c1094a0 msgr2=0x7f2b3c19a9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:26.275 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:26.272+0000 7f2b40c8b640 1 --2- 172.21.15.39:0/4167054222 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2b3c1094a0 0x7f2b3c19a9a0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f2b3000c970 tx=0x7f2b3000ce30 comp rx=0 tx=0).stop 2024-09-21T03:43:26.275 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:26.272+0000 7f2b40c8b640 1 -- 172.21.15.39:0/4167054222 shutdown_connections 2024-09-21T03:43:26.275 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:26.272+0000 7f2b40c8b640 1 --2- 172.21.15.39:0/4167054222 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f2b240b6590 0x7f2b240b8aa0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:26.275 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:26.272+0000 7f2b40c8b640 1 --2- 172.21.15.39:0/4167054222 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2b3c109df0 0x7f2b3c19fc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:26.275 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:26.272+0000 7f2b40c8b640 1 --2- 172.21.15.39:0/4167054222 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2b3c1094a0 0x7f2b3c19a9a0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:26.275 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:26.272+0000 7f2b40c8b640 1 --2- 172.21.15.39:0/4167054222 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2b3c103b60 0x7f2b3c19a400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:26.275 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:26.272+0000 7f2b40c8b640 1 -- 172.21.15.39:0/4167054222 >> 172.21.15.39:0/4167054222 conn(0x7f2b3c07b850 msgr2=0x7f2b3c107490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:43:26.275 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:26.272+0000 7f2b40c8b640 1 -- 172.21.15.39:0/4167054222 shutdown_connections 2024-09-21T03:43:26.276 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:26.272+0000 7f2b40c8b640 1 -- 172.21.15.39:0/4167054222 wait complete. 2024-09-21T03:43:26.583 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:43:26 smithi039 ceph-mon[31211]: osdmap e72: 12 total, 12 up, 12 in 2024-09-21T03:43:26.717 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:43:26 smithi161 ceph-mon[34533]: osdmap e72: 12 total, 12 up, 12 in 2024-09-21T03:43:26.724 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:43:26 smithi150 ceph-mon[34289]: osdmap e72: 12 total, 12 up, 12 in 2024-09-21T03:43:26.922 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":72,"num_osds":12,"num_up_osds":12,"osd_up_since":1726890204,"num_in_osds":12,"osd_in_since":1726890180,"num_remapped_pgs":0} 2024-09-21T03:43:26.923 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 51116d5c-77ca-11ef-baf5-efdc52797490 -- ceph osd dump --format=json 2024-09-21T03:43:27.176 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/51116d5c-77ca-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:43:27.584 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:43:27 smithi039 ceph-mon[31211]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T03:43:27.584 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:43:27 smithi039 ceph-mon[31211]: from='client.? 172.21.15.39:0/4167054222' entity='client.admin' cmd={"prefix": "osd stat", "format": "json"} : dispatch 2024-09-21T03:43:27.714 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:27.695+0000 7f32dea46640 1 -- 172.21.15.39:0/4172685221 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f32d8107630 msgr2=0x7f32d8107a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:27.714 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:27.695+0000 7f32dea46640 1 --2- 172.21.15.39:0/4172685221 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f32d8107630 0x7f32d8107a40 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f32c8009900 tx=0x7f32c802a410 comp rx=0 tx=0).stop 2024-09-21T03:43:27.714 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:27.696+0000 7f32dea46640 1 -- 172.21.15.39:0/4172685221 shutdown_connections 2024-09-21T03:43:27.714 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:27.696+0000 7f32dea46640 1 --2- 172.21.15.39:0/4172685221 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f32d81038f0 0x7f32d810f4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:27.714 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:27.696+0000 7f32dea46640 1 --2- 172.21.15.39:0/4172685221 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f32d8102fa0 0x7f32d81033b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:27.714 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:27.696+0000 7f32dea46640 1 --2- 172.21.15.39:0/4172685221 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f32d8107630 0x7f32d8107a40 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:27.714 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:27.696+0000 7f32dea46640 1 -- 172.21.15.39:0/4172685221 >> 172.21.15.39:0/4172685221 conn(0x7f32d8079e40 msgr2=0x7f32d807a250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:43:27.714 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:27.696+0000 7f32dea46640 1 -- 172.21.15.39:0/4172685221 shutdown_connections 2024-09-21T03:43:27.715 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:27.696+0000 7f32dea46640 1 -- 172.21.15.39:0/4172685221 wait complete. 2024-09-21T03:43:27.715 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:27.696+0000 7f32dea46640 1 Processor -- start 2024-09-21T03:43:27.715 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:27.696+0000 7f32dea46640 1 -- start start 2024-09-21T03:43:27.715 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:27.697+0000 7f32dea46640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f32d8102fa0 0x7f32d819a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:27.715 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:27.697+0000 7f32dea46640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f32d81038f0 0x7f32d819aa20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:27.715 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:27.697+0000 7f32dea46640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f32d8107630 0x7f32d819fcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:27.715 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:27.697+0000 7f32dea46640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f32d81129e0 con 0x7f32d8107630 2024-09-21T03:43:27.715 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:27.697+0000 7f32dea46640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f32d8112860 con 0x7f32d81038f0 2024-09-21T03:43:27.715 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:27.697+0000 7f32dea46640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f32d8112b60 con 0x7f32d8102fa0 2024-09-21T03:43:27.715 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:27.697+0000 7f32dcfe3640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f32d8107630 0x7f32d819fcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:27.715 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:27.697+0000 7f32dcfe3640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f32d8107630 0x7f32d819fcc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:53664/0 (socket says 172.21.15.39:53664) 2024-09-21T03:43:27.715 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:27.697+0000 7f32d77fe640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f32d81038f0 0x7f32d819aa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:27.716 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:27.697+0000 7f32dcfe3640 1 -- 172.21.15.39:0/1117764996 learned_addr learned my addr 172.21.15.39:0/1117764996 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:43:27.716 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:27.697+0000 7f32d7fff640 1 --2- 172.21.15.39:0/1117764996 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f32d8102fa0 0x7f32d819a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:27.716 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:27.697+0000 7f32dcfe3640 1 -- 172.21.15.39:0/1117764996 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f32d8102fa0 msgr2=0x7f32d819a480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:27.716 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:27.697+0000 7f32dcfe3640 1 --2- 172.21.15.39:0/1117764996 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f32d8102fa0 0x7f32d819a480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:27.716 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:27.697+0000 7f32dcfe3640 1 -- 172.21.15.39:0/1117764996 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f32d81038f0 msgr2=0x7f32d819aa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:27.716 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:27.697+0000 7f32dcfe3640 1 --2- 172.21.15.39:0/1117764996 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f32d81038f0 0x7f32d819aa20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:27.716 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:27.697+0000 7f32dcfe3640 1 -- 172.21.15.39:0/1117764996 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f32d81a0330 con 0x7f32d8107630 2024-09-21T03:43:27.716 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:27.698+0000 7f32dcfe3640 1 --2- 172.21.15.39:0/1117764996 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f32d8107630 0x7f32d819fcc0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f32d000cad0 tx=0x7f32d000cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:43:27.716 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:27.698+0000 7f32d57fa640 1 -- 172.21.15.39:0/1117764996 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32d000ae20 con 0x7f32d8107630 2024-09-21T03:43:27.716 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:27.698+0000 7f32dea46640 1 -- 172.21.15.39:0/1117764996 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f32d81a0620 con 0x7f32d8107630 2024-09-21T03:43:27.716 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:27.698+0000 7f32d57fa640 1 -- 172.21.15.39:0/1117764996 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f32d0005020 con 0x7f32d8107630 2024-09-21T03:43:27.716 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:27.698+0000 7f32d57fa640 1 -- 172.21.15.39:0/1117764996 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32d000f6b0 con 0x7f32d8107630 2024-09-21T03:43:27.716 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:27.698+0000 7f32dea46640 1 -- 172.21.15.39:0/1117764996 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f32d81a0b10 con 0x7f32d8107630 2024-09-21T03:43:27.716 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:27.699+0000 7f32dea46640 1 -- 172.21.15.39:0/1117764996 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f32d81033b0 con 0x7f32d8107630 2024-09-21T03:43:27.717 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:27.700+0000 7f32d57fa640 1 -- 172.21.15.39:0/1117764996 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 21) ==== 154529+0+0 (secure 0 0 0) 0x7f32d00051d0 con 0x7f32d8107630 2024-09-21T03:43:27.717 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:27.701+0000 7f32d57fa640 1 --2- 172.21.15.39:0/1117764996 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f32c00b6590 0x7f32c00b8aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:27.717 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:27.701+0000 7f32d7fff640 1 --2- 172.21.15.39:0/1117764996 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f32c00b6590 0x7f32c00b8aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:27.717 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:27.701+0000 7f32d57fa640 1 -- 172.21.15.39:0/1117764996 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 7342+0+0 (secure 0 0 0) 0x7f32d00e7320 con 0x7f32d8107630 2024-09-21T03:43:27.717 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:27.702+0000 7f32d7fff640 1 --2- 172.21.15.39:0/1117764996 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f32c00b6590 0x7f32c00b8aa0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f32c8031440 tx=0x7f32c8005e80 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:43:27.717 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:27.704+0000 7f32d57fa640 1 -- 172.21.15.39:0/1117764996 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f32d81a0b10 con 0x7f32d8107630 2024-09-21T03:43:27.717 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:43:27 smithi161 ceph-mon[34533]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T03:43:27.717 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:43:27 smithi161 ceph-mon[34533]: from='client.? 172.21.15.39:0/4167054222' entity='client.admin' cmd={"prefix": "osd stat", "format": "json"} : dispatch 2024-09-21T03:43:27.724 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:43:27 smithi150 ceph-mon[34289]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T03:43:27.724 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:43:27 smithi150 ceph-mon[34289]: from='client.? 172.21.15.39:0/4167054222' entity='client.admin' cmd={"prefix": "osd stat", "format": "json"} : dispatch 2024-09-21T03:43:27.897 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:27.894+0000 7f32dea46640 1 -- 172.21.15.39:0/1117764996 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f32d819b5b0 con 0x7f32d8107630 2024-09-21T03:43:27.898 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:27.895+0000 7f32d57fa640 1 -- 172.21.15.39:0/1117764996 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v72) ==== 74+0+18759 (secure 0 0 0) 0x7f32d0014030 con 0x7f32d8107630 2024-09-21T03:43:27.898 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T03:43:27.898 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":72,"fsid":"51116d5c-77ca-11ef-baf5-efdc52797490","created":"2024-09-21T03:35:33.631350+0000","modified":"2024-09-21T03:43:25.253062+0000","last_up_change":"2024-09-21T03:43:24.242275+0000","last_in_change":"2024-09-21T03:43:00.886552+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T03:39:29.859542+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1ea7148f-2ace-4c10-86fe-b8019de80c1f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6802","nonce":870188791},{"type":"v1","addr":"172.21.15.39:6803","nonce":870188791}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6804","nonce":870188791},{"type":"v1","addr":"172.21.15.39:6805","nonce":870188791}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6808","nonce":870188791},{"type":"v1","addr":"172.21.15.39:6809","nonce":870188791}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6806","nonce":870188791},{"type":"v1","addr":"172.21.15.39:6807","nonce":870188791}]},"public_addr":"172.21.15.39:6803/870188791","cluster_addr":"172.21.15.39:6805/870188791","heartbeat_back_addr":"172.21.15.39:6809/870188791","heartbeat_front_addr":"172.21.15.39:6807/870188791","state":["exists","up"]},{"osd":1,"uuid":"7cb5db90-28e8-4a30-bedb-c039687d48b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6800","nonce":172427832},{"type":"v1","addr":"172.21.15.150:6801","nonce":172427832}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6802","nonce":172427832},{"type":"v1","addr":"172.21.15.150:6803","nonce":172427832}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6806","nonce":172427832},{"type":"v1","addr":"172.21.15.150:6807","nonce":172427832}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6804","nonce":172427832},{"type":"v1","addr":"172.21.15.150:6805","nonce":172427832}]},"public_addr":"172.21.15.150:6801/172427832","cluster_addr":"172.21.15.150:6803/172427832","heartbeat_back_addr":"172.21.15.150:6807/172427832","heartbeat_front_addr":"172.21.15.150:6805/172427832","state":["exists","up"]},{"osd":2,"uuid":"b25c6ab3-fc95-4751-bd63-6e3fc23e8bd7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":19,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6800","nonce":2727876133},{"type":"v1","addr":"172.21.15.161:6801","nonce":2727876133}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6802","nonce":2727876133},{"type":"v1","addr":"172.21.15.161:6803","nonce":2727876133}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6806","nonce":2727876133},{"type":"v1","addr":"172.21.15.161:6807","nonce":2727876133}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6804","nonce":2727876133},{"type":"v1","addr":"172.21.15.161:6805","nonce":2727876133}]},"public_addr":"172.21.15.161:6801/2727876133","cluster_addr":"172.21.15.161:6803/2727876133","heartbeat_back_addr":"172.21.15.161:6807/2727876133","heartbeat_front_addr":"172.21.15.161:6805/2727876133","state":["exists","up"]},{"osd":3,"uuid":"14417ae1-4536-45ba-9a5c-c3b447d1a72d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6810","nonce":295394468},{"type":"v1","addr":"172.21.15.39:6811","nonce":295394468}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6812","nonce":295394468},{"type":"v1","addr":"172.21.15.39:6813","nonce":295394468}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6816","nonce":295394468},{"type":"v1","addr":"172.21.15.39:6817","nonce":295394468}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6814","nonce":295394468},{"type":"v1","addr":"172.21.15.39:6815","nonce":295394468}]},"public_addr":"172.21.15.39:6811/295394468","cluster_addr":"172.21.15.39:6813/295394468","heartbeat_back_addr":"172.21.15.39:6817/295394468","heartbeat_front_addr":"172.21.15.39:6815/295394468","state":["exists","up"]},{"osd":4,"uuid":"33c732ef-20c2-43c4-abe6-045b687fc521","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6808","nonce":3097721553},{"type":"v1","addr":"172.21.15.150:6809","nonce":3097721553}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6810","nonce":3097721553},{"type":"v1","addr":"172.21.15.150:6811","nonce":3097721553}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6814","nonce":3097721553},{"type":"v1","addr":"172.21.15.150:6815","nonce":3097721553}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6812","nonce":3097721553},{"type":"v1","addr":"172.21.15.150:6813","nonce":3097721553}]},"public_addr":"172.21.15.150:6809/3097721553","cluster_addr":"172.21.15.150:6811/3097721553","heartbeat_back_addr":"172.21.15.150:6815/3097721553","heartbeat_front_addr":"172.21.15.150:6813/3097721553","state":["exists","up"]},{"osd":5,"uuid":"74673722-abd6-400a-bb50-ec3dfba0c5e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6808","nonce":742751023},{"type":"v1","addr":"172.21.15.161:6809","nonce":742751023}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6810","nonce":742751023},{"type":"v1","addr":"172.21.15.161:6811","nonce":742751023}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6814","nonce":742751023},{"type":"v1","addr":"172.21.15.161:6815","nonce":742751023}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6812","nonce":742751023},{"type":"v1","addr":"172.21.15.161:6813","nonce":742751023}]},"public_addr":"172.21.15.161:6809/742751023","cluster_addr":"172.21.15.161:6811/742751023","heartbeat_back_addr":"172.21.15.161:6815/742751023","heartbeat_front_addr":"172.21.15.161:6813/742751023","state":["exists","up"]},{"osd":6,"uuid":"06548677-74c4-4efa-8054-df09b957b649","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6818","nonce":1072700290},{"type":"v1","addr":"172.21.15.39:6819","nonce":1072700290}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6820","nonce":1072700290},{"type":"v1","addr":"172.21.15.39:6821","nonce":1072700290}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6824","nonce":1072700290},{"type":"v1","addr":"172.21.15.39:6825","nonce":1072700290}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6822","nonce":1072700290},{"type":"v1","addr":"172.21.15.39:6823","nonce":1072700290}]},"public_addr":"172.21.15.39:6819/1072700290","cluster_addr":"172.21.15.39:6821/1072700290","heartbeat_back_addr":"172.21.15.39:6825/1072700290","heartbeat_front_addr":"172.21.15.39:6823/1072700290","state":["exists","up"]},{"osd":7,"uuid":"838735ca-c693-4f58-9426-b8ee857dbe3a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6816","nonce":2162619103},{"type":"v1","addr":"172.21.15.150:6817","nonce":2162619103}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6818","nonce":2162619103},{"type":"v1","addr":"172.21.15.150:6819","nonce":2162619103}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6822","nonce":2162619103},{"type":"v1","addr":"172.21.15.150:6823","nonce":2162619103}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6820","nonce":2162619103},{"type":"v1","addr":"172.21.15.150:6821","nonce":2162619103}]},"public_addr":"172.21.15.150:6817/2162619103","cluster_addr":"172.21.15.150:6819/2162619103","heartbeat_back_addr":"172.21.15.150:6823/2162619103","heartbeat_front_addr":"172.21.15.150:6821/2162619103","state":["exists","up"]},{"osd":8,"uuid":"4973b357-b4c2-4c00-b7f7-dd25012a0f72","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6816","nonce":52966633},{"type":"v1","addr":"172.21.15.161:6817","nonce":52966633}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6818","nonce":52966633},{"type":"v1","addr":"172.21.15.161:6819","nonce":52966633}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6822","nonce":52966633},{"type":"v1","addr":"172.21.15.161:6823","nonce":52966633}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6820","nonce":52966633},{"type":"v1","addr":"172.21.15.161:6821","nonce":52966633}]},"public_addr":"172.21.15.161:6817/52966633","cluster_addr":"172.21.15.161:6819/52966633","heartbeat_back_addr":"172.21.15.161:6823/52966633","heartbeat_front_addr":"172.21.15.161:6821/52966633","state":["exists","up"]},{"osd":9,"uuid":"6028613c-ebfd-42ea-9949-3f0d5f6cf025","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6826","nonce":2001303253},{"type":"v1","addr":"172.21.15.39:6827","nonce":2001303253}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6828","nonce":2001303253},{"type":"v1","addr":"172.21.15.39:6829","nonce":2001303253}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6832","nonce":2001303253},{"type":"v1","addr":"172.21.15.39:6833","nonce":2001303253}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6830","nonce":2001303253},{"type":"v1","addr":"172.21.15.39:6831","nonce":2001303253}]},"public_addr":"172.21.15.39:6827/2001303253","cluster_addr":"172.21.15.39:6829/2001303253","heartbeat_back_addr":"172.21.15.39:6833/2001303253","heartbeat_front_addr":"172.21.15.39:6831/2001303253","state":["exists","up"]},{"osd":10,"uuid":"1c71c765-2a6c-4ffe-9fe1-c8adae3d1c32","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6824","nonce":2484627938},{"type":"v1","addr":"172.21.15.150:6825","nonce":2484627938}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6826","nonce":2484627938},{"type":"v1","addr":"172.21.15.150:6827","nonce":2484627938}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6830","nonce":2484627938},{"type":"v1","addr":"172.21.15.150:6831","nonce":2484627938}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6828","nonce":2484627938},{"type":"v1","addr":"172.21.15.150:6829","nonce":2484627938}]},"public_addr":"172.21.15.150:6825/2484627938","cluster_addr":"172.21.15.150:6827/2484627938","heartbeat_back_addr":"172.21.15.150:6831/2484627938","heartbeat_front_addr":"172.21.15.150:6829/2484627938","state":["exists","up"]},{"osd":11,"uuid":"f382f522-62db-46da-997e-2b28f84d1612","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6824","nonce":2178709707},{"type":"v1","addr":"172.21.15.161:6825","nonce":2178709707}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6826","nonce":2178709707},{"type":"v1","addr":"172.21.15.161:6827","nonce":2178709707}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6830","nonce":2178709707},{"type":"v1","addr":"172.21.15.161:6831","nonce":2178709707}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6828","nonce":2178709707},{"type":"v1","addr":"172.21.15.161:6829","nonce":2178709707}]},"public_addr":"172.21.15.161:6825/2178709707","cluster_addr":"172.21.15.161:6827/2178709707","heartbeat_back_addr":"172.21.15.161:6831/2178709707","heartbeat_front_addr":"172.21.15.161:6829/2178709707","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:38:35.540904+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:39:01.455053+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:39:27.052867+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:39:55.173529+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:40:21.298396+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:40:47.131235+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:41:14.475209+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:41:39.920036+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:42:05.587065+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:42:32.624726+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:42:57.479185+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:43:22.203832+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.39:6801/2438110611":"2024-09-22T03:37:25.391457+0000","172.21.15.39:6800/2438110611":"2024-09-22T03:37:25.391457+0000","172.21.15.39:6800/173264695":"2024-09-22T03:35:57.104467+0000","172.21.15.39:6801/173264695":"2024-09-22T03:35:57.104467+0000","172.21.15.39:0/2637182284":"2024-09-22T03:35:57.104467+0000","172.21.15.39:0/1517304561":"2024-09-22T03:36:24.112592+0000","172.21.15.39:6801/1435622199":"2024-09-22T03:36:24.112592+0000","172.21.15.39:6800/1435622199":"2024-09-22T03:36:24.112592+0000","172.21.15.39:0/346832469":"2024-09-22T03:35:57.104467+0000","172.21.15.39:0/3160777371":"2024-09-22T03:36:24.112592+0000","172.21.15.39:0/1730342855":"2024-09-22T03:35:57.104467+0000","172.21.15.39:0/1955726766":"2024-09-22T03:36:24.112592+0000","172.21.15.39:0/3294132796":"2024-09-22T03:37:25.391457+0000","172.21.15.39:0/1295810162":"2024-09-22T03:37:25.391457+0000","172.21.15.39:0/3729230474":"2024-09-22T03:37:25.391457+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T03:43:27.900 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:27.897+0000 7f32dea46640 1 -- 172.21.15.39:0/1117764996 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f32c00b6590 msgr2=0x7f32c00b8aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:27.900 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:27.898+0000 7f32dea46640 1 --2- 172.21.15.39:0/1117764996 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f32c00b6590 0x7f32c00b8aa0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f32c8031440 tx=0x7f32c8005e80 comp rx=0 tx=0).stop 2024-09-21T03:43:27.901 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:27.898+0000 7f32dea46640 1 -- 172.21.15.39:0/1117764996 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f32d8107630 msgr2=0x7f32d819fcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:27.901 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:27.898+0000 7f32dea46640 1 --2- 172.21.15.39:0/1117764996 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f32d8107630 0x7f32d819fcc0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f32d000cad0 tx=0x7f32d000cf90 comp rx=0 tx=0).stop 2024-09-21T03:43:27.901 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:27.898+0000 7f32dea46640 1 -- 172.21.15.39:0/1117764996 shutdown_connections 2024-09-21T03:43:27.901 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:27.898+0000 7f32dea46640 1 --2- 172.21.15.39:0/1117764996 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f32c00b6590 0x7f32c00b8aa0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:27.901 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:27.898+0000 7f32dea46640 1 --2- 172.21.15.39:0/1117764996 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f32d8107630 0x7f32d819fcc0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:27.901 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:27.898+0000 7f32dea46640 1 --2- 172.21.15.39:0/1117764996 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f32d81038f0 0x7f32d819aa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:27.901 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:27.898+0000 7f32dea46640 1 --2- 172.21.15.39:0/1117764996 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f32d8102fa0 0x7f32d819a480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:27.901 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:27.898+0000 7f32dea46640 1 -- 172.21.15.39:0/1117764996 >> 172.21.15.39:0/1117764996 conn(0x7f32d8079e40 msgr2=0x7f32d8106500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:43:27.901 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:27.898+0000 7f32dea46640 1 -- 172.21.15.39:0/1117764996 shutdown_connections 2024-09-21T03:43:27.901 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:27.898+0000 7f32dea46640 1 -- 172.21.15.39:0/1117764996 wait complete. 2024-09-21T03:43:28.583 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:43:28 smithi039 ceph-mon[31211]: from='client.? 172.21.15.39:0/1117764996' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T03:43:28.717 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:43:28 smithi161 ceph-mon[34533]: from='client.? 172.21.15.39:0/1117764996' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T03:43:28.724 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:43:28 smithi150 ceph-mon[34289]: from='client.? 172.21.15.39:0/1117764996' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T03:43:28.728 INFO:tasks.cephadm.ceph_manager.ceph:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-21T03:39:29.859542+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-09-21T03:43:28.729 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 51116d5c-77ca-11ef-baf5-efdc52797490 -- ceph osd pool get .mgr pg_num 2024-09-21T03:43:28.988 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/51116d5c-77ca-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:43:29.363 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:43:29 smithi039 ceph-mon[31211]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T03:43:29.459 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:29.456+0000 7f1483da2640 1 -- 172.21.15.39:0/1260588 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f147c107b70 msgr2=0x7f147c107f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:29.459 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:29.456+0000 7f1483da2640 1 --2- 172.21.15.39:0/1260588 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f147c107b70 0x7f147c107f80 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f146c009900 tx=0x7f146c02a410 comp rx=0 tx=0).stop 2024-09-21T03:43:29.459 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:29.456+0000 7f1483da2640 1 -- 172.21.15.39:0/1260588 shutdown_connections 2024-09-21T03:43:29.459 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:29.456+0000 7f1483da2640 1 --2- 172.21.15.39:0/1260588 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f147c1084c0 0x7f147c10ecf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:29.459 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:29.456+0000 7f1483da2640 1 --2- 172.21.15.39:0/1260588 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f147c107b70 0x7f147c107f80 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:29.460 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:29.456+0000 7f1483da2640 1 --2- 172.21.15.39:0/1260588 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f147c102230 0x7f147c102640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:29.460 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:29.456+0000 7f1483da2640 1 -- 172.21.15.39:0/1260588 >> 172.21.15.39:0/1260588 conn(0x7f147c079e40 msgr2=0x7f147c07a250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:43:29.460 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:29.456+0000 7f1483da2640 1 -- 172.21.15.39:0/1260588 shutdown_connections 2024-09-21T03:43:29.460 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:29.456+0000 7f1483da2640 1 -- 172.21.15.39:0/1260588 wait complete. 2024-09-21T03:43:29.462 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:29.457+0000 7f1483da2640 1 Processor -- start 2024-09-21T03:43:29.462 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:29.457+0000 7f1483da2640 1 -- start start 2024-09-21T03:43:29.462 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:29.458+0000 7f1483da2640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f147c102230 0x7f147c19a5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:29.462 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:29.458+0000 7f1483da2640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f147c107b70 0x7f147c19ab80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:29.462 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:29.458+0000 7f1483da2640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f147c1084c0 0x7f147c19fe20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:29.462 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:29.458+0000 7f1483da2640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f147c113af0 con 0x7f147c107b70 2024-09-21T03:43:29.462 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:29.458+0000 7f1483da2640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f147c113970 con 0x7f147c102230 2024-09-21T03:43:29.462 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:29.458+0000 7f1483da2640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f147c113c70 con 0x7f147c1084c0 2024-09-21T03:43:29.462 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:29.458+0000 7f148133d640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f147c107b70 0x7f147c19ab80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:29.462 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:29.458+0000 7f148133d640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f147c107b70 0x7f147c19ab80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:53682/0 (socket says 172.21.15.39:53682) 2024-09-21T03:43:29.463 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:29.458+0000 7f148133d640 1 -- 172.21.15.39:0/957438347 learned_addr learned my addr 172.21.15.39:0/957438347 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:43:29.463 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:29.458+0000 7f1481b3e640 1 --2- 172.21.15.39:0/957438347 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f147c102230 0x7f147c19a5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:29.463 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:29.458+0000 7f148233f640 1 --2- 172.21.15.39:0/957438347 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f147c1084c0 0x7f147c19fe20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:29.463 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:29.458+0000 7f148133d640 1 -- 172.21.15.39:0/957438347 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f147c1084c0 msgr2=0x7f147c19fe20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:29.463 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:29.458+0000 7f148133d640 1 --2- 172.21.15.39:0/957438347 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f147c1084c0 0x7f147c19fe20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:29.463 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:29.458+0000 7f148133d640 1 -- 172.21.15.39:0/957438347 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f147c102230 msgr2=0x7f147c19a5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:29.463 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:29.458+0000 7f148133d640 1 --2- 172.21.15.39:0/957438347 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f147c102230 0x7f147c19a5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:29.463 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:29.458+0000 7f148133d640 1 -- 172.21.15.39:0/957438347 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f147c1a0490 con 0x7f147c107b70 2024-09-21T03:43:29.463 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:29.458+0000 7f148133d640 1 --2- 172.21.15.39:0/957438347 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f147c107b70 0x7f147c19ab80 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f146c02a930 tx=0x7f146c004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:43:29.463 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:29.458+0000 7f1472ffd640 1 -- 172.21.15.39:0/957438347 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f146c03f030 con 0x7f147c107b70 2024-09-21T03:43:29.463 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:29.459+0000 7f1472ffd640 1 -- 172.21.15.39:0/957438347 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f146c003cb0 con 0x7f147c107b70 2024-09-21T03:43:29.464 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:29.459+0000 7f1472ffd640 1 -- 172.21.15.39:0/957438347 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f146c035070 con 0x7f147c107b70 2024-09-21T03:43:29.464 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:29.459+0000 7f1483da2640 1 -- 172.21.15.39:0/957438347 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f147c1a0720 con 0x7f147c107b70 2024-09-21T03:43:29.464 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:29.459+0000 7f1483da2640 1 -- 172.21.15.39:0/957438347 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f147c1a0bb0 con 0x7f147c107b70 2024-09-21T03:43:29.464 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:29.460+0000 7f1483da2640 1 -- 172.21.15.39:0/957438347 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f147c102640 con 0x7f147c107b70 2024-09-21T03:43:29.464 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:29.461+0000 7f1472ffd640 1 -- 172.21.15.39:0/957438347 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 21) ==== 154529+0+0 (secure 0 0 0) 0x7f146c042030 con 0x7f147c107b70 2024-09-21T03:43:29.465 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:29.462+0000 7f1472ffd640 1 --2- 172.21.15.39:0/957438347 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f14680b6610 0x7f14680b8b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:29.465 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:29.463+0000 7f1481b3e640 1 --2- 172.21.15.39:0/957438347 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f14680b6610 0x7f14680b8b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:29.465 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:29.463+0000 7f1472ffd640 1 -- 172.21.15.39:0/957438347 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 7342+0+0 (secure 0 0 0) 0x7f146c1078e0 con 0x7f147c107b70 2024-09-21T03:43:29.465 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:29.463+0000 7f1481b3e640 1 --2- 172.21.15.39:0/957438347 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f14680b6610 0x7f14680b8b20 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f14780058c0 tx=0x7f147800a250 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:43:29.468 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:29.466+0000 7f1472ffd640 1 -- 172.21.15.39:0/957438347 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f146c0d2920 con 0x7f147c107b70 2024-09-21T03:43:29.657 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:29.654+0000 7f1483da2640 1 -- 172.21.15.39:0/957438347 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f147c10a2c0 con 0x7f147c107b70 2024-09-21T03:43:29.657 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:29.655+0000 7f1472ffd640 1 -- 172.21.15.39:0/957438347 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v72) ==== 93+0+10 (secure 0 0 0) 0x7f146c0d7860 con 0x7f147c107b70 2024-09-21T03:43:29.657 INFO:teuthology.orchestra.run.smithi039.stdout:pg_num: 1 2024-09-21T03:43:29.660 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:29.657+0000 7f1483da2640 1 -- 172.21.15.39:0/957438347 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f14680b6610 msgr2=0x7f14680b8b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:29.660 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:29.657+0000 7f1483da2640 1 --2- 172.21.15.39:0/957438347 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f14680b6610 0x7f14680b8b20 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f14780058c0 tx=0x7f147800a250 comp rx=0 tx=0).stop 2024-09-21T03:43:29.660 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:29.657+0000 7f1483da2640 1 -- 172.21.15.39:0/957438347 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f147c107b70 msgr2=0x7f147c19ab80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:29.660 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:29.657+0000 7f1483da2640 1 --2- 172.21.15.39:0/957438347 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f147c107b70 0x7f147c19ab80 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f146c02a930 tx=0x7f146c004270 comp rx=0 tx=0).stop 2024-09-21T03:43:29.660 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:29.657+0000 7f1483da2640 1 -- 172.21.15.39:0/957438347 shutdown_connections 2024-09-21T03:43:29.660 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:29.657+0000 7f1483da2640 1 --2- 172.21.15.39:0/957438347 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f14680b6610 0x7f14680b8b20 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:29.660 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:29.657+0000 7f1483da2640 1 --2- 172.21.15.39:0/957438347 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f147c1084c0 0x7f147c19fe20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:29.660 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:29.657+0000 7f1483da2640 1 --2- 172.21.15.39:0/957438347 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f147c107b70 0x7f147c19ab80 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:29.660 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:29.657+0000 7f1483da2640 1 --2- 172.21.15.39:0/957438347 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f147c102230 0x7f147c19a5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:29.661 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:29.657+0000 7f1483da2640 1 -- 172.21.15.39:0/957438347 >> 172.21.15.39:0/957438347 conn(0x7f147c079e40 msgr2=0x7f147c103ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:43:29.661 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:29.657+0000 7f1483da2640 1 -- 172.21.15.39:0/957438347 shutdown_connections 2024-09-21T03:43:29.661 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:29.657+0000 7f1483da2640 1 -- 172.21.15.39:0/957438347 wait complete. 2024-09-21T03:43:29.717 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:43:29 smithi161 ceph-mon[34533]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T03:43:29.724 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:43:29 smithi150 ceph-mon[34289]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T03:43:30.401 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:43:30 smithi039 ceph-mon[31211]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T03:43:30.401 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:43:30 smithi039 ceph-mon[31211]: from='client.? 172.21.15.39:0/957438347' entity='client.admin' cmd={"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} : dispatch 2024-09-21T03:43:30.432 INFO:tasks.cephadm:Adding mds.a on smithi039 2024-09-21T03:43:30.432 INFO:tasks.cephadm:Adding mds.d on smithi039 2024-09-21T03:43:30.432 INFO:tasks.cephadm:Adding mds.g on smithi039 2024-09-21T03:43:30.432 INFO:tasks.cephadm:Adding mds.j on smithi039 2024-09-21T03:43:30.433 INFO:tasks.cephadm:Adding mds.b on smithi150 2024-09-21T03:43:30.433 INFO:tasks.cephadm:Adding mds.e on smithi150 2024-09-21T03:43:30.433 INFO:tasks.cephadm:Adding mds.h on smithi150 2024-09-21T03:43:30.433 INFO:tasks.cephadm:Adding mds.k on smithi150 2024-09-21T03:43:30.433 INFO:tasks.cephadm:Adding mds.c on smithi161 2024-09-21T03:43:30.433 INFO:tasks.cephadm:Adding mds.f on smithi161 2024-09-21T03:43:30.433 INFO:tasks.cephadm:Adding mds.i on smithi161 2024-09-21T03:43:30.433 INFO:tasks.cephadm:Adding mds.l on smithi161 2024-09-21T03:43:30.433 DEBUG:teuthology.orchestra.run.smithi161:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 51116d5c-77ca-11ef-baf5-efdc52797490 -- ceph orch apply mds all '12;smithi039=a;smithi039=d;smithi039=g;smithi039=j;smithi150=b;smithi150=e;smithi150=h;smithi150=k;smithi161=c;smithi161=f;smithi161=i;smithi161=l' 2024-09-21T03:43:30.695 INFO:teuthology.orchestra.run.smithi161.stderr:Inferring config /var/lib/ceph/51116d5c-77ca-11ef-baf5-efdc52797490/mon.c/config 2024-09-21T03:43:30.718 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:43:30 smithi161 ceph-mon[34533]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T03:43:30.718 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:43:30 smithi161 ceph-mon[34533]: from='client.? 172.21.15.39:0/957438347' entity='client.admin' cmd={"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} : dispatch 2024-09-21T03:43:30.724 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:43:30 smithi150 ceph-mon[34289]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T03:43:30.724 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:43:30 smithi150 ceph-mon[34289]: from='client.? 172.21.15.39:0/957438347' entity='client.admin' cmd={"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} : dispatch 2024-09-21T03:43:31.150 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-21T03:43:31.147+0000 7fcf7a9d6640 1 -- 172.21.15.161:0/2946965833 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcf7410ddf0 msgr2=0x7fcf74110300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:31.150 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-21T03:43:31.147+0000 7fcf7a9d6640 1 --2- 172.21.15.161:0/2946965833 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcf7410ddf0 0x7fcf74110300 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fcf6c009ca0 tx=0x7fcf6c02a8c0 comp rx=0 tx=0).stop 2024-09-21T03:43:31.150 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-21T03:43:31.148+0000 7fcf7a9d6640 1 -- 172.21.15.161:0/2946965833 shutdown_connections 2024-09-21T03:43:31.150 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-21T03:43:31.148+0000 7fcf7a9d6640 1 --2- 172.21.15.161:0/2946965833 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcf7410ddf0 0x7fcf74110300 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:31.151 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-21T03:43:31.148+0000 7fcf7a9d6640 1 --2- 172.21.15.161:0/2946965833 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcf74105270 0x7fcf7410d8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:31.151 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-21T03:43:31.148+0000 7fcf7a9d6640 1 --2- 172.21.15.161:0/2946965833 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcf74104840 0x7fcf74104c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:31.151 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-21T03:43:31.148+0000 7fcf7a9d6640 1 -- 172.21.15.161:0/2946965833 >> 172.21.15.161:0/2946965833 conn(0x7fcf74079730 msgr2=0x7fcf74079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:43:31.151 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-21T03:43:31.148+0000 7fcf7a9d6640 1 -- 172.21.15.161:0/2946965833 shutdown_connections 2024-09-21T03:43:31.151 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-21T03:43:31.148+0000 7fcf7a9d6640 1 -- 172.21.15.161:0/2946965833 wait complete. 2024-09-21T03:43:31.151 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-21T03:43:31.148+0000 7fcf7a9d6640 1 Processor -- start 2024-09-21T03:43:31.151 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-21T03:43:31.148+0000 7fcf7a9d6640 1 -- start start 2024-09-21T03:43:31.153 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-21T03:43:31.149+0000 7fcf7a9d6640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcf74104840 0x7fcf7419a440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:31.153 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-21T03:43:31.149+0000 7fcf7a9d6640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcf74105270 0x7fcf7419a9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:31.153 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-21T03:43:31.149+0000 7fcf7a9d6640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcf7410ddf0 0x7fcf7419fc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:31.153 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-21T03:43:31.149+0000 7fcf7a9d6640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fcf74113a90 con 0x7fcf7410ddf0 2024-09-21T03:43:31.153 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-21T03:43:31.149+0000 7fcf7a9d6640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fcf74113910 con 0x7fcf74105270 2024-09-21T03:43:31.153 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-21T03:43:31.149+0000 7fcf7a9d6640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fcf74113c10 con 0x7fcf74104840 2024-09-21T03:43:31.153 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-21T03:43:31.149+0000 7fcf73fff640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcf74104840 0x7fcf7419a440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:31.153 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-21T03:43:31.149+0000 7fcf73fff640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcf74104840 0x7fcf7419a440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:57110/0 (socket says 172.21.15.161:57110) 2024-09-21T03:43:31.153 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-21T03:43:31.149+0000 7fcf73fff640 1 -- 172.21.15.161:0/2498049566 learned_addr learned my addr 172.21.15.161:0/2498049566 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-09-21T03:43:31.153 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-21T03:43:31.149+0000 7fcf737fe640 1 --2- 172.21.15.161:0/2498049566 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcf74105270 0x7fcf7419a9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:31.153 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-21T03:43:31.149+0000 7fcf78f73640 1 --2- 172.21.15.161:0/2498049566 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcf7410ddf0 0x7fcf7419fc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:31.153 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-21T03:43:31.149+0000 7fcf73fff640 1 -- 172.21.15.161:0/2498049566 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcf74105270 msgr2=0x7fcf7419a9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:31.154 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-21T03:43:31.149+0000 7fcf73fff640 1 --2- 172.21.15.161:0/2498049566 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcf74105270 0x7fcf7419a9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:31.154 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-21T03:43:31.149+0000 7fcf73fff640 1 -- 172.21.15.161:0/2498049566 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcf7410ddf0 msgr2=0x7fcf7419fc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:31.154 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-21T03:43:31.149+0000 7fcf73fff640 1 --2- 172.21.15.161:0/2498049566 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcf7410ddf0 0x7fcf7419fc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:31.154 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-21T03:43:31.149+0000 7fcf73fff640 1 -- 172.21.15.161:0/2498049566 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcf741a03b0 con 0x7fcf74104840 2024-09-21T03:43:31.154 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-21T03:43:31.150+0000 7fcf73fff640 1 --2- 172.21.15.161:0/2498049566 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcf74104840 0x7fcf7419a440 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fcf6400e9c0 tx=0x7fcf6400ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:43:31.154 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-21T03:43:31.150+0000 7fcf717fa640 1 -- 172.21.15.161:0/2498049566 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf6400cd80 con 0x7fcf74104840 2024-09-21T03:43:31.154 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-21T03:43:31.150+0000 7fcf7a9d6640 1 -- 172.21.15.161:0/2498049566 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcf741a06a0 con 0x7fcf74104840 2024-09-21T03:43:31.154 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-21T03:43:31.150+0000 7fcf717fa640 1 -- 172.21.15.161:0/2498049566 <== mon.1 v2:172.21.15.161:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcf640035f0 con 0x7fcf74104840 2024-09-21T03:43:31.154 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-21T03:43:31.150+0000 7fcf717fa640 1 -- 172.21.15.161:0/2498049566 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf64010690 con 0x7fcf74104840 2024-09-21T03:43:31.154 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-21T03:43:31.150+0000 7fcf7a9d6640 1 -- 172.21.15.161:0/2498049566 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcf741a6e00 con 0x7fcf74104840 2024-09-21T03:43:31.154 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-21T03:43:31.151+0000 7fcf7a9d6640 1 -- 172.21.15.161:0/2498049566 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcf74104c50 con 0x7fcf74104840 2024-09-21T03:43:31.156 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-21T03:43:31.154+0000 7fcf717fa640 1 -- 172.21.15.161:0/2498049566 <== mon.1 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 21) ==== 154529+0+0 (secure 0 0 0) 0x7fcf640037a0 con 0x7fcf74104840 2024-09-21T03:43:31.157 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-21T03:43:31.155+0000 7fcf717fa640 1 --2- 172.21.15.161:0/2498049566 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7fcf5c0b6560 0x7fcf5c0b8a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:31.157 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-21T03:43:31.155+0000 7fcf717fa640 1 -- 172.21.15.161:0/2498049566 <== mon.1 v2:172.21.15.161:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 7342+0+0 (secure 0 0 0) 0x7fcf640e74a0 con 0x7fcf74104840 2024-09-21T03:43:31.157 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-21T03:43:31.155+0000 7fcf737fe640 1 --2- 172.21.15.161:0/2498049566 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7fcf5c0b6560 0x7fcf5c0b8a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:31.157 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-21T03:43:31.156+0000 7fcf737fe640 1 --2- 172.21.15.161:0/2498049566 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7fcf5c0b6560 0x7fcf5c0b8a70 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fcf7410fae0 tx=0x7fcf68005f90 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:43:31.160 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-21T03:43:31.158+0000 7fcf717fa640 1 -- 172.21.15.161:0/2498049566 <== mon.1 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fcf640ae100 con 0x7fcf74104840 2024-09-21T03:43:31.357 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-21T03:43:31.355+0000 7fcf7a9d6640 1 -- 172.21.15.161:0/2498049566 --> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] -- mgr_command(tid 0: {"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi039=a;smithi039=d;smithi039=g;smithi039=j;smithi150=b;smithi150=e;smithi150=h;smithi150=k;smithi161=c;smithi161=f;smithi161=i;smithi161=l", "target": ["mon-mgr", ""]}) -- 0x7fcf74100c70 con 0x7fcf5c0b6560 2024-09-21T03:43:31.366 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-21T03:43:31.364+0000 7fcf717fa640 1 -- 172.21.15.161:0/2498049566 <== mgr.14223 v2:172.21.15.39:6800/1696381982 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+28 (secure 0 0 0) 0x7fcf74100c70 con 0x7fcf5c0b6560 2024-09-21T03:43:31.367 INFO:teuthology.orchestra.run.smithi161.stdout:Scheduled mds.all update... 2024-09-21T03:43:31.369 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-21T03:43:31.367+0000 7fcf7a9d6640 1 -- 172.21.15.161:0/2498049566 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7fcf5c0b6560 msgr2=0x7fcf5c0b8a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:31.369 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-21T03:43:31.367+0000 7fcf7a9d6640 1 --2- 172.21.15.161:0/2498049566 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7fcf5c0b6560 0x7fcf5c0b8a70 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fcf7410fae0 tx=0x7fcf68005f90 comp rx=0 tx=0).stop 2024-09-21T03:43:31.369 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-21T03:43:31.367+0000 7fcf7a9d6640 1 -- 172.21.15.161:0/2498049566 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcf74104840 msgr2=0x7fcf7419a440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:31.369 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-21T03:43:31.367+0000 7fcf7a9d6640 1 --2- 172.21.15.161:0/2498049566 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcf74104840 0x7fcf7419a440 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fcf6400e9c0 tx=0x7fcf6400ee80 comp rx=0 tx=0).stop 2024-09-21T03:43:31.369 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-21T03:43:31.367+0000 7fcf7a9d6640 1 -- 172.21.15.161:0/2498049566 shutdown_connections 2024-09-21T03:43:31.369 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-21T03:43:31.367+0000 7fcf7a9d6640 1 --2- 172.21.15.161:0/2498049566 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7fcf5c0b6560 0x7fcf5c0b8a70 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:31.370 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-21T03:43:31.367+0000 7fcf7a9d6640 1 --2- 172.21.15.161:0/2498049566 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcf7410ddf0 0x7fcf7419fc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:31.370 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-21T03:43:31.367+0000 7fcf7a9d6640 1 --2- 172.21.15.161:0/2498049566 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcf74105270 0x7fcf7419a9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:31.370 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-21T03:43:31.367+0000 7fcf7a9d6640 1 --2- 172.21.15.161:0/2498049566 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcf74104840 0x7fcf7419a440 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:31.370 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-21T03:43:31.367+0000 7fcf7a9d6640 1 -- 172.21.15.161:0/2498049566 >> 172.21.15.161:0/2498049566 conn(0x7fcf74079730 msgr2=0x7fcf740ff550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:43:31.370 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-21T03:43:31.367+0000 7fcf7a9d6640 1 -- 172.21.15.161:0/2498049566 shutdown_connections 2024-09-21T03:43:31.370 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-21T03:43:31.367+0000 7fcf7a9d6640 1 -- 172.21.15.161:0/2498049566 wait complete. 2024-09-21T03:43:32.040 DEBUG:teuthology.orchestra.run.smithi039:mds.a> sudo journalctl -f -n 0 -u ceph-51116d5c-77ca-11ef-baf5-efdc52797490@mds.a.service 2024-09-21T03:43:32.043 DEBUG:teuthology.orchestra.run.smithi039:mds.d> sudo journalctl -f -n 0 -u ceph-51116d5c-77ca-11ef-baf5-efdc52797490@mds.d.service 2024-09-21T03:43:32.086 DEBUG:teuthology.orchestra.run.smithi039:mds.g> sudo journalctl -f -n 0 -u ceph-51116d5c-77ca-11ef-baf5-efdc52797490@mds.g.service 2024-09-21T03:43:32.130 DEBUG:teuthology.orchestra.run.smithi039:mds.j> sudo journalctl -f -n 0 -u ceph-51116d5c-77ca-11ef-baf5-efdc52797490@mds.j.service 2024-09-21T03:43:32.173 DEBUG:teuthology.orchestra.run.smithi150:mds.b> sudo journalctl -f -n 0 -u ceph-51116d5c-77ca-11ef-baf5-efdc52797490@mds.b.service 2024-09-21T03:43:32.176 DEBUG:teuthology.orchestra.run.smithi150:mds.e> sudo journalctl -f -n 0 -u ceph-51116d5c-77ca-11ef-baf5-efdc52797490@mds.e.service 2024-09-21T03:43:32.218 DEBUG:teuthology.orchestra.run.smithi150:mds.h> sudo journalctl -f -n 0 -u ceph-51116d5c-77ca-11ef-baf5-efdc52797490@mds.h.service 2024-09-21T03:43:32.262 DEBUG:teuthology.orchestra.run.smithi150:mds.k> sudo journalctl -f -n 0 -u ceph-51116d5c-77ca-11ef-baf5-efdc52797490@mds.k.service 2024-09-21T03:43:32.306 DEBUG:teuthology.orchestra.run.smithi161:mds.c> sudo journalctl -f -n 0 -u ceph-51116d5c-77ca-11ef-baf5-efdc52797490@mds.c.service 2024-09-21T03:43:32.308 DEBUG:teuthology.orchestra.run.smithi161:mds.f> sudo journalctl -f -n 0 -u ceph-51116d5c-77ca-11ef-baf5-efdc52797490@mds.f.service 2024-09-21T03:43:32.351 DEBUG:teuthology.orchestra.run.smithi161:mds.i> sudo journalctl -f -n 0 -u ceph-51116d5c-77ca-11ef-baf5-efdc52797490@mds.i.service 2024-09-21T03:43:32.393 DEBUG:teuthology.orchestra.run.smithi161:mds.l> sudo journalctl -f -n 0 -u ceph-51116d5c-77ca-11ef-baf5-efdc52797490@mds.l.service 2024-09-21T03:43:32.436 INFO:tasks.cephadm:Setting up CephFS filesystem(s)... 2024-09-21T03:43:32.437 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 51116d5c-77ca-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T03:43:32.455 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:43:32 smithi161 ceph-mon[34533]: from='client.24589 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi039=a;smithi039=d;smithi039=g;smithi039=j;smithi150=b;smithi150=e;smithi150=h;smithi150=k;smithi161=c;smithi161=f;smithi161=i;smithi161=l", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:43:32.456 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:43:32 smithi161 ceph-mon[34533]: Saving service mds.all spec with placement smithi039=a;smithi039=d;smithi039=g;smithi039=j;smithi150=b;smithi150=e;smithi150=h;smithi150=k;smithi161=c;smithi161=f;smithi161=i;smithi161=l;count:12 2024-09-21T03:43:32.456 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:43:32 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:43:32.456 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:43:32 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:43:32.456 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:43:32 smithi161 ceph-mon[34533]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T03:43:32.456 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:43:32 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:43:32.456 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:43:32 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:43:32.456 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:43:32 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:43:32.456 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:43:32 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T03:43:32.456 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:43:32 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T03:43:32.456 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:43:32 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:43:32.456 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:43:32 smithi161 ceph-mon[34533]: Deploying daemon mds.l on smithi161 2024-09-21T03:43:32.460 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:43:32 smithi039 ceph-mon[31211]: from='client.24589 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi039=a;smithi039=d;smithi039=g;smithi039=j;smithi150=b;smithi150=e;smithi150=h;smithi150=k;smithi161=c;smithi161=f;smithi161=i;smithi161=l", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:43:32.460 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:43:32 smithi039 ceph-mon[31211]: Saving service mds.all spec with placement smithi039=a;smithi039=d;smithi039=g;smithi039=j;smithi150=b;smithi150=e;smithi150=h;smithi150=k;smithi161=c;smithi161=f;smithi161=i;smithi161=l;count:12 2024-09-21T03:43:32.460 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:43:32 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:43:32.460 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:43:32 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:43:32.460 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:43:32 smithi039 ceph-mon[31211]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T03:43:32.460 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:43:32 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:43:32.460 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:43:32 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:43:32.460 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:43:32 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:43:32.460 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:43:32 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T03:43:32.460 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:43:32 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T03:43:32.460 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:43:32 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:43:32.461 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:43:32 smithi039 ceph-mon[31211]: Deploying daemon mds.l on smithi161 2024-09-21T03:43:32.474 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:43:32 smithi150 ceph-mon[34289]: from='client.24589 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi039=a;smithi039=d;smithi039=g;smithi039=j;smithi150=b;smithi150=e;smithi150=h;smithi150=k;smithi161=c;smithi161=f;smithi161=i;smithi161=l", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:43:32.474 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:43:32 smithi150 ceph-mon[34289]: Saving service mds.all spec with placement smithi039=a;smithi039=d;smithi039=g;smithi039=j;smithi150=b;smithi150=e;smithi150=h;smithi150=k;smithi161=c;smithi161=f;smithi161=i;smithi161=l;count:12 2024-09-21T03:43:32.475 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:43:32 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:43:32.475 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:43:32 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:43:32.475 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:43:32 smithi150 ceph-mon[34289]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T03:43:32.475 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:43:32 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:43:32.475 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:43:32 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:43:32.475 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:43:32 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:43:32.475 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:43:32 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T03:43:32.475 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:43:32 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T03:43:32.475 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:43:32 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:43:32.475 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:43:32 smithi150 ceph-mon[34289]: Deploying daemon mds.l on smithi161 2024-09-21T03:43:32.689 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/51116d5c-77ca-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:43:33.221 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:33.211+0000 7f7b7231a640 1 -- 172.21.15.39:0/1365373867 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7b6c1094a0 msgr2=0x7f7b6c1098b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:33.222 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:33.211+0000 7f7b7231a640 1 --2- 172.21.15.39:0/1365373867 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7b6c1094a0 0x7f7b6c1098b0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f7b60009900 tx=0x7f7b6002a410 comp rx=0 tx=0).stop 2024-09-21T03:43:33.222 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:33.211+0000 7f7b7231a640 1 -- 172.21.15.39:0/1365373867 shutdown_connections 2024-09-21T03:43:33.222 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:33.211+0000 7f7b7231a640 1 --2- 172.21.15.39:0/1365373867 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7b6c109df0 0x7f7b6c110620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:33.222 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:33.211+0000 7f7b7231a640 1 --2- 172.21.15.39:0/1365373867 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7b6c1094a0 0x7f7b6c1098b0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:33.222 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:33.211+0000 7f7b7231a640 1 --2- 172.21.15.39:0/1365373867 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7b6c103b60 0x7f7b6c103f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:33.222 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:33.211+0000 7f7b7231a640 1 -- 172.21.15.39:0/1365373867 >> 172.21.15.39:0/1365373867 conn(0x7f7b6c07b850 msgr2=0x7f7b6c07bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:43:33.222 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:33.211+0000 7f7b7231a640 1 -- 172.21.15.39:0/1365373867 shutdown_connections 2024-09-21T03:43:33.222 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:33.211+0000 7f7b7231a640 1 -- 172.21.15.39:0/1365373867 wait complete. 2024-09-21T03:43:33.222 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:33.212+0000 7f7b7231a640 1 Processor -- start 2024-09-21T03:43:33.222 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:33.212+0000 7f7b7231a640 1 -- start start 2024-09-21T03:43:33.222 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:33.213+0000 7f7b7231a640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7b6c103b60 0x7f7b6c19a510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:33.222 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:33.213+0000 7f7b7231a640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7b6c1094a0 0x7f7b6c19aab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:33.222 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:33.213+0000 7f7b7231a640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7b6c109df0 0x7f7b6c19fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:33.223 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:33.213+0000 7f7b7231a640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f7b6c113b00 con 0x7f7b6c109df0 2024-09-21T03:43:33.223 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:33.213+0000 7f7b7231a640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f7b6c113980 con 0x7f7b6c103b60 2024-09-21T03:43:33.223 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:33.213+0000 7f7b7231a640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f7b6c113c80 con 0x7f7b6c1094a0 2024-09-21T03:43:33.223 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:33.213+0000 7f7b708b7640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7b6c109df0 0x7f7b6c19fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:33.223 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:33.213+0000 7f7b708b7640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7b6c109df0 0x7f7b6c19fd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:53696/0 (socket says 172.21.15.39:53696) 2024-09-21T03:43:33.223 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:33.213+0000 7f7b708b7640 1 -- 172.21.15.39:0/1498125911 learned_addr learned my addr 172.21.15.39:0/1498125911 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:43:33.223 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:33.213+0000 7f7b6b7fe640 1 --2- 172.21.15.39:0/1498125911 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7b6c1094a0 0x7f7b6c19aab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:33.223 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:33.213+0000 7f7b708b7640 1 -- 172.21.15.39:0/1498125911 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7b6c1094a0 msgr2=0x7f7b6c19aab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:33.223 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:33.213+0000 7f7b708b7640 1 --2- 172.21.15.39:0/1498125911 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7b6c1094a0 0x7f7b6c19aab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:33.223 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:33.213+0000 7f7b708b7640 1 -- 172.21.15.39:0/1498125911 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7b6c103b60 msgr2=0x7f7b6c19a510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:33.223 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:33.213+0000 7f7b708b7640 1 --2- 172.21.15.39:0/1498125911 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7b6c103b60 0x7f7b6c19a510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:33.223 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:33.213+0000 7f7b708b7640 1 -- 172.21.15.39:0/1498125911 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7b6c1a03c0 con 0x7f7b6c109df0 2024-09-21T03:43:33.223 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:33.213+0000 7f7b708b7640 1 --2- 172.21.15.39:0/1498125911 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7b6c109df0 0x7f7b6c19fd50 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f7b6400cce0 tx=0x7f7b6400a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:43:33.224 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:33.213+0000 7f7b697fa640 1 -- 172.21.15.39:0/1498125911 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b6400f450 con 0x7f7b6c109df0 2024-09-21T03:43:33.224 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:33.213+0000 7f7b697fa640 1 -- 172.21.15.39:0/1498125911 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7b6400faa0 con 0x7f7b6c109df0 2024-09-21T03:43:33.224 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:33.214+0000 7f7b697fa640 1 -- 172.21.15.39:0/1498125911 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b64018b70 con 0x7f7b6c109df0 2024-09-21T03:43:33.224 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:33.214+0000 7f7b7231a640 1 -- 172.21.15.39:0/1498125911 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7b6c1a06b0 con 0x7f7b6c109df0 2024-09-21T03:43:33.224 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:33.214+0000 7f7b7231a640 1 -- 172.21.15.39:0/1498125911 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7b6c1a0b70 con 0x7f7b6c109df0 2024-09-21T03:43:33.224 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:33.215+0000 7f7b7231a640 1 -- 172.21.15.39:0/1498125911 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7b6c103f70 con 0x7f7b6c109df0 2024-09-21T03:43:33.224 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:33.217+0000 7f7b697fa640 1 -- 172.21.15.39:0/1498125911 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 21) ==== 154529+0+0 (secure 0 0 0) 0x7f7b64018d10 con 0x7f7b6c109df0 2024-09-21T03:43:33.224 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:33.218+0000 7f7b697fa640 1 --2- 172.21.15.39:0/1498125911 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f7b540b6590 0x7f7b540b8aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:33.224 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:33.218+0000 7f7b6bfff640 1 --2- 172.21.15.39:0/1498125911 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f7b540b6590 0x7f7b540b8aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:33.224 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:33.218+0000 7f7b697fa640 1 -- 172.21.15.39:0/1498125911 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 7342+0+0 (secure 0 0 0) 0x7f7b640e7540 con 0x7f7b6c109df0 2024-09-21T03:43:33.224 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:33.219+0000 7f7b6bfff640 1 --2- 172.21.15.39:0/1498125911 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f7b540b6590 0x7f7b540b8aa0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f7b5c005fd0 tx=0x7f7b5c005850 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:43:33.225 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:33.221+0000 7f7b697fa640 1 -- 172.21.15.39:0/1498125911 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f7b64014030 con 0x7f7b6c109df0 2024-09-21T03:43:33.449 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:33.446+0000 7f7b7231a640 1 -- 172.21.15.39:0/1498125911 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f7b6c19b640 con 0x7f7b6c109df0 2024-09-21T03:43:33.449 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:33.446+0000 7f7b697fa640 1 -- 172.21.15.39:0/1498125911 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 1 v1) ==== 93+0+611 (secure 0 0 0) 0x7f7b6c19b640 con 0x7f7b6c109df0 2024-09-21T03:43:33.449 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T03:43:33.449 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":1,"btime":"2024-09-21T03:35:33:631109+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[]} 2024-09-21T03:43:33.450 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 1 2024-09-21T03:43:33.452 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:33.449+0000 7f7b7231a640 1 -- 172.21.15.39:0/1498125911 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f7b540b6590 msgr2=0x7f7b540b8aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:33.452 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:33.449+0000 7f7b7231a640 1 --2- 172.21.15.39:0/1498125911 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f7b540b6590 0x7f7b540b8aa0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f7b5c005fd0 tx=0x7f7b5c005850 comp rx=0 tx=0).stop 2024-09-21T03:43:33.452 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:33.449+0000 7f7b7231a640 1 -- 172.21.15.39:0/1498125911 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7b6c109df0 msgr2=0x7f7b6c19fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:33.452 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:33.449+0000 7f7b7231a640 1 --2- 172.21.15.39:0/1498125911 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7b6c109df0 0x7f7b6c19fd50 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f7b6400cce0 tx=0x7f7b6400a620 comp rx=0 tx=0).stop 2024-09-21T03:43:33.452 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:33.449+0000 7f7b7231a640 1 -- 172.21.15.39:0/1498125911 shutdown_connections 2024-09-21T03:43:33.452 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:33.449+0000 7f7b7231a640 1 --2- 172.21.15.39:0/1498125911 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f7b540b6590 0x7f7b540b8aa0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:33.452 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:33.449+0000 7f7b7231a640 1 --2- 172.21.15.39:0/1498125911 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7b6c109df0 0x7f7b6c19fd50 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:33.452 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:33.449+0000 7f7b7231a640 1 --2- 172.21.15.39:0/1498125911 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7b6c1094a0 0x7f7b6c19aab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:33.452 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:33.449+0000 7f7b7231a640 1 --2- 172.21.15.39:0/1498125911 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7b6c103b60 0x7f7b6c19a510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:33.452 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:33.449+0000 7f7b7231a640 1 -- 172.21.15.39:0/1498125911 >> 172.21.15.39:0/1498125911 conn(0x7f7b6c07b850 msgr2=0x7f7b6c105560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:43:33.452 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:33.449+0000 7f7b7231a640 1 -- 172.21.15.39:0/1498125911 shutdown_connections 2024-09-21T03:43:33.453 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:33.449+0000 7f7b7231a640 1 -- 172.21.15.39:0/1498125911 wait complete. 2024-09-21T03:43:33.593 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:43:33 smithi161 ceph-mon[34533]: from='client.? 172.21.15.39:0/1498125911' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:43:33.833 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:43:33 smithi039 ceph-mon[31211]: from='client.? 172.21.15.39:0/1498125911' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:43:33.968 INFO:journalctl@ceph.mds.l.smithi161.stdout:Sep 21 03:43:33 smithi161 systemd[1]: Starting Ceph mds.l for 51116d5c-77ca-11ef-baf5-efdc52797490... 2024-09-21T03:43:33.974 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:43:33 smithi150 ceph-mon[34289]: from='client.? 172.21.15.39:0/1498125911' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:43:34.255 INFO:journalctl@ceph.mds.l.smithi161.stdout:Sep 21 03:43:33 smithi161 podman[63079]: 2024-09-21 03:43:33.888698224 +0000 UTC m=+0.015871876 image pull ddce307e1f3263fb077b4f3902e15660050631dad4d2823a20ee3f72bf18d977 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262 2024-09-21T03:43:34.255 INFO:journalctl@ceph.mds.l.smithi161.stdout:Sep 21 03:43:33 smithi161 podman[63079]: 2024-09-21 03:43:33.990932078 +0000 UTC m=+0.118105732 container create 988b75038bf2f4bad8a0510d1676e5ba920c14b4f5483640bdd32b9c274ff8c5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-51116d5c-77ca-11ef-baf5-efdc52797490-mds-l, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, ceph=True, GIT_CLEAN=True) 2024-09-21T03:43:34.256 INFO:journalctl@ceph.mds.l.smithi161.stdout:Sep 21 03:43:34 smithi161 podman[63079]: 2024-09-21 03:43:34.218294928 +0000 UTC m=+0.345468579 container init 988b75038bf2f4bad8a0510d1676e5ba920c14b4f5483640bdd32b9c274ff8c5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-51116d5c-77ca-11ef-baf5-efdc52797490-mds-l, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS) 2024-09-21T03:43:34.256 INFO:journalctl@ceph.mds.l.smithi161.stdout:Sep 21 03:43:34 smithi161 podman[63079]: 2024-09-21 03:43:34.221164572 +0000 UTC m=+0.348338226 container start 988b75038bf2f4bad8a0510d1676e5ba920c14b4f5483640bdd32b9c274ff8c5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-51116d5c-77ca-11ef-baf5-efdc52797490-mds-l, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23) 2024-09-21T03:43:34.718 INFO:journalctl@ceph.mds.l.smithi161.stdout:Sep 21 03:43:34 smithi161 ceph-mds[63098]: -- 172.21.15.161:0/821239180 <== mon.1 v2:172.21.15.161:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5626be39f860 con 0x5626c2547600 2024-09-21T03:43:34.718 INFO:journalctl@ceph.mds.l.smithi161.stdout:Sep 21 03:43:34 smithi161 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-mds-l[63094]: starting mds.l at 2024-09-21T03:43:34.718 INFO:journalctl@ceph.mds.l.smithi161.stdout:Sep 21 03:43:34 smithi161 bash[63079]: 988b75038bf2f4bad8a0510d1676e5ba920c14b4f5483640bdd32b9c274ff8c5 2024-09-21T03:43:34.718 INFO:journalctl@ceph.mds.l.smithi161.stdout:Sep 21 03:43:34 smithi161 systemd[1]: Started Ceph mds.l for 51116d5c-77ca-11ef-baf5-efdc52797490. 2024-09-21T03:43:35.475 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:43:35 smithi150 ceph-mon[34289]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T03:43:35.475 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:43:35 smithi150 ceph-mon[34289]: mds.? [v2:172.21.15.161:6832/2963990654,v1:172.21.15.161:6833/2963990654] up:boot 2024-09-21T03:43:35.475 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:43:35 smithi150 ceph-mon[34289]: fsmap 1 up:standby 2024-09-21T03:43:35.475 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:43:35 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "l"} : dispatch 2024-09-21T03:43:35.475 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:43:35 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:43:35.475 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:43:35 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:43:35.475 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:43:35 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:43:35.475 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:43:35 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T03:43:35.475 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:43:35 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T03:43:35.475 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:43:35 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:43:35.584 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:43:35 smithi039 ceph-mon[31211]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T03:43:35.584 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:43:35 smithi039 ceph-mon[31211]: mds.? [v2:172.21.15.161:6832/2963990654,v1:172.21.15.161:6833/2963990654] up:boot 2024-09-21T03:43:35.584 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:43:35 smithi039 ceph-mon[31211]: fsmap 1 up:standby 2024-09-21T03:43:35.584 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:43:35 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "l"} : dispatch 2024-09-21T03:43:35.584 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:43:35 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:43:35.584 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:43:35 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:43:35.584 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:43:35 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:43:35.584 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:43:35 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T03:43:35.584 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:43:35 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T03:43:35.584 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:43:35 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:43:35.718 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:43:35 smithi161 ceph-mon[34533]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T03:43:35.718 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:43:35 smithi161 ceph-mon[34533]: mds.? [v2:172.21.15.161:6832/2963990654,v1:172.21.15.161:6833/2963990654] up:boot 2024-09-21T03:43:35.718 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:43:35 smithi161 ceph-mon[34533]: fsmap 1 up:standby 2024-09-21T03:43:35.718 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:43:35 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "l"} : dispatch 2024-09-21T03:43:35.718 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:43:35 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:43:35.718 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:43:35 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:43:35.718 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:43:35 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:43:35.718 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:43:35 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T03:43:35.718 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:43:35 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T03:43:35.718 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:43:35 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:43:36.176 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 51116d5c-77ca-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T03:43:36.448 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/51116d5c-77ca-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:43:36.475 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:43:36 smithi150 ceph-mon[34289]: Deploying daemon mds.k on smithi150 2024-09-21T03:43:36.475 INFO:journalctl@ceph.mds.k.smithi150.stdout:Sep 21 03:43:36 smithi150 systemd[1]: Starting Ceph mds.k for 51116d5c-77ca-11ef-baf5-efdc52797490... 2024-09-21T03:43:36.568 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:43:36 smithi039 ceph-mon[31211]: Deploying daemon mds.k on smithi150 2024-09-21T03:43:36.717 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:43:36 smithi161 ceph-mon[34533]: Deploying daemon mds.k on smithi150 2024-09-21T03:43:36.757 INFO:journalctl@ceph.mds.k.smithi150.stdout:Sep 21 03:43:36 smithi150 podman[62683]: 2024-09-21 03:43:36.400076687 +0000 UTC m=+0.015754859 image pull ddce307e1f3263fb077b4f3902e15660050631dad4d2823a20ee3f72bf18d977 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262 2024-09-21T03:43:36.758 INFO:journalctl@ceph.mds.k.smithi150.stdout:Sep 21 03:43:36 smithi150 podman[62683]: 2024-09-21 03:43:36.520630438 +0000 UTC m=+0.136308612 container create f4cfa3dbe0d41529e139e9e981c737f90ad26426d3bd1b97d7133136c0966574 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-51116d5c-77ca-11ef-baf5-efdc52797490-mds-k, ceph=True, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD) 2024-09-21T03:43:36.758 INFO:journalctl@ceph.mds.k.smithi150.stdout:Sep 21 03:43:36 smithi150 podman[62683]: 2024-09-21 03:43:36.75636098 +0000 UTC m=+0.372039150 container init f4cfa3dbe0d41529e139e9e981c737f90ad26426d3bd1b97d7133136c0966574 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-51116d5c-77ca-11ef-baf5-efdc52797490-mds-k, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.build-date=20240731, ceph=True, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23) 2024-09-21T03:43:36.921 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:36.893+0000 7ff93089c640 1 -- 172.21.15.39:0/3769960775 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff92c108d10 msgr2=0x7ff92c10f540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:36.921 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:36.893+0000 7ff93089c640 1 --2- 172.21.15.39:0/3769960775 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff92c108d10 0x7ff92c10f540 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7ff924009ca0 tx=0x7ff92402a8c0 comp rx=0 tx=0).stop 2024-09-21T03:43:36.921 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:36.894+0000 7ff93089c640 1 -- 172.21.15.39:0/3769960775 shutdown_connections 2024-09-21T03:43:36.922 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:36.894+0000 7ff93089c640 1 --2- 172.21.15.39:0/3769960775 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff92c108d10 0x7ff92c10f540 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:36.922 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:36.894+0000 7ff93089c640 1 --2- 172.21.15.39:0/3769960775 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff92c1083c0 0x7ff92c1087d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:36.922 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:36.894+0000 7ff93089c640 1 --2- 172.21.15.39:0/3769960775 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff92c102a80 0x7ff92c102e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:36.922 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:36.894+0000 7ff93089c640 1 -- 172.21.15.39:0/3769960775 >> 172.21.15.39:0/3769960775 conn(0x7ff92c079740 msgr2=0x7ff92c079b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:43:36.922 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:36.894+0000 7ff93089c640 1 -- 172.21.15.39:0/3769960775 shutdown_connections 2024-09-21T03:43:36.922 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:36.894+0000 7ff93089c640 1 -- 172.21.15.39:0/3769960775 wait complete. 2024-09-21T03:43:36.922 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:36.894+0000 7ff93089c640 1 Processor -- start 2024-09-21T03:43:36.922 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:36.894+0000 7ff93089c640 1 -- start start 2024-09-21T03:43:36.922 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:36.895+0000 7ff93089c640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff92c102a80 0x7ff92c0ff690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:36.922 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:36.895+0000 7ff93089c640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff92c1083c0 0x7ff92c0ffbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:36.922 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:36.895+0000 7ff93089c640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff92c108d10 0x7ff92c100110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:36.922 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:36.895+0000 7ff93089c640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7ff92c112980 con 0x7ff92c108d10 2024-09-21T03:43:36.922 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:36.895+0000 7ff93089c640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7ff92c112800 con 0x7ff92c102a80 2024-09-21T03:43:36.922 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:36.895+0000 7ff93089c640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7ff92c112b00 con 0x7ff92c1083c0 2024-09-21T03:43:36.923 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:36.895+0000 7ff92ad9d640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff92c108d10 0x7ff92c100110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:36.923 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:36.895+0000 7ff92ad9d640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff92c108d10 0x7ff92c100110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:53708/0 (socket says 172.21.15.39:53708) 2024-09-21T03:43:36.923 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:36.895+0000 7ff92ad9d640 1 -- 172.21.15.39:0/1599560988 learned_addr learned my addr 172.21.15.39:0/1599560988 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:43:36.923 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:36.895+0000 7ff92a59c640 1 --2- 172.21.15.39:0/1599560988 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff92c102a80 0x7ff92c0ff690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:36.923 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:36.895+0000 7ff923fff640 1 --2- 172.21.15.39:0/1599560988 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff92c1083c0 0x7ff92c0ffbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:36.923 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:36.895+0000 7ff92ad9d640 1 -- 172.21.15.39:0/1599560988 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff92c1083c0 msgr2=0x7ff92c0ffbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:36.923 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:36.895+0000 7ff92ad9d640 1 --2- 172.21.15.39:0/1599560988 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff92c1083c0 0x7ff92c0ffbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:36.923 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:36.895+0000 7ff92ad9d640 1 -- 172.21.15.39:0/1599560988 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff92c102a80 msgr2=0x7ff92c0ff690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:36.923 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:36.895+0000 7ff92ad9d640 1 --2- 172.21.15.39:0/1599560988 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff92c102a80 0x7ff92c0ff690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:36.923 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:36.895+0000 7ff92ad9d640 1 -- 172.21.15.39:0/1599560988 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff92c1a8670 con 0x7ff92c108d10 2024-09-21T03:43:36.923 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:36.895+0000 7ff92ad9d640 1 --2- 172.21.15.39:0/1599560988 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff92c108d10 0x7ff92c100110 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7ff924005cc0 tx=0x7ff92402f380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:43:36.923 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:36.896+0000 7ff9237fe640 1 -- 172.21.15.39:0/1599560988 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff924003bf0 con 0x7ff92c108d10 2024-09-21T03:43:36.923 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:36.896+0000 7ff9237fe640 1 -- 172.21.15.39:0/1599560988 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff924003d90 con 0x7ff92c108d10 2024-09-21T03:43:36.923 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:36.896+0000 7ff93089c640 1 -- 172.21.15.39:0/1599560988 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff92c1a8900 con 0x7ff92c108d10 2024-09-21T03:43:36.923 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:36.896+0000 7ff9237fe640 1 -- 172.21.15.39:0/1599560988 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff92402fb20 con 0x7ff92c108d10 2024-09-21T03:43:36.924 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:36.896+0000 7ff93089c640 1 -- 172.21.15.39:0/1599560988 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff92c1a8e40 con 0x7ff92c108d10 2024-09-21T03:43:36.924 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:36.897+0000 7ff93089c640 1 -- 172.21.15.39:0/1599560988 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff92c103d00 con 0x7ff92c108d10 2024-09-21T03:43:36.924 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:36.899+0000 7ff9237fe640 1 -- 172.21.15.39:0/1599560988 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 21) ==== 154529+0+0 (secure 0 0 0) 0x7ff924004580 con 0x7ff92c108d10 2024-09-21T03:43:36.924 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:36.900+0000 7ff9237fe640 1 --2- 172.21.15.39:0/1599560988 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7ff9180b6560 0x7ff9180b8a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:36.924 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:36.900+0000 7ff92a59c640 1 --2- 172.21.15.39:0/1599560988 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7ff9180b6560 0x7ff9180b8a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:36.924 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:36.900+0000 7ff9237fe640 1 -- 172.21.15.39:0/1599560988 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 7342+0+0 (secure 0 0 0) 0x7ff924108440 con 0x7ff92c108d10 2024-09-21T03:43:36.924 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:36.901+0000 7ff92a59c640 1 --2- 172.21.15.39:0/1599560988 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7ff9180b6560 0x7ff9180b8a70 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7ff92c1010e0 tx=0x7ff91c00a2d0 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:43:36.924 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:36.904+0000 7ff9237fe640 1 -- 172.21.15.39:0/1599560988 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7ff9240cf0a0 con 0x7ff92c108d10 2024-09-21T03:43:37.141 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:37.138+0000 7ff93089c640 1 -- 172.21.15.39:0/1599560988 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff92c100990 con 0x7ff92c108d10 2024-09-21T03:43:37.142 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:37.139+0000 7ff9237fe640 1 -- 172.21.15.39:0/1599560988 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 2 v2) ==== 93+0+1438 (secure 0 0 0) 0x7ff9240098b0 con 0x7ff92c108d10 2024-09-21T03:43:37.142 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T03:43:37.142 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":2,"btime":"2024-09-21T03:43:34:259902+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24601,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6833/2963990654","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":2963990654},{"type":"v1","addr":"172.21.15.161:6833","nonce":2963990654}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-09-21T03:43:37.142 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 2 2024-09-21T03:43:37.144 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:37.141+0000 7ff93089c640 1 -- 172.21.15.39:0/1599560988 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7ff9180b6560 msgr2=0x7ff9180b8a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:37.144 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:37.141+0000 7ff93089c640 1 --2- 172.21.15.39:0/1599560988 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7ff9180b6560 0x7ff9180b8a70 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7ff92c1010e0 tx=0x7ff91c00a2d0 comp rx=0 tx=0).stop 2024-09-21T03:43:37.144 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:37.141+0000 7ff93089c640 1 -- 172.21.15.39:0/1599560988 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff92c108d10 msgr2=0x7ff92c100110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:37.145 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:37.141+0000 7ff93089c640 1 --2- 172.21.15.39:0/1599560988 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff92c108d10 0x7ff92c100110 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7ff924005cc0 tx=0x7ff92402f380 comp rx=0 tx=0).stop 2024-09-21T03:43:37.145 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:37.142+0000 7ff93089c640 1 -- 172.21.15.39:0/1599560988 shutdown_connections 2024-09-21T03:43:37.145 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:37.142+0000 7ff93089c640 1 --2- 172.21.15.39:0/1599560988 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7ff9180b6560 0x7ff9180b8a70 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:37.145 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:37.142+0000 7ff93089c640 1 --2- 172.21.15.39:0/1599560988 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff92c108d10 0x7ff92c100110 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:37.145 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:37.142+0000 7ff93089c640 1 --2- 172.21.15.39:0/1599560988 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff92c1083c0 0x7ff92c0ffbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:37.145 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:37.142+0000 7ff93089c640 1 --2- 172.21.15.39:0/1599560988 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff92c102a80 0x7ff92c0ff690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:37.145 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:37.142+0000 7ff93089c640 1 -- 172.21.15.39:0/1599560988 >> 172.21.15.39:0/1599560988 conn(0x7ff92c079740 msgr2=0x7ff92c06bf20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:43:37.145 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:37.142+0000 7ff93089c640 1 -- 172.21.15.39:0/1599560988 shutdown_connections 2024-09-21T03:43:37.145 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:37.142+0000 7ff93089c640 1 -- 172.21.15.39:0/1599560988 wait complete. 2024-09-21T03:43:37.224 INFO:journalctl@ceph.mds.k.smithi150.stdout:Sep 21 03:43:36 smithi150 podman[62683]: 2024-09-21 03:43:36.759220979 +0000 UTC m=+0.374899159 container start f4cfa3dbe0d41529e139e9e981c737f90ad26426d3bd1b97d7133136c0966574 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-51116d5c-77ca-11ef-baf5-efdc52797490-mds-k, ceph=True, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux ) 2024-09-21T03:43:37.225 INFO:journalctl@ceph.mds.k.smithi150.stdout:Sep 21 03:43:36 smithi150 ceph-mds[62702]: -- 172.21.15.150:0/1089012560 <== mon.2 v2:172.21.15.150:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5624d966b860 con 0x5624dd813600 2024-09-21T03:43:37.225 INFO:journalctl@ceph.mds.k.smithi150.stdout:Sep 21 03:43:36 smithi150 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-mds-k[62698]: starting mds.k at 2024-09-21T03:43:37.225 INFO:journalctl@ceph.mds.k.smithi150.stdout:Sep 21 03:43:36 smithi150 bash[62683]: f4cfa3dbe0d41529e139e9e981c737f90ad26426d3bd1b97d7133136c0966574 2024-09-21T03:43:37.225 INFO:journalctl@ceph.mds.k.smithi150.stdout:Sep 21 03:43:36 smithi150 systemd[1]: Started Ceph mds.k for 51116d5c-77ca-11ef-baf5-efdc52797490. 2024-09-21T03:43:37.583 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:43:37 smithi039 ceph-mon[31211]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T03:43:37.584 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:43:37 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:43:37.584 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:43:37 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:43:37.584 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:43:37 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:43:37.584 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:43:37 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T03:43:37.584 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:43:37 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T03:43:37.584 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:43:37 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:43:37.584 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:43:37 smithi039 ceph-mon[31211]: from='client.? 172.21.15.39:0/1599560988' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:43:37.718 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:43:37 smithi161 ceph-mon[34533]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T03:43:37.718 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:43:37 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:43:37.718 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:43:37 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:43:37.718 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:43:37 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:43:37.718 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:43:37 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T03:43:37.718 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:43:37 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T03:43:37.718 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:43:37 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:43:37.718 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:43:37 smithi161 ceph-mon[34533]: from='client.? 172.21.15.39:0/1599560988' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:43:37.724 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:43:37 smithi150 ceph-mon[34289]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T03:43:37.725 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:43:37 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:43:37.725 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:43:37 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:43:37.725 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:43:37 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:43:37.725 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:43:37 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T03:43:37.725 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:43:37 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T03:43:37.725 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:43:37 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:43:37.725 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:43:37 smithi150 ceph-mon[34289]: from='client.? 172.21.15.39:0/1599560988' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:43:38.584 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:43:38 smithi039 ceph-mon[31211]: Deploying daemon mds.j on smithi039 2024-09-21T03:43:38.584 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:43:38 smithi039 ceph-mon[31211]: mds.? [v2:172.21.15.150:6832/3114643217,v1:172.21.15.150:6833/3114643217] up:boot 2024-09-21T03:43:38.584 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:43:38 smithi039 ceph-mon[31211]: fsmap 2 up:standby 2024-09-21T03:43:38.584 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:43:38 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "k"} : dispatch 2024-09-21T03:43:38.619 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:43:38 smithi161 ceph-mon[34533]: Deploying daemon mds.j on smithi039 2024-09-21T03:43:38.619 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:43:38 smithi161 ceph-mon[34533]: mds.? [v2:172.21.15.150:6832/3114643217,v1:172.21.15.150:6833/3114643217] up:boot 2024-09-21T03:43:38.619 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:43:38 smithi161 ceph-mon[34533]: fsmap 2 up:standby 2024-09-21T03:43:38.619 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:43:38 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "k"} : dispatch 2024-09-21T03:43:38.724 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:43:38 smithi150 ceph-mon[34289]: Deploying daemon mds.j on smithi039 2024-09-21T03:43:38.725 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:43:38 smithi150 ceph-mon[34289]: mds.? [v2:172.21.15.150:6832/3114643217,v1:172.21.15.150:6833/3114643217] up:boot 2024-09-21T03:43:38.725 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:43:38 smithi150 ceph-mon[34289]: fsmap 2 up:standby 2024-09-21T03:43:38.725 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:43:38 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "k"} : dispatch 2024-09-21T03:43:39.334 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:43:39 smithi039 ceph-mon[31211]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T03:43:39.474 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:43:39 smithi150 ceph-mon[34289]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T03:43:39.718 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:43:39 smithi161 ceph-mon[34533]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T03:43:39.802 INFO:journalctl@ceph.mds.j.smithi039.stdout:Sep 21 03:43:39 smithi039 systemd[1]: Starting Ceph mds.j for 51116d5c-77ca-11ef-baf5-efdc52797490... 2024-09-21T03:43:39.915 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 51116d5c-77ca-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T03:43:40.067 INFO:journalctl@ceph.mds.j.smithi039.stdout:Sep 21 03:43:39 smithi039 podman[72051]: 2024-09-21 03:43:39.80052995 +0000 UTC m=+0.113015653 container create ae337640ab42027fb7f045b75f267ec7469d17b989cf80ed21a54070ca3153e9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-51116d5c-77ca-11ef-baf5-efdc52797490-mds-j, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, ceph=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-21T03:43:40.067 INFO:journalctl@ceph.mds.j.smithi039.stdout:Sep 21 03:43:39 smithi039 podman[72051]: 2024-09-21 03:43:39.70393666 +0000 UTC m=+0.016422381 image pull ddce307e1f3263fb077b4f3902e15660050631dad4d2823a20ee3f72bf18d977 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262 2024-09-21T03:43:40.067 INFO:journalctl@ceph.mds.j.smithi039.stdout:Sep 21 03:43:40 smithi039 podman[72051]: 2024-09-21 03:43:40.028041759 +0000 UTC m=+0.340527471 container init ae337640ab42027fb7f045b75f267ec7469d17b989cf80ed21a54070ca3153e9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-51116d5c-77ca-11ef-baf5-efdc52797490-mds-j, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.build-date=20240731, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS) 2024-09-21T03:43:40.067 INFO:journalctl@ceph.mds.j.smithi039.stdout:Sep 21 03:43:40 smithi039 podman[72051]: 2024-09-21 03:43:40.031404536 +0000 UTC m=+0.343890251 container start ae337640ab42027fb7f045b75f267ec7469d17b989cf80ed21a54070ca3153e9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-51116d5c-77ca-11ef-baf5-efdc52797490-mds-j, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.license=GPLv2) 2024-09-21T03:43:40.200 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/51116d5c-77ca-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:43:40.323 INFO:journalctl@ceph.mds.j.smithi039.stdout:Sep 21 03:43:40 smithi039 ceph-mds[72088]: -- 172.21.15.39:0/2895108379 <== mon.0 v2:172.21.15.39:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5592450fb860 con 0x5592492a3600 2024-09-21T03:43:40.323 INFO:journalctl@ceph.mds.j.smithi039.stdout:Sep 21 03:43:40 smithi039 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-mds-j[72066]: starting mds.j at 2024-09-21T03:43:40.323 INFO:journalctl@ceph.mds.j.smithi039.stdout:Sep 21 03:43:40 smithi039 bash[72051]: ae337640ab42027fb7f045b75f267ec7469d17b989cf80ed21a54070ca3153e9 2024-09-21T03:43:40.323 INFO:journalctl@ceph.mds.j.smithi039.stdout:Sep 21 03:43:40 smithi039 systemd[1]: Started Ceph mds.j for 51116d5c-77ca-11ef-baf5-efdc52797490. 2024-09-21T03:43:40.733 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:40.724+0000 7f8e1da1d640 1 -- 172.21.15.39:0/1041424286 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8e18108730 msgr2=0x7f8e18108b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:40.734 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:40.724+0000 7f8e1da1d640 1 --2- 172.21.15.39:0/1041424286 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8e18108730 0x7f8e18108b40 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f8e10009930 tx=0x7f8e1002a420 comp rx=0 tx=0).stop 2024-09-21T03:43:40.734 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:40.724+0000 7f8e1da1d640 1 -- 172.21.15.39:0/1041424286 shutdown_connections 2024-09-21T03:43:40.734 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:40.724+0000 7f8e1da1d640 1 --2- 172.21.15.39:0/1041424286 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8e18102770 0x7f8e181105a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:40.734 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:40.724+0000 7f8e1da1d640 1 --2- 172.21.15.39:0/1041424286 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8e18101e20 0x7f8e18102230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:40.734 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:40.724+0000 7f8e1da1d640 1 --2- 172.21.15.39:0/1041424286 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8e18108730 0x7f8e18108b40 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:40.734 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:40.724+0000 7f8e1da1d640 1 -- 172.21.15.39:0/1041424286 >> 172.21.15.39:0/1041424286 conn(0x7f8e18077cc0 msgr2=0x7f8e18078100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:43:40.734 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:40.724+0000 7f8e1da1d640 1 -- 172.21.15.39:0/1041424286 shutdown_connections 2024-09-21T03:43:40.734 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:40.724+0000 7f8e1da1d640 1 -- 172.21.15.39:0/1041424286 wait complete. 2024-09-21T03:43:40.734 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:40.725+0000 7f8e1da1d640 1 Processor -- start 2024-09-21T03:43:40.734 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:40.725+0000 7f8e1da1d640 1 -- start start 2024-09-21T03:43:40.734 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:40.725+0000 7f8e1da1d640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8e18101e20 0x7f8e1819a4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:40.734 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:40.725+0000 7f8e1da1d640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8e18102770 0x7f8e1819aa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:40.735 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:40.726+0000 7f8e1da1d640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8e18108730 0x7f8e1819fd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:40.735 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:40.726+0000 7f8e1da1d640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f8e18113ab0 con 0x7f8e18101e20 2024-09-21T03:43:40.735 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:40.726+0000 7f8e1da1d640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f8e18113930 con 0x7f8e18102770 2024-09-21T03:43:40.735 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:40.726+0000 7f8e1da1d640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f8e18113c30 con 0x7f8e18108730 2024-09-21T03:43:40.735 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:40.726+0000 7f8e16ffd640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8e18101e20 0x7f8e1819a4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:40.735 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:40.726+0000 7f8e16ffd640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8e18101e20 0x7f8e1819a4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:52160/0 (socket says 172.21.15.39:52160) 2024-09-21T03:43:40.735 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:40.726+0000 7f8e16ffd640 1 -- 172.21.15.39:0/1832026658 learned_addr learned my addr 172.21.15.39:0/1832026658 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:43:40.735 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:40.726+0000 7f8e177fe640 1 --2- 172.21.15.39:0/1832026658 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8e18108730 0x7f8e1819fd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:40.735 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:40.726+0000 7f8e167fc640 1 --2- 172.21.15.39:0/1832026658 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8e18102770 0x7f8e1819aa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:40.735 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:40.726+0000 7f8e16ffd640 1 -- 172.21.15.39:0/1832026658 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8e18108730 msgr2=0x7f8e1819fd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:40.735 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:40.726+0000 7f8e16ffd640 1 --2- 172.21.15.39:0/1832026658 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8e18108730 0x7f8e1819fd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:40.735 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:40.726+0000 7f8e16ffd640 1 -- 172.21.15.39:0/1832026658 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8e18102770 msgr2=0x7f8e1819aa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:40.735 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:40.726+0000 7f8e16ffd640 1 --2- 172.21.15.39:0/1832026658 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8e18102770 0x7f8e1819aa60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:40.735 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:40.726+0000 7f8e16ffd640 1 -- 172.21.15.39:0/1832026658 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8e181a0430 con 0x7f8e18101e20 2024-09-21T03:43:40.736 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:40.726+0000 7f8e16ffd640 1 --2- 172.21.15.39:0/1832026658 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8e18101e20 0x7f8e1819a4c0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f8e10004770 tx=0x7f8e100042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:43:40.736 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:40.726+0000 7f8e1ca1b640 1 -- 172.21.15.39:0/1832026658 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e1003f070 con 0x7f8e18101e20 2024-09-21T03:43:40.736 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:40.727+0000 7f8e1ca1b640 1 -- 172.21.15.39:0/1832026658 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8e1003b460 con 0x7f8e18101e20 2024-09-21T03:43:40.736 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:40.727+0000 7f8e1da1d640 1 -- 172.21.15.39:0/1832026658 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8e181a06c0 con 0x7f8e18101e20 2024-09-21T03:43:40.736 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:40.727+0000 7f8e1ca1b640 1 -- 172.21.15.39:0/1832026658 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e10035070 con 0x7f8e18101e20 2024-09-21T03:43:40.736 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:40.727+0000 7f8e1da1d640 1 -- 172.21.15.39:0/1832026658 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8e181a6e80 con 0x7f8e18101e20 2024-09-21T03:43:40.736 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:40.728+0000 7f8e1da1d640 1 -- 172.21.15.39:0/1832026658 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8e1806b830 con 0x7f8e18101e20 2024-09-21T03:43:40.736 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:40.729+0000 7f8e1ca1b640 1 -- 172.21.15.39:0/1832026658 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 21) ==== 154529+0+0 (secure 0 0 0) 0x7f8e1003b8d0 con 0x7f8e18101e20 2024-09-21T03:43:40.736 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:40.730+0000 7f8e1ca1b640 1 --2- 172.21.15.39:0/1832026658 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f8e040b6560 0x7f8e040b8a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:40.736 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:40.730+0000 7f8e167fc640 1 --2- 172.21.15.39:0/1832026658 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f8e040b6560 0x7f8e040b8a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:40.736 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:40.730+0000 7f8e1ca1b640 1 -- 172.21.15.39:0/1832026658 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 7342+0+0 (secure 0 0 0) 0x7f8e10107570 con 0x7f8e18101e20 2024-09-21T03:43:40.736 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:40.731+0000 7f8e167fc640 1 --2- 172.21.15.39:0/1832026658 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f8e040b6560 0x7f8e040b8a70 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f8e18106090 tx=0x7f8e0000b390 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:43:40.737 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:40.733+0000 7f8e1ca1b640 1 -- 172.21.15.39:0/1832026658 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f8e100d25b0 con 0x7f8e18101e20 2024-09-21T03:43:40.966 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:40.963+0000 7f8e1da1d640 1 -- 172.21.15.39:0/1832026658 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f8e18102230 con 0x7f8e18101e20 2024-09-21T03:43:40.966 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:40.964+0000 7f8e1ca1b640 1 -- 172.21.15.39:0/1832026658 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 4 v4) ==== 93+0+3088 (secure 0 0 0) 0x7f8e100d74f0 con 0x7f8e18101e20 2024-09-21T03:43:40.967 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T03:43:40.967 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":4,"btime":"2024-09-21T03:43:40:275721+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14790,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6835/508995295","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":508995295},{"type":"v1","addr":"172.21.15.39:6835","nonce":508995295}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24599,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6833/3114643217","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":3114643217},{"type":"v1","addr":"172.21.15.150:6833","nonce":3114643217}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24601,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6833/2963990654","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":2963990654},{"type":"v1","addr":"172.21.15.161:6833","nonce":2963990654}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-09-21T03:43:40.967 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 4 2024-09-21T03:43:40.970 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:40.967+0000 7f8e1da1d640 1 -- 172.21.15.39:0/1832026658 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f8e040b6560 msgr2=0x7f8e040b8a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:40.970 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:40.967+0000 7f8e1da1d640 1 --2- 172.21.15.39:0/1832026658 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f8e040b6560 0x7f8e040b8a70 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f8e18106090 tx=0x7f8e0000b390 comp rx=0 tx=0).stop 2024-09-21T03:43:40.970 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:40.967+0000 7f8e1da1d640 1 -- 172.21.15.39:0/1832026658 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8e18101e20 msgr2=0x7f8e1819a4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:40.970 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:40.967+0000 7f8e1da1d640 1 --2- 172.21.15.39:0/1832026658 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8e18101e20 0x7f8e1819a4c0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f8e10004770 tx=0x7f8e100042f0 comp rx=0 tx=0).stop 2024-09-21T03:43:40.970 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:40.967+0000 7f8e1da1d640 1 -- 172.21.15.39:0/1832026658 shutdown_connections 2024-09-21T03:43:40.970 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:40.967+0000 7f8e1da1d640 1 --2- 172.21.15.39:0/1832026658 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f8e040b6560 0x7f8e040b8a70 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:40.970 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:40.967+0000 7f8e1da1d640 1 --2- 172.21.15.39:0/1832026658 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8e18108730 0x7f8e1819fd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:40.970 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:40.967+0000 7f8e1da1d640 1 --2- 172.21.15.39:0/1832026658 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8e18102770 0x7f8e1819aa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:40.970 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:40.967+0000 7f8e1da1d640 1 --2- 172.21.15.39:0/1832026658 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8e18101e20 0x7f8e1819a4c0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:40.970 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:40.967+0000 7f8e1da1d640 1 -- 172.21.15.39:0/1832026658 >> 172.21.15.39:0/1832026658 conn(0x7f8e18077cc0 msgr2=0x7f8e18107590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:43:40.970 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:40.967+0000 7f8e1da1d640 1 -- 172.21.15.39:0/1832026658 shutdown_connections 2024-09-21T03:43:40.970 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:40.967+0000 7f8e1da1d640 1 -- 172.21.15.39:0/1832026658 wait complete. 2024-09-21T03:43:41.468 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:43:41 smithi161 ceph-mon[34533]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T03:43:41.468 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:43:41 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:43:41.468 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:43:41 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:43:41.468 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:43:41 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:43:41.468 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:43:41 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T03:43:41.468 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:43:41 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T03:43:41.468 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:43:41 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:43:41.468 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:43:41 smithi161 ceph-mon[34533]: Deploying daemon mds.d on smithi039 2024-09-21T03:43:41.468 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:43:41 smithi161 ceph-mon[34533]: mds.? [v2:172.21.15.39:6834/508995295,v1:172.21.15.39:6835/508995295] up:boot 2024-09-21T03:43:41.468 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:43:41 smithi161 ceph-mon[34533]: fsmap 3 up:standby 2024-09-21T03:43:41.469 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:43:41 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "j"} : dispatch 2024-09-21T03:43:41.469 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:43:41 smithi161 ceph-mon[34533]: from='client.? 172.21.15.39:0/1832026658' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:43:41.475 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:43:41 smithi150 ceph-mon[34289]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T03:43:41.475 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:43:41 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:43:41.475 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:43:41 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:43:41.475 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:43:41 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:43:41.475 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:43:41 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T03:43:41.475 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:43:41 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T03:43:41.475 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:43:41 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:43:41.475 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:43:41 smithi150 ceph-mon[34289]: Deploying daemon mds.d on smithi039 2024-09-21T03:43:41.475 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:43:41 smithi150 ceph-mon[34289]: mds.? [v2:172.21.15.39:6834/508995295,v1:172.21.15.39:6835/508995295] up:boot 2024-09-21T03:43:41.476 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:43:41 smithi150 ceph-mon[34289]: fsmap 3 up:standby 2024-09-21T03:43:41.476 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:43:41 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "j"} : dispatch 2024-09-21T03:43:41.476 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:43:41 smithi150 ceph-mon[34289]: from='client.? 172.21.15.39:0/1832026658' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:43:41.584 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:43:41 smithi039 ceph-mon[31211]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T03:43:41.584 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:43:41 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:43:41.584 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:43:41 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:43:41.584 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:43:41 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:43:41.584 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:43:41 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T03:43:41.584 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:43:41 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T03:43:41.584 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:43:41 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:43:41.584 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:43:41 smithi039 ceph-mon[31211]: Deploying daemon mds.d on smithi039 2024-09-21T03:43:41.584 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:43:41 smithi039 ceph-mon[31211]: mds.? [v2:172.21.15.39:6834/508995295,v1:172.21.15.39:6835/508995295] up:boot 2024-09-21T03:43:41.585 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:43:41 smithi039 ceph-mon[31211]: fsmap 3 up:standby 2024-09-21T03:43:41.585 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:43:41 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "j"} : dispatch 2024-09-21T03:43:41.585 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:43:41 smithi039 ceph-mon[31211]: from='client.? 172.21.15.39:0/1832026658' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:43:43.438 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:43:43 smithi039 ceph-mon[31211]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T03:43:43.467 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:43:43 smithi161 ceph-mon[34533]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T03:43:43.474 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:43:43 smithi150 ceph-mon[34289]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T03:43:43.721 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 51116d5c-77ca-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T03:43:43.981 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/51116d5c-77ca-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:43:44.490 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:44.487+0000 7f1b4ed33640 1 -- 172.21.15.39:0/940963241 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1b48103270 msgr2=0x7f1b48103680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:44.490 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:44.487+0000 7f1b4ed33640 1 --2- 172.21.15.39:0/940963241 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1b48103270 0x7f1b48103680 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f1b3c009900 tx=0x7f1b3c02a410 comp rx=0 tx=0).stop 2024-09-21T03:43:44.490 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:44.487+0000 7f1b4ed33640 1 -- 172.21.15.39:0/940963241 shutdown_connections 2024-09-21T03:43:44.490 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:44.487+0000 7f1b4ed33640 1 --2- 172.21.15.39:0/940963241 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1b48105b90 0x7f1b4810f5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:44.490 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:44.487+0000 7f1b4ed33640 1 --2- 172.21.15.39:0/940963241 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1b48105240 0x7f1b48105650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:44.490 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:44.487+0000 7f1b4ed33640 1 --2- 172.21.15.39:0/940963241 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1b48103270 0x7f1b48103680 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:44.490 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:44.487+0000 7f1b4ed33640 1 -- 172.21.15.39:0/940963241 >> 172.21.15.39:0/940963241 conn(0x7f1b48079ec0 msgr2=0x7f1b4807a2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:43:44.490 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:44.487+0000 7f1b4ed33640 1 -- 172.21.15.39:0/940963241 shutdown_connections 2024-09-21T03:43:44.490 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:44.487+0000 7f1b4ed33640 1 -- 172.21.15.39:0/940963241 wait complete. 2024-09-21T03:43:44.492 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:44.487+0000 7f1b4ed33640 1 Processor -- start 2024-09-21T03:43:44.492 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:44.488+0000 7f1b4ed33640 1 -- start start 2024-09-21T03:43:44.492 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:44.488+0000 7f1b4ed33640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1b48103270 0x7f1b4819a5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:44.492 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:44.488+0000 7f1b4ed33640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1b48105240 0x7f1b4819ab50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:44.492 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:44.488+0000 7f1b4ed33640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1b48105b90 0x7f1b4819fdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:44.492 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:44.488+0000 7f1b4ed33640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f1b48112a30 con 0x7f1b48105b90 2024-09-21T03:43:44.492 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:44.488+0000 7f1b4ed33640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f1b481128b0 con 0x7f1b48105240 2024-09-21T03:43:44.493 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:44.488+0000 7f1b4ed33640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f1b48112bb0 con 0x7f1b48103270 2024-09-21T03:43:44.493 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:44.488+0000 7f1b4cacf640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1b48103270 0x7f1b4819a5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:44.493 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:44.488+0000 7f1b47fff640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1b48105240 0x7f1b4819ab50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:44.493 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:44.488+0000 7f1b4cacf640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1b48103270 0x7f1b4819a5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.39:36602/0 (socket says 172.21.15.39:36602) 2024-09-21T03:43:44.493 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:44.488+0000 7f1b4cacf640 1 -- 172.21.15.39:0/2815379395 learned_addr learned my addr 172.21.15.39:0/2815379395 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:43:44.493 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:44.488+0000 7f1b4d2d0640 1 --2- 172.21.15.39:0/2815379395 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1b48105b90 0x7f1b4819fdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:44.493 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:44.488+0000 7f1b4d2d0640 1 -- 172.21.15.39:0/2815379395 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1b48103270 msgr2=0x7f1b4819a5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:44.493 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:44.488+0000 7f1b4d2d0640 1 --2- 172.21.15.39:0/2815379395 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1b48103270 0x7f1b4819a5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:44.493 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:44.488+0000 7f1b4d2d0640 1 -- 172.21.15.39:0/2815379395 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1b48105240 msgr2=0x7f1b4819ab50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:44.493 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:44.488+0000 7f1b4d2d0640 1 --2- 172.21.15.39:0/2815379395 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1b48105240 0x7f1b4819ab50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:44.493 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:44.488+0000 7f1b4d2d0640 1 -- 172.21.15.39:0/2815379395 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1b481a0520 con 0x7f1b48105b90 2024-09-21T03:43:44.493 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:44.489+0000 7f1b4d2d0640 1 --2- 172.21.15.39:0/2815379395 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1b48105b90 0x7f1b4819fdf0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f1b4000e940 tx=0x7f1b4000ee00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:43:44.493 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:44.489+0000 7f1b45ffb640 1 -- 172.21.15.39:0/2815379395 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b40004d20 con 0x7f1b48105b90 2024-09-21T03:43:44.493 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:44.489+0000 7f1b4ed33640 1 -- 172.21.15.39:0/2815379395 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1b481a0810 con 0x7f1b48105b90 2024-09-21T03:43:44.494 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:44.489+0000 7f1b45ffb640 1 -- 172.21.15.39:0/2815379395 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1b40005020 con 0x7f1b48105b90 2024-09-21T03:43:44.494 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:44.489+0000 7f1b45ffb640 1 -- 172.21.15.39:0/2815379395 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b400116d0 con 0x7f1b48105b90 2024-09-21T03:43:44.494 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:44.489+0000 7f1b4ed33640 1 -- 172.21.15.39:0/2815379395 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1b481a6f70 con 0x7f1b48105b90 2024-09-21T03:43:44.494 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:44.490+0000 7f1b4ed33640 1 -- 172.21.15.39:0/2815379395 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1b4819b6e0 con 0x7f1b48105b90 2024-09-21T03:43:44.494 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:44.492+0000 7f1b45ffb640 1 -- 172.21.15.39:0/2815379395 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 21) ==== 154529+0+0 (secure 0 0 0) 0x7f1b40005460 con 0x7f1b48105b90 2024-09-21T03:43:44.495 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:44.492+0000 7f1b45ffb640 1 --2- 172.21.15.39:0/2815379395 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f1b380b6560 0x7f1b380b8a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:44.495 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:44.492+0000 7f1b4cacf640 1 --2- 172.21.15.39:0/2815379395 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f1b380b6560 0x7f1b380b8a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:44.495 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:44.492+0000 7f1b45ffb640 1 -- 172.21.15.39:0/2815379395 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 7342+0+0 (secure 0 0 0) 0x7f1b400e7380 con 0x7f1b48105b90 2024-09-21T03:43:44.495 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:44.493+0000 7f1b4cacf640 1 --2- 172.21.15.39:0/2815379395 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f1b380b6560 0x7f1b380b8a70 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f1b3c031440 tx=0x7f1b3c0058d0 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:43:44.498 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:44.495+0000 7f1b45ffb640 1 -- 172.21.15.39:0/2815379395 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f1b40016030 con 0x7f1b48105b90 2024-09-21T03:43:44.727 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:44.724+0000 7f1b4ed33640 1 -- 172.21.15.39:0/2815379395 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1b4810bb10 con 0x7f1b48105b90 2024-09-21T03:43:44.727 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:44.725+0000 7f1b45ffb640 1 -- 172.21.15.39:0/2815379395 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 5 v5) ==== 93+0+3913 (secure 0 0 0) 0x7f1b4810bb10 con 0x7f1b48105b90 2024-09-21T03:43:44.728 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T03:43:44.728 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":5,"btime":"2024-09-21T03:43:44:189874+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14790,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6835/508995295","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":508995295},{"type":"v1","addr":"172.21.15.39:6835","nonce":508995295}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14802,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6837/3915912607","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":3915912607},{"type":"v1","addr":"172.21.15.39:6837","nonce":3915912607}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24599,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6833/3114643217","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":3114643217},{"type":"v1","addr":"172.21.15.150:6833","nonce":3114643217}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24601,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6833/2963990654","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":2963990654},{"type":"v1","addr":"172.21.15.161:6833","nonce":2963990654}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-09-21T03:43:44.728 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 5 2024-09-21T03:43:44.730 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:44.727+0000 7f1b4ed33640 1 -- 172.21.15.39:0/2815379395 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f1b380b6560 msgr2=0x7f1b380b8a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:44.730 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:44.727+0000 7f1b4ed33640 1 --2- 172.21.15.39:0/2815379395 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f1b380b6560 0x7f1b380b8a70 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f1b3c031440 tx=0x7f1b3c0058d0 comp rx=0 tx=0).stop 2024-09-21T03:43:44.730 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:44.727+0000 7f1b4ed33640 1 -- 172.21.15.39:0/2815379395 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1b48105b90 msgr2=0x7f1b4819fdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:44.730 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:44.727+0000 7f1b4ed33640 1 --2- 172.21.15.39:0/2815379395 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1b48105b90 0x7f1b4819fdf0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f1b4000e940 tx=0x7f1b4000ee00 comp rx=0 tx=0).stop 2024-09-21T03:43:44.730 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:44.727+0000 7f1b4ed33640 1 -- 172.21.15.39:0/2815379395 shutdown_connections 2024-09-21T03:43:44.730 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:44.727+0000 7f1b4ed33640 1 --2- 172.21.15.39:0/2815379395 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f1b380b6560 0x7f1b380b8a70 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:44.730 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:44.727+0000 7f1b4ed33640 1 --2- 172.21.15.39:0/2815379395 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1b48105b90 0x7f1b4819fdf0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:44.730 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:44.727+0000 7f1b4ed33640 1 --2- 172.21.15.39:0/2815379395 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1b48105240 0x7f1b4819ab50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:44.730 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:44.727+0000 7f1b4ed33640 1 --2- 172.21.15.39:0/2815379395 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1b48103270 0x7f1b4819a5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:44.730 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:44.727+0000 7f1b4ed33640 1 -- 172.21.15.39:0/2815379395 >> 172.21.15.39:0/2815379395 conn(0x7f1b48079ec0 msgr2=0x7f1b48107580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:43:44.730 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:44.727+0000 7f1b4ed33640 1 -- 172.21.15.39:0/2815379395 shutdown_connections 2024-09-21T03:43:44.730 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:44.727+0000 7f1b4ed33640 1 -- 172.21.15.39:0/2815379395 wait complete. 2024-09-21T03:43:44.968 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:43:44 smithi161 ceph-mon[34533]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T03:43:44.968 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:43:44 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:43:44.968 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:43:44 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:43:44.968 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:43:44 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:43:44.968 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:43:44 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T03:43:44.968 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:43:44 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T03:43:44.968 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:43:44 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:43:44.968 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:43:44 smithi161 ceph-mon[34533]: Deploying daemon mds.e on smithi150 2024-09-21T03:43:44.968 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:43:44 smithi161 ceph-mon[34533]: mds.? [v2:172.21.15.39:6836/3915912607,v1:172.21.15.39:6837/3915912607] up:boot 2024-09-21T03:43:44.969 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:43:44 smithi161 ceph-mon[34533]: fsmap 4 up:standby 2024-09-21T03:43:44.969 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:43:44 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "d"} : dispatch 2024-09-21T03:43:44.975 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:43:44 smithi150 ceph-mon[34289]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T03:43:44.975 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:43:44 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:43:44.975 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:43:44 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:43:44.975 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:43:44 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:43:44.975 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:43:44 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T03:43:44.975 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:43:44 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T03:43:44.975 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:43:44 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:43:44.975 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:43:44 smithi150 ceph-mon[34289]: Deploying daemon mds.e on smithi150 2024-09-21T03:43:44.975 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:43:44 smithi150 ceph-mon[34289]: mds.? [v2:172.21.15.39:6836/3915912607,v1:172.21.15.39:6837/3915912607] up:boot 2024-09-21T03:43:44.975 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:43:44 smithi150 ceph-mon[34289]: fsmap 4 up:standby 2024-09-21T03:43:44.976 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:43:44 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "d"} : dispatch 2024-09-21T03:43:45.084 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:43:44 smithi039 ceph-mon[31211]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T03:43:45.084 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:43:44 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:43:45.084 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:43:44 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:43:45.084 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:43:44 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:43:45.084 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:43:44 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T03:43:45.084 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:43:44 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T03:43:45.084 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:43:44 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:43:45.084 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:43:44 smithi039 ceph-mon[31211]: Deploying daemon mds.e on smithi150 2024-09-21T03:43:45.084 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:43:44 smithi039 ceph-mon[31211]: mds.? [v2:172.21.15.39:6836/3915912607,v1:172.21.15.39:6837/3915912607] up:boot 2024-09-21T03:43:45.084 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:43:44 smithi039 ceph-mon[31211]: fsmap 4 up:standby 2024-09-21T03:43:45.085 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:43:44 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "d"} : dispatch 2024-09-21T03:43:45.720 INFO:journalctl@ceph.mds.e.smithi150.stdout:Sep 21 03:43:45 smithi150 systemd[1]: Starting Ceph mds.e for 51116d5c-77ca-11ef-baf5-efdc52797490... 2024-09-21T03:43:45.833 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:43:45 smithi039 ceph-mon[31211]: from='client.? 172.21.15.39:0/2815379395' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:43:45.974 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:43:45 smithi150 ceph-mon[34289]: from='client.? 172.21.15.39:0/2815379395' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:43:45.975 INFO:journalctl@ceph.mds.e.smithi150.stdout:Sep 21 03:43:45 smithi150 podman[62963]: 2024-09-21 03:43:45.680355849 +0000 UTC m=+0.015956812 image pull ddce307e1f3263fb077b4f3902e15660050631dad4d2823a20ee3f72bf18d977 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262 2024-09-21T03:43:45.975 INFO:journalctl@ceph.mds.e.smithi150.stdout:Sep 21 03:43:45 smithi150 podman[62963]: 2024-09-21 03:43:45.790885177 +0000 UTC m=+0.126486120 container create fd7950219c69717ebeae78a39a2b761ade3ab5de72a7bfbe58e4d8146ec1689c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-51116d5c-77ca-11ef-baf5-efdc52797490-mds-e, CEPH_POINT_RELEASE=, ceph=True, GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_BRANCH=HEAD, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2) 2024-09-21T03:43:46.218 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:43:45 smithi161 ceph-mon[34533]: from='client.? 172.21.15.39:0/2815379395' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:43:46.474 INFO:journalctl@ceph.mds.e.smithi150.stdout:Sep 21 03:43:46 smithi150 podman[62963]: 2024-09-21 03:43:46.093690269 +0000 UTC m=+0.429291219 container init fd7950219c69717ebeae78a39a2b761ade3ab5de72a7bfbe58e4d8146ec1689c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-51116d5c-77ca-11ef-baf5-efdc52797490-mds-e, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, CEPH_POINT_RELEASE=, ceph=True, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-21T03:43:46.475 INFO:journalctl@ceph.mds.e.smithi150.stdout:Sep 21 03:43:46 smithi150 podman[62963]: 2024-09-21 03:43:46.096461123 +0000 UTC m=+0.432062057 container start fd7950219c69717ebeae78a39a2b761ade3ab5de72a7bfbe58e4d8146ec1689c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-51116d5c-77ca-11ef-baf5-efdc52797490-mds-e, org.label-schema.build-date=20240731, ceph=True, org.label-schema.license=GPLv2, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_CLEAN=True, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-21T03:43:46.475 INFO:journalctl@ceph.mds.e.smithi150.stdout:Sep 21 03:43:46 smithi150 ceph-mds[62982]: -- 172.21.15.150:0/3565661336 <== mon.2 v2:172.21.15.150:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5654a3355860 con 0x5654a74fda80 2024-09-21T03:43:46.475 INFO:journalctl@ceph.mds.e.smithi150.stdout:Sep 21 03:43:46 smithi150 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-mds-e[62978]: starting mds.e at 2024-09-21T03:43:46.475 INFO:journalctl@ceph.mds.e.smithi150.stdout:Sep 21 03:43:46 smithi150 bash[62963]: fd7950219c69717ebeae78a39a2b761ade3ab5de72a7bfbe58e4d8146ec1689c 2024-09-21T03:43:46.475 INFO:journalctl@ceph.mds.e.smithi150.stdout:Sep 21 03:43:46 smithi150 systemd[1]: Started Ceph mds.e for 51116d5c-77ca-11ef-baf5-efdc52797490. 2024-09-21T03:43:47.468 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:43:47 smithi161 ceph-mon[34533]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T03:43:47.468 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:43:47 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:43:47.468 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:43:47 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:43:47.468 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:43:47 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:43:47.468 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:43:47 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T03:43:47.468 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:43:47 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T03:43:47.468 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:43:47 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:43:47.468 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:43:47 smithi161 ceph-mon[34533]: Deploying daemon mds.f on smithi161 2024-09-21T03:43:47.468 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:43:47 smithi161 ceph-mon[34533]: mds.? [v2:172.21.15.150:6834/724594008,v1:172.21.15.150:6835/724594008] up:boot 2024-09-21T03:43:47.468 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:43:47 smithi161 ceph-mon[34533]: fsmap 5 up:standby 2024-09-21T03:43:47.469 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:43:47 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "e"} : dispatch 2024-09-21T03:43:47.542 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 51116d5c-77ca-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T03:43:47.572 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:43:47 smithi039 ceph-mon[31211]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T03:43:47.572 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:43:47 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:43:47.572 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:43:47 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:43:47.572 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:43:47 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:43:47.572 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:43:47 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T03:43:47.572 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:43:47 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T03:43:47.572 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:43:47 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:43:47.572 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:43:47 smithi039 ceph-mon[31211]: Deploying daemon mds.f on smithi161 2024-09-21T03:43:47.573 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:43:47 smithi039 ceph-mon[31211]: mds.? [v2:172.21.15.150:6834/724594008,v1:172.21.15.150:6835/724594008] up:boot 2024-09-21T03:43:47.573 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:43:47 smithi039 ceph-mon[31211]: fsmap 5 up:standby 2024-09-21T03:43:47.573 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:43:47 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "e"} : dispatch 2024-09-21T03:43:47.725 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:43:47 smithi150 ceph-mon[34289]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T03:43:47.725 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:43:47 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:43:47.725 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:43:47 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:43:47.725 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:43:47 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:43:47.725 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:43:47 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T03:43:47.725 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:43:47 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T03:43:47.725 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:43:47 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:43:47.725 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:43:47 smithi150 ceph-mon[34289]: Deploying daemon mds.f on smithi161 2024-09-21T03:43:47.726 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:43:47 smithi150 ceph-mon[34289]: mds.? [v2:172.21.15.150:6834/724594008,v1:172.21.15.150:6835/724594008] up:boot 2024-09-21T03:43:47.726 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:43:47 smithi150 ceph-mon[34289]: fsmap 5 up:standby 2024-09-21T03:43:47.726 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:43:47 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "e"} : dispatch 2024-09-21T03:43:47.813 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/51116d5c-77ca-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:43:48.218 INFO:journalctl@ceph.mds.f.smithi161.stdout:Sep 21 03:43:48 smithi161 systemd[1]: Starting Ceph mds.f for 51116d5c-77ca-11ef-baf5-efdc52797490... 2024-09-21T03:43:48.312 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:48.296+0000 7f13e852f640 1 -- 172.21.15.39:0/811620630 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f13e0103b60 msgr2=0x7f13e0103f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:48.312 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:48.296+0000 7f13e852f640 1 --2- 172.21.15.39:0/811620630 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f13e0103b60 0x7f13e0103f70 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f13dc009900 tx=0x7f13dc02a410 comp rx=0 tx=0).stop 2024-09-21T03:43:48.313 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:48.296+0000 7f13e852f640 1 -- 172.21.15.39:0/811620630 shutdown_connections 2024-09-21T03:43:48.313 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:48.296+0000 7f13e852f640 1 --2- 172.21.15.39:0/811620630 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f13e0109df0 0x7f13e0110620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:48.313 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:48.296+0000 7f13e852f640 1 --2- 172.21.15.39:0/811620630 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f13e01094a0 0x7f13e01098b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:48.313 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:48.296+0000 7f13e852f640 1 --2- 172.21.15.39:0/811620630 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f13e0103b60 0x7f13e0103f70 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:48.313 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:48.296+0000 7f13e852f640 1 -- 172.21.15.39:0/811620630 >> 172.21.15.39:0/811620630 conn(0x7f13e007b850 msgr2=0x7f13e007bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:43:48.313 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:48.296+0000 7f13e852f640 1 -- 172.21.15.39:0/811620630 shutdown_connections 2024-09-21T03:43:48.313 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:48.296+0000 7f13e852f640 1 -- 172.21.15.39:0/811620630 wait complete. 2024-09-21T03:43:48.313 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:48.297+0000 7f13e852f640 1 Processor -- start 2024-09-21T03:43:48.313 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:48.297+0000 7f13e852f640 1 -- start start 2024-09-21T03:43:48.313 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:48.297+0000 7f13e852f640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f13e0103b60 0x7f13e019c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:48.313 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:48.297+0000 7f13e852f640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f13e01094a0 0x7f13e019ce90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:48.313 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:48.297+0000 7f13e852f640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f13e0109df0 0x7f13e01a1e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:48.314 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:48.297+0000 7f13e852f640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f13e0075aa0 con 0x7f13e0109df0 2024-09-21T03:43:48.314 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:48.297+0000 7f13e852f640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f13e0075920 con 0x7f13e01094a0 2024-09-21T03:43:48.314 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:48.297+0000 7f13e852f640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f13e0075c20 con 0x7f13e0103b60 2024-09-21T03:43:48.314 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:48.297+0000 7f13e6acc640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f13e0109df0 0x7f13e01a1e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:48.314 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:48.297+0000 7f13e62cb640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f13e0103b60 0x7f13e019c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:48.314 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:48.297+0000 7f13e6acc640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f13e0109df0 0x7f13e01a1e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:59560/0 (socket says 172.21.15.39:59560) 2024-09-21T03:43:48.314 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:48.297+0000 7f13e62cb640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f13e0103b60 0x7f13e019c8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.39:57150/0 (socket says 172.21.15.39:57150) 2024-09-21T03:43:48.314 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:48.297+0000 7f13e6acc640 1 -- 172.21.15.39:0/686394427 learned_addr learned my addr 172.21.15.39:0/686394427 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:43:48.314 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:48.297+0000 7f13e5aca640 1 --2- 172.21.15.39:0/686394427 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f13e01094a0 0x7f13e019ce90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:48.314 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:48.297+0000 7f13e6acc640 1 -- 172.21.15.39:0/686394427 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f13e0103b60 msgr2=0x7f13e019c8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:48.314 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:48.297+0000 7f13e6acc640 1 --2- 172.21.15.39:0/686394427 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f13e0103b60 0x7f13e019c8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:48.314 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:48.297+0000 7f13e6acc640 1 -- 172.21.15.39:0/686394427 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f13e01094a0 msgr2=0x7f13e019ce90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:48.314 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:48.297+0000 7f13e6acc640 1 --2- 172.21.15.39:0/686394427 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f13e01094a0 0x7f13e019ce90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:48.315 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:48.297+0000 7f13e6acc640 1 -- 172.21.15.39:0/686394427 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f13e01a25c0 con 0x7f13e0109df0 2024-09-21T03:43:48.315 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:48.297+0000 7f13e5aca640 1 --2- 172.21.15.39:0/686394427 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f13e01094a0 0x7f13e019ce90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:43:48.315 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:48.298+0000 7f13e6acc640 1 --2- 172.21.15.39:0/686394427 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f13e0109df0 0x7f13e01a1e90 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f13d8009fd0 tx=0x7f13d800b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:43:48.315 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:48.298+0000 7f13d77fe640 1 -- 172.21.15.39:0/686394427 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13d8017070 con 0x7f13e0109df0 2024-09-21T03:43:48.315 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:48.298+0000 7f13d77fe640 1 -- 172.21.15.39:0/686394427 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f13d8003e20 con 0x7f13e0109df0 2024-09-21T03:43:48.315 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:48.298+0000 7f13d77fe640 1 -- 172.21.15.39:0/686394427 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13d8010460 con 0x7f13e0109df0 2024-09-21T03:43:48.315 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:48.298+0000 7f13e852f640 1 -- 172.21.15.39:0/686394427 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f13e01a28b0 con 0x7f13e0109df0 2024-09-21T03:43:48.315 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:48.298+0000 7f13e852f640 1 -- 172.21.15.39:0/686394427 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f13e01a9030 con 0x7f13e0109df0 2024-09-21T03:43:48.315 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:48.299+0000 7f13e852f640 1 -- 172.21.15.39:0/686394427 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f13e0103f70 con 0x7f13e0109df0 2024-09-21T03:43:48.315 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:48.301+0000 7f13d77fe640 1 -- 172.21.15.39:0/686394427 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 21) ==== 154529+0+0 (secure 0 0 0) 0x7f13d8003970 con 0x7f13e0109df0 2024-09-21T03:43:48.315 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:48.302+0000 7f13d77fe640 1 --2- 172.21.15.39:0/686394427 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f13cc0b65e0 0x7f13cc0b8af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:48.315 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:48.302+0000 7f13e62cb640 1 --2- 172.21.15.39:0/686394427 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f13cc0b65e0 0x7f13cc0b8af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:48.315 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:48.302+0000 7f13d77fe640 1 -- 172.21.15.39:0/686394427 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 7342+0+0 (secure 0 0 0) 0x7f13d80e6cd0 con 0x7f13e0109df0 2024-09-21T03:43:48.316 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:48.303+0000 7f13e62cb640 1 --2- 172.21.15.39:0/686394427 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f13cc0b65e0 0x7f13cc0b8af0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f13dc031440 tx=0x7f13dc009560 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:43:48.316 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:48.306+0000 7f13d77fe640 1 -- 172.21.15.39:0/686394427 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f13d80ad9b0 con 0x7f13e0109df0 2024-09-21T03:43:48.535 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:48.533+0000 7f13e852f640 1 -- 172.21.15.39:0/686394427 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f13e01a9310 con 0x7f13e0109df0 2024-09-21T03:43:48.535 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:48.534+0000 7f13d77fe640 1 -- 172.21.15.39:0/686394427 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) ==== 93+0+4738 (secure 0 0 0) 0x7f13d80b1d90 con 0x7f13e0109df0 2024-09-21T03:43:48.535 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T03:43:48.536 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":6,"btime":"2024-09-21T03:43:46:718778+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14790,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6835/508995295","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":508995295},{"type":"v1","addr":"172.21.15.39:6835","nonce":508995295}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14802,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6837/3915912607","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":3915912607},{"type":"v1","addr":"172.21.15.39:6837","nonce":3915912607}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24599,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6833/3114643217","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":3114643217},{"type":"v1","addr":"172.21.15.150:6833","nonce":3114643217}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24601,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6833/2963990654","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":2963990654},{"type":"v1","addr":"172.21.15.161:6833","nonce":2963990654}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24632,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6835/724594008","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6834","nonce":724594008},{"type":"v1","addr":"172.21.15.150:6835","nonce":724594008}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[]} 2024-09-21T03:43:48.536 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 6 2024-09-21T03:43:48.538 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:48.536+0000 7f13e852f640 1 -- 172.21.15.39:0/686394427 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f13cc0b65e0 msgr2=0x7f13cc0b8af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:48.538 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:48.536+0000 7f13e852f640 1 --2- 172.21.15.39:0/686394427 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f13cc0b65e0 0x7f13cc0b8af0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f13dc031440 tx=0x7f13dc009560 comp rx=0 tx=0).stop 2024-09-21T03:43:48.538 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:48.536+0000 7f13e852f640 1 -- 172.21.15.39:0/686394427 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f13e0109df0 msgr2=0x7f13e01a1e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:48.538 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:48.536+0000 7f13e852f640 1 --2- 172.21.15.39:0/686394427 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f13e0109df0 0x7f13e01a1e90 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f13d8009fd0 tx=0x7f13d800b260 comp rx=0 tx=0).stop 2024-09-21T03:43:48.538 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:48.536+0000 7f13e852f640 1 -- 172.21.15.39:0/686394427 shutdown_connections 2024-09-21T03:43:48.538 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:48.536+0000 7f13e852f640 1 --2- 172.21.15.39:0/686394427 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f13cc0b65e0 0x7f13cc0b8af0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:48.538 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:48.536+0000 7f13e852f640 1 --2- 172.21.15.39:0/686394427 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f13e0109df0 0x7f13e01a1e90 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:48.538 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:48.536+0000 7f13e852f640 1 --2- 172.21.15.39:0/686394427 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f13e01094a0 0x7f13e019ce90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:48.538 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:48.536+0000 7f13e852f640 1 --2- 172.21.15.39:0/686394427 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f13e0103b60 0x7f13e019c8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:48.539 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:48.536+0000 7f13e852f640 1 -- 172.21.15.39:0/686394427 >> 172.21.15.39:0/686394427 conn(0x7f13e007b850 msgr2=0x7f13e0100630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:43:48.539 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:48.536+0000 7f13e852f640 1 -- 172.21.15.39:0/686394427 shutdown_connections 2024-09-21T03:43:48.539 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:48.536+0000 7f13e852f640 1 -- 172.21.15.39:0/686394427 wait complete. 2024-09-21T03:43:48.558 INFO:journalctl@ceph.mds.f.smithi161.stdout:Sep 21 03:43:48 smithi161 podman[63347]: 2024-09-21 03:43:48.189074533 +0000 UTC m=+0.015946306 image pull ddce307e1f3263fb077b4f3902e15660050631dad4d2823a20ee3f72bf18d977 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262 2024-09-21T03:43:48.559 INFO:journalctl@ceph.mds.f.smithi161.stdout:Sep 21 03:43:48 smithi161 podman[63347]: 2024-09-21 03:43:48.309955757 +0000 UTC m=+0.136827518 container create eb6e976e5fd4c4e61531a6b24327e749ec80e2314449b00c7baf4e7e2cd94b27 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-51116d5c-77ca-11ef-baf5-efdc52797490-mds-f, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, ceph=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD) 2024-09-21T03:43:48.559 INFO:journalctl@ceph.mds.f.smithi161.stdout:Sep 21 03:43:48 smithi161 podman[63347]: 2024-09-21 03:43:48.520898431 +0000 UTC m=+0.347770193 container init eb6e976e5fd4c4e61531a6b24327e749ec80e2314449b00c7baf4e7e2cd94b27 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-51116d5c-77ca-11ef-baf5-efdc52797490-mds-f, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_CLEAN=True, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, ceph=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS) 2024-09-21T03:43:48.559 INFO:journalctl@ceph.mds.f.smithi161.stdout:Sep 21 03:43:48 smithi161 podman[63347]: 2024-09-21 03:43:48.523816403 +0000 UTC m=+0.350688164 container start eb6e976e5fd4c4e61531a6b24327e749ec80e2314449b00c7baf4e7e2cd94b27 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-51116d5c-77ca-11ef-baf5-efdc52797490-mds-f, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.schema-version=1.0) 2024-09-21T03:43:48.559 INFO:journalctl@ceph.mds.f.smithi161.stdout:Sep 21 03:43:48 smithi161 ceph-mds[63365]: -- 172.21.15.161:0/3883904141 <== mon.1 v2:172.21.15.161:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x556b1be01860 con 0x556b1ffa9180 2024-09-21T03:43:48.968 INFO:journalctl@ceph.mds.f.smithi161.stdout:Sep 21 03:43:48 smithi161 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-mds-f[63361]: starting mds.f at 2024-09-21T03:43:48.968 INFO:journalctl@ceph.mds.f.smithi161.stdout:Sep 21 03:43:48 smithi161 bash[63347]: eb6e976e5fd4c4e61531a6b24327e749ec80e2314449b00c7baf4e7e2cd94b27 2024-09-21T03:43:48.968 INFO:journalctl@ceph.mds.f.smithi161.stdout:Sep 21 03:43:48 smithi161 systemd[1]: Started Ceph mds.f for 51116d5c-77ca-11ef-baf5-efdc52797490. 2024-09-21T03:43:49.400 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:43:49 smithi161 ceph-mon[34533]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T03:43:49.401 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:43:49 smithi161 ceph-mon[34533]: from='client.? 172.21.15.39:0/686394427' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:43:49.401 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:43:49 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:43:49.401 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:43:49 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:43:49.401 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:43:49 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:43:49.401 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:43:49 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T03:43:49.401 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:43:49 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T03:43:49.401 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:43:49 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:43:49.584 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:43:49 smithi039 ceph-mon[31211]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T03:43:49.584 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:43:49 smithi039 ceph-mon[31211]: from='client.? 172.21.15.39:0/686394427' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:43:49.584 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:43:49 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:43:49.584 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:43:49 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:43:49.584 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:43:49 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:43:49.584 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:43:49 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T03:43:49.584 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:43:49 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T03:43:49.584 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:43:49 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:43:49.724 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:43:49 smithi150 ceph-mon[34289]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T03:43:49.725 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:43:49 smithi150 ceph-mon[34289]: from='client.? 172.21.15.39:0/686394427' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:43:49.725 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:43:49 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:43:49.725 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:43:49 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:43:49.725 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:43:49 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:43:49.725 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:43:49 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T03:43:49.725 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:43:49 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T03:43:49.725 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:43:49 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:43:50.381 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:43:50 smithi161 ceph-mon[34533]: Deploying daemon mds.c on smithi161 2024-09-21T03:43:50.381 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:43:50 smithi161 ceph-mon[34533]: mds.? [v2:172.21.15.161:6834/3747882110,v1:172.21.15.161:6835/3747882110] up:boot 2024-09-21T03:43:50.381 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:43:50 smithi161 ceph-mon[34533]: fsmap 6 up:standby 2024-09-21T03:43:50.381 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:43:50 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "f"} : dispatch 2024-09-21T03:43:50.584 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:43:50 smithi039 ceph-mon[31211]: Deploying daemon mds.c on smithi161 2024-09-21T03:43:50.584 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:43:50 smithi039 ceph-mon[31211]: mds.? [v2:172.21.15.161:6834/3747882110,v1:172.21.15.161:6835/3747882110] up:boot 2024-09-21T03:43:50.584 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:43:50 smithi039 ceph-mon[31211]: fsmap 6 up:standby 2024-09-21T03:43:50.584 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:43:50 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "f"} : dispatch 2024-09-21T03:43:50.718 INFO:journalctl@ceph.mds.c.smithi161.stdout:Sep 21 03:43:50 smithi161 systemd[1]: Starting Ceph mds.c for 51116d5c-77ca-11ef-baf5-efdc52797490... 2024-09-21T03:43:50.724 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:43:50 smithi150 ceph-mon[34289]: Deploying daemon mds.c on smithi161 2024-09-21T03:43:50.724 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:43:50 smithi150 ceph-mon[34289]: mds.? [v2:172.21.15.161:6834/3747882110,v1:172.21.15.161:6835/3747882110] up:boot 2024-09-21T03:43:50.725 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:43:50 smithi150 ceph-mon[34289]: fsmap 6 up:standby 2024-09-21T03:43:50.725 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:43:50 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "f"} : dispatch 2024-09-21T03:43:51.028 INFO:journalctl@ceph.mds.c.smithi161.stdout:Sep 21 03:43:50 smithi161 podman[63615]: 2024-09-21 03:43:50.677596075 +0000 UTC m=+0.015792462 image pull ddce307e1f3263fb077b4f3902e15660050631dad4d2823a20ee3f72bf18d977 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262 2024-09-21T03:43:51.028 INFO:journalctl@ceph.mds.c.smithi161.stdout:Sep 21 03:43:50 smithi161 podman[63615]: 2024-09-21 03:43:50.796790318 +0000 UTC m=+0.134986697 container create 21811c0fe34e1544f194ab49cf7af9c7e2f57588432a4ebf98e4cf9e6a19cfb7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-51116d5c-77ca-11ef-baf5-efdc52797490-mds-c, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_CLEAN=True) 2024-09-21T03:43:51.029 INFO:journalctl@ceph.mds.c.smithi161.stdout:Sep 21 03:43:51 smithi161 podman[63615]: 2024-09-21 03:43:51.024318797 +0000 UTC m=+0.362515185 container init 21811c0fe34e1544f194ab49cf7af9c7e2f57588432a4ebf98e4cf9e6a19cfb7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-51116d5c-77ca-11ef-baf5-efdc52797490-mds-c, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, ceph=True, GIT_CLEAN=True) 2024-09-21T03:43:51.029 INFO:journalctl@ceph.mds.c.smithi161.stdout:Sep 21 03:43:51 smithi161 podman[63615]: 2024-09-21 03:43:51.027050845 +0000 UTC m=+0.365247226 container start 21811c0fe34e1544f194ab49cf7af9c7e2f57588432a4ebf98e4cf9e6a19cfb7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-51116d5c-77ca-11ef-baf5-efdc52797490-mds-c, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, ceph=True) 2024-09-21T03:43:51.213 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 51116d5c-77ca-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T03:43:51.468 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:43:51 smithi161 ceph-mon[34533]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T03:43:51.468 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:43:51 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:43:51.468 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:43:51 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:43:51.468 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:43:51 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:43:51.468 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:43:51 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T03:43:51.468 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:43:51 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T03:43:51.468 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:43:51 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:43:51.469 INFO:journalctl@ceph.mds.c.smithi161.stdout:Sep 21 03:43:51 smithi161 ceph-mds[63634]: -- 172.21.15.161:0/1174751850 <== mon.1 v2:172.21.15.161:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5612949df860 con 0x561298b87600 2024-09-21T03:43:51.469 INFO:journalctl@ceph.mds.c.smithi161.stdout:Sep 21 03:43:51 smithi161 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-mds-c[63630]: starting mds.c at 2024-09-21T03:43:51.469 INFO:journalctl@ceph.mds.c.smithi161.stdout:Sep 21 03:43:51 smithi161 bash[63615]: 21811c0fe34e1544f194ab49cf7af9c7e2f57588432a4ebf98e4cf9e6a19cfb7 2024-09-21T03:43:51.469 INFO:journalctl@ceph.mds.c.smithi161.stdout:Sep 21 03:43:51 smithi161 systemd[1]: Started Ceph mds.c for 51116d5c-77ca-11ef-baf5-efdc52797490. 2024-09-21T03:43:51.482 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/51116d5c-77ca-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:43:51.583 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:43:51 smithi039 ceph-mon[31211]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T03:43:51.584 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:43:51 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:43:51.584 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:43:51 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:43:51.584 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:43:51 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:43:51.584 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:43:51 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T03:43:51.584 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:43:51 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T03:43:51.584 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:43:51 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:43:51.617 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:43:51 smithi150 ceph-mon[34289]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T03:43:51.617 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:43:51 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:43:51.617 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:43:51 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:43:51.617 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:43:51 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:43:51.617 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:43:51 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T03:43:51.617 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:43:51 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T03:43:51.617 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:43:51 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:43:51.961 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:51.946+0000 7f103349c640 1 -- 172.21.15.39:0/3459016873 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f102c103c30 msgr2=0x7f102c104040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:51.961 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:51.946+0000 7f103349c640 1 --2- 172.21.15.39:0/3459016873 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f102c103c30 0x7f102c104040 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f1028009900 tx=0x7f102802a410 comp rx=0 tx=0).stop 2024-09-21T03:43:51.961 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:51.946+0000 7f103349c640 1 -- 172.21.15.39:0/3459016873 shutdown_connections 2024-09-21T03:43:51.961 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:51.946+0000 7f103349c640 1 --2- 172.21.15.39:0/3459016873 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f102c109ec0 0x7f102c1106f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:51.961 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:51.946+0000 7f103349c640 1 --2- 172.21.15.39:0/3459016873 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f102c109570 0x7f102c109980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:51.961 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:51.946+0000 7f103349c640 1 --2- 172.21.15.39:0/3459016873 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f102c103c30 0x7f102c104040 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:51.962 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:51.946+0000 7f103349c640 1 -- 172.21.15.39:0/3459016873 >> 172.21.15.39:0/3459016873 conn(0x7f102c07b8d0 msgr2=0x7f102c07bce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:43:51.962 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:51.946+0000 7f103349c640 1 -- 172.21.15.39:0/3459016873 shutdown_connections 2024-09-21T03:43:51.962 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:51.946+0000 7f103349c640 1 -- 172.21.15.39:0/3459016873 wait complete. 2024-09-21T03:43:51.962 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:51.947+0000 7f103349c640 1 Processor -- start 2024-09-21T03:43:51.962 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:51.947+0000 7f103349c640 1 -- start start 2024-09-21T03:43:51.962 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:51.948+0000 7f103349c640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f102c103c30 0x7f102c19a530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:51.962 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:51.948+0000 7f103349c640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f102c109570 0x7f102c19aad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:51.962 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:51.948+0000 7f103349c640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f102c109ec0 0x7f102c19fd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:51.962 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:51.948+0000 7f103349c640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f102c113b20 con 0x7f102c103c30 2024-09-21T03:43:51.962 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:51.948+0000 7f103349c640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f102c1139a0 con 0x7f102c109ec0 2024-09-21T03:43:51.962 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:51.948+0000 7f103349c640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f102c113ca0 con 0x7f102c109570 2024-09-21T03:43:51.962 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:51.948+0000 7f1031238640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f102c103c30 0x7f102c19a530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:51.962 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:51.948+0000 7f1031238640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f102c103c30 0x7f102c19a530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:59574/0 (socket says 172.21.15.39:59574) 2024-09-21T03:43:51.963 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:51.948+0000 7f1031238640 1 -- 172.21.15.39:0/986170113 learned_addr learned my addr 172.21.15.39:0/986170113 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:43:51.963 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:51.948+0000 7f1030a37640 1 --2- 172.21.15.39:0/986170113 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f102c109570 0x7f102c19aad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:51.963 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:51.948+0000 7f1031a39640 1 --2- 172.21.15.39:0/986170113 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f102c109ec0 0x7f102c19fd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:51.963 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:51.948+0000 7f1031238640 1 -- 172.21.15.39:0/986170113 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f102c109570 msgr2=0x7f102c19aad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:51.963 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:51.948+0000 7f1031238640 1 --2- 172.21.15.39:0/986170113 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f102c109570 0x7f102c19aad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:51.963 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:51.948+0000 7f1031238640 1 -- 172.21.15.39:0/986170113 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f102c109ec0 msgr2=0x7f102c19fd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:51.963 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:51.948+0000 7f1031238640 1 --2- 172.21.15.39:0/986170113 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f102c109ec0 0x7f102c19fd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:51.963 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:51.948+0000 7f1031238640 1 -- 172.21.15.39:0/986170113 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f102c1a03e0 con 0x7f102c103c30 2024-09-21T03:43:51.963 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:51.948+0000 7f1031a39640 1 --2- 172.21.15.39:0/986170113 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f102c109ec0 0x7f102c19fd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:43:51.963 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:51.948+0000 7f1031238640 1 --2- 172.21.15.39:0/986170113 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f102c103c30 0x7f102c19a530 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f1028006010 tx=0x7f10280042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:43:51.963 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:51.949+0000 7f10227fc640 1 -- 172.21.15.39:0/986170113 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f102803f030 con 0x7f102c103c30 2024-09-21T03:43:51.963 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:51.949+0000 7f103349c640 1 -- 172.21.15.39:0/986170113 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f102c1a0670 con 0x7f102c103c30 2024-09-21T03:43:51.963 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:51.949+0000 7f10227fc640 1 -- 172.21.15.39:0/986170113 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1028003800 con 0x7f102c103c30 2024-09-21T03:43:51.964 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:51.949+0000 7f10227fc640 1 -- 172.21.15.39:0/986170113 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1028035070 con 0x7f102c103c30 2024-09-21T03:43:51.964 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:51.949+0000 7f103349c640 1 -- 172.21.15.39:0/986170113 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f102c1a6ef0 con 0x7f102c103c30 2024-09-21T03:43:51.964 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:51.950+0000 7f103349c640 1 -- 172.21.15.39:0/986170113 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f102c104040 con 0x7f102c103c30 2024-09-21T03:43:51.964 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:51.953+0000 7f10227fc640 1 -- 172.21.15.39:0/986170113 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 21) ==== 154529+0+0 (secure 0 0 0) 0x7f10280039b0 con 0x7f102c103c30 2024-09-21T03:43:51.964 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:51.954+0000 7f10227fc640 1 --2- 172.21.15.39:0/986170113 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f10180b6590 0x7f10180b8aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:51.964 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:51.954+0000 7f1030a37640 1 --2- 172.21.15.39:0/986170113 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f10180b6590 0x7f10180b8aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:51.964 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:51.954+0000 7f10227fc640 1 -- 172.21.15.39:0/986170113 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 7342+0+0 (secure 0 0 0) 0x7f1028107ad0 con 0x7f102c103c30 2024-09-21T03:43:51.964 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:51.954+0000 7f1030a37640 1 --2- 172.21.15.39:0/986170113 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f10180b6590 0x7f10180b8aa0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f102c106590 tx=0x7f101c009110 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:43:51.964 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:51.956+0000 7f10227fc640 1 -- 172.21.15.39:0/986170113 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f10280ce730 con 0x7f102c103c30 2024-09-21T03:43:52.183 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:52.181+0000 7f103349c640 1 -- 172.21.15.39:0/986170113 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f102c10bc90 con 0x7f102c103c30 2024-09-21T03:43:52.183 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:52.182+0000 7f10227fc640 1 -- 172.21.15.39:0/986170113 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+6394 (secure 0 0 0) 0x7f10280d2b10 con 0x7f102c103c30 2024-09-21T03:43:52.184 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T03:43:52.184 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":8,"btime":"2024-09-21T03:43:51:272564+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14790,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6835/508995295","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":508995295},{"type":"v1","addr":"172.21.15.39:6835","nonce":508995295}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14802,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6837/3915912607","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":3915912607},{"type":"v1","addr":"172.21.15.39:6837","nonce":3915912607}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24599,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6833/3114643217","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":3114643217},{"type":"v1","addr":"172.21.15.150:6833","nonce":3114643217}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24601,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6833/2963990654","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":2963990654},{"type":"v1","addr":"172.21.15.161:6833","nonce":2963990654}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24632,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6835/724594008","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6834","nonce":724594008},{"type":"v1","addr":"172.21.15.150:6835","nonce":724594008}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24640,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6835/3747882110","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":3747882110},{"type":"v1","addr":"172.21.15.161:6835","nonce":3747882110}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24646,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6837/3451544597","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6836","nonce":3451544597},{"type":"v1","addr":"172.21.15.161:6837","nonce":3451544597}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8}],"filesystems":[]} 2024-09-21T03:43:52.185 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 8 2024-09-21T03:43:52.186 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:52.184+0000 7f103349c640 1 -- 172.21.15.39:0/986170113 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f10180b6590 msgr2=0x7f10180b8aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:52.186 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:52.184+0000 7f103349c640 1 --2- 172.21.15.39:0/986170113 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f10180b6590 0x7f10180b8aa0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f102c106590 tx=0x7f101c009110 comp rx=0 tx=0).stop 2024-09-21T03:43:52.186 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:52.184+0000 7f103349c640 1 -- 172.21.15.39:0/986170113 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f102c103c30 msgr2=0x7f102c19a530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:52.186 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:52.184+0000 7f103349c640 1 --2- 172.21.15.39:0/986170113 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f102c103c30 0x7f102c19a530 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f1028006010 tx=0x7f10280042f0 comp rx=0 tx=0).stop 2024-09-21T03:43:52.186 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:52.184+0000 7f103349c640 1 -- 172.21.15.39:0/986170113 shutdown_connections 2024-09-21T03:43:52.187 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:52.184+0000 7f103349c640 1 --2- 172.21.15.39:0/986170113 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f10180b6590 0x7f10180b8aa0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:52.187 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:52.184+0000 7f103349c640 1 --2- 172.21.15.39:0/986170113 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f102c109ec0 0x7f102c19fd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:52.187 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:52.184+0000 7f103349c640 1 --2- 172.21.15.39:0/986170113 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f102c109570 0x7f102c19aad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:52.187 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:52.184+0000 7f103349c640 1 --2- 172.21.15.39:0/986170113 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f102c103c30 0x7f102c19a530 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:52.187 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:52.184+0000 7f103349c640 1 -- 172.21.15.39:0/986170113 >> 172.21.15.39:0/986170113 conn(0x7f102c07b8d0 msgr2=0x7f102c107660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:43:52.187 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:52.185+0000 7f103349c640 1 -- 172.21.15.39:0/986170113 shutdown_connections 2024-09-21T03:43:52.187 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:52.185+0000 7f103349c640 1 -- 172.21.15.39:0/986170113 wait complete. 2024-09-21T03:43:52.548 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:43:52 smithi039 ceph-mon[31211]: Deploying daemon mds.h on smithi150 2024-09-21T03:43:52.548 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:43:52 smithi039 ceph-mon[31211]: mds.? [v2:172.21.15.161:6836/3451544597,v1:172.21.15.161:6837/3451544597] up:boot 2024-09-21T03:43:52.548 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:43:52 smithi039 ceph-mon[31211]: fsmap 7 up:standby 2024-09-21T03:43:52.548 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:43:52 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "c"} : dispatch 2024-09-21T03:43:52.549 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:43:52 smithi039 ceph-mon[31211]: from='client.? 172.21.15.39:0/986170113' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:43:52.640 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:43:52 smithi150 ceph-mon[34289]: Deploying daemon mds.h on smithi150 2024-09-21T03:43:52.640 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:43:52 smithi150 ceph-mon[34289]: mds.? [v2:172.21.15.161:6836/3451544597,v1:172.21.15.161:6837/3451544597] up:boot 2024-09-21T03:43:52.640 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:43:52 smithi150 ceph-mon[34289]: fsmap 7 up:standby 2024-09-21T03:43:52.640 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:43:52 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "c"} : dispatch 2024-09-21T03:43:52.640 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:43:52 smithi150 ceph-mon[34289]: from='client.? 172.21.15.39:0/986170113' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:43:52.718 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:43:52 smithi161 ceph-mon[34533]: Deploying daemon mds.h on smithi150 2024-09-21T03:43:52.718 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:43:52 smithi161 ceph-mon[34533]: mds.? [v2:172.21.15.161:6836/3451544597,v1:172.21.15.161:6837/3451544597] up:boot 2024-09-21T03:43:52.718 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:43:52 smithi161 ceph-mon[34533]: fsmap 7 up:standby 2024-09-21T03:43:52.718 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:43:52 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "c"} : dispatch 2024-09-21T03:43:52.718 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:43:52 smithi161 ceph-mon[34533]: from='client.? 172.21.15.39:0/986170113' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:43:53.278 INFO:journalctl@ceph.mds.h.smithi150.stdout:Sep 21 03:43:53 smithi150 systemd[1]: Starting Ceph mds.h for 51116d5c-77ca-11ef-baf5-efdc52797490... 2024-09-21T03:43:53.583 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:43:53 smithi039 ceph-mon[31211]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T03:43:53.606 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:43:53 smithi150 ceph-mon[34289]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T03:43:53.607 INFO:journalctl@ceph.mds.h.smithi150.stdout:Sep 21 03:43:53 smithi150 podman[63235]: 2024-09-21 03:43:53.278648726 +0000 UTC m=+0.016016846 image pull ddce307e1f3263fb077b4f3902e15660050631dad4d2823a20ee3f72bf18d977 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262 2024-09-21T03:43:53.607 INFO:journalctl@ceph.mds.h.smithi150.stdout:Sep 21 03:43:53 smithi150 podman[63235]: 2024-09-21 03:43:53.39506405 +0000 UTC m=+0.132432163 container create 28ae457df36beee65b54e204dea8e1d5e011aafa663b12e7ef4342f4caefbefb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-51116d5c-77ca-11ef-baf5-efdc52797490-mds-h, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, ceph=True, GIT_CLEAN=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux ) 2024-09-21T03:43:53.717 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:43:53 smithi161 ceph-mon[34533]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T03:43:53.975 INFO:journalctl@ceph.mds.h.smithi150.stdout:Sep 21 03:43:53 smithi150 podman[63235]: 2024-09-21 03:43:53.605733066 +0000 UTC m=+0.343101177 container init 28ae457df36beee65b54e204dea8e1d5e011aafa663b12e7ef4342f4caefbefb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-51116d5c-77ca-11ef-baf5-efdc52797490-mds-h, org.label-schema.vendor=CentOS, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , ceph=True, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-21T03:43:53.975 INFO:journalctl@ceph.mds.h.smithi150.stdout:Sep 21 03:43:53 smithi150 podman[63235]: 2024-09-21 03:43:53.60875062 +0000 UTC m=+0.346118733 container start 28ae457df36beee65b54e204dea8e1d5e011aafa663b12e7ef4342f4caefbefb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-51116d5c-77ca-11ef-baf5-efdc52797490-mds-h, ceph=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_CLEAN=True) 2024-09-21T03:43:53.975 INFO:journalctl@ceph.mds.h.smithi150.stdout:Sep 21 03:43:53 smithi150 ceph-mds[63254]: -- 172.21.15.150:0/2101201062 <== mon.2 v2:172.21.15.150:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55f572d4dc20 con 0x55f576ef5180 2024-09-21T03:43:53.975 INFO:journalctl@ceph.mds.h.smithi150.stdout:Sep 21 03:43:53 smithi150 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-mds-h[63250]: starting mds.h at 2024-09-21T03:43:53.975 INFO:journalctl@ceph.mds.h.smithi150.stdout:Sep 21 03:43:53 smithi150 bash[63235]: 28ae457df36beee65b54e204dea8e1d5e011aafa663b12e7ef4342f4caefbefb 2024-09-21T03:43:53.975 INFO:journalctl@ceph.mds.h.smithi150.stdout:Sep 21 03:43:53 smithi150 systemd[1]: Started Ceph mds.h for 51116d5c-77ca-11ef-baf5-efdc52797490. 2024-09-21T03:43:54.893 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 51116d5c-77ca-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T03:43:55.084 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:43:54 smithi039 ceph-mon[31211]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T03:43:55.084 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:43:54 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:43:55.084 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:43:54 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:43:55.084 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:43:54 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:43:55.084 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:43:54 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T03:43:55.084 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:43:54 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T03:43:55.084 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:43:54 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:43:55.084 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:43:54 smithi039 ceph-mon[31211]: Deploying daemon mds.i on smithi161 2024-09-21T03:43:55.084 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:43:54 smithi039 ceph-mon[31211]: mds.? [v2:172.21.15.150:6836/4017043144,v1:172.21.15.150:6837/4017043144] up:boot 2024-09-21T03:43:55.084 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:43:54 smithi039 ceph-mon[31211]: fsmap 8 up:standby 2024-09-21T03:43:55.084 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:43:54 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "h"} : dispatch 2024-09-21T03:43:55.167 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/51116d5c-77ca-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:43:55.218 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:43:54 smithi161 ceph-mon[34533]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T03:43:55.218 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:43:54 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:43:55.218 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:43:54 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:43:55.218 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:43:54 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:43:55.218 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:43:54 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T03:43:55.218 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:43:54 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T03:43:55.218 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:43:54 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:43:55.218 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:43:54 smithi161 ceph-mon[34533]: Deploying daemon mds.i on smithi161 2024-09-21T03:43:55.218 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:43:54 smithi161 ceph-mon[34533]: mds.? [v2:172.21.15.150:6836/4017043144,v1:172.21.15.150:6837/4017043144] up:boot 2024-09-21T03:43:55.219 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:43:54 smithi161 ceph-mon[34533]: fsmap 8 up:standby 2024-09-21T03:43:55.219 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:43:54 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "h"} : dispatch 2024-09-21T03:43:55.224 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:43:54 smithi150 ceph-mon[34289]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T03:43:55.225 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:43:54 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:43:55.225 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:43:54 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:43:55.225 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:43:54 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:43:55.225 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:43:54 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T03:43:55.225 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:43:54 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T03:43:55.225 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:43:54 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:43:55.225 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:43:54 smithi150 ceph-mon[34289]: Deploying daemon mds.i on smithi161 2024-09-21T03:43:55.225 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:43:54 smithi150 ceph-mon[34289]: mds.? [v2:172.21.15.150:6836/4017043144,v1:172.21.15.150:6837/4017043144] up:boot 2024-09-21T03:43:55.225 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:43:54 smithi150 ceph-mon[34289]: fsmap 8 up:standby 2024-09-21T03:43:55.225 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:43:54 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "h"} : dispatch 2024-09-21T03:43:55.693 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:55.668+0000 7f9a1b186640 1 -- 172.21.15.39:0/2878665574 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9a14109df0 msgr2=0x7f9a14110620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:55.693 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:55.668+0000 7f9a1b186640 1 --2- 172.21.15.39:0/2878665574 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9a14109df0 0x7f9a14110620 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f9a0c009900 tx=0x7f9a0c02a410 comp rx=0 tx=0).stop 2024-09-21T03:43:55.693 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:55.668+0000 7f9a1b186640 1 -- 172.21.15.39:0/2878665574 shutdown_connections 2024-09-21T03:43:55.693 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:55.668+0000 7f9a1b186640 1 --2- 172.21.15.39:0/2878665574 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9a14109df0 0x7f9a14110620 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:55.693 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:55.668+0000 7f9a1b186640 1 --2- 172.21.15.39:0/2878665574 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9a141094a0 0x7f9a141098b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:55.693 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:55.669+0000 7f9a1b186640 1 --2- 172.21.15.39:0/2878665574 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9a14103b60 0x7f9a14103f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:55.693 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:55.669+0000 7f9a1b186640 1 -- 172.21.15.39:0/2878665574 >> 172.21.15.39:0/2878665574 conn(0x7f9a1407b850 msgr2=0x7f9a1407bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:43:55.693 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:55.669+0000 7f9a1b186640 1 -- 172.21.15.39:0/2878665574 shutdown_connections 2024-09-21T03:43:55.694 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:55.669+0000 7f9a1b186640 1 -- 172.21.15.39:0/2878665574 wait complete. 2024-09-21T03:43:55.694 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:55.669+0000 7f9a1b186640 1 Processor -- start 2024-09-21T03:43:55.694 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:55.669+0000 7f9a1b186640 1 -- start start 2024-09-21T03:43:55.694 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:55.670+0000 7f9a1b186640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9a14103b60 0x7f9a1419a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:55.694 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:55.670+0000 7f9a1b186640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9a141094a0 0x7f9a1419aa20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:55.694 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:55.670+0000 7f9a1b186640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9a14109df0 0x7f9a1419fcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:55.694 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:55.670+0000 7f9a1b186640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f9a14113ab0 con 0x7f9a141094a0 2024-09-21T03:43:55.694 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:55.670+0000 7f9a1b186640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f9a14113930 con 0x7f9a14103b60 2024-09-21T03:43:55.694 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:55.670+0000 7f9a1b186640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f9a14113c30 con 0x7f9a14109df0 2024-09-21T03:43:55.694 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:55.670+0000 7f9a13fff640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9a141094a0 0x7f9a1419aa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:55.694 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:55.670+0000 7f9a13fff640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9a141094a0 0x7f9a1419aa20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:59600/0 (socket says 172.21.15.39:59600) 2024-09-21T03:43:55.694 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:55.670+0000 7f9a13fff640 1 -- 172.21.15.39:0/2100168239 learned_addr learned my addr 172.21.15.39:0/2100168239 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:43:55.695 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:55.670+0000 7f9a19723640 1 --2- 172.21.15.39:0/2100168239 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9a14109df0 0x7f9a1419fcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:55.695 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:55.670+0000 7f9a18f22640 1 --2- 172.21.15.39:0/2100168239 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9a14103b60 0x7f9a1419a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:55.695 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:55.670+0000 7f9a13fff640 1 -- 172.21.15.39:0/2100168239 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9a14109df0 msgr2=0x7f9a1419fcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:55.695 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:55.670+0000 7f9a13fff640 1 --2- 172.21.15.39:0/2100168239 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9a14109df0 0x7f9a1419fcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:55.695 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:55.670+0000 7f9a13fff640 1 -- 172.21.15.39:0/2100168239 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9a14103b60 msgr2=0x7f9a1419a480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:55.695 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:55.670+0000 7f9a13fff640 1 --2- 172.21.15.39:0/2100168239 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9a14103b60 0x7f9a1419a480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:55.695 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:55.670+0000 7f9a13fff640 1 -- 172.21.15.39:0/2100168239 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9a141a03f0 con 0x7f9a141094a0 2024-09-21T03:43:55.695 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:55.670+0000 7f9a13fff640 1 --2- 172.21.15.39:0/2100168239 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9a141094a0 0x7f9a1419aa20 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f9a0400ea50 tx=0x7f9a0400ef10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:43:55.695 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:55.670+0000 7f9a11ffb640 1 -- 172.21.15.39:0/2100168239 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a0400cdc0 con 0x7f9a141094a0 2024-09-21T03:43:55.695 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:55.670+0000 7f9a11ffb640 1 -- 172.21.15.39:0/2100168239 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9a04011920 con 0x7f9a141094a0 2024-09-21T03:43:55.695 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:55.670+0000 7f9a11ffb640 1 -- 172.21.15.39:0/2100168239 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a0401aa60 con 0x7f9a141094a0 2024-09-21T03:43:55.695 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:55.670+0000 7f9a1b186640 1 -- 172.21.15.39:0/2100168239 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9a141a06e0 con 0x7f9a141094a0 2024-09-21T03:43:55.695 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:55.671+0000 7f9a1b186640 1 -- 172.21.15.39:0/2100168239 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9a141a6e40 con 0x7f9a141094a0 2024-09-21T03:43:55.696 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:55.672+0000 7f9a1b186640 1 -- 172.21.15.39:0/2100168239 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9a14103f70 con 0x7f9a141094a0 2024-09-21T03:43:55.696 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:55.673+0000 7f9a11ffb640 1 -- 172.21.15.39:0/2100168239 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 21) ==== 154529+0+0 (secure 0 0 0) 0x7f9a04011ad0 con 0x7f9a141094a0 2024-09-21T03:43:55.696 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:55.674+0000 7f9a11ffb640 1 --2- 172.21.15.39:0/2100168239 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f9a000b6560 0x7f9a000b8a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:55.696 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:55.674+0000 7f9a18f22640 1 --2- 172.21.15.39:0/2100168239 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f9a000b6560 0x7f9a000b8a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:55.696 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:55.674+0000 7f9a11ffb640 1 -- 172.21.15.39:0/2100168239 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 7342+0+0 (secure 0 0 0) 0x7f9a040e7700 con 0x7f9a141094a0 2024-09-21T03:43:55.696 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:55.675+0000 7f9a18f22640 1 --2- 172.21.15.39:0/2100168239 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f9a000b6560 0x7f9a000b8a70 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f99fc009b60 tx=0x7f99fc005f90 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:43:55.696 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:55.677+0000 7f9a11ffb640 1 -- 172.21.15.39:0/2100168239 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f9a04058710 con 0x7f9a141094a0 2024-09-21T03:43:55.908 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:55.907+0000 7f9a1b186640 1 -- 172.21.15.39:0/2100168239 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f9a1419b5b0 con 0x7f9a141094a0 2024-09-21T03:43:55.909 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:55.907+0000 7f9a11ffb640 1 -- 172.21.15.39:0/2100168239 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) ==== 93+0+7222 (secure 0 0 0) 0x7f9a1419b5b0 con 0x7f9a141094a0 2024-09-21T03:43:55.909 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T03:43:55.909 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":9,"btime":"2024-09-21T03:43:54:277051+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14790,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6835/508995295","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":508995295},{"type":"v1","addr":"172.21.15.39:6835","nonce":508995295}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14802,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6837/3915912607","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":3915912607},{"type":"v1","addr":"172.21.15.39:6837","nonce":3915912607}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24599,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6833/3114643217","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":3114643217},{"type":"v1","addr":"172.21.15.150:6833","nonce":3114643217}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24601,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6833/2963990654","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":2963990654},{"type":"v1","addr":"172.21.15.161:6833","nonce":2963990654}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24632,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6835/724594008","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6834","nonce":724594008},{"type":"v1","addr":"172.21.15.150:6835","nonce":724594008}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24640,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6835/3747882110","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":3747882110},{"type":"v1","addr":"172.21.15.161:6835","nonce":3747882110}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24646,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6837/3451544597","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6836","nonce":3451544597},{"type":"v1","addr":"172.21.15.161:6837","nonce":3451544597}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24659,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6837/4017043144","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6836","nonce":4017043144},{"type":"v1","addr":"172.21.15.150:6837","nonce":4017043144}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9}],"filesystems":[]} 2024-09-21T03:43:55.910 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 9 2024-09-21T03:43:55.911 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:55.910+0000 7f9a1b186640 1 -- 172.21.15.39:0/2100168239 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f9a000b6560 msgr2=0x7f9a000b8a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:55.912 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:55.910+0000 7f9a1b186640 1 --2- 172.21.15.39:0/2100168239 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f9a000b6560 0x7f9a000b8a70 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f99fc009b60 tx=0x7f99fc005f90 comp rx=0 tx=0).stop 2024-09-21T03:43:55.912 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:55.910+0000 7f9a1b186640 1 -- 172.21.15.39:0/2100168239 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9a141094a0 msgr2=0x7f9a1419aa20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:55.912 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:55.910+0000 7f9a1b186640 1 --2- 172.21.15.39:0/2100168239 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9a141094a0 0x7f9a1419aa20 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f9a0400ea50 tx=0x7f9a0400ef10 comp rx=0 tx=0).stop 2024-09-21T03:43:55.912 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:55.910+0000 7f9a1b186640 1 -- 172.21.15.39:0/2100168239 shutdown_connections 2024-09-21T03:43:55.912 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:55.910+0000 7f9a1b186640 1 --2- 172.21.15.39:0/2100168239 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f9a000b6560 0x7f9a000b8a70 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:55.912 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:55.910+0000 7f9a1b186640 1 --2- 172.21.15.39:0/2100168239 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9a14109df0 0x7f9a1419fcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:55.912 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:55.910+0000 7f9a1b186640 1 --2- 172.21.15.39:0/2100168239 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9a141094a0 0x7f9a1419aa20 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:55.912 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:55.910+0000 7f9a1b186640 1 --2- 172.21.15.39:0/2100168239 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9a14103b60 0x7f9a1419a480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:55.912 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:55.910+0000 7f9a1b186640 1 -- 172.21.15.39:0/2100168239 >> 172.21.15.39:0/2100168239 conn(0x7f9a1407b850 msgr2=0x7f9a14105470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:43:55.912 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:55.910+0000 7f9a1b186640 1 -- 172.21.15.39:0/2100168239 shutdown_connections 2024-09-21T03:43:55.912 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:55.910+0000 7f9a1b186640 1 -- 172.21.15.39:0/2100168239 wait complete. 2024-09-21T03:43:55.917 INFO:journalctl@ceph.mds.i.smithi161.stdout:Sep 21 03:43:55 smithi161 systemd[1]: Starting Ceph mds.i for 51116d5c-77ca-11ef-baf5-efdc52797490... 2024-09-21T03:43:56.196 INFO:journalctl@ceph.mds.i.smithi161.stdout:Sep 21 03:43:55 smithi161 podman[63882]: 2024-09-21 03:43:55.818198789 +0000 UTC m=+0.016175618 image pull ddce307e1f3263fb077b4f3902e15660050631dad4d2823a20ee3f72bf18d977 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262 2024-09-21T03:43:56.196 INFO:journalctl@ceph.mds.i.smithi161.stdout:Sep 21 03:43:55 smithi161 podman[63882]: 2024-09-21 03:43:55.933539217 +0000 UTC m=+0.131516090 container create 97ce66e526b24938fa3e9c132e2f84bc247cb2711e65c9e16e0dbed43fc833a8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-51116d5c-77ca-11ef-baf5-efdc52797490-mds-i, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, io.buildah.version=1.37.2, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, ceph=True, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731) 2024-09-21T03:43:56.468 INFO:journalctl@ceph.mds.i.smithi161.stdout:Sep 21 03:43:56 smithi161 podman[63882]: 2024-09-21 03:43:56.19443821 +0000 UTC m=+0.392415046 container init 97ce66e526b24938fa3e9c132e2f84bc247cb2711e65c9e16e0dbed43fc833a8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-51116d5c-77ca-11ef-baf5-efdc52797490-mds-i, org.label-schema.license=GPLv2, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, CEPH_POINT_RELEASE=, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS) 2024-09-21T03:43:56.468 INFO:journalctl@ceph.mds.i.smithi161.stdout:Sep 21 03:43:56 smithi161 podman[63882]: 2024-09-21 03:43:56.197442766 +0000 UTC m=+0.395419596 container start 97ce66e526b24938fa3e9c132e2f84bc247cb2711e65c9e16e0dbed43fc833a8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-51116d5c-77ca-11ef-baf5-efdc52797490-mds-i, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, maintainer=Guillaume Abrioux , GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-21T03:43:56.468 INFO:journalctl@ceph.mds.i.smithi161.stdout:Sep 21 03:43:56 smithi161 ceph-mds[63901]: -- 172.21.15.161:0/518426681 <== mon.1 v2:172.21.15.161:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55f242c53860 con 0x55f246dfb180 2024-09-21T03:43:56.468 INFO:journalctl@ceph.mds.i.smithi161.stdout:Sep 21 03:43:56 smithi161 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-mds-i[63897]: starting mds.i at 2024-09-21T03:43:56.468 INFO:journalctl@ceph.mds.i.smithi161.stdout:Sep 21 03:43:56 smithi161 bash[63882]: 97ce66e526b24938fa3e9c132e2f84bc247cb2711e65c9e16e0dbed43fc833a8 2024-09-21T03:43:56.468 INFO:journalctl@ceph.mds.i.smithi161.stdout:Sep 21 03:43:56 smithi161 systemd[1]: Started Ceph mds.i for 51116d5c-77ca-11ef-baf5-efdc52797490. 2024-09-21T03:43:56.964 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:43:56 smithi039 ceph-mon[31211]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T03:43:56.964 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:43:56 smithi039 ceph-mon[31211]: from='client.? 172.21.15.39:0/2100168239' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:43:56.964 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:43:56 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:43:56.964 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:43:56 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:43:56.964 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:43:56 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:43:56.964 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:43:56 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T03:43:56.964 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:43:56 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T03:43:56.964 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:43:56 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:43:57.218 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:43:56 smithi161 ceph-mon[34533]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T03:43:57.218 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:43:56 smithi161 ceph-mon[34533]: from='client.? 172.21.15.39:0/2100168239' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:43:57.218 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:43:56 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:43:57.218 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:43:56 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:43:57.218 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:43:56 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:43:57.218 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:43:56 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T03:43:57.218 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:43:56 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T03:43:57.218 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:43:56 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:43:57.224 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:43:56 smithi150 ceph-mon[34289]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T03:43:57.224 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:43:56 smithi150 ceph-mon[34289]: from='client.? 172.21.15.39:0/2100168239' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:43:57.225 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:43:56 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:43:57.225 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:43:56 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:43:57.225 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:43:56 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:43:57.225 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:43:56 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T03:43:57.225 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:43:56 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T03:43:57.225 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:43:56 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:43:57.834 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:43:57 smithi039 ceph-mon[31211]: Deploying daemon mds.g on smithi039 2024-09-21T03:43:57.834 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:43:57 smithi039 ceph-mon[31211]: mds.? [v2:172.21.15.161:6838/2190107270,v1:172.21.15.161:6839/2190107270] up:boot 2024-09-21T03:43:57.834 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:43:57 smithi039 ceph-mon[31211]: fsmap 9 up:standby 2024-09-21T03:43:57.834 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:43:57 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "i"} : dispatch 2024-09-21T03:43:58.218 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:43:57 smithi161 ceph-mon[34533]: Deploying daemon mds.g on smithi039 2024-09-21T03:43:58.218 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:43:57 smithi161 ceph-mon[34533]: mds.? [v2:172.21.15.161:6838/2190107270,v1:172.21.15.161:6839/2190107270] up:boot 2024-09-21T03:43:58.218 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:43:57 smithi161 ceph-mon[34533]: fsmap 9 up:standby 2024-09-21T03:43:58.218 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:43:57 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "i"} : dispatch 2024-09-21T03:43:58.224 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:43:57 smithi150 ceph-mon[34289]: Deploying daemon mds.g on smithi039 2024-09-21T03:43:58.224 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:43:57 smithi150 ceph-mon[34289]: mds.? [v2:172.21.15.161:6838/2190107270,v1:172.21.15.161:6839/2190107270] up:boot 2024-09-21T03:43:58.224 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:43:57 smithi150 ceph-mon[34289]: fsmap 9 up:standby 2024-09-21T03:43:58.225 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:43:57 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "i"} : dispatch 2024-09-21T03:43:58.582 INFO:journalctl@ceph.mds.g.smithi039.stdout:Sep 21 03:43:58 smithi039 systemd[1]: Starting Ceph mds.g for 51116d5c-77ca-11ef-baf5-efdc52797490... 2024-09-21T03:43:58.660 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 51116d5c-77ca-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T03:43:58.833 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:43:58 smithi039 ceph-mon[31211]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T03:43:58.834 INFO:journalctl@ceph.mds.g.smithi039.stdout:Sep 21 03:43:58 smithi039 podman[73179]: 2024-09-21 03:43:58.580164993 +0000 UTC m=+0.106961315 container create c75c7c7aee9065a806a56bf27d116525d838d15b97a3cc62edffaefaa579ddb9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-51116d5c-77ca-11ef-baf5-efdc52797490-mds-g, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_CLEAN=True, ceph=True) 2024-09-21T03:43:58.834 INFO:journalctl@ceph.mds.g.smithi039.stdout:Sep 21 03:43:58 smithi039 podman[73179]: 2024-09-21 03:43:58.489388698 +0000 UTC m=+0.016185037 image pull ddce307e1f3263fb077b4f3902e15660050631dad4d2823a20ee3f72bf18d977 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262 2024-09-21T03:43:58.834 INFO:journalctl@ceph.mds.g.smithi039.stdout:Sep 21 03:43:58 smithi039 podman[73179]: 2024-09-21 03:43:58.782855735 +0000 UTC m=+0.309652076 container init c75c7c7aee9065a806a56bf27d116525d838d15b97a3cc62edffaefaa579ddb9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-51116d5c-77ca-11ef-baf5-efdc52797490-mds-g, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_CLEAN=True, ceph=True, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731) 2024-09-21T03:43:58.834 INFO:journalctl@ceph.mds.g.smithi039.stdout:Sep 21 03:43:58 smithi039 podman[73179]: 2024-09-21 03:43:58.78682837 +0000 UTC m=+0.313624685 container start c75c7c7aee9065a806a56bf27d116525d838d15b97a3cc62edffaefaa579ddb9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-51116d5c-77ca-11ef-baf5-efdc52797490-mds-g, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.build-date=20240731, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, ceph=True) 2024-09-21T03:43:58.834 INFO:journalctl@ceph.mds.g.smithi039.stdout:Sep 21 03:43:58 smithi039 ceph-mds[73216]: -- 172.21.15.39:0/1368368201 <== mon.0 v2:172.21.15.39:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5644379d1860 con 0x56443bb79a80 2024-09-21T03:43:58.834 INFO:journalctl@ceph.mds.g.smithi039.stdout:Sep 21 03:43:58 smithi039 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-mds-g[73194]: starting mds.g at 2024-09-21T03:43:58.952 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/51116d5c-77ca-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:43:59.205 INFO:journalctl@ceph.mds.g.smithi039.stdout:Sep 21 03:43:58 smithi039 bash[73179]: c75c7c7aee9065a806a56bf27d116525d838d15b97a3cc62edffaefaa579ddb9 2024-09-21T03:43:59.205 INFO:journalctl@ceph.mds.g.smithi039.stdout:Sep 21 03:43:58 smithi039 systemd[1]: Started Ceph mds.g for 51116d5c-77ca-11ef-baf5-efdc52797490. 2024-09-21T03:43:59.217 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:43:58 smithi161 ceph-mon[34533]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T03:43:59.224 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:43:58 smithi150 ceph-mon[34289]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T03:43:59.452 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:59.448+0000 7f10fb097640 1 -- 172.21.15.39:0/2353085968 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f10f4102a60 msgr2=0x7f10f4102e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:59.452 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:59.448+0000 7f10fb097640 1 --2- 172.21.15.39:0/2353085968 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f10f4102a60 0x7f10f4102e70 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f10e8009900 tx=0x7f10e802a410 comp rx=0 tx=0).stop 2024-09-21T03:43:59.452 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:59.448+0000 7f10fb097640 1 -- 172.21.15.39:0/2353085968 shutdown_connections 2024-09-21T03:43:59.452 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:59.448+0000 7f10fb097640 1 --2- 172.21.15.39:0/2353085968 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f10f4108cf0 0x7f10f410f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:59.452 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:59.448+0000 7f10fb097640 1 --2- 172.21.15.39:0/2353085968 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f10f41083a0 0x7f10f41087b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:59.452 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:59.448+0000 7f10fb097640 1 --2- 172.21.15.39:0/2353085968 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f10f4102a60 0x7f10f4102e70 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:59.452 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:59.448+0000 7f10fb097640 1 -- 172.21.15.39:0/2353085968 >> 172.21.15.39:0/2353085968 conn(0x7f10f4079730 msgr2=0x7f10f4079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:43:59.452 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:59.448+0000 7f10fb097640 1 -- 172.21.15.39:0/2353085968 shutdown_connections 2024-09-21T03:43:59.452 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:59.448+0000 7f10fb097640 1 -- 172.21.15.39:0/2353085968 wait complete. 2024-09-21T03:43:59.452 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:59.449+0000 7f10fb097640 1 Processor -- start 2024-09-21T03:43:59.453 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:59.449+0000 7f10fb097640 1 -- start start 2024-09-21T03:43:59.453 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:59.449+0000 7f10fb097640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f10f4102a60 0x7f10f4104480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:59.453 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:59.449+0000 7f10fb097640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f10f41083a0 0x7f10f41049c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:59.453 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:59.449+0000 7f10fb097640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f10f4108cf0 0x7f10f4104f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:59.453 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:59.449+0000 7f10fb097640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f10f41129c0 con 0x7f10f41083a0 2024-09-21T03:43:59.453 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:59.449+0000 7f10fb097640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f10f4112840 con 0x7f10f4108cf0 2024-09-21T03:43:59.453 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:59.449+0000 7f10fb097640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f10f4112b40 con 0x7f10f4102a60 2024-09-21T03:43:59.453 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:59.450+0000 7f10f3fff640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f10f41083a0 0x7f10f41049c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:59.453 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:59.450+0000 7f10f8e33640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f10f4102a60 0x7f10f4104480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:59.453 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:59.450+0000 7f10f3fff640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f10f41083a0 0x7f10f41049c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:57824/0 (socket says 172.21.15.39:57824) 2024-09-21T03:43:59.453 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:59.450+0000 7f10f8e33640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f10f4102a60 0x7f10f4104480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.39:37234/0 (socket says 172.21.15.39:37234) 2024-09-21T03:43:59.453 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:59.450+0000 7f10f3fff640 1 -- 172.21.15.39:0/1437035531 learned_addr learned my addr 172.21.15.39:0/1437035531 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:43:59.453 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:59.450+0000 7f10f9634640 1 --2- 172.21.15.39:0/1437035531 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f10f4108cf0 0x7f10f4104f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:59.453 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:59.450+0000 7f10f3fff640 1 -- 172.21.15.39:0/1437035531 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f10f4102a60 msgr2=0x7f10f4104480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:59.454 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:59.450+0000 7f10f3fff640 1 --2- 172.21.15.39:0/1437035531 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f10f4102a60 0x7f10f4104480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:59.454 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:59.450+0000 7f10f3fff640 1 -- 172.21.15.39:0/1437035531 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f10f4108cf0 msgr2=0x7f10f4104f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:59.454 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:59.450+0000 7f10f3fff640 1 --2- 172.21.15.39:0/1437035531 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f10f4108cf0 0x7f10f4104f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:59.454 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:59.450+0000 7f10f3fff640 1 -- 172.21.15.39:0/1437035531 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f10f41a86f0 con 0x7f10f41083a0 2024-09-21T03:43:59.454 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:59.450+0000 7f10f3fff640 1 --2- 172.21.15.39:0/1437035531 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f10f41083a0 0x7f10f41049c0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f10e000b450 tx=0x7f10e000b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:43:59.454 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:59.450+0000 7f10f1ffb640 1 -- 172.21.15.39:0/1437035531 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10e0015020 con 0x7f10f41083a0 2024-09-21T03:43:59.454 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:59.450+0000 7f10f1ffb640 1 -- 172.21.15.39:0/1437035531 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f10e0011d40 con 0x7f10f41083a0 2024-09-21T03:43:59.455 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:59.450+0000 7f10fb097640 1 -- 172.21.15.39:0/1437035531 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f10f41a89e0 con 0x7f10f41083a0 2024-09-21T03:43:59.455 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:59.451+0000 7f10f1ffb640 1 -- 172.21.15.39:0/1437035531 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10e000f340 con 0x7f10f41083a0 2024-09-21T03:43:59.455 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:59.451+0000 7f10fb097640 1 -- 172.21.15.39:0/1437035531 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f10f41a8fa0 con 0x7f10f41083a0 2024-09-21T03:43:59.455 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:59.452+0000 7f10fb097640 1 -- 172.21.15.39:0/1437035531 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f10f4103ce0 con 0x7f10f41083a0 2024-09-21T03:43:59.456 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:59.455+0000 7f10f1ffb640 1 -- 172.21.15.39:0/1437035531 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 21) ==== 154529+0+0 (secure 0 0 0) 0x7f10e0018020 con 0x7f10f41083a0 2024-09-21T03:43:59.457 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:59.456+0000 7f10f1ffb640 1 --2- 172.21.15.39:0/1437035531 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f10e40b6560 0x7f10e40b8a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:59.457 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:59.456+0000 7f10f8e33640 1 --2- 172.21.15.39:0/1437035531 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f10e40b6560 0x7f10e40b8a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:59.457 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:59.456+0000 7f10f1ffb640 1 -- 172.21.15.39:0/1437035531 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 7342+0+0 (secure 0 0 0) 0x7f10e00e6e60 con 0x7f10f41083a0 2024-09-21T03:43:59.458 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:59.456+0000 7f10f8e33640 1 --2- 172.21.15.39:0/1437035531 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f10e40b6560 0x7f10e40b8a70 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f10e8000c00 tx=0x7f10e8031400 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:43:59.462 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:59.461+0000 7f10f1ffb640 1 -- 172.21.15.39:0/1437035531 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f10e001f020 con 0x7f10f41083a0 2024-09-21T03:43:59.695 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:59.693+0000 7f10fb097640 1 -- 172.21.15.39:0/1437035531 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f10f4102e70 con 0x7f10f41083a0 2024-09-21T03:43:59.695 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:59.694+0000 7f10f1ffb640 1 -- 172.21.15.39:0/1437035531 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 10 v10) ==== 94+0+8052 (secure 0 0 0) 0x7f10f4102e70 con 0x7f10f41083a0 2024-09-21T03:43:59.696 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T03:43:59.696 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":10,"btime":"2024-09-21T03:43:56:744718+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14790,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6835/508995295","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":508995295},{"type":"v1","addr":"172.21.15.39:6835","nonce":508995295}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14802,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6837/3915912607","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":3915912607},{"type":"v1","addr":"172.21.15.39:6837","nonce":3915912607}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24599,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6833/3114643217","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":3114643217},{"type":"v1","addr":"172.21.15.150:6833","nonce":3114643217}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24601,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6833/2963990654","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":2963990654},{"type":"v1","addr":"172.21.15.161:6833","nonce":2963990654}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24632,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6835/724594008","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6834","nonce":724594008},{"type":"v1","addr":"172.21.15.150:6835","nonce":724594008}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24640,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6835/3747882110","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":3747882110},{"type":"v1","addr":"172.21.15.161:6835","nonce":3747882110}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24646,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6837/3451544597","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6836","nonce":3451544597},{"type":"v1","addr":"172.21.15.161:6837","nonce":3451544597}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24659,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6837/4017043144","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6836","nonce":4017043144},{"type":"v1","addr":"172.21.15.150:6837","nonce":4017043144}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24667,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6839/2190107270","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6838","nonce":2190107270},{"type":"v1","addr":"172.21.15.161:6839","nonce":2190107270}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[]} 2024-09-21T03:43:59.696 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 10 2024-09-21T03:43:59.698 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:59.696+0000 7f10fb097640 1 -- 172.21.15.39:0/1437035531 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f10e40b6560 msgr2=0x7f10e40b8a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:59.698 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:59.696+0000 7f10fb097640 1 --2- 172.21.15.39:0/1437035531 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f10e40b6560 0x7f10e40b8a70 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f10e8000c00 tx=0x7f10e8031400 comp rx=0 tx=0).stop 2024-09-21T03:43:59.698 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:59.696+0000 7f10fb097640 1 -- 172.21.15.39:0/1437035531 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f10f41083a0 msgr2=0x7f10f41049c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:59.698 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:59.696+0000 7f10fb097640 1 --2- 172.21.15.39:0/1437035531 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f10f41083a0 0x7f10f41049c0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f10e000b450 tx=0x7f10e000b910 comp rx=0 tx=0).stop 2024-09-21T03:43:59.699 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:59.697+0000 7f10fb097640 1 -- 172.21.15.39:0/1437035531 shutdown_connections 2024-09-21T03:43:59.699 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:59.697+0000 7f10fb097640 1 --2- 172.21.15.39:0/1437035531 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f10e40b6560 0x7f10e40b8a70 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:59.699 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:59.697+0000 7f10fb097640 1 --2- 172.21.15.39:0/1437035531 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f10f4108cf0 0x7f10f4104f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:59.699 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:59.697+0000 7f10fb097640 1 --2- 172.21.15.39:0/1437035531 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f10f41083a0 0x7f10f41049c0 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:59.699 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:59.697+0000 7f10fb097640 1 --2- 172.21.15.39:0/1437035531 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f10f4102a60 0x7f10f4104480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:59.699 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:59.697+0000 7f10fb097640 1 -- 172.21.15.39:0/1437035531 >> 172.21.15.39:0/1437035531 conn(0x7f10f4079730 msgr2=0x7f10f4106810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:43:59.699 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:59.697+0000 7f10fb097640 1 -- 172.21.15.39:0/1437035531 shutdown_connections 2024-09-21T03:43:59.699 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:43:59.697+0000 7f10fb097640 1 -- 172.21.15.39:0/1437035531 wait complete. 2024-09-21T03:44:00.084 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:43:59 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:44:00.084 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:43:59 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:44:00.084 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:43:59 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:44:00.084 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:43:59 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T03:44:00.084 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:43:59 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T03:44:00.084 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:43:59 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:44:00.084 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:43:59 smithi039 ceph-mon[31211]: Deploying daemon mds.a on smithi039 2024-09-21T03:44:00.084 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:43:59 smithi039 ceph-mon[31211]: from='client.? 172.21.15.39:0/1437035531' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:44:00.084 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:43:59 smithi039 ceph-mon[31211]: mds.? [v2:172.21.15.39:6838/185509192,v1:172.21.15.39:6839/185509192] up:boot 2024-09-21T03:44:00.084 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:43:59 smithi039 ceph-mon[31211]: fsmap 10 up:standby 2024-09-21T03:44:00.084 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:43:59 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "g"} : dispatch 2024-09-21T03:44:00.218 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:43:59 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:44:00.218 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:43:59 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:44:00.218 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:43:59 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:44:00.218 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:43:59 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T03:44:00.218 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:43:59 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T03:44:00.218 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:43:59 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:44:00.218 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:43:59 smithi161 ceph-mon[34533]: Deploying daemon mds.a on smithi039 2024-09-21T03:44:00.218 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:43:59 smithi161 ceph-mon[34533]: from='client.? 172.21.15.39:0/1437035531' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:44:00.218 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:43:59 smithi161 ceph-mon[34533]: mds.? [v2:172.21.15.39:6838/185509192,v1:172.21.15.39:6839/185509192] up:boot 2024-09-21T03:44:00.219 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:43:59 smithi161 ceph-mon[34533]: fsmap 10 up:standby 2024-09-21T03:44:00.219 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:43:59 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "g"} : dispatch 2024-09-21T03:44:00.224 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:43:59 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:44:00.225 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:43:59 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:44:00.225 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:43:59 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:44:00.225 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:43:59 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T03:44:00.225 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:43:59 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T03:44:00.225 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:43:59 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:44:00.225 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:43:59 smithi150 ceph-mon[34289]: Deploying daemon mds.a on smithi039 2024-09-21T03:44:00.225 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:43:59 smithi150 ceph-mon[34289]: from='client.? 172.21.15.39:0/1437035531' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:44:00.225 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:43:59 smithi150 ceph-mon[34289]: mds.? [v2:172.21.15.39:6838/185509192,v1:172.21.15.39:6839/185509192] up:boot 2024-09-21T03:44:00.225 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:43:59 smithi150 ceph-mon[34289]: fsmap 10 up:standby 2024-09-21T03:44:00.225 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:43:59 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "g"} : dispatch 2024-09-21T03:44:01.217 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:44:00 smithi161 ceph-mon[34533]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T03:44:01.224 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:44:00 smithi150 ceph-mon[34289]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T03:44:01.265 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:44:00 smithi039 ceph-mon[31211]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T03:44:01.834 INFO:journalctl@ceph.mds.a.smithi039.stdout:Sep 21 03:44:01 smithi039 systemd[1]: Starting Ceph mds.a for 51116d5c-77ca-11ef-baf5-efdc52797490... 2024-09-21T03:44:02.225 INFO:journalctl@ceph.mds.a.smithi039.stdout:Sep 21 03:44:01 smithi039 podman[73561]: 2024-09-21 03:44:01.875516151 +0000 UTC m=+0.016388455 image pull ddce307e1f3263fb077b4f3902e15660050631dad4d2823a20ee3f72bf18d977 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262 2024-09-21T03:44:02.225 INFO:journalctl@ceph.mds.a.smithi039.stdout:Sep 21 03:44:01 smithi039 podman[73561]: 2024-09-21 03:44:01.975660958 +0000 UTC m=+0.116533247 container create 4d6edea429af090cfd57687082a277ab3490d8152c67db4f10ef4aac87969fb9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-51116d5c-77ca-11ef-baf5-efdc52797490-mds-a, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, ceph=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, GIT_CLEAN=True) 2024-09-21T03:44:02.225 INFO:journalctl@ceph.mds.a.smithi039.stdout:Sep 21 03:44:02 smithi039 podman[73561]: 2024-09-21 03:44:02.220465182 +0000 UTC m=+0.361337478 container init 4d6edea429af090cfd57687082a277ab3490d8152c67db4f10ef4aac87969fb9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-51116d5c-77ca-11ef-baf5-efdc52797490-mds-a, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.schema-version=1.0, ceph=True) 2024-09-21T03:44:02.225 INFO:journalctl@ceph.mds.a.smithi039.stdout:Sep 21 03:44:02 smithi039 podman[73561]: 2024-09-21 03:44:02.223418501 +0000 UTC m=+0.364290792 container start 4d6edea429af090cfd57687082a277ab3490d8152c67db4f10ef4aac87969fb9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-51116d5c-77ca-11ef-baf5-efdc52797490-mds-a, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, GIT_CLEAN=True, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , ceph=True) 2024-09-21T03:44:02.369 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 51116d5c-77ca-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T03:44:02.583 INFO:journalctl@ceph.mds.a.smithi039.stdout:Sep 21 03:44:02 smithi039 ceph-mds[73580]: -- 172.21.15.39:0/1475681544 <== mon.0 v2:172.21.15.39:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55a61ba87860 con 0x55a61fc2fa80 2024-09-21T03:44:02.584 INFO:journalctl@ceph.mds.a.smithi039.stdout:Sep 21 03:44:02 smithi039 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-mds-a[73576]: starting mds.a at 2024-09-21T03:44:02.584 INFO:journalctl@ceph.mds.a.smithi039.stdout:Sep 21 03:44:02 smithi039 bash[73561]: 4d6edea429af090cfd57687082a277ab3490d8152c67db4f10ef4aac87969fb9 2024-09-21T03:44:02.584 INFO:journalctl@ceph.mds.a.smithi039.stdout:Sep 21 03:44:02 smithi039 systemd[1]: Started Ceph mds.a for 51116d5c-77ca-11ef-baf5-efdc52797490. 2024-09-21T03:44:02.631 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/51116d5c-77ca-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:44:02.997 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:44:02 smithi039 ceph-mon[31211]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T03:44:02.997 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:44:02 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:44:02.997 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:44:02 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:44:02.997 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:44:02 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:44:02.997 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:44:02 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T03:44:02.997 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:44:02 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T03:44:02.998 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:44:02 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:44:03.056 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:44:02 smithi150 ceph-mon[34289]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T03:44:03.056 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:44:02 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:44:03.056 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:44:02 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:44:03.056 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:44:02 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:44:03.056 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:44:02 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T03:44:03.056 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:44:02 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T03:44:03.057 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:44:02 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:44:03.091 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:03.089+0000 7f0a1559d640 1 -- 172.21.15.39:0/404243260 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0a101016d0 msgr2=0x7f0a10101ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:03.091 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:03.089+0000 7f0a1559d640 1 --2- 172.21.15.39:0/404243260 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0a101016d0 0x7f0a10101ae0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f0a04009930 tx=0x7f0a0402a420 comp rx=0 tx=0).stop 2024-09-21T03:44:03.091 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:03.089+0000 7f0a1559d640 1 -- 172.21.15.39:0/404243260 shutdown_connections 2024-09-21T03:44:03.091 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:03.089+0000 7f0a1559d640 1 --2- 172.21.15.39:0/404243260 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0a10102020 0x7f0a10110390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:03.091 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:03.089+0000 7f0a1559d640 1 --2- 172.21.15.39:0/404243260 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0a101016d0 0x7f0a10101ae0 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:03.091 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:03.089+0000 7f0a1559d640 1 --2- 172.21.15.39:0/404243260 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0a101096c0 0x7f0a10109ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:03.092 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:03.089+0000 7f0a1559d640 1 -- 172.21.15.39:0/404243260 >> 172.21.15.39:0/404243260 conn(0x7f0a10078680 msgr2=0x7f0a10078ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:44:03.092 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:03.089+0000 7f0a1559d640 1 -- 172.21.15.39:0/404243260 shutdown_connections 2024-09-21T03:44:03.092 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:03.089+0000 7f0a1559d640 1 -- 172.21.15.39:0/404243260 wait complete. 2024-09-21T03:44:03.094 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:03.090+0000 7f0a1559d640 1 Processor -- start 2024-09-21T03:44:03.094 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:03.090+0000 7f0a1559d640 1 -- start start 2024-09-21T03:44:03.095 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:03.090+0000 7f0a1559d640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0a101016d0 0x7f0a1019a350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:03.095 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:03.090+0000 7f0a1559d640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0a10102020 0x7f0a1019a8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:03.095 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:03.090+0000 7f0a1559d640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0a101096c0 0x7f0a1019fb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:03.095 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:03.090+0000 7f0a1559d640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f0a101138a0 con 0x7f0a101016d0 2024-09-21T03:44:03.095 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:03.090+0000 7f0a1559d640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f0a10113720 con 0x7f0a101096c0 2024-09-21T03:44:03.095 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:03.090+0000 7f0a1559d640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f0a10113a20 con 0x7f0a10102020 2024-09-21T03:44:03.095 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:03.091+0000 7f0a0ffff640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0a101016d0 0x7f0a1019a350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:03.095 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:03.091+0000 7f0a0ffff640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0a101016d0 0x7f0a1019a350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:57862/0 (socket says 172.21.15.39:57862) 2024-09-21T03:44:03.095 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:03.091+0000 7f0a0ffff640 1 -- 172.21.15.39:0/2590423504 learned_addr learned my addr 172.21.15.39:0/2590423504 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:44:03.096 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:03.091+0000 7f0a0f7fe640 1 --2- 172.21.15.39:0/2590423504 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0a10102020 0x7f0a1019a8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:03.096 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:03.091+0000 7f0a14d9c640 1 --2- 172.21.15.39:0/2590423504 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0a101096c0 0x7f0a1019fb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:03.096 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:03.091+0000 7f0a0ffff640 1 -- 172.21.15.39:0/2590423504 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0a10102020 msgr2=0x7f0a1019a8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:03.096 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:03.091+0000 7f0a0ffff640 1 --2- 172.21.15.39:0/2590423504 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0a10102020 0x7f0a1019a8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:03.096 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:03.091+0000 7f0a0ffff640 1 -- 172.21.15.39:0/2590423504 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0a101096c0 msgr2=0x7f0a1019fb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:03.096 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:03.091+0000 7f0a0ffff640 1 --2- 172.21.15.39:0/2590423504 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0a101096c0 0x7f0a1019fb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:03.096 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:03.091+0000 7f0a0ffff640 1 -- 172.21.15.39:0/2590423504 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0a101a02c0 con 0x7f0a101016d0 2024-09-21T03:44:03.096 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:03.091+0000 7f0a0ffff640 1 --2- 172.21.15.39:0/2590423504 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0a101016d0 0x7f0a1019a350 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f0a0000e9f0 tx=0x7f0a0000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:44:03.097 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:03.091+0000 7f0a0d7fa640 1 -- 172.21.15.39:0/2590423504 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a0000cd80 con 0x7f0a101016d0 2024-09-21T03:44:03.097 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:03.091+0000 7f0a0d7fa640 1 -- 172.21.15.39:0/2590423504 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0a000108e0 con 0x7f0a101016d0 2024-09-21T03:44:03.097 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:03.091+0000 7f0a1559d640 1 -- 172.21.15.39:0/2590423504 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0a101a05b0 con 0x7f0a101016d0 2024-09-21T03:44:03.097 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:03.091+0000 7f0a0d7fa640 1 -- 172.21.15.39:0/2590423504 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a00018980 con 0x7f0a101016d0 2024-09-21T03:44:03.097 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:03.091+0000 7f0a1559d640 1 -- 172.21.15.39:0/2590423504 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0a101a6d10 con 0x7f0a101016d0 2024-09-21T03:44:03.097 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:03.093+0000 7f0a1559d640 1 -- 172.21.15.39:0/2590423504 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0a10102a00 con 0x7f0a101016d0 2024-09-21T03:44:03.098 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:03.094+0000 7f0a0d7fa640 1 -- 172.21.15.39:0/2590423504 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 21) ==== 154529+0+0 (secure 0 0 0) 0x7f0a00010430 con 0x7f0a101016d0 2024-09-21T03:44:03.098 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:03.095+0000 7f0a0d7fa640 1 --2- 172.21.15.39:0/2590423504 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f09f80b6560 0x7f09f80b8a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:03.098 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:03.095+0000 7f0a0f7fe640 1 --2- 172.21.15.39:0/2590423504 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f09f80b6560 0x7f09f80b8a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:03.099 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:03.095+0000 7f0a0d7fa640 1 -- 172.21.15.39:0/2590423504 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 7342+0+0 (secure 0 0 0) 0x7f0a000e76e0 con 0x7f0a101016d0 2024-09-21T03:44:03.099 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:03.096+0000 7f0a0f7fe640 1 --2- 172.21.15.39:0/2590423504 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f09f80b6560 0x7f09f80b8a70 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f0a04031440 tx=0x7f0a04000bc0 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:44:03.100 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:03.098+0000 7f0a0d7fa640 1 -- 172.21.15.39:0/2590423504 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f0a000b2720 con 0x7f0a101016d0 2024-09-21T03:44:03.325 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:03.323+0000 7f0a1559d640 1 -- 172.21.15.39:0/2590423504 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f0a10101ae0 con 0x7f0a101016d0 2024-09-21T03:44:03.326 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:03.324+0000 7f0a0d7fa640 1 -- 172.21.15.39:0/2590423504 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 12 v12) ==== 94+0+9701 (secure 0 0 0) 0x7f0a10101ae0 con 0x7f0a101016d0 2024-09-21T03:44:03.326 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T03:44:03.326 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":12,"btime":"2024-09-21T03:44:02:959258+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14790,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6835/508995295","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":508995295},{"type":"v1","addr":"172.21.15.39:6835","nonce":508995295}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14802,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6837/3915912607","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":3915912607},{"type":"v1","addr":"172.21.15.39:6837","nonce":3915912607}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14856,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6839/185509192","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6838","nonce":185509192},{"type":"v1","addr":"172.21.15.39:6839","nonce":185509192}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14868,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6841/4247190456","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6840","nonce":4247190456},{"type":"v1","addr":"172.21.15.39:6841","nonce":4247190456}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24599,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6833/3114643217","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":3114643217},{"type":"v1","addr":"172.21.15.150:6833","nonce":3114643217}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24601,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6833/2963990654","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":2963990654},{"type":"v1","addr":"172.21.15.161:6833","nonce":2963990654}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24632,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6835/724594008","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6834","nonce":724594008},{"type":"v1","addr":"172.21.15.150:6835","nonce":724594008}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24640,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6835/3747882110","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":3747882110},{"type":"v1","addr":"172.21.15.161:6835","nonce":3747882110}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24646,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6837/3451544597","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6836","nonce":3451544597},{"type":"v1","addr":"172.21.15.161:6837","nonce":3451544597}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24659,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6837/4017043144","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6836","nonce":4017043144},{"type":"v1","addr":"172.21.15.150:6837","nonce":4017043144}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24667,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6839/2190107270","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6838","nonce":2190107270},{"type":"v1","addr":"172.21.15.161:6839","nonce":2190107270}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[]} 2024-09-21T03:44:03.326 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 12 2024-09-21T03:44:03.328 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:03.326+0000 7f0a1559d640 1 -- 172.21.15.39:0/2590423504 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f09f80b6560 msgr2=0x7f09f80b8a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:03.328 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:03.326+0000 7f0a1559d640 1 --2- 172.21.15.39:0/2590423504 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f09f80b6560 0x7f09f80b8a70 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f0a04031440 tx=0x7f0a04000bc0 comp rx=0 tx=0).stop 2024-09-21T03:44:03.328 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:03.326+0000 7f0a1559d640 1 -- 172.21.15.39:0/2590423504 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0a101016d0 msgr2=0x7f0a1019a350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:03.328 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:03.326+0000 7f0a1559d640 1 --2- 172.21.15.39:0/2590423504 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0a101016d0 0x7f0a1019a350 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f0a0000e9f0 tx=0x7f0a0000eeb0 comp rx=0 tx=0).stop 2024-09-21T03:44:03.329 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:03.327+0000 7f0a1559d640 1 -- 172.21.15.39:0/2590423504 shutdown_connections 2024-09-21T03:44:03.329 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:03.327+0000 7f0a1559d640 1 --2- 172.21.15.39:0/2590423504 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f09f80b6560 0x7f09f80b8a70 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:03.329 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:03.327+0000 7f0a1559d640 1 --2- 172.21.15.39:0/2590423504 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0a101096c0 0x7f0a1019fb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:03.329 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:03.327+0000 7f0a1559d640 1 --2- 172.21.15.39:0/2590423504 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0a10102020 0x7f0a1019a8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:03.329 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:03.327+0000 7f0a1559d640 1 --2- 172.21.15.39:0/2590423504 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0a101016d0 0x7f0a1019a350 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:03.329 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:03.327+0000 7f0a1559d640 1 -- 172.21.15.39:0/2590423504 >> 172.21.15.39:0/2590423504 conn(0x7f0a10078680 msgr2=0x7f0a101002b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:44:03.329 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:03.327+0000 7f0a1559d640 1 -- 172.21.15.39:0/2590423504 shutdown_connections 2024-09-21T03:44:03.329 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:03.327+0000 7f0a1559d640 1 -- 172.21.15.39:0/2590423504 wait complete. 2024-09-21T03:44:03.468 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:44:02 smithi161 ceph-mon[34533]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T03:44:03.468 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:44:02 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:44:03.468 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:44:02 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:44:03.468 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:44:02 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:44:03.468 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:44:02 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T03:44:03.468 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:44:02 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T03:44:03.468 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:44:02 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:44:04.166 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:44:03 smithi150 ceph-mon[34289]: Deploying daemon mds.b on smithi150 2024-09-21T03:44:04.166 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:44:03 smithi150 ceph-mon[34289]: mds.? [v2:172.21.15.39:6840/4247190456,v1:172.21.15.39:6841/4247190456] up:boot 2024-09-21T03:44:04.166 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:44:03 smithi150 ceph-mon[34289]: fsmap 11 up:standby 2024-09-21T03:44:04.166 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:44:03 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "a"} : dispatch 2024-09-21T03:44:04.166 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:44:03 smithi150 ceph-mon[34289]: from='client.? 172.21.15.39:0/2590423504' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:44:04.334 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:44:03 smithi039 ceph-mon[31211]: Deploying daemon mds.b on smithi150 2024-09-21T03:44:04.334 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:44:03 smithi039 ceph-mon[31211]: mds.? [v2:172.21.15.39:6840/4247190456,v1:172.21.15.39:6841/4247190456] up:boot 2024-09-21T03:44:04.334 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:44:03 smithi039 ceph-mon[31211]: fsmap 11 up:standby 2024-09-21T03:44:04.334 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:44:03 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "a"} : dispatch 2024-09-21T03:44:04.334 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:44:03 smithi039 ceph-mon[31211]: from='client.? 172.21.15.39:0/2590423504' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:44:04.468 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:44:03 smithi161 ceph-mon[34533]: Deploying daemon mds.b on smithi150 2024-09-21T03:44:04.468 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:44:03 smithi161 ceph-mon[34533]: mds.? [v2:172.21.15.39:6840/4247190456,v1:172.21.15.39:6841/4247190456] up:boot 2024-09-21T03:44:04.468 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:44:03 smithi161 ceph-mon[34533]: fsmap 11 up:standby 2024-09-21T03:44:04.468 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:44:03 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "a"} : dispatch 2024-09-21T03:44:04.468 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:44:03 smithi161 ceph-mon[34533]: from='client.? 172.21.15.39:0/2590423504' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:44:04.474 INFO:journalctl@ceph.mds.b.smithi150.stdout:Sep 21 03:44:04 smithi150 systemd[1]: Starting Ceph mds.b for 51116d5c-77ca-11ef-baf5-efdc52797490... 2024-09-21T03:44:04.855 INFO:journalctl@ceph.mds.b.smithi150.stdout:Sep 21 03:44:04 smithi150 podman[63506]: 2024-09-21 03:44:04.473108224 +0000 UTC m=+0.016099247 image pull ddce307e1f3263fb077b4f3902e15660050631dad4d2823a20ee3f72bf18d977 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262 2024-09-21T03:44:04.855 INFO:journalctl@ceph.mds.b.smithi150.stdout:Sep 21 03:44:04 smithi150 podman[63506]: 2024-09-21 03:44:04.614658056 +0000 UTC m=+0.157649063 container create 5600e78780f50d80188ae5ca31c15591bef32c07ce9c7f217de363a23bbad4ed (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-51116d5c-77ca-11ef-baf5-efdc52797490-mds-b, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2) 2024-09-21T03:44:04.855 INFO:journalctl@ceph.mds.b.smithi150.stdout:Sep 21 03:44:04 smithi150 podman[63506]: 2024-09-21 03:44:04.817025049 +0000 UTC m=+0.360016056 container init 5600e78780f50d80188ae5ca31c15591bef32c07ce9c7f217de363a23bbad4ed (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-51116d5c-77ca-11ef-baf5-efdc52797490-mds-b, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, ceph=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731) 2024-09-21T03:44:04.855 INFO:journalctl@ceph.mds.b.smithi150.stdout:Sep 21 03:44:04 smithi150 podman[63506]: 2024-09-21 03:44:04.819927562 +0000 UTC m=+0.362918570 container start 5600e78780f50d80188ae5ca31c15591bef32c07ce9c7f217de363a23bbad4ed (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-51116d5c-77ca-11ef-baf5-efdc52797490-mds-b, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_CLEAN=True, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23) 2024-09-21T03:44:05.225 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:44:04 smithi150 ceph-mon[34289]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T03:44:05.225 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:44:04 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:44:05.225 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:44:04 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:44:05.225 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:44:04 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:44:05.225 INFO:journalctl@ceph.mds.b.smithi150.stdout:Sep 21 03:44:04 smithi150 ceph-mds[63525]: -- 172.21.15.150:0/659421575 <== mon.2 v2:172.21.15.150:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55ab7ccfb860 con 0x55ab80ea3a80 2024-09-21T03:44:05.225 INFO:journalctl@ceph.mds.b.smithi150.stdout:Sep 21 03:44:04 smithi150 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-mds-b[63521]: starting mds.b at 2024-09-21T03:44:05.226 INFO:journalctl@ceph.mds.b.smithi150.stdout:Sep 21 03:44:04 smithi150 bash[63506]: 5600e78780f50d80188ae5ca31c15591bef32c07ce9c7f217de363a23bbad4ed 2024-09-21T03:44:05.226 INFO:journalctl@ceph.mds.b.smithi150.stdout:Sep 21 03:44:04 smithi150 systemd[1]: Started Ceph mds.b for 51116d5c-77ca-11ef-baf5-efdc52797490. 2024-09-21T03:44:05.333 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:44:04 smithi039 ceph-mon[31211]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T03:44:05.334 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:44:04 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:44:05.334 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:44:04 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:44:05.334 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:44:04 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:44:05.468 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:44:04 smithi161 ceph-mon[34533]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T03:44:05.468 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:44:04 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:44:05.468 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:44:04 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:44:05.468 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:44:04 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:44:05.983 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 51116d5c-77ca-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T03:44:06.271 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/51116d5c-77ca-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:44:06.334 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:44:05 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:44:06.334 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:44:05 smithi039 ceph-mon[31211]: mds.? [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] up:boot 2024-09-21T03:44:06.334 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:44:05 smithi039 ceph-mon[31211]: fsmap 12 up:standby 2024-09-21T03:44:06.334 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:44:05 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "b"} : dispatch 2024-09-21T03:44:06.334 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:44:05 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:44:06.334 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:44:05 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:44:06.334 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:44:05 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:44:06.468 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:44:05 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:44:06.468 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:44:05 smithi161 ceph-mon[34533]: mds.? [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] up:boot 2024-09-21T03:44:06.468 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:44:05 smithi161 ceph-mon[34533]: fsmap 12 up:standby 2024-09-21T03:44:06.468 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:44:05 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "b"} : dispatch 2024-09-21T03:44:06.468 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:44:05 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:44:06.468 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:44:05 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:44:06.468 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:44:05 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:44:06.474 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:44:05 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:44:06.474 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:44:05 smithi150 ceph-mon[34289]: mds.? [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] up:boot 2024-09-21T03:44:06.475 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:44:05 smithi150 ceph-mon[34289]: fsmap 12 up:standby 2024-09-21T03:44:06.475 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:44:05 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "b"} : dispatch 2024-09-21T03:44:06.475 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:44:05 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:44:06.475 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:44:05 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:44:06.475 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:44:05 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:44:06.777 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:06.775+0000 7f753c365640 1 -- 172.21.15.39:0/3667061843 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f75341083a0 msgr2=0x7f75341087b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:06.777 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:06.775+0000 7f753c365640 1 --2- 172.21.15.39:0/3667061843 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f75341083a0 0x7f75341087b0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f7528009900 tx=0x7f752802a410 comp rx=0 tx=0).stop 2024-09-21T03:44:06.778 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:06.775+0000 7f753c365640 1 -- 172.21.15.39:0/3667061843 shutdown_connections 2024-09-21T03:44:06.778 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:06.775+0000 7f753c365640 1 --2- 172.21.15.39:0/3667061843 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7534108cf0 0x7f753410f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:06.778 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:06.775+0000 7f753c365640 1 --2- 172.21.15.39:0/3667061843 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f75341083a0 0x7f75341087b0 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:06.778 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:06.775+0000 7f753c365640 1 --2- 172.21.15.39:0/3667061843 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7534102a60 0x7f7534102e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:06.778 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:06.775+0000 7f753c365640 1 -- 172.21.15.39:0/3667061843 >> 172.21.15.39:0/3667061843 conn(0x7f7534079730 msgr2=0x7f7534079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:44:06.778 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:06.775+0000 7f753c365640 1 -- 172.21.15.39:0/3667061843 shutdown_connections 2024-09-21T03:44:06.778 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:06.775+0000 7f753c365640 1 -- 172.21.15.39:0/3667061843 wait complete. 2024-09-21T03:44:06.778 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:06.776+0000 7f753c365640 1 Processor -- start 2024-09-21T03:44:06.778 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:06.776+0000 7f753c365640 1 -- start start 2024-09-21T03:44:06.779 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:06.776+0000 7f753c365640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7534102a60 0x7f753419b2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:06.779 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:06.776+0000 7f753c365640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f75341083a0 0x7f753419b810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:06.780 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:06.776+0000 7f753c365640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7534108cf0 0x7f753419bd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:06.780 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:06.776+0000 7f753c365640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f7534112960 con 0x7f7534108cf0 2024-09-21T03:44:06.780 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:06.776+0000 7f753c365640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f75341127e0 con 0x7f7534102a60 2024-09-21T03:44:06.780 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:06.776+0000 7f753c365640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f7534112ae0 con 0x7f75341083a0 2024-09-21T03:44:06.780 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:06.776+0000 7f753a902640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7534108cf0 0x7f753419bd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:06.780 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:06.776+0000 7f753a902640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7534108cf0 0x7f753419bd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:57872/0 (socket says 172.21.15.39:57872) 2024-09-21T03:44:06.780 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:06.776+0000 7f753a902640 1 -- 172.21.15.39:0/3833939389 learned_addr learned my addr 172.21.15.39:0/3833939389 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:44:06.780 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:06.776+0000 7f753a101640 1 --2- 172.21.15.39:0/3833939389 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7534102a60 0x7f753419b2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:06.780 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:06.777+0000 7f7539900640 1 --2- 172.21.15.39:0/3833939389 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f75341083a0 0x7f753419b810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:06.780 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:06.777+0000 7f753a902640 1 -- 172.21.15.39:0/3833939389 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f75341083a0 msgr2=0x7f753419b810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:06.780 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:06.777+0000 7f753a902640 1 --2- 172.21.15.39:0/3833939389 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f75341083a0 0x7f753419b810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:06.780 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:06.777+0000 7f753a902640 1 -- 172.21.15.39:0/3833939389 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7534102a60 msgr2=0x7f753419b2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:06.780 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:06.777+0000 7f753a902640 1 --2- 172.21.15.39:0/3833939389 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7534102a60 0x7f753419b2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:06.780 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:06.777+0000 7f753a902640 1 -- 172.21.15.39:0/3833939389 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f75341a65e0 con 0x7f7534108cf0 2024-09-21T03:44:06.781 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:06.777+0000 7f753a902640 1 --2- 172.21.15.39:0/3833939389 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7534108cf0 0x7f753419bd50 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f752400d950 tx=0x7f752400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:44:06.781 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:06.777+0000 7f75337fe640 1 -- 172.21.15.39:0/3833939389 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7524017030 con 0x7f7534108cf0 2024-09-21T03:44:06.781 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:06.777+0000 7f75337fe640 1 -- 172.21.15.39:0/3833939389 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7524004020 con 0x7f7534108cf0 2024-09-21T03:44:06.781 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:06.777+0000 7f753c365640 1 -- 172.21.15.39:0/3833939389 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f75341a68d0 con 0x7f7534108cf0 2024-09-21T03:44:06.781 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:06.777+0000 7f75337fe640 1 -- 172.21.15.39:0/3833939389 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7524003b50 con 0x7f7534108cf0 2024-09-21T03:44:06.781 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:06.777+0000 7f753c365640 1 -- 172.21.15.39:0/3833939389 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f75341a6d90 con 0x7f7534108cf0 2024-09-21T03:44:06.781 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:06.778+0000 7f753c365640 1 -- 172.21.15.39:0/3833939389 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7534102e70 con 0x7f7534108cf0 2024-09-21T03:44:06.782 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:06.780+0000 7f75337fe640 1 -- 172.21.15.39:0/3833939389 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 21) ==== 154529+0+0 (secure 0 0 0) 0x7f752401a070 con 0x7f7534108cf0 2024-09-21T03:44:06.783 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:06.781+0000 7f75337fe640 1 --2- 172.21.15.39:0/3833939389 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f751c0b6560 0x7f751c0b8a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:06.783 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:06.781+0000 7f753a101640 1 --2- 172.21.15.39:0/3833939389 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f751c0b6560 0x7f751c0b8a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:06.783 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:06.781+0000 7f75337fe640 1 -- 172.21.15.39:0/3833939389 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 7342+0+0 (secure 0 0 0) 0x7f75240e6d50 con 0x7f7534108cf0 2024-09-21T03:44:06.783 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:06.782+0000 7f753a101640 1 --2- 172.21.15.39:0/3833939389 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f751c0b6560 0x7f751c0b8a70 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f753406ba20 tx=0x7f752c009500 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:44:06.786 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:06.784+0000 7f75337fe640 1 -- 172.21.15.39:0/3833939389 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f75240ad9b0 con 0x7f7534108cf0 2024-09-21T03:44:07.022 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:07.020+0000 7f753c365640 1 -- 172.21.15.39:0/3833939389 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f753406b2d0 con 0x7f7534108cf0 2024-09-21T03:44:07.022 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:07.021+0000 7f75337fe640 1 -- 172.21.15.39:0/3833939389 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 13 v13) ==== 94+0+10530 (secure 0 0 0) 0x7f75240b1d90 con 0x7f7534108cf0 2024-09-21T03:44:07.023 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T03:44:07.023 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":13,"btime":"2024-09-21T03:44:04:981955+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14790,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6835/508995295","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":508995295},{"type":"v1","addr":"172.21.15.39:6835","nonce":508995295}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14802,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6837/3915912607","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":3915912607},{"type":"v1","addr":"172.21.15.39:6837","nonce":3915912607}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14856,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6839/185509192","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6838","nonce":185509192},{"type":"v1","addr":"172.21.15.39:6839","nonce":185509192}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14868,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6841/4247190456","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6840","nonce":4247190456},{"type":"v1","addr":"172.21.15.39:6841","nonce":4247190456}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24599,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6833/3114643217","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":3114643217},{"type":"v1","addr":"172.21.15.150:6833","nonce":3114643217}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24601,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6833/2963990654","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":2963990654},{"type":"v1","addr":"172.21.15.161:6833","nonce":2963990654}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24632,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6835/724594008","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6834","nonce":724594008},{"type":"v1","addr":"172.21.15.150:6835","nonce":724594008}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24640,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6835/3747882110","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":3747882110},{"type":"v1","addr":"172.21.15.161:6835","nonce":3747882110}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24646,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6837/3451544597","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6836","nonce":3451544597},{"type":"v1","addr":"172.21.15.161:6837","nonce":3451544597}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24659,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6837/4017043144","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6836","nonce":4017043144},{"type":"v1","addr":"172.21.15.150:6837","nonce":4017043144}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24667,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6839/2190107270","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6838","nonce":2190107270},{"type":"v1","addr":"172.21.15.161:6839","nonce":2190107270}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":24695,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6839/2949386907","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6838","nonce":2949386907},{"type":"v1","addr":"172.21.15.150:6839","nonce":2949386907}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13}],"filesystems":[]} 2024-09-21T03:44:07.023 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 13 2024-09-21T03:44:07.026 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:07.024+0000 7f753c365640 1 -- 172.21.15.39:0/3833939389 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f751c0b6560 msgr2=0x7f751c0b8a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:07.026 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:07.024+0000 7f753c365640 1 --2- 172.21.15.39:0/3833939389 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f751c0b6560 0x7f751c0b8a70 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f753406ba20 tx=0x7f752c009500 comp rx=0 tx=0).stop 2024-09-21T03:44:07.026 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:07.024+0000 7f753c365640 1 -- 172.21.15.39:0/3833939389 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7534108cf0 msgr2=0x7f753419bd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:07.026 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:07.024+0000 7f753c365640 1 --2- 172.21.15.39:0/3833939389 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7534108cf0 0x7f753419bd50 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f752400d950 tx=0x7f752400de10 comp rx=0 tx=0).stop 2024-09-21T03:44:07.026 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:07.024+0000 7f753c365640 1 -- 172.21.15.39:0/3833939389 shutdown_connections 2024-09-21T03:44:07.027 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:07.024+0000 7f753c365640 1 --2- 172.21.15.39:0/3833939389 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f751c0b6560 0x7f751c0b8a70 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:07.027 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:07.024+0000 7f753c365640 1 --2- 172.21.15.39:0/3833939389 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7534108cf0 0x7f753419bd50 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:07.027 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:07.024+0000 7f753c365640 1 --2- 172.21.15.39:0/3833939389 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f75341083a0 0x7f753419b810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:07.027 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:07.024+0000 7f753c365640 1 --2- 172.21.15.39:0/3833939389 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7534102a60 0x7f753419b2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:07.027 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:07.024+0000 7f753c365640 1 -- 172.21.15.39:0/3833939389 >> 172.21.15.39:0/3833939389 conn(0x7f7534079730 msgr2=0x7f75341044e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:44:07.027 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:07.025+0000 7f753c365640 1 -- 172.21.15.39:0/3833939389 shutdown_connections 2024-09-21T03:44:07.027 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:07.025+0000 7f753c365640 1 -- 172.21.15.39:0/3833939389 wait complete. 2024-09-21T03:44:07.334 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:44:07 smithi039 ceph-mon[31211]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T03:44:07.334 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:44:07 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:44:07.334 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:44:07 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:44:07.334 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:44:07 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:44:07.334 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:44:07 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:44:07.334 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:44:07 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:44:07.334 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:44:07 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:44:07.334 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:44:07 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:44:07.334 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:44:07 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:44:07.335 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:44:07 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:44:07.335 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:44:07 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:44:07.467 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:44:07 smithi161 ceph-mon[34533]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T03:44:07.468 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:44:07 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:44:07.468 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:44:07 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:44:07.468 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:44:07 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:44:07.468 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:44:07 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:44:07.468 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:44:07 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:44:07.468 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:44:07 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:44:07.468 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:44:07 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:44:07.468 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:44:07 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:44:07.468 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:44:07 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:44:07.468 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:44:07 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:44:07.474 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:44:07 smithi150 ceph-mon[34289]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T03:44:07.474 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:44:07 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:44:07.475 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:44:07 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:44:07.475 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:44:07 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:44:07.475 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:44:07 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:44:07.475 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:44:07 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:44:07.475 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:44:07 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:44:07.475 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:44:07 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:44:07.475 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:44:07 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:44:07.475 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:44:07 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:44:07.475 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:44:07 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:44:07.612 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 51116d5c-77ca-11ef-baf5-efdc52797490 -- ceph --format=json-pretty osd lspools 2024-09-21T03:44:08.333 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:44:08 smithi039 ceph-mon[31211]: from='client.? 172.21.15.39:0/3833939389' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:44:08.467 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:44:08 smithi161 ceph-mon[34533]: from='client.? 172.21.15.39:0/3833939389' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:44:08.474 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:44:08 smithi150 ceph-mon[34289]: from='client.? 172.21.15.39:0/3833939389' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:44:08.529 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/51116d5c-77ca-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:44:09.026 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:09.024+0000 7f86b0d37640 1 -- 172.21.15.39:0/2475095259 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f86ac105ae0 msgr2=0x7f86ac10f530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:09.026 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:09.024+0000 7f86b0d37640 1 --2- 172.21.15.39:0/2475095259 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f86ac105ae0 0x7f86ac10f530 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f86a4009ca0 tx=0x7f86a402a8c0 comp rx=0 tx=0).stop 2024-09-21T03:44:09.026 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:09.024+0000 7f86b0d37640 1 -- 172.21.15.39:0/2475095259 shutdown_connections 2024-09-21T03:44:09.026 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:09.024+0000 7f86b0d37640 1 --2- 172.21.15.39:0/2475095259 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f86ac105ae0 0x7f86ac10f530 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:09.026 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:09.024+0000 7f86b0d37640 1 --2- 172.21.15.39:0/2475095259 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f86ac105190 0x7f86ac1055a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:09.027 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:09.024+0000 7f86b0d37640 1 --2- 172.21.15.39:0/2475095259 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f86ac1031c0 0x7f86ac1035d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:09.027 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:09.024+0000 7f86b0d37640 1 -- 172.21.15.39:0/2475095259 >> 172.21.15.39:0/2475095259 conn(0x7f86ac077c60 msgr2=0x7f86ac0780a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:44:09.027 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:09.024+0000 7f86b0d37640 1 -- 172.21.15.39:0/2475095259 shutdown_connections 2024-09-21T03:44:09.027 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:09.024+0000 7f86b0d37640 1 -- 172.21.15.39:0/2475095259 wait complete. 2024-09-21T03:44:09.029 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:09.025+0000 7f86b0d37640 1 Processor -- start 2024-09-21T03:44:09.029 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:09.025+0000 7f86b0d37640 1 -- start start 2024-09-21T03:44:09.029 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:09.026+0000 7f86b0d37640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f86ac1031c0 0x7f86ac19b240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:09.029 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:09.026+0000 7f86b0d37640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f86ac105190 0x7f86ac19b780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:09.029 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:09.026+0000 7f86b0d37640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f86ac105ae0 0x7f86ac19bcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:09.029 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:09.026+0000 7f86b0d37640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f86ac1129d0 con 0x7f86ac105190 2024-09-21T03:44:09.029 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:09.026+0000 7f86b0d37640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f86ac112850 con 0x7f86ac1031c0 2024-09-21T03:44:09.029 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:09.026+0000 7f86b0d37640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f86ac112b50 con 0x7f86ac105ae0 2024-09-21T03:44:09.029 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:09.026+0000 7f86a9d9b640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f86ac105190 0x7f86ac19b780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:09.029 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:09.026+0000 7f86a9d9b640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f86ac105190 0x7f86ac19b780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:50732/0 (socket says 172.21.15.39:50732) 2024-09-21T03:44:09.029 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:09.026+0000 7f86a9d9b640 1 -- 172.21.15.39:0/180476666 learned_addr learned my addr 172.21.15.39:0/180476666 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:44:09.030 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:09.026+0000 7f86aad9d640 1 --2- 172.21.15.39:0/180476666 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f86ac105ae0 0x7f86ac19bcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:09.030 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:09.026+0000 7f86aa59c640 1 --2- 172.21.15.39:0/180476666 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f86ac1031c0 0x7f86ac19b240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:09.030 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:09.026+0000 7f86a9d9b640 1 -- 172.21.15.39:0/180476666 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f86ac105ae0 msgr2=0x7f86ac19bcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:09.030 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:09.026+0000 7f86a9d9b640 1 --2- 172.21.15.39:0/180476666 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f86ac105ae0 0x7f86ac19bcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:09.030 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:09.026+0000 7f86a9d9b640 1 -- 172.21.15.39:0/180476666 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f86ac1031c0 msgr2=0x7f86ac19b240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:09.030 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:09.026+0000 7f86a9d9b640 1 --2- 172.21.15.39:0/180476666 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f86ac1031c0 0x7f86ac19b240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:09.030 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:09.026+0000 7f86a9d9b640 1 -- 172.21.15.39:0/180476666 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f86ac1a6550 con 0x7f86ac105190 2024-09-21T03:44:09.030 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:09.027+0000 7f86a9d9b640 1 --2- 172.21.15.39:0/180476666 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f86ac105190 0x7f86ac19b780 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f86a0005fd0 tx=0x7f86a000dde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:44:09.030 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:09.027+0000 7f869b7fe640 1 -- 172.21.15.39:0/180476666 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86a0017070 con 0x7f86ac105190 2024-09-21T03:44:09.030 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:09.027+0000 7f869b7fe640 1 -- 172.21.15.39:0/180476666 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f86a0009c30 con 0x7f86ac105190 2024-09-21T03:44:09.030 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:09.027+0000 7f869b7fe640 1 -- 172.21.15.39:0/180476666 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86a0003d30 con 0x7f86ac105190 2024-09-21T03:44:09.030 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:09.027+0000 7f86b0d37640 1 -- 172.21.15.39:0/180476666 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f86ac1a6840 con 0x7f86ac105190 2024-09-21T03:44:09.030 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:09.027+0000 7f86b0d37640 1 -- 172.21.15.39:0/180476666 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f86ac1a6d30 con 0x7f86ac105190 2024-09-21T03:44:09.031 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:09.028+0000 7f86b0d37640 1 -- 172.21.15.39:0/180476666 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f86ac1035d0 con 0x7f86ac105190 2024-09-21T03:44:09.031 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:09.029+0000 7f869b7fe640 1 -- 172.21.15.39:0/180476666 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 21) ==== 154529+0+0 (secure 0 0 0) 0x7f86a0003ed0 con 0x7f86ac105190 2024-09-21T03:44:09.032 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:09.030+0000 7f869b7fe640 1 --2- 172.21.15.39:0/180476666 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f86940b65e0 0x7f86940b8af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:09.032 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:09.030+0000 7f86aa59c640 1 --2- 172.21.15.39:0/180476666 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f86940b65e0 0x7f86940b8af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:09.032 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:09.030+0000 7f869b7fe640 1 -- 172.21.15.39:0/180476666 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 7342+0+0 (secure 0 0 0) 0x7f86a00e6b60 con 0x7f86ac105190 2024-09-21T03:44:09.032 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:09.031+0000 7f86aa59c640 1 --2- 172.21.15.39:0/180476666 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f86940b65e0 0x7f86940b8af0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f86ac1029e0 tx=0x7f869c005f90 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:44:09.036 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:09.035+0000 7f869b7fe640 1 -- 172.21.15.39:0/180476666 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f86a0020030 con 0x7f86ac105190 2024-09-21T03:44:09.239 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:09.237+0000 7f86b0d37640 1 -- 172.21.15.39:0/180476666 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f86ac02e640 con 0x7f86ac105190 2024-09-21T03:44:09.240 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:44:09 smithi039 ceph-mon[31211]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T03:44:09.240 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:44:09 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:44:09.240 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:44:09 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:44:09.240 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:44:09 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:44:09.240 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:44:09 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:44:09.240 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:09.238+0000 7f869b7fe640 1 -- 172.21.15.39:0/180476666 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v72) ==== 84+0+65 (secure 0 0 0) 0x7f86a00ad720 con 0x7f86ac105190 2024-09-21T03:44:09.240 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T03:44:09.241 INFO:teuthology.orchestra.run.smithi039.stdout:[ 2024-09-21T03:44:09.241 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-21T03:44:09.241 INFO:teuthology.orchestra.run.smithi039.stdout: "poolnum": 1, 2024-09-21T03:44:09.241 INFO:teuthology.orchestra.run.smithi039.stdout: "poolname": ".mgr" 2024-09-21T03:44:09.241 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-21T03:44:09.241 INFO:teuthology.orchestra.run.smithi039.stdout:] 2024-09-21T03:44:09.242 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:09.240+0000 7f86b0d37640 1 -- 172.21.15.39:0/180476666 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f86940b65e0 msgr2=0x7f86940b8af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:09.243 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:09.240+0000 7f86b0d37640 1 --2- 172.21.15.39:0/180476666 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f86940b65e0 0x7f86940b8af0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f86ac1029e0 tx=0x7f869c005f90 comp rx=0 tx=0).stop 2024-09-21T03:44:09.243 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:09.240+0000 7f86b0d37640 1 -- 172.21.15.39:0/180476666 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f86ac105190 msgr2=0x7f86ac19b780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:09.243 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:09.240+0000 7f86b0d37640 1 --2- 172.21.15.39:0/180476666 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f86ac105190 0x7f86ac19b780 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f86a0005fd0 tx=0x7f86a000dde0 comp rx=0 tx=0).stop 2024-09-21T03:44:09.243 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:09.241+0000 7f86b0d37640 1 -- 172.21.15.39:0/180476666 shutdown_connections 2024-09-21T03:44:09.243 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:09.241+0000 7f86b0d37640 1 --2- 172.21.15.39:0/180476666 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f86940b65e0 0x7f86940b8af0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:09.243 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:09.241+0000 7f86b0d37640 1 --2- 172.21.15.39:0/180476666 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f86ac105ae0 0x7f86ac19bcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:09.243 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:09.241+0000 7f86b0d37640 1 --2- 172.21.15.39:0/180476666 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f86ac105190 0x7f86ac19b780 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:09.243 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:09.241+0000 7f86b0d37640 1 --2- 172.21.15.39:0/180476666 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f86ac1031c0 0x7f86ac19b240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:09.243 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:09.241+0000 7f86b0d37640 1 -- 172.21.15.39:0/180476666 >> 172.21.15.39:0/180476666 conn(0x7f86ac077c60 msgr2=0x7f86ac106400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:44:09.243 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:09.241+0000 7f86b0d37640 1 -- 172.21.15.39:0/180476666 shutdown_connections 2024-09-21T03:44:09.243 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:09.241+0000 7f86b0d37640 1 -- 172.21.15.39:0/180476666 wait complete. 2024-09-21T03:44:09.467 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:44:09 smithi161 ceph-mon[34533]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T03:44:09.468 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:44:09 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:44:09.468 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:44:09 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:44:09.468 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:44:09 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:44:09.468 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:44:09 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:44:09.474 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:44:09 smithi150 ceph-mon[34289]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T03:44:09.474 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:44:09 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:44:09.475 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:44:09 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:44:09.475 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:44:09 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:44:09.475 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:44:09 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:44:09.948 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-09-21T03:44:09.948 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 51116d5c-77ca-11ef-baf5-efdc52797490 -- ceph osd pool create cephfs_metadata --pg_num_min 64 2024-09-21T03:44:10.207 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/51116d5c-77ca-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:44:10.328 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:44:10 smithi039 ceph-mon[31211]: from='client.? 172.21.15.39:0/180476666' entity='client.admin' cmd={"prefix": "osd lspools", "format": "json-pretty"} : dispatch 2024-09-21T03:44:10.328 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:44:10 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:44:10.328 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:44:10 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:44:10.329 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:44:10 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:44:10.329 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:44:10 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:44:10.329 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:44:10 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:44:10.376 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:44:10 smithi150 ceph-mon[34289]: from='client.? 172.21.15.39:0/180476666' entity='client.admin' cmd={"prefix": "osd lspools", "format": "json-pretty"} : dispatch 2024-09-21T03:44:10.376 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:44:10 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:44:10.376 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:44:10 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:44:10.376 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:44:10 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:44:10.376 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:44:10 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:44:10.376 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:44:10 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:44:10.467 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:44:10 smithi161 ceph-mon[34533]: from='client.? 172.21.15.39:0/180476666' entity='client.admin' cmd={"prefix": "osd lspools", "format": "json-pretty"} : dispatch 2024-09-21T03:44:10.468 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:44:10 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:44:10.468 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:44:10 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:44:10.468 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:44:10 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:44:10.468 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:44:10 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:44:10.468 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:44:10 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:44:10.686 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:10.683+0000 7fe0e21e9640 1 -- 172.21.15.39:0/456866619 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe0dc108d20 msgr2=0x7fe0dc10f550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:10.687 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:10.683+0000 7fe0e21e9640 1 --2- 172.21.15.39:0/456866619 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe0dc108d20 0x7fe0dc10f550 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fe0d4009ca0 tx=0x7fe0d402a8c0 comp rx=0 tx=0).stop 2024-09-21T03:44:10.687 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:10.683+0000 7fe0e21e9640 1 -- 172.21.15.39:0/456866619 shutdown_connections 2024-09-21T03:44:10.687 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:10.683+0000 7fe0e21e9640 1 --2- 172.21.15.39:0/456866619 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe0dc108d20 0x7fe0dc10f550 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:10.687 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:10.683+0000 7fe0e21e9640 1 --2- 172.21.15.39:0/456866619 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe0dc1083d0 0x7fe0dc1087e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:10.687 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:10.683+0000 7fe0e21e9640 1 --2- 172.21.15.39:0/456866619 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe0dc102ae0 0x7fe0dc102ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:10.687 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:10.683+0000 7fe0e21e9640 1 -- 172.21.15.39:0/456866619 >> 172.21.15.39:0/456866619 conn(0x7fe0dc079760 msgr2=0x7fe0dc079b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:44:10.687 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:10.683+0000 7fe0e21e9640 1 -- 172.21.15.39:0/456866619 shutdown_connections 2024-09-21T03:44:10.687 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:10.683+0000 7fe0e21e9640 1 -- 172.21.15.39:0/456866619 wait complete. 2024-09-21T03:44:10.687 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:10.684+0000 7fe0e21e9640 1 Processor -- start 2024-09-21T03:44:10.687 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:10.684+0000 7fe0e21e9640 1 -- start start 2024-09-21T03:44:10.687 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:10.684+0000 7fe0e21e9640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe0dc102ae0 0x7fe0dc10ffe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:10.687 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:10.684+0000 7fe0e21e9640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe0dc1083d0 0x7fe0dc110520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:10.688 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:10.684+0000 7fe0e21e9640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe0dc108d20 0x7fe0dc110a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:10.688 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:10.684+0000 7fe0e21e9640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fe0dc113b40 con 0x7fe0dc108d20 2024-09-21T03:44:10.688 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:10.684+0000 7fe0e21e9640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fe0dc1139c0 con 0x7fe0dc102ae0 2024-09-21T03:44:10.688 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:10.684+0000 7fe0e21e9640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fe0dc113cc0 con 0x7fe0dc1083d0 2024-09-21T03:44:10.688 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:10.685+0000 7fe0dbfff640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe0dc108d20 0x7fe0dc110a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:10.688 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:10.685+0000 7fe0dbfff640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe0dc108d20 0x7fe0dc110a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:50746/0 (socket says 172.21.15.39:50746) 2024-09-21T03:44:10.688 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:10.685+0000 7fe0dbfff640 1 -- 172.21.15.39:0/726794046 learned_addr learned my addr 172.21.15.39:0/726794046 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:44:10.688 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:10.685+0000 7fe0db7fe640 1 --2- 172.21.15.39:0/726794046 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe0dc102ae0 0x7fe0dc10ffe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:10.688 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:10.685+0000 7fe0daffd640 1 --2- 172.21.15.39:0/726794046 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe0dc1083d0 0x7fe0dc110520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:10.688 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:10.685+0000 7fe0dbfff640 1 -- 172.21.15.39:0/726794046 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe0dc1083d0 msgr2=0x7fe0dc110520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:10.688 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:10.685+0000 7fe0dbfff640 1 --2- 172.21.15.39:0/726794046 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe0dc1083d0 0x7fe0dc110520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:10.688 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:10.685+0000 7fe0dbfff640 1 -- 172.21.15.39:0/726794046 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe0dc102ae0 msgr2=0x7fe0dc10ffe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:10.688 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:10.685+0000 7fe0dbfff640 1 --2- 172.21.15.39:0/726794046 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe0dc102ae0 0x7fe0dc10ffe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:10.688 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:10.685+0000 7fe0dbfff640 1 -- 172.21.15.39:0/726794046 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe0dc1a8710 con 0x7fe0dc108d20 2024-09-21T03:44:10.689 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:10.685+0000 7fe0db7fe640 1 --2- 172.21.15.39:0/726794046 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe0dc102ae0 0x7fe0dc10ffe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:44:10.689 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:10.685+0000 7fe0dbfff640 1 --2- 172.21.15.39:0/726794046 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe0dc108d20 0x7fe0dc110a60 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fe0d4031a20 tx=0x7fe0d4031a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:44:10.689 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:10.685+0000 7fe0d8ff9640 1 -- 172.21.15.39:0/726794046 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0d4003d30 con 0x7fe0dc108d20 2024-09-21T03:44:10.689 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:10.685+0000 7fe0d8ff9640 1 -- 172.21.15.39:0/726794046 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe0d40044f0 con 0x7fe0dc108d20 2024-09-21T03:44:10.690 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:10.685+0000 7fe0d8ff9640 1 -- 172.21.15.39:0/726794046 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0d402fd90 con 0x7fe0dc108d20 2024-09-21T03:44:10.690 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:10.685+0000 7fe0e21e9640 1 -- 172.21.15.39:0/726794046 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe0dc1a89a0 con 0x7fe0dc108d20 2024-09-21T03:44:10.690 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:10.685+0000 7fe0e21e9640 1 -- 172.21.15.39:0/726794046 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe0dc1a8f60 con 0x7fe0dc108d20 2024-09-21T03:44:10.690 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:10.687+0000 7fe0e21e9640 1 -- 172.21.15.39:0/726794046 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe0dc103d10 con 0x7fe0dc108d20 2024-09-21T03:44:10.690 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:10.688+0000 7fe0d8ff9640 1 -- 172.21.15.39:0/726794046 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 21) ==== 154529+0+0 (secure 0 0 0) 0x7fe0d4004040 con 0x7fe0dc108d20 2024-09-21T03:44:10.691 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:10.689+0000 7fe0d8ff9640 1 --2- 172.21.15.39:0/726794046 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7fe0c80b6560 0x7fe0c80b8a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:10.691 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:10.689+0000 7fe0db7fe640 1 --2- 172.21.15.39:0/726794046 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7fe0c80b6560 0x7fe0c80b8a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:10.691 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:10.689+0000 7fe0d8ff9640 1 -- 172.21.15.39:0/726794046 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 7342+0+0 (secure 0 0 0) 0x7fe0d4108650 con 0x7fe0dc108d20 2024-09-21T03:44:10.691 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:10.690+0000 7fe0db7fe640 1 --2- 172.21.15.39:0/726794046 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7fe0c80b6560 0x7fe0c80b8a70 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fe0dc111a30 tx=0x7fe0c4008040 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:44:10.694 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:10.692+0000 7fe0d8ff9640 1 -- 172.21.15.39:0/726794046 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fe0d40cf340 con 0x7fe0dc108d20 2024-09-21T03:44:10.883 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:10.881+0000 7fe0e21e9640 1 -- 172.21.15.39:0/726794046 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7fe0dc1112e0 con 0x7fe0dc108d20 2024-09-21T03:44:11.286 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:11.284+0000 7fe0d8ff9640 1 -- 172.21.15.39:0/726794046 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v73) ==== 138+0+0 (secure 0 0 0) 0x7fe0d40d3720 con 0x7fe0dc108d20 2024-09-21T03:44:11.286 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:44:11 smithi039 ceph-mon[31211]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T03:44:11.286 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:44:11 smithi039 ceph-mon[31211]: from='client.? 172.21.15.39:0/726794046' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} : dispatch 2024-09-21T03:44:11.287 INFO:teuthology.orchestra.run.smithi039.stderr:pool 'cephfs_metadata' created 2024-09-21T03:44:11.289 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:11.287+0000 7fe0e21e9640 1 -- 172.21.15.39:0/726794046 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7fe0c80b6560 msgr2=0x7fe0c80b8a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:11.289 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:11.287+0000 7fe0e21e9640 1 --2- 172.21.15.39:0/726794046 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7fe0c80b6560 0x7fe0c80b8a70 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fe0dc111a30 tx=0x7fe0c4008040 comp rx=0 tx=0).stop 2024-09-21T03:44:11.289 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:11.287+0000 7fe0e21e9640 1 -- 172.21.15.39:0/726794046 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe0dc108d20 msgr2=0x7fe0dc110a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:11.289 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:11.287+0000 7fe0e21e9640 1 --2- 172.21.15.39:0/726794046 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe0dc108d20 0x7fe0dc110a60 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fe0d4031a20 tx=0x7fe0d4031a50 comp rx=0 tx=0).stop 2024-09-21T03:44:11.289 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:11.287+0000 7fe0e21e9640 1 -- 172.21.15.39:0/726794046 shutdown_connections 2024-09-21T03:44:11.289 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:11.287+0000 7fe0e21e9640 1 --2- 172.21.15.39:0/726794046 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7fe0c80b6560 0x7fe0c80b8a70 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:11.289 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:11.287+0000 7fe0e21e9640 1 --2- 172.21.15.39:0/726794046 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe0dc108d20 0x7fe0dc110a60 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:11.290 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:11.287+0000 7fe0e21e9640 1 --2- 172.21.15.39:0/726794046 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe0dc1083d0 0x7fe0dc110520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:11.290 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:11.287+0000 7fe0e21e9640 1 --2- 172.21.15.39:0/726794046 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe0dc102ae0 0x7fe0dc10ffe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:11.290 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:11.287+0000 7fe0e21e9640 1 -- 172.21.15.39:0/726794046 >> 172.21.15.39:0/726794046 conn(0x7fe0dc079760 msgr2=0x7fe0dc105230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:44:11.290 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:11.287+0000 7fe0e21e9640 1 -- 172.21.15.39:0/726794046 shutdown_connections 2024-09-21T03:44:11.290 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:11.287+0000 7fe0e21e9640 1 -- 172.21.15.39:0/726794046 wait complete. 2024-09-21T03:44:11.389 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:44:11 smithi150 ceph-mon[34289]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T03:44:11.389 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:44:11 smithi150 ceph-mon[34289]: from='client.? 172.21.15.39:0/726794046' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} : dispatch 2024-09-21T03:44:11.468 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:44:11 smithi161 ceph-mon[34533]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T03:44:11.468 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:44:11 smithi161 ceph-mon[34533]: from='client.? 172.21.15.39:0/726794046' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} : dispatch 2024-09-21T03:44:12.030 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 51116d5c-77ca-11ef-baf5-efdc52797490 -- ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-09-21T03:44:12.296 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/51116d5c-77ca-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:44:12.586 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:44:12 smithi039 ceph-mon[31211]: from='client.? 172.21.15.39:0/726794046' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-09-21T03:44:12.586 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:44:12 smithi039 ceph-mon[31211]: osdmap e73: 12 total, 12 up, 12 in 2024-09-21T03:44:12.586 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:44:12 smithi039 ceph-mon[31211]: pgmap v264: 33 pgs: 32 unknown, 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T03:44:12.607 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:44:12 smithi150 ceph-mon[34289]: from='client.? 172.21.15.39:0/726794046' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-09-21T03:44:12.607 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:44:12 smithi150 ceph-mon[34289]: osdmap e73: 12 total, 12 up, 12 in 2024-09-21T03:44:12.607 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:44:12 smithi150 ceph-mon[34289]: pgmap v264: 33 pgs: 32 unknown, 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T03:44:12.718 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:44:12 smithi161 ceph-mon[34533]: from='client.? 172.21.15.39:0/726794046' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-09-21T03:44:12.718 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:44:12 smithi161 ceph-mon[34533]: osdmap e73: 12 total, 12 up, 12 in 2024-09-21T03:44:12.718 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:44:12 smithi161 ceph-mon[34533]: pgmap v264: 33 pgs: 32 unknown, 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T03:44:12.838 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:12.836+0000 7f35e8967640 1 -- 172.21.15.39:0/3291210758 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f35e4102a60 msgr2=0x7f35e4102e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:12.839 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:12.836+0000 7f35e8967640 1 --2- 172.21.15.39:0/3291210758 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f35e4102a60 0x7f35e4102e70 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f35d4009900 tx=0x7f35d402a410 comp rx=0 tx=0).stop 2024-09-21T03:44:12.839 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:12.837+0000 7f35e8967640 1 -- 172.21.15.39:0/3291210758 shutdown_connections 2024-09-21T03:44:12.839 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:12.837+0000 7f35e8967640 1 --2- 172.21.15.39:0/3291210758 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f35e4108cf0 0x7f35e410f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:12.839 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:12.837+0000 7f35e8967640 1 --2- 172.21.15.39:0/3291210758 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f35e41083a0 0x7f35e41087b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:12.839 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:12.837+0000 7f35e8967640 1 --2- 172.21.15.39:0/3291210758 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f35e4102a60 0x7f35e4102e70 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:12.839 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:12.837+0000 7f35e8967640 1 -- 172.21.15.39:0/3291210758 >> 172.21.15.39:0/3291210758 conn(0x7f35e4079730 msgr2=0x7f35e4079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:44:12.839 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:12.837+0000 7f35e8967640 1 -- 172.21.15.39:0/3291210758 shutdown_connections 2024-09-21T03:44:12.839 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:12.837+0000 7f35e8967640 1 -- 172.21.15.39:0/3291210758 wait complete. 2024-09-21T03:44:12.841 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:12.837+0000 7f35e8967640 1 Processor -- start 2024-09-21T03:44:12.841 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:12.838+0000 7f35e8967640 1 -- start start 2024-09-21T03:44:12.841 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:12.838+0000 7f35e8967640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f35e4102a60 0x7f35e410ffb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:12.841 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:12.838+0000 7f35e8967640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f35e41083a0 0x7f35e41104f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:12.841 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:12.838+0000 7f35e8967640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f35e4108cf0 0x7f35e4110a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:12.841 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:12.838+0000 7f35e8967640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f35e4113ac0 con 0x7f35e4102a60 2024-09-21T03:44:12.841 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:12.838+0000 7f35e8967640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f35e4113940 con 0x7f35e41083a0 2024-09-21T03:44:12.842 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:12.838+0000 7f35e8967640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f35e4113c40 con 0x7f35e4108cf0 2024-09-21T03:44:12.842 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:12.838+0000 7f35e259c640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f35e4102a60 0x7f35e410ffb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:12.842 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:12.838+0000 7f35e259c640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f35e4102a60 0x7f35e410ffb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:50754/0 (socket says 172.21.15.39:50754) 2024-09-21T03:44:12.842 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:12.838+0000 7f35e259c640 1 -- 172.21.15.39:0/1405090640 learned_addr learned my addr 172.21.15.39:0/1405090640 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:44:12.842 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:12.838+0000 7f35e1d9b640 1 --2- 172.21.15.39:0/1405090640 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f35e41083a0 0x7f35e41104f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:12.842 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:12.838+0000 7f35e2d9d640 1 --2- 172.21.15.39:0/1405090640 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f35e4108cf0 0x7f35e4110a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:12.842 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:12.838+0000 7f35e259c640 1 -- 172.21.15.39:0/1405090640 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f35e4108cf0 msgr2=0x7f35e4110a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:12.842 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:12.838+0000 7f35e259c640 1 --2- 172.21.15.39:0/1405090640 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f35e4108cf0 0x7f35e4110a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:12.842 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:12.838+0000 7f35e259c640 1 -- 172.21.15.39:0/1405090640 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f35e41083a0 msgr2=0x7f35e41104f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:12.842 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:12.838+0000 7f35e259c640 1 --2- 172.21.15.39:0/1405090640 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f35e41083a0 0x7f35e41104f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:12.842 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:12.838+0000 7f35e259c640 1 -- 172.21.15.39:0/1405090640 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f35e41a8790 con 0x7f35e4102a60 2024-09-21T03:44:12.842 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:12.838+0000 7f35e1d9b640 1 --2- 172.21.15.39:0/1405090640 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f35e41083a0 0x7f35e41104f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:44:12.842 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:12.839+0000 7f35e259c640 1 --2- 172.21.15.39:0/1405090640 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f35e4102a60 0x7f35e410ffb0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f35d4006010 tx=0x7f35d40042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:44:12.842 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:12.839+0000 7f35d37fe640 1 -- 172.21.15.39:0/1405090640 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35d403f030 con 0x7f35e4102a60 2024-09-21T03:44:12.842 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:12.839+0000 7f35d37fe640 1 -- 172.21.15.39:0/1405090640 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f35d4003800 con 0x7f35e4102a60 2024-09-21T03:44:12.842 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:12.839+0000 7f35e8967640 1 -- 172.21.15.39:0/1405090640 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f35e41a8a20 con 0x7f35e4102a60 2024-09-21T03:44:12.843 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:12.839+0000 7f35d37fe640 1 -- 172.21.15.39:0/1405090640 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35d4035070 con 0x7f35e4102a60 2024-09-21T03:44:12.843 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:12.839+0000 7f35e8967640 1 -- 172.21.15.39:0/1405090640 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f35e41a8f80 con 0x7f35e4102a60 2024-09-21T03:44:12.843 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:12.840+0000 7f35e8967640 1 -- 172.21.15.39:0/1405090640 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f35e4102e70 con 0x7f35e4102a60 2024-09-21T03:44:12.843 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:12.842+0000 7f35d37fe640 1 -- 172.21.15.39:0/1405090640 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 21) ==== 154529+0+0 (secure 0 0 0) 0x7f35d4003cb0 con 0x7f35e4102a60 2024-09-21T03:44:12.844 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:12.842+0000 7f35d37fe640 1 --2- 172.21.15.39:0/1405090640 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f35cc0b6560 0x7f35cc0b8a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:12.844 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:12.842+0000 7f35e1d9b640 1 --2- 172.21.15.39:0/1405090640 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f35cc0b6560 0x7f35cc0b8a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:12.844 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:12.843+0000 7f35d37fe640 1 -- 172.21.15.39:0/1405090640 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 7712+0+0 (secure 0 0 0) 0x7f35d4107be0 con 0x7f35e4102a60 2024-09-21T03:44:12.844 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:12.843+0000 7f35e1d9b640 1 --2- 172.21.15.39:0/1405090640 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f35cc0b6560 0x7f35cc0b8a70 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f35e4100cc0 tx=0x7f35d800a590 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:44:12.847 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:12.846+0000 7f35d37fe640 1 -- 172.21.15.39:0/1405090640 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f35d40d2ab0 con 0x7f35e4102a60 2024-09-21T03:44:13.039 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:13.037+0000 7f35e8967640 1 -- 172.21.15.39:0/1405090640 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f35e41112b0 con 0x7f35e4102a60 2024-09-21T03:44:13.296 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:13.294+0000 7f35d37fe640 1 -- 172.21.15.39:0/1405090640 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v75) ==== 170+0+0 (secure 0 0 0) 0x7f35e41112b0 con 0x7f35e4102a60 2024-09-21T03:44:13.296 INFO:teuthology.orchestra.run.smithi039.stderr:pool 'cephfs_data' created 2024-09-21T03:44:13.299 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:13.296+0000 7f35e8967640 1 -- 172.21.15.39:0/1405090640 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f35cc0b6560 msgr2=0x7f35cc0b8a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:13.299 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:13.296+0000 7f35e8967640 1 --2- 172.21.15.39:0/1405090640 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f35cc0b6560 0x7f35cc0b8a70 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f35e4100cc0 tx=0x7f35d800a590 comp rx=0 tx=0).stop 2024-09-21T03:44:13.299 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:13.296+0000 7f35e8967640 1 -- 172.21.15.39:0/1405090640 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f35e4102a60 msgr2=0x7f35e410ffb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:13.299 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:13.296+0000 7f35e8967640 1 --2- 172.21.15.39:0/1405090640 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f35e4102a60 0x7f35e410ffb0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f35d4006010 tx=0x7f35d40042f0 comp rx=0 tx=0).stop 2024-09-21T03:44:13.299 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:13.297+0000 7f35e8967640 1 -- 172.21.15.39:0/1405090640 shutdown_connections 2024-09-21T03:44:13.299 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:13.297+0000 7f35e8967640 1 --2- 172.21.15.39:0/1405090640 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f35cc0b6560 0x7f35cc0b8a70 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:13.299 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:13.297+0000 7f35e8967640 1 --2- 172.21.15.39:0/1405090640 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f35e4108cf0 0x7f35e4110a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:13.299 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:13.297+0000 7f35e8967640 1 --2- 172.21.15.39:0/1405090640 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f35e41083a0 0x7f35e41104f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:13.299 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:13.297+0000 7f35e8967640 1 --2- 172.21.15.39:0/1405090640 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f35e4102a60 0x7f35e410ffb0 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:13.299 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:13.297+0000 7f35e8967640 1 -- 172.21.15.39:0/1405090640 >> 172.21.15.39:0/1405090640 conn(0x7f35e4079730 msgr2=0x7f35e4101440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:44:13.300 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:13.297+0000 7f35e8967640 1 -- 172.21.15.39:0/1405090640 shutdown_connections 2024-09-21T03:44:13.300 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:13.297+0000 7f35e8967640 1 -- 172.21.15.39:0/1405090640 wait complete. 2024-09-21T03:44:13.583 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:44:13 smithi039 ceph-mon[31211]: osdmap e74: 12 total, 12 up, 12 in 2024-09-21T03:44:13.584 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:44:13 smithi039 ceph-mon[31211]: from='client.? 172.21.15.39:0/1405090640' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} : dispatch 2024-09-21T03:44:13.717 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:44:13 smithi161 ceph-mon[34533]: osdmap e74: 12 total, 12 up, 12 in 2024-09-21T03:44:13.718 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:44:13 smithi161 ceph-mon[34533]: from='client.? 172.21.15.39:0/1405090640' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} : dispatch 2024-09-21T03:44:13.724 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:44:13 smithi150 ceph-mon[34289]: osdmap e74: 12 total, 12 up, 12 in 2024-09-21T03:44:13.724 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:44:13 smithi150 ceph-mon[34289]: from='client.? 172.21.15.39:0/1405090640' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} : dispatch 2024-09-21T03:44:13.998 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 51116d5c-77ca-11ef-baf5-efdc52797490 -- ceph fs new cephfs cephfs_metadata cephfs_data 2024-09-21T03:44:14.258 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/51116d5c-77ca-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:44:14.583 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:44:14 smithi039 ceph-mon[31211]: from='client.? 172.21.15.39:0/1405090640' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-09-21T03:44:14.584 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:44:14 smithi039 ceph-mon[31211]: osdmap e75: 12 total, 12 up, 12 in 2024-09-21T03:44:14.584 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:44:14 smithi039 ceph-mon[31211]: pgmap v267: 97 pgs: 11 creating+peering, 77 unknown, 9 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T03:44:14.584 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:44:14 smithi039 ceph-mon[31211]: osdmap e76: 12 total, 12 up, 12 in 2024-09-21T03:44:14.718 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:44:14 smithi161 ceph-mon[34533]: from='client.? 172.21.15.39:0/1405090640' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-09-21T03:44:14.718 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:44:14 smithi161 ceph-mon[34533]: osdmap e75: 12 total, 12 up, 12 in 2024-09-21T03:44:14.718 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:44:14 smithi161 ceph-mon[34533]: pgmap v267: 97 pgs: 11 creating+peering, 77 unknown, 9 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T03:44:14.718 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:44:14 smithi161 ceph-mon[34533]: osdmap e76: 12 total, 12 up, 12 in 2024-09-21T03:44:14.724 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:44:14 smithi150 ceph-mon[34289]: from='client.? 172.21.15.39:0/1405090640' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-09-21T03:44:14.724 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:44:14 smithi150 ceph-mon[34289]: osdmap e75: 12 total, 12 up, 12 in 2024-09-21T03:44:14.724 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:44:14 smithi150 ceph-mon[34289]: pgmap v267: 97 pgs: 11 creating+peering, 77 unknown, 9 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T03:44:14.725 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:44:14 smithi150 ceph-mon[34289]: osdmap e76: 12 total, 12 up, 12 in 2024-09-21T03:44:14.772 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:14.764+0000 7fe786255640 1 -- 172.21.15.39:0/2893141016 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe780103bc0 msgr2=0x7fe780103fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:14.772 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:14.764+0000 7fe786255640 1 --2- 172.21.15.39:0/2893141016 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe780103bc0 0x7fe780103fd0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fe770009900 tx=0x7fe77002a410 comp rx=0 tx=0).stop 2024-09-21T03:44:14.772 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:14.764+0000 7fe786255640 1 -- 172.21.15.39:0/2893141016 shutdown_connections 2024-09-21T03:44:14.772 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:14.764+0000 7fe786255640 1 --2- 172.21.15.39:0/2893141016 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe780109e00 0x7fe780110630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:14.772 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:14.764+0000 7fe786255640 1 --2- 172.21.15.39:0/2893141016 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe7801094b0 0x7fe7801098c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:14.772 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:14.764+0000 7fe786255640 1 --2- 172.21.15.39:0/2893141016 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe780103bc0 0x7fe780103fd0 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:14.772 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:14.764+0000 7fe786255640 1 -- 172.21.15.39:0/2893141016 >> 172.21.15.39:0/2893141016 conn(0x7fe78007b910 msgr2=0x7fe78007bd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:44:14.772 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:14.764+0000 7fe786255640 1 -- 172.21.15.39:0/2893141016 shutdown_connections 2024-09-21T03:44:14.772 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:14.765+0000 7fe786255640 1 -- 172.21.15.39:0/2893141016 wait complete. 2024-09-21T03:44:14.772 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:14.765+0000 7fe786255640 1 Processor -- start 2024-09-21T03:44:14.772 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:14.765+0000 7fe786255640 1 -- start start 2024-09-21T03:44:14.772 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:14.766+0000 7fe786255640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe780103bc0 0x7fe78019a570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:14.773 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:14.766+0000 7fe786255640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe7801094b0 0x7fe78019ab10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:14.773 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:14.766+0000 7fe786255640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe780109e00 0x7fe78019fdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:14.773 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:14.766+0000 7fe786255640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fe780113ac0 con 0x7fe7801094b0 2024-09-21T03:44:14.773 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:14.766+0000 7fe786255640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fe780113940 con 0x7fe780109e00 2024-09-21T03:44:14.773 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:14.766+0000 7fe786255640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fe780113c40 con 0x7fe780103bc0 2024-09-21T03:44:14.773 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:14.766+0000 7fe77effd640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe7801094b0 0x7fe78019ab10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:14.773 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:14.766+0000 7fe77effd640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe7801094b0 0x7fe78019ab10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:50768/0 (socket says 172.21.15.39:50768) 2024-09-21T03:44:14.773 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:14.766+0000 7fe77effd640 1 -- 172.21.15.39:0/3233043517 learned_addr learned my addr 172.21.15.39:0/3233043517 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:44:14.773 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:14.766+0000 7fe77ffff640 1 --2- 172.21.15.39:0/3233043517 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe780109e00 0x7fe78019fdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:14.773 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:14.766+0000 7fe77f7fe640 1 --2- 172.21.15.39:0/3233043517 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe780103bc0 0x7fe78019a570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:14.773 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:14.766+0000 7fe77effd640 1 -- 172.21.15.39:0/3233043517 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe780103bc0 msgr2=0x7fe78019a570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:14.773 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:14.766+0000 7fe77effd640 1 --2- 172.21.15.39:0/3233043517 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe780103bc0 0x7fe78019a570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:14.773 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:14.766+0000 7fe77effd640 1 -- 172.21.15.39:0/3233043517 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe780109e00 msgr2=0x7fe78019fdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:14.773 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:14.766+0000 7fe77effd640 1 --2- 172.21.15.39:0/3233043517 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe780109e00 0x7fe78019fdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:14.774 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:14.766+0000 7fe77effd640 1 -- 172.21.15.39:0/3233043517 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe7801a0420 con 0x7fe7801094b0 2024-09-21T03:44:14.774 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:14.766+0000 7fe77effd640 1 --2- 172.21.15.39:0/3233043517 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe7801094b0 0x7fe78019ab10 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fe77400c970 tx=0x7fe77400ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:44:14.774 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:14.767+0000 7fe77cff9640 1 -- 172.21.15.39:0/3233043517 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe774004220 con 0x7fe7801094b0 2024-09-21T03:44:14.774 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:14.767+0000 7fe77cff9640 1 -- 172.21.15.39:0/3233043517 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe7740043c0 con 0x7fe7801094b0 2024-09-21T03:44:14.774 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:14.767+0000 7fe786255640 1 -- 172.21.15.39:0/3233043517 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe7801a0710 con 0x7fe7801094b0 2024-09-21T03:44:14.774 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:14.767+0000 7fe77cff9640 1 -- 172.21.15.39:0/3233043517 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe774004a00 con 0x7fe7801094b0 2024-09-21T03:44:14.774 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:14.767+0000 7fe786255640 1 -- 172.21.15.39:0/3233043517 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe7801a0c00 con 0x7fe7801094b0 2024-09-21T03:44:14.774 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:14.768+0000 7fe786255640 1 -- 172.21.15.39:0/3233043517 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe780103fd0 con 0x7fe7801094b0 2024-09-21T03:44:14.775 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:14.769+0000 7fe77cff9640 1 -- 172.21.15.39:0/3233043517 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 21) ==== 154529+0+0 (secure 0 0 0) 0x7fe7740079b0 con 0x7fe7801094b0 2024-09-21T03:44:14.775 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:14.771+0000 7fe77cff9640 1 --2- 172.21.15.39:0/3233043517 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7fe7680b6590 0x7fe7680b8aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:14.775 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:14.771+0000 7fe77f7fe640 1 --2- 172.21.15.39:0/3233043517 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7fe7680b6590 0x7fe7680b8aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:14.775 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:14.771+0000 7fe77cff9640 1 -- 172.21.15.39:0/3233043517 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 8094+0+0 (secure 0 0 0) 0x7fe7740e6f40 con 0x7fe7801094b0 2024-09-21T03:44:14.775 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:14.771+0000 7fe77f7fe640 1 --2- 172.21.15.39:0/3233043517 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7fe7680b6590 0x7fe7680b8aa0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fe770031440 tx=0x7fe770005e80 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:44:14.778 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:14.775+0000 7fe77cff9640 1 -- 172.21.15.39:0/3233043517 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fe7801a0c00 con 0x7fe7801094b0 2024-09-21T03:44:15.010 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:15.007+0000 7fe786255640 1 -- 172.21.15.39:0/3233043517 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7fe78002e640 con 0x7fe7801094b0 2024-09-21T03:44:15.331 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:15.328+0000 7fe77cff9640 1 -- 172.21.15.39:0/3233043517 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v14) ==== 172+0+0 (secure 0 0 0) 0x7fe774014030 con 0x7fe7801094b0 2024-09-21T03:44:15.331 INFO:teuthology.orchestra.run.smithi039.stderr:new fs with metadata pool 2 and data pool 3 2024-09-21T03:44:15.333 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:15.330+0000 7fe786255640 1 -- 172.21.15.39:0/3233043517 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7fe7680b6590 msgr2=0x7fe7680b8aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:15.333 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:15.330+0000 7fe786255640 1 --2- 172.21.15.39:0/3233043517 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7fe7680b6590 0x7fe7680b8aa0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fe770031440 tx=0x7fe770005e80 comp rx=0 tx=0).stop 2024-09-21T03:44:15.334 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:15.331+0000 7fe786255640 1 -- 172.21.15.39:0/3233043517 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe7801094b0 msgr2=0x7fe78019ab10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:15.334 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:15.331+0000 7fe786255640 1 --2- 172.21.15.39:0/3233043517 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe7801094b0 0x7fe78019ab10 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fe77400c970 tx=0x7fe77400ce30 comp rx=0 tx=0).stop 2024-09-21T03:44:15.334 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:15.331+0000 7fe786255640 1 -- 172.21.15.39:0/3233043517 shutdown_connections 2024-09-21T03:44:15.334 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:15.331+0000 7fe786255640 1 --2- 172.21.15.39:0/3233043517 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7fe7680b6590 0x7fe7680b8aa0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:15.334 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:15.331+0000 7fe786255640 1 --2- 172.21.15.39:0/3233043517 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe780109e00 0x7fe78019fdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:15.334 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:15.331+0000 7fe786255640 1 --2- 172.21.15.39:0/3233043517 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe7801094b0 0x7fe78019ab10 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:15.334 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:15.331+0000 7fe786255640 1 --2- 172.21.15.39:0/3233043517 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe780103bc0 0x7fe78019a570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:15.334 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:15.331+0000 7fe786255640 1 -- 172.21.15.39:0/3233043517 >> 172.21.15.39:0/3233043517 conn(0x7fe78007b910 msgr2=0x7fe7801075c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:44:15.334 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:15.331+0000 7fe786255640 1 -- 172.21.15.39:0/3233043517 shutdown_connections 2024-09-21T03:44:15.334 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:15.331+0000 7fe786255640 1 -- 172.21.15.39:0/3233043517 wait complete. 2024-09-21T03:44:15.583 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:44:15 smithi039 ceph-mon[31211]: from='client.? 172.21.15.39:0/3233043517' entity='client.admin' cmd={"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} : dispatch 2024-09-21T03:44:15.584 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:44:15 smithi039 ceph-mon[31211]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-09-21T03:44:15.717 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:44:15 smithi161 ceph-mon[34533]: from='client.? 172.21.15.39:0/3233043517' entity='client.admin' cmd={"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} : dispatch 2024-09-21T03:44:15.718 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:44:15 smithi161 ceph-mon[34533]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-09-21T03:44:15.724 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:44:15 smithi150 ceph-mon[34289]: from='client.? 172.21.15.39:0/3233043517' entity='client.admin' cmd={"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} : dispatch 2024-09-21T03:44:15.724 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:44:15 smithi150 ceph-mon[34289]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-09-21T03:44:15.981 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 51116d5c-77ca-11ef-baf5-efdc52797490 -- ceph osd dump --format=json 2024-09-21T03:44:16.239 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/51116d5c-77ca-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:44:16.584 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:44:16 smithi039 ceph-mon[31211]: osdmap e77: 12 total, 12 up, 12 in 2024-09-21T03:44:16.584 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:44:16 smithi039 ceph-mon[31211]: from='client.? 172.21.15.39:0/3233043517' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-09-21T03:44:16.584 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:44:16 smithi039 ceph-mon[31211]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-09-21T03:44:16.584 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:44:16 smithi039 ceph-mon[31211]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-09-21T03:44:16.584 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:44:16 smithi039 ceph-mon[31211]: pgmap v270: 97 pgs: 34 creating+peering, 18 unknown, 45 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T03:44:16.713 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:16.710+0000 7f4b1120e640 1 -- 172.21.15.39:0/3849640320 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4b0c1083d0 msgr2=0x7f4b0c1087e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:16.714 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:16.711+0000 7f4b1120e640 1 --2- 172.21.15.39:0/3849640320 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4b0c1083d0 0x7f4b0c1087e0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f4b00009900 tx=0x7f4b0002a410 comp rx=0 tx=0).stop 2024-09-21T03:44:16.714 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:16.711+0000 7f4b1120e640 1 -- 172.21.15.39:0/3849640320 shutdown_connections 2024-09-21T03:44:16.714 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:16.711+0000 7f4b1120e640 1 --2- 172.21.15.39:0/3849640320 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4b0c108d20 0x7f4b0c10f550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:16.714 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:16.711+0000 7f4b1120e640 1 --2- 172.21.15.39:0/3849640320 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4b0c1083d0 0x7f4b0c1087e0 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:16.714 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:16.711+0000 7f4b1120e640 1 --2- 172.21.15.39:0/3849640320 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4b0c102ae0 0x7f4b0c102ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:16.714 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:16.711+0000 7f4b1120e640 1 -- 172.21.15.39:0/3849640320 >> 172.21.15.39:0/3849640320 conn(0x7f4b0c079760 msgr2=0x7f4b0c079b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:44:16.715 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:16.711+0000 7f4b1120e640 1 -- 172.21.15.39:0/3849640320 shutdown_connections 2024-09-21T03:44:16.715 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:16.711+0000 7f4b1120e640 1 -- 172.21.15.39:0/3849640320 wait complete. 2024-09-21T03:44:16.717 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:16.711+0000 7f4b1120e640 1 Processor -- start 2024-09-21T03:44:16.717 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:16.712+0000 7f4b1120e640 1 -- start start 2024-09-21T03:44:16.717 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:16.712+0000 7f4b1120e640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4b0c102ae0 0x7f4b0c19c720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:16.717 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:16.712+0000 7f4b1120e640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4b0c1083d0 0x7f4b0c19ccc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:16.717 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:16.712+0000 7f4b1120e640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4b0c108d20 0x7f4b0c1a1f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:16.717 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:16.712+0000 7f4b1120e640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f4b0c112990 con 0x7f4b0c102ae0 2024-09-21T03:44:16.718 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:16.712+0000 7f4b1120e640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f4b0c112810 con 0x7f4b0c1083d0 2024-09-21T03:44:16.718 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:16.712+0000 7f4b1120e640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f4b0c112b10 con 0x7f4b0c108d20 2024-09-21T03:44:16.718 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:16.712+0000 7f4b0ad9d640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4b0c102ae0 0x7f4b0c19c720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:16.718 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:16.712+0000 7f4b0ad9d640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4b0c102ae0 0x7f4b0c19c720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:50794/0 (socket says 172.21.15.39:50794) 2024-09-21T03:44:16.718 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:16.712+0000 7f4b0ad9d640 1 -- 172.21.15.39:0/1702145567 learned_addr learned my addr 172.21.15.39:0/1702145567 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:44:16.718 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:16.712+0000 7f4b0b59e640 1 --2- 172.21.15.39:0/1702145567 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4b0c108d20 0x7f4b0c1a1f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:16.718 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:16.712+0000 7f4b0a59c640 1 --2- 172.21.15.39:0/1702145567 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4b0c1083d0 0x7f4b0c19ccc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:16.718 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:16.712+0000 7f4b0ad9d640 1 -- 172.21.15.39:0/1702145567 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4b0c108d20 msgr2=0x7f4b0c1a1f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:16.718 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:16.712+0000 7f4b0ad9d640 1 --2- 172.21.15.39:0/1702145567 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4b0c108d20 0x7f4b0c1a1f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:16.718 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:16.712+0000 7f4b0ad9d640 1 -- 172.21.15.39:0/1702145567 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4b0c1083d0 msgr2=0x7f4b0c19ccc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:16.718 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:16.712+0000 7f4b0ad9d640 1 --2- 172.21.15.39:0/1702145567 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4b0c1083d0 0x7f4b0c19ccc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:16.718 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:16.712+0000 7f4b0ad9d640 1 -- 172.21.15.39:0/1702145567 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4b0c1a2690 con 0x7f4b0c102ae0 2024-09-21T03:44:16.718 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:16.712+0000 7f4b0b59e640 1 --2- 172.21.15.39:0/1702145567 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4b0c108d20 0x7f4b0c1a1f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:44:16.718 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:16.713+0000 7f4b0ad9d640 1 --2- 172.21.15.39:0/1702145567 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4b0c102ae0 0x7f4b0c19c720 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f4afc00d950 tx=0x7f4afc00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:44:16.719 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:16.713+0000 7f4af3fff640 1 -- 172.21.15.39:0/1702145567 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4afc017070 con 0x7f4b0c102ae0 2024-09-21T03:44:16.719 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:16.713+0000 7f4af3fff640 1 -- 172.21.15.39:0/1702145567 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4afc009c30 con 0x7f4b0c102ae0 2024-09-21T03:44:16.719 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:16.713+0000 7f4b1120e640 1 -- 172.21.15.39:0/1702145567 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4b0c1a2980 con 0x7f4b0c102ae0 2024-09-21T03:44:16.719 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:16.713+0000 7f4af3fff640 1 -- 172.21.15.39:0/1702145567 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4afc00f620 con 0x7f4b0c102ae0 2024-09-21T03:44:16.719 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:16.713+0000 7f4b1120e640 1 -- 172.21.15.39:0/1702145567 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4b0c1a90e0 con 0x7f4b0c102ae0 2024-09-21T03:44:16.719 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:16.714+0000 7f4b1120e640 1 -- 172.21.15.39:0/1702145567 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4b0c103d10 con 0x7f4b0c102ae0 2024-09-21T03:44:16.719 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:44:16 smithi161 ceph-mon[34533]: osdmap e77: 12 total, 12 up, 12 in 2024-09-21T03:44:16.720 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:44:16 smithi161 ceph-mon[34533]: from='client.? 172.21.15.39:0/3233043517' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-09-21T03:44:16.720 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:44:16 smithi161 ceph-mon[34533]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-09-21T03:44:16.720 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:44:16 smithi161 ceph-mon[34533]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-09-21T03:44:16.720 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:44:16 smithi161 ceph-mon[34533]: pgmap v270: 97 pgs: 34 creating+peering, 18 unknown, 45 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T03:44:16.720 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:16.716+0000 7f4af3fff640 1 -- 172.21.15.39:0/1702145567 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 21) ==== 154529+0+0 (secure 0 0 0) 0x7f4afc005030 con 0x7f4b0c102ae0 2024-09-21T03:44:16.720 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:16.716+0000 7f4af3fff640 1 --2- 172.21.15.39:0/1702145567 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f4af40b6560 0x7f4af40b8a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:16.721 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:16.717+0000 7f4b0a59c640 1 --2- 172.21.15.39:0/1702145567 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f4af40b6560 0x7f4af40b8a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:16.721 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:16.717+0000 7f4af3fff640 1 -- 172.21.15.39:0/1702145567 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8194+0+0 (secure 0 0 0) 0x7f4afc0e7000 con 0x7f4b0c102ae0 2024-09-21T03:44:16.721 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:16.717+0000 7f4b0a59c640 1 --2- 172.21.15.39:0/1702145567 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f4af40b6560 0x7f4af40b8a70 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f4b00000c00 tx=0x7f4b00031400 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:44:16.722 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:16.720+0000 7f4af3fff640 1 -- 172.21.15.39:0/1702145567 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f4afc0ad8b0 con 0x7f4b0c102ae0 2024-09-21T03:44:16.724 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:44:16 smithi150 ceph-mon[34289]: osdmap e77: 12 total, 12 up, 12 in 2024-09-21T03:44:16.724 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:44:16 smithi150 ceph-mon[34289]: from='client.? 172.21.15.39:0/3233043517' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-09-21T03:44:16.724 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:44:16 smithi150 ceph-mon[34289]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-09-21T03:44:16.725 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:44:16 smithi150 ceph-mon[34289]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-09-21T03:44:16.725 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:44:16 smithi150 ceph-mon[34289]: pgmap v270: 97 pgs: 34 creating+peering, 18 unknown, 45 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T03:44:16.911 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:16.909+0000 7f4b1120e640 1 -- 172.21.15.39:0/1702145567 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f4b0c19d850 con 0x7f4b0c102ae0 2024-09-21T03:44:16.913 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:16.910+0000 7f4af3fff640 1 -- 172.21.15.39:0/1702145567 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v77) ==== 74+0+22570 (secure 0 0 0) 0x7f4b0c19d850 con 0x7f4b0c102ae0 2024-09-21T03:44:16.913 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T03:44:16.913 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":77,"fsid":"51116d5c-77ca-11ef-baf5-efdc52797490","created":"2024-09-21T03:35:33.631350+0000","modified":"2024-09-21T03:44:15.009933+0000","last_up_change":"2024-09-21T03:43:24.242275+0000","last_in_change":"2024-09-21T03:43:00.886552+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T03:39:29.859542+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-21T03:44:10.886435+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"77","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-21T03:44:13.041783+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"77","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1ea7148f-2ace-4c10-86fe-b8019de80c1f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6802","nonce":870188791},{"type":"v1","addr":"172.21.15.39:6803","nonce":870188791}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6804","nonce":870188791},{"type":"v1","addr":"172.21.15.39:6805","nonce":870188791}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6808","nonce":870188791},{"type":"v1","addr":"172.21.15.39:6809","nonce":870188791}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6806","nonce":870188791},{"type":"v1","addr":"172.21.15.39:6807","nonce":870188791}]},"public_addr":"172.21.15.39:6803/870188791","cluster_addr":"172.21.15.39:6805/870188791","heartbeat_back_addr":"172.21.15.39:6809/870188791","heartbeat_front_addr":"172.21.15.39:6807/870188791","state":["exists","up"]},{"osd":1,"uuid":"7cb5db90-28e8-4a30-bedb-c039687d48b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6800","nonce":172427832},{"type":"v1","addr":"172.21.15.150:6801","nonce":172427832}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6802","nonce":172427832},{"type":"v1","addr":"172.21.15.150:6803","nonce":172427832}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6806","nonce":172427832},{"type":"v1","addr":"172.21.15.150:6807","nonce":172427832}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6804","nonce":172427832},{"type":"v1","addr":"172.21.15.150:6805","nonce":172427832}]},"public_addr":"172.21.15.150:6801/172427832","cluster_addr":"172.21.15.150:6803/172427832","heartbeat_back_addr":"172.21.15.150:6807/172427832","heartbeat_front_addr":"172.21.15.150:6805/172427832","state":["exists","up"]},{"osd":2,"uuid":"b25c6ab3-fc95-4751-bd63-6e3fc23e8bd7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":19,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6800","nonce":2727876133},{"type":"v1","addr":"172.21.15.161:6801","nonce":2727876133}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6802","nonce":2727876133},{"type":"v1","addr":"172.21.15.161:6803","nonce":2727876133}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6806","nonce":2727876133},{"type":"v1","addr":"172.21.15.161:6807","nonce":2727876133}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6804","nonce":2727876133},{"type":"v1","addr":"172.21.15.161:6805","nonce":2727876133}]},"public_addr":"172.21.15.161:6801/2727876133","cluster_addr":"172.21.15.161:6803/2727876133","heartbeat_back_addr":"172.21.15.161:6807/2727876133","heartbeat_front_addr":"172.21.15.161:6805/2727876133","state":["exists","up"]},{"osd":3,"uuid":"14417ae1-4536-45ba-9a5c-c3b447d1a72d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6810","nonce":295394468},{"type":"v1","addr":"172.21.15.39:6811","nonce":295394468}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6812","nonce":295394468},{"type":"v1","addr":"172.21.15.39:6813","nonce":295394468}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6816","nonce":295394468},{"type":"v1","addr":"172.21.15.39:6817","nonce":295394468}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6814","nonce":295394468},{"type":"v1","addr":"172.21.15.39:6815","nonce":295394468}]},"public_addr":"172.21.15.39:6811/295394468","cluster_addr":"172.21.15.39:6813/295394468","heartbeat_back_addr":"172.21.15.39:6817/295394468","heartbeat_front_addr":"172.21.15.39:6815/295394468","state":["exists","up"]},{"osd":4,"uuid":"33c732ef-20c2-43c4-abe6-045b687fc521","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6808","nonce":3097721553},{"type":"v1","addr":"172.21.15.150:6809","nonce":3097721553}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6810","nonce":3097721553},{"type":"v1","addr":"172.21.15.150:6811","nonce":3097721553}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6814","nonce":3097721553},{"type":"v1","addr":"172.21.15.150:6815","nonce":3097721553}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6812","nonce":3097721553},{"type":"v1","addr":"172.21.15.150:6813","nonce":3097721553}]},"public_addr":"172.21.15.150:6809/3097721553","cluster_addr":"172.21.15.150:6811/3097721553","heartbeat_back_addr":"172.21.15.150:6815/3097721553","heartbeat_front_addr":"172.21.15.150:6813/3097721553","state":["exists","up"]},{"osd":5,"uuid":"74673722-abd6-400a-bb50-ec3dfba0c5e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6808","nonce":742751023},{"type":"v1","addr":"172.21.15.161:6809","nonce":742751023}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6810","nonce":742751023},{"type":"v1","addr":"172.21.15.161:6811","nonce":742751023}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6814","nonce":742751023},{"type":"v1","addr":"172.21.15.161:6815","nonce":742751023}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6812","nonce":742751023},{"type":"v1","addr":"172.21.15.161:6813","nonce":742751023}]},"public_addr":"172.21.15.161:6809/742751023","cluster_addr":"172.21.15.161:6811/742751023","heartbeat_back_addr":"172.21.15.161:6815/742751023","heartbeat_front_addr":"172.21.15.161:6813/742751023","state":["exists","up"]},{"osd":6,"uuid":"06548677-74c4-4efa-8054-df09b957b649","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6818","nonce":1072700290},{"type":"v1","addr":"172.21.15.39:6819","nonce":1072700290}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6820","nonce":1072700290},{"type":"v1","addr":"172.21.15.39:6821","nonce":1072700290}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6824","nonce":1072700290},{"type":"v1","addr":"172.21.15.39:6825","nonce":1072700290}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6822","nonce":1072700290},{"type":"v1","addr":"172.21.15.39:6823","nonce":1072700290}]},"public_addr":"172.21.15.39:6819/1072700290","cluster_addr":"172.21.15.39:6821/1072700290","heartbeat_back_addr":"172.21.15.39:6825/1072700290","heartbeat_front_addr":"172.21.15.39:6823/1072700290","state":["exists","up"]},{"osd":7,"uuid":"838735ca-c693-4f58-9426-b8ee857dbe3a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6816","nonce":2162619103},{"type":"v1","addr":"172.21.15.150:6817","nonce":2162619103}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6818","nonce":2162619103},{"type":"v1","addr":"172.21.15.150:6819","nonce":2162619103}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6822","nonce":2162619103},{"type":"v1","addr":"172.21.15.150:6823","nonce":2162619103}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6820","nonce":2162619103},{"type":"v1","addr":"172.21.15.150:6821","nonce":2162619103}]},"public_addr":"172.21.15.150:6817/2162619103","cluster_addr":"172.21.15.150:6819/2162619103","heartbeat_back_addr":"172.21.15.150:6823/2162619103","heartbeat_front_addr":"172.21.15.150:6821/2162619103","state":["exists","up"]},{"osd":8,"uuid":"4973b357-b4c2-4c00-b7f7-dd25012a0f72","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6816","nonce":52966633},{"type":"v1","addr":"172.21.15.161:6817","nonce":52966633}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6818","nonce":52966633},{"type":"v1","addr":"172.21.15.161:6819","nonce":52966633}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6822","nonce":52966633},{"type":"v1","addr":"172.21.15.161:6823","nonce":52966633}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6820","nonce":52966633},{"type":"v1","addr":"172.21.15.161:6821","nonce":52966633}]},"public_addr":"172.21.15.161:6817/52966633","cluster_addr":"172.21.15.161:6819/52966633","heartbeat_back_addr":"172.21.15.161:6823/52966633","heartbeat_front_addr":"172.21.15.161:6821/52966633","state":["exists","up"]},{"osd":9,"uuid":"6028613c-ebfd-42ea-9949-3f0d5f6cf025","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6826","nonce":2001303253},{"type":"v1","addr":"172.21.15.39:6827","nonce":2001303253}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6828","nonce":2001303253},{"type":"v1","addr":"172.21.15.39:6829","nonce":2001303253}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6832","nonce":2001303253},{"type":"v1","addr":"172.21.15.39:6833","nonce":2001303253}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6830","nonce":2001303253},{"type":"v1","addr":"172.21.15.39:6831","nonce":2001303253}]},"public_addr":"172.21.15.39:6827/2001303253","cluster_addr":"172.21.15.39:6829/2001303253","heartbeat_back_addr":"172.21.15.39:6833/2001303253","heartbeat_front_addr":"172.21.15.39:6831/2001303253","state":["exists","up"]},{"osd":10,"uuid":"1c71c765-2a6c-4ffe-9fe1-c8adae3d1c32","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6824","nonce":2484627938},{"type":"v1","addr":"172.21.15.150:6825","nonce":2484627938}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6826","nonce":2484627938},{"type":"v1","addr":"172.21.15.150:6827","nonce":2484627938}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6830","nonce":2484627938},{"type":"v1","addr":"172.21.15.150:6831","nonce":2484627938}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6828","nonce":2484627938},{"type":"v1","addr":"172.21.15.150:6829","nonce":2484627938}]},"public_addr":"172.21.15.150:6825/2484627938","cluster_addr":"172.21.15.150:6827/2484627938","heartbeat_back_addr":"172.21.15.150:6831/2484627938","heartbeat_front_addr":"172.21.15.150:6829/2484627938","state":["exists","up"]},{"osd":11,"uuid":"f382f522-62db-46da-997e-2b28f84d1612","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6824","nonce":2178709707},{"type":"v1","addr":"172.21.15.161:6825","nonce":2178709707}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6826","nonce":2178709707},{"type":"v1","addr":"172.21.15.161:6827","nonce":2178709707}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6830","nonce":2178709707},{"type":"v1","addr":"172.21.15.161:6831","nonce":2178709707}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6828","nonce":2178709707},{"type":"v1","addr":"172.21.15.161:6829","nonce":2178709707}]},"public_addr":"172.21.15.161:6825/2178709707","cluster_addr":"172.21.15.161:6827/2178709707","heartbeat_back_addr":"172.21.15.161:6831/2178709707","heartbeat_front_addr":"172.21.15.161:6829/2178709707","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:38:35.540904+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:39:01.455053+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:39:27.052867+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:39:55.173529+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:40:21.298396+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:40:47.131235+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:41:14.475209+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:41:39.920036+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:42:05.587065+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:42:32.624726+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:42:57.479185+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:43:22.203832+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.39:6801/2438110611":"2024-09-22T03:37:25.391457+0000","172.21.15.39:6800/2438110611":"2024-09-22T03:37:25.391457+0000","172.21.15.39:6800/173264695":"2024-09-22T03:35:57.104467+0000","172.21.15.39:6801/173264695":"2024-09-22T03:35:57.104467+0000","172.21.15.39:0/2637182284":"2024-09-22T03:35:57.104467+0000","172.21.15.39:0/1517304561":"2024-09-22T03:36:24.112592+0000","172.21.15.39:6801/1435622199":"2024-09-22T03:36:24.112592+0000","172.21.15.39:6800/1435622199":"2024-09-22T03:36:24.112592+0000","172.21.15.39:0/346832469":"2024-09-22T03:35:57.104467+0000","172.21.15.39:0/3160777371":"2024-09-22T03:36:24.112592+0000","172.21.15.39:0/1730342855":"2024-09-22T03:35:57.104467+0000","172.21.15.39:0/1955726766":"2024-09-22T03:36:24.112592+0000","172.21.15.39:0/3294132796":"2024-09-22T03:37:25.391457+0000","172.21.15.39:0/1295810162":"2024-09-22T03:37:25.391457+0000","172.21.15.39:0/3729230474":"2024-09-22T03:37:25.391457+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T03:44:16.915 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:16.913+0000 7f4b1120e640 1 -- 172.21.15.39:0/1702145567 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f4af40b6560 msgr2=0x7f4af40b8a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:16.916 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:16.913+0000 7f4b1120e640 1 --2- 172.21.15.39:0/1702145567 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f4af40b6560 0x7f4af40b8a70 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f4b00000c00 tx=0x7f4b00031400 comp rx=0 tx=0).stop 2024-09-21T03:44:16.916 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:16.913+0000 7f4b1120e640 1 -- 172.21.15.39:0/1702145567 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4b0c102ae0 msgr2=0x7f4b0c19c720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:16.916 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:16.913+0000 7f4b1120e640 1 --2- 172.21.15.39:0/1702145567 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4b0c102ae0 0x7f4b0c19c720 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f4afc00d950 tx=0x7f4afc00de10 comp rx=0 tx=0).stop 2024-09-21T03:44:16.916 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:16.913+0000 7f4b1120e640 1 -- 172.21.15.39:0/1702145567 shutdown_connections 2024-09-21T03:44:16.916 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:16.913+0000 7f4b1120e640 1 --2- 172.21.15.39:0/1702145567 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f4af40b6560 0x7f4af40b8a70 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:16.916 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:16.913+0000 7f4b1120e640 1 --2- 172.21.15.39:0/1702145567 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4b0c108d20 0x7f4b0c1a1f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:16.916 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:16.913+0000 7f4b1120e640 1 --2- 172.21.15.39:0/1702145567 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4b0c1083d0 0x7f4b0c19ccc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:16.916 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:16.913+0000 7f4b1120e640 1 --2- 172.21.15.39:0/1702145567 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4b0c102ae0 0x7f4b0c19c720 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:16.916 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:16.913+0000 7f4b1120e640 1 -- 172.21.15.39:0/1702145567 >> 172.21.15.39:0/1702145567 conn(0x7f4b0c079760 msgr2=0x7f4b0c104390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:44:16.916 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:16.913+0000 7f4b1120e640 1 -- 172.21.15.39:0/1702145567 shutdown_connections 2024-09-21T03:44:16.916 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:16.913+0000 7f4b1120e640 1 -- 172.21.15.39:0/1702145567 wait complete. 2024-09-21T03:44:17.583 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:44:17 smithi039 ceph-mon[31211]: mds.? [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] up:active 2024-09-21T03:44:17.584 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:44:17 smithi039 ceph-mon[31211]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-21T03:44:17.584 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:44:17 smithi039 ceph-mon[31211]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-21T03:44:17.584 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:44:17 smithi039 ceph-mon[31211]: from='client.? 172.21.15.39:0/1702145567' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T03:44:17.620 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 51116d5c-77ca-11ef-baf5-efdc52797490 -- ceph osd dump --format=json 2024-09-21T03:44:17.718 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:44:17 smithi161 ceph-mon[34533]: mds.? [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] up:active 2024-09-21T03:44:17.718 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:44:17 smithi161 ceph-mon[34533]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-21T03:44:17.718 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:44:17 smithi161 ceph-mon[34533]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-21T03:44:17.718 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:44:17 smithi161 ceph-mon[34533]: from='client.? 172.21.15.39:0/1702145567' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T03:44:17.724 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:44:17 smithi150 ceph-mon[34289]: mds.? [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] up:active 2024-09-21T03:44:17.725 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:44:17 smithi150 ceph-mon[34289]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-21T03:44:17.725 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:44:17 smithi150 ceph-mon[34289]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-21T03:44:17.725 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:44:17 smithi150 ceph-mon[34289]: from='client.? 172.21.15.39:0/1702145567' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T03:44:17.885 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/51116d5c-77ca-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:44:18.396 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:18.377+0000 7f3fdee83640 1 -- 172.21.15.39:0/4233531888 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3fd8103b60 msgr2=0x7f3fd8103f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:18.397 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:18.377+0000 7f3fdee83640 1 --2- 172.21.15.39:0/4233531888 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3fd8103b60 0x7f3fd8103f70 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f3fc0009900 tx=0x7f3fc002a410 comp rx=0 tx=0).stop 2024-09-21T03:44:18.397 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:18.377+0000 7f3fdee83640 1 -- 172.21.15.39:0/4233531888 shutdown_connections 2024-09-21T03:44:18.397 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:18.377+0000 7f3fdee83640 1 --2- 172.21.15.39:0/4233531888 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3fd8109df0 0x7f3fd8110620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:18.397 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:18.377+0000 7f3fdee83640 1 --2- 172.21.15.39:0/4233531888 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3fd81094a0 0x7f3fd81098b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:18.397 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:18.377+0000 7f3fdee83640 1 --2- 172.21.15.39:0/4233531888 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3fd8103b60 0x7f3fd8103f70 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:18.397 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:18.377+0000 7f3fdee83640 1 -- 172.21.15.39:0/4233531888 >> 172.21.15.39:0/4233531888 conn(0x7f3fd807b850 msgr2=0x7f3fd807bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:44:18.397 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:18.377+0000 7f3fdee83640 1 -- 172.21.15.39:0/4233531888 shutdown_connections 2024-09-21T03:44:18.397 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:18.377+0000 7f3fdee83640 1 -- 172.21.15.39:0/4233531888 wait complete. 2024-09-21T03:44:18.397 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:18.378+0000 7f3fdee83640 1 Processor -- start 2024-09-21T03:44:18.397 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:18.378+0000 7f3fdee83640 1 -- start start 2024-09-21T03:44:18.397 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:18.378+0000 7f3fdee83640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3fd8103b60 0x7f3fd819a4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:18.397 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:18.378+0000 7f3fdee83640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3fd81094a0 0x7f3fd819aa40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:18.397 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:18.378+0000 7f3fdee83640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3fd8109df0 0x7f3fd819fce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:18.398 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:18.378+0000 7f3fdee83640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f3fd8113ab0 con 0x7f3fd81094a0 2024-09-21T03:44:18.398 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:18.378+0000 7f3fdee83640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f3fd8113930 con 0x7f3fd8103b60 2024-09-21T03:44:18.398 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:18.378+0000 7f3fdee83640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f3fd8113c30 con 0x7f3fd8109df0 2024-09-21T03:44:18.398 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:18.378+0000 7f3fd7fff640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3fd81094a0 0x7f3fd819aa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:18.398 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:18.378+0000 7f3fd7fff640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3fd81094a0 0x7f3fd819aa40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:57570/0 (socket says 172.21.15.39:57570) 2024-09-21T03:44:18.398 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:18.378+0000 7f3fd7fff640 1 -- 172.21.15.39:0/2647344900 learned_addr learned my addr 172.21.15.39:0/2647344900 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:44:18.398 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:18.378+0000 7f3fdcc1f640 1 --2- 172.21.15.39:0/2647344900 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3fd8103b60 0x7f3fd819a4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:18.398 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:18.379+0000 7f3fdd420640 1 --2- 172.21.15.39:0/2647344900 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3fd8109df0 0x7f3fd819fce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:18.398 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:18.379+0000 7f3fd7fff640 1 -- 172.21.15.39:0/2647344900 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3fd8109df0 msgr2=0x7f3fd819fce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:18.398 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:18.379+0000 7f3fd7fff640 1 --2- 172.21.15.39:0/2647344900 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3fd8109df0 0x7f3fd819fce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:18.398 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:18.379+0000 7f3fd7fff640 1 -- 172.21.15.39:0/2647344900 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3fd8103b60 msgr2=0x7f3fd819a4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:18.398 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:18.379+0000 7f3fd7fff640 1 --2- 172.21.15.39:0/2647344900 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3fd8103b60 0x7f3fd819a4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:18.399 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:18.379+0000 7f3fd7fff640 1 -- 172.21.15.39:0/2647344900 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3fd81a0410 con 0x7f3fd81094a0 2024-09-21T03:44:18.399 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:18.379+0000 7f3fdd420640 1 --2- 172.21.15.39:0/2647344900 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3fd8109df0 0x7f3fd819fce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:44:18.399 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:18.379+0000 7f3fd7fff640 1 --2- 172.21.15.39:0/2647344900 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3fd81094a0 0x7f3fd819aa40 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f3fc800ea20 tx=0x7f3fc800eee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:44:18.399 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:18.379+0000 7f3fd5ffb640 1 -- 172.21.15.39:0/2647344900 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fc800cde0 con 0x7f3fd81094a0 2024-09-21T03:44:18.399 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:18.379+0000 7f3fdee83640 1 -- 172.21.15.39:0/2647344900 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3fd81a0700 con 0x7f3fd81094a0 2024-09-21T03:44:18.399 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:18.379+0000 7f3fd5ffb640 1 -- 172.21.15.39:0/2647344900 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3fc8005020 con 0x7f3fd81094a0 2024-09-21T03:44:18.399 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:18.379+0000 7f3fd5ffb640 1 -- 172.21.15.39:0/2647344900 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fc80116d0 con 0x7f3fd81094a0 2024-09-21T03:44:18.399 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:18.379+0000 7f3fdee83640 1 -- 172.21.15.39:0/2647344900 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3fd81a6e60 con 0x7f3fd81094a0 2024-09-21T03:44:18.399 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:18.380+0000 7f3fdee83640 1 -- 172.21.15.39:0/2647344900 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3fd8103f70 con 0x7f3fd81094a0 2024-09-21T03:44:18.399 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:18.382+0000 7f3fd5ffb640 1 -- 172.21.15.39:0/2647344900 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 21) ==== 154529+0+0 (secure 0 0 0) 0x7f3fc80051d0 con 0x7f3fd81094a0 2024-09-21T03:44:18.399 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:18.383+0000 7f3fd5ffb640 1 --2- 172.21.15.39:0/2647344900 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f3fc40b6560 0x7f3fc40b8a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:18.399 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:18.383+0000 7f3fdcc1f640 1 --2- 172.21.15.39:0/2647344900 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f3fc40b6560 0x7f3fc40b8a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:18.399 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:18.383+0000 7f3fd5ffb640 1 -- 172.21.15.39:0/2647344900 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8194+0+0 (secure 0 0 0) 0x7f3fc80e7920 con 0x7f3fd81094a0 2024-09-21T03:44:18.399 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:18.383+0000 7f3fdcc1f640 1 --2- 172.21.15.39:0/2647344900 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f3fc40b6560 0x7f3fc40b8a70 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f3fc0031440 tx=0x7f3fc0009560 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:44:18.400 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:18.387+0000 7f3fd5ffb640 1 -- 172.21.15.39:0/2647344900 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f3fc8016030 con 0x7f3fd81094a0 2024-09-21T03:44:18.578 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:18.576+0000 7f3fdee83640 1 -- 172.21.15.39:0/2647344900 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f3fd810b630 con 0x7f3fd81094a0 2024-09-21T03:44:18.579 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:44:18 smithi039 ceph-mon[31211]: pgmap v271: 97 pgs: 31 creating+peering, 12 unknown, 54 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 0 op/s 2024-09-21T03:44:18.580 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:18.577+0000 7f3fd5ffb640 1 -- 172.21.15.39:0/2647344900 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v77) ==== 74+0+22570 (secure 0 0 0) 0x7f3fc80ae1d0 con 0x7f3fd81094a0 2024-09-21T03:44:18.580 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T03:44:18.580 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":77,"fsid":"51116d5c-77ca-11ef-baf5-efdc52797490","created":"2024-09-21T03:35:33.631350+0000","modified":"2024-09-21T03:44:15.009933+0000","last_up_change":"2024-09-21T03:43:24.242275+0000","last_in_change":"2024-09-21T03:43:00.886552+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T03:39:29.859542+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-21T03:44:10.886435+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"77","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-21T03:44:13.041783+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"77","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1ea7148f-2ace-4c10-86fe-b8019de80c1f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6802","nonce":870188791},{"type":"v1","addr":"172.21.15.39:6803","nonce":870188791}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6804","nonce":870188791},{"type":"v1","addr":"172.21.15.39:6805","nonce":870188791}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6808","nonce":870188791},{"type":"v1","addr":"172.21.15.39:6809","nonce":870188791}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6806","nonce":870188791},{"type":"v1","addr":"172.21.15.39:6807","nonce":870188791}]},"public_addr":"172.21.15.39:6803/870188791","cluster_addr":"172.21.15.39:6805/870188791","heartbeat_back_addr":"172.21.15.39:6809/870188791","heartbeat_front_addr":"172.21.15.39:6807/870188791","state":["exists","up"]},{"osd":1,"uuid":"7cb5db90-28e8-4a30-bedb-c039687d48b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6800","nonce":172427832},{"type":"v1","addr":"172.21.15.150:6801","nonce":172427832}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6802","nonce":172427832},{"type":"v1","addr":"172.21.15.150:6803","nonce":172427832}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6806","nonce":172427832},{"type":"v1","addr":"172.21.15.150:6807","nonce":172427832}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6804","nonce":172427832},{"type":"v1","addr":"172.21.15.150:6805","nonce":172427832}]},"public_addr":"172.21.15.150:6801/172427832","cluster_addr":"172.21.15.150:6803/172427832","heartbeat_back_addr":"172.21.15.150:6807/172427832","heartbeat_front_addr":"172.21.15.150:6805/172427832","state":["exists","up"]},{"osd":2,"uuid":"b25c6ab3-fc95-4751-bd63-6e3fc23e8bd7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":19,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6800","nonce":2727876133},{"type":"v1","addr":"172.21.15.161:6801","nonce":2727876133}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6802","nonce":2727876133},{"type":"v1","addr":"172.21.15.161:6803","nonce":2727876133}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6806","nonce":2727876133},{"type":"v1","addr":"172.21.15.161:6807","nonce":2727876133}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6804","nonce":2727876133},{"type":"v1","addr":"172.21.15.161:6805","nonce":2727876133}]},"public_addr":"172.21.15.161:6801/2727876133","cluster_addr":"172.21.15.161:6803/2727876133","heartbeat_back_addr":"172.21.15.161:6807/2727876133","heartbeat_front_addr":"172.21.15.161:6805/2727876133","state":["exists","up"]},{"osd":3,"uuid":"14417ae1-4536-45ba-9a5c-c3b447d1a72d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6810","nonce":295394468},{"type":"v1","addr":"172.21.15.39:6811","nonce":295394468}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6812","nonce":295394468},{"type":"v1","addr":"172.21.15.39:6813","nonce":295394468}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6816","nonce":295394468},{"type":"v1","addr":"172.21.15.39:6817","nonce":295394468}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6814","nonce":295394468},{"type":"v1","addr":"172.21.15.39:6815","nonce":295394468}]},"public_addr":"172.21.15.39:6811/295394468","cluster_addr":"172.21.15.39:6813/295394468","heartbeat_back_addr":"172.21.15.39:6817/295394468","heartbeat_front_addr":"172.21.15.39:6815/295394468","state":["exists","up"]},{"osd":4,"uuid":"33c732ef-20c2-43c4-abe6-045b687fc521","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6808","nonce":3097721553},{"type":"v1","addr":"172.21.15.150:6809","nonce":3097721553}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6810","nonce":3097721553},{"type":"v1","addr":"172.21.15.150:6811","nonce":3097721553}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6814","nonce":3097721553},{"type":"v1","addr":"172.21.15.150:6815","nonce":3097721553}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6812","nonce":3097721553},{"type":"v1","addr":"172.21.15.150:6813","nonce":3097721553}]},"public_addr":"172.21.15.150:6809/3097721553","cluster_addr":"172.21.15.150:6811/3097721553","heartbeat_back_addr":"172.21.15.150:6815/3097721553","heartbeat_front_addr":"172.21.15.150:6813/3097721553","state":["exists","up"]},{"osd":5,"uuid":"74673722-abd6-400a-bb50-ec3dfba0c5e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6808","nonce":742751023},{"type":"v1","addr":"172.21.15.161:6809","nonce":742751023}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6810","nonce":742751023},{"type":"v1","addr":"172.21.15.161:6811","nonce":742751023}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6814","nonce":742751023},{"type":"v1","addr":"172.21.15.161:6815","nonce":742751023}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6812","nonce":742751023},{"type":"v1","addr":"172.21.15.161:6813","nonce":742751023}]},"public_addr":"172.21.15.161:6809/742751023","cluster_addr":"172.21.15.161:6811/742751023","heartbeat_back_addr":"172.21.15.161:6815/742751023","heartbeat_front_addr":"172.21.15.161:6813/742751023","state":["exists","up"]},{"osd":6,"uuid":"06548677-74c4-4efa-8054-df09b957b649","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6818","nonce":1072700290},{"type":"v1","addr":"172.21.15.39:6819","nonce":1072700290}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6820","nonce":1072700290},{"type":"v1","addr":"172.21.15.39:6821","nonce":1072700290}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6824","nonce":1072700290},{"type":"v1","addr":"172.21.15.39:6825","nonce":1072700290}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6822","nonce":1072700290},{"type":"v1","addr":"172.21.15.39:6823","nonce":1072700290}]},"public_addr":"172.21.15.39:6819/1072700290","cluster_addr":"172.21.15.39:6821/1072700290","heartbeat_back_addr":"172.21.15.39:6825/1072700290","heartbeat_front_addr":"172.21.15.39:6823/1072700290","state":["exists","up"]},{"osd":7,"uuid":"838735ca-c693-4f58-9426-b8ee857dbe3a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6816","nonce":2162619103},{"type":"v1","addr":"172.21.15.150:6817","nonce":2162619103}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6818","nonce":2162619103},{"type":"v1","addr":"172.21.15.150:6819","nonce":2162619103}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6822","nonce":2162619103},{"type":"v1","addr":"172.21.15.150:6823","nonce":2162619103}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6820","nonce":2162619103},{"type":"v1","addr":"172.21.15.150:6821","nonce":2162619103}]},"public_addr":"172.21.15.150:6817/2162619103","cluster_addr":"172.21.15.150:6819/2162619103","heartbeat_back_addr":"172.21.15.150:6823/2162619103","heartbeat_front_addr":"172.21.15.150:6821/2162619103","state":["exists","up"]},{"osd":8,"uuid":"4973b357-b4c2-4c00-b7f7-dd25012a0f72","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6816","nonce":52966633},{"type":"v1","addr":"172.21.15.161:6817","nonce":52966633}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6818","nonce":52966633},{"type":"v1","addr":"172.21.15.161:6819","nonce":52966633}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6822","nonce":52966633},{"type":"v1","addr":"172.21.15.161:6823","nonce":52966633}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6820","nonce":52966633},{"type":"v1","addr":"172.21.15.161:6821","nonce":52966633}]},"public_addr":"172.21.15.161:6817/52966633","cluster_addr":"172.21.15.161:6819/52966633","heartbeat_back_addr":"172.21.15.161:6823/52966633","heartbeat_front_addr":"172.21.15.161:6821/52966633","state":["exists","up"]},{"osd":9,"uuid":"6028613c-ebfd-42ea-9949-3f0d5f6cf025","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6826","nonce":2001303253},{"type":"v1","addr":"172.21.15.39:6827","nonce":2001303253}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6828","nonce":2001303253},{"type":"v1","addr":"172.21.15.39:6829","nonce":2001303253}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6832","nonce":2001303253},{"type":"v1","addr":"172.21.15.39:6833","nonce":2001303253}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6830","nonce":2001303253},{"type":"v1","addr":"172.21.15.39:6831","nonce":2001303253}]},"public_addr":"172.21.15.39:6827/2001303253","cluster_addr":"172.21.15.39:6829/2001303253","heartbeat_back_addr":"172.21.15.39:6833/2001303253","heartbeat_front_addr":"172.21.15.39:6831/2001303253","state":["exists","up"]},{"osd":10,"uuid":"1c71c765-2a6c-4ffe-9fe1-c8adae3d1c32","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6824","nonce":2484627938},{"type":"v1","addr":"172.21.15.150:6825","nonce":2484627938}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6826","nonce":2484627938},{"type":"v1","addr":"172.21.15.150:6827","nonce":2484627938}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6830","nonce":2484627938},{"type":"v1","addr":"172.21.15.150:6831","nonce":2484627938}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6828","nonce":2484627938},{"type":"v1","addr":"172.21.15.150:6829","nonce":2484627938}]},"public_addr":"172.21.15.150:6825/2484627938","cluster_addr":"172.21.15.150:6827/2484627938","heartbeat_back_addr":"172.21.15.150:6831/2484627938","heartbeat_front_addr":"172.21.15.150:6829/2484627938","state":["exists","up"]},{"osd":11,"uuid":"f382f522-62db-46da-997e-2b28f84d1612","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6824","nonce":2178709707},{"type":"v1","addr":"172.21.15.161:6825","nonce":2178709707}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6826","nonce":2178709707},{"type":"v1","addr":"172.21.15.161:6827","nonce":2178709707}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6830","nonce":2178709707},{"type":"v1","addr":"172.21.15.161:6831","nonce":2178709707}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6828","nonce":2178709707},{"type":"v1","addr":"172.21.15.161:6829","nonce":2178709707}]},"public_addr":"172.21.15.161:6825/2178709707","cluster_addr":"172.21.15.161:6827/2178709707","heartbeat_back_addr":"172.21.15.161:6831/2178709707","heartbeat_front_addr":"172.21.15.161:6829/2178709707","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:38:35.540904+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:39:01.455053+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:39:27.052867+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:39:55.173529+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:40:21.298396+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:40:47.131235+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:41:14.475209+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:41:39.920036+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:42:05.587065+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:42:32.624726+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:42:57.479185+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:43:22.203832+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.39:6801/2438110611":"2024-09-22T03:37:25.391457+0000","172.21.15.39:6800/2438110611":"2024-09-22T03:37:25.391457+0000","172.21.15.39:6800/173264695":"2024-09-22T03:35:57.104467+0000","172.21.15.39:6801/173264695":"2024-09-22T03:35:57.104467+0000","172.21.15.39:0/2637182284":"2024-09-22T03:35:57.104467+0000","172.21.15.39:0/1517304561":"2024-09-22T03:36:24.112592+0000","172.21.15.39:6801/1435622199":"2024-09-22T03:36:24.112592+0000","172.21.15.39:6800/1435622199":"2024-09-22T03:36:24.112592+0000","172.21.15.39:0/346832469":"2024-09-22T03:35:57.104467+0000","172.21.15.39:0/3160777371":"2024-09-22T03:36:24.112592+0000","172.21.15.39:0/1730342855":"2024-09-22T03:35:57.104467+0000","172.21.15.39:0/1955726766":"2024-09-22T03:36:24.112592+0000","172.21.15.39:0/3294132796":"2024-09-22T03:37:25.391457+0000","172.21.15.39:0/1295810162":"2024-09-22T03:37:25.391457+0000","172.21.15.39:0/3729230474":"2024-09-22T03:37:25.391457+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T03:44:18.582 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:18.580+0000 7f3fdee83640 1 -- 172.21.15.39:0/2647344900 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f3fc40b6560 msgr2=0x7f3fc40b8a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:18.582 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:18.580+0000 7f3fdee83640 1 --2- 172.21.15.39:0/2647344900 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f3fc40b6560 0x7f3fc40b8a70 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f3fc0031440 tx=0x7f3fc0009560 comp rx=0 tx=0).stop 2024-09-21T03:44:18.583 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:18.580+0000 7f3fdee83640 1 -- 172.21.15.39:0/2647344900 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3fd81094a0 msgr2=0x7f3fd819aa40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:18.583 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:18.580+0000 7f3fdee83640 1 --2- 172.21.15.39:0/2647344900 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3fd81094a0 0x7f3fd819aa40 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f3fc800ea20 tx=0x7f3fc800eee0 comp rx=0 tx=0).stop 2024-09-21T03:44:18.583 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:18.580+0000 7f3fdee83640 1 -- 172.21.15.39:0/2647344900 shutdown_connections 2024-09-21T03:44:18.583 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:18.580+0000 7f3fdee83640 1 --2- 172.21.15.39:0/2647344900 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f3fc40b6560 0x7f3fc40b8a70 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:18.583 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:18.580+0000 7f3fdee83640 1 --2- 172.21.15.39:0/2647344900 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3fd8109df0 0x7f3fd819fce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:18.583 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:18.580+0000 7f3fdee83640 1 --2- 172.21.15.39:0/2647344900 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3fd81094a0 0x7f3fd819aa40 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:18.583 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:18.580+0000 7f3fdee83640 1 --2- 172.21.15.39:0/2647344900 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3fd8103b60 0x7f3fd819a4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:18.583 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:18.580+0000 7f3fdee83640 1 -- 172.21.15.39:0/2647344900 >> 172.21.15.39:0/2647344900 conn(0x7f3fd807b850 msgr2=0x7f3fd8107530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:44:18.583 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:18.580+0000 7f3fdee83640 1 -- 172.21.15.39:0/2647344900 shutdown_connections 2024-09-21T03:44:18.583 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:18.580+0000 7f3fdee83640 1 -- 172.21.15.39:0/2647344900 wait complete. 2024-09-21T03:44:18.718 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:44:18 smithi161 ceph-mon[34533]: pgmap v271: 97 pgs: 31 creating+peering, 12 unknown, 54 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 0 op/s 2024-09-21T03:44:18.724 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:44:18 smithi150 ceph-mon[34289]: pgmap v271: 97 pgs: 31 creating+peering, 12 unknown, 54 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 0 op/s 2024-09-21T03:44:19.197 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 51116d5c-77ca-11ef-baf5-efdc52797490 -- ceph fs set cephfs standby_count_wanted 0 2024-09-21T03:44:19.455 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/51116d5c-77ca-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:44:19.467 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:44:19 smithi161 ceph-mon[34533]: from='client.? 172.21.15.39:0/2647344900' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T03:44:19.610 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:44:19 smithi039 ceph-mon[31211]: from='client.? 172.21.15.39:0/2647344900' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T03:44:19.724 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:44:19 smithi150 ceph-mon[34289]: from='client.? 172.21.15.39:0/2647344900' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T03:44:19.925 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:19.922+0000 7f940eb67640 1 -- 172.21.15.39:0/3990741481 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9408108d20 msgr2=0x7f940810f550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:19.925 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:19.922+0000 7f940eb67640 1 --2- 172.21.15.39:0/3990741481 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9408108d20 0x7f940810f550 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f9400009ca0 tx=0x7f940002a8c0 comp rx=0 tx=0).stop 2024-09-21T03:44:19.925 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:19.922+0000 7f940eb67640 1 -- 172.21.15.39:0/3990741481 shutdown_connections 2024-09-21T03:44:19.925 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:19.922+0000 7f940eb67640 1 --2- 172.21.15.39:0/3990741481 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9408108d20 0x7f940810f550 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:19.926 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:19.922+0000 7f940eb67640 1 --2- 172.21.15.39:0/3990741481 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f94081083d0 0x7f94081087e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:19.926 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:19.922+0000 7f940eb67640 1 --2- 172.21.15.39:0/3990741481 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9408102ae0 0x7f9408102ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:19.926 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:19.922+0000 7f940eb67640 1 -- 172.21.15.39:0/3990741481 >> 172.21.15.39:0/3990741481 conn(0x7f9408079790 msgr2=0x7f9408079ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:44:19.926 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:19.922+0000 7f940eb67640 1 -- 172.21.15.39:0/3990741481 shutdown_connections 2024-09-21T03:44:19.926 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:19.922+0000 7f940eb67640 1 -- 172.21.15.39:0/3990741481 wait complete. 2024-09-21T03:44:19.928 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:19.923+0000 7f940eb67640 1 Processor -- start 2024-09-21T03:44:19.928 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:19.923+0000 7f940eb67640 1 -- start start 2024-09-21T03:44:19.928 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:19.923+0000 7f940eb67640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9408102ae0 0x7f940819a510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:19.929 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:19.923+0000 7f940eb67640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f94081083d0 0x7f940819aab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:19.929 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:19.923+0000 7f940eb67640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9408108d20 0x7f940819fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:19.929 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:19.923+0000 7f940eb67640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f9408113b00 con 0x7f9408102ae0 2024-09-21T03:44:19.929 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:19.924+0000 7f940eb67640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f9408113980 con 0x7f9408108d20 2024-09-21T03:44:19.929 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:19.924+0000 7f940eb67640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f9408113c80 con 0x7f94081083d0 2024-09-21T03:44:19.929 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:19.924+0000 7f940c903640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9408102ae0 0x7f940819a510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:19.929 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:19.924+0000 7f940c903640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9408102ae0 0x7f940819a510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:57600/0 (socket says 172.21.15.39:57600) 2024-09-21T03:44:19.929 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:19.924+0000 7f940c903640 1 -- 172.21.15.39:0/3228197270 learned_addr learned my addr 172.21.15.39:0/3228197270 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:44:19.929 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:19.924+0000 7f9407fff640 1 --2- 172.21.15.39:0/3228197270 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f94081083d0 0x7f940819aab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:19.929 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:19.924+0000 7f940d104640 1 --2- 172.21.15.39:0/3228197270 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9408108d20 0x7f940819fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:19.929 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:19.924+0000 7f940c903640 1 -- 172.21.15.39:0/3228197270 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f94081083d0 msgr2=0x7f940819aab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:19.929 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:19.924+0000 7f940c903640 1 --2- 172.21.15.39:0/3228197270 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f94081083d0 0x7f940819aab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:19.929 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:19.924+0000 7f940c903640 1 -- 172.21.15.39:0/3228197270 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9408108d20 msgr2=0x7f940819fd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:19.930 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:19.924+0000 7f940c903640 1 --2- 172.21.15.39:0/3228197270 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9408108d20 0x7f940819fd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:19.930 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:19.924+0000 7f940c903640 1 -- 172.21.15.39:0/3228197270 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f94081a0480 con 0x7f9408102ae0 2024-09-21T03:44:19.930 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:19.924+0000 7f9407fff640 1 --2- 172.21.15.39:0/3228197270 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f94081083d0 0x7f940819aab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:44:19.930 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:19.924+0000 7f940c903640 1 --2- 172.21.15.39:0/3228197270 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9408102ae0 0x7f940819a510 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f93fc00d950 tx=0x7f93fc00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:44:19.930 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:19.924+0000 7f9405ffb640 1 -- 172.21.15.39:0/3228197270 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93fc017030 con 0x7f9408102ae0 2024-09-21T03:44:19.930 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:19.924+0000 7f9405ffb640 1 -- 172.21.15.39:0/3228197270 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f93fc004020 con 0x7f9408102ae0 2024-09-21T03:44:19.930 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:19.924+0000 7f940eb67640 1 -- 172.21.15.39:0/3228197270 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f94081a0770 con 0x7f9408102ae0 2024-09-21T03:44:19.930 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:19.924+0000 7f9405ffb640 1 -- 172.21.15.39:0/3228197270 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93fc0047a0 con 0x7f9408102ae0 2024-09-21T03:44:19.930 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:19.925+0000 7f940eb67640 1 -- 172.21.15.39:0/3228197270 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f94081a6ed0 con 0x7f9408102ae0 2024-09-21T03:44:19.930 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:19.926+0000 7f940eb67640 1 -- 172.21.15.39:0/3228197270 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9408103d10 con 0x7f9408102ae0 2024-09-21T03:44:19.931 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:19.927+0000 7f9405ffb640 1 -- 172.21.15.39:0/3228197270 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 21) ==== 154529+0+0 (secure 0 0 0) 0x7f93fc009a30 con 0x7f9408102ae0 2024-09-21T03:44:19.931 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:19.928+0000 7f9405ffb640 1 --2- 172.21.15.39:0/3228197270 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f93f80b6560 0x7f93f80b8a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:19.931 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:19.928+0000 7f9407fff640 1 --2- 172.21.15.39:0/3228197270 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f93f80b6560 0x7f93f80b8a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:19.931 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:19.928+0000 7f9405ffb640 1 -- 172.21.15.39:0/3228197270 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8194+0+0 (secure 0 0 0) 0x7f93fc0e6d00 con 0x7f9408102ae0 2024-09-21T03:44:19.931 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:19.929+0000 7f9407fff640 1 --2- 172.21.15.39:0/3228197270 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f93f80b6560 0x7f93f80b8a70 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f9408107ef0 tx=0x7f93f4005f90 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:44:19.935 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:19.933+0000 7f9405ffb640 1 -- 172.21.15.39:0/3228197270 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f93fc0ad5b0 con 0x7f9408102ae0 2024-09-21T03:44:20.163 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:20.160+0000 7f940eb67640 1 -- 172.21.15.39:0/3228197270 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f940802e640 con 0x7f9408102ae0 2024-09-21T03:44:20.354 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:20.351+0000 7f9405ffb640 1 -- 172.21.15.39:0/3228197270 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v17) ==== 118+0+0 (secure 0 0 0) 0x7f93fc0b1990 con 0x7f9408102ae0 2024-09-21T03:44:20.356 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:20.354+0000 7f940eb67640 1 -- 172.21.15.39:0/3228197270 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f93f80b6560 msgr2=0x7f93f80b8a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:20.357 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:20.354+0000 7f940eb67640 1 --2- 172.21.15.39:0/3228197270 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f93f80b6560 0x7f93f80b8a70 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f9408107ef0 tx=0x7f93f4005f90 comp rx=0 tx=0).stop 2024-09-21T03:44:20.357 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:20.354+0000 7f940eb67640 1 -- 172.21.15.39:0/3228197270 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9408102ae0 msgr2=0x7f940819a510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:20.357 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:20.354+0000 7f940eb67640 1 --2- 172.21.15.39:0/3228197270 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9408102ae0 0x7f940819a510 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f93fc00d950 tx=0x7f93fc00de10 comp rx=0 tx=0).stop 2024-09-21T03:44:20.357 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:20.354+0000 7f940eb67640 1 -- 172.21.15.39:0/3228197270 shutdown_connections 2024-09-21T03:44:20.357 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:20.354+0000 7f940eb67640 1 --2- 172.21.15.39:0/3228197270 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f93f80b6560 0x7f93f80b8a70 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:20.357 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:20.354+0000 7f940eb67640 1 --2- 172.21.15.39:0/3228197270 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9408108d20 0x7f940819fd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:20.357 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:20.354+0000 7f940eb67640 1 --2- 172.21.15.39:0/3228197270 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f94081083d0 0x7f940819aab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:20.357 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:20.354+0000 7f940eb67640 1 --2- 172.21.15.39:0/3228197270 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9408102ae0 0x7f940819a510 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:20.357 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:20.354+0000 7f940eb67640 1 -- 172.21.15.39:0/3228197270 >> 172.21.15.39:0/3228197270 conn(0x7f9408079790 msgr2=0x7f9408106460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:44:20.357 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:20.354+0000 7f940eb67640 1 -- 172.21.15.39:0/3228197270 shutdown_connections 2024-09-21T03:44:20.357 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:20.354+0000 7f940eb67640 1 -- 172.21.15.39:0/3228197270 wait complete. 2024-09-21T03:44:20.583 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:44:20 smithi039 ceph-mon[31211]: pgmap v272: 97 pgs: 97 active+clean; 578 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 993 B/s wr, 5 op/s 2024-09-21T03:44:20.583 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:44:20 smithi039 ceph-mon[31211]: from='client.? 172.21.15.39:0/3228197270' entity='client.admin' cmd={"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} : dispatch 2024-09-21T03:44:20.718 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:44:20 smithi161 ceph-mon[34533]: pgmap v272: 97 pgs: 97 active+clean; 578 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 993 B/s wr, 5 op/s 2024-09-21T03:44:20.718 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:44:20 smithi161 ceph-mon[34533]: from='client.? 172.21.15.39:0/3228197270' entity='client.admin' cmd={"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} : dispatch 2024-09-21T03:44:20.724 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:44:20 smithi150 ceph-mon[34289]: pgmap v272: 97 pgs: 97 active+clean; 578 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 993 B/s wr, 5 op/s 2024-09-21T03:44:20.724 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:44:20 smithi150 ceph-mon[34289]: from='client.? 172.21.15.39:0/3228197270' entity='client.admin' cmd={"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} : dispatch 2024-09-21T03:44:20.972 DEBUG:tasks.cephfs.filesystem:fs_config: {'max_mds': 5, 'session_timeout': 300, 'standby_replay': True, 'subvols': {'create': 2, 'subvol_options': '--namespace-isolated --size 25000000000'}} 2024-09-21T03:44:20.972 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 51116d5c-77ca-11ef-baf5-efdc52797490 -- ceph fs set cephfs max_mds 5 2024-09-21T03:44:21.230 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/51116d5c-77ca-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:44:21.356 INFO:journalctl@ceph.mds.b.smithi150.stdout:Sep 21 03:44:21 smithi150 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-mds-b[63521]: 2024-09-21T03:44:20.997+0000 7efca1db9640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-21T03:44:21.620 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:44:21 smithi039 ceph-mon[31211]: from='client.? 172.21.15.39:0/3228197270' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-09-21T03:44:21.620 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:44:21 smithi039 ceph-mon[31211]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-21T03:44:21.717 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:44:21 smithi161 ceph-mon[34533]: from='client.? 172.21.15.39:0/3228197270' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-09-21T03:44:21.718 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:44:21 smithi161 ceph-mon[34533]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-21T03:44:21.724 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:44:21 smithi150 ceph-mon[34289]: from='client.? 172.21.15.39:0/3228197270' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-09-21T03:44:21.724 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:44:21 smithi150 ceph-mon[34289]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-21T03:44:21.729 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:21.714+0000 7f65009e8640 1 -- 172.21.15.39:0/1510648668 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f64fc102a60 msgr2=0x7f64fc102e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:21.730 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:21.714+0000 7f65009e8640 1 --2- 172.21.15.39:0/1510648668 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f64fc102a60 0x7f64fc102e70 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f64ec009900 tx=0x7f64ec02a410 comp rx=0 tx=0).stop 2024-09-21T03:44:21.730 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:21.714+0000 7f65009e8640 1 -- 172.21.15.39:0/1510648668 shutdown_connections 2024-09-21T03:44:21.730 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:21.714+0000 7f65009e8640 1 --2- 172.21.15.39:0/1510648668 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f64fc108cf0 0x7f64fc10f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:21.730 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:21.714+0000 7f65009e8640 1 --2- 172.21.15.39:0/1510648668 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f64fc1083a0 0x7f64fc1087b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:21.730 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:21.714+0000 7f65009e8640 1 --2- 172.21.15.39:0/1510648668 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f64fc102a60 0x7f64fc102e70 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:21.730 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:21.714+0000 7f65009e8640 1 -- 172.21.15.39:0/1510648668 >> 172.21.15.39:0/1510648668 conn(0x7f64fc079730 msgr2=0x7f64fc079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:44:21.730 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:21.714+0000 7f65009e8640 1 -- 172.21.15.39:0/1510648668 shutdown_connections 2024-09-21T03:44:21.730 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:21.714+0000 7f65009e8640 1 -- 172.21.15.39:0/1510648668 wait complete. 2024-09-21T03:44:21.730 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:21.715+0000 7f65009e8640 1 Processor -- start 2024-09-21T03:44:21.730 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:21.715+0000 7f65009e8640 1 -- start start 2024-09-21T03:44:21.730 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:21.715+0000 7f65009e8640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f64fc102a60 0x7f64fc19a5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:21.730 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:21.715+0000 7f65009e8640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f64fc1083a0 0x7f64fc19ab60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:21.731 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:21.715+0000 7f65009e8640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f64fc108cf0 0x7f64fc19fe00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:21.731 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:21.715+0000 7f65009e8640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f64fc1129c0 con 0x7f64fc1083a0 2024-09-21T03:44:21.731 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:21.715+0000 7f65009e8640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f64fc112840 con 0x7f64fc108cf0 2024-09-21T03:44:21.731 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:21.715+0000 7f65009e8640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f64fc112b40 con 0x7f64fc102a60 2024-09-21T03:44:21.731 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:21.715+0000 7f64f3fff640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f64fc1083a0 0x7f64fc19ab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:21.731 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:21.716+0000 7f64f3fff640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f64fc1083a0 0x7f64fc19ab60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:57606/0 (socket says 172.21.15.39:57606) 2024-09-21T03:44:21.731 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:21.716+0000 7f64f3fff640 1 -- 172.21.15.39:0/4287294636 learned_addr learned my addr 172.21.15.39:0/4287294636 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:44:21.731 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:21.716+0000 7f64fa59c640 1 --2- 172.21.15.39:0/4287294636 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f64fc102a60 0x7f64fc19a5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:21.731 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:21.716+0000 7f64fad9d640 1 --2- 172.21.15.39:0/4287294636 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f64fc108cf0 0x7f64fc19fe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:21.731 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:21.716+0000 7f64f3fff640 1 -- 172.21.15.39:0/4287294636 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f64fc102a60 msgr2=0x7f64fc19a5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:21.731 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:21.716+0000 7f64f3fff640 1 --2- 172.21.15.39:0/4287294636 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f64fc102a60 0x7f64fc19a5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:21.731 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:21.716+0000 7f64f3fff640 1 -- 172.21.15.39:0/4287294636 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f64fc108cf0 msgr2=0x7f64fc19fe00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:21.731 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:21.716+0000 7f64f3fff640 1 --2- 172.21.15.39:0/4287294636 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f64fc108cf0 0x7f64fc19fe00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:21.731 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:21.716+0000 7f64f3fff640 1 -- 172.21.15.39:0/4287294636 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f64fc1a0470 con 0x7f64fc1083a0 2024-09-21T03:44:21.731 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:21.716+0000 7f64f3fff640 1 --2- 172.21.15.39:0/4287294636 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f64fc1083a0 0x7f64fc19ab60 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f64e400b450 tx=0x7f64e400b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:44:21.732 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:21.716+0000 7f64f37fe640 1 -- 172.21.15.39:0/4287294636 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64e4015020 con 0x7f64fc1083a0 2024-09-21T03:44:21.732 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:21.716+0000 7f64f37fe640 1 -- 172.21.15.39:0/4287294636 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f64e4011d40 con 0x7f64fc1083a0 2024-09-21T03:44:21.732 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:21.716+0000 7f65009e8640 1 -- 172.21.15.39:0/4287294636 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f64fc1a0760 con 0x7f64fc1083a0 2024-09-21T03:44:21.732 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:21.716+0000 7f64f37fe640 1 -- 172.21.15.39:0/4287294636 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64e400f340 con 0x7f64fc1083a0 2024-09-21T03:44:21.732 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:21.716+0000 7f65009e8640 1 -- 172.21.15.39:0/4287294636 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f64fc1a6f80 con 0x7f64fc1083a0 2024-09-21T03:44:21.732 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:21.718+0000 7f65009e8640 1 -- 172.21.15.39:0/4287294636 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f64fc102e70 con 0x7f64fc1083a0 2024-09-21T03:44:21.732 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:21.719+0000 7f64f37fe640 1 -- 172.21.15.39:0/4287294636 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 21) ==== 154529+0+0 (secure 0 0 0) 0x7f64e4018020 con 0x7f64fc1083a0 2024-09-21T03:44:21.732 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:21.720+0000 7f64f37fe640 1 --2- 172.21.15.39:0/4287294636 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f64e80b6590 0x7f64e80b8aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:21.732 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:21.720+0000 7f64fa59c640 1 --2- 172.21.15.39:0/4287294636 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f64e80b6590 0x7f64e80b8aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:21.732 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:21.720+0000 7f64f37fe640 1 -- 172.21.15.39:0/4287294636 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8194+0+0 (secure 0 0 0) 0x7f64e40e6dd0 con 0x7f64fc1083a0 2024-09-21T03:44:21.732 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:21.720+0000 7f64fa59c640 1 --2- 172.21.15.39:0/4287294636 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f64e80b6590 0x7f64e80b8aa0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f64ec031440 tx=0x7f64ec005d50 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:44:21.732 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:21.724+0000 7f64f37fe640 1 -- 172.21.15.39:0/4287294636 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f64e401f020 con 0x7f64fc1083a0 2024-09-21T03:44:21.954 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:21.951+0000 7f65009e8640 1 -- 172.21.15.39:0/4287294636 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"} v 0) -- 0x7f64fc1a7150 con 0x7f64fc1083a0 2024-09-21T03:44:22.362 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:22.359+0000 7f64f37fe640 1 -- 172.21.15.39:0/4287294636 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]=0 v18) ==== 105+0+0 (secure 0 0 0) 0x7f64e400d040 con 0x7f64fc1083a0 2024-09-21T03:44:22.364 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:22.361+0000 7f65009e8640 1 -- 172.21.15.39:0/4287294636 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f64e80b6590 msgr2=0x7f64e80b8aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:22.365 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:22.361+0000 7f65009e8640 1 --2- 172.21.15.39:0/4287294636 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f64e80b6590 0x7f64e80b8aa0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f64ec031440 tx=0x7f64ec005d50 comp rx=0 tx=0).stop 2024-09-21T03:44:22.365 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:22.361+0000 7f65009e8640 1 -- 172.21.15.39:0/4287294636 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f64fc1083a0 msgr2=0x7f64fc19ab60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:22.365 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:22.361+0000 7f65009e8640 1 --2- 172.21.15.39:0/4287294636 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f64fc1083a0 0x7f64fc19ab60 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f64e400b450 tx=0x7f64e400b910 comp rx=0 tx=0).stop 2024-09-21T03:44:22.365 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:22.362+0000 7f65009e8640 1 -- 172.21.15.39:0/4287294636 shutdown_connections 2024-09-21T03:44:22.365 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:22.362+0000 7f65009e8640 1 --2- 172.21.15.39:0/4287294636 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f64e80b6590 0x7f64e80b8aa0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:22.365 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:22.362+0000 7f65009e8640 1 --2- 172.21.15.39:0/4287294636 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f64fc108cf0 0x7f64fc19fe00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:22.365 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:22.362+0000 7f65009e8640 1 --2- 172.21.15.39:0/4287294636 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f64fc1083a0 0x7f64fc19ab60 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:22.365 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:22.362+0000 7f65009e8640 1 --2- 172.21.15.39:0/4287294636 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f64fc102a60 0x7f64fc19a5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:22.365 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:22.362+0000 7f65009e8640 1 -- 172.21.15.39:0/4287294636 >> 172.21.15.39:0/4287294636 conn(0x7f64fc079730 msgr2=0x7f64fc106780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:44:22.365 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:22.362+0000 7f65009e8640 1 -- 172.21.15.39:0/4287294636 shutdown_connections 2024-09-21T03:44:22.365 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:22.362+0000 7f65009e8640 1 -- 172.21.15.39:0/4287294636 wait complete. 2024-09-21T03:44:22.681 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:44:22 smithi150 ceph-mon[34289]: pgmap v273: 97 pgs: 97 active+clean; 579 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s wr, 5 op/s 2024-09-21T03:44:22.681 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:44:22 smithi150 ceph-mon[34289]: from='client.? 172.21.15.39:0/4287294636' entity='client.admin' cmd={"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"} : dispatch 2024-09-21T03:44:22.718 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:44:22 smithi161 ceph-mon[34533]: pgmap v273: 97 pgs: 97 active+clean; 579 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s wr, 5 op/s 2024-09-21T03:44:22.718 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:44:22 smithi161 ceph-mon[34533]: from='client.? 172.21.15.39:0/4287294636' entity='client.admin' cmd={"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"} : dispatch 2024-09-21T03:44:22.749 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:44:22 smithi039 ceph-mon[31211]: pgmap v273: 97 pgs: 97 active+clean; 579 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s wr, 5 op/s 2024-09-21T03:44:22.749 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:44:22 smithi039 ceph-mon[31211]: from='client.? 172.21.15.39:0/4287294636' entity='client.admin' cmd={"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"} : dispatch 2024-09-21T03:44:23.088 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 51116d5c-77ca-11ef-baf5-efdc52797490 -- ceph fs set cephfs allow_standby_replay true 2024-09-21T03:44:23.346 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/51116d5c-77ca-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:44:23.615 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:44:23 smithi039 ceph-mon[31211]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-09-21T03:44:23.615 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:44:23 smithi039 ceph-mon[31211]: from='client.? 172.21.15.39:0/4287294636' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]': finished 2024-09-21T03:44:23.616 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:44:23 smithi039 ceph-mon[31211]: daemon mds.i assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-09-21T03:44:23.616 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:44:23 smithi039 ceph-mon[31211]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-21T03:44:23.616 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:44:23 smithi039 ceph-mon[31211]: fsmap cephfs:2 {0=b=up:active,1=i=up:creating} 10 up:standby 2024-09-21T03:44:23.616 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:44:23 smithi039 ceph-mon[31211]: daemon mds.i is now active in filesystem cephfs as rank 1 2024-09-21T03:44:23.718 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:44:23 smithi161 ceph-mon[34533]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-09-21T03:44:23.718 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:44:23 smithi161 ceph-mon[34533]: from='client.? 172.21.15.39:0/4287294636' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]': finished 2024-09-21T03:44:23.718 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:44:23 smithi161 ceph-mon[34533]: daemon mds.i assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-09-21T03:44:23.718 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:44:23 smithi161 ceph-mon[34533]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-21T03:44:23.718 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:44:23 smithi161 ceph-mon[34533]: fsmap cephfs:2 {0=b=up:active,1=i=up:creating} 10 up:standby 2024-09-21T03:44:23.718 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:44:23 smithi161 ceph-mon[34533]: daemon mds.i is now active in filesystem cephfs as rank 1 2024-09-21T03:44:23.724 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:44:23 smithi150 ceph-mon[34289]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-09-21T03:44:23.725 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:44:23 smithi150 ceph-mon[34289]: from='client.? 172.21.15.39:0/4287294636' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]': finished 2024-09-21T03:44:23.725 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:44:23 smithi150 ceph-mon[34289]: daemon mds.i assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-09-21T03:44:23.725 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:44:23 smithi150 ceph-mon[34289]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-21T03:44:23.725 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:44:23 smithi150 ceph-mon[34289]: fsmap cephfs:2 {0=b=up:active,1=i=up:creating} 10 up:standby 2024-09-21T03:44:23.725 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:44:23 smithi150 ceph-mon[34289]: daemon mds.i is now active in filesystem cephfs as rank 1 2024-09-21T03:44:23.825 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:23.822+0000 7f15bf104640 1 -- 172.21.15.39:0/2418252227 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f15b81094a0 msgr2=0x7f15b81098b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:23.825 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:23.822+0000 7f15bf104640 1 --2- 172.21.15.39:0/2418252227 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f15b81094a0 0x7f15b81098b0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f15a8009ca0 tx=0x7f15a802a8c0 comp rx=0 tx=0).stop 2024-09-21T03:44:23.825 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:23.822+0000 7f15bf104640 1 -- 172.21.15.39:0/2418252227 shutdown_connections 2024-09-21T03:44:23.826 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:23.822+0000 7f15bf104640 1 --2- 172.21.15.39:0/2418252227 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f15b8109df0 0x7f15b8110620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:23.826 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:23.822+0000 7f15bf104640 1 --2- 172.21.15.39:0/2418252227 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f15b81094a0 0x7f15b81098b0 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:23.826 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:23.822+0000 7f15bf104640 1 --2- 172.21.15.39:0/2418252227 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f15b8103b60 0x7f15b8103f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:23.826 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:23.822+0000 7f15bf104640 1 -- 172.21.15.39:0/2418252227 >> 172.21.15.39:0/2418252227 conn(0x7f15b807b850 msgr2=0x7f15b807bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:44:23.826 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:23.822+0000 7f15bf104640 1 -- 172.21.15.39:0/2418252227 shutdown_connections 2024-09-21T03:44:23.826 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:23.822+0000 7f15bf104640 1 -- 172.21.15.39:0/2418252227 wait complete. 2024-09-21T03:44:23.828 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:23.823+0000 7f15bf104640 1 Processor -- start 2024-09-21T03:44:23.828 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:23.823+0000 7f15bf104640 1 -- start start 2024-09-21T03:44:23.828 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:23.823+0000 7f15bf104640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f15b8103b60 0x7f15b819a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:23.828 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:23.823+0000 7f15bf104640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f15b81094a0 0x7f15b819aa20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:23.828 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:23.823+0000 7f15bf104640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f15b8109df0 0x7f15b819fcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:23.828 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:23.823+0000 7f15bf104640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f15b8113ab0 con 0x7f15b81094a0 2024-09-21T03:44:23.828 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:23.823+0000 7f15bf104640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f15b8113930 con 0x7f15b8109df0 2024-09-21T03:44:23.828 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:23.823+0000 7f15bf104640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f15b8113c30 con 0x7f15b8103b60 2024-09-21T03:44:23.828 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:23.824+0000 7f15b7fff640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f15b81094a0 0x7f15b819aa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:23.828 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:23.824+0000 7f15b7fff640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f15b81094a0 0x7f15b819aa20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:57614/0 (socket says 172.21.15.39:57614) 2024-09-21T03:44:23.829 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:23.824+0000 7f15b7fff640 1 -- 172.21.15.39:0/1283249687 learned_addr learned my addr 172.21.15.39:0/1283249687 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:44:23.829 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:23.824+0000 7f15bcea0640 1 --2- 172.21.15.39:0/1283249687 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f15b8103b60 0x7f15b819a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:23.829 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:23.824+0000 7f15bd6a1640 1 --2- 172.21.15.39:0/1283249687 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f15b8109df0 0x7f15b819fcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:23.829 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:23.824+0000 7f15b7fff640 1 -- 172.21.15.39:0/1283249687 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f15b8103b60 msgr2=0x7f15b819a480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:23.829 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:23.824+0000 7f15b7fff640 1 --2- 172.21.15.39:0/1283249687 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f15b8103b60 0x7f15b819a480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:23.829 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:23.824+0000 7f15b7fff640 1 -- 172.21.15.39:0/1283249687 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f15b8109df0 msgr2=0x7f15b819fcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:23.829 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:23.824+0000 7f15b7fff640 1 --2- 172.21.15.39:0/1283249687 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f15b8109df0 0x7f15b819fcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:23.829 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:23.824+0000 7f15b7fff640 1 -- 172.21.15.39:0/1283249687 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f15b81a03f0 con 0x7f15b81094a0 2024-09-21T03:44:23.829 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:23.824+0000 7f15bcea0640 1 --2- 172.21.15.39:0/1283249687 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f15b8103b60 0x7f15b819a480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:44:23.829 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:23.824+0000 7f15b7fff640 1 --2- 172.21.15.39:0/1283249687 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f15b81094a0 0x7f15b819aa20 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f15a80319f0 tx=0x7f15a8031a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:44:23.829 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:23.824+0000 7f15b5ffb640 1 -- 172.21.15.39:0/1283249687 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15a8003bf0 con 0x7f15b81094a0 2024-09-21T03:44:23.830 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:23.824+0000 7f15bf104640 1 -- 172.21.15.39:0/1283249687 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f15b81a0680 con 0x7f15b81094a0 2024-09-21T03:44:23.830 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:23.824+0000 7f15b5ffb640 1 -- 172.21.15.39:0/1283249687 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f15a8003d90 con 0x7f15b81094a0 2024-09-21T03:44:23.830 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:23.824+0000 7f15b5ffb640 1 -- 172.21.15.39:0/1283249687 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15a802fb70 con 0x7f15b81094a0 2024-09-21T03:44:23.830 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:23.825+0000 7f15bf104640 1 -- 172.21.15.39:0/1283249687 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f15b81a6e40 con 0x7f15b81094a0 2024-09-21T03:44:23.831 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:23.826+0000 7f15bf104640 1 -- 172.21.15.39:0/1283249687 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f15b8104de0 con 0x7f15b81094a0 2024-09-21T03:44:23.831 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:23.827+0000 7f15b5ffb640 1 -- 172.21.15.39:0/1283249687 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 21) ==== 154529+0+0 (secure 0 0 0) 0x7f15a802fd10 con 0x7f15b81094a0 2024-09-21T03:44:23.831 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:23.828+0000 7f15b5ffb640 1 --2- 172.21.15.39:0/1283249687 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f15a40b6560 0x7f15a40b8a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:23.831 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:23.828+0000 7f15bcea0640 1 --2- 172.21.15.39:0/1283249687 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f15a40b6560 0x7f15a40b8a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:23.831 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:23.828+0000 7f15b5ffb640 1 -- 172.21.15.39:0/1283249687 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8194+0+0 (secure 0 0 0) 0x7f15a8108760 con 0x7f15b81094a0 2024-09-21T03:44:23.831 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:23.829+0000 7f15bcea0640 1 --2- 172.21.15.39:0/1283249687 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f15a40b6560 0x7f15a40b8a70 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f15a00058c0 tx=0x7f15a0005ec0 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:44:23.835 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:23.832+0000 7f15b5ffb640 1 -- 172.21.15.39:0/1283249687 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f15a80cf010 con 0x7f15b81094a0 2024-09-21T03:44:24.064 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:24.061+0000 7f15bf104640 1 -- 172.21.15.39:0/1283249687 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"} v 0) -- 0x7f15b810bc40 con 0x7f15b81094a0 2024-09-21T03:44:24.390 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:24.387+0000 7f15b5ffb640 1 -- 172.21.15.39:0/1283249687 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]=0 v22) ==== 121+0+0 (secure 0 0 0) 0x7f15a80098b0 con 0x7f15b81094a0 2024-09-21T03:44:24.393 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:24.391+0000 7f15bf104640 1 -- 172.21.15.39:0/1283249687 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f15a40b6560 msgr2=0x7f15a40b8a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:24.394 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:24.391+0000 7f15bf104640 1 --2- 172.21.15.39:0/1283249687 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f15a40b6560 0x7f15a40b8a70 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f15a00058c0 tx=0x7f15a0005ec0 comp rx=0 tx=0).stop 2024-09-21T03:44:24.394 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:24.391+0000 7f15bf104640 1 -- 172.21.15.39:0/1283249687 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f15b81094a0 msgr2=0x7f15b819aa20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:24.394 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:24.391+0000 7f15bf104640 1 --2- 172.21.15.39:0/1283249687 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f15b81094a0 0x7f15b819aa20 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f15a80319f0 tx=0x7f15a8031a20 comp rx=0 tx=0).stop 2024-09-21T03:44:24.394 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:24.391+0000 7f15bf104640 1 -- 172.21.15.39:0/1283249687 shutdown_connections 2024-09-21T03:44:24.394 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:24.391+0000 7f15bf104640 1 --2- 172.21.15.39:0/1283249687 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f15a40b6560 0x7f15a40b8a70 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:24.394 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:24.391+0000 7f15bf104640 1 --2- 172.21.15.39:0/1283249687 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f15b8109df0 0x7f15b819fcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:24.394 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:24.391+0000 7f15bf104640 1 --2- 172.21.15.39:0/1283249687 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f15b81094a0 0x7f15b819aa20 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:24.394 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:24.391+0000 7f15bf104640 1 --2- 172.21.15.39:0/1283249687 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f15b8103b60 0x7f15b819a480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:24.394 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:24.391+0000 7f15bf104640 1 -- 172.21.15.39:0/1283249687 >> 172.21.15.39:0/1283249687 conn(0x7f15b807b850 msgr2=0x7f15b8107590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:44:24.394 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:24.391+0000 7f15bf104640 1 -- 172.21.15.39:0/1283249687 shutdown_connections 2024-09-21T03:44:24.394 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:24.391+0000 7f15bf104640 1 -- 172.21.15.39:0/1283249687 wait complete. 2024-09-21T03:44:24.701 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:44:24 smithi039 ceph-mon[31211]: mds.? [v2:172.21.15.161:6838/2190107270,v1:172.21.15.161:6839/2190107270] up:active 2024-09-21T03:44:24.701 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:44:24 smithi039 ceph-mon[31211]: daemon mds.h assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-09-21T03:44:24.701 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:44:24 smithi039 ceph-mon[31211]: fsmap cephfs:2 {0=b=up:active,1=i=up:active} 10 up:standby 2024-09-21T03:44:24.701 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:44:24 smithi039 ceph-mon[31211]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:creating} 9 up:standby 2024-09-21T03:44:24.701 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:44:24 smithi039 ceph-mon[31211]: daemon mds.h is now active in filesystem cephfs as rank 2 2024-09-21T03:44:24.701 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:44:24 smithi039 ceph-mon[31211]: pgmap v274: 97 pgs: 97 active+clean; 579 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s wr, 6 op/s 2024-09-21T03:44:24.701 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:44:24 smithi039 ceph-mon[31211]: from='client.? 172.21.15.39:0/1283249687' entity='client.admin' cmd={"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"} : dispatch 2024-09-21T03:44:24.717 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:44:24 smithi161 ceph-mon[34533]: mds.? [v2:172.21.15.161:6838/2190107270,v1:172.21.15.161:6839/2190107270] up:active 2024-09-21T03:44:24.718 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:44:24 smithi161 ceph-mon[34533]: daemon mds.h assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-09-21T03:44:24.718 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:44:24 smithi161 ceph-mon[34533]: fsmap cephfs:2 {0=b=up:active,1=i=up:active} 10 up:standby 2024-09-21T03:44:24.718 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:44:24 smithi161 ceph-mon[34533]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:creating} 9 up:standby 2024-09-21T03:44:24.718 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:44:24 smithi161 ceph-mon[34533]: daemon mds.h is now active in filesystem cephfs as rank 2 2024-09-21T03:44:24.718 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:44:24 smithi161 ceph-mon[34533]: pgmap v274: 97 pgs: 97 active+clean; 579 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s wr, 6 op/s 2024-09-21T03:44:24.718 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:44:24 smithi161 ceph-mon[34533]: from='client.? 172.21.15.39:0/1283249687' entity='client.admin' cmd={"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"} : dispatch 2024-09-21T03:44:24.724 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:44:24 smithi150 ceph-mon[34289]: mds.? [v2:172.21.15.161:6838/2190107270,v1:172.21.15.161:6839/2190107270] up:active 2024-09-21T03:44:24.724 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:44:24 smithi150 ceph-mon[34289]: daemon mds.h assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-09-21T03:44:24.724 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:44:24 smithi150 ceph-mon[34289]: fsmap cephfs:2 {0=b=up:active,1=i=up:active} 10 up:standby 2024-09-21T03:44:24.725 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:44:24 smithi150 ceph-mon[34289]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:creating} 9 up:standby 2024-09-21T03:44:24.725 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:44:24 smithi150 ceph-mon[34289]: daemon mds.h is now active in filesystem cephfs as rank 2 2024-09-21T03:44:24.725 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:44:24 smithi150 ceph-mon[34289]: pgmap v274: 97 pgs: 97 active+clean; 579 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s wr, 6 op/s 2024-09-21T03:44:24.725 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:44:24 smithi150 ceph-mon[34289]: from='client.? 172.21.15.39:0/1283249687' entity='client.admin' cmd={"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"} : dispatch 2024-09-21T03:44:25.013 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 51116d5c-77ca-11ef-baf5-efdc52797490 -- ceph fs set cephfs session_timeout 300 2024-09-21T03:44:25.272 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/51116d5c-77ca-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:44:25.680 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:44:25 smithi039 ceph-mon[31211]: from='client.? 172.21.15.39:0/1283249687' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-09-21T03:44:25.680 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:44:25 smithi039 ceph-mon[31211]: mds.? [v2:172.21.15.150:6836/4017043144,v1:172.21.15.150:6837/4017043144] up:active 2024-09-21T03:44:25.680 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:44:25 smithi039 ceph-mon[31211]: daemon mds.c assigned to filesystem cephfs as rank 3 (now has 4 ranks) 2024-09-21T03:44:25.680 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:44:25 smithi039 ceph-mon[31211]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 9 up:standby 2024-09-21T03:44:25.680 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:44:25 smithi039 ceph-mon[31211]: fsmap cephfs:4 {0=b=up:active,1=i=up:active,2=h=up:active,3=c=up:creating} 8 up:standby 2024-09-21T03:44:25.680 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:44:25 smithi039 ceph-mon[31211]: daemon mds.c is now active in filesystem cephfs as rank 3 2024-09-21T03:44:25.680 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:44:25 smithi039 ceph-mon[31211]: mds.? [v2:172.21.15.161:6836/3451544597,v1:172.21.15.161:6837/3451544597] up:active 2024-09-21T03:44:25.680 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:44:25 smithi039 ceph-mon[31211]: daemon mds.a assigned to filesystem cephfs as rank 4 (now has 5 ranks) 2024-09-21T03:44:25.681 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:44:25 smithi039 ceph-mon[31211]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-09-21T03:44:25.681 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:44:25 smithi039 ceph-mon[31211]: Cluster is now healthy 2024-09-21T03:44:25.681 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:44:25 smithi039 ceph-mon[31211]: fsmap cephfs:4 {0=b=up:active,1=i=up:active,2=h=up:active,3=c=up:active} 4 up:standby-replay 4 up:standby 2024-09-21T03:44:25.681 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:44:25 smithi039 ceph-mon[31211]: fsmap cephfs:5 4 up:standby-replay 1 up:creating 3 up:standby 4 up:active 2024-09-21T03:44:25.681 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:44:25 smithi039 ceph-mon[31211]: daemon mds.a is now active in filesystem cephfs as rank 4 2024-09-21T03:44:25.718 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:44:25 smithi161 ceph-mon[34533]: from='client.? 172.21.15.39:0/1283249687' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-09-21T03:44:25.718 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:44:25 smithi161 ceph-mon[34533]: mds.? [v2:172.21.15.150:6836/4017043144,v1:172.21.15.150:6837/4017043144] up:active 2024-09-21T03:44:25.718 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:44:25 smithi161 ceph-mon[34533]: daemon mds.c assigned to filesystem cephfs as rank 3 (now has 4 ranks) 2024-09-21T03:44:25.718 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:44:25 smithi161 ceph-mon[34533]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 9 up:standby 2024-09-21T03:44:25.718 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:44:25 smithi161 ceph-mon[34533]: fsmap cephfs:4 {0=b=up:active,1=i=up:active,2=h=up:active,3=c=up:creating} 8 up:standby 2024-09-21T03:44:25.718 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:44:25 smithi161 ceph-mon[34533]: daemon mds.c is now active in filesystem cephfs as rank 3 2024-09-21T03:44:25.718 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:44:25 smithi161 ceph-mon[34533]: mds.? [v2:172.21.15.161:6836/3451544597,v1:172.21.15.161:6837/3451544597] up:active 2024-09-21T03:44:25.718 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:44:25 smithi161 ceph-mon[34533]: daemon mds.a assigned to filesystem cephfs as rank 4 (now has 5 ranks) 2024-09-21T03:44:25.718 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:44:25 smithi161 ceph-mon[34533]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-09-21T03:44:25.718 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:44:25 smithi161 ceph-mon[34533]: Cluster is now healthy 2024-09-21T03:44:25.718 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:44:25 smithi161 ceph-mon[34533]: fsmap cephfs:4 {0=b=up:active,1=i=up:active,2=h=up:active,3=c=up:active} 4 up:standby-replay 4 up:standby 2024-09-21T03:44:25.718 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:44:25 smithi161 ceph-mon[34533]: fsmap cephfs:5 4 up:standby-replay 1 up:creating 3 up:standby 4 up:active 2024-09-21T03:44:25.719 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:44:25 smithi161 ceph-mon[34533]: daemon mds.a is now active in filesystem cephfs as rank 4 2024-09-21T03:44:25.724 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:44:25 smithi150 ceph-mon[34289]: from='client.? 172.21.15.39:0/1283249687' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-09-21T03:44:25.725 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:44:25 smithi150 ceph-mon[34289]: mds.? [v2:172.21.15.150:6836/4017043144,v1:172.21.15.150:6837/4017043144] up:active 2024-09-21T03:44:25.725 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:44:25 smithi150 ceph-mon[34289]: daemon mds.c assigned to filesystem cephfs as rank 3 (now has 4 ranks) 2024-09-21T03:44:25.725 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:44:25 smithi150 ceph-mon[34289]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 9 up:standby 2024-09-21T03:44:25.725 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:44:25 smithi150 ceph-mon[34289]: fsmap cephfs:4 {0=b=up:active,1=i=up:active,2=h=up:active,3=c=up:creating} 8 up:standby 2024-09-21T03:44:25.725 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:44:25 smithi150 ceph-mon[34289]: daemon mds.c is now active in filesystem cephfs as rank 3 2024-09-21T03:44:25.725 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:44:25 smithi150 ceph-mon[34289]: mds.? [v2:172.21.15.161:6836/3451544597,v1:172.21.15.161:6837/3451544597] up:active 2024-09-21T03:44:25.725 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:44:25 smithi150 ceph-mon[34289]: daemon mds.a assigned to filesystem cephfs as rank 4 (now has 5 ranks) 2024-09-21T03:44:25.725 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:44:25 smithi150 ceph-mon[34289]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-09-21T03:44:25.725 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:44:25 smithi150 ceph-mon[34289]: Cluster is now healthy 2024-09-21T03:44:25.725 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:44:25 smithi150 ceph-mon[34289]: fsmap cephfs:4 {0=b=up:active,1=i=up:active,2=h=up:active,3=c=up:active} 4 up:standby-replay 4 up:standby 2024-09-21T03:44:25.725 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:44:25 smithi150 ceph-mon[34289]: fsmap cephfs:5 4 up:standby-replay 1 up:creating 3 up:standby 4 up:active 2024-09-21T03:44:25.725 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:44:25 smithi150 ceph-mon[34289]: daemon mds.a is now active in filesystem cephfs as rank 4 2024-09-21T03:44:25.788 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:25.775+0000 7f6173169640 1 -- 172.21.15.39:0/3565947618 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f616c102ae0 msgr2=0x7f616c102ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:25.788 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:25.775+0000 7f6173169640 1 --2- 172.21.15.39:0/3565947618 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f616c102ae0 0x7f616c102ef0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f6160009900 tx=0x7f616002a410 comp rx=0 tx=0).stop 2024-09-21T03:44:25.788 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:25.775+0000 7f6173169640 1 -- 172.21.15.39:0/3565947618 shutdown_connections 2024-09-21T03:44:25.789 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:25.775+0000 7f6173169640 1 --2- 172.21.15.39:0/3565947618 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f616c108d20 0x7f616c10f550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:25.789 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:25.775+0000 7f6173169640 1 --2- 172.21.15.39:0/3565947618 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f616c1083d0 0x7f616c1087e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:25.789 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:25.775+0000 7f6173169640 1 --2- 172.21.15.39:0/3565947618 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f616c102ae0 0x7f616c102ef0 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:25.789 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:25.775+0000 7f6173169640 1 -- 172.21.15.39:0/3565947618 >> 172.21.15.39:0/3565947618 conn(0x7f616c079760 msgr2=0x7f616c079b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:44:25.789 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:25.775+0000 7f6173169640 1 -- 172.21.15.39:0/3565947618 shutdown_connections 2024-09-21T03:44:25.789 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:25.775+0000 7f6173169640 1 -- 172.21.15.39:0/3565947618 wait complete. 2024-09-21T03:44:25.789 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:25.776+0000 7f6173169640 1 Processor -- start 2024-09-21T03:44:25.789 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:25.776+0000 7f6173169640 1 -- start start 2024-09-21T03:44:25.789 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:25.776+0000 7f6173169640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f616c102ae0 0x7f616c199390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:25.789 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:25.776+0000 7f6173169640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f616c1083d0 0x7f616c199930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:25.789 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:25.776+0000 7f6173169640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f616c108d20 0x7f616c19ebd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:25.789 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:25.776+0000 7f6173169640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f616c1129e0 con 0x7f616c1083d0 2024-09-21T03:44:25.789 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:25.776+0000 7f6173169640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f616c112860 con 0x7f616c108d20 2024-09-21T03:44:25.790 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:25.776+0000 7f6173169640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f616c112b60 con 0x7f616c102ae0 2024-09-21T03:44:25.790 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:25.776+0000 7f616bfff640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f616c1083d0 0x7f616c199930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:25.790 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:25.776+0000 7f616bfff640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f616c1083d0 0x7f616c199930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:57640/0 (socket says 172.21.15.39:57640) 2024-09-21T03:44:25.790 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:25.776+0000 7f616bfff640 1 -- 172.21.15.39:0/1160208137 learned_addr learned my addr 172.21.15.39:0/1160208137 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:44:25.790 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:25.777+0000 7f6170f05640 1 --2- 172.21.15.39:0/1160208137 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f616c102ae0 0x7f616c199390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:25.790 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:25.777+0000 7f616bfff640 1 -- 172.21.15.39:0/1160208137 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f616c102ae0 msgr2=0x7f616c199390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:25.790 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:25.777+0000 7f6171706640 1 --2- 172.21.15.39:0/1160208137 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f616c108d20 0x7f616c19ebd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:25.790 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:25.777+0000 7f616bfff640 1 --2- 172.21.15.39:0/1160208137 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f616c102ae0 0x7f616c199390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:25.790 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:25.777+0000 7f616bfff640 1 -- 172.21.15.39:0/1160208137 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f616c108d20 msgr2=0x7f616c19ebd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:25.790 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:25.777+0000 7f616bfff640 1 --2- 172.21.15.39:0/1160208137 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f616c108d20 0x7f616c19ebd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:25.790 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:25.777+0000 7f616bfff640 1 -- 172.21.15.39:0/1160208137 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f616c19f300 con 0x7f616c1083d0 2024-09-21T03:44:25.790 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:25.777+0000 7f616bfff640 1 --2- 172.21.15.39:0/1160208137 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f616c1083d0 0x7f616c199930 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f615800e9c0 tx=0x7f615800ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:44:25.790 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:25.777+0000 7f6169ffb640 1 -- 172.21.15.39:0/1160208137 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f615800cd80 con 0x7f616c1083d0 2024-09-21T03:44:25.791 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:25.777+0000 7f6169ffb640 1 -- 172.21.15.39:0/1160208137 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f61580054d0 con 0x7f616c1083d0 2024-09-21T03:44:25.791 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:25.777+0000 7f6169ffb640 1 -- 172.21.15.39:0/1160208137 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6158010430 con 0x7f616c1083d0 2024-09-21T03:44:25.791 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:25.777+0000 7f6173169640 1 -- 172.21.15.39:0/1160208137 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f616c19f5f0 con 0x7f616c1083d0 2024-09-21T03:44:25.791 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:25.777+0000 7f6173169640 1 -- 172.21.15.39:0/1160208137 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f616c1a5d50 con 0x7f616c1083d0 2024-09-21T03:44:25.791 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:25.778+0000 7f6173169640 1 -- 172.21.15.39:0/1160208137 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f616c103d10 con 0x7f616c1083d0 2024-09-21T03:44:25.791 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:25.780+0000 7f6169ffb640 1 -- 172.21.15.39:0/1160208137 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 21) ==== 154529+0+0 (secure 0 0 0) 0x7f6158004fd0 con 0x7f616c1083d0 2024-09-21T03:44:25.791 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:25.781+0000 7f6169ffb640 1 --2- 172.21.15.39:0/1160208137 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f615c0b65e0 0x7f615c0b8af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:25.791 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:25.781+0000 7f6170f05640 1 --2- 172.21.15.39:0/1160208137 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f615c0b65e0 0x7f615c0b8af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:25.791 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:25.781+0000 7f6169ffb640 1 -- 172.21.15.39:0/1160208137 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8194+0+0 (secure 0 0 0) 0x7f61580e7880 con 0x7f616c1083d0 2024-09-21T03:44:25.791 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:25.781+0000 7f6170f05640 1 --2- 172.21.15.39:0/1160208137 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f615c0b65e0 0x7f615c0b8af0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f6160000c00 tx=0x7f6160031400 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:44:25.791 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:25.785+0000 7f6169ffb640 1 -- 172.21.15.39:0/1160208137 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f61580ae130 con 0x7f616c1083d0 2024-09-21T03:44:26.017 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:26.015+0000 7f6173169640 1 -- 172.21.15.39:0/1160208137 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"} v 0) -- 0x7f616c10a560 con 0x7f616c1083d0 2024-09-21T03:44:26.394 INFO:journalctl@ceph.mds.b.smithi150.stdout:Sep 21 03:44:25 smithi150 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-mds-b[63521]: 2024-09-21T03:44:25.997+0000 7efca1db9640 -1 mds.pinger is_rank_lagging: rank=1 was never sent ping request. 2024-09-21T03:44:26.394 INFO:journalctl@ceph.mds.b.smithi150.stdout:Sep 21 03:44:25 smithi150 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-mds-b[63521]: 2024-09-21T03:44:25.997+0000 7efca1db9640 -1 mds.pinger is_rank_lagging: rank=2 was never sent ping request. 2024-09-21T03:44:26.394 INFO:journalctl@ceph.mds.b.smithi150.stdout:Sep 21 03:44:25 smithi150 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-mds-b[63521]: 2024-09-21T03:44:25.997+0000 7efca1db9640 -1 mds.pinger is_rank_lagging: rank=3 was never sent ping request. 2024-09-21T03:44:26.394 INFO:journalctl@ceph.mds.b.smithi150.stdout:Sep 21 03:44:25 smithi150 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-mds-b[63521]: 2024-09-21T03:44:25.997+0000 7efca1db9640 -1 mds.pinger is_rank_lagging: rank=4 was never sent ping request. 2024-09-21T03:44:26.706 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:26.704+0000 7f6169ffb640 1 -- 172.21.15.39:0/1160208137 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]=0 v28) ==== 115+0+0 (secure 0 0 0) 0x7f61580b2510 con 0x7f616c1083d0 2024-09-21T03:44:26.707 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:44:26 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.5", "id": [8, 11]} : dispatch 2024-09-21T03:44:26.707 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:44:26 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.8", "id": [2, 11]} : dispatch 2024-09-21T03:44:26.707 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:44:26 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.c", "id": [2, 1]} : dispatch 2024-09-21T03:44:26.707 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:44:26 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1f", "id": [8, 4]} : dispatch 2024-09-21T03:44:26.707 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:44:26 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2", "id": [3, 9]} : dispatch 2024-09-21T03:44:26.707 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:44:26 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.14", "id": [4, 6]} : dispatch 2024-09-21T03:44:26.707 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:44:26 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1a", "id": [5, 7]} : dispatch 2024-09-21T03:44:26.707 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:44:26 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1d", "id": [4, 8]} : dispatch 2024-09-21T03:44:26.707 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:44:26 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2c", "id": [4, 9]} : dispatch 2024-09-21T03:44:26.708 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:44:26 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.38", "id": [3, 9]} : dispatch 2024-09-21T03:44:26.708 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:44:26 smithi039 ceph-mon[31211]: pgmap v275: 97 pgs: 97 active+clean; 582 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 3.3 KiB/s wr, 14 op/s 2024-09-21T03:44:26.708 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:44:26 smithi039 ceph-mon[31211]: mds.? [v2:172.21.15.39:6840/4247190456,v1:172.21.15.39:6841/4247190456] up:active 2024-09-21T03:44:26.708 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:44:26 smithi039 ceph-mon[31211]: fsmap cephfs:5 4 up:standby-replay 3 up:standby 5 up:active 2024-09-21T03:44:26.708 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:44:26 smithi039 ceph-mon[31211]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-21T03:44:26.708 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:44:26 smithi039 ceph-mon[31211]: from='client.? 172.21.15.39:0/1160208137' entity='client.admin' cmd={"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"} : dispatch 2024-09-21T03:44:26.710 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:26.707+0000 7f6173169640 1 -- 172.21.15.39:0/1160208137 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f615c0b65e0 msgr2=0x7f615c0b8af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:26.710 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:26.707+0000 7f6173169640 1 --2- 172.21.15.39:0/1160208137 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f615c0b65e0 0x7f615c0b8af0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f6160000c00 tx=0x7f6160031400 comp rx=0 tx=0).stop 2024-09-21T03:44:26.710 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:26.707+0000 7f6173169640 1 -- 172.21.15.39:0/1160208137 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f616c1083d0 msgr2=0x7f616c199930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:26.710 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:26.707+0000 7f6173169640 1 --2- 172.21.15.39:0/1160208137 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f616c1083d0 0x7f616c199930 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f615800e9c0 tx=0x7f615800ee80 comp rx=0 tx=0).stop 2024-09-21T03:44:26.710 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:26.707+0000 7f6173169640 1 -- 172.21.15.39:0/1160208137 shutdown_connections 2024-09-21T03:44:26.710 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:26.707+0000 7f6173169640 1 --2- 172.21.15.39:0/1160208137 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f615c0b65e0 0x7f615c0b8af0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:26.710 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:26.707+0000 7f6173169640 1 --2- 172.21.15.39:0/1160208137 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f616c108d20 0x7f616c19ebd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:26.710 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:26.707+0000 7f6173169640 1 --2- 172.21.15.39:0/1160208137 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f616c1083d0 0x7f616c199930 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:26.710 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:26.707+0000 7f6173169640 1 --2- 172.21.15.39:0/1160208137 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f616c102ae0 0x7f616c199390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:26.710 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:26.707+0000 7f6173169640 1 -- 172.21.15.39:0/1160208137 >> 172.21.15.39:0/1160208137 conn(0x7f616c079760 msgr2=0x7f616c1014c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:44:26.711 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:26.707+0000 7f6173169640 1 -- 172.21.15.39:0/1160208137 shutdown_connections 2024-09-21T03:44:26.711 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:26.707+0000 7f6173169640 1 -- 172.21.15.39:0/1160208137 wait complete. 2024-09-21T03:44:26.718 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:44:26 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.5", "id": [8, 11]} : dispatch 2024-09-21T03:44:26.718 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:44:26 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.8", "id": [2, 11]} : dispatch 2024-09-21T03:44:26.718 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:44:26 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.c", "id": [2, 1]} : dispatch 2024-09-21T03:44:26.718 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:44:26 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1f", "id": [8, 4]} : dispatch 2024-09-21T03:44:26.718 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:44:26 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2", "id": [3, 9]} : dispatch 2024-09-21T03:44:26.718 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:44:26 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.14", "id": [4, 6]} : dispatch 2024-09-21T03:44:26.718 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:44:26 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1a", "id": [5, 7]} : dispatch 2024-09-21T03:44:26.718 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:44:26 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1d", "id": [4, 8]} : dispatch 2024-09-21T03:44:26.718 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:44:26 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2c", "id": [4, 9]} : dispatch 2024-09-21T03:44:26.718 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:44:26 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.38", "id": [3, 9]} : dispatch 2024-09-21T03:44:26.718 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:44:26 smithi161 ceph-mon[34533]: pgmap v275: 97 pgs: 97 active+clean; 582 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 3.3 KiB/s wr, 14 op/s 2024-09-21T03:44:26.719 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:44:26 smithi161 ceph-mon[34533]: mds.? [v2:172.21.15.39:6840/4247190456,v1:172.21.15.39:6841/4247190456] up:active 2024-09-21T03:44:26.719 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:44:26 smithi161 ceph-mon[34533]: fsmap cephfs:5 4 up:standby-replay 3 up:standby 5 up:active 2024-09-21T03:44:26.719 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:44:26 smithi161 ceph-mon[34533]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-21T03:44:26.719 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:44:26 smithi161 ceph-mon[34533]: from='client.? 172.21.15.39:0/1160208137' entity='client.admin' cmd={"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"} : dispatch 2024-09-21T03:44:26.725 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:44:26 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.5", "id": [8, 11]} : dispatch 2024-09-21T03:44:26.725 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:44:26 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.8", "id": [2, 11]} : dispatch 2024-09-21T03:44:26.725 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:44:26 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.c", "id": [2, 1]} : dispatch 2024-09-21T03:44:26.725 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:44:26 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1f", "id": [8, 4]} : dispatch 2024-09-21T03:44:26.725 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:44:26 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2", "id": [3, 9]} : dispatch 2024-09-21T03:44:26.725 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:44:26 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.14", "id": [4, 6]} : dispatch 2024-09-21T03:44:26.725 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:44:26 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1a", "id": [5, 7]} : dispatch 2024-09-21T03:44:26.725 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:44:26 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1d", "id": [4, 8]} : dispatch 2024-09-21T03:44:26.725 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:44:26 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2c", "id": [4, 9]} : dispatch 2024-09-21T03:44:26.725 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:44:26 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.38", "id": [3, 9]} : dispatch 2024-09-21T03:44:26.725 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:44:26 smithi150 ceph-mon[34289]: pgmap v275: 97 pgs: 97 active+clean; 582 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 3.3 KiB/s wr, 14 op/s 2024-09-21T03:44:26.726 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:44:26 smithi150 ceph-mon[34289]: mds.? [v2:172.21.15.39:6840/4247190456,v1:172.21.15.39:6841/4247190456] up:active 2024-09-21T03:44:26.726 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:44:26 smithi150 ceph-mon[34289]: fsmap cephfs:5 4 up:standby-replay 3 up:standby 5 up:active 2024-09-21T03:44:26.726 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:44:26 smithi150 ceph-mon[34289]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-21T03:44:26.726 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:44:26 smithi150 ceph-mon[34289]: from='client.? 172.21.15.39:0/1160208137' entity='client.admin' cmd={"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"} : dispatch 2024-09-21T03:44:27.542 DEBUG:tasks.cephfs.filesystem:Creating {'create': 2, 'subvol_options': '--namespace-isolated --size 25000000000'} subvols for filesystem 'cephfs' 2024-09-21T03:44:27.543 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 51116d5c-77ca-11ef-baf5-efdc52797490 -- ceph fs subvolumegroup create cephfs qa 2024-09-21T03:44:27.718 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:44:27 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.5", "id": [8, 11]}]': finished 2024-09-21T03:44:27.718 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:44:27 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.8", "id": [2, 11]}]': finished 2024-09-21T03:44:27.718 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:44:27 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.c", "id": [2, 1]}]': finished 2024-09-21T03:44:27.718 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:44:27 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1f", "id": [8, 4]}]': finished 2024-09-21T03:44:27.718 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:44:27 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2", "id": [3, 9]}]': finished 2024-09-21T03:44:27.718 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:44:27 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.14", "id": [4, 6]}]': finished 2024-09-21T03:44:27.718 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:44:27 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1a", "id": [5, 7]}]': finished 2024-09-21T03:44:27.718 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:44:27 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1d", "id": [4, 8]}]': finished 2024-09-21T03:44:27.718 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:44:27 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2c", "id": [4, 9]}]': finished 2024-09-21T03:44:27.719 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:44:27 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.38", "id": [3, 9]}]': finished 2024-09-21T03:44:27.719 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:44:27 smithi161 ceph-mon[34533]: osdmap e78: 12 total, 12 up, 12 in 2024-09-21T03:44:27.719 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:44:27 smithi161 ceph-mon[34533]: from='client.? 172.21.15.39:0/1160208137' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-09-21T03:44:27.719 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:44:27 smithi161 ceph-mon[34533]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-21T03:44:27.725 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:44:27 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.5", "id": [8, 11]}]': finished 2024-09-21T03:44:27.725 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:44:27 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.8", "id": [2, 11]}]': finished 2024-09-21T03:44:27.725 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:44:27 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.c", "id": [2, 1]}]': finished 2024-09-21T03:44:27.725 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:44:27 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1f", "id": [8, 4]}]': finished 2024-09-21T03:44:27.725 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:44:27 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2", "id": [3, 9]}]': finished 2024-09-21T03:44:27.725 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:44:27 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.14", "id": [4, 6]}]': finished 2024-09-21T03:44:27.725 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:44:27 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1a", "id": [5, 7]}]': finished 2024-09-21T03:44:27.725 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:44:27 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1d", "id": [4, 8]}]': finished 2024-09-21T03:44:27.725 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:44:27 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2c", "id": [4, 9]}]': finished 2024-09-21T03:44:27.725 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:44:27 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.38", "id": [3, 9]}]': finished 2024-09-21T03:44:27.725 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:44:27 smithi150 ceph-mon[34289]: osdmap e78: 12 total, 12 up, 12 in 2024-09-21T03:44:27.725 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:44:27 smithi150 ceph-mon[34289]: from='client.? 172.21.15.39:0/1160208137' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-09-21T03:44:27.725 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:44:27 smithi150 ceph-mon[34289]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-21T03:44:27.807 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/51116d5c-77ca-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:44:27.833 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:44:27 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.5", "id": [8, 11]}]': finished 2024-09-21T03:44:27.834 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:44:27 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.8", "id": [2, 11]}]': finished 2024-09-21T03:44:27.834 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:44:27 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.c", "id": [2, 1]}]': finished 2024-09-21T03:44:27.834 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:44:27 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1f", "id": [8, 4]}]': finished 2024-09-21T03:44:27.834 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:44:27 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2", "id": [3, 9]}]': finished 2024-09-21T03:44:27.834 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:44:27 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.14", "id": [4, 6]}]': finished 2024-09-21T03:44:27.834 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:44:27 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1a", "id": [5, 7]}]': finished 2024-09-21T03:44:27.834 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:44:27 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1d", "id": [4, 8]}]': finished 2024-09-21T03:44:27.834 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:44:27 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2c", "id": [4, 9]}]': finished 2024-09-21T03:44:27.834 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:44:27 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.38", "id": [3, 9]}]': finished 2024-09-21T03:44:27.834 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:44:27 smithi039 ceph-mon[31211]: osdmap e78: 12 total, 12 up, 12 in 2024-09-21T03:44:27.834 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:44:27 smithi039 ceph-mon[31211]: from='client.? 172.21.15.39:0/1160208137' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-09-21T03:44:27.834 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:44:27 smithi039 ceph-mon[31211]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-21T03:44:28.292 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:28.287+0000 7fb132bf7640 1 -- 172.21.15.39:0/3790131445 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb12c108cf0 msgr2=0x7fb12c10f520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:28.292 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:28.287+0000 7fb132bf7640 1 --2- 172.21.15.39:0/3790131445 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb12c108cf0 0x7fb12c10f520 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fb124009ca0 tx=0x7fb12402a8c0 comp rx=0 tx=0).stop 2024-09-21T03:44:28.292 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:28.288+0000 7fb132bf7640 1 -- 172.21.15.39:0/3790131445 shutdown_connections 2024-09-21T03:44:28.292 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:28.288+0000 7fb132bf7640 1 --2- 172.21.15.39:0/3790131445 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb12c108cf0 0x7fb12c10f520 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:28.292 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:28.288+0000 7fb132bf7640 1 --2- 172.21.15.39:0/3790131445 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb12c1083a0 0x7fb12c1087b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:28.292 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:28.288+0000 7fb132bf7640 1 --2- 172.21.15.39:0/3790131445 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb12c102a60 0x7fb12c102e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:28.293 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:28.288+0000 7fb132bf7640 1 -- 172.21.15.39:0/3790131445 >> 172.21.15.39:0/3790131445 conn(0x7fb12c079730 msgr2=0x7fb12c079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:44:28.293 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:28.288+0000 7fb132bf7640 1 -- 172.21.15.39:0/3790131445 shutdown_connections 2024-09-21T03:44:28.293 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:28.288+0000 7fb132bf7640 1 -- 172.21.15.39:0/3790131445 wait complete. 2024-09-21T03:44:28.293 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:28.288+0000 7fb132bf7640 1 Processor -- start 2024-09-21T03:44:28.293 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:28.288+0000 7fb132bf7640 1 -- start start 2024-09-21T03:44:28.293 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:28.289+0000 7fb132bf7640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb12c102a60 0x7fb12c19b270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:28.293 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:28.289+0000 7fb132bf7640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb12c1083a0 0x7fb12c19b7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:28.293 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:28.289+0000 7fb132bf7640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb12c108cf0 0x7fb12c19bcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:28.293 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:28.289+0000 7fb132bf7640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fb12c112960 con 0x7fb12c1083a0 2024-09-21T03:44:28.293 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:28.289+0000 7fb132bf7640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fb12c1127e0 con 0x7fb12c108cf0 2024-09-21T03:44:28.293 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:28.289+0000 7fb132bf7640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fb12c112ae0 con 0x7fb12c102a60 2024-09-21T03:44:28.293 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:28.289+0000 7fb12bfff640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb12c1083a0 0x7fb12c19b7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:28.293 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:28.289+0000 7fb12bfff640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb12c1083a0 0x7fb12c19b7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:36656/0 (socket says 172.21.15.39:36656) 2024-09-21T03:44:28.294 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:28.289+0000 7fb12bfff640 1 -- 172.21.15.39:0/3470138756 learned_addr learned my addr 172.21.15.39:0/3470138756 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:44:28.294 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:28.289+0000 7fb130993640 1 --2- 172.21.15.39:0/3470138756 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb12c102a60 0x7fb12c19b270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:28.294 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:28.289+0000 7fb131194640 1 --2- 172.21.15.39:0/3470138756 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb12c108cf0 0x7fb12c19bcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:28.294 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:28.289+0000 7fb12bfff640 1 -- 172.21.15.39:0/3470138756 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb12c102a60 msgr2=0x7fb12c19b270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:28.294 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:28.289+0000 7fb12bfff640 1 --2- 172.21.15.39:0/3470138756 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb12c102a60 0x7fb12c19b270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:28.294 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:28.289+0000 7fb12bfff640 1 -- 172.21.15.39:0/3470138756 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb12c108cf0 msgr2=0x7fb12c19bcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:28.294 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:28.289+0000 7fb12bfff640 1 --2- 172.21.15.39:0/3470138756 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb12c108cf0 0x7fb12c19bcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:28.294 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:28.289+0000 7fb12bfff640 1 -- 172.21.15.39:0/3470138756 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb12c1a6530 con 0x7fb12c1083a0 2024-09-21T03:44:28.294 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:28.289+0000 7fb131194640 1 --2- 172.21.15.39:0/3470138756 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb12c108cf0 0x7fb12c19bcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:44:28.294 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:28.289+0000 7fb12bfff640 1 --2- 172.21.15.39:0/3470138756 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb12c1083a0 0x7fb12c19b7b0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fb11800e9c0 tx=0x7fb11800ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:44:28.294 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:28.289+0000 7fb129ffb640 1 -- 172.21.15.39:0/3470138756 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb11800cd80 con 0x7fb12c1083a0 2024-09-21T03:44:28.295 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:28.290+0000 7fb129ffb640 1 -- 172.21.15.39:0/3470138756 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb1180054d0 con 0x7fb12c1083a0 2024-09-21T03:44:28.295 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:28.290+0000 7fb132bf7640 1 -- 172.21.15.39:0/3470138756 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb12c1a6820 con 0x7fb12c1083a0 2024-09-21T03:44:28.295 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:28.290+0000 7fb129ffb640 1 -- 172.21.15.39:0/3470138756 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb118010430 con 0x7fb12c1083a0 2024-09-21T03:44:28.295 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:28.290+0000 7fb132bf7640 1 -- 172.21.15.39:0/3470138756 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb12c1a6d60 con 0x7fb12c1083a0 2024-09-21T03:44:28.295 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:28.291+0000 7fb132bf7640 1 -- 172.21.15.39:0/3470138756 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb12c102e70 con 0x7fb12c1083a0 2024-09-21T03:44:28.295 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:28.292+0000 7fb129ffb640 1 -- 172.21.15.39:0/3470138756 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 21) ==== 154529+0+0 (secure 0 0 0) 0x7fb118004fd0 con 0x7fb12c1083a0 2024-09-21T03:44:28.296 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:28.293+0000 7fb129ffb640 1 --2- 172.21.15.39:0/3470138756 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7fb11c0b6560 0x7fb11c0b8a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:28.296 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:28.293+0000 7fb130993640 1 --2- 172.21.15.39:0/3470138756 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7fb11c0b6560 0x7fb11c0b8a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:28.296 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:28.293+0000 7fb129ffb640 1 -- 172.21.15.39:0/3470138756 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8484+0+0 (secure 0 0 0) 0x7fb1180e7910 con 0x7fb12c1083a0 2024-09-21T03:44:28.296 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:28.294+0000 7fb130993640 1 --2- 172.21.15.39:0/3470138756 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7fb11c0b6560 0x7fb11c0b8a70 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fb12c06b9e0 tx=0x7fb120005f90 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:44:28.300 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:28.297+0000 7fb129ffb640 1 -- 172.21.15.39:0/3470138756 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fb1180ae0a0 con 0x7fb12c1083a0 2024-09-21T03:44:28.521 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:28.518+0000 7fb132bf7640 1 -- 172.21.15.39:0/3470138756 --> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7fb12c063800 con 0x7fb11c0b6560 2024-09-21T03:44:28.521 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:44:28 smithi039 ceph-mon[31211]: osdmap e79: 12 total, 12 up, 12 in 2024-09-21T03:44:28.521 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:44:28 smithi039 ceph-mon[31211]: pgmap v278: 97 pgs: 97 active+clean; 582 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s rd, 4.5 KiB/s wr, 16 op/s 2024-09-21T03:44:28.554 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:28.552+0000 7fb129ffb640 1 -- 172.21.15.39:0/3470138756 <== mgr.14223 v2:172.21.15.39:6800/1696381982 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fb12c063800 con 0x7fb11c0b6560 2024-09-21T03:44:28.557 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:28.554+0000 7fb132bf7640 1 -- 172.21.15.39:0/3470138756 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7fb11c0b6560 msgr2=0x7fb11c0b8a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:28.557 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:28.554+0000 7fb132bf7640 1 --2- 172.21.15.39:0/3470138756 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7fb11c0b6560 0x7fb11c0b8a70 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fb12c06b9e0 tx=0x7fb120005f90 comp rx=0 tx=0).stop 2024-09-21T03:44:28.557 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:28.554+0000 7fb132bf7640 1 -- 172.21.15.39:0/3470138756 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb12c1083a0 msgr2=0x7fb12c19b7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:28.557 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:28.554+0000 7fb132bf7640 1 --2- 172.21.15.39:0/3470138756 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb12c1083a0 0x7fb12c19b7b0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fb11800e9c0 tx=0x7fb11800ee80 comp rx=0 tx=0).stop 2024-09-21T03:44:28.558 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:28.554+0000 7fb132bf7640 1 -- 172.21.15.39:0/3470138756 shutdown_connections 2024-09-21T03:44:28.558 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:28.554+0000 7fb132bf7640 1 --2- 172.21.15.39:0/3470138756 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7fb11c0b6560 0x7fb11c0b8a70 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:28.558 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:28.554+0000 7fb132bf7640 1 --2- 172.21.15.39:0/3470138756 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb12c108cf0 0x7fb12c19bcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:28.558 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:28.554+0000 7fb132bf7640 1 --2- 172.21.15.39:0/3470138756 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb12c1083a0 0x7fb12c19b7b0 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:28.558 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:28.554+0000 7fb132bf7640 1 --2- 172.21.15.39:0/3470138756 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb12c102a60 0x7fb12c19b270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:28.558 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:28.554+0000 7fb132bf7640 1 -- 172.21.15.39:0/3470138756 >> 172.21.15.39:0/3470138756 conn(0x7fb12c079730 msgr2=0x7fb12c104460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:44:28.558 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:28.555+0000 7fb132bf7640 1 -- 172.21.15.39:0/3470138756 shutdown_connections 2024-09-21T03:44:28.558 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:28.555+0000 7fb132bf7640 1 -- 172.21.15.39:0/3470138756 wait complete. 2024-09-21T03:44:28.718 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:44:28 smithi161 ceph-mon[34533]: osdmap e79: 12 total, 12 up, 12 in 2024-09-21T03:44:28.718 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:44:28 smithi161 ceph-mon[34533]: pgmap v278: 97 pgs: 97 active+clean; 582 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s rd, 4.5 KiB/s wr, 16 op/s 2024-09-21T03:44:28.724 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:44:28 smithi150 ceph-mon[34289]: osdmap e79: 12 total, 12 up, 12 in 2024-09-21T03:44:28.724 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:44:28 smithi150 ceph-mon[34289]: pgmap v278: 97 pgs: 97 active+clean; 582 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s rd, 4.5 KiB/s wr, 16 op/s 2024-09-21T03:44:28.833 INFO:journalctl@ceph.mgr.x.smithi039.stdout:Sep 21 03:44:28 smithi039 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-mgr-x[31438]: 2024-09-21T03:44:28.523+0000 7fe42e8a0640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T03:44:28.834 INFO:journalctl@ceph.mgr.x.smithi039.stdout:Sep 21 03:44:28 smithi039 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-mgr-x[31438]: 2024-09-21T03:44:28.523+0000 7fe42e8a0640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T03:44:28.834 INFO:journalctl@ceph.mgr.x.smithi039.stdout:Sep 21 03:44:28 smithi039 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-mgr-x[31438]: 2024-09-21T03:44:28.523+0000 7fe42e8a0640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T03:44:28.834 INFO:journalctl@ceph.mgr.x.smithi039.stdout:Sep 21 03:44:28 smithi039 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-mgr-x[31438]: 2024-09-21T03:44:28.523+0000 7fe42e8a0640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T03:44:28.834 INFO:journalctl@ceph.mgr.x.smithi039.stdout:Sep 21 03:44:28 smithi039 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-mgr-x[31438]: 2024-09-21T03:44:28.523+0000 7fe42e8a0640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T03:44:29.198 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 51116d5c-77ca-11ef-baf5-efdc52797490 -- ceph fs subvolume create cephfs sv_0 --group_name qa 2024-09-21T03:44:29.460 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/51116d5c-77ca-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:44:29.703 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:44:29 smithi039 ceph-mon[31211]: from='client.14952 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:44:29.717 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:44:29 smithi161 ceph-mon[34533]: from='client.14952 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:44:29.724 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:44:29 smithi150 ceph-mon[34289]: from='client.14952 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:44:29.971 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:29.956+0000 7fe83799a640 1 -- 172.21.15.39:0/496689151 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe830108d20 msgr2=0x7fe83010f550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:29.971 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:29.956+0000 7fe83799a640 1 --2- 172.21.15.39:0/496689151 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe830108d20 0x7fe83010f550 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fe828009ca0 tx=0x7fe82802a8c0 comp rx=0 tx=0).stop 2024-09-21T03:44:29.971 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:29.956+0000 7fe83799a640 1 -- 172.21.15.39:0/496689151 shutdown_connections 2024-09-21T03:44:29.971 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:29.956+0000 7fe83799a640 1 --2- 172.21.15.39:0/496689151 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe830108d20 0x7fe83010f550 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:29.971 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:29.956+0000 7fe83799a640 1 --2- 172.21.15.39:0/496689151 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe8301083d0 0x7fe8301087e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:29.971 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:29.956+0000 7fe83799a640 1 --2- 172.21.15.39:0/496689151 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe830102ae0 0x7fe830102ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:29.971 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:29.957+0000 7fe83799a640 1 -- 172.21.15.39:0/496689151 >> 172.21.15.39:0/496689151 conn(0x7fe830079760 msgr2=0x7fe830079b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:44:29.971 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:29.957+0000 7fe83799a640 1 -- 172.21.15.39:0/496689151 shutdown_connections 2024-09-21T03:44:29.971 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:29.957+0000 7fe83799a640 1 -- 172.21.15.39:0/496689151 wait complete. 2024-09-21T03:44:29.971 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:29.958+0000 7fe83799a640 1 Processor -- start 2024-09-21T03:44:29.972 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:29.958+0000 7fe83799a640 1 -- start start 2024-09-21T03:44:29.972 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:29.958+0000 7fe83799a640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe830102ae0 0x7fe8301992e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:29.972 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:29.958+0000 7fe83799a640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe8301083d0 0x7fe830199880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:29.972 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:29.958+0000 7fe83799a640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe830108d20 0x7fe83019eb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:29.972 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:29.958+0000 7fe83799a640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fe830112990 con 0x7fe8301083d0 2024-09-21T03:44:29.972 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:29.958+0000 7fe83799a640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fe830112810 con 0x7fe830108d20 2024-09-21T03:44:29.972 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:29.958+0000 7fe83799a640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fe830112b10 con 0x7fe830102ae0 2024-09-21T03:44:29.972 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:29.958+0000 7fe834f35640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe8301083d0 0x7fe830199880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:29.972 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:29.958+0000 7fe834f35640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe8301083d0 0x7fe830199880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:36704/0 (socket says 172.21.15.39:36704) 2024-09-21T03:44:29.972 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:29.958+0000 7fe834f35640 1 -- 172.21.15.39:0/845374126 learned_addr learned my addr 172.21.15.39:0/845374126 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:44:29.972 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:29.959+0000 7fe835736640 1 --2- 172.21.15.39:0/845374126 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe830102ae0 0x7fe8301992e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:29.972 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:29.959+0000 7fe835f37640 1 --2- 172.21.15.39:0/845374126 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe830108d20 0x7fe83019eb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:29.972 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:29.959+0000 7fe834f35640 1 -- 172.21.15.39:0/845374126 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe830102ae0 msgr2=0x7fe8301992e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:29.973 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:29.959+0000 7fe834f35640 1 --2- 172.21.15.39:0/845374126 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe830102ae0 0x7fe8301992e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:29.973 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:29.959+0000 7fe834f35640 1 -- 172.21.15.39:0/845374126 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe830108d20 msgr2=0x7fe83019eb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:29.973 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:29.959+0000 7fe834f35640 1 --2- 172.21.15.39:0/845374126 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe830108d20 0x7fe83019eb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:29.973 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:29.959+0000 7fe834f35640 1 -- 172.21.15.39:0/845374126 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe83019f190 con 0x7fe8301083d0 2024-09-21T03:44:29.973 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:29.959+0000 7fe835f37640 1 --2- 172.21.15.39:0/845374126 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe830108d20 0x7fe83019eb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:44:29.973 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:29.959+0000 7fe835736640 1 --2- 172.21.15.39:0/845374126 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe830102ae0 0x7fe8301992e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:44:29.973 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:29.959+0000 7fe834f35640 1 --2- 172.21.15.39:0/845374126 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe8301083d0 0x7fe830199880 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fe82000c970 tx=0x7fe82000ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:44:29.973 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:29.959+0000 7fe8267fc640 1 -- 172.21.15.39:0/845374126 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe820014070 con 0x7fe8301083d0 2024-09-21T03:44:29.973 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:29.959+0000 7fe83799a640 1 -- 172.21.15.39:0/845374126 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe83019f480 con 0x7fe8301083d0 2024-09-21T03:44:29.973 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:29.959+0000 7fe8267fc640 1 -- 172.21.15.39:0/845374126 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe820004440 con 0x7fe8301083d0 2024-09-21T03:44:29.973 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:29.959+0000 7fe8267fc640 1 -- 172.21.15.39:0/845374126 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe820003e50 con 0x7fe8301083d0 2024-09-21T03:44:29.973 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:29.959+0000 7fe83799a640 1 -- 172.21.15.39:0/845374126 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe8301a5ca0 con 0x7fe8301083d0 2024-09-21T03:44:29.973 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:29.960+0000 7fe83799a640 1 -- 172.21.15.39:0/845374126 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe830103d10 con 0x7fe8301083d0 2024-09-21T03:44:29.973 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:29.962+0000 7fe8267fc640 1 -- 172.21.15.39:0/845374126 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 21) ==== 154529+0+0 (secure 0 0 0) 0x7fe820017070 con 0x7fe8301083d0 2024-09-21T03:44:29.974 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:29.962+0000 7fe8267fc640 1 --2- 172.21.15.39:0/845374126 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7fe81c0b6730 0x7fe81c0b8c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:29.974 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:29.963+0000 7fe835736640 1 --2- 172.21.15.39:0/845374126 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7fe81c0b6730 0x7fe81c0b8c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:29.974 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:29.963+0000 7fe8267fc640 1 -- 172.21.15.39:0/845374126 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8484+0+0 (secure 0 0 0) 0x7fe8200e7800 con 0x7fe8301083d0 2024-09-21T03:44:29.974 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:29.963+0000 7fe835736640 1 --2- 172.21.15.39:0/845374126 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7fe81c0b6730 0x7fe81c0b8c40 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fe82c009b40 tx=0x7fe82c005e50 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:44:29.974 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:29.965+0000 7fe8267fc640 1 -- 172.21.15.39:0/845374126 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fe82001e030 con 0x7fe8301083d0 2024-09-21T03:44:30.186 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:30.183+0000 7fe83799a640 1 -- 172.21.15.39:0/845374126 --> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7fe83010ab50 con 0x7fe81c0b6730 2024-09-21T03:44:30.214 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:30.212+0000 7fe8267fc640 1 -- 172.21.15.39:0/845374126 <== mgr.14223 v2:172.21.15.39:6800/1696381982 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fe83010ab50 con 0x7fe81c0b6730 2024-09-21T03:44:30.217 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:30.214+0000 7fe83799a640 1 -- 172.21.15.39:0/845374126 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7fe81c0b6730 msgr2=0x7fe81c0b8c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:30.217 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:30.214+0000 7fe83799a640 1 --2- 172.21.15.39:0/845374126 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7fe81c0b6730 0x7fe81c0b8c40 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fe82c009b40 tx=0x7fe82c005e50 comp rx=0 tx=0).stop 2024-09-21T03:44:30.217 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:30.214+0000 7fe83799a640 1 -- 172.21.15.39:0/845374126 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe8301083d0 msgr2=0x7fe830199880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:30.217 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:30.214+0000 7fe83799a640 1 --2- 172.21.15.39:0/845374126 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe8301083d0 0x7fe830199880 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fe82000c970 tx=0x7fe82000ce30 comp rx=0 tx=0).stop 2024-09-21T03:44:30.217 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:30.214+0000 7fe83799a640 1 -- 172.21.15.39:0/845374126 shutdown_connections 2024-09-21T03:44:30.217 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:30.214+0000 7fe83799a640 1 --2- 172.21.15.39:0/845374126 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7fe81c0b6730 0x7fe81c0b8c40 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:30.217 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:30.214+0000 7fe83799a640 1 --2- 172.21.15.39:0/845374126 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe830108d20 0x7fe83019eb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:30.217 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:30.214+0000 7fe83799a640 1 --2- 172.21.15.39:0/845374126 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe8301083d0 0x7fe830199880 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:30.217 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:30.214+0000 7fe83799a640 1 --2- 172.21.15.39:0/845374126 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe830102ae0 0x7fe8301992e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:30.217 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:30.214+0000 7fe83799a640 1 -- 172.21.15.39:0/845374126 >> 172.21.15.39:0/845374126 conn(0x7fe830079760 msgr2=0x7fe8301043e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:44:30.217 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:30.214+0000 7fe83799a640 1 -- 172.21.15.39:0/845374126 shutdown_connections 2024-09-21T03:44:30.218 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:30.214+0000 7fe83799a640 1 -- 172.21.15.39:0/845374126 wait complete. 2024-09-21T03:44:30.583 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:44:30 smithi039 ceph-mon[31211]: pgmap v279: 97 pgs: 97 active+clean; 587 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 6.5 KiB/s wr, 27 op/s; 146 B/s, 0 objects/s recovering 2024-09-21T03:44:30.583 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:44:30 smithi039 ceph-mon[31211]: from='client.14964 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:44:30.718 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:44:30 smithi161 ceph-mon[34533]: pgmap v279: 97 pgs: 97 active+clean; 587 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 6.5 KiB/s wr, 27 op/s; 146 B/s, 0 objects/s recovering 2024-09-21T03:44:30.718 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:44:30 smithi161 ceph-mon[34533]: from='client.14964 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:44:30.724 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:44:30 smithi150 ceph-mon[34289]: pgmap v279: 97 pgs: 97 active+clean; 587 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 6.5 KiB/s wr, 27 op/s; 146 B/s, 0 objects/s recovering 2024-09-21T03:44:30.725 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:44:30 smithi150 ceph-mon[34289]: from='client.14964 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:44:30.998 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 51116d5c-77ca-11ef-baf5-efdc52797490 -- ceph fs subvolume getpath cephfs --group_name qa sv_0 2024-09-21T03:44:31.257 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/51116d5c-77ca-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:44:31.686 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:44:31 smithi039 ceph-mon[31211]: mgrmap e22: x(active, since 7m), standbys: z, y 2024-09-21T03:44:31.717 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:44:31 smithi161 ceph-mon[34533]: mgrmap e22: x(active, since 7m), standbys: z, y 2024-09-21T03:44:31.724 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:44:31 smithi150 ceph-mon[34289]: mgrmap e22: x(active, since 7m), standbys: z, y 2024-09-21T03:44:31.786 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:31.781+0000 7f1f210c3640 1 -- 172.21.15.39:0/3907238999 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1f1c108cf0 msgr2=0x7f1f1c10f520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:31.787 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:31.781+0000 7f1f210c3640 1 --2- 172.21.15.39:0/3907238999 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1f1c108cf0 0x7f1f1c10f520 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f1f14009900 tx=0x7f1f1402a410 comp rx=0 tx=0).stop 2024-09-21T03:44:31.787 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:31.781+0000 7f1f210c3640 1 -- 172.21.15.39:0/3907238999 shutdown_connections 2024-09-21T03:44:31.787 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:31.781+0000 7f1f210c3640 1 --2- 172.21.15.39:0/3907238999 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1f1c108cf0 0x7f1f1c10f520 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:31.787 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:31.781+0000 7f1f210c3640 1 --2- 172.21.15.39:0/3907238999 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1f1c1083a0 0x7f1f1c1087b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:31.787 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:31.781+0000 7f1f210c3640 1 --2- 172.21.15.39:0/3907238999 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1f1c102a60 0x7f1f1c102e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:31.787 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:31.781+0000 7f1f210c3640 1 -- 172.21.15.39:0/3907238999 >> 172.21.15.39:0/3907238999 conn(0x7f1f1c079730 msgr2=0x7f1f1c079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:44:31.787 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:31.781+0000 7f1f210c3640 1 -- 172.21.15.39:0/3907238999 shutdown_connections 2024-09-21T03:44:31.787 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:31.781+0000 7f1f210c3640 1 -- 172.21.15.39:0/3907238999 wait complete. 2024-09-21T03:44:31.787 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:31.782+0000 7f1f210c3640 1 Processor -- start 2024-09-21T03:44:31.787 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:31.782+0000 7f1f210c3640 1 -- start start 2024-09-21T03:44:31.787 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:31.783+0000 7f1f210c3640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1f1c102a60 0x7f1f1c1050e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:31.787 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:31.783+0000 7f1f210c3640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1f1c1083a0 0x7f1f1c105620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:31.788 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:31.783+0000 7f1f210c3640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1f1c108cf0 0x7f1f1c105b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:31.788 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:31.783+0000 7f1f210c3640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f1f1c1129c0 con 0x7f1f1c108cf0 2024-09-21T03:44:31.788 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:31.783+0000 7f1f210c3640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f1f1c112840 con 0x7f1f1c102a60 2024-09-21T03:44:31.788 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:31.783+0000 7f1f210c3640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f1f1c112b40 con 0x7f1f1c1083a0 2024-09-21T03:44:31.788 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:31.783+0000 7f1f1b59e640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1f1c108cf0 0x7f1f1c105b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:31.788 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:31.783+0000 7f1f1b59e640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1f1c108cf0 0x7f1f1c105b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:36734/0 (socket says 172.21.15.39:36734) 2024-09-21T03:44:31.788 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:31.783+0000 7f1f1b59e640 1 -- 172.21.15.39:0/474987879 learned_addr learned my addr 172.21.15.39:0/474987879 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:44:31.788 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:31.783+0000 7f1f1a59c640 1 --2- 172.21.15.39:0/474987879 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1f1c1083a0 0x7f1f1c105620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:31.788 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:31.783+0000 7f1f1ad9d640 1 --2- 172.21.15.39:0/474987879 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1f1c102a60 0x7f1f1c1050e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:31.788 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:31.783+0000 7f1f1b59e640 1 -- 172.21.15.39:0/474987879 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1f1c1083a0 msgr2=0x7f1f1c105620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:31.788 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:31.783+0000 7f1f1b59e640 1 --2- 172.21.15.39:0/474987879 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1f1c1083a0 0x7f1f1c105620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:31.788 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:31.783+0000 7f1f1b59e640 1 -- 172.21.15.39:0/474987879 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1f1c102a60 msgr2=0x7f1f1c1050e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:31.788 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:31.783+0000 7f1f1b59e640 1 --2- 172.21.15.39:0/474987879 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1f1c102a60 0x7f1f1c1050e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:31.789 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:31.783+0000 7f1f1b59e640 1 -- 172.21.15.39:0/474987879 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1f1c1a86b0 con 0x7f1f1c108cf0 2024-09-21T03:44:31.789 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:31.783+0000 7f1f1ad9d640 1 --2- 172.21.15.39:0/474987879 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1f1c102a60 0x7f1f1c1050e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:44:31.789 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:31.783+0000 7f1f1b59e640 1 --2- 172.21.15.39:0/474987879 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1f1c108cf0 0x7f1f1c105b60 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f1f14006010 tx=0x7f1f14004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:44:31.789 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:31.783+0000 7f1f13fff640 1 -- 172.21.15.39:0/474987879 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f1403f030 con 0x7f1f1c108cf0 2024-09-21T03:44:31.789 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:31.784+0000 7f1f210c3640 1 -- 172.21.15.39:0/474987879 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1f1c1a8940 con 0x7f1f1c108cf0 2024-09-21T03:44:31.789 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:31.784+0000 7f1f13fff640 1 -- 172.21.15.39:0/474987879 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1f14003aa0 con 0x7f1f1c108cf0 2024-09-21T03:44:31.789 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:31.784+0000 7f1f13fff640 1 -- 172.21.15.39:0/474987879 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f14035070 con 0x7f1f1c108cf0 2024-09-21T03:44:31.789 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:31.784+0000 7f1f210c3640 1 -- 172.21.15.39:0/474987879 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1f1c1a8ef0 con 0x7f1f1c108cf0 2024-09-21T03:44:31.790 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:31.785+0000 7f1f210c3640 1 -- 172.21.15.39:0/474987879 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1f1c102e70 con 0x7f1f1c108cf0 2024-09-21T03:44:31.790 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:31.786+0000 7f1f13fff640 1 -- 172.21.15.39:0/474987879 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 22) ==== 154580+0+0 (secure 0 0 0) 0x7f1f14042030 con 0x7f1f1c108cf0 2024-09-21T03:44:31.790 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:31.787+0000 7f1f13fff640 1 --2- 172.21.15.39:0/474987879 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f1f080b6670 0x7f1f080b8b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:31.790 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:31.787+0000 7f1f1ad9d640 1 --2- 172.21.15.39:0/474987879 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f1f080b6670 0x7f1f080b8b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:31.790 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:31.787+0000 7f1f13fff640 1 -- 172.21.15.39:0/474987879 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8484+0+0 (secure 0 0 0) 0x7f1f14107eb0 con 0x7f1f1c108cf0 2024-09-21T03:44:31.790 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:31.788+0000 7f1f1ad9d640 1 --2- 172.21.15.39:0/474987879 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f1f080b6670 0x7f1f080b8b80 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f1f1c106b30 tx=0x7f1f0c008370 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:44:31.794 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:31.791+0000 7f1f13fff640 1 -- 172.21.15.39:0/474987879 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f1f140ce4e0 con 0x7f1f1c108cf0 2024-09-21T03:44:32.013 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:32.011+0000 7f1f210c3640 1 -- 172.21.15.39:0/474987879 --> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}) -- 0x7f1f1c063800 con 0x7f1f080b6670 2024-09-21T03:44:32.019 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:32.017+0000 7f1f13fff640 1 -- 172.21.15.39:0/474987879 <== mgr.14223 v2:172.21.15.39:6800/1696381982 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+54 (secure 0 0 0) 0x7f1f1c063800 con 0x7f1f080b6670 2024-09-21T03:44:32.020 INFO:teuthology.orchestra.run.smithi039.stdout:/volumes/qa/sv_0/a46f377b-17ca-4164-98a7-58f13be1e549 2024-09-21T03:44:32.022 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:32.019+0000 7f1f210c3640 1 -- 172.21.15.39:0/474987879 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f1f080b6670 msgr2=0x7f1f080b8b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:32.022 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:32.019+0000 7f1f210c3640 1 --2- 172.21.15.39:0/474987879 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f1f080b6670 0x7f1f080b8b80 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f1f1c106b30 tx=0x7f1f0c008370 comp rx=0 tx=0).stop 2024-09-21T03:44:32.022 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:32.019+0000 7f1f210c3640 1 -- 172.21.15.39:0/474987879 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1f1c108cf0 msgr2=0x7f1f1c105b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:32.023 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:32.019+0000 7f1f210c3640 1 --2- 172.21.15.39:0/474987879 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1f1c108cf0 0x7f1f1c105b60 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f1f14006010 tx=0x7f1f14004270 comp rx=0 tx=0).stop 2024-09-21T03:44:32.023 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:32.019+0000 7f1f210c3640 1 -- 172.21.15.39:0/474987879 shutdown_connections 2024-09-21T03:44:32.023 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:32.019+0000 7f1f210c3640 1 --2- 172.21.15.39:0/474987879 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f1f080b6670 0x7f1f080b8b80 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:32.023 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:32.019+0000 7f1f210c3640 1 --2- 172.21.15.39:0/474987879 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1f1c108cf0 0x7f1f1c105b60 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:32.023 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:32.019+0000 7f1f210c3640 1 --2- 172.21.15.39:0/474987879 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1f1c1083a0 0x7f1f1c105620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:32.023 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:32.019+0000 7f1f210c3640 1 --2- 172.21.15.39:0/474987879 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1f1c102a60 0x7f1f1c1050e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:32.023 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:32.019+0000 7f1f210c3640 1 -- 172.21.15.39:0/474987879 >> 172.21.15.39:0/474987879 conn(0x7f1f1c079730 msgr2=0x7f1f1c06b1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:44:32.023 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:32.020+0000 7f1f210c3640 1 -- 172.21.15.39:0/474987879 shutdown_connections 2024-09-21T03:44:32.023 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:32.020+0000 7f1f210c3640 1 -- 172.21.15.39:0/474987879 wait complete. 2024-09-21T03:44:32.583 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:44:32 smithi039 ceph-mon[31211]: pgmap v280: 97 pgs: 97 active+clean; 588 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 6.6 KiB/s wr, 28 op/s; 244 B/s, 1 objects/s recovering 2024-09-21T03:44:32.584 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:44:32 smithi039 ceph-mon[31211]: from='client.14970 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:44:32.585 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 51116d5c-77ca-11ef-baf5-efdc52797490 -- ceph fs subvolume create cephfs sv_1 --group_name qa 2024-09-21T03:44:32.717 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:44:32 smithi161 ceph-mon[34533]: pgmap v280: 97 pgs: 97 active+clean; 588 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 6.6 KiB/s wr, 28 op/s; 244 B/s, 1 objects/s recovering 2024-09-21T03:44:32.718 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:44:32 smithi161 ceph-mon[34533]: from='client.14970 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:44:32.724 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:44:32 smithi150 ceph-mon[34289]: pgmap v280: 97 pgs: 97 active+clean; 588 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 6.6 KiB/s wr, 28 op/s; 244 B/s, 1 objects/s recovering 2024-09-21T03:44:32.724 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:44:32 smithi150 ceph-mon[34289]: from='client.14970 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:44:32.839 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/51116d5c-77ca-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:44:33.327 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:33.305+0000 7fa181aeb640 1 -- 172.21.15.39:0/1144526945 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa17c1094a0 msgr2=0x7fa17c1098b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:33.327 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:33.305+0000 7fa181aeb640 1 --2- 172.21.15.39:0/1144526945 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa17c1094a0 0x7fa17c1098b0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fa164009900 tx=0x7fa16402a410 comp rx=0 tx=0).stop 2024-09-21T03:44:33.327 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:33.305+0000 7fa181aeb640 1 -- 172.21.15.39:0/1144526945 shutdown_connections 2024-09-21T03:44:33.327 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:33.305+0000 7fa181aeb640 1 --2- 172.21.15.39:0/1144526945 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa17c109df0 0x7fa17c110620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:33.327 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:33.305+0000 7fa181aeb640 1 --2- 172.21.15.39:0/1144526945 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa17c1094a0 0x7fa17c1098b0 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:33.327 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:33.305+0000 7fa181aeb640 1 --2- 172.21.15.39:0/1144526945 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa17c103b60 0x7fa17c103f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:33.327 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:33.305+0000 7fa181aeb640 1 -- 172.21.15.39:0/1144526945 >> 172.21.15.39:0/1144526945 conn(0x7fa17c07b850 msgr2=0x7fa17c07bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:44:33.327 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:33.305+0000 7fa181aeb640 1 -- 172.21.15.39:0/1144526945 shutdown_connections 2024-09-21T03:44:33.328 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:33.305+0000 7fa181aeb640 1 -- 172.21.15.39:0/1144526945 wait complete. 2024-09-21T03:44:33.328 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:33.305+0000 7fa181aeb640 1 Processor -- start 2024-09-21T03:44:33.328 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:33.306+0000 7fa181aeb640 1 -- start start 2024-09-21T03:44:33.328 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:33.306+0000 7fa181aeb640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa17c103b60 0x7fa17c19a4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:33.328 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:33.306+0000 7fa181aeb640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa17c1094a0 0x7fa17c19aa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:33.328 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:33.306+0000 7fa181aeb640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa17c109df0 0x7fa17c19fd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:33.328 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:33.306+0000 7fa181aeb640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fa17c113ad0 con 0x7fa17c1094a0 2024-09-21T03:44:33.328 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:33.306+0000 7fa181aeb640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fa17c113950 con 0x7fa17c109df0 2024-09-21T03:44:33.328 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:33.306+0000 7fa181aeb640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fa17c113c50 con 0x7fa17c103b60 2024-09-21T03:44:33.328 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:33.306+0000 7fa17affd640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa17c1094a0 0x7fa17c19aa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:33.328 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:33.306+0000 7fa17affd640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa17c1094a0 0x7fa17c19aa60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:36746/0 (socket says 172.21.15.39:36746) 2024-09-21T03:44:33.328 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:33.306+0000 7fa17affd640 1 -- 172.21.15.39:0/557477020 learned_addr learned my addr 172.21.15.39:0/557477020 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:44:33.328 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:33.306+0000 7fa17b7fe640 1 --2- 172.21.15.39:0/557477020 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa17c103b60 0x7fa17c19a4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:33.328 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:33.306+0000 7fa17bfff640 1 --2- 172.21.15.39:0/557477020 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa17c109df0 0x7fa17c19fd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:33.329 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:33.306+0000 7fa17affd640 1 -- 172.21.15.39:0/557477020 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa17c103b60 msgr2=0x7fa17c19a4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:33.329 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:33.306+0000 7fa17affd640 1 --2- 172.21.15.39:0/557477020 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa17c103b60 0x7fa17c19a4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:33.329 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:33.306+0000 7fa17affd640 1 -- 172.21.15.39:0/557477020 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa17c109df0 msgr2=0x7fa17c19fd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:33.329 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:33.306+0000 7fa17affd640 1 --2- 172.21.15.39:0/557477020 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa17c109df0 0x7fa17c19fd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:33.329 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:33.306+0000 7fa17affd640 1 -- 172.21.15.39:0/557477020 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa17c1a0430 con 0x7fa17c1094a0 2024-09-21T03:44:33.329 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:33.306+0000 7fa17b7fe640 1 --2- 172.21.15.39:0/557477020 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa17c103b60 0x7fa17c19a4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:44:33.329 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:33.307+0000 7fa17affd640 1 --2- 172.21.15.39:0/557477020 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa17c1094a0 0x7fa17c19aa60 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fa16402a930 tx=0x7fa164004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:44:33.329 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:33.307+0000 7fa178ff9640 1 -- 172.21.15.39:0/557477020 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa16403f030 con 0x7fa17c1094a0 2024-09-21T03:44:33.329 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:33.307+0000 7fa178ff9640 1 -- 172.21.15.39:0/557477020 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa164003aa0 con 0x7fa17c1094a0 2024-09-21T03:44:33.329 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:33.307+0000 7fa181aeb640 1 -- 172.21.15.39:0/557477020 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa17c1a06c0 con 0x7fa17c1094a0 2024-09-21T03:44:33.329 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:33.307+0000 7fa178ff9640 1 -- 172.21.15.39:0/557477020 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa164035070 con 0x7fa17c1094a0 2024-09-21T03:44:33.329 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:33.307+0000 7fa181aeb640 1 -- 172.21.15.39:0/557477020 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa17c0755f0 con 0x7fa17c1094a0 2024-09-21T03:44:33.329 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:33.308+0000 7fa181aeb640 1 -- 172.21.15.39:0/557477020 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa17c103f70 con 0x7fa17c1094a0 2024-09-21T03:44:33.329 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:33.309+0000 7fa178ff9640 1 -- 172.21.15.39:0/557477020 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 22) ==== 154580+0+0 (secure 0 0 0) 0x7fa164042030 con 0x7fa17c1094a0 2024-09-21T03:44:33.329 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:33.310+0000 7fa178ff9640 1 --2- 172.21.15.39:0/557477020 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7fa1680b65f0 0x7fa1680b8b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:33.329 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:33.310+0000 7fa17b7fe640 1 --2- 172.21.15.39:0/557477020 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7fa1680b65f0 0x7fa1680b8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:33.330 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:33.310+0000 7fa178ff9640 1 -- 172.21.15.39:0/557477020 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8484+0+0 (secure 0 0 0) 0x7fa1641081d0 con 0x7fa17c1094a0 2024-09-21T03:44:33.330 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:33.311+0000 7fa17b7fe640 1 --2- 172.21.15.39:0/557477020 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7fa1680b65f0 0x7fa1680b8b00 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fa16c009fa0 tx=0x7fa16c009570 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:44:33.330 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:33.313+0000 7fa178ff9640 1 -- 172.21.15.39:0/557477020 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fa1640ce800 con 0x7fa17c1094a0 2024-09-21T03:44:33.533 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:33.530+0000 7fa181aeb640 1 -- 172.21.15.39:0/557477020 --> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7fa17c063800 con 0x7fa1680b65f0 2024-09-21T03:44:33.563 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:33.560+0000 7fa178ff9640 1 -- 172.21.15.39:0/557477020 <== mgr.14223 v2:172.21.15.39:6800/1696381982 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fa17c063800 con 0x7fa1680b65f0 2024-09-21T03:44:33.565 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:33.562+0000 7fa181aeb640 1 -- 172.21.15.39:0/557477020 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7fa1680b65f0 msgr2=0x7fa1680b8b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:33.565 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:33.562+0000 7fa181aeb640 1 --2- 172.21.15.39:0/557477020 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7fa1680b65f0 0x7fa1680b8b00 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fa16c009fa0 tx=0x7fa16c009570 comp rx=0 tx=0).stop 2024-09-21T03:44:33.565 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:33.562+0000 7fa181aeb640 1 -- 172.21.15.39:0/557477020 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa17c1094a0 msgr2=0x7fa17c19aa60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:33.565 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:33.562+0000 7fa181aeb640 1 --2- 172.21.15.39:0/557477020 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa17c1094a0 0x7fa17c19aa60 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fa16402a930 tx=0x7fa164004270 comp rx=0 tx=0).stop 2024-09-21T03:44:33.565 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:33.562+0000 7fa181aeb640 1 -- 172.21.15.39:0/557477020 shutdown_connections 2024-09-21T03:44:33.566 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:33.562+0000 7fa181aeb640 1 --2- 172.21.15.39:0/557477020 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7fa1680b65f0 0x7fa1680b8b00 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:33.566 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:33.562+0000 7fa181aeb640 1 --2- 172.21.15.39:0/557477020 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa17c109df0 0x7fa17c19fd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:33.566 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:33.562+0000 7fa181aeb640 1 --2- 172.21.15.39:0/557477020 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa17c1094a0 0x7fa17c19aa60 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:33.566 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:33.562+0000 7fa181aeb640 1 --2- 172.21.15.39:0/557477020 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa17c103b60 0x7fa17c19a4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:33.566 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:33.562+0000 7fa181aeb640 1 -- 172.21.15.39:0/557477020 >> 172.21.15.39:0/557477020 conn(0x7fa17c07b850 msgr2=0x7fa17c107530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:44:33.566 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:33.563+0000 7fa181aeb640 1 -- 172.21.15.39:0/557477020 shutdown_connections 2024-09-21T03:44:33.566 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:33.563+0000 7fa181aeb640 1 -- 172.21.15.39:0/557477020 wait complete. 2024-09-21T03:44:34.210 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 51116d5c-77ca-11ef-baf5-efdc52797490 -- ceph fs subvolume getpath cephfs --group_name qa sv_1 2024-09-21T03:44:34.468 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/51116d5c-77ca-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:44:34.833 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:44:34 smithi039 ceph-mon[31211]: pgmap v281: 97 pgs: 97 active+clean; 588 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 4.4 KiB/s wr, 17 op/s; 244 B/s, 1 objects/s recovering 2024-09-21T03:44:34.834 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:44:34 smithi039 ceph-mon[31211]: from='client.14976 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:44:34.967 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:44:34 smithi161 ceph-mon[34533]: pgmap v281: 97 pgs: 97 active+clean; 588 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 4.4 KiB/s wr, 17 op/s; 244 B/s, 1 objects/s recovering 2024-09-21T03:44:34.968 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:44:34 smithi161 ceph-mon[34533]: from='client.14976 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:44:34.973 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:34.970+0000 7f965359e640 1 -- 172.21.15.39:0/1994609301 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9654102880 msgr2=0x7f9654102c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:34.973 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:34.970+0000 7f965359e640 1 --2- 172.21.15.39:0/1994609301 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9654102880 0x7f9654102c90 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f9644009900 tx=0x7f964402a410 comp rx=0 tx=0).stop 2024-09-21T03:44:34.973 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:34.970+0000 7f965359e640 1 -- 172.21.15.39:0/1994609301 shutdown_connections 2024-09-21T03:44:34.973 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:34.970+0000 7f965359e640 1 --2- 172.21.15.39:0/1994609301 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9654108b10 0x7f965410f340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:34.973 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:34.970+0000 7f965359e640 1 --2- 172.21.15.39:0/1994609301 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f96541081c0 0x7f96541085d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:34.973 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:34.970+0000 7f965359e640 1 --2- 172.21.15.39:0/1994609301 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9654102880 0x7f9654102c90 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:34.973 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:34.970+0000 7f965359e640 1 -- 172.21.15.39:0/1994609301 >> 172.21.15.39:0/1994609301 conn(0x7f96540fe2d0 msgr2=0x7f9654100720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:44:34.973 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:34.970+0000 7f965359e640 1 -- 172.21.15.39:0/1994609301 shutdown_connections 2024-09-21T03:44:34.973 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:34.970+0000 7f965359e640 1 -- 172.21.15.39:0/1994609301 wait complete. 2024-09-21T03:44:34.974 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:34.971+0000 7f965359e640 1 Processor -- start 2024-09-21T03:44:34.974 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:34.971+0000 7f965359e640 1 -- start start 2024-09-21T03:44:34.975 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:34.972+0000 7f965359e640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9654102880 0x7f965419b150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:34.975 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:34.972+0000 7f965359e640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f96541081c0 0x7f965419b690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:34.975 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:34.972+0000 7f965359e640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9654108b10 0x7f965419bbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:34.975 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:34.972+0000 7f965359e640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f96541127c0 con 0x7f9654108b10 2024-09-21T03:44:34.976 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:34.972+0000 7f965359e640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f9654112640 con 0x7f96541081c0 2024-09-21T03:44:34.976 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:34.972+0000 7f965359e640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f9654112940 con 0x7f9654102880 2024-09-21T03:44:34.976 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:34.972+0000 7f9652d9d640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9654108b10 0x7f965419bbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:34.976 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:34.972+0000 7f9652d9d640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9654108b10 0x7f965419bbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:36766/0 (socket says 172.21.15.39:36766) 2024-09-21T03:44:34.976 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:34.972+0000 7f9652d9d640 1 -- 172.21.15.39:0/51763735 learned_addr learned my addr 172.21.15.39:0/51763735 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:44:34.976 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:34.972+0000 7f964bfff640 1 --2- 172.21.15.39:0/51763735 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f96541081c0 0x7f965419b690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:34.976 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:34.972+0000 7f965259c640 1 --2- 172.21.15.39:0/51763735 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9654102880 0x7f965419b150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:34.976 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:34.972+0000 7f9652d9d640 1 -- 172.21.15.39:0/51763735 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9654102880 msgr2=0x7f965419b150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:34.976 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:34.972+0000 7f9652d9d640 1 --2- 172.21.15.39:0/51763735 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9654102880 0x7f965419b150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:34.976 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:34.972+0000 7f9652d9d640 1 -- 172.21.15.39:0/51763735 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f96541081c0 msgr2=0x7f965419b690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:34.976 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:34.972+0000 7f9652d9d640 1 --2- 172.21.15.39:0/51763735 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f96541081c0 0x7f965419b690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:34.976 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:34.972+0000 7f9652d9d640 1 -- 172.21.15.39:0/51763735 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f96541a6350 con 0x7f9654108b10 2024-09-21T03:44:34.976 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:34.972+0000 7f9652d9d640 1 --2- 172.21.15.39:0/51763735 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9654108b10 0x7f965419bbd0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f964c00cce0 tx=0x7f964c00a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:44:34.977 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:34.973+0000 7f964b7fe640 1 -- 172.21.15.39:0/51763735 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f964c00f450 con 0x7f9654108b10 2024-09-21T03:44:34.977 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:34.973+0000 7f965359e640 1 -- 172.21.15.39:0/51763735 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f96541a6640 con 0x7f9654108b10 2024-09-21T03:44:34.977 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:34.973+0000 7f964b7fe640 1 -- 172.21.15.39:0/51763735 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f964c00faa0 con 0x7f9654108b10 2024-09-21T03:44:34.977 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:34.973+0000 7f964b7fe640 1 -- 172.21.15.39:0/51763735 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f964c018b70 con 0x7f9654108b10 2024-09-21T03:44:34.977 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:34.973+0000 7f965359e640 1 -- 172.21.15.39:0/51763735 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f96541a6b80 con 0x7f9654108b10 2024-09-21T03:44:34.977 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:44:34 smithi150 ceph-mon[34289]: pgmap v281: 97 pgs: 97 active+clean; 588 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 4.4 KiB/s wr, 17 op/s; 244 B/s, 1 objects/s recovering 2024-09-21T03:44:34.977 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:44:34 smithi150 ceph-mon[34289]: from='client.14976 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:44:34.977 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:34.974+0000 7f965359e640 1 -- 172.21.15.39:0/51763735 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9654102c90 con 0x7f9654108b10 2024-09-21T03:44:34.978 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:34.975+0000 7f964b7fe640 1 -- 172.21.15.39:0/51763735 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 22) ==== 154580+0+0 (secure 0 0 0) 0x7f964c018d10 con 0x7f9654108b10 2024-09-21T03:44:34.979 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:34.976+0000 7f964b7fe640 1 --2- 172.21.15.39:0/51763735 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f96400b66a0 0x7f96400b8bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:34.979 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:34.976+0000 7f965259c640 1 --2- 172.21.15.39:0/51763735 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f96400b66a0 0x7f96400b8bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:34.979 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:34.976+0000 7f964b7fe640 1 -- 172.21.15.39:0/51763735 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8484+0+0 (secure 0 0 0) 0x7f964c0e7940 con 0x7f9654108b10 2024-09-21T03:44:34.979 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:34.977+0000 7f965259c640 1 --2- 172.21.15.39:0/51763735 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f96400b66a0 0x7f96400b8bb0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f9644000c00 tx=0x7f9644031400 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:44:34.982 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:34.979+0000 7f964b7fe640 1 -- 172.21.15.39:0/51763735 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f964c014030 con 0x7f9654108b10 2024-09-21T03:44:35.198 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:35.195+0000 7f965359e640 1 -- 172.21.15.39:0/51763735 --> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}) -- 0x7f9654063800 con 0x7f96400b66a0 2024-09-21T03:44:35.204 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:35.202+0000 7f964b7fe640 1 -- 172.21.15.39:0/51763735 <== mgr.14223 v2:172.21.15.39:6800/1696381982 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+54 (secure 0 0 0) 0x7f9654063800 con 0x7f96400b66a0 2024-09-21T03:44:35.204 INFO:teuthology.orchestra.run.smithi039.stdout:/volumes/qa/sv_1/a2925cab-9fdb-4028-a544-bf015f65c2bc 2024-09-21T03:44:35.207 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:35.204+0000 7f965359e640 1 -- 172.21.15.39:0/51763735 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f96400b66a0 msgr2=0x7f96400b8bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:35.207 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:35.204+0000 7f965359e640 1 --2- 172.21.15.39:0/51763735 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f96400b66a0 0x7f96400b8bb0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f9644000c00 tx=0x7f9644031400 comp rx=0 tx=0).stop 2024-09-21T03:44:35.207 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:35.204+0000 7f965359e640 1 -- 172.21.15.39:0/51763735 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9654108b10 msgr2=0x7f965419bbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:35.207 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:35.204+0000 7f965359e640 1 --2- 172.21.15.39:0/51763735 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9654108b10 0x7f965419bbd0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f964c00cce0 tx=0x7f964c00a620 comp rx=0 tx=0).stop 2024-09-21T03:44:35.207 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:35.204+0000 7f965359e640 1 -- 172.21.15.39:0/51763735 shutdown_connections 2024-09-21T03:44:35.207 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:35.204+0000 7f965359e640 1 --2- 172.21.15.39:0/51763735 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f96400b66a0 0x7f96400b8bb0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:35.207 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:35.204+0000 7f965359e640 1 --2- 172.21.15.39:0/51763735 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9654108b10 0x7f965419bbd0 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:35.208 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:35.204+0000 7f965359e640 1 --2- 172.21.15.39:0/51763735 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f96541081c0 0x7f965419b690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:35.208 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:35.204+0000 7f965359e640 1 --2- 172.21.15.39:0/51763735 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9654102880 0x7f965419b150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:35.208 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:35.204+0000 7f965359e640 1 -- 172.21.15.39:0/51763735 >> 172.21.15.39:0/51763735 conn(0x7f96540fe2d0 msgr2=0x7f96541006f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:44:35.208 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:35.204+0000 7f965359e640 1 -- 172.21.15.39:0/51763735 shutdown_connections 2024-09-21T03:44:35.208 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:35.204+0000 7f965359e640 1 -- 172.21.15.39:0/51763735 wait complete. 2024-09-21T03:44:35.834 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:44:35 smithi039 ceph-mon[31211]: from='client.14982 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:44:35.919 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 51116d5c-77ca-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T03:44:35.967 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:44:35 smithi161 ceph-mon[34533]: from='client.14982 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:44:35.974 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:44:35 smithi150 ceph-mon[34289]: from='client.14982 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:44:36.173 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/51116d5c-77ca-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:44:36.670 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:36.652+0000 7f90e73d6640 1 -- 172.21.15.39:0/539095184 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f90e0109e40 msgr2=0x7f90e0110670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:36.670 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:36.652+0000 7f90e73d6640 1 --2- 172.21.15.39:0/539095184 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f90e0109e40 0x7f90e0110670 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f90d8009ca0 tx=0x7f90d802a8c0 comp rx=0 tx=0).stop 2024-09-21T03:44:36.670 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:36.652+0000 7f90e73d6640 1 -- 172.21.15.39:0/539095184 shutdown_connections 2024-09-21T03:44:36.670 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:36.652+0000 7f90e73d6640 1 --2- 172.21.15.39:0/539095184 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f90e0109e40 0x7f90e0110670 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:36.670 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:36.652+0000 7f90e73d6640 1 --2- 172.21.15.39:0/539095184 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f90e01094f0 0x7f90e0109900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:36.670 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:36.652+0000 7f90e73d6640 1 --2- 172.21.15.39:0/539095184 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f90e0103bb0 0x7f90e0103fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:36.670 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:36.652+0000 7f90e73d6640 1 -- 172.21.15.39:0/539095184 >> 172.21.15.39:0/539095184 conn(0x7f90e00ff600 msgr2=0x7f90e0101a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:44:36.671 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:36.652+0000 7f90e73d6640 1 -- 172.21.15.39:0/539095184 shutdown_connections 2024-09-21T03:44:36.671 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:36.652+0000 7f90e73d6640 1 -- 172.21.15.39:0/539095184 wait complete. 2024-09-21T03:44:36.671 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:36.653+0000 7f90e73d6640 1 Processor -- start 2024-09-21T03:44:36.671 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:36.653+0000 7f90e73d6640 1 -- start start 2024-09-21T03:44:36.671 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:36.653+0000 7f90e73d6640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f90e0103bb0 0x7f90e0075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:36.671 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:36.653+0000 7f90e73d6640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f90e01094f0 0x7f90e0075c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:36.671 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:36.653+0000 7f90e73d6640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f90e0109e40 0x7f90e0076190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:36.671 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:36.653+0000 7f90e73d6640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f90e0113b00 con 0x7f90e01094f0 2024-09-21T03:44:36.671 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:36.653+0000 7f90e73d6640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f90e0113980 con 0x7f90e0109e40 2024-09-21T03:44:36.671 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:36.653+0000 7f90e73d6640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f90e0113c80 con 0x7f90e0103bb0 2024-09-21T03:44:36.671 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:36.653+0000 7f90e4971640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f90e01094f0 0x7f90e0075c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:36.671 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:36.653+0000 7f90e4971640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f90e01094f0 0x7f90e0075c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:36798/0 (socket says 172.21.15.39:36798) 2024-09-21T03:44:36.671 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:36.653+0000 7f90e4971640 1 -- 172.21.15.39:0/3005753023 learned_addr learned my addr 172.21.15.39:0/3005753023 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:44:36.671 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:36.653+0000 7f90e5973640 1 --2- 172.21.15.39:0/3005753023 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f90e0109e40 0x7f90e0076190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:36.672 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:36.653+0000 7f90e5172640 1 --2- 172.21.15.39:0/3005753023 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f90e0103bb0 0x7f90e0075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:36.672 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:36.654+0000 7f90e4971640 1 -- 172.21.15.39:0/3005753023 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f90e0103bb0 msgr2=0x7f90e0075710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:36.672 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:36.654+0000 7f90e4971640 1 --2- 172.21.15.39:0/3005753023 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f90e0103bb0 0x7f90e0075710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:36.672 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:36.654+0000 7f90e4971640 1 -- 172.21.15.39:0/3005753023 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f90e0109e40 msgr2=0x7f90e0076190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:36.672 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:36.654+0000 7f90e4971640 1 --2- 172.21.15.39:0/3005753023 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f90e0109e40 0x7f90e0076190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:36.672 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:36.654+0000 7f90e4971640 1 -- 172.21.15.39:0/3005753023 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f90e01a8640 con 0x7f90e01094f0 2024-09-21T03:44:36.672 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:36.654+0000 7f90e4971640 1 --2- 172.21.15.39:0/3005753023 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f90e01094f0 0x7f90e0075c50 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f90d000d950 tx=0x7f90d000dd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:44:36.672 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:36.654+0000 7f90d67fc640 1 -- 172.21.15.39:0/3005753023 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90d0017070 con 0x7f90e01094f0 2024-09-21T03:44:36.672 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:36.654+0000 7f90e73d6640 1 -- 172.21.15.39:0/3005753023 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f90e01a8930 con 0x7f90e01094f0 2024-09-21T03:44:36.672 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:36.654+0000 7f90d67fc640 1 -- 172.21.15.39:0/3005753023 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f90d0009c30 con 0x7f90e01094f0 2024-09-21T03:44:36.672 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:36.654+0000 7f90d67fc640 1 -- 172.21.15.39:0/3005753023 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90d00047a0 con 0x7f90e01094f0 2024-09-21T03:44:36.672 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:36.654+0000 7f90e73d6640 1 -- 172.21.15.39:0/3005753023 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f90e01a8ec0 con 0x7f90e01094f0 2024-09-21T03:44:36.672 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:36.655+0000 7f90e73d6640 1 -- 172.21.15.39:0/3005753023 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f90e0103fc0 con 0x7f90e01094f0 2024-09-21T03:44:36.672 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:36.656+0000 7f90d67fc640 1 -- 172.21.15.39:0/3005753023 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 22) ==== 154580+0+0 (secure 0 0 0) 0x7f90d0004020 con 0x7f90e01094f0 2024-09-21T03:44:36.673 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:36.657+0000 7f90d67fc640 1 --2- 172.21.15.39:0/3005753023 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f90cc0b65f0 0x7f90cc0b8b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:36.673 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:36.657+0000 7f90e5172640 1 --2- 172.21.15.39:0/3005753023 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f90cc0b65f0 0x7f90cc0b8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:36.673 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:36.657+0000 7f90d67fc640 1 -- 172.21.15.39:0/3005753023 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8484+0+0 (secure 0 0 0) 0x7f90d00e73f0 con 0x7f90e01094f0 2024-09-21T03:44:36.673 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:36.658+0000 7f90e5172640 1 --2- 172.21.15.39:0/3005753023 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f90cc0b65f0 0x7f90cc0b8b00 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f90e0077160 tx=0x7f90dc00a2d0 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:44:36.673 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:36.660+0000 7f90d67fc640 1 -- 172.21.15.39:0/3005753023 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f90d0020030 con 0x7f90e01094f0 2024-09-21T03:44:36.833 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:44:36 smithi039 ceph-mon[31211]: pgmap v282: 97 pgs: 97 active+clean; 622 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 9.5 KiB/s rd, 8.5 KiB/s wr, 21 op/s; 215 B/s, 1 objects/s recovering 2024-09-21T03:44:36.892 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:36.890+0000 7f90e73d6640 1 -- 172.21.15.39:0/3005753023 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f90e0076a10 con 0x7f90e01094f0 2024-09-21T03:44:36.893 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:36.891+0000 7f90d67fc640 1 -- 172.21.15.39:0/3005753023 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11978 (secure 0 0 0) 0x7f90d00ada20 con 0x7f90e01094f0 2024-09-21T03:44:36.894 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T03:44:36.894 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":28,"btime":"2024-09-21T03:44:26:704224+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14790,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6835/508995295","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":508995295},{"type":"v1","addr":"172.21.15.39:6835","nonce":508995295}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14802,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6837/3915912607","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":3915912607},{"type":"v1","addr":"172.21.15.39:6837","nonce":3915912607}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T03:44:15.009872+0000","modified":"2024-09-21T03:44:26.017327+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24695,"mds_1":24667,"mds_2":24659,"mds_3":24646,"mds_4":14868},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14856":{"gid":14856,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.39:6839/185509192","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6838","nonce":185509192},{"type":"v1","addr":"172.21.15.39:6839","nonce":185509192}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14868":{"gid":14868,"name":"a","rank":4,"incarnation":25,"state":"up:active","state_seq":7,"addr":"172.21.15.39:6841/4247190456","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6840","nonce":4247190456},{"type":"v1","addr":"172.21.15.39:6841","nonce":4247190456}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24599":{"gid":24599,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.150:6833/3114643217","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":3114643217},{"type":"v1","addr":"172.21.15.150:6833","nonce":3114643217}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24601":{"gid":24601,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.161:6833/2963990654","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":2963990654},{"type":"v1","addr":"172.21.15.161:6833","nonce":2963990654}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24632":{"gid":24632,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.150:6835/724594008","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6834","nonce":724594008},{"type":"v1","addr":"172.21.15.150:6835","nonce":724594008}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24640":{"gid":24640,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.161:6835/3747882110","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":3747882110},{"type":"v1","addr":"172.21.15.161:6835","nonce":3747882110}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24646":{"gid":24646,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":10,"addr":"172.21.15.161:6837/3451544597","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6836","nonce":3451544597},{"type":"v1","addr":"172.21.15.161:6837","nonce":3451544597}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24659":{"gid":24659,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.150:6837/4017043144","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6836","nonce":4017043144},{"type":"v1","addr":"172.21.15.150:6837","nonce":4017043144}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24667":{"gid":24667,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.161:6839/2190107270","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6838","nonce":2190107270},{"type":"v1","addr":"172.21.15.161:6839","nonce":2190107270}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24695":{"gid":24695,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.150:6839/2949386907","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6838","nonce":2949386907},{"type":"v1","addr":"172.21.15.150:6839","nonce":2949386907}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24695,"qdb_cluster":[14868,24646,24659,24667,24695]},"id":1}]} 2024-09-21T03:44:36.894 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 28 2024-09-21T03:44:36.896 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:36.893+0000 7f90e73d6640 1 -- 172.21.15.39:0/3005753023 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f90cc0b65f0 msgr2=0x7f90cc0b8b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:36.896 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:36.893+0000 7f90e73d6640 1 --2- 172.21.15.39:0/3005753023 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f90cc0b65f0 0x7f90cc0b8b00 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f90e0077160 tx=0x7f90dc00a2d0 comp rx=0 tx=0).stop 2024-09-21T03:44:36.896 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:36.893+0000 7f90e73d6640 1 -- 172.21.15.39:0/3005753023 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f90e01094f0 msgr2=0x7f90e0075c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:36.896 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:36.893+0000 7f90e73d6640 1 --2- 172.21.15.39:0/3005753023 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f90e01094f0 0x7f90e0075c50 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f90d000d950 tx=0x7f90d000dd50 comp rx=0 tx=0).stop 2024-09-21T03:44:36.896 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:36.893+0000 7f90e73d6640 1 -- 172.21.15.39:0/3005753023 shutdown_connections 2024-09-21T03:44:36.896 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:36.893+0000 7f90e73d6640 1 --2- 172.21.15.39:0/3005753023 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f90cc0b65f0 0x7f90cc0b8b00 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:36.896 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:36.893+0000 7f90e73d6640 1 --2- 172.21.15.39:0/3005753023 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f90e0109e40 0x7f90e0076190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:36.896 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:36.893+0000 7f90e73d6640 1 --2- 172.21.15.39:0/3005753023 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f90e01094f0 0x7f90e0075c50 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:36.896 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:36.893+0000 7f90e73d6640 1 --2- 172.21.15.39:0/3005753023 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f90e0103bb0 0x7f90e0075710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:36.896 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:36.893+0000 7f90e73d6640 1 -- 172.21.15.39:0/3005753023 >> 172.21.15.39:0/3005753023 conn(0x7f90e00ff600 msgr2=0x7f90e0101a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:44:36.897 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:36.893+0000 7f90e73d6640 1 -- 172.21.15.39:0/3005753023 shutdown_connections 2024-09-21T03:44:36.897 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:36.893+0000 7f90e73d6640 1 -- 172.21.15.39:0/3005753023 wait complete. 2024-09-21T03:44:36.968 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:44:36 smithi161 ceph-mon[34533]: pgmap v282: 97 pgs: 97 active+clean; 622 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 9.5 KiB/s rd, 8.5 KiB/s wr, 21 op/s; 215 B/s, 1 objects/s recovering 2024-09-21T03:44:36.974 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:44:36 smithi150 ceph-mon[34289]: pgmap v282: 97 pgs: 97 active+clean; 622 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 9.5 KiB/s rd, 8.5 KiB/s wr, 21 op/s; 215 B/s, 1 objects/s recovering 2024-09-21T03:44:37.521 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 51116d5c-77ca-11ef-baf5-efdc52797490 -- ceph osd dump --format=json 2024-09-21T03:44:37.779 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/51116d5c-77ca-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:44:37.833 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:44:37 smithi039 ceph-mon[31211]: from='client.? 172.21.15.39:0/3005753023' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:44:37.967 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:44:37 smithi161 ceph-mon[34533]: from='client.? 172.21.15.39:0/3005753023' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:44:37.974 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:44:37 smithi150 ceph-mon[34289]: from='client.? 172.21.15.39:0/3005753023' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:44:38.329 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:38.327+0000 7f88d4ed6640 1 -- 172.21.15.39:0/1160410895 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f88d01083a0 msgr2=0x7f88d01087b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:38.330 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:38.327+0000 7f88d4ed6640 1 --2- 172.21.15.39:0/1160410895 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f88d01083a0 0x7f88d01087b0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f88b8009900 tx=0x7f88b802a410 comp rx=0 tx=0).stop 2024-09-21T03:44:38.330 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:38.327+0000 7f88d4ed6640 1 -- 172.21.15.39:0/1160410895 shutdown_connections 2024-09-21T03:44:38.330 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:38.327+0000 7f88d4ed6640 1 --2- 172.21.15.39:0/1160410895 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f88d0108cf0 0x7f88d010f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:38.330 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:38.327+0000 7f88d4ed6640 1 --2- 172.21.15.39:0/1160410895 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f88d01083a0 0x7f88d01087b0 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:38.330 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:38.327+0000 7f88d4ed6640 1 --2- 172.21.15.39:0/1160410895 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f88d0102a60 0x7f88d0102e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:38.330 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:38.327+0000 7f88d4ed6640 1 -- 172.21.15.39:0/1160410895 >> 172.21.15.39:0/1160410895 conn(0x7f88d0079730 msgr2=0x7f88d0079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:44:38.330 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:38.327+0000 7f88d4ed6640 1 -- 172.21.15.39:0/1160410895 shutdown_connections 2024-09-21T03:44:38.330 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:38.327+0000 7f88d4ed6640 1 -- 172.21.15.39:0/1160410895 wait complete. 2024-09-21T03:44:38.332 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:38.328+0000 7f88d4ed6640 1 Processor -- start 2024-09-21T03:44:38.332 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:38.328+0000 7f88d4ed6640 1 -- start start 2024-09-21T03:44:38.332 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:38.328+0000 7f88d4ed6640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f88d0102a60 0x7f88d019a590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:38.332 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:38.328+0000 7f88d4ed6640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f88d01083a0 0x7f88d019ab30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:38.332 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:38.328+0000 7f88d4ed6640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f88d0108cf0 0x7f88d019fdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:38.332 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:38.328+0000 7f88d4ed6640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f88d0113a80 con 0x7f88d01083a0 2024-09-21T03:44:38.332 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:38.328+0000 7f88d4ed6640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f88d0113900 con 0x7f88d0108cf0 2024-09-21T03:44:38.332 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:38.328+0000 7f88d4ed6640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f88d0113c00 con 0x7f88d0102a60 2024-09-21T03:44:38.332 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:38.328+0000 7f88c7fff640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f88d01083a0 0x7f88d019ab30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:38.333 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:38.328+0000 7f88c7fff640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f88d01083a0 0x7f88d019ab30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:53824/0 (socket says 172.21.15.39:53824) 2024-09-21T03:44:38.333 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:38.328+0000 7f88c7fff640 1 -- 172.21.15.39:0/1372346527 learned_addr learned my addr 172.21.15.39:0/1372346527 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:44:38.333 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:38.328+0000 7f88ce59c640 1 --2- 172.21.15.39:0/1372346527 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f88d0102a60 0x7f88d019a590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:38.333 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:38.328+0000 7f88ced9d640 1 --2- 172.21.15.39:0/1372346527 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f88d0108cf0 0x7f88d019fdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:38.333 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:38.328+0000 7f88c7fff640 1 -- 172.21.15.39:0/1372346527 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f88d0102a60 msgr2=0x7f88d019a590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:38.333 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:38.328+0000 7f88c7fff640 1 --2- 172.21.15.39:0/1372346527 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f88d0102a60 0x7f88d019a590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:38.333 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:38.328+0000 7f88c7fff640 1 -- 172.21.15.39:0/1372346527 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f88d0108cf0 msgr2=0x7f88d019fdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:38.333 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:38.328+0000 7f88c7fff640 1 --2- 172.21.15.39:0/1372346527 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f88d0108cf0 0x7f88d019fdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:38.333 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:38.328+0000 7f88c7fff640 1 -- 172.21.15.39:0/1372346527 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f88d01a0500 con 0x7f88d01083a0 2024-09-21T03:44:38.333 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:38.329+0000 7f88c7fff640 1 --2- 172.21.15.39:0/1372346527 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f88d01083a0 0x7f88d019ab30 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f88b8032fd0 tx=0x7f88b8031a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:44:38.333 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:38.329+0000 7f88c77fe640 1 -- 172.21.15.39:0/1372346527 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88b803f030 con 0x7f88d01083a0 2024-09-21T03:44:38.333 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:38.329+0000 7f88c77fe640 1 -- 172.21.15.39:0/1372346527 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f88b8004490 con 0x7f88d01083a0 2024-09-21T03:44:38.333 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:38.329+0000 7f88d4ed6640 1 -- 172.21.15.39:0/1372346527 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f88d01a0790 con 0x7f88d01083a0 2024-09-21T03:44:38.334 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:38.329+0000 7f88c77fe640 1 -- 172.21.15.39:0/1372346527 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88b8035070 con 0x7f88d01083a0 2024-09-21T03:44:38.334 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:38.329+0000 7f88d4ed6640 1 -- 172.21.15.39:0/1372346527 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f88d01a6f50 con 0x7f88d01083a0 2024-09-21T03:44:38.334 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:38.330+0000 7f88d4ed6640 1 -- 172.21.15.39:0/1372346527 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f88d0103ce0 con 0x7f88d01083a0 2024-09-21T03:44:38.334 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:38.332+0000 7f88c77fe640 1 -- 172.21.15.39:0/1372346527 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 22) ==== 154580+0+0 (secure 0 0 0) 0x7f88b8042030 con 0x7f88d01083a0 2024-09-21T03:44:38.335 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:38.332+0000 7f88c77fe640 1 --2- 172.21.15.39:0/1372346527 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f88bc0b6670 0x7f88bc0b8b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:38.338 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:38.332+0000 7f88ce59c640 1 --2- 172.21.15.39:0/1372346527 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f88bc0b6670 0x7f88bc0b8b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:38.339 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:38.333+0000 7f88c77fe640 1 -- 172.21.15.39:0/1372346527 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8484+0+0 (secure 0 0 0) 0x7f88b81078c0 con 0x7f88d01083a0 2024-09-21T03:44:38.339 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:38.333+0000 7f88ce59c640 1 --2- 172.21.15.39:0/1372346527 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f88bc0b6670 0x7f88bc0b8b80 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f88c0009bd0 tx=0x7f88c0005f90 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:44:38.339 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:38.337+0000 7f88c77fe640 1 -- 172.21.15.39:0/1372346527 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f88b80d22d0 con 0x7f88d01083a0 2024-09-21T03:44:38.529 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:38.528+0000 7f88d4ed6640 1 -- 172.21.15.39:0/1372346527 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f88d010aac0 con 0x7f88d01083a0 2024-09-21T03:44:38.531 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:38.530+0000 7f88c77fe640 1 -- 172.21.15.39:0/1372346527 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v79) ==== 74+0+23096 (secure 0 0 0) 0x7f88b80d7210 con 0x7f88d01083a0 2024-09-21T03:44:38.532 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T03:44:38.532 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":79,"fsid":"51116d5c-77ca-11ef-baf5-efdc52797490","created":"2024-09-21T03:35:33.631350+0000","modified":"2024-09-21T03:44:27.402588+0000","last_up_change":"2024-09-21T03:43:24.242275+0000","last_in_change":"2024-09-21T03:43:00.886552+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T03:39:29.859542+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-21T03:44:10.886435+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"77","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-21T03:44:13.041783+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"77","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1ea7148f-2ace-4c10-86fe-b8019de80c1f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6802","nonce":870188791},{"type":"v1","addr":"172.21.15.39:6803","nonce":870188791}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6804","nonce":870188791},{"type":"v1","addr":"172.21.15.39:6805","nonce":870188791}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6808","nonce":870188791},{"type":"v1","addr":"172.21.15.39:6809","nonce":870188791}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6806","nonce":870188791},{"type":"v1","addr":"172.21.15.39:6807","nonce":870188791}]},"public_addr":"172.21.15.39:6803/870188791","cluster_addr":"172.21.15.39:6805/870188791","heartbeat_back_addr":"172.21.15.39:6809/870188791","heartbeat_front_addr":"172.21.15.39:6807/870188791","state":["exists","up"]},{"osd":1,"uuid":"7cb5db90-28e8-4a30-bedb-c039687d48b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6800","nonce":172427832},{"type":"v1","addr":"172.21.15.150:6801","nonce":172427832}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6802","nonce":172427832},{"type":"v1","addr":"172.21.15.150:6803","nonce":172427832}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6806","nonce":172427832},{"type":"v1","addr":"172.21.15.150:6807","nonce":172427832}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6804","nonce":172427832},{"type":"v1","addr":"172.21.15.150:6805","nonce":172427832}]},"public_addr":"172.21.15.150:6801/172427832","cluster_addr":"172.21.15.150:6803/172427832","heartbeat_back_addr":"172.21.15.150:6807/172427832","heartbeat_front_addr":"172.21.15.150:6805/172427832","state":["exists","up"]},{"osd":2,"uuid":"b25c6ab3-fc95-4751-bd63-6e3fc23e8bd7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":19,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6800","nonce":2727876133},{"type":"v1","addr":"172.21.15.161:6801","nonce":2727876133}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6802","nonce":2727876133},{"type":"v1","addr":"172.21.15.161:6803","nonce":2727876133}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6806","nonce":2727876133},{"type":"v1","addr":"172.21.15.161:6807","nonce":2727876133}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6804","nonce":2727876133},{"type":"v1","addr":"172.21.15.161:6805","nonce":2727876133}]},"public_addr":"172.21.15.161:6801/2727876133","cluster_addr":"172.21.15.161:6803/2727876133","heartbeat_back_addr":"172.21.15.161:6807/2727876133","heartbeat_front_addr":"172.21.15.161:6805/2727876133","state":["exists","up"]},{"osd":3,"uuid":"14417ae1-4536-45ba-9a5c-c3b447d1a72d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6810","nonce":295394468},{"type":"v1","addr":"172.21.15.39:6811","nonce":295394468}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6812","nonce":295394468},{"type":"v1","addr":"172.21.15.39:6813","nonce":295394468}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6816","nonce":295394468},{"type":"v1","addr":"172.21.15.39:6817","nonce":295394468}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6814","nonce":295394468},{"type":"v1","addr":"172.21.15.39:6815","nonce":295394468}]},"public_addr":"172.21.15.39:6811/295394468","cluster_addr":"172.21.15.39:6813/295394468","heartbeat_back_addr":"172.21.15.39:6817/295394468","heartbeat_front_addr":"172.21.15.39:6815/295394468","state":["exists","up"]},{"osd":4,"uuid":"33c732ef-20c2-43c4-abe6-045b687fc521","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6808","nonce":3097721553},{"type":"v1","addr":"172.21.15.150:6809","nonce":3097721553}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6810","nonce":3097721553},{"type":"v1","addr":"172.21.15.150:6811","nonce":3097721553}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6814","nonce":3097721553},{"type":"v1","addr":"172.21.15.150:6815","nonce":3097721553}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6812","nonce":3097721553},{"type":"v1","addr":"172.21.15.150:6813","nonce":3097721553}]},"public_addr":"172.21.15.150:6809/3097721553","cluster_addr":"172.21.15.150:6811/3097721553","heartbeat_back_addr":"172.21.15.150:6815/3097721553","heartbeat_front_addr":"172.21.15.150:6813/3097721553","state":["exists","up"]},{"osd":5,"uuid":"74673722-abd6-400a-bb50-ec3dfba0c5e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6808","nonce":742751023},{"type":"v1","addr":"172.21.15.161:6809","nonce":742751023}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6810","nonce":742751023},{"type":"v1","addr":"172.21.15.161:6811","nonce":742751023}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6814","nonce":742751023},{"type":"v1","addr":"172.21.15.161:6815","nonce":742751023}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6812","nonce":742751023},{"type":"v1","addr":"172.21.15.161:6813","nonce":742751023}]},"public_addr":"172.21.15.161:6809/742751023","cluster_addr":"172.21.15.161:6811/742751023","heartbeat_back_addr":"172.21.15.161:6815/742751023","heartbeat_front_addr":"172.21.15.161:6813/742751023","state":["exists","up"]},{"osd":6,"uuid":"06548677-74c4-4efa-8054-df09b957b649","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6818","nonce":1072700290},{"type":"v1","addr":"172.21.15.39:6819","nonce":1072700290}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6820","nonce":1072700290},{"type":"v1","addr":"172.21.15.39:6821","nonce":1072700290}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6824","nonce":1072700290},{"type":"v1","addr":"172.21.15.39:6825","nonce":1072700290}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6822","nonce":1072700290},{"type":"v1","addr":"172.21.15.39:6823","nonce":1072700290}]},"public_addr":"172.21.15.39:6819/1072700290","cluster_addr":"172.21.15.39:6821/1072700290","heartbeat_back_addr":"172.21.15.39:6825/1072700290","heartbeat_front_addr":"172.21.15.39:6823/1072700290","state":["exists","up"]},{"osd":7,"uuid":"838735ca-c693-4f58-9426-b8ee857dbe3a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6816","nonce":2162619103},{"type":"v1","addr":"172.21.15.150:6817","nonce":2162619103}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6818","nonce":2162619103},{"type":"v1","addr":"172.21.15.150:6819","nonce":2162619103}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6822","nonce":2162619103},{"type":"v1","addr":"172.21.15.150:6823","nonce":2162619103}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6820","nonce":2162619103},{"type":"v1","addr":"172.21.15.150:6821","nonce":2162619103}]},"public_addr":"172.21.15.150:6817/2162619103","cluster_addr":"172.21.15.150:6819/2162619103","heartbeat_back_addr":"172.21.15.150:6823/2162619103","heartbeat_front_addr":"172.21.15.150:6821/2162619103","state":["exists","up"]},{"osd":8,"uuid":"4973b357-b4c2-4c00-b7f7-dd25012a0f72","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6816","nonce":52966633},{"type":"v1","addr":"172.21.15.161:6817","nonce":52966633}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6818","nonce":52966633},{"type":"v1","addr":"172.21.15.161:6819","nonce":52966633}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6822","nonce":52966633},{"type":"v1","addr":"172.21.15.161:6823","nonce":52966633}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6820","nonce":52966633},{"type":"v1","addr":"172.21.15.161:6821","nonce":52966633}]},"public_addr":"172.21.15.161:6817/52966633","cluster_addr":"172.21.15.161:6819/52966633","heartbeat_back_addr":"172.21.15.161:6823/52966633","heartbeat_front_addr":"172.21.15.161:6821/52966633","state":["exists","up"]},{"osd":9,"uuid":"6028613c-ebfd-42ea-9949-3f0d5f6cf025","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6826","nonce":2001303253},{"type":"v1","addr":"172.21.15.39:6827","nonce":2001303253}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6828","nonce":2001303253},{"type":"v1","addr":"172.21.15.39:6829","nonce":2001303253}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6832","nonce":2001303253},{"type":"v1","addr":"172.21.15.39:6833","nonce":2001303253}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6830","nonce":2001303253},{"type":"v1","addr":"172.21.15.39:6831","nonce":2001303253}]},"public_addr":"172.21.15.39:6827/2001303253","cluster_addr":"172.21.15.39:6829/2001303253","heartbeat_back_addr":"172.21.15.39:6833/2001303253","heartbeat_front_addr":"172.21.15.39:6831/2001303253","state":["exists","up"]},{"osd":10,"uuid":"1c71c765-2a6c-4ffe-9fe1-c8adae3d1c32","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6824","nonce":2484627938},{"type":"v1","addr":"172.21.15.150:6825","nonce":2484627938}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6826","nonce":2484627938},{"type":"v1","addr":"172.21.15.150:6827","nonce":2484627938}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6830","nonce":2484627938},{"type":"v1","addr":"172.21.15.150:6831","nonce":2484627938}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6828","nonce":2484627938},{"type":"v1","addr":"172.21.15.150:6829","nonce":2484627938}]},"public_addr":"172.21.15.150:6825/2484627938","cluster_addr":"172.21.15.150:6827/2484627938","heartbeat_back_addr":"172.21.15.150:6831/2484627938","heartbeat_front_addr":"172.21.15.150:6829/2484627938","state":["exists","up"]},{"osd":11,"uuid":"f382f522-62db-46da-997e-2b28f84d1612","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6824","nonce":2178709707},{"type":"v1","addr":"172.21.15.161:6825","nonce":2178709707}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6826","nonce":2178709707},{"type":"v1","addr":"172.21.15.161:6827","nonce":2178709707}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6830","nonce":2178709707},{"type":"v1","addr":"172.21.15.161:6831","nonce":2178709707}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6828","nonce":2178709707},{"type":"v1","addr":"172.21.15.161:6829","nonce":2178709707}]},"public_addr":"172.21.15.161:6825/2178709707","cluster_addr":"172.21.15.161:6827/2178709707","heartbeat_back_addr":"172.21.15.161:6831/2178709707","heartbeat_front_addr":"172.21.15.161:6829/2178709707","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:38:35.540904+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:39:01.455053+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:39:27.052867+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:39:55.173529+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:40:21.298396+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:40:47.131235+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:41:14.475209+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:41:39.920036+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:42:05.587065+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:42:32.624726+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:42:57.479185+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:43:22.203832+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.5","mappings":[{"from":8,"to":11}]},{"pgid":"2.8","mappings":[{"from":2,"to":11}]},{"pgid":"2.c","mappings":[{"from":2,"to":1}]},{"pgid":"2.1f","mappings":[{"from":8,"to":4}]},{"pgid":"3.2","mappings":[{"from":3,"to":9}]},{"pgid":"3.14","mappings":[{"from":4,"to":6}]},{"pgid":"3.1a","mappings":[{"from":5,"to":7}]},{"pgid":"3.1d","mappings":[{"from":4,"to":8}]},{"pgid":"3.2c","mappings":[{"from":4,"to":9}]},{"pgid":"3.38","mappings":[{"from":3,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.39:6801/2438110611":"2024-09-22T03:37:25.391457+0000","172.21.15.39:6800/2438110611":"2024-09-22T03:37:25.391457+0000","172.21.15.39:6800/173264695":"2024-09-22T03:35:57.104467+0000","172.21.15.39:6801/173264695":"2024-09-22T03:35:57.104467+0000","172.21.15.39:0/2637182284":"2024-09-22T03:35:57.104467+0000","172.21.15.39:0/1517304561":"2024-09-22T03:36:24.112592+0000","172.21.15.39:6801/1435622199":"2024-09-22T03:36:24.112592+0000","172.21.15.39:6800/1435622199":"2024-09-22T03:36:24.112592+0000","172.21.15.39:0/346832469":"2024-09-22T03:35:57.104467+0000","172.21.15.39:0/3160777371":"2024-09-22T03:36:24.112592+0000","172.21.15.39:0/1730342855":"2024-09-22T03:35:57.104467+0000","172.21.15.39:0/1955726766":"2024-09-22T03:36:24.112592+0000","172.21.15.39:0/3294132796":"2024-09-22T03:37:25.391457+0000","172.21.15.39:0/1295810162":"2024-09-22T03:37:25.391457+0000","172.21.15.39:0/3729230474":"2024-09-22T03:37:25.391457+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T03:44:38.534 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:38.532+0000 7f88d4ed6640 1 -- 172.21.15.39:0/1372346527 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f88bc0b6670 msgr2=0x7f88bc0b8b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:38.534 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:38.532+0000 7f88d4ed6640 1 --2- 172.21.15.39:0/1372346527 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f88bc0b6670 0x7f88bc0b8b80 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f88c0009bd0 tx=0x7f88c0005f90 comp rx=0 tx=0).stop 2024-09-21T03:44:38.534 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:38.532+0000 7f88d4ed6640 1 -- 172.21.15.39:0/1372346527 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f88d01083a0 msgr2=0x7f88d019ab30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:38.534 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:38.532+0000 7f88d4ed6640 1 --2- 172.21.15.39:0/1372346527 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f88d01083a0 0x7f88d019ab30 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f88b8032fd0 tx=0x7f88b8031a20 comp rx=0 tx=0).stop 2024-09-21T03:44:38.534 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:38.532+0000 7f88d4ed6640 1 -- 172.21.15.39:0/1372346527 shutdown_connections 2024-09-21T03:44:38.534 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:38.532+0000 7f88d4ed6640 1 --2- 172.21.15.39:0/1372346527 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f88bc0b6670 0x7f88bc0b8b80 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:38.534 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:38.532+0000 7f88d4ed6640 1 --2- 172.21.15.39:0/1372346527 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f88d0108cf0 0x7f88d019fdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:38.534 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:38.532+0000 7f88d4ed6640 1 --2- 172.21.15.39:0/1372346527 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f88d01083a0 0x7f88d019ab30 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:38.534 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:38.532+0000 7f88d4ed6640 1 --2- 172.21.15.39:0/1372346527 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f88d0102a60 0x7f88d019a590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:38.534 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:38.532+0000 7f88d4ed6640 1 -- 172.21.15.39:0/1372346527 >> 172.21.15.39:0/1372346527 conn(0x7f88d0079730 msgr2=0x7f88d01044c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:44:38.534 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:38.532+0000 7f88d4ed6640 1 -- 172.21.15.39:0/1372346527 shutdown_connections 2024-09-21T03:44:38.535 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:38.532+0000 7f88d4ed6640 1 -- 172.21.15.39:0/1372346527 wait complete. 2024-09-21T03:44:38.833 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:44:38 smithi039 ceph-mon[31211]: pgmap v283: 97 pgs: 97 active+clean; 622 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 8.6 KiB/s rd, 6.8 KiB/s wr, 18 op/s; 193 B/s, 1 objects/s recovering 2024-09-21T03:44:38.834 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:44:38 smithi039 ceph-mon[31211]: from='client.? 172.21.15.39:0/1372346527' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T03:44:38.869 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:44:38 smithi161 ceph-mon[34533]: pgmap v283: 97 pgs: 97 active+clean; 622 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 8.6 KiB/s rd, 6.8 KiB/s wr, 18 op/s; 193 B/s, 1 objects/s recovering 2024-09-21T03:44:38.869 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:44:38 smithi161 ceph-mon[34533]: from='client.? 172.21.15.39:0/1372346527' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T03:44:38.974 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:44:38 smithi150 ceph-mon[34289]: pgmap v283: 97 pgs: 97 active+clean; 622 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 8.6 KiB/s rd, 6.8 KiB/s wr, 18 op/s; 193 B/s, 1 objects/s recovering 2024-09-21T03:44:38.975 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:44:38 smithi150 ceph-mon[34289]: from='client.? 172.21.15.39:0/1372346527' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T03:44:39.356 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-09-21T03:44:39.356 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 51116d5c-77ca-11ef-baf5-efdc52797490 -- ceph pg dump --format=json 2024-09-21T03:44:39.615 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/51116d5c-77ca-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:44:40.128 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:40.092+0000 7fe13619d640 1 -- 172.21.15.39:0/1778310351 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe1301083a0 msgr2=0x7fe1301087b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:40.128 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:40.092+0000 7fe13619d640 1 --2- 172.21.15.39:0/1778310351 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe1301083a0 0x7fe1301087b0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fe118009900 tx=0x7fe11802a410 comp rx=0 tx=0).stop 2024-09-21T03:44:40.128 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:40.092+0000 7fe13619d640 1 -- 172.21.15.39:0/1778310351 shutdown_connections 2024-09-21T03:44:40.128 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:40.092+0000 7fe13619d640 1 --2- 172.21.15.39:0/1778310351 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe130108cf0 0x7fe13010f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:40.128 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:40.092+0000 7fe13619d640 1 --2- 172.21.15.39:0/1778310351 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe1301083a0 0x7fe1301087b0 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:40.128 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:40.092+0000 7fe13619d640 1 --2- 172.21.15.39:0/1778310351 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe130102a60 0x7fe130102e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:40.128 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:40.092+0000 7fe13619d640 1 -- 172.21.15.39:0/1778310351 >> 172.21.15.39:0/1778310351 conn(0x7fe130079730 msgr2=0x7fe130079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:44:40.128 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:40.092+0000 7fe13619d640 1 -- 172.21.15.39:0/1778310351 shutdown_connections 2024-09-21T03:44:40.129 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:40.092+0000 7fe13619d640 1 -- 172.21.15.39:0/1778310351 wait complete. 2024-09-21T03:44:40.129 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:40.093+0000 7fe13619d640 1 Processor -- start 2024-09-21T03:44:40.129 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:40.093+0000 7fe13619d640 1 -- start start 2024-09-21T03:44:40.129 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:40.093+0000 7fe13619d640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe130102a60 0x7fe13019c670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:40.129 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:40.093+0000 7fe13619d640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe1301083a0 0x7fe13019cc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:40.129 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:40.093+0000 7fe13619d640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe130108cf0 0x7fe1301a1eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:40.129 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:40.093+0000 7fe13619d640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fe130112960 con 0x7fe130102a60 2024-09-21T03:44:40.129 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:40.093+0000 7fe13619d640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fe1301127e0 con 0x7fe1301083a0 2024-09-21T03:44:40.129 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:40.093+0000 7fe13619d640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fe130112ae0 con 0x7fe130108cf0 2024-09-21T03:44:40.129 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:40.093+0000 7fe12f7fe640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe130102a60 0x7fe13019c670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:40.129 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:40.093+0000 7fe12f7fe640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe130102a60 0x7fe13019c670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:53842/0 (socket says 172.21.15.39:53842) 2024-09-21T03:44:40.129 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:40.093+0000 7fe12f7fe640 1 -- 172.21.15.39:0/3142416392 learned_addr learned my addr 172.21.15.39:0/3142416392 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:44:40.129 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:40.093+0000 7fe12effd640 1 --2- 172.21.15.39:0/3142416392 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe1301083a0 0x7fe13019cc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:40.130 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:40.093+0000 7fe12ffff640 1 --2- 172.21.15.39:0/3142416392 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe130108cf0 0x7fe1301a1eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:40.130 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:40.094+0000 7fe12f7fe640 1 -- 172.21.15.39:0/3142416392 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe130108cf0 msgr2=0x7fe1301a1eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:40.130 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:40.094+0000 7fe12f7fe640 1 --2- 172.21.15.39:0/3142416392 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe130108cf0 0x7fe1301a1eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:40.130 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:40.094+0000 7fe12f7fe640 1 -- 172.21.15.39:0/3142416392 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe1301083a0 msgr2=0x7fe13019cc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:40.130 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:40.094+0000 7fe12f7fe640 1 --2- 172.21.15.39:0/3142416392 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe1301083a0 0x7fe13019cc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:40.130 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:40.094+0000 7fe12f7fe640 1 -- 172.21.15.39:0/3142416392 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe1301a25e0 con 0x7fe130102a60 2024-09-21T03:44:40.130 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:40.094+0000 7fe12f7fe640 1 --2- 172.21.15.39:0/3142416392 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe130102a60 0x7fe13019c670 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fe12000ea80 tx=0x7fe12000ef40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:44:40.130 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:40.094+0000 7fe12cff9640 1 -- 172.21.15.39:0/3142416392 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe12000ce40 con 0x7fe130102a60 2024-09-21T03:44:40.130 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:40.094+0000 7fe13619d640 1 -- 172.21.15.39:0/3142416392 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe1301a28d0 con 0x7fe130102a60 2024-09-21T03:44:40.130 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:40.094+0000 7fe12cff9640 1 -- 172.21.15.39:0/3142416392 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe120005020 con 0x7fe130102a60 2024-09-21T03:44:40.130 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:40.094+0000 7fe12cff9640 1 -- 172.21.15.39:0/3142416392 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1200116d0 con 0x7fe130102a60 2024-09-21T03:44:40.131 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:40.094+0000 7fe13619d640 1 -- 172.21.15.39:0/3142416392 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe1301a9030 con 0x7fe130102a60 2024-09-21T03:44:40.131 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:40.095+0000 7fe13619d640 1 -- 172.21.15.39:0/3142416392 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe130102e70 con 0x7fe130102a60 2024-09-21T03:44:40.131 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:40.097+0000 7fe12cff9640 1 -- 172.21.15.39:0/3142416392 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 22) ==== 154580+0+0 (secure 0 0 0) 0x7fe1200054d0 con 0x7fe130102a60 2024-09-21T03:44:40.131 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:40.097+0000 7fe12cff9640 1 --2- 172.21.15.39:0/3142416392 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7fe11c0b65f0 0x7fe11c0b8b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:40.131 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:40.098+0000 7fe12effd640 1 --2- 172.21.15.39:0/3142416392 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7fe11c0b65f0 0x7fe11c0b8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:40.131 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:40.098+0000 7fe12cff9640 1 -- 172.21.15.39:0/3142416392 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8484+0+0 (secure 0 0 0) 0x7fe1200e7be0 con 0x7fe130102a60 2024-09-21T03:44:40.131 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:40.098+0000 7fe12effd640 1 --2- 172.21.15.39:0/3142416392 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7fe11c0b65f0 0x7fe11c0b8b00 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fe118000c00 tx=0x7fe118031400 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:44:40.131 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:40.101+0000 7fe12cff9640 1 -- 172.21.15.39:0/3142416392 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fe120016030 con 0x7fe130102a60 2024-09-21T03:44:40.287 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:40.285+0000 7fe13619d640 1 -- 172.21.15.39:0/3142416392 --> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fe130104b40 con 0x7fe11c0b65f0 2024-09-21T03:44:40.291 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:40.290+0000 7fe12cff9640 1 -- 172.21.15.39:0/3142416392 <== mgr.14223 v2:172.21.15.39:6800/1696381982 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258763 (secure 0 0 0) 0x7fe130104b40 con 0x7fe11c0b65f0 2024-09-21T03:44:40.294 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T03:44:40.295 INFO:teuthology.orchestra.run.smithi039.stderr:dumped all 2024-09-21T03:44:40.297 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:40.293+0000 7fe13619d640 1 -- 172.21.15.39:0/3142416392 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7fe11c0b65f0 msgr2=0x7fe11c0b8b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:40.297 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:40.293+0000 7fe13619d640 1 --2- 172.21.15.39:0/3142416392 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7fe11c0b65f0 0x7fe11c0b8b00 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fe118000c00 tx=0x7fe118031400 comp rx=0 tx=0).stop 2024-09-21T03:44:40.297 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:40.293+0000 7fe13619d640 1 -- 172.21.15.39:0/3142416392 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe130102a60 msgr2=0x7fe13019c670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:40.297 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:40.293+0000 7fe13619d640 1 --2- 172.21.15.39:0/3142416392 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe130102a60 0x7fe13019c670 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fe12000ea80 tx=0x7fe12000ef40 comp rx=0 tx=0).stop 2024-09-21T03:44:40.297 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:40.293+0000 7fe13619d640 1 -- 172.21.15.39:0/3142416392 shutdown_connections 2024-09-21T03:44:40.297 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:40.293+0000 7fe13619d640 1 --2- 172.21.15.39:0/3142416392 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7fe11c0b65f0 0x7fe11c0b8b00 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:40.297 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:40.293+0000 7fe13619d640 1 --2- 172.21.15.39:0/3142416392 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe130108cf0 0x7fe1301a1eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:40.298 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:40.293+0000 7fe13619d640 1 --2- 172.21.15.39:0/3142416392 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe1301083a0 0x7fe13019cc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:40.298 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:40.293+0000 7fe13619d640 1 --2- 172.21.15.39:0/3142416392 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe130102a60 0x7fe13019c670 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:40.298 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:40.293+0000 7fe13619d640 1 -- 172.21.15.39:0/3142416392 >> 172.21.15.39:0/3142416392 conn(0x7fe130079730 msgr2=0x7fe130104430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:44:40.298 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:40.293+0000 7fe13619d640 1 -- 172.21.15.39:0/3142416392 shutdown_connections 2024-09-21T03:44:40.298 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:40.293+0000 7fe13619d640 1 -- 172.21.15.39:0/3142416392 wait complete. 2024-09-21T03:44:40.834 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:44:40 smithi039 ceph-mon[31211]: pgmap v284: 97 pgs: 97 active+clean; 634 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 6.9 KiB/s wr, 18 op/s; 162 B/s, 1 objects/s recovering 2024-09-21T03:44:40.967 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:44:40 smithi161 ceph-mon[34533]: pgmap v284: 97 pgs: 97 active+clean; 634 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 6.9 KiB/s wr, 18 op/s; 162 B/s, 1 objects/s recovering 2024-09-21T03:44:40.974 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:44:40 smithi150 ceph-mon[34289]: pgmap v284: 97 pgs: 97 active+clean; 634 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 6.9 KiB/s wr, 18 op/s; 162 B/s, 1 objects/s recovering 2024-09-21T03:44:41.063 INFO:teuthology.orchestra.run.smithi039.stdout:{"pg_ready":true,"pg_map":{"version":284,"stamp":"2024-09-21T03:44:39.492042+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":649137,"num_objects":102,"num_object_clones":0,"num_object_copies":306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":245,"num_read_kb":214,"num_write":353,"num_write_kb":2275,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":23,"num_bytes_recovered":2953794,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":293,"ondisk_log_size":293,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":361676,"kb_used_data":12412,"kb_used_omap":18,"kb_used_meta":349037,"kb_avail":1124580148,"statfs":{"total":1151940427776,"available":1151570071552,"internally_reserved":0,"allocated":12709888,"data_stored":7065290,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19143,"internal_metadata":357414201},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":52869,"num_objects":30,"num_object_clones":0,"num_object_copies":90,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30,"num_whiteouts":0,"num_read":146,"num_read_kb":129,"num_write":81,"num_write_kb":83,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13,"num_bytes_recovered":1954,"num_keys_recovered":0,"num_objects_omap":20,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003332"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":18,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:44:27.420991+0000","last_change":"2024-09-21T03:44:14.327553+0000","last_active":"2024-09-21T03:44:27.420991+0000","last_peered":"2024-09-21T03:44:27.420991+0000","last_clean":"2024-09-21T03:44:27.420991+0000","last_became_active":"2024-09-21T03:44:14.327213+0000","last_became_peered":"2024-09-21T03:44:14.327213+0000","last_unstale":"2024-09-21T03:44:27.420991+0000","last_undegraded":"2024-09-21T03:44:27.420991+0000","last_fullsized":"2024-09-21T03:44:27.420991+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_clean_scrub_stamp":"2024-09-21T03:44:13.292303+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:40:56.550897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":18,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:44:27.417200+0000","last_change":"2024-09-21T03:44:14.321427+0000","last_active":"2024-09-21T03:44:27.417200+0000","last_peered":"2024-09-21T03:44:27.417200+0000","last_clean":"2024-09-21T03:44:27.417200+0000","last_became_active":"2024-09-21T03:44:14.321026+0000","last_became_peered":"2024-09-21T03:44:14.321026+0000","last_unstale":"2024-09-21T03:44:27.417200+0000","last_undegraded":"2024-09-21T03:44:27.417200+0000","last_fullsized":"2024-09-21T03:44:27.417200+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_clean_scrub_stamp":"2024-09-21T03:44:13.292303+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:18:00.889714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":18,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:44:27.416317+0000","last_change":"2024-09-21T03:44:14.324908+0000","last_active":"2024-09-21T03:44:27.416317+0000","last_peered":"2024-09-21T03:44:27.416317+0000","last_clean":"2024-09-21T03:44:27.416317+0000","last_became_active":"2024-09-21T03:44:14.324533+0000","last_became_peered":"2024-09-21T03:44:14.324533+0000","last_unstale":"2024-09-21T03:44:27.416317+0000","last_undegraded":"2024-09-21T03:44:27.416317+0000","last_fullsized":"2024-09-21T03:44:27.416317+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_clean_scrub_stamp":"2024-09-21T03:44:13.292303+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:44:25.235442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":18,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:44:27.613591+0000","last_change":"2024-09-21T03:44:14.326758+0000","last_active":"2024-09-21T03:44:27.613591+0000","last_peered":"2024-09-21T03:44:27.613591+0000","last_clean":"2024-09-21T03:44:27.613591+0000","last_became_active":"2024-09-21T03:44:14.326410+0000","last_became_peered":"2024-09-21T03:44:14.326410+0000","last_unstale":"2024-09-21T03:44:27.613591+0000","last_undegraded":"2024-09-21T03:44:27.613591+0000","last_fullsized":"2024-09-21T03:44:27.613591+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_clean_scrub_stamp":"2024-09-21T03:44:13.292303+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:13:22.716940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":18,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:44:27.419914+0000","last_change":"2024-09-21T03:44:14.318875+0000","last_active":"2024-09-21T03:44:27.419914+0000","last_peered":"2024-09-21T03:44:27.419914+0000","last_clean":"2024-09-21T03:44:27.419914+0000","last_became_active":"2024-09-21T03:44:14.318425+0000","last_became_peered":"2024-09-21T03:44:14.318425+0000","last_unstale":"2024-09-21T03:44:27.419914+0000","last_undegraded":"2024-09-21T03:44:27.419914+0000","last_fullsized":"2024-09-21T03:44:27.419914+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_clean_scrub_stamp":"2024-09-21T03:44:13.292303+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:45:39.916587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":18,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:44:27.611598+0000","last_change":"2024-09-21T03:44:14.325877+0000","last_active":"2024-09-21T03:44:27.611598+0000","last_peered":"2024-09-21T03:44:27.611598+0000","last_clean":"2024-09-21T03:44:27.611598+0000","last_became_active":"2024-09-21T03:44:14.325451+0000","last_became_peered":"2024-09-21T03:44:14.325451+0000","last_unstale":"2024-09-21T03:44:27.611598+0000","last_undegraded":"2024-09-21T03:44:27.611598+0000","last_fullsized":"2024-09-21T03:44:27.611598+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_clean_scrub_stamp":"2024-09-21T03:44:13.292303+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:30:42.427054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":18,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:44:27.418241+0000","last_change":"2024-09-21T03:44:14.323192+0000","last_active":"2024-09-21T03:44:27.418241+0000","last_peered":"2024-09-21T03:44:27.418241+0000","last_clean":"2024-09-21T03:44:27.418241+0000","last_became_active":"2024-09-21T03:44:14.322487+0000","last_became_peered":"2024-09-21T03:44:14.322487+0000","last_unstale":"2024-09-21T03:44:27.418241+0000","last_undegraded":"2024-09-21T03:44:27.418241+0000","last_fullsized":"2024-09-21T03:44:27.418241+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_clean_scrub_stamp":"2024-09-21T03:44:13.292303+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:10:48.770536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:44:27.613320+0000","last_change":"2024-09-21T03:44:27.612681+0000","last_active":"2024-09-21T03:44:27.613320+0000","last_peered":"2024-09-21T03:44:27.613320+0000","last_clean":"2024-09-21T03:44:27.613320+0000","last_became_active":"2024-09-21T03:44:27.612200+0000","last_became_peered":"2024-09-21T03:44:27.612200+0000","last_unstale":"2024-09-21T03:44:27.613320+0000","last_undegraded":"2024-09-21T03:44:27.613320+0000","last_fullsized":"2024-09-21T03:44:27.613320+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_clean_scrub_stamp":"2024-09-21T03:44:13.292303+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:44:58.848571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,5,10],"acting":[9,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":18,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:44:27.414086+0000","last_change":"2024-09-21T03:44:14.325820+0000","last_active":"2024-09-21T03:44:27.414086+0000","last_peered":"2024-09-21T03:44:27.414086+0000","last_clean":"2024-09-21T03:44:27.414086+0000","last_became_active":"2024-09-21T03:44:14.325070+0000","last_became_peered":"2024-09-21T03:44:14.325070+0000","last_unstale":"2024-09-21T03:44:27.414086+0000","last_undegraded":"2024-09-21T03:44:27.414086+0000","last_fullsized":"2024-09-21T03:44:27.414086+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_clean_scrub_stamp":"2024-09-21T03:44:13.292303+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:54:01.072623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":18,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:44:27.612466+0000","last_change":"2024-09-21T03:44:14.323746+0000","last_active":"2024-09-21T03:44:27.612466+0000","last_peered":"2024-09-21T03:44:27.612466+0000","last_clean":"2024-09-21T03:44:27.612466+0000","last_became_active":"2024-09-21T03:44:14.323346+0000","last_became_peered":"2024-09-21T03:44:14.323346+0000","last_unstale":"2024-09-21T03:44:27.612466+0000","last_undegraded":"2024-09-21T03:44:27.612466+0000","last_fullsized":"2024-09-21T03:44:27.612466+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_clean_scrub_stamp":"2024-09-21T03:44:13.292303+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:17:44.603801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":18,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:44:27.419043+0000","last_change":"2024-09-21T03:44:14.328733+0000","last_active":"2024-09-21T03:44:27.419043+0000","last_peered":"2024-09-21T03:44:27.419043+0000","last_clean":"2024-09-21T03:44:27.419043+0000","last_became_active":"2024-09-21T03:44:14.328378+0000","last_became_peered":"2024-09-21T03:44:14.328378+0000","last_unstale":"2024-09-21T03:44:27.419043+0000","last_undegraded":"2024-09-21T03:44:27.419043+0000","last_fullsized":"2024-09-21T03:44:27.419043+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_clean_scrub_stamp":"2024-09-21T03:44:13.292303+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:23:38.497413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":18,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:44:27.613597+0000","last_change":"2024-09-21T03:44:14.328505+0000","last_active":"2024-09-21T03:44:27.613597+0000","last_peered":"2024-09-21T03:44:27.613597+0000","last_clean":"2024-09-21T03:44:27.613597+0000","last_became_active":"2024-09-21T03:44:14.328175+0000","last_became_peered":"2024-09-21T03:44:14.328175+0000","last_unstale":"2024-09-21T03:44:27.613597+0000","last_undegraded":"2024-09-21T03:44:27.613597+0000","last_fullsized":"2024-09-21T03:44:27.613597+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_clean_scrub_stamp":"2024-09-21T03:44:13.292303+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:45:56.628962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":18,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:44:27.410517+0000","last_change":"2024-09-21T03:44:14.314519+0000","last_active":"2024-09-21T03:44:27.410517+0000","last_peered":"2024-09-21T03:44:27.410517+0000","last_clean":"2024-09-21T03:44:27.410517+0000","last_became_active":"2024-09-21T03:44:14.314145+0000","last_became_peered":"2024-09-21T03:44:14.314145+0000","last_unstale":"2024-09-21T03:44:27.410517+0000","last_undegraded":"2024-09-21T03:44:27.410517+0000","last_fullsized":"2024-09-21T03:44:27.410517+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_clean_scrub_stamp":"2024-09-21T03:44:13.292303+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:45:48.494667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":18,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:44:27.416665+0000","last_change":"2024-09-21T03:44:14.312195+0000","last_active":"2024-09-21T03:44:27.416665+0000","last_peered":"2024-09-21T03:44:27.416665+0000","last_clean":"2024-09-21T03:44:27.416665+0000","last_became_active":"2024-09-21T03:44:14.311861+0000","last_became_peered":"2024-09-21T03:44:14.311861+0000","last_unstale":"2024-09-21T03:44:27.416665+0000","last_undegraded":"2024-09-21T03:44:27.416665+0000","last_fullsized":"2024-09-21T03:44:27.416665+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_clean_scrub_stamp":"2024-09-21T03:44:13.292303+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:17:05.025898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":18,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:44:27.416311+0000","last_change":"2024-09-21T03:44:14.323532+0000","last_active":"2024-09-21T03:44:27.416311+0000","last_peered":"2024-09-21T03:44:27.416311+0000","last_clean":"2024-09-21T03:44:27.416311+0000","last_became_active":"2024-09-21T03:44:14.323171+0000","last_became_peered":"2024-09-21T03:44:14.323171+0000","last_unstale":"2024-09-21T03:44:27.416311+0000","last_undegraded":"2024-09-21T03:44:27.416311+0000","last_fullsized":"2024-09-21T03:44:27.416311+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_clean_scrub_stamp":"2024-09-21T03:44:13.292303+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:23:28.999129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":18,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:44:27.413494+0000","last_change":"2024-09-21T03:44:14.324998+0000","last_active":"2024-09-21T03:44:27.413494+0000","last_peered":"2024-09-21T03:44:27.413494+0000","last_clean":"2024-09-21T03:44:27.413494+0000","last_became_active":"2024-09-21T03:44:14.324019+0000","last_became_peered":"2024-09-21T03:44:14.324019+0000","last_unstale":"2024-09-21T03:44:27.413494+0000","last_undegraded":"2024-09-21T03:44:27.413494+0000","last_fullsized":"2024-09-21T03:44:27.413494+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_clean_scrub_stamp":"2024-09-21T03:44:13.292303+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:49:12.965299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":18,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:44:27.421157+0000","last_change":"2024-09-21T03:44:14.325603+0000","last_active":"2024-09-21T03:44:27.421157+0000","last_peered":"2024-09-21T03:44:27.421157+0000","last_clean":"2024-09-21T03:44:27.421157+0000","last_became_active":"2024-09-21T03:44:14.325208+0000","last_became_peered":"2024-09-21T03:44:14.325208+0000","last_unstale":"2024-09-21T03:44:27.421157+0000","last_undegraded":"2024-09-21T03:44:27.421157+0000","last_fullsized":"2024-09-21T03:44:27.421157+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_clean_scrub_stamp":"2024-09-21T03:44:13.292303+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:11:33.794242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":18,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:44:27.414581+0000","last_change":"2024-09-21T03:44:14.317545+0000","last_active":"2024-09-21T03:44:27.414581+0000","last_peered":"2024-09-21T03:44:27.414581+0000","last_clean":"2024-09-21T03:44:27.414581+0000","last_became_active":"2024-09-21T03:44:14.317069+0000","last_became_peered":"2024-09-21T03:44:14.317069+0000","last_unstale":"2024-09-21T03:44:27.414581+0000","last_undegraded":"2024-09-21T03:44:27.414581+0000","last_fullsized":"2024-09-21T03:44:27.414581+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_clean_scrub_stamp":"2024-09-21T03:44:13.292303+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:16:11.734887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":18,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:44:27.418423+0000","last_change":"2024-09-21T03:44:14.322055+0000","last_active":"2024-09-21T03:44:27.418423+0000","last_peered":"2024-09-21T03:44:27.418423+0000","last_clean":"2024-09-21T03:44:27.418423+0000","last_became_active":"2024-09-21T03:44:14.321652+0000","last_became_peered":"2024-09-21T03:44:14.321652+0000","last_unstale":"2024-09-21T03:44:27.418423+0000","last_undegraded":"2024-09-21T03:44:27.418423+0000","last_fullsized":"2024-09-21T03:44:27.418423+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_clean_scrub_stamp":"2024-09-21T03:44:13.292303+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:45:14.285844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":25,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:44:27.423315+0000","last_change":"2024-09-21T03:44:27.423175+0000","last_active":"2024-09-21T03:44:27.423315+0000","last_peered":"2024-09-21T03:44:27.423315+0000","last_clean":"2024-09-21T03:44:27.423315+0000","last_became_active":"2024-09-21T03:44:27.422733+0000","last_became_peered":"2024-09-21T03:44:27.422733+0000","last_unstale":"2024-09-21T03:44:27.423315+0000","last_undegraded":"2024-09-21T03:44:27.423315+0000","last_fullsized":"2024-09-21T03:44:27.423315+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_clean_scrub_stamp":"2024-09-21T03:44:13.292303+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:46:17.874557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,8],"acting":[11,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":18,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:44:27.410602+0000","last_change":"2024-09-21T03:44:14.325829+0000","last_active":"2024-09-21T03:44:27.410602+0000","last_peered":"2024-09-21T03:44:27.410602+0000","last_clean":"2024-09-21T03:44:27.410602+0000","last_became_active":"2024-09-21T03:44:14.325421+0000","last_became_peered":"2024-09-21T03:44:14.325421+0000","last_unstale":"2024-09-21T03:44:27.410602+0000","last_undegraded":"2024-09-21T03:44:27.410602+0000","last_fullsized":"2024-09-21T03:44:27.410602+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_clean_scrub_stamp":"2024-09-21T03:44:13.292303+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:49:00.240771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":18,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:44:27.413691+0000","last_change":"2024-09-21T03:44:14.325255+0000","last_active":"2024-09-21T03:44:27.413691+0000","last_peered":"2024-09-21T03:44:27.413691+0000","last_clean":"2024-09-21T03:44:27.413691+0000","last_became_active":"2024-09-21T03:44:14.324617+0000","last_became_peered":"2024-09-21T03:44:14.324617+0000","last_unstale":"2024-09-21T03:44:27.413691+0000","last_undegraded":"2024-09-21T03:44:27.413691+0000","last_fullsized":"2024-09-21T03:44:27.413691+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_clean_scrub_stamp":"2024-09-21T03:44:13.292303+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:05:03.268522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":18,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:44:27.610583+0000","last_change":"2024-09-21T03:44:14.328053+0000","last_active":"2024-09-21T03:44:27.610583+0000","last_peered":"2024-09-21T03:44:27.610583+0000","last_clean":"2024-09-21T03:44:27.610583+0000","last_became_active":"2024-09-21T03:44:14.327735+0000","last_became_peered":"2024-09-21T03:44:14.327735+0000","last_unstale":"2024-09-21T03:44:27.610583+0000","last_undegraded":"2024-09-21T03:44:27.610583+0000","last_fullsized":"2024-09-21T03:44:27.610583+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_clean_scrub_stamp":"2024-09-21T03:44:13.292303+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:39:48.395784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":18,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:44:27.413859+0000","last_change":"2024-09-21T03:44:14.321181+0000","last_active":"2024-09-21T03:44:27.413859+0000","last_peered":"2024-09-21T03:44:27.413859+0000","last_clean":"2024-09-21T03:44:27.413859+0000","last_became_active":"2024-09-21T03:44:14.320772+0000","last_became_peered":"2024-09-21T03:44:14.320772+0000","last_unstale":"2024-09-21T03:44:27.413859+0000","last_undegraded":"2024-09-21T03:44:27.413859+0000","last_fullsized":"2024-09-21T03:44:27.413859+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_clean_scrub_stamp":"2024-09-21T03:44:13.292303+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:49:26.317404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":18,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:44:27.611124+0000","last_change":"2024-09-21T03:44:14.322643+0000","last_active":"2024-09-21T03:44:27.611124+0000","last_peered":"2024-09-21T03:44:27.611124+0000","last_clean":"2024-09-21T03:44:27.611124+0000","last_became_active":"2024-09-21T03:44:14.322254+0000","last_became_peered":"2024-09-21T03:44:14.322254+0000","last_unstale":"2024-09-21T03:44:27.611124+0000","last_undegraded":"2024-09-21T03:44:27.611124+0000","last_fullsized":"2024-09-21T03:44:27.611124+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_clean_scrub_stamp":"2024-09-21T03:44:13.292303+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:31:47.147845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":18,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:44:27.418815+0000","last_change":"2024-09-21T03:44:14.323090+0000","last_active":"2024-09-21T03:44:27.418815+0000","last_peered":"2024-09-21T03:44:27.418815+0000","last_clean":"2024-09-21T03:44:27.418815+0000","last_became_active":"2024-09-21T03:44:14.322669+0000","last_became_peered":"2024-09-21T03:44:14.322669+0000","last_unstale":"2024-09-21T03:44:27.418815+0000","last_undegraded":"2024-09-21T03:44:27.418815+0000","last_fullsized":"2024-09-21T03:44:27.418815+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_clean_scrub_stamp":"2024-09-21T03:44:13.292303+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:08:56.852422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":18,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:44:27.418948+0000","last_change":"2024-09-21T03:44:14.318146+0000","last_active":"2024-09-21T03:44:27.418948+0000","last_peered":"2024-09-21T03:44:27.418948+0000","last_clean":"2024-09-21T03:44:27.418948+0000","last_became_active":"2024-09-21T03:44:14.317761+0000","last_became_peered":"2024-09-21T03:44:14.317761+0000","last_unstale":"2024-09-21T03:44:27.418948+0000","last_undegraded":"2024-09-21T03:44:27.418948+0000","last_fullsized":"2024-09-21T03:44:27.418948+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_clean_scrub_stamp":"2024-09-21T03:44:13.292303+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:07:31.530728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":18,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:44:27.410569+0000","last_change":"2024-09-21T03:44:14.316832+0000","last_active":"2024-09-21T03:44:27.410569+0000","last_peered":"2024-09-21T03:44:27.410569+0000","last_clean":"2024-09-21T03:44:27.410569+0000","last_became_active":"2024-09-21T03:44:14.316471+0000","last_became_peered":"2024-09-21T03:44:14.316471+0000","last_unstale":"2024-09-21T03:44:27.410569+0000","last_undegraded":"2024-09-21T03:44:27.410569+0000","last_fullsized":"2024-09-21T03:44:27.410569+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_clean_scrub_stamp":"2024-09-21T03:44:13.292303+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:21:23.848778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":18,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:44:27.417002+0000","last_change":"2024-09-21T03:44:14.322277+0000","last_active":"2024-09-21T03:44:27.417002+0000","last_peered":"2024-09-21T03:44:27.417002+0000","last_clean":"2024-09-21T03:44:27.417002+0000","last_became_active":"2024-09-21T03:44:14.321952+0000","last_became_peered":"2024-09-21T03:44:14.321952+0000","last_unstale":"2024-09-21T03:44:27.417002+0000","last_undegraded":"2024-09-21T03:44:27.417002+0000","last_fullsized":"2024-09-21T03:44:27.417002+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_clean_scrub_stamp":"2024-09-21T03:44:13.292303+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:43:55.169380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":18,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:44:27.418939+0000","last_change":"2024-09-21T03:44:14.322626+0000","last_active":"2024-09-21T03:44:27.418939+0000","last_peered":"2024-09-21T03:44:27.418939+0000","last_clean":"2024-09-21T03:44:27.418939+0000","last_became_active":"2024-09-21T03:44:14.321559+0000","last_became_peered":"2024-09-21T03:44:14.321559+0000","last_unstale":"2024-09-21T03:44:27.418939+0000","last_undegraded":"2024-09-21T03:44:27.418939+0000","last_fullsized":"2024-09-21T03:44:27.418939+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_clean_scrub_stamp":"2024-09-21T03:44:13.292303+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:12:23.736500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":18,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:44:27.416654+0000","last_change":"2024-09-21T03:44:14.318057+0000","last_active":"2024-09-21T03:44:27.416654+0000","last_peered":"2024-09-21T03:44:27.416654+0000","last_clean":"2024-09-21T03:44:27.416654+0000","last_became_active":"2024-09-21T03:44:14.317653+0000","last_became_peered":"2024-09-21T03:44:14.317653+0000","last_unstale":"2024-09-21T03:44:27.416654+0000","last_undegraded":"2024-09-21T03:44:27.416654+0000","last_fullsized":"2024-09-21T03:44:27.416654+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_clean_scrub_stamp":"2024-09-21T03:44:13.292303+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:23:17.396599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":18,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:44:27.418421+0000","last_change":"2024-09-21T03:44:14.322331+0000","last_active":"2024-09-21T03:44:27.418421+0000","last_peered":"2024-09-21T03:44:27.418421+0000","last_clean":"2024-09-21T03:44:27.418421+0000","last_became_active":"2024-09-21T03:44:14.321157+0000","last_became_peered":"2024-09-21T03:44:14.321157+0000","last_unstale":"2024-09-21T03:44:27.418421+0000","last_undegraded":"2024-09-21T03:44:27.418421+0000","last_fullsized":"2024-09-21T03:44:27.418421+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_clean_scrub_stamp":"2024-09-21T03:44:13.292303+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:08:27.953449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":25,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:44:27.423182+0000","last_change":"2024-09-21T03:44:27.423081+0000","last_active":"2024-09-21T03:44:27.423182+0000","last_peered":"2024-09-21T03:44:27.423182+0000","last_clean":"2024-09-21T03:44:27.423182+0000","last_became_active":"2024-09-21T03:44:27.422650+0000","last_became_peered":"2024-09-21T03:44:27.422650+0000","last_unstale":"2024-09-21T03:44:27.423182+0000","last_undegraded":"2024-09-21T03:44:27.423182+0000","last_fullsized":"2024-09-21T03:44:27.423182+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_clean_scrub_stamp":"2024-09-21T03:44:13.292303+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:32:44.399954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,7],"acting":[8,9,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1b","version":"79'6","reported_seq":47,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:44:37.717899+0000","last_change":"2024-09-21T03:44:12.307464+0000","last_active":"2024-09-21T03:44:37.717899+0000","last_peered":"2024-09-21T03:44:37.717899+0000","last_clean":"2024-09-21T03:44:37.717899+0000","last_became_active":"2024-09-21T03:44:12.307077+0000","last_became_peered":"2024-09-21T03:44:12.307077+0000","last_unstale":"2024-09-21T03:44:37.717899+0000","last_undegraded":"2024-09-21T03:44:37.717899+0000","last_fullsized":"2024-09-21T03:44:37.717899+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:11.282438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:11.282438+0000","last_clean_scrub_stamp":"2024-09-21T03:44:11.282438+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:50:40.344659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1013,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":18,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":18,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:44:27.414062+0000","last_change":"2024-09-21T03:44:14.319072+0000","last_active":"2024-09-21T03:44:27.414062+0000","last_peered":"2024-09-21T03:44:27.414062+0000","last_clean":"2024-09-21T03:44:27.414062+0000","last_became_active":"2024-09-21T03:44:14.318712+0000","last_became_peered":"2024-09-21T03:44:14.318712+0000","last_unstale":"2024-09-21T03:44:27.414062+0000","last_undegraded":"2024-09-21T03:44:27.414062+0000","last_fullsized":"2024-09-21T03:44:27.414062+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_clean_scrub_stamp":"2024-09-21T03:44:13.292303+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:30:35.618110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1a","version":"77'7","reported_seq":30,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:44:27.611763+0000","last_change":"2024-09-21T03:44:12.310146+0000","last_active":"2024-09-21T03:44:27.611763+0000","last_peered":"2024-09-21T03:44:27.611763+0000","last_clean":"2024-09-21T03:44:27.611763+0000","last_became_active":"2024-09-21T03:44:12.309817+0000","last_became_peered":"2024-09-21T03:44:12.309817+0000","last_unstale":"2024-09-21T03:44:27.611763+0000","last_undegraded":"2024-09-21T03:44:27.611763+0000","last_fullsized":"2024-09-21T03:44:27.611763+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:11.282438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:11.282438+0000","last_clean_scrub_stamp":"2024-09-21T03:44:11.282438+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:46:51.994488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.19","version":"77'3","reported_seq":26,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:44:27.410211+0000","last_change":"2024-09-21T03:44:12.308995+0000","last_active":"2024-09-21T03:44:27.410211+0000","last_peered":"2024-09-21T03:44:27.410211+0000","last_clean":"2024-09-21T03:44:27.410211+0000","last_became_active":"2024-09-21T03:44:12.308058+0000","last_became_peered":"2024-09-21T03:44:12.308058+0000","last_unstale":"2024-09-21T03:44:27.410211+0000","last_undegraded":"2024-09-21T03:44:27.410211+0000","last_fullsized":"2024-09-21T03:44:27.410211+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:11.282438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:11.282438+0000","last_clean_scrub_stamp":"2024-09-21T03:44:11.282438+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:48:39.384985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":18,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:44:27.610583+0000","last_change":"2024-09-21T03:44:14.317878+0000","last_active":"2024-09-21T03:44:27.610583+0000","last_peered":"2024-09-21T03:44:27.610583+0000","last_clean":"2024-09-21T03:44:27.610583+0000","last_became_active":"2024-09-21T03:44:14.317521+0000","last_became_peered":"2024-09-21T03:44:14.317521+0000","last_unstale":"2024-09-21T03:44:27.610583+0000","last_undegraded":"2024-09-21T03:44:27.610583+0000","last_fullsized":"2024-09-21T03:44:27.610583+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_clean_scrub_stamp":"2024-09-21T03:44:13.292303+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:08:00.705210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"77'3","reported_seq":27,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:44:27.414041+0000","last_change":"2024-09-21T03:44:12.308777+0000","last_active":"2024-09-21T03:44:27.414041+0000","last_peered":"2024-09-21T03:44:27.414041+0000","last_clean":"2024-09-21T03:44:27.414041+0000","last_became_active":"2024-09-21T03:44:12.308171+0000","last_became_peered":"2024-09-21T03:44:12.308171+0000","last_unstale":"2024-09-21T03:44:27.414041+0000","last_undegraded":"2024-09-21T03:44:27.414041+0000","last_fullsized":"2024-09-21T03:44:27.414041+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:11.282438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:11.282438+0000","last_clean_scrub_stamp":"2024-09-21T03:44:11.282438+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:41:21.344491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":18,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:44:27.414180+0000","last_change":"2024-09-21T03:44:14.315782+0000","last_active":"2024-09-21T03:44:27.414180+0000","last_peered":"2024-09-21T03:44:27.414180+0000","last_clean":"2024-09-21T03:44:27.414180+0000","last_became_active":"2024-09-21T03:44:14.315314+0000","last_became_peered":"2024-09-21T03:44:14.315314+0000","last_unstale":"2024-09-21T03:44:27.414180+0000","last_undegraded":"2024-09-21T03:44:27.414180+0000","last_fullsized":"2024-09-21T03:44:27.414180+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_clean_scrub_stamp":"2024-09-21T03:44:13.292303+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:22:20.500220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"77'4","reported_seq":55,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:44:37.717074+0000","last_change":"2024-09-21T03:44:12.313251+0000","last_active":"2024-09-21T03:44:37.717074+0000","last_peered":"2024-09-21T03:44:37.717074+0000","last_clean":"2024-09-21T03:44:37.717074+0000","last_became_active":"2024-09-21T03:44:12.312941+0000","last_became_peered":"2024-09-21T03:44:12.312941+0000","last_unstale":"2024-09-21T03:44:37.717074+0000","last_undegraded":"2024-09-21T03:44:37.717074+0000","last_fullsized":"2024-09-21T03:44:37.717074+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:11.282438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:11.282438+0000","last_clean_scrub_stamp":"2024-09-21T03:44:11.282438+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:38:39.876097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":28,"num_read_kb":28,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":18,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:44:27.414153+0000","last_change":"2024-09-21T03:44:14.319621+0000","last_active":"2024-09-21T03:44:27.414153+0000","last_peered":"2024-09-21T03:44:27.414153+0000","last_clean":"2024-09-21T03:44:27.414153+0000","last_became_active":"2024-09-21T03:44:14.316173+0000","last_became_peered":"2024-09-21T03:44:14.316173+0000","last_unstale":"2024-09-21T03:44:27.414153+0000","last_undegraded":"2024-09-21T03:44:27.414153+0000","last_fullsized":"2024-09-21T03:44:27.414153+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_clean_scrub_stamp":"2024-09-21T03:44:13.292303+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:10:21.952989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"77'4","reported_seq":27,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:44:27.419175+0000","last_change":"2024-09-21T03:44:12.308893+0000","last_active":"2024-09-21T03:44:27.419175+0000","last_peered":"2024-09-21T03:44:27.419175+0000","last_clean":"2024-09-21T03:44:27.419175+0000","last_became_active":"2024-09-21T03:44:12.308551+0000","last_became_peered":"2024-09-21T03:44:12.308551+0000","last_unstale":"2024-09-21T03:44:27.419175+0000","last_undegraded":"2024-09-21T03:44:27.419175+0000","last_fullsized":"2024-09-21T03:44:27.419175+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:11.282438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:11.282438+0000","last_clean_scrub_stamp":"2024-09-21T03:44:11.282438+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:48:18.401595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":18,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:44:27.417391+0000","last_change":"2024-09-21T03:44:14.321875+0000","last_active":"2024-09-21T03:44:27.417391+0000","last_peered":"2024-09-21T03:44:27.417391+0000","last_clean":"2024-09-21T03:44:27.417391+0000","last_became_active":"2024-09-21T03:44:14.321483+0000","last_became_peered":"2024-09-21T03:44:14.321483+0000","last_unstale":"2024-09-21T03:44:27.417391+0000","last_undegraded":"2024-09-21T03:44:27.417391+0000","last_fullsized":"2024-09-21T03:44:27.417391+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_clean_scrub_stamp":"2024-09-21T03:44:13.292303+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:40:17.244780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"77'5","reported_seq":29,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:44:27.414461+0000","last_change":"2024-09-21T03:44:12.313208+0000","last_active":"2024-09-21T03:44:27.414461+0000","last_peered":"2024-09-21T03:44:27.414461+0000","last_clean":"2024-09-21T03:44:27.414461+0000","last_became_active":"2024-09-21T03:44:12.312783+0000","last_became_peered":"2024-09-21T03:44:12.312783+0000","last_unstale":"2024-09-21T03:44:27.414461+0000","last_undegraded":"2024-09-21T03:44:27.414461+0000","last_fullsized":"2024-09-21T03:44:27.414461+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:11.282438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:11.282438+0000","last_clean_scrub_stamp":"2024-09-21T03:44:11.282438+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:18:45.114361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":25,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:44:27.421219+0000","last_change":"2024-09-21T03:44:27.421129+0000","last_active":"2024-09-21T03:44:27.421219+0000","last_peered":"2024-09-21T03:44:27.421219+0000","last_clean":"2024-09-21T03:44:27.421219+0000","last_became_active":"2024-09-21T03:44:27.420642+0000","last_became_peered":"2024-09-21T03:44:27.420642+0000","last_unstale":"2024-09-21T03:44:27.421219+0000","last_undegraded":"2024-09-21T03:44:27.421219+0000","last_fullsized":"2024-09-21T03:44:27.421219+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_clean_scrub_stamp":"2024-09-21T03:44:13.292303+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:49:38.065743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,6],"acting":[1,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"79'24","reported_seq":81,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:44:37.725937+0000","last_change":"2024-09-21T03:44:12.314863+0000","last_active":"2024-09-21T03:44:37.725937+0000","last_peered":"2024-09-21T03:44:37.725937+0000","last_clean":"2024-09-21T03:44:37.725937+0000","last_became_active":"2024-09-21T03:44:12.314471+0000","last_became_peered":"2024-09-21T03:44:12.314471+0000","last_unstale":"2024-09-21T03:44:37.725937+0000","last_undegraded":"2024-09-21T03:44:37.725937+0000","last_fullsized":"2024-09-21T03:44:37.725937+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:11.282438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:11.282438+0000","last_clean_scrub_stamp":"2024-09-21T03:44:11.282438+0000","objects_scrubbed":0,"log_size":24,"log_dups_size":0,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:41:03.471226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":50940,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":34,"num_read_kb":66,"num_write":26,"num_write_kb":58,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":18,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:44:27.420428+0000","last_change":"2024-09-21T03:44:14.325471+0000","last_active":"2024-09-21T03:44:27.420428+0000","last_peered":"2024-09-21T03:44:27.420428+0000","last_clean":"2024-09-21T03:44:27.420428+0000","last_became_active":"2024-09-21T03:44:14.325072+0000","last_became_peered":"2024-09-21T03:44:14.325072+0000","last_unstale":"2024-09-21T03:44:27.420428+0000","last_undegraded":"2024-09-21T03:44:27.420428+0000","last_fullsized":"2024-09-21T03:44:27.420428+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_clean_scrub_stamp":"2024-09-21T03:44:13.292303+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:40:40.097240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"77'9","reported_seq":34,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:44:27.416778+0000","last_change":"2024-09-21T03:44:12.312932+0000","last_active":"2024-09-21T03:44:27.416778+0000","last_peered":"2024-09-21T03:44:27.416778+0000","last_clean":"2024-09-21T03:44:27.416778+0000","last_became_active":"2024-09-21T03:44:12.312368+0000","last_became_peered":"2024-09-21T03:44:12.312368+0000","last_unstale":"2024-09-21T03:44:27.416778+0000","last_undegraded":"2024-09-21T03:44:27.416778+0000","last_fullsized":"2024-09-21T03:44:27.416778+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:11.282438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:11.282438+0000","last_clean_scrub_stamp":"2024-09-21T03:44:11.282438+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:28:11.579598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":18,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:44:27.410310+0000","last_change":"2024-09-21T03:44:14.329564+0000","last_active":"2024-09-21T03:44:27.410310+0000","last_peered":"2024-09-21T03:44:27.410310+0000","last_clean":"2024-09-21T03:44:27.410310+0000","last_became_active":"2024-09-21T03:44:14.329122+0000","last_became_peered":"2024-09-21T03:44:14.329122+0000","last_unstale":"2024-09-21T03:44:27.410310+0000","last_undegraded":"2024-09-21T03:44:27.410310+0000","last_fullsized":"2024-09-21T03:44:27.410310+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_clean_scrub_stamp":"2024-09-21T03:44:13.292303+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:16:32.702770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"77'5","reported_seq":29,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:44:27.413875+0000","last_change":"2024-09-21T03:44:12.303339+0000","last_active":"2024-09-21T03:44:27.413875+0000","last_peered":"2024-09-21T03:44:27.413875+0000","last_clean":"2024-09-21T03:44:27.413875+0000","last_became_active":"2024-09-21T03:44:12.302905+0000","last_became_peered":"2024-09-21T03:44:12.302905+0000","last_unstale":"2024-09-21T03:44:27.413875+0000","last_undegraded":"2024-09-21T03:44:27.413875+0000","last_fullsized":"2024-09-21T03:44:27.413875+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:11.282438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:11.282438+0000","last_clean_scrub_stamp":"2024-09-21T03:44:11.282438+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:57:16.424976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":18,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:44:27.418798+0000","last_change":"2024-09-21T03:44:14.323473+0000","last_active":"2024-09-21T03:44:27.418798+0000","last_peered":"2024-09-21T03:44:27.418798+0000","last_clean":"2024-09-21T03:44:27.418798+0000","last_became_active":"2024-09-21T03:44:14.322560+0000","last_became_peered":"2024-09-21T03:44:14.322560+0000","last_unstale":"2024-09-21T03:44:27.418798+0000","last_undegraded":"2024-09-21T03:44:27.418798+0000","last_fullsized":"2024-09-21T03:44:27.418798+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_clean_scrub_stamp":"2024-09-21T03:44:13.292303+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:07:55.714351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"79'6","reported_seq":43,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:44:33.712347+0000","last_change":"2024-09-21T03:44:12.312331+0000","last_active":"2024-09-21T03:44:33.712347+0000","last_peered":"2024-09-21T03:44:33.712347+0000","last_clean":"2024-09-21T03:44:33.712347+0000","last_became_active":"2024-09-21T03:44:12.311984+0000","last_became_peered":"2024-09-21T03:44:12.311984+0000","last_unstale":"2024-09-21T03:44:33.712347+0000","last_undegraded":"2024-09-21T03:44:33.712347+0000","last_fullsized":"2024-09-21T03:44:33.712347+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:11.282438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:11.282438+0000","last_clean_scrub_stamp":"2024-09-21T03:44:11.282438+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:06:11.906592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":979,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":15,"num_read_kb":4,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":18,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:44:27.418181+0000","last_change":"2024-09-21T03:44:14.318437+0000","last_active":"2024-09-21T03:44:27.418181+0000","last_peered":"2024-09-21T03:44:27.418181+0000","last_clean":"2024-09-21T03:44:27.418181+0000","last_became_active":"2024-09-21T03:44:14.318052+0000","last_became_peered":"2024-09-21T03:44:14.318052+0000","last_unstale":"2024-09-21T03:44:27.418181+0000","last_undegraded":"2024-09-21T03:44:27.418181+0000","last_fullsized":"2024-09-21T03:44:27.418181+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_clean_scrub_stamp":"2024-09-21T03:44:13.292303+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:10:12.136159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"77'17","reported_seq":41,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:44:27.419831+0000","last_change":"2024-09-21T03:44:12.310616+0000","last_active":"2024-09-21T03:44:27.419831+0000","last_peered":"2024-09-21T03:44:27.419831+0000","last_clean":"2024-09-21T03:44:27.419831+0000","last_became_active":"2024-09-21T03:44:12.310239+0000","last_became_peered":"2024-09-21T03:44:12.310239+0000","last_unstale":"2024-09-21T03:44:27.419831+0000","last_undegraded":"2024-09-21T03:44:27.419831+0000","last_fullsized":"2024-09-21T03:44:27.419831+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:11.282438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:11.282438+0000","last_clean_scrub_stamp":"2024-09-21T03:44:11.282438+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:43:38.046281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":18,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:44:27.413822+0000","last_change":"2024-09-21T03:44:14.327058+0000","last_active":"2024-09-21T03:44:27.413822+0000","last_peered":"2024-09-21T03:44:27.413822+0000","last_clean":"2024-09-21T03:44:27.413822+0000","last_became_active":"2024-09-21T03:44:14.326641+0000","last_became_peered":"2024-09-21T03:44:14.326641+0000","last_unstale":"2024-09-21T03:44:27.413822+0000","last_undegraded":"2024-09-21T03:44:27.413822+0000","last_fullsized":"2024-09-21T03:44:27.413822+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_clean_scrub_stamp":"2024-09-21T03:44:13.292303+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:10:50.686102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"77'2","reported_seq":26,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:44:27.420648+0000","last_change":"2024-09-21T03:44:12.310426+0000","last_active":"2024-09-21T03:44:27.420648+0000","last_peered":"2024-09-21T03:44:27.420648+0000","last_clean":"2024-09-21T03:44:27.420648+0000","last_became_active":"2024-09-21T03:44:12.309231+0000","last_became_peered":"2024-09-21T03:44:12.309231+0000","last_unstale":"2024-09-21T03:44:27.420648+0000","last_undegraded":"2024-09-21T03:44:27.420648+0000","last_fullsized":"2024-09-21T03:44:27.420648+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:11.282438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:11.282438+0000","last_clean_scrub_stamp":"2024-09-21T03:44:11.282438+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:32:18.284741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":18,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:44:27.420390+0000","last_change":"2024-09-21T03:44:14.319179+0000","last_active":"2024-09-21T03:44:27.420390+0000","last_peered":"2024-09-21T03:44:27.420390+0000","last_clean":"2024-09-21T03:44:27.420390+0000","last_became_active":"2024-09-21T03:44:14.318826+0000","last_became_peered":"2024-09-21T03:44:14.318826+0000","last_unstale":"2024-09-21T03:44:27.420390+0000","last_undegraded":"2024-09-21T03:44:27.420390+0000","last_fullsized":"2024-09-21T03:44:27.420390+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_clean_scrub_stamp":"2024-09-21T03:44:13.292303+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:12:57.985087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.e","version":"77'5","reported_seq":28,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:44:27.410705+0000","last_change":"2024-09-21T03:44:12.315658+0000","last_active":"2024-09-21T03:44:27.410705+0000","last_peered":"2024-09-21T03:44:27.410705+0000","last_clean":"2024-09-21T03:44:27.410705+0000","last_became_active":"2024-09-21T03:44:12.315240+0000","last_became_peered":"2024-09-21T03:44:12.315240+0000","last_unstale":"2024-09-21T03:44:27.410705+0000","last_undegraded":"2024-09-21T03:44:27.410705+0000","last_fullsized":"2024-09-21T03:44:27.410705+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:11.282438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:11.282438+0000","last_clean_scrub_stamp":"2024-09-21T03:44:11.282438+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:22:07.493437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":18,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:44:27.419234+0000","last_change":"2024-09-21T03:44:14.328014+0000","last_active":"2024-09-21T03:44:27.419234+0000","last_peered":"2024-09-21T03:44:27.419234+0000","last_clean":"2024-09-21T03:44:27.419234+0000","last_became_active":"2024-09-21T03:44:14.327638+0000","last_became_peered":"2024-09-21T03:44:14.327638+0000","last_unstale":"2024-09-21T03:44:27.419234+0000","last_undegraded":"2024-09-21T03:44:27.419234+0000","last_fullsized":"2024-09-21T03:44:27.419234+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_clean_scrub_stamp":"2024-09-21T03:44:13.292303+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:48:32.542776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"77'3","reported_seq":27,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:44:27.414656+0000","last_change":"2024-09-21T03:44:12.309272+0000","last_active":"2024-09-21T03:44:27.414656+0000","last_peered":"2024-09-21T03:44:27.414656+0000","last_clean":"2024-09-21T03:44:27.414656+0000","last_became_active":"2024-09-21T03:44:12.308718+0000","last_became_peered":"2024-09-21T03:44:12.308718+0000","last_unstale":"2024-09-21T03:44:27.414656+0000","last_undegraded":"2024-09-21T03:44:27.414656+0000","last_fullsized":"2024-09-21T03:44:27.414656+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:11.282438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:11.282438+0000","last_clean_scrub_stamp":"2024-09-21T03:44:11.282438+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:03:49.433966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"79'1","reported_seq":19,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:44:27.418909+0000","last_change":"2024-09-21T03:44:14.322148+0000","last_active":"2024-09-21T03:44:27.418909+0000","last_peered":"2024-09-21T03:44:27.418909+0000","last_clean":"2024-09-21T03:44:27.418909+0000","last_became_active":"2024-09-21T03:44:14.321784+0000","last_became_peered":"2024-09-21T03:44:14.321784+0000","last_unstale":"2024-09-21T03:44:27.418909+0000","last_undegraded":"2024-09-21T03:44:27.418909+0000","last_fullsized":"2024-09-21T03:44:27.418909+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_clean_scrub_stamp":"2024-09-21T03:44:13.292303+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:40:34.887134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"77'5","reported_seq":32,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:44:37.439633+0000","last_change":"2024-09-21T03:44:27.427722+0000","last_active":"2024-09-21T03:44:37.439633+0000","last_peered":"2024-09-21T03:44:37.439633+0000","last_clean":"2024-09-21T03:44:37.439633+0000","last_became_active":"2024-09-21T03:44:27.419672+0000","last_became_peered":"2024-09-21T03:44:27.419672+0000","last_unstale":"2024-09-21T03:44:37.439633+0000","last_undegraded":"2024-09-21T03:44:37.439633+0000","last_fullsized":"2024-09-21T03:44:37.439633+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:11.282438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:11.282438+0000","last_clean_scrub_stamp":"2024-09-21T03:44:11.282438+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:24:31.890439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1170,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":12,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":1170,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,11,0],"acting":[1,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":18,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:44:27.419112+0000","last_change":"2024-09-21T03:44:14.316503+0000","last_active":"2024-09-21T03:44:27.419112+0000","last_peered":"2024-09-21T03:44:27.419112+0000","last_clean":"2024-09-21T03:44:27.419112+0000","last_became_active":"2024-09-21T03:44:14.316140+0000","last_became_peered":"2024-09-21T03:44:14.316140+0000","last_unstale":"2024-09-21T03:44:27.419112+0000","last_undegraded":"2024-09-21T03:44:27.419112+0000","last_fullsized":"2024-09-21T03:44:27.419112+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_clean_scrub_stamp":"2024-09-21T03:44:13.292303+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:00:30.647579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"21'92","reported_seq":96,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:44:27.418315+0000","last_change":"2024-09-21T03:43:01.568120+0000","last_active":"2024-09-21T03:44:27.418315+0000","last_peered":"2024-09-21T03:44:27.418315+0000","last_clean":"2024-09-21T03:44:27.418315+0000","last_became_active":"2024-09-21T03:43:01.476311+0000","last_became_peered":"2024-09-21T03:43:01.476311+0000","last_unstale":"2024-09-21T03:44:27.418315+0000","last_undegraded":"2024-09-21T03:44:27.418315+0000","last_fullsized":"2024-09-21T03:44:27.418315+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:39:30.121380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:39:30.121380+0000","last_clean_scrub_stamp":"2024-09-21T03:39:30.121380+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:40:27.967442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"77'6","reported_seq":32,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:44:27.613250+0000","last_change":"2024-09-21T03:44:12.311860+0000","last_active":"2024-09-21T03:44:27.613250+0000","last_peered":"2024-09-21T03:44:27.613250+0000","last_clean":"2024-09-21T03:44:27.613250+0000","last_became_active":"2024-09-21T03:44:12.311453+0000","last_became_peered":"2024-09-21T03:44:12.311453+0000","last_unstale":"2024-09-21T03:44:27.613250+0000","last_undegraded":"2024-09-21T03:44:27.613250+0000","last_fullsized":"2024-09-21T03:44:27.613250+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:11.282438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:11.282438+0000","last_clean_scrub_stamp":"2024-09-21T03:44:11.282438+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:23:40.550445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:44:27.418319+0000","last_change":"2024-09-21T03:44:27.416799+0000","last_active":"2024-09-21T03:44:27.418319+0000","last_peered":"2024-09-21T03:44:27.418319+0000","last_clean":"2024-09-21T03:44:27.418319+0000","last_became_active":"2024-09-21T03:44:27.416306+0000","last_became_peered":"2024-09-21T03:44:27.416306+0000","last_unstale":"2024-09-21T03:44:27.418319+0000","last_undegraded":"2024-09-21T03:44:27.418319+0000","last_fullsized":"2024-09-21T03:44:27.418319+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_clean_scrub_stamp":"2024-09-21T03:44:13.292303+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:34:26.547100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,5,1],"acting":[9,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.0","version":"77'4","reported_seq":27,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:44:27.418313+0000","last_change":"2024-09-21T03:44:12.307302+0000","last_active":"2024-09-21T03:44:27.418313+0000","last_peered":"2024-09-21T03:44:27.418313+0000","last_clean":"2024-09-21T03:44:27.418313+0000","last_became_active":"2024-09-21T03:44:12.306856+0000","last_became_peered":"2024-09-21T03:44:12.306856+0000","last_unstale":"2024-09-21T03:44:27.418313+0000","last_undegraded":"2024-09-21T03:44:27.418313+0000","last_fullsized":"2024-09-21T03:44:27.418313+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:11.282438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:11.282438+0000","last_clean_scrub_stamp":"2024-09-21T03:44:11.282438+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:21:20.420020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":18,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:44:27.410210+0000","last_change":"2024-09-21T03:44:14.322913+0000","last_active":"2024-09-21T03:44:27.410210+0000","last_peered":"2024-09-21T03:44:27.410210+0000","last_clean":"2024-09-21T03:44:27.410210+0000","last_became_active":"2024-09-21T03:44:14.322168+0000","last_became_peered":"2024-09-21T03:44:14.322168+0000","last_unstale":"2024-09-21T03:44:27.410210+0000","last_undegraded":"2024-09-21T03:44:27.410210+0000","last_fullsized":"2024-09-21T03:44:27.410210+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_clean_scrub_stamp":"2024-09-21T03:44:13.292303+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:30:09.041211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"77'5","reported_seq":42,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:44:35.714736+0000","last_change":"2024-09-21T03:44:12.309084+0000","last_active":"2024-09-21T03:44:35.714736+0000","last_peered":"2024-09-21T03:44:35.714736+0000","last_clean":"2024-09-21T03:44:35.714736+0000","last_became_active":"2024-09-21T03:44:12.308551+0000","last_became_peered":"2024-09-21T03:44:12.308551+0000","last_unstale":"2024-09-21T03:44:35.714736+0000","last_undegraded":"2024-09-21T03:44:35.714736+0000","last_fullsized":"2024-09-21T03:44:35.714736+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:11.282438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:11.282438+0000","last_clean_scrub_stamp":"2024-09-21T03:44:11.282438+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:46:33.895904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":14,"num_read_kb":14,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":18,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:44:27.413911+0000","last_change":"2024-09-21T03:44:14.322720+0000","last_active":"2024-09-21T03:44:27.413911+0000","last_peered":"2024-09-21T03:44:27.413911+0000","last_clean":"2024-09-21T03:44:27.413911+0000","last_became_active":"2024-09-21T03:44:14.322367+0000","last_became_peered":"2024-09-21T03:44:14.322367+0000","last_unstale":"2024-09-21T03:44:27.413911+0000","last_undegraded":"2024-09-21T03:44:27.413911+0000","last_fullsized":"2024-09-21T03:44:27.413911+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_clean_scrub_stamp":"2024-09-21T03:44:13.292303+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:59:51.692976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"79'6","reported_seq":44,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:44:37.630380+0000","last_change":"2024-09-21T03:44:12.308719+0000","last_active":"2024-09-21T03:44:37.630380+0000","last_peered":"2024-09-21T03:44:37.630380+0000","last_clean":"2024-09-21T03:44:37.630380+0000","last_became_active":"2024-09-21T03:44:12.308380+0000","last_became_peered":"2024-09-21T03:44:12.308380+0000","last_unstale":"2024-09-21T03:44:37.630380+0000","last_undegraded":"2024-09-21T03:44:37.630380+0000","last_fullsized":"2024-09-21T03:44:37.630380+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:11.282438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:11.282438+0000","last_clean_scrub_stamp":"2024-09-21T03:44:11.282438+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:41:02.087501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":979,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":15,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":18,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:44:27.612118+0000","last_change":"2024-09-21T03:44:14.326541+0000","last_active":"2024-09-21T03:44:27.612118+0000","last_peered":"2024-09-21T03:44:27.612118+0000","last_clean":"2024-09-21T03:44:27.612118+0000","last_became_active":"2024-09-21T03:44:14.326132+0000","last_became_peered":"2024-09-21T03:44:14.326132+0000","last_unstale":"2024-09-21T03:44:27.612118+0000","last_undegraded":"2024-09-21T03:44:27.612118+0000","last_fullsized":"2024-09-21T03:44:27.612118+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_clean_scrub_stamp":"2024-09-21T03:44:13.292303+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:09:27.711919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"77'1","reported_seq":24,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:44:27.414339+0000","last_change":"2024-09-21T03:44:12.309463+0000","last_active":"2024-09-21T03:44:27.414339+0000","last_peered":"2024-09-21T03:44:27.414339+0000","last_clean":"2024-09-21T03:44:27.414339+0000","last_became_active":"2024-09-21T03:44:12.309054+0000","last_became_peered":"2024-09-21T03:44:12.309054+0000","last_unstale":"2024-09-21T03:44:27.414339+0000","last_undegraded":"2024-09-21T03:44:27.414339+0000","last_fullsized":"2024-09-21T03:44:27.414339+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:11.282438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:11.282438+0000","last_clean_scrub_stamp":"2024-09-21T03:44:11.282438+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:12:12.505538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"79'1","reported_seq":19,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:44:27.612000+0000","last_change":"2024-09-21T03:44:14.318261+0000","last_active":"2024-09-21T03:44:27.612000+0000","last_peered":"2024-09-21T03:44:27.612000+0000","last_clean":"2024-09-21T03:44:27.612000+0000","last_became_active":"2024-09-21T03:44:14.317839+0000","last_became_peered":"2024-09-21T03:44:14.317839+0000","last_unstale":"2024-09-21T03:44:27.612000+0000","last_undegraded":"2024-09-21T03:44:27.612000+0000","last_fullsized":"2024-09-21T03:44:27.612000+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_clean_scrub_stamp":"2024-09-21T03:44:13.292303+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:19:05.504749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.5","version":"77'4","reported_seq":20,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:44:27.434085+0000","last_change":"2024-09-21T03:44:27.433630+0000","last_active":"2024-09-21T03:44:27.434085+0000","last_peered":"2024-09-21T03:44:27.434085+0000","last_clean":"2024-09-21T03:44:27.434085+0000","last_became_active":"2024-09-21T03:44:27.422093+0000","last_became_peered":"2024-09-21T03:44:27.422093+0000","last_unstale":"2024-09-21T03:44:27.434085+0000","last_undegraded":"2024-09-21T03:44:27.434085+0000","last_fullsized":"2024-09-21T03:44:27.434085+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:11.282438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:11.282438+0000","last_clean_scrub_stamp":"2024-09-21T03:44:11.282438+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:39:42.651024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,9],"acting":[11,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":18,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:44:27.410677+0000","last_change":"2024-09-21T03:44:14.322620+0000","last_active":"2024-09-21T03:44:27.410677+0000","last_peered":"2024-09-21T03:44:27.410677+0000","last_clean":"2024-09-21T03:44:27.410677+0000","last_became_active":"2024-09-21T03:44:14.322214+0000","last_became_peered":"2024-09-21T03:44:14.322214+0000","last_unstale":"2024-09-21T03:44:27.410677+0000","last_undegraded":"2024-09-21T03:44:27.410677+0000","last_fullsized":"2024-09-21T03:44:27.410677+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_clean_scrub_stamp":"2024-09-21T03:44:13.292303+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:53:20.519168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"79'10","reported_seq":33,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:44:30.339054+0000","last_change":"2024-09-21T03:44:12.311302+0000","last_active":"2024-09-21T03:44:30.339054+0000","last_peered":"2024-09-21T03:44:30.339054+0000","last_clean":"2024-09-21T03:44:30.339054+0000","last_became_active":"2024-09-21T03:44:12.310947+0000","last_became_peered":"2024-09-21T03:44:12.310947+0000","last_unstale":"2024-09-21T03:44:30.339054+0000","last_undegraded":"2024-09-21T03:44:30.339054+0000","last_fullsized":"2024-09-21T03:44:30.339054+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:11.282438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:11.282438+0000","last_clean_scrub_stamp":"2024-09-21T03:44:11.282438+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:09:34.308726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":18,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:44:27.611222+0000","last_change":"2024-09-21T03:44:14.324675+0000","last_active":"2024-09-21T03:44:27.611222+0000","last_peered":"2024-09-21T03:44:27.611222+0000","last_clean":"2024-09-21T03:44:27.611222+0000","last_became_active":"2024-09-21T03:44:14.324214+0000","last_became_peered":"2024-09-21T03:44:14.324214+0000","last_unstale":"2024-09-21T03:44:27.611222+0000","last_undegraded":"2024-09-21T03:44:27.611222+0000","last_fullsized":"2024-09-21T03:44:27.611222+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_clean_scrub_stamp":"2024-09-21T03:44:13.292303+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:33:51.071794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"77'7","reported_seq":31,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:44:27.613637+0000","last_change":"2024-09-21T03:44:12.322312+0000","last_active":"2024-09-21T03:44:27.613637+0000","last_peered":"2024-09-21T03:44:27.613637+0000","last_clean":"2024-09-21T03:44:27.613637+0000","last_became_active":"2024-09-21T03:44:12.322018+0000","last_became_peered":"2024-09-21T03:44:12.322018+0000","last_unstale":"2024-09-21T03:44:27.613637+0000","last_undegraded":"2024-09-21T03:44:27.613637+0000","last_fullsized":"2024-09-21T03:44:27.613637+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:11.282438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:11.282438+0000","last_clean_scrub_stamp":"2024-09-21T03:44:11.282438+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:11:41.817574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":18,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:44:27.417254+0000","last_change":"2024-09-21T03:44:14.326682+0000","last_active":"2024-09-21T03:44:27.417254+0000","last_peered":"2024-09-21T03:44:27.417254+0000","last_clean":"2024-09-21T03:44:27.417254+0000","last_became_active":"2024-09-21T03:44:14.326313+0000","last_became_peered":"2024-09-21T03:44:14.326313+0000","last_unstale":"2024-09-21T03:44:27.417254+0000","last_undegraded":"2024-09-21T03:44:27.417254+0000","last_fullsized":"2024-09-21T03:44:27.417254+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_clean_scrub_stamp":"2024-09-21T03:44:13.292303+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:26:44.681245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"77'2","reported_seq":36,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:44:27.436319+0000","last_change":"2024-09-21T03:44:27.436245+0000","last_active":"2024-09-21T03:44:27.436319+0000","last_peered":"2024-09-21T03:44:27.436319+0000","last_clean":"2024-09-21T03:44:27.436319+0000","last_became_active":"2024-09-21T03:44:27.422134+0000","last_became_peered":"2024-09-21T03:44:27.422134+0000","last_unstale":"2024-09-21T03:44:27.436319+0000","last_undegraded":"2024-09-21T03:44:27.436319+0000","last_fullsized":"2024-09-21T03:44:27.436319+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:11.282438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:11.282438+0000","last_clean_scrub_stamp":"2024-09-21T03:44:11.282438+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:50:22.655050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,11,0],"acting":[1,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":18,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:44:27.419704+0000","last_change":"2024-09-21T03:44:14.316312+0000","last_active":"2024-09-21T03:44:27.419704+0000","last_peered":"2024-09-21T03:44:27.419704+0000","last_clean":"2024-09-21T03:44:27.419704+0000","last_became_active":"2024-09-21T03:44:14.315929+0000","last_became_peered":"2024-09-21T03:44:14.315929+0000","last_unstale":"2024-09-21T03:44:27.419704+0000","last_undegraded":"2024-09-21T03:44:27.419704+0000","last_fullsized":"2024-09-21T03:44:27.419704+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_clean_scrub_stamp":"2024-09-21T03:44:13.292303+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:31:33.071983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"77'7","reported_seq":30,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:44:27.610761+0000","last_change":"2024-09-21T03:44:12.312655+0000","last_active":"2024-09-21T03:44:27.610761+0000","last_peered":"2024-09-21T03:44:27.610761+0000","last_clean":"2024-09-21T03:44:27.610761+0000","last_became_active":"2024-09-21T03:44:12.312196+0000","last_became_peered":"2024-09-21T03:44:12.312196+0000","last_unstale":"2024-09-21T03:44:27.610761+0000","last_undegraded":"2024-09-21T03:44:27.610761+0000","last_fullsized":"2024-09-21T03:44:27.610761+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:11.282438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:11.282438+0000","last_clean_scrub_stamp":"2024-09-21T03:44:11.282438+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:24:32.342461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":18,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:44:27.418552+0000","last_change":"2024-09-21T03:44:14.321875+0000","last_active":"2024-09-21T03:44:27.418552+0000","last_peered":"2024-09-21T03:44:27.418552+0000","last_clean":"2024-09-21T03:44:27.418552+0000","last_became_active":"2024-09-21T03:44:14.320682+0000","last_became_peered":"2024-09-21T03:44:14.320682+0000","last_unstale":"2024-09-21T03:44:27.418552+0000","last_undegraded":"2024-09-21T03:44:27.418552+0000","last_fullsized":"2024-09-21T03:44:27.418552+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_clean_scrub_stamp":"2024-09-21T03:44:13.292303+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:41:32.435288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"77'3","reported_seq":26,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:44:27.418692+0000","last_change":"2024-09-21T03:44:12.309100+0000","last_active":"2024-09-21T03:44:27.418692+0000","last_peered":"2024-09-21T03:44:27.418692+0000","last_clean":"2024-09-21T03:44:27.418692+0000","last_became_active":"2024-09-21T03:44:12.308671+0000","last_became_peered":"2024-09-21T03:44:12.308671+0000","last_unstale":"2024-09-21T03:44:27.418692+0000","last_undegraded":"2024-09-21T03:44:27.418692+0000","last_fullsized":"2024-09-21T03:44:27.418692+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:11.282438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:11.282438+0000","last_clean_scrub_stamp":"2024-09-21T03:44:11.282438+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:09:19.347980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":18,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:44:27.610670+0000","last_change":"2024-09-21T03:44:14.320170+0000","last_active":"2024-09-21T03:44:27.610670+0000","last_peered":"2024-09-21T03:44:27.610670+0000","last_clean":"2024-09-21T03:44:27.610670+0000","last_became_active":"2024-09-21T03:44:14.319794+0000","last_became_peered":"2024-09-21T03:44:14.319794+0000","last_unstale":"2024-09-21T03:44:27.610670+0000","last_undegraded":"2024-09-21T03:44:27.610670+0000","last_fullsized":"2024-09-21T03:44:27.610670+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_clean_scrub_stamp":"2024-09-21T03:44:13.292303+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:42:09.684471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"77'4","reported_seq":28,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:44:27.420077+0000","last_change":"2024-09-21T03:44:12.310851+0000","last_active":"2024-09-21T03:44:27.420077+0000","last_peered":"2024-09-21T03:44:27.420077+0000","last_clean":"2024-09-21T03:44:27.420077+0000","last_became_active":"2024-09-21T03:44:12.310302+0000","last_became_peered":"2024-09-21T03:44:12.310302+0000","last_unstale":"2024-09-21T03:44:27.420077+0000","last_undegraded":"2024-09-21T03:44:27.420077+0000","last_fullsized":"2024-09-21T03:44:27.420077+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:11.282438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:11.282438+0000","last_clean_scrub_stamp":"2024-09-21T03:44:11.282438+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:23:46.959121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":18,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:44:27.418894+0000","last_change":"2024-09-21T03:44:14.324544+0000","last_active":"2024-09-21T03:44:27.418894+0000","last_peered":"2024-09-21T03:44:27.418894+0000","last_clean":"2024-09-21T03:44:27.418894+0000","last_became_active":"2024-09-21T03:44:14.322881+0000","last_became_peered":"2024-09-21T03:44:14.322881+0000","last_unstale":"2024-09-21T03:44:27.418894+0000","last_undegraded":"2024-09-21T03:44:27.418894+0000","last_fullsized":"2024-09-21T03:44:27.418894+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_clean_scrub_stamp":"2024-09-21T03:44:13.292303+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:58:42.760185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":25,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:44:27.422566+0000","last_change":"2024-09-21T03:44:27.422438+0000","last_active":"2024-09-21T03:44:27.422566+0000","last_peered":"2024-09-21T03:44:27.422566+0000","last_clean":"2024-09-21T03:44:27.422566+0000","last_became_active":"2024-09-21T03:44:27.421935+0000","last_became_peered":"2024-09-21T03:44:27.421935+0000","last_unstale":"2024-09-21T03:44:27.422566+0000","last_undegraded":"2024-09-21T03:44:27.422566+0000","last_fullsized":"2024-09-21T03:44:27.422566+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_clean_scrub_stamp":"2024-09-21T03:44:13.292303+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:20:50.097783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,1],"acting":[5,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"77'5","reported_seq":29,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:44:27.420510+0000","last_change":"2024-09-21T03:44:12.311509+0000","last_active":"2024-09-21T03:44:27.420510+0000","last_peered":"2024-09-21T03:44:27.420510+0000","last_clean":"2024-09-21T03:44:27.420510+0000","last_became_active":"2024-09-21T03:44:12.310880+0000","last_became_peered":"2024-09-21T03:44:12.310880+0000","last_unstale":"2024-09-21T03:44:27.420510+0000","last_undegraded":"2024-09-21T03:44:27.420510+0000","last_fullsized":"2024-09-21T03:44:27.420510+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:11.282438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:11.282438+0000","last_clean_scrub_stamp":"2024-09-21T03:44:11.282438+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:59:33.887234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":18,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:44:27.413781+0000","last_change":"2024-09-21T03:44:14.330938+0000","last_active":"2024-09-21T03:44:27.413781+0000","last_peered":"2024-09-21T03:44:27.413781+0000","last_clean":"2024-09-21T03:44:27.413781+0000","last_became_active":"2024-09-21T03:44:14.330559+0000","last_became_peered":"2024-09-21T03:44:14.330559+0000","last_unstale":"2024-09-21T03:44:27.413781+0000","last_undegraded":"2024-09-21T03:44:27.413781+0000","last_fullsized":"2024-09-21T03:44:27.413781+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_clean_scrub_stamp":"2024-09-21T03:44:13.292303+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:05:32.535660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"77'12","reported_seq":36,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:44:27.420563+0000","last_change":"2024-09-21T03:44:12.307667+0000","last_active":"2024-09-21T03:44:27.420563+0000","last_peered":"2024-09-21T03:44:27.420563+0000","last_clean":"2024-09-21T03:44:27.420563+0000","last_became_active":"2024-09-21T03:44:12.307074+0000","last_became_peered":"2024-09-21T03:44:12.307074+0000","last_unstale":"2024-09-21T03:44:27.420563+0000","last_undegraded":"2024-09-21T03:44:27.420563+0000","last_fullsized":"2024-09-21T03:44:27.420563+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:11.282438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:11.282438+0000","last_clean_scrub_stamp":"2024-09-21T03:44:11.282438+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:48:41.598375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":18,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:44:27.417322+0000","last_change":"2024-09-21T03:44:14.325487+0000","last_active":"2024-09-21T03:44:27.417322+0000","last_peered":"2024-09-21T03:44:27.417322+0000","last_clean":"2024-09-21T03:44:27.417322+0000","last_became_active":"2024-09-21T03:44:14.325153+0000","last_became_peered":"2024-09-21T03:44:14.325153+0000","last_unstale":"2024-09-21T03:44:27.417322+0000","last_undegraded":"2024-09-21T03:44:27.417322+0000","last_fullsized":"2024-09-21T03:44:27.417322+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_clean_scrub_stamp":"2024-09-21T03:44:13.292303+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:29:17.619130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"77'2","reported_seq":25,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:44:27.611241+0000","last_change":"2024-09-21T03:44:12.307635+0000","last_active":"2024-09-21T03:44:27.611241+0000","last_peered":"2024-09-21T03:44:27.611241+0000","last_clean":"2024-09-21T03:44:27.611241+0000","last_became_active":"2024-09-21T03:44:12.307235+0000","last_became_peered":"2024-09-21T03:44:12.307235+0000","last_unstale":"2024-09-21T03:44:27.611241+0000","last_undegraded":"2024-09-21T03:44:27.611241+0000","last_fullsized":"2024-09-21T03:44:27.611241+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:11.282438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:11.282438+0000","last_clean_scrub_stamp":"2024-09-21T03:44:11.282438+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:18:59.985369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":18,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:44:27.611117+0000","last_change":"2024-09-21T03:44:14.324660+0000","last_active":"2024-09-21T03:44:27.611117+0000","last_peered":"2024-09-21T03:44:27.611117+0000","last_clean":"2024-09-21T03:44:27.611117+0000","last_became_active":"2024-09-21T03:44:14.324309+0000","last_became_peered":"2024-09-21T03:44:14.324309+0000","last_unstale":"2024-09-21T03:44:27.611117+0000","last_undegraded":"2024-09-21T03:44:27.611117+0000","last_fullsized":"2024-09-21T03:44:27.611117+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_clean_scrub_stamp":"2024-09-21T03:44:13.292303+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:13:07.806721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"77'13","reported_seq":67,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:44:33.624020+0000","last_change":"2024-09-21T03:44:27.623910+0000","last_active":"2024-09-21T03:44:33.624020+0000","last_peered":"2024-09-21T03:44:33.624020+0000","last_clean":"2024-09-21T03:44:33.624020+0000","last_became_active":"2024-09-21T03:44:27.614037+0000","last_became_peered":"2024-09-21T03:44:27.614037+0000","last_unstale":"2024-09-21T03:44:33.624020+0000","last_undegraded":"2024-09-21T03:44:33.624020+0000","last_fullsized":"2024-09-21T03:44:33.624020+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:11.282438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:11.282438+0000","last_clean_scrub_stamp":"2024-09-21T03:44:11.282438+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:03:56.753935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":672,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,4],"acting":[9,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":58535,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":164,"num_read_kb":145,"num_write":228,"num_write_kb":123,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13,"num_bytes_recovered":1954,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":540672,"data_stored":175605,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":199,"ondisk_log_size":199,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":71,"seq":304942678033,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27892,"kb_used_data":864,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717260,"statfs":{"total":95995035648,"available":95966474240,"internally_reserved":0,"allocated":884736,"data_stored":428272,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":65,"seq":279172874262,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28556,"kb_used_data":1532,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716596,"statfs":{"total":95995035648,"available":95965794304,"internally_reserved":0,"allocated":1568768,"data_stored":1070520,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":60,"seq":257698037786,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27904,"kb_used_data":880,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717248,"statfs":{"total":95995035648,"available":95966461952,"internally_reserved":0,"allocated":901120,"data_stored":428090,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":54,"seq":231928234015,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27884,"kb_used_data":860,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717268,"statfs":{"total":95995035648,"available":95966482432,"internally_reserved":0,"allocated":880640,"data_stored":427684,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":49,"seq":210453397543,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36804,"kb_used_data":1460,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708348,"statfs":{"total":95995035648,"available":95957348352,"internally_reserved":0,"allocated":1495040,"data_stored":1021768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":36174282},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":43,"seq":184683593771,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27888,"kb_used_data":864,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717264,"statfs":{"total":95995035648,"available":95966478336,"internally_reserved":0,"allocated":884736,"data_stored":428198,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":37,"seq":158913790001,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32656,"kb_used_data":1472,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712496,"statfs":{"total":95995035648,"available":95961595904,"internally_reserved":0,"allocated":1507328,"data_stored":1020664,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":31,"seq":133143986229,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36224,"kb_used_data":872,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708928,"statfs":{"total":95995035648,"available":95957942272,"internally_reserved":0,"allocated":892928,"data_stored":428731,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149755,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32064,"kb_used_data":880,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713088,"statfs":{"total":95995035648,"available":95962202112,"internally_reserved":0,"allocated":901120,"data_stored":427632,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":19,"seq":81604378688,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27944,"kb_used_data":920,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717208,"statfs":{"total":95995035648,"available":95966420992,"internally_reserved":0,"allocated":942080,"data_stored":473364,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542213,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27948,"kb_used_data":924,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717204,"statfs":{"total":95995035648,"available":95966416896,"internally_reserved":0,"allocated":946176,"data_stored":481221,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705739,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27912,"kb_used_data":884,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717240,"statfs":{"total":95995035648,"available":95966453760,"internally_reserved":0,"allocated":905216,"data_stored":429146,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2178,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":54253,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":86016,"data_stored":52133,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1763,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3211,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1113,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":4315,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":716,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":106496,"data_stored":53067,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1304,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T03:44:41.068 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 51116d5c-77ca-11ef-baf5-efdc52797490 -- ceph pg dump --format=json 2024-09-21T03:44:41.333 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/51116d5c-77ca-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:44:41.728 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:44:41 smithi039 ceph-mon[31211]: from='client.15000 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T03:44:41.828 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:41.819+0000 7faf00b34640 1 -- 172.21.15.39:0/139095041 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7faefc1083d0 msgr2=0x7faefc1087e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:41.829 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:41.819+0000 7faf00b34640 1 --2- 172.21.15.39:0/139095041 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7faefc1083d0 0x7faefc1087e0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7faeec009900 tx=0x7faeec02a410 comp rx=0 tx=0).stop 2024-09-21T03:44:41.829 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:41.819+0000 7faf00b34640 1 -- 172.21.15.39:0/139095041 shutdown_connections 2024-09-21T03:44:41.829 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:41.819+0000 7faf00b34640 1 --2- 172.21.15.39:0/139095041 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7faefc108d20 0x7faefc10f550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:41.829 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:41.819+0000 7faf00b34640 1 --2- 172.21.15.39:0/139095041 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7faefc1083d0 0x7faefc1087e0 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:41.829 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:41.819+0000 7faf00b34640 1 --2- 172.21.15.39:0/139095041 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7faefc102ae0 0x7faefc102ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:41.829 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:41.819+0000 7faf00b34640 1 -- 172.21.15.39:0/139095041 >> 172.21.15.39:0/139095041 conn(0x7faefc079760 msgr2=0x7faefc079b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:44:41.829 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:41.820+0000 7faf00b34640 1 -- 172.21.15.39:0/139095041 shutdown_connections 2024-09-21T03:44:41.829 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:41.820+0000 7faf00b34640 1 -- 172.21.15.39:0/139095041 wait complete. 2024-09-21T03:44:41.829 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:41.821+0000 7faf00b34640 1 Processor -- start 2024-09-21T03:44:41.829 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:41.821+0000 7faf00b34640 1 -- start start 2024-09-21T03:44:41.829 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:41.821+0000 7faf00b34640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7faefc102ae0 0x7faefc19b2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:41.829 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:41.821+0000 7faf00b34640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7faefc1083d0 0x7faefc19b7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:41.829 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:41.821+0000 7faf00b34640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7faefc108d20 0x7faefc19bd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:41.830 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:41.821+0000 7faf00b34640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7faefc112990 con 0x7faefc1083d0 2024-09-21T03:44:41.830 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:41.821+0000 7faf00b34640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7faefc112810 con 0x7faefc108d20 2024-09-21T03:44:41.830 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:41.821+0000 7faf00b34640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7faefc112b10 con 0x7faefc102ae0 2024-09-21T03:44:41.830 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:41.821+0000 7faef9d9b640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7faefc1083d0 0x7faefc19b7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:41.830 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:41.821+0000 7faef9d9b640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7faefc1083d0 0x7faefc19b7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:53856/0 (socket says 172.21.15.39:53856) 2024-09-21T03:44:41.830 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:41.821+0000 7faef9d9b640 1 -- 172.21.15.39:0/2011025086 learned_addr learned my addr 172.21.15.39:0/2011025086 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:44:41.830 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:41.821+0000 7faefa59c640 1 --2- 172.21.15.39:0/2011025086 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7faefc102ae0 0x7faefc19b2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:41.830 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:41.821+0000 7faefad9d640 1 --2- 172.21.15.39:0/2011025086 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7faefc108d20 0x7faefc19bd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:41.830 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:41.821+0000 7faef9d9b640 1 -- 172.21.15.39:0/2011025086 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7faefc102ae0 msgr2=0x7faefc19b2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:41.830 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:41.821+0000 7faef9d9b640 1 --2- 172.21.15.39:0/2011025086 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7faefc102ae0 0x7faefc19b2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:41.830 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:41.821+0000 7faef9d9b640 1 -- 172.21.15.39:0/2011025086 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7faefc108d20 msgr2=0x7faefc19bd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:41.831 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:41.821+0000 7faef9d9b640 1 --2- 172.21.15.39:0/2011025086 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7faefc108d20 0x7faefc19bd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:41.831 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:41.821+0000 7faef9d9b640 1 -- 172.21.15.39:0/2011025086 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faefc1a6560 con 0x7faefc1083d0 2024-09-21T03:44:41.831 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:41.822+0000 7faef9d9b640 1 --2- 172.21.15.39:0/2011025086 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7faefc1083d0 0x7faefc19b7e0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7faeec006010 tx=0x7faeec004060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:44:41.831 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:41.822+0000 7faef37fe640 1 -- 172.21.15.39:0/2011025086 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faeec03f030 con 0x7faefc1083d0 2024-09-21T03:44:41.831 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:41.822+0000 7faf00b34640 1 -- 172.21.15.39:0/2011025086 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faefc1a67f0 con 0x7faefc1083d0 2024-09-21T03:44:41.831 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:41.822+0000 7faef37fe640 1 -- 172.21.15.39:0/2011025086 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faeec003aa0 con 0x7faefc1083d0 2024-09-21T03:44:41.831 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:41.822+0000 7faef37fe640 1 -- 172.21.15.39:0/2011025086 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faeec035070 con 0x7faefc1083d0 2024-09-21T03:44:41.831 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:41.822+0000 7faf00b34640 1 -- 172.21.15.39:0/2011025086 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faefc1a6d20 con 0x7faefc1083d0 2024-09-21T03:44:41.831 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:41.823+0000 7faf00b34640 1 -- 172.21.15.39:0/2011025086 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faefc103d10 con 0x7faefc1083d0 2024-09-21T03:44:41.831 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:41.824+0000 7faef37fe640 1 -- 172.21.15.39:0/2011025086 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 22) ==== 154580+0+0 (secure 0 0 0) 0x7faeec042030 con 0x7faefc1083d0 2024-09-21T03:44:41.831 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:41.825+0000 7faef37fe640 1 --2- 172.21.15.39:0/2011025086 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7faee80b65f0 0x7faee80b8b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:41.831 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:41.825+0000 7faefa59c640 1 --2- 172.21.15.39:0/2011025086 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7faee80b65f0 0x7faee80b8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:41.831 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:41.825+0000 7faef37fe640 1 -- 172.21.15.39:0/2011025086 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8484+0+0 (secure 0 0 0) 0x7faeec107de0 con 0x7faefc1083d0 2024-09-21T03:44:41.832 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:41.826+0000 7faefa59c640 1 --2- 172.21.15.39:0/2011025086 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7faee80b65f0 0x7faee80b8b00 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7faefc06c6c0 tx=0x7faee4005f90 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:44:41.832 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:41.828+0000 7faef37fe640 1 -- 172.21.15.39:0/2011025086 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7faeec0d27f0 con 0x7faefc1083d0 2024-09-21T03:44:41.967 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:44:41 smithi161 ceph-mon[34533]: from='client.15000 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T03:44:41.974 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:44:41 smithi150 ceph-mon[34289]: from='client.15000 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T03:44:42.015 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:42.014+0000 7faf00b34640 1 -- 172.21.15.39:0/2011025086 --> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7faefc063800 con 0x7faee80b65f0 2024-09-21T03:44:42.020 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:42.019+0000 7faef37fe640 1 -- 172.21.15.39:0/2011025086 <== mgr.14223 v2:172.21.15.39:6800/1696381982 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258760 (secure 0 0 0) 0x7faefc063800 con 0x7faee80b65f0 2024-09-21T03:44:42.023 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T03:44:42.023 INFO:teuthology.orchestra.run.smithi039.stderr:dumped all 2024-09-21T03:44:42.026 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:42.022+0000 7faf00b34640 1 -- 172.21.15.39:0/2011025086 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7faee80b65f0 msgr2=0x7faee80b8b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:42.026 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:42.022+0000 7faf00b34640 1 --2- 172.21.15.39:0/2011025086 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7faee80b65f0 0x7faee80b8b00 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7faefc06c6c0 tx=0x7faee4005f90 comp rx=0 tx=0).stop 2024-09-21T03:44:42.026 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:42.022+0000 7faf00b34640 1 -- 172.21.15.39:0/2011025086 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7faefc1083d0 msgr2=0x7faefc19b7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:42.026 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:42.022+0000 7faf00b34640 1 --2- 172.21.15.39:0/2011025086 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7faefc1083d0 0x7faefc19b7e0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7faeec006010 tx=0x7faeec004060 comp rx=0 tx=0).stop 2024-09-21T03:44:42.026 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:42.022+0000 7faf00b34640 1 -- 172.21.15.39:0/2011025086 shutdown_connections 2024-09-21T03:44:42.026 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:42.022+0000 7faf00b34640 1 --2- 172.21.15.39:0/2011025086 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7faee80b65f0 0x7faee80b8b00 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:42.026 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:42.022+0000 7faf00b34640 1 --2- 172.21.15.39:0/2011025086 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7faefc108d20 0x7faefc19bd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:42.026 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:42.022+0000 7faf00b34640 1 --2- 172.21.15.39:0/2011025086 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7faefc1083d0 0x7faefc19b7e0 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:42.026 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:42.022+0000 7faf00b34640 1 --2- 172.21.15.39:0/2011025086 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7faefc102ae0 0x7faefc19b2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:42.026 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:42.022+0000 7faf00b34640 1 -- 172.21.15.39:0/2011025086 >> 172.21.15.39:0/2011025086 conn(0x7faefc079760 msgr2=0x7faefc104400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:44:42.026 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:42.022+0000 7faf00b34640 1 -- 172.21.15.39:0/2011025086 shutdown_connections 2024-09-21T03:44:42.026 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:42.022+0000 7faf00b34640 1 -- 172.21.15.39:0/2011025086 wait complete. 2024-09-21T03:44:42.814 INFO:teuthology.orchestra.run.smithi039.stdout:{"pg_ready":true,"pg_map":{"version":285,"stamp":"2024-09-21T03:44:41.492568+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":649137,"num_objects":102,"num_object_clones":0,"num_object_copies":306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":255,"num_read_kb":219,"num_write":359,"num_write_kb":2276,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":23,"num_bytes_recovered":2953794,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":296,"ondisk_log_size":296,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":361692,"kb_used_data":12428,"kb_used_omap":18,"kb_used_meta":349037,"kb_avail":1124580132,"statfs":{"total":1151940427776,"available":1151570055168,"internally_reserved":0,"allocated":12726272,"data_stored":7079179,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19143,"internal_metadata":357414201},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":47637,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":114,"num_read_kb":105,"num_write":32,"num_write_kb":60,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":784,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003452"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":18,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:44:27.420991+0000","last_change":"2024-09-21T03:44:14.327553+0000","last_active":"2024-09-21T03:44:27.420991+0000","last_peered":"2024-09-21T03:44:27.420991+0000","last_clean":"2024-09-21T03:44:27.420991+0000","last_became_active":"2024-09-21T03:44:14.327213+0000","last_became_peered":"2024-09-21T03:44:14.327213+0000","last_unstale":"2024-09-21T03:44:27.420991+0000","last_undegraded":"2024-09-21T03:44:27.420991+0000","last_fullsized":"2024-09-21T03:44:27.420991+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_clean_scrub_stamp":"2024-09-21T03:44:13.292303+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:40:56.550897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":18,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:44:27.417200+0000","last_change":"2024-09-21T03:44:14.321427+0000","last_active":"2024-09-21T03:44:27.417200+0000","last_peered":"2024-09-21T03:44:27.417200+0000","last_clean":"2024-09-21T03:44:27.417200+0000","last_became_active":"2024-09-21T03:44:14.321026+0000","last_became_peered":"2024-09-21T03:44:14.321026+0000","last_unstale":"2024-09-21T03:44:27.417200+0000","last_undegraded":"2024-09-21T03:44:27.417200+0000","last_fullsized":"2024-09-21T03:44:27.417200+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_clean_scrub_stamp":"2024-09-21T03:44:13.292303+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:18:00.889714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":18,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:44:27.416317+0000","last_change":"2024-09-21T03:44:14.324908+0000","last_active":"2024-09-21T03:44:27.416317+0000","last_peered":"2024-09-21T03:44:27.416317+0000","last_clean":"2024-09-21T03:44:27.416317+0000","last_became_active":"2024-09-21T03:44:14.324533+0000","last_became_peered":"2024-09-21T03:44:14.324533+0000","last_unstale":"2024-09-21T03:44:27.416317+0000","last_undegraded":"2024-09-21T03:44:27.416317+0000","last_fullsized":"2024-09-21T03:44:27.416317+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_clean_scrub_stamp":"2024-09-21T03:44:13.292303+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:44:25.235442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":18,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:44:27.613591+0000","last_change":"2024-09-21T03:44:14.326758+0000","last_active":"2024-09-21T03:44:27.613591+0000","last_peered":"2024-09-21T03:44:27.613591+0000","last_clean":"2024-09-21T03:44:27.613591+0000","last_became_active":"2024-09-21T03:44:14.326410+0000","last_became_peered":"2024-09-21T03:44:14.326410+0000","last_unstale":"2024-09-21T03:44:27.613591+0000","last_undegraded":"2024-09-21T03:44:27.613591+0000","last_fullsized":"2024-09-21T03:44:27.613591+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_clean_scrub_stamp":"2024-09-21T03:44:13.292303+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:13:22.716940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":18,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:44:27.419914+0000","last_change":"2024-09-21T03:44:14.318875+0000","last_active":"2024-09-21T03:44:27.419914+0000","last_peered":"2024-09-21T03:44:27.419914+0000","last_clean":"2024-09-21T03:44:27.419914+0000","last_became_active":"2024-09-21T03:44:14.318425+0000","last_became_peered":"2024-09-21T03:44:14.318425+0000","last_unstale":"2024-09-21T03:44:27.419914+0000","last_undegraded":"2024-09-21T03:44:27.419914+0000","last_fullsized":"2024-09-21T03:44:27.419914+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_clean_scrub_stamp":"2024-09-21T03:44:13.292303+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:45:39.916587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":18,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:44:27.611598+0000","last_change":"2024-09-21T03:44:14.325877+0000","last_active":"2024-09-21T03:44:27.611598+0000","last_peered":"2024-09-21T03:44:27.611598+0000","last_clean":"2024-09-21T03:44:27.611598+0000","last_became_active":"2024-09-21T03:44:14.325451+0000","last_became_peered":"2024-09-21T03:44:14.325451+0000","last_unstale":"2024-09-21T03:44:27.611598+0000","last_undegraded":"2024-09-21T03:44:27.611598+0000","last_fullsized":"2024-09-21T03:44:27.611598+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_clean_scrub_stamp":"2024-09-21T03:44:13.292303+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:30:42.427054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":18,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:44:27.418241+0000","last_change":"2024-09-21T03:44:14.323192+0000","last_active":"2024-09-21T03:44:27.418241+0000","last_peered":"2024-09-21T03:44:27.418241+0000","last_clean":"2024-09-21T03:44:27.418241+0000","last_became_active":"2024-09-21T03:44:14.322487+0000","last_became_peered":"2024-09-21T03:44:14.322487+0000","last_unstale":"2024-09-21T03:44:27.418241+0000","last_undegraded":"2024-09-21T03:44:27.418241+0000","last_fullsized":"2024-09-21T03:44:27.418241+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_clean_scrub_stamp":"2024-09-21T03:44:13.292303+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:10:48.770536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:44:27.613320+0000","last_change":"2024-09-21T03:44:27.612681+0000","last_active":"2024-09-21T03:44:27.613320+0000","last_peered":"2024-09-21T03:44:27.613320+0000","last_clean":"2024-09-21T03:44:27.613320+0000","last_became_active":"2024-09-21T03:44:27.612200+0000","last_became_peered":"2024-09-21T03:44:27.612200+0000","last_unstale":"2024-09-21T03:44:27.613320+0000","last_undegraded":"2024-09-21T03:44:27.613320+0000","last_fullsized":"2024-09-21T03:44:27.613320+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_clean_scrub_stamp":"2024-09-21T03:44:13.292303+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:44:58.848571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,5,10],"acting":[9,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":18,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:44:27.414086+0000","last_change":"2024-09-21T03:44:14.325820+0000","last_active":"2024-09-21T03:44:27.414086+0000","last_peered":"2024-09-21T03:44:27.414086+0000","last_clean":"2024-09-21T03:44:27.414086+0000","last_became_active":"2024-09-21T03:44:14.325070+0000","last_became_peered":"2024-09-21T03:44:14.325070+0000","last_unstale":"2024-09-21T03:44:27.414086+0000","last_undegraded":"2024-09-21T03:44:27.414086+0000","last_fullsized":"2024-09-21T03:44:27.414086+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_clean_scrub_stamp":"2024-09-21T03:44:13.292303+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:54:01.072623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":18,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:44:27.612466+0000","last_change":"2024-09-21T03:44:14.323746+0000","last_active":"2024-09-21T03:44:27.612466+0000","last_peered":"2024-09-21T03:44:27.612466+0000","last_clean":"2024-09-21T03:44:27.612466+0000","last_became_active":"2024-09-21T03:44:14.323346+0000","last_became_peered":"2024-09-21T03:44:14.323346+0000","last_unstale":"2024-09-21T03:44:27.612466+0000","last_undegraded":"2024-09-21T03:44:27.612466+0000","last_fullsized":"2024-09-21T03:44:27.612466+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_clean_scrub_stamp":"2024-09-21T03:44:13.292303+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:17:44.603801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":18,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:44:27.419043+0000","last_change":"2024-09-21T03:44:14.328733+0000","last_active":"2024-09-21T03:44:27.419043+0000","last_peered":"2024-09-21T03:44:27.419043+0000","last_clean":"2024-09-21T03:44:27.419043+0000","last_became_active":"2024-09-21T03:44:14.328378+0000","last_became_peered":"2024-09-21T03:44:14.328378+0000","last_unstale":"2024-09-21T03:44:27.419043+0000","last_undegraded":"2024-09-21T03:44:27.419043+0000","last_fullsized":"2024-09-21T03:44:27.419043+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_clean_scrub_stamp":"2024-09-21T03:44:13.292303+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:23:38.497413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":18,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:44:27.613597+0000","last_change":"2024-09-21T03:44:14.328505+0000","last_active":"2024-09-21T03:44:27.613597+0000","last_peered":"2024-09-21T03:44:27.613597+0000","last_clean":"2024-09-21T03:44:27.613597+0000","last_became_active":"2024-09-21T03:44:14.328175+0000","last_became_peered":"2024-09-21T03:44:14.328175+0000","last_unstale":"2024-09-21T03:44:27.613597+0000","last_undegraded":"2024-09-21T03:44:27.613597+0000","last_fullsized":"2024-09-21T03:44:27.613597+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_clean_scrub_stamp":"2024-09-21T03:44:13.292303+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:45:56.628962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":18,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:44:27.410517+0000","last_change":"2024-09-21T03:44:14.314519+0000","last_active":"2024-09-21T03:44:27.410517+0000","last_peered":"2024-09-21T03:44:27.410517+0000","last_clean":"2024-09-21T03:44:27.410517+0000","last_became_active":"2024-09-21T03:44:14.314145+0000","last_became_peered":"2024-09-21T03:44:14.314145+0000","last_unstale":"2024-09-21T03:44:27.410517+0000","last_undegraded":"2024-09-21T03:44:27.410517+0000","last_fullsized":"2024-09-21T03:44:27.410517+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_clean_scrub_stamp":"2024-09-21T03:44:13.292303+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:45:48.494667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":18,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:44:27.416665+0000","last_change":"2024-09-21T03:44:14.312195+0000","last_active":"2024-09-21T03:44:27.416665+0000","last_peered":"2024-09-21T03:44:27.416665+0000","last_clean":"2024-09-21T03:44:27.416665+0000","last_became_active":"2024-09-21T03:44:14.311861+0000","last_became_peered":"2024-09-21T03:44:14.311861+0000","last_unstale":"2024-09-21T03:44:27.416665+0000","last_undegraded":"2024-09-21T03:44:27.416665+0000","last_fullsized":"2024-09-21T03:44:27.416665+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_clean_scrub_stamp":"2024-09-21T03:44:13.292303+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:17:05.025898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":18,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:44:27.416311+0000","last_change":"2024-09-21T03:44:14.323532+0000","last_active":"2024-09-21T03:44:27.416311+0000","last_peered":"2024-09-21T03:44:27.416311+0000","last_clean":"2024-09-21T03:44:27.416311+0000","last_became_active":"2024-09-21T03:44:14.323171+0000","last_became_peered":"2024-09-21T03:44:14.323171+0000","last_unstale":"2024-09-21T03:44:27.416311+0000","last_undegraded":"2024-09-21T03:44:27.416311+0000","last_fullsized":"2024-09-21T03:44:27.416311+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_clean_scrub_stamp":"2024-09-21T03:44:13.292303+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:23:28.999129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":18,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:44:27.413494+0000","last_change":"2024-09-21T03:44:14.324998+0000","last_active":"2024-09-21T03:44:27.413494+0000","last_peered":"2024-09-21T03:44:27.413494+0000","last_clean":"2024-09-21T03:44:27.413494+0000","last_became_active":"2024-09-21T03:44:14.324019+0000","last_became_peered":"2024-09-21T03:44:14.324019+0000","last_unstale":"2024-09-21T03:44:27.413494+0000","last_undegraded":"2024-09-21T03:44:27.413494+0000","last_fullsized":"2024-09-21T03:44:27.413494+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_clean_scrub_stamp":"2024-09-21T03:44:13.292303+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:49:12.965299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":18,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:44:27.421157+0000","last_change":"2024-09-21T03:44:14.325603+0000","last_active":"2024-09-21T03:44:27.421157+0000","last_peered":"2024-09-21T03:44:27.421157+0000","last_clean":"2024-09-21T03:44:27.421157+0000","last_became_active":"2024-09-21T03:44:14.325208+0000","last_became_peered":"2024-09-21T03:44:14.325208+0000","last_unstale":"2024-09-21T03:44:27.421157+0000","last_undegraded":"2024-09-21T03:44:27.421157+0000","last_fullsized":"2024-09-21T03:44:27.421157+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_clean_scrub_stamp":"2024-09-21T03:44:13.292303+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:11:33.794242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":18,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:44:27.414581+0000","last_change":"2024-09-21T03:44:14.317545+0000","last_active":"2024-09-21T03:44:27.414581+0000","last_peered":"2024-09-21T03:44:27.414581+0000","last_clean":"2024-09-21T03:44:27.414581+0000","last_became_active":"2024-09-21T03:44:14.317069+0000","last_became_peered":"2024-09-21T03:44:14.317069+0000","last_unstale":"2024-09-21T03:44:27.414581+0000","last_undegraded":"2024-09-21T03:44:27.414581+0000","last_fullsized":"2024-09-21T03:44:27.414581+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_clean_scrub_stamp":"2024-09-21T03:44:13.292303+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:16:11.734887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":18,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:44:27.418423+0000","last_change":"2024-09-21T03:44:14.322055+0000","last_active":"2024-09-21T03:44:27.418423+0000","last_peered":"2024-09-21T03:44:27.418423+0000","last_clean":"2024-09-21T03:44:27.418423+0000","last_became_active":"2024-09-21T03:44:14.321652+0000","last_became_peered":"2024-09-21T03:44:14.321652+0000","last_unstale":"2024-09-21T03:44:27.418423+0000","last_undegraded":"2024-09-21T03:44:27.418423+0000","last_fullsized":"2024-09-21T03:44:27.418423+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_clean_scrub_stamp":"2024-09-21T03:44:13.292303+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:45:14.285844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":25,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:44:27.423315+0000","last_change":"2024-09-21T03:44:27.423175+0000","last_active":"2024-09-21T03:44:27.423315+0000","last_peered":"2024-09-21T03:44:27.423315+0000","last_clean":"2024-09-21T03:44:27.423315+0000","last_became_active":"2024-09-21T03:44:27.422733+0000","last_became_peered":"2024-09-21T03:44:27.422733+0000","last_unstale":"2024-09-21T03:44:27.423315+0000","last_undegraded":"2024-09-21T03:44:27.423315+0000","last_fullsized":"2024-09-21T03:44:27.423315+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_clean_scrub_stamp":"2024-09-21T03:44:13.292303+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:46:17.874557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,8],"acting":[11,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":18,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:44:27.410602+0000","last_change":"2024-09-21T03:44:14.325829+0000","last_active":"2024-09-21T03:44:27.410602+0000","last_peered":"2024-09-21T03:44:27.410602+0000","last_clean":"2024-09-21T03:44:27.410602+0000","last_became_active":"2024-09-21T03:44:14.325421+0000","last_became_peered":"2024-09-21T03:44:14.325421+0000","last_unstale":"2024-09-21T03:44:27.410602+0000","last_undegraded":"2024-09-21T03:44:27.410602+0000","last_fullsized":"2024-09-21T03:44:27.410602+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_clean_scrub_stamp":"2024-09-21T03:44:13.292303+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:49:00.240771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":18,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:44:27.413691+0000","last_change":"2024-09-21T03:44:14.325255+0000","last_active":"2024-09-21T03:44:27.413691+0000","last_peered":"2024-09-21T03:44:27.413691+0000","last_clean":"2024-09-21T03:44:27.413691+0000","last_became_active":"2024-09-21T03:44:14.324617+0000","last_became_peered":"2024-09-21T03:44:14.324617+0000","last_unstale":"2024-09-21T03:44:27.413691+0000","last_undegraded":"2024-09-21T03:44:27.413691+0000","last_fullsized":"2024-09-21T03:44:27.413691+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_clean_scrub_stamp":"2024-09-21T03:44:13.292303+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:05:03.268522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":18,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:44:27.610583+0000","last_change":"2024-09-21T03:44:14.328053+0000","last_active":"2024-09-21T03:44:27.610583+0000","last_peered":"2024-09-21T03:44:27.610583+0000","last_clean":"2024-09-21T03:44:27.610583+0000","last_became_active":"2024-09-21T03:44:14.327735+0000","last_became_peered":"2024-09-21T03:44:14.327735+0000","last_unstale":"2024-09-21T03:44:27.610583+0000","last_undegraded":"2024-09-21T03:44:27.610583+0000","last_fullsized":"2024-09-21T03:44:27.610583+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_clean_scrub_stamp":"2024-09-21T03:44:13.292303+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:39:48.395784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":18,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:44:27.413859+0000","last_change":"2024-09-21T03:44:14.321181+0000","last_active":"2024-09-21T03:44:27.413859+0000","last_peered":"2024-09-21T03:44:27.413859+0000","last_clean":"2024-09-21T03:44:27.413859+0000","last_became_active":"2024-09-21T03:44:14.320772+0000","last_became_peered":"2024-09-21T03:44:14.320772+0000","last_unstale":"2024-09-21T03:44:27.413859+0000","last_undegraded":"2024-09-21T03:44:27.413859+0000","last_fullsized":"2024-09-21T03:44:27.413859+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_clean_scrub_stamp":"2024-09-21T03:44:13.292303+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:49:26.317404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":18,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:44:27.611124+0000","last_change":"2024-09-21T03:44:14.322643+0000","last_active":"2024-09-21T03:44:27.611124+0000","last_peered":"2024-09-21T03:44:27.611124+0000","last_clean":"2024-09-21T03:44:27.611124+0000","last_became_active":"2024-09-21T03:44:14.322254+0000","last_became_peered":"2024-09-21T03:44:14.322254+0000","last_unstale":"2024-09-21T03:44:27.611124+0000","last_undegraded":"2024-09-21T03:44:27.611124+0000","last_fullsized":"2024-09-21T03:44:27.611124+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_clean_scrub_stamp":"2024-09-21T03:44:13.292303+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:31:47.147845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":18,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:44:27.418815+0000","last_change":"2024-09-21T03:44:14.323090+0000","last_active":"2024-09-21T03:44:27.418815+0000","last_peered":"2024-09-21T03:44:27.418815+0000","last_clean":"2024-09-21T03:44:27.418815+0000","last_became_active":"2024-09-21T03:44:14.322669+0000","last_became_peered":"2024-09-21T03:44:14.322669+0000","last_unstale":"2024-09-21T03:44:27.418815+0000","last_undegraded":"2024-09-21T03:44:27.418815+0000","last_fullsized":"2024-09-21T03:44:27.418815+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_clean_scrub_stamp":"2024-09-21T03:44:13.292303+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:08:56.852422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":18,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:44:27.418948+0000","last_change":"2024-09-21T03:44:14.318146+0000","last_active":"2024-09-21T03:44:27.418948+0000","last_peered":"2024-09-21T03:44:27.418948+0000","last_clean":"2024-09-21T03:44:27.418948+0000","last_became_active":"2024-09-21T03:44:14.317761+0000","last_became_peered":"2024-09-21T03:44:14.317761+0000","last_unstale":"2024-09-21T03:44:27.418948+0000","last_undegraded":"2024-09-21T03:44:27.418948+0000","last_fullsized":"2024-09-21T03:44:27.418948+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_clean_scrub_stamp":"2024-09-21T03:44:13.292303+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:07:31.530728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":18,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:44:27.410569+0000","last_change":"2024-09-21T03:44:14.316832+0000","last_active":"2024-09-21T03:44:27.410569+0000","last_peered":"2024-09-21T03:44:27.410569+0000","last_clean":"2024-09-21T03:44:27.410569+0000","last_became_active":"2024-09-21T03:44:14.316471+0000","last_became_peered":"2024-09-21T03:44:14.316471+0000","last_unstale":"2024-09-21T03:44:27.410569+0000","last_undegraded":"2024-09-21T03:44:27.410569+0000","last_fullsized":"2024-09-21T03:44:27.410569+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_clean_scrub_stamp":"2024-09-21T03:44:13.292303+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:21:23.848778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":18,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:44:27.417002+0000","last_change":"2024-09-21T03:44:14.322277+0000","last_active":"2024-09-21T03:44:27.417002+0000","last_peered":"2024-09-21T03:44:27.417002+0000","last_clean":"2024-09-21T03:44:27.417002+0000","last_became_active":"2024-09-21T03:44:14.321952+0000","last_became_peered":"2024-09-21T03:44:14.321952+0000","last_unstale":"2024-09-21T03:44:27.417002+0000","last_undegraded":"2024-09-21T03:44:27.417002+0000","last_fullsized":"2024-09-21T03:44:27.417002+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_clean_scrub_stamp":"2024-09-21T03:44:13.292303+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:43:55.169380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":18,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:44:27.418939+0000","last_change":"2024-09-21T03:44:14.322626+0000","last_active":"2024-09-21T03:44:27.418939+0000","last_peered":"2024-09-21T03:44:27.418939+0000","last_clean":"2024-09-21T03:44:27.418939+0000","last_became_active":"2024-09-21T03:44:14.321559+0000","last_became_peered":"2024-09-21T03:44:14.321559+0000","last_unstale":"2024-09-21T03:44:27.418939+0000","last_undegraded":"2024-09-21T03:44:27.418939+0000","last_fullsized":"2024-09-21T03:44:27.418939+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_clean_scrub_stamp":"2024-09-21T03:44:13.292303+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:12:23.736500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":18,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:44:27.416654+0000","last_change":"2024-09-21T03:44:14.318057+0000","last_active":"2024-09-21T03:44:27.416654+0000","last_peered":"2024-09-21T03:44:27.416654+0000","last_clean":"2024-09-21T03:44:27.416654+0000","last_became_active":"2024-09-21T03:44:14.317653+0000","last_became_peered":"2024-09-21T03:44:14.317653+0000","last_unstale":"2024-09-21T03:44:27.416654+0000","last_undegraded":"2024-09-21T03:44:27.416654+0000","last_fullsized":"2024-09-21T03:44:27.416654+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_clean_scrub_stamp":"2024-09-21T03:44:13.292303+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:23:17.396599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":18,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:44:27.418421+0000","last_change":"2024-09-21T03:44:14.322331+0000","last_active":"2024-09-21T03:44:27.418421+0000","last_peered":"2024-09-21T03:44:27.418421+0000","last_clean":"2024-09-21T03:44:27.418421+0000","last_became_active":"2024-09-21T03:44:14.321157+0000","last_became_peered":"2024-09-21T03:44:14.321157+0000","last_unstale":"2024-09-21T03:44:27.418421+0000","last_undegraded":"2024-09-21T03:44:27.418421+0000","last_fullsized":"2024-09-21T03:44:27.418421+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_clean_scrub_stamp":"2024-09-21T03:44:13.292303+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:08:27.953449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":25,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:44:27.423182+0000","last_change":"2024-09-21T03:44:27.423081+0000","last_active":"2024-09-21T03:44:27.423182+0000","last_peered":"2024-09-21T03:44:27.423182+0000","last_clean":"2024-09-21T03:44:27.423182+0000","last_became_active":"2024-09-21T03:44:27.422650+0000","last_became_peered":"2024-09-21T03:44:27.422650+0000","last_unstale":"2024-09-21T03:44:27.423182+0000","last_undegraded":"2024-09-21T03:44:27.423182+0000","last_fullsized":"2024-09-21T03:44:27.423182+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_clean_scrub_stamp":"2024-09-21T03:44:13.292303+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:32:44.399954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,7],"acting":[8,9,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1b","version":"79'6","reported_seq":47,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:44:37.717899+0000","last_change":"2024-09-21T03:44:12.307464+0000","last_active":"2024-09-21T03:44:37.717899+0000","last_peered":"2024-09-21T03:44:37.717899+0000","last_clean":"2024-09-21T03:44:37.717899+0000","last_became_active":"2024-09-21T03:44:12.307077+0000","last_became_peered":"2024-09-21T03:44:12.307077+0000","last_unstale":"2024-09-21T03:44:37.717899+0000","last_undegraded":"2024-09-21T03:44:37.717899+0000","last_fullsized":"2024-09-21T03:44:37.717899+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:11.282438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:11.282438+0000","last_clean_scrub_stamp":"2024-09-21T03:44:11.282438+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:50:40.344659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1013,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":18,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":18,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:44:27.414062+0000","last_change":"2024-09-21T03:44:14.319072+0000","last_active":"2024-09-21T03:44:27.414062+0000","last_peered":"2024-09-21T03:44:27.414062+0000","last_clean":"2024-09-21T03:44:27.414062+0000","last_became_active":"2024-09-21T03:44:14.318712+0000","last_became_peered":"2024-09-21T03:44:14.318712+0000","last_unstale":"2024-09-21T03:44:27.414062+0000","last_undegraded":"2024-09-21T03:44:27.414062+0000","last_fullsized":"2024-09-21T03:44:27.414062+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_clean_scrub_stamp":"2024-09-21T03:44:13.292303+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:30:35.618110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1a","version":"77'7","reported_seq":30,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:44:27.611763+0000","last_change":"2024-09-21T03:44:12.310146+0000","last_active":"2024-09-21T03:44:27.611763+0000","last_peered":"2024-09-21T03:44:27.611763+0000","last_clean":"2024-09-21T03:44:27.611763+0000","last_became_active":"2024-09-21T03:44:12.309817+0000","last_became_peered":"2024-09-21T03:44:12.309817+0000","last_unstale":"2024-09-21T03:44:27.611763+0000","last_undegraded":"2024-09-21T03:44:27.611763+0000","last_fullsized":"2024-09-21T03:44:27.611763+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:11.282438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:11.282438+0000","last_clean_scrub_stamp":"2024-09-21T03:44:11.282438+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:46:51.994488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.19","version":"77'3","reported_seq":26,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:44:27.410211+0000","last_change":"2024-09-21T03:44:12.308995+0000","last_active":"2024-09-21T03:44:27.410211+0000","last_peered":"2024-09-21T03:44:27.410211+0000","last_clean":"2024-09-21T03:44:27.410211+0000","last_became_active":"2024-09-21T03:44:12.308058+0000","last_became_peered":"2024-09-21T03:44:12.308058+0000","last_unstale":"2024-09-21T03:44:27.410211+0000","last_undegraded":"2024-09-21T03:44:27.410211+0000","last_fullsized":"2024-09-21T03:44:27.410211+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:11.282438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:11.282438+0000","last_clean_scrub_stamp":"2024-09-21T03:44:11.282438+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:48:39.384985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":18,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:44:27.610583+0000","last_change":"2024-09-21T03:44:14.317878+0000","last_active":"2024-09-21T03:44:27.610583+0000","last_peered":"2024-09-21T03:44:27.610583+0000","last_clean":"2024-09-21T03:44:27.610583+0000","last_became_active":"2024-09-21T03:44:14.317521+0000","last_became_peered":"2024-09-21T03:44:14.317521+0000","last_unstale":"2024-09-21T03:44:27.610583+0000","last_undegraded":"2024-09-21T03:44:27.610583+0000","last_fullsized":"2024-09-21T03:44:27.610583+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_clean_scrub_stamp":"2024-09-21T03:44:13.292303+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:08:00.705210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"77'3","reported_seq":27,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:44:27.414041+0000","last_change":"2024-09-21T03:44:12.308777+0000","last_active":"2024-09-21T03:44:27.414041+0000","last_peered":"2024-09-21T03:44:27.414041+0000","last_clean":"2024-09-21T03:44:27.414041+0000","last_became_active":"2024-09-21T03:44:12.308171+0000","last_became_peered":"2024-09-21T03:44:12.308171+0000","last_unstale":"2024-09-21T03:44:27.414041+0000","last_undegraded":"2024-09-21T03:44:27.414041+0000","last_fullsized":"2024-09-21T03:44:27.414041+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:11.282438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:11.282438+0000","last_clean_scrub_stamp":"2024-09-21T03:44:11.282438+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:41:21.344491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":18,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:44:27.414180+0000","last_change":"2024-09-21T03:44:14.315782+0000","last_active":"2024-09-21T03:44:27.414180+0000","last_peered":"2024-09-21T03:44:27.414180+0000","last_clean":"2024-09-21T03:44:27.414180+0000","last_became_active":"2024-09-21T03:44:14.315314+0000","last_became_peered":"2024-09-21T03:44:14.315314+0000","last_unstale":"2024-09-21T03:44:27.414180+0000","last_undegraded":"2024-09-21T03:44:27.414180+0000","last_fullsized":"2024-09-21T03:44:27.414180+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_clean_scrub_stamp":"2024-09-21T03:44:13.292303+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:22:20.500220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"77'4","reported_seq":55,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:44:37.717074+0000","last_change":"2024-09-21T03:44:12.313251+0000","last_active":"2024-09-21T03:44:37.717074+0000","last_peered":"2024-09-21T03:44:37.717074+0000","last_clean":"2024-09-21T03:44:37.717074+0000","last_became_active":"2024-09-21T03:44:12.312941+0000","last_became_peered":"2024-09-21T03:44:12.312941+0000","last_unstale":"2024-09-21T03:44:37.717074+0000","last_undegraded":"2024-09-21T03:44:37.717074+0000","last_fullsized":"2024-09-21T03:44:37.717074+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:11.282438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:11.282438+0000","last_clean_scrub_stamp":"2024-09-21T03:44:11.282438+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:38:39.876097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":28,"num_read_kb":28,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":18,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:44:27.414153+0000","last_change":"2024-09-21T03:44:14.319621+0000","last_active":"2024-09-21T03:44:27.414153+0000","last_peered":"2024-09-21T03:44:27.414153+0000","last_clean":"2024-09-21T03:44:27.414153+0000","last_became_active":"2024-09-21T03:44:14.316173+0000","last_became_peered":"2024-09-21T03:44:14.316173+0000","last_unstale":"2024-09-21T03:44:27.414153+0000","last_undegraded":"2024-09-21T03:44:27.414153+0000","last_fullsized":"2024-09-21T03:44:27.414153+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_clean_scrub_stamp":"2024-09-21T03:44:13.292303+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:10:21.952989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"77'4","reported_seq":27,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:44:27.419175+0000","last_change":"2024-09-21T03:44:12.308893+0000","last_active":"2024-09-21T03:44:27.419175+0000","last_peered":"2024-09-21T03:44:27.419175+0000","last_clean":"2024-09-21T03:44:27.419175+0000","last_became_active":"2024-09-21T03:44:12.308551+0000","last_became_peered":"2024-09-21T03:44:12.308551+0000","last_unstale":"2024-09-21T03:44:27.419175+0000","last_undegraded":"2024-09-21T03:44:27.419175+0000","last_fullsized":"2024-09-21T03:44:27.419175+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:11.282438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:11.282438+0000","last_clean_scrub_stamp":"2024-09-21T03:44:11.282438+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:48:18.401595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":18,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:44:27.417391+0000","last_change":"2024-09-21T03:44:14.321875+0000","last_active":"2024-09-21T03:44:27.417391+0000","last_peered":"2024-09-21T03:44:27.417391+0000","last_clean":"2024-09-21T03:44:27.417391+0000","last_became_active":"2024-09-21T03:44:14.321483+0000","last_became_peered":"2024-09-21T03:44:14.321483+0000","last_unstale":"2024-09-21T03:44:27.417391+0000","last_undegraded":"2024-09-21T03:44:27.417391+0000","last_fullsized":"2024-09-21T03:44:27.417391+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_clean_scrub_stamp":"2024-09-21T03:44:13.292303+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:40:17.244780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"77'5","reported_seq":29,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:44:27.414461+0000","last_change":"2024-09-21T03:44:12.313208+0000","last_active":"2024-09-21T03:44:27.414461+0000","last_peered":"2024-09-21T03:44:27.414461+0000","last_clean":"2024-09-21T03:44:27.414461+0000","last_became_active":"2024-09-21T03:44:12.312783+0000","last_became_peered":"2024-09-21T03:44:12.312783+0000","last_unstale":"2024-09-21T03:44:27.414461+0000","last_undegraded":"2024-09-21T03:44:27.414461+0000","last_fullsized":"2024-09-21T03:44:27.414461+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:11.282438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:11.282438+0000","last_clean_scrub_stamp":"2024-09-21T03:44:11.282438+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:18:45.114361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":25,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:44:27.421219+0000","last_change":"2024-09-21T03:44:27.421129+0000","last_active":"2024-09-21T03:44:27.421219+0000","last_peered":"2024-09-21T03:44:27.421219+0000","last_clean":"2024-09-21T03:44:27.421219+0000","last_became_active":"2024-09-21T03:44:27.420642+0000","last_became_peered":"2024-09-21T03:44:27.420642+0000","last_unstale":"2024-09-21T03:44:27.421219+0000","last_undegraded":"2024-09-21T03:44:27.421219+0000","last_fullsized":"2024-09-21T03:44:27.421219+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_clean_scrub_stamp":"2024-09-21T03:44:13.292303+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:49:38.065743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,6],"acting":[1,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"79'24","reported_seq":81,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:44:37.725937+0000","last_change":"2024-09-21T03:44:12.314863+0000","last_active":"2024-09-21T03:44:37.725937+0000","last_peered":"2024-09-21T03:44:37.725937+0000","last_clean":"2024-09-21T03:44:37.725937+0000","last_became_active":"2024-09-21T03:44:12.314471+0000","last_became_peered":"2024-09-21T03:44:12.314471+0000","last_unstale":"2024-09-21T03:44:37.725937+0000","last_undegraded":"2024-09-21T03:44:37.725937+0000","last_fullsized":"2024-09-21T03:44:37.725937+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:11.282438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:11.282438+0000","last_clean_scrub_stamp":"2024-09-21T03:44:11.282438+0000","objects_scrubbed":0,"log_size":24,"log_dups_size":0,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:41:03.471226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":50940,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":34,"num_read_kb":66,"num_write":26,"num_write_kb":58,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":18,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:44:27.420428+0000","last_change":"2024-09-21T03:44:14.325471+0000","last_active":"2024-09-21T03:44:27.420428+0000","last_peered":"2024-09-21T03:44:27.420428+0000","last_clean":"2024-09-21T03:44:27.420428+0000","last_became_active":"2024-09-21T03:44:14.325072+0000","last_became_peered":"2024-09-21T03:44:14.325072+0000","last_unstale":"2024-09-21T03:44:27.420428+0000","last_undegraded":"2024-09-21T03:44:27.420428+0000","last_fullsized":"2024-09-21T03:44:27.420428+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_clean_scrub_stamp":"2024-09-21T03:44:13.292303+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:40:40.097240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"77'9","reported_seq":34,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:44:27.416778+0000","last_change":"2024-09-21T03:44:12.312932+0000","last_active":"2024-09-21T03:44:27.416778+0000","last_peered":"2024-09-21T03:44:27.416778+0000","last_clean":"2024-09-21T03:44:27.416778+0000","last_became_active":"2024-09-21T03:44:12.312368+0000","last_became_peered":"2024-09-21T03:44:12.312368+0000","last_unstale":"2024-09-21T03:44:27.416778+0000","last_undegraded":"2024-09-21T03:44:27.416778+0000","last_fullsized":"2024-09-21T03:44:27.416778+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:11.282438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:11.282438+0000","last_clean_scrub_stamp":"2024-09-21T03:44:11.282438+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:28:11.579598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":18,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:44:27.410310+0000","last_change":"2024-09-21T03:44:14.329564+0000","last_active":"2024-09-21T03:44:27.410310+0000","last_peered":"2024-09-21T03:44:27.410310+0000","last_clean":"2024-09-21T03:44:27.410310+0000","last_became_active":"2024-09-21T03:44:14.329122+0000","last_became_peered":"2024-09-21T03:44:14.329122+0000","last_unstale":"2024-09-21T03:44:27.410310+0000","last_undegraded":"2024-09-21T03:44:27.410310+0000","last_fullsized":"2024-09-21T03:44:27.410310+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_clean_scrub_stamp":"2024-09-21T03:44:13.292303+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:16:32.702770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"77'5","reported_seq":29,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:44:27.413875+0000","last_change":"2024-09-21T03:44:12.303339+0000","last_active":"2024-09-21T03:44:27.413875+0000","last_peered":"2024-09-21T03:44:27.413875+0000","last_clean":"2024-09-21T03:44:27.413875+0000","last_became_active":"2024-09-21T03:44:12.302905+0000","last_became_peered":"2024-09-21T03:44:12.302905+0000","last_unstale":"2024-09-21T03:44:27.413875+0000","last_undegraded":"2024-09-21T03:44:27.413875+0000","last_fullsized":"2024-09-21T03:44:27.413875+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:11.282438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:11.282438+0000","last_clean_scrub_stamp":"2024-09-21T03:44:11.282438+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:57:16.424976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":18,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:44:27.418798+0000","last_change":"2024-09-21T03:44:14.323473+0000","last_active":"2024-09-21T03:44:27.418798+0000","last_peered":"2024-09-21T03:44:27.418798+0000","last_clean":"2024-09-21T03:44:27.418798+0000","last_became_active":"2024-09-21T03:44:14.322560+0000","last_became_peered":"2024-09-21T03:44:14.322560+0000","last_unstale":"2024-09-21T03:44:27.418798+0000","last_undegraded":"2024-09-21T03:44:27.418798+0000","last_fullsized":"2024-09-21T03:44:27.418798+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_clean_scrub_stamp":"2024-09-21T03:44:13.292303+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:07:55.714351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"79'6","reported_seq":48,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:44:38.720356+0000","last_change":"2024-09-21T03:44:12.312331+0000","last_active":"2024-09-21T03:44:38.720356+0000","last_peered":"2024-09-21T03:44:38.720356+0000","last_clean":"2024-09-21T03:44:38.720356+0000","last_became_active":"2024-09-21T03:44:12.311984+0000","last_became_peered":"2024-09-21T03:44:12.311984+0000","last_unstale":"2024-09-21T03:44:38.720356+0000","last_undegraded":"2024-09-21T03:44:38.720356+0000","last_fullsized":"2024-09-21T03:44:38.720356+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:11.282438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:11.282438+0000","last_clean_scrub_stamp":"2024-09-21T03:44:11.282438+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:06:11.906592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":979,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":20,"num_read_kb":4,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":18,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:44:27.418181+0000","last_change":"2024-09-21T03:44:14.318437+0000","last_active":"2024-09-21T03:44:27.418181+0000","last_peered":"2024-09-21T03:44:27.418181+0000","last_clean":"2024-09-21T03:44:27.418181+0000","last_became_active":"2024-09-21T03:44:14.318052+0000","last_became_peered":"2024-09-21T03:44:14.318052+0000","last_unstale":"2024-09-21T03:44:27.418181+0000","last_undegraded":"2024-09-21T03:44:27.418181+0000","last_fullsized":"2024-09-21T03:44:27.418181+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_clean_scrub_stamp":"2024-09-21T03:44:13.292303+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:10:12.136159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"77'17","reported_seq":41,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:44:27.419831+0000","last_change":"2024-09-21T03:44:12.310616+0000","last_active":"2024-09-21T03:44:27.419831+0000","last_peered":"2024-09-21T03:44:27.419831+0000","last_clean":"2024-09-21T03:44:27.419831+0000","last_became_active":"2024-09-21T03:44:12.310239+0000","last_became_peered":"2024-09-21T03:44:12.310239+0000","last_unstale":"2024-09-21T03:44:27.419831+0000","last_undegraded":"2024-09-21T03:44:27.419831+0000","last_fullsized":"2024-09-21T03:44:27.419831+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:11.282438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:11.282438+0000","last_clean_scrub_stamp":"2024-09-21T03:44:11.282438+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:43:38.046281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":18,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:44:27.413822+0000","last_change":"2024-09-21T03:44:14.327058+0000","last_active":"2024-09-21T03:44:27.413822+0000","last_peered":"2024-09-21T03:44:27.413822+0000","last_clean":"2024-09-21T03:44:27.413822+0000","last_became_active":"2024-09-21T03:44:14.326641+0000","last_became_peered":"2024-09-21T03:44:14.326641+0000","last_unstale":"2024-09-21T03:44:27.413822+0000","last_undegraded":"2024-09-21T03:44:27.413822+0000","last_fullsized":"2024-09-21T03:44:27.413822+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_clean_scrub_stamp":"2024-09-21T03:44:13.292303+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:10:50.686102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"77'2","reported_seq":26,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:44:27.420648+0000","last_change":"2024-09-21T03:44:12.310426+0000","last_active":"2024-09-21T03:44:27.420648+0000","last_peered":"2024-09-21T03:44:27.420648+0000","last_clean":"2024-09-21T03:44:27.420648+0000","last_became_active":"2024-09-21T03:44:12.309231+0000","last_became_peered":"2024-09-21T03:44:12.309231+0000","last_unstale":"2024-09-21T03:44:27.420648+0000","last_undegraded":"2024-09-21T03:44:27.420648+0000","last_fullsized":"2024-09-21T03:44:27.420648+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:11.282438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:11.282438+0000","last_clean_scrub_stamp":"2024-09-21T03:44:11.282438+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:32:18.284741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":18,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:44:27.420390+0000","last_change":"2024-09-21T03:44:14.319179+0000","last_active":"2024-09-21T03:44:27.420390+0000","last_peered":"2024-09-21T03:44:27.420390+0000","last_clean":"2024-09-21T03:44:27.420390+0000","last_became_active":"2024-09-21T03:44:14.318826+0000","last_became_peered":"2024-09-21T03:44:14.318826+0000","last_unstale":"2024-09-21T03:44:27.420390+0000","last_undegraded":"2024-09-21T03:44:27.420390+0000","last_fullsized":"2024-09-21T03:44:27.420390+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_clean_scrub_stamp":"2024-09-21T03:44:13.292303+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:12:57.985087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.e","version":"77'5","reported_seq":28,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:44:27.410705+0000","last_change":"2024-09-21T03:44:12.315658+0000","last_active":"2024-09-21T03:44:27.410705+0000","last_peered":"2024-09-21T03:44:27.410705+0000","last_clean":"2024-09-21T03:44:27.410705+0000","last_became_active":"2024-09-21T03:44:12.315240+0000","last_became_peered":"2024-09-21T03:44:12.315240+0000","last_unstale":"2024-09-21T03:44:27.410705+0000","last_undegraded":"2024-09-21T03:44:27.410705+0000","last_fullsized":"2024-09-21T03:44:27.410705+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:11.282438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:11.282438+0000","last_clean_scrub_stamp":"2024-09-21T03:44:11.282438+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:22:07.493437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":18,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:44:27.419234+0000","last_change":"2024-09-21T03:44:14.328014+0000","last_active":"2024-09-21T03:44:27.419234+0000","last_peered":"2024-09-21T03:44:27.419234+0000","last_clean":"2024-09-21T03:44:27.419234+0000","last_became_active":"2024-09-21T03:44:14.327638+0000","last_became_peered":"2024-09-21T03:44:14.327638+0000","last_unstale":"2024-09-21T03:44:27.419234+0000","last_undegraded":"2024-09-21T03:44:27.419234+0000","last_fullsized":"2024-09-21T03:44:27.419234+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_clean_scrub_stamp":"2024-09-21T03:44:13.292303+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:48:32.542776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"77'3","reported_seq":27,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:44:27.414656+0000","last_change":"2024-09-21T03:44:12.309272+0000","last_active":"2024-09-21T03:44:27.414656+0000","last_peered":"2024-09-21T03:44:27.414656+0000","last_clean":"2024-09-21T03:44:27.414656+0000","last_became_active":"2024-09-21T03:44:12.308718+0000","last_became_peered":"2024-09-21T03:44:12.308718+0000","last_unstale":"2024-09-21T03:44:27.414656+0000","last_undegraded":"2024-09-21T03:44:27.414656+0000","last_fullsized":"2024-09-21T03:44:27.414656+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:11.282438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:11.282438+0000","last_clean_scrub_stamp":"2024-09-21T03:44:11.282438+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:03:49.433966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"79'1","reported_seq":19,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:44:27.418909+0000","last_change":"2024-09-21T03:44:14.322148+0000","last_active":"2024-09-21T03:44:27.418909+0000","last_peered":"2024-09-21T03:44:27.418909+0000","last_clean":"2024-09-21T03:44:27.418909+0000","last_became_active":"2024-09-21T03:44:14.321784+0000","last_became_peered":"2024-09-21T03:44:14.321784+0000","last_unstale":"2024-09-21T03:44:27.418909+0000","last_undegraded":"2024-09-21T03:44:27.418909+0000","last_fullsized":"2024-09-21T03:44:27.418909+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_clean_scrub_stamp":"2024-09-21T03:44:13.292303+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:40:34.887134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"77'5","reported_seq":32,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:44:37.439633+0000","last_change":"2024-09-21T03:44:27.427722+0000","last_active":"2024-09-21T03:44:37.439633+0000","last_peered":"2024-09-21T03:44:37.439633+0000","last_clean":"2024-09-21T03:44:37.439633+0000","last_became_active":"2024-09-21T03:44:27.419672+0000","last_became_peered":"2024-09-21T03:44:27.419672+0000","last_unstale":"2024-09-21T03:44:37.439633+0000","last_undegraded":"2024-09-21T03:44:37.439633+0000","last_fullsized":"2024-09-21T03:44:37.439633+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:11.282438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:11.282438+0000","last_clean_scrub_stamp":"2024-09-21T03:44:11.282438+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:24:31.890439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1170,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":12,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":1170,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,11,0],"acting":[1,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":18,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:44:27.419112+0000","last_change":"2024-09-21T03:44:14.316503+0000","last_active":"2024-09-21T03:44:27.419112+0000","last_peered":"2024-09-21T03:44:27.419112+0000","last_clean":"2024-09-21T03:44:27.419112+0000","last_became_active":"2024-09-21T03:44:14.316140+0000","last_became_peered":"2024-09-21T03:44:14.316140+0000","last_unstale":"2024-09-21T03:44:27.419112+0000","last_undegraded":"2024-09-21T03:44:27.419112+0000","last_fullsized":"2024-09-21T03:44:27.419112+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_clean_scrub_stamp":"2024-09-21T03:44:13.292303+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:00:30.647579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"21'92","reported_seq":96,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:44:27.418315+0000","last_change":"2024-09-21T03:43:01.568120+0000","last_active":"2024-09-21T03:44:27.418315+0000","last_peered":"2024-09-21T03:44:27.418315+0000","last_clean":"2024-09-21T03:44:27.418315+0000","last_became_active":"2024-09-21T03:43:01.476311+0000","last_became_peered":"2024-09-21T03:43:01.476311+0000","last_unstale":"2024-09-21T03:44:27.418315+0000","last_undegraded":"2024-09-21T03:44:27.418315+0000","last_fullsized":"2024-09-21T03:44:27.418315+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:39:30.121380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:39:30.121380+0000","last_clean_scrub_stamp":"2024-09-21T03:39:30.121380+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:40:27.967442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"77'6","reported_seq":32,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:44:27.613250+0000","last_change":"2024-09-21T03:44:12.311860+0000","last_active":"2024-09-21T03:44:27.613250+0000","last_peered":"2024-09-21T03:44:27.613250+0000","last_clean":"2024-09-21T03:44:27.613250+0000","last_became_active":"2024-09-21T03:44:12.311453+0000","last_became_peered":"2024-09-21T03:44:12.311453+0000","last_unstale":"2024-09-21T03:44:27.613250+0000","last_undegraded":"2024-09-21T03:44:27.613250+0000","last_fullsized":"2024-09-21T03:44:27.613250+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:11.282438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:11.282438+0000","last_clean_scrub_stamp":"2024-09-21T03:44:11.282438+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:23:40.550445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:44:27.418319+0000","last_change":"2024-09-21T03:44:27.416799+0000","last_active":"2024-09-21T03:44:27.418319+0000","last_peered":"2024-09-21T03:44:27.418319+0000","last_clean":"2024-09-21T03:44:27.418319+0000","last_became_active":"2024-09-21T03:44:27.416306+0000","last_became_peered":"2024-09-21T03:44:27.416306+0000","last_unstale":"2024-09-21T03:44:27.418319+0000","last_undegraded":"2024-09-21T03:44:27.418319+0000","last_fullsized":"2024-09-21T03:44:27.418319+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_clean_scrub_stamp":"2024-09-21T03:44:13.292303+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:34:26.547100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,5,1],"acting":[9,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.0","version":"77'4","reported_seq":27,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:44:27.418313+0000","last_change":"2024-09-21T03:44:12.307302+0000","last_active":"2024-09-21T03:44:27.418313+0000","last_peered":"2024-09-21T03:44:27.418313+0000","last_clean":"2024-09-21T03:44:27.418313+0000","last_became_active":"2024-09-21T03:44:12.306856+0000","last_became_peered":"2024-09-21T03:44:12.306856+0000","last_unstale":"2024-09-21T03:44:27.418313+0000","last_undegraded":"2024-09-21T03:44:27.418313+0000","last_fullsized":"2024-09-21T03:44:27.418313+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:11.282438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:11.282438+0000","last_clean_scrub_stamp":"2024-09-21T03:44:11.282438+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:21:20.420020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":18,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:44:27.410210+0000","last_change":"2024-09-21T03:44:14.322913+0000","last_active":"2024-09-21T03:44:27.410210+0000","last_peered":"2024-09-21T03:44:27.410210+0000","last_clean":"2024-09-21T03:44:27.410210+0000","last_became_active":"2024-09-21T03:44:14.322168+0000","last_became_peered":"2024-09-21T03:44:14.322168+0000","last_unstale":"2024-09-21T03:44:27.410210+0000","last_undegraded":"2024-09-21T03:44:27.410210+0000","last_fullsized":"2024-09-21T03:44:27.410210+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_clean_scrub_stamp":"2024-09-21T03:44:13.292303+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:30:09.041211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"77'5","reported_seq":42,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:44:35.714736+0000","last_change":"2024-09-21T03:44:12.309084+0000","last_active":"2024-09-21T03:44:35.714736+0000","last_peered":"2024-09-21T03:44:35.714736+0000","last_clean":"2024-09-21T03:44:35.714736+0000","last_became_active":"2024-09-21T03:44:12.308551+0000","last_became_peered":"2024-09-21T03:44:12.308551+0000","last_unstale":"2024-09-21T03:44:35.714736+0000","last_undegraded":"2024-09-21T03:44:35.714736+0000","last_fullsized":"2024-09-21T03:44:35.714736+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:11.282438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:11.282438+0000","last_clean_scrub_stamp":"2024-09-21T03:44:11.282438+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:46:33.895904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":14,"num_read_kb":14,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":18,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:44:27.413911+0000","last_change":"2024-09-21T03:44:14.322720+0000","last_active":"2024-09-21T03:44:27.413911+0000","last_peered":"2024-09-21T03:44:27.413911+0000","last_clean":"2024-09-21T03:44:27.413911+0000","last_became_active":"2024-09-21T03:44:14.322367+0000","last_became_peered":"2024-09-21T03:44:14.322367+0000","last_unstale":"2024-09-21T03:44:27.413911+0000","last_undegraded":"2024-09-21T03:44:27.413911+0000","last_fullsized":"2024-09-21T03:44:27.413911+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_clean_scrub_stamp":"2024-09-21T03:44:13.292303+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:59:51.692976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"79'6","reported_seq":44,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:44:37.630380+0000","last_change":"2024-09-21T03:44:12.308719+0000","last_active":"2024-09-21T03:44:37.630380+0000","last_peered":"2024-09-21T03:44:37.630380+0000","last_clean":"2024-09-21T03:44:37.630380+0000","last_became_active":"2024-09-21T03:44:12.308380+0000","last_became_peered":"2024-09-21T03:44:12.308380+0000","last_unstale":"2024-09-21T03:44:37.630380+0000","last_undegraded":"2024-09-21T03:44:37.630380+0000","last_fullsized":"2024-09-21T03:44:37.630380+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:11.282438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:11.282438+0000","last_clean_scrub_stamp":"2024-09-21T03:44:11.282438+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:41:02.087501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":979,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":15,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":18,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:44:27.612118+0000","last_change":"2024-09-21T03:44:14.326541+0000","last_active":"2024-09-21T03:44:27.612118+0000","last_peered":"2024-09-21T03:44:27.612118+0000","last_clean":"2024-09-21T03:44:27.612118+0000","last_became_active":"2024-09-21T03:44:14.326132+0000","last_became_peered":"2024-09-21T03:44:14.326132+0000","last_unstale":"2024-09-21T03:44:27.612118+0000","last_undegraded":"2024-09-21T03:44:27.612118+0000","last_fullsized":"2024-09-21T03:44:27.612118+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_clean_scrub_stamp":"2024-09-21T03:44:13.292303+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:09:27.711919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"77'1","reported_seq":24,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:44:27.414339+0000","last_change":"2024-09-21T03:44:12.309463+0000","last_active":"2024-09-21T03:44:27.414339+0000","last_peered":"2024-09-21T03:44:27.414339+0000","last_clean":"2024-09-21T03:44:27.414339+0000","last_became_active":"2024-09-21T03:44:12.309054+0000","last_became_peered":"2024-09-21T03:44:12.309054+0000","last_unstale":"2024-09-21T03:44:27.414339+0000","last_undegraded":"2024-09-21T03:44:27.414339+0000","last_fullsized":"2024-09-21T03:44:27.414339+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:11.282438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:11.282438+0000","last_clean_scrub_stamp":"2024-09-21T03:44:11.282438+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:12:12.505538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"79'1","reported_seq":19,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:44:27.612000+0000","last_change":"2024-09-21T03:44:14.318261+0000","last_active":"2024-09-21T03:44:27.612000+0000","last_peered":"2024-09-21T03:44:27.612000+0000","last_clean":"2024-09-21T03:44:27.612000+0000","last_became_active":"2024-09-21T03:44:14.317839+0000","last_became_peered":"2024-09-21T03:44:14.317839+0000","last_unstale":"2024-09-21T03:44:27.612000+0000","last_undegraded":"2024-09-21T03:44:27.612000+0000","last_fullsized":"2024-09-21T03:44:27.612000+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_clean_scrub_stamp":"2024-09-21T03:44:13.292303+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:19:05.504749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.5","version":"77'4","reported_seq":20,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:44:27.434085+0000","last_change":"2024-09-21T03:44:27.433630+0000","last_active":"2024-09-21T03:44:27.434085+0000","last_peered":"2024-09-21T03:44:27.434085+0000","last_clean":"2024-09-21T03:44:27.434085+0000","last_became_active":"2024-09-21T03:44:27.422093+0000","last_became_peered":"2024-09-21T03:44:27.422093+0000","last_unstale":"2024-09-21T03:44:27.434085+0000","last_undegraded":"2024-09-21T03:44:27.434085+0000","last_fullsized":"2024-09-21T03:44:27.434085+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:11.282438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:11.282438+0000","last_clean_scrub_stamp":"2024-09-21T03:44:11.282438+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:39:42.651024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,9],"acting":[11,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":18,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:44:27.410677+0000","last_change":"2024-09-21T03:44:14.322620+0000","last_active":"2024-09-21T03:44:27.410677+0000","last_peered":"2024-09-21T03:44:27.410677+0000","last_clean":"2024-09-21T03:44:27.410677+0000","last_became_active":"2024-09-21T03:44:14.322214+0000","last_became_peered":"2024-09-21T03:44:14.322214+0000","last_unstale":"2024-09-21T03:44:27.410677+0000","last_undegraded":"2024-09-21T03:44:27.410677+0000","last_fullsized":"2024-09-21T03:44:27.410677+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_clean_scrub_stamp":"2024-09-21T03:44:13.292303+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:53:20.519168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"79'13","reported_seq":36,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:44:36.340306+0000","last_change":"2024-09-21T03:44:12.311302+0000","last_active":"2024-09-21T03:44:36.340306+0000","last_peered":"2024-09-21T03:44:36.340306+0000","last_clean":"2024-09-21T03:44:36.340306+0000","last_became_active":"2024-09-21T03:44:12.310947+0000","last_became_peered":"2024-09-21T03:44:12.310947+0000","last_unstale":"2024-09-21T03:44:36.340306+0000","last_undegraded":"2024-09-21T03:44:36.340306+0000","last_fullsized":"2024-09-21T03:44:36.340306+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:11.282438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:11.282438+0000","last_clean_scrub_stamp":"2024-09-21T03:44:11.282438+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:09:34.308726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":22,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":18,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:44:27.611222+0000","last_change":"2024-09-21T03:44:14.324675+0000","last_active":"2024-09-21T03:44:27.611222+0000","last_peered":"2024-09-21T03:44:27.611222+0000","last_clean":"2024-09-21T03:44:27.611222+0000","last_became_active":"2024-09-21T03:44:14.324214+0000","last_became_peered":"2024-09-21T03:44:14.324214+0000","last_unstale":"2024-09-21T03:44:27.611222+0000","last_undegraded":"2024-09-21T03:44:27.611222+0000","last_fullsized":"2024-09-21T03:44:27.611222+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_clean_scrub_stamp":"2024-09-21T03:44:13.292303+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:33:51.071794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"77'7","reported_seq":31,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:44:27.613637+0000","last_change":"2024-09-21T03:44:12.322312+0000","last_active":"2024-09-21T03:44:27.613637+0000","last_peered":"2024-09-21T03:44:27.613637+0000","last_clean":"2024-09-21T03:44:27.613637+0000","last_became_active":"2024-09-21T03:44:12.322018+0000","last_became_peered":"2024-09-21T03:44:12.322018+0000","last_unstale":"2024-09-21T03:44:27.613637+0000","last_undegraded":"2024-09-21T03:44:27.613637+0000","last_fullsized":"2024-09-21T03:44:27.613637+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:11.282438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:11.282438+0000","last_clean_scrub_stamp":"2024-09-21T03:44:11.282438+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:11:41.817574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":18,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:44:27.417254+0000","last_change":"2024-09-21T03:44:14.326682+0000","last_active":"2024-09-21T03:44:27.417254+0000","last_peered":"2024-09-21T03:44:27.417254+0000","last_clean":"2024-09-21T03:44:27.417254+0000","last_became_active":"2024-09-21T03:44:14.326313+0000","last_became_peered":"2024-09-21T03:44:14.326313+0000","last_unstale":"2024-09-21T03:44:27.417254+0000","last_undegraded":"2024-09-21T03:44:27.417254+0000","last_fullsized":"2024-09-21T03:44:27.417254+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_clean_scrub_stamp":"2024-09-21T03:44:13.292303+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:26:44.681245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"77'2","reported_seq":36,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:44:27.436319+0000","last_change":"2024-09-21T03:44:27.436245+0000","last_active":"2024-09-21T03:44:27.436319+0000","last_peered":"2024-09-21T03:44:27.436319+0000","last_clean":"2024-09-21T03:44:27.436319+0000","last_became_active":"2024-09-21T03:44:27.422134+0000","last_became_peered":"2024-09-21T03:44:27.422134+0000","last_unstale":"2024-09-21T03:44:27.436319+0000","last_undegraded":"2024-09-21T03:44:27.436319+0000","last_fullsized":"2024-09-21T03:44:27.436319+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:11.282438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:11.282438+0000","last_clean_scrub_stamp":"2024-09-21T03:44:11.282438+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:50:22.655050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,11,0],"acting":[1,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":18,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:44:27.419704+0000","last_change":"2024-09-21T03:44:14.316312+0000","last_active":"2024-09-21T03:44:27.419704+0000","last_peered":"2024-09-21T03:44:27.419704+0000","last_clean":"2024-09-21T03:44:27.419704+0000","last_became_active":"2024-09-21T03:44:14.315929+0000","last_became_peered":"2024-09-21T03:44:14.315929+0000","last_unstale":"2024-09-21T03:44:27.419704+0000","last_undegraded":"2024-09-21T03:44:27.419704+0000","last_fullsized":"2024-09-21T03:44:27.419704+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_clean_scrub_stamp":"2024-09-21T03:44:13.292303+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:31:33.071983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"77'7","reported_seq":30,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:44:27.610761+0000","last_change":"2024-09-21T03:44:12.312655+0000","last_active":"2024-09-21T03:44:27.610761+0000","last_peered":"2024-09-21T03:44:27.610761+0000","last_clean":"2024-09-21T03:44:27.610761+0000","last_became_active":"2024-09-21T03:44:12.312196+0000","last_became_peered":"2024-09-21T03:44:12.312196+0000","last_unstale":"2024-09-21T03:44:27.610761+0000","last_undegraded":"2024-09-21T03:44:27.610761+0000","last_fullsized":"2024-09-21T03:44:27.610761+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:11.282438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:11.282438+0000","last_clean_scrub_stamp":"2024-09-21T03:44:11.282438+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:24:32.342461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":18,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:44:27.418552+0000","last_change":"2024-09-21T03:44:14.321875+0000","last_active":"2024-09-21T03:44:27.418552+0000","last_peered":"2024-09-21T03:44:27.418552+0000","last_clean":"2024-09-21T03:44:27.418552+0000","last_became_active":"2024-09-21T03:44:14.320682+0000","last_became_peered":"2024-09-21T03:44:14.320682+0000","last_unstale":"2024-09-21T03:44:27.418552+0000","last_undegraded":"2024-09-21T03:44:27.418552+0000","last_fullsized":"2024-09-21T03:44:27.418552+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_clean_scrub_stamp":"2024-09-21T03:44:13.292303+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:41:32.435288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"77'3","reported_seq":26,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:44:27.418692+0000","last_change":"2024-09-21T03:44:12.309100+0000","last_active":"2024-09-21T03:44:27.418692+0000","last_peered":"2024-09-21T03:44:27.418692+0000","last_clean":"2024-09-21T03:44:27.418692+0000","last_became_active":"2024-09-21T03:44:12.308671+0000","last_became_peered":"2024-09-21T03:44:12.308671+0000","last_unstale":"2024-09-21T03:44:27.418692+0000","last_undegraded":"2024-09-21T03:44:27.418692+0000","last_fullsized":"2024-09-21T03:44:27.418692+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:11.282438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:11.282438+0000","last_clean_scrub_stamp":"2024-09-21T03:44:11.282438+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:09:19.347980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":18,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:44:27.610670+0000","last_change":"2024-09-21T03:44:14.320170+0000","last_active":"2024-09-21T03:44:27.610670+0000","last_peered":"2024-09-21T03:44:27.610670+0000","last_clean":"2024-09-21T03:44:27.610670+0000","last_became_active":"2024-09-21T03:44:14.319794+0000","last_became_peered":"2024-09-21T03:44:14.319794+0000","last_unstale":"2024-09-21T03:44:27.610670+0000","last_undegraded":"2024-09-21T03:44:27.610670+0000","last_fullsized":"2024-09-21T03:44:27.610670+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_clean_scrub_stamp":"2024-09-21T03:44:13.292303+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:42:09.684471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"77'4","reported_seq":28,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:44:27.420077+0000","last_change":"2024-09-21T03:44:12.310851+0000","last_active":"2024-09-21T03:44:27.420077+0000","last_peered":"2024-09-21T03:44:27.420077+0000","last_clean":"2024-09-21T03:44:27.420077+0000","last_became_active":"2024-09-21T03:44:12.310302+0000","last_became_peered":"2024-09-21T03:44:12.310302+0000","last_unstale":"2024-09-21T03:44:27.420077+0000","last_undegraded":"2024-09-21T03:44:27.420077+0000","last_fullsized":"2024-09-21T03:44:27.420077+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:11.282438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:11.282438+0000","last_clean_scrub_stamp":"2024-09-21T03:44:11.282438+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:23:46.959121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":18,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:44:27.418894+0000","last_change":"2024-09-21T03:44:14.324544+0000","last_active":"2024-09-21T03:44:27.418894+0000","last_peered":"2024-09-21T03:44:27.418894+0000","last_clean":"2024-09-21T03:44:27.418894+0000","last_became_active":"2024-09-21T03:44:14.322881+0000","last_became_peered":"2024-09-21T03:44:14.322881+0000","last_unstale":"2024-09-21T03:44:27.418894+0000","last_undegraded":"2024-09-21T03:44:27.418894+0000","last_fullsized":"2024-09-21T03:44:27.418894+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_clean_scrub_stamp":"2024-09-21T03:44:13.292303+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:58:42.760185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":25,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:44:27.422566+0000","last_change":"2024-09-21T03:44:27.422438+0000","last_active":"2024-09-21T03:44:27.422566+0000","last_peered":"2024-09-21T03:44:27.422566+0000","last_clean":"2024-09-21T03:44:27.422566+0000","last_became_active":"2024-09-21T03:44:27.421935+0000","last_became_peered":"2024-09-21T03:44:27.421935+0000","last_unstale":"2024-09-21T03:44:27.422566+0000","last_undegraded":"2024-09-21T03:44:27.422566+0000","last_fullsized":"2024-09-21T03:44:27.422566+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_clean_scrub_stamp":"2024-09-21T03:44:13.292303+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:20:50.097783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,1],"acting":[5,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"77'5","reported_seq":29,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:44:27.420510+0000","last_change":"2024-09-21T03:44:12.311509+0000","last_active":"2024-09-21T03:44:27.420510+0000","last_peered":"2024-09-21T03:44:27.420510+0000","last_clean":"2024-09-21T03:44:27.420510+0000","last_became_active":"2024-09-21T03:44:12.310880+0000","last_became_peered":"2024-09-21T03:44:12.310880+0000","last_unstale":"2024-09-21T03:44:27.420510+0000","last_undegraded":"2024-09-21T03:44:27.420510+0000","last_fullsized":"2024-09-21T03:44:27.420510+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:11.282438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:11.282438+0000","last_clean_scrub_stamp":"2024-09-21T03:44:11.282438+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:59:33.887234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":18,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:44:27.413781+0000","last_change":"2024-09-21T03:44:14.330938+0000","last_active":"2024-09-21T03:44:27.413781+0000","last_peered":"2024-09-21T03:44:27.413781+0000","last_clean":"2024-09-21T03:44:27.413781+0000","last_became_active":"2024-09-21T03:44:14.330559+0000","last_became_peered":"2024-09-21T03:44:14.330559+0000","last_unstale":"2024-09-21T03:44:27.413781+0000","last_undegraded":"2024-09-21T03:44:27.413781+0000","last_fullsized":"2024-09-21T03:44:27.413781+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_clean_scrub_stamp":"2024-09-21T03:44:13.292303+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:05:32.535660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"77'12","reported_seq":36,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:44:27.420563+0000","last_change":"2024-09-21T03:44:12.307667+0000","last_active":"2024-09-21T03:44:27.420563+0000","last_peered":"2024-09-21T03:44:27.420563+0000","last_clean":"2024-09-21T03:44:27.420563+0000","last_became_active":"2024-09-21T03:44:12.307074+0000","last_became_peered":"2024-09-21T03:44:12.307074+0000","last_unstale":"2024-09-21T03:44:27.420563+0000","last_undegraded":"2024-09-21T03:44:27.420563+0000","last_fullsized":"2024-09-21T03:44:27.420563+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:11.282438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:11.282438+0000","last_clean_scrub_stamp":"2024-09-21T03:44:11.282438+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:48:41.598375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":18,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:44:27.417322+0000","last_change":"2024-09-21T03:44:14.325487+0000","last_active":"2024-09-21T03:44:27.417322+0000","last_peered":"2024-09-21T03:44:27.417322+0000","last_clean":"2024-09-21T03:44:27.417322+0000","last_became_active":"2024-09-21T03:44:14.325153+0000","last_became_peered":"2024-09-21T03:44:14.325153+0000","last_unstale":"2024-09-21T03:44:27.417322+0000","last_undegraded":"2024-09-21T03:44:27.417322+0000","last_fullsized":"2024-09-21T03:44:27.417322+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_clean_scrub_stamp":"2024-09-21T03:44:13.292303+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:29:17.619130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"77'2","reported_seq":25,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:44:27.611241+0000","last_change":"2024-09-21T03:44:12.307635+0000","last_active":"2024-09-21T03:44:27.611241+0000","last_peered":"2024-09-21T03:44:27.611241+0000","last_clean":"2024-09-21T03:44:27.611241+0000","last_became_active":"2024-09-21T03:44:12.307235+0000","last_became_peered":"2024-09-21T03:44:12.307235+0000","last_unstale":"2024-09-21T03:44:27.611241+0000","last_undegraded":"2024-09-21T03:44:27.611241+0000","last_fullsized":"2024-09-21T03:44:27.611241+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:11.282438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:11.282438+0000","last_clean_scrub_stamp":"2024-09-21T03:44:11.282438+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:18:59.985369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":18,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:44:27.611117+0000","last_change":"2024-09-21T03:44:14.324660+0000","last_active":"2024-09-21T03:44:27.611117+0000","last_peered":"2024-09-21T03:44:27.611117+0000","last_clean":"2024-09-21T03:44:27.611117+0000","last_became_active":"2024-09-21T03:44:14.324309+0000","last_became_peered":"2024-09-21T03:44:14.324309+0000","last_unstale":"2024-09-21T03:44:27.611117+0000","last_undegraded":"2024-09-21T03:44:27.611117+0000","last_fullsized":"2024-09-21T03:44:27.611117+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_clean_scrub_stamp":"2024-09-21T03:44:13.292303+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:13:07.806721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"77'13","reported_seq":72,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:44:38.630977+0000","last_change":"2024-09-21T03:44:27.623910+0000","last_active":"2024-09-21T03:44:38.630977+0000","last_peered":"2024-09-21T03:44:38.630977+0000","last_clean":"2024-09-21T03:44:38.630977+0000","last_became_active":"2024-09-21T03:44:27.614037+0000","last_became_peered":"2024-09-21T03:44:27.614037+0000","last_unstale":"2024-09-21T03:44:38.630977+0000","last_undegraded":"2024-09-21T03:44:38.630977+0000","last_fullsized":"2024-09-21T03:44:38.630977+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:11.282438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:11.282438+0000","last_clean_scrub_stamp":"2024-09-21T03:44:11.282438+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:03:56.753935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":14,"num_read_kb":14,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":672,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,4],"acting":[9,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":58535,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":174,"num_read_kb":150,"num_write":234,"num_write_kb":124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13,"num_bytes_recovered":1954,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":548864,"data_stored":183757,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":202,"ondisk_log_size":202,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":71,"seq":304942678034,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27892,"kb_used_data":864,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717260,"statfs":{"total":95995035648,"available":95966474240,"internally_reserved":0,"allocated":884736,"data_stored":428272,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":65,"seq":279172874262,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28556,"kb_used_data":1532,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716596,"statfs":{"total":95995035648,"available":95965794304,"internally_reserved":0,"allocated":1568768,"data_stored":1070520,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":60,"seq":257698037787,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27904,"kb_used_data":880,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717248,"statfs":{"total":95995035648,"available":95966461952,"internally_reserved":0,"allocated":901120,"data_stored":428090,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":54,"seq":231928234015,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27884,"kb_used_data":860,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717268,"statfs":{"total":95995035648,"available":95966482432,"internally_reserved":0,"allocated":880640,"data_stored":427684,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":49,"seq":210453397543,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36804,"kb_used_data":1460,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708348,"statfs":{"total":95995035648,"available":95957348352,"internally_reserved":0,"allocated":1495040,"data_stored":1021768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":36174282},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":43,"seq":184683593771,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27888,"kb_used_data":864,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717264,"statfs":{"total":95995035648,"available":95966478336,"internally_reserved":0,"allocated":884736,"data_stored":428198,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":37,"seq":158913790001,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32656,"kb_used_data":1472,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712496,"statfs":{"total":95995035648,"available":95961595904,"internally_reserved":0,"allocated":1507328,"data_stored":1020664,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":31,"seq":133143986230,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36224,"kb_used_data":872,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708928,"statfs":{"total":95995035648,"available":95957942272,"internally_reserved":0,"allocated":892928,"data_stored":428731,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149755,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32064,"kb_used_data":880,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713088,"statfs":{"total":95995035648,"available":95962202112,"internally_reserved":0,"allocated":901120,"data_stored":427632,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":19,"seq":81604378689,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27960,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717192,"statfs":{"total":95995035648,"available":95966404608,"internally_reserved":0,"allocated":958464,"data_stored":487253,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542213,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27948,"kb_used_data":924,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717204,"statfs":{"total":95995035648,"available":95966416896,"internally_reserved":0,"allocated":946176,"data_stored":481221,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705739,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27912,"kb_used_data":884,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717240,"statfs":{"total":95995035648,"available":95966453760,"internally_reserved":0,"allocated":905216,"data_stored":429146,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2178,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":54253,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":60285,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1763,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3211,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1113,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":4315,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":716,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":106496,"data_stored":53067,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1304,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T03:44:42.818 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-09-21T03:44:42.819 INFO:tasks.cephadm:Setting up client nodes... 2024-09-21T03:44:42.819 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 51116d5c-77ca-11ef-baf5-efdc52797490 -- ceph auth get-or-create client.0 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2024-09-21T03:44:42.844 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:44:42 smithi039 ceph-mon[31211]: pgmap v285: 97 pgs: 97 active+clean; 634 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 5.0 KiB/s wr, 12 op/s; 65 B/s, 0 objects/s recovering 2024-09-21T03:44:42.845 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:44:42 smithi039 ceph-mon[31211]: from='client.15006 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T03:44:42.967 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:44:42 smithi161 ceph-mon[34533]: pgmap v285: 97 pgs: 97 active+clean; 634 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 5.0 KiB/s wr, 12 op/s; 65 B/s, 0 objects/s recovering 2024-09-21T03:44:42.968 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:44:42 smithi161 ceph-mon[34533]: from='client.15006 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T03:44:42.974 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:44:42 smithi150 ceph-mon[34289]: pgmap v285: 97 pgs: 97 active+clean; 634 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 5.0 KiB/s wr, 12 op/s; 65 B/s, 0 objects/s recovering 2024-09-21T03:44:42.974 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:44:42 smithi150 ceph-mon[34289]: from='client.15006 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T03:44:43.079 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/51116d5c-77ca-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:44:43.531 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:43.529+0000 7f11af300640 1 -- 172.21.15.39:0/2420665088 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f11a8102a70 msgr2=0x7f11a8102e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:43.531 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:43.529+0000 7f11af300640 1 --2- 172.21.15.39:0/2420665088 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f11a8102a70 0x7f11a8102e80 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f11a4009900 tx=0x7f11a402a410 comp rx=0 tx=0).stop 2024-09-21T03:44:43.531 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:43.529+0000 7f11af300640 1 -- 172.21.15.39:0/2420665088 shutdown_connections 2024-09-21T03:44:43.531 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:43.529+0000 7f11af300640 1 --2- 172.21.15.39:0/2420665088 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f11a8108d00 0x7f11a810f530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:43.531 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:43.529+0000 7f11af300640 1 --2- 172.21.15.39:0/2420665088 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f11a81083b0 0x7f11a81087c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:43.531 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:43.529+0000 7f11af300640 1 --2- 172.21.15.39:0/2420665088 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f11a8102a70 0x7f11a8102e80 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:43.532 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:43.529+0000 7f11af300640 1 -- 172.21.15.39:0/2420665088 >> 172.21.15.39:0/2420665088 conn(0x7f11a8079870 msgr2=0x7f11a8079c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:44:43.532 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:43.529+0000 7f11af300640 1 -- 172.21.15.39:0/2420665088 shutdown_connections 2024-09-21T03:44:43.532 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:43.529+0000 7f11af300640 1 -- 172.21.15.39:0/2420665088 wait complete. 2024-09-21T03:44:43.533 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:43.530+0000 7f11af300640 1 Processor -- start 2024-09-21T03:44:43.533 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:43.530+0000 7f11af300640 1 -- start start 2024-09-21T03:44:43.533 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:43.530+0000 7f11af300640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f11a8102a70 0x7f11a819a460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:43.533 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:43.530+0000 7f11af300640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f11a81083b0 0x7f11a819aa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:43.534 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:43.530+0000 7f11af300640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f11a8108d00 0x7f11a819fca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:43.534 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:43.530+0000 7f11af300640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f11a81129c0 con 0x7f11a81083b0 2024-09-21T03:44:43.534 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:43.530+0000 7f11af300640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f11a8112840 con 0x7f11a8108d00 2024-09-21T03:44:43.534 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:43.530+0000 7f11af300640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f11a8112b40 con 0x7f11a8102a70 2024-09-21T03:44:43.534 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:43.531+0000 7f11ac89b640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f11a81083b0 0x7f11a819aa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:43.534 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:43.531+0000 7f11ac89b640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f11a81083b0 0x7f11a819aa00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:53876/0 (socket says 172.21.15.39:53876) 2024-09-21T03:44:43.534 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:43.531+0000 7f11ac89b640 1 -- 172.21.15.39:0/3367922287 learned_addr learned my addr 172.21.15.39:0/3367922287 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:44:43.534 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:43.531+0000 7f11ad89d640 1 --2- 172.21.15.39:0/3367922287 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f11a8108d00 0x7f11a819fca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:43.534 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:43.531+0000 7f11ad09c640 1 --2- 172.21.15.39:0/3367922287 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f11a8102a70 0x7f11a819a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:43.534 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:43.531+0000 7f11ac89b640 1 -- 172.21.15.39:0/3367922287 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f11a8102a70 msgr2=0x7f11a819a460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:43.534 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:43.531+0000 7f11ac89b640 1 --2- 172.21.15.39:0/3367922287 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f11a8102a70 0x7f11a819a460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:43.534 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:43.531+0000 7f11ac89b640 1 -- 172.21.15.39:0/3367922287 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f11a8108d00 msgr2=0x7f11a819fca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:43.535 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:43.531+0000 7f11ac89b640 1 --2- 172.21.15.39:0/3367922287 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f11a8108d00 0x7f11a819fca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:43.535 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:43.531+0000 7f11ac89b640 1 -- 172.21.15.39:0/3367922287 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f11a81a03d0 con 0x7f11a81083b0 2024-09-21T03:44:43.535 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:43.531+0000 7f11ad09c640 1 --2- 172.21.15.39:0/3367922287 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f11a8102a70 0x7f11a819a460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:44:43.535 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:43.531+0000 7f11ac89b640 1 --2- 172.21.15.39:0/3367922287 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f11a81083b0 0x7f11a819aa00 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f119800d750 tx=0x7f119800dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:44:43.535 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:43.531+0000 7f119e7fc640 1 -- 172.21.15.39:0/3367922287 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1198017070 con 0x7f11a81083b0 2024-09-21T03:44:43.535 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:43.531+0000 7f119e7fc640 1 -- 172.21.15.39:0/3367922287 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f11980054d0 con 0x7f11a81083b0 2024-09-21T03:44:43.535 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:43.531+0000 7f11af300640 1 -- 172.21.15.39:0/3367922287 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f11a81a06c0 con 0x7f11a81083b0 2024-09-21T03:44:43.535 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:43.531+0000 7f119e7fc640 1 -- 172.21.15.39:0/3367922287 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1198004c30 con 0x7f11a81083b0 2024-09-21T03:44:43.535 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:43.531+0000 7f11af300640 1 -- 172.21.15.39:0/3367922287 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f11a81a6e20 con 0x7f11a81083b0 2024-09-21T03:44:43.536 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:43.533+0000 7f11af300640 1 -- 172.21.15.39:0/3367922287 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f11a8102e80 con 0x7f11a81083b0 2024-09-21T03:44:43.536 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:43.534+0000 7f119e7fc640 1 -- 172.21.15.39:0/3367922287 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 22) ==== 154580+0+0 (secure 0 0 0) 0x7f1198004dd0 con 0x7f11a81083b0 2024-09-21T03:44:43.537 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:43.535+0000 7f119e7fc640 1 --2- 172.21.15.39:0/3367922287 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f11940b65f0 0x7f11940b8b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:43.537 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:43.535+0000 7f11ad09c640 1 --2- 172.21.15.39:0/3367922287 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f11940b65f0 0x7f11940b8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:43.537 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:43.535+0000 7f119e7fc640 1 -- 172.21.15.39:0/3367922287 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8484+0+0 (secure 0 0 0) 0x7f11980e73c0 con 0x7f11a81083b0 2024-09-21T03:44:43.537 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:43.536+0000 7f11ad09c640 1 --2- 172.21.15.39:0/3367922287 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f11940b65f0 0x7f11940b8b00 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f11a4031440 tx=0x7f11a4009680 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:44:43.540 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:43.538+0000 7f119e7fc640 1 -- 172.21.15.39:0/3367922287 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f11980ad9f0 con 0x7f11a81083b0 2024-09-21T03:44:43.801 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:43.799+0000 7f11af300640 1 -- 172.21.15.39:0/3367922287 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} v 0) -- 0x7f11a819b590 con 0x7f11a81083b0 2024-09-21T03:44:43.804 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:43.803+0000 7f119e7fc640 1 -- 172.21.15.39:0/3367922287 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]=0 v32) ==== 170+0+59 (secure 0 0 0) 0x7f11a819b590 con 0x7f11a81083b0 2024-09-21T03:44:43.804 INFO:teuthology.orchestra.run.smithi039.stdout:[client.0] 2024-09-21T03:44:43.804 INFO:teuthology.orchestra.run.smithi039.stdout: key = AQArQe5mg4u+LxAAT28lMfRrIHbp0eVpIdybyQ== 2024-09-21T03:44:43.807 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:43.805+0000 7f11af300640 1 -- 172.21.15.39:0/3367922287 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f11940b65f0 msgr2=0x7f11940b8b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:43.807 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:43.805+0000 7f11af300640 1 --2- 172.21.15.39:0/3367922287 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f11940b65f0 0x7f11940b8b00 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f11a4031440 tx=0x7f11a4009680 comp rx=0 tx=0).stop 2024-09-21T03:44:43.807 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:43.805+0000 7f11af300640 1 -- 172.21.15.39:0/3367922287 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f11a81083b0 msgr2=0x7f11a819aa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:43.807 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:43.805+0000 7f11af300640 1 --2- 172.21.15.39:0/3367922287 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f11a81083b0 0x7f11a819aa00 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f119800d750 tx=0x7f119800dc10 comp rx=0 tx=0).stop 2024-09-21T03:44:43.807 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:43.805+0000 7f11af300640 1 -- 172.21.15.39:0/3367922287 shutdown_connections 2024-09-21T03:44:43.807 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:43.805+0000 7f11af300640 1 --2- 172.21.15.39:0/3367922287 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f11940b65f0 0x7f11940b8b00 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:43.807 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:43.805+0000 7f11af300640 1 --2- 172.21.15.39:0/3367922287 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f11a8108d00 0x7f11a819fca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:43.808 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:43.805+0000 7f11af300640 1 --2- 172.21.15.39:0/3367922287 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f11a81083b0 0x7f11a819aa00 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:43.808 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:43.805+0000 7f11af300640 1 --2- 172.21.15.39:0/3367922287 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f11a8102a70 0x7f11a819a460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:43.808 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:43.805+0000 7f11af300640 1 -- 172.21.15.39:0/3367922287 >> 172.21.15.39:0/3367922287 conn(0x7f11a8079870 msgr2=0x7f11a806bb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:44:43.808 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:43.805+0000 7f11af300640 1 -- 172.21.15.39:0/3367922287 shutdown_connections 2024-09-21T03:44:43.808 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:43.805+0000 7f11af300640 1 -- 172.21.15.39:0/3367922287 wait complete. 2024-09-21T03:44:44.529 DEBUG:teuthology.orchestra.run.smithi039:> set -ex 2024-09-21T03:44:44.530 DEBUG:teuthology.orchestra.run.smithi039:> sudo dd of=/etc/ceph/ceph.client.0.keyring 2024-09-21T03:44:44.530 DEBUG:teuthology.orchestra.run.smithi039:> sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-09-21T03:44:44.610 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 51116d5c-77ca-11ef-baf5-efdc52797490 -- ceph config log 1 --format=json 2024-09-21T03:44:44.833 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:44:44 smithi039 ceph-mon[31211]: pgmap v286: 97 pgs: 97 active+clean; 634 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 4.7 KiB/s wr, 11 op/s 2024-09-21T03:44:44.834 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:44:44 smithi039 ceph-mon[31211]: from='client.? 172.21.15.39:0/3367922287' entity='client.admin' cmd={"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} : dispatch 2024-09-21T03:44:44.834 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:44:44 smithi039 ceph-mon[31211]: from='client.? 172.21.15.39:0/3367922287' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-09-21T03:44:44.904 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/51116d5c-77ca-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:44:44.968 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:44:44 smithi161 ceph-mon[34533]: pgmap v286: 97 pgs: 97 active+clean; 634 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 4.7 KiB/s wr, 11 op/s 2024-09-21T03:44:44.968 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:44:44 smithi161 ceph-mon[34533]: from='client.? 172.21.15.39:0/3367922287' entity='client.admin' cmd={"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} : dispatch 2024-09-21T03:44:44.968 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:44:44 smithi161 ceph-mon[34533]: from='client.? 172.21.15.39:0/3367922287' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-09-21T03:44:44.974 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:44:44 smithi150 ceph-mon[34289]: pgmap v286: 97 pgs: 97 active+clean; 634 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 4.7 KiB/s wr, 11 op/s 2024-09-21T03:44:44.974 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:44:44 smithi150 ceph-mon[34289]: from='client.? 172.21.15.39:0/3367922287' entity='client.admin' cmd={"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} : dispatch 2024-09-21T03:44:44.975 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:44:44 smithi150 ceph-mon[34289]: from='client.? 172.21.15.39:0/3367922287' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-09-21T03:44:45.379 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:45.378+0000 7fd4d5e14640 1 -- 172.21.15.39:0/3425625162 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd4d0109e40 msgr2=0x7fd4d0110670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:45.380 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:45.378+0000 7fd4d5e14640 1 --2- 172.21.15.39:0/3425625162 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd4d0109e40 0x7fd4d0110670 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fd4c8009ca0 tx=0x7fd4c802a8c0 comp rx=0 tx=0).stop 2024-09-21T03:44:45.380 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:45.378+0000 7fd4d5e14640 1 -- 172.21.15.39:0/3425625162 shutdown_connections 2024-09-21T03:44:45.380 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:45.378+0000 7fd4d5e14640 1 --2- 172.21.15.39:0/3425625162 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd4d0109e40 0x7fd4d0110670 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:45.380 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:45.378+0000 7fd4d5e14640 1 --2- 172.21.15.39:0/3425625162 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd4d01094f0 0x7fd4d0109900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:45.380 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:45.378+0000 7fd4d5e14640 1 --2- 172.21.15.39:0/3425625162 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd4d0103bb0 0x7fd4d0103fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:45.380 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:45.378+0000 7fd4d5e14640 1 -- 172.21.15.39:0/3425625162 >> 172.21.15.39:0/3425625162 conn(0x7fd4d00ff600 msgr2=0x7fd4d0101a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:44:45.380 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:45.378+0000 7fd4d5e14640 1 -- 172.21.15.39:0/3425625162 shutdown_connections 2024-09-21T03:44:45.380 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:45.378+0000 7fd4d5e14640 1 -- 172.21.15.39:0/3425625162 wait complete. 2024-09-21T03:44:45.382 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:45.379+0000 7fd4d5e14640 1 Processor -- start 2024-09-21T03:44:45.382 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:45.379+0000 7fd4d5e14640 1 -- start start 2024-09-21T03:44:45.382 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:45.379+0000 7fd4d5e14640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd4d0103bb0 0x7fd4d019c6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:45.382 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:45.379+0000 7fd4d5e14640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd4d01094f0 0x7fd4d019cc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:45.383 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:45.379+0000 7fd4d5e14640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd4d0109e40 0x7fd4d01a1f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:45.383 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:45.379+0000 7fd4d5e14640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fd4d0113b60 con 0x7fd4d0103bb0 2024-09-21T03:44:45.383 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:45.379+0000 7fd4d5e14640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fd4d01139e0 con 0x7fd4d0109e40 2024-09-21T03:44:45.383 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:45.379+0000 7fd4d5e14640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fd4d0113ce0 con 0x7fd4d01094f0 2024-09-21T03:44:45.383 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:45.379+0000 7fd4cf7fe640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd4d0103bb0 0x7fd4d019c6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:45.383 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:45.379+0000 7fd4cf7fe640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd4d0103bb0 0x7fd4d019c6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:53892/0 (socket says 172.21.15.39:53892) 2024-09-21T03:44:45.383 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:45.379+0000 7fd4cf7fe640 1 -- 172.21.15.39:0/445775349 learned_addr learned my addr 172.21.15.39:0/445775349 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:44:45.383 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:45.379+0000 7fd4ceffd640 1 --2- 172.21.15.39:0/445775349 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd4d01094f0 0x7fd4d019cc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:45.383 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:45.380+0000 7fd4cffff640 1 --2- 172.21.15.39:0/445775349 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd4d0109e40 0x7fd4d01a1f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:45.383 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:45.380+0000 7fd4cf7fe640 1 -- 172.21.15.39:0/445775349 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd4d01094f0 msgr2=0x7fd4d019cc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:45.383 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:45.380+0000 7fd4cf7fe640 1 --2- 172.21.15.39:0/445775349 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd4d01094f0 0x7fd4d019cc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:45.383 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:45.380+0000 7fd4cf7fe640 1 -- 172.21.15.39:0/445775349 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd4d0109e40 msgr2=0x7fd4d01a1f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:45.383 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:45.380+0000 7fd4cf7fe640 1 --2- 172.21.15.39:0/445775349 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd4d0109e40 0x7fd4d01a1f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:45.384 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:45.380+0000 7fd4cf7fe640 1 -- 172.21.15.39:0/445775349 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd4d01a2660 con 0x7fd4d0103bb0 2024-09-21T03:44:45.384 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:45.380+0000 7fd4cf7fe640 1 --2- 172.21.15.39:0/445775349 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd4d0103bb0 0x7fd4d019c6f0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fd4c000b4a0 tx=0x7fd4c000b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:44:45.384 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:45.380+0000 7fd4ccff9640 1 -- 172.21.15.39:0/445775349 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4c0014070 con 0x7fd4d0103bb0 2024-09-21T03:44:45.384 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:45.380+0000 7fd4d5e14640 1 -- 172.21.15.39:0/445775349 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd4d01a2950 con 0x7fd4d0103bb0 2024-09-21T03:44:45.384 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:45.380+0000 7fd4ccff9640 1 -- 172.21.15.39:0/445775349 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd4c0010ac0 con 0x7fd4d0103bb0 2024-09-21T03:44:45.384 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:45.380+0000 7fd4ccff9640 1 -- 172.21.15.39:0/445775349 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4c0017c30 con 0x7fd4d0103bb0 2024-09-21T03:44:45.384 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:45.380+0000 7fd4d5e14640 1 -- 172.21.15.39:0/445775349 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd4d01a9130 con 0x7fd4d0103bb0 2024-09-21T03:44:45.385 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:45.381+0000 7fd4d5e14640 1 -- 172.21.15.39:0/445775349 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd4d0103fc0 con 0x7fd4d0103bb0 2024-09-21T03:44:45.385 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:45.383+0000 7fd4ccff9640 1 -- 172.21.15.39:0/445775349 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 22) ==== 154580+0+0 (secure 0 0 0) 0x7fd4c0017070 con 0x7fd4d0103bb0 2024-09-21T03:44:45.386 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:45.384+0000 7fd4ccff9640 1 --2- 172.21.15.39:0/445775349 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7fd4b80b65f0 0x7fd4b80b8b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:45.386 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:45.384+0000 7fd4ceffd640 1 --2- 172.21.15.39:0/445775349 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7fd4b80b65f0 0x7fd4b80b8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:45.386 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:45.384+0000 7fd4ccff9640 1 -- 172.21.15.39:0/445775349 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8484+0+0 (secure 0 0 0) 0x7fd4c00e6bc0 con 0x7fd4d0103bb0 2024-09-21T03:44:45.387 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:45.385+0000 7fd4ceffd640 1 --2- 172.21.15.39:0/445775349 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7fd4b80b65f0 0x7fd4b80b8b00 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fd4d0105ba0 tx=0x7fd4c400a2d0 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:44:45.389 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:45.388+0000 7fd4ccff9640 1 -- 172.21.15.39:0/445775349 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fd4c00ad190 con 0x7fd4d0103bb0 2024-09-21T03:44:45.576 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:45.575+0000 7fd4d5e14640 1 -- 172.21.15.39:0/445775349 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "config log", "num": 1, "format": "json"} v 0) -- 0x7fd4d019d820 con 0x7fd4d0103bb0 2024-09-21T03:44:45.577 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:45.575+0000 7fd4ccff9640 1 -- 172.21.15.39:0/445775349 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "config log", "num": 1, "format": "json"}]=0 v54) ==== 86+0+133 (secure 0 0 0) 0x7fd4c00b1570 con 0x7fd4d0103bb0 2024-09-21T03:44:45.577 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T03:44:45.579 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:45.577+0000 7fd4d5e14640 1 -- 172.21.15.39:0/445775349 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7fd4b80b65f0 msgr2=0x7fd4b80b8b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:45.580 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:45.577+0000 7fd4d5e14640 1 --2- 172.21.15.39:0/445775349 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7fd4b80b65f0 0x7fd4b80b8b00 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fd4d0105ba0 tx=0x7fd4c400a2d0 comp rx=0 tx=0).stop 2024-09-21T03:44:45.580 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:45.578+0000 7fd4d5e14640 1 -- 172.21.15.39:0/445775349 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd4d0103bb0 msgr2=0x7fd4d019c6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:45.580 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:45.578+0000 7fd4d5e14640 1 --2- 172.21.15.39:0/445775349 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd4d0103bb0 0x7fd4d019c6f0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fd4c000b4a0 tx=0x7fd4c000b960 comp rx=0 tx=0).stop 2024-09-21T03:44:45.580 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:45.578+0000 7fd4d5e14640 1 -- 172.21.15.39:0/445775349 shutdown_connections 2024-09-21T03:44:45.580 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:45.578+0000 7fd4d5e14640 1 --2- 172.21.15.39:0/445775349 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7fd4b80b65f0 0x7fd4b80b8b00 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:45.580 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:45.578+0000 7fd4d5e14640 1 --2- 172.21.15.39:0/445775349 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd4d0109e40 0x7fd4d01a1f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:45.580 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:45.578+0000 7fd4d5e14640 1 --2- 172.21.15.39:0/445775349 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd4d01094f0 0x7fd4d019cc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:45.580 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:45.578+0000 7fd4d5e14640 1 --2- 172.21.15.39:0/445775349 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd4d0103bb0 0x7fd4d019c6f0 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:45.580 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:45.578+0000 7fd4d5e14640 1 -- 172.21.15.39:0/445775349 >> 172.21.15.39:0/445775349 conn(0x7fd4d00ff600 msgr2=0x7fd4d0108b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:44:45.580 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:45.578+0000 7fd4d5e14640 1 -- 172.21.15.39:0/445775349 shutdown_connections 2024-09-21T03:44:45.580 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:45.578+0000 7fd4d5e14640 1 -- 172.21.15.39:0/445775349 wait complete. 2024-09-21T03:44:45.834 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:44:45 smithi039 ceph-mon[31211]: from='client.? 172.21.15.39:0/445775349' entity='client.admin' cmd={"prefix": "config log", "num": 1, "format": "json"} : dispatch 2024-09-21T03:44:45.968 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:44:45 smithi161 ceph-mon[34533]: from='client.? 172.21.15.39:0/445775349' entity='client.admin' cmd={"prefix": "config log", "num": 1, "format": "json"} : dispatch 2024-09-21T03:44:45.974 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:44:45 smithi150 ceph-mon[34289]: from='client.? 172.21.15.39:0/445775349' entity='client.admin' cmd={"prefix": "config log", "num": 1, "format": "json"} : dispatch 2024-09-21T03:44:46.363 INFO:teuthology.orchestra.run.smithi039.stdout:[{"version":54,"timestamp":"2024-09-21T03:44:04.978929+0000","name":"","changes":[{"name":"mds.all/mds_join_fs","new_value":"all"}]}] 2024-09-21T03:44:46.363 INFO:tasks.ceph_manager:config epoch is 54 2024-09-21T03:44:46.363 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-09-21T03:44:46.363 INFO:tasks.cephadm.ceph_manager.ceph:waiting for mgr available 2024-09-21T03:44:46.363 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 51116d5c-77ca-11ef-baf5-efdc52797490 -- ceph mgr dump --format=json 2024-09-21T03:44:46.621 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/51116d5c-77ca-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:44:46.859 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:44:46 smithi039 ceph-mon[31211]: pgmap v287: 97 pgs: 97 active+clean; 642 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 5.6 KiB/s wr, 15 op/s 2024-09-21T03:44:46.968 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:44:46 smithi161 ceph-mon[34533]: pgmap v287: 97 pgs: 97 active+clean; 642 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 5.6 KiB/s wr, 15 op/s 2024-09-21T03:44:46.974 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:44:46 smithi150 ceph-mon[34289]: pgmap v287: 97 pgs: 97 active+clean; 642 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 5.6 KiB/s wr, 15 op/s 2024-09-21T03:44:47.312 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:47.302+0000 7fc208050640 1 -- 172.21.15.39:0/1789296793 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc200109df0 msgr2=0x7fc200110620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:47.312 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:47.302+0000 7fc208050640 1 --2- 172.21.15.39:0/1789296793 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc200109df0 0x7fc200110620 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fc1f8009900 tx=0x7fc1f802a410 comp rx=0 tx=0).stop 2024-09-21T03:44:47.312 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:47.302+0000 7fc208050640 1 -- 172.21.15.39:0/1789296793 shutdown_connections 2024-09-21T03:44:47.312 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:47.302+0000 7fc208050640 1 --2- 172.21.15.39:0/1789296793 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc200109df0 0x7fc200110620 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:47.313 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:47.302+0000 7fc208050640 1 --2- 172.21.15.39:0/1789296793 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc2001094a0 0x7fc2001098b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:47.313 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:47.302+0000 7fc208050640 1 --2- 172.21.15.39:0/1789296793 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc200103b60 0x7fc200103f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:47.313 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:47.302+0000 7fc208050640 1 -- 172.21.15.39:0/1789296793 >> 172.21.15.39:0/1789296793 conn(0x7fc20007b850 msgr2=0x7fc20007bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:44:47.313 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:47.302+0000 7fc208050640 1 -- 172.21.15.39:0/1789296793 shutdown_connections 2024-09-21T03:44:47.313 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:47.302+0000 7fc208050640 1 -- 172.21.15.39:0/1789296793 wait complete. 2024-09-21T03:44:47.313 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:47.303+0000 7fc208050640 1 Processor -- start 2024-09-21T03:44:47.313 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:47.303+0000 7fc208050640 1 -- start start 2024-09-21T03:44:47.313 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:47.303+0000 7fc208050640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc200103b60 0x7fc20019a4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:47.313 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:47.303+0000 7fc208050640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc2001094a0 0x7fc20019aa40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:47.313 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:47.303+0000 7fc208050640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc200109df0 0x7fc20019fce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:47.313 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:47.303+0000 7fc208050640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fc200113b10 con 0x7fc200103b60 2024-09-21T03:44:47.314 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:47.303+0000 7fc208050640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fc200113990 con 0x7fc2001094a0 2024-09-21T03:44:47.314 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:47.303+0000 7fc208050640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fc200113c90 con 0x7fc200109df0 2024-09-21T03:44:47.314 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:47.304+0000 7fc205dec640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc200103b60 0x7fc20019a4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:47.314 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:47.304+0000 7fc205dec640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc200103b60 0x7fc20019a4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:53900/0 (socket says 172.21.15.39:53900) 2024-09-21T03:44:47.314 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:47.304+0000 7fc205dec640 1 -- 172.21.15.39:0/2846608647 learned_addr learned my addr 172.21.15.39:0/2846608647 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:44:47.314 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:47.304+0000 7fc2065ed640 1 --2- 172.21.15.39:0/2846608647 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc200109df0 0x7fc20019fce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:47.314 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:47.304+0000 7fc205dec640 1 -- 172.21.15.39:0/2846608647 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc200109df0 msgr2=0x7fc20019fce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:47.314 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:47.304+0000 7fc205dec640 1 --2- 172.21.15.39:0/2846608647 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc200109df0 0x7fc20019fce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:47.314 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:47.304+0000 7fc205dec640 1 -- 172.21.15.39:0/2846608647 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc2001094a0 msgr2=0x7fc20019aa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:47.314 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:47.304+0000 7fc2055eb640 1 --2- 172.21.15.39:0/2846608647 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc2001094a0 0x7fc20019aa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:47.314 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:47.304+0000 7fc205dec640 1 --2- 172.21.15.39:0/2846608647 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc2001094a0 0x7fc20019aa40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:47.314 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:47.304+0000 7fc205dec640 1 -- 172.21.15.39:0/2846608647 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc2001a0410 con 0x7fc200103b60 2024-09-21T03:44:47.314 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:47.304+0000 7fc2055eb640 1 --2- 172.21.15.39:0/2846608647 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc2001094a0 0x7fc20019aa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:44:47.315 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:47.304+0000 7fc205dec640 1 --2- 172.21.15.39:0/2846608647 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc200103b60 0x7fc20019a4a0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fc1f000ea80 tx=0x7fc1f000ef40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:44:47.315 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:47.304+0000 7fc1feffd640 1 -- 172.21.15.39:0/2846608647 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1f000ce40 con 0x7fc200103b60 2024-09-21T03:44:47.315 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:47.304+0000 7fc1feffd640 1 -- 172.21.15.39:0/2846608647 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc1f0005020 con 0x7fc200103b60 2024-09-21T03:44:47.315 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:47.304+0000 7fc208050640 1 -- 172.21.15.39:0/2846608647 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc2001a0700 con 0x7fc200103b60 2024-09-21T03:44:47.315 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:47.304+0000 7fc1feffd640 1 -- 172.21.15.39:0/2846608647 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1f00116d0 con 0x7fc200103b60 2024-09-21T03:44:47.315 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:47.304+0000 7fc208050640 1 -- 172.21.15.39:0/2846608647 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc2001a6e60 con 0x7fc200103b60 2024-09-21T03:44:47.315 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:47.305+0000 7fc208050640 1 -- 172.21.15.39:0/2846608647 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc200103f70 con 0x7fc200103b60 2024-09-21T03:44:47.315 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:47.307+0000 7fc1feffd640 1 -- 172.21.15.39:0/2846608647 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 22) ==== 154580+0+0 (secure 0 0 0) 0x7fc1f00054d0 con 0x7fc200103b60 2024-09-21T03:44:47.315 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:47.308+0000 7fc1feffd640 1 --2- 172.21.15.39:0/2846608647 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7fc1e80b65f0 0x7fc1e80b8b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:47.315 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:47.308+0000 7fc2055eb640 1 --2- 172.21.15.39:0/2846608647 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7fc1e80b65f0 0x7fc1e80b8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:47.315 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:47.308+0000 7fc1feffd640 1 -- 172.21.15.39:0/2846608647 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8484+0+0 (secure 0 0 0) 0x7fc1f00e7d40 con 0x7fc200103b60 2024-09-21T03:44:47.315 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:47.308+0000 7fc2055eb640 1 --2- 172.21.15.39:0/2846608647 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7fc1e80b65f0 0x7fc1e80b8b00 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fc1f4006fd0 tx=0x7fc1f4008040 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:44:47.316 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:47.311+0000 7fc1feffd640 1 -- 172.21.15.39:0/2846608647 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fc1f0016030 con 0x7fc200103b60 2024-09-21T03:44:47.551 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:47.550+0000 7fc208050640 1 -- 172.21.15.39:0/2846608647 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) -- 0x7fc20019b5d0 con 0x7fc200103b60 2024-09-21T03:44:47.556 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:47.555+0000 7fc1feffd640 1 -- 172.21.15.39:0/2846608647 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v22) ==== 74+0+296453 (secure 0 0 0) 0x7fc1f00ae370 con 0x7fc200103b60 2024-09-21T03:44:47.560 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T03:44:47.562 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:47.559+0000 7fc208050640 1 -- 172.21.15.39:0/2846608647 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7fc1e80b65f0 msgr2=0x7fc1e80b8b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:47.562 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:47.559+0000 7fc208050640 1 --2- 172.21.15.39:0/2846608647 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7fc1e80b65f0 0x7fc1e80b8b00 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fc1f4006fd0 tx=0x7fc1f4008040 comp rx=0 tx=0).stop 2024-09-21T03:44:47.562 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:47.559+0000 7fc208050640 1 -- 172.21.15.39:0/2846608647 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc200103b60 msgr2=0x7fc20019a4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:47.562 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:47.559+0000 7fc208050640 1 --2- 172.21.15.39:0/2846608647 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc200103b60 0x7fc20019a4a0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fc1f000ea80 tx=0x7fc1f000ef40 comp rx=0 tx=0).stop 2024-09-21T03:44:47.562 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:47.559+0000 7fc208050640 1 -- 172.21.15.39:0/2846608647 shutdown_connections 2024-09-21T03:44:47.562 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:47.559+0000 7fc208050640 1 --2- 172.21.15.39:0/2846608647 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7fc1e80b65f0 0x7fc1e80b8b00 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:47.563 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:47.559+0000 7fc208050640 1 --2- 172.21.15.39:0/2846608647 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc200109df0 0x7fc20019fce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:47.563 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:47.559+0000 7fc208050640 1 --2- 172.21.15.39:0/2846608647 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc2001094a0 0x7fc20019aa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:47.563 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:47.559+0000 7fc208050640 1 --2- 172.21.15.39:0/2846608647 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc200103b60 0x7fc20019a4a0 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:47.563 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:47.559+0000 7fc208050640 1 -- 172.21.15.39:0/2846608647 >> 172.21.15.39:0/2846608647 conn(0x7fc20007b850 msgr2=0x7fc200105470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:44:47.563 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:47.559+0000 7fc208050640 1 -- 172.21.15.39:0/2846608647 shutdown_connections 2024-09-21T03:44:47.563 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:47.559+0000 7fc208050640 1 -- 172.21.15.39:0/2846608647 wait complete. 2024-09-21T03:44:47.834 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:44:47 smithi039 ceph-mon[31211]: from='client.? 172.21.15.39:0/2846608647' entity='client.admin' cmd={"prefix": "mgr dump", "format": "json"} : dispatch 2024-09-21T03:44:47.968 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:44:47 smithi161 ceph-mon[34533]: from='client.? 172.21.15.39:0/2846608647' entity='client.admin' cmd={"prefix": "mgr dump", "format": "json"} : dispatch 2024-09-21T03:44:47.974 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:44:47 smithi150 ceph-mon[34289]: from='client.? 172.21.15.39:0/2846608647' entity='client.admin' cmd={"prefix": "mgr dump", "format": "json"} : dispatch 2024-09-21T03:44:48.227 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":22,"flags":0,"active_gid":14223,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6800","nonce":1696381982},{"type":"v1","addr":"172.21.15.39:6801","nonce":1696381982}]},"active_addr":"172.21.15.39:6801/1696381982","active_change":"2024-09-21T03:37:25.391596+0000","active_mgr_features":4540719139924082687,"available":true,"standbys":[{"gid":24142,"name":"z","mgr_features":4540719139924082687,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to, use commas to separate multiple","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.27.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:10.4.8","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_nginx":{"name":"container_image_nginx","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nginx:sclorg-nginx-126","min":"","max":"","enum_allowed":[],"desc":"Nginx container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.7.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.17","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_oauth2_proxy":{"name":"container_image_oauth2_proxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/oauth2-proxy/oauth2-proxy:v7.6.0","min":"","max":"","enum_allowed":[],"desc":"oauth2-proxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.51.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba_metrics":{"name":"container_image_samba_metrics","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-metrics:latest","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB metrics exporter container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"ssh_keepalive_count_max":{"name":"ssh_keepalive_count_max","type":"int","level":"advanced","flags":0,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"How many times ssh connections can fail liveness checks before the host is marked offline","long_desc":"","tags":[],"see_also":[]},"ssh_keepalive_interval":{"name":"ssh_keepalive_interval","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"How often ssh connections are checked for liveness","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sso_oauth2":{"name":"sso_oauth2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_requests":{"name":"max_requests","type":"int","level":"advanced","flags":0,"default_value":"500","min":"","max":"","enum_allowed":[],"desc":"Maximum number of requests to keep in memory. When new request comes in, the oldest request will be removed if the number of requests exceeds the max request number.if un-finished request is removed, error message will be logged in the ceph-mgr log.","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"secondary_zone_period_retry_limit":{"name":"secondary_zone_period_retry_limit","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"RGW module period update retry limit for secondary site","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"smb","can_run":true,"error_string":"","module_options":{"internal_store_backend":{"name":"internal_store_backend","type":"str","level":"dev","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"set internal store backend. for develoment and testing only","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"update_orchestration":{"name":"update_orchestration","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically update orchestration when smb resources are changed","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]},{"gid":24146,"name":"y","mgr_features":4540719139924082687,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to, use commas to separate multiple","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.27.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:10.4.8","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_nginx":{"name":"container_image_nginx","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nginx:sclorg-nginx-126","min":"","max":"","enum_allowed":[],"desc":"Nginx container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.7.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.17","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_oauth2_proxy":{"name":"container_image_oauth2_proxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/oauth2-proxy/oauth2-proxy:v7.6.0","min":"","max":"","enum_allowed":[],"desc":"oauth2-proxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.51.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba_metrics":{"name":"container_image_samba_metrics","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-metrics:latest","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB metrics exporter container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"ssh_keepalive_count_max":{"name":"ssh_keepalive_count_max","type":"int","level":"advanced","flags":0,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"How many times ssh connections can fail liveness checks before the host is marked offline","long_desc":"","tags":[],"see_also":[]},"ssh_keepalive_interval":{"name":"ssh_keepalive_interval","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"How often ssh connections are checked for liveness","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sso_oauth2":{"name":"sso_oauth2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_requests":{"name":"max_requests","type":"int","level":"advanced","flags":0,"default_value":"500","min":"","max":"","enum_allowed":[],"desc":"Maximum number of requests to keep in memory. When new request comes in, the oldest request will be removed if the number of requests exceeds the max request number.if un-finished request is removed, error message will be logged in the ceph-mgr log.","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"secondary_zone_period_retry_limit":{"name":"secondary_zone_period_retry_limit","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"RGW module period update retry limit for secondary site","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"smb","can_run":true,"error_string":"","module_options":{"internal_store_backend":{"name":"internal_store_backend","type":"str","level":"dev","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"set internal store backend. for develoment and testing only","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"update_orchestration":{"name":"update_orchestration","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically update orchestration when smb resources are changed","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["cephadm","dashboard","iostat","nfs","restful","snap_schedule"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to, use commas to separate multiple","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.27.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:10.4.8","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_nginx":{"name":"container_image_nginx","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nginx:sclorg-nginx-126","min":"","max":"","enum_allowed":[],"desc":"Nginx container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.7.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.17","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_oauth2_proxy":{"name":"container_image_oauth2_proxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/oauth2-proxy/oauth2-proxy:v7.6.0","min":"","max":"","enum_allowed":[],"desc":"oauth2-proxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.51.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba_metrics":{"name":"container_image_samba_metrics","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-metrics:latest","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB metrics exporter container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"ssh_keepalive_count_max":{"name":"ssh_keepalive_count_max","type":"int","level":"advanced","flags":0,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"How many times ssh connections can fail liveness checks before the host is marked offline","long_desc":"","tags":[],"see_also":[]},"ssh_keepalive_interval":{"name":"ssh_keepalive_interval","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"How often ssh connections are checked for liveness","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sso_oauth2":{"name":"sso_oauth2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_requests":{"name":"max_requests","type":"int","level":"advanced","flags":0,"default_value":"500","min":"","max":"","enum_allowed":[],"desc":"Maximum number of requests to keep in memory. When new request comes in, the oldest request will be removed if the number of requests exceeds the max request number.if un-finished request is removed, error message will be logged in the ceph-mgr log.","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"secondary_zone_period_retry_limit":{"name":"secondary_zone_period_retry_limit","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"RGW module period update retry limit for secondary site","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"smb","can_run":true,"error_string":"","module_options":{"internal_store_backend":{"name":"internal_store_backend","type":"str","level":"dev","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"set internal store backend. for develoment and testing only","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"update_orchestration":{"name":"update_orchestration","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically update orchestration when smb resources are changed","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{"dashboard":"https://172.21.15.39:8443/"},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":5,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.39:0","nonce":2409142260}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.39:0","nonce":3867972996}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.39:0","nonce":2292114781}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.39:0","nonce":3602591107}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.39:0","nonce":643948726}]}]} 2024-09-21T03:44:48.234 INFO:tasks.cephadm.ceph_manager.ceph:mgr available! 2024-09-21T03:44:48.234 INFO:tasks.cephadm.ceph_manager.ceph:waiting for all up 2024-09-21T03:44:48.235 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 51116d5c-77ca-11ef-baf5-efdc52797490 -- ceph osd dump --format=json 2024-09-21T03:44:48.489 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/51116d5c-77ca-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:44:48.877 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:44:48 smithi039 ceph-mon[31211]: pgmap v288: 97 pgs: 97 active+clean; 642 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 2.1 KiB/s wr, 10 op/s 2024-09-21T03:44:48.967 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:44:48 smithi161 ceph-mon[34533]: pgmap v288: 97 pgs: 97 active+clean; 642 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 2.1 KiB/s wr, 10 op/s 2024-09-21T03:44:48.974 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:44:48 smithi150 ceph-mon[34289]: pgmap v288: 97 pgs: 97 active+clean; 642 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 2.1 KiB/s wr, 10 op/s 2024-09-21T03:44:49.120 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:49.111+0000 7f656de6b640 1 -- 172.21.15.39:0/798118925 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6568102a60 msgr2=0x7f6568102e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:49.120 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:49.111+0000 7f656de6b640 1 --2- 172.21.15.39:0/798118925 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6568102a60 0x7f6568102e70 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f6558009ca0 tx=0x7f655802a8c0 comp rx=0 tx=0).stop 2024-09-21T03:44:49.120 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:49.111+0000 7f656de6b640 1 -- 172.21.15.39:0/798118925 shutdown_connections 2024-09-21T03:44:49.120 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:49.111+0000 7f656de6b640 1 --2- 172.21.15.39:0/798118925 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6568108cf0 0x7f656810f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:49.121 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:49.111+0000 7f656de6b640 1 --2- 172.21.15.39:0/798118925 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f65681083a0 0x7f65681087b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:49.121 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:49.111+0000 7f656de6b640 1 --2- 172.21.15.39:0/798118925 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6568102a60 0x7f6568102e70 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:49.121 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:49.111+0000 7f656de6b640 1 -- 172.21.15.39:0/798118925 >> 172.21.15.39:0/798118925 conn(0x7f6568079730 msgr2=0x7f6568079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:44:49.121 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:49.111+0000 7f656de6b640 1 -- 172.21.15.39:0/798118925 shutdown_connections 2024-09-21T03:44:49.121 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:49.111+0000 7f656de6b640 1 -- 172.21.15.39:0/798118925 wait complete. 2024-09-21T03:44:49.121 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:49.112+0000 7f656de6b640 1 Processor -- start 2024-09-21T03:44:49.121 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:49.112+0000 7f656de6b640 1 -- start start 2024-09-21T03:44:49.121 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:49.112+0000 7f656de6b640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6568102a60 0x7f656819a4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:49.121 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:49.112+0000 7f656de6b640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f65681083a0 0x7f656819aa40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:49.121 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:49.112+0000 7f656de6b640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6568108cf0 0x7f656819fce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:49.121 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:49.112+0000 7f656de6b640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f65681129e0 con 0x7f6568102a60 2024-09-21T03:44:49.122 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:49.112+0000 7f656de6b640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f6568112860 con 0x7f6568108cf0 2024-09-21T03:44:49.122 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:49.112+0000 7f656de6b640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f6568112b60 con 0x7f65681083a0 2024-09-21T03:44:49.122 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:49.112+0000 7f65677fe640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6568102a60 0x7f656819a4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:49.122 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:49.112+0000 7f65677fe640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6568102a60 0x7f656819a4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:40280/0 (socket says 172.21.15.39:40280) 2024-09-21T03:44:49.122 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:49.112+0000 7f65677fe640 1 -- 172.21.15.39:0/3691466932 learned_addr learned my addr 172.21.15.39:0/3691466932 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:44:49.122 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:49.113+0000 7f6567fff640 1 --2- 172.21.15.39:0/3691466932 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6568108cf0 0x7f656819fce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:49.122 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:49.113+0000 7f6566ffd640 1 --2- 172.21.15.39:0/3691466932 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f65681083a0 0x7f656819aa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:49.122 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:49.113+0000 7f65677fe640 1 -- 172.21.15.39:0/3691466932 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f65681083a0 msgr2=0x7f656819aa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:49.122 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:49.113+0000 7f65677fe640 1 --2- 172.21.15.39:0/3691466932 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f65681083a0 0x7f656819aa40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:49.122 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:49.113+0000 7f65677fe640 1 -- 172.21.15.39:0/3691466932 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6568108cf0 msgr2=0x7f656819fce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:49.122 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:49.113+0000 7f65677fe640 1 --2- 172.21.15.39:0/3691466932 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6568108cf0 0x7f656819fce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:49.122 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:49.113+0000 7f65677fe640 1 -- 172.21.15.39:0/3691466932 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f65681a0410 con 0x7f6568102a60 2024-09-21T03:44:49.122 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:49.113+0000 7f6567fff640 1 --2- 172.21.15.39:0/3691466932 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6568108cf0 0x7f656819fce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:44:49.123 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:49.113+0000 7f65677fe640 1 --2- 172.21.15.39:0/3691466932 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6568102a60 0x7f656819a4a0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f6558006010 tx=0x7f65580319f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:44:49.123 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:49.113+0000 7f6564ff9640 1 -- 172.21.15.39:0/3691466932 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6558003bf0 con 0x7f6568102a60 2024-09-21T03:44:49.123 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:49.113+0000 7f656de6b640 1 -- 172.21.15.39:0/3691466932 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f65681a06a0 con 0x7f6568102a60 2024-09-21T03:44:49.123 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:49.113+0000 7f6564ff9640 1 -- 172.21.15.39:0/3691466932 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6558003d90 con 0x7f6568102a60 2024-09-21T03:44:49.123 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:49.113+0000 7f6564ff9640 1 -- 172.21.15.39:0/3691466932 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f655802fc20 con 0x7f6568102a60 2024-09-21T03:44:49.123 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:49.113+0000 7f656de6b640 1 -- 172.21.15.39:0/3691466932 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f65681a6e60 con 0x7f6568102a60 2024-09-21T03:44:49.123 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:49.114+0000 7f656de6b640 1 -- 172.21.15.39:0/3691466932 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6568102e70 con 0x7f6568102a60 2024-09-21T03:44:49.123 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:49.116+0000 7f6564ff9640 1 -- 172.21.15.39:0/3691466932 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 22) ==== 154580+0+0 (secure 0 0 0) 0x7f65580046a0 con 0x7f6568102a60 2024-09-21T03:44:49.123 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:49.116+0000 7f6564ff9640 1 --2- 172.21.15.39:0/3691466932 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f65540b65f0 0x7f65540b8b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:49.123 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:49.117+0000 7f6566ffd640 1 --2- 172.21.15.39:0/3691466932 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f65540b65f0 0x7f65540b8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:49.123 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:49.117+0000 7f6564ff9640 1 -- 172.21.15.39:0/3691466932 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8484+0+0 (secure 0 0 0) 0x7f6558108c00 con 0x7f6568102a60 2024-09-21T03:44:49.123 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:49.117+0000 7f6566ffd640 1 --2- 172.21.15.39:0/3691466932 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f65540b65f0 0x7f65540b8b00 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f6550005fd0 tx=0x7f6550005850 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:44:49.124 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:49.120+0000 7f6564ff9640 1 -- 172.21.15.39:0/3691466932 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f65580cf230 con 0x7f6568102a60 2024-09-21T03:44:49.312 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:49.310+0000 7f656de6b640 1 -- 172.21.15.39:0/3691466932 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f656819b5d0 con 0x7f6568102a60 2024-09-21T03:44:49.313 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:49.312+0000 7f6564ff9640 1 -- 172.21.15.39:0/3691466932 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v79) ==== 74+0+23096 (secure 0 0 0) 0x7f656819b5d0 con 0x7f6568102a60 2024-09-21T03:44:49.314 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T03:44:49.314 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":79,"fsid":"51116d5c-77ca-11ef-baf5-efdc52797490","created":"2024-09-21T03:35:33.631350+0000","modified":"2024-09-21T03:44:27.402588+0000","last_up_change":"2024-09-21T03:43:24.242275+0000","last_in_change":"2024-09-21T03:43:00.886552+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T03:39:29.859542+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-21T03:44:10.886435+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"77","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-21T03:44:13.041783+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"77","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1ea7148f-2ace-4c10-86fe-b8019de80c1f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6802","nonce":870188791},{"type":"v1","addr":"172.21.15.39:6803","nonce":870188791}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6804","nonce":870188791},{"type":"v1","addr":"172.21.15.39:6805","nonce":870188791}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6808","nonce":870188791},{"type":"v1","addr":"172.21.15.39:6809","nonce":870188791}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6806","nonce":870188791},{"type":"v1","addr":"172.21.15.39:6807","nonce":870188791}]},"public_addr":"172.21.15.39:6803/870188791","cluster_addr":"172.21.15.39:6805/870188791","heartbeat_back_addr":"172.21.15.39:6809/870188791","heartbeat_front_addr":"172.21.15.39:6807/870188791","state":["exists","up"]},{"osd":1,"uuid":"7cb5db90-28e8-4a30-bedb-c039687d48b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6800","nonce":172427832},{"type":"v1","addr":"172.21.15.150:6801","nonce":172427832}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6802","nonce":172427832},{"type":"v1","addr":"172.21.15.150:6803","nonce":172427832}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6806","nonce":172427832},{"type":"v1","addr":"172.21.15.150:6807","nonce":172427832}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6804","nonce":172427832},{"type":"v1","addr":"172.21.15.150:6805","nonce":172427832}]},"public_addr":"172.21.15.150:6801/172427832","cluster_addr":"172.21.15.150:6803/172427832","heartbeat_back_addr":"172.21.15.150:6807/172427832","heartbeat_front_addr":"172.21.15.150:6805/172427832","state":["exists","up"]},{"osd":2,"uuid":"b25c6ab3-fc95-4751-bd63-6e3fc23e8bd7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":19,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6800","nonce":2727876133},{"type":"v1","addr":"172.21.15.161:6801","nonce":2727876133}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6802","nonce":2727876133},{"type":"v1","addr":"172.21.15.161:6803","nonce":2727876133}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6806","nonce":2727876133},{"type":"v1","addr":"172.21.15.161:6807","nonce":2727876133}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6804","nonce":2727876133},{"type":"v1","addr":"172.21.15.161:6805","nonce":2727876133}]},"public_addr":"172.21.15.161:6801/2727876133","cluster_addr":"172.21.15.161:6803/2727876133","heartbeat_back_addr":"172.21.15.161:6807/2727876133","heartbeat_front_addr":"172.21.15.161:6805/2727876133","state":["exists","up"]},{"osd":3,"uuid":"14417ae1-4536-45ba-9a5c-c3b447d1a72d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6810","nonce":295394468},{"type":"v1","addr":"172.21.15.39:6811","nonce":295394468}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6812","nonce":295394468},{"type":"v1","addr":"172.21.15.39:6813","nonce":295394468}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6816","nonce":295394468},{"type":"v1","addr":"172.21.15.39:6817","nonce":295394468}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6814","nonce":295394468},{"type":"v1","addr":"172.21.15.39:6815","nonce":295394468}]},"public_addr":"172.21.15.39:6811/295394468","cluster_addr":"172.21.15.39:6813/295394468","heartbeat_back_addr":"172.21.15.39:6817/295394468","heartbeat_front_addr":"172.21.15.39:6815/295394468","state":["exists","up"]},{"osd":4,"uuid":"33c732ef-20c2-43c4-abe6-045b687fc521","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6808","nonce":3097721553},{"type":"v1","addr":"172.21.15.150:6809","nonce":3097721553}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6810","nonce":3097721553},{"type":"v1","addr":"172.21.15.150:6811","nonce":3097721553}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6814","nonce":3097721553},{"type":"v1","addr":"172.21.15.150:6815","nonce":3097721553}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6812","nonce":3097721553},{"type":"v1","addr":"172.21.15.150:6813","nonce":3097721553}]},"public_addr":"172.21.15.150:6809/3097721553","cluster_addr":"172.21.15.150:6811/3097721553","heartbeat_back_addr":"172.21.15.150:6815/3097721553","heartbeat_front_addr":"172.21.15.150:6813/3097721553","state":["exists","up"]},{"osd":5,"uuid":"74673722-abd6-400a-bb50-ec3dfba0c5e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6808","nonce":742751023},{"type":"v1","addr":"172.21.15.161:6809","nonce":742751023}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6810","nonce":742751023},{"type":"v1","addr":"172.21.15.161:6811","nonce":742751023}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6814","nonce":742751023},{"type":"v1","addr":"172.21.15.161:6815","nonce":742751023}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6812","nonce":742751023},{"type":"v1","addr":"172.21.15.161:6813","nonce":742751023}]},"public_addr":"172.21.15.161:6809/742751023","cluster_addr":"172.21.15.161:6811/742751023","heartbeat_back_addr":"172.21.15.161:6815/742751023","heartbeat_front_addr":"172.21.15.161:6813/742751023","state":["exists","up"]},{"osd":6,"uuid":"06548677-74c4-4efa-8054-df09b957b649","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6818","nonce":1072700290},{"type":"v1","addr":"172.21.15.39:6819","nonce":1072700290}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6820","nonce":1072700290},{"type":"v1","addr":"172.21.15.39:6821","nonce":1072700290}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6824","nonce":1072700290},{"type":"v1","addr":"172.21.15.39:6825","nonce":1072700290}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6822","nonce":1072700290},{"type":"v1","addr":"172.21.15.39:6823","nonce":1072700290}]},"public_addr":"172.21.15.39:6819/1072700290","cluster_addr":"172.21.15.39:6821/1072700290","heartbeat_back_addr":"172.21.15.39:6825/1072700290","heartbeat_front_addr":"172.21.15.39:6823/1072700290","state":["exists","up"]},{"osd":7,"uuid":"838735ca-c693-4f58-9426-b8ee857dbe3a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6816","nonce":2162619103},{"type":"v1","addr":"172.21.15.150:6817","nonce":2162619103}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6818","nonce":2162619103},{"type":"v1","addr":"172.21.15.150:6819","nonce":2162619103}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6822","nonce":2162619103},{"type":"v1","addr":"172.21.15.150:6823","nonce":2162619103}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6820","nonce":2162619103},{"type":"v1","addr":"172.21.15.150:6821","nonce":2162619103}]},"public_addr":"172.21.15.150:6817/2162619103","cluster_addr":"172.21.15.150:6819/2162619103","heartbeat_back_addr":"172.21.15.150:6823/2162619103","heartbeat_front_addr":"172.21.15.150:6821/2162619103","state":["exists","up"]},{"osd":8,"uuid":"4973b357-b4c2-4c00-b7f7-dd25012a0f72","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6816","nonce":52966633},{"type":"v1","addr":"172.21.15.161:6817","nonce":52966633}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6818","nonce":52966633},{"type":"v1","addr":"172.21.15.161:6819","nonce":52966633}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6822","nonce":52966633},{"type":"v1","addr":"172.21.15.161:6823","nonce":52966633}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6820","nonce":52966633},{"type":"v1","addr":"172.21.15.161:6821","nonce":52966633}]},"public_addr":"172.21.15.161:6817/52966633","cluster_addr":"172.21.15.161:6819/52966633","heartbeat_back_addr":"172.21.15.161:6823/52966633","heartbeat_front_addr":"172.21.15.161:6821/52966633","state":["exists","up"]},{"osd":9,"uuid":"6028613c-ebfd-42ea-9949-3f0d5f6cf025","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6826","nonce":2001303253},{"type":"v1","addr":"172.21.15.39:6827","nonce":2001303253}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6828","nonce":2001303253},{"type":"v1","addr":"172.21.15.39:6829","nonce":2001303253}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6832","nonce":2001303253},{"type":"v1","addr":"172.21.15.39:6833","nonce":2001303253}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6830","nonce":2001303253},{"type":"v1","addr":"172.21.15.39:6831","nonce":2001303253}]},"public_addr":"172.21.15.39:6827/2001303253","cluster_addr":"172.21.15.39:6829/2001303253","heartbeat_back_addr":"172.21.15.39:6833/2001303253","heartbeat_front_addr":"172.21.15.39:6831/2001303253","state":["exists","up"]},{"osd":10,"uuid":"1c71c765-2a6c-4ffe-9fe1-c8adae3d1c32","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6824","nonce":2484627938},{"type":"v1","addr":"172.21.15.150:6825","nonce":2484627938}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6826","nonce":2484627938},{"type":"v1","addr":"172.21.15.150:6827","nonce":2484627938}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6830","nonce":2484627938},{"type":"v1","addr":"172.21.15.150:6831","nonce":2484627938}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6828","nonce":2484627938},{"type":"v1","addr":"172.21.15.150:6829","nonce":2484627938}]},"public_addr":"172.21.15.150:6825/2484627938","cluster_addr":"172.21.15.150:6827/2484627938","heartbeat_back_addr":"172.21.15.150:6831/2484627938","heartbeat_front_addr":"172.21.15.150:6829/2484627938","state":["exists","up"]},{"osd":11,"uuid":"f382f522-62db-46da-997e-2b28f84d1612","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6824","nonce":2178709707},{"type":"v1","addr":"172.21.15.161:6825","nonce":2178709707}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6826","nonce":2178709707},{"type":"v1","addr":"172.21.15.161:6827","nonce":2178709707}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6830","nonce":2178709707},{"type":"v1","addr":"172.21.15.161:6831","nonce":2178709707}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6828","nonce":2178709707},{"type":"v1","addr":"172.21.15.161:6829","nonce":2178709707}]},"public_addr":"172.21.15.161:6825/2178709707","cluster_addr":"172.21.15.161:6827/2178709707","heartbeat_back_addr":"172.21.15.161:6831/2178709707","heartbeat_front_addr":"172.21.15.161:6829/2178709707","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:38:35.540904+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:39:01.455053+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:39:27.052867+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:39:55.173529+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:40:21.298396+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:40:47.131235+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:41:14.475209+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:41:39.920036+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:42:05.587065+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:42:32.624726+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:42:57.479185+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:43:22.203832+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.5","mappings":[{"from":8,"to":11}]},{"pgid":"2.8","mappings":[{"from":2,"to":11}]},{"pgid":"2.c","mappings":[{"from":2,"to":1}]},{"pgid":"2.1f","mappings":[{"from":8,"to":4}]},{"pgid":"3.2","mappings":[{"from":3,"to":9}]},{"pgid":"3.14","mappings":[{"from":4,"to":6}]},{"pgid":"3.1a","mappings":[{"from":5,"to":7}]},{"pgid":"3.1d","mappings":[{"from":4,"to":8}]},{"pgid":"3.2c","mappings":[{"from":4,"to":9}]},{"pgid":"3.38","mappings":[{"from":3,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.39:6801/2438110611":"2024-09-22T03:37:25.391457+0000","172.21.15.39:6800/2438110611":"2024-09-22T03:37:25.391457+0000","172.21.15.39:6800/173264695":"2024-09-22T03:35:57.104467+0000","172.21.15.39:6801/173264695":"2024-09-22T03:35:57.104467+0000","172.21.15.39:0/2637182284":"2024-09-22T03:35:57.104467+0000","172.21.15.39:0/1517304561":"2024-09-22T03:36:24.112592+0000","172.21.15.39:6801/1435622199":"2024-09-22T03:36:24.112592+0000","172.21.15.39:6800/1435622199":"2024-09-22T03:36:24.112592+0000","172.21.15.39:0/346832469":"2024-09-22T03:35:57.104467+0000","172.21.15.39:0/3160777371":"2024-09-22T03:36:24.112592+0000","172.21.15.39:0/1730342855":"2024-09-22T03:35:57.104467+0000","172.21.15.39:0/1955726766":"2024-09-22T03:36:24.112592+0000","172.21.15.39:0/3294132796":"2024-09-22T03:37:25.391457+0000","172.21.15.39:0/1295810162":"2024-09-22T03:37:25.391457+0000","172.21.15.39:0/3729230474":"2024-09-22T03:37:25.391457+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T03:44:49.316 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:49.314+0000 7f656de6b640 1 -- 172.21.15.39:0/3691466932 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f65540b65f0 msgr2=0x7f65540b8b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:49.316 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:49.314+0000 7f656de6b640 1 --2- 172.21.15.39:0/3691466932 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f65540b65f0 0x7f65540b8b00 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f6550005fd0 tx=0x7f6550005850 comp rx=0 tx=0).stop 2024-09-21T03:44:49.316 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:49.314+0000 7f656de6b640 1 -- 172.21.15.39:0/3691466932 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6568102a60 msgr2=0x7f656819a4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:49.316 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:49.314+0000 7f656de6b640 1 --2- 172.21.15.39:0/3691466932 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6568102a60 0x7f656819a4a0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f6558006010 tx=0x7f65580319f0 comp rx=0 tx=0).stop 2024-09-21T03:44:49.317 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:49.314+0000 7f656de6b640 1 -- 172.21.15.39:0/3691466932 shutdown_connections 2024-09-21T03:44:49.317 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:49.314+0000 7f656de6b640 1 --2- 172.21.15.39:0/3691466932 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f65540b65f0 0x7f65540b8b00 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:49.317 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:49.314+0000 7f656de6b640 1 --2- 172.21.15.39:0/3691466932 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6568108cf0 0x7f656819fce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:49.317 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:49.314+0000 7f656de6b640 1 --2- 172.21.15.39:0/3691466932 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f65681083a0 0x7f656819aa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:49.317 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:49.314+0000 7f656de6b640 1 --2- 172.21.15.39:0/3691466932 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6568102a60 0x7f656819a4a0 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:49.317 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:49.314+0000 7f656de6b640 1 -- 172.21.15.39:0/3691466932 >> 172.21.15.39:0/3691466932 conn(0x7f6568079730 msgr2=0x7f6568106740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:44:49.317 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:49.315+0000 7f656de6b640 1 -- 172.21.15.39:0/3691466932 shutdown_connections 2024-09-21T03:44:49.317 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:49.315+0000 7f656de6b640 1 -- 172.21.15.39:0/3691466932 wait complete. 2024-09-21T03:44:49.968 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:44:49 smithi161 ceph-mon[34533]: from='client.? 172.21.15.39:0/3691466932' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T03:44:49.974 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:44:49 smithi150 ceph-mon[34289]: from='client.? 172.21.15.39:0/3691466932' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T03:44:50.009 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:44:49 smithi039 ceph-mon[31211]: from='client.? 172.21.15.39:0/3691466932' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T03:44:50.040 INFO:tasks.cephadm.ceph_manager.ceph:all up! 2024-09-21T03:44:50.040 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 51116d5c-77ca-11ef-baf5-efdc52797490 -- ceph osd dump --format=json 2024-09-21T03:44:50.297 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/51116d5c-77ca-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:44:50.679 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:44:50 smithi039 ceph-mon[31211]: pgmap v289: 97 pgs: 97 active+clean; 645 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 2.4 KiB/s wr, 12 op/s 2024-09-21T03:44:50.779 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:50.771+0000 7ff614d50640 1 -- 172.21.15.39:0/236582338 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff610102a80 msgr2=0x7ff610102e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:50.779 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:50.771+0000 7ff614d50640 1 --2- 172.21.15.39:0/236582338 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff610102a80 0x7ff610102e90 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7ff600009900 tx=0x7ff60002a3f0 comp rx=0 tx=0).stop 2024-09-21T03:44:50.779 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:50.771+0000 7ff614d50640 1 -- 172.21.15.39:0/236582338 shutdown_connections 2024-09-21T03:44:50.779 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:50.771+0000 7ff614d50640 1 --2- 172.21.15.39:0/236582338 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff610108d10 0x7ff61010f540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:50.779 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:50.771+0000 7ff614d50640 1 --2- 172.21.15.39:0/236582338 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff6101083c0 0x7ff6101087d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:50.779 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:50.771+0000 7ff614d50640 1 --2- 172.21.15.39:0/236582338 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff610102a80 0x7ff610102e90 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:50.779 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:50.771+0000 7ff614d50640 1 -- 172.21.15.39:0/236582338 >> 172.21.15.39:0/236582338 conn(0x7ff610079740 msgr2=0x7ff610079b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:44:50.780 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:50.771+0000 7ff614d50640 1 -- 172.21.15.39:0/236582338 shutdown_connections 2024-09-21T03:44:50.780 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:50.771+0000 7ff614d50640 1 -- 172.21.15.39:0/236582338 wait complete. 2024-09-21T03:44:50.780 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:50.772+0000 7ff614d50640 1 Processor -- start 2024-09-21T03:44:50.780 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:50.772+0000 7ff614d50640 1 -- start start 2024-09-21T03:44:50.780 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:50.772+0000 7ff614d50640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff610102a80 0x7ff61019b2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:50.780 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:50.772+0000 7ff614d50640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff6101083c0 0x7ff61019b830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:50.780 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:50.772+0000 7ff614d50640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff610108d10 0x7ff61019bd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:50.780 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:50.772+0000 7ff614d50640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7ff6101129e0 con 0x7ff610102a80 2024-09-21T03:44:50.780 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:50.772+0000 7ff614d50640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7ff610112860 con 0x7ff6101083c0 2024-09-21T03:44:50.780 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:50.772+0000 7ff614d50640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7ff610112b60 con 0x7ff610108d10 2024-09-21T03:44:50.780 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:50.772+0000 7ff60e59c640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff610102a80 0x7ff61019b2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:50.780 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:50.772+0000 7ff60e59c640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff610102a80 0x7ff61019b2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:40308/0 (socket says 172.21.15.39:40308) 2024-09-21T03:44:50.780 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:50.772+0000 7ff60e59c640 1 -- 172.21.15.39:0/2998025527 learned_addr learned my addr 172.21.15.39:0/2998025527 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:44:50.780 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:50.772+0000 7ff60dd9b640 1 --2- 172.21.15.39:0/2998025527 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff6101083c0 0x7ff61019b830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:50.781 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:50.772+0000 7ff60ed9d640 1 --2- 172.21.15.39:0/2998025527 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff610108d10 0x7ff61019bd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:50.781 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:50.772+0000 7ff60e59c640 1 -- 172.21.15.39:0/2998025527 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff610108d10 msgr2=0x7ff61019bd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:50.781 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:50.772+0000 7ff60e59c640 1 --2- 172.21.15.39:0/2998025527 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff610108d10 0x7ff61019bd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:50.781 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:50.772+0000 7ff60e59c640 1 -- 172.21.15.39:0/2998025527 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff6101083c0 msgr2=0x7ff61019b830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:50.781 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:50.772+0000 7ff60e59c640 1 --2- 172.21.15.39:0/2998025527 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff6101083c0 0x7ff61019b830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:50.781 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:50.773+0000 7ff60e59c640 1 -- 172.21.15.39:0/2998025527 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff6101a64f0 con 0x7ff610102a80 2024-09-21T03:44:50.781 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:50.773+0000 7ff60ed9d640 1 --2- 172.21.15.39:0/2998025527 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff610108d10 0x7ff61019bd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:44:50.781 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:50.773+0000 7ff60e59c640 1 --2- 172.21.15.39:0/2998025527 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff610102a80 0x7ff61019b2f0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7ff600006010 tx=0x7ff6000042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:44:50.781 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:50.773+0000 7ff5ff7fe640 1 -- 172.21.15.39:0/2998025527 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff60003f030 con 0x7ff610102a80 2024-09-21T03:44:50.781 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:50.773+0000 7ff614d50640 1 -- 172.21.15.39:0/2998025527 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff6101a6780 con 0x7ff610102a80 2024-09-21T03:44:50.781 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:50.773+0000 7ff5ff7fe640 1 -- 172.21.15.39:0/2998025527 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff600003800 con 0x7ff610102a80 2024-09-21T03:44:50.781 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:50.773+0000 7ff614d50640 1 -- 172.21.15.39:0/2998025527 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff6101a6d30 con 0x7ff610102a80 2024-09-21T03:44:50.781 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:50.773+0000 7ff5ff7fe640 1 -- 172.21.15.39:0/2998025527 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff600035070 con 0x7ff610102a80 2024-09-21T03:44:50.782 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:50.774+0000 7ff614d50640 1 -- 172.21.15.39:0/2998025527 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff610103d00 con 0x7ff610102a80 2024-09-21T03:44:50.782 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:50.776+0000 7ff5ff7fe640 1 -- 172.21.15.39:0/2998025527 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 22) ==== 154580+0+0 (secure 0 0 0) 0x7ff600003cb0 con 0x7ff610102a80 2024-09-21T03:44:50.782 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:50.776+0000 7ff5ff7fe640 1 --2- 172.21.15.39:0/2998025527 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7ff5f80b6620 0x7ff5f80b8b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:50.782 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:50.776+0000 7ff60dd9b640 1 --2- 172.21.15.39:0/2998025527 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7ff5f80b6620 0x7ff5f80b8b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:50.782 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:50.777+0000 7ff5ff7fe640 1 -- 172.21.15.39:0/2998025527 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8484+0+0 (secure 0 0 0) 0x7ff600108280 con 0x7ff610102a80 2024-09-21T03:44:50.782 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:50.777+0000 7ff60dd9b640 1 --2- 172.21.15.39:0/2998025527 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7ff5f80b6620 0x7ff5f80b8b30 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7ff604005e10 tx=0x7ff604005da0 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:44:50.782 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:50.780+0000 7ff5ff7fe640 1 -- 172.21.15.39:0/2998025527 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7ff6000ce8b0 con 0x7ff610102a80 2024-09-21T03:44:50.967 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:44:50 smithi161 ceph-mon[34533]: pgmap v289: 97 pgs: 97 active+clean; 645 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 2.4 KiB/s wr, 12 op/s 2024-09-21T03:44:50.969 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:50.968+0000 7ff614d50640 1 -- 172.21.15.39:0/2998025527 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ff61006bf50 con 0x7ff610102a80 2024-09-21T03:44:50.971 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:50.969+0000 7ff5ff7fe640 1 -- 172.21.15.39:0/2998025527 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v79) ==== 74+0+23096 (secure 0 0 0) 0x7ff61006bf50 con 0x7ff610102a80 2024-09-21T03:44:50.971 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T03:44:50.972 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":79,"fsid":"51116d5c-77ca-11ef-baf5-efdc52797490","created":"2024-09-21T03:35:33.631350+0000","modified":"2024-09-21T03:44:27.402588+0000","last_up_change":"2024-09-21T03:43:24.242275+0000","last_in_change":"2024-09-21T03:43:00.886552+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T03:39:29.859542+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-21T03:44:10.886435+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"77","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-21T03:44:13.041783+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"77","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1ea7148f-2ace-4c10-86fe-b8019de80c1f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6802","nonce":870188791},{"type":"v1","addr":"172.21.15.39:6803","nonce":870188791}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6804","nonce":870188791},{"type":"v1","addr":"172.21.15.39:6805","nonce":870188791}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6808","nonce":870188791},{"type":"v1","addr":"172.21.15.39:6809","nonce":870188791}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6806","nonce":870188791},{"type":"v1","addr":"172.21.15.39:6807","nonce":870188791}]},"public_addr":"172.21.15.39:6803/870188791","cluster_addr":"172.21.15.39:6805/870188791","heartbeat_back_addr":"172.21.15.39:6809/870188791","heartbeat_front_addr":"172.21.15.39:6807/870188791","state":["exists","up"]},{"osd":1,"uuid":"7cb5db90-28e8-4a30-bedb-c039687d48b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6800","nonce":172427832},{"type":"v1","addr":"172.21.15.150:6801","nonce":172427832}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6802","nonce":172427832},{"type":"v1","addr":"172.21.15.150:6803","nonce":172427832}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6806","nonce":172427832},{"type":"v1","addr":"172.21.15.150:6807","nonce":172427832}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6804","nonce":172427832},{"type":"v1","addr":"172.21.15.150:6805","nonce":172427832}]},"public_addr":"172.21.15.150:6801/172427832","cluster_addr":"172.21.15.150:6803/172427832","heartbeat_back_addr":"172.21.15.150:6807/172427832","heartbeat_front_addr":"172.21.15.150:6805/172427832","state":["exists","up"]},{"osd":2,"uuid":"b25c6ab3-fc95-4751-bd63-6e3fc23e8bd7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":19,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6800","nonce":2727876133},{"type":"v1","addr":"172.21.15.161:6801","nonce":2727876133}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6802","nonce":2727876133},{"type":"v1","addr":"172.21.15.161:6803","nonce":2727876133}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6806","nonce":2727876133},{"type":"v1","addr":"172.21.15.161:6807","nonce":2727876133}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6804","nonce":2727876133},{"type":"v1","addr":"172.21.15.161:6805","nonce":2727876133}]},"public_addr":"172.21.15.161:6801/2727876133","cluster_addr":"172.21.15.161:6803/2727876133","heartbeat_back_addr":"172.21.15.161:6807/2727876133","heartbeat_front_addr":"172.21.15.161:6805/2727876133","state":["exists","up"]},{"osd":3,"uuid":"14417ae1-4536-45ba-9a5c-c3b447d1a72d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6810","nonce":295394468},{"type":"v1","addr":"172.21.15.39:6811","nonce":295394468}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6812","nonce":295394468},{"type":"v1","addr":"172.21.15.39:6813","nonce":295394468}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6816","nonce":295394468},{"type":"v1","addr":"172.21.15.39:6817","nonce":295394468}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6814","nonce":295394468},{"type":"v1","addr":"172.21.15.39:6815","nonce":295394468}]},"public_addr":"172.21.15.39:6811/295394468","cluster_addr":"172.21.15.39:6813/295394468","heartbeat_back_addr":"172.21.15.39:6817/295394468","heartbeat_front_addr":"172.21.15.39:6815/295394468","state":["exists","up"]},{"osd":4,"uuid":"33c732ef-20c2-43c4-abe6-045b687fc521","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6808","nonce":3097721553},{"type":"v1","addr":"172.21.15.150:6809","nonce":3097721553}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6810","nonce":3097721553},{"type":"v1","addr":"172.21.15.150:6811","nonce":3097721553}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6814","nonce":3097721553},{"type":"v1","addr":"172.21.15.150:6815","nonce":3097721553}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6812","nonce":3097721553},{"type":"v1","addr":"172.21.15.150:6813","nonce":3097721553}]},"public_addr":"172.21.15.150:6809/3097721553","cluster_addr":"172.21.15.150:6811/3097721553","heartbeat_back_addr":"172.21.15.150:6815/3097721553","heartbeat_front_addr":"172.21.15.150:6813/3097721553","state":["exists","up"]},{"osd":5,"uuid":"74673722-abd6-400a-bb50-ec3dfba0c5e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6808","nonce":742751023},{"type":"v1","addr":"172.21.15.161:6809","nonce":742751023}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6810","nonce":742751023},{"type":"v1","addr":"172.21.15.161:6811","nonce":742751023}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6814","nonce":742751023},{"type":"v1","addr":"172.21.15.161:6815","nonce":742751023}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6812","nonce":742751023},{"type":"v1","addr":"172.21.15.161:6813","nonce":742751023}]},"public_addr":"172.21.15.161:6809/742751023","cluster_addr":"172.21.15.161:6811/742751023","heartbeat_back_addr":"172.21.15.161:6815/742751023","heartbeat_front_addr":"172.21.15.161:6813/742751023","state":["exists","up"]},{"osd":6,"uuid":"06548677-74c4-4efa-8054-df09b957b649","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6818","nonce":1072700290},{"type":"v1","addr":"172.21.15.39:6819","nonce":1072700290}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6820","nonce":1072700290},{"type":"v1","addr":"172.21.15.39:6821","nonce":1072700290}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6824","nonce":1072700290},{"type":"v1","addr":"172.21.15.39:6825","nonce":1072700290}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6822","nonce":1072700290},{"type":"v1","addr":"172.21.15.39:6823","nonce":1072700290}]},"public_addr":"172.21.15.39:6819/1072700290","cluster_addr":"172.21.15.39:6821/1072700290","heartbeat_back_addr":"172.21.15.39:6825/1072700290","heartbeat_front_addr":"172.21.15.39:6823/1072700290","state":["exists","up"]},{"osd":7,"uuid":"838735ca-c693-4f58-9426-b8ee857dbe3a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6816","nonce":2162619103},{"type":"v1","addr":"172.21.15.150:6817","nonce":2162619103}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6818","nonce":2162619103},{"type":"v1","addr":"172.21.15.150:6819","nonce":2162619103}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6822","nonce":2162619103},{"type":"v1","addr":"172.21.15.150:6823","nonce":2162619103}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6820","nonce":2162619103},{"type":"v1","addr":"172.21.15.150:6821","nonce":2162619103}]},"public_addr":"172.21.15.150:6817/2162619103","cluster_addr":"172.21.15.150:6819/2162619103","heartbeat_back_addr":"172.21.15.150:6823/2162619103","heartbeat_front_addr":"172.21.15.150:6821/2162619103","state":["exists","up"]},{"osd":8,"uuid":"4973b357-b4c2-4c00-b7f7-dd25012a0f72","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6816","nonce":52966633},{"type":"v1","addr":"172.21.15.161:6817","nonce":52966633}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6818","nonce":52966633},{"type":"v1","addr":"172.21.15.161:6819","nonce":52966633}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6822","nonce":52966633},{"type":"v1","addr":"172.21.15.161:6823","nonce":52966633}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6820","nonce":52966633},{"type":"v1","addr":"172.21.15.161:6821","nonce":52966633}]},"public_addr":"172.21.15.161:6817/52966633","cluster_addr":"172.21.15.161:6819/52966633","heartbeat_back_addr":"172.21.15.161:6823/52966633","heartbeat_front_addr":"172.21.15.161:6821/52966633","state":["exists","up"]},{"osd":9,"uuid":"6028613c-ebfd-42ea-9949-3f0d5f6cf025","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6826","nonce":2001303253},{"type":"v1","addr":"172.21.15.39:6827","nonce":2001303253}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6828","nonce":2001303253},{"type":"v1","addr":"172.21.15.39:6829","nonce":2001303253}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6832","nonce":2001303253},{"type":"v1","addr":"172.21.15.39:6833","nonce":2001303253}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6830","nonce":2001303253},{"type":"v1","addr":"172.21.15.39:6831","nonce":2001303253}]},"public_addr":"172.21.15.39:6827/2001303253","cluster_addr":"172.21.15.39:6829/2001303253","heartbeat_back_addr":"172.21.15.39:6833/2001303253","heartbeat_front_addr":"172.21.15.39:6831/2001303253","state":["exists","up"]},{"osd":10,"uuid":"1c71c765-2a6c-4ffe-9fe1-c8adae3d1c32","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6824","nonce":2484627938},{"type":"v1","addr":"172.21.15.150:6825","nonce":2484627938}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6826","nonce":2484627938},{"type":"v1","addr":"172.21.15.150:6827","nonce":2484627938}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6830","nonce":2484627938},{"type":"v1","addr":"172.21.15.150:6831","nonce":2484627938}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6828","nonce":2484627938},{"type":"v1","addr":"172.21.15.150:6829","nonce":2484627938}]},"public_addr":"172.21.15.150:6825/2484627938","cluster_addr":"172.21.15.150:6827/2484627938","heartbeat_back_addr":"172.21.15.150:6831/2484627938","heartbeat_front_addr":"172.21.15.150:6829/2484627938","state":["exists","up"]},{"osd":11,"uuid":"f382f522-62db-46da-997e-2b28f84d1612","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6824","nonce":2178709707},{"type":"v1","addr":"172.21.15.161:6825","nonce":2178709707}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6826","nonce":2178709707},{"type":"v1","addr":"172.21.15.161:6827","nonce":2178709707}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6830","nonce":2178709707},{"type":"v1","addr":"172.21.15.161:6831","nonce":2178709707}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6828","nonce":2178709707},{"type":"v1","addr":"172.21.15.161:6829","nonce":2178709707}]},"public_addr":"172.21.15.161:6825/2178709707","cluster_addr":"172.21.15.161:6827/2178709707","heartbeat_back_addr":"172.21.15.161:6831/2178709707","heartbeat_front_addr":"172.21.15.161:6829/2178709707","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:38:35.540904+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:39:01.455053+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:39:27.052867+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:39:55.173529+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:40:21.298396+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:40:47.131235+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:41:14.475209+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:41:39.920036+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:42:05.587065+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:42:32.624726+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:42:57.479185+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:43:22.203832+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.5","mappings":[{"from":8,"to":11}]},{"pgid":"2.8","mappings":[{"from":2,"to":11}]},{"pgid":"2.c","mappings":[{"from":2,"to":1}]},{"pgid":"2.1f","mappings":[{"from":8,"to":4}]},{"pgid":"3.2","mappings":[{"from":3,"to":9}]},{"pgid":"3.14","mappings":[{"from":4,"to":6}]},{"pgid":"3.1a","mappings":[{"from":5,"to":7}]},{"pgid":"3.1d","mappings":[{"from":4,"to":8}]},{"pgid":"3.2c","mappings":[{"from":4,"to":9}]},{"pgid":"3.38","mappings":[{"from":3,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.39:6801/2438110611":"2024-09-22T03:37:25.391457+0000","172.21.15.39:6800/2438110611":"2024-09-22T03:37:25.391457+0000","172.21.15.39:6800/173264695":"2024-09-22T03:35:57.104467+0000","172.21.15.39:6801/173264695":"2024-09-22T03:35:57.104467+0000","172.21.15.39:0/2637182284":"2024-09-22T03:35:57.104467+0000","172.21.15.39:0/1517304561":"2024-09-22T03:36:24.112592+0000","172.21.15.39:6801/1435622199":"2024-09-22T03:36:24.112592+0000","172.21.15.39:6800/1435622199":"2024-09-22T03:36:24.112592+0000","172.21.15.39:0/346832469":"2024-09-22T03:35:57.104467+0000","172.21.15.39:0/3160777371":"2024-09-22T03:36:24.112592+0000","172.21.15.39:0/1730342855":"2024-09-22T03:35:57.104467+0000","172.21.15.39:0/1955726766":"2024-09-22T03:36:24.112592+0000","172.21.15.39:0/3294132796":"2024-09-22T03:37:25.391457+0000","172.21.15.39:0/1295810162":"2024-09-22T03:37:25.391457+0000","172.21.15.39:0/3729230474":"2024-09-22T03:37:25.391457+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T03:44:50.974 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:50.972+0000 7ff614d50640 1 -- 172.21.15.39:0/2998025527 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7ff5f80b6620 msgr2=0x7ff5f80b8b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:50.974 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:50.972+0000 7ff614d50640 1 --2- 172.21.15.39:0/2998025527 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7ff5f80b6620 0x7ff5f80b8b30 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7ff604005e10 tx=0x7ff604005da0 comp rx=0 tx=0).stop 2024-09-21T03:44:50.975 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:50.972+0000 7ff614d50640 1 -- 172.21.15.39:0/2998025527 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff610102a80 msgr2=0x7ff61019b2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:50.975 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:50.972+0000 7ff614d50640 1 --2- 172.21.15.39:0/2998025527 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff610102a80 0x7ff61019b2f0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7ff600006010 tx=0x7ff6000042f0 comp rx=0 tx=0).stop 2024-09-21T03:44:50.975 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:50.973+0000 7ff614d50640 1 -- 172.21.15.39:0/2998025527 shutdown_connections 2024-09-21T03:44:50.975 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:50.973+0000 7ff614d50640 1 --2- 172.21.15.39:0/2998025527 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7ff5f80b6620 0x7ff5f80b8b30 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:50.975 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:50.973+0000 7ff614d50640 1 --2- 172.21.15.39:0/2998025527 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff610108d10 0x7ff61019bd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:50.975 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:50.973+0000 7ff614d50640 1 --2- 172.21.15.39:0/2998025527 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff6101083c0 0x7ff61019b830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:50.975 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:50.973+0000 7ff614d50640 1 --2- 172.21.15.39:0/2998025527 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff610102a80 0x7ff61019b2f0 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:50.975 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:50.973+0000 7ff614d50640 1 -- 172.21.15.39:0/2998025527 >> 172.21.15.39:0/2998025527 conn(0x7ff610079740 msgr2=0x7ff610101440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:44:50.975 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:50.973+0000 7ff614d50640 1 -- 172.21.15.39:0/2998025527 shutdown_connections 2024-09-21T03:44:50.975 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:50.973+0000 7ff614d50640 1 -- 172.21.15.39:0/2998025527 wait complete. 2024-09-21T03:44:50.976 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:44:50 smithi150 ceph-mon[34289]: pgmap v289: 97 pgs: 97 active+clean; 645 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 2.4 KiB/s wr, 12 op/s 2024-09-21T03:44:51.773 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 51116d5c-77ca-11ef-baf5-efdc52797490 -- ceph tell osd.0 flush_pg_stats 2024-09-21T03:44:51.773 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 51116d5c-77ca-11ef-baf5-efdc52797490 -- ceph tell osd.1 flush_pg_stats 2024-09-21T03:44:51.774 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 51116d5c-77ca-11ef-baf5-efdc52797490 -- ceph tell osd.2 flush_pg_stats 2024-09-21T03:44:51.774 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 51116d5c-77ca-11ef-baf5-efdc52797490 -- ceph tell osd.3 flush_pg_stats 2024-09-21T03:44:51.775 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 51116d5c-77ca-11ef-baf5-efdc52797490 -- ceph tell osd.4 flush_pg_stats 2024-09-21T03:44:51.775 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 51116d5c-77ca-11ef-baf5-efdc52797490 -- ceph tell osd.5 flush_pg_stats 2024-09-21T03:44:51.775 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 51116d5c-77ca-11ef-baf5-efdc52797490 -- ceph tell osd.6 flush_pg_stats 2024-09-21T03:44:51.776 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 51116d5c-77ca-11ef-baf5-efdc52797490 -- ceph tell osd.7 flush_pg_stats 2024-09-21T03:44:51.776 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 51116d5c-77ca-11ef-baf5-efdc52797490 -- ceph tell osd.8 flush_pg_stats 2024-09-21T03:44:51.777 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 51116d5c-77ca-11ef-baf5-efdc52797490 -- ceph tell osd.9 flush_pg_stats 2024-09-21T03:44:51.777 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 51116d5c-77ca-11ef-baf5-efdc52797490 -- ceph tell osd.10 flush_pg_stats 2024-09-21T03:44:51.777 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 51116d5c-77ca-11ef-baf5-efdc52797490 -- ceph tell osd.11 flush_pg_stats 2024-09-21T03:44:51.873 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:44:51 smithi039 ceph-mon[31211]: from='client.? 172.21.15.39:0/2998025527' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T03:44:51.968 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:44:51 smithi161 ceph-mon[34533]: from='client.? 172.21.15.39:0/2998025527' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T03:44:51.974 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:44:51 smithi150 ceph-mon[34289]: from='client.? 172.21.15.39:0/2998025527' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T03:44:52.423 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/51116d5c-77ca-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:44:52.818 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/51116d5c-77ca-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:44:52.821 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/51116d5c-77ca-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:44:52.821 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/51116d5c-77ca-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:44:52.823 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/51116d5c-77ca-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:44:52.826 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/51116d5c-77ca-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:44:52.836 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:44:52 smithi039 ceph-mon[31211]: pgmap v290: 97 pgs: 97 active+clean; 645 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 1.2 KiB/s wr, 10 op/s 2024-09-21T03:44:52.840 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/51116d5c-77ca-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:44:52.841 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/51116d5c-77ca-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:44:52.841 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/51116d5c-77ca-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:44:52.844 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/51116d5c-77ca-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:44:52.967 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:44:52 smithi161 ceph-mon[34533]: pgmap v290: 97 pgs: 97 active+clean; 645 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 1.2 KiB/s wr, 10 op/s 2024-09-21T03:44:52.974 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:44:52 smithi150 ceph-mon[34289]: pgmap v290: 97 pgs: 97 active+clean; 645 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 1.2 KiB/s wr, 10 op/s 2024-09-21T03:44:53.094 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:53.092+0000 7f31d7021640 1 -- 172.21.15.39:0/1719500163 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f31d0108d20 msgr2=0x7f31d010f550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:53.094 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:53.092+0000 7f31d7021640 1 --2- 172.21.15.39:0/1719500163 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f31d0108d20 0x7f31d010f550 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f31c8009ca0 tx=0x7f31c802a8c0 comp rx=0 tx=0).stop 2024-09-21T03:44:53.094 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:53.092+0000 7f31d7021640 1 -- 172.21.15.39:0/1719500163 shutdown_connections 2024-09-21T03:44:53.095 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:53.092+0000 7f31d7021640 1 --2- 172.21.15.39:0/1719500163 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f31d0108d20 0x7f31d010f550 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:53.095 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:53.092+0000 7f31d7021640 1 --2- 172.21.15.39:0/1719500163 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f31d01083d0 0x7f31d01087e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:53.095 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:53.092+0000 7f31d7021640 1 --2- 172.21.15.39:0/1719500163 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f31d0102a90 0x7f31d0102ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:53.095 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:53.092+0000 7f31d7021640 1 -- 172.21.15.39:0/1719500163 >> 172.21.15.39:0/1719500163 conn(0x7f31d0079730 msgr2=0x7f31d0079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:44:53.095 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:53.092+0000 7f31d7021640 1 -- 172.21.15.39:0/1719500163 shutdown_connections 2024-09-21T03:44:53.095 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:53.092+0000 7f31d7021640 1 -- 172.21.15.39:0/1719500163 wait complete. 2024-09-21T03:44:53.096 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:53.093+0000 7f31d7021640 1 Processor -- start 2024-09-21T03:44:53.096 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:53.093+0000 7f31d7021640 1 -- start start 2024-09-21T03:44:53.096 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:53.094+0000 7f31d7021640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f31d0102a90 0x7f31d019b320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:53.096 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:53.094+0000 7f31d4dbd640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f31d0102a90 0x7f31d019b320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:53.096 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:53.094+0000 7f31d4dbd640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f31d0102a90 0x7f31d019b320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:40326/0 (socket says 172.21.15.39:40326) 2024-09-21T03:44:53.096 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:53.094+0000 7f31d4dbd640 1 -- 172.21.15.39:0/2089465007 learned_addr learned my addr 172.21.15.39:0/2089465007 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:44:53.096 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:53.094+0000 7f31d7021640 1 --2- 172.21.15.39:0/2089465007 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f31d01083d0 0x7f31d019b860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:53.096 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:53.094+0000 7f31d7021640 1 --2- 172.21.15.39:0/2089465007 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f31d0108d20 0x7f31d019bda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:53.097 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:53.095+0000 7f31d7021640 1 -- 172.21.15.39:0/2089465007 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f31d0112a30 con 0x7f31d0102a90 2024-09-21T03:44:53.097 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:53.095+0000 7f31d7021640 1 -- 172.21.15.39:0/2089465007 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f31d01128b0 con 0x7f31d01083d0 2024-09-21T03:44:53.097 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:53.095+0000 7f31d7021640 1 -- 172.21.15.39:0/2089465007 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f31d0112bb0 con 0x7f31d0108d20 2024-09-21T03:44:53.098 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:53.095+0000 7f31d55be640 1 --2- 172.21.15.39:0/2089465007 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f31d0108d20 0x7f31d019bda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:53.098 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:53.095+0000 7f31cffff640 1 --2- 172.21.15.39:0/2089465007 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f31d01083d0 0x7f31d019b860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:53.098 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:53.095+0000 7f31d4dbd640 1 -- 172.21.15.39:0/2089465007 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f31d0108d20 msgr2=0x7f31d019bda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:53.098 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:53.095+0000 7f31d4dbd640 1 --2- 172.21.15.39:0/2089465007 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f31d0108d20 0x7f31d019bda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:53.098 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:53.095+0000 7f31d4dbd640 1 -- 172.21.15.39:0/2089465007 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f31d01083d0 msgr2=0x7f31d019b860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:53.098 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:53.095+0000 7f31d4dbd640 1 --2- 172.21.15.39:0/2089465007 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f31d01083d0 0x7f31d019b860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:53.098 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:53.095+0000 7f31d4dbd640 1 -- 172.21.15.39:0/2089465007 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f31d01a65b0 con 0x7f31d0102a90 2024-09-21T03:44:53.098 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:53.095+0000 7f31d55be640 1 --2- 172.21.15.39:0/2089465007 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f31d0108d20 0x7f31d019bda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:44:53.099 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:53.095+0000 7f31cffff640 1 --2- 172.21.15.39:0/2089465007 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f31d01083d0 0x7f31d019b860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:44:53.099 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:53.095+0000 7f31d4dbd640 1 --2- 172.21.15.39:0/2089465007 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f31d0102a90 0x7f31d019b320 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f31c400d950 tx=0x7f31c400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:44:53.099 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:53.095+0000 7f31cdffb640 1 -- 172.21.15.39:0/2089465007 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31c4017070 con 0x7f31d0102a90 2024-09-21T03:44:53.099 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:53.096+0000 7f31d7021640 1 -- 172.21.15.39:0/2089465007 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f31d01a68a0 con 0x7f31d0102a90 2024-09-21T03:44:53.099 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:53.096+0000 7f31cdffb640 1 -- 172.21.15.39:0/2089465007 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f31c4009c30 con 0x7f31d0102a90 2024-09-21T03:44:53.099 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:53.096+0000 7f31cdffb640 1 -- 172.21.15.39:0/2089465007 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31c400f620 con 0x7f31d0102a90 2024-09-21T03:44:53.099 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:53.096+0000 7f31d7021640 1 -- 172.21.15.39:0/2089465007 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f31d01992e0 con 0x7f31d0102a90 2024-09-21T03:44:53.099 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:53.097+0000 7f31d7021640 1 -- 172.21.15.39:0/2089465007 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f31d0063b50 con 0x7f31d0102a90 2024-09-21T03:44:53.100 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:53.099+0000 7f31cdffb640 1 -- 172.21.15.39:0/2089465007 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 22) ==== 154580+0+0 (secure 0 0 0) 0x7f31c4010460 con 0x7f31d0102a90 2024-09-21T03:44:53.101 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:53.099+0000 7f31cdffb640 1 --2- 172.21.15.39:0/2089465007 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f31c00baa30 0x7f31c00bcf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:53.101 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:53.099+0000 7f31cffff640 1 --2- 172.21.15.39:0/2089465007 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f31c00baa30 0x7f31c00bcf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:53.101 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:53.099+0000 7f31cdffb640 1 -- 172.21.15.39:0/2089465007 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8484+0+0 (secure 0 0 0) 0x7f31c40e7810 con 0x7f31d0102a90 2024-09-21T03:44:53.101 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:53.100+0000 7f31cdffb640 1 --2- 172.21.15.39:0/2089465007 >> [v2:172.21.15.39:6802/870188791,v1:172.21.15.39:6803/870188791] conn(0x7f31c00c48a0 0x7f31c00c6d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:53.101 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:53.100+0000 7f31cdffb640 1 -- 172.21.15.39:0/2089465007 --> [v2:172.21.15.39:6802/870188791,v1:172.21.15.39:6803/870188791] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f31c00c7460 con 0x7f31c00c48a0 2024-09-21T03:44:53.101 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:53.100+0000 7f31cdffb640 1 -- 172.21.15.39:0/2089465007 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_get_version_reply(handle=1 version=79) ==== 24+0+0 (secure 0 0 0) 0x7f31c40e7c60 con 0x7f31d0102a90 2024-09-21T03:44:53.102 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:53.100+0000 7f31d55be640 1 --2- 172.21.15.39:0/2089465007 >> [v2:172.21.15.39:6802/870188791,v1:172.21.15.39:6803/870188791] conn(0x7f31c00c48a0 0x7f31c00c6d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:53.102 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:53.100+0000 7f31cffff640 1 --2- 172.21.15.39:0/2089465007 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f31c00baa30 0x7f31c00bcf40 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f31bc005d70 tx=0x7f31bc005d00 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:44:53.102 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:53.100+0000 7f31d55be640 1 --2- 172.21.15.39:0/2089465007 >> [v2:172.21.15.39:6802/870188791,v1:172.21.15.39:6803/870188791] conn(0x7f31c00c48a0 0x7f31c00c6d90 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:44:53.102 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:53.101+0000 7f31cdffb640 1 -- 172.21.15.39:0/2089465007 <== osd.0 v2:172.21.15.39:6802/870188791 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f31c00c7460 con 0x7f31c00c48a0 2024-09-21T03:44:53.116 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:53.114+0000 7f31d7021640 1 -- 172.21.15.39:0/2089465007 --> [v2:172.21.15.39:6802/870188791,v1:172.21.15.39:6803/870188791] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f31d0063f50 con 0x7f31c00c48a0 2024-09-21T03:44:53.116 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:53.115+0000 7f31cdffb640 1 -- 172.21.15.39:0/2089465007 <== osd.0 v2:172.21.15.39:6802/870188791 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f31d0063f50 con 0x7f31c00c48a0 2024-09-21T03:44:53.117 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:53.115+0000 7f31d7021640 1 -- 172.21.15.39:0/2089465007 >> [v2:172.21.15.39:6802/870188791,v1:172.21.15.39:6803/870188791] conn(0x7f31c00c48a0 msgr2=0x7f31c00c6d90 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:53.117 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:53.115+0000 7f31d7021640 1 --2- 172.21.15.39:0/2089465007 >> [v2:172.21.15.39:6802/870188791,v1:172.21.15.39:6803/870188791] conn(0x7f31c00c48a0 0x7f31c00c6d90 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:53.117 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:53.115+0000 7f31d7021640 1 -- 172.21.15.39:0/2089465007 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f31c00baa30 msgr2=0x7f31c00bcf40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:53.117 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:53.115+0000 7f31d7021640 1 --2- 172.21.15.39:0/2089465007 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f31c00baa30 0x7f31c00bcf40 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f31bc005d70 tx=0x7f31bc005d00 comp rx=0 tx=0).stop 2024-09-21T03:44:53.117 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:53.115+0000 7f31d7021640 1 -- 172.21.15.39:0/2089465007 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f31d0102a90 msgr2=0x7f31d019b320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:53.117 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:53.115+0000 7f31d7021640 1 --2- 172.21.15.39:0/2089465007 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f31d0102a90 0x7f31d019b320 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f31c400d950 tx=0x7f31c400de10 comp rx=0 tx=0).stop 2024-09-21T03:44:53.117 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:53.115+0000 7f31d55be640 1 -- 172.21.15.39:0/2089465007 reap_dead start 2024-09-21T03:44:53.118 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:53.115+0000 7f31d7021640 1 -- 172.21.15.39:0/2089465007 shutdown_connections 2024-09-21T03:44:53.118 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:53.115+0000 7f31d7021640 1 -- 172.21.15.39:0/2089465007 >> 172.21.15.39:0/2089465007 conn(0x7f31d0079730 msgr2=0x7f31d0106420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:44:53.118 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:53.115+0000 7f31d7021640 1 -- 172.21.15.39:0/2089465007 shutdown_connections 2024-09-21T03:44:53.118 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:53.115+0000 7f31d7021640 1 -- 172.21.15.39:0/2089465007 wait complete. 2024-09-21T03:44:54.013 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:54.009+0000 7fcea9680640 1 -- 172.21.15.39:0/2518642673 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcea41052c0 msgr2=0x7fcea410d900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:54.013 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:54.009+0000 7fcea9680640 1 --2- 172.21.15.39:0/2518642673 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcea41052c0 0x7fcea410d900 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fce98009900 tx=0x7fce9802a3f0 comp rx=0 tx=0).stop 2024-09-21T03:44:54.013 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:54.010+0000 7fcea9680640 1 -- 172.21.15.39:0/2518642673 shutdown_connections 2024-09-21T03:44:54.013 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:54.010+0000 7fcea9680640 1 --2- 172.21.15.39:0/2518642673 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcea410de40 0x7fcea4110310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:54.014 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:54.010+0000 7fcea9680640 1 --2- 172.21.15.39:0/2518642673 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcea41052c0 0x7fcea410d900 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:54.014 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:54.010+0000 7fcea9680640 1 --2- 172.21.15.39:0/2518642673 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcea4104890 0x7fcea4104ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:54.014 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:54.010+0000 7fcea9680640 1 -- 172.21.15.39:0/2518642673 >> 172.21.15.39:0/2518642673 conn(0x7fcea4079730 msgr2=0x7fcea4079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:44:54.014 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:54.010+0000 7fcea9680640 1 -- 172.21.15.39:0/2518642673 shutdown_connections 2024-09-21T03:44:54.014 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:54.011+0000 7fcea9680640 1 -- 172.21.15.39:0/2518642673 wait complete. 2024-09-21T03:44:54.018 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:54.012+0000 7fcea9680640 1 Processor -- start 2024-09-21T03:44:54.018 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:54.012+0000 7fcea9680640 1 -- start start 2024-09-21T03:44:54.018 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:54.012+0000 7fcea9680640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcea4104890 0x7fcea419a2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:54.018 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:54.012+0000 7fcea9680640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcea41052c0 0x7fcea419a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:54.018 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:54.012+0000 7fcea9680640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcea410de40 0x7fcea419fae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:54.019 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:54.013+0000 7fcea37fe640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcea410de40 0x7fcea419fae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:54.019 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:54.013+0000 7fcea37fe640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcea410de40 0x7fcea419fae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:40354/0 (socket says 172.21.15.39:40354) 2024-09-21T03:44:54.019 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:54.013+0000 7fcea37fe640 1 -- 172.21.15.39:0/3565839890 learned_addr learned my addr 172.21.15.39:0/3565839890 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:44:54.019 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:54.013+0000 7fcea9680640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fcea4113a80 con 0x7fcea410de40 2024-09-21T03:44:54.019 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:54.013+0000 7fcea27fc640 1 --2- 172.21.15.39:0/3565839890 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcea41052c0 0x7fcea419a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:54.019 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:54.013+0000 7fcea9680640 1 -- 172.21.15.39:0/3565839890 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fcea4113900 con 0x7fcea4104890 2024-09-21T03:44:54.020 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:54.013+0000 7fcea9680640 1 -- 172.21.15.39:0/3565839890 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fcea4113c00 con 0x7fcea41052c0 2024-09-21T03:44:54.020 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:54.013+0000 7fcea2ffd640 1 --2- 172.21.15.39:0/3565839890 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcea4104890 0x7fcea419a2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:54.020 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:54.013+0000 7fcea37fe640 1 -- 172.21.15.39:0/3565839890 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcea41052c0 msgr2=0x7fcea419a840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:54.020 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:54.013+0000 7fcea37fe640 1 --2- 172.21.15.39:0/3565839890 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcea41052c0 0x7fcea419a840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:54.020 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:54.013+0000 7fcea37fe640 1 -- 172.21.15.39:0/3565839890 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcea4104890 msgr2=0x7fcea419a2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:54.020 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:54.013+0000 7fcea37fe640 1 --2- 172.21.15.39:0/3565839890 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcea4104890 0x7fcea419a2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:54.020 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:54.013+0000 7fcea37fe640 1 -- 172.21.15.39:0/3565839890 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcea41a0150 con 0x7fcea410de40 2024-09-21T03:44:54.021 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:54.014+0000 7fcea37fe640 1 --2- 172.21.15.39:0/3565839890 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcea410de40 0x7fcea419fae0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fce9c009fd0 tx=0x7fce9c00b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:44:54.021 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:54.014+0000 7fcea2ffd640 1 --2- 172.21.15.39:0/3565839890 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcea4104890 0x7fcea419a2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:44:54.021 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:54.014+0000 7fce8bfff640 1 -- 172.21.15.39:0/3565839890 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce9c017070 con 0x7fcea410de40 2024-09-21T03:44:54.021 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:54.014+0000 7fce8bfff640 1 -- 172.21.15.39:0/3565839890 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fce9c003e20 con 0x7fcea410de40 2024-09-21T03:44:54.021 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:54.014+0000 7fce8bfff640 1 -- 172.21.15.39:0/3565839890 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce9c004820 con 0x7fcea410de40 2024-09-21T03:44:54.021 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:54.014+0000 7fcea9680640 1 -- 172.21.15.39:0/3565839890 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcea41a0440 con 0x7fcea410de40 2024-09-21T03:44:54.022 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:54.014+0000 7fcea9680640 1 -- 172.21.15.39:0/3565839890 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcea41a0870 con 0x7fcea410de40 2024-09-21T03:44:54.022 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:54.015+0000 7fcea9680640 1 -- 172.21.15.39:0/3565839890 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fcea41072c0 con 0x7fcea410de40 2024-09-21T03:44:54.023 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:54.017+0000 7fce8bfff640 1 -- 172.21.15.39:0/3565839890 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 22) ==== 154580+0+0 (secure 0 0 0) 0x7fce9c004140 con 0x7fcea410de40 2024-09-21T03:44:54.024 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:54.018+0000 7fce8bfff640 1 --2- 172.21.15.39:0/3565839890 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7fce8c0b66a0 0x7fce8c0b8bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:54.024 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:54.018+0000 7fcea2ffd640 1 --2- 172.21.15.39:0/3565839890 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7fce8c0b66a0 0x7fce8c0b8bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:54.024 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:54.018+0000 7fce8bfff640 1 -- 172.21.15.39:0/3565839890 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8484+0+0 (secure 0 0 0) 0x7fce9c0e76b0 con 0x7fcea410de40 2024-09-21T03:44:54.024 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:54.018+0000 7fce8bfff640 1 --2- 172.21.15.39:0/3565839890 >> [v2:172.21.15.39:6818/1072700290,v1:172.21.15.39:6819/1072700290] conn(0x7fce8c0c0510 0x7fce8c0c2a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:54.024 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:54.018+0000 7fce8bfff640 1 -- 172.21.15.39:0/3565839890 --> [v2:172.21.15.39:6818/1072700290,v1:172.21.15.39:6819/1072700290] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fce9c022e70 con 0x7fce8c0c0510 2024-09-21T03:44:54.025 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:54.018+0000 7fce8bfff640 1 -- 172.21.15.39:0/3565839890 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_get_version_reply(handle=1 version=79) ==== 24+0+0 (secure 0 0 0) 0x7fce9c0e7b00 con 0x7fcea410de40 2024-09-21T03:44:54.025 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:54.018+0000 7fcea27fc640 1 --2- 172.21.15.39:0/3565839890 >> [v2:172.21.15.39:6818/1072700290,v1:172.21.15.39:6819/1072700290] conn(0x7fce8c0c0510 0x7fce8c0c2a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:54.025 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:54.018+0000 7fcea2ffd640 1 --2- 172.21.15.39:0/3565839890 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7fce8c0b66a0 0x7fce8c0b8bb0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fcea406c840 tx=0x7fce94005db0 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:44:54.025 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:54.019+0000 7fcea27fc640 1 --2- 172.21.15.39:0/3565839890 >> [v2:172.21.15.39:6818/1072700290,v1:172.21.15.39:6819/1072700290] conn(0x7fce8c0c0510 0x7fce8c0c2a00 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:44:54.025 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:54.019+0000 7fce8bfff640 1 -- 172.21.15.39:0/3565839890 <== osd.6 v2:172.21.15.39:6818/1072700290 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fce9c022e70 con 0x7fce8c0c0510 2024-09-21T03:44:54.035 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:54.033+0000 7fcea9680640 1 -- 172.21.15.39:0/3565839890 --> [v2:172.21.15.39:6818/1072700290,v1:172.21.15.39:6819/1072700290] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fcea406b390 con 0x7fce8c0c0510 2024-09-21T03:44:54.035 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:54.034+0000 7fce8bfff640 1 -- 172.21.15.39:0/3565839890 <== osd.6 v2:172.21.15.39:6818/1072700290 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fcea406b390 con 0x7fce8c0c0510 2024-09-21T03:44:54.037 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:54.034+0000 7fcea9680640 1 -- 172.21.15.39:0/3565839890 >> [v2:172.21.15.39:6818/1072700290,v1:172.21.15.39:6819/1072700290] conn(0x7fce8c0c0510 msgr2=0x7fce8c0c2a00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:54.037 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:54.034+0000 7fcea9680640 1 --2- 172.21.15.39:0/3565839890 >> [v2:172.21.15.39:6818/1072700290,v1:172.21.15.39:6819/1072700290] conn(0x7fce8c0c0510 0x7fce8c0c2a00 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:54.037 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:54.034+0000 7fcea9680640 1 -- 172.21.15.39:0/3565839890 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7fce8c0b66a0 msgr2=0x7fce8c0b8bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:54.037 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:54.034+0000 7fcea9680640 1 --2- 172.21.15.39:0/3565839890 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7fce8c0b66a0 0x7fce8c0b8bb0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fcea406c840 tx=0x7fce94005db0 comp rx=0 tx=0).stop 2024-09-21T03:44:54.038 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:54.034+0000 7fcea9680640 1 -- 172.21.15.39:0/3565839890 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcea410de40 msgr2=0x7fcea419fae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:54.038 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:54.034+0000 7fcea9680640 1 --2- 172.21.15.39:0/3565839890 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcea410de40 0x7fcea419fae0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fce9c009fd0 tx=0x7fce9c00b260 comp rx=0 tx=0).stop 2024-09-21T03:44:54.038 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:54.034+0000 7fcea37fe640 1 -- 172.21.15.39:0/3565839890 reap_dead start 2024-09-21T03:44:54.038 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:54.034+0000 7fcea9680640 1 -- 172.21.15.39:0/3565839890 shutdown_connections 2024-09-21T03:44:54.038 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:54.034+0000 7fcea9680640 1 -- 172.21.15.39:0/3565839890 >> 172.21.15.39:0/3565839890 conn(0x7fcea4079730 msgr2=0x7fcea4105850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:44:54.038 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:54.034+0000 7fcea9680640 1 -- 172.21.15.39:0/3565839890 shutdown_connections 2024-09-21T03:44:54.038 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:54.035+0000 7fcea9680640 1 -- 172.21.15.39:0/3565839890 wait complete. 2024-09-21T03:44:54.276 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:54.237+0000 7f5097436640 1 -- 172.21.15.39:0/2689273638 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f509010de90 msgr2=0x7f5090110330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:54.277 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:54.237+0000 7f5097436640 1 --2- 172.21.15.39:0/2689273638 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f509010de90 0x7f5090110330 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f5088009ca0 tx=0x7f508802a8c0 comp rx=0 tx=0).stop 2024-09-21T03:44:54.277 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:54.237+0000 7f5097436640 1 -- 172.21.15.39:0/2689273638 shutdown_connections 2024-09-21T03:44:54.277 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:54.237+0000 7f5097436640 1 --2- 172.21.15.39:0/2689273638 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f509010de90 0x7f5090110330 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:54.277 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:54.237+0000 7f5097436640 1 --2- 172.21.15.39:0/2689273638 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5090105310 0x7f509010d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:54.277 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:54.237+0000 7f5097436640 1 --2- 172.21.15.39:0/2689273638 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f50901048e0 0x7f5090104cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:54.277 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:54.237+0000 7f5097436640 1 -- 172.21.15.39:0/2689273638 >> 172.21.15.39:0/2689273638 conn(0x7f50900797b0 msgr2=0x7f5090079bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:44:54.277 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:54.237+0000 7f5097436640 1 -- 172.21.15.39:0/2689273638 shutdown_connections 2024-09-21T03:44:54.278 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:54.237+0000 7f5097436640 1 -- 172.21.15.39:0/2689273638 wait complete. 2024-09-21T03:44:54.278 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:54.238+0000 7f5097436640 1 Processor -- start 2024-09-21T03:44:54.278 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:54.238+0000 7f5097436640 1 -- start start 2024-09-21T03:44:54.278 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:54.238+0000 7f5097436640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f50901048e0 0x7f509019a470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:54.278 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:54.238+0000 7f5097436640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5090105310 0x7f509019aa10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:54.278 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:54.238+0000 7f5097436640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f509010de90 0x7f509019fcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:54.279 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:54.238+0000 7f5097436640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f5090113aa0 con 0x7f50901048e0 2024-09-21T03:44:54.279 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:54.238+0000 7f5097436640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f5090113920 con 0x7f509010de90 2024-09-21T03:44:54.279 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:54.238+0000 7f5097436640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f5090113c20 con 0x7f5090105310 2024-09-21T03:44:54.279 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:54.238+0000 7f50951d2640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f50901048e0 0x7f509019a470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:54.279 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:54.239+0000 7f50951d2640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f50901048e0 0x7f509019a470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:40370/0 (socket says 172.21.15.39:40370) 2024-09-21T03:44:54.279 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:54.239+0000 7f50951d2640 1 -- 172.21.15.39:0/1335550145 learned_addr learned my addr 172.21.15.39:0/1335550145 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:44:54.279 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:54.239+0000 7f50949d1640 1 --2- 172.21.15.39:0/1335550145 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5090105310 0x7f509019aa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:54.280 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:54.239+0000 7f50959d3640 1 --2- 172.21.15.39:0/1335550145 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f509010de90 0x7f509019fcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:54.280 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:54.239+0000 7f50951d2640 1 -- 172.21.15.39:0/1335550145 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5090105310 msgr2=0x7f509019aa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:54.280 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:54.239+0000 7f50951d2640 1 --2- 172.21.15.39:0/1335550145 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5090105310 0x7f509019aa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:54.280 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:54.239+0000 7f50951d2640 1 -- 172.21.15.39:0/1335550145 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f509010de90 msgr2=0x7f509019fcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:54.280 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:54.239+0000 7f50951d2640 1 --2- 172.21.15.39:0/1335550145 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f509010de90 0x7f509019fcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:54.280 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:54.239+0000 7f50951d2640 1 -- 172.21.15.39:0/1335550145 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f50901a0320 con 0x7f50901048e0 2024-09-21T03:44:54.281 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:54.239+0000 7f50959d3640 1 --2- 172.21.15.39:0/1335550145 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f509010de90 0x7f509019fcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:44:54.281 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:54.239+0000 7f50951d2640 1 --2- 172.21.15.39:0/1335550145 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f50901048e0 0x7f509019a470 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f508c00c890 tx=0x7f508c00cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:44:54.281 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:54.239+0000 7f50867fc640 1 -- 172.21.15.39:0/1335550145 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f508c004220 con 0x7f50901048e0 2024-09-21T03:44:54.281 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:54.239+0000 7f50867fc640 1 -- 172.21.15.39:0/1335550145 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f508c0043c0 con 0x7f50901048e0 2024-09-21T03:44:54.281 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:54.239+0000 7f50867fc640 1 -- 172.21.15.39:0/1335550145 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f508c004a00 con 0x7f50901048e0 2024-09-21T03:44:54.281 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:54.239+0000 7f5097436640 1 -- 172.21.15.39:0/1335550145 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f50901a0610 con 0x7f50901048e0 2024-09-21T03:44:54.281 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:54.239+0000 7f5097436640 1 -- 172.21.15.39:0/1335550145 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f50901a6e30 con 0x7f50901048e0 2024-09-21T03:44:54.282 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:54.240+0000 7f5097436640 1 -- 172.21.15.39:0/1335550145 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f50901073e0 con 0x7f50901048e0 2024-09-21T03:44:54.282 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:54.243+0000 7f50867fc640 1 -- 172.21.15.39:0/1335550145 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 22) ==== 154580+0+0 (secure 0 0 0) 0x7f508c0079b0 con 0x7f50901048e0 2024-09-21T03:44:54.282 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:54.243+0000 7f50867fc640 1 --2- 172.21.15.39:0/1335550145 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f507c0b6620 0x7f507c0b8b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:54.282 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:54.243+0000 7f50949d1640 1 --2- 172.21.15.39:0/1335550145 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f507c0b6620 0x7f507c0b8b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:54.282 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:54.244+0000 7f50867fc640 1 -- 172.21.15.39:0/1335550145 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8484+0+0 (secure 0 0 0) 0x7f508c0e77c0 con 0x7f50901048e0 2024-09-21T03:44:54.282 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:54.244+0000 7f50867fc640 1 --2- 172.21.15.39:0/1335550145 >> [v2:172.21.15.161:6816/52966633,v1:172.21.15.161:6817/52966633] conn(0x7f507c0c0490 0x7f507c0c2980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:54.283 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:54.244+0000 7f50867fc640 1 -- 172.21.15.39:0/1335550145 --> [v2:172.21.15.161:6816/52966633,v1:172.21.15.161:6817/52966633] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f507c0c3050 con 0x7f507c0c0490 2024-09-21T03:44:54.283 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:54.244+0000 7f50867fc640 1 -- 172.21.15.39:0/1335550145 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_get_version_reply(handle=1 version=79) ==== 24+0+0 (secure 0 0 0) 0x7f508c0e7c10 con 0x7f50901048e0 2024-09-21T03:44:54.283 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:54.244+0000 7f50949d1640 1 --2- 172.21.15.39:0/1335550145 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f507c0b6620 0x7f507c0b8b30 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f5090106430 tx=0x7f508000c4a0 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:44:54.283 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:54.244+0000 7f50959d3640 1 --2- 172.21.15.39:0/1335550145 >> [v2:172.21.15.161:6816/52966633,v1:172.21.15.161:6817/52966633] conn(0x7f507c0c0490 0x7f507c0c2980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:54.283 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:54.244+0000 7f50959d3640 1 --2- 172.21.15.39:0/1335550145 >> [v2:172.21.15.161:6816/52966633,v1:172.21.15.161:6817/52966633] conn(0x7f507c0c0490 0x7f507c0c2980 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:44:54.283 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:54.245+0000 7f50867fc640 1 -- 172.21.15.39:0/1335550145 <== osd.8 v2:172.21.15.161:6816/52966633 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f507c0c3050 con 0x7f507c0c0490 2024-09-21T03:44:54.283 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:54.259+0000 7f5097436640 1 -- 172.21.15.39:0/1335550145 --> [v2:172.21.15.161:6816/52966633,v1:172.21.15.161:6817/52966633] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f509006be50 con 0x7f507c0c0490 2024-09-21T03:44:54.284 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:54.259+0000 7f50867fc640 1 -- 172.21.15.39:0/1335550145 <== osd.8 v2:172.21.15.161:6816/52966633 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f509006be50 con 0x7f507c0c0490 2024-09-21T03:44:54.284 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:54.260+0000 7f5097436640 1 -- 172.21.15.39:0/1335550145 >> [v2:172.21.15.161:6816/52966633,v1:172.21.15.161:6817/52966633] conn(0x7f507c0c0490 msgr2=0x7f507c0c2980 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:54.284 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:54.260+0000 7f5097436640 1 --2- 172.21.15.39:0/1335550145 >> [v2:172.21.15.161:6816/52966633,v1:172.21.15.161:6817/52966633] conn(0x7f507c0c0490 0x7f507c0c2980 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:54.284 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:54.260+0000 7f5097436640 1 -- 172.21.15.39:0/1335550145 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f507c0b6620 msgr2=0x7f507c0b8b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:54.285 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:54.260+0000 7f5097436640 1 --2- 172.21.15.39:0/1335550145 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f507c0b6620 0x7f507c0b8b30 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f5090106430 tx=0x7f508000c4a0 comp rx=0 tx=0).stop 2024-09-21T03:44:54.285 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:54.260+0000 7f5097436640 1 -- 172.21.15.39:0/1335550145 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f50901048e0 msgr2=0x7f509019a470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:54.285 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:54.260+0000 7f5097436640 1 --2- 172.21.15.39:0/1335550145 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f50901048e0 0x7f509019a470 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f508c00c890 tx=0x7f508c00cd50 comp rx=0 tx=0).stop 2024-09-21T03:44:54.285 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:54.260+0000 7f50959d3640 1 -- 172.21.15.39:0/1335550145 reap_dead start 2024-09-21T03:44:54.285 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:54.260+0000 7f5097436640 1 -- 172.21.15.39:0/1335550145 shutdown_connections 2024-09-21T03:44:54.285 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:54.260+0000 7f5097436640 1 -- 172.21.15.39:0/1335550145 >> 172.21.15.39:0/1335550145 conn(0x7f50900797b0 msgr2=0x7f5090106980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:44:54.286 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:54.260+0000 7f5097436640 1 -- 172.21.15.39:0/1335550145 shutdown_connections 2024-09-21T03:44:54.286 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:54.260+0000 7f5097436640 1 -- 172.21.15.39:0/1335550145 wait complete. 2024-09-21T03:44:54.764 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:44:54 smithi039 ceph-mon[31211]: pgmap v291: 97 pgs: 97 active+clean; 645 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 1.2 KiB/s wr, 9 op/s 2024-09-21T03:44:54.788 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:54.762+0000 7fd7f9f14640 1 -- 172.21.15.39:0/1215586221 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd7f4103b60 msgr2=0x7fd7f4103f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:54.788 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:54.762+0000 7fd7f9f14640 1 --2- 172.21.15.39:0/1215586221 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd7f4103b60 0x7fd7f4103f70 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fd7e4009900 tx=0x7fd7e402a410 comp rx=0 tx=0).stop 2024-09-21T03:44:54.788 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:54.762+0000 7fd7f9f14640 1 -- 172.21.15.39:0/1215586221 shutdown_connections 2024-09-21T03:44:54.788 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:54.762+0000 7fd7f9f14640 1 --2- 172.21.15.39:0/1215586221 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd7f4109df0 0x7fd7f4110620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:54.788 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:54.762+0000 7fd7f9f14640 1 --2- 172.21.15.39:0/1215586221 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd7f41094a0 0x7fd7f41098b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:54.788 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:54.762+0000 7fd7f9f14640 1 --2- 172.21.15.39:0/1215586221 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd7f4103b60 0x7fd7f4103f70 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:54.788 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:54.762+0000 7fd7f9f14640 1 -- 172.21.15.39:0/1215586221 >> 172.21.15.39:0/1215586221 conn(0x7fd7f407b850 msgr2=0x7fd7f407bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:44:54.788 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:54.762+0000 7fd7f9f14640 1 -- 172.21.15.39:0/1215586221 shutdown_connections 2024-09-21T03:44:54.789 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:54.762+0000 7fd7f9f14640 1 -- 172.21.15.39:0/1215586221 wait complete. 2024-09-21T03:44:54.789 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:54.763+0000 7fd7f9f14640 1 Processor -- start 2024-09-21T03:44:54.789 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:54.763+0000 7fd7f9f14640 1 -- start start 2024-09-21T03:44:54.789 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:54.764+0000 7fd7f9f14640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd7f4103b60 0x7fd7f419c680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:54.789 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:54.764+0000 7fd7f9f14640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd7f41094a0 0x7fd7f419cc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:54.789 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:54.764+0000 7fd7f9f14640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd7f4109df0 0x7fd7f41a1ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:54.789 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:54.764+0000 7fd7f9f14640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fd7f4113ab0 con 0x7fd7f41094a0 2024-09-21T03:44:54.789 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:54.764+0000 7fd7f9f14640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fd7f4113930 con 0x7fd7f4103b60 2024-09-21T03:44:54.789 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:54.764+0000 7fd7f9f14640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fd7f4113c30 con 0x7fd7f4109df0 2024-09-21T03:44:54.789 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:54.764+0000 7fd7f2ffd640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd7f41094a0 0x7fd7f419cc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:54.789 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:54.764+0000 7fd7f2ffd640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd7f41094a0 0x7fd7f419cc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:40378/0 (socket says 172.21.15.39:40378) 2024-09-21T03:44:54.789 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:54.764+0000 7fd7f2ffd640 1 -- 172.21.15.39:0/3721447715 learned_addr learned my addr 172.21.15.39:0/3721447715 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:44:54.789 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:54.764+0000 7fd7f37fe640 1 --2- 172.21.15.39:0/3721447715 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd7f4103b60 0x7fd7f419c680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:54.790 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:54.764+0000 7fd7f3fff640 1 --2- 172.21.15.39:0/3721447715 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd7f4109df0 0x7fd7f41a1ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:54.790 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:54.764+0000 7fd7f2ffd640 1 -- 172.21.15.39:0/3721447715 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd7f4109df0 msgr2=0x7fd7f41a1ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:54.790 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:54.764+0000 7fd7f2ffd640 1 --2- 172.21.15.39:0/3721447715 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd7f4109df0 0x7fd7f41a1ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:54.790 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:54.764+0000 7fd7f2ffd640 1 -- 172.21.15.39:0/3721447715 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd7f4103b60 msgr2=0x7fd7f419c680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:54.790 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:54.764+0000 7fd7f2ffd640 1 --2- 172.21.15.39:0/3721447715 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd7f4103b60 0x7fd7f419c680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:54.790 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:54.764+0000 7fd7f2ffd640 1 -- 172.21.15.39:0/3721447715 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd7f41a25f0 con 0x7fd7f41094a0 2024-09-21T03:44:54.790 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:54.764+0000 7fd7f37fe640 1 --2- 172.21.15.39:0/3721447715 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd7f4103b60 0x7fd7f419c680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:44:54.790 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:54.764+0000 7fd7f3fff640 1 --2- 172.21.15.39:0/3721447715 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd7f4109df0 0x7fd7f41a1ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:44:54.790 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:54.764+0000 7fd7f2ffd640 1 --2- 172.21.15.39:0/3721447715 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd7f41094a0 0x7fd7f419cc20 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fd7e800d950 tx=0x7fd7e800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:44:54.790 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:54.765+0000 7fd7f0ff9640 1 -- 172.21.15.39:0/3721447715 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7e8017070 con 0x7fd7f41094a0 2024-09-21T03:44:54.790 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:54.765+0000 7fd7f0ff9640 1 -- 172.21.15.39:0/3721447715 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd7e8009c30 con 0x7fd7f41094a0 2024-09-21T03:44:54.790 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:54.765+0000 7fd7f9f14640 1 -- 172.21.15.39:0/3721447715 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd7f41a28e0 con 0x7fd7f41094a0 2024-09-21T03:44:54.790 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:54.765+0000 7fd7f0ff9640 1 -- 172.21.15.39:0/3721447715 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7e80047a0 con 0x7fd7f41094a0 2024-09-21T03:44:54.791 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:54.765+0000 7fd7f9f14640 1 -- 172.21.15.39:0/3721447715 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd7f41a9040 con 0x7fd7f41094a0 2024-09-21T03:44:54.791 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:54.765+0000 7fd7f9f14640 1 -- 172.21.15.39:0/3721447715 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fd7f4107d10 con 0x7fd7f41094a0 2024-09-21T03:44:54.791 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:54.767+0000 7fd7f0ff9640 1 -- 172.21.15.39:0/3721447715 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 22) ==== 154580+0+0 (secure 0 0 0) 0x7fd7e8004020 con 0x7fd7f41094a0 2024-09-21T03:44:54.791 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:54.768+0000 7fd7f0ff9640 1 --2- 172.21.15.39:0/3721447715 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7fd7dc0b65f0 0x7fd7dc0b8b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:54.791 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:54.768+0000 7fd7f37fe640 1 --2- 172.21.15.39:0/3721447715 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7fd7dc0b65f0 0x7fd7dc0b8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:54.791 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:54.768+0000 7fd7f0ff9640 1 -- 172.21.15.39:0/3721447715 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8484+0+0 (secure 0 0 0) 0x7fd7e80e79d0 con 0x7fd7f41094a0 2024-09-21T03:44:54.791 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:54.768+0000 7fd7f0ff9640 1 --2- 172.21.15.39:0/3721447715 >> [v2:172.21.15.39:6826/2001303253,v1:172.21.15.39:6827/2001303253] conn(0x7fd7dc0c0460 0x7fd7dc0c2950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:54.791 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:54.768+0000 7fd7f0ff9640 1 -- 172.21.15.39:0/3721447715 --> [v2:172.21.15.39:6826/2001303253,v1:172.21.15.39:6827/2001303253] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fd7dc0c3020 con 0x7fd7dc0c0460 2024-09-21T03:44:54.791 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:54.768+0000 7fd7f0ff9640 1 -- 172.21.15.39:0/3721447715 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_get_version_reply(handle=1 version=79) ==== 24+0+0 (secure 0 0 0) 0x7fd7e80e7e20 con 0x7fd7f41094a0 2024-09-21T03:44:54.791 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:54.769+0000 7fd7f3fff640 1 --2- 172.21.15.39:0/3721447715 >> [v2:172.21.15.39:6826/2001303253,v1:172.21.15.39:6827/2001303253] conn(0x7fd7dc0c0460 0x7fd7dc0c2950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:54.791 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:54.769+0000 7fd7f37fe640 1 --2- 172.21.15.39:0/3721447715 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7fd7dc0b65f0 0x7fd7dc0b8b00 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fd7e4031440 tx=0x7fd7e4009680 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:44:54.791 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:54.769+0000 7fd7f3fff640 1 --2- 172.21.15.39:0/3721447715 >> [v2:172.21.15.39:6826/2001303253,v1:172.21.15.39:6827/2001303253] conn(0x7fd7dc0c0460 0x7fd7dc0c2950 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.9 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:44:54.791 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:54.770+0000 7fd7f0ff9640 1 -- 172.21.15.39:0/3721447715 <== osd.9 v2:172.21.15.39:6826/2001303253 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fd7dc0c3020 con 0x7fd7dc0c0460 2024-09-21T03:44:54.791 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:54.783+0000 7fd7f9f14640 1 -- 172.21.15.39:0/3721447715 --> [v2:172.21.15.39:6826/2001303253,v1:172.21.15.39:6827/2001303253] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fd7f4104de0 con 0x7fd7dc0c0460 2024-09-21T03:44:54.792 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:54.784+0000 7fd7f0ff9640 1 -- 172.21.15.39:0/3721447715 <== osd.9 v2:172.21.15.39:6826/2001303253 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fd7f4104de0 con 0x7fd7dc0c0460 2024-09-21T03:44:54.792 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:54.784+0000 7fd7f9f14640 1 -- 172.21.15.39:0/3721447715 >> [v2:172.21.15.39:6826/2001303253,v1:172.21.15.39:6827/2001303253] conn(0x7fd7dc0c0460 msgr2=0x7fd7dc0c2950 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:54.792 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:54.784+0000 7fd7f9f14640 1 --2- 172.21.15.39:0/3721447715 >> [v2:172.21.15.39:6826/2001303253,v1:172.21.15.39:6827/2001303253] conn(0x7fd7dc0c0460 0x7fd7dc0c2950 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:54.792 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:54.784+0000 7fd7f9f14640 1 -- 172.21.15.39:0/3721447715 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7fd7dc0b65f0 msgr2=0x7fd7dc0b8b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:54.792 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:54.784+0000 7fd7f9f14640 1 --2- 172.21.15.39:0/3721447715 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7fd7dc0b65f0 0x7fd7dc0b8b00 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fd7e4031440 tx=0x7fd7e4009680 comp rx=0 tx=0).stop 2024-09-21T03:44:54.792 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:54.784+0000 7fd7f9f14640 1 -- 172.21.15.39:0/3721447715 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd7f41094a0 msgr2=0x7fd7f419cc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:54.792 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:54.784+0000 7fd7f9f14640 1 --2- 172.21.15.39:0/3721447715 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd7f41094a0 0x7fd7f419cc20 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fd7e800d950 tx=0x7fd7e800de10 comp rx=0 tx=0).stop 2024-09-21T03:44:54.792 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:54.784+0000 7fd7f3fff640 1 -- 172.21.15.39:0/3721447715 reap_dead start 2024-09-21T03:44:54.793 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:54.784+0000 7fd7f9f14640 1 -- 172.21.15.39:0/3721447715 shutdown_connections 2024-09-21T03:44:54.793 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:54.784+0000 7fd7f9f14640 1 -- 172.21.15.39:0/3721447715 >> 172.21.15.39:0/3721447715 conn(0x7fd7f407b850 msgr2=0x7fd7f41072b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:44:54.793 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:54.784+0000 7fd7f9f14640 1 -- 172.21.15.39:0/3721447715 shutdown_connections 2024-09-21T03:44:54.793 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:54.784+0000 7fd7f9f14640 1 -- 172.21.15.39:0/3721447715 wait complete. 2024-09-21T03:44:54.968 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:44:54 smithi161 ceph-mon[34533]: pgmap v291: 97 pgs: 97 active+clean; 645 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 1.2 KiB/s wr, 9 op/s 2024-09-21T03:44:54.975 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:44:54 smithi150 ceph-mon[34289]: pgmap v291: 97 pgs: 97 active+clean; 645 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 1.2 KiB/s wr, 9 op/s 2024-09-21T03:44:55.872 INFO:teuthology.orchestra.run.smithi039.stdout:231928234019 2024-09-21T03:44:55.873 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 51116d5c-77ca-11ef-baf5-efdc52797490 -- ceph osd last-stat-seq osd.8 2024-09-21T03:44:56.473 INFO:teuthology.orchestra.run.smithi039.stdout:38654705742 2024-09-21T03:44:56.473 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 51116d5c-77ca-11ef-baf5-efdc52797490 -- ceph osd last-stat-seq osd.0 2024-09-21T03:44:56.705 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:44:56 smithi039 ceph-mon[31211]: pgmap v292: 97 pgs: 97 active+clean; 647 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 1.4 KiB/s wr, 13 op/s 2024-09-21T03:44:56.970 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:44:56 smithi161 ceph-mon[34533]: pgmap v292: 97 pgs: 97 active+clean; 647 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 1.4 KiB/s wr, 13 op/s 2024-09-21T03:44:56.974 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:44:56 smithi150 ceph-mon[34289]: pgmap v292: 97 pgs: 97 active+clean; 647 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 1.4 KiB/s wr, 13 op/s 2024-09-21T03:44:57.361 INFO:teuthology.orchestra.run.smithi039.stdout:184683593775 2024-09-21T03:44:57.362 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 51116d5c-77ca-11ef-baf5-efdc52797490 -- ceph osd last-stat-seq osd.6 2024-09-21T03:44:57.418 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:56.797+0000 7f0f66b88640 1 -- 172.21.15.39:0/3316747534 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0f60108d20 msgr2=0x7f0f6010f550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:57.418 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:56.797+0000 7f0f66b88640 1 --2- 172.21.15.39:0/3316747534 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0f60108d20 0x7f0f6010f550 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f0f58009ca0 tx=0x7f0f5802a8c0 comp rx=0 tx=0).stop 2024-09-21T03:44:57.418 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:56.797+0000 7f0f66b88640 1 -- 172.21.15.39:0/3316747534 shutdown_connections 2024-09-21T03:44:57.418 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:56.797+0000 7f0f66b88640 1 --2- 172.21.15.39:0/3316747534 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0f60108d20 0x7f0f6010f550 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:57.418 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:56.797+0000 7f0f66b88640 1 --2- 172.21.15.39:0/3316747534 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0f601083d0 0x7f0f601087e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:57.418 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:56.797+0000 7f0f66b88640 1 --2- 172.21.15.39:0/3316747534 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0f60102a90 0x7f0f60102ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:57.418 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:56.797+0000 7f0f66b88640 1 -- 172.21.15.39:0/3316747534 >> 172.21.15.39:0/3316747534 conn(0x7f0f60079750 msgr2=0x7f0f60079b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:44:57.419 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:56.797+0000 7f0f66b88640 1 -- 172.21.15.39:0/3316747534 shutdown_connections 2024-09-21T03:44:57.419 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:56.797+0000 7f0f66b88640 1 -- 172.21.15.39:0/3316747534 wait complete. 2024-09-21T03:44:57.419 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:56.797+0000 7f0f66b88640 1 Processor -- start 2024-09-21T03:44:57.419 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:56.798+0000 7f0f66b88640 1 -- start start 2024-09-21T03:44:57.419 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:56.798+0000 7f0f66b88640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0f60102a90 0x7f0f6019a3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:57.419 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:56.798+0000 7f0f66b88640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0f601083d0 0x7f0f6019a990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:57.419 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:56.798+0000 7f0f66b88640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0f60108d20 0x7f0f6019fc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:57.419 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:56.798+0000 7f0f66b88640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f0f60112990 con 0x7f0f601083d0 2024-09-21T03:44:57.419 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:56.798+0000 7f0f66b88640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f0f60112810 con 0x7f0f60108d20 2024-09-21T03:44:57.419 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:56.798+0000 7f0f66b88640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f0f60112b10 con 0x7f0f60102a90 2024-09-21T03:44:57.419 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:56.798+0000 7f0f5ffff640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0f601083d0 0x7f0f6019a990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:57.420 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:56.798+0000 7f0f5ffff640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0f601083d0 0x7f0f6019a990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:40406/0 (socket says 172.21.15.39:40406) 2024-09-21T03:44:57.420 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:56.798+0000 7f0f5ffff640 1 -- 172.21.15.39:0/833347 learned_addr learned my addr 172.21.15.39:0/833347 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:44:57.420 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:56.798+0000 7f0f64924640 1 --2- 172.21.15.39:0/833347 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0f60102a90 0x7f0f6019a3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:57.420 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:56.798+0000 7f0f65125640 1 --2- 172.21.15.39:0/833347 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0f60108d20 0x7f0f6019fc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:57.420 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:56.798+0000 7f0f5ffff640 1 -- 172.21.15.39:0/833347 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0f60102a90 msgr2=0x7f0f6019a3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:57.420 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:56.798+0000 7f0f5ffff640 1 --2- 172.21.15.39:0/833347 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0f60102a90 0x7f0f6019a3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:57.420 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:56.798+0000 7f0f5ffff640 1 -- 172.21.15.39:0/833347 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0f60108d20 msgr2=0x7f0f6019fc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:57.420 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:56.798+0000 7f0f5ffff640 1 --2- 172.21.15.39:0/833347 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0f60108d20 0x7f0f6019fc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:57.420 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:56.798+0000 7f0f5ffff640 1 -- 172.21.15.39:0/833347 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0f601a0360 con 0x7f0f601083d0 2024-09-21T03:44:57.420 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:56.798+0000 7f0f65125640 1 --2- 172.21.15.39:0/833347 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0f60108d20 0x7f0f6019fc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:44:57.420 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:56.798+0000 7f0f5ffff640 1 --2- 172.21.15.39:0/833347 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0f601083d0 0x7f0f6019a990 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f0f5400d770 tx=0x7f0f5400dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:44:57.420 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:56.799+0000 7f0f5dffb640 1 -- 172.21.15.39:0/833347 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f54017070 con 0x7f0f601083d0 2024-09-21T03:44:57.420 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:56.799+0000 7f0f66b88640 1 -- 172.21.15.39:0/833347 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0f601a0650 con 0x7f0f601083d0 2024-09-21T03:44:57.420 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:56.799+0000 7f0f5dffb640 1 -- 172.21.15.39:0/833347 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0f54009c30 con 0x7f0f601083d0 2024-09-21T03:44:57.421 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:56.799+0000 7f0f5dffb640 1 -- 172.21.15.39:0/833347 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f5400f620 con 0x7f0f601083d0 2024-09-21T03:44:57.421 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:56.799+0000 7f0f66b88640 1 -- 172.21.15.39:0/833347 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0f601a6db0 con 0x7f0f601083d0 2024-09-21T03:44:57.421 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:56.800+0000 7f0f66b88640 1 -- 172.21.15.39:0/833347 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f0f60063b50 con 0x7f0f601083d0 2024-09-21T03:44:57.421 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:56.801+0000 7f0f5dffb640 1 -- 172.21.15.39:0/833347 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 22) ==== 154580+0+0 (secure 0 0 0) 0x7f0f5401a070 con 0x7f0f601083d0 2024-09-21T03:44:57.421 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:56.802+0000 7f0f5dffb640 1 --2- 172.21.15.39:0/833347 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f0f480b65f0 0x7f0f480b8b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:57.421 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:56.802+0000 7f0f64924640 1 --2- 172.21.15.39:0/833347 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f0f480b65f0 0x7f0f480b8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:57.421 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:56.802+0000 7f0f5dffb640 1 -- 172.21.15.39:0/833347 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8484+0+0 (secure 0 0 0) 0x7f0f540e7560 con 0x7f0f601083d0 2024-09-21T03:44:57.421 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:56.802+0000 7f0f5dffb640 1 --2- 172.21.15.39:0/833347 >> [v2:172.21.15.161:6824/2178709707,v1:172.21.15.161:6825/2178709707] conn(0x7f0f480c0460 0x7f0f480c2950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:57.421 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:56.802+0000 7f0f5dffb640 1 -- 172.21.15.39:0/833347 --> [v2:172.21.15.161:6824/2178709707,v1:172.21.15.161:6825/2178709707] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f0f480c3020 con 0x7f0f480c0460 2024-09-21T03:44:57.421 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:56.802+0000 7f0f5dffb640 1 -- 172.21.15.39:0/833347 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_get_version_reply(handle=1 version=79) ==== 24+0+0 (secure 0 0 0) 0x7f0f540e79b0 con 0x7f0f601083d0 2024-09-21T03:44:57.421 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:56.803+0000 7f0f64924640 1 --2- 172.21.15.39:0/833347 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f0f480b65f0 0x7f0f480b8b00 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f0f50005fd0 tx=0x7f0f5000b390 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:44:57.421 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:56.803+0000 7f0f65125640 1 --2- 172.21.15.39:0/833347 >> [v2:172.21.15.161:6824/2178709707,v1:172.21.15.161:6825/2178709707] conn(0x7f0f480c0460 0x7f0f480c2950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:57.422 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:56.803+0000 7f0f65125640 1 --2- 172.21.15.39:0/833347 >> [v2:172.21.15.161:6824/2178709707,v1:172.21.15.161:6825/2178709707] conn(0x7f0f480c0460 0x7f0f480c2950 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.11 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:44:57.422 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:56.804+0000 7f0f5dffb640 1 -- 172.21.15.39:0/833347 <== osd.11 v2:172.21.15.161:6824/2178709707 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f0f480c3020 con 0x7f0f480c0460 2024-09-21T03:44:57.422 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:56.818+0000 7f0f66b88640 1 -- 172.21.15.39:0/833347 --> [v2:172.21.15.161:6824/2178709707,v1:172.21.15.161:6825/2178709707] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f0f60063f50 con 0x7f0f480c0460 2024-09-21T03:44:57.422 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:56.818+0000 7f0f5dffb640 1 -- 172.21.15.39:0/833347 <== osd.11 v2:172.21.15.161:6824/2178709707 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f0f60063f50 con 0x7f0f480c0460 2024-09-21T03:44:57.422 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:56.819+0000 7f0f66b88640 1 -- 172.21.15.39:0/833347 >> [v2:172.21.15.161:6824/2178709707,v1:172.21.15.161:6825/2178709707] conn(0x7f0f480c0460 msgr2=0x7f0f480c2950 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:57.422 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:56.819+0000 7f0f66b88640 1 --2- 172.21.15.39:0/833347 >> [v2:172.21.15.161:6824/2178709707,v1:172.21.15.161:6825/2178709707] conn(0x7f0f480c0460 0x7f0f480c2950 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:57.423 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:56.819+0000 7f0f66b88640 1 -- 172.21.15.39:0/833347 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f0f480b65f0 msgr2=0x7f0f480b8b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:57.423 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:56.819+0000 7f0f66b88640 1 --2- 172.21.15.39:0/833347 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f0f480b65f0 0x7f0f480b8b00 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f0f50005fd0 tx=0x7f0f5000b390 comp rx=0 tx=0).stop 2024-09-21T03:44:57.423 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:56.819+0000 7f0f66b88640 1 -- 172.21.15.39:0/833347 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0f601083d0 msgr2=0x7f0f6019a990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:57.423 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:56.819+0000 7f0f66b88640 1 --2- 172.21.15.39:0/833347 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0f601083d0 0x7f0f6019a990 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f0f5400d770 tx=0x7f0f5400dc30 comp rx=0 tx=0).stop 2024-09-21T03:44:57.423 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:56.819+0000 7f0f65125640 1 -- 172.21.15.39:0/833347 reap_dead start 2024-09-21T03:44:57.423 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:56.819+0000 7f0f66b88640 1 -- 172.21.15.39:0/833347 shutdown_connections 2024-09-21T03:44:57.423 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:56.819+0000 7f0f66b88640 1 -- 172.21.15.39:0/833347 >> 172.21.15.39:0/833347 conn(0x7f0f60079750 msgr2=0x7f0f60101a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:44:57.423 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:56.819+0000 7f0f66b88640 1 -- 172.21.15.39:0/833347 shutdown_connections 2024-09-21T03:44:57.423 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:56.819+0000 7f0f66b88640 1 -- 172.21.15.39:0/833347 wait complete. 2024-09-21T03:44:58.372 INFO:teuthology.orchestra.run.smithi039.stdout:304942678038 2024-09-21T03:44:58.373 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 51116d5c-77ca-11ef-baf5-efdc52797490 -- ceph osd last-stat-seq osd.11 2024-09-21T03:44:58.967 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:44:58 smithi161 ceph-mon[34533]: pgmap v293: 97 pgs: 97 active+clean; 647 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 597 B/s wr, 9 op/s 2024-09-21T03:44:58.974 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:44:58 smithi150 ceph-mon[34289]: pgmap v293: 97 pgs: 97 active+clean; 647 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 597 B/s wr, 9 op/s 2024-09-21T03:44:59.083 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:44:58 smithi039 ceph-mon[31211]: pgmap v293: 97 pgs: 97 active+clean; 647 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 597 B/s wr, 9 op/s 2024-09-21T03:44:59.246 INFO:teuthology.orchestra.run.smithi039.stdout:257698037790 2024-09-21T03:44:59.246 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 51116d5c-77ca-11ef-baf5-efdc52797490 -- ceph osd last-stat-seq osd.9 2024-09-21T03:44:59.559 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:59.535+0000 7f017a21f640 1 -- 172.21.15.39:0/1847187841 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f01741083a0 msgr2=0x7f01741087b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:59.560 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:59.535+0000 7f017a21f640 1 --2- 172.21.15.39:0/1847187841 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f01741083a0 0x7f01741087b0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f0168009900 tx=0x7f016802a410 comp rx=0 tx=0).stop 2024-09-21T03:44:59.560 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:59.535+0000 7f017a21f640 1 -- 172.21.15.39:0/1847187841 shutdown_connections 2024-09-21T03:44:59.560 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:59.535+0000 7f017a21f640 1 --2- 172.21.15.39:0/1847187841 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0174108cf0 0x7f017410f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:59.560 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:59.535+0000 7f017a21f640 1 --2- 172.21.15.39:0/1847187841 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f01741083a0 0x7f01741087b0 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:59.560 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:59.535+0000 7f017a21f640 1 --2- 172.21.15.39:0/1847187841 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0174102a60 0x7f0174102e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:59.560 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:59.535+0000 7f017a21f640 1 -- 172.21.15.39:0/1847187841 >> 172.21.15.39:0/1847187841 conn(0x7f0174079730 msgr2=0x7f0174079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:44:59.560 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:59.535+0000 7f017a21f640 1 -- 172.21.15.39:0/1847187841 shutdown_connections 2024-09-21T03:44:59.560 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:59.535+0000 7f017a21f640 1 -- 172.21.15.39:0/1847187841 wait complete. 2024-09-21T03:44:59.560 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:59.536+0000 7f017a21f640 1 Processor -- start 2024-09-21T03:44:59.560 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:59.536+0000 7f017a21f640 1 -- start start 2024-09-21T03:44:59.561 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:59.536+0000 7f017a21f640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0174102a60 0x7f017419a550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:59.561 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:59.536+0000 7f017a21f640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f01741083a0 0x7f017419aaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:59.561 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:59.536+0000 7f017a21f640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0174108cf0 0x7f017419fd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:59.561 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:59.536+0000 7f017a21f640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f0174113aa0 con 0x7f0174108cf0 2024-09-21T03:44:59.561 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:59.536+0000 7f017a21f640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f0174113920 con 0x7f01741083a0 2024-09-21T03:44:59.561 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:59.536+0000 7f017a21f640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f0174113c20 con 0x7f0174102a60 2024-09-21T03:44:59.561 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:59.536+0000 7f0173fff640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0174108cf0 0x7f017419fd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:59.561 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:59.537+0000 7f0173fff640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0174108cf0 0x7f017419fd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:39936/0 (socket says 172.21.15.39:39936) 2024-09-21T03:44:59.561 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:59.537+0000 7f0173fff640 1 -- 172.21.15.39:0/4242410217 learned_addr learned my addr 172.21.15.39:0/4242410217 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:44:59.561 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:59.537+0000 7f01737fe640 1 --2- 172.21.15.39:0/4242410217 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0174102a60 0x7f017419a550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:59.561 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:59.537+0000 7f0172ffd640 1 --2- 172.21.15.39:0/4242410217 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f01741083a0 0x7f017419aaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:59.562 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:59.537+0000 7f0173fff640 1 -- 172.21.15.39:0/4242410217 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0174102a60 msgr2=0x7f017419a550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:59.562 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:59.537+0000 7f0173fff640 1 --2- 172.21.15.39:0/4242410217 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0174102a60 0x7f017419a550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:59.562 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:59.537+0000 7f0173fff640 1 -- 172.21.15.39:0/4242410217 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f01741083a0 msgr2=0x7f017419aaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:59.562 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:59.537+0000 7f0173fff640 1 --2- 172.21.15.39:0/4242410217 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f01741083a0 0x7f017419aaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:59.562 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:59.537+0000 7f0173fff640 1 -- 172.21.15.39:0/4242410217 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f01741a04c0 con 0x7f0174108cf0 2024-09-21T03:44:59.562 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:59.537+0000 7f0173fff640 1 --2- 172.21.15.39:0/4242410217 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0174108cf0 0x7f017419fd90 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f016c009fd0 tx=0x7f016c00b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:44:59.562 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:59.537+0000 7f0170ff9640 1 -- 172.21.15.39:0/4242410217 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f016c011830 con 0x7f0174108cf0 2024-09-21T03:44:59.562 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:59.537+0000 7f0170ff9640 1 -- 172.21.15.39:0/4242410217 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f016c016070 con 0x7f0174108cf0 2024-09-21T03:44:59.562 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:59.537+0000 7f0170ff9640 1 -- 172.21.15.39:0/4242410217 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f016c010de0 con 0x7f0174108cf0 2024-09-21T03:44:59.562 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:59.537+0000 7f017a21f640 1 -- 172.21.15.39:0/4242410217 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f01741a07b0 con 0x7f0174108cf0 2024-09-21T03:44:59.562 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:59.537+0000 7f017a21f640 1 -- 172.21.15.39:0/4242410217 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f01741a6f10 con 0x7f0174108cf0 2024-09-21T03:44:59.562 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:59.538+0000 7f017a21f640 1 -- 172.21.15.39:0/4242410217 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f0174063b10 con 0x7f0174108cf0 2024-09-21T03:44:59.563 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:59.540+0000 7f0170ff9640 1 -- 172.21.15.39:0/4242410217 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 22) ==== 154580+0+0 (secure 0 0 0) 0x7f016c0119d0 con 0x7f0174108cf0 2024-09-21T03:44:59.563 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:59.541+0000 7f0170ff9640 1 --2- 172.21.15.39:0/4242410217 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f015c0b6670 0x7f015c0b8b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:59.563 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:59.541+0000 7f01737fe640 1 --2- 172.21.15.39:0/4242410217 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f015c0b6670 0x7f015c0b8b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:59.563 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:59.541+0000 7f0170ff9640 1 -- 172.21.15.39:0/4242410217 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8484+0+0 (secure 0 0 0) 0x7f016c0e75d0 con 0x7f0174108cf0 2024-09-21T03:44:59.563 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:59.541+0000 7f0170ff9640 1 --2- 172.21.15.39:0/4242410217 >> [v2:172.21.15.150:6808/3097721553,v1:172.21.15.150:6809/3097721553] conn(0x7f015c0c04e0 0x7f015c0c29d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:59.563 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:59.541+0000 7f0170ff9640 1 -- 172.21.15.39:0/4242410217 --> [v2:172.21.15.150:6808/3097721553,v1:172.21.15.150:6809/3097721553] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f016c01ee50 con 0x7f015c0c04e0 2024-09-21T03:44:59.563 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:59.541+0000 7f0170ff9640 1 -- 172.21.15.39:0/4242410217 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_get_version_reply(handle=1 version=79) ==== 24+0+0 (secure 0 0 0) 0x7f016c0e7a20 con 0x7f0174108cf0 2024-09-21T03:44:59.563 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:59.541+0000 7f01737fe640 1 --2- 172.21.15.39:0/4242410217 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f015c0b6670 0x7f015c0b8b80 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f0164005fd0 tx=0x7f0164009680 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:44:59.563 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:59.542+0000 7f0172ffd640 1 --2- 172.21.15.39:0/4242410217 >> [v2:172.21.15.150:6808/3097721553,v1:172.21.15.150:6809/3097721553] conn(0x7f015c0c04e0 0x7f015c0c29d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:59.563 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:59.542+0000 7f0172ffd640 1 --2- 172.21.15.39:0/4242410217 >> [v2:172.21.15.150:6808/3097721553,v1:172.21.15.150:6809/3097721553] conn(0x7f015c0c04e0 0x7f015c0c29d0 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:44:59.563 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:59.543+0000 7f0170ff9640 1 -- 172.21.15.39:0/4242410217 <== osd.4 v2:172.21.15.150:6808/3097721553 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f016c01ee50 con 0x7f015c0c04e0 2024-09-21T03:44:59.564 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:59.557+0000 7f017a21f640 1 -- 172.21.15.39:0/4242410217 --> [v2:172.21.15.150:6808/3097721553,v1:172.21.15.150:6809/3097721553] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f0174063f40 con 0x7f015c0c04e0 2024-09-21T03:44:59.564 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:59.557+0000 7f0170ff9640 1 -- 172.21.15.39:0/4242410217 <== osd.4 v2:172.21.15.150:6808/3097721553 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f0174063f40 con 0x7f015c0c04e0 2024-09-21T03:44:59.564 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:59.557+0000 7f017a21f640 1 -- 172.21.15.39:0/4242410217 >> [v2:172.21.15.150:6808/3097721553,v1:172.21.15.150:6809/3097721553] conn(0x7f015c0c04e0 msgr2=0x7f015c0c29d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:59.565 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:59.557+0000 7f017a21f640 1 --2- 172.21.15.39:0/4242410217 >> [v2:172.21.15.150:6808/3097721553,v1:172.21.15.150:6809/3097721553] conn(0x7f015c0c04e0 0x7f015c0c29d0 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:59.565 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:59.558+0000 7f017a21f640 1 -- 172.21.15.39:0/4242410217 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f015c0b6670 msgr2=0x7f015c0b8b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:59.565 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:59.558+0000 7f017a21f640 1 --2- 172.21.15.39:0/4242410217 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f015c0b6670 0x7f015c0b8b80 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f0164005fd0 tx=0x7f0164009680 comp rx=0 tx=0).stop 2024-09-21T03:44:59.565 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:59.558+0000 7f017a21f640 1 -- 172.21.15.39:0/4242410217 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0174108cf0 msgr2=0x7f017419fd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:59.565 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:59.558+0000 7f017a21f640 1 --2- 172.21.15.39:0/4242410217 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0174108cf0 0x7f017419fd90 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f016c009fd0 tx=0x7f016c00b8e0 comp rx=0 tx=0).stop 2024-09-21T03:44:59.565 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:59.558+0000 7f0173fff640 1 -- 172.21.15.39:0/4242410217 reap_dead start 2024-09-21T03:44:59.565 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:59.558+0000 7f017a21f640 1 -- 172.21.15.39:0/4242410217 shutdown_connections 2024-09-21T03:44:59.565 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:59.558+0000 7f017a21f640 1 -- 172.21.15.39:0/4242410217 >> 172.21.15.39:0/4242410217 conn(0x7f0174079730 msgr2=0x7f0174105360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:44:59.565 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:59.558+0000 7f017a21f640 1 -- 172.21.15.39:0/4242410217 shutdown_connections 2024-09-21T03:44:59.565 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:59.558+0000 7f017a21f640 1 -- 172.21.15.39:0/4242410217 wait complete. 2024-09-21T03:44:59.789 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/51116d5c-77ca-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:44:59.964 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:59.962+0000 7f89490d7640 1 -- 172.21.15.39:0/3009290845 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f89441083a0 msgr2=0x7f89441087b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:59.964 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:59.962+0000 7f89490d7640 1 --2- 172.21.15.39:0/3009290845 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f89441083a0 0x7f89441087b0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f892c009900 tx=0x7f892c02a410 comp rx=0 tx=0).stop 2024-09-21T03:44:59.964 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:59.962+0000 7f89490d7640 1 -- 172.21.15.39:0/3009290845 shutdown_connections 2024-09-21T03:44:59.964 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:59.962+0000 7f89490d7640 1 --2- 172.21.15.39:0/3009290845 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8944108cf0 0x7f894410f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:59.964 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:59.962+0000 7f89490d7640 1 --2- 172.21.15.39:0/3009290845 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f89441083a0 0x7f89441087b0 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:59.964 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:59.962+0000 7f89490d7640 1 --2- 172.21.15.39:0/3009290845 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8944102a60 0x7f8944102e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:59.964 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:59.962+0000 7f89490d7640 1 -- 172.21.15.39:0/3009290845 >> 172.21.15.39:0/3009290845 conn(0x7f8944079730 msgr2=0x7f8944079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:44:59.964 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:59.962+0000 7f89490d7640 1 -- 172.21.15.39:0/3009290845 shutdown_connections 2024-09-21T03:44:59.964 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:59.962+0000 7f89490d7640 1 -- 172.21.15.39:0/3009290845 wait complete. 2024-09-21T03:44:59.966 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:59.963+0000 7f89490d7640 1 Processor -- start 2024-09-21T03:44:59.966 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:59.963+0000 7f89490d7640 1 -- start start 2024-09-21T03:44:59.966 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:59.963+0000 7f89490d7640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8944102a60 0x7f894419a510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:59.966 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:59.963+0000 7f89490d7640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f89441083a0 0x7f894419aab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:59.966 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:59.963+0000 7f89490d7640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8944108cf0 0x7f894419fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:59.966 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:59.963+0000 7f89490d7640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f89441129b0 con 0x7f89441083a0 2024-09-21T03:44:59.966 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:59.963+0000 7f89490d7640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f8944112830 con 0x7f8944102a60 2024-09-21T03:44:59.966 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:59.963+0000 7f89490d7640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f8944112b30 con 0x7f8944108cf0 2024-09-21T03:44:59.966 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:59.963+0000 7f893a59c640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f89441083a0 0x7f894419aab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:59.966 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:59.963+0000 7f893a59c640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f89441083a0 0x7f894419aab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:39950/0 (socket says 172.21.15.39:39950) 2024-09-21T03:44:59.967 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:59.963+0000 7f893a59c640 1 -- 172.21.15.39:0/147905292 learned_addr learned my addr 172.21.15.39:0/147905292 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:44:59.967 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:59.963+0000 7f8942d9d640 1 --2- 172.21.15.39:0/147905292 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8944102a60 0x7f894419a510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:59.967 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:59.963+0000 7f894359e640 1 --2- 172.21.15.39:0/147905292 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8944108cf0 0x7f894419fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:59.967 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:59.964+0000 7f893a59c640 1 -- 172.21.15.39:0/147905292 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8944108cf0 msgr2=0x7f894419fd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:59.967 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:59.964+0000 7f893a59c640 1 --2- 172.21.15.39:0/147905292 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8944108cf0 0x7f894419fd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:59.967 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:59.964+0000 7f893a59c640 1 -- 172.21.15.39:0/147905292 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8944102a60 msgr2=0x7f894419a510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:59.967 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:59.964+0000 7f893a59c640 1 --2- 172.21.15.39:0/147905292 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8944102a60 0x7f894419a510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:59.967 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:59.964+0000 7f893a59c640 1 -- 172.21.15.39:0/147905292 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f89441a03c0 con 0x7f89441083a0 2024-09-21T03:44:59.967 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:59.964+0000 7f893a59c640 1 --2- 172.21.15.39:0/147905292 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f89441083a0 0x7f894419aab0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f892c005ab0 tx=0x7f892c003cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:44:59.967 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:59.964+0000 7f8940d99640 1 -- 172.21.15.39:0/147905292 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f892c03f070 con 0x7f89441083a0 2024-09-21T03:44:59.967 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:59.964+0000 7f8940d99640 1 -- 172.21.15.39:0/147905292 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f892c031d60 con 0x7f89441083a0 2024-09-21T03:44:59.967 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:59.964+0000 7f89490d7640 1 -- 172.21.15.39:0/147905292 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f89441a0650 con 0x7f89441083a0 2024-09-21T03:44:59.967 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:59.964+0000 7f8940d99640 1 -- 172.21.15.39:0/147905292 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f892c035030 con 0x7f89441083a0 2024-09-21T03:44:59.967 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:59.964+0000 7f89490d7640 1 -- 172.21.15.39:0/147905292 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f89441a0b30 con 0x7f89441083a0 2024-09-21T03:44:59.968 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:59.965+0000 7f89490d7640 1 -- 172.21.15.39:0/147905292 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f8944104400 con 0x7f89441083a0 2024-09-21T03:44:59.968 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:59.967+0000 7f8940d99640 1 -- 172.21.15.39:0/147905292 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 22) ==== 154580+0+0 (secure 0 0 0) 0x7f892c042070 con 0x7f89441083a0 2024-09-21T03:44:59.969 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:59.968+0000 7f8940d99640 1 --2- 172.21.15.39:0/147905292 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f89300b6620 0x7f89300b8b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:59.969 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:59.968+0000 7f8942d9d640 1 --2- 172.21.15.39:0/147905292 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f89300b6620 0x7f89300b8b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:59.969 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:59.968+0000 7f8940d99640 1 -- 172.21.15.39:0/147905292 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8484+0+0 (secure 0 0 0) 0x7f892c108220 con 0x7f89441083a0 2024-09-21T03:44:59.970 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:59.968+0000 7f8940d99640 1 --2- 172.21.15.39:0/147905292 >> [v2:172.21.15.150:6800/172427832,v1:172.21.15.150:6801/172427832] conn(0x7f89300c0490 0x7f89300c2980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:59.970 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:59.968+0000 7f8940d99640 1 -- 172.21.15.39:0/147905292 --> [v2:172.21.15.150:6800/172427832,v1:172.21.15.150:6801/172427832] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f892c00b0b0 con 0x7f89300c0490 2024-09-21T03:44:59.970 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:59.968+0000 7f8940d99640 1 -- 172.21.15.39:0/147905292 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_get_version_reply(handle=1 version=79) ==== 24+0+0 (secure 0 0 0) 0x7f892c108670 con 0x7f89441083a0 2024-09-21T03:44:59.970 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:59.968+0000 7f8942d9d640 1 --2- 172.21.15.39:0/147905292 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f89300b6620 0x7f89300b8b30 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f893400abe0 tx=0x7f893400a2d0 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:44:59.970 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:59.968+0000 7f894359e640 1 --2- 172.21.15.39:0/147905292 >> [v2:172.21.15.150:6800/172427832,v1:172.21.15.150:6801/172427832] conn(0x7f89300c0490 0x7f89300c2980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:59.970 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:59.969+0000 7f894359e640 1 --2- 172.21.15.39:0/147905292 >> [v2:172.21.15.150:6800/172427832,v1:172.21.15.150:6801/172427832] conn(0x7f89300c0490 0x7f89300c2980 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:44:59.971 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:59.970+0000 7f8940d99640 1 -- 172.21.15.39:0/147905292 <== osd.1 v2:172.21.15.150:6800/172427832 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f892c00b0b0 con 0x7f89300c0490 2024-09-21T03:44:59.985 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:59.983+0000 7f89490d7640 1 -- 172.21.15.39:0/147905292 --> [v2:172.21.15.150:6800/172427832,v1:172.21.15.150:6801/172427832] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f8944103ce0 con 0x7f89300c0490 2024-09-21T03:44:59.985 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:59.984+0000 7f8940d99640 1 -- 172.21.15.39:0/147905292 <== osd.1 v2:172.21.15.150:6800/172427832 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f8944103ce0 con 0x7f89300c0490 2024-09-21T03:44:59.986 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:59.984+0000 7f89490d7640 1 -- 172.21.15.39:0/147905292 >> [v2:172.21.15.150:6800/172427832,v1:172.21.15.150:6801/172427832] conn(0x7f89300c0490 msgr2=0x7f89300c2980 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:59.986 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:59.984+0000 7f89490d7640 1 --2- 172.21.15.39:0/147905292 >> [v2:172.21.15.150:6800/172427832,v1:172.21.15.150:6801/172427832] conn(0x7f89300c0490 0x7f89300c2980 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:59.987 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:59.984+0000 7f89490d7640 1 -- 172.21.15.39:0/147905292 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f89300b6620 msgr2=0x7f89300b8b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:59.987 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:59.984+0000 7f89490d7640 1 --2- 172.21.15.39:0/147905292 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f89300b6620 0x7f89300b8b30 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f893400abe0 tx=0x7f893400a2d0 comp rx=0 tx=0).stop 2024-09-21T03:44:59.987 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:59.984+0000 7f89490d7640 1 -- 172.21.15.39:0/147905292 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f89441083a0 msgr2=0x7f894419aab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:59.987 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:59.984+0000 7f89490d7640 1 --2- 172.21.15.39:0/147905292 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f89441083a0 0x7f894419aab0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f892c005ab0 tx=0x7f892c003cd0 comp rx=0 tx=0).stop 2024-09-21T03:44:59.987 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:59.984+0000 7f894359e640 1 -- 172.21.15.39:0/147905292 reap_dead start 2024-09-21T03:44:59.987 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:59.984+0000 7f89490d7640 1 -- 172.21.15.39:0/147905292 shutdown_connections 2024-09-21T03:44:59.987 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:59.985+0000 7f89490d7640 1 -- 172.21.15.39:0/147905292 >> 172.21.15.39:0/147905292 conn(0x7f8944079730 msgr2=0x7f8944101a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:44:59.987 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:59.985+0000 7f89490d7640 1 -- 172.21.15.39:0/147905292 shutdown_connections 2024-09-21T03:44:59.987 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:59.985+0000 7f89490d7640 1 -- 172.21.15.39:0/147905292 wait complete. 2024-09-21T03:45:00.082 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:58.552+0000 7f1c0ed40640 1 -- 172.21.15.39:0/513216447 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1c08109e00 msgr2=0x7f1c08110630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:00.082 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:58.552+0000 7f1c0ed40640 1 --2- 172.21.15.39:0/513216447 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1c08109e00 0x7f1c08110630 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f1c00009ca0 tx=0x7f1c0002a8c0 comp rx=0 tx=0).stop 2024-09-21T03:45:00.082 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:58.552+0000 7f1c0ed40640 1 -- 172.21.15.39:0/513216447 shutdown_connections 2024-09-21T03:45:00.082 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:58.552+0000 7f1c0ed40640 1 --2- 172.21.15.39:0/513216447 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1c08109e00 0x7f1c08110630 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:00.082 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:58.552+0000 7f1c0ed40640 1 --2- 172.21.15.39:0/513216447 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1c081094b0 0x7f1c081098c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:00.083 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:58.552+0000 7f1c0ed40640 1 --2- 172.21.15.39:0/513216447 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1c08101f70 0x7f1c08102380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:00.083 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:58.552+0000 7f1c0ed40640 1 -- 172.21.15.39:0/513216447 >> 172.21.15.39:0/513216447 conn(0x7f1c08077c60 msgr2=0x7f1c080780a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:45:00.083 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:58.552+0000 7f1c0ed40640 1 -- 172.21.15.39:0/513216447 shutdown_connections 2024-09-21T03:45:00.083 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:58.552+0000 7f1c0ed40640 1 -- 172.21.15.39:0/513216447 wait complete. 2024-09-21T03:45:00.083 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:58.553+0000 7f1c0ed40640 1 Processor -- start 2024-09-21T03:45:00.083 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:58.553+0000 7f1c0ed40640 1 -- start start 2024-09-21T03:45:00.083 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:58.553+0000 7f1c0ed40640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1c08101f70 0x7f1c08102f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:00.083 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:58.554+0000 7f1c0ed40640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1c081094b0 0x7f1c08103450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:00.084 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:58.554+0000 7f1c0ed40640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1c08109e00 0x7f1c08103990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:00.084 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:58.554+0000 7f1c0ed40640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f1c08113ac0 con 0x7f1c08109e00 2024-09-21T03:45:00.084 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:58.554+0000 7f1c0ed40640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f1c08113940 con 0x7f1c081094b0 2024-09-21T03:45:00.084 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:58.554+0000 7f1c0ed40640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f1c08113c40 con 0x7f1c08101f70 2024-09-21T03:45:00.084 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:58.554+0000 7f1c0d2dd640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1c08109e00 0x7f1c08103990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:00.084 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:58.554+0000 7f1c0d2dd640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1c08109e00 0x7f1c08103990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:39930/0 (socket says 172.21.15.39:39930) 2024-09-21T03:45:00.084 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:58.554+0000 7f1c0d2dd640 1 -- 172.21.15.39:0/2652511613 learned_addr learned my addr 172.21.15.39:0/2652511613 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:45:00.084 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:58.554+0000 7f1c0cadc640 1 --2- 172.21.15.39:0/2652511613 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1c08101f70 0x7f1c08102f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:00.084 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:58.554+0000 7f1c07fff640 1 --2- 172.21.15.39:0/2652511613 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1c081094b0 0x7f1c08103450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:00.084 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:58.554+0000 7f1c0d2dd640 1 -- 172.21.15.39:0/2652511613 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1c08101f70 msgr2=0x7f1c08102f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:00.084 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:58.554+0000 7f1c0d2dd640 1 --2- 172.21.15.39:0/2652511613 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1c08101f70 0x7f1c08102f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:00.085 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:58.554+0000 7f1c0d2dd640 1 -- 172.21.15.39:0/2652511613 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1c081094b0 msgr2=0x7f1c08103450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:00.085 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:58.554+0000 7f1c0d2dd640 1 --2- 172.21.15.39:0/2652511613 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1c081094b0 0x7f1c08103450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:00.085 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:58.554+0000 7f1c0d2dd640 1 -- 172.21.15.39:0/2652511613 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1c081a8730 con 0x7f1c08109e00 2024-09-21T03:45:00.085 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:58.555+0000 7f1c0d2dd640 1 --2- 172.21.15.39:0/2652511613 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1c08109e00 0x7f1c08103990 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f1c000098d0 tx=0x7f1c000317b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:45:00.085 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:58.555+0000 7f1c05ffb640 1 -- 172.21.15.39:0/2652511613 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c00003b70 con 0x7f1c08109e00 2024-09-21T03:45:00.085 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:58.555+0000 7f1c0ed40640 1 -- 172.21.15.39:0/2652511613 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1c081a89c0 con 0x7f1c08109e00 2024-09-21T03:45:00.085 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:58.555+0000 7f1c05ffb640 1 -- 172.21.15.39:0/2652511613 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1c00003d10 con 0x7f1c08109e00 2024-09-21T03:45:00.085 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:58.555+0000 7f1c05ffb640 1 -- 172.21.15.39:0/2652511613 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c0002fa20 con 0x7f1c08109e00 2024-09-21T03:45:00.085 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:58.555+0000 7f1c0ed40640 1 -- 172.21.15.39:0/2652511613 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1c081a8f00 con 0x7f1c08109e00 2024-09-21T03:45:00.085 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:58.556+0000 7f1c0ed40640 1 -- 172.21.15.39:0/2652511613 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f1c08063b50 con 0x7f1c08109e00 2024-09-21T03:45:00.085 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:58.558+0000 7f1c05ffb640 1 -- 172.21.15.39:0/2652511613 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 22) ==== 154580+0+0 (secure 0 0 0) 0x7f1c0002fbe0 con 0x7f1c08109e00 2024-09-21T03:45:00.085 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:58.559+0000 7f1c05ffb640 1 --2- 172.21.15.39:0/2652511613 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f1bf80b65f0 0x7f1bf80b8b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:00.086 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:58.559+0000 7f1c0cadc640 1 --2- 172.21.15.39:0/2652511613 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f1bf80b65f0 0x7f1bf80b8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:00.086 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:58.559+0000 7f1c05ffb640 1 -- 172.21.15.39:0/2652511613 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8484+0+0 (secure 0 0 0) 0x7f1c00108bf0 con 0x7f1c08109e00 2024-09-21T03:45:00.086 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:58.559+0000 7f1c05ffb640 1 --2- 172.21.15.39:0/2652511613 >> [v2:172.21.15.150:6824/2484627938,v1:172.21.15.150:6825/2484627938] conn(0x7f1bf80c0460 0x7f1bf80c2950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:00.086 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:58.559+0000 7f1c05ffb640 1 -- 172.21.15.39:0/2652511613 --> [v2:172.21.15.150:6824/2484627938,v1:172.21.15.150:6825/2484627938] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f1bf80c3020 con 0x7f1bf80c0460 2024-09-21T03:45:00.086 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:58.559+0000 7f1c05ffb640 1 -- 172.21.15.39:0/2652511613 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_get_version_reply(handle=1 version=79) ==== 24+0+0 (secure 0 0 0) 0x7f1c00109040 con 0x7f1c08109e00 2024-09-21T03:45:00.086 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:58.559+0000 7f1c0cadc640 1 --2- 172.21.15.39:0/2652511613 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f1bf80b65f0 0x7f1bf80b8b00 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f1c08104960 tx=0x7f1bfc005f90 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:45:00.086 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:58.560+0000 7f1c07fff640 1 --2- 172.21.15.39:0/2652511613 >> [v2:172.21.15.150:6824/2484627938,v1:172.21.15.150:6825/2484627938] conn(0x7f1bf80c0460 0x7f1bf80c2950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:00.086 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:58.560+0000 7f1c07fff640 1 --2- 172.21.15.39:0/2652511613 >> [v2:172.21.15.150:6824/2484627938,v1:172.21.15.150:6825/2484627938] conn(0x7f1bf80c0460 0x7f1bf80c2950 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.10 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:45:00.086 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:58.561+0000 7f1c05ffb640 1 -- 172.21.15.39:0/2652511613 <== osd.10 v2:172.21.15.150:6824/2484627938 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f1bf80c3020 con 0x7f1bf80c0460 2024-09-21T03:45:00.087 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:58.576+0000 7f1c0ed40640 1 -- 172.21.15.39:0/2652511613 --> [v2:172.21.15.150:6824/2484627938,v1:172.21.15.150:6825/2484627938] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f1c08063f50 con 0x7f1bf80c0460 2024-09-21T03:45:00.087 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:58.577+0000 7f1c05ffb640 1 -- 172.21.15.39:0/2652511613 <== osd.10 v2:172.21.15.150:6824/2484627938 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f1c08063f50 con 0x7f1bf80c0460 2024-09-21T03:45:00.087 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:58.577+0000 7f1c0ed40640 1 -- 172.21.15.39:0/2652511613 >> [v2:172.21.15.150:6824/2484627938,v1:172.21.15.150:6825/2484627938] conn(0x7f1bf80c0460 msgr2=0x7f1bf80c2950 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:00.087 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:58.577+0000 7f1c0ed40640 1 --2- 172.21.15.39:0/2652511613 >> [v2:172.21.15.150:6824/2484627938,v1:172.21.15.150:6825/2484627938] conn(0x7f1bf80c0460 0x7f1bf80c2950 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:00.087 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:58.577+0000 7f1c0ed40640 1 -- 172.21.15.39:0/2652511613 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f1bf80b65f0 msgr2=0x7f1bf80b8b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:00.087 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:58.577+0000 7f1c0ed40640 1 --2- 172.21.15.39:0/2652511613 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f1bf80b65f0 0x7f1bf80b8b00 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f1c08104960 tx=0x7f1bfc005f90 comp rx=0 tx=0).stop 2024-09-21T03:45:00.087 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:58.577+0000 7f1c0ed40640 1 -- 172.21.15.39:0/2652511613 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1c08109e00 msgr2=0x7f1c08103990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:00.087 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:58.577+0000 7f1c0ed40640 1 --2- 172.21.15.39:0/2652511613 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1c08109e00 0x7f1c08103990 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f1c000098d0 tx=0x7f1c000317b0 comp rx=0 tx=0).stop 2024-09-21T03:45:00.087 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:58.577+0000 7f1c0d2dd640 1 -- 172.21.15.39:0/2652511613 reap_dead start 2024-09-21T03:45:00.088 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:58.577+0000 7f1c0ed40640 1 -- 172.21.15.39:0/2652511613 shutdown_connections 2024-09-21T03:45:00.088 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:58.577+0000 7f1c0ed40640 1 -- 172.21.15.39:0/2652511613 >> 172.21.15.39:0/2652511613 conn(0x7f1c08077c60 msgr2=0x7f1c081053e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:45:00.088 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:58.577+0000 7f1c0ed40640 1 -- 172.21.15.39:0/2652511613 shutdown_connections 2024-09-21T03:45:00.088 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:44:58.577+0000 7f1c0ed40640 1 -- 172.21.15.39:0/2652511613 wait complete. 2024-09-21T03:45:00.263 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/51116d5c-77ca-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:45:00.774 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:00.764+0000 7fa19f8d8640 1 -- 172.21.15.39:0/3002542419 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa198103bc0 msgr2=0x7fa198103fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:00.774 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:00.764+0000 7fa19f8d8640 1 --2- 172.21.15.39:0/3002542419 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa198103bc0 0x7fa198103fd0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fa188009ca0 tx=0x7fa18802a8e0 comp rx=0 tx=0).stop 2024-09-21T03:45:00.774 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:00.764+0000 7fa19f8d8640 1 -- 172.21.15.39:0/3002542419 shutdown_connections 2024-09-21T03:45:00.775 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:00.764+0000 7fa19f8d8640 1 --2- 172.21.15.39:0/3002542419 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa198109e00 0x7fa198110630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:00.775 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:00.764+0000 7fa19f8d8640 1 --2- 172.21.15.39:0/3002542419 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa1981094b0 0x7fa1981098c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:00.775 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:00.764+0000 7fa19f8d8640 1 --2- 172.21.15.39:0/3002542419 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa198103bc0 0x7fa198103fd0 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:00.775 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:00.764+0000 7fa19f8d8640 1 -- 172.21.15.39:0/3002542419 >> 172.21.15.39:0/3002542419 conn(0x7fa19807b910 msgr2=0x7fa19807bd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:45:00.775 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:00.764+0000 7fa19f8d8640 1 -- 172.21.15.39:0/3002542419 shutdown_connections 2024-09-21T03:45:00.775 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:00.764+0000 7fa19f8d8640 1 -- 172.21.15.39:0/3002542419 wait complete. 2024-09-21T03:45:00.775 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:00.765+0000 7fa19f8d8640 1 Processor -- start 2024-09-21T03:45:00.775 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:00.765+0000 7fa19f8d8640 1 -- start start 2024-09-21T03:45:00.775 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:00.765+0000 7fa19f8d8640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa198103bc0 0x7fa19819a610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:00.776 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:00.765+0000 7fa19f8d8640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa1981094b0 0x7fa19819abb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:00.776 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:00.765+0000 7fa19f8d8640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa198109e00 0x7fa19819fe50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:00.776 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:00.765+0000 7fa19f8d8640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fa198113b00 con 0x7fa1981094b0 2024-09-21T03:45:00.776 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:00.765+0000 7fa19f8d8640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fa198113980 con 0x7fa198109e00 2024-09-21T03:45:00.776 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:00.765+0000 7fa19f8d8640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fa198113c80 con 0x7fa198103bc0 2024-09-21T03:45:00.776 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:00.765+0000 7fa19ce73640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa1981094b0 0x7fa19819abb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:00.776 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:00.765+0000 7fa19ce73640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa1981094b0 0x7fa19819abb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:39962/0 (socket says 172.21.15.39:39962) 2024-09-21T03:45:00.776 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:00.765+0000 7fa19ce73640 1 -- 172.21.15.39:0/3848233395 learned_addr learned my addr 172.21.15.39:0/3848233395 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:45:00.776 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:00.766+0000 7fa19d674640 1 --2- 172.21.15.39:0/3848233395 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa198103bc0 0x7fa19819a610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:00.776 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:00.766+0000 7fa19de75640 1 --2- 172.21.15.39:0/3848233395 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa198109e00 0x7fa19819fe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:00.776 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:00.766+0000 7fa19ce73640 1 -- 172.21.15.39:0/3848233395 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa198103bc0 msgr2=0x7fa19819a610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:00.776 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:00.766+0000 7fa19ce73640 1 --2- 172.21.15.39:0/3848233395 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa198103bc0 0x7fa19819a610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:00.776 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:00.766+0000 7fa19ce73640 1 -- 172.21.15.39:0/3848233395 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa198109e00 msgr2=0x7fa19819fe50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:00.776 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:00.766+0000 7fa19ce73640 1 --2- 172.21.15.39:0/3848233395 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa198109e00 0x7fa19819fe50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:00.776 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:00.766+0000 7fa19ce73640 1 -- 172.21.15.39:0/3848233395 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa1981a0580 con 0x7fa1981094b0 2024-09-21T03:45:00.777 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:00.766+0000 7fa19de75640 1 --2- 172.21.15.39:0/3848233395 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa198109e00 0x7fa19819fe50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:45:00.777 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:00.766+0000 7fa19ce73640 1 --2- 172.21.15.39:0/3848233395 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa1981094b0 0x7fa19819abb0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fa18c006fd0 tx=0x7fa18c00b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:45:00.777 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:00.766+0000 7fa1967fc640 1 -- 172.21.15.39:0/3848233395 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa18c011830 con 0x7fa1981094b0 2024-09-21T03:45:00.777 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:00.766+0000 7fa1967fc640 1 -- 172.21.15.39:0/3848233395 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa18c016070 con 0x7fa1981094b0 2024-09-21T03:45:00.777 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:00.766+0000 7fa1967fc640 1 -- 172.21.15.39:0/3848233395 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa18c010e00 con 0x7fa1981094b0 2024-09-21T03:45:00.777 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:00.766+0000 7fa19f8d8640 1 -- 172.21.15.39:0/3848233395 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa1981a0870 con 0x7fa1981094b0 2024-09-21T03:45:00.777 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:00.766+0000 7fa19f8d8640 1 -- 172.21.15.39:0/3848233395 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa1981a6fd0 con 0x7fa1981094b0 2024-09-21T03:45:00.777 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:00.767+0000 7fa19f8d8640 1 -- 172.21.15.39:0/3848233395 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fa198063ad0 con 0x7fa1981094b0 2024-09-21T03:45:00.777 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:00.769+0000 7fa1967fc640 1 -- 172.21.15.39:0/3848233395 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 22) ==== 154580+0+0 (secure 0 0 0) 0x7fa18c01b460 con 0x7fa1981094b0 2024-09-21T03:45:00.777 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:00.769+0000 7fa1967fc640 1 --2- 172.21.15.39:0/3848233395 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7fa1800b6670 0x7fa1800b8b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:00.777 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:00.769+0000 7fa19d674640 1 --2- 172.21.15.39:0/3848233395 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7fa1800b6670 0x7fa1800b8b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:00.777 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:00.770+0000 7fa1967fc640 1 -- 172.21.15.39:0/3848233395 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8484+0+0 (secure 0 0 0) 0x7fa18c0e7610 con 0x7fa1981094b0 2024-09-21T03:45:00.777 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:00.770+0000 7fa1967fc640 1 --2- 172.21.15.39:0/3848233395 >> [v2:172.21.15.161:6800/2727876133,v1:172.21.15.161:6801/2727876133] conn(0x7fa1800c04e0 0x7fa1800c29d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:00.777 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:00.770+0000 7fa1967fc640 1 -- 172.21.15.39:0/3848233395 --> [v2:172.21.15.161:6800/2727876133,v1:172.21.15.161:6801/2727876133] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fa18c022e50 con 0x7fa1800c04e0 2024-09-21T03:45:00.778 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:00.770+0000 7fa1967fc640 1 -- 172.21.15.39:0/3848233395 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_get_version_reply(handle=1 version=79) ==== 24+0+0 (secure 0 0 0) 0x7fa18c0e7a60 con 0x7fa1981094b0 2024-09-21T03:45:00.778 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:00.770+0000 7fa19de75640 1 --2- 172.21.15.39:0/3848233395 >> [v2:172.21.15.161:6800/2727876133,v1:172.21.15.161:6801/2727876133] conn(0x7fa1800c04e0 0x7fa1800c29d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:00.778 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:00.770+0000 7fa19d674640 1 --2- 172.21.15.39:0/3848233395 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7fa1800b6670 0x7fa1800b8b80 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fa188000f80 tx=0x7fa1880093f0 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:45:00.778 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:00.770+0000 7fa19de75640 1 --2- 172.21.15.39:0/3848233395 >> [v2:172.21.15.161:6800/2727876133,v1:172.21.15.161:6801/2727876133] conn(0x7fa1800c04e0 0x7fa1800c29d0 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:45:00.778 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:00.771+0000 7fa1967fc640 1 -- 172.21.15.39:0/3848233395 <== osd.2 v2:172.21.15.161:6800/2727876133 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fa18c022e50 con 0x7fa1800c04e0 2024-09-21T03:45:00.787 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:00.785+0000 7fa19f8d8640 1 -- 172.21.15.39:0/3848233395 --> [v2:172.21.15.161:6800/2727876133,v1:172.21.15.161:6801/2727876133] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fa198063ed0 con 0x7fa1800c04e0 2024-09-21T03:45:00.787 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:00.785+0000 7fa1967fc640 1 -- 172.21.15.39:0/3848233395 <== osd.2 v2:172.21.15.161:6800/2727876133 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fa198063ed0 con 0x7fa1800c04e0 2024-09-21T03:45:00.789 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:00.786+0000 7fa19f8d8640 1 -- 172.21.15.39:0/3848233395 >> [v2:172.21.15.161:6800/2727876133,v1:172.21.15.161:6801/2727876133] conn(0x7fa1800c04e0 msgr2=0x7fa1800c29d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:00.789 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:00.786+0000 7fa19f8d8640 1 --2- 172.21.15.39:0/3848233395 >> [v2:172.21.15.161:6800/2727876133,v1:172.21.15.161:6801/2727876133] conn(0x7fa1800c04e0 0x7fa1800c29d0 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:00.789 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:00.786+0000 7fa19f8d8640 1 -- 172.21.15.39:0/3848233395 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7fa1800b6670 msgr2=0x7fa1800b8b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:00.789 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:00.786+0000 7fa19f8d8640 1 --2- 172.21.15.39:0/3848233395 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7fa1800b6670 0x7fa1800b8b80 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fa188000f80 tx=0x7fa1880093f0 comp rx=0 tx=0).stop 2024-09-21T03:45:00.789 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:00.786+0000 7fa19f8d8640 1 -- 172.21.15.39:0/3848233395 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa1981094b0 msgr2=0x7fa19819abb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:00.789 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:00.786+0000 7fa19f8d8640 1 --2- 172.21.15.39:0/3848233395 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa1981094b0 0x7fa19819abb0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fa18c006fd0 tx=0x7fa18c00b8e0 comp rx=0 tx=0).stop 2024-09-21T03:45:00.789 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:00.786+0000 7fa19de75640 1 -- 172.21.15.39:0/3848233395 reap_dead start 2024-09-21T03:45:00.789 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:00.786+0000 7fa19f8d8640 1 -- 172.21.15.39:0/3848233395 shutdown_connections 2024-09-21T03:45:00.789 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:00.786+0000 7fa19f8d8640 1 -- 172.21.15.39:0/3848233395 >> 172.21.15.39:0/3848233395 conn(0x7fa19807b910 msgr2=0x7fa198107600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:45:00.789 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:00.786+0000 7fa19f8d8640 1 -- 172.21.15.39:0/3848233395 shutdown_connections 2024-09-21T03:45:00.789 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:00.786+0000 7fa19f8d8640 1 -- 172.21.15.39:0/3848233395 wait complete. 2024-09-21T03:45:00.925 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:45:00 smithi039 ceph-mon[31211]: pgmap v294: 97 pgs: 97 active+clean; 647 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 597 B/s wr, 11 op/s 2024-09-21T03:45:00.927 INFO:teuthology.orchestra.run.smithi039.stdout:279172874267 2024-09-21T03:45:00.927 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 51116d5c-77ca-11ef-baf5-efdc52797490 -- ceph osd last-stat-seq osd.10 2024-09-21T03:45:00.967 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:45:00 smithi161 ceph-mon[34533]: pgmap v294: 97 pgs: 97 active+clean; 647 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 597 B/s wr, 11 op/s 2024-09-21T03:45:00.974 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:45:00 smithi150 ceph-mon[34289]: pgmap v294: 97 pgs: 97 active+clean; 647 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 597 B/s wr, 11 op/s 2024-09-21T03:45:01.565 INFO:teuthology.orchestra.run.smithi039.stdout:81604378694 2024-09-21T03:45:01.565 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 51116d5c-77ca-11ef-baf5-efdc52797490 -- ceph osd last-stat-seq osd.2 2024-09-21T03:45:02.113 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:02.111+0000 7f33e9911640 1 -- 172.21.15.39:0/4007332118 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f33e41049b0 msgr2=0x7f33e4104dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:02.113 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:02.111+0000 7f33e9911640 1 --2- 172.21.15.39:0/4007332118 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f33e41049b0 0x7f33e4104dc0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f33d4009900 tx=0x7f33d402a410 comp rx=0 tx=0).stop 2024-09-21T03:45:02.113 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:02.111+0000 7f33e9911640 1 -- 172.21.15.39:0/4007332118 shutdown_connections 2024-09-21T03:45:02.113 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:02.111+0000 7f33e9911640 1 --2- 172.21.15.39:0/4007332118 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f33e410de10 0x7f33e4110300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:02.113 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:02.111+0000 7f33e9911640 1 --2- 172.21.15.39:0/4007332118 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f33e4105300 0x7f33e410d850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:02.113 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:02.111+0000 7f33e9911640 1 --2- 172.21.15.39:0/4007332118 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f33e41049b0 0x7f33e4104dc0 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:02.113 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:02.111+0000 7f33e9911640 1 -- 172.21.15.39:0/4007332118 >> 172.21.15.39:0/4007332118 conn(0x7f33e4079730 msgr2=0x7f33e4079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:45:02.114 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:02.111+0000 7f33e9911640 1 -- 172.21.15.39:0/4007332118 shutdown_connections 2024-09-21T03:45:02.114 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:02.111+0000 7f33e9911640 1 -- 172.21.15.39:0/4007332118 wait complete. 2024-09-21T03:45:02.115 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:02.112+0000 7f33e9911640 1 Processor -- start 2024-09-21T03:45:02.115 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:02.112+0000 7f33e9911640 1 -- start start 2024-09-21T03:45:02.115 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:02.112+0000 7f33e9911640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f33e41049b0 0x7f33e419a500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:02.116 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:02.112+0000 7f33e9911640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f33e4105300 0x7f33e419aaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:02.116 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:02.112+0000 7f33e9911640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f33e410de10 0x7f33e419fd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:02.116 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:02.113+0000 7f33e9911640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f33e4113a90 con 0x7f33e4105300 2024-09-21T03:45:02.116 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:02.113+0000 7f33e9911640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f33e4113910 con 0x7f33e410de10 2024-09-21T03:45:02.116 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:02.113+0000 7f33e9911640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f33e4113c10 con 0x7f33e41049b0 2024-09-21T03:45:02.116 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:02.113+0000 7f33e27fc640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f33e4105300 0x7f33e419aaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:02.116 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:02.113+0000 7f33e27fc640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f33e4105300 0x7f33e419aaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:39980/0 (socket says 172.21.15.39:39980) 2024-09-21T03:45:02.116 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:02.113+0000 7f33e27fc640 1 -- 172.21.15.39:0/84776888 learned_addr learned my addr 172.21.15.39:0/84776888 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:45:02.116 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:02.113+0000 7f33e2ffd640 1 --2- 172.21.15.39:0/84776888 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f33e41049b0 0x7f33e419a500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:02.116 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:02.113+0000 7f33e37fe640 1 --2- 172.21.15.39:0/84776888 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f33e410de10 0x7f33e419fd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:02.116 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:02.113+0000 7f33e27fc640 1 -- 172.21.15.39:0/84776888 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f33e41049b0 msgr2=0x7f33e419a500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:02.116 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:02.113+0000 7f33e27fc640 1 --2- 172.21.15.39:0/84776888 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f33e41049b0 0x7f33e419a500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:02.116 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:02.113+0000 7f33e27fc640 1 -- 172.21.15.39:0/84776888 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f33e410de10 msgr2=0x7f33e419fd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:02.117 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:02.113+0000 7f33e27fc640 1 --2- 172.21.15.39:0/84776888 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f33e410de10 0x7f33e419fd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:02.117 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:02.113+0000 7f33e27fc640 1 -- 172.21.15.39:0/84776888 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f33e41a0470 con 0x7f33e4105300 2024-09-21T03:45:02.117 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:02.113+0000 7f33e27fc640 1 --2- 172.21.15.39:0/84776888 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f33e4105300 0x7f33e419aaa0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f33d800c970 tx=0x7f33d800ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:45:02.117 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:02.113+0000 7f33e890f640 1 -- 172.21.15.39:0/84776888 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33d8007bb0 con 0x7f33e4105300 2024-09-21T03:45:02.117 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:02.113+0000 7f33e9911640 1 -- 172.21.15.39:0/84776888 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f33e41a0760 con 0x7f33e4105300 2024-09-21T03:45:02.117 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:02.113+0000 7f33e890f640 1 -- 172.21.15.39:0/84776888 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f33d8007d50 con 0x7f33e4105300 2024-09-21T03:45:02.117 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:02.113+0000 7f33e890f640 1 -- 172.21.15.39:0/84776888 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33d800f450 con 0x7f33e4105300 2024-09-21T03:45:02.117 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:02.114+0000 7f33e9911640 1 -- 172.21.15.39:0/84776888 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f33e41a6ec0 con 0x7f33e4105300 2024-09-21T03:45:02.117 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:02.114+0000 7f33e9911640 1 -- 172.21.15.39:0/84776888 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f33e4063b50 con 0x7f33e4105300 2024-09-21T03:45:02.118 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:02.117+0000 7f33e890f640 1 -- 172.21.15.39:0/84776888 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 22) ==== 154580+0+0 (secure 0 0 0) 0x7f33d8004eb0 con 0x7f33e4105300 2024-09-21T03:45:02.119 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:02.117+0000 7f33e890f640 1 --2- 172.21.15.39:0/84776888 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f33cc0b65f0 0x7f33cc0b8b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:02.119 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:02.117+0000 7f33e2ffd640 1 --2- 172.21.15.39:0/84776888 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f33cc0b65f0 0x7f33cc0b8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:02.119 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:02.117+0000 7f33e890f640 1 -- 172.21.15.39:0/84776888 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8484+0+0 (secure 0 0 0) 0x7f33d80e78a0 con 0x7f33e4105300 2024-09-21T03:45:02.119 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:02.118+0000 7f33e890f640 1 --2- 172.21.15.39:0/84776888 >> [v2:172.21.15.161:6808/742751023,v1:172.21.15.161:6809/742751023] conn(0x7f33cc0c0460 0x7f33cc0c2950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:02.120 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:02.118+0000 7f33e890f640 1 -- 172.21.15.39:0/84776888 --> [v2:172.21.15.161:6808/742751023,v1:172.21.15.161:6809/742751023] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f33d8005600 con 0x7f33cc0c0460 2024-09-21T03:45:02.120 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:02.118+0000 7f33e890f640 1 -- 172.21.15.39:0/84776888 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_get_version_reply(handle=1 version=79) ==== 24+0+0 (secure 0 0 0) 0x7f33d80e7cf0 con 0x7f33e4105300 2024-09-21T03:45:02.120 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:02.118+0000 7f33e2ffd640 1 --2- 172.21.15.39:0/84776888 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f33cc0b65f0 0x7f33cc0b8b00 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f33d4000c00 tx=0x7f33d4031400 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:45:02.120 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:02.118+0000 7f33e37fe640 1 --2- 172.21.15.39:0/84776888 >> [v2:172.21.15.161:6808/742751023,v1:172.21.15.161:6809/742751023] conn(0x7f33cc0c0460 0x7f33cc0c2950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:02.120 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:02.119+0000 7f33e37fe640 1 --2- 172.21.15.39:0/84776888 >> [v2:172.21.15.161:6808/742751023,v1:172.21.15.161:6809/742751023] conn(0x7f33cc0c0460 0x7f33cc0c2950 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:45:02.121 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:02.120+0000 7f33e890f640 1 -- 172.21.15.39:0/84776888 <== osd.5 v2:172.21.15.161:6808/742751023 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f33d8005600 con 0x7f33cc0c0460 2024-09-21T03:45:02.135 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:02.133+0000 7f33e9911640 1 -- 172.21.15.39:0/84776888 --> [v2:172.21.15.161:6808/742751023,v1:172.21.15.161:6809/742751023] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f33e4063f50 con 0x7f33cc0c0460 2024-09-21T03:45:02.135 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:02.134+0000 7f33e890f640 1 -- 172.21.15.39:0/84776888 <== osd.5 v2:172.21.15.161:6808/742751023 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f33e4063f50 con 0x7f33cc0c0460 2024-09-21T03:45:02.136 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:02.134+0000 7f33e9911640 1 -- 172.21.15.39:0/84776888 >> [v2:172.21.15.161:6808/742751023,v1:172.21.15.161:6809/742751023] conn(0x7f33cc0c0460 msgr2=0x7f33cc0c2950 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:02.136 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:02.134+0000 7f33e9911640 1 --2- 172.21.15.39:0/84776888 >> [v2:172.21.15.161:6808/742751023,v1:172.21.15.161:6809/742751023] conn(0x7f33cc0c0460 0x7f33cc0c2950 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:02.136 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:02.134+0000 7f33e9911640 1 -- 172.21.15.39:0/84776888 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f33cc0b65f0 msgr2=0x7f33cc0b8b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:02.136 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:02.134+0000 7f33e9911640 1 --2- 172.21.15.39:0/84776888 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f33cc0b65f0 0x7f33cc0b8b00 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f33d4000c00 tx=0x7f33d4031400 comp rx=0 tx=0).stop 2024-09-21T03:45:02.136 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:02.134+0000 7f33e9911640 1 -- 172.21.15.39:0/84776888 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f33e4105300 msgr2=0x7f33e419aaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:02.136 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:02.134+0000 7f33e9911640 1 --2- 172.21.15.39:0/84776888 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f33e4105300 0x7f33e419aaa0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f33d800c970 tx=0x7f33d800ce30 comp rx=0 tx=0).stop 2024-09-21T03:45:02.136 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:02.134+0000 7f33e37fe640 1 -- 172.21.15.39:0/84776888 reap_dead start 2024-09-21T03:45:02.136 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:02.134+0000 7f33e9911640 1 -- 172.21.15.39:0/84776888 shutdown_connections 2024-09-21T03:45:02.136 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:02.134+0000 7f33e9911640 1 -- 172.21.15.39:0/84776888 >> 172.21.15.39:0/84776888 conn(0x7f33e4079730 msgr2=0x7f33e41007b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:45:02.136 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:02.134+0000 7f33e9911640 1 -- 172.21.15.39:0/84776888 shutdown_connections 2024-09-21T03:45:02.137 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:02.134+0000 7f33e9911640 1 -- 172.21.15.39:0/84776888 wait complete. 2024-09-21T03:45:02.968 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:45:02 smithi161 ceph-mon[34533]: pgmap v295: 97 pgs: 97 active+clean; 647 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 255 B/s wr, 9 op/s 2024-09-21T03:45:02.974 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:45:02 smithi150 ceph-mon[34289]: pgmap v295: 97 pgs: 97 active+clean; 647 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 255 B/s wr, 9 op/s 2024-09-21T03:45:03.083 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:45:02 smithi039 ceph-mon[31211]: pgmap v295: 97 pgs: 97 active+clean; 647 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 255 B/s wr, 9 op/s 2024-09-21T03:45:03.203 INFO:teuthology.orchestra.run.smithi039.stdout:60129542218 2024-09-21T03:45:03.203 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 51116d5c-77ca-11ef-baf5-efdc52797490 -- ceph osd last-stat-seq osd.1 2024-09-21T03:45:03.679 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:03.667+0000 7fbdff799640 1 -- 172.21.15.39:0/3040534886 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbdf8109e40 msgr2=0x7fbdf8110670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:03.679 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:03.667+0000 7fbdff799640 1 --2- 172.21.15.39:0/3040534886 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbdf8109e40 0x7fbdf8110670 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fbdf0009ca0 tx=0x7fbdf002a8c0 comp rx=0 tx=0).stop 2024-09-21T03:45:03.679 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:03.667+0000 7fbdff799640 1 -- 172.21.15.39:0/3040534886 shutdown_connections 2024-09-21T03:45:03.679 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:03.667+0000 7fbdff799640 1 --2- 172.21.15.39:0/3040534886 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbdf8109e40 0x7fbdf8110670 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:03.679 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:03.667+0000 7fbdff799640 1 --2- 172.21.15.39:0/3040534886 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbdf81094f0 0x7fbdf8109900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:03.680 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:03.667+0000 7fbdff799640 1 --2- 172.21.15.39:0/3040534886 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbdf8103bb0 0x7fbdf8103fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:03.680 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:03.667+0000 7fbdff799640 1 -- 172.21.15.39:0/3040534886 >> 172.21.15.39:0/3040534886 conn(0x7fbdf80ff600 msgr2=0x7fbdf8101a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:45:03.680 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:03.667+0000 7fbdff799640 1 -- 172.21.15.39:0/3040534886 shutdown_connections 2024-09-21T03:45:03.680 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:03.667+0000 7fbdff799640 1 -- 172.21.15.39:0/3040534886 wait complete. 2024-09-21T03:45:03.680 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:03.668+0000 7fbdff799640 1 Processor -- start 2024-09-21T03:45:03.680 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:03.668+0000 7fbdff799640 1 -- start start 2024-09-21T03:45:03.680 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:03.668+0000 7fbdff799640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbdf8103bb0 0x7fbdf819a470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:03.681 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:03.668+0000 7fbdff799640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbdf81094f0 0x7fbdf819aa10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:03.681 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:03.668+0000 7fbdff799640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbdf8109e40 0x7fbdf819fcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:03.681 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:03.668+0000 7fbdff799640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fbdf8113b00 con 0x7fbdf8103bb0 2024-09-21T03:45:03.681 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:03.668+0000 7fbdff799640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fbdf8113980 con 0x7fbdf81094f0 2024-09-21T03:45:03.681 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:03.668+0000 7fbdff799640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fbdf8113c80 con 0x7fbdf8109e40 2024-09-21T03:45:03.681 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:03.668+0000 7fbdfd535640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbdf8103bb0 0x7fbdf819a470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:03.682 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:03.669+0000 7fbdfd535640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbdf8103bb0 0x7fbdf819a470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:39992/0 (socket says 172.21.15.39:39992) 2024-09-21T03:45:03.682 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:03.669+0000 7fbdfd535640 1 -- 172.21.15.39:0/3966807727 learned_addr learned my addr 172.21.15.39:0/3966807727 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:45:03.682 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:03.669+0000 7fbdfcd34640 1 --2- 172.21.15.39:0/3966807727 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbdf81094f0 0x7fbdf819aa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:03.682 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:03.669+0000 7fbdfdd36640 1 --2- 172.21.15.39:0/3966807727 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbdf8109e40 0x7fbdf819fcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:03.682 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:03.669+0000 7fbdfd535640 1 -- 172.21.15.39:0/3966807727 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbdf8109e40 msgr2=0x7fbdf819fcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:03.682 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:03.669+0000 7fbdfd535640 1 --2- 172.21.15.39:0/3966807727 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbdf8109e40 0x7fbdf819fcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:03.683 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:03.669+0000 7fbdfd535640 1 -- 172.21.15.39:0/3966807727 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbdf81094f0 msgr2=0x7fbdf819aa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:03.683 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:03.669+0000 7fbdfd535640 1 --2- 172.21.15.39:0/3966807727 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbdf81094f0 0x7fbdf819aa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:03.683 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:03.669+0000 7fbdfd535640 1 -- 172.21.15.39:0/3966807727 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbdf81a03e0 con 0x7fbdf8103bb0 2024-09-21T03:45:03.683 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:03.669+0000 7fbdfd535640 1 --2- 172.21.15.39:0/3966807727 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbdf8103bb0 0x7fbdf819a470 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fbdf400d950 tx=0x7fbdf400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:45:03.683 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:03.671+0000 7fbdee7fc640 1 -- 172.21.15.39:0/3966807727 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbdf4017070 con 0x7fbdf8103bb0 2024-09-21T03:45:03.683 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:03.671+0000 7fbdff799640 1 -- 172.21.15.39:0/3966807727 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbdf81a06d0 con 0x7fbdf8103bb0 2024-09-21T03:45:03.684 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:03.671+0000 7fbdee7fc640 1 -- 172.21.15.39:0/3966807727 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbdf4009c30 con 0x7fbdf8103bb0 2024-09-21T03:45:03.684 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:03.671+0000 7fbdee7fc640 1 -- 172.21.15.39:0/3966807727 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbdf400f620 con 0x7fbdf8103bb0 2024-09-21T03:45:03.684 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:03.671+0000 7fbdff799640 1 -- 172.21.15.39:0/3966807727 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbdf81a6e30 con 0x7fbdf8103bb0 2024-09-21T03:45:03.684 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:03.672+0000 7fbdff799640 1 -- 172.21.15.39:0/3966807727 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fbdf8063b50 con 0x7fbdf8103bb0 2024-09-21T03:45:03.684 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:03.673+0000 7fbdee7fc640 1 -- 172.21.15.39:0/3966807727 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 22) ==== 154580+0+0 (secure 0 0 0) 0x7fbdf4010460 con 0x7fbdf8103bb0 2024-09-21T03:45:03.684 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:03.674+0000 7fbdee7fc640 1 --2- 172.21.15.39:0/3966807727 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7fbde40b65f0 0x7fbde40b8b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:03.685 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:03.674+0000 7fbdfcd34640 1 --2- 172.21.15.39:0/3966807727 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7fbde40b65f0 0x7fbde40b8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:03.685 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:03.674+0000 7fbdee7fc640 1 -- 172.21.15.39:0/3966807727 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8484+0+0 (secure 0 0 0) 0x7fbdf40e7720 con 0x7fbdf8103bb0 2024-09-21T03:45:03.685 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:03.674+0000 7fbdee7fc640 1 --2- 172.21.15.39:0/3966807727 >> [v2:172.21.15.39:6810/295394468,v1:172.21.15.39:6811/295394468] conn(0x7fbde40c0460 0x7fbde40c2950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:03.685 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:03.675+0000 7fbdee7fc640 1 -- 172.21.15.39:0/3966807727 --> [v2:172.21.15.39:6810/295394468,v1:172.21.15.39:6811/295394468] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fbdf4015e80 con 0x7fbde40c0460 2024-09-21T03:45:03.685 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:03.675+0000 7fbdee7fc640 1 -- 172.21.15.39:0/3966807727 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_get_version_reply(handle=1 version=79) ==== 24+0+0 (secure 0 0 0) 0x7fbdf40e7b70 con 0x7fbdf8103bb0 2024-09-21T03:45:03.685 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:03.675+0000 7fbdfdd36640 1 --2- 172.21.15.39:0/3966807727 >> [v2:172.21.15.39:6810/295394468,v1:172.21.15.39:6811/295394468] conn(0x7fbde40c0460 0x7fbde40c2950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:03.685 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:03.675+0000 7fbdfcd34640 1 --2- 172.21.15.39:0/3966807727 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7fbde40b65f0 0x7fbde40b8b00 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fbdf8107f60 tx=0x7fbde800b390 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:45:03.686 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:03.675+0000 7fbdfdd36640 1 --2- 172.21.15.39:0/3966807727 >> [v2:172.21.15.39:6810/295394468,v1:172.21.15.39:6811/295394468] conn(0x7fbde40c0460 0x7fbde40c2950 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:45:03.686 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:03.676+0000 7fbdee7fc640 1 -- 172.21.15.39:0/3966807727 <== osd.3 v2:172.21.15.39:6810/295394468 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fbdf4015e80 con 0x7fbde40c0460 2024-09-21T03:45:03.695 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:03.693+0000 7fbdff799640 1 -- 172.21.15.39:0/3966807727 --> [v2:172.21.15.39:6810/295394468,v1:172.21.15.39:6811/295394468] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fbdf8063f80 con 0x7fbde40c0460 2024-09-21T03:45:03.696 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:03.693+0000 7fbdee7fc640 1 -- 172.21.15.39:0/3966807727 <== osd.3 v2:172.21.15.39:6810/295394468 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fbdf8063f80 con 0x7fbde40c0460 2024-09-21T03:45:03.696 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:03.694+0000 7fbdff799640 1 -- 172.21.15.39:0/3966807727 >> [v2:172.21.15.39:6810/295394468,v1:172.21.15.39:6811/295394468] conn(0x7fbde40c0460 msgr2=0x7fbde40c2950 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:03.696 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:03.694+0000 7fbdff799640 1 --2- 172.21.15.39:0/3966807727 >> [v2:172.21.15.39:6810/295394468,v1:172.21.15.39:6811/295394468] conn(0x7fbde40c0460 0x7fbde40c2950 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:03.696 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:03.694+0000 7fbdff799640 1 -- 172.21.15.39:0/3966807727 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7fbde40b65f0 msgr2=0x7fbde40b8b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:03.697 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:03.694+0000 7fbdff799640 1 --2- 172.21.15.39:0/3966807727 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7fbde40b65f0 0x7fbde40b8b00 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fbdf8107f60 tx=0x7fbde800b390 comp rx=0 tx=0).stop 2024-09-21T03:45:03.697 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:03.694+0000 7fbdff799640 1 -- 172.21.15.39:0/3966807727 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbdf8103bb0 msgr2=0x7fbdf819a470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:03.697 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:03.694+0000 7fbdff799640 1 --2- 172.21.15.39:0/3966807727 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbdf8103bb0 0x7fbdf819a470 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fbdf400d950 tx=0x7fbdf400de10 comp rx=0 tx=0).stop 2024-09-21T03:45:03.697 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:03.694+0000 7fbdfdd36640 1 -- 172.21.15.39:0/3966807727 reap_dead start 2024-09-21T03:45:03.697 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:03.694+0000 7fbdff799640 1 -- 172.21.15.39:0/3966807727 shutdown_connections 2024-09-21T03:45:03.697 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:03.694+0000 7fbdff799640 1 -- 172.21.15.39:0/3966807727 >> 172.21.15.39:0/3966807727 conn(0x7fbdf80ff600 msgr2=0x7fbdf8101a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:45:03.697 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:03.694+0000 7fbdff799640 1 -- 172.21.15.39:0/3966807727 shutdown_connections 2024-09-21T03:45:03.697 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:03.694+0000 7fbdff799640 1 -- 172.21.15.39:0/3966807727 wait complete. 2024-09-21T03:45:03.897 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:03.894+0000 7f24bbb19640 1 -- 172.21.15.39:0/1178208961 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f24b4109e10 msgr2=0x7f24b4110640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:03.897 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:03.894+0000 7f24bbb19640 1 --2- 172.21.15.39:0/1178208961 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f24b4109e10 0x7f24b4110640 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f24ac009ca0 tx=0x7f24ac02a8c0 comp rx=0 tx=0).stop 2024-09-21T03:45:03.897 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:03.894+0000 7f24bbb19640 1 -- 172.21.15.39:0/1178208961 shutdown_connections 2024-09-21T03:45:03.897 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:03.894+0000 7f24bbb19640 1 --2- 172.21.15.39:0/1178208961 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f24b4109e10 0x7f24b4110640 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:03.897 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:03.894+0000 7f24bbb19640 1 --2- 172.21.15.39:0/1178208961 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f24b41094c0 0x7f24b41098d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:03.898 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:03.894+0000 7f24bbb19640 1 --2- 172.21.15.39:0/1178208961 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f24b4103b80 0x7f24b4103f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:03.898 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:03.894+0000 7f24bbb19640 1 -- 172.21.15.39:0/1178208961 >> 172.21.15.39:0/1178208961 conn(0x7f24b40ff610 msgr2=0x7f24b4101a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:45:03.898 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:03.894+0000 7f24bbb19640 1 -- 172.21.15.39:0/1178208961 shutdown_connections 2024-09-21T03:45:03.898 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:03.894+0000 7f24bbb19640 1 -- 172.21.15.39:0/1178208961 wait complete. 2024-09-21T03:45:03.899 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:03.895+0000 7f24bbb19640 1 Processor -- start 2024-09-21T03:45:03.899 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:03.895+0000 7f24bbb19640 1 -- start start 2024-09-21T03:45:03.899 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:03.895+0000 7f24bbb19640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f24b4103b80 0x7f24b4075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:03.899 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:03.895+0000 7f24bbb19640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f24b41094c0 0x7f24b4075c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:03.900 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:03.895+0000 7f24bbb19640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f24b4109e10 0x7f24b4076190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:03.900 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:03.895+0000 7f24bbb19640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f24b4113ad0 con 0x7f24b4109e10 2024-09-21T03:45:03.900 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:03.895+0000 7f24bbb19640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f24b4113950 con 0x7f24b41094c0 2024-09-21T03:45:03.900 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:03.895+0000 7f24bbb19640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f24b4113c50 con 0x7f24b4103b80 2024-09-21T03:45:03.900 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:03.895+0000 7f24ba0b6640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f24b4109e10 0x7f24b4076190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:03.900 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:03.896+0000 7f24ba0b6640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f24b4109e10 0x7f24b4076190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:40006/0 (socket says 172.21.15.39:40006) 2024-09-21T03:45:03.900 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:03.896+0000 7f24ba0b6640 1 -- 172.21.15.39:0/2695952062 learned_addr learned my addr 172.21.15.39:0/2695952062 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:45:03.900 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:03.896+0000 7f24b98b5640 1 --2- 172.21.15.39:0/2695952062 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f24b4103b80 0x7f24b4075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:03.900 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:03.896+0000 7f24b90b4640 1 --2- 172.21.15.39:0/2695952062 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f24b41094c0 0x7f24b4075c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:03.900 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:03.896+0000 7f24ba0b6640 1 -- 172.21.15.39:0/2695952062 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f24b4103b80 msgr2=0x7f24b4075710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:03.900 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:03.896+0000 7f24ba0b6640 1 --2- 172.21.15.39:0/2695952062 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f24b4103b80 0x7f24b4075710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:03.901 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:03.896+0000 7f24ba0b6640 1 -- 172.21.15.39:0/2695952062 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f24b41094c0 msgr2=0x7f24b4075c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:03.901 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:03.896+0000 7f24ba0b6640 1 --2- 172.21.15.39:0/2695952062 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f24b41094c0 0x7f24b4075c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:03.901 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:03.896+0000 7f24ba0b6640 1 -- 172.21.15.39:0/2695952062 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f24b41a86e0 con 0x7f24b4109e10 2024-09-21T03:45:03.901 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:03.896+0000 7f24ba0b6640 1 --2- 172.21.15.39:0/2695952062 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f24b4109e10 0x7f24b4076190 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f24ac005cc0 tx=0x7f24ac02f380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:45:03.901 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:03.896+0000 7f24aaffd640 1 -- 172.21.15.39:0/2695952062 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24ac003bf0 con 0x7f24b4109e10 2024-09-21T03:45:03.901 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:03.896+0000 7f24aaffd640 1 -- 172.21.15.39:0/2695952062 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f24ac003d90 con 0x7f24b4109e10 2024-09-21T03:45:03.902 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:03.896+0000 7f24bbb19640 1 -- 172.21.15.39:0/2695952062 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f24b41a8970 con 0x7f24b4109e10 2024-09-21T03:45:03.902 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:03.896+0000 7f24aaffd640 1 -- 172.21.15.39:0/2695952062 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24ac02fbc0 con 0x7f24b4109e10 2024-09-21T03:45:03.902 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:03.897+0000 7f24bbb19640 1 -- 172.21.15.39:0/2695952062 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f24b41a8eb0 con 0x7f24b4109e10 2024-09-21T03:45:03.902 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:03.897+0000 7f24bbb19640 1 -- 172.21.15.39:0/2695952062 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f24b4063b50 con 0x7f24b4109e10 2024-09-21T03:45:03.902 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:03.900+0000 7f24aaffd640 1 -- 172.21.15.39:0/2695952062 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 22) ==== 154580+0+0 (secure 0 0 0) 0x7f24ac02fd60 con 0x7f24b4109e10 2024-09-21T03:45:03.903 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:03.900+0000 7f24aaffd640 1 --2- 172.21.15.39:0/2695952062 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f24a00b65f0 0x7f24a00b8b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:03.903 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:03.901+0000 7f24aaffd640 1 -- 172.21.15.39:0/2695952062 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8484+0+0 (secure 0 0 0) 0x7f24ac108c10 con 0x7f24b4109e10 2024-09-21T03:45:03.904 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:03.901+0000 7f24b98b5640 1 --2- 172.21.15.39:0/2695952062 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f24a00b65f0 0x7f24a00b8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:03.904 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:03.901+0000 7f24aaffd640 1 --2- 172.21.15.39:0/2695952062 >> [v2:172.21.15.150:6816/2162619103,v1:172.21.15.150:6817/2162619103] conn(0x7f24a00c0460 0x7f24a00c2950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:03.904 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:03.901+0000 7f24aaffd640 1 -- 172.21.15.39:0/2695952062 --> [v2:172.21.15.150:6816/2162619103,v1:172.21.15.150:6817/2162619103] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f24a00c3020 con 0x7f24a00c0460 2024-09-21T03:45:03.904 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:03.901+0000 7f24aaffd640 1 -- 172.21.15.39:0/2695952062 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_get_version_reply(handle=1 version=79) ==== 24+0+0 (secure 0 0 0) 0x7f24ac109060 con 0x7f24b4109e10 2024-09-21T03:45:03.904 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:03.901+0000 7f24b90b4640 1 --2- 172.21.15.39:0/2695952062 >> [v2:172.21.15.150:6816/2162619103,v1:172.21.15.150:6817/2162619103] conn(0x7f24a00c0460 0x7f24a00c2950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:03.904 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:03.901+0000 7f24b98b5640 1 --2- 172.21.15.39:0/2695952062 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f24a00b65f0 0x7f24a00b8b00 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f24b00058c0 tx=0x7f24b000a2d0 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:45:03.906 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:03.902+0000 7f24b90b4640 1 --2- 172.21.15.39:0/2695952062 >> [v2:172.21.15.150:6816/2162619103,v1:172.21.15.150:6817/2162619103] conn(0x7f24a00c0460 0x7f24a00c2950 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:45:03.906 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:03.902+0000 7f24aaffd640 1 -- 172.21.15.39:0/2695952062 <== osd.7 v2:172.21.15.150:6816/2162619103 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f24a00c3020 con 0x7f24a00c0460 2024-09-21T03:45:03.919 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:03.916+0000 7f24bbb19640 1 -- 172.21.15.39:0/2695952062 --> [v2:172.21.15.150:6816/2162619103,v1:172.21.15.150:6817/2162619103] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f24b4063f50 con 0x7f24a00c0460 2024-09-21T03:45:03.919 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:03.917+0000 7f24aaffd640 1 -- 172.21.15.39:0/2695952062 <== osd.7 v2:172.21.15.150:6816/2162619103 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f24b4063f50 con 0x7f24a00c0460 2024-09-21T03:45:03.920 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:03.917+0000 7f24bbb19640 1 -- 172.21.15.39:0/2695952062 >> [v2:172.21.15.150:6816/2162619103,v1:172.21.15.150:6817/2162619103] conn(0x7f24a00c0460 msgr2=0x7f24a00c2950 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:03.920 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:03.917+0000 7f24bbb19640 1 --2- 172.21.15.39:0/2695952062 >> [v2:172.21.15.150:6816/2162619103,v1:172.21.15.150:6817/2162619103] conn(0x7f24a00c0460 0x7f24a00c2950 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:03.920 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:03.917+0000 7f24bbb19640 1 -- 172.21.15.39:0/2695952062 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f24a00b65f0 msgr2=0x7f24a00b8b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:03.920 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:03.917+0000 7f24bbb19640 1 --2- 172.21.15.39:0/2695952062 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f24a00b65f0 0x7f24a00b8b00 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f24b00058c0 tx=0x7f24b000a2d0 comp rx=0 tx=0).stop 2024-09-21T03:45:03.920 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:03.917+0000 7f24bbb19640 1 -- 172.21.15.39:0/2695952062 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f24b4109e10 msgr2=0x7f24b4076190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:03.920 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:03.917+0000 7f24bbb19640 1 --2- 172.21.15.39:0/2695952062 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f24b4109e10 0x7f24b4076190 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f24ac005cc0 tx=0x7f24ac02f380 comp rx=0 tx=0).stop 2024-09-21T03:45:03.920 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:03.917+0000 7f24ba0b6640 1 -- 172.21.15.39:0/2695952062 reap_dead start 2024-09-21T03:45:03.920 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:03.917+0000 7f24bbb19640 1 -- 172.21.15.39:0/2695952062 shutdown_connections 2024-09-21T03:45:03.920 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:03.917+0000 7f24bbb19640 1 -- 172.21.15.39:0/2695952062 >> 172.21.15.39:0/2695952062 conn(0x7f24b40ff610 msgr2=0x7f24b4106a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:45:03.921 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:03.917+0000 7f24bbb19640 1 -- 172.21.15.39:0/2695952062 shutdown_connections 2024-09-21T03:45:03.921 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:03.917+0000 7f24bbb19640 1 -- 172.21.15.39:0/2695952062 wait complete. 2024-09-21T03:45:04.556 INFO:teuthology.orchestra.run.smithi039.stdout:111669149761 2024-09-21T03:45:04.556 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 51116d5c-77ca-11ef-baf5-efdc52797490 -- ceph osd last-stat-seq osd.3 2024-09-21T03:45:04.833 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:45:04 smithi039 ceph-mon[31211]: pgmap v296: 97 pgs: 97 active+clean; 647 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 255 B/s wr, 9 op/s 2024-09-21T03:45:04.966 INFO:teuthology.orchestra.run.smithi039.stdout:158913790006 2024-09-21T03:45:04.966 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 51116d5c-77ca-11ef-baf5-efdc52797490 -- ceph osd last-stat-seq osd.5 2024-09-21T03:45:04.968 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:45:04 smithi161 ceph-mon[34533]: pgmap v296: 97 pgs: 97 active+clean; 647 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 255 B/s wr, 9 op/s 2024-09-21T03:45:04.974 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:45:04 smithi150 ceph-mon[34289]: pgmap v296: 97 pgs: 97 active+clean; 647 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 255 B/s wr, 9 op/s 2024-09-21T03:45:05.636 INFO:teuthology.orchestra.run.smithi039.stdout:210453397549 2024-09-21T03:45:05.636 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 51116d5c-77ca-11ef-baf5-efdc52797490 -- ceph osd last-stat-seq osd.7 2024-09-21T03:45:06.097 INFO:teuthology.orchestra.run.smithi039.stdout:133143986234 2024-09-21T03:45:06.098 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 51116d5c-77ca-11ef-baf5-efdc52797490 -- ceph osd last-stat-seq osd.4 2024-09-21T03:45:06.195 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/51116d5c-77ca-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:45:06.195 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/51116d5c-77ca-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:45:06.196 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/51116d5c-77ca-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:45:06.196 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/51116d5c-77ca-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:45:06.196 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/51116d5c-77ca-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:45:06.201 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/51116d5c-77ca-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:45:06.201 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/51116d5c-77ca-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:45:06.202 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/51116d5c-77ca-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:45:06.204 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/51116d5c-77ca-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:45:06.307 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/51116d5c-77ca-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:45:06.938 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:45:06 smithi039 ceph-mon[31211]: pgmap v297: 97 pgs: 97 active+clean; 647 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 255 B/s wr, 12 op/s 2024-09-21T03:45:06.964 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:06.961+0000 7f52e568c640 1 -- 172.21.15.39:0/820533508 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f52e0105310 msgr2=0x7f52e010d860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:06.964 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:06.961+0000 7f52e568c640 1 --2- 172.21.15.39:0/820533508 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f52e0105310 0x7f52e010d860 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f52c8009900 tx=0x7f52c802a410 comp rx=0 tx=0).stop 2024-09-21T03:45:06.964 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:06.961+0000 7f52e568c640 1 -- 172.21.15.39:0/820533508 shutdown_connections 2024-09-21T03:45:06.964 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:06.961+0000 7f52e568c640 1 --2- 172.21.15.39:0/820533508 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f52e010de20 0x7f52e0110310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:06.964 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:06.961+0000 7f52e568c640 1 --2- 172.21.15.39:0/820533508 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f52e0105310 0x7f52e010d860 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:06.964 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:06.961+0000 7f52e568c640 1 --2- 172.21.15.39:0/820533508 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f52e01049c0 0x7f52e0104dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:06.964 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:06.961+0000 7f52e568c640 1 -- 172.21.15.39:0/820533508 >> 172.21.15.39:0/820533508 conn(0x7f52e0079730 msgr2=0x7f52e0079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:45:06.964 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:06.961+0000 7f52e568c640 1 -- 172.21.15.39:0/820533508 shutdown_connections 2024-09-21T03:45:06.964 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:06.961+0000 7f52e568c640 1 -- 172.21.15.39:0/820533508 wait complete. 2024-09-21T03:45:06.966 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:06.962+0000 7f52e568c640 1 Processor -- start 2024-09-21T03:45:06.966 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:06.962+0000 7f52e568c640 1 -- start start 2024-09-21T03:45:06.966 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:06.962+0000 7f52e568c640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f52e01049c0 0x7f52e019a470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:06.966 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:06.962+0000 7f52e568c640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f52e0105310 0x7f52e019aa10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:06.966 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:06.962+0000 7f52e568c640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f52e010de20 0x7f52e019fcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:06.966 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:06.962+0000 7f52e568c640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f52e0113a80 con 0x7f52e0105310 2024-09-21T03:45:06.966 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:06.962+0000 7f52e568c640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f52e0113900 con 0x7f52e01049c0 2024-09-21T03:45:06.967 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:06.962+0000 7f52e568c640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f52e0113c00 con 0x7f52e010de20 2024-09-21T03:45:06.967 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:06.962+0000 7f52de7fc640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f52e0105310 0x7f52e019aa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:06.967 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:06.962+0000 7f52de7fc640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f52e0105310 0x7f52e019aa10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:40018/0 (socket says 172.21.15.39:40018) 2024-09-21T03:45:06.967 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:06.962+0000 7f52deffd640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f52e01049c0 0x7f52e019a470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:06.967 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:06.962+0000 7f52de7fc640 1 -- 172.21.15.39:0/371407795 learned_addr learned my addr 172.21.15.39:0/371407795 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:45:06.967 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:06.962+0000 7f52df7fe640 1 --2- 172.21.15.39:0/371407795 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f52e010de20 0x7f52e019fcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:06.967 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:06.963+0000 7f52de7fc640 1 -- 172.21.15.39:0/371407795 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f52e010de20 msgr2=0x7f52e019fcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:06.967 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:06.963+0000 7f52de7fc640 1 --2- 172.21.15.39:0/371407795 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f52e010de20 0x7f52e019fcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:06.969 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:06.963+0000 7f52de7fc640 1 -- 172.21.15.39:0/371407795 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f52e01049c0 msgr2=0x7f52e019a470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:06.969 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:06.963+0000 7f52de7fc640 1 --2- 172.21.15.39:0/371407795 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f52e01049c0 0x7f52e019a470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:06.970 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:06.963+0000 7f52de7fc640 1 -- 172.21.15.39:0/371407795 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f52e01a03e0 con 0x7f52e0105310 2024-09-21T03:45:06.970 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:06.963+0000 7f52de7fc640 1 --2- 172.21.15.39:0/371407795 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f52e0105310 0x7f52e019aa10 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f52c802a930 tx=0x7f52c8004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:45:06.970 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:06.963+0000 7f52d7fff640 1 -- 172.21.15.39:0/371407795 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52c803f030 con 0x7f52e0105310 2024-09-21T03:45:06.970 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:06.963+0000 7f52e568c640 1 -- 172.21.15.39:0/371407795 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f52e01a0670 con 0x7f52e0105310 2024-09-21T03:45:06.970 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:06.963+0000 7f52d7fff640 1 -- 172.21.15.39:0/371407795 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f52c8003cb0 con 0x7f52e0105310 2024-09-21T03:45:06.970 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:06.963+0000 7f52d7fff640 1 -- 172.21.15.39:0/371407795 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52c8035070 con 0x7f52e0105310 2024-09-21T03:45:06.970 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:06.963+0000 7f52e568c640 1 -- 172.21.15.39:0/371407795 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f52e01a6e30 con 0x7f52e0105310 2024-09-21T03:45:06.970 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:45:06 smithi161 ceph-mon[34533]: pgmap v297: 97 pgs: 97 active+clean; 647 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 255 B/s wr, 12 op/s 2024-09-21T03:45:06.971 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:06.964+0000 7f52e568c640 1 -- 172.21.15.39:0/371407795 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f52e006c1e0 con 0x7f52e0105310 2024-09-21T03:45:06.971 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:06.966+0000 7f52d7fff640 1 -- 172.21.15.39:0/371407795 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 22) ==== 154580+0+0 (secure 0 0 0) 0x7f52c8042030 con 0x7f52e0105310 2024-09-21T03:45:06.971 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:06.966+0000 7f52d7fff640 1 --2- 172.21.15.39:0/371407795 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f52cc0b65f0 0x7f52cc0b8b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:06.971 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:06.966+0000 7f52deffd640 1 --2- 172.21.15.39:0/371407795 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f52cc0b65f0 0x7f52cc0b8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:06.971 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:06.967+0000 7f52d7fff640 1 -- 172.21.15.39:0/371407795 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8484+0+0 (secure 0 0 0) 0x7f52c8107cc0 con 0x7f52e0105310 2024-09-21T03:45:06.972 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:06.967+0000 7f52deffd640 1 --2- 172.21.15.39:0/371407795 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f52cc0b65f0 0x7f52cc0b8b00 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f52e006c8e0 tx=0x7f52d800a2d0 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:45:06.972 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:06.970+0000 7f52d7fff640 1 -- 172.21.15.39:0/371407795 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f52c80ce2f0 con 0x7f52e0105310 2024-09-21T03:45:06.974 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:45:06 smithi150 ceph-mon[34289]: pgmap v297: 97 pgs: 97 active+clean; 647 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 255 B/s wr, 12 op/s 2024-09-21T03:45:07.165 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:07.162+0000 7f52e568c640 1 -- 172.21.15.39:0/371407795 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7f52e0104dd0 con 0x7f52e0105310 2024-09-21T03:45:07.165 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:07.163+0000 7f52d7fff640 1 -- 172.21.15.39:0/371407795 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f52c80d26d0 con 0x7f52e0105310 2024-09-21T03:45:07.166 INFO:teuthology.orchestra.run.smithi039.stdout:38654705745 2024-09-21T03:45:07.167 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:07.165+0000 7f52e568c640 1 -- 172.21.15.39:0/371407795 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f52cc0b65f0 msgr2=0x7f52cc0b8b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:07.167 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:07.165+0000 7f52e568c640 1 --2- 172.21.15.39:0/371407795 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f52cc0b65f0 0x7f52cc0b8b00 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f52e006c8e0 tx=0x7f52d800a2d0 comp rx=0 tx=0).stop 2024-09-21T03:45:07.168 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:07.165+0000 7f52e568c640 1 -- 172.21.15.39:0/371407795 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f52e0105310 msgr2=0x7f52e019aa10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:07.168 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:07.165+0000 7f52e568c640 1 --2- 172.21.15.39:0/371407795 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f52e0105310 0x7f52e019aa10 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f52c802a930 tx=0x7f52c8004270 comp rx=0 tx=0).stop 2024-09-21T03:45:07.168 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:07.165+0000 7f52e568c640 1 -- 172.21.15.39:0/371407795 shutdown_connections 2024-09-21T03:45:07.168 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:07.165+0000 7f52e568c640 1 --2- 172.21.15.39:0/371407795 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f52cc0b65f0 0x7f52cc0b8b00 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:07.168 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:07.165+0000 7f52e568c640 1 --2- 172.21.15.39:0/371407795 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f52e010de20 0x7f52e019fcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:07.168 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:07.165+0000 7f52e568c640 1 --2- 172.21.15.39:0/371407795 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f52e0105310 0x7f52e019aa10 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:07.168 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:07.165+0000 7f52e568c640 1 --2- 172.21.15.39:0/371407795 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f52e01049c0 0x7f52e019a470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:07.169 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:07.165+0000 7f52e568c640 1 -- 172.21.15.39:0/371407795 >> 172.21.15.39:0/371407795 conn(0x7f52e0079730 msgr2=0x7f52e0100880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:45:07.169 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:07.165+0000 7f52e568c640 1 -- 172.21.15.39:0/371407795 shutdown_connections 2024-09-21T03:45:07.169 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:07.165+0000 7f52e568c640 1 -- 172.21.15.39:0/371407795 wait complete. 2024-09-21T03:45:07.594 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:07.591+0000 7f7e181ac640 1 -- 172.21.15.39:0/2903197537 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7e10108cf0 msgr2=0x7f7e1010f520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:07.594 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:07.591+0000 7f7e181ac640 1 --2- 172.21.15.39:0/2903197537 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7e10108cf0 0x7f7e1010f520 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f7e08009ca0 tx=0x7f7e0802a8c0 comp rx=0 tx=0).stop 2024-09-21T03:45:07.594 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:07.591+0000 7f7e181ac640 1 -- 172.21.15.39:0/2903197537 shutdown_connections 2024-09-21T03:45:07.595 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:07.591+0000 7f7e181ac640 1 --2- 172.21.15.39:0/2903197537 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7e10108cf0 0x7f7e1010f520 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:07.595 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:07.591+0000 7f7e181ac640 1 --2- 172.21.15.39:0/2903197537 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7e101083a0 0x7f7e101087b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:07.595 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:07.591+0000 7f7e181ac640 1 --2- 172.21.15.39:0/2903197537 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7e10102a60 0x7f7e10102e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:07.595 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:07.591+0000 7f7e181ac640 1 -- 172.21.15.39:0/2903197537 >> 172.21.15.39:0/2903197537 conn(0x7f7e10079730 msgr2=0x7f7e10079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:45:07.595 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:07.592+0000 7f7e181ac640 1 -- 172.21.15.39:0/2903197537 shutdown_connections 2024-09-21T03:45:07.595 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:07.592+0000 7f7e181ac640 1 -- 172.21.15.39:0/2903197537 wait complete. 2024-09-21T03:45:07.597 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:07.592+0000 7f7e181ac640 1 Processor -- start 2024-09-21T03:45:07.597 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:07.592+0000 7f7e181ac640 1 -- start start 2024-09-21T03:45:07.597 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:07.593+0000 7f7e181ac640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7e10102a60 0x7f7e1019a460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:07.597 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:07.593+0000 7f7e181ac640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7e101083a0 0x7f7e1019aa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:07.597 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:07.593+0000 7f7e181ac640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7e10108cf0 0x7f7e1019fca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:07.597 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:07.593+0000 7f7e181ac640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f7e101129c0 con 0x7f7e10108cf0 2024-09-21T03:45:07.597 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:07.593+0000 7f7e181ac640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f7e10112840 con 0x7f7e10102a60 2024-09-21T03:45:07.597 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:07.593+0000 7f7e181ac640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f7e10112b40 con 0x7f7e101083a0 2024-09-21T03:45:07.597 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:07.593+0000 7f7e16749640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7e10108cf0 0x7f7e1019fca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:07.597 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:07.593+0000 7f7e16749640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7e10108cf0 0x7f7e1019fca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:34794/0 (socket says 172.21.15.39:34794) 2024-09-21T03:45:07.597 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:07.593+0000 7f7e16749640 1 -- 172.21.15.39:0/2734708827 learned_addr learned my addr 172.21.15.39:0/2734708827 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:45:07.598 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:07.593+0000 7f7e15f48640 1 --2- 172.21.15.39:0/2734708827 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7e10102a60 0x7f7e1019a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:07.598 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:07.593+0000 7f7e15747640 1 --2- 172.21.15.39:0/2734708827 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7e101083a0 0x7f7e1019aa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:07.598 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:07.593+0000 7f7e16749640 1 -- 172.21.15.39:0/2734708827 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7e101083a0 msgr2=0x7f7e1019aa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:07.598 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:07.593+0000 7f7e16749640 1 --2- 172.21.15.39:0/2734708827 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7e101083a0 0x7f7e1019aa00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:07.598 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:07.593+0000 7f7e16749640 1 -- 172.21.15.39:0/2734708827 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7e10102a60 msgr2=0x7f7e1019a460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:07.598 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:07.593+0000 7f7e16749640 1 --2- 172.21.15.39:0/2734708827 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7e10102a60 0x7f7e1019a460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:07.598 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:07.593+0000 7f7e16749640 1 -- 172.21.15.39:0/2734708827 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7e101a03d0 con 0x7f7e10108cf0 2024-09-21T03:45:07.598 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:07.593+0000 7f7e16749640 1 --2- 172.21.15.39:0/2734708827 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7e10108cf0 0x7f7e1019fca0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f7e08005cc0 tx=0x7f7e0802f380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:45:07.598 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:07.594+0000 7f7e06ffd640 1 -- 172.21.15.39:0/2734708827 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e08003bf0 con 0x7f7e10108cf0 2024-09-21T03:45:07.598 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:07.594+0000 7f7e181ac640 1 -- 172.21.15.39:0/2734708827 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7e101a0660 con 0x7f7e10108cf0 2024-09-21T03:45:07.598 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:07.594+0000 7f7e06ffd640 1 -- 172.21.15.39:0/2734708827 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7e08003d90 con 0x7f7e10108cf0 2024-09-21T03:45:07.598 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:07.594+0000 7f7e06ffd640 1 -- 172.21.15.39:0/2734708827 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e0802fb20 con 0x7f7e10108cf0 2024-09-21T03:45:07.598 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:07.594+0000 7f7e181ac640 1 -- 172.21.15.39:0/2734708827 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7e10104fc0 con 0x7f7e10108cf0 2024-09-21T03:45:07.599 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:07.595+0000 7f7e181ac640 1 -- 172.21.15.39:0/2734708827 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7e10103ce0 con 0x7f7e10108cf0 2024-09-21T03:45:07.599 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:07.596+0000 7f7e06ffd640 1 -- 172.21.15.39:0/2734708827 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 22) ==== 154580+0+0 (secure 0 0 0) 0x7f7e08004580 con 0x7f7e10108cf0 2024-09-21T03:45:07.600 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:07.597+0000 7f7e06ffd640 1 --2- 172.21.15.39:0/2734708827 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f7dfc0b65f0 0x7f7dfc0b8b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:07.600 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:07.598+0000 7f7e15f48640 1 --2- 172.21.15.39:0/2734708827 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f7dfc0b65f0 0x7f7dfc0b8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:07.601 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:07.598+0000 7f7e06ffd640 1 -- 172.21.15.39:0/2734708827 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8484+0+0 (secure 0 0 0) 0x7f7e08108a20 con 0x7f7e10108cf0 2024-09-21T03:45:07.601 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:07.598+0000 7f7e15f48640 1 --2- 172.21.15.39:0/2734708827 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f7dfc0b65f0 0x7f7dfc0b8b00 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f7e0c00bbe0 tx=0x7f7e0c00b390 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:45:07.603 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:07.601+0000 7f7e06ffd640 1 -- 172.21.15.39:0/2734708827 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f7e080cf050 con 0x7f7e10108cf0 2024-09-21T03:45:07.798 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:07.795+0000 7f7e181ac640 1 -- 172.21.15.39:0/2734708827 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 10} v 0) -- 0x7f7e10102e70 con 0x7f7e10108cf0 2024-09-21T03:45:07.798 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:45:07 smithi039 ceph-mon[31211]: from='client.? 172.21.15.39:0/371407795' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 0} : dispatch 2024-09-21T03:45:07.799 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:07.796+0000 7f7e06ffd640 1 -- 172.21.15.39:0/2734708827 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 10}]=0 v0) ==== 75+0+13 (secure 0 0 0) 0x7f7e10102e70 con 0x7f7e10108cf0 2024-09-21T03:45:07.799 INFO:teuthology.orchestra.run.smithi039.stdout:279172874268 2024-09-21T03:45:07.801 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:07.798+0000 7f7e181ac640 1 -- 172.21.15.39:0/2734708827 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f7dfc0b65f0 msgr2=0x7f7dfc0b8b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:07.801 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:07.798+0000 7f7e181ac640 1 --2- 172.21.15.39:0/2734708827 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f7dfc0b65f0 0x7f7dfc0b8b00 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f7e0c00bbe0 tx=0x7f7e0c00b390 comp rx=0 tx=0).stop 2024-09-21T03:45:07.801 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:07.798+0000 7f7e181ac640 1 -- 172.21.15.39:0/2734708827 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7e10108cf0 msgr2=0x7f7e1019fca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:07.801 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:07.798+0000 7f7e181ac640 1 --2- 172.21.15.39:0/2734708827 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7e10108cf0 0x7f7e1019fca0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f7e08005cc0 tx=0x7f7e0802f380 comp rx=0 tx=0).stop 2024-09-21T03:45:07.801 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:07.798+0000 7f7e181ac640 1 -- 172.21.15.39:0/2734708827 shutdown_connections 2024-09-21T03:45:07.801 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:07.798+0000 7f7e181ac640 1 --2- 172.21.15.39:0/2734708827 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f7dfc0b65f0 0x7f7dfc0b8b00 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:07.801 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:07.798+0000 7f7e181ac640 1 --2- 172.21.15.39:0/2734708827 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7e10108cf0 0x7f7e1019fca0 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:07.801 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:07.798+0000 7f7e181ac640 1 --2- 172.21.15.39:0/2734708827 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7e101083a0 0x7f7e1019aa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:07.801 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:07.798+0000 7f7e181ac640 1 --2- 172.21.15.39:0/2734708827 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7e10102a60 0x7f7e1019a460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:07.801 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:07.798+0000 7f7e181ac640 1 -- 172.21.15.39:0/2734708827 >> 172.21.15.39:0/2734708827 conn(0x7f7e10079730 msgr2=0x7f7e1006bf00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:45:07.801 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:07.798+0000 7f7e181ac640 1 -- 172.21.15.39:0/2734708827 shutdown_connections 2024-09-21T03:45:07.802 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:07.798+0000 7f7e181ac640 1 -- 172.21.15.39:0/2734708827 wait complete. 2024-09-21T03:45:07.968 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:45:07 smithi161 ceph-mon[34533]: from='client.? 172.21.15.39:0/371407795' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 0} : dispatch 2024-09-21T03:45:07.974 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:45:07 smithi150 ceph-mon[34289]: from='client.? 172.21.15.39:0/371407795' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 0} : dispatch 2024-09-21T03:45:08.833 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:45:08 smithi039 ceph-mon[31211]: pgmap v298: 97 pgs: 97 active+clean; 647 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 8 op/s 2024-09-21T03:45:08.833 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:45:08 smithi039 ceph-mon[31211]: from='client.? 172.21.15.39:0/2734708827' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 10} : dispatch 2024-09-21T03:45:08.968 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:45:08 smithi161 ceph-mon[34533]: pgmap v298: 97 pgs: 97 active+clean; 647 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 8 op/s 2024-09-21T03:45:08.968 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:45:08 smithi161 ceph-mon[34533]: from='client.? 172.21.15.39:0/2734708827' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 10} : dispatch 2024-09-21T03:45:08.974 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:45:08 smithi150 ceph-mon[34289]: pgmap v298: 97 pgs: 97 active+clean; 647 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 8 op/s 2024-09-21T03:45:08.974 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:45:08 smithi150 ceph-mon[34289]: from='client.? 172.21.15.39:0/2734708827' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 10} : dispatch 2024-09-21T03:45:09.114 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:09.106+0000 7f764f6f8640 1 -- 172.21.15.39:0/2505678416 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f76481083d0 msgr2=0x7f76481087e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:09.115 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:09.106+0000 7f764f6f8640 1 --2- 172.21.15.39:0/2505678416 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f76481083d0 0x7f76481087e0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f763c009900 tx=0x7f763c02a410 comp rx=0 tx=0).stop 2024-09-21T03:45:09.115 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:09.106+0000 7f764f6f8640 1 -- 172.21.15.39:0/2505678416 shutdown_connections 2024-09-21T03:45:09.115 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:09.106+0000 7f764f6f8640 1 --2- 172.21.15.39:0/2505678416 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7648108d20 0x7f764810f550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:09.115 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:09.106+0000 7f764f6f8640 1 --2- 172.21.15.39:0/2505678416 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f76481083d0 0x7f76481087e0 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:09.115 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:09.106+0000 7f764f6f8640 1 --2- 172.21.15.39:0/2505678416 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7648102ae0 0x7f7648102ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:09.115 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:09.106+0000 7f764f6f8640 1 -- 172.21.15.39:0/2505678416 >> 172.21.15.39:0/2505678416 conn(0x7f76480797b0 msgr2=0x7f7648079bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:45:09.115 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:09.106+0000 7f764f6f8640 1 -- 172.21.15.39:0/2505678416 shutdown_connections 2024-09-21T03:45:09.115 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:09.106+0000 7f764f6f8640 1 -- 172.21.15.39:0/2505678416 wait complete. 2024-09-21T03:45:09.115 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:09.107+0000 7f764f6f8640 1 Processor -- start 2024-09-21T03:45:09.115 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:09.107+0000 7f764f6f8640 1 -- start start 2024-09-21T03:45:09.115 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:09.107+0000 7f764f6f8640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7648102ae0 0x7f764819b330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:09.116 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:09.107+0000 7f764f6f8640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f76481083d0 0x7f764819b870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:09.116 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:09.107+0000 7f764f6f8640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7648108d20 0x7f764819bdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:09.116 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:09.107+0000 7f764f6f8640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f76481129e0 con 0x7f76481083d0 2024-09-21T03:45:09.116 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:09.107+0000 7f764f6f8640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f7648112860 con 0x7f7648108d20 2024-09-21T03:45:09.116 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:09.107+0000 7f764f6f8640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f7648112b60 con 0x7f7648102ae0 2024-09-21T03:45:09.116 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:09.107+0000 7f764cc93640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f76481083d0 0x7f764819b870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:09.116 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:09.107+0000 7f764cc93640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f76481083d0 0x7f764819b870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:34814/0 (socket says 172.21.15.39:34814) 2024-09-21T03:45:09.116 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:09.107+0000 7f764cc93640 1 -- 172.21.15.39:0/1168470667 learned_addr learned my addr 172.21.15.39:0/1168470667 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:45:09.116 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:09.107+0000 7f764d494640 1 --2- 172.21.15.39:0/1168470667 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7648102ae0 0x7f764819b330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:09.116 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:09.107+0000 7f764dc95640 1 --2- 172.21.15.39:0/1168470667 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7648108d20 0x7f764819bdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:09.116 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:09.107+0000 7f764cc93640 1 -- 172.21.15.39:0/1168470667 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7648102ae0 msgr2=0x7f764819b330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:09.116 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:09.107+0000 7f764cc93640 1 --2- 172.21.15.39:0/1168470667 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7648102ae0 0x7f764819b330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:09.116 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:09.107+0000 7f764cc93640 1 -- 172.21.15.39:0/1168470667 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7648108d20 msgr2=0x7f764819bdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:09.117 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:09.107+0000 7f764cc93640 1 --2- 172.21.15.39:0/1168470667 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7648108d20 0x7f764819bdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:09.117 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:09.107+0000 7f764cc93640 1 -- 172.21.15.39:0/1168470667 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f76481a65f0 con 0x7f76481083d0 2024-09-21T03:45:09.117 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:09.108+0000 7f764cc93640 1 --2- 172.21.15.39:0/1168470667 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f76481083d0 0x7f764819b870 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f763c02a930 tx=0x7f763c004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:45:09.117 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:09.108+0000 7f76467fc640 1 -- 172.21.15.39:0/1168470667 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f763c03f030 con 0x7f76481083d0 2024-09-21T03:45:09.117 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:09.108+0000 7f764f6f8640 1 -- 172.21.15.39:0/1168470667 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f76481a6880 con 0x7f76481083d0 2024-09-21T03:45:09.117 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:09.108+0000 7f76467fc640 1 -- 172.21.15.39:0/1168470667 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f763c003aa0 con 0x7f76481083d0 2024-09-21T03:45:09.117 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:09.108+0000 7f76467fc640 1 -- 172.21.15.39:0/1168470667 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f763c035070 con 0x7f76481083d0 2024-09-21T03:45:09.117 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:09.108+0000 7f764f6f8640 1 -- 172.21.15.39:0/1168470667 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f76481a6d60 con 0x7f76481083d0 2024-09-21T03:45:09.117 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:09.109+0000 7f764f6f8640 1 -- 172.21.15.39:0/1168470667 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7648102ef0 con 0x7f76481083d0 2024-09-21T03:45:09.117 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:09.111+0000 7f76467fc640 1 -- 172.21.15.39:0/1168470667 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 22) ==== 154580+0+0 (secure 0 0 0) 0x7f763c042030 con 0x7f76481083d0 2024-09-21T03:45:09.117 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:09.111+0000 7f76467fc640 1 --2- 172.21.15.39:0/1168470667 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f76300b65f0 0x7f76300b8b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:09.117 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:09.112+0000 7f764d494640 1 --2- 172.21.15.39:0/1168470667 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f76300b65f0 0x7f76300b8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:09.117 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:09.112+0000 7f76467fc640 1 -- 172.21.15.39:0/1168470667 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8484+0+0 (secure 0 0 0) 0x7f763c108120 con 0x7f76481083d0 2024-09-21T03:45:09.118 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:09.112+0000 7f764d494640 1 --2- 172.21.15.39:0/1168470667 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f76300b65f0 0x7f76300b8b00 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f764806c3a0 tx=0x7f764000b410 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:45:09.118 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:09.115+0000 7f76467fc640 1 -- 172.21.15.39:0/1168470667 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f763c0ce750 con 0x7f76481083d0 2024-09-21T03:45:09.249 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/51116d5c-77ca-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:45:09.311 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:09.308+0000 7f764f6f8640 1 -- 172.21.15.39:0/1168470667 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7f764806bc50 con 0x7f76481083d0 2024-09-21T03:45:09.311 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:09.308+0000 7f76467fc640 1 -- 172.21.15.39:0/1168470667 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f764806bc50 con 0x7f76481083d0 2024-09-21T03:45:09.311 INFO:teuthology.orchestra.run.smithi039.stdout:60129542219 2024-09-21T03:45:09.313 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:09.310+0000 7f764f6f8640 1 -- 172.21.15.39:0/1168470667 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f76300b65f0 msgr2=0x7f76300b8b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:09.313 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:09.310+0000 7f764f6f8640 1 --2- 172.21.15.39:0/1168470667 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f76300b65f0 0x7f76300b8b00 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f764806c3a0 tx=0x7f764000b410 comp rx=0 tx=0).stop 2024-09-21T03:45:09.313 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:09.310+0000 7f764f6f8640 1 -- 172.21.15.39:0/1168470667 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f76481083d0 msgr2=0x7f764819b870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:09.313 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:09.310+0000 7f764f6f8640 1 --2- 172.21.15.39:0/1168470667 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f76481083d0 0x7f764819b870 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f763c02a930 tx=0x7f763c004270 comp rx=0 tx=0).stop 2024-09-21T03:45:09.313 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:09.310+0000 7f764f6f8640 1 -- 172.21.15.39:0/1168470667 shutdown_connections 2024-09-21T03:45:09.313 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:09.310+0000 7f764f6f8640 1 --2- 172.21.15.39:0/1168470667 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f76300b65f0 0x7f76300b8b00 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:09.314 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:09.310+0000 7f764f6f8640 1 --2- 172.21.15.39:0/1168470667 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7648108d20 0x7f764819bdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:09.314 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:09.310+0000 7f764f6f8640 1 --2- 172.21.15.39:0/1168470667 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f76481083d0 0x7f764819b870 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:09.314 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:09.310+0000 7f764f6f8640 1 --2- 172.21.15.39:0/1168470667 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7648102ae0 0x7f764819b330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:09.314 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:09.310+0000 7f764f6f8640 1 -- 172.21.15.39:0/1168470667 >> 172.21.15.39:0/1168470667 conn(0x7f76480797b0 msgr2=0x7f76481064d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:45:09.314 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:09.310+0000 7f764f6f8640 1 -- 172.21.15.39:0/1168470667 shutdown_connections 2024-09-21T03:45:09.314 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:09.311+0000 7f764f6f8640 1 -- 172.21.15.39:0/1168470667 wait complete. 2024-09-21T03:45:09.578 INFO:tasks.cephadm.ceph_manager.ceph:need seq 38654705742 got 38654705745 for osd.0 2024-09-21T03:45:09.578 DEBUG:teuthology.parallel:result is None 2024-09-21T03:45:09.834 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:45:09 smithi039 ceph-mon[31211]: from='client.? 172.21.15.39:0/1168470667' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 1} : dispatch 2024-09-21T03:45:09.834 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:45:09 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:45:09.968 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:45:09 smithi161 ceph-mon[34533]: from='client.? 172.21.15.39:0/1168470667' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 1} : dispatch 2024-09-21T03:45:09.968 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:45:09 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:45:09.974 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:45:09 smithi150 ceph-mon[34289]: from='client.? 172.21.15.39:0/1168470667' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 1} : dispatch 2024-09-21T03:45:09.974 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:45:09 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:45:10.167 INFO:tasks.cephadm.ceph_manager.ceph:need seq 60129542218 got 60129542219 for osd.1 2024-09-21T03:45:10.168 DEBUG:teuthology.parallel:result is None 2024-09-21T03:45:10.778 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:10.763+0000 7f74c8c60640 1 -- 172.21.15.39:0/326152029 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f74c4103bb0 msgr2=0x7f74c4103fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:10.778 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:10.763+0000 7f74c8c60640 1 --2- 172.21.15.39:0/326152029 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f74c4103bb0 0x7f74c4103fc0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f74b4009900 tx=0x7f74b402a410 comp rx=0 tx=0).stop 2024-09-21T03:45:10.778 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:10.763+0000 7f74c8c60640 1 -- 172.21.15.39:0/326152029 shutdown_connections 2024-09-21T03:45:10.778 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:10.763+0000 7f74c8c60640 1 --2- 172.21.15.39:0/326152029 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f74c4109e40 0x7f74c4110670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:10.778 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:10.763+0000 7f74c8c60640 1 --2- 172.21.15.39:0/326152029 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f74c41094f0 0x7f74c4109900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:10.778 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:10.763+0000 7f74c8c60640 1 --2- 172.21.15.39:0/326152029 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f74c4103bb0 0x7f74c4103fc0 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:10.778 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:10.763+0000 7f74c8c60640 1 -- 172.21.15.39:0/326152029 >> 172.21.15.39:0/326152029 conn(0x7f74c40ff600 msgr2=0x7f74c4101a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:45:10.778 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:10.763+0000 7f74c8c60640 1 -- 172.21.15.39:0/326152029 shutdown_connections 2024-09-21T03:45:10.779 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:10.763+0000 7f74c8c60640 1 -- 172.21.15.39:0/326152029 wait complete. 2024-09-21T03:45:10.779 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:10.764+0000 7f74c8c60640 1 Processor -- start 2024-09-21T03:45:10.779 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:10.764+0000 7f74c8c60640 1 -- start start 2024-09-21T03:45:10.779 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:10.765+0000 7f74c8c60640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f74c4103bb0 0x7f74c419a4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:10.779 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:10.765+0000 7f74c8c60640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f74c41094f0 0x7f74c419aa90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:10.779 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:10.765+0000 7f74c8c60640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f74c4109e40 0x7f74c419fd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:10.779 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:10.765+0000 7f74c8c60640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f74c4113b00 con 0x7f74c41094f0 2024-09-21T03:45:10.779 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:10.765+0000 7f74c8c60640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f74c4113980 con 0x7f74c4103bb0 2024-09-21T03:45:10.779 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:10.765+0000 7f74c8c60640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f74c4113c80 con 0x7f74c4109e40 2024-09-21T03:45:10.779 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:10.765+0000 7f74c1d9b640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f74c41094f0 0x7f74c419aa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:10.779 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:10.765+0000 7f74c1d9b640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f74c41094f0 0x7f74c419aa90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:34824/0 (socket says 172.21.15.39:34824) 2024-09-21T03:45:10.779 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:10.765+0000 7f74c1d9b640 1 -- 172.21.15.39:0/861260342 learned_addr learned my addr 172.21.15.39:0/861260342 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:45:10.779 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:10.765+0000 7f74c259c640 1 --2- 172.21.15.39:0/861260342 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f74c4103bb0 0x7f74c419a4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:10.779 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:10.765+0000 7f74c2d9d640 1 --2- 172.21.15.39:0/861260342 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f74c4109e40 0x7f74c419fd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:10.780 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:10.765+0000 7f74c1d9b640 1 -- 172.21.15.39:0/861260342 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f74c4109e40 msgr2=0x7f74c419fd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:10.780 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:10.765+0000 7f74c1d9b640 1 --2- 172.21.15.39:0/861260342 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f74c4109e40 0x7f74c419fd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:10.780 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:10.765+0000 7f74c1d9b640 1 -- 172.21.15.39:0/861260342 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f74c4103bb0 msgr2=0x7f74c419a4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:10.780 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:10.765+0000 7f74c1d9b640 1 --2- 172.21.15.39:0/861260342 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f74c4103bb0 0x7f74c419a4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:10.780 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:10.765+0000 7f74c1d9b640 1 -- 172.21.15.39:0/861260342 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f74c41a0460 con 0x7f74c41094f0 2024-09-21T03:45:10.780 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:10.765+0000 7f74c1d9b640 1 --2- 172.21.15.39:0/861260342 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f74c41094f0 0x7f74c419aa90 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f74b800d950 tx=0x7f74b800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:45:10.780 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:10.766+0000 7f74b37fe640 1 -- 172.21.15.39:0/861260342 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74b8017070 con 0x7f74c41094f0 2024-09-21T03:45:10.780 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:10.766+0000 7f74b37fe640 1 -- 172.21.15.39:0/861260342 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f74b8009c30 con 0x7f74c41094f0 2024-09-21T03:45:10.780 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:10.766+0000 7f74c8c60640 1 -- 172.21.15.39:0/861260342 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f74c41a0750 con 0x7f74c41094f0 2024-09-21T03:45:10.780 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:10.766+0000 7f74b37fe640 1 -- 172.21.15.39:0/861260342 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74b80047a0 con 0x7f74c41094f0 2024-09-21T03:45:10.780 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:10.766+0000 7f74c8c60640 1 -- 172.21.15.39:0/861260342 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f74c40755f0 con 0x7f74c41094f0 2024-09-21T03:45:10.780 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:10.767+0000 7f74c8c60640 1 -- 172.21.15.39:0/861260342 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f74c4103fc0 con 0x7f74c41094f0 2024-09-21T03:45:10.780 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:10.768+0000 7f74b37fe640 1 -- 172.21.15.39:0/861260342 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 22) ==== 154580+0+0 (secure 0 0 0) 0x7f74b8004020 con 0x7f74c41094f0 2024-09-21T03:45:10.780 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:10.770+0000 7f74b37fe640 1 --2- 172.21.15.39:0/861260342 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f74ac0b6670 0x7f74ac0b8b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:10.780 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:10.770+0000 7f74c259c640 1 --2- 172.21.15.39:0/861260342 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f74ac0b6670 0x7f74ac0b8b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:10.781 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:10.770+0000 7f74b37fe640 1 -- 172.21.15.39:0/861260342 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8484+0+0 (secure 0 0 0) 0x7f74b80e71e0 con 0x7f74c41094f0 2024-09-21T03:45:10.781 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:10.770+0000 7f74c259c640 1 --2- 172.21.15.39:0/861260342 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f74ac0b6670 0x7f74ac0b8b80 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f74b4031440 tx=0x7f74b4009560 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:45:10.781 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:10.774+0000 7f74b37fe640 1 -- 172.21.15.39:0/861260342 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f74b8020020 con 0x7f74c41094f0 2024-09-21T03:45:10.962 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:45:10 smithi039 ceph-mon[31211]: pgmap v299: 97 pgs: 97 active+clean; 647 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-09-21T03:45:10.967 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:45:10 smithi161 ceph-mon[34533]: pgmap v299: 97 pgs: 97 active+clean; 647 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-09-21T03:45:10.972 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:10.969+0000 7fa8412ed640 1 -- 172.21.15.39:0/3758048067 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa83c10de10 msgr2=0x7fa83c110300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:10.972 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:10.969+0000 7fa8412ed640 1 --2- 172.21.15.39:0/3758048067 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa83c10de10 0x7fa83c110300 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fa834009ca0 tx=0x7fa83402a8c0 comp rx=0 tx=0).stop 2024-09-21T03:45:10.972 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:10.969+0000 7fa8412ed640 1 -- 172.21.15.39:0/3758048067 shutdown_connections 2024-09-21T03:45:10.972 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:10.969+0000 7fa8412ed640 1 --2- 172.21.15.39:0/3758048067 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa83c10de10 0x7fa83c110300 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:10.972 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:10.969+0000 7fa8412ed640 1 --2- 172.21.15.39:0/3758048067 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa83c105300 0x7fa83c10d850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:10.972 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:10.969+0000 7fa8412ed640 1 --2- 172.21.15.39:0/3758048067 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa83c1049b0 0x7fa83c104dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:10.972 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:10.969+0000 7fa8412ed640 1 -- 172.21.15.39:0/3758048067 >> 172.21.15.39:0/3758048067 conn(0x7fa83c079730 msgr2=0x7fa83c079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:45:10.972 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:10.969+0000 7fa8412ed640 1 -- 172.21.15.39:0/3758048067 shutdown_connections 2024-09-21T03:45:10.972 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:10.969+0000 7fa8412ed640 1 -- 172.21.15.39:0/3758048067 wait complete. 2024-09-21T03:45:10.974 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:10.970+0000 7fa8412ed640 1 Processor -- start 2024-09-21T03:45:10.974 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:10.970+0000 7fa8412ed640 1 -- start start 2024-09-21T03:45:10.974 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:10.970+0000 7fa8412ed640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa83c1049b0 0x7fa83c19c6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:10.974 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:10.970+0000 7fa8412ed640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa83c105300 0x7fa83c19cc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:10.975 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:10.970+0000 7fa8412ed640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa83c10de10 0x7fa83c1a1f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:10.975 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:10.970+0000 7fa8412ed640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fa83c113a90 con 0x7fa83c1049b0 2024-09-21T03:45:10.975 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:10.970+0000 7fa8412ed640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fa83c113910 con 0x7fa83c10de10 2024-09-21T03:45:10.975 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:10.970+0000 7fa8412ed640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fa83c113c10 con 0x7fa83c105300 2024-09-21T03:45:10.975 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:10.970+0000 7fa83affd640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa83c1049b0 0x7fa83c19c6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:10.975 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:10.970+0000 7fa83affd640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa83c1049b0 0x7fa83c19c6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:34830/0 (socket says 172.21.15.39:34830) 2024-09-21T03:45:10.975 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:10.970+0000 7fa83affd640 1 -- 172.21.15.39:0/2053692059 learned_addr learned my addr 172.21.15.39:0/2053692059 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:45:10.975 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:10.970+0000 7fa83a7fc640 1 --2- 172.21.15.39:0/2053692059 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa83c105300 0x7fa83c19cc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:10.975 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:10.970+0000 7fa83b7fe640 1 --2- 172.21.15.39:0/2053692059 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa83c10de10 0x7fa83c1a1f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:10.975 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:10.971+0000 7fa83affd640 1 -- 172.21.15.39:0/2053692059 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa83c105300 msgr2=0x7fa83c19cc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:10.975 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:10.971+0000 7fa83affd640 1 --2- 172.21.15.39:0/2053692059 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa83c105300 0x7fa83c19cc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:10.975 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:10.971+0000 7fa83affd640 1 -- 172.21.15.39:0/2053692059 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa83c10de10 msgr2=0x7fa83c1a1f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:10.975 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:10.971+0000 7fa83affd640 1 --2- 172.21.15.39:0/2053692059 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa83c10de10 0x7fa83c1a1f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:10.976 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:10.971+0000 7fa83affd640 1 -- 172.21.15.39:0/2053692059 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa83c1a2630 con 0x7fa83c1049b0 2024-09-21T03:45:10.976 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:10.971+0000 7fa83affd640 1 --2- 172.21.15.39:0/2053692059 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa83c1049b0 0x7fa83c19c6c0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fa82c00e9c0 tx=0x7fa82c00ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:45:10.976 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:10.971+0000 7fa823fff640 1 -- 172.21.15.39:0/2053692059 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa82c00cd80 con 0x7fa83c1049b0 2024-09-21T03:45:10.976 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:10.971+0000 7fa823fff640 1 -- 172.21.15.39:0/2053692059 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa82c0035f0 con 0x7fa83c1049b0 2024-09-21T03:45:10.976 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:10.971+0000 7fa8412ed640 1 -- 172.21.15.39:0/2053692059 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa83c1a2920 con 0x7fa83c1049b0 2024-09-21T03:45:10.976 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:10.971+0000 7fa823fff640 1 -- 172.21.15.39:0/2053692059 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa82c010690 con 0x7fa83c1049b0 2024-09-21T03:45:10.976 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:10.971+0000 7fa8412ed640 1 -- 172.21.15.39:0/2053692059 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa83c1a9080 con 0x7fa83c1049b0 2024-09-21T03:45:10.977 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:45:10 smithi150 ceph-mon[34289]: pgmap v299: 97 pgs: 97 active+clean; 647 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-09-21T03:45:10.977 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:10.972+0000 7fa8412ed640 1 -- 172.21.15.39:0/2053692059 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa83c104dc0 con 0x7fa83c1049b0 2024-09-21T03:45:10.977 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:10.974+0000 7fa823fff640 1 -- 172.21.15.39:0/2053692059 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 22) ==== 154580+0+0 (secure 0 0 0) 0x7fa82c004000 con 0x7fa83c1049b0 2024-09-21T03:45:10.977 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:10.975+0000 7fa823fff640 1 --2- 172.21.15.39:0/2053692059 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7fa8240b6670 0x7fa8240b8b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:10.978 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:10.975+0000 7fa823fff640 1 -- 172.21.15.39:0/2053692059 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8484+0+0 (secure 0 0 0) 0x7fa82c0e7db0 con 0x7fa83c1049b0 2024-09-21T03:45:10.978 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:10.975+0000 7fa83a7fc640 1 --2- 172.21.15.39:0/2053692059 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7fa8240b6670 0x7fa8240b8b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:10.978 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:10.976+0000 7fa83a7fc640 1 --2- 172.21.15.39:0/2053692059 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7fa8240b6670 0x7fa8240b8b80 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fa83c10fa60 tx=0x7fa83000b410 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:45:10.979 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:10.976+0000 7f74c8c60640 1 -- 172.21.15.39:0/861260342 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 9} v 0) -- 0x7f74c410bc70 con 0x7f74c41094f0 2024-09-21T03:45:10.981 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:10.978+0000 7fa823fff640 1 -- 172.21.15.39:0/2053692059 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fa82c0ae3e0 con 0x7fa83c1049b0 2024-09-21T03:45:10.981 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:10.978+0000 7f74b37fe640 1 -- 172.21.15.39:0/861260342 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 9}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f74b80ad810 con 0x7f74c41094f0 2024-09-21T03:45:10.981 INFO:teuthology.orchestra.run.smithi039.stdout:257698037793 2024-09-21T03:45:10.984 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:10.981+0000 7f74c8c60640 1 -- 172.21.15.39:0/861260342 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f74ac0b6670 msgr2=0x7f74ac0b8b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:10.984 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:10.981+0000 7f74c8c60640 1 --2- 172.21.15.39:0/861260342 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f74ac0b6670 0x7f74ac0b8b80 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f74b4031440 tx=0x7f74b4009560 comp rx=0 tx=0).stop 2024-09-21T03:45:10.984 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:10.981+0000 7f74c8c60640 1 -- 172.21.15.39:0/861260342 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f74c41094f0 msgr2=0x7f74c419aa90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:10.984 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:10.981+0000 7f74c8c60640 1 --2- 172.21.15.39:0/861260342 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f74c41094f0 0x7f74c419aa90 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f74b800d950 tx=0x7f74b800de10 comp rx=0 tx=0).stop 2024-09-21T03:45:10.984 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:10.981+0000 7f74c8c60640 1 -- 172.21.15.39:0/861260342 shutdown_connections 2024-09-21T03:45:10.984 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:10.981+0000 7f74c8c60640 1 --2- 172.21.15.39:0/861260342 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f74ac0b6670 0x7f74ac0b8b80 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:10.984 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:10.981+0000 7f74c8c60640 1 --2- 172.21.15.39:0/861260342 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f74c4109e40 0x7f74c419fd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:10.984 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:10.981+0000 7f74c8c60640 1 --2- 172.21.15.39:0/861260342 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f74c41094f0 0x7f74c419aa90 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:10.984 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:10.981+0000 7f74c8c60640 1 --2- 172.21.15.39:0/861260342 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f74c4103bb0 0x7f74c419a4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:10.984 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:10.981+0000 7f74c8c60640 1 -- 172.21.15.39:0/861260342 >> 172.21.15.39:0/861260342 conn(0x7f74c40ff600 msgr2=0x7f74c4101a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:45:10.984 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:10.981+0000 7f74c8c60640 1 -- 172.21.15.39:0/861260342 shutdown_connections 2024-09-21T03:45:10.985 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:10.981+0000 7f74c8c60640 1 -- 172.21.15.39:0/861260342 wait complete. 2024-09-21T03:45:11.177 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:11.174+0000 7fa8412ed640 1 -- 172.21.15.39:0/2053692059 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7fa83c1a9430 con 0x7fa83c1049b0 2024-09-21T03:45:11.177 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:11.175+0000 7fa823fff640 1 -- 172.21.15.39:0/2053692059 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fa82c0b27c0 con 0x7fa83c1049b0 2024-09-21T03:45:11.177 INFO:teuthology.orchestra.run.smithi039.stdout:81604378695 2024-09-21T03:45:11.180 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:11.175+0000 7f9de3886640 1 -- 172.21.15.39:0/3512017825 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9ddc109e10 msgr2=0x7f9ddc110640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:11.180 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:11.175+0000 7f9de3886640 1 --2- 172.21.15.39:0/3512017825 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9ddc109e10 0x7f9ddc110640 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f9dd4009ca0 tx=0x7f9dd402a8c0 comp rx=0 tx=0).stop 2024-09-21T03:45:11.180 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:11.175+0000 7f9de3886640 1 -- 172.21.15.39:0/3512017825 shutdown_connections 2024-09-21T03:45:11.180 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:11.175+0000 7f9de3886640 1 --2- 172.21.15.39:0/3512017825 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9ddc109e10 0x7f9ddc110640 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:11.180 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:11.175+0000 7f9de3886640 1 --2- 172.21.15.39:0/3512017825 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9ddc1094c0 0x7f9ddc1098d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:11.180 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:11.175+0000 7f9de3886640 1 --2- 172.21.15.39:0/3512017825 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9ddc103b80 0x7f9ddc103f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:11.180 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:11.175+0000 7f9de3886640 1 -- 172.21.15.39:0/3512017825 >> 172.21.15.39:0/3512017825 conn(0x7f9ddc0ff610 msgr2=0x7f9ddc101a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:45:11.180 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:11.175+0000 7f9de3886640 1 -- 172.21.15.39:0/3512017825 shutdown_connections 2024-09-21T03:45:11.180 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:11.175+0000 7f9de3886640 1 -- 172.21.15.39:0/3512017825 wait complete. 2024-09-21T03:45:11.181 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:11.176+0000 7f9de3886640 1 Processor -- start 2024-09-21T03:45:11.181 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:11.176+0000 7f9de3886640 1 -- start start 2024-09-21T03:45:11.181 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:11.176+0000 7f9de3886640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9ddc103b80 0x7f9ddc19a410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:11.181 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:11.176+0000 7f9de3886640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9ddc1094c0 0x7f9ddc19a9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:11.181 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:11.176+0000 7f9de3886640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9ddc109e10 0x7f9ddc19fc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:11.181 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:11.176+0000 7f9de3886640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f9ddc113b20 con 0x7f9ddc103b80 2024-09-21T03:45:11.181 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:11.176+0000 7f9de3886640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f9ddc1139a0 con 0x7f9ddc1094c0 2024-09-21T03:45:11.181 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:11.176+0000 7f9de3886640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f9ddc113ca0 con 0x7f9ddc109e10 2024-09-21T03:45:11.181 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:11.176+0000 7f9de1622640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9ddc103b80 0x7f9ddc19a410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:11.181 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:11.176+0000 7f9de1622640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9ddc103b80 0x7f9ddc19a410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:34862/0 (socket says 172.21.15.39:34862) 2024-09-21T03:45:11.181 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:11.176+0000 7f9de1622640 1 -- 172.21.15.39:0/411035968 learned_addr learned my addr 172.21.15.39:0/411035968 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:45:11.181 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:11.176+0000 7f9de1e23640 1 --2- 172.21.15.39:0/411035968 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9ddc109e10 0x7f9ddc19fc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:11.181 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:11.176+0000 7f9de0e21640 1 --2- 172.21.15.39:0/411035968 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9ddc1094c0 0x7f9ddc19a9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:11.182 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:11.177+0000 7f9de1622640 1 -- 172.21.15.39:0/411035968 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9ddc109e10 msgr2=0x7f9ddc19fc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:11.182 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:11.177+0000 7f9de1622640 1 --2- 172.21.15.39:0/411035968 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9ddc109e10 0x7f9ddc19fc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:11.182 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:11.177+0000 7f9de1622640 1 -- 172.21.15.39:0/411035968 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9ddc1094c0 msgr2=0x7f9ddc19a9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:11.182 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:11.177+0000 7f9de1622640 1 --2- 172.21.15.39:0/411035968 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9ddc1094c0 0x7f9ddc19a9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:11.182 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:11.177+0000 7f9de1622640 1 -- 172.21.15.39:0/411035968 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ddc1a0380 con 0x7f9ddc103b80 2024-09-21T03:45:11.182 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:11.177+0000 7f9de0e21640 1 --2- 172.21.15.39:0/411035968 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9ddc1094c0 0x7f9ddc19a9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:45:11.182 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:11.177+0000 7f9de1622640 1 --2- 172.21.15.39:0/411035968 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9ddc103b80 0x7f9ddc19a410 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f9dd800d770 tx=0x7f9dd800dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:45:11.182 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:11.177+0000 7f9dd27fc640 1 -- 172.21.15.39:0/411035968 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9dd8017070 con 0x7f9ddc103b80 2024-09-21T03:45:11.182 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:11.177+0000 7f9de3886640 1 -- 172.21.15.39:0/411035968 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9ddc1a0670 con 0x7f9ddc103b80 2024-09-21T03:45:11.182 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:11.177+0000 7fa8412ed640 1 -- 172.21.15.39:0/2053692059 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7fa8240b6670 msgr2=0x7fa8240b8b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:11.182 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:11.177+0000 7fa8412ed640 1 --2- 172.21.15.39:0/2053692059 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7fa8240b6670 0x7fa8240b8b80 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fa83c10fa60 tx=0x7fa83000b410 comp rx=0 tx=0).stop 2024-09-21T03:45:11.183 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:11.177+0000 7fa8412ed640 1 -- 172.21.15.39:0/2053692059 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa83c1049b0 msgr2=0x7fa83c19c6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:11.183 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:11.177+0000 7fa8412ed640 1 --2- 172.21.15.39:0/2053692059 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa83c1049b0 0x7fa83c19c6c0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fa82c00e9c0 tx=0x7fa82c00ee80 comp rx=0 tx=0).stop 2024-09-21T03:45:11.183 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:11.177+0000 7f9de3886640 1 -- 172.21.15.39:0/411035968 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9ddc1a6dd0 con 0x7f9ddc103b80 2024-09-21T03:45:11.183 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:11.177+0000 7f9dd27fc640 1 -- 172.21.15.39:0/411035968 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9dd80054d0 con 0x7f9ddc103b80 2024-09-21T03:45:11.183 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:11.178+0000 7fa8412ed640 1 -- 172.21.15.39:0/2053692059 shutdown_connections 2024-09-21T03:45:11.184 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:11.178+0000 7fa8412ed640 1 --2- 172.21.15.39:0/2053692059 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7fa8240b6670 0x7fa8240b8b80 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:11.184 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:11.178+0000 7fa8412ed640 1 --2- 172.21.15.39:0/2053692059 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa83c10de10 0x7fa83c1a1f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:11.184 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:11.178+0000 7fa8412ed640 1 --2- 172.21.15.39:0/2053692059 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa83c105300 0x7fa83c19cc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:11.184 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:11.178+0000 7fa8412ed640 1 --2- 172.21.15.39:0/2053692059 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa83c1049b0 0x7fa83c19c6c0 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:11.184 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:11.178+0000 7fa8412ed640 1 -- 172.21.15.39:0/2053692059 >> 172.21.15.39:0/2053692059 conn(0x7fa83c079730 msgr2=0x7fa83c1036a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:45:11.184 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:11.178+0000 7f9dd27fc640 1 -- 172.21.15.39:0/411035968 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9dd8004c30 con 0x7f9ddc103b80 2024-09-21T03:45:11.185 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:11.178+0000 7fa8412ed640 1 -- 172.21.15.39:0/2053692059 shutdown_connections 2024-09-21T03:45:11.185 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:11.178+0000 7fa8412ed640 1 -- 172.21.15.39:0/2053692059 wait complete. 2024-09-21T03:45:11.185 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:11.179+0000 7f9de3886640 1 -- 172.21.15.39:0/411035968 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9dc00052c0 con 0x7f9ddc103b80 2024-09-21T03:45:11.185 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:11.181+0000 7f9dd27fc640 1 -- 172.21.15.39:0/411035968 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 22) ==== 154580+0+0 (secure 0 0 0) 0x7f9dd8009c30 con 0x7f9ddc103b80 2024-09-21T03:45:11.185 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:11.182+0000 7f9dd27fc640 1 --2- 172.21.15.39:0/411035968 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f9dc80b6670 0x7f9dc80b8b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:11.186 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:11.182+0000 7f9de0e21640 1 --2- 172.21.15.39:0/411035968 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f9dc80b6670 0x7f9dc80b8b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:11.186 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:11.182+0000 7f9dd27fc640 1 -- 172.21.15.39:0/411035968 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8484+0+0 (secure 0 0 0) 0x7f9dd80e7360 con 0x7f9ddc103b80 2024-09-21T03:45:11.186 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:11.182+0000 7f9de0e21640 1 --2- 172.21.15.39:0/411035968 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f9dc80b6670 0x7f9dc80b8b80 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f9ddc102ec0 tx=0x7f9dcc00b5f0 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:45:11.188 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:11.186+0000 7f9dd27fc640 1 -- 172.21.15.39:0/411035968 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f9dd80ada10 con 0x7f9ddc103b80 2024-09-21T03:45:11.383 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:11.380+0000 7f9de3886640 1 -- 172.21.15.39:0/411035968 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 8} v 0) -- 0x7f9dc0005880 con 0x7f9ddc103b80 2024-09-21T03:45:11.384 INFO:teuthology.orchestra.run.smithi039.stdout:231928234022 2024-09-21T03:45:11.384 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:11.380+0000 7f9dd27fc640 1 -- 172.21.15.39:0/411035968 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 8}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f9dd80b1df0 con 0x7f9ddc103b80 2024-09-21T03:45:11.386 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:11.383+0000 7f9de3886640 1 -- 172.21.15.39:0/411035968 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f9dc80b6670 msgr2=0x7f9dc80b8b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:11.386 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:11.383+0000 7f9de3886640 1 --2- 172.21.15.39:0/411035968 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f9dc80b6670 0x7f9dc80b8b80 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f9ddc102ec0 tx=0x7f9dcc00b5f0 comp rx=0 tx=0).stop 2024-09-21T03:45:11.386 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:11.383+0000 7f9de3886640 1 -- 172.21.15.39:0/411035968 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9ddc103b80 msgr2=0x7f9ddc19a410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:11.386 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:11.383+0000 7f9de3886640 1 --2- 172.21.15.39:0/411035968 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9ddc103b80 0x7f9ddc19a410 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f9dd800d770 tx=0x7f9dd800dc30 comp rx=0 tx=0).stop 2024-09-21T03:45:11.386 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:11.383+0000 7f9de3886640 1 -- 172.21.15.39:0/411035968 shutdown_connections 2024-09-21T03:45:11.386 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:11.383+0000 7f9de3886640 1 --2- 172.21.15.39:0/411035968 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f9dc80b6670 0x7f9dc80b8b80 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:11.387 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:11.383+0000 7f9de3886640 1 --2- 172.21.15.39:0/411035968 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9ddc109e10 0x7f9ddc19fc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:11.387 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:11.383+0000 7f9de3886640 1 --2- 172.21.15.39:0/411035968 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9ddc1094c0 0x7f9ddc19a9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:11.387 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:11.383+0000 7f9de3886640 1 --2- 172.21.15.39:0/411035968 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9ddc103b80 0x7f9ddc19a410 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:11.387 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:11.383+0000 7f9de3886640 1 -- 172.21.15.39:0/411035968 >> 172.21.15.39:0/411035968 conn(0x7f9ddc0ff610 msgr2=0x7f9ddc1069f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:45:11.387 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:11.383+0000 7f9de3886640 1 -- 172.21.15.39:0/411035968 shutdown_connections 2024-09-21T03:45:11.387 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:11.383+0000 7f9de3886640 1 -- 172.21.15.39:0/411035968 wait complete. 2024-09-21T03:45:11.762 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:45:11 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:45:11.762 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:45:11 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:45:11.762 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:45:11 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:45:11.762 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:45:11 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:45:11.762 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:45:11 smithi039 ceph-mon[31211]: from='client.? 172.21.15.39:0/861260342' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 9} : dispatch 2024-09-21T03:45:11.762 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:45:11 smithi039 ceph-mon[31211]: from='client.? 172.21.15.39:0/2053692059' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 2} : dispatch 2024-09-21T03:45:11.762 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:45:11 smithi039 ceph-mon[31211]: from='client.? 172.21.15.39:0/411035968' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 8} : dispatch 2024-09-21T03:45:12.073 INFO:tasks.cephadm.ceph_manager.ceph:need seq 231928234019 got 231928234022 for osd.8 2024-09-21T03:45:12.074 DEBUG:teuthology.parallel:result is None 2024-09-21T03:45:12.218 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:45:11 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:45:12.218 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:45:11 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:45:12.218 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:45:11 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:45:12.218 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:45:11 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:45:12.218 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:45:11 smithi161 ceph-mon[34533]: from='client.? 172.21.15.39:0/861260342' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 9} : dispatch 2024-09-21T03:45:12.218 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:45:11 smithi161 ceph-mon[34533]: from='client.? 172.21.15.39:0/2053692059' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 2} : dispatch 2024-09-21T03:45:12.218 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:45:11 smithi161 ceph-mon[34533]: from='client.? 172.21.15.39:0/411035968' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 8} : dispatch 2024-09-21T03:45:12.224 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:45:11 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:45:12.225 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:45:11 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:45:12.225 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:45:11 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:45:12.225 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:45:11 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:45:12.225 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:45:11 smithi150 ceph-mon[34289]: from='client.? 172.21.15.39:0/861260342' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 9} : dispatch 2024-09-21T03:45:12.225 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:45:11 smithi150 ceph-mon[34289]: from='client.? 172.21.15.39:0/2053692059' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 2} : dispatch 2024-09-21T03:45:12.225 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:45:11 smithi150 ceph-mon[34289]: from='client.? 172.21.15.39:0/411035968' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 8} : dispatch 2024-09-21T03:45:12.615 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:12.605+0000 7f1183d12640 1 -- 172.21.15.39:0/4146418317 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f117c1083c0 msgr2=0x7f117c1087d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:12.615 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:12.605+0000 7f1183d12640 1 --2- 172.21.15.39:0/4146418317 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f117c1083c0 0x7f117c1087d0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f1170009900 tx=0x7f117002a410 comp rx=0 tx=0).stop 2024-09-21T03:45:12.615 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:12.605+0000 7f1183d12640 1 -- 172.21.15.39:0/4146418317 shutdown_connections 2024-09-21T03:45:12.616 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:12.605+0000 7f1183d12640 1 --2- 172.21.15.39:0/4146418317 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f117c108d10 0x7f117c10f540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:12.616 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:12.605+0000 7f1183d12640 1 --2- 172.21.15.39:0/4146418317 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f117c1083c0 0x7f117c1087d0 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:12.616 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:12.605+0000 7f1183d12640 1 --2- 172.21.15.39:0/4146418317 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f117c102a80 0x7f117c102e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:12.616 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:12.605+0000 7f1183d12640 1 -- 172.21.15.39:0/4146418317 >> 172.21.15.39:0/4146418317 conn(0x7f117c079740 msgr2=0x7f117c079b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:45:12.616 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:12.606+0000 7f1183d12640 1 -- 172.21.15.39:0/4146418317 shutdown_connections 2024-09-21T03:45:12.616 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:12.606+0000 7f1183d12640 1 -- 172.21.15.39:0/4146418317 wait complete. 2024-09-21T03:45:12.616 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:12.606+0000 7f1183d12640 1 Processor -- start 2024-09-21T03:45:12.616 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:12.606+0000 7f1183d12640 1 -- start start 2024-09-21T03:45:12.616 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:12.607+0000 7f1183d12640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f117c102a80 0x7f117c19a500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:12.616 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:12.607+0000 7f1183d12640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f117c1083c0 0x7f117c19aaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:12.616 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:12.607+0000 7f1183d12640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f117c108d10 0x7f117c19fd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:12.616 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:12.607+0000 7f1183d12640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f117c112980 con 0x7f117c102a80 2024-09-21T03:45:12.616 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:12.607+0000 7f1183d12640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f117c112800 con 0x7f117c1083c0 2024-09-21T03:45:12.616 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:12.607+0000 7f1183d12640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f117c112b00 con 0x7f117c108d10 2024-09-21T03:45:12.616 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:12.607+0000 7f1181aae640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f117c102a80 0x7f117c19a500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:12.617 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:12.607+0000 7f1181aae640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f117c102a80 0x7f117c19a500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:34880/0 (socket says 172.21.15.39:34880) 2024-09-21T03:45:12.617 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:12.607+0000 7f1181aae640 1 -- 172.21.15.39:0/1958070386 learned_addr learned my addr 172.21.15.39:0/1958070386 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:45:12.617 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:12.607+0000 7f11812ad640 1 --2- 172.21.15.39:0/1958070386 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f117c1083c0 0x7f117c19aaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:12.617 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:12.607+0000 7f11822af640 1 --2- 172.21.15.39:0/1958070386 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f117c108d10 0x7f117c19fd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:12.617 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:12.607+0000 7f1181aae640 1 -- 172.21.15.39:0/1958070386 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f117c108d10 msgr2=0x7f117c19fd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:12.617 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:12.607+0000 7f1181aae640 1 --2- 172.21.15.39:0/1958070386 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f117c108d10 0x7f117c19fd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:12.617 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:12.607+0000 7f1181aae640 1 -- 172.21.15.39:0/1958070386 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f117c1083c0 msgr2=0x7f117c19aaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:12.617 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:12.607+0000 7f1181aae640 1 --2- 172.21.15.39:0/1958070386 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f117c1083c0 0x7f117c19aaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:12.617 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:12.607+0000 7f1181aae640 1 -- 172.21.15.39:0/1958070386 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f117c1a0470 con 0x7f117c102a80 2024-09-21T03:45:12.617 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:12.607+0000 7f11822af640 1 --2- 172.21.15.39:0/1958070386 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f117c108d10 0x7f117c19fd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:45:12.617 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:12.607+0000 7f1181aae640 1 --2- 172.21.15.39:0/1958070386 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f117c102a80 0x7f117c19a500 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f117800d950 tx=0x7f117800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:45:12.617 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:12.607+0000 7f116effd640 1 -- 172.21.15.39:0/1958070386 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1178017070 con 0x7f117c102a80 2024-09-21T03:45:12.617 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:12.608+0000 7f116effd640 1 -- 172.21.15.39:0/1958070386 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1178009c30 con 0x7f117c102a80 2024-09-21T03:45:12.617 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:12.608+0000 7f1183d12640 1 -- 172.21.15.39:0/1958070386 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f117c1a0760 con 0x7f117c102a80 2024-09-21T03:45:12.618 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:12.608+0000 7f116effd640 1 -- 172.21.15.39:0/1958070386 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f117800f620 con 0x7f117c102a80 2024-09-21T03:45:12.618 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:12.608+0000 7f1183d12640 1 -- 172.21.15.39:0/1958070386 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f117c1a6ec0 con 0x7f117c102a80 2024-09-21T03:45:12.618 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:12.609+0000 7f1183d12640 1 -- 172.21.15.39:0/1958070386 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f117c103d00 con 0x7f117c102a80 2024-09-21T03:45:12.618 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:12.610+0000 7f116effd640 1 -- 172.21.15.39:0/1958070386 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 22) ==== 154580+0+0 (secure 0 0 0) 0x7f1178010460 con 0x7f117c102a80 2024-09-21T03:45:12.618 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:12.611+0000 7f116effd640 1 --2- 172.21.15.39:0/1958070386 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f11680b65f0 0x7f11680b8b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:12.618 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:12.611+0000 7f11812ad640 1 --2- 172.21.15.39:0/1958070386 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f11680b65f0 0x7f11680b8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:12.618 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:12.611+0000 7f116effd640 1 -- 172.21.15.39:0/1958070386 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8484+0+0 (secure 0 0 0) 0x7f11780e7270 con 0x7f117c102a80 2024-09-21T03:45:12.618 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:12.612+0000 7f11812ad640 1 --2- 172.21.15.39:0/1958070386 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f11680b65f0 0x7f11680b8b00 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f1170000c00 tx=0x7f1170031400 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:45:12.618 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:12.616+0000 7f116effd640 1 -- 172.21.15.39:0/1958070386 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f11780ad8a0 con 0x7f117c102a80 2024-09-21T03:45:12.808 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:12.806+0000 7f1183d12640 1 -- 172.21.15.39:0/1958070386 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) -- 0x7f117c10baf0 con 0x7f117c102a80 2024-09-21T03:45:12.809 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:12.806+0000 7f116effd640 1 -- 172.21.15.39:0/1958070386 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f11780d0020 con 0x7f117c102a80 2024-09-21T03:45:12.809 INFO:teuthology.orchestra.run.smithi039.stdout:184683593778 2024-09-21T03:45:12.812 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:12.809+0000 7f1183d12640 1 -- 172.21.15.39:0/1958070386 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f11680b65f0 msgr2=0x7f11680b8b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:12.812 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:12.809+0000 7f1183d12640 1 --2- 172.21.15.39:0/1958070386 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f11680b65f0 0x7f11680b8b00 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f1170000c00 tx=0x7f1170031400 comp rx=0 tx=0).stop 2024-09-21T03:45:12.813 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:12.809+0000 7f1183d12640 1 -- 172.21.15.39:0/1958070386 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f117c102a80 msgr2=0x7f117c19a500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:12.813 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:12.809+0000 7f1183d12640 1 --2- 172.21.15.39:0/1958070386 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f117c102a80 0x7f117c19a500 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f117800d950 tx=0x7f117800de10 comp rx=0 tx=0).stop 2024-09-21T03:45:12.813 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:12.809+0000 7f1183d12640 1 -- 172.21.15.39:0/1958070386 shutdown_connections 2024-09-21T03:45:12.813 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:12.809+0000 7f1183d12640 1 --2- 172.21.15.39:0/1958070386 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f11680b65f0 0x7f11680b8b00 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:12.813 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:12.809+0000 7f1183d12640 1 --2- 172.21.15.39:0/1958070386 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f117c108d10 0x7f117c19fd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:12.813 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:12.809+0000 7f1183d12640 1 --2- 172.21.15.39:0/1958070386 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f117c1083c0 0x7f117c19aaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:12.814 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:12.809+0000 7f1183d12640 1 --2- 172.21.15.39:0/1958070386 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f117c102a80 0x7f117c19a500 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:12.814 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:12.809+0000 7f1183d12640 1 -- 172.21.15.39:0/1958070386 >> 172.21.15.39:0/1958070386 conn(0x7f117c079740 msgr2=0x7f117c06bfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:45:12.814 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:12.809+0000 7f1183d12640 1 -- 172.21.15.39:0/1958070386 shutdown_connections 2024-09-21T03:45:12.814 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:12.809+0000 7f1183d12640 1 -- 172.21.15.39:0/1958070386 wait complete. 2024-09-21T03:45:13.083 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:45:12 smithi039 ceph-mon[31211]: pgmap v300: 97 pgs: 97 active+clean; 647 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-21T03:45:13.217 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:45:12 smithi161 ceph-mon[34533]: pgmap v300: 97 pgs: 97 active+clean; 647 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-21T03:45:13.224 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:45:12 smithi150 ceph-mon[34289]: pgmap v300: 97 pgs: 97 active+clean; 647 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-21T03:45:13.288 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:13.286+0000 7fe074802640 1 -- 172.21.15.39:0/2774298852 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe070108b10 msgr2=0x7fe07010f340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:13.289 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:13.286+0000 7fe074802640 1 --2- 172.21.15.39:0/2774298852 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe070108b10 0x7fe07010f340 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fe068009ca0 tx=0x7fe06802a8e0 comp rx=0 tx=0).stop 2024-09-21T03:45:13.289 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:13.286+0000 7fe074802640 1 -- 172.21.15.39:0/2774298852 shutdown_connections 2024-09-21T03:45:13.289 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:13.286+0000 7fe074802640 1 --2- 172.21.15.39:0/2774298852 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe070108b10 0x7fe07010f340 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:13.289 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:13.286+0000 7fe074802640 1 --2- 172.21.15.39:0/2774298852 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe0701081c0 0x7fe0701085d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:13.289 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:13.286+0000 7fe074802640 1 --2- 172.21.15.39:0/2774298852 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe070102880 0x7fe070102c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:13.290 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:13.286+0000 7fe074802640 1 -- 172.21.15.39:0/2774298852 >> 172.21.15.39:0/2774298852 conn(0x7fe0700fe2d0 msgr2=0x7fe070100720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:45:13.290 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:13.286+0000 7fe074802640 1 -- 172.21.15.39:0/2774298852 shutdown_connections 2024-09-21T03:45:13.290 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:13.286+0000 7fe074802640 1 -- 172.21.15.39:0/2774298852 wait complete. 2024-09-21T03:45:13.291 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:13.287+0000 7fe074802640 1 Processor -- start 2024-09-21T03:45:13.292 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:13.287+0000 7fe074802640 1 -- start start 2024-09-21T03:45:13.292 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:13.287+0000 7fe074802640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe070102880 0x7fe07019b100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:13.292 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:13.287+0000 7fe074802640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe0701081c0 0x7fe07019b640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:13.292 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:13.287+0000 7fe074802640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe070108b10 0x7fe07019bb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:13.292 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:13.287+0000 7fe074802640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fe070112830 con 0x7fe070102880 2024-09-21T03:45:13.292 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:13.287+0000 7fe074802640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fe0701126b0 con 0x7fe070108b10 2024-09-21T03:45:13.292 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:13.287+0000 7fe074802640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fe0701129b0 con 0x7fe0701081c0 2024-09-21T03:45:13.292 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:13.287+0000 7fe06ed9d640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe070102880 0x7fe07019b100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:13.293 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:13.287+0000 7fe06ed9d640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe070102880 0x7fe07019b100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:34898/0 (socket says 172.21.15.39:34898) 2024-09-21T03:45:13.293 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:13.287+0000 7fe06ed9d640 1 -- 172.21.15.39:0/1329533456 learned_addr learned my addr 172.21.15.39:0/1329533456 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:45:13.293 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:13.287+0000 7fe06e59c640 1 --2- 172.21.15.39:0/1329533456 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe0701081c0 0x7fe07019b640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:13.293 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:13.288+0000 7fe06f59e640 1 --2- 172.21.15.39:0/1329533456 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe070108b10 0x7fe07019bb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:13.293 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:13.288+0000 7fe06ed9d640 1 -- 172.21.15.39:0/1329533456 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe0701081c0 msgr2=0x7fe07019b640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:13.293 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:13.288+0000 7fe06ed9d640 1 --2- 172.21.15.39:0/1329533456 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe0701081c0 0x7fe07019b640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:13.293 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:13.288+0000 7fe06ed9d640 1 -- 172.21.15.39:0/1329533456 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe070108b10 msgr2=0x7fe07019bb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:13.293 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:13.288+0000 7fe06ed9d640 1 --2- 172.21.15.39:0/1329533456 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe070108b10 0x7fe07019bb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:13.293 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:13.288+0000 7fe06ed9d640 1 -- 172.21.15.39:0/1329533456 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe0701a6390 con 0x7fe070102880 2024-09-21T03:45:13.293 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:13.288+0000 7fe06f59e640 1 --2- 172.21.15.39:0/1329533456 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe070108b10 0x7fe07019bb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:45:13.293 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:13.288+0000 7fe06ed9d640 1 --2- 172.21.15.39:0/1329533456 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe070102880 0x7fe07019b100 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fe06000d770 tx=0x7fe06000db70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:45:13.294 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:13.288+0000 7fe057fff640 1 -- 172.21.15.39:0/1329533456 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe060017070 con 0x7fe070102880 2024-09-21T03:45:13.294 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:13.288+0000 7fe057fff640 1 -- 172.21.15.39:0/1329533456 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe060009820 con 0x7fe070102880 2024-09-21T03:45:13.294 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:13.288+0000 7fe074802640 1 -- 172.21.15.39:0/1329533456 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe0701a6680 con 0x7fe070102880 2024-09-21T03:45:13.294 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:13.288+0000 7fe057fff640 1 -- 172.21.15.39:0/1329533456 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe060004bc0 con 0x7fe070102880 2024-09-21T03:45:13.294 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:13.288+0000 7fe074802640 1 -- 172.21.15.39:0/1329533456 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe0701a6bf0 con 0x7fe070102880 2024-09-21T03:45:13.295 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:13.290+0000 7fe074802640 1 -- 172.21.15.39:0/1329533456 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe070102c90 con 0x7fe070102880 2024-09-21T03:45:13.295 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:13.291+0000 7fe057fff640 1 -- 172.21.15.39:0/1329533456 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 22) ==== 154580+0+0 (secure 0 0 0) 0x7fe0600099d0 con 0x7fe070102880 2024-09-21T03:45:13.295 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:13.292+0000 7fe057fff640 1 --2- 172.21.15.39:0/1329533456 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7fe0580b6620 0x7fe0580b8b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:13.295 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:13.292+0000 7fe06e59c640 1 --2- 172.21.15.39:0/1329533456 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7fe0580b6620 0x7fe0580b8b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:13.295 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:13.292+0000 7fe057fff640 1 -- 172.21.15.39:0/1329533456 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8484+0+0 (secure 0 0 0) 0x7fe0600e70d0 con 0x7fe070102880 2024-09-21T03:45:13.296 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:13.292+0000 7fe06e59c640 1 --2- 172.21.15.39:0/1329533456 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7fe0580b6620 0x7fe0580b8b30 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fe070105c60 tx=0x7fe064009110 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:45:13.298 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:13.296+0000 7fe057fff640 1 -- 172.21.15.39:0/1329533456 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fe0600ad700 con 0x7fe070102880 2024-09-21T03:45:13.489 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:13.486+0000 7fe074802640 1 -- 172.21.15.39:0/1329533456 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 11} v 0) -- 0x7fe07002e640 con 0x7fe070102880 2024-09-21T03:45:13.489 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:13.487+0000 7fe057fff640 1 -- 172.21.15.39:0/1329533456 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 11}]=0 v0) ==== 75+0+13 (secure 0 0 0) 0x7fe0600b1ae0 con 0x7fe070102880 2024-09-21T03:45:13.490 INFO:teuthology.orchestra.run.smithi039.stdout:304942678041 2024-09-21T03:45:13.492 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:13.489+0000 7fe074802640 1 -- 172.21.15.39:0/1329533456 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7fe0580b6620 msgr2=0x7fe0580b8b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:13.492 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:13.489+0000 7fe074802640 1 --2- 172.21.15.39:0/1329533456 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7fe0580b6620 0x7fe0580b8b30 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fe070105c60 tx=0x7fe064009110 comp rx=0 tx=0).stop 2024-09-21T03:45:13.492 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:13.489+0000 7fe074802640 1 -- 172.21.15.39:0/1329533456 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe070102880 msgr2=0x7fe07019b100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:13.492 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:13.489+0000 7fe074802640 1 --2- 172.21.15.39:0/1329533456 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe070102880 0x7fe07019b100 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fe06000d770 tx=0x7fe06000db70 comp rx=0 tx=0).stop 2024-09-21T03:45:13.492 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:13.489+0000 7fe074802640 1 -- 172.21.15.39:0/1329533456 shutdown_connections 2024-09-21T03:45:13.492 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:13.489+0000 7fe074802640 1 --2- 172.21.15.39:0/1329533456 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7fe0580b6620 0x7fe0580b8b30 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:13.493 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:13.489+0000 7fe074802640 1 --2- 172.21.15.39:0/1329533456 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe070108b10 0x7fe07019bb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:13.493 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:13.489+0000 7fe074802640 1 --2- 172.21.15.39:0/1329533456 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe0701081c0 0x7fe07019b640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:13.493 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:13.489+0000 7fe074802640 1 --2- 172.21.15.39:0/1329533456 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe070102880 0x7fe07019b100 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:13.493 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:13.489+0000 7fe074802640 1 -- 172.21.15.39:0/1329533456 >> 172.21.15.39:0/1329533456 conn(0x7fe0700fe2d0 msgr2=0x7fe0701006f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:45:13.493 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:13.489+0000 7fe074802640 1 -- 172.21.15.39:0/1329533456 shutdown_connections 2024-09-21T03:45:13.493 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:13.489+0000 7fe074802640 1 -- 172.21.15.39:0/1329533456 wait complete. 2024-09-21T03:45:13.965 INFO:tasks.cephadm.ceph_manager.ceph:need seq 184683593775 got 184683593778 for osd.6 2024-09-21T03:45:13.965 DEBUG:teuthology.parallel:result is None 2024-09-21T03:45:14.078 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:45:13 smithi039 ceph-mon[31211]: from='client.? 172.21.15.39:0/1958070386' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 6} : dispatch 2024-09-21T03:45:14.078 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:45:13 smithi039 ceph-mon[31211]: from='client.? 172.21.15.39:0/1329533456' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 11} : dispatch 2024-09-21T03:45:14.079 INFO:tasks.cephadm.ceph_manager.ceph:need seq 257698037790 got 257698037793 for osd.9 2024-09-21T03:45:14.079 DEBUG:teuthology.parallel:result is None 2024-09-21T03:45:14.218 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:45:13 smithi161 ceph-mon[34533]: from='client.? 172.21.15.39:0/1958070386' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 6} : dispatch 2024-09-21T03:45:14.218 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:45:13 smithi161 ceph-mon[34533]: from='client.? 172.21.15.39:0/1329533456' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 11} : dispatch 2024-09-21T03:45:14.224 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:45:13 smithi150 ceph-mon[34289]: from='client.? 172.21.15.39:0/1958070386' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 6} : dispatch 2024-09-21T03:45:14.225 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:45:13 smithi150 ceph-mon[34289]: from='client.? 172.21.15.39:0/1329533456' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 11} : dispatch 2024-09-21T03:45:14.461 INFO:tasks.cephadm.ceph_manager.ceph:need seq 279172874267 got 279172874268 for osd.10 2024-09-21T03:45:14.461 DEBUG:teuthology.parallel:result is None 2024-09-21T03:45:14.843 INFO:tasks.cephadm.ceph_manager.ceph:need seq 81604378694 got 81604378695 for osd.2 2024-09-21T03:45:14.843 DEBUG:teuthology.parallel:result is None 2024-09-21T03:45:15.083 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:45:14 smithi039 ceph-mon[31211]: pgmap v301: 97 pgs: 97 active+clean; 647 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-09-21T03:45:15.217 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:45:14 smithi161 ceph-mon[34533]: pgmap v301: 97 pgs: 97 active+clean; 647 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-09-21T03:45:15.224 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:45:14 smithi150 ceph-mon[34289]: pgmap v301: 97 pgs: 97 active+clean; 647 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-09-21T03:45:15.229 INFO:tasks.cephadm.ceph_manager.ceph:need seq 304942678038 got 304942678041 for osd.11 2024-09-21T03:45:15.229 DEBUG:teuthology.parallel:result is None 2024-09-21T03:45:16.137 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:16.134+0000 7ff328834640 1 -- 172.21.15.39:0/1817584074 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff3241083b0 msgr2=0x7ff3241087c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:16.137 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:16.134+0000 7ff328834640 1 --2- 172.21.15.39:0/1817584074 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff3241083b0 0x7ff3241087c0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7ff318009900 tx=0x7ff31802a410 comp rx=0 tx=0).stop 2024-09-21T03:45:16.137 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:16.134+0000 7ff328834640 1 -- 172.21.15.39:0/1817584074 shutdown_connections 2024-09-21T03:45:16.137 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:16.134+0000 7ff328834640 1 --2- 172.21.15.39:0/1817584074 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff324108d00 0x7ff32410f530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:16.138 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:16.134+0000 7ff328834640 1 --2- 172.21.15.39:0/1817584074 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff3241083b0 0x7ff3241087c0 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:16.138 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:16.134+0000 7ff328834640 1 --2- 172.21.15.39:0/1817584074 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff324102a70 0x7ff324102e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:16.138 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:16.134+0000 7ff328834640 1 -- 172.21.15.39:0/1817584074 >> 172.21.15.39:0/1817584074 conn(0x7ff324079710 msgr2=0x7ff324079b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:45:16.138 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:16.134+0000 7ff328834640 1 -- 172.21.15.39:0/1817584074 shutdown_connections 2024-09-21T03:45:16.138 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:16.134+0000 7ff328834640 1 -- 172.21.15.39:0/1817584074 wait complete. 2024-09-21T03:45:16.140 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:16.135+0000 7ff328834640 1 Processor -- start 2024-09-21T03:45:16.140 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:16.135+0000 7ff328834640 1 -- start start 2024-09-21T03:45:16.140 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:16.135+0000 7ff328834640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff324102a70 0x7ff3240ff5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:16.140 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:16.135+0000 7ff328834640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff3241083b0 0x7ff3240ffb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:16.140 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:16.135+0000 7ff328834640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff324108d00 0x7ff324100070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:16.140 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:16.135+0000 7ff328834640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7ff3241129c0 con 0x7ff324102a70 2024-09-21T03:45:16.140 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:16.135+0000 7ff328834640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7ff324112840 con 0x7ff3241083b0 2024-09-21T03:45:16.140 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:16.135+0000 7ff328834640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7ff324112b40 con 0x7ff324108d00 2024-09-21T03:45:16.141 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:16.136+0000 7ff32259c640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff324102a70 0x7ff3240ff5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:16.141 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:16.136+0000 7ff32259c640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff324102a70 0x7ff3240ff5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:34904/0 (socket says 172.21.15.39:34904) 2024-09-21T03:45:16.141 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:16.136+0000 7ff32259c640 1 -- 172.21.15.39:0/1261243227 learned_addr learned my addr 172.21.15.39:0/1261243227 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:45:16.141 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:16.136+0000 7ff321d9b640 1 --2- 172.21.15.39:0/1261243227 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff3241083b0 0x7ff3240ffb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:16.141 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:16.136+0000 7ff322d9d640 1 --2- 172.21.15.39:0/1261243227 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff324108d00 0x7ff324100070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:16.141 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:16.136+0000 7ff32259c640 1 -- 172.21.15.39:0/1261243227 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff324108d00 msgr2=0x7ff324100070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:16.141 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:16.136+0000 7ff32259c640 1 --2- 172.21.15.39:0/1261243227 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff324108d00 0x7ff324100070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:16.141 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:16.136+0000 7ff32259c640 1 -- 172.21.15.39:0/1261243227 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff3241083b0 msgr2=0x7ff3240ffb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:16.141 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:16.136+0000 7ff32259c640 1 --2- 172.21.15.39:0/1261243227 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff3241083b0 0x7ff3240ffb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:16.141 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:16.136+0000 7ff32259c640 1 -- 172.21.15.39:0/1261243227 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff3241a8730 con 0x7ff324102a70 2024-09-21T03:45:16.142 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:16.136+0000 7ff322d9d640 1 --2- 172.21.15.39:0/1261243227 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff324108d00 0x7ff324100070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:45:16.142 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:16.136+0000 7ff32259c640 1 --2- 172.21.15.39:0/1261243227 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff324102a70 0x7ff3240ff5f0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7ff31400c890 tx=0x7ff31400cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:45:16.142 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:16.136+0000 7ff3137fe640 1 -- 172.21.15.39:0/1261243227 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff314004220 con 0x7ff324102a70 2024-09-21T03:45:16.142 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:16.136+0000 7ff328834640 1 -- 172.21.15.39:0/1261243227 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff3241a8a20 con 0x7ff324102a70 2024-09-21T03:45:16.142 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:16.136+0000 7ff3137fe640 1 -- 172.21.15.39:0/1261243227 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff3140043c0 con 0x7ff324102a70 2024-09-21T03:45:16.142 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:16.136+0000 7ff3137fe640 1 -- 172.21.15.39:0/1261243227 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff314004a00 con 0x7ff324102a70 2024-09-21T03:45:16.142 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:16.137+0000 7ff328834640 1 -- 172.21.15.39:0/1261243227 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff3241a8f30 con 0x7ff324102a70 2024-09-21T03:45:16.142 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:16.137+0000 7ff328834640 1 -- 172.21.15.39:0/1261243227 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff324103cf0 con 0x7ff324102a70 2024-09-21T03:45:16.143 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:16.139+0000 7ff3137fe640 1 -- 172.21.15.39:0/1261243227 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 22) ==== 154580+0+0 (secure 0 0 0) 0x7ff3140079b0 con 0x7ff324102a70 2024-09-21T03:45:16.143 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:16.140+0000 7ff3137fe640 1 --2- 172.21.15.39:0/1261243227 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7ff30c0b65f0 0x7ff30c0b8b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:16.143 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:16.140+0000 7ff321d9b640 1 --2- 172.21.15.39:0/1261243227 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7ff30c0b65f0 0x7ff30c0b8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:16.143 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:16.140+0000 7ff3137fe640 1 -- 172.21.15.39:0/1261243227 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8484+0+0 (secure 0 0 0) 0x7ff3140e7780 con 0x7ff324102a70 2024-09-21T03:45:16.143 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:16.141+0000 7ff321d9b640 1 --2- 172.21.15.39:0/1261243227 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7ff30c0b65f0 0x7ff30c0b8b00 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7ff318009230 tx=0x7ff318031400 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:45:16.146 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:16.143+0000 7ff3137fe640 1 -- 172.21.15.39:0/1261243227 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7ff314014030 con 0x7ff324102a70 2024-09-21T03:45:16.272 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:16.269+0000 7fed360b1640 1 -- 172.21.15.39:0/4147973207 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fed30102aa0 msgr2=0x7fed30102eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:16.273 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:16.269+0000 7fed360b1640 1 --2- 172.21.15.39:0/4147973207 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fed30102aa0 0x7fed30102eb0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7fed20009900 tx=0x7fed2002a410 comp rx=0 tx=0).stop 2024-09-21T03:45:16.273 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:16.269+0000 7fed360b1640 1 -- 172.21.15.39:0/4147973207 shutdown_connections 2024-09-21T03:45:16.273 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:16.269+0000 7fed360b1640 1 --2- 172.21.15.39:0/4147973207 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fed30108d30 0x7fed3010f560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:16.273 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:16.269+0000 7fed360b1640 1 --2- 172.21.15.39:0/4147973207 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fed301083e0 0x7fed301087f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:16.273 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:16.269+0000 7fed360b1640 1 --2- 172.21.15.39:0/4147973207 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fed30102aa0 0x7fed30102eb0 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:16.273 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:16.269+0000 7fed360b1640 1 -- 172.21.15.39:0/4147973207 >> 172.21.15.39:0/4147973207 conn(0x7fed300797e0 msgr2=0x7fed30079bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:45:16.273 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:16.269+0000 7fed360b1640 1 -- 172.21.15.39:0/4147973207 shutdown_connections 2024-09-21T03:45:16.273 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:16.269+0000 7fed360b1640 1 -- 172.21.15.39:0/4147973207 wait complete. 2024-09-21T03:45:16.273 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:16.270+0000 7fed360b1640 1 Processor -- start 2024-09-21T03:45:16.273 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:16.270+0000 7fed360b1640 1 -- start start 2024-09-21T03:45:16.275 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:16.270+0000 7fed360b1640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fed30102aa0 0x7fed3019b290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:16.275 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:16.270+0000 7fed360b1640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fed301083e0 0x7fed3019b7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:16.275 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:16.270+0000 7fed360b1640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fed30108d30 0x7fed3019bd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:16.275 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:16.270+0000 7fed360b1640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fed301129a0 con 0x7fed30108d30 2024-09-21T03:45:16.275 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:16.270+0000 7fed360b1640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fed30112820 con 0x7fed301083e0 2024-09-21T03:45:16.275 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:16.270+0000 7fed360b1640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fed30112b20 con 0x7fed30102aa0 2024-09-21T03:45:16.275 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:16.271+0000 7fed2ffff640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fed30108d30 0x7fed3019bd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:16.275 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:16.271+0000 7fed2ffff640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fed30108d30 0x7fed3019bd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:34922/0 (socket says 172.21.15.39:34922) 2024-09-21T03:45:16.275 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:16.271+0000 7fed2ffff640 1 -- 172.21.15.39:0/3240937153 learned_addr learned my addr 172.21.15.39:0/3240937153 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:45:16.275 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:16.271+0000 7fed2f7fe640 1 --2- 172.21.15.39:0/3240937153 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fed30102aa0 0x7fed3019b290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:16.275 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:16.271+0000 7fed2effd640 1 --2- 172.21.15.39:0/3240937153 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fed301083e0 0x7fed3019b7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:16.275 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:16.271+0000 7fed2ffff640 1 -- 172.21.15.39:0/3240937153 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fed30102aa0 msgr2=0x7fed3019b290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:16.276 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:16.271+0000 7fed2ffff640 1 --2- 172.21.15.39:0/3240937153 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fed30102aa0 0x7fed3019b290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:16.276 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:16.271+0000 7fed2ffff640 1 -- 172.21.15.39:0/3240937153 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fed301083e0 msgr2=0x7fed3019b7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:16.276 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:16.271+0000 7fed2ffff640 1 --2- 172.21.15.39:0/3240937153 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fed301083e0 0x7fed3019b7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:16.276 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:16.271+0000 7fed2ffff640 1 -- 172.21.15.39:0/3240937153 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fed30199080 con 0x7fed30108d30 2024-09-21T03:45:16.276 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:16.271+0000 7fed2ffff640 1 --2- 172.21.15.39:0/3240937153 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fed30108d30 0x7fed3019bd10 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7fed2800eaa0 tx=0x7fed2800ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:45:16.276 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:16.272+0000 7fed2cff9640 1 -- 172.21.15.39:0/3240937153 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed2800cdb0 con 0x7fed30108d30 2024-09-21T03:45:16.276 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:16.272+0000 7fed360b1640 1 -- 172.21.15.39:0/3240937153 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fed30199370 con 0x7fed30108d30 2024-09-21T03:45:16.277 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:16.272+0000 7fed2cff9640 1 -- 172.21.15.39:0/3240937153 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fed28005020 con 0x7fed30108d30 2024-09-21T03:45:16.277 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:16.272+0000 7fed360b1640 1 -- 172.21.15.39:0/3240937153 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fed301998b0 con 0x7fed30108d30 2024-09-21T03:45:16.277 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:16.272+0000 7fed2cff9640 1 -- 172.21.15.39:0/3240937153 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed280116d0 con 0x7fed30108d30 2024-09-21T03:45:16.277 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:16.273+0000 7fed360b1640 1 -- 172.21.15.39:0/3240937153 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fed30102eb0 con 0x7fed30108d30 2024-09-21T03:45:16.278 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:16.275+0000 7fed2cff9640 1 -- 172.21.15.39:0/3240937153 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 22) ==== 154580+0+0 (secure 0 0 0) 0x7fed280051d0 con 0x7fed30108d30 2024-09-21T03:45:16.279 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:16.276+0000 7fed2cff9640 1 --2- 172.21.15.39:0/3240937153 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7fed180b66a0 0x7fed180b8bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:16.279 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:16.277+0000 7fed2f7fe640 1 --2- 172.21.15.39:0/3240937153 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7fed180b66a0 0x7fed180b8bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:16.279 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:16.277+0000 7fed2cff9640 1 -- 172.21.15.39:0/3240937153 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8484+0+0 (secure 0 0 0) 0x7fed280e7620 con 0x7fed30108d30 2024-09-21T03:45:16.280 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:16.277+0000 7fed2f7fe640 1 --2- 172.21.15.39:0/3240937153 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7fed180b66a0 0x7fed180b8bb0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fed20000c00 tx=0x7fed20031400 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:45:16.284 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:16.281+0000 7fed2cff9640 1 -- 172.21.15.39:0/3240937153 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fed28016030 con 0x7fed30108d30 2024-09-21T03:45:16.345 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:16.342+0000 7ff328834640 1 -- 172.21.15.39:0/1261243227 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) -- 0x7ff3241008f0 con 0x7ff324102a70 2024-09-21T03:45:16.345 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:16.343+0000 7ff3137fe640 1 -- 172.21.15.39:0/1261243227 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7ff3140addb0 con 0x7ff324102a70 2024-09-21T03:45:16.346 INFO:teuthology.orchestra.run.smithi039.stdout:210453397551 2024-09-21T03:45:16.348 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:16.345+0000 7ff328834640 1 -- 172.21.15.39:0/1261243227 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7ff30c0b65f0 msgr2=0x7ff30c0b8b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:16.348 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:16.345+0000 7ff328834640 1 --2- 172.21.15.39:0/1261243227 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7ff30c0b65f0 0x7ff30c0b8b00 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7ff318009230 tx=0x7ff318031400 comp rx=0 tx=0).stop 2024-09-21T03:45:16.348 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:16.345+0000 7ff328834640 1 -- 172.21.15.39:0/1261243227 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff324102a70 msgr2=0x7ff3240ff5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:16.348 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:16.345+0000 7ff328834640 1 --2- 172.21.15.39:0/1261243227 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff324102a70 0x7ff3240ff5f0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7ff31400c890 tx=0x7ff31400cd50 comp rx=0 tx=0).stop 2024-09-21T03:45:16.348 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:16.345+0000 7ff328834640 1 -- 172.21.15.39:0/1261243227 shutdown_connections 2024-09-21T03:45:16.348 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:16.345+0000 7ff328834640 1 --2- 172.21.15.39:0/1261243227 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7ff30c0b65f0 0x7ff30c0b8b00 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:16.348 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:16.345+0000 7ff328834640 1 --2- 172.21.15.39:0/1261243227 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff324108d00 0x7ff324100070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:16.348 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:16.345+0000 7ff328834640 1 --2- 172.21.15.39:0/1261243227 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff3241083b0 0x7ff3240ffb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:16.348 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:16.345+0000 7ff328834640 1 --2- 172.21.15.39:0/1261243227 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff324102a70 0x7ff3240ff5f0 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:16.348 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:16.345+0000 7ff328834640 1 -- 172.21.15.39:0/1261243227 >> 172.21.15.39:0/1261243227 conn(0x7ff324079710 msgr2=0x7ff3241019c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:45:16.349 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:16.345+0000 7ff328834640 1 -- 172.21.15.39:0/1261243227 shutdown_connections 2024-09-21T03:45:16.349 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:16.345+0000 7ff328834640 1 -- 172.21.15.39:0/1261243227 wait complete. 2024-09-21T03:45:16.486 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:16.483+0000 7fed360b1640 1 -- 172.21.15.39:0/3240937153 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) -- 0x7fed3019ae60 con 0x7fed30108d30 2024-09-21T03:45:16.486 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:16.484+0000 7fed2cff9640 1 -- 172.21.15.39:0/3240937153 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7fed280b2030 con 0x7fed30108d30 2024-09-21T03:45:16.486 INFO:teuthology.orchestra.run.smithi039.stdout:158913790009 2024-09-21T03:45:16.489 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:16.486+0000 7fed360b1640 1 -- 172.21.15.39:0/3240937153 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7fed180b66a0 msgr2=0x7fed180b8bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:16.489 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:16.486+0000 7fed360b1640 1 --2- 172.21.15.39:0/3240937153 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7fed180b66a0 0x7fed180b8bb0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fed20000c00 tx=0x7fed20031400 comp rx=0 tx=0).stop 2024-09-21T03:45:16.489 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:16.486+0000 7fed360b1640 1 -- 172.21.15.39:0/3240937153 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fed30108d30 msgr2=0x7fed3019bd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:16.489 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:16.486+0000 7fed360b1640 1 --2- 172.21.15.39:0/3240937153 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fed30108d30 0x7fed3019bd10 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7fed2800eaa0 tx=0x7fed2800ef60 comp rx=0 tx=0).stop 2024-09-21T03:45:16.489 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:16.486+0000 7fed360b1640 1 -- 172.21.15.39:0/3240937153 shutdown_connections 2024-09-21T03:45:16.489 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:16.486+0000 7fed360b1640 1 --2- 172.21.15.39:0/3240937153 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7fed180b66a0 0x7fed180b8bb0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:16.489 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:16.486+0000 7fed360b1640 1 --2- 172.21.15.39:0/3240937153 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fed30108d30 0x7fed3019bd10 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:16.489 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:16.486+0000 7fed360b1640 1 --2- 172.21.15.39:0/3240937153 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fed301083e0 0x7fed3019b7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:16.489 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:16.486+0000 7fed360b1640 1 --2- 172.21.15.39:0/3240937153 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fed30102aa0 0x7fed3019b290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:16.489 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:16.486+0000 7fed360b1640 1 -- 172.21.15.39:0/3240937153 >> 172.21.15.39:0/3240937153 conn(0x7fed300797e0 msgr2=0x7fed30101410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:45:16.490 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:16.486+0000 7fed360b1640 1 -- 172.21.15.39:0/3240937153 shutdown_connections 2024-09-21T03:45:16.490 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:16.486+0000 7fed360b1640 1 -- 172.21.15.39:0/3240937153 wait complete. 2024-09-21T03:45:16.544 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:16.541+0000 7fb0db62d640 1 -- 172.21.15.39:0/648004419 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb0d41083a0 msgr2=0x7fb0d41087b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:16.544 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:16.541+0000 7fb0db62d640 1 --2- 172.21.15.39:0/648004419 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb0d41083a0 0x7fb0d41087b0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7fb0d0009900 tx=0x7fb0d002a410 comp rx=0 tx=0).stop 2024-09-21T03:45:16.544 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:16.541+0000 7fb0db62d640 1 -- 172.21.15.39:0/648004419 shutdown_connections 2024-09-21T03:45:16.544 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:16.541+0000 7fb0db62d640 1 --2- 172.21.15.39:0/648004419 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb0d4108cf0 0x7fb0d410f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:16.544 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:16.541+0000 7fb0db62d640 1 --2- 172.21.15.39:0/648004419 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb0d41083a0 0x7fb0d41087b0 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:16.544 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:16.541+0000 7fb0db62d640 1 --2- 172.21.15.39:0/648004419 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb0d4102a60 0x7fb0d4102e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:16.544 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:16.541+0000 7fb0db62d640 1 -- 172.21.15.39:0/648004419 >> 172.21.15.39:0/648004419 conn(0x7fb0d4079730 msgr2=0x7fb0d4079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:45:16.544 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:16.541+0000 7fb0db62d640 1 -- 172.21.15.39:0/648004419 shutdown_connections 2024-09-21T03:45:16.544 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:16.541+0000 7fb0db62d640 1 -- 172.21.15.39:0/648004419 wait complete. 2024-09-21T03:45:16.546 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:16.542+0000 7fb0db62d640 1 Processor -- start 2024-09-21T03:45:16.546 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:16.542+0000 7fb0db62d640 1 -- start start 2024-09-21T03:45:16.546 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:16.542+0000 7fb0db62d640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb0d4102a60 0x7fb0d4199350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:16.546 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:16.542+0000 7fb0db62d640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb0d41083a0 0x7fb0d41998f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:16.546 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:16.542+0000 7fb0db62d640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb0d4108cf0 0x7fb0d419eb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:16.546 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:16.542+0000 7fb0db62d640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fb0d4112960 con 0x7fb0d4108cf0 2024-09-21T03:45:16.547 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:16.542+0000 7fb0db62d640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fb0d41127e0 con 0x7fb0d41083a0 2024-09-21T03:45:16.547 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:16.542+0000 7fb0db62d640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fb0d4112ae0 con 0x7fb0d4102a60 2024-09-21T03:45:16.547 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:16.542+0000 7fb0d9bca640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb0d4108cf0 0x7fb0d419eb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:16.547 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:16.542+0000 7fb0d9bca640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb0d4108cf0 0x7fb0d419eb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:34938/0 (socket says 172.21.15.39:34938) 2024-09-21T03:45:16.547 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:16.542+0000 7fb0d9bca640 1 -- 172.21.15.39:0/2751072947 learned_addr learned my addr 172.21.15.39:0/2751072947 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:45:16.547 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:16.542+0000 7fb0d93c9640 1 --2- 172.21.15.39:0/2751072947 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb0d4102a60 0x7fb0d4199350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:16.547 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:16.542+0000 7fb0d8bc8640 1 --2- 172.21.15.39:0/2751072947 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb0d41083a0 0x7fb0d41998f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:16.547 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:16.543+0000 7fb0d9bca640 1 -- 172.21.15.39:0/2751072947 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb0d4102a60 msgr2=0x7fb0d4199350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:16.547 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:16.543+0000 7fb0d9bca640 1 --2- 172.21.15.39:0/2751072947 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb0d4102a60 0x7fb0d4199350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:16.547 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:16.543+0000 7fb0d9bca640 1 -- 172.21.15.39:0/2751072947 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb0d41083a0 msgr2=0x7fb0d41998f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:16.547 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:16.543+0000 7fb0d9bca640 1 --2- 172.21.15.39:0/2751072947 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb0d41083a0 0x7fb0d41998f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:16.547 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:16.543+0000 7fb0d9bca640 1 -- 172.21.15.39:0/2751072947 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb0d419f2c0 con 0x7fb0d4108cf0 2024-09-21T03:45:16.547 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:16.543+0000 7fb0d93c9640 1 --2- 172.21.15.39:0/2751072947 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb0d4102a60 0x7fb0d4199350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:45:16.547 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:16.543+0000 7fb0d9bca640 1 --2- 172.21.15.39:0/2751072947 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb0d4108cf0 0x7fb0d419eb90 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7fb0cc00d750 tx=0x7fb0cc00dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:45:16.547 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:16.543+0000 7fb0c67fc640 1 -- 172.21.15.39:0/2751072947 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0cc017070 con 0x7fb0d4108cf0 2024-09-21T03:45:16.548 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:16.543+0000 7fb0c67fc640 1 -- 172.21.15.39:0/2751072947 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb0cc0054d0 con 0x7fb0d4108cf0 2024-09-21T03:45:16.548 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:16.543+0000 7fb0db62d640 1 -- 172.21.15.39:0/2751072947 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb0d419f5b0 con 0x7fb0d4108cf0 2024-09-21T03:45:16.548 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:16.543+0000 7fb0c67fc640 1 -- 172.21.15.39:0/2751072947 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0cc004c60 con 0x7fb0d4108cf0 2024-09-21T03:45:16.548 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:16.543+0000 7fb0db62d640 1 -- 172.21.15.39:0/2751072947 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb0d406ae60 con 0x7fb0d4108cf0 2024-09-21T03:45:16.548 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:16.545+0000 7fb0db62d640 1 -- 172.21.15.39:0/2751072947 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb0d4103ce0 con 0x7fb0d4108cf0 2024-09-21T03:45:16.548 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:16.546+0000 7fb0c67fc640 1 -- 172.21.15.39:0/2751072947 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 22) ==== 154580+0+0 (secure 0 0 0) 0x7fb0cc009bd0 con 0x7fb0d4108cf0 2024-09-21T03:45:16.550 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:16.547+0000 7fb0c67fc640 1 --2- 172.21.15.39:0/2751072947 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7fb0c00b8800 0x7fb0c00bad10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:16.550 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:16.547+0000 7fb0d93c9640 1 --2- 172.21.15.39:0/2751072947 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7fb0c00b8800 0x7fb0c00bad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:16.550 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:16.547+0000 7fb0c67fc640 1 -- 172.21.15.39:0/2751072947 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8484+0+0 (secure 0 0 0) 0x7fb0cc0e6ef0 con 0x7fb0d4108cf0 2024-09-21T03:45:16.550 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:16.548+0000 7fb0d93c9640 1 --2- 172.21.15.39:0/2751072947 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7fb0c00b8800 0x7fb0c00bad10 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fb0c8005c40 tx=0x7fb0c800a590 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:45:16.554 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:16.552+0000 7fb0c67fc640 1 -- 172.21.15.39:0/2751072947 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fb0cc0b1900 con 0x7fb0d4108cf0 2024-09-21T03:45:16.749 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:16.746+0000 7fb0db62d640 1 -- 172.21.15.39:0/2751072947 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) -- 0x7fb0d406b940 con 0x7fb0d4108cf0 2024-09-21T03:45:16.749 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:16.746+0000 7fb0c67fc640 1 -- 172.21.15.39:0/2751072947 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7fb0cc0b6840 con 0x7fb0d4108cf0 2024-09-21T03:45:16.749 INFO:teuthology.orchestra.run.smithi039.stdout:111669149763 2024-09-21T03:45:16.752 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:16.749+0000 7fb0db62d640 1 -- 172.21.15.39:0/2751072947 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7fb0c00b8800 msgr2=0x7fb0c00bad10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:16.752 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:16.749+0000 7fb0db62d640 1 --2- 172.21.15.39:0/2751072947 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7fb0c00b8800 0x7fb0c00bad10 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fb0c8005c40 tx=0x7fb0c800a590 comp rx=0 tx=0).stop 2024-09-21T03:45:16.752 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:16.749+0000 7fb0db62d640 1 -- 172.21.15.39:0/2751072947 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb0d4108cf0 msgr2=0x7fb0d419eb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:16.752 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:16.749+0000 7fb0db62d640 1 --2- 172.21.15.39:0/2751072947 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb0d4108cf0 0x7fb0d419eb90 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7fb0cc00d750 tx=0x7fb0cc00dc10 comp rx=0 tx=0).stop 2024-09-21T03:45:16.752 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:16.749+0000 7fb0db62d640 1 -- 172.21.15.39:0/2751072947 shutdown_connections 2024-09-21T03:45:16.752 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:16.749+0000 7fb0db62d640 1 --2- 172.21.15.39:0/2751072947 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7fb0c00b8800 0x7fb0c00bad10 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:16.752 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:16.749+0000 7fb0db62d640 1 --2- 172.21.15.39:0/2751072947 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb0d4108cf0 0x7fb0d419eb90 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:16.752 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:16.749+0000 7fb0db62d640 1 --2- 172.21.15.39:0/2751072947 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb0d41083a0 0x7fb0d41998f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:16.752 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:16.749+0000 7fb0db62d640 1 --2- 172.21.15.39:0/2751072947 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb0d4102a60 0x7fb0d4199350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:16.753 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:16.749+0000 7fb0db62d640 1 -- 172.21.15.39:0/2751072947 >> 172.21.15.39:0/2751072947 conn(0x7fb0d4079730 msgr2=0x7fb0d41043d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:45:16.753 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:16.749+0000 7fb0db62d640 1 -- 172.21.15.39:0/2751072947 shutdown_connections 2024-09-21T03:45:16.753 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:16.749+0000 7fb0db62d640 1 -- 172.21.15.39:0/2751072947 wait complete. 2024-09-21T03:45:17.084 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:45:16 smithi039 ceph-mon[31211]: pgmap v302: 97 pgs: 97 active+clean; 647 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-09-21T03:45:17.084 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:45:16 smithi039 ceph-mon[31211]: from='client.? 172.21.15.39:0/1261243227' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 7} : dispatch 2024-09-21T03:45:17.084 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:45:16 smithi039 ceph-mon[31211]: from='client.? 172.21.15.39:0/3240937153' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 5} : dispatch 2024-09-21T03:45:17.084 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:45:16 smithi039 ceph-mon[31211]: from='client.? 172.21.15.39:0/2751072947' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 3} : dispatch 2024-09-21T03:45:17.218 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:45:16 smithi161 ceph-mon[34533]: pgmap v302: 97 pgs: 97 active+clean; 647 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-09-21T03:45:17.218 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:45:16 smithi161 ceph-mon[34533]: from='client.? 172.21.15.39:0/1261243227' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 7} : dispatch 2024-09-21T03:45:17.218 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:45:16 smithi161 ceph-mon[34533]: from='client.? 172.21.15.39:0/3240937153' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 5} : dispatch 2024-09-21T03:45:17.218 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:45:16 smithi161 ceph-mon[34533]: from='client.? 172.21.15.39:0/2751072947' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 3} : dispatch 2024-09-21T03:45:17.224 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:45:16 smithi150 ceph-mon[34289]: pgmap v302: 97 pgs: 97 active+clean; 647 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-09-21T03:45:17.224 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:45:16 smithi150 ceph-mon[34289]: from='client.? 172.21.15.39:0/1261243227' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 7} : dispatch 2024-09-21T03:45:17.224 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:45:16 smithi150 ceph-mon[34289]: from='client.? 172.21.15.39:0/3240937153' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 5} : dispatch 2024-09-21T03:45:17.225 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:45:16 smithi150 ceph-mon[34289]: from='client.? 172.21.15.39:0/2751072947' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 3} : dispatch 2024-09-21T03:45:17.271 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/51116d5c-77ca-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:45:17.682 INFO:tasks.cephadm.ceph_manager.ceph:need seq 158913790006 got 158913790009 for osd.5 2024-09-21T03:45:17.682 DEBUG:teuthology.parallel:result is None 2024-09-21T03:45:18.153 INFO:tasks.cephadm.ceph_manager.ceph:need seq 210453397549 got 210453397551 for osd.7 2024-09-21T03:45:18.153 DEBUG:teuthology.parallel:result is None 2024-09-21T03:45:18.617 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:18.614+0000 7f43b1d88640 1 -- 172.21.15.39:0/3522899565 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f43ac102a60 msgr2=0x7f43ac102e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:18.617 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:18.614+0000 7f43b1d88640 1 --2- 172.21.15.39:0/3522899565 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f43ac102a60 0x7f43ac102e70 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f439c009900 tx=0x7f439c02a410 comp rx=0 tx=0).stop 2024-09-21T03:45:18.617 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:18.614+0000 7f43b1d88640 1 -- 172.21.15.39:0/3522899565 shutdown_connections 2024-09-21T03:45:18.618 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:18.614+0000 7f43b1d88640 1 --2- 172.21.15.39:0/3522899565 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f43ac108cf0 0x7f43ac10f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:18.618 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:18.614+0000 7f43b1d88640 1 --2- 172.21.15.39:0/3522899565 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f43ac1083a0 0x7f43ac1087b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:18.618 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:18.614+0000 7f43b1d88640 1 --2- 172.21.15.39:0/3522899565 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f43ac102a60 0x7f43ac102e70 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:18.618 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:18.614+0000 7f43b1d88640 1 -- 172.21.15.39:0/3522899565 >> 172.21.15.39:0/3522899565 conn(0x7f43ac079730 msgr2=0x7f43ac079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:45:18.618 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:18.614+0000 7f43b1d88640 1 -- 172.21.15.39:0/3522899565 shutdown_connections 2024-09-21T03:45:18.618 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:18.614+0000 7f43b1d88640 1 -- 172.21.15.39:0/3522899565 wait complete. 2024-09-21T03:45:18.620 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:18.615+0000 7f43b1d88640 1 Processor -- start 2024-09-21T03:45:18.620 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:18.616+0000 7f43b1d88640 1 -- start start 2024-09-21T03:45:18.620 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:18.616+0000 7f43b1d88640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f43ac102a60 0x7f43ac19a460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:18.620 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:18.616+0000 7f43b1d88640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f43ac1083a0 0x7f43ac19aa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:18.620 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:18.616+0000 7f43b1d88640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f43ac108cf0 0x7f43ac19fca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:18.620 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:18.616+0000 7f43b1d88640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f43ac112960 con 0x7f43ac1083a0 2024-09-21T03:45:18.620 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:18.616+0000 7f43b1d88640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f43ac1127e0 con 0x7f43ac108cf0 2024-09-21T03:45:18.620 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:18.616+0000 7f43b1d88640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f43ac112ae0 con 0x7f43ac102a60 2024-09-21T03:45:18.620 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:18.616+0000 7f43aaffd640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f43ac1083a0 0x7f43ac19aa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:18.620 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:18.616+0000 7f43aaffd640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f43ac1083a0 0x7f43ac19aa00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:60238/0 (socket says 172.21.15.39:60238) 2024-09-21T03:45:18.620 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:18.616+0000 7f43aaffd640 1 -- 172.21.15.39:0/1204542612 learned_addr learned my addr 172.21.15.39:0/1204542612 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:45:18.620 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:18.616+0000 7f43ab7fe640 1 --2- 172.21.15.39:0/1204542612 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f43ac102a60 0x7f43ac19a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:18.621 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:18.616+0000 7f43abfff640 1 --2- 172.21.15.39:0/1204542612 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f43ac108cf0 0x7f43ac19fca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:18.621 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:18.616+0000 7f43aaffd640 1 -- 172.21.15.39:0/1204542612 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f43ac102a60 msgr2=0x7f43ac19a460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:18.621 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:18.616+0000 7f43aaffd640 1 --2- 172.21.15.39:0/1204542612 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f43ac102a60 0x7f43ac19a460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:18.621 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:18.616+0000 7f43aaffd640 1 -- 172.21.15.39:0/1204542612 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f43ac108cf0 msgr2=0x7f43ac19fca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:18.621 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:18.616+0000 7f43aaffd640 1 --2- 172.21.15.39:0/1204542612 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f43ac108cf0 0x7f43ac19fca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:18.621 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:18.616+0000 7f43aaffd640 1 -- 172.21.15.39:0/1204542612 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f43ac1a03d0 con 0x7f43ac1083a0 2024-09-21T03:45:18.621 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:18.616+0000 7f43ab7fe640 1 --2- 172.21.15.39:0/1204542612 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f43ac102a60 0x7f43ac19a460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:45:18.621 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:18.617+0000 7f43aaffd640 1 --2- 172.21.15.39:0/1204542612 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f43ac1083a0 0x7f43ac19aa00 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f43a000e9f0 tx=0x7f43a000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:45:18.621 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:18.617+0000 7f43a8ff9640 1 -- 172.21.15.39:0/1204542612 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43a000ccc0 con 0x7f43ac1083a0 2024-09-21T03:45:18.621 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:18.617+0000 7f43b1d88640 1 -- 172.21.15.39:0/1204542612 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f43ac1a06c0 con 0x7f43ac1083a0 2024-09-21T03:45:18.621 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:18.617+0000 7f43a8ff9640 1 -- 172.21.15.39:0/1204542612 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f43a00054d0 con 0x7f43ac1083a0 2024-09-21T03:45:18.621 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:18.617+0000 7f43a8ff9640 1 -- 172.21.15.39:0/1204542612 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43a0010430 con 0x7f43ac1083a0 2024-09-21T03:45:18.621 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:18.617+0000 7f43b1d88640 1 -- 172.21.15.39:0/1204542612 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f43ac1a6e20 con 0x7f43ac1083a0 2024-09-21T03:45:18.622 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:18.618+0000 7f43b1d88640 1 -- 172.21.15.39:0/1204542612 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f43ac102e70 con 0x7f43ac1083a0 2024-09-21T03:45:18.622 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:18.620+0000 7f43a8ff9640 1 -- 172.21.15.39:0/1204542612 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 22) ==== 154580+0+0 (secure 0 0 0) 0x7f43a00046c0 con 0x7f43ac1083a0 2024-09-21T03:45:18.623 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:18.621+0000 7f43a8ff9640 1 --2- 172.21.15.39:0/1204542612 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f43940b65f0 0x7f43940b8b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:18.624 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:18.621+0000 7f43ab7fe640 1 --2- 172.21.15.39:0/1204542612 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f43940b65f0 0x7f43940b8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:18.624 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:18.621+0000 7f43a8ff9640 1 -- 172.21.15.39:0/1204542612 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8484+0+0 (secure 0 0 0) 0x7f43a00e7c20 con 0x7f43ac1083a0 2024-09-21T03:45:18.624 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:18.621+0000 7f43ab7fe640 1 --2- 172.21.15.39:0/1204542612 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f43940b65f0 0x7f43940b8b00 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f439c031440 tx=0x7f439c009680 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:45:18.627 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:18.624+0000 7f43a8ff9640 1 -- 172.21.15.39:0/1204542612 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f43a0056710 con 0x7f43ac1083a0 2024-09-21T03:45:18.819 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:18.816+0000 7f43b1d88640 1 -- 172.21.15.39:0/1204542612 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) -- 0x7f43ac1a71b0 con 0x7f43ac1083a0 2024-09-21T03:45:18.819 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:18.816+0000 7f43a8ff9640 1 -- 172.21.15.39:0/1204542612 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f43a00ae250 con 0x7f43ac1083a0 2024-09-21T03:45:18.819 INFO:teuthology.orchestra.run.smithi039.stdout:133143986238 2024-09-21T03:45:18.821 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:18.819+0000 7f43b1d88640 1 -- 172.21.15.39:0/1204542612 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f43940b65f0 msgr2=0x7f43940b8b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:18.821 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:18.819+0000 7f43b1d88640 1 --2- 172.21.15.39:0/1204542612 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f43940b65f0 0x7f43940b8b00 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f439c031440 tx=0x7f439c009680 comp rx=0 tx=0).stop 2024-09-21T03:45:18.822 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:18.819+0000 7f43b1d88640 1 -- 172.21.15.39:0/1204542612 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f43ac1083a0 msgr2=0x7f43ac19aa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:18.822 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:18.819+0000 7f43b1d88640 1 --2- 172.21.15.39:0/1204542612 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f43ac1083a0 0x7f43ac19aa00 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f43a000e9f0 tx=0x7f43a000eeb0 comp rx=0 tx=0).stop 2024-09-21T03:45:18.822 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:18.819+0000 7f43b1d88640 1 -- 172.21.15.39:0/1204542612 shutdown_connections 2024-09-21T03:45:18.822 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:18.819+0000 7f43b1d88640 1 --2- 172.21.15.39:0/1204542612 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f43940b65f0 0x7f43940b8b00 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:18.822 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:18.819+0000 7f43b1d88640 1 --2- 172.21.15.39:0/1204542612 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f43ac108cf0 0x7f43ac19fca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:18.822 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:18.819+0000 7f43b1d88640 1 --2- 172.21.15.39:0/1204542612 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f43ac1083a0 0x7f43ac19aa00 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:18.822 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:18.819+0000 7f43b1d88640 1 --2- 172.21.15.39:0/1204542612 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f43ac102a60 0x7f43ac19a460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:18.822 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:18.819+0000 7f43b1d88640 1 -- 172.21.15.39:0/1204542612 >> 172.21.15.39:0/1204542612 conn(0x7f43ac079730 msgr2=0x7f43ac101a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:45:18.822 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:18.819+0000 7f43b1d88640 1 -- 172.21.15.39:0/1204542612 shutdown_connections 2024-09-21T03:45:18.822 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:18.819+0000 7f43b1d88640 1 -- 172.21.15.39:0/1204542612 wait complete. 2024-09-21T03:45:19.013 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:45:18 smithi039 ceph-mon[31211]: pgmap v303: 97 pgs: 97 active+clean; 647 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-09-21T03:45:19.013 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:45:18 smithi039 ceph-mon[31211]: from='client.? 172.21.15.39:0/1204542612' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 4} : dispatch 2024-09-21T03:45:19.040 INFO:tasks.cephadm.ceph_manager.ceph:need seq 111669149761 got 111669149763 for osd.3 2024-09-21T03:45:19.040 DEBUG:teuthology.parallel:result is None 2024-09-21T03:45:19.218 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:45:18 smithi161 ceph-mon[34533]: pgmap v303: 97 pgs: 97 active+clean; 647 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-09-21T03:45:19.218 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:45:18 smithi161 ceph-mon[34533]: from='client.? 172.21.15.39:0/1204542612' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 4} : dispatch 2024-09-21T03:45:19.224 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:45:18 smithi150 ceph-mon[34289]: pgmap v303: 97 pgs: 97 active+clean; 647 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-09-21T03:45:19.224 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:45:18 smithi150 ceph-mon[34289]: from='client.? 172.21.15.39:0/1204542612' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 4} : dispatch 2024-09-21T03:45:19.664 INFO:tasks.cephadm.ceph_manager.ceph:need seq 133143986234 got 133143986238 for osd.4 2024-09-21T03:45:19.664 DEBUG:teuthology.parallel:result is None 2024-09-21T03:45:19.665 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-09-21T03:45:19.665 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 51116d5c-77ca-11ef-baf5-efdc52797490 -- ceph pg dump --format=json 2024-09-21T03:45:20.487 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/51116d5c-77ca-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:45:20.890 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:45:20 smithi039 ceph-mon[31211]: pgmap v304: 97 pgs: 97 active+clean; 647 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-09-21T03:45:20.982 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:20.979+0000 7f67666df640 1 -- 172.21.15.39:0/2108296829 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f67601083c0 msgr2=0x7f67601087d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:20.982 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:20.979+0000 7f67666df640 1 --2- 172.21.15.39:0/2108296829 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f67601083c0 0x7f67601087d0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f6754009900 tx=0x7f675402a410 comp rx=0 tx=0).stop 2024-09-21T03:45:20.982 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:20.979+0000 7f67666df640 1 -- 172.21.15.39:0/2108296829 shutdown_connections 2024-09-21T03:45:20.983 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:20.979+0000 7f67666df640 1 --2- 172.21.15.39:0/2108296829 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6760108d10 0x7f676010f540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:20.983 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:20.979+0000 7f67666df640 1 --2- 172.21.15.39:0/2108296829 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f67601083c0 0x7f67601087d0 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:20.983 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:20.979+0000 7f67666df640 1 --2- 172.21.15.39:0/2108296829 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6760102a80 0x7f6760102e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:20.983 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:20.979+0000 7f67666df640 1 -- 172.21.15.39:0/2108296829 >> 172.21.15.39:0/2108296829 conn(0x7f6760079740 msgr2=0x7f6760079b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:45:20.983 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:20.979+0000 7f67666df640 1 -- 172.21.15.39:0/2108296829 shutdown_connections 2024-09-21T03:45:20.983 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:20.979+0000 7f67666df640 1 -- 172.21.15.39:0/2108296829 wait complete. 2024-09-21T03:45:20.983 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:20.979+0000 7f67666df640 1 Processor -- start 2024-09-21T03:45:20.983 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:20.980+0000 7f67666df640 1 -- start start 2024-09-21T03:45:20.985 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:20.980+0000 7f67666df640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6760102a80 0x7f676019a460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:20.985 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:20.980+0000 7f67666df640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f67601083c0 0x7f676019aa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:20.985 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:20.980+0000 7f67666df640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6760108d10 0x7f676019fca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:20.985 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:20.980+0000 7f67666df640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f6760112980 con 0x7f6760108d10 2024-09-21T03:45:20.985 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:20.980+0000 7f67666df640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f6760112800 con 0x7f6760102a80 2024-09-21T03:45:20.985 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:20.980+0000 7f67666df640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f6760112b00 con 0x7f67601083c0 2024-09-21T03:45:20.985 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:20.980+0000 7f6764c7c640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6760108d10 0x7f676019fca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:20.985 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:20.980+0000 7f6764c7c640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6760108d10 0x7f676019fca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:60256/0 (socket says 172.21.15.39:60256) 2024-09-21T03:45:20.985 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:20.980+0000 7f6764c7c640 1 -- 172.21.15.39:0/961328204 learned_addr learned my addr 172.21.15.39:0/961328204 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:45:20.985 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:20.980+0000 7f675f7fe640 1 --2- 172.21.15.39:0/961328204 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f67601083c0 0x7f676019aa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:20.985 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:20.980+0000 7f675ffff640 1 --2- 172.21.15.39:0/961328204 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6760102a80 0x7f676019a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:20.985 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:20.981+0000 7f6764c7c640 1 -- 172.21.15.39:0/961328204 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f67601083c0 msgr2=0x7f676019aa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:20.986 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:20.981+0000 7f6764c7c640 1 --2- 172.21.15.39:0/961328204 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f67601083c0 0x7f676019aa00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:20.986 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:20.981+0000 7f6764c7c640 1 -- 172.21.15.39:0/961328204 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6760102a80 msgr2=0x7f676019a460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:20.986 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:20.981+0000 7f6764c7c640 1 --2- 172.21.15.39:0/961328204 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6760102a80 0x7f676019a460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:20.986 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:20.981+0000 7f6764c7c640 1 -- 172.21.15.39:0/961328204 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f67601a03d0 con 0x7f6760108d10 2024-09-21T03:45:20.986 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:20.981+0000 7f6764c7c640 1 --2- 172.21.15.39:0/961328204 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6760108d10 0x7f676019fca0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f675800eaa0 tx=0x7f675800ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:45:20.986 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:20.981+0000 7f675d7fa640 1 -- 172.21.15.39:0/961328204 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f675800cdb0 con 0x7f6760108d10 2024-09-21T03:45:20.986 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:20.981+0000 7f675d7fa640 1 -- 172.21.15.39:0/961328204 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6758005020 con 0x7f6760108d10 2024-09-21T03:45:20.986 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:20.981+0000 7f675d7fa640 1 -- 172.21.15.39:0/961328204 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67580116d0 con 0x7f6760108d10 2024-09-21T03:45:20.986 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:20.981+0000 7f67666df640 1 -- 172.21.15.39:0/961328204 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f67601a06c0 con 0x7f6760108d10 2024-09-21T03:45:20.986 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:20.981+0000 7f67666df640 1 -- 172.21.15.39:0/961328204 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f67601a6e20 con 0x7f6760108d10 2024-09-21T03:45:20.986 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:20.982+0000 7f67666df640 1 -- 172.21.15.39:0/961328204 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6760103d00 con 0x7f6760108d10 2024-09-21T03:45:20.987 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:20.984+0000 7f675d7fa640 1 -- 172.21.15.39:0/961328204 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 22) ==== 154580+0+0 (secure 0 0 0) 0x7f67580051d0 con 0x7f6760108d10 2024-09-21T03:45:20.987 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:20.985+0000 7f675d7fa640 1 --2- 172.21.15.39:0/961328204 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f67480b6670 0x7f67480b8b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:20.987 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:20.985+0000 7f675ffff640 1 --2- 172.21.15.39:0/961328204 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f67480b6670 0x7f67480b8b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:20.987 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:20.985+0000 7f675d7fa640 1 -- 172.21.15.39:0/961328204 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8484+0+0 (secure 0 0 0) 0x7f67580e7aa0 con 0x7f6760108d10 2024-09-21T03:45:20.988 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:20.985+0000 7f675ffff640 1 --2- 172.21.15.39:0/961328204 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f67480b6670 0x7f67480b8b80 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f675000bbe0 tx=0x7f675000b390 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:45:20.992 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:20.989+0000 7f675d7fa640 1 -- 172.21.15.39:0/961328204 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f6758016030 con 0x7f6760108d10 2024-09-21T03:45:21.195 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:21.192+0000 7f67666df640 1 -- 172.21.15.39:0/961328204 --> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f6760104150 con 0x7f67480b6670 2024-09-21T03:45:21.200 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:21.197+0000 7f675d7fa640 1 -- 172.21.15.39:0/961328204 <== mgr.14223 v2:172.21.15.39:6800/1696381982 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258757 (secure 0 0 0) 0x7f6760104150 con 0x7f67480b6670 2024-09-21T03:45:21.202 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T03:45:21.203 INFO:teuthology.orchestra.run.smithi039.stderr:dumped all 2024-09-21T03:45:21.205 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:21.200+0000 7f67666df640 1 -- 172.21.15.39:0/961328204 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f67480b6670 msgr2=0x7f67480b8b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:21.205 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:21.200+0000 7f67666df640 1 --2- 172.21.15.39:0/961328204 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f67480b6670 0x7f67480b8b80 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f675000bbe0 tx=0x7f675000b390 comp rx=0 tx=0).stop 2024-09-21T03:45:21.205 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:21.200+0000 7f67666df640 1 -- 172.21.15.39:0/961328204 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6760108d10 msgr2=0x7f676019fca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:21.205 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:21.200+0000 7f67666df640 1 --2- 172.21.15.39:0/961328204 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6760108d10 0x7f676019fca0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f675800eaa0 tx=0x7f675800ef60 comp rx=0 tx=0).stop 2024-09-21T03:45:21.205 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:21.200+0000 7f67666df640 1 -- 172.21.15.39:0/961328204 shutdown_connections 2024-09-21T03:45:21.205 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:21.200+0000 7f67666df640 1 --2- 172.21.15.39:0/961328204 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f67480b6670 0x7f67480b8b80 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:21.205 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:21.201+0000 7f67666df640 1 --2- 172.21.15.39:0/961328204 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6760108d10 0x7f676019fca0 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:21.205 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:21.201+0000 7f67666df640 1 --2- 172.21.15.39:0/961328204 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f67601083c0 0x7f676019aa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:21.205 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:21.201+0000 7f67666df640 1 --2- 172.21.15.39:0/961328204 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6760102a80 0x7f676019a460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:21.206 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:21.201+0000 7f67666df640 1 -- 172.21.15.39:0/961328204 >> 172.21.15.39:0/961328204 conn(0x7f6760079740 msgr2=0x7f67601019c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:45:21.206 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:21.201+0000 7f67666df640 1 -- 172.21.15.39:0/961328204 shutdown_connections 2024-09-21T03:45:21.206 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:21.201+0000 7f67666df640 1 -- 172.21.15.39:0/961328204 wait complete. 2024-09-21T03:45:21.217 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:45:20 smithi161 ceph-mon[34533]: pgmap v304: 97 pgs: 97 active+clean; 647 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-09-21T03:45:21.224 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:45:20 smithi150 ceph-mon[34289]: pgmap v304: 97 pgs: 97 active+clean; 647 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-09-21T03:45:21.869 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:45:21 smithi039 ceph-mon[31211]: from='client.15186 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T03:45:21.870 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:45:21 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:45:21.870 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:45:21 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:45:21.899 INFO:teuthology.orchestra.run.smithi039.stdout:{"pg_ready":true,"pg_map":{"version":304,"stamp":"2024-09-21T03:45:19.514212+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":662099,"num_objects":102,"num_object_clones":0,"num_object_copies":306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":647,"num_read_kb":428,"num_write":369,"num_write_kb":2293,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":23,"num_bytes_recovered":2953794,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":304,"ondisk_log_size":304,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":361720,"kb_used_data":12456,"kb_used_omap":18,"kb_used_meta":349037,"kb_avail":1124580104,"statfs":{"total":1151940427776,"available":1151570026496,"internally_reserved":0,"allocated":12754944,"data_stored":7109913,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19143,"internal_metadata":357414201},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":134,"num_read_kb":65,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.014585"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":18,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:44:27.420991+0000","last_change":"2024-09-21T03:44:14.327553+0000","last_active":"2024-09-21T03:44:27.420991+0000","last_peered":"2024-09-21T03:44:27.420991+0000","last_clean":"2024-09-21T03:44:27.420991+0000","last_became_active":"2024-09-21T03:44:14.327213+0000","last_became_peered":"2024-09-21T03:44:14.327213+0000","last_unstale":"2024-09-21T03:44:27.420991+0000","last_undegraded":"2024-09-21T03:44:27.420991+0000","last_fullsized":"2024-09-21T03:44:27.420991+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_clean_scrub_stamp":"2024-09-21T03:44:13.292303+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:40:56.550897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":18,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:44:27.417200+0000","last_change":"2024-09-21T03:44:14.321427+0000","last_active":"2024-09-21T03:44:27.417200+0000","last_peered":"2024-09-21T03:44:27.417200+0000","last_clean":"2024-09-21T03:44:27.417200+0000","last_became_active":"2024-09-21T03:44:14.321026+0000","last_became_peered":"2024-09-21T03:44:14.321026+0000","last_unstale":"2024-09-21T03:44:27.417200+0000","last_undegraded":"2024-09-21T03:44:27.417200+0000","last_fullsized":"2024-09-21T03:44:27.417200+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_clean_scrub_stamp":"2024-09-21T03:44:13.292303+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:18:00.889714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":18,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:44:27.416317+0000","last_change":"2024-09-21T03:44:14.324908+0000","last_active":"2024-09-21T03:44:27.416317+0000","last_peered":"2024-09-21T03:44:27.416317+0000","last_clean":"2024-09-21T03:44:27.416317+0000","last_became_active":"2024-09-21T03:44:14.324533+0000","last_became_peered":"2024-09-21T03:44:14.324533+0000","last_unstale":"2024-09-21T03:44:27.416317+0000","last_undegraded":"2024-09-21T03:44:27.416317+0000","last_fullsized":"2024-09-21T03:44:27.416317+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_clean_scrub_stamp":"2024-09-21T03:44:13.292303+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:44:25.235442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":18,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:44:27.613591+0000","last_change":"2024-09-21T03:44:14.326758+0000","last_active":"2024-09-21T03:44:27.613591+0000","last_peered":"2024-09-21T03:44:27.613591+0000","last_clean":"2024-09-21T03:44:27.613591+0000","last_became_active":"2024-09-21T03:44:14.326410+0000","last_became_peered":"2024-09-21T03:44:14.326410+0000","last_unstale":"2024-09-21T03:44:27.613591+0000","last_undegraded":"2024-09-21T03:44:27.613591+0000","last_fullsized":"2024-09-21T03:44:27.613591+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_clean_scrub_stamp":"2024-09-21T03:44:13.292303+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:13:22.716940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":18,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:44:27.419914+0000","last_change":"2024-09-21T03:44:14.318875+0000","last_active":"2024-09-21T03:44:27.419914+0000","last_peered":"2024-09-21T03:44:27.419914+0000","last_clean":"2024-09-21T03:44:27.419914+0000","last_became_active":"2024-09-21T03:44:14.318425+0000","last_became_peered":"2024-09-21T03:44:14.318425+0000","last_unstale":"2024-09-21T03:44:27.419914+0000","last_undegraded":"2024-09-21T03:44:27.419914+0000","last_fullsized":"2024-09-21T03:44:27.419914+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_clean_scrub_stamp":"2024-09-21T03:44:13.292303+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:45:39.916587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":18,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:44:27.611598+0000","last_change":"2024-09-21T03:44:14.325877+0000","last_active":"2024-09-21T03:44:27.611598+0000","last_peered":"2024-09-21T03:44:27.611598+0000","last_clean":"2024-09-21T03:44:27.611598+0000","last_became_active":"2024-09-21T03:44:14.325451+0000","last_became_peered":"2024-09-21T03:44:14.325451+0000","last_unstale":"2024-09-21T03:44:27.611598+0000","last_undegraded":"2024-09-21T03:44:27.611598+0000","last_fullsized":"2024-09-21T03:44:27.611598+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_clean_scrub_stamp":"2024-09-21T03:44:13.292303+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:30:42.427054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":18,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:44:27.418241+0000","last_change":"2024-09-21T03:44:14.323192+0000","last_active":"2024-09-21T03:44:27.418241+0000","last_peered":"2024-09-21T03:44:27.418241+0000","last_clean":"2024-09-21T03:44:27.418241+0000","last_became_active":"2024-09-21T03:44:14.322487+0000","last_became_peered":"2024-09-21T03:44:14.322487+0000","last_unstale":"2024-09-21T03:44:27.418241+0000","last_undegraded":"2024-09-21T03:44:27.418241+0000","last_fullsized":"2024-09-21T03:44:27.418241+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_clean_scrub_stamp":"2024-09-21T03:44:13.292303+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:10:48.770536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:44:27.613320+0000","last_change":"2024-09-21T03:44:27.612681+0000","last_active":"2024-09-21T03:44:27.613320+0000","last_peered":"2024-09-21T03:44:27.613320+0000","last_clean":"2024-09-21T03:44:27.613320+0000","last_became_active":"2024-09-21T03:44:27.612200+0000","last_became_peered":"2024-09-21T03:44:27.612200+0000","last_unstale":"2024-09-21T03:44:27.613320+0000","last_undegraded":"2024-09-21T03:44:27.613320+0000","last_fullsized":"2024-09-21T03:44:27.613320+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_clean_scrub_stamp":"2024-09-21T03:44:13.292303+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:44:58.848571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,5,10],"acting":[9,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":18,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:44:27.414086+0000","last_change":"2024-09-21T03:44:14.325820+0000","last_active":"2024-09-21T03:44:27.414086+0000","last_peered":"2024-09-21T03:44:27.414086+0000","last_clean":"2024-09-21T03:44:27.414086+0000","last_became_active":"2024-09-21T03:44:14.325070+0000","last_became_peered":"2024-09-21T03:44:14.325070+0000","last_unstale":"2024-09-21T03:44:27.414086+0000","last_undegraded":"2024-09-21T03:44:27.414086+0000","last_fullsized":"2024-09-21T03:44:27.414086+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_clean_scrub_stamp":"2024-09-21T03:44:13.292303+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:54:01.072623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":18,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:44:27.612466+0000","last_change":"2024-09-21T03:44:14.323746+0000","last_active":"2024-09-21T03:44:27.612466+0000","last_peered":"2024-09-21T03:44:27.612466+0000","last_clean":"2024-09-21T03:44:27.612466+0000","last_became_active":"2024-09-21T03:44:14.323346+0000","last_became_peered":"2024-09-21T03:44:14.323346+0000","last_unstale":"2024-09-21T03:44:27.612466+0000","last_undegraded":"2024-09-21T03:44:27.612466+0000","last_fullsized":"2024-09-21T03:44:27.612466+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_clean_scrub_stamp":"2024-09-21T03:44:13.292303+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:17:44.603801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":18,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:44:27.419043+0000","last_change":"2024-09-21T03:44:14.328733+0000","last_active":"2024-09-21T03:44:27.419043+0000","last_peered":"2024-09-21T03:44:27.419043+0000","last_clean":"2024-09-21T03:44:27.419043+0000","last_became_active":"2024-09-21T03:44:14.328378+0000","last_became_peered":"2024-09-21T03:44:14.328378+0000","last_unstale":"2024-09-21T03:44:27.419043+0000","last_undegraded":"2024-09-21T03:44:27.419043+0000","last_fullsized":"2024-09-21T03:44:27.419043+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_clean_scrub_stamp":"2024-09-21T03:44:13.292303+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:23:38.497413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":18,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:44:27.613597+0000","last_change":"2024-09-21T03:44:14.328505+0000","last_active":"2024-09-21T03:44:27.613597+0000","last_peered":"2024-09-21T03:44:27.613597+0000","last_clean":"2024-09-21T03:44:27.613597+0000","last_became_active":"2024-09-21T03:44:14.328175+0000","last_became_peered":"2024-09-21T03:44:14.328175+0000","last_unstale":"2024-09-21T03:44:27.613597+0000","last_undegraded":"2024-09-21T03:44:27.613597+0000","last_fullsized":"2024-09-21T03:44:27.613597+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_clean_scrub_stamp":"2024-09-21T03:44:13.292303+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:45:56.628962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":18,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:44:27.410517+0000","last_change":"2024-09-21T03:44:14.314519+0000","last_active":"2024-09-21T03:44:27.410517+0000","last_peered":"2024-09-21T03:44:27.410517+0000","last_clean":"2024-09-21T03:44:27.410517+0000","last_became_active":"2024-09-21T03:44:14.314145+0000","last_became_peered":"2024-09-21T03:44:14.314145+0000","last_unstale":"2024-09-21T03:44:27.410517+0000","last_undegraded":"2024-09-21T03:44:27.410517+0000","last_fullsized":"2024-09-21T03:44:27.410517+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_clean_scrub_stamp":"2024-09-21T03:44:13.292303+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:45:48.494667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":18,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:44:27.416665+0000","last_change":"2024-09-21T03:44:14.312195+0000","last_active":"2024-09-21T03:44:27.416665+0000","last_peered":"2024-09-21T03:44:27.416665+0000","last_clean":"2024-09-21T03:44:27.416665+0000","last_became_active":"2024-09-21T03:44:14.311861+0000","last_became_peered":"2024-09-21T03:44:14.311861+0000","last_unstale":"2024-09-21T03:44:27.416665+0000","last_undegraded":"2024-09-21T03:44:27.416665+0000","last_fullsized":"2024-09-21T03:44:27.416665+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_clean_scrub_stamp":"2024-09-21T03:44:13.292303+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:17:05.025898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":18,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:44:27.416311+0000","last_change":"2024-09-21T03:44:14.323532+0000","last_active":"2024-09-21T03:44:27.416311+0000","last_peered":"2024-09-21T03:44:27.416311+0000","last_clean":"2024-09-21T03:44:27.416311+0000","last_became_active":"2024-09-21T03:44:14.323171+0000","last_became_peered":"2024-09-21T03:44:14.323171+0000","last_unstale":"2024-09-21T03:44:27.416311+0000","last_undegraded":"2024-09-21T03:44:27.416311+0000","last_fullsized":"2024-09-21T03:44:27.416311+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_clean_scrub_stamp":"2024-09-21T03:44:13.292303+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:23:28.999129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":18,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:44:27.413494+0000","last_change":"2024-09-21T03:44:14.324998+0000","last_active":"2024-09-21T03:44:27.413494+0000","last_peered":"2024-09-21T03:44:27.413494+0000","last_clean":"2024-09-21T03:44:27.413494+0000","last_became_active":"2024-09-21T03:44:14.324019+0000","last_became_peered":"2024-09-21T03:44:14.324019+0000","last_unstale":"2024-09-21T03:44:27.413494+0000","last_undegraded":"2024-09-21T03:44:27.413494+0000","last_fullsized":"2024-09-21T03:44:27.413494+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_clean_scrub_stamp":"2024-09-21T03:44:13.292303+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:49:12.965299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":18,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:44:27.421157+0000","last_change":"2024-09-21T03:44:14.325603+0000","last_active":"2024-09-21T03:44:27.421157+0000","last_peered":"2024-09-21T03:44:27.421157+0000","last_clean":"2024-09-21T03:44:27.421157+0000","last_became_active":"2024-09-21T03:44:14.325208+0000","last_became_peered":"2024-09-21T03:44:14.325208+0000","last_unstale":"2024-09-21T03:44:27.421157+0000","last_undegraded":"2024-09-21T03:44:27.421157+0000","last_fullsized":"2024-09-21T03:44:27.421157+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_clean_scrub_stamp":"2024-09-21T03:44:13.292303+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:11:33.794242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":18,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:44:27.414581+0000","last_change":"2024-09-21T03:44:14.317545+0000","last_active":"2024-09-21T03:44:27.414581+0000","last_peered":"2024-09-21T03:44:27.414581+0000","last_clean":"2024-09-21T03:44:27.414581+0000","last_became_active":"2024-09-21T03:44:14.317069+0000","last_became_peered":"2024-09-21T03:44:14.317069+0000","last_unstale":"2024-09-21T03:44:27.414581+0000","last_undegraded":"2024-09-21T03:44:27.414581+0000","last_fullsized":"2024-09-21T03:44:27.414581+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_clean_scrub_stamp":"2024-09-21T03:44:13.292303+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:16:11.734887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":18,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:44:27.418423+0000","last_change":"2024-09-21T03:44:14.322055+0000","last_active":"2024-09-21T03:44:27.418423+0000","last_peered":"2024-09-21T03:44:27.418423+0000","last_clean":"2024-09-21T03:44:27.418423+0000","last_became_active":"2024-09-21T03:44:14.321652+0000","last_became_peered":"2024-09-21T03:44:14.321652+0000","last_unstale":"2024-09-21T03:44:27.418423+0000","last_undegraded":"2024-09-21T03:44:27.418423+0000","last_fullsized":"2024-09-21T03:44:27.418423+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_clean_scrub_stamp":"2024-09-21T03:44:13.292303+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:45:14.285844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":25,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:44:27.423315+0000","last_change":"2024-09-21T03:44:27.423175+0000","last_active":"2024-09-21T03:44:27.423315+0000","last_peered":"2024-09-21T03:44:27.423315+0000","last_clean":"2024-09-21T03:44:27.423315+0000","last_became_active":"2024-09-21T03:44:27.422733+0000","last_became_peered":"2024-09-21T03:44:27.422733+0000","last_unstale":"2024-09-21T03:44:27.423315+0000","last_undegraded":"2024-09-21T03:44:27.423315+0000","last_fullsized":"2024-09-21T03:44:27.423315+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_clean_scrub_stamp":"2024-09-21T03:44:13.292303+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:46:17.874557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,8],"acting":[11,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":18,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:44:27.410602+0000","last_change":"2024-09-21T03:44:14.325829+0000","last_active":"2024-09-21T03:44:27.410602+0000","last_peered":"2024-09-21T03:44:27.410602+0000","last_clean":"2024-09-21T03:44:27.410602+0000","last_became_active":"2024-09-21T03:44:14.325421+0000","last_became_peered":"2024-09-21T03:44:14.325421+0000","last_unstale":"2024-09-21T03:44:27.410602+0000","last_undegraded":"2024-09-21T03:44:27.410602+0000","last_fullsized":"2024-09-21T03:44:27.410602+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_clean_scrub_stamp":"2024-09-21T03:44:13.292303+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:49:00.240771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":18,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:44:27.413691+0000","last_change":"2024-09-21T03:44:14.325255+0000","last_active":"2024-09-21T03:44:27.413691+0000","last_peered":"2024-09-21T03:44:27.413691+0000","last_clean":"2024-09-21T03:44:27.413691+0000","last_became_active":"2024-09-21T03:44:14.324617+0000","last_became_peered":"2024-09-21T03:44:14.324617+0000","last_unstale":"2024-09-21T03:44:27.413691+0000","last_undegraded":"2024-09-21T03:44:27.413691+0000","last_fullsized":"2024-09-21T03:44:27.413691+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_clean_scrub_stamp":"2024-09-21T03:44:13.292303+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:05:03.268522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":18,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:44:27.610583+0000","last_change":"2024-09-21T03:44:14.328053+0000","last_active":"2024-09-21T03:44:27.610583+0000","last_peered":"2024-09-21T03:44:27.610583+0000","last_clean":"2024-09-21T03:44:27.610583+0000","last_became_active":"2024-09-21T03:44:14.327735+0000","last_became_peered":"2024-09-21T03:44:14.327735+0000","last_unstale":"2024-09-21T03:44:27.610583+0000","last_undegraded":"2024-09-21T03:44:27.610583+0000","last_fullsized":"2024-09-21T03:44:27.610583+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_clean_scrub_stamp":"2024-09-21T03:44:13.292303+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:39:48.395784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":18,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:44:27.413859+0000","last_change":"2024-09-21T03:44:14.321181+0000","last_active":"2024-09-21T03:44:27.413859+0000","last_peered":"2024-09-21T03:44:27.413859+0000","last_clean":"2024-09-21T03:44:27.413859+0000","last_became_active":"2024-09-21T03:44:14.320772+0000","last_became_peered":"2024-09-21T03:44:14.320772+0000","last_unstale":"2024-09-21T03:44:27.413859+0000","last_undegraded":"2024-09-21T03:44:27.413859+0000","last_fullsized":"2024-09-21T03:44:27.413859+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_clean_scrub_stamp":"2024-09-21T03:44:13.292303+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:49:26.317404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":18,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:44:27.611124+0000","last_change":"2024-09-21T03:44:14.322643+0000","last_active":"2024-09-21T03:44:27.611124+0000","last_peered":"2024-09-21T03:44:27.611124+0000","last_clean":"2024-09-21T03:44:27.611124+0000","last_became_active":"2024-09-21T03:44:14.322254+0000","last_became_peered":"2024-09-21T03:44:14.322254+0000","last_unstale":"2024-09-21T03:44:27.611124+0000","last_undegraded":"2024-09-21T03:44:27.611124+0000","last_fullsized":"2024-09-21T03:44:27.611124+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_clean_scrub_stamp":"2024-09-21T03:44:13.292303+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:31:47.147845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":18,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:44:27.418815+0000","last_change":"2024-09-21T03:44:14.323090+0000","last_active":"2024-09-21T03:44:27.418815+0000","last_peered":"2024-09-21T03:44:27.418815+0000","last_clean":"2024-09-21T03:44:27.418815+0000","last_became_active":"2024-09-21T03:44:14.322669+0000","last_became_peered":"2024-09-21T03:44:14.322669+0000","last_unstale":"2024-09-21T03:44:27.418815+0000","last_undegraded":"2024-09-21T03:44:27.418815+0000","last_fullsized":"2024-09-21T03:44:27.418815+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_clean_scrub_stamp":"2024-09-21T03:44:13.292303+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:08:56.852422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":18,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:44:27.418948+0000","last_change":"2024-09-21T03:44:14.318146+0000","last_active":"2024-09-21T03:44:27.418948+0000","last_peered":"2024-09-21T03:44:27.418948+0000","last_clean":"2024-09-21T03:44:27.418948+0000","last_became_active":"2024-09-21T03:44:14.317761+0000","last_became_peered":"2024-09-21T03:44:14.317761+0000","last_unstale":"2024-09-21T03:44:27.418948+0000","last_undegraded":"2024-09-21T03:44:27.418948+0000","last_fullsized":"2024-09-21T03:44:27.418948+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_clean_scrub_stamp":"2024-09-21T03:44:13.292303+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:07:31.530728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":18,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:44:27.410569+0000","last_change":"2024-09-21T03:44:14.316832+0000","last_active":"2024-09-21T03:44:27.410569+0000","last_peered":"2024-09-21T03:44:27.410569+0000","last_clean":"2024-09-21T03:44:27.410569+0000","last_became_active":"2024-09-21T03:44:14.316471+0000","last_became_peered":"2024-09-21T03:44:14.316471+0000","last_unstale":"2024-09-21T03:44:27.410569+0000","last_undegraded":"2024-09-21T03:44:27.410569+0000","last_fullsized":"2024-09-21T03:44:27.410569+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_clean_scrub_stamp":"2024-09-21T03:44:13.292303+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:21:23.848778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":18,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:44:27.417002+0000","last_change":"2024-09-21T03:44:14.322277+0000","last_active":"2024-09-21T03:44:27.417002+0000","last_peered":"2024-09-21T03:44:27.417002+0000","last_clean":"2024-09-21T03:44:27.417002+0000","last_became_active":"2024-09-21T03:44:14.321952+0000","last_became_peered":"2024-09-21T03:44:14.321952+0000","last_unstale":"2024-09-21T03:44:27.417002+0000","last_undegraded":"2024-09-21T03:44:27.417002+0000","last_fullsized":"2024-09-21T03:44:27.417002+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_clean_scrub_stamp":"2024-09-21T03:44:13.292303+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:43:55.169380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":18,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:44:27.418939+0000","last_change":"2024-09-21T03:44:14.322626+0000","last_active":"2024-09-21T03:44:27.418939+0000","last_peered":"2024-09-21T03:44:27.418939+0000","last_clean":"2024-09-21T03:44:27.418939+0000","last_became_active":"2024-09-21T03:44:14.321559+0000","last_became_peered":"2024-09-21T03:44:14.321559+0000","last_unstale":"2024-09-21T03:44:27.418939+0000","last_undegraded":"2024-09-21T03:44:27.418939+0000","last_fullsized":"2024-09-21T03:44:27.418939+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_clean_scrub_stamp":"2024-09-21T03:44:13.292303+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:12:23.736500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":18,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:44:27.416654+0000","last_change":"2024-09-21T03:44:14.318057+0000","last_active":"2024-09-21T03:44:27.416654+0000","last_peered":"2024-09-21T03:44:27.416654+0000","last_clean":"2024-09-21T03:44:27.416654+0000","last_became_active":"2024-09-21T03:44:14.317653+0000","last_became_peered":"2024-09-21T03:44:14.317653+0000","last_unstale":"2024-09-21T03:44:27.416654+0000","last_undegraded":"2024-09-21T03:44:27.416654+0000","last_fullsized":"2024-09-21T03:44:27.416654+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_clean_scrub_stamp":"2024-09-21T03:44:13.292303+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:23:17.396599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":18,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:44:27.418421+0000","last_change":"2024-09-21T03:44:14.322331+0000","last_active":"2024-09-21T03:44:27.418421+0000","last_peered":"2024-09-21T03:44:27.418421+0000","last_clean":"2024-09-21T03:44:27.418421+0000","last_became_active":"2024-09-21T03:44:14.321157+0000","last_became_peered":"2024-09-21T03:44:14.321157+0000","last_unstale":"2024-09-21T03:44:27.418421+0000","last_undegraded":"2024-09-21T03:44:27.418421+0000","last_fullsized":"2024-09-21T03:44:27.418421+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_clean_scrub_stamp":"2024-09-21T03:44:13.292303+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:08:27.953449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":25,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:44:27.423182+0000","last_change":"2024-09-21T03:44:27.423081+0000","last_active":"2024-09-21T03:44:27.423182+0000","last_peered":"2024-09-21T03:44:27.423182+0000","last_clean":"2024-09-21T03:44:27.423182+0000","last_became_active":"2024-09-21T03:44:27.422650+0000","last_became_peered":"2024-09-21T03:44:27.422650+0000","last_unstale":"2024-09-21T03:44:27.423182+0000","last_undegraded":"2024-09-21T03:44:27.423182+0000","last_fullsized":"2024-09-21T03:44:27.423182+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_clean_scrub_stamp":"2024-09-21T03:44:13.292303+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:32:44.399954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,7],"acting":[8,9,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1b","version":"79'6","reported_seq":87,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:45:17.780563+0000","last_change":"2024-09-21T03:44:12.307464+0000","last_active":"2024-09-21T03:45:17.780563+0000","last_peered":"2024-09-21T03:45:17.780563+0000","last_clean":"2024-09-21T03:45:17.780563+0000","last_became_active":"2024-09-21T03:44:12.307077+0000","last_became_peered":"2024-09-21T03:44:12.307077+0000","last_unstale":"2024-09-21T03:45:17.780563+0000","last_undegraded":"2024-09-21T03:45:17.780563+0000","last_fullsized":"2024-09-21T03:45:17.780563+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:11.282438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:11.282438+0000","last_clean_scrub_stamp":"2024-09-21T03:44:11.282438+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:50:40.344659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1013,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":58,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":18,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:44:27.414062+0000","last_change":"2024-09-21T03:44:14.319072+0000","last_active":"2024-09-21T03:44:27.414062+0000","last_peered":"2024-09-21T03:44:27.414062+0000","last_clean":"2024-09-21T03:44:27.414062+0000","last_became_active":"2024-09-21T03:44:14.318712+0000","last_became_peered":"2024-09-21T03:44:14.318712+0000","last_unstale":"2024-09-21T03:44:27.414062+0000","last_undegraded":"2024-09-21T03:44:27.414062+0000","last_fullsized":"2024-09-21T03:44:27.414062+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_clean_scrub_stamp":"2024-09-21T03:44:13.292303+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:30:35.618110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1a","version":"77'7","reported_seq":30,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:44:27.611763+0000","last_change":"2024-09-21T03:44:12.310146+0000","last_active":"2024-09-21T03:44:27.611763+0000","last_peered":"2024-09-21T03:44:27.611763+0000","last_clean":"2024-09-21T03:44:27.611763+0000","last_became_active":"2024-09-21T03:44:12.309817+0000","last_became_peered":"2024-09-21T03:44:12.309817+0000","last_unstale":"2024-09-21T03:44:27.611763+0000","last_undegraded":"2024-09-21T03:44:27.611763+0000","last_fullsized":"2024-09-21T03:44:27.611763+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:11.282438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:11.282438+0000","last_clean_scrub_stamp":"2024-09-21T03:44:11.282438+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:46:51.994488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.19","version":"77'3","reported_seq":26,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:44:27.410211+0000","last_change":"2024-09-21T03:44:12.308995+0000","last_active":"2024-09-21T03:44:27.410211+0000","last_peered":"2024-09-21T03:44:27.410211+0000","last_clean":"2024-09-21T03:44:27.410211+0000","last_became_active":"2024-09-21T03:44:12.308058+0000","last_became_peered":"2024-09-21T03:44:12.308058+0000","last_unstale":"2024-09-21T03:44:27.410211+0000","last_undegraded":"2024-09-21T03:44:27.410211+0000","last_fullsized":"2024-09-21T03:44:27.410211+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:11.282438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:11.282438+0000","last_clean_scrub_stamp":"2024-09-21T03:44:11.282438+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:48:39.384985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":18,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:44:27.610583+0000","last_change":"2024-09-21T03:44:14.317878+0000","last_active":"2024-09-21T03:44:27.610583+0000","last_peered":"2024-09-21T03:44:27.610583+0000","last_clean":"2024-09-21T03:44:27.610583+0000","last_became_active":"2024-09-21T03:44:14.317521+0000","last_became_peered":"2024-09-21T03:44:14.317521+0000","last_unstale":"2024-09-21T03:44:27.610583+0000","last_undegraded":"2024-09-21T03:44:27.610583+0000","last_fullsized":"2024-09-21T03:44:27.610583+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_clean_scrub_stamp":"2024-09-21T03:44:13.292303+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:08:00.705210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"77'3","reported_seq":27,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:44:27.414041+0000","last_change":"2024-09-21T03:44:12.308777+0000","last_active":"2024-09-21T03:44:27.414041+0000","last_peered":"2024-09-21T03:44:27.414041+0000","last_clean":"2024-09-21T03:44:27.414041+0000","last_became_active":"2024-09-21T03:44:12.308171+0000","last_became_peered":"2024-09-21T03:44:12.308171+0000","last_unstale":"2024-09-21T03:44:27.414041+0000","last_undegraded":"2024-09-21T03:44:27.414041+0000","last_fullsized":"2024-09-21T03:44:27.414041+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:11.282438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:11.282438+0000","last_clean_scrub_stamp":"2024-09-21T03:44:11.282438+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:41:21.344491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":18,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:44:27.414180+0000","last_change":"2024-09-21T03:44:14.315782+0000","last_active":"2024-09-21T03:44:27.414180+0000","last_peered":"2024-09-21T03:44:27.414180+0000","last_clean":"2024-09-21T03:44:27.414180+0000","last_became_active":"2024-09-21T03:44:14.315314+0000","last_became_peered":"2024-09-21T03:44:14.315314+0000","last_unstale":"2024-09-21T03:44:27.414180+0000","last_undegraded":"2024-09-21T03:44:27.414180+0000","last_fullsized":"2024-09-21T03:44:27.414180+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_clean_scrub_stamp":"2024-09-21T03:44:13.292303+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:22:20.500220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"77'4","reported_seq":135,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:45:17.780016+0000","last_change":"2024-09-21T03:44:12.313251+0000","last_active":"2024-09-21T03:45:17.780016+0000","last_peered":"2024-09-21T03:45:17.780016+0000","last_clean":"2024-09-21T03:45:17.780016+0000","last_became_active":"2024-09-21T03:44:12.312941+0000","last_became_peered":"2024-09-21T03:44:12.312941+0000","last_unstale":"2024-09-21T03:45:17.780016+0000","last_undegraded":"2024-09-21T03:45:17.780016+0000","last_fullsized":"2024-09-21T03:45:17.780016+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:11.282438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:11.282438+0000","last_clean_scrub_stamp":"2024-09-21T03:44:11.282438+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:38:39.876097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":108,"num_read_kb":108,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":18,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:44:27.414153+0000","last_change":"2024-09-21T03:44:14.319621+0000","last_active":"2024-09-21T03:44:27.414153+0000","last_peered":"2024-09-21T03:44:27.414153+0000","last_clean":"2024-09-21T03:44:27.414153+0000","last_became_active":"2024-09-21T03:44:14.316173+0000","last_became_peered":"2024-09-21T03:44:14.316173+0000","last_unstale":"2024-09-21T03:44:27.414153+0000","last_undegraded":"2024-09-21T03:44:27.414153+0000","last_fullsized":"2024-09-21T03:44:27.414153+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_clean_scrub_stamp":"2024-09-21T03:44:13.292303+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:10:21.952989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"77'4","reported_seq":27,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:44:27.419175+0000","last_change":"2024-09-21T03:44:12.308893+0000","last_active":"2024-09-21T03:44:27.419175+0000","last_peered":"2024-09-21T03:44:27.419175+0000","last_clean":"2024-09-21T03:44:27.419175+0000","last_became_active":"2024-09-21T03:44:12.308551+0000","last_became_peered":"2024-09-21T03:44:12.308551+0000","last_unstale":"2024-09-21T03:44:27.419175+0000","last_undegraded":"2024-09-21T03:44:27.419175+0000","last_fullsized":"2024-09-21T03:44:27.419175+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:11.282438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:11.282438+0000","last_clean_scrub_stamp":"2024-09-21T03:44:11.282438+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:48:18.401595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":18,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:44:27.417391+0000","last_change":"2024-09-21T03:44:14.321875+0000","last_active":"2024-09-21T03:44:27.417391+0000","last_peered":"2024-09-21T03:44:27.417391+0000","last_clean":"2024-09-21T03:44:27.417391+0000","last_became_active":"2024-09-21T03:44:14.321483+0000","last_became_peered":"2024-09-21T03:44:14.321483+0000","last_unstale":"2024-09-21T03:44:27.417391+0000","last_undegraded":"2024-09-21T03:44:27.417391+0000","last_fullsized":"2024-09-21T03:44:27.417391+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_clean_scrub_stamp":"2024-09-21T03:44:13.292303+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:40:17.244780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"77'5","reported_seq":29,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:44:27.414461+0000","last_change":"2024-09-21T03:44:12.313208+0000","last_active":"2024-09-21T03:44:27.414461+0000","last_peered":"2024-09-21T03:44:27.414461+0000","last_clean":"2024-09-21T03:44:27.414461+0000","last_became_active":"2024-09-21T03:44:12.312783+0000","last_became_peered":"2024-09-21T03:44:12.312783+0000","last_unstale":"2024-09-21T03:44:27.414461+0000","last_undegraded":"2024-09-21T03:44:27.414461+0000","last_fullsized":"2024-09-21T03:44:27.414461+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:11.282438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:11.282438+0000","last_clean_scrub_stamp":"2024-09-21T03:44:11.282438+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:18:45.114361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":25,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:44:27.421219+0000","last_change":"2024-09-21T03:44:27.421129+0000","last_active":"2024-09-21T03:44:27.421219+0000","last_peered":"2024-09-21T03:44:27.421219+0000","last_clean":"2024-09-21T03:44:27.421219+0000","last_became_active":"2024-09-21T03:44:27.420642+0000","last_became_peered":"2024-09-21T03:44:27.420642+0000","last_unstale":"2024-09-21T03:44:27.421219+0000","last_undegraded":"2024-09-21T03:44:27.421219+0000","last_fullsized":"2024-09-21T03:44:27.421219+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_clean_scrub_stamp":"2024-09-21T03:44:13.292303+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:49:38.065743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,6],"acting":[1,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"79'30","reported_seq":170,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:45:17.794643+0000","last_change":"2024-09-21T03:44:12.314863+0000","last_active":"2024-09-21T03:45:17.794643+0000","last_peered":"2024-09-21T03:45:17.794643+0000","last_clean":"2024-09-21T03:45:17.794643+0000","last_became_active":"2024-09-21T03:44:12.314471+0000","last_became_peered":"2024-09-21T03:44:12.314471+0000","last_unstale":"2024-09-21T03:45:17.794643+0000","last_undegraded":"2024-09-21T03:45:17.794643+0000","last_fullsized":"2024-09-21T03:45:17.794643+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:11.282438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:11.282438+0000","last_clean_scrub_stamp":"2024-09-21T03:44:11.282438+0000","objects_scrubbed":0,"log_size":30,"log_dups_size":0,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:41:03.471226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":63902,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":117,"num_read_kb":120,"num_write":32,"num_write_kb":75,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":18,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:44:27.420428+0000","last_change":"2024-09-21T03:44:14.325471+0000","last_active":"2024-09-21T03:44:27.420428+0000","last_peered":"2024-09-21T03:44:27.420428+0000","last_clean":"2024-09-21T03:44:27.420428+0000","last_became_active":"2024-09-21T03:44:14.325072+0000","last_became_peered":"2024-09-21T03:44:14.325072+0000","last_unstale":"2024-09-21T03:44:27.420428+0000","last_undegraded":"2024-09-21T03:44:27.420428+0000","last_fullsized":"2024-09-21T03:44:27.420428+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_clean_scrub_stamp":"2024-09-21T03:44:13.292303+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:40:40.097240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"77'9","reported_seq":34,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:44:27.416778+0000","last_change":"2024-09-21T03:44:12.312932+0000","last_active":"2024-09-21T03:44:27.416778+0000","last_peered":"2024-09-21T03:44:27.416778+0000","last_clean":"2024-09-21T03:44:27.416778+0000","last_became_active":"2024-09-21T03:44:12.312368+0000","last_became_peered":"2024-09-21T03:44:12.312368+0000","last_unstale":"2024-09-21T03:44:27.416778+0000","last_undegraded":"2024-09-21T03:44:27.416778+0000","last_fullsized":"2024-09-21T03:44:27.416778+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:11.282438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:11.282438+0000","last_clean_scrub_stamp":"2024-09-21T03:44:11.282438+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:28:11.579598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":18,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:44:27.410310+0000","last_change":"2024-09-21T03:44:14.329564+0000","last_active":"2024-09-21T03:44:27.410310+0000","last_peered":"2024-09-21T03:44:27.410310+0000","last_clean":"2024-09-21T03:44:27.410310+0000","last_became_active":"2024-09-21T03:44:14.329122+0000","last_became_peered":"2024-09-21T03:44:14.329122+0000","last_unstale":"2024-09-21T03:44:27.410310+0000","last_undegraded":"2024-09-21T03:44:27.410310+0000","last_fullsized":"2024-09-21T03:44:27.410310+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_clean_scrub_stamp":"2024-09-21T03:44:13.292303+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:16:32.702770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"77'5","reported_seq":29,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:44:27.413875+0000","last_change":"2024-09-21T03:44:12.303339+0000","last_active":"2024-09-21T03:44:27.413875+0000","last_peered":"2024-09-21T03:44:27.413875+0000","last_clean":"2024-09-21T03:44:27.413875+0000","last_became_active":"2024-09-21T03:44:12.302905+0000","last_became_peered":"2024-09-21T03:44:12.302905+0000","last_unstale":"2024-09-21T03:44:27.413875+0000","last_undegraded":"2024-09-21T03:44:27.413875+0000","last_fullsized":"2024-09-21T03:44:27.413875+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:11.282438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:11.282438+0000","last_clean_scrub_stamp":"2024-09-21T03:44:11.282438+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:57:16.424976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":18,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:44:27.418798+0000","last_change":"2024-09-21T03:44:14.323473+0000","last_active":"2024-09-21T03:44:27.418798+0000","last_peered":"2024-09-21T03:44:27.418798+0000","last_clean":"2024-09-21T03:44:27.418798+0000","last_became_active":"2024-09-21T03:44:14.322560+0000","last_became_peered":"2024-09-21T03:44:14.322560+0000","last_unstale":"2024-09-21T03:44:27.418798+0000","last_undegraded":"2024-09-21T03:44:27.418798+0000","last_fullsized":"2024-09-21T03:44:27.418798+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_clean_scrub_stamp":"2024-09-21T03:44:13.292303+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:07:55.714351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"79'6","reported_seq":83,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:45:13.775063+0000","last_change":"2024-09-21T03:44:12.312331+0000","last_active":"2024-09-21T03:45:13.775063+0000","last_peered":"2024-09-21T03:45:13.775063+0000","last_clean":"2024-09-21T03:45:13.775063+0000","last_became_active":"2024-09-21T03:44:12.311984+0000","last_became_peered":"2024-09-21T03:44:12.311984+0000","last_unstale":"2024-09-21T03:45:13.775063+0000","last_undegraded":"2024-09-21T03:45:13.775063+0000","last_fullsized":"2024-09-21T03:45:13.775063+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:11.282438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:11.282438+0000","last_clean_scrub_stamp":"2024-09-21T03:44:11.282438+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:06:11.906592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":979,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":55,"num_read_kb":4,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":18,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:44:27.418181+0000","last_change":"2024-09-21T03:44:14.318437+0000","last_active":"2024-09-21T03:44:27.418181+0000","last_peered":"2024-09-21T03:44:27.418181+0000","last_clean":"2024-09-21T03:44:27.418181+0000","last_became_active":"2024-09-21T03:44:14.318052+0000","last_became_peered":"2024-09-21T03:44:14.318052+0000","last_unstale":"2024-09-21T03:44:27.418181+0000","last_undegraded":"2024-09-21T03:44:27.418181+0000","last_fullsized":"2024-09-21T03:44:27.418181+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_clean_scrub_stamp":"2024-09-21T03:44:13.292303+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:10:12.136159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"77'17","reported_seq":41,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:44:27.419831+0000","last_change":"2024-09-21T03:44:12.310616+0000","last_active":"2024-09-21T03:44:27.419831+0000","last_peered":"2024-09-21T03:44:27.419831+0000","last_clean":"2024-09-21T03:44:27.419831+0000","last_became_active":"2024-09-21T03:44:12.310239+0000","last_became_peered":"2024-09-21T03:44:12.310239+0000","last_unstale":"2024-09-21T03:44:27.419831+0000","last_undegraded":"2024-09-21T03:44:27.419831+0000","last_fullsized":"2024-09-21T03:44:27.419831+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:11.282438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:11.282438+0000","last_clean_scrub_stamp":"2024-09-21T03:44:11.282438+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:43:38.046281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":18,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:44:27.413822+0000","last_change":"2024-09-21T03:44:14.327058+0000","last_active":"2024-09-21T03:44:27.413822+0000","last_peered":"2024-09-21T03:44:27.413822+0000","last_clean":"2024-09-21T03:44:27.413822+0000","last_became_active":"2024-09-21T03:44:14.326641+0000","last_became_peered":"2024-09-21T03:44:14.326641+0000","last_unstale":"2024-09-21T03:44:27.413822+0000","last_undegraded":"2024-09-21T03:44:27.413822+0000","last_fullsized":"2024-09-21T03:44:27.413822+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_clean_scrub_stamp":"2024-09-21T03:44:13.292303+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:10:50.686102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"77'2","reported_seq":26,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:44:27.420648+0000","last_change":"2024-09-21T03:44:12.310426+0000","last_active":"2024-09-21T03:44:27.420648+0000","last_peered":"2024-09-21T03:44:27.420648+0000","last_clean":"2024-09-21T03:44:27.420648+0000","last_became_active":"2024-09-21T03:44:12.309231+0000","last_became_peered":"2024-09-21T03:44:12.309231+0000","last_unstale":"2024-09-21T03:44:27.420648+0000","last_undegraded":"2024-09-21T03:44:27.420648+0000","last_fullsized":"2024-09-21T03:44:27.420648+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:11.282438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:11.282438+0000","last_clean_scrub_stamp":"2024-09-21T03:44:11.282438+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:32:18.284741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":18,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:44:27.420390+0000","last_change":"2024-09-21T03:44:14.319179+0000","last_active":"2024-09-21T03:44:27.420390+0000","last_peered":"2024-09-21T03:44:27.420390+0000","last_clean":"2024-09-21T03:44:27.420390+0000","last_became_active":"2024-09-21T03:44:14.318826+0000","last_became_peered":"2024-09-21T03:44:14.318826+0000","last_unstale":"2024-09-21T03:44:27.420390+0000","last_undegraded":"2024-09-21T03:44:27.420390+0000","last_fullsized":"2024-09-21T03:44:27.420390+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_clean_scrub_stamp":"2024-09-21T03:44:13.292303+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:12:57.985087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.e","version":"77'5","reported_seq":28,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:44:27.410705+0000","last_change":"2024-09-21T03:44:12.315658+0000","last_active":"2024-09-21T03:44:27.410705+0000","last_peered":"2024-09-21T03:44:27.410705+0000","last_clean":"2024-09-21T03:44:27.410705+0000","last_became_active":"2024-09-21T03:44:12.315240+0000","last_became_peered":"2024-09-21T03:44:12.315240+0000","last_unstale":"2024-09-21T03:44:27.410705+0000","last_undegraded":"2024-09-21T03:44:27.410705+0000","last_fullsized":"2024-09-21T03:44:27.410705+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:11.282438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:11.282438+0000","last_clean_scrub_stamp":"2024-09-21T03:44:11.282438+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:22:07.493437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":18,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:44:27.419234+0000","last_change":"2024-09-21T03:44:14.328014+0000","last_active":"2024-09-21T03:44:27.419234+0000","last_peered":"2024-09-21T03:44:27.419234+0000","last_clean":"2024-09-21T03:44:27.419234+0000","last_became_active":"2024-09-21T03:44:14.327638+0000","last_became_peered":"2024-09-21T03:44:14.327638+0000","last_unstale":"2024-09-21T03:44:27.419234+0000","last_undegraded":"2024-09-21T03:44:27.419234+0000","last_fullsized":"2024-09-21T03:44:27.419234+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_clean_scrub_stamp":"2024-09-21T03:44:13.292303+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:48:32.542776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"77'3","reported_seq":27,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:44:27.414656+0000","last_change":"2024-09-21T03:44:12.309272+0000","last_active":"2024-09-21T03:44:27.414656+0000","last_peered":"2024-09-21T03:44:27.414656+0000","last_clean":"2024-09-21T03:44:27.414656+0000","last_became_active":"2024-09-21T03:44:12.308718+0000","last_became_peered":"2024-09-21T03:44:12.308718+0000","last_unstale":"2024-09-21T03:44:27.414656+0000","last_undegraded":"2024-09-21T03:44:27.414656+0000","last_fullsized":"2024-09-21T03:44:27.414656+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:11.282438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:11.282438+0000","last_clean_scrub_stamp":"2024-09-21T03:44:11.282438+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:03:49.433966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"79'1","reported_seq":19,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:44:27.418909+0000","last_change":"2024-09-21T03:44:14.322148+0000","last_active":"2024-09-21T03:44:27.418909+0000","last_peered":"2024-09-21T03:44:27.418909+0000","last_clean":"2024-09-21T03:44:27.418909+0000","last_became_active":"2024-09-21T03:44:14.321784+0000","last_became_peered":"2024-09-21T03:44:14.321784+0000","last_unstale":"2024-09-21T03:44:27.418909+0000","last_undegraded":"2024-09-21T03:44:27.418909+0000","last_fullsized":"2024-09-21T03:44:27.418909+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_clean_scrub_stamp":"2024-09-21T03:44:13.292303+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:40:34.887134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"77'5","reported_seq":72,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:45:17.493522+0000","last_change":"2024-09-21T03:44:27.427722+0000","last_active":"2024-09-21T03:45:17.493522+0000","last_peered":"2024-09-21T03:45:17.493522+0000","last_clean":"2024-09-21T03:45:17.493522+0000","last_became_active":"2024-09-21T03:44:27.419672+0000","last_became_peered":"2024-09-21T03:44:27.419672+0000","last_unstale":"2024-09-21T03:45:17.493522+0000","last_undegraded":"2024-09-21T03:45:17.493522+0000","last_fullsized":"2024-09-21T03:45:17.493522+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:11.282438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:11.282438+0000","last_clean_scrub_stamp":"2024-09-21T03:44:11.282438+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:24:31.890439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1170,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":52,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":1170,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,11,0],"acting":[1,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":18,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:44:27.419112+0000","last_change":"2024-09-21T03:44:14.316503+0000","last_active":"2024-09-21T03:44:27.419112+0000","last_peered":"2024-09-21T03:44:27.419112+0000","last_clean":"2024-09-21T03:44:27.419112+0000","last_became_active":"2024-09-21T03:44:14.316140+0000","last_became_peered":"2024-09-21T03:44:14.316140+0000","last_unstale":"2024-09-21T03:44:27.419112+0000","last_undegraded":"2024-09-21T03:44:27.419112+0000","last_fullsized":"2024-09-21T03:44:27.419112+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_clean_scrub_stamp":"2024-09-21T03:44:13.292303+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:00:30.647579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"21'92","reported_seq":96,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:44:27.418315+0000","last_change":"2024-09-21T03:43:01.568120+0000","last_active":"2024-09-21T03:44:27.418315+0000","last_peered":"2024-09-21T03:44:27.418315+0000","last_clean":"2024-09-21T03:44:27.418315+0000","last_became_active":"2024-09-21T03:43:01.476311+0000","last_became_peered":"2024-09-21T03:43:01.476311+0000","last_unstale":"2024-09-21T03:44:27.418315+0000","last_undegraded":"2024-09-21T03:44:27.418315+0000","last_fullsized":"2024-09-21T03:44:27.418315+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:39:30.121380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:39:30.121380+0000","last_clean_scrub_stamp":"2024-09-21T03:39:30.121380+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:40:27.967442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"77'6","reported_seq":32,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:44:27.613250+0000","last_change":"2024-09-21T03:44:12.311860+0000","last_active":"2024-09-21T03:44:27.613250+0000","last_peered":"2024-09-21T03:44:27.613250+0000","last_clean":"2024-09-21T03:44:27.613250+0000","last_became_active":"2024-09-21T03:44:12.311453+0000","last_became_peered":"2024-09-21T03:44:12.311453+0000","last_unstale":"2024-09-21T03:44:27.613250+0000","last_undegraded":"2024-09-21T03:44:27.613250+0000","last_fullsized":"2024-09-21T03:44:27.613250+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:11.282438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:11.282438+0000","last_clean_scrub_stamp":"2024-09-21T03:44:11.282438+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:23:40.550445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:44:27.418319+0000","last_change":"2024-09-21T03:44:27.416799+0000","last_active":"2024-09-21T03:44:27.418319+0000","last_peered":"2024-09-21T03:44:27.418319+0000","last_clean":"2024-09-21T03:44:27.418319+0000","last_became_active":"2024-09-21T03:44:27.416306+0000","last_became_peered":"2024-09-21T03:44:27.416306+0000","last_unstale":"2024-09-21T03:44:27.418319+0000","last_undegraded":"2024-09-21T03:44:27.418319+0000","last_fullsized":"2024-09-21T03:44:27.418319+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_clean_scrub_stamp":"2024-09-21T03:44:13.292303+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:34:26.547100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,5,1],"acting":[9,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.0","version":"77'4","reported_seq":27,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:44:27.418313+0000","last_change":"2024-09-21T03:44:12.307302+0000","last_active":"2024-09-21T03:44:27.418313+0000","last_peered":"2024-09-21T03:44:27.418313+0000","last_clean":"2024-09-21T03:44:27.418313+0000","last_became_active":"2024-09-21T03:44:12.306856+0000","last_became_peered":"2024-09-21T03:44:12.306856+0000","last_unstale":"2024-09-21T03:44:27.418313+0000","last_undegraded":"2024-09-21T03:44:27.418313+0000","last_fullsized":"2024-09-21T03:44:27.418313+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:11.282438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:11.282438+0000","last_clean_scrub_stamp":"2024-09-21T03:44:11.282438+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:21:20.420020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":18,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:44:27.410210+0000","last_change":"2024-09-21T03:44:14.322913+0000","last_active":"2024-09-21T03:44:27.410210+0000","last_peered":"2024-09-21T03:44:27.410210+0000","last_clean":"2024-09-21T03:44:27.410210+0000","last_became_active":"2024-09-21T03:44:14.322168+0000","last_became_peered":"2024-09-21T03:44:14.322168+0000","last_unstale":"2024-09-21T03:44:27.410210+0000","last_undegraded":"2024-09-21T03:44:27.410210+0000","last_fullsized":"2024-09-21T03:44:27.410210+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_clean_scrub_stamp":"2024-09-21T03:44:13.292303+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:30:09.041211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"77'5","reported_seq":82,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:45:15.777934+0000","last_change":"2024-09-21T03:44:12.309084+0000","last_active":"2024-09-21T03:45:15.777934+0000","last_peered":"2024-09-21T03:45:15.777934+0000","last_clean":"2024-09-21T03:45:15.777934+0000","last_became_active":"2024-09-21T03:44:12.308551+0000","last_became_peered":"2024-09-21T03:44:12.308551+0000","last_unstale":"2024-09-21T03:45:15.777934+0000","last_undegraded":"2024-09-21T03:45:15.777934+0000","last_fullsized":"2024-09-21T03:45:15.777934+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:11.282438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:11.282438+0000","last_clean_scrub_stamp":"2024-09-21T03:44:11.282438+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:46:33.895904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":54,"num_read_kb":54,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":18,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:44:27.413911+0000","last_change":"2024-09-21T03:44:14.322720+0000","last_active":"2024-09-21T03:44:27.413911+0000","last_peered":"2024-09-21T03:44:27.413911+0000","last_clean":"2024-09-21T03:44:27.413911+0000","last_became_active":"2024-09-21T03:44:14.322367+0000","last_became_peered":"2024-09-21T03:44:14.322367+0000","last_unstale":"2024-09-21T03:44:27.413911+0000","last_undegraded":"2024-09-21T03:44:27.413911+0000","last_fullsized":"2024-09-21T03:44:27.413911+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_clean_scrub_stamp":"2024-09-21T03:44:13.292303+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:59:51.692976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"79'6","reported_seq":83,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:45:16.687016+0000","last_change":"2024-09-21T03:44:12.308719+0000","last_active":"2024-09-21T03:45:16.687016+0000","last_peered":"2024-09-21T03:45:16.687016+0000","last_clean":"2024-09-21T03:45:16.687016+0000","last_became_active":"2024-09-21T03:44:12.308380+0000","last_became_peered":"2024-09-21T03:44:12.308380+0000","last_unstale":"2024-09-21T03:45:16.687016+0000","last_undegraded":"2024-09-21T03:45:16.687016+0000","last_fullsized":"2024-09-21T03:45:16.687016+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:11.282438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:11.282438+0000","last_clean_scrub_stamp":"2024-09-21T03:44:11.282438+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:41:02.087501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":979,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":54,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":18,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:44:27.612118+0000","last_change":"2024-09-21T03:44:14.326541+0000","last_active":"2024-09-21T03:44:27.612118+0000","last_peered":"2024-09-21T03:44:27.612118+0000","last_clean":"2024-09-21T03:44:27.612118+0000","last_became_active":"2024-09-21T03:44:14.326132+0000","last_became_peered":"2024-09-21T03:44:14.326132+0000","last_unstale":"2024-09-21T03:44:27.612118+0000","last_undegraded":"2024-09-21T03:44:27.612118+0000","last_fullsized":"2024-09-21T03:44:27.612118+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_clean_scrub_stamp":"2024-09-21T03:44:13.292303+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:09:27.711919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"77'1","reported_seq":24,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:44:27.414339+0000","last_change":"2024-09-21T03:44:12.309463+0000","last_active":"2024-09-21T03:44:27.414339+0000","last_peered":"2024-09-21T03:44:27.414339+0000","last_clean":"2024-09-21T03:44:27.414339+0000","last_became_active":"2024-09-21T03:44:12.309054+0000","last_became_peered":"2024-09-21T03:44:12.309054+0000","last_unstale":"2024-09-21T03:44:27.414339+0000","last_undegraded":"2024-09-21T03:44:27.414339+0000","last_fullsized":"2024-09-21T03:44:27.414339+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:11.282438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:11.282438+0000","last_clean_scrub_stamp":"2024-09-21T03:44:11.282438+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:12:12.505538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"79'1","reported_seq":19,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:44:27.612000+0000","last_change":"2024-09-21T03:44:14.318261+0000","last_active":"2024-09-21T03:44:27.612000+0000","last_peered":"2024-09-21T03:44:27.612000+0000","last_clean":"2024-09-21T03:44:27.612000+0000","last_became_active":"2024-09-21T03:44:14.317839+0000","last_became_peered":"2024-09-21T03:44:14.317839+0000","last_unstale":"2024-09-21T03:44:27.612000+0000","last_undegraded":"2024-09-21T03:44:27.612000+0000","last_fullsized":"2024-09-21T03:44:27.612000+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_clean_scrub_stamp":"2024-09-21T03:44:13.292303+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:19:05.504749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.5","version":"77'4","reported_seq":20,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:44:27.434085+0000","last_change":"2024-09-21T03:44:27.433630+0000","last_active":"2024-09-21T03:44:27.434085+0000","last_peered":"2024-09-21T03:44:27.434085+0000","last_clean":"2024-09-21T03:44:27.434085+0000","last_became_active":"2024-09-21T03:44:27.422093+0000","last_became_peered":"2024-09-21T03:44:27.422093+0000","last_unstale":"2024-09-21T03:44:27.434085+0000","last_undegraded":"2024-09-21T03:44:27.434085+0000","last_fullsized":"2024-09-21T03:44:27.434085+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:11.282438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:11.282438+0000","last_clean_scrub_stamp":"2024-09-21T03:44:11.282438+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:39:42.651024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,9],"acting":[11,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":18,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:44:27.410677+0000","last_change":"2024-09-21T03:44:14.322620+0000","last_active":"2024-09-21T03:44:27.410677+0000","last_peered":"2024-09-21T03:44:27.410677+0000","last_clean":"2024-09-21T03:44:27.410677+0000","last_became_active":"2024-09-21T03:44:14.322214+0000","last_became_peered":"2024-09-21T03:44:14.322214+0000","last_unstale":"2024-09-21T03:44:27.410677+0000","last_undegraded":"2024-09-21T03:44:27.410677+0000","last_fullsized":"2024-09-21T03:44:27.410677+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_clean_scrub_stamp":"2024-09-21T03:44:13.292303+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:53:20.519168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"79'15","reported_seq":38,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:44:40.341077+0000","last_change":"2024-09-21T03:44:12.311302+0000","last_active":"2024-09-21T03:44:40.341077+0000","last_peered":"2024-09-21T03:44:40.341077+0000","last_clean":"2024-09-21T03:44:40.341077+0000","last_became_active":"2024-09-21T03:44:12.310947+0000","last_became_peered":"2024-09-21T03:44:12.310947+0000","last_unstale":"2024-09-21T03:44:40.341077+0000","last_undegraded":"2024-09-21T03:44:40.341077+0000","last_fullsized":"2024-09-21T03:44:40.341077+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:11.282438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:11.282438+0000","last_clean_scrub_stamp":"2024-09-21T03:44:11.282438+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:09:34.308726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":26,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":18,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:44:27.611222+0000","last_change":"2024-09-21T03:44:14.324675+0000","last_active":"2024-09-21T03:44:27.611222+0000","last_peered":"2024-09-21T03:44:27.611222+0000","last_clean":"2024-09-21T03:44:27.611222+0000","last_became_active":"2024-09-21T03:44:14.324214+0000","last_became_peered":"2024-09-21T03:44:14.324214+0000","last_unstale":"2024-09-21T03:44:27.611222+0000","last_undegraded":"2024-09-21T03:44:27.611222+0000","last_fullsized":"2024-09-21T03:44:27.611222+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_clean_scrub_stamp":"2024-09-21T03:44:13.292303+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:33:51.071794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"77'7","reported_seq":31,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:44:27.613637+0000","last_change":"2024-09-21T03:44:12.322312+0000","last_active":"2024-09-21T03:44:27.613637+0000","last_peered":"2024-09-21T03:44:27.613637+0000","last_clean":"2024-09-21T03:44:27.613637+0000","last_became_active":"2024-09-21T03:44:12.322018+0000","last_became_peered":"2024-09-21T03:44:12.322018+0000","last_unstale":"2024-09-21T03:44:27.613637+0000","last_undegraded":"2024-09-21T03:44:27.613637+0000","last_fullsized":"2024-09-21T03:44:27.613637+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:11.282438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:11.282438+0000","last_clean_scrub_stamp":"2024-09-21T03:44:11.282438+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:11:41.817574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":18,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:44:27.417254+0000","last_change":"2024-09-21T03:44:14.326682+0000","last_active":"2024-09-21T03:44:27.417254+0000","last_peered":"2024-09-21T03:44:27.417254+0000","last_clean":"2024-09-21T03:44:27.417254+0000","last_became_active":"2024-09-21T03:44:14.326313+0000","last_became_peered":"2024-09-21T03:44:14.326313+0000","last_unstale":"2024-09-21T03:44:27.417254+0000","last_undegraded":"2024-09-21T03:44:27.417254+0000","last_fullsized":"2024-09-21T03:44:27.417254+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_clean_scrub_stamp":"2024-09-21T03:44:13.292303+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:26:44.681245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"77'2","reported_seq":36,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:44:27.436319+0000","last_change":"2024-09-21T03:44:27.436245+0000","last_active":"2024-09-21T03:44:27.436319+0000","last_peered":"2024-09-21T03:44:27.436319+0000","last_clean":"2024-09-21T03:44:27.436319+0000","last_became_active":"2024-09-21T03:44:27.422134+0000","last_became_peered":"2024-09-21T03:44:27.422134+0000","last_unstale":"2024-09-21T03:44:27.436319+0000","last_undegraded":"2024-09-21T03:44:27.436319+0000","last_fullsized":"2024-09-21T03:44:27.436319+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:11.282438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:11.282438+0000","last_clean_scrub_stamp":"2024-09-21T03:44:11.282438+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:50:22.655050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,11,0],"acting":[1,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":18,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:44:27.419704+0000","last_change":"2024-09-21T03:44:14.316312+0000","last_active":"2024-09-21T03:44:27.419704+0000","last_peered":"2024-09-21T03:44:27.419704+0000","last_clean":"2024-09-21T03:44:27.419704+0000","last_became_active":"2024-09-21T03:44:14.315929+0000","last_became_peered":"2024-09-21T03:44:14.315929+0000","last_unstale":"2024-09-21T03:44:27.419704+0000","last_undegraded":"2024-09-21T03:44:27.419704+0000","last_fullsized":"2024-09-21T03:44:27.419704+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_clean_scrub_stamp":"2024-09-21T03:44:13.292303+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:31:33.071983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"77'7","reported_seq":30,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:44:27.610761+0000","last_change":"2024-09-21T03:44:12.312655+0000","last_active":"2024-09-21T03:44:27.610761+0000","last_peered":"2024-09-21T03:44:27.610761+0000","last_clean":"2024-09-21T03:44:27.610761+0000","last_became_active":"2024-09-21T03:44:12.312196+0000","last_became_peered":"2024-09-21T03:44:12.312196+0000","last_unstale":"2024-09-21T03:44:27.610761+0000","last_undegraded":"2024-09-21T03:44:27.610761+0000","last_fullsized":"2024-09-21T03:44:27.610761+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:11.282438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:11.282438+0000","last_clean_scrub_stamp":"2024-09-21T03:44:11.282438+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:24:32.342461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":18,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:44:27.418552+0000","last_change":"2024-09-21T03:44:14.321875+0000","last_active":"2024-09-21T03:44:27.418552+0000","last_peered":"2024-09-21T03:44:27.418552+0000","last_clean":"2024-09-21T03:44:27.418552+0000","last_became_active":"2024-09-21T03:44:14.320682+0000","last_became_peered":"2024-09-21T03:44:14.320682+0000","last_unstale":"2024-09-21T03:44:27.418552+0000","last_undegraded":"2024-09-21T03:44:27.418552+0000","last_fullsized":"2024-09-21T03:44:27.418552+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_clean_scrub_stamp":"2024-09-21T03:44:13.292303+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:41:32.435288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"77'3","reported_seq":26,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:44:27.418692+0000","last_change":"2024-09-21T03:44:12.309100+0000","last_active":"2024-09-21T03:44:27.418692+0000","last_peered":"2024-09-21T03:44:27.418692+0000","last_clean":"2024-09-21T03:44:27.418692+0000","last_became_active":"2024-09-21T03:44:12.308671+0000","last_became_peered":"2024-09-21T03:44:12.308671+0000","last_unstale":"2024-09-21T03:44:27.418692+0000","last_undegraded":"2024-09-21T03:44:27.418692+0000","last_fullsized":"2024-09-21T03:44:27.418692+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:11.282438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:11.282438+0000","last_clean_scrub_stamp":"2024-09-21T03:44:11.282438+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:09:19.347980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":18,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:44:27.610670+0000","last_change":"2024-09-21T03:44:14.320170+0000","last_active":"2024-09-21T03:44:27.610670+0000","last_peered":"2024-09-21T03:44:27.610670+0000","last_clean":"2024-09-21T03:44:27.610670+0000","last_became_active":"2024-09-21T03:44:14.319794+0000","last_became_peered":"2024-09-21T03:44:14.319794+0000","last_unstale":"2024-09-21T03:44:27.610670+0000","last_undegraded":"2024-09-21T03:44:27.610670+0000","last_fullsized":"2024-09-21T03:44:27.610670+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_clean_scrub_stamp":"2024-09-21T03:44:13.292303+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:42:09.684471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"77'4","reported_seq":28,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:44:27.420077+0000","last_change":"2024-09-21T03:44:12.310851+0000","last_active":"2024-09-21T03:44:27.420077+0000","last_peered":"2024-09-21T03:44:27.420077+0000","last_clean":"2024-09-21T03:44:27.420077+0000","last_became_active":"2024-09-21T03:44:12.310302+0000","last_became_peered":"2024-09-21T03:44:12.310302+0000","last_unstale":"2024-09-21T03:44:27.420077+0000","last_undegraded":"2024-09-21T03:44:27.420077+0000","last_fullsized":"2024-09-21T03:44:27.420077+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:11.282438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:11.282438+0000","last_clean_scrub_stamp":"2024-09-21T03:44:11.282438+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:23:46.959121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":18,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:44:27.418894+0000","last_change":"2024-09-21T03:44:14.324544+0000","last_active":"2024-09-21T03:44:27.418894+0000","last_peered":"2024-09-21T03:44:27.418894+0000","last_clean":"2024-09-21T03:44:27.418894+0000","last_became_active":"2024-09-21T03:44:14.322881+0000","last_became_peered":"2024-09-21T03:44:14.322881+0000","last_unstale":"2024-09-21T03:44:27.418894+0000","last_undegraded":"2024-09-21T03:44:27.418894+0000","last_fullsized":"2024-09-21T03:44:27.418894+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_clean_scrub_stamp":"2024-09-21T03:44:13.292303+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:58:42.760185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":25,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:44:27.422566+0000","last_change":"2024-09-21T03:44:27.422438+0000","last_active":"2024-09-21T03:44:27.422566+0000","last_peered":"2024-09-21T03:44:27.422566+0000","last_clean":"2024-09-21T03:44:27.422566+0000","last_became_active":"2024-09-21T03:44:27.421935+0000","last_became_peered":"2024-09-21T03:44:27.421935+0000","last_unstale":"2024-09-21T03:44:27.422566+0000","last_undegraded":"2024-09-21T03:44:27.422566+0000","last_fullsized":"2024-09-21T03:44:27.422566+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_clean_scrub_stamp":"2024-09-21T03:44:13.292303+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:20:50.097783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,1],"acting":[5,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"77'5","reported_seq":29,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:44:27.420510+0000","last_change":"2024-09-21T03:44:12.311509+0000","last_active":"2024-09-21T03:44:27.420510+0000","last_peered":"2024-09-21T03:44:27.420510+0000","last_clean":"2024-09-21T03:44:27.420510+0000","last_became_active":"2024-09-21T03:44:12.310880+0000","last_became_peered":"2024-09-21T03:44:12.310880+0000","last_unstale":"2024-09-21T03:44:27.420510+0000","last_undegraded":"2024-09-21T03:44:27.420510+0000","last_fullsized":"2024-09-21T03:44:27.420510+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:11.282438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:11.282438+0000","last_clean_scrub_stamp":"2024-09-21T03:44:11.282438+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:59:33.887234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":18,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:44:27.413781+0000","last_change":"2024-09-21T03:44:14.330938+0000","last_active":"2024-09-21T03:44:27.413781+0000","last_peered":"2024-09-21T03:44:27.413781+0000","last_clean":"2024-09-21T03:44:27.413781+0000","last_became_active":"2024-09-21T03:44:14.330559+0000","last_became_peered":"2024-09-21T03:44:14.330559+0000","last_unstale":"2024-09-21T03:44:27.413781+0000","last_undegraded":"2024-09-21T03:44:27.413781+0000","last_fullsized":"2024-09-21T03:44:27.413781+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_clean_scrub_stamp":"2024-09-21T03:44:13.292303+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:05:32.535660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"77'12","reported_seq":36,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:44:27.420563+0000","last_change":"2024-09-21T03:44:12.307667+0000","last_active":"2024-09-21T03:44:27.420563+0000","last_peered":"2024-09-21T03:44:27.420563+0000","last_clean":"2024-09-21T03:44:27.420563+0000","last_became_active":"2024-09-21T03:44:12.307074+0000","last_became_peered":"2024-09-21T03:44:12.307074+0000","last_unstale":"2024-09-21T03:44:27.420563+0000","last_undegraded":"2024-09-21T03:44:27.420563+0000","last_fullsized":"2024-09-21T03:44:27.420563+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:11.282438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:11.282438+0000","last_clean_scrub_stamp":"2024-09-21T03:44:11.282438+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:48:41.598375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":18,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:44:27.417322+0000","last_change":"2024-09-21T03:44:14.325487+0000","last_active":"2024-09-21T03:44:27.417322+0000","last_peered":"2024-09-21T03:44:27.417322+0000","last_clean":"2024-09-21T03:44:27.417322+0000","last_became_active":"2024-09-21T03:44:14.325153+0000","last_became_peered":"2024-09-21T03:44:14.325153+0000","last_unstale":"2024-09-21T03:44:27.417322+0000","last_undegraded":"2024-09-21T03:44:27.417322+0000","last_fullsized":"2024-09-21T03:44:27.417322+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_clean_scrub_stamp":"2024-09-21T03:44:13.292303+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:29:17.619130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"77'2","reported_seq":25,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:44:27.611241+0000","last_change":"2024-09-21T03:44:12.307635+0000","last_active":"2024-09-21T03:44:27.611241+0000","last_peered":"2024-09-21T03:44:27.611241+0000","last_clean":"2024-09-21T03:44:27.611241+0000","last_became_active":"2024-09-21T03:44:12.307235+0000","last_became_peered":"2024-09-21T03:44:12.307235+0000","last_unstale":"2024-09-21T03:44:27.611241+0000","last_undegraded":"2024-09-21T03:44:27.611241+0000","last_fullsized":"2024-09-21T03:44:27.611241+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:11.282438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:11.282438+0000","last_clean_scrub_stamp":"2024-09-21T03:44:11.282438+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:18:59.985369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":18,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:44:27.611117+0000","last_change":"2024-09-21T03:44:14.324660+0000","last_active":"2024-09-21T03:44:27.611117+0000","last_peered":"2024-09-21T03:44:27.611117+0000","last_clean":"2024-09-21T03:44:27.611117+0000","last_became_active":"2024-09-21T03:44:14.324309+0000","last_became_peered":"2024-09-21T03:44:14.324309+0000","last_unstale":"2024-09-21T03:44:27.611117+0000","last_undegraded":"2024-09-21T03:44:27.611117+0000","last_fullsized":"2024-09-21T03:44:27.611117+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_clean_scrub_stamp":"2024-09-21T03:44:13.292303+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:13:07.806721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"77'13","reported_seq":107,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:45:13.681915+0000","last_change":"2024-09-21T03:44:27.623910+0000","last_active":"2024-09-21T03:45:13.681915+0000","last_peered":"2024-09-21T03:45:13.681915+0000","last_clean":"2024-09-21T03:45:13.681915+0000","last_became_active":"2024-09-21T03:44:27.614037+0000","last_became_peered":"2024-09-21T03:44:27.614037+0000","last_unstale":"2024-09-21T03:45:13.681915+0000","last_undegraded":"2024-09-21T03:45:13.681915+0000","last_fullsized":"2024-09-21T03:45:13.681915+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:11.282438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:11.282438+0000","last_clean_scrub_stamp":"2024-09-21T03:44:11.282438+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:03:56.753935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":49,"num_read_kb":49,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":672,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,4],"acting":[9,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":71497,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":566,"num_read_kb":359,"num_write":244,"num_write_kb":141,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13,"num_bytes_recovered":1954,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":577536,"data_stored":214491,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":210,"ondisk_log_size":210,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":71,"seq":304942678042,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27892,"kb_used_data":864,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717260,"statfs":{"total":95995035648,"available":95966474240,"internally_reserved":0,"allocated":884736,"data_stored":428272,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":65,"seq":279172874271,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28568,"kb_used_data":1544,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716584,"statfs":{"total":95995035648,"available":95965782016,"internally_reserved":0,"allocated":1581056,"data_stored":1083482,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":60,"seq":257698037795,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27904,"kb_used_data":880,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717248,"statfs":{"total":95995035648,"available":95966461952,"internally_reserved":0,"allocated":901120,"data_stored":428090,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":54,"seq":231928234024,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27884,"kb_used_data":860,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717268,"statfs":{"total":95995035648,"available":95966482432,"internally_reserved":0,"allocated":880640,"data_stored":427684,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":49,"seq":210453397552,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36804,"kb_used_data":1460,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708348,"statfs":{"total":95995035648,"available":95957348352,"internally_reserved":0,"allocated":1495040,"data_stored":1021768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":36174282},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":43,"seq":184683593780,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27888,"kb_used_data":864,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717264,"statfs":{"total":95995035648,"available":95966478336,"internally_reserved":0,"allocated":884736,"data_stored":428198,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":37,"seq":158913790010,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32656,"kb_used_data":1472,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712496,"statfs":{"total":95995035648,"available":95961595904,"internally_reserved":0,"allocated":1507328,"data_stored":1020664,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":31,"seq":133143986238,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36224,"kb_used_data":872,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708928,"statfs":{"total":95995035648,"available":95957942272,"internally_reserved":0,"allocated":892928,"data_stored":428731,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149764,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32064,"kb_used_data":880,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713088,"statfs":{"total":95995035648,"available":95962202112,"internally_reserved":0,"allocated":901120,"data_stored":427632,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":19,"seq":81604378697,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27964,"kb_used_data":940,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717188,"statfs":{"total":95995035648,"available":95966400512,"internally_reserved":0,"allocated":962560,"data_stored":492063,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542222,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27960,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717192,"statfs":{"total":95995035648,"available":95966404608,"internally_reserved":0,"allocated":958464,"data_stored":494183,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705748,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27912,"kb_used_data":884,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717240,"statfs":{"total":95995035648,"available":95966453760,"internally_reserved":0,"allocated":905216,"data_stored":429146,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2178,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":67215,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":65095,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1763,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3211,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1113,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":4315,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":716,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":118784,"data_stored":66029,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1304,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T03:45:21.903 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 51116d5c-77ca-11ef-baf5-efdc52797490 -- ceph pg dump --format=json 2024-09-21T03:45:22.182 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/51116d5c-77ca-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:45:22.217 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:45:21 smithi161 ceph-mon[34533]: from='client.15186 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T03:45:22.218 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:45:21 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:45:22.218 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:45:21 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:45:22.224 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:45:21 smithi150 ceph-mon[34289]: from='client.15186 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T03:45:22.225 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:45:21 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:45:22.225 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:45:21 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:45:22.698 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:22.689+0000 7f3d0857f640 1 -- 172.21.15.39:0/378922763 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3d001094a0 msgr2=0x7f3d001098b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:22.698 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:22.689+0000 7f3d0857f640 1 --2- 172.21.15.39:0/378922763 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3d001094a0 0x7f3d001098b0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f3cf0009900 tx=0x7f3cf002a410 comp rx=0 tx=0).stop 2024-09-21T03:45:22.698 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:22.689+0000 7f3d0857f640 1 -- 172.21.15.39:0/378922763 shutdown_connections 2024-09-21T03:45:22.698 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:22.689+0000 7f3d0857f640 1 --2- 172.21.15.39:0/378922763 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3d00109df0 0x7f3d00110620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:22.698 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:22.689+0000 7f3d0857f640 1 --2- 172.21.15.39:0/378922763 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3d001094a0 0x7f3d001098b0 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:22.698 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:22.689+0000 7f3d0857f640 1 --2- 172.21.15.39:0/378922763 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3d00103b60 0x7f3d00103f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:22.699 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:22.689+0000 7f3d0857f640 1 -- 172.21.15.39:0/378922763 >> 172.21.15.39:0/378922763 conn(0x7f3d0007b850 msgr2=0x7f3d0007bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:45:22.699 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:22.689+0000 7f3d0857f640 1 -- 172.21.15.39:0/378922763 shutdown_connections 2024-09-21T03:45:22.699 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:22.689+0000 7f3d0857f640 1 -- 172.21.15.39:0/378922763 wait complete. 2024-09-21T03:45:22.699 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:22.690+0000 7f3d0857f640 1 Processor -- start 2024-09-21T03:45:22.699 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:22.690+0000 7f3d0857f640 1 -- start start 2024-09-21T03:45:22.699 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:22.690+0000 7f3d0857f640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3d00103b60 0x7f3d00075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:22.699 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:22.690+0000 7f3d0857f640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3d001094a0 0x7f3d00075c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:22.699 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:22.690+0000 7f3d0857f640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3d00109df0 0x7f3d00076190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:22.699 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:22.690+0000 7f3d0857f640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f3d00113ab0 con 0x7f3d00103b60 2024-09-21T03:45:22.699 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:22.690+0000 7f3d0857f640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f3d00113930 con 0x7f3d001094a0 2024-09-21T03:45:22.699 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:22.690+0000 7f3d0857f640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f3d00113c30 con 0x7f3d00109df0 2024-09-21T03:45:22.699 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:22.690+0000 7f3d0631b640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3d00103b60 0x7f3d00075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:22.699 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:22.690+0000 7f3d0631b640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3d00103b60 0x7f3d00075710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:60272/0 (socket says 172.21.15.39:60272) 2024-09-21T03:45:22.699 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:22.690+0000 7f3d0631b640 1 -- 172.21.15.39:0/2809818632 learned_addr learned my addr 172.21.15.39:0/2809818632 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:45:22.700 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:22.691+0000 7f3d05b1a640 1 --2- 172.21.15.39:0/2809818632 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3d001094a0 0x7f3d00075c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:22.700 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:22.691+0000 7f3d0631b640 1 -- 172.21.15.39:0/2809818632 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3d00109df0 msgr2=0x7f3d00076190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:22.700 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:22.691+0000 7f3d06b1c640 1 --2- 172.21.15.39:0/2809818632 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3d00109df0 0x7f3d00076190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:22.700 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:22.691+0000 7f3d0631b640 1 --2- 172.21.15.39:0/2809818632 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3d00109df0 0x7f3d00076190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:22.700 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:22.691+0000 7f3d0631b640 1 -- 172.21.15.39:0/2809818632 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3d001094a0 msgr2=0x7f3d00075c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:22.700 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:22.691+0000 7f3d0631b640 1 --2- 172.21.15.39:0/2809818632 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3d001094a0 0x7f3d00075c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:22.700 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:22.691+0000 7f3d0631b640 1 -- 172.21.15.39:0/2809818632 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3d001a8820 con 0x7f3d00103b60 2024-09-21T03:45:22.700 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:22.691+0000 7f3d05b1a640 1 --2- 172.21.15.39:0/2809818632 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3d001094a0 0x7f3d00075c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:45:22.700 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:22.691+0000 7f3d0631b640 1 --2- 172.21.15.39:0/2809818632 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3d00103b60 0x7f3d00075710 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f3cfc00d950 tx=0x7f3cfc00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:45:22.700 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:22.691+0000 7f3cf77fe640 1 -- 172.21.15.39:0/2809818632 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3cfc017070 con 0x7f3d00103b60 2024-09-21T03:45:22.700 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:22.691+0000 7f3d0857f640 1 -- 172.21.15.39:0/2809818632 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3d001a8b10 con 0x7f3d00103b60 2024-09-21T03:45:22.700 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:22.691+0000 7f3cf77fe640 1 -- 172.21.15.39:0/2809818632 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3cfc009c30 con 0x7f3d00103b60 2024-09-21T03:45:22.700 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:22.691+0000 7f3cf77fe640 1 -- 172.21.15.39:0/2809818632 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3cfc00f620 con 0x7f3d00103b60 2024-09-21T03:45:22.700 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:22.691+0000 7f3d0857f640 1 -- 172.21.15.39:0/2809818632 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3d001a9020 con 0x7f3d00103b60 2024-09-21T03:45:22.701 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:22.692+0000 7f3d0857f640 1 -- 172.21.15.39:0/2809818632 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3d00103f70 con 0x7f3d00103b60 2024-09-21T03:45:22.701 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:22.694+0000 7f3cf77fe640 1 -- 172.21.15.39:0/2809818632 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 22) ==== 154580+0+0 (secure 0 0 0) 0x7f3cfc005030 con 0x7f3d00103b60 2024-09-21T03:45:22.701 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:22.695+0000 7f3cf77fe640 1 --2- 172.21.15.39:0/2809818632 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f3cec0b65f0 0x7f3cec0b8b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:22.701 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:22.695+0000 7f3d05b1a640 1 --2- 172.21.15.39:0/2809818632 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f3cec0b65f0 0x7f3cec0b8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:22.701 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:22.695+0000 7f3cf77fe640 1 -- 172.21.15.39:0/2809818632 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8484+0+0 (secure 0 0 0) 0x7f3cfc0e72a0 con 0x7f3d00103b60 2024-09-21T03:45:22.701 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:22.695+0000 7f3d05b1a640 1 --2- 172.21.15.39:0/2809818632 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f3cec0b65f0 0x7f3cec0b8b00 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f3cf0009230 tx=0x7f3cf0000bc0 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:45:22.701 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:22.698+0000 7f3cf77fe640 1 -- 172.21.15.39:0/2809818632 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f3cfc0ad8d0 con 0x7f3d00103b60 2024-09-21T03:45:22.888 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:22.886+0000 7f3d0857f640 1 -- 172.21.15.39:0/2809818632 --> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f3d00063800 con 0x7f3cec0b65f0 2024-09-21T03:45:22.893 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:22.890+0000 7f3cf77fe640 1 -- 172.21.15.39:0/2809818632 <== mgr.14223 v2:172.21.15.39:6800/1696381982 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258757 (secure 0 0 0) 0x7f3d00063800 con 0x7f3cec0b65f0 2024-09-21T03:45:22.895 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T03:45:22.896 INFO:teuthology.orchestra.run.smithi039.stderr:dumped all 2024-09-21T03:45:22.898 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:22.894+0000 7f3d0857f640 1 -- 172.21.15.39:0/2809818632 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f3cec0b65f0 msgr2=0x7f3cec0b8b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:22.899 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:22.894+0000 7f3d0857f640 1 --2- 172.21.15.39:0/2809818632 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f3cec0b65f0 0x7f3cec0b8b00 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f3cf0009230 tx=0x7f3cf0000bc0 comp rx=0 tx=0).stop 2024-09-21T03:45:22.899 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:22.894+0000 7f3d0857f640 1 -- 172.21.15.39:0/2809818632 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3d00103b60 msgr2=0x7f3d00075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:22.899 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:22.894+0000 7f3d0857f640 1 --2- 172.21.15.39:0/2809818632 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3d00103b60 0x7f3d00075710 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f3cfc00d950 tx=0x7f3cfc00de10 comp rx=0 tx=0).stop 2024-09-21T03:45:22.899 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:22.894+0000 7f3d0857f640 1 -- 172.21.15.39:0/2809818632 shutdown_connections 2024-09-21T03:45:22.899 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:22.894+0000 7f3d0857f640 1 --2- 172.21.15.39:0/2809818632 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f3cec0b65f0 0x7f3cec0b8b00 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:22.899 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:22.894+0000 7f3d0857f640 1 --2- 172.21.15.39:0/2809818632 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3d00109df0 0x7f3d00076190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:22.899 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:22.894+0000 7f3d0857f640 1 --2- 172.21.15.39:0/2809818632 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3d001094a0 0x7f3d00075c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:22.899 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:22.894+0000 7f3d0857f640 1 --2- 172.21.15.39:0/2809818632 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3d00103b60 0x7f3d00075710 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:22.899 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:22.894+0000 7f3d0857f640 1 -- 172.21.15.39:0/2809818632 >> 172.21.15.39:0/2809818632 conn(0x7f3d0007b850 msgr2=0x7f3d00105560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:45:22.899 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:22.894+0000 7f3d0857f640 1 -- 172.21.15.39:0/2809818632 shutdown_connections 2024-09-21T03:45:22.899 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:22.894+0000 7f3d0857f640 1 -- 172.21.15.39:0/2809818632 wait complete. 2024-09-21T03:45:23.084 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:45:22 smithi039 ceph-mon[31211]: pgmap v305: 97 pgs: 97 active+clean; 647 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-21T03:45:23.084 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:45:22 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:45:23.084 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:45:22 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:45:23.084 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:45:22 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:45:23.218 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:45:22 smithi161 ceph-mon[34533]: pgmap v305: 97 pgs: 97 active+clean; 647 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-21T03:45:23.218 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:45:22 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:45:23.218 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:45:22 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:45:23.218 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:45:22 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:45:23.224 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:45:22 smithi150 ceph-mon[34289]: pgmap v305: 97 pgs: 97 active+clean; 647 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-21T03:45:23.224 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:45:22 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:45:23.225 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:45:22 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:45:23.225 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:45:22 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:45:23.592 INFO:teuthology.orchestra.run.smithi039.stdout:{"pg_ready":true,"pg_map":{"version":305,"stamp":"2024-09-21T03:45:21.514950+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":662099,"num_objects":102,"num_object_clones":0,"num_object_copies":306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":657,"num_read_kb":433,"num_write":369,"num_write_kb":2293,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":23,"num_bytes_recovered":2953794,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":304,"ondisk_log_size":304,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":361720,"kb_used_data":12456,"kb_used_omap":18,"kb_used_meta":349037,"kb_avail":1124580104,"statfs":{"total":1151940427776,"available":1151570026496,"internally_reserved":0,"allocated":12754944,"data_stored":7109913,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19143,"internal_metadata":357414201},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":109,"num_read_kb":55,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.014903"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":18,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:44:27.420991+0000","last_change":"2024-09-21T03:44:14.327553+0000","last_active":"2024-09-21T03:44:27.420991+0000","last_peered":"2024-09-21T03:44:27.420991+0000","last_clean":"2024-09-21T03:44:27.420991+0000","last_became_active":"2024-09-21T03:44:14.327213+0000","last_became_peered":"2024-09-21T03:44:14.327213+0000","last_unstale":"2024-09-21T03:44:27.420991+0000","last_undegraded":"2024-09-21T03:44:27.420991+0000","last_fullsized":"2024-09-21T03:44:27.420991+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_clean_scrub_stamp":"2024-09-21T03:44:13.292303+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:40:56.550897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":18,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:44:27.417200+0000","last_change":"2024-09-21T03:44:14.321427+0000","last_active":"2024-09-21T03:44:27.417200+0000","last_peered":"2024-09-21T03:44:27.417200+0000","last_clean":"2024-09-21T03:44:27.417200+0000","last_became_active":"2024-09-21T03:44:14.321026+0000","last_became_peered":"2024-09-21T03:44:14.321026+0000","last_unstale":"2024-09-21T03:44:27.417200+0000","last_undegraded":"2024-09-21T03:44:27.417200+0000","last_fullsized":"2024-09-21T03:44:27.417200+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_clean_scrub_stamp":"2024-09-21T03:44:13.292303+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:18:00.889714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":18,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:44:27.416317+0000","last_change":"2024-09-21T03:44:14.324908+0000","last_active":"2024-09-21T03:44:27.416317+0000","last_peered":"2024-09-21T03:44:27.416317+0000","last_clean":"2024-09-21T03:44:27.416317+0000","last_became_active":"2024-09-21T03:44:14.324533+0000","last_became_peered":"2024-09-21T03:44:14.324533+0000","last_unstale":"2024-09-21T03:44:27.416317+0000","last_undegraded":"2024-09-21T03:44:27.416317+0000","last_fullsized":"2024-09-21T03:44:27.416317+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_clean_scrub_stamp":"2024-09-21T03:44:13.292303+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:44:25.235442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":18,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:44:27.613591+0000","last_change":"2024-09-21T03:44:14.326758+0000","last_active":"2024-09-21T03:44:27.613591+0000","last_peered":"2024-09-21T03:44:27.613591+0000","last_clean":"2024-09-21T03:44:27.613591+0000","last_became_active":"2024-09-21T03:44:14.326410+0000","last_became_peered":"2024-09-21T03:44:14.326410+0000","last_unstale":"2024-09-21T03:44:27.613591+0000","last_undegraded":"2024-09-21T03:44:27.613591+0000","last_fullsized":"2024-09-21T03:44:27.613591+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_clean_scrub_stamp":"2024-09-21T03:44:13.292303+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:13:22.716940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":18,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:44:27.419914+0000","last_change":"2024-09-21T03:44:14.318875+0000","last_active":"2024-09-21T03:44:27.419914+0000","last_peered":"2024-09-21T03:44:27.419914+0000","last_clean":"2024-09-21T03:44:27.419914+0000","last_became_active":"2024-09-21T03:44:14.318425+0000","last_became_peered":"2024-09-21T03:44:14.318425+0000","last_unstale":"2024-09-21T03:44:27.419914+0000","last_undegraded":"2024-09-21T03:44:27.419914+0000","last_fullsized":"2024-09-21T03:44:27.419914+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_clean_scrub_stamp":"2024-09-21T03:44:13.292303+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:45:39.916587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":18,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:44:27.611598+0000","last_change":"2024-09-21T03:44:14.325877+0000","last_active":"2024-09-21T03:44:27.611598+0000","last_peered":"2024-09-21T03:44:27.611598+0000","last_clean":"2024-09-21T03:44:27.611598+0000","last_became_active":"2024-09-21T03:44:14.325451+0000","last_became_peered":"2024-09-21T03:44:14.325451+0000","last_unstale":"2024-09-21T03:44:27.611598+0000","last_undegraded":"2024-09-21T03:44:27.611598+0000","last_fullsized":"2024-09-21T03:44:27.611598+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_clean_scrub_stamp":"2024-09-21T03:44:13.292303+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:30:42.427054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":18,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:44:27.418241+0000","last_change":"2024-09-21T03:44:14.323192+0000","last_active":"2024-09-21T03:44:27.418241+0000","last_peered":"2024-09-21T03:44:27.418241+0000","last_clean":"2024-09-21T03:44:27.418241+0000","last_became_active":"2024-09-21T03:44:14.322487+0000","last_became_peered":"2024-09-21T03:44:14.322487+0000","last_unstale":"2024-09-21T03:44:27.418241+0000","last_undegraded":"2024-09-21T03:44:27.418241+0000","last_fullsized":"2024-09-21T03:44:27.418241+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_clean_scrub_stamp":"2024-09-21T03:44:13.292303+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:10:48.770536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:44:27.613320+0000","last_change":"2024-09-21T03:44:27.612681+0000","last_active":"2024-09-21T03:44:27.613320+0000","last_peered":"2024-09-21T03:44:27.613320+0000","last_clean":"2024-09-21T03:44:27.613320+0000","last_became_active":"2024-09-21T03:44:27.612200+0000","last_became_peered":"2024-09-21T03:44:27.612200+0000","last_unstale":"2024-09-21T03:44:27.613320+0000","last_undegraded":"2024-09-21T03:44:27.613320+0000","last_fullsized":"2024-09-21T03:44:27.613320+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_clean_scrub_stamp":"2024-09-21T03:44:13.292303+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:44:58.848571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,5,10],"acting":[9,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":18,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:44:27.414086+0000","last_change":"2024-09-21T03:44:14.325820+0000","last_active":"2024-09-21T03:44:27.414086+0000","last_peered":"2024-09-21T03:44:27.414086+0000","last_clean":"2024-09-21T03:44:27.414086+0000","last_became_active":"2024-09-21T03:44:14.325070+0000","last_became_peered":"2024-09-21T03:44:14.325070+0000","last_unstale":"2024-09-21T03:44:27.414086+0000","last_undegraded":"2024-09-21T03:44:27.414086+0000","last_fullsized":"2024-09-21T03:44:27.414086+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_clean_scrub_stamp":"2024-09-21T03:44:13.292303+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:54:01.072623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":18,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:44:27.612466+0000","last_change":"2024-09-21T03:44:14.323746+0000","last_active":"2024-09-21T03:44:27.612466+0000","last_peered":"2024-09-21T03:44:27.612466+0000","last_clean":"2024-09-21T03:44:27.612466+0000","last_became_active":"2024-09-21T03:44:14.323346+0000","last_became_peered":"2024-09-21T03:44:14.323346+0000","last_unstale":"2024-09-21T03:44:27.612466+0000","last_undegraded":"2024-09-21T03:44:27.612466+0000","last_fullsized":"2024-09-21T03:44:27.612466+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_clean_scrub_stamp":"2024-09-21T03:44:13.292303+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:17:44.603801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":18,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:44:27.419043+0000","last_change":"2024-09-21T03:44:14.328733+0000","last_active":"2024-09-21T03:44:27.419043+0000","last_peered":"2024-09-21T03:44:27.419043+0000","last_clean":"2024-09-21T03:44:27.419043+0000","last_became_active":"2024-09-21T03:44:14.328378+0000","last_became_peered":"2024-09-21T03:44:14.328378+0000","last_unstale":"2024-09-21T03:44:27.419043+0000","last_undegraded":"2024-09-21T03:44:27.419043+0000","last_fullsized":"2024-09-21T03:44:27.419043+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_clean_scrub_stamp":"2024-09-21T03:44:13.292303+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:23:38.497413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":18,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:44:27.613597+0000","last_change":"2024-09-21T03:44:14.328505+0000","last_active":"2024-09-21T03:44:27.613597+0000","last_peered":"2024-09-21T03:44:27.613597+0000","last_clean":"2024-09-21T03:44:27.613597+0000","last_became_active":"2024-09-21T03:44:14.328175+0000","last_became_peered":"2024-09-21T03:44:14.328175+0000","last_unstale":"2024-09-21T03:44:27.613597+0000","last_undegraded":"2024-09-21T03:44:27.613597+0000","last_fullsized":"2024-09-21T03:44:27.613597+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_clean_scrub_stamp":"2024-09-21T03:44:13.292303+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:45:56.628962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":18,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:44:27.410517+0000","last_change":"2024-09-21T03:44:14.314519+0000","last_active":"2024-09-21T03:44:27.410517+0000","last_peered":"2024-09-21T03:44:27.410517+0000","last_clean":"2024-09-21T03:44:27.410517+0000","last_became_active":"2024-09-21T03:44:14.314145+0000","last_became_peered":"2024-09-21T03:44:14.314145+0000","last_unstale":"2024-09-21T03:44:27.410517+0000","last_undegraded":"2024-09-21T03:44:27.410517+0000","last_fullsized":"2024-09-21T03:44:27.410517+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_clean_scrub_stamp":"2024-09-21T03:44:13.292303+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:45:48.494667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":18,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:44:27.416665+0000","last_change":"2024-09-21T03:44:14.312195+0000","last_active":"2024-09-21T03:44:27.416665+0000","last_peered":"2024-09-21T03:44:27.416665+0000","last_clean":"2024-09-21T03:44:27.416665+0000","last_became_active":"2024-09-21T03:44:14.311861+0000","last_became_peered":"2024-09-21T03:44:14.311861+0000","last_unstale":"2024-09-21T03:44:27.416665+0000","last_undegraded":"2024-09-21T03:44:27.416665+0000","last_fullsized":"2024-09-21T03:44:27.416665+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_clean_scrub_stamp":"2024-09-21T03:44:13.292303+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:17:05.025898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":18,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:44:27.416311+0000","last_change":"2024-09-21T03:44:14.323532+0000","last_active":"2024-09-21T03:44:27.416311+0000","last_peered":"2024-09-21T03:44:27.416311+0000","last_clean":"2024-09-21T03:44:27.416311+0000","last_became_active":"2024-09-21T03:44:14.323171+0000","last_became_peered":"2024-09-21T03:44:14.323171+0000","last_unstale":"2024-09-21T03:44:27.416311+0000","last_undegraded":"2024-09-21T03:44:27.416311+0000","last_fullsized":"2024-09-21T03:44:27.416311+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_clean_scrub_stamp":"2024-09-21T03:44:13.292303+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:23:28.999129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":18,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:44:27.413494+0000","last_change":"2024-09-21T03:44:14.324998+0000","last_active":"2024-09-21T03:44:27.413494+0000","last_peered":"2024-09-21T03:44:27.413494+0000","last_clean":"2024-09-21T03:44:27.413494+0000","last_became_active":"2024-09-21T03:44:14.324019+0000","last_became_peered":"2024-09-21T03:44:14.324019+0000","last_unstale":"2024-09-21T03:44:27.413494+0000","last_undegraded":"2024-09-21T03:44:27.413494+0000","last_fullsized":"2024-09-21T03:44:27.413494+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_clean_scrub_stamp":"2024-09-21T03:44:13.292303+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:49:12.965299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":18,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:44:27.421157+0000","last_change":"2024-09-21T03:44:14.325603+0000","last_active":"2024-09-21T03:44:27.421157+0000","last_peered":"2024-09-21T03:44:27.421157+0000","last_clean":"2024-09-21T03:44:27.421157+0000","last_became_active":"2024-09-21T03:44:14.325208+0000","last_became_peered":"2024-09-21T03:44:14.325208+0000","last_unstale":"2024-09-21T03:44:27.421157+0000","last_undegraded":"2024-09-21T03:44:27.421157+0000","last_fullsized":"2024-09-21T03:44:27.421157+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_clean_scrub_stamp":"2024-09-21T03:44:13.292303+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:11:33.794242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":18,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:44:27.414581+0000","last_change":"2024-09-21T03:44:14.317545+0000","last_active":"2024-09-21T03:44:27.414581+0000","last_peered":"2024-09-21T03:44:27.414581+0000","last_clean":"2024-09-21T03:44:27.414581+0000","last_became_active":"2024-09-21T03:44:14.317069+0000","last_became_peered":"2024-09-21T03:44:14.317069+0000","last_unstale":"2024-09-21T03:44:27.414581+0000","last_undegraded":"2024-09-21T03:44:27.414581+0000","last_fullsized":"2024-09-21T03:44:27.414581+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_clean_scrub_stamp":"2024-09-21T03:44:13.292303+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:16:11.734887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":18,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:44:27.418423+0000","last_change":"2024-09-21T03:44:14.322055+0000","last_active":"2024-09-21T03:44:27.418423+0000","last_peered":"2024-09-21T03:44:27.418423+0000","last_clean":"2024-09-21T03:44:27.418423+0000","last_became_active":"2024-09-21T03:44:14.321652+0000","last_became_peered":"2024-09-21T03:44:14.321652+0000","last_unstale":"2024-09-21T03:44:27.418423+0000","last_undegraded":"2024-09-21T03:44:27.418423+0000","last_fullsized":"2024-09-21T03:44:27.418423+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_clean_scrub_stamp":"2024-09-21T03:44:13.292303+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:45:14.285844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":25,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:44:27.423315+0000","last_change":"2024-09-21T03:44:27.423175+0000","last_active":"2024-09-21T03:44:27.423315+0000","last_peered":"2024-09-21T03:44:27.423315+0000","last_clean":"2024-09-21T03:44:27.423315+0000","last_became_active":"2024-09-21T03:44:27.422733+0000","last_became_peered":"2024-09-21T03:44:27.422733+0000","last_unstale":"2024-09-21T03:44:27.423315+0000","last_undegraded":"2024-09-21T03:44:27.423315+0000","last_fullsized":"2024-09-21T03:44:27.423315+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_clean_scrub_stamp":"2024-09-21T03:44:13.292303+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:46:17.874557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,8],"acting":[11,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":18,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:44:27.410602+0000","last_change":"2024-09-21T03:44:14.325829+0000","last_active":"2024-09-21T03:44:27.410602+0000","last_peered":"2024-09-21T03:44:27.410602+0000","last_clean":"2024-09-21T03:44:27.410602+0000","last_became_active":"2024-09-21T03:44:14.325421+0000","last_became_peered":"2024-09-21T03:44:14.325421+0000","last_unstale":"2024-09-21T03:44:27.410602+0000","last_undegraded":"2024-09-21T03:44:27.410602+0000","last_fullsized":"2024-09-21T03:44:27.410602+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_clean_scrub_stamp":"2024-09-21T03:44:13.292303+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:49:00.240771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":18,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:44:27.413691+0000","last_change":"2024-09-21T03:44:14.325255+0000","last_active":"2024-09-21T03:44:27.413691+0000","last_peered":"2024-09-21T03:44:27.413691+0000","last_clean":"2024-09-21T03:44:27.413691+0000","last_became_active":"2024-09-21T03:44:14.324617+0000","last_became_peered":"2024-09-21T03:44:14.324617+0000","last_unstale":"2024-09-21T03:44:27.413691+0000","last_undegraded":"2024-09-21T03:44:27.413691+0000","last_fullsized":"2024-09-21T03:44:27.413691+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_clean_scrub_stamp":"2024-09-21T03:44:13.292303+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:05:03.268522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":18,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:44:27.610583+0000","last_change":"2024-09-21T03:44:14.328053+0000","last_active":"2024-09-21T03:44:27.610583+0000","last_peered":"2024-09-21T03:44:27.610583+0000","last_clean":"2024-09-21T03:44:27.610583+0000","last_became_active":"2024-09-21T03:44:14.327735+0000","last_became_peered":"2024-09-21T03:44:14.327735+0000","last_unstale":"2024-09-21T03:44:27.610583+0000","last_undegraded":"2024-09-21T03:44:27.610583+0000","last_fullsized":"2024-09-21T03:44:27.610583+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_clean_scrub_stamp":"2024-09-21T03:44:13.292303+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:39:48.395784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":18,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:44:27.413859+0000","last_change":"2024-09-21T03:44:14.321181+0000","last_active":"2024-09-21T03:44:27.413859+0000","last_peered":"2024-09-21T03:44:27.413859+0000","last_clean":"2024-09-21T03:44:27.413859+0000","last_became_active":"2024-09-21T03:44:14.320772+0000","last_became_peered":"2024-09-21T03:44:14.320772+0000","last_unstale":"2024-09-21T03:44:27.413859+0000","last_undegraded":"2024-09-21T03:44:27.413859+0000","last_fullsized":"2024-09-21T03:44:27.413859+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_clean_scrub_stamp":"2024-09-21T03:44:13.292303+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:49:26.317404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":18,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:44:27.611124+0000","last_change":"2024-09-21T03:44:14.322643+0000","last_active":"2024-09-21T03:44:27.611124+0000","last_peered":"2024-09-21T03:44:27.611124+0000","last_clean":"2024-09-21T03:44:27.611124+0000","last_became_active":"2024-09-21T03:44:14.322254+0000","last_became_peered":"2024-09-21T03:44:14.322254+0000","last_unstale":"2024-09-21T03:44:27.611124+0000","last_undegraded":"2024-09-21T03:44:27.611124+0000","last_fullsized":"2024-09-21T03:44:27.611124+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_clean_scrub_stamp":"2024-09-21T03:44:13.292303+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:31:47.147845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":18,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:44:27.418815+0000","last_change":"2024-09-21T03:44:14.323090+0000","last_active":"2024-09-21T03:44:27.418815+0000","last_peered":"2024-09-21T03:44:27.418815+0000","last_clean":"2024-09-21T03:44:27.418815+0000","last_became_active":"2024-09-21T03:44:14.322669+0000","last_became_peered":"2024-09-21T03:44:14.322669+0000","last_unstale":"2024-09-21T03:44:27.418815+0000","last_undegraded":"2024-09-21T03:44:27.418815+0000","last_fullsized":"2024-09-21T03:44:27.418815+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_clean_scrub_stamp":"2024-09-21T03:44:13.292303+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:08:56.852422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":18,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:44:27.418948+0000","last_change":"2024-09-21T03:44:14.318146+0000","last_active":"2024-09-21T03:44:27.418948+0000","last_peered":"2024-09-21T03:44:27.418948+0000","last_clean":"2024-09-21T03:44:27.418948+0000","last_became_active":"2024-09-21T03:44:14.317761+0000","last_became_peered":"2024-09-21T03:44:14.317761+0000","last_unstale":"2024-09-21T03:44:27.418948+0000","last_undegraded":"2024-09-21T03:44:27.418948+0000","last_fullsized":"2024-09-21T03:44:27.418948+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_clean_scrub_stamp":"2024-09-21T03:44:13.292303+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:07:31.530728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":18,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:44:27.410569+0000","last_change":"2024-09-21T03:44:14.316832+0000","last_active":"2024-09-21T03:44:27.410569+0000","last_peered":"2024-09-21T03:44:27.410569+0000","last_clean":"2024-09-21T03:44:27.410569+0000","last_became_active":"2024-09-21T03:44:14.316471+0000","last_became_peered":"2024-09-21T03:44:14.316471+0000","last_unstale":"2024-09-21T03:44:27.410569+0000","last_undegraded":"2024-09-21T03:44:27.410569+0000","last_fullsized":"2024-09-21T03:44:27.410569+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_clean_scrub_stamp":"2024-09-21T03:44:13.292303+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:21:23.848778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":18,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:44:27.417002+0000","last_change":"2024-09-21T03:44:14.322277+0000","last_active":"2024-09-21T03:44:27.417002+0000","last_peered":"2024-09-21T03:44:27.417002+0000","last_clean":"2024-09-21T03:44:27.417002+0000","last_became_active":"2024-09-21T03:44:14.321952+0000","last_became_peered":"2024-09-21T03:44:14.321952+0000","last_unstale":"2024-09-21T03:44:27.417002+0000","last_undegraded":"2024-09-21T03:44:27.417002+0000","last_fullsized":"2024-09-21T03:44:27.417002+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_clean_scrub_stamp":"2024-09-21T03:44:13.292303+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:43:55.169380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":18,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:44:27.418939+0000","last_change":"2024-09-21T03:44:14.322626+0000","last_active":"2024-09-21T03:44:27.418939+0000","last_peered":"2024-09-21T03:44:27.418939+0000","last_clean":"2024-09-21T03:44:27.418939+0000","last_became_active":"2024-09-21T03:44:14.321559+0000","last_became_peered":"2024-09-21T03:44:14.321559+0000","last_unstale":"2024-09-21T03:44:27.418939+0000","last_undegraded":"2024-09-21T03:44:27.418939+0000","last_fullsized":"2024-09-21T03:44:27.418939+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_clean_scrub_stamp":"2024-09-21T03:44:13.292303+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:12:23.736500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":18,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:44:27.416654+0000","last_change":"2024-09-21T03:44:14.318057+0000","last_active":"2024-09-21T03:44:27.416654+0000","last_peered":"2024-09-21T03:44:27.416654+0000","last_clean":"2024-09-21T03:44:27.416654+0000","last_became_active":"2024-09-21T03:44:14.317653+0000","last_became_peered":"2024-09-21T03:44:14.317653+0000","last_unstale":"2024-09-21T03:44:27.416654+0000","last_undegraded":"2024-09-21T03:44:27.416654+0000","last_fullsized":"2024-09-21T03:44:27.416654+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_clean_scrub_stamp":"2024-09-21T03:44:13.292303+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:23:17.396599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":18,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:44:27.418421+0000","last_change":"2024-09-21T03:44:14.322331+0000","last_active":"2024-09-21T03:44:27.418421+0000","last_peered":"2024-09-21T03:44:27.418421+0000","last_clean":"2024-09-21T03:44:27.418421+0000","last_became_active":"2024-09-21T03:44:14.321157+0000","last_became_peered":"2024-09-21T03:44:14.321157+0000","last_unstale":"2024-09-21T03:44:27.418421+0000","last_undegraded":"2024-09-21T03:44:27.418421+0000","last_fullsized":"2024-09-21T03:44:27.418421+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_clean_scrub_stamp":"2024-09-21T03:44:13.292303+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:08:27.953449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":25,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:44:27.423182+0000","last_change":"2024-09-21T03:44:27.423081+0000","last_active":"2024-09-21T03:44:27.423182+0000","last_peered":"2024-09-21T03:44:27.423182+0000","last_clean":"2024-09-21T03:44:27.423182+0000","last_became_active":"2024-09-21T03:44:27.422650+0000","last_became_peered":"2024-09-21T03:44:27.422650+0000","last_unstale":"2024-09-21T03:44:27.423182+0000","last_undegraded":"2024-09-21T03:44:27.423182+0000","last_fullsized":"2024-09-21T03:44:27.423182+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_clean_scrub_stamp":"2024-09-21T03:44:13.292303+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:32:44.399954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,7],"acting":[8,9,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1b","version":"79'6","reported_seq":87,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:45:17.780563+0000","last_change":"2024-09-21T03:44:12.307464+0000","last_active":"2024-09-21T03:45:17.780563+0000","last_peered":"2024-09-21T03:45:17.780563+0000","last_clean":"2024-09-21T03:45:17.780563+0000","last_became_active":"2024-09-21T03:44:12.307077+0000","last_became_peered":"2024-09-21T03:44:12.307077+0000","last_unstale":"2024-09-21T03:45:17.780563+0000","last_undegraded":"2024-09-21T03:45:17.780563+0000","last_fullsized":"2024-09-21T03:45:17.780563+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:11.282438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:11.282438+0000","last_clean_scrub_stamp":"2024-09-21T03:44:11.282438+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:50:40.344659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1013,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":58,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":18,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:44:27.414062+0000","last_change":"2024-09-21T03:44:14.319072+0000","last_active":"2024-09-21T03:44:27.414062+0000","last_peered":"2024-09-21T03:44:27.414062+0000","last_clean":"2024-09-21T03:44:27.414062+0000","last_became_active":"2024-09-21T03:44:14.318712+0000","last_became_peered":"2024-09-21T03:44:14.318712+0000","last_unstale":"2024-09-21T03:44:27.414062+0000","last_undegraded":"2024-09-21T03:44:27.414062+0000","last_fullsized":"2024-09-21T03:44:27.414062+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_clean_scrub_stamp":"2024-09-21T03:44:13.292303+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:30:35.618110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1a","version":"77'7","reported_seq":30,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:44:27.611763+0000","last_change":"2024-09-21T03:44:12.310146+0000","last_active":"2024-09-21T03:44:27.611763+0000","last_peered":"2024-09-21T03:44:27.611763+0000","last_clean":"2024-09-21T03:44:27.611763+0000","last_became_active":"2024-09-21T03:44:12.309817+0000","last_became_peered":"2024-09-21T03:44:12.309817+0000","last_unstale":"2024-09-21T03:44:27.611763+0000","last_undegraded":"2024-09-21T03:44:27.611763+0000","last_fullsized":"2024-09-21T03:44:27.611763+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:11.282438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:11.282438+0000","last_clean_scrub_stamp":"2024-09-21T03:44:11.282438+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:46:51.994488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.19","version":"77'3","reported_seq":26,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:44:27.410211+0000","last_change":"2024-09-21T03:44:12.308995+0000","last_active":"2024-09-21T03:44:27.410211+0000","last_peered":"2024-09-21T03:44:27.410211+0000","last_clean":"2024-09-21T03:44:27.410211+0000","last_became_active":"2024-09-21T03:44:12.308058+0000","last_became_peered":"2024-09-21T03:44:12.308058+0000","last_unstale":"2024-09-21T03:44:27.410211+0000","last_undegraded":"2024-09-21T03:44:27.410211+0000","last_fullsized":"2024-09-21T03:44:27.410211+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:11.282438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:11.282438+0000","last_clean_scrub_stamp":"2024-09-21T03:44:11.282438+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:48:39.384985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":18,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:44:27.610583+0000","last_change":"2024-09-21T03:44:14.317878+0000","last_active":"2024-09-21T03:44:27.610583+0000","last_peered":"2024-09-21T03:44:27.610583+0000","last_clean":"2024-09-21T03:44:27.610583+0000","last_became_active":"2024-09-21T03:44:14.317521+0000","last_became_peered":"2024-09-21T03:44:14.317521+0000","last_unstale":"2024-09-21T03:44:27.610583+0000","last_undegraded":"2024-09-21T03:44:27.610583+0000","last_fullsized":"2024-09-21T03:44:27.610583+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_clean_scrub_stamp":"2024-09-21T03:44:13.292303+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:08:00.705210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"77'3","reported_seq":27,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:44:27.414041+0000","last_change":"2024-09-21T03:44:12.308777+0000","last_active":"2024-09-21T03:44:27.414041+0000","last_peered":"2024-09-21T03:44:27.414041+0000","last_clean":"2024-09-21T03:44:27.414041+0000","last_became_active":"2024-09-21T03:44:12.308171+0000","last_became_peered":"2024-09-21T03:44:12.308171+0000","last_unstale":"2024-09-21T03:44:27.414041+0000","last_undegraded":"2024-09-21T03:44:27.414041+0000","last_fullsized":"2024-09-21T03:44:27.414041+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:11.282438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:11.282438+0000","last_clean_scrub_stamp":"2024-09-21T03:44:11.282438+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:41:21.344491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":18,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:44:27.414180+0000","last_change":"2024-09-21T03:44:14.315782+0000","last_active":"2024-09-21T03:44:27.414180+0000","last_peered":"2024-09-21T03:44:27.414180+0000","last_clean":"2024-09-21T03:44:27.414180+0000","last_became_active":"2024-09-21T03:44:14.315314+0000","last_became_peered":"2024-09-21T03:44:14.315314+0000","last_unstale":"2024-09-21T03:44:27.414180+0000","last_undegraded":"2024-09-21T03:44:27.414180+0000","last_fullsized":"2024-09-21T03:44:27.414180+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_clean_scrub_stamp":"2024-09-21T03:44:13.292303+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:22:20.500220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"77'4","reported_seq":135,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:45:17.780016+0000","last_change":"2024-09-21T03:44:12.313251+0000","last_active":"2024-09-21T03:45:17.780016+0000","last_peered":"2024-09-21T03:45:17.780016+0000","last_clean":"2024-09-21T03:45:17.780016+0000","last_became_active":"2024-09-21T03:44:12.312941+0000","last_became_peered":"2024-09-21T03:44:12.312941+0000","last_unstale":"2024-09-21T03:45:17.780016+0000","last_undegraded":"2024-09-21T03:45:17.780016+0000","last_fullsized":"2024-09-21T03:45:17.780016+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:11.282438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:11.282438+0000","last_clean_scrub_stamp":"2024-09-21T03:44:11.282438+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:38:39.876097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":108,"num_read_kb":108,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":18,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:44:27.414153+0000","last_change":"2024-09-21T03:44:14.319621+0000","last_active":"2024-09-21T03:44:27.414153+0000","last_peered":"2024-09-21T03:44:27.414153+0000","last_clean":"2024-09-21T03:44:27.414153+0000","last_became_active":"2024-09-21T03:44:14.316173+0000","last_became_peered":"2024-09-21T03:44:14.316173+0000","last_unstale":"2024-09-21T03:44:27.414153+0000","last_undegraded":"2024-09-21T03:44:27.414153+0000","last_fullsized":"2024-09-21T03:44:27.414153+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_clean_scrub_stamp":"2024-09-21T03:44:13.292303+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:10:21.952989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"77'4","reported_seq":27,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:44:27.419175+0000","last_change":"2024-09-21T03:44:12.308893+0000","last_active":"2024-09-21T03:44:27.419175+0000","last_peered":"2024-09-21T03:44:27.419175+0000","last_clean":"2024-09-21T03:44:27.419175+0000","last_became_active":"2024-09-21T03:44:12.308551+0000","last_became_peered":"2024-09-21T03:44:12.308551+0000","last_unstale":"2024-09-21T03:44:27.419175+0000","last_undegraded":"2024-09-21T03:44:27.419175+0000","last_fullsized":"2024-09-21T03:44:27.419175+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:11.282438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:11.282438+0000","last_clean_scrub_stamp":"2024-09-21T03:44:11.282438+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:48:18.401595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":18,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:44:27.417391+0000","last_change":"2024-09-21T03:44:14.321875+0000","last_active":"2024-09-21T03:44:27.417391+0000","last_peered":"2024-09-21T03:44:27.417391+0000","last_clean":"2024-09-21T03:44:27.417391+0000","last_became_active":"2024-09-21T03:44:14.321483+0000","last_became_peered":"2024-09-21T03:44:14.321483+0000","last_unstale":"2024-09-21T03:44:27.417391+0000","last_undegraded":"2024-09-21T03:44:27.417391+0000","last_fullsized":"2024-09-21T03:44:27.417391+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_clean_scrub_stamp":"2024-09-21T03:44:13.292303+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:40:17.244780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"77'5","reported_seq":29,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:44:27.414461+0000","last_change":"2024-09-21T03:44:12.313208+0000","last_active":"2024-09-21T03:44:27.414461+0000","last_peered":"2024-09-21T03:44:27.414461+0000","last_clean":"2024-09-21T03:44:27.414461+0000","last_became_active":"2024-09-21T03:44:12.312783+0000","last_became_peered":"2024-09-21T03:44:12.312783+0000","last_unstale":"2024-09-21T03:44:27.414461+0000","last_undegraded":"2024-09-21T03:44:27.414461+0000","last_fullsized":"2024-09-21T03:44:27.414461+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:11.282438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:11.282438+0000","last_clean_scrub_stamp":"2024-09-21T03:44:11.282438+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:18:45.114361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":25,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:44:27.421219+0000","last_change":"2024-09-21T03:44:27.421129+0000","last_active":"2024-09-21T03:44:27.421219+0000","last_peered":"2024-09-21T03:44:27.421219+0000","last_clean":"2024-09-21T03:44:27.421219+0000","last_became_active":"2024-09-21T03:44:27.420642+0000","last_became_peered":"2024-09-21T03:44:27.420642+0000","last_unstale":"2024-09-21T03:44:27.421219+0000","last_undegraded":"2024-09-21T03:44:27.421219+0000","last_fullsized":"2024-09-21T03:44:27.421219+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_clean_scrub_stamp":"2024-09-21T03:44:13.292303+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:49:38.065743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,6],"acting":[1,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"79'30","reported_seq":170,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:45:17.794643+0000","last_change":"2024-09-21T03:44:12.314863+0000","last_active":"2024-09-21T03:45:17.794643+0000","last_peered":"2024-09-21T03:45:17.794643+0000","last_clean":"2024-09-21T03:45:17.794643+0000","last_became_active":"2024-09-21T03:44:12.314471+0000","last_became_peered":"2024-09-21T03:44:12.314471+0000","last_unstale":"2024-09-21T03:45:17.794643+0000","last_undegraded":"2024-09-21T03:45:17.794643+0000","last_fullsized":"2024-09-21T03:45:17.794643+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:11.282438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:11.282438+0000","last_clean_scrub_stamp":"2024-09-21T03:44:11.282438+0000","objects_scrubbed":0,"log_size":30,"log_dups_size":0,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:41:03.471226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":63902,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":117,"num_read_kb":120,"num_write":32,"num_write_kb":75,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":18,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:44:27.420428+0000","last_change":"2024-09-21T03:44:14.325471+0000","last_active":"2024-09-21T03:44:27.420428+0000","last_peered":"2024-09-21T03:44:27.420428+0000","last_clean":"2024-09-21T03:44:27.420428+0000","last_became_active":"2024-09-21T03:44:14.325072+0000","last_became_peered":"2024-09-21T03:44:14.325072+0000","last_unstale":"2024-09-21T03:44:27.420428+0000","last_undegraded":"2024-09-21T03:44:27.420428+0000","last_fullsized":"2024-09-21T03:44:27.420428+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_clean_scrub_stamp":"2024-09-21T03:44:13.292303+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:40:40.097240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"77'9","reported_seq":34,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:44:27.416778+0000","last_change":"2024-09-21T03:44:12.312932+0000","last_active":"2024-09-21T03:44:27.416778+0000","last_peered":"2024-09-21T03:44:27.416778+0000","last_clean":"2024-09-21T03:44:27.416778+0000","last_became_active":"2024-09-21T03:44:12.312368+0000","last_became_peered":"2024-09-21T03:44:12.312368+0000","last_unstale":"2024-09-21T03:44:27.416778+0000","last_undegraded":"2024-09-21T03:44:27.416778+0000","last_fullsized":"2024-09-21T03:44:27.416778+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:11.282438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:11.282438+0000","last_clean_scrub_stamp":"2024-09-21T03:44:11.282438+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:28:11.579598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":18,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:44:27.410310+0000","last_change":"2024-09-21T03:44:14.329564+0000","last_active":"2024-09-21T03:44:27.410310+0000","last_peered":"2024-09-21T03:44:27.410310+0000","last_clean":"2024-09-21T03:44:27.410310+0000","last_became_active":"2024-09-21T03:44:14.329122+0000","last_became_peered":"2024-09-21T03:44:14.329122+0000","last_unstale":"2024-09-21T03:44:27.410310+0000","last_undegraded":"2024-09-21T03:44:27.410310+0000","last_fullsized":"2024-09-21T03:44:27.410310+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_clean_scrub_stamp":"2024-09-21T03:44:13.292303+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:16:32.702770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"77'5","reported_seq":29,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:44:27.413875+0000","last_change":"2024-09-21T03:44:12.303339+0000","last_active":"2024-09-21T03:44:27.413875+0000","last_peered":"2024-09-21T03:44:27.413875+0000","last_clean":"2024-09-21T03:44:27.413875+0000","last_became_active":"2024-09-21T03:44:12.302905+0000","last_became_peered":"2024-09-21T03:44:12.302905+0000","last_unstale":"2024-09-21T03:44:27.413875+0000","last_undegraded":"2024-09-21T03:44:27.413875+0000","last_fullsized":"2024-09-21T03:44:27.413875+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:11.282438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:11.282438+0000","last_clean_scrub_stamp":"2024-09-21T03:44:11.282438+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:57:16.424976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":18,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:44:27.418798+0000","last_change":"2024-09-21T03:44:14.323473+0000","last_active":"2024-09-21T03:44:27.418798+0000","last_peered":"2024-09-21T03:44:27.418798+0000","last_clean":"2024-09-21T03:44:27.418798+0000","last_became_active":"2024-09-21T03:44:14.322560+0000","last_became_peered":"2024-09-21T03:44:14.322560+0000","last_unstale":"2024-09-21T03:44:27.418798+0000","last_undegraded":"2024-09-21T03:44:27.418798+0000","last_fullsized":"2024-09-21T03:44:27.418798+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_clean_scrub_stamp":"2024-09-21T03:44:13.292303+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:07:55.714351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"79'6","reported_seq":88,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:45:18.782939+0000","last_change":"2024-09-21T03:44:12.312331+0000","last_active":"2024-09-21T03:45:18.782939+0000","last_peered":"2024-09-21T03:45:18.782939+0000","last_clean":"2024-09-21T03:45:18.782939+0000","last_became_active":"2024-09-21T03:44:12.311984+0000","last_became_peered":"2024-09-21T03:44:12.311984+0000","last_unstale":"2024-09-21T03:45:18.782939+0000","last_undegraded":"2024-09-21T03:45:18.782939+0000","last_fullsized":"2024-09-21T03:45:18.782939+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:11.282438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:11.282438+0000","last_clean_scrub_stamp":"2024-09-21T03:44:11.282438+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:06:11.906592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":979,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":60,"num_read_kb":4,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":18,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:44:27.418181+0000","last_change":"2024-09-21T03:44:14.318437+0000","last_active":"2024-09-21T03:44:27.418181+0000","last_peered":"2024-09-21T03:44:27.418181+0000","last_clean":"2024-09-21T03:44:27.418181+0000","last_became_active":"2024-09-21T03:44:14.318052+0000","last_became_peered":"2024-09-21T03:44:14.318052+0000","last_unstale":"2024-09-21T03:44:27.418181+0000","last_undegraded":"2024-09-21T03:44:27.418181+0000","last_fullsized":"2024-09-21T03:44:27.418181+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_clean_scrub_stamp":"2024-09-21T03:44:13.292303+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:10:12.136159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"77'17","reported_seq":41,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:44:27.419831+0000","last_change":"2024-09-21T03:44:12.310616+0000","last_active":"2024-09-21T03:44:27.419831+0000","last_peered":"2024-09-21T03:44:27.419831+0000","last_clean":"2024-09-21T03:44:27.419831+0000","last_became_active":"2024-09-21T03:44:12.310239+0000","last_became_peered":"2024-09-21T03:44:12.310239+0000","last_unstale":"2024-09-21T03:44:27.419831+0000","last_undegraded":"2024-09-21T03:44:27.419831+0000","last_fullsized":"2024-09-21T03:44:27.419831+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:11.282438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:11.282438+0000","last_clean_scrub_stamp":"2024-09-21T03:44:11.282438+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:43:38.046281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":18,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:44:27.413822+0000","last_change":"2024-09-21T03:44:14.327058+0000","last_active":"2024-09-21T03:44:27.413822+0000","last_peered":"2024-09-21T03:44:27.413822+0000","last_clean":"2024-09-21T03:44:27.413822+0000","last_became_active":"2024-09-21T03:44:14.326641+0000","last_became_peered":"2024-09-21T03:44:14.326641+0000","last_unstale":"2024-09-21T03:44:27.413822+0000","last_undegraded":"2024-09-21T03:44:27.413822+0000","last_fullsized":"2024-09-21T03:44:27.413822+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_clean_scrub_stamp":"2024-09-21T03:44:13.292303+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:10:50.686102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"77'2","reported_seq":26,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:44:27.420648+0000","last_change":"2024-09-21T03:44:12.310426+0000","last_active":"2024-09-21T03:44:27.420648+0000","last_peered":"2024-09-21T03:44:27.420648+0000","last_clean":"2024-09-21T03:44:27.420648+0000","last_became_active":"2024-09-21T03:44:12.309231+0000","last_became_peered":"2024-09-21T03:44:12.309231+0000","last_unstale":"2024-09-21T03:44:27.420648+0000","last_undegraded":"2024-09-21T03:44:27.420648+0000","last_fullsized":"2024-09-21T03:44:27.420648+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:11.282438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:11.282438+0000","last_clean_scrub_stamp":"2024-09-21T03:44:11.282438+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:32:18.284741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":18,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:44:27.420390+0000","last_change":"2024-09-21T03:44:14.319179+0000","last_active":"2024-09-21T03:44:27.420390+0000","last_peered":"2024-09-21T03:44:27.420390+0000","last_clean":"2024-09-21T03:44:27.420390+0000","last_became_active":"2024-09-21T03:44:14.318826+0000","last_became_peered":"2024-09-21T03:44:14.318826+0000","last_unstale":"2024-09-21T03:44:27.420390+0000","last_undegraded":"2024-09-21T03:44:27.420390+0000","last_fullsized":"2024-09-21T03:44:27.420390+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_clean_scrub_stamp":"2024-09-21T03:44:13.292303+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:12:57.985087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.e","version":"77'5","reported_seq":28,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:44:27.410705+0000","last_change":"2024-09-21T03:44:12.315658+0000","last_active":"2024-09-21T03:44:27.410705+0000","last_peered":"2024-09-21T03:44:27.410705+0000","last_clean":"2024-09-21T03:44:27.410705+0000","last_became_active":"2024-09-21T03:44:12.315240+0000","last_became_peered":"2024-09-21T03:44:12.315240+0000","last_unstale":"2024-09-21T03:44:27.410705+0000","last_undegraded":"2024-09-21T03:44:27.410705+0000","last_fullsized":"2024-09-21T03:44:27.410705+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:11.282438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:11.282438+0000","last_clean_scrub_stamp":"2024-09-21T03:44:11.282438+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:22:07.493437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":18,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:44:27.419234+0000","last_change":"2024-09-21T03:44:14.328014+0000","last_active":"2024-09-21T03:44:27.419234+0000","last_peered":"2024-09-21T03:44:27.419234+0000","last_clean":"2024-09-21T03:44:27.419234+0000","last_became_active":"2024-09-21T03:44:14.327638+0000","last_became_peered":"2024-09-21T03:44:14.327638+0000","last_unstale":"2024-09-21T03:44:27.419234+0000","last_undegraded":"2024-09-21T03:44:27.419234+0000","last_fullsized":"2024-09-21T03:44:27.419234+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_clean_scrub_stamp":"2024-09-21T03:44:13.292303+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:48:32.542776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"77'3","reported_seq":27,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:44:27.414656+0000","last_change":"2024-09-21T03:44:12.309272+0000","last_active":"2024-09-21T03:44:27.414656+0000","last_peered":"2024-09-21T03:44:27.414656+0000","last_clean":"2024-09-21T03:44:27.414656+0000","last_became_active":"2024-09-21T03:44:12.308718+0000","last_became_peered":"2024-09-21T03:44:12.308718+0000","last_unstale":"2024-09-21T03:44:27.414656+0000","last_undegraded":"2024-09-21T03:44:27.414656+0000","last_fullsized":"2024-09-21T03:44:27.414656+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:11.282438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:11.282438+0000","last_clean_scrub_stamp":"2024-09-21T03:44:11.282438+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:03:49.433966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"79'1","reported_seq":19,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:44:27.418909+0000","last_change":"2024-09-21T03:44:14.322148+0000","last_active":"2024-09-21T03:44:27.418909+0000","last_peered":"2024-09-21T03:44:27.418909+0000","last_clean":"2024-09-21T03:44:27.418909+0000","last_became_active":"2024-09-21T03:44:14.321784+0000","last_became_peered":"2024-09-21T03:44:14.321784+0000","last_unstale":"2024-09-21T03:44:27.418909+0000","last_undegraded":"2024-09-21T03:44:27.418909+0000","last_fullsized":"2024-09-21T03:44:27.418909+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_clean_scrub_stamp":"2024-09-21T03:44:13.292303+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:40:34.887134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"77'5","reported_seq":72,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:45:17.493522+0000","last_change":"2024-09-21T03:44:27.427722+0000","last_active":"2024-09-21T03:45:17.493522+0000","last_peered":"2024-09-21T03:45:17.493522+0000","last_clean":"2024-09-21T03:45:17.493522+0000","last_became_active":"2024-09-21T03:44:27.419672+0000","last_became_peered":"2024-09-21T03:44:27.419672+0000","last_unstale":"2024-09-21T03:45:17.493522+0000","last_undegraded":"2024-09-21T03:45:17.493522+0000","last_fullsized":"2024-09-21T03:45:17.493522+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:11.282438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:11.282438+0000","last_clean_scrub_stamp":"2024-09-21T03:44:11.282438+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:24:31.890439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1170,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":52,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":1170,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,11,0],"acting":[1,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":18,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:44:27.419112+0000","last_change":"2024-09-21T03:44:14.316503+0000","last_active":"2024-09-21T03:44:27.419112+0000","last_peered":"2024-09-21T03:44:27.419112+0000","last_clean":"2024-09-21T03:44:27.419112+0000","last_became_active":"2024-09-21T03:44:14.316140+0000","last_became_peered":"2024-09-21T03:44:14.316140+0000","last_unstale":"2024-09-21T03:44:27.419112+0000","last_undegraded":"2024-09-21T03:44:27.419112+0000","last_fullsized":"2024-09-21T03:44:27.419112+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_clean_scrub_stamp":"2024-09-21T03:44:13.292303+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:00:30.647579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"21'92","reported_seq":96,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:44:27.418315+0000","last_change":"2024-09-21T03:43:01.568120+0000","last_active":"2024-09-21T03:44:27.418315+0000","last_peered":"2024-09-21T03:44:27.418315+0000","last_clean":"2024-09-21T03:44:27.418315+0000","last_became_active":"2024-09-21T03:43:01.476311+0000","last_became_peered":"2024-09-21T03:43:01.476311+0000","last_unstale":"2024-09-21T03:44:27.418315+0000","last_undegraded":"2024-09-21T03:44:27.418315+0000","last_fullsized":"2024-09-21T03:44:27.418315+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:39:30.121380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:39:30.121380+0000","last_clean_scrub_stamp":"2024-09-21T03:39:30.121380+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:40:27.967442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"77'6","reported_seq":32,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:44:27.613250+0000","last_change":"2024-09-21T03:44:12.311860+0000","last_active":"2024-09-21T03:44:27.613250+0000","last_peered":"2024-09-21T03:44:27.613250+0000","last_clean":"2024-09-21T03:44:27.613250+0000","last_became_active":"2024-09-21T03:44:12.311453+0000","last_became_peered":"2024-09-21T03:44:12.311453+0000","last_unstale":"2024-09-21T03:44:27.613250+0000","last_undegraded":"2024-09-21T03:44:27.613250+0000","last_fullsized":"2024-09-21T03:44:27.613250+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:11.282438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:11.282438+0000","last_clean_scrub_stamp":"2024-09-21T03:44:11.282438+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:23:40.550445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:44:27.418319+0000","last_change":"2024-09-21T03:44:27.416799+0000","last_active":"2024-09-21T03:44:27.418319+0000","last_peered":"2024-09-21T03:44:27.418319+0000","last_clean":"2024-09-21T03:44:27.418319+0000","last_became_active":"2024-09-21T03:44:27.416306+0000","last_became_peered":"2024-09-21T03:44:27.416306+0000","last_unstale":"2024-09-21T03:44:27.418319+0000","last_undegraded":"2024-09-21T03:44:27.418319+0000","last_fullsized":"2024-09-21T03:44:27.418319+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_clean_scrub_stamp":"2024-09-21T03:44:13.292303+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:34:26.547100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,5,1],"acting":[9,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.0","version":"77'4","reported_seq":27,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:44:27.418313+0000","last_change":"2024-09-21T03:44:12.307302+0000","last_active":"2024-09-21T03:44:27.418313+0000","last_peered":"2024-09-21T03:44:27.418313+0000","last_clean":"2024-09-21T03:44:27.418313+0000","last_became_active":"2024-09-21T03:44:12.306856+0000","last_became_peered":"2024-09-21T03:44:12.306856+0000","last_unstale":"2024-09-21T03:44:27.418313+0000","last_undegraded":"2024-09-21T03:44:27.418313+0000","last_fullsized":"2024-09-21T03:44:27.418313+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:11.282438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:11.282438+0000","last_clean_scrub_stamp":"2024-09-21T03:44:11.282438+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:21:20.420020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":18,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:44:27.410210+0000","last_change":"2024-09-21T03:44:14.322913+0000","last_active":"2024-09-21T03:44:27.410210+0000","last_peered":"2024-09-21T03:44:27.410210+0000","last_clean":"2024-09-21T03:44:27.410210+0000","last_became_active":"2024-09-21T03:44:14.322168+0000","last_became_peered":"2024-09-21T03:44:14.322168+0000","last_unstale":"2024-09-21T03:44:27.410210+0000","last_undegraded":"2024-09-21T03:44:27.410210+0000","last_fullsized":"2024-09-21T03:44:27.410210+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_clean_scrub_stamp":"2024-09-21T03:44:13.292303+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:30:09.041211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"77'5","reported_seq":82,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:45:15.777934+0000","last_change":"2024-09-21T03:44:12.309084+0000","last_active":"2024-09-21T03:45:15.777934+0000","last_peered":"2024-09-21T03:45:15.777934+0000","last_clean":"2024-09-21T03:45:15.777934+0000","last_became_active":"2024-09-21T03:44:12.308551+0000","last_became_peered":"2024-09-21T03:44:12.308551+0000","last_unstale":"2024-09-21T03:45:15.777934+0000","last_undegraded":"2024-09-21T03:45:15.777934+0000","last_fullsized":"2024-09-21T03:45:15.777934+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:11.282438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:11.282438+0000","last_clean_scrub_stamp":"2024-09-21T03:44:11.282438+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:46:33.895904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":54,"num_read_kb":54,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":18,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:44:27.413911+0000","last_change":"2024-09-21T03:44:14.322720+0000","last_active":"2024-09-21T03:44:27.413911+0000","last_peered":"2024-09-21T03:44:27.413911+0000","last_clean":"2024-09-21T03:44:27.413911+0000","last_became_active":"2024-09-21T03:44:14.322367+0000","last_became_peered":"2024-09-21T03:44:14.322367+0000","last_unstale":"2024-09-21T03:44:27.413911+0000","last_undegraded":"2024-09-21T03:44:27.413911+0000","last_fullsized":"2024-09-21T03:44:27.413911+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_clean_scrub_stamp":"2024-09-21T03:44:13.292303+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:59:51.692976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"79'6","reported_seq":83,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:45:16.687016+0000","last_change":"2024-09-21T03:44:12.308719+0000","last_active":"2024-09-21T03:45:16.687016+0000","last_peered":"2024-09-21T03:45:16.687016+0000","last_clean":"2024-09-21T03:45:16.687016+0000","last_became_active":"2024-09-21T03:44:12.308380+0000","last_became_peered":"2024-09-21T03:44:12.308380+0000","last_unstale":"2024-09-21T03:45:16.687016+0000","last_undegraded":"2024-09-21T03:45:16.687016+0000","last_fullsized":"2024-09-21T03:45:16.687016+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:11.282438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:11.282438+0000","last_clean_scrub_stamp":"2024-09-21T03:44:11.282438+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:41:02.087501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":979,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":54,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":18,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:44:27.612118+0000","last_change":"2024-09-21T03:44:14.326541+0000","last_active":"2024-09-21T03:44:27.612118+0000","last_peered":"2024-09-21T03:44:27.612118+0000","last_clean":"2024-09-21T03:44:27.612118+0000","last_became_active":"2024-09-21T03:44:14.326132+0000","last_became_peered":"2024-09-21T03:44:14.326132+0000","last_unstale":"2024-09-21T03:44:27.612118+0000","last_undegraded":"2024-09-21T03:44:27.612118+0000","last_fullsized":"2024-09-21T03:44:27.612118+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_clean_scrub_stamp":"2024-09-21T03:44:13.292303+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:09:27.711919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"77'1","reported_seq":24,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:44:27.414339+0000","last_change":"2024-09-21T03:44:12.309463+0000","last_active":"2024-09-21T03:44:27.414339+0000","last_peered":"2024-09-21T03:44:27.414339+0000","last_clean":"2024-09-21T03:44:27.414339+0000","last_became_active":"2024-09-21T03:44:12.309054+0000","last_became_peered":"2024-09-21T03:44:12.309054+0000","last_unstale":"2024-09-21T03:44:27.414339+0000","last_undegraded":"2024-09-21T03:44:27.414339+0000","last_fullsized":"2024-09-21T03:44:27.414339+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:11.282438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:11.282438+0000","last_clean_scrub_stamp":"2024-09-21T03:44:11.282438+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:12:12.505538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"79'1","reported_seq":19,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:44:27.612000+0000","last_change":"2024-09-21T03:44:14.318261+0000","last_active":"2024-09-21T03:44:27.612000+0000","last_peered":"2024-09-21T03:44:27.612000+0000","last_clean":"2024-09-21T03:44:27.612000+0000","last_became_active":"2024-09-21T03:44:14.317839+0000","last_became_peered":"2024-09-21T03:44:14.317839+0000","last_unstale":"2024-09-21T03:44:27.612000+0000","last_undegraded":"2024-09-21T03:44:27.612000+0000","last_fullsized":"2024-09-21T03:44:27.612000+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_clean_scrub_stamp":"2024-09-21T03:44:13.292303+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:19:05.504749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.5","version":"77'4","reported_seq":20,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:44:27.434085+0000","last_change":"2024-09-21T03:44:27.433630+0000","last_active":"2024-09-21T03:44:27.434085+0000","last_peered":"2024-09-21T03:44:27.434085+0000","last_clean":"2024-09-21T03:44:27.434085+0000","last_became_active":"2024-09-21T03:44:27.422093+0000","last_became_peered":"2024-09-21T03:44:27.422093+0000","last_unstale":"2024-09-21T03:44:27.434085+0000","last_undegraded":"2024-09-21T03:44:27.434085+0000","last_fullsized":"2024-09-21T03:44:27.434085+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:11.282438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:11.282438+0000","last_clean_scrub_stamp":"2024-09-21T03:44:11.282438+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:39:42.651024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,9],"acting":[11,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":18,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:44:27.410677+0000","last_change":"2024-09-21T03:44:14.322620+0000","last_active":"2024-09-21T03:44:27.410677+0000","last_peered":"2024-09-21T03:44:27.410677+0000","last_clean":"2024-09-21T03:44:27.410677+0000","last_became_active":"2024-09-21T03:44:14.322214+0000","last_became_peered":"2024-09-21T03:44:14.322214+0000","last_unstale":"2024-09-21T03:44:27.410677+0000","last_undegraded":"2024-09-21T03:44:27.410677+0000","last_fullsized":"2024-09-21T03:44:27.410677+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_clean_scrub_stamp":"2024-09-21T03:44:13.292303+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:53:20.519168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"79'15","reported_seq":38,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:44:40.341077+0000","last_change":"2024-09-21T03:44:12.311302+0000","last_active":"2024-09-21T03:44:40.341077+0000","last_peered":"2024-09-21T03:44:40.341077+0000","last_clean":"2024-09-21T03:44:40.341077+0000","last_became_active":"2024-09-21T03:44:12.310947+0000","last_became_peered":"2024-09-21T03:44:12.310947+0000","last_unstale":"2024-09-21T03:44:40.341077+0000","last_undegraded":"2024-09-21T03:44:40.341077+0000","last_fullsized":"2024-09-21T03:44:40.341077+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:11.282438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:11.282438+0000","last_clean_scrub_stamp":"2024-09-21T03:44:11.282438+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:09:34.308726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":26,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":18,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:44:27.611222+0000","last_change":"2024-09-21T03:44:14.324675+0000","last_active":"2024-09-21T03:44:27.611222+0000","last_peered":"2024-09-21T03:44:27.611222+0000","last_clean":"2024-09-21T03:44:27.611222+0000","last_became_active":"2024-09-21T03:44:14.324214+0000","last_became_peered":"2024-09-21T03:44:14.324214+0000","last_unstale":"2024-09-21T03:44:27.611222+0000","last_undegraded":"2024-09-21T03:44:27.611222+0000","last_fullsized":"2024-09-21T03:44:27.611222+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_clean_scrub_stamp":"2024-09-21T03:44:13.292303+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:33:51.071794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"77'7","reported_seq":31,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:44:27.613637+0000","last_change":"2024-09-21T03:44:12.322312+0000","last_active":"2024-09-21T03:44:27.613637+0000","last_peered":"2024-09-21T03:44:27.613637+0000","last_clean":"2024-09-21T03:44:27.613637+0000","last_became_active":"2024-09-21T03:44:12.322018+0000","last_became_peered":"2024-09-21T03:44:12.322018+0000","last_unstale":"2024-09-21T03:44:27.613637+0000","last_undegraded":"2024-09-21T03:44:27.613637+0000","last_fullsized":"2024-09-21T03:44:27.613637+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:11.282438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:11.282438+0000","last_clean_scrub_stamp":"2024-09-21T03:44:11.282438+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:11:41.817574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":18,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:44:27.417254+0000","last_change":"2024-09-21T03:44:14.326682+0000","last_active":"2024-09-21T03:44:27.417254+0000","last_peered":"2024-09-21T03:44:27.417254+0000","last_clean":"2024-09-21T03:44:27.417254+0000","last_became_active":"2024-09-21T03:44:14.326313+0000","last_became_peered":"2024-09-21T03:44:14.326313+0000","last_unstale":"2024-09-21T03:44:27.417254+0000","last_undegraded":"2024-09-21T03:44:27.417254+0000","last_fullsized":"2024-09-21T03:44:27.417254+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_clean_scrub_stamp":"2024-09-21T03:44:13.292303+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:26:44.681245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"77'2","reported_seq":36,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:44:27.436319+0000","last_change":"2024-09-21T03:44:27.436245+0000","last_active":"2024-09-21T03:44:27.436319+0000","last_peered":"2024-09-21T03:44:27.436319+0000","last_clean":"2024-09-21T03:44:27.436319+0000","last_became_active":"2024-09-21T03:44:27.422134+0000","last_became_peered":"2024-09-21T03:44:27.422134+0000","last_unstale":"2024-09-21T03:44:27.436319+0000","last_undegraded":"2024-09-21T03:44:27.436319+0000","last_fullsized":"2024-09-21T03:44:27.436319+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:11.282438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:11.282438+0000","last_clean_scrub_stamp":"2024-09-21T03:44:11.282438+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:50:22.655050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,11,0],"acting":[1,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":18,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:44:27.419704+0000","last_change":"2024-09-21T03:44:14.316312+0000","last_active":"2024-09-21T03:44:27.419704+0000","last_peered":"2024-09-21T03:44:27.419704+0000","last_clean":"2024-09-21T03:44:27.419704+0000","last_became_active":"2024-09-21T03:44:14.315929+0000","last_became_peered":"2024-09-21T03:44:14.315929+0000","last_unstale":"2024-09-21T03:44:27.419704+0000","last_undegraded":"2024-09-21T03:44:27.419704+0000","last_fullsized":"2024-09-21T03:44:27.419704+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_clean_scrub_stamp":"2024-09-21T03:44:13.292303+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:31:33.071983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"77'7","reported_seq":30,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:44:27.610761+0000","last_change":"2024-09-21T03:44:12.312655+0000","last_active":"2024-09-21T03:44:27.610761+0000","last_peered":"2024-09-21T03:44:27.610761+0000","last_clean":"2024-09-21T03:44:27.610761+0000","last_became_active":"2024-09-21T03:44:12.312196+0000","last_became_peered":"2024-09-21T03:44:12.312196+0000","last_unstale":"2024-09-21T03:44:27.610761+0000","last_undegraded":"2024-09-21T03:44:27.610761+0000","last_fullsized":"2024-09-21T03:44:27.610761+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:11.282438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:11.282438+0000","last_clean_scrub_stamp":"2024-09-21T03:44:11.282438+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:24:32.342461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":18,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:44:27.418552+0000","last_change":"2024-09-21T03:44:14.321875+0000","last_active":"2024-09-21T03:44:27.418552+0000","last_peered":"2024-09-21T03:44:27.418552+0000","last_clean":"2024-09-21T03:44:27.418552+0000","last_became_active":"2024-09-21T03:44:14.320682+0000","last_became_peered":"2024-09-21T03:44:14.320682+0000","last_unstale":"2024-09-21T03:44:27.418552+0000","last_undegraded":"2024-09-21T03:44:27.418552+0000","last_fullsized":"2024-09-21T03:44:27.418552+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_clean_scrub_stamp":"2024-09-21T03:44:13.292303+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:41:32.435288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"77'3","reported_seq":26,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:44:27.418692+0000","last_change":"2024-09-21T03:44:12.309100+0000","last_active":"2024-09-21T03:44:27.418692+0000","last_peered":"2024-09-21T03:44:27.418692+0000","last_clean":"2024-09-21T03:44:27.418692+0000","last_became_active":"2024-09-21T03:44:12.308671+0000","last_became_peered":"2024-09-21T03:44:12.308671+0000","last_unstale":"2024-09-21T03:44:27.418692+0000","last_undegraded":"2024-09-21T03:44:27.418692+0000","last_fullsized":"2024-09-21T03:44:27.418692+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:11.282438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:11.282438+0000","last_clean_scrub_stamp":"2024-09-21T03:44:11.282438+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:09:19.347980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":18,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:44:27.610670+0000","last_change":"2024-09-21T03:44:14.320170+0000","last_active":"2024-09-21T03:44:27.610670+0000","last_peered":"2024-09-21T03:44:27.610670+0000","last_clean":"2024-09-21T03:44:27.610670+0000","last_became_active":"2024-09-21T03:44:14.319794+0000","last_became_peered":"2024-09-21T03:44:14.319794+0000","last_unstale":"2024-09-21T03:44:27.610670+0000","last_undegraded":"2024-09-21T03:44:27.610670+0000","last_fullsized":"2024-09-21T03:44:27.610670+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_clean_scrub_stamp":"2024-09-21T03:44:13.292303+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:42:09.684471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"77'4","reported_seq":28,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:44:27.420077+0000","last_change":"2024-09-21T03:44:12.310851+0000","last_active":"2024-09-21T03:44:27.420077+0000","last_peered":"2024-09-21T03:44:27.420077+0000","last_clean":"2024-09-21T03:44:27.420077+0000","last_became_active":"2024-09-21T03:44:12.310302+0000","last_became_peered":"2024-09-21T03:44:12.310302+0000","last_unstale":"2024-09-21T03:44:27.420077+0000","last_undegraded":"2024-09-21T03:44:27.420077+0000","last_fullsized":"2024-09-21T03:44:27.420077+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:11.282438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:11.282438+0000","last_clean_scrub_stamp":"2024-09-21T03:44:11.282438+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:23:46.959121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":18,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:44:27.418894+0000","last_change":"2024-09-21T03:44:14.324544+0000","last_active":"2024-09-21T03:44:27.418894+0000","last_peered":"2024-09-21T03:44:27.418894+0000","last_clean":"2024-09-21T03:44:27.418894+0000","last_became_active":"2024-09-21T03:44:14.322881+0000","last_became_peered":"2024-09-21T03:44:14.322881+0000","last_unstale":"2024-09-21T03:44:27.418894+0000","last_undegraded":"2024-09-21T03:44:27.418894+0000","last_fullsized":"2024-09-21T03:44:27.418894+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_clean_scrub_stamp":"2024-09-21T03:44:13.292303+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:58:42.760185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":25,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:44:27.422566+0000","last_change":"2024-09-21T03:44:27.422438+0000","last_active":"2024-09-21T03:44:27.422566+0000","last_peered":"2024-09-21T03:44:27.422566+0000","last_clean":"2024-09-21T03:44:27.422566+0000","last_became_active":"2024-09-21T03:44:27.421935+0000","last_became_peered":"2024-09-21T03:44:27.421935+0000","last_unstale":"2024-09-21T03:44:27.422566+0000","last_undegraded":"2024-09-21T03:44:27.422566+0000","last_fullsized":"2024-09-21T03:44:27.422566+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_clean_scrub_stamp":"2024-09-21T03:44:13.292303+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:20:50.097783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,1],"acting":[5,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"77'5","reported_seq":29,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:44:27.420510+0000","last_change":"2024-09-21T03:44:12.311509+0000","last_active":"2024-09-21T03:44:27.420510+0000","last_peered":"2024-09-21T03:44:27.420510+0000","last_clean":"2024-09-21T03:44:27.420510+0000","last_became_active":"2024-09-21T03:44:12.310880+0000","last_became_peered":"2024-09-21T03:44:12.310880+0000","last_unstale":"2024-09-21T03:44:27.420510+0000","last_undegraded":"2024-09-21T03:44:27.420510+0000","last_fullsized":"2024-09-21T03:44:27.420510+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:11.282438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:11.282438+0000","last_clean_scrub_stamp":"2024-09-21T03:44:11.282438+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:59:33.887234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":18,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:44:27.413781+0000","last_change":"2024-09-21T03:44:14.330938+0000","last_active":"2024-09-21T03:44:27.413781+0000","last_peered":"2024-09-21T03:44:27.413781+0000","last_clean":"2024-09-21T03:44:27.413781+0000","last_became_active":"2024-09-21T03:44:14.330559+0000","last_became_peered":"2024-09-21T03:44:14.330559+0000","last_unstale":"2024-09-21T03:44:27.413781+0000","last_undegraded":"2024-09-21T03:44:27.413781+0000","last_fullsized":"2024-09-21T03:44:27.413781+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_clean_scrub_stamp":"2024-09-21T03:44:13.292303+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:05:32.535660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"77'12","reported_seq":36,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:44:27.420563+0000","last_change":"2024-09-21T03:44:12.307667+0000","last_active":"2024-09-21T03:44:27.420563+0000","last_peered":"2024-09-21T03:44:27.420563+0000","last_clean":"2024-09-21T03:44:27.420563+0000","last_became_active":"2024-09-21T03:44:12.307074+0000","last_became_peered":"2024-09-21T03:44:12.307074+0000","last_unstale":"2024-09-21T03:44:27.420563+0000","last_undegraded":"2024-09-21T03:44:27.420563+0000","last_fullsized":"2024-09-21T03:44:27.420563+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:11.282438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:11.282438+0000","last_clean_scrub_stamp":"2024-09-21T03:44:11.282438+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:48:41.598375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":18,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:44:27.417322+0000","last_change":"2024-09-21T03:44:14.325487+0000","last_active":"2024-09-21T03:44:27.417322+0000","last_peered":"2024-09-21T03:44:27.417322+0000","last_clean":"2024-09-21T03:44:27.417322+0000","last_became_active":"2024-09-21T03:44:14.325153+0000","last_became_peered":"2024-09-21T03:44:14.325153+0000","last_unstale":"2024-09-21T03:44:27.417322+0000","last_undegraded":"2024-09-21T03:44:27.417322+0000","last_fullsized":"2024-09-21T03:44:27.417322+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_clean_scrub_stamp":"2024-09-21T03:44:13.292303+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:29:17.619130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"77'2","reported_seq":25,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:44:27.611241+0000","last_change":"2024-09-21T03:44:12.307635+0000","last_active":"2024-09-21T03:44:27.611241+0000","last_peered":"2024-09-21T03:44:27.611241+0000","last_clean":"2024-09-21T03:44:27.611241+0000","last_became_active":"2024-09-21T03:44:12.307235+0000","last_became_peered":"2024-09-21T03:44:12.307235+0000","last_unstale":"2024-09-21T03:44:27.611241+0000","last_undegraded":"2024-09-21T03:44:27.611241+0000","last_fullsized":"2024-09-21T03:44:27.611241+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:11.282438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:11.282438+0000","last_clean_scrub_stamp":"2024-09-21T03:44:11.282438+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:18:59.985369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":18,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:44:27.611117+0000","last_change":"2024-09-21T03:44:14.324660+0000","last_active":"2024-09-21T03:44:27.611117+0000","last_peered":"2024-09-21T03:44:27.611117+0000","last_clean":"2024-09-21T03:44:27.611117+0000","last_became_active":"2024-09-21T03:44:14.324309+0000","last_became_peered":"2024-09-21T03:44:14.324309+0000","last_unstale":"2024-09-21T03:44:27.611117+0000","last_undegraded":"2024-09-21T03:44:27.611117+0000","last_fullsized":"2024-09-21T03:44:27.611117+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_clean_scrub_stamp":"2024-09-21T03:44:13.292303+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:13:07.806721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"77'13","reported_seq":112,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:45:18.689491+0000","last_change":"2024-09-21T03:44:27.623910+0000","last_active":"2024-09-21T03:45:18.689491+0000","last_peered":"2024-09-21T03:45:18.689491+0000","last_clean":"2024-09-21T03:45:18.689491+0000","last_became_active":"2024-09-21T03:44:27.614037+0000","last_became_peered":"2024-09-21T03:44:27.614037+0000","last_unstale":"2024-09-21T03:45:18.689491+0000","last_undegraded":"2024-09-21T03:45:18.689491+0000","last_fullsized":"2024-09-21T03:45:18.689491+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:11.282438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:11.282438+0000","last_clean_scrub_stamp":"2024-09-21T03:44:11.282438+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:03:56.753935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":54,"num_read_kb":54,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":672,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,4],"acting":[9,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":71497,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":576,"num_read_kb":364,"num_write":244,"num_write_kb":141,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13,"num_bytes_recovered":1954,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":577536,"data_stored":214491,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":210,"ondisk_log_size":210,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":71,"seq":304942678043,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27892,"kb_used_data":864,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717260,"statfs":{"total":95995035648,"available":95966474240,"internally_reserved":0,"allocated":884736,"data_stored":428272,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":65,"seq":279172874271,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28568,"kb_used_data":1544,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716584,"statfs":{"total":95995035648,"available":95965782016,"internally_reserved":0,"allocated":1581056,"data_stored":1083482,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":60,"seq":257698037796,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27904,"kb_used_data":880,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717248,"statfs":{"total":95995035648,"available":95966461952,"internally_reserved":0,"allocated":901120,"data_stored":428090,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":54,"seq":231928234024,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27884,"kb_used_data":860,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717268,"statfs":{"total":95995035648,"available":95966482432,"internally_reserved":0,"allocated":880640,"data_stored":427684,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":49,"seq":210453397552,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36804,"kb_used_data":1460,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708348,"statfs":{"total":95995035648,"available":95957348352,"internally_reserved":0,"allocated":1495040,"data_stored":1021768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":36174282},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":43,"seq":184683593780,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27888,"kb_used_data":864,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717264,"statfs":{"total":95995035648,"available":95966478336,"internally_reserved":0,"allocated":884736,"data_stored":428198,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":37,"seq":158913790010,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32656,"kb_used_data":1472,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712496,"statfs":{"total":95995035648,"available":95961595904,"internally_reserved":0,"allocated":1507328,"data_stored":1020664,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":31,"seq":133143986239,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36224,"kb_used_data":872,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708928,"statfs":{"total":95995035648,"available":95957942272,"internally_reserved":0,"allocated":892928,"data_stored":428731,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149764,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32064,"kb_used_data":880,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713088,"statfs":{"total":95995035648,"available":95962202112,"internally_reserved":0,"allocated":901120,"data_stored":427632,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":19,"seq":81604378698,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27964,"kb_used_data":940,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717188,"statfs":{"total":95995035648,"available":95966400512,"internally_reserved":0,"allocated":962560,"data_stored":492063,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542222,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27960,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717192,"statfs":{"total":95995035648,"available":95966404608,"internally_reserved":0,"allocated":958464,"data_stored":494183,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705748,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27912,"kb_used_data":884,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717240,"statfs":{"total":95995035648,"available":95966453760,"internally_reserved":0,"allocated":905216,"data_stored":429146,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2178,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":67215,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":65095,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1763,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3211,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1113,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":4315,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":716,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":118784,"data_stored":66029,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1304,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T03:45:23.596 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-09-21T03:45:23.596 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-09-21T03:45:23.596 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy 2024-09-21T03:45:23.596 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 51116d5c-77ca-11ef-baf5-efdc52797490 -- ceph health --format=json 2024-09-21T03:45:23.867 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/51116d5c-77ca-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:45:24.107 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:45:23 smithi039 ceph-mon[31211]: from='client.15192 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T03:45:24.218 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:45:23 smithi161 ceph-mon[34533]: from='client.15192 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T03:45:24.224 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:45:23 smithi150 ceph-mon[34289]: from='client.15192 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T03:45:24.423 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:24.406+0000 7fc9ecf70640 1 -- 172.21.15.39:0/2317670175 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc9e8108cf0 msgr2=0x7fc9e810f520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:24.423 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:24.406+0000 7fc9ecf70640 1 --2- 172.21.15.39:0/2317670175 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc9e8108cf0 0x7fc9e810f520 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7fc9e0009900 tx=0x7fc9e002a410 comp rx=0 tx=0).stop 2024-09-21T03:45:24.423 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:24.406+0000 7fc9ecf70640 1 -- 172.21.15.39:0/2317670175 shutdown_connections 2024-09-21T03:45:24.423 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:24.406+0000 7fc9ecf70640 1 --2- 172.21.15.39:0/2317670175 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc9e8108cf0 0x7fc9e810f520 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:24.423 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:24.406+0000 7fc9ecf70640 1 --2- 172.21.15.39:0/2317670175 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc9e81083a0 0x7fc9e81087b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:24.423 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:24.406+0000 7fc9ecf70640 1 --2- 172.21.15.39:0/2317670175 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc9e8102a60 0x7fc9e8102e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:24.423 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:24.406+0000 7fc9ecf70640 1 -- 172.21.15.39:0/2317670175 >> 172.21.15.39:0/2317670175 conn(0x7fc9e8079730 msgr2=0x7fc9e8079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:45:24.423 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:24.406+0000 7fc9ecf70640 1 -- 172.21.15.39:0/2317670175 shutdown_connections 2024-09-21T03:45:24.423 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:24.406+0000 7fc9ecf70640 1 -- 172.21.15.39:0/2317670175 wait complete. 2024-09-21T03:45:24.424 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:24.407+0000 7fc9ecf70640 1 Processor -- start 2024-09-21T03:45:24.424 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:24.407+0000 7fc9ecf70640 1 -- start start 2024-09-21T03:45:24.424 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:24.408+0000 7fc9ecf70640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc9e8102a60 0x7fc9e8199390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:24.424 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:24.408+0000 7fc9ecf70640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc9e81083a0 0x7fc9e8199930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:24.424 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:24.408+0000 7fc9ecf70640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc9e8108cf0 0x7fc9e819ebd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:24.424 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:24.408+0000 7fc9ecf70640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fc9e8112960 con 0x7fc9e81083a0 2024-09-21T03:45:24.424 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:24.408+0000 7fc9ecf70640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fc9e81127e0 con 0x7fc9e8108cf0 2024-09-21T03:45:24.424 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:24.408+0000 7fc9ecf70640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fc9e8112ae0 con 0x7fc9e8102a60 2024-09-21T03:45:24.424 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:24.408+0000 7fc9e5d9b640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc9e81083a0 0x7fc9e8199930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:24.424 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:24.408+0000 7fc9e5d9b640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc9e81083a0 0x7fc9e8199930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:60284/0 (socket says 172.21.15.39:60284) 2024-09-21T03:45:24.424 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:24.408+0000 7fc9e5d9b640 1 -- 172.21.15.39:0/119997266 learned_addr learned my addr 172.21.15.39:0/119997266 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:45:24.424 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:24.408+0000 7fc9e6d9d640 1 --2- 172.21.15.39:0/119997266 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc9e8108cf0 0x7fc9e819ebd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:24.424 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:24.408+0000 7fc9e659c640 1 --2- 172.21.15.39:0/119997266 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc9e8102a60 0x7fc9e8199390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:24.425 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:24.408+0000 7fc9e5d9b640 1 -- 172.21.15.39:0/119997266 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc9e8102a60 msgr2=0x7fc9e8199390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:24.425 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:24.408+0000 7fc9e5d9b640 1 --2- 172.21.15.39:0/119997266 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc9e8102a60 0x7fc9e8199390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:24.425 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:24.408+0000 7fc9e5d9b640 1 -- 172.21.15.39:0/119997266 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc9e8108cf0 msgr2=0x7fc9e819ebd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:24.425 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:24.408+0000 7fc9e5d9b640 1 --2- 172.21.15.39:0/119997266 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc9e8108cf0 0x7fc9e819ebd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:24.425 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:24.408+0000 7fc9e5d9b640 1 -- 172.21.15.39:0/119997266 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc9e819f300 con 0x7fc9e81083a0 2024-09-21T03:45:24.425 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:24.408+0000 7fc9e5d9b640 1 --2- 172.21.15.39:0/119997266 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc9e81083a0 0x7fc9e8199930 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7fc9dc00ea80 tx=0x7fc9dc00ef40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:45:24.425 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:24.408+0000 7fc9d77fe640 1 -- 172.21.15.39:0/119997266 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9dc00ce40 con 0x7fc9e81083a0 2024-09-21T03:45:24.425 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:24.409+0000 7fc9ecf70640 1 -- 172.21.15.39:0/119997266 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc9e819f5f0 con 0x7fc9e81083a0 2024-09-21T03:45:24.425 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:24.409+0000 7fc9ecf70640 1 -- 172.21.15.39:0/119997266 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc9e81a5d50 con 0x7fc9e81083a0 2024-09-21T03:45:24.425 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:24.409+0000 7fc9d77fe640 1 -- 172.21.15.39:0/119997266 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc9dc011920 con 0x7fc9e81083a0 2024-09-21T03:45:24.425 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:24.409+0000 7fc9d77fe640 1 -- 172.21.15.39:0/119997266 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9dc01a930 con 0x7fc9e81083a0 2024-09-21T03:45:24.425 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:24.410+0000 7fc9ecf70640 1 -- 172.21.15.39:0/119997266 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc9c80052c0 con 0x7fc9e81083a0 2024-09-21T03:45:24.425 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:24.411+0000 7fc9d77fe640 1 -- 172.21.15.39:0/119997266 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 22) ==== 154580+0+0 (secure 0 0 0) 0x7fc9dc011470 con 0x7fc9e81083a0 2024-09-21T03:45:24.425 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:24.412+0000 7fc9d77fe640 1 --2- 172.21.15.39:0/119997266 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7fc9d00b65f0 0x7fc9d00b8b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:24.426 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:24.412+0000 7fc9e659c640 1 --2- 172.21.15.39:0/119997266 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7fc9d00b65f0 0x7fc9d00b8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:24.426 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:24.412+0000 7fc9d77fe640 1 -- 172.21.15.39:0/119997266 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8484+0+0 (secure 0 0 0) 0x7fc9dc0e7920 con 0x7fc9e81083a0 2024-09-21T03:45:24.426 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:24.413+0000 7fc9e659c640 1 --2- 172.21.15.39:0/119997266 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7fc9d00b65f0 0x7fc9d00b8b00 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fc9d8005d00 tx=0x7fc9d800b410 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:45:24.426 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:24.415+0000 7fc9d77fe640 1 -- 172.21.15.39:0/119997266 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fc9dc016030 con 0x7fc9e81083a0 2024-09-21T03:45:24.644 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:24.641+0000 7fc9ecf70640 1 -- 172.21.15.39:0/119997266 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7fc9c80055b0 con 0x7fc9e81083a0 2024-09-21T03:45:24.644 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:24.641+0000 7fc9d77fe640 1 -- 172.21.15.39:0/119997266 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+46 (secure 0 0 0) 0x7fc9dc0adf50 con 0x7fc9e81083a0 2024-09-21T03:45:24.644 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T03:45:24.644 INFO:teuthology.orchestra.run.smithi039.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-09-21T03:45:24.647 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:24.644+0000 7fc9ecf70640 1 -- 172.21.15.39:0/119997266 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7fc9d00b65f0 msgr2=0x7fc9d00b8b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:24.647 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:24.644+0000 7fc9ecf70640 1 --2- 172.21.15.39:0/119997266 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7fc9d00b65f0 0x7fc9d00b8b00 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fc9d8005d00 tx=0x7fc9d800b410 comp rx=0 tx=0).stop 2024-09-21T03:45:24.647 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:24.644+0000 7fc9ecf70640 1 -- 172.21.15.39:0/119997266 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc9e81083a0 msgr2=0x7fc9e8199930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:24.647 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:24.644+0000 7fc9ecf70640 1 --2- 172.21.15.39:0/119997266 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc9e81083a0 0x7fc9e8199930 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7fc9dc00ea80 tx=0x7fc9dc00ef40 comp rx=0 tx=0).stop 2024-09-21T03:45:24.647 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:24.644+0000 7fc9ecf70640 1 -- 172.21.15.39:0/119997266 shutdown_connections 2024-09-21T03:45:24.647 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:24.644+0000 7fc9ecf70640 1 --2- 172.21.15.39:0/119997266 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7fc9d00b65f0 0x7fc9d00b8b00 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:24.647 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:24.644+0000 7fc9ecf70640 1 --2- 172.21.15.39:0/119997266 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc9e8108cf0 0x7fc9e819ebd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:24.647 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:24.644+0000 7fc9ecf70640 1 --2- 172.21.15.39:0/119997266 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc9e81083a0 0x7fc9e8199930 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:24.647 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:24.644+0000 7fc9ecf70640 1 --2- 172.21.15.39:0/119997266 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc9e8102a60 0x7fc9e8199390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:24.648 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:24.644+0000 7fc9ecf70640 1 -- 172.21.15.39:0/119997266 >> 172.21.15.39:0/119997266 conn(0x7fc9e8079730 msgr2=0x7fc9e8104470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:45:24.648 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:24.644+0000 7fc9ecf70640 1 -- 172.21.15.39:0/119997266 shutdown_connections 2024-09-21T03:45:24.648 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:24.644+0000 7fc9ecf70640 1 -- 172.21.15.39:0/119997266 wait complete. 2024-09-21T03:45:25.218 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:45:24 smithi161 ceph-mon[34533]: pgmap v306: 97 pgs: 97 active+clean; 647 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-09-21T03:45:25.218 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:45:24 smithi161 ceph-mon[34533]: from='client.? 172.21.15.39:0/119997266' entity='client.admin' cmd={"prefix": "health", "format": "json"} : dispatch 2024-09-21T03:45:25.224 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:45:24 smithi150 ceph-mon[34289]: pgmap v306: 97 pgs: 97 active+clean; 647 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-09-21T03:45:25.225 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:45:24 smithi150 ceph-mon[34289]: from='client.? 172.21.15.39:0/119997266' entity='client.admin' cmd={"prefix": "health", "format": "json"} : dispatch 2024-09-21T03:45:25.292 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:45:24 smithi039 ceph-mon[31211]: pgmap v306: 97 pgs: 97 active+clean; 647 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-09-21T03:45:25.292 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:45:24 smithi039 ceph-mon[31211]: from='client.? 172.21.15.39:0/119997266' entity='client.admin' cmd={"prefix": "health", "format": "json"} : dispatch 2024-09-21T03:45:25.322 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy done 2024-09-21T03:45:25.322 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 51116d5c-77ca-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T03:45:25.580 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/51116d5c-77ca-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:45:26.048 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:26.043+0000 7f7ba58c5640 1 -- 172.21.15.39:0/680495487 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7ba0108cf0 msgr2=0x7f7ba010f520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:26.048 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:26.043+0000 7f7ba58c5640 1 --2- 172.21.15.39:0/680495487 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7ba0108cf0 0x7f7ba010f520 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f7b98009ca0 tx=0x7f7b9802a8c0 comp rx=0 tx=0).stop 2024-09-21T03:45:26.048 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:26.043+0000 7f7ba58c5640 1 -- 172.21.15.39:0/680495487 shutdown_connections 2024-09-21T03:45:26.048 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:26.043+0000 7f7ba58c5640 1 --2- 172.21.15.39:0/680495487 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7ba0108cf0 0x7f7ba010f520 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:26.048 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:26.043+0000 7f7ba58c5640 1 --2- 172.21.15.39:0/680495487 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7ba01083a0 0x7f7ba01087b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:26.048 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:26.043+0000 7f7ba58c5640 1 --2- 172.21.15.39:0/680495487 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7ba0102a60 0x7f7ba0102e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:26.048 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:26.043+0000 7f7ba58c5640 1 -- 172.21.15.39:0/680495487 >> 172.21.15.39:0/680495487 conn(0x7f7ba0079730 msgr2=0x7f7ba0079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:45:26.048 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:26.043+0000 7f7ba58c5640 1 -- 172.21.15.39:0/680495487 shutdown_connections 2024-09-21T03:45:26.048 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:26.043+0000 7f7ba58c5640 1 -- 172.21.15.39:0/680495487 wait complete. 2024-09-21T03:45:26.048 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:26.044+0000 7f7ba58c5640 1 Processor -- start 2024-09-21T03:45:26.049 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:26.044+0000 7f7ba58c5640 1 -- start start 2024-09-21T03:45:26.049 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:26.044+0000 7f7ba58c5640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7ba0102a60 0x7f7ba019a460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:26.049 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:26.045+0000 7f7ba58c5640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7ba01083a0 0x7f7ba019aa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:26.049 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:26.045+0000 7f7ba58c5640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7ba0108cf0 0x7f7ba019fca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:26.049 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:26.045+0000 7f7ba58c5640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f7ba0112960 con 0x7f7ba01083a0 2024-09-21T03:45:26.049 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:26.045+0000 7f7ba58c5640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f7ba01127e0 con 0x7f7ba0108cf0 2024-09-21T03:45:26.049 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:26.045+0000 7f7ba58c5640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f7ba0112ae0 con 0x7f7ba0102a60 2024-09-21T03:45:26.049 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:26.045+0000 7f7b9e7fc640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7ba01083a0 0x7f7ba019aa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:26.049 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:26.045+0000 7f7b9e7fc640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7ba01083a0 0x7f7ba019aa00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:60310/0 (socket says 172.21.15.39:60310) 2024-09-21T03:45:26.049 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:26.045+0000 7f7b9e7fc640 1 -- 172.21.15.39:0/2567725584 learned_addr learned my addr 172.21.15.39:0/2567725584 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:45:26.049 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:26.045+0000 7f7b9effd640 1 --2- 172.21.15.39:0/2567725584 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7ba0102a60 0x7f7ba019a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:26.049 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:26.045+0000 7f7b9f7fe640 1 --2- 172.21.15.39:0/2567725584 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7ba0108cf0 0x7f7ba019fca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:26.049 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:26.045+0000 7f7b9e7fc640 1 -- 172.21.15.39:0/2567725584 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7ba0102a60 msgr2=0x7f7ba019a460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:26.050 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:26.045+0000 7f7b9e7fc640 1 --2- 172.21.15.39:0/2567725584 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7ba0102a60 0x7f7ba019a460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:26.050 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:26.045+0000 7f7b9e7fc640 1 -- 172.21.15.39:0/2567725584 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7ba0108cf0 msgr2=0x7f7ba019fca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:26.050 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:26.045+0000 7f7b9e7fc640 1 --2- 172.21.15.39:0/2567725584 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7ba0108cf0 0x7f7ba019fca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:26.050 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:26.045+0000 7f7b9e7fc640 1 -- 172.21.15.39:0/2567725584 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ba01a03d0 con 0x7f7ba01083a0 2024-09-21T03:45:26.050 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:26.045+0000 7f7b9e7fc640 1 --2- 172.21.15.39:0/2567725584 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7ba01083a0 0x7f7ba019aa00 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f7b9400c970 tx=0x7f7b9400ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:45:26.051 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:26.045+0000 7f7ba48c3640 1 -- 172.21.15.39:0/2567725584 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b94007bb0 con 0x7f7ba01083a0 2024-09-21T03:45:26.051 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:26.045+0000 7f7ba48c3640 1 -- 172.21.15.39:0/2567725584 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7b94007d50 con 0x7f7ba01083a0 2024-09-21T03:45:26.051 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:26.046+0000 7f7ba58c5640 1 -- 172.21.15.39:0/2567725584 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7ba01a06c0 con 0x7f7ba01083a0 2024-09-21T03:45:26.051 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:26.046+0000 7f7ba48c3640 1 -- 172.21.15.39:0/2567725584 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b9400f450 con 0x7f7ba01083a0 2024-09-21T03:45:26.051 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:26.046+0000 7f7ba58c5640 1 -- 172.21.15.39:0/2567725584 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7ba01a6e20 con 0x7f7ba01083a0 2024-09-21T03:45:26.051 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:26.047+0000 7f7ba58c5640 1 -- 172.21.15.39:0/2567725584 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7ba0102e70 con 0x7f7ba01083a0 2024-09-21T03:45:26.051 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:26.048+0000 7f7ba48c3640 1 -- 172.21.15.39:0/2567725584 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 22) ==== 154580+0+0 (secure 0 0 0) 0x7f7b94004f20 con 0x7f7ba01083a0 2024-09-21T03:45:26.052 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:26.049+0000 7f7ba48c3640 1 --2- 172.21.15.39:0/2567725584 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f7b880b65f0 0x7f7b880b8b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:26.052 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:26.049+0000 7f7b9effd640 1 --2- 172.21.15.39:0/2567725584 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f7b880b65f0 0x7f7b880b8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:26.052 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:26.049+0000 7f7ba48c3640 1 -- 172.21.15.39:0/2567725584 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8484+0+0 (secure 0 0 0) 0x7f7b940e7650 con 0x7f7ba01083a0 2024-09-21T03:45:26.052 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:26.049+0000 7f7b9effd640 1 --2- 172.21.15.39:0/2567725584 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f7b880b65f0 0x7f7b880b8b00 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f7b90005fd0 tx=0x7f7b90005850 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:45:26.056 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:26.053+0000 7f7ba48c3640 1 -- 172.21.15.39:0/2567725584 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f7b94014030 con 0x7f7ba01083a0 2024-09-21T03:45:26.281 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:26.278+0000 7f7ba58c5640 1 -- 172.21.15.39:0/2567725584 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f7ba002e640 con 0x7f7ba01083a0 2024-09-21T03:45:26.281 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:26.279+0000 7f7ba48c3640 1 -- 172.21.15.39:0/2567725584 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11978 (secure 0 0 0) 0x7f7b940adc80 con 0x7f7ba01083a0 2024-09-21T03:45:26.282 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T03:45:26.282 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":28,"btime":"2024-09-21T03:44:26:704224+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14790,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6835/508995295","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":508995295},{"type":"v1","addr":"172.21.15.39:6835","nonce":508995295}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14802,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6837/3915912607","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":3915912607},{"type":"v1","addr":"172.21.15.39:6837","nonce":3915912607}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T03:44:15.009872+0000","modified":"2024-09-21T03:44:26.017327+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24695,"mds_1":24667,"mds_2":24659,"mds_3":24646,"mds_4":14868},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14856":{"gid":14856,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.39:6839/185509192","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6838","nonce":185509192},{"type":"v1","addr":"172.21.15.39:6839","nonce":185509192}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14868":{"gid":14868,"name":"a","rank":4,"incarnation":25,"state":"up:active","state_seq":7,"addr":"172.21.15.39:6841/4247190456","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6840","nonce":4247190456},{"type":"v1","addr":"172.21.15.39:6841","nonce":4247190456}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24599":{"gid":24599,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.150:6833/3114643217","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":3114643217},{"type":"v1","addr":"172.21.15.150:6833","nonce":3114643217}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24601":{"gid":24601,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.161:6833/2963990654","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":2963990654},{"type":"v1","addr":"172.21.15.161:6833","nonce":2963990654}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24632":{"gid":24632,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.150:6835/724594008","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6834","nonce":724594008},{"type":"v1","addr":"172.21.15.150:6835","nonce":724594008}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24640":{"gid":24640,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.161:6835/3747882110","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":3747882110},{"type":"v1","addr":"172.21.15.161:6835","nonce":3747882110}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24646":{"gid":24646,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":10,"addr":"172.21.15.161:6837/3451544597","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6836","nonce":3451544597},{"type":"v1","addr":"172.21.15.161:6837","nonce":3451544597}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24659":{"gid":24659,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.150:6837/4017043144","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6836","nonce":4017043144},{"type":"v1","addr":"172.21.15.150:6837","nonce":4017043144}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24667":{"gid":24667,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.161:6839/2190107270","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6838","nonce":2190107270},{"type":"v1","addr":"172.21.15.161:6839","nonce":2190107270}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24695":{"gid":24695,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.150:6839/2949386907","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6838","nonce":2949386907},{"type":"v1","addr":"172.21.15.150:6839","nonce":2949386907}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24695,"qdb_cluster":[14868,24646,24659,24667,24695]},"id":1}]} 2024-09-21T03:45:26.282 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 28 2024-09-21T03:45:26.285 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:26.282+0000 7f7ba58c5640 1 -- 172.21.15.39:0/2567725584 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f7b880b65f0 msgr2=0x7f7b880b8b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:26.285 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:26.282+0000 7f7ba58c5640 1 --2- 172.21.15.39:0/2567725584 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f7b880b65f0 0x7f7b880b8b00 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f7b90005fd0 tx=0x7f7b90005850 comp rx=0 tx=0).stop 2024-09-21T03:45:26.285 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:26.282+0000 7f7ba58c5640 1 -- 172.21.15.39:0/2567725584 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7ba01083a0 msgr2=0x7f7ba019aa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:26.285 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:26.282+0000 7f7ba58c5640 1 --2- 172.21.15.39:0/2567725584 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7ba01083a0 0x7f7ba019aa00 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f7b9400c970 tx=0x7f7b9400ce30 comp rx=0 tx=0).stop 2024-09-21T03:45:26.285 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:26.282+0000 7f7ba58c5640 1 -- 172.21.15.39:0/2567725584 shutdown_connections 2024-09-21T03:45:26.285 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:26.282+0000 7f7ba58c5640 1 --2- 172.21.15.39:0/2567725584 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f7b880b65f0 0x7f7b880b8b00 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:26.285 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:26.282+0000 7f7ba58c5640 1 --2- 172.21.15.39:0/2567725584 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7ba0108cf0 0x7f7ba019fca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:26.286 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:26.282+0000 7f7ba58c5640 1 --2- 172.21.15.39:0/2567725584 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7ba01083a0 0x7f7ba019aa00 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:26.286 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:26.282+0000 7f7ba58c5640 1 --2- 172.21.15.39:0/2567725584 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7ba0102a60 0x7f7ba019a460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:26.286 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:26.282+0000 7f7ba58c5640 1 -- 172.21.15.39:0/2567725584 >> 172.21.15.39:0/2567725584 conn(0x7f7ba0079730 msgr2=0x7f7ba006b1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:45:26.286 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:26.282+0000 7f7ba58c5640 1 -- 172.21.15.39:0/2567725584 shutdown_connections 2024-09-21T03:45:26.286 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:26.282+0000 7f7ba58c5640 1 -- 172.21.15.39:0/2567725584 wait complete. 2024-09-21T03:45:26.924 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:45:26 smithi039 ceph-mon[31211]: pgmap v307: 97 pgs: 97 active+clean; 647 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-09-21T03:45:26.924 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:45:26 smithi039 ceph-mon[31211]: from='client.? 172.21.15.39:0/2567725584' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:45:26.954 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 51116d5c-77ca-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T03:45:27.212 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/51116d5c-77ca-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:45:27.218 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:45:26 smithi161 ceph-mon[34533]: pgmap v307: 97 pgs: 97 active+clean; 647 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-09-21T03:45:27.218 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:45:26 smithi161 ceph-mon[34533]: from='client.? 172.21.15.39:0/2567725584' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:45:27.225 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:45:26 smithi150 ceph-mon[34289]: pgmap v307: 97 pgs: 97 active+clean; 647 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-09-21T03:45:27.225 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:45:26 smithi150 ceph-mon[34289]: from='client.? 172.21.15.39:0/2567725584' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:45:27.705 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:27.702+0000 7fe768dab640 1 -- 172.21.15.39:0/1550416956 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe764109e00 msgr2=0x7fe764110630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:27.705 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:27.702+0000 7fe768dab640 1 --2- 172.21.15.39:0/1550416956 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe764109e00 0x7fe764110630 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7fe75c009ca0 tx=0x7fe75c02a8c0 comp rx=0 tx=0).stop 2024-09-21T03:45:27.705 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:27.702+0000 7fe768dab640 1 -- 172.21.15.39:0/1550416956 shutdown_connections 2024-09-21T03:45:27.705 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:27.702+0000 7fe768dab640 1 --2- 172.21.15.39:0/1550416956 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe764109e00 0x7fe764110630 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:27.705 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:27.702+0000 7fe768dab640 1 --2- 172.21.15.39:0/1550416956 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe7641094b0 0x7fe7641098c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:27.705 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:27.702+0000 7fe768dab640 1 --2- 172.21.15.39:0/1550416956 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe764103b70 0x7fe764103f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:27.705 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:27.702+0000 7fe768dab640 1 -- 172.21.15.39:0/1550416956 >> 172.21.15.39:0/1550416956 conn(0x7fe7640ff5e0 msgr2=0x7fe764101a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:45:27.705 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:27.702+0000 7fe768dab640 1 -- 172.21.15.39:0/1550416956 shutdown_connections 2024-09-21T03:45:27.706 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:27.702+0000 7fe768dab640 1 -- 172.21.15.39:0/1550416956 wait complete. 2024-09-21T03:45:27.707 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:27.703+0000 7fe768dab640 1 Processor -- start 2024-09-21T03:45:27.707 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:27.703+0000 7fe768dab640 1 -- start start 2024-09-21T03:45:27.708 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:27.703+0000 7fe768dab640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe764103b70 0x7fe76419a4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:27.708 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:27.703+0000 7fe768dab640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe7641094b0 0x7fe76419aa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:27.708 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:27.703+0000 7fe768dab640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe764109e00 0x7fe76419fd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:27.708 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:27.703+0000 7fe768dab640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fe764113b10 con 0x7fe764109e00 2024-09-21T03:45:27.708 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:27.703+0000 7fe768dab640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fe764113990 con 0x7fe7641094b0 2024-09-21T03:45:27.708 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:27.703+0000 7fe768dab640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fe764113c90 con 0x7fe764103b70 2024-09-21T03:45:27.708 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:27.704+0000 7fe762d9d640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe764109e00 0x7fe76419fd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:27.708 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:27.704+0000 7fe762d9d640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe764109e00 0x7fe76419fd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:40482/0 (socket says 172.21.15.39:40482) 2024-09-21T03:45:27.708 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:27.704+0000 7fe762d9d640 1 -- 172.21.15.39:0/661868835 learned_addr learned my addr 172.21.15.39:0/661868835 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:45:27.708 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:27.704+0000 7fe761d9b640 1 --2- 172.21.15.39:0/661868835 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe7641094b0 0x7fe76419aa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:27.708 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:27.704+0000 7fe76259c640 1 --2- 172.21.15.39:0/661868835 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe764103b70 0x7fe76419a4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:27.708 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:27.704+0000 7fe762d9d640 1 -- 172.21.15.39:0/661868835 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe764103b70 msgr2=0x7fe76419a4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:27.708 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:27.704+0000 7fe762d9d640 1 --2- 172.21.15.39:0/661868835 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe764103b70 0x7fe76419a4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:27.709 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:27.704+0000 7fe762d9d640 1 -- 172.21.15.39:0/661868835 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe7641094b0 msgr2=0x7fe76419aa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:27.709 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:27.704+0000 7fe762d9d640 1 --2- 172.21.15.39:0/661868835 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe7641094b0 0x7fe76419aa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:27.709 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:27.704+0000 7fe762d9d640 1 -- 172.21.15.39:0/661868835 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe7641a0450 con 0x7fe764109e00 2024-09-21T03:45:27.709 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:27.704+0000 7fe762d9d640 1 --2- 172.21.15.39:0/661868835 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe764109e00 0x7fe76419fd20 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7fe75c009430 tx=0x7fe75c0317c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:45:27.709 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:27.704+0000 7fe7537fe640 1 -- 172.21.15.39:0/661868835 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe75c003b70 con 0x7fe764109e00 2024-09-21T03:45:27.709 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:27.705+0000 7fe768dab640 1 -- 172.21.15.39:0/661868835 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe7641a06e0 con 0x7fe764109e00 2024-09-21T03:45:27.709 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:27.705+0000 7fe7537fe640 1 -- 172.21.15.39:0/661868835 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe75c003d10 con 0x7fe764109e00 2024-09-21T03:45:27.709 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:27.705+0000 7fe7537fe640 1 -- 172.21.15.39:0/661868835 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe75c02fa30 con 0x7fe764109e00 2024-09-21T03:45:27.709 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:27.705+0000 7fe768dab640 1 -- 172.21.15.39:0/661868835 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe7641a6ea0 con 0x7fe764109e00 2024-09-21T03:45:27.709 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:27.706+0000 7fe768dab640 1 -- 172.21.15.39:0/661868835 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe764103f80 con 0x7fe764109e00 2024-09-21T03:45:27.710 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:27.707+0000 7fe7537fe640 1 -- 172.21.15.39:0/661868835 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 22) ==== 154580+0+0 (secure 0 0 0) 0x7fe75c02fbd0 con 0x7fe764109e00 2024-09-21T03:45:27.711 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:27.708+0000 7fe7537fe640 1 --2- 172.21.15.39:0/661868835 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7fe74c0b65f0 0x7fe74c0b8b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:27.711 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:27.709+0000 7fe76259c640 1 --2- 172.21.15.39:0/661868835 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7fe74c0b65f0 0x7fe74c0b8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:27.711 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:27.709+0000 7fe7537fe640 1 -- 172.21.15.39:0/661868835 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8484+0+0 (secure 0 0 0) 0x7fe75c108a30 con 0x7fe764109e00 2024-09-21T03:45:27.712 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:27.709+0000 7fe76259c640 1 --2- 172.21.15.39:0/661868835 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7fe74c0b65f0 0x7fe74c0b8b00 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fe754009b60 tx=0x7fe754005f90 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:45:27.714 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:27.713+0000 7fe7537fe640 1 -- 172.21.15.39:0/661868835 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fe75c0cf060 con 0x7fe764109e00 2024-09-21T03:45:27.940 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:27.937+0000 7fe768dab640 1 -- 172.21.15.39:0/661868835 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe76419b610 con 0x7fe764109e00 2024-09-21T03:45:27.940 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:27.938+0000 7fe7537fe640 1 -- 172.21.15.39:0/661868835 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11978 (secure 0 0 0) 0x7fe76419b610 con 0x7fe764109e00 2024-09-21T03:45:27.941 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T03:45:27.941 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":28,"btime":"2024-09-21T03:44:26:704224+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14790,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6835/508995295","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":508995295},{"type":"v1","addr":"172.21.15.39:6835","nonce":508995295}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14802,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6837/3915912607","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":3915912607},{"type":"v1","addr":"172.21.15.39:6837","nonce":3915912607}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T03:44:15.009872+0000","modified":"2024-09-21T03:44:26.017327+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24695,"mds_1":24667,"mds_2":24659,"mds_3":24646,"mds_4":14868},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14856":{"gid":14856,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.39:6839/185509192","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6838","nonce":185509192},{"type":"v1","addr":"172.21.15.39:6839","nonce":185509192}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14868":{"gid":14868,"name":"a","rank":4,"incarnation":25,"state":"up:active","state_seq":7,"addr":"172.21.15.39:6841/4247190456","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6840","nonce":4247190456},{"type":"v1","addr":"172.21.15.39:6841","nonce":4247190456}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24599":{"gid":24599,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.150:6833/3114643217","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":3114643217},{"type":"v1","addr":"172.21.15.150:6833","nonce":3114643217}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24601":{"gid":24601,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.161:6833/2963990654","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":2963990654},{"type":"v1","addr":"172.21.15.161:6833","nonce":2963990654}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24632":{"gid":24632,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.150:6835/724594008","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6834","nonce":724594008},{"type":"v1","addr":"172.21.15.150:6835","nonce":724594008}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24640":{"gid":24640,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.161:6835/3747882110","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":3747882110},{"type":"v1","addr":"172.21.15.161:6835","nonce":3747882110}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24646":{"gid":24646,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":10,"addr":"172.21.15.161:6837/3451544597","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6836","nonce":3451544597},{"type":"v1","addr":"172.21.15.161:6837","nonce":3451544597}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24659":{"gid":24659,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.150:6837/4017043144","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6836","nonce":4017043144},{"type":"v1","addr":"172.21.15.150:6837","nonce":4017043144}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24667":{"gid":24667,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.161:6839/2190107270","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6838","nonce":2190107270},{"type":"v1","addr":"172.21.15.161:6839","nonce":2190107270}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24695":{"gid":24695,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.150:6839/2949386907","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6838","nonce":2949386907},{"type":"v1","addr":"172.21.15.150:6839","nonce":2949386907}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24695,"qdb_cluster":[14868,24646,24659,24667,24695]},"id":1}]} 2024-09-21T03:45:27.941 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 28 2024-09-21T03:45:27.943 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:27.940+0000 7fe768dab640 1 -- 172.21.15.39:0/661868835 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7fe74c0b65f0 msgr2=0x7fe74c0b8b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:27.943 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:27.940+0000 7fe768dab640 1 --2- 172.21.15.39:0/661868835 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7fe74c0b65f0 0x7fe74c0b8b00 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fe754009b60 tx=0x7fe754005f90 comp rx=0 tx=0).stop 2024-09-21T03:45:27.943 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:27.940+0000 7fe768dab640 1 -- 172.21.15.39:0/661868835 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe764109e00 msgr2=0x7fe76419fd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:27.943 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:27.940+0000 7fe768dab640 1 --2- 172.21.15.39:0/661868835 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe764109e00 0x7fe76419fd20 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7fe75c009430 tx=0x7fe75c0317c0 comp rx=0 tx=0).stop 2024-09-21T03:45:27.943 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:27.940+0000 7fe768dab640 1 -- 172.21.15.39:0/661868835 shutdown_connections 2024-09-21T03:45:27.943 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:27.940+0000 7fe768dab640 1 --2- 172.21.15.39:0/661868835 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7fe74c0b65f0 0x7fe74c0b8b00 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:27.943 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:27.940+0000 7fe768dab640 1 --2- 172.21.15.39:0/661868835 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe764109e00 0x7fe76419fd20 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:27.943 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:27.940+0000 7fe768dab640 1 --2- 172.21.15.39:0/661868835 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe7641094b0 0x7fe76419aa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:27.943 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:27.940+0000 7fe768dab640 1 --2- 172.21.15.39:0/661868835 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe764103b70 0x7fe76419a4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:27.943 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:27.940+0000 7fe768dab640 1 -- 172.21.15.39:0/661868835 >> 172.21.15.39:0/661868835 conn(0x7fe7640ff5e0 msgr2=0x7fe764106a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:45:27.944 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:27.940+0000 7fe768dab640 1 -- 172.21.15.39:0/661868835 shutdown_connections 2024-09-21T03:45:27.944 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:27.940+0000 7fe768dab640 1 -- 172.21.15.39:0/661868835 wait complete. 2024-09-21T03:45:28.651 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 51116d5c-77ca-11ef-baf5-efdc52797490 -- ceph osd dump --format=json 2024-09-21T03:45:28.918 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/51116d5c-77ca-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:45:29.039 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:45:28 smithi039 ceph-mon[31211]: pgmap v308: 97 pgs: 97 active+clean; 647 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-09-21T03:45:29.039 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:45:28 smithi039 ceph-mon[31211]: from='client.? 172.21.15.39:0/661868835' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:45:29.218 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:45:28 smithi161 ceph-mon[34533]: pgmap v308: 97 pgs: 97 active+clean; 647 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-09-21T03:45:29.218 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:45:28 smithi161 ceph-mon[34533]: from='client.? 172.21.15.39:0/661868835' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:45:29.224 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:45:28 smithi150 ceph-mon[34289]: pgmap v308: 97 pgs: 97 active+clean; 647 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-09-21T03:45:29.224 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:45:28 smithi150 ceph-mon[34289]: from='client.? 172.21.15.39:0/661868835' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:45:29.410 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:29.391+0000 7f31e1d79640 1 -- 172.21.15.39:0/369757961 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f31dc1049f0 msgr2=0x7f31dc104e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:29.410 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:29.391+0000 7f31e1d79640 1 --2- 172.21.15.39:0/369757961 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f31dc1049f0 0x7f31dc104e00 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f31cc009900 tx=0x7f31cc02a410 comp rx=0 tx=0).stop 2024-09-21T03:45:29.410 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:29.392+0000 7f31e1d79640 1 -- 172.21.15.39:0/369757961 shutdown_connections 2024-09-21T03:45:29.410 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:29.392+0000 7f31e1d79640 1 --2- 172.21.15.39:0/369757961 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f31dc10de50 0x7f31dc110340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:29.410 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:29.392+0000 7f31e1d79640 1 --2- 172.21.15.39:0/369757961 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f31dc105340 0x7f31dc10d890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:29.411 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:29.392+0000 7f31e1d79640 1 --2- 172.21.15.39:0/369757961 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f31dc1049f0 0x7f31dc104e00 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:29.411 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:29.392+0000 7f31e1d79640 1 -- 172.21.15.39:0/369757961 >> 172.21.15.39:0/369757961 conn(0x7f31dc079750 msgr2=0x7f31dc079b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:45:29.411 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:29.392+0000 7f31e1d79640 1 -- 172.21.15.39:0/369757961 shutdown_connections 2024-09-21T03:45:29.411 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:29.392+0000 7f31e1d79640 1 -- 172.21.15.39:0/369757961 wait complete. 2024-09-21T03:45:29.411 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:29.392+0000 7f31e1d79640 1 Processor -- start 2024-09-21T03:45:29.411 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:29.392+0000 7f31e1d79640 1 -- start start 2024-09-21T03:45:29.411 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:29.393+0000 7f31e1d79640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f31dc1049f0 0x7f31dc19a560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:29.411 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:29.393+0000 7f31e1d79640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f31dc105340 0x7f31dc19ab00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:29.411 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:29.393+0000 7f31e1d79640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f31dc10de50 0x7f31dc19fda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:29.411 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:29.393+0000 7f31e1d79640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f31dc113b10 con 0x7f31dc10de50 2024-09-21T03:45:29.411 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:29.393+0000 7f31e1d79640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f31dc113990 con 0x7f31dc105340 2024-09-21T03:45:29.411 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:29.393+0000 7f31e1d79640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f31dc113c90 con 0x7f31dc1049f0 2024-09-21T03:45:29.412 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:29.393+0000 7f31dbfff640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f31dc10de50 0x7f31dc19fda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:29.412 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:29.393+0000 7f31dbfff640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f31dc10de50 0x7f31dc19fda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:40500/0 (socket says 172.21.15.39:40500) 2024-09-21T03:45:29.412 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:29.393+0000 7f31dbfff640 1 -- 172.21.15.39:0/3649713553 learned_addr learned my addr 172.21.15.39:0/3649713553 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:45:29.412 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:29.393+0000 7f31db7fe640 1 --2- 172.21.15.39:0/3649713553 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f31dc1049f0 0x7f31dc19a560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:29.412 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:29.393+0000 7f31daffd640 1 --2- 172.21.15.39:0/3649713553 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f31dc105340 0x7f31dc19ab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:29.412 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:29.393+0000 7f31dbfff640 1 -- 172.21.15.39:0/3649713553 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f31dc1049f0 msgr2=0x7f31dc19a560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:29.412 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:29.393+0000 7f31dbfff640 1 --2- 172.21.15.39:0/3649713553 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f31dc1049f0 0x7f31dc19a560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:29.412 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:29.393+0000 7f31dbfff640 1 -- 172.21.15.39:0/3649713553 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f31dc105340 msgr2=0x7f31dc19ab00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:29.412 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:29.393+0000 7f31dbfff640 1 --2- 172.21.15.39:0/3649713553 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f31dc105340 0x7f31dc19ab00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:29.412 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:29.393+0000 7f31dbfff640 1 -- 172.21.15.39:0/3649713553 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f31dc1a04d0 con 0x7f31dc10de50 2024-09-21T03:45:29.412 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:29.393+0000 7f31dbfff640 1 --2- 172.21.15.39:0/3649713553 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f31dc10de50 0x7f31dc19fda0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f31d400cad0 tx=0x7f31d400cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:45:29.412 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:29.394+0000 7f31d8ff9640 1 -- 172.21.15.39:0/3649713553 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31d400ae20 con 0x7f31dc10de50 2024-09-21T03:45:29.412 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:29.394+0000 7f31d8ff9640 1 -- 172.21.15.39:0/3649713553 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f31d4005020 con 0x7f31dc10de50 2024-09-21T03:45:29.412 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:29.394+0000 7f31d8ff9640 1 -- 172.21.15.39:0/3649713553 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31d400f6b0 con 0x7f31dc10de50 2024-09-21T03:45:29.413 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:29.394+0000 7f31e1d79640 1 -- 172.21.15.39:0/3649713553 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f31dc1a07c0 con 0x7f31dc10de50 2024-09-21T03:45:29.413 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:29.394+0000 7f31e1d79640 1 -- 172.21.15.39:0/3649713553 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f31dc1a6f20 con 0x7f31dc10de50 2024-09-21T03:45:29.413 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:29.395+0000 7f31e1d79640 1 -- 172.21.15.39:0/3649713553 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f31dc104e00 con 0x7f31dc10de50 2024-09-21T03:45:29.413 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:29.396+0000 7f31d8ff9640 1 -- 172.21.15.39:0/3649713553 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 22) ==== 154580+0+0 (secure 0 0 0) 0x7f31d40051d0 con 0x7f31dc10de50 2024-09-21T03:45:29.413 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:29.397+0000 7f31d8ff9640 1 --2- 172.21.15.39:0/3649713553 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f31c40b6670 0x7f31c40b8b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:29.413 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:29.397+0000 7f31db7fe640 1 --2- 172.21.15.39:0/3649713553 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f31c40b6670 0x7f31c40b8b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:29.413 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:29.397+0000 7f31d8ff9640 1 -- 172.21.15.39:0/3649713553 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8484+0+0 (secure 0 0 0) 0x7f31d40e75d0 con 0x7f31dc10de50 2024-09-21T03:45:29.413 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:29.398+0000 7f31db7fe640 1 --2- 172.21.15.39:0/3649713553 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f31c40b6670 0x7f31c40b8b80 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f31cc031440 tx=0x7f31cc005e80 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:45:29.413 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:29.401+0000 7f31d8ff9640 1 -- 172.21.15.39:0/3649713553 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f31d4014030 con 0x7f31dc10de50 2024-09-21T03:45:29.592 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:29.590+0000 7f31e1d79640 1 -- 172.21.15.39:0/3649713553 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f31dc1a7280 con 0x7f31dc10de50 2024-09-21T03:45:29.593 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:29.592+0000 7f31d8ff9640 1 -- 172.21.15.39:0/3649713553 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v79) ==== 74+0+23096 (secure 0 0 0) 0x7f31d40adc00 con 0x7f31dc10de50 2024-09-21T03:45:29.593 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T03:45:29.594 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":79,"fsid":"51116d5c-77ca-11ef-baf5-efdc52797490","created":"2024-09-21T03:35:33.631350+0000","modified":"2024-09-21T03:44:27.402588+0000","last_up_change":"2024-09-21T03:43:24.242275+0000","last_in_change":"2024-09-21T03:43:00.886552+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T03:39:29.859542+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-21T03:44:10.886435+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"77","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-21T03:44:13.041783+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"77","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1ea7148f-2ace-4c10-86fe-b8019de80c1f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6802","nonce":870188791},{"type":"v1","addr":"172.21.15.39:6803","nonce":870188791}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6804","nonce":870188791},{"type":"v1","addr":"172.21.15.39:6805","nonce":870188791}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6808","nonce":870188791},{"type":"v1","addr":"172.21.15.39:6809","nonce":870188791}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6806","nonce":870188791},{"type":"v1","addr":"172.21.15.39:6807","nonce":870188791}]},"public_addr":"172.21.15.39:6803/870188791","cluster_addr":"172.21.15.39:6805/870188791","heartbeat_back_addr":"172.21.15.39:6809/870188791","heartbeat_front_addr":"172.21.15.39:6807/870188791","state":["exists","up"]},{"osd":1,"uuid":"7cb5db90-28e8-4a30-bedb-c039687d48b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6800","nonce":172427832},{"type":"v1","addr":"172.21.15.150:6801","nonce":172427832}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6802","nonce":172427832},{"type":"v1","addr":"172.21.15.150:6803","nonce":172427832}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6806","nonce":172427832},{"type":"v1","addr":"172.21.15.150:6807","nonce":172427832}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6804","nonce":172427832},{"type":"v1","addr":"172.21.15.150:6805","nonce":172427832}]},"public_addr":"172.21.15.150:6801/172427832","cluster_addr":"172.21.15.150:6803/172427832","heartbeat_back_addr":"172.21.15.150:6807/172427832","heartbeat_front_addr":"172.21.15.150:6805/172427832","state":["exists","up"]},{"osd":2,"uuid":"b25c6ab3-fc95-4751-bd63-6e3fc23e8bd7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":19,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6800","nonce":2727876133},{"type":"v1","addr":"172.21.15.161:6801","nonce":2727876133}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6802","nonce":2727876133},{"type":"v1","addr":"172.21.15.161:6803","nonce":2727876133}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6806","nonce":2727876133},{"type":"v1","addr":"172.21.15.161:6807","nonce":2727876133}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6804","nonce":2727876133},{"type":"v1","addr":"172.21.15.161:6805","nonce":2727876133}]},"public_addr":"172.21.15.161:6801/2727876133","cluster_addr":"172.21.15.161:6803/2727876133","heartbeat_back_addr":"172.21.15.161:6807/2727876133","heartbeat_front_addr":"172.21.15.161:6805/2727876133","state":["exists","up"]},{"osd":3,"uuid":"14417ae1-4536-45ba-9a5c-c3b447d1a72d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6810","nonce":295394468},{"type":"v1","addr":"172.21.15.39:6811","nonce":295394468}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6812","nonce":295394468},{"type":"v1","addr":"172.21.15.39:6813","nonce":295394468}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6816","nonce":295394468},{"type":"v1","addr":"172.21.15.39:6817","nonce":295394468}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6814","nonce":295394468},{"type":"v1","addr":"172.21.15.39:6815","nonce":295394468}]},"public_addr":"172.21.15.39:6811/295394468","cluster_addr":"172.21.15.39:6813/295394468","heartbeat_back_addr":"172.21.15.39:6817/295394468","heartbeat_front_addr":"172.21.15.39:6815/295394468","state":["exists","up"]},{"osd":4,"uuid":"33c732ef-20c2-43c4-abe6-045b687fc521","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6808","nonce":3097721553},{"type":"v1","addr":"172.21.15.150:6809","nonce":3097721553}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6810","nonce":3097721553},{"type":"v1","addr":"172.21.15.150:6811","nonce":3097721553}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6814","nonce":3097721553},{"type":"v1","addr":"172.21.15.150:6815","nonce":3097721553}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6812","nonce":3097721553},{"type":"v1","addr":"172.21.15.150:6813","nonce":3097721553}]},"public_addr":"172.21.15.150:6809/3097721553","cluster_addr":"172.21.15.150:6811/3097721553","heartbeat_back_addr":"172.21.15.150:6815/3097721553","heartbeat_front_addr":"172.21.15.150:6813/3097721553","state":["exists","up"]},{"osd":5,"uuid":"74673722-abd6-400a-bb50-ec3dfba0c5e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6808","nonce":742751023},{"type":"v1","addr":"172.21.15.161:6809","nonce":742751023}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6810","nonce":742751023},{"type":"v1","addr":"172.21.15.161:6811","nonce":742751023}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6814","nonce":742751023},{"type":"v1","addr":"172.21.15.161:6815","nonce":742751023}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6812","nonce":742751023},{"type":"v1","addr":"172.21.15.161:6813","nonce":742751023}]},"public_addr":"172.21.15.161:6809/742751023","cluster_addr":"172.21.15.161:6811/742751023","heartbeat_back_addr":"172.21.15.161:6815/742751023","heartbeat_front_addr":"172.21.15.161:6813/742751023","state":["exists","up"]},{"osd":6,"uuid":"06548677-74c4-4efa-8054-df09b957b649","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6818","nonce":1072700290},{"type":"v1","addr":"172.21.15.39:6819","nonce":1072700290}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6820","nonce":1072700290},{"type":"v1","addr":"172.21.15.39:6821","nonce":1072700290}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6824","nonce":1072700290},{"type":"v1","addr":"172.21.15.39:6825","nonce":1072700290}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6822","nonce":1072700290},{"type":"v1","addr":"172.21.15.39:6823","nonce":1072700290}]},"public_addr":"172.21.15.39:6819/1072700290","cluster_addr":"172.21.15.39:6821/1072700290","heartbeat_back_addr":"172.21.15.39:6825/1072700290","heartbeat_front_addr":"172.21.15.39:6823/1072700290","state":["exists","up"]},{"osd":7,"uuid":"838735ca-c693-4f58-9426-b8ee857dbe3a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6816","nonce":2162619103},{"type":"v1","addr":"172.21.15.150:6817","nonce":2162619103}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6818","nonce":2162619103},{"type":"v1","addr":"172.21.15.150:6819","nonce":2162619103}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6822","nonce":2162619103},{"type":"v1","addr":"172.21.15.150:6823","nonce":2162619103}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6820","nonce":2162619103},{"type":"v1","addr":"172.21.15.150:6821","nonce":2162619103}]},"public_addr":"172.21.15.150:6817/2162619103","cluster_addr":"172.21.15.150:6819/2162619103","heartbeat_back_addr":"172.21.15.150:6823/2162619103","heartbeat_front_addr":"172.21.15.150:6821/2162619103","state":["exists","up"]},{"osd":8,"uuid":"4973b357-b4c2-4c00-b7f7-dd25012a0f72","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6816","nonce":52966633},{"type":"v1","addr":"172.21.15.161:6817","nonce":52966633}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6818","nonce":52966633},{"type":"v1","addr":"172.21.15.161:6819","nonce":52966633}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6822","nonce":52966633},{"type":"v1","addr":"172.21.15.161:6823","nonce":52966633}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6820","nonce":52966633},{"type":"v1","addr":"172.21.15.161:6821","nonce":52966633}]},"public_addr":"172.21.15.161:6817/52966633","cluster_addr":"172.21.15.161:6819/52966633","heartbeat_back_addr":"172.21.15.161:6823/52966633","heartbeat_front_addr":"172.21.15.161:6821/52966633","state":["exists","up"]},{"osd":9,"uuid":"6028613c-ebfd-42ea-9949-3f0d5f6cf025","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6826","nonce":2001303253},{"type":"v1","addr":"172.21.15.39:6827","nonce":2001303253}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6828","nonce":2001303253},{"type":"v1","addr":"172.21.15.39:6829","nonce":2001303253}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6832","nonce":2001303253},{"type":"v1","addr":"172.21.15.39:6833","nonce":2001303253}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6830","nonce":2001303253},{"type":"v1","addr":"172.21.15.39:6831","nonce":2001303253}]},"public_addr":"172.21.15.39:6827/2001303253","cluster_addr":"172.21.15.39:6829/2001303253","heartbeat_back_addr":"172.21.15.39:6833/2001303253","heartbeat_front_addr":"172.21.15.39:6831/2001303253","state":["exists","up"]},{"osd":10,"uuid":"1c71c765-2a6c-4ffe-9fe1-c8adae3d1c32","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6824","nonce":2484627938},{"type":"v1","addr":"172.21.15.150:6825","nonce":2484627938}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6826","nonce":2484627938},{"type":"v1","addr":"172.21.15.150:6827","nonce":2484627938}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6830","nonce":2484627938},{"type":"v1","addr":"172.21.15.150:6831","nonce":2484627938}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6828","nonce":2484627938},{"type":"v1","addr":"172.21.15.150:6829","nonce":2484627938}]},"public_addr":"172.21.15.150:6825/2484627938","cluster_addr":"172.21.15.150:6827/2484627938","heartbeat_back_addr":"172.21.15.150:6831/2484627938","heartbeat_front_addr":"172.21.15.150:6829/2484627938","state":["exists","up"]},{"osd":11,"uuid":"f382f522-62db-46da-997e-2b28f84d1612","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6824","nonce":2178709707},{"type":"v1","addr":"172.21.15.161:6825","nonce":2178709707}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6826","nonce":2178709707},{"type":"v1","addr":"172.21.15.161:6827","nonce":2178709707}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6830","nonce":2178709707},{"type":"v1","addr":"172.21.15.161:6831","nonce":2178709707}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6828","nonce":2178709707},{"type":"v1","addr":"172.21.15.161:6829","nonce":2178709707}]},"public_addr":"172.21.15.161:6825/2178709707","cluster_addr":"172.21.15.161:6827/2178709707","heartbeat_back_addr":"172.21.15.161:6831/2178709707","heartbeat_front_addr":"172.21.15.161:6829/2178709707","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:38:35.540904+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:39:01.455053+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:39:27.052867+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:39:55.173529+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:40:21.298396+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:40:47.131235+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:41:14.475209+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:41:39.920036+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:42:05.587065+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:42:32.624726+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:42:57.479185+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:43:22.203832+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.5","mappings":[{"from":8,"to":11}]},{"pgid":"2.8","mappings":[{"from":2,"to":11}]},{"pgid":"2.c","mappings":[{"from":2,"to":1}]},{"pgid":"2.1f","mappings":[{"from":8,"to":4}]},{"pgid":"3.2","mappings":[{"from":3,"to":9}]},{"pgid":"3.14","mappings":[{"from":4,"to":6}]},{"pgid":"3.1a","mappings":[{"from":5,"to":7}]},{"pgid":"3.1d","mappings":[{"from":4,"to":8}]},{"pgid":"3.2c","mappings":[{"from":4,"to":9}]},{"pgid":"3.38","mappings":[{"from":3,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.39:6801/2438110611":"2024-09-22T03:37:25.391457+0000","172.21.15.39:6800/2438110611":"2024-09-22T03:37:25.391457+0000","172.21.15.39:6800/173264695":"2024-09-22T03:35:57.104467+0000","172.21.15.39:6801/173264695":"2024-09-22T03:35:57.104467+0000","172.21.15.39:0/2637182284":"2024-09-22T03:35:57.104467+0000","172.21.15.39:0/1517304561":"2024-09-22T03:36:24.112592+0000","172.21.15.39:6801/1435622199":"2024-09-22T03:36:24.112592+0000","172.21.15.39:6800/1435622199":"2024-09-22T03:36:24.112592+0000","172.21.15.39:0/346832469":"2024-09-22T03:35:57.104467+0000","172.21.15.39:0/3160777371":"2024-09-22T03:36:24.112592+0000","172.21.15.39:0/1730342855":"2024-09-22T03:35:57.104467+0000","172.21.15.39:0/1955726766":"2024-09-22T03:36:24.112592+0000","172.21.15.39:0/3294132796":"2024-09-22T03:37:25.391457+0000","172.21.15.39:0/1295810162":"2024-09-22T03:37:25.391457+0000","172.21.15.39:0/3729230474":"2024-09-22T03:37:25.391457+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T03:45:29.596 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:29.594+0000 7f31e1d79640 1 -- 172.21.15.39:0/3649713553 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f31c40b6670 msgr2=0x7f31c40b8b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:29.596 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:29.594+0000 7f31e1d79640 1 --2- 172.21.15.39:0/3649713553 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f31c40b6670 0x7f31c40b8b80 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f31cc031440 tx=0x7f31cc005e80 comp rx=0 tx=0).stop 2024-09-21T03:45:29.596 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:29.594+0000 7f31e1d79640 1 -- 172.21.15.39:0/3649713553 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f31dc10de50 msgr2=0x7f31dc19fda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:29.596 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:29.594+0000 7f31e1d79640 1 --2- 172.21.15.39:0/3649713553 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f31dc10de50 0x7f31dc19fda0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f31d400cad0 tx=0x7f31d400cf90 comp rx=0 tx=0).stop 2024-09-21T03:45:29.596 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:29.594+0000 7f31e1d79640 1 -- 172.21.15.39:0/3649713553 shutdown_connections 2024-09-21T03:45:29.596 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:29.594+0000 7f31e1d79640 1 --2- 172.21.15.39:0/3649713553 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f31c40b6670 0x7f31c40b8b80 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:29.596 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:29.594+0000 7f31e1d79640 1 --2- 172.21.15.39:0/3649713553 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f31dc10de50 0x7f31dc19fda0 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:29.597 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:29.594+0000 7f31e1d79640 1 --2- 172.21.15.39:0/3649713553 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f31dc105340 0x7f31dc19ab00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:29.597 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:29.594+0000 7f31e1d79640 1 --2- 172.21.15.39:0/3649713553 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f31dc1049f0 0x7f31dc19a560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:29.597 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:29.594+0000 7f31e1d79640 1 -- 172.21.15.39:0/3649713553 >> 172.21.15.39:0/3649713553 conn(0x7f31dc079750 msgr2=0x7f31dc106860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:45:29.597 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:29.594+0000 7f31e1d79640 1 -- 172.21.15.39:0/3649713553 shutdown_connections 2024-09-21T03:45:29.597 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:29.594+0000 7f31e1d79640 1 -- 172.21.15.39:0/3649713553 wait complete. 2024-09-21T03:45:30.217 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:45:29 smithi161 ceph-mon[34533]: from='client.? 172.21.15.39:0/3649713553' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T03:45:30.224 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:45:29 smithi150 ceph-mon[34289]: from='client.? 172.21.15.39:0/3649713553' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T03:45:30.230 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:45:29 smithi039 ceph-mon[31211]: from='client.? 172.21.15.39:0/3649713553' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T03:45:30.264 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 51116d5c-77ca-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T03:45:30.525 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/51116d5c-77ca-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:45:30.901 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:45:30 smithi039 ceph-mon[31211]: pgmap v309: 97 pgs: 97 active+clean; 647 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-09-21T03:45:30.995 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:30.991+0000 7f953b950640 1 -- 172.21.15.39:0/1638567559 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9534105330 msgr2=0x7f953410d880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:30.995 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:30.991+0000 7f953b950640 1 --2- 172.21.15.39:0/1638567559 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9534105330 0x7f953410d880 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f9524009900 tx=0x7f952402a410 comp rx=0 tx=0).stop 2024-09-21T03:45:30.995 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:30.991+0000 7f953b950640 1 -- 172.21.15.39:0/1638567559 shutdown_connections 2024-09-21T03:45:30.995 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:30.991+0000 7f953b950640 1 --2- 172.21.15.39:0/1638567559 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f953410de40 0x7f9534110330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:30.995 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:30.991+0000 7f953b950640 1 --2- 172.21.15.39:0/1638567559 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9534105330 0x7f953410d880 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:30.995 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:30.991+0000 7f953b950640 1 --2- 172.21.15.39:0/1638567559 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f95341049e0 0x7f9534104df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:30.996 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:30.991+0000 7f953b950640 1 -- 172.21.15.39:0/1638567559 >> 172.21.15.39:0/1638567559 conn(0x7f9534079750 msgr2=0x7f9534079b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:45:30.996 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:30.991+0000 7f953b950640 1 -- 172.21.15.39:0/1638567559 shutdown_connections 2024-09-21T03:45:30.996 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:30.991+0000 7f953b950640 1 -- 172.21.15.39:0/1638567559 wait complete. 2024-09-21T03:45:30.996 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:30.992+0000 7f953b950640 1 Processor -- start 2024-09-21T03:45:30.996 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:30.992+0000 7f953b950640 1 -- start start 2024-09-21T03:45:30.996 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:30.992+0000 7f953b950640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f95341049e0 0x7f953419a530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:30.996 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:30.992+0000 7f953b950640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9534105330 0x7f953419aad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:30.996 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:30.992+0000 7f953b950640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f953410de40 0x7f953419fd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:30.996 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:30.992+0000 7f953b950640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f9534113ae0 con 0x7f9534105330 2024-09-21T03:45:30.996 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:30.992+0000 7f953b950640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f9534113960 con 0x7f95341049e0 2024-09-21T03:45:30.996 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:30.992+0000 7f953b950640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f9534113c60 con 0x7f953410de40 2024-09-21T03:45:30.996 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:30.992+0000 7f9538eeb640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9534105330 0x7f953419aad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:30.996 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:30.992+0000 7f9538eeb640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9534105330 0x7f953419aad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:40506/0 (socket says 172.21.15.39:40506) 2024-09-21T03:45:30.997 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:30.992+0000 7f9538eeb640 1 -- 172.21.15.39:0/2195682394 learned_addr learned my addr 172.21.15.39:0/2195682394 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:45:30.997 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:30.992+0000 7f95396ec640 1 --2- 172.21.15.39:0/2195682394 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f95341049e0 0x7f953419a530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:30.997 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:30.992+0000 7f9539eed640 1 --2- 172.21.15.39:0/2195682394 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f953410de40 0x7f953419fd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:30.997 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:30.993+0000 7f9538eeb640 1 -- 172.21.15.39:0/2195682394 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f953410de40 msgr2=0x7f953419fd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:30.997 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:30.993+0000 7f9538eeb640 1 --2- 172.21.15.39:0/2195682394 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f953410de40 0x7f953419fd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:30.997 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:30.993+0000 7f9538eeb640 1 -- 172.21.15.39:0/2195682394 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f95341049e0 msgr2=0x7f953419a530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:30.997 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:30.993+0000 7f9538eeb640 1 --2- 172.21.15.39:0/2195682394 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f95341049e0 0x7f953419a530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:30.997 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:30.993+0000 7f9538eeb640 1 -- 172.21.15.39:0/2195682394 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f95341a04a0 con 0x7f9534105330 2024-09-21T03:45:30.997 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:30.993+0000 7f9539eed640 1 --2- 172.21.15.39:0/2195682394 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f953410de40 0x7f953419fd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:45:30.997 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:30.993+0000 7f9538eeb640 1 --2- 172.21.15.39:0/2195682394 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9534105330 0x7f953419aad0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f952402a930 tx=0x7f9524004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:45:30.997 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:30.993+0000 7f952a7fc640 1 -- 172.21.15.39:0/2195682394 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f952403f030 con 0x7f9534105330 2024-09-21T03:45:30.997 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:30.993+0000 7f952a7fc640 1 -- 172.21.15.39:0/2195682394 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9524003cb0 con 0x7f9534105330 2024-09-21T03:45:30.998 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:30.993+0000 7f953b950640 1 -- 172.21.15.39:0/2195682394 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f95341a0730 con 0x7f9534105330 2024-09-21T03:45:30.998 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:30.993+0000 7f952a7fc640 1 -- 172.21.15.39:0/2195682394 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9524035070 con 0x7f9534105330 2024-09-21T03:45:30.998 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:30.993+0000 7f953b950640 1 -- 172.21.15.39:0/2195682394 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f95341a6ef0 con 0x7f9534105330 2024-09-21T03:45:30.999 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:30.994+0000 7f953b950640 1 -- 172.21.15.39:0/2195682394 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f953406c280 con 0x7f9534105330 2024-09-21T03:45:30.999 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:30.996+0000 7f952a7fc640 1 -- 172.21.15.39:0/2195682394 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 22) ==== 154580+0+0 (secure 0 0 0) 0x7f9524042030 con 0x7f9534105330 2024-09-21T03:45:30.999 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:30.996+0000 7f952a7fc640 1 --2- 172.21.15.39:0/2195682394 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f95200b6670 0x7f95200b8b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:30.999 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:30.996+0000 7f95396ec640 1 --2- 172.21.15.39:0/2195682394 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f95200b6670 0x7f95200b8b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:30.999 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:30.997+0000 7f952a7fc640 1 -- 172.21.15.39:0/2195682394 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8484+0+0 (secure 0 0 0) 0x7f9524107f80 con 0x7f9534105330 2024-09-21T03:45:30.999 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:30.997+0000 7f95396ec640 1 --2- 172.21.15.39:0/2195682394 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f95200b6670 0x7f95200b8b80 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f953406c980 tx=0x7f953000a2d0 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:45:31.001 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:31.000+0000 7f952a7fc640 1 -- 172.21.15.39:0/2195682394 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f95240ce5b0 con 0x7f9534105330 2024-09-21T03:45:31.218 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:45:30 smithi161 ceph-mon[34533]: pgmap v309: 97 pgs: 97 active+clean; 647 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-09-21T03:45:31.224 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:45:30 smithi150 ceph-mon[34289]: pgmap v309: 97 pgs: 97 active+clean; 647 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-09-21T03:45:31.229 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:31.227+0000 7f953b950640 1 -- 172.21.15.39:0/2195682394 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f9534104df0 con 0x7f9534105330 2024-09-21T03:45:31.230 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:31.228+0000 7f952a7fc640 1 -- 172.21.15.39:0/2195682394 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11978 (secure 0 0 0) 0x7f9534104df0 con 0x7f9534105330 2024-09-21T03:45:31.230 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T03:45:31.230 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":28,"btime":"2024-09-21T03:44:26:704224+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14790,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6835/508995295","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":508995295},{"type":"v1","addr":"172.21.15.39:6835","nonce":508995295}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14802,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6837/3915912607","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":3915912607},{"type":"v1","addr":"172.21.15.39:6837","nonce":3915912607}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T03:44:15.009872+0000","modified":"2024-09-21T03:44:26.017327+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24695,"mds_1":24667,"mds_2":24659,"mds_3":24646,"mds_4":14868},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14856":{"gid":14856,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.39:6839/185509192","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6838","nonce":185509192},{"type":"v1","addr":"172.21.15.39:6839","nonce":185509192}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14868":{"gid":14868,"name":"a","rank":4,"incarnation":25,"state":"up:active","state_seq":7,"addr":"172.21.15.39:6841/4247190456","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6840","nonce":4247190456},{"type":"v1","addr":"172.21.15.39:6841","nonce":4247190456}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24599":{"gid":24599,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.150:6833/3114643217","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":3114643217},{"type":"v1","addr":"172.21.15.150:6833","nonce":3114643217}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24601":{"gid":24601,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.161:6833/2963990654","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":2963990654},{"type":"v1","addr":"172.21.15.161:6833","nonce":2963990654}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24632":{"gid":24632,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.150:6835/724594008","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6834","nonce":724594008},{"type":"v1","addr":"172.21.15.150:6835","nonce":724594008}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24640":{"gid":24640,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.161:6835/3747882110","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":3747882110},{"type":"v1","addr":"172.21.15.161:6835","nonce":3747882110}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24646":{"gid":24646,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":10,"addr":"172.21.15.161:6837/3451544597","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6836","nonce":3451544597},{"type":"v1","addr":"172.21.15.161:6837","nonce":3451544597}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24659":{"gid":24659,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.150:6837/4017043144","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6836","nonce":4017043144},{"type":"v1","addr":"172.21.15.150:6837","nonce":4017043144}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24667":{"gid":24667,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.161:6839/2190107270","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6838","nonce":2190107270},{"type":"v1","addr":"172.21.15.161:6839","nonce":2190107270}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24695":{"gid":24695,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.150:6839/2949386907","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6838","nonce":2949386907},{"type":"v1","addr":"172.21.15.150:6839","nonce":2949386907}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24695,"qdb_cluster":[14868,24646,24659,24667,24695]},"id":1}]} 2024-09-21T03:45:31.230 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 28 2024-09-21T03:45:31.232 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:31.230+0000 7f953b950640 1 -- 172.21.15.39:0/2195682394 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f95200b6670 msgr2=0x7f95200b8b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:31.232 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:31.230+0000 7f953b950640 1 --2- 172.21.15.39:0/2195682394 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f95200b6670 0x7f95200b8b80 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f953406c980 tx=0x7f953000a2d0 comp rx=0 tx=0).stop 2024-09-21T03:45:31.232 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:31.230+0000 7f953b950640 1 -- 172.21.15.39:0/2195682394 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9534105330 msgr2=0x7f953419aad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:31.232 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:31.230+0000 7f953b950640 1 --2- 172.21.15.39:0/2195682394 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9534105330 0x7f953419aad0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f952402a930 tx=0x7f9524004270 comp rx=0 tx=0).stop 2024-09-21T03:45:31.232 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:31.230+0000 7f953b950640 1 -- 172.21.15.39:0/2195682394 shutdown_connections 2024-09-21T03:45:31.233 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:31.230+0000 7f953b950640 1 --2- 172.21.15.39:0/2195682394 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f95200b6670 0x7f95200b8b80 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:31.233 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:31.230+0000 7f953b950640 1 --2- 172.21.15.39:0/2195682394 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f953410de40 0x7f953419fd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:31.233 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:31.230+0000 7f953b950640 1 --2- 172.21.15.39:0/2195682394 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9534105330 0x7f953419aad0 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:31.233 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:31.230+0000 7f953b950640 1 --2- 172.21.15.39:0/2195682394 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f95341049e0 0x7f953419a530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:31.233 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:31.230+0000 7f953b950640 1 -- 172.21.15.39:0/2195682394 >> 172.21.15.39:0/2195682394 conn(0x7f9534079750 msgr2=0x7f95341008e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:45:31.233 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:31.230+0000 7f953b950640 1 -- 172.21.15.39:0/2195682394 shutdown_connections 2024-09-21T03:45:31.233 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:31.231+0000 7f953b950640 1 -- 172.21.15.39:0/2195682394 wait complete. 2024-09-21T03:45:31.967 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 28, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-09-21T03:44:15.009872+0000', 'modified': '2024-09-21T03:44:26.017327+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 5, 'in': [0, 1, 2, 3, 4], 'up': {'mds_0': 24695, 'mds_1': 24667, 'mds_2': 24659, 'mds_3': 24646, 'mds_4': 14868}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_14856': {'gid': 14856, 'name': 'g', 'rank': 4, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.39:6839/185509192', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.39:6838', 'nonce': 185509192}, {'type': 'v1', 'addr': '172.21.15.39:6839', 'nonce': 185509192}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_14868': {'gid': 14868, 'name': 'a', 'rank': 4, 'incarnation': 25, 'state': 'up:active', 'state_seq': 7, 'addr': '172.21.15.39:6841/4247190456', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.39:6840', 'nonce': 4247190456}, {'type': 'v1', 'addr': '172.21.15.39:6841', 'nonce': 4247190456}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24599': {'gid': 24599, 'name': 'k', 'rank': 3, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.150:6833/3114643217', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.150:6832', 'nonce': 3114643217}, {'type': 'v1', 'addr': '172.21.15.150:6833', 'nonce': 3114643217}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24601': {'gid': 24601, 'name': 'l', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.161:6833/2963990654', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.161:6832', 'nonce': 2963990654}, {'type': 'v1', 'addr': '172.21.15.161:6833', 'nonce': 2963990654}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24632': {'gid': 24632, 'name': 'e', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.150:6835/724594008', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.150:6834', 'nonce': 724594008}, {'type': 'v1', 'addr': '172.21.15.150:6835', 'nonce': 724594008}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24640': {'gid': 24640, 'name': 'f', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.161:6835/3747882110', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.161:6834', 'nonce': 3747882110}, {'type': 'v1', 'addr': '172.21.15.161:6835', 'nonce': 3747882110}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24646': {'gid': 24646, 'name': 'c', 'rank': 3, 'incarnation': 23, 'state': 'up:active', 'state_seq': 10, 'addr': '172.21.15.161:6837/3451544597', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.161:6836', 'nonce': 3451544597}, {'type': 'v1', 'addr': '172.21.15.161:6837', 'nonce': 3451544597}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24659': {'gid': 24659, 'name': 'h', 'rank': 2, 'incarnation': 21, 'state': 'up:active', 'state_seq': 9, 'addr': '172.21.15.150:6837/4017043144', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.150:6836', 'nonce': 4017043144}, {'type': 'v1', 'addr': '172.21.15.150:6837', 'nonce': 4017043144}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24667': {'gid': 24667, 'name': 'i', 'rank': 1, 'incarnation': 19, 'state': 'up:active', 'state_seq': 8, 'addr': '172.21.15.161:6839/2190107270', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.161:6838', 'nonce': 2190107270}, {'type': 'v1', 'addr': '172.21.15.161:6839', 'nonce': 2190107270}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24695': {'gid': 24695, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.150:6839/2949386907', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.150:6838', 'nonce': 2949386907}, {'type': 'v1', 'addr': '172.21.15.150:6839', 'nonce': 2949386907}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24695, 'qdb_cluster': [14868, 24646, 24659, 24667, 24695]} 2024-09-21T03:45:31.967 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 5/5 2024-09-21T03:45:31.968 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 51116d5c-77ca-11ef-baf5-efdc52797490 -- ceph tell mds.a status 2024-09-21T03:45:32.218 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:45:31 smithi161 ceph-mon[34533]: from='client.? 172.21.15.39:0/2195682394' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:45:32.223 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/51116d5c-77ca-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:45:32.225 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:45:31 smithi150 ceph-mon[34289]: from='client.? 172.21.15.39:0/2195682394' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:45:32.333 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:45:31 smithi039 ceph-mon[31211]: from='client.? 172.21.15.39:0/2195682394' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:45:32.701 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:32.700+0000 7f71cd66a640 1 -- 172.21.15.39:0/3149708379 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f71c8102ae0 msgr2=0x7f71c8102ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:32.702 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:32.700+0000 7f71cd66a640 1 --2- 172.21.15.39:0/3149708379 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f71c8102ae0 0x7f71c8102ef0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f71b8009900 tx=0x7f71b802a410 comp rx=0 tx=0).stop 2024-09-21T03:45:32.702 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:32.700+0000 7f71cd66a640 1 -- 172.21.15.39:0/3149708379 shutdown_connections 2024-09-21T03:45:32.702 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:32.700+0000 7f71cd66a640 1 --2- 172.21.15.39:0/3149708379 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f71c8108d20 0x7f71c810f550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:32.702 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:32.700+0000 7f71cd66a640 1 --2- 172.21.15.39:0/3149708379 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f71c81083d0 0x7f71c81087e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:32.702 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:32.700+0000 7f71cd66a640 1 --2- 172.21.15.39:0/3149708379 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f71c8102ae0 0x7f71c8102ef0 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:32.702 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:32.700+0000 7f71cd66a640 1 -- 172.21.15.39:0/3149708379 >> 172.21.15.39:0/3149708379 conn(0x7f71c8079790 msgr2=0x7f71c8079ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:45:32.702 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:32.700+0000 7f71cd66a640 1 -- 172.21.15.39:0/3149708379 shutdown_connections 2024-09-21T03:45:32.702 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:32.700+0000 7f71cd66a640 1 -- 172.21.15.39:0/3149708379 wait complete. 2024-09-21T03:45:32.704 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:32.700+0000 7f71cd66a640 1 Processor -- start 2024-09-21T03:45:32.704 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:32.701+0000 7f71cd66a640 1 -- start start 2024-09-21T03:45:32.704 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:32.701+0000 7f71cd66a640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f71c8102ae0 0x7f71c819c690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:32.704 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:32.701+0000 7f71cd66a640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f71c81083d0 0x7f71c819cc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:32.704 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:32.701+0000 7f71cd66a640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f71c8108d20 0x7f71c81a1ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:32.704 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:32.701+0000 7f71cd66a640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f71c8113b00 con 0x7f71c8102ae0 2024-09-21T03:45:32.705 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:32.701+0000 7f71cd66a640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f71c8113980 con 0x7f71c8108d20 2024-09-21T03:45:32.705 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:32.701+0000 7f71cd66a640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f71c8113c80 con 0x7f71c81083d0 2024-09-21T03:45:32.705 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:32.701+0000 7f71c6ffd640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f71c8102ae0 0x7f71c819c690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:32.705 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:32.701+0000 7f71c6ffd640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f71c8102ae0 0x7f71c819c690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:40538/0 (socket says 172.21.15.39:40538) 2024-09-21T03:45:32.705 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:32.701+0000 7f71c6ffd640 1 -- 172.21.15.39:0/1051941114 learned_addr learned my addr 172.21.15.39:0/1051941114 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:45:32.705 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:32.701+0000 7f71c77fe640 1 --2- 172.21.15.39:0/1051941114 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f71c8108d20 0x7f71c81a1ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:32.705 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:32.701+0000 7f71be7fc640 1 --2- 172.21.15.39:0/1051941114 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f71c81083d0 0x7f71c819cc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:32.705 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:32.701+0000 7f71c6ffd640 1 -- 172.21.15.39:0/1051941114 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f71c81083d0 msgr2=0x7f71c819cc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:32.705 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:32.701+0000 7f71c6ffd640 1 --2- 172.21.15.39:0/1051941114 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f71c81083d0 0x7f71c819cc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:32.705 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:32.701+0000 7f71c6ffd640 1 -- 172.21.15.39:0/1051941114 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f71c8108d20 msgr2=0x7f71c81a1ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:32.705 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:32.701+0000 7f71c6ffd640 1 --2- 172.21.15.39:0/1051941114 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f71c8108d20 0x7f71c81a1ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:32.705 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:32.701+0000 7f71c6ffd640 1 -- 172.21.15.39:0/1051941114 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f71c81a2600 con 0x7f71c8102ae0 2024-09-21T03:45:32.705 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:32.701+0000 7f71c77fe640 1 --2- 172.21.15.39:0/1051941114 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f71c8108d20 0x7f71c81a1ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:45:32.706 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:32.702+0000 7f71c6ffd640 1 --2- 172.21.15.39:0/1051941114 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f71c8102ae0 0x7f71c819c690 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f71b8005940 tx=0x7f71b80042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:45:32.706 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:32.702+0000 7f71c4ff9640 1 -- 172.21.15.39:0/1051941114 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71b803f030 con 0x7f71c8102ae0 2024-09-21T03:45:32.706 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:32.702+0000 7f71c4ff9640 1 -- 172.21.15.39:0/1051941114 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f71b8003cb0 con 0x7f71c8102ae0 2024-09-21T03:45:32.706 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:32.702+0000 7f71cd66a640 1 -- 172.21.15.39:0/1051941114 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f71c81a2890 con 0x7f71c8102ae0 2024-09-21T03:45:32.706 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:32.702+0000 7f71c4ff9640 1 -- 172.21.15.39:0/1051941114 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71b8035070 con 0x7f71c8102ae0 2024-09-21T03:45:32.706 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:32.702+0000 7f71cd66a640 1 -- 172.21.15.39:0/1051941114 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f71c81a9050 con 0x7f71c8102ae0 2024-09-21T03:45:32.708 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:32.704+0000 7f71c4ff9640 1 -- 172.21.15.39:0/1051941114 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 22) ==== 154580+0+0 (secure 0 0 0) 0x7f71b8042030 con 0x7f71c8102ae0 2024-09-21T03:45:32.708 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:32.705+0000 7f71d3bde740 1 Processor -- start 2024-09-21T03:45:32.708 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:32.705+0000 7f71d3bde740 1 -- start start 2024-09-21T03:45:32.708 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:32.705+0000 7f71d3bde740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55fb989aaf60 0x55fb98d50f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:32.708 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:32.705+0000 7f71d3bde740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55fb989820a0 0x55fb98d51440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:32.708 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:32.705+0000 7f71d3bde740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55fb9898a280 0x55fb98d51980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:32.709 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:32.705+0000 7f71d3bde740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x55fb98cc8c10 con 0x55fb989820a0 2024-09-21T03:45:32.709 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:32.705+0000 7f71d3bde740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x55fb98cb8040 con 0x55fb989aaf60 2024-09-21T03:45:32.709 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:32.705+0000 7f71d3bde740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55fb98cbbe50 con 0x55fb9898a280 2024-09-21T03:45:32.709 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:32.705+0000 7f71c6ffd640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55fb989820a0 0x55fb98d51440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:32.709 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:32.705+0000 7f71c77fe640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55fb989aaf60 0x55fb98d50f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:32.709 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:32.705+0000 7f71c6ffd640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55fb989820a0 0x55fb98d51440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:40552/0 (socket says 172.21.15.39:40552) 2024-09-21T03:45:32.709 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:32.705+0000 7f71c6ffd640 1 -- 172.21.15.39:0/2754314000 learned_addr learned my addr 172.21.15.39:0/2754314000 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:45:32.709 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:32.705+0000 7f71be7fc640 1 --2- 172.21.15.39:0/2754314000 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55fb9898a280 0x55fb98d51980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:32.709 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:32.706+0000 7f71c6ffd640 1 -- 172.21.15.39:0/2754314000 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55fb9898a280 msgr2=0x55fb98d51980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:45:32.709 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:32.706+0000 7f71c6ffd640 1 --2- 172.21.15.39:0/2754314000 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55fb9898a280 0x55fb98d51980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:32.709 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:32.706+0000 7f71c6ffd640 1 -- 172.21.15.39:0/2754314000 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55fb989aaf60 msgr2=0x55fb98d50f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:45:32.709 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:32.706+0000 7f71c6ffd640 1 --2- 172.21.15.39:0/2754314000 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55fb989aaf60 0x55fb98d50f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:32.709 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:32.706+0000 7f71c6ffd640 1 -- 172.21.15.39:0/2754314000 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fb98a16540 con 0x55fb989820a0 2024-09-21T03:45:32.710 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:32.706+0000 7f71c77fe640 1 --2- 172.21.15.39:0/2754314000 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55fb989aaf60 0x55fb98d50f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:45:32.710 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:32.706+0000 7f71c6ffd640 1 --2- 172.21.15.39:0/2754314000 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55fb989820a0 0x55fb98d51440 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f71b8105440 tx=0x7f71b810bf20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b441ecbed736f24d server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:45:32.710 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:32.706+0000 7f71c4ff9640 1 --2- 172.21.15.39:0/1051941114 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f71b40b65f0 0x7f71b40b8b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:32.710 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:32.706+0000 7f71c77fe640 1 --2- 172.21.15.39:0/1051941114 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f71b40b65f0 0x7f71b40b8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:32.710 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:32.706+0000 7f71bd7fa640 1 -- 172.21.15.39:0/2754314000 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71b80ca1d0 con 0x55fb989820a0 2024-09-21T03:45:32.710 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:32.706+0000 7f71bd7fa640 1 -- 172.21.15.39:0/2754314000 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f71b8115070 con 0x55fb989820a0 2024-09-21T03:45:32.710 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:32.706+0000 7f71bd7fa640 1 -- 172.21.15.39:0/2754314000 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71b80cbd30 con 0x55fb989820a0 2024-09-21T03:45:32.710 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:32.706+0000 7f71c4ff9640 1 -- 172.21.15.39:0/1051941114 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8484+0+0 (secure 0 0 0) 0x7f71b8107c50 con 0x7f71c8102ae0 2024-09-21T03:45:32.714 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:32.707+0000 7f71d3bde740 1 -- 172.21.15.39:0/2754314000 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55fb989820a0 msgr2=0x55fb98d51440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:32.714 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:32.707+0000 7f71c77fe640 1 --2- 172.21.15.39:0/1051941114 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f71b40b65f0 0x7f71b40b8b00 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f71c0005fd0 tx=0x7f71c0009800 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:45:32.714 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:32.707+0000 7f71d3bde740 1 --2- 172.21.15.39:0/2754314000 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55fb989820a0 0x55fb98d51440 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f71b8105440 tx=0x7f71b810bf20 comp rx=0 tx=0).stop 2024-09-21T03:45:32.714 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:32.707+0000 7f71d3bde740 1 -- 172.21.15.39:0/2754314000 shutdown_connections 2024-09-21T03:45:32.714 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:32.707+0000 7f71d3bde740 1 --2- 172.21.15.39:0/2754314000 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55fb9898a280 0x55fb98d51980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:32.714 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:32.707+0000 7f71d3bde740 1 --2- 172.21.15.39:0/2754314000 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55fb989820a0 0x55fb98d51440 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:32.714 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:32.707+0000 7f71d3bde740 1 --2- 172.21.15.39:0/2754314000 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55fb989aaf60 0x55fb98d50f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:32.714 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:32.707+0000 7f71d3bde740 1 -- 172.21.15.39:0/2754314000 >> 172.21.15.39:0/2754314000 conn(0x55fb9899f350 msgr2=0x55fb98d4edf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:45:32.715 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:32.707+0000 7f71d3bde740 1 -- 172.21.15.39:0/2754314000 shutdown_connections 2024-09-21T03:45:32.715 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:32.707+0000 7f71d3bde740 1 -- 172.21.15.39:0/2754314000 wait complete. 2024-09-21T03:45:32.715 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:32.708+0000 7f71d3bde740 1 -- start start 2024-09-21T03:45:32.715 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:32.708+0000 7f71d3bde740 1 Processor -- start 2024-09-21T03:45:32.715 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:32.708+0000 7f71d3bde740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55fb989820a0 0x55fb98d5ca10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:32.715 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:32.708+0000 7f71d3bde740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55fb9898a280 0x55fb98d5cf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:32.715 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:32.708+0000 7f71d3bde740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55fb989aaf60 0x55fb98d68700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:32.715 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:32.708+0000 7f71d3bde740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x55fb98b195f0 con 0x55fb989aaf60 2024-09-21T03:45:32.715 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:32.708+0000 7f71d3bde740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x55fb98c53010 con 0x55fb989820a0 2024-09-21T03:45:32.715 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:32.708+0000 7f71d3bde740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55fb98d5f4a0 con 0x55fb9898a280 2024-09-21T03:45:32.715 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:32.708+0000 7f71d3bde740 10 client.0 _ll_register_callbacks cb 0x55fb98cb2340 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:45:32.715 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:32.709+0000 7f71c77fe640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55fb989aaf60 0x55fb98d68700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:32.715 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:32.709+0000 7f71c77fe640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55fb989aaf60 0x55fb98d68700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:40568/0 (socket says 172.21.15.39:40568) 2024-09-21T03:45:32.716 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:32.709+0000 7f71c77fe640 1 -- 172.21.15.39:0/34984217 learned_addr learned my addr 172.21.15.39:0/34984217 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:45:32.716 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:32.709+0000 7f71be7fc640 1 --2- 172.21.15.39:0/34984217 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55fb9898a280 0x55fb98d5cf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:32.716 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:32.709+0000 7f71c6ffd640 1 --2- 172.21.15.39:0/34984217 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55fb989820a0 0x55fb98d5ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:32.716 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:32.709+0000 7f71c77fe640 1 -- 172.21.15.39:0/34984217 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55fb9898a280 msgr2=0x55fb98d5cf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:45:32.716 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:32.709+0000 7f71c77fe640 1 --2- 172.21.15.39:0/34984217 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55fb9898a280 0x55fb98d5cf50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:32.716 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:32.709+0000 7f71c77fe640 1 -- 172.21.15.39:0/34984217 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55fb989820a0 msgr2=0x55fb98d5ca10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:45:32.716 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:32.709+0000 7f71c77fe640 1 --2- 172.21.15.39:0/34984217 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55fb989820a0 0x55fb98d5ca10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:32.716 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:32.709+0000 7f71c77fe640 1 -- 172.21.15.39:0/34984217 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fb98c80a00 con 0x55fb989aaf60 2024-09-21T03:45:32.716 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:32.709+0000 7f71c6ffd640 1 --2- 172.21.15.39:0/34984217 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55fb989820a0 0x55fb98d5ca10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:45:32.716 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:32.709+0000 7f71c77fe640 1 --2- 172.21.15.39:0/34984217 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55fb989aaf60 0x55fb98d68700 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f71c0011ea0 tx=0x7f71c0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b128c01f101a28df server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:45:32.716 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:32.709+0000 7f71ae7fc640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-09-21T03:45:32.716 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:32.709+0000 7f71ae7fc640 1 -- 172.21.15.39:0/34984217 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71c001ed30 con 0x55fb989aaf60 2024-09-21T03:45:32.716 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:32.709+0000 7f71ae7fc640 1 -- 172.21.15.39:0/34984217 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8484+0+0 (secure 0 0 0) 0x7f71c002a480 con 0x55fb989aaf60 2024-09-21T03:45:32.716 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:32.709+0000 7f71d3bde740 1 -- 172.21.15.39:0/34984217 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f71b8115070 con 0x55fb989aaf60 2024-09-21T03:45:32.716 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:32.710+0000 7f71ae7fc640 1 -- 172.21.15.39:0/34984217 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f71c002a680 con 0x55fb989aaf60 2024-09-21T03:45:32.717 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:32.710+0000 7f71d3bde740 10 client.15234 fetch_fsmap learned FSMap version 28 2024-09-21T03:45:32.717 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:32.710+0000 7f71d3bde740 1 -- 172.21.15.39:0/34984217 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55fb98d6b370 con 0x55fb989aaf60 2024-09-21T03:45:32.717 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:32.710+0000 7f71ae7fc640 1 -- 172.21.15.39:0/34984217 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f71b8115070 con 0x55fb989aaf60 2024-09-21T03:45:32.717 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:32.710+0000 7f71ae7fc640 1 -- 172.21.15.39:0/34984217 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f71c000f040 con 0x55fb989aaf60 2024-09-21T03:45:32.717 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:32.710+0000 7f71d3bde740 10 client.15234 fetch_fsmap finished waiting for FSMap version 28 2024-09-21T03:45:32.717 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:32.710+0000 7f71d3bde740 10 client.15234 resolve_mds: resolved name 'a' to daemon mds.a 2024-09-21T03:45:32.717 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:32.710+0000 7f71d3bde740 20 client.15234 populate_metadata read hostname 'smithi039' 2024-09-21T03:45:32.717 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:32.710+0000 7f71d3bde740 1 --2- 172.21.15.39:0/34984217 >> [v2:172.21.15.39:6840/4247190456,v1:172.21.15.39:6841/4247190456] conn(0x55fb989698e0 0x55fb98d6ee00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:32.717 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:32.710+0000 7f71d3bde740 4 client.15234 mds_command: new command op to 14868 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:45:32.717 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:32.710+0000 7f71d3bde740 1 -- 172.21.15.39:0/34984217 --> [v2:172.21.15.39:6840/4247190456,v1:172.21.15.39:6841/4247190456] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fb98ccab60 con 0x55fb989698e0 2024-09-21T03:45:32.717 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:32.710+0000 7f71c6ffd640 1 --2- 172.21.15.39:0/34984217 >> [v2:172.21.15.39:6840/4247190456,v1:172.21.15.39:6841/4247190456] conn(0x55fb989698e0 0x55fb98d6ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:32.717 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:32.711+0000 7f71ae7fc640 10 client.15234 ms_handle_connect on v2:172.21.15.39:6840/4247190456 2024-09-21T03:45:32.717 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:32.711+0000 7f71c6ffd640 1 --2- 172.21.15.39:0/34984217 >> [v2:172.21.15.39:6840/4247190456,v1:172.21.15.39:6841/4247190456] conn(0x55fb989698e0 0x55fb98d6ee00 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=7e8dd08526b46126 server_cookie=b6924985a1eb598d in_seq=0 out_seq=0 2024-09-21T03:45:32.717 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:32.712+0000 7f71ae7fc640 1 -- 172.21.15.39:0/34984217 <== mds.4 v2:172.21.15.39:6840/4247190456 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55fb98ccab60 con 0x55fb989698e0 2024-09-21T03:45:32.718 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:32.712+0000 7f71ae7fc640 10 client.15234 handle_command_reply: tid=0 2024-09-21T03:45:32.718 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:32.712+0000 7f71d3bde740 1 client.15234 shutdown 2024-09-21T03:45:32.718 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:32.712+0000 7f71d3bde740 1 -- 172.21.15.39:0/34984217 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55fb989aaf60 msgr2=0x55fb98d68700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:32.718 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:32.712+0000 7f71d3bde740 1 --2- 172.21.15.39:0/34984217 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55fb989aaf60 0x55fb98d68700 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f71c0011ea0 tx=0x7f71c0004a40 comp rx=0 tx=0).stop 2024-09-21T03:45:32.718 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:32.712+0000 7f71d3bde740 1 -- 172.21.15.39:0/34984217 shutdown_connections 2024-09-21T03:45:32.718 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:32.712+0000 7f71d3bde740 1 --2- 172.21.15.39:0/34984217 >> [v2:172.21.15.39:6840/4247190456,v1:172.21.15.39:6841/4247190456] conn(0x55fb989698e0 0x55fb98d6ee00 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:32.718 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:32.712+0000 7f71d3bde740 1 --2- 172.21.15.39:0/34984217 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55fb989aaf60 0x55fb98d68700 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:32.718 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:32.712+0000 7f71d3bde740 1 --2- 172.21.15.39:0/34984217 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55fb9898a280 0x55fb98d5cf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:32.718 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:32.712+0000 7f71d3bde740 1 --2- 172.21.15.39:0/34984217 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55fb989820a0 0x55fb98d5ca10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:32.718 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:32.712+0000 7f71d3bde740 1 -- 172.21.15.39:0/34984217 >> 172.21.15.39:0/34984217 conn(0x55fb9899f350 msgr2=0x55fb98d51f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:45:32.718 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:32.712+0000 7f71ae7fc640 0 client.15234 ms_handle_reset on v2:172.21.15.39:6840/4247190456 2024-09-21T03:45:32.718 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:32.712+0000 7f71d3bde740 1 -- 172.21.15.39:0/34984217 shutdown_connections 2024-09-21T03:45:32.718 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:32.712+0000 7f71d3bde740 1 -- 172.21.15.39:0/34984217 wait complete. 2024-09-21T03:45:32.718 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:32.712+0000 7f71d3bde740 20 client.15234 trim_cache size 0 max 16384 2024-09-21T03:45:32.732 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:32.730+0000 7f71d3bde740 1 Processor -- start 2024-09-21T03:45:32.732 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:32.731+0000 7f71d3bde740 1 -- start start 2024-09-21T03:45:32.733 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:32.731+0000 7f71d3bde740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55fb989820a0 0x55fb98d4e730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:32.733 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:32.731+0000 7f71d3bde740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55fb9898a280 0x55fb98d4ec70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:32.734 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:32.731+0000 7f71d3bde740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55fb989aaf60 0x55fb98d579c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:32.734 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:32.731+0000 7f71d3bde740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x55fb98d6d140 con 0x55fb989aaf60 2024-09-21T03:45:32.734 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:32.731+0000 7f71d3bde740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x55fb98d6d810 con 0x55fb989820a0 2024-09-21T03:45:32.734 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:32.731+0000 7f71d3bde740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55fb98d6dee0 con 0x55fb9898a280 2024-09-21T03:45:32.734 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:32.731+0000 7f71c77fe640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55fb989aaf60 0x55fb98d579c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:32.734 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:32.731+0000 7f71c77fe640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55fb989aaf60 0x55fb98d579c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:40576/0 (socket says 172.21.15.39:40576) 2024-09-21T03:45:32.734 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:32.731+0000 7f71c77fe640 1 -- 172.21.15.39:0/2199869429 learned_addr learned my addr 172.21.15.39:0/2199869429 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:45:32.734 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:32.731+0000 7f71be7fc640 1 --2- 172.21.15.39:0/2199869429 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55fb9898a280 0x55fb98d4ec70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:32.734 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:32.731+0000 7f71c6ffd640 1 --2- 172.21.15.39:0/2199869429 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55fb989820a0 0x55fb98d4e730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:32.734 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:32.731+0000 7f71c77fe640 1 -- 172.21.15.39:0/2199869429 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55fb9898a280 msgr2=0x55fb98d4ec70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:45:32.734 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:32.731+0000 7f71c77fe640 1 --2- 172.21.15.39:0/2199869429 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55fb9898a280 0x55fb98d4ec70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:32.734 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:32.731+0000 7f71c77fe640 1 -- 172.21.15.39:0/2199869429 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55fb989820a0 msgr2=0x55fb98d4e730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:45:32.734 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:32.731+0000 7f71c77fe640 1 --2- 172.21.15.39:0/2199869429 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55fb989820a0 0x55fb98d4e730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:32.735 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:32.731+0000 7f71c77fe640 1 -- 172.21.15.39:0/2199869429 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fb98d5dad0 con 0x55fb989aaf60 2024-09-21T03:45:32.735 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:32.731+0000 7f71be7fc640 1 --2- 172.21.15.39:0/2199869429 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55fb9898a280 0x55fb98d4ec70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:45:32.735 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:32.732+0000 7f71c77fe640 1 --2- 172.21.15.39:0/2199869429 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55fb989aaf60 0x55fb98d579c0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f71c0000f80 tx=0x7f71c0011a80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1710501683b30350 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:45:32.735 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:32.732+0000 7f71bcff9640 1 -- 172.21.15.39:0/2199869429 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71c001eb30 con 0x55fb989aaf60 2024-09-21T03:45:32.735 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:32.732+0000 7f71bcff9640 1 -- 172.21.15.39:0/2199869429 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f71c0038070 con 0x55fb989aaf60 2024-09-21T03:45:32.735 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:32.732+0000 7f71bcff9640 1 -- 172.21.15.39:0/2199869429 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71c001f6a0 con 0x55fb989aaf60 2024-09-21T03:45:32.738 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:32.733+0000 7f71d3bde740 1 -- 172.21.15.39:0/2199869429 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55fb989aaf60 msgr2=0x55fb98d579c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:32.739 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:32.733+0000 7f71d3bde740 1 --2- 172.21.15.39:0/2199869429 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55fb989aaf60 0x55fb98d579c0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f71c0000f80 tx=0x7f71c0011a80 comp rx=0 tx=0).stop 2024-09-21T03:45:32.739 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:32.733+0000 7f71d3bde740 1 -- 172.21.15.39:0/2199869429 shutdown_connections 2024-09-21T03:45:32.739 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:32.733+0000 7f71d3bde740 1 --2- 172.21.15.39:0/2199869429 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55fb989aaf60 0x55fb98d579c0 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:32.739 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:32.733+0000 7f71d3bde740 1 --2- 172.21.15.39:0/2199869429 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55fb9898a280 0x55fb98d4ec70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:32.739 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:32.733+0000 7f71d3bde740 1 --2- 172.21.15.39:0/2199869429 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55fb989820a0 0x55fb98d4e730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:32.739 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:32.733+0000 7f71d3bde740 1 -- 172.21.15.39:0/2199869429 >> 172.21.15.39:0/2199869429 conn(0x55fb9899f350 msgr2=0x55fb98d5c9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:45:32.739 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:32.733+0000 7f71d3bde740 1 -- 172.21.15.39:0/2199869429 shutdown_connections 2024-09-21T03:45:32.739 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:32.733+0000 7f71d3bde740 1 -- 172.21.15.39:0/2199869429 wait complete. 2024-09-21T03:45:32.739 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:32.733+0000 7f71d3bde740 1 -- start start 2024-09-21T03:45:32.739 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:32.733+0000 7f71d3bde740 1 Processor -- start 2024-09-21T03:45:32.739 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:32.734+0000 7f71d3bde740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55fb989820a0 0x55fb98d707f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:32.739 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:32.734+0000 7f71d3bde740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55fb9898a280 0x55fb98d70db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:32.740 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:32.734+0000 7f71d3bde740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55fb989aaf60 0x55fb98d75df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:32.740 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:32.734+0000 7f71d3bde740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x55fb98d6ec80 con 0x55fb989820a0 2024-09-21T03:45:32.740 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:32.734+0000 7f71d3bde740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x55fb98d6e5b0 con 0x55fb9898a280 2024-09-21T03:45:32.740 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:32.734+0000 7f71d3bde740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55fb98d6f350 con 0x55fb989aaf60 2024-09-21T03:45:32.740 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:32.734+0000 7f71d3bde740 10 client.0 _ll_register_callbacks cb 0x7f71c0009a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:45:32.740 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:32.734+0000 7f71c6ffd640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55fb989820a0 0x55fb98d707f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:32.740 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:32.734+0000 7f71c6ffd640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55fb989820a0 0x55fb98d707f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:40590/0 (socket says 172.21.15.39:40590) 2024-09-21T03:45:32.740 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:32.734+0000 7f71c6ffd640 1 -- 172.21.15.39:0/2952554287 learned_addr learned my addr 172.21.15.39:0/2952554287 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:45:32.740 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:32.734+0000 7f71c77fe640 1 --2- 172.21.15.39:0/2952554287 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55fb989aaf60 0x55fb98d75df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:32.740 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:32.734+0000 7f71be7fc640 1 --2- 172.21.15.39:0/2952554287 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55fb9898a280 0x55fb98d70db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:32.740 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:32.734+0000 7f71c6ffd640 1 -- 172.21.15.39:0/2952554287 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55fb989aaf60 msgr2=0x55fb98d75df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:45:32.740 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:32.734+0000 7f71c6ffd640 1 --2- 172.21.15.39:0/2952554287 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55fb989aaf60 0x55fb98d75df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:32.740 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:32.734+0000 7f71c6ffd640 1 -- 172.21.15.39:0/2952554287 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55fb9898a280 msgr2=0x55fb98d70db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:45:32.740 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:32.734+0000 7f71c6ffd640 1 --2- 172.21.15.39:0/2952554287 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55fb9898a280 0x55fb98d70db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:32.741 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:32.734+0000 7f71c6ffd640 1 -- 172.21.15.39:0/2952554287 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fb98d76460 con 0x55fb989820a0 2024-09-21T03:45:32.741 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:32.734+0000 7f71c6ffd640 1 --2- 172.21.15.39:0/2952554287 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55fb989820a0 0x55fb98d707f0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f71b80cd880 tx=0x7f71b802a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c14486cceeccf358 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:45:32.741 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:32.734+0000 7f71ae7fc640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-09-21T03:45:32.741 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:32.735+0000 7f71ae7fc640 1 -- 172.21.15.39:0/2952554287 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71b803ce60 con 0x55fb989820a0 2024-09-21T03:45:32.741 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:32.735+0000 7f71ae7fc640 1 -- 172.21.15.39:0/2952554287 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8484+0+0 (secure 0 0 0) 0x7f71b8105750 con 0x55fb989820a0 2024-09-21T03:45:32.741 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:32.735+0000 7f71d3bde740 1 -- 172.21.15.39:0/2952554287 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f71c0038070 con 0x55fb989820a0 2024-09-21T03:45:32.741 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:32.735+0000 7f71ae7fc640 1 -- 172.21.15.39:0/2952554287 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f71b8049030 con 0x55fb989820a0 2024-09-21T03:45:32.741 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:32.735+0000 7f71d3bde740 10 client.15240 fetch_fsmap learned FSMap version 28 2024-09-21T03:45:32.741 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:32.735+0000 7f71d3bde740 1 -- 172.21.15.39:0/2952554287 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55fb98d77790 con 0x55fb989820a0 2024-09-21T03:45:32.741 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:32.735+0000 7f71ae7fc640 1 -- 172.21.15.39:0/2952554287 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f71b810b070 con 0x55fb989820a0 2024-09-21T03:45:32.741 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:32.735+0000 7f71ae7fc640 1 -- 172.21.15.39:0/2952554287 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f71b80f10b0 con 0x55fb989820a0 2024-09-21T03:45:32.741 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:32.735+0000 7f71d3bde740 10 client.15240 fetch_fsmap finished waiting for FSMap version 28 2024-09-21T03:45:32.741 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:32.735+0000 7f71d3bde740 10 client.15240 resolve_mds: resolved name 'a' to daemon mds.a 2024-09-21T03:45:32.741 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:32.735+0000 7f71d3bde740 20 client.15240 populate_metadata read hostname 'smithi039' 2024-09-21T03:45:32.742 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:32.735+0000 7f71d3bde740 1 --2- 172.21.15.39:0/2952554287 >> [v2:172.21.15.39:6840/4247190456,v1:172.21.15.39:6841/4247190456] conn(0x55fb9893ec90 0x55fb98ddd100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:32.742 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:32.735+0000 7f71d3bde740 4 client.15240 mds_command: new command op to 14868 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-21T03:45:32.742 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:32.735+0000 7f71d3bde740 1 -- 172.21.15.39:0/2952554287 --> [v2:172.21.15.39:6840/4247190456,v1:172.21.15.39:6841/4247190456] -- command(tid 0: {"prefix": "status"}) -- 0x55fb98cc6610 con 0x55fb9893ec90 2024-09-21T03:45:32.742 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:32.736+0000 7f71be7fc640 1 --2- 172.21.15.39:0/2952554287 >> [v2:172.21.15.39:6840/4247190456,v1:172.21.15.39:6841/4247190456] conn(0x55fb9893ec90 0x55fb98ddd100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:32.742 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:32.736+0000 7f71ae7fc640 10 client.15240 ms_handle_connect on v2:172.21.15.39:6840/4247190456 2024-09-21T03:45:32.742 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:32.736+0000 7f71be7fc640 1 --2- 172.21.15.39:0/2952554287 >> [v2:172.21.15.39:6840/4247190456,v1:172.21.15.39:6841/4247190456] conn(0x55fb9893ec90 0x55fb98ddd100 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=da1741bd71606979 server_cookie=f6a6ea5d363166ab in_seq=0 out_seq=0 2024-09-21T03:45:32.742 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:32.736+0000 7f71ae7fc640 1 -- 172.21.15.39:0/2952554287 <== mds.4 v2:172.21.15.39:6840/4247190456 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x55fb98cc6610 con 0x55fb9893ec90 2024-09-21T03:45:32.742 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:32.736+0000 7f71ae7fc640 10 client.15240 handle_command_reply: tid=0 2024-09-21T03:45:32.742 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:32.736+0000 7f71d3bde740 1 client.15240 shutdown 2024-09-21T03:45:32.742 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:32.736+0000 7f71d3bde740 1 -- 172.21.15.39:0/2952554287 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55fb989820a0 msgr2=0x55fb98d707f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:32.742 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:32.736+0000 7f71d3bde740 1 --2- 172.21.15.39:0/2952554287 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55fb989820a0 0x55fb98d707f0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f71b80cd880 tx=0x7f71b802a410 comp rx=0 tx=0).stop 2024-09-21T03:45:32.742 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:32.736+0000 7f71d3bde740 1 -- 172.21.15.39:0/2952554287 shutdown_connections 2024-09-21T03:45:32.742 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:32.736+0000 7f71d3bde740 1 --2- 172.21.15.39:0/2952554287 >> [v2:172.21.15.39:6840/4247190456,v1:172.21.15.39:6841/4247190456] conn(0x55fb9893ec90 0x55fb98ddd100 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:32.743 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:32.736+0000 7f71d3bde740 1 --2- 172.21.15.39:0/2952554287 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55fb989aaf60 0x55fb98d75df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:32.743 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:32.736+0000 7f71d3bde740 1 --2- 172.21.15.39:0/2952554287 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55fb9898a280 0x55fb98d70db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:32.743 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:32.736+0000 7f71d3bde740 1 --2- 172.21.15.39:0/2952554287 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55fb989820a0 0x55fb98d707f0 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:32.743 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:32.736+0000 7f71d3bde740 1 -- 172.21.15.39:0/2952554287 >> 172.21.15.39:0/2952554287 conn(0x55fb9899f350 msgr2=0x55fb98d579c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:45:32.743 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:32.736+0000 7f71ae7fc640 0 client.15240 ms_handle_reset on v2:172.21.15.39:6840/4247190456 2024-09-21T03:45:32.743 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:32.737+0000 7f71d3bde740 1 -- 172.21.15.39:0/2952554287 shutdown_connections 2024-09-21T03:45:32.743 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:32.737+0000 7f71d3bde740 1 -- 172.21.15.39:0/2952554287 wait complete. 2024-09-21T03:45:32.743 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:32.737+0000 7f71d3bde740 20 client.15240 trim_cache size 0 max 16384 2024-09-21T03:45:32.743 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-09-21T03:45:32.743 INFO:teuthology.orchestra.run.smithi039.stdout: "cluster_fsid": "51116d5c-77ca-11ef-baf5-efdc52797490", 2024-09-21T03:45:32.744 INFO:teuthology.orchestra.run.smithi039.stdout: "whoami": 4, 2024-09-21T03:45:32.744 INFO:teuthology.orchestra.run.smithi039.stdout: "id": 14868, 2024-09-21T03:45:32.744 INFO:teuthology.orchestra.run.smithi039.stdout: "want_state": "up:active", 2024-09-21T03:45:32.744 INFO:teuthology.orchestra.run.smithi039.stdout: "state": "up:active", 2024-09-21T03:45:32.744 INFO:teuthology.orchestra.run.smithi039.stdout: "fs_name": "cephfs", 2024-09-21T03:45:32.744 INFO:teuthology.orchestra.run.smithi039.stdout: "rank_uptime": 68.042641012999994, 2024-09-21T03:45:32.744 INFO:teuthology.orchestra.run.smithi039.stdout: "mdsmap_epoch": 28, 2024-09-21T03:45:32.744 INFO:teuthology.orchestra.run.smithi039.stdout: "osdmap_epoch": 79, 2024-09-21T03:45:32.744 INFO:teuthology.orchestra.run.smithi039.stdout: "osdmap_epoch_barrier": 78, 2024-09-21T03:45:32.744 INFO:teuthology.orchestra.run.smithi039.stdout: "uptime": 90.475741049999996 2024-09-21T03:45:32.744 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-09-21T03:45:32.745 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:32.737+0000 7f71cd66a640 1 -- 172.21.15.39:0/1051941114 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f71b40b65f0 msgr2=0x7f71b40b8b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:32.745 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:32.737+0000 7f71cd66a640 1 --2- 172.21.15.39:0/1051941114 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f71b40b65f0 0x7f71b40b8b00 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f71c0005fd0 tx=0x7f71c0009800 comp rx=0 tx=0).stop 2024-09-21T03:45:32.745 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:32.737+0000 7f71cd66a640 1 -- 172.21.15.39:0/1051941114 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f71c8102ae0 msgr2=0x7f71c819c690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:32.745 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:32.737+0000 7f71cd66a640 1 --2- 172.21.15.39:0/1051941114 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f71c8102ae0 0x7f71c819c690 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f71b8005940 tx=0x7f71b80042f0 comp rx=0 tx=0).stop 2024-09-21T03:45:32.745 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:32.737+0000 7f71cd66a640 1 -- 172.21.15.39:0/1051941114 shutdown_connections 2024-09-21T03:45:32.745 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:32.737+0000 7f71cd66a640 1 --2- 172.21.15.39:0/1051941114 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f71b40b65f0 0x7f71b40b8b00 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:32.745 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:32.737+0000 7f71cd66a640 1 --2- 172.21.15.39:0/1051941114 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f71c8108d20 0x7f71c81a1ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:32.745 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:32.737+0000 7f71cd66a640 1 --2- 172.21.15.39:0/1051941114 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f71c81083d0 0x7f71c819cc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:32.745 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:32.737+0000 7f71cd66a640 1 --2- 172.21.15.39:0/1051941114 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f71c8102ae0 0x7f71c819c690 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:32.745 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:32.737+0000 7f71cd66a640 1 -- 172.21.15.39:0/1051941114 >> 172.21.15.39:0/1051941114 conn(0x7f71c8079790 msgr2=0x7f71c8106280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:45:32.746 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:32.737+0000 7f71cd66a640 1 -- 172.21.15.39:0/1051941114 shutdown_connections 2024-09-21T03:45:32.746 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:32.737+0000 7f71cd66a640 1 -- 172.21.15.39:0/1051941114 wait complete. 2024-09-21T03:45:33.213 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:45:32 smithi039 ceph-mon[31211]: pgmap v310: 97 pgs: 97 active+clean; 647 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-21T03:45:33.217 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:45:32 smithi161 ceph-mon[34533]: pgmap v310: 97 pgs: 97 active+clean; 647 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-21T03:45:33.224 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:45:32 smithi150 ceph-mon[34289]: pgmap v310: 97 pgs: 97 active+clean; 647 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-21T03:45:33.551 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 51116d5c-77ca-11ef-baf5-efdc52797490 -- ceph tell mds.c status 2024-09-21T03:45:33.811 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/51116d5c-77ca-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:45:34.267 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:34.265+0000 7efe90ae4640 1 -- 172.21.15.39:0/1918081107 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efe8c1094a0 msgr2=0x7efe8c1098b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:34.268 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:34.266+0000 7efe90ae4640 1 --2- 172.21.15.39:0/1918081107 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efe8c1094a0 0x7efe8c1098b0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7efe74009900 tx=0x7efe7402a410 comp rx=0 tx=0).stop 2024-09-21T03:45:34.268 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:34.266+0000 7efe90ae4640 1 -- 172.21.15.39:0/1918081107 shutdown_connections 2024-09-21T03:45:34.268 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:34.266+0000 7efe90ae4640 1 --2- 172.21.15.39:0/1918081107 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7efe8c109df0 0x7efe8c110620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:34.268 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:34.266+0000 7efe90ae4640 1 --2- 172.21.15.39:0/1918081107 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efe8c1094a0 0x7efe8c1098b0 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:34.268 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:34.266+0000 7efe90ae4640 1 --2- 172.21.15.39:0/1918081107 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7efe8c103b60 0x7efe8c103f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:34.268 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:34.266+0000 7efe90ae4640 1 -- 172.21.15.39:0/1918081107 >> 172.21.15.39:0/1918081107 conn(0x7efe8c07b850 msgr2=0x7efe8c07bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:45:34.268 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:34.266+0000 7efe90ae4640 1 -- 172.21.15.39:0/1918081107 shutdown_connections 2024-09-21T03:45:34.268 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:34.266+0000 7efe90ae4640 1 -- 172.21.15.39:0/1918081107 wait complete. 2024-09-21T03:45:34.270 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:34.266+0000 7efe90ae4640 1 Processor -- start 2024-09-21T03:45:34.270 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:34.267+0000 7efe90ae4640 1 -- start start 2024-09-21T03:45:34.270 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:34.267+0000 7efe90ae4640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7efe8c103b60 0x7efe8c19a4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:34.270 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:34.267+0000 7efe90ae4640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7efe8c1094a0 0x7efe8c19aa40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:34.270 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:34.267+0000 7efe90ae4640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efe8c109df0 0x7efe8c19fce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:34.270 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:34.267+0000 7efe90ae4640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7efe8c113ab0 con 0x7efe8c109df0 2024-09-21T03:45:34.270 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:34.267+0000 7efe90ae4640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7efe8c113930 con 0x7efe8c1094a0 2024-09-21T03:45:34.270 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:34.267+0000 7efe90ae4640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7efe8c113c30 con 0x7efe8c103b60 2024-09-21T03:45:34.270 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:34.267+0000 7efe8ad9d640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efe8c109df0 0x7efe8c19fce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:34.271 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:34.267+0000 7efe8ad9d640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efe8c109df0 0x7efe8c19fce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:40608/0 (socket says 172.21.15.39:40608) 2024-09-21T03:45:34.271 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:34.267+0000 7efe8ad9d640 1 -- 172.21.15.39:0/3305583911 learned_addr learned my addr 172.21.15.39:0/3305583911 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:45:34.271 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:34.267+0000 7efe81d9b640 1 --2- 172.21.15.39:0/3305583911 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7efe8c1094a0 0x7efe8c19aa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:34.271 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:34.267+0000 7efe8a59c640 1 --2- 172.21.15.39:0/3305583911 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7efe8c103b60 0x7efe8c19a4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:34.271 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:34.268+0000 7efe8ad9d640 1 -- 172.21.15.39:0/3305583911 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7efe8c103b60 msgr2=0x7efe8c19a4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:34.271 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:34.268+0000 7efe8ad9d640 1 --2- 172.21.15.39:0/3305583911 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7efe8c103b60 0x7efe8c19a4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:34.271 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:34.268+0000 7efe8ad9d640 1 -- 172.21.15.39:0/3305583911 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7efe8c1094a0 msgr2=0x7efe8c19aa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:34.271 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:34.268+0000 7efe8ad9d640 1 --2- 172.21.15.39:0/3305583911 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7efe8c1094a0 0x7efe8c19aa40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:34.271 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:34.268+0000 7efe8ad9d640 1 -- 172.21.15.39:0/3305583911 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efe8c1a0410 con 0x7efe8c109df0 2024-09-21T03:45:34.271 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:34.268+0000 7efe8ad9d640 1 --2- 172.21.15.39:0/3305583911 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efe8c109df0 0x7efe8c19fce0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7efe8400cad0 tx=0x7efe8400cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:45:34.271 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:34.268+0000 7efe83fff640 1 -- 172.21.15.39:0/3305583911 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe8400ae20 con 0x7efe8c109df0 2024-09-21T03:45:34.271 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:34.268+0000 7efe90ae4640 1 -- 172.21.15.39:0/3305583911 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efe8c1a0700 con 0x7efe8c109df0 2024-09-21T03:45:34.271 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:34.268+0000 7efe83fff640 1 -- 172.21.15.39:0/3305583911 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efe84005020 con 0x7efe8c109df0 2024-09-21T03:45:34.271 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:34.268+0000 7efe83fff640 1 -- 172.21.15.39:0/3305583911 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe8400f6b0 con 0x7efe8c109df0 2024-09-21T03:45:34.272 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:34.268+0000 7efe90ae4640 1 -- 172.21.15.39:0/3305583911 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efe8c1a6e60 con 0x7efe8c109df0 2024-09-21T03:45:34.272 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:34.271+0000 7efe97058740 1 Processor -- start 2024-09-21T03:45:34.273 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:34.272+0000 7efe83fff640 1 -- 172.21.15.39:0/3305583911 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 22) ==== 154580+0+0 (secure 0 0 0) 0x7efe84004b20 con 0x7efe8c109df0 2024-09-21T03:45:34.273 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:34.272+0000 7efe97058740 1 -- start start 2024-09-21T03:45:34.275 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:34.272+0000 7efe97058740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564ba60a8f60 0x564ba644ef00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:34.275 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:34.272+0000 7efe97058740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564ba60800a0 0x564ba644f440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:34.275 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:34.272+0000 7efe97058740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564ba6088280 0x564ba644f980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:34.275 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:34.272+0000 7efe97058740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x564ba63c6c10 con 0x564ba60a8f60 2024-09-21T03:45:34.275 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:34.272+0000 7efe97058740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x564ba63b6040 con 0x564ba6088280 2024-09-21T03:45:34.275 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:34.272+0000 7efe97058740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x564ba63b9e50 con 0x564ba60800a0 2024-09-21T03:45:34.275 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:34.272+0000 7efe81d9b640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564ba60a8f60 0x564ba644ef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:34.275 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:34.272+0000 7efe81d9b640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564ba60a8f60 0x564ba644ef00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:40622/0 (socket says 172.21.15.39:40622) 2024-09-21T03:45:34.276 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:34.272+0000 7efe81d9b640 1 -- 172.21.15.39:0/2481440668 learned_addr learned my addr 172.21.15.39:0/2481440668 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:45:34.276 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:34.272+0000 7efe8a59c640 1 --2- 172.21.15.39:0/2481440668 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564ba60800a0 0x564ba644f440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:34.276 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:34.273+0000 7efe81d9b640 1 --2- 172.21.15.39:0/2481440668 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564ba6088280 0x564ba644f980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:34.276 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:34.273+0000 7efe81d9b640 1 -- 172.21.15.39:0/2481440668 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564ba60800a0 msgr2=0x564ba644f440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:45:34.276 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:34.273+0000 7efe81d9b640 1 --2- 172.21.15.39:0/2481440668 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564ba60800a0 0x564ba644f440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:34.276 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:34.273+0000 7efe81d9b640 1 -- 172.21.15.39:0/2481440668 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564ba6088280 msgr2=0x564ba644f980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:45:34.276 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:34.273+0000 7efe81d9b640 1 --2- 172.21.15.39:0/2481440668 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564ba6088280 0x564ba644f980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:34.276 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:34.273+0000 7efe81d9b640 1 -- 172.21.15.39:0/2481440668 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564ba6114540 con 0x564ba60a8f60 2024-09-21T03:45:34.276 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:34.273+0000 7efe83fff640 1 --2- 172.21.15.39:0/3305583911 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7efe780b65f0 0x7efe780b8b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:34.276 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:34.273+0000 7efe81d9b640 1 --2- 172.21.15.39:0/2481440668 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564ba60a8f60 0x564ba644ef00 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7efe8c06c9f0 tx=0x7efe74000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8707e0af90431748 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:45:34.276 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:34.273+0000 7efe8a59c640 1 --2- 172.21.15.39:0/3305583911 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7efe780b65f0 0x7efe780b8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:34.277 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:34.273+0000 7efe83fff640 1 -- 172.21.15.39:0/3305583911 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8484+0+0 (secure 0 0 0) 0x7efe840e7590 con 0x7efe8c109df0 2024-09-21T03:45:34.277 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:34.273+0000 7efe80d99640 1 -- 172.21.15.39:0/2481440668 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe74044020 con 0x564ba60a8f60 2024-09-21T03:45:34.277 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:34.273+0000 7efe80d99640 1 -- 172.21.15.39:0/2481440668 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efe74031820 con 0x564ba60a8f60 2024-09-21T03:45:34.277 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:34.273+0000 7efe80d99640 1 -- 172.21.15.39:0/2481440668 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe74035040 con 0x564ba60a8f60 2024-09-21T03:45:34.281 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:34.274+0000 7efe97058740 1 -- 172.21.15.39:0/2481440668 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564ba60a8f60 msgr2=0x564ba644ef00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:34.281 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:34.274+0000 7efe8a59c640 1 --2- 172.21.15.39:0/3305583911 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7efe780b65f0 0x7efe780b8b00 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x564ba644db80 tx=0x7efe7c00b390 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:45:34.281 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:34.274+0000 7efe97058740 1 --2- 172.21.15.39:0/2481440668 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564ba60a8f60 0x564ba644ef00 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7efe8c06c9f0 tx=0x7efe74000bc0 comp rx=0 tx=0).stop 2024-09-21T03:45:34.281 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:34.274+0000 7efe97058740 1 -- 172.21.15.39:0/2481440668 shutdown_connections 2024-09-21T03:45:34.281 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:34.274+0000 7efe97058740 1 --2- 172.21.15.39:0/2481440668 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564ba6088280 0x564ba644f980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:34.281 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:34.274+0000 7efe97058740 1 --2- 172.21.15.39:0/2481440668 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564ba60800a0 0x564ba644f440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:34.281 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:34.274+0000 7efe97058740 1 --2- 172.21.15.39:0/2481440668 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564ba60a8f60 0x564ba644ef00 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:34.281 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:34.274+0000 7efe97058740 1 -- 172.21.15.39:0/2481440668 >> 172.21.15.39:0/2481440668 conn(0x564ba609d350 msgr2=0x564ba644cdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:45:34.281 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:34.274+0000 7efe97058740 1 -- 172.21.15.39:0/2481440668 shutdown_connections 2024-09-21T03:45:34.281 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:34.274+0000 7efe97058740 1 -- 172.21.15.39:0/2481440668 wait complete. 2024-09-21T03:45:34.281 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:34.275+0000 7efe97058740 1 -- start start 2024-09-21T03:45:34.282 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:34.275+0000 7efe97058740 1 Processor -- start 2024-09-21T03:45:34.282 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:34.275+0000 7efe97058740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564ba60800a0 0x564ba645ee40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:34.282 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:34.275+0000 7efe97058740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564ba6088280 0x564ba645f380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:34.282 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:34.275+0000 7efe97058740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564ba60a8f60 0x564ba646ad40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:34.282 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:34.275+0000 7efe97058740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x564ba62175f0 con 0x564ba60a8f60 2024-09-21T03:45:34.282 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:34.275+0000 7efe97058740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x564ba6351010 con 0x564ba60800a0 2024-09-21T03:45:34.282 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:34.275+0000 7efe97058740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x564ba64619e0 con 0x564ba6088280 2024-09-21T03:45:34.282 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:34.275+0000 7efe97058740 10 client.0 _ll_register_callbacks cb 0x564ba63b0340 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:45:34.282 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:34.275+0000 7efe8ad9d640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564ba60a8f60 0x564ba646ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:34.282 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:34.275+0000 7efe8ad9d640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564ba60a8f60 0x564ba646ad40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:40628/0 (socket says 172.21.15.39:40628) 2024-09-21T03:45:34.282 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:34.275+0000 7efe8ad9d640 1 -- 172.21.15.39:0/1924027791 learned_addr learned my addr 172.21.15.39:0/1924027791 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:45:34.282 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:34.276+0000 7efe8a59c640 1 --2- 172.21.15.39:0/1924027791 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564ba60800a0 0x564ba645ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:34.282 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:34.276+0000 7efe81d9b640 1 --2- 172.21.15.39:0/1924027791 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564ba6088280 0x564ba645f380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:34.283 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:34.276+0000 7efe8ad9d640 1 -- 172.21.15.39:0/1924027791 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564ba6088280 msgr2=0x564ba645f380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:45:34.283 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:34.276+0000 7efe8ad9d640 1 --2- 172.21.15.39:0/1924027791 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564ba6088280 0x564ba645f380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:34.283 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:34.276+0000 7efe8ad9d640 1 -- 172.21.15.39:0/1924027791 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564ba60800a0 msgr2=0x564ba645ee40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:45:34.283 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:34.276+0000 7efe8ad9d640 1 --2- 172.21.15.39:0/1924027791 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564ba60800a0 0x564ba645ee40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:34.283 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:34.276+0000 7efe8ad9d640 1 -- 172.21.15.39:0/1924027791 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564ba637ea00 con 0x564ba60a8f60 2024-09-21T03:45:34.283 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:34.276+0000 7efe81d9b640 1 --2- 172.21.15.39:0/1924027791 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564ba6088280 0x564ba645f380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:45:34.283 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:34.276+0000 7efe8a59c640 1 --2- 172.21.15.39:0/1924027791 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564ba60800a0 0x564ba645ee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:45:34.283 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:34.276+0000 7efe8ad9d640 1 --2- 172.21.15.39:0/1924027791 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564ba60a8f60 0x564ba646ad40 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7efe840a9b40 tx=0x7efe84004e10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b260ee6a2a07eccd server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:45:34.283 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:34.276+0000 7efe71ffb640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-09-21T03:45:34.283 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:34.276+0000 7efe71ffb640 1 -- 172.21.15.39:0/1924027791 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe8401d030 con 0x564ba60a8f60 2024-09-21T03:45:34.283 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:34.276+0000 7efe71ffb640 1 -- 172.21.15.39:0/1924027791 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8484+0+0 (secure 0 0 0) 0x7efe840e72e0 con 0x564ba60a8f60 2024-09-21T03:45:34.283 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:34.276+0000 7efe97058740 1 -- 172.21.15.39:0/1924027791 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efe74031820 con 0x564ba60a8f60 2024-09-21T03:45:34.283 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:34.276+0000 7efe71ffb640 1 -- 172.21.15.39:0/1924027791 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7efe840b6ee0 con 0x564ba60a8f60 2024-09-21T03:45:34.284 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:34.276+0000 7efe97058740 10 client.15252 fetch_fsmap learned FSMap version 28 2024-09-21T03:45:34.284 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:34.276+0000 7efe97058740 1 -- 172.21.15.39:0/1924027791 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x564ba646da10 con 0x564ba60a8f60 2024-09-21T03:45:34.284 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:34.277+0000 7efe71ffb640 1 -- 172.21.15.39:0/1924027791 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efe74031820 con 0x564ba60a8f60 2024-09-21T03:45:34.284 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:34.277+0000 7efe71ffb640 1 -- 172.21.15.39:0/1924027791 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7efe840b3130 con 0x564ba60a8f60 2024-09-21T03:45:34.284 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:34.277+0000 7efe97058740 10 client.15252 fetch_fsmap finished waiting for FSMap version 28 2024-09-21T03:45:34.284 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:34.277+0000 7efe97058740 10 client.15252 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-21T03:45:34.284 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:34.277+0000 7efe97058740 20 client.15252 populate_metadata read hostname 'smithi039' 2024-09-21T03:45:34.284 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:34.277+0000 7efe97058740 1 --2- 172.21.15.39:0/1924027791 >> [v2:172.21.15.161:6836/3451544597,v1:172.21.15.161:6837/3451544597] conn(0x564ba60678e0 0x564ba6471420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:34.284 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:34.277+0000 7efe97058740 4 client.15252 mds_command: new command op to 24646 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:45:34.284 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:34.277+0000 7efe97058740 1 -- 172.21.15.39:0/1924027791 --> [v2:172.21.15.161:6836/3451544597,v1:172.21.15.161:6837/3451544597] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564ba63c8b60 con 0x564ba60678e0 2024-09-21T03:45:34.284 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:34.277+0000 7efe8a59c640 1 --2- 172.21.15.39:0/1924027791 >> [v2:172.21.15.161:6836/3451544597,v1:172.21.15.161:6837/3451544597] conn(0x564ba60678e0 0x564ba6471420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:34.284 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:34.278+0000 7efe71ffb640 10 client.15252 ms_handle_connect on v2:172.21.15.161:6836/3451544597 2024-09-21T03:45:34.284 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:34.278+0000 7efe8a59c640 1 --2- 172.21.15.39:0/1924027791 >> [v2:172.21.15.161:6836/3451544597,v1:172.21.15.161:6837/3451544597] conn(0x564ba60678e0 0x564ba6471420 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=1b1fc4ec98f490fc server_cookie=57ceab0ba345f623 in_seq=0 out_seq=0 2024-09-21T03:45:34.285 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:34.278+0000 7efe71ffb640 1 -- 172.21.15.39:0/1924027791 <== mds.3 v2:172.21.15.161:6836/3451544597 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x564ba63c8b60 con 0x564ba60678e0 2024-09-21T03:45:34.285 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:34.278+0000 7efe71ffb640 10 client.15252 handle_command_reply: tid=0 2024-09-21T03:45:34.285 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:34.279+0000 7efe97058740 1 client.15252 shutdown 2024-09-21T03:45:34.285 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:34.279+0000 7efe97058740 1 -- 172.21.15.39:0/1924027791 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564ba60a8f60 msgr2=0x564ba646ad40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:34.285 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:34.279+0000 7efe97058740 1 --2- 172.21.15.39:0/1924027791 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564ba60a8f60 0x564ba646ad40 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7efe840a9b40 tx=0x7efe84004e10 comp rx=0 tx=0).stop 2024-09-21T03:45:34.285 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:34.279+0000 7efe97058740 1 -- 172.21.15.39:0/1924027791 shutdown_connections 2024-09-21T03:45:34.285 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:34.279+0000 7efe97058740 1 --2- 172.21.15.39:0/1924027791 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564ba60a8f60 0x564ba646ad40 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:34.285 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:34.279+0000 7efe97058740 1 --2- 172.21.15.39:0/1924027791 >> [v2:172.21.15.161:6836/3451544597,v1:172.21.15.161:6837/3451544597] conn(0x564ba60678e0 0x564ba6471420 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:34.285 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:34.279+0000 7efe97058740 1 --2- 172.21.15.39:0/1924027791 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564ba6088280 0x564ba645f380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:34.285 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:34.279+0000 7efe97058740 1 --2- 172.21.15.39:0/1924027791 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564ba60800a0 0x564ba645ee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:34.285 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:34.279+0000 7efe71ffb640 0 client.15252 ms_handle_reset on v2:172.21.15.161:6836/3451544597 2024-09-21T03:45:34.285 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:34.279+0000 7efe97058740 1 -- 172.21.15.39:0/1924027791 >> 172.21.15.39:0/1924027791 conn(0x564ba609d350 msgr2=0x564ba644b3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:45:34.285 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:34.279+0000 7efe97058740 1 -- 172.21.15.39:0/1924027791 shutdown_connections 2024-09-21T03:45:34.286 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:34.279+0000 7efe97058740 1 -- 172.21.15.39:0/1924027791 wait complete. 2024-09-21T03:45:34.286 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:34.279+0000 7efe97058740 20 client.15252 trim_cache size 0 max 16384 2024-09-21T03:45:34.299 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:34.297+0000 7efe97058740 1 Processor -- start 2024-09-21T03:45:34.299 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:34.297+0000 7efe97058740 1 -- start start 2024-09-21T03:45:34.300 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:34.297+0000 7efe97058740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564ba60800a0 0x564ba6459d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:34.300 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:34.297+0000 7efe97058740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564ba6088280 0x564ba645a2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:34.300 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:34.297+0000 7efe97058740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564ba60a8f60 0x564ba645d4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:34.300 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:34.297+0000 7efe97058740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x564ba646f760 con 0x564ba6088280 2024-09-21T03:45:34.300 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:34.297+0000 7efe97058740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x564ba646fe30 con 0x564ba60a8f60 2024-09-21T03:45:34.300 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:34.297+0000 7efe97058740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x564ba6470500 con 0x564ba60800a0 2024-09-21T03:45:34.300 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:34.298+0000 7efe81d9b640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564ba6088280 0x564ba645a2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:34.300 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:34.298+0000 7efe81d9b640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564ba6088280 0x564ba645a2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:40630/0 (socket says 172.21.15.39:40630) 2024-09-21T03:45:34.300 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:34.298+0000 7efe81d9b640 1 -- 172.21.15.39:0/3000294762 learned_addr learned my addr 172.21.15.39:0/3000294762 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:45:34.300 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:34.298+0000 7efe81d9b640 1 -- 172.21.15.39:0/3000294762 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564ba60800a0 msgr2=0x564ba6459d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:45:34.300 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:34.298+0000 7efe81d9b640 1 --2- 172.21.15.39:0/3000294762 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564ba60800a0 0x564ba6459d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:34.300 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:34.298+0000 7efe81d9b640 1 -- 172.21.15.39:0/3000294762 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564ba60a8f60 msgr2=0x564ba645d4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:45:34.300 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:34.298+0000 7efe81d9b640 1 --2- 172.21.15.39:0/3000294762 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564ba60a8f60 0x564ba645d4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:34.300 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:34.298+0000 7efe81d9b640 1 -- 172.21.15.39:0/3000294762 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564ba6450e40 con 0x564ba6088280 2024-09-21T03:45:34.301 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:34.298+0000 7efe81d9b640 1 --2- 172.21.15.39:0/3000294762 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564ba6088280 0x564ba645a2c0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7efe74004650 tx=0x7efe74005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe81262da7e26a7e server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:45:34.304 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:34.298+0000 7efe73fff640 1 -- 172.21.15.39:0/3000294762 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe74044030 con 0x564ba6088280 2024-09-21T03:45:34.304 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:34.298+0000 7efe73fff640 1 -- 172.21.15.39:0/3000294762 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efe74031820 con 0x564ba6088280 2024-09-21T03:45:34.304 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:34.298+0000 7efe73fff640 1 -- 172.21.15.39:0/3000294762 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe74035040 con 0x564ba6088280 2024-09-21T03:45:34.304 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:34.299+0000 7efe97058740 1 -- 172.21.15.39:0/3000294762 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564ba6088280 msgr2=0x564ba645a2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:34.304 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:34.299+0000 7efe97058740 1 --2- 172.21.15.39:0/3000294762 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564ba6088280 0x564ba645a2c0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7efe74004650 tx=0x7efe74005fd0 comp rx=0 tx=0).stop 2024-09-21T03:45:34.304 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:34.299+0000 7efe97058740 1 -- 172.21.15.39:0/3000294762 shutdown_connections 2024-09-21T03:45:34.304 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:34.299+0000 7efe97058740 1 --2- 172.21.15.39:0/3000294762 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564ba60a8f60 0x564ba645d4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:34.304 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:34.299+0000 7efe97058740 1 --2- 172.21.15.39:0/3000294762 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564ba6088280 0x564ba645a2c0 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:34.304 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:34.299+0000 7efe97058740 1 --2- 172.21.15.39:0/3000294762 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564ba60800a0 0x564ba6459d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:34.304 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:34.299+0000 7efe97058740 1 -- 172.21.15.39:0/3000294762 >> 172.21.15.39:0/3000294762 conn(0x564ba609d350 msgr2=0x564ba645ee20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:45:34.304 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:34.299+0000 7efe97058740 1 -- 172.21.15.39:0/3000294762 shutdown_connections 2024-09-21T03:45:34.304 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:34.299+0000 7efe97058740 1 -- 172.21.15.39:0/3000294762 wait complete. 2024-09-21T03:45:34.304 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:34.299+0000 7efe97058740 1 -- start start 2024-09-21T03:45:34.305 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:34.300+0000 7efe97058740 1 Processor -- start 2024-09-21T03:45:34.305 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:34.300+0000 7efe97058740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564ba60800a0 0x564ba6479ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:34.305 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:34.300+0000 7efe97058740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564ba6088280 0x564ba64734e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:34.305 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:34.300+0000 7efe97058740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564ba60a8f60 0x564ba6473a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:34.305 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:34.300+0000 7efe97058740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x564ba64712a0 con 0x564ba60800a0 2024-09-21T03:45:34.305 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:34.300+0000 7efe97058740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x564ba6470bd0 con 0x564ba60a8f60 2024-09-21T03:45:34.305 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:34.300+0000 7efe97058740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x564ba6471970 con 0x564ba6088280 2024-09-21T03:45:34.305 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:34.300+0000 7efe97058740 10 client.0 _ll_register_callbacks cb 0x7efe840a9c40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:45:34.305 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:34.300+0000 7efe8a59c640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564ba60800a0 0x564ba6479ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:34.305 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:34.300+0000 7efe8a59c640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564ba60800a0 0x564ba6479ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:40638/0 (socket says 172.21.15.39:40638) 2024-09-21T03:45:34.305 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:34.300+0000 7efe8a59c640 1 -- 172.21.15.39:0/593216504 learned_addr learned my addr 172.21.15.39:0/593216504 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:45:34.305 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:34.300+0000 7efe81d9b640 1 --2- 172.21.15.39:0/593216504 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564ba6088280 0x564ba64734e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:34.305 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:34.300+0000 7efe8ad9d640 1 --2- 172.21.15.39:0/593216504 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564ba60a8f60 0x564ba6473a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:34.305 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:34.300+0000 7efe8a59c640 1 -- 172.21.15.39:0/593216504 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564ba6088280 msgr2=0x564ba64734e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:45:34.305 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:34.300+0000 7efe8a59c640 1 --2- 172.21.15.39:0/593216504 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564ba6088280 0x564ba64734e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:34.306 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:34.300+0000 7efe8a59c640 1 -- 172.21.15.39:0/593216504 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564ba60a8f60 msgr2=0x564ba6473a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:45:34.306 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:34.300+0000 7efe8a59c640 1 --2- 172.21.15.39:0/593216504 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564ba60a8f60 0x564ba6473a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:34.306 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:34.300+0000 7efe8a59c640 1 -- 172.21.15.39:0/593216504 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564ba64741f0 con 0x564ba60800a0 2024-09-21T03:45:34.306 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:34.300+0000 7efe8ad9d640 1 --2- 172.21.15.39:0/593216504 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564ba60a8f60 0x564ba6473a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:45:34.306 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:34.301+0000 7efe8a59c640 1 --2- 172.21.15.39:0/593216504 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564ba60800a0 0x564ba6479ef0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7efe7c003d00 tx=0x7efe7c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb4f1e62c16e9101 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:45:34.306 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:34.301+0000 7efe71ffb640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-09-21T03:45:34.306 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:34.301+0000 7efe71ffb640 1 -- 172.21.15.39:0/593216504 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe7c01ede0 con 0x564ba60800a0 2024-09-21T03:45:34.306 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:34.301+0000 7efe71ffb640 1 -- 172.21.15.39:0/593216504 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8484+0+0 (secure 0 0 0) 0x7efe7c02a420 con 0x564ba60800a0 2024-09-21T03:45:34.306 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:34.301+0000 7efe97058740 1 -- 172.21.15.39:0/593216504 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efe74031820 con 0x564ba60800a0 2024-09-21T03:45:34.306 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:34.301+0000 7efe71ffb640 1 -- 172.21.15.39:0/593216504 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7efe7c02a620 con 0x564ba60800a0 2024-09-21T03:45:34.306 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:34.301+0000 7efe97058740 10 client.15258 fetch_fsmap learned FSMap version 28 2024-09-21T03:45:34.306 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:34.301+0000 7efe97058740 1 -- 172.21.15.39:0/593216504 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x564ba64dc8f0 con 0x564ba60800a0 2024-09-21T03:45:34.306 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:34.301+0000 7efe71ffb640 1 -- 172.21.15.39:0/593216504 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efe74031820 con 0x564ba60800a0 2024-09-21T03:45:34.306 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:34.301+0000 7efe71ffb640 1 -- 172.21.15.39:0/593216504 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7efe7c038020 con 0x564ba60800a0 2024-09-21T03:45:34.307 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:34.301+0000 7efe97058740 10 client.15258 fetch_fsmap finished waiting for FSMap version 28 2024-09-21T03:45:34.307 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:34.301+0000 7efe97058740 10 client.15258 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-21T03:45:34.307 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:34.301+0000 7efe97058740 20 client.15258 populate_metadata read hostname 'smithi039' 2024-09-21T03:45:34.307 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:34.301+0000 7efe97058740 1 --2- 172.21.15.39:0/593216504 >> [v2:172.21.15.161:6836/3451544597,v1:172.21.15.161:6837/3451544597] conn(0x564ba603cc90 0x564ba64e0120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:34.307 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:34.301+0000 7efe97058740 4 client.15258 mds_command: new command op to 24646 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-21T03:45:34.307 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:34.302+0000 7efe97058740 1 -- 172.21.15.39:0/593216504 --> [v2:172.21.15.161:6836/3451544597,v1:172.21.15.161:6837/3451544597] -- command(tid 0: {"prefix": "status"}) -- 0x564ba63c4610 con 0x564ba603cc90 2024-09-21T03:45:34.307 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:34.302+0000 7efe81d9b640 1 --2- 172.21.15.39:0/593216504 >> [v2:172.21.15.161:6836/3451544597,v1:172.21.15.161:6837/3451544597] conn(0x564ba603cc90 0x564ba64e0120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:34.307 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:34.302+0000 7efe81d9b640 1 --2- 172.21.15.39:0/593216504 >> [v2:172.21.15.161:6836/3451544597,v1:172.21.15.161:6837/3451544597] conn(0x564ba603cc90 0x564ba64e0120 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=4e5c94a486d47c07 server_cookie=9277672f70538403 in_seq=0 out_seq=0 2024-09-21T03:45:34.307 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:34.302+0000 7efe71ffb640 10 client.15258 ms_handle_connect on v2:172.21.15.161:6836/3451544597 2024-09-21T03:45:34.307 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:34.302+0000 7efe71ffb640 1 -- 172.21.15.39:0/593216504 <== mds.3 v2:172.21.15.161:6836/3451544597 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x564ba63c4610 con 0x564ba603cc90 2024-09-21T03:45:34.307 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:34.302+0000 7efe71ffb640 10 client.15258 handle_command_reply: tid=0 2024-09-21T03:45:34.307 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:34.302+0000 7efe97058740 1 client.15258 shutdown 2024-09-21T03:45:34.308 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:34.302+0000 7efe97058740 1 -- 172.21.15.39:0/593216504 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564ba60800a0 msgr2=0x564ba6479ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:34.308 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:34.302+0000 7efe97058740 1 --2- 172.21.15.39:0/593216504 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564ba60800a0 0x564ba6479ef0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7efe7c003d00 tx=0x7efe7c004730 comp rx=0 tx=0).stop 2024-09-21T03:45:34.308 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:34.302+0000 7efe97058740 1 -- 172.21.15.39:0/593216504 shutdown_connections 2024-09-21T03:45:34.308 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:34.303+0000 7efe97058740 1 --2- 172.21.15.39:0/593216504 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564ba60a8f60 0x564ba6473a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:34.308 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:34.303+0000 7efe97058740 1 --2- 172.21.15.39:0/593216504 >> [v2:172.21.15.161:6836/3451544597,v1:172.21.15.161:6837/3451544597] conn(0x564ba603cc90 0x564ba64e0120 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:34.308 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:34.303+0000 7efe97058740 1 --2- 172.21.15.39:0/593216504 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564ba6088280 0x564ba64734e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:34.309 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:34.303+0000 7efe97058740 1 --2- 172.21.15.39:0/593216504 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564ba60800a0 0x564ba6479ef0 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:34.309 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:34.303+0000 7efe97058740 1 -- 172.21.15.39:0/593216504 >> 172.21.15.39:0/593216504 conn(0x564ba609d350 msgr2=0x564ba6459d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:45:34.309 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:34.303+0000 7efe71ffb640 0 client.15258 ms_handle_reset on v2:172.21.15.161:6836/3451544597 2024-09-21T03:45:34.309 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:34.303+0000 7efe97058740 1 -- 172.21.15.39:0/593216504 shutdown_connections 2024-09-21T03:45:34.309 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:34.303+0000 7efe97058740 1 -- 172.21.15.39:0/593216504 wait complete. 2024-09-21T03:45:34.309 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:34.303+0000 7efe97058740 20 client.15258 trim_cache size 0 max 16384 2024-09-21T03:45:34.309 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-09-21T03:45:34.309 INFO:teuthology.orchestra.run.smithi039.stdout: "cluster_fsid": "51116d5c-77ca-11ef-baf5-efdc52797490", 2024-09-21T03:45:34.309 INFO:teuthology.orchestra.run.smithi039.stdout: "whoami": 3, 2024-09-21T03:45:34.309 INFO:teuthology.orchestra.run.smithi039.stdout: "id": 24646, 2024-09-21T03:45:34.309 INFO:teuthology.orchestra.run.smithi039.stdout: "want_state": "up:active", 2024-09-21T03:45:34.309 INFO:teuthology.orchestra.run.smithi039.stdout: "state": "up:active", 2024-09-21T03:45:34.310 INFO:teuthology.orchestra.run.smithi039.stdout: "fs_name": "cephfs", 2024-09-21T03:45:34.310 INFO:teuthology.orchestra.run.smithi039.stdout: "rank_uptime": 69.906424697999995, 2024-09-21T03:45:34.310 INFO:teuthology.orchestra.run.smithi039.stdout: "mdsmap_epoch": 28, 2024-09-21T03:45:34.310 INFO:teuthology.orchestra.run.smithi039.stdout: "osdmap_epoch": 79, 2024-09-21T03:45:34.310 INFO:teuthology.orchestra.run.smithi039.stdout: "osdmap_epoch_barrier": 78, 2024-09-21T03:45:34.310 INFO:teuthology.orchestra.run.smithi039.stdout: "uptime": 103.24015147599999 2024-09-21T03:45:34.310 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-09-21T03:45:34.311 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:34.303+0000 7efe90ae4640 1 -- 172.21.15.39:0/3305583911 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7efe780b65f0 msgr2=0x7efe780b8b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:34.311 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:34.303+0000 7efe90ae4640 1 --2- 172.21.15.39:0/3305583911 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7efe780b65f0 0x7efe780b8b00 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x564ba644db80 tx=0x7efe7c00b390 comp rx=0 tx=0).stop 2024-09-21T03:45:34.311 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:34.303+0000 7efe90ae4640 1 -- 172.21.15.39:0/3305583911 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efe8c109df0 msgr2=0x7efe8c19fce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:34.311 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:34.303+0000 7efe90ae4640 1 --2- 172.21.15.39:0/3305583911 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efe8c109df0 0x7efe8c19fce0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7efe8400cad0 tx=0x7efe8400cf90 comp rx=0 tx=0).stop 2024-09-21T03:45:34.311 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:34.303+0000 7efe90ae4640 1 -- 172.21.15.39:0/3305583911 shutdown_connections 2024-09-21T03:45:34.311 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:34.303+0000 7efe90ae4640 1 --2- 172.21.15.39:0/3305583911 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7efe780b65f0 0x7efe780b8b00 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:34.311 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:34.303+0000 7efe90ae4640 1 --2- 172.21.15.39:0/3305583911 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efe8c109df0 0x7efe8c19fce0 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:34.311 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:34.303+0000 7efe90ae4640 1 --2- 172.21.15.39:0/3305583911 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7efe8c1094a0 0x7efe8c19aa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:34.311 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:34.303+0000 7efe90ae4640 1 --2- 172.21.15.39:0/3305583911 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7efe8c103b60 0x7efe8c19a4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:34.311 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:34.303+0000 7efe90ae4640 1 -- 172.21.15.39:0/3305583911 >> 172.21.15.39:0/3305583911 conn(0x7efe8c07b850 msgr2=0x7efe8c105510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:45:34.311 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:34.304+0000 7efe90ae4640 1 -- 172.21.15.39:0/3305583911 shutdown_connections 2024-09-21T03:45:34.311 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:34.304+0000 7efe90ae4640 1 -- 172.21.15.39:0/3305583911 wait complete. 2024-09-21T03:45:35.056 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 51116d5c-77ca-11ef-baf5-efdc52797490 -- ceph tell mds.h status 2024-09-21T03:45:35.218 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:45:34 smithi161 ceph-mon[34533]: pgmap v311: 97 pgs: 97 active+clean; 647 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-09-21T03:45:35.225 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:45:34 smithi150 ceph-mon[34289]: pgmap v311: 97 pgs: 97 active+clean; 647 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-09-21T03:45:35.308 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/51116d5c-77ca-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:45:35.333 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:45:34 smithi039 ceph-mon[31211]: pgmap v311: 97 pgs: 97 active+clean; 647 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-09-21T03:45:35.824 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:35.814+0000 7fcfc12da640 1 -- 172.21.15.39:0/1585159236 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcfbc102a60 msgr2=0x7fcfbc102e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:35.824 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:35.814+0000 7fcfc12da640 1 --2- 172.21.15.39:0/1585159236 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcfbc102a60 0x7fcfbc102e70 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7fcfac009900 tx=0x7fcfac02a410 comp rx=0 tx=0).stop 2024-09-21T03:45:35.825 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:35.814+0000 7fcfc12da640 1 -- 172.21.15.39:0/1585159236 shutdown_connections 2024-09-21T03:45:35.825 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:35.814+0000 7fcfc12da640 1 --2- 172.21.15.39:0/1585159236 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcfbc108cf0 0x7fcfbc10f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:35.825 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:35.814+0000 7fcfc12da640 1 --2- 172.21.15.39:0/1585159236 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcfbc1083a0 0x7fcfbc1087b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:35.825 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:35.814+0000 7fcfc12da640 1 --2- 172.21.15.39:0/1585159236 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcfbc102a60 0x7fcfbc102e70 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:35.825 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:35.814+0000 7fcfc12da640 1 -- 172.21.15.39:0/1585159236 >> 172.21.15.39:0/1585159236 conn(0x7fcfbc079730 msgr2=0x7fcfbc079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:45:35.825 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:35.814+0000 7fcfc12da640 1 -- 172.21.15.39:0/1585159236 shutdown_connections 2024-09-21T03:45:35.825 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:35.814+0000 7fcfc12da640 1 -- 172.21.15.39:0/1585159236 wait complete. 2024-09-21T03:45:35.825 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:35.815+0000 7fcfc12da640 1 Processor -- start 2024-09-21T03:45:35.825 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:35.815+0000 7fcfc12da640 1 -- start start 2024-09-21T03:45:35.825 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:35.816+0000 7fcfc12da640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcfbc102a60 0x7fcfbc199380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:35.825 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:35.816+0000 7fcfc12da640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcfbc1083a0 0x7fcfbc199920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:35.825 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:35.816+0000 7fcfc12da640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcfbc108cf0 0x7fcfbc19ebc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:35.825 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:35.816+0000 7fcfc12da640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fcfbc1129b0 con 0x7fcfbc108cf0 2024-09-21T03:45:35.826 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:35.816+0000 7fcfc12da640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fcfbc112830 con 0x7fcfbc102a60 2024-09-21T03:45:35.826 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:35.816+0000 7fcfc12da640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fcfbc112b30 con 0x7fcfbc1083a0 2024-09-21T03:45:35.826 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:35.816+0000 7fcfbb7fe640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcfbc108cf0 0x7fcfbc19ebc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:35.826 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:35.816+0000 7fcfbb7fe640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcfbc108cf0 0x7fcfbc19ebc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:40666/0 (socket says 172.21.15.39:40666) 2024-09-21T03:45:35.826 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:35.816+0000 7fcfbb7fe640 1 -- 172.21.15.39:0/3682034190 learned_addr learned my addr 172.21.15.39:0/3682034190 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:45:35.826 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:35.816+0000 7fcfba7fc640 1 --2- 172.21.15.39:0/3682034190 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcfbc1083a0 0x7fcfbc199920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:35.826 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:35.816+0000 7fcfbaffd640 1 --2- 172.21.15.39:0/3682034190 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcfbc102a60 0x7fcfbc199380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:35.826 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:35.816+0000 7fcfbb7fe640 1 -- 172.21.15.39:0/3682034190 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcfbc1083a0 msgr2=0x7fcfbc199920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:35.826 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:35.816+0000 7fcfbb7fe640 1 --2- 172.21.15.39:0/3682034190 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcfbc1083a0 0x7fcfbc199920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:35.826 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:35.816+0000 7fcfbb7fe640 1 -- 172.21.15.39:0/3682034190 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcfbc102a60 msgr2=0x7fcfbc199380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:35.826 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:35.816+0000 7fcfbb7fe640 1 --2- 172.21.15.39:0/3682034190 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcfbc102a60 0x7fcfbc199380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:35.826 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:35.816+0000 7fcfbb7fe640 1 -- 172.21.15.39:0/3682034190 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcfbc19f2f0 con 0x7fcfbc108cf0 2024-09-21T03:45:35.826 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:35.816+0000 7fcfbaffd640 1 --2- 172.21.15.39:0/3682034190 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcfbc102a60 0x7fcfbc199380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:45:35.827 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:35.816+0000 7fcfbb7fe640 1 --2- 172.21.15.39:0/3682034190 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcfbc108cf0 0x7fcfbc19ebc0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7fcfb400eb10 tx=0x7fcfb400c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:45:35.827 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:35.816+0000 7fcfa3fff640 1 -- 172.21.15.39:0/3682034190 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfb4004b20 con 0x7fcfbc108cf0 2024-09-21T03:45:35.827 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:35.817+0000 7fcfa3fff640 1 -- 172.21.15.39:0/3682034190 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcfb4004cc0 con 0x7fcfbc108cf0 2024-09-21T03:45:35.827 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:35.817+0000 7fcfc12da640 1 -- 172.21.15.39:0/3682034190 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcfbc19f5e0 con 0x7fcfbc108cf0 2024-09-21T03:45:35.827 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:35.817+0000 7fcfa3fff640 1 -- 172.21.15.39:0/3682034190 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfb4010670 con 0x7fcfbc108cf0 2024-09-21T03:45:35.827 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:35.817+0000 7fcfc12da640 1 -- 172.21.15.39:0/3682034190 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcfbc1a5d40 con 0x7fcfbc108cf0 2024-09-21T03:45:35.827 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:35.820+0000 7fcfc784e740 1 Processor -- start 2024-09-21T03:45:35.827 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:35.820+0000 7fcfa3fff640 1 -- 172.21.15.39:0/3682034190 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 22) ==== 154580+0+0 (secure 0 0 0) 0x7fcfb4005020 con 0x7fcfbc108cf0 2024-09-21T03:45:35.827 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:35.820+0000 7fcfc784e740 1 -- start start 2024-09-21T03:45:35.827 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:35.820+0000 7fcfc784e740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56020bc19f60 0x56020bfbff00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:35.827 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:35.820+0000 7fcfc784e740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56020bbf10a0 0x56020bfc0440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:35.827 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:35.820+0000 7fcfc784e740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56020bbf9280 0x56020bfc0980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:35.827 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:35.820+0000 7fcfc784e740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x56020bf37c10 con 0x56020bbf9280 2024-09-21T03:45:35.827 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:35.820+0000 7fcfc784e740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x56020bf27040 con 0x56020bbf10a0 2024-09-21T03:45:35.828 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:35.820+0000 7fcfc784e740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x56020bf2ae50 con 0x56020bc19f60 2024-09-21T03:45:35.828 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:35.821+0000 7fcfba7fc640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56020bbf9280 0x56020bfc0980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:35.828 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:35.821+0000 7fcfba7fc640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56020bbf9280 0x56020bfc0980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:40672/0 (socket says 172.21.15.39:40672) 2024-09-21T03:45:35.828 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:35.821+0000 7fcfba7fc640 1 -- 172.21.15.39:0/3345941510 learned_addr learned my addr 172.21.15.39:0/3345941510 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:45:35.828 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:35.821+0000 7fcfbaffd640 1 --2- 172.21.15.39:0/3345941510 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56020bbf10a0 0x56020bfc0440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:35.828 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:35.821+0000 7fcfba7fc640 1 --2- 172.21.15.39:0/3345941510 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56020bc19f60 0x56020bfbff00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:35.828 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:35.821+0000 7fcfba7fc640 1 -- 172.21.15.39:0/3345941510 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56020bc19f60 msgr2=0x56020bfbff00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:45:35.828 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:35.821+0000 7fcfba7fc640 1 --2- 172.21.15.39:0/3345941510 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56020bc19f60 0x56020bfbff00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:35.828 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:35.821+0000 7fcfba7fc640 1 -- 172.21.15.39:0/3345941510 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56020bbf10a0 msgr2=0x56020bfc0440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:45:35.828 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:35.821+0000 7fcfba7fc640 1 --2- 172.21.15.39:0/3345941510 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56020bbf10a0 0x56020bfc0440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:35.828 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:35.821+0000 7fcfba7fc640 1 -- 172.21.15.39:0/3345941510 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56020bc85540 con 0x56020bbf9280 2024-09-21T03:45:35.828 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:35.821+0000 7fcfa3fff640 1 --2- 172.21.15.39:0/3682034190 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7fcfa40b65f0 0x7fcfa40b8b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:35.828 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:35.821+0000 7fcfbaffd640 1 --2- 172.21.15.39:0/3682034190 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7fcfa40b65f0 0x7fcfa40b8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:35.828 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:35.821+0000 7fcfba7fc640 1 --2- 172.21.15.39:0/3345941510 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56020bbf9280 0x56020bfc0980 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7fcfb000ae20 tx=0x7fcfb000a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=12ff619eacc6a874 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:45:35.829 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:35.821+0000 7fcfa3fff640 1 -- 172.21.15.39:0/3682034190 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8484+0+0 (secure 0 0 0) 0x7fcfb40e7c80 con 0x7fcfbc108cf0 2024-09-21T03:45:35.829 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:35.821+0000 7fcfa17fa640 1 -- 172.21.15.39:0/3345941510 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfb0005020 con 0x56020bbf9280 2024-09-21T03:45:35.829 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:35.821+0000 7fcfa17fa640 1 -- 172.21.15.39:0/3345941510 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcfb0017030 con 0x56020bbf9280 2024-09-21T03:45:35.829 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:35.821+0000 7fcfa17fa640 1 -- 172.21.15.39:0/3345941510 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfb000f450 con 0x56020bbf9280 2024-09-21T03:45:35.829 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:35.822+0000 7fcfc784e740 1 -- 172.21.15.39:0/3345941510 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56020bbf9280 msgr2=0x56020bfc0980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:35.829 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:35.822+0000 7fcfbaffd640 1 --2- 172.21.15.39:0/3682034190 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7fcfa40b65f0 0x7fcfa40b8b00 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fcfac000c00 tx=0x7fcfac009640 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:45:35.829 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:35.822+0000 7fcfc784e740 1 --2- 172.21.15.39:0/3345941510 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56020bbf9280 0x56020bfc0980 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7fcfb000ae20 tx=0x7fcfb000a710 comp rx=0 tx=0).stop 2024-09-21T03:45:35.829 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:35.822+0000 7fcfc784e740 1 -- 172.21.15.39:0/3345941510 shutdown_connections 2024-09-21T03:45:35.829 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:35.822+0000 7fcfc784e740 1 --2- 172.21.15.39:0/3345941510 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56020bbf9280 0x56020bfc0980 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:35.829 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:35.822+0000 7fcfc784e740 1 --2- 172.21.15.39:0/3345941510 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56020bbf10a0 0x56020bfc0440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:35.829 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:35.822+0000 7fcfc784e740 1 --2- 172.21.15.39:0/3345941510 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56020bc19f60 0x56020bfbff00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:35.829 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:35.822+0000 7fcfc784e740 1 -- 172.21.15.39:0/3345941510 >> 172.21.15.39:0/3345941510 conn(0x56020bc0e350 msgr2=0x56020bfbddf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:45:35.829 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:35.822+0000 7fcfc784e740 1 -- 172.21.15.39:0/3345941510 shutdown_connections 2024-09-21T03:45:35.829 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:35.822+0000 7fcfc784e740 1 -- 172.21.15.39:0/3345941510 wait complete. 2024-09-21T03:45:35.830 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:35.823+0000 7fcfc784e740 1 -- start start 2024-09-21T03:45:35.830 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:35.823+0000 7fcfc784e740 1 Processor -- start 2024-09-21T03:45:35.833 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:35.823+0000 7fcfc784e740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56020bbf10a0 0x56020bfcb740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:35.833 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:35.823+0000 7fcfc784e740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56020bbf9280 0x56020bfcbd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:35.833 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:35.823+0000 7fcfc784e740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56020bc19f60 0x56020bfd79d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:35.833 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:35.823+0000 7fcfc784e740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x56020bd885f0 con 0x56020bbf10a0 2024-09-21T03:45:35.833 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:35.823+0000 7fcfc784e740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x56020bec2010 con 0x56020bc19f60 2024-09-21T03:45:35.833 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:35.823+0000 7fcfc784e740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x56020bfce6e0 con 0x56020bbf9280 2024-09-21T03:45:35.833 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:35.823+0000 7fcfbaffd640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56020bbf10a0 0x56020bfcb740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:35.833 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:35.823+0000 7fcfc784e740 10 client.0 _ll_register_callbacks cb 0x56020bf21340 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:45:35.833 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:35.823+0000 7fcfbaffd640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56020bbf10a0 0x56020bfcb740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:40678/0 (socket says 172.21.15.39:40678) 2024-09-21T03:45:35.833 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:35.823+0000 7fcfbaffd640 1 -- 172.21.15.39:0/2353709326 learned_addr learned my addr 172.21.15.39:0/2353709326 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:45:35.833 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:35.823+0000 7fcfba7fc640 1 --2- 172.21.15.39:0/2353709326 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56020bbf9280 0x56020bfcbd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:35.833 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:35.823+0000 7fcfbb7fe640 1 --2- 172.21.15.39:0/2353709326 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56020bc19f60 0x56020bfd79d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:35.834 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:35.824+0000 7fcfbaffd640 1 -- 172.21.15.39:0/2353709326 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56020bbf9280 msgr2=0x56020bfcbd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:45:35.834 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:35.824+0000 7fcfbaffd640 1 --2- 172.21.15.39:0/2353709326 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56020bbf9280 0x56020bfcbd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:35.834 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:35.824+0000 7fcfbaffd640 1 -- 172.21.15.39:0/2353709326 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56020bc19f60 msgr2=0x56020bfd79d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:45:35.834 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:35.824+0000 7fcfbaffd640 1 --2- 172.21.15.39:0/2353709326 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56020bc19f60 0x56020bfd79d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:35.834 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:35.824+0000 7fcfbaffd640 1 -- 172.21.15.39:0/2353709326 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56020beefa00 con 0x56020bbf10a0 2024-09-21T03:45:35.834 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:35.824+0000 7fcfba7fc640 1 --2- 172.21.15.39:0/2353709326 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56020bbf9280 0x56020bfcbd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:45:35.834 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:35.824+0000 7fcfbaffd640 1 --2- 172.21.15.39:0/2353709326 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56020bbf10a0 0x56020bfcb740 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7fcfac009900 tx=0x7fcfac009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f5c81905f0a91bfb server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:45:35.834 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:35.824+0000 7fcf9eff5640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-09-21T03:45:35.834 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:35.824+0000 7fcf9eff5640 1 -- 172.21.15.39:0/2353709326 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfac04c070 con 0x56020bbf10a0 2024-09-21T03:45:35.834 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:35.824+0000 7fcf9eff5640 1 -- 172.21.15.39:0/2353709326 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8484+0+0 (secure 0 0 0) 0x7fcfac004c50 con 0x56020bbf10a0 2024-09-21T03:45:35.834 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:35.824+0000 7fcfc784e740 1 -- 172.21.15.39:0/2353709326 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcfb0017030 con 0x56020bbf10a0 2024-09-21T03:45:35.834 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:35.824+0000 7fcf9eff5640 1 -- 172.21.15.39:0/2353709326 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fcfac058050 con 0x56020bbf10a0 2024-09-21T03:45:35.834 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:35.824+0000 7fcfc784e740 10 client.15270 fetch_fsmap learned FSMap version 28 2024-09-21T03:45:35.835 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:35.824+0000 7fcfc784e740 1 -- 172.21.15.39:0/2353709326 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x56020bfd9810 con 0x56020bbf10a0 2024-09-21T03:45:35.835 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:35.825+0000 7fcf9eff5640 1 -- 172.21.15.39:0/2353709326 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcfb0017030 con 0x56020bbf10a0 2024-09-21T03:45:35.835 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:35.825+0000 7fcf9eff5640 1 -- 172.21.15.39:0/2353709326 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fcfac032560 con 0x56020bbf10a0 2024-09-21T03:45:35.835 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:35.825+0000 7fcfc784e740 10 client.15270 fetch_fsmap finished waiting for FSMap version 28 2024-09-21T03:45:35.835 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:35.825+0000 7fcfc784e740 10 client.15270 resolve_mds: resolved name 'h' to daemon mds.h 2024-09-21T03:45:35.835 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:35.825+0000 7fcfc784e740 20 client.15270 populate_metadata read hostname 'smithi039' 2024-09-21T03:45:35.835 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:35.825+0000 7fcfc784e740 1 --2- 172.21.15.39:0/2353709326 >> [v2:172.21.15.150:6836/4017043144,v1:172.21.15.150:6837/4017043144] conn(0x56020bbd88e0 0x56020bfe2420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:35.835 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:35.825+0000 7fcfc784e740 4 client.15270 mds_command: new command op to 24659 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:45:35.835 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:35.825+0000 7fcfc784e740 1 -- 172.21.15.39:0/2353709326 --> [v2:172.21.15.150:6836/4017043144,v1:172.21.15.150:6837/4017043144] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56020bf39b60 con 0x56020bbd88e0 2024-09-21T03:45:35.835 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:35.825+0000 7fcfba7fc640 1 --2- 172.21.15.39:0/2353709326 >> [v2:172.21.15.150:6836/4017043144,v1:172.21.15.150:6837/4017043144] conn(0x56020bbd88e0 0x56020bfe2420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:35.835 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:35.826+0000 7fcfba7fc640 1 --2- 172.21.15.39:0/2353709326 >> [v2:172.21.15.150:6836/4017043144,v1:172.21.15.150:6837/4017043144] conn(0x56020bbd88e0 0x56020bfe2420 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=c2888bfd9b1e87cf server_cookie=bbbf74d8be259f80 in_seq=0 out_seq=0 2024-09-21T03:45:35.835 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:35.826+0000 7fcf9eff5640 10 client.15270 ms_handle_connect on v2:172.21.15.150:6836/4017043144 2024-09-21T03:45:35.835 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:35.826+0000 7fcf9eff5640 1 -- 172.21.15.39:0/2353709326 <== mds.2 v2:172.21.15.150:6836/4017043144 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x56020bf39b60 con 0x56020bbd88e0 2024-09-21T03:45:35.835 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:35.826+0000 7fcf9eff5640 10 client.15270 handle_command_reply: tid=0 2024-09-21T03:45:35.836 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:35.827+0000 7fcfc784e740 1 client.15270 shutdown 2024-09-21T03:45:35.836 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:35.827+0000 7fcfc784e740 1 -- 172.21.15.39:0/2353709326 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56020bbf10a0 msgr2=0x56020bfcb740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:35.836 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:35.827+0000 7fcfc784e740 1 --2- 172.21.15.39:0/2353709326 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56020bbf10a0 0x56020bfcb740 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7fcfac009900 tx=0x7fcfac009260 comp rx=0 tx=0).stop 2024-09-21T03:45:35.836 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:35.827+0000 7fcfc784e740 1 -- 172.21.15.39:0/2353709326 shutdown_connections 2024-09-21T03:45:35.836 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:35.827+0000 7fcfc784e740 1 --2- 172.21.15.39:0/2353709326 >> [v2:172.21.15.150:6836/4017043144,v1:172.21.15.150:6837/4017043144] conn(0x56020bbd88e0 0x56020bfe2420 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:35.836 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:35.827+0000 7fcfc784e740 1 --2- 172.21.15.39:0/2353709326 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56020bc19f60 0x56020bfd79d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:35.836 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:35.827+0000 7fcfc784e740 1 --2- 172.21.15.39:0/2353709326 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56020bbf9280 0x56020bfcbd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:35.836 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:35.827+0000 7fcfc784e740 1 --2- 172.21.15.39:0/2353709326 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56020bbf10a0 0x56020bfcb740 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:35.836 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:35.827+0000 7fcfc784e740 1 -- 172.21.15.39:0/2353709326 >> 172.21.15.39:0/2353709326 conn(0x56020bc0e350 msgr2=0x56020bfc3090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:45:35.836 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:35.827+0000 7fcf9eff5640 0 client.15270 ms_handle_reset on v2:172.21.15.150:6836/4017043144 2024-09-21T03:45:35.836 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:35.827+0000 7fcfc784e740 1 -- 172.21.15.39:0/2353709326 shutdown_connections 2024-09-21T03:45:35.836 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:35.827+0000 7fcfc784e740 1 -- 172.21.15.39:0/2353709326 wait complete. 2024-09-21T03:45:35.836 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:35.827+0000 7fcfc784e740 20 client.15270 trim_cache size 0 max 16384 2024-09-21T03:45:35.847 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:35.845+0000 7fcfc784e740 1 Processor -- start 2024-09-21T03:45:35.848 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:35.845+0000 7fcfc784e740 1 -- start start 2024-09-21T03:45:35.848 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:35.845+0000 7fcfc784e740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56020bbf10a0 0x56020bfd9b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:35.848 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:35.845+0000 7fcfc784e740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56020bbf9280 0x56020bfda040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:35.848 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:35.845+0000 7fcfc784e740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56020bc19f60 0x56020bfc68f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:35.848 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:35.845+0000 7fcfc784e740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x56020bfe0760 con 0x56020bbf10a0 2024-09-21T03:45:35.848 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:35.845+0000 7fcfc784e740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x56020bfe0e30 con 0x56020bc19f60 2024-09-21T03:45:35.848 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:35.845+0000 7fcfc784e740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x56020bfe1500 con 0x56020bbf9280 2024-09-21T03:45:35.848 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:35.845+0000 7fcfbaffd640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56020bbf10a0 0x56020bfd9b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:35.848 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:35.845+0000 7fcfbaffd640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56020bbf10a0 0x56020bfd9b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:40688/0 (socket says 172.21.15.39:40688) 2024-09-21T03:45:35.848 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:35.845+0000 7fcfbaffd640 1 -- 172.21.15.39:0/188281445 learned_addr learned my addr 172.21.15.39:0/188281445 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:45:35.848 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:35.846+0000 7fcfba7fc640 1 --2- 172.21.15.39:0/188281445 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56020bbf9280 0x56020bfda040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:35.848 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:35.846+0000 7fcfbaffd640 1 -- 172.21.15.39:0/188281445 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56020bbf9280 msgr2=0x56020bfda040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:45:35.849 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:35.846+0000 7fcfbaffd640 1 --2- 172.21.15.39:0/188281445 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56020bbf9280 0x56020bfda040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:35.849 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:35.846+0000 7fcfbaffd640 1 -- 172.21.15.39:0/188281445 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56020bc19f60 msgr2=0x56020bfc68f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:45:35.849 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:35.846+0000 7fcfbaffd640 1 --2- 172.21.15.39:0/188281445 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56020bc19f60 0x56020bfc68f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:35.849 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:35.846+0000 7fcfbaffd640 1 -- 172.21.15.39:0/188281445 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56020bfb8080 con 0x56020bbf10a0 2024-09-21T03:45:35.849 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:35.846+0000 7fcfba7fc640 1 --2- 172.21.15.39:0/188281445 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56020bbf9280 0x56020bfda040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:45:35.849 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:35.846+0000 7fcfbaffd640 1 --2- 172.21.15.39:0/188281445 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56020bbf10a0 0x56020bfd9b00 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7fcfac032b20 tx=0x7fcfac0097c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af6c3a759735c798 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:45:35.849 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:35.846+0000 7fcfa0ff9640 1 -- 172.21.15.39:0/188281445 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfac04c070 con 0x56020bbf10a0 2024-09-21T03:45:35.849 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:35.846+0000 7fcfa0ff9640 1 -- 172.21.15.39:0/188281445 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcfac061070 con 0x56020bbf10a0 2024-09-21T03:45:35.849 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:35.846+0000 7fcfa0ff9640 1 -- 172.21.15.39:0/188281445 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfac035050 con 0x56020bbf10a0 2024-09-21T03:45:35.853 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:35.847+0000 7fcfc784e740 1 -- 172.21.15.39:0/188281445 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56020bbf10a0 msgr2=0x56020bfd9b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:35.853 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:35.847+0000 7fcfc784e740 1 --2- 172.21.15.39:0/188281445 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56020bbf10a0 0x56020bfd9b00 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7fcfac032b20 tx=0x7fcfac0097c0 comp rx=0 tx=0).stop 2024-09-21T03:45:35.853 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:35.847+0000 7fcfc784e740 1 -- 172.21.15.39:0/188281445 shutdown_connections 2024-09-21T03:45:35.853 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:35.847+0000 7fcfc784e740 1 --2- 172.21.15.39:0/188281445 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56020bc19f60 0x56020bfc68f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:35.853 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:35.847+0000 7fcfc784e740 1 --2- 172.21.15.39:0/188281445 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56020bbf9280 0x56020bfda040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:35.853 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:35.847+0000 7fcfc784e740 1 --2- 172.21.15.39:0/188281445 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56020bbf10a0 0x56020bfd9b00 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:35.853 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:35.847+0000 7fcfc784e740 1 -- 172.21.15.39:0/188281445 >> 172.21.15.39:0/188281445 conn(0x56020bc0e350 msgr2=0x56020bfcb4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:45:35.853 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:35.847+0000 7fcfc784e740 1 -- 172.21.15.39:0/188281445 shutdown_connections 2024-09-21T03:45:35.853 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:35.847+0000 7fcfc784e740 1 -- 172.21.15.39:0/188281445 wait complete. 2024-09-21T03:45:35.853 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:35.847+0000 7fcfc784e740 1 -- start start 2024-09-21T03:45:35.853 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:35.847+0000 7fcfc784e740 1 Processor -- start 2024-09-21T03:45:35.854 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:35.848+0000 7fcfc784e740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56020bbf10a0 0x56020bfe4c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:35.854 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:35.848+0000 7fcfc784e740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56020bbf9280 0x56020bfe5140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:35.854 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:35.848+0000 7fcfc784e740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56020bc19f60 0x56020bfe5680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:35.854 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:35.848+0000 7fcfc784e740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x56020bfe22a0 con 0x56020bbf9280 2024-09-21T03:45:35.854 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:35.848+0000 7fcfc784e740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x56020bfe1bd0 con 0x56020bbf10a0 2024-09-21T03:45:35.854 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:35.848+0000 7fcfc784e740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x56020bfe2970 con 0x56020bc19f60 2024-09-21T03:45:35.854 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:35.848+0000 7fcfc784e740 10 client.0 _ll_register_callbacks cb 0x7fcfb40093a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:45:35.854 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:35.848+0000 7fcfba7fc640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56020bbf9280 0x56020bfe5140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:35.854 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:35.848+0000 7fcfba7fc640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56020bbf9280 0x56020bfe5140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:40694/0 (socket says 172.21.15.39:40694) 2024-09-21T03:45:35.854 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:35.848+0000 7fcfba7fc640 1 -- 172.21.15.39:0/240783456 learned_addr learned my addr 172.21.15.39:0/240783456 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:45:35.854 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:35.848+0000 7fcfbb7fe640 1 --2- 172.21.15.39:0/240783456 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56020bc19f60 0x56020bfe5680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:35.854 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:35.848+0000 7fcfbaffd640 1 --2- 172.21.15.39:0/240783456 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56020bbf10a0 0x56020bfe4c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:35.854 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:35.848+0000 7fcfba7fc640 1 -- 172.21.15.39:0/240783456 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56020bc19f60 msgr2=0x56020bfe5680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:45:35.855 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:35.848+0000 7fcfba7fc640 1 --2- 172.21.15.39:0/240783456 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56020bc19f60 0x56020bfe5680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:35.855 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:35.848+0000 7fcfba7fc640 1 -- 172.21.15.39:0/240783456 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56020bbf10a0 msgr2=0x56020bfe4c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:45:35.855 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:35.848+0000 7fcfba7fc640 1 --2- 172.21.15.39:0/240783456 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56020bbf10a0 0x56020bfe4c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:35.855 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:35.848+0000 7fcfba7fc640 1 -- 172.21.15.39:0/240783456 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56020bfe9d50 con 0x56020bbf9280 2024-09-21T03:45:35.855 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:35.848+0000 7fcfba7fc640 1 --2- 172.21.15.39:0/240783456 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56020bbf9280 0x56020bfe5140 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7fcfb0000c00 tx=0x7fcfb0004b10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a8ecc7a7fea8a46 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:45:35.855 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:35.848+0000 7fcf9eff5640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-09-21T03:45:35.855 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:35.849+0000 7fcf9eff5640 1 -- 172.21.15.39:0/240783456 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfb000a8e0 con 0x56020bbf9280 2024-09-21T03:45:35.855 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:35.849+0000 7fcf9eff5640 1 -- 172.21.15.39:0/240783456 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8484+0+0 (secure 0 0 0) 0x7fcfb0004680 con 0x56020bbf9280 2024-09-21T03:45:35.855 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:35.849+0000 7fcfc784e740 1 -- 172.21.15.39:0/240783456 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcfac061070 con 0x56020bbf9280 2024-09-21T03:45:35.855 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:35.849+0000 7fcf9eff5640 1 -- 172.21.15.39:0/240783456 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fcfb0036050 con 0x56020bbf9280 2024-09-21T03:45:35.855 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:35.849+0000 7fcfc784e740 10 client.15276 fetch_fsmap learned FSMap version 28 2024-09-21T03:45:35.855 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:35.849+0000 7fcfc784e740 1 -- 172.21.15.39:0/240783456 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x56020bfeaf20 con 0x56020bbf9280 2024-09-21T03:45:35.855 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:35.849+0000 7fcf9eff5640 1 -- 172.21.15.39:0/240783456 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcfac061070 con 0x56020bbf9280 2024-09-21T03:45:35.855 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:35.849+0000 7fcf9eff5640 1 -- 172.21.15.39:0/240783456 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fcfb0026070 con 0x56020bbf9280 2024-09-21T03:45:35.856 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:35.849+0000 7fcfc784e740 10 client.15276 fetch_fsmap finished waiting for FSMap version 28 2024-09-21T03:45:35.856 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:35.849+0000 7fcfc784e740 10 client.15276 resolve_mds: resolved name 'h' to daemon mds.h 2024-09-21T03:45:35.856 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:35.849+0000 7fcfc784e740 20 client.15276 populate_metadata read hostname 'smithi039' 2024-09-21T03:45:35.856 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:35.849+0000 7fcfc784e740 1 --2- 172.21.15.39:0/240783456 >> [v2:172.21.15.150:6836/4017043144,v1:172.21.15.150:6837/4017043144] conn(0x56020bbadc90 0x56020c0506c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:35.856 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:35.849+0000 7fcfc784e740 4 client.15276 mds_command: new command op to 24659 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-21T03:45:35.856 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:35.849+0000 7fcfc784e740 1 -- 172.21.15.39:0/240783456 --> [v2:172.21.15.150:6836/4017043144,v1:172.21.15.150:6837/4017043144] -- command(tid 0: {"prefix": "status"}) -- 0x56020bf35610 con 0x56020bbadc90 2024-09-21T03:45:35.856 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:35.850+0000 7fcfbaffd640 1 --2- 172.21.15.39:0/240783456 >> [v2:172.21.15.150:6836/4017043144,v1:172.21.15.150:6837/4017043144] conn(0x56020bbadc90 0x56020c0506c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:35.856 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:35.850+0000 7fcfbaffd640 1 --2- 172.21.15.39:0/240783456 >> [v2:172.21.15.150:6836/4017043144,v1:172.21.15.150:6837/4017043144] conn(0x56020bbadc90 0x56020c0506c0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=723666a65a429d07 server_cookie=6a16f5c1ea670525 in_seq=0 out_seq=0 2024-09-21T03:45:35.856 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:35.850+0000 7fcf9eff5640 10 client.15276 ms_handle_connect on v2:172.21.15.150:6836/4017043144 2024-09-21T03:45:35.856 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:35.850+0000 7fcf9eff5640 1 -- 172.21.15.39:0/240783456 <== mds.2 v2:172.21.15.150:6836/4017043144 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x56020bf35610 con 0x56020bbadc90 2024-09-21T03:45:35.856 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:35.850+0000 7fcf9eff5640 10 client.15276 handle_command_reply: tid=0 2024-09-21T03:45:35.856 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:35.850+0000 7fcfc784e740 1 client.15276 shutdown 2024-09-21T03:45:35.856 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:35.851+0000 7fcfc784e740 1 -- 172.21.15.39:0/240783456 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56020bbf9280 msgr2=0x56020bfe5140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:35.856 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:35.851+0000 7fcfc784e740 1 --2- 172.21.15.39:0/240783456 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56020bbf9280 0x56020bfe5140 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7fcfb0000c00 tx=0x7fcfb0004b10 comp rx=0 tx=0).stop 2024-09-21T03:45:35.857 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:35.851+0000 7fcfc784e740 1 -- 172.21.15.39:0/240783456 shutdown_connections 2024-09-21T03:45:35.857 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:35.851+0000 7fcfc784e740 1 --2- 172.21.15.39:0/240783456 >> [v2:172.21.15.150:6836/4017043144,v1:172.21.15.150:6837/4017043144] conn(0x56020bbadc90 0x56020c0506c0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:35.857 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:35.851+0000 7fcfc784e740 1 --2- 172.21.15.39:0/240783456 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56020bc19f60 0x56020bfe5680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:35.857 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:35.851+0000 7fcfc784e740 1 --2- 172.21.15.39:0/240783456 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56020bbf9280 0x56020bfe5140 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:35.857 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:35.851+0000 7fcfc784e740 1 --2- 172.21.15.39:0/240783456 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56020bbf10a0 0x56020bfe4c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:35.857 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:35.851+0000 7fcfc784e740 1 -- 172.21.15.39:0/240783456 >> 172.21.15.39:0/240783456 conn(0x56020bc0e350 msgr2=0x56020bfd9aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:45:35.857 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:35.851+0000 7fcf9eff5640 0 client.15276 ms_handle_reset on v2:172.21.15.150:6836/4017043144 2024-09-21T03:45:35.857 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:35.851+0000 7fcfc784e740 1 -- 172.21.15.39:0/240783456 shutdown_connections 2024-09-21T03:45:35.857 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:35.851+0000 7fcfc784e740 1 -- 172.21.15.39:0/240783456 wait complete. 2024-09-21T03:45:35.857 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:35.851+0000 7fcfc784e740 20 client.15276 trim_cache size 0 max 16384 2024-09-21T03:45:35.857 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-09-21T03:45:35.857 INFO:teuthology.orchestra.run.smithi039.stdout: "cluster_fsid": "51116d5c-77ca-11ef-baf5-efdc52797490", 2024-09-21T03:45:35.858 INFO:teuthology.orchestra.run.smithi039.stdout: "whoami": 2, 2024-09-21T03:45:35.858 INFO:teuthology.orchestra.run.smithi039.stdout: "id": 24659, 2024-09-21T03:45:35.858 INFO:teuthology.orchestra.run.smithi039.stdout: "want_state": "up:active", 2024-09-21T03:45:35.858 INFO:teuthology.orchestra.run.smithi039.stdout: "state": "up:active", 2024-09-21T03:45:35.858 INFO:teuthology.orchestra.run.smithi039.stdout: "fs_name": "cephfs", 2024-09-21T03:45:35.858 INFO:teuthology.orchestra.run.smithi039.stdout: "rank_uptime": 72.473745862000001, 2024-09-21T03:45:35.858 INFO:teuthology.orchestra.run.smithi039.stdout: "mdsmap_epoch": 28, 2024-09-21T03:45:35.858 INFO:teuthology.orchestra.run.smithi039.stdout: "osdmap_epoch": 79, 2024-09-21T03:45:35.858 INFO:teuthology.orchestra.run.smithi039.stdout: "osdmap_epoch_barrier": 78, 2024-09-21T03:45:35.858 INFO:teuthology.orchestra.run.smithi039.stdout: "uptime": 102.20671441899999 2024-09-21T03:45:35.858 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-09-21T03:45:35.859 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:35.851+0000 7fcfc12da640 1 -- 172.21.15.39:0/3682034190 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7fcfa40b65f0 msgr2=0x7fcfa40b8b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:35.859 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:35.851+0000 7fcfc12da640 1 --2- 172.21.15.39:0/3682034190 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7fcfa40b65f0 0x7fcfa40b8b00 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fcfac000c00 tx=0x7fcfac009640 comp rx=0 tx=0).stop 2024-09-21T03:45:35.859 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:35.851+0000 7fcfc12da640 1 -- 172.21.15.39:0/3682034190 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcfbc108cf0 msgr2=0x7fcfbc19ebc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:35.859 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:35.851+0000 7fcfc12da640 1 --2- 172.21.15.39:0/3682034190 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcfbc108cf0 0x7fcfbc19ebc0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7fcfb400eb10 tx=0x7fcfb400c2d0 comp rx=0 tx=0).stop 2024-09-21T03:45:35.859 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:35.851+0000 7fcfc12da640 1 -- 172.21.15.39:0/3682034190 shutdown_connections 2024-09-21T03:45:35.859 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:35.852+0000 7fcfc12da640 1 --2- 172.21.15.39:0/3682034190 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7fcfa40b65f0 0x7fcfa40b8b00 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:35.860 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:35.852+0000 7fcfc12da640 1 --2- 172.21.15.39:0/3682034190 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcfbc108cf0 0x7fcfbc19ebc0 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:35.860 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:35.852+0000 7fcfc12da640 1 --2- 172.21.15.39:0/3682034190 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcfbc1083a0 0x7fcfbc199920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:35.860 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:35.852+0000 7fcfc12da640 1 --2- 172.21.15.39:0/3682034190 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcfbc102a60 0x7fcfbc199380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:35.860 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:35.852+0000 7fcfc12da640 1 -- 172.21.15.39:0/3682034190 >> 172.21.15.39:0/3682034190 conn(0x7fcfbc079730 msgr2=0x7fcfbc101370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:45:35.860 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:35.852+0000 7fcfc12da640 1 -- 172.21.15.39:0/3682034190 shutdown_connections 2024-09-21T03:45:35.860 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:35.852+0000 7fcfc12da640 1 -- 172.21.15.39:0/3682034190 wait complete. 2024-09-21T03:45:36.539 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 51116d5c-77ca-11ef-baf5-efdc52797490 -- ceph tell mds.i status 2024-09-21T03:45:36.795 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/51116d5c-77ca-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:45:37.207 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:45:36 smithi039 ceph-mon[31211]: pgmap v312: 97 pgs: 97 active+clean; 647 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 12 op/s 2024-09-21T03:45:37.217 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:45:36 smithi161 ceph-mon[34533]: pgmap v312: 97 pgs: 97 active+clean; 647 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 12 op/s 2024-09-21T03:45:37.224 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:45:36 smithi150 ceph-mon[34289]: pgmap v312: 97 pgs: 97 active+clean; 647 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 12 op/s 2024-09-21T03:45:37.315 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:37.297+0000 7f524d1ec640 1 -- 172.21.15.39:0/1713651197 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f524810ddf0 msgr2=0x7f5248110300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:37.315 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:37.297+0000 7f524d1ec640 1 --2- 172.21.15.39:0/1713651197 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f524810ddf0 0x7f5248110300 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f5240009ca0 tx=0x7f524002a8c0 comp rx=0 tx=0).stop 2024-09-21T03:45:37.315 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:37.298+0000 7f524d1ec640 1 -- 172.21.15.39:0/1713651197 shutdown_connections 2024-09-21T03:45:37.315 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:37.298+0000 7f524d1ec640 1 --2- 172.21.15.39:0/1713651197 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f524810ddf0 0x7f5248110300 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:37.316 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:37.298+0000 7f524d1ec640 1 --2- 172.21.15.39:0/1713651197 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5248105270 0x7f524810d8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:37.316 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:37.298+0000 7f524d1ec640 1 --2- 172.21.15.39:0/1713651197 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5248104840 0x7f5248104c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:37.316 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:37.298+0000 7f524d1ec640 1 -- 172.21.15.39:0/1713651197 >> 172.21.15.39:0/1713651197 conn(0x7f5248079730 msgr2=0x7f5248079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:45:37.316 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:37.298+0000 7f524d1ec640 1 -- 172.21.15.39:0/1713651197 shutdown_connections 2024-09-21T03:45:37.316 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:37.298+0000 7f524d1ec640 1 -- 172.21.15.39:0/1713651197 wait complete. 2024-09-21T03:45:37.316 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:37.298+0000 7f524d1ec640 1 Processor -- start 2024-09-21T03:45:37.316 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:37.299+0000 7f524d1ec640 1 -- start start 2024-09-21T03:45:37.316 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:37.299+0000 7f524d1ec640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5248104840 0x7f524819a3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:37.316 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:37.299+0000 7f524d1ec640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5248105270 0x7f524819a980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:37.317 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:37.299+0000 7f524d1ec640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f524810ddf0 0x7f524819fc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:37.317 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:37.299+0000 7f524d1ec640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f5248113a90 con 0x7f5248104840 2024-09-21T03:45:37.317 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:37.299+0000 7f524d1ec640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f5248113910 con 0x7f5248105270 2024-09-21T03:45:37.317 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:37.299+0000 7f524d1ec640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f5248113c10 con 0x7f524810ddf0 2024-09-21T03:45:37.317 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:37.299+0000 7f5246d9d640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5248104840 0x7f524819a3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:37.317 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:37.299+0000 7f5246d9d640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5248104840 0x7f524819a3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:40708/0 (socket says 172.21.15.39:40708) 2024-09-21T03:45:37.317 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:37.299+0000 7f5246d9d640 1 -- 172.21.15.39:0/3231983357 learned_addr learned my addr 172.21.15.39:0/3231983357 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:45:37.317 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:37.299+0000 7f524759e640 1 --2- 172.21.15.39:0/3231983357 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f524810ddf0 0x7f524819fc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:37.317 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:37.299+0000 7f524659c640 1 --2- 172.21.15.39:0/3231983357 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5248105270 0x7f524819a980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:37.317 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:37.299+0000 7f5246d9d640 1 -- 172.21.15.39:0/3231983357 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f524810ddf0 msgr2=0x7f524819fc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:37.317 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:37.299+0000 7f5246d9d640 1 --2- 172.21.15.39:0/3231983357 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f524810ddf0 0x7f524819fc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:37.317 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:37.299+0000 7f5246d9d640 1 -- 172.21.15.39:0/3231983357 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5248105270 msgr2=0x7f524819a980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:37.317 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:37.299+0000 7f5246d9d640 1 --2- 172.21.15.39:0/3231983357 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5248105270 0x7f524819a980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:37.318 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:37.299+0000 7f5246d9d640 1 -- 172.21.15.39:0/3231983357 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f52481a0350 con 0x7f5248104840 2024-09-21T03:45:37.318 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:37.299+0000 7f524659c640 1 --2- 172.21.15.39:0/3231983357 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5248105270 0x7f524819a980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:45:37.318 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:37.300+0000 7f5246d9d640 1 --2- 172.21.15.39:0/3231983357 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5248104840 0x7f524819a3e0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f523800d950 tx=0x7f523800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:45:37.318 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:37.300+0000 7f522ffff640 1 -- 172.21.15.39:0/3231983357 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5238017070 con 0x7f5248104840 2024-09-21T03:45:37.318 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:37.300+0000 7f524d1ec640 1 -- 172.21.15.39:0/3231983357 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f52481a0640 con 0x7f5248104840 2024-09-21T03:45:37.318 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:37.300+0000 7f522ffff640 1 -- 172.21.15.39:0/3231983357 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5238009c30 con 0x7f5248104840 2024-09-21T03:45:37.318 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:37.300+0000 7f522ffff640 1 -- 172.21.15.39:0/3231983357 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f523800f620 con 0x7f5248104840 2024-09-21T03:45:37.318 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:37.300+0000 7f524d1ec640 1 -- 172.21.15.39:0/3231983357 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f52481a6da0 con 0x7f5248104840 2024-09-21T03:45:37.318 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:37.303+0000 7f5253760740 1 Processor -- start 2024-09-21T03:45:37.318 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:37.303+0000 7f522ffff640 1 -- 172.21.15.39:0/3231983357 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 22) ==== 154580+0+0 (secure 0 0 0) 0x7f523801a070 con 0x7f5248104840 2024-09-21T03:45:37.318 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:37.303+0000 7f5253760740 1 -- start start 2024-09-21T03:45:37.318 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:37.303+0000 7f5253760740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5600f367ff60 0x5600f3a25f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:37.318 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:37.303+0000 7f5253760740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5600f36570a0 0x5600f3a26440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:37.319 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:37.303+0000 7f5253760740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5600f365f280 0x5600f3a26980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:37.319 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:37.303+0000 7f5253760740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x5600f399dc10 con 0x5600f365f280 2024-09-21T03:45:37.319 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:37.303+0000 7f5253760740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x5600f398d040 con 0x5600f36570a0 2024-09-21T03:45:37.319 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:37.303+0000 7f5253760740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x5600f3990e50 con 0x5600f367ff60 2024-09-21T03:45:37.319 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:37.303+0000 7f524659c640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5600f365f280 0x5600f3a26980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:37.319 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:37.303+0000 7f524659c640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5600f365f280 0x5600f3a26980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:40712/0 (socket says 172.21.15.39:40712) 2024-09-21T03:45:37.319 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:37.303+0000 7f524659c640 1 -- 172.21.15.39:0/200848208 learned_addr learned my addr 172.21.15.39:0/200848208 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:45:37.319 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:37.304+0000 7f5246d9d640 1 --2- 172.21.15.39:0/200848208 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5600f36570a0 0x5600f3a26440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:37.319 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:37.304+0000 7f524759e640 1 --2- 172.21.15.39:0/200848208 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5600f367ff60 0x5600f3a25f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:37.319 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:37.304+0000 7f524659c640 1 -- 172.21.15.39:0/200848208 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5600f367ff60 msgr2=0x5600f3a25f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:45:37.319 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:37.304+0000 7f524659c640 1 --2- 172.21.15.39:0/200848208 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5600f367ff60 0x5600f3a25f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:37.319 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:37.304+0000 7f524659c640 1 -- 172.21.15.39:0/200848208 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5600f36570a0 msgr2=0x5600f3a26440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:45:37.319 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:37.304+0000 7f524659c640 1 --2- 172.21.15.39:0/200848208 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5600f36570a0 0x5600f3a26440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:37.319 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:37.304+0000 7f524659c640 1 -- 172.21.15.39:0/200848208 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5600f36eb540 con 0x5600f365f280 2024-09-21T03:45:37.320 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:37.304+0000 7f522ffff640 1 --2- 172.21.15.39:0/3231983357 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f52300b65f0 0x7f52300b8b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:37.320 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:37.304+0000 7f524759e640 1 --2- 172.21.15.39:0/3231983357 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f52300b65f0 0x7f52300b8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:37.320 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:37.304+0000 7f524659c640 1 --2- 172.21.15.39:0/200848208 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5600f365f280 0x5600f3a26980 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f523c00bcd0 tx=0x7f523c00f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e4ae7a4763179f5 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:45:37.320 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:37.304+0000 7f522ffff640 1 -- 172.21.15.39:0/3231983357 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8484+0+0 (secure 0 0 0) 0x7f52380e70a0 con 0x7f5248104840 2024-09-21T03:45:37.320 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:37.304+0000 7f522d7fa640 1 -- 172.21.15.39:0/200848208 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f523c015070 con 0x5600f365f280 2024-09-21T03:45:37.320 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:37.304+0000 7f522d7fa640 1 -- 172.21.15.39:0/200848208 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f523c01a910 con 0x5600f365f280 2024-09-21T03:45:37.320 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:37.304+0000 7f522d7fa640 1 -- 172.21.15.39:0/200848208 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f523c0119a0 con 0x5600f365f280 2024-09-21T03:45:37.320 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:37.305+0000 7f5253760740 1 -- 172.21.15.39:0/200848208 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5600f365f280 msgr2=0x5600f3a26980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:37.320 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:37.305+0000 7f5253760740 1 --2- 172.21.15.39:0/200848208 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5600f365f280 0x5600f3a26980 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f523c00bcd0 tx=0x7f523c00f730 comp rx=0 tx=0).stop 2024-09-21T03:45:37.320 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:37.305+0000 7f524759e640 1 --2- 172.21.15.39:0/3231983357 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f52300b65f0 0x7f52300b8b00 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f5240006010 tx=0x7f5240009440 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:45:37.320 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:37.305+0000 7f5253760740 1 -- 172.21.15.39:0/200848208 shutdown_connections 2024-09-21T03:45:37.320 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:37.305+0000 7f5253760740 1 --2- 172.21.15.39:0/200848208 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5600f365f280 0x5600f3a26980 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:37.320 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:37.305+0000 7f5253760740 1 --2- 172.21.15.39:0/200848208 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5600f36570a0 0x5600f3a26440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:37.321 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:37.305+0000 7f5253760740 1 --2- 172.21.15.39:0/200848208 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5600f367ff60 0x5600f3a25f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:37.321 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:37.305+0000 7f5253760740 1 -- 172.21.15.39:0/200848208 >> 172.21.15.39:0/200848208 conn(0x5600f3674350 msgr2=0x5600f3a23df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:45:37.321 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:37.305+0000 7f5253760740 1 -- 172.21.15.39:0/200848208 shutdown_connections 2024-09-21T03:45:37.321 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:37.305+0000 7f5253760740 1 -- 172.21.15.39:0/200848208 wait complete. 2024-09-21T03:45:37.321 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:37.306+0000 7f5253760740 1 -- start start 2024-09-21T03:45:37.321 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:37.306+0000 7f5253760740 1 Processor -- start 2024-09-21T03:45:37.321 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:37.306+0000 7f5253760740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5600f36570a0 0x5600f3a31770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:37.321 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:37.306+0000 7f5253760740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5600f365f280 0x5600f3a31d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:37.321 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:37.306+0000 7f5253760740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5600f367ff60 0x5600f3a3da00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:37.321 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:37.306+0000 7f5253760740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x5600f37ee5f0 con 0x5600f36570a0 2024-09-21T03:45:37.321 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:37.306+0000 7f5253760740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x5600f3928010 con 0x5600f367ff60 2024-09-21T03:45:37.321 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:37.306+0000 7f5253760740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x5600f3a346e0 con 0x5600f365f280 2024-09-21T03:45:37.321 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:37.306+0000 7f5253760740 10 client.0 _ll_register_callbacks cb 0x5600f3987340 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:45:37.321 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:37.306+0000 7f5246d9d640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5600f36570a0 0x5600f3a31770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:37.322 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:37.306+0000 7f5246d9d640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5600f36570a0 0x5600f3a31770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:40720/0 (socket says 172.21.15.39:40720) 2024-09-21T03:45:37.322 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:37.306+0000 7f5246d9d640 1 -- 172.21.15.39:0/3021347161 learned_addr learned my addr 172.21.15.39:0/3021347161 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:45:37.322 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:37.306+0000 7f524759e640 1 --2- 172.21.15.39:0/3021347161 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5600f367ff60 0x5600f3a3da00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:37.322 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:37.307+0000 7f5246d9d640 1 -- 172.21.15.39:0/3021347161 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5600f365f280 msgr2=0x5600f3a31d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:45:37.322 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:37.307+0000 7f524659c640 1 --2- 172.21.15.39:0/3021347161 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5600f365f280 0x5600f3a31d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:37.322 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:37.307+0000 7f5246d9d640 1 --2- 172.21.15.39:0/3021347161 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5600f365f280 0x5600f3a31d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:37.322 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:37.307+0000 7f5246d9d640 1 -- 172.21.15.39:0/3021347161 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5600f367ff60 msgr2=0x5600f3a3da00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:45:37.322 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:37.307+0000 7f5246d9d640 1 --2- 172.21.15.39:0/3021347161 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5600f367ff60 0x5600f3a3da00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:37.322 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:37.307+0000 7f5246d9d640 1 -- 172.21.15.39:0/3021347161 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5600f3955a00 con 0x5600f36570a0 2024-09-21T03:45:37.322 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:37.307+0000 7f524759e640 1 --2- 172.21.15.39:0/3021347161 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5600f367ff60 0x5600f3a3da00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:45:37.323 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:37.307+0000 7f5246d9d640 1 --2- 172.21.15.39:0/3021347161 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5600f36570a0 0x5600f3a31770 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f52380e4980 tx=0x7f523800bf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1127adbcc36042e2 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:45:37.323 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:37.307+0000 7f522aff5640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-09-21T03:45:37.323 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:37.307+0000 7f522aff5640 1 -- 172.21.15.39:0/3021347161 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5238021030 con 0x5600f36570a0 2024-09-21T03:45:37.323 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:37.307+0000 7f522aff5640 1 -- 172.21.15.39:0/3021347161 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8484+0+0 (secure 0 0 0) 0x7f52380e90d0 con 0x5600f36570a0 2024-09-21T03:45:37.323 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:37.307+0000 7f5253760740 1 -- 172.21.15.39:0/3021347161 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f523c01a910 con 0x5600f36570a0 2024-09-21T03:45:37.323 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:37.307+0000 7f522aff5640 1 -- 172.21.15.39:0/3021347161 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f52380ac920 con 0x5600f36570a0 2024-09-21T03:45:37.323 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:37.307+0000 7f5253760740 10 client.15288 fetch_fsmap learned FSMap version 28 2024-09-21T03:45:37.323 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:37.307+0000 7f5253760740 1 -- 172.21.15.39:0/3021347161 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5600f3a405d0 con 0x5600f36570a0 2024-09-21T03:45:37.323 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:37.307+0000 7f522aff5640 1 -- 172.21.15.39:0/3021347161 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f523c01a910 con 0x5600f36570a0 2024-09-21T03:45:37.323 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:37.307+0000 7f522aff5640 1 -- 172.21.15.39:0/3021347161 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f5238098190 con 0x5600f36570a0 2024-09-21T03:45:37.323 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:37.308+0000 7f5253760740 10 client.15288 fetch_fsmap finished waiting for FSMap version 28 2024-09-21T03:45:37.323 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:37.308+0000 7f5253760740 10 client.15288 resolve_mds: resolved name 'i' to daemon mds.i 2024-09-21T03:45:37.323 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:37.308+0000 7f5253760740 20 client.15288 populate_metadata read hostname 'smithi039' 2024-09-21T03:45:37.323 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:37.308+0000 7f5253760740 1 --2- 172.21.15.39:0/3021347161 >> [v2:172.21.15.161:6838/2190107270,v1:172.21.15.161:6839/2190107270] conn(0x5600f363e8e0 0x5600f3a44050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:37.323 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:37.308+0000 7f5253760740 4 client.15288 mds_command: new command op to 24667 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:45:37.324 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:37.308+0000 7f5253760740 1 -- 172.21.15.39:0/3021347161 --> [v2:172.21.15.161:6838/2190107270,v1:172.21.15.161:6839/2190107270] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5600f399fb60 con 0x5600f363e8e0 2024-09-21T03:45:37.324 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:37.308+0000 7f524659c640 1 --2- 172.21.15.39:0/3021347161 >> [v2:172.21.15.161:6838/2190107270,v1:172.21.15.161:6839/2190107270] conn(0x5600f363e8e0 0x5600f3a44050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:37.324 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:37.308+0000 7f524659c640 1 --2- 172.21.15.39:0/3021347161 >> [v2:172.21.15.161:6838/2190107270,v1:172.21.15.161:6839/2190107270] conn(0x5600f363e8e0 0x5600f3a44050 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=345040aeb04cf88c server_cookie=7d96a55d642c6e98 in_seq=0 out_seq=0 2024-09-21T03:45:37.324 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:37.308+0000 7f522aff5640 10 client.15288 ms_handle_connect on v2:172.21.15.161:6838/2190107270 2024-09-21T03:45:37.324 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:37.309+0000 7f522aff5640 1 -- 172.21.15.39:0/3021347161 <== mds.1 v2:172.21.15.161:6838/2190107270 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5600f399fb60 con 0x5600f363e8e0 2024-09-21T03:45:37.324 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:37.309+0000 7f522aff5640 10 client.15288 handle_command_reply: tid=0 2024-09-21T03:45:37.324 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:37.309+0000 7f5253760740 1 client.15288 shutdown 2024-09-21T03:45:37.324 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:37.309+0000 7f5253760740 1 -- 172.21.15.39:0/3021347161 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5600f36570a0 msgr2=0x5600f3a31770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:37.324 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:37.309+0000 7f5253760740 1 --2- 172.21.15.39:0/3021347161 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5600f36570a0 0x5600f3a31770 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f52380e4980 tx=0x7f523800bf60 comp rx=0 tx=0).stop 2024-09-21T03:45:37.324 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:37.309+0000 7f5253760740 1 -- 172.21.15.39:0/3021347161 shutdown_connections 2024-09-21T03:45:37.324 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:37.309+0000 7f5253760740 1 --2- 172.21.15.39:0/3021347161 >> [v2:172.21.15.161:6838/2190107270,v1:172.21.15.161:6839/2190107270] conn(0x5600f363e8e0 0x5600f3a44050 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:37.324 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:37.310+0000 7f5253760740 1 --2- 172.21.15.39:0/3021347161 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5600f367ff60 0x5600f3a3da00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:37.324 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:37.310+0000 7f5253760740 1 --2- 172.21.15.39:0/3021347161 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5600f365f280 0x5600f3a31d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:37.325 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:37.310+0000 7f5253760740 1 --2- 172.21.15.39:0/3021347161 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5600f36570a0 0x5600f3a31770 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:37.325 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:37.310+0000 7f522aff5640 0 client.15288 ms_handle_reset on v2:172.21.15.161:6838/2190107270 2024-09-21T03:45:37.325 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:37.310+0000 7f5253760740 1 -- 172.21.15.39:0/3021347161 >> 172.21.15.39:0/3021347161 conn(0x5600f3674350 msgr2=0x5600f3a29090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:45:37.325 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:37.310+0000 7f5253760740 1 -- 172.21.15.39:0/3021347161 shutdown_connections 2024-09-21T03:45:37.325 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:37.310+0000 7f5253760740 1 -- 172.21.15.39:0/3021347161 wait complete. 2024-09-21T03:45:37.325 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:37.310+0000 7f5253760740 20 client.15288 trim_cache size 0 max 16384 2024-09-21T03:45:37.331 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:37.329+0000 7f5253760740 1 Processor -- start 2024-09-21T03:45:37.332 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:37.329+0000 7f5253760740 1 -- start start 2024-09-21T03:45:37.332 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:37.329+0000 7f5253760740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5600f36570a0 0x5600f3a3db80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:37.332 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:37.329+0000 7f5253760740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5600f365f280 0x5600f3a2c920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:37.332 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:37.329+0000 7f5253760740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5600f367ff60 0x5600f3a2ce60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:37.332 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:37.329+0000 7f5253760740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x5600f3a42390 con 0x5600f36570a0 2024-09-21T03:45:37.332 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:37.329+0000 7f5253760740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x5600f3a42a60 con 0x5600f367ff60 2024-09-21T03:45:37.332 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:37.329+0000 7f5253760740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x5600f3a43130 con 0x5600f365f280 2024-09-21T03:45:37.332 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:37.329+0000 7f5246d9d640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5600f36570a0 0x5600f3a3db80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:37.332 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:37.329+0000 7f5246d9d640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5600f36570a0 0x5600f3a3db80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:40732/0 (socket says 172.21.15.39:40732) 2024-09-21T03:45:37.332 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:37.329+0000 7f5246d9d640 1 -- 172.21.15.39:0/1634700361 learned_addr learned my addr 172.21.15.39:0/1634700361 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:45:37.332 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:37.330+0000 7f524659c640 1 --2- 172.21.15.39:0/1634700361 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5600f365f280 0x5600f3a2c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:37.332 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:37.330+0000 7f5246d9d640 1 -- 172.21.15.39:0/1634700361 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5600f365f280 msgr2=0x5600f3a2c920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:45:37.332 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:37.330+0000 7f5246d9d640 1 --2- 172.21.15.39:0/1634700361 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5600f365f280 0x5600f3a2c920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:37.333 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:37.330+0000 7f5246d9d640 1 -- 172.21.15.39:0/1634700361 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5600f367ff60 msgr2=0x5600f3a2ce60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:45:37.333 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:37.330+0000 7f5246d9d640 1 --2- 172.21.15.39:0/1634700361 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5600f367ff60 0x5600f3a2ce60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:37.333 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:37.330+0000 7f5246d9d640 1 -- 172.21.15.39:0/1634700361 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5600f3a32a30 con 0x5600f36570a0 2024-09-21T03:45:37.333 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:37.330+0000 7f5246d9d640 1 --2- 172.21.15.39:0/1634700361 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5600f36570a0 0x5600f3a3db80 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f52380accd0 tx=0x7f52380e4910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb9516ea83d56ce3 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:45:37.333 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:37.330+0000 7f522cff9640 1 -- 172.21.15.39:0/1634700361 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5238021070 con 0x5600f36570a0 2024-09-21T03:45:37.333 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:37.330+0000 7f522cff9640 1 -- 172.21.15.39:0/1634700361 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5238009de0 con 0x5600f36570a0 2024-09-21T03:45:37.333 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:37.330+0000 7f522cff9640 1 -- 172.21.15.39:0/1634700361 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52380ac740 con 0x5600f36570a0 2024-09-21T03:45:37.337 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:37.331+0000 7f5253760740 1 -- 172.21.15.39:0/1634700361 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5600f36570a0 msgr2=0x5600f3a3db80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:37.337 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:37.331+0000 7f5253760740 1 --2- 172.21.15.39:0/1634700361 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5600f36570a0 0x5600f3a3db80 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f52380accd0 tx=0x7f52380e4910 comp rx=0 tx=0).stop 2024-09-21T03:45:37.337 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:37.331+0000 7f5253760740 1 -- 172.21.15.39:0/1634700361 shutdown_connections 2024-09-21T03:45:37.337 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:37.331+0000 7f5253760740 1 --2- 172.21.15.39:0/1634700361 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5600f367ff60 0x5600f3a2ce60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:37.337 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:37.331+0000 7f5253760740 1 --2- 172.21.15.39:0/1634700361 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5600f365f280 0x5600f3a2c920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:37.337 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:37.331+0000 7f5253760740 1 --2- 172.21.15.39:0/1634700361 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5600f36570a0 0x5600f3a3db80 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:37.337 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:37.331+0000 7f5253760740 1 -- 172.21.15.39:0/1634700361 >> 172.21.15.39:0/1634700361 conn(0x5600f3674350 msgr2=0x5600f3a41a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:45:37.337 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:37.331+0000 7f5253760740 1 -- 172.21.15.39:0/1634700361 shutdown_connections 2024-09-21T03:45:37.337 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:37.331+0000 7f5253760740 1 -- 172.21.15.39:0/1634700361 wait complete. 2024-09-21T03:45:37.338 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:37.331+0000 7f5253760740 1 -- start start 2024-09-21T03:45:37.338 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:37.332+0000 7f5253760740 1 Processor -- start 2024-09-21T03:45:37.338 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:37.332+0000 7f5253760740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5600f36570a0 0x5600f3a46a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:37.338 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:37.332+0000 7f5253760740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5600f365f280 0x5600f3a46f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:37.338 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:37.332+0000 7f5253760740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5600f367ff60 0x5600f3a4b880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:37.338 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:37.332+0000 7f5253760740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x5600f3a43ed0 con 0x5600f365f280 2024-09-21T03:45:37.338 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:37.332+0000 7f5253760740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x5600f3a43800 con 0x5600f367ff60 2024-09-21T03:45:37.338 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:37.332+0000 7f5253760740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x5600f3a445a0 con 0x5600f36570a0 2024-09-21T03:45:37.338 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:37.332+0000 7f5253760740 10 client.0 _ll_register_callbacks cb 0x7f52380a9620 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:45:37.338 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:37.332+0000 7f524659c640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5600f365f280 0x5600f3a46f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:37.338 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:37.332+0000 7f524659c640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5600f365f280 0x5600f3a46f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:40746/0 (socket says 172.21.15.39:40746) 2024-09-21T03:45:37.338 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:37.332+0000 7f524659c640 1 -- 172.21.15.39:0/976098935 learned_addr learned my addr 172.21.15.39:0/976098935 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:45:37.338 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:37.332+0000 7f5246d9d640 1 --2- 172.21.15.39:0/976098935 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5600f36570a0 0x5600f3a46a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:37.338 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:37.332+0000 7f524759e640 1 --2- 172.21.15.39:0/976098935 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5600f367ff60 0x5600f3a4b880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:37.338 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:37.332+0000 7f524659c640 1 -- 172.21.15.39:0/976098935 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5600f36570a0 msgr2=0x5600f3a46a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:45:37.339 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:37.332+0000 7f524659c640 1 --2- 172.21.15.39:0/976098935 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5600f36570a0 0x5600f3a46a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:37.339 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:37.332+0000 7f524659c640 1 -- 172.21.15.39:0/976098935 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5600f367ff60 msgr2=0x5600f3a4b880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:45:37.339 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:37.332+0000 7f524659c640 1 --2- 172.21.15.39:0/976098935 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5600f367ff60 0x5600f3a4b880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:37.339 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:37.332+0000 7f524659c640 1 -- 172.21.15.39:0/976098935 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5600f3a4bdc0 con 0x5600f365f280 2024-09-21T03:45:37.339 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:37.333+0000 7f524659c640 1 --2- 172.21.15.39:0/976098935 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5600f365f280 0x5600f3a46f70 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f523c0114b0 tx=0x7f523c0114e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c55cc4cd858966d6 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:45:37.339 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:37.333+0000 7f522aff5640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-09-21T03:45:37.339 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:37.333+0000 7f522aff5640 1 -- 172.21.15.39:0/976098935 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f523c031060 con 0x5600f365f280 2024-09-21T03:45:37.339 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:37.333+0000 7f522aff5640 1 -- 172.21.15.39:0/976098935 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8484+0+0 (secure 0 0 0) 0x7f523c004250 con 0x5600f365f280 2024-09-21T03:45:37.339 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:37.333+0000 7f5253760740 1 -- 172.21.15.39:0/976098935 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5238009de0 con 0x5600f365f280 2024-09-21T03:45:37.339 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:37.333+0000 7f522aff5640 1 -- 172.21.15.39:0/976098935 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f523c015030 con 0x5600f365f280 2024-09-21T03:45:37.339 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:37.333+0000 7f5253760740 10 client.15294 fetch_fsmap learned FSMap version 28 2024-09-21T03:45:37.339 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:37.333+0000 7f5253760740 1 -- 172.21.15.39:0/976098935 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5600f3a4d040 con 0x5600f365f280 2024-09-21T03:45:37.339 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:37.333+0000 7f522aff5640 1 -- 172.21.15.39:0/976098935 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5238009de0 con 0x5600f365f280 2024-09-21T03:45:37.339 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:37.333+0000 7f522aff5640 1 -- 172.21.15.39:0/976098935 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f523c004890 con 0x5600f365f280 2024-09-21T03:45:37.340 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:37.334+0000 7f5253760740 10 client.15294 fetch_fsmap finished waiting for FSMap version 28 2024-09-21T03:45:37.340 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:37.334+0000 7f5253760740 10 client.15294 resolve_mds: resolved name 'i' to daemon mds.i 2024-09-21T03:45:37.340 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:37.334+0000 7f5253760740 20 client.15294 populate_metadata read hostname 'smithi039' 2024-09-21T03:45:37.340 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:37.334+0000 7f5253760740 1 --2- 172.21.15.39:0/976098935 >> [v2:172.21.15.161:6838/2190107270,v1:172.21.15.161:6839/2190107270] conn(0x5600f3613c90 0x5600f3ab2790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:37.340 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:37.334+0000 7f5253760740 4 client.15294 mds_command: new command op to 24667 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-21T03:45:37.340 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:37.334+0000 7f5253760740 1 -- 172.21.15.39:0/976098935 --> [v2:172.21.15.161:6838/2190107270,v1:172.21.15.161:6839/2190107270] -- command(tid 0: {"prefix": "status"}) -- 0x5600f399b610 con 0x5600f3613c90 2024-09-21T03:45:37.340 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:37.334+0000 7f5246d9d640 1 --2- 172.21.15.39:0/976098935 >> [v2:172.21.15.161:6838/2190107270,v1:172.21.15.161:6839/2190107270] conn(0x5600f3613c90 0x5600f3ab2790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:37.340 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:37.334+0000 7f5246d9d640 1 --2- 172.21.15.39:0/976098935 >> [v2:172.21.15.161:6838/2190107270,v1:172.21.15.161:6839/2190107270] conn(0x5600f3613c90 0x5600f3ab2790 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=3d6b75ea3e67d796 server_cookie=c253e15a189389d6 in_seq=0 out_seq=0 2024-09-21T03:45:37.340 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:37.334+0000 7f522aff5640 10 client.15294 ms_handle_connect on v2:172.21.15.161:6838/2190107270 2024-09-21T03:45:37.340 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:37.334+0000 7f522aff5640 1 -- 172.21.15.39:0/976098935 <== mds.1 v2:172.21.15.161:6838/2190107270 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x5600f399b610 con 0x5600f3613c90 2024-09-21T03:45:37.340 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:37.335+0000 7f522aff5640 10 client.15294 handle_command_reply: tid=0 2024-09-21T03:45:37.340 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:37.335+0000 7f5253760740 1 client.15294 shutdown 2024-09-21T03:45:37.341 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:37.335+0000 7f5253760740 1 -- 172.21.15.39:0/976098935 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5600f365f280 msgr2=0x5600f3a46f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:37.341 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:37.335+0000 7f5253760740 1 --2- 172.21.15.39:0/976098935 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5600f365f280 0x5600f3a46f70 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f523c0114b0 tx=0x7f523c0114e0 comp rx=0 tx=0).stop 2024-09-21T03:45:37.341 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:37.335+0000 7f5253760740 1 -- 172.21.15.39:0/976098935 shutdown_connections 2024-09-21T03:45:37.341 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:37.335+0000 7f5253760740 1 --2- 172.21.15.39:0/976098935 >> [v2:172.21.15.161:6838/2190107270,v1:172.21.15.161:6839/2190107270] conn(0x5600f3613c90 0x5600f3ab2790 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:37.341 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:37.335+0000 7f5253760740 1 --2- 172.21.15.39:0/976098935 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5600f367ff60 0x5600f3a4b880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:37.341 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:37.335+0000 7f5253760740 1 --2- 172.21.15.39:0/976098935 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5600f365f280 0x5600f3a46f70 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:37.341 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:37.335+0000 7f5253760740 1 --2- 172.21.15.39:0/976098935 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5600f36570a0 0x5600f3a46a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:37.341 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:37.335+0000 7f5253760740 1 -- 172.21.15.39:0/976098935 >> 172.21.15.39:0/976098935 conn(0x5600f3674350 msgr2=0x5600f3a3db80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:45:37.341 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:37.335+0000 7f522aff5640 0 client.15294 ms_handle_reset on v2:172.21.15.161:6838/2190107270 2024-09-21T03:45:37.341 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:37.335+0000 7f5253760740 1 -- 172.21.15.39:0/976098935 shutdown_connections 2024-09-21T03:45:37.341 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:37.335+0000 7f5253760740 1 -- 172.21.15.39:0/976098935 wait complete. 2024-09-21T03:45:37.341 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:37.335+0000 7f5253760740 20 client.15294 trim_cache size 0 max 16384 2024-09-21T03:45:37.341 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-09-21T03:45:37.342 INFO:teuthology.orchestra.run.smithi039.stdout: "cluster_fsid": "51116d5c-77ca-11ef-baf5-efdc52797490", 2024-09-21T03:45:37.342 INFO:teuthology.orchestra.run.smithi039.stdout: "whoami": 1, 2024-09-21T03:45:37.342 INFO:teuthology.orchestra.run.smithi039.stdout: "id": 24667, 2024-09-21T03:45:37.342 INFO:teuthology.orchestra.run.smithi039.stdout: "want_state": "up:active", 2024-09-21T03:45:37.342 INFO:teuthology.orchestra.run.smithi039.stdout: "state": "up:active", 2024-09-21T03:45:37.342 INFO:teuthology.orchestra.run.smithi039.stdout: "fs_name": "cephfs", 2024-09-21T03:45:37.342 INFO:teuthology.orchestra.run.smithi039.stdout: "rank_uptime": 74.967656947999998, 2024-09-21T03:45:37.342 INFO:teuthology.orchestra.run.smithi039.stdout: "mdsmap_epoch": 28, 2024-09-21T03:45:37.342 INFO:teuthology.orchestra.run.smithi039.stdout: "osdmap_epoch": 79, 2024-09-21T03:45:37.342 INFO:teuthology.orchestra.run.smithi039.stdout: "osdmap_epoch_barrier": 78, 2024-09-21T03:45:37.342 INFO:teuthology.orchestra.run.smithi039.stdout: "uptime": 101.101598052 2024-09-21T03:45:37.342 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-09-21T03:45:37.343 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:37.336+0000 7f524d1ec640 1 -- 172.21.15.39:0/3231983357 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f52300b65f0 msgr2=0x7f52300b8b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:37.343 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:37.336+0000 7f524d1ec640 1 --2- 172.21.15.39:0/3231983357 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f52300b65f0 0x7f52300b8b00 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f5240006010 tx=0x7f5240009440 comp rx=0 tx=0).stop 2024-09-21T03:45:37.343 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:37.336+0000 7f524d1ec640 1 -- 172.21.15.39:0/3231983357 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5248104840 msgr2=0x7f524819a3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:37.343 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:37.336+0000 7f524d1ec640 1 --2- 172.21.15.39:0/3231983357 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5248104840 0x7f524819a3e0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f523800d950 tx=0x7f523800de10 comp rx=0 tx=0).stop 2024-09-21T03:45:37.343 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:37.336+0000 7f524d1ec640 1 -- 172.21.15.39:0/3231983357 shutdown_connections 2024-09-21T03:45:37.343 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:37.336+0000 7f524d1ec640 1 --2- 172.21.15.39:0/3231983357 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f52300b65f0 0x7f52300b8b00 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:37.343 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:37.336+0000 7f524d1ec640 1 --2- 172.21.15.39:0/3231983357 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f524810ddf0 0x7f524819fc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:37.343 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:37.336+0000 7f524d1ec640 1 --2- 172.21.15.39:0/3231983357 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5248105270 0x7f524819a980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:37.344 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:37.336+0000 7f524d1ec640 1 --2- 172.21.15.39:0/3231983357 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5248104840 0x7f524819a3e0 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:37.344 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:37.336+0000 7f524d1ec640 1 -- 172.21.15.39:0/3231983357 >> 172.21.15.39:0/3231983357 conn(0x7f5248079730 msgr2=0x7f52481062d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:45:37.344 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:37.336+0000 7f524d1ec640 1 -- 172.21.15.39:0/3231983357 shutdown_connections 2024-09-21T03:45:37.344 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:37.336+0000 7f524d1ec640 1 -- 172.21.15.39:0/3231983357 wait complete. 2024-09-21T03:45:38.029 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 51116d5c-77ca-11ef-baf5-efdc52797490 -- ceph tell mds.b status 2024-09-21T03:45:38.283 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/51116d5c-77ca-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:45:38.741 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:38.739+0000 7f5ac8d92640 1 -- 172.21.15.39:0/67414483 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5ac41083b0 msgr2=0x7f5ac41087c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:38.741 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:38.739+0000 7f5ac8d92640 1 --2- 172.21.15.39:0/67414483 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5ac41083b0 0x7f5ac41087c0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f5ab8009900 tx=0x7f5ab802a410 comp rx=0 tx=0).stop 2024-09-21T03:45:38.741 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:38.739+0000 7f5ac8d92640 1 -- 172.21.15.39:0/67414483 shutdown_connections 2024-09-21T03:45:38.741 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:38.739+0000 7f5ac8d92640 1 --2- 172.21.15.39:0/67414483 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5ac4108d00 0x7f5ac410f530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:38.741 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:38.739+0000 7f5ac8d92640 1 --2- 172.21.15.39:0/67414483 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5ac41083b0 0x7f5ac41087c0 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:38.741 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:38.739+0000 7f5ac8d92640 1 --2- 172.21.15.39:0/67414483 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5ac4102a70 0x7f5ac4102e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:38.741 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:38.739+0000 7f5ac8d92640 1 -- 172.21.15.39:0/67414483 >> 172.21.15.39:0/67414483 conn(0x7f5ac4079730 msgr2=0x7f5ac4079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:45:38.741 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:38.739+0000 7f5ac8d92640 1 -- 172.21.15.39:0/67414483 shutdown_connections 2024-09-21T03:45:38.742 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:38.739+0000 7f5ac8d92640 1 -- 172.21.15.39:0/67414483 wait complete. 2024-09-21T03:45:38.743 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:38.740+0000 7f5ac8d92640 1 Processor -- start 2024-09-21T03:45:38.744 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:38.740+0000 7f5ac8d92640 1 -- start start 2024-09-21T03:45:38.744 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:38.741+0000 7f5ac8d92640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5ac4102a70 0x7f5ac4199360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:38.744 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:38.741+0000 7f5ac8d92640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5ac41083b0 0x7f5ac4199900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:38.744 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:38.741+0000 7f5ac8d92640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5ac4108d00 0x7f5ac419eba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:38.744 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:38.741+0000 7f5ac8d92640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f5ac4112970 con 0x7f5ac4102a70 2024-09-21T03:45:38.744 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:38.741+0000 7f5ac8d92640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f5ac41127f0 con 0x7f5ac4108d00 2024-09-21T03:45:38.744 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:38.741+0000 7f5ac8d92640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f5ac4112af0 con 0x7f5ac41083b0 2024-09-21T03:45:38.744 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:38.741+0000 7f5ac259c640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5ac4102a70 0x7f5ac4199360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:38.744 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:38.741+0000 7f5ac259c640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5ac4102a70 0x7f5ac4199360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:55712/0 (socket says 172.21.15.39:55712) 2024-09-21T03:45:38.744 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:38.741+0000 7f5ac259c640 1 -- 172.21.15.39:0/842288765 learned_addr learned my addr 172.21.15.39:0/842288765 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:45:38.744 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:38.741+0000 7f5ac2d9d640 1 --2- 172.21.15.39:0/842288765 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5ac4108d00 0x7f5ac419eba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:38.744 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:38.741+0000 7f5ac1d9b640 1 --2- 172.21.15.39:0/842288765 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5ac41083b0 0x7f5ac4199900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:38.744 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:38.741+0000 7f5ac259c640 1 -- 172.21.15.39:0/842288765 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5ac41083b0 msgr2=0x7f5ac4199900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:38.744 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:38.741+0000 7f5ac259c640 1 --2- 172.21.15.39:0/842288765 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5ac41083b0 0x7f5ac4199900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:38.745 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:38.741+0000 7f5ac259c640 1 -- 172.21.15.39:0/842288765 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5ac4108d00 msgr2=0x7f5ac419eba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:38.745 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:38.741+0000 7f5ac259c640 1 --2- 172.21.15.39:0/842288765 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5ac4108d00 0x7f5ac419eba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:38.745 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:38.741+0000 7f5ac259c640 1 -- 172.21.15.39:0/842288765 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ac419f2d0 con 0x7f5ac4102a70 2024-09-21T03:45:38.745 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:38.741+0000 7f5ac1d9b640 1 --2- 172.21.15.39:0/842288765 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5ac41083b0 0x7f5ac4199900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:45:38.745 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:38.741+0000 7f5ac259c640 1 --2- 172.21.15.39:0/842288765 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5ac4102a70 0x7f5ac4199360 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f5ab400ebc0 tx=0x7f5ab4007590 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:45:38.745 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:38.742+0000 7f5ab37fe640 1 -- 172.21.15.39:0/842288765 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ab401c070 con 0x7f5ac4102a70 2024-09-21T03:45:38.745 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:38.742+0000 7f5ab37fe640 1 -- 172.21.15.39:0/842288765 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5ab4007ca0 con 0x7f5ac4102a70 2024-09-21T03:45:38.745 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:38.742+0000 7f5ac8d92640 1 -- 172.21.15.39:0/842288765 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5ac419f5c0 con 0x7f5ac4102a70 2024-09-21T03:45:38.745 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:38.742+0000 7f5ab37fe640 1 -- 172.21.15.39:0/842288765 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ab4017410 con 0x7f5ac4102a70 2024-09-21T03:45:38.745 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:38.742+0000 7f5ac8d92640 1 -- 172.21.15.39:0/842288765 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5ac406bae0 con 0x7f5ac4102a70 2024-09-21T03:45:38.746 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:38.744+0000 7f5acf306740 1 Processor -- start 2024-09-21T03:45:38.746 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:38.744+0000 7f5acf306740 1 -- start start 2024-09-21T03:45:38.747 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:38.745+0000 7f5acf306740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5568c0bd0f60 0x5568c0f76f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:38.747 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:38.745+0000 7f5acf306740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5568c0ba80a0 0x5568c0f77440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:38.747 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:38.745+0000 7f5acf306740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5568c0bb0280 0x5568c0f77980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:38.747 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:38.745+0000 7f5acf306740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x5568c0eeec10 con 0x5568c0bb0280 2024-09-21T03:45:38.747 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:38.745+0000 7f5acf306740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x5568c0ede040 con 0x5568c0ba80a0 2024-09-21T03:45:38.747 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:38.745+0000 7f5acf306740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x5568c0ee1e50 con 0x5568c0bd0f60 2024-09-21T03:45:38.748 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:38.745+0000 7f5ab37fe640 1 -- 172.21.15.39:0/842288765 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 22) ==== 154580+0+0 (secure 0 0 0) 0x7f5ab401f070 con 0x7f5ac4102a70 2024-09-21T03:45:38.748 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:38.745+0000 7f5ac1d9b640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5568c0bb0280 0x5568c0f77980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:38.748 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:38.745+0000 7f5ac1d9b640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5568c0bb0280 0x5568c0f77980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:55716/0 (socket says 172.21.15.39:55716) 2024-09-21T03:45:38.748 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:38.745+0000 7f5ac1d9b640 1 -- 172.21.15.39:0/3953506667 learned_addr learned my addr 172.21.15.39:0/3953506667 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:45:38.748 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:38.745+0000 7f5ac2d9d640 1 --2- 172.21.15.39:0/3953506667 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5568c0bd0f60 0x5568c0f76f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:38.748 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:38.745+0000 7f5ac259c640 1 --2- 172.21.15.39:0/3953506667 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5568c0ba80a0 0x5568c0f77440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:38.748 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:38.745+0000 7f5ac1d9b640 1 -- 172.21.15.39:0/3953506667 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5568c0bd0f60 msgr2=0x5568c0f76f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:45:38.748 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:38.745+0000 7f5ac1d9b640 1 --2- 172.21.15.39:0/3953506667 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5568c0bd0f60 0x5568c0f76f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:38.748 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:38.745+0000 7f5ac1d9b640 1 -- 172.21.15.39:0/3953506667 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5568c0ba80a0 msgr2=0x5568c0f77440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:45:38.748 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:38.745+0000 7f5ac1d9b640 1 --2- 172.21.15.39:0/3953506667 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5568c0ba80a0 0x5568c0f77440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:38.748 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:38.745+0000 7f5ac1d9b640 1 -- 172.21.15.39:0/3953506667 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5568c0c3c540 con 0x5568c0bb0280 2024-09-21T03:45:38.748 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:38.746+0000 7f5ac1d9b640 1 --2- 172.21.15.39:0/3953506667 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5568c0bb0280 0x5568c0f77980 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f5ab8031440 tx=0x7f5ab8032d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd15736fe24921a4 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:45:38.752 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:38.746+0000 7f5ab0ff9640 1 -- 172.21.15.39:0/3953506667 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ab803f070 con 0x5568c0bb0280 2024-09-21T03:45:38.752 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:38.746+0000 7f5ab0ff9640 1 -- 172.21.15.39:0/3953506667 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5ab80316a0 con 0x5568c0bb0280 2024-09-21T03:45:38.752 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:38.746+0000 7f5ab0ff9640 1 -- 172.21.15.39:0/3953506667 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ab8035070 con 0x5568c0bb0280 2024-09-21T03:45:38.752 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:38.746+0000 7f5ab37fe640 1 --2- 172.21.15.39:0/842288765 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f5aac0b6670 0x7f5aac0b8b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:38.752 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:38.746+0000 7f5ac2d9d640 1 --2- 172.21.15.39:0/842288765 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f5aac0b6670 0x7f5aac0b8b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:38.752 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:38.746+0000 7f5ab37fe640 1 -- 172.21.15.39:0/842288765 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8484+0+0 (secure 0 0 0) 0x7f5ab40e8890 con 0x7f5ac4102a70 2024-09-21T03:45:38.752 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:38.747+0000 7f5acf306740 1 -- 172.21.15.39:0/3953506667 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5568c0bb0280 msgr2=0x5568c0f77980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:38.752 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:38.747+0000 7f5ac2d9d640 1 --2- 172.21.15.39:0/842288765 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f5aac0b6670 0x7f5aac0b8b80 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f5abc00b7f0 tx=0x7f5abc00b2e0 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:45:38.752 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:38.747+0000 7f5acf306740 1 --2- 172.21.15.39:0/3953506667 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5568c0bb0280 0x5568c0f77980 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f5ab8031440 tx=0x7f5ab8032d00 comp rx=0 tx=0).stop 2024-09-21T03:45:38.752 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:38.747+0000 7f5acf306740 1 -- 172.21.15.39:0/3953506667 shutdown_connections 2024-09-21T03:45:38.752 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:38.747+0000 7f5acf306740 1 --2- 172.21.15.39:0/3953506667 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5568c0bb0280 0x5568c0f77980 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:38.753 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:38.747+0000 7f5acf306740 1 --2- 172.21.15.39:0/3953506667 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5568c0ba80a0 0x5568c0f77440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:38.753 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:38.747+0000 7f5acf306740 1 --2- 172.21.15.39:0/3953506667 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5568c0bd0f60 0x5568c0f76f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:38.753 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:38.747+0000 7f5acf306740 1 -- 172.21.15.39:0/3953506667 >> 172.21.15.39:0/3953506667 conn(0x5568c0bc5350 msgr2=0x5568c0f74df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:45:38.753 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:38.747+0000 7f5acf306740 1 -- 172.21.15.39:0/3953506667 shutdown_connections 2024-09-21T03:45:38.753 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:38.747+0000 7f5acf306740 1 -- 172.21.15.39:0/3953506667 wait complete. 2024-09-21T03:45:38.753 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:38.747+0000 7f5acf306740 1 -- start start 2024-09-21T03:45:38.753 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:38.748+0000 7f5acf306740 1 Processor -- start 2024-09-21T03:45:38.753 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:38.748+0000 7f5acf306740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5568c0ba80a0 0x5568c0f82a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:38.753 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:38.748+0000 7f5acf306740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5568c0bb0280 0x5568c0f82fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:38.753 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:38.748+0000 7f5acf306740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5568c0bd0f60 0x5568c0f8e970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:38.753 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:38.748+0000 7f5acf306740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x5568c0d3f5f0 con 0x5568c0bd0f60 2024-09-21T03:45:38.753 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:38.748+0000 7f5acf306740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x5568c0e79010 con 0x5568c0ba80a0 2024-09-21T03:45:38.753 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:38.748+0000 7f5acf306740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x5568c0f85680 con 0x5568c0bb0280 2024-09-21T03:45:38.753 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:38.748+0000 7f5acf306740 10 client.0 _ll_register_callbacks cb 0x5568c0ed8340 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:45:38.754 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:38.748+0000 7f5ac2d9d640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5568c0bd0f60 0x5568c0f8e970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:38.754 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:38.748+0000 7f5ac2d9d640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5568c0bd0f60 0x5568c0f8e970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:55720/0 (socket says 172.21.15.39:55720) 2024-09-21T03:45:38.754 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:38.748+0000 7f5ac2d9d640 1 -- 172.21.15.39:0/1400011109 learned_addr learned my addr 172.21.15.39:0/1400011109 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:45:38.754 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:38.748+0000 7f5ac259c640 1 --2- 172.21.15.39:0/1400011109 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5568c0ba80a0 0x5568c0f82a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:38.754 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:38.748+0000 7f5ac1d9b640 1 --2- 172.21.15.39:0/1400011109 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5568c0bb0280 0x5568c0f82fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:38.754 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:38.748+0000 7f5ac2d9d640 1 -- 172.21.15.39:0/1400011109 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5568c0bb0280 msgr2=0x5568c0f82fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:45:38.754 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:38.748+0000 7f5ac2d9d640 1 --2- 172.21.15.39:0/1400011109 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5568c0bb0280 0x5568c0f82fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:38.754 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:38.748+0000 7f5ac2d9d640 1 -- 172.21.15.39:0/1400011109 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5568c0ba80a0 msgr2=0x5568c0f82a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:45:38.754 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:38.748+0000 7f5ac2d9d640 1 --2- 172.21.15.39:0/1400011109 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5568c0ba80a0 0x5568c0f82a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:38.754 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:38.748+0000 7f5ac2d9d640 1 -- 172.21.15.39:0/1400011109 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5568c0ea6a00 con 0x5568c0bd0f60 2024-09-21T03:45:38.754 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:38.749+0000 7f5ac2d9d640 1 --2- 172.21.15.39:0/1400011109 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5568c0bd0f60 0x5568c0f8e970 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f5abc003c60 tx=0x7f5abc004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c9544288508c8f4 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:45:38.754 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:38.749+0000 7f5aa9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-09-21T03:45:38.754 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:38.749+0000 7f5aa9ffb640 1 -- 172.21.15.39:0/1400011109 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5abc003ea0 con 0x5568c0bd0f60 2024-09-21T03:45:38.754 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:38.749+0000 7f5aa9ffb640 1 -- 172.21.15.39:0/1400011109 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8484+0+0 (secure 0 0 0) 0x7f5abc017030 con 0x5568c0bd0f60 2024-09-21T03:45:38.754 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:38.749+0000 7f5acf306740 1 -- 172.21.15.39:0/1400011109 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5ab80316a0 con 0x5568c0bd0f60 2024-09-21T03:45:38.755 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:38.749+0000 7f5aa9ffb640 1 -- 172.21.15.39:0/1400011109 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f5abc01fe40 con 0x5568c0bd0f60 2024-09-21T03:45:38.755 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:38.749+0000 7f5acf306740 10 client.15306 fetch_fsmap learned FSMap version 28 2024-09-21T03:45:38.755 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:38.749+0000 7f5acf306740 1 -- 172.21.15.39:0/1400011109 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5568c0f916a0 con 0x5568c0bd0f60 2024-09-21T03:45:38.755 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:38.749+0000 7f5aa9ffb640 1 -- 172.21.15.39:0/1400011109 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5ab80316a0 con 0x5568c0bd0f60 2024-09-21T03:45:38.755 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:38.749+0000 7f5aa9ffb640 1 -- 172.21.15.39:0/1400011109 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f5abc02e020 con 0x5568c0bd0f60 2024-09-21T03:45:38.755 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:38.749+0000 7f5acf306740 10 client.15306 fetch_fsmap finished waiting for FSMap version 28 2024-09-21T03:45:38.755 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:38.749+0000 7f5acf306740 10 client.15306 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-21T03:45:38.755 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:38.750+0000 7f5acf306740 20 client.15306 populate_metadata read hostname 'smithi039' 2024-09-21T03:45:38.755 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:38.750+0000 7f5acf306740 1 --2- 172.21.15.39:0/1400011109 >> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] conn(0x5568c0b8f8e0 0x5568c0f95170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:38.755 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:38.750+0000 7f5acf306740 4 client.15306 mds_command: new command op to 24695 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:45:38.755 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:38.750+0000 7f5acf306740 1 -- 172.21.15.39:0/1400011109 --> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5568c0ef0b60 con 0x5568c0b8f8e0 2024-09-21T03:45:38.755 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:38.750+0000 7f5ac259c640 1 --2- 172.21.15.39:0/1400011109 >> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] conn(0x5568c0b8f8e0 0x5568c0f95170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:38.755 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:38.750+0000 7f5ac259c640 1 --2- 172.21.15.39:0/1400011109 >> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] conn(0x5568c0b8f8e0 0x5568c0f95170 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=445ff284dcd73488 server_cookie=d9ba323fb78cb653 in_seq=0 out_seq=0 2024-09-21T03:45:38.755 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:38.750+0000 7f5aa9ffb640 10 client.15306 ms_handle_connect on v2:172.21.15.150:6838/2949386907 2024-09-21T03:45:38.756 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:38.751+0000 7f5aa9ffb640 1 -- 172.21.15.39:0/1400011109 <== mds.0 v2:172.21.15.150:6838/2949386907 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5568c0ef0b60 con 0x5568c0b8f8e0 2024-09-21T03:45:38.756 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:38.751+0000 7f5aa9ffb640 10 client.15306 handle_command_reply: tid=0 2024-09-21T03:45:38.757 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:38.751+0000 7f5acf306740 1 client.15306 shutdown 2024-09-21T03:45:38.757 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:38.751+0000 7f5acf306740 1 -- 172.21.15.39:0/1400011109 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5568c0bd0f60 msgr2=0x5568c0f8e970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:38.757 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:38.751+0000 7f5acf306740 1 --2- 172.21.15.39:0/1400011109 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5568c0bd0f60 0x5568c0f8e970 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f5abc003c60 tx=0x7f5abc004730 comp rx=0 tx=0).stop 2024-09-21T03:45:38.757 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:38.752+0000 7f5acf306740 1 -- 172.21.15.39:0/1400011109 shutdown_connections 2024-09-21T03:45:38.757 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:38.752+0000 7f5acf306740 1 --2- 172.21.15.39:0/1400011109 >> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] conn(0x5568c0b8f8e0 0x5568c0f95170 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:38.757 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:38.752+0000 7f5acf306740 1 --2- 172.21.15.39:0/1400011109 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5568c0bd0f60 0x5568c0f8e970 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:38.757 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:38.752+0000 7f5acf306740 1 --2- 172.21.15.39:0/1400011109 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5568c0bb0280 0x5568c0f82fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:38.757 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:38.752+0000 7f5aa9ffb640 0 client.15306 ms_handle_reset on v2:172.21.15.150:6838/2949386907 2024-09-21T03:45:38.757 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:38.752+0000 7f5acf306740 1 --2- 172.21.15.39:0/1400011109 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5568c0ba80a0 0x5568c0f82a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:38.757 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:38.752+0000 7f5acf306740 1 -- 172.21.15.39:0/1400011109 >> 172.21.15.39:0/1400011109 conn(0x5568c0bc5350 msgr2=0x5568c0f7a090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:45:38.757 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:38.752+0000 7f5acf306740 1 -- 172.21.15.39:0/1400011109 shutdown_connections 2024-09-21T03:45:38.757 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:38.752+0000 7f5acf306740 1 -- 172.21.15.39:0/1400011109 wait complete. 2024-09-21T03:45:38.757 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:38.752+0000 7f5acf306740 20 client.15306 trim_cache size 0 max 16384 2024-09-21T03:45:38.773 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:38.770+0000 7f5acf306740 1 Processor -- start 2024-09-21T03:45:38.773 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:38.770+0000 7f5acf306740 1 -- start start 2024-09-21T03:45:38.773 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:38.770+0000 7f5acf306740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5568c0ba80a0 0x5568c0f82a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:38.773 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:38.770+0000 7f5acf306740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5568c0bb0280 0x5568c0f82f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:38.773 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:38.770+0000 7f5acf306740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5568c0bd0f60 0x5568c0f7da20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:38.773 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:38.770+0000 7f5acf306740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x5568c0f934b0 con 0x5568c0ba80a0 2024-09-21T03:45:38.773 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:38.770+0000 7f5acf306740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x5568c0f93b80 con 0x5568c0bd0f60 2024-09-21T03:45:38.773 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:38.770+0000 7f5acf306740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x5568c0f94250 con 0x5568c0bb0280 2024-09-21T03:45:38.773 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:38.770+0000 7f5ac259c640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5568c0ba80a0 0x5568c0f82a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:38.773 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:38.770+0000 7f5ac259c640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5568c0ba80a0 0x5568c0f82a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:55730/0 (socket says 172.21.15.39:55730) 2024-09-21T03:45:38.773 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:38.770+0000 7f5ac259c640 1 -- 172.21.15.39:0/627329124 learned_addr learned my addr 172.21.15.39:0/627329124 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:45:38.773 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:38.770+0000 7f5ac1d9b640 1 --2- 172.21.15.39:0/627329124 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5568c0bb0280 0x5568c0f82f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:38.773 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:38.770+0000 7f5ac259c640 1 -- 172.21.15.39:0/627329124 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5568c0bb0280 msgr2=0x5568c0f82f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:45:38.774 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:38.770+0000 7f5ac259c640 1 --2- 172.21.15.39:0/627329124 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5568c0bb0280 0x5568c0f82f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:38.774 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:38.771+0000 7f5ac259c640 1 -- 172.21.15.39:0/627329124 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5568c0bd0f60 msgr2=0x5568c0f7da20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:45:38.774 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:38.771+0000 7f5ac2d9d640 1 --2- 172.21.15.39:0/627329124 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5568c0bd0f60 0x5568c0f7da20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:38.774 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:38.771+0000 7f5ac259c640 1 --2- 172.21.15.39:0/627329124 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5568c0bd0f60 0x5568c0f7da20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:38.774 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:38.771+0000 7f5ac259c640 1 -- 172.21.15.39:0/627329124 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5568c0f791f0 con 0x5568c0ba80a0 2024-09-21T03:45:38.774 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:38.771+0000 7f5ac2d9d640 1 --2- 172.21.15.39:0/627329124 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5568c0bd0f60 0x5568c0f7da20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:45:38.774 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:38.771+0000 7f5ac259c640 1 --2- 172.21.15.39:0/627329124 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5568c0ba80a0 0x5568c0f82a50 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f5ab400cfd0 tx=0x7f5ab400cab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=539ba87c5237c5d9 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:45:38.774 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:38.771+0000 7f5aabfff640 1 -- 172.21.15.39:0/627329124 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ab40eae60 con 0x5568c0ba80a0 2024-09-21T03:45:38.774 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:38.771+0000 7f5aabfff640 1 -- 172.21.15.39:0/627329124 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5ab40ae380 con 0x5568c0ba80a0 2024-09-21T03:45:38.775 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:38.771+0000 7f5aabfff640 1 -- 172.21.15.39:0/627329124 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ab40e6470 con 0x5568c0ba80a0 2024-09-21T03:45:38.778 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:38.772+0000 7f5acf306740 1 -- 172.21.15.39:0/627329124 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5568c0ba80a0 msgr2=0x5568c0f82a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:38.778 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:38.772+0000 7f5acf306740 1 --2- 172.21.15.39:0/627329124 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5568c0ba80a0 0x5568c0f82a50 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f5ab400cfd0 tx=0x7f5ab400cab0 comp rx=0 tx=0).stop 2024-09-21T03:45:38.778 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:38.772+0000 7f5acf306740 1 -- 172.21.15.39:0/627329124 shutdown_connections 2024-09-21T03:45:38.778 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:38.772+0000 7f5acf306740 1 --2- 172.21.15.39:0/627329124 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5568c0bd0f60 0x5568c0f7da20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:38.778 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:38.772+0000 7f5acf306740 1 --2- 172.21.15.39:0/627329124 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5568c0bb0280 0x5568c0f82f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:38.779 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:38.772+0000 7f5acf306740 1 --2- 172.21.15.39:0/627329124 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5568c0ba80a0 0x5568c0f82a50 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:38.779 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:38.772+0000 7f5acf306740 1 -- 172.21.15.39:0/627329124 >> 172.21.15.39:0/627329124 conn(0x5568c0bc5350 msgr2=0x5568c0f7cd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:45:38.779 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:38.772+0000 7f5acf306740 1 -- 172.21.15.39:0/627329124 shutdown_connections 2024-09-21T03:45:38.779 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:38.772+0000 7f5acf306740 1 -- 172.21.15.39:0/627329124 wait complete. 2024-09-21T03:45:38.779 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:38.772+0000 7f5acf306740 1 -- start start 2024-09-21T03:45:38.779 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:38.772+0000 7f5acf306740 1 Processor -- start 2024-09-21T03:45:38.779 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:38.773+0000 7f5acf306740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5568c0ba80a0 0x5568c0f9d940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:38.779 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:38.773+0000 7f5acf306740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5568c0bb0280 0x5568c0f96df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:38.779 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:38.773+0000 7f5acf306740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5568c0bd0f60 0x5568c0f97330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:38.779 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:38.773+0000 7f5acf306740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x5568c0f94ff0 con 0x5568c0bd0f60 2024-09-21T03:45:38.779 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:38.773+0000 7f5acf306740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x5568c0f94920 con 0x5568c0bb0280 2024-09-21T03:45:38.779 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:38.773+0000 7f5acf306740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x5568c0f956c0 con 0x5568c0ba80a0 2024-09-21T03:45:38.779 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:38.773+0000 7f5acf306740 10 client.0 _ll_register_callbacks cb 0x7f5abc00bf30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:45:38.779 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:38.773+0000 7f5ac2d9d640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5568c0bd0f60 0x5568c0f97330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:38.779 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:38.773+0000 7f5ac2d9d640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5568c0bd0f60 0x5568c0f97330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:55734/0 (socket says 172.21.15.39:55734) 2024-09-21T03:45:38.780 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:38.773+0000 7f5ac2d9d640 1 -- 172.21.15.39:0/2805246585 learned_addr learned my addr 172.21.15.39:0/2805246585 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:45:38.780 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:38.773+0000 7f5ac259c640 1 --2- 172.21.15.39:0/2805246585 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5568c0ba80a0 0x5568c0f9d940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:38.780 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:38.773+0000 7f5ac1d9b640 1 --2- 172.21.15.39:0/2805246585 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5568c0bb0280 0x5568c0f96df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:38.780 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:38.773+0000 7f5ac2d9d640 1 -- 172.21.15.39:0/2805246585 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5568c0ba80a0 msgr2=0x5568c0f9d940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:45:38.780 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:38.773+0000 7f5ac2d9d640 1 --2- 172.21.15.39:0/2805246585 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5568c0ba80a0 0x5568c0f9d940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:38.780 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:38.773+0000 7f5ac2d9d640 1 -- 172.21.15.39:0/2805246585 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5568c0bb0280 msgr2=0x5568c0f96df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:45:38.780 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:38.773+0000 7f5ac2d9d640 1 --2- 172.21.15.39:0/2805246585 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5568c0bb0280 0x5568c0f96df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:38.780 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:38.773+0000 7f5ac2d9d640 1 -- 172.21.15.39:0/2805246585 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5568c0f97870 con 0x5568c0bd0f60 2024-09-21T03:45:38.780 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:38.773+0000 7f5ac2d9d640 1 --2- 172.21.15.39:0/2805246585 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5568c0bd0f60 0x5568c0f97330 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f5abc018040 tx=0x7f5abc004a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4fdf5c9de2f6f833 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:45:38.780 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:38.773+0000 7f5aa9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-09-21T03:45:38.780 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:38.773+0000 7f5aa9ffb640 1 -- 172.21.15.39:0/2805246585 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5abc003a40 con 0x5568c0bd0f60 2024-09-21T03:45:38.780 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:38.773+0000 7f5aa9ffb640 1 -- 172.21.15.39:0/2805246585 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8484+0+0 (secure 0 0 0) 0x7f5abc027a70 con 0x5568c0bd0f60 2024-09-21T03:45:38.780 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:38.773+0000 7f5acf306740 1 -- 172.21.15.39:0/2805246585 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5ab40ae380 con 0x5568c0bd0f60 2024-09-21T03:45:38.781 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:38.774+0000 7f5aa9ffb640 1 -- 172.21.15.39:0/2805246585 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f5abc027c70 con 0x5568c0bd0f60 2024-09-21T03:45:38.781 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:38.774+0000 7f5acf306740 10 client.15312 fetch_fsmap learned FSMap version 28 2024-09-21T03:45:38.781 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:38.774+0000 7f5acf306740 1 -- 172.21.15.39:0/2805246585 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5568c0f9d210 con 0x5568c0bd0f60 2024-09-21T03:45:38.781 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:38.774+0000 7f5aa9ffb640 1 -- 172.21.15.39:0/2805246585 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5ab40ae380 con 0x5568c0bd0f60 2024-09-21T03:45:38.781 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:38.774+0000 7f5aa9ffb640 1 -- 172.21.15.39:0/2805246585 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f5abc040070 con 0x5568c0bd0f60 2024-09-21T03:45:38.781 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:38.774+0000 7f5acf306740 10 client.15312 fetch_fsmap finished waiting for FSMap version 28 2024-09-21T03:45:38.781 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:38.774+0000 7f5acf306740 10 client.15312 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-21T03:45:38.781 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:38.774+0000 7f5acf306740 20 client.15312 populate_metadata read hostname 'smithi039' 2024-09-21T03:45:38.781 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:38.774+0000 7f5acf306740 1 --2- 172.21.15.39:0/2805246585 >> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] conn(0x5568c0b64c90 0x5568c1003c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:38.781 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:38.774+0000 7f5acf306740 4 client.15312 mds_command: new command op to 24695 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-21T03:45:38.781 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:38.774+0000 7f5acf306740 1 -- 172.21.15.39:0/2805246585 --> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] -- command(tid 0: {"prefix": "status"}) -- 0x5568c0eec610 con 0x5568c0b64c90 2024-09-21T03:45:38.781 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:38.775+0000 7f5ac259c640 1 --2- 172.21.15.39:0/2805246585 >> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] conn(0x5568c0b64c90 0x5568c1003c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:38.781 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:38.775+0000 7f5aa9ffb640 10 client.15312 ms_handle_connect on v2:172.21.15.150:6838/2949386907 2024-09-21T03:45:38.782 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:38.775+0000 7f5ac259c640 1 --2- 172.21.15.39:0/2805246585 >> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] conn(0x5568c0b64c90 0x5568c1003c40 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3a5ce2e1e6f3fa7e server_cookie=9c720c6415516e92 in_seq=0 out_seq=0 2024-09-21T03:45:38.782 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:38.775+0000 7f5aa9ffb640 1 -- 172.21.15.39:0/2805246585 <== mds.0 v2:172.21.15.150:6838/2949386907 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x5568c0eec610 con 0x5568c0b64c90 2024-09-21T03:45:38.782 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:38.775+0000 7f5aa9ffb640 10 client.15312 handle_command_reply: tid=0 2024-09-21T03:45:38.782 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:38.775+0000 7f5acf306740 1 client.15312 shutdown 2024-09-21T03:45:38.782 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:38.775+0000 7f5acf306740 1 -- 172.21.15.39:0/2805246585 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5568c0bd0f60 msgr2=0x5568c0f97330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:38.782 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:38.775+0000 7f5acf306740 1 --2- 172.21.15.39:0/2805246585 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5568c0bd0f60 0x5568c0f97330 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f5abc018040 tx=0x7f5abc004a50 comp rx=0 tx=0).stop 2024-09-21T03:45:38.782 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:38.775+0000 7f5acf306740 1 -- 172.21.15.39:0/2805246585 shutdown_connections 2024-09-21T03:45:38.782 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:38.775+0000 7f5acf306740 1 --2- 172.21.15.39:0/2805246585 >> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] conn(0x5568c0b64c90 0x5568c1003c40 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:38.782 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:38.775+0000 7f5acf306740 1 --2- 172.21.15.39:0/2805246585 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5568c0bd0f60 0x5568c0f97330 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:38.782 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:38.775+0000 7f5acf306740 1 --2- 172.21.15.39:0/2805246585 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5568c0bb0280 0x5568c0f96df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:38.782 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:38.775+0000 7f5acf306740 1 --2- 172.21.15.39:0/2805246585 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5568c0ba80a0 0x5568c0f9d940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:38.782 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:38.775+0000 7f5aa9ffb640 0 client.15312 ms_handle_reset on v2:172.21.15.150:6838/2949386907 2024-09-21T03:45:38.782 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:38.775+0000 7f5acf306740 1 -- 172.21.15.39:0/2805246585 >> 172.21.15.39:0/2805246585 conn(0x5568c0bc5350 msgr2=0x5568c0f7da20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:45:38.783 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:38.776+0000 7f5acf306740 1 -- 172.21.15.39:0/2805246585 shutdown_connections 2024-09-21T03:45:38.783 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:38.776+0000 7f5acf306740 1 -- 172.21.15.39:0/2805246585 wait complete. 2024-09-21T03:45:38.783 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:38.776+0000 7f5acf306740 20 client.15312 trim_cache size 0 max 16384 2024-09-21T03:45:38.783 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-09-21T03:45:38.783 INFO:teuthology.orchestra.run.smithi039.stdout: "cluster_fsid": "51116d5c-77ca-11ef-baf5-efdc52797490", 2024-09-21T03:45:38.783 INFO:teuthology.orchestra.run.smithi039.stdout: "whoami": 0, 2024-09-21T03:45:38.783 INFO:teuthology.orchestra.run.smithi039.stdout: "id": 24695, 2024-09-21T03:45:38.783 INFO:teuthology.orchestra.run.smithi039.stdout: "want_state": "up:active", 2024-09-21T03:45:38.783 INFO:teuthology.orchestra.run.smithi039.stdout: "state": "up:active", 2024-09-21T03:45:38.783 INFO:teuthology.orchestra.run.smithi039.stdout: "fs_name": "cephfs", 2024-09-21T03:45:38.783 INFO:teuthology.orchestra.run.smithi039.stdout: "rank_uptime": 83.444505282999998, 2024-09-21T03:45:38.784 INFO:teuthology.orchestra.run.smithi039.stdout: "mdsmap_epoch": 28, 2024-09-21T03:45:38.784 INFO:teuthology.orchestra.run.smithi039.stdout: "osdmap_epoch": 79, 2024-09-21T03:45:38.784 INFO:teuthology.orchestra.run.smithi039.stdout: "osdmap_epoch_barrier": 78, 2024-09-21T03:45:38.784 INFO:teuthology.orchestra.run.smithi039.stdout: "uptime": 93.919850100999994 2024-09-21T03:45:38.784 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-09-21T03:45:38.784 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:38.776+0000 7f5ac8d92640 1 -- 172.21.15.39:0/842288765 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f5aac0b6670 msgr2=0x7f5aac0b8b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:38.784 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:38.776+0000 7f5ac8d92640 1 --2- 172.21.15.39:0/842288765 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f5aac0b6670 0x7f5aac0b8b80 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f5abc00b7f0 tx=0x7f5abc00b2e0 comp rx=0 tx=0).stop 2024-09-21T03:45:38.785 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:38.776+0000 7f5ac8d92640 1 -- 172.21.15.39:0/842288765 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5ac4102a70 msgr2=0x7f5ac4199360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:38.785 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:38.776+0000 7f5ac8d92640 1 --2- 172.21.15.39:0/842288765 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5ac4102a70 0x7f5ac4199360 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f5ab400ebc0 tx=0x7f5ab4007590 comp rx=0 tx=0).stop 2024-09-21T03:45:38.785 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:38.776+0000 7f5ac8d92640 1 -- 172.21.15.39:0/842288765 shutdown_connections 2024-09-21T03:45:38.785 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:38.776+0000 7f5ac8d92640 1 --2- 172.21.15.39:0/842288765 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f5aac0b6670 0x7f5aac0b8b80 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:38.785 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:38.776+0000 7f5ac8d92640 1 --2- 172.21.15.39:0/842288765 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5ac4108d00 0x7f5ac419eba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:38.785 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:38.776+0000 7f5ac8d92640 1 --2- 172.21.15.39:0/842288765 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5ac41083b0 0x7f5ac4199900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:38.785 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:38.776+0000 7f5ac8d92640 1 --2- 172.21.15.39:0/842288765 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5ac4102a70 0x7f5ac4199360 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:38.785 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:38.776+0000 7f5ac8d92640 1 -- 172.21.15.39:0/842288765 >> 172.21.15.39:0/842288765 conn(0x7f5ac4079730 msgr2=0x7f5ac4104480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:45:38.785 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:38.776+0000 7f5ac8d92640 1 -- 172.21.15.39:0/842288765 shutdown_connections 2024-09-21T03:45:38.785 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:38.776+0000 7f5ac8d92640 1 -- 172.21.15.39:0/842288765 wait complete. 2024-09-21T03:45:39.084 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:45:38 smithi039 ceph-mon[31211]: pgmap v313: 97 pgs: 97 active+clean; 647 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-09-21T03:45:39.217 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:45:38 smithi161 ceph-mon[34533]: pgmap v313: 97 pgs: 97 active+clean; 647 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-09-21T03:45:39.224 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:45:38 smithi150 ceph-mon[34289]: pgmap v313: 97 pgs: 97 active+clean; 647 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-09-21T03:45:39.655 INFO:tasks.cephadm:Setup complete, yielding 2024-09-21T03:45:39.655 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-09-21T03:45:39.665 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi039.front.sepia.ceph.com 2024-09-21T03:45:39.666 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 51116d5c-77ca-11ef-baf5-efdc52797490 -- bash -c 'ceph orch status' 2024-09-21T03:45:39.922 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/51116d5c-77ca-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:45:40.381 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:40.359+0000 7f173f935640 1 -- 172.21.15.39:0/3000549920 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1738103b80 msgr2=0x7f1738103f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:40.381 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:40.359+0000 7f173f935640 1 --2- 172.21.15.39:0/3000549920 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1738103b80 0x7f1738103f90 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f1734009900 tx=0x7f173402a410 comp rx=0 tx=0).stop 2024-09-21T03:45:40.381 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:40.359+0000 7f173f935640 1 -- 172.21.15.39:0/3000549920 shutdown_connections 2024-09-21T03:45:40.381 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:40.359+0000 7f173f935640 1 --2- 172.21.15.39:0/3000549920 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1738109e10 0x7f1738110640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:40.381 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:40.359+0000 7f173f935640 1 --2- 172.21.15.39:0/3000549920 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f17381094c0 0x7f17381098d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:40.381 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:40.359+0000 7f173f935640 1 --2- 172.21.15.39:0/3000549920 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1738103b80 0x7f1738103f90 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:40.381 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:40.359+0000 7f173f935640 1 -- 172.21.15.39:0/3000549920 >> 172.21.15.39:0/3000549920 conn(0x7f17380ff610 msgr2=0x7f1738101a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:45:40.381 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:40.359+0000 7f173f935640 1 -- 172.21.15.39:0/3000549920 shutdown_connections 2024-09-21T03:45:40.381 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:40.359+0000 7f173f935640 1 -- 172.21.15.39:0/3000549920 wait complete. 2024-09-21T03:45:40.381 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:40.360+0000 7f173f935640 1 Processor -- start 2024-09-21T03:45:40.382 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:40.360+0000 7f173f935640 1 -- start start 2024-09-21T03:45:40.382 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:40.360+0000 7f173f935640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1738103b80 0x7f1738075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:40.382 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:40.360+0000 7f173f935640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f17381094c0 0x7f1738075c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:40.382 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:40.360+0000 7f173f935640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1738109e10 0x7f1738076190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:40.382 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:40.360+0000 7f173f935640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f1738113ad0 con 0x7f1738109e10 2024-09-21T03:45:40.382 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:40.360+0000 7f173f935640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f1738113950 con 0x7f1738103b80 2024-09-21T03:45:40.382 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:40.360+0000 7f173f935640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f1738113c50 con 0x7f17381094c0 2024-09-21T03:45:40.382 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:40.360+0000 7f173ded2640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1738109e10 0x7f1738076190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:40.382 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:40.360+0000 7f173ded2640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1738109e10 0x7f1738076190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:55748/0 (socket says 172.21.15.39:55748) 2024-09-21T03:45:40.382 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:40.360+0000 7f173ded2640 1 -- 172.21.15.39:0/1615385807 learned_addr learned my addr 172.21.15.39:0/1615385807 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:45:40.382 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:40.361+0000 7f173d6d1640 1 --2- 172.21.15.39:0/1615385807 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1738103b80 0x7f1738075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:40.382 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:40.361+0000 7f173ced0640 1 --2- 172.21.15.39:0/1615385807 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f17381094c0 0x7f1738075c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:40.382 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:40.361+0000 7f173ded2640 1 -- 172.21.15.39:0/1615385807 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f17381094c0 msgr2=0x7f1738075c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:40.383 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:40.361+0000 7f173ded2640 1 --2- 172.21.15.39:0/1615385807 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f17381094c0 0x7f1738075c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:40.383 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:40.361+0000 7f173ded2640 1 -- 172.21.15.39:0/1615385807 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1738103b80 msgr2=0x7f1738075710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:40.383 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:40.361+0000 7f173ded2640 1 --2- 172.21.15.39:0/1615385807 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1738103b80 0x7f1738075710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:40.383 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:40.361+0000 7f173ded2640 1 -- 172.21.15.39:0/1615385807 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f17381a87a0 con 0x7f1738109e10 2024-09-21T03:45:40.383 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:40.361+0000 7f173d6d1640 1 --2- 172.21.15.39:0/1615385807 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1738103b80 0x7f1738075710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:45:40.383 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:40.361+0000 7f173ded2640 1 --2- 172.21.15.39:0/1615385807 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1738109e10 0x7f1738076190 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f173000eaa0 tx=0x7f173000ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:45:40.383 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:40.361+0000 7f172e7fc640 1 -- 172.21.15.39:0/1615385807 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f173000cdb0 con 0x7f1738109e10 2024-09-21T03:45:40.383 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:40.361+0000 7f173f935640 1 -- 172.21.15.39:0/1615385807 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f17381a8a90 con 0x7f1738109e10 2024-09-21T03:45:40.383 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:40.361+0000 7f172e7fc640 1 -- 172.21.15.39:0/1615385807 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1730005020 con 0x7f1738109e10 2024-09-21T03:45:40.383 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:40.361+0000 7f172e7fc640 1 -- 172.21.15.39:0/1615385807 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17300116d0 con 0x7f1738109e10 2024-09-21T03:45:40.383 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:40.361+0000 7f173f935640 1 -- 172.21.15.39:0/1615385807 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f17381a8f80 con 0x7f1738109e10 2024-09-21T03:45:40.383 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:40.362+0000 7f173f935640 1 -- 172.21.15.39:0/1615385807 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1738103f90 con 0x7f1738109e10 2024-09-21T03:45:40.383 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:40.365+0000 7f172e7fc640 1 -- 172.21.15.39:0/1615385807 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 22) ==== 154580+0+0 (secure 0 0 0) 0x7f17300051d0 con 0x7f1738109e10 2024-09-21T03:45:40.384 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:40.366+0000 7f172e7fc640 1 --2- 172.21.15.39:0/1615385807 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f17240b65f0 0x7f17240b8b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:40.384 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:40.366+0000 7f173d6d1640 1 --2- 172.21.15.39:0/1615385807 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f17240b65f0 0x7f17240b8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:40.384 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:40.366+0000 7f172e7fc640 1 -- 172.21.15.39:0/1615385807 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8484+0+0 (secure 0 0 0) 0x7f17300e7cd0 con 0x7f1738109e10 2024-09-21T03:45:40.384 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:40.367+0000 7f173d6d1640 1 --2- 172.21.15.39:0/1615385807 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f17240b65f0 0x7f17240b8b00 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f1734000c00 tx=0x7f1734031400 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:45:40.384 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:40.369+0000 7f172e7fc640 1 -- 172.21.15.39:0/1615385807 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f1730016030 con 0x7f1738109e10 2024-09-21T03:45:40.579 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:40.577+0000 7f173f935640 1 -- 172.21.15.39:0/1615385807 --> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] -- mgr_command(tid 0: {"prefix": "orch status", "target": ["mon-mgr", ""]}) -- 0x7f1738063800 con 0x7f17240b65f0 2024-09-21T03:45:40.580 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:40.578+0000 7f172e7fc640 1 -- 172.21.15.39:0/1615385807 <== mgr.14223 v2:172.21.15.39:6800/1696381982 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+43 (secure 0 0 0) 0x7f1738063800 con 0x7f17240b65f0 2024-09-21T03:45:40.580 INFO:teuthology.orchestra.run.smithi039.stdout:Backend: cephadm 2024-09-21T03:45:40.580 INFO:teuthology.orchestra.run.smithi039.stdout:Available: Yes 2024-09-21T03:45:40.580 INFO:teuthology.orchestra.run.smithi039.stdout:Paused: No 2024-09-21T03:45:40.583 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:40.581+0000 7f173f935640 1 -- 172.21.15.39:0/1615385807 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f17240b65f0 msgr2=0x7f17240b8b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:40.583 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:40.581+0000 7f173f935640 1 --2- 172.21.15.39:0/1615385807 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f17240b65f0 0x7f17240b8b00 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f1734000c00 tx=0x7f1734031400 comp rx=0 tx=0).stop 2024-09-21T03:45:40.583 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:40.581+0000 7f173f935640 1 -- 172.21.15.39:0/1615385807 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1738109e10 msgr2=0x7f1738076190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:40.583 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:40.581+0000 7f173f935640 1 --2- 172.21.15.39:0/1615385807 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1738109e10 0x7f1738076190 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f173000eaa0 tx=0x7f173000ef60 comp rx=0 tx=0).stop 2024-09-21T03:45:40.583 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:40.581+0000 7f173f935640 1 -- 172.21.15.39:0/1615385807 shutdown_connections 2024-09-21T03:45:40.583 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:40.581+0000 7f173f935640 1 --2- 172.21.15.39:0/1615385807 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f17240b65f0 0x7f17240b8b00 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:40.583 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:40.581+0000 7f173f935640 1 --2- 172.21.15.39:0/1615385807 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1738109e10 0x7f1738076190 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:40.583 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:40.581+0000 7f173f935640 1 --2- 172.21.15.39:0/1615385807 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f17381094c0 0x7f1738075c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:40.583 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:40.581+0000 7f173f935640 1 --2- 172.21.15.39:0/1615385807 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1738103b80 0x7f1738075710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:40.583 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:40.581+0000 7f173f935640 1 -- 172.21.15.39:0/1615385807 >> 172.21.15.39:0/1615385807 conn(0x7f17380ff610 msgr2=0x7f1738101a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:45:40.584 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:40.581+0000 7f173f935640 1 -- 172.21.15.39:0/1615385807 shutdown_connections 2024-09-21T03:45:40.584 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:40.581+0000 7f173f935640 1 -- 172.21.15.39:0/1615385807 wait complete. 2024-09-21T03:45:41.192 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:45:40 smithi039 ceph-mon[31211]: pgmap v314: 97 pgs: 97 active+clean; 647 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-09-21T03:45:41.217 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:45:40 smithi161 ceph-mon[34533]: pgmap v314: 97 pgs: 97 active+clean; 647 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-09-21T03:45:41.220 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 51116d5c-77ca-11ef-baf5-efdc52797490 -- bash -c 'ceph orch ps' 2024-09-21T03:45:41.224 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:45:40 smithi150 ceph-mon[34289]: pgmap v314: 97 pgs: 97 active+clean; 647 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-09-21T03:45:41.480 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/51116d5c-77ca-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:45:41.971 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:41.964+0000 7ffb81a47640 1 -- 172.21.15.39:0/1073059771 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ffb7c102a60 msgr2=0x7ffb7c102e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:41.971 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:41.964+0000 7ffb81a47640 1 --2- 172.21.15.39:0/1073059771 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ffb7c102a60 0x7ffb7c102e70 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7ffb6c009900 tx=0x7ffb6c02a410 comp rx=0 tx=0).stop 2024-09-21T03:45:41.971 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:41.964+0000 7ffb81a47640 1 -- 172.21.15.39:0/1073059771 shutdown_connections 2024-09-21T03:45:41.971 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:41.964+0000 7ffb81a47640 1 --2- 172.21.15.39:0/1073059771 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ffb7c108cf0 0x7ffb7c10f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:41.971 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:41.964+0000 7ffb81a47640 1 --2- 172.21.15.39:0/1073059771 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ffb7c1083a0 0x7ffb7c1087b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:41.972 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:41.964+0000 7ffb81a47640 1 --2- 172.21.15.39:0/1073059771 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ffb7c102a60 0x7ffb7c102e70 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:41.972 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:41.964+0000 7ffb81a47640 1 -- 172.21.15.39:0/1073059771 >> 172.21.15.39:0/1073059771 conn(0x7ffb7c079730 msgr2=0x7ffb7c079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:45:41.972 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:41.964+0000 7ffb81a47640 1 -- 172.21.15.39:0/1073059771 shutdown_connections 2024-09-21T03:45:41.972 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:41.964+0000 7ffb81a47640 1 -- 172.21.15.39:0/1073059771 wait complete. 2024-09-21T03:45:41.972 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:41.965+0000 7ffb81a47640 1 Processor -- start 2024-09-21T03:45:41.972 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:41.965+0000 7ffb81a47640 1 -- start start 2024-09-21T03:45:41.972 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:41.965+0000 7ffb81a47640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ffb7c102a60 0x7ffb7c104250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:41.972 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:41.965+0000 7ffb81a47640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ffb7c1083a0 0x7ffb7c104790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:41.972 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:41.965+0000 7ffb81a47640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ffb7c108cf0 0x7ffb7c104cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:41.972 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:41.965+0000 7ffb81a47640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7ffb7c1129c0 con 0x7ffb7c102a60 2024-09-21T03:45:41.972 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:41.965+0000 7ffb81a47640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7ffb7c112840 con 0x7ffb7c108cf0 2024-09-21T03:45:41.972 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:41.965+0000 7ffb81a47640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7ffb7c112b40 con 0x7ffb7c1083a0 2024-09-21T03:45:41.973 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:41.966+0000 7ffb7affd640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ffb7c102a60 0x7ffb7c104250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:41.973 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:41.966+0000 7ffb7affd640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ffb7c102a60 0x7ffb7c104250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:55762/0 (socket says 172.21.15.39:55762) 2024-09-21T03:45:41.973 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:41.966+0000 7ffb7affd640 1 -- 172.21.15.39:0/1003630003 learned_addr learned my addr 172.21.15.39:0/1003630003 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:45:41.973 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:41.966+0000 7ffb7a7fc640 1 --2- 172.21.15.39:0/1003630003 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ffb7c1083a0 0x7ffb7c104790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:41.973 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:41.966+0000 7ffb7affd640 1 -- 172.21.15.39:0/1003630003 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ffb7c1083a0 msgr2=0x7ffb7c104790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:41.973 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:41.966+0000 7ffb7affd640 1 --2- 172.21.15.39:0/1003630003 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ffb7c1083a0 0x7ffb7c104790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:41.973 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:41.966+0000 7ffb7affd640 1 -- 172.21.15.39:0/1003630003 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ffb7c108cf0 msgr2=0x7ffb7c104cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:41.973 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:41.966+0000 7ffb7b7fe640 1 --2- 172.21.15.39:0/1003630003 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ffb7c108cf0 0x7ffb7c104cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:41.973 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:41.966+0000 7ffb7affd640 1 --2- 172.21.15.39:0/1003630003 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ffb7c108cf0 0x7ffb7c104cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:41.973 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:41.966+0000 7ffb7affd640 1 -- 172.21.15.39:0/1003630003 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffb7c1a86d0 con 0x7ffb7c102a60 2024-09-21T03:45:41.973 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:41.966+0000 7ffb7b7fe640 1 --2- 172.21.15.39:0/1003630003 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ffb7c108cf0 0x7ffb7c104cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:45:41.973 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:41.966+0000 7ffb7affd640 1 --2- 172.21.15.39:0/1003630003 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ffb7c102a60 0x7ffb7c104250 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7ffb6c006010 tx=0x7ffb6c0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:45:41.973 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:41.966+0000 7ffb80a45640 1 -- 172.21.15.39:0/1003630003 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb6c03f030 con 0x7ffb7c102a60 2024-09-21T03:45:41.973 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:41.966+0000 7ffb80a45640 1 -- 172.21.15.39:0/1003630003 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ffb6c003800 con 0x7ffb7c102a60 2024-09-21T03:45:41.974 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:41.966+0000 7ffb81a47640 1 -- 172.21.15.39:0/1003630003 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffb7c1a8960 con 0x7ffb7c102a60 2024-09-21T03:45:41.974 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:41.966+0000 7ffb80a45640 1 -- 172.21.15.39:0/1003630003 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb6c035070 con 0x7ffb7c102a60 2024-09-21T03:45:41.974 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:41.966+0000 7ffb81a47640 1 -- 172.21.15.39:0/1003630003 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffb7c1a8ec0 con 0x7ffb7c102a60 2024-09-21T03:45:41.974 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:41.967+0000 7ffb81a47640 1 -- 172.21.15.39:0/1003630003 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffb7c102e70 con 0x7ffb7c102a60 2024-09-21T03:45:41.974 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:41.969+0000 7ffb80a45640 1 -- 172.21.15.39:0/1003630003 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 22) ==== 154580+0+0 (secure 0 0 0) 0x7ffb6c003cb0 con 0x7ffb7c102a60 2024-09-21T03:45:41.975 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:41.970+0000 7ffb80a45640 1 --2- 172.21.15.39:0/1003630003 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7ffb640b65f0 0x7ffb640b8b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:41.975 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:41.970+0000 7ffb7a7fc640 1 --2- 172.21.15.39:0/1003630003 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7ffb640b65f0 0x7ffb640b8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:41.975 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:41.970+0000 7ffb80a45640 1 -- 172.21.15.39:0/1003630003 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8484+0+0 (secure 0 0 0) 0x7ffb6c108280 con 0x7ffb7c102a60 2024-09-21T03:45:41.975 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:41.971+0000 7ffb7a7fc640 1 --2- 172.21.15.39:0/1003630003 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7ffb640b65f0 0x7ffb640b8b00 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7ffb7c100d20 tx=0x7ffb70008040 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:45:41.975 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:41.973+0000 7ffb80a45640 1 -- 172.21.15.39:0/1003630003 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7ffb6c0ce8b0 con 0x7ffb7c102a60 2024-09-21T03:45:42.170 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:42.168+0000 7ffb81a47640 1 -- 172.21.15.39:0/1003630003 --> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) -- 0x7ffb7c06aeb0 con 0x7ffb640b65f0 2024-09-21T03:45:42.170 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:45:41 smithi039 ceph-mon[31211]: from='client.15318 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:45:42.182 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:42.181+0000 7ffb80a45640 1 -- 172.21.15.39:0/1003630003 <== mgr.14223 v2:172.21.15.39:6800/1696381982 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+4185 (secure 0 0 0) 0x7ffb7c06aeb0 con 0x7ffb640b65f0 2024-09-21T03:45:42.182 INFO:teuthology.orchestra.run.smithi039.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-09-21T03:45:42.183 INFO:teuthology.orchestra.run.smithi039.stdout:mds.a smithi039 running (100s) 20s ago 99s 85.8M - 19.3.0-5120-g18432218 ddce307e1f32 4d6edea429af 2024-09-21T03:45:42.183 INFO:teuthology.orchestra.run.smithi039.stdout:mds.b smithi150 running (97s) 31s ago 97s 95.7M - 19.3.0-5120-g18432218 ddce307e1f32 5600e78780f5 2024-09-21T03:45:42.183 INFO:teuthology.orchestra.run.smithi039.stdout:mds.c smithi161 running (111s) 31s ago 111s 88.1M - 19.3.0-5120-g18432218 ddce307e1f32 21811c0fe34e 2024-09-21T03:45:42.183 INFO:teuthology.orchestra.run.smithi039.stdout:mds.d smithi039 running (118s) 20s ago 118s 82.0M - 19.3.0-5120-g18432218 ddce307e1f32 d295f08191b8 2024-09-21T03:45:42.183 INFO:teuthology.orchestra.run.smithi039.stdout:mds.e smithi150 running (116s) 31s ago 115s 85.4M - 19.3.0-5120-g18432218 ddce307e1f32 fd7950219c69 2024-09-21T03:45:42.183 INFO:teuthology.orchestra.run.smithi039.stdout:mds.f smithi161 running (113s) 31s ago 113s 88.2M - 19.3.0-5120-g18432218 ddce307e1f32 eb6e976e5fd4 2024-09-21T03:45:42.183 INFO:teuthology.orchestra.run.smithi039.stdout:mds.g smithi039 running (103s) 20s ago 103s 87.0M - 19.3.0-5120-g18432218 ddce307e1f32 c75c7c7aee90 2024-09-21T03:45:42.183 INFO:teuthology.orchestra.run.smithi039.stdout:mds.h smithi150 running (108s) 31s ago 108s 86.5M - 19.3.0-5120-g18432218 ddce307e1f32 28ae457df36b 2024-09-21T03:45:42.183 INFO:teuthology.orchestra.run.smithi039.stdout:mds.i smithi161 running (106s) 31s ago 105s 86.1M - 19.3.0-5120-g18432218 ddce307e1f32 97ce66e526b2 2024-09-21T03:45:42.183 INFO:teuthology.orchestra.run.smithi039.stdout:mds.j smithi039 running (2m) 20s ago 2m 82.4M - 19.3.0-5120-g18432218 ddce307e1f32 ae337640ab42 2024-09-21T03:45:42.183 INFO:teuthology.orchestra.run.smithi039.stdout:mds.k smithi150 running (2m) 31s ago 2m 84.5M - 19.3.0-5120-g18432218 ddce307e1f32 f4cfa3dbe0d4 2024-09-21T03:45:42.183 INFO:teuthology.orchestra.run.smithi039.stdout:mds.l smithi161 running (2m) 31s ago 2m 84.9M - 19.3.0-5120-g18432218 ddce307e1f32 988b75038bf2 2024-09-21T03:45:42.183 INFO:teuthology.orchestra.run.smithi039.stdout:mgr.x smithi039 *:9283,8765 running (10m) 20s ago 10m 585M - 19.3.0-5120-g18432218 ddce307e1f32 622e116e4f4c 2024-09-21T03:45:42.183 INFO:teuthology.orchestra.run.smithi039.stdout:mgr.y smithi150 *:8443,8765 running (8m) 31s ago 8m 527M - 19.3.0-5120-g18432218 ddce307e1f32 5ebc4c7778bc 2024-09-21T03:45:42.183 INFO:teuthology.orchestra.run.smithi039.stdout:mgr.z smithi161 *:8443,8765 running (8m) 31s ago 8m 523M - 19.3.0-5120-g18432218 ddce307e1f32 4f7e5e425925 2024-09-21T03:45:42.184 INFO:teuthology.orchestra.run.smithi039.stdout:mon.a smithi039 running (10m) 20s ago 10m 123M 2048M 19.3.0-5120-g18432218 ddce307e1f32 861e490fc665 2024-09-21T03:45:42.184 INFO:teuthology.orchestra.run.smithi039.stdout:mon.b smithi150 running (8m) 31s ago 8m 110M 2048M 19.3.0-5120-g18432218 ddce307e1f32 6cc60c3531e0 2024-09-21T03:45:42.184 INFO:teuthology.orchestra.run.smithi039.stdout:mon.c smithi161 running (8m) 31s ago 8m 110M 2048M 19.3.0-5120-g18432218 ddce307e1f32 4ea176de02d9 2024-09-21T03:45:42.184 INFO:teuthology.orchestra.run.smithi039.stdout:osd.0 smithi039 running (7m) 20s ago 7m 115M 4272M 19.3.0-5120-g18432218 ddce307e1f32 fa4243328756 2024-09-21T03:45:42.184 INFO:teuthology.orchestra.run.smithi039.stdout:osd.1 smithi150 running (6m) 31s ago 6m 116M 4272M 19.3.0-5120-g18432218 ddce307e1f32 c075cba916a0 2024-09-21T03:45:42.184 INFO:teuthology.orchestra.run.smithi039.stdout:osd.2 smithi161 running (6m) 31s ago 6m 116M 4272M 19.3.0-5120-g18432218 ddce307e1f32 f6a27d40f337 2024-09-21T03:45:42.184 INFO:teuthology.orchestra.run.smithi039.stdout:osd.3 smithi039 running (5m) 20s ago 5m 122M 4272M 19.3.0-5120-g18432218 ddce307e1f32 f836a786f909 2024-09-21T03:45:42.184 INFO:teuthology.orchestra.run.smithi039.stdout:osd.4 smithi150 running (5m) 31s ago 5m 119M 4272M 19.3.0-5120-g18432218 ddce307e1f32 8fcc733b9e66 2024-09-21T03:45:42.184 INFO:teuthology.orchestra.run.smithi039.stdout:osd.5 smithi161 running (5m) 31s ago 5m 122M 4272M 19.3.0-5120-g18432218 ddce307e1f32 23913b296872 2024-09-21T03:45:42.184 INFO:teuthology.orchestra.run.smithi039.stdout:osd.6 smithi039 running (4m) 20s ago 4m 112M 4272M 19.3.0-5120-g18432218 ddce307e1f32 1cac07383877 2024-09-21T03:45:42.184 INFO:teuthology.orchestra.run.smithi039.stdout:osd.7 smithi150 running (4m) 31s ago 4m 127M 4272M 19.3.0-5120-g18432218 ddce307e1f32 0ca8bb1614c1 2024-09-21T03:45:42.184 INFO:teuthology.orchestra.run.smithi039.stdout:osd.8 smithi161 running (3m) 31s ago 3m 116M 4272M 19.3.0-5120-g18432218 ddce307e1f32 6f8b1dd48857 2024-09-21T03:45:42.184 INFO:teuthology.orchestra.run.smithi039.stdout:osd.9 smithi039 running (3m) 20s ago 3m 112M 4272M 19.3.0-5120-g18432218 ddce307e1f32 71eed5410760 2024-09-21T03:45:42.184 INFO:teuthology.orchestra.run.smithi039.stdout:osd.10 smithi150 running (2m) 31s ago 2m 112M 4272M 19.3.0-5120-g18432218 ddce307e1f32 5e25d378fe9b 2024-09-21T03:45:42.185 INFO:teuthology.orchestra.run.smithi039.stdout:osd.11 smithi161 running (2m) 31s ago 2m 111M 4272M 19.3.0-5120-g18432218 ddce307e1f32 3b22bc180075 2024-09-21T03:45:42.185 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:42.183+0000 7ffb81a47640 1 -- 172.21.15.39:0/1003630003 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7ffb640b65f0 msgr2=0x7ffb640b8b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:42.185 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:42.183+0000 7ffb81a47640 1 --2- 172.21.15.39:0/1003630003 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7ffb640b65f0 0x7ffb640b8b00 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7ffb7c100d20 tx=0x7ffb70008040 comp rx=0 tx=0).stop 2024-09-21T03:45:42.186 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:42.183+0000 7ffb81a47640 1 -- 172.21.15.39:0/1003630003 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ffb7c102a60 msgr2=0x7ffb7c104250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:42.186 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:42.183+0000 7ffb81a47640 1 --2- 172.21.15.39:0/1003630003 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ffb7c102a60 0x7ffb7c104250 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7ffb6c006010 tx=0x7ffb6c0042f0 comp rx=0 tx=0).stop 2024-09-21T03:45:42.186 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:42.184+0000 7ffb81a47640 1 -- 172.21.15.39:0/1003630003 shutdown_connections 2024-09-21T03:45:42.186 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:42.184+0000 7ffb81a47640 1 --2- 172.21.15.39:0/1003630003 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7ffb640b65f0 0x7ffb640b8b00 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:42.186 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:42.184+0000 7ffb81a47640 1 --2- 172.21.15.39:0/1003630003 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ffb7c108cf0 0x7ffb7c104cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:42.186 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:42.184+0000 7ffb81a47640 1 --2- 172.21.15.39:0/1003630003 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ffb7c1083a0 0x7ffb7c104790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:42.186 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:42.184+0000 7ffb81a47640 1 --2- 172.21.15.39:0/1003630003 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ffb7c102a60 0x7ffb7c104250 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:42.186 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:42.184+0000 7ffb81a47640 1 -- 172.21.15.39:0/1003630003 >> 172.21.15.39:0/1003630003 conn(0x7ffb7c079730 msgr2=0x7ffb7c101af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:45:42.186 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:42.184+0000 7ffb81a47640 1 -- 172.21.15.39:0/1003630003 shutdown_connections 2024-09-21T03:45:42.186 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:42.184+0000 7ffb81a47640 1 -- 172.21.15.39:0/1003630003 wait complete. 2024-09-21T03:45:42.217 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:45:41 smithi161 ceph-mon[34533]: from='client.15318 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:45:42.224 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:45:41 smithi150 ceph-mon[34289]: from='client.15318 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:45:42.853 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 51116d5c-77ca-11ef-baf5-efdc52797490 -- bash -c 'ceph orch ls' 2024-09-21T03:45:43.083 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:45:42 smithi039 ceph-mon[31211]: pgmap v315: 97 pgs: 97 active+clean; 647 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-21T03:45:43.084 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:45:42 smithi039 ceph-mon[31211]: from='client.15324 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:45:43.112 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/51116d5c-77ca-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:45:43.218 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:45:42 smithi161 ceph-mon[34533]: pgmap v315: 97 pgs: 97 active+clean; 647 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-21T03:45:43.218 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:45:42 smithi161 ceph-mon[34533]: from='client.15324 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:45:43.224 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:45:42 smithi150 ceph-mon[34289]: pgmap v315: 97 pgs: 97 active+clean; 647 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-21T03:45:43.224 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:45:42 smithi150 ceph-mon[34289]: from='client.15324 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:45:43.639 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:43.637+0000 7f6698bae640 1 -- 172.21.15.39:0/108616458 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f66941094a0 msgr2=0x7f66941098b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:43.640 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:43.637+0000 7f6698bae640 1 --2- 172.21.15.39:0/108616458 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f66941094a0 0x7f66941098b0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f667c009900 tx=0x7f667c02a410 comp rx=0 tx=0).stop 2024-09-21T03:45:43.640 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:43.637+0000 7f6698bae640 1 -- 172.21.15.39:0/108616458 shutdown_connections 2024-09-21T03:45:43.640 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:43.637+0000 7f6698bae640 1 --2- 172.21.15.39:0/108616458 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6694109df0 0x7f6694110620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:43.640 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:43.637+0000 7f6698bae640 1 --2- 172.21.15.39:0/108616458 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f66941094a0 0x7f66941098b0 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:43.640 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:43.637+0000 7f6698bae640 1 --2- 172.21.15.39:0/108616458 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6694103b60 0x7f6694103f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:43.640 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:43.637+0000 7f6698bae640 1 -- 172.21.15.39:0/108616458 >> 172.21.15.39:0/108616458 conn(0x7f669407b850 msgr2=0x7f669407bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:45:43.640 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:43.637+0000 7f6698bae640 1 -- 172.21.15.39:0/108616458 shutdown_connections 2024-09-21T03:45:43.640 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:43.637+0000 7f6698bae640 1 -- 172.21.15.39:0/108616458 wait complete. 2024-09-21T03:45:43.641 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:43.638+0000 7f6698bae640 1 Processor -- start 2024-09-21T03:45:43.641 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:43.638+0000 7f6698bae640 1 -- start start 2024-09-21T03:45:43.643 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:43.638+0000 7f6698bae640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6694103b60 0x7f6694075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:43.643 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:43.638+0000 7f6698bae640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f66941094a0 0x7f6694075c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:43.643 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:43.638+0000 7f6698bae640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6694109df0 0x7f6694076190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:43.643 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:43.638+0000 7f6698bae640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f6694113ab0 con 0x7f66941094a0 2024-09-21T03:45:43.643 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:43.638+0000 7f6698bae640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f6694113930 con 0x7f6694109df0 2024-09-21T03:45:43.643 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:43.638+0000 7f6698bae640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f6694113c30 con 0x7f6694103b60 2024-09-21T03:45:43.643 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:43.638+0000 7f6691d9b640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f66941094a0 0x7f6694075c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:43.643 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:43.638+0000 7f6691d9b640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f66941094a0 0x7f6694075c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:55782/0 (socket says 172.21.15.39:55782) 2024-09-21T03:45:43.643 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:43.638+0000 7f6691d9b640 1 -- 172.21.15.39:0/4122962683 learned_addr learned my addr 172.21.15.39:0/4122962683 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:45:43.644 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:43.638+0000 7f669259c640 1 --2- 172.21.15.39:0/4122962683 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6694103b60 0x7f6694075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:43.644 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:43.638+0000 7f6692d9d640 1 --2- 172.21.15.39:0/4122962683 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6694109df0 0x7f6694076190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:43.644 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:43.639+0000 7f6691d9b640 1 -- 172.21.15.39:0/4122962683 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6694103b60 msgr2=0x7f6694075710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:43.644 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:43.639+0000 7f6691d9b640 1 --2- 172.21.15.39:0/4122962683 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6694103b60 0x7f6694075710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:43.644 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:43.639+0000 7f6691d9b640 1 -- 172.21.15.39:0/4122962683 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6694109df0 msgr2=0x7f6694076190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:43.644 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:43.639+0000 7f6691d9b640 1 --2- 172.21.15.39:0/4122962683 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6694109df0 0x7f6694076190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:43.644 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:43.639+0000 7f6691d9b640 1 -- 172.21.15.39:0/4122962683 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f66941a8780 con 0x7f66941094a0 2024-09-21T03:45:43.644 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:43.639+0000 7f6691d9b640 1 --2- 172.21.15.39:0/4122962683 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f66941094a0 0x7f6694075c50 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f667c02a930 tx=0x7f667c004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:45:43.644 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:43.639+0000 7f668b7fe640 1 -- 172.21.15.39:0/4122962683 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f667c03f030 con 0x7f66941094a0 2024-09-21T03:45:43.645 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:43.639+0000 7f6698bae640 1 -- 172.21.15.39:0/4122962683 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f66941a8a10 con 0x7f66941094a0 2024-09-21T03:45:43.645 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:43.639+0000 7f668b7fe640 1 -- 172.21.15.39:0/4122962683 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f667c003aa0 con 0x7f66941094a0 2024-09-21T03:45:43.645 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:43.639+0000 7f668b7fe640 1 -- 172.21.15.39:0/4122962683 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f667c035070 con 0x7f66941094a0 2024-09-21T03:45:43.645 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:43.639+0000 7f6698bae640 1 -- 172.21.15.39:0/4122962683 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f66941a8ef0 con 0x7f66941094a0 2024-09-21T03:45:43.645 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:43.640+0000 7f6698bae640 1 -- 172.21.15.39:0/4122962683 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6694103f70 con 0x7f66941094a0 2024-09-21T03:45:43.646 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:43.642+0000 7f668b7fe640 1 -- 172.21.15.39:0/4122962683 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 22) ==== 154580+0+0 (secure 0 0 0) 0x7f667c042030 con 0x7f66941094a0 2024-09-21T03:45:43.646 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:43.643+0000 7f668b7fe640 1 --2- 172.21.15.39:0/4122962683 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f66800b65f0 0x7f66800b8b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:43.646 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:43.643+0000 7f669259c640 1 --2- 172.21.15.39:0/4122962683 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f66800b65f0 0x7f66800b8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:43.646 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:43.643+0000 7f668b7fe640 1 -- 172.21.15.39:0/4122962683 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8484+0+0 (secure 0 0 0) 0x7f667c108090 con 0x7f66941094a0 2024-09-21T03:45:43.646 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:43.644+0000 7f669259c640 1 --2- 172.21.15.39:0/4122962683 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f66800b65f0 0x7f66800b8b00 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f6694077160 tx=0x7f6684009680 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:45:43.647 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:43.646+0000 7f668b7fe640 1 -- 172.21.15.39:0/4122962683 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f667c0d2aa0 con 0x7f66941094a0 2024-09-21T03:45:43.845 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:43.844+0000 7f6698bae640 1 -- 172.21.15.39:0/4122962683 --> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] -- mgr_command(tid 0: {"prefix": "orch ls", "target": ["mon-mgr", ""]}) -- 0x7f6694063800 con 0x7f66800b65f0 2024-09-21T03:45:43.851 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:43.849+0000 7f668b7fe640 1 -- 172.21.15.39:0/4122962683 <== mgr.14223 v2:172.21.15.39:6800/1696381982 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+980 (secure 0 0 0) 0x7f6694063800 con 0x7f66800b65f0 2024-09-21T03:45:43.851 INFO:teuthology.orchestra.run.smithi039.stdout:NAME PORTS RUNNING REFRESHED AGE PLACEMENT 2024-09-21T03:45:43.851 INFO:teuthology.orchestra.run.smithi039.stdout:mds.all 12/12 33s ago 2m smithi039=a;smithi039=d;smithi039=g;smithi039=j;smithi150=b;smithi150=e;smithi150=h;smithi150=k;smithi161=c;smithi161=f;smithi161=i;smithi161=l;count:12 2024-09-21T03:45:43.851 INFO:teuthology.orchestra.run.smithi039.stdout:mgr 3/3 33s ago 8m smithi039=x;smithi150=y;smithi161=z;count:3 2024-09-21T03:45:43.851 INFO:teuthology.orchestra.run.smithi039.stdout:mon 3/3 33s ago 8m smithi039:172.21.15.39=a;smithi150:172.21.15.150=b;smithi161:172.21.15.161=c;count:3 2024-09-21T03:45:43.851 INFO:teuthology.orchestra.run.smithi039.stdout:osd 12 33s ago - 2024-09-21T03:45:43.853 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:43.851+0000 7f6698bae640 1 -- 172.21.15.39:0/4122962683 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f66800b65f0 msgr2=0x7f66800b8b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:43.854 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:43.851+0000 7f6698bae640 1 --2- 172.21.15.39:0/4122962683 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f66800b65f0 0x7f66800b8b00 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f6694077160 tx=0x7f6684009680 comp rx=0 tx=0).stop 2024-09-21T03:45:43.854 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:43.851+0000 7f6698bae640 1 -- 172.21.15.39:0/4122962683 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f66941094a0 msgr2=0x7f6694075c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:43.854 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:43.851+0000 7f6698bae640 1 --2- 172.21.15.39:0/4122962683 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f66941094a0 0x7f6694075c50 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f667c02a930 tx=0x7f667c004270 comp rx=0 tx=0).stop 2024-09-21T03:45:43.854 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:43.852+0000 7f6698bae640 1 -- 172.21.15.39:0/4122962683 shutdown_connections 2024-09-21T03:45:43.854 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:43.852+0000 7f6698bae640 1 --2- 172.21.15.39:0/4122962683 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f66800b65f0 0x7f66800b8b00 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:43.854 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:43.852+0000 7f6698bae640 1 --2- 172.21.15.39:0/4122962683 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6694109df0 0x7f6694076190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:43.854 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:43.852+0000 7f6698bae640 1 --2- 172.21.15.39:0/4122962683 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f66941094a0 0x7f6694075c50 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:43.854 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:43.852+0000 7f6698bae640 1 --2- 172.21.15.39:0/4122962683 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6694103b60 0x7f6694075710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:43.854 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:43.852+0000 7f6698bae640 1 -- 172.21.15.39:0/4122962683 >> 172.21.15.39:0/4122962683 conn(0x7f669407b850 msgr2=0x7f6694105510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:45:43.854 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:43.852+0000 7f6698bae640 1 -- 172.21.15.39:0/4122962683 shutdown_connections 2024-09-21T03:45:43.854 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:43.852+0000 7f6698bae640 1 -- 172.21.15.39:0/4122962683 wait complete. 2024-09-21T03:45:44.483 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 51116d5c-77ca-11ef-baf5-efdc52797490 -- bash -c 'ceph orch host ls' 2024-09-21T03:45:44.747 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/51116d5c-77ca-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:45:45.128 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:45:44 smithi039 ceph-mon[31211]: pgmap v316: 97 pgs: 97 active+clean; 647 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-09-21T03:45:45.128 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:45:44 smithi039 ceph-mon[31211]: from='client.15330 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:45:45.218 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:45:44 smithi161 ceph-mon[34533]: pgmap v316: 97 pgs: 97 active+clean; 647 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-09-21T03:45:45.218 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:45:44 smithi161 ceph-mon[34533]: from='client.15330 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:45:45.224 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:45:44 smithi150 ceph-mon[34289]: pgmap v316: 97 pgs: 97 active+clean; 647 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-09-21T03:45:45.225 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:45:44 smithi150 ceph-mon[34289]: from='client.15330 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:45:45.236 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:45.221+0000 7f1e828f0640 1 -- 172.21.15.39:0/3013768069 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1e7c109df0 msgr2=0x7f1e7c110620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:45.236 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:45.221+0000 7f1e828f0640 1 --2- 172.21.15.39:0/3013768069 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1e7c109df0 0x7f1e7c110620 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f1e74009ca0 tx=0x7f1e7402a8c0 comp rx=0 tx=0).stop 2024-09-21T03:45:45.236 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:45.221+0000 7f1e828f0640 1 -- 172.21.15.39:0/3013768069 shutdown_connections 2024-09-21T03:45:45.236 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:45.221+0000 7f1e828f0640 1 --2- 172.21.15.39:0/3013768069 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1e7c109df0 0x7f1e7c110620 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:45.237 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:45.221+0000 7f1e828f0640 1 --2- 172.21.15.39:0/3013768069 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1e7c1094a0 0x7f1e7c1098b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:45.237 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:45.221+0000 7f1e828f0640 1 --2- 172.21.15.39:0/3013768069 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1e7c103b60 0x7f1e7c103f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:45.237 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:45.221+0000 7f1e828f0640 1 -- 172.21.15.39:0/3013768069 >> 172.21.15.39:0/3013768069 conn(0x7f1e7c07b850 msgr2=0x7f1e7c07bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:45:45.237 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:45.222+0000 7f1e828f0640 1 -- 172.21.15.39:0/3013768069 shutdown_connections 2024-09-21T03:45:45.237 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:45.222+0000 7f1e828f0640 1 -- 172.21.15.39:0/3013768069 wait complete. 2024-09-21T03:45:45.237 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:45.222+0000 7f1e828f0640 1 Processor -- start 2024-09-21T03:45:45.237 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:45.222+0000 7f1e828f0640 1 -- start start 2024-09-21T03:45:45.237 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:45.223+0000 7f1e828f0640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1e7c103b60 0x7f1e7c19a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:45.237 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:45.223+0000 7f1e828f0640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1e7c1094a0 0x7f1e7c19a9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:45.237 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:45.223+0000 7f1e828f0640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1e7c109df0 0x7f1e7c19fc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:45.237 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:45.223+0000 7f1e828f0640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f1e7c113b00 con 0x7f1e7c103b60 2024-09-21T03:45:45.237 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:45.223+0000 7f1e828f0640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f1e7c113980 con 0x7f1e7c109df0 2024-09-21T03:45:45.237 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:45.223+0000 7f1e828f0640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f1e7c113c80 con 0x7f1e7c1094a0 2024-09-21T03:45:45.238 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:45.223+0000 7f1e7bfff640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1e7c103b60 0x7f1e7c19a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:45.238 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:45.223+0000 7f1e7bfff640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1e7c103b60 0x7f1e7c19a430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:55802/0 (socket says 172.21.15.39:55802) 2024-09-21T03:45:45.238 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:45.223+0000 7f1e7bfff640 1 -- 172.21.15.39:0/991489300 learned_addr learned my addr 172.21.15.39:0/991489300 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:45:45.238 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:45.223+0000 7f1e80e8d640 1 --2- 172.21.15.39:0/991489300 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1e7c109df0 0x7f1e7c19fc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:45.238 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:45.223+0000 7f1e7bfff640 1 -- 172.21.15.39:0/991489300 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1e7c1094a0 msgr2=0x7f1e7c19a9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:45.238 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:45.223+0000 7f1e7b7fe640 1 --2- 172.21.15.39:0/991489300 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1e7c1094a0 0x7f1e7c19a9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:45.238 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:45.223+0000 7f1e7bfff640 1 --2- 172.21.15.39:0/991489300 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1e7c1094a0 0x7f1e7c19a9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:45.238 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:45.223+0000 7f1e7bfff640 1 -- 172.21.15.39:0/991489300 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1e7c109df0 msgr2=0x7f1e7c19fc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:45.238 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:45.223+0000 7f1e7bfff640 1 --2- 172.21.15.39:0/991489300 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1e7c109df0 0x7f1e7c19fc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:45.238 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:45.223+0000 7f1e7bfff640 1 -- 172.21.15.39:0/991489300 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1e7c1a03a0 con 0x7f1e7c103b60 2024-09-21T03:45:45.238 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:45.223+0000 7f1e80e8d640 1 --2- 172.21.15.39:0/991489300 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1e7c109df0 0x7f1e7c19fc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:45:45.238 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:45.223+0000 7f1e7b7fe640 1 --2- 172.21.15.39:0/991489300 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1e7c1094a0 0x7f1e7c19a9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:45:45.238 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:45.223+0000 7f1e7bfff640 1 --2- 172.21.15.39:0/991489300 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1e7c103b60 0x7f1e7c19a430 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f1e6c00d950 tx=0x7f1e6c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:45:45.239 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:45.223+0000 7f1e797fa640 1 -- 172.21.15.39:0/991489300 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e6c017070 con 0x7f1e7c103b60 2024-09-21T03:45:45.239 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:45.224+0000 7f1e797fa640 1 -- 172.21.15.39:0/991489300 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1e6c009c30 con 0x7f1e7c103b60 2024-09-21T03:45:45.239 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:45.224+0000 7f1e828f0640 1 -- 172.21.15.39:0/991489300 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1e7c1a0690 con 0x7f1e7c103b60 2024-09-21T03:45:45.239 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:45.224+0000 7f1e797fa640 1 -- 172.21.15.39:0/991489300 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e6c00f620 con 0x7f1e7c103b60 2024-09-21T03:45:45.239 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:45.224+0000 7f1e828f0640 1 -- 172.21.15.39:0/991489300 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1e7c1a6df0 con 0x7f1e7c103b60 2024-09-21T03:45:45.239 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:45.225+0000 7f1e828f0640 1 -- 172.21.15.39:0/991489300 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1e7c103f70 con 0x7f1e7c103b60 2024-09-21T03:45:45.239 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:45.226+0000 7f1e797fa640 1 -- 172.21.15.39:0/991489300 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 22) ==== 154580+0+0 (secure 0 0 0) 0x7f1e6c010460 con 0x7f1e7c103b60 2024-09-21T03:45:45.239 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:45.227+0000 7f1e797fa640 1 --2- 172.21.15.39:0/991489300 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f1e640b65f0 0x7f1e640b8b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:45.239 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:45.227+0000 7f1e7b7fe640 1 --2- 172.21.15.39:0/991489300 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f1e640b65f0 0x7f1e640b8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:45.239 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:45.227+0000 7f1e797fa640 1 -- 172.21.15.39:0/991489300 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8484+0+0 (secure 0 0 0) 0x7f1e6c0e7320 con 0x7f1e7c103b60 2024-09-21T03:45:45.239 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:45.227+0000 7f1e7b7fe640 1 --2- 172.21.15.39:0/991489300 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f1e640b65f0 0x7f1e640b8b00 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f1e7c109060 tx=0x7f1e70005d30 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:45:45.239 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:45.231+0000 7f1e797fa640 1 -- 172.21.15.39:0/991489300 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f1e6c0ad950 con 0x7f1e7c103b60 2024-09-21T03:45:45.429 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:45.427+0000 7f1e828f0640 1 -- 172.21.15.39:0/991489300 --> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""]}) -- 0x7f1e7c063800 con 0x7f1e640b65f0 2024-09-21T03:45:45.430 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:45.429+0000 7f1e797fa640 1 -- 172.21.15.39:0/991489300 <== mgr.14223 v2:172.21.15.39:6800/1696381982 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+191 (secure 0 0 0) 0x7f1e7c063800 con 0x7f1e640b65f0 2024-09-21T03:45:45.431 INFO:teuthology.orchestra.run.smithi039.stdout:HOST ADDR LABELS STATUS 2024-09-21T03:45:45.431 INFO:teuthology.orchestra.run.smithi039.stdout:smithi039 172.21.15.39 2024-09-21T03:45:45.431 INFO:teuthology.orchestra.run.smithi039.stdout:smithi150 172.21.15.150 2024-09-21T03:45:45.431 INFO:teuthology.orchestra.run.smithi039.stdout:smithi161 172.21.15.161 2024-09-21T03:45:45.431 INFO:teuthology.orchestra.run.smithi039.stdout:3 hosts in cluster 2024-09-21T03:45:45.433 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:45.431+0000 7f1e828f0640 1 -- 172.21.15.39:0/991489300 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f1e640b65f0 msgr2=0x7f1e640b8b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:45.433 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:45.431+0000 7f1e828f0640 1 --2- 172.21.15.39:0/991489300 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f1e640b65f0 0x7f1e640b8b00 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f1e7c109060 tx=0x7f1e70005d30 comp rx=0 tx=0).stop 2024-09-21T03:45:45.433 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:45.431+0000 7f1e828f0640 1 -- 172.21.15.39:0/991489300 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1e7c103b60 msgr2=0x7f1e7c19a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:45.433 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:45.431+0000 7f1e828f0640 1 --2- 172.21.15.39:0/991489300 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1e7c103b60 0x7f1e7c19a430 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f1e6c00d950 tx=0x7f1e6c00de10 comp rx=0 tx=0).stop 2024-09-21T03:45:45.433 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:45.431+0000 7f1e828f0640 1 -- 172.21.15.39:0/991489300 shutdown_connections 2024-09-21T03:45:45.433 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:45.431+0000 7f1e828f0640 1 --2- 172.21.15.39:0/991489300 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f1e640b65f0 0x7f1e640b8b00 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:45.433 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:45.431+0000 7f1e828f0640 1 --2- 172.21.15.39:0/991489300 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1e7c109df0 0x7f1e7c19fc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:45.433 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:45.431+0000 7f1e828f0640 1 --2- 172.21.15.39:0/991489300 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1e7c1094a0 0x7f1e7c19a9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:45.433 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:45.431+0000 7f1e828f0640 1 --2- 172.21.15.39:0/991489300 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1e7c103b60 0x7f1e7c19a430 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:45.434 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:45.431+0000 7f1e828f0640 1 -- 172.21.15.39:0/991489300 >> 172.21.15.39:0/991489300 conn(0x7f1e7c07b850 msgr2=0x7f1e7c107490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:45:45.434 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:45.431+0000 7f1e828f0640 1 -- 172.21.15.39:0/991489300 shutdown_connections 2024-09-21T03:45:45.434 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:45.431+0000 7f1e828f0640 1 -- 172.21.15.39:0/991489300 wait complete. 2024-09-21T03:45:46.214 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 51116d5c-77ca-11ef-baf5-efdc52797490 -- bash -c 'ceph orch device ls' 2024-09-21T03:45:46.474 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/51116d5c-77ca-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:45:47.002 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:46.969+0000 7fe4390d5640 1 -- 172.21.15.39:0/3629378761 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe434102a60 msgr2=0x7fe434102e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:47.002 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:46.969+0000 7fe4390d5640 1 --2- 172.21.15.39:0/3629378761 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe434102a60 0x7fe434102e70 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7fe424009900 tx=0x7fe42402a410 comp rx=0 tx=0).stop 2024-09-21T03:45:47.002 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:46.969+0000 7fe4390d5640 1 -- 172.21.15.39:0/3629378761 shutdown_connections 2024-09-21T03:45:47.002 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:46.969+0000 7fe4390d5640 1 --2- 172.21.15.39:0/3629378761 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe434108cf0 0x7fe43410f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:47.002 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:46.969+0000 7fe4390d5640 1 --2- 172.21.15.39:0/3629378761 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe4341083a0 0x7fe4341087b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:47.003 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:46.969+0000 7fe4390d5640 1 --2- 172.21.15.39:0/3629378761 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe434102a60 0x7fe434102e70 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:47.003 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:46.969+0000 7fe4390d5640 1 -- 172.21.15.39:0/3629378761 >> 172.21.15.39:0/3629378761 conn(0x7fe434079730 msgr2=0x7fe434079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:45:47.003 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:46.969+0000 7fe4390d5640 1 -- 172.21.15.39:0/3629378761 shutdown_connections 2024-09-21T03:45:47.003 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:46.969+0000 7fe4390d5640 1 -- 172.21.15.39:0/3629378761 wait complete. 2024-09-21T03:45:47.003 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:46.970+0000 7fe4390d5640 1 Processor -- start 2024-09-21T03:45:47.003 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:46.970+0000 7fe4390d5640 1 -- start start 2024-09-21T03:45:47.003 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:46.971+0000 7fe4390d5640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe434102a60 0x7fe434199330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:47.003 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:46.971+0000 7fe4390d5640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe4341083a0 0x7fe4341998d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:47.003 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:46.971+0000 7fe4390d5640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe434108cf0 0x7fe43419eb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:47.003 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:46.971+0000 7fe4390d5640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fe434112960 con 0x7fe4341083a0 2024-09-21T03:45:47.003 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:46.971+0000 7fe4390d5640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fe4341127e0 con 0x7fe434108cf0 2024-09-21T03:45:47.003 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:46.971+0000 7fe4390d5640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fe434112ae0 con 0x7fe434102a60 2024-09-21T03:45:47.004 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:46.971+0000 7fe43259c640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe4341083a0 0x7fe4341998d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:47.004 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:46.971+0000 7fe43259c640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe4341083a0 0x7fe4341998d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:55830/0 (socket says 172.21.15.39:55830) 2024-09-21T03:45:47.004 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:46.971+0000 7fe43259c640 1 -- 172.21.15.39:0/3879213454 learned_addr learned my addr 172.21.15.39:0/3879213454 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:45:47.004 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:46.971+0000 7fe432d9d640 1 --2- 172.21.15.39:0/3879213454 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe434102a60 0x7fe434199330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:47.004 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:46.971+0000 7fe43359e640 1 --2- 172.21.15.39:0/3879213454 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe434108cf0 0x7fe43419eb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:47.004 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:46.971+0000 7fe43259c640 1 -- 172.21.15.39:0/3879213454 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe434102a60 msgr2=0x7fe434199330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:47.004 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:46.971+0000 7fe43259c640 1 --2- 172.21.15.39:0/3879213454 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe434102a60 0x7fe434199330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:47.004 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:46.971+0000 7fe43259c640 1 -- 172.21.15.39:0/3879213454 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe434108cf0 msgr2=0x7fe43419eb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:47.004 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:46.971+0000 7fe43259c640 1 --2- 172.21.15.39:0/3879213454 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe434108cf0 0x7fe43419eb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:47.004 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:46.971+0000 7fe43259c640 1 -- 172.21.15.39:0/3879213454 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe43419f2a0 con 0x7fe4341083a0 2024-09-21T03:45:47.004 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:46.971+0000 7fe43359e640 1 --2- 172.21.15.39:0/3879213454 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe434108cf0 0x7fe43419eb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:45:47.004 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:46.971+0000 7fe43259c640 1 --2- 172.21.15.39:0/3879213454 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe4341083a0 0x7fe4341998d0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7fe42800d770 tx=0x7fe42800dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:45:47.005 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:46.972+0000 7fe41bfff640 1 -- 172.21.15.39:0/3879213454 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe428017070 con 0x7fe4341083a0 2024-09-21T03:45:47.005 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:46.972+0000 7fe41bfff640 1 -- 172.21.15.39:0/3879213454 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe428009c30 con 0x7fe4341083a0 2024-09-21T03:45:47.005 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:46.972+0000 7fe41bfff640 1 -- 172.21.15.39:0/3879213454 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe42800f620 con 0x7fe4341083a0 2024-09-21T03:45:47.005 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:46.972+0000 7fe4390d5640 1 -- 172.21.15.39:0/3879213454 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe43419f590 con 0x7fe4341083a0 2024-09-21T03:45:47.005 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:46.972+0000 7fe4390d5640 1 -- 172.21.15.39:0/3879213454 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe4341a5cf0 con 0x7fe4341083a0 2024-09-21T03:45:47.005 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:46.973+0000 7fe4390d5640 1 -- 172.21.15.39:0/3879213454 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe434102e70 con 0x7fe4341083a0 2024-09-21T03:45:47.005 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:46.974+0000 7fe41bfff640 1 -- 172.21.15.39:0/3879213454 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 22) ==== 154580+0+0 (secure 0 0 0) 0x7fe42801a070 con 0x7fe4341083a0 2024-09-21T03:45:47.005 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:46.975+0000 7fe41bfff640 1 --2- 172.21.15.39:0/3879213454 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7fe41c0b6670 0x7fe41c0b8b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:47.005 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:46.975+0000 7fe432d9d640 1 --2- 172.21.15.39:0/3879213454 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7fe41c0b6670 0x7fe41c0b8b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:47.005 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:46.975+0000 7fe41bfff640 1 -- 172.21.15.39:0/3879213454 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8484+0+0 (secure 0 0 0) 0x7fe4280e7050 con 0x7fe4341083a0 2024-09-21T03:45:47.005 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:46.976+0000 7fe432d9d640 1 --2- 172.21.15.39:0/3879213454 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7fe41c0b6670 0x7fe41c0b8b80 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fe424031440 tx=0x7fe424009560 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:45:47.005 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:46.978+0000 7fe41bfff640 1 -- 172.21.15.39:0/3879213454 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fe428009de0 con 0x7fe4341083a0 2024-09-21T03:45:47.176 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:47.174+0000 7fe4390d5640 1 -- 172.21.15.39:0/3879213454 --> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] -- mgr_command(tid 0: {"prefix": "orch device ls", "target": ["mon-mgr", ""]}) -- 0x7fe434063800 con 0x7fe41c0b6670 2024-09-21T03:45:47.176 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:45:46 smithi039 ceph-mon[31211]: from='client.15336 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:45:47.177 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:45:46 smithi039 ceph-mon[31211]: pgmap v317: 97 pgs: 97 active+clean; 647 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 12 op/s 2024-09-21T03:45:47.178 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:47.176+0000 7fe41bfff640 1 -- 172.21.15.39:0/3879213454 <== mgr.14223 v2:172.21.15.39:6800/1696381982 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+532 (secure 0 0 0) 0x7fe434063800 con 0x7fe41c0b6670 2024-09-21T03:45:47.178 INFO:teuthology.orchestra.run.smithi039.stdout:HOST PATH TYPE DEVICE ID SIZE AVAILABLE REFRESHED REJECT REASONS 2024-09-21T03:45:47.178 INFO:teuthology.orchestra.run.smithi039.stdout:smithi039 /dev/nvme0n1 ssd INTEL_SSDPEDKE040T7_BTLE736102QX4P0KGN 3726G No 3m ago Has a FileSystem, LVM detected 2024-09-21T03:45:47.178 INFO:teuthology.orchestra.run.smithi039.stdout:smithi150 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_CVFT623300GX400BGN 372G No 2m ago Has a FileSystem, LVM detected 2024-09-21T03:45:47.178 INFO:teuthology.orchestra.run.smithi039.stdout:smithi161 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_CVFT6233001X400BGN 372G No 2m ago Has a FileSystem, LVM detected 2024-09-21T03:45:47.180 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:47.178+0000 7fe4390d5640 1 -- 172.21.15.39:0/3879213454 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7fe41c0b6670 msgr2=0x7fe41c0b8b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:47.180 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:47.178+0000 7fe4390d5640 1 --2- 172.21.15.39:0/3879213454 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7fe41c0b6670 0x7fe41c0b8b80 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fe424031440 tx=0x7fe424009560 comp rx=0 tx=0).stop 2024-09-21T03:45:47.180 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:47.178+0000 7fe4390d5640 1 -- 172.21.15.39:0/3879213454 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe4341083a0 msgr2=0x7fe4341998d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:47.180 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:47.178+0000 7fe4390d5640 1 --2- 172.21.15.39:0/3879213454 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe4341083a0 0x7fe4341998d0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7fe42800d770 tx=0x7fe42800dc30 comp rx=0 tx=0).stop 2024-09-21T03:45:47.180 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:47.178+0000 7fe4390d5640 1 -- 172.21.15.39:0/3879213454 shutdown_connections 2024-09-21T03:45:47.181 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:47.178+0000 7fe4390d5640 1 --2- 172.21.15.39:0/3879213454 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7fe41c0b6670 0x7fe41c0b8b80 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:47.181 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:47.178+0000 7fe4390d5640 1 --2- 172.21.15.39:0/3879213454 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe434108cf0 0x7fe43419eb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:47.181 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:47.178+0000 7fe4390d5640 1 --2- 172.21.15.39:0/3879213454 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe4341083a0 0x7fe4341998d0 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:47.181 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:47.178+0000 7fe4390d5640 1 --2- 172.21.15.39:0/3879213454 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe434102a60 0x7fe434199330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:47.181 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:47.178+0000 7fe4390d5640 1 -- 172.21.15.39:0/3879213454 >> 172.21.15.39:0/3879213454 conn(0x7fe434079730 msgr2=0x7fe434101350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:45:47.181 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:47.178+0000 7fe4390d5640 1 -- 172.21.15.39:0/3879213454 shutdown_connections 2024-09-21T03:45:47.181 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:47.178+0000 7fe4390d5640 1 -- 172.21.15.39:0/3879213454 wait complete. 2024-09-21T03:45:47.218 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:45:46 smithi161 ceph-mon[34533]: from='client.15336 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:45:47.218 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:45:46 smithi161 ceph-mon[34533]: pgmap v317: 97 pgs: 97 active+clean; 647 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 12 op/s 2024-09-21T03:45:47.224 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:45:46 smithi150 ceph-mon[34289]: from='client.15336 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:45:47.225 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:45:46 smithi150 ceph-mon[34289]: pgmap v317: 97 pgs: 97 active+clean; 647 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 12 op/s 2024-09-21T03:45:47.948 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-09-21T03:45:47.958 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi039.front.sepia.ceph.com 2024-09-21T03:45:47.958 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 51116d5c-77ca-11ef-baf5-efdc52797490 -- bash -c 'ceph fs dump' 2024-09-21T03:45:48.216 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/51116d5c-77ca-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:45:48.217 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:45:47 smithi161 ceph-mon[34533]: from='client.15342 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:45:48.224 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:45:47 smithi150 ceph-mon[34289]: from='client.15342 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:45:48.334 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:45:47 smithi039 ceph-mon[31211]: from='client.15342 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:45:48.769 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:48.693+0000 7f912880d640 1 -- 172.21.15.39:0/16253161 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9124108d10 msgr2=0x7f912410f540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:48.769 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:48.693+0000 7f912880d640 1 --2- 172.21.15.39:0/16253161 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9124108d10 0x7f912410f540 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f911c009ca0 tx=0x7f911c02a8c0 comp rx=0 tx=0).stop 2024-09-21T03:45:48.769 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:48.693+0000 7f912880d640 1 -- 172.21.15.39:0/16253161 shutdown_connections 2024-09-21T03:45:48.769 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:48.693+0000 7f912880d640 1 --2- 172.21.15.39:0/16253161 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9124108d10 0x7f912410f540 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:48.769 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:48.693+0000 7f912880d640 1 --2- 172.21.15.39:0/16253161 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f91241083c0 0x7f91241087d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:48.769 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:48.693+0000 7f912880d640 1 --2- 172.21.15.39:0/16253161 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9124102a80 0x7f9124102e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:48.769 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:48.693+0000 7f912880d640 1 -- 172.21.15.39:0/16253161 >> 172.21.15.39:0/16253161 conn(0x7f9124079740 msgr2=0x7f9124079b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:45:48.770 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:48.693+0000 7f912880d640 1 -- 172.21.15.39:0/16253161 shutdown_connections 2024-09-21T03:45:48.770 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:48.693+0000 7f912880d640 1 -- 172.21.15.39:0/16253161 wait complete. 2024-09-21T03:45:48.770 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:48.694+0000 7f912880d640 1 Processor -- start 2024-09-21T03:45:48.770 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:48.694+0000 7f912880d640 1 -- start start 2024-09-21T03:45:48.770 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:48.694+0000 7f912880d640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9124102a80 0x7f912419b300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:48.770 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:48.694+0000 7f912880d640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f91241083c0 0x7f912419b840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:48.770 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:48.694+0000 7f912880d640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9124108d10 0x7f912419bd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:48.770 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:48.694+0000 7f912880d640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f91241129d0 con 0x7f9124102a80 2024-09-21T03:45:48.770 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:48.694+0000 7f912880d640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f9124112850 con 0x7f9124108d10 2024-09-21T03:45:48.770 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:48.694+0000 7f912880d640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f9124112b50 con 0x7f91241083c0 2024-09-21T03:45:48.770 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:48.694+0000 7f912259c640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9124102a80 0x7f912419b300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:48.770 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:48.694+0000 7f912259c640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9124102a80 0x7f912419b300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:35480/0 (socket says 172.21.15.39:35480) 2024-09-21T03:45:48.770 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:48.694+0000 7f912259c640 1 -- 172.21.15.39:0/329949813 learned_addr learned my addr 172.21.15.39:0/329949813 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:45:48.771 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:48.695+0000 7f912259c640 1 -- 172.21.15.39:0/329949813 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f91241083c0 msgr2=0x7f912419b840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:48.771 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:48.695+0000 7f9121d9b640 1 --2- 172.21.15.39:0/329949813 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f91241083c0 0x7f912419b840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:48.771 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:48.695+0000 7f9122d9d640 1 --2- 172.21.15.39:0/329949813 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9124108d10 0x7f912419bd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:48.771 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:48.695+0000 7f912259c640 1 --2- 172.21.15.39:0/329949813 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f91241083c0 0x7f912419b840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:48.771 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:48.695+0000 7f912259c640 1 -- 172.21.15.39:0/329949813 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9124108d10 msgr2=0x7f912419bd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:48.771 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:48.695+0000 7f912259c640 1 --2- 172.21.15.39:0/329949813 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9124108d10 0x7f912419bd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:48.771 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:48.695+0000 7f912259c640 1 -- 172.21.15.39:0/329949813 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f91241a6500 con 0x7f9124102a80 2024-09-21T03:45:48.771 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:48.695+0000 7f912259c640 1 --2- 172.21.15.39:0/329949813 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9124102a80 0x7f912419b300 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f910c00c920 tx=0x7f910c00cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:45:48.771 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:48.695+0000 7f911b7fe640 1 -- 172.21.15.39:0/329949813 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f910c007c70 con 0x7f9124102a80 2024-09-21T03:45:48.771 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:48.695+0000 7f911b7fe640 1 -- 172.21.15.39:0/329949813 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f910c007e10 con 0x7f9124102a80 2024-09-21T03:45:48.771 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:48.695+0000 7f912880d640 1 -- 172.21.15.39:0/329949813 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f91241a67f0 con 0x7f9124102a80 2024-09-21T03:45:48.771 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:48.695+0000 7f911b7fe640 1 -- 172.21.15.39:0/329949813 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f910c018820 con 0x7f9124102a80 2024-09-21T03:45:48.771 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:48.695+0000 7f912880d640 1 -- 172.21.15.39:0/329949813 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f91241a6d30 con 0x7f9124102a80 2024-09-21T03:45:48.771 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:48.696+0000 7f912880d640 1 -- 172.21.15.39:0/329949813 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9124103d00 con 0x7f9124102a80 2024-09-21T03:45:48.772 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:48.699+0000 7f911b7fe640 1 -- 172.21.15.39:0/329949813 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 22) ==== 154580+0+0 (secure 0 0 0) 0x7f910c00f450 con 0x7f9124102a80 2024-09-21T03:45:48.772 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:48.699+0000 7f911b7fe640 1 --2- 172.21.15.39:0/329949813 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f91100b6620 0x7f91100b8b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:48.772 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:48.700+0000 7f9121d9b640 1 --2- 172.21.15.39:0/329949813 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f91100b6620 0x7f91100b8b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:48.772 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:48.700+0000 7f911b7fe640 1 -- 172.21.15.39:0/329949813 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8484+0+0 (secure 0 0 0) 0x7f910c0e7a90 con 0x7f9124102a80 2024-09-21T03:45:48.772 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:48.700+0000 7f9121d9b640 1 --2- 172.21.15.39:0/329949813 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f91100b6620 0x7f91100b8b30 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f9114005d80 tx=0x7f9114005c90 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:45:48.772 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:48.703+0000 7f911b7fe640 1 -- 172.21.15.39:0/329949813 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f910c014030 con 0x7f9124102a80 2024-09-21T03:45:48.933 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:48.932+0000 7f912880d640 1 -- 172.21.15.39:0/329949813 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump"} v 0) -- 0x7f912406bf50 con 0x7f9124102a80 2024-09-21T03:45:48.935 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:48.933+0000 7f911b7fe640 1 -- 172.21.15.39:0/329949813 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 28 v28) ==== 76+0+3131 (secure 0 0 0) 0x7f910c0ae150 con 0x7f9124102a80 2024-09-21T03:45:48.935 INFO:teuthology.orchestra.run.smithi039.stdout:e28 2024-09-21T03:45:48.935 INFO:teuthology.orchestra.run.smithi039.stdout:btime 2024-09-21T03:44:26:704224+0000 2024-09-21T03:45:48.935 INFO:teuthology.orchestra.run.smithi039.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-09-21T03:45:48.935 INFO:teuthology.orchestra.run.smithi039.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-21T03:45:48.935 INFO:teuthology.orchestra.run.smithi039.stdout:legacy client fscid: 1 2024-09-21T03:45:48.935 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T03:45:48.935 INFO:teuthology.orchestra.run.smithi039.stdout:Filesystem 'cephfs' (1) 2024-09-21T03:45:48.936 INFO:teuthology.orchestra.run.smithi039.stdout:fs_name cephfs 2024-09-21T03:45:48.936 INFO:teuthology.orchestra.run.smithi039.stdout:epoch 28 2024-09-21T03:45:48.936 INFO:teuthology.orchestra.run.smithi039.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-09-21T03:45:48.936 INFO:teuthology.orchestra.run.smithi039.stdout:created 2024-09-21T03:44:15.009872+0000 2024-09-21T03:45:48.936 INFO:teuthology.orchestra.run.smithi039.stdout:modified 2024-09-21T03:44:26.017327+0000 2024-09-21T03:45:48.936 INFO:teuthology.orchestra.run.smithi039.stdout:tableserver 0 2024-09-21T03:45:48.936 INFO:teuthology.orchestra.run.smithi039.stdout:root 0 2024-09-21T03:45:48.936 INFO:teuthology.orchestra.run.smithi039.stdout:session_timeout 300 2024-09-21T03:45:48.936 INFO:teuthology.orchestra.run.smithi039.stdout:session_autoclose 300 2024-09-21T03:45:48.936 INFO:teuthology.orchestra.run.smithi039.stdout:max_file_size 1099511627776 2024-09-21T03:45:48.936 INFO:teuthology.orchestra.run.smithi039.stdout:max_xattr_size 65536 2024-09-21T03:45:48.936 INFO:teuthology.orchestra.run.smithi039.stdout:required_client_features {} 2024-09-21T03:45:48.936 INFO:teuthology.orchestra.run.smithi039.stdout:last_failure 0 2024-09-21T03:45:48.936 INFO:teuthology.orchestra.run.smithi039.stdout:last_failure_osd_epoch 0 2024-09-21T03:45:48.937 INFO:teuthology.orchestra.run.smithi039.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-21T03:45:48.937 INFO:teuthology.orchestra.run.smithi039.stdout:max_mds 5 2024-09-21T03:45:48.937 INFO:teuthology.orchestra.run.smithi039.stdout:in 0,1,2,3,4 2024-09-21T03:45:48.937 INFO:teuthology.orchestra.run.smithi039.stdout:up {0=24695,1=24667,2=24659,3=24646,4=14868} 2024-09-21T03:45:48.937 INFO:teuthology.orchestra.run.smithi039.stdout:failed 2024-09-21T03:45:48.937 INFO:teuthology.orchestra.run.smithi039.stdout:damaged 2024-09-21T03:45:48.937 INFO:teuthology.orchestra.run.smithi039.stdout:stopped 2024-09-21T03:45:48.937 INFO:teuthology.orchestra.run.smithi039.stdout:data_pools [3] 2024-09-21T03:45:48.937 INFO:teuthology.orchestra.run.smithi039.stdout:metadata_pool 2 2024-09-21T03:45:48.937 INFO:teuthology.orchestra.run.smithi039.stdout:inline_data disabled 2024-09-21T03:45:48.937 INFO:teuthology.orchestra.run.smithi039.stdout:balancer 2024-09-21T03:45:48.937 INFO:teuthology.orchestra.run.smithi039.stdout:bal_rank_mask -1 2024-09-21T03:45:48.937 INFO:teuthology.orchestra.run.smithi039.stdout:standby_count_wanted 0 2024-09-21T03:45:48.937 INFO:teuthology.orchestra.run.smithi039.stdout:qdb_cluster leader: 24695 members: 14868,24646,24659,24667,24695 2024-09-21T03:45:48.938 INFO:teuthology.orchestra.run.smithi039.stdout:[mds.b{0:24695} state up:active seq 4 addr [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] compat {c=[1],r=[1],i=[1fff]}] 2024-09-21T03:45:48.938 INFO:teuthology.orchestra.run.smithi039.stdout:[mds.f{0:24640} state up:standby-replay seq 1 addr [v2:172.21.15.161:6834/3747882110,v1:172.21.15.161:6835/3747882110] compat {c=[1],r=[1],i=[1fff]}] 2024-09-21T03:45:48.938 INFO:teuthology.orchestra.run.smithi039.stdout:[mds.i{1:24667} state up:active seq 8 addr [v2:172.21.15.161:6838/2190107270,v1:172.21.15.161:6839/2190107270] compat {c=[1],r=[1],i=[1fff]}] 2024-09-21T03:45:48.938 INFO:teuthology.orchestra.run.smithi039.stdout:[mds.e{1:24632} state up:standby-replay seq 1 addr [v2:172.21.15.150:6834/724594008,v1:172.21.15.150:6835/724594008] compat {c=[1],r=[1],i=[1fff]}] 2024-09-21T03:45:48.938 INFO:teuthology.orchestra.run.smithi039.stdout:[mds.h{2:24659} state up:active seq 9 addr [v2:172.21.15.150:6836/4017043144,v1:172.21.15.150:6837/4017043144] compat {c=[1],r=[1],i=[1fff]}] 2024-09-21T03:45:48.938 INFO:teuthology.orchestra.run.smithi039.stdout:[mds.l{2:24601} state up:standby-replay seq 1 addr [v2:172.21.15.161:6832/2963990654,v1:172.21.15.161:6833/2963990654] compat {c=[1],r=[1],i=[1fff]}] 2024-09-21T03:45:48.938 INFO:teuthology.orchestra.run.smithi039.stdout:[mds.c{3:24646} state up:active seq 10 addr [v2:172.21.15.161:6836/3451544597,v1:172.21.15.161:6837/3451544597] compat {c=[1],r=[1],i=[1fff]}] 2024-09-21T03:45:48.938 INFO:teuthology.orchestra.run.smithi039.stdout:[mds.k{3:24599} state up:standby-replay seq 1 addr [v2:172.21.15.150:6832/3114643217,v1:172.21.15.150:6833/3114643217] compat {c=[1],r=[1],i=[1fff]}] 2024-09-21T03:45:48.938 INFO:teuthology.orchestra.run.smithi039.stdout:[mds.a{4:14868} state up:active seq 7 addr [v2:172.21.15.39:6840/4247190456,v1:172.21.15.39:6841/4247190456] compat {c=[1],r=[1],i=[1fff]}] 2024-09-21T03:45:48.938 INFO:teuthology.orchestra.run.smithi039.stdout:[mds.g{4:14856} state up:standby-replay seq 1 addr [v2:172.21.15.39:6838/185509192,v1:172.21.15.39:6839/185509192] compat {c=[1],r=[1],i=[1fff]}] 2024-09-21T03:45:48.938 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T03:45:48.938 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T03:45:48.938 INFO:teuthology.orchestra.run.smithi039.stdout:Standby daemons: 2024-09-21T03:45:48.938 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T03:45:48.939 INFO:teuthology.orchestra.run.smithi039.stdout:[mds.j{-1:14790} state up:standby seq 1 addr [v2:172.21.15.39:6834/508995295,v1:172.21.15.39:6835/508995295] compat {c=[1],r=[1],i=[1fff]}] 2024-09-21T03:45:48.939 INFO:teuthology.orchestra.run.smithi039.stdout:[mds.d{-1:14802} state up:standby seq 1 addr [v2:172.21.15.39:6836/3915912607,v1:172.21.15.39:6837/3915912607] compat {c=[1],r=[1],i=[1fff]}] 2024-09-21T03:45:48.939 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 28 2024-09-21T03:45:48.939 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:48.935+0000 7f912880d640 1 -- 172.21.15.39:0/329949813 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f91100b6620 msgr2=0x7f91100b8b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:48.940 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:48.935+0000 7f912880d640 1 --2- 172.21.15.39:0/329949813 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f91100b6620 0x7f91100b8b30 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f9114005d80 tx=0x7f9114005c90 comp rx=0 tx=0).stop 2024-09-21T03:45:48.940 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:48.935+0000 7f912880d640 1 -- 172.21.15.39:0/329949813 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9124102a80 msgr2=0x7f912419b300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:48.940 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:48.935+0000 7f912880d640 1 --2- 172.21.15.39:0/329949813 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9124102a80 0x7f912419b300 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f910c00c920 tx=0x7f910c00cde0 comp rx=0 tx=0).stop 2024-09-21T03:45:48.940 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:48.935+0000 7f912880d640 1 -- 172.21.15.39:0/329949813 shutdown_connections 2024-09-21T03:45:48.940 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:48.935+0000 7f912880d640 1 --2- 172.21.15.39:0/329949813 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f91100b6620 0x7f91100b8b30 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:48.940 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:48.935+0000 7f912880d640 1 --2- 172.21.15.39:0/329949813 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9124108d10 0x7f912419bd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:48.940 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:48.935+0000 7f912880d640 1 --2- 172.21.15.39:0/329949813 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f91241083c0 0x7f912419b840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:48.940 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:48.935+0000 7f912880d640 1 --2- 172.21.15.39:0/329949813 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9124102a80 0x7f912419b300 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:48.940 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:48.935+0000 7f912880d640 1 -- 172.21.15.39:0/329949813 >> 172.21.15.39:0/329949813 conn(0x7f9124079740 msgr2=0x7f9124104430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:45:48.940 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:48.936+0000 7f912880d640 1 -- 172.21.15.39:0/329949813 shutdown_connections 2024-09-21T03:45:48.940 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:48.936+0000 7f912880d640 1 -- 172.21.15.39:0/329949813 wait complete. 2024-09-21T03:45:49.217 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:45:48 smithi161 ceph-mon[34533]: pgmap v318: 97 pgs: 97 active+clean; 647 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-09-21T03:45:49.224 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:45:48 smithi150 ceph-mon[34289]: pgmap v318: 97 pgs: 97 active+clean; 647 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-09-21T03:45:49.264 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:45:48 smithi039 ceph-mon[31211]: pgmap v318: 97 pgs: 97 active+clean; 647 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-09-21T03:45:49.724 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 51116d5c-77ca-11ef-baf5-efdc52797490 -- bash -c 'ceph osd dump' 2024-09-21T03:45:49.984 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/51116d5c-77ca-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:45:50.083 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:45:49 smithi039 ceph-mon[31211]: from='client.? 172.21.15.39:0/329949813' entity='client.admin' cmd={"prefix": "fs dump"} : dispatch 2024-09-21T03:45:50.217 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:45:49 smithi161 ceph-mon[34533]: from='client.? 172.21.15.39:0/329949813' entity='client.admin' cmd={"prefix": "fs dump"} : dispatch 2024-09-21T03:45:50.224 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:45:49 smithi150 ceph-mon[34289]: from='client.? 172.21.15.39:0/329949813' entity='client.admin' cmd={"prefix": "fs dump"} : dispatch 2024-09-21T03:45:50.476 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:50.467+0000 7fafa6891640 1 -- 172.21.15.39:0/3901002880 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fafa01048a0 msgr2=0x7fafa0104cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:50.477 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:50.467+0000 7fafa6891640 1 --2- 172.21.15.39:0/3901002880 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fafa01048a0 0x7fafa0104cb0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7faf90009900 tx=0x7faf9002a410 comp rx=0 tx=0).stop 2024-09-21T03:45:50.477 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:50.467+0000 7fafa6891640 1 -- 172.21.15.39:0/3901002880 shutdown_connections 2024-09-21T03:45:50.477 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:50.467+0000 7fafa6891640 1 --2- 172.21.15.39:0/3901002880 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fafa010de50 0x7fafa01102f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:50.477 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:50.467+0000 7fafa6891640 1 --2- 172.21.15.39:0/3901002880 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fafa01052d0 0x7fafa010d910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:50.477 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:50.467+0000 7fafa6891640 1 --2- 172.21.15.39:0/3901002880 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fafa01048a0 0x7fafa0104cb0 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:50.477 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:50.467+0000 7fafa6891640 1 -- 172.21.15.39:0/3901002880 >> 172.21.15.39:0/3901002880 conn(0x7fafa0079730 msgr2=0x7fafa0079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:45:50.477 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:50.467+0000 7fafa6891640 1 -- 172.21.15.39:0/3901002880 shutdown_connections 2024-09-21T03:45:50.477 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:50.467+0000 7fafa6891640 1 -- 172.21.15.39:0/3901002880 wait complete. 2024-09-21T03:45:50.477 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:50.468+0000 7fafa6891640 1 Processor -- start 2024-09-21T03:45:50.477 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:50.468+0000 7fafa6891640 1 -- start start 2024-09-21T03:45:50.477 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:50.468+0000 7fafa6891640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fafa01048a0 0x7fafa01027b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:50.477 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:50.468+0000 7fafa6891640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fafa01052d0 0x7fafa0102cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:50.478 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:50.468+0000 7fafa6891640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fafa010de50 0x7fafa0103230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:50.478 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:50.468+0000 7fafa6891640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fafa0113b90 con 0x7fafa01052d0 2024-09-21T03:45:50.478 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:50.468+0000 7fafa6891640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fafa0113a10 con 0x7fafa01048a0 2024-09-21T03:45:50.478 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:50.468+0000 7fafa6891640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fafa0113d10 con 0x7fafa010de50 2024-09-21T03:45:50.478 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:50.468+0000 7faf97fff640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fafa01052d0 0x7fafa0102cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:50.478 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:50.468+0000 7faf97fff640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fafa01052d0 0x7fafa0102cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:35498/0 (socket says 172.21.15.39:35498) 2024-09-21T03:45:50.478 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:50.468+0000 7faf97fff640 1 -- 172.21.15.39:0/1630662883 learned_addr learned my addr 172.21.15.39:0/1630662883 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:45:50.478 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:50.468+0000 7fafa4e2e640 1 --2- 172.21.15.39:0/1630662883 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fafa010de50 0x7fafa0103230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:50.478 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:50.468+0000 7faf9ffff640 1 --2- 172.21.15.39:0/1630662883 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fafa01048a0 0x7fafa01027b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:50.478 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:50.469+0000 7faf97fff640 1 -- 172.21.15.39:0/1630662883 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fafa010de50 msgr2=0x7fafa0103230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:50.478 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:50.469+0000 7faf97fff640 1 --2- 172.21.15.39:0/1630662883 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fafa010de50 0x7fafa0103230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:50.478 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:50.469+0000 7faf97fff640 1 -- 172.21.15.39:0/1630662883 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fafa01048a0 msgr2=0x7fafa01027b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:50.479 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:50.469+0000 7faf97fff640 1 --2- 172.21.15.39:0/1630662883 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fafa01048a0 0x7fafa01027b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:50.479 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:50.469+0000 7faf97fff640 1 -- 172.21.15.39:0/1630662883 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fafa01a8880 con 0x7fafa01052d0 2024-09-21T03:45:50.479 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:50.469+0000 7faf9ffff640 1 --2- 172.21.15.39:0/1630662883 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fafa01048a0 0x7fafa01027b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:45:50.479 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:50.469+0000 7faf97fff640 1 --2- 172.21.15.39:0/1630662883 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fafa01052d0 0x7fafa0102cf0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7faf8800b450 tx=0x7faf8800b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:45:50.479 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:50.469+0000 7faf9dffb640 1 -- 172.21.15.39:0/1630662883 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf88015020 con 0x7fafa01052d0 2024-09-21T03:45:50.479 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:50.469+0000 7fafa6891640 1 -- 172.21.15.39:0/1630662883 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fafa01a8b70 con 0x7fafa01052d0 2024-09-21T03:45:50.479 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:50.469+0000 7faf9dffb640 1 -- 172.21.15.39:0/1630662883 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faf88011d40 con 0x7fafa01052d0 2024-09-21T03:45:50.479 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:50.469+0000 7faf9dffb640 1 -- 172.21.15.39:0/1630662883 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf8800f340 con 0x7fafa01052d0 2024-09-21T03:45:50.479 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:50.469+0000 7fafa6891640 1 -- 172.21.15.39:0/1630662883 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fafa01a90b0 con 0x7fafa01052d0 2024-09-21T03:45:50.479 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:50.470+0000 7fafa6891640 1 -- 172.21.15.39:0/1630662883 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fafa0104d30 con 0x7fafa01052d0 2024-09-21T03:45:50.479 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:50.472+0000 7faf9dffb640 1 -- 172.21.15.39:0/1630662883 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 22) ==== 154580+0+0 (secure 0 0 0) 0x7faf88018020 con 0x7fafa01052d0 2024-09-21T03:45:50.479 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:50.472+0000 7faf9dffb640 1 --2- 172.21.15.39:0/1630662883 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7faf8c0b65f0 0x7faf8c0b8b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:50.479 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:50.472+0000 7faf9ffff640 1 --2- 172.21.15.39:0/1630662883 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7faf8c0b65f0 0x7faf8c0b8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:50.479 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:50.472+0000 7faf9dffb640 1 -- 172.21.15.39:0/1630662883 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8484+0+0 (secure 0 0 0) 0x7faf880e7540 con 0x7fafa01052d0 2024-09-21T03:45:50.480 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:50.473+0000 7faf9ffff640 1 --2- 172.21.15.39:0/1630662883 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7faf8c0b65f0 0x7faf8c0b8b00 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fafa0104200 tx=0x7faf90031400 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:45:50.480 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:50.475+0000 7faf9dffb640 1 -- 172.21.15.39:0/1630662883 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7faf8801f020 con 0x7fafa01052d0 2024-09-21T03:45:50.670 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:50.668+0000 7fafa6891640 1 -- 172.21.15.39:0/1630662883 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd dump"} v 0) -- 0x7fafa0103ab0 con 0x7fafa01052d0 2024-09-21T03:45:50.671 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:50.669+0000 7faf9dffb640 1 -- 172.21.15.39:0/1630662883 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump"}]=0 v79) ==== 56+0+5707 (secure 0 0 0) 0x7fafa0103ab0 con 0x7fafa01052d0 2024-09-21T03:45:50.671 INFO:teuthology.orchestra.run.smithi039.stdout:epoch 79 2024-09-21T03:45:50.671 INFO:teuthology.orchestra.run.smithi039.stdout:fsid 51116d5c-77ca-11ef-baf5-efdc52797490 2024-09-21T03:45:50.671 INFO:teuthology.orchestra.run.smithi039.stdout:created 2024-09-21T03:35:33.631350+0000 2024-09-21T03:45:50.671 INFO:teuthology.orchestra.run.smithi039.stdout:modified 2024-09-21T03:44:27.402588+0000 2024-09-21T03:45:50.672 INFO:teuthology.orchestra.run.smithi039.stdout:flags sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit 2024-09-21T03:45:50.672 INFO:teuthology.orchestra.run.smithi039.stdout:crush_version 26 2024-09-21T03:45:50.672 INFO:teuthology.orchestra.run.smithi039.stdout:full_ratio 0.95 2024-09-21T03:45:50.672 INFO:teuthology.orchestra.run.smithi039.stdout:backfillfull_ratio 0.9 2024-09-21T03:45:50.672 INFO:teuthology.orchestra.run.smithi039.stdout:nearfull_ratio 0.85 2024-09-21T03:45:50.672 INFO:teuthology.orchestra.run.smithi039.stdout:require_min_compat_client luminous 2024-09-21T03:45:50.672 INFO:teuthology.orchestra.run.smithi039.stdout:min_compat_client luminous 2024-09-21T03:45:50.672 INFO:teuthology.orchestra.run.smithi039.stdout:require_osd_release squid 2024-09-21T03:45:50.672 INFO:teuthology.orchestra.run.smithi039.stdout:stretch_mode_enabled false 2024-09-21T03:45:50.672 INFO:teuthology.orchestra.run.smithi039.stdout:pool 1 '.mgr' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 1 pgp_num 1 autoscale_mode off last_change 22 flags hashpspool stripe_width 0 pg_num_max 32 pg_num_min 1 application mgr read_balance_score 12.00 2024-09-21T03:45:50.672 INFO:teuthology.orchestra.run.smithi039.stdout:pool 2 'cephfs_metadata' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode off last_change 77 flags hashpspool stripe_width 0 pg_autoscale_bias 4 pg_num_min 16 recovery_priority 5 application cephfs read_balance_score 1.88 2024-09-21T03:45:50.672 INFO:teuthology.orchestra.run.smithi039.stdout:pool 3 'cephfs_data' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 64 pgp_num 64 autoscale_mode off last_change 77 flags hashpspool stripe_width 0 pg_num_min 64 target_size_ratio 0.9 application cephfs read_balance_score 1.50 2024-09-21T03:45:50.672 INFO:teuthology.orchestra.run.smithi039.stdout:max_osd 12 2024-09-21T03:45:50.672 INFO:teuthology.orchestra.run.smithi039.stdout:osd.0 up in weight 1 up_from 9 up_thru 75 down_at 0 last_clean_interval [0,0) [v2:172.21.15.39:6802/870188791,v1:172.21.15.39:6803/870188791] [v2:172.21.15.39:6804/870188791,v1:172.21.15.39:6805/870188791] exists,up 1ea7148f-2ace-4c10-86fe-b8019de80c1f 2024-09-21T03:45:50.673 INFO:teuthology.orchestra.run.smithi039.stdout:osd.1 up in weight 1 up_from 14 up_thru 78 down_at 0 last_clean_interval [0,0) [v2:172.21.15.150:6800/172427832,v1:172.21.15.150:6801/172427832] [v2:172.21.15.150:6802/172427832,v1:172.21.15.150:6803/172427832] exists,up 7cb5db90-28e8-4a30-bedb-c039687d48b0 2024-09-21T03:45:50.673 INFO:teuthology.orchestra.run.smithi039.stdout:osd.2 up in weight 1 up_from 19 up_thru 75 down_at 0 last_clean_interval [0,0) [v2:172.21.15.161:6800/2727876133,v1:172.21.15.161:6801/2727876133] [v2:172.21.15.161:6802/2727876133,v1:172.21.15.161:6803/2727876133] exists,up b25c6ab3-fc95-4751-bd63-6e3fc23e8bd7 2024-09-21T03:45:50.673 INFO:teuthology.orchestra.run.smithi039.stdout:osd.3 up in weight 1 up_from 26 up_thru 75 down_at 0 last_clean_interval [0,0) [v2:172.21.15.39:6810/295394468,v1:172.21.15.39:6811/295394468] [v2:172.21.15.39:6812/295394468,v1:172.21.15.39:6813/295394468] exists,up 14417ae1-4536-45ba-9a5c-c3b447d1a72d 2024-09-21T03:45:50.673 INFO:teuthology.orchestra.run.smithi039.stdout:osd.4 up in weight 1 up_from 31 up_thru 75 down_at 0 last_clean_interval [0,0) [v2:172.21.15.150:6808/3097721553,v1:172.21.15.150:6809/3097721553] [v2:172.21.15.150:6810/3097721553,v1:172.21.15.150:6811/3097721553] exists,up 33c732ef-20c2-43c4-abe6-045b687fc521 2024-09-21T03:45:50.673 INFO:teuthology.orchestra.run.smithi039.stdout:osd.5 up in weight 1 up_from 37 up_thru 78 down_at 0 last_clean_interval [0,0) [v2:172.21.15.161:6808/742751023,v1:172.21.15.161:6809/742751023] [v2:172.21.15.161:6810/742751023,v1:172.21.15.161:6811/742751023] exists,up 74673722-abd6-400a-bb50-ec3dfba0c5e9 2024-09-21T03:45:50.673 INFO:teuthology.orchestra.run.smithi039.stdout:osd.6 up in weight 1 up_from 43 up_thru 75 down_at 0 last_clean_interval [0,0) [v2:172.21.15.39:6818/1072700290,v1:172.21.15.39:6819/1072700290] [v2:172.21.15.39:6820/1072700290,v1:172.21.15.39:6821/1072700290] exists,up 06548677-74c4-4efa-8054-df09b957b649 2024-09-21T03:45:50.673 INFO:teuthology.orchestra.run.smithi039.stdout:osd.7 up in weight 1 up_from 49 up_thru 75 down_at 0 last_clean_interval [0,0) [v2:172.21.15.150:6816/2162619103,v1:172.21.15.150:6817/2162619103] [v2:172.21.15.150:6818/2162619103,v1:172.21.15.150:6819/2162619103] exists,up 838735ca-c693-4f58-9426-b8ee857dbe3a 2024-09-21T03:45:50.673 INFO:teuthology.orchestra.run.smithi039.stdout:osd.8 up in weight 1 up_from 54 up_thru 78 down_at 0 last_clean_interval [0,0) [v2:172.21.15.161:6816/52966633,v1:172.21.15.161:6817/52966633] [v2:172.21.15.161:6818/52966633,v1:172.21.15.161:6819/52966633] exists,up 4973b357-b4c2-4c00-b7f7-dd25012a0f72 2024-09-21T03:45:50.673 INFO:teuthology.orchestra.run.smithi039.stdout:osd.9 up in weight 1 up_from 60 up_thru 78 down_at 0 last_clean_interval [0,0) [v2:172.21.15.39:6826/2001303253,v1:172.21.15.39:6827/2001303253] [v2:172.21.15.39:6828/2001303253,v1:172.21.15.39:6829/2001303253] exists,up 6028613c-ebfd-42ea-9949-3f0d5f6cf025 2024-09-21T03:45:50.673 INFO:teuthology.orchestra.run.smithi039.stdout:osd.10 up in weight 1 up_from 65 up_thru 75 down_at 0 last_clean_interval [0,0) [v2:172.21.15.150:6824/2484627938,v1:172.21.15.150:6825/2484627938] [v2:172.21.15.150:6826/2484627938,v1:172.21.15.150:6827/2484627938] exists,up 1c71c765-2a6c-4ffe-9fe1-c8adae3d1c32 2024-09-21T03:45:50.673 INFO:teuthology.orchestra.run.smithi039.stdout:osd.11 up in weight 1 up_from 71 up_thru 78 down_at 0 last_clean_interval [0,0) [v2:172.21.15.161:6824/2178709707,v1:172.21.15.161:6825/2178709707] [v2:172.21.15.161:6826/2178709707,v1:172.21.15.161:6827/2178709707] exists,up f382f522-62db-46da-997e-2b28f84d1612 2024-09-21T03:45:50.674 INFO:teuthology.orchestra.run.smithi039.stdout:pg_upmap_items 2.5 [8,11] 2024-09-21T03:45:50.674 INFO:teuthology.orchestra.run.smithi039.stdout:pg_upmap_items 2.8 [2,11] 2024-09-21T03:45:50.674 INFO:teuthology.orchestra.run.smithi039.stdout:pg_upmap_items 2.c [2,1] 2024-09-21T03:45:50.674 INFO:teuthology.orchestra.run.smithi039.stdout:pg_upmap_items 2.1f [8,4] 2024-09-21T03:45:50.674 INFO:teuthology.orchestra.run.smithi039.stdout:pg_upmap_items 3.2 [3,9] 2024-09-21T03:45:50.674 INFO:teuthology.orchestra.run.smithi039.stdout:pg_upmap_items 3.14 [4,6] 2024-09-21T03:45:50.674 INFO:teuthology.orchestra.run.smithi039.stdout:pg_upmap_items 3.1a [5,7] 2024-09-21T03:45:50.674 INFO:teuthology.orchestra.run.smithi039.stdout:pg_upmap_items 3.1d [4,8] 2024-09-21T03:45:50.674 INFO:teuthology.orchestra.run.smithi039.stdout:pg_upmap_items 3.2c [4,9] 2024-09-21T03:45:50.674 INFO:teuthology.orchestra.run.smithi039.stdout:pg_upmap_items 3.38 [3,9] 2024-09-21T03:45:50.674 INFO:teuthology.orchestra.run.smithi039.stdout:blocklist 172.21.15.39:6801/2438110611 expires 2024-09-22T03:37:25.391457+0000 2024-09-21T03:45:50.674 INFO:teuthology.orchestra.run.smithi039.stdout:blocklist 172.21.15.39:6800/2438110611 expires 2024-09-22T03:37:25.391457+0000 2024-09-21T03:45:50.675 INFO:teuthology.orchestra.run.smithi039.stdout:blocklist 172.21.15.39:6800/173264695 expires 2024-09-22T03:35:57.104467+0000 2024-09-21T03:45:50.675 INFO:teuthology.orchestra.run.smithi039.stdout:blocklist 172.21.15.39:6801/173264695 expires 2024-09-22T03:35:57.104467+0000 2024-09-21T03:45:50.675 INFO:teuthology.orchestra.run.smithi039.stdout:blocklist 172.21.15.39:0/2637182284 expires 2024-09-22T03:35:57.104467+0000 2024-09-21T03:45:50.675 INFO:teuthology.orchestra.run.smithi039.stdout:blocklist 172.21.15.39:0/1517304561 expires 2024-09-22T03:36:24.112592+0000 2024-09-21T03:45:50.675 INFO:teuthology.orchestra.run.smithi039.stdout:blocklist 172.21.15.39:6801/1435622199 expires 2024-09-22T03:36:24.112592+0000 2024-09-21T03:45:50.675 INFO:teuthology.orchestra.run.smithi039.stdout:blocklist 172.21.15.39:6800/1435622199 expires 2024-09-22T03:36:24.112592+0000 2024-09-21T03:45:50.675 INFO:teuthology.orchestra.run.smithi039.stdout:blocklist 172.21.15.39:0/346832469 expires 2024-09-22T03:35:57.104467+0000 2024-09-21T03:45:50.675 INFO:teuthology.orchestra.run.smithi039.stdout:blocklist 172.21.15.39:0/3160777371 expires 2024-09-22T03:36:24.112592+0000 2024-09-21T03:45:50.675 INFO:teuthology.orchestra.run.smithi039.stdout:blocklist 172.21.15.39:0/1730342855 expires 2024-09-22T03:35:57.104467+0000 2024-09-21T03:45:50.675 INFO:teuthology.orchestra.run.smithi039.stdout:blocklist 172.21.15.39:0/1955726766 expires 2024-09-22T03:36:24.112592+0000 2024-09-21T03:45:50.675 INFO:teuthology.orchestra.run.smithi039.stdout:blocklist 172.21.15.39:0/3294132796 expires 2024-09-22T03:37:25.391457+0000 2024-09-21T03:45:50.675 INFO:teuthology.orchestra.run.smithi039.stdout:blocklist 172.21.15.39:0/1295810162 expires 2024-09-22T03:37:25.391457+0000 2024-09-21T03:45:50.675 INFO:teuthology.orchestra.run.smithi039.stdout:blocklist 172.21.15.39:0/3729230474 expires 2024-09-22T03:37:25.391457+0000 2024-09-21T03:45:50.676 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:50.672+0000 7fafa6891640 1 -- 172.21.15.39:0/1630662883 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7faf8c0b65f0 msgr2=0x7faf8c0b8b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:50.676 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:50.672+0000 7fafa6891640 1 --2- 172.21.15.39:0/1630662883 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7faf8c0b65f0 0x7faf8c0b8b00 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fafa0104200 tx=0x7faf90031400 comp rx=0 tx=0).stop 2024-09-21T03:45:50.676 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:50.672+0000 7fafa6891640 1 -- 172.21.15.39:0/1630662883 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fafa01052d0 msgr2=0x7fafa0102cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:50.676 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:50.672+0000 7fafa6891640 1 --2- 172.21.15.39:0/1630662883 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fafa01052d0 0x7fafa0102cf0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7faf8800b450 tx=0x7faf8800b910 comp rx=0 tx=0).stop 2024-09-21T03:45:50.676 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:50.672+0000 7fafa6891640 1 -- 172.21.15.39:0/1630662883 shutdown_connections 2024-09-21T03:45:50.676 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:50.672+0000 7fafa6891640 1 --2- 172.21.15.39:0/1630662883 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7faf8c0b65f0 0x7faf8c0b8b00 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:50.677 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:50.672+0000 7fafa6891640 1 --2- 172.21.15.39:0/1630662883 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fafa010de50 0x7fafa0103230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:50.677 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:50.672+0000 7fafa6891640 1 --2- 172.21.15.39:0/1630662883 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fafa01052d0 0x7fafa0102cf0 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:50.677 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:50.672+0000 7fafa6891640 1 --2- 172.21.15.39:0/1630662883 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fafa01048a0 0x7fafa01027b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:50.677 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:50.672+0000 7fafa6891640 1 -- 172.21.15.39:0/1630662883 >> 172.21.15.39:0/1630662883 conn(0x7fafa0079730 msgr2=0x7fafa0100960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:45:50.677 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:50.672+0000 7fafa6891640 1 -- 172.21.15.39:0/1630662883 shutdown_connections 2024-09-21T03:45:50.677 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:50.672+0000 7fafa6891640 1 -- 172.21.15.39:0/1630662883 wait complete. 2024-09-21T03:45:51.217 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:45:50 smithi161 ceph-mon[34533]: pgmap v319: 97 pgs: 97 active+clean; 647 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-09-21T03:45:51.218 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:45:50 smithi161 ceph-mon[34533]: from='client.? 172.21.15.39:0/1630662883' entity='client.admin' cmd={"prefix": "osd dump"} : dispatch 2024-09-21T03:45:51.224 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:45:50 smithi150 ceph-mon[34289]: pgmap v319: 97 pgs: 97 active+clean; 647 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-09-21T03:45:51.224 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:45:50 smithi150 ceph-mon[34289]: from='client.? 172.21.15.39:0/1630662883' entity='client.admin' cmd={"prefix": "osd dump"} : dispatch 2024-09-21T03:45:51.243 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:45:50 smithi039 ceph-mon[31211]: pgmap v319: 97 pgs: 97 active+clean; 647 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-09-21T03:45:51.243 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:45:50 smithi039 ceph-mon[31211]: from='client.? 172.21.15.39:0/1630662883' entity='client.admin' cmd={"prefix": "osd dump"} : dispatch 2024-09-21T03:45:51.350 INFO:teuthology.run_tasks:Running task fs.ready... 2024-09-21T03:45:51.361 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 51116d5c-77ca-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T03:45:51.620 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/51116d5c-77ca-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:45:52.076 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:52.074+0000 7f86181f6640 1 -- 172.21.15.39:0/438042606 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f86101094a0 msgr2=0x7f86101098b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:52.076 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:52.074+0000 7f86181f6640 1 --2- 172.21.15.39:0/438042606 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f86101094a0 0x7f86101098b0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f8600009900 tx=0x7f860002a410 comp rx=0 tx=0).stop 2024-09-21T03:45:52.076 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:52.074+0000 7f86181f6640 1 -- 172.21.15.39:0/438042606 shutdown_connections 2024-09-21T03:45:52.076 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:52.074+0000 7f86181f6640 1 --2- 172.21.15.39:0/438042606 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8610109df0 0x7f8610110620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:52.077 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:52.074+0000 7f86181f6640 1 --2- 172.21.15.39:0/438042606 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f86101094a0 0x7f86101098b0 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:52.077 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:52.074+0000 7f86181f6640 1 --2- 172.21.15.39:0/438042606 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8610103b60 0x7f8610103f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:52.077 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:52.074+0000 7f86181f6640 1 -- 172.21.15.39:0/438042606 >> 172.21.15.39:0/438042606 conn(0x7f861007b850 msgr2=0x7f861007bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:45:52.077 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:52.074+0000 7f86181f6640 1 -- 172.21.15.39:0/438042606 shutdown_connections 2024-09-21T03:45:52.077 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:52.074+0000 7f86181f6640 1 -- 172.21.15.39:0/438042606 wait complete. 2024-09-21T03:45:52.078 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:52.075+0000 7f86181f6640 1 Processor -- start 2024-09-21T03:45:52.079 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:52.075+0000 7f86181f6640 1 -- start start 2024-09-21T03:45:52.079 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:52.075+0000 7f86181f6640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8610103b60 0x7f8610075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:52.079 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:52.075+0000 7f86181f6640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f86101094a0 0x7f8610075c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:52.079 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:52.075+0000 7f86181f6640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8610109df0 0x7f8610076190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:52.079 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:52.075+0000 7f86181f6640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f8610113ad0 con 0x7f8610109df0 2024-09-21T03:45:52.079 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:52.075+0000 7f86181f6640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f8610113950 con 0x7f86101094a0 2024-09-21T03:45:52.079 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:52.075+0000 7f86181f6640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f8610113c50 con 0x7f8610103b60 2024-09-21T03:45:52.079 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:52.075+0000 7f8616793640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8610109df0 0x7f8610076190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:52.079 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:52.076+0000 7f8616793640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8610109df0 0x7f8610076190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:35522/0 (socket says 172.21.15.39:35522) 2024-09-21T03:45:52.079 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:52.076+0000 7f8616793640 1 -- 172.21.15.39:0/1473553186 learned_addr learned my addr 172.21.15.39:0/1473553186 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:45:52.080 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:52.076+0000 7f8615791640 1 --2- 172.21.15.39:0/1473553186 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f86101094a0 0x7f8610075c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:52.080 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:52.076+0000 7f8615f92640 1 --2- 172.21.15.39:0/1473553186 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8610103b60 0x7f8610075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:52.080 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:52.076+0000 7f8616793640 1 -- 172.21.15.39:0/1473553186 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8610103b60 msgr2=0x7f8610075710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:52.080 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:52.076+0000 7f8616793640 1 --2- 172.21.15.39:0/1473553186 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8610103b60 0x7f8610075710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:52.080 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:52.076+0000 7f8616793640 1 -- 172.21.15.39:0/1473553186 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f86101094a0 msgr2=0x7f8610075c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:52.080 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:52.076+0000 7f8616793640 1 --2- 172.21.15.39:0/1473553186 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f86101094a0 0x7f8610075c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:52.080 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:52.076+0000 7f8616793640 1 -- 172.21.15.39:0/1473553186 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f86101a8780 con 0x7f8610109df0 2024-09-21T03:45:52.080 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:52.076+0000 7f8616793640 1 --2- 172.21.15.39:0/1473553186 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8610109df0 0x7f8610076190 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f860800eaa0 tx=0x7f860800ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:45:52.080 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:52.076+0000 7f8606ffd640 1 -- 172.21.15.39:0/1473553186 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f860800cdb0 con 0x7f8610109df0 2024-09-21T03:45:52.080 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:52.076+0000 7f86181f6640 1 -- 172.21.15.39:0/1473553186 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f86101a8a70 con 0x7f8610109df0 2024-09-21T03:45:52.080 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:52.076+0000 7f8606ffd640 1 -- 172.21.15.39:0/1473553186 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8608005020 con 0x7f8610109df0 2024-09-21T03:45:52.080 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:52.076+0000 7f8606ffd640 1 -- 172.21.15.39:0/1473553186 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86080116d0 con 0x7f8610109df0 2024-09-21T03:45:52.080 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:52.076+0000 7f86181f6640 1 -- 172.21.15.39:0/1473553186 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f86101a8fb0 con 0x7f8610109df0 2024-09-21T03:45:52.081 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:52.077+0000 7f86181f6640 1 -- 172.21.15.39:0/1473553186 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8610103f70 con 0x7f8610109df0 2024-09-21T03:45:52.081 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:52.080+0000 7f8606ffd640 1 -- 172.21.15.39:0/1473553186 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 22) ==== 154580+0+0 (secure 0 0 0) 0x7f86080051d0 con 0x7f8610109df0 2024-09-21T03:45:52.082 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:52.081+0000 7f8606ffd640 1 --2- 172.21.15.39:0/1473553186 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f85fc0b65f0 0x7f85fc0b8b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:52.082 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:52.081+0000 7f8615f92640 1 --2- 172.21.15.39:0/1473553186 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f85fc0b65f0 0x7f85fc0b8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:52.083 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:52.081+0000 7f8606ffd640 1 -- 172.21.15.39:0/1473553186 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8484+0+0 (secure 0 0 0) 0x7f86080e7c20 con 0x7f8610109df0 2024-09-21T03:45:52.083 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:52.081+0000 7f8615f92640 1 --2- 172.21.15.39:0/1473553186 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f85fc0b65f0 0x7f85fc0b8b00 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f8610077160 tx=0x7f860c008040 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:45:52.085 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:52.084+0000 7f8606ffd640 1 -- 172.21.15.39:0/1473553186 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f8608016030 con 0x7f8610109df0 2024-09-21T03:45:52.315 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:52.313+0000 7f86181f6640 1 -- 172.21.15.39:0/1473553186 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f8610076a10 con 0x7f8610109df0 2024-09-21T03:45:52.316 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:52.314+0000 7f8606ffd640 1 -- 172.21.15.39:0/1473553186 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11978 (secure 0 0 0) 0x7f8610076a10 con 0x7f8610109df0 2024-09-21T03:45:52.316 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T03:45:52.316 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":28,"btime":"2024-09-21T03:44:26:704224+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14790,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6835/508995295","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":508995295},{"type":"v1","addr":"172.21.15.39:6835","nonce":508995295}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14802,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6837/3915912607","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":3915912607},{"type":"v1","addr":"172.21.15.39:6837","nonce":3915912607}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T03:44:15.009872+0000","modified":"2024-09-21T03:44:26.017327+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24695,"mds_1":24667,"mds_2":24659,"mds_3":24646,"mds_4":14868},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14856":{"gid":14856,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.39:6839/185509192","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6838","nonce":185509192},{"type":"v1","addr":"172.21.15.39:6839","nonce":185509192}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14868":{"gid":14868,"name":"a","rank":4,"incarnation":25,"state":"up:active","state_seq":7,"addr":"172.21.15.39:6841/4247190456","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6840","nonce":4247190456},{"type":"v1","addr":"172.21.15.39:6841","nonce":4247190456}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24599":{"gid":24599,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.150:6833/3114643217","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":3114643217},{"type":"v1","addr":"172.21.15.150:6833","nonce":3114643217}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24601":{"gid":24601,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.161:6833/2963990654","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":2963990654},{"type":"v1","addr":"172.21.15.161:6833","nonce":2963990654}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24632":{"gid":24632,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.150:6835/724594008","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6834","nonce":724594008},{"type":"v1","addr":"172.21.15.150:6835","nonce":724594008}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24640":{"gid":24640,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.161:6835/3747882110","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":3747882110},{"type":"v1","addr":"172.21.15.161:6835","nonce":3747882110}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24646":{"gid":24646,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":10,"addr":"172.21.15.161:6837/3451544597","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6836","nonce":3451544597},{"type":"v1","addr":"172.21.15.161:6837","nonce":3451544597}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24659":{"gid":24659,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.150:6837/4017043144","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6836","nonce":4017043144},{"type":"v1","addr":"172.21.15.150:6837","nonce":4017043144}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24667":{"gid":24667,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.161:6839/2190107270","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6838","nonce":2190107270},{"type":"v1","addr":"172.21.15.161:6839","nonce":2190107270}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24695":{"gid":24695,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.150:6839/2949386907","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6838","nonce":2949386907},{"type":"v1","addr":"172.21.15.150:6839","nonce":2949386907}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24695,"qdb_cluster":[14868,24646,24659,24667,24695]},"id":1}]} 2024-09-21T03:45:52.317 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 28 2024-09-21T03:45:52.319 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:52.317+0000 7f86181f6640 1 -- 172.21.15.39:0/1473553186 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f85fc0b65f0 msgr2=0x7f85fc0b8b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:52.319 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:52.317+0000 7f86181f6640 1 --2- 172.21.15.39:0/1473553186 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f85fc0b65f0 0x7f85fc0b8b00 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f8610077160 tx=0x7f860c008040 comp rx=0 tx=0).stop 2024-09-21T03:45:52.319 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:52.317+0000 7f86181f6640 1 -- 172.21.15.39:0/1473553186 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8610109df0 msgr2=0x7f8610076190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:52.319 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:52.317+0000 7f86181f6640 1 --2- 172.21.15.39:0/1473553186 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8610109df0 0x7f8610076190 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f860800eaa0 tx=0x7f860800ef60 comp rx=0 tx=0).stop 2024-09-21T03:45:52.319 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:52.317+0000 7f86181f6640 1 -- 172.21.15.39:0/1473553186 shutdown_connections 2024-09-21T03:45:52.319 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:52.317+0000 7f86181f6640 1 --2- 172.21.15.39:0/1473553186 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f85fc0b65f0 0x7f85fc0b8b00 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:52.319 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:52.317+0000 7f86181f6640 1 --2- 172.21.15.39:0/1473553186 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8610109df0 0x7f8610076190 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:52.319 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:52.317+0000 7f86181f6640 1 --2- 172.21.15.39:0/1473553186 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f86101094a0 0x7f8610075c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:52.319 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:52.317+0000 7f86181f6640 1 --2- 172.21.15.39:0/1473553186 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8610103b60 0x7f8610075710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:52.319 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:52.317+0000 7f86181f6640 1 -- 172.21.15.39:0/1473553186 >> 172.21.15.39:0/1473553186 conn(0x7f861007b850 msgr2=0x7f8610107530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:45:52.319 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:52.317+0000 7f86181f6640 1 -- 172.21.15.39:0/1473553186 shutdown_connections 2024-09-21T03:45:52.319 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:52.317+0000 7f86181f6640 1 -- 172.21.15.39:0/1473553186 wait complete. 2024-09-21T03:45:52.975 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 51116d5c-77ca-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T03:45:53.217 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:45:52 smithi161 ceph-mon[34533]: pgmap v320: 97 pgs: 97 active+clean; 647 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-21T03:45:53.218 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:45:52 smithi161 ceph-mon[34533]: from='client.? 172.21.15.39:0/1473553186' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:45:53.224 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:45:52 smithi150 ceph-mon[34289]: pgmap v320: 97 pgs: 97 active+clean; 647 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-21T03:45:53.224 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:45:52 smithi150 ceph-mon[34289]: from='client.? 172.21.15.39:0/1473553186' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:45:53.229 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/51116d5c-77ca-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:45:53.334 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:45:52 smithi039 ceph-mon[31211]: pgmap v320: 97 pgs: 97 active+clean; 647 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-21T03:45:53.334 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:45:52 smithi039 ceph-mon[31211]: from='client.? 172.21.15.39:0/1473553186' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:45:53.724 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:53.705+0000 7fce95518640 1 -- 172.21.15.39:0/1092521316 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fce90108d20 msgr2=0x7fce9010f550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:53.724 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:53.705+0000 7fce95518640 1 --2- 172.21.15.39:0/1092521316 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fce90108d20 0x7fce9010f550 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7fce80009ca0 tx=0x7fce8002a8c0 comp rx=0 tx=0).stop 2024-09-21T03:45:53.725 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:53.705+0000 7fce95518640 1 -- 172.21.15.39:0/1092521316 shutdown_connections 2024-09-21T03:45:53.725 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:53.705+0000 7fce95518640 1 --2- 172.21.15.39:0/1092521316 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fce90108d20 0x7fce9010f550 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:53.725 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:53.705+0000 7fce95518640 1 --2- 172.21.15.39:0/1092521316 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fce901083d0 0x7fce901087e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:53.725 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:53.705+0000 7fce95518640 1 --2- 172.21.15.39:0/1092521316 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fce90102ae0 0x7fce90102ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:53.725 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:53.705+0000 7fce95518640 1 -- 172.21.15.39:0/1092521316 >> 172.21.15.39:0/1092521316 conn(0x7fce900797b0 msgr2=0x7fce90079bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:45:53.725 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:53.705+0000 7fce95518640 1 -- 172.21.15.39:0/1092521316 shutdown_connections 2024-09-21T03:45:53.725 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:53.705+0000 7fce95518640 1 -- 172.21.15.39:0/1092521316 wait complete. 2024-09-21T03:45:53.725 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:53.706+0000 7fce95518640 1 Processor -- start 2024-09-21T03:45:53.725 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:53.706+0000 7fce95518640 1 -- start start 2024-09-21T03:45:53.725 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:53.706+0000 7fce95518640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fce90102ae0 0x7fce9019a450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:53.725 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:53.706+0000 7fce95518640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fce901083d0 0x7fce9019a9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:53.726 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:53.706+0000 7fce95518640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fce90108d20 0x7fce9019fc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:53.726 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:53.706+0000 7fce95518640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fce90113b00 con 0x7fce90108d20 2024-09-21T03:45:53.726 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:53.706+0000 7fce95518640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fce90113980 con 0x7fce90102ae0 2024-09-21T03:45:53.726 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:53.706+0000 7fce95518640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fce90113c80 con 0x7fce901083d0 2024-09-21T03:45:53.726 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:53.706+0000 7fce8f7fe640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fce90108d20 0x7fce9019fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:53.726 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:53.706+0000 7fce8f7fe640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fce90108d20 0x7fce9019fc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:35538/0 (socket says 172.21.15.39:35538) 2024-09-21T03:45:53.726 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:53.707+0000 7fce8f7fe640 1 -- 172.21.15.39:0/2535477703 learned_addr learned my addr 172.21.15.39:0/2535477703 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:45:53.726 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:53.707+0000 7fce8e7fc640 1 --2- 172.21.15.39:0/2535477703 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fce901083d0 0x7fce9019a9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:53.726 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:53.707+0000 7fce8effd640 1 --2- 172.21.15.39:0/2535477703 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fce90102ae0 0x7fce9019a450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:53.726 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:53.707+0000 7fce8f7fe640 1 -- 172.21.15.39:0/2535477703 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fce901083d0 msgr2=0x7fce9019a9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:53.726 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:53.707+0000 7fce8f7fe640 1 --2- 172.21.15.39:0/2535477703 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fce901083d0 0x7fce9019a9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:53.726 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:53.707+0000 7fce8f7fe640 1 -- 172.21.15.39:0/2535477703 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fce90102ae0 msgr2=0x7fce9019a450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:53.727 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:53.707+0000 7fce8f7fe640 1 --2- 172.21.15.39:0/2535477703 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fce90102ae0 0x7fce9019a450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:53.727 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:53.707+0000 7fce8f7fe640 1 -- 172.21.15.39:0/2535477703 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fce901a03c0 con 0x7fce90108d20 2024-09-21T03:45:53.727 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:53.707+0000 7fce8f7fe640 1 --2- 172.21.15.39:0/2535477703 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fce90108d20 0x7fce9019fc90 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7fce800098d0 tx=0x7fce800317b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:45:53.727 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:53.707+0000 7fce87fff640 1 -- 172.21.15.39:0/2535477703 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce80003b70 con 0x7fce90108d20 2024-09-21T03:45:53.727 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:53.707+0000 7fce95518640 1 -- 172.21.15.39:0/2535477703 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fce901a0650 con 0x7fce90108d20 2024-09-21T03:45:53.727 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:53.707+0000 7fce87fff640 1 -- 172.21.15.39:0/2535477703 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fce80003d10 con 0x7fce90108d20 2024-09-21T03:45:53.727 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:53.707+0000 7fce87fff640 1 -- 172.21.15.39:0/2535477703 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce8002fa20 con 0x7fce90108d20 2024-09-21T03:45:53.727 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:53.707+0000 7fce95518640 1 -- 172.21.15.39:0/2535477703 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fce901a6e10 con 0x7fce90108d20 2024-09-21T03:45:53.727 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:53.708+0000 7fce95518640 1 -- 172.21.15.39:0/2535477703 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fce90102ef0 con 0x7fce90108d20 2024-09-21T03:45:53.727 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:53.710+0000 7fce87fff640 1 -- 172.21.15.39:0/2535477703 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 22) ==== 154580+0+0 (secure 0 0 0) 0x7fce8002fc20 con 0x7fce90108d20 2024-09-21T03:45:53.727 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:53.711+0000 7fce87fff640 1 --2- 172.21.15.39:0/2535477703 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7fce7c0b65f0 0x7fce7c0b8b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:53.727 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:53.711+0000 7fce8effd640 1 --2- 172.21.15.39:0/2535477703 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7fce7c0b65f0 0x7fce7c0b8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:53.727 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:53.711+0000 7fce87fff640 1 -- 172.21.15.39:0/2535477703 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8484+0+0 (secure 0 0 0) 0x7fce80108a10 con 0x7fce90108d20 2024-09-21T03:45:53.728 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:53.712+0000 7fce8effd640 1 --2- 172.21.15.39:0/2535477703 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7fce7c0b65f0 0x7fce7c0b8b00 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fce88005d00 tx=0x7fce8800b410 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:45:53.728 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:53.714+0000 7fce87fff640 1 -- 172.21.15.39:0/2535477703 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fce800cf040 con 0x7fce90108d20 2024-09-21T03:45:53.946 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:53.943+0000 7fce95518640 1 -- 172.21.15.39:0/2535477703 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fce9010ab70 con 0x7fce90108d20 2024-09-21T03:45:53.947 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:53.945+0000 7fce87fff640 1 -- 172.21.15.39:0/2535477703 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11978 (secure 0 0 0) 0x7fce800d3420 con 0x7fce90108d20 2024-09-21T03:45:53.948 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T03:45:53.948 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":28,"btime":"2024-09-21T03:44:26:704224+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14790,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6835/508995295","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":508995295},{"type":"v1","addr":"172.21.15.39:6835","nonce":508995295}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14802,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6837/3915912607","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":3915912607},{"type":"v1","addr":"172.21.15.39:6837","nonce":3915912607}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T03:44:15.009872+0000","modified":"2024-09-21T03:44:26.017327+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24695,"mds_1":24667,"mds_2":24659,"mds_3":24646,"mds_4":14868},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14856":{"gid":14856,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.39:6839/185509192","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6838","nonce":185509192},{"type":"v1","addr":"172.21.15.39:6839","nonce":185509192}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14868":{"gid":14868,"name":"a","rank":4,"incarnation":25,"state":"up:active","state_seq":7,"addr":"172.21.15.39:6841/4247190456","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6840","nonce":4247190456},{"type":"v1","addr":"172.21.15.39:6841","nonce":4247190456}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24599":{"gid":24599,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.150:6833/3114643217","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":3114643217},{"type":"v1","addr":"172.21.15.150:6833","nonce":3114643217}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24601":{"gid":24601,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.161:6833/2963990654","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":2963990654},{"type":"v1","addr":"172.21.15.161:6833","nonce":2963990654}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24632":{"gid":24632,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.150:6835/724594008","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6834","nonce":724594008},{"type":"v1","addr":"172.21.15.150:6835","nonce":724594008}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24640":{"gid":24640,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.161:6835/3747882110","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":3747882110},{"type":"v1","addr":"172.21.15.161:6835","nonce":3747882110}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24646":{"gid":24646,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":10,"addr":"172.21.15.161:6837/3451544597","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6836","nonce":3451544597},{"type":"v1","addr":"172.21.15.161:6837","nonce":3451544597}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24659":{"gid":24659,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.150:6837/4017043144","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6836","nonce":4017043144},{"type":"v1","addr":"172.21.15.150:6837","nonce":4017043144}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24667":{"gid":24667,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.161:6839/2190107270","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6838","nonce":2190107270},{"type":"v1","addr":"172.21.15.161:6839","nonce":2190107270}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24695":{"gid":24695,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.150:6839/2949386907","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6838","nonce":2949386907},{"type":"v1","addr":"172.21.15.150:6839","nonce":2949386907}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24695,"qdb_cluster":[14868,24646,24659,24667,24695]},"id":1}]} 2024-09-21T03:45:53.948 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 28 2024-09-21T03:45:53.951 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:53.948+0000 7fce95518640 1 -- 172.21.15.39:0/2535477703 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7fce7c0b65f0 msgr2=0x7fce7c0b8b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:53.951 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:53.948+0000 7fce95518640 1 --2- 172.21.15.39:0/2535477703 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7fce7c0b65f0 0x7fce7c0b8b00 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fce88005d00 tx=0x7fce8800b410 comp rx=0 tx=0).stop 2024-09-21T03:45:53.951 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:53.948+0000 7fce95518640 1 -- 172.21.15.39:0/2535477703 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fce90108d20 msgr2=0x7fce9019fc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:53.951 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:53.948+0000 7fce95518640 1 --2- 172.21.15.39:0/2535477703 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fce90108d20 0x7fce9019fc90 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7fce800098d0 tx=0x7fce800317b0 comp rx=0 tx=0).stop 2024-09-21T03:45:53.951 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:53.948+0000 7fce95518640 1 -- 172.21.15.39:0/2535477703 shutdown_connections 2024-09-21T03:45:53.951 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:53.948+0000 7fce95518640 1 --2- 172.21.15.39:0/2535477703 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7fce7c0b65f0 0x7fce7c0b8b00 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:53.952 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:53.948+0000 7fce95518640 1 --2- 172.21.15.39:0/2535477703 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fce90108d20 0x7fce9019fc90 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:53.952 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:53.948+0000 7fce95518640 1 --2- 172.21.15.39:0/2535477703 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fce901083d0 0x7fce9019a9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:53.952 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:53.948+0000 7fce95518640 1 --2- 172.21.15.39:0/2535477703 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fce90102ae0 0x7fce9019a450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:53.952 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:53.948+0000 7fce95518640 1 -- 172.21.15.39:0/2535477703 >> 172.21.15.39:0/2535477703 conn(0x7fce900797b0 msgr2=0x7fce90106460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:45:53.952 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:53.948+0000 7fce95518640 1 -- 172.21.15.39:0/2535477703 shutdown_connections 2024-09-21T03:45:53.952 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:53.948+0000 7fce95518640 1 -- 172.21.15.39:0/2535477703 wait complete. 2024-09-21T03:45:54.664 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 51116d5c-77ca-11ef-baf5-efdc52797490 -- ceph osd dump --format=json 2024-09-21T03:45:54.925 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/51116d5c-77ca-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:45:55.204 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:45:54 smithi039 ceph-mon[31211]: pgmap v321: 97 pgs: 97 active+clean; 647 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-09-21T03:45:55.204 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:45:54 smithi039 ceph-mon[31211]: from='client.? 172.21.15.39:0/2535477703' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:45:55.217 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:45:54 smithi161 ceph-mon[34533]: pgmap v321: 97 pgs: 97 active+clean; 647 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-09-21T03:45:55.218 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:45:54 smithi161 ceph-mon[34533]: from='client.? 172.21.15.39:0/2535477703' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:45:55.225 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:45:54 smithi150 ceph-mon[34289]: pgmap v321: 97 pgs: 97 active+clean; 647 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-09-21T03:45:55.225 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:45:54 smithi150 ceph-mon[34289]: from='client.? 172.21.15.39:0/2535477703' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:45:55.403 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:55.399+0000 7f39c0291640 1 -- 172.21.15.39:0/4200933113 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f39b810de10 msgr2=0x7f39b8110300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:55.403 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:55.399+0000 7f39c0291640 1 --2- 172.21.15.39:0/4200933113 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f39b810de10 0x7f39b8110300 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f39b0009ca0 tx=0x7f39b002a8c0 comp rx=0 tx=0).stop 2024-09-21T03:45:55.403 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:55.399+0000 7f39c0291640 1 -- 172.21.15.39:0/4200933113 shutdown_connections 2024-09-21T03:45:55.403 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:55.399+0000 7f39c0291640 1 --2- 172.21.15.39:0/4200933113 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f39b810de10 0x7f39b8110300 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:55.403 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:55.399+0000 7f39c0291640 1 --2- 172.21.15.39:0/4200933113 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f39b8105300 0x7f39b810d850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:55.403 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:55.399+0000 7f39c0291640 1 --2- 172.21.15.39:0/4200933113 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f39b81049b0 0x7f39b8104dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:55.403 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:55.399+0000 7f39c0291640 1 -- 172.21.15.39:0/4200933113 >> 172.21.15.39:0/4200933113 conn(0x7f39b8079730 msgr2=0x7f39b8079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:45:55.403 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:55.399+0000 7f39c0291640 1 -- 172.21.15.39:0/4200933113 shutdown_connections 2024-09-21T03:45:55.403 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:55.399+0000 7f39c0291640 1 -- 172.21.15.39:0/4200933113 wait complete. 2024-09-21T03:45:55.403 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:55.400+0000 7f39c0291640 1 Processor -- start 2024-09-21T03:45:55.403 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:55.400+0000 7f39c0291640 1 -- start start 2024-09-21T03:45:55.404 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:55.400+0000 7f39c0291640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f39b81049b0 0x7f39b819a500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:55.404 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:55.400+0000 7f39c0291640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f39b8105300 0x7f39b819aaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:55.404 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:55.400+0000 7f39c0291640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f39b810de10 0x7f39b819fd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:55.405 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:55.400+0000 7f39c0291640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f39b8113a90 con 0x7f39b810de10 2024-09-21T03:45:55.405 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:55.400+0000 7f39c0291640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f39b8113910 con 0x7f39b81049b0 2024-09-21T03:45:55.405 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:55.400+0000 7f39c0291640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f39b8113c10 con 0x7f39b8105300 2024-09-21T03:45:55.405 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:55.400+0000 7f39be82e640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f39b810de10 0x7f39b819fd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:55.406 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:55.400+0000 7f39be82e640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f39b810de10 0x7f39b819fd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:35552/0 (socket says 172.21.15.39:35552) 2024-09-21T03:45:55.406 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:55.400+0000 7f39be82e640 1 -- 172.21.15.39:0/73162835 learned_addr learned my addr 172.21.15.39:0/73162835 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:45:55.406 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:55.400+0000 7f39bd82c640 1 --2- 172.21.15.39:0/73162835 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f39b8105300 0x7f39b819aaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:55.406 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:55.400+0000 7f39be02d640 1 --2- 172.21.15.39:0/73162835 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f39b81049b0 0x7f39b819a500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:55.406 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:55.401+0000 7f39be82e640 1 -- 172.21.15.39:0/73162835 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f39b8105300 msgr2=0x7f39b819aaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:55.406 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:55.401+0000 7f39be82e640 1 --2- 172.21.15.39:0/73162835 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f39b8105300 0x7f39b819aaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:55.406 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:55.401+0000 7f39be82e640 1 -- 172.21.15.39:0/73162835 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f39b81049b0 msgr2=0x7f39b819a500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:55.406 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:55.401+0000 7f39be82e640 1 --2- 172.21.15.39:0/73162835 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f39b81049b0 0x7f39b819a500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:55.406 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:55.401+0000 7f39be82e640 1 -- 172.21.15.39:0/73162835 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f39b81a0470 con 0x7f39b810de10 2024-09-21T03:45:55.406 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:55.401+0000 7f39be82e640 1 --2- 172.21.15.39:0/73162835 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f39b810de10 0x7f39b819fd40 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f39b00098d0 tx=0x7f39b00317b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:45:55.406 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:55.401+0000 7f39af7fe640 1 -- 172.21.15.39:0/73162835 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39b0003b70 con 0x7f39b810de10 2024-09-21T03:45:55.406 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:55.401+0000 7f39c0291640 1 -- 172.21.15.39:0/73162835 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f39b81a0700 con 0x7f39b810de10 2024-09-21T03:45:55.406 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:55.401+0000 7f39c0291640 1 -- 172.21.15.39:0/73162835 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f39b81a6ec0 con 0x7f39b810de10 2024-09-21T03:45:55.406 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:55.401+0000 7f39af7fe640 1 -- 172.21.15.39:0/73162835 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f39b0003d10 con 0x7f39b810de10 2024-09-21T03:45:55.407 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:55.401+0000 7f39af7fe640 1 -- 172.21.15.39:0/73162835 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39b002fa20 con 0x7f39b810de10 2024-09-21T03:45:55.407 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:55.403+0000 7f39c0291640 1 -- 172.21.15.39:0/73162835 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f39b8104dc0 con 0x7f39b810de10 2024-09-21T03:45:55.407 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:55.404+0000 7f39af7fe640 1 -- 172.21.15.39:0/73162835 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 22) ==== 154580+0+0 (secure 0 0 0) 0x7f39b002fbe0 con 0x7f39b810de10 2024-09-21T03:45:55.407 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:55.405+0000 7f39af7fe640 1 --2- 172.21.15.39:0/73162835 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f39a40b65f0 0x7f39a40b8b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:55.408 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:55.405+0000 7f39be02d640 1 --2- 172.21.15.39:0/73162835 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f39a40b65f0 0x7f39a40b8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:55.408 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:55.405+0000 7f39af7fe640 1 -- 172.21.15.39:0/73162835 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8484+0+0 (secure 0 0 0) 0x7f39b0108a40 con 0x7f39b810de10 2024-09-21T03:45:55.408 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:55.406+0000 7f39be02d640 1 --2- 172.21.15.39:0/73162835 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f39a40b65f0 0x7f39a40b8b00 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f39b40058c0 tx=0x7f39b400a2d0 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:45:55.411 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:55.408+0000 7f39af7fe640 1 -- 172.21.15.39:0/73162835 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f39b00cf100 con 0x7f39b810de10 2024-09-21T03:45:55.603 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:55.600+0000 7f39c0291640 1 -- 172.21.15.39:0/73162835 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f39b8108f10 con 0x7f39b810de10 2024-09-21T03:45:55.604 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:55.602+0000 7f39af7fe640 1 -- 172.21.15.39:0/73162835 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v79) ==== 74+0+23096 (secure 0 0 0) 0x7f39b00d34e0 con 0x7f39b810de10 2024-09-21T03:45:55.605 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T03:45:55.605 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":79,"fsid":"51116d5c-77ca-11ef-baf5-efdc52797490","created":"2024-09-21T03:35:33.631350+0000","modified":"2024-09-21T03:44:27.402588+0000","last_up_change":"2024-09-21T03:43:24.242275+0000","last_in_change":"2024-09-21T03:43:00.886552+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T03:39:29.859542+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-21T03:44:10.886435+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"77","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-21T03:44:13.041783+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"77","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1ea7148f-2ace-4c10-86fe-b8019de80c1f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6802","nonce":870188791},{"type":"v1","addr":"172.21.15.39:6803","nonce":870188791}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6804","nonce":870188791},{"type":"v1","addr":"172.21.15.39:6805","nonce":870188791}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6808","nonce":870188791},{"type":"v1","addr":"172.21.15.39:6809","nonce":870188791}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6806","nonce":870188791},{"type":"v1","addr":"172.21.15.39:6807","nonce":870188791}]},"public_addr":"172.21.15.39:6803/870188791","cluster_addr":"172.21.15.39:6805/870188791","heartbeat_back_addr":"172.21.15.39:6809/870188791","heartbeat_front_addr":"172.21.15.39:6807/870188791","state":["exists","up"]},{"osd":1,"uuid":"7cb5db90-28e8-4a30-bedb-c039687d48b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6800","nonce":172427832},{"type":"v1","addr":"172.21.15.150:6801","nonce":172427832}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6802","nonce":172427832},{"type":"v1","addr":"172.21.15.150:6803","nonce":172427832}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6806","nonce":172427832},{"type":"v1","addr":"172.21.15.150:6807","nonce":172427832}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6804","nonce":172427832},{"type":"v1","addr":"172.21.15.150:6805","nonce":172427832}]},"public_addr":"172.21.15.150:6801/172427832","cluster_addr":"172.21.15.150:6803/172427832","heartbeat_back_addr":"172.21.15.150:6807/172427832","heartbeat_front_addr":"172.21.15.150:6805/172427832","state":["exists","up"]},{"osd":2,"uuid":"b25c6ab3-fc95-4751-bd63-6e3fc23e8bd7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":19,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6800","nonce":2727876133},{"type":"v1","addr":"172.21.15.161:6801","nonce":2727876133}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6802","nonce":2727876133},{"type":"v1","addr":"172.21.15.161:6803","nonce":2727876133}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6806","nonce":2727876133},{"type":"v1","addr":"172.21.15.161:6807","nonce":2727876133}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6804","nonce":2727876133},{"type":"v1","addr":"172.21.15.161:6805","nonce":2727876133}]},"public_addr":"172.21.15.161:6801/2727876133","cluster_addr":"172.21.15.161:6803/2727876133","heartbeat_back_addr":"172.21.15.161:6807/2727876133","heartbeat_front_addr":"172.21.15.161:6805/2727876133","state":["exists","up"]},{"osd":3,"uuid":"14417ae1-4536-45ba-9a5c-c3b447d1a72d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6810","nonce":295394468},{"type":"v1","addr":"172.21.15.39:6811","nonce":295394468}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6812","nonce":295394468},{"type":"v1","addr":"172.21.15.39:6813","nonce":295394468}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6816","nonce":295394468},{"type":"v1","addr":"172.21.15.39:6817","nonce":295394468}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6814","nonce":295394468},{"type":"v1","addr":"172.21.15.39:6815","nonce":295394468}]},"public_addr":"172.21.15.39:6811/295394468","cluster_addr":"172.21.15.39:6813/295394468","heartbeat_back_addr":"172.21.15.39:6817/295394468","heartbeat_front_addr":"172.21.15.39:6815/295394468","state":["exists","up"]},{"osd":4,"uuid":"33c732ef-20c2-43c4-abe6-045b687fc521","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6808","nonce":3097721553},{"type":"v1","addr":"172.21.15.150:6809","nonce":3097721553}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6810","nonce":3097721553},{"type":"v1","addr":"172.21.15.150:6811","nonce":3097721553}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6814","nonce":3097721553},{"type":"v1","addr":"172.21.15.150:6815","nonce":3097721553}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6812","nonce":3097721553},{"type":"v1","addr":"172.21.15.150:6813","nonce":3097721553}]},"public_addr":"172.21.15.150:6809/3097721553","cluster_addr":"172.21.15.150:6811/3097721553","heartbeat_back_addr":"172.21.15.150:6815/3097721553","heartbeat_front_addr":"172.21.15.150:6813/3097721553","state":["exists","up"]},{"osd":5,"uuid":"74673722-abd6-400a-bb50-ec3dfba0c5e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6808","nonce":742751023},{"type":"v1","addr":"172.21.15.161:6809","nonce":742751023}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6810","nonce":742751023},{"type":"v1","addr":"172.21.15.161:6811","nonce":742751023}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6814","nonce":742751023},{"type":"v1","addr":"172.21.15.161:6815","nonce":742751023}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6812","nonce":742751023},{"type":"v1","addr":"172.21.15.161:6813","nonce":742751023}]},"public_addr":"172.21.15.161:6809/742751023","cluster_addr":"172.21.15.161:6811/742751023","heartbeat_back_addr":"172.21.15.161:6815/742751023","heartbeat_front_addr":"172.21.15.161:6813/742751023","state":["exists","up"]},{"osd":6,"uuid":"06548677-74c4-4efa-8054-df09b957b649","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6818","nonce":1072700290},{"type":"v1","addr":"172.21.15.39:6819","nonce":1072700290}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6820","nonce":1072700290},{"type":"v1","addr":"172.21.15.39:6821","nonce":1072700290}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6824","nonce":1072700290},{"type":"v1","addr":"172.21.15.39:6825","nonce":1072700290}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6822","nonce":1072700290},{"type":"v1","addr":"172.21.15.39:6823","nonce":1072700290}]},"public_addr":"172.21.15.39:6819/1072700290","cluster_addr":"172.21.15.39:6821/1072700290","heartbeat_back_addr":"172.21.15.39:6825/1072700290","heartbeat_front_addr":"172.21.15.39:6823/1072700290","state":["exists","up"]},{"osd":7,"uuid":"838735ca-c693-4f58-9426-b8ee857dbe3a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6816","nonce":2162619103},{"type":"v1","addr":"172.21.15.150:6817","nonce":2162619103}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6818","nonce":2162619103},{"type":"v1","addr":"172.21.15.150:6819","nonce":2162619103}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6822","nonce":2162619103},{"type":"v1","addr":"172.21.15.150:6823","nonce":2162619103}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6820","nonce":2162619103},{"type":"v1","addr":"172.21.15.150:6821","nonce":2162619103}]},"public_addr":"172.21.15.150:6817/2162619103","cluster_addr":"172.21.15.150:6819/2162619103","heartbeat_back_addr":"172.21.15.150:6823/2162619103","heartbeat_front_addr":"172.21.15.150:6821/2162619103","state":["exists","up"]},{"osd":8,"uuid":"4973b357-b4c2-4c00-b7f7-dd25012a0f72","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6816","nonce":52966633},{"type":"v1","addr":"172.21.15.161:6817","nonce":52966633}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6818","nonce":52966633},{"type":"v1","addr":"172.21.15.161:6819","nonce":52966633}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6822","nonce":52966633},{"type":"v1","addr":"172.21.15.161:6823","nonce":52966633}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6820","nonce":52966633},{"type":"v1","addr":"172.21.15.161:6821","nonce":52966633}]},"public_addr":"172.21.15.161:6817/52966633","cluster_addr":"172.21.15.161:6819/52966633","heartbeat_back_addr":"172.21.15.161:6823/52966633","heartbeat_front_addr":"172.21.15.161:6821/52966633","state":["exists","up"]},{"osd":9,"uuid":"6028613c-ebfd-42ea-9949-3f0d5f6cf025","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6826","nonce":2001303253},{"type":"v1","addr":"172.21.15.39:6827","nonce":2001303253}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6828","nonce":2001303253},{"type":"v1","addr":"172.21.15.39:6829","nonce":2001303253}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6832","nonce":2001303253},{"type":"v1","addr":"172.21.15.39:6833","nonce":2001303253}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6830","nonce":2001303253},{"type":"v1","addr":"172.21.15.39:6831","nonce":2001303253}]},"public_addr":"172.21.15.39:6827/2001303253","cluster_addr":"172.21.15.39:6829/2001303253","heartbeat_back_addr":"172.21.15.39:6833/2001303253","heartbeat_front_addr":"172.21.15.39:6831/2001303253","state":["exists","up"]},{"osd":10,"uuid":"1c71c765-2a6c-4ffe-9fe1-c8adae3d1c32","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6824","nonce":2484627938},{"type":"v1","addr":"172.21.15.150:6825","nonce":2484627938}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6826","nonce":2484627938},{"type":"v1","addr":"172.21.15.150:6827","nonce":2484627938}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6830","nonce":2484627938},{"type":"v1","addr":"172.21.15.150:6831","nonce":2484627938}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6828","nonce":2484627938},{"type":"v1","addr":"172.21.15.150:6829","nonce":2484627938}]},"public_addr":"172.21.15.150:6825/2484627938","cluster_addr":"172.21.15.150:6827/2484627938","heartbeat_back_addr":"172.21.15.150:6831/2484627938","heartbeat_front_addr":"172.21.15.150:6829/2484627938","state":["exists","up"]},{"osd":11,"uuid":"f382f522-62db-46da-997e-2b28f84d1612","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6824","nonce":2178709707},{"type":"v1","addr":"172.21.15.161:6825","nonce":2178709707}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6826","nonce":2178709707},{"type":"v1","addr":"172.21.15.161:6827","nonce":2178709707}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6830","nonce":2178709707},{"type":"v1","addr":"172.21.15.161:6831","nonce":2178709707}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6828","nonce":2178709707},{"type":"v1","addr":"172.21.15.161:6829","nonce":2178709707}]},"public_addr":"172.21.15.161:6825/2178709707","cluster_addr":"172.21.15.161:6827/2178709707","heartbeat_back_addr":"172.21.15.161:6831/2178709707","heartbeat_front_addr":"172.21.15.161:6829/2178709707","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:38:35.540904+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:39:01.455053+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:39:27.052867+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:39:55.173529+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:40:21.298396+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:40:47.131235+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:41:14.475209+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:41:39.920036+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:42:05.587065+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:42:32.624726+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:42:57.479185+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:43:22.203832+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.5","mappings":[{"from":8,"to":11}]},{"pgid":"2.8","mappings":[{"from":2,"to":11}]},{"pgid":"2.c","mappings":[{"from":2,"to":1}]},{"pgid":"2.1f","mappings":[{"from":8,"to":4}]},{"pgid":"3.2","mappings":[{"from":3,"to":9}]},{"pgid":"3.14","mappings":[{"from":4,"to":6}]},{"pgid":"3.1a","mappings":[{"from":5,"to":7}]},{"pgid":"3.1d","mappings":[{"from":4,"to":8}]},{"pgid":"3.2c","mappings":[{"from":4,"to":9}]},{"pgid":"3.38","mappings":[{"from":3,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.39:6801/2438110611":"2024-09-22T03:37:25.391457+0000","172.21.15.39:6800/2438110611":"2024-09-22T03:37:25.391457+0000","172.21.15.39:6800/173264695":"2024-09-22T03:35:57.104467+0000","172.21.15.39:6801/173264695":"2024-09-22T03:35:57.104467+0000","172.21.15.39:0/2637182284":"2024-09-22T03:35:57.104467+0000","172.21.15.39:0/1517304561":"2024-09-22T03:36:24.112592+0000","172.21.15.39:6801/1435622199":"2024-09-22T03:36:24.112592+0000","172.21.15.39:6800/1435622199":"2024-09-22T03:36:24.112592+0000","172.21.15.39:0/346832469":"2024-09-22T03:35:57.104467+0000","172.21.15.39:0/3160777371":"2024-09-22T03:36:24.112592+0000","172.21.15.39:0/1730342855":"2024-09-22T03:35:57.104467+0000","172.21.15.39:0/1955726766":"2024-09-22T03:36:24.112592+0000","172.21.15.39:0/3294132796":"2024-09-22T03:37:25.391457+0000","172.21.15.39:0/1295810162":"2024-09-22T03:37:25.391457+0000","172.21.15.39:0/3729230474":"2024-09-22T03:37:25.391457+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T03:45:55.608 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:55.604+0000 7f39c0291640 1 -- 172.21.15.39:0/73162835 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f39a40b65f0 msgr2=0x7f39a40b8b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:55.608 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:55.604+0000 7f39c0291640 1 --2- 172.21.15.39:0/73162835 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f39a40b65f0 0x7f39a40b8b00 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f39b40058c0 tx=0x7f39b400a2d0 comp rx=0 tx=0).stop 2024-09-21T03:45:55.608 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:55.604+0000 7f39c0291640 1 -- 172.21.15.39:0/73162835 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f39b810de10 msgr2=0x7f39b819fd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:55.608 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:55.604+0000 7f39c0291640 1 --2- 172.21.15.39:0/73162835 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f39b810de10 0x7f39b819fd40 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f39b00098d0 tx=0x7f39b00317b0 comp rx=0 tx=0).stop 2024-09-21T03:45:55.608 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:55.605+0000 7f39c0291640 1 -- 172.21.15.39:0/73162835 shutdown_connections 2024-09-21T03:45:55.608 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:55.605+0000 7f39c0291640 1 --2- 172.21.15.39:0/73162835 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f39a40b65f0 0x7f39a40b8b00 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:55.608 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:55.605+0000 7f39c0291640 1 --2- 172.21.15.39:0/73162835 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f39b810de10 0x7f39b819fd40 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:55.608 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:55.605+0000 7f39c0291640 1 --2- 172.21.15.39:0/73162835 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f39b8105300 0x7f39b819aaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:55.608 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:55.605+0000 7f39c0291640 1 --2- 172.21.15.39:0/73162835 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f39b81049b0 0x7f39b819a500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:55.609 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:55.605+0000 7f39c0291640 1 -- 172.21.15.39:0/73162835 >> 172.21.15.39:0/73162835 conn(0x7f39b8079730 msgr2=0x7f39b8100730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:45:55.609 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:55.605+0000 7f39c0291640 1 -- 172.21.15.39:0/73162835 shutdown_connections 2024-09-21T03:45:55.609 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:55.605+0000 7f39c0291640 1 -- 172.21.15.39:0/73162835 wait complete. 2024-09-21T03:45:56.217 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:45:55 smithi161 ceph-mon[34533]: from='client.? 172.21.15.39:0/73162835' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T03:45:56.224 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:45:55 smithi150 ceph-mon[34289]: from='client.? 172.21.15.39:0/73162835' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T03:45:56.302 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:45:55 smithi039 ceph-mon[31211]: from='client.? 172.21.15.39:0/73162835' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T03:45:56.333 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 28, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-09-21T03:44:15.009872+0000', 'modified': '2024-09-21T03:44:26.017327+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 5, 'in': [0, 1, 2, 3, 4], 'up': {'mds_0': 24695, 'mds_1': 24667, 'mds_2': 24659, 'mds_3': 24646, 'mds_4': 14868}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_14856': {'gid': 14856, 'name': 'g', 'rank': 4, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.39:6839/185509192', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.39:6838', 'nonce': 185509192}, {'type': 'v1', 'addr': '172.21.15.39:6839', 'nonce': 185509192}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_14868': {'gid': 14868, 'name': 'a', 'rank': 4, 'incarnation': 25, 'state': 'up:active', 'state_seq': 7, 'addr': '172.21.15.39:6841/4247190456', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.39:6840', 'nonce': 4247190456}, {'type': 'v1', 'addr': '172.21.15.39:6841', 'nonce': 4247190456}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24599': {'gid': 24599, 'name': 'k', 'rank': 3, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.150:6833/3114643217', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.150:6832', 'nonce': 3114643217}, {'type': 'v1', 'addr': '172.21.15.150:6833', 'nonce': 3114643217}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24601': {'gid': 24601, 'name': 'l', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.161:6833/2963990654', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.161:6832', 'nonce': 2963990654}, {'type': 'v1', 'addr': '172.21.15.161:6833', 'nonce': 2963990654}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24632': {'gid': 24632, 'name': 'e', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.150:6835/724594008', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.150:6834', 'nonce': 724594008}, {'type': 'v1', 'addr': '172.21.15.150:6835', 'nonce': 724594008}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24640': {'gid': 24640, 'name': 'f', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.161:6835/3747882110', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.161:6834', 'nonce': 3747882110}, {'type': 'v1', 'addr': '172.21.15.161:6835', 'nonce': 3747882110}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24646': {'gid': 24646, 'name': 'c', 'rank': 3, 'incarnation': 23, 'state': 'up:active', 'state_seq': 10, 'addr': '172.21.15.161:6837/3451544597', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.161:6836', 'nonce': 3451544597}, {'type': 'v1', 'addr': '172.21.15.161:6837', 'nonce': 3451544597}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24659': {'gid': 24659, 'name': 'h', 'rank': 2, 'incarnation': 21, 'state': 'up:active', 'state_seq': 9, 'addr': '172.21.15.150:6837/4017043144', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.150:6836', 'nonce': 4017043144}, {'type': 'v1', 'addr': '172.21.15.150:6837', 'nonce': 4017043144}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24667': {'gid': 24667, 'name': 'i', 'rank': 1, 'incarnation': 19, 'state': 'up:active', 'state_seq': 8, 'addr': '172.21.15.161:6839/2190107270', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.161:6838', 'nonce': 2190107270}, {'type': 'v1', 'addr': '172.21.15.161:6839', 'nonce': 2190107270}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24695': {'gid': 24695, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.150:6839/2949386907', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.150:6838', 'nonce': 2949386907}, {'type': 'v1', 'addr': '172.21.15.150:6839', 'nonce': 2949386907}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24695, 'qdb_cluster': [14868, 24646, 24659, 24667, 24695]} 2024-09-21T03:45:56.333 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 5/5 2024-09-21T03:45:56.334 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 51116d5c-77ca-11ef-baf5-efdc52797490 -- ceph tell mds.a status 2024-09-21T03:45:56.592 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/51116d5c-77ca-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:45:56.989 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:45:56 smithi039 ceph-mon[31211]: pgmap v322: 97 pgs: 97 active+clean; 647 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 12 op/s 2024-09-21T03:45:57.083 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:57.080+0000 7f8e809dd640 1 -- 172.21.15.39:0/915438675 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8e7c1094a0 msgr2=0x7f8e7c1098b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:57.083 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:57.080+0000 7f8e809dd640 1 --2- 172.21.15.39:0/915438675 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8e7c1094a0 0x7f8e7c1098b0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f8e64009900 tx=0x7f8e6402a3f0 comp rx=0 tx=0).stop 2024-09-21T03:45:57.083 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:57.080+0000 7f8e809dd640 1 -- 172.21.15.39:0/915438675 shutdown_connections 2024-09-21T03:45:57.083 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:57.080+0000 7f8e809dd640 1 --2- 172.21.15.39:0/915438675 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8e7c109df0 0x7f8e7c110620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:57.084 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:57.080+0000 7f8e809dd640 1 --2- 172.21.15.39:0/915438675 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8e7c1094a0 0x7f8e7c1098b0 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:57.084 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:57.080+0000 7f8e809dd640 1 --2- 172.21.15.39:0/915438675 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8e7c103b60 0x7f8e7c103f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:57.084 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:57.080+0000 7f8e809dd640 1 -- 172.21.15.39:0/915438675 >> 172.21.15.39:0/915438675 conn(0x7f8e7c07b850 msgr2=0x7f8e7c07bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:45:57.084 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:57.080+0000 7f8e809dd640 1 -- 172.21.15.39:0/915438675 shutdown_connections 2024-09-21T03:45:57.084 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:57.080+0000 7f8e809dd640 1 -- 172.21.15.39:0/915438675 wait complete. 2024-09-21T03:45:57.085 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:57.081+0000 7f8e809dd640 1 Processor -- start 2024-09-21T03:45:57.085 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:57.081+0000 7f8e809dd640 1 -- start start 2024-09-21T03:45:57.086 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:57.082+0000 7f8e809dd640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8e7c103b60 0x7f8e7c19a500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:57.086 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:57.082+0000 7f8e809dd640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8e7c1094a0 0x7f8e7c19aaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:57.086 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:57.082+0000 7f8e809dd640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8e7c109df0 0x7f8e7c19fd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:57.086 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:57.082+0000 7f8e809dd640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f8e7c113ad0 con 0x7f8e7c109df0 2024-09-21T03:45:57.086 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:57.082+0000 7f8e809dd640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f8e7c113950 con 0x7f8e7c1094a0 2024-09-21T03:45:57.086 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:57.082+0000 7f8e809dd640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f8e7c113c50 con 0x7f8e7c103b60 2024-09-21T03:45:57.086 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:57.082+0000 7f8e7ad9d640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8e7c109df0 0x7f8e7c19fd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:57.086 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:57.082+0000 7f8e7ad9d640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8e7c109df0 0x7f8e7c19fd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:35576/0 (socket says 172.21.15.39:35576) 2024-09-21T03:45:57.086 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:57.082+0000 7f8e7ad9d640 1 -- 172.21.15.39:0/3139532328 learned_addr learned my addr 172.21.15.39:0/3139532328 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:45:57.086 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:57.082+0000 7f8e7a59c640 1 --2- 172.21.15.39:0/3139532328 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8e7c103b60 0x7f8e7c19a500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:57.086 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:57.082+0000 7f8e73fff640 1 --2- 172.21.15.39:0/3139532328 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8e7c1094a0 0x7f8e7c19aaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:57.086 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:57.082+0000 7f8e7ad9d640 1 -- 172.21.15.39:0/3139532328 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8e7c103b60 msgr2=0x7f8e7c19a500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:57.086 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:57.082+0000 7f8e7ad9d640 1 --2- 172.21.15.39:0/3139532328 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8e7c103b60 0x7f8e7c19a500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:57.087 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:57.082+0000 7f8e7ad9d640 1 -- 172.21.15.39:0/3139532328 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8e7c1094a0 msgr2=0x7f8e7c19aaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:57.087 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:57.082+0000 7f8e7ad9d640 1 --2- 172.21.15.39:0/3139532328 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8e7c1094a0 0x7f8e7c19aaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:57.087 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:57.082+0000 7f8e7ad9d640 1 -- 172.21.15.39:0/3139532328 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8e7c1a0470 con 0x7f8e7c109df0 2024-09-21T03:45:57.087 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:57.082+0000 7f8e7ad9d640 1 --2- 172.21.15.39:0/3139532328 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8e7c109df0 0x7f8e7c19fd40 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f8e7400b970 tx=0x7f8e7400be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:45:57.087 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:57.083+0000 7f8e737fe640 1 -- 172.21.15.39:0/3139532328 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e74019070 con 0x7f8e7c109df0 2024-09-21T03:45:57.087 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:57.083+0000 7f8e809dd640 1 -- 172.21.15.39:0/3139532328 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8e7c1a0760 con 0x7f8e7c109df0 2024-09-21T03:45:57.087 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:57.083+0000 7f8e737fe640 1 -- 172.21.15.39:0/3139532328 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8e74014440 con 0x7f8e7c109df0 2024-09-21T03:45:57.087 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:57.083+0000 7f8e737fe640 1 -- 172.21.15.39:0/3139532328 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e740075d0 con 0x7f8e7c109df0 2024-09-21T03:45:57.087 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:57.083+0000 7f8e809dd640 1 -- 172.21.15.39:0/3139532328 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8e7c1a6ec0 con 0x7f8e7c109df0 2024-09-21T03:45:57.088 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:57.085+0000 7f8e86f51740 1 Processor -- start 2024-09-21T03:45:57.088 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:57.085+0000 7f8e737fe640 1 -- 172.21.15.39:0/3139532328 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 22) ==== 154580+0+0 (secure 0 0 0) 0x7f8e7401c070 con 0x7f8e7c109df0 2024-09-21T03:45:57.090 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:57.086+0000 7f8e86f51740 1 -- start start 2024-09-21T03:45:57.090 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:57.086+0000 7f8e86f51740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e656e24f60 0x55e6571caf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:57.090 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:57.086+0000 7f8e86f51740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55e656dfc0a0 0x55e6571cb440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:57.090 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:57.086+0000 7f8e86f51740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e656e04280 0x55e6571cb980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:57.090 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:57.086+0000 7f8e86f51740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x55e657142c10 con 0x55e656dfc0a0 2024-09-21T03:45:57.090 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:57.086+0000 7f8e86f51740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x55e657132040 con 0x55e656e04280 2024-09-21T03:45:57.090 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:57.086+0000 7f8e86f51740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55e657135e50 con 0x55e656e24f60 2024-09-21T03:45:57.090 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:57.086+0000 7f8e7a59c640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55e656dfc0a0 0x55e6571cb440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:57.090 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:57.086+0000 7f8e7a59c640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55e656dfc0a0 0x55e6571cb440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:35588/0 (socket says 172.21.15.39:35588) 2024-09-21T03:45:57.090 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:57.086+0000 7f8e7a59c640 1 -- 172.21.15.39:0/3996382848 learned_addr learned my addr 172.21.15.39:0/3996382848 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:45:57.091 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:57.086+0000 7f8e73fff640 1 --2- 172.21.15.39:0/3996382848 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e656e24f60 0x55e6571caf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:57.091 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:57.086+0000 7f8e73fff640 1 --2- 172.21.15.39:0/3996382848 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e656e04280 0x55e6571cb980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:57.091 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:57.087+0000 7f8e7a59c640 1 -- 172.21.15.39:0/3996382848 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e656e24f60 msgr2=0x55e6571caf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:45:57.091 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:57.087+0000 7f8e7a59c640 1 --2- 172.21.15.39:0/3996382848 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e656e24f60 0x55e6571caf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:57.091 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:57.087+0000 7f8e7a59c640 1 -- 172.21.15.39:0/3996382848 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e656e04280 msgr2=0x55e6571cb980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:45:57.091 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:57.087+0000 7f8e7a59c640 1 --2- 172.21.15.39:0/3996382848 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e656e04280 0x55e6571cb980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:57.091 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:57.087+0000 7f8e7a59c640 1 -- 172.21.15.39:0/3996382848 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e656e90540 con 0x55e656dfc0a0 2024-09-21T03:45:57.091 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:57.087+0000 7f8e73fff640 1 --2- 172.21.15.39:0/3996382848 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e656e04280 0x55e6571cb980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:45:57.091 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:57.087+0000 7f8e737fe640 1 --2- 172.21.15.39:0/3139532328 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f8e680b65f0 0x7f8e680b8b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:57.091 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:57.087+0000 7f8e7a59c640 1 --2- 172.21.15.39:0/3996382848 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55e656dfc0a0 0x55e6571cb440 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f8e6c009cf0 tx=0x7f8e6c011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2604197f963c47a6 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:45:57.091 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:57.087+0000 7f8e73fff640 1 --2- 172.21.15.39:0/3139532328 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f8e680b65f0 0x7f8e680b8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:57.091 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:57.087+0000 7f8e737fe640 1 -- 172.21.15.39:0/3139532328 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8484+0+0 (secure 0 0 0) 0x7f8e740e6ab0 con 0x7f8e7c109df0 2024-09-21T03:45:57.091 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:57.087+0000 7f8e70ff9640 1 -- 172.21.15.39:0/3996382848 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e6c01e070 con 0x55e656dfc0a0 2024-09-21T03:45:57.091 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:57.087+0000 7f8e70ff9640 1 -- 172.21.15.39:0/3996382848 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8e6c019ae0 con 0x55e656dfc0a0 2024-09-21T03:45:57.092 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:57.087+0000 7f8e70ff9640 1 -- 172.21.15.39:0/3996382848 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e6c00e910 con 0x55e656dfc0a0 2024-09-21T03:45:57.095 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:57.088+0000 7f8e86f51740 1 -- 172.21.15.39:0/3996382848 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55e656dfc0a0 msgr2=0x55e6571cb440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:57.095 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:57.088+0000 7f8e86f51740 1 --2- 172.21.15.39:0/3996382848 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55e656dfc0a0 0x55e6571cb440 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f8e6c009cf0 tx=0x7f8e6c011810 comp rx=0 tx=0).stop 2024-09-21T03:45:57.095 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:57.088+0000 7f8e73fff640 1 --2- 172.21.15.39:0/3139532328 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f8e680b65f0 0x7f8e680b8b00 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f8e64009230 tx=0x7f8e6402a3f0 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:45:57.095 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:57.088+0000 7f8e86f51740 1 -- 172.21.15.39:0/3996382848 shutdown_connections 2024-09-21T03:45:57.095 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:57.088+0000 7f8e86f51740 1 --2- 172.21.15.39:0/3996382848 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e656e04280 0x55e6571cb980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:57.095 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:57.088+0000 7f8e86f51740 1 --2- 172.21.15.39:0/3996382848 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55e656dfc0a0 0x55e6571cb440 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:57.095 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:57.088+0000 7f8e86f51740 1 --2- 172.21.15.39:0/3996382848 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e656e24f60 0x55e6571caf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:57.095 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:57.088+0000 7f8e86f51740 1 -- 172.21.15.39:0/3996382848 >> 172.21.15.39:0/3996382848 conn(0x55e656e19350 msgr2=0x55e6571c8df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:45:57.095 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:57.088+0000 7f8e86f51740 1 -- 172.21.15.39:0/3996382848 shutdown_connections 2024-09-21T03:45:57.096 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:57.088+0000 7f8e86f51740 1 -- 172.21.15.39:0/3996382848 wait complete. 2024-09-21T03:45:57.096 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:57.089+0000 7f8e86f51740 1 -- start start 2024-09-21T03:45:57.096 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:57.089+0000 7f8e86f51740 1 Processor -- start 2024-09-21T03:45:57.096 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:57.089+0000 7f8e86f51740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e656dfc0a0 0x55e6571d67d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:57.096 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:57.089+0000 7f8e86f51740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e656e04280 0x55e6571d6db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:57.096 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:57.089+0000 7f8e86f51740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55e656e24f60 0x55e6571e2a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:57.096 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:57.089+0000 7f8e86f51740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x55e656f935f0 con 0x55e656e24f60 2024-09-21T03:45:57.096 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:57.089+0000 7f8e86f51740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x55e6570cd010 con 0x55e656dfc0a0 2024-09-21T03:45:57.096 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:57.089+0000 7f8e86f51740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55e6571d96e0 con 0x55e656e04280 2024-09-21T03:45:57.096 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:57.089+0000 7f8e86f51740 10 client.0 _ll_register_callbacks cb 0x55e65712c340 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:45:57.096 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:57.089+0000 7f8e7ad9d640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55e656e24f60 0x55e6571e2a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:57.096 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:57.089+0000 7f8e7ad9d640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55e656e24f60 0x55e6571e2a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:35598/0 (socket says 172.21.15.39:35598) 2024-09-21T03:45:57.096 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:57.089+0000 7f8e7ad9d640 1 -- 172.21.15.39:0/977799163 learned_addr learned my addr 172.21.15.39:0/977799163 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:45:57.096 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:57.089+0000 7f8e7a59c640 1 --2- 172.21.15.39:0/977799163 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e656dfc0a0 0x55e6571d67d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:57.097 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:57.089+0000 7f8e73fff640 1 --2- 172.21.15.39:0/977799163 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e656e04280 0x55e6571d6db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:57.097 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:57.090+0000 7f8e7ad9d640 1 -- 172.21.15.39:0/977799163 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e656e04280 msgr2=0x55e6571d6db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:45:57.097 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:57.090+0000 7f8e7ad9d640 1 --2- 172.21.15.39:0/977799163 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e656e04280 0x55e6571d6db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:57.097 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:57.090+0000 7f8e7ad9d640 1 -- 172.21.15.39:0/977799163 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e656dfc0a0 msgr2=0x55e6571d67d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:45:57.097 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:57.090+0000 7f8e7ad9d640 1 --2- 172.21.15.39:0/977799163 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e656dfc0a0 0x55e6571d67d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:57.097 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:57.090+0000 7f8e7ad9d640 1 -- 172.21.15.39:0/977799163 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e6570faa00 con 0x55e656e24f60 2024-09-21T03:45:57.097 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:57.090+0000 7f8e73fff640 1 --2- 172.21.15.39:0/977799163 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e656e04280 0x55e6571d6db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:45:57.097 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:57.090+0000 7f8e7ad9d640 1 --2- 172.21.15.39:0/977799163 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55e656e24f60 0x55e6571e2a00 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f8e740a9060 tx=0x7f8e740e47e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c7496945ac1a5b7f server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:45:57.097 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:57.090+0000 7f8e61ffb640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-09-21T03:45:57.097 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:57.090+0000 7f8e61ffb640 1 -- 172.21.15.39:0/977799163 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e7400cd40 con 0x55e656e24f60 2024-09-21T03:45:57.097 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:57.090+0000 7f8e61ffb640 1 -- 172.21.15.39:0/977799163 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8484+0+0 (secure 0 0 0) 0x7f8e740abcf0 con 0x55e656e24f60 2024-09-21T03:45:57.097 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:57.090+0000 7f8e86f51740 1 -- 172.21.15.39:0/977799163 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8e6c019ae0 con 0x55e656e24f60 2024-09-21T03:45:57.097 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:57.090+0000 7f8e61ffb640 1 -- 172.21.15.39:0/977799163 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f8e740b14c0 con 0x55e656e24f60 2024-09-21T03:45:57.097 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:57.090+0000 7f8e86f51740 10 client.15384 fetch_fsmap learned FSMap version 28 2024-09-21T03:45:57.097 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:57.090+0000 7f8e86f51740 1 -- 172.21.15.39:0/977799163 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55e6571e55d0 con 0x55e656e24f60 2024-09-21T03:45:57.098 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:57.090+0000 7f8e61ffb640 1 -- 172.21.15.39:0/977799163 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8e6c019ae0 con 0x55e656e24f60 2024-09-21T03:45:57.098 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:57.091+0000 7f8e61ffb640 1 -- 172.21.15.39:0/977799163 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f8e740ab760 con 0x55e656e24f60 2024-09-21T03:45:57.098 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:57.091+0000 7f8e86f51740 10 client.15384 fetch_fsmap finished waiting for FSMap version 28 2024-09-21T03:45:57.098 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:57.091+0000 7f8e86f51740 10 client.15384 resolve_mds: resolved name 'a' to daemon mds.a 2024-09-21T03:45:57.098 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:57.091+0000 7f8e86f51740 20 client.15384 populate_metadata read hostname 'smithi039' 2024-09-21T03:45:57.098 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:57.091+0000 7f8e86f51740 1 --2- 172.21.15.39:0/977799163 >> [v2:172.21.15.39:6840/4247190456,v1:172.21.15.39:6841/4247190456] conn(0x55e656de38e0 0x55e6571e9010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:57.098 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:57.091+0000 7f8e86f51740 4 client.15384 mds_command: new command op to 14868 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:45:57.098 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:57.091+0000 7f8e86f51740 1 -- 172.21.15.39:0/977799163 --> [v2:172.21.15.39:6840/4247190456,v1:172.21.15.39:6841/4247190456] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e657144b60 con 0x55e656de38e0 2024-09-21T03:45:57.098 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:57.091+0000 7f8e7a59c640 1 --2- 172.21.15.39:0/977799163 >> [v2:172.21.15.39:6840/4247190456,v1:172.21.15.39:6841/4247190456] conn(0x55e656de38e0 0x55e6571e9010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:57.098 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:57.091+0000 7f8e7a59c640 1 --2- 172.21.15.39:0/977799163 >> [v2:172.21.15.39:6840/4247190456,v1:172.21.15.39:6841/4247190456] conn(0x55e656de38e0 0x55e6571e9010 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=60414820bac925df server_cookie=babf685e6dbf5390 in_seq=0 out_seq=0 2024-09-21T03:45:57.098 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:57.091+0000 7f8e61ffb640 10 client.15384 ms_handle_connect on v2:172.21.15.39:6840/4247190456 2024-09-21T03:45:57.098 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:57.092+0000 7f8e61ffb640 1 -- 172.21.15.39:0/977799163 <== mds.4 v2:172.21.15.39:6840/4247190456 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55e657144b60 con 0x55e656de38e0 2024-09-21T03:45:57.098 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:57.092+0000 7f8e61ffb640 10 client.15384 handle_command_reply: tid=0 2024-09-21T03:45:57.099 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:57.092+0000 7f8e86f51740 1 client.15384 shutdown 2024-09-21T03:45:57.099 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:57.092+0000 7f8e86f51740 1 -- 172.21.15.39:0/977799163 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55e656e24f60 msgr2=0x55e6571e2a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:57.099 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:57.092+0000 7f8e86f51740 1 --2- 172.21.15.39:0/977799163 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55e656e24f60 0x55e6571e2a00 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f8e740a9060 tx=0x7f8e740e47e0 comp rx=0 tx=0).stop 2024-09-21T03:45:57.099 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:57.092+0000 7f8e86f51740 1 -- 172.21.15.39:0/977799163 shutdown_connections 2024-09-21T03:45:57.099 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:57.092+0000 7f8e86f51740 1 --2- 172.21.15.39:0/977799163 >> [v2:172.21.15.39:6840/4247190456,v1:172.21.15.39:6841/4247190456] conn(0x55e656de38e0 0x55e6571e9010 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:57.099 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:57.092+0000 7f8e86f51740 1 --2- 172.21.15.39:0/977799163 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55e656e24f60 0x55e6571e2a00 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:57.099 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:57.092+0000 7f8e86f51740 1 --2- 172.21.15.39:0/977799163 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e656e04280 0x55e6571d6db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:57.099 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:57.092+0000 7f8e86f51740 1 --2- 172.21.15.39:0/977799163 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e656dfc0a0 0x55e6571d67d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:57.099 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:57.092+0000 7f8e86f51740 1 -- 172.21.15.39:0/977799163 >> 172.21.15.39:0/977799163 conn(0x55e656e19350 msgr2=0x55e6571ce090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:45:57.099 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:57.092+0000 7f8e61ffb640 0 client.15384 ms_handle_reset on v2:172.21.15.39:6840/4247190456 2024-09-21T03:45:57.099 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:57.092+0000 7f8e86f51740 1 -- 172.21.15.39:0/977799163 shutdown_connections 2024-09-21T03:45:57.099 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:57.092+0000 7f8e86f51740 1 -- 172.21.15.39:0/977799163 wait complete. 2024-09-21T03:45:57.099 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:57.092+0000 7f8e86f51740 20 client.15384 trim_cache size 0 max 16384 2024-09-21T03:45:57.113 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:57.110+0000 7f8e86f51740 1 Processor -- start 2024-09-21T03:45:57.113 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:57.110+0000 7f8e86f51740 1 -- start start 2024-09-21T03:45:57.114 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:57.111+0000 7f8e86f51740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e656dfc0a0 0x55e6571d1020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:57.114 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:57.111+0000 7f8e86f51740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e656e04280 0x55e6571c3ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:57.114 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:57.111+0000 7f8e86f51740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55e656e24f60 0x55e6571e2970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:57.114 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:57.111+0000 7f8e86f51740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x55e6571e7350 con 0x55e656e24f60 2024-09-21T03:45:57.114 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:57.111+0000 7f8e86f51740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x55e6571e7a20 con 0x55e656dfc0a0 2024-09-21T03:45:57.114 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:57.111+0000 7f8e86f51740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55e6571e80f0 con 0x55e656e04280 2024-09-21T03:45:57.114 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:57.111+0000 7f8e7ad9d640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55e656e24f60 0x55e6571e2970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:57.114 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:57.111+0000 7f8e7ad9d640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55e656e24f60 0x55e6571e2970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:35606/0 (socket says 172.21.15.39:35606) 2024-09-21T03:45:57.114 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:57.111+0000 7f8e7ad9d640 1 -- 172.21.15.39:0/1670269029 learned_addr learned my addr 172.21.15.39:0/1670269029 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:45:57.118 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:57.111+0000 7f8e73fff640 1 --2- 172.21.15.39:0/1670269029 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e656e04280 0x55e6571c3ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:57.118 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:57.111+0000 7f8e7a59c640 1 --2- 172.21.15.39:0/1670269029 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e656dfc0a0 0x55e6571d1020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:57.118 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:57.111+0000 7f8e7ad9d640 1 -- 172.21.15.39:0/1670269029 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e656e04280 msgr2=0x55e6571c3ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:45:57.118 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:57.111+0000 7f8e7ad9d640 1 --2- 172.21.15.39:0/1670269029 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e656e04280 0x55e6571c3ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:57.118 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:57.111+0000 7f8e7ad9d640 1 -- 172.21.15.39:0/1670269029 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e656dfc0a0 msgr2=0x55e6571d1020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:45:57.118 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:57.111+0000 7f8e7ad9d640 1 --2- 172.21.15.39:0/1670269029 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e656dfc0a0 0x55e6571d1020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:57.118 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:57.111+0000 7f8e7ad9d640 1 -- 172.21.15.39:0/1670269029 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e6571cd790 con 0x55e656e24f60 2024-09-21T03:45:57.118 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:57.111+0000 7f8e7a59c640 1 --2- 172.21.15.39:0/1670269029 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e656dfc0a0 0x55e6571d1020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:45:57.118 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:57.111+0000 7f8e7ad9d640 1 --2- 172.21.15.39:0/1670269029 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55e656e24f60 0x55e6571e2970 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f8e740ac790 tx=0x7f8e74004b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e656e4c70c3caea server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:45:57.118 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:57.112+0000 7f8e63fff640 1 -- 172.21.15.39:0/1670269029 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e740b2650 con 0x55e656e24f60 2024-09-21T03:45:57.118 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:57.112+0000 7f8e63fff640 1 -- 172.21.15.39:0/1670269029 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8e740b6400 con 0x55e656e24f60 2024-09-21T03:45:57.118 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:57.112+0000 7f8e63fff640 1 -- 172.21.15.39:0/1670269029 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e740ac3e0 con 0x55e656e24f60 2024-09-21T03:45:57.119 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:57.112+0000 7f8e86f51740 1 -- 172.21.15.39:0/1670269029 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55e656e24f60 msgr2=0x55e6571e2970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:57.119 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:57.112+0000 7f8e86f51740 1 --2- 172.21.15.39:0/1670269029 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55e656e24f60 0x55e6571e2970 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f8e740ac790 tx=0x7f8e74004b50 comp rx=0 tx=0).stop 2024-09-21T03:45:57.119 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:57.113+0000 7f8e86f51740 1 -- 172.21.15.39:0/1670269029 shutdown_connections 2024-09-21T03:45:57.119 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:57.113+0000 7f8e86f51740 1 --2- 172.21.15.39:0/1670269029 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55e656e24f60 0x55e6571e2970 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:57.119 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:57.113+0000 7f8e86f51740 1 --2- 172.21.15.39:0/1670269029 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e656e04280 0x55e6571c3ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:57.119 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:57.113+0000 7f8e86f51740 1 --2- 172.21.15.39:0/1670269029 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e656dfc0a0 0x55e6571d1020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:57.119 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:57.113+0000 7f8e86f51740 1 -- 172.21.15.39:0/1670269029 >> 172.21.15.39:0/1670269029 conn(0x55e656e19350 msgr2=0x55e6571d6740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:45:57.119 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:57.113+0000 7f8e86f51740 1 -- 172.21.15.39:0/1670269029 shutdown_connections 2024-09-21T03:45:57.119 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:57.113+0000 7f8e86f51740 1 -- 172.21.15.39:0/1670269029 wait complete. 2024-09-21T03:45:57.119 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:57.113+0000 7f8e86f51740 1 -- start start 2024-09-21T03:45:57.119 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:57.113+0000 7f8e86f51740 1 Processor -- start 2024-09-21T03:45:57.119 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:57.113+0000 7f8e86f51740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55e656dfc0a0 0x55e6571eb980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:57.119 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:57.113+0000 7f8e86f51740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e656e04280 0x55e6571ebec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:57.119 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:57.113+0000 7f8e86f51740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e656e24f60 0x55e6571f0f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:57.119 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:57.113+0000 7f8e86f51740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x55e6571e8e90 con 0x55e656dfc0a0 2024-09-21T03:45:57.120 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:57.113+0000 7f8e86f51740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x55e6571e87c0 con 0x55e656e24f60 2024-09-21T03:45:57.120 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:57.113+0000 7f8e86f51740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55e6571e9560 con 0x55e656e04280 2024-09-21T03:45:57.120 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:57.113+0000 7f8e86f51740 10 client.0 _ll_register_callbacks cb 0x7f8e74014c00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:45:57.120 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:57.114+0000 7f8e7a59c640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55e656dfc0a0 0x55e6571eb980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:57.120 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:57.114+0000 7f8e7a59c640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55e656dfc0a0 0x55e6571eb980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:35622/0 (socket says 172.21.15.39:35622) 2024-09-21T03:45:57.120 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:57.114+0000 7f8e7a59c640 1 -- 172.21.15.39:0/593346231 learned_addr learned my addr 172.21.15.39:0/593346231 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:45:57.120 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:57.114+0000 7f8e73fff640 1 --2- 172.21.15.39:0/593346231 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e656e04280 0x55e6571ebec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:57.120 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:57.114+0000 7f8e7a59c640 1 -- 172.21.15.39:0/593346231 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e656e04280 msgr2=0x55e6571ebec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:45:57.120 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:57.114+0000 7f8e7a59c640 1 --2- 172.21.15.39:0/593346231 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e656e04280 0x55e6571ebec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:57.120 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:57.114+0000 7f8e7a59c640 1 -- 172.21.15.39:0/593346231 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e656e24f60 msgr2=0x55e6571f0f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:45:57.120 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:57.114+0000 7f8e7a59c640 1 --2- 172.21.15.39:0/593346231 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e656e24f60 0x55e6571f0f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:57.120 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:57.114+0000 7f8e7a59c640 1 -- 172.21.15.39:0/593346231 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e6571f1450 con 0x55e656dfc0a0 2024-09-21T03:45:57.120 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:57.114+0000 7f8e7a59c640 1 --2- 172.21.15.39:0/593346231 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55e656dfc0a0 0x55e6571eb980 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f8e6c009250 tx=0x7f8e6c009c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a45398afc74ea6fa server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:45:57.120 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:57.114+0000 7f8e61ffb640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-09-21T03:45:57.121 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:57.114+0000 7f8e61ffb640 1 -- 172.21.15.39:0/593346231 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e6c032060 con 0x55e656dfc0a0 2024-09-21T03:45:57.121 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:57.114+0000 7f8e86f51740 1 -- 172.21.15.39:0/593346231 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8e740b6400 con 0x55e656dfc0a0 2024-09-21T03:45:57.121 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:57.114+0000 7f8e61ffb640 1 -- 172.21.15.39:0/593346231 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8484+0+0 (secure 0 0 0) 0x7f8e6c005310 con 0x55e656dfc0a0 2024-09-21T03:45:57.121 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:57.114+0000 7f8e61ffb640 1 -- 172.21.15.39:0/593346231 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f8e6c01e040 con 0x55e656dfc0a0 2024-09-21T03:45:57.121 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:57.114+0000 7f8e86f51740 10 client.15390 fetch_fsmap learned FSMap version 28 2024-09-21T03:45:57.121 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:57.114+0000 7f8e86f51740 1 -- 172.21.15.39:0/593346231 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55e657254c60 con 0x55e656dfc0a0 2024-09-21T03:45:57.121 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:57.115+0000 7f8e61ffb640 1 -- 172.21.15.39:0/593346231 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8e740b6400 con 0x55e656dfc0a0 2024-09-21T03:45:57.121 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:57.115+0000 7f8e61ffb640 1 -- 172.21.15.39:0/593346231 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f8e6c0035f0 con 0x55e656dfc0a0 2024-09-21T03:45:57.121 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:57.115+0000 7f8e86f51740 10 client.15390 fetch_fsmap finished waiting for FSMap version 28 2024-09-21T03:45:57.121 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:57.115+0000 7f8e86f51740 10 client.15390 resolve_mds: resolved name 'a' to daemon mds.a 2024-09-21T03:45:57.121 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:57.115+0000 7f8e86f51740 20 client.15390 populate_metadata read hostname 'smithi039' 2024-09-21T03:45:57.121 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:57.115+0000 7f8e86f51740 1 --2- 172.21.15.39:0/593346231 >> [v2:172.21.15.39:6840/4247190456,v1:172.21.15.39:6841/4247190456] conn(0x55e656db8c90 0x55e6572583e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:57.121 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:57.115+0000 7f8e86f51740 4 client.15390 mds_command: new command op to 14868 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-21T03:45:57.121 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:57.115+0000 7f8e86f51740 1 -- 172.21.15.39:0/593346231 --> [v2:172.21.15.39:6840/4247190456,v1:172.21.15.39:6841/4247190456] -- command(tid 0: {"prefix": "status"}) -- 0x55e657140610 con 0x55e656db8c90 2024-09-21T03:45:57.121 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:57.115+0000 7f8e73fff640 1 --2- 172.21.15.39:0/593346231 >> [v2:172.21.15.39:6840/4247190456,v1:172.21.15.39:6841/4247190456] conn(0x55e656db8c90 0x55e6572583e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:57.123 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:57.115+0000 7f8e61ffb640 10 client.15390 ms_handle_connect on v2:172.21.15.39:6840/4247190456 2024-09-21T03:45:57.123 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:57.115+0000 7f8e73fff640 1 --2- 172.21.15.39:0/593346231 >> [v2:172.21.15.39:6840/4247190456,v1:172.21.15.39:6841/4247190456] conn(0x55e656db8c90 0x55e6572583e0 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=be45059c51d58519 server_cookie=c1e6b19819937552 in_seq=0 out_seq=0 2024-09-21T03:45:57.123 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:57.116+0000 7f8e61ffb640 1 -- 172.21.15.39:0/593346231 <== mds.4 v2:172.21.15.39:6840/4247190456 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x55e657140610 con 0x55e656db8c90 2024-09-21T03:45:57.123 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:57.116+0000 7f8e61ffb640 10 client.15390 handle_command_reply: tid=0 2024-09-21T03:45:57.123 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:57.116+0000 7f8e86f51740 1 client.15390 shutdown 2024-09-21T03:45:57.123 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:57.116+0000 7f8e86f51740 1 -- 172.21.15.39:0/593346231 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55e656dfc0a0 msgr2=0x55e6571eb980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:57.123 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:57.116+0000 7f8e86f51740 1 --2- 172.21.15.39:0/593346231 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55e656dfc0a0 0x55e6571eb980 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f8e6c009250 tx=0x7f8e6c009c40 comp rx=0 tx=0).stop 2024-09-21T03:45:57.123 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:57.116+0000 7f8e86f51740 1 -- 172.21.15.39:0/593346231 shutdown_connections 2024-09-21T03:45:57.123 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:57.116+0000 7f8e86f51740 1 --2- 172.21.15.39:0/593346231 >> [v2:172.21.15.39:6840/4247190456,v1:172.21.15.39:6841/4247190456] conn(0x55e656db8c90 0x55e6572583e0 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:57.123 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:57.116+0000 7f8e86f51740 1 --2- 172.21.15.39:0/593346231 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e656e24f60 0x55e6571f0f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:57.123 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:57.116+0000 7f8e86f51740 1 --2- 172.21.15.39:0/593346231 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e656e04280 0x55e6571ebec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:57.123 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:57.116+0000 7f8e86f51740 1 --2- 172.21.15.39:0/593346231 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55e656dfc0a0 0x55e6571eb980 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:57.123 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:57.116+0000 7f8e61ffb640 0 client.15390 ms_handle_reset on v2:172.21.15.39:6840/4247190456 2024-09-21T03:45:57.124 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:57.116+0000 7f8e86f51740 1 -- 172.21.15.39:0/593346231 >> 172.21.15.39:0/593346231 conn(0x55e656e19350 msgr2=0x55e6571c8600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:45:57.124 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:57.116+0000 7f8e86f51740 1 -- 172.21.15.39:0/593346231 shutdown_connections 2024-09-21T03:45:57.124 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:57.116+0000 7f8e86f51740 1 -- 172.21.15.39:0/593346231 wait complete. 2024-09-21T03:45:57.124 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:57.116+0000 7f8e86f51740 20 client.15390 trim_cache size 0 max 16384 2024-09-21T03:45:57.124 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-09-21T03:45:57.124 INFO:teuthology.orchestra.run.smithi039.stdout: "cluster_fsid": "51116d5c-77ca-11ef-baf5-efdc52797490", 2024-09-21T03:45:57.124 INFO:teuthology.orchestra.run.smithi039.stdout: "whoami": 4, 2024-09-21T03:45:57.124 INFO:teuthology.orchestra.run.smithi039.stdout: "id": 14868, 2024-09-21T03:45:57.124 INFO:teuthology.orchestra.run.smithi039.stdout: "want_state": "up:active", 2024-09-21T03:45:57.124 INFO:teuthology.orchestra.run.smithi039.stdout: "state": "up:active", 2024-09-21T03:45:57.124 INFO:teuthology.orchestra.run.smithi039.stdout: "fs_name": "cephfs", 2024-09-21T03:45:57.124 INFO:teuthology.orchestra.run.smithi039.stdout: "rank_uptime": 92.423195727999996, 2024-09-21T03:45:57.125 INFO:teuthology.orchestra.run.smithi039.stdout: "mdsmap_epoch": 28, 2024-09-21T03:45:57.125 INFO:teuthology.orchestra.run.smithi039.stdout: "osdmap_epoch": 79, 2024-09-21T03:45:57.125 INFO:teuthology.orchestra.run.smithi039.stdout: "osdmap_epoch_barrier": 78, 2024-09-21T03:45:57.125 INFO:teuthology.orchestra.run.smithi039.stdout: "uptime": 114.85629545899999 2024-09-21T03:45:57.125 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-09-21T03:45:57.125 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:57.117+0000 7f8e809dd640 1 -- 172.21.15.39:0/3139532328 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f8e680b65f0 msgr2=0x7f8e680b8b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:57.126 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:57.117+0000 7f8e809dd640 1 --2- 172.21.15.39:0/3139532328 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f8e680b65f0 0x7f8e680b8b00 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f8e64009230 tx=0x7f8e6402a3f0 comp rx=0 tx=0).stop 2024-09-21T03:45:57.126 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:57.117+0000 7f8e809dd640 1 -- 172.21.15.39:0/3139532328 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8e7c109df0 msgr2=0x7f8e7c19fd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:57.126 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:57.117+0000 7f8e809dd640 1 --2- 172.21.15.39:0/3139532328 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8e7c109df0 0x7f8e7c19fd40 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f8e7400b970 tx=0x7f8e7400be30 comp rx=0 tx=0).stop 2024-09-21T03:45:57.126 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:57.117+0000 7f8e809dd640 1 -- 172.21.15.39:0/3139532328 shutdown_connections 2024-09-21T03:45:57.126 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:57.117+0000 7f8e809dd640 1 --2- 172.21.15.39:0/3139532328 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f8e680b65f0 0x7f8e680b8b00 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:57.126 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:57.117+0000 7f8e809dd640 1 --2- 172.21.15.39:0/3139532328 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8e7c109df0 0x7f8e7c19fd40 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:57.126 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:57.117+0000 7f8e809dd640 1 --2- 172.21.15.39:0/3139532328 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8e7c1094a0 0x7f8e7c19aaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:57.126 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:57.117+0000 7f8e809dd640 1 --2- 172.21.15.39:0/3139532328 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8e7c103b60 0x7f8e7c19a500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:57.126 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:57.117+0000 7f8e809dd640 1 -- 172.21.15.39:0/3139532328 >> 172.21.15.39:0/3139532328 conn(0x7f8e7c07b850 msgr2=0x7f8e7c107590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:45:57.126 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:57.117+0000 7f8e809dd640 1 -- 172.21.15.39:0/3139532328 shutdown_connections 2024-09-21T03:45:57.126 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:57.117+0000 7f8e809dd640 1 -- 172.21.15.39:0/3139532328 wait complete. 2024-09-21T03:45:57.218 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:45:56 smithi161 ceph-mon[34533]: pgmap v322: 97 pgs: 97 active+clean; 647 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 12 op/s 2024-09-21T03:45:57.224 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:45:56 smithi150 ceph-mon[34289]: pgmap v322: 97 pgs: 97 active+clean; 647 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 12 op/s 2024-09-21T03:45:57.935 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 51116d5c-77ca-11ef-baf5-efdc52797490 -- ceph tell mds.c status 2024-09-21T03:45:58.197 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/51116d5c-77ca-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:45:58.767 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:58.741+0000 7feee51f1640 1 -- 172.21.15.39:0/118589478 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7feee0108d70 msgr2=0x7feee010f5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:58.767 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:58.741+0000 7feee51f1640 1 --2- 172.21.15.39:0/118589478 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7feee0108d70 0x7feee010f5a0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7feed8009ca0 tx=0x7feed802a8e0 comp rx=0 tx=0).stop 2024-09-21T03:45:58.767 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:58.741+0000 7feee51f1640 1 -- 172.21.15.39:0/118589478 shutdown_connections 2024-09-21T03:45:58.768 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:58.741+0000 7feee51f1640 1 --2- 172.21.15.39:0/118589478 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7feee0108d70 0x7feee010f5a0 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:58.768 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:58.741+0000 7feee51f1640 1 --2- 172.21.15.39:0/118589478 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7feee0108420 0x7feee0108830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:58.768 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:58.741+0000 7feee51f1640 1 --2- 172.21.15.39:0/118589478 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7feee0102ae0 0x7feee0102ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:58.768 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:58.742+0000 7feee51f1640 1 -- 172.21.15.39:0/118589478 >> 172.21.15.39:0/118589478 conn(0x7feee0079770 msgr2=0x7feee0079b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:45:58.768 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:58.742+0000 7feee51f1640 1 -- 172.21.15.39:0/118589478 shutdown_connections 2024-09-21T03:45:58.768 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:58.742+0000 7feee51f1640 1 -- 172.21.15.39:0/118589478 wait complete. 2024-09-21T03:45:58.768 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:58.743+0000 7feee51f1640 1 Processor -- start 2024-09-21T03:45:58.768 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:58.743+0000 7feee51f1640 1 -- start start 2024-09-21T03:45:58.768 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:58.743+0000 7feee51f1640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7feee0102ae0 0x7feee0199540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:58.768 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:58.743+0000 7feee51f1640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7feee0108420 0x7feee0199ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:58.768 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:58.743+0000 7feee51f1640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7feee0108d70 0x7feee019ed80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:58.768 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:58.743+0000 7feee51f1640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7feee0112af0 con 0x7feee0108d70 2024-09-21T03:45:58.768 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:58.743+0000 7feee51f1640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7feee0112970 con 0x7feee0102ae0 2024-09-21T03:45:58.769 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:58.743+0000 7feee51f1640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7feee0112c70 con 0x7feee0108420 2024-09-21T03:45:58.769 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:58.743+0000 7feedf59e640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7feee0108d70 0x7feee019ed80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:58.769 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:58.743+0000 7feedf59e640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7feee0108d70 0x7feee019ed80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:42472/0 (socket says 172.21.15.39:42472) 2024-09-21T03:45:58.769 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:58.743+0000 7feedf59e640 1 -- 172.21.15.39:0/3252743552 learned_addr learned my addr 172.21.15.39:0/3252743552 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:45:58.769 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:58.744+0000 7feeded9d640 1 --2- 172.21.15.39:0/3252743552 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7feee0102ae0 0x7feee0199540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:58.769 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:58.744+0000 7feede59c640 1 --2- 172.21.15.39:0/3252743552 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7feee0108420 0x7feee0199ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:58.769 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:58.744+0000 7feedf59e640 1 -- 172.21.15.39:0/3252743552 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7feee0108420 msgr2=0x7feee0199ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:58.769 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:58.744+0000 7feedf59e640 1 --2- 172.21.15.39:0/3252743552 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7feee0108420 0x7feee0199ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:58.769 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:58.744+0000 7feedf59e640 1 -- 172.21.15.39:0/3252743552 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7feee0102ae0 msgr2=0x7feee0199540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:58.769 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:58.744+0000 7feedf59e640 1 --2- 172.21.15.39:0/3252743552 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7feee0102ae0 0x7feee0199540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:58.769 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:58.744+0000 7feedf59e640 1 -- 172.21.15.39:0/3252743552 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feee019f4b0 con 0x7feee0108d70 2024-09-21T03:45:58.769 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:58.744+0000 7feedf59e640 1 --2- 172.21.15.39:0/3252743552 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7feee0108d70 0x7feee019ed80 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7feed8005cc0 tx=0x7feed8003ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:45:58.769 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:58.744+0000 7feec7fff640 1 -- 172.21.15.39:0/3252743552 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feed8004040 con 0x7feee0108d70 2024-09-21T03:45:58.769 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:58.744+0000 7feee51f1640 1 -- 172.21.15.39:0/3252743552 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feee019f740 con 0x7feee0108d70 2024-09-21T03:45:58.770 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:58.744+0000 7feec7fff640 1 -- 172.21.15.39:0/3252743552 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7feed8004690 con 0x7feee0108d70 2024-09-21T03:45:58.770 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:58.744+0000 7feec7fff640 1 -- 172.21.15.39:0/3252743552 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feed8030e70 con 0x7feee0108d70 2024-09-21T03:45:58.770 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:58.744+0000 7feee51f1640 1 -- 172.21.15.39:0/3252743552 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feee006bb60 con 0x7feee0108d70 2024-09-21T03:45:58.770 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:58.747+0000 7feec7fff640 1 -- 172.21.15.39:0/3252743552 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 23) ==== 154529+0+0 (secure 0 0 0) 0x7feed80305e0 con 0x7feee0108d70 2024-09-21T03:45:58.770 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:58.747+0000 7feeeb765740 1 Processor -- start 2024-09-21T03:45:58.770 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:58.747+0000 7feeeb765740 1 -- start start 2024-09-21T03:45:58.770 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:58.747+0000 7feeeb765740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561b2fe42f60 0x561b301e8f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:58.770 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:58.747+0000 7feeeb765740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561b2fe1a0a0 0x561b301e9440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:58.770 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:58.747+0000 7feeeb765740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561b2fe22280 0x561b301e9980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:58.770 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:58.747+0000 7feeeb765740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x561b30160c10 con 0x561b2fe1a0a0 2024-09-21T03:45:58.770 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:58.747+0000 7feeeb765740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x561b30150040 con 0x561b2fe42f60 2024-09-21T03:45:58.770 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:58.747+0000 7feeeb765740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x561b30153e50 con 0x561b2fe22280 2024-09-21T03:45:58.770 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:58.747+0000 7feeded9d640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561b2fe1a0a0 0x561b301e9440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:58.770 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:58.747+0000 7feeded9d640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561b2fe1a0a0 0x561b301e9440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:42474/0 (socket says 172.21.15.39:42474) 2024-09-21T03:45:58.770 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:58.747+0000 7feeded9d640 1 -- 172.21.15.39:0/645790886 learned_addr learned my addr 172.21.15.39:0/645790886 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:45:58.771 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:58.747+0000 7feede59c640 1 --2- 172.21.15.39:0/645790886 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561b2fe42f60 0x561b301e8f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:58.771 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:58.748+0000 7feede59c640 1 --2- 172.21.15.39:0/645790886 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561b2fe22280 0x561b301e9980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:58.771 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:58.748+0000 7feeded9d640 1 -- 172.21.15.39:0/645790886 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561b2fe22280 msgr2=0x561b301e9980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:45:58.771 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:58.748+0000 7feeded9d640 1 --2- 172.21.15.39:0/645790886 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561b2fe22280 0x561b301e9980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:58.771 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:58.748+0000 7feeded9d640 1 -- 172.21.15.39:0/645790886 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561b2fe42f60 msgr2=0x561b301e8f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:45:58.771 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:58.748+0000 7feeded9d640 1 --2- 172.21.15.39:0/645790886 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561b2fe42f60 0x561b301e8f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:58.771 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:58.748+0000 7feeded9d640 1 -- 172.21.15.39:0/645790886 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561b2feae540 con 0x561b2fe1a0a0 2024-09-21T03:45:58.771 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:58.748+0000 7feec7fff640 1 --2- 172.21.15.39:0/3252743552 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7feec80b6560 0x7feec80b8a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:58.771 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:58.748+0000 7feede59c640 1 --2- 172.21.15.39:0/3252743552 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7feec80b6560 0x7feec80b8a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:58.771 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:58.748+0000 7feeded9d640 1 --2- 172.21.15.39:0/645790886 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561b2fe1a0a0 0x561b301e9440 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7feed000b820 tx=0x7feed000bce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c3ec5b109507fda server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:45:58.771 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:58.748+0000 7feec7fff640 1 -- 172.21.15.39:0/3252743552 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8484+0+0 (secure 0 0 0) 0x7feed8108940 con 0x7feee0108d70 2024-09-21T03:45:58.771 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:58.748+0000 7feec57fa640 1 -- 172.21.15.39:0/645790886 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feed000d870 con 0x561b2fe1a0a0 2024-09-21T03:45:58.771 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:58.748+0000 7feec57fa640 1 -- 172.21.15.39:0/645790886 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7feed0019070 con 0x561b2fe1a0a0 2024-09-21T03:45:58.771 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:58.748+0000 7feec57fa640 1 -- 172.21.15.39:0/645790886 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feed0014dc0 con 0x561b2fe1a0a0 2024-09-21T03:45:58.772 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:58.749+0000 7feeeb765740 1 -- 172.21.15.39:0/645790886 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561b2fe1a0a0 msgr2=0x561b301e9440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:58.772 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:58.749+0000 7feeeb765740 1 --2- 172.21.15.39:0/645790886 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561b2fe1a0a0 0x561b301e9440 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7feed000b820 tx=0x7feed000bce0 comp rx=0 tx=0).stop 2024-09-21T03:45:58.772 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:58.749+0000 7feede59c640 1 --2- 172.21.15.39:0/3252743552 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7feec80b6560 0x7feec80b8a70 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7feed4006fd0 tx=0x7feed400a230 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:45:58.772 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:58.749+0000 7feeeb765740 1 -- 172.21.15.39:0/645790886 shutdown_connections 2024-09-21T03:45:58.772 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:58.749+0000 7feeeb765740 1 --2- 172.21.15.39:0/645790886 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561b2fe22280 0x561b301e9980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:58.772 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:58.749+0000 7feeeb765740 1 --2- 172.21.15.39:0/645790886 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561b2fe1a0a0 0x561b301e9440 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:58.772 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:58.749+0000 7feeeb765740 1 --2- 172.21.15.39:0/645790886 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561b2fe42f60 0x561b301e8f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:58.772 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:58.749+0000 7feeeb765740 1 -- 172.21.15.39:0/645790886 >> 172.21.15.39:0/645790886 conn(0x561b2fe37350 msgr2=0x561b301e6df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:45:58.772 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:58.749+0000 7feeeb765740 1 -- 172.21.15.39:0/645790886 shutdown_connections 2024-09-21T03:45:58.772 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:58.749+0000 7feeeb765740 1 -- 172.21.15.39:0/645790886 wait complete. 2024-09-21T03:45:58.772 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:58.750+0000 7feeeb765740 1 -- start start 2024-09-21T03:45:58.772 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:58.750+0000 7feeeb765740 1 Processor -- start 2024-09-21T03:45:58.772 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:58.750+0000 7feeeb765740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561b2fe1a0a0 0x561b301f8b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:58.772 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:58.750+0000 7feeeb765740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561b2fe22280 0x561b301f90d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:58.772 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:58.750+0000 7feeeb765740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561b2fe42f60 0x561b30204d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:58.773 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:58.750+0000 7feeeb765740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x561b2ffb15f0 con 0x561b2fe42f60 2024-09-21T03:45:58.773 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:58.750+0000 7feeeb765740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x561b300eb010 con 0x561b2fe22280 2024-09-21T03:45:58.773 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:58.750+0000 7feeeb765740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x561b301fba40 con 0x561b2fe1a0a0 2024-09-21T03:45:58.773 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:58.750+0000 7feeeb765740 10 client.0 _ll_register_callbacks cb 0x561b3014a340 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:45:58.773 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:58.750+0000 7feedf59e640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561b2fe42f60 0x561b30204d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:58.773 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:58.750+0000 7feedf59e640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561b2fe42f60 0x561b30204d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:42488/0 (socket says 172.21.15.39:42488) 2024-09-21T03:45:58.773 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:58.750+0000 7feedf59e640 1 -- 172.21.15.39:0/3717381504 learned_addr learned my addr 172.21.15.39:0/3717381504 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:45:58.773 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:58.750+0000 7feede59c640 1 --2- 172.21.15.39:0/3717381504 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561b2fe22280 0x561b301f90d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:58.773 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:58.750+0000 7feeded9d640 1 --2- 172.21.15.39:0/3717381504 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561b2fe1a0a0 0x561b301f8b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:58.773 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:58.750+0000 7feedf59e640 1 -- 172.21.15.39:0/3717381504 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561b2fe1a0a0 msgr2=0x561b301f8b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:45:58.773 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:58.750+0000 7feedf59e640 1 --2- 172.21.15.39:0/3717381504 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561b2fe1a0a0 0x561b301f8b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:58.773 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:58.750+0000 7feedf59e640 1 -- 172.21.15.39:0/3717381504 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561b2fe22280 msgr2=0x561b301f90d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:45:58.774 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:58.750+0000 7feedf59e640 1 --2- 172.21.15.39:0/3717381504 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561b2fe22280 0x561b301f90d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:58.774 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:58.750+0000 7feedf59e640 1 -- 172.21.15.39:0/3717381504 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561b30118a00 con 0x561b2fe42f60 2024-09-21T03:45:58.774 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:58.750+0000 7feede59c640 1 --2- 172.21.15.39:0/3717381504 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561b2fe22280 0x561b301f90d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:45:58.774 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:58.750+0000 7feeded9d640 1 --2- 172.21.15.39:0/3717381504 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561b2fe1a0a0 0x561b301f8b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:45:58.774 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:58.751+0000 7feedf59e640 1 --2- 172.21.15.39:0/3717381504 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561b2fe42f60 0x561b30204d70 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7feed80cafa0 tx=0x7feed80ce3e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc01975d28ee6e29 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:45:58.774 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:58.751+0000 7feec2ff5640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-09-21T03:45:58.774 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:58.751+0000 7feec2ff5640 1 -- 172.21.15.39:0/3717381504 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feed80098d0 con 0x561b2fe42f60 2024-09-21T03:45:58.775 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:58.751+0000 7feec2ff5640 1 -- 172.21.15.39:0/3717381504 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8484+0+0 (secure 0 0 0) 0x7feed80d3180 con 0x561b2fe42f60 2024-09-21T03:45:58.775 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:58.751+0000 7feeeb765740 1 -- 172.21.15.39:0/3717381504 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7feed0019070 con 0x561b2fe42f60 2024-09-21T03:45:58.775 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:58.751+0000 7feec2ff5640 1 -- 172.21.15.39:0/3717381504 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7feed80ce4e0 con 0x561b2fe42f60 2024-09-21T03:45:58.775 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:58.751+0000 7feeeb765740 10 client.15402 fetch_fsmap learned FSMap version 28 2024-09-21T03:45:58.775 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:58.751+0000 7feeeb765740 1 -- 172.21.15.39:0/3717381504 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x561b302078e0 con 0x561b2fe42f60 2024-09-21T03:45:58.775 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:58.751+0000 7feec2ff5640 1 -- 172.21.15.39:0/3717381504 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7feed0019070 con 0x561b2fe42f60 2024-09-21T03:45:58.775 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:58.751+0000 7feec2ff5640 1 -- 172.21.15.39:0/3717381504 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7feed80e4b80 con 0x561b2fe42f60 2024-09-21T03:45:58.776 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:58.751+0000 7feeeb765740 10 client.15402 fetch_fsmap finished waiting for FSMap version 28 2024-09-21T03:45:58.776 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:58.751+0000 7feeeb765740 10 client.15402 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-21T03:45:58.776 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:58.751+0000 7feeeb765740 20 client.15402 populate_metadata read hostname 'smithi039' 2024-09-21T03:45:58.776 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:58.751+0000 7feeeb765740 1 --2- 172.21.15.39:0/3717381504 >> [v2:172.21.15.161:6836/3451544597,v1:172.21.15.161:6837/3451544597] conn(0x561b2fe018e0 0x561b3020b3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:58.776 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:58.752+0000 7feeeb765740 4 client.15402 mds_command: new command op to 24646 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:45:58.776 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:58.752+0000 7feeeb765740 1 -- 172.21.15.39:0/3717381504 --> [v2:172.21.15.161:6836/3451544597,v1:172.21.15.161:6837/3451544597] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561b30162b60 con 0x561b2fe018e0 2024-09-21T03:45:58.776 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:58.752+0000 7feeded9d640 1 --2- 172.21.15.39:0/3717381504 >> [v2:172.21.15.161:6836/3451544597,v1:172.21.15.161:6837/3451544597] conn(0x561b2fe018e0 0x561b3020b3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:58.776 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:58.752+0000 7feec2ff5640 10 client.15402 ms_handle_connect on v2:172.21.15.161:6836/3451544597 2024-09-21T03:45:58.776 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:58.752+0000 7feeded9d640 1 --2- 172.21.15.39:0/3717381504 >> [v2:172.21.15.161:6836/3451544597,v1:172.21.15.161:6837/3451544597] conn(0x561b2fe018e0 0x561b3020b3f0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=bdfa36cdf462119b server_cookie=d6332f641c0f4b1c in_seq=0 out_seq=0 2024-09-21T03:45:58.776 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:58.753+0000 7feec2ff5640 1 -- 172.21.15.39:0/3717381504 <== mds.3 v2:172.21.15.161:6836/3451544597 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x561b30162b60 con 0x561b2fe018e0 2024-09-21T03:45:58.776 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:58.753+0000 7feec2ff5640 10 client.15402 handle_command_reply: tid=0 2024-09-21T03:45:58.777 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:58.753+0000 7feeeb765740 1 client.15402 shutdown 2024-09-21T03:45:58.777 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:58.753+0000 7feeeb765740 1 -- 172.21.15.39:0/3717381504 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561b2fe42f60 msgr2=0x561b30204d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:58.777 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:58.753+0000 7feeeb765740 1 --2- 172.21.15.39:0/3717381504 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561b2fe42f60 0x561b30204d70 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7feed80cafa0 tx=0x7feed80ce3e0 comp rx=0 tx=0).stop 2024-09-21T03:45:58.777 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:58.753+0000 7feeeb765740 1 -- 172.21.15.39:0/3717381504 shutdown_connections 2024-09-21T03:45:58.777 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:58.753+0000 7feeeb765740 1 --2- 172.21.15.39:0/3717381504 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561b2fe42f60 0x561b30204d70 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:58.777 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:58.753+0000 7feeeb765740 1 --2- 172.21.15.39:0/3717381504 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561b2fe22280 0x561b301f90d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:58.777 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:58.753+0000 7feeeb765740 1 --2- 172.21.15.39:0/3717381504 >> [v2:172.21.15.161:6836/3451544597,v1:172.21.15.161:6837/3451544597] conn(0x561b2fe018e0 0x561b3020b3f0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:58.777 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:58.753+0000 7feeeb765740 1 --2- 172.21.15.39:0/3717381504 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561b2fe1a0a0 0x561b301f8b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:58.777 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:58.753+0000 7feec2ff5640 0 client.15402 ms_handle_reset on v2:172.21.15.161:6836/3451544597 2024-09-21T03:45:58.777 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:58.753+0000 7feeeb765740 1 -- 172.21.15.39:0/3717381504 >> 172.21.15.39:0/3717381504 conn(0x561b2fe37350 msgr2=0x561b301e53b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:45:58.778 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:58.753+0000 7feeeb765740 1 -- 172.21.15.39:0/3717381504 shutdown_connections 2024-09-21T03:45:58.778 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:58.753+0000 7feeeb765740 1 -- 172.21.15.39:0/3717381504 wait complete. 2024-09-21T03:45:58.778 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:58.753+0000 7feeeb765740 20 client.15402 trim_cache size 0 max 16384 2024-09-21T03:45:58.783 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:58.771+0000 7feeeb765740 1 Processor -- start 2024-09-21T03:45:58.783 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:58.772+0000 7feeeb765740 1 -- start start 2024-09-21T03:45:58.783 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:58.772+0000 7feeeb765740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561b2fe1a0a0 0x561b30204ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:58.783 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:58.772+0000 7feeeb765740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561b2fe22280 0x561b30205220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:58.783 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:58.772+0000 7feeeb765740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561b2fe42f60 0x561b301e53b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:58.783 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:58.772+0000 7feeeb765740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x561b30209730 con 0x561b2fe42f60 2024-09-21T03:45:58.783 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:58.772+0000 7feeeb765740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x561b30209e00 con 0x561b2fe1a0a0 2024-09-21T03:45:58.783 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:58.772+0000 7feeeb765740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x561b3020a4d0 con 0x561b2fe22280 2024-09-21T03:45:58.784 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:58.772+0000 7feedf59e640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561b2fe42f60 0x561b301e53b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:58.784 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:58.772+0000 7feedf59e640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561b2fe42f60 0x561b301e53b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:42504/0 (socket says 172.21.15.39:42504) 2024-09-21T03:45:58.784 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:58.772+0000 7feedf59e640 1 -- 172.21.15.39:0/3688727020 learned_addr learned my addr 172.21.15.39:0/3688727020 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:45:58.784 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:58.772+0000 7feeded9d640 1 --2- 172.21.15.39:0/3688727020 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561b2fe1a0a0 0x561b30204ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:58.784 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:58.772+0000 7feede59c640 1 --2- 172.21.15.39:0/3688727020 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561b2fe22280 0x561b30205220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:58.784 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:58.772+0000 7feedf59e640 1 -- 172.21.15.39:0/3688727020 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561b2fe22280 msgr2=0x561b30205220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:45:58.784 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:58.772+0000 7feedf59e640 1 --2- 172.21.15.39:0/3688727020 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561b2fe22280 0x561b30205220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:58.784 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:58.772+0000 7feedf59e640 1 -- 172.21.15.39:0/3688727020 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561b2fe1a0a0 msgr2=0x561b30204ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:45:58.784 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:58.772+0000 7feedf59e640 1 --2- 172.21.15.39:0/3688727020 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561b2fe1a0a0 0x561b30204ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:58.784 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:58.772+0000 7feedf59e640 1 -- 172.21.15.39:0/3688727020 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561b301eb810 con 0x561b2fe42f60 2024-09-21T03:45:58.784 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:58.772+0000 7feeded9d640 1 --2- 172.21.15.39:0/3688727020 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561b2fe1a0a0 0x561b30204ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:45:58.784 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:58.772+0000 7feedf59e640 1 --2- 172.21.15.39:0/3688727020 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561b2fe42f60 0x561b301e53b0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7feed8006010 tx=0x7feed80ce450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7089af737c968d91 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:45:58.784 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:58.773+0000 7feec4ff9640 1 -- 172.21.15.39:0/3688727020 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feed80b8070 con 0x561b2fe42f60 2024-09-21T03:45:58.784 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:58.773+0000 7feec4ff9640 1 -- 172.21.15.39:0/3688727020 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7feed8036070 con 0x561b2fe42f60 2024-09-21T03:45:58.784 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:58.773+0000 7feec4ff9640 1 -- 172.21.15.39:0/3688727020 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feed80cf0d0 con 0x561b2fe42f60 2024-09-21T03:45:58.785 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:58.773+0000 7feeeb765740 1 -- 172.21.15.39:0/3688727020 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561b2fe42f60 msgr2=0x561b301e53b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:58.785 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:58.773+0000 7feeeb765740 1 --2- 172.21.15.39:0/3688727020 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561b2fe42f60 0x561b301e53b0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7feed8006010 tx=0x7feed80ce450 comp rx=0 tx=0).stop 2024-09-21T03:45:58.785 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:58.773+0000 7feeeb765740 1 -- 172.21.15.39:0/3688727020 shutdown_connections 2024-09-21T03:45:58.785 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:58.773+0000 7feeeb765740 1 --2- 172.21.15.39:0/3688727020 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561b2fe42f60 0x561b301e53b0 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:58.785 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:58.773+0000 7feeeb765740 1 --2- 172.21.15.39:0/3688727020 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561b2fe22280 0x561b30205220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:58.785 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:58.773+0000 7feeeb765740 1 --2- 172.21.15.39:0/3688727020 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561b2fe1a0a0 0x561b30204ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:58.785 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:58.774+0000 7feeeb765740 1 -- 172.21.15.39:0/3688727020 >> 172.21.15.39:0/3688727020 conn(0x561b2fe37350 msgr2=0x561b301f73d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:45:58.785 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:58.774+0000 7feeeb765740 1 -- 172.21.15.39:0/3688727020 shutdown_connections 2024-09-21T03:45:58.785 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:58.774+0000 7feeeb765740 1 -- 172.21.15.39:0/3688727020 wait complete. 2024-09-21T03:45:58.785 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:58.774+0000 7feeeb765740 1 -- start start 2024-09-21T03:45:58.785 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:58.774+0000 7feeeb765740 1 Processor -- start 2024-09-21T03:45:58.785 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:58.774+0000 7feeeb765740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561b2fe1a0a0 0x561b3020de80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:58.785 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:58.774+0000 7feeeb765740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561b2fe22280 0x561b3020e400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:58.785 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:58.774+0000 7feeeb765740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561b2fe42f60 0x561b30213410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:58.786 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:58.774+0000 7feeeb765740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x561b3020b270 con 0x561b2fe42f60 2024-09-21T03:45:58.786 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:58.774+0000 7feeeb765740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x561b3020aba0 con 0x561b2fe22280 2024-09-21T03:45:58.786 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:58.774+0000 7feeeb765740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x561b3020b940 con 0x561b2fe1a0a0 2024-09-21T03:45:58.786 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:58.774+0000 7feeeb765740 10 client.0 _ll_register_callbacks cb 0x7feed8009460 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:45:58.786 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:58.775+0000 7feedf59e640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561b2fe42f60 0x561b30213410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:58.786 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:58.775+0000 7feedf59e640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561b2fe42f60 0x561b30213410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:42506/0 (socket says 172.21.15.39:42506) 2024-09-21T03:45:58.786 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:58.775+0000 7feedf59e640 1 -- 172.21.15.39:0/2367697248 learned_addr learned my addr 172.21.15.39:0/2367697248 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:45:58.786 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:58.775+0000 7feeded9d640 1 --2- 172.21.15.39:0/2367697248 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561b2fe1a0a0 0x561b3020de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:58.786 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:58.775+0000 7feede59c640 1 --2- 172.21.15.39:0/2367697248 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561b2fe22280 0x561b3020e400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:58.786 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:58.775+0000 7feedf59e640 1 -- 172.21.15.39:0/2367697248 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561b2fe1a0a0 msgr2=0x561b3020de80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:45:58.786 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:58.775+0000 7feedf59e640 1 --2- 172.21.15.39:0/2367697248 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561b2fe1a0a0 0x561b3020de80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:58.786 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:58.775+0000 7feedf59e640 1 -- 172.21.15.39:0/2367697248 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561b2fe22280 msgr2=0x561b3020e400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:45:58.786 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:58.775+0000 7feedf59e640 1 --2- 172.21.15.39:0/2367697248 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561b2fe22280 0x561b3020e400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:58.787 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:58.775+0000 7feedf59e640 1 -- 172.21.15.39:0/2367697248 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561b30213a20 con 0x561b2fe42f60 2024-09-21T03:45:58.787 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:58.775+0000 7feede59c640 1 --2- 172.21.15.39:0/2367697248 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561b2fe22280 0x561b3020e400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:45:58.787 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:58.775+0000 7feedf59e640 1 --2- 172.21.15.39:0/2367697248 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561b2fe42f60 0x561b30213410 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7feed80e4f20 tx=0x7feed80ce3e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=24bb1e13de7b551c server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:45:58.787 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:58.775+0000 7feec2ff5640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-09-21T03:45:58.787 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:58.775+0000 7feec2ff5640 1 -- 172.21.15.39:0/2367697248 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feed8091060 con 0x561b2fe42f60 2024-09-21T03:45:58.787 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:58.775+0000 7feec2ff5640 1 -- 172.21.15.39:0/2367697248 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8484+0+0 (secure 0 0 0) 0x7feed8004b40 con 0x561b2fe42f60 2024-09-21T03:45:58.787 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:58.775+0000 7feeeb765740 1 -- 172.21.15.39:0/2367697248 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7feed8036070 con 0x561b2fe42f60 2024-09-21T03:45:58.787 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:58.775+0000 7feec2ff5640 1 -- 172.21.15.39:0/2367697248 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7feed80b8070 con 0x561b2fe42f60 2024-09-21T03:45:58.787 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:58.775+0000 7feeeb765740 10 client.15408 fetch_fsmap learned FSMap version 28 2024-09-21T03:45:58.787 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:58.775+0000 7feeeb765740 1 -- 172.21.15.39:0/2367697248 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x561b30213a20 con 0x561b2fe42f60 2024-09-21T03:45:58.787 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:58.776+0000 7feec2ff5640 1 -- 172.21.15.39:0/2367697248 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7feed8036070 con 0x561b2fe42f60 2024-09-21T03:45:58.787 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:58.776+0000 7feec2ff5640 1 -- 172.21.15.39:0/2367697248 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7feed80cd6a0 con 0x561b2fe42f60 2024-09-21T03:45:58.787 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:58.776+0000 7feeeb765740 10 client.15408 fetch_fsmap finished waiting for FSMap version 28 2024-09-21T03:45:58.787 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:58.776+0000 7feeeb765740 10 client.15408 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-21T03:45:58.788 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:58.776+0000 7feeeb765740 20 client.15408 populate_metadata read hostname 'smithi039' 2024-09-21T03:45:58.788 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:58.776+0000 7feeeb765740 1 --2- 172.21.15.39:0/2367697248 >> [v2:172.21.15.161:6836/3451544597,v1:172.21.15.161:6837/3451544597] conn(0x561b2fdd6c90 0x561b3027a290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:58.788 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:58.776+0000 7feeeb765740 4 client.15408 mds_command: new command op to 24646 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-21T03:45:58.788 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:58.776+0000 7feeeb765740 1 -- 172.21.15.39:0/2367697248 --> [v2:172.21.15.161:6836/3451544597,v1:172.21.15.161:6837/3451544597] -- command(tid 0: {"prefix": "status"}) -- 0x561b301eb560 con 0x561b2fdd6c90 2024-09-21T03:45:58.788 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:58.776+0000 7feeded9d640 1 --2- 172.21.15.39:0/2367697248 >> [v2:172.21.15.161:6836/3451544597,v1:172.21.15.161:6837/3451544597] conn(0x561b2fdd6c90 0x561b3027a290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:58.788 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:58.776+0000 7feeded9d640 1 --2- 172.21.15.39:0/2367697248 >> [v2:172.21.15.161:6836/3451544597,v1:172.21.15.161:6837/3451544597] conn(0x561b2fdd6c90 0x561b3027a290 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=28ae907e96672d18 server_cookie=5b267dfc0b99c2f0 in_seq=0 out_seq=0 2024-09-21T03:45:58.788 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:58.776+0000 7feec2ff5640 10 client.15408 ms_handle_connect on v2:172.21.15.161:6836/3451544597 2024-09-21T03:45:58.788 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:58.777+0000 7feec2ff5640 1 -- 172.21.15.39:0/2367697248 <== mds.3 v2:172.21.15.161:6836/3451544597 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x561b301eb560 con 0x561b2fdd6c90 2024-09-21T03:45:58.788 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:58.777+0000 7feec2ff5640 10 client.15408 handle_command_reply: tid=0 2024-09-21T03:45:58.788 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:58.777+0000 7feeeb765740 1 client.15408 shutdown 2024-09-21T03:45:58.788 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:58.777+0000 7feeeb765740 1 -- 172.21.15.39:0/2367697248 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561b2fe42f60 msgr2=0x561b30213410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:58.788 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:58.777+0000 7feeeb765740 1 --2- 172.21.15.39:0/2367697248 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561b2fe42f60 0x561b30213410 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7feed80e4f20 tx=0x7feed80ce3e0 comp rx=0 tx=0).stop 2024-09-21T03:45:58.788 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:58.777+0000 7feeeb765740 1 -- 172.21.15.39:0/2367697248 shutdown_connections 2024-09-21T03:45:58.789 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:58.777+0000 7feeeb765740 1 --2- 172.21.15.39:0/2367697248 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561b2fe42f60 0x561b30213410 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:58.789 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:58.777+0000 7feeeb765740 1 --2- 172.21.15.39:0/2367697248 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561b2fe22280 0x561b3020e400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:58.789 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:58.777+0000 7feeeb765740 1 --2- 172.21.15.39:0/2367697248 >> [v2:172.21.15.161:6836/3451544597,v1:172.21.15.161:6837/3451544597] conn(0x561b2fdd6c90 0x561b3027a290 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:58.789 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:58.777+0000 7feec2ff5640 0 client.15408 ms_handle_reset on v2:172.21.15.161:6836/3451544597 2024-09-21T03:45:58.789 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:58.777+0000 7feeeb765740 1 --2- 172.21.15.39:0/2367697248 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561b2fe1a0a0 0x561b3020de80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:58.789 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:58.777+0000 7feeeb765740 1 -- 172.21.15.39:0/2367697248 >> 172.21.15.39:0/2367697248 conn(0x561b2fe37350 msgr2=0x561b301e9f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:45:58.789 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:58.777+0000 7feeeb765740 1 -- 172.21.15.39:0/2367697248 shutdown_connections 2024-09-21T03:45:58.789 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:58.777+0000 7feeeb765740 1 -- 172.21.15.39:0/2367697248 wait complete. 2024-09-21T03:45:58.789 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:58.777+0000 7feeeb765740 20 client.15408 trim_cache size 0 max 16384 2024-09-21T03:45:58.789 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-09-21T03:45:58.790 INFO:teuthology.orchestra.run.smithi039.stdout: "cluster_fsid": "51116d5c-77ca-11ef-baf5-efdc52797490", 2024-09-21T03:45:58.790 INFO:teuthology.orchestra.run.smithi039.stdout: "whoami": 3, 2024-09-21T03:45:58.790 INFO:teuthology.orchestra.run.smithi039.stdout: "id": 24646, 2024-09-21T03:45:58.790 INFO:teuthology.orchestra.run.smithi039.stdout: "want_state": "up:active", 2024-09-21T03:45:58.790 INFO:teuthology.orchestra.run.smithi039.stdout: "state": "up:active", 2024-09-21T03:45:58.790 INFO:teuthology.orchestra.run.smithi039.stdout: "fs_name": "cephfs", 2024-09-21T03:45:58.790 INFO:teuthology.orchestra.run.smithi039.stdout: "rank_uptime": 94.381807449999997, 2024-09-21T03:45:58.790 INFO:teuthology.orchestra.run.smithi039.stdout: "mdsmap_epoch": 28, 2024-09-21T03:45:58.790 INFO:teuthology.orchestra.run.smithi039.stdout: "osdmap_epoch": 79, 2024-09-21T03:45:58.790 INFO:teuthology.orchestra.run.smithi039.stdout: "osdmap_epoch_barrier": 78, 2024-09-21T03:45:58.790 INFO:teuthology.orchestra.run.smithi039.stdout: "uptime": 127.715533705 2024-09-21T03:45:58.790 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-09-21T03:45:58.791 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:58.778+0000 7feee51f1640 1 -- 172.21.15.39:0/3252743552 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7feec80b6560 msgr2=0x7feec80b8a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:58.791 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:58.778+0000 7feee51f1640 1 --2- 172.21.15.39:0/3252743552 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7feec80b6560 0x7feec80b8a70 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7feed4006fd0 tx=0x7feed400a230 comp rx=0 tx=0).stop 2024-09-21T03:45:58.791 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:58.778+0000 7feee51f1640 1 -- 172.21.15.39:0/3252743552 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7feee0108d70 msgr2=0x7feee019ed80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:58.791 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:58.778+0000 7feee51f1640 1 --2- 172.21.15.39:0/3252743552 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7feee0108d70 0x7feee019ed80 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7feed8005cc0 tx=0x7feed8003ba0 comp rx=0 tx=0).stop 2024-09-21T03:45:58.791 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:58.778+0000 7feee51f1640 1 -- 172.21.15.39:0/3252743552 shutdown_connections 2024-09-21T03:45:58.791 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:58.778+0000 7feee51f1640 1 --2- 172.21.15.39:0/3252743552 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7feec80b6560 0x7feec80b8a70 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:58.791 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:58.778+0000 7feee51f1640 1 --2- 172.21.15.39:0/3252743552 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7feee0108d70 0x7feee019ed80 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:58.791 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:58.778+0000 7feee51f1640 1 --2- 172.21.15.39:0/3252743552 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7feee0108420 0x7feee0199ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:58.791 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:58.778+0000 7feee51f1640 1 --2- 172.21.15.39:0/3252743552 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7feee0102ae0 0x7feee0199540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:58.792 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:58.778+0000 7feee51f1640 1 -- 172.21.15.39:0/3252743552 >> 172.21.15.39:0/3252743552 conn(0x7feee0079770 msgr2=0x7feee0106510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:45:58.792 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:58.778+0000 7feee51f1640 1 -- 172.21.15.39:0/3252743552 shutdown_connections 2024-09-21T03:45:58.792 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:45:58.778+0000 7feee51f1640 1 -- 172.21.15.39:0/3252743552 wait complete. 2024-09-21T03:45:59.217 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:45:58 smithi161 ceph-mon[34533]: pgmap v323: 97 pgs: 97 active+clean; 647 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 85 B/s wr, 8 op/s 2024-09-21T03:45:59.218 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:45:58 smithi161 ceph-mon[34533]: mgrmap e23: x(active, since 8m), standbys: z, y 2024-09-21T03:45:59.224 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:45:58 smithi150 ceph-mon[34289]: pgmap v323: 97 pgs: 97 active+clean; 647 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 85 B/s wr, 8 op/s 2024-09-21T03:45:59.224 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:45:58 smithi150 ceph-mon[34289]: mgrmap e23: x(active, since 8m), standbys: z, y 2024-09-21T03:45:59.333 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:45:58 smithi039 ceph-mon[31211]: pgmap v323: 97 pgs: 97 active+clean; 647 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 85 B/s wr, 8 op/s 2024-09-21T03:45:59.334 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:45:58 smithi039 ceph-mon[31211]: mgrmap e23: x(active, since 8m), standbys: z, y 2024-09-21T03:45:59.577 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 51116d5c-77ca-11ef-baf5-efdc52797490 -- ceph tell mds.h status 2024-09-21T03:45:59.835 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/51116d5c-77ca-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:46:00.333 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:00.323+0000 7f21c4cf8640 1 -- 172.21.15.39:0/2425543003 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f21c0102430 msgr2=0x7f21c0102840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:46:00.333 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:00.323+0000 7f21c4cf8640 1 --2- 172.21.15.39:0/2425543003 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f21c0102430 0x7f21c0102840 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f21b0009900 tx=0x7f21b002a410 comp rx=0 tx=0).stop 2024-09-21T03:46:00.333 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:00.323+0000 7f21c4cf8640 1 -- 172.21.15.39:0/2425543003 shutdown_connections 2024-09-21T03:46:00.333 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:00.323+0000 7f21c4cf8640 1 --2- 172.21.15.39:0/2425543003 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f21c0105230 0x7f21c010eca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:00.334 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:00.323+0000 7f21c4cf8640 1 --2- 172.21.15.39:0/2425543003 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f21c01048e0 0x7f21c0104cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:00.334 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:00.323+0000 7f21c4cf8640 1 --2- 172.21.15.39:0/2425543003 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f21c0102430 0x7f21c0102840 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:00.334 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:00.323+0000 7f21c4cf8640 1 -- 172.21.15.39:0/2425543003 >> 172.21.15.39:0/2425543003 conn(0x7f21c0077800 msgr2=0x7f21c0077c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:46:00.334 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:00.323+0000 7f21c4cf8640 1 -- 172.21.15.39:0/2425543003 shutdown_connections 2024-09-21T03:46:00.334 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:00.323+0000 7f21c4cf8640 1 -- 172.21.15.39:0/2425543003 wait complete. 2024-09-21T03:46:00.334 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:00.324+0000 7f21c4cf8640 1 Processor -- start 2024-09-21T03:46:00.334 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:00.324+0000 7f21c4cf8640 1 -- start start 2024-09-21T03:46:00.334 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:00.324+0000 7f21c4cf8640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f21c0102430 0x7f21c0199490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:00.334 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:00.324+0000 7f21c4cf8640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f21c01048e0 0x7f21c0199a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:00.334 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:00.324+0000 7f21c4cf8640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f21c0105230 0x7f21c019ecd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:00.334 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:00.324+0000 7f21c4cf8640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f21c0112170 con 0x7f21c0105230 2024-09-21T03:46:00.334 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:00.324+0000 7f21c4cf8640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f21c0111ff0 con 0x7f21c0102430 2024-09-21T03:46:00.334 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:00.324+0000 7f21c4cf8640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f21c01122f0 con 0x7f21c01048e0 2024-09-21T03:46:00.335 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:00.324+0000 7f21bed9d640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f21c0105230 0x7f21c019ecd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:46:00.335 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:00.324+0000 7f21bed9d640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f21c0105230 0x7f21c019ecd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:42524/0 (socket says 172.21.15.39:42524) 2024-09-21T03:46:00.335 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:00.324+0000 7f21bed9d640 1 -- 172.21.15.39:0/4178237856 learned_addr learned my addr 172.21.15.39:0/4178237856 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:46:00.335 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:00.325+0000 7f21be59c640 1 --2- 172.21.15.39:0/4178237856 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f21c0102430 0x7f21c0199490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:46:00.335 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:00.325+0000 7f21bdd9b640 1 --2- 172.21.15.39:0/4178237856 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f21c01048e0 0x7f21c0199a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:46:00.335 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:00.325+0000 7f21bed9d640 1 -- 172.21.15.39:0/4178237856 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f21c01048e0 msgr2=0x7f21c0199a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:46:00.335 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:00.325+0000 7f21bed9d640 1 --2- 172.21.15.39:0/4178237856 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f21c01048e0 0x7f21c0199a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:00.335 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:00.325+0000 7f21bed9d640 1 -- 172.21.15.39:0/4178237856 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f21c0102430 msgr2=0x7f21c0199490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:46:00.335 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:00.325+0000 7f21bed9d640 1 --2- 172.21.15.39:0/4178237856 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f21c0102430 0x7f21c0199490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:00.335 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:00.325+0000 7f21bed9d640 1 -- 172.21.15.39:0/4178237856 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f21c019f340 con 0x7f21c0105230 2024-09-21T03:46:00.335 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:00.325+0000 7f21be59c640 1 --2- 172.21.15.39:0/4178237856 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f21c0102430 0x7f21c0199490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:46:00.335 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:00.325+0000 7f21bdd9b640 1 --2- 172.21.15.39:0/4178237856 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f21c01048e0 0x7f21c0199a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:46:00.335 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:00.325+0000 7f21bed9d640 1 --2- 172.21.15.39:0/4178237856 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f21c0105230 0x7f21c019ecd0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f21b800cce0 tx=0x7f21b800a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:46:00.335 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:00.325+0000 7f21af7fe640 1 -- 172.21.15.39:0/4178237856 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21b800f450 con 0x7f21c0105230 2024-09-21T03:46:00.336 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:00.325+0000 7f21c4cf8640 1 -- 172.21.15.39:0/4178237856 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f21c019f630 con 0x7f21c0105230 2024-09-21T03:46:00.336 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:00.325+0000 7f21af7fe640 1 -- 172.21.15.39:0/4178237856 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f21b800faa0 con 0x7f21c0105230 2024-09-21T03:46:00.336 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:00.325+0000 7f21af7fe640 1 -- 172.21.15.39:0/4178237856 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21b8018b70 con 0x7f21c0105230 2024-09-21T03:46:00.336 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:00.325+0000 7f21c4cf8640 1 -- 172.21.15.39:0/4178237856 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f21c019fb20 con 0x7f21c0105230 2024-09-21T03:46:00.336 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:00.328+0000 7f21cb26c740 1 Processor -- start 2024-09-21T03:46:00.336 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:00.328+0000 7f21af7fe640 1 -- 172.21.15.39:0/4178237856 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 23) ==== 154529+0+0 (secure 0 0 0) 0x7f21b8018d10 con 0x7f21c0105230 2024-09-21T03:46:00.336 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:00.328+0000 7f21cb26c740 1 -- start start 2024-09-21T03:46:00.336 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:00.329+0000 7f21cb26c740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56038ebc0f60 0x56038ef66f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:00.336 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:00.329+0000 7f21cb26c740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56038eb980a0 0x56038ef67440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:00.336 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:00.329+0000 7f21cb26c740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56038eba0280 0x56038ef67980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:00.336 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:00.329+0000 7f21cb26c740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x56038eedec10 con 0x56038eba0280 2024-09-21T03:46:00.336 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:00.329+0000 7f21cb26c740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x56038eece040 con 0x56038ebc0f60 2024-09-21T03:46:00.336 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:00.329+0000 7f21cb26c740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x56038eed1e50 con 0x56038eb980a0 2024-09-21T03:46:00.336 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:00.329+0000 7f21bdd9b640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56038eba0280 0x56038ef67980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:46:00.336 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:00.329+0000 7f21bdd9b640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56038eba0280 0x56038ef67980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:42540/0 (socket says 172.21.15.39:42540) 2024-09-21T03:46:00.337 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:00.329+0000 7f21bdd9b640 1 -- 172.21.15.39:0/872736309 learned_addr learned my addr 172.21.15.39:0/872736309 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:46:00.337 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:00.329+0000 7f21bdd9b640 1 --2- 172.21.15.39:0/872736309 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56038ebc0f60 0x56038ef66f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:46:00.337 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:00.329+0000 7f21be59c640 1 --2- 172.21.15.39:0/872736309 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56038eb980a0 0x56038ef67440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:46:00.337 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:00.329+0000 7f21bdd9b640 1 -- 172.21.15.39:0/872736309 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56038eb980a0 msgr2=0x56038ef67440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:46:00.337 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:00.329+0000 7f21bdd9b640 1 --2- 172.21.15.39:0/872736309 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56038eb980a0 0x56038ef67440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:00.337 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:00.329+0000 7f21bdd9b640 1 -- 172.21.15.39:0/872736309 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56038ebc0f60 msgr2=0x56038ef66f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:46:00.337 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:00.329+0000 7f21bdd9b640 1 --2- 172.21.15.39:0/872736309 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56038ebc0f60 0x56038ef66f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:00.337 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:00.329+0000 7f21bdd9b640 1 -- 172.21.15.39:0/872736309 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56038ec2c540 con 0x56038eba0280 2024-09-21T03:46:00.337 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:00.329+0000 7f21af7fe640 1 --2- 172.21.15.39:0/4178237856 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f21a80b6590 0x7f21a80b8aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:00.337 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:00.329+0000 7f21be59c640 1 --2- 172.21.15.39:0/4178237856 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f21a80b6590 0x7f21a80b8aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:46:00.337 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:00.329+0000 7f21bdd9b640 1 --2- 172.21.15.39:0/872736309 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56038eba0280 0x56038ef67980 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f21b400cfc0 tx=0x7f21b400c1d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6cf2e4dd4954f6e1 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:46:00.337 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:00.330+0000 7f21af7fe640 1 -- 172.21.15.39:0/4178237856 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8484+0+0 (secure 0 0 0) 0x7f21b80e7970 con 0x7f21c0105230 2024-09-21T03:46:00.337 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:00.330+0000 7f21acff9640 1 -- 172.21.15.39:0/872736309 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21b401d070 con 0x56038eba0280 2024-09-21T03:46:00.337 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:00.330+0000 7f21acff9640 1 -- 172.21.15.39:0/872736309 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f21b400c6b0 con 0x56038eba0280 2024-09-21T03:46:00.338 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:00.330+0000 7f21acff9640 1 -- 172.21.15.39:0/872736309 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21b400f040 con 0x56038eba0280 2024-09-21T03:46:00.341 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:00.331+0000 7f21cb26c740 1 -- 172.21.15.39:0/872736309 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56038eba0280 msgr2=0x56038ef67980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:46:00.341 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:00.331+0000 7f21be59c640 1 --2- 172.21.15.39:0/4178237856 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f21a80b6590 0x7f21a80b8aa0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x56038ef65b80 tx=0x7f21b0032b20 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:46:00.342 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:00.331+0000 7f21cb26c740 1 --2- 172.21.15.39:0/872736309 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56038eba0280 0x56038ef67980 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f21b400cfc0 tx=0x7f21b400c1d0 comp rx=0 tx=0).stop 2024-09-21T03:46:00.342 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:00.331+0000 7f21cb26c740 1 -- 172.21.15.39:0/872736309 shutdown_connections 2024-09-21T03:46:00.342 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:00.331+0000 7f21cb26c740 1 --2- 172.21.15.39:0/872736309 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56038eba0280 0x56038ef67980 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:00.342 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:00.331+0000 7f21cb26c740 1 --2- 172.21.15.39:0/872736309 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56038eb980a0 0x56038ef67440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:00.342 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:00.331+0000 7f21cb26c740 1 --2- 172.21.15.39:0/872736309 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56038ebc0f60 0x56038ef66f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:00.342 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:00.331+0000 7f21cb26c740 1 -- 172.21.15.39:0/872736309 >> 172.21.15.39:0/872736309 conn(0x56038ebb5350 msgr2=0x56038ef64df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:46:00.342 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:00.331+0000 7f21cb26c740 1 -- 172.21.15.39:0/872736309 shutdown_connections 2024-09-21T03:46:00.342 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:00.331+0000 7f21cb26c740 1 -- 172.21.15.39:0/872736309 wait complete. 2024-09-21T03:46:00.342 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:00.332+0000 7f21cb26c740 1 -- start start 2024-09-21T03:46:00.342 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:00.332+0000 7f21cb26c740 1 Processor -- start 2024-09-21T03:46:00.342 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:00.332+0000 7f21cb26c740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56038eb980a0 0x56038ef72a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:00.342 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:00.332+0000 7f21cb26c740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56038eba0280 0x56038ef72f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:00.342 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:00.332+0000 7f21cb26c740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56038ebc0f60 0x56038ef7ea30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:00.343 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:00.332+0000 7f21cb26c740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x56038ed2f5f0 con 0x56038eb980a0 2024-09-21T03:46:00.343 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:00.332+0000 7f21cb26c740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x56038ee69010 con 0x56038ebc0f60 2024-09-21T03:46:00.343 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:00.332+0000 7f21cb26c740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x56038ef756e0 con 0x56038eba0280 2024-09-21T03:46:00.343 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:00.332+0000 7f21cb26c740 10 client.0 _ll_register_callbacks cb 0x56038eec8340 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:46:00.343 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:00.332+0000 7f21be59c640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56038eb980a0 0x56038ef72a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:46:00.343 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:00.332+0000 7f21be59c640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56038eb980a0 0x56038ef72a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:42552/0 (socket says 172.21.15.39:42552) 2024-09-21T03:46:00.343 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:00.332+0000 7f21be59c640 1 -- 172.21.15.39:0/3573438189 learned_addr learned my addr 172.21.15.39:0/3573438189 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:46:00.343 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:00.332+0000 7f21bdd9b640 1 --2- 172.21.15.39:0/3573438189 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56038eba0280 0x56038ef72f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:46:00.343 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:00.332+0000 7f21bed9d640 1 --2- 172.21.15.39:0/3573438189 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56038ebc0f60 0x56038ef7ea30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:46:00.343 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:00.333+0000 7f21be59c640 1 -- 172.21.15.39:0/3573438189 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56038eba0280 msgr2=0x56038ef72f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:46:00.343 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:00.333+0000 7f21be59c640 1 --2- 172.21.15.39:0/3573438189 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56038eba0280 0x56038ef72f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:00.343 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:00.333+0000 7f21be59c640 1 -- 172.21.15.39:0/3573438189 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56038ebc0f60 msgr2=0x56038ef7ea30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:46:00.343 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:00.333+0000 7f21be59c640 1 --2- 172.21.15.39:0/3573438189 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56038ebc0f60 0x56038ef7ea30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:00.343 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:00.333+0000 7f21be59c640 1 -- 172.21.15.39:0/3573438189 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56038ee96a00 con 0x56038eb980a0 2024-09-21T03:46:00.344 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:00.333+0000 7f21bdd9b640 1 --2- 172.21.15.39:0/3573438189 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56038eba0280 0x56038ef72f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:46:00.344 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:00.333+0000 7f21be59c640 1 --2- 172.21.15.39:0/3573438189 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56038eb980a0 0x56038ef72a10 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f21b0033000 tx=0x7f21b002a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4656e978ce997514 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:46:00.344 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:00.333+0000 7f21a5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-09-21T03:46:00.344 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:00.333+0000 7f21a5ffb640 1 -- 172.21.15.39:0/3573438189 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21b0052060 con 0x56038eb980a0 2024-09-21T03:46:00.344 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:00.333+0000 7f21a5ffb640 1 -- 172.21.15.39:0/3573438189 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8484+0+0 (secure 0 0 0) 0x7f21b0003a00 con 0x56038eb980a0 2024-09-21T03:46:00.344 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:00.333+0000 7f21cb26c740 1 -- 172.21.15.39:0/3573438189 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f21b400c6b0 con 0x56038eb980a0 2024-09-21T03:46:00.344 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:00.333+0000 7f21a5ffb640 1 -- 172.21.15.39:0/3573438189 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f21b0035050 con 0x56038eb980a0 2024-09-21T03:46:00.344 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:00.333+0000 7f21cb26c740 10 client.15420 fetch_fsmap learned FSMap version 28 2024-09-21T03:46:00.344 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:00.333+0000 7f21cb26c740 1 -- 172.21.15.39:0/3573438189 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x56038ef816a0 con 0x56038eb980a0 2024-09-21T03:46:00.344 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:00.333+0000 7f21a5ffb640 1 -- 172.21.15.39:0/3573438189 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f21b400c6b0 con 0x56038eb980a0 2024-09-21T03:46:00.344 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:00.334+0000 7f21a5ffb640 1 -- 172.21.15.39:0/3573438189 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f21b004c030 con 0x56038eb980a0 2024-09-21T03:46:00.344 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:00.334+0000 7f21cb26c740 10 client.15420 fetch_fsmap finished waiting for FSMap version 28 2024-09-21T03:46:00.344 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:00.334+0000 7f21cb26c740 10 client.15420 resolve_mds: resolved name 'h' to daemon mds.h 2024-09-21T03:46:00.344 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:00.334+0000 7f21cb26c740 20 client.15420 populate_metadata read hostname 'smithi039' 2024-09-21T03:46:00.345 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:00.334+0000 7f21cb26c740 1 --2- 172.21.15.39:0/3573438189 >> [v2:172.21.15.150:6836/4017043144,v1:172.21.15.150:6837/4017043144] conn(0x56038eb7f8e0 0x56038ef85160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:00.345 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:00.334+0000 7f21cb26c740 4 client.15420 mds_command: new command op to 24659 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:46:00.345 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:00.334+0000 7f21cb26c740 1 -- 172.21.15.39:0/3573438189 --> [v2:172.21.15.150:6836/4017043144,v1:172.21.15.150:6837/4017043144] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56038eee0b60 con 0x56038eb7f8e0 2024-09-21T03:46:00.345 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:00.334+0000 7f21bdd9b640 1 --2- 172.21.15.39:0/3573438189 >> [v2:172.21.15.150:6836/4017043144,v1:172.21.15.150:6837/4017043144] conn(0x56038eb7f8e0 0x56038ef85160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:46:00.345 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:00.334+0000 7f21bdd9b640 1 --2- 172.21.15.39:0/3573438189 >> [v2:172.21.15.150:6836/4017043144,v1:172.21.15.150:6837/4017043144] conn(0x56038eb7f8e0 0x56038ef85160 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=c9c12d77ac1eaa7b server_cookie=c300484503138b75 in_seq=0 out_seq=0 2024-09-21T03:46:00.345 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:00.334+0000 7f21a5ffb640 10 client.15420 ms_handle_connect on v2:172.21.15.150:6836/4017043144 2024-09-21T03:46:00.345 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:00.335+0000 7f21a5ffb640 1 -- 172.21.15.39:0/3573438189 <== mds.2 v2:172.21.15.150:6836/4017043144 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x56038eee0b60 con 0x56038eb7f8e0 2024-09-21T03:46:00.345 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:00.335+0000 7f21a5ffb640 10 client.15420 handle_command_reply: tid=0 2024-09-21T03:46:00.345 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:00.335+0000 7f21cb26c740 1 client.15420 shutdown 2024-09-21T03:46:00.345 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:00.335+0000 7f21cb26c740 1 -- 172.21.15.39:0/3573438189 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56038eb980a0 msgr2=0x56038ef72a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:46:00.345 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:00.335+0000 7f21cb26c740 1 --2- 172.21.15.39:0/3573438189 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56038eb980a0 0x56038ef72a10 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f21b0033000 tx=0x7f21b002a410 comp rx=0 tx=0).stop 2024-09-21T03:46:00.345 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:00.335+0000 7f21cb26c740 1 -- 172.21.15.39:0/3573438189 shutdown_connections 2024-09-21T03:46:00.345 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:00.335+0000 7f21cb26c740 1 --2- 172.21.15.39:0/3573438189 >> [v2:172.21.15.150:6836/4017043144,v1:172.21.15.150:6837/4017043144] conn(0x56038eb7f8e0 0x56038ef85160 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:00.345 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:00.335+0000 7f21cb26c740 1 --2- 172.21.15.39:0/3573438189 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56038ebc0f60 0x56038ef7ea30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:00.346 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:00.335+0000 7f21cb26c740 1 --2- 172.21.15.39:0/3573438189 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56038eba0280 0x56038ef72f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:00.346 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:00.335+0000 7f21a5ffb640 0 client.15420 ms_handle_reset on v2:172.21.15.150:6836/4017043144 2024-09-21T03:46:00.346 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:00.335+0000 7f21cb26c740 1 --2- 172.21.15.39:0/3573438189 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56038eb980a0 0x56038ef72a10 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:00.346 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:00.336+0000 7f21cb26c740 1 -- 172.21.15.39:0/3573438189 >> 172.21.15.39:0/3573438189 conn(0x56038ebb5350 msgr2=0x56038ef6a090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:46:00.346 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:00.336+0000 7f21cb26c740 1 -- 172.21.15.39:0/3573438189 shutdown_connections 2024-09-21T03:46:00.346 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:00.336+0000 7f21cb26c740 1 -- 172.21.15.39:0/3573438189 wait complete. 2024-09-21T03:46:00.346 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:00.336+0000 7f21cb26c740 20 client.15420 trim_cache size 0 max 16384 2024-09-21T03:46:00.357 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:00.354+0000 7f21cb26c740 1 Processor -- start 2024-09-21T03:46:00.357 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:00.354+0000 7f21cb26c740 1 -- start start 2024-09-21T03:46:00.357 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:00.354+0000 7f21cb26c740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56038eb7f8e0 0x56038ef7e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:00.357 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:00.354+0000 7f21cb26c740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56038eb980a0 0x56038ef7eee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:00.357 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:00.354+0000 7f21cb26c740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56038eba0280 0x56038ef64600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:00.357 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:00.354+0000 7f21cb26c740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x56038ef834a0 con 0x56038eb980a0 2024-09-21T03:46:00.357 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:00.354+0000 7f21cb26c740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x56038ef83b70 con 0x56038eb7f8e0 2024-09-21T03:46:00.357 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:00.354+0000 7f21cb26c740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x56038ef84240 con 0x56038eba0280 2024-09-21T03:46:00.357 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:00.354+0000 7f21bdd9b640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56038eb980a0 0x56038ef7eee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:46:00.357 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:00.354+0000 7f21bdd9b640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56038eb980a0 0x56038ef7eee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:42562/0 (socket says 172.21.15.39:42562) 2024-09-21T03:46:00.358 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:00.354+0000 7f21bdd9b640 1 -- 172.21.15.39:0/3345637164 learned_addr learned my addr 172.21.15.39:0/3345637164 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:46:00.361 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:00.355+0000 7f21be59c640 1 --2- 172.21.15.39:0/3345637164 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56038eb7f8e0 0x56038ef7e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:46:00.361 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:00.355+0000 7f21bed9d640 1 --2- 172.21.15.39:0/3345637164 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56038eba0280 0x56038ef64600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:46:00.361 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:00.355+0000 7f21bdd9b640 1 -- 172.21.15.39:0/3345637164 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56038eba0280 msgr2=0x56038ef64600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:46:00.361 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:00.355+0000 7f21bdd9b640 1 --2- 172.21.15.39:0/3345637164 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56038eba0280 0x56038ef64600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:00.361 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:00.355+0000 7f21bdd9b640 1 -- 172.21.15.39:0/3345637164 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56038eb7f8e0 msgr2=0x56038ef7e9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:46:00.361 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:00.355+0000 7f21bdd9b640 1 --2- 172.21.15.39:0/3345637164 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56038eb7f8e0 0x56038ef7e9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:00.361 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:00.355+0000 7f21bdd9b640 1 -- 172.21.15.39:0/3345637164 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56038ef691f0 con 0x56038eb980a0 2024-09-21T03:46:00.361 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:00.355+0000 7f21bed9d640 1 --2- 172.21.15.39:0/3345637164 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56038eba0280 0x56038ef64600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:46:00.361 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:00.355+0000 7f21be59c640 1 --2- 172.21.15.39:0/3345637164 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56038eb7f8e0 0x56038ef7e9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:46:00.361 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:00.355+0000 7f21bdd9b640 1 --2- 172.21.15.39:0/3345637164 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56038eb980a0 0x56038ef7eee0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f21b40155f0 tx=0x7f21b400cd00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=73a6bf03c1f40208 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:46:00.361 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:00.355+0000 7f21a7fff640 1 -- 172.21.15.39:0/3345637164 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21b401d040 con 0x56038eb980a0 2024-09-21T03:46:00.361 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:00.355+0000 7f21a7fff640 1 -- 172.21.15.39:0/3345637164 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f21b4030070 con 0x56038eb980a0 2024-09-21T03:46:00.362 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:00.355+0000 7f21a7fff640 1 -- 172.21.15.39:0/3345637164 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21b4013dc0 con 0x56038eb980a0 2024-09-21T03:46:00.362 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:00.356+0000 7f21cb26c740 1 -- 172.21.15.39:0/3345637164 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56038eb980a0 msgr2=0x56038ef7eee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:46:00.362 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:00.356+0000 7f21cb26c740 1 --2- 172.21.15.39:0/3345637164 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56038eb980a0 0x56038ef7eee0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f21b40155f0 tx=0x7f21b400cd00 comp rx=0 tx=0).stop 2024-09-21T03:46:00.362 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:00.356+0000 7f21cb26c740 1 -- 172.21.15.39:0/3345637164 shutdown_connections 2024-09-21T03:46:00.362 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:00.356+0000 7f21cb26c740 1 --2- 172.21.15.39:0/3345637164 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56038eba0280 0x56038ef64600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:00.362 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:00.356+0000 7f21cb26c740 1 --2- 172.21.15.39:0/3345637164 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56038eb980a0 0x56038ef7eee0 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:00.362 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:00.356+0000 7f21cb26c740 1 --2- 172.21.15.39:0/3345637164 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56038eb7f8e0 0x56038ef7e9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:00.362 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:00.356+0000 7f21cb26c740 1 -- 172.21.15.39:0/3345637164 >> 172.21.15.39:0/3345637164 conn(0x56038ebb5350 msgr2=0x56038ef71090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:46:00.362 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:00.356+0000 7f21cb26c740 1 -- 172.21.15.39:0/3345637164 shutdown_connections 2024-09-21T03:46:00.362 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:00.356+0000 7f21cb26c740 1 -- 172.21.15.39:0/3345637164 wait complete. 2024-09-21T03:46:00.362 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:00.356+0000 7f21cb26c740 1 -- start start 2024-09-21T03:46:00.362 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:00.356+0000 7f21cb26c740 1 Processor -- start 2024-09-21T03:46:00.362 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:00.357+0000 7f21cb26c740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56038eb7f8e0 0x56038ef8ddc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:00.362 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:00.357+0000 7f21cb26c740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56038eb980a0 0x56038ef87270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:00.363 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:00.357+0000 7f21cb26c740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56038eba0280 0x56038ef87980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:00.363 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:00.357+0000 7f21cb26c740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x56038ef84fe0 con 0x56038eb7f8e0 2024-09-21T03:46:00.363 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:00.357+0000 7f21cb26c740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x56038ef84910 con 0x56038eba0280 2024-09-21T03:46:00.363 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:00.357+0000 7f21cb26c740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x56038ef856b0 con 0x56038eb980a0 2024-09-21T03:46:00.363 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:00.357+0000 7f21cb26c740 10 client.0 _ll_register_callbacks cb 0x7f21b003f070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:46:00.363 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:00.357+0000 7f21be59c640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56038eb7f8e0 0x56038ef8ddc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:46:00.363 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:00.357+0000 7f21be59c640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56038eb7f8e0 0x56038ef8ddc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:42578/0 (socket says 172.21.15.39:42578) 2024-09-21T03:46:00.363 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:00.357+0000 7f21be59c640 1 -- 172.21.15.39:0/1433729957 learned_addr learned my addr 172.21.15.39:0/1433729957 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:46:00.363 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:00.357+0000 7f21bdd9b640 1 --2- 172.21.15.39:0/1433729957 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56038eb980a0 0x56038ef87270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:46:00.363 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:00.357+0000 7f21bed9d640 1 --2- 172.21.15.39:0/1433729957 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56038eba0280 0x56038ef87980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:46:00.363 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:00.357+0000 7f21be59c640 1 -- 172.21.15.39:0/1433729957 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56038eb980a0 msgr2=0x56038ef87270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:46:00.363 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:00.357+0000 7f21be59c640 1 --2- 172.21.15.39:0/1433729957 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56038eb980a0 0x56038ef87270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:00.363 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:00.357+0000 7f21be59c640 1 -- 172.21.15.39:0/1433729957 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56038eba0280 msgr2=0x56038ef87980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:46:00.363 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:00.357+0000 7f21be59c640 1 --2- 172.21.15.39:0/1433729957 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56038eba0280 0x56038ef87980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:00.364 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:00.357+0000 7f21be59c640 1 -- 172.21.15.39:0/1433729957 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56038ef8ca00 con 0x56038eb7f8e0 2024-09-21T03:46:00.364 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:00.357+0000 7f21be59c640 1 --2- 172.21.15.39:0/1433729957 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56038eb7f8e0 0x56038ef8ddc0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f21b0009900 tx=0x7f21b00314a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c5b7e48041613a server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:46:00.364 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:00.357+0000 7f21a5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-09-21T03:46:00.364 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:00.357+0000 7f21a5ffb640 1 -- 172.21.15.39:0/1433729957 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21b0052040 con 0x56038eb7f8e0 2024-09-21T03:46:00.364 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:00.357+0000 7f21a5ffb640 1 -- 172.21.15.39:0/1433729957 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8484+0+0 (secure 0 0 0) 0x7f21b0003a00 con 0x56038eb7f8e0 2024-09-21T03:46:00.364 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:00.357+0000 7f21cb26c740 1 -- 172.21.15.39:0/1433729957 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f21b4030070 con 0x56038eb7f8e0 2024-09-21T03:46:00.364 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:00.358+0000 7f21a5ffb640 1 -- 172.21.15.39:0/1433729957 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f21b0067050 con 0x56038eb7f8e0 2024-09-21T03:46:00.364 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:00.358+0000 7f21cb26c740 10 client.15426 fetch_fsmap learned FSMap version 28 2024-09-21T03:46:00.364 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:00.358+0000 7f21cb26c740 1 -- 172.21.15.39:0/1433729957 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x56038eff0630 con 0x56038eb7f8e0 2024-09-21T03:46:00.364 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:00.358+0000 7f21a5ffb640 1 -- 172.21.15.39:0/1433729957 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f21b4030070 con 0x56038eb7f8e0 2024-09-21T03:46:00.364 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:00.358+0000 7f21a5ffb640 1 -- 172.21.15.39:0/1433729957 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f21b005e070 con 0x56038eb7f8e0 2024-09-21T03:46:00.364 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:00.358+0000 7f21cb26c740 10 client.15426 fetch_fsmap finished waiting for FSMap version 28 2024-09-21T03:46:00.364 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:00.358+0000 7f21cb26c740 10 client.15426 resolve_mds: resolved name 'h' to daemon mds.h 2024-09-21T03:46:00.364 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:00.358+0000 7f21cb26c740 20 client.15426 populate_metadata read hostname 'smithi039' 2024-09-21T03:46:00.364 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:00.358+0000 7f21cb26c740 1 --2- 172.21.15.39:0/1433729957 >> [v2:172.21.15.150:6836/4017043144,v1:172.21.15.150:6837/4017043144] conn(0x56038ebc0f60 0x56038eff3f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:00.365 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:00.358+0000 7f21cb26c740 4 client.15426 mds_command: new command op to 24659 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-21T03:46:00.365 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:00.358+0000 7f21cb26c740 1 -- 172.21.15.39:0/1433729957 --> [v2:172.21.15.150:6836/4017043144,v1:172.21.15.150:6837/4017043144] -- command(tid 0: {"prefix": "status"}) -- 0x56038eedc610 con 0x56038ebc0f60 2024-09-21T03:46:00.366 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:00.359+0000 7f21bdd9b640 1 --2- 172.21.15.39:0/1433729957 >> [v2:172.21.15.150:6836/4017043144,v1:172.21.15.150:6837/4017043144] conn(0x56038ebc0f60 0x56038eff3f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:46:00.366 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:00.359+0000 7f21bdd9b640 1 --2- 172.21.15.39:0/1433729957 >> [v2:172.21.15.150:6836/4017043144,v1:172.21.15.150:6837/4017043144] conn(0x56038ebc0f60 0x56038eff3f30 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=65b8ce55f9035772 server_cookie=3c0d06ab862e70f8 in_seq=0 out_seq=0 2024-09-21T03:46:00.366 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:00.359+0000 7f21a5ffb640 10 client.15426 ms_handle_connect on v2:172.21.15.150:6836/4017043144 2024-09-21T03:46:00.366 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:00.359+0000 7f21a5ffb640 1 -- 172.21.15.39:0/1433729957 <== mds.2 v2:172.21.15.150:6836/4017043144 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x56038eedc610 con 0x56038ebc0f60 2024-09-21T03:46:00.366 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:00.359+0000 7f21a5ffb640 10 client.15426 handle_command_reply: tid=0 2024-09-21T03:46:00.366 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:00.359+0000 7f21cb26c740 1 client.15426 shutdown 2024-09-21T03:46:00.366 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:00.359+0000 7f21cb26c740 1 -- 172.21.15.39:0/1433729957 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56038eb7f8e0 msgr2=0x56038ef8ddc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:46:00.367 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:00.359+0000 7f21cb26c740 1 --2- 172.21.15.39:0/1433729957 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56038eb7f8e0 0x56038ef8ddc0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f21b0009900 tx=0x7f21b00314a0 comp rx=0 tx=0).stop 2024-09-21T03:46:00.367 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:00.359+0000 7f21cb26c740 1 -- 172.21.15.39:0/1433729957 shutdown_connections 2024-09-21T03:46:00.367 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:00.359+0000 7f21cb26c740 1 --2- 172.21.15.39:0/1433729957 >> [v2:172.21.15.150:6836/4017043144,v1:172.21.15.150:6837/4017043144] conn(0x56038ebc0f60 0x56038eff3f30 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:00.367 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:00.359+0000 7f21cb26c740 1 --2- 172.21.15.39:0/1433729957 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56038eba0280 0x56038ef87980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:00.367 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:00.359+0000 7f21cb26c740 1 --2- 172.21.15.39:0/1433729957 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56038eb980a0 0x56038ef87270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:00.367 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:00.359+0000 7f21cb26c740 1 --2- 172.21.15.39:0/1433729957 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56038eb7f8e0 0x56038ef8ddc0 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:00.367 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:00.359+0000 7f21cb26c740 1 -- 172.21.15.39:0/1433729957 >> 172.21.15.39:0/1433729957 conn(0x56038ebb5350 msgr2=0x56038ef70cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:46:00.367 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:00.359+0000 7f21a5ffb640 0 client.15426 ms_handle_reset on v2:172.21.15.150:6836/4017043144 2024-09-21T03:46:00.367 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:00.360+0000 7f21cb26c740 1 -- 172.21.15.39:0/1433729957 shutdown_connections 2024-09-21T03:46:00.367 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:00.360+0000 7f21cb26c740 1 -- 172.21.15.39:0/1433729957 wait complete. 2024-09-21T03:46:00.367 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:00.360+0000 7f21cb26c740 20 client.15426 trim_cache size 0 max 16384 2024-09-21T03:46:00.367 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-09-21T03:46:00.368 INFO:teuthology.orchestra.run.smithi039.stdout: "cluster_fsid": "51116d5c-77ca-11ef-baf5-efdc52797490", 2024-09-21T03:46:00.368 INFO:teuthology.orchestra.run.smithi039.stdout: "whoami": 2, 2024-09-21T03:46:00.368 INFO:teuthology.orchestra.run.smithi039.stdout: "id": 24659, 2024-09-21T03:46:00.368 INFO:teuthology.orchestra.run.smithi039.stdout: "want_state": "up:active", 2024-09-21T03:46:00.368 INFO:teuthology.orchestra.run.smithi039.stdout: "state": "up:active", 2024-09-21T03:46:00.368 INFO:teuthology.orchestra.run.smithi039.stdout: "fs_name": "cephfs", 2024-09-21T03:46:00.368 INFO:teuthology.orchestra.run.smithi039.stdout: "rank_uptime": 96.983563029999999, 2024-09-21T03:46:00.368 INFO:teuthology.orchestra.run.smithi039.stdout: "mdsmap_epoch": 28, 2024-09-21T03:46:00.368 INFO:teuthology.orchestra.run.smithi039.stdout: "osdmap_epoch": 79, 2024-09-21T03:46:00.368 INFO:teuthology.orchestra.run.smithi039.stdout: "osdmap_epoch_barrier": 78, 2024-09-21T03:46:00.368 INFO:teuthology.orchestra.run.smithi039.stdout: "uptime": 126.71653205699999 2024-09-21T03:46:00.368 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-09-21T03:46:00.369 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:00.360+0000 7f21c4cf8640 1 -- 172.21.15.39:0/4178237856 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f21a80b6590 msgr2=0x7f21a80b8aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:46:00.369 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:00.360+0000 7f21c4cf8640 1 --2- 172.21.15.39:0/4178237856 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f21a80b6590 0x7f21a80b8aa0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x56038ef65b80 tx=0x7f21b0032b20 comp rx=0 tx=0).stop 2024-09-21T03:46:00.369 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:00.360+0000 7f21c4cf8640 1 -- 172.21.15.39:0/4178237856 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f21c0105230 msgr2=0x7f21c019ecd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:46:00.369 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:00.360+0000 7f21c4cf8640 1 --2- 172.21.15.39:0/4178237856 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f21c0105230 0x7f21c019ecd0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f21b800cce0 tx=0x7f21b800a620 comp rx=0 tx=0).stop 2024-09-21T03:46:00.369 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:00.360+0000 7f21c4cf8640 1 -- 172.21.15.39:0/4178237856 shutdown_connections 2024-09-21T03:46:00.369 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:00.360+0000 7f21c4cf8640 1 --2- 172.21.15.39:0/4178237856 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f21a80b6590 0x7f21a80b8aa0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:00.369 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:00.360+0000 7f21c4cf8640 1 --2- 172.21.15.39:0/4178237856 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f21c0105230 0x7f21c019ecd0 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:00.370 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:00.360+0000 7f21c4cf8640 1 --2- 172.21.15.39:0/4178237856 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f21c01048e0 0x7f21c0199a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:00.370 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:00.360+0000 7f21c4cf8640 1 --2- 172.21.15.39:0/4178237856 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f21c0102430 0x7f21c0199490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:00.370 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:00.360+0000 7f21c4cf8640 1 -- 172.21.15.39:0/4178237856 >> 172.21.15.39:0/4178237856 conn(0x7f21c0077800 msgr2=0x7f21c00ff9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:46:00.370 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:00.360+0000 7f21c4cf8640 1 -- 172.21.15.39:0/4178237856 shutdown_connections 2024-09-21T03:46:00.370 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:00.360+0000 7f21c4cf8640 1 -- 172.21.15.39:0/4178237856 wait complete. 2024-09-21T03:46:01.083 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:46:00 smithi039 ceph-mon[31211]: pgmap v324: 97 pgs: 97 active+clean; 647 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 597 B/s wr, 12 op/s 2024-09-21T03:46:01.169 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 51116d5c-77ca-11ef-baf5-efdc52797490 -- ceph tell mds.i status 2024-09-21T03:46:01.217 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:46:00 smithi161 ceph-mon[34533]: pgmap v324: 97 pgs: 97 active+clean; 647 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 597 B/s wr, 12 op/s 2024-09-21T03:46:01.224 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:46:00 smithi150 ceph-mon[34289]: pgmap v324: 97 pgs: 97 active+clean; 647 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 597 B/s wr, 12 op/s 2024-09-21T03:46:01.429 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/51116d5c-77ca-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:46:01.953 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:01.950+0000 7fa332dc1640 1 -- 172.21.15.39:0/1371303398 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa32c1083c0 msgr2=0x7fa32c1087d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:46:01.953 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:01.950+0000 7fa332dc1640 1 --2- 172.21.15.39:0/1371303398 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa32c1083c0 0x7fa32c1087d0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7fa320009900 tx=0x7fa32002a410 comp rx=0 tx=0).stop 2024-09-21T03:46:01.953 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:01.950+0000 7fa332dc1640 1 -- 172.21.15.39:0/1371303398 shutdown_connections 2024-09-21T03:46:01.953 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:01.950+0000 7fa332dc1640 1 --2- 172.21.15.39:0/1371303398 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa32c108d10 0x7fa32c10f540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:01.953 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:01.950+0000 7fa332dc1640 1 --2- 172.21.15.39:0/1371303398 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa32c1083c0 0x7fa32c1087d0 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:01.953 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:01.950+0000 7fa332dc1640 1 --2- 172.21.15.39:0/1371303398 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa32c102a80 0x7fa32c102e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:01.954 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:01.950+0000 7fa332dc1640 1 -- 172.21.15.39:0/1371303398 >> 172.21.15.39:0/1371303398 conn(0x7fa32c079740 msgr2=0x7fa32c079b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:46:01.954 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:01.950+0000 7fa332dc1640 1 -- 172.21.15.39:0/1371303398 shutdown_connections 2024-09-21T03:46:01.954 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:01.950+0000 7fa332dc1640 1 -- 172.21.15.39:0/1371303398 wait complete. 2024-09-21T03:46:01.954 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:01.950+0000 7fa332dc1640 1 Processor -- start 2024-09-21T03:46:01.954 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:01.951+0000 7fa332dc1640 1 -- start start 2024-09-21T03:46:01.955 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:01.951+0000 7fa332dc1640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa32c102a80 0x7fa32c19a560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:01.955 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:01.951+0000 7fa332dc1640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa32c1083c0 0x7fa32c19ab00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:01.955 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:01.951+0000 7fa332dc1640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa32c108d10 0x7fa32c19fda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:01.955 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:01.951+0000 7fa332dc1640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fa32c1129c0 con 0x7fa32c102a80 2024-09-21T03:46:01.956 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:01.951+0000 7fa332dc1640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fa32c112840 con 0x7fa32c1083c0 2024-09-21T03:46:01.956 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:01.951+0000 7fa332dc1640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fa32c112b40 con 0x7fa32c108d10 2024-09-21T03:46:01.956 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:01.951+0000 7fa330b5d640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa32c102a80 0x7fa32c19a560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:46:01.956 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:01.951+0000 7fa330b5d640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa32c102a80 0x7fa32c19a560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:42606/0 (socket says 172.21.15.39:42606) 2024-09-21T03:46:01.956 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:01.951+0000 7fa330b5d640 1 -- 172.21.15.39:0/728645390 learned_addr learned my addr 172.21.15.39:0/728645390 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:46:01.956 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:01.951+0000 7fa32bfff640 1 --2- 172.21.15.39:0/728645390 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa32c1083c0 0x7fa32c19ab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:46:01.956 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:01.951+0000 7fa33135e640 1 --2- 172.21.15.39:0/728645390 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa32c108d10 0x7fa32c19fda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:46:01.956 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:01.951+0000 7fa330b5d640 1 -- 172.21.15.39:0/728645390 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa32c108d10 msgr2=0x7fa32c19fda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:46:01.956 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:01.951+0000 7fa330b5d640 1 --2- 172.21.15.39:0/728645390 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa32c108d10 0x7fa32c19fda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:01.956 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:01.951+0000 7fa330b5d640 1 -- 172.21.15.39:0/728645390 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa32c1083c0 msgr2=0x7fa32c19ab00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:46:01.956 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:01.951+0000 7fa330b5d640 1 --2- 172.21.15.39:0/728645390 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa32c1083c0 0x7fa32c19ab00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:01.956 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:01.951+0000 7fa330b5d640 1 -- 172.21.15.39:0/728645390 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa32c1a0410 con 0x7fa32c102a80 2024-09-21T03:46:01.956 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:01.952+0000 7fa330b5d640 1 --2- 172.21.15.39:0/728645390 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa32c102a80 0x7fa32c19a560 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7fa31c00b4a0 tx=0x7fa31c00b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:46:01.956 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:01.952+0000 7fa329ffb640 1 -- 172.21.15.39:0/728645390 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa31c014070 con 0x7fa32c102a80 2024-09-21T03:46:01.957 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:01.952+0000 7fa329ffb640 1 -- 172.21.15.39:0/728645390 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa31c010ac0 con 0x7fa32c102a80 2024-09-21T03:46:01.957 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:01.952+0000 7fa332dc1640 1 -- 172.21.15.39:0/728645390 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa32c1a0700 con 0x7fa32c102a80 2024-09-21T03:46:01.957 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:01.952+0000 7fa329ffb640 1 -- 172.21.15.39:0/728645390 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa31c017c30 con 0x7fa32c102a80 2024-09-21T03:46:01.957 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:01.952+0000 7fa332dc1640 1 -- 172.21.15.39:0/728645390 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa32c1a6f20 con 0x7fa32c102a80 2024-09-21T03:46:01.958 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:01.954+0000 7fa339335740 1 Processor -- start 2024-09-21T03:46:01.958 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:01.955+0000 7fa339335740 1 -- start start 2024-09-21T03:46:01.958 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:01.955+0000 7fa329ffb640 1 -- 172.21.15.39:0/728645390 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 23) ==== 154529+0+0 (secure 0 0 0) 0x7fa31c017070 con 0x7fa32c102a80 2024-09-21T03:46:01.958 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:01.955+0000 7fa339335740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56511176ef60 0x565111b14f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:01.958 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:01.955+0000 7fa339335740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5651117460a0 0x565111b15440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:01.959 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:01.955+0000 7fa339335740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56511174e280 0x565111b15980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:01.959 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:01.955+0000 7fa339335740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x565111a8cc10 con 0x56511174e280 2024-09-21T03:46:01.959 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:01.955+0000 7fa339335740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x565111a7c040 con 0x5651117460a0 2024-09-21T03:46:01.959 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:01.955+0000 7fa339335740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x565111a7fe50 con 0x56511176ef60 2024-09-21T03:46:01.959 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:01.955+0000 7fa32bfff640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56511174e280 0x565111b15980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:46:01.959 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:01.955+0000 7fa32bfff640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56511174e280 0x565111b15980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:42608/0 (socket says 172.21.15.39:42608) 2024-09-21T03:46:01.959 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:01.955+0000 7fa32bfff640 1 -- 172.21.15.39:0/3199798902 learned_addr learned my addr 172.21.15.39:0/3199798902 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:46:01.959 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:01.955+0000 7fa330b5d640 1 --2- 172.21.15.39:0/3199798902 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5651117460a0 0x565111b15440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:46:01.959 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:01.955+0000 7fa33135e640 1 --2- 172.21.15.39:0/3199798902 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56511176ef60 0x565111b14f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:46:01.959 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:01.955+0000 7fa32bfff640 1 -- 172.21.15.39:0/3199798902 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56511176ef60 msgr2=0x565111b14f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:46:01.959 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:01.955+0000 7fa32bfff640 1 --2- 172.21.15.39:0/3199798902 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56511176ef60 0x565111b14f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:01.959 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:01.955+0000 7fa32bfff640 1 -- 172.21.15.39:0/3199798902 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5651117460a0 msgr2=0x565111b15440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:46:01.959 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:01.955+0000 7fa32bfff640 1 --2- 172.21.15.39:0/3199798902 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5651117460a0 0x565111b15440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:01.959 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:01.955+0000 7fa32bfff640 1 -- 172.21.15.39:0/3199798902 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5651117da540 con 0x56511174e280 2024-09-21T03:46:01.960 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:01.956+0000 7fa32bfff640 1 --2- 172.21.15.39:0/3199798902 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56511174e280 0x565111b15980 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7fa32c06b6c0 tx=0x7fa320031cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=595f8d16bad2efec server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:46:01.963 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:01.956+0000 7fa312ffd640 1 -- 172.21.15.39:0/3199798902 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa32003f070 con 0x56511174e280 2024-09-21T03:46:01.963 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:01.956+0000 7fa329ffb640 1 --2- 172.21.15.39:0/728645390 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7fa3140b6610 0x7fa3140b8b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:01.963 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:01.956+0000 7fa312ffd640 1 -- 172.21.15.39:0/3199798902 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa3200316a0 con 0x56511174e280 2024-09-21T03:46:01.963 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:01.956+0000 7fa312ffd640 1 -- 172.21.15.39:0/3199798902 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa320035070 con 0x56511174e280 2024-09-21T03:46:01.963 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:01.956+0000 7fa33135e640 1 --2- 172.21.15.39:0/728645390 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7fa3140b6610 0x7fa3140b8b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:46:01.964 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:01.956+0000 7fa329ffb640 1 -- 172.21.15.39:0/728645390 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8484+0+0 (secure 0 0 0) 0x7fa31c0e66f0 con 0x7fa32c102a80 2024-09-21T03:46:01.964 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:01.957+0000 7fa33135e640 1 --2- 172.21.15.39:0/728645390 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7fa3140b6610 0x7fa3140b8b20 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fa324007ed0 tx=0x7fa324007680 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:46:01.964 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:01.957+0000 7fa339335740 1 -- 172.21.15.39:0/3199798902 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56511174e280 msgr2=0x565111b15980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:46:01.964 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:01.957+0000 7fa339335740 1 --2- 172.21.15.39:0/3199798902 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56511174e280 0x565111b15980 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7fa32c06b6c0 tx=0x7fa320031cd0 comp rx=0 tx=0).stop 2024-09-21T03:46:01.964 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:01.957+0000 7fa339335740 1 -- 172.21.15.39:0/3199798902 shutdown_connections 2024-09-21T03:46:01.964 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:01.957+0000 7fa339335740 1 --2- 172.21.15.39:0/3199798902 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56511174e280 0x565111b15980 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:01.964 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:01.957+0000 7fa339335740 1 --2- 172.21.15.39:0/3199798902 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5651117460a0 0x565111b15440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:01.964 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:01.957+0000 7fa339335740 1 --2- 172.21.15.39:0/3199798902 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56511176ef60 0x565111b14f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:01.964 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:01.957+0000 7fa339335740 1 -- 172.21.15.39:0/3199798902 >> 172.21.15.39:0/3199798902 conn(0x565111763350 msgr2=0x565111b12df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:46:01.964 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:01.957+0000 7fa339335740 1 -- 172.21.15.39:0/3199798902 shutdown_connections 2024-09-21T03:46:01.964 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:01.957+0000 7fa339335740 1 -- 172.21.15.39:0/3199798902 wait complete. 2024-09-21T03:46:01.964 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:01.957+0000 7fa339335740 1 -- start start 2024-09-21T03:46:01.964 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:01.958+0000 7fa339335740 1 Processor -- start 2024-09-21T03:46:01.964 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:01.958+0000 7fa339335740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5651117460a0 0x565111b24b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:01.965 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:01.958+0000 7fa339335740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56511174e280 0x565111b250b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:01.965 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:01.958+0000 7fa339335740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56511176ef60 0x565111b30d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:01.965 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:01.958+0000 7fa339335740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x5651118dd5f0 con 0x56511176ef60 2024-09-21T03:46:01.965 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:01.958+0000 7fa339335740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x565111a17010 con 0x56511174e280 2024-09-21T03:46:01.965 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:01.958+0000 7fa339335740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x565111b279e0 con 0x5651117460a0 2024-09-21T03:46:01.965 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:01.958+0000 7fa339335740 10 client.0 _ll_register_callbacks cb 0x565111a76340 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:46:01.965 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:01.958+0000 7fa33135e640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56511176ef60 0x565111b30d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:46:01.965 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:01.958+0000 7fa33135e640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56511176ef60 0x565111b30d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:42614/0 (socket says 172.21.15.39:42614) 2024-09-21T03:46:01.965 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:01.958+0000 7fa33135e640 1 -- 172.21.15.39:0/2101982324 learned_addr learned my addr 172.21.15.39:0/2101982324 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:46:01.965 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:01.958+0000 7fa330b5d640 1 --2- 172.21.15.39:0/2101982324 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5651117460a0 0x565111b24b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:46:01.965 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:01.958+0000 7fa32bfff640 1 --2- 172.21.15.39:0/2101982324 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56511174e280 0x565111b250b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:46:01.965 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:01.959+0000 7fa33135e640 1 -- 172.21.15.39:0/2101982324 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5651117460a0 msgr2=0x565111b24b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:46:01.965 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:01.959+0000 7fa33135e640 1 --2- 172.21.15.39:0/2101982324 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5651117460a0 0x565111b24b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:01.965 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:01.959+0000 7fa33135e640 1 -- 172.21.15.39:0/2101982324 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56511174e280 msgr2=0x565111b250b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:46:01.965 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:01.959+0000 7fa33135e640 1 --2- 172.21.15.39:0/2101982324 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56511174e280 0x565111b250b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:01.966 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:01.959+0000 7fa33135e640 1 -- 172.21.15.39:0/2101982324 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x565111a44a00 con 0x56511176ef60 2024-09-21T03:46:01.966 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:01.959+0000 7fa33135e640 1 --2- 172.21.15.39:0/2101982324 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56511176ef60 0x565111b30d40 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7fa32400ed30 tx=0x7fa324004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=268db353a7ba3b7b server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:46:01.966 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:01.959+0000 7fa30bfff640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-09-21T03:46:01.966 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:01.959+0000 7fa30bfff640 1 -- 172.21.15.39:0/2101982324 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa32401cd30 con 0x56511176ef60 2024-09-21T03:46:01.966 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:01.959+0000 7fa30bfff640 1 -- 172.21.15.39:0/2101982324 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8484+0+0 (secure 0 0 0) 0x7fa32401d410 con 0x56511176ef60 2024-09-21T03:46:01.966 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:01.959+0000 7fa339335740 1 -- 172.21.15.39:0/2101982324 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa3200316a0 con 0x56511176ef60 2024-09-21T03:46:01.966 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:01.959+0000 7fa30bfff640 1 -- 172.21.15.39:0/2101982324 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fa32401d610 con 0x56511176ef60 2024-09-21T03:46:01.966 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:01.959+0000 7fa339335740 10 client.15438 fetch_fsmap learned FSMap version 28 2024-09-21T03:46:01.966 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:01.959+0000 7fa339335740 1 -- 172.21.15.39:0/2101982324 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x565111b33910 con 0x56511176ef60 2024-09-21T03:46:01.966 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:01.959+0000 7fa30bfff640 1 -- 172.21.15.39:0/2101982324 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa3200316a0 con 0x56511176ef60 2024-09-21T03:46:01.966 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:01.960+0000 7fa30bfff640 1 -- 172.21.15.39:0/2101982324 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fa32402f020 con 0x56511176ef60 2024-09-21T03:46:01.966 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:01.960+0000 7fa339335740 10 client.15438 fetch_fsmap finished waiting for FSMap version 28 2024-09-21T03:46:01.966 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:01.960+0000 7fa339335740 10 client.15438 resolve_mds: resolved name 'i' to daemon mds.i 2024-09-21T03:46:01.967 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:01.960+0000 7fa339335740 20 client.15438 populate_metadata read hostname 'smithi039' 2024-09-21T03:46:01.967 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:01.960+0000 7fa339335740 1 --2- 172.21.15.39:0/2101982324 >> [v2:172.21.15.161:6838/2190107270,v1:172.21.15.161:6839/2190107270] conn(0x56511172d8e0 0x565111b37830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:01.967 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:01.960+0000 7fa339335740 4 client.15438 mds_command: new command op to 24667 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:46:01.967 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:01.960+0000 7fa339335740 1 -- 172.21.15.39:0/2101982324 --> [v2:172.21.15.161:6838/2190107270,v1:172.21.15.161:6839/2190107270] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x565111a8eb60 con 0x56511172d8e0 2024-09-21T03:46:01.967 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:01.960+0000 7fa330b5d640 1 --2- 172.21.15.39:0/2101982324 >> [v2:172.21.15.161:6838/2190107270,v1:172.21.15.161:6839/2190107270] conn(0x56511172d8e0 0x565111b37830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:46:01.967 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:01.961+0000 7fa330b5d640 1 --2- 172.21.15.39:0/2101982324 >> [v2:172.21.15.161:6838/2190107270,v1:172.21.15.161:6839/2190107270] conn(0x56511172d8e0 0x565111b37830 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=f839c1b64ed63302 server_cookie=256fa72a74d1bb7c in_seq=0 out_seq=0 2024-09-21T03:46:01.967 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:01.961+0000 7fa30bfff640 10 client.15438 ms_handle_connect on v2:172.21.15.161:6838/2190107270 2024-09-21T03:46:01.968 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:01.961+0000 7fa30bfff640 1 -- 172.21.15.39:0/2101982324 <== mds.1 v2:172.21.15.161:6838/2190107270 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x565111a8eb60 con 0x56511172d8e0 2024-09-21T03:46:01.968 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:01.961+0000 7fa30bfff640 10 client.15438 handle_command_reply: tid=0 2024-09-21T03:46:01.968 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:01.961+0000 7fa339335740 1 client.15438 shutdown 2024-09-21T03:46:01.968 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:01.962+0000 7fa339335740 1 -- 172.21.15.39:0/2101982324 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56511176ef60 msgr2=0x565111b30d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:46:01.968 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:01.962+0000 7fa339335740 1 --2- 172.21.15.39:0/2101982324 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56511176ef60 0x565111b30d40 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7fa32400ed30 tx=0x7fa324004a40 comp rx=0 tx=0).stop 2024-09-21T03:46:01.968 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:01.962+0000 7fa339335740 1 -- 172.21.15.39:0/2101982324 shutdown_connections 2024-09-21T03:46:01.968 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:01.962+0000 7fa339335740 1 --2- 172.21.15.39:0/2101982324 >> [v2:172.21.15.161:6838/2190107270,v1:172.21.15.161:6839/2190107270] conn(0x56511172d8e0 0x565111b37830 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:01.969 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:01.962+0000 7fa339335740 1 --2- 172.21.15.39:0/2101982324 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56511176ef60 0x565111b30d40 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:01.969 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:01.962+0000 7fa339335740 1 --2- 172.21.15.39:0/2101982324 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56511174e280 0x565111b250b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:01.969 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:01.962+0000 7fa339335740 1 --2- 172.21.15.39:0/2101982324 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5651117460a0 0x565111b24b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:01.969 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:01.962+0000 7fa339335740 1 -- 172.21.15.39:0/2101982324 >> 172.21.15.39:0/2101982324 conn(0x565111763350 msgr2=0x565111b168f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:46:01.969 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:01.962+0000 7fa30bfff640 0 client.15438 ms_handle_reset on v2:172.21.15.161:6838/2190107270 2024-09-21T03:46:01.969 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:01.962+0000 7fa339335740 1 -- 172.21.15.39:0/2101982324 shutdown_connections 2024-09-21T03:46:01.969 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:01.962+0000 7fa339335740 1 -- 172.21.15.39:0/2101982324 wait complete. 2024-09-21T03:46:01.969 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:01.962+0000 7fa339335740 20 client.15438 trim_cache size 0 max 16384 2024-09-21T03:46:01.983 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:01.980+0000 7fa339335740 1 Processor -- start 2024-09-21T03:46:01.984 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:01.980+0000 7fa339335740 1 -- start start 2024-09-21T03:46:01.984 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:01.981+0000 7fa339335740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5651117460a0 0x565111b30cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:01.984 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:01.981+0000 7fa339335740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56511174e280 0x565111b311f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:01.984 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:01.981+0000 7fa339335740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56511176ef60 0x565111b233b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:01.984 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:01.981+0000 7fa339335740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x565111b35b70 con 0x56511176ef60 2024-09-21T03:46:01.984 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:01.981+0000 7fa339335740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x565111b36240 con 0x56511174e280 2024-09-21T03:46:01.984 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:01.981+0000 7fa339335740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x565111b36910 con 0x5651117460a0 2024-09-21T03:46:01.984 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:01.981+0000 7fa33135e640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56511176ef60 0x565111b233b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:46:01.984 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:01.981+0000 7fa33135e640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56511176ef60 0x565111b233b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:42616/0 (socket says 172.21.15.39:42616) 2024-09-21T03:46:01.984 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:01.981+0000 7fa33135e640 1 -- 172.21.15.39:0/1279492693 learned_addr learned my addr 172.21.15.39:0/1279492693 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:46:01.984 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:01.981+0000 7fa32bfff640 1 --2- 172.21.15.39:0/1279492693 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56511174e280 0x565111b311f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:46:01.984 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:01.981+0000 7fa330b5d640 1 --2- 172.21.15.39:0/1279492693 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5651117460a0 0x565111b30cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:46:01.984 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:01.981+0000 7fa33135e640 1 -- 172.21.15.39:0/1279492693 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5651117460a0 msgr2=0x565111b30cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:46:01.984 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:01.981+0000 7fa33135e640 1 --2- 172.21.15.39:0/1279492693 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5651117460a0 0x565111b30cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:01.985 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:01.981+0000 7fa33135e640 1 -- 172.21.15.39:0/1279492693 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56511174e280 msgr2=0x565111b311f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:46:01.985 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:01.981+0000 7fa33135e640 1 --2- 172.21.15.39:0/1279492693 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56511174e280 0x565111b311f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:01.985 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:01.981+0000 7fa33135e640 1 -- 172.21.15.39:0/1279492693 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x565111b203c0 con 0x56511176ef60 2024-09-21T03:46:01.987 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:01.981+0000 7fa33135e640 1 --2- 172.21.15.39:0/1279492693 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56511176ef60 0x565111b233b0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7fa324011d40 tx=0x7fa32400ea90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a3a89754142e6f58 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:46:01.987 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:01.982+0000 7fa3127fc640 1 -- 172.21.15.39:0/1279492693 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa32401c770 con 0x56511176ef60 2024-09-21T03:46:01.987 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:01.982+0000 7fa3127fc640 1 -- 172.21.15.39:0/1279492693 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa32401d410 con 0x56511176ef60 2024-09-21T03:46:01.987 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:01.982+0000 7fa3127fc640 1 -- 172.21.15.39:0/1279492693 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa324004dc0 con 0x56511176ef60 2024-09-21T03:46:01.987 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:01.982+0000 7fa339335740 1 -- 172.21.15.39:0/1279492693 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56511176ef60 msgr2=0x565111b233b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:46:01.988 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:01.982+0000 7fa339335740 1 --2- 172.21.15.39:0/1279492693 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56511176ef60 0x565111b233b0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7fa324011d40 tx=0x7fa32400ea90 comp rx=0 tx=0).stop 2024-09-21T03:46:01.988 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:01.983+0000 7fa339335740 1 -- 172.21.15.39:0/1279492693 shutdown_connections 2024-09-21T03:46:01.988 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:01.983+0000 7fa339335740 1 --2- 172.21.15.39:0/1279492693 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56511176ef60 0x565111b233b0 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:01.988 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:01.983+0000 7fa339335740 1 --2- 172.21.15.39:0/1279492693 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56511174e280 0x565111b311f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:01.988 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:01.983+0000 7fa339335740 1 --2- 172.21.15.39:0/1279492693 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5651117460a0 0x565111b30cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:01.988 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:01.983+0000 7fa339335740 1 -- 172.21.15.39:0/1279492693 >> 172.21.15.39:0/1279492693 conn(0x565111763350 msgr2=0x565111b1a030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:46:01.988 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:01.983+0000 7fa339335740 1 -- 172.21.15.39:0/1279492693 shutdown_connections 2024-09-21T03:46:01.988 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:01.983+0000 7fa339335740 1 -- 172.21.15.39:0/1279492693 wait complete. 2024-09-21T03:46:01.988 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:01.983+0000 7fa339335740 1 -- start start 2024-09-21T03:46:01.988 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:01.983+0000 7fa339335740 1 Processor -- start 2024-09-21T03:46:01.988 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:01.983+0000 7fa339335740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5651117460a0 0x565111b39ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:01.988 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:01.984+0000 7fa339335740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56511174e280 0x565111b3a010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:01.988 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:01.984+0000 7fa339335740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56511176ef60 0x565111b3a550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:01.988 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:01.984+0000 7fa339335740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x565111b376b0 con 0x56511174e280 2024-09-21T03:46:01.989 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:01.984+0000 7fa339335740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x565111b36fe0 con 0x56511176ef60 2024-09-21T03:46:01.989 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:01.984+0000 7fa339335740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x565111b37d80 con 0x5651117460a0 2024-09-21T03:46:01.989 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:01.984+0000 7fa339335740 10 client.0 _ll_register_callbacks cb 0x7fa324007b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:46:01.989 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:01.984+0000 7fa32bfff640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56511174e280 0x565111b3a010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:46:01.989 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:01.984+0000 7fa32bfff640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56511174e280 0x565111b3a010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:42626/0 (socket says 172.21.15.39:42626) 2024-09-21T03:46:01.989 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:01.984+0000 7fa32bfff640 1 -- 172.21.15.39:0/2735120583 learned_addr learned my addr 172.21.15.39:0/2735120583 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:46:01.989 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:01.984+0000 7fa330b5d640 1 --2- 172.21.15.39:0/2735120583 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5651117460a0 0x565111b39ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:46:01.989 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:01.984+0000 7fa33135e640 1 --2- 172.21.15.39:0/2735120583 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56511176ef60 0x565111b3a550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:46:01.989 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:01.984+0000 7fa330b5d640 1 -- 172.21.15.39:0/2735120583 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56511176ef60 msgr2=0x565111b3a550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:46:01.989 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:01.984+0000 7fa330b5d640 1 --2- 172.21.15.39:0/2735120583 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56511176ef60 0x565111b3a550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:01.989 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:01.984+0000 7fa330b5d640 1 -- 172.21.15.39:0/2735120583 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56511174e280 msgr2=0x565111b3a010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:46:01.989 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:01.984+0000 7fa330b5d640 1 --2- 172.21.15.39:0/2735120583 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56511174e280 0x565111b3a010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:01.989 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:01.984+0000 7fa330b5d640 1 -- 172.21.15.39:0/2735120583 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) -- 0x565111b3f290 con 0x5651117460a0 2024-09-21T03:46:01.989 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:01.984+0000 7fa32bfff640 1 --2- 172.21.15.39:0/2735120583 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56511174e280 0x565111b3a010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:46:01.990 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:01.984+0000 7fa330b5d640 1 --2- 172.21.15.39:0/2735120583 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5651117460a0 0x565111b39ad0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fa31c0e4140 tx=0x7fa31c0e68f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=acdb6037b9a8f50a server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:46:01.990 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:01.984+0000 7fa30bfff640 10 client.0 ms_handle_connect on v2:172.21.15.161:3300/0 2024-09-21T03:46:01.990 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:01.984+0000 7fa30bfff640 1 -- 172.21.15.39:0/2735120583 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa31c01e040 con 0x5651117460a0 2024-09-21T03:46:01.990 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:01.985+0000 7fa339335740 1 -- 172.21.15.39:0/2735120583 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa32401d410 con 0x5651117460a0 2024-09-21T03:46:01.990 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:01.985+0000 7fa30bfff640 1 -- 172.21.15.39:0/2735120583 <== mon.1 v2:172.21.15.161:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8484+0+0 (secure 0 0 0) 0x7fa31c0a89f0 con 0x5651117460a0 2024-09-21T03:46:01.990 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:01.985+0000 7fa30bfff640 1 -- 172.21.15.39:0/2735120583 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fa31c017360 con 0x5651117460a0 2024-09-21T03:46:01.990 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:01.985+0000 7fa339335740 10 client.25117 fetch_fsmap learned FSMap version 28 2024-09-21T03:46:01.990 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:01.985+0000 7fa339335740 1 -- 172.21.15.39:0/2735120583 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x565111ba2ff0 con 0x5651117460a0 2024-09-21T03:46:01.991 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:01.985+0000 7fa30bfff640 1 -- 172.21.15.39:0/2735120583 <== mon.1 v2:172.21.15.161:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa32401d410 con 0x5651117460a0 2024-09-21T03:46:01.992 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:01.985+0000 7fa30bfff640 1 -- 172.21.15.39:0/2735120583 <== mon.1 v2:172.21.15.161:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fa31c0e9ce0 con 0x5651117460a0 2024-09-21T03:46:01.992 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:01.985+0000 7fa339335740 10 client.25117 fetch_fsmap finished waiting for FSMap version 28 2024-09-21T03:46:01.992 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:01.985+0000 7fa339335740 10 client.25117 resolve_mds: resolved name 'i' to daemon mds.i 2024-09-21T03:46:01.992 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:01.985+0000 7fa339335740 20 client.25117 populate_metadata read hostname 'smithi039' 2024-09-21T03:46:01.992 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:01.985+0000 7fa339335740 1 --2- 172.21.15.39:0/2735120583 >> [v2:172.21.15.161:6838/2190107270,v1:172.21.15.161:6839/2190107270] conn(0x565111702c90 0x565111ba68e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:01.992 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:01.985+0000 7fa339335740 4 client.25117 mds_command: new command op to 24667 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-21T03:46:01.992 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:01.985+0000 7fa339335740 1 -- 172.21.15.39:0/2735120583 --> [v2:172.21.15.161:6838/2190107270,v1:172.21.15.161:6839/2190107270] -- command(tid 0: {"prefix": "status"}) -- 0x565111a8a610 con 0x565111702c90 2024-09-21T03:46:01.992 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:01.986+0000 7fa32bfff640 1 --2- 172.21.15.39:0/2735120583 >> [v2:172.21.15.161:6838/2190107270,v1:172.21.15.161:6839/2190107270] conn(0x565111702c90 0x565111ba68e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:46:01.992 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:01.986+0000 7fa32bfff640 1 --2- 172.21.15.39:0/2735120583 >> [v2:172.21.15.161:6838/2190107270,v1:172.21.15.161:6839/2190107270] conn(0x565111702c90 0x565111ba68e0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=805a83b9eb944f52 server_cookie=2c57e36dcac976ef in_seq=0 out_seq=0 2024-09-21T03:46:01.992 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:01.986+0000 7fa30bfff640 10 client.25117 ms_handle_connect on v2:172.21.15.161:6838/2190107270 2024-09-21T03:46:01.992 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:01.986+0000 7fa30bfff640 1 -- 172.21.15.39:0/2735120583 <== mds.1 v2:172.21.15.161:6838/2190107270 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x565111a8a610 con 0x565111702c90 2024-09-21T03:46:01.992 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:01.986+0000 7fa30bfff640 10 client.25117 handle_command_reply: tid=0 2024-09-21T03:46:01.992 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:01.986+0000 7fa339335740 1 client.25117 shutdown 2024-09-21T03:46:01.993 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:01.986+0000 7fa339335740 1 -- 172.21.15.39:0/2735120583 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5651117460a0 msgr2=0x565111b39ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:46:01.993 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:01.986+0000 7fa339335740 1 --2- 172.21.15.39:0/2735120583 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5651117460a0 0x565111b39ad0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fa31c0e4140 tx=0x7fa31c0e68f0 comp rx=0 tx=0).stop 2024-09-21T03:46:01.993 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:01.987+0000 7fa339335740 1 -- 172.21.15.39:0/2735120583 shutdown_connections 2024-09-21T03:46:01.993 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:01.987+0000 7fa339335740 1 --2- 172.21.15.39:0/2735120583 >> [v2:172.21.15.161:6838/2190107270,v1:172.21.15.161:6839/2190107270] conn(0x565111702c90 0x565111ba68e0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:01.993 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:01.987+0000 7fa339335740 1 --2- 172.21.15.39:0/2735120583 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56511176ef60 0x565111b3a550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:01.993 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:01.987+0000 7fa339335740 1 --2- 172.21.15.39:0/2735120583 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56511174e280 0x565111b3a010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:01.993 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:01.987+0000 7fa339335740 1 --2- 172.21.15.39:0/2735120583 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5651117460a0 0x565111b39ad0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:01.993 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:01.987+0000 7fa30bfff640 0 client.25117 ms_handle_reset on v2:172.21.15.161:6838/2190107270 2024-09-21T03:46:01.993 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:01.987+0000 7fa339335740 1 -- 172.21.15.39:0/2735120583 >> 172.21.15.39:0/2735120583 conn(0x565111763350 msgr2=0x565111b24aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:46:01.993 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:01.987+0000 7fa339335740 1 -- 172.21.15.39:0/2735120583 shutdown_connections 2024-09-21T03:46:01.993 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:01.987+0000 7fa339335740 1 -- 172.21.15.39:0/2735120583 wait complete. 2024-09-21T03:46:01.993 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:01.987+0000 7fa339335740 20 client.25117 trim_cache size 0 max 16384 2024-09-21T03:46:01.993 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-09-21T03:46:01.994 INFO:teuthology.orchestra.run.smithi039.stdout: "cluster_fsid": "51116d5c-77ca-11ef-baf5-efdc52797490", 2024-09-21T03:46:01.994 INFO:teuthology.orchestra.run.smithi039.stdout: "whoami": 1, 2024-09-21T03:46:01.994 INFO:teuthology.orchestra.run.smithi039.stdout: "id": 24667, 2024-09-21T03:46:01.994 INFO:teuthology.orchestra.run.smithi039.stdout: "want_state": "up:active", 2024-09-21T03:46:01.994 INFO:teuthology.orchestra.run.smithi039.stdout: "state": "up:active", 2024-09-21T03:46:01.994 INFO:teuthology.orchestra.run.smithi039.stdout: "fs_name": "cephfs", 2024-09-21T03:46:01.994 INFO:teuthology.orchestra.run.smithi039.stdout: "rank_uptime": 99.620514236000005, 2024-09-21T03:46:01.994 INFO:teuthology.orchestra.run.smithi039.stdout: "mdsmap_epoch": 28, 2024-09-21T03:46:01.994 INFO:teuthology.orchestra.run.smithi039.stdout: "osdmap_epoch": 79, 2024-09-21T03:46:01.994 INFO:teuthology.orchestra.run.smithi039.stdout: "osdmap_epoch_barrier": 78, 2024-09-21T03:46:01.994 INFO:teuthology.orchestra.run.smithi039.stdout: "uptime": 125.754454778 2024-09-21T03:46:01.994 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-09-21T03:46:01.995 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:01.987+0000 7fa332dc1640 1 -- 172.21.15.39:0/728645390 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7fa3140b6610 msgr2=0x7fa3140b8b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:46:01.995 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:01.987+0000 7fa332dc1640 1 --2- 172.21.15.39:0/728645390 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7fa3140b6610 0x7fa3140b8b20 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fa324007ed0 tx=0x7fa324007680 comp rx=0 tx=0).stop 2024-09-21T03:46:01.995 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:01.987+0000 7fa332dc1640 1 -- 172.21.15.39:0/728645390 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa32c102a80 msgr2=0x7fa32c19a560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:46:01.995 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:01.987+0000 7fa332dc1640 1 --2- 172.21.15.39:0/728645390 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa32c102a80 0x7fa32c19a560 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7fa31c00b4a0 tx=0x7fa31c00b960 comp rx=0 tx=0).stop 2024-09-21T03:46:01.995 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:01.988+0000 7fa332dc1640 1 -- 172.21.15.39:0/728645390 shutdown_connections 2024-09-21T03:46:01.995 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:01.988+0000 7fa332dc1640 1 --2- 172.21.15.39:0/728645390 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7fa3140b6610 0x7fa3140b8b20 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:01.995 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:01.988+0000 7fa332dc1640 1 --2- 172.21.15.39:0/728645390 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa32c108d10 0x7fa32c19fda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:01.995 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:01.988+0000 7fa332dc1640 1 --2- 172.21.15.39:0/728645390 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa32c1083c0 0x7fa32c19ab00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:01.996 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:01.988+0000 7fa332dc1640 1 --2- 172.21.15.39:0/728645390 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa32c102a80 0x7fa32c19a560 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:01.996 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:01.988+0000 7fa332dc1640 1 -- 172.21.15.39:0/728645390 >> 172.21.15.39:0/728645390 conn(0x7fa32c079740 msgr2=0x7fa32c06bf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:46:01.996 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:01.988+0000 7fa332dc1640 1 -- 172.21.15.39:0/728645390 shutdown_connections 2024-09-21T03:46:01.996 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:01.988+0000 7fa332dc1640 1 -- 172.21.15.39:0/728645390 wait complete. 2024-09-21T03:46:02.736 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 51116d5c-77ca-11ef-baf5-efdc52797490 -- ceph tell mds.b status 2024-09-21T03:46:02.992 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/51116d5c-77ca-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:46:03.217 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:46:02 smithi161 ceph-mon[34533]: pgmap v325: 97 pgs: 97 active+clean; 647 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 767 B/s wr, 10 op/s 2024-09-21T03:46:03.225 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:46:02 smithi150 ceph-mon[34289]: pgmap v325: 97 pgs: 97 active+clean; 647 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 767 B/s wr, 10 op/s 2024-09-21T03:46:03.227 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:46:02 smithi039 ceph-mon[31211]: pgmap v325: 97 pgs: 97 active+clean; 647 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 767 B/s wr, 10 op/s 2024-09-21T03:46:03.427 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:03.417+0000 7fead4493640 1 -- 172.21.15.39:0/1020504445 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7feacc102a60 msgr2=0x7feacc102e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:46:03.427 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:03.417+0000 7fead4493640 1 --2- 172.21.15.39:0/1020504445 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7feacc102a60 0x7feacc102e70 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7feac8009900 tx=0x7feac802a410 comp rx=0 tx=0).stop 2024-09-21T03:46:03.428 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:03.417+0000 7fead4493640 1 -- 172.21.15.39:0/1020504445 shutdown_connections 2024-09-21T03:46:03.428 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:03.417+0000 7fead4493640 1 --2- 172.21.15.39:0/1020504445 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7feacc108cf0 0x7feacc10f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:03.428 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:03.417+0000 7fead4493640 1 --2- 172.21.15.39:0/1020504445 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7feacc1083a0 0x7feacc1087b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:03.428 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:03.417+0000 7fead4493640 1 --2- 172.21.15.39:0/1020504445 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7feacc102a60 0x7feacc102e70 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:03.428 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:03.417+0000 7fead4493640 1 -- 172.21.15.39:0/1020504445 >> 172.21.15.39:0/1020504445 conn(0x7feacc079730 msgr2=0x7feacc079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:46:03.428 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:03.417+0000 7fead4493640 1 -- 172.21.15.39:0/1020504445 shutdown_connections 2024-09-21T03:46:03.428 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:03.418+0000 7fead4493640 1 -- 172.21.15.39:0/1020504445 wait complete. 2024-09-21T03:46:03.428 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:03.418+0000 7fead4493640 1 Processor -- start 2024-09-21T03:46:03.428 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:03.418+0000 7fead4493640 1 -- start start 2024-09-21T03:46:03.429 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:03.419+0000 7fead4493640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7feacc102a60 0x7feacc19a3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:03.429 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:03.419+0000 7fead4493640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7feacc1083a0 0x7feacc19a980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:03.429 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:03.419+0000 7fead4493640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7feacc108cf0 0x7feacc19fc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:03.429 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:03.419+0000 7fead4493640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7feacc112960 con 0x7feacc108cf0 2024-09-21T03:46:03.429 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:03.419+0000 7fead4493640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7feacc1127e0 con 0x7feacc102a60 2024-09-21T03:46:03.429 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:03.419+0000 7fead4493640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7feacc112ae0 con 0x7feacc1083a0 2024-09-21T03:46:03.429 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:03.419+0000 7fead2a30640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7feacc108cf0 0x7feacc19fc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:46:03.429 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:03.419+0000 7fead2a30640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7feacc108cf0 0x7feacc19fc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:42638/0 (socket says 172.21.15.39:42638) 2024-09-21T03:46:03.430 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:03.419+0000 7fead2a30640 1 -- 172.21.15.39:0/2502335289 learned_addr learned my addr 172.21.15.39:0/2502335289 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:46:03.430 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:03.419+0000 7fead1a2e640 1 --2- 172.21.15.39:0/2502335289 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7feacc1083a0 0x7feacc19a980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:46:03.430 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:03.419+0000 7fead222f640 1 --2- 172.21.15.39:0/2502335289 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7feacc102a60 0x7feacc19a3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:46:03.431 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:03.419+0000 7fead2a30640 1 -- 172.21.15.39:0/2502335289 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7feacc1083a0 msgr2=0x7feacc19a980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:46:03.431 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:03.419+0000 7fead2a30640 1 --2- 172.21.15.39:0/2502335289 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7feacc1083a0 0x7feacc19a980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:03.431 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:03.419+0000 7fead2a30640 1 -- 172.21.15.39:0/2502335289 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7feacc102a60 msgr2=0x7feacc19a3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:46:03.431 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:03.419+0000 7fead2a30640 1 --2- 172.21.15.39:0/2502335289 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7feacc102a60 0x7feacc19a3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:03.431 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:03.419+0000 7fead2a30640 1 -- 172.21.15.39:0/2502335289 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feacc1a0350 con 0x7feacc108cf0 2024-09-21T03:46:03.431 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:03.419+0000 7fead2a30640 1 --2- 172.21.15.39:0/2502335289 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7feacc108cf0 0x7feacc19fc20 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7feac400eaa0 tx=0x7feac400ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:46:03.431 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:03.419+0000 7feac37fe640 1 -- 172.21.15.39:0/2502335289 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feac400cdb0 con 0x7feacc108cf0 2024-09-21T03:46:03.431 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:03.419+0000 7feac37fe640 1 -- 172.21.15.39:0/2502335289 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7feac4005020 con 0x7feacc108cf0 2024-09-21T03:46:03.431 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:03.419+0000 7fead4493640 1 -- 172.21.15.39:0/2502335289 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feacc1a0640 con 0x7feacc108cf0 2024-09-21T03:46:03.431 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:03.420+0000 7feac37fe640 1 -- 172.21.15.39:0/2502335289 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feac40116d0 con 0x7feacc108cf0 2024-09-21T03:46:03.431 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:03.420+0000 7fead4493640 1 -- 172.21.15.39:0/2502335289 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feacc1a6da0 con 0x7feacc108cf0 2024-09-21T03:46:03.432 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:03.422+0000 7feadaa07740 1 Processor -- start 2024-09-21T03:46:03.432 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:03.422+0000 7feac37fe640 1 -- 172.21.15.39:0/2502335289 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 23) ==== 154529+0+0 (secure 0 0 0) 0x7feac40051d0 con 0x7feacc108cf0 2024-09-21T03:46:03.432 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:03.422+0000 7feadaa07740 1 -- start start 2024-09-21T03:46:03.432 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:03.423+0000 7feadaa07740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5642e3abbf60 0x5642e3e61f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:03.432 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:03.423+0000 7feadaa07740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5642e3a930a0 0x5642e3e62440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:03.432 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:03.423+0000 7feadaa07740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5642e3a9b280 0x5642e3e62980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:03.432 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:03.423+0000 7feadaa07740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x5642e3dd9c10 con 0x5642e3abbf60 2024-09-21T03:46:03.432 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:03.423+0000 7feadaa07740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x5642e3dc9040 con 0x5642e3a9b280 2024-09-21T03:46:03.432 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:03.423+0000 7feadaa07740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x5642e3dcce50 con 0x5642e3a930a0 2024-09-21T03:46:03.432 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:03.423+0000 7fead222f640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5642e3a930a0 0x5642e3e62440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:46:03.432 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:03.423+0000 7fead1a2e640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5642e3abbf60 0x5642e3e61f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:46:03.432 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:03.423+0000 7fead222f640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5642e3a930a0 0x5642e3e62440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.39:36880/0 (socket says 172.21.15.39:36880) 2024-09-21T03:46:03.432 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:03.423+0000 7fead222f640 1 -- 172.21.15.39:0/69540221 learned_addr learned my addr 172.21.15.39:0/69540221 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:46:03.432 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:03.423+0000 7fead1a2e640 1 --2- 172.21.15.39:0/69540221 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5642e3a9b280 0x5642e3e62980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:46:03.433 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:03.423+0000 7feac37fe640 1 --2- 172.21.15.39:0/2502335289 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7feab80b65e0 0x7feab80b8af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:03.433 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:03.423+0000 7fead222f640 1 -- 172.21.15.39:0/69540221 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5642e3a9b280 msgr2=0x5642e3e62980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:46:03.433 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:03.423+0000 7fead222f640 1 --2- 172.21.15.39:0/69540221 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5642e3a9b280 0x5642e3e62980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:03.433 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:03.423+0000 7fead222f640 1 -- 172.21.15.39:0/69540221 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5642e3abbf60 msgr2=0x5642e3e61f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:46:03.433 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:03.423+0000 7fead222f640 1 --2- 172.21.15.39:0/69540221 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5642e3abbf60 0x5642e3e61f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:03.433 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:03.423+0000 7fead222f640 1 -- 172.21.15.39:0/69540221 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5642e3b27540 con 0x5642e3a930a0 2024-09-21T03:46:03.433 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:03.424+0000 7fead2a30640 1 --2- 172.21.15.39:0/2502335289 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7feab80b65e0 0x7feab80b8af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:46:03.433 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:03.424+0000 7fead222f640 1 --2- 172.21.15.39:0/69540221 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5642e3a930a0 0x5642e3e62440 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7feac8009900 tx=0x7feac8032c40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e954356c89b1b856 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:46:03.433 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:03.424+0000 7feac37fe640 1 -- 172.21.15.39:0/2502335289 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8484+0+0 (secure 0 0 0) 0x7feac40e7510 con 0x7feacc108cf0 2024-09-21T03:46:03.433 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:03.424+0000 7feac0d9a640 1 -- 172.21.15.39:0/69540221 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feac8035040 con 0x5642e3a930a0 2024-09-21T03:46:03.433 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:03.424+0000 7feac0d9a640 1 -- 172.21.15.39:0/69540221 <== mon.1 v2:172.21.15.161:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7feac8004030 con 0x5642e3a930a0 2024-09-21T03:46:03.433 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:03.424+0000 7feac0d9a640 1 -- 172.21.15.39:0/69540221 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feac8005460 con 0x5642e3a930a0 2024-09-21T03:46:03.437 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:03.425+0000 7feadaa07740 1 -- 172.21.15.39:0/69540221 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5642e3a930a0 msgr2=0x5642e3e62440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:46:03.437 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:03.425+0000 7fead2a30640 1 --2- 172.21.15.39:0/2502335289 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7feab80b65e0 0x7feab80b8af0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7feac40e5280 tx=0x7feac40770a0 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:46:03.437 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:03.425+0000 7feadaa07740 1 --2- 172.21.15.39:0/69540221 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5642e3a930a0 0x5642e3e62440 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7feac8009900 tx=0x7feac8032c40 comp rx=0 tx=0).stop 2024-09-21T03:46:03.437 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:03.425+0000 7feadaa07740 1 -- 172.21.15.39:0/69540221 shutdown_connections 2024-09-21T03:46:03.437 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:03.425+0000 7feadaa07740 1 --2- 172.21.15.39:0/69540221 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5642e3a9b280 0x5642e3e62980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:03.437 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:03.425+0000 7feadaa07740 1 --2- 172.21.15.39:0/69540221 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5642e3a930a0 0x5642e3e62440 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:03.437 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:03.425+0000 7feadaa07740 1 --2- 172.21.15.39:0/69540221 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5642e3abbf60 0x5642e3e61f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:03.437 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:03.425+0000 7feadaa07740 1 -- 172.21.15.39:0/69540221 >> 172.21.15.39:0/69540221 conn(0x5642e3ab0350 msgr2=0x5642e3e5fdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:46:03.437 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:03.425+0000 7feadaa07740 1 -- 172.21.15.39:0/69540221 shutdown_connections 2024-09-21T03:46:03.437 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:03.425+0000 7feadaa07740 1 -- 172.21.15.39:0/69540221 wait complete. 2024-09-21T03:46:03.437 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:03.426+0000 7feadaa07740 1 -- start start 2024-09-21T03:46:03.437 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:03.426+0000 7feadaa07740 1 Processor -- start 2024-09-21T03:46:03.437 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:03.426+0000 7feadaa07740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5642e3a930a0 0x5642e3e6d7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:03.438 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:03.426+0000 7feadaa07740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5642e3a9b280 0x5642e3e6ddb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:03.438 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:03.426+0000 7feadaa07740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5642e3abbf60 0x5642e3e799a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:03.438 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:03.426+0000 7feadaa07740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x5642e3c2a5f0 con 0x5642e3a9b280 2024-09-21T03:46:03.438 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:03.426+0000 7feadaa07740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x5642e3d64010 con 0x5642e3abbf60 2024-09-21T03:46:03.438 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:03.426+0000 7feadaa07740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x5642e3e70680 con 0x5642e3a930a0 2024-09-21T03:46:03.438 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:03.426+0000 7feadaa07740 10 client.0 _ll_register_callbacks cb 0x5642e3dc3340 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:46:03.438 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:03.426+0000 7fead222f640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5642e3a9b280 0x5642e3e6ddb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:46:03.438 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:03.426+0000 7fead222f640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5642e3a9b280 0x5642e3e6ddb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:42646/0 (socket says 172.21.15.39:42646) 2024-09-21T03:46:03.438 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:03.426+0000 7fead222f640 1 -- 172.21.15.39:0/1084968078 learned_addr learned my addr 172.21.15.39:0/1084968078 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:46:03.438 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:03.427+0000 7fead1a2e640 1 --2- 172.21.15.39:0/1084968078 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5642e3a930a0 0x5642e3e6d7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:46:03.438 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:03.427+0000 7fead1a2e640 1 --2- 172.21.15.39:0/1084968078 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5642e3abbf60 0x5642e3e799a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:46:03.438 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:03.427+0000 7fead222f640 1 -- 172.21.15.39:0/1084968078 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5642e3a930a0 msgr2=0x5642e3e6d7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:46:03.438 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:03.427+0000 7fead222f640 1 --2- 172.21.15.39:0/1084968078 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5642e3a930a0 0x5642e3e6d7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:03.438 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:03.427+0000 7fead222f640 1 -- 172.21.15.39:0/1084968078 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5642e3abbf60 msgr2=0x5642e3e799a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:46:03.439 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:03.427+0000 7fead222f640 1 --2- 172.21.15.39:0/1084968078 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5642e3abbf60 0x5642e3e799a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:03.439 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:03.427+0000 7fead222f640 1 -- 172.21.15.39:0/1084968078 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5642e3d91a00 con 0x5642e3a9b280 2024-09-21T03:46:03.439 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:03.427+0000 7fead1a2e640 1 --2- 172.21.15.39:0/1084968078 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5642e3abbf60 0x5642e3e799a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:46:03.439 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:03.427+0000 7fead222f640 1 --2- 172.21.15.39:0/1084968078 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5642e3a9b280 0x5642e3e6ddb0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7feac8000c00 tx=0x7feac802abb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb0fbf7c818dd2bd server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:46:03.439 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:03.427+0000 7feab5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-09-21T03:46:03.439 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:03.427+0000 7feab5ffb640 1 -- 172.21.15.39:0/1084968078 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feac8035050 con 0x5642e3a9b280 2024-09-21T03:46:03.439 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:03.427+0000 7feadaa07740 1 -- 172.21.15.39:0/1084968078 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7feac8004030 con 0x5642e3a9b280 2024-09-21T03:46:03.439 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:03.427+0000 7feab5ffb640 1 -- 172.21.15.39:0/1084968078 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8484+0+0 (secure 0 0 0) 0x7feac80043f0 con 0x5642e3a9b280 2024-09-21T03:46:03.439 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:03.427+0000 7feab5ffb640 1 -- 172.21.15.39:0/1084968078 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7feac8005260 con 0x5642e3a9b280 2024-09-21T03:46:03.439 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:03.428+0000 7feadaa07740 10 client.15456 fetch_fsmap learned FSMap version 28 2024-09-21T03:46:03.439 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:03.428+0000 7feadaa07740 1 -- 172.21.15.39:0/1084968078 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5642e3e7c570 con 0x5642e3a9b280 2024-09-21T03:46:03.439 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:03.428+0000 7feab5ffb640 1 -- 172.21.15.39:0/1084968078 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7feac8004030 con 0x5642e3a9b280 2024-09-21T03:46:03.439 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:03.428+0000 7feab5ffb640 1 -- 172.21.15.39:0/1084968078 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7feac8059070 con 0x5642e3a9b280 2024-09-21T03:46:03.440 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:03.428+0000 7feadaa07740 10 client.15456 fetch_fsmap finished waiting for FSMap version 28 2024-09-21T03:46:03.440 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:03.428+0000 7feadaa07740 10 client.15456 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-21T03:46:03.440 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:03.428+0000 7feadaa07740 20 client.15456 populate_metadata read hostname 'smithi039' 2024-09-21T03:46:03.440 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:03.428+0000 7feadaa07740 1 --2- 172.21.15.39:0/1084968078 >> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] conn(0x5642e3a7a8e0 0x5642e3e7fb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:03.440 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:03.428+0000 7feadaa07740 4 client.15456 mds_command: new command op to 24695 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:46:03.440 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:03.428+0000 7feadaa07740 1 -- 172.21.15.39:0/1084968078 --> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5642e3ddbb60 con 0x5642e3a7a8e0 2024-09-21T03:46:03.440 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:03.428+0000 7fead1a2e640 1 --2- 172.21.15.39:0/1084968078 >> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] conn(0x5642e3a7a8e0 0x5642e3e7fb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:46:03.440 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:03.429+0000 7fead1a2e640 1 --2- 172.21.15.39:0/1084968078 >> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] conn(0x5642e3a7a8e0 0x5642e3e7fb10 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=119377998ad4728d server_cookie=51906bc47ae60e90 in_seq=0 out_seq=0 2024-09-21T03:46:03.440 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:03.429+0000 7feab5ffb640 10 client.15456 ms_handle_connect on v2:172.21.15.150:6838/2949386907 2024-09-21T03:46:03.440 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:03.430+0000 7feab5ffb640 1 -- 172.21.15.39:0/1084968078 <== mds.0 v2:172.21.15.150:6838/2949386907 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5642e3ddbb60 con 0x5642e3a7a8e0 2024-09-21T03:46:03.440 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:03.430+0000 7feab5ffb640 10 client.15456 handle_command_reply: tid=0 2024-09-21T03:46:03.440 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:03.430+0000 7feadaa07740 1 client.15456 shutdown 2024-09-21T03:46:03.440 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:03.430+0000 7feadaa07740 1 -- 172.21.15.39:0/1084968078 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5642e3a9b280 msgr2=0x5642e3e6ddb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:46:03.440 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:03.430+0000 7feadaa07740 1 --2- 172.21.15.39:0/1084968078 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5642e3a9b280 0x5642e3e6ddb0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7feac8000c00 tx=0x7feac802abb0 comp rx=0 tx=0).stop 2024-09-21T03:46:03.441 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:03.430+0000 7feadaa07740 1 -- 172.21.15.39:0/1084968078 shutdown_connections 2024-09-21T03:46:03.441 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:03.430+0000 7feadaa07740 1 --2- 172.21.15.39:0/1084968078 >> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] conn(0x5642e3a7a8e0 0x5642e3e7fb10 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:03.441 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:03.430+0000 7feab5ffb640 0 client.15456 ms_handle_reset on v2:172.21.15.150:6838/2949386907 2024-09-21T03:46:03.441 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:03.430+0000 7feadaa07740 1 --2- 172.21.15.39:0/1084968078 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5642e3abbf60 0x5642e3e799a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:03.441 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:03.430+0000 7feadaa07740 1 --2- 172.21.15.39:0/1084968078 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5642e3a9b280 0x5642e3e6ddb0 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:03.441 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:03.430+0000 7feadaa07740 1 --2- 172.21.15.39:0/1084968078 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5642e3a930a0 0x5642e3e6d7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:03.441 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:03.430+0000 7feadaa07740 1 -- 172.21.15.39:0/1084968078 >> 172.21.15.39:0/1084968078 conn(0x5642e3ab0350 msgr2=0x5642e3e650e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:46:03.441 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:03.430+0000 7feadaa07740 1 -- 172.21.15.39:0/1084968078 shutdown_connections 2024-09-21T03:46:03.441 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:03.430+0000 7feadaa07740 1 -- 172.21.15.39:0/1084968078 wait complete. 2024-09-21T03:46:03.441 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:03.430+0000 7feadaa07740 20 client.15456 trim_cache size 0 max 16384 2024-09-21T03:46:03.452 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:03.448+0000 7feadaa07740 1 Processor -- start 2024-09-21T03:46:03.452 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:03.448+0000 7feadaa07740 1 -- start start 2024-09-21T03:46:03.452 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:03.448+0000 7feadaa07740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5642e3a7a8e0 0x5642e3e68020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:03.452 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:03.448+0000 7feadaa07740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5642e3a930a0 0x5642e3e5aad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:03.452 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:03.448+0000 7feadaa07740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5642e3a9b280 0x5642e3e79910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:03.452 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:03.448+0000 7feadaa07740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x5642e3e7de50 con 0x5642e3a930a0 2024-09-21T03:46:03.452 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:03.448+0000 7feadaa07740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x5642e3e7e520 con 0x5642e3a9b280 2024-09-21T03:46:03.452 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:03.448+0000 7feadaa07740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x5642e3e7ebf0 con 0x5642e3a7a8e0 2024-09-21T03:46:03.453 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:03.449+0000 7fead222f640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5642e3a930a0 0x5642e3e5aad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:46:03.453 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:03.449+0000 7fead222f640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5642e3a930a0 0x5642e3e5aad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:42650/0 (socket says 172.21.15.39:42650) 2024-09-21T03:46:03.453 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:03.449+0000 7fead222f640 1 -- 172.21.15.39:0/1320453358 learned_addr learned my addr 172.21.15.39:0/1320453358 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:46:03.453 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:03.449+0000 7fead222f640 1 -- 172.21.15.39:0/1320453358 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5642e3a7a8e0 msgr2=0x5642e3e68020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:46:03.453 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:03.449+0000 7fead1a2e640 1 --2- 172.21.15.39:0/1320453358 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5642e3a7a8e0 0x5642e3e68020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:46:03.453 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:03.449+0000 7fead222f640 1 --2- 172.21.15.39:0/1320453358 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5642e3a7a8e0 0x5642e3e68020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:03.453 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:03.449+0000 7fead222f640 1 -- 172.21.15.39:0/1320453358 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5642e3a9b280 msgr2=0x5642e3e79910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:46:03.453 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:03.449+0000 7fead222f640 1 --2- 172.21.15.39:0/1320453358 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5642e3a9b280 0x5642e3e79910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:03.453 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:03.449+0000 7fead222f640 1 -- 172.21.15.39:0/1320453358 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5642e3e6ff10 con 0x5642e3a930a0 2024-09-21T03:46:03.453 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:03.449+0000 7fead1a2e640 1 --2- 172.21.15.39:0/1320453358 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5642e3a7a8e0 0x5642e3e68020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:46:03.453 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:03.449+0000 7fead222f640 1 --2- 172.21.15.39:0/1320453358 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5642e3a930a0 0x5642e3e5aad0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7feac8033000 tx=0x7feac8031400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=75e659f4749fdf5b server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:46:03.453 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:03.449+0000 7feab7fff640 1 -- 172.21.15.39:0/1320453358 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feac805f020 con 0x5642e3a930a0 2024-09-21T03:46:03.454 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:03.449+0000 7feab7fff640 1 -- 172.21.15.39:0/1320453358 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7feac8005400 con 0x5642e3a930a0 2024-09-21T03:46:03.454 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:03.449+0000 7feab7fff640 1 -- 172.21.15.39:0/1320453358 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feac8035040 con 0x5642e3a930a0 2024-09-21T03:46:03.458 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:03.450+0000 7feadaa07740 1 -- 172.21.15.39:0/1320453358 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5642e3a930a0 msgr2=0x5642e3e5aad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:46:03.458 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:03.450+0000 7feadaa07740 1 --2- 172.21.15.39:0/1320453358 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5642e3a930a0 0x5642e3e5aad0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7feac8033000 tx=0x7feac8031400 comp rx=0 tx=0).stop 2024-09-21T03:46:03.458 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:03.450+0000 7feadaa07740 1 -- 172.21.15.39:0/1320453358 shutdown_connections 2024-09-21T03:46:03.458 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:03.450+0000 7feadaa07740 1 --2- 172.21.15.39:0/1320453358 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5642e3a9b280 0x5642e3e79910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:03.458 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:03.450+0000 7feadaa07740 1 --2- 172.21.15.39:0/1320453358 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5642e3a930a0 0x5642e3e5aad0 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:03.458 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:03.450+0000 7feadaa07740 1 --2- 172.21.15.39:0/1320453358 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5642e3a7a8e0 0x5642e3e68020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:03.458 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:03.450+0000 7feadaa07740 1 -- 172.21.15.39:0/1320453358 >> 172.21.15.39:0/1320453358 conn(0x5642e3ab0350 msgr2=0x5642e3e68980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:46:03.458 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:03.450+0000 7feadaa07740 1 -- 172.21.15.39:0/1320453358 shutdown_connections 2024-09-21T03:46:03.458 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:03.450+0000 7feadaa07740 1 -- 172.21.15.39:0/1320453358 wait complete. 2024-09-21T03:46:03.458 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:03.450+0000 7feadaa07740 1 -- start start 2024-09-21T03:46:03.458 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:03.451+0000 7feadaa07740 1 Processor -- start 2024-09-21T03:46:03.459 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:03.451+0000 7feadaa07740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5642e3a7a8e0 0x5642e3e82080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:03.459 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:03.451+0000 7feadaa07740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5642e3a930a0 0x5642e3e825c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:03.459 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:03.451+0000 7feadaa07740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5642e3a9b280 0x5642e3e87810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:03.459 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:03.451+0000 7feadaa07740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x5642e3e7f990 con 0x5642e3a930a0 2024-09-21T03:46:03.459 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:03.451+0000 7feadaa07740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x5642e3e7f2c0 con 0x5642e3a7a8e0 2024-09-21T03:46:03.459 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:03.451+0000 7feadaa07740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x5642e3e80060 con 0x5642e3a9b280 2024-09-21T03:46:03.459 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:03.451+0000 7feadaa07740 10 client.0 _ll_register_callbacks cb 0x7feabc009a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:46:03.459 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:03.451+0000 7fead222f640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5642e3a930a0 0x5642e3e825c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:46:03.459 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:03.451+0000 7fead222f640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5642e3a930a0 0x5642e3e825c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:42656/0 (socket says 172.21.15.39:42656) 2024-09-21T03:46:03.459 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:03.451+0000 7fead222f640 1 -- 172.21.15.39:0/509118864 learned_addr learned my addr 172.21.15.39:0/509118864 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:46:03.459 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:03.451+0000 7fead1a2e640 1 --2- 172.21.15.39:0/509118864 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5642e3a7a8e0 0x5642e3e82080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:46:03.459 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:03.451+0000 7fead1a2e640 1 --2- 172.21.15.39:0/509118864 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5642e3a9b280 0x5642e3e87810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:46:03.459 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:03.451+0000 7fead222f640 1 -- 172.21.15.39:0/509118864 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5642e3a9b280 msgr2=0x5642e3e87810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:46:03.459 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:03.451+0000 7fead222f640 1 --2- 172.21.15.39:0/509118864 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5642e3a9b280 0x5642e3e87810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:03.460 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:03.451+0000 7fead222f640 1 -- 172.21.15.39:0/509118864 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5642e3a7a8e0 msgr2=0x5642e3e82080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:46:03.460 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:03.451+0000 7fead222f640 1 --2- 172.21.15.39:0/509118864 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5642e3a7a8e0 0x5642e3e82080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:03.460 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:03.451+0000 7fead222f640 1 -- 172.21.15.39:0/509118864 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5642e3e82e40 con 0x5642e3a930a0 2024-09-21T03:46:03.460 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:03.451+0000 7fead1a2e640 1 --2- 172.21.15.39:0/509118864 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5642e3a9b280 0x5642e3e87810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:46:03.460 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:03.451+0000 7fead222f640 1 --2- 172.21.15.39:0/509118864 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5642e3a930a0 0x5642e3e825c0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7feac804b040 tx=0x7feac802ab10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be2d52f12a8e5e93 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:46:03.460 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:03.451+0000 7feab5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-09-21T03:46:03.460 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:03.452+0000 7feab5ffb640 1 -- 172.21.15.39:0/509118864 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feac8035070 con 0x5642e3a930a0 2024-09-21T03:46:03.460 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:03.452+0000 7feab5ffb640 1 -- 172.21.15.39:0/509118864 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8484+0+0 (secure 0 0 0) 0x7feac803fb10 con 0x5642e3a930a0 2024-09-21T03:46:03.460 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:03.452+0000 7feadaa07740 1 -- 172.21.15.39:0/509118864 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7feac8005400 con 0x5642e3a930a0 2024-09-21T03:46:03.460 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:03.452+0000 7feab5ffb640 1 -- 172.21.15.39:0/509118864 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7feac805f040 con 0x5642e3a930a0 2024-09-21T03:46:03.460 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:03.452+0000 7feadaa07740 10 client.15462 fetch_fsmap learned FSMap version 28 2024-09-21T03:46:03.460 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:03.452+0000 7feadaa07740 1 -- 172.21.15.39:0/509118864 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5642e3eeb4c0 con 0x5642e3a930a0 2024-09-21T03:46:03.460 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:03.452+0000 7feab5ffb640 1 -- 172.21.15.39:0/509118864 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7feac8005400 con 0x5642e3a930a0 2024-09-21T03:46:03.460 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:03.452+0000 7feab5ffb640 1 -- 172.21.15.39:0/509118864 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7feac806a030 con 0x5642e3a930a0 2024-09-21T03:46:03.460 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:03.452+0000 7feadaa07740 10 client.15462 fetch_fsmap finished waiting for FSMap version 28 2024-09-21T03:46:03.461 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:03.452+0000 7feadaa07740 10 client.15462 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-21T03:46:03.461 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:03.452+0000 7feadaa07740 20 client.15462 populate_metadata read hostname 'smithi039' 2024-09-21T03:46:03.461 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:03.452+0000 7feadaa07740 1 --2- 172.21.15.39:0/509118864 >> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] conn(0x5642e3abbf60 0x5642e3eeeb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:03.461 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:03.452+0000 7feadaa07740 4 client.15462 mds_command: new command op to 24695 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-21T03:46:03.461 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:03.452+0000 7feadaa07740 1 -- 172.21.15.39:0/509118864 --> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] -- command(tid 0: {"prefix": "status"}) -- 0x5642e3dd7610 con 0x5642e3abbf60 2024-09-21T03:46:03.461 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:03.453+0000 7fead1a2e640 1 --2- 172.21.15.39:0/509118864 >> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] conn(0x5642e3abbf60 0x5642e3eeeb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:46:03.461 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:03.453+0000 7fead1a2e640 1 --2- 172.21.15.39:0/509118864 >> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] conn(0x5642e3abbf60 0x5642e3eeeb60 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f883c7664524e82e server_cookie=245b94b55f9243e9 in_seq=0 out_seq=0 2024-09-21T03:46:03.461 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:03.453+0000 7feab5ffb640 10 client.15462 ms_handle_connect on v2:172.21.15.150:6838/2949386907 2024-09-21T03:46:03.461 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:03.453+0000 7feab5ffb640 1 -- 172.21.15.39:0/509118864 <== mds.0 v2:172.21.15.150:6838/2949386907 1 ==== command_reply(tid 0: 0 ) ==== 8+0+322 (crc 0 0 0) 0x5642e3dd7610 con 0x5642e3abbf60 2024-09-21T03:46:03.461 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:03.453+0000 7feab5ffb640 10 client.15462 handle_command_reply: tid=0 2024-09-21T03:46:03.461 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:03.453+0000 7feadaa07740 1 client.15462 shutdown 2024-09-21T03:46:03.461 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:03.453+0000 7feadaa07740 1 -- 172.21.15.39:0/509118864 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5642e3a930a0 msgr2=0x5642e3e825c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:46:03.461 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:03.453+0000 7feadaa07740 1 --2- 172.21.15.39:0/509118864 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5642e3a930a0 0x5642e3e825c0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7feac804b040 tx=0x7feac802ab10 comp rx=0 tx=0).stop 2024-09-21T03:46:03.462 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:03.453+0000 7feadaa07740 1 -- 172.21.15.39:0/509118864 shutdown_connections 2024-09-21T03:46:03.462 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:03.453+0000 7feadaa07740 1 --2- 172.21.15.39:0/509118864 >> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] conn(0x5642e3abbf60 0x5642e3eeeb60 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:03.462 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:03.453+0000 7feadaa07740 1 --2- 172.21.15.39:0/509118864 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5642e3a9b280 0x5642e3e87810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:03.462 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:03.453+0000 7feadaa07740 1 --2- 172.21.15.39:0/509118864 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5642e3a930a0 0x5642e3e825c0 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:03.462 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:03.453+0000 7feab5ffb640 0 client.15462 ms_handle_reset on v2:172.21.15.150:6838/2949386907 2024-09-21T03:46:03.462 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:03.453+0000 7feadaa07740 1 --2- 172.21.15.39:0/509118864 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5642e3a7a8e0 0x5642e3e82080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:03.462 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:03.453+0000 7feadaa07740 1 -- 172.21.15.39:0/509118864 >> 172.21.15.39:0/509118864 conn(0x5642e3ab0350 msgr2=0x5642e3e68980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:46:03.462 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:03.454+0000 7feadaa07740 1 -- 172.21.15.39:0/509118864 shutdown_connections 2024-09-21T03:46:03.462 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:03.454+0000 7feadaa07740 1 -- 172.21.15.39:0/509118864 wait complete. 2024-09-21T03:46:03.462 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:03.454+0000 7feadaa07740 20 client.15462 trim_cache size 0 max 16384 2024-09-21T03:46:03.462 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-09-21T03:46:03.462 INFO:teuthology.orchestra.run.smithi039.stdout: "cluster_fsid": "51116d5c-77ca-11ef-baf5-efdc52797490", 2024-09-21T03:46:03.463 INFO:teuthology.orchestra.run.smithi039.stdout: "whoami": 0, 2024-09-21T03:46:03.463 INFO:teuthology.orchestra.run.smithi039.stdout: "id": 24695, 2024-09-21T03:46:03.463 INFO:teuthology.orchestra.run.smithi039.stdout: "want_state": "up:active", 2024-09-21T03:46:03.463 INFO:teuthology.orchestra.run.smithi039.stdout: "state": "up:active", 2024-09-21T03:46:03.463 INFO:teuthology.orchestra.run.smithi039.stdout: "fs_name": "cephfs", 2024-09-21T03:46:03.463 INFO:teuthology.orchestra.run.smithi039.stdout: "rank_uptime": 108.123518437, 2024-09-21T03:46:03.463 INFO:teuthology.orchestra.run.smithi039.stdout: "mdsmap_epoch": 28, 2024-09-21T03:46:03.463 INFO:teuthology.orchestra.run.smithi039.stdout: "osdmap_epoch": 79, 2024-09-21T03:46:03.463 INFO:teuthology.orchestra.run.smithi039.stdout: "osdmap_epoch_barrier": 78, 2024-09-21T03:46:03.463 INFO:teuthology.orchestra.run.smithi039.stdout: "uptime": 118.598861322 2024-09-21T03:46:03.463 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-09-21T03:46:03.464 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:03.454+0000 7fead4493640 1 -- 172.21.15.39:0/2502335289 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7feab80b65e0 msgr2=0x7feab80b8af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:46:03.464 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:03.454+0000 7fead4493640 1 --2- 172.21.15.39:0/2502335289 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7feab80b65e0 0x7feab80b8af0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7feac40e5280 tx=0x7feac40770a0 comp rx=0 tx=0).stop 2024-09-21T03:46:03.464 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:03.454+0000 7fead4493640 1 -- 172.21.15.39:0/2502335289 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7feacc108cf0 msgr2=0x7feacc19fc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:46:03.464 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:03.454+0000 7fead4493640 1 --2- 172.21.15.39:0/2502335289 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7feacc108cf0 0x7feacc19fc20 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7feac400eaa0 tx=0x7feac400ef60 comp rx=0 tx=0).stop 2024-09-21T03:46:03.464 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:03.454+0000 7fead4493640 1 -- 172.21.15.39:0/2502335289 shutdown_connections 2024-09-21T03:46:03.464 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:03.454+0000 7fead4493640 1 --2- 172.21.15.39:0/2502335289 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7feab80b65e0 0x7feab80b8af0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:03.464 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:03.454+0000 7fead4493640 1 --2- 172.21.15.39:0/2502335289 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7feacc108cf0 0x7feacc19fc20 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:03.464 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:03.454+0000 7fead4493640 1 --2- 172.21.15.39:0/2502335289 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7feacc1083a0 0x7feacc19a980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:03.464 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:03.454+0000 7fead4493640 1 --2- 172.21.15.39:0/2502335289 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7feacc102a60 0x7feacc19a3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:03.465 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:03.454+0000 7fead4493640 1 -- 172.21.15.39:0/2502335289 >> 172.21.15.39:0/2502335289 conn(0x7feacc079730 msgr2=0x7feacc101ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:46:03.465 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:03.455+0000 7fead4493640 1 -- 172.21.15.39:0/2502335289 shutdown_connections 2024-09-21T03:46:03.465 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:03.455+0000 7fead4493640 1 -- 172.21.15.39:0/2502335289 wait complete. 2024-09-21T03:46:04.354 INFO:teuthology.run_tasks:Running task kclient... 2024-09-21T03:46:04.378 INFO:tasks.kclient:Mounting kernel clients... 2024-09-21T03:46:04.378 INFO:tasks.kclient:config is {'client.0': {}} 2024-09-21T03:46:04.379 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T03:46:04.379 DEBUG:teuthology.orchestra.run.smithi039:> ip netns list 2024-09-21T03:46:04.397 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T03:46:04.397 DEBUG:teuthology.orchestra.run.smithi039:> sudo ip link delete ceph-brx 2024-09-21T03:46:04.468 INFO:teuthology.orchestra.run.smithi039.stderr:Cannot find device "ceph-brx" 2024-09-21T03:46:04.469 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T03:46:04.470 INFO:tasks.kclient:client.0 config is {'mntopts': ['ms_mode=secure', 'wsync'], 'syntax': 'v1', 'mount_subvol_num': 1} 2024-09-21T03:46:04.470 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-09-21T03:46:04.470 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-21T03:46:04.470 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 51116d5c-77ca-11ef-baf5-efdc52797490 -- ceph fs ls 2024-09-21T03:46:04.768 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/51116d5c-77ca-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:46:05.194 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:46:04 smithi039 ceph-mon[31211]: pgmap v326: 97 pgs: 97 active+clean; 647 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 767 B/s wr, 10 op/s 2024-09-21T03:46:05.201 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:05.191+0000 7f96d40fa640 1 -- 172.21.15.39:0/1188908553 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f96cc108cf0 msgr2=0x7f96cc10f520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:46:05.201 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:05.191+0000 7f96d40fa640 1 --2- 172.21.15.39:0/1188908553 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f96cc108cf0 0x7f96cc10f520 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f96c4009ca0 tx=0x7f96c402a8c0 comp rx=0 tx=0).stop 2024-09-21T03:46:05.201 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:05.191+0000 7f96d40fa640 1 -- 172.21.15.39:0/1188908553 shutdown_connections 2024-09-21T03:46:05.201 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:05.191+0000 7f96d40fa640 1 --2- 172.21.15.39:0/1188908553 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f96cc108cf0 0x7f96cc10f520 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:05.201 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:05.191+0000 7f96d40fa640 1 --2- 172.21.15.39:0/1188908553 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f96cc1083a0 0x7f96cc1087b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:05.202 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:05.191+0000 7f96d40fa640 1 --2- 172.21.15.39:0/1188908553 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f96cc102a60 0x7f96cc102e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:05.202 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:05.191+0000 7f96d40fa640 1 -- 172.21.15.39:0/1188908553 >> 172.21.15.39:0/1188908553 conn(0x7f96cc079730 msgr2=0x7f96cc079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:46:05.202 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:05.192+0000 7f96d40fa640 1 -- 172.21.15.39:0/1188908553 shutdown_connections 2024-09-21T03:46:05.202 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:05.192+0000 7f96d40fa640 1 -- 172.21.15.39:0/1188908553 wait complete. 2024-09-21T03:46:05.202 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:05.193+0000 7f96d40fa640 1 Processor -- start 2024-09-21T03:46:05.202 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:05.193+0000 7f96d40fa640 1 -- start start 2024-09-21T03:46:05.202 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:05.193+0000 7f96d40fa640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f96cc102a60 0x7f96cc19a4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:05.202 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:05.193+0000 7f96d40fa640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f96cc1083a0 0x7f96cc19aa40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:05.202 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:05.193+0000 7f96d40fa640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f96cc108cf0 0x7f96cc19fce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:05.202 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:05.193+0000 7f96d40fa640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f96cc112960 con 0x7f96cc1083a0 2024-09-21T03:46:05.202 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:05.193+0000 7f96d40fa640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f96cc1127e0 con 0x7f96cc102a60 2024-09-21T03:46:05.202 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:05.193+0000 7f96d40fa640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f96cc112ae0 con 0x7f96cc108cf0 2024-09-21T03:46:05.202 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:05.193+0000 7f96d1695640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f96cc1083a0 0x7f96cc19aa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:46:05.203 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:05.193+0000 7f96d2697640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f96cc108cf0 0x7f96cc19fce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:46:05.203 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:05.193+0000 7f96d2697640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f96cc108cf0 0x7f96cc19fce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.39:36904/0 (socket says 172.21.15.39:36904) 2024-09-21T03:46:05.203 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:05.193+0000 7f96d2697640 1 -- 172.21.15.39:0/4273142855 learned_addr learned my addr 172.21.15.39:0/4273142855 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:46:05.203 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:05.193+0000 7f96d1695640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f96cc1083a0 0x7f96cc19aa40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:42684/0 (socket says 172.21.15.39:42684) 2024-09-21T03:46:05.203 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:05.193+0000 7f96d1e96640 1 --2- 172.21.15.39:0/4273142855 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f96cc102a60 0x7f96cc19a4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:46:05.203 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:05.193+0000 7f96d1695640 1 -- 172.21.15.39:0/4273142855 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f96cc108cf0 msgr2=0x7f96cc19fce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:46:05.203 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:05.193+0000 7f96d1695640 1 --2- 172.21.15.39:0/4273142855 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f96cc108cf0 0x7f96cc19fce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:05.203 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:05.194+0000 7f96d1695640 1 -- 172.21.15.39:0/4273142855 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f96cc102a60 msgr2=0x7f96cc19a4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:46:05.203 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:05.194+0000 7f96d1695640 1 --2- 172.21.15.39:0/4273142855 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f96cc102a60 0x7f96cc19a4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:05.203 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:05.194+0000 7f96d1695640 1 -- 172.21.15.39:0/4273142855 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f96cc1a0410 con 0x7f96cc1083a0 2024-09-21T03:46:05.203 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:05.194+0000 7f96d1695640 1 --2- 172.21.15.39:0/4273142855 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f96cc1083a0 0x7f96cc19aa40 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f96bc00c940 tx=0x7f96bc00ce00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:46:05.203 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:05.194+0000 7f96c2ffd640 1 -- 172.21.15.39:0/4273142855 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96bc007bb0 con 0x7f96cc1083a0 2024-09-21T03:46:05.203 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:05.194+0000 7f96c2ffd640 1 -- 172.21.15.39:0/4273142855 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f96bc007d50 con 0x7f96cc1083a0 2024-09-21T03:46:05.204 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:05.194+0000 7f96d40fa640 1 -- 172.21.15.39:0/4273142855 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f96cc1a0700 con 0x7f96cc1083a0 2024-09-21T03:46:05.204 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:05.194+0000 7f96c2ffd640 1 -- 172.21.15.39:0/4273142855 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96bc004db0 con 0x7f96cc1083a0 2024-09-21T03:46:05.204 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:05.194+0000 7f96d40fa640 1 -- 172.21.15.39:0/4273142855 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f96cc1a6e60 con 0x7f96cc1083a0 2024-09-21T03:46:05.204 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:05.195+0000 7f96d40fa640 1 -- 172.21.15.39:0/4273142855 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f96cc103ce0 con 0x7f96cc1083a0 2024-09-21T03:46:05.204 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:05.197+0000 7f96c2ffd640 1 -- 172.21.15.39:0/4273142855 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 23) ==== 154529+0+0 (secure 0 0 0) 0x7f96bc005000 con 0x7f96cc1083a0 2024-09-21T03:46:05.204 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:05.198+0000 7f96c2ffd640 1 --2- 172.21.15.39:0/4273142855 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f96b80b6560 0x7f96b80b8a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:05.204 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:05.198+0000 7f96d1e96640 1 --2- 172.21.15.39:0/4273142855 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f96b80b6560 0x7f96b80b8a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:46:05.204 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:05.198+0000 7f96c2ffd640 1 -- 172.21.15.39:0/4273142855 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8484+0+0 (secure 0 0 0) 0x7f96bc0e7410 con 0x7f96cc1083a0 2024-09-21T03:46:05.204 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:05.198+0000 7f96d1e96640 1 --2- 172.21.15.39:0/4273142855 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f96b80b6560 0x7f96b80b8a70 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f96c800bbe0 tx=0x7f96c800b390 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:46:05.204 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:05.201+0000 7f96c2ffd640 1 -- 172.21.15.39:0/4273142855 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f96bc014030 con 0x7f96cc1083a0 2024-09-21T03:46:05.225 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:46:04 smithi150 ceph-mon[34289]: pgmap v326: 97 pgs: 97 active+clean; 647 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 767 B/s wr, 10 op/s 2024-09-21T03:46:05.436 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:05.433+0000 7f96d40fa640 1 -- 172.21.15.39:0/4273142855 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f96cc10bb50 con 0x7f96cc1083a0 2024-09-21T03:46:05.437 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:05.434+0000 7f96c2ffd640 1 -- 172.21.15.39:0/4273142855 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v28) ==== 53+0+73 (secure 0 0 0) 0x7f96bc0b1f80 con 0x7f96cc1083a0 2024-09-21T03:46:05.437 INFO:teuthology.orchestra.run.smithi039.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-21T03:46:05.439 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:05.436+0000 7f96d40fa640 1 -- 172.21.15.39:0/4273142855 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f96b80b6560 msgr2=0x7f96b80b8a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:46:05.439 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:05.436+0000 7f96d40fa640 1 --2- 172.21.15.39:0/4273142855 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f96b80b6560 0x7f96b80b8a70 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f96c800bbe0 tx=0x7f96c800b390 comp rx=0 tx=0).stop 2024-09-21T03:46:05.439 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:05.436+0000 7f96d40fa640 1 -- 172.21.15.39:0/4273142855 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f96cc1083a0 msgr2=0x7f96cc19aa40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:46:05.439 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:05.436+0000 7f96d40fa640 1 --2- 172.21.15.39:0/4273142855 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f96cc1083a0 0x7f96cc19aa40 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f96bc00c940 tx=0x7f96bc00ce00 comp rx=0 tx=0).stop 2024-09-21T03:46:05.439 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:05.436+0000 7f96d40fa640 1 -- 172.21.15.39:0/4273142855 shutdown_connections 2024-09-21T03:46:05.439 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:05.436+0000 7f96d40fa640 1 --2- 172.21.15.39:0/4273142855 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f96b80b6560 0x7f96b80b8a70 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:05.440 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:05.436+0000 7f96d40fa640 1 --2- 172.21.15.39:0/4273142855 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f96cc108cf0 0x7f96cc19fce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:05.440 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:05.436+0000 7f96d40fa640 1 --2- 172.21.15.39:0/4273142855 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f96cc1083a0 0x7f96cc19aa40 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:05.440 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:05.436+0000 7f96d40fa640 1 --2- 172.21.15.39:0/4273142855 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f96cc102a60 0x7f96cc19a4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:05.440 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:05.436+0000 7f96d40fa640 1 -- 172.21.15.39:0/4273142855 >> 172.21.15.39:0/4273142855 conn(0x7f96cc079730 msgr2=0x7f96cc06b200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:46:05.440 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:05.437+0000 7f96d40fa640 1 -- 172.21.15.39:0/4273142855 shutdown_connections 2024-09-21T03:46:05.440 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:05.437+0000 7f96d40fa640 1 -- 172.21.15.39:0/4273142855 wait complete. 2024-09-21T03:46:05.467 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:46:04 smithi161 ceph-mon[34533]: pgmap v326: 97 pgs: 97 active+clean; 647 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 767 B/s wr, 10 op/s 2024-09-21T03:46:06.272 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:46:05 smithi039 ceph-mon[31211]: from='client.? 172.21.15.39:0/4273142855' entity='client.admin' cmd={"prefix": "fs ls"} : dispatch 2024-09-21T03:46:06.302 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-09-21T03:46:06.302 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-21T03:46:06.302 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi039.front.sepia.ceph.com 2024-09-21T03:46:06.303 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-09-21T03:46:06.303 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-21T03:46:06.303 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-09-21T03:46:06.303 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-09-21T03:46:06.303 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-21T03:46:06.303 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-21T03:46:06.303 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T03:46:06.303 DEBUG:teuthology.orchestra.run.smithi039:> ip addr 2024-09-21T03:46:06.320 INFO:teuthology.orchestra.run.smithi039.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-21T03:46:06.320 INFO:teuthology.orchestra.run.smithi039.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-21T03:46:06.321 INFO:teuthology.orchestra.run.smithi039.stdout: inet 127.0.0.1/8 scope host lo 2024-09-21T03:46:06.321 INFO:teuthology.orchestra.run.smithi039.stdout: valid_lft forever preferred_lft forever 2024-09-21T03:46:06.321 INFO:teuthology.orchestra.run.smithi039.stdout: inet6 ::1/128 scope host 2024-09-21T03:46:06.321 INFO:teuthology.orchestra.run.smithi039.stdout: valid_lft forever preferred_lft forever 2024-09-21T03:46:06.321 INFO:teuthology.orchestra.run.smithi039.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-21T03:46:06.321 INFO:teuthology.orchestra.run.smithi039.stdout: link/ether 0c:c4:7a:6b:fe:c6 brd ff:ff:ff:ff:ff:ff 2024-09-21T03:46:06.321 INFO:teuthology.orchestra.run.smithi039.stdout: altname enp7s0f0 2024-09-21T03:46:06.321 INFO:teuthology.orchestra.run.smithi039.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-21T03:46:06.321 INFO:teuthology.orchestra.run.smithi039.stdout: link/ether 0c:c4:7a:6b:fe:c7 brd ff:ff:ff:ff:ff:ff 2024-09-21T03:46:06.321 INFO:teuthology.orchestra.run.smithi039.stdout: altname enp7s0f1 2024-09-21T03:46:06.321 INFO:teuthology.orchestra.run.smithi039.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-21T03:46:06.321 INFO:teuthology.orchestra.run.smithi039.stdout: link/ether 0c:c4:7a:bd:13:70 brd ff:ff:ff:ff:ff:ff 2024-09-21T03:46:06.321 INFO:teuthology.orchestra.run.smithi039.stdout: altname enp1s0f0 2024-09-21T03:46:06.322 INFO:teuthology.orchestra.run.smithi039.stdout: inet 172.21.15.39/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-09-21T03:46:06.322 INFO:teuthology.orchestra.run.smithi039.stdout: valid_lft 42049sec preferred_lft 42049sec 2024-09-21T03:46:06.322 INFO:teuthology.orchestra.run.smithi039.stdout: inet6 fe80::ec4:7aff:febd:1370/64 scope link 2024-09-21T03:46:06.322 INFO:teuthology.orchestra.run.smithi039.stdout: valid_lft forever preferred_lft forever 2024-09-21T03:46:06.322 INFO:teuthology.orchestra.run.smithi039.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-21T03:46:06.322 INFO:teuthology.orchestra.run.smithi039.stdout: link/ether 0c:c4:7a:bd:13:71 brd ff:ff:ff:ff:ff:ff 2024-09-21T03:46:06.322 INFO:teuthology.orchestra.run.smithi039.stdout: altname enp1s0f1 2024-09-21T03:46:06.322 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-09-21T03:46:06.322 DEBUG:teuthology.orchestra.run.smithi039:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-21T03:46:06.323 DEBUG:teuthology.orchestra.run.smithi039:> set -e 2024-09-21T03:46:06.323 DEBUG:teuthology.orchestra.run.smithi039:> sudo ip link add name ceph-brx type bridge 2024-09-21T03:46:06.323 DEBUG:teuthology.orchestra.run.smithi039:> sudo ip addr flush dev ceph-brx 2024-09-21T03:46:06.323 DEBUG:teuthology.orchestra.run.smithi039:> sudo ip link set ceph-brx up 2024-09-21T03:46:06.323 DEBUG:teuthology.orchestra.run.smithi039:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-09-21T03:46:06.323 DEBUG:teuthology.orchestra.run.smithi039:> ') 2024-09-21T03:46:06.410 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:06 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-21T03:46:06.468 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:46:05 smithi161 ceph-mon[34533]: from='client.? 172.21.15.39:0/4273142855' entity='client.admin' cmd={"prefix": "fs ls"} : dispatch 2024-09-21T03:46:06.474 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:46:05 smithi150 ceph-mon[34289]: from='client.? 172.21.15.39:0/4273142855' entity='client.admin' cmd={"prefix": "fs ls"} : dispatch 2024-09-21T03:46:06.517 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:06 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T03:46:06.523 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T03:46:06.524 DEBUG:teuthology.orchestra.run.smithi039:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-21T03:46:06.550 INFO:teuthology.orchestra.run.smithi039.stdout:1 2024-09-21T03:46:06.552 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T03:46:06.552 DEBUG:teuthology.orchestra.run.smithi039:> route 2024-09-21T03:46:06.617 INFO:teuthology.orchestra.run.smithi039.stdout:Kernel IP routing table 2024-09-21T03:46:06.618 INFO:teuthology.orchestra.run.smithi039.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-21T03:46:06.618 INFO:teuthology.orchestra.run.smithi039.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-09-21T03:46:06.618 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-09-21T03:46:06.618 INFO:teuthology.orchestra.run.smithi039.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-21T03:46:06.618 DEBUG:teuthology.orchestra.run.smithi039:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-21T03:46:06.619 DEBUG:teuthology.orchestra.run.smithi039:> set -e 2024-09-21T03:46:06.619 DEBUG:teuthology.orchestra.run.smithi039:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-09-21T03:46:06.619 DEBUG:teuthology.orchestra.run.smithi039:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-09-21T03:46:06.619 DEBUG:teuthology.orchestra.run.smithi039:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-09-21T03:46:06.619 DEBUG:teuthology.orchestra.run.smithi039:> ') 2024-09-21T03:46:06.707 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:06 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-21T03:46:06.922 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:06 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T03:46:06.928 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T03:46:06.928 DEBUG:teuthology.orchestra.run.smithi039:> ip netns list 2024-09-21T03:46:06.944 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T03:46:06.944 DEBUG:teuthology.orchestra.run.smithi039:> ip netns list-id 2024-09-21T03:46:07.002 DEBUG:teuthology.orchestra.run.smithi039:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-21T03:46:07.002 DEBUG:teuthology.orchestra.run.smithi039:> set -e 2024-09-21T03:46:07.003 DEBUG:teuthology.orchestra.run.smithi039:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-21T03:46:07.003 DEBUG:teuthology.orchestra.run.smithi039:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2024-09-21T03:46:07.003 DEBUG:teuthology.orchestra.run.smithi039:> ') 2024-09-21T03:46:07.083 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:46:06 smithi039 ceph-mon[31211]: pgmap v327: 97 pgs: 97 active+clean; 647 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 767 B/s wr, 13 op/s 2024-09-21T03:46:07.092 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:07 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-21T03:46:07.116 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:07 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T03:46:07.123 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2024-09-21T03:46:07.123 DEBUG:teuthology.orchestra.run.smithi039:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-21T03:46:07.123 DEBUG:teuthology.orchestra.run.smithi039:> set -e 2024-09-21T03:46:07.123 DEBUG:teuthology.orchestra.run.smithi039:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2024-09-21T03:46:07.123 DEBUG:teuthology.orchestra.run.smithi039:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-09-21T03:46:07.123 DEBUG:teuthology.orchestra.run.smithi039:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2024-09-21T03:46:07.123 DEBUG:teuthology.orchestra.run.smithi039:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2024-09-21T03:46:07.123 DEBUG:teuthology.orchestra.run.smithi039:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2024-09-21T03:46:07.123 DEBUG:teuthology.orchestra.run.smithi039:> ') 2024-09-21T03:46:07.212 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:07 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-21T03:46:07.284 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:07 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T03:46:07.290 DEBUG:teuthology.orchestra.run.smithi039:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-21T03:46:07.290 DEBUG:teuthology.orchestra.run.smithi039:> set -e 2024-09-21T03:46:07.290 DEBUG:teuthology.orchestra.run.smithi039:> sudo ip link set brx.0 up 2024-09-21T03:46:07.290 DEBUG:teuthology.orchestra.run.smithi039:> sudo ip link set dev brx.0 master ceph-brx 2024-09-21T03:46:07.290 DEBUG:teuthology.orchestra.run.smithi039:> ') 2024-09-21T03:46:07.340 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:07 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-21T03:46:07.363 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:07 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T03:46:07.369 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-21T03:46:07.370 DEBUG:teuthology.orchestra.run.smithi039:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-09-21T03:46:07.385 INFO:teuthology.orchestra.run.smithi039.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-09-21T03:46:07.385 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-21T03:46:07.386 DEBUG:teuthology.orchestra.run.smithi039:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-09-21T03:46:07.441 WARNING:tasks.cephfs.mount:You cannot specify both: cephfs_mntpt and mount_subvol_num 2024-09-21T03:46:07.441 INFO:tasks.cephfs.mount:Mounting subvol 1 for now 2024-09-21T03:46:07.441 INFO:tasks.cephfs.kernel_mount:mounting using device: :/volumes/qa/sv_1/a2925cab-9fdb-4028-a544-bf015f65c2bc 2024-09-21T03:46:07.441 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T03:46:07.441 DEBUG:teuthology.orchestra.run.smithi039:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/volumes/qa/sv_1/a2925cab-9fdb-4028-a544-bf015f65c2bc /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs,ms_mode=secure,wsync 2024-09-21T03:46:07.468 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:46:06 smithi161 ceph-mon[34533]: pgmap v327: 97 pgs: 97 active+clean; 647 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 767 B/s wr, 13 op/s 2024-09-21T03:46:07.474 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:46:06 smithi150 ceph-mon[34289]: pgmap v327: 97 pgs: 97 active+clean; 647 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 767 B/s wr, 13 op/s 2024-09-21T03:46:07.548 INFO:teuthology.orchestra.run.smithi039.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs,ms_mode=secure,wsync 2024-09-21T03:46:07.548 INFO:teuthology.orchestra.run.smithi039.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,ms_mode=secure,wsync". 2024-09-21T03:46:07.548 INFO:teuthology.orchestra.run.smithi039.stdout:invalid new device string format 2024-09-21T03:46:07.697 INFO:teuthology.orchestra.run.smithi039.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs,ms_mode=secure,wsync 2024-09-21T03:46:07.697 INFO:teuthology.orchestra.run.smithi039.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,ms_mode=secure,wsync". 2024-09-21T03:46:07.697 INFO:teuthology.orchestra.run.smithi039.stdout:invalid new device string format 2024-09-21T03:46:07.698 INFO:teuthology.orchestra.run.smithi039.stdout:mount.ceph: resolved to: "172.21.15.39:3300,172.21.15.150:3300,172.21.15.161:3300" 2024-09-21T03:46:07.698 INFO:teuthology.orchestra.run.smithi039.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.39:3300,172.21.15.150:3300,172.21.15.161:3300:/volumes/qa/sv_1/a2925cab-9fdb-4028-a544-bf015f65c2bc 2024-09-21T03:46:07.698 INFO:teuthology.orchestra.run.smithi039.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,ms_mode=secure,wsync,key=0,fsid=51116d5c-77ca-11ef-baf5-efdc52797490" will pass to kernel 2024-09-21T03:46:07.698 INFO:teuthology.orchestra.run.smithi039.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-09-21T03:46:07.699 INFO:teuthology.orchestra.run.smithi039.stdout: You just mounted a file system that supports labels which does not 2024-09-21T03:46:07.699 INFO:teuthology.orchestra.run.smithi039.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-09-21T03:46:07.699 INFO:teuthology.orchestra.run.smithi039.stdout: applications will generate AVC messages and not be allowed access to 2024-09-21T03:46:07.699 INFO:teuthology.orchestra.run.smithi039.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-09-21T03:46:07.700 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-09-21T03:46:07.700 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T03:46:07.700 DEBUG:teuthology.orchestra.run.smithi039:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-09-21T03:46:07.728 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-21T03:46:07.818 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:07 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-21T03:46:07.822 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:07 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T03:46:07.827 INFO:teuthology.orchestra.run.smithi039.stdout:client15471 2024-09-21T03:46:07.827 DEBUG:teuthology.orchestra.run.smithi039:> set -ex 2024-09-21T03:46:07.827 DEBUG:teuthology.orchestra.run.smithi039:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T03:46:07.844 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-21T03:46:07.935 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:07 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-21T03:46:07.937 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:07 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T03:46:07.943 INFO:teuthology.orchestra.run.smithi039.stdout:client15471 2024-09-21T03:46:07.943 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-21T03:46:07.993 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:07 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-21T03:46:07.996 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:07 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T03:46:08.002 INFO:teuthology.orchestra.run.smithi039.stdout:client15471 2024-09-21T03:46:08.002 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=300 -- bash -c 'sudo dd if=/sys/kernel/debug/ceph/51116d5c-77ca-11ef-baf5-efdc52797490.client15471/status') 2024-09-21T03:46:08.051 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:08 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-21T03:46:08.062 INFO:teuthology.orchestra.run.smithi039.stdout:instance: client.15471 (3)192.168.0.1:0/3808177376 2024-09-21T03:46:08.062 INFO:teuthology.orchestra.run.smithi039.stdout:blocklisted: false 2024-09-21T03:46:08.062 INFO:teuthology.orchestra.run.smithi039.stderr:0+1 records in 2024-09-21T03:46:08.062 INFO:teuthology.orchestra.run.smithi039.stderr:0+1 records out 2024-09-21T03:46:08.062 INFO:teuthology.orchestra.run.smithi039.stderr:70 bytes copied, 3.0097e-05 s, 2.3 MB/s 2024-09-21T03:46:08.063 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:08 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T03:46:08.069 INFO:teuthology.run_tasks:Running task exec... 2024-09-21T03:46:08.079 INFO:teuthology.task.exec:Executing custom commands... 2024-09-21T03:46:08.079 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi039.front.sepia.ceph.com 2024-09-21T03:46:08.079 DEBUG:teuthology.orchestra.run.smithi039:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs set cephfs balance_automate false' 2024-09-21T03:46:08.194 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:08.191+0000 7fea2cd71640 1 -- 172.21.15.39:0/1902335110 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fea281029e0 msgr2=0x7fea28102df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:46:08.194 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:08.191+0000 7fea2cd71640 1 --2- 172.21.15.39:0/1902335110 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fea281029e0 0x7fea28102df0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7fea18009900 tx=0x7fea1802a410 comp rx=0 tx=0).stop 2024-09-21T03:46:08.194 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:08.191+0000 7fea2cd71640 1 -- 172.21.15.39:0/1902335110 shutdown_connections 2024-09-21T03:46:08.194 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:08.191+0000 7fea2cd71640 1 --2- 172.21.15.39:0/1902335110 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fea28108c60 0x7fea2810f490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:08.194 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:08.191+0000 7fea2cd71640 1 --2- 172.21.15.39:0/1902335110 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fea28108310 0x7fea28108720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:08.194 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:08.191+0000 7fea2cd71640 1 --2- 172.21.15.39:0/1902335110 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fea281029e0 0x7fea28102df0 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:08.194 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:08.191+0000 7fea2cd71640 1 -- 172.21.15.39:0/1902335110 >> 172.21.15.39:0/1902335110 conn(0x7fea280fe3c0 msgr2=0x7fea28100810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:46:08.194 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:08.191+0000 7fea2cd71640 1 -- 172.21.15.39:0/1902335110 shutdown_connections 2024-09-21T03:46:08.194 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:08.191+0000 7fea2cd71640 1 -- 172.21.15.39:0/1902335110 wait complete. 2024-09-21T03:46:08.196 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:08.192+0000 7fea2cd71640 1 Processor -- start 2024-09-21T03:46:08.196 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:08.192+0000 7fea2cd71640 1 -- start start 2024-09-21T03:46:08.196 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:08.192+0000 7fea2cd71640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fea281029e0 0x7fea2819b1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:08.196 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:08.192+0000 7fea2cd71640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fea28108310 0x7fea2819b730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:08.196 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:08.192+0000 7fea2cd71640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fea28108c60 0x7fea2819bc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:08.196 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:08.192+0000 7fea2cd71640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fea28112920 con 0x7fea281029e0 2024-09-21T03:46:08.197 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:08.192+0000 7fea2cd71640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fea281127a0 con 0x7fea28108310 2024-09-21T03:46:08.197 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:08.192+0000 7fea2cd71640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fea28112aa0 con 0x7fea28108c60 2024-09-21T03:46:08.197 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:08.192+0000 7fea2659c640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fea281029e0 0x7fea2819b1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:46:08.197 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:08.192+0000 7fea2659c640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fea281029e0 0x7fea2819b1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:43050/0 (socket says 172.21.15.39:43050) 2024-09-21T03:46:08.197 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:08.192+0000 7fea2659c640 1 -- 172.21.15.39:0/2964373851 learned_addr learned my addr 172.21.15.39:0/2964373851 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:46:08.197 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:08.193+0000 7fea26d9d640 1 --2- 172.21.15.39:0/2964373851 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fea28108c60 0x7fea2819bc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:46:08.197 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:08.193+0000 7fea25d9b640 1 --2- 172.21.15.39:0/2964373851 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fea28108310 0x7fea2819b730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:46:08.197 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:08.193+0000 7fea2659c640 1 -- 172.21.15.39:0/2964373851 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fea28108c60 msgr2=0x7fea2819bc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:46:08.197 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:08.193+0000 7fea2659c640 1 --2- 172.21.15.39:0/2964373851 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fea28108c60 0x7fea2819bc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:08.197 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:08.193+0000 7fea2659c640 1 -- 172.21.15.39:0/2964373851 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fea28108310 msgr2=0x7fea2819b730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:46:08.197 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:08.193+0000 7fea2659c640 1 --2- 172.21.15.39:0/2964373851 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fea28108310 0x7fea2819b730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:08.197 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:08.193+0000 7fea2659c640 1 -- 172.21.15.39:0/2964373851 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fea281a6470 con 0x7fea281029e0 2024-09-21T03:46:08.197 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:08.193+0000 7fea2659c640 1 --2- 172.21.15.39:0/2964373851 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fea281029e0 0x7fea2819b1f0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7fea18005940 tx=0x7fea180042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:46:08.197 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:08.193+0000 7fea177fe640 1 -- 172.21.15.39:0/2964373851 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea1803f030 con 0x7fea281029e0 2024-09-21T03:46:08.198 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:08.193+0000 7fea177fe640 1 -- 172.21.15.39:0/2964373851 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fea18003cb0 con 0x7fea281029e0 2024-09-21T03:46:08.198 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:08.193+0000 7fea177fe640 1 -- 172.21.15.39:0/2964373851 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea18035070 con 0x7fea281029e0 2024-09-21T03:46:08.198 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:08.193+0000 7fea2cd71640 1 -- 172.21.15.39:0/2964373851 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fea281a6700 con 0x7fea281029e0 2024-09-21T03:46:08.198 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:08.193+0000 7fea2cd71640 1 -- 172.21.15.39:0/2964373851 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fea281a6bf0 con 0x7fea281029e0 2024-09-21T03:46:08.198 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:08.194+0000 7fea2cd71640 1 -- 172.21.15.39:0/2964373851 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fea28102df0 con 0x7fea281029e0 2024-09-21T03:46:08.198 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:08.196+0000 7fea177fe640 1 -- 172.21.15.39:0/2964373851 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 23) ==== 154529+0+0 (secure 0 0 0) 0x7fea18042030 con 0x7fea281029e0 2024-09-21T03:46:08.199 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:08.197+0000 7fea177fe640 1 --2- 172.21.15.39:0/2964373851 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7fea100b6560 0x7fea100b8a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:08.199 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:08.197+0000 7fea25d9b640 1 --2- 172.21.15.39:0/2964373851 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7fea100b6560 0x7fea100b8a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:46:08.199 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:08.197+0000 7fea177fe640 1 -- 172.21.15.39:0/2964373851 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8484+0+0 (secure 0 0 0) 0x7fea18108080 con 0x7fea281029e0 2024-09-21T03:46:08.200 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:08.197+0000 7fea25d9b640 1 --2- 172.21.15.39:0/2964373851 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7fea100b6560 0x7fea100b8a70 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fea1c005fd0 tx=0x7fea1c005850 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:46:08.202 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:08.200+0000 7fea177fe640 1 -- 172.21.15.39:0/2964373851 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fea180ce810 con 0x7fea281029e0 2024-09-21T03:46:08.430 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:08.428+0000 7fea2cd71640 1 -- 172.21.15.39:0/2964373851 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"} v 0) -- 0x7fea281a6fb0 con 0x7fea281029e0 2024-09-21T03:46:08.986 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:08.983+0000 7fea177fe640 1 -- 172.21.15.39:0/2964373851 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]=0 v29) ==== 118+0+0 (secure 0 0 0) 0x7fea180d2bf0 con 0x7fea281029e0 2024-09-21T03:46:08.989 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:08.987+0000 7fea2cd71640 1 -- 172.21.15.39:0/2964373851 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7fea100b6560 msgr2=0x7fea100b8a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:46:08.990 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:08.987+0000 7fea2cd71640 1 --2- 172.21.15.39:0/2964373851 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7fea100b6560 0x7fea100b8a70 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fea1c005fd0 tx=0x7fea1c005850 comp rx=0 tx=0).stop 2024-09-21T03:46:08.990 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:08.987+0000 7fea2cd71640 1 -- 172.21.15.39:0/2964373851 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fea281029e0 msgr2=0x7fea2819b1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:46:08.990 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:08.987+0000 7fea2cd71640 1 --2- 172.21.15.39:0/2964373851 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fea281029e0 0x7fea2819b1f0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7fea18005940 tx=0x7fea180042f0 comp rx=0 tx=0).stop 2024-09-21T03:46:08.990 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:08.987+0000 7fea2cd71640 1 -- 172.21.15.39:0/2964373851 shutdown_connections 2024-09-21T03:46:08.990 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:08.987+0000 7fea2cd71640 1 --2- 172.21.15.39:0/2964373851 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7fea100b6560 0x7fea100b8a70 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:08.990 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:08.987+0000 7fea2cd71640 1 --2- 172.21.15.39:0/2964373851 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fea28108c60 0x7fea2819bc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:08.990 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:08.987+0000 7fea2cd71640 1 --2- 172.21.15.39:0/2964373851 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fea28108310 0x7fea2819b730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:08.990 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:08.987+0000 7fea2cd71640 1 --2- 172.21.15.39:0/2964373851 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fea281029e0 0x7fea2819b1f0 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:08.990 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:08.987+0000 7fea2cd71640 1 -- 172.21.15.39:0/2964373851 >> 172.21.15.39:0/2964373851 conn(0x7fea280fe3c0 msgr2=0x7fea28106ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:46:08.990 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:08.987+0000 7fea2cd71640 1 -- 172.21.15.39:0/2964373851 shutdown_connections 2024-09-21T03:46:08.991 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:08.987+0000 7fea2cd71640 1 -- 172.21.15.39:0/2964373851 wait complete. 2024-09-21T03:46:09.005 DEBUG:teuthology.orchestra.run.smithi039:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs subvolumegroup pin cephfs qa random 0.10' 2024-09-21T03:46:09.126 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:09.123+0000 7f0664eb6640 1 -- 172.21.15.39:0/471680761 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f06601031a0 msgr2=0x7f06601035b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:46:09.127 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:09.123+0000 7f0664eb6640 1 --2- 172.21.15.39:0/471680761 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f06601031a0 0x7f06601035b0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f0648009900 tx=0x7f064802a410 comp rx=0 tx=0).stop 2024-09-21T03:46:09.127 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:09.123+0000 7f0664eb6640 1 -- 172.21.15.39:0/471680761 shutdown_connections 2024-09-21T03:46:09.127 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:09.123+0000 7f0664eb6640 1 --2- 172.21.15.39:0/471680761 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0660105a20 0x7f066010f490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:09.127 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:09.123+0000 7f0664eb6640 1 --2- 172.21.15.39:0/471680761 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f06601050d0 0x7f06601054e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:09.127 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:09.123+0000 7f0664eb6640 1 --2- 172.21.15.39:0/471680761 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f06601031a0 0x7f06601035b0 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:09.127 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:09.123+0000 7f0664eb6640 1 -- 172.21.15.39:0/471680761 >> 172.21.15.39:0/471680761 conn(0x7f0660079ec0 msgr2=0x7f066007a2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:46:09.127 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:09.123+0000 7f0664eb6640 1 -- 172.21.15.39:0/471680761 shutdown_connections 2024-09-21T03:46:09.127 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:09.123+0000 7f0664eb6640 1 -- 172.21.15.39:0/471680761 wait complete. 2024-09-21T03:46:09.129 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:09.124+0000 7f0664eb6640 1 Processor -- start 2024-09-21T03:46:09.129 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:09.124+0000 7f0664eb6640 1 -- start start 2024-09-21T03:46:09.129 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:09.124+0000 7f0664eb6640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f06601031a0 0x7f066019b250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:09.130 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:09.124+0000 7f0664eb6640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f06601050d0 0x7f066019b790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:09.130 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:09.124+0000 7f0664eb6640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0660105a20 0x7f066019bcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:09.130 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:09.124+0000 7f0664eb6640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f0660112920 con 0x7f06601031a0 2024-09-21T03:46:09.130 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:09.124+0000 7f0664eb6640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f06601127a0 con 0x7f06601050d0 2024-09-21T03:46:09.130 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:09.124+0000 7f0664eb6640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f0660112aa0 con 0x7f0660105a20 2024-09-21T03:46:09.130 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:09.125+0000 7f065e59c640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f06601031a0 0x7f066019b250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:46:09.130 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:09.125+0000 7f065e59c640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f06601031a0 0x7f066019b250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:43064/0 (socket says 172.21.15.39:43064) 2024-09-21T03:46:09.130 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:09.125+0000 7f065e59c640 1 -- 172.21.15.39:0/1311203808 learned_addr learned my addr 172.21.15.39:0/1311203808 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:46:09.130 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:09.125+0000 7f065dd9b640 1 --2- 172.21.15.39:0/1311203808 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f06601050d0 0x7f066019b790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:46:09.130 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:09.125+0000 7f065ed9d640 1 --2- 172.21.15.39:0/1311203808 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0660105a20 0x7f066019bcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:46:09.130 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:09.125+0000 7f065e59c640 1 -- 172.21.15.39:0/1311203808 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0660105a20 msgr2=0x7f066019bcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:46:09.131 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:09.125+0000 7f065e59c640 1 --2- 172.21.15.39:0/1311203808 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0660105a20 0x7f066019bcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:09.131 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:09.125+0000 7f065e59c640 1 -- 172.21.15.39:0/1311203808 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f06601050d0 msgr2=0x7f066019b790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:46:09.131 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:09.125+0000 7f065e59c640 1 --2- 172.21.15.39:0/1311203808 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f06601050d0 0x7f066019b790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:09.131 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:09.125+0000 7f065e59c640 1 -- 172.21.15.39:0/1311203808 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f06601a6480 con 0x7f06601031a0 2024-09-21T03:46:09.131 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:09.125+0000 7f065e59c640 1 --2- 172.21.15.39:0/1311203808 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f06601031a0 0x7f066019b250 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f0648005940 tx=0x7f06480042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:46:09.131 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:09.125+0000 7f06577fe640 1 -- 172.21.15.39:0/1311203808 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f064803f030 con 0x7f06601031a0 2024-09-21T03:46:09.131 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:09.125+0000 7f06577fe640 1 -- 172.21.15.39:0/1311203808 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0648003cb0 con 0x7f06601031a0 2024-09-21T03:46:09.131 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:09.125+0000 7f0664eb6640 1 -- 172.21.15.39:0/1311203808 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f06601a6710 con 0x7f06601031a0 2024-09-21T03:46:09.131 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:09.125+0000 7f06577fe640 1 -- 172.21.15.39:0/1311203808 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0648035070 con 0x7f06601031a0 2024-09-21T03:46:09.131 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:09.125+0000 7f0664eb6640 1 -- 172.21.15.39:0/1311203808 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f06601a6c00 con 0x7f06601031a0 2024-09-21T03:46:09.131 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:09.126+0000 7f0664eb6640 1 -- 172.21.15.39:0/1311203808 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f06601035b0 con 0x7f06601031a0 2024-09-21T03:46:09.132 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:09.128+0000 7f06577fe640 1 -- 172.21.15.39:0/1311203808 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 23) ==== 154529+0+0 (secure 0 0 0) 0x7f0648042030 con 0x7f06601031a0 2024-09-21T03:46:09.132 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:09.129+0000 7f06577fe640 1 --2- 172.21.15.39:0/1311203808 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f064c0b6560 0x7f064c0b8a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:09.132 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:09.129+0000 7f065dd9b640 1 --2- 172.21.15.39:0/1311203808 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f064c0b6560 0x7f064c0b8a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:46:09.132 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:09.129+0000 7f06577fe640 1 -- 172.21.15.39:0/1311203808 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8484+0+0 (secure 0 0 0) 0x7f0648107fc0 con 0x7f06601031a0 2024-09-21T03:46:09.132 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:09.129+0000 7f065dd9b640 1 --2- 172.21.15.39:0/1311203808 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f064c0b6560 0x7f064c0b8a70 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f066006afe0 tx=0x7f065000a2d0 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:46:09.135 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:09.132+0000 7f06577fe640 1 -- 172.21.15.39:0/1311203808 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f06480ce750 con 0x7f06601031a0 2024-09-21T03:46:09.270 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:46:08 smithi161 ceph-mon[34533]: pgmap v328: 97 pgs: 97 active+clean; 647 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 767 B/s wr, 10 op/s 2024-09-21T03:46:09.270 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:46:08 smithi161 ceph-mon[34533]: from='client.? 172.21.15.39:0/2964373851' entity='client.admin' cmd={"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"} : dispatch 2024-09-21T03:46:09.333 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:46:08 smithi039 ceph-mon[31211]: pgmap v328: 97 pgs: 97 active+clean; 647 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 767 B/s wr, 10 op/s 2024-09-21T03:46:09.334 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:46:08 smithi039 ceph-mon[31211]: from='client.? 172.21.15.39:0/2964373851' entity='client.admin' cmd={"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"} : dispatch 2024-09-21T03:46:09.351 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:09.348+0000 7f0664eb6640 1 -- 172.21.15.39:0/1311203808 --> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup pin", "vol_name": "cephfs", "group_name": "qa", "pin_type": "random", "pin_setting": "0.10", "target": ["mon-mgr", ""]}) -- 0x7f0660063800 con 0x7f064c0b6560 2024-09-21T03:46:09.376 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:09.373+0000 7f06577fe640 1 -- 172.21.15.39:0/1311203808 <== mgr.14223 v2:172.21.15.39:6800/1696381982 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+3 (secure 0 0 0) 0x7f0660063800 con 0x7f064c0b6560 2024-09-21T03:46:09.376 INFO:teuthology.orchestra.run.smithi039.stdout:{} 2024-09-21T03:46:09.379 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:09.376+0000 7f0664eb6640 1 -- 172.21.15.39:0/1311203808 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f064c0b6560 msgr2=0x7f064c0b8a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:46:09.379 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:09.376+0000 7f0664eb6640 1 --2- 172.21.15.39:0/1311203808 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f064c0b6560 0x7f064c0b8a70 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f066006afe0 tx=0x7f065000a2d0 comp rx=0 tx=0).stop 2024-09-21T03:46:09.379 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:09.376+0000 7f0664eb6640 1 -- 172.21.15.39:0/1311203808 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f06601031a0 msgr2=0x7f066019b250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:46:09.379 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:09.376+0000 7f0664eb6640 1 --2- 172.21.15.39:0/1311203808 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f06601031a0 0x7f066019b250 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f0648005940 tx=0x7f06480042f0 comp rx=0 tx=0).stop 2024-09-21T03:46:09.379 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:09.376+0000 7f0664eb6640 1 -- 172.21.15.39:0/1311203808 shutdown_connections 2024-09-21T03:46:09.379 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:09.376+0000 7f0664eb6640 1 --2- 172.21.15.39:0/1311203808 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f064c0b6560 0x7f064c0b8a70 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:09.379 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:09.376+0000 7f0664eb6640 1 --2- 172.21.15.39:0/1311203808 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0660105a20 0x7f066019bcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:09.379 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:09.376+0000 7f0664eb6640 1 --2- 172.21.15.39:0/1311203808 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f06601050d0 0x7f066019b790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:09.379 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:09.376+0000 7f0664eb6640 1 --2- 172.21.15.39:0/1311203808 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f06601031a0 0x7f066019b250 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:09.380 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:09.376+0000 7f0664eb6640 1 -- 172.21.15.39:0/1311203808 >> 172.21.15.39:0/1311203808 conn(0x7f0660079ec0 msgr2=0x7f0660107290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:46:09.380 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:09.376+0000 7f0664eb6640 1 -- 172.21.15.39:0/1311203808 shutdown_connections 2024-09-21T03:46:09.380 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:09.376+0000 7f0664eb6640 1 -- 172.21.15.39:0/1311203808 wait complete. 2024-09-21T03:46:09.390 INFO:teuthology.run_tasks:Running task check-counter... 2024-09-21T03:46:09.403 INFO:teuthology.run_tasks:Running task fwd_scrub... 2024-09-21T03:46:09.451 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T03:46:09.474 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:46:08 smithi150 ceph-mon[34289]: pgmap v328: 97 pgs: 97 active+clean; 647 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 767 B/s wr, 10 op/s 2024-09-21T03:46:09.474 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:46:08 smithi150 ceph-mon[34289]: from='client.? 172.21.15.39:0/2964373851' entity='client.admin' cmd={"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"} : dispatch 2024-09-21T03:46:09.565 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:09.562+0000 7f2ec1e0e640 1 -- 172.21.15.39:0/1012095163 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2ebc108c30 msgr2=0x7f2ebc10f460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:46:09.566 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:09.562+0000 7f2ec1e0e640 1 --2- 172.21.15.39:0/1012095163 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2ebc108c30 0x7f2ebc10f460 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f2eb4009ca0 tx=0x7f2eb402a8c0 comp rx=0 tx=0).stop 2024-09-21T03:46:09.566 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:09.562+0000 7f2ec1e0e640 1 -- 172.21.15.39:0/1012095163 shutdown_connections 2024-09-21T03:46:09.566 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:09.562+0000 7f2ec1e0e640 1 --2- 172.21.15.39:0/1012095163 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2ebc108c30 0x7f2ebc10f460 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:09.566 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:09.562+0000 7f2ec1e0e640 1 --2- 172.21.15.39:0/1012095163 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2ebc1082e0 0x7f2ebc1086f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:09.566 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:09.562+0000 7f2ec1e0e640 1 --2- 172.21.15.39:0/1012095163 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2ebc1029b0 0x7f2ebc102dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:09.566 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:09.562+0000 7f2ec1e0e640 1 -- 172.21.15.39:0/1012095163 >> 172.21.15.39:0/1012095163 conn(0x7f2ebc0fe3b0 msgr2=0x7f2ebc100800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:46:09.566 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:09.562+0000 7f2ec1e0e640 1 -- 172.21.15.39:0/1012095163 shutdown_connections 2024-09-21T03:46:09.566 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:09.562+0000 7f2ec1e0e640 1 -- 172.21.15.39:0/1012095163 wait complete. 2024-09-21T03:46:09.566 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:09.563+0000 7f2ec1e0e640 1 Processor -- start 2024-09-21T03:46:09.568 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:09.563+0000 7f2ec1e0e640 1 -- start start 2024-09-21T03:46:09.568 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:09.564+0000 7f2ec1e0e640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2ebc1029b0 0x7f2ebc199330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:09.568 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:09.564+0000 7f2ec1e0e640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2ebc1082e0 0x7f2ebc1998d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:09.568 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:09.564+0000 7f2ec1e0e640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2ebc108c30 0x7f2ebc19eb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:09.568 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:09.564+0000 7f2ec1e0e640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f2ebc1128f0 con 0x7f2ebc1029b0 2024-09-21T03:46:09.568 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:09.564+0000 7f2ec1e0e640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f2ebc112770 con 0x7f2ebc108c30 2024-09-21T03:46:09.569 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:09.564+0000 7f2ec1e0e640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f2ebc112a70 con 0x7f2ebc1082e0 2024-09-21T03:46:09.569 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:09.564+0000 7f2ebb7fe640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2ebc1029b0 0x7f2ebc199330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:46:09.569 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:09.564+0000 7f2ebb7fe640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2ebc1029b0 0x7f2ebc199330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:43106/0 (socket says 172.21.15.39:43106) 2024-09-21T03:46:09.569 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:09.564+0000 7f2ebb7fe640 1 -- 172.21.15.39:0/947925955 learned_addr learned my addr 172.21.15.39:0/947925955 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:46:09.569 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:09.564+0000 7f2ebaffd640 1 --2- 172.21.15.39:0/947925955 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2ebc1082e0 0x7f2ebc1998d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:46:09.569 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:09.564+0000 7f2ebbfff640 1 --2- 172.21.15.39:0/947925955 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2ebc108c30 0x7f2ebc19eb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:46:09.569 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:09.564+0000 7f2ebb7fe640 1 -- 172.21.15.39:0/947925955 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2ebc1082e0 msgr2=0x7f2ebc1998d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:46:09.569 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:09.564+0000 7f2ebb7fe640 1 --2- 172.21.15.39:0/947925955 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2ebc1082e0 0x7f2ebc1998d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:09.569 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:09.564+0000 7f2ebb7fe640 1 -- 172.21.15.39:0/947925955 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2ebc108c30 msgr2=0x7f2ebc19eb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:46:09.569 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:09.564+0000 7f2ebb7fe640 1 --2- 172.21.15.39:0/947925955 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2ebc108c30 0x7f2ebc19eb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:09.569 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:09.564+0000 7f2ebb7fe640 1 -- 172.21.15.39:0/947925955 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ebc19f260 con 0x7f2ebc1029b0 2024-09-21T03:46:09.569 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:09.564+0000 7f2ebaffd640 1 --2- 172.21.15.39:0/947925955 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2ebc1082e0 0x7f2ebc1998d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:46:09.569 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:09.564+0000 7f2ebb7fe640 1 --2- 172.21.15.39:0/947925955 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2ebc1029b0 0x7f2ebc199330 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f2ea400d950 tx=0x7f2ea400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:46:09.569 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:09.564+0000 7f2eb8ff9640 1 -- 172.21.15.39:0/947925955 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ea4017070 con 0x7f2ebc1029b0 2024-09-21T03:46:09.570 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:09.564+0000 7f2ec1e0e640 1 -- 172.21.15.39:0/947925955 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2ebc19f550 con 0x7f2ebc1029b0 2024-09-21T03:46:09.570 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:09.564+0000 7f2eb8ff9640 1 -- 172.21.15.39:0/947925955 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2ea4009c30 con 0x7f2ebc1029b0 2024-09-21T03:46:09.570 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:09.564+0000 7f2eb8ff9640 1 -- 172.21.15.39:0/947925955 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ea400f620 con 0x7f2ebc1029b0 2024-09-21T03:46:09.570 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:09.564+0000 7f2ec1e0e640 1 -- 172.21.15.39:0/947925955 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2ebc1a5ce0 con 0x7f2ebc1029b0 2024-09-21T03:46:09.570 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:09.566+0000 7f2ec1e0e640 1 -- 172.21.15.39:0/947925955 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2ebc102dc0 con 0x7f2ebc1029b0 2024-09-21T03:46:09.571 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:09.567+0000 7f2eb8ff9640 1 -- 172.21.15.39:0/947925955 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 23) ==== 154529+0+0 (secure 0 0 0) 0x7f2ea4010460 con 0x7f2ebc1029b0 2024-09-21T03:46:09.571 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:09.568+0000 7f2eb8ff9640 1 --2- 172.21.15.39:0/947925955 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f2ea80b6560 0x7f2ea80b8a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:09.571 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:09.568+0000 7f2ebaffd640 1 --2- 172.21.15.39:0/947925955 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f2ea80b6560 0x7f2ea80b8a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:46:09.571 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:09.568+0000 7f2eb8ff9640 1 -- 172.21.15.39:0/947925955 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8484+0+0 (secure 0 0 0) 0x7f2ea40e70f0 con 0x7f2ebc1029b0 2024-09-21T03:46:09.571 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:09.568+0000 7f2ebaffd640 1 --2- 172.21.15.39:0/947925955 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f2ea80b6560 0x7f2ea80b8a70 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f2eac005fd0 tx=0x7f2eac00b5d0 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:46:09.574 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:09.572+0000 7f2eb8ff9640 1 -- 172.21.15.39:0/947925955 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f2ea40ad920 con 0x7f2ebc1029b0 2024-09-21T03:46:09.766 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:09.763+0000 7f2ec1e0e640 1 -- 172.21.15.39:0/947925955 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f2ebc10a490 con 0x7f2ebc1029b0 2024-09-21T03:46:09.767 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:09.765+0000 7f2eb8ff9640 1 -- 172.21.15.39:0/947925955 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v79) ==== 74+0+23096 (secure 0 0 0) 0x7f2ea4009de0 con 0x7f2ebc1029b0 2024-09-21T03:46:09.768 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T03:46:09.768 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":79,"fsid":"51116d5c-77ca-11ef-baf5-efdc52797490","created":"2024-09-21T03:35:33.631350+0000","modified":"2024-09-21T03:44:27.402588+0000","last_up_change":"2024-09-21T03:43:24.242275+0000","last_in_change":"2024-09-21T03:43:00.886552+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T03:39:29.859542+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-21T03:44:10.886435+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"77","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-21T03:44:13.041783+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"77","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1ea7148f-2ace-4c10-86fe-b8019de80c1f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6802","nonce":870188791},{"type":"v1","addr":"172.21.15.39:6803","nonce":870188791}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6804","nonce":870188791},{"type":"v1","addr":"172.21.15.39:6805","nonce":870188791}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6808","nonce":870188791},{"type":"v1","addr":"172.21.15.39:6809","nonce":870188791}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6806","nonce":870188791},{"type":"v1","addr":"172.21.15.39:6807","nonce":870188791}]},"public_addr":"172.21.15.39:6803/870188791","cluster_addr":"172.21.15.39:6805/870188791","heartbeat_back_addr":"172.21.15.39:6809/870188791","heartbeat_front_addr":"172.21.15.39:6807/870188791","state":["exists","up"]},{"osd":1,"uuid":"7cb5db90-28e8-4a30-bedb-c039687d48b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6800","nonce":172427832},{"type":"v1","addr":"172.21.15.150:6801","nonce":172427832}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6802","nonce":172427832},{"type":"v1","addr":"172.21.15.150:6803","nonce":172427832}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6806","nonce":172427832},{"type":"v1","addr":"172.21.15.150:6807","nonce":172427832}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6804","nonce":172427832},{"type":"v1","addr":"172.21.15.150:6805","nonce":172427832}]},"public_addr":"172.21.15.150:6801/172427832","cluster_addr":"172.21.15.150:6803/172427832","heartbeat_back_addr":"172.21.15.150:6807/172427832","heartbeat_front_addr":"172.21.15.150:6805/172427832","state":["exists","up"]},{"osd":2,"uuid":"b25c6ab3-fc95-4751-bd63-6e3fc23e8bd7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":19,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6800","nonce":2727876133},{"type":"v1","addr":"172.21.15.161:6801","nonce":2727876133}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6802","nonce":2727876133},{"type":"v1","addr":"172.21.15.161:6803","nonce":2727876133}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6806","nonce":2727876133},{"type":"v1","addr":"172.21.15.161:6807","nonce":2727876133}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6804","nonce":2727876133},{"type":"v1","addr":"172.21.15.161:6805","nonce":2727876133}]},"public_addr":"172.21.15.161:6801/2727876133","cluster_addr":"172.21.15.161:6803/2727876133","heartbeat_back_addr":"172.21.15.161:6807/2727876133","heartbeat_front_addr":"172.21.15.161:6805/2727876133","state":["exists","up"]},{"osd":3,"uuid":"14417ae1-4536-45ba-9a5c-c3b447d1a72d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6810","nonce":295394468},{"type":"v1","addr":"172.21.15.39:6811","nonce":295394468}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6812","nonce":295394468},{"type":"v1","addr":"172.21.15.39:6813","nonce":295394468}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6816","nonce":295394468},{"type":"v1","addr":"172.21.15.39:6817","nonce":295394468}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6814","nonce":295394468},{"type":"v1","addr":"172.21.15.39:6815","nonce":295394468}]},"public_addr":"172.21.15.39:6811/295394468","cluster_addr":"172.21.15.39:6813/295394468","heartbeat_back_addr":"172.21.15.39:6817/295394468","heartbeat_front_addr":"172.21.15.39:6815/295394468","state":["exists","up"]},{"osd":4,"uuid":"33c732ef-20c2-43c4-abe6-045b687fc521","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6808","nonce":3097721553},{"type":"v1","addr":"172.21.15.150:6809","nonce":3097721553}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6810","nonce":3097721553},{"type":"v1","addr":"172.21.15.150:6811","nonce":3097721553}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6814","nonce":3097721553},{"type":"v1","addr":"172.21.15.150:6815","nonce":3097721553}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6812","nonce":3097721553},{"type":"v1","addr":"172.21.15.150:6813","nonce":3097721553}]},"public_addr":"172.21.15.150:6809/3097721553","cluster_addr":"172.21.15.150:6811/3097721553","heartbeat_back_addr":"172.21.15.150:6815/3097721553","heartbeat_front_addr":"172.21.15.150:6813/3097721553","state":["exists","up"]},{"osd":5,"uuid":"74673722-abd6-400a-bb50-ec3dfba0c5e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6808","nonce":742751023},{"type":"v1","addr":"172.21.15.161:6809","nonce":742751023}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6810","nonce":742751023},{"type":"v1","addr":"172.21.15.161:6811","nonce":742751023}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6814","nonce":742751023},{"type":"v1","addr":"172.21.15.161:6815","nonce":742751023}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6812","nonce":742751023},{"type":"v1","addr":"172.21.15.161:6813","nonce":742751023}]},"public_addr":"172.21.15.161:6809/742751023","cluster_addr":"172.21.15.161:6811/742751023","heartbeat_back_addr":"172.21.15.161:6815/742751023","heartbeat_front_addr":"172.21.15.161:6813/742751023","state":["exists","up"]},{"osd":6,"uuid":"06548677-74c4-4efa-8054-df09b957b649","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6818","nonce":1072700290},{"type":"v1","addr":"172.21.15.39:6819","nonce":1072700290}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6820","nonce":1072700290},{"type":"v1","addr":"172.21.15.39:6821","nonce":1072700290}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6824","nonce":1072700290},{"type":"v1","addr":"172.21.15.39:6825","nonce":1072700290}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6822","nonce":1072700290},{"type":"v1","addr":"172.21.15.39:6823","nonce":1072700290}]},"public_addr":"172.21.15.39:6819/1072700290","cluster_addr":"172.21.15.39:6821/1072700290","heartbeat_back_addr":"172.21.15.39:6825/1072700290","heartbeat_front_addr":"172.21.15.39:6823/1072700290","state":["exists","up"]},{"osd":7,"uuid":"838735ca-c693-4f58-9426-b8ee857dbe3a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6816","nonce":2162619103},{"type":"v1","addr":"172.21.15.150:6817","nonce":2162619103}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6818","nonce":2162619103},{"type":"v1","addr":"172.21.15.150:6819","nonce":2162619103}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6822","nonce":2162619103},{"type":"v1","addr":"172.21.15.150:6823","nonce":2162619103}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6820","nonce":2162619103},{"type":"v1","addr":"172.21.15.150:6821","nonce":2162619103}]},"public_addr":"172.21.15.150:6817/2162619103","cluster_addr":"172.21.15.150:6819/2162619103","heartbeat_back_addr":"172.21.15.150:6823/2162619103","heartbeat_front_addr":"172.21.15.150:6821/2162619103","state":["exists","up"]},{"osd":8,"uuid":"4973b357-b4c2-4c00-b7f7-dd25012a0f72","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6816","nonce":52966633},{"type":"v1","addr":"172.21.15.161:6817","nonce":52966633}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6818","nonce":52966633},{"type":"v1","addr":"172.21.15.161:6819","nonce":52966633}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6822","nonce":52966633},{"type":"v1","addr":"172.21.15.161:6823","nonce":52966633}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6820","nonce":52966633},{"type":"v1","addr":"172.21.15.161:6821","nonce":52966633}]},"public_addr":"172.21.15.161:6817/52966633","cluster_addr":"172.21.15.161:6819/52966633","heartbeat_back_addr":"172.21.15.161:6823/52966633","heartbeat_front_addr":"172.21.15.161:6821/52966633","state":["exists","up"]},{"osd":9,"uuid":"6028613c-ebfd-42ea-9949-3f0d5f6cf025","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6826","nonce":2001303253},{"type":"v1","addr":"172.21.15.39:6827","nonce":2001303253}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6828","nonce":2001303253},{"type":"v1","addr":"172.21.15.39:6829","nonce":2001303253}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6832","nonce":2001303253},{"type":"v1","addr":"172.21.15.39:6833","nonce":2001303253}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6830","nonce":2001303253},{"type":"v1","addr":"172.21.15.39:6831","nonce":2001303253}]},"public_addr":"172.21.15.39:6827/2001303253","cluster_addr":"172.21.15.39:6829/2001303253","heartbeat_back_addr":"172.21.15.39:6833/2001303253","heartbeat_front_addr":"172.21.15.39:6831/2001303253","state":["exists","up"]},{"osd":10,"uuid":"1c71c765-2a6c-4ffe-9fe1-c8adae3d1c32","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6824","nonce":2484627938},{"type":"v1","addr":"172.21.15.150:6825","nonce":2484627938}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6826","nonce":2484627938},{"type":"v1","addr":"172.21.15.150:6827","nonce":2484627938}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6830","nonce":2484627938},{"type":"v1","addr":"172.21.15.150:6831","nonce":2484627938}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6828","nonce":2484627938},{"type":"v1","addr":"172.21.15.150:6829","nonce":2484627938}]},"public_addr":"172.21.15.150:6825/2484627938","cluster_addr":"172.21.15.150:6827/2484627938","heartbeat_back_addr":"172.21.15.150:6831/2484627938","heartbeat_front_addr":"172.21.15.150:6829/2484627938","state":["exists","up"]},{"osd":11,"uuid":"f382f522-62db-46da-997e-2b28f84d1612","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6824","nonce":2178709707},{"type":"v1","addr":"172.21.15.161:6825","nonce":2178709707}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6826","nonce":2178709707},{"type":"v1","addr":"172.21.15.161:6827","nonce":2178709707}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6830","nonce":2178709707},{"type":"v1","addr":"172.21.15.161:6831","nonce":2178709707}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6828","nonce":2178709707},{"type":"v1","addr":"172.21.15.161:6829","nonce":2178709707}]},"public_addr":"172.21.15.161:6825/2178709707","cluster_addr":"172.21.15.161:6827/2178709707","heartbeat_back_addr":"172.21.15.161:6831/2178709707","heartbeat_front_addr":"172.21.15.161:6829/2178709707","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:38:35.540904+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:39:01.455053+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:39:27.052867+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:39:55.173529+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:40:21.298396+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:40:47.131235+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:41:14.475209+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:41:39.920036+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:42:05.587065+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:42:32.624726+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:42:57.479185+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:43:22.203832+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.5","mappings":[{"from":8,"to":11}]},{"pgid":"2.8","mappings":[{"from":2,"to":11}]},{"pgid":"2.c","mappings":[{"from":2,"to":1}]},{"pgid":"2.1f","mappings":[{"from":8,"to":4}]},{"pgid":"3.2","mappings":[{"from":3,"to":9}]},{"pgid":"3.14","mappings":[{"from":4,"to":6}]},{"pgid":"3.1a","mappings":[{"from":5,"to":7}]},{"pgid":"3.1d","mappings":[{"from":4,"to":8}]},{"pgid":"3.2c","mappings":[{"from":4,"to":9}]},{"pgid":"3.38","mappings":[{"from":3,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.39:6801/2438110611":"2024-09-22T03:37:25.391457+0000","172.21.15.39:6800/2438110611":"2024-09-22T03:37:25.391457+0000","172.21.15.39:6800/173264695":"2024-09-22T03:35:57.104467+0000","172.21.15.39:6801/173264695":"2024-09-22T03:35:57.104467+0000","172.21.15.39:0/2637182284":"2024-09-22T03:35:57.104467+0000","172.21.15.39:0/1517304561":"2024-09-22T03:36:24.112592+0000","172.21.15.39:6801/1435622199":"2024-09-22T03:36:24.112592+0000","172.21.15.39:6800/1435622199":"2024-09-22T03:36:24.112592+0000","172.21.15.39:0/346832469":"2024-09-22T03:35:57.104467+0000","172.21.15.39:0/3160777371":"2024-09-22T03:36:24.112592+0000","172.21.15.39:0/1730342855":"2024-09-22T03:35:57.104467+0000","172.21.15.39:0/1955726766":"2024-09-22T03:36:24.112592+0000","172.21.15.39:0/3294132796":"2024-09-22T03:37:25.391457+0000","172.21.15.39:0/1295810162":"2024-09-22T03:37:25.391457+0000","172.21.15.39:0/3729230474":"2024-09-22T03:37:25.391457+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T03:46:09.770 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:09.767+0000 7f2ec1e0e640 1 -- 172.21.15.39:0/947925955 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f2ea80b6560 msgr2=0x7f2ea80b8a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:46:09.770 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:09.767+0000 7f2ec1e0e640 1 --2- 172.21.15.39:0/947925955 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f2ea80b6560 0x7f2ea80b8a70 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f2eac005fd0 tx=0x7f2eac00b5d0 comp rx=0 tx=0).stop 2024-09-21T03:46:09.770 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:09.767+0000 7f2ec1e0e640 1 -- 172.21.15.39:0/947925955 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2ebc1029b0 msgr2=0x7f2ebc199330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:46:09.770 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:09.767+0000 7f2ec1e0e640 1 --2- 172.21.15.39:0/947925955 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2ebc1029b0 0x7f2ebc199330 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f2ea400d950 tx=0x7f2ea400de10 comp rx=0 tx=0).stop 2024-09-21T03:46:09.770 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:09.767+0000 7f2ec1e0e640 1 -- 172.21.15.39:0/947925955 shutdown_connections 2024-09-21T03:46:09.770 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:09.767+0000 7f2ec1e0e640 1 --2- 172.21.15.39:0/947925955 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f2ea80b6560 0x7f2ea80b8a70 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:09.771 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:09.767+0000 7f2ec1e0e640 1 --2- 172.21.15.39:0/947925955 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2ebc108c30 0x7f2ebc19eb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:09.771 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:09.767+0000 7f2ec1e0e640 1 --2- 172.21.15.39:0/947925955 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2ebc1082e0 0x7f2ebc1998d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:09.771 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:09.767+0000 7f2ec1e0e640 1 --2- 172.21.15.39:0/947925955 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2ebc1029b0 0x7f2ebc199330 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:09.771 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:09.767+0000 7f2ec1e0e640 1 -- 172.21.15.39:0/947925955 >> 172.21.15.39:0/947925955 conn(0x7f2ebc0fe3b0 msgr2=0x7f2ebc1078b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:46:09.771 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:09.767+0000 7f2ec1e0e640 1 -- 172.21.15.39:0/947925955 shutdown_connections 2024-09-21T03:46:09.771 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:09.767+0000 7f2ec1e0e640 1 -- 172.21.15.39:0/947925955 wait complete. 2024-09-21T03:46:09.781 INFO:journalctl@ceph.mgr.x.smithi039.stdout:Sep 21 03:46:09 smithi039 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-mgr-x[31438]: 2024-09-21T03:46:09.353+0000 7fe42e8a0640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T03:46:09.781 INFO:journalctl@ceph.mgr.x.smithi039.stdout:Sep 21 03:46:09 smithi039 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-mgr-x[31438]: 2024-09-21T03:46:09.353+0000 7fe42e8a0640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T03:46:09.781 INFO:journalctl@ceph.mgr.x.smithi039.stdout:Sep 21 03:46:09 smithi039 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-mgr-x[31438]: 2024-09-21T03:46:09.353+0000 7fe42e8a0640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T03:46:09.781 INFO:journalctl@ceph.mgr.x.smithi039.stdout:Sep 21 03:46:09 smithi039 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-mgr-x[31438]: 2024-09-21T03:46:09.353+0000 7fe42e8a0640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T03:46:09.781 INFO:journalctl@ceph.mgr.x.smithi039.stdout:Sep 21 03:46:09 smithi039 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-mgr-x[31438]: 2024-09-21T03:46:09.353+0000 7fe42e8a0640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T03:46:09.782 INFO:tasks.fwd_scrub.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-21T03:39:29.859542+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-09-21T03:44:10.886435+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '77', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-09-21T03:44:13.041783+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '77', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-09-21T03:46:09.783 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-21T03:46:09.900 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:09.897+0000 7ff5ccb94640 1 -- 172.21.15.39:0/1471803620 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff5c8105240 msgr2=0x7ff5c810d780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:46:09.900 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:09.897+0000 7ff5ccb94640 1 --2- 172.21.15.39:0/1471803620 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff5c8105240 0x7ff5c810d780 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7ff5bc009900 tx=0x7ff5bc02a410 comp rx=0 tx=0).stop 2024-09-21T03:46:09.900 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:09.897+0000 7ff5ccb94640 1 -- 172.21.15.39:0/1471803620 shutdown_connections 2024-09-21T03:46:09.900 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:09.897+0000 7ff5ccb94640 1 --2- 172.21.15.39:0/1471803620 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff5c810dd50 0x7ff5c8110240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:09.900 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:09.897+0000 7ff5ccb94640 1 --2- 172.21.15.39:0/1471803620 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff5c8105240 0x7ff5c810d780 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:09.900 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:09.897+0000 7ff5ccb94640 1 --2- 172.21.15.39:0/1471803620 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff5c81048f0 0x7ff5c8104d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:09.900 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:09.897+0000 7ff5ccb94640 1 -- 172.21.15.39:0/1471803620 >> 172.21.15.39:0/1471803620 conn(0x7ff5c80fe3c0 msgr2=0x7ff5c8100810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:46:09.900 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:09.897+0000 7ff5ccb94640 1 -- 172.21.15.39:0/1471803620 shutdown_connections 2024-09-21T03:46:09.900 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:09.897+0000 7ff5ccb94640 1 -- 172.21.15.39:0/1471803620 wait complete. 2024-09-21T03:46:09.902 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:09.898+0000 7ff5ccb94640 1 Processor -- start 2024-09-21T03:46:09.903 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:09.898+0000 7ff5ccb94640 1 -- start start 2024-09-21T03:46:09.903 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:09.898+0000 7ff5ccb94640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff5c81048f0 0x7ff5c819a3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:09.903 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:09.898+0000 7ff5ccb94640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff5c8105240 0x7ff5c819a990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:09.903 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:09.898+0000 7ff5ccb94640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff5c810dd50 0x7ff5c819fc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:09.903 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:09.898+0000 7ff5ccb94640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7ff5c81139b0 con 0x7ff5c810dd50 2024-09-21T03:46:09.903 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:09.898+0000 7ff5ccb94640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7ff5c8113830 con 0x7ff5c8105240 2024-09-21T03:46:09.903 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:09.898+0000 7ff5ccb94640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7ff5c8113b30 con 0x7ff5c81048f0 2024-09-21T03:46:09.903 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:09.898+0000 7ff5c6d9d640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff5c810dd50 0x7ff5c819fc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:46:09.903 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:09.898+0000 7ff5c6d9d640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff5c810dd50 0x7ff5c819fc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:43124/0 (socket says 172.21.15.39:43124) 2024-09-21T03:46:09.903 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:09.898+0000 7ff5c6d9d640 1 -- 172.21.15.39:0/2022635456 learned_addr learned my addr 172.21.15.39:0/2022635456 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:46:09.903 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:09.898+0000 7ff5c659c640 1 --2- 172.21.15.39:0/2022635456 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff5c81048f0 0x7ff5c819a3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:46:09.903 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:09.898+0000 7ff5c5d9b640 1 --2- 172.21.15.39:0/2022635456 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff5c8105240 0x7ff5c819a990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:46:09.904 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:09.898+0000 7ff5c6d9d640 1 -- 172.21.15.39:0/2022635456 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff5c81048f0 msgr2=0x7ff5c819a3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:46:09.904 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:09.898+0000 7ff5c6d9d640 1 --2- 172.21.15.39:0/2022635456 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff5c81048f0 0x7ff5c819a3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:09.904 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:09.898+0000 7ff5c6d9d640 1 -- 172.21.15.39:0/2022635456 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff5c8105240 msgr2=0x7ff5c819a990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:46:09.904 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:09.898+0000 7ff5c6d9d640 1 --2- 172.21.15.39:0/2022635456 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff5c8105240 0x7ff5c819a990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:09.904 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:09.898+0000 7ff5c6d9d640 1 -- 172.21.15.39:0/2022635456 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff5c81a0290 con 0x7ff5c810dd50 2024-09-21T03:46:09.904 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:09.899+0000 7ff5c6d9d640 1 --2- 172.21.15.39:0/2022635456 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff5c810dd50 0x7ff5c819fc20 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7ff5c000cce0 tx=0x7ff5c000a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:46:09.904 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:09.899+0000 7ff5b77fe640 1 -- 172.21.15.39:0/2022635456 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5c000f450 con 0x7ff5c810dd50 2024-09-21T03:46:09.905 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:09.899+0000 7ff5b77fe640 1 -- 172.21.15.39:0/2022635456 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff5c000faa0 con 0x7ff5c810dd50 2024-09-21T03:46:09.906 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:09.899+0000 7ff5ccb94640 1 -- 172.21.15.39:0/2022635456 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff5c81a0580 con 0x7ff5c810dd50 2024-09-21T03:46:09.906 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:09.899+0000 7ff5b77fe640 1 -- 172.21.15.39:0/2022635456 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5c0018b70 con 0x7ff5c810dd50 2024-09-21T03:46:09.906 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:09.899+0000 7ff5ccb94640 1 -- 172.21.15.39:0/2022635456 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff5c81a0a50 con 0x7ff5c810dd50 2024-09-21T03:46:09.906 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:09.900+0000 7ff5ccb94640 1 -- 172.21.15.39:0/2022635456 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff5c806bfa0 con 0x7ff5c810dd50 2024-09-21T03:46:09.906 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:09.902+0000 7ff5b77fe640 1 -- 172.21.15.39:0/2022635456 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 23) ==== 154529+0+0 (secure 0 0 0) 0x7ff5c0018d10 con 0x7ff5c810dd50 2024-09-21T03:46:09.907 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:09.902+0000 7ff5b77fe640 1 --2- 172.21.15.39:0/2022635456 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7ff5b00b6560 0x7ff5b00b8a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:09.907 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:09.902+0000 7ff5c659c640 1 --2- 172.21.15.39:0/2022635456 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7ff5b00b6560 0x7ff5b00b8a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:46:09.907 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:09.903+0000 7ff5b77fe640 1 -- 172.21.15.39:0/2022635456 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8484+0+0 (secure 0 0 0) 0x7ff5c00e78d0 con 0x7ff5c810dd50 2024-09-21T03:46:09.907 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:09.903+0000 7ff5c659c640 1 --2- 172.21.15.39:0/2022635456 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7ff5b00b6560 0x7ff5b00b8a70 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7ff5b8005fd0 tx=0x7ff5b800b390 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:46:09.908 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:09.905+0000 7ff5b77fe640 1 -- 172.21.15.39:0/2022635456 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7ff5c81a0a50 con 0x7ff5c810dd50 2024-09-21T03:46:10.015 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:10.013+0000 7ff5b77fe640 1 -- 172.21.15.39:0/2022635456 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mgrmap(e 24) ==== 154580+0+0 (secure 0 0 0) 0x7ff5c00a9c60 con 0x7ff5c810dd50 2024-09-21T03:46:10.083 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:46:10 smithi039 ceph-mon[31211]: from='client.? 172.21.15.39:0/2964373851' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]': finished 2024-09-21T03:46:10.084 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:46:10 smithi039 ceph-mon[31211]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-21T03:46:10.084 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:46:10 smithi039 ceph-mon[31211]: from='client.? 172.21.15.39:0/947925955' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T03:46:10.104 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:10.101+0000 7ff5ccb94640 1 -- 172.21.15.39:0/2022635456 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7ff5c819b510 con 0x7ff5c810dd50 2024-09-21T03:46:10.104 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:10.102+0000 7ff5b77fe640 1 -- 172.21.15.39:0/2022635456 <== mon.0 v2:172.21.15.39:3300/0 8 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v79) ==== 93+0+10 (secure 0 0 0) 0x7ff5c0014030 con 0x7ff5c810dd50 2024-09-21T03:46:10.105 INFO:teuthology.orchestra.run.smithi039.stdout:pg_num: 1 2024-09-21T03:46:10.107 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:10.105+0000 7ff5ccb94640 1 -- 172.21.15.39:0/2022635456 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7ff5b00b6560 msgr2=0x7ff5b00b8a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:46:10.108 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:10.105+0000 7ff5ccb94640 1 --2- 172.21.15.39:0/2022635456 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7ff5b00b6560 0x7ff5b00b8a70 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7ff5b8005fd0 tx=0x7ff5b800b390 comp rx=0 tx=0).stop 2024-09-21T03:46:10.108 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:10.105+0000 7ff5ccb94640 1 -- 172.21.15.39:0/2022635456 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff5c810dd50 msgr2=0x7ff5c819fc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:46:10.108 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:10.105+0000 7ff5ccb94640 1 --2- 172.21.15.39:0/2022635456 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff5c810dd50 0x7ff5c819fc20 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7ff5c000cce0 tx=0x7ff5c000a620 comp rx=0 tx=0).stop 2024-09-21T03:46:10.108 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:10.105+0000 7ff5ccb94640 1 -- 172.21.15.39:0/2022635456 shutdown_connections 2024-09-21T03:46:10.108 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:10.105+0000 7ff5ccb94640 1 --2- 172.21.15.39:0/2022635456 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7ff5b00b6560 0x7ff5b00b8a70 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:10.108 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:10.105+0000 7ff5ccb94640 1 --2- 172.21.15.39:0/2022635456 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff5c810dd50 0x7ff5c819fc20 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:10.108 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:10.105+0000 7ff5ccb94640 1 --2- 172.21.15.39:0/2022635456 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff5c8105240 0x7ff5c819a990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:10.108 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:10.105+0000 7ff5ccb94640 1 --2- 172.21.15.39:0/2022635456 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff5c81048f0 0x7ff5c819a3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:10.108 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:10.105+0000 7ff5ccb94640 1 -- 172.21.15.39:0/2022635456 >> 172.21.15.39:0/2022635456 conn(0x7ff5c80fe3c0 msgr2=0x7ff5c8107860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:46:10.108 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:10.105+0000 7ff5ccb94640 1 -- 172.21.15.39:0/2022635456 shutdown_connections 2024-09-21T03:46:10.108 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:10.105+0000 7ff5ccb94640 1 -- 172.21.15.39:0/2022635456 wait complete. 2024-09-21T03:46:10.120 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-09-21T03:46:10.238 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:10.235+0000 7fd0254c5640 1 -- 172.21.15.39:0/1874450656 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd020107220 msgr2=0x7fd020107630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:46:10.239 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:10.235+0000 7fd0254c5640 1 --2- 172.21.15.39:0/1874450656 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd020107220 0x7fd020107630 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7fd008009900 tx=0x7fd00802a410 comp rx=0 tx=0).stop 2024-09-21T03:46:10.239 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:10.235+0000 7fd0254c5640 1 -- 172.21.15.39:0/1874450656 shutdown_connections 2024-09-21T03:46:10.239 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:10.235+0000 7fd0254c5640 1 --2- 172.21.15.39:0/1874450656 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd020107b70 0x7fd02010f450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:10.239 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:10.235+0000 7fd0254c5640 1 --2- 172.21.15.39:0/1874450656 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd020107220 0x7fd020107630 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:10.239 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:10.235+0000 7fd0254c5640 1 --2- 172.21.15.39:0/1874450656 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd020108540 0x7fd020108950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:10.239 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:10.235+0000 7fd0254c5640 1 -- 172.21.15.39:0/1874450656 >> 172.21.15.39:0/1874450656 conn(0x7fd0200765f0 msgr2=0x7fd020076a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:46:10.239 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:10.236+0000 7fd0254c5640 1 -- 172.21.15.39:0/1874450656 shutdown_connections 2024-09-21T03:46:10.239 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:10.236+0000 7fd0254c5640 1 -- 172.21.15.39:0/1874450656 wait complete. 2024-09-21T03:46:10.241 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:10.236+0000 7fd0254c5640 1 Processor -- start 2024-09-21T03:46:10.241 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:10.236+0000 7fd0254c5640 1 -- start start 2024-09-21T03:46:10.241 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:10.237+0000 7fd0254c5640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd020107220 0x7fd02019a410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:10.241 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:10.237+0000 7fd0254c5640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd020107b70 0x7fd02019a9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:10.242 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:10.237+0000 7fd0254c5640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd020108540 0x7fd02019fc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:10.242 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:10.237+0000 7fd0254c5640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fd0201128e0 con 0x7fd020108540 2024-09-21T03:46:10.242 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:10.237+0000 7fd0254c5640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fd020112760 con 0x7fd020107220 2024-09-21T03:46:10.242 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:10.237+0000 7fd0254c5640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fd020112a60 con 0x7fd020107b70 2024-09-21T03:46:10.242 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:10.237+0000 7fd01f7fe640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd020108540 0x7fd02019fc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:46:10.242 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:10.237+0000 7fd01f7fe640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd020108540 0x7fd02019fc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:43128/0 (socket says 172.21.15.39:43128) 2024-09-21T03:46:10.242 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:10.237+0000 7fd01f7fe640 1 -- 172.21.15.39:0/3385151792 learned_addr learned my addr 172.21.15.39:0/3385151792 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:46:10.242 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:10.237+0000 7fd017fff640 1 --2- 172.21.15.39:0/3385151792 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd020107b70 0x7fd02019a9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:46:10.242 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:10.237+0000 7fd01effd640 1 --2- 172.21.15.39:0/3385151792 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd020107220 0x7fd02019a410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:46:10.242 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:10.237+0000 7fd01f7fe640 1 -- 172.21.15.39:0/3385151792 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd020107b70 msgr2=0x7fd02019a9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:46:10.242 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:10.237+0000 7fd01f7fe640 1 --2- 172.21.15.39:0/3385151792 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd020107b70 0x7fd02019a9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:10.242 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:10.237+0000 7fd01f7fe640 1 -- 172.21.15.39:0/3385151792 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd020107220 msgr2=0x7fd02019a410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:46:10.243 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:10.237+0000 7fd01f7fe640 1 --2- 172.21.15.39:0/3385151792 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd020107220 0x7fd02019a410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:10.243 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:10.237+0000 7fd01f7fe640 1 -- 172.21.15.39:0/3385151792 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd0201a0340 con 0x7fd020108540 2024-09-21T03:46:10.243 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:10.237+0000 7fd01f7fe640 1 --2- 172.21.15.39:0/3385151792 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd020108540 0x7fd02019fc40 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7fd018009fd0 tx=0x7fd01800b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:46:10.243 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:10.237+0000 7fd01cff9640 1 -- 172.21.15.39:0/3385151792 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd018017070 con 0x7fd020108540 2024-09-21T03:46:10.243 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:10.238+0000 7fd01cff9640 1 -- 172.21.15.39:0/3385151792 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd018003e20 con 0x7fd020108540 2024-09-21T03:46:10.243 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:10.238+0000 7fd0254c5640 1 -- 172.21.15.39:0/3385151792 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd0201a0630 con 0x7fd020108540 2024-09-21T03:46:10.243 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:10.238+0000 7fd01cff9640 1 -- 172.21.15.39:0/3385151792 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd018010460 con 0x7fd020108540 2024-09-21T03:46:10.243 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:10.238+0000 7fd0254c5640 1 -- 172.21.15.39:0/3385151792 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd0201a6dc0 con 0x7fd020108540 2024-09-21T03:46:10.243 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:10.239+0000 7fd0254c5640 1 -- 172.21.15.39:0/3385151792 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd020107630 con 0x7fd020108540 2024-09-21T03:46:10.244 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:10.241+0000 7fd01cff9640 1 -- 172.21.15.39:0/3385151792 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 24) ==== 154580+0+0 (secure 0 0 0) 0x7fd018003970 con 0x7fd020108540 2024-09-21T03:46:10.245 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:10.242+0000 7fd01cff9640 1 --2- 172.21.15.39:0/3385151792 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7fd00c0b65f0 0x7fd00c0b8b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:10.245 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:10.242+0000 7fd01effd640 1 --2- 172.21.15.39:0/3385151792 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7fd00c0b65f0 0x7fd00c0b8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:46:10.245 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:10.242+0000 7fd01cff9640 1 -- 172.21.15.39:0/3385151792 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8484+0+0 (secure 0 0 0) 0x7fd0180e7470 con 0x7fd020108540 2024-09-21T03:46:10.245 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:10.243+0000 7fd01effd640 1 --2- 172.21.15.39:0/3385151792 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7fd00c0b65f0 0x7fd00c0b8b00 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fd010005e00 tx=0x7fd010005c50 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:46:10.247 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:10.244+0000 7fd01cff9640 1 -- 172.21.15.39:0/3385151792 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fd0180ada40 con 0x7fd020108540 2024-09-21T03:46:10.440 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:10.438+0000 7fd0254c5640 1 -- 172.21.15.39:0/3385151792 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fd0201a6fc0 con 0x7fd020108540 2024-09-21T03:46:10.441 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:10.438+0000 7fd01cff9640 1 -- 172.21.15.39:0/3385151792 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v79) ==== 104+0+11 (secure 0 0 0) 0x7fd0180b1e20 con 0x7fd020108540 2024-09-21T03:46:10.441 INFO:teuthology.orchestra.run.smithi039.stdout:pg_num: 32 2024-09-21T03:46:10.443 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:10.441+0000 7fd0254c5640 1 -- 172.21.15.39:0/3385151792 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7fd00c0b65f0 msgr2=0x7fd00c0b8b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:46:10.444 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:10.441+0000 7fd0254c5640 1 --2- 172.21.15.39:0/3385151792 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7fd00c0b65f0 0x7fd00c0b8b00 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fd010005e00 tx=0x7fd010005c50 comp rx=0 tx=0).stop 2024-09-21T03:46:10.444 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:10.441+0000 7fd0254c5640 1 -- 172.21.15.39:0/3385151792 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd020108540 msgr2=0x7fd02019fc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:46:10.444 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:10.441+0000 7fd0254c5640 1 --2- 172.21.15.39:0/3385151792 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd020108540 0x7fd02019fc40 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7fd018009fd0 tx=0x7fd01800b260 comp rx=0 tx=0).stop 2024-09-21T03:46:10.444 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:10.441+0000 7fd0254c5640 1 -- 172.21.15.39:0/3385151792 shutdown_connections 2024-09-21T03:46:10.444 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:10.441+0000 7fd0254c5640 1 --2- 172.21.15.39:0/3385151792 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7fd00c0b65f0 0x7fd00c0b8b00 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:10.444 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:10.441+0000 7fd0254c5640 1 --2- 172.21.15.39:0/3385151792 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd020108540 0x7fd02019fc40 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:10.444 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:10.441+0000 7fd0254c5640 1 --2- 172.21.15.39:0/3385151792 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd020107b70 0x7fd02019a9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:10.444 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:10.441+0000 7fd0254c5640 1 --2- 172.21.15.39:0/3385151792 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd020107220 0x7fd02019a410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:10.444 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:10.441+0000 7fd0254c5640 1 -- 172.21.15.39:0/3385151792 >> 172.21.15.39:0/3385151792 conn(0x7fd0200765f0 msgr2=0x7fd020100bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:46:10.444 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:10.441+0000 7fd0254c5640 1 -- 172.21.15.39:0/3385151792 shutdown_connections 2024-09-21T03:46:10.444 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:10.441+0000 7fd0254c5640 1 -- 172.21.15.39:0/3385151792 wait complete. 2024-09-21T03:46:10.456 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-09-21T03:46:10.468 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:46:10 smithi161 ceph-mon[34533]: from='client.? 172.21.15.39:0/2964373851' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]': finished 2024-09-21T03:46:10.468 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:46:10 smithi161 ceph-mon[34533]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-21T03:46:10.468 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:46:10 smithi161 ceph-mon[34533]: from='client.? 172.21.15.39:0/947925955' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T03:46:10.474 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:46:10 smithi150 ceph-mon[34289]: from='client.? 172.21.15.39:0/2964373851' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]': finished 2024-09-21T03:46:10.475 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:46:10 smithi150 ceph-mon[34289]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-21T03:46:10.475 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:46:10 smithi150 ceph-mon[34289]: from='client.? 172.21.15.39:0/947925955' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T03:46:10.575 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:10.572+0000 7f12ee121640 1 -- 172.21.15.39:0/258740273 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f12e81082f0 msgr2=0x7f12e8108700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:46:10.575 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:10.572+0000 7f12ee121640 1 --2- 172.21.15.39:0/258740273 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f12e81082f0 0x7f12e8108700 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f12d8009900 tx=0x7f12d802a410 comp rx=0 tx=0).stop 2024-09-21T03:46:10.575 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:10.572+0000 7f12ee121640 1 -- 172.21.15.39:0/258740273 shutdown_connections 2024-09-21T03:46:10.575 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:10.572+0000 7f12ee121640 1 --2- 172.21.15.39:0/258740273 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f12e8108c40 0x7f12e810f470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:10.575 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:10.572+0000 7f12ee121640 1 --2- 172.21.15.39:0/258740273 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f12e81082f0 0x7f12e8108700 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:10.575 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:10.572+0000 7f12ee121640 1 --2- 172.21.15.39:0/258740273 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f12e81029c0 0x7f12e8102dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:10.575 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:10.572+0000 7f12ee121640 1 -- 172.21.15.39:0/258740273 >> 172.21.15.39:0/258740273 conn(0x7f12e80fe400 msgr2=0x7f12e8100850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:46:10.575 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:10.572+0000 7f12ee121640 1 -- 172.21.15.39:0/258740273 shutdown_connections 2024-09-21T03:46:10.576 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:10.572+0000 7f12ee121640 1 -- 172.21.15.39:0/258740273 wait complete. 2024-09-21T03:46:10.578 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:10.573+0000 7f12ee121640 1 Processor -- start 2024-09-21T03:46:10.578 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:10.573+0000 7f12ee121640 1 -- start start 2024-09-21T03:46:10.578 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:10.573+0000 7f12ee121640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f12e81029c0 0x7f12e819a440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:10.578 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:10.573+0000 7f12ee121640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f12e81082f0 0x7f12e819a9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:10.578 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:10.573+0000 7f12ee121640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f12e8108c40 0x7f12e819fc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:10.578 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:10.573+0000 7f12ee121640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f12e8112950 con 0x7f12e81082f0 2024-09-21T03:46:10.578 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:10.573+0000 7f12ee121640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f12e81127d0 con 0x7f12e81029c0 2024-09-21T03:46:10.578 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:10.573+0000 7f12ee121640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f12e8112ad0 con 0x7f12e8108c40 2024-09-21T03:46:10.578 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:10.573+0000 7f12e6ffd640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f12e81082f0 0x7f12e819a9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:46:10.578 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:10.573+0000 7f12e6ffd640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f12e81082f0 0x7f12e819a9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:43148/0 (socket says 172.21.15.39:43148) 2024-09-21T03:46:10.578 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:10.573+0000 7f12e6ffd640 1 -- 172.21.15.39:0/1742390767 learned_addr learned my addr 172.21.15.39:0/1742390767 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:46:10.579 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:10.573+0000 7f12e77fe640 1 --2- 172.21.15.39:0/1742390767 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f12e81029c0 0x7f12e819a440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:46:10.579 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:10.573+0000 7f12e7fff640 1 --2- 172.21.15.39:0/1742390767 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f12e8108c40 0x7f12e819fc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:46:10.579 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:10.574+0000 7f12e6ffd640 1 -- 172.21.15.39:0/1742390767 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f12e8108c40 msgr2=0x7f12e819fc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:46:10.579 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:10.574+0000 7f12e6ffd640 1 --2- 172.21.15.39:0/1742390767 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f12e8108c40 0x7f12e819fc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:10.579 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:10.574+0000 7f12e6ffd640 1 -- 172.21.15.39:0/1742390767 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f12e81029c0 msgr2=0x7f12e819a440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:46:10.579 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:10.574+0000 7f12e6ffd640 1 --2- 172.21.15.39:0/1742390767 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f12e81029c0 0x7f12e819a440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:10.579 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:10.574+0000 7f12e6ffd640 1 -- 172.21.15.39:0/1742390767 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f12e81a0370 con 0x7f12e81082f0 2024-09-21T03:46:10.579 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:10.574+0000 7f12e6ffd640 1 --2- 172.21.15.39:0/1742390767 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f12e81082f0 0x7f12e819a9e0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f12d8005ab0 tx=0x7f12d8003b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:46:10.579 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:10.574+0000 7f12e4ff9640 1 -- 172.21.15.39:0/1742390767 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12d803f070 con 0x7f12e81082f0 2024-09-21T03:46:10.579 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:10.574+0000 7f12e4ff9640 1 -- 172.21.15.39:0/1742390767 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f12d8031b50 con 0x7f12e81082f0 2024-09-21T03:46:10.579 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:10.574+0000 7f12ee121640 1 -- 172.21.15.39:0/1742390767 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f12e81a0600 con 0x7f12e81082f0 2024-09-21T03:46:10.579 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:10.574+0000 7f12e4ff9640 1 -- 172.21.15.39:0/1742390767 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12d8035030 con 0x7f12e81082f0 2024-09-21T03:46:10.579 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:10.574+0000 7f12ee121640 1 -- 172.21.15.39:0/1742390767 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f12e81a6df0 con 0x7f12e81082f0 2024-09-21T03:46:10.579 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:10.575+0000 7f12ee121640 1 -- 172.21.15.39:0/1742390767 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f12e8103c30 con 0x7f12e81082f0 2024-09-21T03:46:10.580 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:10.577+0000 7f12e4ff9640 1 -- 172.21.15.39:0/1742390767 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 24) ==== 154580+0+0 (secure 0 0 0) 0x7f12d8042070 con 0x7f12e81082f0 2024-09-21T03:46:10.580 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:10.578+0000 7f12e4ff9640 1 --2- 172.21.15.39:0/1742390767 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f12d40b65f0 0x7f12d40b8b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:10.580 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:10.578+0000 7f12e4ff9640 1 -- 172.21.15.39:0/1742390767 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8484+0+0 (secure 0 0 0) 0x7f12d8108050 con 0x7f12e81082f0 2024-09-21T03:46:10.581 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:10.578+0000 7f12e77fe640 1 --2- 172.21.15.39:0/1742390767 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f12d40b65f0 0x7f12d40b8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:46:10.581 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:10.578+0000 7f12e77fe640 1 --2- 172.21.15.39:0/1742390767 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f12d40b65f0 0x7f12d40b8b00 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f12d0009bd0 tx=0x7f12d0005f90 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:46:10.584 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:10.581+0000 7f12e4ff9640 1 -- 172.21.15.39:0/1742390767 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f12d80ce680 con 0x7f12e81082f0 2024-09-21T03:46:10.773 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:10.770+0000 7f12ee121640 1 -- 172.21.15.39:0/1742390767 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f12e819b560 con 0x7f12e81082f0 2024-09-21T03:46:10.774 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:10.771+0000 7f12e4ff9640 1 -- 172.21.15.39:0/1742390767 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v79) ==== 100+0+11 (secure 0 0 0) 0x7f12d80d2a60 con 0x7f12e81082f0 2024-09-21T03:46:10.774 INFO:teuthology.orchestra.run.smithi039.stdout:pg_num: 64 2024-09-21T03:46:10.776 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:10.773+0000 7f12ee121640 1 -- 172.21.15.39:0/1742390767 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f12d40b65f0 msgr2=0x7f12d40b8b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:46:10.776 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:10.773+0000 7f12ee121640 1 --2- 172.21.15.39:0/1742390767 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f12d40b65f0 0x7f12d40b8b00 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f12d0009bd0 tx=0x7f12d0005f90 comp rx=0 tx=0).stop 2024-09-21T03:46:10.776 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:10.773+0000 7f12ee121640 1 -- 172.21.15.39:0/1742390767 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f12e81082f0 msgr2=0x7f12e819a9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:46:10.776 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:10.773+0000 7f12ee121640 1 --2- 172.21.15.39:0/1742390767 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f12e81082f0 0x7f12e819a9e0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f12d8005ab0 tx=0x7f12d8003b90 comp rx=0 tx=0).stop 2024-09-21T03:46:10.776 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:10.773+0000 7f12ee121640 1 -- 172.21.15.39:0/1742390767 shutdown_connections 2024-09-21T03:46:10.776 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:10.773+0000 7f12ee121640 1 --2- 172.21.15.39:0/1742390767 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f12d40b65f0 0x7f12d40b8b00 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:10.776 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:10.773+0000 7f12ee121640 1 --2- 172.21.15.39:0/1742390767 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f12e8108c40 0x7f12e819fc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:10.777 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:10.773+0000 7f12ee121640 1 --2- 172.21.15.39:0/1742390767 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f12e81082f0 0x7f12e819a9e0 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:10.777 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:10.773+0000 7f12ee121640 1 --2- 172.21.15.39:0/1742390767 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f12e81029c0 0x7f12e819a440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:10.777 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:10.773+0000 7f12ee121640 1 -- 172.21.15.39:0/1742390767 >> 172.21.15.39:0/1742390767 conn(0x7f12e80fe400 msgr2=0x7f12e806bca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:46:10.777 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:10.773+0000 7f12ee121640 1 -- 172.21.15.39:0/1742390767 shutdown_connections 2024-09-21T03:46:10.777 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:10.773+0000 7f12ee121640 1 -- 172.21.15.39:0/1742390767 wait complete. 2024-09-21T03:46:10.788 INFO:tasks.fwd_scrub:Wait for all MDSs to reach steady state... 2024-09-21T03:46:10.788 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 51116d5c-77ca-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T03:46:11.051 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/51116d5c-77ca-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:46:11.083 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:46:11 smithi039 ceph-mon[31211]: from='client.15483 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup pin", "vol_name": "cephfs", "group_name": "qa", "pin_type": "random", "pin_setting": "0.10", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:46:11.084 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:46:11 smithi039 ceph-mon[31211]: pgmap v329: 97 pgs: 97 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 853 B/s wr, 12 op/s 2024-09-21T03:46:11.084 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:46:11 smithi039 ceph-mon[31211]: mgrmap e24: x(active, since 8m), standbys: z, y 2024-09-21T03:46:11.084 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:46:11 smithi039 ceph-mon[31211]: from='client.? 172.21.15.39:0/2022635456' entity='client.admin' cmd={"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} : dispatch 2024-09-21T03:46:11.084 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:46:11 smithi039 ceph-mon[31211]: from='client.? 172.21.15.39:0/3385151792' entity='client.admin' cmd={"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} : dispatch 2024-09-21T03:46:11.084 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:46:11 smithi039 ceph-mon[31211]: from='client.? 172.21.15.39:0/1742390767' entity='client.admin' cmd={"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} : dispatch 2024-09-21T03:46:11.376 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:46:11 smithi150 ceph-mon[34289]: from='client.15483 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup pin", "vol_name": "cephfs", "group_name": "qa", "pin_type": "random", "pin_setting": "0.10", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:46:11.376 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:46:11 smithi150 ceph-mon[34289]: pgmap v329: 97 pgs: 97 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 853 B/s wr, 12 op/s 2024-09-21T03:46:11.376 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:46:11 smithi150 ceph-mon[34289]: mgrmap e24: x(active, since 8m), standbys: z, y 2024-09-21T03:46:11.376 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:46:11 smithi150 ceph-mon[34289]: from='client.? 172.21.15.39:0/2022635456' entity='client.admin' cmd={"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} : dispatch 2024-09-21T03:46:11.376 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:46:11 smithi150 ceph-mon[34289]: from='client.? 172.21.15.39:0/3385151792' entity='client.admin' cmd={"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} : dispatch 2024-09-21T03:46:11.377 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:46:11 smithi150 ceph-mon[34289]: from='client.? 172.21.15.39:0/1742390767' entity='client.admin' cmd={"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} : dispatch 2024-09-21T03:46:11.467 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:46:11 smithi161 ceph-mon[34533]: from='client.15483 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup pin", "vol_name": "cephfs", "group_name": "qa", "pin_type": "random", "pin_setting": "0.10", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:46:11.468 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:46:11 smithi161 ceph-mon[34533]: pgmap v329: 97 pgs: 97 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 853 B/s wr, 12 op/s 2024-09-21T03:46:11.468 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:46:11 smithi161 ceph-mon[34533]: mgrmap e24: x(active, since 8m), standbys: z, y 2024-09-21T03:46:11.468 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:46:11 smithi161 ceph-mon[34533]: from='client.? 172.21.15.39:0/2022635456' entity='client.admin' cmd={"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} : dispatch 2024-09-21T03:46:11.468 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:46:11 smithi161 ceph-mon[34533]: from='client.? 172.21.15.39:0/3385151792' entity='client.admin' cmd={"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} : dispatch 2024-09-21T03:46:11.468 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:46:11 smithi161 ceph-mon[34533]: from='client.? 172.21.15.39:0/1742390767' entity='client.admin' cmd={"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} : dispatch 2024-09-21T03:46:11.629 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:11.612+0000 7f0a5877a640 1 -- 172.21.15.39:0/1982513278 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0a50108cf0 msgr2=0x7f0a5010f520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:46:11.629 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:11.612+0000 7f0a5877a640 1 --2- 172.21.15.39:0/1982513278 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0a50108cf0 0x7f0a5010f520 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f0a48009ca0 tx=0x7f0a4802a8c0 comp rx=0 tx=0).stop 2024-09-21T03:46:11.629 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:11.613+0000 7f0a5877a640 1 -- 172.21.15.39:0/1982513278 shutdown_connections 2024-09-21T03:46:11.629 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:11.613+0000 7f0a5877a640 1 --2- 172.21.15.39:0/1982513278 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0a50108cf0 0x7f0a5010f520 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:11.629 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:11.613+0000 7f0a5877a640 1 --2- 172.21.15.39:0/1982513278 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0a501083a0 0x7f0a501087b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:11.630 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:11.613+0000 7f0a5877a640 1 --2- 172.21.15.39:0/1982513278 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0a50102a60 0x7f0a50102e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:11.630 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:11.613+0000 7f0a5877a640 1 -- 172.21.15.39:0/1982513278 >> 172.21.15.39:0/1982513278 conn(0x7f0a50079730 msgr2=0x7f0a50079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:46:11.630 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:11.613+0000 7f0a5877a640 1 -- 172.21.15.39:0/1982513278 shutdown_connections 2024-09-21T03:46:11.630 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:11.613+0000 7f0a5877a640 1 -- 172.21.15.39:0/1982513278 wait complete. 2024-09-21T03:46:11.630 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:11.613+0000 7f0a5877a640 1 Processor -- start 2024-09-21T03:46:11.630 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:11.613+0000 7f0a5877a640 1 -- start start 2024-09-21T03:46:11.630 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:11.614+0000 7f0a5877a640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0a50102a60 0x7f0a5019a3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:11.630 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:11.614+0000 7f0a5877a640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0a501083a0 0x7f0a5019a990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:11.630 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:11.614+0000 7f0a5877a640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0a50108cf0 0x7f0a5019fc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:11.630 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:11.614+0000 7f0a5877a640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f0a50113a80 con 0x7f0a501083a0 2024-09-21T03:46:11.630 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:11.614+0000 7f0a5877a640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f0a50113900 con 0x7f0a50108cf0 2024-09-21T03:46:11.630 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:11.614+0000 7f0a5877a640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f0a50113c00 con 0x7f0a50102a60 2024-09-21T03:46:11.631 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:11.614+0000 7f0a55d15640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0a501083a0 0x7f0a5019a990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:46:11.631 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:11.614+0000 7f0a56516640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0a50102a60 0x7f0a5019a3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:46:11.631 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:11.614+0000 7f0a55d15640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0a501083a0 0x7f0a5019a990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:43172/0 (socket says 172.21.15.39:43172) 2024-09-21T03:46:11.631 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:11.614+0000 7f0a55d15640 1 -- 172.21.15.39:0/1338028907 learned_addr learned my addr 172.21.15.39:0/1338028907 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:46:11.631 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:11.614+0000 7f0a56d17640 1 --2- 172.21.15.39:0/1338028907 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0a50108cf0 0x7f0a5019fc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:46:11.631 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:11.614+0000 7f0a55d15640 1 -- 172.21.15.39:0/1338028907 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0a50102a60 msgr2=0x7f0a5019a3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:46:11.631 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:11.614+0000 7f0a55d15640 1 --2- 172.21.15.39:0/1338028907 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0a50102a60 0x7f0a5019a3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:11.631 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:11.614+0000 7f0a55d15640 1 -- 172.21.15.39:0/1338028907 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0a50108cf0 msgr2=0x7f0a5019fc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:46:11.631 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:11.614+0000 7f0a55d15640 1 --2- 172.21.15.39:0/1338028907 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0a50108cf0 0x7f0a5019fc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:11.631 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:11.614+0000 7f0a55d15640 1 -- 172.21.15.39:0/1338028907 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0a501a0360 con 0x7f0a501083a0 2024-09-21T03:46:11.631 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:11.614+0000 7f0a56516640 1 --2- 172.21.15.39:0/1338028907 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0a50102a60 0x7f0a5019a3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:46:11.631 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:11.615+0000 7f0a55d15640 1 --2- 172.21.15.39:0/1338028907 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0a501083a0 0x7f0a5019a990 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f0a4000c970 tx=0x7f0a4000ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:46:11.631 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:11.615+0000 7f0a477fe640 1 -- 172.21.15.39:0/1338028907 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a40007bb0 con 0x7f0a501083a0 2024-09-21T03:46:11.631 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:11.615+0000 7f0a477fe640 1 -- 172.21.15.39:0/1338028907 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0a40007d50 con 0x7f0a501083a0 2024-09-21T03:46:11.632 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:11.615+0000 7f0a5877a640 1 -- 172.21.15.39:0/1338028907 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0a501a0650 con 0x7f0a501083a0 2024-09-21T03:46:11.632 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:11.615+0000 7f0a477fe640 1 -- 172.21.15.39:0/1338028907 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a4000f450 con 0x7f0a501083a0 2024-09-21T03:46:11.632 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:11.615+0000 7f0a5877a640 1 -- 172.21.15.39:0/1338028907 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0a501a6db0 con 0x7f0a501083a0 2024-09-21T03:46:11.632 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:11.616+0000 7f0a5877a640 1 -- 172.21.15.39:0/1338028907 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0a50103ce0 con 0x7f0a501083a0 2024-09-21T03:46:11.632 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:11.618+0000 7f0a477fe640 1 -- 172.21.15.39:0/1338028907 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 24) ==== 154580+0+0 (secure 0 0 0) 0x7f0a40004f20 con 0x7f0a501083a0 2024-09-21T03:46:11.632 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:11.618+0000 7f0a477fe640 1 --2- 172.21.15.39:0/1338028907 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f0a3c0b6670 0x7f0a3c0b8b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:11.632 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:11.619+0000 7f0a56516640 1 --2- 172.21.15.39:0/1338028907 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f0a3c0b6670 0x7f0a3c0b8b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:46:11.632 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:11.619+0000 7f0a477fe640 1 -- 172.21.15.39:0/1338028907 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8484+0+0 (secure 0 0 0) 0x7f0a400e7530 con 0x7f0a501083a0 2024-09-21T03:46:11.632 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:11.619+0000 7f0a56516640 1 --2- 172.21.15.39:0/1338028907 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f0a3c0b6670 0x7f0a3c0b8b80 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f0a4c00bfd0 tx=0x7f0a4c00b390 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:46:11.632 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:11.623+0000 7f0a477fe640 1 -- 172.21.15.39:0/1338028907 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f0a40014030 con 0x7f0a501083a0 2024-09-21T03:46:11.854 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:11.851+0000 7f0a5877a640 1 -- 172.21.15.39:0/1338028907 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f0a50102e70 con 0x7f0a501083a0 2024-09-21T03:46:11.855 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:11.852+0000 7f0a477fe640 1 -- 172.21.15.39:0/1338028907 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11978 (secure 0 0 0) 0x7f0a400b1f40 con 0x7f0a501083a0 2024-09-21T03:46:11.855 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T03:46:11.855 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":29,"btime":"2024-09-21T03:46:08:982881+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14790,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6835/508995295","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":508995295},{"type":"v1","addr":"172.21.15.39:6835","nonce":508995295}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14802,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6837/3915912607","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":3915912607},{"type":"v1","addr":"172.21.15.39:6837","nonce":3915912607}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":29,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T03:44:15.009872+0000","modified":"2024-09-21T03:46:08.430113+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24695,"mds_1":24667,"mds_2":24659,"mds_3":24646,"mds_4":14868},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14856":{"gid":14856,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.39:6839/185509192","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6838","nonce":185509192},{"type":"v1","addr":"172.21.15.39:6839","nonce":185509192}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14868":{"gid":14868,"name":"a","rank":4,"incarnation":25,"state":"up:active","state_seq":7,"addr":"172.21.15.39:6841/4247190456","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6840","nonce":4247190456},{"type":"v1","addr":"172.21.15.39:6841","nonce":4247190456}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24599":{"gid":24599,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.150:6833/3114643217","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":3114643217},{"type":"v1","addr":"172.21.15.150:6833","nonce":3114643217}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24601":{"gid":24601,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.161:6833/2963990654","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":2963990654},{"type":"v1","addr":"172.21.15.161:6833","nonce":2963990654}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24632":{"gid":24632,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.150:6835/724594008","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6834","nonce":724594008},{"type":"v1","addr":"172.21.15.150:6835","nonce":724594008}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24640":{"gid":24640,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.161:6835/3747882110","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":3747882110},{"type":"v1","addr":"172.21.15.161:6835","nonce":3747882110}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24646":{"gid":24646,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":10,"addr":"172.21.15.161:6837/3451544597","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6836","nonce":3451544597},{"type":"v1","addr":"172.21.15.161:6837","nonce":3451544597}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24659":{"gid":24659,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.150:6837/4017043144","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6836","nonce":4017043144},{"type":"v1","addr":"172.21.15.150:6837","nonce":4017043144}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24667":{"gid":24667,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.161:6839/2190107270","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6838","nonce":2190107270},{"type":"v1","addr":"172.21.15.161:6839","nonce":2190107270}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24695":{"gid":24695,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.150:6839/2949386907","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6838","nonce":2949386907},{"type":"v1","addr":"172.21.15.150:6839","nonce":2949386907}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24695,"qdb_cluster":[24695,24667,24659,24646,14868]},"id":1}]} 2024-09-21T03:46:11.855 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 29 2024-09-21T03:46:11.857 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:11.854+0000 7f0a5877a640 1 -- 172.21.15.39:0/1338028907 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f0a3c0b6670 msgr2=0x7f0a3c0b8b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:46:11.857 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:11.854+0000 7f0a5877a640 1 --2- 172.21.15.39:0/1338028907 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f0a3c0b6670 0x7f0a3c0b8b80 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f0a4c00bfd0 tx=0x7f0a4c00b390 comp rx=0 tx=0).stop 2024-09-21T03:46:11.857 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:11.854+0000 7f0a5877a640 1 -- 172.21.15.39:0/1338028907 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0a501083a0 msgr2=0x7f0a5019a990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:46:11.857 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:11.854+0000 7f0a5877a640 1 --2- 172.21.15.39:0/1338028907 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0a501083a0 0x7f0a5019a990 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f0a4000c970 tx=0x7f0a4000ce30 comp rx=0 tx=0).stop 2024-09-21T03:46:11.858 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:11.854+0000 7f0a5877a640 1 -- 172.21.15.39:0/1338028907 shutdown_connections 2024-09-21T03:46:11.858 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:11.854+0000 7f0a5877a640 1 --2- 172.21.15.39:0/1338028907 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f0a3c0b6670 0x7f0a3c0b8b80 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:11.858 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:11.854+0000 7f0a5877a640 1 --2- 172.21.15.39:0/1338028907 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0a50108cf0 0x7f0a5019fc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:11.858 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:11.854+0000 7f0a5877a640 1 --2- 172.21.15.39:0/1338028907 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0a501083a0 0x7f0a5019a990 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:11.858 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:11.854+0000 7f0a5877a640 1 --2- 172.21.15.39:0/1338028907 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0a50102a60 0x7f0a5019a3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:11.858 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:11.854+0000 7f0a5877a640 1 -- 172.21.15.39:0/1338028907 >> 172.21.15.39:0/1338028907 conn(0x7f0a50079730 msgr2=0x7f0a501051c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:46:11.858 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:11.854+0000 7f0a5877a640 1 -- 172.21.15.39:0/1338028907 shutdown_connections 2024-09-21T03:46:11.858 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:11.854+0000 7f0a5877a640 1 -- 172.21.15.39:0/1338028907 wait complete. 2024-09-21T03:46:12.237 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:46:12 smithi039 ceph-mon[31211]: from='client.? 172.21.15.39:0/1338028907' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:46:12.467 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:46:12 smithi161 ceph-mon[34533]: from='client.? 172.21.15.39:0/1338028907' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:46:12.474 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:46:12 smithi150 ceph-mon[34289]: from='client.? 172.21.15.39:0/1338028907' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:46:12.617 INFO:tasks.fwd_scrub:Ready to start scrub thrashing 2024-09-21T03:46:12.618 INFO:tasks.fwd_scrub.ceph_manager:waiting for clean 2024-09-21T03:46:12.618 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-21T03:46:12.739 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:12.736+0000 7fcb16d41640 1 -- 172.21.15.39:0/1952580345 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcb10108c40 msgr2=0x7fcb1010f470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:46:12.739 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:12.736+0000 7fcb16d41640 1 --2- 172.21.15.39:0/1952580345 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcb10108c40 0x7fcb1010f470 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7fcb08009ca0 tx=0x7fcb0802a8c0 comp rx=0 tx=0).stop 2024-09-21T03:46:12.739 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:12.736+0000 7fcb16d41640 1 -- 172.21.15.39:0/1952580345 shutdown_connections 2024-09-21T03:46:12.740 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:12.736+0000 7fcb16d41640 1 --2- 172.21.15.39:0/1952580345 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcb10108c40 0x7fcb1010f470 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:12.740 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:12.736+0000 7fcb16d41640 1 --2- 172.21.15.39:0/1952580345 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcb101082f0 0x7fcb10108700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:12.740 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:12.736+0000 7fcb16d41640 1 --2- 172.21.15.39:0/1952580345 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcb10103130 0x7fcb10103540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:12.740 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:12.736+0000 7fcb16d41640 1 -- 172.21.15.39:0/1952580345 >> 172.21.15.39:0/1952580345 conn(0x7fcb10079e50 msgr2=0x7fcb1007a260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:46:12.740 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:12.736+0000 7fcb16d41640 1 -- 172.21.15.39:0/1952580345 shutdown_connections 2024-09-21T03:46:12.740 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:12.736+0000 7fcb16d41640 1 -- 172.21.15.39:0/1952580345 wait complete. 2024-09-21T03:46:12.742 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:12.737+0000 7fcb16d41640 1 Processor -- start 2024-09-21T03:46:12.742 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:12.737+0000 7fcb16d41640 1 -- start start 2024-09-21T03:46:12.742 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:12.738+0000 7fcb16d41640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcb10103130 0x7fcb1019a380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:12.742 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:12.738+0000 7fcb16d41640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcb101082f0 0x7fcb1019a920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:12.742 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:12.738+0000 7fcb16d41640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcb10108c40 0x7fcb1019fbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:12.743 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:12.738+0000 7fcb16d41640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fcb10113a20 con 0x7fcb10103130 2024-09-21T03:46:12.743 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:12.738+0000 7fcb16d41640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fcb101138a0 con 0x7fcb101082f0 2024-09-21T03:46:12.743 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:12.738+0000 7fcb16d41640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fcb10113ba0 con 0x7fcb10108c40 2024-09-21T03:46:12.743 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:12.738+0000 7fcb14add640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcb10103130 0x7fcb1019a380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:46:12.743 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:12.738+0000 7fcb14add640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcb10103130 0x7fcb1019a380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:43190/0 (socket says 172.21.15.39:43190) 2024-09-21T03:46:12.743 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:12.738+0000 7fcb14add640 1 -- 172.21.15.39:0/775959510 learned_addr learned my addr 172.21.15.39:0/775959510 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:46:12.743 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:12.738+0000 7fcb152de640 1 --2- 172.21.15.39:0/775959510 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcb10108c40 0x7fcb1019fbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:46:12.743 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:12.738+0000 7fcb0ffff640 1 --2- 172.21.15.39:0/775959510 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcb101082f0 0x7fcb1019a920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:46:12.743 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:12.738+0000 7fcb14add640 1 -- 172.21.15.39:0/775959510 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcb10108c40 msgr2=0x7fcb1019fbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:46:12.743 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:12.738+0000 7fcb14add640 1 --2- 172.21.15.39:0/775959510 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcb10108c40 0x7fcb1019fbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:12.743 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:12.738+0000 7fcb14add640 1 -- 172.21.15.39:0/775959510 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcb101082f0 msgr2=0x7fcb1019a920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:46:12.743 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:12.738+0000 7fcb14add640 1 --2- 172.21.15.39:0/775959510 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcb101082f0 0x7fcb1019a920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:12.743 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:12.738+0000 7fcb14add640 1 -- 172.21.15.39:0/775959510 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcb101a02b0 con 0x7fcb10103130 2024-09-21T03:46:12.744 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:12.738+0000 7fcb152de640 1 --2- 172.21.15.39:0/775959510 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcb10108c40 0x7fcb1019fbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:46:12.744 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:12.738+0000 7fcb0ffff640 1 --2- 172.21.15.39:0/775959510 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcb101082f0 0x7fcb1019a920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:46:12.744 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:12.738+0000 7fcb14add640 1 --2- 172.21.15.39:0/775959510 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcb10103130 0x7fcb1019a380 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7fcb0000d950 tx=0x7fcb0000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:46:12.744 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:12.738+0000 7fcb0dffb640 1 -- 172.21.15.39:0/775959510 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb00017070 con 0x7fcb10103130 2024-09-21T03:46:12.744 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:12.738+0000 7fcb0dffb640 1 -- 172.21.15.39:0/775959510 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcb00009c30 con 0x7fcb10103130 2024-09-21T03:46:12.744 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:12.738+0000 7fcb0dffb640 1 -- 172.21.15.39:0/775959510 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb0000f620 con 0x7fcb10103130 2024-09-21T03:46:12.744 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:12.738+0000 7fcb16d41640 1 -- 172.21.15.39:0/775959510 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcb101a05a0 con 0x7fcb10103130 2024-09-21T03:46:12.744 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:12.739+0000 7fcb16d41640 1 -- 172.21.15.39:0/775959510 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcb101a6d30 con 0x7fcb10103130 2024-09-21T03:46:12.744 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:12.740+0000 7fcb16d41640 1 -- 172.21.15.39:0/775959510 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcb1006b720 con 0x7fcb10103130 2024-09-21T03:46:12.745 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:12.741+0000 7fcb0dffb640 1 -- 172.21.15.39:0/775959510 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 24) ==== 154580+0+0 (secure 0 0 0) 0x7fcb00010460 con 0x7fcb10103130 2024-09-21T03:46:12.745 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:12.742+0000 7fcb0dffb640 1 --2- 172.21.15.39:0/775959510 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7fcaf80b6670 0x7fcaf80b8b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:12.745 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:12.742+0000 7fcb0ffff640 1 --2- 172.21.15.39:0/775959510 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7fcaf80b6670 0x7fcaf80b8b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:46:12.745 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:12.742+0000 7fcb0dffb640 1 -- 172.21.15.39:0/775959510 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8484+0+0 (secure 0 0 0) 0x7fcb000e7290 con 0x7fcb10103130 2024-09-21T03:46:12.746 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:12.743+0000 7fcb0ffff640 1 --2- 172.21.15.39:0/775959510 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7fcaf80b6670 0x7fcaf80b8b80 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fcb10105ee0 tx=0x7fcb04005d30 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:46:12.748 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:12.745+0000 7fcb0dffb640 1 -- 172.21.15.39:0/775959510 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fcb000ad940 con 0x7fcb10103130 2024-09-21T03:46:12.932 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:12.930+0000 7fcb16d41640 1 -- 172.21.15.39:0/775959510 --> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fcb10063800 con 0x7fcaf80b6670 2024-09-21T03:46:12.937 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:12.935+0000 7fcb0dffb640 1 -- 172.21.15.39:0/775959510 <== mgr.14223 v2:172.21.15.39:6800/1696381982 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258883 (secure 0 0 0) 0x7fcb10063800 con 0x7fcaf80b6670 2024-09-21T03:46:12.940 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T03:46:12.941 INFO:teuthology.orchestra.run.smithi039.stderr:dumped all 2024-09-21T03:46:12.943 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:12.938+0000 7fcb16d41640 1 -- 172.21.15.39:0/775959510 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7fcaf80b6670 msgr2=0x7fcaf80b8b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:46:12.943 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:12.938+0000 7fcb16d41640 1 --2- 172.21.15.39:0/775959510 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7fcaf80b6670 0x7fcaf80b8b80 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fcb10105ee0 tx=0x7fcb04005d30 comp rx=0 tx=0).stop 2024-09-21T03:46:12.943 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:12.938+0000 7fcb16d41640 1 -- 172.21.15.39:0/775959510 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcb10103130 msgr2=0x7fcb1019a380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:46:12.943 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:12.938+0000 7fcb16d41640 1 --2- 172.21.15.39:0/775959510 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcb10103130 0x7fcb1019a380 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7fcb0000d950 tx=0x7fcb0000de10 comp rx=0 tx=0).stop 2024-09-21T03:46:12.943 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:12.938+0000 7fcb16d41640 1 -- 172.21.15.39:0/775959510 shutdown_connections 2024-09-21T03:46:12.943 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:12.938+0000 7fcb16d41640 1 --2- 172.21.15.39:0/775959510 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7fcaf80b6670 0x7fcaf80b8b80 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:12.943 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:12.938+0000 7fcb16d41640 1 --2- 172.21.15.39:0/775959510 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcb10108c40 0x7fcb1019fbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:12.943 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:12.938+0000 7fcb16d41640 1 --2- 172.21.15.39:0/775959510 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcb101082f0 0x7fcb1019a920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:12.943 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:12.938+0000 7fcb16d41640 1 --2- 172.21.15.39:0/775959510 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcb10103130 0x7fcb1019a380 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:12.943 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:12.938+0000 7fcb16d41640 1 -- 172.21.15.39:0/775959510 >> 172.21.15.39:0/775959510 conn(0x7fcb10079e50 msgr2=0x7fcb10104280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:46:12.943 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:12.938+0000 7fcb16d41640 1 -- 172.21.15.39:0/775959510 shutdown_connections 2024-09-21T03:46:12.943 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:12.938+0000 7fcb16d41640 1 -- 172.21.15.39:0/775959510 wait complete. 2024-09-21T03:46:12.952 INFO:teuthology.orchestra.run.smithi039.stdout:{"pg_ready":true,"pg_map":{"version":330,"stamp":"2024-09-21T03:46:11.529574+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":663778,"num_objects":102,"num_object_clones":0,"num_object_copies":306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":1165,"num_read_kb":691,"num_write":385,"num_write_kb":2306,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":23,"num_bytes_recovered":2953794,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":818,"ondisk_log_size":818,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":361704,"kb_used_data":12456,"kb_used_omap":18,"kb_used_meta":349037,"kb_avail":1124580120,"statfs":{"total":1151940427776,"available":1151570042880,"internally_reserved":0,"allocated":12754944,"data_stored":7113860,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19143,"internal_metadata":357414201},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":1148,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":114,"num_read_kb":59,"num_write":9,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003767"},"pg_stats":[{"pgid":"3.3f","version":"79'8","reported_seq":26,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:45:55.900065+0000","last_change":"2024-09-21T03:44:14.327553+0000","last_active":"2024-09-21T03:45:55.900065+0000","last_peered":"2024-09-21T03:45:55.900065+0000","last_clean":"2024-09-21T03:45:55.900065+0000","last_became_active":"2024-09-21T03:44:14.327213+0000","last_became_peered":"2024-09-21T03:44:14.327213+0000","last_unstale":"2024-09-21T03:45:55.900065+0000","last_undegraded":"2024-09-21T03:45:55.900065+0000","last_fullsized":"2024-09-21T03:45:55.900065+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_clean_scrub_stamp":"2024-09-21T03:44:13.292303+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:40:56.550897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"79'4","reported_seq":22,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:45:55.902384+0000","last_change":"2024-09-21T03:44:14.321427+0000","last_active":"2024-09-21T03:45:55.902384+0000","last_peered":"2024-09-21T03:45:55.902384+0000","last_clean":"2024-09-21T03:45:55.902384+0000","last_became_active":"2024-09-21T03:44:14.321026+0000","last_became_peered":"2024-09-21T03:44:14.321026+0000","last_unstale":"2024-09-21T03:45:55.902384+0000","last_undegraded":"2024-09-21T03:45:55.902384+0000","last_fullsized":"2024-09-21T03:45:55.902384+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_clean_scrub_stamp":"2024-09-21T03:44:13.292303+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:18:00.889714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"79'7","reported_seq":25,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:45:55.906660+0000","last_change":"2024-09-21T03:44:14.324908+0000","last_active":"2024-09-21T03:45:55.906660+0000","last_peered":"2024-09-21T03:45:55.906660+0000","last_clean":"2024-09-21T03:45:55.906660+0000","last_became_active":"2024-09-21T03:44:14.324533+0000","last_became_peered":"2024-09-21T03:44:14.324533+0000","last_unstale":"2024-09-21T03:45:55.906660+0000","last_undegraded":"2024-09-21T03:45:55.906660+0000","last_fullsized":"2024-09-21T03:45:55.906660+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_clean_scrub_stamp":"2024-09-21T03:44:13.292303+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:44:25.235442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"79'8","reported_seq":26,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:45:55.885244+0000","last_change":"2024-09-21T03:44:14.326758+0000","last_active":"2024-09-21T03:45:55.885244+0000","last_peered":"2024-09-21T03:45:55.885244+0000","last_clean":"2024-09-21T03:45:55.885244+0000","last_became_active":"2024-09-21T03:44:14.326410+0000","last_became_peered":"2024-09-21T03:44:14.326410+0000","last_unstale":"2024-09-21T03:45:55.885244+0000","last_undegraded":"2024-09-21T03:45:55.885244+0000","last_fullsized":"2024-09-21T03:45:55.885244+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_clean_scrub_stamp":"2024-09-21T03:44:13.292303+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:13:22.716940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"79'4","reported_seq":22,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:45:55.874578+0000","last_change":"2024-09-21T03:44:14.318875+0000","last_active":"2024-09-21T03:45:55.874578+0000","last_peered":"2024-09-21T03:45:55.874578+0000","last_clean":"2024-09-21T03:45:55.874578+0000","last_became_active":"2024-09-21T03:44:14.318425+0000","last_became_peered":"2024-09-21T03:44:14.318425+0000","last_unstale":"2024-09-21T03:45:55.874578+0000","last_undegraded":"2024-09-21T03:45:55.874578+0000","last_fullsized":"2024-09-21T03:45:55.874578+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_clean_scrub_stamp":"2024-09-21T03:44:13.292303+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:45:39.916587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"79'6","reported_seq":24,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:45:55.892089+0000","last_change":"2024-09-21T03:44:14.325877+0000","last_active":"2024-09-21T03:45:55.892089+0000","last_peered":"2024-09-21T03:45:55.892089+0000","last_clean":"2024-09-21T03:45:55.892089+0000","last_became_active":"2024-09-21T03:44:14.325451+0000","last_became_peered":"2024-09-21T03:44:14.325451+0000","last_unstale":"2024-09-21T03:45:55.892089+0000","last_undegraded":"2024-09-21T03:45:55.892089+0000","last_fullsized":"2024-09-21T03:45:55.892089+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_clean_scrub_stamp":"2024-09-21T03:44:13.292303+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:30:42.427054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"79'5","reported_seq":23,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:45:55.888426+0000","last_change":"2024-09-21T03:44:14.323192+0000","last_active":"2024-09-21T03:45:55.888426+0000","last_peered":"2024-09-21T03:45:55.888426+0000","last_clean":"2024-09-21T03:45:55.888426+0000","last_became_active":"2024-09-21T03:44:14.322487+0000","last_became_peered":"2024-09-21T03:44:14.322487+0000","last_unstale":"2024-09-21T03:45:55.888426+0000","last_undegraded":"2024-09-21T03:45:55.888426+0000","last_fullsized":"2024-09-21T03:45:55.888426+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_clean_scrub_stamp":"2024-09-21T03:44:13.292303+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:10:48.770536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"79'9","reported_seq":21,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:45:55.892165+0000","last_change":"2024-09-21T03:44:27.612681+0000","last_active":"2024-09-21T03:45:55.892165+0000","last_peered":"2024-09-21T03:45:55.892165+0000","last_clean":"2024-09-21T03:45:55.892165+0000","last_became_active":"2024-09-21T03:44:27.612200+0000","last_became_peered":"2024-09-21T03:44:27.612200+0000","last_unstale":"2024-09-21T03:45:55.892165+0000","last_undegraded":"2024-09-21T03:45:55.892165+0000","last_fullsized":"2024-09-21T03:45:55.892165+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_clean_scrub_stamp":"2024-09-21T03:44:13.292303+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:44:58.848571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,5,10],"acting":[9,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.37","version":"79'11","reported_seq":29,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:45:55.900520+0000","last_change":"2024-09-21T03:44:14.325820+0000","last_active":"2024-09-21T03:45:55.900520+0000","last_peered":"2024-09-21T03:45:55.900520+0000","last_clean":"2024-09-21T03:45:55.900520+0000","last_became_active":"2024-09-21T03:44:14.325070+0000","last_became_peered":"2024-09-21T03:44:14.325070+0000","last_unstale":"2024-09-21T03:45:55.900520+0000","last_undegraded":"2024-09-21T03:45:55.900520+0000","last_fullsized":"2024-09-21T03:45:55.900520+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_clean_scrub_stamp":"2024-09-21T03:44:13.292303+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:54:01.072623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"79'16","reported_seq":34,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:45:55.894398+0000","last_change":"2024-09-21T03:44:14.323746+0000","last_active":"2024-09-21T03:45:55.894398+0000","last_peered":"2024-09-21T03:45:55.894398+0000","last_clean":"2024-09-21T03:45:55.894398+0000","last_became_active":"2024-09-21T03:44:14.323346+0000","last_became_peered":"2024-09-21T03:44:14.323346+0000","last_unstale":"2024-09-21T03:45:55.894398+0000","last_undegraded":"2024-09-21T03:45:55.894398+0000","last_fullsized":"2024-09-21T03:45:55.894398+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_clean_scrub_stamp":"2024-09-21T03:44:13.292303+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:17:44.603801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"79'6","reported_seq":24,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:45:55.896278+0000","last_change":"2024-09-21T03:44:14.328733+0000","last_active":"2024-09-21T03:45:55.896278+0000","last_peered":"2024-09-21T03:45:55.896278+0000","last_clean":"2024-09-21T03:45:55.896278+0000","last_became_active":"2024-09-21T03:44:14.328378+0000","last_became_peered":"2024-09-21T03:44:14.328378+0000","last_unstale":"2024-09-21T03:45:55.896278+0000","last_undegraded":"2024-09-21T03:45:55.896278+0000","last_fullsized":"2024-09-21T03:45:55.896278+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_clean_scrub_stamp":"2024-09-21T03:44:13.292303+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:23:38.497413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"79'7","reported_seq":25,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:45:55.895287+0000","last_change":"2024-09-21T03:44:14.328505+0000","last_active":"2024-09-21T03:45:55.895287+0000","last_peered":"2024-09-21T03:45:55.895287+0000","last_clean":"2024-09-21T03:45:55.895287+0000","last_became_active":"2024-09-21T03:44:14.328175+0000","last_became_peered":"2024-09-21T03:44:14.328175+0000","last_unstale":"2024-09-21T03:45:55.895287+0000","last_undegraded":"2024-09-21T03:45:55.895287+0000","last_fullsized":"2024-09-21T03:45:55.895287+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_clean_scrub_stamp":"2024-09-21T03:44:13.292303+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:45:56.628962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"79'10","reported_seq":28,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:45:55.903680+0000","last_change":"2024-09-21T03:44:14.314519+0000","last_active":"2024-09-21T03:45:55.903680+0000","last_peered":"2024-09-21T03:45:55.903680+0000","last_clean":"2024-09-21T03:45:55.903680+0000","last_became_active":"2024-09-21T03:44:14.314145+0000","last_became_peered":"2024-09-21T03:44:14.314145+0000","last_unstale":"2024-09-21T03:45:55.903680+0000","last_undegraded":"2024-09-21T03:45:55.903680+0000","last_fullsized":"2024-09-21T03:45:55.903680+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_clean_scrub_stamp":"2024-09-21T03:44:13.292303+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:45:48.494667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"79'7","reported_seq":25,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:45:55.902926+0000","last_change":"2024-09-21T03:44:14.312195+0000","last_active":"2024-09-21T03:45:55.902926+0000","last_peered":"2024-09-21T03:45:55.902926+0000","last_clean":"2024-09-21T03:45:55.902926+0000","last_became_active":"2024-09-21T03:44:14.311861+0000","last_became_peered":"2024-09-21T03:44:14.311861+0000","last_unstale":"2024-09-21T03:45:55.902926+0000","last_undegraded":"2024-09-21T03:45:55.902926+0000","last_fullsized":"2024-09-21T03:45:55.902926+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_clean_scrub_stamp":"2024-09-21T03:44:13.292303+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:17:05.025898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"79'7","reported_seq":25,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:45:55.905677+0000","last_change":"2024-09-21T03:44:14.323532+0000","last_active":"2024-09-21T03:45:55.905677+0000","last_peered":"2024-09-21T03:45:55.905677+0000","last_clean":"2024-09-21T03:45:55.905677+0000","last_became_active":"2024-09-21T03:44:14.323171+0000","last_became_peered":"2024-09-21T03:44:14.323171+0000","last_unstale":"2024-09-21T03:45:55.905677+0000","last_undegraded":"2024-09-21T03:45:55.905677+0000","last_fullsized":"2024-09-21T03:45:55.905677+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_clean_scrub_stamp":"2024-09-21T03:44:13.292303+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:23:28.999129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"79'9","reported_seq":27,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:45:55.903498+0000","last_change":"2024-09-21T03:44:14.324998+0000","last_active":"2024-09-21T03:45:55.903498+0000","last_peered":"2024-09-21T03:45:55.903498+0000","last_clean":"2024-09-21T03:45:55.903498+0000","last_became_active":"2024-09-21T03:44:14.324019+0000","last_became_peered":"2024-09-21T03:44:14.324019+0000","last_unstale":"2024-09-21T03:45:55.903498+0000","last_undegraded":"2024-09-21T03:45:55.903498+0000","last_fullsized":"2024-09-21T03:45:55.903498+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_clean_scrub_stamp":"2024-09-21T03:44:13.292303+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:49:12.965299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"79'8","reported_seq":26,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:45:55.900363+0000","last_change":"2024-09-21T03:44:14.325603+0000","last_active":"2024-09-21T03:45:55.900363+0000","last_peered":"2024-09-21T03:45:55.900363+0000","last_clean":"2024-09-21T03:45:55.900363+0000","last_became_active":"2024-09-21T03:44:14.325208+0000","last_became_peered":"2024-09-21T03:44:14.325208+0000","last_unstale":"2024-09-21T03:45:55.900363+0000","last_undegraded":"2024-09-21T03:45:55.900363+0000","last_fullsized":"2024-09-21T03:45:55.900363+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_clean_scrub_stamp":"2024-09-21T03:44:13.292303+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:11:33.794242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"79'10","reported_seq":28,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:45:55.891232+0000","last_change":"2024-09-21T03:44:14.317545+0000","last_active":"2024-09-21T03:45:55.891232+0000","last_peered":"2024-09-21T03:45:55.891232+0000","last_clean":"2024-09-21T03:45:55.891232+0000","last_became_active":"2024-09-21T03:44:14.317069+0000","last_became_peered":"2024-09-21T03:44:14.317069+0000","last_unstale":"2024-09-21T03:45:55.891232+0000","last_undegraded":"2024-09-21T03:45:55.891232+0000","last_fullsized":"2024-09-21T03:45:55.891232+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_clean_scrub_stamp":"2024-09-21T03:44:13.292303+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:16:11.734887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"79'7","reported_seq":25,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:45:55.903043+0000","last_change":"2024-09-21T03:44:14.322055+0000","last_active":"2024-09-21T03:45:55.903043+0000","last_peered":"2024-09-21T03:45:55.903043+0000","last_clean":"2024-09-21T03:45:55.903043+0000","last_became_active":"2024-09-21T03:44:14.321652+0000","last_became_peered":"2024-09-21T03:44:14.321652+0000","last_unstale":"2024-09-21T03:45:55.903043+0000","last_undegraded":"2024-09-21T03:45:55.903043+0000","last_fullsized":"2024-09-21T03:45:55.903043+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_clean_scrub_stamp":"2024-09-21T03:44:13.292303+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:45:14.285844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2c","version":"79'8","reported_seq":33,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:45:55.896619+0000","last_change":"2024-09-21T03:44:27.423175+0000","last_active":"2024-09-21T03:45:55.896619+0000","last_peered":"2024-09-21T03:45:55.896619+0000","last_clean":"2024-09-21T03:45:55.896619+0000","last_became_active":"2024-09-21T03:44:27.422733+0000","last_became_peered":"2024-09-21T03:44:27.422733+0000","last_unstale":"2024-09-21T03:45:55.896619+0000","last_undegraded":"2024-09-21T03:45:55.896619+0000","last_fullsized":"2024-09-21T03:45:55.896619+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_clean_scrub_stamp":"2024-09-21T03:44:13.292303+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:46:17.874557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,8],"acting":[11,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2b","version":"79'11","reported_seq":29,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:45:55.903246+0000","last_change":"2024-09-21T03:44:14.325829+0000","last_active":"2024-09-21T03:45:55.903246+0000","last_peered":"2024-09-21T03:45:55.903246+0000","last_clean":"2024-09-21T03:45:55.903246+0000","last_became_active":"2024-09-21T03:44:14.325421+0000","last_became_peered":"2024-09-21T03:44:14.325421+0000","last_unstale":"2024-09-21T03:45:55.903246+0000","last_undegraded":"2024-09-21T03:45:55.903246+0000","last_fullsized":"2024-09-21T03:45:55.903246+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_clean_scrub_stamp":"2024-09-21T03:44:13.292303+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:49:00.240771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2a","version":"79'10","reported_seq":28,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:45:55.907357+0000","last_change":"2024-09-21T03:44:14.325255+0000","last_active":"2024-09-21T03:45:55.907357+0000","last_peered":"2024-09-21T03:45:55.907357+0000","last_clean":"2024-09-21T03:45:55.907357+0000","last_became_active":"2024-09-21T03:44:14.324617+0000","last_became_peered":"2024-09-21T03:44:14.324617+0000","last_unstale":"2024-09-21T03:45:55.907357+0000","last_undegraded":"2024-09-21T03:45:55.907357+0000","last_fullsized":"2024-09-21T03:45:55.907357+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_clean_scrub_stamp":"2024-09-21T03:44:13.292303+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:05:03.268522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"79'8","reported_seq":26,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:45:55.885845+0000","last_change":"2024-09-21T03:44:14.328053+0000","last_active":"2024-09-21T03:45:55.885845+0000","last_peered":"2024-09-21T03:45:55.885845+0000","last_clean":"2024-09-21T03:45:55.885845+0000","last_became_active":"2024-09-21T03:44:14.327735+0000","last_became_peered":"2024-09-21T03:44:14.327735+0000","last_unstale":"2024-09-21T03:45:55.885845+0000","last_undegraded":"2024-09-21T03:45:55.885845+0000","last_fullsized":"2024-09-21T03:45:55.885845+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_clean_scrub_stamp":"2024-09-21T03:44:13.292303+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:39:48.395784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"79'6","reported_seq":24,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:45:55.888696+0000","last_change":"2024-09-21T03:44:14.321181+0000","last_active":"2024-09-21T03:45:55.888696+0000","last_peered":"2024-09-21T03:45:55.888696+0000","last_clean":"2024-09-21T03:45:55.888696+0000","last_became_active":"2024-09-21T03:44:14.320772+0000","last_became_peered":"2024-09-21T03:44:14.320772+0000","last_unstale":"2024-09-21T03:45:55.888696+0000","last_undegraded":"2024-09-21T03:45:55.888696+0000","last_fullsized":"2024-09-21T03:45:55.888696+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_clean_scrub_stamp":"2024-09-21T03:44:13.292303+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:49:26.317404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"79'2","reported_seq":20,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:45:55.884461+0000","last_change":"2024-09-21T03:44:14.322643+0000","last_active":"2024-09-21T03:45:55.884461+0000","last_peered":"2024-09-21T03:45:55.884461+0000","last_clean":"2024-09-21T03:45:55.884461+0000","last_became_active":"2024-09-21T03:44:14.322254+0000","last_became_peered":"2024-09-21T03:44:14.322254+0000","last_unstale":"2024-09-21T03:45:55.884461+0000","last_undegraded":"2024-09-21T03:45:55.884461+0000","last_fullsized":"2024-09-21T03:45:55.884461+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_clean_scrub_stamp":"2024-09-21T03:44:13.292303+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:31:47.147845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"79'15","reported_seq":33,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:45:55.907955+0000","last_change":"2024-09-21T03:44:14.323090+0000","last_active":"2024-09-21T03:45:55.907955+0000","last_peered":"2024-09-21T03:45:55.907955+0000","last_clean":"2024-09-21T03:45:55.907955+0000","last_became_active":"2024-09-21T03:44:14.322669+0000","last_became_peered":"2024-09-21T03:44:14.322669+0000","last_unstale":"2024-09-21T03:45:55.907955+0000","last_undegraded":"2024-09-21T03:45:55.907955+0000","last_fullsized":"2024-09-21T03:45:55.907955+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_clean_scrub_stamp":"2024-09-21T03:44:13.292303+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:08:56.852422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"79'6","reported_seq":24,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:45:55.896312+0000","last_change":"2024-09-21T03:44:14.318146+0000","last_active":"2024-09-21T03:45:55.896312+0000","last_peered":"2024-09-21T03:45:55.896312+0000","last_clean":"2024-09-21T03:45:55.896312+0000","last_became_active":"2024-09-21T03:44:14.317761+0000","last_became_peered":"2024-09-21T03:44:14.317761+0000","last_unstale":"2024-09-21T03:45:55.896312+0000","last_undegraded":"2024-09-21T03:45:55.896312+0000","last_fullsized":"2024-09-21T03:45:55.896312+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_clean_scrub_stamp":"2024-09-21T03:44:13.292303+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:07:31.530728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.24","version":"79'4","reported_seq":22,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:45:55.902467+0000","last_change":"2024-09-21T03:44:14.316832+0000","last_active":"2024-09-21T03:45:55.902467+0000","last_peered":"2024-09-21T03:45:55.902467+0000","last_clean":"2024-09-21T03:45:55.902467+0000","last_became_active":"2024-09-21T03:44:14.316471+0000","last_became_peered":"2024-09-21T03:44:14.316471+0000","last_unstale":"2024-09-21T03:45:55.902467+0000","last_undegraded":"2024-09-21T03:45:55.902467+0000","last_fullsized":"2024-09-21T03:45:55.902467+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_clean_scrub_stamp":"2024-09-21T03:44:13.292303+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:21:23.848778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"79'7","reported_seq":25,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:45:55.905316+0000","last_change":"2024-09-21T03:44:14.322277+0000","last_active":"2024-09-21T03:45:55.905316+0000","last_peered":"2024-09-21T03:45:55.905316+0000","last_clean":"2024-09-21T03:45:55.905316+0000","last_became_active":"2024-09-21T03:44:14.321952+0000","last_became_peered":"2024-09-21T03:44:14.321952+0000","last_unstale":"2024-09-21T03:45:55.905316+0000","last_undegraded":"2024-09-21T03:45:55.905316+0000","last_fullsized":"2024-09-21T03:45:55.905316+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_clean_scrub_stamp":"2024-09-21T03:44:13.292303+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:43:55.169380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"79'8","reported_seq":26,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:45:55.886351+0000","last_change":"2024-09-21T03:44:14.322626+0000","last_active":"2024-09-21T03:45:55.886351+0000","last_peered":"2024-09-21T03:45:55.886351+0000","last_clean":"2024-09-21T03:45:55.886351+0000","last_became_active":"2024-09-21T03:44:14.321559+0000","last_became_peered":"2024-09-21T03:44:14.321559+0000","last_unstale":"2024-09-21T03:45:55.886351+0000","last_undegraded":"2024-09-21T03:45:55.886351+0000","last_fullsized":"2024-09-21T03:45:55.886351+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_clean_scrub_stamp":"2024-09-21T03:44:13.292303+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:12:23.736500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"79'12","reported_seq":30,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:45:55.886765+0000","last_change":"2024-09-21T03:44:14.318057+0000","last_active":"2024-09-21T03:45:55.886765+0000","last_peered":"2024-09-21T03:45:55.886765+0000","last_clean":"2024-09-21T03:45:55.886765+0000","last_became_active":"2024-09-21T03:44:14.317653+0000","last_became_peered":"2024-09-21T03:44:14.317653+0000","last_unstale":"2024-09-21T03:45:55.886765+0000","last_undegraded":"2024-09-21T03:45:55.886765+0000","last_fullsized":"2024-09-21T03:45:55.886765+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_clean_scrub_stamp":"2024-09-21T03:44:13.292303+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:23:17.396599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"79'9","reported_seq":27,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:45:55.895204+0000","last_change":"2024-09-21T03:44:14.322331+0000","last_active":"2024-09-21T03:45:55.895204+0000","last_peered":"2024-09-21T03:45:55.895204+0000","last_clean":"2024-09-21T03:45:55.895204+0000","last_became_active":"2024-09-21T03:44:14.321157+0000","last_became_peered":"2024-09-21T03:44:14.321157+0000","last_unstale":"2024-09-21T03:45:55.895204+0000","last_undegraded":"2024-09-21T03:45:55.895204+0000","last_fullsized":"2024-09-21T03:45:55.895204+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_clean_scrub_stamp":"2024-09-21T03:44:13.292303+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:08:27.953449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1a","version":"79'7","reported_seq":32,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:45:55.890546+0000","last_change":"2024-09-21T03:44:27.423081+0000","last_active":"2024-09-21T03:45:55.890546+0000","last_peered":"2024-09-21T03:45:55.890546+0000","last_clean":"2024-09-21T03:45:55.890546+0000","last_became_active":"2024-09-21T03:44:27.422650+0000","last_became_peered":"2024-09-21T03:44:27.422650+0000","last_unstale":"2024-09-21T03:45:55.890546+0000","last_undegraded":"2024-09-21T03:45:55.890546+0000","last_fullsized":"2024-09-21T03:45:55.890546+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_clean_scrub_stamp":"2024-09-21T03:44:13.292303+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:32:44.399954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,7],"acting":[8,9,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1b","version":"79'7","reported_seq":139,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:46:07.857454+0000","last_change":"2024-09-21T03:44:12.307464+0000","last_active":"2024-09-21T03:46:07.857454+0000","last_peered":"2024-09-21T03:46:07.857454+0000","last_clean":"2024-09-21T03:46:07.857454+0000","last_became_active":"2024-09-21T03:44:12.307077+0000","last_became_peered":"2024-09-21T03:44:12.307077+0000","last_unstale":"2024-09-21T03:46:07.857454+0000","last_undegraded":"2024-09-21T03:46:07.857454+0000","last_fullsized":"2024-09-21T03:46:07.857454+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:11.282438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:11.282438+0000","last_clean_scrub_stamp":"2024-09-21T03:44:11.282438+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:50:40.344659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1154,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":109,"num_read_kb":4,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"79'8","reported_seq":26,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:45:55.891478+0000","last_change":"2024-09-21T03:44:14.319072+0000","last_active":"2024-09-21T03:45:55.891478+0000","last_peered":"2024-09-21T03:45:55.891478+0000","last_clean":"2024-09-21T03:45:55.891478+0000","last_became_active":"2024-09-21T03:44:14.318712+0000","last_became_peered":"2024-09-21T03:44:14.318712+0000","last_unstale":"2024-09-21T03:45:55.891478+0000","last_undegraded":"2024-09-21T03:45:55.891478+0000","last_fullsized":"2024-09-21T03:45:55.891478+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_clean_scrub_stamp":"2024-09-21T03:44:13.292303+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:30:35.618110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1a","version":"77'7","reported_seq":30,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:44:27.611763+0000","last_change":"2024-09-21T03:44:12.310146+0000","last_active":"2024-09-21T03:44:27.611763+0000","last_peered":"2024-09-21T03:44:27.611763+0000","last_clean":"2024-09-21T03:44:27.611763+0000","last_became_active":"2024-09-21T03:44:12.309817+0000","last_became_peered":"2024-09-21T03:44:12.309817+0000","last_unstale":"2024-09-21T03:44:27.611763+0000","last_undegraded":"2024-09-21T03:44:27.611763+0000","last_fullsized":"2024-09-21T03:44:27.611763+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:11.282438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:11.282438+0000","last_clean_scrub_stamp":"2024-09-21T03:44:11.282438+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:46:51.994488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.19","version":"77'3","reported_seq":26,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:44:27.410211+0000","last_change":"2024-09-21T03:44:12.308995+0000","last_active":"2024-09-21T03:44:27.410211+0000","last_peered":"2024-09-21T03:44:27.410211+0000","last_clean":"2024-09-21T03:44:27.410211+0000","last_became_active":"2024-09-21T03:44:12.308058+0000","last_became_peered":"2024-09-21T03:44:12.308058+0000","last_unstale":"2024-09-21T03:44:27.410211+0000","last_undegraded":"2024-09-21T03:44:27.410211+0000","last_fullsized":"2024-09-21T03:44:27.410211+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:11.282438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:11.282438+0000","last_clean_scrub_stamp":"2024-09-21T03:44:11.282438+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:48:39.384985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"79'8","reported_seq":26,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:45:55.886620+0000","last_change":"2024-09-21T03:44:14.317878+0000","last_active":"2024-09-21T03:45:55.886620+0000","last_peered":"2024-09-21T03:45:55.886620+0000","last_clean":"2024-09-21T03:45:55.886620+0000","last_became_active":"2024-09-21T03:44:14.317521+0000","last_became_peered":"2024-09-21T03:44:14.317521+0000","last_unstale":"2024-09-21T03:45:55.886620+0000","last_undegraded":"2024-09-21T03:45:55.886620+0000","last_fullsized":"2024-09-21T03:45:55.886620+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_clean_scrub_stamp":"2024-09-21T03:44:13.292303+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:08:00.705210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"77'3","reported_seq":27,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:44:27.414041+0000","last_change":"2024-09-21T03:44:12.308777+0000","last_active":"2024-09-21T03:44:27.414041+0000","last_peered":"2024-09-21T03:44:27.414041+0000","last_clean":"2024-09-21T03:44:27.414041+0000","last_became_active":"2024-09-21T03:44:12.308171+0000","last_became_peered":"2024-09-21T03:44:12.308171+0000","last_unstale":"2024-09-21T03:44:27.414041+0000","last_undegraded":"2024-09-21T03:44:27.414041+0000","last_fullsized":"2024-09-21T03:44:27.414041+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:11.282438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:11.282438+0000","last_clean_scrub_stamp":"2024-09-21T03:44:11.282438+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:41:21.344491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"79'11","reported_seq":29,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:45:55.892137+0000","last_change":"2024-09-21T03:44:14.315782+0000","last_active":"2024-09-21T03:45:55.892137+0000","last_peered":"2024-09-21T03:45:55.892137+0000","last_clean":"2024-09-21T03:45:55.892137+0000","last_became_active":"2024-09-21T03:44:14.315314+0000","last_became_peered":"2024-09-21T03:44:14.315314+0000","last_unstale":"2024-09-21T03:45:55.892137+0000","last_undegraded":"2024-09-21T03:45:55.892137+0000","last_fullsized":"2024-09-21T03:45:55.892137+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_clean_scrub_stamp":"2024-09-21T03:44:13.292303+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:22:20.500220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"79'5","reported_seq":236,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:46:07.856800+0000","last_change":"2024-09-21T03:44:12.313251+0000","last_active":"2024-09-21T03:46:07.856800+0000","last_peered":"2024-09-21T03:46:07.856800+0000","last_clean":"2024-09-21T03:46:07.856800+0000","last_became_active":"2024-09-21T03:44:12.312941+0000","last_became_peered":"2024-09-21T03:44:12.312941+0000","last_unstale":"2024-09-21T03:46:07.856800+0000","last_undegraded":"2024-09-21T03:46:07.856800+0000","last_fullsized":"2024-09-21T03:46:07.856800+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:11.282438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:11.282438+0000","last_clean_scrub_stamp":"2024-09-21T03:44:11.282438+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:38:39.876097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":208,"num_read_kb":208,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"79'10","reported_seq":28,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:45:55.905106+0000","last_change":"2024-09-21T03:44:14.319621+0000","last_active":"2024-09-21T03:45:55.905106+0000","last_peered":"2024-09-21T03:45:55.905106+0000","last_clean":"2024-09-21T03:45:55.905106+0000","last_became_active":"2024-09-21T03:44:14.316173+0000","last_became_peered":"2024-09-21T03:44:14.316173+0000","last_unstale":"2024-09-21T03:45:55.905106+0000","last_undegraded":"2024-09-21T03:45:55.905106+0000","last_fullsized":"2024-09-21T03:45:55.905106+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_clean_scrub_stamp":"2024-09-21T03:44:13.292303+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:10:21.952989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"77'4","reported_seq":27,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:44:27.419175+0000","last_change":"2024-09-21T03:44:12.308893+0000","last_active":"2024-09-21T03:44:27.419175+0000","last_peered":"2024-09-21T03:44:27.419175+0000","last_clean":"2024-09-21T03:44:27.419175+0000","last_became_active":"2024-09-21T03:44:12.308551+0000","last_became_peered":"2024-09-21T03:44:12.308551+0000","last_unstale":"2024-09-21T03:44:27.419175+0000","last_undegraded":"2024-09-21T03:44:27.419175+0000","last_fullsized":"2024-09-21T03:44:27.419175+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:11.282438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:11.282438+0000","last_clean_scrub_stamp":"2024-09-21T03:44:11.282438+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:48:18.401595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"79'9","reported_seq":27,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:45:55.904971+0000","last_change":"2024-09-21T03:44:14.321875+0000","last_active":"2024-09-21T03:45:55.904971+0000","last_peered":"2024-09-21T03:45:55.904971+0000","last_clean":"2024-09-21T03:45:55.904971+0000","last_became_active":"2024-09-21T03:44:14.321483+0000","last_became_peered":"2024-09-21T03:44:14.321483+0000","last_unstale":"2024-09-21T03:45:55.904971+0000","last_undegraded":"2024-09-21T03:45:55.904971+0000","last_fullsized":"2024-09-21T03:45:55.904971+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_clean_scrub_stamp":"2024-09-21T03:44:13.292303+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:40:17.244780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"77'5","reported_seq":29,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:44:27.414461+0000","last_change":"2024-09-21T03:44:12.313208+0000","last_active":"2024-09-21T03:44:27.414461+0000","last_peered":"2024-09-21T03:44:27.414461+0000","last_clean":"2024-09-21T03:44:27.414461+0000","last_became_active":"2024-09-21T03:44:12.312783+0000","last_became_peered":"2024-09-21T03:44:12.312783+0000","last_unstale":"2024-09-21T03:44:27.414461+0000","last_undegraded":"2024-09-21T03:44:27.414461+0000","last_fullsized":"2024-09-21T03:44:27.414461+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:11.282438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:11.282438+0000","last_clean_scrub_stamp":"2024-09-21T03:44:11.282438+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:18:45.114361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"79'11","reported_seq":36,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:45:55.907006+0000","last_change":"2024-09-21T03:44:27.421129+0000","last_active":"2024-09-21T03:45:55.907006+0000","last_peered":"2024-09-21T03:45:55.907006+0000","last_clean":"2024-09-21T03:45:55.907006+0000","last_became_active":"2024-09-21T03:44:27.420642+0000","last_became_peered":"2024-09-21T03:44:27.420642+0000","last_unstale":"2024-09-21T03:45:55.907006+0000","last_undegraded":"2024-09-21T03:45:55.907006+0000","last_fullsized":"2024-09-21T03:45:55.907006+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_clean_scrub_stamp":"2024-09-21T03:44:13.292303+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:49:38.065743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,6],"acting":[1,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"79'34","reported_seq":277,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:46:07.917025+0000","last_change":"2024-09-21T03:44:12.314863+0000","last_active":"2024-09-21T03:46:07.917025+0000","last_peered":"2024-09-21T03:46:07.917025+0000","last_clean":"2024-09-21T03:46:07.917025+0000","last_became_active":"2024-09-21T03:44:12.314471+0000","last_became_peered":"2024-09-21T03:44:12.314471+0000","last_unstale":"2024-09-21T03:46:07.917025+0000","last_undegraded":"2024-09-21T03:46:07.917025+0000","last_fullsized":"2024-09-21T03:46:07.917025+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:11.282438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:11.282438+0000","last_clean_scrub_stamp":"2024-09-21T03:44:11.282438+0000","objects_scrubbed":0,"log_size":34,"log_dups_size":0,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:41:03.471226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":64459,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":220,"num_read_kb":173,"num_write":36,"num_write_kb":79,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"79'6","reported_seq":24,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:45:55.897616+0000","last_change":"2024-09-21T03:44:14.325471+0000","last_active":"2024-09-21T03:45:55.897616+0000","last_peered":"2024-09-21T03:45:55.897616+0000","last_clean":"2024-09-21T03:45:55.897616+0000","last_became_active":"2024-09-21T03:44:14.325072+0000","last_became_peered":"2024-09-21T03:44:14.325072+0000","last_unstale":"2024-09-21T03:45:55.897616+0000","last_undegraded":"2024-09-21T03:45:55.897616+0000","last_fullsized":"2024-09-21T03:45:55.897616+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_clean_scrub_stamp":"2024-09-21T03:44:13.292303+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:40:40.097240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"77'9","reported_seq":34,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:44:27.416778+0000","last_change":"2024-09-21T03:44:12.312932+0000","last_active":"2024-09-21T03:44:27.416778+0000","last_peered":"2024-09-21T03:44:27.416778+0000","last_clean":"2024-09-21T03:44:27.416778+0000","last_became_active":"2024-09-21T03:44:12.312368+0000","last_became_peered":"2024-09-21T03:44:12.312368+0000","last_unstale":"2024-09-21T03:44:27.416778+0000","last_undegraded":"2024-09-21T03:44:27.416778+0000","last_fullsized":"2024-09-21T03:44:27.416778+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:11.282438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:11.282438+0000","last_clean_scrub_stamp":"2024-09-21T03:44:11.282438+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:28:11.579598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"79'9","reported_seq":27,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:45:55.901852+0000","last_change":"2024-09-21T03:44:14.329564+0000","last_active":"2024-09-21T03:45:55.901852+0000","last_peered":"2024-09-21T03:45:55.901852+0000","last_clean":"2024-09-21T03:45:55.901852+0000","last_became_active":"2024-09-21T03:44:14.329122+0000","last_became_peered":"2024-09-21T03:44:14.329122+0000","last_unstale":"2024-09-21T03:45:55.901852+0000","last_undegraded":"2024-09-21T03:45:55.901852+0000","last_fullsized":"2024-09-21T03:45:55.901852+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_clean_scrub_stamp":"2024-09-21T03:44:13.292303+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:16:32.702770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"77'5","reported_seq":29,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:44:27.413875+0000","last_change":"2024-09-21T03:44:12.303339+0000","last_active":"2024-09-21T03:44:27.413875+0000","last_peered":"2024-09-21T03:44:27.413875+0000","last_clean":"2024-09-21T03:44:27.413875+0000","last_became_active":"2024-09-21T03:44:12.302905+0000","last_became_peered":"2024-09-21T03:44:12.302905+0000","last_unstale":"2024-09-21T03:44:27.413875+0000","last_undegraded":"2024-09-21T03:44:27.413875+0000","last_fullsized":"2024-09-21T03:44:27.413875+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:11.282438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:11.282438+0000","last_clean_scrub_stamp":"2024-09-21T03:44:11.282438+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:57:16.424976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"79'15","reported_seq":33,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:45:55.905704+0000","last_change":"2024-09-21T03:44:14.323473+0000","last_active":"2024-09-21T03:45:55.905704+0000","last_peered":"2024-09-21T03:45:55.905704+0000","last_clean":"2024-09-21T03:45:55.905704+0000","last_became_active":"2024-09-21T03:44:14.322560+0000","last_became_peered":"2024-09-21T03:44:14.322560+0000","last_unstale":"2024-09-21T03:45:55.905704+0000","last_undegraded":"2024-09-21T03:45:55.905704+0000","last_fullsized":"2024-09-21T03:45:55.905704+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_clean_scrub_stamp":"2024-09-21T03:44:13.292303+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:07:55.714351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"79'8","reported_seq":142,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:46:09.861993+0000","last_change":"2024-09-21T03:44:12.312331+0000","last_active":"2024-09-21T03:46:09.861993+0000","last_peered":"2024-09-21T03:46:09.861993+0000","last_clean":"2024-09-21T03:46:09.861993+0000","last_became_active":"2024-09-21T03:44:12.311984+0000","last_became_peered":"2024-09-21T03:44:12.311984+0000","last_unstale":"2024-09-21T03:46:09.861993+0000","last_undegraded":"2024-09-21T03:46:09.861993+0000","last_fullsized":"2024-09-21T03:46:09.861993+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:11.282438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:11.282438+0000","last_clean_scrub_stamp":"2024-09-21T03:44:11.282438+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:06:11.906592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1511,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":112,"num_read_kb":6,"num_write":8,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"79'6","reported_seq":24,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:45:55.885280+0000","last_change":"2024-09-21T03:44:14.318437+0000","last_active":"2024-09-21T03:45:55.885280+0000","last_peered":"2024-09-21T03:45:55.885280+0000","last_clean":"2024-09-21T03:45:55.885280+0000","last_became_active":"2024-09-21T03:44:14.318052+0000","last_became_peered":"2024-09-21T03:44:14.318052+0000","last_unstale":"2024-09-21T03:45:55.885280+0000","last_undegraded":"2024-09-21T03:45:55.885280+0000","last_fullsized":"2024-09-21T03:45:55.885280+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_clean_scrub_stamp":"2024-09-21T03:44:13.292303+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:10:12.136159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"77'17","reported_seq":41,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:44:27.419831+0000","last_change":"2024-09-21T03:44:12.310616+0000","last_active":"2024-09-21T03:44:27.419831+0000","last_peered":"2024-09-21T03:44:27.419831+0000","last_clean":"2024-09-21T03:44:27.419831+0000","last_became_active":"2024-09-21T03:44:12.310239+0000","last_became_peered":"2024-09-21T03:44:12.310239+0000","last_unstale":"2024-09-21T03:44:27.419831+0000","last_undegraded":"2024-09-21T03:44:27.419831+0000","last_fullsized":"2024-09-21T03:44:27.419831+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:11.282438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:11.282438+0000","last_clean_scrub_stamp":"2024-09-21T03:44:11.282438+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:43:38.046281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"79'6","reported_seq":24,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:45:55.894707+0000","last_change":"2024-09-21T03:44:14.327058+0000","last_active":"2024-09-21T03:45:55.894707+0000","last_peered":"2024-09-21T03:45:55.894707+0000","last_clean":"2024-09-21T03:45:55.894707+0000","last_became_active":"2024-09-21T03:44:14.326641+0000","last_became_peered":"2024-09-21T03:44:14.326641+0000","last_unstale":"2024-09-21T03:45:55.894707+0000","last_undegraded":"2024-09-21T03:45:55.894707+0000","last_fullsized":"2024-09-21T03:45:55.894707+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_clean_scrub_stamp":"2024-09-21T03:44:13.292303+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:10:50.686102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"77'2","reported_seq":26,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:44:27.420648+0000","last_change":"2024-09-21T03:44:12.310426+0000","last_active":"2024-09-21T03:44:27.420648+0000","last_peered":"2024-09-21T03:44:27.420648+0000","last_clean":"2024-09-21T03:44:27.420648+0000","last_became_active":"2024-09-21T03:44:12.309231+0000","last_became_peered":"2024-09-21T03:44:12.309231+0000","last_unstale":"2024-09-21T03:44:27.420648+0000","last_undegraded":"2024-09-21T03:44:27.420648+0000","last_fullsized":"2024-09-21T03:44:27.420648+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:11.282438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:11.282438+0000","last_clean_scrub_stamp":"2024-09-21T03:44:11.282438+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:32:18.284741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"79'5","reported_seq":23,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:45:55.887191+0000","last_change":"2024-09-21T03:44:14.319179+0000","last_active":"2024-09-21T03:45:55.887191+0000","last_peered":"2024-09-21T03:45:55.887191+0000","last_clean":"2024-09-21T03:45:55.887191+0000","last_became_active":"2024-09-21T03:44:14.318826+0000","last_became_peered":"2024-09-21T03:44:14.318826+0000","last_unstale":"2024-09-21T03:45:55.887191+0000","last_undegraded":"2024-09-21T03:45:55.887191+0000","last_fullsized":"2024-09-21T03:45:55.887191+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_clean_scrub_stamp":"2024-09-21T03:44:13.292303+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:12:57.985087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.e","version":"77'5","reported_seq":28,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:44:27.410705+0000","last_change":"2024-09-21T03:44:12.315658+0000","last_active":"2024-09-21T03:44:27.410705+0000","last_peered":"2024-09-21T03:44:27.410705+0000","last_clean":"2024-09-21T03:44:27.410705+0000","last_became_active":"2024-09-21T03:44:12.315240+0000","last_became_peered":"2024-09-21T03:44:12.315240+0000","last_unstale":"2024-09-21T03:44:27.410705+0000","last_undegraded":"2024-09-21T03:44:27.410705+0000","last_fullsized":"2024-09-21T03:44:27.410705+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:11.282438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:11.282438+0000","last_clean_scrub_stamp":"2024-09-21T03:44:11.282438+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:22:07.493437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"79'9","reported_seq":27,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:45:55.898231+0000","last_change":"2024-09-21T03:44:14.328014+0000","last_active":"2024-09-21T03:45:55.898231+0000","last_peered":"2024-09-21T03:45:55.898231+0000","last_clean":"2024-09-21T03:45:55.898231+0000","last_became_active":"2024-09-21T03:44:14.327638+0000","last_became_peered":"2024-09-21T03:44:14.327638+0000","last_unstale":"2024-09-21T03:45:55.898231+0000","last_undegraded":"2024-09-21T03:45:55.898231+0000","last_fullsized":"2024-09-21T03:45:55.898231+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_clean_scrub_stamp":"2024-09-21T03:44:13.292303+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:48:32.542776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"77'3","reported_seq":27,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:44:27.414656+0000","last_change":"2024-09-21T03:44:12.309272+0000","last_active":"2024-09-21T03:44:27.414656+0000","last_peered":"2024-09-21T03:44:27.414656+0000","last_clean":"2024-09-21T03:44:27.414656+0000","last_became_active":"2024-09-21T03:44:12.308718+0000","last_became_peered":"2024-09-21T03:44:12.308718+0000","last_unstale":"2024-09-21T03:44:27.414656+0000","last_undegraded":"2024-09-21T03:44:27.414656+0000","last_fullsized":"2024-09-21T03:44:27.414656+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:11.282438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:11.282438+0000","last_clean_scrub_stamp":"2024-09-21T03:44:11.282438+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:03:49.433966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"79'7","reported_seq":25,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:45:55.889962+0000","last_change":"2024-09-21T03:44:14.322148+0000","last_active":"2024-09-21T03:45:55.889962+0000","last_peered":"2024-09-21T03:45:55.889962+0000","last_clean":"2024-09-21T03:45:55.889962+0000","last_became_active":"2024-09-21T03:44:14.321784+0000","last_became_peered":"2024-09-21T03:44:14.321784+0000","last_unstale":"2024-09-21T03:45:55.889962+0000","last_undegraded":"2024-09-21T03:45:55.889962+0000","last_fullsized":"2024-09-21T03:45:55.889962+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_clean_scrub_stamp":"2024-09-21T03:44:13.292303+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:40:34.887134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"77'5","reported_seq":122,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:46:07.560527+0000","last_change":"2024-09-21T03:44:27.427722+0000","last_active":"2024-09-21T03:46:07.560527+0000","last_peered":"2024-09-21T03:46:07.560527+0000","last_clean":"2024-09-21T03:46:07.560527+0000","last_became_active":"2024-09-21T03:44:27.419672+0000","last_became_peered":"2024-09-21T03:44:27.419672+0000","last_unstale":"2024-09-21T03:46:07.560527+0000","last_undegraded":"2024-09-21T03:46:07.560527+0000","last_fullsized":"2024-09-21T03:46:07.560527+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:11.282438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:11.282438+0000","last_clean_scrub_stamp":"2024-09-21T03:44:11.282438+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:24:31.890439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1170,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":102,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":1170,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,11,0],"acting":[1,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.d","version":"79'7","reported_seq":25,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:45:55.897960+0000","last_change":"2024-09-21T03:44:14.316503+0000","last_active":"2024-09-21T03:45:55.897960+0000","last_peered":"2024-09-21T03:45:55.897960+0000","last_clean":"2024-09-21T03:45:55.897960+0000","last_became_active":"2024-09-21T03:44:14.316140+0000","last_became_peered":"2024-09-21T03:44:14.316140+0000","last_unstale":"2024-09-21T03:45:55.897960+0000","last_undegraded":"2024-09-21T03:45:55.897960+0000","last_fullsized":"2024-09-21T03:45:55.897960+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_clean_scrub_stamp":"2024-09-21T03:44:13.292303+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:00:30.647579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"21'92","reported_seq":97,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:44:27.418361+0000","last_change":"2024-09-21T03:43:01.568120+0000","last_active":"2024-09-21T03:44:27.418361+0000","last_peered":"2024-09-21T03:44:27.418361+0000","last_clean":"2024-09-21T03:44:27.418361+0000","last_became_active":"2024-09-21T03:43:01.476311+0000","last_became_peered":"2024-09-21T03:43:01.476311+0000","last_unstale":"2024-09-21T03:44:27.418361+0000","last_undegraded":"2024-09-21T03:44:27.418361+0000","last_fullsized":"2024-09-21T03:44:27.418361+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:39:30.121380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:39:30.121380+0000","last_clean_scrub_stamp":"2024-09-21T03:39:30.121380+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:40:27.967442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"77'6","reported_seq":32,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:44:27.613250+0000","last_change":"2024-09-21T03:44:12.311860+0000","last_active":"2024-09-21T03:44:27.613250+0000","last_peered":"2024-09-21T03:44:27.613250+0000","last_clean":"2024-09-21T03:44:27.613250+0000","last_became_active":"2024-09-21T03:44:12.311453+0000","last_became_peered":"2024-09-21T03:44:12.311453+0000","last_unstale":"2024-09-21T03:44:27.613250+0000","last_undegraded":"2024-09-21T03:44:27.613250+0000","last_fullsized":"2024-09-21T03:44:27.613250+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:11.282438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:11.282438+0000","last_clean_scrub_stamp":"2024-09-21T03:44:11.282438+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:23:40.550445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2","version":"79'8","reported_seq":20,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:45:55.883471+0000","last_change":"2024-09-21T03:44:27.416799+0000","last_active":"2024-09-21T03:45:55.883471+0000","last_peered":"2024-09-21T03:45:55.883471+0000","last_clean":"2024-09-21T03:45:55.883471+0000","last_became_active":"2024-09-21T03:44:27.416306+0000","last_became_peered":"2024-09-21T03:44:27.416306+0000","last_unstale":"2024-09-21T03:45:55.883471+0000","last_undegraded":"2024-09-21T03:45:55.883471+0000","last_fullsized":"2024-09-21T03:45:55.883471+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_clean_scrub_stamp":"2024-09-21T03:44:13.292303+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:34:26.547100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,5,1],"acting":[9,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.0","version":"77'4","reported_seq":27,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:44:27.418313+0000","last_change":"2024-09-21T03:44:12.307302+0000","last_active":"2024-09-21T03:44:27.418313+0000","last_peered":"2024-09-21T03:44:27.418313+0000","last_clean":"2024-09-21T03:44:27.418313+0000","last_became_active":"2024-09-21T03:44:12.306856+0000","last_became_peered":"2024-09-21T03:44:12.306856+0000","last_unstale":"2024-09-21T03:44:27.418313+0000","last_undegraded":"2024-09-21T03:44:27.418313+0000","last_fullsized":"2024-09-21T03:44:27.418313+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:11.282438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:11.282438+0000","last_clean_scrub_stamp":"2024-09-21T03:44:11.282438+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:21:20.420020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"79'5","reported_seq":23,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:45:55.899176+0000","last_change":"2024-09-21T03:44:14.322913+0000","last_active":"2024-09-21T03:45:55.899176+0000","last_peered":"2024-09-21T03:45:55.899176+0000","last_clean":"2024-09-21T03:45:55.899176+0000","last_became_active":"2024-09-21T03:44:14.322168+0000","last_became_peered":"2024-09-21T03:44:14.322168+0000","last_unstale":"2024-09-21T03:45:55.899176+0000","last_undegraded":"2024-09-21T03:45:55.899176+0000","last_fullsized":"2024-09-21T03:45:55.899176+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_clean_scrub_stamp":"2024-09-21T03:44:13.292303+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:30:09.041211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"79'6","reported_seq":133,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:46:05.855033+0000","last_change":"2024-09-21T03:44:12.309084+0000","last_active":"2024-09-21T03:46:05.855033+0000","last_peered":"2024-09-21T03:46:05.855033+0000","last_clean":"2024-09-21T03:46:05.855033+0000","last_became_active":"2024-09-21T03:44:12.308551+0000","last_became_peered":"2024-09-21T03:44:12.308551+0000","last_unstale":"2024-09-21T03:46:05.855033+0000","last_undegraded":"2024-09-21T03:46:05.855033+0000","last_fullsized":"2024-09-21T03:46:05.855033+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:11.282438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:11.282438+0000","last_clean_scrub_stamp":"2024-09-21T03:44:11.282438+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:46:33.895904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":104,"num_read_kb":104,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"79'8","reported_seq":26,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:45:55.884045+0000","last_change":"2024-09-21T03:44:14.322720+0000","last_active":"2024-09-21T03:45:55.884045+0000","last_peered":"2024-09-21T03:45:55.884045+0000","last_clean":"2024-09-21T03:45:55.884045+0000","last_became_active":"2024-09-21T03:44:14.322367+0000","last_became_peered":"2024-09-21T03:44:14.322367+0000","last_unstale":"2024-09-21T03:45:55.884045+0000","last_undegraded":"2024-09-21T03:45:55.884045+0000","last_fullsized":"2024-09-21T03:45:55.884045+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_clean_scrub_stamp":"2024-09-21T03:44:13.292303+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:59:51.692976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"79'8","reported_seq":137,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:46:07.760063+0000","last_change":"2024-09-21T03:44:12.308719+0000","last_active":"2024-09-21T03:46:07.760063+0000","last_peered":"2024-09-21T03:46:07.760063+0000","last_clean":"2024-09-21T03:46:07.760063+0000","last_became_active":"2024-09-21T03:44:12.308380+0000","last_became_peered":"2024-09-21T03:44:12.308380+0000","last_unstale":"2024-09-21T03:46:07.760063+0000","last_undegraded":"2024-09-21T03:46:07.760063+0000","last_fullsized":"2024-09-21T03:46:07.760063+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:11.282438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:11.282438+0000","last_clean_scrub_stamp":"2024-09-21T03:44:11.282438+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:41:02.087501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1428,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":106,"num_read_kb":4,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"79'7","reported_seq":25,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:45:55.901997+0000","last_change":"2024-09-21T03:44:14.326541+0000","last_active":"2024-09-21T03:45:55.901997+0000","last_peered":"2024-09-21T03:45:55.901997+0000","last_clean":"2024-09-21T03:45:55.901997+0000","last_became_active":"2024-09-21T03:44:14.326132+0000","last_became_peered":"2024-09-21T03:44:14.326132+0000","last_unstale":"2024-09-21T03:45:55.901997+0000","last_undegraded":"2024-09-21T03:45:55.901997+0000","last_fullsized":"2024-09-21T03:45:55.901997+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_clean_scrub_stamp":"2024-09-21T03:44:13.292303+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:09:27.711919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"77'1","reported_seq":24,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:44:27.414339+0000","last_change":"2024-09-21T03:44:12.309463+0000","last_active":"2024-09-21T03:44:27.414339+0000","last_peered":"2024-09-21T03:44:27.414339+0000","last_clean":"2024-09-21T03:44:27.414339+0000","last_became_active":"2024-09-21T03:44:12.309054+0000","last_became_peered":"2024-09-21T03:44:12.309054+0000","last_unstale":"2024-09-21T03:44:27.414339+0000","last_undegraded":"2024-09-21T03:44:27.414339+0000","last_fullsized":"2024-09-21T03:44:27.414339+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:11.282438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:11.282438+0000","last_clean_scrub_stamp":"2024-09-21T03:44:11.282438+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:12:12.505538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"79'5","reported_seq":23,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:45:55.890793+0000","last_change":"2024-09-21T03:44:14.318261+0000","last_active":"2024-09-21T03:45:55.890793+0000","last_peered":"2024-09-21T03:45:55.890793+0000","last_clean":"2024-09-21T03:45:55.890793+0000","last_became_active":"2024-09-21T03:44:14.317839+0000","last_became_peered":"2024-09-21T03:44:14.317839+0000","last_unstale":"2024-09-21T03:45:55.890793+0000","last_undegraded":"2024-09-21T03:45:55.890793+0000","last_fullsized":"2024-09-21T03:45:55.890793+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_clean_scrub_stamp":"2024-09-21T03:44:13.292303+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:19:05.504749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.5","version":"77'4","reported_seq":20,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:44:27.434085+0000","last_change":"2024-09-21T03:44:27.433630+0000","last_active":"2024-09-21T03:44:27.434085+0000","last_peered":"2024-09-21T03:44:27.434085+0000","last_clean":"2024-09-21T03:44:27.434085+0000","last_became_active":"2024-09-21T03:44:27.422093+0000","last_became_peered":"2024-09-21T03:44:27.422093+0000","last_unstale":"2024-09-21T03:44:27.434085+0000","last_undegraded":"2024-09-21T03:44:27.434085+0000","last_fullsized":"2024-09-21T03:44:27.434085+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:11.282438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:11.282438+0000","last_clean_scrub_stamp":"2024-09-21T03:44:11.282438+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:39:42.651024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,9],"acting":[11,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.4","version":"79'9","reported_seq":27,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:45:55.899012+0000","last_change":"2024-09-21T03:44:14.322620+0000","last_active":"2024-09-21T03:45:55.899012+0000","last_peered":"2024-09-21T03:45:55.899012+0000","last_clean":"2024-09-21T03:45:55.899012+0000","last_became_active":"2024-09-21T03:44:14.322214+0000","last_became_peered":"2024-09-21T03:44:14.322214+0000","last_unstale":"2024-09-21T03:45:55.899012+0000","last_undegraded":"2024-09-21T03:45:55.899012+0000","last_fullsized":"2024-09-21T03:45:55.899012+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_clean_scrub_stamp":"2024-09-21T03:44:13.292303+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:53:20.519168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"79'17","reported_seq":40,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:45:56.352616+0000","last_change":"2024-09-21T03:44:12.311302+0000","last_active":"2024-09-21T03:45:56.352616+0000","last_peered":"2024-09-21T03:45:56.352616+0000","last_clean":"2024-09-21T03:45:56.352616+0000","last_became_active":"2024-09-21T03:44:12.310947+0000","last_became_peered":"2024-09-21T03:44:12.310947+0000","last_unstale":"2024-09-21T03:45:56.352616+0000","last_undegraded":"2024-09-21T03:45:56.352616+0000","last_fullsized":"2024-09-21T03:45:56.352616+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:11.282438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:11.282438+0000","last_clean_scrub_stamp":"2024-09-21T03:44:11.282438+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:09:34.308726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":30,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"79'5","reported_seq":23,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:45:55.891717+0000","last_change":"2024-09-21T03:44:14.324675+0000","last_active":"2024-09-21T03:45:55.891717+0000","last_peered":"2024-09-21T03:45:55.891717+0000","last_clean":"2024-09-21T03:45:55.891717+0000","last_became_active":"2024-09-21T03:44:14.324214+0000","last_became_peered":"2024-09-21T03:44:14.324214+0000","last_unstale":"2024-09-21T03:45:55.891717+0000","last_undegraded":"2024-09-21T03:45:55.891717+0000","last_fullsized":"2024-09-21T03:45:55.891717+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_clean_scrub_stamp":"2024-09-21T03:44:13.292303+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:33:51.071794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"77'7","reported_seq":31,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:44:27.613637+0000","last_change":"2024-09-21T03:44:12.322312+0000","last_active":"2024-09-21T03:44:27.613637+0000","last_peered":"2024-09-21T03:44:27.613637+0000","last_clean":"2024-09-21T03:44:27.613637+0000","last_became_active":"2024-09-21T03:44:12.322018+0000","last_became_peered":"2024-09-21T03:44:12.322018+0000","last_unstale":"2024-09-21T03:44:27.613637+0000","last_undegraded":"2024-09-21T03:44:27.613637+0000","last_fullsized":"2024-09-21T03:44:27.613637+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:11.282438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:11.282438+0000","last_clean_scrub_stamp":"2024-09-21T03:44:11.282438+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:11:41.817574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"79'7","reported_seq":25,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:45:55.904952+0000","last_change":"2024-09-21T03:44:14.326682+0000","last_active":"2024-09-21T03:45:55.904952+0000","last_peered":"2024-09-21T03:45:55.904952+0000","last_clean":"2024-09-21T03:45:55.904952+0000","last_became_active":"2024-09-21T03:44:14.326313+0000","last_became_peered":"2024-09-21T03:44:14.326313+0000","last_unstale":"2024-09-21T03:45:55.904952+0000","last_undegraded":"2024-09-21T03:45:55.904952+0000","last_fullsized":"2024-09-21T03:45:55.904952+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_clean_scrub_stamp":"2024-09-21T03:44:13.292303+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:26:44.681245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"77'2","reported_seq":36,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:44:27.436319+0000","last_change":"2024-09-21T03:44:27.436245+0000","last_active":"2024-09-21T03:44:27.436319+0000","last_peered":"2024-09-21T03:44:27.436319+0000","last_clean":"2024-09-21T03:44:27.436319+0000","last_became_active":"2024-09-21T03:44:27.422134+0000","last_became_peered":"2024-09-21T03:44:27.422134+0000","last_unstale":"2024-09-21T03:44:27.436319+0000","last_undegraded":"2024-09-21T03:44:27.436319+0000","last_fullsized":"2024-09-21T03:44:27.436319+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:11.282438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:11.282438+0000","last_clean_scrub_stamp":"2024-09-21T03:44:11.282438+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:50:22.655050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,11,0],"acting":[1,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"79'2","reported_seq":20,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:45:55.854212+0000","last_change":"2024-09-21T03:44:14.316312+0000","last_active":"2024-09-21T03:45:55.854212+0000","last_peered":"2024-09-21T03:45:55.854212+0000","last_clean":"2024-09-21T03:45:55.854212+0000","last_became_active":"2024-09-21T03:44:14.315929+0000","last_became_peered":"2024-09-21T03:44:14.315929+0000","last_unstale":"2024-09-21T03:45:55.854212+0000","last_undegraded":"2024-09-21T03:45:55.854212+0000","last_fullsized":"2024-09-21T03:45:55.854212+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_clean_scrub_stamp":"2024-09-21T03:44:13.292303+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:31:33.071983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"77'7","reported_seq":30,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:44:27.610761+0000","last_change":"2024-09-21T03:44:12.312655+0000","last_active":"2024-09-21T03:44:27.610761+0000","last_peered":"2024-09-21T03:44:27.610761+0000","last_clean":"2024-09-21T03:44:27.610761+0000","last_became_active":"2024-09-21T03:44:12.312196+0000","last_became_peered":"2024-09-21T03:44:12.312196+0000","last_unstale":"2024-09-21T03:44:27.610761+0000","last_undegraded":"2024-09-21T03:44:27.610761+0000","last_fullsized":"2024-09-21T03:44:27.610761+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:11.282438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:11.282438+0000","last_clean_scrub_stamp":"2024-09-21T03:44:11.282438+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:24:32.342461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"79'9","reported_seq":27,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:45:55.897072+0000","last_change":"2024-09-21T03:44:14.321875+0000","last_active":"2024-09-21T03:45:55.897072+0000","last_peered":"2024-09-21T03:45:55.897072+0000","last_clean":"2024-09-21T03:45:55.897072+0000","last_became_active":"2024-09-21T03:44:14.320682+0000","last_became_peered":"2024-09-21T03:44:14.320682+0000","last_unstale":"2024-09-21T03:45:55.897072+0000","last_undegraded":"2024-09-21T03:45:55.897072+0000","last_fullsized":"2024-09-21T03:45:55.897072+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_clean_scrub_stamp":"2024-09-21T03:44:13.292303+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:41:32.435288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"77'3","reported_seq":26,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:44:27.418692+0000","last_change":"2024-09-21T03:44:12.309100+0000","last_active":"2024-09-21T03:44:27.418692+0000","last_peered":"2024-09-21T03:44:27.418692+0000","last_clean":"2024-09-21T03:44:27.418692+0000","last_became_active":"2024-09-21T03:44:12.308671+0000","last_became_peered":"2024-09-21T03:44:12.308671+0000","last_unstale":"2024-09-21T03:44:27.418692+0000","last_undegraded":"2024-09-21T03:44:27.418692+0000","last_fullsized":"2024-09-21T03:44:27.418692+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:11.282438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:11.282438+0000","last_clean_scrub_stamp":"2024-09-21T03:44:11.282438+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:09:19.347980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"79'8","reported_seq":26,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:45:55.890407+0000","last_change":"2024-09-21T03:44:14.320170+0000","last_active":"2024-09-21T03:45:55.890407+0000","last_peered":"2024-09-21T03:45:55.890407+0000","last_clean":"2024-09-21T03:45:55.890407+0000","last_became_active":"2024-09-21T03:44:14.319794+0000","last_became_peered":"2024-09-21T03:44:14.319794+0000","last_unstale":"2024-09-21T03:45:55.890407+0000","last_undegraded":"2024-09-21T03:45:55.890407+0000","last_fullsized":"2024-09-21T03:45:55.890407+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_clean_scrub_stamp":"2024-09-21T03:44:13.292303+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:42:09.684471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"77'4","reported_seq":28,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:44:27.420077+0000","last_change":"2024-09-21T03:44:12.310851+0000","last_active":"2024-09-21T03:44:27.420077+0000","last_peered":"2024-09-21T03:44:27.420077+0000","last_clean":"2024-09-21T03:44:27.420077+0000","last_became_active":"2024-09-21T03:44:12.310302+0000","last_became_peered":"2024-09-21T03:44:12.310302+0000","last_unstale":"2024-09-21T03:44:27.420077+0000","last_undegraded":"2024-09-21T03:44:27.420077+0000","last_fullsized":"2024-09-21T03:44:27.420077+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:11.282438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:11.282438+0000","last_clean_scrub_stamp":"2024-09-21T03:44:11.282438+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:23:46.959121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"79'5","reported_seq":23,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:45:55.904667+0000","last_change":"2024-09-21T03:44:14.324544+0000","last_active":"2024-09-21T03:45:55.904667+0000","last_peered":"2024-09-21T03:45:55.904667+0000","last_clean":"2024-09-21T03:45:55.904667+0000","last_became_active":"2024-09-21T03:44:14.322881+0000","last_became_peered":"2024-09-21T03:44:14.322881+0000","last_unstale":"2024-09-21T03:45:55.904667+0000","last_undegraded":"2024-09-21T03:45:55.904667+0000","last_fullsized":"2024-09-21T03:45:55.904667+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_clean_scrub_stamp":"2024-09-21T03:44:13.292303+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:58:42.760185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"79'15","reported_seq":40,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:45:55.895829+0000","last_change":"2024-09-21T03:44:27.422438+0000","last_active":"2024-09-21T03:45:55.895829+0000","last_peered":"2024-09-21T03:45:55.895829+0000","last_clean":"2024-09-21T03:45:55.895829+0000","last_became_active":"2024-09-21T03:44:27.421935+0000","last_became_peered":"2024-09-21T03:44:27.421935+0000","last_unstale":"2024-09-21T03:45:55.895829+0000","last_undegraded":"2024-09-21T03:45:55.895829+0000","last_fullsized":"2024-09-21T03:45:55.895829+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_clean_scrub_stamp":"2024-09-21T03:44:13.292303+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:20:50.097783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,1],"acting":[5,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"77'5","reported_seq":29,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:44:27.420510+0000","last_change":"2024-09-21T03:44:12.311509+0000","last_active":"2024-09-21T03:44:27.420510+0000","last_peered":"2024-09-21T03:44:27.420510+0000","last_clean":"2024-09-21T03:44:27.420510+0000","last_became_active":"2024-09-21T03:44:12.310880+0000","last_became_peered":"2024-09-21T03:44:12.310880+0000","last_unstale":"2024-09-21T03:44:27.420510+0000","last_undegraded":"2024-09-21T03:44:27.420510+0000","last_fullsized":"2024-09-21T03:44:27.420510+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:11.282438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:11.282438+0000","last_clean_scrub_stamp":"2024-09-21T03:44:11.282438+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:59:33.887234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"79'9","reported_seq":27,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:45:55.905320+0000","last_change":"2024-09-21T03:44:14.330938+0000","last_active":"2024-09-21T03:45:55.905320+0000","last_peered":"2024-09-21T03:45:55.905320+0000","last_clean":"2024-09-21T03:45:55.905320+0000","last_became_active":"2024-09-21T03:44:14.330559+0000","last_became_peered":"2024-09-21T03:44:14.330559+0000","last_unstale":"2024-09-21T03:45:55.905320+0000","last_undegraded":"2024-09-21T03:45:55.905320+0000","last_fullsized":"2024-09-21T03:45:55.905320+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_clean_scrub_stamp":"2024-09-21T03:44:13.292303+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:05:32.535660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"77'12","reported_seq":36,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:44:27.420563+0000","last_change":"2024-09-21T03:44:12.307667+0000","last_active":"2024-09-21T03:44:27.420563+0000","last_peered":"2024-09-21T03:44:27.420563+0000","last_clean":"2024-09-21T03:44:27.420563+0000","last_became_active":"2024-09-21T03:44:12.307074+0000","last_became_peered":"2024-09-21T03:44:12.307074+0000","last_unstale":"2024-09-21T03:44:27.420563+0000","last_undegraded":"2024-09-21T03:44:27.420563+0000","last_fullsized":"2024-09-21T03:44:27.420563+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:11.282438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:11.282438+0000","last_clean_scrub_stamp":"2024-09-21T03:44:11.282438+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:48:41.598375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1f","version":"79'6","reported_seq":24,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:45:55.903388+0000","last_change":"2024-09-21T03:44:14.325487+0000","last_active":"2024-09-21T03:45:55.903388+0000","last_peered":"2024-09-21T03:45:55.903388+0000","last_clean":"2024-09-21T03:45:55.903388+0000","last_became_active":"2024-09-21T03:44:14.325153+0000","last_became_peered":"2024-09-21T03:44:14.325153+0000","last_unstale":"2024-09-21T03:45:55.903388+0000","last_undegraded":"2024-09-21T03:45:55.903388+0000","last_fullsized":"2024-09-21T03:45:55.903388+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_clean_scrub_stamp":"2024-09-21T03:44:13.292303+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:29:17.619130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"77'2","reported_seq":25,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:44:27.611241+0000","last_change":"2024-09-21T03:44:12.307635+0000","last_active":"2024-09-21T03:44:27.611241+0000","last_peered":"2024-09-21T03:44:27.611241+0000","last_clean":"2024-09-21T03:44:27.611241+0000","last_became_active":"2024-09-21T03:44:12.307235+0000","last_became_peered":"2024-09-21T03:44:12.307235+0000","last_unstale":"2024-09-21T03:44:27.611241+0000","last_undegraded":"2024-09-21T03:44:27.611241+0000","last_fullsized":"2024-09-21T03:44:27.611241+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:11.282438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:11.282438+0000","last_clean_scrub_stamp":"2024-09-21T03:44:11.282438+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:18:59.985369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"79'5","reported_seq":23,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:45:55.881464+0000","last_change":"2024-09-21T03:44:14.324660+0000","last_active":"2024-09-21T03:45:55.881464+0000","last_peered":"2024-09-21T03:45:55.881464+0000","last_clean":"2024-09-21T03:45:55.881464+0000","last_became_active":"2024-09-21T03:44:14.324309+0000","last_became_peered":"2024-09-21T03:44:14.324309+0000","last_unstale":"2024-09-21T03:45:55.881464+0000","last_undegraded":"2024-09-21T03:45:55.881464+0000","last_fullsized":"2024-09-21T03:45:55.881464+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_clean_scrub_stamp":"2024-09-21T03:44:13.292303+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:13:07.806721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"79'14","reported_seq":163,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:46:08.761874+0000","last_change":"2024-09-21T03:44:27.623910+0000","last_active":"2024-09-21T03:46:08.761874+0000","last_peered":"2024-09-21T03:46:08.761874+0000","last_clean":"2024-09-21T03:46:08.761874+0000","last_became_active":"2024-09-21T03:44:27.614037+0000","last_became_peered":"2024-09-21T03:44:27.614037+0000","last_unstale":"2024-09-21T03:46:08.761874+0000","last_undegraded":"2024-09-21T03:46:08.761874+0000","last_fullsized":"2024-09-21T03:46:08.761874+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:11.282438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:11.282438+0000","last_clean_scrub_stamp":"2024-09-21T03:44:11.282438+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:03:56.753935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":104,"num_read_kb":104,"num_write":15,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":672,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,4],"acting":[9,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":73176,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":1084,"num_read_kb":622,"num_write":260,"num_write_kb":154,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13,"num_bytes_recovered":1954,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":585728,"data_stored":227183,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":224,"ondisk_log_size":224,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":71,"seq":304942678053,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27888,"kb_used_data":864,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717264,"statfs":{"total":95995035648,"available":95966478336,"internally_reserved":0,"allocated":884736,"data_stored":428272,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":65,"seq":279172874281,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28568,"kb_used_data":1544,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716584,"statfs":{"total":95995035648,"available":95965782016,"internally_reserved":0,"allocated":1581056,"data_stored":1084180,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":60,"seq":257698037806,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27904,"kb_used_data":880,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717248,"statfs":{"total":95995035648,"available":95966461952,"internally_reserved":0,"allocated":901120,"data_stored":428090,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":54,"seq":231928234034,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27884,"kb_used_data":860,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717268,"statfs":{"total":95995035648,"available":95966482432,"internally_reserved":0,"allocated":880640,"data_stored":427684,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":49,"seq":210453397562,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36804,"kb_used_data":1460,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708348,"statfs":{"total":95995035648,"available":95957348352,"internally_reserved":0,"allocated":1495040,"data_stored":1022499,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":36174282},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":43,"seq":184683593790,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27888,"kb_used_data":864,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717264,"statfs":{"total":95995035648,"available":95966478336,"internally_reserved":0,"allocated":884736,"data_stored":428339,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":37,"seq":158913790020,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32656,"kb_used_data":1472,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712496,"statfs":{"total":95995035648,"available":95961595904,"internally_reserved":0,"allocated":1507328,"data_stored":1020805,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":31,"seq":133143986249,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36216,"kb_used_data":872,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708936,"statfs":{"total":95995035648,"available":95957950464,"internally_reserved":0,"allocated":892928,"data_stored":429263,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149774,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32064,"kb_used_data":880,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713088,"statfs":{"total":95995035648,"available":95962202112,"internally_reserved":0,"allocated":901120,"data_stored":427632,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":19,"seq":81604378708,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27964,"kb_used_data":940,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717188,"statfs":{"total":95995035648,"available":95966400512,"internally_reserved":0,"allocated":962560,"data_stored":492761,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542232,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27960,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717192,"statfs":{"total":95995035648,"available":95966404608,"internally_reserved":0,"allocated":958464,"data_stored":495189,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705758,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27908,"kb_used_data":884,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717244,"statfs":{"total":95995035648,"available":95966457856,"internally_reserved":0,"allocated":905216,"data_stored":429146,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2178,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":68221,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":106496,"data_stored":74538,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":2295,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3352,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1254,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":5046,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":716,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":118784,"data_stored":66727,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1304,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T03:46:12.956 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-21T03:46:13.106 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:13.103+0000 7fd492152640 1 -- 172.21.15.39:0/1965333722 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd48c105250 msgr2=0x7fd48c10d790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:46:13.106 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:13.103+0000 7fd492152640 1 --2- 172.21.15.39:0/1965333722 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd48c105250 0x7fd48c10d790 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7fd480009900 tx=0x7fd48002a410 comp rx=0 tx=0).stop 2024-09-21T03:46:13.106 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:13.103+0000 7fd492152640 1 -- 172.21.15.39:0/1965333722 shutdown_connections 2024-09-21T03:46:13.106 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:13.103+0000 7fd492152640 1 --2- 172.21.15.39:0/1965333722 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd48c10dd60 0x7fd48c110250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:13.107 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:13.103+0000 7fd492152640 1 --2- 172.21.15.39:0/1965333722 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd48c105250 0x7fd48c10d790 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:13.107 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:13.103+0000 7fd492152640 1 --2- 172.21.15.39:0/1965333722 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd48c104900 0x7fd48c104d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:13.107 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:13.103+0000 7fd492152640 1 -- 172.21.15.39:0/1965333722 >> 172.21.15.39:0/1965333722 conn(0x7fd48c0fe3b0 msgr2=0x7fd48c100800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:46:13.107 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:13.103+0000 7fd492152640 1 -- 172.21.15.39:0/1965333722 shutdown_connections 2024-09-21T03:46:13.107 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:13.103+0000 7fd492152640 1 -- 172.21.15.39:0/1965333722 wait complete. 2024-09-21T03:46:13.109 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:13.104+0000 7fd492152640 1 Processor -- start 2024-09-21T03:46:13.109 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:13.104+0000 7fd492152640 1 -- start start 2024-09-21T03:46:13.109 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:13.104+0000 7fd492152640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd48c104900 0x7fd48c19a460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:13.109 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:13.104+0000 7fd492152640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd48c105250 0x7fd48c19aa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:13.109 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:13.104+0000 7fd492152640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd48c10dd60 0x7fd48c19fc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:13.109 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:13.104+0000 7fd492152640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fd48c1139c0 con 0x7fd48c10dd60 2024-09-21T03:46:13.109 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:13.104+0000 7fd492152640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fd48c113840 con 0x7fd48c104900 2024-09-21T03:46:13.109 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:13.104+0000 7fd492152640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fd48c113b40 con 0x7fd48c105250 2024-09-21T03:46:13.109 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:13.105+0000 7fd48bfff640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd48c10dd60 0x7fd48c19fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:46:13.110 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:13.105+0000 7fd48bfff640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd48c10dd60 0x7fd48c19fc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:43202/0 (socket says 172.21.15.39:43202) 2024-09-21T03:46:13.110 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:13.105+0000 7fd48bfff640 1 -- 172.21.15.39:0/1500966372 learned_addr learned my addr 172.21.15.39:0/1500966372 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:46:13.110 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:13.105+0000 7fd48affd640 1 --2- 172.21.15.39:0/1500966372 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd48c105250 0x7fd48c19aa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:46:13.110 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:13.105+0000 7fd48b7fe640 1 --2- 172.21.15.39:0/1500966372 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd48c104900 0x7fd48c19a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:46:13.110 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:13.105+0000 7fd48bfff640 1 -- 172.21.15.39:0/1500966372 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd48c105250 msgr2=0x7fd48c19aa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:46:13.110 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:13.105+0000 7fd48bfff640 1 --2- 172.21.15.39:0/1500966372 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd48c105250 0x7fd48c19aa00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:13.110 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:13.105+0000 7fd48bfff640 1 -- 172.21.15.39:0/1500966372 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd48c104900 msgr2=0x7fd48c19a460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:46:13.110 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:13.105+0000 7fd48bfff640 1 --2- 172.21.15.39:0/1500966372 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd48c104900 0x7fd48c19a460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:13.110 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:13.105+0000 7fd48bfff640 1 -- 172.21.15.39:0/1500966372 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd48c1a0390 con 0x7fd48c10dd60 2024-09-21T03:46:13.110 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:13.105+0000 7fd48bfff640 1 --2- 172.21.15.39:0/1500966372 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd48c10dd60 0x7fd48c19fc90 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7fd484009fd0 tx=0x7fd48400b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:46:13.110 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:13.105+0000 7fd488ff9640 1 -- 172.21.15.39:0/1500966372 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd484017070 con 0x7fd48c10dd60 2024-09-21T03:46:13.110 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:13.105+0000 7fd488ff9640 1 -- 172.21.15.39:0/1500966372 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd484003e20 con 0x7fd48c10dd60 2024-09-21T03:46:13.110 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:13.105+0000 7fd492152640 1 -- 172.21.15.39:0/1500966372 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd48c1a0680 con 0x7fd48c10dd60 2024-09-21T03:46:13.111 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:13.105+0000 7fd488ff9640 1 -- 172.21.15.39:0/1500966372 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd484010460 con 0x7fd48c10dd60 2024-09-21T03:46:13.111 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:13.105+0000 7fd492152640 1 -- 172.21.15.39:0/1500966372 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd48c1a6e10 con 0x7fd48c10dd60 2024-09-21T03:46:13.111 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:13.106+0000 7fd492152640 1 -- 172.21.15.39:0/1500966372 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd48c104d10 con 0x7fd48c10dd60 2024-09-21T03:46:13.111 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:13.108+0000 7fd488ff9640 1 -- 172.21.15.39:0/1500966372 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 24) ==== 154580+0+0 (secure 0 0 0) 0x7fd484003970 con 0x7fd48c10dd60 2024-09-21T03:46:13.112 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:13.109+0000 7fd488ff9640 1 --2- 172.21.15.39:0/1500966372 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7fd4740b65f0 0x7fd4740b8b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:13.112 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:13.110+0000 7fd48b7fe640 1 --2- 172.21.15.39:0/1500966372 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7fd4740b65f0 0x7fd4740b8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:46:13.112 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:13.110+0000 7fd488ff9640 1 -- 172.21.15.39:0/1500966372 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8484+0+0 (secure 0 0 0) 0x7fd4840e72b0 con 0x7fd48c10dd60 2024-09-21T03:46:13.113 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:13.110+0000 7fd48b7fe640 1 --2- 172.21.15.39:0/1500966372 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7fd4740b65f0 0x7fd4740b8b00 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fd47c009bd0 tx=0x7fd47c005f90 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:46:13.115 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:13.113+0000 7fd488ff9640 1 -- 172.21.15.39:0/1500966372 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fd4840b1cc0 con 0x7fd48c10dd60 2024-09-21T03:46:13.300 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:13.298+0000 7fd492152640 1 -- 172.21.15.39:0/1500966372 --> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fd48c107fe0 con 0x7fd4740b65f0 2024-09-21T03:46:13.305 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:13.302+0000 7fd488ff9640 1 -- 172.21.15.39:0/1500966372 <== mgr.14223 v2:172.21.15.39:6800/1696381982 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258883 (secure 0 0 0) 0x7fd48c107fe0 con 0x7fd4740b65f0 2024-09-21T03:46:13.308 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T03:46:13.309 INFO:teuthology.orchestra.run.smithi039.stderr:dumped all 2024-09-21T03:46:13.310 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:13.305+0000 7fd492152640 1 -- 172.21.15.39:0/1500966372 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7fd4740b65f0 msgr2=0x7fd4740b8b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:46:13.310 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:13.305+0000 7fd492152640 1 --2- 172.21.15.39:0/1500966372 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7fd4740b65f0 0x7fd4740b8b00 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fd47c009bd0 tx=0x7fd47c005f90 comp rx=0 tx=0).stop 2024-09-21T03:46:13.310 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:13.305+0000 7fd492152640 1 -- 172.21.15.39:0/1500966372 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd48c10dd60 msgr2=0x7fd48c19fc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:46:13.310 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:13.305+0000 7fd492152640 1 --2- 172.21.15.39:0/1500966372 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd48c10dd60 0x7fd48c19fc90 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7fd484009fd0 tx=0x7fd48400b260 comp rx=0 tx=0).stop 2024-09-21T03:46:13.311 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:13.305+0000 7fd492152640 1 -- 172.21.15.39:0/1500966372 shutdown_connections 2024-09-21T03:46:13.311 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:13.305+0000 7fd492152640 1 --2- 172.21.15.39:0/1500966372 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7fd4740b65f0 0x7fd4740b8b00 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:13.311 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:13.305+0000 7fd492152640 1 --2- 172.21.15.39:0/1500966372 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd48c10dd60 0x7fd48c19fc90 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:13.311 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:13.305+0000 7fd492152640 1 --2- 172.21.15.39:0/1500966372 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd48c105250 0x7fd48c19aa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:13.311 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:13.305+0000 7fd492152640 1 --2- 172.21.15.39:0/1500966372 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd48c104900 0x7fd48c19a460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:13.311 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:13.305+0000 7fd492152640 1 -- 172.21.15.39:0/1500966372 >> 172.21.15.39:0/1500966372 conn(0x7fd48c0fe3b0 msgr2=0x7fd48c1078d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:46:13.311 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:13.306+0000 7fd492152640 1 -- 172.21.15.39:0/1500966372 shutdown_connections 2024-09-21T03:46:13.311 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:13.306+0000 7fd492152640 1 -- 172.21.15.39:0/1500966372 wait complete. 2024-09-21T03:46:13.318 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:46:13 smithi039 ceph-mon[31211]: pgmap v330: 97 pgs: 97 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 511 B/s wr, 10 op/s 2024-09-21T03:46:13.320 INFO:teuthology.orchestra.run.smithi039.stdout:{"pg_ready":true,"pg_map":{"version":330,"stamp":"2024-09-21T03:46:11.529574+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":663778,"num_objects":102,"num_object_clones":0,"num_object_copies":306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":1165,"num_read_kb":691,"num_write":385,"num_write_kb":2306,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":23,"num_bytes_recovered":2953794,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":818,"ondisk_log_size":818,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":361704,"kb_used_data":12456,"kb_used_omap":18,"kb_used_meta":349037,"kb_avail":1124580120,"statfs":{"total":1151940427776,"available":1151570042880,"internally_reserved":0,"allocated":12754944,"data_stored":7113860,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19143,"internal_metadata":357414201},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":1148,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":114,"num_read_kb":59,"num_write":9,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003767"},"pg_stats":[{"pgid":"3.3f","version":"79'8","reported_seq":26,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:45:55.900065+0000","last_change":"2024-09-21T03:44:14.327553+0000","last_active":"2024-09-21T03:45:55.900065+0000","last_peered":"2024-09-21T03:45:55.900065+0000","last_clean":"2024-09-21T03:45:55.900065+0000","last_became_active":"2024-09-21T03:44:14.327213+0000","last_became_peered":"2024-09-21T03:44:14.327213+0000","last_unstale":"2024-09-21T03:45:55.900065+0000","last_undegraded":"2024-09-21T03:45:55.900065+0000","last_fullsized":"2024-09-21T03:45:55.900065+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_clean_scrub_stamp":"2024-09-21T03:44:13.292303+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:40:56.550897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"79'4","reported_seq":22,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:45:55.902384+0000","last_change":"2024-09-21T03:44:14.321427+0000","last_active":"2024-09-21T03:45:55.902384+0000","last_peered":"2024-09-21T03:45:55.902384+0000","last_clean":"2024-09-21T03:45:55.902384+0000","last_became_active":"2024-09-21T03:44:14.321026+0000","last_became_peered":"2024-09-21T03:44:14.321026+0000","last_unstale":"2024-09-21T03:45:55.902384+0000","last_undegraded":"2024-09-21T03:45:55.902384+0000","last_fullsized":"2024-09-21T03:45:55.902384+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_clean_scrub_stamp":"2024-09-21T03:44:13.292303+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:18:00.889714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"79'7","reported_seq":25,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:45:55.906660+0000","last_change":"2024-09-21T03:44:14.324908+0000","last_active":"2024-09-21T03:45:55.906660+0000","last_peered":"2024-09-21T03:45:55.906660+0000","last_clean":"2024-09-21T03:45:55.906660+0000","last_became_active":"2024-09-21T03:44:14.324533+0000","last_became_peered":"2024-09-21T03:44:14.324533+0000","last_unstale":"2024-09-21T03:45:55.906660+0000","last_undegraded":"2024-09-21T03:45:55.906660+0000","last_fullsized":"2024-09-21T03:45:55.906660+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_clean_scrub_stamp":"2024-09-21T03:44:13.292303+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:44:25.235442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"79'8","reported_seq":26,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:45:55.885244+0000","last_change":"2024-09-21T03:44:14.326758+0000","last_active":"2024-09-21T03:45:55.885244+0000","last_peered":"2024-09-21T03:45:55.885244+0000","last_clean":"2024-09-21T03:45:55.885244+0000","last_became_active":"2024-09-21T03:44:14.326410+0000","last_became_peered":"2024-09-21T03:44:14.326410+0000","last_unstale":"2024-09-21T03:45:55.885244+0000","last_undegraded":"2024-09-21T03:45:55.885244+0000","last_fullsized":"2024-09-21T03:45:55.885244+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_clean_scrub_stamp":"2024-09-21T03:44:13.292303+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:13:22.716940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"79'4","reported_seq":22,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:45:55.874578+0000","last_change":"2024-09-21T03:44:14.318875+0000","last_active":"2024-09-21T03:45:55.874578+0000","last_peered":"2024-09-21T03:45:55.874578+0000","last_clean":"2024-09-21T03:45:55.874578+0000","last_became_active":"2024-09-21T03:44:14.318425+0000","last_became_peered":"2024-09-21T03:44:14.318425+0000","last_unstale":"2024-09-21T03:45:55.874578+0000","last_undegraded":"2024-09-21T03:45:55.874578+0000","last_fullsized":"2024-09-21T03:45:55.874578+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_clean_scrub_stamp":"2024-09-21T03:44:13.292303+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:45:39.916587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"79'6","reported_seq":24,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:45:55.892089+0000","last_change":"2024-09-21T03:44:14.325877+0000","last_active":"2024-09-21T03:45:55.892089+0000","last_peered":"2024-09-21T03:45:55.892089+0000","last_clean":"2024-09-21T03:45:55.892089+0000","last_became_active":"2024-09-21T03:44:14.325451+0000","last_became_peered":"2024-09-21T03:44:14.325451+0000","last_unstale":"2024-09-21T03:45:55.892089+0000","last_undegraded":"2024-09-21T03:45:55.892089+0000","last_fullsized":"2024-09-21T03:45:55.892089+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_clean_scrub_stamp":"2024-09-21T03:44:13.292303+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:30:42.427054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"79'5","reported_seq":23,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:45:55.888426+0000","last_change":"2024-09-21T03:44:14.323192+0000","last_active":"2024-09-21T03:45:55.888426+0000","last_peered":"2024-09-21T03:45:55.888426+0000","last_clean":"2024-09-21T03:45:55.888426+0000","last_became_active":"2024-09-21T03:44:14.322487+0000","last_became_peered":"2024-09-21T03:44:14.322487+0000","last_unstale":"2024-09-21T03:45:55.888426+0000","last_undegraded":"2024-09-21T03:45:55.888426+0000","last_fullsized":"2024-09-21T03:45:55.888426+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_clean_scrub_stamp":"2024-09-21T03:44:13.292303+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:10:48.770536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"79'9","reported_seq":21,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:45:55.892165+0000","last_change":"2024-09-21T03:44:27.612681+0000","last_active":"2024-09-21T03:45:55.892165+0000","last_peered":"2024-09-21T03:45:55.892165+0000","last_clean":"2024-09-21T03:45:55.892165+0000","last_became_active":"2024-09-21T03:44:27.612200+0000","last_became_peered":"2024-09-21T03:44:27.612200+0000","last_unstale":"2024-09-21T03:45:55.892165+0000","last_undegraded":"2024-09-21T03:45:55.892165+0000","last_fullsized":"2024-09-21T03:45:55.892165+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_clean_scrub_stamp":"2024-09-21T03:44:13.292303+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:44:58.848571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,5,10],"acting":[9,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.37","version":"79'11","reported_seq":29,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:45:55.900520+0000","last_change":"2024-09-21T03:44:14.325820+0000","last_active":"2024-09-21T03:45:55.900520+0000","last_peered":"2024-09-21T03:45:55.900520+0000","last_clean":"2024-09-21T03:45:55.900520+0000","last_became_active":"2024-09-21T03:44:14.325070+0000","last_became_peered":"2024-09-21T03:44:14.325070+0000","last_unstale":"2024-09-21T03:45:55.900520+0000","last_undegraded":"2024-09-21T03:45:55.900520+0000","last_fullsized":"2024-09-21T03:45:55.900520+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_clean_scrub_stamp":"2024-09-21T03:44:13.292303+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:54:01.072623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"79'16","reported_seq":34,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:45:55.894398+0000","last_change":"2024-09-21T03:44:14.323746+0000","last_active":"2024-09-21T03:45:55.894398+0000","last_peered":"2024-09-21T03:45:55.894398+0000","last_clean":"2024-09-21T03:45:55.894398+0000","last_became_active":"2024-09-21T03:44:14.323346+0000","last_became_peered":"2024-09-21T03:44:14.323346+0000","last_unstale":"2024-09-21T03:45:55.894398+0000","last_undegraded":"2024-09-21T03:45:55.894398+0000","last_fullsized":"2024-09-21T03:45:55.894398+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_clean_scrub_stamp":"2024-09-21T03:44:13.292303+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:17:44.603801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"79'6","reported_seq":24,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:45:55.896278+0000","last_change":"2024-09-21T03:44:14.328733+0000","last_active":"2024-09-21T03:45:55.896278+0000","last_peered":"2024-09-21T03:45:55.896278+0000","last_clean":"2024-09-21T03:45:55.896278+0000","last_became_active":"2024-09-21T03:44:14.328378+0000","last_became_peered":"2024-09-21T03:44:14.328378+0000","last_unstale":"2024-09-21T03:45:55.896278+0000","last_undegraded":"2024-09-21T03:45:55.896278+0000","last_fullsized":"2024-09-21T03:45:55.896278+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_clean_scrub_stamp":"2024-09-21T03:44:13.292303+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:23:38.497413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"79'7","reported_seq":25,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:45:55.895287+0000","last_change":"2024-09-21T03:44:14.328505+0000","last_active":"2024-09-21T03:45:55.895287+0000","last_peered":"2024-09-21T03:45:55.895287+0000","last_clean":"2024-09-21T03:45:55.895287+0000","last_became_active":"2024-09-21T03:44:14.328175+0000","last_became_peered":"2024-09-21T03:44:14.328175+0000","last_unstale":"2024-09-21T03:45:55.895287+0000","last_undegraded":"2024-09-21T03:45:55.895287+0000","last_fullsized":"2024-09-21T03:45:55.895287+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_clean_scrub_stamp":"2024-09-21T03:44:13.292303+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:45:56.628962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"79'10","reported_seq":28,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:45:55.903680+0000","last_change":"2024-09-21T03:44:14.314519+0000","last_active":"2024-09-21T03:45:55.903680+0000","last_peered":"2024-09-21T03:45:55.903680+0000","last_clean":"2024-09-21T03:45:55.903680+0000","last_became_active":"2024-09-21T03:44:14.314145+0000","last_became_peered":"2024-09-21T03:44:14.314145+0000","last_unstale":"2024-09-21T03:45:55.903680+0000","last_undegraded":"2024-09-21T03:45:55.903680+0000","last_fullsized":"2024-09-21T03:45:55.903680+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_clean_scrub_stamp":"2024-09-21T03:44:13.292303+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:45:48.494667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"79'7","reported_seq":25,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:45:55.902926+0000","last_change":"2024-09-21T03:44:14.312195+0000","last_active":"2024-09-21T03:45:55.902926+0000","last_peered":"2024-09-21T03:45:55.902926+0000","last_clean":"2024-09-21T03:45:55.902926+0000","last_became_active":"2024-09-21T03:44:14.311861+0000","last_became_peered":"2024-09-21T03:44:14.311861+0000","last_unstale":"2024-09-21T03:45:55.902926+0000","last_undegraded":"2024-09-21T03:45:55.902926+0000","last_fullsized":"2024-09-21T03:45:55.902926+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_clean_scrub_stamp":"2024-09-21T03:44:13.292303+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:17:05.025898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"79'7","reported_seq":25,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:45:55.905677+0000","last_change":"2024-09-21T03:44:14.323532+0000","last_active":"2024-09-21T03:45:55.905677+0000","last_peered":"2024-09-21T03:45:55.905677+0000","last_clean":"2024-09-21T03:45:55.905677+0000","last_became_active":"2024-09-21T03:44:14.323171+0000","last_became_peered":"2024-09-21T03:44:14.323171+0000","last_unstale":"2024-09-21T03:45:55.905677+0000","last_undegraded":"2024-09-21T03:45:55.905677+0000","last_fullsized":"2024-09-21T03:45:55.905677+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_clean_scrub_stamp":"2024-09-21T03:44:13.292303+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:23:28.999129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"79'9","reported_seq":27,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:45:55.903498+0000","last_change":"2024-09-21T03:44:14.324998+0000","last_active":"2024-09-21T03:45:55.903498+0000","last_peered":"2024-09-21T03:45:55.903498+0000","last_clean":"2024-09-21T03:45:55.903498+0000","last_became_active":"2024-09-21T03:44:14.324019+0000","last_became_peered":"2024-09-21T03:44:14.324019+0000","last_unstale":"2024-09-21T03:45:55.903498+0000","last_undegraded":"2024-09-21T03:45:55.903498+0000","last_fullsized":"2024-09-21T03:45:55.903498+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_clean_scrub_stamp":"2024-09-21T03:44:13.292303+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:49:12.965299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"79'8","reported_seq":26,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:45:55.900363+0000","last_change":"2024-09-21T03:44:14.325603+0000","last_active":"2024-09-21T03:45:55.900363+0000","last_peered":"2024-09-21T03:45:55.900363+0000","last_clean":"2024-09-21T03:45:55.900363+0000","last_became_active":"2024-09-21T03:44:14.325208+0000","last_became_peered":"2024-09-21T03:44:14.325208+0000","last_unstale":"2024-09-21T03:45:55.900363+0000","last_undegraded":"2024-09-21T03:45:55.900363+0000","last_fullsized":"2024-09-21T03:45:55.900363+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_clean_scrub_stamp":"2024-09-21T03:44:13.292303+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:11:33.794242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"79'10","reported_seq":28,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:45:55.891232+0000","last_change":"2024-09-21T03:44:14.317545+0000","last_active":"2024-09-21T03:45:55.891232+0000","last_peered":"2024-09-21T03:45:55.891232+0000","last_clean":"2024-09-21T03:45:55.891232+0000","last_became_active":"2024-09-21T03:44:14.317069+0000","last_became_peered":"2024-09-21T03:44:14.317069+0000","last_unstale":"2024-09-21T03:45:55.891232+0000","last_undegraded":"2024-09-21T03:45:55.891232+0000","last_fullsized":"2024-09-21T03:45:55.891232+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_clean_scrub_stamp":"2024-09-21T03:44:13.292303+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:16:11.734887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"79'7","reported_seq":25,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:45:55.903043+0000","last_change":"2024-09-21T03:44:14.322055+0000","last_active":"2024-09-21T03:45:55.903043+0000","last_peered":"2024-09-21T03:45:55.903043+0000","last_clean":"2024-09-21T03:45:55.903043+0000","last_became_active":"2024-09-21T03:44:14.321652+0000","last_became_peered":"2024-09-21T03:44:14.321652+0000","last_unstale":"2024-09-21T03:45:55.903043+0000","last_undegraded":"2024-09-21T03:45:55.903043+0000","last_fullsized":"2024-09-21T03:45:55.903043+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_clean_scrub_stamp":"2024-09-21T03:44:13.292303+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:45:14.285844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2c","version":"79'8","reported_seq":33,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:45:55.896619+0000","last_change":"2024-09-21T03:44:27.423175+0000","last_active":"2024-09-21T03:45:55.896619+0000","last_peered":"2024-09-21T03:45:55.896619+0000","last_clean":"2024-09-21T03:45:55.896619+0000","last_became_active":"2024-09-21T03:44:27.422733+0000","last_became_peered":"2024-09-21T03:44:27.422733+0000","last_unstale":"2024-09-21T03:45:55.896619+0000","last_undegraded":"2024-09-21T03:45:55.896619+0000","last_fullsized":"2024-09-21T03:45:55.896619+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_clean_scrub_stamp":"2024-09-21T03:44:13.292303+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:46:17.874557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,8],"acting":[11,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2b","version":"79'11","reported_seq":29,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:45:55.903246+0000","last_change":"2024-09-21T03:44:14.325829+0000","last_active":"2024-09-21T03:45:55.903246+0000","last_peered":"2024-09-21T03:45:55.903246+0000","last_clean":"2024-09-21T03:45:55.903246+0000","last_became_active":"2024-09-21T03:44:14.325421+0000","last_became_peered":"2024-09-21T03:44:14.325421+0000","last_unstale":"2024-09-21T03:45:55.903246+0000","last_undegraded":"2024-09-21T03:45:55.903246+0000","last_fullsized":"2024-09-21T03:45:55.903246+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_clean_scrub_stamp":"2024-09-21T03:44:13.292303+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:49:00.240771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2a","version":"79'10","reported_seq":28,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:45:55.907357+0000","last_change":"2024-09-21T03:44:14.325255+0000","last_active":"2024-09-21T03:45:55.907357+0000","last_peered":"2024-09-21T03:45:55.907357+0000","last_clean":"2024-09-21T03:45:55.907357+0000","last_became_active":"2024-09-21T03:44:14.324617+0000","last_became_peered":"2024-09-21T03:44:14.324617+0000","last_unstale":"2024-09-21T03:45:55.907357+0000","last_undegraded":"2024-09-21T03:45:55.907357+0000","last_fullsized":"2024-09-21T03:45:55.907357+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_clean_scrub_stamp":"2024-09-21T03:44:13.292303+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:05:03.268522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"79'8","reported_seq":26,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:45:55.885845+0000","last_change":"2024-09-21T03:44:14.328053+0000","last_active":"2024-09-21T03:45:55.885845+0000","last_peered":"2024-09-21T03:45:55.885845+0000","last_clean":"2024-09-21T03:45:55.885845+0000","last_became_active":"2024-09-21T03:44:14.327735+0000","last_became_peered":"2024-09-21T03:44:14.327735+0000","last_unstale":"2024-09-21T03:45:55.885845+0000","last_undegraded":"2024-09-21T03:45:55.885845+0000","last_fullsized":"2024-09-21T03:45:55.885845+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_clean_scrub_stamp":"2024-09-21T03:44:13.292303+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:39:48.395784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"79'6","reported_seq":24,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:45:55.888696+0000","last_change":"2024-09-21T03:44:14.321181+0000","last_active":"2024-09-21T03:45:55.888696+0000","last_peered":"2024-09-21T03:45:55.888696+0000","last_clean":"2024-09-21T03:45:55.888696+0000","last_became_active":"2024-09-21T03:44:14.320772+0000","last_became_peered":"2024-09-21T03:44:14.320772+0000","last_unstale":"2024-09-21T03:45:55.888696+0000","last_undegraded":"2024-09-21T03:45:55.888696+0000","last_fullsized":"2024-09-21T03:45:55.888696+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_clean_scrub_stamp":"2024-09-21T03:44:13.292303+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:49:26.317404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"79'2","reported_seq":20,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:45:55.884461+0000","last_change":"2024-09-21T03:44:14.322643+0000","last_active":"2024-09-21T03:45:55.884461+0000","last_peered":"2024-09-21T03:45:55.884461+0000","last_clean":"2024-09-21T03:45:55.884461+0000","last_became_active":"2024-09-21T03:44:14.322254+0000","last_became_peered":"2024-09-21T03:44:14.322254+0000","last_unstale":"2024-09-21T03:45:55.884461+0000","last_undegraded":"2024-09-21T03:45:55.884461+0000","last_fullsized":"2024-09-21T03:45:55.884461+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_clean_scrub_stamp":"2024-09-21T03:44:13.292303+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:31:47.147845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"79'15","reported_seq":33,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:45:55.907955+0000","last_change":"2024-09-21T03:44:14.323090+0000","last_active":"2024-09-21T03:45:55.907955+0000","last_peered":"2024-09-21T03:45:55.907955+0000","last_clean":"2024-09-21T03:45:55.907955+0000","last_became_active":"2024-09-21T03:44:14.322669+0000","last_became_peered":"2024-09-21T03:44:14.322669+0000","last_unstale":"2024-09-21T03:45:55.907955+0000","last_undegraded":"2024-09-21T03:45:55.907955+0000","last_fullsized":"2024-09-21T03:45:55.907955+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_clean_scrub_stamp":"2024-09-21T03:44:13.292303+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:08:56.852422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"79'6","reported_seq":24,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:45:55.896312+0000","last_change":"2024-09-21T03:44:14.318146+0000","last_active":"2024-09-21T03:45:55.896312+0000","last_peered":"2024-09-21T03:45:55.896312+0000","last_clean":"2024-09-21T03:45:55.896312+0000","last_became_active":"2024-09-21T03:44:14.317761+0000","last_became_peered":"2024-09-21T03:44:14.317761+0000","last_unstale":"2024-09-21T03:45:55.896312+0000","last_undegraded":"2024-09-21T03:45:55.896312+0000","last_fullsized":"2024-09-21T03:45:55.896312+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_clean_scrub_stamp":"2024-09-21T03:44:13.292303+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:07:31.530728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.24","version":"79'4","reported_seq":22,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:45:55.902467+0000","last_change":"2024-09-21T03:44:14.316832+0000","last_active":"2024-09-21T03:45:55.902467+0000","last_peered":"2024-09-21T03:45:55.902467+0000","last_clean":"2024-09-21T03:45:55.902467+0000","last_became_active":"2024-09-21T03:44:14.316471+0000","last_became_peered":"2024-09-21T03:44:14.316471+0000","last_unstale":"2024-09-21T03:45:55.902467+0000","last_undegraded":"2024-09-21T03:45:55.902467+0000","last_fullsized":"2024-09-21T03:45:55.902467+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_clean_scrub_stamp":"2024-09-21T03:44:13.292303+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:21:23.848778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"79'7","reported_seq":25,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:45:55.905316+0000","last_change":"2024-09-21T03:44:14.322277+0000","last_active":"2024-09-21T03:45:55.905316+0000","last_peered":"2024-09-21T03:45:55.905316+0000","last_clean":"2024-09-21T03:45:55.905316+0000","last_became_active":"2024-09-21T03:44:14.321952+0000","last_became_peered":"2024-09-21T03:44:14.321952+0000","last_unstale":"2024-09-21T03:45:55.905316+0000","last_undegraded":"2024-09-21T03:45:55.905316+0000","last_fullsized":"2024-09-21T03:45:55.905316+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_clean_scrub_stamp":"2024-09-21T03:44:13.292303+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:43:55.169380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"79'8","reported_seq":26,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:45:55.886351+0000","last_change":"2024-09-21T03:44:14.322626+0000","last_active":"2024-09-21T03:45:55.886351+0000","last_peered":"2024-09-21T03:45:55.886351+0000","last_clean":"2024-09-21T03:45:55.886351+0000","last_became_active":"2024-09-21T03:44:14.321559+0000","last_became_peered":"2024-09-21T03:44:14.321559+0000","last_unstale":"2024-09-21T03:45:55.886351+0000","last_undegraded":"2024-09-21T03:45:55.886351+0000","last_fullsized":"2024-09-21T03:45:55.886351+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_clean_scrub_stamp":"2024-09-21T03:44:13.292303+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:12:23.736500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"79'12","reported_seq":30,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:45:55.886765+0000","last_change":"2024-09-21T03:44:14.318057+0000","last_active":"2024-09-21T03:45:55.886765+0000","last_peered":"2024-09-21T03:45:55.886765+0000","last_clean":"2024-09-21T03:45:55.886765+0000","last_became_active":"2024-09-21T03:44:14.317653+0000","last_became_peered":"2024-09-21T03:44:14.317653+0000","last_unstale":"2024-09-21T03:45:55.886765+0000","last_undegraded":"2024-09-21T03:45:55.886765+0000","last_fullsized":"2024-09-21T03:45:55.886765+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_clean_scrub_stamp":"2024-09-21T03:44:13.292303+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:23:17.396599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"79'9","reported_seq":27,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:45:55.895204+0000","last_change":"2024-09-21T03:44:14.322331+0000","last_active":"2024-09-21T03:45:55.895204+0000","last_peered":"2024-09-21T03:45:55.895204+0000","last_clean":"2024-09-21T03:45:55.895204+0000","last_became_active":"2024-09-21T03:44:14.321157+0000","last_became_peered":"2024-09-21T03:44:14.321157+0000","last_unstale":"2024-09-21T03:45:55.895204+0000","last_undegraded":"2024-09-21T03:45:55.895204+0000","last_fullsized":"2024-09-21T03:45:55.895204+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_clean_scrub_stamp":"2024-09-21T03:44:13.292303+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:08:27.953449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1a","version":"79'7","reported_seq":32,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:45:55.890546+0000","last_change":"2024-09-21T03:44:27.423081+0000","last_active":"2024-09-21T03:45:55.890546+0000","last_peered":"2024-09-21T03:45:55.890546+0000","last_clean":"2024-09-21T03:45:55.890546+0000","last_became_active":"2024-09-21T03:44:27.422650+0000","last_became_peered":"2024-09-21T03:44:27.422650+0000","last_unstale":"2024-09-21T03:45:55.890546+0000","last_undegraded":"2024-09-21T03:45:55.890546+0000","last_fullsized":"2024-09-21T03:45:55.890546+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_clean_scrub_stamp":"2024-09-21T03:44:13.292303+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:32:44.399954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,7],"acting":[8,9,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1b","version":"79'7","reported_seq":139,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:46:07.857454+0000","last_change":"2024-09-21T03:44:12.307464+0000","last_active":"2024-09-21T03:46:07.857454+0000","last_peered":"2024-09-21T03:46:07.857454+0000","last_clean":"2024-09-21T03:46:07.857454+0000","last_became_active":"2024-09-21T03:44:12.307077+0000","last_became_peered":"2024-09-21T03:44:12.307077+0000","last_unstale":"2024-09-21T03:46:07.857454+0000","last_undegraded":"2024-09-21T03:46:07.857454+0000","last_fullsized":"2024-09-21T03:46:07.857454+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:11.282438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:11.282438+0000","last_clean_scrub_stamp":"2024-09-21T03:44:11.282438+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:50:40.344659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1154,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":109,"num_read_kb":4,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"79'8","reported_seq":26,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:45:55.891478+0000","last_change":"2024-09-21T03:44:14.319072+0000","last_active":"2024-09-21T03:45:55.891478+0000","last_peered":"2024-09-21T03:45:55.891478+0000","last_clean":"2024-09-21T03:45:55.891478+0000","last_became_active":"2024-09-21T03:44:14.318712+0000","last_became_peered":"2024-09-21T03:44:14.318712+0000","last_unstale":"2024-09-21T03:45:55.891478+0000","last_undegraded":"2024-09-21T03:45:55.891478+0000","last_fullsized":"2024-09-21T03:45:55.891478+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_clean_scrub_stamp":"2024-09-21T03:44:13.292303+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:30:35.618110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1a","version":"77'7","reported_seq":30,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:44:27.611763+0000","last_change":"2024-09-21T03:44:12.310146+0000","last_active":"2024-09-21T03:44:27.611763+0000","last_peered":"2024-09-21T03:44:27.611763+0000","last_clean":"2024-09-21T03:44:27.611763+0000","last_became_active":"2024-09-21T03:44:12.309817+0000","last_became_peered":"2024-09-21T03:44:12.309817+0000","last_unstale":"2024-09-21T03:44:27.611763+0000","last_undegraded":"2024-09-21T03:44:27.611763+0000","last_fullsized":"2024-09-21T03:44:27.611763+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:11.282438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:11.282438+0000","last_clean_scrub_stamp":"2024-09-21T03:44:11.282438+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:46:51.994488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.19","version":"77'3","reported_seq":26,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:44:27.410211+0000","last_change":"2024-09-21T03:44:12.308995+0000","last_active":"2024-09-21T03:44:27.410211+0000","last_peered":"2024-09-21T03:44:27.410211+0000","last_clean":"2024-09-21T03:44:27.410211+0000","last_became_active":"2024-09-21T03:44:12.308058+0000","last_became_peered":"2024-09-21T03:44:12.308058+0000","last_unstale":"2024-09-21T03:44:27.410211+0000","last_undegraded":"2024-09-21T03:44:27.410211+0000","last_fullsized":"2024-09-21T03:44:27.410211+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:11.282438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:11.282438+0000","last_clean_scrub_stamp":"2024-09-21T03:44:11.282438+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:48:39.384985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"79'8","reported_seq":26,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:45:55.886620+0000","last_change":"2024-09-21T03:44:14.317878+0000","last_active":"2024-09-21T03:45:55.886620+0000","last_peered":"2024-09-21T03:45:55.886620+0000","last_clean":"2024-09-21T03:45:55.886620+0000","last_became_active":"2024-09-21T03:44:14.317521+0000","last_became_peered":"2024-09-21T03:44:14.317521+0000","last_unstale":"2024-09-21T03:45:55.886620+0000","last_undegraded":"2024-09-21T03:45:55.886620+0000","last_fullsized":"2024-09-21T03:45:55.886620+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_clean_scrub_stamp":"2024-09-21T03:44:13.292303+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:08:00.705210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"77'3","reported_seq":27,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:44:27.414041+0000","last_change":"2024-09-21T03:44:12.308777+0000","last_active":"2024-09-21T03:44:27.414041+0000","last_peered":"2024-09-21T03:44:27.414041+0000","last_clean":"2024-09-21T03:44:27.414041+0000","last_became_active":"2024-09-21T03:44:12.308171+0000","last_became_peered":"2024-09-21T03:44:12.308171+0000","last_unstale":"2024-09-21T03:44:27.414041+0000","last_undegraded":"2024-09-21T03:44:27.414041+0000","last_fullsized":"2024-09-21T03:44:27.414041+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:11.282438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:11.282438+0000","last_clean_scrub_stamp":"2024-09-21T03:44:11.282438+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:41:21.344491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"79'11","reported_seq":29,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:45:55.892137+0000","last_change":"2024-09-21T03:44:14.315782+0000","last_active":"2024-09-21T03:45:55.892137+0000","last_peered":"2024-09-21T03:45:55.892137+0000","last_clean":"2024-09-21T03:45:55.892137+0000","last_became_active":"2024-09-21T03:44:14.315314+0000","last_became_peered":"2024-09-21T03:44:14.315314+0000","last_unstale":"2024-09-21T03:45:55.892137+0000","last_undegraded":"2024-09-21T03:45:55.892137+0000","last_fullsized":"2024-09-21T03:45:55.892137+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_clean_scrub_stamp":"2024-09-21T03:44:13.292303+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:22:20.500220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"79'5","reported_seq":236,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:46:07.856800+0000","last_change":"2024-09-21T03:44:12.313251+0000","last_active":"2024-09-21T03:46:07.856800+0000","last_peered":"2024-09-21T03:46:07.856800+0000","last_clean":"2024-09-21T03:46:07.856800+0000","last_became_active":"2024-09-21T03:44:12.312941+0000","last_became_peered":"2024-09-21T03:44:12.312941+0000","last_unstale":"2024-09-21T03:46:07.856800+0000","last_undegraded":"2024-09-21T03:46:07.856800+0000","last_fullsized":"2024-09-21T03:46:07.856800+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:11.282438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:11.282438+0000","last_clean_scrub_stamp":"2024-09-21T03:44:11.282438+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:38:39.876097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":208,"num_read_kb":208,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"79'10","reported_seq":28,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:45:55.905106+0000","last_change":"2024-09-21T03:44:14.319621+0000","last_active":"2024-09-21T03:45:55.905106+0000","last_peered":"2024-09-21T03:45:55.905106+0000","last_clean":"2024-09-21T03:45:55.905106+0000","last_became_active":"2024-09-21T03:44:14.316173+0000","last_became_peered":"2024-09-21T03:44:14.316173+0000","last_unstale":"2024-09-21T03:45:55.905106+0000","last_undegraded":"2024-09-21T03:45:55.905106+0000","last_fullsized":"2024-09-21T03:45:55.905106+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_clean_scrub_stamp":"2024-09-21T03:44:13.292303+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:10:21.952989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"77'4","reported_seq":27,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:44:27.419175+0000","last_change":"2024-09-21T03:44:12.308893+0000","last_active":"2024-09-21T03:44:27.419175+0000","last_peered":"2024-09-21T03:44:27.419175+0000","last_clean":"2024-09-21T03:44:27.419175+0000","last_became_active":"2024-09-21T03:44:12.308551+0000","last_became_peered":"2024-09-21T03:44:12.308551+0000","last_unstale":"2024-09-21T03:44:27.419175+0000","last_undegraded":"2024-09-21T03:44:27.419175+0000","last_fullsized":"2024-09-21T03:44:27.419175+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:11.282438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:11.282438+0000","last_clean_scrub_stamp":"2024-09-21T03:44:11.282438+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:48:18.401595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"79'9","reported_seq":27,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:45:55.904971+0000","last_change":"2024-09-21T03:44:14.321875+0000","last_active":"2024-09-21T03:45:55.904971+0000","last_peered":"2024-09-21T03:45:55.904971+0000","last_clean":"2024-09-21T03:45:55.904971+0000","last_became_active":"2024-09-21T03:44:14.321483+0000","last_became_peered":"2024-09-21T03:44:14.321483+0000","last_unstale":"2024-09-21T03:45:55.904971+0000","last_undegraded":"2024-09-21T03:45:55.904971+0000","last_fullsized":"2024-09-21T03:45:55.904971+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_clean_scrub_stamp":"2024-09-21T03:44:13.292303+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:40:17.244780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"77'5","reported_seq":29,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:44:27.414461+0000","last_change":"2024-09-21T03:44:12.313208+0000","last_active":"2024-09-21T03:44:27.414461+0000","last_peered":"2024-09-21T03:44:27.414461+0000","last_clean":"2024-09-21T03:44:27.414461+0000","last_became_active":"2024-09-21T03:44:12.312783+0000","last_became_peered":"2024-09-21T03:44:12.312783+0000","last_unstale":"2024-09-21T03:44:27.414461+0000","last_undegraded":"2024-09-21T03:44:27.414461+0000","last_fullsized":"2024-09-21T03:44:27.414461+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:11.282438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:11.282438+0000","last_clean_scrub_stamp":"2024-09-21T03:44:11.282438+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:18:45.114361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"79'11","reported_seq":36,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:45:55.907006+0000","last_change":"2024-09-21T03:44:27.421129+0000","last_active":"2024-09-21T03:45:55.907006+0000","last_peered":"2024-09-21T03:45:55.907006+0000","last_clean":"2024-09-21T03:45:55.907006+0000","last_became_active":"2024-09-21T03:44:27.420642+0000","last_became_peered":"2024-09-21T03:44:27.420642+0000","last_unstale":"2024-09-21T03:45:55.907006+0000","last_undegraded":"2024-09-21T03:45:55.907006+0000","last_fullsized":"2024-09-21T03:45:55.907006+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_clean_scrub_stamp":"2024-09-21T03:44:13.292303+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:49:38.065743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,6],"acting":[1,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"79'34","reported_seq":277,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:46:07.917025+0000","last_change":"2024-09-21T03:44:12.314863+0000","last_active":"2024-09-21T03:46:07.917025+0000","last_peered":"2024-09-21T03:46:07.917025+0000","last_clean":"2024-09-21T03:46:07.917025+0000","last_became_active":"2024-09-21T03:44:12.314471+0000","last_became_peered":"2024-09-21T03:44:12.314471+0000","last_unstale":"2024-09-21T03:46:07.917025+0000","last_undegraded":"2024-09-21T03:46:07.917025+0000","last_fullsized":"2024-09-21T03:46:07.917025+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:11.282438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:11.282438+0000","last_clean_scrub_stamp":"2024-09-21T03:44:11.282438+0000","objects_scrubbed":0,"log_size":34,"log_dups_size":0,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:41:03.471226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":64459,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":220,"num_read_kb":173,"num_write":36,"num_write_kb":79,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"79'6","reported_seq":24,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:45:55.897616+0000","last_change":"2024-09-21T03:44:14.325471+0000","last_active":"2024-09-21T03:45:55.897616+0000","last_peered":"2024-09-21T03:45:55.897616+0000","last_clean":"2024-09-21T03:45:55.897616+0000","last_became_active":"2024-09-21T03:44:14.325072+0000","last_became_peered":"2024-09-21T03:44:14.325072+0000","last_unstale":"2024-09-21T03:45:55.897616+0000","last_undegraded":"2024-09-21T03:45:55.897616+0000","last_fullsized":"2024-09-21T03:45:55.897616+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_clean_scrub_stamp":"2024-09-21T03:44:13.292303+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:40:40.097240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"77'9","reported_seq":34,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:44:27.416778+0000","last_change":"2024-09-21T03:44:12.312932+0000","last_active":"2024-09-21T03:44:27.416778+0000","last_peered":"2024-09-21T03:44:27.416778+0000","last_clean":"2024-09-21T03:44:27.416778+0000","last_became_active":"2024-09-21T03:44:12.312368+0000","last_became_peered":"2024-09-21T03:44:12.312368+0000","last_unstale":"2024-09-21T03:44:27.416778+0000","last_undegraded":"2024-09-21T03:44:27.416778+0000","last_fullsized":"2024-09-21T03:44:27.416778+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:11.282438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:11.282438+0000","last_clean_scrub_stamp":"2024-09-21T03:44:11.282438+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:28:11.579598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"79'9","reported_seq":27,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:45:55.901852+0000","last_change":"2024-09-21T03:44:14.329564+0000","last_active":"2024-09-21T03:45:55.901852+0000","last_peered":"2024-09-21T03:45:55.901852+0000","last_clean":"2024-09-21T03:45:55.901852+0000","last_became_active":"2024-09-21T03:44:14.329122+0000","last_became_peered":"2024-09-21T03:44:14.329122+0000","last_unstale":"2024-09-21T03:45:55.901852+0000","last_undegraded":"2024-09-21T03:45:55.901852+0000","last_fullsized":"2024-09-21T03:45:55.901852+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_clean_scrub_stamp":"2024-09-21T03:44:13.292303+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:16:32.702770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"77'5","reported_seq":29,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:44:27.413875+0000","last_change":"2024-09-21T03:44:12.303339+0000","last_active":"2024-09-21T03:44:27.413875+0000","last_peered":"2024-09-21T03:44:27.413875+0000","last_clean":"2024-09-21T03:44:27.413875+0000","last_became_active":"2024-09-21T03:44:12.302905+0000","last_became_peered":"2024-09-21T03:44:12.302905+0000","last_unstale":"2024-09-21T03:44:27.413875+0000","last_undegraded":"2024-09-21T03:44:27.413875+0000","last_fullsized":"2024-09-21T03:44:27.413875+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:11.282438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:11.282438+0000","last_clean_scrub_stamp":"2024-09-21T03:44:11.282438+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:57:16.424976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"79'15","reported_seq":33,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:45:55.905704+0000","last_change":"2024-09-21T03:44:14.323473+0000","last_active":"2024-09-21T03:45:55.905704+0000","last_peered":"2024-09-21T03:45:55.905704+0000","last_clean":"2024-09-21T03:45:55.905704+0000","last_became_active":"2024-09-21T03:44:14.322560+0000","last_became_peered":"2024-09-21T03:44:14.322560+0000","last_unstale":"2024-09-21T03:45:55.905704+0000","last_undegraded":"2024-09-21T03:45:55.905704+0000","last_fullsized":"2024-09-21T03:45:55.905704+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_clean_scrub_stamp":"2024-09-21T03:44:13.292303+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:07:55.714351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"79'8","reported_seq":142,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:46:09.861993+0000","last_change":"2024-09-21T03:44:12.312331+0000","last_active":"2024-09-21T03:46:09.861993+0000","last_peered":"2024-09-21T03:46:09.861993+0000","last_clean":"2024-09-21T03:46:09.861993+0000","last_became_active":"2024-09-21T03:44:12.311984+0000","last_became_peered":"2024-09-21T03:44:12.311984+0000","last_unstale":"2024-09-21T03:46:09.861993+0000","last_undegraded":"2024-09-21T03:46:09.861993+0000","last_fullsized":"2024-09-21T03:46:09.861993+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:11.282438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:11.282438+0000","last_clean_scrub_stamp":"2024-09-21T03:44:11.282438+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:06:11.906592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1511,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":112,"num_read_kb":6,"num_write":8,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"79'6","reported_seq":24,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:45:55.885280+0000","last_change":"2024-09-21T03:44:14.318437+0000","last_active":"2024-09-21T03:45:55.885280+0000","last_peered":"2024-09-21T03:45:55.885280+0000","last_clean":"2024-09-21T03:45:55.885280+0000","last_became_active":"2024-09-21T03:44:14.318052+0000","last_became_peered":"2024-09-21T03:44:14.318052+0000","last_unstale":"2024-09-21T03:45:55.885280+0000","last_undegraded":"2024-09-21T03:45:55.885280+0000","last_fullsized":"2024-09-21T03:45:55.885280+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_clean_scrub_stamp":"2024-09-21T03:44:13.292303+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:10:12.136159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"77'17","reported_seq":41,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:44:27.419831+0000","last_change":"2024-09-21T03:44:12.310616+0000","last_active":"2024-09-21T03:44:27.419831+0000","last_peered":"2024-09-21T03:44:27.419831+0000","last_clean":"2024-09-21T03:44:27.419831+0000","last_became_active":"2024-09-21T03:44:12.310239+0000","last_became_peered":"2024-09-21T03:44:12.310239+0000","last_unstale":"2024-09-21T03:44:27.419831+0000","last_undegraded":"2024-09-21T03:44:27.419831+0000","last_fullsized":"2024-09-21T03:44:27.419831+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:11.282438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:11.282438+0000","last_clean_scrub_stamp":"2024-09-21T03:44:11.282438+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:43:38.046281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"79'6","reported_seq":24,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:45:55.894707+0000","last_change":"2024-09-21T03:44:14.327058+0000","last_active":"2024-09-21T03:45:55.894707+0000","last_peered":"2024-09-21T03:45:55.894707+0000","last_clean":"2024-09-21T03:45:55.894707+0000","last_became_active":"2024-09-21T03:44:14.326641+0000","last_became_peered":"2024-09-21T03:44:14.326641+0000","last_unstale":"2024-09-21T03:45:55.894707+0000","last_undegraded":"2024-09-21T03:45:55.894707+0000","last_fullsized":"2024-09-21T03:45:55.894707+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_clean_scrub_stamp":"2024-09-21T03:44:13.292303+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:10:50.686102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"77'2","reported_seq":26,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:44:27.420648+0000","last_change":"2024-09-21T03:44:12.310426+0000","last_active":"2024-09-21T03:44:27.420648+0000","last_peered":"2024-09-21T03:44:27.420648+0000","last_clean":"2024-09-21T03:44:27.420648+0000","last_became_active":"2024-09-21T03:44:12.309231+0000","last_became_peered":"2024-09-21T03:44:12.309231+0000","last_unstale":"2024-09-21T03:44:27.420648+0000","last_undegraded":"2024-09-21T03:44:27.420648+0000","last_fullsized":"2024-09-21T03:44:27.420648+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:11.282438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:11.282438+0000","last_clean_scrub_stamp":"2024-09-21T03:44:11.282438+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:32:18.284741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"79'5","reported_seq":23,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:45:55.887191+0000","last_change":"2024-09-21T03:44:14.319179+0000","last_active":"2024-09-21T03:45:55.887191+0000","last_peered":"2024-09-21T03:45:55.887191+0000","last_clean":"2024-09-21T03:45:55.887191+0000","last_became_active":"2024-09-21T03:44:14.318826+0000","last_became_peered":"2024-09-21T03:44:14.318826+0000","last_unstale":"2024-09-21T03:45:55.887191+0000","last_undegraded":"2024-09-21T03:45:55.887191+0000","last_fullsized":"2024-09-21T03:45:55.887191+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_clean_scrub_stamp":"2024-09-21T03:44:13.292303+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:12:57.985087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.e","version":"77'5","reported_seq":28,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:44:27.410705+0000","last_change":"2024-09-21T03:44:12.315658+0000","last_active":"2024-09-21T03:44:27.410705+0000","last_peered":"2024-09-21T03:44:27.410705+0000","last_clean":"2024-09-21T03:44:27.410705+0000","last_became_active":"2024-09-21T03:44:12.315240+0000","last_became_peered":"2024-09-21T03:44:12.315240+0000","last_unstale":"2024-09-21T03:44:27.410705+0000","last_undegraded":"2024-09-21T03:44:27.410705+0000","last_fullsized":"2024-09-21T03:44:27.410705+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:11.282438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:11.282438+0000","last_clean_scrub_stamp":"2024-09-21T03:44:11.282438+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:22:07.493437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"79'9","reported_seq":27,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:45:55.898231+0000","last_change":"2024-09-21T03:44:14.328014+0000","last_active":"2024-09-21T03:45:55.898231+0000","last_peered":"2024-09-21T03:45:55.898231+0000","last_clean":"2024-09-21T03:45:55.898231+0000","last_became_active":"2024-09-21T03:44:14.327638+0000","last_became_peered":"2024-09-21T03:44:14.327638+0000","last_unstale":"2024-09-21T03:45:55.898231+0000","last_undegraded":"2024-09-21T03:45:55.898231+0000","last_fullsized":"2024-09-21T03:45:55.898231+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_clean_scrub_stamp":"2024-09-21T03:44:13.292303+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:48:32.542776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"77'3","reported_seq":27,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:44:27.414656+0000","last_change":"2024-09-21T03:44:12.309272+0000","last_active":"2024-09-21T03:44:27.414656+0000","last_peered":"2024-09-21T03:44:27.414656+0000","last_clean":"2024-09-21T03:44:27.414656+0000","last_became_active":"2024-09-21T03:44:12.308718+0000","last_became_peered":"2024-09-21T03:44:12.308718+0000","last_unstale":"2024-09-21T03:44:27.414656+0000","last_undegraded":"2024-09-21T03:44:27.414656+0000","last_fullsized":"2024-09-21T03:44:27.414656+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:11.282438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:11.282438+0000","last_clean_scrub_stamp":"2024-09-21T03:44:11.282438+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:03:49.433966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"79'7","reported_seq":25,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:45:55.889962+0000","last_change":"2024-09-21T03:44:14.322148+0000","last_active":"2024-09-21T03:45:55.889962+0000","last_peered":"2024-09-21T03:45:55.889962+0000","last_clean":"2024-09-21T03:45:55.889962+0000","last_became_active":"2024-09-21T03:44:14.321784+0000","last_became_peered":"2024-09-21T03:44:14.321784+0000","last_unstale":"2024-09-21T03:45:55.889962+0000","last_undegraded":"2024-09-21T03:45:55.889962+0000","last_fullsized":"2024-09-21T03:45:55.889962+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_clean_scrub_stamp":"2024-09-21T03:44:13.292303+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:40:34.887134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"77'5","reported_seq":122,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:46:07.560527+0000","last_change":"2024-09-21T03:44:27.427722+0000","last_active":"2024-09-21T03:46:07.560527+0000","last_peered":"2024-09-21T03:46:07.560527+0000","last_clean":"2024-09-21T03:46:07.560527+0000","last_became_active":"2024-09-21T03:44:27.419672+0000","last_became_peered":"2024-09-21T03:44:27.419672+0000","last_unstale":"2024-09-21T03:46:07.560527+0000","last_undegraded":"2024-09-21T03:46:07.560527+0000","last_fullsized":"2024-09-21T03:46:07.560527+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:11.282438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:11.282438+0000","last_clean_scrub_stamp":"2024-09-21T03:44:11.282438+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:24:31.890439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1170,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":102,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":1170,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,11,0],"acting":[1,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.d","version":"79'7","reported_seq":25,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:45:55.897960+0000","last_change":"2024-09-21T03:44:14.316503+0000","last_active":"2024-09-21T03:45:55.897960+0000","last_peered":"2024-09-21T03:45:55.897960+0000","last_clean":"2024-09-21T03:45:55.897960+0000","last_became_active":"2024-09-21T03:44:14.316140+0000","last_became_peered":"2024-09-21T03:44:14.316140+0000","last_unstale":"2024-09-21T03:45:55.897960+0000","last_undegraded":"2024-09-21T03:45:55.897960+0000","last_fullsized":"2024-09-21T03:45:55.897960+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_clean_scrub_stamp":"2024-09-21T03:44:13.292303+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:00:30.647579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"21'92","reported_seq":97,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:44:27.418361+0000","last_change":"2024-09-21T03:43:01.568120+0000","last_active":"2024-09-21T03:44:27.418361+0000","last_peered":"2024-09-21T03:44:27.418361+0000","last_clean":"2024-09-21T03:44:27.418361+0000","last_became_active":"2024-09-21T03:43:01.476311+0000","last_became_peered":"2024-09-21T03:43:01.476311+0000","last_unstale":"2024-09-21T03:44:27.418361+0000","last_undegraded":"2024-09-21T03:44:27.418361+0000","last_fullsized":"2024-09-21T03:44:27.418361+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:39:30.121380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:39:30.121380+0000","last_clean_scrub_stamp":"2024-09-21T03:39:30.121380+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:40:27.967442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"77'6","reported_seq":32,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:44:27.613250+0000","last_change":"2024-09-21T03:44:12.311860+0000","last_active":"2024-09-21T03:44:27.613250+0000","last_peered":"2024-09-21T03:44:27.613250+0000","last_clean":"2024-09-21T03:44:27.613250+0000","last_became_active":"2024-09-21T03:44:12.311453+0000","last_became_peered":"2024-09-21T03:44:12.311453+0000","last_unstale":"2024-09-21T03:44:27.613250+0000","last_undegraded":"2024-09-21T03:44:27.613250+0000","last_fullsized":"2024-09-21T03:44:27.613250+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:11.282438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:11.282438+0000","last_clean_scrub_stamp":"2024-09-21T03:44:11.282438+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:23:40.550445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2","version":"79'8","reported_seq":20,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:45:55.883471+0000","last_change":"2024-09-21T03:44:27.416799+0000","last_active":"2024-09-21T03:45:55.883471+0000","last_peered":"2024-09-21T03:45:55.883471+0000","last_clean":"2024-09-21T03:45:55.883471+0000","last_became_active":"2024-09-21T03:44:27.416306+0000","last_became_peered":"2024-09-21T03:44:27.416306+0000","last_unstale":"2024-09-21T03:45:55.883471+0000","last_undegraded":"2024-09-21T03:45:55.883471+0000","last_fullsized":"2024-09-21T03:45:55.883471+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_clean_scrub_stamp":"2024-09-21T03:44:13.292303+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:34:26.547100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,5,1],"acting":[9,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.0","version":"77'4","reported_seq":27,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:44:27.418313+0000","last_change":"2024-09-21T03:44:12.307302+0000","last_active":"2024-09-21T03:44:27.418313+0000","last_peered":"2024-09-21T03:44:27.418313+0000","last_clean":"2024-09-21T03:44:27.418313+0000","last_became_active":"2024-09-21T03:44:12.306856+0000","last_became_peered":"2024-09-21T03:44:12.306856+0000","last_unstale":"2024-09-21T03:44:27.418313+0000","last_undegraded":"2024-09-21T03:44:27.418313+0000","last_fullsized":"2024-09-21T03:44:27.418313+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:11.282438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:11.282438+0000","last_clean_scrub_stamp":"2024-09-21T03:44:11.282438+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:21:20.420020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"79'5","reported_seq":23,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:45:55.899176+0000","last_change":"2024-09-21T03:44:14.322913+0000","last_active":"2024-09-21T03:45:55.899176+0000","last_peered":"2024-09-21T03:45:55.899176+0000","last_clean":"2024-09-21T03:45:55.899176+0000","last_became_active":"2024-09-21T03:44:14.322168+0000","last_became_peered":"2024-09-21T03:44:14.322168+0000","last_unstale":"2024-09-21T03:45:55.899176+0000","last_undegraded":"2024-09-21T03:45:55.899176+0000","last_fullsized":"2024-09-21T03:45:55.899176+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_clean_scrub_stamp":"2024-09-21T03:44:13.292303+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:30:09.041211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"79'6","reported_seq":133,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:46:05.855033+0000","last_change":"2024-09-21T03:44:12.309084+0000","last_active":"2024-09-21T03:46:05.855033+0000","last_peered":"2024-09-21T03:46:05.855033+0000","last_clean":"2024-09-21T03:46:05.855033+0000","last_became_active":"2024-09-21T03:44:12.308551+0000","last_became_peered":"2024-09-21T03:44:12.308551+0000","last_unstale":"2024-09-21T03:46:05.855033+0000","last_undegraded":"2024-09-21T03:46:05.855033+0000","last_fullsized":"2024-09-21T03:46:05.855033+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:11.282438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:11.282438+0000","last_clean_scrub_stamp":"2024-09-21T03:44:11.282438+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:46:33.895904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":104,"num_read_kb":104,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"79'8","reported_seq":26,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:45:55.884045+0000","last_change":"2024-09-21T03:44:14.322720+0000","last_active":"2024-09-21T03:45:55.884045+0000","last_peered":"2024-09-21T03:45:55.884045+0000","last_clean":"2024-09-21T03:45:55.884045+0000","last_became_active":"2024-09-21T03:44:14.322367+0000","last_became_peered":"2024-09-21T03:44:14.322367+0000","last_unstale":"2024-09-21T03:45:55.884045+0000","last_undegraded":"2024-09-21T03:45:55.884045+0000","last_fullsized":"2024-09-21T03:45:55.884045+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_clean_scrub_stamp":"2024-09-21T03:44:13.292303+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:59:51.692976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"79'8","reported_seq":137,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:46:07.760063+0000","last_change":"2024-09-21T03:44:12.308719+0000","last_active":"2024-09-21T03:46:07.760063+0000","last_peered":"2024-09-21T03:46:07.760063+0000","last_clean":"2024-09-21T03:46:07.760063+0000","last_became_active":"2024-09-21T03:44:12.308380+0000","last_became_peered":"2024-09-21T03:44:12.308380+0000","last_unstale":"2024-09-21T03:46:07.760063+0000","last_undegraded":"2024-09-21T03:46:07.760063+0000","last_fullsized":"2024-09-21T03:46:07.760063+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:11.282438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:11.282438+0000","last_clean_scrub_stamp":"2024-09-21T03:44:11.282438+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:41:02.087501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1428,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":106,"num_read_kb":4,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"79'7","reported_seq":25,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:45:55.901997+0000","last_change":"2024-09-21T03:44:14.326541+0000","last_active":"2024-09-21T03:45:55.901997+0000","last_peered":"2024-09-21T03:45:55.901997+0000","last_clean":"2024-09-21T03:45:55.901997+0000","last_became_active":"2024-09-21T03:44:14.326132+0000","last_became_peered":"2024-09-21T03:44:14.326132+0000","last_unstale":"2024-09-21T03:45:55.901997+0000","last_undegraded":"2024-09-21T03:45:55.901997+0000","last_fullsized":"2024-09-21T03:45:55.901997+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_clean_scrub_stamp":"2024-09-21T03:44:13.292303+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:09:27.711919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"77'1","reported_seq":24,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:44:27.414339+0000","last_change":"2024-09-21T03:44:12.309463+0000","last_active":"2024-09-21T03:44:27.414339+0000","last_peered":"2024-09-21T03:44:27.414339+0000","last_clean":"2024-09-21T03:44:27.414339+0000","last_became_active":"2024-09-21T03:44:12.309054+0000","last_became_peered":"2024-09-21T03:44:12.309054+0000","last_unstale":"2024-09-21T03:44:27.414339+0000","last_undegraded":"2024-09-21T03:44:27.414339+0000","last_fullsized":"2024-09-21T03:44:27.414339+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:11.282438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:11.282438+0000","last_clean_scrub_stamp":"2024-09-21T03:44:11.282438+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:12:12.505538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"79'5","reported_seq":23,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:45:55.890793+0000","last_change":"2024-09-21T03:44:14.318261+0000","last_active":"2024-09-21T03:45:55.890793+0000","last_peered":"2024-09-21T03:45:55.890793+0000","last_clean":"2024-09-21T03:45:55.890793+0000","last_became_active":"2024-09-21T03:44:14.317839+0000","last_became_peered":"2024-09-21T03:44:14.317839+0000","last_unstale":"2024-09-21T03:45:55.890793+0000","last_undegraded":"2024-09-21T03:45:55.890793+0000","last_fullsized":"2024-09-21T03:45:55.890793+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_clean_scrub_stamp":"2024-09-21T03:44:13.292303+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:19:05.504749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.5","version":"77'4","reported_seq":20,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:44:27.434085+0000","last_change":"2024-09-21T03:44:27.433630+0000","last_active":"2024-09-21T03:44:27.434085+0000","last_peered":"2024-09-21T03:44:27.434085+0000","last_clean":"2024-09-21T03:44:27.434085+0000","last_became_active":"2024-09-21T03:44:27.422093+0000","last_became_peered":"2024-09-21T03:44:27.422093+0000","last_unstale":"2024-09-21T03:44:27.434085+0000","last_undegraded":"2024-09-21T03:44:27.434085+0000","last_fullsized":"2024-09-21T03:44:27.434085+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:11.282438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:11.282438+0000","last_clean_scrub_stamp":"2024-09-21T03:44:11.282438+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:39:42.651024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,9],"acting":[11,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.4","version":"79'9","reported_seq":27,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:45:55.899012+0000","last_change":"2024-09-21T03:44:14.322620+0000","last_active":"2024-09-21T03:45:55.899012+0000","last_peered":"2024-09-21T03:45:55.899012+0000","last_clean":"2024-09-21T03:45:55.899012+0000","last_became_active":"2024-09-21T03:44:14.322214+0000","last_became_peered":"2024-09-21T03:44:14.322214+0000","last_unstale":"2024-09-21T03:45:55.899012+0000","last_undegraded":"2024-09-21T03:45:55.899012+0000","last_fullsized":"2024-09-21T03:45:55.899012+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_clean_scrub_stamp":"2024-09-21T03:44:13.292303+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:53:20.519168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"79'17","reported_seq":40,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:45:56.352616+0000","last_change":"2024-09-21T03:44:12.311302+0000","last_active":"2024-09-21T03:45:56.352616+0000","last_peered":"2024-09-21T03:45:56.352616+0000","last_clean":"2024-09-21T03:45:56.352616+0000","last_became_active":"2024-09-21T03:44:12.310947+0000","last_became_peered":"2024-09-21T03:44:12.310947+0000","last_unstale":"2024-09-21T03:45:56.352616+0000","last_undegraded":"2024-09-21T03:45:56.352616+0000","last_fullsized":"2024-09-21T03:45:56.352616+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:11.282438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:11.282438+0000","last_clean_scrub_stamp":"2024-09-21T03:44:11.282438+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:09:34.308726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":30,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"79'5","reported_seq":23,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:45:55.891717+0000","last_change":"2024-09-21T03:44:14.324675+0000","last_active":"2024-09-21T03:45:55.891717+0000","last_peered":"2024-09-21T03:45:55.891717+0000","last_clean":"2024-09-21T03:45:55.891717+0000","last_became_active":"2024-09-21T03:44:14.324214+0000","last_became_peered":"2024-09-21T03:44:14.324214+0000","last_unstale":"2024-09-21T03:45:55.891717+0000","last_undegraded":"2024-09-21T03:45:55.891717+0000","last_fullsized":"2024-09-21T03:45:55.891717+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_clean_scrub_stamp":"2024-09-21T03:44:13.292303+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:33:51.071794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"77'7","reported_seq":31,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:44:27.613637+0000","last_change":"2024-09-21T03:44:12.322312+0000","last_active":"2024-09-21T03:44:27.613637+0000","last_peered":"2024-09-21T03:44:27.613637+0000","last_clean":"2024-09-21T03:44:27.613637+0000","last_became_active":"2024-09-21T03:44:12.322018+0000","last_became_peered":"2024-09-21T03:44:12.322018+0000","last_unstale":"2024-09-21T03:44:27.613637+0000","last_undegraded":"2024-09-21T03:44:27.613637+0000","last_fullsized":"2024-09-21T03:44:27.613637+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:11.282438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:11.282438+0000","last_clean_scrub_stamp":"2024-09-21T03:44:11.282438+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:11:41.817574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"79'7","reported_seq":25,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:45:55.904952+0000","last_change":"2024-09-21T03:44:14.326682+0000","last_active":"2024-09-21T03:45:55.904952+0000","last_peered":"2024-09-21T03:45:55.904952+0000","last_clean":"2024-09-21T03:45:55.904952+0000","last_became_active":"2024-09-21T03:44:14.326313+0000","last_became_peered":"2024-09-21T03:44:14.326313+0000","last_unstale":"2024-09-21T03:45:55.904952+0000","last_undegraded":"2024-09-21T03:45:55.904952+0000","last_fullsized":"2024-09-21T03:45:55.904952+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_clean_scrub_stamp":"2024-09-21T03:44:13.292303+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:26:44.681245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"77'2","reported_seq":36,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:44:27.436319+0000","last_change":"2024-09-21T03:44:27.436245+0000","last_active":"2024-09-21T03:44:27.436319+0000","last_peered":"2024-09-21T03:44:27.436319+0000","last_clean":"2024-09-21T03:44:27.436319+0000","last_became_active":"2024-09-21T03:44:27.422134+0000","last_became_peered":"2024-09-21T03:44:27.422134+0000","last_unstale":"2024-09-21T03:44:27.436319+0000","last_undegraded":"2024-09-21T03:44:27.436319+0000","last_fullsized":"2024-09-21T03:44:27.436319+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:11.282438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:11.282438+0000","last_clean_scrub_stamp":"2024-09-21T03:44:11.282438+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:50:22.655050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,11,0],"acting":[1,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"79'2","reported_seq":20,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:45:55.854212+0000","last_change":"2024-09-21T03:44:14.316312+0000","last_active":"2024-09-21T03:45:55.854212+0000","last_peered":"2024-09-21T03:45:55.854212+0000","last_clean":"2024-09-21T03:45:55.854212+0000","last_became_active":"2024-09-21T03:44:14.315929+0000","last_became_peered":"2024-09-21T03:44:14.315929+0000","last_unstale":"2024-09-21T03:45:55.854212+0000","last_undegraded":"2024-09-21T03:45:55.854212+0000","last_fullsized":"2024-09-21T03:45:55.854212+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_clean_scrub_stamp":"2024-09-21T03:44:13.292303+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:31:33.071983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"77'7","reported_seq":30,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:44:27.610761+0000","last_change":"2024-09-21T03:44:12.312655+0000","last_active":"2024-09-21T03:44:27.610761+0000","last_peered":"2024-09-21T03:44:27.610761+0000","last_clean":"2024-09-21T03:44:27.610761+0000","last_became_active":"2024-09-21T03:44:12.312196+0000","last_became_peered":"2024-09-21T03:44:12.312196+0000","last_unstale":"2024-09-21T03:44:27.610761+0000","last_undegraded":"2024-09-21T03:44:27.610761+0000","last_fullsized":"2024-09-21T03:44:27.610761+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:11.282438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:11.282438+0000","last_clean_scrub_stamp":"2024-09-21T03:44:11.282438+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:24:32.342461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"79'9","reported_seq":27,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:45:55.897072+0000","last_change":"2024-09-21T03:44:14.321875+0000","last_active":"2024-09-21T03:45:55.897072+0000","last_peered":"2024-09-21T03:45:55.897072+0000","last_clean":"2024-09-21T03:45:55.897072+0000","last_became_active":"2024-09-21T03:44:14.320682+0000","last_became_peered":"2024-09-21T03:44:14.320682+0000","last_unstale":"2024-09-21T03:45:55.897072+0000","last_undegraded":"2024-09-21T03:45:55.897072+0000","last_fullsized":"2024-09-21T03:45:55.897072+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_clean_scrub_stamp":"2024-09-21T03:44:13.292303+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:41:32.435288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"77'3","reported_seq":26,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:44:27.418692+0000","last_change":"2024-09-21T03:44:12.309100+0000","last_active":"2024-09-21T03:44:27.418692+0000","last_peered":"2024-09-21T03:44:27.418692+0000","last_clean":"2024-09-21T03:44:27.418692+0000","last_became_active":"2024-09-21T03:44:12.308671+0000","last_became_peered":"2024-09-21T03:44:12.308671+0000","last_unstale":"2024-09-21T03:44:27.418692+0000","last_undegraded":"2024-09-21T03:44:27.418692+0000","last_fullsized":"2024-09-21T03:44:27.418692+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:11.282438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:11.282438+0000","last_clean_scrub_stamp":"2024-09-21T03:44:11.282438+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:09:19.347980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"79'8","reported_seq":26,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:45:55.890407+0000","last_change":"2024-09-21T03:44:14.320170+0000","last_active":"2024-09-21T03:45:55.890407+0000","last_peered":"2024-09-21T03:45:55.890407+0000","last_clean":"2024-09-21T03:45:55.890407+0000","last_became_active":"2024-09-21T03:44:14.319794+0000","last_became_peered":"2024-09-21T03:44:14.319794+0000","last_unstale":"2024-09-21T03:45:55.890407+0000","last_undegraded":"2024-09-21T03:45:55.890407+0000","last_fullsized":"2024-09-21T03:45:55.890407+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_clean_scrub_stamp":"2024-09-21T03:44:13.292303+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:42:09.684471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"77'4","reported_seq":28,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:44:27.420077+0000","last_change":"2024-09-21T03:44:12.310851+0000","last_active":"2024-09-21T03:44:27.420077+0000","last_peered":"2024-09-21T03:44:27.420077+0000","last_clean":"2024-09-21T03:44:27.420077+0000","last_became_active":"2024-09-21T03:44:12.310302+0000","last_became_peered":"2024-09-21T03:44:12.310302+0000","last_unstale":"2024-09-21T03:44:27.420077+0000","last_undegraded":"2024-09-21T03:44:27.420077+0000","last_fullsized":"2024-09-21T03:44:27.420077+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:11.282438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:11.282438+0000","last_clean_scrub_stamp":"2024-09-21T03:44:11.282438+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:23:46.959121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"79'5","reported_seq":23,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:45:55.904667+0000","last_change":"2024-09-21T03:44:14.324544+0000","last_active":"2024-09-21T03:45:55.904667+0000","last_peered":"2024-09-21T03:45:55.904667+0000","last_clean":"2024-09-21T03:45:55.904667+0000","last_became_active":"2024-09-21T03:44:14.322881+0000","last_became_peered":"2024-09-21T03:44:14.322881+0000","last_unstale":"2024-09-21T03:45:55.904667+0000","last_undegraded":"2024-09-21T03:45:55.904667+0000","last_fullsized":"2024-09-21T03:45:55.904667+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_clean_scrub_stamp":"2024-09-21T03:44:13.292303+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:58:42.760185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"79'15","reported_seq":40,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:45:55.895829+0000","last_change":"2024-09-21T03:44:27.422438+0000","last_active":"2024-09-21T03:45:55.895829+0000","last_peered":"2024-09-21T03:45:55.895829+0000","last_clean":"2024-09-21T03:45:55.895829+0000","last_became_active":"2024-09-21T03:44:27.421935+0000","last_became_peered":"2024-09-21T03:44:27.421935+0000","last_unstale":"2024-09-21T03:45:55.895829+0000","last_undegraded":"2024-09-21T03:45:55.895829+0000","last_fullsized":"2024-09-21T03:45:55.895829+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_clean_scrub_stamp":"2024-09-21T03:44:13.292303+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:20:50.097783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,1],"acting":[5,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"77'5","reported_seq":29,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:44:27.420510+0000","last_change":"2024-09-21T03:44:12.311509+0000","last_active":"2024-09-21T03:44:27.420510+0000","last_peered":"2024-09-21T03:44:27.420510+0000","last_clean":"2024-09-21T03:44:27.420510+0000","last_became_active":"2024-09-21T03:44:12.310880+0000","last_became_peered":"2024-09-21T03:44:12.310880+0000","last_unstale":"2024-09-21T03:44:27.420510+0000","last_undegraded":"2024-09-21T03:44:27.420510+0000","last_fullsized":"2024-09-21T03:44:27.420510+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:11.282438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:11.282438+0000","last_clean_scrub_stamp":"2024-09-21T03:44:11.282438+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:59:33.887234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"79'9","reported_seq":27,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:45:55.905320+0000","last_change":"2024-09-21T03:44:14.330938+0000","last_active":"2024-09-21T03:45:55.905320+0000","last_peered":"2024-09-21T03:45:55.905320+0000","last_clean":"2024-09-21T03:45:55.905320+0000","last_became_active":"2024-09-21T03:44:14.330559+0000","last_became_peered":"2024-09-21T03:44:14.330559+0000","last_unstale":"2024-09-21T03:45:55.905320+0000","last_undegraded":"2024-09-21T03:45:55.905320+0000","last_fullsized":"2024-09-21T03:45:55.905320+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_clean_scrub_stamp":"2024-09-21T03:44:13.292303+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:05:32.535660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"77'12","reported_seq":36,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:44:27.420563+0000","last_change":"2024-09-21T03:44:12.307667+0000","last_active":"2024-09-21T03:44:27.420563+0000","last_peered":"2024-09-21T03:44:27.420563+0000","last_clean":"2024-09-21T03:44:27.420563+0000","last_became_active":"2024-09-21T03:44:12.307074+0000","last_became_peered":"2024-09-21T03:44:12.307074+0000","last_unstale":"2024-09-21T03:44:27.420563+0000","last_undegraded":"2024-09-21T03:44:27.420563+0000","last_fullsized":"2024-09-21T03:44:27.420563+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:11.282438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:11.282438+0000","last_clean_scrub_stamp":"2024-09-21T03:44:11.282438+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:48:41.598375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1f","version":"79'6","reported_seq":24,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:45:55.903388+0000","last_change":"2024-09-21T03:44:14.325487+0000","last_active":"2024-09-21T03:45:55.903388+0000","last_peered":"2024-09-21T03:45:55.903388+0000","last_clean":"2024-09-21T03:45:55.903388+0000","last_became_active":"2024-09-21T03:44:14.325153+0000","last_became_peered":"2024-09-21T03:44:14.325153+0000","last_unstale":"2024-09-21T03:45:55.903388+0000","last_undegraded":"2024-09-21T03:45:55.903388+0000","last_fullsized":"2024-09-21T03:45:55.903388+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_clean_scrub_stamp":"2024-09-21T03:44:13.292303+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:29:17.619130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"77'2","reported_seq":25,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:44:27.611241+0000","last_change":"2024-09-21T03:44:12.307635+0000","last_active":"2024-09-21T03:44:27.611241+0000","last_peered":"2024-09-21T03:44:27.611241+0000","last_clean":"2024-09-21T03:44:27.611241+0000","last_became_active":"2024-09-21T03:44:12.307235+0000","last_became_peered":"2024-09-21T03:44:12.307235+0000","last_unstale":"2024-09-21T03:44:27.611241+0000","last_undegraded":"2024-09-21T03:44:27.611241+0000","last_fullsized":"2024-09-21T03:44:27.611241+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:11.282438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:11.282438+0000","last_clean_scrub_stamp":"2024-09-21T03:44:11.282438+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:18:59.985369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"79'5","reported_seq":23,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:45:55.881464+0000","last_change":"2024-09-21T03:44:14.324660+0000","last_active":"2024-09-21T03:45:55.881464+0000","last_peered":"2024-09-21T03:45:55.881464+0000","last_clean":"2024-09-21T03:45:55.881464+0000","last_became_active":"2024-09-21T03:44:14.324309+0000","last_became_peered":"2024-09-21T03:44:14.324309+0000","last_unstale":"2024-09-21T03:45:55.881464+0000","last_undegraded":"2024-09-21T03:45:55.881464+0000","last_fullsized":"2024-09-21T03:45:55.881464+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_clean_scrub_stamp":"2024-09-21T03:44:13.292303+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:13:07.806721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"79'14","reported_seq":163,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:46:08.761874+0000","last_change":"2024-09-21T03:44:27.623910+0000","last_active":"2024-09-21T03:46:08.761874+0000","last_peered":"2024-09-21T03:46:08.761874+0000","last_clean":"2024-09-21T03:46:08.761874+0000","last_became_active":"2024-09-21T03:44:27.614037+0000","last_became_peered":"2024-09-21T03:44:27.614037+0000","last_unstale":"2024-09-21T03:46:08.761874+0000","last_undegraded":"2024-09-21T03:46:08.761874+0000","last_fullsized":"2024-09-21T03:46:08.761874+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:11.282438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:11.282438+0000","last_clean_scrub_stamp":"2024-09-21T03:44:11.282438+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:03:56.753935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":104,"num_read_kb":104,"num_write":15,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":672,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,4],"acting":[9,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":73176,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":1084,"num_read_kb":622,"num_write":260,"num_write_kb":154,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13,"num_bytes_recovered":1954,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":585728,"data_stored":227183,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":224,"ondisk_log_size":224,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":71,"seq":304942678053,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27888,"kb_used_data":864,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717264,"statfs":{"total":95995035648,"available":95966478336,"internally_reserved":0,"allocated":884736,"data_stored":428272,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":65,"seq":279172874281,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28568,"kb_used_data":1544,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716584,"statfs":{"total":95995035648,"available":95965782016,"internally_reserved":0,"allocated":1581056,"data_stored":1084180,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":60,"seq":257698037806,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27904,"kb_used_data":880,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717248,"statfs":{"total":95995035648,"available":95966461952,"internally_reserved":0,"allocated":901120,"data_stored":428090,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":54,"seq":231928234034,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27884,"kb_used_data":860,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717268,"statfs":{"total":95995035648,"available":95966482432,"internally_reserved":0,"allocated":880640,"data_stored":427684,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":49,"seq":210453397562,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36804,"kb_used_data":1460,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708348,"statfs":{"total":95995035648,"available":95957348352,"internally_reserved":0,"allocated":1495040,"data_stored":1022499,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":36174282},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":43,"seq":184683593790,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27888,"kb_used_data":864,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717264,"statfs":{"total":95995035648,"available":95966478336,"internally_reserved":0,"allocated":884736,"data_stored":428339,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":37,"seq":158913790020,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32656,"kb_used_data":1472,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712496,"statfs":{"total":95995035648,"available":95961595904,"internally_reserved":0,"allocated":1507328,"data_stored":1020805,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":31,"seq":133143986249,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36216,"kb_used_data":872,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708936,"statfs":{"total":95995035648,"available":95957950464,"internally_reserved":0,"allocated":892928,"data_stored":429263,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149774,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32064,"kb_used_data":880,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713088,"statfs":{"total":95995035648,"available":95962202112,"internally_reserved":0,"allocated":901120,"data_stored":427632,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":19,"seq":81604378708,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27964,"kb_used_data":940,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717188,"statfs":{"total":95995035648,"available":95966400512,"internally_reserved":0,"allocated":962560,"data_stored":492761,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542232,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27960,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717192,"statfs":{"total":95995035648,"available":95966404608,"internally_reserved":0,"allocated":958464,"data_stored":495189,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705758,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27908,"kb_used_data":884,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717244,"statfs":{"total":95995035648,"available":95966457856,"internally_reserved":0,"allocated":905216,"data_stored":429146,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2178,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":68221,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":106496,"data_stored":74538,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":2295,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3352,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1254,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":5046,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":716,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":118784,"data_stored":66727,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1304,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T03:46:13.323 INFO:tasks.fwd_scrub.ceph_manager:clean! 2024-09-21T03:46:13.324 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-21T03:46:13.467 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:46:13 smithi161 ceph-mon[34533]: pgmap v330: 97 pgs: 97 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 511 B/s wr, 10 op/s 2024-09-21T03:46:13.474 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:13.471+0000 7f5edbcd5640 1 -- 172.21.15.39:0/555868271 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5ed410dd60 msgr2=0x7f5ed4110250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:46:13.474 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:13.471+0000 7f5edbcd5640 1 --2- 172.21.15.39:0/555868271 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5ed410dd60 0x7f5ed4110250 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f5ecc009ca0 tx=0x7f5ecc02a8c0 comp rx=0 tx=0).stop 2024-09-21T03:46:13.474 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:13.471+0000 7f5edbcd5640 1 -- 172.21.15.39:0/555868271 shutdown_connections 2024-09-21T03:46:13.474 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:13.471+0000 7f5edbcd5640 1 --2- 172.21.15.39:0/555868271 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5ed410dd60 0x7f5ed4110250 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:13.474 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:13.471+0000 7f5edbcd5640 1 --2- 172.21.15.39:0/555868271 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5ed4105250 0x7f5ed410d790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:13.474 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:13.471+0000 7f5edbcd5640 1 --2- 172.21.15.39:0/555868271 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5ed4104900 0x7f5ed4104d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:13.474 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:13.471+0000 7f5edbcd5640 1 -- 172.21.15.39:0/555868271 >> 172.21.15.39:0/555868271 conn(0x7f5ed407a670 msgr2=0x7f5ed407aa80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:46:13.474 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:13.471+0000 7f5edbcd5640 1 -- 172.21.15.39:0/555868271 shutdown_connections 2024-09-21T03:46:13.474 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:13.471+0000 7f5edbcd5640 1 -- 172.21.15.39:0/555868271 wait complete. 2024-09-21T03:46:13.475 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:46:13 smithi150 ceph-mon[34289]: pgmap v330: 97 pgs: 97 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 511 B/s wr, 10 op/s 2024-09-21T03:46:13.477 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:13.472+0000 7f5edbcd5640 1 Processor -- start 2024-09-21T03:46:13.477 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:13.472+0000 7f5edbcd5640 1 -- start start 2024-09-21T03:46:13.477 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:13.472+0000 7f5edbcd5640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5ed4104900 0x7f5ed419a340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:13.477 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:13.472+0000 7f5edbcd5640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5ed4105250 0x7f5ed419a8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:13.477 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:13.472+0000 7f5edbcd5640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5ed410dd60 0x7f5ed419fb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:13.477 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:13.472+0000 7f5edbcd5640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f5ed41139e0 con 0x7f5ed4105250 2024-09-21T03:46:13.477 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:13.472+0000 7f5edbcd5640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f5ed4113860 con 0x7f5ed4104900 2024-09-21T03:46:13.477 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:13.472+0000 7f5edbcd5640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f5ed4113b60 con 0x7f5ed410dd60 2024-09-21T03:46:13.477 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:13.472+0000 7f5ed9270640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5ed4105250 0x7f5ed419a8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:46:13.477 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:13.472+0000 7f5ed9270640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5ed4105250 0x7f5ed419a8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:43220/0 (socket says 172.21.15.39:43220) 2024-09-21T03:46:13.477 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:13.472+0000 7f5ed9270640 1 -- 172.21.15.39:0/491611480 learned_addr learned my addr 172.21.15.39:0/491611480 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:46:13.477 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:13.472+0000 7f5ed9a71640 1 --2- 172.21.15.39:0/491611480 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5ed4104900 0x7f5ed419a340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:46:13.478 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:13.472+0000 7f5eda272640 1 --2- 172.21.15.39:0/491611480 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5ed410dd60 0x7f5ed419fb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:46:13.478 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:13.472+0000 7f5ed9270640 1 -- 172.21.15.39:0/491611480 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5ed410dd60 msgr2=0x7f5ed419fb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:46:13.478 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:13.472+0000 7f5ed9270640 1 --2- 172.21.15.39:0/491611480 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5ed410dd60 0x7f5ed419fb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:13.478 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:13.472+0000 7f5ed9270640 1 -- 172.21.15.39:0/491611480 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5ed4104900 msgr2=0x7f5ed419a340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:46:13.478 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:13.472+0000 7f5ed9270640 1 --2- 172.21.15.39:0/491611480 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5ed4104900 0x7f5ed419a340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:13.478 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:13.472+0000 7f5ed9270640 1 -- 172.21.15.39:0/491611480 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ed41a0270 con 0x7f5ed4105250 2024-09-21T03:46:13.478 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:13.472+0000 7f5eda272640 1 --2- 172.21.15.39:0/491611480 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5ed410dd60 0x7f5ed419fb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:46:13.478 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:13.473+0000 7f5ed9270640 1 --2- 172.21.15.39:0/491611480 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5ed4105250 0x7f5ed419a8e0 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f5ec400e9c0 tx=0x7f5ec400ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:46:13.478 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:13.473+0000 7f5ecaffd640 1 -- 172.21.15.39:0/491611480 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ec400ccc0 con 0x7f5ed4105250 2024-09-21T03:46:13.478 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:13.473+0000 7f5ecaffd640 1 -- 172.21.15.39:0/491611480 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5ec40054d0 con 0x7f5ed4105250 2024-09-21T03:46:13.478 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:13.473+0000 7f5edbcd5640 1 -- 172.21.15.39:0/491611480 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5ed41a0560 con 0x7f5ed4105250 2024-09-21T03:46:13.478 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:13.473+0000 7f5ecaffd640 1 -- 172.21.15.39:0/491611480 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ec4010430 con 0x7f5ed4105250 2024-09-21T03:46:13.478 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:13.473+0000 7f5edbcd5640 1 -- 172.21.15.39:0/491611480 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5ed41a6cf0 con 0x7f5ed4105250 2024-09-21T03:46:13.478 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:13.474+0000 7f5edbcd5640 1 -- 172.21.15.39:0/491611480 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5ed4104d10 con 0x7f5ed4105250 2024-09-21T03:46:13.479 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:13.475+0000 7f5ecaffd640 1 -- 172.21.15.39:0/491611480 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 24) ==== 154580+0+0 (secure 0 0 0) 0x7f5ec4005000 con 0x7f5ed4105250 2024-09-21T03:46:13.479 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:13.476+0000 7f5ecaffd640 1 --2- 172.21.15.39:0/491611480 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f5ec00b6670 0x7f5ec00b8b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:13.479 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:13.476+0000 7f5ed9a71640 1 --2- 172.21.15.39:0/491611480 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f5ec00b6670 0x7f5ec00b8b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:46:13.479 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:13.476+0000 7f5ecaffd640 1 -- 172.21.15.39:0/491611480 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8484+0+0 (secure 0 0 0) 0x7f5ec40e7ae0 con 0x7f5ed4105250 2024-09-21T03:46:13.480 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:13.477+0000 7f5ed9a71640 1 --2- 172.21.15.39:0/491611480 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f5ec00b6670 0x7f5ec00b8b80 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f5ed0005d00 tx=0x7f5ed000b410 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:46:13.482 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:13.479+0000 7f5ecaffd640 1 -- 172.21.15.39:0/491611480 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f5ec4056710 con 0x7f5ed4105250 2024-09-21T03:46:13.668 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:13.666+0000 7f5edbcd5640 1 -- 172.21.15.39:0/491611480 --> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f5ed4100b50 con 0x7f5ec00b6670 2024-09-21T03:46:13.673 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:13.670+0000 7f5ecaffd640 1 -- 172.21.15.39:0/491611480 <== mgr.14223 v2:172.21.15.39:6800/1696381982 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258883 (secure 0 0 0) 0x7f5ed4100b50 con 0x7f5ec00b6670 2024-09-21T03:46:13.675 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T03:46:13.676 INFO:teuthology.orchestra.run.smithi039.stderr:dumped all 2024-09-21T03:46:13.678 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:13.673+0000 7f5edbcd5640 1 -- 172.21.15.39:0/491611480 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f5ec00b6670 msgr2=0x7f5ec00b8b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:46:13.678 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:13.673+0000 7f5edbcd5640 1 --2- 172.21.15.39:0/491611480 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f5ec00b6670 0x7f5ec00b8b80 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f5ed0005d00 tx=0x7f5ed000b410 comp rx=0 tx=0).stop 2024-09-21T03:46:13.678 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:13.673+0000 7f5edbcd5640 1 -- 172.21.15.39:0/491611480 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5ed4105250 msgr2=0x7f5ed419a8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:46:13.678 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:13.673+0000 7f5edbcd5640 1 --2- 172.21.15.39:0/491611480 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5ed4105250 0x7f5ed419a8e0 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f5ec400e9c0 tx=0x7f5ec400ee80 comp rx=0 tx=0).stop 2024-09-21T03:46:13.678 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:13.673+0000 7f5edbcd5640 1 -- 172.21.15.39:0/491611480 shutdown_connections 2024-09-21T03:46:13.678 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:13.674+0000 7f5edbcd5640 1 --2- 172.21.15.39:0/491611480 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f5ec00b6670 0x7f5ec00b8b80 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:13.678 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:13.674+0000 7f5edbcd5640 1 --2- 172.21.15.39:0/491611480 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5ed410dd60 0x7f5ed419fb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:13.678 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:13.674+0000 7f5edbcd5640 1 --2- 172.21.15.39:0/491611480 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5ed4105250 0x7f5ed419a8e0 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:13.678 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:13.674+0000 7f5edbcd5640 1 --2- 172.21.15.39:0/491611480 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5ed4104900 0x7f5ed419a340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:13.679 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:13.674+0000 7f5edbcd5640 1 -- 172.21.15.39:0/491611480 >> 172.21.15.39:0/491611480 conn(0x7f5ed407a670 msgr2=0x7f5ed40ff430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:46:13.679 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:13.674+0000 7f5edbcd5640 1 -- 172.21.15.39:0/491611480 shutdown_connections 2024-09-21T03:46:13.679 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:13.674+0000 7f5edbcd5640 1 -- 172.21.15.39:0/491611480 wait complete. 2024-09-21T03:46:13.687 INFO:teuthology.orchestra.run.smithi039.stdout:{"pg_ready":true,"pg_map":{"version":331,"stamp":"2024-09-21T03:46:13.530003+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":663778,"num_objects":102,"num_object_clones":0,"num_object_copies":306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":1170,"num_read_kb":696,"num_write":387,"num_write_kb":2307,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":23,"num_bytes_recovered":2953794,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":819,"ondisk_log_size":819,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":361704,"kb_used_data":12456,"kb_used_omap":18,"kb_used_meta":349037,"kb_avail":1124580120,"statfs":{"total":1151940427776,"available":1151570042880,"internally_reserved":0,"allocated":12754944,"data_stored":7114559,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19143,"internal_metadata":357414201},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":1007,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":108,"num_read_kb":58,"num_write":7,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003544"},"pg_stats":[{"pgid":"3.3f","version":"79'8","reported_seq":26,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:45:55.900065+0000","last_change":"2024-09-21T03:44:14.327553+0000","last_active":"2024-09-21T03:45:55.900065+0000","last_peered":"2024-09-21T03:45:55.900065+0000","last_clean":"2024-09-21T03:45:55.900065+0000","last_became_active":"2024-09-21T03:44:14.327213+0000","last_became_peered":"2024-09-21T03:44:14.327213+0000","last_unstale":"2024-09-21T03:45:55.900065+0000","last_undegraded":"2024-09-21T03:45:55.900065+0000","last_fullsized":"2024-09-21T03:45:55.900065+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_clean_scrub_stamp":"2024-09-21T03:44:13.292303+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:40:56.550897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"79'4","reported_seq":22,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:45:55.902384+0000","last_change":"2024-09-21T03:44:14.321427+0000","last_active":"2024-09-21T03:45:55.902384+0000","last_peered":"2024-09-21T03:45:55.902384+0000","last_clean":"2024-09-21T03:45:55.902384+0000","last_became_active":"2024-09-21T03:44:14.321026+0000","last_became_peered":"2024-09-21T03:44:14.321026+0000","last_unstale":"2024-09-21T03:45:55.902384+0000","last_undegraded":"2024-09-21T03:45:55.902384+0000","last_fullsized":"2024-09-21T03:45:55.902384+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_clean_scrub_stamp":"2024-09-21T03:44:13.292303+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:18:00.889714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"79'7","reported_seq":25,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:45:55.906660+0000","last_change":"2024-09-21T03:44:14.324908+0000","last_active":"2024-09-21T03:45:55.906660+0000","last_peered":"2024-09-21T03:45:55.906660+0000","last_clean":"2024-09-21T03:45:55.906660+0000","last_became_active":"2024-09-21T03:44:14.324533+0000","last_became_peered":"2024-09-21T03:44:14.324533+0000","last_unstale":"2024-09-21T03:45:55.906660+0000","last_undegraded":"2024-09-21T03:45:55.906660+0000","last_fullsized":"2024-09-21T03:45:55.906660+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_clean_scrub_stamp":"2024-09-21T03:44:13.292303+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:44:25.235442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"79'8","reported_seq":26,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:45:55.885244+0000","last_change":"2024-09-21T03:44:14.326758+0000","last_active":"2024-09-21T03:45:55.885244+0000","last_peered":"2024-09-21T03:45:55.885244+0000","last_clean":"2024-09-21T03:45:55.885244+0000","last_became_active":"2024-09-21T03:44:14.326410+0000","last_became_peered":"2024-09-21T03:44:14.326410+0000","last_unstale":"2024-09-21T03:45:55.885244+0000","last_undegraded":"2024-09-21T03:45:55.885244+0000","last_fullsized":"2024-09-21T03:45:55.885244+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_clean_scrub_stamp":"2024-09-21T03:44:13.292303+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:13:22.716940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"79'4","reported_seq":22,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:45:55.874578+0000","last_change":"2024-09-21T03:44:14.318875+0000","last_active":"2024-09-21T03:45:55.874578+0000","last_peered":"2024-09-21T03:45:55.874578+0000","last_clean":"2024-09-21T03:45:55.874578+0000","last_became_active":"2024-09-21T03:44:14.318425+0000","last_became_peered":"2024-09-21T03:44:14.318425+0000","last_unstale":"2024-09-21T03:45:55.874578+0000","last_undegraded":"2024-09-21T03:45:55.874578+0000","last_fullsized":"2024-09-21T03:45:55.874578+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_clean_scrub_stamp":"2024-09-21T03:44:13.292303+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:45:39.916587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"79'6","reported_seq":24,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:45:55.892089+0000","last_change":"2024-09-21T03:44:14.325877+0000","last_active":"2024-09-21T03:45:55.892089+0000","last_peered":"2024-09-21T03:45:55.892089+0000","last_clean":"2024-09-21T03:45:55.892089+0000","last_became_active":"2024-09-21T03:44:14.325451+0000","last_became_peered":"2024-09-21T03:44:14.325451+0000","last_unstale":"2024-09-21T03:45:55.892089+0000","last_undegraded":"2024-09-21T03:45:55.892089+0000","last_fullsized":"2024-09-21T03:45:55.892089+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_clean_scrub_stamp":"2024-09-21T03:44:13.292303+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:30:42.427054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"79'5","reported_seq":23,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:45:55.888426+0000","last_change":"2024-09-21T03:44:14.323192+0000","last_active":"2024-09-21T03:45:55.888426+0000","last_peered":"2024-09-21T03:45:55.888426+0000","last_clean":"2024-09-21T03:45:55.888426+0000","last_became_active":"2024-09-21T03:44:14.322487+0000","last_became_peered":"2024-09-21T03:44:14.322487+0000","last_unstale":"2024-09-21T03:45:55.888426+0000","last_undegraded":"2024-09-21T03:45:55.888426+0000","last_fullsized":"2024-09-21T03:45:55.888426+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_clean_scrub_stamp":"2024-09-21T03:44:13.292303+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:10:48.770536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"79'9","reported_seq":21,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:45:55.892165+0000","last_change":"2024-09-21T03:44:27.612681+0000","last_active":"2024-09-21T03:45:55.892165+0000","last_peered":"2024-09-21T03:45:55.892165+0000","last_clean":"2024-09-21T03:45:55.892165+0000","last_became_active":"2024-09-21T03:44:27.612200+0000","last_became_peered":"2024-09-21T03:44:27.612200+0000","last_unstale":"2024-09-21T03:45:55.892165+0000","last_undegraded":"2024-09-21T03:45:55.892165+0000","last_fullsized":"2024-09-21T03:45:55.892165+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_clean_scrub_stamp":"2024-09-21T03:44:13.292303+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:44:58.848571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,5,10],"acting":[9,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.37","version":"79'11","reported_seq":29,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:45:55.900520+0000","last_change":"2024-09-21T03:44:14.325820+0000","last_active":"2024-09-21T03:45:55.900520+0000","last_peered":"2024-09-21T03:45:55.900520+0000","last_clean":"2024-09-21T03:45:55.900520+0000","last_became_active":"2024-09-21T03:44:14.325070+0000","last_became_peered":"2024-09-21T03:44:14.325070+0000","last_unstale":"2024-09-21T03:45:55.900520+0000","last_undegraded":"2024-09-21T03:45:55.900520+0000","last_fullsized":"2024-09-21T03:45:55.900520+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_clean_scrub_stamp":"2024-09-21T03:44:13.292303+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:54:01.072623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"79'16","reported_seq":34,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:45:55.894398+0000","last_change":"2024-09-21T03:44:14.323746+0000","last_active":"2024-09-21T03:45:55.894398+0000","last_peered":"2024-09-21T03:45:55.894398+0000","last_clean":"2024-09-21T03:45:55.894398+0000","last_became_active":"2024-09-21T03:44:14.323346+0000","last_became_peered":"2024-09-21T03:44:14.323346+0000","last_unstale":"2024-09-21T03:45:55.894398+0000","last_undegraded":"2024-09-21T03:45:55.894398+0000","last_fullsized":"2024-09-21T03:45:55.894398+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_clean_scrub_stamp":"2024-09-21T03:44:13.292303+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:17:44.603801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"79'6","reported_seq":24,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:45:55.896278+0000","last_change":"2024-09-21T03:44:14.328733+0000","last_active":"2024-09-21T03:45:55.896278+0000","last_peered":"2024-09-21T03:45:55.896278+0000","last_clean":"2024-09-21T03:45:55.896278+0000","last_became_active":"2024-09-21T03:44:14.328378+0000","last_became_peered":"2024-09-21T03:44:14.328378+0000","last_unstale":"2024-09-21T03:45:55.896278+0000","last_undegraded":"2024-09-21T03:45:55.896278+0000","last_fullsized":"2024-09-21T03:45:55.896278+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_clean_scrub_stamp":"2024-09-21T03:44:13.292303+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:23:38.497413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"79'7","reported_seq":25,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:45:55.895287+0000","last_change":"2024-09-21T03:44:14.328505+0000","last_active":"2024-09-21T03:45:55.895287+0000","last_peered":"2024-09-21T03:45:55.895287+0000","last_clean":"2024-09-21T03:45:55.895287+0000","last_became_active":"2024-09-21T03:44:14.328175+0000","last_became_peered":"2024-09-21T03:44:14.328175+0000","last_unstale":"2024-09-21T03:45:55.895287+0000","last_undegraded":"2024-09-21T03:45:55.895287+0000","last_fullsized":"2024-09-21T03:45:55.895287+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_clean_scrub_stamp":"2024-09-21T03:44:13.292303+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:45:56.628962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"79'10","reported_seq":28,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:45:55.903680+0000","last_change":"2024-09-21T03:44:14.314519+0000","last_active":"2024-09-21T03:45:55.903680+0000","last_peered":"2024-09-21T03:45:55.903680+0000","last_clean":"2024-09-21T03:45:55.903680+0000","last_became_active":"2024-09-21T03:44:14.314145+0000","last_became_peered":"2024-09-21T03:44:14.314145+0000","last_unstale":"2024-09-21T03:45:55.903680+0000","last_undegraded":"2024-09-21T03:45:55.903680+0000","last_fullsized":"2024-09-21T03:45:55.903680+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_clean_scrub_stamp":"2024-09-21T03:44:13.292303+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:45:48.494667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"79'7","reported_seq":25,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:45:55.902926+0000","last_change":"2024-09-21T03:44:14.312195+0000","last_active":"2024-09-21T03:45:55.902926+0000","last_peered":"2024-09-21T03:45:55.902926+0000","last_clean":"2024-09-21T03:45:55.902926+0000","last_became_active":"2024-09-21T03:44:14.311861+0000","last_became_peered":"2024-09-21T03:44:14.311861+0000","last_unstale":"2024-09-21T03:45:55.902926+0000","last_undegraded":"2024-09-21T03:45:55.902926+0000","last_fullsized":"2024-09-21T03:45:55.902926+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_clean_scrub_stamp":"2024-09-21T03:44:13.292303+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:17:05.025898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"79'7","reported_seq":25,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:45:55.905677+0000","last_change":"2024-09-21T03:44:14.323532+0000","last_active":"2024-09-21T03:45:55.905677+0000","last_peered":"2024-09-21T03:45:55.905677+0000","last_clean":"2024-09-21T03:45:55.905677+0000","last_became_active":"2024-09-21T03:44:14.323171+0000","last_became_peered":"2024-09-21T03:44:14.323171+0000","last_unstale":"2024-09-21T03:45:55.905677+0000","last_undegraded":"2024-09-21T03:45:55.905677+0000","last_fullsized":"2024-09-21T03:45:55.905677+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_clean_scrub_stamp":"2024-09-21T03:44:13.292303+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:23:28.999129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"79'9","reported_seq":27,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:45:55.903498+0000","last_change":"2024-09-21T03:44:14.324998+0000","last_active":"2024-09-21T03:45:55.903498+0000","last_peered":"2024-09-21T03:45:55.903498+0000","last_clean":"2024-09-21T03:45:55.903498+0000","last_became_active":"2024-09-21T03:44:14.324019+0000","last_became_peered":"2024-09-21T03:44:14.324019+0000","last_unstale":"2024-09-21T03:45:55.903498+0000","last_undegraded":"2024-09-21T03:45:55.903498+0000","last_fullsized":"2024-09-21T03:45:55.903498+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_clean_scrub_stamp":"2024-09-21T03:44:13.292303+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:49:12.965299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"79'8","reported_seq":26,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:45:55.900363+0000","last_change":"2024-09-21T03:44:14.325603+0000","last_active":"2024-09-21T03:45:55.900363+0000","last_peered":"2024-09-21T03:45:55.900363+0000","last_clean":"2024-09-21T03:45:55.900363+0000","last_became_active":"2024-09-21T03:44:14.325208+0000","last_became_peered":"2024-09-21T03:44:14.325208+0000","last_unstale":"2024-09-21T03:45:55.900363+0000","last_undegraded":"2024-09-21T03:45:55.900363+0000","last_fullsized":"2024-09-21T03:45:55.900363+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_clean_scrub_stamp":"2024-09-21T03:44:13.292303+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:11:33.794242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"79'10","reported_seq":28,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:45:55.891232+0000","last_change":"2024-09-21T03:44:14.317545+0000","last_active":"2024-09-21T03:45:55.891232+0000","last_peered":"2024-09-21T03:45:55.891232+0000","last_clean":"2024-09-21T03:45:55.891232+0000","last_became_active":"2024-09-21T03:44:14.317069+0000","last_became_peered":"2024-09-21T03:44:14.317069+0000","last_unstale":"2024-09-21T03:45:55.891232+0000","last_undegraded":"2024-09-21T03:45:55.891232+0000","last_fullsized":"2024-09-21T03:45:55.891232+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_clean_scrub_stamp":"2024-09-21T03:44:13.292303+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:16:11.734887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"79'7","reported_seq":25,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:45:55.903043+0000","last_change":"2024-09-21T03:44:14.322055+0000","last_active":"2024-09-21T03:45:55.903043+0000","last_peered":"2024-09-21T03:45:55.903043+0000","last_clean":"2024-09-21T03:45:55.903043+0000","last_became_active":"2024-09-21T03:44:14.321652+0000","last_became_peered":"2024-09-21T03:44:14.321652+0000","last_unstale":"2024-09-21T03:45:55.903043+0000","last_undegraded":"2024-09-21T03:45:55.903043+0000","last_fullsized":"2024-09-21T03:45:55.903043+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_clean_scrub_stamp":"2024-09-21T03:44:13.292303+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:45:14.285844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2c","version":"79'8","reported_seq":33,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:45:55.896619+0000","last_change":"2024-09-21T03:44:27.423175+0000","last_active":"2024-09-21T03:45:55.896619+0000","last_peered":"2024-09-21T03:45:55.896619+0000","last_clean":"2024-09-21T03:45:55.896619+0000","last_became_active":"2024-09-21T03:44:27.422733+0000","last_became_peered":"2024-09-21T03:44:27.422733+0000","last_unstale":"2024-09-21T03:45:55.896619+0000","last_undegraded":"2024-09-21T03:45:55.896619+0000","last_fullsized":"2024-09-21T03:45:55.896619+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_clean_scrub_stamp":"2024-09-21T03:44:13.292303+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:46:17.874557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,8],"acting":[11,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2b","version":"79'11","reported_seq":29,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:45:55.903246+0000","last_change":"2024-09-21T03:44:14.325829+0000","last_active":"2024-09-21T03:45:55.903246+0000","last_peered":"2024-09-21T03:45:55.903246+0000","last_clean":"2024-09-21T03:45:55.903246+0000","last_became_active":"2024-09-21T03:44:14.325421+0000","last_became_peered":"2024-09-21T03:44:14.325421+0000","last_unstale":"2024-09-21T03:45:55.903246+0000","last_undegraded":"2024-09-21T03:45:55.903246+0000","last_fullsized":"2024-09-21T03:45:55.903246+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_clean_scrub_stamp":"2024-09-21T03:44:13.292303+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:49:00.240771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2a","version":"79'10","reported_seq":28,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:45:55.907357+0000","last_change":"2024-09-21T03:44:14.325255+0000","last_active":"2024-09-21T03:45:55.907357+0000","last_peered":"2024-09-21T03:45:55.907357+0000","last_clean":"2024-09-21T03:45:55.907357+0000","last_became_active":"2024-09-21T03:44:14.324617+0000","last_became_peered":"2024-09-21T03:44:14.324617+0000","last_unstale":"2024-09-21T03:45:55.907357+0000","last_undegraded":"2024-09-21T03:45:55.907357+0000","last_fullsized":"2024-09-21T03:45:55.907357+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_clean_scrub_stamp":"2024-09-21T03:44:13.292303+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:05:03.268522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"79'8","reported_seq":26,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:45:55.885845+0000","last_change":"2024-09-21T03:44:14.328053+0000","last_active":"2024-09-21T03:45:55.885845+0000","last_peered":"2024-09-21T03:45:55.885845+0000","last_clean":"2024-09-21T03:45:55.885845+0000","last_became_active":"2024-09-21T03:44:14.327735+0000","last_became_peered":"2024-09-21T03:44:14.327735+0000","last_unstale":"2024-09-21T03:45:55.885845+0000","last_undegraded":"2024-09-21T03:45:55.885845+0000","last_fullsized":"2024-09-21T03:45:55.885845+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_clean_scrub_stamp":"2024-09-21T03:44:13.292303+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:39:48.395784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"79'6","reported_seq":24,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:45:55.888696+0000","last_change":"2024-09-21T03:44:14.321181+0000","last_active":"2024-09-21T03:45:55.888696+0000","last_peered":"2024-09-21T03:45:55.888696+0000","last_clean":"2024-09-21T03:45:55.888696+0000","last_became_active":"2024-09-21T03:44:14.320772+0000","last_became_peered":"2024-09-21T03:44:14.320772+0000","last_unstale":"2024-09-21T03:45:55.888696+0000","last_undegraded":"2024-09-21T03:45:55.888696+0000","last_fullsized":"2024-09-21T03:45:55.888696+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_clean_scrub_stamp":"2024-09-21T03:44:13.292303+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:49:26.317404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"79'2","reported_seq":20,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:45:55.884461+0000","last_change":"2024-09-21T03:44:14.322643+0000","last_active":"2024-09-21T03:45:55.884461+0000","last_peered":"2024-09-21T03:45:55.884461+0000","last_clean":"2024-09-21T03:45:55.884461+0000","last_became_active":"2024-09-21T03:44:14.322254+0000","last_became_peered":"2024-09-21T03:44:14.322254+0000","last_unstale":"2024-09-21T03:45:55.884461+0000","last_undegraded":"2024-09-21T03:45:55.884461+0000","last_fullsized":"2024-09-21T03:45:55.884461+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_clean_scrub_stamp":"2024-09-21T03:44:13.292303+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:31:47.147845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"79'15","reported_seq":33,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:45:55.907955+0000","last_change":"2024-09-21T03:44:14.323090+0000","last_active":"2024-09-21T03:45:55.907955+0000","last_peered":"2024-09-21T03:45:55.907955+0000","last_clean":"2024-09-21T03:45:55.907955+0000","last_became_active":"2024-09-21T03:44:14.322669+0000","last_became_peered":"2024-09-21T03:44:14.322669+0000","last_unstale":"2024-09-21T03:45:55.907955+0000","last_undegraded":"2024-09-21T03:45:55.907955+0000","last_fullsized":"2024-09-21T03:45:55.907955+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_clean_scrub_stamp":"2024-09-21T03:44:13.292303+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:08:56.852422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"79'6","reported_seq":24,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:45:55.896312+0000","last_change":"2024-09-21T03:44:14.318146+0000","last_active":"2024-09-21T03:45:55.896312+0000","last_peered":"2024-09-21T03:45:55.896312+0000","last_clean":"2024-09-21T03:45:55.896312+0000","last_became_active":"2024-09-21T03:44:14.317761+0000","last_became_peered":"2024-09-21T03:44:14.317761+0000","last_unstale":"2024-09-21T03:45:55.896312+0000","last_undegraded":"2024-09-21T03:45:55.896312+0000","last_fullsized":"2024-09-21T03:45:55.896312+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_clean_scrub_stamp":"2024-09-21T03:44:13.292303+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:07:31.530728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.24","version":"79'4","reported_seq":22,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:45:55.902467+0000","last_change":"2024-09-21T03:44:14.316832+0000","last_active":"2024-09-21T03:45:55.902467+0000","last_peered":"2024-09-21T03:45:55.902467+0000","last_clean":"2024-09-21T03:45:55.902467+0000","last_became_active":"2024-09-21T03:44:14.316471+0000","last_became_peered":"2024-09-21T03:44:14.316471+0000","last_unstale":"2024-09-21T03:45:55.902467+0000","last_undegraded":"2024-09-21T03:45:55.902467+0000","last_fullsized":"2024-09-21T03:45:55.902467+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_clean_scrub_stamp":"2024-09-21T03:44:13.292303+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:21:23.848778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"79'7","reported_seq":25,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:45:55.905316+0000","last_change":"2024-09-21T03:44:14.322277+0000","last_active":"2024-09-21T03:45:55.905316+0000","last_peered":"2024-09-21T03:45:55.905316+0000","last_clean":"2024-09-21T03:45:55.905316+0000","last_became_active":"2024-09-21T03:44:14.321952+0000","last_became_peered":"2024-09-21T03:44:14.321952+0000","last_unstale":"2024-09-21T03:45:55.905316+0000","last_undegraded":"2024-09-21T03:45:55.905316+0000","last_fullsized":"2024-09-21T03:45:55.905316+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_clean_scrub_stamp":"2024-09-21T03:44:13.292303+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:43:55.169380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"79'8","reported_seq":26,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:45:55.886351+0000","last_change":"2024-09-21T03:44:14.322626+0000","last_active":"2024-09-21T03:45:55.886351+0000","last_peered":"2024-09-21T03:45:55.886351+0000","last_clean":"2024-09-21T03:45:55.886351+0000","last_became_active":"2024-09-21T03:44:14.321559+0000","last_became_peered":"2024-09-21T03:44:14.321559+0000","last_unstale":"2024-09-21T03:45:55.886351+0000","last_undegraded":"2024-09-21T03:45:55.886351+0000","last_fullsized":"2024-09-21T03:45:55.886351+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_clean_scrub_stamp":"2024-09-21T03:44:13.292303+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:12:23.736500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"79'12","reported_seq":30,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:45:55.886765+0000","last_change":"2024-09-21T03:44:14.318057+0000","last_active":"2024-09-21T03:45:55.886765+0000","last_peered":"2024-09-21T03:45:55.886765+0000","last_clean":"2024-09-21T03:45:55.886765+0000","last_became_active":"2024-09-21T03:44:14.317653+0000","last_became_peered":"2024-09-21T03:44:14.317653+0000","last_unstale":"2024-09-21T03:45:55.886765+0000","last_undegraded":"2024-09-21T03:45:55.886765+0000","last_fullsized":"2024-09-21T03:45:55.886765+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_clean_scrub_stamp":"2024-09-21T03:44:13.292303+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:23:17.396599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"79'9","reported_seq":27,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:45:55.895204+0000","last_change":"2024-09-21T03:44:14.322331+0000","last_active":"2024-09-21T03:45:55.895204+0000","last_peered":"2024-09-21T03:45:55.895204+0000","last_clean":"2024-09-21T03:45:55.895204+0000","last_became_active":"2024-09-21T03:44:14.321157+0000","last_became_peered":"2024-09-21T03:44:14.321157+0000","last_unstale":"2024-09-21T03:45:55.895204+0000","last_undegraded":"2024-09-21T03:45:55.895204+0000","last_fullsized":"2024-09-21T03:45:55.895204+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_clean_scrub_stamp":"2024-09-21T03:44:13.292303+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:08:27.953449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1a","version":"79'7","reported_seq":32,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:45:55.890546+0000","last_change":"2024-09-21T03:44:27.423081+0000","last_active":"2024-09-21T03:45:55.890546+0000","last_peered":"2024-09-21T03:45:55.890546+0000","last_clean":"2024-09-21T03:45:55.890546+0000","last_became_active":"2024-09-21T03:44:27.422650+0000","last_became_peered":"2024-09-21T03:44:27.422650+0000","last_unstale":"2024-09-21T03:45:55.890546+0000","last_undegraded":"2024-09-21T03:45:55.890546+0000","last_fullsized":"2024-09-21T03:45:55.890546+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_clean_scrub_stamp":"2024-09-21T03:44:13.292303+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:32:44.399954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,7],"acting":[8,9,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1b","version":"79'7","reported_seq":139,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:46:07.857454+0000","last_change":"2024-09-21T03:44:12.307464+0000","last_active":"2024-09-21T03:46:07.857454+0000","last_peered":"2024-09-21T03:46:07.857454+0000","last_clean":"2024-09-21T03:46:07.857454+0000","last_became_active":"2024-09-21T03:44:12.307077+0000","last_became_peered":"2024-09-21T03:44:12.307077+0000","last_unstale":"2024-09-21T03:46:07.857454+0000","last_undegraded":"2024-09-21T03:46:07.857454+0000","last_fullsized":"2024-09-21T03:46:07.857454+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:11.282438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:11.282438+0000","last_clean_scrub_stamp":"2024-09-21T03:44:11.282438+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:50:40.344659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1154,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":109,"num_read_kb":4,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"79'8","reported_seq":26,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:45:55.891478+0000","last_change":"2024-09-21T03:44:14.319072+0000","last_active":"2024-09-21T03:45:55.891478+0000","last_peered":"2024-09-21T03:45:55.891478+0000","last_clean":"2024-09-21T03:45:55.891478+0000","last_became_active":"2024-09-21T03:44:14.318712+0000","last_became_peered":"2024-09-21T03:44:14.318712+0000","last_unstale":"2024-09-21T03:45:55.891478+0000","last_undegraded":"2024-09-21T03:45:55.891478+0000","last_fullsized":"2024-09-21T03:45:55.891478+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_clean_scrub_stamp":"2024-09-21T03:44:13.292303+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:30:35.618110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1a","version":"77'7","reported_seq":30,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:44:27.611763+0000","last_change":"2024-09-21T03:44:12.310146+0000","last_active":"2024-09-21T03:44:27.611763+0000","last_peered":"2024-09-21T03:44:27.611763+0000","last_clean":"2024-09-21T03:44:27.611763+0000","last_became_active":"2024-09-21T03:44:12.309817+0000","last_became_peered":"2024-09-21T03:44:12.309817+0000","last_unstale":"2024-09-21T03:44:27.611763+0000","last_undegraded":"2024-09-21T03:44:27.611763+0000","last_fullsized":"2024-09-21T03:44:27.611763+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:11.282438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:11.282438+0000","last_clean_scrub_stamp":"2024-09-21T03:44:11.282438+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:46:51.994488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.19","version":"77'3","reported_seq":26,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:44:27.410211+0000","last_change":"2024-09-21T03:44:12.308995+0000","last_active":"2024-09-21T03:44:27.410211+0000","last_peered":"2024-09-21T03:44:27.410211+0000","last_clean":"2024-09-21T03:44:27.410211+0000","last_became_active":"2024-09-21T03:44:12.308058+0000","last_became_peered":"2024-09-21T03:44:12.308058+0000","last_unstale":"2024-09-21T03:44:27.410211+0000","last_undegraded":"2024-09-21T03:44:27.410211+0000","last_fullsized":"2024-09-21T03:44:27.410211+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:11.282438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:11.282438+0000","last_clean_scrub_stamp":"2024-09-21T03:44:11.282438+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:48:39.384985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"79'8","reported_seq":26,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:45:55.886620+0000","last_change":"2024-09-21T03:44:14.317878+0000","last_active":"2024-09-21T03:45:55.886620+0000","last_peered":"2024-09-21T03:45:55.886620+0000","last_clean":"2024-09-21T03:45:55.886620+0000","last_became_active":"2024-09-21T03:44:14.317521+0000","last_became_peered":"2024-09-21T03:44:14.317521+0000","last_unstale":"2024-09-21T03:45:55.886620+0000","last_undegraded":"2024-09-21T03:45:55.886620+0000","last_fullsized":"2024-09-21T03:45:55.886620+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_clean_scrub_stamp":"2024-09-21T03:44:13.292303+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:08:00.705210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"77'3","reported_seq":27,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:44:27.414041+0000","last_change":"2024-09-21T03:44:12.308777+0000","last_active":"2024-09-21T03:44:27.414041+0000","last_peered":"2024-09-21T03:44:27.414041+0000","last_clean":"2024-09-21T03:44:27.414041+0000","last_became_active":"2024-09-21T03:44:12.308171+0000","last_became_peered":"2024-09-21T03:44:12.308171+0000","last_unstale":"2024-09-21T03:44:27.414041+0000","last_undegraded":"2024-09-21T03:44:27.414041+0000","last_fullsized":"2024-09-21T03:44:27.414041+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:11.282438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:11.282438+0000","last_clean_scrub_stamp":"2024-09-21T03:44:11.282438+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:41:21.344491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"79'11","reported_seq":29,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:45:55.892137+0000","last_change":"2024-09-21T03:44:14.315782+0000","last_active":"2024-09-21T03:45:55.892137+0000","last_peered":"2024-09-21T03:45:55.892137+0000","last_clean":"2024-09-21T03:45:55.892137+0000","last_became_active":"2024-09-21T03:44:14.315314+0000","last_became_peered":"2024-09-21T03:44:14.315314+0000","last_unstale":"2024-09-21T03:45:55.892137+0000","last_undegraded":"2024-09-21T03:45:55.892137+0000","last_fullsized":"2024-09-21T03:45:55.892137+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_clean_scrub_stamp":"2024-09-21T03:44:13.292303+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:22:20.500220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"79'5","reported_seq":236,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:46:07.856800+0000","last_change":"2024-09-21T03:44:12.313251+0000","last_active":"2024-09-21T03:46:07.856800+0000","last_peered":"2024-09-21T03:46:07.856800+0000","last_clean":"2024-09-21T03:46:07.856800+0000","last_became_active":"2024-09-21T03:44:12.312941+0000","last_became_peered":"2024-09-21T03:44:12.312941+0000","last_unstale":"2024-09-21T03:46:07.856800+0000","last_undegraded":"2024-09-21T03:46:07.856800+0000","last_fullsized":"2024-09-21T03:46:07.856800+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:11.282438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:11.282438+0000","last_clean_scrub_stamp":"2024-09-21T03:44:11.282438+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:38:39.876097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":208,"num_read_kb":208,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"79'10","reported_seq":28,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:45:55.905106+0000","last_change":"2024-09-21T03:44:14.319621+0000","last_active":"2024-09-21T03:45:55.905106+0000","last_peered":"2024-09-21T03:45:55.905106+0000","last_clean":"2024-09-21T03:45:55.905106+0000","last_became_active":"2024-09-21T03:44:14.316173+0000","last_became_peered":"2024-09-21T03:44:14.316173+0000","last_unstale":"2024-09-21T03:45:55.905106+0000","last_undegraded":"2024-09-21T03:45:55.905106+0000","last_fullsized":"2024-09-21T03:45:55.905106+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_clean_scrub_stamp":"2024-09-21T03:44:13.292303+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:10:21.952989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"77'4","reported_seq":27,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:44:27.419175+0000","last_change":"2024-09-21T03:44:12.308893+0000","last_active":"2024-09-21T03:44:27.419175+0000","last_peered":"2024-09-21T03:44:27.419175+0000","last_clean":"2024-09-21T03:44:27.419175+0000","last_became_active":"2024-09-21T03:44:12.308551+0000","last_became_peered":"2024-09-21T03:44:12.308551+0000","last_unstale":"2024-09-21T03:44:27.419175+0000","last_undegraded":"2024-09-21T03:44:27.419175+0000","last_fullsized":"2024-09-21T03:44:27.419175+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:11.282438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:11.282438+0000","last_clean_scrub_stamp":"2024-09-21T03:44:11.282438+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:48:18.401595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"79'9","reported_seq":27,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:45:55.904971+0000","last_change":"2024-09-21T03:44:14.321875+0000","last_active":"2024-09-21T03:45:55.904971+0000","last_peered":"2024-09-21T03:45:55.904971+0000","last_clean":"2024-09-21T03:45:55.904971+0000","last_became_active":"2024-09-21T03:44:14.321483+0000","last_became_peered":"2024-09-21T03:44:14.321483+0000","last_unstale":"2024-09-21T03:45:55.904971+0000","last_undegraded":"2024-09-21T03:45:55.904971+0000","last_fullsized":"2024-09-21T03:45:55.904971+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_clean_scrub_stamp":"2024-09-21T03:44:13.292303+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:40:17.244780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"77'5","reported_seq":29,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:44:27.414461+0000","last_change":"2024-09-21T03:44:12.313208+0000","last_active":"2024-09-21T03:44:27.414461+0000","last_peered":"2024-09-21T03:44:27.414461+0000","last_clean":"2024-09-21T03:44:27.414461+0000","last_became_active":"2024-09-21T03:44:12.312783+0000","last_became_peered":"2024-09-21T03:44:12.312783+0000","last_unstale":"2024-09-21T03:44:27.414461+0000","last_undegraded":"2024-09-21T03:44:27.414461+0000","last_fullsized":"2024-09-21T03:44:27.414461+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:11.282438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:11.282438+0000","last_clean_scrub_stamp":"2024-09-21T03:44:11.282438+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:18:45.114361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"79'11","reported_seq":36,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:45:55.907006+0000","last_change":"2024-09-21T03:44:27.421129+0000","last_active":"2024-09-21T03:45:55.907006+0000","last_peered":"2024-09-21T03:45:55.907006+0000","last_clean":"2024-09-21T03:45:55.907006+0000","last_became_active":"2024-09-21T03:44:27.420642+0000","last_became_peered":"2024-09-21T03:44:27.420642+0000","last_unstale":"2024-09-21T03:45:55.907006+0000","last_undegraded":"2024-09-21T03:45:55.907006+0000","last_fullsized":"2024-09-21T03:45:55.907006+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_clean_scrub_stamp":"2024-09-21T03:44:13.292303+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:49:38.065743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,6],"acting":[1,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"79'34","reported_seq":277,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:46:07.917025+0000","last_change":"2024-09-21T03:44:12.314863+0000","last_active":"2024-09-21T03:46:07.917025+0000","last_peered":"2024-09-21T03:46:07.917025+0000","last_clean":"2024-09-21T03:46:07.917025+0000","last_became_active":"2024-09-21T03:44:12.314471+0000","last_became_peered":"2024-09-21T03:44:12.314471+0000","last_unstale":"2024-09-21T03:46:07.917025+0000","last_undegraded":"2024-09-21T03:46:07.917025+0000","last_fullsized":"2024-09-21T03:46:07.917025+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:11.282438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:11.282438+0000","last_clean_scrub_stamp":"2024-09-21T03:44:11.282438+0000","objects_scrubbed":0,"log_size":34,"log_dups_size":0,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:41:03.471226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":64459,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":220,"num_read_kb":173,"num_write":36,"num_write_kb":79,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"79'6","reported_seq":24,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:45:55.897616+0000","last_change":"2024-09-21T03:44:14.325471+0000","last_active":"2024-09-21T03:45:55.897616+0000","last_peered":"2024-09-21T03:45:55.897616+0000","last_clean":"2024-09-21T03:45:55.897616+0000","last_became_active":"2024-09-21T03:44:14.325072+0000","last_became_peered":"2024-09-21T03:44:14.325072+0000","last_unstale":"2024-09-21T03:45:55.897616+0000","last_undegraded":"2024-09-21T03:45:55.897616+0000","last_fullsized":"2024-09-21T03:45:55.897616+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_clean_scrub_stamp":"2024-09-21T03:44:13.292303+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:40:40.097240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"77'9","reported_seq":34,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:44:27.416778+0000","last_change":"2024-09-21T03:44:12.312932+0000","last_active":"2024-09-21T03:44:27.416778+0000","last_peered":"2024-09-21T03:44:27.416778+0000","last_clean":"2024-09-21T03:44:27.416778+0000","last_became_active":"2024-09-21T03:44:12.312368+0000","last_became_peered":"2024-09-21T03:44:12.312368+0000","last_unstale":"2024-09-21T03:44:27.416778+0000","last_undegraded":"2024-09-21T03:44:27.416778+0000","last_fullsized":"2024-09-21T03:44:27.416778+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:11.282438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:11.282438+0000","last_clean_scrub_stamp":"2024-09-21T03:44:11.282438+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:28:11.579598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"79'9","reported_seq":27,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:45:55.901852+0000","last_change":"2024-09-21T03:44:14.329564+0000","last_active":"2024-09-21T03:45:55.901852+0000","last_peered":"2024-09-21T03:45:55.901852+0000","last_clean":"2024-09-21T03:45:55.901852+0000","last_became_active":"2024-09-21T03:44:14.329122+0000","last_became_peered":"2024-09-21T03:44:14.329122+0000","last_unstale":"2024-09-21T03:45:55.901852+0000","last_undegraded":"2024-09-21T03:45:55.901852+0000","last_fullsized":"2024-09-21T03:45:55.901852+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_clean_scrub_stamp":"2024-09-21T03:44:13.292303+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:16:32.702770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"77'5","reported_seq":29,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:44:27.413875+0000","last_change":"2024-09-21T03:44:12.303339+0000","last_active":"2024-09-21T03:44:27.413875+0000","last_peered":"2024-09-21T03:44:27.413875+0000","last_clean":"2024-09-21T03:44:27.413875+0000","last_became_active":"2024-09-21T03:44:12.302905+0000","last_became_peered":"2024-09-21T03:44:12.302905+0000","last_unstale":"2024-09-21T03:44:27.413875+0000","last_undegraded":"2024-09-21T03:44:27.413875+0000","last_fullsized":"2024-09-21T03:44:27.413875+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:11.282438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:11.282438+0000","last_clean_scrub_stamp":"2024-09-21T03:44:11.282438+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:57:16.424976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"79'15","reported_seq":33,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:45:55.905704+0000","last_change":"2024-09-21T03:44:14.323473+0000","last_active":"2024-09-21T03:45:55.905704+0000","last_peered":"2024-09-21T03:45:55.905704+0000","last_clean":"2024-09-21T03:45:55.905704+0000","last_became_active":"2024-09-21T03:44:14.322560+0000","last_became_peered":"2024-09-21T03:44:14.322560+0000","last_unstale":"2024-09-21T03:45:55.905704+0000","last_undegraded":"2024-09-21T03:45:55.905704+0000","last_fullsized":"2024-09-21T03:45:55.905704+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_clean_scrub_stamp":"2024-09-21T03:44:13.292303+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:07:55.714351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"79'8","reported_seq":142,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:46:09.861993+0000","last_change":"2024-09-21T03:44:12.312331+0000","last_active":"2024-09-21T03:46:09.861993+0000","last_peered":"2024-09-21T03:46:09.861993+0000","last_clean":"2024-09-21T03:46:09.861993+0000","last_became_active":"2024-09-21T03:44:12.311984+0000","last_became_peered":"2024-09-21T03:44:12.311984+0000","last_unstale":"2024-09-21T03:46:09.861993+0000","last_undegraded":"2024-09-21T03:46:09.861993+0000","last_fullsized":"2024-09-21T03:46:09.861993+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:11.282438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:11.282438+0000","last_clean_scrub_stamp":"2024-09-21T03:44:11.282438+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:06:11.906592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1511,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":112,"num_read_kb":6,"num_write":8,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"79'6","reported_seq":24,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:45:55.885280+0000","last_change":"2024-09-21T03:44:14.318437+0000","last_active":"2024-09-21T03:45:55.885280+0000","last_peered":"2024-09-21T03:45:55.885280+0000","last_clean":"2024-09-21T03:45:55.885280+0000","last_became_active":"2024-09-21T03:44:14.318052+0000","last_became_peered":"2024-09-21T03:44:14.318052+0000","last_unstale":"2024-09-21T03:45:55.885280+0000","last_undegraded":"2024-09-21T03:45:55.885280+0000","last_fullsized":"2024-09-21T03:45:55.885280+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_clean_scrub_stamp":"2024-09-21T03:44:13.292303+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:10:12.136159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"77'17","reported_seq":41,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:44:27.419831+0000","last_change":"2024-09-21T03:44:12.310616+0000","last_active":"2024-09-21T03:44:27.419831+0000","last_peered":"2024-09-21T03:44:27.419831+0000","last_clean":"2024-09-21T03:44:27.419831+0000","last_became_active":"2024-09-21T03:44:12.310239+0000","last_became_peered":"2024-09-21T03:44:12.310239+0000","last_unstale":"2024-09-21T03:44:27.419831+0000","last_undegraded":"2024-09-21T03:44:27.419831+0000","last_fullsized":"2024-09-21T03:44:27.419831+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:11.282438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:11.282438+0000","last_clean_scrub_stamp":"2024-09-21T03:44:11.282438+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:43:38.046281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"79'6","reported_seq":24,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:45:55.894707+0000","last_change":"2024-09-21T03:44:14.327058+0000","last_active":"2024-09-21T03:45:55.894707+0000","last_peered":"2024-09-21T03:45:55.894707+0000","last_clean":"2024-09-21T03:45:55.894707+0000","last_became_active":"2024-09-21T03:44:14.326641+0000","last_became_peered":"2024-09-21T03:44:14.326641+0000","last_unstale":"2024-09-21T03:45:55.894707+0000","last_undegraded":"2024-09-21T03:45:55.894707+0000","last_fullsized":"2024-09-21T03:45:55.894707+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_clean_scrub_stamp":"2024-09-21T03:44:13.292303+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:10:50.686102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"79'3","reported_seq":27,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:44:27.420726+0000","last_change":"2024-09-21T03:44:12.310426+0000","last_active":"2024-09-21T03:44:27.420726+0000","last_peered":"2024-09-21T03:44:27.420726+0000","last_clean":"2024-09-21T03:44:27.420726+0000","last_became_active":"2024-09-21T03:44:12.309231+0000","last_became_peered":"2024-09-21T03:44:12.309231+0000","last_unstale":"2024-09-21T03:44:27.420726+0000","last_undegraded":"2024-09-21T03:44:27.420726+0000","last_fullsized":"2024-09-21T03:44:27.420726+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:11.282438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:11.282438+0000","last_clean_scrub_stamp":"2024-09-21T03:44:11.282438+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:32:18.284741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"79'5","reported_seq":23,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:45:55.887191+0000","last_change":"2024-09-21T03:44:14.319179+0000","last_active":"2024-09-21T03:45:55.887191+0000","last_peered":"2024-09-21T03:45:55.887191+0000","last_clean":"2024-09-21T03:45:55.887191+0000","last_became_active":"2024-09-21T03:44:14.318826+0000","last_became_peered":"2024-09-21T03:44:14.318826+0000","last_unstale":"2024-09-21T03:45:55.887191+0000","last_undegraded":"2024-09-21T03:45:55.887191+0000","last_fullsized":"2024-09-21T03:45:55.887191+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_clean_scrub_stamp":"2024-09-21T03:44:13.292303+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:12:57.985087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.e","version":"77'5","reported_seq":28,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:44:27.410705+0000","last_change":"2024-09-21T03:44:12.315658+0000","last_active":"2024-09-21T03:44:27.410705+0000","last_peered":"2024-09-21T03:44:27.410705+0000","last_clean":"2024-09-21T03:44:27.410705+0000","last_became_active":"2024-09-21T03:44:12.315240+0000","last_became_peered":"2024-09-21T03:44:12.315240+0000","last_unstale":"2024-09-21T03:44:27.410705+0000","last_undegraded":"2024-09-21T03:44:27.410705+0000","last_fullsized":"2024-09-21T03:44:27.410705+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:11.282438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:11.282438+0000","last_clean_scrub_stamp":"2024-09-21T03:44:11.282438+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:22:07.493437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"79'9","reported_seq":27,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:45:55.898231+0000","last_change":"2024-09-21T03:44:14.328014+0000","last_active":"2024-09-21T03:45:55.898231+0000","last_peered":"2024-09-21T03:45:55.898231+0000","last_clean":"2024-09-21T03:45:55.898231+0000","last_became_active":"2024-09-21T03:44:14.327638+0000","last_became_peered":"2024-09-21T03:44:14.327638+0000","last_unstale":"2024-09-21T03:45:55.898231+0000","last_undegraded":"2024-09-21T03:45:55.898231+0000","last_fullsized":"2024-09-21T03:45:55.898231+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_clean_scrub_stamp":"2024-09-21T03:44:13.292303+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:48:32.542776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"77'3","reported_seq":27,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:44:27.414656+0000","last_change":"2024-09-21T03:44:12.309272+0000","last_active":"2024-09-21T03:44:27.414656+0000","last_peered":"2024-09-21T03:44:27.414656+0000","last_clean":"2024-09-21T03:44:27.414656+0000","last_became_active":"2024-09-21T03:44:12.308718+0000","last_became_peered":"2024-09-21T03:44:12.308718+0000","last_unstale":"2024-09-21T03:44:27.414656+0000","last_undegraded":"2024-09-21T03:44:27.414656+0000","last_fullsized":"2024-09-21T03:44:27.414656+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:11.282438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:11.282438+0000","last_clean_scrub_stamp":"2024-09-21T03:44:11.282438+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:03:49.433966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"79'7","reported_seq":25,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:45:55.889962+0000","last_change":"2024-09-21T03:44:14.322148+0000","last_active":"2024-09-21T03:45:55.889962+0000","last_peered":"2024-09-21T03:45:55.889962+0000","last_clean":"2024-09-21T03:45:55.889962+0000","last_became_active":"2024-09-21T03:44:14.321784+0000","last_became_peered":"2024-09-21T03:44:14.321784+0000","last_unstale":"2024-09-21T03:45:55.889962+0000","last_undegraded":"2024-09-21T03:45:55.889962+0000","last_fullsized":"2024-09-21T03:45:55.889962+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_clean_scrub_stamp":"2024-09-21T03:44:13.292303+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:40:34.887134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"77'5","reported_seq":122,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:46:07.560527+0000","last_change":"2024-09-21T03:44:27.427722+0000","last_active":"2024-09-21T03:46:07.560527+0000","last_peered":"2024-09-21T03:46:07.560527+0000","last_clean":"2024-09-21T03:46:07.560527+0000","last_became_active":"2024-09-21T03:44:27.419672+0000","last_became_peered":"2024-09-21T03:44:27.419672+0000","last_unstale":"2024-09-21T03:46:07.560527+0000","last_undegraded":"2024-09-21T03:46:07.560527+0000","last_fullsized":"2024-09-21T03:46:07.560527+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:11.282438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:11.282438+0000","last_clean_scrub_stamp":"2024-09-21T03:44:11.282438+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:24:31.890439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1170,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":102,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":1170,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,11,0],"acting":[1,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.d","version":"79'7","reported_seq":25,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:45:55.897960+0000","last_change":"2024-09-21T03:44:14.316503+0000","last_active":"2024-09-21T03:45:55.897960+0000","last_peered":"2024-09-21T03:45:55.897960+0000","last_clean":"2024-09-21T03:45:55.897960+0000","last_became_active":"2024-09-21T03:44:14.316140+0000","last_became_peered":"2024-09-21T03:44:14.316140+0000","last_unstale":"2024-09-21T03:45:55.897960+0000","last_undegraded":"2024-09-21T03:45:55.897960+0000","last_fullsized":"2024-09-21T03:45:55.897960+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_clean_scrub_stamp":"2024-09-21T03:44:13.292303+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:00:30.647579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"21'92","reported_seq":97,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:44:27.418361+0000","last_change":"2024-09-21T03:43:01.568120+0000","last_active":"2024-09-21T03:44:27.418361+0000","last_peered":"2024-09-21T03:44:27.418361+0000","last_clean":"2024-09-21T03:44:27.418361+0000","last_became_active":"2024-09-21T03:43:01.476311+0000","last_became_peered":"2024-09-21T03:43:01.476311+0000","last_unstale":"2024-09-21T03:44:27.418361+0000","last_undegraded":"2024-09-21T03:44:27.418361+0000","last_fullsized":"2024-09-21T03:44:27.418361+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:39:30.121380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:39:30.121380+0000","last_clean_scrub_stamp":"2024-09-21T03:39:30.121380+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:40:27.967442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"77'6","reported_seq":32,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:44:27.613250+0000","last_change":"2024-09-21T03:44:12.311860+0000","last_active":"2024-09-21T03:44:27.613250+0000","last_peered":"2024-09-21T03:44:27.613250+0000","last_clean":"2024-09-21T03:44:27.613250+0000","last_became_active":"2024-09-21T03:44:12.311453+0000","last_became_peered":"2024-09-21T03:44:12.311453+0000","last_unstale":"2024-09-21T03:44:27.613250+0000","last_undegraded":"2024-09-21T03:44:27.613250+0000","last_fullsized":"2024-09-21T03:44:27.613250+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:11.282438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:11.282438+0000","last_clean_scrub_stamp":"2024-09-21T03:44:11.282438+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:23:40.550445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2","version":"79'8","reported_seq":20,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:45:55.883471+0000","last_change":"2024-09-21T03:44:27.416799+0000","last_active":"2024-09-21T03:45:55.883471+0000","last_peered":"2024-09-21T03:45:55.883471+0000","last_clean":"2024-09-21T03:45:55.883471+0000","last_became_active":"2024-09-21T03:44:27.416306+0000","last_became_peered":"2024-09-21T03:44:27.416306+0000","last_unstale":"2024-09-21T03:45:55.883471+0000","last_undegraded":"2024-09-21T03:45:55.883471+0000","last_fullsized":"2024-09-21T03:45:55.883471+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_clean_scrub_stamp":"2024-09-21T03:44:13.292303+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:34:26.547100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,5,1],"acting":[9,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.0","version":"77'4","reported_seq":27,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:44:27.418313+0000","last_change":"2024-09-21T03:44:12.307302+0000","last_active":"2024-09-21T03:44:27.418313+0000","last_peered":"2024-09-21T03:44:27.418313+0000","last_clean":"2024-09-21T03:44:27.418313+0000","last_became_active":"2024-09-21T03:44:12.306856+0000","last_became_peered":"2024-09-21T03:44:12.306856+0000","last_unstale":"2024-09-21T03:44:27.418313+0000","last_undegraded":"2024-09-21T03:44:27.418313+0000","last_fullsized":"2024-09-21T03:44:27.418313+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:11.282438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:11.282438+0000","last_clean_scrub_stamp":"2024-09-21T03:44:11.282438+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:21:20.420020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"79'5","reported_seq":23,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:45:55.899176+0000","last_change":"2024-09-21T03:44:14.322913+0000","last_active":"2024-09-21T03:45:55.899176+0000","last_peered":"2024-09-21T03:45:55.899176+0000","last_clean":"2024-09-21T03:45:55.899176+0000","last_became_active":"2024-09-21T03:44:14.322168+0000","last_became_peered":"2024-09-21T03:44:14.322168+0000","last_unstale":"2024-09-21T03:45:55.899176+0000","last_undegraded":"2024-09-21T03:45:55.899176+0000","last_fullsized":"2024-09-21T03:45:55.899176+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_clean_scrub_stamp":"2024-09-21T03:44:13.292303+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:30:09.041211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"79'6","reported_seq":138,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:46:10.863417+0000","last_change":"2024-09-21T03:44:12.309084+0000","last_active":"2024-09-21T03:46:10.863417+0000","last_peered":"2024-09-21T03:46:10.863417+0000","last_clean":"2024-09-21T03:46:10.863417+0000","last_became_active":"2024-09-21T03:44:12.308551+0000","last_became_peered":"2024-09-21T03:44:12.308551+0000","last_unstale":"2024-09-21T03:46:10.863417+0000","last_undegraded":"2024-09-21T03:46:10.863417+0000","last_fullsized":"2024-09-21T03:46:10.863417+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:11.282438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:11.282438+0000","last_clean_scrub_stamp":"2024-09-21T03:44:11.282438+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:46:33.895904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":109,"num_read_kb":109,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"79'8","reported_seq":26,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:45:55.884045+0000","last_change":"2024-09-21T03:44:14.322720+0000","last_active":"2024-09-21T03:45:55.884045+0000","last_peered":"2024-09-21T03:45:55.884045+0000","last_clean":"2024-09-21T03:45:55.884045+0000","last_became_active":"2024-09-21T03:44:14.322367+0000","last_became_peered":"2024-09-21T03:44:14.322367+0000","last_unstale":"2024-09-21T03:45:55.884045+0000","last_undegraded":"2024-09-21T03:45:55.884045+0000","last_fullsized":"2024-09-21T03:45:55.884045+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_clean_scrub_stamp":"2024-09-21T03:44:13.292303+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:59:51.692976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"79'8","reported_seq":137,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:46:07.760063+0000","last_change":"2024-09-21T03:44:12.308719+0000","last_active":"2024-09-21T03:46:07.760063+0000","last_peered":"2024-09-21T03:46:07.760063+0000","last_clean":"2024-09-21T03:46:07.760063+0000","last_became_active":"2024-09-21T03:44:12.308380+0000","last_became_peered":"2024-09-21T03:44:12.308380+0000","last_unstale":"2024-09-21T03:46:07.760063+0000","last_undegraded":"2024-09-21T03:46:07.760063+0000","last_fullsized":"2024-09-21T03:46:07.760063+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:11.282438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:11.282438+0000","last_clean_scrub_stamp":"2024-09-21T03:44:11.282438+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:41:02.087501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1428,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":106,"num_read_kb":4,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"79'7","reported_seq":25,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:45:55.901997+0000","last_change":"2024-09-21T03:44:14.326541+0000","last_active":"2024-09-21T03:45:55.901997+0000","last_peered":"2024-09-21T03:45:55.901997+0000","last_clean":"2024-09-21T03:45:55.901997+0000","last_became_active":"2024-09-21T03:44:14.326132+0000","last_became_peered":"2024-09-21T03:44:14.326132+0000","last_unstale":"2024-09-21T03:45:55.901997+0000","last_undegraded":"2024-09-21T03:45:55.901997+0000","last_fullsized":"2024-09-21T03:45:55.901997+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_clean_scrub_stamp":"2024-09-21T03:44:13.292303+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:09:27.711919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"77'1","reported_seq":24,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:44:27.414339+0000","last_change":"2024-09-21T03:44:12.309463+0000","last_active":"2024-09-21T03:44:27.414339+0000","last_peered":"2024-09-21T03:44:27.414339+0000","last_clean":"2024-09-21T03:44:27.414339+0000","last_became_active":"2024-09-21T03:44:12.309054+0000","last_became_peered":"2024-09-21T03:44:12.309054+0000","last_unstale":"2024-09-21T03:44:27.414339+0000","last_undegraded":"2024-09-21T03:44:27.414339+0000","last_fullsized":"2024-09-21T03:44:27.414339+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:11.282438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:11.282438+0000","last_clean_scrub_stamp":"2024-09-21T03:44:11.282438+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:12:12.505538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"79'5","reported_seq":23,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:45:55.890793+0000","last_change":"2024-09-21T03:44:14.318261+0000","last_active":"2024-09-21T03:45:55.890793+0000","last_peered":"2024-09-21T03:45:55.890793+0000","last_clean":"2024-09-21T03:45:55.890793+0000","last_became_active":"2024-09-21T03:44:14.317839+0000","last_became_peered":"2024-09-21T03:44:14.317839+0000","last_unstale":"2024-09-21T03:45:55.890793+0000","last_undegraded":"2024-09-21T03:45:55.890793+0000","last_fullsized":"2024-09-21T03:45:55.890793+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_clean_scrub_stamp":"2024-09-21T03:44:13.292303+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:19:05.504749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.5","version":"77'4","reported_seq":20,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:44:27.434085+0000","last_change":"2024-09-21T03:44:27.433630+0000","last_active":"2024-09-21T03:44:27.434085+0000","last_peered":"2024-09-21T03:44:27.434085+0000","last_clean":"2024-09-21T03:44:27.434085+0000","last_became_active":"2024-09-21T03:44:27.422093+0000","last_became_peered":"2024-09-21T03:44:27.422093+0000","last_unstale":"2024-09-21T03:44:27.434085+0000","last_undegraded":"2024-09-21T03:44:27.434085+0000","last_fullsized":"2024-09-21T03:44:27.434085+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:11.282438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:11.282438+0000","last_clean_scrub_stamp":"2024-09-21T03:44:11.282438+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:39:42.651024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,9],"acting":[11,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.4","version":"79'9","reported_seq":27,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:45:55.899012+0000","last_change":"2024-09-21T03:44:14.322620+0000","last_active":"2024-09-21T03:45:55.899012+0000","last_peered":"2024-09-21T03:45:55.899012+0000","last_clean":"2024-09-21T03:45:55.899012+0000","last_became_active":"2024-09-21T03:44:14.322214+0000","last_became_peered":"2024-09-21T03:44:14.322214+0000","last_unstale":"2024-09-21T03:45:55.899012+0000","last_undegraded":"2024-09-21T03:45:55.899012+0000","last_fullsized":"2024-09-21T03:45:55.899012+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_clean_scrub_stamp":"2024-09-21T03:44:13.292303+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:53:20.519168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"79'17","reported_seq":40,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:45:56.352616+0000","last_change":"2024-09-21T03:44:12.311302+0000","last_active":"2024-09-21T03:45:56.352616+0000","last_peered":"2024-09-21T03:45:56.352616+0000","last_clean":"2024-09-21T03:45:56.352616+0000","last_became_active":"2024-09-21T03:44:12.310947+0000","last_became_peered":"2024-09-21T03:44:12.310947+0000","last_unstale":"2024-09-21T03:45:56.352616+0000","last_undegraded":"2024-09-21T03:45:56.352616+0000","last_fullsized":"2024-09-21T03:45:56.352616+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:11.282438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:11.282438+0000","last_clean_scrub_stamp":"2024-09-21T03:44:11.282438+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:09:34.308726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":30,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"79'5","reported_seq":23,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:45:55.891717+0000","last_change":"2024-09-21T03:44:14.324675+0000","last_active":"2024-09-21T03:45:55.891717+0000","last_peered":"2024-09-21T03:45:55.891717+0000","last_clean":"2024-09-21T03:45:55.891717+0000","last_became_active":"2024-09-21T03:44:14.324214+0000","last_became_peered":"2024-09-21T03:44:14.324214+0000","last_unstale":"2024-09-21T03:45:55.891717+0000","last_undegraded":"2024-09-21T03:45:55.891717+0000","last_fullsized":"2024-09-21T03:45:55.891717+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_clean_scrub_stamp":"2024-09-21T03:44:13.292303+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:33:51.071794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"77'7","reported_seq":31,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:44:27.613637+0000","last_change":"2024-09-21T03:44:12.322312+0000","last_active":"2024-09-21T03:44:27.613637+0000","last_peered":"2024-09-21T03:44:27.613637+0000","last_clean":"2024-09-21T03:44:27.613637+0000","last_became_active":"2024-09-21T03:44:12.322018+0000","last_became_peered":"2024-09-21T03:44:12.322018+0000","last_unstale":"2024-09-21T03:44:27.613637+0000","last_undegraded":"2024-09-21T03:44:27.613637+0000","last_fullsized":"2024-09-21T03:44:27.613637+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:11.282438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:11.282438+0000","last_clean_scrub_stamp":"2024-09-21T03:44:11.282438+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:11:41.817574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"79'7","reported_seq":25,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:45:55.904952+0000","last_change":"2024-09-21T03:44:14.326682+0000","last_active":"2024-09-21T03:45:55.904952+0000","last_peered":"2024-09-21T03:45:55.904952+0000","last_clean":"2024-09-21T03:45:55.904952+0000","last_became_active":"2024-09-21T03:44:14.326313+0000","last_became_peered":"2024-09-21T03:44:14.326313+0000","last_unstale":"2024-09-21T03:45:55.904952+0000","last_undegraded":"2024-09-21T03:45:55.904952+0000","last_fullsized":"2024-09-21T03:45:55.904952+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_clean_scrub_stamp":"2024-09-21T03:44:13.292303+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:26:44.681245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"77'2","reported_seq":36,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:44:27.436319+0000","last_change":"2024-09-21T03:44:27.436245+0000","last_active":"2024-09-21T03:44:27.436319+0000","last_peered":"2024-09-21T03:44:27.436319+0000","last_clean":"2024-09-21T03:44:27.436319+0000","last_became_active":"2024-09-21T03:44:27.422134+0000","last_became_peered":"2024-09-21T03:44:27.422134+0000","last_unstale":"2024-09-21T03:44:27.436319+0000","last_undegraded":"2024-09-21T03:44:27.436319+0000","last_fullsized":"2024-09-21T03:44:27.436319+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:11.282438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:11.282438+0000","last_clean_scrub_stamp":"2024-09-21T03:44:11.282438+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:50:22.655050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,11,0],"acting":[1,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"79'2","reported_seq":20,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:45:55.854212+0000","last_change":"2024-09-21T03:44:14.316312+0000","last_active":"2024-09-21T03:45:55.854212+0000","last_peered":"2024-09-21T03:45:55.854212+0000","last_clean":"2024-09-21T03:45:55.854212+0000","last_became_active":"2024-09-21T03:44:14.315929+0000","last_became_peered":"2024-09-21T03:44:14.315929+0000","last_unstale":"2024-09-21T03:45:55.854212+0000","last_undegraded":"2024-09-21T03:45:55.854212+0000","last_fullsized":"2024-09-21T03:45:55.854212+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_clean_scrub_stamp":"2024-09-21T03:44:13.292303+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:31:33.071983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"77'7","reported_seq":30,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:44:27.610761+0000","last_change":"2024-09-21T03:44:12.312655+0000","last_active":"2024-09-21T03:44:27.610761+0000","last_peered":"2024-09-21T03:44:27.610761+0000","last_clean":"2024-09-21T03:44:27.610761+0000","last_became_active":"2024-09-21T03:44:12.312196+0000","last_became_peered":"2024-09-21T03:44:12.312196+0000","last_unstale":"2024-09-21T03:44:27.610761+0000","last_undegraded":"2024-09-21T03:44:27.610761+0000","last_fullsized":"2024-09-21T03:44:27.610761+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:11.282438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:11.282438+0000","last_clean_scrub_stamp":"2024-09-21T03:44:11.282438+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:24:32.342461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"79'9","reported_seq":27,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:45:55.897072+0000","last_change":"2024-09-21T03:44:14.321875+0000","last_active":"2024-09-21T03:45:55.897072+0000","last_peered":"2024-09-21T03:45:55.897072+0000","last_clean":"2024-09-21T03:45:55.897072+0000","last_became_active":"2024-09-21T03:44:14.320682+0000","last_became_peered":"2024-09-21T03:44:14.320682+0000","last_unstale":"2024-09-21T03:45:55.897072+0000","last_undegraded":"2024-09-21T03:45:55.897072+0000","last_fullsized":"2024-09-21T03:45:55.897072+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_clean_scrub_stamp":"2024-09-21T03:44:13.292303+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:41:32.435288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"77'3","reported_seq":26,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:44:27.418692+0000","last_change":"2024-09-21T03:44:12.309100+0000","last_active":"2024-09-21T03:44:27.418692+0000","last_peered":"2024-09-21T03:44:27.418692+0000","last_clean":"2024-09-21T03:44:27.418692+0000","last_became_active":"2024-09-21T03:44:12.308671+0000","last_became_peered":"2024-09-21T03:44:12.308671+0000","last_unstale":"2024-09-21T03:44:27.418692+0000","last_undegraded":"2024-09-21T03:44:27.418692+0000","last_fullsized":"2024-09-21T03:44:27.418692+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:11.282438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:11.282438+0000","last_clean_scrub_stamp":"2024-09-21T03:44:11.282438+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:09:19.347980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"79'8","reported_seq":26,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:45:55.890407+0000","last_change":"2024-09-21T03:44:14.320170+0000","last_active":"2024-09-21T03:45:55.890407+0000","last_peered":"2024-09-21T03:45:55.890407+0000","last_clean":"2024-09-21T03:45:55.890407+0000","last_became_active":"2024-09-21T03:44:14.319794+0000","last_became_peered":"2024-09-21T03:44:14.319794+0000","last_unstale":"2024-09-21T03:45:55.890407+0000","last_undegraded":"2024-09-21T03:45:55.890407+0000","last_fullsized":"2024-09-21T03:45:55.890407+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_clean_scrub_stamp":"2024-09-21T03:44:13.292303+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:42:09.684471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"77'4","reported_seq":28,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:44:27.420077+0000","last_change":"2024-09-21T03:44:12.310851+0000","last_active":"2024-09-21T03:44:27.420077+0000","last_peered":"2024-09-21T03:44:27.420077+0000","last_clean":"2024-09-21T03:44:27.420077+0000","last_became_active":"2024-09-21T03:44:12.310302+0000","last_became_peered":"2024-09-21T03:44:12.310302+0000","last_unstale":"2024-09-21T03:44:27.420077+0000","last_undegraded":"2024-09-21T03:44:27.420077+0000","last_fullsized":"2024-09-21T03:44:27.420077+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:11.282438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:11.282438+0000","last_clean_scrub_stamp":"2024-09-21T03:44:11.282438+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:23:46.959121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"79'5","reported_seq":23,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:45:55.904667+0000","last_change":"2024-09-21T03:44:14.324544+0000","last_active":"2024-09-21T03:45:55.904667+0000","last_peered":"2024-09-21T03:45:55.904667+0000","last_clean":"2024-09-21T03:45:55.904667+0000","last_became_active":"2024-09-21T03:44:14.322881+0000","last_became_peered":"2024-09-21T03:44:14.322881+0000","last_unstale":"2024-09-21T03:45:55.904667+0000","last_undegraded":"2024-09-21T03:45:55.904667+0000","last_fullsized":"2024-09-21T03:45:55.904667+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_clean_scrub_stamp":"2024-09-21T03:44:13.292303+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:58:42.760185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"79'15","reported_seq":40,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:45:55.895829+0000","last_change":"2024-09-21T03:44:27.422438+0000","last_active":"2024-09-21T03:45:55.895829+0000","last_peered":"2024-09-21T03:45:55.895829+0000","last_clean":"2024-09-21T03:45:55.895829+0000","last_became_active":"2024-09-21T03:44:27.421935+0000","last_became_peered":"2024-09-21T03:44:27.421935+0000","last_unstale":"2024-09-21T03:45:55.895829+0000","last_undegraded":"2024-09-21T03:45:55.895829+0000","last_fullsized":"2024-09-21T03:45:55.895829+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_clean_scrub_stamp":"2024-09-21T03:44:13.292303+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:20:50.097783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,1],"acting":[5,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"77'5","reported_seq":29,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:44:27.420510+0000","last_change":"2024-09-21T03:44:12.311509+0000","last_active":"2024-09-21T03:44:27.420510+0000","last_peered":"2024-09-21T03:44:27.420510+0000","last_clean":"2024-09-21T03:44:27.420510+0000","last_became_active":"2024-09-21T03:44:12.310880+0000","last_became_peered":"2024-09-21T03:44:12.310880+0000","last_unstale":"2024-09-21T03:44:27.420510+0000","last_undegraded":"2024-09-21T03:44:27.420510+0000","last_fullsized":"2024-09-21T03:44:27.420510+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:11.282438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:11.282438+0000","last_clean_scrub_stamp":"2024-09-21T03:44:11.282438+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:59:33.887234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"79'9","reported_seq":27,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:45:55.905320+0000","last_change":"2024-09-21T03:44:14.330938+0000","last_active":"2024-09-21T03:45:55.905320+0000","last_peered":"2024-09-21T03:45:55.905320+0000","last_clean":"2024-09-21T03:45:55.905320+0000","last_became_active":"2024-09-21T03:44:14.330559+0000","last_became_peered":"2024-09-21T03:44:14.330559+0000","last_unstale":"2024-09-21T03:45:55.905320+0000","last_undegraded":"2024-09-21T03:45:55.905320+0000","last_fullsized":"2024-09-21T03:45:55.905320+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_clean_scrub_stamp":"2024-09-21T03:44:13.292303+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:05:32.535660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"77'12","reported_seq":36,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:44:27.420563+0000","last_change":"2024-09-21T03:44:12.307667+0000","last_active":"2024-09-21T03:44:27.420563+0000","last_peered":"2024-09-21T03:44:27.420563+0000","last_clean":"2024-09-21T03:44:27.420563+0000","last_became_active":"2024-09-21T03:44:12.307074+0000","last_became_peered":"2024-09-21T03:44:12.307074+0000","last_unstale":"2024-09-21T03:44:27.420563+0000","last_undegraded":"2024-09-21T03:44:27.420563+0000","last_fullsized":"2024-09-21T03:44:27.420563+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:11.282438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:11.282438+0000","last_clean_scrub_stamp":"2024-09-21T03:44:11.282438+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:48:41.598375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1f","version":"79'6","reported_seq":24,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:45:55.903388+0000","last_change":"2024-09-21T03:44:14.325487+0000","last_active":"2024-09-21T03:45:55.903388+0000","last_peered":"2024-09-21T03:45:55.903388+0000","last_clean":"2024-09-21T03:45:55.903388+0000","last_became_active":"2024-09-21T03:44:14.325153+0000","last_became_peered":"2024-09-21T03:44:14.325153+0000","last_unstale":"2024-09-21T03:45:55.903388+0000","last_undegraded":"2024-09-21T03:45:55.903388+0000","last_fullsized":"2024-09-21T03:45:55.903388+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_clean_scrub_stamp":"2024-09-21T03:44:13.292303+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:29:17.619130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"77'2","reported_seq":26,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:44:27.611295+0000","last_change":"2024-09-21T03:44:12.307635+0000","last_active":"2024-09-21T03:44:27.611295+0000","last_peered":"2024-09-21T03:44:27.611295+0000","last_clean":"2024-09-21T03:44:27.611295+0000","last_became_active":"2024-09-21T03:44:12.307235+0000","last_became_peered":"2024-09-21T03:44:12.307235+0000","last_unstale":"2024-09-21T03:44:27.611295+0000","last_undegraded":"2024-09-21T03:44:27.611295+0000","last_fullsized":"2024-09-21T03:44:27.611295+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:11.282438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:11.282438+0000","last_clean_scrub_stamp":"2024-09-21T03:44:11.282438+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:18:59.985369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"79'5","reported_seq":23,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:45:55.881464+0000","last_change":"2024-09-21T03:44:14.324660+0000","last_active":"2024-09-21T03:45:55.881464+0000","last_peered":"2024-09-21T03:45:55.881464+0000","last_clean":"2024-09-21T03:45:55.881464+0000","last_became_active":"2024-09-21T03:44:14.324309+0000","last_became_peered":"2024-09-21T03:44:14.324309+0000","last_unstale":"2024-09-21T03:45:55.881464+0000","last_undegraded":"2024-09-21T03:45:55.881464+0000","last_fullsized":"2024-09-21T03:45:55.881464+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:13.292303+0000","last_clean_scrub_stamp":"2024-09-21T03:44:13.292303+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:13:07.806721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"79'14","reported_seq":163,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:46:08.761874+0000","last_change":"2024-09-21T03:44:27.623910+0000","last_active":"2024-09-21T03:46:08.761874+0000","last_peered":"2024-09-21T03:46:08.761874+0000","last_clean":"2024-09-21T03:46:08.761874+0000","last_became_active":"2024-09-21T03:44:27.614037+0000","last_became_peered":"2024-09-21T03:44:27.614037+0000","last_unstale":"2024-09-21T03:46:08.761874+0000","last_undegraded":"2024-09-21T03:46:08.761874+0000","last_fullsized":"2024-09-21T03:46:08.761874+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:44:11.282438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:44:11.282438+0000","last_clean_scrub_stamp":"2024-09-21T03:44:11.282438+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:03:56.753935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":104,"num_read_kb":104,"num_write":15,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":672,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,4],"acting":[9,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":73176,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":1089,"num_read_kb":627,"num_write":262,"num_write_kb":155,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13,"num_bytes_recovered":1954,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":585728,"data_stored":227882,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":225,"ondisk_log_size":225,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":71,"seq":304942678053,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27888,"kb_used_data":864,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717264,"statfs":{"total":95995035648,"available":95966478336,"internally_reserved":0,"allocated":884736,"data_stored":428272,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":65,"seq":279172874281,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28568,"kb_used_data":1544,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716584,"statfs":{"total":95995035648,"available":95965782016,"internally_reserved":0,"allocated":1581056,"data_stored":1084180,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":60,"seq":257698037806,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27904,"kb_used_data":880,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717248,"statfs":{"total":95995035648,"available":95966461952,"internally_reserved":0,"allocated":901120,"data_stored":428090,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":54,"seq":231928234035,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27884,"kb_used_data":860,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717268,"statfs":{"total":95995035648,"available":95966482432,"internally_reserved":0,"allocated":880640,"data_stored":427684,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":49,"seq":210453397562,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36804,"kb_used_data":1460,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708348,"statfs":{"total":95995035648,"available":95957348352,"internally_reserved":0,"allocated":1495040,"data_stored":1022499,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":36174282},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":43,"seq":184683593791,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27888,"kb_used_data":864,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717264,"statfs":{"total":95995035648,"available":95966478336,"internally_reserved":0,"allocated":884736,"data_stored":429038,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":37,"seq":158913790020,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32656,"kb_used_data":1472,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712496,"statfs":{"total":95995035648,"available":95961595904,"internally_reserved":0,"allocated":1507328,"data_stored":1020805,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":31,"seq":133143986249,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36216,"kb_used_data":872,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708936,"statfs":{"total":95995035648,"available":95957950464,"internally_reserved":0,"allocated":892928,"data_stored":429263,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149775,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32064,"kb_used_data":880,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713088,"statfs":{"total":95995035648,"available":95962202112,"internally_reserved":0,"allocated":901120,"data_stored":427632,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":19,"seq":81604378708,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27964,"kb_used_data":940,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717188,"statfs":{"total":95995035648,"available":95966400512,"internally_reserved":0,"allocated":962560,"data_stored":492761,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542232,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27960,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717192,"statfs":{"total":95995035648,"available":95966404608,"internally_reserved":0,"allocated":958464,"data_stored":495189,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705759,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27908,"kb_used_data":884,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717244,"statfs":{"total":95995035648,"available":95966457856,"internally_reserved":0,"allocated":905216,"data_stored":429146,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2178,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":68221,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":106496,"data_stored":74538,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":2295,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3352,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1953,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":5046,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":716,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":118784,"data_stored":66727,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1304,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T03:46:13.692 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 51116d5c-77ca-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T03:46:13.982 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/51116d5c-77ca-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:46:14.333 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:46:14 smithi039 ceph-mon[31211]: from='client.15525 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T03:46:14.334 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:46:14 smithi039 ceph-mon[31211]: from='client.15531 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T03:46:14.467 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:46:14 smithi161 ceph-mon[34533]: from='client.15525 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T03:46:14.468 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:46:14 smithi161 ceph-mon[34533]: from='client.15531 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T03:46:14.471 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:14.461+0000 7f9074b5c640 1 -- 172.21.15.39:0/4250410076 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9070102ae0 msgr2=0x7f9070102ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:46:14.471 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:14.461+0000 7f9074b5c640 1 --2- 172.21.15.39:0/4250410076 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9070102ae0 0x7f9070102ef0 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f9060009900 tx=0x7f906002a410 comp rx=0 tx=0).stop 2024-09-21T03:46:14.471 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:14.461+0000 7f9074b5c640 1 -- 172.21.15.39:0/4250410076 shutdown_connections 2024-09-21T03:46:14.471 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:14.461+0000 7f9074b5c640 1 --2- 172.21.15.39:0/4250410076 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9070108d70 0x7f907010f5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:14.471 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:14.461+0000 7f9074b5c640 1 --2- 172.21.15.39:0/4250410076 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9070108420 0x7f9070108830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:14.471 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:14.461+0000 7f9074b5c640 1 --2- 172.21.15.39:0/4250410076 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9070102ae0 0x7f9070102ef0 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:14.471 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:14.461+0000 7f9074b5c640 1 -- 172.21.15.39:0/4250410076 >> 172.21.15.39:0/4250410076 conn(0x7f9070079770 msgr2=0x7f9070079b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:46:14.471 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:14.461+0000 7f9074b5c640 1 -- 172.21.15.39:0/4250410076 shutdown_connections 2024-09-21T03:46:14.471 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:14.461+0000 7f9074b5c640 1 -- 172.21.15.39:0/4250410076 wait complete. 2024-09-21T03:46:14.471 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:14.462+0000 7f9074b5c640 1 Processor -- start 2024-09-21T03:46:14.472 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:14.462+0000 7f9074b5c640 1 -- start start 2024-09-21T03:46:14.472 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:14.462+0000 7f9074b5c640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9070102ae0 0x7f907019b380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:14.472 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:14.462+0000 7f9074b5c640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9070108420 0x7f907019b8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:14.472 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:14.462+0000 7f9074b5c640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9070108d70 0x7f907019be00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:14.472 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:14.462+0000 7f9074b5c640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f9070112a30 con 0x7f9070108420 2024-09-21T03:46:14.472 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:14.462+0000 7f9074b5c640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f90701128b0 con 0x7f9070108d70 2024-09-21T03:46:14.472 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:14.462+0000 7f9074b5c640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f9070112bb0 con 0x7f9070102ae0 2024-09-21T03:46:14.472 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:14.462+0000 7f906dd9b640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9070108420 0x7f907019b8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:46:14.472 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:14.462+0000 7f906dd9b640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9070108420 0x7f907019b8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:43230/0 (socket says 172.21.15.39:43230) 2024-09-21T03:46:14.472 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:14.462+0000 7f906dd9b640 1 -- 172.21.15.39:0/4111517178 learned_addr learned my addr 172.21.15.39:0/4111517178 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:46:14.472 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:14.462+0000 7f906e59c640 1 --2- 172.21.15.39:0/4111517178 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9070102ae0 0x7f907019b380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:46:14.472 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:14.462+0000 7f906ed9d640 1 --2- 172.21.15.39:0/4111517178 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9070108d70 0x7f907019be00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:46:14.472 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:14.463+0000 7f906dd9b640 1 -- 172.21.15.39:0/4111517178 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9070102ae0 msgr2=0x7f907019b380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:46:14.472 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:14.463+0000 7f906dd9b640 1 --2- 172.21.15.39:0/4111517178 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9070102ae0 0x7f907019b380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:14.472 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:14.463+0000 7f906dd9b640 1 -- 172.21.15.39:0/4111517178 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9070108d70 msgr2=0x7f907019be00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:46:14.473 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:14.463+0000 7f906dd9b640 1 --2- 172.21.15.39:0/4111517178 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9070108d70 0x7f907019be00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:14.473 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:14.463+0000 7f906dd9b640 1 -- 172.21.15.39:0/4111517178 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f90701a6690 con 0x7f9070108420 2024-09-21T03:46:14.473 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:14.463+0000 7f906ed9d640 1 --2- 172.21.15.39:0/4111517178 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9070108d70 0x7f907019be00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:46:14.473 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:14.463+0000 7f906dd9b640 1 --2- 172.21.15.39:0/4111517178 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9070108420 0x7f907019b8c0 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f905800d750 tx=0x7f905800dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:46:14.473 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:14.463+0000 7f90677fe640 1 -- 172.21.15.39:0/4111517178 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9058017070 con 0x7f9070108420 2024-09-21T03:46:14.473 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:14.463+0000 7f9074b5c640 1 -- 172.21.15.39:0/4111517178 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f90701a6980 con 0x7f9070108420 2024-09-21T03:46:14.473 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:14.463+0000 7f90677fe640 1 -- 172.21.15.39:0/4111517178 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f90580054d0 con 0x7f9070108420 2024-09-21T03:46:14.473 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:14.463+0000 7f90677fe640 1 -- 172.21.15.39:0/4111517178 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9058004c30 con 0x7f9070108420 2024-09-21T03:46:14.473 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:14.463+0000 7f9074b5c640 1 -- 172.21.15.39:0/4111517178 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f90701a6ec0 con 0x7f9070108420 2024-09-21T03:46:14.473 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:14.464+0000 7f9074b5c640 1 -- 172.21.15.39:0/4111517178 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9070103d60 con 0x7f9070108420 2024-09-21T03:46:14.473 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:14.466+0000 7f90677fe640 1 -- 172.21.15.39:0/4111517178 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 24) ==== 154580+0+0 (secure 0 0 0) 0x7f9058004dd0 con 0x7f9070108420 2024-09-21T03:46:14.473 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:14.466+0000 7f90677fe640 1 --2- 172.21.15.39:0/4111517178 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f905c0b65f0 0x7f905c0b8b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:14.473 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:14.466+0000 7f906e59c640 1 --2- 172.21.15.39:0/4111517178 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f905c0b65f0 0x7f905c0b8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:46:14.473 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:14.466+0000 7f90677fe640 1 -- 172.21.15.39:0/4111517178 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8484+0+0 (secure 0 0 0) 0x7f90580e74d0 con 0x7f9070108420 2024-09-21T03:46:14.474 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:14.467+0000 7f906e59c640 1 --2- 172.21.15.39:0/4111517178 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f905c0b65f0 0x7f905c0b8b00 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f9060000c00 tx=0x7f9060031400 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:46:14.474 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:14.469+0000 7f90677fe640 1 -- 172.21.15.39:0/4111517178 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f90580adb00 con 0x7f9070108420 2024-09-21T03:46:14.474 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:46:14 smithi150 ceph-mon[34289]: from='client.15525 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T03:46:14.474 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:46:14 smithi150 ceph-mon[34289]: from='client.15531 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T03:46:14.697 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:14.694+0000 7f9074b5c640 1 -- 172.21.15.39:0/4111517178 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f9070102ef0 con 0x7f9070108420 2024-09-21T03:46:14.697 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:14.695+0000 7f90677fe640 1 -- 172.21.15.39:0/4111517178 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11978 (secure 0 0 0) 0x7f9070102ef0 con 0x7f9070108420 2024-09-21T03:46:14.698 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T03:46:14.698 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":29,"btime":"2024-09-21T03:46:08:982881+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14790,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6835/508995295","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":508995295},{"type":"v1","addr":"172.21.15.39:6835","nonce":508995295}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14802,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6837/3915912607","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":3915912607},{"type":"v1","addr":"172.21.15.39:6837","nonce":3915912607}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":29,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T03:44:15.009872+0000","modified":"2024-09-21T03:46:08.430113+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24695,"mds_1":24667,"mds_2":24659,"mds_3":24646,"mds_4":14868},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14856":{"gid":14856,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.39:6839/185509192","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6838","nonce":185509192},{"type":"v1","addr":"172.21.15.39:6839","nonce":185509192}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14868":{"gid":14868,"name":"a","rank":4,"incarnation":25,"state":"up:active","state_seq":7,"addr":"172.21.15.39:6841/4247190456","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6840","nonce":4247190456},{"type":"v1","addr":"172.21.15.39:6841","nonce":4247190456}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24599":{"gid":24599,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.150:6833/3114643217","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":3114643217},{"type":"v1","addr":"172.21.15.150:6833","nonce":3114643217}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24601":{"gid":24601,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.161:6833/2963990654","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":2963990654},{"type":"v1","addr":"172.21.15.161:6833","nonce":2963990654}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24632":{"gid":24632,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.150:6835/724594008","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6834","nonce":724594008},{"type":"v1","addr":"172.21.15.150:6835","nonce":724594008}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24640":{"gid":24640,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.161:6835/3747882110","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":3747882110},{"type":"v1","addr":"172.21.15.161:6835","nonce":3747882110}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24646":{"gid":24646,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":10,"addr":"172.21.15.161:6837/3451544597","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6836","nonce":3451544597},{"type":"v1","addr":"172.21.15.161:6837","nonce":3451544597}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24659":{"gid":24659,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.150:6837/4017043144","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6836","nonce":4017043144},{"type":"v1","addr":"172.21.15.150:6837","nonce":4017043144}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24667":{"gid":24667,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.161:6839/2190107270","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6838","nonce":2190107270},{"type":"v1","addr":"172.21.15.161:6839","nonce":2190107270}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24695":{"gid":24695,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.150:6839/2949386907","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6838","nonce":2949386907},{"type":"v1","addr":"172.21.15.150:6839","nonce":2949386907}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24695,"qdb_cluster":[24695,24667,24659,24646,14868]},"id":1}]} 2024-09-21T03:46:14.698 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 29 2024-09-21T03:46:14.700 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:14.697+0000 7f9074b5c640 1 -- 172.21.15.39:0/4111517178 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f905c0b65f0 msgr2=0x7f905c0b8b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:46:14.700 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:14.697+0000 7f9074b5c640 1 --2- 172.21.15.39:0/4111517178 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f905c0b65f0 0x7f905c0b8b00 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f9060000c00 tx=0x7f9060031400 comp rx=0 tx=0).stop 2024-09-21T03:46:14.700 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:14.697+0000 7f9074b5c640 1 -- 172.21.15.39:0/4111517178 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9070108420 msgr2=0x7f907019b8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:46:14.700 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:14.697+0000 7f9074b5c640 1 --2- 172.21.15.39:0/4111517178 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9070108420 0x7f907019b8c0 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f905800d750 tx=0x7f905800dc10 comp rx=0 tx=0).stop 2024-09-21T03:46:14.700 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:14.697+0000 7f9074b5c640 1 -- 172.21.15.39:0/4111517178 shutdown_connections 2024-09-21T03:46:14.700 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:14.697+0000 7f9074b5c640 1 --2- 172.21.15.39:0/4111517178 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f905c0b65f0 0x7f905c0b8b00 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:14.700 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:14.697+0000 7f9074b5c640 1 --2- 172.21.15.39:0/4111517178 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9070108d70 0x7f907019be00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:14.701 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:14.697+0000 7f9074b5c640 1 --2- 172.21.15.39:0/4111517178 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9070108420 0x7f907019b8c0 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:14.701 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:14.697+0000 7f9074b5c640 1 --2- 172.21.15.39:0/4111517178 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9070102ae0 0x7f907019b380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:14.701 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:14.697+0000 7f9074b5c640 1 -- 172.21.15.39:0/4111517178 >> 172.21.15.39:0/4111517178 conn(0x7f9070079770 msgr2=0x7f9070106570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:46:14.701 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:14.697+0000 7f9074b5c640 1 -- 172.21.15.39:0/4111517178 shutdown_connections 2024-09-21T03:46:14.701 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:14.698+0000 7f9074b5c640 1 -- 172.21.15.39:0/4111517178 wait complete. 2024-09-21T03:46:15.334 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:46:15 smithi039 ceph-mon[31211]: pgmap v331: 97 pgs: 97 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 426 B/s wr, 9 op/s 2024-09-21T03:46:15.334 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:46:15 smithi039 ceph-mon[31211]: from='client.15537 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T03:46:15.334 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:46:15 smithi039 ceph-mon[31211]: from='client.? 172.21.15.39:0/4111517178' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:46:15.468 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:46:15 smithi161 ceph-mon[34533]: pgmap v331: 97 pgs: 97 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 426 B/s wr, 9 op/s 2024-09-21T03:46:15.468 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:46:15 smithi161 ceph-mon[34533]: from='client.15537 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T03:46:15.468 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:46:15 smithi161 ceph-mon[34533]: from='client.? 172.21.15.39:0/4111517178' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:46:15.474 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:46:15 smithi150 ceph-mon[34289]: pgmap v331: 97 pgs: 97 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 426 B/s wr, 9 op/s 2024-09-21T03:46:15.475 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:46:15 smithi150 ceph-mon[34289]: from='client.15537 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T03:46:15.475 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:46:15 smithi150 ceph-mon[34289]: from='client.? 172.21.15.39:0/4111517178' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:46:15.509 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 51116d5c-77ca-11ef-baf5-efdc52797490 -- ceph osd dump --format=json 2024-09-21T03:46:15.771 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/51116d5c-77ca-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:46:16.237 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:16.227+0000 7fe51b353640 1 -- 172.21.15.39:0/2302910613 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe514108cf0 msgr2=0x7fe51410f520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:46:16.237 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:16.227+0000 7fe51b353640 1 --2- 172.21.15.39:0/2302910613 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe514108cf0 0x7fe51410f520 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7fe510009ca0 tx=0x7fe51002a8c0 comp rx=0 tx=0).stop 2024-09-21T03:46:16.237 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:16.228+0000 7fe51b353640 1 -- 172.21.15.39:0/2302910613 shutdown_connections 2024-09-21T03:46:16.237 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:16.228+0000 7fe51b353640 1 --2- 172.21.15.39:0/2302910613 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe514108cf0 0x7fe51410f520 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:16.237 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:16.228+0000 7fe51b353640 1 --2- 172.21.15.39:0/2302910613 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe5141083a0 0x7fe5141087b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:16.237 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:16.228+0000 7fe51b353640 1 --2- 172.21.15.39:0/2302910613 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe514102a60 0x7fe514102e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:16.237 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:16.228+0000 7fe51b353640 1 -- 172.21.15.39:0/2302910613 >> 172.21.15.39:0/2302910613 conn(0x7fe514079730 msgr2=0x7fe514079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:46:16.237 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:16.228+0000 7fe51b353640 1 -- 172.21.15.39:0/2302910613 shutdown_connections 2024-09-21T03:46:16.238 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:16.228+0000 7fe51b353640 1 -- 172.21.15.39:0/2302910613 wait complete. 2024-09-21T03:46:16.238 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:16.228+0000 7fe51b353640 1 Processor -- start 2024-09-21T03:46:16.238 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:16.228+0000 7fe51b353640 1 -- start start 2024-09-21T03:46:16.238 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:16.229+0000 7fe51b353640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe514102a60 0x7fe5141050e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:16.238 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:16.229+0000 7fe51b353640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe5141083a0 0x7fe514105620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:16.238 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:16.229+0000 7fe51b353640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe514108cf0 0x7fe514105b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:16.238 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:16.229+0000 7fe51b353640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fe5141129c0 con 0x7fe514108cf0 2024-09-21T03:46:16.238 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:16.229+0000 7fe51b353640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fe514112840 con 0x7fe5141083a0 2024-09-21T03:46:16.238 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:16.229+0000 7fe51b353640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fe514112b40 con 0x7fe514102a60 2024-09-21T03:46:16.238 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:16.229+0000 7fe5198f0640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe514108cf0 0x7fe514105b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:46:16.238 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:16.229+0000 7fe5198f0640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe514108cf0 0x7fe514105b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:43260/0 (socket says 172.21.15.39:43260) 2024-09-21T03:46:16.238 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:16.229+0000 7fe5198f0640 1 -- 172.21.15.39:0/833538571 learned_addr learned my addr 172.21.15.39:0/833538571 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:46:16.238 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:16.229+0000 7fe5190ef640 1 --2- 172.21.15.39:0/833538571 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe514102a60 0x7fe5141050e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:46:16.239 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:16.229+0000 7fe5188ee640 1 --2- 172.21.15.39:0/833538571 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe5141083a0 0x7fe514105620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:46:16.239 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:16.229+0000 7fe5198f0640 1 -- 172.21.15.39:0/833538571 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe514102a60 msgr2=0x7fe5141050e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:46:16.239 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:16.229+0000 7fe5198f0640 1 --2- 172.21.15.39:0/833538571 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe514102a60 0x7fe5141050e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:16.239 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:16.229+0000 7fe5198f0640 1 -- 172.21.15.39:0/833538571 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe5141083a0 msgr2=0x7fe514105620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:46:16.239 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:16.229+0000 7fe5198f0640 1 --2- 172.21.15.39:0/833538571 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe5141083a0 0x7fe514105620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:16.239 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:16.229+0000 7fe5198f0640 1 -- 172.21.15.39:0/833538571 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe5141a86f0 con 0x7fe514108cf0 2024-09-21T03:46:16.239 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:16.230+0000 7fe5198f0640 1 --2- 172.21.15.39:0/833538571 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe514108cf0 0x7fe514105b60 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7fe510005cc0 tx=0x7fe51002f380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:46:16.239 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:16.230+0000 7fe50a7fc640 1 -- 172.21.15.39:0/833538571 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe510003bf0 con 0x7fe514108cf0 2024-09-21T03:46:16.239 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:16.230+0000 7fe51b353640 1 -- 172.21.15.39:0/833538571 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe5141a8980 con 0x7fe514108cf0 2024-09-21T03:46:16.239 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:16.230+0000 7fe50a7fc640 1 -- 172.21.15.39:0/833538571 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe510003d90 con 0x7fe514108cf0 2024-09-21T03:46:16.239 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:16.230+0000 7fe50a7fc640 1 -- 172.21.15.39:0/833538571 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe51002fbc0 con 0x7fe514108cf0 2024-09-21T03:46:16.239 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:16.230+0000 7fe51b353640 1 -- 172.21.15.39:0/833538571 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe5141a8f40 con 0x7fe514108cf0 2024-09-21T03:46:16.239 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:16.231+0000 7fe51b353640 1 -- 172.21.15.39:0/833538571 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe514102e70 con 0x7fe514108cf0 2024-09-21T03:46:16.239 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:16.232+0000 7fe50a7fc640 1 -- 172.21.15.39:0/833538571 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 24) ==== 154580+0+0 (secure 0 0 0) 0x7fe51002fd60 con 0x7fe514108cf0 2024-09-21T03:46:16.239 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:16.233+0000 7fe50a7fc640 1 --2- 172.21.15.39:0/833538571 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7fe5000b65f0 0x7fe5000b8b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:16.240 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:16.233+0000 7fe5190ef640 1 --2- 172.21.15.39:0/833538571 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7fe5000b65f0 0x7fe5000b8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:46:16.240 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:16.233+0000 7fe50a7fc640 1 -- 172.21.15.39:0/833538571 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8484+0+0 (secure 0 0 0) 0x7fe510108cd0 con 0x7fe514108cf0 2024-09-21T03:46:16.240 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:16.234+0000 7fe5190ef640 1 --2- 172.21.15.39:0/833538571 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7fe5000b65f0 0x7fe5000b8b00 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fe514106b30 tx=0x7fe50c00b390 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:46:16.240 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:16.237+0000 7fe50a7fc640 1 -- 172.21.15.39:0/833538571 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fe5100cf300 con 0x7fe514108cf0 2024-09-21T03:46:16.427 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:16.425+0000 7fe51b353640 1 -- 172.21.15.39:0/833538571 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fe51402e640 con 0x7fe514108cf0 2024-09-21T03:46:16.429 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:16.426+0000 7fe50a7fc640 1 -- 172.21.15.39:0/833538571 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v79) ==== 74+0+23096 (secure 0 0 0) 0x7fe5100d36e0 con 0x7fe514108cf0 2024-09-21T03:46:16.429 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T03:46:16.429 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":79,"fsid":"51116d5c-77ca-11ef-baf5-efdc52797490","created":"2024-09-21T03:35:33.631350+0000","modified":"2024-09-21T03:44:27.402588+0000","last_up_change":"2024-09-21T03:43:24.242275+0000","last_in_change":"2024-09-21T03:43:00.886552+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T03:39:29.859542+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-21T03:44:10.886435+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"77","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-21T03:44:13.041783+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"77","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1ea7148f-2ace-4c10-86fe-b8019de80c1f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6802","nonce":870188791},{"type":"v1","addr":"172.21.15.39:6803","nonce":870188791}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6804","nonce":870188791},{"type":"v1","addr":"172.21.15.39:6805","nonce":870188791}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6808","nonce":870188791},{"type":"v1","addr":"172.21.15.39:6809","nonce":870188791}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6806","nonce":870188791},{"type":"v1","addr":"172.21.15.39:6807","nonce":870188791}]},"public_addr":"172.21.15.39:6803/870188791","cluster_addr":"172.21.15.39:6805/870188791","heartbeat_back_addr":"172.21.15.39:6809/870188791","heartbeat_front_addr":"172.21.15.39:6807/870188791","state":["exists","up"]},{"osd":1,"uuid":"7cb5db90-28e8-4a30-bedb-c039687d48b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6800","nonce":172427832},{"type":"v1","addr":"172.21.15.150:6801","nonce":172427832}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6802","nonce":172427832},{"type":"v1","addr":"172.21.15.150:6803","nonce":172427832}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6806","nonce":172427832},{"type":"v1","addr":"172.21.15.150:6807","nonce":172427832}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6804","nonce":172427832},{"type":"v1","addr":"172.21.15.150:6805","nonce":172427832}]},"public_addr":"172.21.15.150:6801/172427832","cluster_addr":"172.21.15.150:6803/172427832","heartbeat_back_addr":"172.21.15.150:6807/172427832","heartbeat_front_addr":"172.21.15.150:6805/172427832","state":["exists","up"]},{"osd":2,"uuid":"b25c6ab3-fc95-4751-bd63-6e3fc23e8bd7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":19,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6800","nonce":2727876133},{"type":"v1","addr":"172.21.15.161:6801","nonce":2727876133}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6802","nonce":2727876133},{"type":"v1","addr":"172.21.15.161:6803","nonce":2727876133}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6806","nonce":2727876133},{"type":"v1","addr":"172.21.15.161:6807","nonce":2727876133}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6804","nonce":2727876133},{"type":"v1","addr":"172.21.15.161:6805","nonce":2727876133}]},"public_addr":"172.21.15.161:6801/2727876133","cluster_addr":"172.21.15.161:6803/2727876133","heartbeat_back_addr":"172.21.15.161:6807/2727876133","heartbeat_front_addr":"172.21.15.161:6805/2727876133","state":["exists","up"]},{"osd":3,"uuid":"14417ae1-4536-45ba-9a5c-c3b447d1a72d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6810","nonce":295394468},{"type":"v1","addr":"172.21.15.39:6811","nonce":295394468}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6812","nonce":295394468},{"type":"v1","addr":"172.21.15.39:6813","nonce":295394468}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6816","nonce":295394468},{"type":"v1","addr":"172.21.15.39:6817","nonce":295394468}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6814","nonce":295394468},{"type":"v1","addr":"172.21.15.39:6815","nonce":295394468}]},"public_addr":"172.21.15.39:6811/295394468","cluster_addr":"172.21.15.39:6813/295394468","heartbeat_back_addr":"172.21.15.39:6817/295394468","heartbeat_front_addr":"172.21.15.39:6815/295394468","state":["exists","up"]},{"osd":4,"uuid":"33c732ef-20c2-43c4-abe6-045b687fc521","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6808","nonce":3097721553},{"type":"v1","addr":"172.21.15.150:6809","nonce":3097721553}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6810","nonce":3097721553},{"type":"v1","addr":"172.21.15.150:6811","nonce":3097721553}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6814","nonce":3097721553},{"type":"v1","addr":"172.21.15.150:6815","nonce":3097721553}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6812","nonce":3097721553},{"type":"v1","addr":"172.21.15.150:6813","nonce":3097721553}]},"public_addr":"172.21.15.150:6809/3097721553","cluster_addr":"172.21.15.150:6811/3097721553","heartbeat_back_addr":"172.21.15.150:6815/3097721553","heartbeat_front_addr":"172.21.15.150:6813/3097721553","state":["exists","up"]},{"osd":5,"uuid":"74673722-abd6-400a-bb50-ec3dfba0c5e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6808","nonce":742751023},{"type":"v1","addr":"172.21.15.161:6809","nonce":742751023}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6810","nonce":742751023},{"type":"v1","addr":"172.21.15.161:6811","nonce":742751023}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6814","nonce":742751023},{"type":"v1","addr":"172.21.15.161:6815","nonce":742751023}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6812","nonce":742751023},{"type":"v1","addr":"172.21.15.161:6813","nonce":742751023}]},"public_addr":"172.21.15.161:6809/742751023","cluster_addr":"172.21.15.161:6811/742751023","heartbeat_back_addr":"172.21.15.161:6815/742751023","heartbeat_front_addr":"172.21.15.161:6813/742751023","state":["exists","up"]},{"osd":6,"uuid":"06548677-74c4-4efa-8054-df09b957b649","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6818","nonce":1072700290},{"type":"v1","addr":"172.21.15.39:6819","nonce":1072700290}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6820","nonce":1072700290},{"type":"v1","addr":"172.21.15.39:6821","nonce":1072700290}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6824","nonce":1072700290},{"type":"v1","addr":"172.21.15.39:6825","nonce":1072700290}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6822","nonce":1072700290},{"type":"v1","addr":"172.21.15.39:6823","nonce":1072700290}]},"public_addr":"172.21.15.39:6819/1072700290","cluster_addr":"172.21.15.39:6821/1072700290","heartbeat_back_addr":"172.21.15.39:6825/1072700290","heartbeat_front_addr":"172.21.15.39:6823/1072700290","state":["exists","up"]},{"osd":7,"uuid":"838735ca-c693-4f58-9426-b8ee857dbe3a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6816","nonce":2162619103},{"type":"v1","addr":"172.21.15.150:6817","nonce":2162619103}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6818","nonce":2162619103},{"type":"v1","addr":"172.21.15.150:6819","nonce":2162619103}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6822","nonce":2162619103},{"type":"v1","addr":"172.21.15.150:6823","nonce":2162619103}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6820","nonce":2162619103},{"type":"v1","addr":"172.21.15.150:6821","nonce":2162619103}]},"public_addr":"172.21.15.150:6817/2162619103","cluster_addr":"172.21.15.150:6819/2162619103","heartbeat_back_addr":"172.21.15.150:6823/2162619103","heartbeat_front_addr":"172.21.15.150:6821/2162619103","state":["exists","up"]},{"osd":8,"uuid":"4973b357-b4c2-4c00-b7f7-dd25012a0f72","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6816","nonce":52966633},{"type":"v1","addr":"172.21.15.161:6817","nonce":52966633}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6818","nonce":52966633},{"type":"v1","addr":"172.21.15.161:6819","nonce":52966633}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6822","nonce":52966633},{"type":"v1","addr":"172.21.15.161:6823","nonce":52966633}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6820","nonce":52966633},{"type":"v1","addr":"172.21.15.161:6821","nonce":52966633}]},"public_addr":"172.21.15.161:6817/52966633","cluster_addr":"172.21.15.161:6819/52966633","heartbeat_back_addr":"172.21.15.161:6823/52966633","heartbeat_front_addr":"172.21.15.161:6821/52966633","state":["exists","up"]},{"osd":9,"uuid":"6028613c-ebfd-42ea-9949-3f0d5f6cf025","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6826","nonce":2001303253},{"type":"v1","addr":"172.21.15.39:6827","nonce":2001303253}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6828","nonce":2001303253},{"type":"v1","addr":"172.21.15.39:6829","nonce":2001303253}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6832","nonce":2001303253},{"type":"v1","addr":"172.21.15.39:6833","nonce":2001303253}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6830","nonce":2001303253},{"type":"v1","addr":"172.21.15.39:6831","nonce":2001303253}]},"public_addr":"172.21.15.39:6827/2001303253","cluster_addr":"172.21.15.39:6829/2001303253","heartbeat_back_addr":"172.21.15.39:6833/2001303253","heartbeat_front_addr":"172.21.15.39:6831/2001303253","state":["exists","up"]},{"osd":10,"uuid":"1c71c765-2a6c-4ffe-9fe1-c8adae3d1c32","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6824","nonce":2484627938},{"type":"v1","addr":"172.21.15.150:6825","nonce":2484627938}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6826","nonce":2484627938},{"type":"v1","addr":"172.21.15.150:6827","nonce":2484627938}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6830","nonce":2484627938},{"type":"v1","addr":"172.21.15.150:6831","nonce":2484627938}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6828","nonce":2484627938},{"type":"v1","addr":"172.21.15.150:6829","nonce":2484627938}]},"public_addr":"172.21.15.150:6825/2484627938","cluster_addr":"172.21.15.150:6827/2484627938","heartbeat_back_addr":"172.21.15.150:6831/2484627938","heartbeat_front_addr":"172.21.15.150:6829/2484627938","state":["exists","up"]},{"osd":11,"uuid":"f382f522-62db-46da-997e-2b28f84d1612","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6824","nonce":2178709707},{"type":"v1","addr":"172.21.15.161:6825","nonce":2178709707}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6826","nonce":2178709707},{"type":"v1","addr":"172.21.15.161:6827","nonce":2178709707}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6830","nonce":2178709707},{"type":"v1","addr":"172.21.15.161:6831","nonce":2178709707}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6828","nonce":2178709707},{"type":"v1","addr":"172.21.15.161:6829","nonce":2178709707}]},"public_addr":"172.21.15.161:6825/2178709707","cluster_addr":"172.21.15.161:6827/2178709707","heartbeat_back_addr":"172.21.15.161:6831/2178709707","heartbeat_front_addr":"172.21.15.161:6829/2178709707","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:38:35.540904+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:39:01.455053+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:39:27.052867+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:39:55.173529+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:40:21.298396+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:40:47.131235+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:41:14.475209+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:41:39.920036+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:42:05.587065+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:42:32.624726+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:42:57.479185+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:43:22.203832+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.5","mappings":[{"from":8,"to":11}]},{"pgid":"2.8","mappings":[{"from":2,"to":11}]},{"pgid":"2.c","mappings":[{"from":2,"to":1}]},{"pgid":"2.1f","mappings":[{"from":8,"to":4}]},{"pgid":"3.2","mappings":[{"from":3,"to":9}]},{"pgid":"3.14","mappings":[{"from":4,"to":6}]},{"pgid":"3.1a","mappings":[{"from":5,"to":7}]},{"pgid":"3.1d","mappings":[{"from":4,"to":8}]},{"pgid":"3.2c","mappings":[{"from":4,"to":9}]},{"pgid":"3.38","mappings":[{"from":3,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.39:6801/2438110611":"2024-09-22T03:37:25.391457+0000","172.21.15.39:6800/2438110611":"2024-09-22T03:37:25.391457+0000","172.21.15.39:6800/173264695":"2024-09-22T03:35:57.104467+0000","172.21.15.39:6801/173264695":"2024-09-22T03:35:57.104467+0000","172.21.15.39:0/2637182284":"2024-09-22T03:35:57.104467+0000","172.21.15.39:0/1517304561":"2024-09-22T03:36:24.112592+0000","172.21.15.39:6801/1435622199":"2024-09-22T03:36:24.112592+0000","172.21.15.39:6800/1435622199":"2024-09-22T03:36:24.112592+0000","172.21.15.39:0/346832469":"2024-09-22T03:35:57.104467+0000","172.21.15.39:0/3160777371":"2024-09-22T03:36:24.112592+0000","172.21.15.39:0/1730342855":"2024-09-22T03:35:57.104467+0000","172.21.15.39:0/1955726766":"2024-09-22T03:36:24.112592+0000","172.21.15.39:0/3294132796":"2024-09-22T03:37:25.391457+0000","172.21.15.39:0/1295810162":"2024-09-22T03:37:25.391457+0000","172.21.15.39:0/3729230474":"2024-09-22T03:37:25.391457+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T03:46:16.431 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:16.428+0000 7fe51b353640 1 -- 172.21.15.39:0/833538571 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7fe5000b65f0 msgr2=0x7fe5000b8b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:46:16.431 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:16.428+0000 7fe51b353640 1 --2- 172.21.15.39:0/833538571 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7fe5000b65f0 0x7fe5000b8b00 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fe514106b30 tx=0x7fe50c00b390 comp rx=0 tx=0).stop 2024-09-21T03:46:16.431 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:16.428+0000 7fe51b353640 1 -- 172.21.15.39:0/833538571 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe514108cf0 msgr2=0x7fe514105b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:46:16.431 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:16.428+0000 7fe51b353640 1 --2- 172.21.15.39:0/833538571 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe514108cf0 0x7fe514105b60 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7fe510005cc0 tx=0x7fe51002f380 comp rx=0 tx=0).stop 2024-09-21T03:46:16.431 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:16.429+0000 7fe51b353640 1 -- 172.21.15.39:0/833538571 shutdown_connections 2024-09-21T03:46:16.431 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:16.429+0000 7fe51b353640 1 --2- 172.21.15.39:0/833538571 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7fe5000b65f0 0x7fe5000b8b00 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:16.432 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:16.429+0000 7fe51b353640 1 --2- 172.21.15.39:0/833538571 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe514108cf0 0x7fe514105b60 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:16.432 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:16.429+0000 7fe51b353640 1 --2- 172.21.15.39:0/833538571 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe5141083a0 0x7fe514105620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:16.432 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:16.429+0000 7fe51b353640 1 --2- 172.21.15.39:0/833538571 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe514102a60 0x7fe5141050e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:16.432 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:16.429+0000 7fe51b353640 1 -- 172.21.15.39:0/833538571 >> 172.21.15.39:0/833538571 conn(0x7fe514079730 msgr2=0x7fe51406b1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:46:16.432 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:16.429+0000 7fe51b353640 1 -- 172.21.15.39:0/833538571 shutdown_connections 2024-09-21T03:46:16.432 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:16.429+0000 7fe51b353640 1 -- 172.21.15.39:0/833538571 wait complete. 2024-09-21T03:46:17.104 DEBUG:tasks.fwd_scrub:Yielding 2024-09-21T03:46:17.104 INFO:teuthology.run_tasks:Running task exec... 2024-09-21T03:46:17.113 INFO:teuthology.task.exec:Executing custom commands... 2024-09-21T03:46:17.114 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi039.front.sepia.ceph.com 2024-09-21T03:46:17.114 DEBUG:teuthology.orchestra.run.smithi039:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule add --fs=cephfs --path=/ --snap_schedule=1m' 2024-09-21T03:46:17.114 INFO:tasks.fwd_scrub.fs.[cephfs]:start scrubbing fs: cephfs 2024-09-21T03:46:17.114 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-21T03:46:17.114 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 51116d5c-77ca-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-21T03:46:17.234 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:17.231+0000 7f935c9f1640 1 -- 172.21.15.39:0/2597837157 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f93581029d0 msgr2=0x7f9358102de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:46:17.234 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:17.231+0000 7f935c9f1640 1 --2- 172.21.15.39:0/2597837157 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f93581029d0 0x7f9358102de0 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f9340009900 tx=0x7f934002a410 comp rx=0 tx=0).stop 2024-09-21T03:46:17.235 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:17.231+0000 7f935c9f1640 1 -- 172.21.15.39:0/2597837157 shutdown_connections 2024-09-21T03:46:17.235 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:17.231+0000 7f935c9f1640 1 --2- 172.21.15.39:0/2597837157 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9358108c50 0x7f935810f480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:17.235 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:17.231+0000 7f935c9f1640 1 --2- 172.21.15.39:0/2597837157 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9358108300 0x7f9358108710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:17.235 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:17.231+0000 7f935c9f1640 1 --2- 172.21.15.39:0/2597837157 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f93581029d0 0x7f9358102de0 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:17.235 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:17.231+0000 7f935c9f1640 1 -- 172.21.15.39:0/2597837157 >> 172.21.15.39:0/2597837157 conn(0x7f935807a6f0 msgr2=0x7f935807ab00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:46:17.235 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:17.231+0000 7f935c9f1640 1 -- 172.21.15.39:0/2597837157 shutdown_connections 2024-09-21T03:46:17.235 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:17.231+0000 7f935c9f1640 1 -- 172.21.15.39:0/2597837157 wait complete. 2024-09-21T03:46:17.237 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:17.233+0000 7f935c9f1640 1 Processor -- start 2024-09-21T03:46:17.237 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:17.233+0000 7f935c9f1640 1 -- start start 2024-09-21T03:46:17.237 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:17.233+0000 7f935c9f1640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f93581029d0 0x7f935819a470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:17.237 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:17.233+0000 7f935c9f1640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9358108300 0x7f935819aa10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:17.237 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:17.233+0000 7f935c9f1640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9358108c50 0x7f935819fca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:17.237 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:17.233+0000 7f935c9f1640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f9358113a30 con 0x7f93581029d0 2024-09-21T03:46:17.238 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:17.233+0000 7f935c9f1640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f93581138b0 con 0x7f9358108c50 2024-09-21T03:46:17.238 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:17.233+0000 7f935c9f1640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f9358113bb0 con 0x7f9358108300 2024-09-21T03:46:17.238 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:17.233+0000 7f935659c640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f93581029d0 0x7f935819a470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:46:17.238 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:17.233+0000 7f935659c640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f93581029d0 0x7f935819a470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:43274/0 (socket says 172.21.15.39:43274) 2024-09-21T03:46:17.238 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:17.233+0000 7f935659c640 1 -- 172.21.15.39:0/1532348942 learned_addr learned my addr 172.21.15.39:0/1532348942 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:46:17.238 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:17.233+0000 7f9355d9b640 1 --2- 172.21.15.39:0/1532348942 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9358108300 0x7f935819aa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:46:17.238 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:17.233+0000 7f9356d9d640 1 --2- 172.21.15.39:0/1532348942 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9358108c50 0x7f935819fca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:46:17.238 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:17.233+0000 7f935659c640 1 -- 172.21.15.39:0/1532348942 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9358108300 msgr2=0x7f935819aa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:46:17.238 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:17.233+0000 7f935659c640 1 --2- 172.21.15.39:0/1532348942 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9358108300 0x7f935819aa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:17.238 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:17.233+0000 7f935659c640 1 -- 172.21.15.39:0/1532348942 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9358108c50 msgr2=0x7f935819fca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:46:17.238 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:17.233+0000 7f935659c640 1 --2- 172.21.15.39:0/1532348942 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9358108c50 0x7f935819fca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:17.238 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:17.233+0000 7f935659c640 1 -- 172.21.15.39:0/1532348942 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f93581a03a0 con 0x7f93581029d0 2024-09-21T03:46:17.238 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:17.233+0000 7f9355d9b640 1 --2- 172.21.15.39:0/1532348942 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9358108300 0x7f935819aa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:46:17.238 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:17.233+0000 7f9356d9d640 1 --2- 172.21.15.39:0/1532348942 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9358108c50 0x7f935819fca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:46:17.239 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:17.234+0000 7f935659c640 1 --2- 172.21.15.39:0/1532348942 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f93581029d0 0x7f935819a470 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f9340005940 tx=0x7f93400042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:46:17.239 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:17.234+0000 7f934f7fe640 1 -- 172.21.15.39:0/1532348942 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f934003f030 con 0x7f93581029d0 2024-09-21T03:46:17.239 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:17.234+0000 7f935c9f1640 1 -- 172.21.15.39:0/1532348942 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f93581a0630 con 0x7f93581029d0 2024-09-21T03:46:17.239 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:17.234+0000 7f934f7fe640 1 -- 172.21.15.39:0/1532348942 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9340003cb0 con 0x7f93581029d0 2024-09-21T03:46:17.239 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:17.234+0000 7f935c9f1640 1 -- 172.21.15.39:0/1532348942 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f93581a6e20 con 0x7f93581029d0 2024-09-21T03:46:17.239 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:17.234+0000 7f934f7fe640 1 -- 172.21.15.39:0/1532348942 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9340035070 con 0x7f93581029d0 2024-09-21T03:46:17.239 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:17.235+0000 7f935c9f1640 1 -- 172.21.15.39:0/1532348942 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9358102de0 con 0x7f93581029d0 2024-09-21T03:46:17.239 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:17.237+0000 7f934f7fe640 1 -- 172.21.15.39:0/1532348942 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 24) ==== 154580+0+0 (secure 0 0 0) 0x7f9340042030 con 0x7f93581029d0 2024-09-21T03:46:17.240 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:17.237+0000 7f934f7fe640 1 --2- 172.21.15.39:0/1532348942 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f93440b65f0 0x7f93440b8b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:17.240 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:17.238+0000 7f9355d9b640 1 --2- 172.21.15.39:0/1532348942 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f93440b65f0 0x7f93440b8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:46:17.240 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:17.238+0000 7f934f7fe640 1 -- 172.21.15.39:0/1532348942 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8484+0+0 (secure 0 0 0) 0x7f93401082a0 con 0x7f93581029d0 2024-09-21T03:46:17.241 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:17.238+0000 7f9355d9b640 1 --2- 172.21.15.39:0/1532348942 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f93440b65f0 0x7f93440b8b00 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f9348005d00 tx=0x7f9348005c70 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:46:17.247 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:17.245+0000 7f934f7fe640 1 -- 172.21.15.39:0/1532348942 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f93400ce8d0 con 0x7f93581029d0 2024-09-21T03:46:17.333 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:46:17 smithi039 ceph-mon[31211]: pgmap v332: 97 pgs: 97 active+clean; 657 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 1.4 KiB/s wr, 13 op/s 2024-09-21T03:46:17.334 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:46:17 smithi039 ceph-mon[31211]: from='client.? 172.21.15.39:0/833538571' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T03:46:17.381 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/51116d5c-77ca-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:46:17.468 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:17.465+0000 7f935c9f1640 1 -- 172.21.15.39:0/1532348942 --> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}) -- 0x7f935810aa80 con 0x7f93440b65f0 2024-09-21T03:46:17.468 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:46:17 smithi161 ceph-mon[34533]: pgmap v332: 97 pgs: 97 active+clean; 657 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 1.4 KiB/s wr, 13 op/s 2024-09-21T03:46:17.468 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:46:17 smithi161 ceph-mon[34533]: from='client.? 172.21.15.39:0/833538571' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T03:46:17.474 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:46:17 smithi150 ceph-mon[34289]: pgmap v332: 97 pgs: 97 active+clean; 657 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 1.4 KiB/s wr, 13 op/s 2024-09-21T03:46:17.474 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:46:17 smithi150 ceph-mon[34289]: from='client.? 172.21.15.39:0/833538571' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T03:46:17.643 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:17.640+0000 7f934f7fe640 1 -- 172.21.15.39:0/1532348942 <== mgr.14223 v2:172.21.15.39:6800/1696381982 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f935810aa80 con 0x7f93440b65f0 2024-09-21T03:46:17.643 INFO:teuthology.orchestra.run.smithi039.stdout:Schedule set for path / 2024-09-21T03:46:17.646 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:17.643+0000 7f935c9f1640 1 -- 172.21.15.39:0/1532348942 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f93440b65f0 msgr2=0x7f93440b8b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:46:17.646 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:17.643+0000 7f935c9f1640 1 --2- 172.21.15.39:0/1532348942 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f93440b65f0 0x7f93440b8b00 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f9348005d00 tx=0x7f9348005c70 comp rx=0 tx=0).stop 2024-09-21T03:46:17.646 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:17.643+0000 7f935c9f1640 1 -- 172.21.15.39:0/1532348942 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f93581029d0 msgr2=0x7f935819a470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:46:17.646 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:17.643+0000 7f935c9f1640 1 --2- 172.21.15.39:0/1532348942 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f93581029d0 0x7f935819a470 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f9340005940 tx=0x7f93400042f0 comp rx=0 tx=0).stop 2024-09-21T03:46:17.646 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:17.643+0000 7f935c9f1640 1 -- 172.21.15.39:0/1532348942 shutdown_connections 2024-09-21T03:46:17.646 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:17.643+0000 7f935c9f1640 1 --2- 172.21.15.39:0/1532348942 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f93440b65f0 0x7f93440b8b00 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:17.646 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:17.643+0000 7f935c9f1640 1 --2- 172.21.15.39:0/1532348942 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9358108c50 0x7f935819fca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:17.647 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:17.643+0000 7f935c9f1640 1 --2- 172.21.15.39:0/1532348942 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9358108300 0x7f935819aa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:17.647 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:17.643+0000 7f935c9f1640 1 --2- 172.21.15.39:0/1532348942 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f93581029d0 0x7f935819a470 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:17.647 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:17.643+0000 7f935c9f1640 1 -- 172.21.15.39:0/1532348942 >> 172.21.15.39:0/1532348942 conn(0x7f935807a6f0 msgr2=0x7f93581063f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:46:17.647 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:17.643+0000 7f935c9f1640 1 -- 172.21.15.39:0/1532348942 shutdown_connections 2024-09-21T03:46:17.647 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:17.643+0000 7f935c9f1640 1 -- 172.21.15.39:0/1532348942 wait complete. 2024-09-21T03:46:17.658 DEBUG:teuthology.orchestra.run.smithi039:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule retention add --fs=cephfs --path=/ --retention-spec-or-period=6m3h' 2024-09-21T03:46:17.779 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:17.774+0000 7f6b0efe6640 1 -- 172.21.15.39:0/3735913021 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6b08108c30 msgr2=0x7f6b0810f460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:46:17.779 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:17.774+0000 7f6b0efe6640 1 --2- 172.21.15.39:0/3735913021 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6b08108c30 0x7f6b0810f460 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f6b00009ca0 tx=0x7f6b0002a8c0 comp rx=0 tx=0).stop 2024-09-21T03:46:17.780 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:17.774+0000 7f6b0efe6640 1 -- 172.21.15.39:0/3735913021 shutdown_connections 2024-09-21T03:46:17.780 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:17.774+0000 7f6b0efe6640 1 --2- 172.21.15.39:0/3735913021 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6b08108c30 0x7f6b0810f460 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:17.780 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:17.774+0000 7f6b0efe6640 1 --2- 172.21.15.39:0/3735913021 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6b081082e0 0x7f6b081086f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:17.780 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:17.774+0000 7f6b0efe6640 1 --2- 172.21.15.39:0/3735913021 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6b081029b0 0x7f6b08102dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:17.780 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:17.774+0000 7f6b0efe6640 1 -- 172.21.15.39:0/3735913021 >> 172.21.15.39:0/3735913021 conn(0x7f6b0807a740 msgr2=0x7f6b0807ab50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:46:17.780 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:17.774+0000 7f6b0efe6640 1 -- 172.21.15.39:0/3735913021 shutdown_connections 2024-09-21T03:46:17.780 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:17.774+0000 7f6b0efe6640 1 -- 172.21.15.39:0/3735913021 wait complete. 2024-09-21T03:46:17.780 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:17.775+0000 7f6b0efe6640 1 Processor -- start 2024-09-21T03:46:17.780 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:17.775+0000 7f6b0efe6640 1 -- start start 2024-09-21T03:46:17.780 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:17.775+0000 7f6b0efe6640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6b081029b0 0x7f6b08109550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:17.781 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:17.775+0000 7f6b0efe6640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6b081082e0 0x7f6b08109a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:17.781 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:17.775+0000 7f6b0efe6640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6b08108c30 0x7f6b08109fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:17.781 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:17.775+0000 7f6b0efe6640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f6b081128f0 con 0x7f6b081029b0 2024-09-21T03:46:17.781 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:17.775+0000 7f6b0efe6640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f6b08112770 con 0x7f6b08108c30 2024-09-21T03:46:17.781 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:17.775+0000 7f6b0efe6640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f6b08112a70 con 0x7f6b081082e0 2024-09-21T03:46:17.781 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:17.776+0000 7f6b0cd82640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6b081029b0 0x7f6b08109550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:46:17.781 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:17.776+0000 7f6b0cd82640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6b081029b0 0x7f6b08109550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:33386/0 (socket says 172.21.15.39:33386) 2024-09-21T03:46:17.781 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:17.776+0000 7f6b0cd82640 1 -- 172.21.15.39:0/1842672114 learned_addr learned my addr 172.21.15.39:0/1842672114 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:46:17.781 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:17.776+0000 7f6b0d583640 1 --2- 172.21.15.39:0/1842672114 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6b08108c30 0x7f6b08109fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:46:17.781 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:17.776+0000 7f6b07fff640 1 --2- 172.21.15.39:0/1842672114 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6b081082e0 0x7f6b08109a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:46:17.781 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:17.776+0000 7f6b0cd82640 1 -- 172.21.15.39:0/1842672114 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6b081082e0 msgr2=0x7f6b08109a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:46:17.781 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:17.776+0000 7f6b0cd82640 1 --2- 172.21.15.39:0/1842672114 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6b081082e0 0x7f6b08109a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:17.781 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:17.776+0000 7f6b0cd82640 1 -- 172.21.15.39:0/1842672114 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6b08108c30 msgr2=0x7f6b08109fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:46:17.781 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:17.776+0000 7f6b0cd82640 1 --2- 172.21.15.39:0/1842672114 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6b08108c30 0x7f6b08109fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:17.782 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:17.776+0000 7f6b0cd82640 1 -- 172.21.15.39:0/1842672114 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6b081a8690 con 0x7f6b081029b0 2024-09-21T03:46:17.782 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:17.776+0000 7f6b07fff640 1 --2- 172.21.15.39:0/1842672114 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6b081082e0 0x7f6b08109a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:46:17.782 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:17.776+0000 7f6b0cd82640 1 --2- 172.21.15.39:0/1842672114 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6b081029b0 0x7f6b08109550 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f6afc00e9f0 tx=0x7f6afc00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:46:17.782 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:17.776+0000 7f6b05ffb640 1 -- 172.21.15.39:0/1842672114 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6afc00cd80 con 0x7f6b081029b0 2024-09-21T03:46:17.782 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:17.776+0000 7f6b05ffb640 1 -- 172.21.15.39:0/1842672114 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6afc0108e0 con 0x7f6b081029b0 2024-09-21T03:46:17.782 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:17.776+0000 7f6b0efe6640 1 -- 172.21.15.39:0/1842672114 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6b081a8980 con 0x7f6b081029b0 2024-09-21T03:46:17.782 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:17.776+0000 7f6b05ffb640 1 -- 172.21.15.39:0/1842672114 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6afc018980 con 0x7f6b081029b0 2024-09-21T03:46:17.782 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:17.777+0000 7f6b0efe6640 1 -- 172.21.15.39:0/1842672114 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6b081a8ed0 con 0x7f6b081029b0 2024-09-21T03:46:17.783 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:17.778+0000 7f6b0efe6640 1 -- 172.21.15.39:0/1842672114 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6b08102dc0 con 0x7f6b081029b0 2024-09-21T03:46:17.783 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:17.779+0000 7f6b05ffb640 1 -- 172.21.15.39:0/1842672114 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 24) ==== 154580+0+0 (secure 0 0 0) 0x7f6afc010430 con 0x7f6b081029b0 2024-09-21T03:46:17.783 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:17.780+0000 7f6b05ffb640 1 --2- 172.21.15.39:0/1842672114 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f6af80b6620 0x7f6af80b8b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:17.783 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:17.780+0000 7f6b07fff640 1 --2- 172.21.15.39:0/1842672114 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f6af80b6620 0x7f6af80b8b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:46:17.783 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:17.780+0000 7f6b05ffb640 1 -- 172.21.15.39:0/1842672114 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8484+0+0 (secure 0 0 0) 0x7f6afc0e79c0 con 0x7f6b081029b0 2024-09-21T03:46:17.783 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:17.781+0000 7f6b07fff640 1 --2- 172.21.15.39:0/1842672114 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f6af80b6620 0x7f6af80b8b30 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f6b081064c0 tx=0x7f6af4005f90 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:46:17.787 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:17.784+0000 7f6b05ffb640 1 -- 172.21.15.39:0/1842672114 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f6afc0adff0 con 0x7f6b081029b0 2024-09-21T03:46:17.870 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:17.851+0000 7f5ec9ced640 1 -- 172.21.15.39:0/3562211033 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5ec4108cf0 msgr2=0x7f5ec410f520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:46:17.871 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:17.851+0000 7f5ec9ced640 1 --2- 172.21.15.39:0/3562211033 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5ec4108cf0 0x7f5ec410f520 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f5ebc009900 tx=0x7f5ebc02a410 comp rx=0 tx=0).stop 2024-09-21T03:46:17.871 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:17.851+0000 7f5ec9ced640 1 -- 172.21.15.39:0/3562211033 shutdown_connections 2024-09-21T03:46:17.871 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:17.851+0000 7f5ec9ced640 1 --2- 172.21.15.39:0/3562211033 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5ec4108cf0 0x7f5ec410f520 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:17.871 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:17.851+0000 7f5ec9ced640 1 --2- 172.21.15.39:0/3562211033 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5ec41083a0 0x7f5ec41087b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:17.871 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:17.851+0000 7f5ec9ced640 1 --2- 172.21.15.39:0/3562211033 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5ec4102a60 0x7f5ec4102e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:17.871 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:17.851+0000 7f5ec9ced640 1 -- 172.21.15.39:0/3562211033 >> 172.21.15.39:0/3562211033 conn(0x7f5ec4079730 msgr2=0x7f5ec4079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:46:17.871 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:17.851+0000 7f5ec9ced640 1 -- 172.21.15.39:0/3562211033 shutdown_connections 2024-09-21T03:46:17.871 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:17.851+0000 7f5ec9ced640 1 -- 172.21.15.39:0/3562211033 wait complete. 2024-09-21T03:46:17.871 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:17.852+0000 7f5ec9ced640 1 Processor -- start 2024-09-21T03:46:17.871 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:17.852+0000 7f5ec9ced640 1 -- start start 2024-09-21T03:46:17.871 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:17.852+0000 7f5ec9ced640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5ec4102a60 0x7f5ec4199350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:17.871 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:17.852+0000 7f5ec9ced640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5ec41083a0 0x7f5ec41998f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:17.871 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:17.852+0000 7f5ec9ced640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5ec4108cf0 0x7f5ec419eb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:17.872 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:17.852+0000 7f5ec9ced640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f5ec4112960 con 0x7f5ec4108cf0 2024-09-21T03:46:17.872 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:17.852+0000 7f5ec9ced640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f5ec41127e0 con 0x7f5ec41083a0 2024-09-21T03:46:17.872 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:17.852+0000 7f5ec9ced640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f5ec4112ae0 con 0x7f5ec4102a60 2024-09-21T03:46:17.872 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:17.852+0000 7f5ec3fff640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5ec4108cf0 0x7f5ec419eb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:46:17.872 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:17.852+0000 7f5ec3fff640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5ec4108cf0 0x7f5ec419eb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:33408/0 (socket says 172.21.15.39:33408) 2024-09-21T03:46:17.872 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:17.852+0000 7f5ec3fff640 1 -- 172.21.15.39:0/3780889612 learned_addr learned my addr 172.21.15.39:0/3780889612 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:46:17.872 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:17.853+0000 7f5ec2ffd640 1 --2- 172.21.15.39:0/3780889612 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5ec41083a0 0x7f5ec41998f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:46:17.872 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:17.853+0000 7f5ec3fff640 1 -- 172.21.15.39:0/3780889612 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5ec4102a60 msgr2=0x7f5ec4199350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:46:17.872 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:17.853+0000 7f5ec3fff640 1 --2- 172.21.15.39:0/3780889612 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5ec4102a60 0x7f5ec4199350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:17.872 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:17.853+0000 7f5ec3fff640 1 -- 172.21.15.39:0/3780889612 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5ec41083a0 msgr2=0x7f5ec41998f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:46:17.872 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:17.853+0000 7f5ec3fff640 1 --2- 172.21.15.39:0/3780889612 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5ec41083a0 0x7f5ec41998f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:17.872 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:17.853+0000 7f5ec3fff640 1 -- 172.21.15.39:0/3780889612 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ec419f2c0 con 0x7f5ec4108cf0 2024-09-21T03:46:17.872 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:17.853+0000 7f5ec3fff640 1 --2- 172.21.15.39:0/3780889612 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5ec4108cf0 0x7f5ec419eb90 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f5ebc004770 tx=0x7f5ebc004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:46:17.872 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:17.853+0000 7f5ec0ff9640 1 -- 172.21.15.39:0/3780889612 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ebc046070 con 0x7f5ec4108cf0 2024-09-21T03:46:17.872 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:17.853+0000 7f5ec0ff9640 1 -- 172.21.15.39:0/3780889612 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5ebc003aa0 con 0x7f5ec4108cf0 2024-09-21T03:46:17.873 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:17.853+0000 7f5ec0ff9640 1 -- 172.21.15.39:0/3780889612 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ebc035050 con 0x7f5ec4108cf0 2024-09-21T03:46:17.873 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:17.853+0000 7f5ec9ced640 1 -- 172.21.15.39:0/3780889612 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5ec419f550 con 0x7f5ec4108cf0 2024-09-21T03:46:17.873 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:17.853+0000 7f5ec9ced640 1 -- 172.21.15.39:0/3780889612 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5ec406ae60 con 0x7f5ec4108cf0 2024-09-21T03:46:17.873 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:17.856+0000 7f5ed0261740 1 Processor -- start 2024-09-21T03:46:17.873 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:17.856+0000 7f5ec0ff9640 1 -- 172.21.15.39:0/3780889612 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 24) ==== 154580+0+0 (secure 0 0 0) 0x7f5ebc049070 con 0x7f5ec4108cf0 2024-09-21T03:46:17.873 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:17.856+0000 7f5ed0261740 1 -- start start 2024-09-21T03:46:17.873 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:17.857+0000 7f5ed0261740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5632bc745350 0x5632bcaec430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:17.873 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:17.857+0000 7f5ed0261740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5632bc71c490 0x5632bcaec970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:17.873 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:17.857+0000 7f5ed0261740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5632bc724670 0x5632bcaeceb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:17.873 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:17.857+0000 7f5ed0261740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x5632bca62e60 con 0x5632bc71c490 2024-09-21T03:46:17.873 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:17.857+0000 7f5ed0261740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x5632bca52290 con 0x5632bc724670 2024-09-21T03:46:17.873 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:17.857+0000 7f5ed0261740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x5632bca560a0 con 0x5632bc745350 2024-09-21T03:46:17.873 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:17.857+0000 7f5ec37fe640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5632bc71c490 0x5632bcaec970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:46:17.873 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:17.857+0000 7f5ec37fe640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5632bc71c490 0x5632bcaec970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:33416/0 (socket says 172.21.15.39:33416) 2024-09-21T03:46:17.874 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:17.857+0000 7f5ec37fe640 1 -- 172.21.15.39:0/2411438132 learned_addr learned my addr 172.21.15.39:0/2411438132 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:46:17.874 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:17.857+0000 7f5ec2ffd640 1 --2- 172.21.15.39:0/2411438132 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5632bc745350 0x5632bcaec430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:46:17.874 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:17.857+0000 7f5ec2ffd640 1 --2- 172.21.15.39:0/2411438132 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5632bc724670 0x5632bcaeceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:46:17.874 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:17.857+0000 7f5ec37fe640 1 -- 172.21.15.39:0/2411438132 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5632bc745350 msgr2=0x5632bcaec430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:46:17.874 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:17.857+0000 7f5ec37fe640 1 --2- 172.21.15.39:0/2411438132 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5632bc745350 0x5632bcaec430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:17.874 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:17.857+0000 7f5ec37fe640 1 -- 172.21.15.39:0/2411438132 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5632bc724670 msgr2=0x5632bcaeceb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:46:17.874 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:17.857+0000 7f5ec37fe640 1 --2- 172.21.15.39:0/2411438132 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5632bc724670 0x5632bcaeceb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:17.874 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:17.857+0000 7f5ec37fe640 1 -- 172.21.15.39:0/2411438132 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5632bc7b0930 con 0x5632bc71c490 2024-09-21T03:46:17.874 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:17.857+0000 7f5ec2ffd640 1 --2- 172.21.15.39:0/2411438132 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5632bc724670 0x5632bcaeceb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:46:17.874 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:17.857+0000 7f5ec0ff9640 1 --2- 172.21.15.39:0/3780889612 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f5eac0b6670 0x7f5eac0b8b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:17.874 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:17.857+0000 7f5ec37fe640 1 --2- 172.21.15.39:0/2411438132 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5632bc71c490 0x5632bcaec970 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f5eb4009ea0 tx=0x7f5eb4009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd8bb16e8630492e server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:46:17.874 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:17.857+0000 7f5ec2ffd640 1 --2- 172.21.15.39:0/3780889612 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f5eac0b6670 0x7f5eac0b8b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:46:17.874 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:17.858+0000 7f5ec0ff9640 1 -- 172.21.15.39:0/3780889612 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8484+0+0 (secure 0 0 0) 0x7f5ebc108f20 con 0x7f5ec4108cf0 2024-09-21T03:46:17.875 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:17.858+0000 7f5ea9ffb640 1 -- 172.21.15.39:0/2411438132 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5eb4004d20 con 0x5632bc71c490 2024-09-21T03:46:17.875 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:17.858+0000 7f5ea9ffb640 1 -- 172.21.15.39:0/2411438132 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5eb40054d0 con 0x5632bc71c490 2024-09-21T03:46:17.875 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:17.858+0000 7f5ea9ffb640 1 -- 172.21.15.39:0/2411438132 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5eb40107f0 con 0x5632bc71c490 2024-09-21T03:46:17.875 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:17.859+0000 7f5ed0261740 1 -- 172.21.15.39:0/2411438132 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5632bc71c490 msgr2=0x5632bcaec970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:46:17.875 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:17.859+0000 7f5ed0261740 1 --2- 172.21.15.39:0/2411438132 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5632bc71c490 0x5632bcaec970 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f5eb4009ea0 tx=0x7f5eb4009f90 comp rx=0 tx=0).stop 2024-09-21T03:46:17.875 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:17.859+0000 7f5ec2ffd640 1 --2- 172.21.15.39:0/3780889612 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f5eac0b6670 0x7f5eac0b8b80 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f5ec4105f20 tx=0x7f5eb800b390 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:46:17.875 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:17.859+0000 7f5ed0261740 1 -- 172.21.15.39:0/2411438132 shutdown_connections 2024-09-21T03:46:17.875 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:17.859+0000 7f5ed0261740 1 --2- 172.21.15.39:0/2411438132 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5632bc724670 0x5632bcaeceb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:17.875 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:17.859+0000 7f5ed0261740 1 --2- 172.21.15.39:0/2411438132 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5632bc71c490 0x5632bcaec970 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:17.875 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:17.859+0000 7f5ed0261740 1 --2- 172.21.15.39:0/2411438132 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5632bc745350 0x5632bcaec430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:17.875 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:17.859+0000 7f5ed0261740 1 -- 172.21.15.39:0/2411438132 >> 172.21.15.39:0/2411438132 conn(0x5632bc739740 msgr2=0x5632bcaea320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:46:17.875 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:17.859+0000 7f5ed0261740 1 -- 172.21.15.39:0/2411438132 shutdown_connections 2024-09-21T03:46:17.875 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:17.859+0000 7f5ed0261740 1 -- 172.21.15.39:0/2411438132 wait complete. 2024-09-21T03:46:17.875 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:17.859+0000 7f5ed0261740 1 -- start start 2024-09-21T03:46:17.875 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:17.860+0000 7f5ed0261740 1 Processor -- start 2024-09-21T03:46:17.876 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:17.860+0000 7f5ed0261740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5632bc71c490 0x5632bcaf7f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:17.876 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:17.860+0000 7f5ed0261740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5632bc724670 0x5632bcaf8480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:17.876 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:17.860+0000 7f5ed0261740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5632bc745350 0x5632bcb03f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:17.876 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:17.860+0000 7f5ed0261740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x5632bc8b3890 con 0x5632bc724670 2024-09-21T03:46:17.876 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:17.860+0000 7f5ed0261740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x5632bc9ecd40 con 0x5632bc71c490 2024-09-21T03:46:17.876 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:17.860+0000 7f5ed0261740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x5632bcafac10 con 0x5632bc745350 2024-09-21T03:46:17.876 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:17.860+0000 7f5ed0261740 10 client.0 _ll_register_callbacks cb 0x5632bca50c80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:46:17.876 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:17.860+0000 7f5ec2ffd640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5632bc724670 0x5632bcaf8480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:46:17.876 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:17.860+0000 7f5ec2ffd640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5632bc724670 0x5632bcaf8480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:33426/0 (socket says 172.21.15.39:33426) 2024-09-21T03:46:17.876 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:17.860+0000 7f5ec2ffd640 1 -- 172.21.15.39:0/2516933348 learned_addr learned my addr 172.21.15.39:0/2516933348 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:46:17.876 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:17.860+0000 7f5ec3fff640 1 --2- 172.21.15.39:0/2516933348 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5632bc745350 0x5632bcb03f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:46:17.876 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:17.860+0000 7f5ec37fe640 1 --2- 172.21.15.39:0/2516933348 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5632bc71c490 0x5632bcaf7f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:46:17.876 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:17.861+0000 7f5ec2ffd640 1 -- 172.21.15.39:0/2516933348 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5632bc745350 msgr2=0x5632bcb03f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:46:17.876 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:17.861+0000 7f5ec2ffd640 1 --2- 172.21.15.39:0/2516933348 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5632bc745350 0x5632bcb03f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:17.877 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:17.861+0000 7f5ec2ffd640 1 -- 172.21.15.39:0/2516933348 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5632bc71c490 msgr2=0x5632bcaf7f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:46:17.877 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:17.861+0000 7f5ec2ffd640 1 --2- 172.21.15.39:0/2516933348 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5632bc71c490 0x5632bcaf7f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:17.877 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:17.861+0000 7f5ec2ffd640 1 -- 172.21.15.39:0/2516933348 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5632bcb04440 con 0x5632bc724670 2024-09-21T03:46:17.877 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:17.861+0000 7f5ec3fff640 1 --2- 172.21.15.39:0/2516933348 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5632bc745350 0x5632bcb03f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:46:17.877 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:17.861+0000 7f5ec2ffd640 1 --2- 172.21.15.39:0/2516933348 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5632bc724670 0x5632bcaf8480 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f5eb8005d00 tx=0x7f5eb8005c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f9a23738ab29f3ad server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:46:17.877 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:17.861+0000 7f5ea2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-09-21T03:46:17.877 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:17.861+0000 7f5ea2ffd640 1 -- 172.21.15.39:0/2516933348 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5eb801ea40 con 0x5632bc724670 2024-09-21T03:46:17.877 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:17.861+0000 7f5ea2ffd640 1 -- 172.21.15.39:0/2516933348 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8484+0+0 (secure 0 0 0) 0x7f5eb801ebe0 con 0x5632bc724670 2024-09-21T03:46:17.878 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:17.861+0000 7f5ed0261740 1 -- 172.21.15.39:0/2516933348 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5eb40054d0 con 0x5632bc724670 2024-09-21T03:46:17.878 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:17.861+0000 7f5ea2ffd640 1 -- 172.21.15.39:0/2516933348 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f5eb801ede0 con 0x5632bc724670 2024-09-21T03:46:17.878 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:17.861+0000 7f5ed0261740 10 client.15579 fetch_fsmap learned FSMap version 29 2024-09-21T03:46:17.878 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:17.861+0000 7f5ed0261740 1 -- 172.21.15.39:0/2516933348 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5632bcb06d10 con 0x5632bc724670 2024-09-21T03:46:17.878 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:17.862+0000 7f5ea2ffd640 1 -- 172.21.15.39:0/2516933348 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5eb40054d0 con 0x5632bc724670 2024-09-21T03:46:17.878 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:17.862+0000 7f5ea2ffd640 1 -- 172.21.15.39:0/2516933348 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f5eb802e020 con 0x5632bc724670 2024-09-21T03:46:17.878 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:17.862+0000 7f5ed0261740 10 client.15579 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T03:46:17.878 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:17.862+0000 7f5ed0261740 10 client.15579 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T03:46:17.878 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:17.862+0000 7f5ed0261740 20 client.15579 populate_metadata read hostname 'smithi039' 2024-09-21T03:46:17.878 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:17.862+0000 7f5ed0261740 1 --2- 172.21.15.39:0/2516933348 >> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] conn(0x5632bc703cd0 0x5632bcb0a320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:17.878 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:17.862+0000 7f5ed0261740 4 client.15579 mds_command: new command op to 24695 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:46:17.878 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:17.862+0000 7f5ed0261740 1 -- 172.21.15.39:0/2516933348 --> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5632bca64db0 con 0x5632bc703cd0 2024-09-21T03:46:17.878 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:17.862+0000 7f5ec37fe640 1 --2- 172.21.15.39:0/2516933348 >> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] conn(0x5632bc703cd0 0x5632bcb0a320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:46:17.879 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:17.863+0000 7f5ec37fe640 1 --2- 172.21.15.39:0/2516933348 >> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] conn(0x5632bc703cd0 0x5632bcb0a320 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=998ee886044e31ca server_cookie=5c7105490e61892b in_seq=0 out_seq=0 2024-09-21T03:46:17.879 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:17.863+0000 7f5ea2ffd640 10 client.15579 ms_handle_connect on v2:172.21.15.150:6838/2949386907 2024-09-21T03:46:17.879 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:17.863+0000 7f5ea2ffd640 1 -- 172.21.15.39:0/2516933348 <== mds.0 v2:172.21.15.150:6838/2949386907 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5632bca64db0 con 0x5632bc703cd0 2024-09-21T03:46:17.879 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:17.863+0000 7f5ea2ffd640 10 client.15579 handle_command_reply: tid=0 2024-09-21T03:46:17.879 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:17.863+0000 7f5ed0261740 1 client.15579 shutdown 2024-09-21T03:46:17.879 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:17.863+0000 7f5ed0261740 1 -- 172.21.15.39:0/2516933348 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5632bc724670 msgr2=0x5632bcaf8480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:46:17.879 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:17.864+0000 7f5ed0261740 1 --2- 172.21.15.39:0/2516933348 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5632bc724670 0x5632bcaf8480 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f5eb8005d00 tx=0x7f5eb8005c90 comp rx=0 tx=0).stop 2024-09-21T03:46:17.879 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:17.864+0000 7f5ed0261740 1 -- 172.21.15.39:0/2516933348 shutdown_connections 2024-09-21T03:46:17.879 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:17.864+0000 7f5ed0261740 1 --2- 172.21.15.39:0/2516933348 >> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] conn(0x5632bc703cd0 0x5632bcb0a320 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:17.879 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:17.864+0000 7f5ea2ffd640 0 client.15579 ms_handle_reset on v2:172.21.15.150:6838/2949386907 2024-09-21T03:46:17.879 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:17.864+0000 7f5ed0261740 1 --2- 172.21.15.39:0/2516933348 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5632bc745350 0x5632bcb03f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:17.879 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:17.864+0000 7f5ed0261740 1 --2- 172.21.15.39:0/2516933348 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5632bc724670 0x5632bcaf8480 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:17.879 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:17.864+0000 7f5ed0261740 1 --2- 172.21.15.39:0/2516933348 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5632bc71c490 0x5632bcaf7f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:17.879 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:17.864+0000 7f5ed0261740 1 -- 172.21.15.39:0/2516933348 >> 172.21.15.39:0/2516933348 conn(0x5632bc739740 msgr2=0x5632bcaef5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:46:17.880 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:17.864+0000 7f5ed0261740 1 -- 172.21.15.39:0/2516933348 shutdown_connections 2024-09-21T03:46:17.880 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:17.864+0000 7f5ed0261740 1 -- 172.21.15.39:0/2516933348 wait complete. 2024-09-21T03:46:17.880 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:17.864+0000 7f5ed0261740 20 client.15579 trim_cache size 0 max 16384 2024-09-21T03:46:17.884 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:17.882+0000 7f5ed0261740 1 Processor -- start 2024-09-21T03:46:17.884 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:17.882+0000 7f5ed0261740 1 -- start start 2024-09-21T03:46:17.884 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:17.882+0000 7f5ed0261740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5632bc703cd0 0x5632bcb03e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:17.885 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:17.882+0000 7f5ed0261740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5632bc71c490 0x5632bcaf2550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:17.885 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:17.882+0000 7f5ed0261740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5632bc724670 0x5632bcaf4f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:17.885 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:17.882+0000 7f5ed0261740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f5ec41ad520 con 0x5632bc71c490 2024-09-21T03:46:17.885 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:17.882+0000 7f5ed0261740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f5ec41af060 con 0x5632bc703cd0 2024-09-21T03:46:17.885 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:17.882+0000 7f5ed0261740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f5ec41adbf0 con 0x5632bc724670 2024-09-21T03:46:17.886 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:17.882+0000 7f5ec2ffd640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5632bc71c490 0x5632bcaf2550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:46:17.886 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:17.882+0000 7f5ec2ffd640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5632bc71c490 0x5632bcaf2550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:33442/0 (socket says 172.21.15.39:33442) 2024-09-21T03:46:17.886 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:17.882+0000 7f5ec2ffd640 1 -- 172.21.15.39:0/790098993 learned_addr learned my addr 172.21.15.39:0/790098993 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:46:17.886 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:17.882+0000 7f5ec37fe640 1 --2- 172.21.15.39:0/790098993 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5632bc703cd0 0x5632bcb03e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:46:17.886 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:17.882+0000 7f5ec3fff640 1 --2- 172.21.15.39:0/790098993 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5632bc724670 0x5632bcaf4f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:46:17.886 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:17.882+0000 7f5ec2ffd640 1 -- 172.21.15.39:0/790098993 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5632bc724670 msgr2=0x5632bcaf4f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:46:17.886 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:17.882+0000 7f5ec2ffd640 1 --2- 172.21.15.39:0/790098993 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5632bc724670 0x5632bcaf4f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:17.886 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:17.882+0000 7f5ec2ffd640 1 -- 172.21.15.39:0/790098993 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5632bc703cd0 msgr2=0x5632bcb03e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:46:17.886 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:17.882+0000 7f5ec2ffd640 1 --2- 172.21.15.39:0/790098993 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5632bc703cd0 0x5632bcb03e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:17.887 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:17.883+0000 7f5ec2ffd640 1 -- 172.21.15.39:0/790098993 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5632bcaf35d0 con 0x5632bc71c490 2024-09-21T03:46:17.887 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:17.883+0000 7f5ec2ffd640 1 --2- 172.21.15.39:0/790098993 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5632bc71c490 0x5632bcaf2550 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f5eb801c570 tx=0x7f5eb80103d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=88beed08ba7e35c4 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:46:17.887 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:17.883+0000 7f5ea97fa640 1 -- 172.21.15.39:0/790098993 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5eb801ea40 con 0x5632bc71c490 2024-09-21T03:46:17.887 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:17.883+0000 7f5ea97fa640 1 -- 172.21.15.39:0/790098993 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5eb801ebe0 con 0x5632bc71c490 2024-09-21T03:46:17.887 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:17.883+0000 7f5ea97fa640 1 -- 172.21.15.39:0/790098993 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5eb8004640 con 0x5632bc71c490 2024-09-21T03:46:17.889 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:17.884+0000 7f5ed0261740 1 -- 172.21.15.39:0/790098993 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5632bc71c490 msgr2=0x5632bcaf2550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:46:17.889 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:17.884+0000 7f5ed0261740 1 --2- 172.21.15.39:0/790098993 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5632bc71c490 0x5632bcaf2550 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f5eb801c570 tx=0x7f5eb80103d0 comp rx=0 tx=0).stop 2024-09-21T03:46:17.889 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:17.884+0000 7f5ed0261740 1 -- 172.21.15.39:0/790098993 shutdown_connections 2024-09-21T03:46:17.889 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:17.884+0000 7f5ed0261740 1 --2- 172.21.15.39:0/790098993 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5632bc724670 0x5632bcaf4f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:17.889 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:17.884+0000 7f5ed0261740 1 --2- 172.21.15.39:0/790098993 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5632bc71c490 0x5632bcaf2550 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:17.889 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:17.884+0000 7f5ed0261740 1 --2- 172.21.15.39:0/790098993 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5632bc703cd0 0x5632bcb03e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:17.889 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:17.884+0000 7f5ed0261740 1 -- 172.21.15.39:0/790098993 >> 172.21.15.39:0/790098993 conn(0x5632bc739740 msgr2=0x5632bcaf65c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:46:17.889 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:17.884+0000 7f5ed0261740 1 -- 172.21.15.39:0/790098993 shutdown_connections 2024-09-21T03:46:17.889 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:17.884+0000 7f5ed0261740 1 -- 172.21.15.39:0/790098993 wait complete. 2024-09-21T03:46:17.889 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:17.884+0000 7f5ed0261740 1 -- start start 2024-09-21T03:46:17.890 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:17.885+0000 7f5ed0261740 1 Processor -- start 2024-09-21T03:46:17.890 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:17.885+0000 7f5ed0261740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5632bc703cd0 0x5632bcafc6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:17.890 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:17.885+0000 7f5ed0261740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5632bc71c490 0x5632bcafccc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:17.890 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:17.885+0000 7f5ed0261740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5632bc724670 0x5632bcb720c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:17.890 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:17.885+0000 7f5ed0261740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f5ec41ae990 con 0x5632bc71c490 2024-09-21T03:46:17.890 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:17.885+0000 7f5ed0261740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f5ec41ae2c0 con 0x5632bc703cd0 2024-09-21T03:46:17.890 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:17.885+0000 7f5ed0261740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f5ec41ab500 con 0x5632bc724670 2024-09-21T03:46:17.890 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:17.885+0000 7f5ed0261740 10 client.0 _ll_register_callbacks cb 0x7f5eb800bc10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:46:17.890 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:17.885+0000 7f5ec2ffd640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5632bc71c490 0x5632bcafccc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:46:17.890 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:17.885+0000 7f5ec2ffd640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5632bc71c490 0x5632bcafccc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:33456/0 (socket says 172.21.15.39:33456) 2024-09-21T03:46:17.890 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:17.885+0000 7f5ec2ffd640 1 -- 172.21.15.39:0/3421503650 learned_addr learned my addr 172.21.15.39:0/3421503650 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:46:17.890 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:17.885+0000 7f5ec3fff640 1 --2- 172.21.15.39:0/3421503650 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5632bc724670 0x5632bcb720c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:46:17.890 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:17.885+0000 7f5ec37fe640 1 --2- 172.21.15.39:0/3421503650 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5632bc703cd0 0x5632bcafc6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:46:17.890 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:17.886+0000 7f5ec2ffd640 1 -- 172.21.15.39:0/3421503650 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5632bc724670 msgr2=0x5632bcb720c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:46:17.891 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:17.886+0000 7f5ec2ffd640 1 --2- 172.21.15.39:0/3421503650 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5632bc724670 0x5632bcb720c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:17.891 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:17.886+0000 7f5ec2ffd640 1 -- 172.21.15.39:0/3421503650 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5632bc703cd0 msgr2=0x5632bcafc6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:46:17.891 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:17.886+0000 7f5ec2ffd640 1 --2- 172.21.15.39:0/3421503650 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5632bc703cd0 0x5632bcafc6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:17.891 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:17.886+0000 7f5ec2ffd640 1 -- 172.21.15.39:0/3421503650 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5632bcb72730 con 0x5632bc71c490 2024-09-21T03:46:17.891 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:17.886+0000 7f5ec2ffd640 1 --2- 172.21.15.39:0/3421503650 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5632bc71c490 0x5632bcafccc0 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f5eb801ca50 tx=0x7f5eb801ce00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d7cab09a18f9485a server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:46:17.891 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:17.886+0000 7f5ea2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-09-21T03:46:17.891 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:17.886+0000 7f5ea2ffd640 1 -- 172.21.15.39:0/3421503650 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5eb8005240 con 0x5632bc71c490 2024-09-21T03:46:17.891 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:17.886+0000 7f5ea2ffd640 1 -- 172.21.15.39:0/3421503650 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8484+0+0 (secure 0 0 0) 0x7f5eb80053e0 con 0x5632bc71c490 2024-09-21T03:46:17.891 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:17.886+0000 7f5ed0261740 1 -- 172.21.15.39:0/3421503650 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5eb801ebe0 con 0x5632bc71c490 2024-09-21T03:46:17.891 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:17.886+0000 7f5ea2ffd640 1 -- 172.21.15.39:0/3421503650 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f5eb8048050 con 0x5632bc71c490 2024-09-21T03:46:17.891 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:17.886+0000 7f5ed0261740 10 client.15585 fetch_fsmap learned FSMap version 29 2024-09-21T03:46:17.891 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:17.886+0000 7f5ed0261740 1 -- 172.21.15.39:0/3421503650 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5632bcb79a40 con 0x5632bc71c490 2024-09-21T03:46:17.891 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:17.886+0000 7f5ea2ffd640 1 -- 172.21.15.39:0/3421503650 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5eb801ebe0 con 0x5632bc71c490 2024-09-21T03:46:17.892 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:17.887+0000 7f5ea2ffd640 1 -- 172.21.15.39:0/3421503650 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f5eb8038030 con 0x5632bc71c490 2024-09-21T03:46:17.893 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:17.887+0000 7f5ed0261740 10 client.15585 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T03:46:17.893 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:17.887+0000 7f5ed0261740 10 client.15585 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T03:46:17.893 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:17.887+0000 7f5ed0261740 20 client.15585 populate_metadata read hostname 'smithi039' 2024-09-21T03:46:17.893 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:17.887+0000 7f5ed0261740 1 --2- 172.21.15.39:0/3421503650 >> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] conn(0x5632bc745350 0x5632bcb7c100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:17.893 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:17.887+0000 7f5ed0261740 4 client.15585 mds_command: new command op to 24695 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-21T03:46:17.893 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:17.887+0000 7f5ed0261740 1 -- 172.21.15.39:0/3421503650 --> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x5632bca60860 con 0x5632bc745350 2024-09-21T03:46:17.893 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:17.887+0000 7f5ec37fe640 1 --2- 172.21.15.39:0/3421503650 >> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] conn(0x5632bc745350 0x5632bcb7c100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:46:17.893 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:17.888+0000 7f5ea2ffd640 10 client.15585 ms_handle_connect on v2:172.21.15.150:6838/2949386907 2024-09-21T03:46:17.893 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:17.888+0000 7f5ec37fe640 1 --2- 172.21.15.39:0/3421503650 >> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] conn(0x5632bc745350 0x5632bcb7c100 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=af0e56d4709697ae server_cookie=6a2e215fcc6b3943 in_seq=0 out_seq=0 2024-09-21T03:46:17.893 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:17.890+0000 7f5ea2ffd640 1 -- 172.21.15.39:0/3421503650 <== mds.0 v2:172.21.15.150:6838/2949386907 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x5632bca60860 con 0x5632bc745350 2024-09-21T03:46:17.893 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:17.890+0000 7f5ea2ffd640 10 client.15585 handle_command_reply: tid=0 2024-09-21T03:46:17.893 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:17.890+0000 7f5ed0261740 1 client.15585 shutdown 2024-09-21T03:46:17.893 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:17.890+0000 7f5ed0261740 1 -- 172.21.15.39:0/3421503650 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5632bc71c490 msgr2=0x5632bcafccc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:46:17.894 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:17.890+0000 7f5ed0261740 1 --2- 172.21.15.39:0/3421503650 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5632bc71c490 0x5632bcafccc0 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f5eb801ca50 tx=0x7f5eb801ce00 comp rx=0 tx=0).stop 2024-09-21T03:46:17.894 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:17.890+0000 7f5ed0261740 1 -- 172.21.15.39:0/3421503650 shutdown_connections 2024-09-21T03:46:17.894 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:17.890+0000 7f5ed0261740 1 --2- 172.21.15.39:0/3421503650 >> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] conn(0x5632bc745350 0x5632bcb7c100 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:17.894 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:17.890+0000 7f5ea2ffd640 0 client.15585 ms_handle_reset on v2:172.21.15.150:6838/2949386907 2024-09-21T03:46:17.895 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:17.890+0000 7f5ed0261740 1 --2- 172.21.15.39:0/3421503650 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5632bc724670 0x5632bcb720c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:17.895 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:17.890+0000 7f5ed0261740 1 --2- 172.21.15.39:0/3421503650 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5632bc71c490 0x5632bcafccc0 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:17.895 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:17.890+0000 7f5ed0261740 1 --2- 172.21.15.39:0/3421503650 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5632bc703cd0 0x5632bcafc6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:17.895 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:17.890+0000 7f5ed0261740 1 -- 172.21.15.39:0/3421503650 >> 172.21.15.39:0/3421503650 conn(0x5632bc739740 msgr2=0x5632bcaed460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:46:17.895 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:17.890+0000 7f5ed0261740 1 -- 172.21.15.39:0/3421503650 shutdown_connections 2024-09-21T03:46:17.895 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:17.890+0000 7f5ed0261740 1 -- 172.21.15.39:0/3421503650 wait complete. 2024-09-21T03:46:17.895 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:17.891+0000 7f5ed0261740 20 client.15585 trim_cache size 0 max 16384 2024-09-21T03:46:17.895 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-09-21T03:46:17.895 INFO:teuthology.orchestra.run.smithi039.stdout: "return_code": 0, 2024-09-21T03:46:17.895 INFO:teuthology.orchestra.run.smithi039.stdout: "scrub_tag": "02509e4f-3911-4983-b16f-df8ded0b9998", 2024-09-21T03:46:17.895 INFO:teuthology.orchestra.run.smithi039.stdout: "mode": "asynchronous" 2024-09-21T03:46:17.896 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-09-21T03:46:17.896 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:17.891+0000 7f5ec9ced640 1 -- 172.21.15.39:0/3780889612 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f5eac0b6670 msgr2=0x7f5eac0b8b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:46:17.896 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:17.891+0000 7f5ec9ced640 1 --2- 172.21.15.39:0/3780889612 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f5eac0b6670 0x7f5eac0b8b80 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f5ec4105f20 tx=0x7f5eb800b390 comp rx=0 tx=0).stop 2024-09-21T03:46:17.896 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:17.891+0000 7f5ec9ced640 1 -- 172.21.15.39:0/3780889612 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5ec4108cf0 msgr2=0x7f5ec419eb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:46:17.896 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:17.891+0000 7f5ec9ced640 1 --2- 172.21.15.39:0/3780889612 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5ec4108cf0 0x7f5ec419eb90 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f5ebc004770 tx=0x7f5ebc004270 comp rx=0 tx=0).stop 2024-09-21T03:46:17.896 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:17.891+0000 7f5ec9ced640 1 -- 172.21.15.39:0/3780889612 shutdown_connections 2024-09-21T03:46:17.896 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:17.891+0000 7f5ec9ced640 1 --2- 172.21.15.39:0/3780889612 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f5eac0b6670 0x7f5eac0b8b80 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:17.896 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:17.891+0000 7f5ec9ced640 1 --2- 172.21.15.39:0/3780889612 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5ec4108cf0 0x7f5ec419eb90 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:17.896 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:17.891+0000 7f5ec9ced640 1 --2- 172.21.15.39:0/3780889612 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5ec41083a0 0x7f5ec41998f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:17.897 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:17.891+0000 7f5ec9ced640 1 --2- 172.21.15.39:0/3780889612 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5ec4102a60 0x7f5ec4199350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:17.897 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:17.891+0000 7f5ec9ced640 1 -- 172.21.15.39:0/3780889612 >> 172.21.15.39:0/3780889612 conn(0x7f5ec4079730 msgr2=0x7f5ec41043b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:46:17.897 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:17.892+0000 7f5ec9ced640 1 -- 172.21.15.39:0/3780889612 shutdown_connections 2024-09-21T03:46:17.897 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:17.892+0000 7f5ec9ced640 1 -- 172.21.15.39:0/3780889612 wait complete. 2024-09-21T03:46:18.018 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:18.015+0000 7f6b0efe6640 1 -- 172.21.15.39:0/1842672114 --> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] -- mgr_command(tid 0: {"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}) -- 0x7f6b08101bf0 con 0x7f6af80b6620 2024-09-21T03:46:18.065 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:18.062+0000 7f6b05ffb640 1 -- 172.21.15.39:0/1842672114 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mgrmap(e 25) ==== 154637+0+0 (secure 0 0 0) 0x7f6afc0a9b60 con 0x7f6b081029b0 2024-09-21T03:46:18.093 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:18.090+0000 7f6b05ffb640 1 -- 172.21.15.39:0/1842672114 <== mgr.14223 v2:172.21.15.39:6800/1696381982 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+26 (secure 0 0 0) 0x7f6b08101bf0 con 0x7f6af80b6620 2024-09-21T03:46:18.093 INFO:teuthology.orchestra.run.smithi039.stdout:Retention added to path / 2024-09-21T03:46:18.095 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:18.092+0000 7f6b0efe6640 1 -- 172.21.15.39:0/1842672114 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f6af80b6620 msgr2=0x7f6af80b8b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:46:18.095 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:18.092+0000 7f6b0efe6640 1 --2- 172.21.15.39:0/1842672114 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f6af80b6620 0x7f6af80b8b30 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f6b081064c0 tx=0x7f6af4005f90 comp rx=0 tx=0).stop 2024-09-21T03:46:18.095 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:18.092+0000 7f6b0efe6640 1 -- 172.21.15.39:0/1842672114 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6b081029b0 msgr2=0x7f6b08109550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:46:18.095 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:18.092+0000 7f6b0efe6640 1 --2- 172.21.15.39:0/1842672114 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6b081029b0 0x7f6b08109550 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f6afc00e9f0 tx=0x7f6afc00eeb0 comp rx=0 tx=0).stop 2024-09-21T03:46:18.096 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:18.092+0000 7f6b0efe6640 1 -- 172.21.15.39:0/1842672114 shutdown_connections 2024-09-21T03:46:18.096 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:18.092+0000 7f6b0efe6640 1 --2- 172.21.15.39:0/1842672114 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f6af80b6620 0x7f6af80b8b30 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:18.096 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:18.092+0000 7f6b0efe6640 1 --2- 172.21.15.39:0/1842672114 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6b08108c30 0x7f6b08109fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:18.096 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:18.092+0000 7f6b0efe6640 1 --2- 172.21.15.39:0/1842672114 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6b081082e0 0x7f6b08109a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:18.096 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:18.092+0000 7f6b0efe6640 1 --2- 172.21.15.39:0/1842672114 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6b081029b0 0x7f6b08109550 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:18.096 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:18.092+0000 7f6b0efe6640 1 -- 172.21.15.39:0/1842672114 >> 172.21.15.39:0/1842672114 conn(0x7f6b0807a740 msgr2=0x7f6b08101440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:46:18.096 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:18.093+0000 7f6b0efe6640 1 -- 172.21.15.39:0/1842672114 shutdown_connections 2024-09-21T03:46:18.096 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:18.093+0000 7f6b0efe6640 1 -- 172.21.15.39:0/1842672114 wait complete. 2024-09-21T03:46:18.107 DEBUG:teuthology.orchestra.run.smithi039:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule status --fs=cephfs --path=/' 2024-09-21T03:46:18.224 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:18.221+0000 7fa9b50bd640 1 -- 172.21.15.39:0/2144780867 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa9b0109d80 msgr2=0x7fa9b01105b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:46:18.224 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:18.221+0000 7fa9b50bd640 1 --2- 172.21.15.39:0/2144780867 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa9b0109d80 0x7fa9b01105b0 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7fa9a8009ca0 tx=0x7fa9a802a8c0 comp rx=0 tx=0).stop 2024-09-21T03:46:18.224 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:18.221+0000 7fa9b50bd640 1 -- 172.21.15.39:0/2144780867 shutdown_connections 2024-09-21T03:46:18.224 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:18.221+0000 7fa9b50bd640 1 --2- 172.21.15.39:0/2144780867 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa9b0109d80 0x7fa9b01105b0 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:18.224 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:18.221+0000 7fa9b50bd640 1 --2- 172.21.15.39:0/2144780867 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa9b0109430 0x7fa9b0109840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:18.224 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:18.221+0000 7fa9b50bd640 1 --2- 172.21.15.39:0/2144780867 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa9b0103b00 0x7fa9b0103f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:18.224 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:18.221+0000 7fa9b50bd640 1 -- 172.21.15.39:0/2144780867 >> 172.21.15.39:0/2144780867 conn(0x7fa9b00ff500 msgr2=0x7fa9b0101950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:46:18.224 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:18.221+0000 7fa9b50bd640 1 -- 172.21.15.39:0/2144780867 shutdown_connections 2024-09-21T03:46:18.224 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:18.221+0000 7fa9b50bd640 1 -- 172.21.15.39:0/2144780867 wait complete. 2024-09-21T03:46:18.226 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:18.222+0000 7fa9b50bd640 1 Processor -- start 2024-09-21T03:46:18.226 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:18.222+0000 7fa9b50bd640 1 -- start start 2024-09-21T03:46:18.226 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:18.222+0000 7fa9b50bd640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa9b0103b00 0x7fa9b019a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:18.227 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:18.222+0000 7fa9b50bd640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa9b0109430 0x7fa9b019a9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:18.227 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:18.222+0000 7fa9b50bd640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa9b0109d80 0x7fa9b019fc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:18.227 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:18.222+0000 7fa9b50bd640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fa9b0113a90 con 0x7fa9b0103b00 2024-09-21T03:46:18.227 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:18.222+0000 7fa9b50bd640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fa9b0113910 con 0x7fa9b0109430 2024-09-21T03:46:18.227 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:18.222+0000 7fa9b50bd640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fa9b0113c10 con 0x7fa9b0109d80 2024-09-21T03:46:18.227 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:18.222+0000 7fa9aed9d640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa9b0103b00 0x7fa9b019a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:46:18.227 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:18.222+0000 7fa9aed9d640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa9b0103b00 0x7fa9b019a430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:33480/0 (socket says 172.21.15.39:33480) 2024-09-21T03:46:18.227 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:18.222+0000 7fa9aed9d640 1 -- 172.21.15.39:0/3202380205 learned_addr learned my addr 172.21.15.39:0/3202380205 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:46:18.227 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:18.222+0000 7fa9af59e640 1 --2- 172.21.15.39:0/3202380205 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa9b0109d80 0x7fa9b019fc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:46:18.227 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:18.222+0000 7fa9ae59c640 1 --2- 172.21.15.39:0/3202380205 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa9b0109430 0x7fa9b019a9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:46:18.227 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:18.223+0000 7fa9aed9d640 1 -- 172.21.15.39:0/3202380205 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa9b0109d80 msgr2=0x7fa9b019fc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:46:18.227 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:18.223+0000 7fa9aed9d640 1 --2- 172.21.15.39:0/3202380205 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa9b0109d80 0x7fa9b019fc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:18.227 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:18.223+0000 7fa9aed9d640 1 -- 172.21.15.39:0/3202380205 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa9b0109430 msgr2=0x7fa9b019a9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:46:18.227 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:18.223+0000 7fa9aed9d640 1 --2- 172.21.15.39:0/3202380205 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa9b0109430 0x7fa9b019a9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:18.228 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:18.223+0000 7fa9aed9d640 1 -- 172.21.15.39:0/3202380205 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa9b01a0360 con 0x7fa9b0103b00 2024-09-21T03:46:18.228 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:18.223+0000 7fa9aed9d640 1 --2- 172.21.15.39:0/3202380205 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa9b0103b00 0x7fa9b019a430 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7fa9a000d950 tx=0x7fa9a000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:46:18.228 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:18.223+0000 7fa997fff640 1 -- 172.21.15.39:0/3202380205 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9a0017030 con 0x7fa9b0103b00 2024-09-21T03:46:18.228 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:18.223+0000 7fa997fff640 1 -- 172.21.15.39:0/3202380205 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa9a0004020 con 0x7fa9b0103b00 2024-09-21T03:46:18.228 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:18.223+0000 7fa9b50bd640 1 -- 172.21.15.39:0/3202380205 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa9b01a0650 con 0x7fa9b0103b00 2024-09-21T03:46:18.228 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:18.223+0000 7fa997fff640 1 -- 172.21.15.39:0/3202380205 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9a00047a0 con 0x7fa9b0103b00 2024-09-21T03:46:18.228 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:18.223+0000 7fa9b50bd640 1 -- 172.21.15.39:0/3202380205 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa9b01a6de0 con 0x7fa9b0103b00 2024-09-21T03:46:18.228 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:18.224+0000 7fa9b50bd640 1 -- 172.21.15.39:0/3202380205 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa9b0103f10 con 0x7fa9b0103b00 2024-09-21T03:46:18.228 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:18.226+0000 7fa997fff640 1 -- 172.21.15.39:0/3202380205 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 25) ==== 154637+0+0 (secure 0 0 0) 0x7fa9a0009a30 con 0x7fa9b0103b00 2024-09-21T03:46:18.229 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:18.227+0000 7fa997fff640 1 --2- 172.21.15.39:0/3202380205 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7fa9980b6680 0x7fa9980b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:18.229 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:18.227+0000 7fa9ae59c640 1 --2- 172.21.15.39:0/3202380205 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7fa9980b6680 0x7fa9980b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:46:18.229 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:18.227+0000 7fa997fff640 1 -- 172.21.15.39:0/3202380205 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8484+0+0 (secure 0 0 0) 0x7fa9a00e7760 con 0x7fa9b0103b00 2024-09-21T03:46:18.230 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:18.227+0000 7fa9ae59c640 1 --2- 172.21.15.39:0/3202380205 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7fa9980b6680 0x7fa9980b8b90 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fa9b0102d50 tx=0x7fa9a4005f90 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:46:18.232 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:18.230+0000 7fa997fff640 1 -- 172.21.15.39:0/3202380205 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fa9a00adcd0 con 0x7fa9b0103b00 2024-09-21T03:46:18.443 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:18.441+0000 7fa9b50bd640 1 -- 172.21.15.39:0/3202380205 --> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}) -- 0x7fa9b0063800 con 0x7fa9980b6680 2024-09-21T03:46:18.451 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:18.449+0000 7fa997fff640 1 -- 172.21.15.39:0/3202380205 <== mgr.14223 v2:172.21.15.39:6800/1696381982 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+298 (secure 0 0 0) 0x7fa9b0063800 con 0x7fa9980b6680 2024-09-21T03:46:18.451 INFO:teuthology.orchestra.run.smithi039.stdout:{"fs": "cephfs", "subvol": null, "group": null, "path": "/", "rel_path": "/", "schedule": "1m", "retention": {"m": 6, "h": 3}, "start": "2024-09-21T00:00:00", "created": "2024-09-21T03:46:17", "first": null, "last": null, "last_pruned": null, "created_count": 0, "pruned_count": 0, "active": true} 2024-09-21T03:46:18.454 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:18.452+0000 7fa9b50bd640 1 -- 172.21.15.39:0/3202380205 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7fa9980b6680 msgr2=0x7fa9980b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:46:18.454 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:18.452+0000 7fa9b50bd640 1 --2- 172.21.15.39:0/3202380205 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7fa9980b6680 0x7fa9980b8b90 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fa9b0102d50 tx=0x7fa9a4005f90 comp rx=0 tx=0).stop 2024-09-21T03:46:18.454 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:18.452+0000 7fa9b50bd640 1 -- 172.21.15.39:0/3202380205 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa9b0103b00 msgr2=0x7fa9b019a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:46:18.454 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:18.452+0000 7fa9b50bd640 1 --2- 172.21.15.39:0/3202380205 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa9b0103b00 0x7fa9b019a430 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7fa9a000d950 tx=0x7fa9a000de10 comp rx=0 tx=0).stop 2024-09-21T03:46:18.454 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:18.452+0000 7fa9b50bd640 1 -- 172.21.15.39:0/3202380205 shutdown_connections 2024-09-21T03:46:18.454 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:18.452+0000 7fa9b50bd640 1 --2- 172.21.15.39:0/3202380205 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7fa9980b6680 0x7fa9980b8b90 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:18.454 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:18.452+0000 7fa9b50bd640 1 --2- 172.21.15.39:0/3202380205 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa9b0109d80 0x7fa9b019fc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:18.454 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:18.452+0000 7fa9b50bd640 1 --2- 172.21.15.39:0/3202380205 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa9b0109430 0x7fa9b019a9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:18.454 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:18.452+0000 7fa9b50bd640 1 --2- 172.21.15.39:0/3202380205 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa9b0103b00 0x7fa9b019a430 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:18.454 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:18.452+0000 7fa9b50bd640 1 -- 172.21.15.39:0/3202380205 >> 172.21.15.39:0/3202380205 conn(0x7fa9b00ff500 msgr2=0x7fa9b01089e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:46:18.455 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:18.452+0000 7fa9b50bd640 1 -- 172.21.15.39:0/3202380205 shutdown_connections 2024-09-21T03:46:18.455 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:18.452+0000 7fa9b50bd640 1 -- 172.21.15.39:0/3202380205 wait complete. 2024-09-21T03:46:18.465 DEBUG:teuthology.orchestra.run.smithi039:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true' 2024-09-21T03:46:18.582 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:18.580+0000 7fb303ea2640 1 -- 172.21.15.39:0/1598017672 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb2fc104960 msgr2=0x7fb2fc104d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:46:18.582 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:18.580+0000 7fb303ea2640 1 --2- 172.21.15.39:0/1598017672 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb2fc104960 0x7fb2fc104d70 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7fb2f8009900 tx=0x7fb2f802a410 comp rx=0 tx=0).stop 2024-09-21T03:46:18.582 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:18.580+0000 7fb303ea2640 1 -- 172.21.15.39:0/1598017672 shutdown_connections 2024-09-21T03:46:18.582 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:18.580+0000 7fb303ea2640 1 --2- 172.21.15.39:0/1598017672 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb2fc10ddc0 0x7fb2fc1102b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:18.582 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:18.580+0000 7fb303ea2640 1 --2- 172.21.15.39:0/1598017672 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb2fc1052b0 0x7fb2fc10d7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:18.582 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:18.580+0000 7fb303ea2640 1 --2- 172.21.15.39:0/1598017672 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb2fc104960 0x7fb2fc104d70 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:18.583 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:18.580+0000 7fb303ea2640 1 -- 172.21.15.39:0/1598017672 >> 172.21.15.39:0/1598017672 conn(0x7fb2fc0fe410 msgr2=0x7fb2fc100860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:46:18.583 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:18.580+0000 7fb303ea2640 1 -- 172.21.15.39:0/1598017672 shutdown_connections 2024-09-21T03:46:18.583 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:18.580+0000 7fb303ea2640 1 -- 172.21.15.39:0/1598017672 wait complete. 2024-09-21T03:46:18.585 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:18.581+0000 7fb303ea2640 1 Processor -- start 2024-09-21T03:46:18.585 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:18.581+0000 7fb303ea2640 1 -- start start 2024-09-21T03:46:18.585 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:18.581+0000 7fb303ea2640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb2fc104960 0x7fb2fc19a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:18.585 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:18.581+0000 7fb303ea2640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb2fc1052b0 0x7fb2fc19aa20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:18.585 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:18.581+0000 7fb303ea2640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb2fc10ddc0 0x7fb2fc19fcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:18.585 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:18.581+0000 7fb303ea2640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fb2fc113a20 con 0x7fb2fc10ddc0 2024-09-21T03:46:18.585 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:18.581+0000 7fb303ea2640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fb2fc1138a0 con 0x7fb2fc104960 2024-09-21T03:46:18.585 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:18.581+0000 7fb303ea2640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fb2fc113ba0 con 0x7fb2fc1052b0 2024-09-21T03:46:18.585 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:18.581+0000 7fb30243f640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb2fc10ddc0 0x7fb2fc19fcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:46:18.585 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:18.582+0000 7fb30243f640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb2fc10ddc0 0x7fb2fc19fcb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:33506/0 (socket says 172.21.15.39:33506) 2024-09-21T03:46:18.585 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:18.582+0000 7fb30243f640 1 -- 172.21.15.39:0/1413261057 learned_addr learned my addr 172.21.15.39:0/1413261057 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:46:18.585 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:18.582+0000 7fb30143d640 1 --2- 172.21.15.39:0/1413261057 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb2fc1052b0 0x7fb2fc19aa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:46:18.586 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:18.582+0000 7fb301c3e640 1 --2- 172.21.15.39:0/1413261057 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb2fc104960 0x7fb2fc19a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:46:18.586 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:18.582+0000 7fb30243f640 1 -- 172.21.15.39:0/1413261057 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb2fc1052b0 msgr2=0x7fb2fc19aa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:46:18.586 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:18.582+0000 7fb30243f640 1 --2- 172.21.15.39:0/1413261057 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb2fc1052b0 0x7fb2fc19aa20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:18.586 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:18.582+0000 7fb30243f640 1 -- 172.21.15.39:0/1413261057 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb2fc104960 msgr2=0x7fb2fc19a480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:46:18.586 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:18.582+0000 7fb30243f640 1 --2- 172.21.15.39:0/1413261057 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb2fc104960 0x7fb2fc19a480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:18.586 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:18.582+0000 7fb30243f640 1 -- 172.21.15.39:0/1413261057 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb2fc1a03b0 con 0x7fb2fc10ddc0 2024-09-21T03:46:18.586 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:18.582+0000 7fb30243f640 1 --2- 172.21.15.39:0/1413261057 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb2fc10ddc0 0x7fb2fc19fcb0 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7fb2f400eb10 tx=0x7fb2f400c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:46:18.586 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:18.582+0000 7fb2f2ffd640 1 -- 172.21.15.39:0/1413261057 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2f4004b20 con 0x7fb2fc10ddc0 2024-09-21T03:46:18.586 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:18.582+0000 7fb2f2ffd640 1 -- 172.21.15.39:0/1413261057 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb2f4004cc0 con 0x7fb2fc10ddc0 2024-09-21T03:46:18.586 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:18.582+0000 7fb2f2ffd640 1 -- 172.21.15.39:0/1413261057 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2f4010670 con 0x7fb2fc10ddc0 2024-09-21T03:46:18.586 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:18.582+0000 7fb303ea2640 1 -- 172.21.15.39:0/1413261057 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb2fc1a06a0 con 0x7fb2fc10ddc0 2024-09-21T03:46:18.586 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:18.582+0000 7fb303ea2640 1 -- 172.21.15.39:0/1413261057 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb2fc1a6e30 con 0x7fb2fc10ddc0 2024-09-21T03:46:18.587 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:18.584+0000 7fb303ea2640 1 -- 172.21.15.39:0/1413261057 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb2fc104d70 con 0x7fb2fc10ddc0 2024-09-21T03:46:18.587 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:18.585+0000 7fb2f2ffd640 1 -- 172.21.15.39:0/1413261057 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 25) ==== 154637+0+0 (secure 0 0 0) 0x7fb2f4005000 con 0x7fb2fc10ddc0 2024-09-21T03:46:18.588 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:18.587+0000 7fb2f2ffd640 1 --2- 172.21.15.39:0/1413261057 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7fb2e80b6680 0x7fb2e80b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:18.588 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:18.587+0000 7fb301c3e640 1 --2- 172.21.15.39:0/1413261057 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7fb2e80b6680 0x7fb2e80b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:46:18.588 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:18.587+0000 7fb2f2ffd640 1 -- 172.21.15.39:0/1413261057 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8484+0+0 (secure 0 0 0) 0x7fb2f40e7f50 con 0x7fb2fc10ddc0 2024-09-21T03:46:18.589 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:18.587+0000 7fb301c3e640 1 --2- 172.21.15.39:0/1413261057 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7fb2e80b6680 0x7fb2e80b8b90 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fb2f8031440 tx=0x7fb2f8005d50 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:46:18.592 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:18.590+0000 7fb2f2ffd640 1 -- 172.21.15.39:0/1413261057 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fb2f4056710 con 0x7fb2fc10ddc0 2024-09-21T03:46:18.606 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 51116d5c-77ca-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub status 2024-09-21T03:46:18.804 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:18.802+0000 7fb303ea2640 1 -- 172.21.15.39:0/1413261057 --> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}) -- 0x7fb2fc107fc0 con 0x7fb2e80b6680 2024-09-21T03:46:18.811 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:18.810+0000 7fb2f2ffd640 1 -- 172.21.15.39:0/1413261057 <== mgr.14223 v2:172.21.15.39:6800/1696381982 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+10 (secure 0 0 0) 0x7fb2fc107fc0 con 0x7fb2e80b6680 2024-09-21T03:46:18.811 INFO:teuthology.orchestra.run.smithi039.stdout:/ 1m 6m3h 2024-09-21T03:46:18.814 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:18.812+0000 7fb303ea2640 1 -- 172.21.15.39:0/1413261057 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7fb2e80b6680 msgr2=0x7fb2e80b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:46:18.814 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:18.812+0000 7fb303ea2640 1 --2- 172.21.15.39:0/1413261057 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7fb2e80b6680 0x7fb2e80b8b90 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fb2f8031440 tx=0x7fb2f8005d50 comp rx=0 tx=0).stop 2024-09-21T03:46:18.814 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:18.812+0000 7fb303ea2640 1 -- 172.21.15.39:0/1413261057 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb2fc10ddc0 msgr2=0x7fb2fc19fcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:46:18.814 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:18.812+0000 7fb303ea2640 1 --2- 172.21.15.39:0/1413261057 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb2fc10ddc0 0x7fb2fc19fcb0 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7fb2f400eb10 tx=0x7fb2f400c2d0 comp rx=0 tx=0).stop 2024-09-21T03:46:18.814 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:18.812+0000 7fb303ea2640 1 -- 172.21.15.39:0/1413261057 shutdown_connections 2024-09-21T03:46:18.814 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:18.812+0000 7fb303ea2640 1 --2- 172.21.15.39:0/1413261057 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7fb2e80b6680 0x7fb2e80b8b90 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:18.814 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:18.812+0000 7fb303ea2640 1 --2- 172.21.15.39:0/1413261057 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb2fc10ddc0 0x7fb2fc19fcb0 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:18.814 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:18.812+0000 7fb303ea2640 1 --2- 172.21.15.39:0/1413261057 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb2fc1052b0 0x7fb2fc19aa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:18.814 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:18.812+0000 7fb303ea2640 1 --2- 172.21.15.39:0/1413261057 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb2fc104960 0x7fb2fc19a480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:18.814 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:18.812+0000 7fb303ea2640 1 -- 172.21.15.39:0/1413261057 >> 172.21.15.39:0/1413261057 conn(0x7fb2fc0fe410 msgr2=0x7fb2fc1078b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:46:18.814 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:18.812+0000 7fb303ea2640 1 -- 172.21.15.39:0/1413261057 shutdown_connections 2024-09-21T03:46:18.814 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:18.812+0000 7fb303ea2640 1 -- 172.21.15.39:0/1413261057 wait complete. 2024-09-21T03:46:18.827 DEBUG:teuthology.orchestra.run.smithi039:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'date +%s > START_TIME' 2024-09-21T03:46:18.855 INFO:teuthology.run_tasks:Running task full_sequential_finally... 2024-09-21T03:46:18.871 INFO:teuthology.run_tasks:Running task workunit... 2024-09-21T03:46:18.903 INFO:tasks.workunit:Pulling workunits from ref 1843221895adccfb3b692060251dd6a0183bc410 2024-09-21T03:46:18.903 INFO:tasks.workunit:Making a separate scratch dir for every client... 2024-09-21T03:46:18.903 INFO:tasks.workunit:timeout=3h 2024-09-21T03:46:18.903 INFO:tasks.workunit:cleanup=True 2024-09-21T03:46:18.903 DEBUG:teuthology.orchestra.run.smithi039:> stat -- /home/ubuntu/cephtest/mnt.0 2024-09-21T03:46:18.904 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/51116d5c-77ca-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:46:18.920 INFO:teuthology.orchestra.run.smithi039.stdout: File: /home/ubuntu/cephtest/mnt.0 2024-09-21T03:46:18.920 INFO:teuthology.orchestra.run.smithi039.stdout: Size: 0 Blocks: 0 IO Block: 65536 directory 2024-09-21T03:46:18.920 INFO:teuthology.orchestra.run.smithi039.stdout:Device: 78h/120d Inode: 1099511628283 Links: 2 2024-09-21T03:46:18.920 INFO:teuthology.orchestra.run.smithi039.stdout:Access: (1777/drwxrwxrwt) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-21T03:46:18.920 INFO:teuthology.orchestra.run.smithi039.stdout:Context: system_u:object_r:unlabeled_t:s0 2024-09-21T03:46:18.920 INFO:teuthology.orchestra.run.smithi039.stdout:Access: 2024-09-21 03:44:33.536184419 +0000 2024-09-21T03:46:18.920 INFO:teuthology.orchestra.run.smithi039.stdout:Modify: 2024-09-21 03:44:33.536184419 +0000 2024-09-21T03:46:18.921 INFO:teuthology.orchestra.run.smithi039.stdout:Change: 2024-09-21 03:46:07.722712960 +0000 2024-09-21T03:46:18.921 INFO:teuthology.orchestra.run.smithi039.stdout: Birth: 2024-09-21 03:44:33.536184419 +0000 2024-09-21T03:46:18.921 INFO:tasks.workunit:Did not need to create dir /home/ubuntu/cephtest/mnt.0 2024-09-21T03:46:18.921 DEBUG:teuthology.orchestra.run.smithi039:> cd -- /home/ubuntu/cephtest/mnt.0 && sudo install -d -m 0755 --owner=ubuntu -- client.0 2024-09-21T03:46:18.995 DEBUG:teuthology.orchestra.run.smithi039:> rm -rf /home/ubuntu/cephtest/clone.client.0 && git clone https://git.ceph.com/ceph-ci.git /home/ubuntu/cephtest/clone.client.0 && cd /home/ubuntu/cephtest/clone.client.0 && git checkout 1843221895adccfb3b692060251dd6a0183bc410 2024-09-21T03:46:19.053 INFO:tasks.workunit.client.0.smithi039.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.0'... 2024-09-21T03:46:19.084 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:46:19 smithi039 ceph-mon[31211]: from='client.15555 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:46:19.084 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:46:19 smithi039 ceph-mon[31211]: pgmap v333: 97 pgs: 97 active+clean; 657 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 1.4 KiB/s wr, 10 op/s 2024-09-21T03:46:19.084 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:46:19 smithi039 ceph-mon[31211]: scrub queued for path: / 2024-09-21T03:46:19.084 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:46:19 smithi039 ceph-mon[31211]: scrub summary: idle+waiting paths [/] 2024-09-21T03:46:19.084 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:46:19 smithi039 ceph-mon[31211]: scrub summary: active paths [/] 2024-09-21T03:46:19.084 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:46:19 smithi039 ceph-mon[31211]: scrub summary: idle+waiting paths [/] 2024-09-21T03:46:19.084 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:46:19 smithi039 ceph-mon[31211]: from='client.15567 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:46:19.084 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:46:19 smithi039 ceph-mon[31211]: mgrmap e25: x(active, since 8m), standbys: z, y 2024-09-21T03:46:19.437 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:19.432+0000 7fef72a5e640 1 -- 172.21.15.39:0/3700855794 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fef6c103b90 msgr2=0x7fef6c103fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:46:19.438 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:19.432+0000 7fef72a5e640 1 --2- 172.21.15.39:0/3700855794 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fef6c103b90 0x7fef6c103fa0 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7fef5c009900 tx=0x7fef5c02a410 comp rx=0 tx=0).stop 2024-09-21T03:46:19.438 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:19.432+0000 7fef72a5e640 1 -- 172.21.15.39:0/3700855794 shutdown_connections 2024-09-21T03:46:19.438 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:19.432+0000 7fef72a5e640 1 --2- 172.21.15.39:0/3700855794 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fef6c109e20 0x7fef6c110650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:19.438 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:19.432+0000 7fef72a5e640 1 --2- 172.21.15.39:0/3700855794 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fef6c1094d0 0x7fef6c1098e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:19.438 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:19.432+0000 7fef72a5e640 1 --2- 172.21.15.39:0/3700855794 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fef6c103b90 0x7fef6c103fa0 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:19.438 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:19.432+0000 7fef72a5e640 1 -- 172.21.15.39:0/3700855794 >> 172.21.15.39:0/3700855794 conn(0x7fef6c0ff600 msgr2=0x7fef6c101a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:46:19.438 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:19.432+0000 7fef72a5e640 1 -- 172.21.15.39:0/3700855794 shutdown_connections 2024-09-21T03:46:19.438 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:19.432+0000 7fef72a5e640 1 -- 172.21.15.39:0/3700855794 wait complete. 2024-09-21T03:46:19.438 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:19.433+0000 7fef72a5e640 1 Processor -- start 2024-09-21T03:46:19.438 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:19.433+0000 7fef72a5e640 1 -- start start 2024-09-21T03:46:19.439 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:19.433+0000 7fef72a5e640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fef6c103b90 0x7fef6c075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:19.439 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:19.433+0000 7fef72a5e640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fef6c1094d0 0x7fef6c075c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:19.439 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:19.433+0000 7fef72a5e640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fef6c109e20 0x7fef6c076190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:19.439 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:19.434+0000 7fef72a5e640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fef6c113ae0 con 0x7fef6c109e20 2024-09-21T03:46:19.439 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:19.434+0000 7fef72a5e640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fef6c113960 con 0x7fef6c1094d0 2024-09-21T03:46:19.439 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:19.434+0000 7fef72a5e640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fef6c113c60 con 0x7fef6c103b90 2024-09-21T03:46:19.439 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:19.434+0000 7fef70ffb640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fef6c109e20 0x7fef6c076190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:46:19.439 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:19.434+0000 7fef70ffb640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fef6c109e20 0x7fef6c076190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:33520/0 (socket says 172.21.15.39:33520) 2024-09-21T03:46:19.439 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:19.434+0000 7fef70ffb640 1 -- 172.21.15.39:0/353422161 learned_addr learned my addr 172.21.15.39:0/353422161 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:46:19.439 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:19.434+0000 7fef6bfff640 1 --2- 172.21.15.39:0/353422161 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fef6c103b90 0x7fef6c075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:46:19.439 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:19.434+0000 7fef6b7fe640 1 --2- 172.21.15.39:0/353422161 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fef6c1094d0 0x7fef6c075c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:46:19.439 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:19.434+0000 7fef70ffb640 1 -- 172.21.15.39:0/353422161 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fef6c103b90 msgr2=0x7fef6c075710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:46:19.439 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:19.434+0000 7fef70ffb640 1 --2- 172.21.15.39:0/353422161 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fef6c103b90 0x7fef6c075710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:19.439 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:19.434+0000 7fef70ffb640 1 -- 172.21.15.39:0/353422161 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fef6c1094d0 msgr2=0x7fef6c075c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:46:19.439 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:19.434+0000 7fef70ffb640 1 --2- 172.21.15.39:0/353422161 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fef6c1094d0 0x7fef6c075c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:19.440 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:19.434+0000 7fef70ffb640 1 -- 172.21.15.39:0/353422161 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fef6c1a8810 con 0x7fef6c109e20 2024-09-21T03:46:19.440 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:19.434+0000 7fef6bfff640 1 --2- 172.21.15.39:0/353422161 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fef6c103b90 0x7fef6c075710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:46:19.440 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:19.434+0000 7fef70ffb640 1 --2- 172.21.15.39:0/353422161 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fef6c109e20 0x7fef6c076190 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7fef6400eaa0 tx=0x7fef6400ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:46:19.440 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:19.434+0000 7fef697fa640 1 -- 172.21.15.39:0/353422161 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef6400cdb0 con 0x7fef6c109e20 2024-09-21T03:46:19.440 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:19.434+0000 7fef72a5e640 1 -- 172.21.15.39:0/353422161 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fef6c1a8b00 con 0x7fef6c109e20 2024-09-21T03:46:19.440 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:19.434+0000 7fef697fa640 1 -- 172.21.15.39:0/353422161 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fef64005020 con 0x7fef6c109e20 2024-09-21T03:46:19.440 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:19.434+0000 7fef697fa640 1 -- 172.21.15.39:0/353422161 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef640116d0 con 0x7fef6c109e20 2024-09-21T03:46:19.440 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:19.435+0000 7fef72a5e640 1 -- 172.21.15.39:0/353422161 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fef6c1a8ff0 con 0x7fef6c109e20 2024-09-21T03:46:19.442 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:19.437+0000 7fef78fd2740 1 Processor -- start 2024-09-21T03:46:19.442 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:19.437+0000 7fef697fa640 1 -- 172.21.15.39:0/353422161 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 25) ==== 154637+0+0 (secure 0 0 0) 0x7fef64011870 con 0x7fef6c109e20 2024-09-21T03:46:19.442 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:19.438+0000 7fef78fd2740 1 -- start start 2024-09-21T03:46:19.442 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:19.438+0000 7fef78fd2740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a5ab10d070 0x55a5ab4b4270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:19.442 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:19.438+0000 7fef78fd2740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a5ab0e41b0 0x55a5ab4b47b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:19.443 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:19.438+0000 7fef78fd2740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a5ab0ec390 0x55a5ab4b4cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:19.443 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:19.438+0000 7fef78fd2740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x55a5ab42ae50 con 0x55a5ab0ec390 2024-09-21T03:46:19.443 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:19.438+0000 7fef78fd2740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x55a5ab41a280 con 0x55a5ab10d070 2024-09-21T03:46:19.443 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:19.438+0000 7fef78fd2740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55a5ab41e090 con 0x55a5ab0e41b0 2024-09-21T03:46:19.443 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:19.438+0000 7fef6b7fe640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a5ab0ec390 0x55a5ab4b4cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:46:19.443 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:19.438+0000 7fef6b7fe640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a5ab0ec390 0x55a5ab4b4cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:33526/0 (socket says 172.21.15.39:33526) 2024-09-21T03:46:19.443 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:19.438+0000 7fef6b7fe640 1 -- 172.21.15.39:0/3348595738 learned_addr learned my addr 172.21.15.39:0/3348595738 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:46:19.443 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:19.438+0000 7fef6bfff640 1 --2- 172.21.15.39:0/3348595738 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a5ab0e41b0 0x55a5ab4b47b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:46:19.443 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:19.438+0000 7fef6b7fe640 1 --2- 172.21.15.39:0/3348595738 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a5ab10d070 0x55a5ab4b4270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:46:19.443 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:19.438+0000 7fef697fa640 1 --2- 172.21.15.39:0/353422161 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7fef540b6680 0x7fef540b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:19.443 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:19.438+0000 7fef6b7fe640 1 -- 172.21.15.39:0/3348595738 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a5ab0e41b0 msgr2=0x55a5ab4b47b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:46:19.443 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:19.438+0000 7fef6b7fe640 1 --2- 172.21.15.39:0/3348595738 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a5ab0e41b0 0x55a5ab4b47b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:19.443 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:19.438+0000 7fef6b7fe640 1 -- 172.21.15.39:0/3348595738 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a5ab10d070 msgr2=0x55a5ab4b4270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:46:19.443 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:19.438+0000 7fef6b7fe640 1 --2- 172.21.15.39:0/3348595738 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a5ab10d070 0x55a5ab4b4270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:19.444 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:19.438+0000 7fef6b7fe640 1 -- 172.21.15.39:0/3348595738 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a5ab178650 con 0x55a5ab0ec390 2024-09-21T03:46:19.444 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:19.439+0000 7fef697fa640 1 -- 172.21.15.39:0/353422161 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8484+0+0 (secure 0 0 0) 0x7fef640e7940 con 0x7fef6c109e20 2024-09-21T03:46:19.444 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:19.439+0000 7fef70ffb640 1 --2- 172.21.15.39:0/353422161 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7fef540b6680 0x7fef540b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:46:19.444 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:19.439+0000 7fef6b7fe640 1 --2- 172.21.15.39:0/3348595738 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a5ab0ec390 0x55a5ab4b4cf0 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7fef6000cf90 tx=0x7fef6000c630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d8cdc0d7f53da8c server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:46:19.444 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:19.439+0000 7fef527fc640 1 -- 172.21.15.39:0/3348595738 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef60019030 con 0x55a5ab0ec390 2024-09-21T03:46:19.444 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:19.439+0000 7fef527fc640 1 -- 172.21.15.39:0/3348595738 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fef600054d0 con 0x55a5ab0ec390 2024-09-21T03:46:19.444 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:19.439+0000 7fef527fc640 1 -- 172.21.15.39:0/3348595738 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef60011520 con 0x55a5ab0ec390 2024-09-21T03:46:19.444 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:19.440+0000 7fef78fd2740 1 -- 172.21.15.39:0/3348595738 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a5ab0ec390 msgr2=0x55a5ab4b4cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:46:19.444 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:19.440+0000 7fef70ffb640 1 --2- 172.21.15.39:0/353422161 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7fef540b6680 0x7fef540b8b90 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fef640aaa20 tx=0x7fef640e7c40 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:46:19.444 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:19.440+0000 7fef78fd2740 1 --2- 172.21.15.39:0/3348595738 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a5ab0ec390 0x55a5ab4b4cf0 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7fef6000cf90 tx=0x7fef6000c630 comp rx=0 tx=0).stop 2024-09-21T03:46:19.444 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:19.440+0000 7fef78fd2740 1 -- 172.21.15.39:0/3348595738 shutdown_connections 2024-09-21T03:46:19.444 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:19.440+0000 7fef78fd2740 1 --2- 172.21.15.39:0/3348595738 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a5ab0ec390 0x55a5ab4b4cf0 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:19.444 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:19.440+0000 7fef78fd2740 1 --2- 172.21.15.39:0/3348595738 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a5ab0e41b0 0x55a5ab4b47b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:19.444 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:19.440+0000 7fef78fd2740 1 --2- 172.21.15.39:0/3348595738 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a5ab10d070 0x55a5ab4b4270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:19.444 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:19.440+0000 7fef78fd2740 1 -- 172.21.15.39:0/3348595738 >> 172.21.15.39:0/3348595738 conn(0x55a5ab101460 msgr2=0x55a5ab4b2160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:46:19.445 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:19.440+0000 7fef78fd2740 1 -- 172.21.15.39:0/3348595738 shutdown_connections 2024-09-21T03:46:19.445 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:19.440+0000 7fef78fd2740 1 -- 172.21.15.39:0/3348595738 wait complete. 2024-09-21T03:46:19.445 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:19.440+0000 7fef78fd2740 1 -- start start 2024-09-21T03:46:19.445 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:19.441+0000 7fef78fd2740 1 Processor -- start 2024-09-21T03:46:19.448 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:19.441+0000 7fef78fd2740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a5ab0e41b0 0x55a5ab4bf930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:19.448 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:19.441+0000 7fef78fd2740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a5ab0ec390 0x55a5ab4bfe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:19.448 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:19.441+0000 7fef78fd2740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a5ab10d070 0x55a5ab4cbd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:19.448 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:19.441+0000 7fef78fd2740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x55a5ab27b700 con 0x55a5ab0ec390 2024-09-21T03:46:19.448 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:19.441+0000 7fef78fd2740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x55a5ab3b5110 con 0x55a5ab0e41b0 2024-09-21T03:46:19.448 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:19.441+0000 7fef78fd2740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55a5ab4c2a50 con 0x55a5ab10d070 2024-09-21T03:46:19.448 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:19.441+0000 7fef78fd2740 10 client.0 _ll_register_callbacks cb 0x55a5ab3ce990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:46:19.449 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:19.441+0000 7fef6bfff640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a5ab0ec390 0x55a5ab4bfe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:46:19.449 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:19.441+0000 7fef6bfff640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a5ab0ec390 0x55a5ab4bfe70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:33534/0 (socket says 172.21.15.39:33534) 2024-09-21T03:46:19.449 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:19.441+0000 7fef6bfff640 1 -- 172.21.15.39:0/431099882 learned_addr learned my addr 172.21.15.39:0/431099882 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:46:19.449 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:19.441+0000 7fef6b7fe640 1 --2- 172.21.15.39:0/431099882 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a5ab0e41b0 0x55a5ab4bf930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:46:19.449 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:19.441+0000 7fef6b7fe640 1 --2- 172.21.15.39:0/431099882 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a5ab10d070 0x55a5ab4cbd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:46:19.449 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:19.442+0000 7fef6bfff640 1 -- 172.21.15.39:0/431099882 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a5ab10d070 msgr2=0x55a5ab4cbd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:46:19.449 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:19.442+0000 7fef6bfff640 1 --2- 172.21.15.39:0/431099882 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a5ab10d070 0x55a5ab4cbd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:19.449 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:19.442+0000 7fef6bfff640 1 -- 172.21.15.39:0/431099882 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a5ab0e41b0 msgr2=0x55a5ab4bf930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:46:19.449 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:19.442+0000 7fef6bfff640 1 --2- 172.21.15.39:0/431099882 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a5ab0e41b0 0x55a5ab4bf930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:19.449 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:19.442+0000 7fef6bfff640 1 -- 172.21.15.39:0/431099882 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a5ab3e2c40 con 0x55a5ab0ec390 2024-09-21T03:46:19.449 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:19.442+0000 7fef6b7fe640 1 --2- 172.21.15.39:0/431099882 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a5ab10d070 0x55a5ab4cbd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:46:19.449 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:19.442+0000 7fef6bfff640 1 --2- 172.21.15.39:0/431099882 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a5ab0ec390 0x55a5ab4bfe70 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7fef5c005940 tx=0x7fef5c031720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=41e2eeca2bcdbbd server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:46:19.449 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:19.442+0000 7fef4b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-09-21T03:46:19.449 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:19.442+0000 7fef4b7fe640 1 -- 172.21.15.39:0/431099882 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef5c035040 con 0x55a5ab0ec390 2024-09-21T03:46:19.450 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:19.442+0000 7fef4b7fe640 1 -- 172.21.15.39:0/431099882 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8484+0+0 (secure 0 0 0) 0x7fef5c004240 con 0x55a5ab0ec390 2024-09-21T03:46:19.450 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:19.442+0000 7fef78fd2740 1 -- 172.21.15.39:0/431099882 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fef600054d0 con 0x55a5ab0ec390 2024-09-21T03:46:19.450 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:19.442+0000 7fef4b7fe640 1 -- 172.21.15.39:0/431099882 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fef5c004440 con 0x55a5ab0ec390 2024-09-21T03:46:19.450 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:19.442+0000 7fef78fd2740 10 client.15609 fetch_fsmap learned FSMap version 29 2024-09-21T03:46:19.450 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:19.442+0000 7fef78fd2740 1 -- 172.21.15.39:0/431099882 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55a5ab4ce760 con 0x55a5ab0ec390 2024-09-21T03:46:19.450 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:19.442+0000 7fef4b7fe640 1 -- 172.21.15.39:0/431099882 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fef600054d0 con 0x55a5ab0ec390 2024-09-21T03:46:19.450 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:19.443+0000 7fef4b7fe640 1 -- 172.21.15.39:0/431099882 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fef5c04c020 con 0x55a5ab0ec390 2024-09-21T03:46:19.450 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:19.443+0000 7fef78fd2740 10 client.15609 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T03:46:19.450 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:19.443+0000 7fef78fd2740 10 client.15609 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T03:46:19.450 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:19.443+0000 7fef78fd2740 20 client.15609 populate_metadata read hostname 'smithi039' 2024-09-21T03:46:19.450 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:19.443+0000 7fef78fd2740 1 --2- 172.21.15.39:0/431099882 >> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] conn(0x55a5ab0cb9f0 0x55a5ab4d1be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:19.451 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:19.443+0000 7fef78fd2740 4 client.15609 mds_command: new command op to 24695 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:46:19.451 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:19.443+0000 7fef78fd2740 1 -- 172.21.15.39:0/431099882 --> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a5ab42cda0 con 0x55a5ab0cb9f0 2024-09-21T03:46:19.451 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:19.443+0000 7fef6b7fe640 1 --2- 172.21.15.39:0/431099882 >> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] conn(0x55a5ab0cb9f0 0x55a5ab4d1be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:46:19.451 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:19.443+0000 7fef6b7fe640 1 --2- 172.21.15.39:0/431099882 >> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] conn(0x55a5ab0cb9f0 0x55a5ab4d1be0 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c222999a529d8c23 server_cookie=499dc160818f9981 in_seq=0 out_seq=0 2024-09-21T03:46:19.451 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:19.443+0000 7fef4b7fe640 10 client.15609 ms_handle_connect on v2:172.21.15.150:6838/2949386907 2024-09-21T03:46:19.451 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:19.444+0000 7fef4b7fe640 1 -- 172.21.15.39:0/431099882 <== mds.0 v2:172.21.15.150:6838/2949386907 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55a5ab42cda0 con 0x55a5ab0cb9f0 2024-09-21T03:46:19.451 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:19.444+0000 7fef4b7fe640 10 client.15609 handle_command_reply: tid=0 2024-09-21T03:46:19.451 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:19.444+0000 7fef78fd2740 1 client.15609 shutdown 2024-09-21T03:46:19.451 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:19.444+0000 7fef78fd2740 1 -- 172.21.15.39:0/431099882 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a5ab0ec390 msgr2=0x55a5ab4bfe70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:46:19.452 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:19.444+0000 7fef78fd2740 1 --2- 172.21.15.39:0/431099882 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a5ab0ec390 0x55a5ab4bfe70 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7fef5c005940 tx=0x7fef5c031720 comp rx=0 tx=0).stop 2024-09-21T03:46:19.452 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:19.444+0000 7fef78fd2740 1 -- 172.21.15.39:0/431099882 shutdown_connections 2024-09-21T03:46:19.452 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:19.444+0000 7fef78fd2740 1 --2- 172.21.15.39:0/431099882 >> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] conn(0x55a5ab0cb9f0 0x55a5ab4d1be0 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:19.452 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:19.444+0000 7fef78fd2740 1 --2- 172.21.15.39:0/431099882 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a5ab10d070 0x55a5ab4cbd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:19.452 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:19.445+0000 7fef78fd2740 1 --2- 172.21.15.39:0/431099882 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a5ab0ec390 0x55a5ab4bfe70 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:19.452 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:19.445+0000 7fef4b7fe640 0 client.15609 ms_handle_reset on v2:172.21.15.150:6838/2949386907 2024-09-21T03:46:19.452 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:19.445+0000 7fef78fd2740 1 --2- 172.21.15.39:0/431099882 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a5ab0e41b0 0x55a5ab4bf930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:19.452 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:19.445+0000 7fef78fd2740 1 -- 172.21.15.39:0/431099882 >> 172.21.15.39:0/431099882 conn(0x55a5ab101460 msgr2=0x55a5ab4b7450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:46:19.452 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:19.445+0000 7fef78fd2740 1 -- 172.21.15.39:0/431099882 shutdown_connections 2024-09-21T03:46:19.453 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:19.445+0000 7fef78fd2740 1 -- 172.21.15.39:0/431099882 wait complete. 2024-09-21T03:46:19.453 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:19.445+0000 7fef78fd2740 20 client.15609 trim_cache size 0 max 16384 2024-09-21T03:46:19.464 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:19.462+0000 7fef78fd2740 1 Processor -- start 2024-09-21T03:46:19.464 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:19.462+0000 7fef78fd2740 1 -- start start 2024-09-21T03:46:19.465 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:19.463+0000 7fef78fd2740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a5ab0e41b0 0x55a5ab4b5c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:19.465 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:19.463+0000 7fef78fd2740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a5ab0ec390 0x55a5ab4c1560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:19.465 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:19.463+0000 7fef78fd2740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a5ab10d070 0x55a5ab4cbc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:19.465 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:19.463+0000 7fef78fd2740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fef6c1b19a0 con 0x55a5ab10d070 2024-09-21T03:46:19.465 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:19.463+0000 7fef78fd2740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fef6c1b34e0 con 0x55a5ab0e41b0 2024-09-21T03:46:19.465 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:19.463+0000 7fef78fd2740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fef6c1b2070 con 0x55a5ab0ec390 2024-09-21T03:46:19.465 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:19.463+0000 7fef6b7fe640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a5ab10d070 0x55a5ab4cbc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:46:19.465 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:19.463+0000 7fef6b7fe640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a5ab10d070 0x55a5ab4cbc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:33538/0 (socket says 172.21.15.39:33538) 2024-09-21T03:46:19.465 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:19.463+0000 7fef6b7fe640 1 -- 172.21.15.39:0/1558300394 learned_addr learned my addr 172.21.15.39:0/1558300394 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:46:19.466 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:19.463+0000 7fef6b7fe640 1 -- 172.21.15.39:0/1558300394 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a5ab0ec390 msgr2=0x55a5ab4c1560 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-21T03:46:19.466 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:19.463+0000 7fef6b7fe640 1 --2- 172.21.15.39:0/1558300394 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a5ab0ec390 0x55a5ab4c1560 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:19.467 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:19.463+0000 7fef6b7fe640 1 -- 172.21.15.39:0/1558300394 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a5ab0e41b0 msgr2=0x55a5ab4b5c60 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-21T03:46:19.467 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:19.463+0000 7fef6b7fe640 1 --2- 172.21.15.39:0/1558300394 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a5ab0e41b0 0x55a5ab4b5c60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:19.467 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:19.463+0000 7fef6b7fe640 1 -- 172.21.15.39:0/1558300394 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a5ab4c0bf0 con 0x55a5ab10d070 2024-09-21T03:46:19.467 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:19.463+0000 7fef6b7fe640 1 --2- 172.21.15.39:0/1558300394 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a5ab10d070 0x55a5ab4cbc40 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7fef60004bc0 tx=0x7fef60004bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a15613fdaf435af server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:46:19.467 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:19.464+0000 7fef51ffb640 1 -- 172.21.15.39:0/1558300394 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef6002c040 con 0x55a5ab10d070 2024-09-21T03:46:19.467 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:19.464+0000 7fef51ffb640 1 -- 172.21.15.39:0/1558300394 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fef60004e20 con 0x55a5ab10d070 2024-09-21T03:46:19.467 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:19.464+0000 7fef51ffb640 1 -- 172.21.15.39:0/1558300394 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef60019040 con 0x55a5ab10d070 2024-09-21T03:46:19.467 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:19.464+0000 7fef78fd2740 1 -- 172.21.15.39:0/1558300394 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a5ab10d070 msgr2=0x55a5ab4cbc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:46:19.467 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:19.464+0000 7fef78fd2740 1 --2- 172.21.15.39:0/1558300394 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a5ab10d070 0x55a5ab4cbc40 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7fef60004bc0 tx=0x7fef60004bf0 comp rx=0 tx=0).stop 2024-09-21T03:46:19.467 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:19.464+0000 7fef78fd2740 1 -- 172.21.15.39:0/1558300394 shutdown_connections 2024-09-21T03:46:19.467 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:19.464+0000 7fef78fd2740 1 --2- 172.21.15.39:0/1558300394 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a5ab10d070 0x55a5ab4cbc40 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:19.468 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:19.464+0000 7fef78fd2740 1 --2- 172.21.15.39:0/1558300394 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a5ab0ec390 0x55a5ab4c1560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:19.468 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:19.464+0000 7fef78fd2740 1 --2- 172.21.15.39:0/1558300394 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a5ab0e41b0 0x55a5ab4b5c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:19.468 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:19.464+0000 7fef78fd2740 1 -- 172.21.15.39:0/1558300394 >> 172.21.15.39:0/1558300394 conn(0x55a5ab101460 msgr2=0x55a5ab4c0e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:46:19.468 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:19.464+0000 7fef78fd2740 1 -- 172.21.15.39:0/1558300394 shutdown_connections 2024-09-21T03:46:19.468 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:19.464+0000 7fef78fd2740 1 -- 172.21.15.39:0/1558300394 wait complete. 2024-09-21T03:46:19.468 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:19.465+0000 7fef78fd2740 1 -- start start 2024-09-21T03:46:19.468 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:19.465+0000 7fef78fd2740 1 Processor -- start 2024-09-21T03:46:19.469 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:46:19 smithi161 ceph-mon[34533]: from='client.15555 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:46:19.469 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:46:19 smithi161 ceph-mon[34533]: pgmap v333: 97 pgs: 97 active+clean; 657 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 1.4 KiB/s wr, 10 op/s 2024-09-21T03:46:19.469 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:46:19 smithi161 ceph-mon[34533]: scrub queued for path: / 2024-09-21T03:46:19.469 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:46:19 smithi161 ceph-mon[34533]: scrub summary: idle+waiting paths [/] 2024-09-21T03:46:19.469 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:46:19 smithi161 ceph-mon[34533]: scrub summary: active paths [/] 2024-09-21T03:46:19.469 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:46:19 smithi161 ceph-mon[34533]: scrub summary: idle+waiting paths [/] 2024-09-21T03:46:19.469 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:46:19 smithi161 ceph-mon[34533]: from='client.15567 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:46:19.469 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:46:19 smithi161 ceph-mon[34533]: mgrmap e25: x(active, since 8m), standbys: z, y 2024-09-21T03:46:19.472 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:19.465+0000 7fef78fd2740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a5ab0e41b0 0x55a5ab4c8440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:19.472 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:19.465+0000 7fef78fd2740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a5ab0ec390 0x55a5ab4c38f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:19.472 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:19.465+0000 7fef78fd2740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a5ab10d070 0x55a5ab4c3e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:19.472 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:19.465+0000 7fef78fd2740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fef6c1b2e10 con 0x55a5ab0ec390 2024-09-21T03:46:19.472 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:19.465+0000 7fef78fd2740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fef6c1b2740 con 0x55a5ab0e41b0 2024-09-21T03:46:19.472 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:19.465+0000 7fef78fd2740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fef6c1af980 con 0x55a5ab10d070 2024-09-21T03:46:19.472 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:19.465+0000 7fef78fd2740 10 client.0 _ll_register_callbacks cb 0x7fef5c005eb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:46:19.472 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:19.466+0000 7fef6bfff640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a5ab0ec390 0x55a5ab4c38f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:46:19.472 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:19.466+0000 7fef6bfff640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a5ab0ec390 0x55a5ab4c38f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:33542/0 (socket says 172.21.15.39:33542) 2024-09-21T03:46:19.472 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:19.466+0000 7fef6bfff640 1 -- 172.21.15.39:0/2042122496 learned_addr learned my addr 172.21.15.39:0/2042122496 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:46:19.472 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:19.466+0000 7fef6b7fe640 1 --2- 172.21.15.39:0/2042122496 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a5ab0e41b0 0x55a5ab4c8440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:46:19.472 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:19.466+0000 7fef6bfff640 1 -- 172.21.15.39:0/2042122496 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a5ab10d070 msgr2=0x55a5ab4c3e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:46:19.473 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:19.466+0000 7fef6bfff640 1 --2- 172.21.15.39:0/2042122496 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a5ab10d070 0x55a5ab4c3e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:19.473 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:19.466+0000 7fef6bfff640 1 -- 172.21.15.39:0/2042122496 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a5ab0e41b0 msgr2=0x55a5ab4c8440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:46:19.473 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:19.466+0000 7fef6bfff640 1 --2- 172.21.15.39:0/2042122496 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a5ab0e41b0 0x55a5ab4c8440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:19.473 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:19.466+0000 7fef6bfff640 1 -- 172.21.15.39:0/2042122496 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a5ab4c4570 con 0x55a5ab0ec390 2024-09-21T03:46:19.473 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:19.466+0000 7fef6b7fe640 1 --2- 172.21.15.39:0/2042122496 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a5ab0e41b0 0x55a5ab4c8440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:46:19.473 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:19.466+0000 7fef6bfff640 1 --2- 172.21.15.39:0/2042122496 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a5ab0ec390 0x55a5ab4c38f0 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7fef5c004b50 tx=0x7fef5c0314a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b1ff4dda98a6146e server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:46:19.473 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:19.466+0000 7fef4b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-09-21T03:46:19.473 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:19.466+0000 7fef4b7fe640 1 -- 172.21.15.39:0/2042122496 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef5c035040 con 0x55a5ab0ec390 2024-09-21T03:46:19.473 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:19.466+0000 7fef4b7fe640 1 -- 172.21.15.39:0/2042122496 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8484+0+0 (secure 0 0 0) 0x7fef5c0055b0 con 0x55a5ab0ec390 2024-09-21T03:46:19.473 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:19.466+0000 7fef78fd2740 1 -- 172.21.15.39:0/2042122496 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fef60004e20 con 0x55a5ab0ec390 2024-09-21T03:46:19.473 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:19.466+0000 7fef4b7fe640 1 -- 172.21.15.39:0/2042122496 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fef5c05e050 con 0x55a5ab0ec390 2024-09-21T03:46:19.473 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:19.466+0000 7fef78fd2740 10 client.15615 fetch_fsmap learned FSMap version 29 2024-09-21T03:46:19.473 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:19.466+0000 7fef78fd2740 1 -- 172.21.15.39:0/2042122496 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55a5ab53c4a0 con 0x55a5ab0ec390 2024-09-21T03:46:19.473 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:19.467+0000 7fef4b7fe640 1 -- 172.21.15.39:0/2042122496 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fef60004e20 con 0x55a5ab0ec390 2024-09-21T03:46:19.474 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:19.467+0000 7fef4b7fe640 1 -- 172.21.15.39:0/2042122496 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fef5c04c070 con 0x55a5ab0ec390 2024-09-21T03:46:19.474 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:19.467+0000 7fef78fd2740 10 client.15615 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T03:46:19.474 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:19.467+0000 7fef78fd2740 10 client.15615 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T03:46:19.474 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:19.467+0000 7fef78fd2740 20 client.15615 populate_metadata read hostname 'smithi039' 2024-09-21T03:46:19.474 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:19.467+0000 7fef78fd2740 1 --2- 172.21.15.39:0/2042122496 >> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] conn(0x55a5ab0a0da0 0x55a5ab53ecb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:19.474 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:19.467+0000 7fef78fd2740 4 client.15615 mds_command: new command op to 24695 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-21T03:46:19.474 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:19.467+0000 7fef78fd2740 1 -- 172.21.15.39:0/2042122496 --> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55a5ab428850 con 0x55a5ab0a0da0 2024-09-21T03:46:19.474 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:19.467+0000 7fef6b7fe640 1 --2- 172.21.15.39:0/2042122496 >> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] conn(0x55a5ab0a0da0 0x55a5ab53ecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:46:19.474 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:19.468+0000 7fef4b7fe640 10 client.15615 ms_handle_connect on v2:172.21.15.150:6838/2949386907 2024-09-21T03:46:19.474 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:19.468+0000 7fef6b7fe640 1 --2- 172.21.15.39:0/2042122496 >> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] conn(0x55a5ab0a0da0 0x55a5ab53ecb0 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e90f7a2bca015445 server_cookie=89b08e77a5f2add2 in_seq=0 out_seq=0 2024-09-21T03:46:19.474 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:19.468+0000 7fef4b7fe640 1 -- 172.21.15.39:0/2042122496 <== mds.0 v2:172.21.15.150:6838/2949386907 1 ==== command_reply(tid 0: 0 ) ==== 8+0+570 (crc 0 0 0) 0x55a5ab428850 con 0x55a5ab0a0da0 2024-09-21T03:46:19.474 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:19.468+0000 7fef4b7fe640 10 client.15615 handle_command_reply: tid=0 2024-09-21T03:46:19.474 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:19.468+0000 7fef78fd2740 1 client.15615 shutdown 2024-09-21T03:46:19.474 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:19.468+0000 7fef78fd2740 1 -- 172.21.15.39:0/2042122496 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a5ab0ec390 msgr2=0x55a5ab4c38f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:46:19.475 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:19.468+0000 7fef78fd2740 1 --2- 172.21.15.39:0/2042122496 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a5ab0ec390 0x55a5ab4c38f0 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7fef5c004b50 tx=0x7fef5c0314a0 comp rx=0 tx=0).stop 2024-09-21T03:46:19.475 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:19.468+0000 7fef78fd2740 1 -- 172.21.15.39:0/2042122496 shutdown_connections 2024-09-21T03:46:19.475 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:19.468+0000 7fef78fd2740 1 --2- 172.21.15.39:0/2042122496 >> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] conn(0x55a5ab0a0da0 0x55a5ab53ecb0 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:19.475 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:19.468+0000 7fef78fd2740 1 --2- 172.21.15.39:0/2042122496 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a5ab10d070 0x55a5ab4c3e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:19.475 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:19.468+0000 7fef78fd2740 1 --2- 172.21.15.39:0/2042122496 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a5ab0ec390 0x55a5ab4c38f0 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:19.475 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:19.468+0000 7fef4b7fe640 0 client.15615 ms_handle_reset on v2:172.21.15.150:6838/2949386907 2024-09-21T03:46:19.475 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:19.468+0000 7fef78fd2740 1 --2- 172.21.15.39:0/2042122496 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a5ab0e41b0 0x55a5ab4c8440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:19.475 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:19.468+0000 7fef78fd2740 1 -- 172.21.15.39:0/2042122496 >> 172.21.15.39:0/2042122496 conn(0x55a5ab101460 msgr2=0x55a5ab4b5a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:46:19.475 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:19.468+0000 7fef78fd2740 1 -- 172.21.15.39:0/2042122496 shutdown_connections 2024-09-21T03:46:19.475 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:19.468+0000 7fef78fd2740 1 -- 172.21.15.39:0/2042122496 wait complete. 2024-09-21T03:46:19.475 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:19.469+0000 7fef78fd2740 20 client.15615 trim_cache size 0 max 16384 2024-09-21T03:46:19.475 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-09-21T03:46:19.476 INFO:teuthology.orchestra.run.smithi039.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-21T03:46:19.476 INFO:teuthology.orchestra.run.smithi039.stdout: "scrubs": { 2024-09-21T03:46:19.476 INFO:teuthology.orchestra.run.smithi039.stdout: "02509e4f-3911-4983-b16f-df8ded0b9998": { 2024-09-21T03:46:19.476 INFO:teuthology.orchestra.run.smithi039.stdout: "path": "/", 2024-09-21T03:46:19.476 INFO:teuthology.orchestra.run.smithi039.stdout: "tag": "02509e4f-3911-4983-b16f-df8ded0b9998", 2024-09-21T03:46:19.476 INFO:teuthology.orchestra.run.smithi039.stdout: "options": "recursive,force" 2024-09-21T03:46:19.476 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-21T03:46:19.476 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:46:19.476 INFO:teuthology.orchestra.run.smithi039.stdout: "scrub_stats": { 2024-09-21T03:46:19.476 INFO:teuthology.orchestra.run.smithi039.stdout: "02509e4f-3911-4983-b16f-df8ded0b9998": { 2024-09-21T03:46:19.476 INFO:teuthology.orchestra.run.smithi039.stdout: "start_time": "2024-09-21T03:46:17.889576+0000", 2024-09-21T03:46:19.476 INFO:teuthology.orchestra.run.smithi039.stdout: "path": "/", 2024-09-21T03:46:19.478 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_started": 0, 2024-09-21T03:46:19.478 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_passed": 0, 2024-09-21T03:46:19.478 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_failed": 0, 2024-09-21T03:46:19.478 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_skipped": 0 2024-09-21T03:46:19.478 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-21T03:46:19.478 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-21T03:46:19.478 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-09-21T03:46:19.479 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:19.469+0000 7fef72a5e640 1 -- 172.21.15.39:0/353422161 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7fef540b6680 msgr2=0x7fef540b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:46:19.479 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:19.469+0000 7fef72a5e640 1 --2- 172.21.15.39:0/353422161 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7fef540b6680 0x7fef540b8b90 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fef640aaa20 tx=0x7fef640e7c40 comp rx=0 tx=0).stop 2024-09-21T03:46:19.479 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:19.469+0000 7fef72a5e640 1 -- 172.21.15.39:0/353422161 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fef6c109e20 msgr2=0x7fef6c076190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:46:19.479 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:19.469+0000 7fef72a5e640 1 --2- 172.21.15.39:0/353422161 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fef6c109e20 0x7fef6c076190 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7fef6400eaa0 tx=0x7fef6400ef60 comp rx=0 tx=0).stop 2024-09-21T03:46:19.479 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:19.469+0000 7fef72a5e640 1 -- 172.21.15.39:0/353422161 shutdown_connections 2024-09-21T03:46:19.479 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:19.469+0000 7fef72a5e640 1 --2- 172.21.15.39:0/353422161 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7fef540b6680 0x7fef540b8b90 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:19.480 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:19.469+0000 7fef72a5e640 1 --2- 172.21.15.39:0/353422161 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fef6c109e20 0x7fef6c076190 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:19.480 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:19.469+0000 7fef72a5e640 1 --2- 172.21.15.39:0/353422161 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fef6c1094d0 0x7fef6c075c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:19.480 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:19.469+0000 7fef72a5e640 1 --2- 172.21.15.39:0/353422161 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fef6c103b90 0x7fef6c075710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:19.480 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:19.469+0000 7fef72a5e640 1 -- 172.21.15.39:0/353422161 >> 172.21.15.39:0/353422161 conn(0x7fef6c0ff600 msgr2=0x7fef6c101a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:46:19.480 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:19.469+0000 7fef72a5e640 1 -- 172.21.15.39:0/353422161 shutdown_connections 2024-09-21T03:46:19.480 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:19.470+0000 7fef72a5e640 1 -- 172.21.15.39:0/353422161 wait complete. 2024-09-21T03:46:19.480 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:46:19 smithi150 ceph-mon[34289]: from='client.15555 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:46:19.480 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:46:19 smithi150 ceph-mon[34289]: pgmap v333: 97 pgs: 97 active+clean; 657 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 1.4 KiB/s wr, 10 op/s 2024-09-21T03:46:19.480 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:46:19 smithi150 ceph-mon[34289]: scrub queued for path: / 2024-09-21T03:46:19.480 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:46:19 smithi150 ceph-mon[34289]: scrub summary: idle+waiting paths [/] 2024-09-21T03:46:19.481 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:46:19 smithi150 ceph-mon[34289]: scrub summary: active paths [/] 2024-09-21T03:46:19.481 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:46:19 smithi150 ceph-mon[34289]: scrub summary: idle+waiting paths [/] 2024-09-21T03:46:19.481 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:46:19 smithi150 ceph-mon[34289]: from='client.15567 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:46:19.481 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:46:19 smithi150 ceph-mon[34289]: mgrmap e25: x(active, since 8m), standbys: z, y 2024-09-21T03:46:20.184 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:46:20 smithi039 ceph-mon[31211]: from='client.15591 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:46:20.184 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:46:20 smithi039 ceph-mon[31211]: from='client.15597 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:46:20.213 INFO:tasks.cephfs.filesystem:scrub status for tag:02509e4f-3911-4983-b16f-df8ded0b9998 - {'path': '/', 'tag': '02509e4f-3911-4983-b16f-df8ded0b9998', 'options': 'recursive,force'} 2024-09-21T03:46:20.468 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:46:20 smithi161 ceph-mon[34533]: from='client.15591 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:46:20.468 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:46:20 smithi161 ceph-mon[34533]: from='client.15597 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:46:20.474 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:46:20 smithi150 ceph-mon[34289]: from='client.15591 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:46:20.474 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:46:20 smithi150 ceph-mon[34289]: from='client.15597 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:46:21.333 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:46:21 smithi039 ceph-mon[31211]: pgmap v334: 97 pgs: 97 active+clean; 1.0 MiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 74 KiB/s wr, 21 op/s 2024-09-21T03:46:21.467 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:46:21 smithi161 ceph-mon[34533]: pgmap v334: 97 pgs: 97 active+clean; 1.0 MiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 74 KiB/s wr, 21 op/s 2024-09-21T03:46:21.474 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:46:21 smithi150 ceph-mon[34289]: pgmap v334: 97 pgs: 97 active+clean; 1.0 MiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 74 KiB/s wr, 21 op/s 2024-09-21T03:46:23.225 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:46:23 smithi150 ceph-mon[34289]: pgmap v335: 97 pgs: 97 active+clean; 1.4 MiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 175 KiB/s wr, 57 op/s 2024-09-21T03:46:23.225 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:46:23 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:46:23.225 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:46:23 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:46:23.225 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:46:23 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:46:23.334 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:46:23 smithi039 ceph-mon[31211]: pgmap v335: 97 pgs: 97 active+clean; 1.4 MiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 175 KiB/s wr, 57 op/s 2024-09-21T03:46:23.334 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:46:23 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:46:23.334 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:46:23 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:46:23.334 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:46:23 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:46:23.467 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:46:23 smithi161 ceph-mon[34533]: pgmap v335: 97 pgs: 97 active+clean; 1.4 MiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 175 KiB/s wr, 57 op/s 2024-09-21T03:46:23.468 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:46:23 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:46:23.468 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:46:23 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:46:23.468 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:46:23 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:46:24.834 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:46:24 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:46:24.834 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:46:24 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:46:24.834 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:46:24 smithi039 ceph-mon[31211]: pgmap v336: 97 pgs: 97 active+clean; 1.4 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 175 KiB/s wr, 56 op/s 2024-09-21T03:46:24.834 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:46:24 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:46:24.834 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:46:24 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:46:24.834 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:46:24 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:46:24.834 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:46:24 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:46:24.834 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:46:24 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:46:24.835 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:46:24 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:46:24.835 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:46:24 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:46:24.968 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:46:24 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:46:24.992 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:46:24 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:46:24.992 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:46:24 smithi161 ceph-mon[34533]: pgmap v336: 97 pgs: 97 active+clean; 1.4 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 175 KiB/s wr, 56 op/s 2024-09-21T03:46:24.992 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:46:24 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:46:24.992 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:46:24 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:46:24.992 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:46:24 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:46:24.992 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:46:24 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:46:24.992 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:46:24 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:46:24.992 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:46:24 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:46:24.992 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:46:24 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:46:24.993 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:46:24 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:46:24.993 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:46:24 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:46:24.993 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:46:24 smithi150 ceph-mon[34289]: pgmap v336: 97 pgs: 97 active+clean; 1.4 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 175 KiB/s wr, 56 op/s 2024-09-21T03:46:24.993 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:46:24 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:46:24.993 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:46:24 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:46:24.993 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:46:24 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:46:24.993 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:46:24 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:46:24.993 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:46:24 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:46:24.993 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:46:24 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:46:24.994 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:46:24 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:46:26.967 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:46:26 smithi161 ceph-mon[34533]: pgmap v337: 97 pgs: 97 active+clean; 1.4 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 176 KiB/s wr, 60 op/s 2024-09-21T03:46:26.974 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:46:26 smithi150 ceph-mon[34289]: pgmap v337: 97 pgs: 97 active+clean; 1.4 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 176 KiB/s wr, 60 op/s 2024-09-21T03:46:27.083 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:46:26 smithi039 ceph-mon[31211]: pgmap v337: 97 pgs: 97 active+clean; 1.4 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 176 KiB/s wr, 60 op/s 2024-09-21T03:46:28.968 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:46:28 smithi161 ceph-mon[34533]: pgmap v338: 97 pgs: 97 active+clean; 1.4 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 175 KiB/s wr, 56 op/s 2024-09-21T03:46:28.974 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:46:28 smithi150 ceph-mon[34289]: pgmap v338: 97 pgs: 97 active+clean; 1.4 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 175 KiB/s wr, 56 op/s 2024-09-21T03:46:29.084 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:46:28 smithi039 ceph-mon[31211]: pgmap v338: 97 pgs: 97 active+clean; 1.4 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 175 KiB/s wr, 56 op/s 2024-09-21T03:46:30.968 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:46:30 smithi161 ceph-mon[34533]: pgmap v339: 97 pgs: 97 active+clean; 1.4 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 175 KiB/s wr, 58 op/s 2024-09-21T03:46:30.968 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:46:30 smithi161 ceph-mon[34533]: scrub summary: idle 2024-09-21T03:46:30.974 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:46:30 smithi150 ceph-mon[34289]: pgmap v339: 97 pgs: 97 active+clean; 1.4 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 175 KiB/s wr, 58 op/s 2024-09-21T03:46:30.974 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:46:30 smithi150 ceph-mon[34289]: scrub summary: idle 2024-09-21T03:46:31.084 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:46:30 smithi039 ceph-mon[31211]: pgmap v339: 97 pgs: 97 active+clean; 1.4 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 175 KiB/s wr, 58 op/s 2024-09-21T03:46:31.084 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:46:30 smithi039 ceph-mon[31211]: scrub summary: idle 2024-09-21T03:46:32.967 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:46:32 smithi161 ceph-mon[34533]: pgmap v340: 97 pgs: 97 active+clean; 1.4 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 102 KiB/s wr, 48 op/s 2024-09-21T03:46:32.974 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:46:32 smithi150 ceph-mon[34289]: pgmap v340: 97 pgs: 97 active+clean; 1.4 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 102 KiB/s wr, 48 op/s 2024-09-21T03:46:33.083 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:46:32 smithi039 ceph-mon[31211]: pgmap v340: 97 pgs: 97 active+clean; 1.4 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 102 KiB/s wr, 48 op/s 2024-09-21T03:46:34.968 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:46:34 smithi161 ceph-mon[34533]: pgmap v341: 97 pgs: 97 active+clean; 1.4 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 767 B/s wr, 9 op/s 2024-09-21T03:46:34.974 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:46:34 smithi150 ceph-mon[34289]: pgmap v341: 97 pgs: 97 active+clean; 1.4 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 767 B/s wr, 9 op/s 2024-09-21T03:46:35.084 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:46:34 smithi039 ceph-mon[31211]: pgmap v341: 97 pgs: 97 active+clean; 1.4 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 767 B/s wr, 9 op/s 2024-09-21T03:46:36.968 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:46:36 smithi161 ceph-mon[34533]: pgmap v342: 97 pgs: 97 active+clean; 1.4 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 597 B/s wr, 12 op/s 2024-09-21T03:46:36.974 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:46:36 smithi150 ceph-mon[34289]: pgmap v342: 97 pgs: 97 active+clean; 1.4 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 597 B/s wr, 12 op/s 2024-09-21T03:46:37.083 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:46:36 smithi039 ceph-mon[31211]: pgmap v342: 97 pgs: 97 active+clean; 1.4 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 597 B/s wr, 12 op/s 2024-09-21T03:46:38.968 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:46:38 smithi161 ceph-mon[34533]: pgmap v343: 97 pgs: 97 active+clean; 1.4 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-09-21T03:46:38.974 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:46:38 smithi150 ceph-mon[34289]: pgmap v343: 97 pgs: 97 active+clean; 1.4 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-09-21T03:46:39.083 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:46:38 smithi039 ceph-mon[31211]: pgmap v343: 97 pgs: 97 active+clean; 1.4 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-09-21T03:46:40.885 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:46:40 smithi039 ceph-mon[31211]: pgmap v344: 97 pgs: 97 active+clean; 1.4 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-09-21T03:46:40.967 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:46:40 smithi161 ceph-mon[34533]: pgmap v344: 97 pgs: 97 active+clean; 1.4 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-09-21T03:46:40.974 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:46:40 smithi150 ceph-mon[34289]: pgmap v344: 97 pgs: 97 active+clean; 1.4 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-09-21T03:46:42.968 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:46:42 smithi161 ceph-mon[34533]: pgmap v345: 97 pgs: 97 active+clean; 1.4 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-21T03:46:42.974 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:46:42 smithi150 ceph-mon[34289]: pgmap v345: 97 pgs: 97 active+clean; 1.4 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-21T03:46:43.083 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:46:42 smithi039 ceph-mon[31211]: pgmap v345: 97 pgs: 97 active+clean; 1.4 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-21T03:46:44.968 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:46:44 smithi161 ceph-mon[34533]: pgmap v346: 97 pgs: 97 active+clean; 1.4 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-09-21T03:46:44.975 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:46:44 smithi150 ceph-mon[34289]: pgmap v346: 97 pgs: 97 active+clean; 1.4 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-09-21T03:46:45.083 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:46:44 smithi039 ceph-mon[31211]: pgmap v346: 97 pgs: 97 active+clean; 1.4 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-09-21T03:46:46.968 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:46:46 smithi161 ceph-mon[34533]: pgmap v347: 97 pgs: 97 active+clean; 1.4 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 12 op/s 2024-09-21T03:46:46.974 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:46:46 smithi150 ceph-mon[34289]: pgmap v347: 97 pgs: 97 active+clean; 1.4 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 12 op/s 2024-09-21T03:46:47.083 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:46:46 smithi039 ceph-mon[31211]: pgmap v347: 97 pgs: 97 active+clean; 1.4 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 12 op/s 2024-09-21T03:46:48.968 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:46:48 smithi161 ceph-mon[34533]: pgmap v348: 97 pgs: 97 active+clean; 1.4 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-09-21T03:46:48.974 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:46:48 smithi150 ceph-mon[34289]: pgmap v348: 97 pgs: 97 active+clean; 1.4 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-09-21T03:46:49.083 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:46:48 smithi039 ceph-mon[31211]: pgmap v348: 97 pgs: 97 active+clean; 1.4 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-09-21T03:46:50.214 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 51116d5c-77ca-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub status 2024-09-21T03:46:50.507 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/51116d5c-77ca-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:46:50.949 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:46:50 smithi039 ceph-mon[31211]: pgmap v349: 97 pgs: 97 active+clean; 1.4 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-09-21T03:46:50.967 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:46:50 smithi161 ceph-mon[34533]: pgmap v349: 97 pgs: 97 active+clean; 1.4 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-09-21T03:46:50.974 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:46:50 smithi150 ceph-mon[34289]: pgmap v349: 97 pgs: 97 active+clean; 1.4 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-09-21T03:46:51.092 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:51.089+0000 7fd7c5ca6640 1 -- 172.21.15.39:0/2853628355 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd7c0102a60 msgr2=0x7fd7c0102e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:46:51.092 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:51.089+0000 7fd7c5ca6640 1 --2- 172.21.15.39:0/2853628355 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd7c0102a60 0x7fd7c0102e70 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7fd7b0009900 tx=0x7fd7b002a410 comp rx=0 tx=0).stop 2024-09-21T03:46:51.092 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:51.089+0000 7fd7c5ca6640 1 -- 172.21.15.39:0/2853628355 shutdown_connections 2024-09-21T03:46:51.092 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:51.089+0000 7fd7c5ca6640 1 --2- 172.21.15.39:0/2853628355 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd7c0108cf0 0x7fd7c010f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:51.092 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:51.089+0000 7fd7c5ca6640 1 --2- 172.21.15.39:0/2853628355 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd7c01083a0 0x7fd7c01087b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:51.092 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:51.089+0000 7fd7c5ca6640 1 --2- 172.21.15.39:0/2853628355 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd7c0102a60 0x7fd7c0102e70 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:51.093 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:51.089+0000 7fd7c5ca6640 1 -- 172.21.15.39:0/2853628355 >> 172.21.15.39:0/2853628355 conn(0x7fd7c0079730 msgr2=0x7fd7c0079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:46:51.093 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:51.089+0000 7fd7c5ca6640 1 -- 172.21.15.39:0/2853628355 shutdown_connections 2024-09-21T03:46:51.093 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:51.089+0000 7fd7c5ca6640 1 -- 172.21.15.39:0/2853628355 wait complete. 2024-09-21T03:46:51.094 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:51.090+0000 7fd7c5ca6640 1 Processor -- start 2024-09-21T03:46:51.094 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:51.090+0000 7fd7c5ca6640 1 -- start start 2024-09-21T03:46:51.094 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:51.090+0000 7fd7c5ca6640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd7c0102a60 0x7fd7c01a0a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:51.094 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:51.090+0000 7fd7c5ca6640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd7c01083a0 0x7fd7c01a1030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:51.094 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:51.090+0000 7fd7c5ca6640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd7c0108cf0 0x7fd7c01a62d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:51.094 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:51.091+0000 7fd7c5ca6640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fd7c0112960 con 0x7fd7c01083a0 2024-09-21T03:46:51.094 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:51.091+0000 7fd7c5ca6640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fd7c01127e0 con 0x7fd7c0102a60 2024-09-21T03:46:51.094 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:51.091+0000 7fd7c5ca6640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fd7c0112ae0 con 0x7fd7c0108cf0 2024-09-21T03:46:51.095 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:51.091+0000 7fd7beffd640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd7c01083a0 0x7fd7c01a1030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:46:51.095 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:51.091+0000 7fd7beffd640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd7c01083a0 0x7fd7c01a1030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:57318/0 (socket says 172.21.15.39:57318) 2024-09-21T03:46:51.095 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:51.091+0000 7fd7beffd640 1 -- 172.21.15.39:0/200675937 learned_addr learned my addr 172.21.15.39:0/200675937 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:46:51.095 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:51.091+0000 7fd7bf7fe640 1 --2- 172.21.15.39:0/200675937 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd7c0102a60 0x7fd7c01a0a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:46:51.095 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:51.091+0000 7fd7bffff640 1 --2- 172.21.15.39:0/200675937 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd7c0108cf0 0x7fd7c01a62d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:46:51.095 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:51.091+0000 7fd7beffd640 1 -- 172.21.15.39:0/200675937 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd7c0108cf0 msgr2=0x7fd7c01a62d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:46:51.095 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:51.091+0000 7fd7beffd640 1 --2- 172.21.15.39:0/200675937 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd7c0108cf0 0x7fd7c01a62d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:51.095 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:51.091+0000 7fd7beffd640 1 -- 172.21.15.39:0/200675937 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd7c0102a60 msgr2=0x7fd7c01a0a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:46:51.095 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:51.091+0000 7fd7beffd640 1 --2- 172.21.15.39:0/200675937 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd7c0102a60 0x7fd7c01a0a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:51.095 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:51.091+0000 7fd7beffd640 1 -- 172.21.15.39:0/200675937 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd7c01a6940 con 0x7fd7c01083a0 2024-09-21T03:46:51.095 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:51.091+0000 7fd7bf7fe640 1 --2- 172.21.15.39:0/200675937 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd7c0102a60 0x7fd7c01a0a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:46:51.095 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:51.091+0000 7fd7beffd640 1 --2- 172.21.15.39:0/200675937 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd7c01083a0 0x7fd7c01a1030 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7fd7b400c970 tx=0x7fd7b400ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:46:51.096 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:51.092+0000 7fd7bcff9640 1 -- 172.21.15.39:0/200675937 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7b4007bb0 con 0x7fd7c01083a0 2024-09-21T03:46:51.096 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:51.092+0000 7fd7c5ca6640 1 -- 172.21.15.39:0/200675937 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd7c01a6c30 con 0x7fd7c01083a0 2024-09-21T03:46:51.096 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:51.092+0000 7fd7bcff9640 1 -- 172.21.15.39:0/200675937 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd7b4007d50 con 0x7fd7c01083a0 2024-09-21T03:46:51.096 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:51.092+0000 7fd7bcff9640 1 -- 172.21.15.39:0/200675937 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7b4018730 con 0x7fd7c01083a0 2024-09-21T03:46:51.096 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:51.092+0000 7fd7c5ca6640 1 -- 172.21.15.39:0/200675937 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd7c01ad450 con 0x7fd7c01083a0 2024-09-21T03:46:51.097 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:51.095+0000 7fd7cc21a740 1 Processor -- start 2024-09-21T03:46:51.098 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:51.095+0000 7fd7cc21a740 1 -- start start 2024-09-21T03:46:51.098 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:51.095+0000 7fd7cc21a740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564311c3c070 0x564311fe3650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:51.098 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:51.095+0000 7fd7cc21a740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564311c131b0 0x564311fe3b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:51.099 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:51.095+0000 7fd7cc21a740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564311c1b390 0x564311fe40d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:51.099 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:51.095+0000 7fd7cc21a740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x564311f59e50 con 0x564311c131b0 2024-09-21T03:46:51.099 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:51.095+0000 7fd7cc21a740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x564311f49280 con 0x564311c1b390 2024-09-21T03:46:51.099 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:51.095+0000 7fd7cc21a740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x564311f4d090 con 0x564311c3c070 2024-09-21T03:46:51.099 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:51.096+0000 7fd7bf7fe640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564311c131b0 0x564311fe3b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:46:51.100 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:51.096+0000 7fd7bf7fe640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564311c131b0 0x564311fe3b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:57326/0 (socket says 172.21.15.39:57326) 2024-09-21T03:46:51.100 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:51.096+0000 7fd7bf7fe640 1 -- 172.21.15.39:0/2729542364 learned_addr learned my addr 172.21.15.39:0/2729542364 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:46:51.100 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:51.096+0000 7fd7bffff640 1 --2- 172.21.15.39:0/2729542364 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564311c3c070 0x564311fe3650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:46:51.100 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:51.096+0000 7fd7bcff9640 1 -- 172.21.15.39:0/200675937 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 25) ==== 154637+0+0 (secure 0 0 0) 0x7fd7b400f990 con 0x7fd7c01083a0 2024-09-21T03:46:51.100 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:51.096+0000 7fd7bffff640 1 -- 172.21.15.39:0/2729542364 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564311c1b390 msgr2=0x564311fe40d0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-21T03:46:51.101 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:51.096+0000 7fd7bffff640 1 --2- 172.21.15.39:0/2729542364 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564311c1b390 0x564311fe40d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:51.102 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:51.096+0000 7fd7bffff640 1 -- 172.21.15.39:0/2729542364 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564311c131b0 msgr2=0x564311fe3b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:46:51.102 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:51.096+0000 7fd7bffff640 1 --2- 172.21.15.39:0/2729542364 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564311c131b0 0x564311fe3b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:51.102 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:51.096+0000 7fd7bffff640 1 -- 172.21.15.39:0/2729542364 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564311ca7650 con 0x564311c3c070 2024-09-21T03:46:51.102 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:51.097+0000 7fd7bf7fe640 1 --2- 172.21.15.39:0/2729542364 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564311c131b0 0x564311fe3b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:46:51.102 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:51.097+0000 7fd7bffff640 1 --2- 172.21.15.39:0/2729542364 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564311c3c070 0x564311fe3650 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fd7b8007d90 tx=0x7fd7b800e720 comp rx=0 tx=0).ready entity=mon.1 client_cookie=88cbf95a6b91bb86 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:46:51.102 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:51.097+0000 7fd7a5ffb640 1 -- 172.21.15.39:0/2729542364 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7b8014070 con 0x564311c3c070 2024-09-21T03:46:51.102 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:51.097+0000 7fd7a5ffb640 1 -- 172.21.15.39:0/2729542364 <== mon.1 v2:172.21.15.161:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd7b8003e20 con 0x564311c3c070 2024-09-21T03:46:51.102 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:51.097+0000 7fd7a5ffb640 1 -- 172.21.15.39:0/2729542364 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7b8010690 con 0x564311c3c070 2024-09-21T03:46:51.102 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:51.097+0000 7fd7bcff9640 1 --2- 172.21.15.39:0/200675937 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7fd7a80b67b0 0x7fd7a80b8cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:51.102 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:51.098+0000 7fd7bf7fe640 1 --2- 172.21.15.39:0/200675937 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7fd7a80b67b0 0x7fd7a80b8cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:46:51.102 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:51.098+0000 7fd7bcff9640 1 -- 172.21.15.39:0/200675937 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8484+0+0 (secure 0 0 0) 0x7fd7b40e77e0 con 0x7fd7c01083a0 2024-09-21T03:46:51.102 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:51.098+0000 7fd7cc21a740 1 -- 172.21.15.39:0/2729542364 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564311c3c070 msgr2=0x564311fe3650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:46:51.102 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:51.098+0000 7fd7cc21a740 1 --2- 172.21.15.39:0/2729542364 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564311c3c070 0x564311fe3650 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fd7b8007d90 tx=0x7fd7b800e720 comp rx=0 tx=0).stop 2024-09-21T03:46:51.102 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:51.098+0000 7fd7bf7fe640 1 --2- 172.21.15.39:0/200675937 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7fd7a80b67b0 0x7fd7a80b8cc0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x564311fe22d0 tx=0x7fd7b0009640 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:46:51.103 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:51.098+0000 7fd7cc21a740 1 -- 172.21.15.39:0/2729542364 shutdown_connections 2024-09-21T03:46:51.103 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:51.098+0000 7fd7cc21a740 1 --2- 172.21.15.39:0/2729542364 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564311c1b390 0x564311fe40d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:51.103 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:51.098+0000 7fd7cc21a740 1 --2- 172.21.15.39:0/2729542364 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564311c131b0 0x564311fe3b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:51.103 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:51.098+0000 7fd7cc21a740 1 --2- 172.21.15.39:0/2729542364 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564311c3c070 0x564311fe3650 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:51.103 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:51.098+0000 7fd7cc21a740 1 -- 172.21.15.39:0/2729542364 >> 172.21.15.39:0/2729542364 conn(0x564311c30460 msgr2=0x564311fe1540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:46:51.103 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:51.098+0000 7fd7cc21a740 1 -- 172.21.15.39:0/2729542364 shutdown_connections 2024-09-21T03:46:51.103 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:51.098+0000 7fd7cc21a740 1 -- 172.21.15.39:0/2729542364 wait complete. 2024-09-21T03:46:51.103 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:51.099+0000 7fd7cc21a740 1 -- start start 2024-09-21T03:46:51.104 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:51.099+0000 7fd7cc21a740 1 Processor -- start 2024-09-21T03:46:51.105 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:51.100+0000 7fd7cc21a740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564311c131b0 0x564311feed50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:51.105 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:51.100+0000 7fd7cc21a740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564311c1b390 0x564311fef290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:51.105 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:51.100+0000 7fd7cc21a740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564311c3c070 0x564311ffb120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:51.105 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:51.100+0000 7fd7bf7fe640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564311c131b0 0x564311feed50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:46:51.105 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:51.100+0000 7fd7cc21a740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x564311daa700 con 0x564311c131b0 2024-09-21T03:46:51.105 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:51.100+0000 7fd7cc21a740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x564311ee4110 con 0x564311c3c070 2024-09-21T03:46:51.105 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:51.100+0000 7fd7cc21a740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x564311ff1e30 con 0x564311c1b390 2024-09-21T03:46:51.105 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:51.100+0000 7fd7bf7fe640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564311c131b0 0x564311feed50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:57330/0 (socket says 172.21.15.39:57330) 2024-09-21T03:46:51.105 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:51.100+0000 7fd7bf7fe640 1 -- 172.21.15.39:0/3997714185 learned_addr learned my addr 172.21.15.39:0/3997714185 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:46:51.105 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:51.100+0000 7fd7cc21a740 10 client.0 _ll_register_callbacks cb 0x564311efd990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:46:51.105 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:51.100+0000 7fd7beffd640 1 --2- 172.21.15.39:0/3997714185 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564311c1b390 0x564311fef290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:46:51.105 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:51.100+0000 7fd7bffff640 1 --2- 172.21.15.39:0/3997714185 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564311c3c070 0x564311ffb120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:46:51.106 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:51.100+0000 7fd7bf7fe640 1 -- 172.21.15.39:0/3997714185 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564311c1b390 msgr2=0x564311fef290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:46:51.106 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:51.100+0000 7fd7bf7fe640 1 --2- 172.21.15.39:0/3997714185 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564311c1b390 0x564311fef290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:51.106 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:51.100+0000 7fd7bf7fe640 1 -- 172.21.15.39:0/3997714185 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564311c3c070 msgr2=0x564311ffb120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:46:51.106 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:51.100+0000 7fd7bf7fe640 1 --2- 172.21.15.39:0/3997714185 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564311c3c070 0x564311ffb120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:51.106 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:51.100+0000 7fd7bf7fe640 1 -- 172.21.15.39:0/3997714185 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564311f11c40 con 0x564311c131b0 2024-09-21T03:46:51.106 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:51.100+0000 7fd7cc21a740 1 -- 172.21.15.39:0/3997714185 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd7b8003e20 con 0x564311c131b0 2024-09-21T03:46:51.106 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:51.100+0000 7fd7beffd640 1 --2- 172.21.15.39:0/3997714185 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564311c1b390 0x564311fef290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:46:51.106 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:51.101+0000 7fd7bf7fe640 1 --2- 172.21.15.39:0/3997714185 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564311c131b0 0x564311feed50 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7fd7b0006010 tx=0x7fd7b0009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=34d52ec0dc8d0048 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:46:51.106 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:51.101+0000 7fd79effd640 10 client.15627 ms_handle_connect on v2:172.21.15.39:3300/0 2024-09-21T03:46:51.106 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:51.101+0000 7fd79effd640 1 -- 172.21.15.39:0/3997714185 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7b004b030 con 0x564311c131b0 2024-09-21T03:46:51.106 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:51.101+0000 7fd79effd640 1 -- 172.21.15.39:0/3997714185 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8484+0+0 (secure 0 0 0) 0x7fd7b0003800 con 0x564311c131b0 2024-09-21T03:46:51.106 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:51.101+0000 7fd79effd640 1 -- 172.21.15.39:0/3997714185 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fd7b0035050 con 0x564311c131b0 2024-09-21T03:46:51.106 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:51.101+0000 7fd7cc21a740 10 client.15627 fetch_fsmap learned FSMap version 29 2024-09-21T03:46:51.106 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:51.101+0000 7fd7cc21a740 1 -- 172.21.15.39:0/3997714185 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({fsmap=29}) -- 0x564311ffb6d0 con 0x564311c131b0 2024-09-21T03:46:51.107 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:51.102+0000 7fd79effd640 1 -- 172.21.15.39:0/3997714185 <== mon.0 v2:172.21.15.39:3300/0 4 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fd7b0003a00 con 0x564311c131b0 2024-09-21T03:46:51.107 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:51.102+0000 7fd7cc21a740 10 client.15627 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T03:46:51.107 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:51.102+0000 7fd7cc21a740 10 client.15627 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T03:46:51.107 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:51.102+0000 7fd7cc21a740 20 client.15627 populate_metadata read hostname 'smithi039' 2024-09-21T03:46:51.107 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:51.102+0000 7fd7cc21a740 1 --2- 172.21.15.39:0/3997714185 >> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] conn(0x564311bfa9f0 0x564311ffec20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:51.107 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:51.102+0000 7fd7cc21a740 4 client.15627 mds_command: new command op to 24695 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:46:51.107 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:51.102+0000 7fd7cc21a740 1 -- 172.21.15.39:0/3997714185 --> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564311f5bda0 con 0x564311bfa9f0 2024-09-21T03:46:51.107 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:51.102+0000 7fd7beffd640 1 --2- 172.21.15.39:0/3997714185 >> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] conn(0x564311bfa9f0 0x564311ffec20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:46:51.108 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:51.103+0000 7fd7beffd640 1 --2- 172.21.15.39:0/3997714185 >> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] conn(0x564311bfa9f0 0x564311ffec20 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=507fc3b8a61a283 server_cookie=4ce1b30137d896bf in_seq=0 out_seq=0 2024-09-21T03:46:51.108 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:51.103+0000 7fd79effd640 10 client.15627 ms_handle_connect on v2:172.21.15.150:6838/2949386907 2024-09-21T03:46:51.108 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:51.103+0000 7fd79effd640 1 -- 172.21.15.39:0/3997714185 <== mds.0 v2:172.21.15.150:6838/2949386907 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x564311f5bda0 con 0x564311bfa9f0 2024-09-21T03:46:51.108 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:51.103+0000 7fd79effd640 10 client.15627 handle_command_reply: tid=0 2024-09-21T03:46:51.108 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:51.103+0000 7fd7cc21a740 1 client.15627 shutdown 2024-09-21T03:46:51.108 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:51.104+0000 7fd7cc21a740 1 -- 172.21.15.39:0/3997714185 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564311c131b0 msgr2=0x564311feed50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:46:51.108 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:51.104+0000 7fd7cc21a740 1 --2- 172.21.15.39:0/3997714185 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564311c131b0 0x564311feed50 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7fd7b0006010 tx=0x7fd7b0009260 comp rx=0 tx=0).stop 2024-09-21T03:46:51.109 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:51.104+0000 7fd7cc21a740 1 -- 172.21.15.39:0/3997714185 shutdown_connections 2024-09-21T03:46:51.109 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:51.104+0000 7fd7cc21a740 1 --2- 172.21.15.39:0/3997714185 >> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] conn(0x564311bfa9f0 0x564311ffec20 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:51.109 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:51.104+0000 7fd7cc21a740 1 --2- 172.21.15.39:0/3997714185 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564311c3c070 0x564311ffb120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:51.109 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:51.104+0000 7fd7cc21a740 1 --2- 172.21.15.39:0/3997714185 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564311c1b390 0x564311fef290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:51.109 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:51.104+0000 7fd7cc21a740 1 --2- 172.21.15.39:0/3997714185 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564311c131b0 0x564311feed50 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:51.109 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:51.104+0000 7fd7cc21a740 1 -- 172.21.15.39:0/3997714185 >> 172.21.15.39:0/3997714185 conn(0x564311c30460 msgr2=0x564311fe67e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:46:51.109 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:51.104+0000 7fd79effd640 0 client.15627 ms_handle_reset on v2:172.21.15.150:6838/2949386907 2024-09-21T03:46:51.109 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:51.104+0000 7fd7cc21a740 1 -- 172.21.15.39:0/3997714185 shutdown_connections 2024-09-21T03:46:51.109 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:51.104+0000 7fd7cc21a740 1 -- 172.21.15.39:0/3997714185 wait complete. 2024-09-21T03:46:51.109 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:51.104+0000 7fd7cc21a740 20 client.15627 trim_cache size 0 max 16384 2024-09-21T03:46:51.125 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:51.122+0000 7fd7cc21a740 1 Processor -- start 2024-09-21T03:46:51.125 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:51.122+0000 7fd7cc21a740 1 -- start start 2024-09-21T03:46:51.125 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:51.122+0000 7fd7cc21a740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564311c131b0 0x564311ffb020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:51.125 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:51.122+0000 7fd7cc21a740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564311c1b390 0x56431200c140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:51.125 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:51.122+0000 7fd7cc21a740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564311c3c070 0x564311febf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:51.125 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:51.122+0000 7fd7cc21a740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fd7c0075250 con 0x564311c3c070 2024-09-21T03:46:51.125 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:51.122+0000 7fd7cc21a740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fd7c01aedc0 con 0x564311c131b0 2024-09-21T03:46:51.125 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:51.122+0000 7fd7cc21a740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fd7c01ad950 con 0x564311c1b390 2024-09-21T03:46:51.126 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:51.122+0000 7fd7bffff640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564311c3c070 0x564311febf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:46:51.126 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:51.122+0000 7fd7bffff640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564311c3c070 0x564311febf40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:57340/0 (socket says 172.21.15.39:57340) 2024-09-21T03:46:51.126 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:51.122+0000 7fd7bffff640 1 -- 172.21.15.39:0/3770416708 learned_addr learned my addr 172.21.15.39:0/3770416708 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:46:51.126 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:51.123+0000 7fd7beffd640 1 --2- 172.21.15.39:0/3770416708 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564311c1b390 0x56431200c140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:46:51.127 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:51.123+0000 7fd7bf7fe640 1 --2- 172.21.15.39:0/3770416708 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564311c131b0 0x564311ffb020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:46:51.127 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:51.123+0000 7fd7bffff640 1 -- 172.21.15.39:0/3770416708 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564311c1b390 msgr2=0x56431200c140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:46:51.127 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:51.123+0000 7fd7bffff640 1 --2- 172.21.15.39:0/3770416708 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564311c1b390 0x56431200c140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:51.127 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:51.123+0000 7fd7bffff640 1 -- 172.21.15.39:0/3770416708 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564311c131b0 msgr2=0x564311ffb020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:46:51.127 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:51.123+0000 7fd7bffff640 1 --2- 172.21.15.39:0/3770416708 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564311c131b0 0x564311ffb020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:51.127 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:51.123+0000 7fd7bffff640 1 -- 172.21.15.39:0/3770416708 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564311fe5940 con 0x564311c3c070 2024-09-21T03:46:51.127 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:51.123+0000 7fd7beffd640 1 --2- 172.21.15.39:0/3770416708 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564311c1b390 0x56431200c140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:46:51.127 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:51.123+0000 7fd7bffff640 1 --2- 172.21.15.39:0/3770416708 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564311c3c070 0x564311febf40 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7fd7b8007d90 tx=0x7fd7b8007550 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fdccf5d13f0d9ce server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:46:51.127 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:51.123+0000 7fd7a57fa640 1 -- 172.21.15.39:0/3770416708 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7b804a070 con 0x564311c3c070 2024-09-21T03:46:51.127 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:51.124+0000 7fd7a57fa640 1 -- 172.21.15.39:0/3770416708 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd7b8003b70 con 0x564311c3c070 2024-09-21T03:46:51.127 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:51.124+0000 7fd7a57fa640 1 -- 172.21.15.39:0/3770416708 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7b8014050 con 0x564311c3c070 2024-09-21T03:46:51.128 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:51.124+0000 7fd7cc21a740 1 -- 172.21.15.39:0/3770416708 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564311c3c070 msgr2=0x564311febf40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:46:51.128 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:51.124+0000 7fd7cc21a740 1 --2- 172.21.15.39:0/3770416708 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564311c3c070 0x564311febf40 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7fd7b8007d90 tx=0x7fd7b8007550 comp rx=0 tx=0).stop 2024-09-21T03:46:51.128 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:51.125+0000 7fd7cc21a740 1 -- 172.21.15.39:0/3770416708 shutdown_connections 2024-09-21T03:46:51.128 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:51.125+0000 7fd7cc21a740 1 --2- 172.21.15.39:0/3770416708 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564311c3c070 0x564311febf40 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:51.128 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:51.125+0000 7fd7cc21a740 1 --2- 172.21.15.39:0/3770416708 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564311c1b390 0x56431200c140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:51.129 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:51.125+0000 7fd7cc21a740 1 --2- 172.21.15.39:0/3770416708 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564311c131b0 0x564311ffb020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:51.129 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:51.125+0000 7fd7cc21a740 1 -- 172.21.15.39:0/3770416708 >> 172.21.15.39:0/3770416708 conn(0x564311c30460 msgr2=0x564311fed5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:46:51.129 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:51.125+0000 7fd7cc21a740 1 -- 172.21.15.39:0/3770416708 shutdown_connections 2024-09-21T03:46:51.129 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:51.125+0000 7fd7cc21a740 1 -- 172.21.15.39:0/3770416708 wait complete. 2024-09-21T03:46:51.129 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:51.125+0000 7fd7cc21a740 1 -- start start 2024-09-21T03:46:51.129 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:51.126+0000 7fd7cc21a740 1 Processor -- start 2024-09-21T03:46:51.130 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:51.126+0000 7fd7cc21a740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564311c131b0 0x564311ff7670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:51.130 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:51.126+0000 7fd7cc21a740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564311c1b390 0x564311ff2040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:51.130 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:51.126+0000 7fd7cc21a740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564311c3c070 0x564311ff2580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:51.130 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:51.126+0000 7fd7beffd640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564311c1b390 0x564311ff2040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:46:51.130 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:51.126+0000 7fd7beffd640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564311c1b390 0x564311ff2040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:57348/0 (socket says 172.21.15.39:57348) 2024-09-21T03:46:51.131 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:51.126+0000 7fd7beffd640 1 -- 172.21.15.39:0/2938168572 learned_addr learned my addr 172.21.15.39:0/2938168572 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:46:51.131 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:51.127+0000 7fd7bf7fe640 1 --2- 172.21.15.39:0/2938168572 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564311c131b0 0x564311ff7670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:46:51.131 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:51.127+0000 7fd7cc21a740 1 -- 172.21.15.39:0/2938168572 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fd7c01ae6f0 con 0x564311c1b390 2024-09-21T03:46:51.131 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:51.127+0000 7fd7cc21a740 1 -- 172.21.15.39:0/2938168572 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fd7c01ae020 con 0x564311c3c070 2024-09-21T03:46:51.131 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:51.127+0000 7fd7cc21a740 1 -- 172.21.15.39:0/2938168572 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fd7c0102eb0 con 0x564311c131b0 2024-09-21T03:46:51.131 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:51.127+0000 7fd7bffff640 1 --2- 172.21.15.39:0/2938168572 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564311c3c070 0x564311ff2580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:46:51.131 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:51.127+0000 7fd7beffd640 1 -- 172.21.15.39:0/2938168572 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564311c131b0 msgr2=0x564311ff7670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:46:51.131 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:51.127+0000 7fd7beffd640 1 --2- 172.21.15.39:0/2938168572 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564311c131b0 0x564311ff7670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:51.131 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:51.127+0000 7fd7beffd640 1 -- 172.21.15.39:0/2938168572 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564311c3c070 msgr2=0x564311ff2580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:46:51.131 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:51.127+0000 7fd7beffd640 1 --2- 172.21.15.39:0/2938168572 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564311c3c070 0x564311ff2580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:51.131 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:51.127+0000 7fd7beffd640 1 -- 172.21.15.39:0/2938168572 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564311ff2b10 con 0x564311c1b390 2024-09-21T03:46:51.131 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:51.127+0000 7fd7bf7fe640 1 --2- 172.21.15.39:0/2938168572 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564311c131b0 0x564311ff7670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:46:51.133 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:51.127+0000 7fd7bffff640 1 --2- 172.21.15.39:0/2938168572 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564311c3c070 0x564311ff2580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:46:51.133 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:51.127+0000 7fd7cc21a740 10 client.0 _ll_register_callbacks cb 0x7fd7b0009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:46:51.134 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:51.128+0000 7fd7beffd640 1 --2- 172.21.15.39:0/2938168572 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564311c1b390 0x564311ff2040 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7fd7b40aa860 tx=0x7fd7b400fc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a5975588ce864a00 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:46:51.134 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:51.128+0000 7fd79effd640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-09-21T03:46:51.134 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:51.128+0000 7fd7cc21a740 1 -- 172.21.15.39:0/2938168572 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd7b8003b70 con 0x564311c1b390 2024-09-21T03:46:51.134 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:51.128+0000 7fd79effd640 1 -- 172.21.15.39:0/2938168572 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7b40e5380 con 0x564311c1b390 2024-09-21T03:46:51.134 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:51.128+0000 7fd79effd640 1 -- 172.21.15.39:0/2938168572 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8484+0+0 (secure 0 0 0) 0x7fd7b40aca50 con 0x564311c1b390 2024-09-21T03:46:51.134 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:51.128+0000 7fd79effd640 1 -- 172.21.15.39:0/2938168572 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fd7b40ad100 con 0x564311c1b390 2024-09-21T03:46:51.134 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:51.128+0000 7fd7cc21a740 10 client.15633 fetch_fsmap learned FSMap version 29 2024-09-21T03:46:51.134 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:51.128+0000 7fd7cc21a740 1 -- 172.21.15.39:0/2938168572 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({fsmap=29}) -- 0x56431206b8e0 con 0x564311c1b390 2024-09-21T03:46:51.134 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:51.128+0000 7fd79effd640 1 -- 172.21.15.39:0/2938168572 <== mon.0 v2:172.21.15.39:3300/0 4 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fd7b40a8f40 con 0x564311c1b390 2024-09-21T03:46:51.134 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:51.129+0000 7fd7cc21a740 10 client.15633 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T03:46:51.135 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:51.129+0000 7fd7cc21a740 10 client.15633 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T03:46:51.135 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:51.129+0000 7fd7cc21a740 20 client.15633 populate_metadata read hostname 'smithi039' 2024-09-21T03:46:51.135 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:51.129+0000 7fd7cc21a740 1 --2- 172.21.15.39:0/2938168572 >> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] conn(0x564311bcfda0 0x56431206dfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:51.135 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:51.129+0000 7fd7cc21a740 4 client.15633 mds_command: new command op to 24695 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-21T03:46:51.135 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:51.129+0000 7fd7cc21a740 1 -- 172.21.15.39:0/2938168572 --> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] -- command(tid 0: {"prefix": "scrub status"}) -- 0x564311f57850 con 0x564311bcfda0 2024-09-21T03:46:51.135 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:51.129+0000 7fd7bf7fe640 1 --2- 172.21.15.39:0/2938168572 >> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] conn(0x564311bcfda0 0x56431206dfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:46:51.135 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:51.129+0000 7fd7bf7fe640 1 --2- 172.21.15.39:0/2938168572 >> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] conn(0x564311bcfda0 0x56431206dfe0 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=696a7665f35e1fdc server_cookie=a0d35a48244da2f8 in_seq=0 out_seq=0 2024-09-21T03:46:51.135 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:51.129+0000 7fd79effd640 10 client.15633 ms_handle_connect on v2:172.21.15.150:6838/2949386907 2024-09-21T03:46:51.135 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:51.130+0000 7fd79effd640 1 -- 172.21.15.39:0/2938168572 <== mds.0 v2:172.21.15.150:6838/2949386907 1 ==== command_reply(tid 0: 0 ) ==== 8+0+374 (crc 0 0 0) 0x564311f57850 con 0x564311bcfda0 2024-09-21T03:46:51.135 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:51.130+0000 7fd79effd640 10 client.15633 handle_command_reply: tid=0 2024-09-21T03:46:51.135 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:51.130+0000 7fd7cc21a740 1 client.15633 shutdown 2024-09-21T03:46:51.135 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:51.130+0000 7fd7cc21a740 1 -- 172.21.15.39:0/2938168572 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564311c1b390 msgr2=0x564311ff2040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:46:51.135 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:51.130+0000 7fd7cc21a740 1 --2- 172.21.15.39:0/2938168572 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564311c1b390 0x564311ff2040 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7fd7b40aa860 tx=0x7fd7b400fc80 comp rx=0 tx=0).stop 2024-09-21T03:46:51.135 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:51.130+0000 7fd7cc21a740 1 -- 172.21.15.39:0/2938168572 shutdown_connections 2024-09-21T03:46:51.135 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:51.130+0000 7fd7cc21a740 1 --2- 172.21.15.39:0/2938168572 >> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] conn(0x564311bcfda0 0x56431206dfe0 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:51.136 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:51.130+0000 7fd7cc21a740 1 --2- 172.21.15.39:0/2938168572 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564311c3c070 0x564311ff2580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:51.136 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:51.130+0000 7fd7cc21a740 1 --2- 172.21.15.39:0/2938168572 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564311c1b390 0x564311ff2040 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:51.136 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:51.130+0000 7fd79effd640 0 client.15633 ms_handle_reset on v2:172.21.15.150:6838/2949386907 2024-09-21T03:46:51.136 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:51.130+0000 7fd7cc21a740 1 --2- 172.21.15.39:0/2938168572 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564311c131b0 0x564311ff7670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:51.136 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:51.130+0000 7fd7cc21a740 1 -- 172.21.15.39:0/2938168572 >> 172.21.15.39:0/2938168572 conn(0x564311c30460 msgr2=0x564311ff0200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:46:51.136 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:51.130+0000 7fd7cc21a740 1 -- 172.21.15.39:0/2938168572 shutdown_connections 2024-09-21T03:46:51.136 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:51.131+0000 7fd7cc21a740 1 -- 172.21.15.39:0/2938168572 wait complete. 2024-09-21T03:46:51.136 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:51.131+0000 7fd7cc21a740 20 client.15633 trim_cache size 0 max 16384 2024-09-21T03:46:51.137 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-09-21T03:46:51.137 INFO:teuthology.orchestra.run.smithi039.stdout: "status": "no active scrubs running", 2024-09-21T03:46:51.137 INFO:teuthology.orchestra.run.smithi039.stdout: "scrubs": {}, 2024-09-21T03:46:51.137 INFO:teuthology.orchestra.run.smithi039.stdout: "scrub_stats": { 2024-09-21T03:46:51.137 INFO:teuthology.orchestra.run.smithi039.stdout: "02509e4f-3911-4983-b16f-df8ded0b9998": { 2024-09-21T03:46:51.137 INFO:teuthology.orchestra.run.smithi039.stdout: "start_time": "2024-09-21T03:46:17.889576+0000", 2024-09-21T03:46:51.137 INFO:teuthology.orchestra.run.smithi039.stdout: "path": "/", 2024-09-21T03:46:51.137 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_started": 0, 2024-09-21T03:46:51.137 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_passed": 0, 2024-09-21T03:46:51.137 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_failed": 0, 2024-09-21T03:46:51.137 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_skipped": 0 2024-09-21T03:46:51.137 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-21T03:46:51.137 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-21T03:46:51.138 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-09-21T03:46:51.138 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:51.131+0000 7fd7c5ca6640 1 -- 172.21.15.39:0/200675937 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7fd7a80b67b0 msgr2=0x7fd7a80b8cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:46:51.138 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:51.131+0000 7fd7c5ca6640 1 --2- 172.21.15.39:0/200675937 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7fd7a80b67b0 0x7fd7a80b8cc0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x564311fe22d0 tx=0x7fd7b0009640 comp rx=0 tx=0).stop 2024-09-21T03:46:51.138 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:51.131+0000 7fd7c5ca6640 1 -- 172.21.15.39:0/200675937 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd7c01083a0 msgr2=0x7fd7c01a1030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:46:51.138 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:51.131+0000 7fd7c5ca6640 1 --2- 172.21.15.39:0/200675937 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd7c01083a0 0x7fd7c01a1030 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7fd7b400c970 tx=0x7fd7b400ce30 comp rx=0 tx=0).stop 2024-09-21T03:46:51.138 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:51.131+0000 7fd7c5ca6640 1 -- 172.21.15.39:0/200675937 shutdown_connections 2024-09-21T03:46:51.139 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:51.131+0000 7fd7c5ca6640 1 --2- 172.21.15.39:0/200675937 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7fd7a80b67b0 0x7fd7a80b8cc0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:51.139 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:51.131+0000 7fd7c5ca6640 1 --2- 172.21.15.39:0/200675937 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd7c0108cf0 0x7fd7c01a62d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:51.139 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:51.131+0000 7fd7c5ca6640 1 --2- 172.21.15.39:0/200675937 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd7c01083a0 0x7fd7c01a1030 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:51.139 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:51.131+0000 7fd7c5ca6640 1 --2- 172.21.15.39:0/200675937 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd7c0102a60 0x7fd7c01a0a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:51.139 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:51.131+0000 7fd7c5ca6640 1 -- 172.21.15.39:0/200675937 >> 172.21.15.39:0/200675937 conn(0x7fd7c0079730 msgr2=0x7fd7c0101820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:46:51.139 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:51.132+0000 7fd7c5ca6640 1 -- 172.21.15.39:0/200675937 shutdown_connections 2024-09-21T03:46:51.139 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:51.132+0000 7fd7c5ca6640 1 -- 172.21.15.39:0/200675937 wait complete. 2024-09-21T03:46:51.802 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-21T03:46:51.802 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 51116d5c-77ca-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T03:46:52.108 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/51116d5c-77ca-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:46:52.612 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:52.609+0000 7f500c3f4640 1 -- 172.21.15.39:0/1748523435 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5004108cf0 msgr2=0x7f500410f520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:46:52.612 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:52.609+0000 7f500c3f4640 1 --2- 172.21.15.39:0/1748523435 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5004108cf0 0x7f500410f520 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f4ffc009ca0 tx=0x7f4ffc02a8c0 comp rx=0 tx=0).stop 2024-09-21T03:46:52.613 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:52.609+0000 7f500c3f4640 1 -- 172.21.15.39:0/1748523435 shutdown_connections 2024-09-21T03:46:52.613 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:52.609+0000 7f500c3f4640 1 --2- 172.21.15.39:0/1748523435 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5004108cf0 0x7f500410f520 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:52.613 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:52.609+0000 7f500c3f4640 1 --2- 172.21.15.39:0/1748523435 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f50041083a0 0x7f50041087b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:52.613 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:52.609+0000 7f500c3f4640 1 --2- 172.21.15.39:0/1748523435 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5004102a60 0x7f5004102e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:52.613 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:52.609+0000 7f500c3f4640 1 -- 172.21.15.39:0/1748523435 >> 172.21.15.39:0/1748523435 conn(0x7f5004079730 msgr2=0x7f5004079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:46:52.613 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:52.609+0000 7f500c3f4640 1 -- 172.21.15.39:0/1748523435 shutdown_connections 2024-09-21T03:46:52.613 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:52.609+0000 7f500c3f4640 1 -- 172.21.15.39:0/1748523435 wait complete. 2024-09-21T03:46:52.615 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:52.610+0000 7f500c3f4640 1 Processor -- start 2024-09-21T03:46:52.615 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:52.610+0000 7f500c3f4640 1 -- start start 2024-09-21T03:46:52.615 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:52.611+0000 7f500c3f4640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5004102a60 0x7f500419a4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:52.615 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:52.611+0000 7f500c3f4640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f50041083a0 0x7f500419aa90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:52.615 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:52.611+0000 7f500c3f4640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5004108cf0 0x7f500419fd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:52.615 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:52.611+0000 7f500c3f4640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f5004113a80 con 0x7f50041083a0 2024-09-21T03:46:52.615 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:52.611+0000 7f500c3f4640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f5004113900 con 0x7f5004102a60 2024-09-21T03:46:52.615 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:52.611+0000 7f500c3f4640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f5004113c00 con 0x7f5004108cf0 2024-09-21T03:46:52.615 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:52.611+0000 7f500998f640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f50041083a0 0x7f500419aa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:46:52.615 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:52.611+0000 7f500998f640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f50041083a0 0x7f500419aa90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:57360/0 (socket says 172.21.15.39:57360) 2024-09-21T03:46:52.616 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:52.611+0000 7f500998f640 1 -- 172.21.15.39:0/2480229859 learned_addr learned my addr 172.21.15.39:0/2480229859 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:46:52.616 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:52.611+0000 7f500a190640 1 --2- 172.21.15.39:0/2480229859 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5004102a60 0x7f500419a4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:46:52.616 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:52.611+0000 7f500a991640 1 --2- 172.21.15.39:0/2480229859 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5004108cf0 0x7f500419fd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:46:52.616 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:52.611+0000 7f500998f640 1 -- 172.21.15.39:0/2480229859 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5004108cf0 msgr2=0x7f500419fd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:46:52.616 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:52.611+0000 7f500998f640 1 --2- 172.21.15.39:0/2480229859 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5004108cf0 0x7f500419fd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:52.616 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:52.611+0000 7f500998f640 1 -- 172.21.15.39:0/2480229859 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5004102a60 msgr2=0x7f500419a4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:46:52.616 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:52.611+0000 7f500998f640 1 --2- 172.21.15.39:0/2480229859 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5004102a60 0x7f500419a4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:52.616 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:52.611+0000 7f500998f640 1 -- 172.21.15.39:0/2480229859 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f50041a03a0 con 0x7f50041083a0 2024-09-21T03:46:52.616 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:52.612+0000 7f500998f640 1 --2- 172.21.15.39:0/2480229859 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f50041083a0 0x7f500419aa90 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f4ff400c970 tx=0x7f4ff400ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:46:52.616 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:52.612+0000 7f4ffb7fe640 1 -- 172.21.15.39:0/2480229859 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ff4014070 con 0x7f50041083a0 2024-09-21T03:46:52.616 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:52.612+0000 7f500c3f4640 1 -- 172.21.15.39:0/2480229859 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f50041a0690 con 0x7f50041083a0 2024-09-21T03:46:52.616 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:52.612+0000 7f500c3f4640 1 -- 172.21.15.39:0/2480229859 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f50041a6eb0 con 0x7f50041083a0 2024-09-21T03:46:52.616 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:52.612+0000 7f4ffb7fe640 1 -- 172.21.15.39:0/2480229859 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4ff4004440 con 0x7f50041083a0 2024-09-21T03:46:52.616 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:52.612+0000 7f4ffb7fe640 1 -- 172.21.15.39:0/2480229859 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ff4003e50 con 0x7f50041083a0 2024-09-21T03:46:52.617 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:52.613+0000 7f500c3f4640 1 -- 172.21.15.39:0/2480229859 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4fe8005290 con 0x7f50041083a0 2024-09-21T03:46:52.619 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:52.617+0000 7f4ffb7fe640 1 -- 172.21.15.39:0/2480229859 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 25) ==== 154637+0+0 (secure 0 0 0) 0x7f4ff4017070 con 0x7f50041083a0 2024-09-21T03:46:52.620 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:52.618+0000 7f4ffb7fe640 1 --2- 172.21.15.39:0/2480229859 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f4ff00b68d0 0x7f4ff00b8de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:52.620 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:52.618+0000 7f4ffb7fe640 1 -- 172.21.15.39:0/2480229859 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8484+0+0 (secure 0 0 0) 0x7f4ff40e75e0 con 0x7f50041083a0 2024-09-21T03:46:52.621 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:52.618+0000 7f500a190640 1 --2- 172.21.15.39:0/2480229859 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f4ff00b68d0 0x7f4ff00b8de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:46:52.622 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:52.619+0000 7f500a190640 1 --2- 172.21.15.39:0/2480229859 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f4ff00b68d0 0x7f4ff00b8de0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f5000005d00 tx=0x7f500000b410 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:46:52.622 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:52.619+0000 7f4ffb7fe640 1 -- 172.21.15.39:0/2480229859 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f4ff401e020 con 0x7f50041083a0 2024-09-21T03:46:52.833 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:46:52 smithi039 ceph-mon[31211]: pgmap v350: 97 pgs: 97 active+clean; 1.4 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-21T03:46:52.937 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:52.935+0000 7f500c3f4640 1 -- 172.21.15.39:0/2480229859 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f4fe8005580 con 0x7f50041083a0 2024-09-21T03:46:52.938 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:52.935+0000 7f4ffb7fe640 1 -- 172.21.15.39:0/2480229859 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11978 (secure 0 0 0) 0x7f4ff40adbd0 con 0x7f50041083a0 2024-09-21T03:46:52.938 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T03:46:52.939 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":29,"btime":"2024-09-21T03:46:08:982881+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14790,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6835/508995295","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":508995295},{"type":"v1","addr":"172.21.15.39:6835","nonce":508995295}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14802,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6837/3915912607","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":3915912607},{"type":"v1","addr":"172.21.15.39:6837","nonce":3915912607}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":29,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T03:44:15.009872+0000","modified":"2024-09-21T03:46:08.430113+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24695,"mds_1":24667,"mds_2":24659,"mds_3":24646,"mds_4":14868},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14856":{"gid":14856,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.39:6839/185509192","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6838","nonce":185509192},{"type":"v1","addr":"172.21.15.39:6839","nonce":185509192}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14868":{"gid":14868,"name":"a","rank":4,"incarnation":25,"state":"up:active","state_seq":7,"addr":"172.21.15.39:6841/4247190456","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6840","nonce":4247190456},{"type":"v1","addr":"172.21.15.39:6841","nonce":4247190456}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24599":{"gid":24599,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.150:6833/3114643217","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":3114643217},{"type":"v1","addr":"172.21.15.150:6833","nonce":3114643217}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24601":{"gid":24601,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.161:6833/2963990654","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":2963990654},{"type":"v1","addr":"172.21.15.161:6833","nonce":2963990654}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24632":{"gid":24632,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.150:6835/724594008","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6834","nonce":724594008},{"type":"v1","addr":"172.21.15.150:6835","nonce":724594008}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24640":{"gid":24640,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.161:6835/3747882110","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":3747882110},{"type":"v1","addr":"172.21.15.161:6835","nonce":3747882110}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24646":{"gid":24646,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":10,"addr":"172.21.15.161:6837/3451544597","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6836","nonce":3451544597},{"type":"v1","addr":"172.21.15.161:6837","nonce":3451544597}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24659":{"gid":24659,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.150:6837/4017043144","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6836","nonce":4017043144},{"type":"v1","addr":"172.21.15.150:6837","nonce":4017043144}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24667":{"gid":24667,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.161:6839/2190107270","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6838","nonce":2190107270},{"type":"v1","addr":"172.21.15.161:6839","nonce":2190107270}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24695":{"gid":24695,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.150:6839/2949386907","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6838","nonce":2949386907},{"type":"v1","addr":"172.21.15.150:6839","nonce":2949386907}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24695,"qdb_cluster":[24695,24667,24659,24646,14868]},"id":1}]} 2024-09-21T03:46:52.939 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 29 2024-09-21T03:46:52.942 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:52.939+0000 7f500c3f4640 1 -- 172.21.15.39:0/2480229859 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f4ff00b68d0 msgr2=0x7f4ff00b8de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:46:52.942 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:52.939+0000 7f500c3f4640 1 --2- 172.21.15.39:0/2480229859 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f4ff00b68d0 0x7f4ff00b8de0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f5000005d00 tx=0x7f500000b410 comp rx=0 tx=0).stop 2024-09-21T03:46:52.942 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:52.939+0000 7f500c3f4640 1 -- 172.21.15.39:0/2480229859 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f50041083a0 msgr2=0x7f500419aa90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:46:52.942 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:52.939+0000 7f500c3f4640 1 --2- 172.21.15.39:0/2480229859 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f50041083a0 0x7f500419aa90 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f4ff400c970 tx=0x7f4ff400ce30 comp rx=0 tx=0).stop 2024-09-21T03:46:52.942 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:52.939+0000 7f500c3f4640 1 -- 172.21.15.39:0/2480229859 shutdown_connections 2024-09-21T03:46:52.942 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:52.939+0000 7f500c3f4640 1 --2- 172.21.15.39:0/2480229859 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f4ff00b68d0 0x7f4ff00b8de0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:52.942 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:52.939+0000 7f500c3f4640 1 --2- 172.21.15.39:0/2480229859 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5004108cf0 0x7f500419fd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:52.942 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:52.939+0000 7f500c3f4640 1 --2- 172.21.15.39:0/2480229859 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f50041083a0 0x7f500419aa90 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:52.942 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:52.939+0000 7f500c3f4640 1 --2- 172.21.15.39:0/2480229859 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5004102a60 0x7f500419a4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:52.943 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:52.939+0000 7f500c3f4640 1 -- 172.21.15.39:0/2480229859 >> 172.21.15.39:0/2480229859 conn(0x7f5004079730 msgr2=0x7f5004105300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:46:52.943 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:52.939+0000 7f500c3f4640 1 -- 172.21.15.39:0/2480229859 shutdown_connections 2024-09-21T03:46:52.943 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:52.939+0000 7f500c3f4640 1 -- 172.21.15.39:0/2480229859 wait complete. 2024-09-21T03:46:52.968 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:46:52 smithi161 ceph-mon[34533]: pgmap v350: 97 pgs: 97 active+clean; 1.4 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-21T03:46:52.974 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:46:52 smithi150 ceph-mon[34289]: pgmap v350: 97 pgs: 97 active+clean; 1.4 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-21T03:46:53.553 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 51116d5c-77ca-11ef-baf5-efdc52797490 -- ceph tell mds.1:4 damage ls 2024-09-21T03:46:53.834 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:46:53 smithi039 ceph-mon[31211]: from='client.? 172.21.15.39:0/2480229859' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:46:53.885 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/51116d5c-77ca-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:46:53.967 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:46:53 smithi161 ceph-mon[34533]: from='client.? 172.21.15.39:0/2480229859' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:46:53.974 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:46:53 smithi150 ceph-mon[34289]: from='client.? 172.21.15.39:0/2480229859' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:46:54.429 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:54.426+0000 7f93b2c3c640 1 -- 172.21.15.39:0/4135216781 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f93ac108cf0 msgr2=0x7f93ac10f520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:46:54.430 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:54.426+0000 7f93b2c3c640 1 --2- 172.21.15.39:0/4135216781 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f93ac108cf0 0x7f93ac10f520 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f93a4009ca0 tx=0x7f93a402a8c0 comp rx=0 tx=0).stop 2024-09-21T03:46:54.430 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:54.426+0000 7f93b2c3c640 1 -- 172.21.15.39:0/4135216781 shutdown_connections 2024-09-21T03:46:54.430 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:54.426+0000 7f93b2c3c640 1 --2- 172.21.15.39:0/4135216781 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f93ac108cf0 0x7f93ac10f520 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:54.430 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:54.426+0000 7f93b2c3c640 1 --2- 172.21.15.39:0/4135216781 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f93ac1083a0 0x7f93ac1087b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:54.430 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:54.426+0000 7f93b2c3c640 1 --2- 172.21.15.39:0/4135216781 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f93ac102a60 0x7f93ac102e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:54.430 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:54.426+0000 7f93b2c3c640 1 -- 172.21.15.39:0/4135216781 >> 172.21.15.39:0/4135216781 conn(0x7f93ac079730 msgr2=0x7f93ac079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:46:54.430 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:54.426+0000 7f93b2c3c640 1 -- 172.21.15.39:0/4135216781 shutdown_connections 2024-09-21T03:46:54.430 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:54.427+0000 7f93b2c3c640 1 -- 172.21.15.39:0/4135216781 wait complete. 2024-09-21T03:46:54.432 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:54.427+0000 7f93b2c3c640 1 Processor -- start 2024-09-21T03:46:54.432 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:54.428+0000 7f93b2c3c640 1 -- start start 2024-09-21T03:46:54.432 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:54.428+0000 7f93b2c3c640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f93ac102a60 0x7f93ac19a490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:54.432 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:54.428+0000 7f93b2c3c640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f93ac1083a0 0x7f93ac19aa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:54.432 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:54.428+0000 7f93b2c3c640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f93ac108cf0 0x7f93ac19fcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:54.432 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:54.428+0000 7f93b2c3c640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f93ac113a80 con 0x7f93ac102a60 2024-09-21T03:46:54.432 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:54.428+0000 7f93b2c3c640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f93ac113900 con 0x7f93ac1083a0 2024-09-21T03:46:54.432 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:54.428+0000 7f93b2c3c640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f93ac113c00 con 0x7f93ac108cf0 2024-09-21T03:46:54.433 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:54.428+0000 7f93b09d8640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f93ac102a60 0x7f93ac19a490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:46:54.433 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:54.428+0000 7f93b09d8640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f93ac102a60 0x7f93ac19a490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:57378/0 (socket says 172.21.15.39:57378) 2024-09-21T03:46:54.433 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:54.428+0000 7f93b09d8640 1 -- 172.21.15.39:0/1808513267 learned_addr learned my addr 172.21.15.39:0/1808513267 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:46:54.433 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:54.428+0000 7f93b11d9640 1 --2- 172.21.15.39:0/1808513267 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f93ac108cf0 0x7f93ac19fcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:46:54.433 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:54.428+0000 7f93abfff640 1 --2- 172.21.15.39:0/1808513267 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f93ac1083a0 0x7f93ac19aa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:46:54.433 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:54.428+0000 7f93b09d8640 1 -- 172.21.15.39:0/1808513267 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f93ac108cf0 msgr2=0x7f93ac19fcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:46:54.433 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:54.428+0000 7f93b09d8640 1 --2- 172.21.15.39:0/1808513267 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f93ac108cf0 0x7f93ac19fcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:54.433 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:54.429+0000 7f93b09d8640 1 -- 172.21.15.39:0/1808513267 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f93ac1083a0 msgr2=0x7f93ac19aa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:46:54.433 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:54.429+0000 7f93b09d8640 1 --2- 172.21.15.39:0/1808513267 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f93ac1083a0 0x7f93ac19aa30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:54.433 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:54.429+0000 7f93b09d8640 1 -- 172.21.15.39:0/1808513267 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f93ac1a0400 con 0x7f93ac102a60 2024-09-21T03:46:54.433 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:54.429+0000 7f93b09d8640 1 --2- 172.21.15.39:0/1808513267 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f93ac102a60 0x7f93ac19a490 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f93a000d770 tx=0x7f93a000dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:46:54.433 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:54.429+0000 7f93a9ffb640 1 -- 172.21.15.39:0/1808513267 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93a001c070 con 0x7f93ac102a60 2024-09-21T03:46:54.433 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:54.429+0000 7f93a9ffb640 1 -- 172.21.15.39:0/1808513267 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f93a00054d0 con 0x7f93ac102a60 2024-09-21T03:46:54.434 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:54.429+0000 7f93b2c3c640 1 -- 172.21.15.39:0/1808513267 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f93ac1a06f0 con 0x7f93ac102a60 2024-09-21T03:46:54.434 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:54.429+0000 7f93a9ffb640 1 -- 172.21.15.39:0/1808513267 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93a00047a0 con 0x7f93ac102a60 2024-09-21T03:46:54.434 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:54.430+0000 7f93b2c3c640 1 -- 172.21.15.39:0/1808513267 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f93ac10fe90 con 0x7f93ac102a60 2024-09-21T03:46:54.435 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:54.433+0000 7f93b91b0740 1 Processor -- start 2024-09-21T03:46:54.437 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:54.433+0000 7f93b91b0740 1 -- start start 2024-09-21T03:46:54.438 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:54.433+0000 7f93b91b0740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558741986070 0x558741d2d290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:54.438 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:54.433+0000 7f93b91b0740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55874195d1b0 0x558741d2d7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:54.438 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:54.433+0000 7f93b91b0740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558741965390 0x558741d2dd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:54.438 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:54.433+0000 7f93b91b0740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x558741ca3ea0 con 0x55874195d1b0 2024-09-21T03:46:54.438 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:54.433+0000 7f93b91b0740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x558741c932d0 con 0x558741986070 2024-09-21T03:46:54.438 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:54.433+0000 7f93b91b0740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x558741c970e0 con 0x558741965390 2024-09-21T03:46:54.438 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:54.434+0000 7f93b11d9640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558741986070 0x558741d2d290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:46:54.438 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:54.434+0000 7f93b11d9640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558741986070 0x558741d2d290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.39:38188/0 (socket says 172.21.15.39:38188) 2024-09-21T03:46:54.438 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:54.434+0000 7f93a9ffb640 1 -- 172.21.15.39:0/1808513267 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 25) ==== 154637+0+0 (secure 0 0 0) 0x7f93a00049a0 con 0x7f93ac102a60 2024-09-21T03:46:54.438 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:54.434+0000 7f93b11d9640 1 -- 172.21.15.39:0/1346905780 learned_addr learned my addr 172.21.15.39:0/1346905780 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:46:54.438 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:54.434+0000 7f93abfff640 1 --2- 172.21.15.39:0/1346905780 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558741965390 0x558741d2dd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:46:54.438 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:54.434+0000 7f93b09d8640 1 --2- 172.21.15.39:0/1346905780 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55874195d1b0 0x558741d2d7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:46:54.439 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:54.434+0000 7f93b11d9640 1 -- 172.21.15.39:0/1346905780 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558741965390 msgr2=0x558741d2dd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:46:54.439 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:54.434+0000 7f93b11d9640 1 --2- 172.21.15.39:0/1346905780 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558741965390 0x558741d2dd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:54.439 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:54.434+0000 7f93b11d9640 1 -- 172.21.15.39:0/1346905780 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55874195d1b0 msgr2=0x558741d2d7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:46:54.439 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:54.434+0000 7f93b11d9640 1 --2- 172.21.15.39:0/1346905780 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55874195d1b0 0x558741d2d7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:54.439 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:54.434+0000 7f93b11d9640 1 -- 172.21.15.39:0/1346905780 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5587419f1650 con 0x558741986070 2024-09-21T03:46:54.439 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:54.434+0000 7f93b11d9640 1 --2- 172.21.15.39:0/1346905780 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558741986070 0x558741d2d290 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f93a40098b0 tx=0x7f93a40098e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8f156eb1d3d973c4 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:46:54.439 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:54.435+0000 7f9396ffd640 1 -- 172.21.15.39:0/1346905780 <== mon.2 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93a4031cf0 con 0x558741986070 2024-09-21T03:46:54.440 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:54.435+0000 7f9396ffd640 1 -- 172.21.15.39:0/1346905780 <== mon.2 v2:172.21.15.150:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f93a4036030 con 0x558741986070 2024-09-21T03:46:54.441 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:54.435+0000 7f93a9ffb640 1 --2- 172.21.15.39:0/1808513267 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f939c0b6680 0x7f939c0b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:54.441 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:54.435+0000 7f93b09d8640 1 --2- 172.21.15.39:0/1808513267 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f939c0b6680 0x7f939c0b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:46:54.441 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:54.435+0000 7f93a9ffb640 1 -- 172.21.15.39:0/1808513267 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8484+0+0 (secure 0 0 0) 0x7f93a00ef670 con 0x7f93ac102a60 2024-09-21T03:46:54.441 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:54.435+0000 7f9396ffd640 1 -- 172.21.15.39:0/1346905780 <== mon.2 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93a40042a0 con 0x558741986070 2024-09-21T03:46:54.441 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:54.436+0000 7f93b91b0740 1 -- 172.21.15.39:0/1346905780 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558741986070 msgr2=0x558741d2d290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:46:54.441 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:54.436+0000 7f93b09d8640 1 --2- 172.21.15.39:0/1808513267 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f939c0b6680 0x7f939c0b8b90 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x558741d2bf10 tx=0x7f93a0009ae0 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:46:54.441 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:54.436+0000 7f93b91b0740 1 --2- 172.21.15.39:0/1346905780 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558741986070 0x558741d2d290 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f93a40098b0 tx=0x7f93a40098e0 comp rx=0 tx=0).stop 2024-09-21T03:46:54.441 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:54.436+0000 7f93b91b0740 1 -- 172.21.15.39:0/1346905780 shutdown_connections 2024-09-21T03:46:54.441 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:54.436+0000 7f93b91b0740 1 --2- 172.21.15.39:0/1346905780 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558741965390 0x558741d2dd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:54.441 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:54.436+0000 7f93b91b0740 1 --2- 172.21.15.39:0/1346905780 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55874195d1b0 0x558741d2d7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:54.441 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:54.436+0000 7f93b91b0740 1 --2- 172.21.15.39:0/1346905780 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558741986070 0x558741d2d290 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:54.442 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:54.436+0000 7f93b91b0740 1 -- 172.21.15.39:0/1346905780 >> 172.21.15.39:0/1346905780 conn(0x55874197a460 msgr2=0x558741d2b180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:46:54.442 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:54.436+0000 7f93b91b0740 1 -- 172.21.15.39:0/1346905780 shutdown_connections 2024-09-21T03:46:54.442 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:54.436+0000 7f93b91b0740 1 -- 172.21.15.39:0/1346905780 wait complete. 2024-09-21T03:46:54.442 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:54.437+0000 7f93b91b0740 1 -- start start 2024-09-21T03:46:54.442 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:54.437+0000 7f93b91b0740 1 Processor -- start 2024-09-21T03:46:54.442 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:54.437+0000 7f93b91b0740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55874195d1b0 0x558741d38810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:54.442 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:54.437+0000 7f93b91b0740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558741965390 0x558741d38d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:54.442 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:54.438+0000 7f93b91b0740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558741986070 0x558741d39290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:54.442 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:54.438+0000 7f93b91b0740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x558741af4700 con 0x55874195d1b0 2024-09-21T03:46:54.442 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:54.438+0000 7f93b91b0740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x558741c2e160 con 0x558741965390 2024-09-21T03:46:54.442 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:54.438+0000 7f93b91b0740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x558741d3b890 con 0x558741986070 2024-09-21T03:46:54.442 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:54.438+0000 7f93b91b0740 10 client.0 _ll_register_callbacks cb 0x558741c479e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:46:54.443 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:54.438+0000 7f93abfff640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558741965390 0x558741d38d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:46:54.443 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:54.438+0000 7f93b11d9640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55874195d1b0 0x558741d38810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:46:54.443 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:54.438+0000 7f93abfff640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558741965390 0x558741d38d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.39:38194/0 (socket says 172.21.15.39:38194) 2024-09-21T03:46:54.445 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:54.438+0000 7f93abfff640 1 -- 172.21.15.39:0/611556177 learned_addr learned my addr 172.21.15.39:0/611556177 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:46:54.445 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:54.438+0000 7f93abfff640 1 -- 172.21.15.39:0/611556177 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558741986070 msgr2=0x558741d39290 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-21T03:46:54.446 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:54.438+0000 7f93abfff640 1 --2- 172.21.15.39:0/611556177 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558741986070 0x558741d39290 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:54.446 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:54.438+0000 7f93abfff640 1 -- 172.21.15.39:0/611556177 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55874195d1b0 msgr2=0x558741d38810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:46:54.446 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:54.438+0000 7f93abfff640 1 --2- 172.21.15.39:0/611556177 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55874195d1b0 0x558741d38810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:54.446 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:54.438+0000 7f93abfff640 1 -- 172.21.15.39:0/611556177 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558741c5bc90 con 0x558741965390 2024-09-21T03:46:54.446 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:54.438+0000 7f93b11d9640 1 --2- 172.21.15.39:0/611556177 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55874195d1b0 0x558741d38810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:46:54.446 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:54.439+0000 7f93abfff640 1 --2- 172.21.15.39:0/611556177 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558741965390 0x558741d38d50 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f9398004580 tx=0x7f9398012ca0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b1454331af6a83d5 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:46:54.446 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:54.439+0000 7f938bfff640 10 client.0 ms_handle_connect on v2:172.21.15.150:3300/0 2024-09-21T03:46:54.446 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:54.439+0000 7f938bfff640 1 -- 172.21.15.39:0/611556177 <== mon.2 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f939801f060 con 0x558741965390 2024-09-21T03:46:54.446 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:54.439+0000 7f93b91b0740 1 -- 172.21.15.39:0/611556177 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f93a4036030 con 0x558741965390 2024-09-21T03:46:54.446 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:54.439+0000 7f938bfff640 1 -- 172.21.15.39:0/611556177 <== mon.2 v2:172.21.15.150:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8484+0+0 (secure 0 0 0) 0x7f939800b570 con 0x558741965390 2024-09-21T03:46:54.446 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:54.439+0000 7f938bfff640 1 -- 172.21.15.39:0/611556177 <== mon.2 v2:172.21.15.150:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f939800b770 con 0x558741965390 2024-09-21T03:46:54.447 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:54.439+0000 7f93b91b0740 10 client.25262 fetch_fsmap learned FSMap version 29 2024-09-21T03:46:54.447 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:54.439+0000 7f93b91b0740 1 -- 172.21.15.39:0/611556177 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x558741d47590 con 0x558741965390 2024-09-21T03:46:54.447 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:54.440+0000 7f938bfff640 1 -- 172.21.15.39:0/611556177 <== mon.2 v2:172.21.15.150:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f93a4036030 con 0x558741965390 2024-09-21T03:46:54.447 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:54.440+0000 7f938bfff640 1 -- 172.21.15.39:0/611556177 <== mon.2 v2:172.21.15.150:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f9398019030 con 0x558741965390 2024-09-21T03:46:54.447 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:54.440+0000 7f93b91b0740 10 client.25262 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T03:46:54.447 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:54.440+0000 7f93b91b0740 10 client.25262 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.a 2024-09-21T03:46:54.447 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:54.440+0000 7f93b91b0740 20 client.25262 populate_metadata read hostname 'smithi039' 2024-09-21T03:46:54.447 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:54.440+0000 7f93b91b0740 1 --2- 172.21.15.39:0/611556177 >> [v2:172.21.15.39:6840/4247190456,v1:172.21.15.39:6841/4247190456] conn(0x5587419449f0 0x558741d4ab50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:54.447 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:54.440+0000 7f93b91b0740 4 client.25262 mds_command: new command op to 14868 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:46:54.447 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:54.440+0000 7f93b91b0740 1 -- 172.21.15.39:0/611556177 --> [v2:172.21.15.39:6840/4247190456,v1:172.21.15.39:6841/4247190456] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558741ca5df0 con 0x5587419449f0 2024-09-21T03:46:54.447 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:54.440+0000 7f93b11d9640 1 --2- 172.21.15.39:0/611556177 >> [v2:172.21.15.39:6840/4247190456,v1:172.21.15.39:6841/4247190456] conn(0x5587419449f0 0x558741d4ab50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:46:54.447 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:54.441+0000 7f938bfff640 10 client.25262 ms_handle_connect on v2:172.21.15.39:6840/4247190456 2024-09-21T03:46:54.448 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:54.441+0000 7f93b11d9640 1 --2- 172.21.15.39:0/611556177 >> [v2:172.21.15.39:6840/4247190456,v1:172.21.15.39:6841/4247190456] conn(0x5587419449f0 0x558741d4ab50 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=da9a668e3320016a server_cookie=2235b3f85e9e9413 in_seq=0 out_seq=0 2024-09-21T03:46:54.448 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:54.441+0000 7f938bfff640 1 -- 172.21.15.39:0/611556177 <== mds.4 v2:172.21.15.39:6840/4247190456 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x558741ca5df0 con 0x5587419449f0 2024-09-21T03:46:54.448 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:54.441+0000 7f938bfff640 10 client.25262 handle_command_reply: tid=0 2024-09-21T03:46:54.448 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:54.442+0000 7f93b91b0740 1 client.25262 shutdown 2024-09-21T03:46:54.448 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:54.442+0000 7f93b91b0740 1 -- 172.21.15.39:0/611556177 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558741965390 msgr2=0x558741d38d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:46:54.448 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:54.442+0000 7f93b91b0740 1 --2- 172.21.15.39:0/611556177 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558741965390 0x558741d38d50 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f9398004580 tx=0x7f9398012ca0 comp rx=0 tx=0).stop 2024-09-21T03:46:54.448 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:54.442+0000 7f93b91b0740 1 -- 172.21.15.39:0/611556177 shutdown_connections 2024-09-21T03:46:54.448 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:54.442+0000 7f93b91b0740 1 --2- 172.21.15.39:0/611556177 >> [v2:172.21.15.39:6840/4247190456,v1:172.21.15.39:6841/4247190456] conn(0x5587419449f0 0x558741d4ab50 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:54.448 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:54.442+0000 7f93b91b0740 1 --2- 172.21.15.39:0/611556177 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558741986070 0x558741d39290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:54.448 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:54.442+0000 7f93b91b0740 1 --2- 172.21.15.39:0/611556177 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558741965390 0x558741d38d50 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:54.448 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:54.442+0000 7f93b91b0740 1 --2- 172.21.15.39:0/611556177 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55874195d1b0 0x558741d38810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:54.448 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:54.442+0000 7f93b91b0740 1 -- 172.21.15.39:0/611556177 >> 172.21.15.39:0/611556177 conn(0x55874197a460 msgr2=0x558741d30420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:46:54.449 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:54.442+0000 7f938bfff640 0 client.25262 ms_handle_reset on v2:172.21.15.39:6840/4247190456 2024-09-21T03:46:54.449 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:54.442+0000 7f93b91b0740 1 -- 172.21.15.39:0/611556177 shutdown_connections 2024-09-21T03:46:54.449 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:54.442+0000 7f93b91b0740 1 -- 172.21.15.39:0/611556177 wait complete. 2024-09-21T03:46:54.449 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:54.442+0000 7f93b91b0740 20 client.25262 trim_cache size 0 max 16384 2024-09-21T03:46:54.459 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:54.456+0000 7f93b91b0740 1 Processor -- start 2024-09-21T03:46:54.459 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:54.456+0000 7f93b91b0740 1 -- start start 2024-09-21T03:46:54.459 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:54.456+0000 7f93b91b0740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55874195d1b0 0x558741d3a6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:54.459 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:54.456+0000 7f93b91b0740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558741965390 0x558741d30aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:54.460 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:54.456+0000 7f93b91b0740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558741986070 0x558741d2a730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:54.460 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:54.456+0000 7f93b91b0740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f93ac1ad460 con 0x558741986070 2024-09-21T03:46:54.460 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:54.456+0000 7f93b91b0740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f93ac1aefa0 con 0x558741965390 2024-09-21T03:46:54.460 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:54.456+0000 7f93b91b0740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f93ac1adb30 con 0x55874195d1b0 2024-09-21T03:46:54.460 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:54.456+0000 7f93b11d9640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558741986070 0x558741d2a730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:46:54.460 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:54.456+0000 7f93abfff640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558741965390 0x558741d30aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:46:54.460 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:54.456+0000 7f93abfff640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558741965390 0x558741d30aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.39:38202/0 (socket says 172.21.15.39:38202) 2024-09-21T03:46:54.460 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:54.456+0000 7f93abfff640 1 -- 172.21.15.39:0/3401556684 learned_addr learned my addr 172.21.15.39:0/3401556684 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:46:54.460 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:54.457+0000 7f93abfff640 1 -- 172.21.15.39:0/3401556684 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55874195d1b0 msgr2=0x558741d3a6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:46:54.460 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:54.457+0000 7f93abfff640 1 --2- 172.21.15.39:0/3401556684 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55874195d1b0 0x558741d3a6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:54.460 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:54.457+0000 7f93abfff640 1 -- 172.21.15.39:0/3401556684 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558741986070 msgr2=0x558741d2a730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:46:54.460 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:54.457+0000 7f93abfff640 1 --2- 172.21.15.39:0/3401556684 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558741986070 0x558741d2a730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:54.461 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:54.457+0000 7f93abfff640 1 -- 172.21.15.39:0/3401556684 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558741d2b690 con 0x558741965390 2024-09-21T03:46:54.462 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:54.457+0000 7f93b11d9640 1 --2- 172.21.15.39:0/3401556684 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558741986070 0x558741d2a730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:46:54.462 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:54.457+0000 7f93abfff640 1 --2- 172.21.15.39:0/3401556684 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558741965390 0x558741d30aa0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f9398000c00 tx=0x7f9398003ac0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6248cd7baa334542 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:46:54.462 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:54.458+0000 7f93967fc640 1 -- 172.21.15.39:0/3401556684 <== mon.2 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9398003ea0 con 0x558741965390 2024-09-21T03:46:54.462 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:54.458+0000 7f93967fc640 1 -- 172.21.15.39:0/3401556684 <== mon.2 v2:172.21.15.150:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f93980054e0 con 0x558741965390 2024-09-21T03:46:54.462 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:54.458+0000 7f93967fc640 1 -- 172.21.15.39:0/3401556684 <== mon.2 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f939800bea0 con 0x558741965390 2024-09-21T03:46:54.462 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:54.459+0000 7f93b91b0740 1 -- 172.21.15.39:0/3401556684 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558741965390 msgr2=0x558741d30aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:46:54.463 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:54.459+0000 7f93b91b0740 1 --2- 172.21.15.39:0/3401556684 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558741965390 0x558741d30aa0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f9398000c00 tx=0x7f9398003ac0 comp rx=0 tx=0).stop 2024-09-21T03:46:54.463 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:54.459+0000 7f93b91b0740 1 -- 172.21.15.39:0/3401556684 shutdown_connections 2024-09-21T03:46:54.463 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:54.459+0000 7f93b91b0740 1 --2- 172.21.15.39:0/3401556684 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558741986070 0x558741d2a730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:54.463 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:54.459+0000 7f93b91b0740 1 --2- 172.21.15.39:0/3401556684 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558741965390 0x558741d30aa0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:54.463 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:54.459+0000 7f93b91b0740 1 --2- 172.21.15.39:0/3401556684 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55874195d1b0 0x558741d3a6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:54.463 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:54.459+0000 7f93b91b0740 1 -- 172.21.15.39:0/3401556684 >> 172.21.15.39:0/3401556684 conn(0x55874197a460 msgr2=0x558741d30390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:46:54.463 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:54.459+0000 7f93b91b0740 1 -- 172.21.15.39:0/3401556684 shutdown_connections 2024-09-21T03:46:54.463 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:54.459+0000 7f93b91b0740 1 -- 172.21.15.39:0/3401556684 wait complete. 2024-09-21T03:46:54.463 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:54.459+0000 7f93b91b0740 1 -- start start 2024-09-21T03:46:54.463 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:54.459+0000 7f93b91b0740 1 Processor -- start 2024-09-21T03:46:54.466 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:54.460+0000 7f93b91b0740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55874195d1b0 0x558741d415c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:54.466 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:54.460+0000 7f93b91b0740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558741965390 0x558741d3cad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:54.466 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:54.460+0000 7f93b91b0740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558741986070 0x558741d3d1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:54.466 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:54.460+0000 7f93b91b0740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f93ac1ae8d0 con 0x558741965390 2024-09-21T03:46:54.466 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:54.460+0000 7f93b91b0740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f93ac1ae200 con 0x55874195d1b0 2024-09-21T03:46:54.466 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:54.460+0000 7f93b91b0740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f93ac1ab440 con 0x558741986070 2024-09-21T03:46:54.467 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:54.460+0000 7f93b91b0740 10 client.0 _ll_register_callbacks cb 0x7f93980113e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:46:54.467 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:54.460+0000 7f93abfff640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558741965390 0x558741d3cad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:46:54.467 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:54.460+0000 7f93abfff640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558741965390 0x558741d3cad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:57418/0 (socket says 172.21.15.39:57418) 2024-09-21T03:46:54.467 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:54.460+0000 7f93abfff640 1 -- 172.21.15.39:0/3101117821 learned_addr learned my addr 172.21.15.39:0/3101117821 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:46:54.467 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:54.461+0000 7f93b11d9640 1 --2- 172.21.15.39:0/3101117821 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55874195d1b0 0x558741d415c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:46:54.467 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:54.461+0000 7f93abfff640 1 -- 172.21.15.39:0/3101117821 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558741986070 msgr2=0x558741d3d1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:46:54.467 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:54.461+0000 7f93abfff640 1 --2- 172.21.15.39:0/3101117821 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558741986070 0x558741d3d1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:54.467 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:54.461+0000 7f93abfff640 1 -- 172.21.15.39:0/3101117821 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55874195d1b0 msgr2=0x558741d415c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:46:54.467 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:54.461+0000 7f93abfff640 1 --2- 172.21.15.39:0/3101117821 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55874195d1b0 0x558741d415c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:54.467 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:54.461+0000 7f93abfff640 1 -- 172.21.15.39:0/3101117821 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558741db4510 con 0x558741965390 2024-09-21T03:46:54.467 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:54.461+0000 7f93b11d9640 1 --2- 172.21.15.39:0/3101117821 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55874195d1b0 0x558741d415c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:46:54.467 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:54.461+0000 7f93abfff640 1 --2- 172.21.15.39:0/3101117821 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558741965390 0x558741d3cad0 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f939800fda0 tx=0x7f939800bbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3be418ff88bab916 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:46:54.468 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:54.461+0000 7f938bfff640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-09-21T03:46:54.468 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:54.462+0000 7f938bfff640 1 -- 172.21.15.39:0/3101117821 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9398039060 con 0x558741965390 2024-09-21T03:46:54.468 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:54.462+0000 7f93b91b0740 1 -- 172.21.15.39:0/3101117821 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f93980054e0 con 0x558741965390 2024-09-21T03:46:54.468 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:54.462+0000 7f938bfff640 1 -- 172.21.15.39:0/3101117821 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8484+0+0 (secure 0 0 0) 0x7f939800f5f0 con 0x558741965390 2024-09-21T03:46:54.468 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:54.462+0000 7f938bfff640 1 -- 172.21.15.39:0/3101117821 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f939801f040 con 0x558741965390 2024-09-21T03:46:54.468 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:54.462+0000 7f93b91b0740 10 client.15657 fetch_fsmap learned FSMap version 29 2024-09-21T03:46:54.468 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:54.462+0000 7f93b91b0740 1 -- 172.21.15.39:0/3101117821 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x558741db5690 con 0x558741965390 2024-09-21T03:46:54.468 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:54.462+0000 7f938bfff640 1 -- 172.21.15.39:0/3101117821 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f93980054e0 con 0x558741965390 2024-09-21T03:46:54.468 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:54.462+0000 7f938bfff640 1 -- 172.21.15.39:0/3101117821 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f9398019070 con 0x558741965390 2024-09-21T03:46:54.468 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:54.462+0000 7f93b91b0740 10 client.15657 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T03:46:54.468 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:54.463+0000 7f93b91b0740 10 client.15657 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.a 2024-09-21T03:46:54.468 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:54.463+0000 7f93b91b0740 20 client.15657 populate_metadata read hostname 'smithi039' 2024-09-21T03:46:54.468 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:54.463+0000 7f93b91b0740 1 --2- 172.21.15.39:0/3101117821 >> [v2:172.21.15.39:6840/4247190456,v1:172.21.15.39:6841/4247190456] conn(0x558741919da0 0x558741db7cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:54.469 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:54.463+0000 7f93b91b0740 4 client.15657 mds_command: new command op to 14868 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T03:46:54.469 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:54.463+0000 7f93b91b0740 1 -- 172.21.15.39:0/3101117821 --> [v2:172.21.15.39:6840/4247190456,v1:172.21.15.39:6841/4247190456] -- command(tid 0: {"prefix": "damage ls"}) -- 0x558741ca18a0 con 0x558741919da0 2024-09-21T03:46:54.469 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:54.463+0000 7f93b11d9640 1 --2- 172.21.15.39:0/3101117821 >> [v2:172.21.15.39:6840/4247190456,v1:172.21.15.39:6841/4247190456] conn(0x558741919da0 0x558741db7cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:46:54.469 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:54.463+0000 7f938bfff640 10 client.15657 ms_handle_connect on v2:172.21.15.39:6840/4247190456 2024-09-21T03:46:54.470 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:54.464+0000 7f93b11d9640 1 --2- 172.21.15.39:0/3101117821 >> [v2:172.21.15.39:6840/4247190456,v1:172.21.15.39:6841/4247190456] conn(0x558741919da0 0x558741db7cb0 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=926a09e395b8a7fd server_cookie=b54937ffded4f725 in_seq=0 out_seq=0 2024-09-21T03:46:54.470 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:54.464+0000 7f938bfff640 1 -- 172.21.15.39:0/3101117821 <== mds.4 v2:172.21.15.39:6840/4247190456 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x558741ca18a0 con 0x558741919da0 2024-09-21T03:46:54.470 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:54.464+0000 7f938bfff640 10 client.15657 handle_command_reply: tid=0 2024-09-21T03:46:54.470 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:54.464+0000 7f93b91b0740 1 client.15657 shutdown 2024-09-21T03:46:54.470 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:54.464+0000 7f93b91b0740 1 -- 172.21.15.39:0/3101117821 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558741965390 msgr2=0x558741d3cad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:46:54.471 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:54.464+0000 7f93b91b0740 1 --2- 172.21.15.39:0/3101117821 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558741965390 0x558741d3cad0 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f939800fda0 tx=0x7f939800bbc0 comp rx=0 tx=0).stop 2024-09-21T03:46:54.471 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:54.464+0000 7f93b91b0740 1 -- 172.21.15.39:0/3101117821 shutdown_connections 2024-09-21T03:46:54.471 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:54.464+0000 7f93b91b0740 1 --2- 172.21.15.39:0/3101117821 >> [v2:172.21.15.39:6840/4247190456,v1:172.21.15.39:6841/4247190456] conn(0x558741919da0 0x558741db7cb0 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:54.471 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:54.464+0000 7f938bfff640 0 client.15657 ms_handle_reset on v2:172.21.15.39:6840/4247190456 2024-09-21T03:46:54.471 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:54.464+0000 7f93b91b0740 1 --2- 172.21.15.39:0/3101117821 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558741986070 0x558741d3d1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:54.471 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:54.464+0000 7f93b91b0740 1 --2- 172.21.15.39:0/3101117821 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558741965390 0x558741d3cad0 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:54.471 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:54.465+0000 7f93b91b0740 1 --2- 172.21.15.39:0/3101117821 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55874195d1b0 0x558741d415c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:54.471 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:54.465+0000 7f93b91b0740 1 -- 172.21.15.39:0/3101117821 >> 172.21.15.39:0/3101117821 conn(0x55874197a460 msgr2=0x558741d3a450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:46:54.471 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:54.465+0000 7f93b91b0740 1 -- 172.21.15.39:0/3101117821 shutdown_connections 2024-09-21T03:46:54.471 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:54.465+0000 7f93b91b0740 1 -- 172.21.15.39:0/3101117821 wait complete. 2024-09-21T03:46:54.471 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:54.465+0000 7f93b91b0740 20 client.15657 trim_cache size 0 max 16384 2024-09-21T03:46:54.472 INFO:teuthology.orchestra.run.smithi039.stdout:[] 2024-09-21T03:46:54.472 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:54.466+0000 7f93b2c3c640 1 -- 172.21.15.39:0/1808513267 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f939c0b6680 msgr2=0x7f939c0b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:46:54.472 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:54.466+0000 7f93b2c3c640 1 --2- 172.21.15.39:0/1808513267 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f939c0b6680 0x7f939c0b8b90 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x558741d2bf10 tx=0x7f93a0009ae0 comp rx=0 tx=0).stop 2024-09-21T03:46:54.473 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:54.466+0000 7f93b2c3c640 1 -- 172.21.15.39:0/1808513267 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f93ac102a60 msgr2=0x7f93ac19a490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:46:54.473 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:54.466+0000 7f93b2c3c640 1 --2- 172.21.15.39:0/1808513267 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f93ac102a60 0x7f93ac19a490 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f93a000d770 tx=0x7f93a000dc30 comp rx=0 tx=0).stop 2024-09-21T03:46:54.473 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:54.466+0000 7f93b2c3c640 1 -- 172.21.15.39:0/1808513267 shutdown_connections 2024-09-21T03:46:54.473 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:54.466+0000 7f93b2c3c640 1 --2- 172.21.15.39:0/1808513267 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f939c0b6680 0x7f939c0b8b90 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:54.473 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:54.466+0000 7f93b2c3c640 1 --2- 172.21.15.39:0/1808513267 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f93ac108cf0 0x7f93ac19fcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:54.473 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:54.466+0000 7f93b2c3c640 1 --2- 172.21.15.39:0/1808513267 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f93ac1083a0 0x7f93ac19aa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:54.473 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:54.466+0000 7f93b2c3c640 1 --2- 172.21.15.39:0/1808513267 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f93ac102a60 0x7f93ac19a490 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:54.473 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:54.466+0000 7f93b2c3c640 1 -- 172.21.15.39:0/1808513267 >> 172.21.15.39:0/1808513267 conn(0x7f93ac079730 msgr2=0x7f93ac104460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:46:54.473 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:54.466+0000 7f93b2c3c640 1 -- 172.21.15.39:0/1808513267 shutdown_connections 2024-09-21T03:46:54.473 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:54.466+0000 7f93b2c3c640 1 -- 172.21.15.39:0/1808513267 wait complete. 2024-09-21T03:46:54.967 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:46:54 smithi161 ceph-mon[34533]: pgmap v351: 97 pgs: 97 active+clean; 1.4 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-09-21T03:46:54.974 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:46:54 smithi150 ceph-mon[34289]: pgmap v351: 97 pgs: 97 active+clean; 1.4 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-09-21T03:46:55.040 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:46:54 smithi039 ceph-mon[31211]: pgmap v351: 97 pgs: 97 active+clean; 1.4 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-09-21T03:46:55.279 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 51116d5c-77ca-11ef-baf5-efdc52797490 -- ceph tell mds.1:3 damage ls 2024-09-21T03:46:55.568 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/51116d5c-77ca-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:46:56.072 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:56.064+0000 7efe9dffb640 1 -- 172.21.15.39:0/3259069730 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe900318a0 con 0x7efea0102a60 2024-09-21T03:46:56.072 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:56.064+0000 7efea587a640 1 -- 172.21.15.39:0/3259069730 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efea0102a60 msgr2=0x7efea0102e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:46:56.072 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:56.064+0000 7efea587a640 1 --2- 172.21.15.39:0/3259069730 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efea0102a60 0x7efea0102e70 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7efe90009900 tx=0x7efe9002a410 comp rx=0 tx=0).stop 2024-09-21T03:46:56.072 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:56.065+0000 7efea587a640 1 -- 172.21.15.39:0/3259069730 shutdown_connections 2024-09-21T03:46:56.072 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:56.065+0000 7efea587a640 1 --2- 172.21.15.39:0/3259069730 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7efea0108cf0 0x7efea010f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:56.072 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:56.065+0000 7efea587a640 1 --2- 172.21.15.39:0/3259069730 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7efea01083a0 0x7efea01087b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:56.072 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:56.065+0000 7efea587a640 1 --2- 172.21.15.39:0/3259069730 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efea0102a60 0x7efea0102e70 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:56.072 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:56.065+0000 7efea587a640 1 -- 172.21.15.39:0/3259069730 >> 172.21.15.39:0/3259069730 conn(0x7efea0079730 msgr2=0x7efea0079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:46:56.072 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:56.065+0000 7efea587a640 1 -- 172.21.15.39:0/3259069730 shutdown_connections 2024-09-21T03:46:56.073 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:56.065+0000 7efea587a640 1 -- 172.21.15.39:0/3259069730 wait complete. 2024-09-21T03:46:56.073 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:56.066+0000 7efea587a640 1 Processor -- start 2024-09-21T03:46:56.073 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:56.066+0000 7efea587a640 1 -- start start 2024-09-21T03:46:56.073 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:56.066+0000 7efea587a640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7efea0102a60 0x7efea019a440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:56.073 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:56.066+0000 7efea587a640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7efea01083a0 0x7efea019a9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:56.073 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:56.066+0000 7efea587a640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efea0108cf0 0x7efea019fc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:56.073 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:56.066+0000 7efea587a640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7efea0112960 con 0x7efea0108cf0 2024-09-21T03:46:56.073 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:56.066+0000 7efea587a640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7efea01127e0 con 0x7efea0102a60 2024-09-21T03:46:56.073 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:56.066+0000 7efea587a640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7efea0112ae0 con 0x7efea01083a0 2024-09-21T03:46:56.073 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:56.067+0000 7efe9f7fe640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efea0108cf0 0x7efea019fc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:46:56.073 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:56.067+0000 7efe9f7fe640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efea0108cf0 0x7efea019fc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:57430/0 (socket says 172.21.15.39:57430) 2024-09-21T03:46:56.073 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:56.067+0000 7efe9f7fe640 1 -- 172.21.15.39:0/2273433823 learned_addr learned my addr 172.21.15.39:0/2273433823 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:46:56.073 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:56.067+0000 7efe9e7fc640 1 --2- 172.21.15.39:0/2273433823 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7efea01083a0 0x7efea019a9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:46:56.073 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:56.067+0000 7efe9effd640 1 --2- 172.21.15.39:0/2273433823 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7efea0102a60 0x7efea019a440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:46:56.074 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:56.067+0000 7efe9f7fe640 1 -- 172.21.15.39:0/2273433823 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7efea01083a0 msgr2=0x7efea019a9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:46:56.074 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:56.067+0000 7efe9f7fe640 1 --2- 172.21.15.39:0/2273433823 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7efea01083a0 0x7efea019a9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:56.074 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:56.067+0000 7efe9f7fe640 1 -- 172.21.15.39:0/2273433823 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7efea0102a60 msgr2=0x7efea019a440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:46:56.074 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:56.067+0000 7efe9f7fe640 1 --2- 172.21.15.39:0/2273433823 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7efea0102a60 0x7efea019a440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:56.074 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:56.067+0000 7efe9f7fe640 1 -- 172.21.15.39:0/2273433823 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efea01a03b0 con 0x7efea0108cf0 2024-09-21T03:46:56.074 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:56.067+0000 7efe9f7fe640 1 --2- 172.21.15.39:0/2273433823 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efea0108cf0 0x7efea019fc80 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7efe9800ea20 tx=0x7efe9800eee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:46:56.074 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:56.068+0000 7efea4878640 1 -- 172.21.15.39:0/2273433823 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe98004d20 con 0x7efea0108cf0 2024-09-21T03:46:56.074 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:56.068+0000 7efea587a640 1 -- 172.21.15.39:0/2273433823 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efea01a06a0 con 0x7efea0108cf0 2024-09-21T03:46:56.074 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:56.068+0000 7efea4878640 1 -- 172.21.15.39:0/2273433823 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efe98017e40 con 0x7efea0108cf0 2024-09-21T03:46:56.074 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:56.068+0000 7efea587a640 1 -- 172.21.15.39:0/2273433823 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efea0104130 con 0x7efea0108cf0 2024-09-21T03:46:56.074 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:56.068+0000 7efea4878640 1 -- 172.21.15.39:0/2273433823 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe98005880 con 0x7efea0108cf0 2024-09-21T03:46:56.075 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:56.071+0000 7efeabdee740 1 Processor -- start 2024-09-21T03:46:56.075 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:56.072+0000 7efea4878640 1 -- 172.21.15.39:0/2273433823 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 25) ==== 154637+0+0 (secure 0 0 0) 0x7efe98004ec0 con 0x7efea0108cf0 2024-09-21T03:46:56.075 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:56.072+0000 7efeabdee740 1 -- start start 2024-09-21T03:46:56.075 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:56.072+0000 7efeabdee740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5567738b5070 0x556773c5c6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:56.076 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:56.072+0000 7efeabdee740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55677388c1b0 0x556773c5cbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:56.076 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:56.072+0000 7efeabdee740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556773894390 0x556773c5d120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:56.076 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:56.072+0000 7efeabdee740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x556773bd2ea0 con 0x55677388c1b0 2024-09-21T03:46:56.076 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:56.072+0000 7efeabdee740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x556773bc22d0 con 0x556773894390 2024-09-21T03:46:56.076 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:56.072+0000 7efeabdee740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x556773bc60e0 con 0x5567738b5070 2024-09-21T03:46:56.076 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:56.073+0000 7efe9effd640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55677388c1b0 0x556773c5cbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:46:56.076 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:56.073+0000 7efe9effd640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55677388c1b0 0x556773c5cbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:57446/0 (socket says 172.21.15.39:57446) 2024-09-21T03:46:56.077 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:56.073+0000 7efe9effd640 1 -- 172.21.15.39:0/632351913 learned_addr learned my addr 172.21.15.39:0/632351913 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:46:56.077 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:56.073+0000 7efe9e7fc640 1 --2- 172.21.15.39:0/632351913 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5567738b5070 0x556773c5c6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:46:56.077 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:56.073+0000 7efe9e7fc640 1 --2- 172.21.15.39:0/632351913 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556773894390 0x556773c5d120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:46:56.077 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:56.073+0000 7efea4878640 1 --2- 172.21.15.39:0/2273433823 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7efe880b6700 0x7efe880b8c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:56.077 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:56.073+0000 7efe9effd640 1 -- 172.21.15.39:0/632351913 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5567738b5070 msgr2=0x556773c5c6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:46:56.077 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:56.073+0000 7efe9effd640 1 --2- 172.21.15.39:0/632351913 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5567738b5070 0x556773c5c6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:56.077 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:56.073+0000 7efe9f7fe640 1 --2- 172.21.15.39:0/2273433823 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7efe880b6700 0x7efe880b8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:46:56.077 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:56.073+0000 7efe9effd640 1 -- 172.21.15.39:0/632351913 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556773894390 msgr2=0x556773c5d120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:46:56.077 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:56.073+0000 7efe9effd640 1 --2- 172.21.15.39:0/632351913 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556773894390 0x556773c5d120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:56.077 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:56.073+0000 7efe9effd640 1 -- 172.21.15.39:0/632351913 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556773920650 con 0x55677388c1b0 2024-09-21T03:46:56.078 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:56.074+0000 7efea4878640 1 -- 172.21.15.39:0/2273433823 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8484+0+0 (secure 0 0 0) 0x7efe980efba0 con 0x7efea0108cf0 2024-09-21T03:46:56.078 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:56.074+0000 7efe9effd640 1 --2- 172.21.15.39:0/632351913 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55677388c1b0 0x556773c5cbe0 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7efe90009900 tx=0x7efe90032cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e9ee06f06c5ed51c server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:46:56.078 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:56.074+0000 7efe9f7fe640 1 --2- 172.21.15.39:0/2273433823 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7efe880b6700 0x7efe880b8c10 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7efe980b2030 tx=0x7efe980e6450 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:46:56.078 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:56.074+0000 7efe85ffb640 1 -- 172.21.15.39:0/632351913 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe90040070 con 0x55677388c1b0 2024-09-21T03:46:56.078 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:56.074+0000 7efe85ffb640 1 -- 172.21.15.39:0/632351913 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efe90032e40 con 0x55677388c1b0 2024-09-21T03:46:56.079 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:56.074+0000 7efe85ffb640 1 -- 172.21.15.39:0/632351913 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe90035040 con 0x55677388c1b0 2024-09-21T03:46:56.079 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:56.075+0000 7efeabdee740 1 -- 172.21.15.39:0/632351913 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55677388c1b0 msgr2=0x556773c5cbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:46:56.079 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:56.075+0000 7efeabdee740 1 --2- 172.21.15.39:0/632351913 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55677388c1b0 0x556773c5cbe0 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7efe90009900 tx=0x7efe90032cd0 comp rx=0 tx=0).stop 2024-09-21T03:46:56.079 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:56.075+0000 7efeabdee740 1 -- 172.21.15.39:0/632351913 shutdown_connections 2024-09-21T03:46:56.079 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:56.075+0000 7efeabdee740 1 --2- 172.21.15.39:0/632351913 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556773894390 0x556773c5d120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:56.079 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:56.075+0000 7efeabdee740 1 --2- 172.21.15.39:0/632351913 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55677388c1b0 0x556773c5cbe0 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:56.079 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:56.075+0000 7efeabdee740 1 --2- 172.21.15.39:0/632351913 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5567738b5070 0x556773c5c6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:56.079 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:56.075+0000 7efeabdee740 1 -- 172.21.15.39:0/632351913 >> 172.21.15.39:0/632351913 conn(0x5567738a9460 msgr2=0x556773c5a590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:46:56.079 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:56.075+0000 7efeabdee740 1 -- 172.21.15.39:0/632351913 shutdown_connections 2024-09-21T03:46:56.079 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:56.076+0000 7efeabdee740 1 -- 172.21.15.39:0/632351913 wait complete. 2024-09-21T03:46:56.080 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:56.076+0000 7efeabdee740 1 -- start start 2024-09-21T03:46:56.080 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:56.076+0000 7efeabdee740 1 Processor -- start 2024-09-21T03:46:56.080 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:56.077+0000 7efeabdee740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55677388c1b0 0x556773c67e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:56.080 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:56.077+0000 7efeabdee740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556773894390 0x556773c68360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:56.080 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:56.077+0000 7efeabdee740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5567738b5070 0x556773c74110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:56.080 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:56.077+0000 7efeabdee740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x556773a23700 con 0x556773894390 2024-09-21T03:46:56.080 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:56.077+0000 7efeabdee740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x556773b5d160 con 0x5567738b5070 2024-09-21T03:46:56.081 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:56.077+0000 7efeabdee740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x556773c6ae20 con 0x55677388c1b0 2024-09-21T03:46:56.081 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:56.077+0000 7efeabdee740 10 client.0 _ll_register_callbacks cb 0x556773b769e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:46:56.081 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:56.077+0000 7efe9effd640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556773894390 0x556773c68360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:46:56.081 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:56.077+0000 7efe9e7fc640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55677388c1b0 0x556773c67e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:46:56.081 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:56.077+0000 7efe9effd640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556773894390 0x556773c68360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:57462/0 (socket says 172.21.15.39:57462) 2024-09-21T03:46:56.081 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:56.077+0000 7efe9effd640 1 -- 172.21.15.39:0/445399496 learned_addr learned my addr 172.21.15.39:0/445399496 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:46:56.082 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:56.078+0000 7efe9e7fc640 1 --2- 172.21.15.39:0/445399496 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5567738b5070 0x556773c74110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:46:56.082 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:56.078+0000 7efe9effd640 1 -- 172.21.15.39:0/445399496 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55677388c1b0 msgr2=0x556773c67e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:46:56.082 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:56.078+0000 7efe9effd640 1 --2- 172.21.15.39:0/445399496 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55677388c1b0 0x556773c67e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:56.082 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:56.078+0000 7efe9effd640 1 -- 172.21.15.39:0/445399496 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5567738b5070 msgr2=0x556773c74110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:46:56.082 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:56.078+0000 7efe9effd640 1 --2- 172.21.15.39:0/445399496 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5567738b5070 0x556773c74110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:56.082 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:56.078+0000 7efe9effd640 1 -- 172.21.15.39:0/445399496 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556773b8ac90 con 0x556773894390 2024-09-21T03:46:56.083 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:56.078+0000 7efe9e7fc640 1 --2- 172.21.15.39:0/445399496 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5567738b5070 0x556773c74110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:46:56.083 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:56.078+0000 7efe9effd640 1 --2- 172.21.15.39:0/445399496 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556773894390 0x556773c68360 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7efe90005610 tx=0x7efe900051c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b54610d840e26f37 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:46:56.083 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:56.078+0000 7efe7effd640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-09-21T03:46:56.083 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:56.078+0000 7efe7effd640 1 -- 172.21.15.39:0/445399496 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe90035040 con 0x556773894390 2024-09-21T03:46:56.083 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:56.078+0000 7efe7effd640 1 -- 172.21.15.39:0/445399496 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8484+0+0 (secure 0 0 0) 0x7efe90004030 con 0x556773894390 2024-09-21T03:46:56.083 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:56.078+0000 7efeabdee740 1 -- 172.21.15.39:0/445399496 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x556773bd4df0 con 0x556773894390 2024-09-21T03:46:56.083 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:56.078+0000 7efe7effd640 1 -- 172.21.15.39:0/445399496 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7efe90040040 con 0x556773894390 2024-09-21T03:46:56.083 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:56.079+0000 7efeabdee740 10 client.15669 fetch_fsmap learned FSMap version 29 2024-09-21T03:46:56.083 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:56.079+0000 7efeabdee740 1 -- 172.21.15.39:0/445399496 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x556773c76c00 con 0x556773894390 2024-09-21T03:46:56.083 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:56.079+0000 7efe7effd640 1 -- 172.21.15.39:0/445399496 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x556773bd4df0 con 0x556773894390 2024-09-21T03:46:56.083 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:56.079+0000 7efe7effd640 1 -- 172.21.15.39:0/445399496 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7efe90053020 con 0x556773894390 2024-09-21T03:46:56.083 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:56.079+0000 7efeabdee740 10 client.15669 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T03:46:56.083 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:56.079+0000 7efeabdee740 10 client.15669 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-21T03:46:56.083 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:56.079+0000 7efeabdee740 20 client.15669 populate_metadata read hostname 'smithi039' 2024-09-21T03:46:56.084 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:56.079+0000 7efeabdee740 1 --2- 172.21.15.39:0/445399496 >> [v2:172.21.15.161:6836/3451544597,v1:172.21.15.161:6837/3451544597] conn(0x5567738739f0 0x556773c79f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:56.084 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:56.079+0000 7efeabdee740 4 client.15669 mds_command: new command op to 24646 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:46:56.084 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:56.079+0000 7efeabdee740 1 -- 172.21.15.39:0/445399496 --> [v2:172.21.15.161:6836/3451544597,v1:172.21.15.161:6837/3451544597] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556773bd08a0 con 0x5567738739f0 2024-09-21T03:46:56.084 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:56.080+0000 7efe9e7fc640 1 --2- 172.21.15.39:0/445399496 >> [v2:172.21.15.161:6836/3451544597,v1:172.21.15.161:6837/3451544597] conn(0x5567738739f0 0x556773c79f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:46:56.085 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:56.080+0000 7efe9e7fc640 1 --2- 172.21.15.39:0/445399496 >> [v2:172.21.15.161:6836/3451544597,v1:172.21.15.161:6837/3451544597] conn(0x5567738739f0 0x556773c79f60 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=3989121ac5e3cb1f server_cookie=35b91ee9c255680c in_seq=0 out_seq=0 2024-09-21T03:46:56.085 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:56.080+0000 7efe7effd640 10 client.15669 ms_handle_connect on v2:172.21.15.161:6836/3451544597 2024-09-21T03:46:56.085 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:56.081+0000 7efe7effd640 1 -- 172.21.15.39:0/445399496 <== mds.3 v2:172.21.15.161:6836/3451544597 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x556773bd08a0 con 0x5567738739f0 2024-09-21T03:46:56.085 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:56.081+0000 7efe7effd640 10 client.15669 handle_command_reply: tid=0 2024-09-21T03:46:56.085 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:56.081+0000 7efeabdee740 1 client.15669 shutdown 2024-09-21T03:46:56.085 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:56.081+0000 7efeabdee740 1 -- 172.21.15.39:0/445399496 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556773894390 msgr2=0x556773c68360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:46:56.085 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:56.081+0000 7efeabdee740 1 --2- 172.21.15.39:0/445399496 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556773894390 0x556773c68360 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7efe90005610 tx=0x7efe900051c0 comp rx=0 tx=0).stop 2024-09-21T03:46:56.085 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:56.081+0000 7efeabdee740 1 -- 172.21.15.39:0/445399496 shutdown_connections 2024-09-21T03:46:56.085 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:56.081+0000 7efeabdee740 1 --2- 172.21.15.39:0/445399496 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5567738b5070 0x556773c74110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:56.086 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:56.081+0000 7efeabdee740 1 --2- 172.21.15.39:0/445399496 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556773894390 0x556773c68360 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:56.086 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:56.081+0000 7efeabdee740 1 --2- 172.21.15.39:0/445399496 >> [v2:172.21.15.161:6836/3451544597,v1:172.21.15.161:6837/3451544597] conn(0x5567738739f0 0x556773c79f60 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:56.086 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:56.081+0000 7efe7effd640 0 client.15669 ms_handle_reset on v2:172.21.15.161:6836/3451544597 2024-09-21T03:46:56.086 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:56.081+0000 7efeabdee740 1 --2- 172.21.15.39:0/445399496 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55677388c1b0 0x556773c67e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:56.086 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:56.082+0000 7efeabdee740 1 -- 172.21.15.39:0/445399496 >> 172.21.15.39:0/445399496 conn(0x5567738a9460 msgr2=0x556773c5f830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:46:56.086 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:56.082+0000 7efeabdee740 1 -- 172.21.15.39:0/445399496 shutdown_connections 2024-09-21T03:46:56.086 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:56.082+0000 7efeabdee740 1 -- 172.21.15.39:0/445399496 wait complete. 2024-09-21T03:46:56.086 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:56.082+0000 7efeabdee740 20 client.15669 trim_cache size 0 max 16384 2024-09-21T03:46:56.098 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:56.095+0000 7efeabdee740 1 Processor -- start 2024-09-21T03:46:56.098 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:56.095+0000 7efeabdee740 1 -- start start 2024-09-21T03:46:56.099 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:56.096+0000 7efeabdee740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5567738739f0 0x556773c682e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:56.099 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:56.096+0000 7efeabdee740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55677388c1b0 0x556773c74080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:56.099 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:56.096+0000 7efeabdee740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556773894390 0x556773c65010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:56.099 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:56.096+0000 7efeabdee740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7efea01ad4c0 con 0x55677388c1b0 2024-09-21T03:46:56.099 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:56.096+0000 7efeabdee740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7efea01af000 con 0x5567738739f0 2024-09-21T03:46:56.099 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:56.096+0000 7efeabdee740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7efea01adb90 con 0x556773894390 2024-09-21T03:46:56.100 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:56.096+0000 7efe9effd640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55677388c1b0 0x556773c74080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:46:56.100 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:56.096+0000 7efe9effd640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55677388c1b0 0x556773c74080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:57466/0 (socket says 172.21.15.39:57466) 2024-09-21T03:46:56.100 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:56.096+0000 7efe9effd640 1 -- 172.21.15.39:0/2414914498 learned_addr learned my addr 172.21.15.39:0/2414914498 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:46:56.100 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:56.096+0000 7efe9e7fc640 1 --2- 172.21.15.39:0/2414914498 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5567738739f0 0x556773c682e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:46:56.100 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:56.096+0000 7efe9e7fc640 1 --2- 172.21.15.39:0/2414914498 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556773894390 0x556773c65010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:46:56.100 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:56.096+0000 7efe9effd640 1 -- 172.21.15.39:0/2414914498 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556773894390 msgr2=0x556773c65010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:46:56.100 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:56.096+0000 7efe9effd640 1 --2- 172.21.15.39:0/2414914498 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556773894390 0x556773c65010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:56.100 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:56.097+0000 7efe9effd640 1 -- 172.21.15.39:0/2414914498 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5567738739f0 msgr2=0x556773c682e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:46:56.100 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:56.097+0000 7efe9effd640 1 --2- 172.21.15.39:0/2414914498 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5567738739f0 0x556773c682e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:56.100 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:56.097+0000 7efe9effd640 1 -- 172.21.15.39:0/2414914498 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556773c690e0 con 0x55677388c1b0 2024-09-21T03:46:56.101 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:56.097+0000 7efe9effd640 1 --2- 172.21.15.39:0/2414914498 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55677388c1b0 0x556773c74080 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7efe90033000 tx=0x7efe90000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d91df3191622ef9 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:46:56.101 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:56.097+0000 7efe857fa640 1 -- 172.21.15.39:0/2414914498 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe90035040 con 0x55677388c1b0 2024-09-21T03:46:56.101 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:56.097+0000 7efe857fa640 1 -- 172.21.15.39:0/2414914498 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efe9003a790 con 0x55677388c1b0 2024-09-21T03:46:56.102 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:56.097+0000 7efe857fa640 1 -- 172.21.15.39:0/2414914498 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe90030bb0 con 0x55677388c1b0 2024-09-21T03:46:56.102 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:56.098+0000 7efeabdee740 1 -- 172.21.15.39:0/2414914498 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55677388c1b0 msgr2=0x556773c74080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:46:56.102 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:56.098+0000 7efeabdee740 1 --2- 172.21.15.39:0/2414914498 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55677388c1b0 0x556773c74080 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7efe90033000 tx=0x7efe90000bc0 comp rx=0 tx=0).stop 2024-09-21T03:46:56.102 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:56.098+0000 7efeabdee740 1 -- 172.21.15.39:0/2414914498 shutdown_connections 2024-09-21T03:46:56.102 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:56.098+0000 7efeabdee740 1 --2- 172.21.15.39:0/2414914498 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556773894390 0x556773c65010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:56.102 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:56.098+0000 7efeabdee740 1 --2- 172.21.15.39:0/2414914498 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55677388c1b0 0x556773c74080 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:56.102 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:56.098+0000 7efeabdee740 1 --2- 172.21.15.39:0/2414914498 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5567738739f0 0x556773c682e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:56.102 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:56.098+0000 7efeabdee740 1 -- 172.21.15.39:0/2414914498 >> 172.21.15.39:0/2414914498 conn(0x5567738a9460 msgr2=0x556773c67bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:46:56.102 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:56.098+0000 7efeabdee740 1 -- 172.21.15.39:0/2414914498 shutdown_connections 2024-09-21T03:46:56.102 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:56.099+0000 7efeabdee740 1 -- 172.21.15.39:0/2414914498 wait complete. 2024-09-21T03:46:56.106 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:56.099+0000 7efeabdee740 1 -- start start 2024-09-21T03:46:56.106 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:56.099+0000 7efeabdee740 1 Processor -- start 2024-09-21T03:46:56.106 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:56.100+0000 7efeabdee740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5567738739f0 0x556773c7d7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:56.106 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:56.100+0000 7efeabdee740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55677388c1b0 0x556773c7aca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:56.106 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:56.100+0000 7efeabdee740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556773894390 0x556773c7b240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:56.106 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:56.100+0000 7efeabdee740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7efea01ae930 con 0x55677388c1b0 2024-09-21T03:46:56.106 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:56.100+0000 7efe9effd640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55677388c1b0 0x556773c7aca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:46:56.106 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:56.100+0000 7efe9effd640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55677388c1b0 0x556773c7aca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:57474/0 (socket says 172.21.15.39:57474) 2024-09-21T03:46:56.106 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:56.100+0000 7efeabdee740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7efea01ae260 con 0x556773894390 2024-09-21T03:46:56.106 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:56.100+0000 7efe9effd640 1 -- 172.21.15.39:0/901720042 learned_addr learned my addr 172.21.15.39:0/901720042 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:46:56.107 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:56.100+0000 7efeabdee740 1 -- 172.21.15.39:0/901720042 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7efea01ab4a0 con 0x5567738739f0 2024-09-21T03:46:56.107 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:56.100+0000 7efeabdee740 10 client.0 _ll_register_callbacks cb 0x7efe94011920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:46:56.107 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:56.101+0000 7efe9e7fc640 1 --2- 172.21.15.39:0/901720042 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5567738739f0 0x556773c7d7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:46:56.107 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:56.101+0000 7efe9e7fc640 1 --2- 172.21.15.39:0/901720042 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556773894390 0x556773c7b240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:46:56.107 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:56.101+0000 7efe9effd640 1 -- 172.21.15.39:0/901720042 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5567738739f0 msgr2=0x556773c7d7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:46:56.107 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:56.101+0000 7efe9effd640 1 --2- 172.21.15.39:0/901720042 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5567738739f0 0x556773c7d7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:56.107 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:56.101+0000 7efe9effd640 1 -- 172.21.15.39:0/901720042 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556773894390 msgr2=0x556773c7b240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:46:56.107 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:56.101+0000 7efe9effd640 1 --2- 172.21.15.39:0/901720042 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556773894390 0x556773c7b240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:56.107 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:56.101+0000 7efe9effd640 1 -- 172.21.15.39:0/901720042 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556773c7b9d0 con 0x55677388c1b0 2024-09-21T03:46:56.107 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:56.101+0000 7efe9effd640 1 --2- 172.21.15.39:0/901720042 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55677388c1b0 0x556773c7aca0 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7efe9003afb0 tx=0x7efe9002a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c24e9391603d32b6 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:46:56.107 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:56.101+0000 7efe7effd640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-09-21T03:46:56.107 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:56.101+0000 7efe7effd640 1 -- 172.21.15.39:0/901720042 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe90063060 con 0x55677388c1b0 2024-09-21T03:46:56.107 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:56.102+0000 7efe7effd640 1 -- 172.21.15.39:0/901720042 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8484+0+0 (secure 0 0 0) 0x7efe90030bb0 con 0x55677388c1b0 2024-09-21T03:46:56.107 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:56.102+0000 7efeabdee740 1 -- 172.21.15.39:0/901720042 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x556773bcdcd0 con 0x55677388c1b0 2024-09-21T03:46:56.108 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:56.102+0000 7efe7effd640 1 -- 172.21.15.39:0/901720042 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7efe90035070 con 0x55677388c1b0 2024-09-21T03:46:56.108 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:56.102+0000 7efeabdee740 10 client.15675 fetch_fsmap learned FSMap version 29 2024-09-21T03:46:56.108 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:56.102+0000 7efeabdee740 1 -- 172.21.15.39:0/901720042 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x556773ce4690 con 0x55677388c1b0 2024-09-21T03:46:56.108 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:56.102+0000 7efe7effd640 1 -- 172.21.15.39:0/901720042 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x556773bcdcd0 con 0x55677388c1b0 2024-09-21T03:46:56.108 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:56.102+0000 7efe7effd640 1 -- 172.21.15.39:0/901720042 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7efe9005d070 con 0x55677388c1b0 2024-09-21T03:46:56.108 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:56.103+0000 7efeabdee740 10 client.15675 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T03:46:56.108 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:56.103+0000 7efeabdee740 10 client.15675 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-21T03:46:56.108 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:56.103+0000 7efeabdee740 20 client.15675 populate_metadata read hostname 'smithi039' 2024-09-21T03:46:56.108 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:56.103+0000 7efeabdee740 1 --2- 172.21.15.39:0/901720042 >> [v2:172.21.15.161:6836/3451544597,v1:172.21.15.161:6837/3451544597] conn(0x5567738b5070 0x556773ce6dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:56.108 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:56.103+0000 7efeabdee740 4 client.15675 mds_command: new command op to 24646 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T03:46:56.108 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:56.103+0000 7efeabdee740 1 -- 172.21.15.39:0/901720042 --> [v2:172.21.15.161:6836/3451544597,v1:172.21.15.161:6837/3451544597] -- command(tid 0: {"prefix": "damage ls"}) -- 0x556773b43aa0 con 0x5567738b5070 2024-09-21T03:46:56.110 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:56.104+0000 7efe9e7fc640 1 --2- 172.21.15.39:0/901720042 >> [v2:172.21.15.161:6836/3451544597,v1:172.21.15.161:6837/3451544597] conn(0x5567738b5070 0x556773ce6dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:46:56.110 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:56.104+0000 7efe7effd640 10 client.15675 ms_handle_connect on v2:172.21.15.161:6836/3451544597 2024-09-21T03:46:56.110 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:56.104+0000 7efe9e7fc640 1 --2- 172.21.15.39:0/901720042 >> [v2:172.21.15.161:6836/3451544597,v1:172.21.15.161:6837/3451544597] conn(0x5567738b5070 0x556773ce6dc0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=c9c01b479f82a43 server_cookie=4d4607972c21cc82 in_seq=0 out_seq=0 2024-09-21T03:46:56.110 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:56.104+0000 7efe7effd640 1 -- 172.21.15.39:0/901720042 <== mds.3 v2:172.21.15.161:6836/3451544597 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x556773b43aa0 con 0x5567738b5070 2024-09-21T03:46:56.110 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:56.104+0000 7efe7effd640 10 client.15675 handle_command_reply: tid=0 2024-09-21T03:46:56.110 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:56.104+0000 7efeabdee740 1 client.15675 shutdown 2024-09-21T03:46:56.111 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:56.105+0000 7efeabdee740 1 -- 172.21.15.39:0/901720042 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55677388c1b0 msgr2=0x556773c7aca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:46:56.111 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:56.105+0000 7efeabdee740 1 --2- 172.21.15.39:0/901720042 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55677388c1b0 0x556773c7aca0 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7efe9003afb0 tx=0x7efe9002a410 comp rx=0 tx=0).stop 2024-09-21T03:46:56.111 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:56.105+0000 7efeabdee740 1 -- 172.21.15.39:0/901720042 shutdown_connections 2024-09-21T03:46:56.111 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:56.105+0000 7efeabdee740 1 --2- 172.21.15.39:0/901720042 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556773894390 0x556773c7b240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:56.111 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:56.105+0000 7efeabdee740 1 --2- 172.21.15.39:0/901720042 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55677388c1b0 0x556773c7aca0 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:56.111 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:56.105+0000 7efeabdee740 1 --2- 172.21.15.39:0/901720042 >> [v2:172.21.15.161:6836/3451544597,v1:172.21.15.161:6837/3451544597] conn(0x5567738b5070 0x556773ce6dc0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:56.111 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:56.105+0000 7efeabdee740 1 --2- 172.21.15.39:0/901720042 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5567738739f0 0x556773c7d7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:56.111 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:56.105+0000 7efeabdee740 1 -- 172.21.15.39:0/901720042 >> 172.21.15.39:0/901720042 conn(0x5567738a9460 msgr2=0x556773c5d6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:46:56.111 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:56.105+0000 7efe7effd640 0 client.15675 ms_handle_reset on v2:172.21.15.161:6836/3451544597 2024-09-21T03:46:56.111 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:56.105+0000 7efeabdee740 1 -- 172.21.15.39:0/901720042 shutdown_connections 2024-09-21T03:46:56.111 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:56.105+0000 7efeabdee740 1 -- 172.21.15.39:0/901720042 wait complete. 2024-09-21T03:46:56.111 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:56.105+0000 7efeabdee740 20 client.15675 trim_cache size 0 max 16384 2024-09-21T03:46:56.112 INFO:teuthology.orchestra.run.smithi039.stdout:[] 2024-09-21T03:46:56.112 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:56.106+0000 7efea587a640 1 -- 172.21.15.39:0/2273433823 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7efe880b6700 msgr2=0x7efe880b8c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:46:56.112 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:56.106+0000 7efea587a640 1 --2- 172.21.15.39:0/2273433823 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7efe880b6700 0x7efe880b8c10 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7efe980b2030 tx=0x7efe980e6450 comp rx=0 tx=0).stop 2024-09-21T03:46:56.112 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:56.106+0000 7efea587a640 1 -- 172.21.15.39:0/2273433823 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efea0108cf0 msgr2=0x7efea019fc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:46:56.112 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:56.106+0000 7efea587a640 1 --2- 172.21.15.39:0/2273433823 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efea0108cf0 0x7efea019fc80 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7efe9800ea20 tx=0x7efe9800eee0 comp rx=0 tx=0).stop 2024-09-21T03:46:56.112 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:56.106+0000 7efea587a640 1 -- 172.21.15.39:0/2273433823 shutdown_connections 2024-09-21T03:46:56.112 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:56.106+0000 7efea587a640 1 --2- 172.21.15.39:0/2273433823 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7efe880b6700 0x7efe880b8c10 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:56.112 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:56.106+0000 7efea587a640 1 --2- 172.21.15.39:0/2273433823 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efea0108cf0 0x7efea019fc80 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:56.112 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:56.106+0000 7efea587a640 1 --2- 172.21.15.39:0/2273433823 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7efea01083a0 0x7efea019a9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:56.113 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:56.106+0000 7efea587a640 1 --2- 172.21.15.39:0/2273433823 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7efea0102a60 0x7efea019a440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:56.113 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:56.106+0000 7efea587a640 1 -- 172.21.15.39:0/2273433823 >> 172.21.15.39:0/2273433823 conn(0x7efea0079730 msgr2=0x7efea0101a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:46:56.113 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:56.106+0000 7efea587a640 1 -- 172.21.15.39:0/2273433823 shutdown_connections 2024-09-21T03:46:56.113 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:56.106+0000 7efea587a640 1 -- 172.21.15.39:0/2273433823 wait complete. 2024-09-21T03:46:56.768 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:46:56 smithi039 ceph-mon[31211]: pgmap v352: 97 pgs: 97 active+clean; 1.4 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 682 B/s wr, 12 op/s 2024-09-21T03:46:56.810 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 51116d5c-77ca-11ef-baf5-efdc52797490 -- ceph tell mds.1:2 damage ls 2024-09-21T03:46:56.967 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:46:56 smithi161 ceph-mon[34533]: pgmap v352: 97 pgs: 97 active+clean; 1.4 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 682 B/s wr, 12 op/s 2024-09-21T03:46:56.974 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:46:56 smithi150 ceph-mon[34289]: pgmap v352: 97 pgs: 97 active+clean; 1.4 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 682 B/s wr, 12 op/s 2024-09-21T03:46:57.203 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/51116d5c-77ca-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:46:57.719 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:57.715+0000 7fe00d663640 1 -- 172.21.15.39:0/137110475 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe0081021f0 msgr2=0x7fe008102600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:46:57.719 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:57.715+0000 7fe00d663640 1 --2- 172.21.15.39:0/137110475 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe0081021f0 0x7fe008102600 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7fdff8009900 tx=0x7fdff802a410 comp rx=0 tx=0).stop 2024-09-21T03:46:57.719 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:57.716+0000 7fe00d663640 1 -- 172.21.15.39:0/137110475 shutdown_connections 2024-09-21T03:46:57.719 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:57.716+0000 7fe00d663640 1 --2- 172.21.15.39:0/137110475 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe008108480 0x7fe00810ecb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:57.719 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:57.716+0000 7fe00d663640 1 --2- 172.21.15.39:0/137110475 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe008107b30 0x7fe008107f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:57.719 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:57.716+0000 7fe00d663640 1 --2- 172.21.15.39:0/137110475 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe0081021f0 0x7fe008102600 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:57.720 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:57.716+0000 7fe00d663640 1 -- 172.21.15.39:0/137110475 >> 172.21.15.39:0/137110475 conn(0x7fe008077c60 msgr2=0x7fe0080780a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:46:57.720 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:57.716+0000 7fe00d663640 1 -- 172.21.15.39:0/137110475 shutdown_connections 2024-09-21T03:46:57.720 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:57.716+0000 7fe00d663640 1 -- 172.21.15.39:0/137110475 wait complete. 2024-09-21T03:46:57.721 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:57.717+0000 7fe00d663640 1 Processor -- start 2024-09-21T03:46:57.721 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:57.717+0000 7fe00d663640 1 -- start start 2024-09-21T03:46:57.721 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:57.718+0000 7fe00d663640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe0081021f0 0x7fe00819a4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:57.721 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:57.718+0000 7fe00d663640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe008107b30 0x7fe00819aa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:57.721 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:57.718+0000 7fe00d663640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe008108480 0x7fe00819fd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:57.721 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:57.718+0000 7fe00d663640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fe008113ab0 con 0x7fe0081021f0 2024-09-21T03:46:57.721 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:57.718+0000 7fe00d663640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fe008113930 con 0x7fe008107b30 2024-09-21T03:46:57.721 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:57.718+0000 7fe00d663640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fe008113c30 con 0x7fe008108480 2024-09-21T03:46:57.721 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:57.718+0000 7fe006ffd640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe0081021f0 0x7fe00819a4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:46:57.721 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:57.718+0000 7fe006ffd640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe0081021f0 0x7fe00819a4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:52066/0 (socket says 172.21.15.39:52066) 2024-09-21T03:46:57.722 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:57.718+0000 7fe006ffd640 1 -- 172.21.15.39:0/2276793384 learned_addr learned my addr 172.21.15.39:0/2276793384 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:46:57.722 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:57.718+0000 7fe0077fe640 1 --2- 172.21.15.39:0/2276793384 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe008108480 0x7fe00819fd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:46:57.722 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:57.718+0000 7fe0067fc640 1 --2- 172.21.15.39:0/2276793384 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe008107b30 0x7fe00819aa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:46:57.722 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:57.719+0000 7fe006ffd640 1 -- 172.21.15.39:0/2276793384 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe008108480 msgr2=0x7fe00819fd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:46:57.722 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:57.719+0000 7fe006ffd640 1 --2- 172.21.15.39:0/2276793384 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe008108480 0x7fe00819fd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:57.722 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:57.719+0000 7fe006ffd640 1 -- 172.21.15.39:0/2276793384 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe008107b30 msgr2=0x7fe00819aa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:46:57.722 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:57.719+0000 7fe006ffd640 1 --2- 172.21.15.39:0/2276793384 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe008107b30 0x7fe00819aa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:57.722 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:57.719+0000 7fe006ffd640 1 -- 172.21.15.39:0/2276793384 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe0081a0450 con 0x7fe0081021f0 2024-09-21T03:46:57.723 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:57.719+0000 7fe006ffd640 1 --2- 172.21.15.39:0/2276793384 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe0081021f0 0x7fe00819a4e0 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7fdff8005940 tx=0x7fdff80042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:46:57.723 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:57.719+0000 7fdfeffff640 1 -- 172.21.15.39:0/2276793384 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdff803f030 con 0x7fe0081021f0 2024-09-21T03:46:57.723 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:57.719+0000 7fdfeffff640 1 -- 172.21.15.39:0/2276793384 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdff8003cb0 con 0x7fe0081021f0 2024-09-21T03:46:57.723 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:57.719+0000 7fe00d663640 1 -- 172.21.15.39:0/2276793384 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe0081a06e0 con 0x7fe0081021f0 2024-09-21T03:46:57.723 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:57.719+0000 7fdfeffff640 1 -- 172.21.15.39:0/2276793384 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdff8035070 con 0x7fe0081021f0 2024-09-21T03:46:57.723 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:57.719+0000 7fe00d663640 1 -- 172.21.15.39:0/2276793384 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe0081a6ea0 con 0x7fe0081021f0 2024-09-21T03:46:57.725 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:57.722+0000 7fe013bd7740 1 Processor -- start 2024-09-21T03:46:57.725 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:57.723+0000 7fe013bd7740 1 -- start start 2024-09-21T03:46:57.726 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:57.723+0000 7fe013bd7740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556dbfd3a070 0x556dc00e16a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:57.726 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:57.723+0000 7fe013bd7740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556dbfd111b0 0x556dc00e1be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:57.726 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:57.723+0000 7fe013bd7740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556dbfd19390 0x556dc00e2120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:57.726 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:57.723+0000 7fe013bd7740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x556dc0057ea0 con 0x556dbfd3a070 2024-09-21T03:46:57.726 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:57.723+0000 7fe013bd7740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x556dc00472d0 con 0x556dbfd19390 2024-09-21T03:46:57.726 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:57.723+0000 7fe013bd7740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x556dc004b0e0 con 0x556dbfd111b0 2024-09-21T03:46:57.727 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:57.723+0000 7fe0077fe640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556dbfd3a070 0x556dc00e16a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:46:57.727 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:57.724+0000 7fe0067fc640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556dbfd19390 0x556dc00e2120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:46:57.727 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:57.724+0000 7fe0077fe640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556dbfd3a070 0x556dc00e16a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:52082/0 (socket says 172.21.15.39:52082) 2024-09-21T03:46:57.727 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:57.724+0000 7fe0077fe640 1 -- 172.21.15.39:0/2704817784 learned_addr learned my addr 172.21.15.39:0/2704817784 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:46:57.727 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:57.724+0000 7fdfeffff640 1 -- 172.21.15.39:0/2276793384 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 25) ==== 154637+0+0 (secure 0 0 0) 0x7fdff8042030 con 0x7fe0081021f0 2024-09-21T03:46:57.727 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:57.724+0000 7fe0077fe640 1 -- 172.21.15.39:0/2704817784 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556dbfd111b0 msgr2=0x556dc00e1be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:46:57.728 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:57.724+0000 7fe006ffd640 1 --2- 172.21.15.39:0/2704817784 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556dbfd111b0 0x556dc00e1be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:46:57.728 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:57.725+0000 7fe0077fe640 1 --2- 172.21.15.39:0/2704817784 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556dbfd111b0 0x556dc00e1be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:57.728 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:57.725+0000 7fe0077fe640 1 -- 172.21.15.39:0/2704817784 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556dbfd19390 msgr2=0x556dc00e2120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:46:57.728 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:57.725+0000 7fe0077fe640 1 --2- 172.21.15.39:0/2704817784 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556dbfd19390 0x556dc00e2120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:57.728 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:57.725+0000 7fe0077fe640 1 -- 172.21.15.39:0/2704817784 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556dbfda5650 con 0x556dbfd3a070 2024-09-21T03:46:57.728 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:57.725+0000 7fe0067fc640 1 --2- 172.21.15.39:0/2704817784 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556dbfd19390 0x556dc00e2120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:46:57.728 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:57.725+0000 7fe006ffd640 1 --2- 172.21.15.39:0/2704817784 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556dbfd111b0 0x556dc00e1be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:46:57.728 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:57.725+0000 7fe0077fe640 1 --2- 172.21.15.39:0/2704817784 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556dbfd3a070 0x556dc00e16a0 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7fe0000103f0 tx=0x7fe0000108b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e66ef433e30cafe3 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:46:57.729 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:57.726+0000 7fdfed7fa640 1 -- 172.21.15.39:0/2704817784 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe000016070 con 0x556dbfd3a070 2024-09-21T03:46:57.729 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:57.726+0000 7fdfed7fa640 1 -- 172.21.15.39:0/2704817784 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe000004d20 con 0x556dbfd3a070 2024-09-21T03:46:57.729 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:57.726+0000 7fdfed7fa640 1 -- 172.21.15.39:0/2704817784 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe000005350 con 0x556dbfd3a070 2024-09-21T03:46:57.730 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:57.726+0000 7fdfeffff640 1 --2- 172.21.15.39:0/2276793384 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7fdff00b6850 0x7fdff00b8d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:57.730 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:57.726+0000 7fe006ffd640 1 --2- 172.21.15.39:0/2276793384 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7fdff00b6850 0x7fdff00b8d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:46:57.730 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:57.726+0000 7fdfeffff640 1 -- 172.21.15.39:0/2276793384 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8484+0+0 (secure 0 0 0) 0x7fdff8107ba0 con 0x7fe0081021f0 2024-09-21T03:46:57.730 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:57.727+0000 7fe013bd7740 1 -- 172.21.15.39:0/2704817784 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556dbfd3a070 msgr2=0x556dc00e16a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:46:57.730 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:57.727+0000 7fe006ffd640 1 --2- 172.21.15.39:0/2276793384 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7fdff00b6850 0x7fdff00b8d60 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x556dc00e0320 tx=0x7fdff8097480 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:46:57.730 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:57.727+0000 7fe013bd7740 1 --2- 172.21.15.39:0/2704817784 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556dbfd3a070 0x556dc00e16a0 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7fe0000103f0 tx=0x7fe0000108b0 comp rx=0 tx=0).stop 2024-09-21T03:46:57.731 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:57.727+0000 7fe013bd7740 1 -- 172.21.15.39:0/2704817784 shutdown_connections 2024-09-21T03:46:57.731 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:57.727+0000 7fe013bd7740 1 --2- 172.21.15.39:0/2704817784 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556dbfd19390 0x556dc00e2120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:57.731 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:57.727+0000 7fe013bd7740 1 --2- 172.21.15.39:0/2704817784 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556dbfd111b0 0x556dc00e1be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:57.731 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:57.727+0000 7fe013bd7740 1 --2- 172.21.15.39:0/2704817784 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556dbfd3a070 0x556dc00e16a0 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:57.731 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:57.727+0000 7fe013bd7740 1 -- 172.21.15.39:0/2704817784 >> 172.21.15.39:0/2704817784 conn(0x556dbfd2e460 msgr2=0x556dc00df590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:46:57.731 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:57.727+0000 7fe013bd7740 1 -- 172.21.15.39:0/2704817784 shutdown_connections 2024-09-21T03:46:57.731 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:57.727+0000 7fe013bd7740 1 -- 172.21.15.39:0/2704817784 wait complete. 2024-09-21T03:46:57.731 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:57.728+0000 7fe013bd7740 1 -- start start 2024-09-21T03:46:57.732 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:57.728+0000 7fe013bd7740 1 Processor -- start 2024-09-21T03:46:57.732 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:57.728+0000 7fe013bd7740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556dbfd111b0 0x556dc00f1240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:57.732 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:57.728+0000 7fe013bd7740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556dbfd19390 0x556dc00f1780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:57.732 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:57.728+0000 7fe013bd7740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556dbfd3a070 0x556dc00fd4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:57.733 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:57.729+0000 7fe0077fe640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556dbfd3a070 0x556dc00fd4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:46:57.734 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:57.729+0000 7fe013bd7740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x556dbfea8700 con 0x556dbfd3a070 2024-09-21T03:46:57.734 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:57.729+0000 7fe013bd7740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x556dbffe2160 con 0x556dbfd19390 2024-09-21T03:46:57.734 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:57.729+0000 7fe013bd7740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x556dc00f41b0 con 0x556dbfd111b0 2024-09-21T03:46:57.734 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:57.729+0000 7fe0077fe640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556dbfd3a070 0x556dc00fd4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:52084/0 (socket says 172.21.15.39:52084) 2024-09-21T03:46:57.734 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:57.729+0000 7fe0077fe640 1 -- 172.21.15.39:0/1769732600 learned_addr learned my addr 172.21.15.39:0/1769732600 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:46:57.734 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:57.729+0000 7fe013bd7740 10 client.0 _ll_register_callbacks cb 0x556dbfffb9e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:46:57.734 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:57.729+0000 7fe0077fe640 1 --2- 172.21.15.39:0/1769732600 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556dbfd111b0 0x556dc00f1240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:46:57.734 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:57.729+0000 7fe0077fe640 1 -- 172.21.15.39:0/1769732600 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556dbfd111b0 msgr2=0x556dc00f1240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:46:57.734 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:57.729+0000 7fe0077fe640 1 --2- 172.21.15.39:0/1769732600 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556dbfd111b0 0x556dc00f1240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:57.734 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:57.729+0000 7fe0077fe640 1 -- 172.21.15.39:0/1769732600 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556dbfd19390 msgr2=0x556dc00f1780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:46:57.734 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:57.729+0000 7fe0077fe640 1 --2- 172.21.15.39:0/1769732600 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556dbfd19390 0x556dc00f1780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:57.734 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:57.729+0000 7fe0077fe640 1 -- 172.21.15.39:0/1769732600 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556dc000fc90 con 0x556dbfd3a070 2024-09-21T03:46:57.735 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:57.729+0000 7fe0077fe640 1 --2- 172.21.15.39:0/1769732600 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556dbfd3a070 0x556dc00fd4e0 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x556dc00da310 tx=0x7fe000012ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea71f2fe144bfa0a server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:46:57.735 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:57.729+0000 7fdfeaff5640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-09-21T03:46:57.735 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:57.730+0000 7fdfeaff5640 1 -- 172.21.15.39:0/1769732600 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe000016070 con 0x556dbfd3a070 2024-09-21T03:46:57.735 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:57.730+0000 7fdfeaff5640 1 -- 172.21.15.39:0/1769732600 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8484+0+0 (secure 0 0 0) 0x7fe000022040 con 0x556dbfd3a070 2024-09-21T03:46:57.735 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:57.730+0000 7fe013bd7740 1 -- 172.21.15.39:0/1769732600 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe000004d20 con 0x556dbfd3a070 2024-09-21T03:46:57.735 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:57.730+0000 7fdfeaff5640 1 -- 172.21.15.39:0/1769732600 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fe000005630 con 0x556dbfd3a070 2024-09-21T03:46:57.735 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:57.730+0000 7fe013bd7740 10 client.15687 fetch_fsmap learned FSMap version 29 2024-09-21T03:46:57.735 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:57.730+0000 7fe013bd7740 1 -- 172.21.15.39:0/1769732600 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x556dc00fff20 con 0x556dbfd3a070 2024-09-21T03:46:57.735 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:57.730+0000 7fdfeaff5640 1 -- 172.21.15.39:0/1769732600 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe0000057d0 con 0x556dbfd3a070 2024-09-21T03:46:57.735 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:57.730+0000 7fdfeaff5640 1 -- 172.21.15.39:0/1769732600 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fe000025070 con 0x556dbfd3a070 2024-09-21T03:46:57.735 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:57.731+0000 7fe013bd7740 10 client.15687 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T03:46:57.735 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:57.731+0000 7fe013bd7740 10 client.15687 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-21T03:46:57.735 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:57.731+0000 7fe013bd7740 20 client.15687 populate_metadata read hostname 'smithi039' 2024-09-21T03:46:57.735 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:57.731+0000 7fe013bd7740 1 --2- 172.21.15.39:0/1769732600 >> [v2:172.21.15.150:6836/4017043144,v1:172.21.15.150:6837/4017043144] conn(0x556dbfcf89f0 0x556dc0103280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:57.737 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:57.731+0000 7fe013bd7740 4 client.15687 mds_command: new command op to 24659 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:46:57.738 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:57.731+0000 7fe013bd7740 1 -- 172.21.15.39:0/1769732600 --> [v2:172.21.15.150:6836/4017043144,v1:172.21.15.150:6837/4017043144] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fe000004d20 con 0x556dbfcf89f0 2024-09-21T03:46:57.738 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:57.732+0000 7fe0067fc640 1 --2- 172.21.15.39:0/1769732600 >> [v2:172.21.15.150:6836/4017043144,v1:172.21.15.150:6837/4017043144] conn(0x556dbfcf89f0 0x556dc0103280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:46:57.738 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:57.732+0000 7fdfeaff5640 10 client.15687 ms_handle_connect on v2:172.21.15.150:6836/4017043144 2024-09-21T03:46:57.738 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:57.732+0000 7fe0067fc640 1 --2- 172.21.15.39:0/1769732600 >> [v2:172.21.15.150:6836/4017043144,v1:172.21.15.150:6837/4017043144] conn(0x556dbfcf89f0 0x556dc0103280 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=b2c1937b3a6f98c0 server_cookie=150d2c7953915ede in_seq=0 out_seq=0 2024-09-21T03:46:57.738 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:57.733+0000 7fdfeaff5640 1 -- 172.21.15.39:0/1769732600 <== mds.2 v2:172.21.15.150:6836/4017043144 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x7fe000004d20 con 0x556dbfcf89f0 2024-09-21T03:46:57.738 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:57.733+0000 7fdfeaff5640 10 client.15687 handle_command_reply: tid=0 2024-09-21T03:46:57.738 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:57.733+0000 7fe013bd7740 1 client.15687 shutdown 2024-09-21T03:46:57.738 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:57.733+0000 7fe013bd7740 1 -- 172.21.15.39:0/1769732600 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556dbfd3a070 msgr2=0x556dc00fd4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:46:57.738 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:57.733+0000 7fe013bd7740 1 --2- 172.21.15.39:0/1769732600 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556dbfd3a070 0x556dc00fd4e0 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x556dc00da310 tx=0x7fe000012ef0 comp rx=0 tx=0).stop 2024-09-21T03:46:57.738 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:57.733+0000 7fe013bd7740 1 -- 172.21.15.39:0/1769732600 shutdown_connections 2024-09-21T03:46:57.738 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:57.733+0000 7fe013bd7740 1 --2- 172.21.15.39:0/1769732600 >> [v2:172.21.15.150:6836/4017043144,v1:172.21.15.150:6837/4017043144] conn(0x556dbfcf89f0 0x556dc0103280 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:57.738 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:57.733+0000 7fe013bd7740 1 --2- 172.21.15.39:0/1769732600 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556dbfd3a070 0x556dc00fd4e0 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:57.739 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:57.733+0000 7fdfeaff5640 0 client.15687 ms_handle_reset on v2:172.21.15.150:6836/4017043144 2024-09-21T03:46:57.739 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:57.733+0000 7fe013bd7740 1 --2- 172.21.15.39:0/1769732600 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556dbfd19390 0x556dc00f1780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:57.739 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:57.734+0000 7fe013bd7740 1 --2- 172.21.15.39:0/1769732600 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556dbfd111b0 0x556dc00f1240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:57.739 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:57.734+0000 7fe013bd7740 1 -- 172.21.15.39:0/1769732600 >> 172.21.15.39:0/1769732600 conn(0x556dbfd2e460 msgr2=0x556dc00ddbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:46:57.739 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:57.734+0000 7fe013bd7740 1 -- 172.21.15.39:0/1769732600 shutdown_connections 2024-09-21T03:46:57.739 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:57.734+0000 7fe013bd7740 1 -- 172.21.15.39:0/1769732600 wait complete. 2024-09-21T03:46:57.739 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:57.734+0000 7fe013bd7740 20 client.15687 trim_cache size 0 max 16384 2024-09-21T03:46:57.750 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:57.747+0000 7fe013bd7740 1 Processor -- start 2024-09-21T03:46:57.750 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:57.747+0000 7fe013bd7740 1 -- start start 2024-09-21T03:46:57.750 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:57.748+0000 7fe013bd7740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556dbfcf89f0 0x556dc00e6e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:57.751 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:57.748+0000 7fe013bd7740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556dbfd111b0 0x556dc00ee430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:57.751 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:57.748+0000 7fe013bd7740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556dbfd19390 0x556dc00ee970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:57.751 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:57.748+0000 7fe013bd7740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fe0081ad600 con 0x556dbfd19390 2024-09-21T03:46:57.751 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:57.748+0000 7fe013bd7740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fe0081af140 con 0x556dbfd111b0 2024-09-21T03:46:57.751 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:57.748+0000 7fe013bd7740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fe0081adcd0 con 0x556dbfcf89f0 2024-09-21T03:46:57.752 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:57.748+0000 7fe0077fe640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556dbfd19390 0x556dc00ee970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:46:57.752 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:57.748+0000 7fe0077fe640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556dbfd19390 0x556dc00ee970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:52090/0 (socket says 172.21.15.39:52090) 2024-09-21T03:46:57.752 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:57.748+0000 7fe0077fe640 1 -- 172.21.15.39:0/1859722214 learned_addr learned my addr 172.21.15.39:0/1859722214 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:46:57.752 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:57.748+0000 7fe0067fc640 1 --2- 172.21.15.39:0/1859722214 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556dbfd111b0 0x556dc00ee430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:46:57.752 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:57.748+0000 7fe0077fe640 1 --2- 172.21.15.39:0/1859722214 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556dbfcf89f0 0x556dc00e6e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:46:57.752 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:57.748+0000 7fe0077fe640 1 -- 172.21.15.39:0/1859722214 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556dbfcf89f0 msgr2=0x556dc00e6e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:46:57.752 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:57.748+0000 7fe0077fe640 1 --2- 172.21.15.39:0/1859722214 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556dbfcf89f0 0x556dc00e6e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:57.752 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:57.748+0000 7fe0077fe640 1 -- 172.21.15.39:0/1859722214 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556dbfd111b0 msgr2=0x556dc00ee430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:46:57.752 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:57.748+0000 7fe0077fe640 1 --2- 172.21.15.39:0/1859722214 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556dbfd111b0 0x556dc00ee430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:57.752 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:57.748+0000 7fe0077fe640 1 -- 172.21.15.39:0/1859722214 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556dc00fdc90 con 0x556dbfd19390 2024-09-21T03:46:57.752 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:57.748+0000 7fe0067fc640 1 --2- 172.21.15.39:0/1859722214 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556dbfd111b0 0x556dc00ee430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:46:57.753 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:57.749+0000 7fe0077fe640 1 --2- 172.21.15.39:0/1859722214 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556dbfd19390 0x556dc00ee970 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7fe0000098b0 tx=0x7fe0000108b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4004b893ebd21d66 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:46:57.753 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:57.749+0000 7fdfecff9640 1 -- 172.21.15.39:0/1859722214 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe000036020 con 0x556dbfd19390 2024-09-21T03:46:57.753 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:57.749+0000 7fdfecff9640 1 -- 172.21.15.39:0/1859722214 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe000037040 con 0x556dbfd19390 2024-09-21T03:46:57.753 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:57.749+0000 7fdfecff9640 1 -- 172.21.15.39:0/1859722214 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe000016040 con 0x556dbfd19390 2024-09-21T03:46:57.754 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:57.750+0000 7fe013bd7740 1 -- 172.21.15.39:0/1859722214 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556dbfd19390 msgr2=0x556dc00ee970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:46:57.754 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:57.750+0000 7fe013bd7740 1 --2- 172.21.15.39:0/1859722214 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556dbfd19390 0x556dc00ee970 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7fe0000098b0 tx=0x7fe0000108b0 comp rx=0 tx=0).stop 2024-09-21T03:46:57.754 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:57.750+0000 7fe013bd7740 1 -- 172.21.15.39:0/1859722214 shutdown_connections 2024-09-21T03:46:57.754 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:57.750+0000 7fe013bd7740 1 --2- 172.21.15.39:0/1859722214 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556dbfd19390 0x556dc00ee970 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:57.754 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:57.750+0000 7fe013bd7740 1 --2- 172.21.15.39:0/1859722214 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556dbfd111b0 0x556dc00ee430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:57.754 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:57.750+0000 7fe013bd7740 1 --2- 172.21.15.39:0/1859722214 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556dbfcf89f0 0x556dc00e6e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:57.754 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:57.750+0000 7fe013bd7740 1 -- 172.21.15.39:0/1859722214 >> 172.21.15.39:0/1859722214 conn(0x556dbfd2e460 msgr2=0x556dc00e6760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:46:57.754 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:57.750+0000 7fe013bd7740 1 -- 172.21.15.39:0/1859722214 shutdown_connections 2024-09-21T03:46:57.755 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:57.750+0000 7fe013bd7740 1 -- 172.21.15.39:0/1859722214 wait complete. 2024-09-21T03:46:57.755 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:57.750+0000 7fe013bd7740 1 -- start start 2024-09-21T03:46:57.755 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:57.751+0000 7fe013bd7740 1 Processor -- start 2024-09-21T03:46:57.755 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:57.751+0000 7fe013bd7740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556dbfcf89f0 0x556dc00e8740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:57.755 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:57.751+0000 7fe013bd7740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556dbfd111b0 0x556dc00e8d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:57.755 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:57.751+0000 7fe013bd7740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556dbfd19390 0x556dc00f93e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:57.755 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:57.751+0000 7fe013bd7740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fe0081aea70 con 0x556dbfd19390 2024-09-21T03:46:57.755 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:57.751+0000 7fe013bd7740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fe0081ae3a0 con 0x556dbfcf89f0 2024-09-21T03:46:57.755 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:57.751+0000 7fe013bd7740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fe0081ab5e0 con 0x556dbfd111b0 2024-09-21T03:46:57.756 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:57.751+0000 7fe013bd7740 10 client.0 _ll_register_callbacks cb 0x7fdffc00b3a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:46:57.756 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:57.751+0000 7fe0077fe640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556dbfd19390 0x556dc00f93e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:46:57.756 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:57.751+0000 7fe0077fe640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556dbfd19390 0x556dc00f93e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:52106/0 (socket says 172.21.15.39:52106) 2024-09-21T03:46:57.756 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:57.751+0000 7fe0077fe640 1 -- 172.21.15.39:0/1985786565 learned_addr learned my addr 172.21.15.39:0/1985786565 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:46:57.756 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:57.751+0000 7fe0077fe640 1 --2- 172.21.15.39:0/1985786565 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556dbfcf89f0 0x556dc00e8740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:46:57.756 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:57.751+0000 7fe0067fc640 1 --2- 172.21.15.39:0/1985786565 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556dbfd111b0 0x556dc00e8d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:46:57.757 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:57.752+0000 7fe0077fe640 1 -- 172.21.15.39:0/1985786565 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556dbfd111b0 msgr2=0x556dc00e8d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:46:57.757 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:57.752+0000 7fe0077fe640 1 --2- 172.21.15.39:0/1985786565 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556dbfd111b0 0x556dc00e8d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:57.757 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:57.752+0000 7fe0077fe640 1 -- 172.21.15.39:0/1985786565 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556dbfcf89f0 msgr2=0x556dc00e8740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:46:57.757 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:57.752+0000 7fe0077fe640 1 --2- 172.21.15.39:0/1985786565 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556dbfcf89f0 0x556dc00e8740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:57.757 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:57.752+0000 7fe0077fe640 1 -- 172.21.15.39:0/1985786565 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556dc00f9a50 con 0x556dbfd19390 2024-09-21T03:46:57.757 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:57.752+0000 7fe0077fe640 1 --2- 172.21.15.39:0/1985786565 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556dbfd19390 0x556dc00f93e0 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7fe000009770 tx=0x7fe000005040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f9850e3736c054a0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:46:57.757 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:57.752+0000 7fdfeaff5640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-09-21T03:46:57.757 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:57.752+0000 7fdfeaff5640 1 -- 172.21.15.39:0/1985786565 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe000036070 con 0x556dbfd19390 2024-09-21T03:46:57.757 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:57.752+0000 7fdfeaff5640 1 -- 172.21.15.39:0/1985786565 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8484+0+0 (secure 0 0 0) 0x7fe00001f040 con 0x556dbfd19390 2024-09-21T03:46:57.757 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:57.752+0000 7fe013bd7740 1 -- 172.21.15.39:0/1985786565 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe000037040 con 0x556dbfd19390 2024-09-21T03:46:57.757 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:57.752+0000 7fdfeaff5640 1 -- 172.21.15.39:0/1985786565 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fe00004f050 con 0x556dbfd19390 2024-09-21T03:46:57.757 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:57.752+0000 7fe013bd7740 10 client.15693 fetch_fsmap learned FSMap version 29 2024-09-21T03:46:57.757 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:57.752+0000 7fe013bd7740 1 -- 172.21.15.39:0/1985786565 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x556dc00f6d00 con 0x556dbfd19390 2024-09-21T03:46:57.757 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:57.753+0000 7fdfeaff5640 1 -- 172.21.15.39:0/1985786565 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe000037040 con 0x556dbfd19390 2024-09-21T03:46:57.758 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:57.753+0000 7fdfeaff5640 1 -- 172.21.15.39:0/1985786565 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fe00003c020 con 0x556dbfd19390 2024-09-21T03:46:57.758 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:57.753+0000 7fe013bd7740 10 client.15693 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T03:46:57.758 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:57.753+0000 7fe013bd7740 10 client.15693 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-21T03:46:57.758 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:57.753+0000 7fe013bd7740 20 client.15693 populate_metadata read hostname 'smithi039' 2024-09-21T03:46:57.758 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:57.753+0000 7fe013bd7740 1 --2- 172.21.15.39:0/1985786565 >> [v2:172.21.15.150:6836/4017043144,v1:172.21.15.150:6837/4017043144] conn(0x556dbfd3a070 0x556dc0174390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:57.758 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:57.753+0000 7fe013bd7740 4 client.15693 mds_command: new command op to 24659 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T03:46:57.758 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:57.753+0000 7fe013bd7740 1 -- 172.21.15.39:0/1985786565 --> [v2:172.21.15.150:6836/4017043144,v1:172.21.15.150:6837/4017043144] -- command(tid 0: {"prefix": "damage ls"}) -- 0x556dc0059df0 con 0x556dbfd3a070 2024-09-21T03:46:57.758 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:57.753+0000 7fe0067fc640 1 --2- 172.21.15.39:0/1985786565 >> [v2:172.21.15.150:6836/4017043144,v1:172.21.15.150:6837/4017043144] conn(0x556dbfd3a070 0x556dc0174390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:46:57.758 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:57.754+0000 7fdfeaff5640 10 client.15693 ms_handle_connect on v2:172.21.15.150:6836/4017043144 2024-09-21T03:46:57.759 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:57.754+0000 7fe0067fc640 1 --2- 172.21.15.39:0/1985786565 >> [v2:172.21.15.150:6836/4017043144,v1:172.21.15.150:6837/4017043144] conn(0x556dbfd3a070 0x556dc0174390 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=d3176965ffb71280 server_cookie=8d2ac2042fa67d80 in_seq=0 out_seq=0 2024-09-21T03:46:57.759 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:57.754+0000 7fdfeaff5640 1 -- 172.21.15.39:0/1985786565 <== mds.2 v2:172.21.15.150:6836/4017043144 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x556dc0059df0 con 0x556dbfd3a070 2024-09-21T03:46:57.759 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:57.754+0000 7fdfeaff5640 10 client.15693 handle_command_reply: tid=0 2024-09-21T03:46:57.759 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:57.754+0000 7fe013bd7740 1 client.15693 shutdown 2024-09-21T03:46:57.760 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:57.754+0000 7fe013bd7740 1 -- 172.21.15.39:0/1985786565 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556dbfd19390 msgr2=0x556dc00f93e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:46:57.760 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:57.754+0000 7fe013bd7740 1 --2- 172.21.15.39:0/1985786565 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556dbfd19390 0x556dc00f93e0 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7fe000009770 tx=0x7fe000005040 comp rx=0 tx=0).stop 2024-09-21T03:46:57.760 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:57.755+0000 7fe013bd7740 1 -- 172.21.15.39:0/1985786565 shutdown_connections 2024-09-21T03:46:57.760 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:57.755+0000 7fe013bd7740 1 --2- 172.21.15.39:0/1985786565 >> [v2:172.21.15.150:6836/4017043144,v1:172.21.15.150:6837/4017043144] conn(0x556dbfd3a070 0x556dc0174390 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:57.760 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:57.755+0000 7fe013bd7740 1 --2- 172.21.15.39:0/1985786565 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556dbfd19390 0x556dc00f93e0 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:57.760 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:57.755+0000 7fe013bd7740 1 --2- 172.21.15.39:0/1985786565 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556dbfd111b0 0x556dc00e8d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:57.760 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:57.755+0000 7fe013bd7740 1 --2- 172.21.15.39:0/1985786565 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556dbfcf89f0 0x556dc00e8740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:57.760 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:57.755+0000 7fe013bd7740 1 -- 172.21.15.39:0/1985786565 >> 172.21.15.39:0/1985786565 conn(0x556dbfd2e460 msgr2=0x556dc00e2670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:46:57.760 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:57.755+0000 7fdfeaff5640 0 client.15693 ms_handle_reset on v2:172.21.15.150:6836/4017043144 2024-09-21T03:46:57.760 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:57.755+0000 7fe013bd7740 1 -- 172.21.15.39:0/1985786565 shutdown_connections 2024-09-21T03:46:57.760 INFO:teuthology.orchestra.run.smithi039.stdout:[] 2024-09-21T03:46:57.761 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:57.755+0000 7fe013bd7740 1 -- 172.21.15.39:0/1985786565 wait complete. 2024-09-21T03:46:57.761 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:57.755+0000 7fe013bd7740 20 client.15693 trim_cache size 0 max 16384 2024-09-21T03:46:57.761 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:57.755+0000 7fe00d663640 1 -- 172.21.15.39:0/2276793384 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7fdff00b6850 msgr2=0x7fdff00b8d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:46:57.761 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:57.755+0000 7fe00d663640 1 --2- 172.21.15.39:0/2276793384 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7fdff00b6850 0x7fdff00b8d60 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x556dc00e0320 tx=0x7fdff8097480 comp rx=0 tx=0).stop 2024-09-21T03:46:57.762 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:57.755+0000 7fe00d663640 1 -- 172.21.15.39:0/2276793384 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe0081021f0 msgr2=0x7fe00819a4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:46:57.762 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:57.755+0000 7fe00d663640 1 --2- 172.21.15.39:0/2276793384 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe0081021f0 0x7fe00819a4e0 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7fdff8005940 tx=0x7fdff80042f0 comp rx=0 tx=0).stop 2024-09-21T03:46:57.762 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:57.756+0000 7fe00d663640 1 -- 172.21.15.39:0/2276793384 shutdown_connections 2024-09-21T03:46:57.762 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:57.756+0000 7fe00d663640 1 --2- 172.21.15.39:0/2276793384 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7fdff00b6850 0x7fdff00b8d60 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:57.762 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:57.756+0000 7fe00d663640 1 --2- 172.21.15.39:0/2276793384 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe008108480 0x7fe00819fd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:57.762 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:57.756+0000 7fe00d663640 1 --2- 172.21.15.39:0/2276793384 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe008107b30 0x7fe00819aa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:57.762 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:57.756+0000 7fe00d663640 1 --2- 172.21.15.39:0/2276793384 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe0081021f0 0x7fe00819a4e0 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:57.762 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:57.756+0000 7fe00d663640 1 -- 172.21.15.39:0/2276793384 >> 172.21.15.39:0/2276793384 conn(0x7fe008077c60 msgr2=0x7fe008105c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:46:57.762 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:57.756+0000 7fe00d663640 1 -- 172.21.15.39:0/2276793384 shutdown_connections 2024-09-21T03:46:57.762 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:57.756+0000 7fe00d663640 1 -- 172.21.15.39:0/2276793384 wait complete. 2024-09-21T03:46:58.486 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 51116d5c-77ca-11ef-baf5-efdc52797490 -- ceph tell mds.1:1 damage ls 2024-09-21T03:46:58.855 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/51116d5c-77ca-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:46:58.967 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:46:58 smithi161 ceph-mon[34533]: pgmap v353: 97 pgs: 97 active+clean; 1.4 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 682 B/s wr, 9 op/s 2024-09-21T03:46:58.974 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:46:58 smithi150 ceph-mon[34289]: pgmap v353: 97 pgs: 97 active+clean; 1.4 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 682 B/s wr, 9 op/s 2024-09-21T03:46:58.981 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:46:58 smithi039 ceph-mon[31211]: pgmap v353: 97 pgs: 97 active+clean; 1.4 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 682 B/s wr, 9 op/s 2024-09-21T03:46:59.421 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:59.418+0000 7fd760d1d640 1 -- 172.21.15.39:0/46731637 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd75c1049b0 msgr2=0x7fd75c104dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:46:59.421 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:59.418+0000 7fd760d1d640 1 --2- 172.21.15.39:0/46731637 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd75c1049b0 0x7fd75c104dc0 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7fd74c009900 tx=0x7fd74c02a410 comp rx=0 tx=0).stop 2024-09-21T03:46:59.421 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:59.418+0000 7fd760d1d640 1 -- 172.21.15.39:0/46731637 shutdown_connections 2024-09-21T03:46:59.421 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:59.418+0000 7fd760d1d640 1 --2- 172.21.15.39:0/46731637 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd75c10de10 0x7fd75c110300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:59.421 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:59.418+0000 7fd760d1d640 1 --2- 172.21.15.39:0/46731637 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd75c105300 0x7fd75c10d850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:59.421 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:59.418+0000 7fd760d1d640 1 --2- 172.21.15.39:0/46731637 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd75c1049b0 0x7fd75c104dc0 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:59.421 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:59.418+0000 7fd760d1d640 1 -- 172.21.15.39:0/46731637 >> 172.21.15.39:0/46731637 conn(0x7fd75c079730 msgr2=0x7fd75c079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:46:59.421 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:59.418+0000 7fd760d1d640 1 -- 172.21.15.39:0/46731637 shutdown_connections 2024-09-21T03:46:59.422 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:59.418+0000 7fd760d1d640 1 -- 172.21.15.39:0/46731637 wait complete. 2024-09-21T03:46:59.422 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:59.419+0000 7fd760d1d640 1 Processor -- start 2024-09-21T03:46:59.422 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:59.419+0000 7fd760d1d640 1 -- start start 2024-09-21T03:46:59.423 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:59.419+0000 7fd760d1d640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd75c1049b0 0x7fd75c102710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:59.423 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:59.419+0000 7fd760d1d640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd75c105300 0x7fd75c102c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:59.423 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:59.419+0000 7fd760d1d640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd75c10de10 0x7fd75c103190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:59.423 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:59.419+0000 7fd760d1d640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fd75c113ae0 con 0x7fd75c1049b0 2024-09-21T03:46:59.423 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:59.419+0000 7fd760d1d640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fd75c113960 con 0x7fd75c105300 2024-09-21T03:46:59.423 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:59.419+0000 7fd760d1d640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fd75c113c60 con 0x7fd75c10de10 2024-09-21T03:46:59.423 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:59.420+0000 7fd759d9b640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd75c105300 0x7fd75c102c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:46:59.423 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:59.420+0000 7fd759d9b640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd75c105300 0x7fd75c102c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.39:35286/0 (socket says 172.21.15.39:35286) 2024-09-21T03:46:59.424 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:59.420+0000 7fd759d9b640 1 -- 172.21.15.39:0/1834079303 learned_addr learned my addr 172.21.15.39:0/1834079303 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:46:59.424 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:59.420+0000 7fd75a59c640 1 --2- 172.21.15.39:0/1834079303 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd75c1049b0 0x7fd75c102710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:46:59.424 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:59.420+0000 7fd759d9b640 1 -- 172.21.15.39:0/1834079303 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd75c10de10 msgr2=0x7fd75c103190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:46:59.424 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:59.420+0000 7fd759d9b640 1 --2- 172.21.15.39:0/1834079303 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd75c10de10 0x7fd75c103190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:59.424 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:59.420+0000 7fd759d9b640 1 -- 172.21.15.39:0/1834079303 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd75c1049b0 msgr2=0x7fd75c102710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:46:59.425 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:59.420+0000 7fd759d9b640 1 --2- 172.21.15.39:0/1834079303 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd75c1049b0 0x7fd75c102710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:59.425 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:59.420+0000 7fd759d9b640 1 -- 172.21.15.39:0/1834079303 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd75c074150 con 0x7fd75c105300 2024-09-21T03:46:59.425 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:59.420+0000 7fd75a59c640 1 --2- 172.21.15.39:0/1834079303 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd75c1049b0 0x7fd75c102710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:46:59.425 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:59.421+0000 7fd759d9b640 1 --2- 172.21.15.39:0/1834079303 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd75c105300 0x7fd75c102c50 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fd75000e9f0 tx=0x7fd75000eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:46:59.425 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:59.421+0000 7fd74b7fe640 1 -- 172.21.15.39:0/1834079303 <== mon.2 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd75000ccc0 con 0x7fd75c105300 2024-09-21T03:46:59.425 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:59.421+0000 7fd760d1d640 1 -- 172.21.15.39:0/1834079303 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd75c074440 con 0x7fd75c105300 2024-09-21T03:46:59.425 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:59.421+0000 7fd760d1d640 1 -- 172.21.15.39:0/1834079303 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd75c0749a0 con 0x7fd75c105300 2024-09-21T03:46:59.425 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:59.421+0000 7fd74b7fe640 1 -- 172.21.15.39:0/1834079303 <== mon.2 v2:172.21.15.150:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd7500108e0 con 0x7fd75c105300 2024-09-21T03:46:59.425 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:59.421+0000 7fd74b7fe640 1 -- 172.21.15.39:0/1834079303 <== mon.2 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd750018860 con 0x7fd75c105300 2024-09-21T03:46:59.427 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:59.424+0000 7fd767291740 1 Processor -- start 2024-09-21T03:46:59.427 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:59.424+0000 7fd767291740 1 -- start start 2024-09-21T03:46:59.429 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:59.425+0000 7fd767291740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562af10bc070 0x562af14632c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:59.429 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:59.425+0000 7fd767291740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562af10931b0 0x562af1463800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:59.429 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:59.425+0000 7fd767291740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562af109b390 0x562af1463d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:59.429 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:59.425+0000 7fd767291740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x562af13d9ea0 con 0x562af10bc070 2024-09-21T03:46:59.429 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:59.425+0000 7fd767291740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x562af13c92d0 con 0x562af10931b0 2024-09-21T03:46:59.429 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:59.425+0000 7fd767291740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x562af13cd0e0 con 0x562af109b390 2024-09-21T03:46:59.429 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:59.425+0000 7fd75ad9d640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562af10bc070 0x562af14632c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:46:59.429 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:59.425+0000 7fd75ad9d640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562af10bc070 0x562af14632c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:52124/0 (socket says 172.21.15.39:52124) 2024-09-21T03:46:59.429 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:59.425+0000 7fd75ad9d640 1 -- 172.21.15.39:0/3627489104 learned_addr learned my addr 172.21.15.39:0/3627489104 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:46:59.430 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:59.426+0000 7fd75a59c640 1 --2- 172.21.15.39:0/3627489104 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562af10931b0 0x562af1463800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:46:59.430 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:59.426+0000 7fd74b7fe640 1 -- 172.21.15.39:0/1834079303 <== mon.2 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 25) ==== 154637+0+0 (secure 0 0 0) 0x7fd750018a00 con 0x7fd75c105300 2024-09-21T03:46:59.430 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:59.426+0000 7fd75a59c640 1 -- 172.21.15.39:0/3627489104 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562af109b390 msgr2=0x562af1463d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:46:59.430 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:59.426+0000 7fd759d9b640 1 --2- 172.21.15.39:0/3627489104 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562af109b390 0x562af1463d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:46:59.430 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:59.426+0000 7fd75a59c640 1 --2- 172.21.15.39:0/3627489104 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562af109b390 0x562af1463d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:59.430 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:59.426+0000 7fd75a59c640 1 -- 172.21.15.39:0/3627489104 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562af10bc070 msgr2=0x562af14632c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:46:59.430 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:59.426+0000 7fd75a59c640 1 --2- 172.21.15.39:0/3627489104 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562af10bc070 0x562af14632c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:59.430 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:59.426+0000 7fd75a59c640 1 -- 172.21.15.39:0/3627489104 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562af1127650 con 0x562af10931b0 2024-09-21T03:46:59.431 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:59.426+0000 7fd75ad9d640 1 --2- 172.21.15.39:0/3627489104 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562af10bc070 0x562af14632c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:46:59.432 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:59.426+0000 7fd759d9b640 1 --2- 172.21.15.39:0/3627489104 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562af109b390 0x562af1463d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:46:59.432 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:59.427+0000 7fd75a59c640 1 --2- 172.21.15.39:0/3627489104 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562af10931b0 0x562af1463800 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fd74c02a930 tx=0x7fd74c004540 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7d87cb2b4639c615 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:46:59.432 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:59.427+0000 7fd748ff9640 1 -- 172.21.15.39:0/3627489104 <== mon.2 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd74c035040 con 0x562af10931b0 2024-09-21T03:46:59.432 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:59.427+0000 7fd748ff9640 1 -- 172.21.15.39:0/3627489104 <== mon.2 v2:172.21.15.150:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd74c004770 con 0x562af10931b0 2024-09-21T03:46:59.432 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:59.427+0000 7fd74b7fe640 1 --2- 172.21.15.39:0/1834079303 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7fd7440b6780 0x7fd7440b8c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:59.432 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:59.428+0000 7fd75ad9d640 1 --2- 172.21.15.39:0/1834079303 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7fd7440b6780 0x7fd7440b8c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:46:59.432 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:59.428+0000 7fd74b7fe640 1 -- 172.21.15.39:0/1834079303 <== mon.2 v2:172.21.15.150:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8484+0+0 (secure 0 0 0) 0x7fd7500e7830 con 0x7fd75c105300 2024-09-21T03:46:59.432 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:59.428+0000 7fd748ff9640 1 -- 172.21.15.39:0/3627489104 <== mon.2 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd74c003a50 con 0x562af10931b0 2024-09-21T03:46:59.432 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:59.428+0000 7fd75ad9d640 1 --2- 172.21.15.39:0/1834079303 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7fd7440b6780 0x7fd7440b8c90 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fd754009270 tx=0x7fd754009680 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:46:59.432 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:59.428+0000 7fd767291740 1 -- 172.21.15.39:0/3627489104 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562af10931b0 msgr2=0x562af1463800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:46:59.432 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:59.428+0000 7fd767291740 1 --2- 172.21.15.39:0/3627489104 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562af10931b0 0x562af1463800 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fd74c02a930 tx=0x7fd74c004540 comp rx=0 tx=0).stop 2024-09-21T03:46:59.432 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:59.429+0000 7fd767291740 1 -- 172.21.15.39:0/3627489104 shutdown_connections 2024-09-21T03:46:59.432 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:59.429+0000 7fd767291740 1 --2- 172.21.15.39:0/3627489104 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562af109b390 0x562af1463d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:59.433 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:59.429+0000 7fd767291740 1 --2- 172.21.15.39:0/3627489104 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562af10931b0 0x562af1463800 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:59.433 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:59.429+0000 7fd767291740 1 --2- 172.21.15.39:0/3627489104 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562af10bc070 0x562af14632c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:59.433 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:59.429+0000 7fd767291740 1 -- 172.21.15.39:0/3627489104 >> 172.21.15.39:0/3627489104 conn(0x562af10b0460 msgr2=0x562af14611b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:46:59.433 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:59.429+0000 7fd767291740 1 -- 172.21.15.39:0/3627489104 shutdown_connections 2024-09-21T03:46:59.437 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:59.429+0000 7fd767291740 1 -- 172.21.15.39:0/3627489104 wait complete. 2024-09-21T03:46:59.437 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:59.429+0000 7fd767291740 1 -- start start 2024-09-21T03:46:59.437 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:59.429+0000 7fd767291740 1 Processor -- start 2024-09-21T03:46:59.437 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:59.430+0000 7fd767291740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562af10931b0 0x562af146e870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:59.437 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:59.430+0000 7fd767291740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562af109b390 0x562af146edb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:59.437 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:59.430+0000 7fd767291740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562af10bc070 0x562af146f2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:59.437 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:59.430+0000 7fd767291740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x562af122a700 con 0x562af10931b0 2024-09-21T03:46:59.437 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:59.430+0000 7fd767291740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x562af1364160 con 0x562af109b390 2024-09-21T03:46:59.437 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:59.430+0000 7fd767291740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x562af1471ae0 con 0x562af10bc070 2024-09-21T03:46:59.438 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:59.430+0000 7fd767291740 10 client.0 _ll_register_callbacks cb 0x562af137d9e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:46:59.438 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:59.430+0000 7fd759d9b640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562af109b390 0x562af146edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:46:59.438 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:59.430+0000 7fd759d9b640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562af109b390 0x562af146edb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.39:35294/0 (socket says 172.21.15.39:35294) 2024-09-21T03:46:59.438 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:59.430+0000 7fd759d9b640 1 -- 172.21.15.39:0/2404383957 learned_addr learned my addr 172.21.15.39:0/2404383957 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:46:59.438 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:59.431+0000 7fd75a59c640 1 --2- 172.21.15.39:0/2404383957 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562af10931b0 0x562af146e870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:46:59.438 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:59.431+0000 7fd759d9b640 1 -- 172.21.15.39:0/2404383957 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562af10bc070 msgr2=0x562af146f2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:46:59.438 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:59.431+0000 7fd75ad9d640 1 --2- 172.21.15.39:0/2404383957 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562af10bc070 0x562af146f2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:46:59.438 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:59.431+0000 7fd759d9b640 1 --2- 172.21.15.39:0/2404383957 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562af10bc070 0x562af146f2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:59.438 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:59.431+0000 7fd759d9b640 1 -- 172.21.15.39:0/2404383957 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562af10931b0 msgr2=0x562af146e870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:46:59.438 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:59.431+0000 7fd759d9b640 1 --2- 172.21.15.39:0/2404383957 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562af10931b0 0x562af146e870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:59.438 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:59.431+0000 7fd759d9b640 1 -- 172.21.15.39:0/2404383957 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562af1391c90 con 0x562af109b390 2024-09-21T03:46:59.438 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:59.431+0000 7fd75a59c640 1 --2- 172.21.15.39:0/2404383957 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562af10931b0 0x562af146e870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:46:59.439 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:59.431+0000 7fd75ad9d640 1 --2- 172.21.15.39:0/2404383957 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562af10bc070 0x562af146f2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:46:59.439 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:59.431+0000 7fd759d9b640 1 --2- 172.21.15.39:0/2404383957 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562af109b390 0x562af146edb0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fd7500ad5d0 tx=0x7fd750010800 comp rx=0 tx=0).ready entity=mon.2 client_cookie=42e33df59e65d4ac server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:46:59.439 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:59.431+0000 7fd741ffb640 10 client.0 ms_handle_connect on v2:172.21.15.150:3300/0 2024-09-21T03:46:59.439 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:59.432+0000 7fd741ffb640 1 -- 172.21.15.39:0/2404383957 <== mon.2 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7500acdb0 con 0x562af109b390 2024-09-21T03:46:59.439 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:59.432+0000 7fd767291740 1 -- 172.21.15.39:0/2404383957 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd74c004770 con 0x562af109b390 2024-09-21T03:46:59.439 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:59.432+0000 7fd741ffb640 1 -- 172.21.15.39:0/2404383957 <== mon.2 v2:172.21.15.150:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8484+0+0 (secure 0 0 0) 0x7fd7500b6040 con 0x562af109b390 2024-09-21T03:46:59.439 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:59.432+0000 7fd741ffb640 1 -- 172.21.15.39:0/2404383957 <== mon.2 v2:172.21.15.150:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fd75000ce60 con 0x562af109b390 2024-09-21T03:46:59.439 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:59.432+0000 7fd767291740 10 client.25307 fetch_fsmap learned FSMap version 29 2024-09-21T03:46:59.439 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:59.432+0000 7fd767291740 1 -- 172.21.15.39:0/2404383957 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x562af147d6d0 con 0x562af109b390 2024-09-21T03:46:59.439 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:59.433+0000 7fd741ffb640 1 -- 172.21.15.39:0/2404383957 <== mon.2 v2:172.21.15.150:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd74c004770 con 0x562af109b390 2024-09-21T03:46:59.439 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:59.433+0000 7fd741ffb640 1 -- 172.21.15.39:0/2404383957 <== mon.2 v2:172.21.15.150:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fd7500ac310 con 0x562af109b390 2024-09-21T03:46:59.439 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:59.433+0000 7fd767291740 10 client.25307 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T03:46:59.439 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:59.433+0000 7fd767291740 10 client.25307 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-21T03:46:59.439 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:59.433+0000 7fd767291740 20 client.25307 populate_metadata read hostname 'smithi039' 2024-09-21T03:46:59.440 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:59.433+0000 7fd767291740 1 --2- 172.21.15.39:0/2404383957 >> [v2:172.21.15.161:6838/2190107270,v1:172.21.15.161:6839/2190107270] conn(0x562af107a9f0 0x562af1480c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:59.440 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:59.433+0000 7fd767291740 4 client.25307 mds_command: new command op to 24667 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:46:59.440 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:59.433+0000 7fd767291740 1 -- 172.21.15.39:0/2404383957 --> [v2:172.21.15.161:6838/2190107270,v1:172.21.15.161:6839/2190107270] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562af13dbdf0 con 0x562af107a9f0 2024-09-21T03:46:59.440 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:59.434+0000 7fd75a59c640 1 --2- 172.21.15.39:0/2404383957 >> [v2:172.21.15.161:6838/2190107270,v1:172.21.15.161:6839/2190107270] conn(0x562af107a9f0 0x562af1480c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:46:59.440 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:59.434+0000 7fd741ffb640 10 client.25307 ms_handle_connect on v2:172.21.15.161:6838/2190107270 2024-09-21T03:46:59.440 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:59.434+0000 7fd75a59c640 1 --2- 172.21.15.39:0/2404383957 >> [v2:172.21.15.161:6838/2190107270,v1:172.21.15.161:6839/2190107270] conn(0x562af107a9f0 0x562af1480c20 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=2fe1f53ef7d0d5a0 server_cookie=a244a0e0862c1608 in_seq=0 out_seq=0 2024-09-21T03:46:59.441 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:59.435+0000 7fd741ffb640 1 -- 172.21.15.39:0/2404383957 <== mds.1 v2:172.21.15.161:6838/2190107270 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x562af13dbdf0 con 0x562af107a9f0 2024-09-21T03:46:59.441 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:59.435+0000 7fd741ffb640 10 client.25307 handle_command_reply: tid=0 2024-09-21T03:46:59.441 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:59.435+0000 7fd767291740 1 client.25307 shutdown 2024-09-21T03:46:59.441 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:59.435+0000 7fd767291740 1 -- 172.21.15.39:0/2404383957 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562af109b390 msgr2=0x562af146edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:46:59.441 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:59.435+0000 7fd767291740 1 --2- 172.21.15.39:0/2404383957 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562af109b390 0x562af146edb0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fd7500ad5d0 tx=0x7fd750010800 comp rx=0 tx=0).stop 2024-09-21T03:46:59.442 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:59.435+0000 7fd767291740 1 -- 172.21.15.39:0/2404383957 shutdown_connections 2024-09-21T03:46:59.442 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:59.435+0000 7fd767291740 1 --2- 172.21.15.39:0/2404383957 >> [v2:172.21.15.161:6838/2190107270,v1:172.21.15.161:6839/2190107270] conn(0x562af107a9f0 0x562af1480c20 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:59.442 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:59.435+0000 7fd767291740 1 --2- 172.21.15.39:0/2404383957 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562af10bc070 0x562af146f2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:59.442 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:59.435+0000 7fd767291740 1 --2- 172.21.15.39:0/2404383957 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562af109b390 0x562af146edb0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:59.442 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:59.435+0000 7fd767291740 1 --2- 172.21.15.39:0/2404383957 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562af10931b0 0x562af146e870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:59.442 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:59.435+0000 7fd767291740 1 -- 172.21.15.39:0/2404383957 >> 172.21.15.39:0/2404383957 conn(0x562af10b0460 msgr2=0x562af1466450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:46:59.442 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:59.435+0000 7fd741ffb640 0 client.25307 ms_handle_reset on v2:172.21.15.161:6838/2190107270 2024-09-21T03:46:59.442 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:59.435+0000 7fd767291740 1 -- 172.21.15.39:0/2404383957 shutdown_connections 2024-09-21T03:46:59.442 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:59.436+0000 7fd767291740 1 -- 172.21.15.39:0/2404383957 wait complete. 2024-09-21T03:46:59.442 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:59.436+0000 7fd767291740 20 client.25307 trim_cache size 0 max 16384 2024-09-21T03:46:59.452 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:59.449+0000 7fd767291740 1 Processor -- start 2024-09-21T03:46:59.453 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:59.449+0000 7fd767291740 1 -- start start 2024-09-21T03:46:59.453 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:59.450+0000 7fd767291740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562af10931b0 0x562af1470750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:59.453 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:59.450+0000 7fd767291740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562af109b390 0x562af1464a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:59.453 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:59.450+0000 7fd767291740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562af10bc070 0x562af1460760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:59.453 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:59.450+0000 7fd767291740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fd73c00e740 con 0x562af109b390 2024-09-21T03:46:59.453 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:59.450+0000 7fd767291740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fd73c010280 con 0x562af10931b0 2024-09-21T03:46:59.453 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:59.450+0000 7fd767291740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fd73c00ee10 con 0x562af10bc070 2024-09-21T03:46:59.453 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:59.450+0000 7fd75ad9d640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562af10bc070 0x562af1460760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:46:59.455 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:59.450+0000 7fd75a59c640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562af10931b0 0x562af1470750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:46:59.455 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:59.450+0000 7fd75a59c640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562af10931b0 0x562af1470750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.39:35298/0 (socket says 172.21.15.39:35298) 2024-09-21T03:46:59.455 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:59.450+0000 7fd75a59c640 1 -- 172.21.15.39:0/4291409502 learned_addr learned my addr 172.21.15.39:0/4291409502 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:46:59.455 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:59.450+0000 7fd75ad9d640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562af10bc070 0x562af1460760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.39:37466/0 (socket says 172.21.15.39:37466) 2024-09-21T03:46:59.455 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:59.450+0000 7fd759d9b640 1 --2- 172.21.15.39:0/4291409502 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562af109b390 0x562af1464a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:46:59.455 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:59.451+0000 7fd75a59c640 1 -- 172.21.15.39:0/4291409502 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562af10bc070 msgr2=0x562af1460760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:46:59.455 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:59.451+0000 7fd75a59c640 1 --2- 172.21.15.39:0/4291409502 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562af10bc070 0x562af1460760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:59.455 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:59.451+0000 7fd75a59c640 1 -- 172.21.15.39:0/4291409502 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562af109b390 msgr2=0x562af1464a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:46:59.455 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:59.451+0000 7fd75a59c640 1 --2- 172.21.15.39:0/4291409502 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562af109b390 0x562af1464a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:59.455 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:59.451+0000 7fd75a59c640 1 -- 172.21.15.39:0/4291409502 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562af146f910 con 0x562af10931b0 2024-09-21T03:46:59.455 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:59.451+0000 7fd75ad9d640 1 --2- 172.21.15.39:0/4291409502 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562af10bc070 0x562af1460760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:46:59.456 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:59.451+0000 7fd759d9b640 1 --2- 172.21.15.39:0/4291409502 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562af109b390 0x562af1464a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:46:59.456 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:59.451+0000 7fd75a59c640 1 --2- 172.21.15.39:0/4291409502 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562af10931b0 0x562af1470750 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fd74c009900 tx=0x7fd74c02a410 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9d5c18101cf1d33a server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:46:59.456 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:59.452+0000 7fd743fff640 1 -- 172.21.15.39:0/4291409502 <== mon.2 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd74c05b070 con 0x562af10931b0 2024-09-21T03:46:59.456 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:59.452+0000 7fd743fff640 1 -- 172.21.15.39:0/4291409502 <== mon.2 v2:172.21.15.150:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd74c004c20 con 0x562af10931b0 2024-09-21T03:46:59.456 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:59.452+0000 7fd743fff640 1 -- 172.21.15.39:0/4291409502 <== mon.2 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd74c035040 con 0x562af10931b0 2024-09-21T03:46:59.458 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:59.453+0000 7fd767291740 1 -- 172.21.15.39:0/4291409502 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562af10931b0 msgr2=0x562af1470750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:46:59.459 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:59.453+0000 7fd767291740 1 --2- 172.21.15.39:0/4291409502 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562af10931b0 0x562af1470750 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fd74c009900 tx=0x7fd74c02a410 comp rx=0 tx=0).stop 2024-09-21T03:46:59.459 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:59.453+0000 7fd767291740 1 -- 172.21.15.39:0/4291409502 shutdown_connections 2024-09-21T03:46:59.459 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:59.453+0000 7fd767291740 1 --2- 172.21.15.39:0/4291409502 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562af10bc070 0x562af1460760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:59.459 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:59.453+0000 7fd767291740 1 --2- 172.21.15.39:0/4291409502 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562af109b390 0x562af1464a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:59.459 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:59.453+0000 7fd767291740 1 --2- 172.21.15.39:0/4291409502 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562af10931b0 0x562af1470750 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:59.459 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:59.453+0000 7fd767291740 1 -- 172.21.15.39:0/4291409502 >> 172.21.15.39:0/4291409502 conn(0x562af10b0460 msgr2=0x562af14642f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:46:59.459 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:59.453+0000 7fd767291740 1 -- 172.21.15.39:0/4291409502 shutdown_connections 2024-09-21T03:46:59.459 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:59.453+0000 7fd767291740 1 -- 172.21.15.39:0/4291409502 wait complete. 2024-09-21T03:46:59.459 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:59.453+0000 7fd767291740 1 -- start start 2024-09-21T03:46:59.459 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:59.453+0000 7fd767291740 1 Processor -- start 2024-09-21T03:46:59.459 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:59.454+0000 7fd767291740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562af109b390 0x562af1472d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:59.459 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:59.454+0000 7fd767291740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562af10bc070 0x562af1473280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:59.459 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:59.454+0000 7fd767291740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562af104fda0 0x562af14737c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:59.460 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:59.454+0000 7fd75a59c640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562af109b390 0x562af1472d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:46:59.460 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:59.454+0000 7fd75a59c640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562af109b390 0x562af1472d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.39:35300/0 (socket says 172.21.15.39:35300) 2024-09-21T03:46:59.460 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:59.454+0000 7fd767291740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fd73c00fbb0 con 0x562af104fda0 2024-09-21T03:46:59.460 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:59.454+0000 7fd767291740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fd73c00f4e0 con 0x562af109b390 2024-09-21T03:46:59.460 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:59.454+0000 7fd767291740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fd73c00c720 con 0x562af10bc070 2024-09-21T03:46:59.460 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:59.454+0000 7fd75a59c640 1 -- 172.21.15.39:0/874274529 learned_addr learned my addr 172.21.15.39:0/874274529 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:46:59.460 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:59.454+0000 7fd767291740 10 client.0 _ll_register_callbacks cb 0x7fd7500a9ae0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:46:59.460 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:59.454+0000 7fd75ad9d640 1 --2- 172.21.15.39:0/874274529 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562af104fda0 0x562af14737c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:46:59.460 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:59.454+0000 7fd759d9b640 1 --2- 172.21.15.39:0/874274529 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562af10bc070 0x562af1473280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:46:59.460 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:59.455+0000 7fd75ad9d640 1 -- 172.21.15.39:0/874274529 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562af10bc070 msgr2=0x562af1473280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:46:59.461 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:59.455+0000 7fd75ad9d640 1 --2- 172.21.15.39:0/874274529 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562af10bc070 0x562af1473280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:59.461 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:59.455+0000 7fd75ad9d640 1 -- 172.21.15.39:0/874274529 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562af109b390 msgr2=0x562af1472d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:46:59.461 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:59.455+0000 7fd75ad9d640 1 --2- 172.21.15.39:0/874274529 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562af109b390 0x562af1472d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:59.461 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:59.455+0000 7fd75ad9d640 1 -- 172.21.15.39:0/874274529 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562af14ea5e0 con 0x562af104fda0 2024-09-21T03:46:59.461 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:59.455+0000 7fd75a59c640 1 --2- 172.21.15.39:0/874274529 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562af109b390 0x562af1472d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:46:59.461 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:59.455+0000 7fd75ad9d640 1 --2- 172.21.15.39:0/874274529 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562af104fda0 0x562af14737c0 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7fd754012010 tx=0x7fd754004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab8bf83d367fb5c9 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:46:59.461 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:59.455+0000 7fd741ffb640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-09-21T03:46:59.461 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:59.455+0000 7fd741ffb640 1 -- 172.21.15.39:0/874274529 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd754003c40 con 0x562af104fda0 2024-09-21T03:46:59.461 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:59.456+0000 7fd741ffb640 1 -- 172.21.15.39:0/874274529 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8484+0+0 (secure 0 0 0) 0x7fd754003de0 con 0x562af104fda0 2024-09-21T03:46:59.461 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:59.456+0000 7fd767291740 1 -- 172.21.15.39:0/874274529 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd74c004c20 con 0x562af104fda0 2024-09-21T03:46:59.461 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:59.456+0000 7fd741ffb640 1 -- 172.21.15.39:0/874274529 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fd754004050 con 0x562af104fda0 2024-09-21T03:46:59.461 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:59.456+0000 7fd767291740 10 client.15708 fetch_fsmap learned FSMap version 29 2024-09-21T03:46:59.461 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:59.456+0000 7fd767291740 1 -- 172.21.15.39:0/874274529 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x562af14eb880 con 0x562af104fda0 2024-09-21T03:46:59.463 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:59.456+0000 7fd741ffb640 1 -- 172.21.15.39:0/874274529 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd74c004c20 con 0x562af104fda0 2024-09-21T03:46:59.463 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:59.456+0000 7fd741ffb640 1 -- 172.21.15.39:0/874274529 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fd75400f040 con 0x562af104fda0 2024-09-21T03:46:59.463 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:59.456+0000 7fd767291740 10 client.15708 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T03:46:59.463 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:59.456+0000 7fd767291740 10 client.15708 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-21T03:46:59.464 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:59.456+0000 7fd767291740 20 client.15708 populate_metadata read hostname 'smithi039' 2024-09-21T03:46:59.464 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:59.456+0000 7fd767291740 1 --2- 172.21.15.39:0/874274529 >> [v2:172.21.15.161:6838/2190107270,v1:172.21.15.161:6839/2190107270] conn(0x562af14ebfb0 0x562af14ee3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:59.464 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:59.457+0000 7fd767291740 4 client.15708 mds_command: new command op to 24667 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T03:46:59.464 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:59.457+0000 7fd767291740 1 -- 172.21.15.39:0/874274529 --> [v2:172.21.15.161:6838/2190107270,v1:172.21.15.161:6839/2190107270] -- command(tid 0: {"prefix": "damage ls"}) -- 0x562af13d78a0 con 0x562af14ebfb0 2024-09-21T03:46:59.464 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:59.457+0000 7fd75a59c640 1 --2- 172.21.15.39:0/874274529 >> [v2:172.21.15.161:6838/2190107270,v1:172.21.15.161:6839/2190107270] conn(0x562af14ebfb0 0x562af14ee3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:46:59.464 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:59.457+0000 7fd741ffb640 10 client.15708 ms_handle_connect on v2:172.21.15.161:6838/2190107270 2024-09-21T03:46:59.464 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:59.457+0000 7fd75a59c640 1 --2- 172.21.15.39:0/874274529 >> [v2:172.21.15.161:6838/2190107270,v1:172.21.15.161:6839/2190107270] conn(0x562af14ebfb0 0x562af14ee3d0 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=1566d0594c565dfa server_cookie=d639b99376a98b5e in_seq=0 out_seq=0 2024-09-21T03:46:59.464 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:59.458+0000 7fd741ffb640 1 -- 172.21.15.39:0/874274529 <== mds.1 v2:172.21.15.161:6838/2190107270 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x562af13d78a0 con 0x562af14ebfb0 2024-09-21T03:46:59.464 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:59.458+0000 7fd741ffb640 10 client.15708 handle_command_reply: tid=0 2024-09-21T03:46:59.464 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:59.458+0000 7fd767291740 1 client.15708 shutdown 2024-09-21T03:46:59.464 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:59.459+0000 7fd767291740 1 -- 172.21.15.39:0/874274529 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562af104fda0 msgr2=0x562af14737c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:46:59.464 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:59.459+0000 7fd767291740 1 --2- 172.21.15.39:0/874274529 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562af104fda0 0x562af14737c0 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7fd754012010 tx=0x7fd754004a40 comp rx=0 tx=0).stop 2024-09-21T03:46:59.464 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:59.459+0000 7fd767291740 1 -- 172.21.15.39:0/874274529 shutdown_connections 2024-09-21T03:46:59.464 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:59.459+0000 7fd767291740 1 --2- 172.21.15.39:0/874274529 >> [v2:172.21.15.161:6838/2190107270,v1:172.21.15.161:6839/2190107270] conn(0x562af14ebfb0 0x562af14ee3d0 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:59.465 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:59.459+0000 7fd767291740 1 --2- 172.21.15.39:0/874274529 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562af104fda0 0x562af14737c0 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:59.465 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:59.459+0000 7fd767291740 1 --2- 172.21.15.39:0/874274529 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562af10bc070 0x562af1473280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:59.465 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:59.459+0000 7fd767291740 1 --2- 172.21.15.39:0/874274529 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562af109b390 0x562af1472d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:59.465 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:59.459+0000 7fd767291740 1 -- 172.21.15.39:0/874274529 >> 172.21.15.39:0/874274529 conn(0x562af10b0460 msgr2=0x562af147a730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:46:59.465 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:59.459+0000 7fd741ffb640 0 client.15708 ms_handle_reset on v2:172.21.15.161:6838/2190107270 2024-09-21T03:46:59.465 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:59.459+0000 7fd767291740 1 -- 172.21.15.39:0/874274529 shutdown_connections 2024-09-21T03:46:59.465 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:59.459+0000 7fd767291740 1 -- 172.21.15.39:0/874274529 wait complete. 2024-09-21T03:46:59.465 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:59.459+0000 7fd767291740 20 client.15708 trim_cache size 0 max 16384 2024-09-21T03:46:59.465 INFO:teuthology.orchestra.run.smithi039.stdout:[] 2024-09-21T03:46:59.466 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:59.460+0000 7fd760d1d640 1 -- 172.21.15.39:0/1834079303 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7fd7440b6780 msgr2=0x7fd7440b8c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:46:59.466 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:59.460+0000 7fd760d1d640 1 --2- 172.21.15.39:0/1834079303 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7fd7440b6780 0x7fd7440b8c90 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fd754009270 tx=0x7fd754009680 comp rx=0 tx=0).stop 2024-09-21T03:46:59.466 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:59.460+0000 7fd760d1d640 1 -- 172.21.15.39:0/1834079303 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd75c105300 msgr2=0x7fd75c102c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:46:59.466 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:59.460+0000 7fd760d1d640 1 --2- 172.21.15.39:0/1834079303 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd75c105300 0x7fd75c102c50 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fd75000e9f0 tx=0x7fd75000eeb0 comp rx=0 tx=0).stop 2024-09-21T03:46:59.466 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:59.460+0000 7fd760d1d640 1 -- 172.21.15.39:0/1834079303 shutdown_connections 2024-09-21T03:46:59.466 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:59.460+0000 7fd760d1d640 1 --2- 172.21.15.39:0/1834079303 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7fd7440b6780 0x7fd7440b8c90 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:59.466 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:59.460+0000 7fd760d1d640 1 --2- 172.21.15.39:0/1834079303 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd75c10de10 0x7fd75c103190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:59.466 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:59.460+0000 7fd760d1d640 1 --2- 172.21.15.39:0/1834079303 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd75c105300 0x7fd75c102c50 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:59.467 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:59.460+0000 7fd760d1d640 1 --2- 172.21.15.39:0/1834079303 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd75c1049b0 0x7fd75c102710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:59.467 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:59.461+0000 7fd760d1d640 1 -- 172.21.15.39:0/1834079303 >> 172.21.15.39:0/1834079303 conn(0x7fd75c079730 msgr2=0x7fd75c0ff550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:46:59.467 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:59.461+0000 7fd760d1d640 1 -- 172.21.15.39:0/1834079303 shutdown_connections 2024-09-21T03:46:59.467 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:46:59.461+0000 7fd760d1d640 1 -- 172.21.15.39:0/1834079303 wait complete. 2024-09-21T03:47:00.166 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 51116d5c-77ca-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 damage ls 2024-09-21T03:47:00.571 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/51116d5c-77ca-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:47:00.583 INFO:journalctl@ceph.mgr.x.smithi039.stdout:Sep 21 03:47:00 smithi039 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-mgr-x[31438]: 2024-09-21T03:47:00.104+0000 7fe40dc25640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T03:47:00.584 INFO:journalctl@ceph.mgr.x.smithi039.stdout:Sep 21 03:47:00 smithi039 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-mgr-x[31438]: 2024-09-21T03:47:00.104+0000 7fe40dc25640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T03:47:00.584 INFO:journalctl@ceph.mgr.x.smithi039.stdout:Sep 21 03:47:00 smithi039 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-mgr-x[31438]: 2024-09-21T03:47:00.104+0000 7fe40dc25640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T03:47:00.584 INFO:journalctl@ceph.mgr.x.smithi039.stdout:Sep 21 03:47:00 smithi039 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-mgr-x[31438]: 2024-09-21T03:47:00.104+0000 7fe40dc25640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T03:47:00.584 INFO:journalctl@ceph.mgr.x.smithi039.stdout:Sep 21 03:47:00 smithi039 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-mgr-x[31438]: 2024-09-21T03:47:00.104+0000 7fe40dc25640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T03:47:00.945 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:47:00 smithi039 ceph-mon[31211]: pgmap v354: 97 pgs: 97 active+clean; 1.4 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 682 B/s wr, 11 op/s 2024-09-21T03:47:00.968 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:47:00 smithi161 ceph-mon[34533]: pgmap v354: 97 pgs: 97 active+clean; 1.4 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 682 B/s wr, 11 op/s 2024-09-21T03:47:00.975 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:47:00 smithi150 ceph-mon[34289]: pgmap v354: 97 pgs: 97 active+clean; 1.4 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 682 B/s wr, 11 op/s 2024-09-21T03:47:01.095 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:01.092+0000 7fadce19f640 1 -- 172.21.15.39:0/2531900233 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fadc8108b50 msgr2=0x7fadc810f380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:47:01.095 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:01.092+0000 7fadce19f640 1 --2- 172.21.15.39:0/2531900233 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fadc8108b50 0x7fadc810f380 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7fadc0009ca0 tx=0x7fadc002a8c0 comp rx=0 tx=0).stop 2024-09-21T03:47:01.096 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:01.093+0000 7fadce19f640 1 -- 172.21.15.39:0/2531900233 shutdown_connections 2024-09-21T03:47:01.096 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:01.093+0000 7fadce19f640 1 --2- 172.21.15.39:0/2531900233 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fadc8108b50 0x7fadc810f380 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:01.096 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:01.093+0000 7fadce19f640 1 --2- 172.21.15.39:0/2531900233 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fadc8108200 0x7fadc8108610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:01.096 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:01.093+0000 7fadce19f640 1 --2- 172.21.15.39:0/2531900233 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fadc81028c0 0x7fadc8102cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:01.096 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:01.093+0000 7fadce19f640 1 -- 172.21.15.39:0/2531900233 >> 172.21.15.39:0/2531900233 conn(0x7fadc80fe350 msgr2=0x7fadc81007a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:47:01.096 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:01.093+0000 7fadce19f640 1 -- 172.21.15.39:0/2531900233 shutdown_connections 2024-09-21T03:47:01.096 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:01.093+0000 7fadce19f640 1 -- 172.21.15.39:0/2531900233 wait complete. 2024-09-21T03:47:01.097 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:01.094+0000 7fadce19f640 1 Processor -- start 2024-09-21T03:47:01.097 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:01.094+0000 7fadce19f640 1 -- start start 2024-09-21T03:47:01.097 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:01.094+0000 7fadce19f640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fadc81028c0 0x7fadc8100290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:47:01.097 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:01.094+0000 7fadce19f640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fadc8108200 0x7fadc80fe880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:47:01.097 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:01.094+0000 7fadce19f640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fadc8108b50 0x7fadc80fedc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:47:01.098 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:01.095+0000 7fadce19f640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fadc8112870 con 0x7fadc8108200 2024-09-21T03:47:01.098 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:01.095+0000 7fadce19f640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fadc81126f0 con 0x7fadc8108b50 2024-09-21T03:47:01.098 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:01.095+0000 7fadce19f640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fadc81129f0 con 0x7fadc81028c0 2024-09-21T03:47:01.098 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:01.095+0000 7fadcc99c640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fadc8108200 0x7fadc80fe880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:47:01.098 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:01.095+0000 7fadcc99c640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fadc8108200 0x7fadc80fe880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:52180/0 (socket says 172.21.15.39:52180) 2024-09-21T03:47:01.099 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:01.095+0000 7fadcc99c640 1 -- 172.21.15.39:0/1740393894 learned_addr learned my addr 172.21.15.39:0/1740393894 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:47:01.099 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:01.095+0000 7fadcd99e640 1 --2- 172.21.15.39:0/1740393894 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fadc8108b50 0x7fadc80fedc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:47:01.099 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:01.095+0000 7fadcd19d640 1 --2- 172.21.15.39:0/1740393894 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fadc81028c0 0x7fadc8100290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:47:01.100 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:01.095+0000 7fadcc99c640 1 -- 172.21.15.39:0/1740393894 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fadc81028c0 msgr2=0x7fadc8100290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:47:01.100 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:01.095+0000 7fadcc99c640 1 --2- 172.21.15.39:0/1740393894 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fadc81028c0 0x7fadc8100290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:01.100 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:01.095+0000 7fadcc99c640 1 -- 172.21.15.39:0/1740393894 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fadc8108b50 msgr2=0x7fadc80fedc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:47:01.100 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:01.095+0000 7fadcc99c640 1 --2- 172.21.15.39:0/1740393894 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fadc8108b50 0x7fadc80fedc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:01.100 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:01.095+0000 7fadcc99c640 1 -- 172.21.15.39:0/1740393894 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fadc80ff5e0 con 0x7fadc8108200 2024-09-21T03:47:01.100 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:01.095+0000 7fadcd19d640 1 --2- 172.21.15.39:0/1740393894 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fadc81028c0 0x7fadc8100290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:47:01.100 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:01.096+0000 7fadcc99c640 1 --2- 172.21.15.39:0/1740393894 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fadc8108200 0x7fadc80fe880 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7fadb800e9c0 tx=0x7fadb800ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:47:01.100 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:01.096+0000 7fadbe7fc640 1 -- 172.21.15.39:0/1740393894 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadb800ccc0 con 0x7fadc8108200 2024-09-21T03:47:01.100 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:01.096+0000 7fadbe7fc640 1 -- 172.21.15.39:0/1740393894 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fadb80054d0 con 0x7fadc8108200 2024-09-21T03:47:01.100 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:01.096+0000 7fadce19f640 1 -- 172.21.15.39:0/1740393894 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fadc81acb10 con 0x7fadc8108200 2024-09-21T03:47:01.101 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:01.096+0000 7fadbe7fc640 1 -- 172.21.15.39:0/1740393894 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadb800ccc0 con 0x7fadc8108200 2024-09-21T03:47:01.101 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:01.096+0000 7fadce19f640 1 -- 172.21.15.39:0/1740393894 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fadc81ad000 con 0x7fadc8108200 2024-09-21T03:47:01.103 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:01.100+0000 7fadd5975740 1 Processor -- start 2024-09-21T03:47:01.103 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:01.100+0000 7fadd5975740 1 -- start start 2024-09-21T03:47:01.103 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:01.100+0000 7fadd5975740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559bdb105070 0x559bdb4ac460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:47:01.103 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:01.100+0000 7fadd5975740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559bdb0dc1b0 0x559bdb4ac9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:47:01.103 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:01.100+0000 7fadd5975740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x559bdb0e4390 0x559bdb4acee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:47:01.104 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:01.100+0000 7fadd5975740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x559bdb422ea0 con 0x559bdb0dc1b0 2024-09-21T03:47:01.104 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:01.100+0000 7fadd5975740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x559bdb4122d0 con 0x559bdb0e4390 2024-09-21T03:47:01.104 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:01.100+0000 7fadd5975740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x559bdb4160e0 con 0x559bdb105070 2024-09-21T03:47:01.104 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:01.101+0000 7fadcd19d640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559bdb0dc1b0 0x559bdb4ac9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:47:01.104 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:01.101+0000 7fadcd99e640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559bdb105070 0x559bdb4ac460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:47:01.104 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:01.101+0000 7fadcd19d640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559bdb0dc1b0 0x559bdb4ac9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:52192/0 (socket says 172.21.15.39:52192) 2024-09-21T03:47:01.104 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:01.101+0000 7fadcd19d640 1 -- 172.21.15.39:0/3022070584 learned_addr learned my addr 172.21.15.39:0/3022070584 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:47:01.108 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:01.101+0000 7fadcd19d640 1 -- 172.21.15.39:0/3022070584 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559bdb105070 msgr2=0x559bdb4ac460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:47:01.108 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:01.101+0000 7fadcd19d640 1 --2- 172.21.15.39:0/3022070584 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559bdb105070 0x559bdb4ac460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:01.108 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:01.101+0000 7fadcd19d640 1 -- 172.21.15.39:0/3022070584 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x559bdb0e4390 msgr2=0x559bdb4acee0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-21T03:47:01.108 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:01.101+0000 7fadcd19d640 1 --2- 172.21.15.39:0/3022070584 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x559bdb0e4390 0x559bdb4acee0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:01.108 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:01.101+0000 7fadcd19d640 1 -- 172.21.15.39:0/3022070584 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559bdb170650 con 0x559bdb0dc1b0 2024-09-21T03:47:01.108 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:01.101+0000 7fadcd19d640 1 --2- 172.21.15.39:0/3022070584 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559bdb0dc1b0 0x559bdb4ac9a0 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7fadc4009c20 tx=0x7fadc400e6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9befb264bb95abb1 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:47:01.108 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:01.102+0000 7fadbe7fc640 1 -- 172.21.15.39:0/1740393894 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 25) ==== 154637+0+0 (secure 0 0 0) 0x7fadb8021440 con 0x7fadc8108200 2024-09-21T03:47:01.109 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:01.102+0000 7fadb37fe640 1 -- 172.21.15.39:0/3022070584 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadc4016070 con 0x559bdb0dc1b0 2024-09-21T03:47:01.109 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:01.102+0000 7fadb37fe640 1 -- 172.21.15.39:0/3022070584 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fadc40035f0 con 0x559bdb0dc1b0 2024-09-21T03:47:01.109 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:01.102+0000 7fadb37fe640 1 -- 172.21.15.39:0/3022070584 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadc40116d0 con 0x559bdb0dc1b0 2024-09-21T03:47:01.109 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:01.103+0000 7fadbe7fc640 1 --2- 172.21.15.39:0/1740393894 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7fadb40b6880 0x7fadb40b8d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:47:01.109 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:01.103+0000 7fadd5975740 1 -- 172.21.15.39:0/3022070584 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559bdb0dc1b0 msgr2=0x559bdb4ac9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:47:01.109 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:01.103+0000 7fadd5975740 1 --2- 172.21.15.39:0/3022070584 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559bdb0dc1b0 0x559bdb4ac9a0 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7fadc4009c20 tx=0x7fadc400e6c0 comp rx=0 tx=0).stop 2024-09-21T03:47:01.109 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:01.103+0000 7fadbe7fc640 1 -- 172.21.15.39:0/1740393894 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8484+0+0 (secure 0 0 0) 0x7fadb80e7750 con 0x7fadc8108200 2024-09-21T03:47:01.109 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:01.104+0000 7fadcd99e640 1 --2- 172.21.15.39:0/1740393894 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7fadb40b6880 0x7fadb40b8d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:47:01.109 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:01.104+0000 7fadcd99e640 1 --2- 172.21.15.39:0/1740393894 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7fadb40b6880 0x7fadb40b8d90 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fadc0004af0 tx=0x7fadc0004a80 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:47:01.109 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:01.104+0000 7fadd5975740 1 -- 172.21.15.39:0/3022070584 shutdown_connections 2024-09-21T03:47:01.109 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:01.104+0000 7fadd5975740 1 --2- 172.21.15.39:0/3022070584 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x559bdb0e4390 0x559bdb4acee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:01.109 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:01.104+0000 7fadd5975740 1 --2- 172.21.15.39:0/3022070584 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559bdb0dc1b0 0x559bdb4ac9a0 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:01.110 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:01.104+0000 7fadd5975740 1 --2- 172.21.15.39:0/3022070584 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559bdb105070 0x559bdb4ac460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:01.110 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:01.104+0000 7fadd5975740 1 -- 172.21.15.39:0/3022070584 >> 172.21.15.39:0/3022070584 conn(0x559bdb0f9460 msgr2=0x559bdb4aa350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:47:01.110 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:01.104+0000 7fadd5975740 1 -- 172.21.15.39:0/3022070584 shutdown_connections 2024-09-21T03:47:01.110 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:01.105+0000 7fadd5975740 1 -- 172.21.15.39:0/3022070584 wait complete. 2024-09-21T03:47:01.110 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:01.105+0000 7fadd5975740 1 -- start start 2024-09-21T03:47:01.110 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:01.105+0000 7fadd5975740 1 Processor -- start 2024-09-21T03:47:01.111 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:01.106+0000 7fadd5975740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x559bdb0dc1b0 0x559bdb4b7b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:47:01.112 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:01.106+0000 7fadd5975740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559bdb0e4390 0x559bdb4b80b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:47:01.112 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:01.106+0000 7fadd5975740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559bdb105070 0x559bdb4c3f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:47:01.112 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:01.106+0000 7fadd5975740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x559bdb273700 con 0x559bdb105070 2024-09-21T03:47:01.112 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:01.106+0000 7fadd5975740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x559bdb3ad160 con 0x559bdb0dc1b0 2024-09-21T03:47:01.112 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:01.106+0000 7fadd5975740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x559bdb4bac40 con 0x559bdb0e4390 2024-09-21T03:47:01.112 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:01.106+0000 7fadcd19d640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x559bdb0dc1b0 0x559bdb4b7b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:47:01.112 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:01.106+0000 7fadcd19d640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x559bdb0dc1b0 0x559bdb4b7b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.39:35346/0 (socket says 172.21.15.39:35346) 2024-09-21T03:47:01.112 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:01.106+0000 7fadcd19d640 1 -- 172.21.15.39:0/589719745 learned_addr learned my addr 172.21.15.39:0/589719745 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:47:01.112 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:01.106+0000 7fadd5975740 10 client.0 _ll_register_callbacks cb 0x559bdb3c69e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:47:01.112 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:01.106+0000 7fadcd99e640 1 --2- 172.21.15.39:0/589719745 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559bdb105070 0x559bdb4c3f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:47:01.112 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:01.106+0000 7fadcc99c640 1 --2- 172.21.15.39:0/589719745 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559bdb0e4390 0x559bdb4b80b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:47:01.112 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:01.106+0000 7fadcd19d640 1 -- 172.21.15.39:0/589719745 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559bdb0e4390 msgr2=0x559bdb4b80b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:47:01.113 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:01.106+0000 7fadcd19d640 1 --2- 172.21.15.39:0/589719745 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559bdb0e4390 0x559bdb4b80b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:01.113 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:01.106+0000 7fadcd19d640 1 -- 172.21.15.39:0/589719745 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559bdb105070 msgr2=0x559bdb4c3f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:47:01.113 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:01.107+0000 7fadcd19d640 1 --2- 172.21.15.39:0/589719745 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559bdb105070 0x559bdb4c3f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:01.113 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:01.107+0000 7fadcd19d640 1 -- 172.21.15.39:0/589719745 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559bdb3dac90 con 0x559bdb0dc1b0 2024-09-21T03:47:01.113 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:01.107+0000 7fadcc99c640 1 --2- 172.21.15.39:0/589719745 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559bdb0e4390 0x559bdb4b80b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:47:01.113 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:01.107+0000 7fadcd99e640 1 --2- 172.21.15.39:0/589719745 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559bdb105070 0x559bdb4c3f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:47:01.113 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:01.107+0000 7fadcd19d640 1 --2- 172.21.15.39:0/589719745 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x559bdb0dc1b0 0x559bdb4b7b70 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fadc40099c0 tx=0x7fadc400ef60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2fc8276e3c09641b server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:47:01.113 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:01.107+0000 7fadb0ff9640 10 client.0 ms_handle_connect on v2:172.21.15.150:3300/0 2024-09-21T03:47:01.113 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:01.107+0000 7fadb0ff9640 1 -- 172.21.15.39:0/589719745 <== mon.2 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadc4028060 con 0x559bdb0dc1b0 2024-09-21T03:47:01.113 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:01.107+0000 7fadb0ff9640 1 -- 172.21.15.39:0/589719745 <== mon.2 v2:172.21.15.150:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8484+0+0 (secure 0 0 0) 0x7fadc4004210 con 0x559bdb0dc1b0 2024-09-21T03:47:01.113 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:01.107+0000 7fadd5975740 1 -- 172.21.15.39:0/589719745 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fadc40035f0 con 0x559bdb0dc1b0 2024-09-21T03:47:01.113 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:01.107+0000 7fadb0ff9640 1 -- 172.21.15.39:0/589719745 <== mon.2 v2:172.21.15.150:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fadc4016040 con 0x559bdb0dc1b0 2024-09-21T03:47:01.113 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:01.107+0000 7fadd5975740 10 client.25325 fetch_fsmap learned FSMap version 29 2024-09-21T03:47:01.113 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:01.107+0000 7fadd5975740 1 -- 172.21.15.39:0/589719745 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x559bdb4c6960 con 0x559bdb0dc1b0 2024-09-21T03:47:01.113 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:01.108+0000 7fadb0ff9640 1 -- 172.21.15.39:0/589719745 <== mon.2 v2:172.21.15.150:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fadc4004410 con 0x559bdb0dc1b0 2024-09-21T03:47:01.114 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:01.108+0000 7fadb0ff9640 1 -- 172.21.15.39:0/589719745 <== mon.2 v2:172.21.15.150:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fadc400f040 con 0x559bdb0dc1b0 2024-09-21T03:47:01.114 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:01.108+0000 7fadd5975740 10 client.25325 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T03:47:01.114 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:01.108+0000 7fadd5975740 10 client.25325 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T03:47:01.114 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:01.108+0000 7fadd5975740 20 client.25325 populate_metadata read hostname 'smithi039' 2024-09-21T03:47:01.114 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:01.108+0000 7fadd5975740 1 --2- 172.21.15.39:0/589719745 >> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] conn(0x559bdb0c39f0 0x559bdb4c9cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:47:01.114 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:01.108+0000 7fadd5975740 4 client.25325 mds_command: new command op to 24695 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:47:01.114 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:01.108+0000 7fadd5975740 1 -- 172.21.15.39:0/589719745 --> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fadc40035f0 con 0x559bdb0c39f0 2024-09-21T03:47:01.114 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:01.109+0000 7fadcc99c640 1 --2- 172.21.15.39:0/589719745 >> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] conn(0x559bdb0c39f0 0x559bdb4c9cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:47:01.115 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:01.109+0000 7fadb0ff9640 10 client.25325 ms_handle_connect on v2:172.21.15.150:6838/2949386907 2024-09-21T03:47:01.115 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:01.109+0000 7fadcc99c640 1 --2- 172.21.15.39:0/589719745 >> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] conn(0x559bdb0c39f0 0x559bdb4c9cc0 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e3b9eea2f8e01fd8 server_cookie=506925250c6804c6 in_seq=0 out_seq=0 2024-09-21T03:47:01.115 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:01.110+0000 7fadb0ff9640 1 -- 172.21.15.39:0/589719745 <== mds.0 v2:172.21.15.150:6838/2949386907 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x7fadc40035f0 con 0x559bdb0c39f0 2024-09-21T03:47:01.115 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:01.110+0000 7fadb0ff9640 10 client.25325 handle_command_reply: tid=0 2024-09-21T03:47:01.115 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:01.110+0000 7fadd5975740 1 client.25325 shutdown 2024-09-21T03:47:01.115 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:01.110+0000 7fadd5975740 1 -- 172.21.15.39:0/589719745 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x559bdb0dc1b0 msgr2=0x559bdb4b7b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:47:01.115 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:01.110+0000 7fadd5975740 1 --2- 172.21.15.39:0/589719745 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x559bdb0dc1b0 0x559bdb4b7b70 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fadc40099c0 tx=0x7fadc400ef60 comp rx=0 tx=0).stop 2024-09-21T03:47:01.115 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:01.111+0000 7fadd5975740 1 -- 172.21.15.39:0/589719745 shutdown_connections 2024-09-21T03:47:01.115 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:01.111+0000 7fadd5975740 1 --2- 172.21.15.39:0/589719745 >> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] conn(0x559bdb0c39f0 0x559bdb4c9cc0 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:01.116 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:01.111+0000 7fadd5975740 1 --2- 172.21.15.39:0/589719745 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559bdb105070 0x559bdb4c3f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:01.116 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:01.111+0000 7fadd5975740 1 --2- 172.21.15.39:0/589719745 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559bdb0e4390 0x559bdb4b80b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:01.116 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:01.111+0000 7fadb0ff9640 0 client.25325 ms_handle_reset on v2:172.21.15.150:6838/2949386907 2024-09-21T03:47:01.116 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:01.111+0000 7fadd5975740 1 --2- 172.21.15.39:0/589719745 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x559bdb0dc1b0 0x559bdb4b7b70 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:01.116 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:01.111+0000 7fadd5975740 1 -- 172.21.15.39:0/589719745 >> 172.21.15.39:0/589719745 conn(0x559bdb0f9460 msgr2=0x559bdb4af5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:47:01.116 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:01.111+0000 7fadd5975740 1 -- 172.21.15.39:0/589719745 shutdown_connections 2024-09-21T03:47:01.116 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:01.111+0000 7fadd5975740 1 -- 172.21.15.39:0/589719745 wait complete. 2024-09-21T03:47:01.116 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:01.111+0000 7fadd5975740 20 client.25325 trim_cache size 0 max 16384 2024-09-21T03:47:01.139 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:01.136+0000 7fadd5975740 1 Processor -- start 2024-09-21T03:47:01.140 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:01.136+0000 7fadd5975740 1 -- start start 2024-09-21T03:47:01.141 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:01.137+0000 7fadd5975740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559bdb0dc1b0 0x559bdb4b6a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:47:01.141 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:01.137+0000 7fadd5975740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x559bdb0e4390 0x559bdb4adba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:47:01.141 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:01.137+0000 7fadd5975740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559bdb105070 0x559bdb4c3ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:47:01.141 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:01.137+0000 7fadcd99e640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559bdb105070 0x559bdb4c3ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:47:01.141 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:01.137+0000 7fadcd99e640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559bdb105070 0x559bdb4c3ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:52208/0 (socket says 172.21.15.39:52208) 2024-09-21T03:47:01.141 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:01.137+0000 7fadcd99e640 1 -- 172.21.15.39:0/2002136748 learned_addr learned my addr 172.21.15.39:0/2002136748 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:47:01.141 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:01.137+0000 7fadd5975740 1 -- 172.21.15.39:0/2002136748 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fadc81b5cd0 con 0x559bdb105070 2024-09-21T03:47:01.141 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:01.137+0000 7fadd5975740 1 -- 172.21.15.39:0/2002136748 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fadc81b7810 con 0x559bdb0e4390 2024-09-21T03:47:01.142 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:01.137+0000 7fadd5975740 1 -- 172.21.15.39:0/2002136748 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fadc81b63a0 con 0x559bdb0dc1b0 2024-09-21T03:47:01.142 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:01.137+0000 7fadcc99c640 1 --2- 172.21.15.39:0/2002136748 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x559bdb0e4390 0x559bdb4adba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:47:01.142 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:01.137+0000 7fadcd19d640 1 --2- 172.21.15.39:0/2002136748 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559bdb0dc1b0 0x559bdb4b6a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:47:01.142 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:01.138+0000 7fadcd99e640 1 -- 172.21.15.39:0/2002136748 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559bdb0dc1b0 msgr2=0x559bdb4b6a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:47:01.142 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:01.138+0000 7fadcd99e640 1 --2- 172.21.15.39:0/2002136748 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559bdb0dc1b0 0x559bdb4b6a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:01.142 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:01.138+0000 7fadcd99e640 1 -- 172.21.15.39:0/2002136748 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x559bdb0e4390 msgr2=0x559bdb4adba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:47:01.142 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:01.138+0000 7fadcd99e640 1 --2- 172.21.15.39:0/2002136748 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x559bdb0e4390 0x559bdb4adba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:01.142 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:01.138+0000 7fadcd99e640 1 -- 172.21.15.39:0/2002136748 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559bdb4b86d0 con 0x559bdb105070 2024-09-21T03:47:01.142 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:01.138+0000 7fadcd99e640 1 --2- 172.21.15.39:0/2002136748 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559bdb105070 0x559bdb4c3ed0 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7fadc0035000 tx=0x7fadc0040000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dcd3ae882785084 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:47:01.143 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:01.139+0000 7fadb2ffd640 1 -- 172.21.15.39:0/2002136748 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadc00318a0 con 0x559bdb105070 2024-09-21T03:47:01.143 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:01.139+0000 7fadb2ffd640 1 -- 172.21.15.39:0/2002136748 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fadc0036040 con 0x559bdb105070 2024-09-21T03:47:01.143 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:01.139+0000 7fadb2ffd640 1 -- 172.21.15.39:0/2002136748 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadc002f610 con 0x559bdb105070 2024-09-21T03:47:01.143 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:01.140+0000 7fadd5975740 1 -- 172.21.15.39:0/2002136748 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559bdb105070 msgr2=0x559bdb4c3ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:47:01.143 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:01.140+0000 7fadd5975740 1 --2- 172.21.15.39:0/2002136748 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559bdb105070 0x559bdb4c3ed0 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7fadc0035000 tx=0x7fadc0040000 comp rx=0 tx=0).stop 2024-09-21T03:47:01.143 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:01.140+0000 7fadd5975740 1 -- 172.21.15.39:0/2002136748 shutdown_connections 2024-09-21T03:47:01.143 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:01.140+0000 7fadd5975740 1 --2- 172.21.15.39:0/2002136748 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559bdb105070 0x559bdb4c3ed0 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:01.143 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:01.140+0000 7fadd5975740 1 --2- 172.21.15.39:0/2002136748 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x559bdb0e4390 0x559bdb4adba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:01.143 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:01.140+0000 7fadd5975740 1 --2- 172.21.15.39:0/2002136748 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559bdb0dc1b0 0x559bdb4b6a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:01.144 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:01.140+0000 7fadd5975740 1 -- 172.21.15.39:0/2002136748 >> 172.21.15.39:0/2002136748 conn(0x559bdb0f9460 msgr2=0x559bdb4ad490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:47:01.144 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:01.140+0000 7fadd5975740 1 -- 172.21.15.39:0/2002136748 shutdown_connections 2024-09-21T03:47:01.144 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:01.140+0000 7fadd5975740 1 -- 172.21.15.39:0/2002136748 wait complete. 2024-09-21T03:47:01.144 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:01.141+0000 7fadd5975740 1 -- start start 2024-09-21T03:47:01.145 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:01.141+0000 7fadd5975740 1 Processor -- start 2024-09-21T03:47:01.145 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:01.141+0000 7fadd5975740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x559bdb0dc1b0 0x559bdb4cb130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:47:01.145 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:01.142+0000 7fadd5975740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559bdb0e4390 0x559bdb4cb6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:47:01.145 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:01.142+0000 7fadd5975740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559bdb105070 0x559bdb533440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:47:01.145 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:01.142+0000 7fadd5975740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fadc81b7140 con 0x559bdb105070 2024-09-21T03:47:01.145 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:01.142+0000 7fadd5975740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fadc81b6a70 con 0x559bdb0dc1b0 2024-09-21T03:47:01.145 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:01.142+0000 7fadd5975740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fadc81b3cb0 con 0x559bdb0e4390 2024-09-21T03:47:01.145 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:01.142+0000 7fadd5975740 10 client.0 _ll_register_callbacks cb 0x7fadc4009aa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:47:01.145 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:01.142+0000 7fadcd19d640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x559bdb0dc1b0 0x559bdb4cb130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:47:01.145 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:01.142+0000 7fadcd19d640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x559bdb0dc1b0 0x559bdb4cb130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.39:35374/0 (socket says 172.21.15.39:35374) 2024-09-21T03:47:01.145 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:01.142+0000 7fadcd19d640 1 -- 172.21.15.39:0/2604704563 learned_addr learned my addr 172.21.15.39:0/2604704563 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:47:01.145 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:01.142+0000 7fadcc99c640 1 --2- 172.21.15.39:0/2604704563 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559bdb0e4390 0x559bdb4cb6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:47:01.147 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:01.142+0000 7fadcd19d640 1 -- 172.21.15.39:0/2604704563 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559bdb0e4390 msgr2=0x559bdb4cb6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:47:01.147 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:01.142+0000 7fadcd19d640 1 --2- 172.21.15.39:0/2604704563 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559bdb0e4390 0x559bdb4cb6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:01.147 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:01.142+0000 7fadcd19d640 1 -- 172.21.15.39:0/2604704563 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559bdb105070 msgr2=0x559bdb533440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:47:01.147 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:01.142+0000 7fadcd19d640 1 --2- 172.21.15.39:0/2604704563 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559bdb105070 0x559bdb533440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:01.147 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:01.142+0000 7fadcd19d640 1 -- 172.21.15.39:0/2604704563 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559bdb533980 con 0x559bdb0dc1b0 2024-09-21T03:47:01.148 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:01.142+0000 7fadcc99c640 1 --2- 172.21.15.39:0/2604704563 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559bdb0e4390 0x559bdb4cb6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:47:01.148 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:01.143+0000 7fadcd19d640 1 --2- 172.21.15.39:0/2604704563 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x559bdb0dc1b0 0x559bdb4cb130 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fadc400ec40 tx=0x7fadc4023930 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6542ddb1fcc7db91 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:47:01.148 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:01.143+0000 7fadb0ff9640 10 client.0 ms_handle_connect on v2:172.21.15.150:3300/0 2024-09-21T03:47:01.148 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:01.143+0000 7fadb0ff9640 1 -- 172.21.15.39:0/2604704563 <== mon.2 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadc4028040 con 0x559bdb0dc1b0 2024-09-21T03:47:01.148 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:01.143+0000 7fadb0ff9640 1 -- 172.21.15.39:0/2604704563 <== mon.2 v2:172.21.15.150:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8484+0+0 (secure 0 0 0) 0x7fadc4023d30 con 0x559bdb0dc1b0 2024-09-21T03:47:01.148 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:01.143+0000 7fadd5975740 1 -- 172.21.15.39:0/2604704563 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fadc0036040 con 0x559bdb0dc1b0 2024-09-21T03:47:01.148 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:01.143+0000 7fadb0ff9640 1 -- 172.21.15.39:0/2604704563 <== mon.2 v2:172.21.15.150:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fadc40035f0 con 0x559bdb0dc1b0 2024-09-21T03:47:01.148 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:01.143+0000 7fadd5975740 10 client.25331 fetch_fsmap learned FSMap version 29 2024-09-21T03:47:01.148 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:01.143+0000 7fadd5975740 1 -- 172.21.15.39:0/2604704563 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x559bdb534b10 con 0x559bdb0dc1b0 2024-09-21T03:47:01.148 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:01.143+0000 7fadb0ff9640 1 -- 172.21.15.39:0/2604704563 <== mon.2 v2:172.21.15.150:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fadc40094c0 con 0x559bdb0dc1b0 2024-09-21T03:47:01.148 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:01.143+0000 7fadb0ff9640 1 -- 172.21.15.39:0/2604704563 <== mon.2 v2:172.21.15.150:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fadc4004940 con 0x559bdb0dc1b0 2024-09-21T03:47:01.148 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:01.144+0000 7fadd5975740 10 client.25331 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T03:47:01.148 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:01.144+0000 7fadd5975740 10 client.25331 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T03:47:01.148 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:01.144+0000 7fadd5975740 20 client.25331 populate_metadata read hostname 'smithi039' 2024-09-21T03:47:01.149 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:01.144+0000 7fadd5975740 1 --2- 172.21.15.39:0/2604704563 >> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] conn(0x559bdb098da0 0x559bdb537190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:47:01.149 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:01.144+0000 7fadd5975740 4 client.25331 mds_command: new command op to 24695 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T03:47:01.149 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:01.144+0000 7fadd5975740 1 -- 172.21.15.39:0/2604704563 --> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7fadc0036040 con 0x559bdb098da0 2024-09-21T03:47:01.149 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:01.144+0000 7fadcc99c640 1 --2- 172.21.15.39:0/2604704563 >> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] conn(0x559bdb098da0 0x559bdb537190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:47:01.149 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:01.144+0000 7fadb0ff9640 10 client.25331 ms_handle_connect on v2:172.21.15.150:6838/2949386907 2024-09-21T03:47:01.150 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:01.145+0000 7fadcc99c640 1 --2- 172.21.15.39:0/2604704563 >> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] conn(0x559bdb098da0 0x559bdb537190 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f66a20d2097cc199 server_cookie=79b9f0d1d5fcd7cf in_seq=0 out_seq=0 2024-09-21T03:47:01.150 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:01.145+0000 7fadb0ff9640 1 -- 172.21.15.39:0/2604704563 <== mds.0 v2:172.21.15.150:6838/2949386907 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7fadc0036040 con 0x559bdb098da0 2024-09-21T03:47:01.150 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:01.145+0000 7fadb0ff9640 10 client.25331 handle_command_reply: tid=0 2024-09-21T03:47:01.150 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:01.145+0000 7fadd5975740 1 client.25331 shutdown 2024-09-21T03:47:01.150 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:01.146+0000 7fadd5975740 1 -- 172.21.15.39:0/2604704563 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x559bdb0dc1b0 msgr2=0x559bdb4cb130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:47:01.150 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:01.146+0000 7fadd5975740 1 --2- 172.21.15.39:0/2604704563 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x559bdb0dc1b0 0x559bdb4cb130 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fadc400ec40 tx=0x7fadc4023930 comp rx=0 tx=0).stop 2024-09-21T03:47:01.150 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:01.146+0000 7fadd5975740 1 -- 172.21.15.39:0/2604704563 shutdown_connections 2024-09-21T03:47:01.150 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:01.146+0000 7fadd5975740 1 --2- 172.21.15.39:0/2604704563 >> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] conn(0x559bdb098da0 0x559bdb537190 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:01.150 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:01.146+0000 7fadd5975740 1 --2- 172.21.15.39:0/2604704563 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559bdb105070 0x559bdb533440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:01.150 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:01.146+0000 7fadd5975740 1 --2- 172.21.15.39:0/2604704563 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559bdb0e4390 0x559bdb4cb6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:01.151 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:01.146+0000 7fadd5975740 1 --2- 172.21.15.39:0/2604704563 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x559bdb0dc1b0 0x559bdb4cb130 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:01.151 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:01.146+0000 7fadd5975740 1 -- 172.21.15.39:0/2604704563 >> 172.21.15.39:0/2604704563 conn(0x559bdb0f9460 msgr2=0x559bdb4b9020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:47:01.151 INFO:teuthology.orchestra.run.smithi039.stdout:[] 2024-09-21T03:47:01.152 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:01.146+0000 7fadb0ff9640 0 client.25331 ms_handle_reset on v2:172.21.15.150:6838/2949386907 2024-09-21T03:47:01.152 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:01.146+0000 7fadd5975740 1 -- 172.21.15.39:0/2604704563 shutdown_connections 2024-09-21T03:47:01.152 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:01.146+0000 7fadd5975740 1 -- 172.21.15.39:0/2604704563 wait complete. 2024-09-21T03:47:01.152 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:01.146+0000 7fadd5975740 20 client.25331 trim_cache size 0 max 16384 2024-09-21T03:47:01.152 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:01.147+0000 7fadce19f640 1 -- 172.21.15.39:0/1740393894 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7fadb40b6880 msgr2=0x7fadb40b8d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:47:01.152 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:01.147+0000 7fadce19f640 1 --2- 172.21.15.39:0/1740393894 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7fadb40b6880 0x7fadb40b8d90 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fadc0004af0 tx=0x7fadc0004a80 comp rx=0 tx=0).stop 2024-09-21T03:47:01.152 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:01.147+0000 7fadce19f640 1 -- 172.21.15.39:0/1740393894 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fadc8108200 msgr2=0x7fadc80fe880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:47:01.152 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:01.147+0000 7fadce19f640 1 --2- 172.21.15.39:0/1740393894 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fadc8108200 0x7fadc80fe880 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7fadb800e9c0 tx=0x7fadb800ee80 comp rx=0 tx=0).stop 2024-09-21T03:47:01.152 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:01.147+0000 7fadce19f640 1 -- 172.21.15.39:0/1740393894 shutdown_connections 2024-09-21T03:47:01.153 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:01.147+0000 7fadce19f640 1 --2- 172.21.15.39:0/1740393894 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7fadb40b6880 0x7fadb40b8d90 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:01.153 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:01.147+0000 7fadce19f640 1 --2- 172.21.15.39:0/1740393894 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fadc8108b50 0x7fadc80fedc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:01.153 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:01.147+0000 7fadce19f640 1 --2- 172.21.15.39:0/1740393894 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fadc8108200 0x7fadc80fe880 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:01.153 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:01.147+0000 7fadce19f640 1 --2- 172.21.15.39:0/1740393894 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fadc81028c0 0x7fadc8100290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:01.153 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:01.147+0000 7fadce19f640 1 -- 172.21.15.39:0/1740393894 >> 172.21.15.39:0/1740393894 conn(0x7fadc80fe350 msgr2=0x7fadc8100970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:47:01.153 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:01.148+0000 7fadce19f640 1 -- 172.21.15.39:0/1740393894 shutdown_connections 2024-09-21T03:47:01.153 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:01.148+0000 7fadce19f640 1 -- 172.21.15.39:0/1740393894 wait complete. 2024-09-21T03:47:02.917 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-21T03:47:02.917 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 51116d5c-77ca-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-21T03:47:02.950 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:47:02 smithi039 ceph-mon[31211]: pgmap v355: 97 pgs: 97 active+clean; 1.4 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 682 B/s wr, 9 op/s 2024-09-21T03:47:02.950 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:47:02 smithi039 ceph-mon[31211]: mgrmap e26: x(active, since 9m), standbys: z, y 2024-09-21T03:47:02.968 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:47:02 smithi161 ceph-mon[34533]: pgmap v355: 97 pgs: 97 active+clean; 1.4 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 682 B/s wr, 9 op/s 2024-09-21T03:47:02.968 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:47:02 smithi161 ceph-mon[34533]: mgrmap e26: x(active, since 9m), standbys: z, y 2024-09-21T03:47:02.974 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:47:02 smithi150 ceph-mon[34289]: pgmap v355: 97 pgs: 97 active+clean; 1.4 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 682 B/s wr, 9 op/s 2024-09-21T03:47:02.974 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:47:02 smithi150 ceph-mon[34289]: mgrmap e26: x(active, since 9m), standbys: z, y 2024-09-21T03:47:03.317 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/51116d5c-77ca-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:47:03.906 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:03.903+0000 7f1154bf1640 1 -- 172.21.15.39:0/1779485892 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1150109e00 msgr2=0x7f1150110630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:47:03.906 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:03.903+0000 7f1154bf1640 1 --2- 172.21.15.39:0/1779485892 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1150109e00 0x7f1150110630 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f1148009ca0 tx=0x7f114802a8c0 comp rx=0 tx=0).stop 2024-09-21T03:47:03.906 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:03.903+0000 7f1154bf1640 1 -- 172.21.15.39:0/1779485892 shutdown_connections 2024-09-21T03:47:03.907 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:03.903+0000 7f1154bf1640 1 --2- 172.21.15.39:0/1779485892 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1150109e00 0x7f1150110630 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:03.907 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:03.903+0000 7f1154bf1640 1 --2- 172.21.15.39:0/1779485892 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f11501094b0 0x7f11501098c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:03.907 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:03.903+0000 7f1154bf1640 1 --2- 172.21.15.39:0/1779485892 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1150101f70 0x7f1150102380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:03.907 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:03.903+0000 7f1154bf1640 1 -- 172.21.15.39:0/1779485892 >> 172.21.15.39:0/1779485892 conn(0x7f1150077c60 msgr2=0x7f11500780a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:47:03.907 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:03.903+0000 7f1154bf1640 1 -- 172.21.15.39:0/1779485892 shutdown_connections 2024-09-21T03:47:03.907 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:03.903+0000 7f1154bf1640 1 -- 172.21.15.39:0/1779485892 wait complete. 2024-09-21T03:47:03.908 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:03.904+0000 7f1154bf1640 1 Processor -- start 2024-09-21T03:47:03.908 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:03.904+0000 7f1154bf1640 1 -- start start 2024-09-21T03:47:03.908 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:03.904+0000 7f1154bf1640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1150101f70 0x7f115019a4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:47:03.908 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:03.905+0000 7f114e59c640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1150101f70 0x7f115019a4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:47:03.908 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:03.905+0000 7f114e59c640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1150101f70 0x7f115019a4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:52226/0 (socket says 172.21.15.39:52226) 2024-09-21T03:47:03.908 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:03.905+0000 7f1154bf1640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f11501094b0 0x7f115019aa90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:47:03.908 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:03.905+0000 7f114e59c640 1 -- 172.21.15.39:0/2598719099 learned_addr learned my addr 172.21.15.39:0/2598719099 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:47:03.908 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:03.905+0000 7f1154bf1640 1 --2- 172.21.15.39:0/2598719099 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1150109e00 0x7f115019fd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:47:03.908 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:03.905+0000 7f1154bf1640 1 -- 172.21.15.39:0/2598719099 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f1150113ac0 con 0x7f1150101f70 2024-09-21T03:47:03.908 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:03.905+0000 7f1154bf1640 1 -- 172.21.15.39:0/2598719099 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f1150113940 con 0x7f1150109e00 2024-09-21T03:47:03.909 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:03.905+0000 7f1154bf1640 1 -- 172.21.15.39:0/2598719099 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f1150113c40 con 0x7f11501094b0 2024-09-21T03:47:03.909 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:03.905+0000 7f114dd9b640 1 --2- 172.21.15.39:0/2598719099 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f11501094b0 0x7f115019aa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:47:03.909 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:03.905+0000 7f114dd9b640 1 -- 172.21.15.39:0/2598719099 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1150109e00 msgr2=0x7f115019fd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:47:03.909 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:03.906+0000 7f114dd9b640 1 --2- 172.21.15.39:0/2598719099 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1150109e00 0x7f115019fd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:03.909 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:03.906+0000 7f114dd9b640 1 -- 172.21.15.39:0/2598719099 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1150101f70 msgr2=0x7f115019a4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:47:03.910 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:03.906+0000 7f114dd9b640 1 --2- 172.21.15.39:0/2598719099 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1150101f70 0x7f115019a4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:03.910 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:03.906+0000 7f114dd9b640 1 -- 172.21.15.39:0/2598719099 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f11501a0460 con 0x7f11501094b0 2024-09-21T03:47:03.910 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:03.906+0000 7f114e59c640 1 --2- 172.21.15.39:0/2598719099 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1150101f70 0x7f115019a4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:47:03.910 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:03.906+0000 7f114dd9b640 1 --2- 172.21.15.39:0/2598719099 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f11501094b0 0x7f115019aa90 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f1150075ab0 tx=0x7f114400ee50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:47:03.910 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:03.906+0000 7f113f7fe640 1 -- 172.21.15.39:0/2598719099 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f114400cc40 con 0x7f11501094b0 2024-09-21T03:47:03.910 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:03.906+0000 7f113f7fe640 1 -- 172.21.15.39:0/2598719099 <== mon.1 v2:172.21.15.161:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f114400cde0 con 0x7f11501094b0 2024-09-21T03:47:03.910 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:03.906+0000 7f113f7fe640 1 -- 172.21.15.39:0/2598719099 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11440187f0 con 0x7f11501094b0 2024-09-21T03:47:03.910 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:03.907+0000 7f1154bf1640 1 -- 172.21.15.39:0/2598719099 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f11501a0750 con 0x7f11501094b0 2024-09-21T03:47:03.911 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:03.908+0000 7f1154bf1640 1 -- 172.21.15.39:0/2598719099 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f11501a0b50 con 0x7f11501094b0 2024-09-21T03:47:03.914 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:03.911+0000 7f115b165740 1 Processor -- start 2024-09-21T03:47:03.915 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:03.912+0000 7f115b165740 1 -- start start 2024-09-21T03:47:03.915 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:03.912+0000 7f113f7fe640 1 -- 172.21.15.39:0/2598719099 <== mon.1 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 26) ==== 154694+0+0 (secure 0 0 0) 0x7f114401f020 con 0x7f11501094b0 2024-09-21T03:47:03.915 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:03.912+0000 7f115b165740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5572d450a350 0x5572d48b1430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:47:03.915 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:03.912+0000 7f115b165740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5572d44e1490 0x5572d48b1970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:47:03.915 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:03.912+0000 7f115b165740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5572d44e9670 0x5572d48b1eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:47:03.915 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:03.912+0000 7f114e59c640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5572d44e1490 0x5572d48b1970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:47:03.916 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:03.912+0000 7f114e59c640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5572d44e1490 0x5572d48b1970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.39:35398/0 (socket says 172.21.15.39:35398) 2024-09-21T03:47:03.916 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:03.912+0000 7f114ed9d640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5572d450a350 0x5572d48b1430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:47:03.916 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:03.913+0000 7f114e59c640 1 -- 172.21.15.39:0/942120780 learned_addr learned my addr 172.21.15.39:0/942120780 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:47:03.916 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:03.913+0000 7f115b165740 1 -- 172.21.15.39:0/942120780 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x5572d4827e60 con 0x5572d450a350 2024-09-21T03:47:03.916 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:03.913+0000 7f114ed9d640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5572d450a350 0x5572d48b1430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:52242/0 (socket says 172.21.15.39:52242) 2024-09-21T03:47:03.916 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:03.913+0000 7f115b165740 1 -- 172.21.15.39:0/942120780 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x5572d4817290 con 0x5572d44e1490 2024-09-21T03:47:03.918 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:03.913+0000 7f115b165740 1 -- 172.21.15.39:0/942120780 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x5572d481b0a0 con 0x5572d44e9670 2024-09-21T03:47:03.918 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:03.913+0000 7f114dd9b640 1 --2- 172.21.15.39:0/942120780 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5572d44e9670 0x5572d48b1eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:47:03.921 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:03.913+0000 7f114e59c640 1 -- 172.21.15.39:0/942120780 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5572d44e9670 msgr2=0x5572d48b1eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:47:03.921 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:03.913+0000 7f114e59c640 1 --2- 172.21.15.39:0/942120780 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5572d44e9670 0x5572d48b1eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:03.921 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:03.913+0000 7f114e59c640 1 -- 172.21.15.39:0/942120780 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5572d450a350 msgr2=0x5572d48b1430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:47:03.921 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:03.913+0000 7f113f7fe640 1 --2- 172.21.15.39:0/2598719099 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f11380b6790 0x7f11380b8ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:47:03.921 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:03.914+0000 7f113f7fe640 1 -- 172.21.15.39:0/2598719099 <== mon.1 v2:172.21.15.161:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8484+0+0 (secure 0 0 0) 0x7f11440e7920 con 0x7f11501094b0 2024-09-21T03:47:03.921 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:03.914+0000 7f114e59c640 1 --2- 172.21.15.39:0/942120780 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5572d450a350 0x5572d48b1430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:03.921 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:03.914+0000 7f114e59c640 1 -- 172.21.15.39:0/942120780 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5572d4575930 con 0x5572d44e1490 2024-09-21T03:47:03.921 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:03.914+0000 7f114dd9b640 1 --2- 172.21.15.39:0/942120780 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5572d44e9670 0x5572d48b1eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:47:03.921 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:03.914+0000 7f114ed9d640 1 --2- 172.21.15.39:0/942120780 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5572d450a350 0x5572d48b1430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:47:03.921 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:03.914+0000 7f114ed9d640 1 --2- 172.21.15.39:0/2598719099 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f11380b6790 0x7f11380b8ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:47:03.922 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:03.914+0000 7f114e59c640 1 --2- 172.21.15.39:0/942120780 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5572d44e1490 0x5572d48b1970 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f1140010580 tx=0x7f1140010a40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=17d88bd534270654 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:47:03.922 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:03.914+0000 7f113cff9640 1 -- 172.21.15.39:0/942120780 <== mon.2 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f114000e890 con 0x5572d44e1490 2024-09-21T03:47:03.922 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:03.915+0000 7f113cff9640 1 -- 172.21.15.39:0/942120780 <== mon.2 v2:172.21.15.150:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f114001c480 con 0x5572d44e1490 2024-09-21T03:47:03.922 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:03.915+0000 7f113cff9640 1 -- 172.21.15.39:0/942120780 <== mon.2 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11400116a0 con 0x5572d44e1490 2024-09-21T03:47:03.922 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:03.916+0000 7f115b165740 1 -- 172.21.15.39:0/942120780 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5572d44e1490 msgr2=0x5572d48b1970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:47:03.922 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:03.916+0000 7f115b165740 1 --2- 172.21.15.39:0/942120780 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5572d44e1490 0x5572d48b1970 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f1140010580 tx=0x7f1140010a40 comp rx=0 tx=0).stop 2024-09-21T03:47:03.922 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:03.916+0000 7f114ed9d640 1 --2- 172.21.15.39:0/2598719099 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f11380b6790 0x7f11380b8ca0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f1148005e30 tx=0x7f11480095c0 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:47:03.922 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:03.916+0000 7f115b165740 1 -- 172.21.15.39:0/942120780 shutdown_connections 2024-09-21T03:47:03.922 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:03.916+0000 7f115b165740 1 --2- 172.21.15.39:0/942120780 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5572d44e9670 0x5572d48b1eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:03.922 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:03.916+0000 7f115b165740 1 --2- 172.21.15.39:0/942120780 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5572d44e1490 0x5572d48b1970 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:03.922 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:03.916+0000 7f115b165740 1 --2- 172.21.15.39:0/942120780 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5572d450a350 0x5572d48b1430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:03.922 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:03.916+0000 7f115b165740 1 -- 172.21.15.39:0/942120780 >> 172.21.15.39:0/942120780 conn(0x5572d44fe740 msgr2=0x5572d48af320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:47:03.922 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:03.916+0000 7f115b165740 1 -- 172.21.15.39:0/942120780 shutdown_connections 2024-09-21T03:47:03.923 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:03.916+0000 7f115b165740 1 -- 172.21.15.39:0/942120780 wait complete. 2024-09-21T03:47:03.923 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:03.917+0000 7f115b165740 1 -- start start 2024-09-21T03:47:03.923 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:03.917+0000 7f115b165740 1 Processor -- start 2024-09-21T03:47:03.923 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:03.917+0000 7f115b165740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5572d44e1490 0x5572d48b8f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:47:03.923 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:03.917+0000 7f115b165740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5572d44e9670 0x5572d48b9440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:47:03.923 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:03.917+0000 7f115b165740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5572d450a350 0x5572d48d16d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:47:03.923 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:03.917+0000 7f115b165740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x5572d4678890 con 0x5572d44e9670 2024-09-21T03:47:03.923 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:03.917+0000 7f115b165740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x5572d47b1d40 con 0x5572d450a350 2024-09-21T03:47:03.923 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:03.917+0000 7f115b165740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x5572d48c83a0 con 0x5572d44e1490 2024-09-21T03:47:03.923 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:03.917+0000 7f115b165740 10 client.0 _ll_register_callbacks cb 0x5572d4815c80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:47:03.923 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:03.918+0000 7f114dd9b640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5572d44e9670 0x5572d48b9440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:47:03.923 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:03.918+0000 7f114e59c640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5572d44e1490 0x5572d48b8f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:47:03.926 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:03.918+0000 7f114dd9b640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5572d44e9670 0x5572d48b9440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:52246/0 (socket says 172.21.15.39:52246) 2024-09-21T03:47:03.926 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:03.918+0000 7f114dd9b640 1 -- 172.21.15.39:0/4214399716 learned_addr learned my addr 172.21.15.39:0/4214399716 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:47:03.926 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:03.918+0000 7f114ed9d640 1 --2- 172.21.15.39:0/4214399716 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5572d450a350 0x5572d48d16d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:47:03.927 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:03.918+0000 7f114dd9b640 1 -- 172.21.15.39:0/4214399716 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5572d44e1490 msgr2=0x5572d48b8f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:47:03.927 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:03.918+0000 7f114dd9b640 1 --2- 172.21.15.39:0/4214399716 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5572d44e1490 0x5572d48b8f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:03.927 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:03.918+0000 7f114dd9b640 1 -- 172.21.15.39:0/4214399716 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5572d450a350 msgr2=0x5572d48d16d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:47:03.927 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:03.918+0000 7f114dd9b640 1 --2- 172.21.15.39:0/4214399716 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5572d450a350 0x5572d48d16d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:03.927 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:03.918+0000 7f114dd9b640 1 -- 172.21.15.39:0/4214399716 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5572d48d1c10 con 0x5572d44e9670 2024-09-21T03:47:03.927 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:03.918+0000 7f114dd9b640 1 --2- 172.21.15.39:0/4214399716 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5572d44e9670 0x5572d48b9440 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f11440ad320 tx=0x7f11440e55a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d0fc49740286fe63 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:47:03.927 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:03.918+0000 7f1135ffb640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-09-21T03:47:03.927 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:03.918+0000 7f1135ffb640 1 -- 172.21.15.39:0/4214399716 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11440acf70 con 0x5572d44e9670 2024-09-21T03:47:03.927 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:03.919+0000 7f1135ffb640 1 -- 172.21.15.39:0/4214399716 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8484+0+0 (secure 0 0 0) 0x7f11440ac880 con 0x5572d44e9670 2024-09-21T03:47:03.927 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:03.919+0000 7f115b165740 1 -- 172.21.15.39:0/4214399716 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f114001c480 con 0x5572d44e9670 2024-09-21T03:47:03.927 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:03.919+0000 7f1135ffb640 1 -- 172.21.15.39:0/4214399716 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f11440acd90 con 0x5572d44e9670 2024-09-21T03:47:03.927 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:03.919+0000 7f115b165740 10 client.15741 fetch_fsmap learned FSMap version 29 2024-09-21T03:47:03.928 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:03.919+0000 7f115b165740 1 -- 172.21.15.39:0/4214399716 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5572d48d44f0 con 0x5572d44e9670 2024-09-21T03:47:03.928 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:03.919+0000 7f1135ffb640 1 -- 172.21.15.39:0/4214399716 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f11440ad130 con 0x5572d44e9670 2024-09-21T03:47:03.928 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:03.919+0000 7f1135ffb640 1 -- 172.21.15.39:0/4214399716 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f11440ac2f0 con 0x5572d44e9670 2024-09-21T03:47:03.928 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:03.919+0000 7f115b165740 10 client.15741 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T03:47:03.928 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:03.919+0000 7f115b165740 10 client.15741 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T03:47:03.928 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:03.919+0000 7f115b165740 20 client.15741 populate_metadata read hostname 'smithi039' 2024-09-21T03:47:03.928 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:03.919+0000 7f115b165740 1 --2- 172.21.15.39:0/4214399716 >> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] conn(0x5572d44c8cd0 0x5572d48d78c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:47:03.928 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:03.919+0000 7f115b165740 4 client.15741 mds_command: new command op to 24695 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:47:03.928 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:03.919+0000 7f115b165740 1 -- 172.21.15.39:0/4214399716 --> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f114001c480 con 0x5572d44c8cd0 2024-09-21T03:47:03.928 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:03.920+0000 7f114e59c640 1 --2- 172.21.15.39:0/4214399716 >> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] conn(0x5572d44c8cd0 0x5572d48d78c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:47:03.928 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:03.920+0000 7f114e59c640 1 --2- 172.21.15.39:0/4214399716 >> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] conn(0x5572d44c8cd0 0x5572d48d78c0 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7c7e20ac072f3da9 server_cookie=904d8af26486a129 in_seq=0 out_seq=0 2024-09-21T03:47:03.929 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:03.920+0000 7f1135ffb640 10 client.15741 ms_handle_connect on v2:172.21.15.150:6838/2949386907 2024-09-21T03:47:03.929 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:03.921+0000 7f1135ffb640 1 -- 172.21.15.39:0/4214399716 <== mds.0 v2:172.21.15.150:6838/2949386907 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x7f114001c480 con 0x5572d44c8cd0 2024-09-21T03:47:03.929 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:03.921+0000 7f1135ffb640 10 client.15741 handle_command_reply: tid=0 2024-09-21T03:47:03.929 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:03.921+0000 7f115b165740 1 client.15741 shutdown 2024-09-21T03:47:03.930 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:03.921+0000 7f115b165740 1 -- 172.21.15.39:0/4214399716 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5572d44e9670 msgr2=0x5572d48b9440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:47:03.930 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:03.921+0000 7f115b165740 1 --2- 172.21.15.39:0/4214399716 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5572d44e9670 0x5572d48b9440 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f11440ad320 tx=0x7f11440e55a0 comp rx=0 tx=0).stop 2024-09-21T03:47:03.930 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:03.921+0000 7f115b165740 1 -- 172.21.15.39:0/4214399716 shutdown_connections 2024-09-21T03:47:03.930 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:03.921+0000 7f115b165740 1 --2- 172.21.15.39:0/4214399716 >> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] conn(0x5572d44c8cd0 0x5572d48d78c0 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:03.931 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:03.922+0000 7f1135ffb640 0 client.15741 ms_handle_reset on v2:172.21.15.150:6838/2949386907 2024-09-21T03:47:03.931 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:03.921+0000 7f115b165740 1 --2- 172.21.15.39:0/4214399716 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5572d450a350 0x5572d48d16d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:03.931 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:03.922+0000 7f115b165740 1 --2- 172.21.15.39:0/4214399716 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5572d44e9670 0x5572d48b9440 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:03.931 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:03.922+0000 7f115b165740 1 --2- 172.21.15.39:0/4214399716 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5572d44e1490 0x5572d48b8f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:03.931 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:03.922+0000 7f115b165740 1 -- 172.21.15.39:0/4214399716 >> 172.21.15.39:0/4214399716 conn(0x5572d44fe740 msgr2=0x5572d48ad850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:47:03.931 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:03.922+0000 7f115b165740 1 -- 172.21.15.39:0/4214399716 shutdown_connections 2024-09-21T03:47:03.931 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:03.922+0000 7f115b165740 1 -- 172.21.15.39:0/4214399716 wait complete. 2024-09-21T03:47:03.931 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:03.922+0000 7f115b165740 20 client.15741 trim_cache size 0 max 16384 2024-09-21T03:47:03.960 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:03.957+0000 7f115b165740 1 Processor -- start 2024-09-21T03:47:03.960 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:03.957+0000 7f115b165740 1 -- start start 2024-09-21T03:47:03.961 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:03.958+0000 7f115b165740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5572d44fe740 0x5572d48b8590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:47:03.961 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:03.958+0000 7f115b165740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5572d44c8cd0 0x5572d48adf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:47:03.961 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:03.958+0000 7f115b165740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5572d44e1490 0x5572d48d15d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:47:03.961 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:03.958+0000 7f115b165740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f11501ad5d0 con 0x5572d44e1490 2024-09-21T03:47:03.962 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:03.958+0000 7f115b165740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f11501af110 con 0x5572d44c8cd0 2024-09-21T03:47:03.962 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:03.958+0000 7f115b165740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f11501adca0 con 0x5572d44fe740 2024-09-21T03:47:03.962 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:03.958+0000 7f114dd9b640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5572d44c8cd0 0x5572d48adf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:47:03.962 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:03.958+0000 7f114ed9d640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5572d44e1490 0x5572d48d15d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:47:03.962 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:03.958+0000 7f114dd9b640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5572d44c8cd0 0x5572d48adf60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.39:35402/0 (socket says 172.21.15.39:35402) 2024-09-21T03:47:03.963 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:03.958+0000 7f114dd9b640 1 -- 172.21.15.39:0/2624212040 learned_addr learned my addr 172.21.15.39:0/2624212040 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:47:03.963 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:03.958+0000 7f114e59c640 1 --2- 172.21.15.39:0/2624212040 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5572d44fe740 0x5572d48b8590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:47:03.963 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:03.958+0000 7f114dd9b640 1 -- 172.21.15.39:0/2624212040 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5572d44fe740 msgr2=0x5572d48b8590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:47:03.963 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:03.958+0000 7f114dd9b640 1 --2- 172.21.15.39:0/2624212040 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5572d44fe740 0x5572d48b8590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:03.963 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:03.958+0000 7f114dd9b640 1 -- 172.21.15.39:0/2624212040 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5572d44e1490 msgr2=0x5572d48d15d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:47:03.963 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:03.958+0000 7f114dd9b640 1 --2- 172.21.15.39:0/2624212040 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5572d44e1490 0x5572d48d15d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:03.963 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:03.958+0000 7f114dd9b640 1 -- 172.21.15.39:0/2624212040 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5572d48b7200 con 0x5572d44c8cd0 2024-09-21T03:47:03.963 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:03.958+0000 7f114e59c640 1 --2- 172.21.15.39:0/2624212040 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5572d44fe740 0x5572d48b8590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:47:03.963 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:03.958+0000 7f114ed9d640 1 --2- 172.21.15.39:0/2624212040 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5572d44e1490 0x5572d48d15d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:47:03.963 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:03.959+0000 7f114dd9b640 1 --2- 172.21.15.39:0/2624212040 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5572d44c8cd0 0x5572d48adf60 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f11440ad5b0 tx=0x7f11440e4f10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=66029c1552a18df3 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:47:03.963 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:03.959+0000 7f1137fff640 1 -- 172.21.15.39:0/2624212040 <== mon.2 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11440ad130 con 0x5572d44c8cd0 2024-09-21T03:47:03.963 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:03.959+0000 7f1137fff640 1 -- 172.21.15.39:0/2624212040 <== mon.2 v2:172.21.15.150:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f11440add20 con 0x5572d44c8cd0 2024-09-21T03:47:03.963 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:03.959+0000 7f1137fff640 1 -- 172.21.15.39:0/2624212040 <== mon.2 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11440a92a0 con 0x5572d44c8cd0 2024-09-21T03:47:03.965 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:03.960+0000 7f115b165740 1 -- 172.21.15.39:0/2624212040 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5572d44c8cd0 msgr2=0x5572d48adf60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:47:03.965 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:03.960+0000 7f115b165740 1 --2- 172.21.15.39:0/2624212040 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5572d44c8cd0 0x5572d48adf60 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f11440ad5b0 tx=0x7f11440e4f10 comp rx=0 tx=0).stop 2024-09-21T03:47:03.966 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:03.960+0000 7f115b165740 1 -- 172.21.15.39:0/2624212040 shutdown_connections 2024-09-21T03:47:03.966 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:03.960+0000 7f115b165740 1 --2- 172.21.15.39:0/2624212040 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5572d44e1490 0x5572d48d15d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:03.966 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:03.960+0000 7f115b165740 1 --2- 172.21.15.39:0/2624212040 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5572d44c8cd0 0x5572d48adf60 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:03.966 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:03.960+0000 7f115b165740 1 --2- 172.21.15.39:0/2624212040 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5572d44fe740 0x5572d48b8590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:03.966 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:03.960+0000 7f115b165740 1 -- 172.21.15.39:0/2624212040 >> 172.21.15.39:0/2624212040 conn(0x5572d48c6a70 msgr2=0x5572d48ad850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:47:03.966 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:03.960+0000 7f115b165740 1 -- 172.21.15.39:0/2624212040 shutdown_connections 2024-09-21T03:47:03.966 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:03.960+0000 7f115b165740 1 -- 172.21.15.39:0/2624212040 wait complete. 2024-09-21T03:47:03.966 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:03.961+0000 7f115b165740 1 -- start start 2024-09-21T03:47:03.966 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:03.961+0000 7f115b165740 1 Processor -- start 2024-09-21T03:47:03.966 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:03.961+0000 7f115b165740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5572d44c8cd0 0x5572d48cdcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:47:03.966 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:03.961+0000 7f115b165740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5572d44e1490 0x5572d48ca180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:47:03.966 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:03.961+0000 7f115b165740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5572d44fe740 0x5572d48c3b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:47:03.966 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:03.961+0000 7f115b165740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f11501aea40 con 0x5572d44fe740 2024-09-21T03:47:03.966 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:03.961+0000 7f115b165740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f11501ae370 con 0x5572d44c8cd0 2024-09-21T03:47:03.967 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:03.961+0000 7f115b165740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f11501ab5b0 con 0x5572d44e1490 2024-09-21T03:47:03.967 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:03.961+0000 7f115b165740 10 client.0 _ll_register_callbacks cb 0x7f11440e4e60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:47:03.967 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:03.961+0000 7f114ed9d640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5572d44fe740 0x5572d48c3b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:47:03.967 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:03.961+0000 7f114ed9d640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5572d44fe740 0x5572d48c3b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:52270/0 (socket says 172.21.15.39:52270) 2024-09-21T03:47:03.967 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:03.961+0000 7f114ed9d640 1 -- 172.21.15.39:0/1986468672 learned_addr learned my addr 172.21.15.39:0/1986468672 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:47:03.967 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:03.962+0000 7f114dd9b640 1 --2- 172.21.15.39:0/1986468672 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5572d44e1490 0x5572d48ca180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:47:03.967 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:03.962+0000 7f114e59c640 1 --2- 172.21.15.39:0/1986468672 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5572d44c8cd0 0x5572d48cdcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:47:03.967 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:03.962+0000 7f114ed9d640 1 -- 172.21.15.39:0/1986468672 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5572d44e1490 msgr2=0x5572d48ca180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:47:03.967 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:03.962+0000 7f114ed9d640 1 --2- 172.21.15.39:0/1986468672 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5572d44e1490 0x5572d48ca180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:03.967 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:03.962+0000 7f114ed9d640 1 -- 172.21.15.39:0/1986468672 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5572d44c8cd0 msgr2=0x5572d48cdcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:47:03.967 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:03.962+0000 7f114ed9d640 1 --2- 172.21.15.39:0/1986468672 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5572d44c8cd0 0x5572d48cdcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:03.967 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:03.962+0000 7f114ed9d640 1 -- 172.21.15.39:0/1986468672 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5572d48c4190 con 0x5572d44fe740 2024-09-21T03:47:03.967 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:03.962+0000 7f114dd9b640 1 --2- 172.21.15.39:0/1986468672 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5572d44e1490 0x5572d48ca180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:47:03.967 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:03.962+0000 7f114ed9d640 1 --2- 172.21.15.39:0/1986468672 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5572d44fe740 0x5572d48c3b50 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f1148003c20 tx=0x7f1148005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=969209664108911f server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:47:03.968 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:03.962+0000 7f1135ffb640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-09-21T03:47:03.968 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:03.962+0000 7f1135ffb640 1 -- 172.21.15.39:0/1986468672 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11480047c0 con 0x5572d44fe740 2024-09-21T03:47:03.968 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:03.962+0000 7f115b165740 1 -- 172.21.15.39:0/1986468672 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5572d48c81f0 con 0x5572d44fe740 2024-09-21T03:47:03.968 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:03.962+0000 7f1135ffb640 1 -- 172.21.15.39:0/1986468672 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8484+0+0 (secure 0 0 0) 0x7f114802f660 con 0x5572d44fe740 2024-09-21T03:47:03.968 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:03.963+0000 7f1135ffb640 1 -- 172.21.15.39:0/1986468672 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f114802f860 con 0x5572d44fe740 2024-09-21T03:47:03.968 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:03.963+0000 7f115b165740 10 client.15747 fetch_fsmap learned FSMap version 29 2024-09-21T03:47:03.968 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:03.963+0000 7f115b165740 1 -- 172.21.15.39:0/1986468672 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5572d493a8b0 con 0x5572d44fe740 2024-09-21T03:47:03.970 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:03.963+0000 7f1135ffb640 1 -- 172.21.15.39:0/1986468672 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x5572d48c81f0 con 0x5572d44fe740 2024-09-21T03:47:03.970 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:03.963+0000 7f1135ffb640 1 -- 172.21.15.39:0/1986468672 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f1148030bb0 con 0x5572d44fe740 2024-09-21T03:47:03.970 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:03.963+0000 7f115b165740 10 client.15747 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T03:47:03.970 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:03.963+0000 7f115b165740 10 client.15747 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T03:47:03.970 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:03.963+0000 7f115b165740 20 client.15747 populate_metadata read hostname 'smithi039' 2024-09-21T03:47:03.970 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:03.963+0000 7f115b165740 1 --2- 172.21.15.39:0/1986468672 >> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] conn(0x5572d44e9670 0x5572d493d0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:47:03.970 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:03.963+0000 7f115b165740 4 client.15747 mds_command: new command op to 24695 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-21T03:47:03.970 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:03.963+0000 7f115b165740 1 -- 172.21.15.39:0/1986468672 --> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x5572d4829db0 con 0x5572d44e9670 2024-09-21T03:47:03.970 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:03.964+0000 7f114e59c640 1 --2- 172.21.15.39:0/1986468672 >> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] conn(0x5572d44e9670 0x5572d493d0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:47:03.970 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:03.964+0000 7f1135ffb640 10 client.15747 ms_handle_connect on v2:172.21.15.150:6838/2949386907 2024-09-21T03:47:03.971 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:03.964+0000 7f114e59c640 1 --2- 172.21.15.39:0/1986468672 >> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] conn(0x5572d44e9670 0x5572d493d0d0 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=23e690f65cd4713b server_cookie=2dfb4c7762c5f382 in_seq=0 out_seq=0 2024-09-21T03:47:03.971 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:03.966+0000 7f1135ffb640 1 -- 172.21.15.39:0/1986468672 <== mds.0 v2:172.21.15.150:6838/2949386907 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x5572d4829db0 con 0x5572d44e9670 2024-09-21T03:47:03.971 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:03.966+0000 7f1135ffb640 10 client.15747 handle_command_reply: tid=0 2024-09-21T03:47:03.971 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:03.966+0000 7f115b165740 1 client.15747 shutdown 2024-09-21T03:47:03.971 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:03.966+0000 7f115b165740 1 -- 172.21.15.39:0/1986468672 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5572d44fe740 msgr2=0x5572d48c3b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:47:03.971 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:03.966+0000 7f115b165740 1 --2- 172.21.15.39:0/1986468672 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5572d44fe740 0x5572d48c3b50 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f1148003c20 tx=0x7f1148005fd0 comp rx=0 tx=0).stop 2024-09-21T03:47:03.971 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:03.966+0000 7f115b165740 1 -- 172.21.15.39:0/1986468672 shutdown_connections 2024-09-21T03:47:03.971 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:03.966+0000 7f115b165740 1 --2- 172.21.15.39:0/1986468672 >> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] conn(0x5572d44e9670 0x5572d493d0d0 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:03.971 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:03.967+0000 7f115b165740 1 --2- 172.21.15.39:0/1986468672 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5572d44fe740 0x5572d48c3b50 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:03.971 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:03.967+0000 7f1135ffb640 0 client.15747 ms_handle_reset on v2:172.21.15.150:6838/2949386907 2024-09-21T03:47:03.971 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:03.967+0000 7f115b165740 1 --2- 172.21.15.39:0/1986468672 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5572d44e1490 0x5572d48ca180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:03.971 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:03.967+0000 7f115b165740 1 --2- 172.21.15.39:0/1986468672 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5572d44c8cd0 0x5572d48cdcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:03.971 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:03.967+0000 7f115b165740 1 -- 172.21.15.39:0/1986468672 >> 172.21.15.39:0/1986468672 conn(0x5572d48c6a70 msgr2=0x5572d48b2460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:47:03.971 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:03.967+0000 7f115b165740 1 -- 172.21.15.39:0/1986468672 shutdown_connections 2024-09-21T03:47:03.972 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:03.967+0000 7f115b165740 1 -- 172.21.15.39:0/1986468672 wait complete. 2024-09-21T03:47:03.972 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:03.967+0000 7f115b165740 20 client.15747 trim_cache size 0 max 16384 2024-09-21T03:47:03.972 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-09-21T03:47:03.972 INFO:teuthology.orchestra.run.smithi039.stdout: "return_code": 0, 2024-09-21T03:47:03.972 INFO:teuthology.orchestra.run.smithi039.stdout: "scrub_tag": "10728de8-c49d-4edc-be2b-639249255e94", 2024-09-21T03:47:03.972 INFO:teuthology.orchestra.run.smithi039.stdout: "mode": "asynchronous" 2024-09-21T03:47:03.972 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-09-21T03:47:03.973 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:03.968+0000 7f1154bf1640 1 -- 172.21.15.39:0/2598719099 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f11380b6790 msgr2=0x7f11380b8ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:47:03.973 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:03.968+0000 7f1154bf1640 1 --2- 172.21.15.39:0/2598719099 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f11380b6790 0x7f11380b8ca0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f1148005e30 tx=0x7f11480095c0 comp rx=0 tx=0).stop 2024-09-21T03:47:03.973 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:03.968+0000 7f1154bf1640 1 -- 172.21.15.39:0/2598719099 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f11501094b0 msgr2=0x7f115019aa90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:47:03.973 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:03.968+0000 7f1154bf1640 1 --2- 172.21.15.39:0/2598719099 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f11501094b0 0x7f115019aa90 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f1150075ab0 tx=0x7f114400ee50 comp rx=0 tx=0).stop 2024-09-21T03:47:03.973 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:03.968+0000 7f1154bf1640 1 -- 172.21.15.39:0/2598719099 shutdown_connections 2024-09-21T03:47:03.974 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:03.968+0000 7f1154bf1640 1 --2- 172.21.15.39:0/2598719099 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f11380b6790 0x7f11380b8ca0 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:03.974 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:03.968+0000 7f1154bf1640 1 --2- 172.21.15.39:0/2598719099 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1150109e00 0x7f115019fd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:03.974 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:03.968+0000 7f1154bf1640 1 --2- 172.21.15.39:0/2598719099 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f11501094b0 0x7f115019aa90 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:03.974 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:03.968+0000 7f1154bf1640 1 --2- 172.21.15.39:0/2598719099 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1150101f70 0x7f115019a4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:03.974 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:03.968+0000 7f1154bf1640 1 -- 172.21.15.39:0/2598719099 >> 172.21.15.39:0/2598719099 conn(0x7f1150077c60 msgr2=0x7f1150107540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:47:03.974 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:03.968+0000 7f1154bf1640 1 -- 172.21.15.39:0/2598719099 shutdown_connections 2024-09-21T03:47:03.974 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:03.968+0000 7f1154bf1640 1 -- 172.21.15.39:0/2598719099 wait complete. 2024-09-21T03:47:04.819 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:47:04 smithi039 ceph-mon[31211]: pgmap v356: 97 pgs: 97 active+clean; 1.4 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 767 B/s wr, 9 op/s 2024-09-21T03:47:04.862 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 51116d5c-77ca-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub status 2024-09-21T03:47:04.968 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:47:04 smithi161 ceph-mon[34533]: pgmap v356: 97 pgs: 97 active+clean; 1.4 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 767 B/s wr, 9 op/s 2024-09-21T03:47:04.974 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:47:04 smithi150 ceph-mon[34289]: pgmap v356: 97 pgs: 97 active+clean; 1.4 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 767 B/s wr, 9 op/s 2024-09-21T03:47:05.272 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/51116d5c-77ca-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:47:05.824 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:05.820+0000 7f24d7228640 1 -- 172.21.15.39:0/36399559 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f24d0108cf0 msgr2=0x7f24d010f520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:47:05.824 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:05.820+0000 7f24d7228640 1 --2- 172.21.15.39:0/36399559 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f24d0108cf0 0x7f24d010f520 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f24c8009ca0 tx=0x7f24c802a8c0 comp rx=0 tx=0).stop 2024-09-21T03:47:05.824 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:05.821+0000 7f24d7228640 1 -- 172.21.15.39:0/36399559 shutdown_connections 2024-09-21T03:47:05.824 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:05.821+0000 7f24d7228640 1 --2- 172.21.15.39:0/36399559 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f24d0108cf0 0x7f24d010f520 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:05.824 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:05.821+0000 7f24d7228640 1 --2- 172.21.15.39:0/36399559 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f24d01083a0 0x7f24d01087b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:05.824 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:05.821+0000 7f24d7228640 1 --2- 172.21.15.39:0/36399559 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f24d0102a60 0x7f24d0102e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:05.824 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:05.821+0000 7f24d7228640 1 -- 172.21.15.39:0/36399559 >> 172.21.15.39:0/36399559 conn(0x7f24d0079730 msgr2=0x7f24d0079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:47:05.824 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:05.821+0000 7f24d7228640 1 -- 172.21.15.39:0/36399559 shutdown_connections 2024-09-21T03:47:05.824 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:05.821+0000 7f24d7228640 1 -- 172.21.15.39:0/36399559 wait complete. 2024-09-21T03:47:05.825 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:05.822+0000 7f24d7228640 1 Processor -- start 2024-09-21T03:47:05.825 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:05.822+0000 7f24d7228640 1 -- start start 2024-09-21T03:47:05.826 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:05.822+0000 7f24d7228640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f24d0102a60 0x7f24d01993e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:47:05.826 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:05.822+0000 7f24d7228640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f24d01083a0 0x7f24d0199980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:47:05.826 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:05.822+0000 7f24d7228640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f24d0108cf0 0x7f24d019ec20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:47:05.826 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:05.822+0000 7f24d7228640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f24d01129b0 con 0x7f24d01083a0 2024-09-21T03:47:05.826 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:05.822+0000 7f24d7228640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f24d0112830 con 0x7f24d0108cf0 2024-09-21T03:47:05.826 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:05.822+0000 7f24d7228640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f24d0112b30 con 0x7f24d0102a60 2024-09-21T03:47:05.826 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:05.823+0000 7f24d4fc4640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f24d0102a60 0x7f24d01993e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:47:05.826 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:05.823+0000 7f24d4fc4640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f24d0102a60 0x7f24d01993e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.39:37658/0 (socket says 172.21.15.39:37658) 2024-09-21T03:47:05.826 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:05.823+0000 7f24d4fc4640 1 -- 172.21.15.39:0/1515058788 learned_addr learned my addr 172.21.15.39:0/1515058788 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:47:05.826 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:05.823+0000 7f24d57c5640 1 --2- 172.21.15.39:0/1515058788 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f24d0108cf0 0x7f24d019ec20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:47:05.826 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:05.823+0000 7f24cffff640 1 --2- 172.21.15.39:0/1515058788 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f24d01083a0 0x7f24d0199980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:47:05.828 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:05.823+0000 7f24d4fc4640 1 -- 172.21.15.39:0/1515058788 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f24d0108cf0 msgr2=0x7f24d019ec20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:47:05.828 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:05.823+0000 7f24d4fc4640 1 --2- 172.21.15.39:0/1515058788 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f24d0108cf0 0x7f24d019ec20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:05.828 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:05.823+0000 7f24d4fc4640 1 -- 172.21.15.39:0/1515058788 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f24d01083a0 msgr2=0x7f24d0199980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:47:05.828 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:05.823+0000 7f24d4fc4640 1 --2- 172.21.15.39:0/1515058788 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f24d01083a0 0x7f24d0199980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:05.828 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:05.823+0000 7f24d4fc4640 1 -- 172.21.15.39:0/1515058788 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f24d019f320 con 0x7f24d0102a60 2024-09-21T03:47:05.828 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:05.823+0000 7f24cffff640 1 --2- 172.21.15.39:0/1515058788 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f24d01083a0 0x7f24d0199980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:47:05.828 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:05.823+0000 7f24d4fc4640 1 --2- 172.21.15.39:0/1515058788 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f24d0102a60 0x7f24d01993e0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f24c400ea20 tx=0x7f24c400eee0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:47:05.828 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:05.824+0000 7f24cdffb640 1 -- 172.21.15.39:0/1515058788 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24c400cd80 con 0x7f24d0102a60 2024-09-21T03:47:05.828 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:05.824+0000 7f24cdffb640 1 -- 172.21.15.39:0/1515058788 <== mon.1 v2:172.21.15.161:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f24c40108e0 con 0x7f24d0102a60 2024-09-21T03:47:05.828 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:05.824+0000 7f24d7228640 1 -- 172.21.15.39:0/1515058788 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f24d019f610 con 0x7f24d0102a60 2024-09-21T03:47:05.828 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:05.824+0000 7f24cdffb640 1 -- 172.21.15.39:0/1515058788 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24c4018840 con 0x7f24d0102a60 2024-09-21T03:47:05.829 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:05.824+0000 7f24d7228640 1 -- 172.21.15.39:0/1515058788 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f24d006ae60 con 0x7f24d0102a60 2024-09-21T03:47:05.830 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:05.828+0000 7f24dd79c740 1 Processor -- start 2024-09-21T03:47:05.831 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:05.828+0000 7f24dd79c740 1 -- start start 2024-09-21T03:47:05.831 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:05.828+0000 7f24cdffb640 1 -- 172.21.15.39:0/1515058788 <== mon.1 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 26) ==== 154694+0+0 (secure 0 0 0) 0x7f24c4010430 con 0x7f24d0102a60 2024-09-21T03:47:05.832 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:05.828+0000 7f24dd79c740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561e7c31b070 0x561e7c6caa80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:47:05.832 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:05.828+0000 7f24dd79c740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561e7c2f21b0 0x561e7c6cafc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:47:05.832 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:05.828+0000 7f24dd79c740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561e7c2fa390 0x561e7c6cb500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:47:05.832 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:05.828+0000 7f24dd79c740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x561e7c638e50 con 0x561e7c2fa390 2024-09-21T03:47:05.832 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:05.828+0000 7f24dd79c740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x561e7c628280 con 0x561e7c2f21b0 2024-09-21T03:47:05.832 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:05.828+0000 7f24dd79c740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x561e7c62c090 con 0x561e7c31b070 2024-09-21T03:47:05.832 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:05.828+0000 7f24cffff640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561e7c2fa390 0x561e7c6cb500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:47:05.832 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:05.829+0000 7f24cffff640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561e7c2fa390 0x561e7c6cb500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:52308/0 (socket says 172.21.15.39:52308) 2024-09-21T03:47:05.833 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:05.829+0000 7f24cffff640 1 -- 172.21.15.39:0/1747322969 learned_addr learned my addr 172.21.15.39:0/1747322969 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:47:05.833 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:05.829+0000 7f24d4fc4640 1 --2- 172.21.15.39:0/1747322969 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561e7c2f21b0 0x561e7c6cafc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:47:05.833 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:05.829+0000 7f24d57c5640 1 --2- 172.21.15.39:0/1747322969 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561e7c31b070 0x561e7c6caa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:47:05.834 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:05.829+0000 7f24cffff640 1 -- 172.21.15.39:0/1747322969 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561e7c31b070 msgr2=0x561e7c6caa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:47:05.834 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:05.829+0000 7f24cffff640 1 --2- 172.21.15.39:0/1747322969 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561e7c31b070 0x561e7c6caa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:05.834 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:05.829+0000 7f24cffff640 1 -- 172.21.15.39:0/1747322969 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561e7c2f21b0 msgr2=0x561e7c6cafc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:47:05.834 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:05.829+0000 7f24cffff640 1 --2- 172.21.15.39:0/1747322969 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561e7c2f21b0 0x561e7c6cafc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:05.834 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:05.830+0000 7f24cdffb640 1 --2- 172.21.15.39:0/1515058788 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f24c00b6810 0x7f24c00b8d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:47:05.834 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:05.830+0000 7f24cffff640 1 -- 172.21.15.39:0/1747322969 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561e7c386650 con 0x561e7c2fa390 2024-09-21T03:47:05.834 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:05.830+0000 7f24cdffb640 1 -- 172.21.15.39:0/1515058788 <== mon.1 v2:172.21.15.161:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8484+0+0 (secure 0 0 0) 0x7f24c40e7c10 con 0x7f24d0102a60 2024-09-21T03:47:05.834 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:05.830+0000 7f24d57c5640 1 --2- 172.21.15.39:0/1515058788 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f24c00b6810 0x7f24c00b8d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:47:05.834 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:05.830+0000 7f24cffff640 1 --2- 172.21.15.39:0/1747322969 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561e7c2fa390 0x561e7c6cb500 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f24bc009b60 tx=0x7f24bc00e6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e737409c1c0c4066 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:47:05.834 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:05.831+0000 7f24baffd640 1 -- 172.21.15.39:0/1747322969 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24bc016070 con 0x561e7c2fa390 2024-09-21T03:47:05.835 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:05.831+0000 7f24baffd640 1 -- 172.21.15.39:0/1747322969 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f24bc009e50 con 0x561e7c2fa390 2024-09-21T03:47:05.835 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:05.831+0000 7f24baffd640 1 -- 172.21.15.39:0/1747322969 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24bc004e60 con 0x561e7c2fa390 2024-09-21T03:47:05.841 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:05.832+0000 7f24dd79c740 1 -- 172.21.15.39:0/1747322969 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561e7c2fa390 msgr2=0x561e7c6cb500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:47:05.841 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:05.832+0000 7f24d57c5640 1 --2- 172.21.15.39:0/1515058788 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f24c00b6810 0x7f24c00b8d20 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f24c8006010 tx=0x7f24c8005d80 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:47:05.841 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:05.832+0000 7f24dd79c740 1 --2- 172.21.15.39:0/1747322969 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561e7c2fa390 0x561e7c6cb500 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f24bc009b60 tx=0x7f24bc00e6c0 comp rx=0 tx=0).stop 2024-09-21T03:47:05.841 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:05.832+0000 7f24dd79c740 1 -- 172.21.15.39:0/1747322969 shutdown_connections 2024-09-21T03:47:05.841 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:05.832+0000 7f24dd79c740 1 --2- 172.21.15.39:0/1747322969 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561e7c2fa390 0x561e7c6cb500 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:05.841 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:05.832+0000 7f24dd79c740 1 --2- 172.21.15.39:0/1747322969 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561e7c2f21b0 0x561e7c6cafc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:05.842 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:05.832+0000 7f24dd79c740 1 --2- 172.21.15.39:0/1747322969 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561e7c31b070 0x561e7c6caa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:05.842 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:05.832+0000 7f24dd79c740 1 -- 172.21.15.39:0/1747322969 >> 172.21.15.39:0/1747322969 conn(0x561e7c30f460 msgr2=0x561e7c6c8970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:47:05.842 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:05.832+0000 7f24dd79c740 1 -- 172.21.15.39:0/1747322969 shutdown_connections 2024-09-21T03:47:05.842 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:05.832+0000 7f24dd79c740 1 -- 172.21.15.39:0/1747322969 wait complete. 2024-09-21T03:47:05.842 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:05.833+0000 7f24dd79c740 1 -- start start 2024-09-21T03:47:05.842 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:05.833+0000 7f24dd79c740 1 Processor -- start 2024-09-21T03:47:05.843 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:05.834+0000 7f24dd79c740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561e7c2f21b0 0x561e7c6d6250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:47:05.843 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:05.834+0000 7f24dd79c740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561e7c31b070 0x561e7c6d6790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:47:05.843 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:05.834+0000 7f24dd79c740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561e7c2d99f0 0x561e7c6e2550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:47:05.843 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:05.834+0000 7f24dd79c740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x561e7c489700 con 0x561e7c2d99f0 2024-09-21T03:47:05.843 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:05.834+0000 7f24dd79c740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x561e7c5c3110 con 0x561e7c2f21b0 2024-09-21T03:47:05.843 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:05.834+0000 7f24dd79c740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x561e7c6d9260 con 0x561e7c31b070 2024-09-21T03:47:05.844 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:05.834+0000 7f24dd79c740 10 client.0 _ll_register_callbacks cb 0x561e7c5dc990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:47:05.844 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:05.834+0000 7f24d4fc4640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561e7c2f21b0 0x561e7c6d6250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:47:05.844 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:05.834+0000 7f24d4fc4640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561e7c2f21b0 0x561e7c6d6250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.39:35440/0 (socket says 172.21.15.39:35440) 2024-09-21T03:47:05.844 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:05.834+0000 7f24d4fc4640 1 -- 172.21.15.39:0/1461984976 learned_addr learned my addr 172.21.15.39:0/1461984976 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:47:05.844 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:05.834+0000 7f24d57c5640 1 --2- 172.21.15.39:0/1461984976 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561e7c2d99f0 0x561e7c6e2550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:47:05.844 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:05.835+0000 7f24cffff640 1 --2- 172.21.15.39:0/1461984976 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561e7c31b070 0x561e7c6d6790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:47:05.845 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:05.835+0000 7f24d57c5640 1 -- 172.21.15.39:0/1461984976 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561e7c31b070 msgr2=0x561e7c6d6790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:47:05.845 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:05.835+0000 7f24d57c5640 1 --2- 172.21.15.39:0/1461984976 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561e7c31b070 0x561e7c6d6790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:05.845 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:05.835+0000 7f24d57c5640 1 -- 172.21.15.39:0/1461984976 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561e7c2f21b0 msgr2=0x561e7c6d6250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:47:05.845 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:05.835+0000 7f24d57c5640 1 --2- 172.21.15.39:0/1461984976 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561e7c2f21b0 0x561e7c6d6250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:05.845 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:05.835+0000 7f24d57c5640 1 -- 172.21.15.39:0/1461984976 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561e7c5f0c40 con 0x561e7c2d99f0 2024-09-21T03:47:05.845 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:05.835+0000 7f24cffff640 1 --2- 172.21.15.39:0/1461984976 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561e7c31b070 0x561e7c6d6790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:47:05.846 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:05.835+0000 7f24d57c5640 1 --2- 172.21.15.39:0/1461984976 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561e7c2d99f0 0x561e7c6e2550 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f24c8005e30 tx=0x7f24c802a8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b199e98098fd0402 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:47:05.846 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:05.835+0000 7f24affff640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-09-21T03:47:05.846 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:05.835+0000 7f24d4fc4640 1 --2- 172.21.15.39:0/1461984976 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561e7c2f21b0 0x561e7c6d6250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:47:05.846 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:05.836+0000 7f24affff640 1 -- 172.21.15.39:0/1461984976 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24c802f610 con 0x561e7c2d99f0 2024-09-21T03:47:05.846 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:05.836+0000 7f24affff640 1 -- 172.21.15.39:0/1461984976 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8484+0+0 (secure 0 0 0) 0x7f24c802f7b0 con 0x561e7c2d99f0 2024-09-21T03:47:05.846 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:05.836+0000 7f24dd79c740 1 -- 172.21.15.39:0/1461984976 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f24bc009e50 con 0x561e7c2d99f0 2024-09-21T03:47:05.846 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:05.836+0000 7f24affff640 1 -- 172.21.15.39:0/1461984976 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f24c802f9b0 con 0x561e7c2d99f0 2024-09-21T03:47:05.847 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:05.836+0000 7f24dd79c740 10 client.15756 fetch_fsmap learned FSMap version 29 2024-09-21T03:47:05.847 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:05.836+0000 7f24dd79c740 1 -- 172.21.15.39:0/1461984976 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x561e7c6e5090 con 0x561e7c2d99f0 2024-09-21T03:47:05.847 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:05.836+0000 7f24affff640 1 -- 172.21.15.39:0/1461984976 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f24c8036050 con 0x561e7c2d99f0 2024-09-21T03:47:05.847 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:05.836+0000 7f24affff640 1 -- 172.21.15.39:0/1461984976 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f24c805a070 con 0x561e7c2d99f0 2024-09-21T03:47:05.847 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:05.836+0000 7f24dd79c740 10 client.15756 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T03:47:05.848 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:05.836+0000 7f24dd79c740 10 client.15756 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T03:47:05.848 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:05.837+0000 7f24dd79c740 20 client.15756 populate_metadata read hostname 'smithi039' 2024-09-21T03:47:05.848 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:05.837+0000 7f24dd79c740 1 --2- 172.21.15.39:0/1461984976 >> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] conn(0x561e7c2aeda0 0x561e7c6e8520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:47:05.848 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:05.837+0000 7f24dd79c740 4 client.15756 mds_command: new command op to 24695 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:47:05.848 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:05.837+0000 7f24dd79c740 1 -- 172.21.15.39:0/1461984976 --> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f24bc009e50 con 0x561e7c2aeda0 2024-09-21T03:47:05.848 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:05.837+0000 7f24d4fc4640 1 --2- 172.21.15.39:0/1461984976 >> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] conn(0x561e7c2aeda0 0x561e7c6e8520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:47:05.848 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:05.837+0000 7f24affff640 10 client.15756 ms_handle_connect on v2:172.21.15.150:6838/2949386907 2024-09-21T03:47:05.849 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:05.838+0000 7f24d4fc4640 1 --2- 172.21.15.39:0/1461984976 >> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] conn(0x561e7c2aeda0 0x561e7c6e8520 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a9407b30ebb069f7 server_cookie=3ad6d56ef1523ebd in_seq=0 out_seq=0 2024-09-21T03:47:05.851 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:05.839+0000 7f24affff640 1 -- 172.21.15.39:0/1461984976 <== mds.0 v2:172.21.15.150:6838/2949386907 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x7f24bc009e50 con 0x561e7c2aeda0 2024-09-21T03:47:05.851 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:05.839+0000 7f24affff640 10 client.15756 handle_command_reply: tid=0 2024-09-21T03:47:05.851 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:05.839+0000 7f24dd79c740 1 client.15756 shutdown 2024-09-21T03:47:05.851 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:05.839+0000 7f24dd79c740 1 -- 172.21.15.39:0/1461984976 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561e7c2d99f0 msgr2=0x561e7c6e2550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:47:05.851 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:05.839+0000 7f24dd79c740 1 --2- 172.21.15.39:0/1461984976 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561e7c2d99f0 0x561e7c6e2550 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f24c8005e30 tx=0x7f24c802a8c0 comp rx=0 tx=0).stop 2024-09-21T03:47:05.852 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:05.839+0000 7f24dd79c740 1 -- 172.21.15.39:0/1461984976 shutdown_connections 2024-09-21T03:47:05.852 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:05.839+0000 7f24dd79c740 1 --2- 172.21.15.39:0/1461984976 >> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] conn(0x561e7c2aeda0 0x561e7c6e8520 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:05.852 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:05.839+0000 7f24dd79c740 1 --2- 172.21.15.39:0/1461984976 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561e7c2d99f0 0x561e7c6e2550 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:05.852 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:05.839+0000 7f24dd79c740 1 --2- 172.21.15.39:0/1461984976 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561e7c31b070 0x561e7c6d6790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:05.852 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:05.839+0000 7f24dd79c740 1 --2- 172.21.15.39:0/1461984976 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561e7c2f21b0 0x561e7c6d6250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:05.852 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:05.839+0000 7f24affff640 0 client.15756 ms_handle_reset on v2:172.21.15.150:6838/2949386907 2024-09-21T03:47:05.853 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:05.839+0000 7f24dd79c740 1 -- 172.21.15.39:0/1461984976 >> 172.21.15.39:0/1461984976 conn(0x561e7c30f460 msgr2=0x561e7c6cdc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:47:05.853 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:05.840+0000 7f24dd79c740 1 -- 172.21.15.39:0/1461984976 shutdown_connections 2024-09-21T03:47:05.853 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:05.840+0000 7f24dd79c740 1 -- 172.21.15.39:0/1461984976 wait complete. 2024-09-21T03:47:05.853 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:05.840+0000 7f24dd79c740 20 client.15756 trim_cache size 0 max 16384 2024-09-21T03:47:05.877 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:05.875+0000 7f24dd79c740 1 Processor -- start 2024-09-21T03:47:05.878 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:05.875+0000 7f24dd79c740 1 -- start start 2024-09-21T03:47:05.878 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:05.875+0000 7f24dd79c740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561e7c2d99f0 0x561e7c6d8130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:47:05.878 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:05.875+0000 7f24dd79c740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561e7c2f21b0 0x561e7c6cc1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:47:05.878 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:05.875+0000 7f24dd79c740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561e7c31b070 0x561e7c6e2450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:47:05.879 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:05.875+0000 7f24dd79c740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f24d01ad540 con 0x561e7c31b070 2024-09-21T03:47:05.879 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:05.875+0000 7f24dd79c740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f24d01af080 con 0x561e7c2f21b0 2024-09-21T03:47:05.879 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:05.875+0000 7f24dd79c740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f24d01adc10 con 0x561e7c2d99f0 2024-09-21T03:47:05.879 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:05.876+0000 7f24cffff640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561e7c2f21b0 0x561e7c6cc1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:47:05.879 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:05.876+0000 7f24d4fc4640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561e7c2d99f0 0x561e7c6d8130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:47:05.879 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:05.876+0000 7f24cffff640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561e7c2f21b0 0x561e7c6cc1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.39:35448/0 (socket says 172.21.15.39:35448) 2024-09-21T03:47:05.879 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:05.876+0000 7f24cffff640 1 -- 172.21.15.39:0/2173057463 learned_addr learned my addr 172.21.15.39:0/2173057463 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:47:05.879 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:05.876+0000 7f24d57c5640 1 --2- 172.21.15.39:0/2173057463 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561e7c31b070 0x561e7c6e2450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:47:05.880 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:05.876+0000 7f24d57c5640 1 -- 172.21.15.39:0/2173057463 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561e7c2d99f0 msgr2=0x561e7c6d8130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:47:05.880 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:05.876+0000 7f24d57c5640 1 --2- 172.21.15.39:0/2173057463 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561e7c2d99f0 0x561e7c6d8130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:05.880 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:05.876+0000 7f24d57c5640 1 -- 172.21.15.39:0/2173057463 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561e7c2f21b0 msgr2=0x561e7c6cc1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:47:05.880 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:05.876+0000 7f24d57c5640 1 --2- 172.21.15.39:0/2173057463 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561e7c2f21b0 0x561e7c6cc1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:05.880 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:05.876+0000 7f24d57c5640 1 -- 172.21.15.39:0/2173057463 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561e7c6d4750 con 0x561e7c31b070 2024-09-21T03:47:05.880 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:05.876+0000 7f24d4fc4640 1 --2- 172.21.15.39:0/2173057463 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561e7c2d99f0 0x561e7c6d8130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:47:05.880 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:05.876+0000 7f24cffff640 1 --2- 172.21.15.39:0/2173057463 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561e7c2f21b0 0x561e7c6cc1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:47:05.880 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:05.876+0000 7f24d57c5640 1 --2- 172.21.15.39:0/2173057463 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561e7c31b070 0x561e7c6e2450 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f24c8030f90 tx=0x7f24c8031820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff030e0e47175368 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:47:05.880 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:05.877+0000 7f24ba7fc640 1 -- 172.21.15.39:0/2173057463 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24c8060070 con 0x561e7c31b070 2024-09-21T03:47:05.881 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:05.877+0000 7f24ba7fc640 1 -- 172.21.15.39:0/2173057463 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f24c8053760 con 0x561e7c31b070 2024-09-21T03:47:05.881 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:05.877+0000 7f24ba7fc640 1 -- 172.21.15.39:0/2173057463 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24c8053d80 con 0x561e7c31b070 2024-09-21T03:47:05.883 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:05.878+0000 7f24dd79c740 1 -- 172.21.15.39:0/2173057463 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561e7c31b070 msgr2=0x561e7c6e2450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:47:05.883 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:05.878+0000 7f24dd79c740 1 --2- 172.21.15.39:0/2173057463 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561e7c31b070 0x561e7c6e2450 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f24c8030f90 tx=0x7f24c8031820 comp rx=0 tx=0).stop 2024-09-21T03:47:05.883 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:05.878+0000 7f24dd79c740 1 -- 172.21.15.39:0/2173057463 shutdown_connections 2024-09-21T03:47:05.883 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:05.878+0000 7f24dd79c740 1 --2- 172.21.15.39:0/2173057463 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561e7c31b070 0x561e7c6e2450 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:05.883 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:05.878+0000 7f24dd79c740 1 --2- 172.21.15.39:0/2173057463 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561e7c2f21b0 0x561e7c6cc1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:05.883 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:05.878+0000 7f24dd79c740 1 --2- 172.21.15.39:0/2173057463 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561e7c2d99f0 0x561e7c6d8130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:05.883 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:05.878+0000 7f24dd79c740 1 -- 172.21.15.39:0/2173057463 >> 172.21.15.39:0/2173057463 conn(0x561e7c30f460 msgr2=0x561e7c6cbab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:47:05.883 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:05.878+0000 7f24dd79c740 1 -- 172.21.15.39:0/2173057463 shutdown_connections 2024-09-21T03:47:05.884 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:05.878+0000 7f24dd79c740 1 -- 172.21.15.39:0/2173057463 wait complete. 2024-09-21T03:47:05.884 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:05.879+0000 7f24dd79c740 1 -- start start 2024-09-21T03:47:05.884 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:05.879+0000 7f24dd79c740 1 Processor -- start 2024-09-21T03:47:05.884 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:05.879+0000 7f24dd79c740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561e7c2d99f0 0x561e7c6dea10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:47:05.884 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:05.879+0000 7f24dd79c740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561e7c2f21b0 0x561e7c6daf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:47:05.884 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:05.879+0000 7f24dd79c740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561e7c31b070 0x561e7c6db4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:47:05.884 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:05.880+0000 7f24d4fc4640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561e7c2d99f0 0x561e7c6dea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:47:05.884 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:05.880+0000 7f24d57c5640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561e7c31b070 0x561e7c6db4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:47:05.884 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:05.880+0000 7f24d4fc4640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561e7c2d99f0 0x561e7c6dea10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.39:35464/0 (socket says 172.21.15.39:35464) 2024-09-21T03:47:05.884 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:05.880+0000 7f24d57c5640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561e7c31b070 0x561e7c6db4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:52322/0 (socket says 172.21.15.39:52322) 2024-09-21T03:47:05.884 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:05.880+0000 7f24dd79c740 1 -- 172.21.15.39:0/1980802999 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f24d01ae9b0 con 0x561e7c31b070 2024-09-21T03:47:05.884 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:05.880+0000 7f24d4fc4640 1 -- 172.21.15.39:0/1980802999 learned_addr learned my addr 172.21.15.39:0/1980802999 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:47:05.884 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:05.880+0000 7f24dd79c740 1 -- 172.21.15.39:0/1980802999 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f24d01ae2e0 con 0x561e7c2d99f0 2024-09-21T03:47:05.885 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:05.880+0000 7f24dd79c740 1 -- 172.21.15.39:0/1980802999 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f24d01ab520 con 0x561e7c2f21b0 2024-09-21T03:47:05.885 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:05.880+0000 7f24cffff640 1 --2- 172.21.15.39:0/1980802999 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561e7c2f21b0 0x561e7c6daf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:47:05.886 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:05.880+0000 7f24dd79c740 10 client.0 _ll_register_callbacks cb 0x7f24c8009460 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:47:05.886 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:05.880+0000 7f24cffff640 1 -- 172.21.15.39:0/1980802999 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561e7c2d99f0 msgr2=0x561e7c6dea10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:47:05.886 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:05.880+0000 7f24cffff640 1 --2- 172.21.15.39:0/1980802999 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561e7c2d99f0 0x561e7c6dea10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:05.886 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:05.880+0000 7f24cffff640 1 -- 172.21.15.39:0/1980802999 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561e7c31b070 msgr2=0x561e7c6db4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:47:05.886 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:05.880+0000 7f24cffff640 1 --2- 172.21.15.39:0/1980802999 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561e7c31b070 0x561e7c6db4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:05.886 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:05.880+0000 7f24cffff640 1 -- 172.21.15.39:0/1980802999 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561e7c6db9f0 con 0x561e7c2f21b0 2024-09-21T03:47:05.887 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:05.880+0000 7f24d57c5640 1 --2- 172.21.15.39:0/1980802999 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561e7c31b070 0x561e7c6db4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:47:05.887 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:05.880+0000 7f24d4fc4640 1 --2- 172.21.15.39:0/1980802999 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561e7c2d99f0 0x561e7c6dea10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:47:05.887 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:05.881+0000 7f24cffff640 1 --2- 172.21.15.39:0/1980802999 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561e7c2f21b0 0x561e7c6daf70 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f24bc009ab0 tx=0x7f24bc00efc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1fb0787576a7549f server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:47:05.887 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:05.881+0000 7f24affff640 10 client.0 ms_handle_connect on v2:172.21.15.161:3300/0 2024-09-21T03:47:05.887 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:05.881+0000 7f24affff640 1 -- 172.21.15.39:0/1980802999 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24bc02a020 con 0x561e7c2f21b0 2024-09-21T03:47:05.887 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:05.881+0000 7f24affff640 1 -- 172.21.15.39:0/1980802999 <== mon.1 v2:172.21.15.161:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8484+0+0 (secure 0 0 0) 0x7f24bc011470 con 0x561e7c2f21b0 2024-09-21T03:47:05.887 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:05.881+0000 7f24dd79c740 1 -- 172.21.15.39:0/1980802999 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x561e7c6d6b20 con 0x561e7c2f21b0 2024-09-21T03:47:05.887 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:05.881+0000 7f24affff640 1 -- 172.21.15.39:0/1980802999 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f24bc035050 con 0x561e7c2f21b0 2024-09-21T03:47:05.887 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:05.881+0000 7f24dd79c740 10 client.25375 fetch_fsmap learned FSMap version 29 2024-09-21T03:47:05.887 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:05.881+0000 7f24dd79c740 1 -- 172.21.15.39:0/1980802999 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x561e7c753b70 con 0x561e7c2f21b0 2024-09-21T03:47:05.887 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:05.881+0000 7f24affff640 1 -- 172.21.15.39:0/1980802999 <== mon.1 v2:172.21.15.161:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x561e7c6d6b20 con 0x561e7c2f21b0 2024-09-21T03:47:05.887 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:05.882+0000 7f24affff640 1 -- 172.21.15.39:0/1980802999 <== mon.1 v2:172.21.15.161:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f24bc00f040 con 0x561e7c2f21b0 2024-09-21T03:47:05.887 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:05.882+0000 7f24dd79c740 10 client.25375 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T03:47:05.887 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:05.882+0000 7f24dd79c740 10 client.25375 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T03:47:05.888 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:05.882+0000 7f24dd79c740 20 client.25375 populate_metadata read hostname 'smithi039' 2024-09-21T03:47:05.888 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:05.882+0000 7f24dd79c740 1 --2- 172.21.15.39:0/1980802999 >> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] conn(0x561e7c7540f0 0x561e7c756510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:47:05.888 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:05.882+0000 7f24dd79c740 4 client.25375 mds_command: new command op to 24695 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-21T03:47:05.888 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:05.882+0000 7f24dd79c740 1 -- 172.21.15.39:0/1980802999 --> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] -- command(tid 0: {"prefix": "scrub status"}) -- 0x561e7c63ada0 con 0x561e7c7540f0 2024-09-21T03:47:05.888 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:05.882+0000 7f24d4fc4640 1 --2- 172.21.15.39:0/1980802999 >> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] conn(0x561e7c7540f0 0x561e7c756510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:47:05.888 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:05.883+0000 7f24affff640 10 client.25375 ms_handle_connect on v2:172.21.15.150:6838/2949386907 2024-09-21T03:47:05.888 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:05.883+0000 7f24d4fc4640 1 --2- 172.21.15.39:0/1980802999 >> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] conn(0x561e7c7540f0 0x561e7c756510 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=77988ea8231b3921 server_cookie=3707ba9c5cbac27c in_seq=0 out_seq=0 2024-09-21T03:47:05.888 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:05.883+0000 7f24affff640 1 -- 172.21.15.39:0/1980802999 <== mds.0 v2:172.21.15.150:6838/2949386907 1 ==== command_reply(tid 0: 0 ) ==== 8+0+658 (crc 0 0 0) 0x561e7c63ada0 con 0x561e7c7540f0 2024-09-21T03:47:05.888 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:05.883+0000 7f24affff640 10 client.25375 handle_command_reply: tid=0 2024-09-21T03:47:05.888 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:05.883+0000 7f24dd79c740 1 client.25375 shutdown 2024-09-21T03:47:05.888 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:05.883+0000 7f24dd79c740 1 -- 172.21.15.39:0/1980802999 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561e7c2f21b0 msgr2=0x561e7c6daf70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:47:05.888 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:05.883+0000 7f24dd79c740 1 --2- 172.21.15.39:0/1980802999 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561e7c2f21b0 0x561e7c6daf70 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f24bc009ab0 tx=0x7f24bc00efc0 comp rx=0 tx=0).stop 2024-09-21T03:47:05.889 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:05.884+0000 7f24dd79c740 1 -- 172.21.15.39:0/1980802999 shutdown_connections 2024-09-21T03:47:05.889 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:05.884+0000 7f24dd79c740 1 --2- 172.21.15.39:0/1980802999 >> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] conn(0x561e7c7540f0 0x561e7c756510 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:05.889 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:05.884+0000 7f24dd79c740 1 --2- 172.21.15.39:0/1980802999 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561e7c31b070 0x561e7c6db4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:05.889 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:05.884+0000 7f24affff640 0 client.25375 ms_handle_reset on v2:172.21.15.150:6838/2949386907 2024-09-21T03:47:05.889 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:05.884+0000 7f24dd79c740 1 --2- 172.21.15.39:0/1980802999 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561e7c2f21b0 0x561e7c6daf70 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:05.889 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:05.884+0000 7f24dd79c740 1 --2- 172.21.15.39:0/1980802999 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561e7c2d99f0 0x561e7c6dea10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:05.889 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:05.884+0000 7f24dd79c740 1 -- 172.21.15.39:0/1980802999 >> 172.21.15.39:0/1980802999 conn(0x561e7c30f460 msgr2=0x561e7c6d7e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:47:05.890 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-09-21T03:47:05.890 INFO:teuthology.orchestra.run.smithi039.stdout: "status": "no active scrubs running", 2024-09-21T03:47:05.890 INFO:teuthology.orchestra.run.smithi039.stdout: "scrubs": {}, 2024-09-21T03:47:05.890 INFO:teuthology.orchestra.run.smithi039.stdout: "scrub_stats": { 2024-09-21T03:47:05.890 INFO:teuthology.orchestra.run.smithi039.stdout: "10728de8-c49d-4edc-be2b-639249255e94": { 2024-09-21T03:47:05.890 INFO:teuthology.orchestra.run.smithi039.stdout: "start_time": "2024-09-21T03:47:03.965623+0000", 2024-09-21T03:47:05.890 INFO:teuthology.orchestra.run.smithi039.stdout: "path": "/", 2024-09-21T03:47:05.890 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_started": 0, 2024-09-21T03:47:05.891 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_passed": 0, 2024-09-21T03:47:05.891 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_failed": 0, 2024-09-21T03:47:05.891 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_skipped": 0 2024-09-21T03:47:05.891 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:47:05.891 INFO:teuthology.orchestra.run.smithi039.stdout: "02509e4f-3911-4983-b16f-df8ded0b9998": { 2024-09-21T03:47:05.891 INFO:teuthology.orchestra.run.smithi039.stdout: "start_time": "2024-09-21T03:46:17.889576+0000", 2024-09-21T03:47:05.891 INFO:teuthology.orchestra.run.smithi039.stdout: "path": "/", 2024-09-21T03:47:05.891 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_started": 0, 2024-09-21T03:47:05.892 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_passed": 0, 2024-09-21T03:47:05.892 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_failed": 0, 2024-09-21T03:47:05.892 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_skipped": 0 2024-09-21T03:47:05.892 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-21T03:47:05.892 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-21T03:47:05.892 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-09-21T03:47:05.892 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:05.884+0000 7f24dd79c740 1 -- 172.21.15.39:0/1980802999 shutdown_connections 2024-09-21T03:47:05.893 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:05.884+0000 7f24dd79c740 1 -- 172.21.15.39:0/1980802999 wait complete. 2024-09-21T03:47:05.893 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:05.884+0000 7f24dd79c740 20 client.25375 trim_cache size 0 max 16384 2024-09-21T03:47:05.893 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:05.884+0000 7f24d7228640 1 -- 172.21.15.39:0/1515058788 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f24c00b6810 msgr2=0x7f24c00b8d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:47:05.893 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:05.885+0000 7f24d7228640 1 --2- 172.21.15.39:0/1515058788 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f24c00b6810 0x7f24c00b8d20 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f24c8006010 tx=0x7f24c8005d80 comp rx=0 tx=0).stop 2024-09-21T03:47:05.893 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:05.885+0000 7f24d7228640 1 -- 172.21.15.39:0/1515058788 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f24d0102a60 msgr2=0x7f24d01993e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:47:05.893 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:05.885+0000 7f24d7228640 1 --2- 172.21.15.39:0/1515058788 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f24d0102a60 0x7f24d01993e0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f24c400ea20 tx=0x7f24c400eee0 comp rx=0 tx=0).stop 2024-09-21T03:47:05.893 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:05.885+0000 7f24d7228640 1 -- 172.21.15.39:0/1515058788 shutdown_connections 2024-09-21T03:47:05.893 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:05.885+0000 7f24d7228640 1 --2- 172.21.15.39:0/1515058788 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f24c00b6810 0x7f24c00b8d20 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:05.893 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:05.885+0000 7f24d7228640 1 --2- 172.21.15.39:0/1515058788 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f24d0108cf0 0x7f24d019ec20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:05.894 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:05.885+0000 7f24d7228640 1 --2- 172.21.15.39:0/1515058788 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f24d01083a0 0x7f24d0199980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:05.894 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:05.885+0000 7f24d7228640 1 --2- 172.21.15.39:0/1515058788 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f24d0102a60 0x7f24d01993e0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:05.894 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:05.885+0000 7f24d7228640 1 -- 172.21.15.39:0/1515058788 >> 172.21.15.39:0/1515058788 conn(0x7f24d0079730 msgr2=0x7f24d0104410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:47:05.894 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:05.885+0000 7f24d7228640 1 -- 172.21.15.39:0/1515058788 shutdown_connections 2024-09-21T03:47:05.894 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:05.885+0000 7f24d7228640 1 -- 172.21.15.39:0/1515058788 wait complete. 2024-09-21T03:47:05.968 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:47:05 smithi161 ceph-mon[34533]: scrub queued for path: / 2024-09-21T03:47:05.968 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:47:05 smithi161 ceph-mon[34533]: scrub summary: idle+waiting paths [/] 2024-09-21T03:47:05.968 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:47:05 smithi161 ceph-mon[34533]: scrub summary: active paths [/] 2024-09-21T03:47:05.968 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:47:05 smithi161 ceph-mon[34533]: scrub summary: idle+waiting paths [/] 2024-09-21T03:47:05.968 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:47:05 smithi161 ceph-mon[34533]: scrub summary: idle 2024-09-21T03:47:05.975 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:47:05 smithi150 ceph-mon[34289]: scrub queued for path: / 2024-09-21T03:47:05.975 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:47:05 smithi150 ceph-mon[34289]: scrub summary: idle+waiting paths [/] 2024-09-21T03:47:05.975 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:47:05 smithi150 ceph-mon[34289]: scrub summary: active paths [/] 2024-09-21T03:47:05.975 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:47:05 smithi150 ceph-mon[34289]: scrub summary: idle+waiting paths [/] 2024-09-21T03:47:05.975 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:47:05 smithi150 ceph-mon[34289]: scrub summary: idle 2024-09-21T03:47:06.084 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:47:05 smithi039 ceph-mon[31211]: scrub queued for path: / 2024-09-21T03:47:06.084 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:47:05 smithi039 ceph-mon[31211]: scrub summary: idle+waiting paths [/] 2024-09-21T03:47:06.084 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:47:05 smithi039 ceph-mon[31211]: scrub summary: active paths [/] 2024-09-21T03:47:06.084 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:47:05 smithi039 ceph-mon[31211]: scrub summary: idle+waiting paths [/] 2024-09-21T03:47:06.084 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:47:05 smithi039 ceph-mon[31211]: scrub summary: idle 2024-09-21T03:47:06.743 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-21T03:47:06.744 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 51116d5c-77ca-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T03:47:06.834 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:47:06 smithi039 ceph-mon[31211]: pgmap v357: 97 pgs: 97 active+clean; 1.4 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 23 KiB/s wr, 27 op/s 2024-09-21T03:47:06.967 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:47:06 smithi161 ceph-mon[34533]: pgmap v357: 97 pgs: 97 active+clean; 1.4 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 23 KiB/s wr, 27 op/s 2024-09-21T03:47:06.974 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:47:06 smithi150 ceph-mon[34289]: pgmap v357: 97 pgs: 97 active+clean; 1.4 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 23 KiB/s wr, 27 op/s 2024-09-21T03:47:07.043 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/51116d5c-77ca-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:47:07.629 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:07.626+0000 7fcc94bcb640 1 -- 172.21.15.39:0/4240977207 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcc90104880 msgr2=0x7fcc90104c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:47:07.630 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:07.626+0000 7fcc94bcb640 1 --2- 172.21.15.39:0/4240977207 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcc90104880 0x7fcc90104c90 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7fcc80009900 tx=0x7fcc8002a410 comp rx=0 tx=0).stop 2024-09-21T03:47:07.630 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:07.626+0000 7fcc94bcb640 1 -- 172.21.15.39:0/4240977207 shutdown_connections 2024-09-21T03:47:07.630 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:07.626+0000 7fcc94bcb640 1 --2- 172.21.15.39:0/4240977207 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcc9010de30 0x7fcc90110300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:07.630 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:07.626+0000 7fcc94bcb640 1 --2- 172.21.15.39:0/4240977207 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcc901052b0 0x7fcc9010d8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:07.630 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:07.626+0000 7fcc94bcb640 1 --2- 172.21.15.39:0/4240977207 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcc90104880 0x7fcc90104c90 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:07.630 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:07.626+0000 7fcc94bcb640 1 -- 172.21.15.39:0/4240977207 >> 172.21.15.39:0/4240977207 conn(0x7fcc90079730 msgr2=0x7fcc90079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:47:07.630 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:07.627+0000 7fcc94bcb640 1 -- 172.21.15.39:0/4240977207 shutdown_connections 2024-09-21T03:47:07.630 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:07.627+0000 7fcc94bcb640 1 -- 172.21.15.39:0/4240977207 wait complete. 2024-09-21T03:47:07.631 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:07.628+0000 7fcc94bcb640 1 Processor -- start 2024-09-21T03:47:07.631 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:07.628+0000 7fcc94bcb640 1 -- start start 2024-09-21T03:47:07.632 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:07.629+0000 7fcc94bcb640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcc90104880 0x7fcc90198270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:47:07.632 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:07.629+0000 7fcc94bcb640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcc901052b0 0x7fcc90198810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:47:07.632 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:07.629+0000 7fcc94bcb640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcc9010de30 0x7fcc9019dab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:47:07.633 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:07.629+0000 7fcc94bcb640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fcc90111880 con 0x7fcc9010de30 2024-09-21T03:47:07.633 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:07.629+0000 7fcc94bcb640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fcc90111700 con 0x7fcc901052b0 2024-09-21T03:47:07.633 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:07.629+0000 7fcc94bcb640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fcc90111a00 con 0x7fcc90104880 2024-09-21T03:47:07.633 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:07.629+0000 7fcc8dd9b640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcc901052b0 0x7fcc90198810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:47:07.633 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:07.629+0000 7fcc8e59c640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcc90104880 0x7fcc90198270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:47:07.633 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:07.629+0000 7fcc8dd9b640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcc901052b0 0x7fcc90198810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.39:56842/0 (socket says 172.21.15.39:56842) 2024-09-21T03:47:07.633 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:07.629+0000 7fcc8dd9b640 1 -- 172.21.15.39:0/1921151407 learned_addr learned my addr 172.21.15.39:0/1921151407 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:47:07.633 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:07.629+0000 7fcc8e59c640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcc90104880 0x7fcc90198270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.39:37756/0 (socket says 172.21.15.39:37756) 2024-09-21T03:47:07.634 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:07.629+0000 7fcc8ed9d640 1 --2- 172.21.15.39:0/1921151407 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcc9010de30 0x7fcc9019dab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:47:07.635 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:07.629+0000 7fcc8e59c640 1 -- 172.21.15.39:0/1921151407 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcc901052b0 msgr2=0x7fcc90198810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:47:07.635 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:07.629+0000 7fcc8e59c640 1 --2- 172.21.15.39:0/1921151407 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcc901052b0 0x7fcc90198810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:07.635 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:07.629+0000 7fcc8e59c640 1 -- 172.21.15.39:0/1921151407 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcc9010de30 msgr2=0x7fcc9019dab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:47:07.635 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:07.629+0000 7fcc8e59c640 1 --2- 172.21.15.39:0/1921151407 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcc9010de30 0x7fcc9019dab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:07.635 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:07.629+0000 7fcc8e59c640 1 -- 172.21.15.39:0/1921151407 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcc9019e1e0 con 0x7fcc90104880 2024-09-21T03:47:07.635 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:07.630+0000 7fcc8dd9b640 1 --2- 172.21.15.39:0/1921151407 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcc901052b0 0x7fcc90198810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:47:07.635 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:07.630+0000 7fcc8e59c640 1 --2- 172.21.15.39:0/1921151407 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcc90104880 0x7fcc90198270 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fcc80006010 tx=0x7fcc80004060 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:47:07.635 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:07.630+0000 7fcc7f7fe640 1 -- 172.21.15.39:0/1921151407 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc8003f070 con 0x7fcc90104880 2024-09-21T03:47:07.636 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:07.630+0000 7fcc7f7fe640 1 -- 172.21.15.39:0/1921151407 <== mon.1 v2:172.21.15.161:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcc80003ca0 con 0x7fcc90104880 2024-09-21T03:47:07.636 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:07.630+0000 7fcc7f7fe640 1 -- 172.21.15.39:0/1921151407 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc80035070 con 0x7fcc90104880 2024-09-21T03:47:07.636 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:07.630+0000 7fcc94bcb640 1 -- 172.21.15.39:0/1921151407 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcc9019e470 con 0x7fcc90104880 2024-09-21T03:47:07.636 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:07.630+0000 7fcc94bcb640 1 -- 172.21.15.39:0/1921151407 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcc9019e890 con 0x7fcc90104880 2024-09-21T03:47:07.636 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:07.632+0000 7fcc94bcb640 1 -- 172.21.15.39:0/1921151407 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcc90104c90 con 0x7fcc90104880 2024-09-21T03:47:07.637 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:07.634+0000 7fcc7f7fe640 1 -- 172.21.15.39:0/1921151407 <== mon.1 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 26) ==== 154694+0+0 (secure 0 0 0) 0x7fcc8003b460 con 0x7fcc90104880 2024-09-21T03:47:07.638 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:07.636+0000 7fcc7f7fe640 1 --2- 172.21.15.39:0/1921151407 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7fcc780b6710 0x7fcc780b8c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:47:07.639 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:07.636+0000 7fcc8dd9b640 1 --2- 172.21.15.39:0/1921151407 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7fcc780b6710 0x7fcc780b8c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:47:07.639 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:07.636+0000 7fcc7f7fe640 1 -- 172.21.15.39:0/1921151407 <== mon.1 v2:172.21.15.161:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8484+0+0 (secure 0 0 0) 0x7fcc80107f60 con 0x7fcc90104880 2024-09-21T03:47:07.639 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:07.637+0000 7fcc8dd9b640 1 --2- 172.21.15.39:0/1921151407 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7fcc780b6710 0x7fcc780b8c20 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fcc901024e0 tx=0x7fcc84005f90 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:47:07.640 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:07.638+0000 7fcc7f7fe640 1 -- 172.21.15.39:0/1921151407 <== mon.1 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fcc800d26e0 con 0x7fcc90104880 2024-09-21T03:47:08.103 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:08.100+0000 7fcc94bcb640 1 -- 172.21.15.39:0/1921151407 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fcc901993a0 con 0x7fcc90104880 2024-09-21T03:47:08.104 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:08.101+0000 7fcc7f7fe640 1 -- 172.21.15.39:0/1921151407 <== mon.1 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11978 (secure 0 0 0) 0x7fcc800d7620 con 0x7fcc90104880 2024-09-21T03:47:08.104 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T03:47:08.104 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 29 2024-09-21T03:47:08.105 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":29,"btime":"2024-09-21T03:46:08:982881+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14790,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6835/508995295","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":508995295},{"type":"v1","addr":"172.21.15.39:6835","nonce":508995295}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14802,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6837/3915912607","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":3915912607},{"type":"v1","addr":"172.21.15.39:6837","nonce":3915912607}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":29,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T03:44:15.009872+0000","modified":"2024-09-21T03:46:08.430113+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24695,"mds_1":24667,"mds_2":24659,"mds_3":24646,"mds_4":14868},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14856":{"gid":14856,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.39:6839/185509192","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6838","nonce":185509192},{"type":"v1","addr":"172.21.15.39:6839","nonce":185509192}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14868":{"gid":14868,"name":"a","rank":4,"incarnation":25,"state":"up:active","state_seq":7,"addr":"172.21.15.39:6841/4247190456","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6840","nonce":4247190456},{"type":"v1","addr":"172.21.15.39:6841","nonce":4247190456}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24599":{"gid":24599,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.150:6833/3114643217","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":3114643217},{"type":"v1","addr":"172.21.15.150:6833","nonce":3114643217}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24601":{"gid":24601,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.161:6833/2963990654","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":2963990654},{"type":"v1","addr":"172.21.15.161:6833","nonce":2963990654}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24632":{"gid":24632,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.150:6835/724594008","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6834","nonce":724594008},{"type":"v1","addr":"172.21.15.150:6835","nonce":724594008}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24640":{"gid":24640,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.161:6835/3747882110","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":3747882110},{"type":"v1","addr":"172.21.15.161:6835","nonce":3747882110}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24646":{"gid":24646,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":10,"addr":"172.21.15.161:6837/3451544597","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6836","nonce":3451544597},{"type":"v1","addr":"172.21.15.161:6837","nonce":3451544597}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24659":{"gid":24659,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.150:6837/4017043144","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6836","nonce":4017043144},{"type":"v1","addr":"172.21.15.150:6837","nonce":4017043144}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24667":{"gid":24667,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.161:6839/2190107270","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6838","nonce":2190107270},{"type":"v1","addr":"172.21.15.161:6839","nonce":2190107270}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24695":{"gid":24695,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.150:6839/2949386907","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6838","nonce":2949386907},{"type":"v1","addr":"172.21.15.150:6839","nonce":2949386907}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24695,"qdb_cluster":[24695,24667,24659,24646,14868]},"id":1}]} 2024-09-21T03:47:08.108 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:08.104+0000 7fcc94bcb640 1 -- 172.21.15.39:0/1921151407 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7fcc780b6710 msgr2=0x7fcc780b8c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:47:08.108 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:08.104+0000 7fcc94bcb640 1 --2- 172.21.15.39:0/1921151407 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7fcc780b6710 0x7fcc780b8c20 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fcc901024e0 tx=0x7fcc84005f90 comp rx=0 tx=0).stop 2024-09-21T03:47:08.108 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:08.104+0000 7fcc94bcb640 1 -- 172.21.15.39:0/1921151407 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcc90104880 msgr2=0x7fcc90198270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:47:08.108 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:08.104+0000 7fcc94bcb640 1 --2- 172.21.15.39:0/1921151407 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcc90104880 0x7fcc90198270 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fcc80006010 tx=0x7fcc80004060 comp rx=0 tx=0).stop 2024-09-21T03:47:08.108 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:08.104+0000 7fcc94bcb640 1 -- 172.21.15.39:0/1921151407 shutdown_connections 2024-09-21T03:47:08.108 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:08.104+0000 7fcc94bcb640 1 --2- 172.21.15.39:0/1921151407 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7fcc780b6710 0x7fcc780b8c20 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:08.108 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:08.104+0000 7fcc94bcb640 1 --2- 172.21.15.39:0/1921151407 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcc9010de30 0x7fcc9019dab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:08.108 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:08.104+0000 7fcc94bcb640 1 --2- 172.21.15.39:0/1921151407 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcc901052b0 0x7fcc90198810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:08.108 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:08.104+0000 7fcc94bcb640 1 --2- 172.21.15.39:0/1921151407 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcc90104880 0x7fcc90198270 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:08.108 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:08.104+0000 7fcc94bcb640 1 -- 172.21.15.39:0/1921151407 >> 172.21.15.39:0/1921151407 conn(0x7fcc90079730 msgr2=0x7fcc90100890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:47:08.108 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:08.105+0000 7fcc94bcb640 1 -- 172.21.15.39:0/1921151407 shutdown_connections 2024-09-21T03:47:08.108 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:08.105+0000 7fcc94bcb640 1 -- 172.21.15.39:0/1921151407 wait complete. 2024-09-21T03:47:08.834 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:47:08 smithi039 ceph-mon[31211]: pgmap v358: 97 pgs: 97 active+clean; 1.4 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 24 op/s 2024-09-21T03:47:08.834 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:47:08 smithi039 ceph-mon[31211]: from='client.? 172.21.15.39:0/1921151407' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:47:08.908 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 51116d5c-77ca-11ef-baf5-efdc52797490 -- ceph tell mds.1:4 damage ls 2024-09-21T03:47:08.968 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:47:08 smithi161 ceph-mon[34533]: pgmap v358: 97 pgs: 97 active+clean; 1.4 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 24 op/s 2024-09-21T03:47:08.968 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:47:08 smithi161 ceph-mon[34533]: from='client.? 172.21.15.39:0/1921151407' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:47:08.975 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:47:08 smithi150 ceph-mon[34289]: pgmap v358: 97 pgs: 97 active+clean; 1.4 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 24 op/s 2024-09-21T03:47:08.975 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:47:08 smithi150 ceph-mon[34289]: from='client.? 172.21.15.39:0/1921151407' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:47:09.195 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/51116d5c-77ca-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:47:09.706 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:09.704+0000 7f2b7126f640 1 -- 172.21.15.39:0/372011286 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2b6c10c020 msgr2=0x7f2b6c112850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:47:09.706 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:09.704+0000 7f2b7126f640 1 --2- 172.21.15.39:0/372011286 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2b6c10c020 0x7f2b6c112850 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f2b64009ca0 tx=0x7f2b6402a8c0 comp rx=0 tx=0).stop 2024-09-21T03:47:09.706 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:09.704+0000 7f2b7126f640 1 -- 172.21.15.39:0/372011286 shutdown_connections 2024-09-21T03:47:09.707 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:09.704+0000 7f2b7126f640 1 --2- 172.21.15.39:0/372011286 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2b6c10c020 0x7f2b6c112850 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:09.707 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:09.704+0000 7f2b7126f640 1 --2- 172.21.15.39:0/372011286 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2b6c10b6d0 0x7f2b6c10bae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:09.707 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:09.704+0000 7f2b7126f640 1 --2- 172.21.15.39:0/372011286 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2b6c105d90 0x7f2b6c1061a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:09.707 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:09.704+0000 7f2b7126f640 1 -- 172.21.15.39:0/372011286 >> 172.21.15.39:0/372011286 conn(0x7f2b6c101860 msgr2=0x7f2b6c103cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:47:09.707 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:09.705+0000 7f2b7126f640 1 -- 172.21.15.39:0/372011286 shutdown_connections 2024-09-21T03:47:09.707 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:09.705+0000 7f2b7126f640 1 -- 172.21.15.39:0/372011286 wait complete. 2024-09-21T03:47:09.707 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:09.706+0000 7f2b7126f640 1 Processor -- start 2024-09-21T03:47:09.708 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:09.706+0000 7f2b7126f640 1 -- start start 2024-09-21T03:47:09.709 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:09.706+0000 7f2b7126f640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2b6c105d90 0x7f2b6c19c480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:47:09.709 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:09.706+0000 7f2b7126f640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2b6c10b6d0 0x7f2b6c19ca20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:47:09.709 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:09.706+0000 7f2b7126f640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2b6c10c020 0x7f2b6c1a1cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:47:09.709 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:09.707+0000 7f2b7126f640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f2b6c115ce0 con 0x7f2b6c10c020 2024-09-21T03:47:09.710 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:09.707+0000 7f2b7126f640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f2b6c115b60 con 0x7f2b6c105d90 2024-09-21T03:47:09.710 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:09.707+0000 7f2b7126f640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f2b6c115e60 con 0x7f2b6c10b6d0 2024-09-21T03:47:09.710 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:09.707+0000 7f2b6a7fc640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2b6c10b6d0 0x7f2b6c19ca20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:47:09.710 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:09.707+0000 7f2b6a7fc640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2b6c10b6d0 0x7f2b6c19ca20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.39:37786/0 (socket says 172.21.15.39:37786) 2024-09-21T03:47:09.710 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:09.707+0000 7f2b6a7fc640 1 -- 172.21.15.39:0/3855483620 learned_addr learned my addr 172.21.15.39:0/3855483620 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:47:09.710 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:09.707+0000 7f2b6affd640 1 --2- 172.21.15.39:0/3855483620 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2b6c105d90 0x7f2b6c19c480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:47:09.710 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:09.707+0000 7f2b6b7fe640 1 --2- 172.21.15.39:0/3855483620 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2b6c10c020 0x7f2b6c1a1cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:47:09.710 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:09.707+0000 7f2b6a7fc640 1 -- 172.21.15.39:0/3855483620 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2b6c105d90 msgr2=0x7f2b6c19c480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:47:09.710 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:09.707+0000 7f2b6a7fc640 1 --2- 172.21.15.39:0/3855483620 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2b6c105d90 0x7f2b6c19c480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:09.710 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:09.707+0000 7f2b6a7fc640 1 -- 172.21.15.39:0/3855483620 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2b6c10c020 msgr2=0x7f2b6c1a1cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:47:09.710 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:09.707+0000 7f2b6a7fc640 1 --2- 172.21.15.39:0/3855483620 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2b6c10c020 0x7f2b6c1a1cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:09.710 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:09.707+0000 7f2b6a7fc640 1 -- 172.21.15.39:0/3855483620 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2b6c1a23f0 con 0x7f2b6c10b6d0 2024-09-21T03:47:09.710 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:09.707+0000 7f2b6b7fe640 1 --2- 172.21.15.39:0/3855483620 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2b6c10c020 0x7f2b6c1a1cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:47:09.711 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:09.707+0000 7f2b6a7fc640 1 --2- 172.21.15.39:0/3855483620 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2b6c10b6d0 0x7f2b6c19ca20 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f2b6000e9f0 tx=0x7f2b6000eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:47:09.711 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:09.708+0000 7f2b53fff640 1 -- 172.21.15.39:0/3855483620 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b6000ccc0 con 0x7f2b6c10b6d0 2024-09-21T03:47:09.711 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:09.708+0000 7f2b7126f640 1 -- 172.21.15.39:0/3855483620 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2b6c1a26e0 con 0x7f2b6c10b6d0 2024-09-21T03:47:09.711 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:09.708+0000 7f2b53fff640 1 -- 172.21.15.39:0/3855483620 <== mon.1 v2:172.21.15.161:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2b600108e0 con 0x7f2b6c10b6d0 2024-09-21T03:47:09.711 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:09.708+0000 7f2b53fff640 1 -- 172.21.15.39:0/3855483620 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b60018860 con 0x7f2b6c10b6d0 2024-09-21T03:47:09.711 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:09.708+0000 7f2b7126f640 1 -- 172.21.15.39:0/3855483620 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2b6c1a8e40 con 0x7f2b6c10b6d0 2024-09-21T03:47:09.712 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:09.711+0000 7f2b777e3740 1 Processor -- start 2024-09-21T03:47:09.712 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:09.711+0000 7f2b777e3740 1 -- start start 2024-09-21T03:47:09.714 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:09.711+0000 7f2b777e3740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5625c73fb070 0x5625c77a26a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:47:09.714 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:09.711+0000 7f2b777e3740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5625c73d21b0 0x5625c77a2be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:47:09.714 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:09.711+0000 7f2b777e3740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5625c73da390 0x5625c77a3120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:47:09.714 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:09.711+0000 7f2b777e3740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x5625c7718ea0 con 0x5625c73da390 2024-09-21T03:47:09.714 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:09.711+0000 7f2b777e3740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x5625c77082d0 con 0x5625c73d21b0 2024-09-21T03:47:09.714 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:09.711+0000 7f2b777e3740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x5625c770c0e0 con 0x5625c73fb070 2024-09-21T03:47:09.714 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:09.711+0000 7f2b53fff640 1 -- 172.21.15.39:0/3855483620 <== mon.1 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 26) ==== 154694+0+0 (secure 0 0 0) 0x7f2b60010430 con 0x7f2b6c10b6d0 2024-09-21T03:47:09.714 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:09.712+0000 7f2b6b7fe640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5625c73fb070 0x5625c77a26a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:47:09.715 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:09.712+0000 7f2b6b7fe640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5625c73fb070 0x5625c77a26a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.39:37802/0 (socket says 172.21.15.39:37802) 2024-09-21T03:47:09.715 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:09.712+0000 7f2b6b7fe640 1 -- 172.21.15.39:0/3335056298 learned_addr learned my addr 172.21.15.39:0/3335056298 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:47:09.715 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:09.712+0000 7f2b6a7fc640 1 --2- 172.21.15.39:0/3335056298 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5625c73da390 0x5625c77a3120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:47:09.715 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:09.712+0000 7f2b6affd640 1 --2- 172.21.15.39:0/3335056298 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5625c73d21b0 0x5625c77a2be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:47:09.715 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:09.712+0000 7f2b6b7fe640 1 -- 172.21.15.39:0/3335056298 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5625c73d21b0 msgr2=0x5625c77a2be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:47:09.715 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:09.712+0000 7f2b6b7fe640 1 --2- 172.21.15.39:0/3335056298 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5625c73d21b0 0x5625c77a2be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:09.715 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:09.712+0000 7f2b6b7fe640 1 -- 172.21.15.39:0/3335056298 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5625c73da390 msgr2=0x5625c77a3120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:47:09.715 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:09.712+0000 7f2b6b7fe640 1 --2- 172.21.15.39:0/3335056298 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5625c73da390 0x5625c77a3120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:09.715 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:09.712+0000 7f2b6b7fe640 1 -- 172.21.15.39:0/3335056298 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5625c7466650 con 0x5625c73fb070 2024-09-21T03:47:09.715 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:09.712+0000 7f2b6affd640 1 --2- 172.21.15.39:0/3335056298 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5625c73d21b0 0x5625c77a2be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:47:09.717 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:09.713+0000 7f2b6b7fe640 1 --2- 172.21.15.39:0/3335056298 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5625c73fb070 0x5625c77a26a0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f2b64009420 tx=0x7f2b64009450 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d37fc33203d46be3 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:47:09.717 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:09.713+0000 7f2b53fff640 1 --2- 172.21.15.39:0/3855483620 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f2b540b6790 0x7f2b540b8ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:47:09.717 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:09.713+0000 7f2b6affd640 1 --2- 172.21.15.39:0/3855483620 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f2b540b6790 0x7f2b540b8ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:47:09.717 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:09.713+0000 7f2b53fff640 1 -- 172.21.15.39:0/3855483620 <== mon.1 v2:172.21.15.161:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8484+0+0 (secure 0 0 0) 0x7f2b600e7b50 con 0x7f2b6c10b6d0 2024-09-21T03:47:09.717 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:09.713+0000 7f2b517fa640 1 -- 172.21.15.39:0/3335056298 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b64031af0 con 0x5625c73fb070 2024-09-21T03:47:09.717 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:09.713+0000 7f2b517fa640 1 -- 172.21.15.39:0/3335056298 <== mon.1 v2:172.21.15.161:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2b64036040 con 0x5625c73fb070 2024-09-21T03:47:09.717 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:09.713+0000 7f2b517fa640 1 -- 172.21.15.39:0/3335056298 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b64004040 con 0x5625c73fb070 2024-09-21T03:47:09.718 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:09.714+0000 7f2b777e3740 1 -- 172.21.15.39:0/3335056298 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5625c73fb070 msgr2=0x5625c77a26a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:47:09.718 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:09.714+0000 7f2b777e3740 1 --2- 172.21.15.39:0/3335056298 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5625c73fb070 0x5625c77a26a0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f2b64009420 tx=0x7f2b64009450 comp rx=0 tx=0).stop 2024-09-21T03:47:09.718 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:09.714+0000 7f2b6affd640 1 --2- 172.21.15.39:0/3855483620 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f2b540b6790 0x7f2b540b8ca0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x5625c77a1320 tx=0x7f2b5c005f90 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:47:09.718 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:09.714+0000 7f2b777e3740 1 -- 172.21.15.39:0/3335056298 shutdown_connections 2024-09-21T03:47:09.718 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:09.714+0000 7f2b777e3740 1 --2- 172.21.15.39:0/3335056298 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5625c73da390 0x5625c77a3120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:09.718 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:09.714+0000 7f2b777e3740 1 --2- 172.21.15.39:0/3335056298 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5625c73d21b0 0x5625c77a2be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:09.718 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:09.714+0000 7f2b777e3740 1 --2- 172.21.15.39:0/3335056298 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5625c73fb070 0x5625c77a26a0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:09.718 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:09.714+0000 7f2b777e3740 1 -- 172.21.15.39:0/3335056298 >> 172.21.15.39:0/3335056298 conn(0x5625c73ef460 msgr2=0x5625c77a0590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:47:09.718 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:09.714+0000 7f2b777e3740 1 -- 172.21.15.39:0/3335056298 shutdown_connections 2024-09-21T03:47:09.718 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:09.714+0000 7f2b777e3740 1 -- 172.21.15.39:0/3335056298 wait complete. 2024-09-21T03:47:09.718 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:09.715+0000 7f2b777e3740 1 -- start start 2024-09-21T03:47:09.718 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:09.715+0000 7f2b777e3740 1 Processor -- start 2024-09-21T03:47:09.718 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:09.715+0000 7f2b777e3740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5625c73d21b0 0x5625c77add70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:47:09.718 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:09.715+0000 7f2b777e3740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5625c73da390 0x5625c77ae2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:47:09.720 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:09.715+0000 7f2b777e3740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5625c73fb070 0x5625c77ba140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:47:09.720 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:09.715+0000 7f2b777e3740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x5625c7569700 con 0x5625c73da390 2024-09-21T03:47:09.720 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:09.715+0000 7f2b777e3740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x5625c76a3160 con 0x5625c73d21b0 2024-09-21T03:47:09.720 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:09.715+0000 7f2b777e3740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x5625c77b0e20 con 0x5625c73fb070 2024-09-21T03:47:09.721 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:09.715+0000 7f2b777e3740 10 client.0 _ll_register_callbacks cb 0x5625c76bc9e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:47:09.721 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:09.716+0000 7f2b6a7fc640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5625c73da390 0x5625c77ae2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:47:09.721 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:09.716+0000 7f2b6a7fc640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5625c73da390 0x5625c77ae2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:51188/0 (socket says 172.21.15.39:51188) 2024-09-21T03:47:09.721 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:09.716+0000 7f2b6a7fc640 1 -- 172.21.15.39:0/166760934 learned_addr learned my addr 172.21.15.39:0/166760934 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:47:09.721 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:09.716+0000 7f2b6affd640 1 --2- 172.21.15.39:0/166760934 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5625c73d21b0 0x5625c77add70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:47:09.721 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:09.716+0000 7f2b6b7fe640 1 --2- 172.21.15.39:0/166760934 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5625c73fb070 0x5625c77ba140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:47:09.721 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:09.716+0000 7f2b6a7fc640 1 -- 172.21.15.39:0/166760934 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5625c73fb070 msgr2=0x5625c77ba140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:47:09.721 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:09.716+0000 7f2b6a7fc640 1 --2- 172.21.15.39:0/166760934 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5625c73fb070 0x5625c77ba140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:09.721 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:09.716+0000 7f2b6a7fc640 1 -- 172.21.15.39:0/166760934 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5625c73d21b0 msgr2=0x5625c77add70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:47:09.721 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:09.716+0000 7f2b6a7fc640 1 --2- 172.21.15.39:0/166760934 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5625c73d21b0 0x5625c77add70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:09.721 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:09.716+0000 7f2b6a7fc640 1 -- 172.21.15.39:0/166760934 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5625c76d0c90 con 0x5625c73da390 2024-09-21T03:47:09.721 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:09.716+0000 7f2b6a7fc640 1 --2- 172.21.15.39:0/166760934 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5625c73da390 0x5625c77ae2b0 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f2b600e5290 tx=0x7f2b600e5880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5934e5ce5ae47b4b server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:47:09.721 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:09.716+0000 7f2b4eff5640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-09-21T03:47:09.722 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:09.719+0000 7f2b4eff5640 1 -- 172.21.15.39:0/166760934 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b600a9ea0 con 0x5625c73da390 2024-09-21T03:47:09.722 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:09.719+0000 7f2b4eff5640 1 -- 172.21.15.39:0/166760934 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8484+0+0 (secure 0 0 0) 0x7f2b600ac520 con 0x5625c73da390 2024-09-21T03:47:09.722 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:09.719+0000 7f2b777e3740 1 -- 172.21.15.39:0/166760934 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2b64036040 con 0x5625c73da390 2024-09-21T03:47:09.722 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:09.719+0000 7f2b4eff5640 1 -- 172.21.15.39:0/166760934 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f2b600acfc0 con 0x5625c73da390 2024-09-21T03:47:09.722 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:09.719+0000 7f2b777e3740 10 client.15780 fetch_fsmap learned FSMap version 29 2024-09-21T03:47:09.723 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:09.719+0000 7f2b777e3740 1 -- 172.21.15.39:0/166760934 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5625c77c53e0 con 0x5625c73da390 2024-09-21T03:47:09.723 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:09.720+0000 7f2b4eff5640 1 -- 172.21.15.39:0/166760934 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2b64036040 con 0x5625c73da390 2024-09-21T03:47:09.723 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:09.720+0000 7f2b4eff5640 1 -- 172.21.15.39:0/166760934 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f2b600ac720 con 0x5625c73da390 2024-09-21T03:47:09.723 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:09.720+0000 7f2b777e3740 10 client.15780 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T03:47:09.723 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:09.720+0000 7f2b777e3740 10 client.15780 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.a 2024-09-21T03:47:09.723 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:09.720+0000 7f2b777e3740 20 client.15780 populate_metadata read hostname 'smithi039' 2024-09-21T03:47:09.723 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:09.720+0000 7f2b777e3740 1 --2- 172.21.15.39:0/166760934 >> [v2:172.21.15.39:6840/4247190456,v1:172.21.15.39:6841/4247190456] conn(0x5625c73b99f0 0x5625c77c88b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:47:09.723 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:09.720+0000 7f2b6affd640 1 --2- 172.21.15.39:0/166760934 >> [v2:172.21.15.39:6840/4247190456,v1:172.21.15.39:6841/4247190456] conn(0x5625c73b99f0 0x5625c77c88b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:47:09.723 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:09.721+0000 7f2b777e3740 4 client.15780 mds_command: new command op to 14868 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:47:09.723 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:09.721+0000 7f2b777e3740 1 -- 172.21.15.39:0/166760934 --> [v2:172.21.15.39:6840/4247190456,v1:172.21.15.39:6841/4247190456] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5625c771adf0 con 0x5625c73b99f0 2024-09-21T03:47:09.724 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:09.721+0000 7f2b4eff5640 10 client.15780 ms_handle_connect on v2:172.21.15.39:6840/4247190456 2024-09-21T03:47:09.724 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:09.721+0000 7f2b6affd640 1 --2- 172.21.15.39:0/166760934 >> [v2:172.21.15.39:6840/4247190456,v1:172.21.15.39:6841/4247190456] conn(0x5625c73b99f0 0x5625c77c88b0 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=1ffa417e4ddb8fe server_cookie=f08a38dc43110e1c in_seq=0 out_seq=0 2024-09-21T03:47:09.724 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:09.722+0000 7f2b4eff5640 1 -- 172.21.15.39:0/166760934 <== mds.4 v2:172.21.15.39:6840/4247190456 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5625c771adf0 con 0x5625c73b99f0 2024-09-21T03:47:09.725 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:09.722+0000 7f2b4eff5640 10 client.15780 handle_command_reply: tid=0 2024-09-21T03:47:09.725 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:09.722+0000 7f2b777e3740 1 client.15780 shutdown 2024-09-21T03:47:09.725 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:09.722+0000 7f2b777e3740 1 -- 172.21.15.39:0/166760934 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5625c73da390 msgr2=0x5625c77ae2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:47:09.725 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:09.722+0000 7f2b777e3740 1 --2- 172.21.15.39:0/166760934 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5625c73da390 0x5625c77ae2b0 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f2b600e5290 tx=0x7f2b600e5880 comp rx=0 tx=0).stop 2024-09-21T03:47:09.725 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:09.723+0000 7f2b777e3740 1 -- 172.21.15.39:0/166760934 shutdown_connections 2024-09-21T03:47:09.725 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:09.723+0000 7f2b777e3740 1 --2- 172.21.15.39:0/166760934 >> [v2:172.21.15.39:6840/4247190456,v1:172.21.15.39:6841/4247190456] conn(0x5625c73b99f0 0x5625c77c88b0 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:09.725 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:09.723+0000 7f2b4eff5640 0 client.15780 ms_handle_reset on v2:172.21.15.39:6840/4247190456 2024-09-21T03:47:09.725 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:09.723+0000 7f2b777e3740 1 --2- 172.21.15.39:0/166760934 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5625c73fb070 0x5625c77ba140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:09.725 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:09.723+0000 7f2b777e3740 1 --2- 172.21.15.39:0/166760934 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5625c73da390 0x5625c77ae2b0 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:09.725 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:09.723+0000 7f2b777e3740 1 --2- 172.21.15.39:0/166760934 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5625c73d21b0 0x5625c77add70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:09.725 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:09.723+0000 7f2b777e3740 1 -- 172.21.15.39:0/166760934 >> 172.21.15.39:0/166760934 conn(0x5625c73ef460 msgr2=0x5625c77a5830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:47:09.725 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:09.723+0000 7f2b777e3740 1 -- 172.21.15.39:0/166760934 shutdown_connections 2024-09-21T03:47:09.725 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:09.723+0000 7f2b777e3740 1 -- 172.21.15.39:0/166760934 wait complete. 2024-09-21T03:47:09.726 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:09.723+0000 7f2b777e3740 20 client.15780 trim_cache size 0 max 16384 2024-09-21T03:47:09.740 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:09.738+0000 7f2b777e3740 1 Processor -- start 2024-09-21T03:47:09.740 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:09.739+0000 7f2b777e3740 1 -- start start 2024-09-21T03:47:09.742 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:09.739+0000 7f2b777e3740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5625c73b99f0 0x5625c77ae230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:47:09.742 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:09.739+0000 7f2b777e3740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5625c73d21b0 0x5625c77ba040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:47:09.742 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:09.739+0000 7f2b6affd640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5625c73b99f0 0x5625c77ae230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:47:09.743 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:09.739+0000 7f2b6affd640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5625c73b99f0 0x5625c77ae230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:51196/0 (socket says 172.21.15.39:51196) 2024-09-21T03:47:09.743 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:09.739+0000 7f2b6affd640 1 -- 172.21.15.39:0/1998308351 learned_addr learned my addr 172.21.15.39:0/1998308351 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:47:09.743 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:09.739+0000 7f2b777e3740 1 --2- 172.21.15.39:0/1998308351 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5625c73da390 0x5625c77aaf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:47:09.743 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:09.739+0000 7f2b777e3740 1 -- 172.21.15.39:0/1998308351 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f2b6c1ad600 con 0x5625c73b99f0 2024-09-21T03:47:09.743 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:09.739+0000 7f2b777e3740 1 -- 172.21.15.39:0/1998308351 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f2b6c1af140 con 0x5625c73d21b0 2024-09-21T03:47:09.743 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:09.739+0000 7f2b777e3740 1 -- 172.21.15.39:0/1998308351 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f2b6c1adcd0 con 0x5625c73da390 2024-09-21T03:47:09.743 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:09.739+0000 7f2b6a7fc640 1 --2- 172.21.15.39:0/1998308351 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5625c73d21b0 0x5625c77ba040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:47:09.743 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:09.740+0000 7f2b6affd640 1 -- 172.21.15.39:0/1998308351 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5625c73da390 msgr2=0x5625c77aaf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:47:09.743 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:09.740+0000 7f2b6affd640 1 --2- 172.21.15.39:0/1998308351 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5625c73da390 0x5625c77aaf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:09.743 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:09.740+0000 7f2b6affd640 1 -- 172.21.15.39:0/1998308351 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5625c73d21b0 msgr2=0x5625c77ba040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:47:09.743 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:09.740+0000 7f2b6affd640 1 --2- 172.21.15.39:0/1998308351 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5625c73d21b0 0x5625c77ba040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:09.743 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:09.740+0000 7f2b6affd640 1 -- 172.21.15.39:0/1998308351 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5625c77a4990 con 0x5625c73b99f0 2024-09-21T03:47:09.744 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:09.740+0000 7f2b6a7fc640 1 --2- 172.21.15.39:0/1998308351 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5625c73d21b0 0x5625c77ba040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:47:09.744 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:09.740+0000 7f2b6affd640 1 --2- 172.21.15.39:0/1998308351 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5625c73b99f0 0x5625c77ae230 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f2b5c012cb0 tx=0x7f2b5c01b000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=133cfd92c563dae1 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:47:09.744 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:09.740+0000 7f2b50ff9640 1 -- 172.21.15.39:0/1998308351 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b5c01dd20 con 0x5625c73b99f0 2024-09-21T03:47:09.744 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:09.741+0000 7f2b50ff9640 1 -- 172.21.15.39:0/1998308351 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2b5c034070 con 0x5625c73b99f0 2024-09-21T03:47:09.744 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:09.741+0000 7f2b50ff9640 1 -- 172.21.15.39:0/1998308351 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b5c029650 con 0x5625c73b99f0 2024-09-21T03:47:09.746 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:09.742+0000 7f2b777e3740 1 -- 172.21.15.39:0/1998308351 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5625c73b99f0 msgr2=0x5625c77ae230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:47:09.746 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:09.742+0000 7f2b777e3740 1 --2- 172.21.15.39:0/1998308351 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5625c73b99f0 0x5625c77ae230 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f2b5c012cb0 tx=0x7f2b5c01b000 comp rx=0 tx=0).stop 2024-09-21T03:47:09.746 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:09.742+0000 7f2b777e3740 1 -- 172.21.15.39:0/1998308351 shutdown_connections 2024-09-21T03:47:09.746 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:09.742+0000 7f2b777e3740 1 --2- 172.21.15.39:0/1998308351 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5625c73da390 0x5625c77aaf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:09.746 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:09.742+0000 7f2b777e3740 1 --2- 172.21.15.39:0/1998308351 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5625c73d21b0 0x5625c77ba040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:09.747 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:09.742+0000 7f2b777e3740 1 --2- 172.21.15.39:0/1998308351 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5625c73b99f0 0x5625c77ae230 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:09.747 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:09.742+0000 7f2b777e3740 1 -- 172.21.15.39:0/1998308351 >> 172.21.15.39:0/1998308351 conn(0x5625c73ef460 msgr2=0x5625c77adb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:47:09.747 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:09.742+0000 7f2b777e3740 1 -- 172.21.15.39:0/1998308351 shutdown_connections 2024-09-21T03:47:09.747 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:09.742+0000 7f2b777e3740 1 -- 172.21.15.39:0/1998308351 wait complete. 2024-09-21T03:47:09.747 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:09.743+0000 7f2b777e3740 1 -- start start 2024-09-21T03:47:09.747 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:09.743+0000 7f2b777e3740 1 Processor -- start 2024-09-21T03:47:09.747 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:09.743+0000 7f2b777e3740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5625c73b99f0 0x5625c77b23a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:47:09.747 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:09.743+0000 7f2b777e3740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5625c73d21b0 0x5625c77b28e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:47:09.747 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:09.744+0000 7f2b6a7fc640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5625c73d21b0 0x5625c77b28e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:47:09.747 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:09.744+0000 7f2b6a7fc640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5625c73d21b0 0x5625c77b28e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:51198/0 (socket says 172.21.15.39:51198) 2024-09-21T03:47:09.747 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:09.744+0000 7f2b6a7fc640 1 -- 172.21.15.39:0/291122729 learned_addr learned my addr 172.21.15.39:0/291122729 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:47:09.747 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:09.744+0000 7f2b777e3740 1 --2- 172.21.15.39:0/291122729 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5625c73da390 0x5625c7832230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:47:09.747 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:09.744+0000 7f2b6affd640 1 --2- 172.21.15.39:0/291122729 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5625c73b99f0 0x5625c77b23a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:47:09.748 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:09.744+0000 7f2b777e3740 1 -- 172.21.15.39:0/291122729 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f2b6c1aea70 con 0x5625c73d21b0 2024-09-21T03:47:09.748 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:09.744+0000 7f2b777e3740 1 -- 172.21.15.39:0/291122729 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f2b6c1ae3a0 con 0x5625c73b99f0 2024-09-21T03:47:09.748 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:09.744+0000 7f2b777e3740 1 -- 172.21.15.39:0/291122729 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f2b6c1ab640 con 0x5625c73da390 2024-09-21T03:47:09.748 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:09.744+0000 7f2b777e3740 10 client.0 _ll_register_callbacks cb 0x7f2b600a7f50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:47:09.748 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:09.744+0000 7f2b6b7fe640 1 --2- 172.21.15.39:0/291122729 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5625c73da390 0x5625c7832230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:47:09.748 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:09.744+0000 7f2b6affd640 1 -- 172.21.15.39:0/291122729 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5625c73da390 msgr2=0x5625c7832230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:47:09.748 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:09.744+0000 7f2b6affd640 1 --2- 172.21.15.39:0/291122729 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5625c73da390 0x5625c7832230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:09.748 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:09.744+0000 7f2b6affd640 1 -- 172.21.15.39:0/291122729 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5625c73d21b0 msgr2=0x5625c77b28e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:47:09.748 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:09.744+0000 7f2b6affd640 1 --2- 172.21.15.39:0/291122729 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5625c73d21b0 0x5625c77b28e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:09.748 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:09.744+0000 7f2b6affd640 1 -- 172.21.15.39:0/291122729 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5625c77b3180 con 0x5625c73b99f0 2024-09-21T03:47:09.748 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:09.744+0000 7f2b6a7fc640 1 --2- 172.21.15.39:0/291122729 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5625c73d21b0 0x5625c77b28e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:47:09.748 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:09.745+0000 7f2b6affd640 1 --2- 172.21.15.39:0/291122729 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5625c73b99f0 0x5625c77b23a0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f2b5c033000 tx=0x7f2b5c009fb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8f41156db6c998a3 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:47:09.748 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:09.745+0000 7f2b4eff5640 10 client.0 ms_handle_connect on v2:172.21.15.150:3300/0 2024-09-21T03:47:09.751 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:09.745+0000 7f2b4eff5640 1 -- 172.21.15.39:0/291122729 <== mon.2 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b5c03e070 con 0x5625c73b99f0 2024-09-21T03:47:09.751 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:09.745+0000 7f2b4eff5640 1 -- 172.21.15.39:0/291122729 <== mon.2 v2:172.21.15.150:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8484+0+0 (secure 0 0 0) 0x7f2b5c02f600 con 0x5625c73b99f0 2024-09-21T03:47:09.751 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:09.745+0000 7f2b777e3740 1 -- 172.21.15.39:0/291122729 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2b5c034070 con 0x5625c73b99f0 2024-09-21T03:47:09.752 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:09.745+0000 7f2b4eff5640 1 -- 172.21.15.39:0/291122729 <== mon.2 v2:172.21.15.150:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f2b5c04a050 con 0x5625c73b99f0 2024-09-21T03:47:09.752 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:09.745+0000 7f2b777e3740 10 client.25382 fetch_fsmap learned FSMap version 29 2024-09-21T03:47:09.752 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:09.745+0000 7f2b777e3740 1 -- 172.21.15.39:0/291122729 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5625c7833720 con 0x5625c73b99f0 2024-09-21T03:47:09.752 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:09.745+0000 7f2b4eff5640 1 -- 172.21.15.39:0/291122729 <== mon.2 v2:172.21.15.150:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2b5c02f800 con 0x5625c73b99f0 2024-09-21T03:47:09.752 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:09.747+0000 7f2b4eff5640 1 -- 172.21.15.39:0/291122729 <== mon.2 v2:172.21.15.150:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f2b5c03a440 con 0x5625c73b99f0 2024-09-21T03:47:09.752 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:09.747+0000 7f2b777e3740 10 client.25382 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T03:47:09.752 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:09.747+0000 7f2b777e3740 10 client.25382 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.a 2024-09-21T03:47:09.752 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:09.747+0000 7f2b777e3740 20 client.25382 populate_metadata read hostname 'smithi039' 2024-09-21T03:47:09.752 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:09.747+0000 7f2b777e3740 1 --2- 172.21.15.39:0/291122729 >> [v2:172.21.15.39:6840/4247190456,v1:172.21.15.39:6841/4247190456] conn(0x5625c73fb070 0x5625c7835db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:47:09.752 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:09.747+0000 7f2b777e3740 4 client.25382 mds_command: new command op to 14868 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T03:47:09.752 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:09.747+0000 7f2b777e3740 1 -- 172.21.15.39:0/291122729 --> [v2:172.21.15.39:6840/4247190456,v1:172.21.15.39:6841/4247190456] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7f2b5c034070 con 0x5625c73fb070 2024-09-21T03:47:09.752 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:09.747+0000 7f2b6a7fc640 1 --2- 172.21.15.39:0/291122729 >> [v2:172.21.15.39:6840/4247190456,v1:172.21.15.39:6841/4247190456] conn(0x5625c73fb070 0x5625c7835db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:47:09.752 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:09.748+0000 7f2b4eff5640 10 client.25382 ms_handle_connect on v2:172.21.15.39:6840/4247190456 2024-09-21T03:47:09.753 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:09.748+0000 7f2b6a7fc640 1 --2- 172.21.15.39:0/291122729 >> [v2:172.21.15.39:6840/4247190456,v1:172.21.15.39:6841/4247190456] conn(0x5625c73fb070 0x5625c7835db0 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=e3b2122970c7f68b server_cookie=62b0e4a88c25ffaf in_seq=0 out_seq=0 2024-09-21T03:47:09.753 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:09.748+0000 7f2b4eff5640 1 -- 172.21.15.39:0/291122729 <== mds.4 v2:172.21.15.39:6840/4247190456 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7f2b5c034070 con 0x5625c73fb070 2024-09-21T03:47:09.753 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:09.748+0000 7f2b4eff5640 10 client.25382 handle_command_reply: tid=0 2024-09-21T03:47:09.753 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:09.748+0000 7f2b777e3740 1 client.25382 shutdown 2024-09-21T03:47:09.753 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:09.748+0000 7f2b777e3740 1 -- 172.21.15.39:0/291122729 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5625c73b99f0 msgr2=0x5625c77b23a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:47:09.753 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:09.748+0000 7f2b777e3740 1 --2- 172.21.15.39:0/291122729 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5625c73b99f0 0x5625c77b23a0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f2b5c033000 tx=0x7f2b5c009fb0 comp rx=0 tx=0).stop 2024-09-21T03:47:09.753 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:09.748+0000 7f2b777e3740 1 -- 172.21.15.39:0/291122729 shutdown_connections 2024-09-21T03:47:09.753 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:09.748+0000 7f2b777e3740 1 --2- 172.21.15.39:0/291122729 >> [v2:172.21.15.39:6840/4247190456,v1:172.21.15.39:6841/4247190456] conn(0x5625c73fb070 0x5625c7835db0 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:09.753 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:09.748+0000 7f2b4eff5640 0 client.25382 ms_handle_reset on v2:172.21.15.39:6840/4247190456 2024-09-21T03:47:09.753 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:09.748+0000 7f2b777e3740 1 --2- 172.21.15.39:0/291122729 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5625c73da390 0x5625c7832230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:09.753 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:09.748+0000 7f2b777e3740 1 --2- 172.21.15.39:0/291122729 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5625c73d21b0 0x5625c77b28e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:09.753 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:09.748+0000 7f2b777e3740 1 --2- 172.21.15.39:0/291122729 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5625c73b99f0 0x5625c77b23a0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:09.753 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:09.748+0000 7f2b777e3740 1 -- 172.21.15.39:0/291122729 >> 172.21.15.39:0/291122729 conn(0x5625c73ef460 msgr2=0x5625c77ac1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:47:09.753 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:09.749+0000 7f2b777e3740 1 -- 172.21.15.39:0/291122729 shutdown_connections 2024-09-21T03:47:09.753 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:09.749+0000 7f2b777e3740 1 -- 172.21.15.39:0/291122729 wait complete. 2024-09-21T03:47:09.754 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:09.749+0000 7f2b777e3740 20 client.25382 trim_cache size 0 max 16384 2024-09-21T03:47:09.754 INFO:teuthology.orchestra.run.smithi039.stdout:[] 2024-09-21T03:47:09.755 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:09.750+0000 7f2b7126f640 1 -- 172.21.15.39:0/3855483620 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f2b540b6790 msgr2=0x7f2b540b8ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:47:09.755 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:09.750+0000 7f2b7126f640 1 --2- 172.21.15.39:0/3855483620 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f2b540b6790 0x7f2b540b8ca0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x5625c77a1320 tx=0x7f2b5c005f90 comp rx=0 tx=0).stop 2024-09-21T03:47:09.755 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:09.750+0000 7f2b7126f640 1 -- 172.21.15.39:0/3855483620 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2b6c10b6d0 msgr2=0x7f2b6c19ca20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:47:09.755 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:09.750+0000 7f2b7126f640 1 --2- 172.21.15.39:0/3855483620 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2b6c10b6d0 0x7f2b6c19ca20 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f2b6000e9f0 tx=0x7f2b6000eeb0 comp rx=0 tx=0).stop 2024-09-21T03:47:09.755 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:09.750+0000 7f2b7126f640 1 -- 172.21.15.39:0/3855483620 shutdown_connections 2024-09-21T03:47:09.755 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:09.750+0000 7f2b7126f640 1 --2- 172.21.15.39:0/3855483620 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f2b540b6790 0x7f2b540b8ca0 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:09.755 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:09.750+0000 7f2b7126f640 1 --2- 172.21.15.39:0/3855483620 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2b6c10c020 0x7f2b6c1a1cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:09.755 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:09.750+0000 7f2b7126f640 1 --2- 172.21.15.39:0/3855483620 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2b6c10b6d0 0x7f2b6c19ca20 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:09.755 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:09.750+0000 7f2b7126f640 1 --2- 172.21.15.39:0/3855483620 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2b6c105d90 0x7f2b6c19c480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:09.755 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:09.750+0000 7f2b7126f640 1 -- 172.21.15.39:0/3855483620 >> 172.21.15.39:0/3855483620 conn(0x7f2b6c101860 msgr2=0x7f2b6c109540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:47:09.756 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:09.750+0000 7f2b7126f640 1 -- 172.21.15.39:0/3855483620 shutdown_connections 2024-09-21T03:47:09.756 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:09.750+0000 7f2b7126f640 1 -- 172.21.15.39:0/3855483620 wait complete. 2024-09-21T03:47:10.538 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 51116d5c-77ca-11ef-baf5-efdc52797490 -- ceph tell mds.1:3 damage ls 2024-09-21T03:47:10.805 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/51116d5c-77ca-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:47:10.974 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:47:10 smithi150 ceph-mon[34289]: pgmap v359: 97 pgs: 97 active+clean; 1.4 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 22 KiB/s wr, 26 op/s 2024-09-21T03:47:11.072 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:47:10 smithi039 ceph-mon[31211]: pgmap v359: 97 pgs: 97 active+clean; 1.4 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 22 KiB/s wr, 26 op/s 2024-09-21T03:47:11.218 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:47:10 smithi161 ceph-mon[34533]: pgmap v359: 97 pgs: 97 active+clean; 1.4 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 22 KiB/s wr, 26 op/s 2024-09-21T03:47:11.368 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:11.365+0000 7f058cbf3640 1 -- 172.21.15.39:0/3193314643 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f05881083a0 msgr2=0x7f05881087b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:47:11.368 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:11.365+0000 7f058cbf3640 1 --2- 172.21.15.39:0/3193314643 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f05881083a0 0x7f05881087b0 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f057c009900 tx=0x7f057c02a410 comp rx=0 tx=0).stop 2024-09-21T03:47:11.369 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:11.366+0000 7f058cbf3640 1 -- 172.21.15.39:0/3193314643 shutdown_connections 2024-09-21T03:47:11.369 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:11.366+0000 7f058cbf3640 1 --2- 172.21.15.39:0/3193314643 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0588108cf0 0x7f058810f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:11.369 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:11.366+0000 7f058cbf3640 1 --2- 172.21.15.39:0/3193314643 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f05881083a0 0x7f05881087b0 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:11.369 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:11.366+0000 7f058cbf3640 1 --2- 172.21.15.39:0/3193314643 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0588102a60 0x7f0588102e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:11.369 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:11.366+0000 7f058cbf3640 1 -- 172.21.15.39:0/3193314643 >> 172.21.15.39:0/3193314643 conn(0x7f0588079730 msgr2=0x7f0588079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:47:11.369 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:11.367+0000 7f058cbf3640 1 -- 172.21.15.39:0/3193314643 shutdown_connections 2024-09-21T03:47:11.369 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:11.367+0000 7f058cbf3640 1 -- 172.21.15.39:0/3193314643 wait complete. 2024-09-21T03:47:11.369 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:11.368+0000 7f058cbf3640 1 Processor -- start 2024-09-21T03:47:11.370 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:11.368+0000 7f058cbf3640 1 -- start start 2024-09-21T03:47:11.371 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:11.369+0000 7f058cbf3640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0588102a60 0x7f0588199340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:47:11.371 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:11.369+0000 7f058659c640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0588102a60 0x7f0588199340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:47:11.371 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:11.370+0000 7f058659c640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0588102a60 0x7f0588199340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:51220/0 (socket says 172.21.15.39:51220) 2024-09-21T03:47:11.372 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:11.370+0000 7f058659c640 1 -- 172.21.15.39:0/3016989592 learned_addr learned my addr 172.21.15.39:0/3016989592 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:47:11.372 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:11.370+0000 7f058cbf3640 1 --2- 172.21.15.39:0/3016989592 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f05881083a0 0x7f05881998e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:47:11.372 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:11.370+0000 7f058cbf3640 1 --2- 172.21.15.39:0/3016989592 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0588108cf0 0x7f058819eb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:47:11.372 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:11.370+0000 7f058cbf3640 1 -- 172.21.15.39:0/3016989592 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f05881129b0 con 0x7f0588102a60 2024-09-21T03:47:11.372 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:11.370+0000 7f058cbf3640 1 -- 172.21.15.39:0/3016989592 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f0588112830 con 0x7f0588108cf0 2024-09-21T03:47:11.372 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:11.370+0000 7f058cbf3640 1 -- 172.21.15.39:0/3016989592 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f0588112b30 con 0x7f05881083a0 2024-09-21T03:47:11.373 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:11.370+0000 7f0585d9b640 1 --2- 172.21.15.39:0/3016989592 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f05881083a0 0x7f05881998e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:47:11.373 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:11.370+0000 7f0586d9d640 1 --2- 172.21.15.39:0/3016989592 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0588108cf0 0x7f058819eb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:47:11.373 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:11.371+0000 7f0585d9b640 1 -- 172.21.15.39:0/3016989592 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0588108cf0 msgr2=0x7f058819eb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:47:11.374 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:11.371+0000 7f0585d9b640 1 --2- 172.21.15.39:0/3016989592 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0588108cf0 0x7f058819eb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:11.374 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:11.371+0000 7f0585d9b640 1 -- 172.21.15.39:0/3016989592 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0588102a60 msgr2=0x7f0588199340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:47:11.374 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:11.371+0000 7f0585d9b640 1 --2- 172.21.15.39:0/3016989592 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0588102a60 0x7f0588199340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:11.374 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:11.371+0000 7f0585d9b640 1 -- 172.21.15.39:0/3016989592 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f058819f2b0 con 0x7f05881083a0 2024-09-21T03:47:11.374 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:11.371+0000 7f058659c640 1 --2- 172.21.15.39:0/3016989592 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0588102a60 0x7f0588199340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:47:11.374 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:11.371+0000 7f0585d9b640 1 --2- 172.21.15.39:0/3016989592 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f05881083a0 0x7f05881998e0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f057c02a930 tx=0x7f057c003b90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:47:11.374 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:11.372+0000 7f05777fe640 1 -- 172.21.15.39:0/3016989592 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f057c035070 con 0x7f05881083a0 2024-09-21T03:47:11.374 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:11.372+0000 7f05777fe640 1 -- 172.21.15.39:0/3016989592 <== mon.1 v2:172.21.15.161:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f057c031950 con 0x7f05881083a0 2024-09-21T03:47:11.374 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:11.372+0000 7f058cbf3640 1 -- 172.21.15.39:0/3016989592 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f058819f540 con 0x7f05881083a0 2024-09-21T03:47:11.375 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:11.372+0000 7f058cbf3640 1 -- 172.21.15.39:0/3016989592 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f058819f980 con 0x7f05881083a0 2024-09-21T03:47:11.375 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:11.373+0000 7f05777fe640 1 -- 172.21.15.39:0/3016989592 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f057c038940 con 0x7f05881083a0 2024-09-21T03:47:11.377 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:11.375+0000 7f0593167740 1 Processor -- start 2024-09-21T03:47:11.377 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:11.376+0000 7f0593167740 1 -- start start 2024-09-21T03:47:11.380 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:11.376+0000 7f0593167740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a01b902070 0x55a01bca6d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:47:11.380 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:11.376+0000 7f0593167740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a01b8d91b0 0x55a01bca72a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:47:11.380 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:11.376+0000 7f0593167740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a01b8e1390 0x55a01bcb9670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:47:11.380 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:11.376+0000 7f0593167740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x55a01bc1fea0 con 0x55a01b902070 2024-09-21T03:47:11.380 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:11.376+0000 7f0593167740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x55a01bc0f2d0 con 0x55a01b8e1390 2024-09-21T03:47:11.380 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:11.376+0000 7f0593167740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55a01bc130e0 con 0x55a01b8d91b0 2024-09-21T03:47:11.380 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:11.376+0000 7f0586d9d640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a01b902070 0x55a01bca6d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:47:11.380 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:11.376+0000 7f0586d9d640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a01b902070 0x55a01bca6d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:51230/0 (socket says 172.21.15.39:51230) 2024-09-21T03:47:11.380 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:11.377+0000 7f0586d9d640 1 -- 172.21.15.39:0/386582157 learned_addr learned my addr 172.21.15.39:0/386582157 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:47:11.380 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:11.377+0000 7f0586d9d640 1 -- 172.21.15.39:0/386582157 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a01b8d91b0 msgr2=0x55a01bca72a0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-21T03:47:11.380 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:11.377+0000 7f0586d9d640 1 --2- 172.21.15.39:0/386582157 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a01b8d91b0 0x55a01bca72a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:11.380 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:11.377+0000 7f0586d9d640 1 -- 172.21.15.39:0/386582157 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a01b8e1390 msgr2=0x55a01bcb9670 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-21T03:47:11.381 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:11.377+0000 7f0586d9d640 1 --2- 172.21.15.39:0/386582157 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a01b8e1390 0x55a01bcb9670 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:11.381 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:11.377+0000 7f0586d9d640 1 -- 172.21.15.39:0/386582157 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a01b96d650 con 0x55a01b902070 2024-09-21T03:47:11.381 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:11.377+0000 7f05777fe640 1 -- 172.21.15.39:0/3016989592 <== mon.1 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 26) ==== 154694+0+0 (secure 0 0 0) 0x7f057c0314a0 con 0x7f05881083a0 2024-09-21T03:47:11.381 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:11.377+0000 7f0586d9d640 1 --2- 172.21.15.39:0/386582157 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a01b902070 0x55a01bca6d20 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f0580009fd0 tx=0x7f05800118d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3c4380a0f65652a7 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:47:11.381 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:11.378+0000 7f0574ff9640 1 -- 172.21.15.39:0/386582157 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0580004d20 con 0x55a01b902070 2024-09-21T03:47:11.381 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:11.378+0000 7f0574ff9640 1 -- 172.21.15.39:0/386582157 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0580004050 con 0x55a01b902070 2024-09-21T03:47:11.381 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:11.378+0000 7f0574ff9640 1 -- 172.21.15.39:0/386582157 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0580005280 con 0x55a01b902070 2024-09-21T03:47:11.382 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:11.378+0000 7f05777fe640 1 --2- 172.21.15.39:0/3016989592 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f05700b6810 0x7f05700b8d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:47:11.382 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:11.379+0000 7f058659c640 1 --2- 172.21.15.39:0/3016989592 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f05700b6810 0x7f05700b8d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:47:11.382 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:11.379+0000 7f05777fe640 1 -- 172.21.15.39:0/3016989592 <== mon.1 v2:172.21.15.161:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8484+0+0 (secure 0 0 0) 0x7f057c1075f0 con 0x7f05881083a0 2024-09-21T03:47:11.382 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:11.379+0000 7f0593167740 1 -- 172.21.15.39:0/386582157 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a01b902070 msgr2=0x55a01bca6d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:47:11.382 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:11.379+0000 7f0593167740 1 --2- 172.21.15.39:0/386582157 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a01b902070 0x55a01bca6d20 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f0580009fd0 tx=0x7f05800118d0 comp rx=0 tx=0).stop 2024-09-21T03:47:11.383 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:11.379+0000 7f0593167740 1 -- 172.21.15.39:0/386582157 shutdown_connections 2024-09-21T03:47:11.383 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:11.379+0000 7f0593167740 1 --2- 172.21.15.39:0/386582157 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a01b8e1390 0x55a01bcb9670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:11.383 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:11.379+0000 7f0593167740 1 --2- 172.21.15.39:0/386582157 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a01b8d91b0 0x55a01bca72a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:11.383 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:11.379+0000 7f0593167740 1 --2- 172.21.15.39:0/386582157 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a01b902070 0x55a01bca6d20 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:11.383 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:11.379+0000 7f0593167740 1 -- 172.21.15.39:0/386582157 >> 172.21.15.39:0/386582157 conn(0x55a01b8f6460 msgr2=0x55a01bca9240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:47:11.383 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:11.380+0000 7f0593167740 1 -- 172.21.15.39:0/386582157 shutdown_connections 2024-09-21T03:47:11.383 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:11.380+0000 7f058659c640 1 --2- 172.21.15.39:0/3016989592 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f05700b6810 0x7f05700b8d20 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x55a01bca7ec0 tx=0x7f0578005f90 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:47:11.383 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:11.380+0000 7f0593167740 1 -- 172.21.15.39:0/386582157 wait complete. 2024-09-21T03:47:11.383 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:11.380+0000 7f0593167740 1 -- start start 2024-09-21T03:47:11.383 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:11.380+0000 7f0593167740 1 Processor -- start 2024-09-21T03:47:11.386 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:11.381+0000 7f0593167740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a01b8d91b0 0x55a01bcbd9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:47:11.386 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:11.381+0000 7f0593167740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a01b8e1390 0x55a01bcbdf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:47:11.386 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:11.381+0000 7f0593167740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a01b902070 0x55a01bcc2fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:47:11.386 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:11.381+0000 7f0593167740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x55a01ba70700 con 0x55a01b8d91b0 2024-09-21T03:47:11.386 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:11.381+0000 7f0593167740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x55a01bbaa160 con 0x55a01b8e1390 2024-09-21T03:47:11.386 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:11.381+0000 7f0593167740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55a01bcba170 con 0x55a01b902070 2024-09-21T03:47:11.386 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:11.381+0000 7f0593167740 10 client.0 _ll_register_callbacks cb 0x55a01bbc39e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:47:11.386 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:11.381+0000 7f0585d9b640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a01b8e1390 0x55a01bcbdf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:47:11.387 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:11.381+0000 7f0586d9d640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a01b902070 0x55a01bcc2fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:47:11.387 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:11.381+0000 7f0585d9b640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a01b8e1390 0x55a01bcbdf80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.39:56910/0 (socket says 172.21.15.39:56910) 2024-09-21T03:47:11.387 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:11.381+0000 7f0585d9b640 1 -- 172.21.15.39:0/3161024492 learned_addr learned my addr 172.21.15.39:0/3161024492 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:47:11.387 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:11.382+0000 7f0585d9b640 1 -- 172.21.15.39:0/3161024492 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a01b902070 msgr2=0x55a01bcc2fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:47:11.387 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:11.382+0000 7f0585d9b640 1 --2- 172.21.15.39:0/3161024492 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a01b902070 0x55a01bcc2fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:11.387 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:11.382+0000 7f0585d9b640 1 -- 172.21.15.39:0/3161024492 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a01b8d91b0 msgr2=0x55a01bcbd9a0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-21T03:47:11.387 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:11.382+0000 7f0585d9b640 1 --2- 172.21.15.39:0/3161024492 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a01b8d91b0 0x55a01bcbd9a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:11.387 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:11.382+0000 7f0585d9b640 1 -- 172.21.15.39:0/3161024492 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a01bbd7c90 con 0x55a01b8e1390 2024-09-21T03:47:11.387 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:11.382+0000 7f0586d9d640 1 --2- 172.21.15.39:0/3161024492 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a01b902070 0x55a01bcc2fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:47:11.387 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:11.382+0000 7f0585d9b640 1 --2- 172.21.15.39:0/3161024492 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a01b8e1390 0x55a01bcbdf80 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f057c0ca500 tx=0x7f057c0c91f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d14f9291b90439bf server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:47:11.387 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:11.382+0000 7f056dffb640 10 client.0 ms_handle_connect on v2:172.21.15.150:3300/0 2024-09-21T03:47:11.387 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:11.382+0000 7f056dffb640 1 -- 172.21.15.39:0/3161024492 <== mon.2 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f057c031790 con 0x55a01b8e1390 2024-09-21T03:47:11.387 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:11.382+0000 7f0593167740 1 -- 172.21.15.39:0/3161024492 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0580004050 con 0x55a01b8e1390 2024-09-21T03:47:11.388 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:11.382+0000 7f056dffb640 1 -- 172.21.15.39:0/3161024492 <== mon.2 v2:172.21.15.150:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8484+0+0 (secure 0 0 0) 0x7f057c104ef0 con 0x55a01b8e1390 2024-09-21T03:47:11.388 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:11.382+0000 7f056dffb640 1 -- 172.21.15.39:0/3161024492 <== mon.2 v2:172.21.15.150:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f057c0ccda0 con 0x55a01b8e1390 2024-09-21T03:47:11.388 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:11.382+0000 7f0593167740 10 client.25391 fetch_fsmap learned FSMap version 29 2024-09-21T03:47:11.388 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:11.382+0000 7f0593167740 1 -- 172.21.15.39:0/3161024492 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55a01bccbef0 con 0x55a01b8e1390 2024-09-21T03:47:11.388 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:11.383+0000 7f056dffb640 1 -- 172.21.15.39:0/3161024492 <== mon.2 v2:172.21.15.150:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0580004050 con 0x55a01b8e1390 2024-09-21T03:47:11.388 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:11.383+0000 7f056dffb640 1 -- 172.21.15.39:0/3161024492 <== mon.2 v2:172.21.15.150:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f057c0b6be0 con 0x55a01b8e1390 2024-09-21T03:47:11.388 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:11.383+0000 7f0593167740 10 client.25391 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T03:47:11.390 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:11.383+0000 7f0593167740 10 client.25391 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-21T03:47:11.390 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:11.383+0000 7f0593167740 20 client.25391 populate_metadata read hostname 'smithi039' 2024-09-21T03:47:11.390 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:11.383+0000 7f0593167740 1 --2- 172.21.15.39:0/3161024492 >> [v2:172.21.15.161:6836/3451544597,v1:172.21.15.161:6837/3451544597] conn(0x55a01b8c09f0 0x55a01bccf460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:47:11.390 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:11.383+0000 7f0593167740 4 client.25391 mds_command: new command op to 24646 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:47:11.390 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:11.383+0000 7f0593167740 1 -- 172.21.15.39:0/3161024492 --> [v2:172.21.15.161:6836/3451544597,v1:172.21.15.161:6837/3451544597] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a01bc21df0 con 0x55a01b8c09f0 2024-09-21T03:47:11.390 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:11.386+0000 7f058659c640 1 --2- 172.21.15.39:0/3161024492 >> [v2:172.21.15.161:6836/3451544597,v1:172.21.15.161:6837/3451544597] conn(0x55a01b8c09f0 0x55a01bccf460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:47:11.390 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:11.386+0000 7f056dffb640 10 client.25391 ms_handle_connect on v2:172.21.15.161:6836/3451544597 2024-09-21T03:47:11.390 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:11.386+0000 7f058659c640 1 --2- 172.21.15.39:0/3161024492 >> [v2:172.21.15.161:6836/3451544597,v1:172.21.15.161:6837/3451544597] conn(0x55a01b8c09f0 0x55a01bccf460 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=702081b98bc92761 server_cookie=789fbceb1bdaa0e7 in_seq=0 out_seq=0 2024-09-21T03:47:11.390 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:11.387+0000 7f056dffb640 1 -- 172.21.15.39:0/3161024492 <== mds.3 v2:172.21.15.161:6836/3451544597 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55a01bc21df0 con 0x55a01b8c09f0 2024-09-21T03:47:11.390 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:11.387+0000 7f056dffb640 10 client.25391 handle_command_reply: tid=0 2024-09-21T03:47:11.391 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:11.387+0000 7f0593167740 1 client.25391 shutdown 2024-09-21T03:47:11.391 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:11.388+0000 7f0593167740 1 -- 172.21.15.39:0/3161024492 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a01b8e1390 msgr2=0x55a01bcbdf80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:47:11.391 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:11.388+0000 7f0593167740 1 --2- 172.21.15.39:0/3161024492 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a01b8e1390 0x55a01bcbdf80 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f057c0ca500 tx=0x7f057c0c91f0 comp rx=0 tx=0).stop 2024-09-21T03:47:11.391 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:11.388+0000 7f0593167740 1 -- 172.21.15.39:0/3161024492 shutdown_connections 2024-09-21T03:47:11.391 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:11.388+0000 7f0593167740 1 --2- 172.21.15.39:0/3161024492 >> [v2:172.21.15.161:6836/3451544597,v1:172.21.15.161:6837/3451544597] conn(0x55a01b8c09f0 0x55a01bccf460 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:11.392 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:11.388+0000 7f0593167740 1 --2- 172.21.15.39:0/3161024492 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a01b902070 0x55a01bcc2fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:11.392 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:11.388+0000 7f056dffb640 0 client.25391 ms_handle_reset on v2:172.21.15.161:6836/3451544597 2024-09-21T03:47:11.392 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:11.388+0000 7f0593167740 1 --2- 172.21.15.39:0/3161024492 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a01b8e1390 0x55a01bcbdf80 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:11.392 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:11.388+0000 7f0593167740 1 --2- 172.21.15.39:0/3161024492 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a01b8d91b0 0x55a01bcbd9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:11.392 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:11.388+0000 7f0593167740 1 -- 172.21.15.39:0/3161024492 >> 172.21.15.39:0/3161024492 conn(0x55a01b8f6460 msgr2=0x55a01bca81f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:47:11.392 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:11.389+0000 7f0593167740 1 -- 172.21.15.39:0/3161024492 shutdown_connections 2024-09-21T03:47:11.392 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:11.389+0000 7f0593167740 1 -- 172.21.15.39:0/3161024492 wait complete. 2024-09-21T03:47:11.392 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:11.389+0000 7f0593167740 20 client.25391 trim_cache size 0 max 16384 2024-09-21T03:47:11.406 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:11.404+0000 7f0593167740 1 Processor -- start 2024-09-21T03:47:11.406 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:11.404+0000 7f0593167740 1 -- start start 2024-09-21T03:47:11.406 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:11.405+0000 7f0593167740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a01b8d91b0 0x55a01bca9170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:47:11.406 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:11.405+0000 7f0593167740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a01b8e1390 0x55a01bcb8160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:47:11.407 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:11.405+0000 7f0593167740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a01b902070 0x55a01bcb86a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:47:11.407 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:11.405+0000 7f058659c640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a01b8d91b0 0x55a01bca9170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:47:11.407 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:11.405+0000 7f058659c640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a01b8d91b0 0x55a01bca9170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:51238/0 (socket says 172.21.15.39:51238) 2024-09-21T03:47:11.407 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:11.405+0000 7f0593167740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f05881ad3f0 con 0x55a01b8d91b0 2024-09-21T03:47:11.407 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:11.405+0000 7f058659c640 1 -- 172.21.15.39:0/3721390974 learned_addr learned my addr 172.21.15.39:0/3721390974 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:47:11.407 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:11.405+0000 7f0593167740 1 -- 172.21.15.39:0/3721390974 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f05881aef30 con 0x55a01b8e1390 2024-09-21T03:47:11.407 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:11.405+0000 7f0593167740 1 -- 172.21.15.39:0/3721390974 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f05881adac0 con 0x55a01b902070 2024-09-21T03:47:11.408 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:11.405+0000 7f0585d9b640 1 --2- 172.21.15.39:0/3721390974 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a01b8e1390 0x55a01bcb8160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:47:11.408 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:11.405+0000 7f0586d9d640 1 --2- 172.21.15.39:0/3721390974 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a01b902070 0x55a01bcb86a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:47:11.408 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:11.406+0000 7f058659c640 1 -- 172.21.15.39:0/3721390974 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a01b902070 msgr2=0x55a01bcb86a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:47:11.408 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:11.406+0000 7f058659c640 1 --2- 172.21.15.39:0/3721390974 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a01b902070 0x55a01bcb86a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:11.408 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:11.406+0000 7f058659c640 1 -- 172.21.15.39:0/3721390974 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a01b8e1390 msgr2=0x55a01bcb8160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:47:11.408 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:11.406+0000 7f058659c640 1 --2- 172.21.15.39:0/3721390974 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a01b8e1390 0x55a01bcb8160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:11.409 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:11.406+0000 7f058659c640 1 -- 172.21.15.39:0/3721390974 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a01bca9b30 con 0x55a01b8d91b0 2024-09-21T03:47:11.409 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:11.406+0000 7f0585d9b640 1 --2- 172.21.15.39:0/3721390974 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a01b8e1390 0x55a01bcb8160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:47:11.409 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:11.406+0000 7f058659c640 1 --2- 172.21.15.39:0/3721390974 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a01b8d91b0 0x55a01bca9170 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f057800f750 tx=0x7f0578014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fad49066475a048e server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:47:11.409 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:11.406+0000 7f056ffff640 1 -- 172.21.15.39:0/3721390974 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0578029030 con 0x55a01b8d91b0 2024-09-21T03:47:11.409 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:11.406+0000 7f056ffff640 1 -- 172.21.15.39:0/3721390974 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f057801c8c0 con 0x55a01b8d91b0 2024-09-21T03:47:11.409 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:11.406+0000 7f056ffff640 1 -- 172.21.15.39:0/3721390974 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f057801b7d0 con 0x55a01b8d91b0 2024-09-21T03:47:11.410 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:11.407+0000 7f0593167740 1 -- 172.21.15.39:0/3721390974 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a01b8d91b0 msgr2=0x55a01bca9170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:47:11.410 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:11.407+0000 7f0593167740 1 --2- 172.21.15.39:0/3721390974 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a01b8d91b0 0x55a01bca9170 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f057800f750 tx=0x7f0578014000 comp rx=0 tx=0).stop 2024-09-21T03:47:11.410 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:11.407+0000 7f0593167740 1 -- 172.21.15.39:0/3721390974 shutdown_connections 2024-09-21T03:47:11.410 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:11.407+0000 7f0593167740 1 --2- 172.21.15.39:0/3721390974 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a01b902070 0x55a01bcb86a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:11.410 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:11.407+0000 7f0593167740 1 --2- 172.21.15.39:0/3721390974 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a01b8e1390 0x55a01bcb8160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:11.410 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:11.407+0000 7f0593167740 1 --2- 172.21.15.39:0/3721390974 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a01b8d91b0 0x55a01bca9170 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:11.410 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:11.407+0000 7f0593167740 1 -- 172.21.15.39:0/3721390974 >> 172.21.15.39:0/3721390974 conn(0x55a01b8f6460 msgr2=0x55a01bca6cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:47:11.410 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:11.408+0000 7f0593167740 1 -- 172.21.15.39:0/3721390974 shutdown_connections 2024-09-21T03:47:11.410 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:11.408+0000 7f0593167740 1 -- 172.21.15.39:0/3721390974 wait complete. 2024-09-21T03:47:11.410 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:11.408+0000 7f0593167740 1 -- start start 2024-09-21T03:47:11.411 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:11.408+0000 7f0593167740 1 Processor -- start 2024-09-21T03:47:11.411 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:11.409+0000 7f0593167740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a01b8d91b0 0x55a01bcc6070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:47:11.411 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:11.409+0000 7f0593167740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a01b8e1390 0x55a01bcc65b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:47:11.411 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:11.409+0000 7f0593167740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a01b902070 0x55a01bcc6af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:47:11.411 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:11.409+0000 7f0593167740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f05881ae860 con 0x55a01b8e1390 2024-09-21T03:47:11.411 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:11.409+0000 7f0593167740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f05881ae190 con 0x55a01b902070 2024-09-21T03:47:11.411 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:11.409+0000 7f0593167740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f05881ab3d0 con 0x55a01b8d91b0 2024-09-21T03:47:11.411 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:11.409+0000 7f0593167740 10 client.0 _ll_register_callbacks cb 0x7f057c104ad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:47:11.412 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:11.409+0000 7f0585d9b640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a01b8e1390 0x55a01bcc65b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:47:11.412 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:11.409+0000 7f0585d9b640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a01b8e1390 0x55a01bcc65b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:51246/0 (socket says 172.21.15.39:51246) 2024-09-21T03:47:11.412 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:11.409+0000 7f0585d9b640 1 -- 172.21.15.39:0/739773788 learned_addr learned my addr 172.21.15.39:0/739773788 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:47:11.412 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:11.409+0000 7f058659c640 1 --2- 172.21.15.39:0/739773788 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a01b8d91b0 0x55a01bcc6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:47:11.412 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:11.409+0000 7f0586d9d640 1 --2- 172.21.15.39:0/739773788 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a01b902070 0x55a01bcc6af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:47:11.412 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:11.410+0000 7f0585d9b640 1 -- 172.21.15.39:0/739773788 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a01b8d91b0 msgr2=0x55a01bcc6070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:47:11.413 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:11.410+0000 7f0585d9b640 1 --2- 172.21.15.39:0/739773788 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a01b8d91b0 0x55a01bcc6070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:11.413 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:11.410+0000 7f0585d9b640 1 -- 172.21.15.39:0/739773788 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a01b902070 msgr2=0x55a01bcc6af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:47:11.413 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:11.410+0000 7f0585d9b640 1 --2- 172.21.15.39:0/739773788 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a01b902070 0x55a01bcc6af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:11.413 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:11.410+0000 7f0585d9b640 1 -- 172.21.15.39:0/739773788 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a01bd38fa0 con 0x55a01b8e1390 2024-09-21T03:47:11.413 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:11.410+0000 7f0585d9b640 1 --2- 172.21.15.39:0/739773788 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a01b8e1390 0x55a01bcc65b0 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f057c0ccf90 tx=0x7f057c0045e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7dfcceaac8d707e8 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:47:11.413 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:11.410+0000 7f056dffb640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-09-21T03:47:11.414 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:11.411+0000 7f056dffb640 1 -- 172.21.15.39:0/739773788 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f057c0c98e0 con 0x55a01b8e1390 2024-09-21T03:47:11.414 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:11.411+0000 7f056dffb640 1 -- 172.21.15.39:0/739773788 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8484+0+0 (secure 0 0 0) 0x7f057c104b80 con 0x55a01b8e1390 2024-09-21T03:47:11.414 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:11.411+0000 7f0593167740 1 -- 172.21.15.39:0/739773788 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f057801c8c0 con 0x55a01b8e1390 2024-09-21T03:47:11.414 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:11.411+0000 7f056dffb640 1 -- 172.21.15.39:0/739773788 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f057c0ccbe0 con 0x55a01b8e1390 2024-09-21T03:47:11.414 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:11.411+0000 7f0593167740 10 client.15801 fetch_fsmap learned FSMap version 29 2024-09-21T03:47:11.414 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:11.411+0000 7f0593167740 1 -- 172.21.15.39:0/739773788 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55a01bd3a220 con 0x55a01b8e1390 2024-09-21T03:47:11.414 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:11.411+0000 7f056dffb640 1 -- 172.21.15.39:0/739773788 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f057801c8c0 con 0x55a01b8e1390 2024-09-21T03:47:11.414 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:11.412+0000 7f056dffb640 1 -- 172.21.15.39:0/739773788 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f057c0c8be0 con 0x55a01b8e1390 2024-09-21T03:47:11.414 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:11.412+0000 7f0593167740 10 client.15801 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T03:47:11.414 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:11.412+0000 7f0593167740 10 client.15801 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-21T03:47:11.415 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:11.412+0000 7f0593167740 20 client.15801 populate_metadata read hostname 'smithi039' 2024-09-21T03:47:11.415 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:11.412+0000 7f0593167740 1 --2- 172.21.15.39:0/739773788 >> [v2:172.21.15.161:6836/3451544597,v1:172.21.15.161:6837/3451544597] conn(0x55a01b895da0 0x55a01bd3c880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:47:11.415 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:11.412+0000 7f0593167740 4 client.15801 mds_command: new command op to 24646 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T03:47:11.415 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:11.412+0000 7f0593167740 1 -- 172.21.15.39:0/739773788 --> [v2:172.21.15.161:6836/3451544597,v1:172.21.15.161:6837/3451544597] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55a01bc1d8a0 con 0x55a01b895da0 2024-09-21T03:47:11.415 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:11.412+0000 7f058659c640 1 --2- 172.21.15.39:0/739773788 >> [v2:172.21.15.161:6836/3451544597,v1:172.21.15.161:6837/3451544597] conn(0x55a01b895da0 0x55a01bd3c880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:47:11.416 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:11.413+0000 7f058659c640 1 --2- 172.21.15.39:0/739773788 >> [v2:172.21.15.161:6836/3451544597,v1:172.21.15.161:6837/3451544597] conn(0x55a01b895da0 0x55a01bd3c880 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=e280baa1d8ba51a7 server_cookie=4b421c451594cebd in_seq=0 out_seq=0 2024-09-21T03:47:11.416 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:11.413+0000 7f056dffb640 10 client.15801 ms_handle_connect on v2:172.21.15.161:6836/3451544597 2024-09-21T03:47:11.416 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:11.413+0000 7f056dffb640 1 -- 172.21.15.39:0/739773788 <== mds.3 v2:172.21.15.161:6836/3451544597 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55a01bc1d8a0 con 0x55a01b895da0 2024-09-21T03:47:11.416 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:11.413+0000 7f056dffb640 10 client.15801 handle_command_reply: tid=0 2024-09-21T03:47:11.417 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:11.413+0000 7f0593167740 1 client.15801 shutdown 2024-09-21T03:47:11.417 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:11.413+0000 7f0593167740 1 -- 172.21.15.39:0/739773788 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a01b8e1390 msgr2=0x55a01bcc65b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:47:11.417 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:11.413+0000 7f0593167740 1 --2- 172.21.15.39:0/739773788 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a01b8e1390 0x55a01bcc65b0 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f057c0ccf90 tx=0x7f057c0045e0 comp rx=0 tx=0).stop 2024-09-21T03:47:11.417 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:11.414+0000 7f0593167740 1 -- 172.21.15.39:0/739773788 shutdown_connections 2024-09-21T03:47:11.417 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:11.414+0000 7f0593167740 1 --2- 172.21.15.39:0/739773788 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a01b902070 0x55a01bcc6af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:11.417 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:11.414+0000 7f0593167740 1 --2- 172.21.15.39:0/739773788 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a01b8e1390 0x55a01bcc65b0 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:11.417 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:11.414+0000 7f0593167740 1 --2- 172.21.15.39:0/739773788 >> [v2:172.21.15.161:6836/3451544597,v1:172.21.15.161:6837/3451544597] conn(0x55a01b895da0 0x55a01bd3c880 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:11.417 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:11.414+0000 7f0593167740 1 --2- 172.21.15.39:0/739773788 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a01b8d91b0 0x55a01bcc6070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:11.417 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:11.414+0000 7f0593167740 1 -- 172.21.15.39:0/739773788 >> 172.21.15.39:0/739773788 conn(0x55a01b8f6460 msgr2=0x55a01bcb8160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:47:11.417 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:11.414+0000 7f056dffb640 0 client.15801 ms_handle_reset on v2:172.21.15.161:6836/3451544597 2024-09-21T03:47:11.417 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:11.414+0000 7f0593167740 1 -- 172.21.15.39:0/739773788 shutdown_connections 2024-09-21T03:47:11.417 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:11.414+0000 7f0593167740 1 -- 172.21.15.39:0/739773788 wait complete. 2024-09-21T03:47:11.417 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:11.414+0000 7f0593167740 20 client.15801 trim_cache size 0 max 16384 2024-09-21T03:47:11.418 INFO:teuthology.orchestra.run.smithi039.stdout:[] 2024-09-21T03:47:11.418 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:11.415+0000 7f058cbf3640 1 -- 172.21.15.39:0/3016989592 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f05700b6810 msgr2=0x7f05700b8d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:47:11.418 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:11.415+0000 7f058cbf3640 1 --2- 172.21.15.39:0/3016989592 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f05700b6810 0x7f05700b8d20 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x55a01bca7ec0 tx=0x7f0578005f90 comp rx=0 tx=0).stop 2024-09-21T03:47:11.418 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:11.415+0000 7f058cbf3640 1 -- 172.21.15.39:0/3016989592 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f05881083a0 msgr2=0x7f05881998e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:47:11.419 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:11.415+0000 7f058cbf3640 1 --2- 172.21.15.39:0/3016989592 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f05881083a0 0x7f05881998e0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f057c02a930 tx=0x7f057c003b90 comp rx=0 tx=0).stop 2024-09-21T03:47:11.419 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:11.415+0000 7f058cbf3640 1 -- 172.21.15.39:0/3016989592 shutdown_connections 2024-09-21T03:47:11.419 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:11.415+0000 7f058cbf3640 1 --2- 172.21.15.39:0/3016989592 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f05700b6810 0x7f05700b8d20 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:11.419 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:11.415+0000 7f058cbf3640 1 --2- 172.21.15.39:0/3016989592 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0588108cf0 0x7f058819eb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:11.419 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:11.415+0000 7f058cbf3640 1 --2- 172.21.15.39:0/3016989592 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f05881083a0 0x7f05881998e0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:11.419 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:11.415+0000 7f058cbf3640 1 --2- 172.21.15.39:0/3016989592 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0588102a60 0x7f0588199340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:11.419 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:11.415+0000 7f058cbf3640 1 -- 172.21.15.39:0/3016989592 >> 172.21.15.39:0/3016989592 conn(0x7f0588079730 msgr2=0x7f05881043b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:47:11.419 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:11.416+0000 7f058cbf3640 1 -- 172.21.15.39:0/3016989592 shutdown_connections 2024-09-21T03:47:11.419 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:11.416+0000 7f058cbf3640 1 -- 172.21.15.39:0/3016989592 wait complete. 2024-09-21T03:47:12.245 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 51116d5c-77ca-11ef-baf5-efdc52797490 -- ceph tell mds.1:2 damage ls 2024-09-21T03:47:12.534 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/51116d5c-77ca-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:47:12.955 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:47:12 smithi039 ceph-mon[31211]: pgmap v360: 97 pgs: 97 active+clean; 1.4 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 24 op/s 2024-09-21T03:47:12.974 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:47:12 smithi150 ceph-mon[34289]: pgmap v360: 97 pgs: 97 active+clean; 1.4 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 24 op/s 2024-09-21T03:47:13.062 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:13.053+0000 7f358cb22640 1 -- 172.21.15.39:0/146918627 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f35881083e0 msgr2=0x7f35881087f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:47:13.063 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:13.053+0000 7f358cb22640 1 --2- 172.21.15.39:0/146918627 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f35881083e0 0x7f35881087f0 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f357c009900 tx=0x7f357c02a3f0 comp rx=0 tx=0).stop 2024-09-21T03:47:13.063 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:13.054+0000 7f358cb22640 1 -- 172.21.15.39:0/146918627 shutdown_connections 2024-09-21T03:47:13.063 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:13.054+0000 7f358cb22640 1 --2- 172.21.15.39:0/146918627 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3588108d30 0x7f358810f560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:13.063 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:13.054+0000 7f358cb22640 1 --2- 172.21.15.39:0/146918627 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f35881083e0 0x7f35881087f0 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:13.063 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:13.054+0000 7f358cb22640 1 --2- 172.21.15.39:0/146918627 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3588102aa0 0x7f3588102eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:13.063 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:13.054+0000 7f358cb22640 1 -- 172.21.15.39:0/146918627 >> 172.21.15.39:0/146918627 conn(0x7f3588079750 msgr2=0x7f3588079b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:47:13.063 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:13.054+0000 7f358cb22640 1 -- 172.21.15.39:0/146918627 shutdown_connections 2024-09-21T03:47:13.063 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:13.054+0000 7f358cb22640 1 -- 172.21.15.39:0/146918627 wait complete. 2024-09-21T03:47:13.063 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:13.055+0000 7f358cb22640 1 Processor -- start 2024-09-21T03:47:13.063 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:13.055+0000 7f358cb22640 1 -- start start 2024-09-21T03:47:13.063 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:13.056+0000 7f358cb22640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3588102aa0 0x7f3588198f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:47:13.063 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:13.056+0000 7f358cb22640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f35881083e0 0x7f3588199460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:47:13.064 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:13.056+0000 7f358cb22640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3588108d30 0x7f35881999a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:47:13.064 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:13.056+0000 7f358cb22640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f3588112a40 con 0x7f3588108d30 2024-09-21T03:47:13.064 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:13.056+0000 7f3586d9d640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3588108d30 0x7f35881999a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:47:13.064 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:13.056+0000 7f3586d9d640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3588108d30 0x7f35881999a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:51258/0 (socket says 172.21.15.39:51258) 2024-09-21T03:47:13.064 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:13.056+0000 7f3586d9d640 1 -- 172.21.15.39:0/3887792008 learned_addr learned my addr 172.21.15.39:0/3887792008 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:47:13.064 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:13.056+0000 7f358659c640 1 --2- 172.21.15.39:0/3887792008 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3588102aa0 0x7f3588198f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:47:13.064 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:13.056+0000 7f358cb22640 1 -- 172.21.15.39:0/3887792008 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f35881128c0 con 0x7f35881083e0 2024-09-21T03:47:13.064 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:13.056+0000 7f358cb22640 1 -- 172.21.15.39:0/3887792008 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f3588112bc0 con 0x7f3588102aa0 2024-09-21T03:47:13.064 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:13.056+0000 7f3585d9b640 1 --2- 172.21.15.39:0/3887792008 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f35881083e0 0x7f3588199460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:47:13.064 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:13.057+0000 7f3586d9d640 1 -- 172.21.15.39:0/3887792008 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3588102aa0 msgr2=0x7f3588198f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:47:13.064 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:13.057+0000 7f3586d9d640 1 --2- 172.21.15.39:0/3887792008 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3588102aa0 0x7f3588198f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:13.064 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:13.057+0000 7f3586d9d640 1 -- 172.21.15.39:0/3887792008 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f35881083e0 msgr2=0x7f3588199460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:47:13.065 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:13.057+0000 7f3586d9d640 1 --2- 172.21.15.39:0/3887792008 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f35881083e0 0x7f3588199460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:13.065 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:13.057+0000 7f3586d9d640 1 -- 172.21.15.39:0/3887792008 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f35881a85d0 con 0x7f3588108d30 2024-09-21T03:47:13.065 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:13.057+0000 7f3586d9d640 1 --2- 172.21.15.39:0/3887792008 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3588108d30 0x7f35881999a0 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f358000eaa0 tx=0x7f358000ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:47:13.065 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:13.058+0000 7f35777fe640 1 -- 172.21.15.39:0/3887792008 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f358000cdb0 con 0x7f3588108d30 2024-09-21T03:47:13.065 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:13.058+0000 7f35777fe640 1 -- 172.21.15.39:0/3887792008 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3580005020 con 0x7f3588108d30 2024-09-21T03:47:13.065 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:13.058+0000 7f358cb22640 1 -- 172.21.15.39:0/3887792008 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f35881a88c0 con 0x7f3588108d30 2024-09-21T03:47:13.065 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:13.058+0000 7f358cb22640 1 -- 172.21.15.39:0/3887792008 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f35881a8da0 con 0x7f3588108d30 2024-09-21T03:47:13.065 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:13.058+0000 7f35777fe640 1 -- 172.21.15.39:0/3887792008 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35800116d0 con 0x7f3588108d30 2024-09-21T03:47:13.066 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:13.062+0000 7f35777fe640 1 -- 172.21.15.39:0/3887792008 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 26) ==== 154694+0+0 (secure 0 0 0) 0x7f35800051d0 con 0x7f3588108d30 2024-09-21T03:47:13.066 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:13.062+0000 7f3593096740 1 Processor -- start 2024-09-21T03:47:13.066 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:13.063+0000 7f3593096740 1 -- start start 2024-09-21T03:47:13.066 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:13.063+0000 7f3593096740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56009a9e6070 0x56009ad95eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:47:13.066 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:13.063+0000 7f3593096740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56009a9bd1b0 0x56009ad963f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:47:13.066 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:13.063+0000 7f3593096740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56009a9c5390 0x56009ad96930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:47:13.067 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:13.063+0000 7f3593096740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x56009ad03ea0 con 0x56009a9c5390 2024-09-21T03:47:13.067 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:13.064+0000 7f3593096740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x56009acf32d0 con 0x56009a9bd1b0 2024-09-21T03:47:13.067 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:13.064+0000 7f3593096740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x56009acf70e0 con 0x56009a9e6070 2024-09-21T03:47:13.067 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:13.064+0000 7f35777fe640 1 --2- 172.21.15.39:0/3887792008 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f35700b6850 0x7f35700b8d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:47:13.067 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:13.064+0000 7f35777fe640 1 -- 172.21.15.39:0/3887792008 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8484+0+0 (secure 0 0 0) 0x7f35800e79b0 con 0x7f3588108d30 2024-09-21T03:47:13.067 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:13.064+0000 7f3586d9d640 1 --2- 172.21.15.39:0/3887792008 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f35700b6850 0x7f35700b8d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:47:13.067 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:13.064+0000 7f3585d9b640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56009a9e6070 0x56009ad95eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:47:13.067 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:13.064+0000 7f3585d9b640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56009a9e6070 0x56009ad95eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.39:37902/0 (socket says 172.21.15.39:37902) 2024-09-21T03:47:13.067 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:13.064+0000 7f3585d9b640 1 -- 172.21.15.39:0/3205643701 learned_addr learned my addr 172.21.15.39:0/3205643701 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:47:13.067 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:13.064+0000 7f358659c640 1 --2- 172.21.15.39:0/3205643701 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56009a9bd1b0 0x56009ad963f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:47:13.068 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:13.065+0000 7f3586d9d640 1 --2- 172.21.15.39:0/3887792008 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f35700b6850 0x7f35700b8d60 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f35800e5060 tx=0x7f358000cf50 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:47:13.068 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:13.065+0000 7f3585d9b640 1 --2- 172.21.15.39:0/3205643701 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56009a9c5390 0x56009ad96930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:47:13.068 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:13.065+0000 7f3585d9b640 1 -- 172.21.15.39:0/3205643701 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56009a9bd1b0 msgr2=0x56009ad963f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:47:13.068 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:13.065+0000 7f3585d9b640 1 --2- 172.21.15.39:0/3205643701 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56009a9bd1b0 0x56009ad963f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:13.069 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:13.065+0000 7f3585d9b640 1 -- 172.21.15.39:0/3205643701 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56009a9c5390 msgr2=0x56009ad96930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:47:13.069 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:13.065+0000 7f3585d9b640 1 --2- 172.21.15.39:0/3205643701 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56009a9c5390 0x56009ad96930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:13.069 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:13.065+0000 7f3585d9b640 1 -- 172.21.15.39:0/3205643701 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56009aa51650 con 0x56009a9e6070 2024-09-21T03:47:13.069 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:13.066+0000 7f3585d9b640 1 --2- 172.21.15.39:0/3205643701 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56009a9e6070 0x56009ad95eb0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f357c009900 tx=0x7f357c031400 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b7af6670da223945 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:47:13.069 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:13.066+0000 7f3574ff9640 1 -- 172.21.15.39:0/3205643701 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f357c035040 con 0x56009a9e6070 2024-09-21T03:47:13.069 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:13.066+0000 7f3574ff9640 1 -- 172.21.15.39:0/3205643701 <== mon.1 v2:172.21.15.161:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f357c004020 con 0x56009a9e6070 2024-09-21T03:47:13.069 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:13.066+0000 7f3574ff9640 1 -- 172.21.15.39:0/3205643701 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f357c003c00 con 0x56009a9e6070 2024-09-21T03:47:13.070 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:13.068+0000 7f3593096740 1 -- 172.21.15.39:0/3205643701 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56009a9e6070 msgr2=0x56009ad95eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:47:13.070 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:13.068+0000 7f3593096740 1 --2- 172.21.15.39:0/3205643701 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56009a9e6070 0x56009ad95eb0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f357c009900 tx=0x7f357c031400 comp rx=0 tx=0).stop 2024-09-21T03:47:13.070 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:13.068+0000 7f3593096740 1 -- 172.21.15.39:0/3205643701 shutdown_connections 2024-09-21T03:47:13.070 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:13.068+0000 7f3593096740 1 --2- 172.21.15.39:0/3205643701 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56009a9c5390 0x56009ad96930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:13.070 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:13.068+0000 7f3593096740 1 --2- 172.21.15.39:0/3205643701 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56009a9bd1b0 0x56009ad963f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:13.070 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:13.068+0000 7f3593096740 1 --2- 172.21.15.39:0/3205643701 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56009a9e6070 0x56009ad95eb0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:13.070 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:13.068+0000 7f3593096740 1 -- 172.21.15.39:0/3205643701 >> 172.21.15.39:0/3205643701 conn(0x56009a9da460 msgr2=0x56009ad93da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:47:13.070 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:13.068+0000 7f3593096740 1 -- 172.21.15.39:0/3205643701 shutdown_connections 2024-09-21T03:47:13.072 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:13.070+0000 7f3593096740 1 -- 172.21.15.39:0/3205643701 wait complete. 2024-09-21T03:47:13.072 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:13.071+0000 7f3593096740 1 -- start start 2024-09-21T03:47:13.072 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:13.071+0000 7f3593096740 1 Processor -- start 2024-09-21T03:47:13.073 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:13.071+0000 7f3593096740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56009a9bd1b0 0x56009ada1520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:47:13.073 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:13.071+0000 7f3593096740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56009a9c5390 0x56009ada1a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:47:13.073 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:13.071+0000 7f3593096740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56009a9e6070 0x56009adad950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:47:13.073 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:13.072+0000 7f3593096740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x56009ab54700 con 0x56009a9c5390 2024-09-21T03:47:13.073 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:13.072+0000 7f3593096740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x56009ac8e160 con 0x56009a9e6070 2024-09-21T03:47:13.074 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:13.072+0000 7f3593096740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x56009ada4630 con 0x56009a9bd1b0 2024-09-21T03:47:13.074 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:13.072+0000 7f3585d9b640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56009a9bd1b0 0x56009ada1520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:47:13.074 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:13.072+0000 7f3585d9b640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56009a9bd1b0 0x56009ada1520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.39:37904/0 (socket says 172.21.15.39:37904) 2024-09-21T03:47:13.074 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:13.072+0000 7f3585d9b640 1 -- 172.21.15.39:0/4252277772 learned_addr learned my addr 172.21.15.39:0/4252277772 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:47:13.074 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:13.072+0000 7f358659c640 1 --2- 172.21.15.39:0/4252277772 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56009a9c5390 0x56009ada1a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:47:13.075 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:13.072+0000 7f3593096740 10 client.0 _ll_register_callbacks cb 0x56009aca79e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:47:13.075 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:13.072+0000 7f3585d9b640 1 -- 172.21.15.39:0/4252277772 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56009a9e6070 msgr2=0x56009adad950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:47:13.075 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:13.072+0000 7f3585d9b640 1 --2- 172.21.15.39:0/4252277772 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56009a9e6070 0x56009adad950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:13.075 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:13.072+0000 7f3585d9b640 1 -- 172.21.15.39:0/4252277772 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56009a9c5390 msgr2=0x56009ada1a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:47:13.075 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:13.072+0000 7f3585d9b640 1 --2- 172.21.15.39:0/4252277772 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56009a9c5390 0x56009ada1a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:13.075 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:13.072+0000 7f3585d9b640 1 -- 172.21.15.39:0/4252277772 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56009acbbc90 con 0x56009a9bd1b0 2024-09-21T03:47:13.075 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:13.073+0000 7f358659c640 1 --2- 172.21.15.39:0/4252277772 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56009a9c5390 0x56009ada1a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:47:13.075 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:13.073+0000 7f3585d9b640 1 --2- 172.21.15.39:0/4252277772 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56009a9bd1b0 0x56009ada1520 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f357c02abb0 tx=0x7f357c005fd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5bfd4c0eb5250e4b server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:47:13.075 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:13.073+0000 7f356dffb640 10 client.0 ms_handle_connect on v2:172.21.15.161:3300/0 2024-09-21T03:47:13.076 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:13.073+0000 7f356dffb640 1 -- 172.21.15.39:0/4252277772 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f357c04f020 con 0x56009a9bd1b0 2024-09-21T03:47:13.076 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:13.073+0000 7f3593096740 1 -- 172.21.15.39:0/4252277772 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f357c004020 con 0x56009a9bd1b0 2024-09-21T03:47:13.076 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:13.073+0000 7f356dffb640 1 -- 172.21.15.39:0/4252277772 <== mon.1 v2:172.21.15.161:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8484+0+0 (secure 0 0 0) 0x7f357c032d10 con 0x56009a9bd1b0 2024-09-21T03:47:13.076 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:13.074+0000 7f356dffb640 1 -- 172.21.15.39:0/4252277772 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f357c05a050 con 0x56009a9bd1b0 2024-09-21T03:47:13.076 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:13.074+0000 7f3593096740 10 client.25423 fetch_fsmap learned FSMap version 29 2024-09-21T03:47:13.076 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:13.074+0000 7f3593096740 1 -- 172.21.15.39:0/4252277772 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x56009adb0310 con 0x56009a9bd1b0 2024-09-21T03:47:13.078 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:13.074+0000 7f356dffb640 1 -- 172.21.15.39:0/4252277772 <== mon.1 v2:172.21.15.161:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f357c004020 con 0x56009a9bd1b0 2024-09-21T03:47:13.078 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:13.075+0000 7f356dffb640 1 -- 172.21.15.39:0/4252277772 <== mon.1 v2:172.21.15.161:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f357c0499a0 con 0x56009a9bd1b0 2024-09-21T03:47:13.078 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:13.075+0000 7f3593096740 10 client.25423 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T03:47:13.078 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:13.075+0000 7f3593096740 10 client.25423 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-21T03:47:13.078 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:13.075+0000 7f3593096740 20 client.25423 populate_metadata read hostname 'smithi039' 2024-09-21T03:47:13.078 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:13.075+0000 7f3593096740 1 --2- 172.21.15.39:0/4252277772 >> [v2:172.21.15.150:6836/4017043144,v1:172.21.15.150:6837/4017043144] conn(0x56009a9a49f0 0x56009adb3670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:47:13.078 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:13.075+0000 7f3593096740 4 client.25423 mds_command: new command op to 24659 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:47:13.078 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:13.075+0000 7f3593096740 1 -- 172.21.15.39:0/4252277772 --> [v2:172.21.15.150:6836/4017043144,v1:172.21.15.150:6837/4017043144] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56009ad05df0 con 0x56009a9a49f0 2024-09-21T03:47:13.079 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:13.076+0000 7f358659c640 1 --2- 172.21.15.39:0/4252277772 >> [v2:172.21.15.150:6836/4017043144,v1:172.21.15.150:6837/4017043144] conn(0x56009a9a49f0 0x56009adb3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:47:13.079 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:13.076+0000 7f358659c640 1 --2- 172.21.15.39:0/4252277772 >> [v2:172.21.15.150:6836/4017043144,v1:172.21.15.150:6837/4017043144] conn(0x56009a9a49f0 0x56009adb3670 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=72526ec93a88ed8 server_cookie=b1c9fe8e41a94440 in_seq=0 out_seq=0 2024-09-21T03:47:13.079 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:13.076+0000 7f356dffb640 10 client.25423 ms_handle_connect on v2:172.21.15.150:6836/4017043144 2024-09-21T03:47:13.079 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:13.077+0000 7f356dffb640 1 -- 172.21.15.39:0/4252277772 <== mds.2 v2:172.21.15.150:6836/4017043144 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x56009ad05df0 con 0x56009a9a49f0 2024-09-21T03:47:13.080 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:13.077+0000 7f356dffb640 10 client.25423 handle_command_reply: tid=0 2024-09-21T03:47:13.080 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:13.077+0000 7f3593096740 1 client.25423 shutdown 2024-09-21T03:47:13.080 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:13.078+0000 7f3593096740 1 -- 172.21.15.39:0/4252277772 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56009a9bd1b0 msgr2=0x56009ada1520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:47:13.080 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:13.078+0000 7f3593096740 1 --2- 172.21.15.39:0/4252277772 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56009a9bd1b0 0x56009ada1520 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f357c02abb0 tx=0x7f357c005fd0 comp rx=0 tx=0).stop 2024-09-21T03:47:13.080 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:13.078+0000 7f3593096740 1 -- 172.21.15.39:0/4252277772 shutdown_connections 2024-09-21T03:47:13.080 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:13.078+0000 7f3593096740 1 --2- 172.21.15.39:0/4252277772 >> [v2:172.21.15.150:6836/4017043144,v1:172.21.15.150:6837/4017043144] conn(0x56009a9a49f0 0x56009adb3670 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:13.080 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:13.078+0000 7f3593096740 1 --2- 172.21.15.39:0/4252277772 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56009a9e6070 0x56009adad950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:13.080 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:13.078+0000 7f3593096740 1 --2- 172.21.15.39:0/4252277772 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56009a9c5390 0x56009ada1a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:13.080 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:13.078+0000 7f3593096740 1 --2- 172.21.15.39:0/4252277772 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56009a9bd1b0 0x56009ada1520 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:13.080 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:13.078+0000 7f3593096740 1 -- 172.21.15.39:0/4252277772 >> 172.21.15.39:0/4252277772 conn(0x56009a9da460 msgr2=0x56009ad99040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:47:13.080 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:13.078+0000 7f356dffb640 0 client.25423 ms_handle_reset on v2:172.21.15.150:6836/4017043144 2024-09-21T03:47:13.080 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:13.078+0000 7f3593096740 1 -- 172.21.15.39:0/4252277772 shutdown_connections 2024-09-21T03:47:13.081 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:13.079+0000 7f3593096740 1 -- 172.21.15.39:0/4252277772 wait complete. 2024-09-21T03:47:13.081 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:13.079+0000 7f3593096740 20 client.25423 trim_cache size 0 max 16384 2024-09-21T03:47:13.107 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:13.105+0000 7f3593096740 1 Processor -- start 2024-09-21T03:47:13.107 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:13.105+0000 7f3593096740 1 -- start start 2024-09-21T03:47:13.108 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:13.105+0000 7f3593096740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56009a9bd1b0 0x56009ada0410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:47:13.108 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:13.105+0000 7f3593096740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56009a9c5390 0x56009adad8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:47:13.108 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:13.106+0000 7f3593096740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56009a9e6070 0x56009ad975f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:47:13.108 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:13.106+0000 7f3593096740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f35881ad530 con 0x56009a9c5390 2024-09-21T03:47:13.108 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:13.106+0000 7f358659c640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56009a9c5390 0x56009adad8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:47:13.108 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:13.106+0000 7f3593096740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f35881af070 con 0x56009a9e6070 2024-09-21T03:47:13.108 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:13.106+0000 7f358659c640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56009a9c5390 0x56009adad8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:51292/0 (socket says 172.21.15.39:51292) 2024-09-21T03:47:13.108 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:13.106+0000 7f358659c640 1 -- 172.21.15.39:0/1520465833 learned_addr learned my addr 172.21.15.39:0/1520465833 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:47:13.108 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:13.106+0000 7f3593096740 1 -- 172.21.15.39:0/1520465833 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f35881adc00 con 0x56009a9bd1b0 2024-09-21T03:47:13.108 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:13.106+0000 7f3585d9b640 1 --2- 172.21.15.39:0/1520465833 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56009a9bd1b0 0x56009ada0410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:47:13.109 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:13.106+0000 7f358659c640 1 -- 172.21.15.39:0/1520465833 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56009a9bd1b0 msgr2=0x56009ada0410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:47:13.109 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:13.106+0000 7f358659c640 1 --2- 172.21.15.39:0/1520465833 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56009a9bd1b0 0x56009ada0410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:13.109 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:13.106+0000 7f358659c640 1 -- 172.21.15.39:0/1520465833 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56009a9e6070 msgr2=0x56009ad975f0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-21T03:47:13.109 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:13.106+0000 7f358659c640 1 --2- 172.21.15.39:0/1520465833 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56009a9e6070 0x56009ad975f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:13.109 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:13.106+0000 7f358659c640 1 -- 172.21.15.39:0/1520465833 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56009ada27e0 con 0x56009a9c5390 2024-09-21T03:47:13.109 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:13.107+0000 7f358659c640 1 --2- 172.21.15.39:0/1520465833 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56009a9c5390 0x56009adad8c0 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f3578000c00 tx=0x7f3578004410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c4a1b4f996633cc server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:47:13.109 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:13.107+0000 7f356ffff640 1 -- 172.21.15.39:0/1520465833 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f357800bdc0 con 0x56009a9c5390 2024-09-21T03:47:13.110 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:13.107+0000 7f356ffff640 1 -- 172.21.15.39:0/1520465833 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3578005020 con 0x56009a9c5390 2024-09-21T03:47:13.110 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:13.107+0000 7f356ffff640 1 -- 172.21.15.39:0/1520465833 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3578004b50 con 0x56009a9c5390 2024-09-21T03:47:13.111 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:13.108+0000 7f3593096740 1 -- 172.21.15.39:0/1520465833 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56009a9c5390 msgr2=0x56009adad8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:47:13.112 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:13.108+0000 7f3593096740 1 --2- 172.21.15.39:0/1520465833 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56009a9c5390 0x56009adad8c0 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f3578000c00 tx=0x7f3578004410 comp rx=0 tx=0).stop 2024-09-21T03:47:13.112 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:13.108+0000 7f3593096740 1 -- 172.21.15.39:0/1520465833 shutdown_connections 2024-09-21T03:47:13.112 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:13.108+0000 7f3593096740 1 --2- 172.21.15.39:0/1520465833 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56009a9e6070 0x56009ad975f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:13.112 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:13.108+0000 7f3593096740 1 --2- 172.21.15.39:0/1520465833 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56009a9c5390 0x56009adad8c0 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:13.112 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:13.108+0000 7f3593096740 1 --2- 172.21.15.39:0/1520465833 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56009a9bd1b0 0x56009ada0410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:13.112 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:13.108+0000 7f3593096740 1 -- 172.21.15.39:0/1520465833 >> 172.21.15.39:0/1520465833 conn(0x56009a9da460 msgr2=0x56009ad96ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:47:13.112 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:13.108+0000 7f3593096740 1 -- 172.21.15.39:0/1520465833 shutdown_connections 2024-09-21T03:47:13.112 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:13.108+0000 7f3593096740 1 -- 172.21.15.39:0/1520465833 wait complete. 2024-09-21T03:47:13.112 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:13.109+0000 7f3593096740 1 -- start start 2024-09-21T03:47:13.112 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:13.109+0000 7f3593096740 1 Processor -- start 2024-09-21T03:47:13.112 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:13.109+0000 7f3593096740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56009a9bd1b0 0x56009adb7040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:47:13.112 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:13.109+0000 7f3593096740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56009a9c5390 0x56009adb4510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:47:13.113 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:13.110+0000 7f3593096740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56009a9e6070 0x56009adb4c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:47:13.113 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:13.110+0000 7f3585d9b640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56009a9bd1b0 0x56009adb7040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:47:13.113 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:13.110+0000 7f3593096740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f35881ae9a0 con 0x56009a9bd1b0 2024-09-21T03:47:13.113 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:13.110+0000 7f3585d9b640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56009a9bd1b0 0x56009adb7040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:51306/0 (socket says 172.21.15.39:51306) 2024-09-21T03:47:13.113 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:13.110+0000 7f3585d9b640 1 -- 172.21.15.39:0/732866014 learned_addr learned my addr 172.21.15.39:0/732866014 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:47:13.113 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:13.110+0000 7f3593096740 1 -- 172.21.15.39:0/732866014 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f35881ae2d0 con 0x56009a9e6070 2024-09-21T03:47:13.113 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:13.110+0000 7f3593096740 1 -- 172.21.15.39:0/732866014 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f35881ab550 con 0x56009a9c5390 2024-09-21T03:47:13.113 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:13.110+0000 7f358659c640 1 --2- 172.21.15.39:0/732866014 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56009a9c5390 0x56009adb4510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:47:13.113 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:13.110+0000 7f3593096740 10 client.0 _ll_register_callbacks cb 0x7f357c0324b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:47:13.113 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:13.110+0000 7f3585d9b640 1 --2- 172.21.15.39:0/732866014 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56009a9e6070 0x56009adb4c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:47:13.116 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:13.110+0000 7f3585d9b640 1 -- 172.21.15.39:0/732866014 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56009a9c5390 msgr2=0x56009adb4510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:47:13.116 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:13.110+0000 7f3585d9b640 1 --2- 172.21.15.39:0/732866014 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56009a9c5390 0x56009adb4510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:13.116 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:13.110+0000 7f3585d9b640 1 -- 172.21.15.39:0/732866014 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56009a9e6070 msgr2=0x56009adb4c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:47:13.116 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:13.110+0000 7f3585d9b640 1 --2- 172.21.15.39:0/732866014 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56009a9e6070 0x56009adb4c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:13.116 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:13.110+0000 7f3585d9b640 1 -- 172.21.15.39:0/732866014 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56009ae1cdf0 con 0x56009a9bd1b0 2024-09-21T03:47:13.116 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:13.111+0000 7f3585d9b640 1 --2- 172.21.15.39:0/732866014 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56009a9bd1b0 0x56009adb7040 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f357c049f10 tx=0x7f357c02a3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9343d05e45400d4 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:47:13.117 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:13.111+0000 7f356dffb640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-09-21T03:47:13.117 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:13.111+0000 7f356dffb640 1 -- 172.21.15.39:0/732866014 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f357c04f070 con 0x56009a9bd1b0 2024-09-21T03:47:13.117 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:13.111+0000 7f356dffb640 1 -- 172.21.15.39:0/732866014 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8484+0+0 (secure 0 0 0) 0x7f357c03a3f0 con 0x56009a9bd1b0 2024-09-21T03:47:13.117 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:13.111+0000 7f3593096740 1 -- 172.21.15.39:0/732866014 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3578005020 con 0x56009a9bd1b0 2024-09-21T03:47:13.117 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:13.111+0000 7f356dffb640 1 -- 172.21.15.39:0/732866014 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f357c035040 con 0x56009a9bd1b0 2024-09-21T03:47:13.117 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:13.111+0000 7f3593096740 10 client.15816 fetch_fsmap learned FSMap version 29 2024-09-21T03:47:13.117 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:13.111+0000 7f3593096740 1 -- 172.21.15.39:0/732866014 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x56009ae1dec0 con 0x56009a9bd1b0 2024-09-21T03:47:13.117 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:13.111+0000 7f356dffb640 1 -- 172.21.15.39:0/732866014 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3578005020 con 0x56009a9bd1b0 2024-09-21T03:47:13.117 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:13.112+0000 7f356dffb640 1 -- 172.21.15.39:0/732866014 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f357c067050 con 0x56009a9bd1b0 2024-09-21T03:47:13.117 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:13.112+0000 7f3593096740 10 client.15816 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T03:47:13.117 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:13.112+0000 7f3593096740 10 client.15816 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-21T03:47:13.117 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:13.112+0000 7f3593096740 20 client.15816 populate_metadata read hostname 'smithi039' 2024-09-21T03:47:13.117 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:13.112+0000 7f3593096740 1 --2- 172.21.15.39:0/732866014 >> [v2:172.21.15.150:6836/4017043144,v1:172.21.15.150:6837/4017043144] conn(0x56009a979da0 0x56009ae20600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:47:13.118 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:13.112+0000 7f3593096740 4 client.15816 mds_command: new command op to 24659 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T03:47:13.118 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:13.112+0000 7f3593096740 1 -- 172.21.15.39:0/732866014 --> [v2:172.21.15.150:6836/4017043144,v1:172.21.15.150:6837/4017043144] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56009ad018a0 con 0x56009a979da0 2024-09-21T03:47:13.118 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:13.112+0000 7f358659c640 1 --2- 172.21.15.39:0/732866014 >> [v2:172.21.15.150:6836/4017043144,v1:172.21.15.150:6837/4017043144] conn(0x56009a979da0 0x56009ae20600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:47:13.118 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:13.113+0000 7f358659c640 1 --2- 172.21.15.39:0/732866014 >> [v2:172.21.15.150:6836/4017043144,v1:172.21.15.150:6837/4017043144] conn(0x56009a979da0 0x56009ae20600 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=2d3c9db018375693 server_cookie=dc5355f55516ebf4 in_seq=0 out_seq=0 2024-09-21T03:47:13.118 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:13.113+0000 7f356dffb640 10 client.15816 ms_handle_connect on v2:172.21.15.150:6836/4017043144 2024-09-21T03:47:13.118 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:13.113+0000 7f356dffb640 1 -- 172.21.15.39:0/732866014 <== mds.2 v2:172.21.15.150:6836/4017043144 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56009ad018a0 con 0x56009a979da0 2024-09-21T03:47:13.118 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:13.113+0000 7f356dffb640 10 client.15816 handle_command_reply: tid=0 2024-09-21T03:47:13.118 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:13.113+0000 7f3593096740 1 client.15816 shutdown 2024-09-21T03:47:13.118 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:13.113+0000 7f3593096740 1 -- 172.21.15.39:0/732866014 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56009a9bd1b0 msgr2=0x56009adb7040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:47:13.118 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:13.113+0000 7f3593096740 1 --2- 172.21.15.39:0/732866014 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56009a9bd1b0 0x56009adb7040 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f357c049f10 tx=0x7f357c02a3f0 comp rx=0 tx=0).stop 2024-09-21T03:47:13.118 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:13.114+0000 7f3593096740 1 -- 172.21.15.39:0/732866014 shutdown_connections 2024-09-21T03:47:13.118 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:13.114+0000 7f3593096740 1 --2- 172.21.15.39:0/732866014 >> [v2:172.21.15.150:6836/4017043144,v1:172.21.15.150:6837/4017043144] conn(0x56009a979da0 0x56009ae20600 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:13.119 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:13.114+0000 7f3593096740 1 --2- 172.21.15.39:0/732866014 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56009a9e6070 0x56009adb4c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:13.119 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:13.114+0000 7f3593096740 1 --2- 172.21.15.39:0/732866014 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56009a9c5390 0x56009adb4510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:13.119 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:13.114+0000 7f3593096740 1 --2- 172.21.15.39:0/732866014 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56009a9bd1b0 0x56009adb7040 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:13.119 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:13.114+0000 7f356dffb640 0 client.15816 ms_handle_reset on v2:172.21.15.150:6836/4017043144 2024-09-21T03:47:13.119 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:13.114+0000 7f3593096740 1 -- 172.21.15.39:0/732866014 >> 172.21.15.39:0/732866014 conn(0x56009a9da460 msgr2=0x56009ad96e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:47:13.119 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:13.114+0000 7f3593096740 1 -- 172.21.15.39:0/732866014 shutdown_connections 2024-09-21T03:47:13.119 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:13.114+0000 7f3593096740 1 -- 172.21.15.39:0/732866014 wait complete. 2024-09-21T03:47:13.119 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:13.114+0000 7f3593096740 20 client.15816 trim_cache size 0 max 16384 2024-09-21T03:47:13.119 INFO:teuthology.orchestra.run.smithi039.stdout:[] 2024-09-21T03:47:13.120 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:13.115+0000 7f358cb22640 1 -- 172.21.15.39:0/3887792008 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f35700b6850 msgr2=0x7f35700b8d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:47:13.120 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:13.115+0000 7f358cb22640 1 --2- 172.21.15.39:0/3887792008 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f35700b6850 0x7f35700b8d60 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f35800e5060 tx=0x7f358000cf50 comp rx=0 tx=0).stop 2024-09-21T03:47:13.120 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:13.115+0000 7f358cb22640 1 -- 172.21.15.39:0/3887792008 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3588108d30 msgr2=0x7f35881999a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:47:13.120 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:13.115+0000 7f358cb22640 1 --2- 172.21.15.39:0/3887792008 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3588108d30 0x7f35881999a0 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f358000eaa0 tx=0x7f358000ef60 comp rx=0 tx=0).stop 2024-09-21T03:47:13.120 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:13.115+0000 7f358cb22640 1 -- 172.21.15.39:0/3887792008 shutdown_connections 2024-09-21T03:47:13.120 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:13.115+0000 7f358cb22640 1 --2- 172.21.15.39:0/3887792008 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f35700b6850 0x7f35700b8d60 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:13.121 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:13.115+0000 7f358cb22640 1 --2- 172.21.15.39:0/3887792008 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3588108d30 0x7f35881999a0 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:13.121 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:13.115+0000 7f358cb22640 1 --2- 172.21.15.39:0/3887792008 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f35881083e0 0x7f3588199460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:13.121 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:13.115+0000 7f358cb22640 1 --2- 172.21.15.39:0/3887792008 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3588102aa0 0x7f3588198f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:13.121 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:13.115+0000 7f358cb22640 1 -- 172.21.15.39:0/3887792008 >> 172.21.15.39:0/3887792008 conn(0x7f3588079750 msgr2=0x7f3588106290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:47:13.121 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:13.115+0000 7f358cb22640 1 -- 172.21.15.39:0/3887792008 shutdown_connections 2024-09-21T03:47:13.121 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:13.115+0000 7f358cb22640 1 -- 172.21.15.39:0/3887792008 wait complete. 2024-09-21T03:47:13.218 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:47:12 smithi161 ceph-mon[34533]: pgmap v360: 97 pgs: 97 active+clean; 1.4 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 24 op/s 2024-09-21T03:47:13.849 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 51116d5c-77ca-11ef-baf5-efdc52797490 -- ceph tell mds.1:1 damage ls 2024-09-21T03:47:14.250 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/51116d5c-77ca-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:47:14.832 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:14.829+0000 7fdcf36d5640 1 -- 172.21.15.39:0/140280949 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdcec1052c0 msgr2=0x7fdcec10d900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:47:14.832 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:14.829+0000 7fdcf36d5640 1 --2- 172.21.15.39:0/140280949 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdcec1052c0 0x7fdcec10d900 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7fdcdc009900 tx=0x7fdcdc02a3f0 comp rx=0 tx=0).stop 2024-09-21T03:47:14.832 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:14.829+0000 7fdcf36d5640 1 -- 172.21.15.39:0/140280949 shutdown_connections 2024-09-21T03:47:14.832 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:14.829+0000 7fdcf36d5640 1 --2- 172.21.15.39:0/140280949 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdcec10de40 0x7fdcec110310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:14.832 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:14.829+0000 7fdcf36d5640 1 --2- 172.21.15.39:0/140280949 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdcec1052c0 0x7fdcec10d900 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:14.832 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:14.829+0000 7fdcf36d5640 1 --2- 172.21.15.39:0/140280949 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdcec104890 0x7fdcec104ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:14.832 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:14.829+0000 7fdcf36d5640 1 -- 172.21.15.39:0/140280949 >> 172.21.15.39:0/140280949 conn(0x7fdcec079730 msgr2=0x7fdcec079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:47:14.832 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:14.830+0000 7fdcf36d5640 1 -- 172.21.15.39:0/140280949 shutdown_connections 2024-09-21T03:47:14.833 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:14.830+0000 7fdcf36d5640 1 -- 172.21.15.39:0/140280949 wait complete. 2024-09-21T03:47:14.833 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:14.831+0000 7fdcf36d5640 1 Processor -- start 2024-09-21T03:47:14.833 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:14.831+0000 7fdcf36d5640 1 -- start start 2024-09-21T03:47:14.834 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:14.832+0000 7fdcf36d5640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdcec104890 0x7fdcec1027a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:47:14.834 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:14.832+0000 7fdcf36d5640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdcec1052c0 0x7fdcec102ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:47:14.834 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:14.832+0000 7fdcf36d5640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdcec10de40 0x7fdcec103220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:47:14.834 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:14.832+0000 7fdcf0c70640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdcec1052c0 0x7fdcec102ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:47:14.834 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:14.832+0000 7fdcf36d5640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fdcec113a80 con 0x7fdcec1052c0 2024-09-21T03:47:14.834 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:14.832+0000 7fdcf36d5640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fdcec113900 con 0x7fdcec104890 2024-09-21T03:47:14.834 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:14.832+0000 7fdcf36d5640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fdcec113c00 con 0x7fdcec10de40 2024-09-21T03:47:14.836 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:14.832+0000 7fdcf1471640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdcec104890 0x7fdcec1027a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:47:14.836 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:14.832+0000 7fdcf1471640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdcec104890 0x7fdcec1027a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.39:56998/0 (socket says 172.21.15.39:56998) 2024-09-21T03:47:14.836 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:14.832+0000 7fdcf1471640 1 -- 172.21.15.39:0/995753761 learned_addr learned my addr 172.21.15.39:0/995753761 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:47:14.836 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:14.832+0000 7fdcf0c70640 1 --2- 172.21.15.39:0/995753761 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdcec1052c0 0x7fdcec102ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:51320/0 (socket says 172.21.15.39:51320) 2024-09-21T03:47:14.836 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:14.832+0000 7fdcf1c72640 1 --2- 172.21.15.39:0/995753761 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdcec10de40 0x7fdcec103220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:47:14.837 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:14.832+0000 7fdcf1471640 1 -- 172.21.15.39:0/995753761 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdcec10de40 msgr2=0x7fdcec103220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:47:14.837 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:14.832+0000 7fdcf1471640 1 --2- 172.21.15.39:0/995753761 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdcec10de40 0x7fdcec103220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:14.837 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:14.832+0000 7fdcf1471640 1 -- 172.21.15.39:0/995753761 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdcec1052c0 msgr2=0x7fdcec102ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:47:14.837 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:14.832+0000 7fdcf1471640 1 --2- 172.21.15.39:0/995753761 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdcec1052c0 0x7fdcec102ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:14.837 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:14.832+0000 7fdcf1471640 1 -- 172.21.15.39:0/995753761 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdcec1a87d0 con 0x7fdcec104890 2024-09-21T03:47:14.837 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:14.833+0000 7fdcf1471640 1 --2- 172.21.15.39:0/995753761 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdcec104890 0x7fdcec1027a0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fdce800e990 tx=0x7fdce800ee50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:47:14.837 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:14.833+0000 7fdce27fc640 1 -- 172.21.15.39:0/995753761 <== mon.2 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdce800cbe0 con 0x7fdcec104890 2024-09-21T03:47:14.837 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:14.833+0000 7fdcf36d5640 1 -- 172.21.15.39:0/995753761 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdcec1a8ac0 con 0x7fdcec104890 2024-09-21T03:47:14.837 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:14.833+0000 7fdce27fc640 1 -- 172.21.15.39:0/995753761 <== mon.2 v2:172.21.15.150:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdce800cd80 con 0x7fdcec104890 2024-09-21T03:47:14.837 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:14.833+0000 7fdce27fc640 1 -- 172.21.15.39:0/995753761 <== mon.2 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdce8011470 con 0x7fdcec104890 2024-09-21T03:47:14.837 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:14.834+0000 7fdcf36d5640 1 -- 172.21.15.39:0/995753761 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdcec1a9020 con 0x7fdcec104890 2024-09-21T03:47:14.840 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:14.837+0000 7fdce27fc640 1 -- 172.21.15.39:0/995753761 <== mon.2 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 26) ==== 154694+0+0 (secure 0 0 0) 0x7fdce8004020 con 0x7fdcec104890 2024-09-21T03:47:14.840 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:14.838+0000 7fdcf9c49740 1 Processor -- start 2024-09-21T03:47:14.840 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:14.838+0000 7fdcf9c49740 1 -- start start 2024-09-21T03:47:14.840 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:14.838+0000 7fdcf9c49740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55aa24d06070 0x55aa250ad6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:47:14.841 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:14.838+0000 7fdcf9c49740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55aa24cdd1b0 0x55aa250adbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:47:14.841 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:14.838+0000 7fdcf9c49740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55aa24ce5390 0x55aa250ae120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:47:14.841 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:14.838+0000 7fdcf9c49740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x55aa25023ea0 con 0x55aa24ce5390 2024-09-21T03:47:14.841 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:14.838+0000 7fdcf9c49740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x55aa250132d0 con 0x55aa24cdd1b0 2024-09-21T03:47:14.841 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:14.838+0000 7fdcf9c49740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55aa250170e0 con 0x55aa24d06070 2024-09-21T03:47:14.841 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:14.839+0000 7fdcf0c70640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55aa24ce5390 0x55aa250ae120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:47:14.841 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:14.839+0000 7fdcf0c70640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55aa24ce5390 0x55aa250ae120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:51336/0 (socket says 172.21.15.39:51336) 2024-09-21T03:47:14.842 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:14.839+0000 7fdcf0c70640 1 -- 172.21.15.39:0/2462317005 learned_addr learned my addr 172.21.15.39:0/2462317005 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:47:14.842 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:14.839+0000 7fdcf1c72640 1 --2- 172.21.15.39:0/2462317005 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55aa24d06070 0x55aa250ad6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:47:14.842 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:14.839+0000 7fdcf1471640 1 --2- 172.21.15.39:0/2462317005 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55aa24cdd1b0 0x55aa250adbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:47:14.842 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:14.839+0000 7fdcf0c70640 1 -- 172.21.15.39:0/2462317005 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55aa24d06070 msgr2=0x55aa250ad6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:47:14.843 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:14.839+0000 7fdcf0c70640 1 --2- 172.21.15.39:0/2462317005 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55aa24d06070 0x55aa250ad6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:14.843 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:14.839+0000 7fdcf0c70640 1 -- 172.21.15.39:0/2462317005 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55aa24cdd1b0 msgr2=0x55aa250adbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:47:14.843 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:14.839+0000 7fdcf0c70640 1 --2- 172.21.15.39:0/2462317005 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55aa24cdd1b0 0x55aa250adbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:14.843 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:14.839+0000 7fdcf0c70640 1 -- 172.21.15.39:0/2462317005 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55aa24d71650 con 0x55aa24ce5390 2024-09-21T03:47:14.843 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:14.840+0000 7fdcf1471640 1 --2- 172.21.15.39:0/2462317005 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55aa24cdd1b0 0x55aa250adbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:47:14.843 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:14.840+0000 7fdcf0c70640 1 --2- 172.21.15.39:0/2462317005 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55aa24ce5390 0x55aa250ae120 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7fdcdc009900 tx=0x7fdcdc032b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6425463c85b8f1f5 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:47:14.843 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:14.840+0000 7fdce27fc640 1 --2- 172.21.15.39:0/995753761 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7fdcd80b6790 0x7fdcd80b8ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:47:14.843 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:14.840+0000 7fdce27fc640 1 -- 172.21.15.39:0/995753761 <== mon.2 v2:172.21.15.150:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8484+0+0 (secure 0 0 0) 0x7fdce80e7790 con 0x7fdcec104890 2024-09-21T03:47:14.843 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:14.840+0000 7fdcf1c72640 1 --2- 172.21.15.39:0/995753761 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7fdcd80b6790 0x7fdcd80b8ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:47:14.843 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:14.841+0000 7fdcd77fe640 1 -- 172.21.15.39:0/2462317005 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcdc040070 con 0x55aa24ce5390 2024-09-21T03:47:14.844 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:14.841+0000 7fdcd77fe640 1 -- 172.21.15.39:0/2462317005 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdcdc004030 con 0x55aa24ce5390 2024-09-21T03:47:14.844 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:14.841+0000 7fdcd77fe640 1 -- 172.21.15.39:0/2462317005 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcdc035040 con 0x55aa24ce5390 2024-09-21T03:47:14.844 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:14.842+0000 7fdcf9c49740 1 -- 172.21.15.39:0/2462317005 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55aa24ce5390 msgr2=0x55aa250ae120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:47:14.844 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:14.842+0000 7fdcf1c72640 1 --2- 172.21.15.39:0/995753761 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7fdcd80b6790 0x7fdcd80b8ca0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fdce4005fd0 tx=0x7fdce4009500 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:47:14.844 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:14.842+0000 7fdcf9c49740 1 --2- 172.21.15.39:0/2462317005 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55aa24ce5390 0x55aa250ae120 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7fdcdc009900 tx=0x7fdcdc032b20 comp rx=0 tx=0).stop 2024-09-21T03:47:14.844 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:14.842+0000 7fdcf9c49740 1 -- 172.21.15.39:0/2462317005 shutdown_connections 2024-09-21T03:47:14.844 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:14.842+0000 7fdcf9c49740 1 --2- 172.21.15.39:0/2462317005 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55aa24ce5390 0x55aa250ae120 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:14.845 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:14.842+0000 7fdcf9c49740 1 --2- 172.21.15.39:0/2462317005 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55aa24cdd1b0 0x55aa250adbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:14.845 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:14.842+0000 7fdcf9c49740 1 --2- 172.21.15.39:0/2462317005 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55aa24d06070 0x55aa250ad6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:14.845 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:14.842+0000 7fdcf9c49740 1 -- 172.21.15.39:0/2462317005 >> 172.21.15.39:0/2462317005 conn(0x55aa24cfa460 msgr2=0x55aa250ab590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:47:14.845 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:14.842+0000 7fdcf9c49740 1 -- 172.21.15.39:0/2462317005 shutdown_connections 2024-09-21T03:47:14.845 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:14.842+0000 7fdcf9c49740 1 -- 172.21.15.39:0/2462317005 wait complete. 2024-09-21T03:47:14.845 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:14.843+0000 7fdcf9c49740 1 -- start start 2024-09-21T03:47:14.845 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:14.843+0000 7fdcf9c49740 1 Processor -- start 2024-09-21T03:47:14.846 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:14.844+0000 7fdcf9c49740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55aa24cdd1b0 0x55aa250c15a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:47:14.846 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:14.844+0000 7fdcf9c49740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55aa24ce5390 0x55aa250c1ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:47:14.846 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:14.844+0000 7fdcf9c49740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55aa24d06070 0x55aa250cd8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:47:14.846 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:14.844+0000 7fdcf9c49740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x55aa24e74700 con 0x55aa24d06070 2024-09-21T03:47:14.846 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:14.844+0000 7fdcf9c49740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x55aa24fae160 con 0x55aa24cdd1b0 2024-09-21T03:47:14.846 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:14.844+0000 7fdcf9c49740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55aa250c4550 con 0x55aa24ce5390 2024-09-21T03:47:14.847 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:14.844+0000 7fdcf1c72640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55aa24d06070 0x55aa250cd8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:47:14.847 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:14.844+0000 7fdcf1c72640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55aa24d06070 0x55aa250cd8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:51340/0 (socket says 172.21.15.39:51340) 2024-09-21T03:47:14.847 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:14.844+0000 7fdcf1c72640 1 -- 172.21.15.39:0/854963929 learned_addr learned my addr 172.21.15.39:0/854963929 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:47:14.847 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:14.844+0000 7fdcf1471640 1 --2- 172.21.15.39:0/854963929 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55aa24cdd1b0 0x55aa250c15a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:47:14.847 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:14.845+0000 7fdcf0c70640 1 --2- 172.21.15.39:0/854963929 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55aa24ce5390 0x55aa250c1ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:47:14.847 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:14.845+0000 7fdcf9c49740 10 client.0 _ll_register_callbacks cb 0x55aa24fc79e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:47:14.849 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:14.845+0000 7fdcf1c72640 1 -- 172.21.15.39:0/854963929 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55aa24ce5390 msgr2=0x55aa250c1ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:47:14.849 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:14.845+0000 7fdcf1c72640 1 --2- 172.21.15.39:0/854963929 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55aa24ce5390 0x55aa250c1ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:14.849 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:14.845+0000 7fdcf1c72640 1 -- 172.21.15.39:0/854963929 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55aa24cdd1b0 msgr2=0x55aa250c15a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:47:14.849 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:14.845+0000 7fdcf1c72640 1 --2- 172.21.15.39:0/854963929 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55aa24cdd1b0 0x55aa250c15a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:14.849 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:14.845+0000 7fdcf1c72640 1 -- 172.21.15.39:0/854963929 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55aa24fdbc90 con 0x55aa24d06070 2024-09-21T03:47:14.849 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:14.845+0000 7fdcf1c72640 1 --2- 172.21.15.39:0/854963929 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55aa24d06070 0x55aa250cd8b0 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7fdce4003f70 tx=0x7fdce4004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec07e639c51bbd5d server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:47:14.849 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:14.845+0000 7fdcd4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-09-21T03:47:14.849 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:14.846+0000 7fdcd4ff9640 1 -- 172.21.15.39:0/854963929 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdce4004b20 con 0x55aa24d06070 2024-09-21T03:47:14.850 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:14.846+0000 7fdcf9c49740 1 -- 172.21.15.39:0/854963929 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdcdc004030 con 0x55aa24d06070 2024-09-21T03:47:14.850 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:14.846+0000 7fdcd4ff9640 1 -- 172.21.15.39:0/854963929 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8484+0+0 (secure 0 0 0) 0x7fdce4004cc0 con 0x55aa24d06070 2024-09-21T03:47:14.850 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:14.846+0000 7fdcd4ff9640 1 -- 172.21.15.39:0/854963929 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fdce400f8f0 con 0x55aa24d06070 2024-09-21T03:47:14.850 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:14.846+0000 7fdcf9c49740 10 client.15828 fetch_fsmap learned FSMap version 29 2024-09-21T03:47:14.850 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:14.846+0000 7fdcf9c49740 1 -- 172.21.15.39:0/854963929 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55aa250d02b0 con 0x55aa24d06070 2024-09-21T03:47:14.850 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:14.846+0000 7fdcd4ff9640 1 -- 172.21.15.39:0/854963929 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdcdc004030 con 0x55aa24d06070 2024-09-21T03:47:14.850 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:14.847+0000 7fdcd4ff9640 1 -- 172.21.15.39:0/854963929 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fdce402e070 con 0x55aa24d06070 2024-09-21T03:47:14.850 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:14.847+0000 7fdcf9c49740 10 client.15828 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T03:47:14.850 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:14.847+0000 7fdcf9c49740 10 client.15828 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-21T03:47:14.850 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:14.847+0000 7fdcf9c49740 20 client.15828 populate_metadata read hostname 'smithi039' 2024-09-21T03:47:14.850 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:14.847+0000 7fdcf9c49740 1 --2- 172.21.15.39:0/854963929 >> [v2:172.21.15.161:6838/2190107270,v1:172.21.15.161:6839/2190107270] conn(0x55aa24cc49f0 0x55aa250d38d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:47:14.850 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:14.847+0000 7fdcf9c49740 4 client.15828 mds_command: new command op to 24667 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:47:14.850 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:14.847+0000 7fdcf9c49740 1 -- 172.21.15.39:0/854963929 --> [v2:172.21.15.161:6838/2190107270,v1:172.21.15.161:6839/2190107270] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55aa25025df0 con 0x55aa24cc49f0 2024-09-21T03:47:14.850 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:14.847+0000 7fdcf1471640 1 --2- 172.21.15.39:0/854963929 >> [v2:172.21.15.161:6838/2190107270,v1:172.21.15.161:6839/2190107270] conn(0x55aa24cc49f0 0x55aa250d38d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:47:14.852 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:14.848+0000 7fdcd4ff9640 10 client.15828 ms_handle_connect on v2:172.21.15.161:6838/2190107270 2024-09-21T03:47:14.852 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:14.848+0000 7fdcf1471640 1 --2- 172.21.15.39:0/854963929 >> [v2:172.21.15.161:6838/2190107270,v1:172.21.15.161:6839/2190107270] conn(0x55aa24cc49f0 0x55aa250d38d0 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=df36924a72a5de7c server_cookie=1d82fb470e729918 in_seq=0 out_seq=0 2024-09-21T03:47:14.852 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:14.849+0000 7fdcd4ff9640 1 -- 172.21.15.39:0/854963929 <== mds.1 v2:172.21.15.161:6838/2190107270 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55aa25025df0 con 0x55aa24cc49f0 2024-09-21T03:47:14.852 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:14.849+0000 7fdcd4ff9640 10 client.15828 handle_command_reply: tid=0 2024-09-21T03:47:14.853 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:14.849+0000 7fdcf9c49740 1 client.15828 shutdown 2024-09-21T03:47:14.853 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:14.849+0000 7fdcf9c49740 1 -- 172.21.15.39:0/854963929 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55aa24d06070 msgr2=0x55aa250cd8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:47:14.853 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:14.849+0000 7fdcf9c49740 1 --2- 172.21.15.39:0/854963929 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55aa24d06070 0x55aa250cd8b0 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7fdce4003f70 tx=0x7fdce4004ab0 comp rx=0 tx=0).stop 2024-09-21T03:47:14.853 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:14.850+0000 7fdcf9c49740 1 -- 172.21.15.39:0/854963929 shutdown_connections 2024-09-21T03:47:14.853 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:14.850+0000 7fdcf9c49740 1 --2- 172.21.15.39:0/854963929 >> [v2:172.21.15.161:6838/2190107270,v1:172.21.15.161:6839/2190107270] conn(0x55aa24cc49f0 0x55aa250d38d0 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:14.853 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:14.850+0000 7fdcf9c49740 1 --2- 172.21.15.39:0/854963929 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55aa24d06070 0x55aa250cd8b0 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:14.853 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:14.850+0000 7fdcf9c49740 1 --2- 172.21.15.39:0/854963929 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55aa24ce5390 0x55aa250c1ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:14.853 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:14.850+0000 7fdcf9c49740 1 --2- 172.21.15.39:0/854963929 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55aa24cdd1b0 0x55aa250c15a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:14.853 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:14.850+0000 7fdcf9c49740 1 -- 172.21.15.39:0/854963929 >> 172.21.15.39:0/854963929 conn(0x55aa24cfa460 msgr2=0x55aa250a9bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:47:14.853 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:14.850+0000 7fdcd4ff9640 0 client.15828 ms_handle_reset on v2:172.21.15.161:6838/2190107270 2024-09-21T03:47:14.853 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:14.850+0000 7fdcf9c49740 1 -- 172.21.15.39:0/854963929 shutdown_connections 2024-09-21T03:47:14.853 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:14.850+0000 7fdcf9c49740 1 -- 172.21.15.39:0/854963929 wait complete. 2024-09-21T03:47:14.853 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:14.850+0000 7fdcf9c49740 20 client.15828 trim_cache size 0 max 16384 2024-09-21T03:47:14.878 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:14.877+0000 7fdcf9c49740 1 Processor -- start 2024-09-21T03:47:14.879 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:14.877+0000 7fdcf9c49740 1 -- start start 2024-09-21T03:47:14.879 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:14.877+0000 7fdcf9c49740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55aa24cdd1b0 0x55aa250b2f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:47:14.880 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:14.877+0000 7fdcf9c49740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55aa24ce5390 0x55aa250cd7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:47:14.880 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:14.877+0000 7fdcf9c49740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55aa24d06070 0x55aa250be790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:47:14.881 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:14.877+0000 7fdcf9c49740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fdcec1ad520 con 0x55aa24d06070 2024-09-21T03:47:14.881 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:14.877+0000 7fdcf9c49740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fdcec1af060 con 0x55aa24ce5390 2024-09-21T03:47:14.881 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:14.877+0000 7fdcf9c49740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fdcec1adbf0 con 0x55aa24cdd1b0 2024-09-21T03:47:14.881 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:14.877+0000 7fdcf1471640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55aa24cdd1b0 0x55aa250b2f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:47:14.881 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:14.878+0000 7fdcf1471640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55aa24cdd1b0 0x55aa250b2f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.39:37974/0 (socket says 172.21.15.39:37974) 2024-09-21T03:47:14.881 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:14.878+0000 7fdcf1471640 1 -- 172.21.15.39:0/3132538704 learned_addr learned my addr 172.21.15.39:0/3132538704 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:47:14.881 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:14.878+0000 7fdcf1c72640 1 --2- 172.21.15.39:0/3132538704 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55aa24d06070 0x55aa250be790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:47:14.881 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:14.878+0000 7fdcf0c70640 1 --2- 172.21.15.39:0/3132538704 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55aa24ce5390 0x55aa250cd7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:47:14.881 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:14.878+0000 7fdcf1471640 1 -- 172.21.15.39:0/3132538704 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55aa24ce5390 msgr2=0x55aa250cd7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:47:14.881 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:14.878+0000 7fdcf1471640 1 --2- 172.21.15.39:0/3132538704 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55aa24ce5390 0x55aa250cd7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:14.881 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:14.878+0000 7fdcf1471640 1 -- 172.21.15.39:0/3132538704 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55aa24d06070 msgr2=0x55aa250be790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:47:14.881 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:14.878+0000 7fdcf1471640 1 --2- 172.21.15.39:0/3132538704 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55aa24d06070 0x55aa250be790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:14.881 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:14.878+0000 7fdcf1471640 1 -- 172.21.15.39:0/3132538704 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55aa250b04b0 con 0x55aa24cdd1b0 2024-09-21T03:47:14.881 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:14.878+0000 7fdcf0c70640 1 --2- 172.21.15.39:0/3132538704 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55aa24ce5390 0x55aa250cd7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:47:14.882 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:14.878+0000 7fdcf1c72640 1 --2- 172.21.15.39:0/3132538704 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55aa24d06070 0x55aa250be790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:47:14.882 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:14.878+0000 7fdcf1471640 1 --2- 172.21.15.39:0/3132538704 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55aa24cdd1b0 0x55aa250b2f20 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fdce8003a20 tx=0x7fdce8000bc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ebf6e52b0ca7039b server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:47:14.882 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:14.879+0000 7fdcd6ffd640 1 -- 172.21.15.39:0/3132538704 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdce801f040 con 0x55aa24cdd1b0 2024-09-21T03:47:14.882 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:14.879+0000 7fdcd6ffd640 1 -- 172.21.15.39:0/3132538704 <== mon.1 v2:172.21.15.161:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdce80b1f10 con 0x55aa24cdd1b0 2024-09-21T03:47:14.882 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:14.879+0000 7fdcd6ffd640 1 -- 172.21.15.39:0/3132538704 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdce80a7be0 con 0x55aa24cdd1b0 2024-09-21T03:47:14.883 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:14.880+0000 7fdcf9c49740 1 -- 172.21.15.39:0/3132538704 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55aa24cdd1b0 msgr2=0x55aa250b2f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:47:14.883 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:14.880+0000 7fdcf9c49740 1 --2- 172.21.15.39:0/3132538704 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55aa24cdd1b0 0x55aa250b2f20 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fdce8003a20 tx=0x7fdce8000bc0 comp rx=0 tx=0).stop 2024-09-21T03:47:14.883 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:14.880+0000 7fdcf9c49740 1 -- 172.21.15.39:0/3132538704 shutdown_connections 2024-09-21T03:47:14.883 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:14.880+0000 7fdcf9c49740 1 --2- 172.21.15.39:0/3132538704 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55aa24d06070 0x55aa250be790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:14.883 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:14.880+0000 7fdcf9c49740 1 --2- 172.21.15.39:0/3132538704 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55aa24ce5390 0x55aa250cd7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:14.883 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:14.880+0000 7fdcf9c49740 1 --2- 172.21.15.39:0/3132538704 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55aa24cdd1b0 0x55aa250b2f20 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:14.883 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:14.880+0000 7fdcf9c49740 1 -- 172.21.15.39:0/3132538704 >> 172.21.15.39:0/3132538704 conn(0x55aa24cfa460 msgr2=0x55aa250b2810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:47:14.883 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:14.880+0000 7fdcf9c49740 1 -- 172.21.15.39:0/3132538704 shutdown_connections 2024-09-21T03:47:14.883 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:14.881+0000 7fdcf9c49740 1 -- 172.21.15.39:0/3132538704 wait complete. 2024-09-21T03:47:14.883 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:14.881+0000 7fdcf9c49740 1 -- start start 2024-09-21T03:47:14.884 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:14.881+0000 7fdcf9c49740 1 Processor -- start 2024-09-21T03:47:14.886 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:14.882+0000 7fdcf9c49740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55aa24cdd1b0 0x55aa250c5990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:47:14.886 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:14.882+0000 7fdcf9c49740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55aa24ce5390 0x55aa250c5ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:47:14.886 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:14.882+0000 7fdcf9c49740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55aa24d06070 0x55aa250c6410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:47:14.886 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:14.882+0000 7fdcf9c49740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fdcec1ae990 con 0x55aa24cdd1b0 2024-09-21T03:47:14.886 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:14.882+0000 7fdcf9c49740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fdcec1ae2c0 con 0x55aa24ce5390 2024-09-21T03:47:14.886 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:14.882+0000 7fdcf9c49740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fdcec1ab500 con 0x55aa24d06070 2024-09-21T03:47:14.886 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:14.882+0000 7fdcf9c49740 10 client.0 _ll_register_callbacks cb 0x7fdce4009e40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:47:14.886 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:14.882+0000 7fdcf1471640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55aa24cdd1b0 0x55aa250c5990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:47:14.886 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:14.882+0000 7fdcf1c72640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55aa24d06070 0x55aa250c6410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:47:14.886 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:14.882+0000 7fdcf1c72640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55aa24d06070 0x55aa250c6410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.39:37982/0 (socket says 172.21.15.39:37982) 2024-09-21T03:47:14.887 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:14.882+0000 7fdcf1c72640 1 -- 172.21.15.39:0/2508859189 learned_addr learned my addr 172.21.15.39:0/2508859189 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:47:14.887 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:14.883+0000 7fdcf0c70640 1 --2- 172.21.15.39:0/2508859189 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55aa24ce5390 0x55aa250c5ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:47:14.887 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:14.883+0000 7fdcf1c72640 1 -- 172.21.15.39:0/2508859189 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55aa24ce5390 msgr2=0x55aa250c5ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:47:14.887 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:14.883+0000 7fdcf1c72640 1 --2- 172.21.15.39:0/2508859189 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55aa24ce5390 0x55aa250c5ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:14.887 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:14.883+0000 7fdcf1c72640 1 -- 172.21.15.39:0/2508859189 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55aa24cdd1b0 msgr2=0x55aa250c5990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:47:14.887 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:14.883+0000 7fdcf1c72640 1 --2- 172.21.15.39:0/2508859189 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55aa24cdd1b0 0x55aa250c5990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:14.887 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:14.883+0000 7fdcf1c72640 1 -- 172.21.15.39:0/2508859189 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55aa2513d480 con 0x55aa24d06070 2024-09-21T03:47:14.887 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:14.883+0000 7fdcf1471640 1 --2- 172.21.15.39:0/2508859189 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55aa24cdd1b0 0x55aa250c5990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:47:14.887 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:14.883+0000 7fdcf0c70640 1 --2- 172.21.15.39:0/2508859189 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55aa24ce5390 0x55aa250c5ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:47:14.887 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:14.883+0000 7fdcf1c72640 1 --2- 172.21.15.39:0/2508859189 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55aa24d06070 0x55aa250c6410 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fdce4011e20 tx=0x7fdce401e980 comp rx=0 tx=0).ready entity=mon.1 client_cookie=854ce49ce1387f1 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:47:14.887 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:14.883+0000 7fdcd4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.161:3300/0 2024-09-21T03:47:14.887 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:14.883+0000 7fdcd4ff9640 1 -- 172.21.15.39:0/2508859189 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdce4034040 con 0x55aa24d06070 2024-09-21T03:47:14.889 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:14.883+0000 7fdcd4ff9640 1 -- 172.21.15.39:0/2508859189 <== mon.1 v2:172.21.15.161:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8484+0+0 (secure 0 0 0) 0x7fdce401ec70 con 0x55aa24d06070 2024-09-21T03:47:14.889 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:14.883+0000 7fdcf9c49740 1 -- 172.21.15.39:0/2508859189 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdce80b1f10 con 0x55aa24d06070 2024-09-21T03:47:14.889 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:14.884+0000 7fdcd4ff9640 1 -- 172.21.15.39:0/2508859189 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fdce400fe50 con 0x55aa24d06070 2024-09-21T03:47:14.889 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:14.884+0000 7fdcf9c49740 10 client.25444 fetch_fsmap learned FSMap version 29 2024-09-21T03:47:14.889 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:14.884+0000 7fdcf9c49740 1 -- 172.21.15.39:0/2508859189 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55aa2513e700 con 0x55aa24d06070 2024-09-21T03:47:14.889 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:14.884+0000 7fdcd4ff9640 1 -- 172.21.15.39:0/2508859189 <== mon.1 v2:172.21.15.161:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdce80b1f10 con 0x55aa24d06070 2024-09-21T03:47:14.889 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:14.884+0000 7fdcd4ff9640 1 -- 172.21.15.39:0/2508859189 <== mon.1 v2:172.21.15.161:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fdce4040070 con 0x55aa24d06070 2024-09-21T03:47:14.889 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:14.885+0000 7fdcf9c49740 10 client.25444 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T03:47:14.889 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:14.885+0000 7fdcf9c49740 10 client.25444 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-21T03:47:14.889 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:14.885+0000 7fdcf9c49740 20 client.25444 populate_metadata read hostname 'smithi039' 2024-09-21T03:47:14.889 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:14.885+0000 7fdcf9c49740 1 --2- 172.21.15.39:0/2508859189 >> [v2:172.21.15.161:6838/2190107270,v1:172.21.15.161:6839/2190107270] conn(0x55aa24c99da0 0x55aa25140f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:47:14.889 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:14.885+0000 7fdcf9c49740 4 client.25444 mds_command: new command op to 24667 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T03:47:14.889 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:14.885+0000 7fdcf9c49740 1 -- 172.21.15.39:0/2508859189 --> [v2:172.21.15.161:6838/2190107270,v1:172.21.15.161:6839/2190107270] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55aa250218a0 con 0x55aa24c99da0 2024-09-21T03:47:14.890 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:14.885+0000 7fdcf1471640 1 --2- 172.21.15.39:0/2508859189 >> [v2:172.21.15.161:6838/2190107270,v1:172.21.15.161:6839/2190107270] conn(0x55aa24c99da0 0x55aa25140f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:47:14.890 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:14.886+0000 7fdcd4ff9640 10 client.25444 ms_handle_connect on v2:172.21.15.161:6838/2190107270 2024-09-21T03:47:14.890 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:14.886+0000 7fdcf1471640 1 --2- 172.21.15.39:0/2508859189 >> [v2:172.21.15.161:6838/2190107270,v1:172.21.15.161:6839/2190107270] conn(0x55aa24c99da0 0x55aa25140f40 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=5e3c219f02977c38 server_cookie=3b9ff59996eb8cfa in_seq=0 out_seq=0 2024-09-21T03:47:14.890 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:14.886+0000 7fdcd4ff9640 1 -- 172.21.15.39:0/2508859189 <== mds.1 v2:172.21.15.161:6838/2190107270 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55aa250218a0 con 0x55aa24c99da0 2024-09-21T03:47:14.890 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:14.886+0000 7fdcd4ff9640 10 client.25444 handle_command_reply: tid=0 2024-09-21T03:47:14.890 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:14.886+0000 7fdcf9c49740 1 client.25444 shutdown 2024-09-21T03:47:14.890 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:14.887+0000 7fdcf9c49740 1 -- 172.21.15.39:0/2508859189 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55aa24d06070 msgr2=0x55aa250c6410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:47:14.890 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:14.887+0000 7fdcf9c49740 1 --2- 172.21.15.39:0/2508859189 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55aa24d06070 0x55aa250c6410 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fdce4011e20 tx=0x7fdce401e980 comp rx=0 tx=0).stop 2024-09-21T03:47:14.890 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:14.887+0000 7fdcf9c49740 1 -- 172.21.15.39:0/2508859189 shutdown_connections 2024-09-21T03:47:14.890 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:14.887+0000 7fdcf9c49740 1 --2- 172.21.15.39:0/2508859189 >> [v2:172.21.15.161:6838/2190107270,v1:172.21.15.161:6839/2190107270] conn(0x55aa24c99da0 0x55aa25140f40 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:14.890 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:14.887+0000 7fdcf9c49740 1 --2- 172.21.15.39:0/2508859189 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55aa24d06070 0x55aa250c6410 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:14.890 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:14.887+0000 7fdcf9c49740 1 --2- 172.21.15.39:0/2508859189 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55aa24ce5390 0x55aa250c5ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:14.890 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:14.887+0000 7fdcd4ff9640 0 client.25444 ms_handle_reset on v2:172.21.15.161:6838/2190107270 2024-09-21T03:47:14.890 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:14.887+0000 7fdcf9c49740 1 --2- 172.21.15.39:0/2508859189 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55aa24cdd1b0 0x55aa250c5990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:14.891 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:14.887+0000 7fdcf9c49740 1 -- 172.21.15.39:0/2508859189 >> 172.21.15.39:0/2508859189 conn(0x55aa24cfa460 msgr2=0x55aa250ae6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:47:14.891 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:14.887+0000 7fdcf9c49740 1 -- 172.21.15.39:0/2508859189 shutdown_connections 2024-09-21T03:47:14.891 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:14.887+0000 7fdcf9c49740 1 -- 172.21.15.39:0/2508859189 wait complete. 2024-09-21T03:47:14.891 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:14.888+0000 7fdcf9c49740 20 client.25444 trim_cache size 0 max 16384 2024-09-21T03:47:14.891 INFO:teuthology.orchestra.run.smithi039.stdout:[] 2024-09-21T03:47:14.892 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:14.889+0000 7fdcf36d5640 1 -- 172.21.15.39:0/995753761 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7fdcd80b6790 msgr2=0x7fdcd80b8ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:47:14.892 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:14.889+0000 7fdcf36d5640 1 --2- 172.21.15.39:0/995753761 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7fdcd80b6790 0x7fdcd80b8ca0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fdce4005fd0 tx=0x7fdce4009500 comp rx=0 tx=0).stop 2024-09-21T03:47:14.892 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:14.889+0000 7fdcf36d5640 1 -- 172.21.15.39:0/995753761 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdcec104890 msgr2=0x7fdcec1027a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:47:14.892 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:14.889+0000 7fdcf36d5640 1 --2- 172.21.15.39:0/995753761 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdcec104890 0x7fdcec1027a0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fdce800e990 tx=0x7fdce800ee50 comp rx=0 tx=0).stop 2024-09-21T03:47:14.892 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:14.889+0000 7fdcf36d5640 1 -- 172.21.15.39:0/995753761 shutdown_connections 2024-09-21T03:47:14.892 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:14.889+0000 7fdcf36d5640 1 --2- 172.21.15.39:0/995753761 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7fdcd80b6790 0x7fdcd80b8ca0 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:14.892 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:14.889+0000 7fdcf36d5640 1 --2- 172.21.15.39:0/995753761 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdcec10de40 0x7fdcec103220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:14.893 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:14.889+0000 7fdcf36d5640 1 --2- 172.21.15.39:0/995753761 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdcec1052c0 0x7fdcec102ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:14.893 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:14.889+0000 7fdcf36d5640 1 --2- 172.21.15.39:0/995753761 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdcec104890 0x7fdcec1027a0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:14.893 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:14.889+0000 7fdcf36d5640 1 -- 172.21.15.39:0/995753761 >> 172.21.15.39:0/995753761 conn(0x7fdcec079730 msgr2=0x7fdcec1009f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:47:14.893 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:14.889+0000 7fdcf36d5640 1 -- 172.21.15.39:0/995753761 shutdown_connections 2024-09-21T03:47:14.893 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:14.889+0000 7fdcf36d5640 1 -- 172.21.15.39:0/995753761 wait complete. 2024-09-21T03:47:15.084 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:47:14 smithi039 ceph-mon[31211]: pgmap v361: 97 pgs: 97 active+clean; 1.4 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 24 op/s 2024-09-21T03:47:15.218 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:47:14 smithi161 ceph-mon[34533]: pgmap v361: 97 pgs: 97 active+clean; 1.4 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 24 op/s 2024-09-21T03:47:15.224 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:47:14 smithi150 ceph-mon[34289]: pgmap v361: 97 pgs: 97 active+clean; 1.4 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 24 op/s 2024-09-21T03:47:15.716 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 51116d5c-77ca-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 damage ls 2024-09-21T03:47:16.095 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/51116d5c-77ca-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:47:16.573 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:16.559+0000 7f4fb73c2640 1 -- 172.21.15.39:0/1624232342 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4fb01052b0 msgr2=0x7f4fb010d8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:47:16.573 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:16.559+0000 7f4fb73c2640 1 --2- 172.21.15.39:0/1624232342 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4fb01052b0 0x7f4fb010d8f0 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f4fa0009900 tx=0x7f4fa002a410 comp rx=0 tx=0).stop 2024-09-21T03:47:16.573 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:16.559+0000 7f4fb73c2640 1 -- 172.21.15.39:0/1624232342 shutdown_connections 2024-09-21T03:47:16.573 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:16.559+0000 7f4fb73c2640 1 --2- 172.21.15.39:0/1624232342 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4fb010de30 0x7f4fb0110340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:16.573 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:16.559+0000 7f4fb73c2640 1 --2- 172.21.15.39:0/1624232342 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4fb01052b0 0x7f4fb010d8f0 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:16.573 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:16.559+0000 7f4fb73c2640 1 --2- 172.21.15.39:0/1624232342 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4fb0104880 0x7f4fb0104c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:16.573 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:16.559+0000 7f4fb73c2640 1 -- 172.21.15.39:0/1624232342 >> 172.21.15.39:0/1624232342 conn(0x7f4fb0079750 msgr2=0x7f4fb0079b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:47:16.574 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:16.559+0000 7f4fb73c2640 1 -- 172.21.15.39:0/1624232342 shutdown_connections 2024-09-21T03:47:16.574 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:16.559+0000 7f4fb73c2640 1 -- 172.21.15.39:0/1624232342 wait complete. 2024-09-21T03:47:16.574 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:16.560+0000 7f4fb73c2640 1 Processor -- start 2024-09-21T03:47:16.574 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:16.560+0000 7f4fb73c2640 1 -- start start 2024-09-21T03:47:16.574 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:16.561+0000 7f4fb73c2640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4fb0104880 0x7f4fb019c730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:47:16.574 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:16.561+0000 7f4fb73c2640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4fb01052b0 0x7f4fb019ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:47:16.574 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:16.561+0000 7f4fb73c2640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4fb010de30 0x7f4fb01a1f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:47:16.574 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:16.561+0000 7f4fb73c2640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f4fb0113a70 con 0x7f4fb0104880 2024-09-21T03:47:16.574 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:16.561+0000 7f4fb73c2640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f4fb01138f0 con 0x7f4fb01052b0 2024-09-21T03:47:16.574 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:16.561+0000 7f4fb73c2640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f4fb0113bf0 con 0x7f4fb010de30 2024-09-21T03:47:16.574 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:16.561+0000 7f4fb515e640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4fb0104880 0x7f4fb019c730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:47:16.574 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:16.561+0000 7f4fb515e640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4fb0104880 0x7f4fb019c730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:51376/0 (socket says 172.21.15.39:51376) 2024-09-21T03:47:16.574 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:16.561+0000 7f4fb515e640 1 -- 172.21.15.39:0/2214940677 learned_addr learned my addr 172.21.15.39:0/2214940677 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:47:16.574 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:16.561+0000 7f4fb495d640 1 --2- 172.21.15.39:0/2214940677 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4fb01052b0 0x7f4fb019ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:47:16.575 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:16.561+0000 7f4fb595f640 1 --2- 172.21.15.39:0/2214940677 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4fb010de30 0x7f4fb01a1f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:47:16.575 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:16.561+0000 7f4fb515e640 1 -- 172.21.15.39:0/2214940677 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4fb010de30 msgr2=0x7f4fb01a1f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:47:16.575 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:16.561+0000 7f4fb515e640 1 --2- 172.21.15.39:0/2214940677 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4fb010de30 0x7f4fb01a1f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:16.575 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:16.561+0000 7f4fb515e640 1 -- 172.21.15.39:0/2214940677 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4fb01052b0 msgr2=0x7f4fb019ccd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:47:16.575 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:16.561+0000 7f4fb515e640 1 --2- 172.21.15.39:0/2214940677 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4fb01052b0 0x7f4fb019ccd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:16.575 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:16.561+0000 7f4fb515e640 1 -- 172.21.15.39:0/2214940677 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4fb01a26a0 con 0x7f4fb0104880 2024-09-21T03:47:16.575 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:16.561+0000 7f4fb595f640 1 --2- 172.21.15.39:0/2214940677 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4fb010de30 0x7f4fb01a1f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:47:16.575 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:16.562+0000 7f4fb515e640 1 --2- 172.21.15.39:0/2214940677 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4fb0104880 0x7f4fb019c730 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f4fac00e9f0 tx=0x7f4fac00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:47:16.575 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:16.562+0000 7f4fa67fc640 1 -- 172.21.15.39:0/2214940677 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fac00cd80 con 0x7f4fb0104880 2024-09-21T03:47:16.575 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:16.562+0000 7f4fa67fc640 1 -- 172.21.15.39:0/2214940677 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4fac0108e0 con 0x7f4fb0104880 2024-09-21T03:47:16.575 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:16.562+0000 7f4fb73c2640 1 -- 172.21.15.39:0/2214940677 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4fb01a2990 con 0x7f4fb0104880 2024-09-21T03:47:16.575 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:16.562+0000 7f4fb73c2640 1 -- 172.21.15.39:0/2214940677 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4fb01a90f0 con 0x7f4fb0104880 2024-09-21T03:47:16.575 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:16.562+0000 7f4fa67fc640 1 -- 172.21.15.39:0/2214940677 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fac018980 con 0x7f4fb0104880 2024-09-21T03:47:16.575 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:16.566+0000 7f4fbd936740 1 Processor -- start 2024-09-21T03:47:16.576 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:16.566+0000 7f4fbd936740 1 -- start start 2024-09-21T03:47:16.576 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:16.566+0000 7f4fa67fc640 1 -- 172.21.15.39:0/2214940677 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 26) ==== 154694+0+0 (secure 0 0 0) 0x7f4fac010430 con 0x7f4fb0104880 2024-09-21T03:47:16.576 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:16.567+0000 7f4fbd936740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55bbb15b7070 0x55bbb195e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:47:16.576 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:16.567+0000 7f4fbd936740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55bbb158e1b0 0x55bbb195e800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:47:16.576 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:16.567+0000 7f4fa67fc640 1 --2- 172.21.15.39:0/2214940677 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f4f9c0b6710 0x7f4f9c0b8c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:47:16.576 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:16.567+0000 7f4fb595f640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55bbb15b7070 0x55bbb195e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:47:16.576 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:16.567+0000 7f4fb595f640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55bbb15b7070 0x55bbb195e2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:51392/0 (socket says 172.21.15.39:51392) 2024-09-21T03:47:16.576 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:16.567+0000 7f4fa67fc640 1 -- 172.21.15.39:0/2214940677 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8484+0+0 (secure 0 0 0) 0x7f4fac0e7fb0 con 0x7f4fb0104880 2024-09-21T03:47:16.576 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:16.567+0000 7f4fb515e640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55bbb158e1b0 0x55bbb195e800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:47:16.576 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:16.568+0000 7f4fb495d640 1 --2- 172.21.15.39:0/2214940677 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f4f9c0b6710 0x7f4f9c0b8c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:47:16.576 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:16.568+0000 7f4fb515e640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55bbb158e1b0 0x55bbb195e800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.39:38016/0 (socket says 172.21.15.39:38016) 2024-09-21T03:47:16.576 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:16.568+0000 7f4fb515e640 1 -- 172.21.15.39:0/3078258465 learned_addr learned my addr 172.21.15.39:0/3078258465 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:47:16.576 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:16.568+0000 7f4fbd936740 1 --2- 172.21.15.39:0/3078258465 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55bbb1596390 0x55bbb195ed40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:47:16.576 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:16.568+0000 7f4fbd936740 1 -- 172.21.15.39:0/3078258465 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x55bbb18d4ea0 con 0x55bbb15b7070 2024-09-21T03:47:16.576 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:16.568+0000 7f4fbd936740 1 -- 172.21.15.39:0/3078258465 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x55bbb18c42d0 con 0x55bbb1596390 2024-09-21T03:47:16.577 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:16.568+0000 7f4fbd936740 1 -- 172.21.15.39:0/3078258465 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55bbb18c80e0 con 0x55bbb158e1b0 2024-09-21T03:47:16.577 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:16.568+0000 7f4fb515e640 1 -- 172.21.15.39:0/3078258465 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55bbb1596390 msgr2=0x55bbb195ed40 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-21T03:47:16.577 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:16.568+0000 7f4fb515e640 1 --2- 172.21.15.39:0/3078258465 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55bbb1596390 0x55bbb195ed40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:16.577 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:16.568+0000 7f4fb515e640 1 -- 172.21.15.39:0/3078258465 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55bbb15b7070 msgr2=0x55bbb195e2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:47:16.577 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:16.568+0000 7f4fb515e640 1 --2- 172.21.15.39:0/3078258465 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55bbb15b7070 0x55bbb195e2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:16.577 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:16.568+0000 7f4fb515e640 1 -- 172.21.15.39:0/3078258465 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bbb1622650 con 0x55bbb158e1b0 2024-09-21T03:47:16.577 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:16.569+0000 7f4fb495d640 1 --2- 172.21.15.39:0/2214940677 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f4f9c0b6710 0x7f4f9c0b8c20 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f4fa0009230 tx=0x7f4fa0031400 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:47:16.577 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:16.569+0000 7f4fb515e640 1 --2- 172.21.15.39:0/3078258465 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55bbb158e1b0 0x55bbb195e800 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f4fac0aa280 tx=0x7f4fac0aa430 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6c8699733f6febce server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:47:16.577 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:16.569+0000 7f4f9b7fe640 1 -- 172.21.15.39:0/3078258465 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fac0ad7c0 con 0x55bbb158e1b0 2024-09-21T03:47:16.577 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:16.569+0000 7f4f9b7fe640 1 -- 172.21.15.39:0/3078258465 <== mon.1 v2:172.21.15.161:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4fac0ae3b0 con 0x55bbb158e1b0 2024-09-21T03:47:16.577 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:16.569+0000 7f4f9b7fe640 1 -- 172.21.15.39:0/3078258465 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fac018b20 con 0x55bbb158e1b0 2024-09-21T03:47:16.577 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:16.570+0000 7f4fbd936740 1 -- 172.21.15.39:0/3078258465 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55bbb158e1b0 msgr2=0x55bbb195e800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:47:16.577 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:16.570+0000 7f4fbd936740 1 --2- 172.21.15.39:0/3078258465 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55bbb158e1b0 0x55bbb195e800 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f4fac0aa280 tx=0x7f4fac0aa430 comp rx=0 tx=0).stop 2024-09-21T03:47:16.577 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:16.570+0000 7f4fbd936740 1 -- 172.21.15.39:0/3078258465 shutdown_connections 2024-09-21T03:47:16.577 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:16.570+0000 7f4fbd936740 1 --2- 172.21.15.39:0/3078258465 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55bbb1596390 0x55bbb195ed40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:16.578 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:16.570+0000 7f4fbd936740 1 --2- 172.21.15.39:0/3078258465 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55bbb158e1b0 0x55bbb195e800 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:16.578 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:16.570+0000 7f4fbd936740 1 --2- 172.21.15.39:0/3078258465 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55bbb15b7070 0x55bbb195e2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:16.578 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:16.570+0000 7f4fbd936740 1 -- 172.21.15.39:0/3078258465 >> 172.21.15.39:0/3078258465 conn(0x55bbb15ab460 msgr2=0x55bbb195c1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:47:16.578 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:16.570+0000 7f4fbd936740 1 -- 172.21.15.39:0/3078258465 shutdown_connections 2024-09-21T03:47:16.578 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:16.570+0000 7f4fbd936740 1 -- 172.21.15.39:0/3078258465 wait complete. 2024-09-21T03:47:16.578 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:16.571+0000 7f4fbd936740 1 -- start start 2024-09-21T03:47:16.578 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:16.571+0000 7f4fbd936740 1 Processor -- start 2024-09-21T03:47:16.578 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:16.571+0000 7f4fbd936740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55bbb158e1b0 0x55bbb1969960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:47:16.581 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:16.571+0000 7f4fbd936740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55bbb1596390 0x55bbb1969ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:47:16.581 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:16.571+0000 7f4fbd936740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55bbb15b7070 0x55bbb1975d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:47:16.581 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:16.571+0000 7f4fbd936740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x55bbb1725700 con 0x55bbb15b7070 2024-09-21T03:47:16.581 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:16.571+0000 7f4fbd936740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x55bbb185f160 con 0x55bbb1596390 2024-09-21T03:47:16.582 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:16.571+0000 7f4fbd936740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55bbb196ca40 con 0x55bbb158e1b0 2024-09-21T03:47:16.582 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:16.571+0000 7f4fbd936740 10 client.0 _ll_register_callbacks cb 0x55bbb18789e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:47:16.582 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:16.572+0000 7f4fb595f640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55bbb15b7070 0x55bbb1975d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:47:16.582 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:16.572+0000 7f4fb595f640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55bbb15b7070 0x55bbb1975d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:51406/0 (socket says 172.21.15.39:51406) 2024-09-21T03:47:16.582 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:16.572+0000 7f4fb595f640 1 -- 172.21.15.39:0/4105631356 learned_addr learned my addr 172.21.15.39:0/4105631356 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:47:16.582 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:16.572+0000 7f4fb495d640 1 --2- 172.21.15.39:0/4105631356 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55bbb1596390 0x55bbb1969ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:47:16.582 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:16.572+0000 7f4fb595f640 1 -- 172.21.15.39:0/4105631356 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55bbb158e1b0 msgr2=0x55bbb1969960 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-21T03:47:16.582 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:16.572+0000 7f4fb595f640 1 --2- 172.21.15.39:0/4105631356 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55bbb158e1b0 0x55bbb1969960 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:16.582 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:16.572+0000 7f4fb595f640 1 -- 172.21.15.39:0/4105631356 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55bbb1596390 msgr2=0x55bbb1969ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:47:16.582 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:16.572+0000 7f4fb595f640 1 --2- 172.21.15.39:0/4105631356 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55bbb1596390 0x55bbb1969ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:16.582 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:16.572+0000 7f4fb595f640 1 -- 172.21.15.39:0/4105631356 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bbb188cc90 con 0x55bbb15b7070 2024-09-21T03:47:16.582 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:16.572+0000 7f4fb595f640 1 --2- 172.21.15.39:0/4105631356 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55bbb15b7070 0x55bbb1975d30 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f4fa80106b0 tx=0x7f4fa8010b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe99204c73ce52e7 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:47:16.582 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:16.572+0000 7f4f98ff9640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-09-21T03:47:16.583 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:16.573+0000 7f4f98ff9640 1 -- 172.21.15.39:0/4105631356 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fa8004d20 con 0x55bbb15b7070 2024-09-21T03:47:16.583 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:16.573+0000 7f4fbd936740 1 -- 172.21.15.39:0/4105631356 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4fac0ae3b0 con 0x55bbb15b7070 2024-09-21T03:47:16.583 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:16.573+0000 7f4f98ff9640 1 -- 172.21.15.39:0/4105631356 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8484+0+0 (secure 0 0 0) 0x7f4fa800e930 con 0x55bbb15b7070 2024-09-21T03:47:16.583 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:16.573+0000 7f4f98ff9640 1 -- 172.21.15.39:0/4105631356 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f4fa800eb30 con 0x55bbb15b7070 2024-09-21T03:47:16.583 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:16.573+0000 7f4fbd936740 10 client.15843 fetch_fsmap learned FSMap version 29 2024-09-21T03:47:16.583 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:16.573+0000 7f4fbd936740 1 -- 172.21.15.39:0/4105631356 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55bbb1980ff0 con 0x55bbb15b7070 2024-09-21T03:47:16.583 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:16.574+0000 7f4f98ff9640 1 -- 172.21.15.39:0/4105631356 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4fac0ae3b0 con 0x55bbb15b7070 2024-09-21T03:47:16.583 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:16.574+0000 7f4f98ff9640 1 -- 172.21.15.39:0/4105631356 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f4fa8011440 con 0x55bbb15b7070 2024-09-21T03:47:16.583 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:16.574+0000 7f4fbd936740 10 client.15843 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T03:47:16.583 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:16.574+0000 7f4fbd936740 10 client.15843 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T03:47:16.583 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:16.574+0000 7f4fbd936740 20 client.15843 populate_metadata read hostname 'smithi039' 2024-09-21T03:47:16.583 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:16.574+0000 7f4fbd936740 1 --2- 172.21.15.39:0/4105631356 >> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] conn(0x55bbb15759f0 0x55bbb19848c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:47:16.583 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:16.574+0000 7f4fbd936740 4 client.15843 mds_command: new command op to 24695 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:47:16.583 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:16.574+0000 7f4fbd936740 1 -- 172.21.15.39:0/4105631356 --> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bbb18d6df0 con 0x55bbb15759f0 2024-09-21T03:47:16.583 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:16.575+0000 7f4fb515e640 1 --2- 172.21.15.39:0/4105631356 >> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] conn(0x55bbb15759f0 0x55bbb19848c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:47:16.584 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:16.575+0000 7f4f98ff9640 10 client.15843 ms_handle_connect on v2:172.21.15.150:6838/2949386907 2024-09-21T03:47:16.584 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:16.575+0000 7f4fb515e640 1 --2- 172.21.15.39:0/4105631356 >> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] conn(0x55bbb15759f0 0x55bbb19848c0 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=423234ac53788837 server_cookie=f71a44ccded099dc in_seq=0 out_seq=0 2024-09-21T03:47:16.584 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:16.576+0000 7f4f98ff9640 1 -- 172.21.15.39:0/4105631356 <== mds.0 v2:172.21.15.150:6838/2949386907 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55bbb18d6df0 con 0x55bbb15759f0 2024-09-21T03:47:16.584 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:16.576+0000 7f4f98ff9640 10 client.15843 handle_command_reply: tid=0 2024-09-21T03:47:16.584 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:16.576+0000 7f4fbd936740 1 client.15843 shutdown 2024-09-21T03:47:16.584 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:16.576+0000 7f4fbd936740 1 -- 172.21.15.39:0/4105631356 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55bbb15b7070 msgr2=0x55bbb1975d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:47:16.584 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:16.576+0000 7f4fbd936740 1 --2- 172.21.15.39:0/4105631356 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55bbb15b7070 0x55bbb1975d30 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f4fa80106b0 tx=0x7f4fa8010b70 comp rx=0 tx=0).stop 2024-09-21T03:47:16.584 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:16.577+0000 7f4fbd936740 1 -- 172.21.15.39:0/4105631356 shutdown_connections 2024-09-21T03:47:16.584 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:16.577+0000 7f4fbd936740 1 --2- 172.21.15.39:0/4105631356 >> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] conn(0x55bbb15759f0 0x55bbb19848c0 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:16.584 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:16.577+0000 7f4fbd936740 1 --2- 172.21.15.39:0/4105631356 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55bbb15b7070 0x55bbb1975d30 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:16.584 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:16.577+0000 7f4fbd936740 1 --2- 172.21.15.39:0/4105631356 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55bbb1596390 0x55bbb1969ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:16.584 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:16.577+0000 7f4f98ff9640 0 client.15843 ms_handle_reset on v2:172.21.15.150:6838/2949386907 2024-09-21T03:47:16.585 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:16.577+0000 7f4fbd936740 1 --2- 172.21.15.39:0/4105631356 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55bbb158e1b0 0x55bbb1969960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:16.585 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:16.577+0000 7f4fbd936740 1 -- 172.21.15.39:0/4105631356 >> 172.21.15.39:0/4105631356 conn(0x55bbb15ab460 msgr2=0x55bbb19614a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:47:16.585 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:16.577+0000 7f4fbd936740 1 -- 172.21.15.39:0/4105631356 shutdown_connections 2024-09-21T03:47:16.585 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:16.577+0000 7f4fbd936740 1 -- 172.21.15.39:0/4105631356 wait complete. 2024-09-21T03:47:16.585 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:16.577+0000 7f4fbd936740 20 client.15843 trim_cache size 0 max 16384 2024-09-21T03:47:16.592 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:16.591+0000 7f4fbd936740 1 Processor -- start 2024-09-21T03:47:16.592 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:16.591+0000 7f4fbd936740 1 -- start start 2024-09-21T03:47:16.593 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:16.591+0000 7f4fbd936740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55bbb158e1b0 0x55bbb196a0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:47:16.593 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:16.591+0000 7f4fbd936740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55bbb1596390 0x55bbb195fa00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:47:16.593 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:16.591+0000 7f4fbd936740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55bbb15b7070 0x55bbb195b760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:47:16.594 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:16.591+0000 7f4fbd936740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f4fb01ad630 con 0x55bbb15b7070 2024-09-21T03:47:16.594 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:16.591+0000 7f4fbd936740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f4fb01af170 con 0x55bbb158e1b0 2024-09-21T03:47:16.594 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:16.591+0000 7f4fbd936740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f4fb01add00 con 0x55bbb1596390 2024-09-21T03:47:16.594 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:16.592+0000 7f4fb595f640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55bbb15b7070 0x55bbb195b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:47:16.594 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:16.592+0000 7f4fb595f640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55bbb15b7070 0x55bbb195b760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:51418/0 (socket says 172.21.15.39:51418) 2024-09-21T03:47:16.594 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:16.592+0000 7f4fb595f640 1 -- 172.21.15.39:0/3263248703 learned_addr learned my addr 172.21.15.39:0/3263248703 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:47:16.594 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:16.592+0000 7f4fb495d640 1 --2- 172.21.15.39:0/3263248703 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55bbb1596390 0x55bbb195fa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:47:16.595 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:16.592+0000 7f4fb515e640 1 --2- 172.21.15.39:0/3263248703 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55bbb158e1b0 0x55bbb196a0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:47:16.595 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:16.592+0000 7f4fb595f640 1 -- 172.21.15.39:0/3263248703 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55bbb1596390 msgr2=0x55bbb195fa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:47:16.595 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:16.592+0000 7f4fb595f640 1 --2- 172.21.15.39:0/3263248703 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55bbb1596390 0x55bbb195fa00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:16.595 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:16.592+0000 7f4fb595f640 1 -- 172.21.15.39:0/3263248703 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55bbb158e1b0 msgr2=0x55bbb196a0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:47:16.595 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:16.592+0000 7f4fb595f640 1 --2- 172.21.15.39:0/3263248703 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55bbb158e1b0 0x55bbb196a0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:16.595 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:16.592+0000 7f4fb595f640 1 -- 172.21.15.39:0/3263248703 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bbb19605b0 con 0x55bbb15b7070 2024-09-21T03:47:16.595 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:16.592+0000 7f4fb595f640 1 --2- 172.21.15.39:0/3263248703 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55bbb15b7070 0x55bbb195b760 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f4fa8009cb0 tx=0x7f4fa80059d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a04166e34a909830 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:47:16.595 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:16.593+0000 7f4f9affd640 1 -- 172.21.15.39:0/3263248703 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fa800e720 con 0x55bbb15b7070 2024-09-21T03:47:16.595 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:16.593+0000 7f4f9affd640 1 -- 172.21.15.39:0/3263248703 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4fa80098d0 con 0x55bbb15b7070 2024-09-21T03:47:16.595 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:16.593+0000 7f4f9affd640 1 -- 172.21.15.39:0/3263248703 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fa801b8e0 con 0x55bbb15b7070 2024-09-21T03:47:16.596 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:16.594+0000 7f4fbd936740 1 -- 172.21.15.39:0/3263248703 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55bbb15b7070 msgr2=0x55bbb195b760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:47:16.596 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:16.594+0000 7f4fbd936740 1 --2- 172.21.15.39:0/3263248703 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55bbb15b7070 0x55bbb195b760 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f4fa8009cb0 tx=0x7f4fa80059d0 comp rx=0 tx=0).stop 2024-09-21T03:47:16.596 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:16.594+0000 7f4fbd936740 1 -- 172.21.15.39:0/3263248703 shutdown_connections 2024-09-21T03:47:16.596 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:16.594+0000 7f4fbd936740 1 --2- 172.21.15.39:0/3263248703 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55bbb15b7070 0x55bbb195b760 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:16.596 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:16.594+0000 7f4fbd936740 1 --2- 172.21.15.39:0/3263248703 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55bbb1596390 0x55bbb195fa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:16.596 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:16.594+0000 7f4fbd936740 1 --2- 172.21.15.39:0/3263248703 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55bbb158e1b0 0x55bbb196a0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:16.597 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:16.594+0000 7f4fbd936740 1 -- 172.21.15.39:0/3263248703 >> 172.21.15.39:0/3263248703 conn(0x55bbb15ab460 msgr2=0x55bbb195f2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:47:16.597 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:16.594+0000 7f4fbd936740 1 -- 172.21.15.39:0/3263248703 shutdown_connections 2024-09-21T03:47:16.597 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:16.594+0000 7f4fbd936740 1 -- 172.21.15.39:0/3263248703 wait complete. 2024-09-21T03:47:16.597 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:16.595+0000 7f4fbd936740 1 -- start start 2024-09-21T03:47:16.597 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:16.595+0000 7f4fbd936740 1 Processor -- start 2024-09-21T03:47:16.598 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:16.595+0000 7f4fbd936740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55bbb158e1b0 0x55bbb1972330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:47:16.598 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:16.595+0000 7f4fbd936740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55bbb1596390 0x55bbb196d7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:47:16.598 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:16.595+0000 7f4fbd936740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55bbb15b7070 0x55bbb196dd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:47:16.598 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:16.596+0000 7f4fbd936740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f4fb01aeaa0 con 0x55bbb158e1b0 2024-09-21T03:47:16.598 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:16.596+0000 7f4fbd936740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f4fb01ae3d0 con 0x55bbb1596390 2024-09-21T03:47:16.598 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:16.596+0000 7f4fbd936740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f4fb01ab670 con 0x55bbb15b7070 2024-09-21T03:47:16.599 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:16.596+0000 7f4fbd936740 10 client.0 _ll_register_callbacks cb 0x7f4fa80097b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:47:16.599 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:16.596+0000 7f4fb515e640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55bbb158e1b0 0x55bbb1972330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:47:16.599 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:16.596+0000 7f4fb495d640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55bbb1596390 0x55bbb196d7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:47:16.599 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:16.596+0000 7f4fb495d640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55bbb1596390 0x55bbb196d7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.39:57112/0 (socket says 172.21.15.39:57112) 2024-09-21T03:47:16.599 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:16.596+0000 7f4fb495d640 1 -- 172.21.15.39:0/3491195684 learned_addr learned my addr 172.21.15.39:0/3491195684 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:47:16.599 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:16.596+0000 7f4fb595f640 1 --2- 172.21.15.39:0/3491195684 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55bbb15b7070 0x55bbb196dd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:47:16.599 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:16.596+0000 7f4fb495d640 1 -- 172.21.15.39:0/3491195684 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55bbb15b7070 msgr2=0x55bbb196dd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:47:16.599 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:16.596+0000 7f4fb495d640 1 --2- 172.21.15.39:0/3491195684 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55bbb15b7070 0x55bbb196dd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:16.599 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:16.596+0000 7f4fb495d640 1 -- 172.21.15.39:0/3491195684 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55bbb158e1b0 msgr2=0x55bbb1972330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:47:16.599 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:16.596+0000 7f4fb495d640 1 --2- 172.21.15.39:0/3491195684 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55bbb158e1b0 0x55bbb1972330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:16.599 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:16.597+0000 7f4fb495d640 1 -- 172.21.15.39:0/3491195684 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bbb196e360 con 0x55bbb1596390 2024-09-21T03:47:16.599 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:16.597+0000 7f4fb515e640 1 --2- 172.21.15.39:0/3491195684 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55bbb158e1b0 0x55bbb1972330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:47:16.601 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:16.597+0000 7f4fb495d640 1 --2- 172.21.15.39:0/3491195684 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55bbb1596390 0x55bbb196d7e0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f4fa0009900 tx=0x7f4fa0009260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=59668277cc6680fd server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:47:16.601 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:16.597+0000 7f4f98ff9640 10 client.0 ms_handle_connect on v2:172.21.15.150:3300/0 2024-09-21T03:47:16.601 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:16.597+0000 7f4f98ff9640 1 -- 172.21.15.39:0/3491195684 <== mon.2 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fa0035040 con 0x55bbb1596390 2024-09-21T03:47:16.601 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:16.597+0000 7f4f98ff9640 1 -- 172.21.15.39:0/3491195684 <== mon.2 v2:172.21.15.150:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8484+0+0 (secure 0 0 0) 0x7f4fa00043d0 con 0x55bbb1596390 2024-09-21T03:47:16.601 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:16.597+0000 7f4fbd936740 1 -- 172.21.15.39:0/3491195684 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4fa80098d0 con 0x55bbb1596390 2024-09-21T03:47:16.601 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:16.598+0000 7f4f98ff9640 1 -- 172.21.15.39:0/3491195684 <== mon.2 v2:172.21.15.150:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f4fa0005040 con 0x55bbb1596390 2024-09-21T03:47:16.601 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:16.598+0000 7f4fbd936740 10 client.25433 fetch_fsmap learned FSMap version 29 2024-09-21T03:47:16.601 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:16.598+0000 7f4fbd936740 1 -- 172.21.15.39:0/3491195684 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55bbb19ef070 con 0x55bbb1596390 2024-09-21T03:47:16.601 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:16.598+0000 7f4f98ff9640 1 -- 172.21.15.39:0/3491195684 <== mon.2 v2:172.21.15.150:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4fa0004750 con 0x55bbb1596390 2024-09-21T03:47:16.602 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:16.598+0000 7f4f98ff9640 1 -- 172.21.15.39:0/3491195684 <== mon.2 v2:172.21.15.150:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f4fa0051020 con 0x55bbb1596390 2024-09-21T03:47:16.602 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:16.598+0000 7f4fbd936740 10 client.25433 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T03:47:16.602 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:16.598+0000 7f4fbd936740 10 client.25433 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T03:47:16.602 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:16.598+0000 7f4fbd936740 20 client.25433 populate_metadata read hostname 'smithi039' 2024-09-21T03:47:16.602 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:16.598+0000 7f4fbd936740 1 --2- 172.21.15.39:0/3491195684 >> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] conn(0x55bbb154ada0 0x55bbb19f1890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:47:16.602 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:16.598+0000 7f4fbd936740 4 client.25433 mds_command: new command op to 24695 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T03:47:16.602 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:16.598+0000 7f4fbd936740 1 -- 172.21.15.39:0/3491195684 --> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7f4fa80098d0 con 0x55bbb154ada0 2024-09-21T03:47:16.602 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:16.599+0000 7f4fb515e640 1 --2- 172.21.15.39:0/3491195684 >> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] conn(0x55bbb154ada0 0x55bbb19f1890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:47:16.602 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:16.599+0000 7f4f98ff9640 10 client.25433 ms_handle_connect on v2:172.21.15.150:6838/2949386907 2024-09-21T03:47:16.602 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:16.599+0000 7f4fb515e640 1 --2- 172.21.15.39:0/3491195684 >> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] conn(0x55bbb154ada0 0x55bbb19f1890 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=caa7daeef990f673 server_cookie=b8b18d44be9fef96 in_seq=0 out_seq=0 2024-09-21T03:47:16.603 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:16.600+0000 7f4f98ff9640 1 -- 172.21.15.39:0/3491195684 <== mds.0 v2:172.21.15.150:6838/2949386907 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7f4fa80098d0 con 0x55bbb154ada0 2024-09-21T03:47:16.603 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:16.600+0000 7f4f98ff9640 10 client.25433 handle_command_reply: tid=0 2024-09-21T03:47:16.603 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:16.600+0000 7f4fbd936740 1 client.25433 shutdown 2024-09-21T03:47:16.603 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:16.600+0000 7f4fbd936740 1 -- 172.21.15.39:0/3491195684 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55bbb1596390 msgr2=0x55bbb196d7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:47:16.603 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:16.600+0000 7f4fbd936740 1 --2- 172.21.15.39:0/3491195684 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55bbb1596390 0x55bbb196d7e0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f4fa0009900 tx=0x7f4fa0009260 comp rx=0 tx=0).stop 2024-09-21T03:47:16.604 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:16.600+0000 7f4fbd936740 1 -- 172.21.15.39:0/3491195684 shutdown_connections 2024-09-21T03:47:16.604 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:16.600+0000 7f4fbd936740 1 --2- 172.21.15.39:0/3491195684 >> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] conn(0x55bbb154ada0 0x55bbb19f1890 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:16.604 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:16.600+0000 7f4fbd936740 1 --2- 172.21.15.39:0/3491195684 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55bbb15b7070 0x55bbb196dd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:16.604 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:16.600+0000 7f4fbd936740 1 --2- 172.21.15.39:0/3491195684 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55bbb1596390 0x55bbb196d7e0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:16.604 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:16.600+0000 7f4fbd936740 1 --2- 172.21.15.39:0/3491195684 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55bbb158e1b0 0x55bbb1972330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:16.604 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:16.600+0000 7f4fbd936740 1 -- 172.21.15.39:0/3491195684 >> 172.21.15.39:0/3491195684 conn(0x55bbb15ab460 msgr2=0x55bbb195fcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:47:16.604 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:16.600+0000 7f4f98ff9640 0 client.25433 ms_handle_reset on v2:172.21.15.150:6838/2949386907 2024-09-21T03:47:16.604 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:16.600+0000 7f4fbd936740 1 -- 172.21.15.39:0/3491195684 shutdown_connections 2024-09-21T03:47:16.604 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:16.600+0000 7f4fbd936740 1 -- 172.21.15.39:0/3491195684 wait complete. 2024-09-21T03:47:16.604 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:16.600+0000 7f4fbd936740 20 client.25433 trim_cache size 0 max 16384 2024-09-21T03:47:16.605 INFO:teuthology.orchestra.run.smithi039.stdout:[] 2024-09-21T03:47:16.605 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:16.601+0000 7f4fb73c2640 1 -- 172.21.15.39:0/2214940677 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f4f9c0b6710 msgr2=0x7f4f9c0b8c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:47:16.605 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:16.601+0000 7f4fb73c2640 1 --2- 172.21.15.39:0/2214940677 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f4f9c0b6710 0x7f4f9c0b8c20 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f4fa0009230 tx=0x7f4fa0031400 comp rx=0 tx=0).stop 2024-09-21T03:47:16.606 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:16.601+0000 7f4fb73c2640 1 -- 172.21.15.39:0/2214940677 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4fb0104880 msgr2=0x7f4fb019c730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:47:16.606 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:16.601+0000 7f4fb73c2640 1 --2- 172.21.15.39:0/2214940677 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4fb0104880 0x7f4fb019c730 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f4fac00e9f0 tx=0x7f4fac00eeb0 comp rx=0 tx=0).stop 2024-09-21T03:47:16.606 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:16.601+0000 7f4fb73c2640 1 -- 172.21.15.39:0/2214940677 shutdown_connections 2024-09-21T03:47:16.606 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:16.601+0000 7f4fb73c2640 1 --2- 172.21.15.39:0/2214940677 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f4f9c0b6710 0x7f4f9c0b8c20 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:16.606 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:16.601+0000 7f4fb73c2640 1 --2- 172.21.15.39:0/2214940677 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4fb010de30 0x7f4fb01a1f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:16.606 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:16.601+0000 7f4fb73c2640 1 --2- 172.21.15.39:0/2214940677 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4fb01052b0 0x7f4fb019ccd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:16.606 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:16.601+0000 7f4fb73c2640 1 --2- 172.21.15.39:0/2214940677 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4fb0104880 0x7f4fb019c730 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:16.606 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:16.601+0000 7f4fb73c2640 1 -- 172.21.15.39:0/2214940677 >> 172.21.15.39:0/2214940677 conn(0x7f4fb0079750 msgr2=0x7f4fb0100710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:47:16.606 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:16.601+0000 7f4fb73c2640 1 -- 172.21.15.39:0/2214940677 shutdown_connections 2024-09-21T03:47:16.606 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:16.601+0000 7f4fb73c2640 1 -- 172.21.15.39:0/2214940677 wait complete. 2024-09-21T03:47:16.735 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:47:16 smithi039 ceph-mon[31211]: pgmap v362: 97 pgs: 97 active+clean; 1.4 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 23 KiB/s wr, 27 op/s 2024-09-21T03:47:17.218 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:47:16 smithi161 ceph-mon[34533]: pgmap v362: 97 pgs: 97 active+clean; 1.4 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 23 KiB/s wr, 27 op/s 2024-09-21T03:47:17.225 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:47:16 smithi150 ceph-mon[34289]: pgmap v362: 97 pgs: 97 active+clean; 1.4 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 23 KiB/s wr, 27 op/s 2024-09-21T03:47:18.484 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-21T03:47:18.484 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 51116d5c-77ca-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-21T03:47:18.752 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/51116d5c-77ca-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:47:18.833 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:47:18 smithi039 ceph-mon[31211]: pgmap v363: 97 pgs: 97 active+clean; 1.4 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 255 B/s wr, 9 op/s 2024-09-21T03:47:19.218 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:47:18 smithi161 ceph-mon[34533]: pgmap v363: 97 pgs: 97 active+clean; 1.4 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 255 B/s wr, 9 op/s 2024-09-21T03:47:19.224 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:47:18 smithi150 ceph-mon[34289]: pgmap v363: 97 pgs: 97 active+clean; 1.4 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 255 B/s wr, 9 op/s 2024-09-21T03:47:19.341 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:19.339+0000 7f693272e640 1 -- 172.21.15.39:0/3162966174 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f692c10ddf0 msgr2=0x7f692c110300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:47:19.342 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:19.339+0000 7f693272e640 1 --2- 172.21.15.39:0/3162966174 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f692c10ddf0 0x7f692c110300 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f6924009ca0 tx=0x7f692402a8c0 comp rx=0 tx=0).stop 2024-09-21T03:47:19.342 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:19.339+0000 7f693272e640 1 -- 172.21.15.39:0/3162966174 shutdown_connections 2024-09-21T03:47:19.342 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:19.339+0000 7f693272e640 1 --2- 172.21.15.39:0/3162966174 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f692c10ddf0 0x7f692c110300 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:19.342 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:19.339+0000 7f693272e640 1 --2- 172.21.15.39:0/3162966174 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f692c105270 0x7f692c10d8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:19.342 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:19.339+0000 7f693272e640 1 --2- 172.21.15.39:0/3162966174 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f692c104840 0x7f692c104c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:19.342 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:19.339+0000 7f693272e640 1 -- 172.21.15.39:0/3162966174 >> 172.21.15.39:0/3162966174 conn(0x7f692c079730 msgr2=0x7f692c079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:47:19.342 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:19.339+0000 7f693272e640 1 -- 172.21.15.39:0/3162966174 shutdown_connections 2024-09-21T03:47:19.342 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:19.339+0000 7f693272e640 1 -- 172.21.15.39:0/3162966174 wait complete. 2024-09-21T03:47:19.343 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:19.340+0000 7f693272e640 1 Processor -- start 2024-09-21T03:47:19.343 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:19.340+0000 7f693272e640 1 -- start start 2024-09-21T03:47:19.344 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:19.341+0000 7f693272e640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f692c104840 0x7f692c1a1770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:47:19.344 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:19.341+0000 7f693272e640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f692c105270 0x7f692c1a1cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:47:19.344 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:19.341+0000 7f693272e640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f692c10ddf0 0x7f692c1ac7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:47:19.345 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:19.341+0000 7f693272e640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f692c113a90 con 0x7f692c105270 2024-09-21T03:47:19.345 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:19.341+0000 7f693272e640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f692c113910 con 0x7f692c104840 2024-09-21T03:47:19.345 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:19.341+0000 7f693272e640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f692c113c10 con 0x7f692c10ddf0 2024-09-21T03:47:19.345 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:19.341+0000 7f692b7fe640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f692c105270 0x7f692c1a1cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:47:19.345 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:19.341+0000 7f692b7fe640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f692c105270 0x7f692c1a1cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:38594/0 (socket says 172.21.15.39:38594) 2024-09-21T03:47:19.345 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:19.341+0000 7f692b7fe640 1 -- 172.21.15.39:0/3716103549 learned_addr learned my addr 172.21.15.39:0/3716103549 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:47:19.345 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:19.341+0000 7f692bfff640 1 --2- 172.21.15.39:0/3716103549 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f692c104840 0x7f692c1a1770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:47:19.345 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:19.341+0000 7f6930ccb640 1 --2- 172.21.15.39:0/3716103549 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f692c10ddf0 0x7f692c1ac7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:47:19.345 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:19.342+0000 7f692b7fe640 1 -- 172.21.15.39:0/3716103549 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f692c10ddf0 msgr2=0x7f692c1ac7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:47:19.345 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:19.342+0000 7f692b7fe640 1 --2- 172.21.15.39:0/3716103549 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f692c10ddf0 0x7f692c1ac7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:19.345 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:19.342+0000 7f692b7fe640 1 -- 172.21.15.39:0/3716103549 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f692c104840 msgr2=0x7f692c1a1770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:47:19.346 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:19.342+0000 7f692b7fe640 1 --2- 172.21.15.39:0/3716103549 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f692c104840 0x7f692c1a1770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:19.346 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:19.342+0000 7f692b7fe640 1 -- 172.21.15.39:0/3716103549 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f692c1a2550 con 0x7f692c105270 2024-09-21T03:47:19.346 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:19.342+0000 7f6930ccb640 1 --2- 172.21.15.39:0/3716103549 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f692c10ddf0 0x7f692c1ac7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:47:19.346 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:19.342+0000 7f692bfff640 1 --2- 172.21.15.39:0/3716103549 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f692c104840 0x7f692c1a1770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:47:19.346 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:19.342+0000 7f692b7fe640 1 --2- 172.21.15.39:0/3716103549 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f692c105270 0x7f692c1a1cb0 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f692000d950 tx=0x7f692000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:47:19.346 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:19.342+0000 7f69297fa640 1 -- 172.21.15.39:0/3716103549 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6920017070 con 0x7f692c105270 2024-09-21T03:47:19.346 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:19.342+0000 7f693272e640 1 -- 172.21.15.39:0/3716103549 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f692c1acdf0 con 0x7f692c105270 2024-09-21T03:47:19.346 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:19.342+0000 7f69297fa640 1 -- 172.21.15.39:0/3716103549 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6920009c30 con 0x7f692c105270 2024-09-21T03:47:19.346 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:19.342+0000 7f69297fa640 1 -- 172.21.15.39:0/3716103549 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69200047a0 con 0x7f692c105270 2024-09-21T03:47:19.346 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:19.343+0000 7f693272e640 1 -- 172.21.15.39:0/3716103549 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f692c1ad2a0 con 0x7f692c105270 2024-09-21T03:47:19.348 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:19.346+0000 7f6938ca2740 1 Processor -- start 2024-09-21T03:47:19.348 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:19.347+0000 7f6938ca2740 1 -- start start 2024-09-21T03:47:19.349 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:19.347+0000 7f69297fa640 1 -- 172.21.15.39:0/3716103549 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 26) ==== 154694+0+0 (secure 0 0 0) 0x7f6920004020 con 0x7f692c105270 2024-09-21T03:47:19.349 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:19.347+0000 7f6938ca2740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a27255f350 0x55a272906430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:47:19.349 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:19.347+0000 7f6938ca2740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a272536490 0x55a272906970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:47:19.349 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:19.347+0000 7f6938ca2740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a27253e670 0x55a272906eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:47:19.349 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:19.347+0000 7f6938ca2740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x55a27287ce60 con 0x55a272536490 2024-09-21T03:47:19.349 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:19.347+0000 7f6938ca2740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x55a27286c290 con 0x55a27255f350 2024-09-21T03:47:19.349 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:19.347+0000 7f6938ca2740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55a2728700a0 con 0x55a27253e670 2024-09-21T03:47:19.349 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:19.347+0000 7f692bfff640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a272536490 0x55a272906970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:47:19.350 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:19.347+0000 7f692bfff640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a272536490 0x55a272906970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:38596/0 (socket says 172.21.15.39:38596) 2024-09-21T03:47:19.350 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:19.347+0000 7f692bfff640 1 -- 172.21.15.39:0/968089180 learned_addr learned my addr 172.21.15.39:0/968089180 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:47:19.350 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:19.347+0000 7f6930ccb640 1 --2- 172.21.15.39:0/968089180 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a27255f350 0x55a272906430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:47:19.350 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:19.347+0000 7f692b7fe640 1 --2- 172.21.15.39:0/968089180 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a27253e670 0x55a272906eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:47:19.350 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:19.348+0000 7f692bfff640 1 -- 172.21.15.39:0/968089180 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a27253e670 msgr2=0x55a272906eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:47:19.351 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:19.348+0000 7f692bfff640 1 --2- 172.21.15.39:0/968089180 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a27253e670 0x55a272906eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:19.351 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:19.348+0000 7f692bfff640 1 -- 172.21.15.39:0/968089180 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a27255f350 msgr2=0x55a272906430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:47:19.351 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:19.348+0000 7f692bfff640 1 --2- 172.21.15.39:0/968089180 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a27255f350 0x55a272906430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:19.351 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:19.348+0000 7f692bfff640 1 -- 172.21.15.39:0/968089180 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a2725ca930 con 0x55a272536490 2024-09-21T03:47:19.351 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:19.348+0000 7f692bfff640 1 --2- 172.21.15.39:0/968089180 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a272536490 0x55a272906970 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f691c0114f0 tx=0x7f691c0119b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6801ed288cebfa23 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:47:19.351 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:19.349+0000 7f69127fc640 1 -- 172.21.15.39:0/968089180 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f691c01e070 con 0x55a272536490 2024-09-21T03:47:19.352 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:19.349+0000 7f69127fc640 1 -- 172.21.15.39:0/968089180 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f691c00fad0 con 0x55a272536490 2024-09-21T03:47:19.352 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:19.349+0000 7f69297fa640 1 --2- 172.21.15.39:0/3716103549 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f69140b6810 0x7f69140b8d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:47:19.352 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:19.349+0000 7f69127fc640 1 -- 172.21.15.39:0/968089180 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f691c019aa0 con 0x55a272536490 2024-09-21T03:47:19.352 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:19.349+0000 7f69297fa640 1 -- 172.21.15.39:0/3716103549 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8484+0+0 (secure 0 0 0) 0x7f69200e7450 con 0x7f692c105270 2024-09-21T03:47:19.352 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:19.350+0000 7f6930ccb640 1 --2- 172.21.15.39:0/3716103549 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f69140b6810 0x7f69140b8d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:47:19.352 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:19.350+0000 7f6938ca2740 1 -- 172.21.15.39:0/968089180 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a272536490 msgr2=0x55a272906970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:47:19.352 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:19.350+0000 7f6938ca2740 1 --2- 172.21.15.39:0/968089180 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a272536490 0x55a272906970 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f691c0114f0 tx=0x7f691c0119b0 comp rx=0 tx=0).stop 2024-09-21T03:47:19.352 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:19.350+0000 7f6930ccb640 1 --2- 172.21.15.39:0/3716103549 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f69140b6810 0x7f69140b8d20 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f6924006010 tx=0x7f6924032ed0 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:47:19.352 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:19.351+0000 7f6938ca2740 1 -- 172.21.15.39:0/968089180 shutdown_connections 2024-09-21T03:47:19.352 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:19.351+0000 7f6938ca2740 1 --2- 172.21.15.39:0/968089180 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a27253e670 0x55a272906eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:19.353 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:19.351+0000 7f6938ca2740 1 --2- 172.21.15.39:0/968089180 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a272536490 0x55a272906970 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:19.353 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:19.351+0000 7f6938ca2740 1 --2- 172.21.15.39:0/968089180 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a27255f350 0x55a272906430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:19.353 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:19.351+0000 7f6938ca2740 1 -- 172.21.15.39:0/968089180 >> 172.21.15.39:0/968089180 conn(0x55a272553740 msgr2=0x55a272904320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:47:19.353 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:19.351+0000 7f6938ca2740 1 -- 172.21.15.39:0/968089180 shutdown_connections 2024-09-21T03:47:19.353 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:19.351+0000 7f6938ca2740 1 -- 172.21.15.39:0/968089180 wait complete. 2024-09-21T03:47:19.354 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:19.351+0000 7f6938ca2740 1 -- start start 2024-09-21T03:47:19.354 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:19.351+0000 7f6938ca2740 1 Processor -- start 2024-09-21T03:47:19.354 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:19.352+0000 7f6938ca2740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a272536490 0x55a272911cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:47:19.354 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:19.352+0000 7f6938ca2740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a27253e670 0x55a272912260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:47:19.354 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:19.352+0000 7f6938ca2740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a27255f350 0x55a27291e020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:47:19.354 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:19.352+0000 7f6938ca2740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x55a2726cd890 con 0x55a272536490 2024-09-21T03:47:19.354 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:19.352+0000 7f6938ca2740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x55a272806d40 con 0x55a27255f350 2024-09-21T03:47:19.355 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:19.352+0000 7f6938ca2740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55a272914cb0 con 0x55a27253e670 2024-09-21T03:47:19.355 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:19.352+0000 7f6938ca2740 10 client.0 _ll_register_callbacks cb 0x55a27286ac80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:47:19.355 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:19.352+0000 7f692bfff640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a272536490 0x55a272911cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:47:19.355 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:19.352+0000 7f692bfff640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a272536490 0x55a272911cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:38606/0 (socket says 172.21.15.39:38606) 2024-09-21T03:47:19.355 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:19.352+0000 7f692bfff640 1 -- 172.21.15.39:0/1075050829 learned_addr learned my addr 172.21.15.39:0/1075050829 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:47:19.355 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:19.352+0000 7f692b7fe640 1 --2- 172.21.15.39:0/1075050829 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a27253e670 0x55a272912260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:47:19.355 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:19.352+0000 7f6930ccb640 1 --2- 172.21.15.39:0/1075050829 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a27255f350 0x55a27291e020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:47:19.356 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:19.353+0000 7f692b7fe640 1 -- 172.21.15.39:0/1075050829 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a27255f350 msgr2=0x55a27291e020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:47:19.356 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:19.353+0000 7f692b7fe640 1 --2- 172.21.15.39:0/1075050829 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a27255f350 0x55a27291e020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:19.356 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:19.353+0000 7f692b7fe640 1 -- 172.21.15.39:0/1075050829 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a272536490 msgr2=0x55a272911cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:47:19.356 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:19.353+0000 7f692b7fe640 1 --2- 172.21.15.39:0/1075050829 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a272536490 0x55a272911cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:19.356 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:19.353+0000 7f692b7fe640 1 -- 172.21.15.39:0/1075050829 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a27291e560 con 0x55a27253e670 2024-09-21T03:47:19.356 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:19.353+0000 7f692bfff640 1 --2- 172.21.15.39:0/1075050829 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a272536490 0x55a272911cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:47:19.357 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:19.353+0000 7f692b7fe640 1 --2- 172.21.15.39:0/1075050829 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a27253e670 0x55a272912260 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f69200e4aa0 tx=0x7f69200ea9b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cfaaa012d73f63ff server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:47:19.357 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:19.353+0000 7f690b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.161:3300/0 2024-09-21T03:47:19.357 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:19.353+0000 7f690b7fe640 1 -- 172.21.15.39:0/1075050829 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69200a9740 con 0x55a27253e670 2024-09-21T03:47:19.357 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:19.353+0000 7f690b7fe640 1 -- 172.21.15.39:0/1075050829 <== mon.1 v2:172.21.15.161:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8484+0+0 (secure 0 0 0) 0x7f69200a8c40 con 0x55a27253e670 2024-09-21T03:47:19.357 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:19.353+0000 7f6938ca2740 1 -- 172.21.15.39:0/1075050829 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f691c00fad0 con 0x55a27253e670 2024-09-21T03:47:19.357 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:19.353+0000 7f690b7fe640 1 -- 172.21.15.39:0/1075050829 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f69200ac860 con 0x55a27253e670 2024-09-21T03:47:19.357 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:19.353+0000 7f6938ca2740 10 client.25468 fetch_fsmap learned FSMap version 29 2024-09-21T03:47:19.357 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:19.353+0000 7f6938ca2740 1 -- 172.21.15.39:0/1075050829 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55a272920df0 con 0x55a27253e670 2024-09-21T03:47:19.357 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:19.354+0000 7f690b7fe640 1 -- 172.21.15.39:0/1075050829 <== mon.1 v2:172.21.15.161:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f69200e4f60 con 0x55a27253e670 2024-09-21T03:47:19.357 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:19.354+0000 7f690b7fe640 1 -- 172.21.15.39:0/1075050829 <== mon.1 v2:172.21.15.161:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f6920098190 con 0x55a27253e670 2024-09-21T03:47:19.357 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:19.354+0000 7f6938ca2740 10 client.25468 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T03:47:19.357 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:19.354+0000 7f6938ca2740 10 client.25468 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T03:47:19.357 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:19.354+0000 7f6938ca2740 20 client.25468 populate_metadata read hostname 'smithi039' 2024-09-21T03:47:19.358 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:19.354+0000 7f6938ca2740 1 --2- 172.21.15.39:0/1075050829 >> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] conn(0x55a27251dcd0 0x55a272924750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:47:19.358 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:19.354+0000 7f6938ca2740 4 client.25468 mds_command: new command op to 24695 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:47:19.358 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:19.354+0000 7f6938ca2740 1 -- 172.21.15.39:0/1075050829 --> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f691c00fad0 con 0x55a27251dcd0 2024-09-21T03:47:19.358 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:19.354+0000 7f692bfff640 1 --2- 172.21.15.39:0/1075050829 >> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] conn(0x55a27251dcd0 0x55a272924750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:47:19.359 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:19.355+0000 7f690b7fe640 10 client.25468 ms_handle_connect on v2:172.21.15.150:6838/2949386907 2024-09-21T03:47:19.359 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:19.355+0000 7f692bfff640 1 --2- 172.21.15.39:0/1075050829 >> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] conn(0x55a27251dcd0 0x55a272924750 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=de0c125d5ce69bc8 server_cookie=842b0014357a94e5 in_seq=0 out_seq=0 2024-09-21T03:47:19.359 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:19.356+0000 7f690b7fe640 1 -- 172.21.15.39:0/1075050829 <== mds.0 v2:172.21.15.150:6838/2949386907 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x7f691c00fad0 con 0x55a27251dcd0 2024-09-21T03:47:19.359 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:19.356+0000 7f690b7fe640 10 client.25468 handle_command_reply: tid=0 2024-09-21T03:47:19.359 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:19.356+0000 7f6938ca2740 1 client.25468 shutdown 2024-09-21T03:47:19.359 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:19.356+0000 7f6938ca2740 1 -- 172.21.15.39:0/1075050829 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a27253e670 msgr2=0x55a272912260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:47:19.359 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:19.356+0000 7f6938ca2740 1 --2- 172.21.15.39:0/1075050829 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a27253e670 0x55a272912260 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f69200e4aa0 tx=0x7f69200ea9b0 comp rx=0 tx=0).stop 2024-09-21T03:47:19.359 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:19.356+0000 7f6938ca2740 1 -- 172.21.15.39:0/1075050829 shutdown_connections 2024-09-21T03:47:19.359 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:19.356+0000 7f6938ca2740 1 --2- 172.21.15.39:0/1075050829 >> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] conn(0x55a27251dcd0 0x55a272924750 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:19.359 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:19.356+0000 7f6938ca2740 1 --2- 172.21.15.39:0/1075050829 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a27255f350 0x55a27291e020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:19.359 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:19.356+0000 7f690b7fe640 0 client.25468 ms_handle_reset on v2:172.21.15.150:6838/2949386907 2024-09-21T03:47:19.359 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:19.356+0000 7f6938ca2740 1 --2- 172.21.15.39:0/1075050829 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a27253e670 0x55a272912260 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:19.360 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:19.356+0000 7f6938ca2740 1 --2- 172.21.15.39:0/1075050829 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a272536490 0x55a272911cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:19.360 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:19.356+0000 7f6938ca2740 1 -- 172.21.15.39:0/1075050829 >> 172.21.15.39:0/1075050829 conn(0x55a272553740 msgr2=0x55a2729095c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:47:19.360 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:19.356+0000 7f6938ca2740 1 -- 172.21.15.39:0/1075050829 shutdown_connections 2024-09-21T03:47:19.360 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:19.356+0000 7f6938ca2740 1 -- 172.21.15.39:0/1075050829 wait complete. 2024-09-21T03:47:19.360 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:19.356+0000 7f6938ca2740 20 client.25468 trim_cache size 0 max 16384 2024-09-21T03:47:19.394 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:19.392+0000 7f6938ca2740 1 Processor -- start 2024-09-21T03:47:19.394 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:19.392+0000 7f6938ca2740 1 -- start start 2024-09-21T03:47:19.394 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:19.393+0000 7f6938ca2740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a27251dcd0 0x55a27290d4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:47:19.395 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:19.393+0000 7f6938ca2740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a272536490 0x55a27290c550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:47:19.395 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:19.393+0000 7f692b7fe640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a272536490 0x55a27290c550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:47:19.395 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:19.393+0000 7f692b7fe640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a272536490 0x55a27290c550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:38616/0 (socket says 172.21.15.39:38616) 2024-09-21T03:47:19.395 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:19.393+0000 7f6938ca2740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a27253e670 0x55a27291df20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:47:19.396 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:19.393+0000 7f692b7fe640 1 -- 172.21.15.39:0/1050013500 learned_addr learned my addr 172.21.15.39:0/1050013500 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:47:19.396 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:19.393+0000 7f6938ca2740 1 -- 172.21.15.39:0/1050013500 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f692c1b5cd0 con 0x55a272536490 2024-09-21T03:47:19.396 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:19.393+0000 7f6938ca2740 1 -- 172.21.15.39:0/1050013500 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f692c1b7810 con 0x55a27251dcd0 2024-09-21T03:47:19.396 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:19.393+0000 7f6938ca2740 1 -- 172.21.15.39:0/1050013500 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f692c1b63a0 con 0x55a27253e670 2024-09-21T03:47:19.396 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:19.393+0000 7f692bfff640 1 --2- 172.21.15.39:0/1050013500 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a27251dcd0 0x55a27290d4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:47:19.396 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:19.393+0000 7f692b7fe640 1 -- 172.21.15.39:0/1050013500 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a27253e670 msgr2=0x55a27291df20 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-21T03:47:19.396 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:19.393+0000 7f692b7fe640 1 --2- 172.21.15.39:0/1050013500 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a27253e670 0x55a27291df20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:19.396 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:19.394+0000 7f692b7fe640 1 -- 172.21.15.39:0/1050013500 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a27251dcd0 msgr2=0x55a27290d4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:47:19.396 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:19.394+0000 7f692b7fe640 1 --2- 172.21.15.39:0/1050013500 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a27251dcd0 0x55a27290d4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:19.396 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:19.394+0000 7f692b7fe640 1 -- 172.21.15.39:0/1050013500 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a272910c20 con 0x55a272536490 2024-09-21T03:47:19.396 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:19.394+0000 7f692b7fe640 1 --2- 172.21.15.39:0/1050013500 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a272536490 0x55a27290c550 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f69200aa360 tx=0x7f69200eaf40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=97a9f4b34f5afd80 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:47:19.397 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:19.394+0000 7f6911ffb640 1 -- 172.21.15.39:0/1050013500 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69200aca40 con 0x55a272536490 2024-09-21T03:47:19.397 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:19.394+0000 7f6911ffb640 1 -- 172.21.15.39:0/1050013500 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f69200acc00 con 0x55a272536490 2024-09-21T03:47:19.397 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:19.394+0000 7f6911ffb640 1 -- 172.21.15.39:0/1050013500 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69200acec0 con 0x55a272536490 2024-09-21T03:47:19.398 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:19.395+0000 7f6938ca2740 1 -- 172.21.15.39:0/1050013500 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a272536490 msgr2=0x55a27290c550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:47:19.398 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:19.395+0000 7f6938ca2740 1 --2- 172.21.15.39:0/1050013500 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a272536490 0x55a27290c550 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f69200aa360 tx=0x7f69200eaf40 comp rx=0 tx=0).stop 2024-09-21T03:47:19.398 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:19.396+0000 7f6938ca2740 1 -- 172.21.15.39:0/1050013500 shutdown_connections 2024-09-21T03:47:19.398 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:19.396+0000 7f6938ca2740 1 --2- 172.21.15.39:0/1050013500 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a27253e670 0x55a27291df20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:19.398 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:19.396+0000 7f6938ca2740 1 --2- 172.21.15.39:0/1050013500 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a272536490 0x55a27290c550 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:19.398 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:19.396+0000 7f6938ca2740 1 --2- 172.21.15.39:0/1050013500 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a27251dcd0 0x55a27290d4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:19.398 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:19.396+0000 7f6938ca2740 1 -- 172.21.15.39:0/1050013500 >> 172.21.15.39:0/1050013500 conn(0x55a272553740 msgr2=0x55a272907460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:47:19.398 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:19.396+0000 7f6938ca2740 1 -- 172.21.15.39:0/1050013500 shutdown_connections 2024-09-21T03:47:19.398 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:19.396+0000 7f6938ca2740 1 -- 172.21.15.39:0/1050013500 wait complete. 2024-09-21T03:47:19.401 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:19.396+0000 7f6938ca2740 1 -- start start 2024-09-21T03:47:19.401 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:19.397+0000 7f6938ca2740 1 Processor -- start 2024-09-21T03:47:19.401 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:19.397+0000 7f6938ca2740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a27251dcd0 0x55a27291a860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:47:19.401 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:19.397+0000 7f6938ca2740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a272536490 0x55a272915d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:47:19.401 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:19.397+0000 7f692b7fe640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a272536490 0x55a272915d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:47:19.401 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:19.397+0000 7f692b7fe640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a272536490 0x55a272915d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:38626/0 (socket says 172.21.15.39:38626) 2024-09-21T03:47:19.401 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:19.397+0000 7f692b7fe640 1 -- 172.21.15.39:0/3146973819 learned_addr learned my addr 172.21.15.39:0/3146973819 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:47:19.401 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:19.397+0000 7f692bfff640 1 --2- 172.21.15.39:0/3146973819 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a27251dcd0 0x55a27291a860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:47:19.401 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:19.397+0000 7f6938ca2740 1 --2- 172.21.15.39:0/3146973819 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a27253e670 0x55a272916340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:47:19.402 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:19.397+0000 7f6938ca2740 1 -- 172.21.15.39:0/3146973819 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f692c1b7140 con 0x55a272536490 2024-09-21T03:47:19.402 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:19.397+0000 7f6938ca2740 1 -- 172.21.15.39:0/3146973819 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f692c1b6a70 con 0x55a27251dcd0 2024-09-21T03:47:19.402 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:19.397+0000 7f6938ca2740 1 -- 172.21.15.39:0/3146973819 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f692c1b3cb0 con 0x55a27253e670 2024-09-21T03:47:19.402 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:19.398+0000 7f6930ccb640 1 --2- 172.21.15.39:0/3146973819 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a27253e670 0x55a272916340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:47:19.402 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:19.398+0000 7f692b7fe640 1 -- 172.21.15.39:0/3146973819 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a27253e670 msgr2=0x55a272916340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:47:19.402 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:19.398+0000 7f692b7fe640 1 --2- 172.21.15.39:0/3146973819 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a27253e670 0x55a272916340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:19.402 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:19.398+0000 7f692b7fe640 1 -- 172.21.15.39:0/3146973819 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a27251dcd0 msgr2=0x55a27291a860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:47:19.402 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:19.398+0000 7f692b7fe640 1 --2- 172.21.15.39:0/3146973819 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a27251dcd0 0x55a27291a860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:19.402 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:19.398+0000 7f692b7fe640 1 -- 172.21.15.39:0/3146973819 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a272916ad0 con 0x55a272536490 2024-09-21T03:47:19.402 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:19.398+0000 7f692b7fe640 1 --2- 172.21.15.39:0/3146973819 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a272536490 0x55a272915d10 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f69200e4c10 tx=0x7f69200eadd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=18aa51b14909ccc3 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:47:19.402 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:19.398+0000 7f690b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-09-21T03:47:19.402 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:19.398+0000 7f6938ca2740 10 client.0 _ll_register_callbacks cb 0x7f69200b4f50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:47:19.403 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:19.398+0000 7f690b7fe640 1 -- 172.21.15.39:0/3146973819 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69200a8f90 con 0x55a272536490 2024-09-21T03:47:19.403 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:19.399+0000 7f6938ca2740 1 -- 172.21.15.39:0/3146973819 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f69200acc00 con 0x55a272536490 2024-09-21T03:47:19.403 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:19.399+0000 7f690b7fe640 1 -- 172.21.15.39:0/3146973819 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8484+0+0 (secure 0 0 0) 0x7f69200eab10 con 0x55a272536490 2024-09-21T03:47:19.403 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:19.399+0000 7f690b7fe640 1 -- 172.21.15.39:0/3146973819 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f69200aca40 con 0x55a272536490 2024-09-21T03:47:19.403 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:19.399+0000 7f6938ca2740 10 client.15867 fetch_fsmap learned FSMap version 29 2024-09-21T03:47:19.403 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:19.399+0000 7f6938ca2740 1 -- 172.21.15.39:0/3146973819 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({fsmap=29}) -- 0x55a27298ee00 con 0x55a272536490 2024-09-21T03:47:19.405 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:19.399+0000 7f690b7fe640 1 -- 172.21.15.39:0/3146973819 <== mon.0 v2:172.21.15.39:3300/0 4 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f69200abfd0 con 0x55a272536490 2024-09-21T03:47:19.405 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:19.400+0000 7f6938ca2740 10 client.15867 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T03:47:19.405 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:19.400+0000 7f6938ca2740 10 client.15867 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T03:47:19.406 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:19.400+0000 7f6938ca2740 20 client.15867 populate_metadata read hostname 'smithi039' 2024-09-21T03:47:19.406 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:19.400+0000 7f6938ca2740 1 --2- 172.21.15.39:0/3146973819 >> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] conn(0x55a27255f350 0x55a272991420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:47:19.406 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:19.400+0000 7f6938ca2740 4 client.15867 mds_command: new command op to 24695 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-21T03:47:19.406 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:19.400+0000 7f6938ca2740 1 -- 172.21.15.39:0/3146973819 --> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55a27287edb0 con 0x55a27255f350 2024-09-21T03:47:19.406 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:19.400+0000 7f692bfff640 1 --2- 172.21.15.39:0/3146973819 >> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] conn(0x55a27255f350 0x55a272991420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:47:19.406 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:19.400+0000 7f690b7fe640 10 client.15867 ms_handle_connect on v2:172.21.15.150:6838/2949386907 2024-09-21T03:47:19.406 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:19.400+0000 7f692bfff640 1 --2- 172.21.15.39:0/3146973819 >> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] conn(0x55a27255f350 0x55a272991420 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2603c610eb729e35 server_cookie=312d45f061e0fec3 in_seq=0 out_seq=0 2024-09-21T03:47:19.406 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:19.402+0000 7f690b7fe640 1 -- 172.21.15.39:0/3146973819 <== mds.0 v2:172.21.15.150:6838/2949386907 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55a27287edb0 con 0x55a27255f350 2024-09-21T03:47:19.406 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:19.402+0000 7f690b7fe640 10 client.15867 handle_command_reply: tid=0 2024-09-21T03:47:19.406 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:19.402+0000 7f6938ca2740 1 client.15867 shutdown 2024-09-21T03:47:19.406 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:19.402+0000 7f6938ca2740 1 -- 172.21.15.39:0/3146973819 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a272536490 msgr2=0x55a272915d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:47:19.407 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:19.402+0000 7f6938ca2740 1 --2- 172.21.15.39:0/3146973819 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a272536490 0x55a272915d10 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f69200e4c10 tx=0x7f69200eadd0 comp rx=0 tx=0).stop 2024-09-21T03:47:19.407 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:19.403+0000 7f6938ca2740 1 -- 172.21.15.39:0/3146973819 shutdown_connections 2024-09-21T03:47:19.407 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:19.403+0000 7f6938ca2740 1 --2- 172.21.15.39:0/3146973819 >> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] conn(0x55a27255f350 0x55a272991420 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:19.407 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:19.403+0000 7f6938ca2740 1 --2- 172.21.15.39:0/3146973819 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a27253e670 0x55a272916340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:19.407 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:19.403+0000 7f690b7fe640 0 client.15867 ms_handle_reset on v2:172.21.15.150:6838/2949386907 2024-09-21T03:47:19.407 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:19.403+0000 7f6938ca2740 1 --2- 172.21.15.39:0/3146973819 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a272536490 0x55a272915d10 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:19.407 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:19.403+0000 7f6938ca2740 1 --2- 172.21.15.39:0/3146973819 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a27251dcd0 0x55a27291a860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:19.407 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:19.403+0000 7f6938ca2740 1 -- 172.21.15.39:0/3146973819 >> 172.21.15.39:0/3146973819 conn(0x55a272553740 msgr2=0x55a272907bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:47:19.407 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:19.403+0000 7f6938ca2740 1 -- 172.21.15.39:0/3146973819 shutdown_connections 2024-09-21T03:47:19.407 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:19.403+0000 7f6938ca2740 1 -- 172.21.15.39:0/3146973819 wait complete. 2024-09-21T03:47:19.407 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:19.403+0000 7f6938ca2740 20 client.15867 trim_cache size 0 max 16384 2024-09-21T03:47:19.408 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-09-21T03:47:19.408 INFO:teuthology.orchestra.run.smithi039.stdout: "return_code": 0, 2024-09-21T03:47:19.408 INFO:teuthology.orchestra.run.smithi039.stdout: "scrub_tag": "c7e2b8e4-057d-435f-b2bd-b633de98035f", 2024-09-21T03:47:19.408 INFO:teuthology.orchestra.run.smithi039.stdout: "mode": "asynchronous" 2024-09-21T03:47:19.408 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-09-21T03:47:19.408 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:19.404+0000 7f693272e640 1 -- 172.21.15.39:0/3716103549 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f69140b6810 msgr2=0x7f69140b8d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:47:19.409 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:19.404+0000 7f693272e640 1 --2- 172.21.15.39:0/3716103549 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f69140b6810 0x7f69140b8d20 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f6924006010 tx=0x7f6924032ed0 comp rx=0 tx=0).stop 2024-09-21T03:47:19.409 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:19.404+0000 7f693272e640 1 -- 172.21.15.39:0/3716103549 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f692c105270 msgr2=0x7f692c1a1cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:47:19.409 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:19.404+0000 7f693272e640 1 --2- 172.21.15.39:0/3716103549 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f692c105270 0x7f692c1a1cb0 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f692000d950 tx=0x7f692000de10 comp rx=0 tx=0).stop 2024-09-21T03:47:19.409 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:19.405+0000 7f693272e640 1 -- 172.21.15.39:0/3716103549 shutdown_connections 2024-09-21T03:47:19.409 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:19.405+0000 7f693272e640 1 --2- 172.21.15.39:0/3716103549 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f69140b6810 0x7f69140b8d20 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:19.409 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:19.405+0000 7f693272e640 1 --2- 172.21.15.39:0/3716103549 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f692c10ddf0 0x7f692c1ac7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:19.409 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:19.405+0000 7f693272e640 1 --2- 172.21.15.39:0/3716103549 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f692c105270 0x7f692c1a1cb0 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:19.409 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:19.405+0000 7f693272e640 1 --2- 172.21.15.39:0/3716103549 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f692c104840 0x7f692c1a1770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:19.409 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:19.405+0000 7f693272e640 1 -- 172.21.15.39:0/3716103549 >> 172.21.15.39:0/3716103549 conn(0x7f692c079730 msgr2=0x7f692c107970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:47:19.409 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:19.405+0000 7f693272e640 1 -- 172.21.15.39:0/3716103549 shutdown_connections 2024-09-21T03:47:19.410 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:19.405+0000 7f693272e640 1 -- 172.21.15.39:0/3716103549 wait complete. 2024-09-21T03:47:20.226 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 51116d5c-77ca-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub status 2024-09-21T03:47:20.513 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/51116d5c-77ca-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:47:21.051 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:47:20 smithi039 ceph-mon[31211]: scrub queued for path: / 2024-09-21T03:47:21.051 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:47:20 smithi039 ceph-mon[31211]: scrub summary: idle+waiting paths [/] 2024-09-21T03:47:21.051 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:47:20 smithi039 ceph-mon[31211]: scrub summary: active paths [/] 2024-09-21T03:47:21.051 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:47:20 smithi039 ceph-mon[31211]: scrub summary: idle+waiting paths [/] 2024-09-21T03:47:21.052 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:47:20 smithi039 ceph-mon[31211]: pgmap v364: 97 pgs: 97 active+clean; 1.4 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 255 B/s wr, 11 op/s 2024-09-21T03:47:21.052 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:47:20 smithi039 ceph-mon[31211]: scrub summary: idle 2024-09-21T03:47:21.175 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:21.151+0000 7f92bc0ee640 1 -- 172.21.15.39:0/2449327319 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f92b4102ae0 msgr2=0x7f92b4102ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:47:21.175 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:21.151+0000 7f92bc0ee640 1 --2- 172.21.15.39:0/2449327319 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f92b4102ae0 0x7f92b4102ef0 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f92b0009900 tx=0x7f92b002a410 comp rx=0 tx=0).stop 2024-09-21T03:47:21.175 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:21.151+0000 7f92bc0ee640 1 -- 172.21.15.39:0/2449327319 shutdown_connections 2024-09-21T03:47:21.175 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:21.151+0000 7f92bc0ee640 1 --2- 172.21.15.39:0/2449327319 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f92b4108d20 0x7f92b410f550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:21.175 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:21.151+0000 7f92bc0ee640 1 --2- 172.21.15.39:0/2449327319 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f92b41083d0 0x7f92b41087e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:21.175 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:21.151+0000 7f92bc0ee640 1 --2- 172.21.15.39:0/2449327319 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f92b4102ae0 0x7f92b4102ef0 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:21.175 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:21.151+0000 7f92bc0ee640 1 -- 172.21.15.39:0/2449327319 >> 172.21.15.39:0/2449327319 conn(0x7f92b40797b0 msgr2=0x7f92b4079bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:47:21.175 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:21.151+0000 7f92bc0ee640 1 -- 172.21.15.39:0/2449327319 shutdown_connections 2024-09-21T03:47:21.175 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:21.151+0000 7f92bc0ee640 1 -- 172.21.15.39:0/2449327319 wait complete. 2024-09-21T03:47:21.176 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:21.152+0000 7f92bc0ee640 1 Processor -- start 2024-09-21T03:47:21.176 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:21.152+0000 7f92bc0ee640 1 -- start start 2024-09-21T03:47:21.176 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:21.152+0000 7f92bc0ee640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f92b4102ae0 0x7f92b419a540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:47:21.176 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:21.152+0000 7f92bc0ee640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f92b41083d0 0x7f92b419aae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:47:21.176 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:21.152+0000 7f92bc0ee640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f92b4108d20 0x7f92b419fd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:47:21.176 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:21.152+0000 7f92bc0ee640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f92b41129e0 con 0x7f92b4102ae0 2024-09-21T03:47:21.176 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:21.152+0000 7f92bc0ee640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f92b4112860 con 0x7f92b4108d20 2024-09-21T03:47:21.176 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:21.152+0000 7f92bc0ee640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f92b4112b60 con 0x7f92b41083d0 2024-09-21T03:47:21.176 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:21.152+0000 7f92b9e8a640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f92b4102ae0 0x7f92b419a540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:47:21.176 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:21.152+0000 7f92b9e8a640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f92b4102ae0 0x7f92b419a540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:38644/0 (socket says 172.21.15.39:38644) 2024-09-21T03:47:21.176 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:21.152+0000 7f92b9e8a640 1 -- 172.21.15.39:0/1424819386 learned_addr learned my addr 172.21.15.39:0/1424819386 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:47:21.176 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:21.152+0000 7f92b9689640 1 --2- 172.21.15.39:0/1424819386 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f92b41083d0 0x7f92b419aae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:47:21.176 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:21.152+0000 7f92ba68b640 1 --2- 172.21.15.39:0/1424819386 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f92b4108d20 0x7f92b419fd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:47:21.177 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:21.152+0000 7f92b9e8a640 1 -- 172.21.15.39:0/1424819386 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f92b41083d0 msgr2=0x7f92b419aae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:47:21.177 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:21.153+0000 7f92b9e8a640 1 --2- 172.21.15.39:0/1424819386 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f92b41083d0 0x7f92b419aae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:21.177 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:21.153+0000 7f92b9e8a640 1 -- 172.21.15.39:0/1424819386 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f92b4108d20 msgr2=0x7f92b419fd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:47:21.177 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:21.153+0000 7f92b9e8a640 1 --2- 172.21.15.39:0/1424819386 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f92b4108d20 0x7f92b419fd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:21.177 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:21.153+0000 7f92b9e8a640 1 -- 172.21.15.39:0/1424819386 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f92b41a04b0 con 0x7f92b4102ae0 2024-09-21T03:47:21.177 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:21.153+0000 7f92b9e8a640 1 --2- 172.21.15.39:0/1424819386 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f92b4102ae0 0x7f92b419a540 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f92b0005940 tx=0x7f92b00042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:47:21.177 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:21.153+0000 7f92aaffd640 1 -- 172.21.15.39:0/1424819386 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92b003f030 con 0x7f92b4102ae0 2024-09-21T03:47:21.177 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:21.153+0000 7f92bc0ee640 1 -- 172.21.15.39:0/1424819386 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f92b41a0740 con 0x7f92b4102ae0 2024-09-21T03:47:21.177 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:21.153+0000 7f92aaffd640 1 -- 172.21.15.39:0/1424819386 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f92b0003cb0 con 0x7f92b4102ae0 2024-09-21T03:47:21.177 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:21.153+0000 7f92aaffd640 1 -- 172.21.15.39:0/1424819386 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92b0035070 con 0x7f92b4102ae0 2024-09-21T03:47:21.177 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:21.153+0000 7f92bc0ee640 1 -- 172.21.15.39:0/1424819386 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f92b41a6f00 con 0x7f92b4102ae0 2024-09-21T03:47:21.177 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:21.156+0000 7f92aaffd640 1 -- 172.21.15.39:0/1424819386 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 26) ==== 154694+0+0 (secure 0 0 0) 0x7f92b0042030 con 0x7f92b4102ae0 2024-09-21T03:47:21.177 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:21.156+0000 7f92c2662740 1 Processor -- start 2024-09-21T03:47:21.177 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:21.156+0000 7f92c2662740 1 -- start start 2024-09-21T03:47:21.178 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:21.156+0000 7f92c2662740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55f3c90c5070 0x55f3c946c270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:47:21.178 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:21.156+0000 7f92c2662740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f3c909c1b0 0x55f3c946c7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:47:21.178 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:21.156+0000 7f92c2662740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f3c90a4390 0x55f3c946ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:47:21.178 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:21.156+0000 7f92c2662740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x55f3c93e2e50 con 0x55f3c90c5070 2024-09-21T03:47:21.178 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:21.156+0000 7f92c2662740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x55f3c93d2280 con 0x55f3c909c1b0 2024-09-21T03:47:21.178 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:21.157+0000 7f92c2662740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55f3c93d6090 con 0x55f3c90a4390 2024-09-21T03:47:21.178 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:21.157+0000 7f92ba68b640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55f3c90c5070 0x55f3c946c270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:47:21.178 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:21.157+0000 7f92ba68b640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55f3c90c5070 0x55f3c946c270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:38650/0 (socket says 172.21.15.39:38650) 2024-09-21T03:47:21.178 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:21.157+0000 7f92ba68b640 1 -- 172.21.15.39:0/3128335700 learned_addr learned my addr 172.21.15.39:0/3128335700 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:47:21.178 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:21.157+0000 7f92aaffd640 1 --2- 172.21.15.39:0/1424819386 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f92a00b6710 0x7f92a00b8c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:47:21.178 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:21.157+0000 7f92b9689640 1 --2- 172.21.15.39:0/3128335700 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f3c90a4390 0x55f3c946ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:47:21.178 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:21.157+0000 7f92b9e8a640 1 --2- 172.21.15.39:0/3128335700 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f3c909c1b0 0x55f3c946c7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:47:21.179 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:21.157+0000 7f92ba68b640 1 --2- 172.21.15.39:0/1424819386 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f92a00b6710 0x7f92a00b8c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:47:21.179 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:21.157+0000 7f92aaffd640 1 -- 172.21.15.39:0/1424819386 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8484+0+0 (secure 0 0 0) 0x7f92b0108090 con 0x7f92b4102ae0 2024-09-21T03:47:21.179 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:21.157+0000 7f92b9689640 1 -- 172.21.15.39:0/3128335700 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f3c909c1b0 msgr2=0x55f3c946c7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:47:21.179 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:21.157+0000 7f92b9689640 1 --2- 172.21.15.39:0/3128335700 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f3c909c1b0 0x55f3c946c7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:21.179 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:21.157+0000 7f92b9689640 1 -- 172.21.15.39:0/3128335700 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55f3c90c5070 msgr2=0x55f3c946c270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:47:21.179 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:21.157+0000 7f92b9689640 1 --2- 172.21.15.39:0/3128335700 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55f3c90c5070 0x55f3c946c270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:21.179 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:21.157+0000 7f92b9689640 1 -- 172.21.15.39:0/3128335700 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f3c9130650 con 0x55f3c90a4390 2024-09-21T03:47:21.179 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:21.157+0000 7f92ba68b640 1 --2- 172.21.15.39:0/3128335700 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55f3c90c5070 0x55f3c946c270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:47:21.179 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:21.158+0000 7f92b9689640 1 --2- 172.21.15.39:0/3128335700 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f3c90a4390 0x55f3c946ccf0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f92a4005d00 tx=0x7f92a40107a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=83689ae0037fb716 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:47:21.179 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:21.158+0000 7f92ba68b640 1 --2- 172.21.15.39:0/1424819386 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f92a00b6710 0x7f92a00b8c20 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f92ac009fd0 tx=0x7f92ac004ab0 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:47:21.179 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:21.158+0000 7f929ffff640 1 -- 172.21.15.39:0/3128335700 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92a401c480 con 0x55f3c90a4390 2024-09-21T03:47:21.179 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:21.158+0000 7f929ffff640 1 -- 172.21.15.39:0/3128335700 <== mon.1 v2:172.21.15.161:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f92a401cad0 con 0x55f3c90a4390 2024-09-21T03:47:21.179 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:21.158+0000 7f929ffff640 1 -- 172.21.15.39:0/3128335700 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92a4011a50 con 0x55f3c90a4390 2024-09-21T03:47:21.179 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:21.159+0000 7f92c2662740 1 -- 172.21.15.39:0/3128335700 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f3c90a4390 msgr2=0x55f3c946ccf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:47:21.180 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:21.159+0000 7f92c2662740 1 --2- 172.21.15.39:0/3128335700 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f3c90a4390 0x55f3c946ccf0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f92a4005d00 tx=0x7f92a40107a0 comp rx=0 tx=0).stop 2024-09-21T03:47:21.180 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:21.159+0000 7f92c2662740 1 -- 172.21.15.39:0/3128335700 shutdown_connections 2024-09-21T03:47:21.180 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:21.159+0000 7f92c2662740 1 --2- 172.21.15.39:0/3128335700 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f3c90a4390 0x55f3c946ccf0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:21.180 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:21.159+0000 7f92c2662740 1 --2- 172.21.15.39:0/3128335700 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f3c909c1b0 0x55f3c946c7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:21.180 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:21.159+0000 7f92c2662740 1 --2- 172.21.15.39:0/3128335700 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55f3c90c5070 0x55f3c946c270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:21.180 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:21.159+0000 7f92c2662740 1 -- 172.21.15.39:0/3128335700 >> 172.21.15.39:0/3128335700 conn(0x55f3c90b9460 msgr2=0x55f3c946a160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:47:21.180 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:21.159+0000 7f92c2662740 1 -- 172.21.15.39:0/3128335700 shutdown_connections 2024-09-21T03:47:21.180 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:21.159+0000 7f92c2662740 1 -- 172.21.15.39:0/3128335700 wait complete. 2024-09-21T03:47:21.180 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:21.159+0000 7f92c2662740 1 -- start start 2024-09-21T03:47:21.180 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:21.159+0000 7f92c2662740 1 Processor -- start 2024-09-21T03:47:21.180 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:21.160+0000 7f92c2662740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f3c909c1b0 0x55f3c9477900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:47:21.180 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:21.160+0000 7f92c2662740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55f3c90a4390 0x55f3c9477e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:47:21.180 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:21.160+0000 7f92c2662740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f3c90c5070 0x55f3c94783c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:47:21.180 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:21.160+0000 7f92c2662740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x55f3c9233700 con 0x55f3c90a4390 2024-09-21T03:47:21.181 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:21.160+0000 7f92c2662740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x55f3c936d110 con 0x55f3c909c1b0 2024-09-21T03:47:21.181 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:21.160+0000 7f92c2662740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55f3c947aa50 con 0x55f3c90c5070 2024-09-21T03:47:21.181 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:21.160+0000 7f92c2662740 10 client.0 _ll_register_callbacks cb 0x55f3c9386990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:47:21.181 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:21.160+0000 7f92b9689640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55f3c90a4390 0x55f3c9477e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:47:21.181 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:21.160+0000 7f92b9e8a640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f3c909c1b0 0x55f3c9477900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:47:21.181 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:21.160+0000 7f92b9689640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55f3c90a4390 0x55f3c9477e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:38664/0 (socket says 172.21.15.39:38664) 2024-09-21T03:47:21.181 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:21.160+0000 7f92b9689640 1 -- 172.21.15.39:0/474736191 learned_addr learned my addr 172.21.15.39:0/474736191 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:47:21.181 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:21.160+0000 7f92ba68b640 1 --2- 172.21.15.39:0/474736191 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f3c90c5070 0x55f3c94783c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:47:21.181 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:21.160+0000 7f92b9e8a640 1 -- 172.21.15.39:0/474736191 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f3c90c5070 msgr2=0x55f3c94783c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:47:21.181 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:21.160+0000 7f92b9e8a640 1 --2- 172.21.15.39:0/474736191 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f3c90c5070 0x55f3c94783c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:21.181 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:21.160+0000 7f92b9e8a640 1 -- 172.21.15.39:0/474736191 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55f3c90a4390 msgr2=0x55f3c9477e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:47:21.181 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:21.160+0000 7f92b9e8a640 1 --2- 172.21.15.39:0/474736191 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55f3c90a4390 0x55f3c9477e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:21.181 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:21.160+0000 7f92b9e8a640 1 -- 172.21.15.39:0/474736191 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f3c939ac40 con 0x55f3c909c1b0 2024-09-21T03:47:21.182 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:21.161+0000 7f92b9689640 1 --2- 172.21.15.39:0/474736191 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55f3c90a4390 0x55f3c9477e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:47:21.182 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:21.161+0000 7f92b9e8a640 1 --2- 172.21.15.39:0/474736191 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f3c909c1b0 0x55f3c9477900 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f92b0105740 tx=0x7f92b0035000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7d700afc18042b67 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:47:21.182 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:21.161+0000 7f929d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.150:3300/0 2024-09-21T03:47:21.182 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:21.161+0000 7f929d7fa640 1 -- 172.21.15.39:0/474736191 <== mon.2 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92b0049070 con 0x55f3c909c1b0 2024-09-21T03:47:21.182 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:21.161+0000 7f92c2662740 1 -- 172.21.15.39:0/474736191 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f92a401cad0 con 0x55f3c909c1b0 2024-09-21T03:47:21.182 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:21.161+0000 7f929d7fa640 1 -- 172.21.15.39:0/474736191 <== mon.2 v2:172.21.15.150:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8484+0+0 (secure 0 0 0) 0x7f92b00b50e0 con 0x55f3c909c1b0 2024-09-21T03:47:21.182 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:21.161+0000 7f929d7fa640 1 -- 172.21.15.39:0/474736191 <== mon.2 v2:172.21.15.150:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f92b00c9ce0 con 0x55f3c909c1b0 2024-09-21T03:47:21.182 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:21.161+0000 7f92c2662740 10 client.25463 fetch_fsmap learned FSMap version 29 2024-09-21T03:47:21.182 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:21.161+0000 7f92c2662740 1 -- 172.21.15.39:0/474736191 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55f3c94866f0 con 0x55f3c909c1b0 2024-09-21T03:47:21.182 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:21.161+0000 7f929d7fa640 1 -- 172.21.15.39:0/474736191 <== mon.2 v2:172.21.15.150:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f92a401cad0 con 0x55f3c909c1b0 2024-09-21T03:47:21.182 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:21.162+0000 7f929d7fa640 1 -- 172.21.15.39:0/474736191 <== mon.2 v2:172.21.15.150:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f92b01059f0 con 0x55f3c909c1b0 2024-09-21T03:47:21.182 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:21.162+0000 7f92c2662740 10 client.25463 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T03:47:21.182 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:21.162+0000 7f92c2662740 10 client.25463 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T03:47:21.183 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:21.162+0000 7f92c2662740 20 client.25463 populate_metadata read hostname 'smithi039' 2024-09-21T03:47:21.183 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:21.162+0000 7f92c2662740 1 --2- 172.21.15.39:0/474736191 >> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] conn(0x55f3c90839f0 0x55f3c9489c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:47:21.183 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:21.162+0000 7f92c2662740 4 client.25463 mds_command: new command op to 24695 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:47:21.183 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:21.162+0000 7f92c2662740 1 -- 172.21.15.39:0/474736191 --> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f3c93e4da0 con 0x55f3c90839f0 2024-09-21T03:47:21.183 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:21.162+0000 7f92b9689640 1 --2- 172.21.15.39:0/474736191 >> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] conn(0x55f3c90839f0 0x55f3c9489c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:47:21.183 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:21.162+0000 7f929d7fa640 10 client.25463 ms_handle_connect on v2:172.21.15.150:6838/2949386907 2024-09-21T03:47:21.183 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:21.162+0000 7f92b9689640 1 --2- 172.21.15.39:0/474736191 >> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] conn(0x55f3c90839f0 0x55f3c9489c80 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9db9a81b98b3596d server_cookie=8f299148a75e76d9 in_seq=0 out_seq=0 2024-09-21T03:47:21.183 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:21.163+0000 7f929d7fa640 1 -- 172.21.15.39:0/474736191 <== mds.0 v2:172.21.15.150:6838/2949386907 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55f3c93e4da0 con 0x55f3c90839f0 2024-09-21T03:47:21.183 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:21.163+0000 7f929d7fa640 10 client.25463 handle_command_reply: tid=0 2024-09-21T03:47:21.183 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:21.163+0000 7f92c2662740 1 client.25463 shutdown 2024-09-21T03:47:21.183 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:21.163+0000 7f92c2662740 1 -- 172.21.15.39:0/474736191 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f3c909c1b0 msgr2=0x55f3c9477900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:47:21.183 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:21.163+0000 7f92c2662740 1 --2- 172.21.15.39:0/474736191 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f3c909c1b0 0x55f3c9477900 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f92b0105740 tx=0x7f92b0035000 comp rx=0 tx=0).stop 2024-09-21T03:47:21.183 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:21.164+0000 7f92c2662740 1 -- 172.21.15.39:0/474736191 shutdown_connections 2024-09-21T03:47:21.183 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:21.164+0000 7f92c2662740 1 --2- 172.21.15.39:0/474736191 >> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] conn(0x55f3c90839f0 0x55f3c9489c80 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:21.184 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:21.164+0000 7f92c2662740 1 --2- 172.21.15.39:0/474736191 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f3c90c5070 0x55f3c94783c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:21.184 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:21.164+0000 7f92c2662740 1 --2- 172.21.15.39:0/474736191 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55f3c90a4390 0x55f3c9477e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:21.184 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:21.164+0000 7f929d7fa640 0 client.25463 ms_handle_reset on v2:172.21.15.150:6838/2949386907 2024-09-21T03:47:21.184 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:21.164+0000 7f92c2662740 1 --2- 172.21.15.39:0/474736191 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f3c909c1b0 0x55f3c9477900 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:21.184 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:21.164+0000 7f92c2662740 1 -- 172.21.15.39:0/474736191 >> 172.21.15.39:0/474736191 conn(0x55f3c90b9460 msgr2=0x55f3c946f450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:47:21.184 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:21.164+0000 7f92c2662740 1 -- 172.21.15.39:0/474736191 shutdown_connections 2024-09-21T03:47:21.184 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:21.164+0000 7f92c2662740 1 -- 172.21.15.39:0/474736191 wait complete. 2024-09-21T03:47:21.184 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:21.164+0000 7f92c2662740 20 client.25463 trim_cache size 0 max 16384 2024-09-21T03:47:21.186 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:21.182+0000 7f92c2662740 1 Processor -- start 2024-09-21T03:47:21.186 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:21.182+0000 7f92c2662740 1 -- start start 2024-09-21T03:47:21.186 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:21.182+0000 7f92c2662740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55f3c909c1b0 0x55f3c946d9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:47:21.186 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:21.182+0000 7f92c2662740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f3c90a4390 0x55f3c9474af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:47:21.186 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:21.182+0000 7f92c2662740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f3c90c5070 0x55f3c9475030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:47:21.186 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:21.182+0000 7f92c2662740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f92b41af690 con 0x55f3c909c1b0 2024-09-21T03:47:21.187 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:21.182+0000 7f92c2662740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f92b41b11d0 con 0x55f3c90c5070 2024-09-21T03:47:21.187 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:21.182+0000 7f92c2662740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f92b41afd60 con 0x55f3c90a4390 2024-09-21T03:47:21.187 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:21.182+0000 7f92b9e8a640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55f3c909c1b0 0x55f3c946d9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:47:21.187 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:21.182+0000 7f92b9e8a640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55f3c909c1b0 0x55f3c946d9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:38666/0 (socket says 172.21.15.39:38666) 2024-09-21T03:47:21.187 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:21.182+0000 7f92b9e8a640 1 -- 172.21.15.39:0/2200238530 learned_addr learned my addr 172.21.15.39:0/2200238530 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:47:21.187 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:21.182+0000 7f92ba68b640 1 --2- 172.21.15.39:0/2200238530 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f3c90c5070 0x55f3c9475030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:47:21.187 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:21.182+0000 7f92b9689640 1 --2- 172.21.15.39:0/2200238530 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f3c90a4390 0x55f3c9474af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:47:21.187 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:21.183+0000 7f92b9e8a640 1 -- 172.21.15.39:0/2200238530 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f3c90a4390 msgr2=0x55f3c9474af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:47:21.187 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:21.183+0000 7f92b9e8a640 1 --2- 172.21.15.39:0/2200238530 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f3c90a4390 0x55f3c9474af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:21.187 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:21.183+0000 7f92b9e8a640 1 -- 172.21.15.39:0/2200238530 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f3c90c5070 msgr2=0x55f3c9475030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:47:21.187 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:21.183+0000 7f92b9e8a640 1 --2- 172.21.15.39:0/2200238530 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f3c90c5070 0x55f3c9475030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:21.187 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:21.183+0000 7f92b9e8a640 1 -- 172.21.15.39:0/2200238530 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f3c946e560 con 0x55f3c909c1b0 2024-09-21T03:47:21.187 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:21.183+0000 7f92b9e8a640 1 --2- 172.21.15.39:0/2200238530 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55f3c909c1b0 0x55f3c946d9b0 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f92b00cddd0 tx=0x7f92b002a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aa56f7423dacfafe server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:47:21.188 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:21.183+0000 7f929f7fe640 1 -- 172.21.15.39:0/2200238530 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92b0049070 con 0x55f3c909c1b0 2024-09-21T03:47:21.188 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:21.183+0000 7f929f7fe640 1 -- 172.21.15.39:0/2200238530 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f92b00cd950 con 0x55f3c909c1b0 2024-09-21T03:47:21.188 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:21.183+0000 7f929f7fe640 1 -- 172.21.15.39:0/2200238530 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92b00d1050 con 0x55f3c909c1b0 2024-09-21T03:47:21.188 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:21.184+0000 7f92c2662740 1 -- 172.21.15.39:0/2200238530 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55f3c909c1b0 msgr2=0x55f3c946d9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:47:21.188 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:21.184+0000 7f92c2662740 1 --2- 172.21.15.39:0/2200238530 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55f3c909c1b0 0x55f3c946d9b0 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f92b00cddd0 tx=0x7f92b002a410 comp rx=0 tx=0).stop 2024-09-21T03:47:21.188 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:21.184+0000 7f92c2662740 1 -- 172.21.15.39:0/2200238530 shutdown_connections 2024-09-21T03:47:21.188 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:21.184+0000 7f92c2662740 1 --2- 172.21.15.39:0/2200238530 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f3c90c5070 0x55f3c9475030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:21.188 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:21.184+0000 7f92c2662740 1 --2- 172.21.15.39:0/2200238530 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f3c90a4390 0x55f3c9474af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:21.188 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:21.184+0000 7f92c2662740 1 --2- 172.21.15.39:0/2200238530 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55f3c909c1b0 0x55f3c946d9b0 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:21.188 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:21.184+0000 7f92c2662740 1 -- 172.21.15.39:0/2200238530 >> 172.21.15.39:0/2200238530 conn(0x55f3c90b9460 msgr2=0x55f3c946d2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:47:21.188 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:21.184+0000 7f92c2662740 1 -- 172.21.15.39:0/2200238530 shutdown_connections 2024-09-21T03:47:21.188 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:21.184+0000 7f92c2662740 1 -- 172.21.15.39:0/2200238530 wait complete. 2024-09-21T03:47:21.188 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:21.185+0000 7f92c2662740 1 -- start start 2024-09-21T03:47:21.188 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:21.185+0000 7f92c2662740 1 Processor -- start 2024-09-21T03:47:21.191 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:21.185+0000 7f92c2662740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f3c909c1b0 0x55f3c94806b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:47:21.191 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:21.185+0000 7f92c2662740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f3c90a4390 0x55f3c947bbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:47:21.192 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:21.185+0000 7f92c2662740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55f3c90c5070 0x55f3c947c2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:47:21.192 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:21.185+0000 7f92c2662740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f92b41b0b00 con 0x55f3c90c5070 2024-09-21T03:47:21.192 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:21.185+0000 7f92c2662740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f92b41b0430 con 0x55f3c90a4390 2024-09-21T03:47:21.192 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:21.185+0000 7f92c2662740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f92b41ad670 con 0x55f3c909c1b0 2024-09-21T03:47:21.192 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:21.185+0000 7f92c2662740 10 client.0 _ll_register_callbacks cb 0x7f92b00ca510 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:47:21.192 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:21.185+0000 7f92ba68b640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55f3c90c5070 0x55f3c947c2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:47:21.192 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:21.185+0000 7f92ba68b640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55f3c90c5070 0x55f3c947c2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:38680/0 (socket says 172.21.15.39:38680) 2024-09-21T03:47:21.192 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:21.185+0000 7f92ba68b640 1 -- 172.21.15.39:0/1692435667 learned_addr learned my addr 172.21.15.39:0/1692435667 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:47:21.192 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:21.186+0000 7f92b9e8a640 1 --2- 172.21.15.39:0/1692435667 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f3c909c1b0 0x55f3c94806b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:47:21.192 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:21.186+0000 7f92b9689640 1 --2- 172.21.15.39:0/1692435667 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f3c90a4390 0x55f3c947bbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:47:21.192 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:21.186+0000 7f92ba68b640 1 -- 172.21.15.39:0/1692435667 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f3c909c1b0 msgr2=0x55f3c94806b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:47:21.192 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:21.186+0000 7f92ba68b640 1 --2- 172.21.15.39:0/1692435667 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f3c909c1b0 0x55f3c94806b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:21.192 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:21.186+0000 7f92ba68b640 1 -- 172.21.15.39:0/1692435667 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f3c90a4390 msgr2=0x55f3c947bbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:47:21.193 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:21.186+0000 7f92ba68b640 1 --2- 172.21.15.39:0/1692435667 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f3c90a4390 0x55f3c947bbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:21.193 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:21.186+0000 7f92ba68b640 1 -- 172.21.15.39:0/1692435667 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f3c94f3640 con 0x55f3c90c5070 2024-09-21T03:47:21.193 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:21.186+0000 7f92ba68b640 1 --2- 172.21.15.39:0/1692435667 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55f3c90c5070 0x55f3c947c2d0 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f92ac0123b0 tx=0x7f92ac004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c6921c262aa981b server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:47:21.193 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:21.186+0000 7f929d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-09-21T03:47:21.193 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:21.186+0000 7f929d7fa640 1 -- 172.21.15.39:0/1692435667 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92ac018070 con 0x55f3c90c5070 2024-09-21T03:47:21.193 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:21.186+0000 7f92c2662740 1 -- 172.21.15.39:0/1692435667 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f92b00cd950 con 0x55f3c90c5070 2024-09-21T03:47:21.193 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:21.186+0000 7f929d7fa640 1 -- 172.21.15.39:0/1692435667 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8484+0+0 (secure 0 0 0) 0x7f92ac003970 con 0x55f3c90c5070 2024-09-21T03:47:21.193 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:21.187+0000 7f929d7fa640 1 -- 172.21.15.39:0/1692435667 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f92ac003b70 con 0x55f3c90c5070 2024-09-21T03:47:21.193 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:21.187+0000 7f92c2662740 10 client.15885 fetch_fsmap learned FSMap version 29 2024-09-21T03:47:21.193 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:21.187+0000 7f92c2662740 1 -- 172.21.15.39:0/1692435667 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55f3c94f4830 con 0x55f3c90c5070 2024-09-21T03:47:21.193 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:21.187+0000 7f929d7fa640 1 -- 172.21.15.39:0/1692435667 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f92b00cd950 con 0x55f3c90c5070 2024-09-21T03:47:21.193 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:21.187+0000 7f929d7fa640 1 -- 172.21.15.39:0/1692435667 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f92ac033050 con 0x55f3c90c5070 2024-09-21T03:47:21.193 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:21.187+0000 7f92c2662740 10 client.15885 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T03:47:21.193 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:21.187+0000 7f92c2662740 10 client.15885 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T03:47:21.194 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:21.187+0000 7f92c2662740 20 client.15885 populate_metadata read hostname 'smithi039' 2024-09-21T03:47:21.194 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:21.187+0000 7f92c2662740 1 --2- 172.21.15.39:0/1692435667 >> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] conn(0x55f3c9058da0 0x55f3c94f6fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:47:21.194 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:21.187+0000 7f92c2662740 4 client.15885 mds_command: new command op to 24695 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-21T03:47:21.194 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:21.187+0000 7f92c2662740 1 -- 172.21.15.39:0/1692435667 --> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55f3c93e0850 con 0x55f3c9058da0 2024-09-21T03:47:21.194 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:21.187+0000 7f92b9e8a640 1 --2- 172.21.15.39:0/1692435667 >> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] conn(0x55f3c9058da0 0x55f3c94f6fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:47:21.194 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:21.188+0000 7f92b9e8a640 1 --2- 172.21.15.39:0/1692435667 >> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] conn(0x55f3c9058da0 0x55f3c94f6fd0 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=db36ae4158fd54c8 server_cookie=b5caccebd37ec7e2 in_seq=0 out_seq=0 2024-09-21T03:47:21.194 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:21.188+0000 7f929d7fa640 10 client.15885 ms_handle_connect on v2:172.21.15.150:6838/2949386907 2024-09-21T03:47:21.194 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:21.188+0000 7f929d7fa640 1 -- 172.21.15.39:0/1692435667 <== mds.0 v2:172.21.15.150:6838/2949386907 1 ==== command_reply(tid 0: 0 ) ==== 8+0+942 (crc 0 0 0) 0x55f3c93e0850 con 0x55f3c9058da0 2024-09-21T03:47:21.194 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:21.188+0000 7f929d7fa640 10 client.15885 handle_command_reply: tid=0 2024-09-21T03:47:21.194 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:21.188+0000 7f92c2662740 1 client.15885 shutdown 2024-09-21T03:47:21.194 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:21.188+0000 7f92c2662740 1 -- 172.21.15.39:0/1692435667 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55f3c90c5070 msgr2=0x55f3c947c2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:47:21.194 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:21.188+0000 7f92c2662740 1 --2- 172.21.15.39:0/1692435667 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55f3c90c5070 0x55f3c947c2d0 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f92ac0123b0 tx=0x7f92ac004a40 comp rx=0 tx=0).stop 2024-09-21T03:47:21.194 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:21.189+0000 7f92c2662740 1 -- 172.21.15.39:0/1692435667 shutdown_connections 2024-09-21T03:47:21.195 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:21.189+0000 7f92c2662740 1 --2- 172.21.15.39:0/1692435667 >> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] conn(0x55f3c9058da0 0x55f3c94f6fd0 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:21.195 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:21.189+0000 7f92c2662740 1 --2- 172.21.15.39:0/1692435667 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55f3c90c5070 0x55f3c947c2d0 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:21.195 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:21.189+0000 7f92c2662740 1 --2- 172.21.15.39:0/1692435667 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f3c90a4390 0x55f3c947bbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:21.195 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:21.189+0000 7f92c2662740 1 --2- 172.21.15.39:0/1692435667 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f3c909c1b0 0x55f3c94806b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:21.195 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:21.189+0000 7f929d7fa640 0 client.15885 ms_handle_reset on v2:172.21.15.150:6838/2949386907 2024-09-21T03:47:21.195 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:21.189+0000 7f92c2662740 1 -- 172.21.15.39:0/1692435667 >> 172.21.15.39:0/1692435667 conn(0x55f3c90b9460 msgr2=0x55f3c94835f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:47:21.195 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:21.189+0000 7f92c2662740 1 -- 172.21.15.39:0/1692435667 shutdown_connections 2024-09-21T03:47:21.195 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:21.189+0000 7f92c2662740 1 -- 172.21.15.39:0/1692435667 wait complete. 2024-09-21T03:47:21.195 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:21.189+0000 7f92c2662740 20 client.15885 trim_cache size 0 max 16384 2024-09-21T03:47:21.195 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-09-21T03:47:21.196 INFO:teuthology.orchestra.run.smithi039.stdout: "status": "no active scrubs running", 2024-09-21T03:47:21.196 INFO:teuthology.orchestra.run.smithi039.stdout: "scrubs": {}, 2024-09-21T03:47:21.196 INFO:teuthology.orchestra.run.smithi039.stdout: "scrub_stats": { 2024-09-21T03:47:21.196 INFO:teuthology.orchestra.run.smithi039.stdout: "c7e2b8e4-057d-435f-b2bd-b633de98035f": { 2024-09-21T03:47:21.196 INFO:teuthology.orchestra.run.smithi039.stdout: "start_time": "2024-09-21T03:47:19.400305+0000", 2024-09-21T03:47:21.196 INFO:teuthology.orchestra.run.smithi039.stdout: "path": "/", 2024-09-21T03:47:21.196 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_started": 0, 2024-09-21T03:47:21.196 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_passed": 0, 2024-09-21T03:47:21.196 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_failed": 0, 2024-09-21T03:47:21.196 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_skipped": 0 2024-09-21T03:47:21.196 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:47:21.196 INFO:teuthology.orchestra.run.smithi039.stdout: "10728de8-c49d-4edc-be2b-639249255e94": { 2024-09-21T03:47:21.196 INFO:teuthology.orchestra.run.smithi039.stdout: "start_time": "2024-09-21T03:47:03.965623+0000", 2024-09-21T03:47:21.197 INFO:teuthology.orchestra.run.smithi039.stdout: "path": "/", 2024-09-21T03:47:21.197 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_started": 0, 2024-09-21T03:47:21.197 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_passed": 0, 2024-09-21T03:47:21.197 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_failed": 0, 2024-09-21T03:47:21.197 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_skipped": 0 2024-09-21T03:47:21.197 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:47:21.197 INFO:teuthology.orchestra.run.smithi039.stdout: "02509e4f-3911-4983-b16f-df8ded0b9998": { 2024-09-21T03:47:21.197 INFO:teuthology.orchestra.run.smithi039.stdout: "start_time": "2024-09-21T03:46:17.889576+0000", 2024-09-21T03:47:21.197 INFO:teuthology.orchestra.run.smithi039.stdout: "path": "/", 2024-09-21T03:47:21.197 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_started": 0, 2024-09-21T03:47:21.197 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_passed": 0, 2024-09-21T03:47:21.197 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_failed": 0, 2024-09-21T03:47:21.197 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_skipped": 0 2024-09-21T03:47:21.197 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-21T03:47:21.198 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-21T03:47:21.198 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-09-21T03:47:21.198 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:21.190+0000 7f92bc0ee640 1 -- 172.21.15.39:0/1424819386 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f92a00b6710 msgr2=0x7f92a00b8c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:47:21.199 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:21.190+0000 7f92bc0ee640 1 --2- 172.21.15.39:0/1424819386 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f92a00b6710 0x7f92a00b8c20 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f92ac009fd0 tx=0x7f92ac004ab0 comp rx=0 tx=0).stop 2024-09-21T03:47:21.199 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:21.190+0000 7f92bc0ee640 1 -- 172.21.15.39:0/1424819386 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f92b4102ae0 msgr2=0x7f92b419a540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:47:21.199 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:21.190+0000 7f92bc0ee640 1 --2- 172.21.15.39:0/1424819386 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f92b4102ae0 0x7f92b419a540 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f92b0005940 tx=0x7f92b00042f0 comp rx=0 tx=0).stop 2024-09-21T03:47:21.199 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:21.190+0000 7f92bc0ee640 1 -- 172.21.15.39:0/1424819386 shutdown_connections 2024-09-21T03:47:21.199 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:21.190+0000 7f92bc0ee640 1 --2- 172.21.15.39:0/1424819386 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f92a00b6710 0x7f92a00b8c20 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:21.199 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:21.190+0000 7f92bc0ee640 1 --2- 172.21.15.39:0/1424819386 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f92b4108d20 0x7f92b419fd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:21.199 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:21.190+0000 7f92bc0ee640 1 --2- 172.21.15.39:0/1424819386 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f92b41083d0 0x7f92b419aae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:21.199 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:21.190+0000 7f92bc0ee640 1 --2- 172.21.15.39:0/1424819386 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f92b4102ae0 0x7f92b419a540 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:21.199 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:21.190+0000 7f92bc0ee640 1 -- 172.21.15.39:0/1424819386 >> 172.21.15.39:0/1424819386 conn(0x7f92b40797b0 msgr2=0x7f92b4101a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:47:21.199 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:21.190+0000 7f92bc0ee640 1 -- 172.21.15.39:0/1424819386 shutdown_connections 2024-09-21T03:47:21.199 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:21.190+0000 7f92bc0ee640 1 -- 172.21.15.39:0/1424819386 wait complete. 2024-09-21T03:47:21.217 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:47:20 smithi161 ceph-mon[34533]: scrub queued for path: / 2024-09-21T03:47:21.218 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:47:20 smithi161 ceph-mon[34533]: scrub summary: idle+waiting paths [/] 2024-09-21T03:47:21.218 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:47:20 smithi161 ceph-mon[34533]: scrub summary: active paths [/] 2024-09-21T03:47:21.218 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:47:20 smithi161 ceph-mon[34533]: scrub summary: idle+waiting paths [/] 2024-09-21T03:47:21.218 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:47:20 smithi161 ceph-mon[34533]: pgmap v364: 97 pgs: 97 active+clean; 1.4 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 255 B/s wr, 11 op/s 2024-09-21T03:47:21.218 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:47:20 smithi161 ceph-mon[34533]: scrub summary: idle 2024-09-21T03:47:21.224 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:47:20 smithi150 ceph-mon[34289]: scrub queued for path: / 2024-09-21T03:47:21.224 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:47:20 smithi150 ceph-mon[34289]: scrub summary: idle+waiting paths [/] 2024-09-21T03:47:21.225 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:47:20 smithi150 ceph-mon[34289]: scrub summary: active paths [/] 2024-09-21T03:47:21.225 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:47:20 smithi150 ceph-mon[34289]: scrub summary: idle+waiting paths [/] 2024-09-21T03:47:21.225 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:47:20 smithi150 ceph-mon[34289]: pgmap v364: 97 pgs: 97 active+clean; 1.4 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 255 B/s wr, 11 op/s 2024-09-21T03:47:21.225 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:47:20 smithi150 ceph-mon[34289]: scrub summary: idle 2024-09-21T03:47:22.413 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-21T03:47:22.413 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 51116d5c-77ca-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T03:47:22.685 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/51116d5c-77ca-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:47:23.084 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:47:22 smithi039 ceph-mon[31211]: pgmap v365: 97 pgs: 97 active+clean; 1.4 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 170 B/s wr, 9 op/s 2024-09-21T03:47:23.218 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:47:22 smithi161 ceph-mon[34533]: pgmap v365: 97 pgs: 97 active+clean; 1.4 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 170 B/s wr, 9 op/s 2024-09-21T03:47:23.224 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:47:22 smithi150 ceph-mon[34289]: pgmap v365: 97 pgs: 97 active+clean; 1.4 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 170 B/s wr, 9 op/s 2024-09-21T03:47:23.271 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:23.254+0000 7f49f2557640 1 -- 172.21.15.39:0/3361047627 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f49ec102f20 msgr2=0x7f49ec10fc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:47:23.271 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:23.254+0000 7f49f2557640 1 --2- 172.21.15.39:0/3361047627 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f49ec102f20 0x7f49ec10fc40 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f49e0009900 tx=0x7f49e002a480 comp rx=0 tx=0).stop 2024-09-21T03:47:23.271 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:23.255+0000 7f49f2557640 1 -- 172.21.15.39:0/3361047627 shutdown_connections 2024-09-21T03:47:23.271 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:23.255+0000 7f49f2557640 1 --2- 172.21.15.39:0/3361047627 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f49ec110180 0x7f49ec1125a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:23.271 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:23.255+0000 7f49f2557640 1 --2- 172.21.15.39:0/3361047627 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f49ec102f20 0x7f49ec10fc40 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:23.271 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:23.255+0000 7f49f2557640 1 --2- 172.21.15.39:0/3361047627 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f49ec1025d0 0x7f49ec1029e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:23.271 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:23.255+0000 7f49f2557640 1 -- 172.21.15.39:0/3361047627 >> 172.21.15.39:0/3361047627 conn(0x7f49ec079730 msgr2=0x7f49ec079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:47:23.271 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:23.255+0000 7f49f2557640 1 -- 172.21.15.39:0/3361047627 shutdown_connections 2024-09-21T03:47:23.271 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:23.255+0000 7f49f2557640 1 -- 172.21.15.39:0/3361047627 wait complete. 2024-09-21T03:47:23.271 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:23.255+0000 7f49f2557640 1 Processor -- start 2024-09-21T03:47:23.271 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:23.255+0000 7f49f2557640 1 -- start start 2024-09-21T03:47:23.272 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:23.256+0000 7f49f2557640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f49ec1025d0 0x7f49ec19c640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:47:23.272 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:23.256+0000 7f49f2557640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f49ec102f20 0x7f49ec19cbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:47:23.272 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:23.256+0000 7f49f2557640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f49ec110180 0x7f49ec1a1e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:47:23.272 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:23.256+0000 7f49f2557640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f49ec115c70 con 0x7f49ec110180 2024-09-21T03:47:23.272 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:23.256+0000 7f49f2557640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f49ec115af0 con 0x7f49ec1025d0 2024-09-21T03:47:23.272 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:23.256+0000 7f49f2557640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f49ec115df0 con 0x7f49ec102f20 2024-09-21T03:47:23.272 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:23.256+0000 7f49f0af4640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f49ec110180 0x7f49ec1a1e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:47:23.272 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:23.256+0000 7f49f0af4640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f49ec110180 0x7f49ec1a1e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:38706/0 (socket says 172.21.15.39:38706) 2024-09-21T03:47:23.272 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:23.256+0000 7f49eb7fe640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f49ec102f20 0x7f49ec19cbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:47:23.272 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:23.256+0000 7f49f0af4640 1 -- 172.21.15.39:0/192770096 learned_addr learned my addr 172.21.15.39:0/192770096 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:47:23.272 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:23.256+0000 7f49ebfff640 1 --2- 172.21.15.39:0/192770096 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f49ec1025d0 0x7f49ec19c640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:47:23.272 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:23.256+0000 7f49f0af4640 1 -- 172.21.15.39:0/192770096 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f49ec102f20 msgr2=0x7f49ec19cbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:47:23.272 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:23.256+0000 7f49f0af4640 1 --2- 172.21.15.39:0/192770096 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f49ec102f20 0x7f49ec19cbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:23.273 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:23.256+0000 7f49f0af4640 1 -- 172.21.15.39:0/192770096 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f49ec1025d0 msgr2=0x7f49ec19c640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:47:23.273 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:23.256+0000 7f49f0af4640 1 --2- 172.21.15.39:0/192770096 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f49ec1025d0 0x7f49ec19c640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:23.273 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:23.256+0000 7f49f0af4640 1 -- 172.21.15.39:0/192770096 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f49ec1a25b0 con 0x7f49ec110180 2024-09-21T03:47:23.273 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:23.257+0000 7f49f0af4640 1 --2- 172.21.15.39:0/192770096 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f49ec110180 0x7f49ec1a1e80 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f49e400eaa0 tx=0x7f49e400ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:47:23.273 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:23.257+0000 7f49e97fa640 1 -- 172.21.15.39:0/192770096 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49e400cdb0 con 0x7f49ec110180 2024-09-21T03:47:23.273 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:23.257+0000 7f49f2557640 1 -- 172.21.15.39:0/192770096 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f49ec1a28a0 con 0x7f49ec110180 2024-09-21T03:47:23.273 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:23.257+0000 7f49e97fa640 1 -- 172.21.15.39:0/192770096 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f49e4005020 con 0x7f49ec110180 2024-09-21T03:47:23.273 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:23.257+0000 7f49e97fa640 1 -- 172.21.15.39:0/192770096 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49e40116d0 con 0x7f49ec110180 2024-09-21T03:47:23.273 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:23.257+0000 7f49f2557640 1 -- 172.21.15.39:0/192770096 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f49ec1a9000 con 0x7f49ec110180 2024-09-21T03:47:23.273 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:23.258+0000 7f49f2557640 1 -- 172.21.15.39:0/192770096 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f49ec06b560 con 0x7f49ec110180 2024-09-21T03:47:23.273 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:23.260+0000 7f49e97fa640 1 -- 172.21.15.39:0/192770096 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 26) ==== 154694+0+0 (secure 0 0 0) 0x7f49e40051d0 con 0x7f49ec110180 2024-09-21T03:47:23.273 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:23.260+0000 7f49e97fa640 1 --2- 172.21.15.39:0/192770096 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f49d40b6710 0x7f49d40b8c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:47:23.273 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:23.261+0000 7f49ebfff640 1 --2- 172.21.15.39:0/192770096 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f49d40b6710 0x7f49d40b8c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:47:23.273 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:23.261+0000 7f49e97fa640 1 -- 172.21.15.39:0/192770096 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8484+0+0 (secure 0 0 0) 0x7f49e40e7d80 con 0x7f49ec110180 2024-09-21T03:47:23.274 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:23.261+0000 7f49ebfff640 1 --2- 172.21.15.39:0/192770096 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f49d40b6710 0x7f49d40b8c20 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f49dc00bbe0 tx=0x7f49dc00b390 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:47:23.274 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:23.264+0000 7f49e97fa640 1 -- 172.21.15.39:0/192770096 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f49e4016030 con 0x7f49ec110180 2024-09-21T03:47:23.494 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:23.492+0000 7f49f2557640 1 -- 172.21.15.39:0/192770096 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f49ec1029e0 con 0x7f49ec110180 2024-09-21T03:47:23.495 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:23.493+0000 7f49e97fa640 1 -- 172.21.15.39:0/192770096 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11978 (secure 0 0 0) 0x7f49ec1029e0 con 0x7f49ec110180 2024-09-21T03:47:23.495 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T03:47:23.496 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":29,"btime":"2024-09-21T03:46:08:982881+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14790,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6835/508995295","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":508995295},{"type":"v1","addr":"172.21.15.39:6835","nonce":508995295}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14802,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6837/3915912607","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":3915912607},{"type":"v1","addr":"172.21.15.39:6837","nonce":3915912607}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":29,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T03:44:15.009872+0000","modified":"2024-09-21T03:46:08.430113+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24695,"mds_1":24667,"mds_2":24659,"mds_3":24646,"mds_4":14868},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14856":{"gid":14856,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.39:6839/185509192","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6838","nonce":185509192},{"type":"v1","addr":"172.21.15.39:6839","nonce":185509192}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14868":{"gid":14868,"name":"a","rank":4,"incarnation":25,"state":"up:active","state_seq":7,"addr":"172.21.15.39:6841/4247190456","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6840","nonce":4247190456},{"type":"v1","addr":"172.21.15.39:6841","nonce":4247190456}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24599":{"gid":24599,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.150:6833/3114643217","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":3114643217},{"type":"v1","addr":"172.21.15.150:6833","nonce":3114643217}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24601":{"gid":24601,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.161:6833/2963990654","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":2963990654},{"type":"v1","addr":"172.21.15.161:6833","nonce":2963990654}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24632":{"gid":24632,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.150:6835/724594008","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6834","nonce":724594008},{"type":"v1","addr":"172.21.15.150:6835","nonce":724594008}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24640":{"gid":24640,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.161:6835/3747882110","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":3747882110},{"type":"v1","addr":"172.21.15.161:6835","nonce":3747882110}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24646":{"gid":24646,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":10,"addr":"172.21.15.161:6837/3451544597","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6836","nonce":3451544597},{"type":"v1","addr":"172.21.15.161:6837","nonce":3451544597}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24659":{"gid":24659,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.150:6837/4017043144","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6836","nonce":4017043144},{"type":"v1","addr":"172.21.15.150:6837","nonce":4017043144}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24667":{"gid":24667,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.161:6839/2190107270","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6838","nonce":2190107270},{"type":"v1","addr":"172.21.15.161:6839","nonce":2190107270}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24695":{"gid":24695,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.150:6839/2949386907","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6838","nonce":2949386907},{"type":"v1","addr":"172.21.15.150:6839","nonce":2949386907}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24695,"qdb_cluster":[24695,24667,24659,24646,14868]},"id":1}]} 2024-09-21T03:47:23.496 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 29 2024-09-21T03:47:23.499 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:23.497+0000 7f49f2557640 1 -- 172.21.15.39:0/192770096 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f49d40b6710 msgr2=0x7f49d40b8c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:47:23.499 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:23.497+0000 7f49f2557640 1 --2- 172.21.15.39:0/192770096 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f49d40b6710 0x7f49d40b8c20 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f49dc00bbe0 tx=0x7f49dc00b390 comp rx=0 tx=0).stop 2024-09-21T03:47:23.499 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:23.497+0000 7f49f2557640 1 -- 172.21.15.39:0/192770096 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f49ec110180 msgr2=0x7f49ec1a1e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:47:23.499 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:23.497+0000 7f49f2557640 1 --2- 172.21.15.39:0/192770096 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f49ec110180 0x7f49ec1a1e80 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f49e400eaa0 tx=0x7f49e400ef60 comp rx=0 tx=0).stop 2024-09-21T03:47:23.500 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:23.497+0000 7f49f2557640 1 -- 172.21.15.39:0/192770096 shutdown_connections 2024-09-21T03:47:23.500 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:23.497+0000 7f49f2557640 1 --2- 172.21.15.39:0/192770096 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f49d40b6710 0x7f49d40b8c20 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:23.500 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:23.497+0000 7f49f2557640 1 --2- 172.21.15.39:0/192770096 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f49ec110180 0x7f49ec1a1e80 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:23.500 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:23.497+0000 7f49f2557640 1 --2- 172.21.15.39:0/192770096 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f49ec102f20 0x7f49ec19cbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:23.500 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:23.497+0000 7f49f2557640 1 --2- 172.21.15.39:0/192770096 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f49ec1025d0 0x7f49ec19c640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:23.500 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:23.497+0000 7f49f2557640 1 -- 172.21.15.39:0/192770096 >> 172.21.15.39:0/192770096 conn(0x7f49ec079730 msgr2=0x7f49ec111600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:47:23.500 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:23.497+0000 7f49f2557640 1 -- 172.21.15.39:0/192770096 shutdown_connections 2024-09-21T03:47:23.500 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:23.497+0000 7f49f2557640 1 -- 172.21.15.39:0/192770096 wait complete. 2024-09-21T03:47:24.084 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:47:23 smithi039 ceph-mon[31211]: from='client.? 172.21.15.39:0/192770096' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:47:24.218 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:47:23 smithi161 ceph-mon[34533]: from='client.? 172.21.15.39:0/192770096' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:47:24.224 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:47:23 smithi150 ceph-mon[34289]: from='client.? 172.21.15.39:0/192770096' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:47:24.344 INFO:tasks.workunit.client.0.smithi039.stderr:Updating files: 88% (11220/12697) Updating files: 89% (11301/12697) Updating files: 90% (11428/12697) Updating files: 91% (11555/12697) Updating files: 92% (11682/12697) Updating files: 93% (11809/12697) Updating files: 94% (11936/12697) Updating files: 95% (12063/12697) Updating files: 96% (12190/12697) Updating files: 97% (12317/12697) Updating files: 98% (12444/12697) Updating files: 99% (12571/12697) Updating files: 100% (12697/12697) Updating files: 100% (12697/12697), done. 2024-09-21T03:47:24.404 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 51116d5c-77ca-11ef-baf5-efdc52797490 -- ceph tell mds.1:4 damage ls 2024-09-21T03:47:24.711 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/51116d5c-77ca-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:47:24.982 INFO:tasks.workunit.client.0.smithi039.stderr:Note: switching to '1843221895adccfb3b692060251dd6a0183bc410'. 2024-09-21T03:47:24.982 INFO:tasks.workunit.client.0.smithi039.stderr: 2024-09-21T03:47:24.982 INFO:tasks.workunit.client.0.smithi039.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2024-09-21T03:47:24.982 INFO:tasks.workunit.client.0.smithi039.stderr:changes and commit them, and you can discard any commits you make in this 2024-09-21T03:47:24.983 INFO:tasks.workunit.client.0.smithi039.stderr:state without impacting any branches by switching back to a branch. 2024-09-21T03:47:24.983 INFO:tasks.workunit.client.0.smithi039.stderr: 2024-09-21T03:47:24.983 INFO:tasks.workunit.client.0.smithi039.stderr:If you want to create a new branch to retain commits you create, you may 2024-09-21T03:47:24.983 INFO:tasks.workunit.client.0.smithi039.stderr:do so (now or later) by using -c with the switch command. Example: 2024-09-21T03:47:24.983 INFO:tasks.workunit.client.0.smithi039.stderr: 2024-09-21T03:47:24.983 INFO:tasks.workunit.client.0.smithi039.stderr: git switch -c 2024-09-21T03:47:24.983 INFO:tasks.workunit.client.0.smithi039.stderr: 2024-09-21T03:47:24.983 INFO:tasks.workunit.client.0.smithi039.stderr:Or undo this operation with: 2024-09-21T03:47:24.983 INFO:tasks.workunit.client.0.smithi039.stderr: 2024-09-21T03:47:24.983 INFO:tasks.workunit.client.0.smithi039.stderr: git switch - 2024-09-21T03:47:24.983 INFO:tasks.workunit.client.0.smithi039.stderr: 2024-09-21T03:47:24.983 INFO:tasks.workunit.client.0.smithi039.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2024-09-21T03:47:24.983 INFO:tasks.workunit.client.0.smithi039.stderr: 2024-09-21T03:47:24.983 INFO:tasks.workunit.client.0.smithi039.stderr:HEAD is now at 1843221895a Merge PR #56776 into wip-pdonnell-testing-20240920.212106-debug 2024-09-21T03:47:24.988 DEBUG:teuthology.orchestra.run.smithi039:> cd -- /home/ubuntu/cephtest/clone.client.0/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.0 2024-09-21T03:47:25.029 INFO:tasks.workunit.client.0.smithi039.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2024-09-21T03:47:25.031 INFO:tasks.workunit.client.0.smithi039.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-09-21T03:47:25.032 INFO:tasks.workunit.client.0.smithi039.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2024-09-21T03:47:25.041 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:47:24 smithi039 ceph-mon[31211]: pgmap v366: 97 pgs: 97 active+clean; 1.4 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 170 B/s wr, 9 op/s 2024-09-21T03:47:25.041 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:47:24 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:47:25.041 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:47:24 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:47:25.041 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:47:24 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:47:25.041 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:47:24 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:47:25.041 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:47:24 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:47:25.218 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:47:24 smithi161 ceph-mon[34533]: pgmap v366: 97 pgs: 97 active+clean; 1.4 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 170 B/s wr, 9 op/s 2024-09-21T03:47:25.218 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:47:24 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:47:25.218 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:47:24 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:47:25.218 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:47:24 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:47:25.218 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:47:24 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:47:25.218 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:47:24 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:47:25.224 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:47:24 smithi150 ceph-mon[34289]: pgmap v366: 97 pgs: 97 active+clean; 1.4 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 170 B/s wr, 9 op/s 2024-09-21T03:47:25.224 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:47:24 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:47:25.225 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:47:24 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:47:25.225 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:47:24 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:47:25.225 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:47:24 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:47:25.225 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:47:24 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:47:25.406 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:25.327+0000 7fd5bc393640 1 -- 172.21.15.39:0/3633190867 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd5b41083d0 msgr2=0x7fd5b41087e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:47:25.406 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:25.327+0000 7fd5bc393640 1 --2- 172.21.15.39:0/3633190867 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd5b41083d0 0x7fd5b41087e0 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7fd5a4009900 tx=0x7fd5a402a410 comp rx=0 tx=0).stop 2024-09-21T03:47:25.406 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:25.327+0000 7fd5bc393640 1 -- 172.21.15.39:0/3633190867 shutdown_connections 2024-09-21T03:47:25.406 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:25.327+0000 7fd5bc393640 1 --2- 172.21.15.39:0/3633190867 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd5b4108d20 0x7fd5b410f550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:25.406 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:25.327+0000 7fd5bc393640 1 --2- 172.21.15.39:0/3633190867 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd5b41083d0 0x7fd5b41087e0 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:25.406 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:25.327+0000 7fd5bc393640 1 --2- 172.21.15.39:0/3633190867 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd5b4102ae0 0x7fd5b4102ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:25.406 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:25.327+0000 7fd5bc393640 1 -- 172.21.15.39:0/3633190867 >> 172.21.15.39:0/3633190867 conn(0x7fd5b4079760 msgr2=0x7fd5b4079b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:47:25.407 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:25.327+0000 7fd5bc393640 1 -- 172.21.15.39:0/3633190867 shutdown_connections 2024-09-21T03:47:25.407 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:25.327+0000 7fd5bc393640 1 -- 172.21.15.39:0/3633190867 wait complete. 2024-09-21T03:47:25.407 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:25.328+0000 7fd5bc393640 1 Processor -- start 2024-09-21T03:47:25.407 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:25.328+0000 7fd5bc393640 1 -- start start 2024-09-21T03:47:25.407 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:25.329+0000 7fd5bc393640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd5b4102ae0 0x7fd5b419a520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:47:25.407 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:25.329+0000 7fd5bc393640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd5b41083d0 0x7fd5b419aac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:47:25.407 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:25.329+0000 7fd5bc393640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd5b4108d20 0x7fd5b419fd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:47:25.407 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:25.329+0000 7fd5bc393640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fd5b4113ab0 con 0x7fd5b4102ae0 2024-09-21T03:47:25.407 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:25.329+0000 7fd5bc393640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fd5b4113930 con 0x7fd5b4108d20 2024-09-21T03:47:25.407 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:25.329+0000 7fd5bc393640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fd5b4113c30 con 0x7fd5b41083d0 2024-09-21T03:47:25.407 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:25.329+0000 7fd5ba12f640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd5b4102ae0 0x7fd5b419a520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:47:25.407 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:25.329+0000 7fd5ba12f640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd5b4102ae0 0x7fd5b419a520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:38728/0 (socket says 172.21.15.39:38728) 2024-09-21T03:47:25.407 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:25.329+0000 7fd5ba12f640 1 -- 172.21.15.39:0/786730333 learned_addr learned my addr 172.21.15.39:0/786730333 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:47:25.407 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:25.329+0000 7fd5b992e640 1 --2- 172.21.15.39:0/786730333 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd5b41083d0 0x7fd5b419aac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:47:25.407 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:25.329+0000 7fd5ba930640 1 --2- 172.21.15.39:0/786730333 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd5b4108d20 0x7fd5b419fd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:47:25.408 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:25.329+0000 7fd5ba12f640 1 -- 172.21.15.39:0/786730333 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd5b41083d0 msgr2=0x7fd5b419aac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:47:25.408 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:25.329+0000 7fd5ba12f640 1 --2- 172.21.15.39:0/786730333 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd5b41083d0 0x7fd5b419aac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:25.408 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:25.329+0000 7fd5ba12f640 1 -- 172.21.15.39:0/786730333 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd5b4108d20 msgr2=0x7fd5b419fd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:47:25.408 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:25.329+0000 7fd5ba12f640 1 --2- 172.21.15.39:0/786730333 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd5b4108d20 0x7fd5b419fd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:25.408 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:25.329+0000 7fd5ba12f640 1 -- 172.21.15.39:0/786730333 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd5b41a03d0 con 0x7fd5b4102ae0 2024-09-21T03:47:25.408 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:25.330+0000 7fd5ba12f640 1 --2- 172.21.15.39:0/786730333 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd5b4102ae0 0x7fd5b419a520 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7fd5b000d950 tx=0x7fd5b000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:47:25.408 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:25.330+0000 7fd5ab7fe640 1 -- 172.21.15.39:0/786730333 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5b0017030 con 0x7fd5b4102ae0 2024-09-21T03:47:25.408 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:25.330+0000 7fd5ab7fe640 1 -- 172.21.15.39:0/786730333 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd5b0004020 con 0x7fd5b4102ae0 2024-09-21T03:47:25.408 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:25.330+0000 7fd5bc393640 1 -- 172.21.15.39:0/786730333 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd5b41a06c0 con 0x7fd5b4102ae0 2024-09-21T03:47:25.408 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:25.330+0000 7fd5ab7fe640 1 -- 172.21.15.39:0/786730333 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5b00047a0 con 0x7fd5b4102ae0 2024-09-21T03:47:25.408 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:25.330+0000 7fd5bc393640 1 -- 172.21.15.39:0/786730333 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd5b41a6ee0 con 0x7fd5b4102ae0 2024-09-21T03:47:25.408 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:25.332+0000 7fd5c2907740 1 Processor -- start 2024-09-21T03:47:25.408 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:25.333+0000 7fd5c2907740 1 -- start start 2024-09-21T03:47:25.408 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:25.333+0000 7fd5ab7fe640 1 -- 172.21.15.39:0/786730333 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 26) ==== 154694+0+0 (secure 0 0 0) 0x7fd5b0009a30 con 0x7fd5b4102ae0 2024-09-21T03:47:25.409 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:25.333+0000 7fd5c2907740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56419b6c7070 0x56419ba6e6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:47:25.409 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:25.333+0000 7fd5c2907740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56419b69e1b0 0x56419ba6ebe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:47:25.409 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:25.333+0000 7fd5c2907740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56419b6a6390 0x56419ba6f120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:47:25.409 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:25.333+0000 7fd5c2907740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x56419b9e4ea0 con 0x56419b6a6390 2024-09-21T03:47:25.409 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:25.333+0000 7fd5c2907740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x56419b9d42d0 con 0x56419b6c7070 2024-09-21T03:47:25.409 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:25.333+0000 7fd5c2907740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x56419b9d80e0 con 0x56419b69e1b0 2024-09-21T03:47:25.409 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:25.333+0000 7fd5b992e640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56419b6a6390 0x56419ba6f120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:47:25.409 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:25.333+0000 7fd5b992e640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56419b6a6390 0x56419ba6f120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:38742/0 (socket says 172.21.15.39:38742) 2024-09-21T03:47:25.409 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:25.333+0000 7fd5b992e640 1 -- 172.21.15.39:0/2836696197 learned_addr learned my addr 172.21.15.39:0/2836696197 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:47:25.409 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:25.333+0000 7fd5ba930640 1 --2- 172.21.15.39:0/2836696197 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56419b6c7070 0x56419ba6e6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:47:25.409 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:25.333+0000 7fd5ba12f640 1 --2- 172.21.15.39:0/2836696197 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56419b69e1b0 0x56419ba6ebe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:47:25.409 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:25.334+0000 7fd5ba930640 1 -- 172.21.15.39:0/2836696197 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56419b69e1b0 msgr2=0x56419ba6ebe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:47:25.409 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:25.334+0000 7fd5ba930640 1 --2- 172.21.15.39:0/2836696197 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56419b69e1b0 0x56419ba6ebe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:25.409 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:25.334+0000 7fd5ba930640 1 -- 172.21.15.39:0/2836696197 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56419b6a6390 msgr2=0x56419ba6f120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:47:25.410 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:25.334+0000 7fd5ba930640 1 --2- 172.21.15.39:0/2836696197 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56419b6a6390 0x56419ba6f120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:25.410 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:25.334+0000 7fd5ba930640 1 -- 172.21.15.39:0/2836696197 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56419b732650 con 0x56419b6c7070 2024-09-21T03:47:25.410 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:25.334+0000 7fd5ba12f640 1 --2- 172.21.15.39:0/2836696197 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56419b69e1b0 0x56419ba6ebe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:47:25.410 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:25.334+0000 7fd5b992e640 1 --2- 172.21.15.39:0/2836696197 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56419b6a6390 0x56419ba6f120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:47:25.410 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:25.334+0000 7fd5ab7fe640 1 --2- 172.21.15.39:0/786730333 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7fd5a00b6740 0x7fd5a00b8c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:47:25.410 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:25.334+0000 7fd5ba930640 1 --2- 172.21.15.39:0/2836696197 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56419b6c7070 0x56419ba6e6a0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fd5ac007dc0 tx=0x7fd5ac00f790 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e6748fd9c3519fe5 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:47:25.410 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:25.334+0000 7fd5ba12f640 1 --2- 172.21.15.39:0/786730333 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7fd5a00b6740 0x7fd5a00b8c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:47:25.410 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:25.334+0000 7fd5a8d9a640 1 -- 172.21.15.39:0/2836696197 <== mon.2 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5ac00fea0 con 0x56419b6c7070 2024-09-21T03:47:25.410 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:25.334+0000 7fd5a8d9a640 1 -- 172.21.15.39:0/2836696197 <== mon.2 v2:172.21.15.150:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd5ac003b70 con 0x56419b6c7070 2024-09-21T03:47:25.410 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:25.334+0000 7fd5a8d9a640 1 -- 172.21.15.39:0/2836696197 <== mon.2 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5ac0108f0 con 0x56419b6c7070 2024-09-21T03:47:25.410 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:25.334+0000 7fd5ab7fe640 1 -- 172.21.15.39:0/786730333 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8484+0+0 (secure 0 0 0) 0x7fd5b00e7240 con 0x7fd5b4102ae0 2024-09-21T03:47:25.411 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:25.335+0000 7fd5c2907740 1 -- 172.21.15.39:0/2836696197 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56419b6c7070 msgr2=0x56419ba6e6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:47:25.411 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:25.335+0000 7fd5ba12f640 1 --2- 172.21.15.39:0/786730333 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7fd5a00b6740 0x7fd5a00b8c50 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fd5b00e4950 tx=0x7fd5b00585f0 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:47:25.411 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:25.335+0000 7fd5c2907740 1 --2- 172.21.15.39:0/2836696197 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56419b6c7070 0x56419ba6e6a0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fd5ac007dc0 tx=0x7fd5ac00f790 comp rx=0 tx=0).stop 2024-09-21T03:47:25.411 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:25.335+0000 7fd5c2907740 1 -- 172.21.15.39:0/2836696197 shutdown_connections 2024-09-21T03:47:25.411 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:25.335+0000 7fd5c2907740 1 --2- 172.21.15.39:0/2836696197 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56419b6a6390 0x56419ba6f120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:25.411 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:25.335+0000 7fd5c2907740 1 --2- 172.21.15.39:0/2836696197 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56419b69e1b0 0x56419ba6ebe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:25.411 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:25.335+0000 7fd5c2907740 1 --2- 172.21.15.39:0/2836696197 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56419b6c7070 0x56419ba6e6a0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:25.411 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:25.335+0000 7fd5c2907740 1 -- 172.21.15.39:0/2836696197 >> 172.21.15.39:0/2836696197 conn(0x56419b6bb460 msgr2=0x56419ba6c590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:47:25.411 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:25.335+0000 7fd5c2907740 1 -- 172.21.15.39:0/2836696197 shutdown_connections 2024-09-21T03:47:25.411 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:25.335+0000 7fd5c2907740 1 -- 172.21.15.39:0/2836696197 wait complete. 2024-09-21T03:47:25.411 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:25.336+0000 7fd5c2907740 1 -- start start 2024-09-21T03:47:25.411 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:25.336+0000 7fd5c2907740 1 Processor -- start 2024-09-21T03:47:25.411 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:25.336+0000 7fd5c2907740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56419b69e1b0 0x56419ba7e0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:47:25.411 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:25.336+0000 7fd5c2907740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56419b6a6390 0x56419ba7e620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:47:25.412 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:25.336+0000 7fd5c2907740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56419b6c7070 0x56419ba7eb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:47:25.412 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:25.336+0000 7fd5c2907740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x56419b835700 con 0x56419b6c7070 2024-09-21T03:47:25.412 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:25.336+0000 7fd5c2907740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x56419b96f160 con 0x56419b69e1b0 2024-09-21T03:47:25.412 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:25.336+0000 7fd5c2907740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x56419ba81180 con 0x56419b6a6390 2024-09-21T03:47:25.412 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:25.336+0000 7fd5c2907740 10 client.0 _ll_register_callbacks cb 0x56419b9889e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:47:25.412 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:25.336+0000 7fd5ba930640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56419b6c7070 0x56419ba7eb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:47:25.412 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:25.336+0000 7fd5ba930640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56419b6c7070 0x56419ba7eb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:38744/0 (socket says 172.21.15.39:38744) 2024-09-21T03:47:25.412 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:25.336+0000 7fd5ba930640 1 -- 172.21.15.39:0/2845707766 learned_addr learned my addr 172.21.15.39:0/2845707766 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:47:25.412 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:25.336+0000 7fd5b992e640 1 --2- 172.21.15.39:0/2845707766 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56419b6a6390 0x56419ba7e620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:47:25.412 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:25.336+0000 7fd5ba930640 1 --2- 172.21.15.39:0/2845707766 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56419b69e1b0 0x56419ba7e0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:47:25.412 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:25.337+0000 7fd5ba930640 1 -- 172.21.15.39:0/2845707766 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56419b6a6390 msgr2=0x56419ba7e620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:47:25.412 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:25.337+0000 7fd5ba930640 1 --2- 172.21.15.39:0/2845707766 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56419b6a6390 0x56419ba7e620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:25.412 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:25.337+0000 7fd5ba930640 1 -- 172.21.15.39:0/2845707766 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56419b69e1b0 msgr2=0x56419ba7e0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:47:25.412 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:25.337+0000 7fd5ba930640 1 --2- 172.21.15.39:0/2845707766 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56419b69e1b0 0x56419ba7e0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:25.412 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:25.337+0000 7fd5ba930640 1 -- 172.21.15.39:0/2845707766 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56419b99cc90 con 0x56419b6c7070 2024-09-21T03:47:25.413 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:25.337+0000 7fd5ba930640 1 --2- 172.21.15.39:0/2845707766 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56419b6c7070 0x56419ba7eb60 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7fd5ac010010 tx=0x7fd5ac0072a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f2d223dbb93188f server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:47:25.413 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:25.337+0000 7fd59dffb640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-09-21T03:47:25.413 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:25.337+0000 7fd59dffb640 1 -- 172.21.15.39:0/2845707766 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5ac004a40 con 0x56419b6c7070 2024-09-21T03:47:25.413 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:25.337+0000 7fd59dffb640 1 -- 172.21.15.39:0/2845707766 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8484+0+0 (secure 0 0 0) 0x7fd5ac004be0 con 0x56419b6c7070 2024-09-21T03:47:25.413 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:25.337+0000 7fd5c2907740 1 -- 172.21.15.39:0/2845707766 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd5ac003b70 con 0x56419b6c7070 2024-09-21T03:47:25.413 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:25.337+0000 7fd59dffb640 1 -- 172.21.15.39:0/2845707766 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fd5ac004de0 con 0x56419b6c7070 2024-09-21T03:47:25.413 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:25.337+0000 7fd5c2907740 10 client.15903 fetch_fsmap learned FSMap version 29 2024-09-21T03:47:25.413 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:25.337+0000 7fd5c2907740 1 -- 172.21.15.39:0/2845707766 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x56419ba8cdf0 con 0x56419b6c7070 2024-09-21T03:47:25.413 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:25.337+0000 7fd59dffb640 1 -- 172.21.15.39:0/2845707766 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd5ac003b70 con 0x56419b6c7070 2024-09-21T03:47:25.413 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:25.338+0000 7fd59dffb640 1 -- 172.21.15.39:0/2845707766 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fd5ac02d020 con 0x56419b6c7070 2024-09-21T03:47:25.413 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:25.338+0000 7fd5c2907740 10 client.15903 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T03:47:25.413 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:25.338+0000 7fd5c2907740 10 client.15903 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.a 2024-09-21T03:47:25.413 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:25.338+0000 7fd5c2907740 20 client.15903 populate_metadata read hostname 'smithi039' 2024-09-21T03:47:25.413 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:25.338+0000 7fd5c2907740 1 --2- 172.21.15.39:0/2845707766 >> [v2:172.21.15.39:6840/4247190456,v1:172.21.15.39:6841/4247190456] conn(0x56419b6859f0 0x56419ba90200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:47:25.413 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:25.338+0000 7fd5c2907740 4 client.15903 mds_command: new command op to 14868 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:47:25.413 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:25.338+0000 7fd5c2907740 1 -- 172.21.15.39:0/2845707766 --> [v2:172.21.15.39:6840/4247190456,v1:172.21.15.39:6841/4247190456] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56419b9e6df0 con 0x56419b6859f0 2024-09-21T03:47:25.414 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:25.338+0000 7fd5b992e640 1 --2- 172.21.15.39:0/2845707766 >> [v2:172.21.15.39:6840/4247190456,v1:172.21.15.39:6841/4247190456] conn(0x56419b6859f0 0x56419ba90200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:47:25.414 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:25.338+0000 7fd5b992e640 1 --2- 172.21.15.39:0/2845707766 >> [v2:172.21.15.39:6840/4247190456,v1:172.21.15.39:6841/4247190456] conn(0x56419b6859f0 0x56419ba90200 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=35d7e9a10cf7a611 server_cookie=a264029a9e4083ec in_seq=0 out_seq=0 2024-09-21T03:47:25.414 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:25.338+0000 7fd59dffb640 10 client.15903 ms_handle_connect on v2:172.21.15.39:6840/4247190456 2024-09-21T03:47:25.414 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:25.339+0000 7fd59dffb640 1 -- 172.21.15.39:0/2845707766 <== mds.4 v2:172.21.15.39:6840/4247190456 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x56419b9e6df0 con 0x56419b6859f0 2024-09-21T03:47:25.414 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:25.339+0000 7fd59dffb640 10 client.15903 handle_command_reply: tid=0 2024-09-21T03:47:25.414 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:25.339+0000 7fd5c2907740 1 client.15903 shutdown 2024-09-21T03:47:25.414 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:25.339+0000 7fd5c2907740 1 -- 172.21.15.39:0/2845707766 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56419b6c7070 msgr2=0x56419ba7eb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:47:25.414 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:25.339+0000 7fd5c2907740 1 --2- 172.21.15.39:0/2845707766 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56419b6c7070 0x56419ba7eb60 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7fd5ac010010 tx=0x7fd5ac0072a0 comp rx=0 tx=0).stop 2024-09-21T03:47:25.414 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:25.339+0000 7fd5c2907740 1 -- 172.21.15.39:0/2845707766 shutdown_connections 2024-09-21T03:47:25.414 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:25.339+0000 7fd5c2907740 1 --2- 172.21.15.39:0/2845707766 >> [v2:172.21.15.39:6840/4247190456,v1:172.21.15.39:6841/4247190456] conn(0x56419b6859f0 0x56419ba90200 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:25.414 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:25.339+0000 7fd59dffb640 0 client.15903 ms_handle_reset on v2:172.21.15.39:6840/4247190456 2024-09-21T03:47:25.414 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:25.339+0000 7fd5c2907740 1 --2- 172.21.15.39:0/2845707766 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56419b6c7070 0x56419ba7eb60 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:25.414 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:25.339+0000 7fd5c2907740 1 --2- 172.21.15.39:0/2845707766 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56419b6a6390 0x56419ba7e620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:25.414 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:25.339+0000 7fd5c2907740 1 --2- 172.21.15.39:0/2845707766 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56419b69e1b0 0x56419ba7e0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:25.415 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:25.339+0000 7fd5c2907740 1 -- 172.21.15.39:0/2845707766 >> 172.21.15.39:0/2845707766 conn(0x56419b6bb460 msgr2=0x56419ba6abb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:47:25.415 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:25.339+0000 7fd5c2907740 1 -- 172.21.15.39:0/2845707766 shutdown_connections 2024-09-21T03:47:25.415 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:25.339+0000 7fd5c2907740 1 -- 172.21.15.39:0/2845707766 wait complete. 2024-09-21T03:47:25.415 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:25.339+0000 7fd5c2907740 20 client.15903 trim_cache size 0 max 16384 2024-09-21T03:47:25.415 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:25.353+0000 7fd5c2907740 1 Processor -- start 2024-09-21T03:47:25.415 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:25.353+0000 7fd5c2907740 1 -- start start 2024-09-21T03:47:25.415 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:25.353+0000 7fd5c2907740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56419b6859f0 0x56419ba70090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:47:25.415 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:25.353+0000 7fd5c2907740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56419b69e1b0 0x56419ba7fd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:47:25.415 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:25.353+0000 7fd5c2907740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56419b6a6390 0x56419ba6bbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:47:25.415 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:25.353+0000 7fd5c2907740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fd5b41ad560 con 0x56419b6a6390 2024-09-21T03:47:25.415 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:25.353+0000 7fd5c2907740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fd5b41af0a0 con 0x56419b6859f0 2024-09-21T03:47:25.415 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:25.353+0000 7fd5c2907740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fd5b41adc30 con 0x56419b69e1b0 2024-09-21T03:47:25.416 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:25.353+0000 7fd5ba930640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56419b6a6390 0x56419ba6bbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:47:25.416 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:25.353+0000 7fd5ba930640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56419b6a6390 0x56419ba6bbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:38752/0 (socket says 172.21.15.39:38752) 2024-09-21T03:47:25.416 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:25.353+0000 7fd5ba930640 1 -- 172.21.15.39:0/2391730034 learned_addr learned my addr 172.21.15.39:0/2391730034 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:47:25.416 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:25.353+0000 7fd5ba930640 1 --2- 172.21.15.39:0/2391730034 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56419b6859f0 0x56419ba70090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:47:25.416 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:25.353+0000 7fd5b992e640 1 --2- 172.21.15.39:0/2391730034 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56419b69e1b0 0x56419ba7fd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:47:25.416 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:25.354+0000 7fd5ba930640 1 -- 172.21.15.39:0/2391730034 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56419b69e1b0 msgr2=0x56419ba7fd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:47:25.416 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:25.354+0000 7fd5ba930640 1 --2- 172.21.15.39:0/2391730034 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56419b69e1b0 0x56419ba7fd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:25.416 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:25.354+0000 7fd5ba930640 1 -- 172.21.15.39:0/2391730034 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56419b6859f0 msgr2=0x56419ba70090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:47:25.416 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:25.354+0000 7fd5ba930640 1 --2- 172.21.15.39:0/2391730034 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56419b6859f0 0x56419ba70090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:25.416 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:25.354+0000 7fd5ba930640 1 -- 172.21.15.39:0/2391730034 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56419ba7c5e0 con 0x56419b6a6390 2024-09-21T03:47:25.416 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:25.354+0000 7fd5b992e640 1 --2- 172.21.15.39:0/2391730034 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56419b69e1b0 0x56419ba7fd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:47:25.416 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:25.354+0000 7fd5ba930640 1 --2- 172.21.15.39:0/2391730034 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56419b6a6390 0x56419ba6bbc0 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7fd5ac0076b0 tx=0x7fd5ac00f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=705f427c3a90675a server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:47:25.416 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:25.354+0000 7fd59ffff640 1 -- 172.21.15.39:0/2391730034 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5ac003b30 con 0x56419b6a6390 2024-09-21T03:47:25.416 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:25.354+0000 7fd59ffff640 1 -- 172.21.15.39:0/2391730034 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd5ac035030 con 0x56419b6a6390 2024-09-21T03:47:25.416 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:25.354+0000 7fd59ffff640 1 -- 172.21.15.39:0/2391730034 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5ac0049c0 con 0x56419b6a6390 2024-09-21T03:47:25.417 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:25.355+0000 7fd5c2907740 1 -- 172.21.15.39:0/2391730034 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56419b6a6390 msgr2=0x56419ba6bbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:47:25.417 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:25.355+0000 7fd5c2907740 1 --2- 172.21.15.39:0/2391730034 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56419b6a6390 0x56419ba6bbc0 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7fd5ac0076b0 tx=0x7fd5ac00f790 comp rx=0 tx=0).stop 2024-09-21T03:47:25.417 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:25.355+0000 7fd5c2907740 1 -- 172.21.15.39:0/2391730034 shutdown_connections 2024-09-21T03:47:25.417 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:25.355+0000 7fd5c2907740 1 --2- 172.21.15.39:0/2391730034 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56419b6a6390 0x56419ba6bbc0 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:25.417 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:25.355+0000 7fd5c2907740 1 --2- 172.21.15.39:0/2391730034 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56419b69e1b0 0x56419ba7fd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:25.417 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:25.355+0000 7fd5c2907740 1 --2- 172.21.15.39:0/2391730034 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56419b6859f0 0x56419ba70090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:25.417 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:25.355+0000 7fd5c2907740 1 -- 172.21.15.39:0/2391730034 >> 172.21.15.39:0/2391730034 conn(0x56419b6bb460 msgr2=0x56419ba7f600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:47:25.417 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:25.355+0000 7fd5c2907740 1 -- 172.21.15.39:0/2391730034 shutdown_connections 2024-09-21T03:47:25.417 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:25.355+0000 7fd5c2907740 1 -- 172.21.15.39:0/2391730034 wait complete. 2024-09-21T03:47:25.417 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:25.355+0000 7fd5c2907740 1 -- start start 2024-09-21T03:47:25.417 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:25.355+0000 7fd5c2907740 1 Processor -- start 2024-09-21T03:47:25.417 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:25.356+0000 7fd5c2907740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56419b6859f0 0x56419ba93b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:47:25.417 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:25.356+0000 7fd5c2907740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56419b69e1b0 0x56419ba90fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:47:25.417 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:25.356+0000 7fd5c2907740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56419b6a6390 0x56419ba91620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:47:25.418 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:25.356+0000 7fd5c2907740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fd5b41ae9d0 con 0x56419b69e1b0 2024-09-21T03:47:25.418 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:25.356+0000 7fd5c2907740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fd5b41ae300 con 0x56419b6a6390 2024-09-21T03:47:25.418 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:25.356+0000 7fd5c2907740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fd5b41ab540 con 0x56419b6859f0 2024-09-21T03:47:25.418 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:25.356+0000 7fd5c2907740 10 client.0 _ll_register_callbacks cb 0x7fd5a4009640 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:47:25.418 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:25.356+0000 7fd5b992e640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56419b69e1b0 0x56419ba90fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:47:25.418 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:25.356+0000 7fd5b992e640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56419b69e1b0 0x56419ba90fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:38756/0 (socket says 172.21.15.39:38756) 2024-09-21T03:47:25.418 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:25.356+0000 7fd5b992e640 1 -- 172.21.15.39:0/2980824457 learned_addr learned my addr 172.21.15.39:0/2980824457 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:47:25.418 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:25.356+0000 7fd5ba930640 1 --2- 172.21.15.39:0/2980824457 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56419b6a6390 0x56419ba91620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:47:25.418 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:25.356+0000 7fd5ba930640 1 --2- 172.21.15.39:0/2980824457 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56419b6859f0 0x56419ba93b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:47:25.418 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:25.356+0000 7fd5b992e640 1 -- 172.21.15.39:0/2980824457 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56419b6859f0 msgr2=0x56419ba93b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:47:25.418 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:25.356+0000 7fd5b992e640 1 --2- 172.21.15.39:0/2980824457 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56419b6859f0 0x56419ba93b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:25.418 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:25.356+0000 7fd5b992e640 1 -- 172.21.15.39:0/2980824457 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56419b6a6390 msgr2=0x56419ba91620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:47:25.418 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:25.356+0000 7fd5b992e640 1 --2- 172.21.15.39:0/2980824457 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56419b6a6390 0x56419ba91620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:25.418 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:25.356+0000 7fd5b992e640 1 -- 172.21.15.39:0/2980824457 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56419ba91d90 con 0x56419b69e1b0 2024-09-21T03:47:25.418 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:25.356+0000 7fd5ba930640 1 --2- 172.21.15.39:0/2980824457 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56419b6859f0 0x56419ba93b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:47:25.419 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:25.356+0000 7fd5b992e640 1 --2- 172.21.15.39:0/2980824457 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56419b69e1b0 0x56419ba90fd0 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7fd5b406b680 tx=0x7fd5a40042a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=50c04023b8660e30 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:47:25.419 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:25.356+0000 7fd59dffb640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-09-21T03:47:25.419 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:25.356+0000 7fd59dffb640 1 -- 172.21.15.39:0/2980824457 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5a4035040 con 0x56419b69e1b0 2024-09-21T03:47:25.419 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:25.356+0000 7fd59dffb640 1 -- 172.21.15.39:0/2980824457 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8484+0+0 (secure 0 0 0) 0x7fd5a4005240 con 0x56419b69e1b0 2024-09-21T03:47:25.419 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:25.357+0000 7fd5c2907740 1 -- 172.21.15.39:0/2980824457 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd5ac035030 con 0x56419b69e1b0 2024-09-21T03:47:25.419 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:25.357+0000 7fd59dffb640 1 -- 172.21.15.39:0/2980824457 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fd5a4005440 con 0x56419b69e1b0 2024-09-21T03:47:25.419 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:25.357+0000 7fd5c2907740 10 client.15909 fetch_fsmap learned FSMap version 29 2024-09-21T03:47:25.419 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:25.357+0000 7fd5c2907740 1 -- 172.21.15.39:0/2980824457 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x56419bafaa20 con 0x56419b69e1b0 2024-09-21T03:47:25.419 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:25.357+0000 7fd59dffb640 1 -- 172.21.15.39:0/2980824457 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd5ac035030 con 0x56419b69e1b0 2024-09-21T03:47:25.419 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:25.357+0000 7fd59dffb640 1 -- 172.21.15.39:0/2980824457 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fd5a4055020 con 0x56419b69e1b0 2024-09-21T03:47:25.419 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:25.357+0000 7fd5c2907740 10 client.15909 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T03:47:25.419 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:25.357+0000 7fd5c2907740 10 client.15909 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.a 2024-09-21T03:47:25.419 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:25.357+0000 7fd5c2907740 20 client.15909 populate_metadata read hostname 'smithi039' 2024-09-21T03:47:25.419 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:25.357+0000 7fd5c2907740 1 --2- 172.21.15.39:0/2980824457 >> [v2:172.21.15.39:6840/4247190456,v1:172.21.15.39:6841/4247190456] conn(0x56419b6c7070 0x56419bafd100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:47:25.420 INFO:teuthology.orchestra.run.smithi039.stdout:[] 2024-09-21T03:47:25.420 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:25.357+0000 7fd5c2907740 4 client.15909 mds_command: new command op to 14868 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T03:47:25.420 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:25.357+0000 7fd5c2907740 1 -- 172.21.15.39:0/2980824457 --> [v2:172.21.15.39:6840/4247190456,v1:172.21.15.39:6841/4247190456] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56419b9e28a0 con 0x56419b6c7070 2024-09-21T03:47:25.420 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:25.357+0000 7fd5ba930640 1 --2- 172.21.15.39:0/2980824457 >> [v2:172.21.15.39:6840/4247190456,v1:172.21.15.39:6841/4247190456] conn(0x56419b6c7070 0x56419bafd100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:47:25.420 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:25.358+0000 7fd5ba930640 1 --2- 172.21.15.39:0/2980824457 >> [v2:172.21.15.39:6840/4247190456,v1:172.21.15.39:6841/4247190456] conn(0x56419b6c7070 0x56419bafd100 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=de4bffdc25e93fc0 server_cookie=a58356758d37d56e in_seq=0 out_seq=0 2024-09-21T03:47:25.420 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:25.358+0000 7fd59dffb640 10 client.15909 ms_handle_connect on v2:172.21.15.39:6840/4247190456 2024-09-21T03:47:25.420 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:25.358+0000 7fd59dffb640 1 -- 172.21.15.39:0/2980824457 <== mds.4 v2:172.21.15.39:6840/4247190456 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56419b9e28a0 con 0x56419b6c7070 2024-09-21T03:47:25.420 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:25.358+0000 7fd59dffb640 10 client.15909 handle_command_reply: tid=0 2024-09-21T03:47:25.421 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:25.358+0000 7fd5c2907740 1 client.15909 shutdown 2024-09-21T03:47:25.421 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:25.358+0000 7fd5c2907740 1 -- 172.21.15.39:0/2980824457 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56419b69e1b0 msgr2=0x56419ba90fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:47:25.421 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:25.358+0000 7fd5c2907740 1 --2- 172.21.15.39:0/2980824457 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56419b69e1b0 0x56419ba90fd0 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7fd5b406b680 tx=0x7fd5a40042a0 comp rx=0 tx=0).stop 2024-09-21T03:47:25.421 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:25.358+0000 7fd5c2907740 1 -- 172.21.15.39:0/2980824457 shutdown_connections 2024-09-21T03:47:25.421 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:25.358+0000 7fd5c2907740 1 --2- 172.21.15.39:0/2980824457 >> [v2:172.21.15.39:6840/4247190456,v1:172.21.15.39:6841/4247190456] conn(0x56419b6c7070 0x56419bafd100 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:25.421 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:25.358+0000 7fd5c2907740 1 --2- 172.21.15.39:0/2980824457 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56419b6a6390 0x56419ba91620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:25.421 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:25.358+0000 7fd59dffb640 0 client.15909 ms_handle_reset on v2:172.21.15.39:6840/4247190456 2024-09-21T03:47:25.421 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:25.358+0000 7fd5c2907740 1 --2- 172.21.15.39:0/2980824457 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56419b69e1b0 0x56419ba90fd0 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:25.421 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:25.358+0000 7fd5c2907740 1 --2- 172.21.15.39:0/2980824457 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56419b6859f0 0x56419ba93b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:25.421 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:25.358+0000 7fd5c2907740 1 -- 172.21.15.39:0/2980824457 >> 172.21.15.39:0/2980824457 conn(0x56419b6bb460 msgr2=0x56419ba6fe60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:47:25.421 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:25.358+0000 7fd5c2907740 1 -- 172.21.15.39:0/2980824457 shutdown_connections 2024-09-21T03:47:25.421 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:25.358+0000 7fd5c2907740 1 -- 172.21.15.39:0/2980824457 wait complete. 2024-09-21T03:47:25.421 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:25.358+0000 7fd5c2907740 20 client.15909 trim_cache size 0 max 16384 2024-09-21T03:47:25.421 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:25.359+0000 7fd5bc393640 1 -- 172.21.15.39:0/786730333 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7fd5a00b6740 msgr2=0x7fd5a00b8c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:47:25.421 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:25.359+0000 7fd5bc393640 1 --2- 172.21.15.39:0/786730333 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7fd5a00b6740 0x7fd5a00b8c50 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fd5b00e4950 tx=0x7fd5b00585f0 comp rx=0 tx=0).stop 2024-09-21T03:47:25.422 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:25.359+0000 7fd5bc393640 1 -- 172.21.15.39:0/786730333 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd5b4102ae0 msgr2=0x7fd5b419a520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:47:25.422 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:25.359+0000 7fd5bc393640 1 --2- 172.21.15.39:0/786730333 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd5b4102ae0 0x7fd5b419a520 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7fd5b000d950 tx=0x7fd5b000de10 comp rx=0 tx=0).stop 2024-09-21T03:47:25.422 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:25.359+0000 7fd5bc393640 1 -- 172.21.15.39:0/786730333 shutdown_connections 2024-09-21T03:47:25.422 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:25.359+0000 7fd5bc393640 1 --2- 172.21.15.39:0/786730333 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7fd5a00b6740 0x7fd5a00b8c50 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:25.422 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:25.359+0000 7fd5bc393640 1 --2- 172.21.15.39:0/786730333 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd5b4108d20 0x7fd5b419fd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:25.422 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:25.359+0000 7fd5bc393640 1 --2- 172.21.15.39:0/786730333 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd5b41083d0 0x7fd5b419aac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:25.422 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:25.359+0000 7fd5bc393640 1 --2- 172.21.15.39:0/786730333 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd5b4102ae0 0x7fd5b419a520 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:25.422 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:25.359+0000 7fd5bc393640 1 -- 172.21.15.39:0/786730333 >> 172.21.15.39:0/786730333 conn(0x7fd5b4079760 msgr2=0x7fd5b41044f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:47:25.422 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:25.359+0000 7fd5bc393640 1 -- 172.21.15.39:0/786730333 shutdown_connections 2024-09-21T03:47:25.422 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:25.359+0000 7fd5bc393640 1 -- 172.21.15.39:0/786730333 wait complete. 2024-09-21T03:47:26.084 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:47:25 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:47:26.084 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:47:25 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:47:26.084 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:47:25 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:47:26.218 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:47:25 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:47:26.218 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:47:25 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:47:26.218 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:47:25 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:47:26.225 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:47:25 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:47:26.225 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:47:25 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:47:26.225 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:47:25 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:47:26.499 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 51116d5c-77ca-11ef-baf5-efdc52797490 -- ceph tell mds.1:3 damage ls 2024-09-21T03:47:26.764 INFO:tasks.workunit.client.0.smithi039.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2024-09-21T03:47:26.804 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/51116d5c-77ca-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:47:26.826 INFO:tasks.workunit.client.0.smithi039.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2024-09-21T03:47:26.872 INFO:tasks.workunit.client.0.smithi039.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-09-21T03:47:26.873 INFO:tasks.workunit.client.0.smithi039.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-09-21T03:47:26.873 INFO:tasks.workunit.client.0.smithi039.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2024-09-21T03:47:26.923 INFO:tasks.workunit.client.0.smithi039.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-09-21T03:47:26.927 DEBUG:teuthology.orchestra.run.smithi039:> set -ex 2024-09-21T03:47:26.927 DEBUG:teuthology.orchestra.run.smithi039:> dd if=/home/ubuntu/cephtest/workunits.list.client.0 of=/dev/stdout 2024-09-21T03:47:26.943 INFO:tasks.workunit:Running workunits matching suites/iozone.sh on client.0... 2024-09-21T03:47:26.945 INFO:tasks.workunit:Running workunit suites/iozone.sh... 2024-09-21T03:47:26.945 DEBUG:teuthology.orchestra.run.smithi039:workunit test suites/iozone.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=1843221895adccfb3b692060251dd6a0183bc410 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/iozone.sh 2024-09-21T03:47:27.006 INFO:tasks.workunit.client.0.smithi039.stderr:+ iozone -c -e -s 1024M -r 16K -t 1 -F f1 -i 0 -i 1 2024-09-21T03:47:27.115 INFO:tasks.workunit.client.0.smithi039.stdout: Iozone: Performance Test of File I/O 2024-09-21T03:47:27.115 INFO:tasks.workunit.client.0.smithi039.stdout: Version $Revision: 3.494 $ 2024-09-21T03:47:27.115 INFO:tasks.workunit.client.0.smithi039.stdout: Compiled for 64 bit mode. 2024-09-21T03:47:27.115 INFO:tasks.workunit.client.0.smithi039.stdout: Build: linux-AMD64 2024-09-21T03:47:27.115 INFO:tasks.workunit.client.0.smithi039.stdout: 2024-09-21T03:47:27.115 INFO:tasks.workunit.client.0.smithi039.stdout: Contributors:William Norcott, Don Capps, Isom Crawford, Kirby Collins 2024-09-21T03:47:27.116 INFO:tasks.workunit.client.0.smithi039.stdout: Al Slater, Scott Rhine, Mike Wisner, Ken Goss 2024-09-21T03:47:27.116 INFO:tasks.workunit.client.0.smithi039.stdout: Steve Landherr, Brad Smith, Mark Kelly, Dr. Alain CYR, 2024-09-21T03:47:27.116 INFO:tasks.workunit.client.0.smithi039.stdout: Randy Dunlap, Mark Montague, Dan Million, Gavin Brebner, 2024-09-21T03:47:27.116 INFO:tasks.workunit.client.0.smithi039.stdout: Jean-Marc Zucconi, Jeff Blomberg, Benny Halevy, Dave Boone, 2024-09-21T03:47:27.116 INFO:tasks.workunit.client.0.smithi039.stdout: Erik Habbinga, Kris Strecker, Walter Wong, Joshua Root, 2024-09-21T03:47:27.116 INFO:tasks.workunit.client.0.smithi039.stdout: Fabrice Bacchella, Zhenghua Xue, Qin Li, Darren Sawyer, 2024-09-21T03:47:27.116 INFO:tasks.workunit.client.0.smithi039.stdout: Vangel Bojaxhi, Ben England, Vikentsi Lapa, 2024-09-21T03:47:27.116 INFO:tasks.workunit.client.0.smithi039.stdout: Alexey Skidanov, Sudhir Kumar. 2024-09-21T03:47:27.116 INFO:tasks.workunit.client.0.smithi039.stdout: 2024-09-21T03:47:27.116 INFO:tasks.workunit.client.0.smithi039.stdout: Run began: Sat Sep 21 03:47:27 2024 2024-09-21T03:47:27.116 INFO:tasks.workunit.client.0.smithi039.stdout: 2024-09-21T03:47:27.116 INFO:tasks.workunit.client.0.smithi039.stdout: Include close in write timing 2024-09-21T03:47:27.116 INFO:tasks.workunit.client.0.smithi039.stdout: Include fsync in write timing 2024-09-21T03:47:27.116 INFO:tasks.workunit.client.0.smithi039.stdout: File size set to 1048576 kB 2024-09-21T03:47:27.116 INFO:tasks.workunit.client.0.smithi039.stdout: Record Size 16 kB 2024-09-21T03:47:27.117 INFO:tasks.workunit.client.0.smithi039.stdout: Command line used: iozone -c -e -s 1024M -r 16K -t 1 -F f1 -i 0 -i 1 2024-09-21T03:47:27.117 INFO:tasks.workunit.client.0.smithi039.stdout: Output is in kBytes/sec 2024-09-21T03:47:27.117 INFO:tasks.workunit.client.0.smithi039.stdout: Time Resolution = 0.000001 seconds. 2024-09-21T03:47:27.117 INFO:tasks.workunit.client.0.smithi039.stdout: Processor cache size set to 1024 kBytes. 2024-09-21T03:47:27.117 INFO:tasks.workunit.client.0.smithi039.stdout: Processor cache line size set to 32 bytes. 2024-09-21T03:47:27.117 INFO:tasks.workunit.client.0.smithi039.stdout: File stride size set to 17 * record size. 2024-09-21T03:47:27.117 INFO:tasks.workunit.client.0.smithi039.stdout: Throughput test with 1 process 2024-09-21T03:47:27.117 INFO:tasks.workunit.client.0.smithi039.stdout: Each process writes a 1048576 kByte file in 16 kByte records 2024-09-21T03:47:27.217 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:47:26 smithi161 ceph-mon[34533]: pgmap v367: 97 pgs: 97 active+clean; 1.4 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 170 B/s wr, 12 op/s 2024-09-21T03:47:27.218 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:47:26 smithi039 ceph-mon[31211]: pgmap v367: 97 pgs: 97 active+clean; 1.4 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 170 B/s wr, 12 op/s 2024-09-21T03:47:27.224 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:47:26 smithi150 ceph-mon[34289]: pgmap v367: 97 pgs: 97 active+clean; 1.4 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 170 B/s wr, 12 op/s 2024-09-21T03:47:27.327 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:27.316+0000 7f6e0737f640 1 -- 172.21.15.39:0/2975012171 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6e00102ae0 msgr2=0x7f6e00102ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:47:27.328 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:27.316+0000 7f6e0737f640 1 --2- 172.21.15.39:0/2975012171 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6e00102ae0 0x7f6e00102ef0 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f6dfc009900 tx=0x7f6dfc02a410 comp rx=0 tx=0).stop 2024-09-21T03:47:27.328 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:27.316+0000 7f6e0737f640 1 -- 172.21.15.39:0/2975012171 shutdown_connections 2024-09-21T03:47:27.328 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:27.316+0000 7f6e0737f640 1 --2- 172.21.15.39:0/2975012171 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6e00108d20 0x7f6e0010f550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:27.328 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:27.316+0000 7f6e0737f640 1 --2- 172.21.15.39:0/2975012171 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6e001083d0 0x7f6e001087e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:27.328 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:27.316+0000 7f6e0737f640 1 --2- 172.21.15.39:0/2975012171 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6e00102ae0 0x7f6e00102ef0 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:27.328 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:27.316+0000 7f6e0737f640 1 -- 172.21.15.39:0/2975012171 >> 172.21.15.39:0/2975012171 conn(0x7f6e00079790 msgr2=0x7f6e00079ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:47:27.328 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:27.316+0000 7f6e0737f640 1 -- 172.21.15.39:0/2975012171 shutdown_connections 2024-09-21T03:47:27.328 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:27.316+0000 7f6e0737f640 1 -- 172.21.15.39:0/2975012171 wait complete. 2024-09-21T03:47:27.328 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:27.317+0000 7f6e0737f640 1 Processor -- start 2024-09-21T03:47:27.328 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:27.317+0000 7f6e0737f640 1 -- start start 2024-09-21T03:47:27.328 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:27.317+0000 7f6e0737f640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6e00102ae0 0x7f6e000ff670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:47:27.328 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:27.317+0000 7f6e0737f640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6e001083d0 0x7f6e000ffbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:47:27.329 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:27.317+0000 7f6e0737f640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6e00108d20 0x7f6e001000f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:47:27.329 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:27.317+0000 7f6e0737f640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f6e001129e0 con 0x7f6e00102ae0 2024-09-21T03:47:27.329 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:27.317+0000 7f6e0737f640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f6e00112860 con 0x7f6e001083d0 2024-09-21T03:47:27.329 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:27.317+0000 7f6e0737f640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f6e00112b60 con 0x7f6e00108d20 2024-09-21T03:47:27.329 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:27.317+0000 7f6e0511b640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6e00102ae0 0x7f6e000ff670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:47:27.329 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:27.317+0000 7f6e0511b640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6e00102ae0 0x7f6e000ff670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:38774/0 (socket says 172.21.15.39:38774) 2024-09-21T03:47:27.329 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:27.317+0000 7f6e0511b640 1 -- 172.21.15.39:0/910803181 learned_addr learned my addr 172.21.15.39:0/910803181 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:47:27.329 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:27.317+0000 7f6e0591c640 1 --2- 172.21.15.39:0/910803181 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6e00108d20 0x7f6e001000f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:47:27.329 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:27.317+0000 7f6e0491a640 1 --2- 172.21.15.39:0/910803181 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6e001083d0 0x7f6e000ffbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:47:27.329 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:27.317+0000 7f6e0511b640 1 -- 172.21.15.39:0/910803181 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6e00108d20 msgr2=0x7f6e001000f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:47:27.329 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:27.317+0000 7f6e0511b640 1 --2- 172.21.15.39:0/910803181 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6e00108d20 0x7f6e001000f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:27.329 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:27.317+0000 7f6e0511b640 1 -- 172.21.15.39:0/910803181 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6e001083d0 msgr2=0x7f6e000ffbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:47:27.330 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:27.317+0000 7f6e0511b640 1 --2- 172.21.15.39:0/910803181 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6e001083d0 0x7f6e000ffbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:27.330 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:27.318+0000 7f6e0511b640 1 -- 172.21.15.39:0/910803181 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6e001a8780 con 0x7f6e00102ae0 2024-09-21T03:47:27.330 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:27.318+0000 7f6e0511b640 1 --2- 172.21.15.39:0/910803181 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6e00102ae0 0x7f6e000ff670 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f6dfc005940 tx=0x7f6dfc0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:47:27.330 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:27.318+0000 7f6df67fc640 1 -- 172.21.15.39:0/910803181 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6dfc03f030 con 0x7f6e00102ae0 2024-09-21T03:47:27.330 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:27.318+0000 7f6e0737f640 1 -- 172.21.15.39:0/910803181 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6e001a8a10 con 0x7f6e00102ae0 2024-09-21T03:47:27.330 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:27.318+0000 7f6df67fc640 1 -- 172.21.15.39:0/910803181 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6dfc003cb0 con 0x7f6e00102ae0 2024-09-21T03:47:27.330 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:27.318+0000 7f6e0737f640 1 -- 172.21.15.39:0/910803181 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6e001a8f40 con 0x7f6e00102ae0 2024-09-21T03:47:27.330 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:27.318+0000 7f6df67fc640 1 -- 172.21.15.39:0/910803181 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6dfc035070 con 0x7f6e00102ae0 2024-09-21T03:47:27.330 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:27.321+0000 7f6e0d8f3740 1 Processor -- start 2024-09-21T03:47:27.330 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:27.321+0000 7f6e0d8f3740 1 -- start start 2024-09-21T03:47:27.330 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:27.321+0000 7f6e0d8f3740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5649c1b81070 0x5649c1f286a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:47:27.330 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:27.321+0000 7f6e0d8f3740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5649c1b581b0 0x5649c1f28be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:47:27.330 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:27.321+0000 7f6e0d8f3740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5649c1b60390 0x5649c1f29120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:47:27.331 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:27.321+0000 7f6e0d8f3740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x5649c1e9eea0 con 0x5649c1b60390 2024-09-21T03:47:27.331 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:27.321+0000 7f6e0d8f3740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x5649c1e8e2d0 con 0x5649c1b81070 2024-09-21T03:47:27.331 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:27.321+0000 7f6e0d8f3740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x5649c1e920e0 con 0x5649c1b581b0 2024-09-21T03:47:27.331 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:27.321+0000 7f6e0491a640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5649c1b60390 0x5649c1f29120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:47:27.331 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:27.321+0000 7f6df67fc640 1 -- 172.21.15.39:0/910803181 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 26) ==== 154694+0+0 (secure 0 0 0) 0x7f6dfc042030 con 0x7f6e00102ae0 2024-09-21T03:47:27.331 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:27.321+0000 7f6e0491a640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5649c1b60390 0x5649c1f29120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:38786/0 (socket says 172.21.15.39:38786) 2024-09-21T03:47:27.331 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:27.321+0000 7f6e0491a640 1 -- 172.21.15.39:0/1319576389 learned_addr learned my addr 172.21.15.39:0/1319576389 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:47:27.331 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:27.322+0000 7f6e0591c640 1 --2- 172.21.15.39:0/1319576389 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5649c1b81070 0x5649c1f286a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:47:27.331 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:27.322+0000 7f6e0491a640 1 -- 172.21.15.39:0/1319576389 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5649c1b581b0 msgr2=0x5649c1f28be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:47:27.331 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:27.322+0000 7f6e0491a640 1 --2- 172.21.15.39:0/1319576389 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5649c1b581b0 0x5649c1f28be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:27.331 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:27.322+0000 7f6e0491a640 1 -- 172.21.15.39:0/1319576389 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5649c1b81070 msgr2=0x5649c1f286a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:47:27.331 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:27.322+0000 7f6e0491a640 1 --2- 172.21.15.39:0/1319576389 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5649c1b81070 0x5649c1f286a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:27.331 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:27.322+0000 7f6e0491a640 1 -- 172.21.15.39:0/1319576389 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5649c1bec650 con 0x5649c1b60390 2024-09-21T03:47:27.331 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:27.322+0000 7f6e0491a640 1 --2- 172.21.15.39:0/1319576389 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5649c1b60390 0x5649c1f29120 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f6df000bc30 tx=0x7f6df00107a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ae03146de07ed85 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:47:27.332 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:27.322+0000 7f6deb7fe640 1 -- 172.21.15.39:0/1319576389 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6df001c480 con 0x5649c1b60390 2024-09-21T03:47:27.332 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:27.322+0000 7f6deb7fe640 1 -- 172.21.15.39:0/1319576389 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6df001cad0 con 0x5649c1b60390 2024-09-21T03:47:27.332 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:27.322+0000 7f6deb7fe640 1 -- 172.21.15.39:0/1319576389 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6df0011aa0 con 0x5649c1b60390 2024-09-21T03:47:27.332 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:27.322+0000 7f6df67fc640 1 --2- 172.21.15.39:0/910803181 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f6dec0b6710 0x7f6dec0b8c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:47:27.332 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:27.323+0000 7f6e0591c640 1 --2- 172.21.15.39:0/910803181 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f6dec0b6710 0x7f6dec0b8c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:47:27.332 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:27.323+0000 7f6df67fc640 1 -- 172.21.15.39:0/910803181 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8484+0+0 (secure 0 0 0) 0x7f6dfc108120 con 0x7f6e00102ae0 2024-09-21T03:47:27.332 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:27.323+0000 7f6e0d8f3740 1 -- 172.21.15.39:0/1319576389 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5649c1b60390 msgr2=0x5649c1f29120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:47:27.332 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:27.323+0000 7f6e0591c640 1 --2- 172.21.15.39:0/910803181 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f6dec0b6710 0x7f6dec0b8c20 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f6df8005fd0 tx=0x7f6df8009600 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:47:27.332 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:27.323+0000 7f6e0d8f3740 1 --2- 172.21.15.39:0/1319576389 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5649c1b60390 0x5649c1f29120 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f6df000bc30 tx=0x7f6df00107a0 comp rx=0 tx=0).stop 2024-09-21T03:47:27.332 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:27.323+0000 7f6e0d8f3740 1 -- 172.21.15.39:0/1319576389 shutdown_connections 2024-09-21T03:47:27.332 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:27.323+0000 7f6e0d8f3740 1 --2- 172.21.15.39:0/1319576389 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5649c1b60390 0x5649c1f29120 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:27.332 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:27.323+0000 7f6e0d8f3740 1 --2- 172.21.15.39:0/1319576389 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5649c1b581b0 0x5649c1f28be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:27.332 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:27.323+0000 7f6e0d8f3740 1 --2- 172.21.15.39:0/1319576389 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5649c1b81070 0x5649c1f286a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:27.332 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:27.323+0000 7f6e0d8f3740 1 -- 172.21.15.39:0/1319576389 >> 172.21.15.39:0/1319576389 conn(0x5649c1b75460 msgr2=0x5649c1f26590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:47:27.333 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:27.324+0000 7f6e0d8f3740 1 -- 172.21.15.39:0/1319576389 shutdown_connections 2024-09-21T03:47:27.333 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:27.324+0000 7f6e0d8f3740 1 -- 172.21.15.39:0/1319576389 wait complete. 2024-09-21T03:47:27.333 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:27.324+0000 7f6e0d8f3740 1 -- start start 2024-09-21T03:47:27.333 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:27.324+0000 7f6e0d8f3740 1 Processor -- start 2024-09-21T03:47:27.334 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:27.325+0000 7f6e0d8f3740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5649c1b581b0 0x5649c1f33dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:47:27.334 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:27.325+0000 7f6e0d8f3740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5649c1b60390 0x5649c1f34300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:47:27.334 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:27.325+0000 7f6e0d8f3740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5649c1b81070 0x5649c1f40170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:47:27.334 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:27.325+0000 7f6e0d8f3740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x5649c1cef700 con 0x5649c1b81070 2024-09-21T03:47:27.334 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:27.325+0000 7f6e0d8f3740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x5649c1e29160 con 0x5649c1b581b0 2024-09-21T03:47:27.334 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:27.325+0000 7f6e0d8f3740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x5649c1f36e80 con 0x5649c1b60390 2024-09-21T03:47:27.334 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:27.325+0000 7f6e0591c640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5649c1b81070 0x5649c1f40170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:47:27.334 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:27.325+0000 7f6e0d8f3740 10 client.0 _ll_register_callbacks cb 0x5649c1e429e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:47:27.334 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:27.325+0000 7f6e0591c640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5649c1b81070 0x5649c1f40170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:38798/0 (socket says 172.21.15.39:38798) 2024-09-21T03:47:27.334 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:27.325+0000 7f6e0591c640 1 -- 172.21.15.39:0/2705066958 learned_addr learned my addr 172.21.15.39:0/2705066958 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:47:27.334 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:27.325+0000 7f6e0511b640 1 --2- 172.21.15.39:0/2705066958 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5649c1b581b0 0x5649c1f33dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:47:27.334 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:27.325+0000 7f6e0491a640 1 --2- 172.21.15.39:0/2705066958 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5649c1b60390 0x5649c1f34300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:47:27.335 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:27.326+0000 7f6e0591c640 1 -- 172.21.15.39:0/2705066958 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5649c1b60390 msgr2=0x5649c1f34300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:47:27.335 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:27.326+0000 7f6e0591c640 1 --2- 172.21.15.39:0/2705066958 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5649c1b60390 0x5649c1f34300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:27.335 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:27.326+0000 7f6e0591c640 1 -- 172.21.15.39:0/2705066958 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5649c1b581b0 msgr2=0x5649c1f33dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:47:27.335 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:27.326+0000 7f6e0591c640 1 --2- 172.21.15.39:0/2705066958 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5649c1b581b0 0x5649c1f33dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:27.335 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:27.326+0000 7f6e0591c640 1 -- 172.21.15.39:0/2705066958 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5649c1e56c90 con 0x5649c1b81070 2024-09-21T03:47:27.335 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:27.326+0000 7f6e0591c640 1 --2- 172.21.15.39:0/2705066958 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5649c1b81070 0x5649c1f40170 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f6df8005c40 tx=0x7f6df8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=72f2df2e826b0d18 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:47:27.335 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:27.326+0000 7f6de8ff9640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-09-21T03:47:27.338 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:27.326+0000 7f6de8ff9640 1 -- 172.21.15.39:0/2705066958 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6df8020620 con 0x5649c1b81070 2024-09-21T03:47:27.338 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:27.326+0000 7f6de8ff9640 1 -- 172.21.15.39:0/2705066958 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8484+0+0 (secure 0 0 0) 0x7f6df80207c0 con 0x5649c1b81070 2024-09-21T03:47:27.338 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:27.326+0000 7f6e0d8f3740 1 -- 172.21.15.39:0/2705066958 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6df001cad0 con 0x5649c1b81070 2024-09-21T03:47:27.338 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:27.326+0000 7f6de8ff9640 1 -- 172.21.15.39:0/2705066958 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f6df8020a00 con 0x5649c1b81070 2024-09-21T03:47:27.338 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:27.326+0000 7f6e0d8f3740 10 client.15921 fetch_fsmap learned FSMap version 29 2024-09-21T03:47:27.338 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:27.327+0000 7f6e0d8f3740 1 -- 172.21.15.39:0/2705066958 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5649c1f42c50 con 0x5649c1b81070 2024-09-21T03:47:27.338 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:27.327+0000 7f6de8ff9640 1 -- 172.21.15.39:0/2705066958 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6df001cad0 con 0x5649c1b81070 2024-09-21T03:47:27.338 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:27.327+0000 7f6de8ff9640 1 -- 172.21.15.39:0/2705066958 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f6df802f020 con 0x5649c1b81070 2024-09-21T03:47:27.339 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:27.327+0000 7f6e0d8f3740 10 client.15921 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T03:47:27.339 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:27.327+0000 7f6e0d8f3740 10 client.15921 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-21T03:47:27.339 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:27.327+0000 7f6e0d8f3740 20 client.15921 populate_metadata read hostname 'smithi039' 2024-09-21T03:47:27.339 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:27.327+0000 7f6e0d8f3740 1 --2- 172.21.15.39:0/2705066958 >> [v2:172.21.15.161:6836/3451544597,v1:172.21.15.161:6837/3451544597] conn(0x5649c1b3f9f0 0x5649c1f461f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:47:27.339 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:27.327+0000 7f6e0d8f3740 4 client.15921 mds_command: new command op to 24646 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:47:27.339 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:27.327+0000 7f6e0d8f3740 1 -- 172.21.15.39:0/2705066958 --> [v2:172.21.15.161:6836/3451544597,v1:172.21.15.161:6837/3451544597] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5649c1ea0df0 con 0x5649c1b3f9f0 2024-09-21T03:47:27.339 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:27.327+0000 7f6e0511b640 1 --2- 172.21.15.39:0/2705066958 >> [v2:172.21.15.161:6836/3451544597,v1:172.21.15.161:6837/3451544597] conn(0x5649c1b3f9f0 0x5649c1f461f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:47:27.339 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:27.328+0000 7f6de8ff9640 10 client.15921 ms_handle_connect on v2:172.21.15.161:6836/3451544597 2024-09-21T03:47:27.339 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:27.328+0000 7f6e0511b640 1 --2- 172.21.15.39:0/2705066958 >> [v2:172.21.15.161:6836/3451544597,v1:172.21.15.161:6837/3451544597] conn(0x5649c1b3f9f0 0x5649c1f461f0 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=603afde480fafc81 server_cookie=c5a83f833b25c01a in_seq=0 out_seq=0 2024-09-21T03:47:27.339 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:27.329+0000 7f6de8ff9640 1 -- 172.21.15.39:0/2705066958 <== mds.3 v2:172.21.15.161:6836/3451544597 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5649c1ea0df0 con 0x5649c1b3f9f0 2024-09-21T03:47:27.339 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:27.329+0000 7f6de8ff9640 10 client.15921 handle_command_reply: tid=0 2024-09-21T03:47:27.339 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:27.329+0000 7f6e0d8f3740 1 client.15921 shutdown 2024-09-21T03:47:27.340 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:27.329+0000 7f6e0d8f3740 1 -- 172.21.15.39:0/2705066958 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5649c1b81070 msgr2=0x5649c1f40170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:47:27.340 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:27.329+0000 7f6e0d8f3740 1 --2- 172.21.15.39:0/2705066958 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5649c1b81070 0x5649c1f40170 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f6df8005c40 tx=0x7f6df8004a40 comp rx=0 tx=0).stop 2024-09-21T03:47:27.340 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:27.329+0000 7f6e0d8f3740 1 -- 172.21.15.39:0/2705066958 shutdown_connections 2024-09-21T03:47:27.340 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:27.329+0000 7f6e0d8f3740 1 --2- 172.21.15.39:0/2705066958 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5649c1b81070 0x5649c1f40170 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:27.340 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:27.329+0000 7f6e0d8f3740 1 --2- 172.21.15.39:0/2705066958 >> [v2:172.21.15.161:6836/3451544597,v1:172.21.15.161:6837/3451544597] conn(0x5649c1b3f9f0 0x5649c1f461f0 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:27.340 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:27.329+0000 7f6e0d8f3740 1 --2- 172.21.15.39:0/2705066958 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5649c1b60390 0x5649c1f34300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:27.340 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:27.329+0000 7f6de8ff9640 0 client.15921 ms_handle_reset on v2:172.21.15.161:6836/3451544597 2024-09-21T03:47:27.340 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:27.329+0000 7f6e0d8f3740 1 --2- 172.21.15.39:0/2705066958 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5649c1b581b0 0x5649c1f33dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:27.340 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:27.329+0000 7f6e0d8f3740 1 -- 172.21.15.39:0/2705066958 >> 172.21.15.39:0/2705066958 conn(0x5649c1b75460 msgr2=0x5649c1f2b880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:47:27.340 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:27.329+0000 7f6e0d8f3740 1 -- 172.21.15.39:0/2705066958 shutdown_connections 2024-09-21T03:47:27.340 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:27.329+0000 7f6e0d8f3740 1 -- 172.21.15.39:0/2705066958 wait complete. 2024-09-21T03:47:27.340 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:27.330+0000 7f6e0d8f3740 20 client.15921 trim_cache size 0 max 16384 2024-09-21T03:47:27.360 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:27.358+0000 7f6e0d8f3740 1 Processor -- start 2024-09-21T03:47:27.360 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:27.358+0000 7f6e0d8f3740 1 -- start start 2024-09-21T03:47:27.362 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:27.358+0000 7f6e0d8f3740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5649c1b3f9f0 0x5649c1f2a090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:47:27.362 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:27.359+0000 7f6e0d8f3740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5649c1b581b0 0x5649c1f359f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:47:27.362 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:27.359+0000 7f6e0d8f3740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5649c1b60390 0x5649c1f40070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:47:27.362 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:27.359+0000 7f6e0d8f3740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f6e001b18b0 con 0x5649c1b581b0 2024-09-21T03:47:27.362 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:27.359+0000 7f6e0d8f3740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f6e001b33f0 con 0x5649c1b60390 2024-09-21T03:47:27.362 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:27.359+0000 7f6e0d8f3740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f6e001b1f80 con 0x5649c1b3f9f0 2024-09-21T03:47:27.362 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:27.359+0000 7f6e0491a640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5649c1b581b0 0x5649c1f359f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:47:27.362 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:27.359+0000 7f6e0491a640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5649c1b581b0 0x5649c1f359f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:36264/0 (socket says 172.21.15.39:36264) 2024-09-21T03:47:27.362 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:27.359+0000 7f6e0491a640 1 -- 172.21.15.39:0/2656634478 learned_addr learned my addr 172.21.15.39:0/2656634478 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:47:27.362 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:27.359+0000 7f6e0511b640 1 --2- 172.21.15.39:0/2656634478 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5649c1b3f9f0 0x5649c1f2a090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:47:27.362 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:27.359+0000 7f6e0591c640 1 --2- 172.21.15.39:0/2656634478 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5649c1b60390 0x5649c1f40070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:47:27.362 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:27.359+0000 7f6e0491a640 1 -- 172.21.15.39:0/2656634478 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5649c1b3f9f0 msgr2=0x5649c1f2a090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:47:27.363 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:27.359+0000 7f6e0491a640 1 --2- 172.21.15.39:0/2656634478 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5649c1b3f9f0 0x5649c1f2a090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:27.363 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:27.359+0000 7f6e0491a640 1 -- 172.21.15.39:0/2656634478 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5649c1b60390 msgr2=0x5649c1f40070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:47:27.363 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:27.359+0000 7f6e0491a640 1 --2- 172.21.15.39:0/2656634478 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5649c1b60390 0x5649c1f40070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:27.363 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:27.359+0000 7f6e0491a640 1 -- 172.21.15.39:0/2656634478 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5649c1f35080 con 0x5649c1b581b0 2024-09-21T03:47:27.363 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:27.359+0000 7f6e0591c640 1 --2- 172.21.15.39:0/2656634478 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5649c1b60390 0x5649c1f40070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:47:27.363 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:27.360+0000 7f6e0491a640 1 --2- 172.21.15.39:0/2656634478 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5649c1b581b0 0x5649c1f359f0 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f6df000bc30 tx=0x7f6df0005cb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a895592927d62c7 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:47:27.363 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:27.360+0000 7f6deaffd640 1 -- 172.21.15.39:0/2656634478 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6df000b770 con 0x5649c1b581b0 2024-09-21T03:47:27.363 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:27.360+0000 7f6deaffd640 1 -- 172.21.15.39:0/2656634478 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6df0005040 con 0x5649c1b581b0 2024-09-21T03:47:27.363 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:27.360+0000 7f6deaffd640 1 -- 172.21.15.39:0/2656634478 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6df00118f0 con 0x5649c1b581b0 2024-09-21T03:47:27.365 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:27.361+0000 7f6e0d8f3740 1 -- 172.21.15.39:0/2656634478 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5649c1b581b0 msgr2=0x5649c1f359f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:47:27.365 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:27.361+0000 7f6e0d8f3740 1 --2- 172.21.15.39:0/2656634478 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5649c1b581b0 0x5649c1f359f0 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f6df000bc30 tx=0x7f6df0005cb0 comp rx=0 tx=0).stop 2024-09-21T03:47:27.365 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:27.361+0000 7f6e0d8f3740 1 -- 172.21.15.39:0/2656634478 shutdown_connections 2024-09-21T03:47:27.366 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:27.361+0000 7f6e0d8f3740 1 --2- 172.21.15.39:0/2656634478 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5649c1b60390 0x5649c1f40070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:27.366 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:27.361+0000 7f6e0d8f3740 1 --2- 172.21.15.39:0/2656634478 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5649c1b581b0 0x5649c1f359f0 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:27.366 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:27.361+0000 7f6e0d8f3740 1 --2- 172.21.15.39:0/2656634478 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5649c1b3f9f0 0x5649c1f2a090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:27.366 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:27.361+0000 7f6e0d8f3740 1 -- 172.21.15.39:0/2656634478 >> 172.21.15.39:0/2656634478 conn(0x5649c1b75460 msgr2=0x5649c1f352e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:47:27.366 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:27.361+0000 7f6e0d8f3740 1 -- 172.21.15.39:0/2656634478 shutdown_connections 2024-09-21T03:47:27.366 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:27.361+0000 7f6e0d8f3740 1 -- 172.21.15.39:0/2656634478 wait complete. 2024-09-21T03:47:27.366 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:27.361+0000 7f6e0d8f3740 1 -- start start 2024-09-21T03:47:27.366 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:27.361+0000 7f6e0d8f3740 1 Processor -- start 2024-09-21T03:47:27.366 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:27.362+0000 7f6e0d8f3740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5649c1b3f9f0 0x5649c1f383f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:47:27.366 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:27.362+0000 7f6e0d8f3740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5649c1b581b0 0x5649c1f38930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:47:27.366 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:27.362+0000 7f6e0d8f3740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5649c1b60390 0x5649c1fafbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:47:27.366 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:27.362+0000 7f6e0d8f3740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f6e001b2d20 con 0x5649c1b3f9f0 2024-09-21T03:47:27.366 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:27.362+0000 7f6e0d8f3740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f6e001b2650 con 0x5649c1b581b0 2024-09-21T03:47:27.366 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:27.362+0000 7f6e0d8f3740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f6e001af890 con 0x5649c1b60390 2024-09-21T03:47:27.367 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:27.362+0000 7f6e0d8f3740 10 client.0 _ll_register_callbacks cb 0x7f6df8009df0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:47:27.367 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:27.362+0000 7f6e0511b640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5649c1b3f9f0 0x5649c1f383f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:47:27.367 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:27.362+0000 7f6e0511b640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5649c1b3f9f0 0x5649c1f383f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:36276/0 (socket says 172.21.15.39:36276) 2024-09-21T03:47:27.367 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:27.362+0000 7f6e0511b640 1 -- 172.21.15.39:0/793589872 learned_addr learned my addr 172.21.15.39:0/793589872 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:47:27.367 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:27.362+0000 7f6e0591c640 1 --2- 172.21.15.39:0/793589872 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5649c1b60390 0x5649c1fafbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:47:27.367 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:27.362+0000 7f6e0491a640 1 --2- 172.21.15.39:0/793589872 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5649c1b581b0 0x5649c1f38930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:47:27.367 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:27.363+0000 7f6e0511b640 1 -- 172.21.15.39:0/793589872 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5649c1b60390 msgr2=0x5649c1fafbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:47:27.367 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:27.363+0000 7f6e0511b640 1 --2- 172.21.15.39:0/793589872 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5649c1b60390 0x5649c1fafbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:27.367 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:27.363+0000 7f6e0511b640 1 -- 172.21.15.39:0/793589872 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5649c1b581b0 msgr2=0x5649c1f38930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:47:27.367 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:27.363+0000 7f6e0511b640 1 --2- 172.21.15.39:0/793589872 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5649c1b581b0 0x5649c1f38930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:27.367 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:27.363+0000 7f6e0511b640 1 -- 172.21.15.39:0/793589872 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5649c1f391b0 con 0x5649c1b3f9f0 2024-09-21T03:47:27.367 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:27.363+0000 7f6e0511b640 1 --2- 172.21.15.39:0/793589872 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5649c1b3f9f0 0x5649c1f383f0 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f6dfc0cb140 tx=0x7f6dfc03b6b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=21b152917a82f86b server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:47:27.367 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:27.363+0000 7f6de8ff9640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-09-21T03:47:27.368 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:27.363+0000 7f6de8ff9640 1 -- 172.21.15.39:0/793589872 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6dfc049040 con 0x5649c1b3f9f0 2024-09-21T03:47:27.368 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:27.363+0000 7f6de8ff9640 1 -- 172.21.15.39:0/793589872 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8484+0+0 (secure 0 0 0) 0x7f6dfc0ca140 con 0x5649c1b3f9f0 2024-09-21T03:47:27.368 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:27.363+0000 7f6e0d8f3740 1 -- 172.21.15.39:0/793589872 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6df0005040 con 0x5649c1b3f9f0 2024-09-21T03:47:27.368 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:27.363+0000 7f6de8ff9640 1 -- 172.21.15.39:0/793589872 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f6dfc0cd9e0 con 0x5649c1b3f9f0 2024-09-21T03:47:27.368 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:27.363+0000 7f6e0d8f3740 10 client.15927 fetch_fsmap learned FSMap version 29 2024-09-21T03:47:27.368 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:27.363+0000 7f6e0d8f3740 1 -- 172.21.15.39:0/793589872 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5649c1fb1190 con 0x5649c1b3f9f0 2024-09-21T03:47:27.368 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:27.363+0000 7f6de8ff9640 1 -- 172.21.15.39:0/793589872 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6df0005040 con 0x5649c1b3f9f0 2024-09-21T03:47:27.368 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:27.364+0000 7f6de8ff9640 1 -- 172.21.15.39:0/793589872 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f6dfc105b10 con 0x5649c1b3f9f0 2024-09-21T03:47:27.368 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:27.364+0000 7f6e0d8f3740 10 client.15927 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T03:47:27.368 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:27.364+0000 7f6e0d8f3740 10 client.15927 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-21T03:47:27.368 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:27.364+0000 7f6e0d8f3740 20 client.15927 populate_metadata read hostname 'smithi039' 2024-09-21T03:47:27.368 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:27.364+0000 7f6e0d8f3740 1 --2- 172.21.15.39:0/793589872 >> [v2:172.21.15.161:6836/3451544597,v1:172.21.15.161:6837/3451544597] conn(0x5649c1b81070 0x5649c1fb39a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:47:27.368 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:27.364+0000 7f6e0d8f3740 4 client.15927 mds_command: new command op to 24646 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T03:47:27.369 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:27.364+0000 7f6e0d8f3740 1 -- 172.21.15.39:0/793589872 --> [v2:172.21.15.161:6836/3451544597,v1:172.21.15.161:6837/3451544597] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5649c1e9c8a0 con 0x5649c1b81070 2024-09-21T03:47:27.370 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:27.364+0000 7f6e0491a640 1 --2- 172.21.15.39:0/793589872 >> [v2:172.21.15.161:6836/3451544597,v1:172.21.15.161:6837/3451544597] conn(0x5649c1b81070 0x5649c1fb39a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:47:27.370 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:27.365+0000 7f6de8ff9640 10 client.15927 ms_handle_connect on v2:172.21.15.161:6836/3451544597 2024-09-21T03:47:27.370 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:27.365+0000 7f6e0491a640 1 --2- 172.21.15.39:0/793589872 >> [v2:172.21.15.161:6836/3451544597,v1:172.21.15.161:6837/3451544597] conn(0x5649c1b81070 0x5649c1fb39a0 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=daefc811fcb47c44 server_cookie=d5df0b5c754d0a1b in_seq=0 out_seq=0 2024-09-21T03:47:27.370 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:27.365+0000 7f6de8ff9640 1 -- 172.21.15.39:0/793589872 <== mds.3 v2:172.21.15.161:6836/3451544597 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5649c1e9c8a0 con 0x5649c1b81070 2024-09-21T03:47:27.370 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:27.365+0000 7f6de8ff9640 10 client.15927 handle_command_reply: tid=0 2024-09-21T03:47:27.370 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:27.365+0000 7f6e0d8f3740 1 client.15927 shutdown 2024-09-21T03:47:27.370 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:27.365+0000 7f6e0d8f3740 1 -- 172.21.15.39:0/793589872 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5649c1b3f9f0 msgr2=0x5649c1f383f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:47:27.370 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:27.365+0000 7f6e0d8f3740 1 --2- 172.21.15.39:0/793589872 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5649c1b3f9f0 0x5649c1f383f0 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f6dfc0cb140 tx=0x7f6dfc03b6b0 comp rx=0 tx=0).stop 2024-09-21T03:47:27.370 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:27.365+0000 7f6e0d8f3740 1 -- 172.21.15.39:0/793589872 shutdown_connections 2024-09-21T03:47:27.370 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:27.365+0000 7f6e0d8f3740 1 --2- 172.21.15.39:0/793589872 >> [v2:172.21.15.161:6836/3451544597,v1:172.21.15.161:6837/3451544597] conn(0x5649c1b81070 0x5649c1fb39a0 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:27.370 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:27.365+0000 7f6e0d8f3740 1 --2- 172.21.15.39:0/793589872 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5649c1b60390 0x5649c1fafbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:27.370 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:27.366+0000 7f6de8ff9640 0 client.15927 ms_handle_reset on v2:172.21.15.161:6836/3451544597 2024-09-21T03:47:27.371 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:27.366+0000 7f6e0d8f3740 1 --2- 172.21.15.39:0/793589872 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5649c1b581b0 0x5649c1f38930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:27.371 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:27.366+0000 7f6e0d8f3740 1 --2- 172.21.15.39:0/793589872 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5649c1b3f9f0 0x5649c1f383f0 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:27.371 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:27.366+0000 7f6e0d8f3740 1 -- 172.21.15.39:0/793589872 >> 172.21.15.39:0/793589872 conn(0x5649c1b75460 msgr2=0x5649c1f29e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:47:27.371 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:27.366+0000 7f6e0d8f3740 1 -- 172.21.15.39:0/793589872 shutdown_connections 2024-09-21T03:47:27.371 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:27.366+0000 7f6e0d8f3740 1 -- 172.21.15.39:0/793589872 wait complete. 2024-09-21T03:47:27.371 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:27.366+0000 7f6e0d8f3740 20 client.15927 trim_cache size 0 max 16384 2024-09-21T03:47:27.371 INFO:teuthology.orchestra.run.smithi039.stdout:[] 2024-09-21T03:47:27.372 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:27.366+0000 7f6e0737f640 1 -- 172.21.15.39:0/910803181 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f6dec0b6710 msgr2=0x7f6dec0b8c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:47:27.372 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:27.366+0000 7f6e0737f640 1 --2- 172.21.15.39:0/910803181 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f6dec0b6710 0x7f6dec0b8c20 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f6df8005fd0 tx=0x7f6df8009600 comp rx=0 tx=0).stop 2024-09-21T03:47:27.372 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:27.366+0000 7f6e0737f640 1 -- 172.21.15.39:0/910803181 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6e00102ae0 msgr2=0x7f6e000ff670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:47:27.372 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:27.366+0000 7f6e0737f640 1 --2- 172.21.15.39:0/910803181 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6e00102ae0 0x7f6e000ff670 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f6dfc005940 tx=0x7f6dfc0042f0 comp rx=0 tx=0).stop 2024-09-21T03:47:27.372 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:27.367+0000 7f6e0737f640 1 -- 172.21.15.39:0/910803181 shutdown_connections 2024-09-21T03:47:27.372 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:27.367+0000 7f6e0737f640 1 --2- 172.21.15.39:0/910803181 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f6dec0b6710 0x7f6dec0b8c20 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:27.372 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:27.367+0000 7f6e0737f640 1 --2- 172.21.15.39:0/910803181 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6e00108d20 0x7f6e001000f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:27.372 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:27.367+0000 7f6e0737f640 1 --2- 172.21.15.39:0/910803181 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6e001083d0 0x7f6e000ffbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:27.372 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:27.367+0000 7f6e0737f640 1 --2- 172.21.15.39:0/910803181 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6e00102ae0 0x7f6e000ff670 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:27.372 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:27.367+0000 7f6e0737f640 1 -- 172.21.15.39:0/910803181 >> 172.21.15.39:0/910803181 conn(0x7f6e00079790 msgr2=0x7f6e00101af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:47:27.372 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:27.367+0000 7f6e0737f640 1 -- 172.21.15.39:0/910803181 shutdown_connections 2024-09-21T03:47:27.372 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:27.367+0000 7f6e0737f640 1 -- 172.21.15.39:0/910803181 wait complete. 2024-09-21T03:47:28.300 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 51116d5c-77ca-11ef-baf5-efdc52797490 -- ceph tell mds.1:2 damage ls 2024-09-21T03:47:28.756 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/51116d5c-77ca-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:47:28.813 INFO:tasks.workunit.client.0.smithi039.stdout: 2024-09-21T03:47:28.814 INFO:tasks.workunit.client.0.smithi039.stdout: Children see throughput for 1 initial writers = 618199.62 kB/sec 2024-09-21T03:47:28.814 INFO:tasks.workunit.client.0.smithi039.stdout: Parent sees throughput for 1 initial writers = 618147.52 kB/sec 2024-09-21T03:47:28.814 INFO:tasks.workunit.client.0.smithi039.stdout: Min throughput per process = 618199.62 kB/sec 2024-09-21T03:47:28.814 INFO:tasks.workunit.client.0.smithi039.stdout: Max throughput per process = 618199.62 kB/sec 2024-09-21T03:47:28.814 INFO:tasks.workunit.client.0.smithi039.stdout: Avg throughput per process = 618199.62 kB/sec 2024-09-21T03:47:28.814 INFO:tasks.workunit.client.0.smithi039.stdout: Min xfer = 1048576.00 kB 2024-09-21T03:47:29.333 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:47:29 smithi039 ceph-mon[31211]: pgmap v368: 97 pgs: 97 active+clean; 1.4 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 85 B/s wr, 8 op/s 2024-09-21T03:47:29.334 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:47:29 smithi039 ceph-mon[31211]: mgrmap e27: x(active, since 10m), standbys: z, y 2024-09-21T03:47:29.468 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:47:29 smithi161 ceph-mon[34533]: pgmap v368: 97 pgs: 97 active+clean; 1.4 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 85 B/s wr, 8 op/s 2024-09-21T03:47:29.468 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:47:29 smithi161 ceph-mon[34533]: mgrmap e27: x(active, since 10m), standbys: z, y 2024-09-21T03:47:29.474 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:47:29 smithi150 ceph-mon[34289]: pgmap v368: 97 pgs: 97 active+clean; 1.4 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 85 B/s wr, 8 op/s 2024-09-21T03:47:29.474 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:47:29 smithi150 ceph-mon[34289]: mgrmap e27: x(active, since 10m), standbys: z, y 2024-09-21T03:47:31.467 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:47:31 smithi161 ceph-mon[34533]: pgmap v369: 97 pgs: 97 active+clean; 441 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 37 MiB/s wr, 22 op/s 2024-09-21T03:47:31.474 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:47:31 smithi150 ceph-mon[34289]: pgmap v369: 97 pgs: 97 active+clean; 441 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 37 MiB/s wr, 22 op/s 2024-09-21T03:47:31.584 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:47:31 smithi039 ceph-mon[31211]: pgmap v369: 97 pgs: 97 active+clean; 441 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 37 MiB/s wr, 22 op/s 2024-09-21T03:47:31.724 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:31.709+0000 7f595f873640 1 -- 172.21.15.39:0/323549463 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5958103b60 msgr2=0x7f5958103f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:47:31.724 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:31.709+0000 7f595f873640 1 --2- 172.21.15.39:0/323549463 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5958103b60 0x7f5958103f70 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f5954009900 tx=0x7f595402a410 comp rx=0 tx=0).stop 2024-09-21T03:47:31.724 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:31.709+0000 7f595f873640 1 -- 172.21.15.39:0/323549463 shutdown_connections 2024-09-21T03:47:31.724 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:31.709+0000 7f595f873640 1 --2- 172.21.15.39:0/323549463 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5958109df0 0x7f5958110620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:31.724 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:31.709+0000 7f595f873640 1 --2- 172.21.15.39:0/323549463 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f59581094a0 0x7f59581098b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:31.724 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:31.709+0000 7f595f873640 1 --2- 172.21.15.39:0/323549463 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5958103b60 0x7f5958103f70 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:31.724 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:31.709+0000 7f595f873640 1 -- 172.21.15.39:0/323549463 >> 172.21.15.39:0/323549463 conn(0x7f595807b850 msgr2=0x7f595807bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:47:31.724 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:31.709+0000 7f595f873640 1 -- 172.21.15.39:0/323549463 shutdown_connections 2024-09-21T03:47:31.724 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:31.709+0000 7f595f873640 1 -- 172.21.15.39:0/323549463 wait complete. 2024-09-21T03:47:31.725 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:31.709+0000 7f595f873640 1 Processor -- start 2024-09-21T03:47:31.725 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:31.709+0000 7f595f873640 1 -- start start 2024-09-21T03:47:31.725 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:31.710+0000 7f595f873640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5958103b60 0x7f5958075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:47:31.725 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:31.710+0000 7f595f873640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f59581094a0 0x7f5958075c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:47:31.725 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:31.710+0000 7f595f873640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5958109df0 0x7f5958076190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:47:31.725 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:31.710+0000 7f595f873640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f5958113ab0 con 0x7f59581094a0 2024-09-21T03:47:31.725 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:31.710+0000 7f595f873640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f5958113930 con 0x7f5958103b60 2024-09-21T03:47:31.725 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:31.710+0000 7f595f873640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f5958113c30 con 0x7f5958109df0 2024-09-21T03:47:31.725 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:31.710+0000 7f595ce0e640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f59581094a0 0x7f5958075c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:47:31.725 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:31.710+0000 7f595ce0e640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f59581094a0 0x7f5958075c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:36296/0 (socket says 172.21.15.39:36296) 2024-09-21T03:47:31.725 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:31.710+0000 7f595ce0e640 1 -- 172.21.15.39:0/3559820241 learned_addr learned my addr 172.21.15.39:0/3559820241 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:47:31.725 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:31.710+0000 7f595d60f640 1 --2- 172.21.15.39:0/3559820241 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5958103b60 0x7f5958075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:47:31.725 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:31.710+0000 7f595de10640 1 --2- 172.21.15.39:0/3559820241 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5958109df0 0x7f5958076190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:47:31.726 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:31.710+0000 7f595ce0e640 1 -- 172.21.15.39:0/3559820241 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5958109df0 msgr2=0x7f5958076190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:47:31.726 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:31.710+0000 7f595ce0e640 1 --2- 172.21.15.39:0/3559820241 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5958109df0 0x7f5958076190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:31.726 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:31.710+0000 7f595ce0e640 1 -- 172.21.15.39:0/3559820241 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5958103b60 msgr2=0x7f5958075710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:47:31.726 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:31.710+0000 7f595ce0e640 1 --2- 172.21.15.39:0/3559820241 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5958103b60 0x7f5958075710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:31.726 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:31.710+0000 7f595ce0e640 1 -- 172.21.15.39:0/3559820241 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f59581a8670 con 0x7f59581094a0 2024-09-21T03:47:31.726 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:31.710+0000 7f595d60f640 1 --2- 172.21.15.39:0/3559820241 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5958103b60 0x7f5958075710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:47:31.726 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:31.710+0000 7f595ce0e640 1 --2- 172.21.15.39:0/3559820241 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f59581094a0 0x7f5958075c50 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f594800c8c0 tx=0x7f594800cd80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:47:31.726 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:31.711+0000 7f594e7fc640 1 -- 172.21.15.39:0/3559820241 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5948007bb0 con 0x7f59581094a0 2024-09-21T03:47:31.726 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:31.711+0000 7f594e7fc640 1 -- 172.21.15.39:0/3559820241 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5948007d50 con 0x7f59581094a0 2024-09-21T03:47:31.726 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:31.711+0000 7f594e7fc640 1 -- 172.21.15.39:0/3559820241 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59480187c0 con 0x7f59581094a0 2024-09-21T03:47:31.726 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:31.711+0000 7f595f873640 1 -- 172.21.15.39:0/3559820241 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f59581a8960 con 0x7f59581094a0 2024-09-21T03:47:31.726 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:31.711+0000 7f595f873640 1 -- 172.21.15.39:0/3559820241 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f59581a8ea0 con 0x7f59581094a0 2024-09-21T03:47:31.726 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:31.713+0000 7f5965de7740 1 Processor -- start 2024-09-21T03:47:31.726 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:31.714+0000 7f5965de7740 1 -- start start 2024-09-21T03:47:31.727 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:31.714+0000 7f594e7fc640 1 -- 172.21.15.39:0/3559820241 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 27) ==== 154643+0+0 (secure 0 0 0) 0x7f594800fa20 con 0x7f59581094a0 2024-09-21T03:47:31.727 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:31.714+0000 7f5965de7740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563e97c5d070 0x563e980042c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:47:31.727 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:31.714+0000 7f5965de7740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563e97c341b0 0x563e98004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:47:31.727 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:31.714+0000 7f5965de7740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563e97c3c390 0x563e98004d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:47:31.727 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:31.714+0000 7f5965de7740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x563e97f7aea0 con 0x563e97c341b0 2024-09-21T03:47:31.727 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:31.714+0000 7f5965de7740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x563e97f6a2d0 con 0x563e97c3c390 2024-09-21T03:47:31.727 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:31.714+0000 7f5965de7740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x563e97f6e0e0 con 0x563e97c5d070 2024-09-21T03:47:31.727 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:31.714+0000 7f595d60f640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563e97c341b0 0x563e98004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:47:31.727 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:31.714+0000 7f595de10640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563e97c5d070 0x563e980042c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:47:31.727 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:31.714+0000 7f595d60f640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563e97c341b0 0x563e98004800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:36302/0 (socket says 172.21.15.39:36302) 2024-09-21T03:47:31.727 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:31.714+0000 7f595de10640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563e97c5d070 0x563e980042c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.39:56382/0 (socket says 172.21.15.39:56382) 2024-09-21T03:47:31.727 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:31.714+0000 7f595d60f640 1 -- 172.21.15.39:0/1219645595 learned_addr learned my addr 172.21.15.39:0/1219645595 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:47:31.728 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:31.714+0000 7f595ce0e640 1 --2- 172.21.15.39:0/1219645595 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563e97c3c390 0x563e98004d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:47:31.728 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:31.715+0000 7f595d60f640 1 -- 172.21.15.39:0/1219645595 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563e97c5d070 msgr2=0x563e980042c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:47:31.728 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:31.715+0000 7f595d60f640 1 --2- 172.21.15.39:0/1219645595 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563e97c5d070 0x563e980042c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:31.728 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:31.715+0000 7f595d60f640 1 -- 172.21.15.39:0/1219645595 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563e97c3c390 msgr2=0x563e98004d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:47:31.728 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:31.715+0000 7f595d60f640 1 --2- 172.21.15.39:0/1219645595 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563e97c3c390 0x563e98004d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:31.728 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:31.715+0000 7f595d60f640 1 -- 172.21.15.39:0/1219645595 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563e97cc8650 con 0x563e97c341b0 2024-09-21T03:47:31.728 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:31.715+0000 7f595d60f640 1 --2- 172.21.15.39:0/1219645595 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563e97c341b0 0x563e98004800 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f595402a930 tx=0x7f5954032e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bab818fa2e02f9a server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:47:31.728 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:31.715+0000 7f594e7fc640 1 --2- 172.21.15.39:0/3559820241 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f59440b6680 0x7f59440b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:47:31.728 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:31.715+0000 7f59437fe640 1 -- 172.21.15.39:0/1219645595 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f595403f070 con 0x563e97c341b0 2024-09-21T03:47:31.728 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:31.715+0000 7f595de10640 1 --2- 172.21.15.39:0/3559820241 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f59440b6680 0x7f59440b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:47:31.728 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:31.715+0000 7f59437fe640 1 -- 172.21.15.39:0/1219645595 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f59540316a0 con 0x563e97c341b0 2024-09-21T03:47:31.728 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:31.715+0000 7f59437fe640 1 -- 172.21.15.39:0/1219645595 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5954035070 con 0x563e97c341b0 2024-09-21T03:47:31.728 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:31.715+0000 7f594e7fc640 1 -- 172.21.15.39:0/3559820241 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8484+0+0 (secure 0 0 0) 0x7f59480e78d0 con 0x7f59581094a0 2024-09-21T03:47:31.728 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:31.716+0000 7f5965de7740 1 -- 172.21.15.39:0/1219645595 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563e97c341b0 msgr2=0x563e98004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:47:31.729 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:31.716+0000 7f595de10640 1 --2- 172.21.15.39:0/3559820241 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f59440b6680 0x7f59440b8b90 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f5950005fd0 tx=0x7f59500097e0 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:47:31.729 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:31.716+0000 7f5965de7740 1 --2- 172.21.15.39:0/1219645595 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563e97c341b0 0x563e98004800 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f595402a930 tx=0x7f5954032e70 comp rx=0 tx=0).stop 2024-09-21T03:47:31.729 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:31.716+0000 7f5965de7740 1 -- 172.21.15.39:0/1219645595 shutdown_connections 2024-09-21T03:47:31.729 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:31.716+0000 7f5965de7740 1 --2- 172.21.15.39:0/1219645595 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563e97c3c390 0x563e98004d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:31.729 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:31.716+0000 7f5965de7740 1 --2- 172.21.15.39:0/1219645595 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563e97c341b0 0x563e98004800 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:31.729 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:31.716+0000 7f5965de7740 1 --2- 172.21.15.39:0/1219645595 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563e97c5d070 0x563e980042c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:31.729 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:31.716+0000 7f5965de7740 1 -- 172.21.15.39:0/1219645595 >> 172.21.15.39:0/1219645595 conn(0x563e97c51460 msgr2=0x563e980021b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:47:31.729 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:31.716+0000 7f5965de7740 1 -- 172.21.15.39:0/1219645595 shutdown_connections 2024-09-21T03:47:31.729 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:31.716+0000 7f5965de7740 1 -- 172.21.15.39:0/1219645595 wait complete. 2024-09-21T03:47:31.729 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:31.717+0000 7f5965de7740 1 -- start start 2024-09-21T03:47:31.729 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:31.717+0000 7f5965de7740 1 Processor -- start 2024-09-21T03:47:31.729 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:31.717+0000 7f5965de7740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563e97c341b0 0x563e9800f970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:47:31.729 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:31.717+0000 7f5965de7740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563e97c3c390 0x563e9800feb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:47:31.729 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:31.717+0000 7f5965de7740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563e97c5d070 0x563e9801bdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:47:31.730 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:31.717+0000 7f5965de7740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x563e97dcb700 con 0x563e97c5d070 2024-09-21T03:47:31.730 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:31.717+0000 7f5965de7740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x563e97f05160 con 0x563e97c3c390 2024-09-21T03:47:31.730 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:31.717+0000 7f5965de7740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x563e98012a40 con 0x563e97c341b0 2024-09-21T03:47:31.730 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:31.717+0000 7f5965de7740 10 client.0 _ll_register_callbacks cb 0x563e97f1e9e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:47:31.730 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:31.717+0000 7f595de10640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563e97c5d070 0x563e9801bdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:47:31.730 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:31.718+0000 7f595de10640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563e97c5d070 0x563e9801bdc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:36318/0 (socket says 172.21.15.39:36318) 2024-09-21T03:47:31.730 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:31.718+0000 7f595de10640 1 -- 172.21.15.39:0/467453050 learned_addr learned my addr 172.21.15.39:0/467453050 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:47:31.730 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:31.718+0000 7f595d60f640 1 --2- 172.21.15.39:0/467453050 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563e97c341b0 0x563e9800f970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:47:31.730 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:31.718+0000 7f595ce0e640 1 --2- 172.21.15.39:0/467453050 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563e97c3c390 0x563e9800feb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:47:31.730 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:31.718+0000 7f595de10640 1 -- 172.21.15.39:0/467453050 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563e97c341b0 msgr2=0x563e9800f970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:47:31.730 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:31.718+0000 7f595de10640 1 --2- 172.21.15.39:0/467453050 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563e97c341b0 0x563e9800f970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:31.730 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:31.718+0000 7f595de10640 1 -- 172.21.15.39:0/467453050 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563e97c3c390 msgr2=0x563e9800feb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:47:31.730 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:31.718+0000 7f595de10640 1 --2- 172.21.15.39:0/467453050 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563e97c3c390 0x563e9800feb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:31.731 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:31.718+0000 7f595de10640 1 -- 172.21.15.39:0/467453050 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563e97f32c90 con 0x563e97c5d070 2024-09-21T03:47:31.731 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:31.718+0000 7f595d60f640 1 --2- 172.21.15.39:0/467453050 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563e97c341b0 0x563e9800f970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:47:31.731 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:31.718+0000 7f595de10640 1 --2- 172.21.15.39:0/467453050 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563e97c5d070 0x563e9801bdc0 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f5950003e60 tx=0x7f5950004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a42def3976b448b8 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:47:31.731 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:31.718+0000 7f5940ff9640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-09-21T03:47:31.731 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:31.718+0000 7f5940ff9640 1 -- 172.21.15.39:0/467453050 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5950010900 con 0x563e97c5d070 2024-09-21T03:47:31.731 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:31.718+0000 7f5940ff9640 1 -- 172.21.15.39:0/467453050 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8484+0+0 (secure 0 0 0) 0x7f5950010aa0 con 0x563e97c5d070 2024-09-21T03:47:31.731 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:31.718+0000 7f5965de7740 1 -- 172.21.15.39:0/467453050 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f59540316a0 con 0x563e97c5d070 2024-09-21T03:47:31.731 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:31.718+0000 7f5940ff9640 1 -- 172.21.15.39:0/467453050 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f5950010ca0 con 0x563e97c5d070 2024-09-21T03:47:31.731 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:31.719+0000 7f5965de7740 10 client.15939 fetch_fsmap learned FSMap version 29 2024-09-21T03:47:31.731 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:31.719+0000 7f5965de7740 1 -- 172.21.15.39:0/467453050 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x563e9801e7e0 con 0x563e97c5d070 2024-09-21T03:47:31.731 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:31.719+0000 7f5940ff9640 1 -- 172.21.15.39:0/467453050 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f59540316a0 con 0x563e97c5d070 2024-09-21T03:47:31.731 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:31.719+0000 7f5940ff9640 1 -- 172.21.15.39:0/467453050 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f595002e020 con 0x563e97c5d070 2024-09-21T03:47:31.731 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:31.719+0000 7f5965de7740 10 client.15939 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T03:47:31.731 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:31.719+0000 7f5965de7740 10 client.15939 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-21T03:47:31.732 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:31.719+0000 7f5965de7740 20 client.15939 populate_metadata read hostname 'smithi039' 2024-09-21T03:47:31.732 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:31.719+0000 7f5965de7740 1 --2- 172.21.15.39:0/467453050 >> [v2:172.21.15.150:6836/4017043144,v1:172.21.15.150:6837/4017043144] conn(0x563e97c1b9f0 0x563e98021d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:47:31.732 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:31.719+0000 7f5965de7740 4 client.15939 mds_command: new command op to 24659 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:47:31.732 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:31.719+0000 7f5965de7740 1 -- 172.21.15.39:0/467453050 --> [v2:172.21.15.150:6836/4017043144,v1:172.21.15.150:6837/4017043144] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563e97f7cdf0 con 0x563e97c1b9f0 2024-09-21T03:47:31.732 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:31.719+0000 7f595d60f640 1 --2- 172.21.15.39:0/467453050 >> [v2:172.21.15.150:6836/4017043144,v1:172.21.15.150:6837/4017043144] conn(0x563e97c1b9f0 0x563e98021d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:47:31.732 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:31.720+0000 7f595d60f640 1 --2- 172.21.15.39:0/467453050 >> [v2:172.21.15.150:6836/4017043144,v1:172.21.15.150:6837/4017043144] conn(0x563e97c1b9f0 0x563e98021d00 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=c62ef32b5004a734 server_cookie=efb7e290e41b78f1 in_seq=0 out_seq=0 2024-09-21T03:47:31.732 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:31.720+0000 7f5940ff9640 10 client.15939 ms_handle_connect on v2:172.21.15.150:6836/4017043144 2024-09-21T03:47:31.732 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:31.721+0000 7f5940ff9640 1 -- 172.21.15.39:0/467453050 <== mds.2 v2:172.21.15.150:6836/4017043144 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x563e97f7cdf0 con 0x563e97c1b9f0 2024-09-21T03:47:31.732 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:31.721+0000 7f5940ff9640 10 client.15939 handle_command_reply: tid=0 2024-09-21T03:47:31.732 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:31.721+0000 7f5965de7740 1 client.15939 shutdown 2024-09-21T03:47:31.732 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:31.721+0000 7f5965de7740 1 -- 172.21.15.39:0/467453050 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563e97c5d070 msgr2=0x563e9801bdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:47:31.732 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:31.721+0000 7f5965de7740 1 --2- 172.21.15.39:0/467453050 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563e97c5d070 0x563e9801bdc0 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f5950003e60 tx=0x7f5950004a40 comp rx=0 tx=0).stop 2024-09-21T03:47:31.732 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:31.721+0000 7f5965de7740 1 -- 172.21.15.39:0/467453050 shutdown_connections 2024-09-21T03:47:31.732 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:31.721+0000 7f5965de7740 1 --2- 172.21.15.39:0/467453050 >> [v2:172.21.15.150:6836/4017043144,v1:172.21.15.150:6837/4017043144] conn(0x563e97c1b9f0 0x563e98021d00 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:31.733 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:31.721+0000 7f5965de7740 1 --2- 172.21.15.39:0/467453050 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563e97c5d070 0x563e9801bdc0 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:31.733 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:31.721+0000 7f5965de7740 1 --2- 172.21.15.39:0/467453050 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563e97c3c390 0x563e9800feb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:31.733 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:31.721+0000 7f5965de7740 1 --2- 172.21.15.39:0/467453050 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563e97c341b0 0x563e9800f970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:31.733 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:31.721+0000 7f5940ff9640 0 client.15939 ms_handle_reset on v2:172.21.15.150:6836/4017043144 2024-09-21T03:47:31.733 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:31.721+0000 7f5965de7740 1 -- 172.21.15.39:0/467453050 >> 172.21.15.39:0/467453050 conn(0x563e97c51460 msgr2=0x563e98007450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:47:31.733 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:31.721+0000 7f5965de7740 1 -- 172.21.15.39:0/467453050 shutdown_connections 2024-09-21T03:47:31.733 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:31.721+0000 7f5965de7740 1 -- 172.21.15.39:0/467453050 wait complete. 2024-09-21T03:47:31.733 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:31.721+0000 7f5965de7740 20 client.15939 trim_cache size 0 max 16384 2024-09-21T03:47:31.737 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:31.734+0000 7f5965de7740 1 Processor -- start 2024-09-21T03:47:31.737 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:31.734+0000 7f5965de7740 1 -- start start 2024-09-21T03:47:31.737 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:31.735+0000 7f5965de7740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563e97c341b0 0x563e98005a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:47:31.737 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:31.735+0000 7f5965de7740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563e97c3c390 0x563e98001760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:47:31.737 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:31.735+0000 7f5965de7740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563e97c5d070 0x563e98001ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:47:31.738 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:31.735+0000 7f5965de7740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f59581b1740 con 0x563e97c3c390 2024-09-21T03:47:31.738 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:31.735+0000 7f5965de7740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f59581b3280 con 0x563e97c5d070 2024-09-21T03:47:31.738 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:31.735+0000 7f5965de7740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f59581b1e10 con 0x563e97c341b0 2024-09-21T03:47:31.738 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:31.735+0000 7f595ce0e640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563e97c3c390 0x563e98001760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:47:31.738 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:31.735+0000 7f595ce0e640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563e97c3c390 0x563e98001760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:36332/0 (socket says 172.21.15.39:36332) 2024-09-21T03:47:31.738 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:31.735+0000 7f595ce0e640 1 -- 172.21.15.39:0/4080750982 learned_addr learned my addr 172.21.15.39:0/4080750982 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:47:31.738 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:31.735+0000 7f595d60f640 1 --2- 172.21.15.39:0/4080750982 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563e97c341b0 0x563e98005a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:47:31.738 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:31.735+0000 7f595de10640 1 --2- 172.21.15.39:0/4080750982 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563e97c5d070 0x563e98001ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:47:31.738 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:31.735+0000 7f595ce0e640 1 -- 172.21.15.39:0/4080750982 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563e97c341b0 msgr2=0x563e98005a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:47:31.738 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:31.735+0000 7f595ce0e640 1 --2- 172.21.15.39:0/4080750982 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563e97c341b0 0x563e98005a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:31.738 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:31.735+0000 7f595ce0e640 1 -- 172.21.15.39:0/4080750982 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563e97c5d070 msgr2=0x563e98001ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:47:31.738 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:31.735+0000 7f595ce0e640 1 --2- 172.21.15.39:0/4080750982 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563e97c5d070 0x563e98001ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:31.738 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:31.735+0000 7f595ce0e640 1 -- 172.21.15.39:0/4080750982 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563e980065b0 con 0x563e97c3c390 2024-09-21T03:47:31.739 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:31.735+0000 7f595d60f640 1 --2- 172.21.15.39:0/4080750982 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563e97c341b0 0x563e98005a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:47:31.739 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:31.735+0000 7f595ce0e640 1 --2- 172.21.15.39:0/4080750982 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563e97c3c390 0x563e98001760 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f59480a9d10 tx=0x7f594800fdc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b7f965e1cd4e536 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:47:31.739 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:31.736+0000 7f5942ffd640 1 -- 172.21.15.39:0/4080750982 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59480aceb0 con 0x563e97c3c390 2024-09-21T03:47:31.739 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:31.736+0000 7f5942ffd640 1 -- 172.21.15.39:0/4080750982 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f59480ad250 con 0x563e97c3c390 2024-09-21T03:47:31.739 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:31.736+0000 7f5942ffd640 1 -- 172.21.15.39:0/4080750982 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59480b6f30 con 0x563e97c3c390 2024-09-21T03:47:31.743 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:31.736+0000 7f5965de7740 1 -- 172.21.15.39:0/4080750982 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563e97c3c390 msgr2=0x563e98001760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:47:31.745 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:31.736+0000 7f5965de7740 1 --2- 172.21.15.39:0/4080750982 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563e97c3c390 0x563e98001760 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f59480a9d10 tx=0x7f594800fdc0 comp rx=0 tx=0).stop 2024-09-21T03:47:31.746 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:31.736+0000 7f5965de7740 1 -- 172.21.15.39:0/4080750982 shutdown_connections 2024-09-21T03:47:31.746 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:31.736+0000 7f5965de7740 1 --2- 172.21.15.39:0/4080750982 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563e97c5d070 0x563e98001ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:31.746 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:31.736+0000 7f5965de7740 1 --2- 172.21.15.39:0/4080750982 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563e97c3c390 0x563e98001760 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:31.746 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:31.736+0000 7f5965de7740 1 --2- 172.21.15.39:0/4080750982 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563e97c341b0 0x563e98005a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:31.746 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:31.736+0000 7f5965de7740 1 -- 172.21.15.39:0/4080750982 >> 172.21.15.39:0/4080750982 conn(0x563e97c51460 msgr2=0x563e980052f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:47:31.746 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:31.737+0000 7f5965de7740 1 -- 172.21.15.39:0/4080750982 shutdown_connections 2024-09-21T03:47:31.746 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:31.737+0000 7f5965de7740 1 -- 172.21.15.39:0/4080750982 wait complete. 2024-09-21T03:47:31.746 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:31.737+0000 7f5965de7740 1 -- start start 2024-09-21T03:47:31.746 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:31.737+0000 7f5965de7740 1 Processor -- start 2024-09-21T03:47:31.746 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:31.738+0000 7f5965de7740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563e97c341b0 0x563e98013fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:47:31.746 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:31.738+0000 7f5965de7740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563e97c3c390 0x563e98014500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:47:31.746 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:31.738+0000 7f5965de7740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563e97c5d070 0x563e9808b6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:47:31.747 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:31.738+0000 7f5965de7740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f59581b2bb0 con 0x563e97c341b0 2024-09-21T03:47:31.747 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:31.738+0000 7f5965de7740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f59581b24e0 con 0x563e97c5d070 2024-09-21T03:47:31.747 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:31.738+0000 7f5965de7740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f59581af720 con 0x563e97c3c390 2024-09-21T03:47:31.747 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:31.738+0000 7f5965de7740 10 client.0 _ll_register_callbacks cb 0x7f5950009c80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:47:31.747 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:31.738+0000 7f595d60f640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563e97c341b0 0x563e98013fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:47:31.747 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:31.738+0000 7f595d60f640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563e97c341b0 0x563e98013fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:36346/0 (socket says 172.21.15.39:36346) 2024-09-21T03:47:31.747 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:31.738+0000 7f595d60f640 1 -- 172.21.15.39:0/612016252 learned_addr learned my addr 172.21.15.39:0/612016252 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:47:31.747 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:31.738+0000 7f595ce0e640 1 --2- 172.21.15.39:0/612016252 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563e97c3c390 0x563e98014500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:47:31.747 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:31.738+0000 7f595de10640 1 --2- 172.21.15.39:0/612016252 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563e97c5d070 0x563e9808b6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:47:31.747 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:31.738+0000 7f595d60f640 1 -- 172.21.15.39:0/612016252 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563e97c3c390 msgr2=0x563e98014500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:47:31.747 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:31.738+0000 7f595d60f640 1 --2- 172.21.15.39:0/612016252 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563e97c3c390 0x563e98014500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:31.747 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:31.738+0000 7f595d60f640 1 -- 172.21.15.39:0/612016252 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563e97c5d070 msgr2=0x563e9808b6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:47:31.747 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:31.738+0000 7f595d60f640 1 --2- 172.21.15.39:0/612016252 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563e97c5d070 0x563e9808b6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:31.747 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:31.738+0000 7f595d60f640 1 -- 172.21.15.39:0/612016252 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563e98014d50 con 0x563e97c341b0 2024-09-21T03:47:31.748 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:31.738+0000 7f595d60f640 1 --2- 172.21.15.39:0/612016252 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563e97c341b0 0x563e98013fc0 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f5954005420 tx=0x7f5954005310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee49bf9c9e623f4c server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:47:31.748 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:31.738+0000 7f5940ff9640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-09-21T03:47:31.748 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:31.739+0000 7f5940ff9640 1 -- 172.21.15.39:0/612016252 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f595403f040 con 0x563e97c341b0 2024-09-21T03:47:31.748 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:31.739+0000 7f5940ff9640 1 -- 172.21.15.39:0/612016252 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8484+0+0 (secure 0 0 0) 0x7f5954031b10 con 0x563e97c341b0 2024-09-21T03:47:31.748 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:31.739+0000 7f5965de7740 1 -- 172.21.15.39:0/612016252 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f59480ad250 con 0x563e97c341b0 2024-09-21T03:47:31.748 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:31.739+0000 7f5940ff9640 1 -- 172.21.15.39:0/612016252 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f5954035040 con 0x563e97c341b0 2024-09-21T03:47:31.748 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:31.739+0000 7f5965de7740 10 client.15945 fetch_fsmap learned FSMap version 29 2024-09-21T03:47:31.748 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:31.739+0000 7f5965de7740 1 -- 172.21.15.39:0/612016252 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x563e9808cc00 con 0x563e97c341b0 2024-09-21T03:47:31.748 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:31.739+0000 7f5940ff9640 1 -- 172.21.15.39:0/612016252 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f59480ad250 con 0x563e97c341b0 2024-09-21T03:47:31.748 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:31.739+0000 7f5940ff9640 1 -- 172.21.15.39:0/612016252 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f595405a070 con 0x563e97c341b0 2024-09-21T03:47:31.748 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:31.739+0000 7f5965de7740 10 client.15945 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T03:47:31.748 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:31.739+0000 7f5965de7740 10 client.15945 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-21T03:47:31.748 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:31.739+0000 7f5965de7740 20 client.15945 populate_metadata read hostname 'smithi039' 2024-09-21T03:47:31.748 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:31.739+0000 7f5965de7740 1 --2- 172.21.15.39:0/612016252 >> [v2:172.21.15.150:6836/4017043144,v1:172.21.15.150:6837/4017043144] conn(0x563e97bf0da0 0x563e9808ef80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:47:31.749 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:31.739+0000 7f5965de7740 4 client.15945 mds_command: new command op to 24659 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T03:47:31.749 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:31.739+0000 7f5965de7740 1 -- 172.21.15.39:0/612016252 --> [v2:172.21.15.150:6836/4017043144,v1:172.21.15.150:6837/4017043144] -- command(tid 0: {"prefix": "damage ls"}) -- 0x563e97f788a0 con 0x563e97bf0da0 2024-09-21T03:47:31.749 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:31.740+0000 7f595ce0e640 1 --2- 172.21.15.39:0/612016252 >> [v2:172.21.15.150:6836/4017043144,v1:172.21.15.150:6837/4017043144] conn(0x563e97bf0da0 0x563e9808ef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:47:31.749 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:31.740+0000 7f595ce0e640 1 --2- 172.21.15.39:0/612016252 >> [v2:172.21.15.150:6836/4017043144,v1:172.21.15.150:6837/4017043144] conn(0x563e97bf0da0 0x563e9808ef80 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=dc016469ceccb6cd server_cookie=1dd3961f64fbe3da in_seq=0 out_seq=0 2024-09-21T03:47:31.749 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:31.740+0000 7f5940ff9640 10 client.15945 ms_handle_connect on v2:172.21.15.150:6836/4017043144 2024-09-21T03:47:31.749 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:31.740+0000 7f5940ff9640 1 -- 172.21.15.39:0/612016252 <== mds.2 v2:172.21.15.150:6836/4017043144 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x563e97f788a0 con 0x563e97bf0da0 2024-09-21T03:47:31.749 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:31.740+0000 7f5940ff9640 10 client.15945 handle_command_reply: tid=0 2024-09-21T03:47:31.749 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:31.740+0000 7f5965de7740 1 client.15945 shutdown 2024-09-21T03:47:31.749 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:31.740+0000 7f5965de7740 1 -- 172.21.15.39:0/612016252 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563e97c341b0 msgr2=0x563e98013fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:47:31.749 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:31.740+0000 7f5965de7740 1 --2- 172.21.15.39:0/612016252 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563e97c341b0 0x563e98013fc0 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f5954005420 tx=0x7f5954005310 comp rx=0 tx=0).stop 2024-09-21T03:47:31.749 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:31.740+0000 7f5965de7740 1 -- 172.21.15.39:0/612016252 shutdown_connections 2024-09-21T03:47:31.749 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:31.740+0000 7f5965de7740 1 --2- 172.21.15.39:0/612016252 >> [v2:172.21.15.150:6836/4017043144,v1:172.21.15.150:6837/4017043144] conn(0x563e97bf0da0 0x563e9808ef80 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:31.749 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:31.740+0000 7f5965de7740 1 --2- 172.21.15.39:0/612016252 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563e97c5d070 0x563e9808b6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:31.749 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:31.740+0000 7f5965de7740 1 --2- 172.21.15.39:0/612016252 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563e97c3c390 0x563e98014500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:31.750 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:31.740+0000 7f5965de7740 1 --2- 172.21.15.39:0/612016252 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563e97c341b0 0x563e98013fc0 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:31.750 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:31.740+0000 7f5940ff9640 0 client.15945 ms_handle_reset on v2:172.21.15.150:6836/4017043144 2024-09-21T03:47:31.750 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:31.740+0000 7f5965de7740 1 -- 172.21.15.39:0/612016252 >> 172.21.15.39:0/612016252 conn(0x563e97c51460 msgr2=0x563e98011740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:47:31.750 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:31.741+0000 7f5965de7740 1 -- 172.21.15.39:0/612016252 shutdown_connections 2024-09-21T03:47:31.750 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:31.741+0000 7f5965de7740 1 -- 172.21.15.39:0/612016252 wait complete. 2024-09-21T03:47:31.750 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:31.741+0000 7f5965de7740 20 client.15945 trim_cache size 0 max 16384 2024-09-21T03:47:31.750 INFO:teuthology.orchestra.run.smithi039.stdout:[] 2024-09-21T03:47:31.751 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:31.741+0000 7f595f873640 1 -- 172.21.15.39:0/3559820241 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f59440b6680 msgr2=0x7f59440b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:47:31.751 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:31.741+0000 7f595f873640 1 --2- 172.21.15.39:0/3559820241 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f59440b6680 0x7f59440b8b90 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f5950005fd0 tx=0x7f59500097e0 comp rx=0 tx=0).stop 2024-09-21T03:47:31.751 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:31.741+0000 7f595f873640 1 -- 172.21.15.39:0/3559820241 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f59581094a0 msgr2=0x7f5958075c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:47:31.751 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:31.741+0000 7f595f873640 1 --2- 172.21.15.39:0/3559820241 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f59581094a0 0x7f5958075c50 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f594800c8c0 tx=0x7f594800cd80 comp rx=0 tx=0).stop 2024-09-21T03:47:31.751 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:31.741+0000 7f595f873640 1 -- 172.21.15.39:0/3559820241 shutdown_connections 2024-09-21T03:47:31.751 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:31.741+0000 7f595f873640 1 --2- 172.21.15.39:0/3559820241 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f59440b6680 0x7f59440b8b90 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:31.751 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:31.741+0000 7f595f873640 1 --2- 172.21.15.39:0/3559820241 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5958109df0 0x7f5958076190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:31.751 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:31.741+0000 7f595f873640 1 --2- 172.21.15.39:0/3559820241 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f59581094a0 0x7f5958075c50 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:31.752 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:31.741+0000 7f595f873640 1 --2- 172.21.15.39:0/3559820241 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5958103b60 0x7f5958075710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:31.752 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:31.741+0000 7f595f873640 1 -- 172.21.15.39:0/3559820241 >> 172.21.15.39:0/3559820241 conn(0x7f595807b850 msgr2=0x7f5958107530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:47:31.752 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:31.742+0000 7f595f873640 1 -- 172.21.15.39:0/3559820241 shutdown_connections 2024-09-21T03:47:31.752 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:31.742+0000 7f595f873640 1 -- 172.21.15.39:0/3559820241 wait complete. 2024-09-21T03:47:32.551 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 51116d5c-77ca-11ef-baf5-efdc52797490 -- ceph tell mds.1:1 damage ls 2024-09-21T03:47:32.809 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/51116d5c-77ca-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:47:33.217 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:47:33 smithi039 ceph-mon[31211]: pgmap v370: 97 pgs: 97 active+clean; 725 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 60 MiB/s wr, 27 op/s 2024-09-21T03:47:33.315 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:33.313+0000 7fadf1917640 1 -- 172.21.15.39:0/961174124 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fadec108d20 msgr2=0x7fadec10f550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:47:33.315 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:33.313+0000 7fadf1917640 1 --2- 172.21.15.39:0/961174124 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fadec108d20 0x7fadec10f550 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7fade4009ca0 tx=0x7fade402a8c0 comp rx=0 tx=0).stop 2024-09-21T03:47:33.315 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:33.313+0000 7fadf1917640 1 -- 172.21.15.39:0/961174124 shutdown_connections 2024-09-21T03:47:33.315 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:33.313+0000 7fadf1917640 1 --2- 172.21.15.39:0/961174124 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fadec108d20 0x7fadec10f550 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:33.315 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:33.313+0000 7fadf1917640 1 --2- 172.21.15.39:0/961174124 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fadec1083d0 0x7fadec1087e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:33.316 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:33.313+0000 7fadf1917640 1 --2- 172.21.15.39:0/961174124 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fadec102ae0 0x7fadec102ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:33.316 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:33.313+0000 7fadf1917640 1 -- 172.21.15.39:0/961174124 >> 172.21.15.39:0/961174124 conn(0x7fadec079760 msgr2=0x7fadec079b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:47:33.316 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:33.313+0000 7fadf1917640 1 -- 172.21.15.39:0/961174124 shutdown_connections 2024-09-21T03:47:33.316 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:33.313+0000 7fadf1917640 1 -- 172.21.15.39:0/961174124 wait complete. 2024-09-21T03:47:33.317 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:33.314+0000 7fadf1917640 1 Processor -- start 2024-09-21T03:47:33.317 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:33.314+0000 7fadf1917640 1 -- start start 2024-09-21T03:47:33.317 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:33.314+0000 7fadf1917640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fadec102ae0 0x7fadec0ff670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:47:33.318 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:33.314+0000 7fadf1917640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fadec1083d0 0x7fadec0ffbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:47:33.318 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:33.314+0000 7fadf1917640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fadec108d20 0x7fadec1000f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:47:33.318 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:33.314+0000 7fadf1917640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fadec112990 con 0x7fadec102ae0 2024-09-21T03:47:33.318 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:33.314+0000 7fadf1917640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fadec112810 con 0x7fadec1083d0 2024-09-21T03:47:33.318 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:33.314+0000 7fadf1917640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fadec112b10 con 0x7fadec108d20 2024-09-21T03:47:33.318 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:33.315+0000 7fadeaffd640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fadec102ae0 0x7fadec0ff670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:47:33.318 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:33.315+0000 7fadeaffd640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fadec102ae0 0x7fadec0ff670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:36364/0 (socket says 172.21.15.39:36364) 2024-09-21T03:47:33.318 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:33.315+0000 7fadeaffd640 1 -- 172.21.15.39:0/4124535785 learned_addr learned my addr 172.21.15.39:0/4124535785 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:47:33.318 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:33.315+0000 7fadeb7fe640 1 --2- 172.21.15.39:0/4124535785 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fadec108d20 0x7fadec1000f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:47:33.318 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:33.315+0000 7fadea7fc640 1 --2- 172.21.15.39:0/4124535785 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fadec1083d0 0x7fadec0ffbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:47:33.318 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:33.315+0000 7fadeaffd640 1 -- 172.21.15.39:0/4124535785 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fadec108d20 msgr2=0x7fadec1000f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:47:33.318 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:33.315+0000 7fadeaffd640 1 --2- 172.21.15.39:0/4124535785 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fadec108d20 0x7fadec1000f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:33.319 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:33.315+0000 7fadeaffd640 1 -- 172.21.15.39:0/4124535785 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fadec1083d0 msgr2=0x7fadec0ffbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:47:33.319 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:33.315+0000 7fadeaffd640 1 --2- 172.21.15.39:0/4124535785 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fadec1083d0 0x7fadec0ffbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:33.319 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:33.315+0000 7fadeaffd640 1 -- 172.21.15.39:0/4124535785 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fadec1a86d0 con 0x7fadec102ae0 2024-09-21T03:47:33.319 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:33.315+0000 7fadeaffd640 1 --2- 172.21.15.39:0/4124535785 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fadec102ae0 0x7fadec0ff670 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7faddc00e9f0 tx=0x7faddc00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:47:33.319 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:33.315+0000 7fadf0915640 1 -- 172.21.15.39:0/4124535785 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faddc00cd80 con 0x7fadec102ae0 2024-09-21T03:47:33.319 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:33.315+0000 7fadf0915640 1 -- 172.21.15.39:0/4124535785 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faddc0108e0 con 0x7fadec102ae0 2024-09-21T03:47:33.319 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:33.315+0000 7fadf1917640 1 -- 172.21.15.39:0/4124535785 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fadec1a89c0 con 0x7fadec102ae0 2024-09-21T03:47:33.319 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:33.316+0000 7fadf0915640 1 -- 172.21.15.39:0/4124535785 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faddc018980 con 0x7fadec102ae0 2024-09-21T03:47:33.319 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:33.316+0000 7fadf1917640 1 -- 172.21.15.39:0/4124535785 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fadec1a8f00 con 0x7fadec102ae0 2024-09-21T03:47:33.320 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:33.318+0000 7fadf7e8b740 1 Processor -- start 2024-09-21T03:47:33.320 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:33.318+0000 7fadf7e8b740 1 -- start start 2024-09-21T03:47:33.321 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:33.319+0000 7fadf7e8b740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5604d5d43070 0x5604d60ea6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:47:33.321 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:33.319+0000 7fadf7e8b740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5604d5d1a1b0 0x5604d60eabe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:47:33.322 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:33.319+0000 7fadf7e8b740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5604d5d22390 0x5604d60eb120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:47:33.322 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:33.319+0000 7fadf7e8b740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x5604d6060ea0 con 0x5604d5d22390 2024-09-21T03:47:33.322 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:33.319+0000 7fadf7e8b740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x5604d60502d0 con 0x5604d5d43070 2024-09-21T03:47:33.322 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:33.319+0000 7fadf7e8b740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x5604d60540e0 con 0x5604d5d1a1b0 2024-09-21T03:47:33.322 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:33.319+0000 7fadea7fc640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5604d5d22390 0x5604d60eb120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:47:33.322 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:33.319+0000 7fadea7fc640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5604d5d22390 0x5604d60eb120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:36372/0 (socket says 172.21.15.39:36372) 2024-09-21T03:47:33.322 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:33.319+0000 7fadea7fc640 1 -- 172.21.15.39:0/3363456407 learned_addr learned my addr 172.21.15.39:0/3363456407 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:47:33.322 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:33.319+0000 7fadeb7fe640 1 --2- 172.21.15.39:0/3363456407 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5604d5d43070 0x5604d60ea6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:47:33.322 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:33.319+0000 7fadea7fc640 1 -- 172.21.15.39:0/3363456407 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5604d5d1a1b0 msgr2=0x5604d60eabe0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-21T03:47:33.322 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:33.319+0000 7fadea7fc640 1 --2- 172.21.15.39:0/3363456407 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5604d5d1a1b0 0x5604d60eabe0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:33.322 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:33.319+0000 7fadea7fc640 1 -- 172.21.15.39:0/3363456407 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5604d5d43070 msgr2=0x5604d60ea6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:47:33.322 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:33.319+0000 7fadea7fc640 1 --2- 172.21.15.39:0/3363456407 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5604d5d43070 0x5604d60ea6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:33.322 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:33.319+0000 7fadea7fc640 1 -- 172.21.15.39:0/3363456407 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5604d5dae650 con 0x5604d5d22390 2024-09-21T03:47:33.323 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:33.319+0000 7fadea7fc640 1 --2- 172.21.15.39:0/3363456407 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5604d5d22390 0x5604d60eb120 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7fade000b6c0 tx=0x7fade000bb80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a2879eb4616cafef server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:47:33.323 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:33.320+0000 7fadf0915640 1 -- 172.21.15.39:0/4124535785 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 27) ==== 154643+0+0 (secure 0 0 0) 0x7faddc010430 con 0x7fadec102ae0 2024-09-21T03:47:33.323 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:33.320+0000 7fadd1ffb640 1 -- 172.21.15.39:0/3363456407 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fade0003bd0 con 0x5604d5d22390 2024-09-21T03:47:33.323 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:33.320+0000 7fadd1ffb640 1 -- 172.21.15.39:0/3363456407 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fade0015070 con 0x5604d5d22390 2024-09-21T03:47:33.323 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:33.320+0000 7fadd1ffb640 1 -- 172.21.15.39:0/3363456407 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fade0004d10 con 0x5604d5d22390 2024-09-21T03:47:33.326 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:33.320+0000 7fadf0915640 1 --2- 172.21.15.39:0/4124535785 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7fadd40b6680 0x7fadd40b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:47:33.326 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:33.321+0000 7fadeb7fe640 1 --2- 172.21.15.39:0/4124535785 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7fadd40b6680 0x7fadd40b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:47:33.326 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:33.321+0000 7fadf7e8b740 1 -- 172.21.15.39:0/3363456407 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5604d5d22390 msgr2=0x5604d60eb120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:47:33.326 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:33.321+0000 7fadf7e8b740 1 --2- 172.21.15.39:0/3363456407 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5604d5d22390 0x5604d60eb120 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7fade000b6c0 tx=0x7fade000bb80 comp rx=0 tx=0).stop 2024-09-21T03:47:33.326 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:33.321+0000 7fadf0915640 1 -- 172.21.15.39:0/4124535785 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8484+0+0 (secure 0 0 0) 0x7faddc0e7a30 con 0x7fadec102ae0 2024-09-21T03:47:33.326 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:33.321+0000 7fadf7e8b740 1 -- 172.21.15.39:0/3363456407 shutdown_connections 2024-09-21T03:47:33.326 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:33.321+0000 7fadf7e8b740 1 --2- 172.21.15.39:0/3363456407 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5604d5d22390 0x5604d60eb120 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:33.326 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:33.321+0000 7fadf7e8b740 1 --2- 172.21.15.39:0/3363456407 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5604d5d1a1b0 0x5604d60eabe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:33.326 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:33.321+0000 7fadf7e8b740 1 --2- 172.21.15.39:0/3363456407 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5604d5d43070 0x5604d60ea6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:33.326 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:33.321+0000 7fadf7e8b740 1 -- 172.21.15.39:0/3363456407 >> 172.21.15.39:0/3363456407 conn(0x5604d5d37460 msgr2=0x5604d60e8590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:47:33.326 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:33.321+0000 7fadf7e8b740 1 -- 172.21.15.39:0/3363456407 shutdown_connections 2024-09-21T03:47:33.326 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:33.321+0000 7fadf7e8b740 1 -- 172.21.15.39:0/3363456407 wait complete. 2024-09-21T03:47:33.326 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:33.321+0000 7fadeb7fe640 1 --2- 172.21.15.39:0/4124535785 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7fadd40b6680 0x7fadd40b8b90 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fade402f500 tx=0x7fade402f310 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:47:33.327 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:33.321+0000 7fadf7e8b740 1 -- start start 2024-09-21T03:47:33.327 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:33.322+0000 7fadf7e8b740 1 Processor -- start 2024-09-21T03:47:33.327 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:33.322+0000 7fadf7e8b740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5604d5d1a1b0 0x5604d60f5d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:47:33.327 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:33.322+0000 7fadf7e8b740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5604d5d22390 0x5604d60f62d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:47:33.327 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:33.322+0000 7fadf7e8b740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5604d5d43070 0x5604d61021a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:47:33.327 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:33.322+0000 7fadf7e8b740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x5604d5eb1700 con 0x5604d5d1a1b0 2024-09-21T03:47:33.327 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:33.322+0000 7fadf7e8b740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x5604d5feb160 con 0x5604d5d22390 2024-09-21T03:47:33.327 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:33.322+0000 7fadf7e8b740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x5604d60f8e80 con 0x5604d5d43070 2024-09-21T03:47:33.327 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:33.322+0000 7fadf7e8b740 10 client.0 _ll_register_callbacks cb 0x5604d60049e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:47:33.327 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:33.322+0000 7fadeaffd640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5604d5d1a1b0 0x5604d60f5d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:47:33.327 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:33.322+0000 7fadeaffd640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5604d5d1a1b0 0x5604d60f5d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:36374/0 (socket says 172.21.15.39:36374) 2024-09-21T03:47:33.327 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:33.322+0000 7fadeaffd640 1 -- 172.21.15.39:0/2170029350 learned_addr learned my addr 172.21.15.39:0/2170029350 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:47:33.327 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:33.322+0000 7fadeb7fe640 1 --2- 172.21.15.39:0/2170029350 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5604d5d43070 0x5604d61021a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:47:33.328 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:33.323+0000 7fadeaffd640 1 -- 172.21.15.39:0/2170029350 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5604d5d43070 msgr2=0x5604d61021a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:47:33.328 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:33.323+0000 7fadeaffd640 1 --2- 172.21.15.39:0/2170029350 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5604d5d43070 0x5604d61021a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:33.328 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:33.323+0000 7fadeaffd640 1 -- 172.21.15.39:0/2170029350 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5604d5d22390 msgr2=0x5604d60f62d0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-21T03:47:33.328 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:33.323+0000 7fadeaffd640 1 --2- 172.21.15.39:0/2170029350 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5604d5d22390 0x5604d60f62d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:33.328 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:33.323+0000 7fadeaffd640 1 -- 172.21.15.39:0/2170029350 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5604d6018c90 con 0x5604d5d1a1b0 2024-09-21T03:47:33.328 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:33.323+0000 7fadeaffd640 1 --2- 172.21.15.39:0/2170029350 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5604d5d1a1b0 0x5604d60f5d90 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7faddc0e5250 tx=0x7faddc0ad2a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d952290cdcc4f42a server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:47:33.328 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:33.323+0000 7fadcaffd640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-09-21T03:47:33.328 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:33.323+0000 7fadcaffd640 1 -- 172.21.15.39:0/2170029350 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faddc0aedd0 con 0x5604d5d1a1b0 2024-09-21T03:47:33.328 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:33.323+0000 7fadcaffd640 1 -- 172.21.15.39:0/2170029350 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8484+0+0 (secure 0 0 0) 0x7faddc018b20 con 0x5604d5d1a1b0 2024-09-21T03:47:33.328 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:33.323+0000 7fadf7e8b740 1 -- 172.21.15.39:0/2170029350 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fade0015070 con 0x5604d5d1a1b0 2024-09-21T03:47:33.328 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:33.323+0000 7fadcaffd640 1 -- 172.21.15.39:0/2170029350 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7faddc0ad460 con 0x5604d5d1a1b0 2024-09-21T03:47:33.328 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:33.323+0000 7fadf7e8b740 10 client.15957 fetch_fsmap learned FSMap version 29 2024-09-21T03:47:33.328 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:33.323+0000 7fadf7e8b740 1 -- 172.21.15.39:0/2170029350 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5604d6104bf0 con 0x5604d5d1a1b0 2024-09-21T03:47:33.329 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:33.323+0000 7fadcaffd640 1 -- 172.21.15.39:0/2170029350 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fade0015070 con 0x5604d5d1a1b0 2024-09-21T03:47:33.329 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:33.324+0000 7fadcaffd640 1 -- 172.21.15.39:0/2170029350 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7faddc0ac620 con 0x5604d5d1a1b0 2024-09-21T03:47:33.329 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:33.324+0000 7fadf7e8b740 10 client.15957 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T03:47:33.329 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:33.324+0000 7fadf7e8b740 10 client.15957 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-21T03:47:33.329 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:33.324+0000 7fadf7e8b740 20 client.15957 populate_metadata read hostname 'smithi039' 2024-09-21T03:47:33.329 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:33.324+0000 7fadf7e8b740 1 --2- 172.21.15.39:0/2170029350 >> [v2:172.21.15.161:6838/2190107270,v1:172.21.15.161:6839/2190107270] conn(0x5604d5d019f0 0x5604d61080e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:47:33.329 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:33.324+0000 7fadf7e8b740 4 client.15957 mds_command: new command op to 24667 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:47:33.329 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:33.324+0000 7fadf7e8b740 1 -- 172.21.15.39:0/2170029350 --> [v2:172.21.15.161:6838/2190107270,v1:172.21.15.161:6839/2190107270] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5604d6062df0 con 0x5604d5d019f0 2024-09-21T03:47:33.329 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:33.324+0000 7fadea7fc640 1 --2- 172.21.15.39:0/2170029350 >> [v2:172.21.15.161:6838/2190107270,v1:172.21.15.161:6839/2190107270] conn(0x5604d5d019f0 0x5604d61080e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:47:33.330 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:33.325+0000 7fadcaffd640 10 client.15957 ms_handle_connect on v2:172.21.15.161:6838/2190107270 2024-09-21T03:47:33.330 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:33.325+0000 7fadea7fc640 1 --2- 172.21.15.39:0/2170029350 >> [v2:172.21.15.161:6838/2190107270,v1:172.21.15.161:6839/2190107270] conn(0x5604d5d019f0 0x5604d61080e0 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=cdb56556c94e33a5 server_cookie=afb9a9c0cd8b963b in_seq=0 out_seq=0 2024-09-21T03:47:33.330 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:33.325+0000 7fadcaffd640 1 -- 172.21.15.39:0/2170029350 <== mds.1 v2:172.21.15.161:6838/2190107270 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5604d6062df0 con 0x5604d5d019f0 2024-09-21T03:47:33.331 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:33.325+0000 7fadcaffd640 10 client.15957 handle_command_reply: tid=0 2024-09-21T03:47:33.331 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:33.325+0000 7fadf7e8b740 1 client.15957 shutdown 2024-09-21T03:47:33.331 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:33.325+0000 7fadf7e8b740 1 -- 172.21.15.39:0/2170029350 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5604d5d1a1b0 msgr2=0x5604d60f5d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:47:33.331 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:33.325+0000 7fadf7e8b740 1 --2- 172.21.15.39:0/2170029350 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5604d5d1a1b0 0x5604d60f5d90 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7faddc0e5250 tx=0x7faddc0ad2a0 comp rx=0 tx=0).stop 2024-09-21T03:47:33.331 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:33.326+0000 7fadf7e8b740 1 -- 172.21.15.39:0/2170029350 shutdown_connections 2024-09-21T03:47:33.331 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:33.326+0000 7fadf7e8b740 1 --2- 172.21.15.39:0/2170029350 >> [v2:172.21.15.161:6838/2190107270,v1:172.21.15.161:6839/2190107270] conn(0x5604d5d019f0 0x5604d61080e0 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:33.331 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:33.326+0000 7fadcaffd640 0 client.15957 ms_handle_reset on v2:172.21.15.161:6838/2190107270 2024-09-21T03:47:33.331 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:33.326+0000 7fadf7e8b740 1 --2- 172.21.15.39:0/2170029350 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5604d5d43070 0x5604d61021a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:33.331 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:33.326+0000 7fadf7e8b740 1 --2- 172.21.15.39:0/2170029350 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5604d5d22390 0x5604d60f62d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:33.331 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:33.326+0000 7fadf7e8b740 1 --2- 172.21.15.39:0/2170029350 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5604d5d1a1b0 0x5604d60f5d90 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:33.331 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:33.326+0000 7fadf7e8b740 1 -- 172.21.15.39:0/2170029350 >> 172.21.15.39:0/2170029350 conn(0x5604d5d37460 msgr2=0x5604d60ed830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:47:33.331 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:33.326+0000 7fadf7e8b740 1 -- 172.21.15.39:0/2170029350 shutdown_connections 2024-09-21T03:47:33.331 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:33.326+0000 7fadf7e8b740 1 -- 172.21.15.39:0/2170029350 wait complete. 2024-09-21T03:47:33.331 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:33.326+0000 7fadf7e8b740 20 client.15957 trim_cache size 0 max 16384 2024-09-21T03:47:33.342 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:33.339+0000 7fadf7e8b740 1 Processor -- start 2024-09-21T03:47:33.342 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:33.339+0000 7fadf7e8b740 1 -- start start 2024-09-21T03:47:33.342 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:33.339+0000 7fadf7e8b740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5604d5d019f0 0x5604d60ebde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:47:33.342 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:33.339+0000 7fadf7e8b740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5604d5d1a1b0 0x5604d60e7bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:47:33.342 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:33.339+0000 7fadf7e8b740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5604d5d22390 0x5604d60e8100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:47:33.342 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:33.339+0000 7fadf7e8b740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fadec1b1890 con 0x5604d5d22390 2024-09-21T03:47:33.342 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:33.339+0000 7fadf7e8b740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fadec1b33d0 con 0x5604d5d1a1b0 2024-09-21T03:47:33.342 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:33.339+0000 7fadf7e8b740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fadec1b1f60 con 0x5604d5d019f0 2024-09-21T03:47:33.342 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:33.340+0000 7fadeb7fe640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5604d5d22390 0x5604d60e8100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:47:33.342 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:33.340+0000 7fadeb7fe640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5604d5d22390 0x5604d60e8100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:36378/0 (socket says 172.21.15.39:36378) 2024-09-21T03:47:33.342 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:33.340+0000 7fadeb7fe640 1 -- 172.21.15.39:0/3382615062 learned_addr learned my addr 172.21.15.39:0/3382615062 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:47:33.342 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:33.340+0000 7fadeaffd640 1 --2- 172.21.15.39:0/3382615062 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5604d5d019f0 0x5604d60ebde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:47:33.342 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:33.340+0000 7fadea7fc640 1 --2- 172.21.15.39:0/3382615062 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5604d5d1a1b0 0x5604d60e7bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:47:33.343 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:33.340+0000 7fadeb7fe640 1 -- 172.21.15.39:0/3382615062 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5604d5d019f0 msgr2=0x5604d60ebde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:47:33.343 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:33.340+0000 7fadeb7fe640 1 --2- 172.21.15.39:0/3382615062 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5604d5d019f0 0x5604d60ebde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:33.343 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:33.340+0000 7fadeb7fe640 1 -- 172.21.15.39:0/3382615062 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5604d5d1a1b0 msgr2=0x5604d60e7bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:47:33.343 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:33.340+0000 7fadeb7fe640 1 --2- 172.21.15.39:0/3382615062 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5604d5d1a1b0 0x5604d60e7bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:33.343 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:33.340+0000 7fadeb7fe640 1 -- 172.21.15.39:0/3382615062 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5604d60f8370 con 0x5604d5d22390 2024-09-21T03:47:33.343 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:33.340+0000 7fadea7fc640 1 --2- 172.21.15.39:0/3382615062 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5604d5d1a1b0 0x5604d60e7bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:47:33.343 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:33.340+0000 7fadeb7fe640 1 --2- 172.21.15.39:0/3382615062 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5604d5d22390 0x5604d60e8100 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7fade4035000 tx=0x7fade4041000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=651a15eb12c346e1 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:47:33.345 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:33.340+0000 7fadd17fa640 1 -- 172.21.15.39:0/3382615062 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fade4003970 con 0x5604d5d22390 2024-09-21T03:47:33.345 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:33.340+0000 7fadd17fa640 1 -- 172.21.15.39:0/3382615062 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fade4036050 con 0x5604d5d22390 2024-09-21T03:47:33.345 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:33.340+0000 7fadd17fa640 1 -- 172.21.15.39:0/3382615062 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fade402f660 con 0x5604d5d22390 2024-09-21T03:47:33.345 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:33.341+0000 7fadf7e8b740 1 -- 172.21.15.39:0/3382615062 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5604d5d22390 msgr2=0x5604d60e8100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:47:33.345 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:33.341+0000 7fadf7e8b740 1 --2- 172.21.15.39:0/3382615062 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5604d5d22390 0x5604d60e8100 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7fade4035000 tx=0x7fade4041000 comp rx=0 tx=0).stop 2024-09-21T03:47:33.346 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:33.341+0000 7fadf7e8b740 1 -- 172.21.15.39:0/3382615062 shutdown_connections 2024-09-21T03:47:33.346 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:33.341+0000 7fadf7e8b740 1 --2- 172.21.15.39:0/3382615062 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5604d5d22390 0x5604d60e8100 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:33.346 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:33.341+0000 7fadf7e8b740 1 --2- 172.21.15.39:0/3382615062 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5604d5d1a1b0 0x5604d60e7bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:33.346 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:33.341+0000 7fadf7e8b740 1 --2- 172.21.15.39:0/3382615062 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5604d5d019f0 0x5604d60ebde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:33.346 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:33.341+0000 7fadf7e8b740 1 -- 172.21.15.39:0/3382615062 >> 172.21.15.39:0/3382615062 conn(0x5604d5d37460 msgr2=0x5604d60eb6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:47:33.346 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:33.341+0000 7fadf7e8b740 1 -- 172.21.15.39:0/3382615062 shutdown_connections 2024-09-21T03:47:33.346 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:33.342+0000 7fadf7e8b740 1 -- 172.21.15.39:0/3382615062 wait complete. 2024-09-21T03:47:33.346 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:33.342+0000 7fadf7e8b740 1 -- start start 2024-09-21T03:47:33.346 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:33.342+0000 7fadf7e8b740 1 Processor -- start 2024-09-21T03:47:33.346 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:33.342+0000 7fadf7e8b740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5604d5d019f0 0x5604d60fa000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:47:33.346 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:33.342+0000 7fadf7e8b740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5604d5d1a1b0 0x5604d60fa540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:47:33.346 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:33.342+0000 7fadf7e8b740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5604d5d22390 0x5604d60faa80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:47:33.346 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:33.342+0000 7fadf7e8b740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fadec1b2d00 con 0x5604d5d1a1b0 2024-09-21T03:47:33.347 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:33.342+0000 7fadf7e8b740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fadec1b2630 con 0x5604d5d019f0 2024-09-21T03:47:33.347 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:33.342+0000 7fadf7e8b740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fadec1af870 con 0x5604d5d22390 2024-09-21T03:47:33.347 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:33.343+0000 7fadf7e8b740 10 client.0 _ll_register_callbacks cb 0x7faddc009700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:47:33.347 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:33.343+0000 7fadea7fc640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5604d5d1a1b0 0x5604d60fa540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:47:33.347 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:33.343+0000 7fadea7fc640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5604d5d1a1b0 0x5604d60fa540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:36384/0 (socket says 172.21.15.39:36384) 2024-09-21T03:47:33.347 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:33.343+0000 7fadea7fc640 1 -- 172.21.15.39:0/1063101139 learned_addr learned my addr 172.21.15.39:0/1063101139 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:47:33.347 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:33.343+0000 7fadeaffd640 1 --2- 172.21.15.39:0/1063101139 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5604d5d019f0 0x5604d60fa000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:47:33.347 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:33.343+0000 7fadeb7fe640 1 --2- 172.21.15.39:0/1063101139 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5604d5d22390 0x5604d60faa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:47:33.347 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:33.343+0000 7fadea7fc640 1 -- 172.21.15.39:0/1063101139 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5604d5d22390 msgr2=0x5604d60faa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:47:33.347 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:33.343+0000 7fadea7fc640 1 --2- 172.21.15.39:0/1063101139 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5604d5d22390 0x5604d60faa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:33.347 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:33.343+0000 7fadea7fc640 1 -- 172.21.15.39:0/1063101139 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5604d5d019f0 msgr2=0x5604d60fa000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:47:33.347 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:33.343+0000 7fadea7fc640 1 --2- 172.21.15.39:0/1063101139 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5604d5d019f0 0x5604d60fa000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:33.347 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:33.343+0000 7fadea7fc640 1 -- 172.21.15.39:0/1063101139 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5604d6171a70 con 0x5604d5d1a1b0 2024-09-21T03:47:33.347 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:33.343+0000 7fadea7fc640 1 --2- 172.21.15.39:0/1063101139 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5604d5d1a1b0 0x5604d60fa540 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7fade000bbc0 tx=0x7fade00047e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b3a3a230a971179 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:47:33.347 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:33.343+0000 7fadcaffd640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-09-21T03:47:33.348 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:33.343+0000 7fadcaffd640 1 -- 172.21.15.39:0/1063101139 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fade0031060 con 0x5604d5d1a1b0 2024-09-21T03:47:33.348 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:33.344+0000 7fadcaffd640 1 -- 172.21.15.39:0/1063101139 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8484+0+0 (secure 0 0 0) 0x7fade0003de0 con 0x5604d5d1a1b0 2024-09-21T03:47:33.348 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:33.344+0000 7fadf7e8b740 1 -- 172.21.15.39:0/1063101139 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fade4036050 con 0x5604d5d1a1b0 2024-09-21T03:47:33.348 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:33.344+0000 7fadcaffd640 1 -- 172.21.15.39:0/1063101139 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fade0004290 con 0x5604d5d1a1b0 2024-09-21T03:47:33.348 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:33.344+0000 7fadf7e8b740 10 client.15963 fetch_fsmap learned FSMap version 29 2024-09-21T03:47:33.348 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:33.344+0000 7fadf7e8b740 1 -- 172.21.15.39:0/1063101139 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5604d6172c70 con 0x5604d5d1a1b0 2024-09-21T03:47:33.349 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:33.344+0000 7fadcaffd640 1 -- 172.21.15.39:0/1063101139 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fade4036050 con 0x5604d5d1a1b0 2024-09-21T03:47:33.349 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:33.344+0000 7fadcaffd640 1 -- 172.21.15.39:0/1063101139 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fade0019520 con 0x5604d5d1a1b0 2024-09-21T03:47:33.350 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:33.344+0000 7fadf7e8b740 10 client.15963 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T03:47:33.350 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:33.344+0000 7fadf7e8b740 10 client.15963 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-21T03:47:33.350 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:33.344+0000 7fadf7e8b740 20 client.15963 populate_metadata read hostname 'smithi039' 2024-09-21T03:47:33.350 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:33.344+0000 7fadf7e8b740 1 --2- 172.21.15.39:0/1063101139 >> [v2:172.21.15.161:6838/2190107270,v1:172.21.15.161:6839/2190107270] conn(0x5604d5d43070 0x5604d6175430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:47:33.350 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:33.344+0000 7fadf7e8b740 4 client.15963 mds_command: new command op to 24667 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T03:47:33.350 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:33.344+0000 7fadf7e8b740 1 -- 172.21.15.39:0/1063101139 --> [v2:172.21.15.161:6838/2190107270,v1:172.21.15.161:6839/2190107270] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5604d605e8a0 con 0x5604d5d43070 2024-09-21T03:47:33.350 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:33.345+0000 7fadeaffd640 1 --2- 172.21.15.39:0/1063101139 >> [v2:172.21.15.161:6838/2190107270,v1:172.21.15.161:6839/2190107270] conn(0x5604d5d43070 0x5604d6175430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:47:33.350 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:33.345+0000 7fadeaffd640 1 --2- 172.21.15.39:0/1063101139 >> [v2:172.21.15.161:6838/2190107270,v1:172.21.15.161:6839/2190107270] conn(0x5604d5d43070 0x5604d6175430 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=91c1238277026c46 server_cookie=730bac0a3a6ad8ca in_seq=0 out_seq=0 2024-09-21T03:47:33.350 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:33.345+0000 7fadcaffd640 10 client.15963 ms_handle_connect on v2:172.21.15.161:6838/2190107270 2024-09-21T03:47:33.350 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:33.345+0000 7fadcaffd640 1 -- 172.21.15.39:0/1063101139 <== mds.1 v2:172.21.15.161:6838/2190107270 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5604d605e8a0 con 0x5604d5d43070 2024-09-21T03:47:33.350 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:33.345+0000 7fadcaffd640 10 client.15963 handle_command_reply: tid=0 2024-09-21T03:47:33.350 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:33.345+0000 7fadf7e8b740 1 client.15963 shutdown 2024-09-21T03:47:33.350 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:33.345+0000 7fadf7e8b740 1 -- 172.21.15.39:0/1063101139 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5604d5d1a1b0 msgr2=0x5604d60fa540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:47:33.350 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:33.345+0000 7fadf7e8b740 1 --2- 172.21.15.39:0/1063101139 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5604d5d1a1b0 0x5604d60fa540 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7fade000bbc0 tx=0x7fade00047e0 comp rx=0 tx=0).stop 2024-09-21T03:47:33.350 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:33.346+0000 7fadf7e8b740 1 -- 172.21.15.39:0/1063101139 shutdown_connections 2024-09-21T03:47:33.351 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:33.346+0000 7fadf7e8b740 1 --2- 172.21.15.39:0/1063101139 >> [v2:172.21.15.161:6838/2190107270,v1:172.21.15.161:6839/2190107270] conn(0x5604d5d43070 0x5604d6175430 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:33.351 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:33.346+0000 7fadf7e8b740 1 --2- 172.21.15.39:0/1063101139 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5604d5d22390 0x5604d60faa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:33.351 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:33.346+0000 7fadf7e8b740 1 --2- 172.21.15.39:0/1063101139 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5604d5d1a1b0 0x5604d60fa540 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:33.351 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:33.346+0000 7fadf7e8b740 1 --2- 172.21.15.39:0/1063101139 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5604d5d019f0 0x5604d60fa000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:33.351 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:33.346+0000 7fadf7e8b740 1 -- 172.21.15.39:0/1063101139 >> 172.21.15.39:0/1063101139 conn(0x5604d5d37460 msgr2=0x5604d60eb660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:47:33.351 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:33.346+0000 7fadcaffd640 0 client.15963 ms_handle_reset on v2:172.21.15.161:6838/2190107270 2024-09-21T03:47:33.351 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:33.346+0000 7fadf7e8b740 1 -- 172.21.15.39:0/1063101139 shutdown_connections 2024-09-21T03:47:33.351 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:33.346+0000 7fadf7e8b740 1 -- 172.21.15.39:0/1063101139 wait complete. 2024-09-21T03:47:33.351 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:33.346+0000 7fadf7e8b740 20 client.15963 trim_cache size 0 max 16384 2024-09-21T03:47:33.351 INFO:teuthology.orchestra.run.smithi039.stdout:[] 2024-09-21T03:47:33.352 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:33.346+0000 7fadf1917640 1 -- 172.21.15.39:0/4124535785 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7fadd40b6680 msgr2=0x7fadd40b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:47:33.352 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:33.346+0000 7fadf1917640 1 --2- 172.21.15.39:0/4124535785 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7fadd40b6680 0x7fadd40b8b90 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fade402f500 tx=0x7fade402f310 comp rx=0 tx=0).stop 2024-09-21T03:47:33.352 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:33.346+0000 7fadf1917640 1 -- 172.21.15.39:0/4124535785 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fadec102ae0 msgr2=0x7fadec0ff670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:47:33.352 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:33.346+0000 7fadf1917640 1 --2- 172.21.15.39:0/4124535785 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fadec102ae0 0x7fadec0ff670 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7faddc00e9f0 tx=0x7faddc00eeb0 comp rx=0 tx=0).stop 2024-09-21T03:47:33.352 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:33.347+0000 7fadf1917640 1 -- 172.21.15.39:0/4124535785 shutdown_connections 2024-09-21T03:47:33.352 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:33.347+0000 7fadf1917640 1 --2- 172.21.15.39:0/4124535785 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7fadd40b6680 0x7fadd40b8b90 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:33.352 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:33.347+0000 7fadf1917640 1 --2- 172.21.15.39:0/4124535785 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fadec108d20 0x7fadec1000f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:33.352 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:33.347+0000 7fadf1917640 1 --2- 172.21.15.39:0/4124535785 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fadec1083d0 0x7fadec0ffbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:33.353 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:33.347+0000 7fadf1917640 1 --2- 172.21.15.39:0/4124535785 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fadec102ae0 0x7fadec0ff670 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:33.353 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:33.347+0000 7fadf1917640 1 -- 172.21.15.39:0/4124535785 >> 172.21.15.39:0/4124535785 conn(0x7fadec079760 msgr2=0x7fadec06be90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:47:33.353 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:33.347+0000 7fadf1917640 1 -- 172.21.15.39:0/4124535785 shutdown_connections 2024-09-21T03:47:33.353 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:33.347+0000 7fadf1917640 1 -- 172.21.15.39:0/4124535785 wait complete. 2024-09-21T03:47:33.468 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:47:33 smithi161 ceph-mon[34533]: pgmap v370: 97 pgs: 97 active+clean; 725 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 60 MiB/s wr, 27 op/s 2024-09-21T03:47:33.474 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:47:33 smithi150 ceph-mon[34289]: pgmap v370: 97 pgs: 97 active+clean; 725 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 60 MiB/s wr, 27 op/s 2024-09-21T03:47:34.384 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 51116d5c-77ca-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 damage ls 2024-09-21T03:47:34.758 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/51116d5c-77ca-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:47:34.942 INFO:tasks.workunit.client.0.smithi039.stdout: 2024-09-21T03:47:34.942 INFO:tasks.workunit.client.0.smithi039.stdout: Children see throughput for 1 rewriters = 706460.94 kB/sec 2024-09-21T03:47:34.942 INFO:tasks.workunit.client.0.smithi039.stdout: Parent sees throughput for 1 rewriters = 706376.67 kB/sec 2024-09-21T03:47:34.943 INFO:tasks.workunit.client.0.smithi039.stdout: Min throughput per process = 706460.94 kB/sec 2024-09-21T03:47:34.943 INFO:tasks.workunit.client.0.smithi039.stdout: Max throughput per process = 706460.94 kB/sec 2024-09-21T03:47:34.943 INFO:tasks.workunit.client.0.smithi039.stdout: Avg throughput per process = 706460.94 kB/sec 2024-09-21T03:47:34.943 INFO:tasks.workunit.client.0.smithi039.stdout: Min xfer = 1048576.00 kB 2024-09-21T03:47:35.153 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:47:35 smithi039 ceph-mon[31211]: pgmap v371: 97 pgs: 97 active+clean; 945 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 79 MiB/s wr, 31 op/s 2024-09-21T03:47:35.468 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:47:35 smithi161 ceph-mon[34533]: pgmap v371: 97 pgs: 97 active+clean; 945 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 79 MiB/s wr, 31 op/s 2024-09-21T03:47:35.474 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:47:35 smithi150 ceph-mon[34289]: pgmap v371: 97 pgs: 97 active+clean; 945 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 79 MiB/s wr, 31 op/s 2024-09-21T03:47:35.577 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:35.573+0000 7f099b224640 1 -- 172.21.15.39:0/1592957308 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0994104a30 msgr2=0x7f0994104e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:47:35.577 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:35.573+0000 7f099b224640 1 --2- 172.21.15.39:0/1592957308 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0994104a30 0x7f0994104e40 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f0984009900 tx=0x7f098402a410 comp rx=0 tx=0).stop 2024-09-21T03:47:35.577 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:35.573+0000 7f099b224640 1 -- 172.21.15.39:0/1592957308 shutdown_connections 2024-09-21T03:47:35.577 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:35.573+0000 7f099b224640 1 --2- 172.21.15.39:0/1592957308 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f099410de60 0x7f0994110330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:35.577 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:35.573+0000 7f099b224640 1 --2- 172.21.15.39:0/1592957308 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0994105380 0x7f099410d8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:35.577 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:35.573+0000 7f099b224640 1 --2- 172.21.15.39:0/1592957308 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0994104a30 0x7f0994104e40 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:35.577 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:35.573+0000 7f099b224640 1 -- 172.21.15.39:0/1592957308 >> 172.21.15.39:0/1592957308 conn(0x7f0994079760 msgr2=0x7f0994079b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:47:35.577 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:35.573+0000 7f099b224640 1 -- 172.21.15.39:0/1592957308 shutdown_connections 2024-09-21T03:47:35.577 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:35.573+0000 7f099b224640 1 -- 172.21.15.39:0/1592957308 wait complete. 2024-09-21T03:47:35.577 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:35.574+0000 7f099b224640 1 Processor -- start 2024-09-21T03:47:35.577 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:35.574+0000 7f099b224640 1 -- start start 2024-09-21T03:47:35.579 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:35.574+0000 7f099b224640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0994104a30 0x7f099419a510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:47:35.579 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:35.574+0000 7f099b224640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0994105380 0x7f099419aab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:47:35.579 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:35.574+0000 7f099b224640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f099410de60 0x7f099419fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:47:35.579 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:35.574+0000 7f099b224640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f0994113ac0 con 0x7f0994104a30 2024-09-21T03:47:35.580 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:35.574+0000 7f099b224640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f0994113940 con 0x7f099410de60 2024-09-21T03:47:35.580 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:35.574+0000 7f099b224640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f0994113c40 con 0x7f0994105380 2024-09-21T03:47:35.580 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:35.574+0000 7f0998fc0640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0994104a30 0x7f099419a510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:47:35.580 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:35.574+0000 7f0998fc0640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0994104a30 0x7f099419a510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:36394/0 (socket says 172.21.15.39:36394) 2024-09-21T03:47:35.580 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:35.575+0000 7f0998fc0640 1 -- 172.21.15.39:0/1476302276 learned_addr learned my addr 172.21.15.39:0/1476302276 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:47:35.580 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:35.575+0000 7f09997c1640 1 --2- 172.21.15.39:0/1476302276 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f099410de60 0x7f099419fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:47:35.580 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:35.575+0000 7f0993fff640 1 --2- 172.21.15.39:0/1476302276 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0994105380 0x7f099419aab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:47:35.580 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:35.575+0000 7f0998fc0640 1 -- 172.21.15.39:0/1476302276 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0994105380 msgr2=0x7f099419aab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:47:35.580 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:35.575+0000 7f0998fc0640 1 --2- 172.21.15.39:0/1476302276 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0994105380 0x7f099419aab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:35.580 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:35.575+0000 7f0998fc0640 1 -- 172.21.15.39:0/1476302276 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f099410de60 msgr2=0x7f099419fd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:47:35.580 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:35.575+0000 7f0998fc0640 1 --2- 172.21.15.39:0/1476302276 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f099410de60 0x7f099419fd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:35.580 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:35.575+0000 7f0998fc0640 1 -- 172.21.15.39:0/1476302276 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f09941a0480 con 0x7f0994104a30 2024-09-21T03:47:35.580 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:35.575+0000 7f0998fc0640 1 --2- 172.21.15.39:0/1476302276 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0994104a30 0x7f099419a510 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f0984032fd0 tx=0x7f0984004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:47:35.580 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:35.575+0000 7f0991ffb640 1 -- 172.21.15.39:0/1476302276 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f098403f030 con 0x7f0994104a30 2024-09-21T03:47:35.581 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:35.575+0000 7f099b224640 1 -- 172.21.15.39:0/1476302276 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f09941a0710 con 0x7f0994104a30 2024-09-21T03:47:35.581 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:35.575+0000 7f0991ffb640 1 -- 172.21.15.39:0/1476302276 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0984004490 con 0x7f0994104a30 2024-09-21T03:47:35.581 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:35.575+0000 7f0991ffb640 1 -- 172.21.15.39:0/1476302276 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0984035070 con 0x7f0994104a30 2024-09-21T03:47:35.581 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:35.576+0000 7f099b224640 1 -- 172.21.15.39:0/1476302276 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f09941a6ed0 con 0x7f0994104a30 2024-09-21T03:47:35.583 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:35.578+0000 7f09a1798740 1 Processor -- start 2024-09-21T03:47:35.583 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:35.578+0000 7f09a1798740 1 -- start start 2024-09-21T03:47:35.583 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:35.578+0000 7f09a1798740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5607652f2070 0x5607656992c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:47:35.583 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:35.579+0000 7f09a1798740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5607652c91b0 0x560765699800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:47:35.583 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:35.579+0000 7f09a1798740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5607652d1390 0x560765699d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:47:35.583 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:35.579+0000 7f0991ffb640 1 -- 172.21.15.39:0/1476302276 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 27) ==== 154643+0+0 (secure 0 0 0) 0x7f0984042030 con 0x7f0994104a30 2024-09-21T03:47:35.583 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:35.579+0000 7f09a1798740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x56076560fea0 con 0x5607652c91b0 2024-09-21T03:47:35.583 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:35.579+0000 7f09a1798740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x5607655ff2d0 con 0x5607652d1390 2024-09-21T03:47:35.583 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:35.579+0000 7f09a1798740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x5607656030e0 con 0x5607652f2070 2024-09-21T03:47:35.583 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:35.579+0000 7f0998fc0640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5607652c91b0 0x560765699800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:47:35.583 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:35.579+0000 7f09997c1640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5607652f2070 0x5607656992c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:47:35.583 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:35.579+0000 7f09997c1640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5607652f2070 0x5607656992c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.39:56504/0 (socket says 172.21.15.39:56504) 2024-09-21T03:47:35.584 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:35.579+0000 7f09997c1640 1 -- 172.21.15.39:0/1765560313 learned_addr learned my addr 172.21.15.39:0/1765560313 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:47:35.584 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:35.579+0000 7f0993fff640 1 --2- 172.21.15.39:0/1765560313 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5607652d1390 0x560765699d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:47:35.584 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:35.579+0000 7f09997c1640 1 -- 172.21.15.39:0/1765560313 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5607652d1390 msgr2=0x560765699d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:47:35.584 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:35.579+0000 7f09997c1640 1 --2- 172.21.15.39:0/1765560313 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5607652d1390 0x560765699d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:35.584 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:35.579+0000 7f09997c1640 1 -- 172.21.15.39:0/1765560313 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5607652c91b0 msgr2=0x560765699800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:47:35.584 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:35.579+0000 7f09997c1640 1 --2- 172.21.15.39:0/1765560313 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5607652c91b0 0x560765699800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:35.584 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:35.579+0000 7f09997c1640 1 -- 172.21.15.39:0/1765560313 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56076535d650 con 0x5607652f2070 2024-09-21T03:47:35.584 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:35.579+0000 7f0993fff640 1 --2- 172.21.15.39:0/1765560313 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5607652d1390 0x560765699d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:47:35.584 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:35.579+0000 7f09997c1640 1 --2- 172.21.15.39:0/1765560313 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5607652f2070 0x5607656992c0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f098c010370 tx=0x7f098c010830 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5bad9214d1ee5db5 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:47:35.584 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:35.580+0000 7f097affd640 1 -- 172.21.15.39:0/1765560313 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f098c016070 con 0x5607652f2070 2024-09-21T03:47:35.584 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:35.580+0000 7f097affd640 1 -- 172.21.15.39:0/1765560313 <== mon.1 v2:172.21.15.161:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f098c004d20 con 0x5607652f2070 2024-09-21T03:47:35.584 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:35.580+0000 7f097affd640 1 -- 172.21.15.39:0/1765560313 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f098c005420 con 0x5607652f2070 2024-09-21T03:47:35.584 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:35.580+0000 7f0991ffb640 1 --2- 172.21.15.39:0/1476302276 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f097c0b6680 0x7f097c0b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:47:35.584 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:35.580+0000 7f0998fc0640 1 --2- 172.21.15.39:0/1476302276 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f097c0b6680 0x7f097c0b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:47:35.585 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:35.580+0000 7f0991ffb640 1 -- 172.21.15.39:0/1476302276 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8484+0+0 (secure 0 0 0) 0x7f09841077b0 con 0x7f0994104a30 2024-09-21T03:47:35.587 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:35.581+0000 7f09a1798740 1 -- 172.21.15.39:0/1765560313 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5607652f2070 msgr2=0x5607656992c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:47:35.587 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:35.581+0000 7f09a1798740 1 --2- 172.21.15.39:0/1765560313 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5607652f2070 0x5607656992c0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f098c010370 tx=0x7f098c010830 comp rx=0 tx=0).stop 2024-09-21T03:47:35.588 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:35.581+0000 7f0998fc0640 1 --2- 172.21.15.39:0/1476302276 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f097c0b6680 0x7f097c0b8b90 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f0984104fd0 tx=0x7f098410d550 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:47:35.588 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:35.581+0000 7f09a1798740 1 -- 172.21.15.39:0/1765560313 shutdown_connections 2024-09-21T03:47:35.588 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:35.581+0000 7f09a1798740 1 --2- 172.21.15.39:0/1765560313 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5607652d1390 0x560765699d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:35.588 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:35.581+0000 7f09a1798740 1 --2- 172.21.15.39:0/1765560313 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5607652c91b0 0x560765699800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:35.588 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:35.581+0000 7f09a1798740 1 --2- 172.21.15.39:0/1765560313 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5607652f2070 0x5607656992c0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:35.588 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:35.581+0000 7f09a1798740 1 -- 172.21.15.39:0/1765560313 >> 172.21.15.39:0/1765560313 conn(0x5607652e6460 msgr2=0x5607656971b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:47:35.588 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:35.581+0000 7f09a1798740 1 -- 172.21.15.39:0/1765560313 shutdown_connections 2024-09-21T03:47:35.588 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:35.581+0000 7f09a1798740 1 -- 172.21.15.39:0/1765560313 wait complete. 2024-09-21T03:47:35.588 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:35.581+0000 7f09a1798740 1 -- start start 2024-09-21T03:47:35.588 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:35.581+0000 7f09a1798740 1 Processor -- start 2024-09-21T03:47:35.588 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:35.582+0000 7f09a1798740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5607652c91b0 0x5607656a4900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:47:35.588 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:35.582+0000 7f09a1798740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5607652d1390 0x5607656a4e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:47:35.588 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:35.582+0000 7f09a1798740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5607652f2070 0x5607656a53c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:47:35.588 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:35.582+0000 7f09a1798740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x560765460700 con 0x5607652f2070 2024-09-21T03:47:35.589 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:35.582+0000 7f09a1798740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x56076559a160 con 0x5607652d1390 2024-09-21T03:47:35.589 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:35.582+0000 7f09a1798740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x5607656a7aa0 con 0x5607652c91b0 2024-09-21T03:47:35.589 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:35.582+0000 7f09a1798740 10 client.0 _ll_register_callbacks cb 0x5607655b39e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:47:35.589 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:35.582+0000 7f09997c1640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5607652c91b0 0x5607656a4900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:47:35.589 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:35.582+0000 7f09997c1640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5607652c91b0 0x5607656a4900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.39:56514/0 (socket says 172.21.15.39:56514) 2024-09-21T03:47:35.589 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:35.582+0000 7f09997c1640 1 -- 172.21.15.39:0/1391393461 learned_addr learned my addr 172.21.15.39:0/1391393461 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:47:35.589 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:35.582+0000 7f09997c1640 1 --2- 172.21.15.39:0/1391393461 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5607652f2070 0x5607656a53c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:47:35.589 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:35.582+0000 7f0993fff640 1 --2- 172.21.15.39:0/1391393461 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5607652d1390 0x5607656a4e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:47:35.589 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:35.582+0000 7f09997c1640 1 -- 172.21.15.39:0/1391393461 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5607652d1390 msgr2=0x5607656a4e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:47:35.589 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:35.582+0000 7f09997c1640 1 --2- 172.21.15.39:0/1391393461 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5607652d1390 0x5607656a4e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:35.589 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:35.582+0000 7f09997c1640 1 -- 172.21.15.39:0/1391393461 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5607652f2070 msgr2=0x5607656a53c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:47:35.589 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:35.582+0000 7f09997c1640 1 --2- 172.21.15.39:0/1391393461 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5607652f2070 0x5607656a53c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:35.589 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:35.582+0000 7f09997c1640 1 -- 172.21.15.39:0/1391393461 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5607655c7c90 con 0x5607652c91b0 2024-09-21T03:47:35.589 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:35.582+0000 7f0993fff640 1 --2- 172.21.15.39:0/1391393461 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5607652d1390 0x5607656a4e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:47:35.589 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:35.583+0000 7f09997c1640 1 --2- 172.21.15.39:0/1391393461 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5607652c91b0 0x5607656a4900 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f098c004cd0 tx=0x7f098c012f00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d5ef06aea786a5df server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:47:35.590 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:35.583+0000 7f0973fff640 10 client.0 ms_handle_connect on v2:172.21.15.161:3300/0 2024-09-21T03:47:35.590 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:35.583+0000 7f0973fff640 1 -- 172.21.15.39:0/1391393461 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f098c016040 con 0x5607652c91b0 2024-09-21T03:47:35.590 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:35.583+0000 7f0973fff640 1 -- 172.21.15.39:0/1391393461 <== mon.1 v2:172.21.15.161:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8484+0+0 (secure 0 0 0) 0x7f098c005850 con 0x5607652c91b0 2024-09-21T03:47:35.590 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:35.583+0000 7f09a1798740 1 -- 172.21.15.39:0/1391393461 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f098c004d20 con 0x5607652c91b0 2024-09-21T03:47:35.590 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:35.583+0000 7f0973fff640 1 -- 172.21.15.39:0/1391393461 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f098c005a50 con 0x5607652c91b0 2024-09-21T03:47:35.590 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:35.583+0000 7f09a1798740 10 client.25573 fetch_fsmap learned FSMap version 29 2024-09-21T03:47:35.590 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:35.583+0000 7f09a1798740 1 -- 172.21.15.39:0/1391393461 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5607656b3740 con 0x5607652c91b0 2024-09-21T03:47:35.590 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:35.583+0000 7f0973fff640 1 -- 172.21.15.39:0/1391393461 <== mon.1 v2:172.21.15.161:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f098c004d20 con 0x5607652c91b0 2024-09-21T03:47:35.590 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:35.584+0000 7f0973fff640 1 -- 172.21.15.39:0/1391393461 <== mon.1 v2:172.21.15.161:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f098c031070 con 0x5607652c91b0 2024-09-21T03:47:35.590 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:35.584+0000 7f09a1798740 10 client.25573 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T03:47:35.590 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:35.584+0000 7f09a1798740 10 client.25573 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T03:47:35.590 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:35.584+0000 7f09a1798740 20 client.25573 populate_metadata read hostname 'smithi039' 2024-09-21T03:47:35.590 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:35.584+0000 7f09a1798740 1 --2- 172.21.15.39:0/1391393461 >> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] conn(0x5607652b09f0 0x5607656b6b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:47:35.591 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:35.584+0000 7f09a1798740 4 client.25573 mds_command: new command op to 24695 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:47:35.591 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:35.584+0000 7f09a1798740 1 -- 172.21.15.39:0/1391393461 --> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560765611df0 con 0x5607652b09f0 2024-09-21T03:47:35.591 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:35.584+0000 7f0993fff640 1 --2- 172.21.15.39:0/1391393461 >> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] conn(0x5607652b09f0 0x5607656b6b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:47:35.591 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:35.584+0000 7f0973fff640 10 client.25573 ms_handle_connect on v2:172.21.15.150:6838/2949386907 2024-09-21T03:47:35.591 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:35.584+0000 7f0993fff640 1 --2- 172.21.15.39:0/1391393461 >> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] conn(0x5607652b09f0 0x5607656b6b00 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b27f43623847a642 server_cookie=b7a48aa5c885cf8c in_seq=0 out_seq=0 2024-09-21T03:47:35.592 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:35.585+0000 7f0973fff640 1 -- 172.21.15.39:0/1391393461 <== mds.0 v2:172.21.15.150:6838/2949386907 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x560765611df0 con 0x5607652b09f0 2024-09-21T03:47:35.592 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:35.585+0000 7f0973fff640 10 client.25573 handle_command_reply: tid=0 2024-09-21T03:47:35.592 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:35.585+0000 7f09a1798740 1 client.25573 shutdown 2024-09-21T03:47:35.592 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:35.585+0000 7f09a1798740 1 -- 172.21.15.39:0/1391393461 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5607652c91b0 msgr2=0x5607656a4900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:47:35.592 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:35.585+0000 7f09a1798740 1 --2- 172.21.15.39:0/1391393461 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5607652c91b0 0x5607656a4900 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f098c004cd0 tx=0x7f098c012f00 comp rx=0 tx=0).stop 2024-09-21T03:47:35.592 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:35.586+0000 7f09a1798740 1 -- 172.21.15.39:0/1391393461 shutdown_connections 2024-09-21T03:47:35.592 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:35.586+0000 7f09a1798740 1 --2- 172.21.15.39:0/1391393461 >> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] conn(0x5607652b09f0 0x5607656b6b00 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:35.592 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:35.586+0000 7f09a1798740 1 --2- 172.21.15.39:0/1391393461 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5607652f2070 0x5607656a53c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:35.592 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:35.586+0000 7f0973fff640 0 client.25573 ms_handle_reset on v2:172.21.15.150:6838/2949386907 2024-09-21T03:47:35.592 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:35.586+0000 7f09a1798740 1 --2- 172.21.15.39:0/1391393461 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5607652d1390 0x5607656a4e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:35.592 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:35.586+0000 7f09a1798740 1 --2- 172.21.15.39:0/1391393461 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5607652c91b0 0x5607656a4900 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:35.593 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:35.586+0000 7f09a1798740 1 -- 172.21.15.39:0/1391393461 >> 172.21.15.39:0/1391393461 conn(0x5607652e6460 msgr2=0x56076569c450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:47:35.593 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:35.586+0000 7f09a1798740 1 -- 172.21.15.39:0/1391393461 shutdown_connections 2024-09-21T03:47:35.593 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:35.586+0000 7f09a1798740 1 -- 172.21.15.39:0/1391393461 wait complete. 2024-09-21T03:47:35.593 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:35.586+0000 7f09a1798740 20 client.25573 trim_cache size 0 max 16384 2024-09-21T03:47:35.602 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:35.599+0000 7f09a1798740 1 Processor -- start 2024-09-21T03:47:35.602 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:35.599+0000 7f09a1798740 1 -- start start 2024-09-21T03:47:35.602 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:35.599+0000 7f09a1798740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5607652b09f0 0x5607656a67e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:47:35.603 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:35.599+0000 7f09a1798740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5607652c91b0 0x56076569aa00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:47:35.603 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:35.599+0000 7f09a1798740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5607652d1390 0x560765696760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:47:35.603 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:35.599+0000 7f09a1798740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f09941af7c0 con 0x5607652b09f0 2024-09-21T03:47:35.603 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:35.599+0000 7f09a1798740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f09941b1300 con 0x5607652d1390 2024-09-21T03:47:35.603 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:35.599+0000 7f09a1798740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f09941afe90 con 0x5607652c91b0 2024-09-21T03:47:35.603 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:35.599+0000 7f09997c1640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5607652b09f0 0x5607656a67e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:47:35.603 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:35.599+0000 7f09997c1640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5607652b09f0 0x5607656a67e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:36424/0 (socket says 172.21.15.39:36424) 2024-09-21T03:47:35.603 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:35.600+0000 7f09997c1640 1 -- 172.21.15.39:0/2383127342 learned_addr learned my addr 172.21.15.39:0/2383127342 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:47:35.603 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:35.600+0000 7f0993fff640 1 --2- 172.21.15.39:0/2383127342 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5607652c91b0 0x56076569aa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:47:35.603 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:35.600+0000 7f09997c1640 1 -- 172.21.15.39:0/2383127342 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5607652c91b0 msgr2=0x56076569aa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:47:35.603 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:35.600+0000 7f09997c1640 1 --2- 172.21.15.39:0/2383127342 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5607652c91b0 0x56076569aa00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:35.603 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:35.600+0000 7f09997c1640 1 -- 172.21.15.39:0/2383127342 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5607652d1390 msgr2=0x560765696760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:47:35.604 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:35.600+0000 7f09997c1640 1 --2- 172.21.15.39:0/2383127342 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5607652d1390 0x560765696760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:35.604 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:35.600+0000 7f09997c1640 1 -- 172.21.15.39:0/2383127342 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56076569c120 con 0x5607652b09f0 2024-09-21T03:47:35.608 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:35.600+0000 7f09997c1640 1 --2- 172.21.15.39:0/2383127342 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5607652b09f0 0x5607656a67e0 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f098c002230 tx=0x7f098c004bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=98742ba603ecf7e8 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:47:35.608 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:35.600+0000 7f097a7fc640 1 -- 172.21.15.39:0/2383127342 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f098c037030 con 0x5607652b09f0 2024-09-21T03:47:35.608 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:35.600+0000 7f097a7fc640 1 -- 172.21.15.39:0/2383127342 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f098c012510 con 0x5607652b09f0 2024-09-21T03:47:35.608 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:35.600+0000 7f097a7fc640 1 -- 172.21.15.39:0/2383127342 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f098c016040 con 0x5607652b09f0 2024-09-21T03:47:35.608 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:35.601+0000 7f09a1798740 1 -- 172.21.15.39:0/2383127342 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5607652b09f0 msgr2=0x5607656a67e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:47:35.608 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:35.601+0000 7f09a1798740 1 --2- 172.21.15.39:0/2383127342 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5607652b09f0 0x5607656a67e0 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f098c002230 tx=0x7f098c004bc0 comp rx=0 tx=0).stop 2024-09-21T03:47:35.608 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:35.601+0000 7f09a1798740 1 -- 172.21.15.39:0/2383127342 shutdown_connections 2024-09-21T03:47:35.608 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:35.601+0000 7f09a1798740 1 --2- 172.21.15.39:0/2383127342 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5607652d1390 0x560765696760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:35.608 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:35.601+0000 7f09a1798740 1 --2- 172.21.15.39:0/2383127342 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5607652c91b0 0x56076569aa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:35.608 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:35.601+0000 7f09a1798740 1 --2- 172.21.15.39:0/2383127342 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5607652b09f0 0x5607656a67e0 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:35.609 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:35.601+0000 7f09a1798740 1 -- 172.21.15.39:0/2383127342 >> 172.21.15.39:0/2383127342 conn(0x5607652e6460 msgr2=0x56076569a2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:47:35.609 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:35.601+0000 7f09a1798740 1 -- 172.21.15.39:0/2383127342 shutdown_connections 2024-09-21T03:47:35.609 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:35.601+0000 7f09a1798740 1 -- 172.21.15.39:0/2383127342 wait complete. 2024-09-21T03:47:35.609 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:35.602+0000 7f09a1798740 1 -- start start 2024-09-21T03:47:35.609 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:35.602+0000 7f09a1798740 1 Processor -- start 2024-09-21T03:47:35.609 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:35.602+0000 7f09a1798740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5607652b09f0 0x5607656ad580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:47:35.609 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:35.602+0000 7f09a1798740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5607652c91b0 0x5607656a8a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:47:35.609 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:35.602+0000 7f09a1798740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5607652d1390 0x5607656a9060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:47:35.609 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:35.602+0000 7f09a1798740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f09941b0c30 con 0x5607652d1390 2024-09-21T03:47:35.609 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:35.602+0000 7f09a1798740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f09941b0560 con 0x5607652b09f0 2024-09-21T03:47:35.609 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:35.602+0000 7f09a1798740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f09941ad7a0 con 0x5607652c91b0 2024-09-21T03:47:35.609 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:35.602+0000 7f09a1798740 10 client.0 _ll_register_callbacks cb 0x7f098410d420 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:47:35.610 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:35.602+0000 7f09997c1640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5607652d1390 0x5607656a9060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:47:35.610 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:35.602+0000 7f09997c1640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5607652d1390 0x5607656a9060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:36440/0 (socket says 172.21.15.39:36440) 2024-09-21T03:47:35.610 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:35.602+0000 7f09997c1640 1 -- 172.21.15.39:0/3328752583 learned_addr learned my addr 172.21.15.39:0/3328752583 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:47:35.610 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:35.602+0000 7f0993fff640 1 --2- 172.21.15.39:0/3328752583 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5607652c91b0 0x5607656a8a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:47:35.610 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:35.603+0000 7f09997c1640 1 --2- 172.21.15.39:0/3328752583 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5607652b09f0 0x5607656ad580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:47:35.610 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:35.603+0000 7f0993fff640 1 -- 172.21.15.39:0/3328752583 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5607652b09f0 msgr2=0x5607656ad580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:47:35.610 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:35.603+0000 7f0993fff640 1 --2- 172.21.15.39:0/3328752583 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5607652b09f0 0x5607656ad580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:35.610 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:35.603+0000 7f0993fff640 1 -- 172.21.15.39:0/3328752583 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5607652d1390 msgr2=0x5607656a9060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:47:35.610 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:35.603+0000 7f0993fff640 1 --2- 172.21.15.39:0/3328752583 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5607652d1390 0x5607656a9060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:35.610 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:35.603+0000 7f0993fff640 1 -- 172.21.15.39:0/3328752583 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5607656a97f0 con 0x5607652c91b0 2024-09-21T03:47:35.610 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:35.603+0000 7f0993fff640 1 --2- 172.21.15.39:0/3328752583 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5607652c91b0 0x5607656a8a80 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x5607656b7070 tx=0x7f09880039c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f9581b4ca9741d29 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:47:35.610 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:35.603+0000 7f0973fff640 10 client.0 ms_handle_connect on v2:172.21.15.161:3300/0 2024-09-21T03:47:35.610 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:35.603+0000 7f0973fff640 1 -- 172.21.15.39:0/3328752583 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0988005020 con 0x5607652c91b0 2024-09-21T03:47:35.611 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:35.603+0000 7f0973fff640 1 -- 172.21.15.39:0/3328752583 <== mon.1 v2:172.21.15.161:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8484+0+0 (secure 0 0 0) 0x7f09880051c0 con 0x5607652c91b0 2024-09-21T03:47:35.611 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:35.603+0000 7f09a1798740 1 -- 172.21.15.39:0/3328752583 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f098c012510 con 0x5607652c91b0 2024-09-21T03:47:35.611 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:35.603+0000 7f0973fff640 1 -- 172.21.15.39:0/3328752583 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f09880053c0 con 0x5607652c91b0 2024-09-21T03:47:35.611 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:35.603+0000 7f09a1798740 10 client.25579 fetch_fsmap learned FSMap version 29 2024-09-21T03:47:35.611 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:35.603+0000 7f09a1798740 1 -- 172.21.15.39:0/3328752583 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x560765721340 con 0x5607652c91b0 2024-09-21T03:47:35.611 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:35.604+0000 7f0973fff640 1 -- 172.21.15.39:0/3328752583 <== mon.1 v2:172.21.15.161:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f098c012510 con 0x5607652c91b0 2024-09-21T03:47:35.611 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:35.604+0000 7f0973fff640 1 -- 172.21.15.39:0/3328752583 <== mon.1 v2:172.21.15.161:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f098802a030 con 0x5607652c91b0 2024-09-21T03:47:35.611 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:35.604+0000 7f09a1798740 10 client.25579 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T03:47:35.611 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:35.604+0000 7f09a1798740 10 client.25579 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T03:47:35.611 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:35.604+0000 7f09a1798740 20 client.25579 populate_metadata read hostname 'smithi039' 2024-09-21T03:47:35.611 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:35.604+0000 7f09a1798740 1 --2- 172.21.15.39:0/3328752583 >> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] conn(0x5607652f2070 0x560765723b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:47:35.611 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:35.604+0000 7f09a1798740 4 client.25579 mds_command: new command op to 24695 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T03:47:35.611 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:35.604+0000 7f09a1798740 1 -- 172.21.15.39:0/3328752583 --> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56076560d8a0 con 0x5607652f2070 2024-09-21T03:47:35.611 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:35.604+0000 7f09997c1640 1 --2- 172.21.15.39:0/3328752583 >> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] conn(0x5607652f2070 0x560765723b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:47:35.612 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:35.604+0000 7f0973fff640 10 client.25579 ms_handle_connect on v2:172.21.15.150:6838/2949386907 2024-09-21T03:47:35.612 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:35.604+0000 7f09997c1640 1 --2- 172.21.15.39:0/3328752583 >> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] conn(0x5607652f2070 0x560765723b00 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f94b06f7f444bd89 server_cookie=21ecaf546c235060 in_seq=0 out_seq=0 2024-09-21T03:47:35.612 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:35.605+0000 7f0973fff640 1 -- 172.21.15.39:0/3328752583 <== mds.0 v2:172.21.15.150:6838/2949386907 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56076560d8a0 con 0x5607652f2070 2024-09-21T03:47:35.612 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:35.605+0000 7f0973fff640 10 client.25579 handle_command_reply: tid=0 2024-09-21T03:47:35.612 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:35.605+0000 7f09a1798740 1 client.25579 shutdown 2024-09-21T03:47:35.612 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:35.605+0000 7f09a1798740 1 -- 172.21.15.39:0/3328752583 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5607652c91b0 msgr2=0x5607656a8a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:47:35.612 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:35.605+0000 7f09a1798740 1 --2- 172.21.15.39:0/3328752583 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5607652c91b0 0x5607656a8a80 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x5607656b7070 tx=0x7f09880039c0 comp rx=0 tx=0).stop 2024-09-21T03:47:35.612 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:35.605+0000 7f09a1798740 1 -- 172.21.15.39:0/3328752583 shutdown_connections 2024-09-21T03:47:35.612 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:35.605+0000 7f09a1798740 1 --2- 172.21.15.39:0/3328752583 >> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] conn(0x5607652f2070 0x560765723b00 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:35.612 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:35.605+0000 7f09a1798740 1 --2- 172.21.15.39:0/3328752583 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5607652d1390 0x5607656a9060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:35.612 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:35.605+0000 7f09a1798740 1 --2- 172.21.15.39:0/3328752583 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5607652c91b0 0x5607656a8a80 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:35.612 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:35.605+0000 7f0973fff640 0 client.25579 ms_handle_reset on v2:172.21.15.150:6838/2949386907 2024-09-21T03:47:35.612 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:35.605+0000 7f09a1798740 1 --2- 172.21.15.39:0/3328752583 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5607652b09f0 0x5607656ad580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:35.612 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:35.605+0000 7f09a1798740 1 -- 172.21.15.39:0/3328752583 >> 172.21.15.39:0/3328752583 conn(0x5607652e6460 msgr2=0x5607656b0410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:47:35.612 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:35.605+0000 7f09a1798740 1 -- 172.21.15.39:0/3328752583 shutdown_connections 2024-09-21T03:47:35.612 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:35.605+0000 7f09a1798740 1 -- 172.21.15.39:0/3328752583 wait complete. 2024-09-21T03:47:35.613 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:35.605+0000 7f09a1798740 20 client.25579 trim_cache size 0 max 16384 2024-09-21T03:47:35.613 INFO:teuthology.orchestra.run.smithi039.stdout:[] 2024-09-21T03:47:35.614 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:35.606+0000 7f099b224640 1 -- 172.21.15.39:0/1476302276 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f097c0b6680 msgr2=0x7f097c0b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:47:35.614 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:35.606+0000 7f099b224640 1 --2- 172.21.15.39:0/1476302276 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f097c0b6680 0x7f097c0b8b90 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f0984104fd0 tx=0x7f098410d550 comp rx=0 tx=0).stop 2024-09-21T03:47:35.614 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:35.606+0000 7f099b224640 1 -- 172.21.15.39:0/1476302276 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0994104a30 msgr2=0x7f099419a510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:47:35.614 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:35.606+0000 7f099b224640 1 --2- 172.21.15.39:0/1476302276 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0994104a30 0x7f099419a510 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f0984032fd0 tx=0x7f0984004270 comp rx=0 tx=0).stop 2024-09-21T03:47:35.614 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:35.606+0000 7f099b224640 1 -- 172.21.15.39:0/1476302276 shutdown_connections 2024-09-21T03:47:35.614 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:35.606+0000 7f099b224640 1 --2- 172.21.15.39:0/1476302276 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f097c0b6680 0x7f097c0b8b90 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:35.614 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:35.606+0000 7f099b224640 1 --2- 172.21.15.39:0/1476302276 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f099410de60 0x7f099419fd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:35.614 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:35.606+0000 7f099b224640 1 --2- 172.21.15.39:0/1476302276 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0994105380 0x7f099419aab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:35.614 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:35.606+0000 7f099b224640 1 --2- 172.21.15.39:0/1476302276 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0994104a30 0x7f099419a510 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:35.614 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:35.606+0000 7f099b224640 1 -- 172.21.15.39:0/1476302276 >> 172.21.15.39:0/1476302276 conn(0x7f0994079760 msgr2=0x7f09941008f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:47:35.614 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:35.606+0000 7f099b224640 1 -- 172.21.15.39:0/1476302276 shutdown_connections 2024-09-21T03:47:35.614 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:35.606+0000 7f099b224640 1 -- 172.21.15.39:0/1476302276 wait complete. 2024-09-21T03:47:37.304 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-21T03:47:37.304 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 51116d5c-77ca-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-21T03:47:37.415 INFO:tasks.workunit.client.0.smithi039.stdout: 2024-09-21T03:47:37.416 INFO:tasks.workunit.client.0.smithi039.stdout: Children see throughput for 1 readers = 6579127.00 kB/sec 2024-09-21T03:47:37.416 INFO:tasks.workunit.client.0.smithi039.stdout: Parent sees throughput for 1 readers = 6570968.51 kB/sec 2024-09-21T03:47:37.416 INFO:tasks.workunit.client.0.smithi039.stdout: Min throughput per process = 6579127.00 kB/sec 2024-09-21T03:47:37.416 INFO:tasks.workunit.client.0.smithi039.stdout: Max throughput per process = 6579127.00 kB/sec 2024-09-21T03:47:37.416 INFO:tasks.workunit.client.0.smithi039.stdout: Avg throughput per process = 6579127.00 kB/sec 2024-09-21T03:47:37.416 INFO:tasks.workunit.client.0.smithi039.stdout: Min xfer = 1048576.00 kB 2024-09-21T03:47:37.467 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:47:37 smithi161 ceph-mon[34533]: pgmap v372: 97 pgs: 97 active+clean; 1.0 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 114 MiB/s wr, 43 op/s 2024-09-21T03:47:37.475 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:47:37 smithi150 ceph-mon[34289]: pgmap v372: 97 pgs: 97 active+clean; 1.0 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 114 MiB/s wr, 43 op/s 2024-09-21T03:47:37.577 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/51116d5c-77ca-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:47:37.583 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:47:37 smithi039 ceph-mon[31211]: pgmap v372: 97 pgs: 97 active+clean; 1.0 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 114 MiB/s wr, 43 op/s 2024-09-21T03:47:38.071 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:38.057+0000 7f461ed08640 1 -- 172.21.15.39:0/574648980 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4618108cf0 msgr2=0x7f461810f520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:47:38.071 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:38.057+0000 7f461ed08640 1 --2- 172.21.15.39:0/574648980 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4618108cf0 0x7f461810f520 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f4610009ca0 tx=0x7f461002a8c0 comp rx=0 tx=0).stop 2024-09-21T03:47:38.071 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:38.057+0000 7f461ed08640 1 -- 172.21.15.39:0/574648980 shutdown_connections 2024-09-21T03:47:38.071 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:38.057+0000 7f461ed08640 1 --2- 172.21.15.39:0/574648980 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4618108cf0 0x7f461810f520 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:38.071 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:38.057+0000 7f461ed08640 1 --2- 172.21.15.39:0/574648980 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f46181083a0 0x7f46181087b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:38.071 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:38.057+0000 7f461ed08640 1 --2- 172.21.15.39:0/574648980 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4618102a60 0x7f4618102e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:38.071 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:38.057+0000 7f461ed08640 1 -- 172.21.15.39:0/574648980 >> 172.21.15.39:0/574648980 conn(0x7f4618079730 msgr2=0x7f4618079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:47:38.071 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:38.057+0000 7f461ed08640 1 -- 172.21.15.39:0/574648980 shutdown_connections 2024-09-21T03:47:38.071 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:38.057+0000 7f461ed08640 1 -- 172.21.15.39:0/574648980 wait complete. 2024-09-21T03:47:38.071 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:38.058+0000 7f461ed08640 1 Processor -- start 2024-09-21T03:47:38.072 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:38.058+0000 7f461ed08640 1 -- start start 2024-09-21T03:47:38.072 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:38.058+0000 7f461ed08640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4618102a60 0x7f46181992f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:47:38.072 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:38.058+0000 7f461ed08640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f46181083a0 0x7f4618199890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:47:38.072 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:38.058+0000 7f461ed08640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4618108cf0 0x7f461819eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:47:38.072 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:38.058+0000 7f461ed08640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f4618112960 con 0x7f4618102a60 2024-09-21T03:47:38.072 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:38.058+0000 7f461ed08640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f46181127e0 con 0x7f46181083a0 2024-09-21T03:47:38.072 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:38.058+0000 7f461ed08640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f4618112ae0 con 0x7f4618108cf0 2024-09-21T03:47:38.072 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:38.058+0000 7f461caa4640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4618102a60 0x7f46181992f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:47:38.072 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:38.058+0000 7f461caa4640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4618102a60 0x7f46181992f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:47296/0 (socket says 172.21.15.39:47296) 2024-09-21T03:47:38.072 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:38.058+0000 7f461caa4640 1 -- 172.21.15.39:0/2514137625 learned_addr learned my addr 172.21.15.39:0/2514137625 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:47:38.072 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:38.058+0000 7f4617fff640 1 --2- 172.21.15.39:0/2514137625 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f46181083a0 0x7f4618199890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:47:38.072 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:38.058+0000 7f461d2a5640 1 --2- 172.21.15.39:0/2514137625 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4618108cf0 0x7f461819eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:47:38.072 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:38.058+0000 7f461caa4640 1 -- 172.21.15.39:0/2514137625 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4618108cf0 msgr2=0x7f461819eb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:47:38.072 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:38.058+0000 7f461caa4640 1 --2- 172.21.15.39:0/2514137625 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4618108cf0 0x7f461819eb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:38.073 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:38.058+0000 7f461caa4640 1 -- 172.21.15.39:0/2514137625 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f46181083a0 msgr2=0x7f4618199890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:47:38.073 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:38.058+0000 7f461caa4640 1 --2- 172.21.15.39:0/2514137625 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f46181083a0 0x7f4618199890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:38.073 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:38.058+0000 7f461caa4640 1 -- 172.21.15.39:0/2514137625 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f461819f1a0 con 0x7f4618102a60 2024-09-21T03:47:38.073 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:38.058+0000 7f461d2a5640 1 --2- 172.21.15.39:0/2514137625 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4618108cf0 0x7f461819eb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:47:38.073 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:38.058+0000 7f4617fff640 1 --2- 172.21.15.39:0/2514137625 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f46181083a0 0x7f4618199890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:47:38.073 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:38.059+0000 7f461caa4640 1 --2- 172.21.15.39:0/2514137625 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4618102a60 0x7f46181992f0 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f460c00d950 tx=0x7f460c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:47:38.073 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:38.059+0000 7f4615ffb640 1 -- 172.21.15.39:0/2514137625 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f460c017070 con 0x7f4618102a60 2024-09-21T03:47:38.073 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:38.059+0000 7f461ed08640 1 -- 172.21.15.39:0/2514137625 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f461819f490 con 0x7f4618102a60 2024-09-21T03:47:38.073 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:38.059+0000 7f4615ffb640 1 -- 172.21.15.39:0/2514137625 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f460c009c30 con 0x7f4618102a60 2024-09-21T03:47:38.073 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:38.059+0000 7f4615ffb640 1 -- 172.21.15.39:0/2514137625 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f460c00f620 con 0x7f4618102a60 2024-09-21T03:47:38.073 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:38.059+0000 7f461ed08640 1 -- 172.21.15.39:0/2514137625 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f46181a5cb0 con 0x7f4618102a60 2024-09-21T03:47:38.073 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:38.062+0000 7f462527c740 1 Processor -- start 2024-09-21T03:47:38.073 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:38.062+0000 7f462527c740 1 -- start start 2024-09-21T03:47:38.073 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:38.062+0000 7f4615ffb640 1 -- 172.21.15.39:0/2514137625 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 27) ==== 154643+0+0 (secure 0 0 0) 0x7f460c010460 con 0x7f4618102a60 2024-09-21T03:47:38.074 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:38.062+0000 7f462527c740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a12d979350 0x55a12dd20430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:47:38.074 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:38.062+0000 7f462527c740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a12d950490 0x55a12dd20970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:47:38.074 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:38.062+0000 7f462527c740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a12d958670 0x55a12dd20eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:47:38.074 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:38.062+0000 7f462527c740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x55a12dc96e60 con 0x55a12d979350 2024-09-21T03:47:38.074 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:38.062+0000 7f462527c740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x55a12dc86290 con 0x55a12d958670 2024-09-21T03:47:38.074 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:38.062+0000 7f462527c740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55a12dc8a0a0 con 0x55a12d950490 2024-09-21T03:47:38.074 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:38.062+0000 7f461d2a5640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a12d979350 0x55a12dd20430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:47:38.074 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:38.062+0000 7f461d2a5640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a12d979350 0x55a12dd20430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:47300/0 (socket says 172.21.15.39:47300) 2024-09-21T03:47:38.074 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:38.062+0000 7f461d2a5640 1 -- 172.21.15.39:0/215624611 learned_addr learned my addr 172.21.15.39:0/215624611 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:47:38.074 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:38.062+0000 7f461caa4640 1 --2- 172.21.15.39:0/215624611 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a12d950490 0x55a12dd20970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:47:38.074 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:38.062+0000 7f4617fff640 1 --2- 172.21.15.39:0/215624611 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a12d958670 0x55a12dd20eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:47:38.074 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:38.063+0000 7f461d2a5640 1 -- 172.21.15.39:0/215624611 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a12d950490 msgr2=0x55a12dd20970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:47:38.075 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:38.063+0000 7f461d2a5640 1 --2- 172.21.15.39:0/215624611 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a12d950490 0x55a12dd20970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:38.075 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:38.063+0000 7f461d2a5640 1 -- 172.21.15.39:0/215624611 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a12d958670 msgr2=0x55a12dd20eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:47:38.075 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:38.063+0000 7f461d2a5640 1 --2- 172.21.15.39:0/215624611 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a12d958670 0x55a12dd20eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:38.075 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:38.063+0000 7f461d2a5640 1 -- 172.21.15.39:0/215624611 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a12d9e4930 con 0x55a12d979350 2024-09-21T03:47:38.075 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:38.063+0000 7f461caa4640 1 --2- 172.21.15.39:0/215624611 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a12d950490 0x55a12dd20970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:47:38.075 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:38.063+0000 7f4615ffb640 1 --2- 172.21.15.39:0/2514137625 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f46080b66b0 0x7f46080b8bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:47:38.075 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:38.063+0000 7f461d2a5640 1 --2- 172.21.15.39:0/215624611 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a12d979350 0x55a12dd20430 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f4610006010 tx=0x7f4610009880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b11ff2f88c942835 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:47:38.075 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:38.063+0000 7f461caa4640 1 --2- 172.21.15.39:0/2514137625 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f46080b66b0 0x7f46080b8bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:47:38.075 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:38.063+0000 7f4602ffd640 1 -- 172.21.15.39:0/215624611 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4610031d70 con 0x55a12d979350 2024-09-21T03:47:38.075 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:38.063+0000 7f4615ffb640 1 -- 172.21.15.39:0/2514137625 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8484+0+0 (secure 0 0 0) 0x7f460c0e73f0 con 0x7f4618102a60 2024-09-21T03:47:38.075 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:38.063+0000 7f4602ffd640 1 -- 172.21.15.39:0/215624611 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4610036070 con 0x55a12d979350 2024-09-21T03:47:38.075 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:38.063+0000 7f4602ffd640 1 -- 172.21.15.39:0/215624611 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46100042a0 con 0x55a12d979350 2024-09-21T03:47:38.075 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:38.064+0000 7f462527c740 1 -- 172.21.15.39:0/215624611 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a12d979350 msgr2=0x55a12dd20430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:47:38.075 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:38.064+0000 7f462527c740 1 --2- 172.21.15.39:0/215624611 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a12d979350 0x55a12dd20430 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f4610006010 tx=0x7f4610009880 comp rx=0 tx=0).stop 2024-09-21T03:47:38.076 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:38.064+0000 7f461caa4640 1 --2- 172.21.15.39:0/2514137625 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f46080b66b0 0x7f46080b8bc0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f460c0e4c10 tx=0x7f460c000bc0 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:47:38.076 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:38.064+0000 7f462527c740 1 -- 172.21.15.39:0/215624611 shutdown_connections 2024-09-21T03:47:38.076 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:38.064+0000 7f462527c740 1 --2- 172.21.15.39:0/215624611 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a12d958670 0x55a12dd20eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:38.076 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:38.064+0000 7f462527c740 1 --2- 172.21.15.39:0/215624611 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a12d950490 0x55a12dd20970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:38.076 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:38.064+0000 7f462527c740 1 --2- 172.21.15.39:0/215624611 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a12d979350 0x55a12dd20430 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:38.076 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:38.064+0000 7f462527c740 1 -- 172.21.15.39:0/215624611 >> 172.21.15.39:0/215624611 conn(0x55a12d96d740 msgr2=0x55a12dd1e320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:47:38.076 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:38.064+0000 7f462527c740 1 -- 172.21.15.39:0/215624611 shutdown_connections 2024-09-21T03:47:38.076 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:38.064+0000 7f462527c740 1 -- 172.21.15.39:0/215624611 wait complete. 2024-09-21T03:47:38.076 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:38.064+0000 7f462527c740 1 -- start start 2024-09-21T03:47:38.076 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:38.065+0000 7f462527c740 1 Processor -- start 2024-09-21T03:47:38.076 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:38.065+0000 7f462527c740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a12d950490 0x55a12dd2e160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:47:38.076 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:38.065+0000 7f462527c740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a12d958670 0x55a12dd2e6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:47:38.076 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:38.065+0000 7f462527c740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a12d979350 0x55a12dd3a040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:47:38.076 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:38.065+0000 7f462527c740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x55a12dae7890 con 0x55a12d979350 2024-09-21T03:47:38.077 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:38.065+0000 7f462527c740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x55a12dc20d40 con 0x55a12d958670 2024-09-21T03:47:38.077 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:38.065+0000 7f462527c740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55a12dd30d10 con 0x55a12d950490 2024-09-21T03:47:38.077 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:38.065+0000 7f462527c740 10 client.0 _ll_register_callbacks cb 0x55a12dc84c80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:47:38.077 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:38.065+0000 7f461d2a5640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a12d979350 0x55a12dd3a040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:47:38.077 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:38.065+0000 7f461d2a5640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a12d979350 0x55a12dd3a040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:47310/0 (socket says 172.21.15.39:47310) 2024-09-21T03:47:38.077 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:38.065+0000 7f461d2a5640 1 -- 172.21.15.39:0/2483829117 learned_addr learned my addr 172.21.15.39:0/2483829117 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:47:38.077 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:38.065+0000 7f461d2a5640 1 --2- 172.21.15.39:0/2483829117 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a12d950490 0x55a12dd2e160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:47:38.077 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:38.065+0000 7f4617fff640 1 --2- 172.21.15.39:0/2483829117 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a12d958670 0x55a12dd2e6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:47:38.077 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:38.066+0000 7f461d2a5640 1 -- 172.21.15.39:0/2483829117 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a12d950490 msgr2=0x55a12dd2e160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:47:38.077 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:38.066+0000 7f461d2a5640 1 --2- 172.21.15.39:0/2483829117 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a12d950490 0x55a12dd2e160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:38.077 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:38.066+0000 7f461d2a5640 1 -- 172.21.15.39:0/2483829117 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a12d958670 msgr2=0x55a12dd2e6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:47:38.077 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:38.066+0000 7f461d2a5640 1 --2- 172.21.15.39:0/2483829117 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a12d958670 0x55a12dd2e6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:38.077 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:38.066+0000 7f461d2a5640 1 -- 172.21.15.39:0/2483829117 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a12dd3a580 con 0x55a12d979350 2024-09-21T03:47:38.077 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:38.066+0000 7f461d2a5640 1 --2- 172.21.15.39:0/2483829117 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a12d979350 0x55a12dd3a040 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f4610003ce0 tx=0x7f461003d040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d517a9e934cd0511 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:47:38.078 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:38.066+0000 7f45f7fff640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-09-21T03:47:38.078 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:38.066+0000 7f45f7fff640 1 -- 172.21.15.39:0/2483829117 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4610058060 con 0x55a12d979350 2024-09-21T03:47:38.078 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:38.066+0000 7f45f7fff640 1 -- 172.21.15.39:0/2483829117 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8484+0+0 (secure 0 0 0) 0x7f4610031650 con 0x55a12d979350 2024-09-21T03:47:38.078 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:38.066+0000 7f462527c740 1 -- 172.21.15.39:0/2483829117 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4610036070 con 0x55a12d979350 2024-09-21T03:47:38.078 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:38.066+0000 7f45f7fff640 1 -- 172.21.15.39:0/2483829117 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f4610031850 con 0x55a12d979350 2024-09-21T03:47:38.078 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:38.066+0000 7f462527c740 10 client.15993 fetch_fsmap learned FSMap version 29 2024-09-21T03:47:38.078 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:38.066+0000 7f462527c740 1 -- 172.21.15.39:0/2483829117 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55a12dd3ce60 con 0x55a12d979350 2024-09-21T03:47:38.078 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:38.067+0000 7f45f7fff640 1 -- 172.21.15.39:0/2483829117 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4610036070 con 0x55a12d979350 2024-09-21T03:47:38.078 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:38.067+0000 7f45f7fff640 1 -- 172.21.15.39:0/2483829117 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f461002f660 con 0x55a12d979350 2024-09-21T03:47:38.078 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:38.067+0000 7f462527c740 10 client.15993 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T03:47:38.078 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:38.067+0000 7f462527c740 10 client.15993 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T03:47:38.078 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:38.067+0000 7f462527c740 20 client.15993 populate_metadata read hostname 'smithi039' 2024-09-21T03:47:38.078 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:38.067+0000 7f462527c740 1 --2- 172.21.15.39:0/2483829117 >> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] conn(0x55a12d937cd0 0x55a12dd401c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:47:38.078 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:38.067+0000 7f462527c740 4 client.15993 mds_command: new command op to 24695 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:47:38.079 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:38.067+0000 7f462527c740 1 -- 172.21.15.39:0/2483829117 --> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a12dc98db0 con 0x55a12d937cd0 2024-09-21T03:47:38.079 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:38.067+0000 7f4617fff640 1 --2- 172.21.15.39:0/2483829117 >> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] conn(0x55a12d937cd0 0x55a12dd401c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:47:38.079 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:38.068+0000 7f4617fff640 1 --2- 172.21.15.39:0/2483829117 >> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] conn(0x55a12d937cd0 0x55a12dd401c0 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4750e8a984734d3f server_cookie=e0deb68677cb0d87 in_seq=0 out_seq=0 2024-09-21T03:47:38.079 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:38.068+0000 7f45f7fff640 10 client.15993 ms_handle_connect on v2:172.21.15.150:6838/2949386907 2024-09-21T03:47:38.080 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:38.068+0000 7f45f7fff640 1 -- 172.21.15.39:0/2483829117 <== mds.0 v2:172.21.15.150:6838/2949386907 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55a12dc98db0 con 0x55a12d937cd0 2024-09-21T03:47:38.080 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:38.068+0000 7f45f7fff640 10 client.15993 handle_command_reply: tid=0 2024-09-21T03:47:38.080 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:38.068+0000 7f462527c740 1 client.15993 shutdown 2024-09-21T03:47:38.080 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:38.069+0000 7f462527c740 1 -- 172.21.15.39:0/2483829117 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a12d979350 msgr2=0x55a12dd3a040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:47:38.080 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:38.069+0000 7f462527c740 1 --2- 172.21.15.39:0/2483829117 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a12d979350 0x55a12dd3a040 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f4610003ce0 tx=0x7f461003d040 comp rx=0 tx=0).stop 2024-09-21T03:47:38.080 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:38.069+0000 7f462527c740 1 -- 172.21.15.39:0/2483829117 shutdown_connections 2024-09-21T03:47:38.080 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:38.069+0000 7f462527c740 1 --2- 172.21.15.39:0/2483829117 >> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] conn(0x55a12d937cd0 0x55a12dd401c0 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:38.080 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:38.069+0000 7f45f7fff640 0 client.15993 ms_handle_reset on v2:172.21.15.150:6838/2949386907 2024-09-21T03:47:38.081 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:38.069+0000 7f462527c740 1 --2- 172.21.15.39:0/2483829117 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a12d979350 0x55a12dd3a040 unknown :-1 s=CLOSED pgs=597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:38.081 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:38.069+0000 7f462527c740 1 --2- 172.21.15.39:0/2483829117 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a12d958670 0x55a12dd2e6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:38.081 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:38.069+0000 7f462527c740 1 --2- 172.21.15.39:0/2483829117 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a12d950490 0x55a12dd2e160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:38.081 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:38.069+0000 7f462527c740 1 -- 172.21.15.39:0/2483829117 >> 172.21.15.39:0/2483829117 conn(0x55a12d96d740 msgr2=0x55a12dd1c850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:47:38.081 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:38.069+0000 7f462527c740 1 -- 172.21.15.39:0/2483829117 shutdown_connections 2024-09-21T03:47:38.081 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:38.069+0000 7f462527c740 1 -- 172.21.15.39:0/2483829117 wait complete. 2024-09-21T03:47:38.081 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:38.069+0000 7f462527c740 20 client.15993 trim_cache size 0 max 16384 2024-09-21T03:47:38.090 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:38.087+0000 7f462527c740 1 Processor -- start 2024-09-21T03:47:38.091 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:38.087+0000 7f462527c740 1 -- start start 2024-09-21T03:47:38.091 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:38.087+0000 7f462527c740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a12d950490 0x55a12dd39fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:47:38.091 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:38.087+0000 7f462527c740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a12d958670 0x55a12dd1c850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:47:38.091 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:38.087+0000 7f462527c740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a12d979350 0x55a12dd1cd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:47:38.091 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:38.087+0000 7f462527c740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f46181ad3f0 con 0x55a12d958670 2024-09-21T03:47:38.091 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:38.087+0000 7f462527c740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f46181aef30 con 0x55a12d979350 2024-09-21T03:47:38.091 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:38.087+0000 7f462527c740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f46181adac0 con 0x55a12d950490 2024-09-21T03:47:38.091 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:38.087+0000 7f4617fff640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a12d958670 0x55a12dd1c850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:47:38.091 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:38.087+0000 7f4617fff640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a12d958670 0x55a12dd1c850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:47318/0 (socket says 172.21.15.39:47318) 2024-09-21T03:47:38.091 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:38.087+0000 7f4617fff640 1 -- 172.21.15.39:0/2016385419 learned_addr learned my addr 172.21.15.39:0/2016385419 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:47:38.091 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:38.087+0000 7f461d2a5640 1 --2- 172.21.15.39:0/2016385419 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a12d950490 0x55a12dd39fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:47:38.091 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:38.087+0000 7f461d2a5640 1 --2- 172.21.15.39:0/2016385419 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a12d979350 0x55a12dd1cd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:47:38.091 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:38.087+0000 7f4617fff640 1 -- 172.21.15.39:0/2016385419 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a12d950490 msgr2=0x55a12dd39fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:47:38.091 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:38.087+0000 7f4617fff640 1 --2- 172.21.15.39:0/2016385419 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a12d950490 0x55a12dd39fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:38.091 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:38.087+0000 7f4617fff640 1 -- 172.21.15.39:0/2016385419 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a12d979350 msgr2=0x55a12dd1cd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:47:38.092 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:38.087+0000 7f4617fff640 1 --2- 172.21.15.39:0/2016385419 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a12d979350 0x55a12dd1cd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:38.092 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:38.088+0000 7f4617fff640 1 -- 172.21.15.39:0/2016385419 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a12dd2f220 con 0x55a12d958670 2024-09-21T03:47:38.092 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:38.088+0000 7f4617fff640 1 --2- 172.21.15.39:0/2016385419 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a12d958670 0x55a12dd1c850 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f460400c880 tx=0x7f4604013830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ffff43ed1f9003e1 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:47:38.092 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:38.088+0000 7f46027fc640 1 -- 172.21.15.39:0/2016385419 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4604004050 con 0x55a12d958670 2024-09-21T03:47:38.092 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:38.088+0000 7f46027fc640 1 -- 172.21.15.39:0/2016385419 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f46040041f0 con 0x55a12d958670 2024-09-21T03:47:38.092 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:38.088+0000 7f46027fc640 1 -- 172.21.15.39:0/2016385419 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46040116a0 con 0x55a12d958670 2024-09-21T03:47:38.094 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:38.089+0000 7f462527c740 1 -- 172.21.15.39:0/2016385419 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a12d958670 msgr2=0x55a12dd1c850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:47:38.095 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:38.089+0000 7f462527c740 1 --2- 172.21.15.39:0/2016385419 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a12d958670 0x55a12dd1c850 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f460400c880 tx=0x7f4604013830 comp rx=0 tx=0).stop 2024-09-21T03:47:38.095 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:38.089+0000 7f462527c740 1 -- 172.21.15.39:0/2016385419 shutdown_connections 2024-09-21T03:47:38.095 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:38.089+0000 7f462527c740 1 --2- 172.21.15.39:0/2016385419 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a12d979350 0x55a12dd1cd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:38.095 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:38.089+0000 7f462527c740 1 --2- 172.21.15.39:0/2016385419 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a12d958670 0x55a12dd1c850 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:38.095 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:38.089+0000 7f462527c740 1 --2- 172.21.15.39:0/2016385419 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a12d950490 0x55a12dd39fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:38.095 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:38.089+0000 7f462527c740 1 -- 172.21.15.39:0/2016385419 >> 172.21.15.39:0/2016385419 conn(0x55a12d96d740 msgr2=0x55a12dd2c7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:47:38.095 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:38.089+0000 7f462527c740 1 -- 172.21.15.39:0/2016385419 shutdown_connections 2024-09-21T03:47:38.095 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:38.089+0000 7f462527c740 1 -- 172.21.15.39:0/2016385419 wait complete. 2024-09-21T03:47:38.095 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:38.089+0000 7f462527c740 1 -- start start 2024-09-21T03:47:38.095 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:38.089+0000 7f462527c740 1 Processor -- start 2024-09-21T03:47:38.095 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:38.090+0000 7f462527c740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a12d950490 0x55a12dd437d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:47:38.095 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:38.090+0000 7f462527c740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a12d958670 0x55a12dd3fc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:47:38.095 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:38.090+0000 7f462527c740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a12d979350 0x55a12dd401d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:47:38.095 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:38.090+0000 7f462527c740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f46181ae860 con 0x55a12d979350 2024-09-21T03:47:38.095 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:38.090+0000 7f462527c740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f46181ae190 con 0x55a12d958670 2024-09-21T03:47:38.096 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:38.090+0000 7f462527c740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f46181ab3d0 con 0x55a12d950490 2024-09-21T03:47:38.096 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:38.090+0000 7f462527c740 10 client.0 _ll_register_callbacks cb 0x7f4610009420 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:47:38.096 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:38.090+0000 7f461d2a5640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a12d979350 0x55a12dd401d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:47:38.096 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:38.090+0000 7f461d2a5640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a12d979350 0x55a12dd401d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:47326/0 (socket says 172.21.15.39:47326) 2024-09-21T03:47:38.096 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:38.090+0000 7f461d2a5640 1 -- 172.21.15.39:0/1940519527 learned_addr learned my addr 172.21.15.39:0/1940519527 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:47:38.096 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:38.090+0000 7f461d2a5640 1 --2- 172.21.15.39:0/1940519527 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a12d950490 0x55a12dd437d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:47:38.096 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:38.090+0000 7f4617fff640 1 --2- 172.21.15.39:0/1940519527 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a12d958670 0x55a12dd3fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:47:38.096 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:38.090+0000 7f461d2a5640 1 -- 172.21.15.39:0/1940519527 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a12d950490 msgr2=0x55a12dd437d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:47:38.096 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:38.090+0000 7f461d2a5640 1 --2- 172.21.15.39:0/1940519527 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a12d950490 0x55a12dd437d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:38.096 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:38.090+0000 7f461d2a5640 1 -- 172.21.15.39:0/1940519527 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a12d958670 msgr2=0x55a12dd3fc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:47:38.096 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:38.090+0000 7f461d2a5640 1 --2- 172.21.15.39:0/1940519527 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a12d958670 0x55a12dd3fc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:38.096 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:38.090+0000 7f461d2a5640 1 -- 172.21.15.39:0/1940519527 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a12dd40790 con 0x55a12d979350 2024-09-21T03:47:38.096 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:38.090+0000 7f461d2a5640 1 --2- 172.21.15.39:0/1940519527 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a12d979350 0x55a12dd401d0 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x55a12dd2ed10 tx=0x7f461003d040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df6ee57ed581c963 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:47:38.096 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:38.091+0000 7f45f7fff640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-09-21T03:47:38.097 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:38.091+0000 7f45f7fff640 1 -- 172.21.15.39:0/1940519527 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4610058040 con 0x55a12d979350 2024-09-21T03:47:38.097 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:38.091+0000 7f45f7fff640 1 -- 172.21.15.39:0/1940519527 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8484+0+0 (secure 0 0 0) 0x7f4610031450 con 0x55a12d979350 2024-09-21T03:47:38.097 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:38.091+0000 7f462527c740 1 -- 172.21.15.39:0/1940519527 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f46040041f0 con 0x55a12d979350 2024-09-21T03:47:38.097 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:38.091+0000 7f45f7fff640 1 -- 172.21.15.39:0/1940519527 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f4610009540 con 0x55a12d979350 2024-09-21T03:47:38.097 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:38.091+0000 7f462527c740 10 client.15999 fetch_fsmap learned FSMap version 29 2024-09-21T03:47:38.097 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:38.091+0000 7f462527c740 1 -- 172.21.15.39:0/1940519527 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55a12ddab6b0 con 0x55a12d979350 2024-09-21T03:47:38.097 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:38.091+0000 7f45f7fff640 1 -- 172.21.15.39:0/1940519527 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f46040041f0 con 0x55a12d979350 2024-09-21T03:47:38.097 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:38.091+0000 7f45f7fff640 1 -- 172.21.15.39:0/1940519527 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f4610004400 con 0x55a12d979350 2024-09-21T03:47:38.097 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:38.091+0000 7f462527c740 10 client.15999 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T03:47:38.097 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:38.091+0000 7f462527c740 10 client.15999 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T03:47:38.097 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:38.091+0000 7f462527c740 20 client.15999 populate_metadata read hostname 'smithi039' 2024-09-21T03:47:38.097 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:38.091+0000 7f462527c740 1 --2- 172.21.15.39:0/1940519527 >> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] conn(0x55a12d90d080 0x55a12ddadcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:47:38.097 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:38.091+0000 7f462527c740 4 client.15999 mds_command: new command op to 24695 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-21T03:47:38.097 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:38.091+0000 7f462527c740 1 -- 172.21.15.39:0/1940519527 --> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55a12dc94860 con 0x55a12d90d080 2024-09-21T03:47:38.098 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:38.092+0000 7f4617fff640 1 --2- 172.21.15.39:0/1940519527 >> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] conn(0x55a12d90d080 0x55a12ddadcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:47:38.098 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:38.092+0000 7f45f7fff640 10 client.15999 ms_handle_connect on v2:172.21.15.150:6838/2949386907 2024-09-21T03:47:38.098 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:38.092+0000 7f4617fff640 1 --2- 172.21.15.39:0/1940519527 >> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] conn(0x55a12d90d080 0x55a12ddadcd0 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d0d18a6796685f1a server_cookie=7710ad3773773aeb in_seq=0 out_seq=0 2024-09-21T03:47:38.099 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:38.094+0000 7f45f7fff640 1 -- 172.21.15.39:0/1940519527 <== mds.0 v2:172.21.15.150:6838/2949386907 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55a12dc94860 con 0x55a12d90d080 2024-09-21T03:47:38.099 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:38.094+0000 7f45f7fff640 10 client.15999 handle_command_reply: tid=0 2024-09-21T03:47:38.099 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:38.094+0000 7f462527c740 1 client.15999 shutdown 2024-09-21T03:47:38.099 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:38.094+0000 7f462527c740 1 -- 172.21.15.39:0/1940519527 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a12d979350 msgr2=0x55a12dd401d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:47:38.099 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:38.094+0000 7f462527c740 1 --2- 172.21.15.39:0/1940519527 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a12d979350 0x55a12dd401d0 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x55a12dd2ed10 tx=0x7f461003d040 comp rx=0 tx=0).stop 2024-09-21T03:47:38.099 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:38.094+0000 7f462527c740 1 -- 172.21.15.39:0/1940519527 shutdown_connections 2024-09-21T03:47:38.099 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:38.094+0000 7f462527c740 1 --2- 172.21.15.39:0/1940519527 >> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] conn(0x55a12d90d080 0x55a12ddadcd0 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:38.099 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:38.094+0000 7f462527c740 1 --2- 172.21.15.39:0/1940519527 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a12d979350 0x55a12dd401d0 unknown :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:38.099 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:38.094+0000 7f45f7fff640 0 client.15999 ms_handle_reset on v2:172.21.15.150:6838/2949386907 2024-09-21T03:47:38.099 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:38.094+0000 7f462527c740 1 --2- 172.21.15.39:0/1940519527 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a12d958670 0x55a12dd3fc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:38.099 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:38.094+0000 7f462527c740 1 --2- 172.21.15.39:0/1940519527 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a12d950490 0x55a12dd437d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:38.100 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:38.094+0000 7f462527c740 1 -- 172.21.15.39:0/1940519527 >> 172.21.15.39:0/1940519527 conn(0x55a12d96d740 msgr2=0x55a12dd1c850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:47:38.100 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:38.094+0000 7f462527c740 1 -- 172.21.15.39:0/1940519527 shutdown_connections 2024-09-21T03:47:38.100 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:38.094+0000 7f462527c740 1 -- 172.21.15.39:0/1940519527 wait complete. 2024-09-21T03:47:38.100 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:38.094+0000 7f462527c740 20 client.15999 trim_cache size 0 max 16384 2024-09-21T03:47:38.100 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-09-21T03:47:38.100 INFO:teuthology.orchestra.run.smithi039.stdout: "return_code": 0, 2024-09-21T03:47:38.100 INFO:teuthology.orchestra.run.smithi039.stdout: "scrub_tag": "29411307-eb8d-4339-8d2b-47ebd32c0080", 2024-09-21T03:47:38.100 INFO:teuthology.orchestra.run.smithi039.stdout: "mode": "asynchronous" 2024-09-21T03:47:38.100 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-09-21T03:47:38.101 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:38.095+0000 7f461ed08640 1 -- 172.21.15.39:0/2514137625 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f46080b66b0 msgr2=0x7f46080b8bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:47:38.101 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:38.095+0000 7f461ed08640 1 --2- 172.21.15.39:0/2514137625 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f46080b66b0 0x7f46080b8bc0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f460c0e4c10 tx=0x7f460c000bc0 comp rx=0 tx=0).stop 2024-09-21T03:47:38.101 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:38.095+0000 7f461ed08640 1 -- 172.21.15.39:0/2514137625 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4618102a60 msgr2=0x7f46181992f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:47:38.101 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:38.095+0000 7f461ed08640 1 --2- 172.21.15.39:0/2514137625 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4618102a60 0x7f46181992f0 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f460c00d950 tx=0x7f460c00de10 comp rx=0 tx=0).stop 2024-09-21T03:47:38.101 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:38.095+0000 7f461ed08640 1 -- 172.21.15.39:0/2514137625 shutdown_connections 2024-09-21T03:47:38.101 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:38.095+0000 7f461ed08640 1 --2- 172.21.15.39:0/2514137625 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f46080b66b0 0x7f46080b8bc0 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:38.101 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:38.095+0000 7f461ed08640 1 --2- 172.21.15.39:0/2514137625 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4618108cf0 0x7f461819eb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:38.102 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:38.095+0000 7f461ed08640 1 --2- 172.21.15.39:0/2514137625 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f46181083a0 0x7f4618199890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:38.102 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:38.095+0000 7f461ed08640 1 --2- 172.21.15.39:0/2514137625 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4618102a60 0x7f46181992f0 unknown :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:38.102 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:38.095+0000 7f461ed08640 1 -- 172.21.15.39:0/2514137625 >> 172.21.15.39:0/2514137625 conn(0x7f4618079730 msgr2=0x7f46181043b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:47:38.102 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:38.095+0000 7f461ed08640 1 -- 172.21.15.39:0/2514137625 shutdown_connections 2024-09-21T03:47:38.102 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:38.095+0000 7f461ed08640 1 -- 172.21.15.39:0/2514137625 wait complete. 2024-09-21T03:47:38.737 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 51116d5c-77ca-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub status 2024-09-21T03:47:38.996 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/51116d5c-77ca-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:47:39.462 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:47:39 smithi039 ceph-mon[31211]: pgmap v373: 97 pgs: 97 active+clean; 1.0 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 KiB/s rd, 122 MiB/s wr, 42 op/s 2024-09-21T03:47:39.463 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:47:39 smithi039 ceph-mon[31211]: scrub queued for path: / 2024-09-21T03:47:39.463 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:47:39 smithi039 ceph-mon[31211]: scrub summary: idle+waiting paths [/] 2024-09-21T03:47:39.463 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:47:39 smithi039 ceph-mon[31211]: scrub summary: active paths [/] 2024-09-21T03:47:39.463 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:47:39 smithi039 ceph-mon[31211]: scrub summary: idle+waiting paths [/] 2024-09-21T03:47:39.467 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:47:39 smithi161 ceph-mon[34533]: pgmap v373: 97 pgs: 97 active+clean; 1.0 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 KiB/s rd, 122 MiB/s wr, 42 op/s 2024-09-21T03:47:39.467 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:47:39 smithi161 ceph-mon[34533]: scrub queued for path: / 2024-09-21T03:47:39.468 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:47:39 smithi161 ceph-mon[34533]: scrub summary: idle+waiting paths [/] 2024-09-21T03:47:39.468 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:47:39 smithi161 ceph-mon[34533]: scrub summary: active paths [/] 2024-09-21T03:47:39.468 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:47:39 smithi161 ceph-mon[34533]: scrub summary: idle+waiting paths [/] 2024-09-21T03:47:39.474 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:47:39 smithi150 ceph-mon[34289]: pgmap v373: 97 pgs: 97 active+clean; 1.0 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 KiB/s rd, 122 MiB/s wr, 42 op/s 2024-09-21T03:47:39.474 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:47:39 smithi150 ceph-mon[34289]: scrub queued for path: / 2024-09-21T03:47:39.475 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:47:39 smithi150 ceph-mon[34289]: scrub summary: idle+waiting paths [/] 2024-09-21T03:47:39.475 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:47:39 smithi150 ceph-mon[34289]: scrub summary: active paths [/] 2024-09-21T03:47:39.475 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:47:39 smithi150 ceph-mon[34289]: scrub summary: idle+waiting paths [/] 2024-09-21T03:47:39.572 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:39.559+0000 7f145ef12640 1 -- 172.21.15.39:0/3380949508 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1458108d30 msgr2=0x7f145810f560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:47:39.572 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:39.559+0000 7f145ef12640 1 --2- 172.21.15.39:0/3380949508 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1458108d30 0x7f145810f560 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f1450009ca0 tx=0x7f145002a8c0 comp rx=0 tx=0).stop 2024-09-21T03:47:39.572 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:39.559+0000 7f145ef12640 1 -- 172.21.15.39:0/3380949508 shutdown_connections 2024-09-21T03:47:39.572 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:39.559+0000 7f145ef12640 1 --2- 172.21.15.39:0/3380949508 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1458108d30 0x7f145810f560 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:39.572 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:39.559+0000 7f145ef12640 1 --2- 172.21.15.39:0/3380949508 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f14581083e0 0x7f14581087f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:39.572 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:39.559+0000 7f145ef12640 1 --2- 172.21.15.39:0/3380949508 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1458102aa0 0x7f1458102eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:39.572 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:39.559+0000 7f145ef12640 1 -- 172.21.15.39:0/3380949508 >> 172.21.15.39:0/3380949508 conn(0x7f14580797e0 msgr2=0x7f1458079bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:47:39.572 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:39.559+0000 7f145ef12640 1 -- 172.21.15.39:0/3380949508 shutdown_connections 2024-09-21T03:47:39.572 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:39.559+0000 7f145ef12640 1 -- 172.21.15.39:0/3380949508 wait complete. 2024-09-21T03:47:39.572 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:39.559+0000 7f145ef12640 1 Processor -- start 2024-09-21T03:47:39.572 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:39.560+0000 7f145ef12640 1 -- start start 2024-09-21T03:47:39.573 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:39.560+0000 7f145ef12640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1458102aa0 0x7f1458199390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:47:39.573 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:39.560+0000 7f145ef12640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f14581083e0 0x7f1458199930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:47:39.573 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:39.560+0000 7f145ef12640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1458108d30 0x7f145819ebd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:47:39.573 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:39.560+0000 7f145ef12640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f14581129a0 con 0x7f1458102aa0 2024-09-21T03:47:39.573 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:39.560+0000 7f145ef12640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f1458112820 con 0x7f1458108d30 2024-09-21T03:47:39.573 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:39.560+0000 7f145ef12640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f1458112b20 con 0x7f14581083e0 2024-09-21T03:47:39.573 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:39.560+0000 7f145ccae640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1458102aa0 0x7f1458199390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:47:39.573 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:39.560+0000 7f145ccae640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1458102aa0 0x7f1458199390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:47338/0 (socket says 172.21.15.39:47338) 2024-09-21T03:47:39.573 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:39.560+0000 7f145ccae640 1 -- 172.21.15.39:0/2159983133 learned_addr learned my addr 172.21.15.39:0/2159983133 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:47:39.573 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:39.560+0000 7f1457fff640 1 --2- 172.21.15.39:0/2159983133 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f14581083e0 0x7f1458199930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:47:39.573 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:39.560+0000 7f145d4af640 1 --2- 172.21.15.39:0/2159983133 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1458108d30 0x7f145819ebd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:47:39.573 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:39.560+0000 7f145ccae640 1 -- 172.21.15.39:0/2159983133 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f14581083e0 msgr2=0x7f1458199930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:47:39.573 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:39.560+0000 7f145ccae640 1 --2- 172.21.15.39:0/2159983133 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f14581083e0 0x7f1458199930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:39.573 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:39.560+0000 7f145ccae640 1 -- 172.21.15.39:0/2159983133 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1458108d30 msgr2=0x7f145819ebd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:47:39.574 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:39.560+0000 7f145ccae640 1 --2- 172.21.15.39:0/2159983133 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1458108d30 0x7f145819ebd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:39.574 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:39.560+0000 7f145ccae640 1 -- 172.21.15.39:0/2159983133 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f145819f300 con 0x7f1458102aa0 2024-09-21T03:47:39.574 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:39.560+0000 7f1457fff640 1 --2- 172.21.15.39:0/2159983133 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f14581083e0 0x7f1458199930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:47:39.574 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:39.561+0000 7f145ccae640 1 --2- 172.21.15.39:0/2159983133 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1458102aa0 0x7f1458199390 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f144c00eb00 tx=0x7f144c009f20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:47:39.574 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:39.561+0000 7f1455ffb640 1 -- 172.21.15.39:0/2159983133 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f144c01c070 con 0x7f1458102aa0 2024-09-21T03:47:39.574 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:39.561+0000 7f1455ffb640 1 -- 172.21.15.39:0/2159983133 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f144c00bc30 con 0x7f1458102aa0 2024-09-21T03:47:39.574 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:39.561+0000 7f1455ffb640 1 -- 172.21.15.39:0/2159983133 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f144c016610 con 0x7f1458102aa0 2024-09-21T03:47:39.574 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:39.561+0000 7f145ef12640 1 -- 172.21.15.39:0/2159983133 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f145819f5f0 con 0x7f1458102aa0 2024-09-21T03:47:39.574 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:39.561+0000 7f145ef12640 1 -- 172.21.15.39:0/2159983133 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f145806b820 con 0x7f1458102aa0 2024-09-21T03:47:39.574 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:39.564+0000 7f1465486740 1 Processor -- start 2024-09-21T03:47:39.574 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:39.564+0000 7f1465486740 1 -- start start 2024-09-21T03:47:39.574 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:39.565+0000 7f1465486740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5651ec491070 0x5651ec838650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:47:39.574 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:39.565+0000 7f1465486740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5651ec4681b0 0x5651ec838b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:47:39.574 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:39.565+0000 7f1465486740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5651ec470390 0x5651ec8390d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:47:39.574 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:39.565+0000 7f1465486740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x5651ec7aee50 con 0x5651ec470390 2024-09-21T03:47:39.575 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:39.565+0000 7f1465486740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x5651ec79e280 con 0x5651ec491070 2024-09-21T03:47:39.575 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:39.565+0000 7f1465486740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x5651ec7a2090 con 0x5651ec4681b0 2024-09-21T03:47:39.575 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:39.565+0000 7f1457fff640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5651ec470390 0x5651ec8390d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:47:39.575 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:39.565+0000 7f1457fff640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5651ec470390 0x5651ec8390d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:47346/0 (socket says 172.21.15.39:47346) 2024-09-21T03:47:39.575 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:39.565+0000 7f1457fff640 1 -- 172.21.15.39:0/1845402524 learned_addr learned my addr 172.21.15.39:0/1845402524 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:47:39.575 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:39.565+0000 7f1455ffb640 1 -- 172.21.15.39:0/2159983133 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 27) ==== 154643+0+0 (secure 0 0 0) 0x7f144c01f070 con 0x7f1458102aa0 2024-09-21T03:47:39.575 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:39.565+0000 7f145d4af640 1 --2- 172.21.15.39:0/1845402524 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5651ec491070 0x5651ec838650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:47:39.575 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:39.565+0000 7f1457fff640 1 -- 172.21.15.39:0/1845402524 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5651ec4681b0 msgr2=0x5651ec838b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:47:39.575 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:39.565+0000 7f1457fff640 1 --2- 172.21.15.39:0/1845402524 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5651ec4681b0 0x5651ec838b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:39.575 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:39.565+0000 7f1457fff640 1 -- 172.21.15.39:0/1845402524 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5651ec491070 msgr2=0x5651ec838650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:47:39.575 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:39.565+0000 7f1457fff640 1 --2- 172.21.15.39:0/1845402524 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5651ec491070 0x5651ec838650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:39.575 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:39.565+0000 7f1457fff640 1 -- 172.21.15.39:0/1845402524 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5651ec4fc650 con 0x5651ec470390 2024-09-21T03:47:39.576 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:39.566+0000 7f1457fff640 1 --2- 172.21.15.39:0/1845402524 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5651ec470390 0x5651ec8390d0 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f14440114f0 tx=0x7f14440119b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f304b6a6a8d7cb0a server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:47:39.576 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:39.566+0000 7f1442ffd640 1 -- 172.21.15.39:0/1845402524 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1444005020 con 0x5651ec470390 2024-09-21T03:47:39.576 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:39.566+0000 7f1442ffd640 1 -- 172.21.15.39:0/1845402524 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f14440047a0 con 0x5651ec470390 2024-09-21T03:47:39.576 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:39.566+0000 7f1442ffd640 1 -- 172.21.15.39:0/1845402524 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f144400f880 con 0x5651ec470390 2024-09-21T03:47:39.576 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:39.566+0000 7f1455ffb640 1 --2- 172.21.15.39:0/2159983133 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f14480b6680 0x7f14480b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:47:39.576 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:39.566+0000 7f145d4af640 1 --2- 172.21.15.39:0/2159983133 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f14480b6680 0x7f14480b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:47:39.576 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:39.566+0000 7f1455ffb640 1 -- 172.21.15.39:0/2159983133 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8484+0+0 (secure 0 0 0) 0x7f144c0e8ce0 con 0x7f1458102aa0 2024-09-21T03:47:39.576 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:39.567+0000 7f1465486740 1 -- 172.21.15.39:0/1845402524 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5651ec470390 msgr2=0x5651ec8390d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:47:39.576 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:39.567+0000 7f145d4af640 1 --2- 172.21.15.39:0/2159983133 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f14480b6680 0x7f14480b8b90 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f1450005e30 tx=0x7f1450005c30 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:47:39.576 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:39.567+0000 7f1465486740 1 --2- 172.21.15.39:0/1845402524 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5651ec470390 0x5651ec8390d0 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f14440114f0 tx=0x7f14440119b0 comp rx=0 tx=0).stop 2024-09-21T03:47:39.576 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:39.567+0000 7f1465486740 1 -- 172.21.15.39:0/1845402524 shutdown_connections 2024-09-21T03:47:39.576 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:39.567+0000 7f1465486740 1 --2- 172.21.15.39:0/1845402524 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5651ec470390 0x5651ec8390d0 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:39.576 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:39.567+0000 7f1465486740 1 --2- 172.21.15.39:0/1845402524 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5651ec4681b0 0x5651ec838b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:39.577 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:39.567+0000 7f1465486740 1 --2- 172.21.15.39:0/1845402524 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5651ec491070 0x5651ec838650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:39.577 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:39.567+0000 7f1465486740 1 -- 172.21.15.39:0/1845402524 >> 172.21.15.39:0/1845402524 conn(0x5651ec485460 msgr2=0x5651ec836540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:47:39.577 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:39.567+0000 7f1465486740 1 -- 172.21.15.39:0/1845402524 shutdown_connections 2024-09-21T03:47:39.577 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:39.567+0000 7f1465486740 1 -- 172.21.15.39:0/1845402524 wait complete. 2024-09-21T03:47:39.577 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:39.567+0000 7f1465486740 1 -- start start 2024-09-21T03:47:39.577 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:39.568+0000 7f1465486740 1 Processor -- start 2024-09-21T03:47:39.577 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:39.568+0000 7f1465486740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5651ec4681b0 0x5651ec843cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:47:39.577 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:39.568+0000 7f1465486740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5651ec470390 0x5651ec844200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:47:39.577 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:39.568+0000 7f1465486740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5651ec491070 0x5651ec850150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:47:39.577 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:39.568+0000 7f1465486740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x5651ec5ff700 con 0x5651ec470390 2024-09-21T03:47:39.577 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:39.568+0000 7f1465486740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x5651ec739110 con 0x5651ec4681b0 2024-09-21T03:47:39.577 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:39.568+0000 7f1465486740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x5651ec846e30 con 0x5651ec491070 2024-09-21T03:47:39.577 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:39.568+0000 7f1465486740 10 client.0 _ll_register_callbacks cb 0x5651ec752990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:47:39.577 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:39.568+0000 7f1457fff640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5651ec470390 0x5651ec844200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:47:39.578 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:39.568+0000 7f1457fff640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5651ec470390 0x5651ec844200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:47348/0 (socket says 172.21.15.39:47348) 2024-09-21T03:47:39.578 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:39.568+0000 7f1457fff640 1 -- 172.21.15.39:0/57171602 learned_addr learned my addr 172.21.15.39:0/57171602 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:47:39.578 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:39.569+0000 7f145d4af640 1 --2- 172.21.15.39:0/57171602 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5651ec491070 0x5651ec850150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:47:39.578 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:39.569+0000 7f145ccae640 1 --2- 172.21.15.39:0/57171602 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5651ec4681b0 0x5651ec843cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:47:39.578 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:39.569+0000 7f1457fff640 1 -- 172.21.15.39:0/57171602 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5651ec491070 msgr2=0x5651ec850150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:47:39.578 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:39.569+0000 7f1457fff640 1 --2- 172.21.15.39:0/57171602 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5651ec491070 0x5651ec850150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:39.578 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:39.569+0000 7f1457fff640 1 -- 172.21.15.39:0/57171602 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5651ec4681b0 msgr2=0x5651ec843cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:47:39.578 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:39.569+0000 7f1457fff640 1 --2- 172.21.15.39:0/57171602 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5651ec4681b0 0x5651ec843cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:39.578 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:39.569+0000 7f1457fff640 1 -- 172.21.15.39:0/57171602 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5651ec766c40 con 0x5651ec470390 2024-09-21T03:47:39.578 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:39.569+0000 7f1457fff640 1 --2- 172.21.15.39:0/57171602 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5651ec470390 0x5651ec844200 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f144401b040 tx=0x7f1444005130 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec7b39e6ea43b67b server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:47:39.578 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:39.569+0000 7f1437fff640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-09-21T03:47:39.580 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:39.569+0000 7f1437fff640 1 -- 172.21.15.39:0/57171602 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1444028070 con 0x5651ec470390 2024-09-21T03:47:39.580 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:39.569+0000 7f1465486740 1 -- 172.21.15.39:0/57171602 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f14440047a0 con 0x5651ec470390 2024-09-21T03:47:39.581 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:39.569+0000 7f1437fff640 1 -- 172.21.15.39:0/57171602 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8484+0+0 (secure 0 0 0) 0x7f144400f620 con 0x5651ec470390 2024-09-21T03:47:39.581 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:39.570+0000 7f1437fff640 1 -- 172.21.15.39:0/57171602 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f1444034050 con 0x5651ec470390 2024-09-21T03:47:39.581 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:39.570+0000 7f1465486740 10 client.16011 fetch_fsmap learned FSMap version 29 2024-09-21T03:47:39.581 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:39.570+0000 7f1465486740 1 -- 172.21.15.39:0/57171602 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5651ec852b30 con 0x5651ec470390 2024-09-21T03:47:39.581 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:39.570+0000 7f1437fff640 1 -- 172.21.15.39:0/57171602 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f14440047a0 con 0x5651ec470390 2024-09-21T03:47:39.581 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:39.570+0000 7f1437fff640 1 -- 172.21.15.39:0/57171602 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f1444024670 con 0x5651ec470390 2024-09-21T03:47:39.581 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:39.570+0000 7f1465486740 10 client.16011 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T03:47:39.581 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:39.570+0000 7f1465486740 10 client.16011 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T03:47:39.581 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:39.570+0000 7f1465486740 20 client.16011 populate_metadata read hostname 'smithi039' 2024-09-21T03:47:39.581 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:39.570+0000 7f1465486740 1 --2- 172.21.15.39:0/57171602 >> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] conn(0x5651ec44f9f0 0x5651ec855ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:47:39.581 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:39.570+0000 7f1465486740 4 client.16011 mds_command: new command op to 24695 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:47:39.581 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:39.570+0000 7f1465486740 1 -- 172.21.15.39:0/57171602 --> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5651ec7b0da0 con 0x5651ec44f9f0 2024-09-21T03:47:39.581 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:39.571+0000 7f145ccae640 1 --2- 172.21.15.39:0/57171602 >> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] conn(0x5651ec44f9f0 0x5651ec855ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:47:39.582 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:39.571+0000 7f145ccae640 1 --2- 172.21.15.39:0/57171602 >> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] conn(0x5651ec44f9f0 0x5651ec855ef0 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f93f04d54bb47804 server_cookie=8ff32ab18b258b02 in_seq=0 out_seq=0 2024-09-21T03:47:39.582 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:39.571+0000 7f1437fff640 10 client.16011 ms_handle_connect on v2:172.21.15.150:6838/2949386907 2024-09-21T03:47:39.582 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:39.572+0000 7f1437fff640 1 -- 172.21.15.39:0/57171602 <== mds.0 v2:172.21.15.150:6838/2949386907 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5651ec7b0da0 con 0x5651ec44f9f0 2024-09-21T03:47:39.582 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:39.572+0000 7f1437fff640 10 client.16011 handle_command_reply: tid=0 2024-09-21T03:47:39.582 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:39.572+0000 7f1465486740 1 client.16011 shutdown 2024-09-21T03:47:39.582 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:39.572+0000 7f1465486740 1 -- 172.21.15.39:0/57171602 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5651ec470390 msgr2=0x5651ec844200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:47:39.582 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:39.572+0000 7f1465486740 1 --2- 172.21.15.39:0/57171602 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5651ec470390 0x5651ec844200 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f144401b040 tx=0x7f1444005130 comp rx=0 tx=0).stop 2024-09-21T03:47:39.582 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:39.572+0000 7f1465486740 1 -- 172.21.15.39:0/57171602 shutdown_connections 2024-09-21T03:47:39.582 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:39.572+0000 7f1465486740 1 --2- 172.21.15.39:0/57171602 >> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] conn(0x5651ec44f9f0 0x5651ec855ef0 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:39.582 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:39.572+0000 7f1465486740 1 --2- 172.21.15.39:0/57171602 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5651ec491070 0x5651ec850150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:39.582 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:39.572+0000 7f1465486740 1 --2- 172.21.15.39:0/57171602 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5651ec470390 0x5651ec844200 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:39.582 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:39.572+0000 7f1465486740 1 --2- 172.21.15.39:0/57171602 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5651ec4681b0 0x5651ec843cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:39.582 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:39.572+0000 7f1437fff640 0 client.16011 ms_handle_reset on v2:172.21.15.150:6838/2949386907 2024-09-21T03:47:39.582 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:39.572+0000 7f1465486740 1 -- 172.21.15.39:0/57171602 >> 172.21.15.39:0/57171602 conn(0x5651ec485460 msgr2=0x5651ec83b7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:47:39.582 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:39.572+0000 7f1465486740 1 -- 172.21.15.39:0/57171602 shutdown_connections 2024-09-21T03:47:39.583 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:39.572+0000 7f1465486740 1 -- 172.21.15.39:0/57171602 wait complete. 2024-09-21T03:47:39.583 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:39.573+0000 7f1465486740 20 client.16011 trim_cache size 0 max 16384 2024-09-21T03:47:39.594 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:39.591+0000 7f1465486740 1 Processor -- start 2024-09-21T03:47:39.594 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:39.591+0000 7f1465486740 1 -- start start 2024-09-21T03:47:39.594 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:39.591+0000 7f1465486740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5651ec4681b0 0x5651ec83a040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:47:39.594 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:39.591+0000 7f1465486740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5651ec470390 0x5651ec8458f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:47:39.594 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:39.591+0000 7f1465486740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5651ec491070 0x5651ec835b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:47:39.594 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:39.591+0000 7f1465486740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f14581ad510 con 0x5651ec491070 2024-09-21T03:47:39.594 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:39.591+0000 7f1465486740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f14581af050 con 0x5651ec4681b0 2024-09-21T03:47:39.594 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:39.591+0000 7f1465486740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f14581adbe0 con 0x5651ec470390 2024-09-21T03:47:39.595 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:39.591+0000 7f145d4af640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5651ec491070 0x5651ec835b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:47:39.595 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:39.591+0000 7f145d4af640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5651ec491070 0x5651ec835b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:47356/0 (socket says 172.21.15.39:47356) 2024-09-21T03:47:39.595 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:39.591+0000 7f145d4af640 1 -- 172.21.15.39:0/1990174072 learned_addr learned my addr 172.21.15.39:0/1990174072 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:47:39.595 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:39.591+0000 7f1457fff640 1 --2- 172.21.15.39:0/1990174072 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5651ec470390 0x5651ec8458f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:47:39.595 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:39.591+0000 7f145ccae640 1 --2- 172.21.15.39:0/1990174072 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5651ec4681b0 0x5651ec83a040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:47:39.596 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:39.592+0000 7f145d4af640 1 -- 172.21.15.39:0/1990174072 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5651ec470390 msgr2=0x5651ec8458f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:47:39.596 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:39.592+0000 7f145d4af640 1 --2- 172.21.15.39:0/1990174072 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5651ec470390 0x5651ec8458f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:39.596 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:39.592+0000 7f145d4af640 1 -- 172.21.15.39:0/1990174072 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5651ec4681b0 msgr2=0x5651ec83a040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:47:39.596 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:39.592+0000 7f145d4af640 1 --2- 172.21.15.39:0/1990174072 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5651ec4681b0 0x5651ec83a040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:39.597 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:39.592+0000 7f145d4af640 1 -- 172.21.15.39:0/1990174072 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5651ec844f80 con 0x5651ec491070 2024-09-21T03:47:39.597 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:39.592+0000 7f1457fff640 1 --2- 172.21.15.39:0/1990174072 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5651ec470390 0x5651ec8458f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:47:39.597 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:39.592+0000 7f145d4af640 1 --2- 172.21.15.39:0/1990174072 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5651ec491070 0x5651ec835b70 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f1450032e20 tx=0x7f1450044000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f8d7b3a2cf4a2781 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:47:39.597 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:39.592+0000 7f14427fc640 1 -- 172.21.15.39:0/1990174072 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1450003970 con 0x5651ec491070 2024-09-21T03:47:39.597 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:39.592+0000 7f14427fc640 1 -- 172.21.15.39:0/1990174072 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1450036040 con 0x5651ec491070 2024-09-21T03:47:39.597 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:39.592+0000 7f14427fc640 1 -- 172.21.15.39:0/1990174072 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f145002f660 con 0x5651ec491070 2024-09-21T03:47:39.597 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:39.593+0000 7f1465486740 1 -- 172.21.15.39:0/1990174072 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5651ec491070 msgr2=0x5651ec835b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:47:39.597 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:39.593+0000 7f1465486740 1 --2- 172.21.15.39:0/1990174072 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5651ec491070 0x5651ec835b70 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f1450032e20 tx=0x7f1450044000 comp rx=0 tx=0).stop 2024-09-21T03:47:39.597 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:39.593+0000 7f1465486740 1 -- 172.21.15.39:0/1990174072 shutdown_connections 2024-09-21T03:47:39.597 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:39.593+0000 7f1465486740 1 --2- 172.21.15.39:0/1990174072 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5651ec491070 0x5651ec835b70 unknown :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:39.597 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:39.593+0000 7f1465486740 1 --2- 172.21.15.39:0/1990174072 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5651ec470390 0x5651ec8458f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:39.597 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:39.593+0000 7f1465486740 1 --2- 172.21.15.39:0/1990174072 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5651ec4681b0 0x5651ec83a040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:39.597 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:39.593+0000 7f1465486740 1 -- 172.21.15.39:0/1990174072 >> 172.21.15.39:0/1990174072 conn(0x5651ec485460 msgr2=0x5651ec8451e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:47:39.597 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:39.593+0000 7f1465486740 1 -- 172.21.15.39:0/1990174072 shutdown_connections 2024-09-21T03:47:39.598 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:39.593+0000 7f1465486740 1 -- 172.21.15.39:0/1990174072 wait complete. 2024-09-21T03:47:39.598 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:39.594+0000 7f1465486740 1 -- start start 2024-09-21T03:47:39.598 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:39.594+0000 7f1465486740 1 Processor -- start 2024-09-21T03:47:39.601 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:39.594+0000 7f1465486740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5651ec4681b0 0x5651ec84c7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:47:39.601 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:39.594+0000 7f1465486740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5651ec470390 0x5651ec847c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:47:39.601 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:39.594+0000 7f1465486740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5651ec491070 0x5651ec8481c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:47:39.601 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:39.594+0000 7f1465486740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f14581ae980 con 0x5651ec4681b0 2024-09-21T03:47:39.601 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:39.594+0000 7f1465486740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f14581ae2b0 con 0x5651ec470390 2024-09-21T03:47:39.601 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:39.594+0000 7f1465486740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f14581ab4f0 con 0x5651ec491070 2024-09-21T03:47:39.601 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:39.594+0000 7f1465486740 10 client.0 _ll_register_callbacks cb 0x7f144c005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:47:39.602 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:39.595+0000 7f145ccae640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5651ec4681b0 0x5651ec84c7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:47:39.602 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:39.595+0000 7f145ccae640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5651ec4681b0 0x5651ec84c7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:47364/0 (socket says 172.21.15.39:47364) 2024-09-21T03:47:39.602 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:39.595+0000 7f145ccae640 1 -- 172.21.15.39:0/3454278990 learned_addr learned my addr 172.21.15.39:0/3454278990 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:47:39.602 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:39.595+0000 7f1457fff640 1 --2- 172.21.15.39:0/3454278990 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5651ec470390 0x5651ec847c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:47:39.602 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:39.595+0000 7f145d4af640 1 --2- 172.21.15.39:0/3454278990 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5651ec491070 0x5651ec8481c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:47:39.602 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:39.595+0000 7f145ccae640 1 -- 172.21.15.39:0/3454278990 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5651ec491070 msgr2=0x5651ec8481c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:47:39.602 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:39.595+0000 7f145ccae640 1 --2- 172.21.15.39:0/3454278990 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5651ec491070 0x5651ec8481c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:39.602 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:39.595+0000 7f145ccae640 1 -- 172.21.15.39:0/3454278990 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5651ec470390 msgr2=0x5651ec847c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:47:39.602 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:39.595+0000 7f145ccae640 1 --2- 172.21.15.39:0/3454278990 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5651ec470390 0x5651ec847c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:39.602 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:39.595+0000 7f145ccae640 1 -- 172.21.15.39:0/3454278990 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5651ec848880 con 0x5651ec4681b0 2024-09-21T03:47:39.602 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:39.595+0000 7f145d4af640 1 --2- 172.21.15.39:0/3454278990 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5651ec491070 0x5651ec8481c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:47:39.602 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:39.595+0000 7f1457fff640 1 --2- 172.21.15.39:0/3454278990 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5651ec470390 0x5651ec847c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:47:39.602 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:39.595+0000 7f145ccae640 1 --2- 172.21.15.39:0/3454278990 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5651ec4681b0 0x5651ec84c7d0 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f144c0ab1d0 tx=0x7f144c096040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1db509db3a608c13 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:47:39.602 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:39.595+0000 7f1437fff640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-09-21T03:47:39.602 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:39.595+0000 7f1437fff640 1 -- 172.21.15.39:0/3454278990 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f144c0d6070 con 0x5651ec4681b0 2024-09-21T03:47:39.603 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:39.595+0000 7f1437fff640 1 -- 172.21.15.39:0/3454278990 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8484+0+0 (secure 0 0 0) 0x7f144c0d0b60 con 0x5651ec4681b0 2024-09-21T03:47:39.603 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:39.595+0000 7f1465486740 1 -- 172.21.15.39:0/3454278990 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1450036040 con 0x5651ec4681b0 2024-09-21T03:47:39.603 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:39.596+0000 7f1437fff640 1 -- 172.21.15.39:0/3454278990 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f144c0c9050 con 0x5651ec4681b0 2024-09-21T03:47:39.603 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:39.596+0000 7f1465486740 10 client.16017 fetch_fsmap learned FSMap version 29 2024-09-21T03:47:39.603 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:39.596+0000 7f1465486740 1 -- 172.21.15.39:0/3454278990 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5651ec8c05d0 con 0x5651ec4681b0 2024-09-21T03:47:39.603 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:39.596+0000 7f1437fff640 1 -- 172.21.15.39:0/3454278990 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1450036040 con 0x5651ec4681b0 2024-09-21T03:47:39.603 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:39.596+0000 7f1437fff640 1 -- 172.21.15.39:0/3454278990 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f144c0b30c0 con 0x5651ec4681b0 2024-09-21T03:47:39.603 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:39.596+0000 7f1465486740 10 client.16017 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T03:47:39.603 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:39.596+0000 7f1465486740 10 client.16017 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T03:47:39.603 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:39.596+0000 7f1465486740 20 client.16017 populate_metadata read hostname 'smithi039' 2024-09-21T03:47:39.603 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:39.596+0000 7f1465486740 1 --2- 172.21.15.39:0/3454278990 >> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] conn(0x5651ec424da0 0x5651ec8c2d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:47:39.603 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:39.596+0000 7f1465486740 4 client.16017 mds_command: new command op to 24695 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-21T03:47:39.603 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:39.596+0000 7f1465486740 1 -- 172.21.15.39:0/3454278990 --> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5651ec7ac850 con 0x5651ec424da0 2024-09-21T03:47:39.604 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:39.597+0000 7f1457fff640 1 --2- 172.21.15.39:0/3454278990 >> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] conn(0x5651ec424da0 0x5651ec8c2d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:47:39.604 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:39.597+0000 7f1437fff640 10 client.16017 ms_handle_connect on v2:172.21.15.150:6838/2949386907 2024-09-21T03:47:39.604 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:39.597+0000 7f1457fff640 1 --2- 172.21.15.39:0/3454278990 >> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] conn(0x5651ec424da0 0x5651ec8c2d70 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5ceb446740042254 server_cookie=f9f8bf780be5933b in_seq=0 out_seq=0 2024-09-21T03:47:39.604 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:39.597+0000 7f1437fff640 1 -- 172.21.15.39:0/3454278990 <== mds.0 v2:172.21.15.150:6838/2949386907 1 ==== command_reply(tid 0: 0 ) ==== 8+0+1422 (crc 0 0 0) 0x5651ec7ac850 con 0x5651ec424da0 2024-09-21T03:47:39.604 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:39.597+0000 7f1437fff640 10 client.16017 handle_command_reply: tid=0 2024-09-21T03:47:39.604 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:39.597+0000 7f1465486740 1 client.16017 shutdown 2024-09-21T03:47:39.604 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:39.598+0000 7f1465486740 1 -- 172.21.15.39:0/3454278990 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5651ec4681b0 msgr2=0x5651ec84c7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:47:39.604 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:39.598+0000 7f1465486740 1 --2- 172.21.15.39:0/3454278990 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5651ec4681b0 0x5651ec84c7d0 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f144c0ab1d0 tx=0x7f144c096040 comp rx=0 tx=0).stop 2024-09-21T03:47:39.604 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:39.598+0000 7f1465486740 1 -- 172.21.15.39:0/3454278990 shutdown_connections 2024-09-21T03:47:39.604 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:39.598+0000 7f1465486740 1 --2- 172.21.15.39:0/3454278990 >> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] conn(0x5651ec424da0 0x5651ec8c2d70 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:39.604 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:39.598+0000 7f1465486740 1 --2- 172.21.15.39:0/3454278990 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5651ec491070 0x5651ec8481c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:39.604 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:39.598+0000 7f1465486740 1 --2- 172.21.15.39:0/3454278990 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5651ec470390 0x5651ec847c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:39.604 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:39.598+0000 7f1437fff640 0 client.16017 ms_handle_reset on v2:172.21.15.150:6838/2949386907 2024-09-21T03:47:39.604 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:39.598+0000 7f1465486740 1 --2- 172.21.15.39:0/3454278990 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5651ec4681b0 0x5651ec84c7d0 unknown :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:39.605 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:39.598+0000 7f1465486740 1 -- 172.21.15.39:0/3454278990 >> 172.21.15.39:0/3454278990 conn(0x5651ec485460 msgr2=0x5651ec839e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:47:39.605 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:39.598+0000 7f1465486740 1 -- 172.21.15.39:0/3454278990 shutdown_connections 2024-09-21T03:47:39.605 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:39.598+0000 7f1465486740 1 -- 172.21.15.39:0/3454278990 wait complete. 2024-09-21T03:47:39.605 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:39.598+0000 7f1465486740 20 client.16017 trim_cache size 0 max 16384 2024-09-21T03:47:39.605 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-09-21T03:47:39.605 INFO:teuthology.orchestra.run.smithi039.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-21T03:47:39.605 INFO:teuthology.orchestra.run.smithi039.stdout: "scrubs": { 2024-09-21T03:47:39.605 INFO:teuthology.orchestra.run.smithi039.stdout: "29411307-eb8d-4339-8d2b-47ebd32c0080": { 2024-09-21T03:47:39.605 INFO:teuthology.orchestra.run.smithi039.stdout: "path": "/", 2024-09-21T03:47:39.605 INFO:teuthology.orchestra.run.smithi039.stdout: "tag": "29411307-eb8d-4339-8d2b-47ebd32c0080", 2024-09-21T03:47:39.605 INFO:teuthology.orchestra.run.smithi039.stdout: "options": "recursive,force" 2024-09-21T03:47:39.606 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-21T03:47:39.606 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:47:39.606 INFO:teuthology.orchestra.run.smithi039.stdout: "scrub_stats": { 2024-09-21T03:47:39.606 INFO:teuthology.orchestra.run.smithi039.stdout: "29411307-eb8d-4339-8d2b-47ebd32c0080": { 2024-09-21T03:47:39.606 INFO:teuthology.orchestra.run.smithi039.stdout: "start_time": "2024-09-21T03:47:38.092920+0000", 2024-09-21T03:47:39.606 INFO:teuthology.orchestra.run.smithi039.stdout: "path": "/", 2024-09-21T03:47:39.606 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_started": 0, 2024-09-21T03:47:39.606 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_passed": 0, 2024-09-21T03:47:39.606 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_failed": 0, 2024-09-21T03:47:39.606 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_skipped": 0 2024-09-21T03:47:39.606 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:47:39.606 INFO:teuthology.orchestra.run.smithi039.stdout: "c7e2b8e4-057d-435f-b2bd-b633de98035f": { 2024-09-21T03:47:39.606 INFO:teuthology.orchestra.run.smithi039.stdout: "start_time": "2024-09-21T03:47:19.400305+0000", 2024-09-21T03:47:39.607 INFO:teuthology.orchestra.run.smithi039.stdout: "path": "/", 2024-09-21T03:47:39.607 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_started": 0, 2024-09-21T03:47:39.607 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_passed": 0, 2024-09-21T03:47:39.607 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_failed": 0, 2024-09-21T03:47:39.607 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_skipped": 0 2024-09-21T03:47:39.607 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:47:39.607 INFO:teuthology.orchestra.run.smithi039.stdout: "10728de8-c49d-4edc-be2b-639249255e94": { 2024-09-21T03:47:39.607 INFO:teuthology.orchestra.run.smithi039.stdout: "start_time": "2024-09-21T03:47:03.965623+0000", 2024-09-21T03:47:39.607 INFO:teuthology.orchestra.run.smithi039.stdout: "path": "/", 2024-09-21T03:47:39.607 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_started": 0, 2024-09-21T03:47:39.607 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_passed": 0, 2024-09-21T03:47:39.607 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_failed": 0, 2024-09-21T03:47:39.607 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_skipped": 0 2024-09-21T03:47:39.608 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:47:39.608 INFO:teuthology.orchestra.run.smithi039.stdout: "02509e4f-3911-4983-b16f-df8ded0b9998": { 2024-09-21T03:47:39.608 INFO:teuthology.orchestra.run.smithi039.stdout: "start_time": "2024-09-21T03:46:17.889576+0000", 2024-09-21T03:47:39.608 INFO:teuthology.orchestra.run.smithi039.stdout: "path": "/", 2024-09-21T03:47:39.608 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_started": 0, 2024-09-21T03:47:39.608 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_passed": 0, 2024-09-21T03:47:39.608 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_failed": 0, 2024-09-21T03:47:39.608 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_skipped": 0 2024-09-21T03:47:39.608 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-21T03:47:39.608 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-21T03:47:39.608 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-09-21T03:47:39.609 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:39.599+0000 7f145ef12640 1 -- 172.21.15.39:0/2159983133 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f14480b6680 msgr2=0x7f14480b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:47:39.609 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:39.599+0000 7f145ef12640 1 --2- 172.21.15.39:0/2159983133 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f14480b6680 0x7f14480b8b90 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f1450005e30 tx=0x7f1450005c30 comp rx=0 tx=0).stop 2024-09-21T03:47:39.609 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:39.599+0000 7f145ef12640 1 -- 172.21.15.39:0/2159983133 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1458102aa0 msgr2=0x7f1458199390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:47:39.609 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:39.599+0000 7f145ef12640 1 --2- 172.21.15.39:0/2159983133 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1458102aa0 0x7f1458199390 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f144c00eb00 tx=0x7f144c009f20 comp rx=0 tx=0).stop 2024-09-21T03:47:39.609 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:39.599+0000 7f145ef12640 1 -- 172.21.15.39:0/2159983133 shutdown_connections 2024-09-21T03:47:39.609 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:39.599+0000 7f145ef12640 1 --2- 172.21.15.39:0/2159983133 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f14480b6680 0x7f14480b8b90 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:39.609 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:39.599+0000 7f145ef12640 1 --2- 172.21.15.39:0/2159983133 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1458108d30 0x7f145819ebd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:39.609 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:39.599+0000 7f145ef12640 1 --2- 172.21.15.39:0/2159983133 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f14581083e0 0x7f1458199930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:39.609 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:39.599+0000 7f145ef12640 1 --2- 172.21.15.39:0/2159983133 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1458102aa0 0x7f1458199390 unknown :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:39.609 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:39.599+0000 7f145ef12640 1 -- 172.21.15.39:0/2159983133 >> 172.21.15.39:0/2159983133 conn(0x7f14580797e0 msgr2=0x7f14581043f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:47:39.610 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:39.599+0000 7f145ef12640 1 -- 172.21.15.39:0/2159983133 shutdown_connections 2024-09-21T03:47:39.610 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:47:39.599+0000 7f145ef12640 1 -- 172.21.15.39:0/2159983133 wait complete. 2024-09-21T03:47:39.674 INFO:tasks.workunit.client.0.smithi039.stdout: 2024-09-21T03:47:39.674 INFO:tasks.workunit.client.0.smithi039.stdout: Children see throughput for 1 re-readers = 6594032.00 kB/sec 2024-09-21T03:47:39.674 INFO:tasks.workunit.client.0.smithi039.stdout: Parent sees throughput for 1 re-readers = 6585284.03 kB/sec 2024-09-21T03:47:39.674 INFO:tasks.workunit.client.0.smithi039.stdout: Min throughput per process = 6594032.00 kB/sec 2024-09-21T03:47:39.674 INFO:tasks.workunit.client.0.smithi039.stdout: Max throughput per process = 6594032.00 kB/sec 2024-09-21T03:47:39.674 INFO:tasks.workunit.client.0.smithi039.stdout: Avg throughput per process = 6594032.00 kB/sec 2024-09-21T03:47:39.675 INFO:tasks.workunit.client.0.smithi039.stdout: Min xfer = 1048576.00 kB 2024-09-21T03:47:40.334 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:47:40 smithi039 ceph-mon[31211]: scrub summary: idle 2024-09-21T03:47:40.468 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:47:40 smithi161 ceph-mon[34533]: scrub summary: idle 2024-09-21T03:47:40.474 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:47:40 smithi150 ceph-mon[34289]: scrub summary: idle 2024-09-21T03:47:40.719 INFO:tasks.cephfs.filesystem:scrub status for tag:29411307-eb8d-4339-8d2b-47ebd32c0080 - {'path': '/', 'tag': '29411307-eb8d-4339-8d2b-47ebd32c0080', 'options': 'recursive,force'} 2024-09-21T03:47:41.468 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:47:41 smithi161 ceph-mon[34533]: pgmap v374: 97 pgs: 97 active+clean; 1.0 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 171 MiB/s wr, 57 op/s 2024-09-21T03:47:41.474 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:47:41 smithi150 ceph-mon[34289]: pgmap v374: 97 pgs: 97 active+clean; 1.0 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 171 MiB/s wr, 57 op/s 2024-09-21T03:47:41.584 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:47:41 smithi039 ceph-mon[31211]: pgmap v374: 97 pgs: 97 active+clean; 1.0 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 171 MiB/s wr, 57 op/s 2024-09-21T03:47:43.468 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:47:43 smithi161 ceph-mon[34533]: pgmap v375: 97 pgs: 97 active+clean; 1.0 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 134 MiB/s wr, 43 op/s 2024-09-21T03:47:43.474 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:47:43 smithi150 ceph-mon[34289]: pgmap v375: 97 pgs: 97 active+clean; 1.0 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 134 MiB/s wr, 43 op/s 2024-09-21T03:47:43.584 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:47:43 smithi039 ceph-mon[31211]: pgmap v375: 97 pgs: 97 active+clean; 1.0 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 134 MiB/s wr, 43 op/s 2024-09-21T03:47:45.468 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:47:45 smithi161 ceph-mon[34533]: pgmap v376: 97 pgs: 97 active+clean; 1.0 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 110 MiB/s wr, 36 op/s 2024-09-21T03:47:45.474 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:47:45 smithi150 ceph-mon[34289]: pgmap v376: 97 pgs: 97 active+clean; 1.0 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 110 MiB/s wr, 36 op/s 2024-09-21T03:47:45.584 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:47:45 smithi039 ceph-mon[31211]: pgmap v376: 97 pgs: 97 active+clean; 1.0 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 110 MiB/s wr, 36 op/s 2024-09-21T03:47:46.231 INFO:tasks.workunit.client.0.smithi039.stdout: 2024-09-21T03:47:46.231 INFO:tasks.workunit.client.0.smithi039.stdout: 2024-09-21T03:47:46.231 INFO:tasks.workunit.client.0.smithi039.stdout: 2024-09-21T03:47:46.231 INFO:tasks.workunit.client.0.smithi039.stdout:iozone test complete. 2024-09-21T03:47:46.232 INFO:tasks.workunit.client.0.smithi039.stderr:+ iozone -c -e -s 1024M -r 1M -t 1 -F f2 -i 0 -i 1 2024-09-21T03:47:46.299 INFO:tasks.workunit.client.0.smithi039.stdout: Iozone: Performance Test of File I/O 2024-09-21T03:47:46.299 INFO:tasks.workunit.client.0.smithi039.stdout: Version $Revision: 3.494 $ 2024-09-21T03:47:46.299 INFO:tasks.workunit.client.0.smithi039.stdout: Compiled for 64 bit mode. 2024-09-21T03:47:46.299 INFO:tasks.workunit.client.0.smithi039.stdout: Build: linux-AMD64 2024-09-21T03:47:46.299 INFO:tasks.workunit.client.0.smithi039.stdout: 2024-09-21T03:47:46.300 INFO:tasks.workunit.client.0.smithi039.stdout: Contributors:William Norcott, Don Capps, Isom Crawford, Kirby Collins 2024-09-21T03:47:46.300 INFO:tasks.workunit.client.0.smithi039.stdout: Al Slater, Scott Rhine, Mike Wisner, Ken Goss 2024-09-21T03:47:46.300 INFO:tasks.workunit.client.0.smithi039.stdout: Steve Landherr, Brad Smith, Mark Kelly, Dr. Alain CYR, 2024-09-21T03:47:46.300 INFO:tasks.workunit.client.0.smithi039.stdout: Randy Dunlap, Mark Montague, Dan Million, Gavin Brebner, 2024-09-21T03:47:46.300 INFO:tasks.workunit.client.0.smithi039.stdout: Jean-Marc Zucconi, Jeff Blomberg, Benny Halevy, Dave Boone, 2024-09-21T03:47:46.300 INFO:tasks.workunit.client.0.smithi039.stdout: Erik Habbinga, Kris Strecker, Walter Wong, Joshua Root, 2024-09-21T03:47:46.300 INFO:tasks.workunit.client.0.smithi039.stdout: Fabrice Bacchella, Zhenghua Xue, Qin Li, Darren Sawyer, 2024-09-21T03:47:46.300 INFO:tasks.workunit.client.0.smithi039.stdout: Vangel Bojaxhi, Ben England, Vikentsi Lapa, 2024-09-21T03:47:46.300 INFO:tasks.workunit.client.0.smithi039.stdout: Alexey Skidanov, Sudhir Kumar. 2024-09-21T03:47:46.300 INFO:tasks.workunit.client.0.smithi039.stdout: 2024-09-21T03:47:46.300 INFO:tasks.workunit.client.0.smithi039.stdout: Run began: Sat Sep 21 03:47:46 2024 2024-09-21T03:47:46.300 INFO:tasks.workunit.client.0.smithi039.stdout: 2024-09-21T03:47:46.301 INFO:tasks.workunit.client.0.smithi039.stdout: Include close in write timing 2024-09-21T03:47:46.301 INFO:tasks.workunit.client.0.smithi039.stdout: Include fsync in write timing 2024-09-21T03:47:46.301 INFO:tasks.workunit.client.0.smithi039.stdout: File size set to 1048576 kB 2024-09-21T03:47:46.301 INFO:tasks.workunit.client.0.smithi039.stdout: Record Size 1024 kB 2024-09-21T03:47:46.301 INFO:tasks.workunit.client.0.smithi039.stdout: Command line used: iozone -c -e -s 1024M -r 1M -t 1 -F f2 -i 0 -i 1 2024-09-21T03:47:46.301 INFO:tasks.workunit.client.0.smithi039.stdout: Output is in kBytes/sec 2024-09-21T03:47:46.301 INFO:tasks.workunit.client.0.smithi039.stdout: Time Resolution = 0.000001 seconds. 2024-09-21T03:47:46.301 INFO:tasks.workunit.client.0.smithi039.stdout: Processor cache size set to 1024 kBytes. 2024-09-21T03:47:46.301 INFO:tasks.workunit.client.0.smithi039.stdout: Processor cache line size set to 32 bytes. 2024-09-21T03:47:46.301 INFO:tasks.workunit.client.0.smithi039.stdout: File stride size set to 17 * record size. 2024-09-21T03:47:46.301 INFO:tasks.workunit.client.0.smithi039.stdout: Throughput test with 1 process 2024-09-21T03:47:46.301 INFO:tasks.workunit.client.0.smithi039.stdout: Each process writes a 1048576 kByte file in 1024 kByte records 2024-09-21T03:47:47.468 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:47:47 smithi161 ceph-mon[34533]: pgmap v377: 97 pgs: 97 active+clean; 1.0 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 92 MiB/s wr, 35 op/s 2024-09-21T03:47:47.474 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:47:47 smithi150 ceph-mon[34289]: pgmap v377: 97 pgs: 97 active+clean; 1.0 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 92 MiB/s wr, 35 op/s 2024-09-21T03:47:47.584 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:47:47 smithi039 ceph-mon[31211]: pgmap v377: 97 pgs: 97 active+clean; 1.0 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 92 MiB/s wr, 35 op/s 2024-09-21T03:47:47.857 INFO:tasks.workunit.client.0.smithi039.stdout: 2024-09-21T03:47:47.857 INFO:tasks.workunit.client.0.smithi039.stdout: Children see throughput for 1 initial writers = 688666.88 kB/sec 2024-09-21T03:47:47.857 INFO:tasks.workunit.client.0.smithi039.stdout: Parent sees throughput for 1 initial writers = 688574.20 kB/sec 2024-09-21T03:47:47.857 INFO:tasks.workunit.client.0.smithi039.stdout: Min throughput per process = 688666.88 kB/sec 2024-09-21T03:47:47.857 INFO:tasks.workunit.client.0.smithi039.stdout: Max throughput per process = 688666.88 kB/sec 2024-09-21T03:47:47.857 INFO:tasks.workunit.client.0.smithi039.stdout: Avg throughput per process = 688666.88 kB/sec 2024-09-21T03:47:47.858 INFO:tasks.workunit.client.0.smithi039.stdout: Min xfer = 1048576.00 kB 2024-09-21T03:47:49.468 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:47:49 smithi161 ceph-mon[34533]: pgmap v378: 97 pgs: 97 active+clean; 1.0 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 59 MiB/s wr, 24 op/s 2024-09-21T03:47:49.474 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:47:49 smithi150 ceph-mon[34289]: pgmap v378: 97 pgs: 97 active+clean; 1.0 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 59 MiB/s wr, 24 op/s 2024-09-21T03:47:49.584 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:47:49 smithi039 ceph-mon[31211]: pgmap v378: 97 pgs: 97 active+clean; 1.0 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 59 MiB/s wr, 24 op/s 2024-09-21T03:47:51.528 INFO:tasks.workunit.client.0.smithi039.stdout: 2024-09-21T03:47:51.528 INFO:tasks.workunit.client.0.smithi039.stdout: Children see throughput for 1 rewriters = 671063.25 kB/sec 2024-09-21T03:47:51.528 INFO:tasks.workunit.client.0.smithi039.stdout: Parent sees throughput for 1 rewriters = 670963.65 kB/sec 2024-09-21T03:47:51.529 INFO:tasks.workunit.client.0.smithi039.stdout: Min throughput per process = 671063.25 kB/sec 2024-09-21T03:47:51.529 INFO:tasks.workunit.client.0.smithi039.stdout: Max throughput per process = 671063.25 kB/sec 2024-09-21T03:47:51.529 INFO:tasks.workunit.client.0.smithi039.stdout: Avg throughput per process = 671063.25 kB/sec 2024-09-21T03:47:51.529 INFO:tasks.workunit.client.0.smithi039.stdout: Min xfer = 1048576.00 kB 2024-09-21T03:47:51.584 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:47:51 smithi039 ceph-mon[31211]: pgmap v379: 97 pgs: 97 active+clean; 1.3 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 105 MiB/s wr, 47 op/s 2024-09-21T03:47:51.717 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:47:51 smithi161 ceph-mon[34533]: pgmap v379: 97 pgs: 97 active+clean; 1.3 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 105 MiB/s wr, 47 op/s 2024-09-21T03:47:51.724 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:47:51 smithi150 ceph-mon[34289]: pgmap v379: 97 pgs: 97 active+clean; 1.3 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 105 MiB/s wr, 47 op/s 2024-09-21T03:47:53.474 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:47:53 smithi150 ceph-mon[34289]: pgmap v380: 97 pgs: 97 active+clean; 1.3 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 78 MiB/s wr, 44 op/s 2024-09-21T03:47:53.584 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:47:53 smithi039 ceph-mon[31211]: pgmap v380: 97 pgs: 97 active+clean; 1.3 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 78 MiB/s wr, 44 op/s 2024-09-21T03:47:53.717 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:47:53 smithi161 ceph-mon[34533]: pgmap v380: 97 pgs: 97 active+clean; 1.3 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 78 MiB/s wr, 44 op/s 2024-09-21T03:47:53.855 INFO:tasks.workunit.client.0.smithi039.stdout: 2024-09-21T03:47:53.856 INFO:tasks.workunit.client.0.smithi039.stdout: Children see throughput for 1 readers = 7871959.50 kB/sec 2024-09-21T03:47:53.856 INFO:tasks.workunit.client.0.smithi039.stdout: Parent sees throughput for 1 readers = 7857206.98 kB/sec 2024-09-21T03:47:53.856 INFO:tasks.workunit.client.0.smithi039.stdout: Min throughput per process = 7871959.50 kB/sec 2024-09-21T03:47:53.856 INFO:tasks.workunit.client.0.smithi039.stdout: Max throughput per process = 7871959.50 kB/sec 2024-09-21T03:47:53.856 INFO:tasks.workunit.client.0.smithi039.stdout: Avg throughput per process = 7871959.50 kB/sec 2024-09-21T03:47:53.856 INFO:tasks.workunit.client.0.smithi039.stdout: Min xfer = 1048576.00 kB 2024-09-21T03:47:55.584 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:47:55 smithi039 ceph-mon[31211]: pgmap v381: 97 pgs: 97 active+clean; 1.0 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 118 MiB/s wr, 62 op/s 2024-09-21T03:47:55.718 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:47:55 smithi161 ceph-mon[34533]: pgmap v381: 97 pgs: 97 active+clean; 1.0 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 118 MiB/s wr, 62 op/s 2024-09-21T03:47:55.724 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:47:55 smithi150 ceph-mon[34289]: pgmap v381: 97 pgs: 97 active+clean; 1.0 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 118 MiB/s wr, 62 op/s 2024-09-21T03:47:56.073 INFO:tasks.workunit.client.0.smithi039.stdout: 2024-09-21T03:47:56.073 INFO:tasks.workunit.client.0.smithi039.stdout: Children see throughput for 1 re-readers = 7979491.00 kB/sec 2024-09-21T03:47:56.073 INFO:tasks.workunit.client.0.smithi039.stdout: Parent sees throughput for 1 re-readers = 7967175.27 kB/sec 2024-09-21T03:47:56.073 INFO:tasks.workunit.client.0.smithi039.stdout: Min throughput per process = 7979491.00 kB/sec 2024-09-21T03:47:56.073 INFO:tasks.workunit.client.0.smithi039.stdout: Max throughput per process = 7979491.00 kB/sec 2024-09-21T03:47:56.073 INFO:tasks.workunit.client.0.smithi039.stdout: Avg throughput per process = 7979491.00 kB/sec 2024-09-21T03:47:56.073 INFO:tasks.workunit.client.0.smithi039.stdout: Min xfer = 1048576.00 kB 2024-09-21T03:47:57.584 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:47:57 smithi039 ceph-mon[31211]: pgmap v382: 97 pgs: 97 active+clean; 1.0 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 171 MiB/s wr, 79 op/s 2024-09-21T03:47:57.718 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:47:57 smithi161 ceph-mon[34533]: pgmap v382: 97 pgs: 97 active+clean; 1.0 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 171 MiB/s wr, 79 op/s 2024-09-21T03:47:57.724 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:47:57 smithi150 ceph-mon[34289]: pgmap v382: 97 pgs: 97 active+clean; 1.0 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 171 MiB/s wr, 79 op/s 2024-09-21T03:47:59.583 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:47:59 smithi039 ceph-mon[31211]: pgmap v383: 97 pgs: 97 active+clean; 1.0 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 171 MiB/s wr, 75 op/s 2024-09-21T03:47:59.718 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:47:59 smithi161 ceph-mon[34533]: pgmap v383: 97 pgs: 97 active+clean; 1.0 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 171 MiB/s wr, 75 op/s 2024-09-21T03:47:59.725 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:47:59 smithi150 ceph-mon[34289]: pgmap v383: 97 pgs: 97 active+clean; 1.0 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 171 MiB/s wr, 75 op/s 2024-09-21T03:48:01.584 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:48:01 smithi039 ceph-mon[31211]: pgmap v384: 97 pgs: 97 active+clean; 1.0 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 169 MiB/s wr, 78 op/s 2024-09-21T03:48:01.718 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:48:01 smithi161 ceph-mon[34533]: pgmap v384: 97 pgs: 97 active+clean; 1.0 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 169 MiB/s wr, 78 op/s 2024-09-21T03:48:01.724 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:48:01 smithi150 ceph-mon[34289]: pgmap v384: 97 pgs: 97 active+clean; 1.0 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 169 MiB/s wr, 78 op/s 2024-09-21T03:48:02.323 INFO:tasks.workunit.client.0.smithi039.stdout: 2024-09-21T03:48:02.324 INFO:tasks.workunit.client.0.smithi039.stdout: 2024-09-21T03:48:02.324 INFO:tasks.workunit.client.0.smithi039.stdout: 2024-09-21T03:48:02.324 INFO:tasks.workunit.client.0.smithi039.stdout:iozone test complete. 2024-09-21T03:48:02.324 INFO:tasks.workunit.client.0.smithi039.stderr:+ iozone -c -e -s 10240M -r 1M -t 1 -F f3 -i 0 -i 1 2024-09-21T03:48:02.391 INFO:tasks.workunit.client.0.smithi039.stdout: Iozone: Performance Test of File I/O 2024-09-21T03:48:02.392 INFO:tasks.workunit.client.0.smithi039.stdout: Version $Revision: 3.494 $ 2024-09-21T03:48:02.392 INFO:tasks.workunit.client.0.smithi039.stdout: Compiled for 64 bit mode. 2024-09-21T03:48:02.392 INFO:tasks.workunit.client.0.smithi039.stdout: Build: linux-AMD64 2024-09-21T03:48:02.392 INFO:tasks.workunit.client.0.smithi039.stdout: 2024-09-21T03:48:02.392 INFO:tasks.workunit.client.0.smithi039.stdout: Contributors:William Norcott, Don Capps, Isom Crawford, Kirby Collins 2024-09-21T03:48:02.392 INFO:tasks.workunit.client.0.smithi039.stdout: Al Slater, Scott Rhine, Mike Wisner, Ken Goss 2024-09-21T03:48:02.392 INFO:tasks.workunit.client.0.smithi039.stdout: Steve Landherr, Brad Smith, Mark Kelly, Dr. Alain CYR, 2024-09-21T03:48:02.392 INFO:tasks.workunit.client.0.smithi039.stdout: Randy Dunlap, Mark Montague, Dan Million, Gavin Brebner, 2024-09-21T03:48:02.392 INFO:tasks.workunit.client.0.smithi039.stdout: Jean-Marc Zucconi, Jeff Blomberg, Benny Halevy, Dave Boone, 2024-09-21T03:48:02.393 INFO:tasks.workunit.client.0.smithi039.stdout: Erik Habbinga, Kris Strecker, Walter Wong, Joshua Root, 2024-09-21T03:48:02.393 INFO:tasks.workunit.client.0.smithi039.stdout: Fabrice Bacchella, Zhenghua Xue, Qin Li, Darren Sawyer, 2024-09-21T03:48:02.393 INFO:tasks.workunit.client.0.smithi039.stdout: Vangel Bojaxhi, Ben England, Vikentsi Lapa, 2024-09-21T03:48:02.393 INFO:tasks.workunit.client.0.smithi039.stdout: Alexey Skidanov, Sudhir Kumar. 2024-09-21T03:48:02.393 INFO:tasks.workunit.client.0.smithi039.stdout: 2024-09-21T03:48:02.393 INFO:tasks.workunit.client.0.smithi039.stdout: Run began: Sat Sep 21 03:48:02 2024 2024-09-21T03:48:02.393 INFO:tasks.workunit.client.0.smithi039.stdout: 2024-09-21T03:48:02.393 INFO:tasks.workunit.client.0.smithi039.stdout: Include close in write timing 2024-09-21T03:48:02.393 INFO:tasks.workunit.client.0.smithi039.stdout: Include fsync in write timing 2024-09-21T03:48:02.393 INFO:tasks.workunit.client.0.smithi039.stdout: File size set to 10485760 kB 2024-09-21T03:48:02.393 INFO:tasks.workunit.client.0.smithi039.stdout: Record Size 1024 kB 2024-09-21T03:48:02.393 INFO:tasks.workunit.client.0.smithi039.stdout: Command line used: iozone -c -e -s 10240M -r 1M -t 1 -F f3 -i 0 -i 1 2024-09-21T03:48:02.393 INFO:tasks.workunit.client.0.smithi039.stdout: Output is in kBytes/sec 2024-09-21T03:48:02.394 INFO:tasks.workunit.client.0.smithi039.stdout: Time Resolution = 0.000001 seconds. 2024-09-21T03:48:02.394 INFO:tasks.workunit.client.0.smithi039.stdout: Processor cache size set to 1024 kBytes. 2024-09-21T03:48:02.394 INFO:tasks.workunit.client.0.smithi039.stdout: Processor cache line size set to 32 bytes. 2024-09-21T03:48:02.394 INFO:tasks.workunit.client.0.smithi039.stdout: File stride size set to 17 * record size. 2024-09-21T03:48:02.394 INFO:tasks.workunit.client.0.smithi039.stdout: Throughput test with 1 process 2024-09-21T03:48:02.394 INFO:tasks.workunit.client.0.smithi039.stdout: Each process writes a 10485760 kByte file in 1024 kByte records 2024-09-21T03:48:03.585 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:48:03 smithi039 ceph-mon[31211]: pgmap v385: 97 pgs: 97 active+clean; 1.0 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 114 MiB/s wr, 53 op/s 2024-09-21T03:48:03.717 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:48:03 smithi161 ceph-mon[34533]: pgmap v385: 97 pgs: 97 active+clean; 1.0 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 114 MiB/s wr, 53 op/s 2024-09-21T03:48:03.724 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:48:03 smithi150 ceph-mon[34289]: pgmap v385: 97 pgs: 97 active+clean; 1.0 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 114 MiB/s wr, 53 op/s 2024-09-21T03:48:05.584 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:48:05 smithi039 ceph-mon[31211]: pgmap v386: 97 pgs: 97 active+clean; 1.0 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 93 MiB/s wr, 41 op/s 2024-09-21T03:48:05.718 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:48:05 smithi161 ceph-mon[34533]: pgmap v386: 97 pgs: 97 active+clean; 1.0 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 93 MiB/s wr, 41 op/s 2024-09-21T03:48:05.724 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:48:05 smithi150 ceph-mon[34289]: pgmap v386: 97 pgs: 97 active+clean; 1.0 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 93 MiB/s wr, 41 op/s 2024-09-21T03:48:06.718 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:48:06 smithi161 ceph-mon[34533]: pgmap v387: 97 pgs: 97 active+clean; 1.4 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 100 MiB/s wr, 74 op/s 2024-09-21T03:48:06.725 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:48:06 smithi150 ceph-mon[34289]: pgmap v387: 97 pgs: 97 active+clean; 1.4 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 100 MiB/s wr, 74 op/s 2024-09-21T03:48:06.838 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:48:06 smithi039 ceph-mon[31211]: pgmap v387: 97 pgs: 97 active+clean; 1.4 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 100 MiB/s wr, 74 op/s 2024-09-21T03:48:08.968 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:48:08 smithi161 ceph-mon[34533]: pgmap v388: 97 pgs: 97 active+clean; 1.6 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 75 MiB/s wr, 66 op/s 2024-09-21T03:48:08.974 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:48:08 smithi150 ceph-mon[34289]: pgmap v388: 97 pgs: 97 active+clean; 1.6 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 75 MiB/s wr, 66 op/s 2024-09-21T03:48:09.085 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:48:08 smithi039 ceph-mon[31211]: pgmap v388: 97 pgs: 97 active+clean; 1.6 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 75 MiB/s wr, 66 op/s 2024-09-21T03:48:10.720 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 51116d5c-77ca-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub status 2024-09-21T03:48:11.085 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:48:10 smithi039 ceph-mon[31211]: pgmap v389: 97 pgs: 97 active+clean; 3.6 GiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 389 KiB/s rd, 300 MiB/s wr, 285 op/s 2024-09-21T03:48:11.217 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:48:10 smithi161 ceph-mon[34533]: pgmap v389: 97 pgs: 97 active+clean; 3.6 GiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 389 KiB/s rd, 300 MiB/s wr, 285 op/s 2024-09-21T03:48:11.224 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:48:10 smithi150 ceph-mon[34289]: pgmap v389: 97 pgs: 97 active+clean; 3.6 GiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 389 KiB/s rd, 300 MiB/s wr, 285 op/s 2024-09-21T03:48:11.491 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/51116d5c-77ca-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:48:12.483 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:12.480+0000 7fc318c4b640 1 -- 172.21.15.39:0/3217608577 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc31410ad10 msgr2=0x7fc3141183c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:48:12.483 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:12.480+0000 7fc31159a640 1 -- 172.21.15.39:0/3217608577 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc308004230 con 0x7fc31410ad10 2024-09-21T03:48:12.484 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:12.480+0000 7fc318c4b640 1 --2- 172.21.15.39:0/3217608577 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc31410ad10 0x7fc3141183c0 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7fc308009900 tx=0x7fc30802a440 comp rx=0 tx=0).stop 2024-09-21T03:48:12.484 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:12.482+0000 7fc318c4b640 1 -- 172.21.15.39:0/3217608577 shutdown_connections 2024-09-21T03:48:12.484 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:12.482+0000 7fc318c4b640 1 --2- 172.21.15.39:0/3217608577 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc314118900 0x7fc31411ad40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:12.484 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:12.482+0000 7fc318c4b640 1 --2- 172.21.15.39:0/3217608577 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc31410ad10 0x7fc3141183c0 unknown :-1 s=CLOSED pgs=606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:12.484 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:12.482+0000 7fc318c4b640 1 --2- 172.21.15.39:0/3217608577 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc31410a3c0 0x7fc31410a7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:12.484 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:12.482+0000 7fc318c4b640 1 -- 172.21.15.39:0/3217608577 >> 172.21.15.39:0/3217608577 conn(0x7fc3140712f0 msgr2=0x7fc314073e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:48:12.485 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:12.483+0000 7fc318c4b640 1 -- 172.21.15.39:0/3217608577 shutdown_connections 2024-09-21T03:48:12.487 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:12.483+0000 7fc318c4b640 1 -- 172.21.15.39:0/3217608577 wait complete. 2024-09-21T03:48:12.487 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:12.485+0000 7fc318c4b640 1 Processor -- start 2024-09-21T03:48:12.487 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:12.485+0000 7fc318c4b640 1 -- start start 2024-09-21T03:48:12.487 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:12.485+0000 7fc318c4b640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc31410a3c0 0x7fc31410df70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:48:12.487 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:12.485+0000 7fc318c4b640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc31410ad10 0x7fc31410e4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:48:12.487 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:12.485+0000 7fc318c4b640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc314118900 0x7fc314116fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:48:12.487 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:12.485+0000 7fc318c4b640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fc31411e4e0 con 0x7fc31410ad10 2024-09-21T03:48:12.488 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:12.485+0000 7fc318c4b640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fc31411e360 con 0x7fc31410a3c0 2024-09-21T03:48:12.488 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:12.485+0000 7fc318c4b640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fc31411e660 con 0x7fc314118900 2024-09-21T03:48:12.488 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:12.486+0000 7fc312d9d640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc314118900 0x7fc314116fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:48:12.488 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:12.486+0000 7fc312d9d640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc314118900 0x7fc314116fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.39:46098/0 (socket says 172.21.15.39:46098) 2024-09-21T03:48:12.488 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:12.486+0000 7fc312d9d640 1 -- 172.21.15.39:0/2025319476 learned_addr learned my addr 172.21.15.39:0/2025319476 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:48:12.490 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:12.488+0000 7fc31259c640 1 --2- 172.21.15.39:0/2025319476 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc31410a3c0 0x7fc31410df70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:48:12.493 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:12.488+0000 7fc312d9d640 1 -- 172.21.15.39:0/2025319476 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc31410a3c0 msgr2=0x7fc31410df70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:48:12.493 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:12.488+0000 7fc312d9d640 1 --2- 172.21.15.39:0/2025319476 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc31410a3c0 0x7fc31410df70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:12.493 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:12.488+0000 7fc312d9d640 1 -- 172.21.15.39:0/2025319476 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc31410ad10 msgr2=0x7fc31410e4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:48:12.493 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:12.488+0000 7fc312d9d640 1 --2- 172.21.15.39:0/2025319476 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc31410ad10 0x7fc31410e4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:12.493 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:12.488+0000 7fc312d9d640 1 -- 172.21.15.39:0/2025319476 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc31410ed30 con 0x7fc314118900 2024-09-21T03:48:12.496 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:12.494+0000 7fc312d9d640 1 --2- 172.21.15.39:0/2025319476 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc314118900 0x7fc314116fe0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fc30c00cab0 tx=0x7fc30c00cf70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:48:12.500 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:12.498+0000 7fc3037fe640 1 -- 172.21.15.39:0/2025319476 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc30c00ae20 con 0x7fc314118900 2024-09-21T03:48:12.500 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:12.499+0000 7fc3037fe640 1 -- 172.21.15.39:0/2025319476 <== mon.1 v2:172.21.15.161:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc30c00f900 con 0x7fc314118900 2024-09-21T03:48:12.501 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:12.499+0000 7fc3037fe640 1 -- 172.21.15.39:0/2025319476 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc30c018a50 con 0x7fc314118900 2024-09-21T03:48:12.504 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:12.501+0000 7fc318c4b640 1 -- 172.21.15.39:0/2025319476 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc3141175e0 con 0x7fc314118900 2024-09-21T03:48:12.504 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:12.501+0000 7fc318c4b640 1 -- 172.21.15.39:0/2025319476 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc314117aa0 con 0x7fc314118900 2024-09-21T03:48:12.514 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:12.512+0000 7fc31f1bf740 1 Processor -- start 2024-09-21T03:48:12.520 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:12.517+0000 7fc31f1bf740 1 -- start start 2024-09-21T03:48:12.520 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:12.518+0000 7fc31f1bf740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561b99f16070 0x561b9a2bd270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:48:12.520 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:12.518+0000 7fc31f1bf740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561b99eed1b0 0x561b9a2bd7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:48:12.520 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:12.518+0000 7fc31f1bf740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561b99ef5390 0x561b9a2bdcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:48:12.520 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:12.518+0000 7fc31f1bf740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x561b9a233e50 con 0x561b99ef5390 2024-09-21T03:48:12.521 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:12.518+0000 7fc31f1bf740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x561b9a223280 con 0x561b99f16070 2024-09-21T03:48:12.521 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:12.518+0000 7fc31f1bf740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x561b9a227090 con 0x561b99eed1b0 2024-09-21T03:48:12.523 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:12.519+0000 7fc311d9b640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561b99ef5390 0x561b9a2bdcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:48:12.524 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:12.519+0000 7fc311d9b640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561b99ef5390 0x561b9a2bdcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:44234/0 (socket says 172.21.15.39:44234) 2024-09-21T03:48:12.524 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:12.519+0000 7fc311d9b640 1 -- 172.21.15.39:0/3995060 learned_addr learned my addr 172.21.15.39:0/3995060 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:48:12.526 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:12.519+0000 7fc311d9b640 1 --2- 172.21.15.39:0/3995060 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561b99f16070 0x561b9a2bd270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:48:12.528 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:12.525+0000 7fc311d9b640 1 -- 172.21.15.39:0/3995060 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561b99eed1b0 msgr2=0x561b9a2bd7b0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-21T03:48:12.528 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:12.525+0000 7fc311d9b640 1 --2- 172.21.15.39:0/3995060 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561b99eed1b0 0x561b9a2bd7b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:12.528 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:12.525+0000 7fc311d9b640 1 -- 172.21.15.39:0/3995060 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561b99f16070 msgr2=0x561b9a2bd270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:48:12.528 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:12.525+0000 7fc311d9b640 1 --2- 172.21.15.39:0/3995060 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561b99f16070 0x561b9a2bd270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:12.528 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:12.525+0000 7fc311d9b640 1 -- 172.21.15.39:0/3995060 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561b99f81650 con 0x561b99ef5390 2024-09-21T03:48:12.528 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:12.526+0000 7fc311d9b640 1 --2- 172.21.15.39:0/3995060 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561b99ef5390 0x561b9a2bdcf0 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7fc308005940 tx=0x7fc3080058b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=41cae6cdd8c329fc server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:48:12.532 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:12.527+0000 7fc300ff9640 1 -- 172.21.15.39:0/3995060 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc308004250 con 0x561b99ef5390 2024-09-21T03:48:12.533 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:12.527+0000 7fc300ff9640 1 -- 172.21.15.39:0/3995060 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc308045070 con 0x561b99ef5390 2024-09-21T03:48:12.534 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:12.527+0000 7fc300ff9640 1 -- 172.21.15.39:0/3995060 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc308003a50 con 0x561b99ef5390 2024-09-21T03:48:12.534 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:12.528+0000 7fc3037fe640 1 -- 172.21.15.39:0/2025319476 <== mon.1 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 27) ==== 154643+0+0 (secure 0 0 0) 0x7fc30c018bf0 con 0x7fc314118900 2024-09-21T03:48:12.535 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:12.529+0000 7fc31f1bf740 1 -- 172.21.15.39:0/3995060 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561b99ef5390 msgr2=0x561b9a2bdcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:48:12.535 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:12.529+0000 7fc31f1bf740 1 --2- 172.21.15.39:0/3995060 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561b99ef5390 0x561b9a2bdcf0 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7fc308005940 tx=0x7fc3080058b0 comp rx=0 tx=0).stop 2024-09-21T03:48:12.535 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:12.529+0000 7fc3037fe640 1 --2- 172.21.15.39:0/2025319476 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7fc2fc0b6880 0x7fc2fc0b8d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:48:12.536 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:12.530+0000 7fc3037fe640 1 -- 172.21.15.39:0/2025319476 <== mon.1 v2:172.21.15.161:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8484+0+0 (secure 0 0 0) 0x7fc30c0e7fb0 con 0x7fc314118900 2024-09-21T03:48:12.537 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:12.533+0000 7fc31f1bf740 1 -- 172.21.15.39:0/3995060 shutdown_connections 2024-09-21T03:48:12.537 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:12.533+0000 7fc31f1bf740 1 --2- 172.21.15.39:0/3995060 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561b99ef5390 0x561b9a2bdcf0 unknown :-1 s=CLOSED pgs=607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:12.537 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:12.533+0000 7fc31f1bf740 1 --2- 172.21.15.39:0/3995060 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561b99eed1b0 0x561b9a2bd7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:12.537 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:12.533+0000 7fc31f1bf740 1 --2- 172.21.15.39:0/3995060 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561b99f16070 0x561b9a2bd270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:12.537 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:12.533+0000 7fc31f1bf740 1 -- 172.21.15.39:0/3995060 >> 172.21.15.39:0/3995060 conn(0x561b99f0a460 msgr2=0x561b9a2bb160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:48:12.538 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:12.536+0000 7fc311d9b640 1 --2- 172.21.15.39:0/2025319476 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7fc2fc0b6880 0x7fc2fc0b8d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:48:12.542 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:12.539+0000 7fc31f1bf740 1 -- 172.21.15.39:0/3995060 shutdown_connections 2024-09-21T03:48:12.542 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:12.539+0000 7fc311d9b640 1 --2- 172.21.15.39:0/2025319476 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7fc2fc0b6880 0x7fc2fc0b8d90 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fc308000ea0 tx=0x7fc308004bb0 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:48:12.550 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:12.544+0000 7fc31f1bf740 1 -- 172.21.15.39:0/3995060 wait complete. 2024-09-21T03:48:12.550 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:12.545+0000 7fc31f1bf740 1 -- start start 2024-09-21T03:48:12.550 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:12.546+0000 7fc31f1bf740 1 Processor -- start 2024-09-21T03:48:12.550 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:12.547+0000 7fc31f1bf740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561b99eed1b0 0x561b9a2c89e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:48:12.550 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:12.547+0000 7fc31f1bf740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561b99ef5390 0x561b9a2c8f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:48:12.550 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:12.547+0000 7fc31f1bf740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561b99f16070 0x561b9a2d4d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:48:12.551 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:12.547+0000 7fc31f1bf740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x561b9a084700 con 0x561b99eed1b0 2024-09-21T03:48:12.551 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:12.547+0000 7fc31f1bf740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x561b9a1be110 con 0x561b99ef5390 2024-09-21T03:48:12.551 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:12.547+0000 7fc31f1bf740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x561b9a2cb9f0 con 0x561b99f16070 2024-09-21T03:48:12.551 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:12.547+0000 7fc31f1bf740 10 client.0 _ll_register_callbacks cb 0x561b9a1d7990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:48:12.553 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:12.551+0000 7fc311d9b640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561b99ef5390 0x561b9a2c8f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:48:12.553 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:12.551+0000 7fc312d9d640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561b99f16070 0x561b9a2d4d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:48:12.556 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:12.552+0000 7fc312d9d640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561b99f16070 0x561b9a2d4d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.39:46126/0 (socket says 172.21.15.39:46126) 2024-09-21T03:48:12.556 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:12.552+0000 7fc312d9d640 1 -- 172.21.15.39:0/2190463342 learned_addr learned my addr 172.21.15.39:0/2190463342 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:48:12.556 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:12.552+0000 7fc31259c640 1 --2- 172.21.15.39:0/2190463342 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561b99eed1b0 0x561b9a2c89e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:48:12.557 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:12.554+0000 7fc312d9d640 1 -- 172.21.15.39:0/2190463342 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561b99ef5390 msgr2=0x561b9a2c8f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:48:12.557 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:12.554+0000 7fc312d9d640 1 --2- 172.21.15.39:0/2190463342 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561b99ef5390 0x561b9a2c8f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:12.557 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:12.554+0000 7fc312d9d640 1 -- 172.21.15.39:0/2190463342 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561b99eed1b0 msgr2=0x561b9a2c89e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:48:12.557 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:12.554+0000 7fc312d9d640 1 --2- 172.21.15.39:0/2190463342 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561b99eed1b0 0x561b9a2c89e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:12.557 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:12.554+0000 7fc312d9d640 1 -- 172.21.15.39:0/2190463342 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561b9a1ebc40 con 0x561b99f16070 2024-09-21T03:48:12.566 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:12.564+0000 7fc312d9d640 1 --2- 172.21.15.39:0/2190463342 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561b99f16070 0x561b9a2d4d10 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fc30c0addd0 tx=0x7fc30c0ad670 comp rx=0 tx=0).ready entity=mon.1 client_cookie=42a1222b37112c5f server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:48:12.567 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:12.565+0000 7fc2f9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.161:3300/0 2024-09-21T03:48:12.568 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:12.565+0000 7fc2f9ffb640 1 -- 172.21.15.39:0/2190463342 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc30c0b0040 con 0x561b99f16070 2024-09-21T03:48:12.568 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:12.565+0000 7fc31f1bf740 1 -- 172.21.15.39:0/2190463342 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc308045070 con 0x561b99f16070 2024-09-21T03:48:12.568 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:12.565+0000 7fc2f9ffb640 1 -- 172.21.15.39:0/2190463342 <== mon.1 v2:172.21.15.161:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8484+0+0 (secure 0 0 0) 0x7fc30c05d1f0 con 0x561b99f16070 2024-09-21T03:48:12.569 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:12.566+0000 7fc2f9ffb640 1 -- 172.21.15.39:0/2190463342 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fc30c0b7860 con 0x561b99f16070 2024-09-21T03:48:12.569 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:12.567+0000 7fc31f1bf740 10 client.25609 fetch_fsmap learned FSMap version 29 2024-09-21T03:48:12.569 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:12.567+0000 7fc31f1bf740 1 -- 172.21.15.39:0/2190463342 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x561b9a2d7840 con 0x561b99f16070 2024-09-21T03:48:12.576 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:12.571+0000 7fc2f9ffb640 1 -- 172.21.15.39:0/2190463342 <== mon.1 v2:172.21.15.161:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc308045070 con 0x561b99f16070 2024-09-21T03:48:12.577 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:12.571+0000 7fc2f9ffb640 1 -- 172.21.15.39:0/2190463342 <== mon.1 v2:172.21.15.161:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fc30c020070 con 0x561b99f16070 2024-09-21T03:48:12.577 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:12.572+0000 7fc31f1bf740 10 client.25609 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T03:48:12.577 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:12.572+0000 7fc31f1bf740 10 client.25609 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T03:48:12.577 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:12.572+0000 7fc31f1bf740 20 client.25609 populate_metadata read hostname 'smithi039' 2024-09-21T03:48:12.577 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:12.572+0000 7fc31f1bf740 1 --2- 172.21.15.39:0/2190463342 >> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] conn(0x561b99ed49f0 0x561b9a2dacd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:48:12.577 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:12.572+0000 7fc31f1bf740 4 client.25609 mds_command: new command op to 24695 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:48:12.577 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:12.572+0000 7fc31f1bf740 1 -- 172.21.15.39:0/2190463342 --> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561b9a235da0 con 0x561b99ed49f0 2024-09-21T03:48:12.579 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:12.577+0000 7fc31259c640 1 --2- 172.21.15.39:0/2190463342 >> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] conn(0x561b99ed49f0 0x561b9a2dacd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:48:12.588 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:12.586+0000 7fc31259c640 1 --2- 172.21.15.39:0/2190463342 >> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] conn(0x561b99ed49f0 0x561b9a2dacd0 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b858111a5a39107 server_cookie=c366b189088bc586 in_seq=0 out_seq=0 2024-09-21T03:48:12.588 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:12.586+0000 7fc2f9ffb640 10 client.25609 ms_handle_connect on v2:172.21.15.150:6838/2949386907 2024-09-21T03:48:12.591 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:12.589+0000 7fc2f9ffb640 1 -- 172.21.15.39:0/2190463342 <== mds.0 v2:172.21.15.150:6838/2949386907 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x561b9a235da0 con 0x561b99ed49f0 2024-09-21T03:48:12.591 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:12.589+0000 7fc2f9ffb640 10 client.25609 handle_command_reply: tid=0 2024-09-21T03:48:12.591 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:12.589+0000 7fc31f1bf740 1 client.25609 shutdown 2024-09-21T03:48:12.592 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:12.590+0000 7fc31f1bf740 1 -- 172.21.15.39:0/2190463342 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561b99f16070 msgr2=0x561b9a2d4d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:48:12.592 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:12.590+0000 7fc31f1bf740 1 --2- 172.21.15.39:0/2190463342 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561b99f16070 0x561b9a2d4d10 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fc30c0addd0 tx=0x7fc30c0ad670 comp rx=0 tx=0).stop 2024-09-21T03:48:12.593 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:12.590+0000 7fc31f1bf740 1 -- 172.21.15.39:0/2190463342 shutdown_connections 2024-09-21T03:48:12.593 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:12.590+0000 7fc31f1bf740 1 --2- 172.21.15.39:0/2190463342 >> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] conn(0x561b99ed49f0 0x561b9a2dacd0 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:12.593 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:12.590+0000 7fc31f1bf740 1 --2- 172.21.15.39:0/2190463342 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561b99f16070 0x561b9a2d4d10 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:12.593 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:12.590+0000 7fc31f1bf740 1 --2- 172.21.15.39:0/2190463342 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561b99ef5390 0x561b9a2c8f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:12.593 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:12.590+0000 7fc31f1bf740 1 --2- 172.21.15.39:0/2190463342 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561b99eed1b0 0x561b9a2c89e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:12.593 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:12.590+0000 7fc31f1bf740 1 -- 172.21.15.39:0/2190463342 >> 172.21.15.39:0/2190463342 conn(0x561b99f0a460 msgr2=0x561b9a2c0400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:48:12.593 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:12.591+0000 7fc2f9ffb640 0 client.25609 ms_handle_reset on v2:172.21.15.150:6838/2949386907 2024-09-21T03:48:12.593 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:12.591+0000 7fc31f1bf740 1 -- 172.21.15.39:0/2190463342 shutdown_connections 2024-09-21T03:48:12.597 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:12.595+0000 7fc31f1bf740 1 -- 172.21.15.39:0/2190463342 wait complete. 2024-09-21T03:48:12.598 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:12.596+0000 7fc31f1bf740 20 client.25609 trim_cache size 0 max 16384 2024-09-21T03:48:12.646 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:12.644+0000 7fc31f1bf740 1 Processor -- start 2024-09-21T03:48:12.646 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:12.644+0000 7fc31f1bf740 1 -- start start 2024-09-21T03:48:12.647 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:12.645+0000 7fc31f1bf740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561b99eed1b0 0x561b9a2c0370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:48:12.651 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:12.649+0000 7fc31f1bf740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561b99ef5390 0x561b9a2c08b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:48:12.651 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:12.649+0000 7fc31f1bf740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561b99f16070 0x561b9a2c5bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:48:12.652 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:12.649+0000 7fc31f1bf740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fc3141c6d40 con 0x561b99eed1b0 2024-09-21T03:48:12.652 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:12.650+0000 7fc311d9b640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561b99ef5390 0x561b9a2c08b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:48:12.652 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:12.650+0000 7fc311d9b640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561b99ef5390 0x561b9a2c08b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.39:53070/0 (socket says 172.21.15.39:53070) 2024-09-21T03:48:12.652 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:12.650+0000 7fc311d9b640 1 -- 172.21.15.39:0/1657139926 learned_addr learned my addr 172.21.15.39:0/1657139926 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:48:12.652 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:12.650+0000 7fc312d9d640 1 --2- 172.21.15.39:0/1657139926 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561b99f16070 0x561b9a2c5bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:48:12.652 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:12.650+0000 7fc31f1bf740 1 -- 172.21.15.39:0/1657139926 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fc3141c8880 con 0x561b99ef5390 2024-09-21T03:48:12.653 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:12.650+0000 7fc31f1bf740 1 -- 172.21.15.39:0/1657139926 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fc3141c7410 con 0x561b99f16070 2024-09-21T03:48:12.655 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:12.652+0000 7fc31259c640 1 --2- 172.21.15.39:0/1657139926 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561b99eed1b0 0x561b9a2c0370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:48:12.655 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:12.653+0000 7fc312d9d640 1 -- 172.21.15.39:0/1657139926 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561b99ef5390 msgr2=0x561b9a2c08b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:48:12.655 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:12.653+0000 7fc312d9d640 1 --2- 172.21.15.39:0/1657139926 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561b99ef5390 0x561b9a2c08b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:12.655 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:12.653+0000 7fc312d9d640 1 -- 172.21.15.39:0/1657139926 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561b99eed1b0 msgr2=0x561b9a2c0370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:48:12.655 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:12.653+0000 7fc312d9d640 1 --2- 172.21.15.39:0/1657139926 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561b99eed1b0 0x561b9a2c0370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:12.655 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:12.653+0000 7fc312d9d640 1 -- 172.21.15.39:0/1657139926 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561b9a2c9ca0 con 0x561b99f16070 2024-09-21T03:48:12.656 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:12.654+0000 7fc312d9d640 1 --2- 172.21.15.39:0/1657139926 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561b99f16070 0x561b9a2c5bd0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fc30c00a6e0 tx=0x7fc30c0a9ce0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cca1d60b77532437 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:48:12.659 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:12.658+0000 7fc2fbfff640 1 -- 172.21.15.39:0/1657139926 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc30c0b0070 con 0x561b99f16070 2024-09-21T03:48:12.659 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:12.658+0000 7fc2fbfff640 1 -- 172.21.15.39:0/1657139926 <== mon.1 v2:172.21.15.161:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc30c0d51c0 con 0x561b99f16070 2024-09-21T03:48:12.661 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:12.659+0000 7fc31f1bf740 1 -- 172.21.15.39:0/1657139926 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561b99f16070 msgr2=0x561b9a2c5bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:48:12.661 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:12.659+0000 7fc31f1bf740 1 --2- 172.21.15.39:0/1657139926 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561b99f16070 0x561b9a2c5bd0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fc30c00a6e0 tx=0x7fc30c0a9ce0 comp rx=0 tx=0).stop 2024-09-21T03:48:12.667 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:12.664+0000 7fc31f1bf740 1 -- 172.21.15.39:0/1657139926 shutdown_connections 2024-09-21T03:48:12.667 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:12.664+0000 7fc31f1bf740 1 --2- 172.21.15.39:0/1657139926 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561b99f16070 0x561b9a2c5bd0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:12.667 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:12.664+0000 7fc31f1bf740 1 --2- 172.21.15.39:0/1657139926 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561b99ef5390 0x561b9a2c08b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:12.667 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:12.664+0000 7fc31f1bf740 1 --2- 172.21.15.39:0/1657139926 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561b99eed1b0 0x561b9a2c0370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:12.667 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:12.664+0000 7fc31f1bf740 1 -- 172.21.15.39:0/1657139926 >> 172.21.15.39:0/1657139926 conn(0x561b99f0a460 msgr2=0x561b9a2c7260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:48:12.671 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:12.666+0000 7fc31f1bf740 1 -- 172.21.15.39:0/1657139926 shutdown_connections 2024-09-21T03:48:12.671 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:12.666+0000 7fc31f1bf740 1 -- 172.21.15.39:0/1657139926 wait complete. 2024-09-21T03:48:12.671 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:12.666+0000 7fc31f1bf740 1 -- start start 2024-09-21T03:48:12.672 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:12.667+0000 7fc31f1bf740 1 Processor -- start 2024-09-21T03:48:12.672 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:12.668+0000 7fc31f1bf740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561b99eed1b0 0x561b9a2d16b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:48:12.672 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:12.668+0000 7fc31f1bf740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561b99ef5390 0x561b9a2ccc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:48:12.672 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:12.668+0000 7fc31f1bf740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561b99f16070 0x561b9a2cd340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:48:12.672 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:12.668+0000 7fc31f1bf740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fc3141c81b0 con 0x561b99eed1b0 2024-09-21T03:48:12.672 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:12.669+0000 7fc31f1bf740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fc3141c7ae0 con 0x561b99ef5390 2024-09-21T03:48:12.672 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:12.669+0000 7fc31f1bf740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fc3141c4d20 con 0x561b99f16070 2024-09-21T03:48:12.672 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:12.669+0000 7fc311d9b640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561b99ef5390 0x561b9a2ccc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:48:12.672 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:12.669+0000 7fc311d9b640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561b99ef5390 0x561b9a2ccc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.39:53074/0 (socket says 172.21.15.39:53074) 2024-09-21T03:48:12.672 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:12.669+0000 7fc311d9b640 1 -- 172.21.15.39:0/3093810014 learned_addr learned my addr 172.21.15.39:0/3093810014 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:48:12.672 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:12.669+0000 7fc31259c640 1 --2- 172.21.15.39:0/3093810014 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561b99eed1b0 0x561b9a2d16b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:48:12.673 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:12.669+0000 7fc31f1bf740 10 client.0 _ll_register_callbacks cb 0x7fc30c004a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:48:12.673 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:12.669+0000 7fc312d9d640 1 --2- 172.21.15.39:0/3093810014 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561b99f16070 0x561b9a2cd340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:48:12.673 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:12.670+0000 7fc311d9b640 1 -- 172.21.15.39:0/3093810014 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561b99f16070 msgr2=0x561b9a2cd340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:48:12.674 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:12.670+0000 7fc311d9b640 1 --2- 172.21.15.39:0/3093810014 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561b99f16070 0x561b9a2cd340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:12.674 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:12.670+0000 7fc311d9b640 1 -- 172.21.15.39:0/3093810014 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561b99eed1b0 msgr2=0x561b9a2d16b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:48:12.674 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:12.670+0000 7fc311d9b640 1 --2- 172.21.15.39:0/3093810014 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561b99eed1b0 0x561b9a2d16b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:12.674 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:12.670+0000 7fc311d9b640 1 -- 172.21.15.39:0/3093810014 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561b9a344690 con 0x561b99ef5390 2024-09-21T03:48:12.674 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:12.671+0000 7fc311d9b640 1 --2- 172.21.15.39:0/3093810014 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561b99ef5390 0x561b9a2ccc30 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fc308005940 tx=0x7fc308037000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d425bf74a3e55f46 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:48:12.674 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:12.671+0000 7fc2f9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.150:3300/0 2024-09-21T03:48:12.677 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:12.674+0000 7fc2f9ffb640 1 -- 172.21.15.39:0/3093810014 <== mon.2 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc308050040 con 0x561b99ef5390 2024-09-21T03:48:12.677 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:12.674+0000 7fc2f9ffb640 1 -- 172.21.15.39:0/3093810014 <== mon.2 v2:172.21.15.150:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8484+0+0 (secure 0 0 0) 0x7fc308040a00 con 0x561b99ef5390 2024-09-21T03:48:12.677 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:12.675+0000 7fc31f1bf740 1 -- 172.21.15.39:0/3093810014 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc30c0d51c0 con 0x561b99ef5390 2024-09-21T03:48:12.678 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:12.676+0000 7fc2f9ffb640 1 -- 172.21.15.39:0/3093810014 <== mon.2 v2:172.21.15.150:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fc308067050 con 0x561b99ef5390 2024-09-21T03:48:12.678 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:12.676+0000 7fc31f1bf740 10 client.25604 fetch_fsmap learned FSMap version 29 2024-09-21T03:48:12.678 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:12.676+0000 7fc31f1bf740 1 -- 172.21.15.39:0/3093810014 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x561b9a345810 con 0x561b99ef5390 2024-09-21T03:48:12.680 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:12.677+0000 7fc2f9ffb640 1 -- 172.21.15.39:0/3093810014 <== mon.2 v2:172.21.15.150:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc308045030 con 0x561b99ef5390 2024-09-21T03:48:12.680 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:12.677+0000 7fc2f9ffb640 1 -- 172.21.15.39:0/3093810014 <== mon.2 v2:172.21.15.150:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fc308056040 con 0x561b99ef5390 2024-09-21T03:48:12.680 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:12.678+0000 7fc31f1bf740 10 client.25604 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T03:48:12.680 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:12.678+0000 7fc31f1bf740 10 client.25604 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T03:48:12.680 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:12.678+0000 7fc31f1bf740 20 client.25604 populate_metadata read hostname 'smithi039' 2024-09-21T03:48:12.680 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:12.678+0000 7fc31f1bf740 1 --2- 172.21.15.39:0/3093810014 >> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] conn(0x561b99ea9da0 0x561b9a347af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:48:12.680 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:12.678+0000 7fc31f1bf740 4 client.25604 mds_command: new command op to 24695 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-21T03:48:12.680 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:12.678+0000 7fc31f1bf740 1 -- 172.21.15.39:0/3093810014 --> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] -- command(tid 0: {"prefix": "scrub status"}) -- 0x7fc30c0d51c0 con 0x561b99ea9da0 2024-09-21T03:48:12.681 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:12.678+0000 7fc31259c640 1 --2- 172.21.15.39:0/3093810014 >> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] conn(0x561b99ea9da0 0x561b9a347af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:48:12.682 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:12.680+0000 7fc2f9ffb640 10 client.25604 ms_handle_connect on v2:172.21.15.150:6838/2949386907 2024-09-21T03:48:12.682 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:12.680+0000 7fc31259c640 1 --2- 172.21.15.39:0/3093810014 >> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] conn(0x561b99ea9da0 0x561b9a347af0 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3127fc3e2ba359a4 server_cookie=a17f5333d6641a3b in_seq=0 out_seq=0 2024-09-21T03:48:12.684 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:12.681+0000 7fc2f9ffb640 1 -- 172.21.15.39:0/3093810014 <== mds.0 v2:172.21.15.150:6838/2949386907 1 ==== command_reply(tid 0: 0 ) ==== 8+0+1226 (crc 0 0 0) 0x7fc30c0d51c0 con 0x561b99ea9da0 2024-09-21T03:48:12.684 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:12.681+0000 7fc2f9ffb640 10 client.25604 handle_command_reply: tid=0 2024-09-21T03:48:12.684 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:12.681+0000 7fc31f1bf740 1 client.25604 shutdown 2024-09-21T03:48:12.685 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:12.682+0000 7fc31f1bf740 1 -- 172.21.15.39:0/3093810014 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561b99ef5390 msgr2=0x561b9a2ccc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:48:12.685 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:12.682+0000 7fc31f1bf740 1 --2- 172.21.15.39:0/3093810014 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561b99ef5390 0x561b9a2ccc30 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fc308005940 tx=0x7fc308037000 comp rx=0 tx=0).stop 2024-09-21T03:48:12.685 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:12.683+0000 7fc31f1bf740 1 -- 172.21.15.39:0/3093810014 shutdown_connections 2024-09-21T03:48:12.686 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:12.683+0000 7fc31f1bf740 1 --2- 172.21.15.39:0/3093810014 >> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] conn(0x561b99ea9da0 0x561b9a347af0 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:12.686 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:12.683+0000 7fc31f1bf740 1 --2- 172.21.15.39:0/3093810014 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561b99f16070 0x561b9a2cd340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:12.686 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:12.683+0000 7fc31f1bf740 1 --2- 172.21.15.39:0/3093810014 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561b99ef5390 0x561b9a2ccc30 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:12.686 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:12.683+0000 7fc31f1bf740 1 --2- 172.21.15.39:0/3093810014 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561b99eed1b0 0x561b9a2d16b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:12.686 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:12.683+0000 7fc31f1bf740 1 -- 172.21.15.39:0/3093810014 >> 172.21.15.39:0/3093810014 conn(0x561b99f0a460 msgr2=0x561b9a2c6e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:48:12.686 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:12.683+0000 7fc2f9ffb640 0 client.25604 ms_handle_reset on v2:172.21.15.150:6838/2949386907 2024-09-21T03:48:12.686 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:12.684+0000 7fc31f1bf740 1 -- 172.21.15.39:0/3093810014 shutdown_connections 2024-09-21T03:48:12.686 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:12.684+0000 7fc31f1bf740 1 -- 172.21.15.39:0/3093810014 wait complete. 2024-09-21T03:48:12.687 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:12.684+0000 7fc31f1bf740 20 client.25604 trim_cache size 0 max 16384 2024-09-21T03:48:12.687 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-09-21T03:48:12.688 INFO:teuthology.orchestra.run.smithi039.stdout: "status": "no active scrubs running", 2024-09-21T03:48:12.688 INFO:teuthology.orchestra.run.smithi039.stdout: "scrubs": {}, 2024-09-21T03:48:12.688 INFO:teuthology.orchestra.run.smithi039.stdout: "scrub_stats": { 2024-09-21T03:48:12.688 INFO:teuthology.orchestra.run.smithi039.stdout: "29411307-eb8d-4339-8d2b-47ebd32c0080": { 2024-09-21T03:48:12.688 INFO:teuthology.orchestra.run.smithi039.stdout: "start_time": "2024-09-21T03:47:38.092920+0000", 2024-09-21T03:48:12.688 INFO:teuthology.orchestra.run.smithi039.stdout: "path": "/", 2024-09-21T03:48:12.688 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_started": 0, 2024-09-21T03:48:12.688 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_passed": 0, 2024-09-21T03:48:12.688 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_failed": 0, 2024-09-21T03:48:12.688 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_skipped": 0 2024-09-21T03:48:12.688 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:48:12.689 INFO:teuthology.orchestra.run.smithi039.stdout: "c7e2b8e4-057d-435f-b2bd-b633de98035f": { 2024-09-21T03:48:12.689 INFO:teuthology.orchestra.run.smithi039.stdout: "start_time": "2024-09-21T03:47:19.400305+0000", 2024-09-21T03:48:12.689 INFO:teuthology.orchestra.run.smithi039.stdout: "path": "/", 2024-09-21T03:48:12.689 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_started": 0, 2024-09-21T03:48:12.689 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_passed": 0, 2024-09-21T03:48:12.689 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_failed": 0, 2024-09-21T03:48:12.689 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_skipped": 0 2024-09-21T03:48:12.689 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:48:12.689 INFO:teuthology.orchestra.run.smithi039.stdout: "10728de8-c49d-4edc-be2b-639249255e94": { 2024-09-21T03:48:12.689 INFO:teuthology.orchestra.run.smithi039.stdout: "start_time": "2024-09-21T03:47:03.965623+0000", 2024-09-21T03:48:12.689 INFO:teuthology.orchestra.run.smithi039.stdout: "path": "/", 2024-09-21T03:48:12.689 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_started": 0, 2024-09-21T03:48:12.690 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_passed": 0, 2024-09-21T03:48:12.690 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_failed": 0, 2024-09-21T03:48:12.690 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_skipped": 0 2024-09-21T03:48:12.690 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:48:12.690 INFO:teuthology.orchestra.run.smithi039.stdout: "02509e4f-3911-4983-b16f-df8ded0b9998": { 2024-09-21T03:48:12.690 INFO:teuthology.orchestra.run.smithi039.stdout: "start_time": "2024-09-21T03:46:17.889576+0000", 2024-09-21T03:48:12.690 INFO:teuthology.orchestra.run.smithi039.stdout: "path": "/", 2024-09-21T03:48:12.690 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_started": 0, 2024-09-21T03:48:12.690 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_passed": 0, 2024-09-21T03:48:12.690 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_failed": 0, 2024-09-21T03:48:12.690 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_skipped": 0 2024-09-21T03:48:12.690 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-21T03:48:12.691 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-21T03:48:12.691 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-09-21T03:48:12.692 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:12.689+0000 7fc318c4b640 1 -- 172.21.15.39:0/2025319476 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7fc2fc0b6880 msgr2=0x7fc2fc0b8d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:48:12.692 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:12.689+0000 7fc318c4b640 1 --2- 172.21.15.39:0/2025319476 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7fc2fc0b6880 0x7fc2fc0b8d90 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fc308000ea0 tx=0x7fc308004bb0 comp rx=0 tx=0).stop 2024-09-21T03:48:12.692 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:12.689+0000 7fc318c4b640 1 -- 172.21.15.39:0/2025319476 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc314118900 msgr2=0x7fc314116fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:48:12.692 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:12.689+0000 7fc318c4b640 1 --2- 172.21.15.39:0/2025319476 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc314118900 0x7fc314116fe0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fc30c00cab0 tx=0x7fc30c00cf70 comp rx=0 tx=0).stop 2024-09-21T03:48:12.692 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:12.689+0000 7fc318c4b640 1 -- 172.21.15.39:0/2025319476 shutdown_connections 2024-09-21T03:48:12.692 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:12.689+0000 7fc318c4b640 1 --2- 172.21.15.39:0/2025319476 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7fc2fc0b6880 0x7fc2fc0b8d90 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:12.692 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:12.690+0000 7fc318c4b640 1 --2- 172.21.15.39:0/2025319476 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc314118900 0x7fc314116fe0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:12.692 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:12.690+0000 7fc318c4b640 1 --2- 172.21.15.39:0/2025319476 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc31410ad10 0x7fc31410e4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:12.693 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:12.690+0000 7fc318c4b640 1 --2- 172.21.15.39:0/2025319476 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc31410a3c0 0x7fc31410df70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:12.693 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:12.690+0000 7fc318c4b640 1 -- 172.21.15.39:0/2025319476 >> 172.21.15.39:0/2025319476 conn(0x7fc3140712f0 msgr2=0x7fc31411a970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:48:12.693 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:12.690+0000 7fc318c4b640 1 -- 172.21.15.39:0/2025319476 shutdown_connections 2024-09-21T03:48:12.693 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:12.690+0000 7fc318c4b640 1 -- 172.21.15.39:0/2025319476 wait complete. 2024-09-21T03:48:13.084 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:48:12 smithi039 ceph-mon[31211]: pgmap v390: 97 pgs: 97 active+clean; 4.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 387 KiB/s rd, 386 MiB/s wr, 307 op/s 2024-09-21T03:48:13.218 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:48:12 smithi161 ceph-mon[34533]: pgmap v390: 97 pgs: 97 active+clean; 4.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 387 KiB/s rd, 386 MiB/s wr, 307 op/s 2024-09-21T03:48:13.224 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:48:12 smithi150 ceph-mon[34289]: pgmap v390: 97 pgs: 97 active+clean; 4.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 387 KiB/s rd, 386 MiB/s wr, 307 op/s 2024-09-21T03:48:13.511 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-21T03:48:13.511 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 51116d5c-77ca-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T03:48:13.932 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/51116d5c-77ca-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:48:14.567 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:14.565+0000 7f6e445ed640 1 -- 172.21.15.39:0/2477093152 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6e3c108cf0 msgr2=0x7f6e3c10f520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:48:14.567 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:14.565+0000 7f6e445ed640 1 --2- 172.21.15.39:0/2477093152 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6e3c108cf0 0x7f6e3c10f520 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f6e34009ca0 tx=0x7f6e3402a8d0 comp rx=0 tx=0).stop 2024-09-21T03:48:14.567 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:14.565+0000 7f6e445ed640 1 -- 172.21.15.39:0/2477093152 shutdown_connections 2024-09-21T03:48:14.567 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:14.565+0000 7f6e445ed640 1 --2- 172.21.15.39:0/2477093152 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6e3c108cf0 0x7f6e3c10f520 unknown :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:14.568 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:14.565+0000 7f6e445ed640 1 --2- 172.21.15.39:0/2477093152 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6e3c1083a0 0x7f6e3c1087b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:14.568 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:14.565+0000 7f6e445ed640 1 --2- 172.21.15.39:0/2477093152 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6e3c102a60 0x7f6e3c102e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:14.568 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:14.565+0000 7f6e445ed640 1 -- 172.21.15.39:0/2477093152 >> 172.21.15.39:0/2477093152 conn(0x7f6e3c079730 msgr2=0x7f6e3c079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:48:14.568 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:14.565+0000 7f6e445ed640 1 -- 172.21.15.39:0/2477093152 shutdown_connections 2024-09-21T03:48:14.568 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:14.565+0000 7f6e445ed640 1 -- 172.21.15.39:0/2477093152 wait complete. 2024-09-21T03:48:14.568 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:14.566+0000 7f6e445ed640 1 Processor -- start 2024-09-21T03:48:14.568 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:14.567+0000 7f6e445ed640 1 -- start start 2024-09-21T03:48:14.570 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:14.567+0000 7f6e445ed640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6e3c102a60 0x7f6e3c10ffb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:48:14.571 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:14.567+0000 7f6e445ed640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6e3c1083a0 0x7f6e3c1104f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:48:14.571 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:14.567+0000 7f6e445ed640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6e3c108cf0 0x7f6e3c110a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:48:14.571 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:14.567+0000 7f6e445ed640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f6e3c113ae0 con 0x7f6e3c102a60 2024-09-21T03:48:14.571 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:14.567+0000 7f6e445ed640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f6e3c113960 con 0x7f6e3c108cf0 2024-09-21T03:48:14.571 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:14.567+0000 7f6e445ed640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f6e3c113c60 con 0x7f6e3c1083a0 2024-09-21T03:48:14.571 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:14.567+0000 7f6e42389640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6e3c102a60 0x7f6e3c10ffb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:48:14.571 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:14.567+0000 7f6e42389640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6e3c102a60 0x7f6e3c10ffb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:44254/0 (socket says 172.21.15.39:44254) 2024-09-21T03:48:14.571 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:14.567+0000 7f6e42389640 1 -- 172.21.15.39:0/3533890306 learned_addr learned my addr 172.21.15.39:0/3533890306 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:48:14.571 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:14.568+0000 7f6e42b8a640 1 --2- 172.21.15.39:0/3533890306 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6e3c108cf0 0x7f6e3c110a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:48:14.571 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:14.568+0000 7f6e42389640 1 -- 172.21.15.39:0/3533890306 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6e3c1083a0 msgr2=0x7f6e3c1104f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:48:14.571 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:14.568+0000 7f6e42389640 1 --2- 172.21.15.39:0/3533890306 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6e3c1083a0 0x7f6e3c1104f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:14.571 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:14.568+0000 7f6e42389640 1 -- 172.21.15.39:0/3533890306 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6e3c108cf0 msgr2=0x7f6e3c110a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:48:14.571 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:14.568+0000 7f6e42389640 1 --2- 172.21.15.39:0/3533890306 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6e3c108cf0 0x7f6e3c110a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:14.571 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:14.568+0000 7f6e42389640 1 -- 172.21.15.39:0/3533890306 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6e3c1a8700 con 0x7f6e3c102a60 2024-09-21T03:48:14.572 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:14.568+0000 7f6e42b8a640 1 --2- 172.21.15.39:0/3533890306 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6e3c108cf0 0x7f6e3c110a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:48:14.572 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:14.568+0000 7f6e42389640 1 --2- 172.21.15.39:0/3533890306 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6e3c102a60 0x7f6e3c10ffb0 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f6e3800b450 tx=0x7f6e3800b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:48:14.572 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:14.569+0000 7f6e337fe640 1 -- 172.21.15.39:0/3533890306 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e38015020 con 0x7f6e3c102a60 2024-09-21T03:48:14.572 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:14.569+0000 7f6e337fe640 1 -- 172.21.15.39:0/3533890306 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6e38011d40 con 0x7f6e3c102a60 2024-09-21T03:48:14.572 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:14.569+0000 7f6e337fe640 1 -- 172.21.15.39:0/3533890306 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e3800f340 con 0x7f6e3c102a60 2024-09-21T03:48:14.572 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:14.569+0000 7f6e445ed640 1 -- 172.21.15.39:0/3533890306 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6e3c1a89f0 con 0x7f6e3c102a60 2024-09-21T03:48:14.572 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:14.569+0000 7f6e445ed640 1 -- 172.21.15.39:0/3533890306 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6e3c1a8f60 con 0x7f6e3c102a60 2024-09-21T03:48:14.573 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:14.571+0000 7f6e445ed640 1 -- 172.21.15.39:0/3533890306 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6e3c102e70 con 0x7f6e3c102a60 2024-09-21T03:48:14.576 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:14.574+0000 7f6e337fe640 1 -- 172.21.15.39:0/3533890306 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 27) ==== 154643+0+0 (secure 0 0 0) 0x7f6e38018020 con 0x7f6e3c102a60 2024-09-21T03:48:14.578 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:14.576+0000 7f6e337fe640 1 --2- 172.21.15.39:0/3533890306 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f6e280b6780 0x7f6e280b8c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:48:14.579 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:14.577+0000 7f6e337fe640 1 -- 172.21.15.39:0/3533890306 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8484+0+0 (secure 0 0 0) 0x7f6e380e6e10 con 0x7f6e3c102a60 2024-09-21T03:48:14.580 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:14.578+0000 7f6e41b88640 1 --2- 172.21.15.39:0/3533890306 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f6e280b6780 0x7f6e280b8c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:48:14.581 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:14.578+0000 7f6e41b88640 1 --2- 172.21.15.39:0/3533890306 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f6e280b6780 0x7f6e280b8c90 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f6e3c105eb0 tx=0x7f6e2c005c70 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:48:14.585 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:14.584+0000 7f6e337fe640 1 -- 172.21.15.39:0/3533890306 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f6e3801f030 con 0x7f6e3c102a60 2024-09-21T03:48:14.889 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:14.887+0000 7f6e445ed640 1 -- 172.21.15.39:0/3533890306 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f6e3c02e640 con 0x7f6e3c102a60 2024-09-21T03:48:14.890 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:14.888+0000 7f6e337fe640 1 -- 172.21.15.39:0/3533890306 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11978 (secure 0 0 0) 0x7f6e3800d040 con 0x7f6e3c102a60 2024-09-21T03:48:14.891 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T03:48:14.891 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":29,"btime":"2024-09-21T03:46:08:982881+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14790,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6835/508995295","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":508995295},{"type":"v1","addr":"172.21.15.39:6835","nonce":508995295}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14802,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6837/3915912607","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":3915912607},{"type":"v1","addr":"172.21.15.39:6837","nonce":3915912607}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":29,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T03:44:15.009872+0000","modified":"2024-09-21T03:46:08.430113+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24695,"mds_1":24667,"mds_2":24659,"mds_3":24646,"mds_4":14868},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14856":{"gid":14856,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.39:6839/185509192","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6838","nonce":185509192},{"type":"v1","addr":"172.21.15.39:6839","nonce":185509192}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14868":{"gid":14868,"name":"a","rank":4,"incarnation":25,"state":"up:active","state_seq":7,"addr":"172.21.15.39:6841/4247190456","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6840","nonce":4247190456},{"type":"v1","addr":"172.21.15.39:6841","nonce":4247190456}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24599":{"gid":24599,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.150:6833/3114643217","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":3114643217},{"type":"v1","addr":"172.21.15.150:6833","nonce":3114643217}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24601":{"gid":24601,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.161:6833/2963990654","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":2963990654},{"type":"v1","addr":"172.21.15.161:6833","nonce":2963990654}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24632":{"gid":24632,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.150:6835/724594008","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6834","nonce":724594008},{"type":"v1","addr":"172.21.15.150:6835","nonce":724594008}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24640":{"gid":24640,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.161:6835/3747882110","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":3747882110},{"type":"v1","addr":"172.21.15.161:6835","nonce":3747882110}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24646":{"gid":24646,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":10,"addr":"172.21.15.161:6837/3451544597","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6836","nonce":3451544597},{"type":"v1","addr":"172.21.15.161:6837","nonce":3451544597}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24659":{"gid":24659,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.150:6837/4017043144","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6836","nonce":4017043144},{"type":"v1","addr":"172.21.15.150:6837","nonce":4017043144}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24667":{"gid":24667,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.161:6839/2190107270","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6838","nonce":2190107270},{"type":"v1","addr":"172.21.15.161:6839","nonce":2190107270}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24695":{"gid":24695,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.150:6839/2949386907","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6838","nonce":2949386907},{"type":"v1","addr":"172.21.15.150:6839","nonce":2949386907}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24695,"qdb_cluster":[24695,24667,24659,24646,14868]},"id":1}]} 2024-09-21T03:48:14.891 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 29 2024-09-21T03:48:14.895 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:14.892+0000 7f6e445ed640 1 -- 172.21.15.39:0/3533890306 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f6e280b6780 msgr2=0x7f6e280b8c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:48:14.895 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:14.892+0000 7f6e445ed640 1 --2- 172.21.15.39:0/3533890306 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f6e280b6780 0x7f6e280b8c90 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f6e3c105eb0 tx=0x7f6e2c005c70 comp rx=0 tx=0).stop 2024-09-21T03:48:14.895 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:14.892+0000 7f6e445ed640 1 -- 172.21.15.39:0/3533890306 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6e3c102a60 msgr2=0x7f6e3c10ffb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:48:14.895 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:14.892+0000 7f6e445ed640 1 --2- 172.21.15.39:0/3533890306 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6e3c102a60 0x7f6e3c10ffb0 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f6e3800b450 tx=0x7f6e3800b910 comp rx=0 tx=0).stop 2024-09-21T03:48:14.895 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:14.893+0000 7f6e445ed640 1 -- 172.21.15.39:0/3533890306 shutdown_connections 2024-09-21T03:48:14.895 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:14.893+0000 7f6e445ed640 1 --2- 172.21.15.39:0/3533890306 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f6e280b6780 0x7f6e280b8c90 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:14.895 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:14.893+0000 7f6e445ed640 1 --2- 172.21.15.39:0/3533890306 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6e3c108cf0 0x7f6e3c110a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:14.895 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:14.893+0000 7f6e445ed640 1 --2- 172.21.15.39:0/3533890306 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6e3c1083a0 0x7f6e3c1104f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:14.895 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:14.893+0000 7f6e445ed640 1 --2- 172.21.15.39:0/3533890306 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6e3c102a60 0x7f6e3c10ffb0 unknown :-1 s=CLOSED pgs=609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:14.895 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:14.893+0000 7f6e445ed640 1 -- 172.21.15.39:0/3533890306 >> 172.21.15.39:0/3533890306 conn(0x7f6e3c079730 msgr2=0x7f6e3c104410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:48:14.896 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:14.893+0000 7f6e445ed640 1 -- 172.21.15.39:0/3533890306 shutdown_connections 2024-09-21T03:48:14.896 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:14.893+0000 7f6e445ed640 1 -- 172.21.15.39:0/3533890306 wait complete. 2024-09-21T03:48:15.207 INFO:tasks.workunit.client.0.smithi039.stdout: 2024-09-21T03:48:15.207 INFO:tasks.workunit.client.0.smithi039.stdout: Children see throughput for 1 initial writers = 820463.31 kB/sec 2024-09-21T03:48:15.207 INFO:tasks.workunit.client.0.smithi039.stdout: Parent sees throughput for 1 initial writers = 820448.69 kB/sec 2024-09-21T03:48:15.207 INFO:tasks.workunit.client.0.smithi039.stdout: Min throughput per process = 820463.31 kB/sec 2024-09-21T03:48:15.207 INFO:tasks.workunit.client.0.smithi039.stdout: Max throughput per process = 820463.31 kB/sec 2024-09-21T03:48:15.207 INFO:tasks.workunit.client.0.smithi039.stdout: Avg throughput per process = 820463.31 kB/sec 2024-09-21T03:48:15.207 INFO:tasks.workunit.client.0.smithi039.stdout: Min xfer = 10485760.00 kB 2024-09-21T03:48:15.334 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:48:14 smithi039 ceph-mon[31211]: pgmap v391: 97 pgs: 97 active+clean; 6.6 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 387 KiB/s rd, 565 MiB/s wr, 351 op/s 2024-09-21T03:48:15.468 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:48:15 smithi161 ceph-mon[34533]: pgmap v391: 97 pgs: 97 active+clean; 6.6 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 387 KiB/s rd, 565 MiB/s wr, 351 op/s 2024-09-21T03:48:15.474 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:48:15 smithi150 ceph-mon[34289]: pgmap v391: 97 pgs: 97 active+clean; 6.6 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 387 KiB/s rd, 565 MiB/s wr, 351 op/s 2024-09-21T03:48:16.013 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 51116d5c-77ca-11ef-baf5-efdc52797490 -- ceph tell mds.1:4 damage ls 2024-09-21T03:48:16.279 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/51116d5c-77ca-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:48:16.334 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:48:15 smithi039 ceph-mon[31211]: from='client.? 172.21.15.39:0/3533890306' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:48:16.467 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:48:15 smithi161 ceph-mon[34533]: from='client.? 172.21.15.39:0/3533890306' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:48:16.474 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:48:15 smithi150 ceph-mon[34289]: from='client.? 172.21.15.39:0/3533890306' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:48:16.875 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:16.858+0000 7fc1918a2640 1 -- 172.21.15.39:0/1425051760 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc18c1083d0 msgr2=0x7fc18c1087e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:48:16.876 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:16.858+0000 7fc1918a2640 1 --2- 172.21.15.39:0/1425051760 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc18c1083d0 0x7fc18c1087e0 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7fc174009900 tx=0x7fc17402a410 comp rx=0 tx=0).stop 2024-09-21T03:48:16.876 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:16.858+0000 7fc1918a2640 1 -- 172.21.15.39:0/1425051760 shutdown_connections 2024-09-21T03:48:16.876 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:16.858+0000 7fc1918a2640 1 --2- 172.21.15.39:0/1425051760 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc18c108d20 0x7fc18c10f550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:16.876 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:16.858+0000 7fc1918a2640 1 --2- 172.21.15.39:0/1425051760 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc18c1083d0 0x7fc18c1087e0 unknown :-1 s=CLOSED pgs=610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:16.876 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:16.858+0000 7fc1918a2640 1 --2- 172.21.15.39:0/1425051760 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc18c102ae0 0x7fc18c102ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:16.876 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:16.858+0000 7fc1918a2640 1 -- 172.21.15.39:0/1425051760 >> 172.21.15.39:0/1425051760 conn(0x7fc18c079760 msgr2=0x7fc18c079b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:48:16.876 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:16.858+0000 7fc1918a2640 1 -- 172.21.15.39:0/1425051760 shutdown_connections 2024-09-21T03:48:16.876 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:16.859+0000 7fc1918a2640 1 -- 172.21.15.39:0/1425051760 wait complete. 2024-09-21T03:48:16.876 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:16.859+0000 7fc1918a2640 1 Processor -- start 2024-09-21T03:48:16.876 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:16.859+0000 7fc1918a2640 1 -- start start 2024-09-21T03:48:16.876 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:16.860+0000 7fc1918a2640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc18c102ae0 0x7fc18c19a460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:48:16.876 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:16.860+0000 7fc1918a2640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc18c1083d0 0x7fc18c19aa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:48:16.876 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:16.860+0000 7fc1918a2640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc18c108d20 0x7fc18c19fca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:48:16.877 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:16.860+0000 7fc1918a2640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fc18c113ab0 con 0x7fc18c102ae0 2024-09-21T03:48:16.877 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:16.860+0000 7fc1918a2640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fc18c113930 con 0x7fc18c1083d0 2024-09-21T03:48:16.877 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:16.860+0000 7fc1918a2640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fc18c113c30 con 0x7fc18c108d20 2024-09-21T03:48:16.877 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:16.860+0000 7fc18affd640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc18c102ae0 0x7fc18c19a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:48:16.877 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:16.860+0000 7fc18affd640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc18c102ae0 0x7fc18c19a460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:44286/0 (socket says 172.21.15.39:44286) 2024-09-21T03:48:16.877 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:16.860+0000 7fc18affd640 1 -- 172.21.15.39:0/1558573262 learned_addr learned my addr 172.21.15.39:0/1558573262 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:48:16.877 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:16.860+0000 7fc1827fc640 1 --2- 172.21.15.39:0/1558573262 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc18c1083d0 0x7fc18c19aa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:48:16.877 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:16.860+0000 7fc18b7fe640 1 --2- 172.21.15.39:0/1558573262 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc18c108d20 0x7fc18c19fca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:48:16.877 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:16.860+0000 7fc18affd640 1 -- 172.21.15.39:0/1558573262 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc18c108d20 msgr2=0x7fc18c19fca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:48:16.877 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:16.860+0000 7fc18affd640 1 --2- 172.21.15.39:0/1558573262 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc18c108d20 0x7fc18c19fca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:16.877 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:16.860+0000 7fc18affd640 1 -- 172.21.15.39:0/1558573262 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc18c1083d0 msgr2=0x7fc18c19aa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:48:16.877 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:16.860+0000 7fc18affd640 1 --2- 172.21.15.39:0/1558573262 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc18c1083d0 0x7fc18c19aa00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:16.877 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:16.860+0000 7fc18affd640 1 -- 172.21.15.39:0/1558573262 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc18c1a03d0 con 0x7fc18c102ae0 2024-09-21T03:48:16.877 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:16.860+0000 7fc1827fc640 1 --2- 172.21.15.39:0/1558573262 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc18c1083d0 0x7fc18c19aa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:48:16.877 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:16.860+0000 7fc18b7fe640 1 --2- 172.21.15.39:0/1558573262 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc18c108d20 0x7fc18c19fca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:48:16.878 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:16.860+0000 7fc18affd640 1 --2- 172.21.15.39:0/1558573262 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc18c102ae0 0x7fc18c19a460 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7fc17c00d770 tx=0x7fc17c00dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:48:16.878 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:16.860+0000 7fc1908a0640 1 -- 172.21.15.39:0/1558573262 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc17c017070 con 0x7fc18c102ae0 2024-09-21T03:48:16.878 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:16.861+0000 7fc1918a2640 1 -- 172.21.15.39:0/1558573262 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc18c1a06c0 con 0x7fc18c102ae0 2024-09-21T03:48:16.878 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:16.861+0000 7fc1908a0640 1 -- 172.21.15.39:0/1558573262 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc17c0054d0 con 0x7fc18c102ae0 2024-09-21T03:48:16.878 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:16.861+0000 7fc1908a0640 1 -- 172.21.15.39:0/1558573262 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc17c004c30 con 0x7fc18c102ae0 2024-09-21T03:48:16.878 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:16.861+0000 7fc1918a2640 1 -- 172.21.15.39:0/1558573262 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc18c1a6e20 con 0x7fc18c102ae0 2024-09-21T03:48:16.879 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:16.863+0000 7fc197e16740 1 Processor -- start 2024-09-21T03:48:16.879 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:16.863+0000 7fc197e16740 1 -- start start 2024-09-21T03:48:16.879 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:16.864+0000 7fc1908a0640 1 -- 172.21.15.39:0/1558573262 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 27) ==== 154643+0+0 (secure 0 0 0) 0x7fc17c004e10 con 0x7fc18c102ae0 2024-09-21T03:48:16.879 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:16.864+0000 7fc197e16740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56205083a070 0x562050be16a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:48:16.879 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:16.864+0000 7fc197e16740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5620508111b0 0x562050be1be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:48:16.879 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:16.864+0000 7fc197e16740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562050819390 0x562050be2120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:48:16.879 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:16.864+0000 7fc197e16740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x562050b57ea0 con 0x5620508111b0 2024-09-21T03:48:16.880 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:16.864+0000 7fc197e16740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x562050b472d0 con 0x562050819390 2024-09-21T03:48:16.880 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:16.864+0000 7fc197e16740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x562050b4b0e0 con 0x56205083a070 2024-09-21T03:48:16.880 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:16.864+0000 7fc18affd640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5620508111b0 0x562050be1be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:48:16.880 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:16.864+0000 7fc18affd640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5620508111b0 0x562050be1be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:44294/0 (socket says 172.21.15.39:44294) 2024-09-21T03:48:16.880 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:16.864+0000 7fc18affd640 1 -- 172.21.15.39:0/1729406539 learned_addr learned my addr 172.21.15.39:0/1729406539 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:48:16.880 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:16.864+0000 7fc18b7fe640 1 --2- 172.21.15.39:0/1729406539 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56205083a070 0x562050be16a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:48:16.880 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:16.864+0000 7fc1827fc640 1 --2- 172.21.15.39:0/1729406539 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562050819390 0x562050be2120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:48:16.881 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:16.864+0000 7fc18affd640 1 -- 172.21.15.39:0/1729406539 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56205083a070 msgr2=0x562050be16a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:48:16.881 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:16.864+0000 7fc18affd640 1 --2- 172.21.15.39:0/1729406539 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56205083a070 0x562050be16a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:16.881 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:16.864+0000 7fc18affd640 1 -- 172.21.15.39:0/1729406539 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562050819390 msgr2=0x562050be2120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:48:16.881 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:16.864+0000 7fc18affd640 1 --2- 172.21.15.39:0/1729406539 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562050819390 0x562050be2120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:16.881 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:16.864+0000 7fc18affd640 1 -- 172.21.15.39:0/1729406539 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5620508a5650 con 0x5620508111b0 2024-09-21T03:48:16.881 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:16.864+0000 7fc1827fc640 1 --2- 172.21.15.39:0/1729406539 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562050819390 0x562050be2120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:48:16.881 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:16.864+0000 7fc18affd640 1 --2- 172.21.15.39:0/1729406539 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5620508111b0 0x562050be1be0 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7fc17c0e49a0 tx=0x7fc17c0ef9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=baffd7a6f95428d server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:48:16.882 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:16.865+0000 7fc1908a0640 1 --2- 172.21.15.39:0/1558573262 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7fc1780b6680 0x7fc1780b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:48:16.882 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:16.865+0000 7fc181ffb640 1 -- 172.21.15.39:0/1729406539 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc17c0fb070 con 0x5620508111b0 2024-09-21T03:48:16.882 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:16.865+0000 7fc18b7fe640 1 --2- 172.21.15.39:0/1558573262 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7fc1780b6680 0x7fc1780b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:48:16.882 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:16.865+0000 7fc181ffb640 1 -- 172.21.15.39:0/1729406539 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc17c0acc40 con 0x5620508111b0 2024-09-21T03:48:16.882 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:16.865+0000 7fc181ffb640 1 -- 172.21.15.39:0/1729406539 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc17c08db70 con 0x5620508111b0 2024-09-21T03:48:16.882 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:16.865+0000 7fc1908a0640 1 -- 172.21.15.39:0/1558573262 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8484+0+0 (secure 0 0 0) 0x7fc17c0e72c0 con 0x7fc18c102ae0 2024-09-21T03:48:16.882 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:16.866+0000 7fc197e16740 1 -- 172.21.15.39:0/1729406539 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5620508111b0 msgr2=0x562050be1be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:48:16.883 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:16.866+0000 7fc197e16740 1 --2- 172.21.15.39:0/1729406539 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5620508111b0 0x562050be1be0 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7fc17c0e49a0 tx=0x7fc17c0ef9b0 comp rx=0 tx=0).stop 2024-09-21T03:48:16.883 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:16.866+0000 7fc18b7fe640 1 --2- 172.21.15.39:0/1558573262 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7fc1780b6680 0x7fc1780b8b90 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fc184005fd0 tx=0x7fc184009800 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:48:16.883 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:16.866+0000 7fc197e16740 1 -- 172.21.15.39:0/1729406539 shutdown_connections 2024-09-21T03:48:16.883 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:16.866+0000 7fc197e16740 1 --2- 172.21.15.39:0/1729406539 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562050819390 0x562050be2120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:16.883 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:16.866+0000 7fc197e16740 1 --2- 172.21.15.39:0/1729406539 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5620508111b0 0x562050be1be0 unknown :-1 s=CLOSED pgs=612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:16.883 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:16.866+0000 7fc197e16740 1 --2- 172.21.15.39:0/1729406539 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56205083a070 0x562050be16a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:16.883 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:16.866+0000 7fc197e16740 1 -- 172.21.15.39:0/1729406539 >> 172.21.15.39:0/1729406539 conn(0x56205082e460 msgr2=0x562050bdf590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:48:16.884 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:16.866+0000 7fc197e16740 1 -- 172.21.15.39:0/1729406539 shutdown_connections 2024-09-21T03:48:16.884 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:16.866+0000 7fc197e16740 1 -- 172.21.15.39:0/1729406539 wait complete. 2024-09-21T03:48:16.884 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:16.866+0000 7fc197e16740 1 -- start start 2024-09-21T03:48:16.884 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:16.866+0000 7fc197e16740 1 Processor -- start 2024-09-21T03:48:16.884 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:16.867+0000 7fc197e16740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5620508111b0 0x562050beccd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:48:16.884 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:16.867+0000 7fc197e16740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562050819390 0x562050bed210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:48:16.885 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:16.867+0000 7fc197e16740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56205083a070 0x562050bed750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:48:16.885 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:16.867+0000 7fc197e16740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x5620509a8700 con 0x56205083a070 2024-09-21T03:48:16.885 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:16.867+0000 7fc197e16740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x562050ae2160 con 0x5620508111b0 2024-09-21T03:48:16.885 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:16.867+0000 7fc197e16740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x562050befe80 con 0x562050819390 2024-09-21T03:48:16.885 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:16.867+0000 7fc197e16740 10 client.0 _ll_register_callbacks cb 0x562050afb9e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:48:16.885 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:16.867+0000 7fc18b7fe640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56205083a070 0x562050bed750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:48:16.886 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:16.867+0000 7fc18b7fe640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56205083a070 0x562050bed750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:44306/0 (socket says 172.21.15.39:44306) 2024-09-21T03:48:16.886 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:16.867+0000 7fc18b7fe640 1 -- 172.21.15.39:0/171876895 learned_addr learned my addr 172.21.15.39:0/171876895 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:48:16.886 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:16.867+0000 7fc18affd640 1 --2- 172.21.15.39:0/171876895 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5620508111b0 0x562050beccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:48:16.886 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:16.867+0000 7fc1827fc640 1 --2- 172.21.15.39:0/171876895 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562050819390 0x562050bed210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:48:16.886 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:16.867+0000 7fc18b7fe640 1 -- 172.21.15.39:0/171876895 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562050819390 msgr2=0x562050bed210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:48:16.886 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:16.867+0000 7fc18b7fe640 1 --2- 172.21.15.39:0/171876895 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562050819390 0x562050bed210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:16.887 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:16.867+0000 7fc18b7fe640 1 -- 172.21.15.39:0/171876895 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5620508111b0 msgr2=0x562050beccd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:48:16.887 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:16.867+0000 7fc18b7fe640 1 --2- 172.21.15.39:0/171876895 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5620508111b0 0x562050beccd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:16.887 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:16.867+0000 7fc18b7fe640 1 -- 172.21.15.39:0/171876895 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562050b0fc90 con 0x56205083a070 2024-09-21T03:48:16.887 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:16.868+0000 7fc18b7fe640 1 --2- 172.21.15.39:0/171876895 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56205083a070 0x562050bed750 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7fc184011cc0 tx=0x7fc184004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6bf122839860022 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:48:16.887 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:16.868+0000 7fc16affd640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-09-21T03:48:16.887 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:16.868+0000 7fc16affd640 1 -- 172.21.15.39:0/171876895 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc18401ed30 con 0x56205083a070 2024-09-21T03:48:16.888 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:16.868+0000 7fc16affd640 1 -- 172.21.15.39:0/171876895 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8484+0+0 (secure 0 0 0) 0x7fc18401f410 con 0x56205083a070 2024-09-21T03:48:16.888 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:16.868+0000 7fc197e16740 1 -- 172.21.15.39:0/171876895 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc17c0acc40 con 0x56205083a070 2024-09-21T03:48:16.888 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:16.868+0000 7fc16affd640 1 -- 172.21.15.39:0/171876895 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fc184011e70 con 0x56205083a070 2024-09-21T03:48:16.888 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:16.868+0000 7fc197e16740 10 client.16050 fetch_fsmap learned FSMap version 29 2024-09-21T03:48:16.888 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:16.868+0000 7fc197e16740 1 -- 172.21.15.39:0/171876895 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x562050bfbb40 con 0x56205083a070 2024-09-21T03:48:16.888 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:16.868+0000 7fc16affd640 1 -- 172.21.15.39:0/171876895 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc17c0acc40 con 0x56205083a070 2024-09-21T03:48:16.888 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:16.868+0000 7fc16affd640 1 -- 172.21.15.39:0/171876895 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fc184031020 con 0x56205083a070 2024-09-21T03:48:16.888 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:16.868+0000 7fc197e16740 10 client.16050 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T03:48:16.889 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:16.868+0000 7fc197e16740 10 client.16050 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.a 2024-09-21T03:48:16.889 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:16.868+0000 7fc197e16740 20 client.16050 populate_metadata read hostname 'smithi039' 2024-09-21T03:48:16.889 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:16.868+0000 7fc197e16740 1 --2- 172.21.15.39:0/171876895 >> [v2:172.21.15.39:6840/4247190456,v1:172.21.15.39:6841/4247190456] conn(0x5620507f89f0 0x562050bfef80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:48:16.889 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:16.868+0000 7fc197e16740 4 client.16050 mds_command: new command op to 14868 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:48:16.889 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:16.868+0000 7fc197e16740 1 -- 172.21.15.39:0/171876895 --> [v2:172.21.15.39:6840/4247190456,v1:172.21.15.39:6841/4247190456] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562050b59df0 con 0x5620507f89f0 2024-09-21T03:48:16.889 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:16.869+0000 7fc18affd640 1 --2- 172.21.15.39:0/171876895 >> [v2:172.21.15.39:6840/4247190456,v1:172.21.15.39:6841/4247190456] conn(0x5620507f89f0 0x562050bfef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:48:16.889 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:16.869+0000 7fc16affd640 10 client.16050 ms_handle_connect on v2:172.21.15.39:6840/4247190456 2024-09-21T03:48:16.890 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:16.869+0000 7fc18affd640 1 --2- 172.21.15.39:0/171876895 >> [v2:172.21.15.39:6840/4247190456,v1:172.21.15.39:6841/4247190456] conn(0x5620507f89f0 0x562050bfef80 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=617cfb1aa2a0d167 server_cookie=abda889101106a62 in_seq=0 out_seq=0 2024-09-21T03:48:16.890 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:16.870+0000 7fc16affd640 1 -- 172.21.15.39:0/171876895 <== mds.4 v2:172.21.15.39:6840/4247190456 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x562050b59df0 con 0x5620507f89f0 2024-09-21T03:48:16.890 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:16.870+0000 7fc16affd640 10 client.16050 handle_command_reply: tid=0 2024-09-21T03:48:16.890 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:16.870+0000 7fc197e16740 1 client.16050 shutdown 2024-09-21T03:48:16.890 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:16.870+0000 7fc197e16740 1 -- 172.21.15.39:0/171876895 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56205083a070 msgr2=0x562050bed750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:48:16.890 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:16.870+0000 7fc197e16740 1 --2- 172.21.15.39:0/171876895 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56205083a070 0x562050bed750 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7fc184011cc0 tx=0x7fc184004a40 comp rx=0 tx=0).stop 2024-09-21T03:48:16.890 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:16.870+0000 7fc197e16740 1 -- 172.21.15.39:0/171876895 shutdown_connections 2024-09-21T03:48:16.891 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:16.870+0000 7fc197e16740 1 --2- 172.21.15.39:0/171876895 >> [v2:172.21.15.39:6840/4247190456,v1:172.21.15.39:6841/4247190456] conn(0x5620507f89f0 0x562050bfef80 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:16.891 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:16.870+0000 7fc197e16740 1 --2- 172.21.15.39:0/171876895 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56205083a070 0x562050bed750 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:16.891 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:16.870+0000 7fc197e16740 1 --2- 172.21.15.39:0/171876895 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562050819390 0x562050bed210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:16.891 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:16.870+0000 7fc197e16740 1 --2- 172.21.15.39:0/171876895 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5620508111b0 0x562050beccd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:16.891 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:16.870+0000 7fc197e16740 1 -- 172.21.15.39:0/171876895 >> 172.21.15.39:0/171876895 conn(0x56205082e460 msgr2=0x562050be4830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:48:16.891 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:16.870+0000 7fc16affd640 0 client.16050 ms_handle_reset on v2:172.21.15.39:6840/4247190456 2024-09-21T03:48:16.891 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:16.870+0000 7fc197e16740 1 -- 172.21.15.39:0/171876895 shutdown_connections 2024-09-21T03:48:16.892 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:16.870+0000 7fc197e16740 1 -- 172.21.15.39:0/171876895 wait complete. 2024-09-21T03:48:16.892 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:16.870+0000 7fc197e16740 20 client.16050 trim_cache size 0 max 16384 2024-09-21T03:48:16.900 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:16.883+0000 7fc197e16740 1 Processor -- start 2024-09-21T03:48:16.900 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:16.883+0000 7fc197e16740 1 -- start start 2024-09-21T03:48:16.900 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:16.884+0000 7fc197e16740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56205082e460 0x562050be5160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:48:16.900 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:16.884+0000 7fc197e16740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5620508111b0 0x562050bee900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:48:16.901 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:16.884+0000 7fc197e16740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562050819390 0x562050be9ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:48:16.901 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:16.884+0000 7fc197e16740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fc18c1af6b0 con 0x5620508111b0 2024-09-21T03:48:16.901 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:16.884+0000 7fc197e16740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fc18c1b11f0 con 0x562050819390 2024-09-21T03:48:16.901 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:16.884+0000 7fc197e16740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fc18c1afd80 con 0x56205082e460 2024-09-21T03:48:16.901 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:16.884+0000 7fc1827fc640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5620508111b0 0x562050bee900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:48:16.901 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:16.884+0000 7fc1827fc640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5620508111b0 0x562050bee900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:44320/0 (socket says 172.21.15.39:44320) 2024-09-21T03:48:16.901 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:16.884+0000 7fc1827fc640 1 -- 172.21.15.39:0/194830374 learned_addr learned my addr 172.21.15.39:0/194830374 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:48:16.902 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:16.884+0000 7fc18affd640 1 --2- 172.21.15.39:0/194830374 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56205082e460 0x562050be5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:48:16.902 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:16.884+0000 7fc18b7fe640 1 --2- 172.21.15.39:0/194830374 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562050819390 0x562050be9ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:48:16.902 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:16.884+0000 7fc1827fc640 1 -- 172.21.15.39:0/194830374 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56205082e460 msgr2=0x562050be5160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:48:16.902 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:16.884+0000 7fc1827fc640 1 --2- 172.21.15.39:0/194830374 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56205082e460 0x562050be5160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:16.902 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:16.884+0000 7fc1827fc640 1 -- 172.21.15.39:0/194830374 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562050819390 msgr2=0x562050be9ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:48:16.902 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:16.884+0000 7fc1827fc640 1 --2- 172.21.15.39:0/194830374 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562050819390 0x562050be9ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:16.903 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:16.884+0000 7fc1827fc640 1 -- 172.21.15.39:0/194830374 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562050be4500 con 0x5620508111b0 2024-09-21T03:48:16.903 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:16.884+0000 7fc18affd640 1 --2- 172.21.15.39:0/194830374 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56205082e460 0x562050be5160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:48:16.903 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:16.884+0000 7fc1827fc640 1 --2- 172.21.15.39:0/194830374 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5620508111b0 0x562050bee900 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7fc174005ab0 tx=0x7fc174032e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=36530bbeb304dd3c server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:48:16.903 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:16.884+0000 7fc1817fa640 1 -- 172.21.15.39:0/194830374 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc174035040 con 0x5620508111b0 2024-09-21T03:48:16.903 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:16.885+0000 7fc1817fa640 1 -- 172.21.15.39:0/194830374 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc174031dd0 con 0x5620508111b0 2024-09-21T03:48:16.903 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:16.885+0000 7fc1817fa640 1 -- 172.21.15.39:0/194830374 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1740053c0 con 0x5620508111b0 2024-09-21T03:48:16.903 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:16.885+0000 7fc197e16740 1 -- 172.21.15.39:0/194830374 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5620508111b0 msgr2=0x562050bee900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:48:16.904 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:16.885+0000 7fc197e16740 1 --2- 172.21.15.39:0/194830374 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5620508111b0 0x562050bee900 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7fc174005ab0 tx=0x7fc174032e50 comp rx=0 tx=0).stop 2024-09-21T03:48:16.904 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:16.885+0000 7fc197e16740 1 -- 172.21.15.39:0/194830374 shutdown_connections 2024-09-21T03:48:16.904 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:16.885+0000 7fc197e16740 1 --2- 172.21.15.39:0/194830374 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562050819390 0x562050be9ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:16.904 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:16.885+0000 7fc197e16740 1 --2- 172.21.15.39:0/194830374 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5620508111b0 0x562050bee900 unknown :-1 s=CLOSED pgs=614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:16.904 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:16.885+0000 7fc197e16740 1 --2- 172.21.15.39:0/194830374 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56205082e460 0x562050be5160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:16.904 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:16.885+0000 7fc197e16740 1 -- 172.21.15.39:0/194830374 >> 172.21.15.39:0/194830374 conn(0x562050bff2d0 msgr2=0x562050bee1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:48:16.904 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:16.886+0000 7fc197e16740 1 -- 172.21.15.39:0/194830374 shutdown_connections 2024-09-21T03:48:16.905 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:16.886+0000 7fc197e16740 1 -- 172.21.15.39:0/194830374 wait complete. 2024-09-21T03:48:16.905 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:16.886+0000 7fc197e16740 1 -- start start 2024-09-21T03:48:16.905 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:16.886+0000 7fc197e16740 1 Processor -- start 2024-09-21T03:48:16.905 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:16.886+0000 7fc197e16740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5620508111b0 0x562050bf5a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:48:16.905 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:16.886+0000 7fc197e16740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562050819390 0x562050bf2f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:48:16.905 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:16.886+0000 7fc197e16740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56205082e460 0x562050bf3620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:48:16.905 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:16.887+0000 7fc197e16740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fc18c1b0b20 con 0x5620508111b0 2024-09-21T03:48:16.906 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:16.887+0000 7fc197e16740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fc18c1b0450 con 0x56205082e460 2024-09-21T03:48:16.906 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:16.887+0000 7fc197e16740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fc18c1ad690 con 0x562050819390 2024-09-21T03:48:16.906 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:16.887+0000 7fc197e16740 10 client.0 _ll_register_callbacks cb 0x7fc184009d60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:48:16.906 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:16.887+0000 7fc18affd640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5620508111b0 0x562050bf5a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:48:16.906 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:16.887+0000 7fc18affd640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5620508111b0 0x562050bf5a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:44332/0 (socket says 172.21.15.39:44332) 2024-09-21T03:48:16.906 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:16.887+0000 7fc18affd640 1 -- 172.21.15.39:0/1640770937 learned_addr learned my addr 172.21.15.39:0/1640770937 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:48:16.907 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:16.887+0000 7fc1827fc640 1 --2- 172.21.15.39:0/1640770937 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562050819390 0x562050bf2f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:48:16.907 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:16.887+0000 7fc18b7fe640 1 --2- 172.21.15.39:0/1640770937 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56205082e460 0x562050bf3620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:48:16.907 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:16.887+0000 7fc18affd640 1 -- 172.21.15.39:0/1640770937 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562050819390 msgr2=0x562050bf2f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:48:16.907 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:16.887+0000 7fc18affd640 1 --2- 172.21.15.39:0/1640770937 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562050819390 0x562050bf2f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:16.907 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:16.887+0000 7fc18affd640 1 -- 172.21.15.39:0/1640770937 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56205082e460 msgr2=0x562050bf3620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:48:16.907 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:16.887+0000 7fc18affd640 1 --2- 172.21.15.39:0/1640770937 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56205082e460 0x562050bf3620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:16.907 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:16.887+0000 7fc18affd640 1 -- 172.21.15.39:0/1640770937 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562050c68950 con 0x5620508111b0 2024-09-21T03:48:16.908 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:16.887+0000 7fc18affd640 1 --2- 172.21.15.39:0/1640770937 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5620508111b0 0x562050bf5a80 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7fc17c0aa3a0 tx=0x7fc17c00de80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7b81700fefc45b8e server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:48:16.908 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:16.887+0000 7fc16affd640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-09-21T03:48:16.908 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:16.887+0000 7fc16affd640 1 -- 172.21.15.39:0/1640770937 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc17c092ce0 con 0x5620508111b0 2024-09-21T03:48:16.908 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:16.887+0000 7fc16affd640 1 -- 172.21.15.39:0/1640770937 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8484+0+0 (secure 0 0 0) 0x7fc17c0a93a0 con 0x5620508111b0 2024-09-21T03:48:16.908 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:16.887+0000 7fc197e16740 1 -- 172.21.15.39:0/1640770937 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc174031dd0 con 0x5620508111b0 2024-09-21T03:48:16.908 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:16.888+0000 7fc16affd640 1 -- 172.21.15.39:0/1640770937 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fc17c091240 con 0x5620508111b0 2024-09-21T03:48:16.909 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:16.888+0000 7fc197e16740 10 client.16056 fetch_fsmap learned FSMap version 29 2024-09-21T03:48:16.909 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:16.888+0000 7fc197e16740 1 -- 172.21.15.39:0/1640770937 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x562050c69a20 con 0x5620508111b0 2024-09-21T03:48:16.909 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:16.888+0000 7fc16affd640 1 -- 172.21.15.39:0/1640770937 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc174031dd0 con 0x5620508111b0 2024-09-21T03:48:16.909 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:16.888+0000 7fc16affd640 1 -- 172.21.15.39:0/1640770937 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fc17c005100 con 0x5620508111b0 2024-09-21T03:48:16.909 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:16.888+0000 7fc197e16740 10 client.16056 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T03:48:16.909 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:16.888+0000 7fc197e16740 10 client.16056 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.a 2024-09-21T03:48:16.910 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:16.888+0000 7fc197e16740 20 client.16056 populate_metadata read hostname 'smithi039' 2024-09-21T03:48:16.910 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:16.888+0000 7fc197e16740 1 --2- 172.21.15.39:0/1640770937 >> [v2:172.21.15.39:6840/4247190456,v1:172.21.15.39:6841/4247190456] conn(0x56205083a070 0x562050c6bdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:48:16.910 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:16.888+0000 7fc197e16740 4 client.16056 mds_command: new command op to 14868 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T03:48:16.910 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:16.888+0000 7fc197e16740 1 -- 172.21.15.39:0/1640770937 --> [v2:172.21.15.39:6840/4247190456,v1:172.21.15.39:6841/4247190456] -- command(tid 0: {"prefix": "damage ls"}) -- 0x562050b558a0 con 0x56205083a070 2024-09-21T03:48:16.910 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:16.888+0000 7fc1827fc640 1 --2- 172.21.15.39:0/1640770937 >> [v2:172.21.15.39:6840/4247190456,v1:172.21.15.39:6841/4247190456] conn(0x56205083a070 0x562050c6bdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:48:16.910 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:16.889+0000 7fc16affd640 10 client.16056 ms_handle_connect on v2:172.21.15.39:6840/4247190456 2024-09-21T03:48:16.911 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:16.889+0000 7fc1827fc640 1 --2- 172.21.15.39:0/1640770937 >> [v2:172.21.15.39:6840/4247190456,v1:172.21.15.39:6841/4247190456] conn(0x56205083a070 0x562050c6bdc0 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=679632a3cd560042 server_cookie=845f6339935b6e72 in_seq=0 out_seq=0 2024-09-21T03:48:16.911 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:16.889+0000 7fc16affd640 1 -- 172.21.15.39:0/1640770937 <== mds.4 v2:172.21.15.39:6840/4247190456 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x562050b558a0 con 0x56205083a070 2024-09-21T03:48:16.911 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:16.889+0000 7fc16affd640 10 client.16056 handle_command_reply: tid=0 2024-09-21T03:48:16.911 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:16.889+0000 7fc197e16740 1 client.16056 shutdown 2024-09-21T03:48:16.911 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:16.889+0000 7fc197e16740 1 -- 172.21.15.39:0/1640770937 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5620508111b0 msgr2=0x562050bf5a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:48:16.911 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:16.889+0000 7fc197e16740 1 --2- 172.21.15.39:0/1640770937 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5620508111b0 0x562050bf5a80 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7fc17c0aa3a0 tx=0x7fc17c00de80 comp rx=0 tx=0).stop 2024-09-21T03:48:16.912 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:16.889+0000 7fc197e16740 1 -- 172.21.15.39:0/1640770937 shutdown_connections 2024-09-21T03:48:16.912 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:16.889+0000 7fc197e16740 1 --2- 172.21.15.39:0/1640770937 >> [v2:172.21.15.39:6840/4247190456,v1:172.21.15.39:6841/4247190456] conn(0x56205083a070 0x562050c6bdc0 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:16.912 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:16.889+0000 7fc197e16740 1 --2- 172.21.15.39:0/1640770937 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56205082e460 0x562050bf3620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:16.912 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:16.889+0000 7fc16affd640 0 client.16056 ms_handle_reset on v2:172.21.15.39:6840/4247190456 2024-09-21T03:48:16.912 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:16.889+0000 7fc197e16740 1 --2- 172.21.15.39:0/1640770937 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562050819390 0x562050bf2f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:16.914 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:16.889+0000 7fc197e16740 1 --2- 172.21.15.39:0/1640770937 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5620508111b0 0x562050bf5a80 unknown :-1 s=CLOSED pgs=615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:16.914 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:16.889+0000 7fc197e16740 1 -- 172.21.15.39:0/1640770937 >> 172.21.15.39:0/1640770937 conn(0x562050bff2d0 msgr2=0x562050beb150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:48:16.914 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:16.889+0000 7fc197e16740 1 -- 172.21.15.39:0/1640770937 shutdown_connections 2024-09-21T03:48:16.914 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:16.889+0000 7fc197e16740 1 -- 172.21.15.39:0/1640770937 wait complete. 2024-09-21T03:48:16.914 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:16.889+0000 7fc197e16740 20 client.16056 trim_cache size 0 max 16384 2024-09-21T03:48:16.915 INFO:teuthology.orchestra.run.smithi039.stdout:[] 2024-09-21T03:48:16.916 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:16.890+0000 7fc1918a2640 1 -- 172.21.15.39:0/1558573262 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7fc1780b6680 msgr2=0x7fc1780b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:48:16.916 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:16.890+0000 7fc1918a2640 1 --2- 172.21.15.39:0/1558573262 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7fc1780b6680 0x7fc1780b8b90 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fc184005fd0 tx=0x7fc184009800 comp rx=0 tx=0).stop 2024-09-21T03:48:16.916 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:16.890+0000 7fc1918a2640 1 -- 172.21.15.39:0/1558573262 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc18c102ae0 msgr2=0x7fc18c19a460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:48:16.916 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:16.890+0000 7fc1918a2640 1 --2- 172.21.15.39:0/1558573262 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc18c102ae0 0x7fc18c19a460 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7fc17c00d770 tx=0x7fc17c00dc30 comp rx=0 tx=0).stop 2024-09-21T03:48:16.916 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:16.890+0000 7fc1918a2640 1 -- 172.21.15.39:0/1558573262 shutdown_connections 2024-09-21T03:48:16.916 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:16.890+0000 7fc1918a2640 1 --2- 172.21.15.39:0/1558573262 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7fc1780b6680 0x7fc1780b8b90 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:16.917 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:16.890+0000 7fc1918a2640 1 --2- 172.21.15.39:0/1558573262 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc18c108d20 0x7fc18c19fca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:16.917 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:16.890+0000 7fc1918a2640 1 --2- 172.21.15.39:0/1558573262 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc18c1083d0 0x7fc18c19aa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:16.917 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:16.890+0000 7fc1918a2640 1 --2- 172.21.15.39:0/1558573262 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc18c102ae0 0x7fc18c19a460 unknown :-1 s=CLOSED pgs=611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:16.917 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:16.890+0000 7fc1918a2640 1 -- 172.21.15.39:0/1558573262 >> 172.21.15.39:0/1558573262 conn(0x7fc18c079760 msgr2=0x7fc18c104450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:48:16.917 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:16.890+0000 7fc1918a2640 1 -- 172.21.15.39:0/1558573262 shutdown_connections 2024-09-21T03:48:16.917 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:16.890+0000 7fc1918a2640 1 -- 172.21.15.39:0/1558573262 wait complete. 2024-09-21T03:48:17.084 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:48:16 smithi039 ceph-mon[31211]: pgmap v392: 97 pgs: 97 active+clean; 9.0 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 503 KiB/s rd, 766 MiB/s wr, 429 op/s 2024-09-21T03:48:17.468 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:48:16 smithi161 ceph-mon[34533]: pgmap v392: 97 pgs: 97 active+clean; 9.0 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 503 KiB/s rd, 766 MiB/s wr, 429 op/s 2024-09-21T03:48:17.474 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:48:16 smithi150 ceph-mon[34289]: pgmap v392: 97 pgs: 97 active+clean; 9.0 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 503 KiB/s rd, 766 MiB/s wr, 429 op/s 2024-09-21T03:48:17.532 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 51116d5c-77ca-11ef-baf5-efdc52797490 -- ceph tell mds.1:3 damage ls 2024-09-21T03:48:17.814 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/51116d5c-77ca-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:48:18.573 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:18.570+0000 7f6949112640 1 -- 172.21.15.39:0/4272129304 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6944102ae0 msgr2=0x7f6944102ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:48:18.573 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:18.570+0000 7f6949112640 1 --2- 172.21.15.39:0/4272129304 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6944102ae0 0x7f6944102ef0 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f6934009900 tx=0x7f693402a410 comp rx=0 tx=0).stop 2024-09-21T03:48:18.573 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:18.570+0000 7f6949112640 1 -- 172.21.15.39:0/4272129304 shutdown_connections 2024-09-21T03:48:18.573 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:18.570+0000 7f6949112640 1 --2- 172.21.15.39:0/4272129304 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6944108d20 0x7f694410f550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:18.573 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:18.570+0000 7f6949112640 1 --2- 172.21.15.39:0/4272129304 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f69441083d0 0x7f69441087e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:18.574 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:18.570+0000 7f6949112640 1 --2- 172.21.15.39:0/4272129304 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6944102ae0 0x7f6944102ef0 unknown :-1 s=CLOSED pgs=616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:18.574 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:18.570+0000 7f6949112640 1 -- 172.21.15.39:0/4272129304 >> 172.21.15.39:0/4272129304 conn(0x7f6944079790 msgr2=0x7f6944079ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:48:18.574 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:18.571+0000 7f6949112640 1 -- 172.21.15.39:0/4272129304 shutdown_connections 2024-09-21T03:48:18.574 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:18.571+0000 7f6949112640 1 -- 172.21.15.39:0/4272129304 wait complete. 2024-09-21T03:48:18.574 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:18.572+0000 7f6949112640 1 Processor -- start 2024-09-21T03:48:18.574 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:18.572+0000 7f6949112640 1 -- start start 2024-09-21T03:48:18.575 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:18.573+0000 7f6949112640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6944102ae0 0x7f69441a0c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:48:18.576 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:18.573+0000 7f6949112640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f69441083d0 0x7f69441a11a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:48:18.576 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:18.573+0000 7f6942d9d640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6944102ae0 0x7f69441a0c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:48:18.576 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:18.573+0000 7f6949112640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6944108d20 0x7f69441a6440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:48:18.576 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:18.573+0000 7f6949112640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f6944112a40 con 0x7f6944102ae0 2024-09-21T03:48:18.576 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:18.573+0000 7f6949112640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f69441128c0 con 0x7f6944108d20 2024-09-21T03:48:18.576 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:18.573+0000 7f6949112640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f6944112bc0 con 0x7f69441083d0 2024-09-21T03:48:18.576 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:18.574+0000 7f6942d9d640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6944102ae0 0x7f69441a0c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:48664/0 (socket says 172.21.15.39:48664) 2024-09-21T03:48:18.576 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:18.574+0000 7f6942d9d640 1 -- 172.21.15.39:0/2508571639 learned_addr learned my addr 172.21.15.39:0/2508571639 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:48:18.577 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:18.574+0000 7f694259c640 1 --2- 172.21.15.39:0/2508571639 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f69441083d0 0x7f69441a11a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:48:18.577 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:18.574+0000 7f6942d9d640 1 -- 172.21.15.39:0/2508571639 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f69441083d0 msgr2=0x7f69441a11a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:48:18.577 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:18.574+0000 7f6942d9d640 1 --2- 172.21.15.39:0/2508571639 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f69441083d0 0x7f69441a11a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:18.577 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:18.574+0000 7f6942d9d640 1 -- 172.21.15.39:0/2508571639 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6944108d20 msgr2=0x7f69441a6440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:48:18.577 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:18.574+0000 7f6942d9d640 1 --2- 172.21.15.39:0/2508571639 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6944108d20 0x7f69441a6440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:18.577 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:18.574+0000 7f6942d9d640 1 -- 172.21.15.39:0/2508571639 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f69441a6b70 con 0x7f6944102ae0 2024-09-21T03:48:18.577 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:18.575+0000 7f6942d9d640 1 --2- 172.21.15.39:0/2508571639 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6944102ae0 0x7f69441a0c00 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f6934005940 tx=0x7f6934004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:48:18.578 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:18.575+0000 7f692bfff640 1 -- 172.21.15.39:0/2508571639 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f693403f030 con 0x7f6944102ae0 2024-09-21T03:48:18.578 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:18.575+0000 7f692bfff640 1 -- 172.21.15.39:0/2508571639 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6934003aa0 con 0x7f6944102ae0 2024-09-21T03:48:18.578 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:18.575+0000 7f692bfff640 1 -- 172.21.15.39:0/2508571639 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6934035070 con 0x7f6944102ae0 2024-09-21T03:48:18.578 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:18.575+0000 7f6949112640 1 -- 172.21.15.39:0/2508571639 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f69441a6e00 con 0x7f6944102ae0 2024-09-21T03:48:18.578 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:18.576+0000 7f6949112640 1 -- 172.21.15.39:0/2508571639 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f69441ad5c0 con 0x7f6944102ae0 2024-09-21T03:48:18.582 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:18.580+0000 7f692bfff640 1 -- 172.21.15.39:0/2508571639 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 27) ==== 154643+0+0 (secure 0 0 0) 0x7f6934042030 con 0x7f6944102ae0 2024-09-21T03:48:18.583 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:18.580+0000 7f694f686740 1 Processor -- start 2024-09-21T03:48:18.583 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:18.580+0000 7f694f686740 1 -- start start 2024-09-21T03:48:18.583 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:18.580+0000 7f694f686740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562209dcf070 0x56220a1766a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:48:18.583 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:18.580+0000 7f694f686740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562209da61b0 0x56220a176be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:48:18.583 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:18.580+0000 7f694f686740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562209dae390 0x56220a177120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:48:18.583 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:18.580+0000 7f694f686740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x56220a0ecea0 con 0x562209dae390 2024-09-21T03:48:18.583 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:18.580+0000 7f694f686740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x56220a0dc2d0 con 0x562209da61b0 2024-09-21T03:48:18.583 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:18.580+0000 7f694f686740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x56220a0e00e0 con 0x562209dcf070 2024-09-21T03:48:18.583 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:18.581+0000 7f694259c640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562209dae390 0x56220a177120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:48:18.583 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:18.581+0000 7f694259c640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562209dae390 0x56220a177120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:48678/0 (socket says 172.21.15.39:48678) 2024-09-21T03:48:18.584 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:18.581+0000 7f694259c640 1 -- 172.21.15.39:0/1676662656 learned_addr learned my addr 172.21.15.39:0/1676662656 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:48:18.584 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:18.581+0000 7f6942d9d640 1 --2- 172.21.15.39:0/1676662656 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562209da61b0 0x56220a176be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:48:18.584 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:18.581+0000 7f694259c640 1 -- 172.21.15.39:0/1676662656 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562209dcf070 msgr2=0x56220a1766a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:48:18.585 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:18.581+0000 7f694259c640 1 --2- 172.21.15.39:0/1676662656 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562209dcf070 0x56220a1766a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:18.585 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:18.581+0000 7f694259c640 1 -- 172.21.15.39:0/1676662656 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562209da61b0 msgr2=0x56220a176be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:48:18.585 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:18.581+0000 7f694259c640 1 --2- 172.21.15.39:0/1676662656 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562209da61b0 0x56220a176be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:18.585 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:18.581+0000 7f694259c640 1 -- 172.21.15.39:0/1676662656 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562209e3a650 con 0x562209dae390 2024-09-21T03:48:18.585 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:18.581+0000 7f692bfff640 1 --2- 172.21.15.39:0/2508571639 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f692c0b6700 0x7f692c0b8c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:48:18.585 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:18.581+0000 7f694259c640 1 --2- 172.21.15.39:0/1676662656 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562209dae390 0x56220a177120 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f693800b790 tx=0x7f693800bc50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3335d9d45c096cc1 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:48:18.585 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:18.582+0000 7f692bfff640 1 -- 172.21.15.39:0/2508571639 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8484+0+0 (secure 0 0 0) 0x7f6934107c90 con 0x7f6944102ae0 2024-09-21T03:48:18.585 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:18.582+0000 7f694359e640 1 --2- 172.21.15.39:0/2508571639 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f692c0b6700 0x7f692c0b8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:48:18.586 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:18.582+0000 7f69297fa640 1 -- 172.21.15.39:0/1676662656 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f693800d950 con 0x562209dae390 2024-09-21T03:48:18.586 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:18.582+0000 7f69297fa640 1 -- 172.21.15.39:0/1676662656 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6938018070 con 0x562209dae390 2024-09-21T03:48:18.587 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:18.582+0000 7f69297fa640 1 -- 172.21.15.39:0/1676662656 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6938013db0 con 0x562209dae390 2024-09-21T03:48:18.587 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:18.583+0000 7f694f686740 1 -- 172.21.15.39:0/1676662656 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562209dae390 msgr2=0x56220a177120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:48:18.587 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:18.583+0000 7f694359e640 1 --2- 172.21.15.39:0/2508571639 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f692c0b6700 0x7f692c0b8c10 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f693c009270 tx=0x7f693c0091e0 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:48:18.587 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:18.583+0000 7f694f686740 1 --2- 172.21.15.39:0/1676662656 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562209dae390 0x56220a177120 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f693800b790 tx=0x7f693800bc50 comp rx=0 tx=0).stop 2024-09-21T03:48:18.587 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:18.583+0000 7f694f686740 1 -- 172.21.15.39:0/1676662656 shutdown_connections 2024-09-21T03:48:18.587 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:18.583+0000 7f694f686740 1 --2- 172.21.15.39:0/1676662656 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562209dae390 0x56220a177120 unknown :-1 s=CLOSED pgs=618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:18.588 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:18.583+0000 7f694f686740 1 --2- 172.21.15.39:0/1676662656 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562209da61b0 0x56220a176be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:18.588 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:18.583+0000 7f694f686740 1 --2- 172.21.15.39:0/1676662656 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562209dcf070 0x56220a1766a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:18.588 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:18.583+0000 7f694f686740 1 -- 172.21.15.39:0/1676662656 >> 172.21.15.39:0/1676662656 conn(0x562209dc3460 msgr2=0x56220a174590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:48:18.588 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:18.583+0000 7f694f686740 1 -- 172.21.15.39:0/1676662656 shutdown_connections 2024-09-21T03:48:18.588 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:18.583+0000 7f694f686740 1 -- 172.21.15.39:0/1676662656 wait complete. 2024-09-21T03:48:18.589 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:18.584+0000 7f694f686740 1 -- start start 2024-09-21T03:48:18.589 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:18.584+0000 7f694f686740 1 Processor -- start 2024-09-21T03:48:18.589 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:18.584+0000 7f694f686740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562209da61b0 0x56220a181d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:48:18.589 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:18.584+0000 7f694f686740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562209dae390 0x56220a1822a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:48:18.589 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:18.584+0000 7f694f686740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562209dcf070 0x56220a19f180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:48:18.590 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:18.584+0000 7f694f686740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x562209f3d700 con 0x562209dcf070 2024-09-21T03:48:18.590 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:18.584+0000 7f694f686740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x56220a077160 con 0x562209dae390 2024-09-21T03:48:18.590 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:18.584+0000 7f694f686740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x56220a184e80 con 0x562209da61b0 2024-09-21T03:48:18.590 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:18.584+0000 7f694f686740 10 client.0 _ll_register_callbacks cb 0x56220a0909e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:48:18.591 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:18.584+0000 7f694359e640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562209dcf070 0x56220a19f180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:48:18.591 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:18.584+0000 7f6942d9d640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562209da61b0 0x56220a181d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:48:18.591 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:18.584+0000 7f6942d9d640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562209da61b0 0x56220a181d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.39:54862/0 (socket says 172.21.15.39:54862) 2024-09-21T03:48:18.594 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:18.585+0000 7f6942d9d640 1 -- 172.21.15.39:0/1826496888 learned_addr learned my addr 172.21.15.39:0/1826496888 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:48:18.594 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:18.585+0000 7f694359e640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562209dcf070 0x56220a19f180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:48694/0 (socket says 172.21.15.39:48694) 2024-09-21T03:48:18.594 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:18.585+0000 7f694359e640 1 -- 172.21.15.39:0/1826496888 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562209da61b0 msgr2=0x56220a181d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:48:18.595 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:18.585+0000 7f694359e640 1 --2- 172.21.15.39:0/1826496888 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562209da61b0 0x56220a181d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:18.595 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:18.585+0000 7f694359e640 1 -- 172.21.15.39:0/1826496888 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562209dae390 msgr2=0x56220a1822a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:48:18.595 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:18.585+0000 7f694359e640 1 --2- 172.21.15.39:0/1826496888 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562209dae390 0x56220a1822a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:18.595 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:18.585+0000 7f694359e640 1 -- 172.21.15.39:0/1826496888 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56220a0a4c90 con 0x562209dcf070 2024-09-21T03:48:18.595 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:18.585+0000 7f694359e640 1 --2- 172.21.15.39:0/1826496888 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562209dcf070 0x56220a19f180 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f693c005c40 tx=0x7f693c009500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7441450c0134d8f6 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:48:18.595 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:18.585+0000 7f6926ff5640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-09-21T03:48:18.595 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:18.585+0000 7f6926ff5640 1 -- 172.21.15.39:0/1826496888 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f693c020070 con 0x562209dcf070 2024-09-21T03:48:18.595 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:18.586+0000 7f694f686740 1 -- 172.21.15.39:0/1826496888 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6938018070 con 0x562209dcf070 2024-09-21T03:48:18.595 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:18.586+0000 7f6926ff5640 1 -- 172.21.15.39:0/1826496888 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8484+0+0 (secure 0 0 0) 0x7f693c00fd20 con 0x562209dcf070 2024-09-21T03:48:18.595 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:18.586+0000 7f6926ff5640 1 -- 172.21.15.39:0/1826496888 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f693c025460 con 0x562209dcf070 2024-09-21T03:48:18.595 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:18.586+0000 7f694f686740 10 client.16068 fetch_fsmap learned FSMap version 29 2024-09-21T03:48:18.595 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:18.586+0000 7f694f686740 1 -- 172.21.15.39:0/1826496888 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x56220a1c3c60 con 0x562209dcf070 2024-09-21T03:48:18.595 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:18.586+0000 7f6926ff5640 1 -- 172.21.15.39:0/1826496888 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f693c01c8d0 con 0x562209dcf070 2024-09-21T03:48:18.596 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:18.586+0000 7f6926ff5640 1 -- 172.21.15.39:0/1826496888 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f693c02e550 con 0x562209dcf070 2024-09-21T03:48:18.596 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:18.586+0000 7f694f686740 10 client.16068 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T03:48:18.596 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:18.586+0000 7f694f686740 10 client.16068 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-21T03:48:18.596 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:18.586+0000 7f694f686740 20 client.16068 populate_metadata read hostname 'smithi039' 2024-09-21T03:48:18.596 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:18.586+0000 7f694f686740 1 --2- 172.21.15.39:0/1826496888 >> [v2:172.21.15.161:6836/3451544597,v1:172.21.15.161:6837/3451544597] conn(0x562209d8d9f0 0x56220a1c7520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:48:18.596 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:18.586+0000 7f694f686740 4 client.16068 mds_command: new command op to 24646 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:48:18.596 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:18.586+0000 7f694f686740 1 -- 172.21.15.39:0/1826496888 --> [v2:172.21.15.161:6836/3451544597,v1:172.21.15.161:6837/3451544597] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f6938018070 con 0x562209d8d9f0 2024-09-21T03:48:18.596 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:18.587+0000 7f6942d9d640 1 --2- 172.21.15.39:0/1826496888 >> [v2:172.21.15.161:6836/3451544597,v1:172.21.15.161:6837/3451544597] conn(0x562209d8d9f0 0x56220a1c7520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:48:18.596 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:18.587+0000 7f6942d9d640 1 --2- 172.21.15.39:0/1826496888 >> [v2:172.21.15.161:6836/3451544597,v1:172.21.15.161:6837/3451544597] conn(0x562209d8d9f0 0x56220a1c7520 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=49d4021054581af3 server_cookie=7a02d5dcb843ae48 in_seq=0 out_seq=0 2024-09-21T03:48:18.596 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:18.587+0000 7f6926ff5640 10 client.16068 ms_handle_connect on v2:172.21.15.161:6836/3451544597 2024-09-21T03:48:18.596 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:18.589+0000 7f6926ff5640 1 -- 172.21.15.39:0/1826496888 <== mds.3 v2:172.21.15.161:6836/3451544597 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x7f6938018070 con 0x562209d8d9f0 2024-09-21T03:48:18.596 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:18.589+0000 7f6926ff5640 10 client.16068 handle_command_reply: tid=0 2024-09-21T03:48:18.596 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:18.589+0000 7f694f686740 1 client.16068 shutdown 2024-09-21T03:48:18.596 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:18.589+0000 7f694f686740 1 -- 172.21.15.39:0/1826496888 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562209dcf070 msgr2=0x56220a19f180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:48:18.597 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:18.589+0000 7f694f686740 1 --2- 172.21.15.39:0/1826496888 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562209dcf070 0x56220a19f180 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f693c005c40 tx=0x7f693c009500 comp rx=0 tx=0).stop 2024-09-21T03:48:18.597 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:18.589+0000 7f694f686740 1 -- 172.21.15.39:0/1826496888 shutdown_connections 2024-09-21T03:48:18.597 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:18.589+0000 7f694f686740 1 --2- 172.21.15.39:0/1826496888 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562209dcf070 0x56220a19f180 unknown :-1 s=CLOSED pgs=619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:18.597 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:18.589+0000 7f694f686740 1 --2- 172.21.15.39:0/1826496888 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562209dae390 0x56220a1822a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:18.597 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:18.589+0000 7f694f686740 1 --2- 172.21.15.39:0/1826496888 >> [v2:172.21.15.161:6836/3451544597,v1:172.21.15.161:6837/3451544597] conn(0x562209d8d9f0 0x56220a1c7520 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:18.597 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:18.589+0000 7f694f686740 1 --2- 172.21.15.39:0/1826496888 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562209da61b0 0x56220a181d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:18.597 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:18.589+0000 7f6926ff5640 0 client.16068 ms_handle_reset on v2:172.21.15.161:6836/3451544597 2024-09-21T03:48:18.597 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:18.589+0000 7f694f686740 1 -- 172.21.15.39:0/1826496888 >> 172.21.15.39:0/1826496888 conn(0x562209dc3460 msgr2=0x56220a179830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:48:18.597 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:18.589+0000 7f694f686740 1 -- 172.21.15.39:0/1826496888 shutdown_connections 2024-09-21T03:48:18.597 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:18.589+0000 7f694f686740 1 -- 172.21.15.39:0/1826496888 wait complete. 2024-09-21T03:48:18.597 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:18.589+0000 7f694f686740 20 client.16068 trim_cache size 0 max 16384 2024-09-21T03:48:18.607 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:18.603+0000 7f694f686740 1 Processor -- start 2024-09-21T03:48:18.607 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:18.603+0000 7f694f686740 1 -- start start 2024-09-21T03:48:18.607 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:18.603+0000 7f694f686740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562209da61b0 0x56220a182220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:48:18.607 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:18.603+0000 7f694f686740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562209dae390 0x56220a19f100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:48:18.607 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:18.603+0000 7f694f686740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562209dcf070 0x56220a173bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:48:18.607 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:18.603+0000 7f694f686740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f6944075420 con 0x562209da61b0 2024-09-21T03:48:18.607 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:18.603+0000 7f694f686740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f6944199880 con 0x562209dcf070 2024-09-21T03:48:18.607 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:18.603+0000 7f694f686740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f6944198410 con 0x562209dae390 2024-09-21T03:48:18.608 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:18.603+0000 7f6942d9d640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562209da61b0 0x56220a182220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:48:18.608 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:18.603+0000 7f6942d9d640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562209da61b0 0x56220a182220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:48704/0 (socket says 172.21.15.39:48704) 2024-09-21T03:48:18.608 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:18.603+0000 7f6942d9d640 1 -- 172.21.15.39:0/3747396814 learned_addr learned my addr 172.21.15.39:0/3747396814 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:48:18.608 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:18.603+0000 7f694259c640 1 --2- 172.21.15.39:0/3747396814 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562209dae390 0x56220a19f100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:48:18.608 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:18.604+0000 7f694359e640 1 --2- 172.21.15.39:0/3747396814 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562209dcf070 0x56220a173bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:48:18.608 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:18.604+0000 7f6942d9d640 1 -- 172.21.15.39:0/3747396814 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562209dae390 msgr2=0x56220a19f100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:48:18.608 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:18.604+0000 7f6942d9d640 1 --2- 172.21.15.39:0/3747396814 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562209dae390 0x56220a19f100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:18.608 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:18.604+0000 7f6942d9d640 1 -- 172.21.15.39:0/3747396814 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562209dcf070 msgr2=0x56220a173bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:48:18.608 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:18.604+0000 7f6942d9d640 1 --2- 172.21.15.39:0/3747396814 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562209dcf070 0x56220a173bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:18.608 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:18.604+0000 7f6942d9d640 1 -- 172.21.15.39:0/3747396814 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56220a184930 con 0x562209da61b0 2024-09-21T03:48:18.608 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:18.604+0000 7f6942d9d640 1 --2- 172.21.15.39:0/3747396814 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562209da61b0 0x56220a182220 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f69340cd800 tx=0x7f69340f1040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8bfb46a00bc8a561 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:48:18.608 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:18.604+0000 7f6928ff9640 1 -- 172.21.15.39:0/3747396814 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6934049040 con 0x562209da61b0 2024-09-21T03:48:18.608 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:18.604+0000 7f6928ff9640 1 -- 172.21.15.39:0/3747396814 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f69340cd610 con 0x562209da61b0 2024-09-21T03:48:18.608 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:18.604+0000 7f6928ff9640 1 -- 172.21.15.39:0/3747396814 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6934105680 con 0x562209da61b0 2024-09-21T03:48:18.611 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:18.605+0000 7f694f686740 1 -- 172.21.15.39:0/3747396814 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562209da61b0 msgr2=0x56220a182220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:48:18.611 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:18.605+0000 7f694f686740 1 --2- 172.21.15.39:0/3747396814 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562209da61b0 0x56220a182220 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f69340cd800 tx=0x7f69340f1040 comp rx=0 tx=0).stop 2024-09-21T03:48:18.611 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:18.605+0000 7f694f686740 1 -- 172.21.15.39:0/3747396814 shutdown_connections 2024-09-21T03:48:18.611 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:18.605+0000 7f694f686740 1 --2- 172.21.15.39:0/3747396814 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562209dcf070 0x56220a173bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:18.611 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:18.605+0000 7f694f686740 1 --2- 172.21.15.39:0/3747396814 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562209dae390 0x56220a19f100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:18.611 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:18.605+0000 7f694f686740 1 --2- 172.21.15.39:0/3747396814 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562209da61b0 0x56220a182220 unknown :-1 s=CLOSED pgs=620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:18.611 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:18.605+0000 7f694f686740 1 -- 172.21.15.39:0/3747396814 >> 172.21.15.39:0/3747396814 conn(0x562209dc3460 msgr2=0x56220a181b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:48:18.611 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:18.606+0000 7f694f686740 1 -- 172.21.15.39:0/3747396814 shutdown_connections 2024-09-21T03:48:18.611 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:18.606+0000 7f694f686740 1 -- 172.21.15.39:0/3747396814 wait complete. 2024-09-21T03:48:18.612 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:18.606+0000 7f694f686740 1 -- start start 2024-09-21T03:48:18.612 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:18.606+0000 7f694f686740 1 Processor -- start 2024-09-21T03:48:18.612 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:18.606+0000 7f694f686740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562209da61b0 0x56220a19b740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:48:18.612 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:18.606+0000 7f694f686740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562209dae390 0x56220a196bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:48:18.612 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:18.606+0000 7f694f686740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562209dcf070 0x56220a197130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:48:18.612 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:18.607+0000 7f694f686740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f69441991b0 con 0x562209da61b0 2024-09-21T03:48:18.612 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:18.607+0000 7f694f686740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f6944198ae0 con 0x562209dae390 2024-09-21T03:48:18.612 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:18.607+0000 7f694f686740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f6944102ef0 con 0x562209dcf070 2024-09-21T03:48:18.612 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:18.607+0000 7f6942d9d640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562209da61b0 0x56220a19b740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:48:18.612 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:18.607+0000 7f694259c640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562209dae390 0x56220a196bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:48:18.612 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:18.607+0000 7f6942d9d640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562209da61b0 0x56220a19b740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:48716/0 (socket says 172.21.15.39:48716) 2024-09-21T03:48:18.612 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:18.607+0000 7f6942d9d640 1 -- 172.21.15.39:0/3128103053 learned_addr learned my addr 172.21.15.39:0/3128103053 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:48:18.612 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:18.607+0000 7f694f686740 10 client.0 _ll_register_callbacks cb 0x7f693c014050 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:48:18.612 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:18.607+0000 7f694359e640 1 --2- 172.21.15.39:0/3128103053 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562209dcf070 0x56220a197130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:48:18.613 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:18.607+0000 7f6942d9d640 1 -- 172.21.15.39:0/3128103053 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562209dcf070 msgr2=0x56220a197130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:48:18.613 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:18.607+0000 7f6942d9d640 1 --2- 172.21.15.39:0/3128103053 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562209dcf070 0x56220a197130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:18.613 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:18.607+0000 7f6942d9d640 1 -- 172.21.15.39:0/3128103053 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562209dae390 msgr2=0x56220a196bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:48:18.613 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:18.607+0000 7f6942d9d640 1 --2- 172.21.15.39:0/3128103053 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562209dae390 0x56220a196bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:18.613 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:18.607+0000 7f6942d9d640 1 -- 172.21.15.39:0/3128103053 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56220a1976f0 con 0x562209da61b0 2024-09-21T03:48:18.613 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:18.607+0000 7f6942d9d640 1 --2- 172.21.15.39:0/3128103053 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562209da61b0 0x56220a19b740 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f69340cda90 tx=0x7f6934009640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fef1e132782bc7be server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:48:18.613 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:18.607+0000 7f6926ff5640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-09-21T03:48:18.613 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:18.607+0000 7f6926ff5640 1 -- 172.21.15.39:0/3128103053 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6934049050 con 0x562209da61b0 2024-09-21T03:48:18.613 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:18.608+0000 7f6926ff5640 1 -- 172.21.15.39:0/3128103053 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8484+0+0 (secure 0 0 0) 0x7f69340c9d70 con 0x562209da61b0 2024-09-21T03:48:18.613 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:18.608+0000 7f694f686740 1 -- 172.21.15.39:0/3128103053 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f69340cd610 con 0x562209da61b0 2024-09-21T03:48:18.613 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:18.608+0000 7f6926ff5640 1 -- 172.21.15.39:0/3128103053 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f69340d2040 con 0x562209da61b0 2024-09-21T03:48:18.613 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:18.608+0000 7f694f686740 10 client.16074 fetch_fsmap learned FSMap version 29 2024-09-21T03:48:18.613 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:18.608+0000 7f694f686740 1 -- 172.21.15.39:0/3128103053 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x56220a231c80 con 0x562209da61b0 2024-09-21T03:48:18.614 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:18.608+0000 7f6926ff5640 1 -- 172.21.15.39:0/3128103053 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f69340cd610 con 0x562209da61b0 2024-09-21T03:48:18.615 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:18.609+0000 7f6926ff5640 1 -- 172.21.15.39:0/3128103053 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f69340d21e0 con 0x562209da61b0 2024-09-21T03:48:18.616 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:18.610+0000 7f694f686740 10 client.16074 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T03:48:18.616 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:18.610+0000 7f694f686740 10 client.16074 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-21T03:48:18.616 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:18.610+0000 7f694f686740 20 client.16074 populate_metadata read hostname 'smithi039' 2024-09-21T03:48:18.616 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:18.610+0000 7f694f686740 1 --2- 172.21.15.39:0/3128103053 >> [v2:172.21.15.161:6836/3451544597,v1:172.21.15.161:6837/3451544597] conn(0x562209d62da0 0x56220a234260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:48:18.616 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:18.610+0000 7f694f686740 4 client.16074 mds_command: new command op to 24646 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T03:48:18.616 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:18.610+0000 7f694f686740 1 -- 172.21.15.39:0/3128103053 --> [v2:172.21.15.161:6836/3451544597,v1:172.21.15.161:6837/3451544597] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56220a0eedf0 con 0x562209d62da0 2024-09-21T03:48:18.616 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:18.610+0000 7f694259c640 1 --2- 172.21.15.39:0/3128103053 >> [v2:172.21.15.161:6836/3451544597,v1:172.21.15.161:6837/3451544597] conn(0x562209d62da0 0x56220a234260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:48:18.616 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:18.611+0000 7f6926ff5640 10 client.16074 ms_handle_connect on v2:172.21.15.161:6836/3451544597 2024-09-21T03:48:18.616 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:18.611+0000 7f694259c640 1 --2- 172.21.15.39:0/3128103053 >> [v2:172.21.15.161:6836/3451544597,v1:172.21.15.161:6837/3451544597] conn(0x562209d62da0 0x56220a234260 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=1dc9837fba8ae335 server_cookie=630f29a964b533b4 in_seq=0 out_seq=0 2024-09-21T03:48:18.616 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:18.612+0000 7f6926ff5640 1 -- 172.21.15.39:0/3128103053 <== mds.3 v2:172.21.15.161:6836/3451544597 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7f6938018040 con 0x562209d62da0 2024-09-21T03:48:18.616 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:18.612+0000 7f6926ff5640 10 client.16074 handle_command_reply: tid=0 2024-09-21T03:48:18.616 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:18.612+0000 7f694f686740 1 client.16074 shutdown 2024-09-21T03:48:18.617 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:18.613+0000 7f694f686740 1 -- 172.21.15.39:0/3128103053 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562209da61b0 msgr2=0x56220a19b740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:48:18.617 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:18.613+0000 7f694f686740 1 --2- 172.21.15.39:0/3128103053 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562209da61b0 0x56220a19b740 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f69340cda90 tx=0x7f6934009640 comp rx=0 tx=0).stop 2024-09-21T03:48:18.617 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:18.613+0000 7f694f686740 1 -- 172.21.15.39:0/3128103053 shutdown_connections 2024-09-21T03:48:18.617 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:18.613+0000 7f694f686740 1 --2- 172.21.15.39:0/3128103053 >> [v2:172.21.15.161:6836/3451544597,v1:172.21.15.161:6837/3451544597] conn(0x562209d62da0 0x56220a234260 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:18.617 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:18.613+0000 7f694f686740 1 --2- 172.21.15.39:0/3128103053 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562209dcf070 0x56220a197130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:18.617 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:18.613+0000 7f694f686740 1 --2- 172.21.15.39:0/3128103053 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562209dae390 0x56220a196bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:18.617 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:18.613+0000 7f694f686740 1 --2- 172.21.15.39:0/3128103053 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562209da61b0 0x56220a19b740 secure :-1 s=CLOSED pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f69340cda90 tx=0x7f6934009640 comp rx=0 tx=0).stop 2024-09-21T03:48:18.618 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:18.613+0000 7f694f686740 1 -- 172.21.15.39:0/3128103053 >> 172.21.15.39:0/3128103053 conn(0x562209dc3460 msgr2=0x56220a1801b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:48:18.618 INFO:teuthology.orchestra.run.smithi039.stdout:[] 2024-09-21T03:48:18.619 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:18.615+0000 7f6926ff5640 0 client.16074 ms_handle_reset on v2:172.21.15.161:6836/3451544597 2024-09-21T03:48:18.619 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:18.616+0000 7f694f686740 1 -- 172.21.15.39:0/3128103053 shutdown_connections 2024-09-21T03:48:18.619 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:18.616+0000 7f694f686740 1 -- 172.21.15.39:0/3128103053 wait complete. 2024-09-21T03:48:18.619 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:18.616+0000 7f694f686740 20 client.16074 trim_cache size 0 max 16384 2024-09-21T03:48:18.619 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:18.617+0000 7f6949112640 1 -- 172.21.15.39:0/2508571639 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f692c0b6700 msgr2=0x7f692c0b8c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:48:18.619 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:18.617+0000 7f6949112640 1 --2- 172.21.15.39:0/2508571639 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f692c0b6700 0x7f692c0b8c10 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f693c009270 tx=0x7f693c0091e0 comp rx=0 tx=0).stop 2024-09-21T03:48:18.619 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:18.617+0000 7f6949112640 1 -- 172.21.15.39:0/2508571639 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6944102ae0 msgr2=0x7f69441a0c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:48:18.619 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:18.617+0000 7f6949112640 1 --2- 172.21.15.39:0/2508571639 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6944102ae0 0x7f69441a0c00 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f6934005940 tx=0x7f6934004270 comp rx=0 tx=0).stop 2024-09-21T03:48:18.619 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:18.617+0000 7f6949112640 1 -- 172.21.15.39:0/2508571639 shutdown_connections 2024-09-21T03:48:18.620 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:18.617+0000 7f6949112640 1 --2- 172.21.15.39:0/2508571639 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f692c0b6700 0x7f692c0b8c10 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:18.620 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:18.617+0000 7f6949112640 1 --2- 172.21.15.39:0/2508571639 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6944108d20 0x7f69441a6440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:18.620 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:18.617+0000 7f6949112640 1 --2- 172.21.15.39:0/2508571639 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f69441083d0 0x7f69441a11a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:18.620 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:18.617+0000 7f6949112640 1 --2- 172.21.15.39:0/2508571639 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6944102ae0 0x7f69441a0c00 unknown :-1 s=CLOSED pgs=617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:18.620 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:18.617+0000 7f6949112640 1 -- 172.21.15.39:0/2508571639 >> 172.21.15.39:0/2508571639 conn(0x7f6944079790 msgr2=0x7f694406baa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:48:18.620 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:18.618+0000 7f6949112640 1 -- 172.21.15.39:0/2508571639 shutdown_connections 2024-09-21T03:48:18.620 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:18.618+0000 7f6949112640 1 -- 172.21.15.39:0/2508571639 wait complete. 2024-09-21T03:48:19.333 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:48:19 smithi039 ceph-mon[31211]: pgmap v393: 97 pgs: 97 active+clean; 9.2 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 487 KiB/s rd, 733 MiB/s wr, 381 op/s 2024-09-21T03:48:19.467 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:48:19 smithi161 ceph-mon[34533]: pgmap v393: 97 pgs: 97 active+clean; 9.2 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 487 KiB/s rd, 733 MiB/s wr, 381 op/s 2024-09-21T03:48:19.474 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:48:19 smithi150 ceph-mon[34289]: pgmap v393: 97 pgs: 97 active+clean; 9.2 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 487 KiB/s rd, 733 MiB/s wr, 381 op/s 2024-09-21T03:48:20.112 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 51116d5c-77ca-11ef-baf5-efdc52797490 -- ceph tell mds.1:2 damage ls 2024-09-21T03:48:20.636 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/51116d5c-77ca-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:48:21.334 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:48:21 smithi039 ceph-mon[31211]: pgmap v394: 97 pgs: 97 active+clean; 10 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 545 KiB/s rd, 798 MiB/s wr, 429 op/s 2024-09-21T03:48:21.467 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:48:21 smithi161 ceph-mon[34533]: pgmap v394: 97 pgs: 97 active+clean; 10 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 545 KiB/s rd, 798 MiB/s wr, 429 op/s 2024-09-21T03:48:21.474 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:48:21 smithi150 ceph-mon[34289]: pgmap v394: 97 pgs: 97 active+clean; 10 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 545 KiB/s rd, 798 MiB/s wr, 429 op/s 2024-09-21T03:48:21.659 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:21.641+0000 7f913edb2640 1 -- 172.21.15.39:0/1336463737 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9138103ba0 msgr2=0x7f9138103fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:48:21.659 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:21.641+0000 7f913edb2640 1 --2- 172.21.15.39:0/1336463737 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9138103ba0 0x7f9138103fb0 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f912c009900 tx=0x7f912c02a410 comp rx=0 tx=0).stop 2024-09-21T03:48:21.659 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:21.641+0000 7f913edb2640 1 -- 172.21.15.39:0/1336463737 shutdown_connections 2024-09-21T03:48:21.659 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:21.641+0000 7f913edb2640 1 --2- 172.21.15.39:0/1336463737 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9138109e30 0x7f9138110660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:21.659 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:21.641+0000 7f913edb2640 1 --2- 172.21.15.39:0/1336463737 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f91381094e0 0x7f91381098f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:21.659 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:21.641+0000 7f913edb2640 1 --2- 172.21.15.39:0/1336463737 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9138103ba0 0x7f9138103fb0 unknown :-1 s=CLOSED pgs=622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:21.659 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:21.641+0000 7f913edb2640 1 -- 172.21.15.39:0/1336463737 >> 172.21.15.39:0/1336463737 conn(0x7f91380ff630 msgr2=0x7f9138101a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:48:21.660 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:21.641+0000 7f913edb2640 1 -- 172.21.15.39:0/1336463737 shutdown_connections 2024-09-21T03:48:21.660 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:21.641+0000 7f913edb2640 1 -- 172.21.15.39:0/1336463737 wait complete. 2024-09-21T03:48:21.660 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:21.642+0000 7f913edb2640 1 Processor -- start 2024-09-21T03:48:21.660 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:21.642+0000 7f913edb2640 1 -- start start 2024-09-21T03:48:21.660 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:21.643+0000 7f913edb2640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9138103ba0 0x7f913819a610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:48:21.660 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:21.643+0000 7f913edb2640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f91381094e0 0x7f913819abb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:48:21.660 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:21.643+0000 7f913edb2640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9138109e30 0x7f913819fe50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:48:21.660 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:21.643+0000 7f913edb2640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f9138113c00 con 0x7f9138109e30 2024-09-21T03:48:21.660 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:21.644+0000 7f913edb2640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f9138113a80 con 0x7f91381094e0 2024-09-21T03:48:21.660 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:21.644+0000 7f913cb4e640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9138103ba0 0x7f913819a610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:48:21.660 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:21.644+0000 7f913cb4e640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9138103ba0 0x7f913819a610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.39:54892/0 (socket says 172.21.15.39:54892) 2024-09-21T03:48:21.660 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:21.644+0000 7f913cb4e640 1 -- 172.21.15.39:0/41636786 learned_addr learned my addr 172.21.15.39:0/41636786 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:48:21.660 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:21.644+0000 7f913edb2640 1 -- 172.21.15.39:0/41636786 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f9138113d80 con 0x7f9138103ba0 2024-09-21T03:48:21.661 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:21.644+0000 7f9137fff640 1 --2- 172.21.15.39:0/41636786 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f91381094e0 0x7f913819abb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:48:21.661 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:21.645+0000 7f913d34f640 1 --2- 172.21.15.39:0/41636786 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9138109e30 0x7f913819fe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:48:21.661 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:21.645+0000 7f913cb4e640 1 -- 172.21.15.39:0/41636786 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f91381094e0 msgr2=0x7f913819abb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:48:21.661 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:21.645+0000 7f913cb4e640 1 --2- 172.21.15.39:0/41636786 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f91381094e0 0x7f913819abb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:21.661 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:21.645+0000 7f913cb4e640 1 -- 172.21.15.39:0/41636786 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9138109e30 msgr2=0x7f913819fe50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:48:21.661 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:21.645+0000 7f913cb4e640 1 --2- 172.21.15.39:0/41636786 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9138109e30 0x7f913819fe50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:21.661 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:21.645+0000 7f913cb4e640 1 -- 172.21.15.39:0/41636786 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f91381a0580 con 0x7f9138103ba0 2024-09-21T03:48:21.661 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:21.647+0000 7f913cb4e640 1 --2- 172.21.15.39:0/41636786 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9138103ba0 0x7f913819a610 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f912c005ab0 tx=0x7f912c003d50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:48:21.661 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:21.647+0000 7f9135ffb640 1 -- 172.21.15.39:0/41636786 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f912c03f070 con 0x7f9138103ba0 2024-09-21T03:48:21.661 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:21.648+0000 7f9135ffb640 1 -- 172.21.15.39:0/41636786 <== mon.1 v2:172.21.15.161:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f912c031d60 con 0x7f9138103ba0 2024-09-21T03:48:21.661 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:21.649+0000 7f9135ffb640 1 -- 172.21.15.39:0/41636786 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f912c035030 con 0x7f9138103ba0 2024-09-21T03:48:21.661 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:21.649+0000 7f913edb2640 1 -- 172.21.15.39:0/41636786 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f91381a0810 con 0x7f9138103ba0 2024-09-21T03:48:21.661 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:21.650+0000 7f913edb2640 1 -- 172.21.15.39:0/41636786 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f91381a6fd0 con 0x7f9138103ba0 2024-09-21T03:48:21.661 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:21.654+0000 7f9145326740 1 Processor -- start 2024-09-21T03:48:21.661 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:21.655+0000 7f9135ffb640 1 -- 172.21.15.39:0/41636786 <== mon.1 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 27) ==== 154643+0+0 (secure 0 0 0) 0x7f912c042070 con 0x7f9138103ba0 2024-09-21T03:48:21.662 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:21.656+0000 7f9135ffb640 1 --2- 172.21.15.39:0/41636786 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f91280b6780 0x7f91280b8c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:48:21.662 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:21.657+0000 7f9135ffb640 1 -- 172.21.15.39:0/41636786 <== mon.1 v2:172.21.15.161:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8484+0+0 (secure 0 0 0) 0x7f912c1082a0 con 0x7f9138103ba0 2024-09-21T03:48:21.662 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:21.657+0000 7f9145326740 1 -- start start 2024-09-21T03:48:21.662 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:21.658+0000 7f9145326740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559c50d15070 0x559c510bc2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:48:21.662 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:21.658+0000 7f9145326740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559c50cec1b0 0x559c510bc800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:48:21.663 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:21.659+0000 7f913cb4e640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559c50d15070 0x559c510bc2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:48:21.663 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:21.659+0000 7f913cb4e640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559c50d15070 0x559c510bc2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.39:54896/0 (socket says 172.21.15.39:54896) 2024-09-21T03:48:21.663 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:21.659+0000 7f913d34f640 1 --2- 172.21.15.39:0/41636786 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f91280b6780 0x7f91280b8c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:48:21.663 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:21.659+0000 7f913cb4e640 1 -- 172.21.15.39:0/424293931 learned_addr learned my addr 172.21.15.39:0/424293931 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:48:21.663 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:21.659+0000 7f9145326740 1 --2- 172.21.15.39:0/424293931 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x559c50cf4390 0x559c510bcd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:48:21.663 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:21.660+0000 7f9145326740 1 -- 172.21.15.39:0/424293931 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x559c51032ea0 con 0x559c50cec1b0 2024-09-21T03:48:21.663 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:21.660+0000 7f9145326740 1 -- 172.21.15.39:0/424293931 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x559c510222d0 con 0x559c50cf4390 2024-09-21T03:48:21.664 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:21.660+0000 7f9145326740 1 -- 172.21.15.39:0/424293931 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x559c510260e0 con 0x559c50d15070 2024-09-21T03:48:21.664 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:21.660+0000 7f913d34f640 1 --2- 172.21.15.39:0/424293931 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x559c50cf4390 0x559c510bcd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:48:21.664 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:21.661+0000 7f913d34f640 1 --2- 172.21.15.39:0/41636786 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f91280b6780 0x7f91280b8c90 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f9130009ef0 tx=0x7f9130009410 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:48:21.664 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:21.662+0000 7f913cb4e640 1 -- 172.21.15.39:0/424293931 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x559c50cf4390 msgr2=0x559c510bcd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:48:21.664 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:21.662+0000 7f913cb4e640 1 --2- 172.21.15.39:0/424293931 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x559c50cf4390 0x559c510bcd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:21.664 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:21.662+0000 7f913cb4e640 1 -- 172.21.15.39:0/424293931 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559c50cec1b0 msgr2=0x559c510bc800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:48:21.665 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:21.662+0000 7f913cb4e640 1 --2- 172.21.15.39:0/424293931 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559c50cec1b0 0x559c510bc800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:21.665 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:21.662+0000 7f913cb4e640 1 -- 172.21.15.39:0/424293931 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559c50d80650 con 0x559c50d15070 2024-09-21T03:48:21.670 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:21.667+0000 7f913cb4e640 1 --2- 172.21.15.39:0/424293931 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559c50d15070 0x559c510bc2c0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f912c0ce040 tx=0x7f912c009260 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b403eb2d8a7d097e server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:48:21.670 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:21.668+0000 7f9122ffd640 1 -- 172.21.15.39:0/424293931 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f912c0ce7b0 con 0x559c50d15070 2024-09-21T03:48:21.670 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:21.668+0000 7f9122ffd640 1 -- 172.21.15.39:0/424293931 <== mon.1 v2:172.21.15.161:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f912c03be50 con 0x559c50d15070 2024-09-21T03:48:21.671 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:21.669+0000 7f9122ffd640 1 -- 172.21.15.39:0/424293931 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f912c0d7640 con 0x559c50d15070 2024-09-21T03:48:21.671 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:21.670+0000 7f9145326740 1 -- 172.21.15.39:0/424293931 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559c50d15070 msgr2=0x559c510bc2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:48:21.671 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:21.670+0000 7f9145326740 1 --2- 172.21.15.39:0/424293931 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559c50d15070 0x559c510bc2c0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f912c0ce040 tx=0x7f912c009260 comp rx=0 tx=0).stop 2024-09-21T03:48:21.674 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:21.670+0000 7f9145326740 1 -- 172.21.15.39:0/424293931 shutdown_connections 2024-09-21T03:48:21.674 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:21.670+0000 7f9145326740 1 --2- 172.21.15.39:0/424293931 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x559c50cf4390 0x559c510bcd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:21.674 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:21.670+0000 7f9145326740 1 --2- 172.21.15.39:0/424293931 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559c50cec1b0 0x559c510bc800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:21.674 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:21.670+0000 7f9145326740 1 --2- 172.21.15.39:0/424293931 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559c50d15070 0x559c510bc2c0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:21.674 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:21.670+0000 7f9145326740 1 -- 172.21.15.39:0/424293931 >> 172.21.15.39:0/424293931 conn(0x559c50d09460 msgr2=0x559c510ba1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:48:21.674 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:21.671+0000 7f9145326740 1 -- 172.21.15.39:0/424293931 shutdown_connections 2024-09-21T03:48:21.675 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:21.671+0000 7f9145326740 1 -- 172.21.15.39:0/424293931 wait complete. 2024-09-21T03:48:21.675 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:21.672+0000 7f9145326740 1 -- start start 2024-09-21T03:48:21.675 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:21.672+0000 7f9145326740 1 Processor -- start 2024-09-21T03:48:21.676 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:21.673+0000 7f9145326740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559c50cec1b0 0x559c510c79e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:48:21.676 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:21.673+0000 7f9145326740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559c50cf4390 0x559c510c7f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:48:21.676 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:21.673+0000 7f9145326740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x559c50d15070 0x559c510dc2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:48:21.676 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:21.673+0000 7f9145326740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x559c50e83700 con 0x559c50cf4390 2024-09-21T03:48:21.676 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:21.673+0000 7f9145326740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x559c50fbd160 con 0x559c50d15070 2024-09-21T03:48:21.676 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:21.673+0000 7f9145326740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x559c510ca860 con 0x559c50cec1b0 2024-09-21T03:48:21.676 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:21.673+0000 7f9145326740 10 client.0 _ll_register_callbacks cb 0x559c50fd69e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:48:21.676 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:21.674+0000 7f913d34f640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x559c50d15070 0x559c510dc2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:48:21.676 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:21.674+0000 7f913d34f640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x559c50d15070 0x559c510dc2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.39:41074/0 (socket says 172.21.15.39:41074) 2024-09-21T03:48:21.677 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:21.674+0000 7f913d34f640 1 -- 172.21.15.39:0/3518016953 learned_addr learned my addr 172.21.15.39:0/3518016953 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:48:21.677 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:21.674+0000 7f9137fff640 1 --2- 172.21.15.39:0/3518016953 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559c50cf4390 0x559c510c7f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:48:21.677 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:21.675+0000 7f913cb4e640 1 --2- 172.21.15.39:0/3518016953 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559c50cec1b0 0x559c510c79e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:48:21.678 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:21.675+0000 7f913d34f640 1 -- 172.21.15.39:0/3518016953 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559c50cec1b0 msgr2=0x559c510c79e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:48:21.678 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:21.675+0000 7f913d34f640 1 --2- 172.21.15.39:0/3518016953 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559c50cec1b0 0x559c510c79e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:21.678 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:21.675+0000 7f913d34f640 1 -- 172.21.15.39:0/3518016953 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559c50cf4390 msgr2=0x559c510c7f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:48:21.678 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:21.675+0000 7f913d34f640 1 --2- 172.21.15.39:0/3518016953 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559c50cf4390 0x559c510c7f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:21.678 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:21.675+0000 7f913d34f640 1 -- 172.21.15.39:0/3518016953 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559c50feac90 con 0x559c50d15070 2024-09-21T03:48:21.681 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:21.678+0000 7f913d34f640 1 --2- 172.21.15.39:0/3518016953 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x559c50d15070 0x559c510dc2d0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f9130017d90 tx=0x7f9130011000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8eaf0f04534c02b4 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:48:21.681 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:21.678+0000 7f9117fff640 10 client.0 ms_handle_connect on v2:172.21.15.150:3300/0 2024-09-21T03:48:21.682 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:21.679+0000 7f9117fff640 1 -- 172.21.15.39:0/3518016953 <== mon.2 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9130029060 con 0x559c50d15070 2024-09-21T03:48:21.682 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:21.679+0000 7f9117fff640 1 -- 172.21.15.39:0/3518016953 <== mon.2 v2:172.21.15.150:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8484+0+0 (secure 0 0 0) 0x7f9130016ab0 con 0x559c50d15070 2024-09-21T03:48:21.683 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:21.681+0000 7f9145326740 1 -- 172.21.15.39:0/3518016953 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f912c03be50 con 0x559c50d15070 2024-09-21T03:48:21.684 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:21.682+0000 7f9117fff640 1 -- 172.21.15.39:0/3518016953 <== mon.2 v2:172.21.15.150:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f9130016cb0 con 0x559c50d15070 2024-09-21T03:48:21.684 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:21.682+0000 7f9145326740 10 client.25637 fetch_fsmap learned FSMap version 29 2024-09-21T03:48:21.685 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:21.682+0000 7f9145326740 1 -- 172.21.15.39:0/3518016953 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x559c510cd000 con 0x559c50d15070 2024-09-21T03:48:21.688 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:21.686+0000 7f9117fff640 1 -- 172.21.15.39:0/3518016953 <== mon.2 v2:172.21.15.150:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f912c03be50 con 0x559c50d15070 2024-09-21T03:48:21.692 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:21.689+0000 7f9117fff640 1 -- 172.21.15.39:0/3518016953 <== mon.2 v2:172.21.15.150:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f9130023070 con 0x559c50d15070 2024-09-21T03:48:21.692 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:21.689+0000 7f9145326740 10 client.25637 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T03:48:21.692 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:21.689+0000 7f9145326740 10 client.25637 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-21T03:48:21.692 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:21.689+0000 7f9145326740 20 client.25637 populate_metadata read hostname 'smithi039' 2024-09-21T03:48:21.693 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:21.689+0000 7f9145326740 1 --2- 172.21.15.39:0/3518016953 >> [v2:172.21.15.150:6836/4017043144,v1:172.21.15.150:6837/4017043144] conn(0x559c50cd39f0 0x559c510d08d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:48:21.693 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:21.689+0000 7f9145326740 4 client.25637 mds_command: new command op to 24659 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:48:21.693 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:21.689+0000 7f9145326740 1 -- 172.21.15.39:0/3518016953 --> [v2:172.21.15.150:6836/4017043144,v1:172.21.15.150:6837/4017043144] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559c51034df0 con 0x559c50cd39f0 2024-09-21T03:48:21.693 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:21.691+0000 7f913cb4e640 1 --2- 172.21.15.39:0/3518016953 >> [v2:172.21.15.150:6836/4017043144,v1:172.21.15.150:6837/4017043144] conn(0x559c50cd39f0 0x559c510d08d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:48:21.698 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:21.696+0000 7f9117fff640 10 client.25637 ms_handle_connect on v2:172.21.15.150:6836/4017043144 2024-09-21T03:48:21.698 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:21.696+0000 7f913cb4e640 1 --2- 172.21.15.39:0/3518016953 >> [v2:172.21.15.150:6836/4017043144,v1:172.21.15.150:6837/4017043144] conn(0x559c50cd39f0 0x559c510d08d0 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=b1bf6006cf781a8a server_cookie=18a13e7ea19768a1 in_seq=0 out_seq=0 2024-09-21T03:48:21.701 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:21.699+0000 7f9117fff640 1 -- 172.21.15.39:0/3518016953 <== mds.2 v2:172.21.15.150:6836/4017043144 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x559c51034df0 con 0x559c50cd39f0 2024-09-21T03:48:21.701 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:21.699+0000 7f9117fff640 10 client.25637 handle_command_reply: tid=0 2024-09-21T03:48:21.701 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:21.699+0000 7f9145326740 1 client.25637 shutdown 2024-09-21T03:48:21.703 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:21.700+0000 7f9145326740 1 -- 172.21.15.39:0/3518016953 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x559c50d15070 msgr2=0x559c510dc2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:48:21.703 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:21.700+0000 7f9145326740 1 --2- 172.21.15.39:0/3518016953 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x559c50d15070 0x559c510dc2d0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f9130017d90 tx=0x7f9130011000 comp rx=0 tx=0).stop 2024-09-21T03:48:21.704 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:21.700+0000 7f9145326740 1 -- 172.21.15.39:0/3518016953 shutdown_connections 2024-09-21T03:48:21.704 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:21.700+0000 7f9145326740 1 --2- 172.21.15.39:0/3518016953 >> [v2:172.21.15.150:6836/4017043144,v1:172.21.15.150:6837/4017043144] conn(0x559c50cd39f0 0x559c510d08d0 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:21.704 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:21.700+0000 7f9145326740 1 --2- 172.21.15.39:0/3518016953 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x559c50d15070 0x559c510dc2d0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:21.704 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:21.700+0000 7f9145326740 1 --2- 172.21.15.39:0/3518016953 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559c50cf4390 0x559c510c7f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:21.704 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:21.700+0000 7f9145326740 1 --2- 172.21.15.39:0/3518016953 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559c50cec1b0 0x559c510c79e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:21.704 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:21.700+0000 7f9145326740 1 -- 172.21.15.39:0/3518016953 >> 172.21.15.39:0/3518016953 conn(0x559c50d09460 msgr2=0x559c510bf450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:48:21.705 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:21.701+0000 7f9117fff640 0 client.25637 ms_handle_reset on v2:172.21.15.150:6836/4017043144 2024-09-21T03:48:21.705 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:21.701+0000 7f9145326740 1 -- 172.21.15.39:0/3518016953 shutdown_connections 2024-09-21T03:48:21.705 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:21.701+0000 7f9145326740 1 -- 172.21.15.39:0/3518016953 wait complete. 2024-09-21T03:48:21.705 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:21.701+0000 7f9145326740 20 client.25637 trim_cache size 0 max 16384 2024-09-21T03:48:21.745 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:21.743+0000 7f9145326740 1 Processor -- start 2024-09-21T03:48:21.746 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:21.743+0000 7f9145326740 1 -- start start 2024-09-21T03:48:21.747 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:21.744+0000 7f9145326740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x559c50cec1b0 0x559c510c7ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:48:21.747 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:21.745+0000 7f9145326740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559c50cf4390 0x559c510dc090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:48:21.747 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:21.745+0000 7f9145326740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559c50d15070 0x559c510c4bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:48:21.747 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:21.745+0000 7f9145326740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f91381ad510 con 0x559c50d15070 2024-09-21T03:48:21.748 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:21.745+0000 7f9145326740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f91381af050 con 0x559c50cec1b0 2024-09-21T03:48:21.748 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:21.745+0000 7f9145326740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f91381adbe0 con 0x559c50cf4390 2024-09-21T03:48:21.748 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:21.746+0000 7f913cb4e640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x559c50cec1b0 0x559c510c7ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:48:21.749 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:21.746+0000 7f913cb4e640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x559c50cec1b0 0x559c510c7ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.39:41076/0 (socket says 172.21.15.39:41076) 2024-09-21T03:48:21.749 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:21.746+0000 7f913cb4e640 1 -- 172.21.15.39:0/4134065441 learned_addr learned my addr 172.21.15.39:0/4134065441 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:48:21.749 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:21.746+0000 7f913d34f640 1 --2- 172.21.15.39:0/4134065441 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559c50d15070 0x559c510c4bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:48:21.750 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:21.747+0000 7f9137fff640 1 --2- 172.21.15.39:0/4134065441 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559c50cf4390 0x559c510dc090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:48:21.750 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:21.747+0000 7f913cb4e640 1 -- 172.21.15.39:0/4134065441 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559c50cf4390 msgr2=0x559c510dc090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:48:21.750 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:21.747+0000 7f913cb4e640 1 --2- 172.21.15.39:0/4134065441 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559c50cf4390 0x559c510dc090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:21.750 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:21.747+0000 7f913cb4e640 1 -- 172.21.15.39:0/4134065441 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559c50d15070 msgr2=0x559c510c4bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:48:21.751 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:21.747+0000 7f913cb4e640 1 --2- 172.21.15.39:0/4134065441 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559c50d15070 0x559c510c4bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:21.751 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:21.747+0000 7f913cb4e640 1 -- 172.21.15.39:0/4134065441 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559c510bea10 con 0x559c50cec1b0 2024-09-21T03:48:21.751 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:21.748+0000 7f913cb4e640 1 --2- 172.21.15.39:0/4134065441 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x559c50cec1b0 0x559c510c7ea0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f912c0ca680 tx=0x7f912c009680 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2ac350b34ec19be7 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:48:21.754 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:21.748+0000 7f91227fc640 1 -- 172.21.15.39:0/4134065441 <== mon.2 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f912c0d7ad0 con 0x559c50cec1b0 2024-09-21T03:48:21.755 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:21.748+0000 7f91227fc640 1 -- 172.21.15.39:0/4134065441 <== mon.2 v2:172.21.15.150:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f912c03be50 con 0x559c50cec1b0 2024-09-21T03:48:21.755 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:21.748+0000 7f91227fc640 1 -- 172.21.15.39:0/4134065441 <== mon.2 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f912c0b9430 con 0x559c50cec1b0 2024-09-21T03:48:21.755 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:21.750+0000 7f9145326740 1 -- 172.21.15.39:0/4134065441 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x559c50cec1b0 msgr2=0x559c510c7ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:48:21.755 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:21.750+0000 7f9145326740 1 --2- 172.21.15.39:0/4134065441 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x559c50cec1b0 0x559c510c7ea0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f912c0ca680 tx=0x7f912c009680 comp rx=0 tx=0).stop 2024-09-21T03:48:21.755 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:21.750+0000 7f9145326740 1 -- 172.21.15.39:0/4134065441 shutdown_connections 2024-09-21T03:48:21.756 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:21.750+0000 7f9145326740 1 --2- 172.21.15.39:0/4134065441 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559c50d15070 0x559c510c4bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:21.756 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:21.750+0000 7f9145326740 1 --2- 172.21.15.39:0/4134065441 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559c50cf4390 0x559c510dc090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:21.756 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:21.750+0000 7f9145326740 1 --2- 172.21.15.39:0/4134065441 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x559c50cec1b0 0x559c510c7ea0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:21.756 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:21.750+0000 7f9145326740 1 -- 172.21.15.39:0/4134065441 >> 172.21.15.39:0/4134065441 conn(0x559c50d09460 msgr2=0x559c510c7790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:48:21.756 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:21.751+0000 7f9145326740 1 -- 172.21.15.39:0/4134065441 shutdown_connections 2024-09-21T03:48:21.756 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:21.751+0000 7f9145326740 1 -- 172.21.15.39:0/4134065441 wait complete. 2024-09-21T03:48:21.756 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:21.752+0000 7f9145326740 1 -- start start 2024-09-21T03:48:21.757 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:21.752+0000 7f9145326740 1 Processor -- start 2024-09-21T03:48:21.759 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:21.753+0000 7f9145326740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559c50cec1b0 0x559c5114dc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:48:21.759 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:21.754+0000 7f9145326740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x559c50cf4390 0x559c5114e1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:48:21.759 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:21.754+0000 7f9145326740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559c50d15070 0x559c510d5f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:48:21.760 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:21.754+0000 7f913cb4e640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559c50cec1b0 0x559c5114dc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:48:21.760 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:21.754+0000 7f913cb4e640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559c50cec1b0 0x559c5114dc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.39:54918/0 (socket says 172.21.15.39:54918) 2024-09-21T03:48:21.760 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:21.755+0000 7f9137fff640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x559c50cf4390 0x559c5114e1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:48:21.760 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:21.755+0000 7f9137fff640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x559c50cf4390 0x559c5114e1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.39:41092/0 (socket says 172.21.15.39:41092) 2024-09-21T03:48:21.760 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:21.755+0000 7f913cb4e640 1 -- 172.21.15.39:0/500191621 learned_addr learned my addr 172.21.15.39:0/500191621 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:48:21.760 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:21.756+0000 7f9145326740 1 -- 172.21.15.39:0/500191621 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f91381ae980 con 0x559c50d15070 2024-09-21T03:48:21.761 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:21.756+0000 7f9145326740 1 -- 172.21.15.39:0/500191621 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f91381ae2b0 con 0x559c50cf4390 2024-09-21T03:48:21.761 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:21.756+0000 7f9145326740 1 -- 172.21.15.39:0/500191621 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f91381ab5c0 con 0x559c50cec1b0 2024-09-21T03:48:21.761 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:21.756+0000 7f9145326740 10 client.0 _ll_register_callbacks cb 0x7f91300174b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:48:21.761 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:21.756+0000 7f913d34f640 1 --2- 172.21.15.39:0/500191621 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559c50d15070 0x559c510d5f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:48:21.763 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:21.757+0000 7f9137fff640 1 -- 172.21.15.39:0/500191621 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559c50cec1b0 msgr2=0x559c5114dc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:48:21.763 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:21.757+0000 7f9137fff640 1 --2- 172.21.15.39:0/500191621 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559c50cec1b0 0x559c5114dc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:21.763 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:21.757+0000 7f9137fff640 1 -- 172.21.15.39:0/500191621 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559c50d15070 msgr2=0x559c510d5f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:48:21.763 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:21.757+0000 7f9137fff640 1 --2- 172.21.15.39:0/500191621 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559c50d15070 0x559c510d5f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:21.763 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:21.757+0000 7f9137fff640 1 -- 172.21.15.39:0/500191621 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559c510d6580 con 0x559c50cf4390 2024-09-21T03:48:21.764 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:21.759+0000 7f9137fff640 1 --2- 172.21.15.39:0/500191621 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x559c50cf4390 0x559c5114e1e0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x559c510baf40 tx=0x7f9124014ad0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d1efdb26ec7db61b server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:48:21.764 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:21.759+0000 7f9117fff640 10 client.0 ms_handle_connect on v2:172.21.15.150:3300/0 2024-09-21T03:48:21.764 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:21.760+0000 7f9117fff640 1 -- 172.21.15.39:0/500191621 <== mon.2 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91240167c0 con 0x559c50cf4390 2024-09-21T03:48:21.764 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:21.760+0000 7f9145326740 1 -- 172.21.15.39:0/500191621 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f912c03be50 con 0x559c50cf4390 2024-09-21T03:48:21.764 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:21.760+0000 7f9117fff640 1 -- 172.21.15.39:0/500191621 <== mon.2 v2:172.21.15.150:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8484+0+0 (secure 0 0 0) 0x7f9124016960 con 0x559c50cf4390 2024-09-21T03:48:21.764 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:21.761+0000 7f9117fff640 1 -- 172.21.15.39:0/500191621 <== mon.2 v2:172.21.15.150:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f9124016b60 con 0x559c50cf4390 2024-09-21T03:48:21.770 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:21.769+0000 7f9145326740 10 client.25643 fetch_fsmap learned FSMap version 29 2024-09-21T03:48:21.771 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:21.769+0000 7f9145326740 1 -- 172.21.15.39:0/500191621 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x559c511554a0 con 0x559c50cf4390 2024-09-21T03:48:21.786 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:21.779+0000 7f9117fff640 1 -- 172.21.15.39:0/500191621 <== mon.2 v2:172.21.15.150:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9124016d00 con 0x559c50cf4390 2024-09-21T03:48:21.786 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:21.781+0000 7f9117fff640 1 -- 172.21.15.39:0/500191621 <== mon.2 v2:172.21.15.150:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f9124025020 con 0x559c50cf4390 2024-09-21T03:48:21.786 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:21.782+0000 7f9145326740 10 client.25643 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T03:48:21.786 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:21.782+0000 7f9145326740 10 client.25643 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-21T03:48:21.786 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:21.782+0000 7f9145326740 20 client.25643 populate_metadata read hostname 'smithi039' 2024-09-21T03:48:21.786 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:21.782+0000 7f9145326740 1 --2- 172.21.15.39:0/500191621 >> [v2:172.21.15.150:6836/4017043144,v1:172.21.15.150:6837/4017043144] conn(0x559c50ca8da0 0x559c51157c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:48:21.787 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:21.782+0000 7f9145326740 4 client.25643 mds_command: new command op to 24659 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T03:48:21.787 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:21.782+0000 7f9145326740 1 -- 172.21.15.39:0/500191621 --> [v2:172.21.15.150:6836/4017043144,v1:172.21.15.150:6837/4017043144] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7f912c03be50 con 0x559c50ca8da0 2024-09-21T03:48:21.793 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:21.790+0000 7f913cb4e640 1 --2- 172.21.15.39:0/500191621 >> [v2:172.21.15.150:6836/4017043144,v1:172.21.15.150:6837/4017043144] conn(0x559c50ca8da0 0x559c51157c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:48:21.798 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:21.796+0000 7f913cb4e640 1 --2- 172.21.15.39:0/500191621 >> [v2:172.21.15.150:6836/4017043144,v1:172.21.15.150:6837/4017043144] conn(0x559c50ca8da0 0x559c51157c30 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=4f2a32bc7a2a77ca server_cookie=c66c6ab6d5f59846 in_seq=0 out_seq=0 2024-09-21T03:48:21.798 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:21.796+0000 7f9117fff640 10 client.25643 ms_handle_connect on v2:172.21.15.150:6836/4017043144 2024-09-21T03:48:21.800 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:21.798+0000 7f9117fff640 1 -- 172.21.15.39:0/500191621 <== mds.2 v2:172.21.15.150:6836/4017043144 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7f912c03be50 con 0x559c50ca8da0 2024-09-21T03:48:21.800 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:21.798+0000 7f9117fff640 10 client.25643 handle_command_reply: tid=0 2024-09-21T03:48:21.800 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:21.798+0000 7f9145326740 1 client.25643 shutdown 2024-09-21T03:48:21.802 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:21.799+0000 7f9145326740 1 -- 172.21.15.39:0/500191621 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x559c50cf4390 msgr2=0x559c5114e1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:48:21.802 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:21.799+0000 7f9145326740 1 --2- 172.21.15.39:0/500191621 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x559c50cf4390 0x559c5114e1e0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x559c510baf40 tx=0x7f9124014ad0 comp rx=0 tx=0).stop 2024-09-21T03:48:21.802 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:21.799+0000 7f9145326740 1 -- 172.21.15.39:0/500191621 shutdown_connections 2024-09-21T03:48:21.802 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:21.799+0000 7f9145326740 1 --2- 172.21.15.39:0/500191621 >> [v2:172.21.15.150:6836/4017043144,v1:172.21.15.150:6837/4017043144] conn(0x559c50ca8da0 0x559c51157c30 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:21.802 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:21.799+0000 7f9145326740 1 --2- 172.21.15.39:0/500191621 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559c50d15070 0x559c510d5f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:21.802 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:21.799+0000 7f9117fff640 0 client.25643 ms_handle_reset on v2:172.21.15.150:6836/4017043144 2024-09-21T03:48:21.802 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:21.800+0000 7f9145326740 1 --2- 172.21.15.39:0/500191621 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x559c50cf4390 0x559c5114e1e0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:21.802 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:21.800+0000 7f9145326740 1 --2- 172.21.15.39:0/500191621 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559c50cec1b0 0x559c5114dc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:21.802 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:21.800+0000 7f9145326740 1 -- 172.21.15.39:0/500191621 >> 172.21.15.39:0/500191621 conn(0x559c50d09460 msgr2=0x559c510bd2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:48:21.803 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:21.800+0000 7f9145326740 1 -- 172.21.15.39:0/500191621 shutdown_connections 2024-09-21T03:48:21.803 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:21.801+0000 7f9145326740 1 -- 172.21.15.39:0/500191621 wait complete. 2024-09-21T03:48:21.803 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:21.801+0000 7f9145326740 20 client.25643 trim_cache size 0 max 16384 2024-09-21T03:48:21.804 INFO:teuthology.orchestra.run.smithi039.stdout:[] 2024-09-21T03:48:21.810 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:21.808+0000 7f913edb2640 1 -- 172.21.15.39:0/41636786 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f91280b6780 msgr2=0x7f91280b8c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:48:21.810 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:21.808+0000 7f913edb2640 1 --2- 172.21.15.39:0/41636786 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f91280b6780 0x7f91280b8c90 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f9130009ef0 tx=0x7f9130009410 comp rx=0 tx=0).stop 2024-09-21T03:48:21.810 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:21.808+0000 7f913edb2640 1 -- 172.21.15.39:0/41636786 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9138103ba0 msgr2=0x7f913819a610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:48:21.810 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:21.808+0000 7f913edb2640 1 --2- 172.21.15.39:0/41636786 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9138103ba0 0x7f913819a610 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f912c005ab0 tx=0x7f912c003d50 comp rx=0 tx=0).stop 2024-09-21T03:48:21.813 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:21.810+0000 7f913edb2640 1 -- 172.21.15.39:0/41636786 shutdown_connections 2024-09-21T03:48:21.813 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:21.810+0000 7f913edb2640 1 --2- 172.21.15.39:0/41636786 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f91280b6780 0x7f91280b8c90 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:21.814 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:21.810+0000 7f913edb2640 1 --2- 172.21.15.39:0/41636786 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9138109e30 0x7f913819fe50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:21.814 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:21.810+0000 7f913edb2640 1 --2- 172.21.15.39:0/41636786 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f91381094e0 0x7f913819abb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:21.814 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:21.810+0000 7f913edb2640 1 --2- 172.21.15.39:0/41636786 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9138103ba0 0x7f913819a610 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:21.814 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:21.810+0000 7f913edb2640 1 -- 172.21.15.39:0/41636786 >> 172.21.15.39:0/41636786 conn(0x7f91380ff630 msgr2=0x7f9138101a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:48:21.814 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:21.811+0000 7f913edb2640 1 -- 172.21.15.39:0/41636786 shutdown_connections 2024-09-21T03:48:21.814 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:21.811+0000 7f913edb2640 1 -- 172.21.15.39:0/41636786 wait complete. 2024-09-21T03:48:23.319 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 51116d5c-77ca-11ef-baf5-efdc52797490 -- ceph tell mds.1:1 damage ls 2024-09-21T03:48:23.334 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:48:23 smithi039 ceph-mon[31211]: pgmap v395: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 176 KiB/s rd, 606 MiB/s wr, 219 op/s 2024-09-21T03:48:23.468 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:48:23 smithi161 ceph-mon[34533]: pgmap v395: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 176 KiB/s rd, 606 MiB/s wr, 219 op/s 2024-09-21T03:48:23.474 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:48:23 smithi150 ceph-mon[34289]: pgmap v395: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 176 KiB/s rd, 606 MiB/s wr, 219 op/s 2024-09-21T03:48:23.875 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/51116d5c-77ca-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:48:24.845 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:24.840+0000 7f6bbfd2f640 1 -- 172.21.15.39:0/1506372783 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6bb810ad10 msgr2=0x7f6bb8116fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:48:24.845 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:24.840+0000 7f6bbfd2f640 1 --2- 172.21.15.39:0/1506372783 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6bb810ad10 0x7f6bb8116fc0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f6bb0005fd0 tx=0x7f6bb002b430 comp rx=0 tx=0).stop 2024-09-21T03:48:24.845 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:24.841+0000 7f6bbfd2f640 1 -- 172.21.15.39:0/1506372783 shutdown_connections 2024-09-21T03:48:24.845 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:24.841+0000 7f6bbfd2f640 1 --2- 172.21.15.39:0/1506372783 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6bb810ad10 0x7f6bb8116fc0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:24.845 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:24.841+0000 7f6bbfd2f640 1 --2- 172.21.15.39:0/1506372783 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6bb810a3c0 0x7f6bb810a7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:24.845 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:24.841+0000 7f6bbfd2f640 1 --2- 172.21.15.39:0/1506372783 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6bb810f150 0x7f6bb810f560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:24.845 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:24.841+0000 7f6bbfd2f640 1 -- 172.21.15.39:0/1506372783 >> 172.21.15.39:0/1506372783 conn(0x7f6bb80712f0 msgr2=0x7f6bb8073e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:48:24.847 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:24.843+0000 7f6bbfd2f640 1 -- 172.21.15.39:0/1506372783 shutdown_connections 2024-09-21T03:48:24.847 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:24.844+0000 7f6bbfd2f640 1 -- 172.21.15.39:0/1506372783 wait complete. 2024-09-21T03:48:24.849 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:24.845+0000 7f6bbfd2f640 1 Processor -- start 2024-09-21T03:48:24.849 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:24.845+0000 7f6bbfd2f640 1 -- start start 2024-09-21T03:48:24.849 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:24.846+0000 7f6bbfd2f640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6bb810a3c0 0x7f6bb81a2e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:48:24.849 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:24.846+0000 7f6bbfd2f640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6bb810ad10 0x7f6bb81a33f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:48:24.849 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:24.846+0000 7f6bbfd2f640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6bb810f150 0x7f6bb81a8690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:48:24.850 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:24.846+0000 7f6bbfd2f640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f6bb811a190 con 0x7f6bb810ad10 2024-09-21T03:48:24.850 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:24.846+0000 7f6bbfd2f640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f6bb811a010 con 0x7f6bb810f150 2024-09-21T03:48:24.850 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:24.846+0000 7f6bbfd2f640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f6bb811a310 con 0x7f6bb810a3c0 2024-09-21T03:48:24.850 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:24.847+0000 7f6bbd2ca640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6bb810ad10 0x7f6bb81a33f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:48:24.850 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:24.847+0000 7f6bbd2ca640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6bb810ad10 0x7f6bb81a33f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:48792/0 (socket says 172.21.15.39:48792) 2024-09-21T03:48:24.850 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:24.847+0000 7f6bbd2ca640 1 -- 172.21.15.39:0/680507360 learned_addr learned my addr 172.21.15.39:0/680507360 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:48:24.850 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:24.847+0000 7f6bbdacb640 1 --2- 172.21.15.39:0/680507360 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6bb810a3c0 0x7f6bb81a2e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:48:24.851 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:24.847+0000 7f6bbd2ca640 1 -- 172.21.15.39:0/680507360 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6bb810a3c0 msgr2=0x7f6bb81a2e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:48:24.851 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:24.847+0000 7f6bbd2ca640 1 --2- 172.21.15.39:0/680507360 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6bb810a3c0 0x7f6bb81a2e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:24.851 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:24.848+0000 7f6bbd2ca640 1 -- 172.21.15.39:0/680507360 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6bb810f150 msgr2=0x7f6bb81a8690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:48:24.851 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:24.848+0000 7f6bbd2ca640 1 --2- 172.21.15.39:0/680507360 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6bb810f150 0x7f6bb81a8690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:24.851 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:24.848+0000 7f6bbd2ca640 1 -- 172.21.15.39:0/680507360 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6bb81a8bd0 con 0x7f6bb810ad10 2024-09-21T03:48:24.852 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:24.848+0000 7f6bbd2ca640 1 --2- 172.21.15.39:0/680507360 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6bb810ad10 0x7f6bb81a33f0 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f6bb400b450 tx=0x7f6bb400b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:48:24.852 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:24.849+0000 7f6baeffd640 1 -- 172.21.15.39:0/680507360 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6bb4015020 con 0x7f6bb810ad10 2024-09-21T03:48:24.852 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:24.849+0000 7f6baeffd640 1 -- 172.21.15.39:0/680507360 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6bb4011d40 con 0x7f6bb810ad10 2024-09-21T03:48:24.853 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:24.849+0000 7f6baeffd640 1 -- 172.21.15.39:0/680507360 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6bb4010de0 con 0x7f6bb810ad10 2024-09-21T03:48:24.853 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:24.849+0000 7f6bbfd2f640 1 -- 172.21.15.39:0/680507360 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6bb81a8ec0 con 0x7f6bb810ad10 2024-09-21T03:48:24.853 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:24.850+0000 7f6bbfd2f640 1 -- 172.21.15.39:0/680507360 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6bb81a9430 con 0x7f6bb810ad10 2024-09-21T03:48:24.858 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:24.855+0000 7f6bc62a3740 1 Processor -- start 2024-09-21T03:48:24.860 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:24.855+0000 7f6bc62a3740 1 -- start start 2024-09-21T03:48:24.860 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:24.856+0000 7f6bc62a3740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56274cb2a070 0x56274ced12c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:48:24.860 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:24.857+0000 7f6bc62a3740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56274cb011b0 0x56274ced1800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:48:24.862 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:24.858+0000 7f6bc62a3740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56274cb09390 0x56274ced1d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:48:24.862 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:24.858+0000 7f6bc62a3740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x56274ce47ea0 con 0x56274cb011b0 2024-09-21T03:48:24.862 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:24.858+0000 7f6bc62a3740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x56274ce372d0 con 0x56274cb09390 2024-09-21T03:48:24.862 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:24.858+0000 7f6bc62a3740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x56274ce3b0e0 con 0x56274cb2a070 2024-09-21T03:48:24.862 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:24.858+0000 7f6bbe2cc640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56274cb2a070 0x56274ced12c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:48:24.862 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:24.858+0000 7f6baeffd640 1 -- 172.21.15.39:0/680507360 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 27) ==== 154643+0+0 (secure 0 0 0) 0x7f6bb4018020 con 0x7f6bb810ad10 2024-09-21T03:48:24.863 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:24.858+0000 7f6bbdacb640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56274cb011b0 0x56274ced1800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:48:24.863 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:24.858+0000 7f6bbdacb640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56274cb011b0 0x56274ced1800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:48806/0 (socket says 172.21.15.39:48806) 2024-09-21T03:48:24.864 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:24.858+0000 7f6bbdacb640 1 -- 172.21.15.39:0/2157941430 learned_addr learned my addr 172.21.15.39:0/2157941430 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:48:24.864 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:24.859+0000 7f6bbdacb640 1 -- 172.21.15.39:0/2157941430 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56274cb2a070 msgr2=0x56274ced12c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:48:24.864 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:24.859+0000 7f6bbdacb640 1 --2- 172.21.15.39:0/2157941430 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56274cb2a070 0x56274ced12c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:24.864 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:24.859+0000 7f6bbd2ca640 1 --2- 172.21.15.39:0/2157941430 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56274cb09390 0x56274ced1d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:48:24.864 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:24.859+0000 7f6bbdacb640 1 -- 172.21.15.39:0/2157941430 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56274cb09390 msgr2=0x56274ced1d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:48:24.864 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:24.859+0000 7f6bbdacb640 1 --2- 172.21.15.39:0/2157941430 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56274cb09390 0x56274ced1d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:24.864 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:24.859+0000 7f6bbdacb640 1 -- 172.21.15.39:0/2157941430 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56274cb95650 con 0x56274cb011b0 2024-09-21T03:48:24.864 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:24.859+0000 7f6baeffd640 1 --2- 172.21.15.39:0/680507360 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f6ba40b6880 0x7f6ba40b8d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:48:24.865 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:24.859+0000 7f6bbdacb640 1 --2- 172.21.15.39:0/2157941430 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56274cb011b0 0x56274ced1800 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f6ba800b820 tx=0x7f6ba800bce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6eea4761f2418258 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:48:24.865 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:24.860+0000 7f6baeffd640 1 -- 172.21.15.39:0/680507360 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8484+0+0 (secure 0 0 0) 0x7f6bb40e70f0 con 0x7f6bb810ad10 2024-09-21T03:48:24.865 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:24.860+0000 7f6ba3fff640 1 -- 172.21.15.39:0/2157941430 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ba800d950 con 0x56274cb011b0 2024-09-21T03:48:24.865 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:24.860+0000 7f6ba3fff640 1 -- 172.21.15.39:0/2157941430 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6ba8019070 con 0x56274cb011b0 2024-09-21T03:48:24.866 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:24.860+0000 7f6bbe2cc640 1 --2- 172.21.15.39:0/680507360 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f6ba40b6880 0x7f6ba40b8d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:48:24.866 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:24.860+0000 7f6ba3fff640 1 -- 172.21.15.39:0/2157941430 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ba80123e0 con 0x56274cb011b0 2024-09-21T03:48:24.866 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:24.862+0000 7f6bbe2cc640 1 --2- 172.21.15.39:0/680507360 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f6ba40b6880 0x7f6ba40b8d90 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f6bb0035040 tx=0x7f6bb000d000 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:48:24.866 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:24.862+0000 7f6bc62a3740 1 -- 172.21.15.39:0/2157941430 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56274cb011b0 msgr2=0x56274ced1800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:48:24.866 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:24.862+0000 7f6bc62a3740 1 --2- 172.21.15.39:0/2157941430 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56274cb011b0 0x56274ced1800 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f6ba800b820 tx=0x7f6ba800bce0 comp rx=0 tx=0).stop 2024-09-21T03:48:24.867 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:24.863+0000 7f6bc62a3740 1 -- 172.21.15.39:0/2157941430 shutdown_connections 2024-09-21T03:48:24.867 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:24.863+0000 7f6bc62a3740 1 --2- 172.21.15.39:0/2157941430 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56274cb09390 0x56274ced1d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:24.867 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:24.863+0000 7f6bc62a3740 1 --2- 172.21.15.39:0/2157941430 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56274cb011b0 0x56274ced1800 unknown :-1 s=CLOSED pgs=624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:24.867 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:24.863+0000 7f6bc62a3740 1 --2- 172.21.15.39:0/2157941430 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56274cb2a070 0x56274ced12c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:24.867 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:24.863+0000 7f6bc62a3740 1 -- 172.21.15.39:0/2157941430 >> 172.21.15.39:0/2157941430 conn(0x56274cb1e460 msgr2=0x56274cecf1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:48:24.867 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:24.863+0000 7f6bc62a3740 1 -- 172.21.15.39:0/2157941430 shutdown_connections 2024-09-21T03:48:24.868 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:24.863+0000 7f6bc62a3740 1 -- 172.21.15.39:0/2157941430 wait complete. 2024-09-21T03:48:24.868 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:24.864+0000 7f6bc62a3740 1 -- start start 2024-09-21T03:48:24.868 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:24.865+0000 7f6bc62a3740 1 Processor -- start 2024-09-21T03:48:24.868 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:24.865+0000 7f6bc62a3740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56274cb011b0 0x56274cedca00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:48:24.869 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:24.865+0000 7f6bc62a3740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56274cb09390 0x56274cedcf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:48:24.869 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:24.866+0000 7f6bc62a3740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56274cb2a070 0x56274cee8d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:48:24.869 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:24.866+0000 7f6bc62a3740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x56274cc98700 con 0x56274cb2a070 2024-09-21T03:48:24.869 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:24.866+0000 7f6bc62a3740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x56274cdd2160 con 0x56274cb011b0 2024-09-21T03:48:24.870 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:24.866+0000 7f6bc62a3740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x56274cedfaa0 con 0x56274cb09390 2024-09-21T03:48:24.870 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:24.866+0000 7f6bc62a3740 10 client.0 _ll_register_callbacks cb 0x56274cdeb9e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:48:24.870 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:24.866+0000 7f6bbe2cc640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56274cb2a070 0x56274cee8d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:48:24.870 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:24.866+0000 7f6bbe2cc640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56274cb2a070 0x56274cee8d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:48816/0 (socket says 172.21.15.39:48816) 2024-09-21T03:48:24.870 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:24.866+0000 7f6bbe2cc640 1 -- 172.21.15.39:0/437922356 learned_addr learned my addr 172.21.15.39:0/437922356 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:48:24.870 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:24.866+0000 7f6bbe2cc640 1 -- 172.21.15.39:0/437922356 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56274cb09390 msgr2=0x56274cedcf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:48:24.870 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:24.866+0000 7f6bbe2cc640 1 --2- 172.21.15.39:0/437922356 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56274cb09390 0x56274cedcf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:24.870 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:24.866+0000 7f6bbe2cc640 1 -- 172.21.15.39:0/437922356 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56274cb011b0 msgr2=0x56274cedca00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:48:24.870 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:24.866+0000 7f6bbe2cc640 1 --2- 172.21.15.39:0/437922356 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56274cb011b0 0x56274cedca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:24.870 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:24.866+0000 7f6bbe2cc640 1 -- 172.21.15.39:0/437922356 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56274cdffc90 con 0x56274cb2a070 2024-09-21T03:48:24.871 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:24.867+0000 7f6bbe2cc640 1 --2- 172.21.15.39:0/437922356 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56274cb2a070 0x56274cee8d90 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f6bb0034000 tx=0x7f6bb002b930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a0c5c8ac2edeb7d2 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:48:24.871 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:24.867+0000 7f6ba17fa640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-09-21T03:48:24.871 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:24.867+0000 7f6ba17fa640 1 -- 172.21.15.39:0/437922356 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6bb0043040 con 0x56274cb2a070 2024-09-21T03:48:24.871 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:24.867+0000 7f6ba17fa640 1 -- 172.21.15.39:0/437922356 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8484+0+0 (secure 0 0 0) 0x7f6bb000a0d0 con 0x56274cb2a070 2024-09-21T03:48:24.871 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:24.868+0000 7f6bc62a3740 1 -- 172.21.15.39:0/437922356 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6ba8019070 con 0x56274cb2a070 2024-09-21T03:48:24.872 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:24.868+0000 7f6ba17fa640 1 -- 172.21.15.39:0/437922356 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f6bb0037040 con 0x56274cb2a070 2024-09-21T03:48:24.872 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:24.868+0000 7f6bc62a3740 10 client.16095 fetch_fsmap learned FSMap version 29 2024-09-21T03:48:24.872 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:24.868+0000 7f6bc62a3740 1 -- 172.21.15.39:0/437922356 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x56274ceeb830 con 0x56274cb2a070 2024-09-21T03:48:24.873 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:24.869+0000 7f6ba17fa640 1 -- 172.21.15.39:0/437922356 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6ba8019070 con 0x56274cb2a070 2024-09-21T03:48:24.873 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:24.869+0000 7f6ba17fa640 1 -- 172.21.15.39:0/437922356 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f6bb003a030 con 0x56274cb2a070 2024-09-21T03:48:24.874 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:24.869+0000 7f6bc62a3740 10 client.16095 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T03:48:24.874 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:24.869+0000 7f6bc62a3740 10 client.16095 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-21T03:48:24.874 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:24.869+0000 7f6bc62a3740 20 client.16095 populate_metadata read hostname 'smithi039' 2024-09-21T03:48:24.874 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:24.869+0000 7f6bc62a3740 1 --2- 172.21.15.39:0/437922356 >> [v2:172.21.15.161:6838/2190107270,v1:172.21.15.161:6839/2190107270] conn(0x56274cae89f0 0x56274ceeeee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:48:24.874 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:24.869+0000 7f6bc62a3740 4 client.16095 mds_command: new command op to 24667 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:48:24.874 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:24.869+0000 7f6bc62a3740 1 -- 172.21.15.39:0/437922356 --> [v2:172.21.15.161:6838/2190107270,v1:172.21.15.161:6839/2190107270] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56274ce49df0 con 0x56274cae89f0 2024-09-21T03:48:24.874 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:24.870+0000 7f6bbdacb640 1 --2- 172.21.15.39:0/437922356 >> [v2:172.21.15.161:6838/2190107270,v1:172.21.15.161:6839/2190107270] conn(0x56274cae89f0 0x56274ceeeee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:48:24.874 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:24.871+0000 7f6bbdacb640 1 --2- 172.21.15.39:0/437922356 >> [v2:172.21.15.161:6838/2190107270,v1:172.21.15.161:6839/2190107270] conn(0x56274cae89f0 0x56274ceeeee0 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=39346873da9a4b82 server_cookie=90b66a15ad060ab2 in_seq=0 out_seq=0 2024-09-21T03:48:24.874 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:24.871+0000 7f6ba17fa640 10 client.16095 ms_handle_connect on v2:172.21.15.161:6838/2190107270 2024-09-21T03:48:24.877 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:24.874+0000 7f6ba17fa640 1 -- 172.21.15.39:0/437922356 <== mds.1 v2:172.21.15.161:6838/2190107270 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x56274ce49df0 con 0x56274cae89f0 2024-09-21T03:48:24.877 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:24.874+0000 7f6ba17fa640 10 client.16095 handle_command_reply: tid=0 2024-09-21T03:48:24.878 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:24.874+0000 7f6bc62a3740 1 client.16095 shutdown 2024-09-21T03:48:24.878 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:24.874+0000 7f6bc62a3740 1 -- 172.21.15.39:0/437922356 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56274cb2a070 msgr2=0x56274cee8d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:48:24.878 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:24.874+0000 7f6bc62a3740 1 --2- 172.21.15.39:0/437922356 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56274cb2a070 0x56274cee8d90 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f6bb0034000 tx=0x7f6bb002b930 comp rx=0 tx=0).stop 2024-09-21T03:48:24.878 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:24.874+0000 7f6bc62a3740 1 -- 172.21.15.39:0/437922356 shutdown_connections 2024-09-21T03:48:24.878 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:24.874+0000 7f6bc62a3740 1 --2- 172.21.15.39:0/437922356 >> [v2:172.21.15.161:6838/2190107270,v1:172.21.15.161:6839/2190107270] conn(0x56274cae89f0 0x56274ceeeee0 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:24.878 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:24.874+0000 7f6ba17fa640 0 client.16095 ms_handle_reset on v2:172.21.15.161:6838/2190107270 2024-09-21T03:48:24.878 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:24.874+0000 7f6bc62a3740 1 --2- 172.21.15.39:0/437922356 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56274cb2a070 0x56274cee8d90 unknown :-1 s=CLOSED pgs=625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:24.879 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:24.874+0000 7f6bc62a3740 1 --2- 172.21.15.39:0/437922356 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56274cb09390 0x56274cedcf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:24.879 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:24.874+0000 7f6bc62a3740 1 --2- 172.21.15.39:0/437922356 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56274cb011b0 0x56274cedca00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:24.879 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:24.874+0000 7f6bc62a3740 1 -- 172.21.15.39:0/437922356 >> 172.21.15.39:0/437922356 conn(0x56274cb1e460 msgr2=0x56274ced4450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:48:24.879 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:24.874+0000 7f6bc62a3740 1 -- 172.21.15.39:0/437922356 shutdown_connections 2024-09-21T03:48:24.879 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:24.874+0000 7f6bc62a3740 1 -- 172.21.15.39:0/437922356 wait complete. 2024-09-21T03:48:24.879 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:24.875+0000 7f6bc62a3740 20 client.16095 trim_cache size 0 max 16384 2024-09-21T03:48:24.900 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:24.897+0000 7f6bc62a3740 1 Processor -- start 2024-09-21T03:48:24.900 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:24.897+0000 7f6bc62a3740 1 -- start start 2024-09-21T03:48:24.900 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:24.897+0000 7f6bc62a3740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56274cb011b0 0x56274cedcec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:48:24.901 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:24.898+0000 7f6bc62a3740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56274cb09390 0x56274ced43c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:48:24.901 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:24.898+0000 7f6bc62a3740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56274cb2a070 0x56274ced4900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:48:24.901 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:24.898+0000 7f6bc62a3740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f6bb81be720 con 0x56274cb2a070 2024-09-21T03:48:24.902 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:24.898+0000 7f6bc62a3740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f6bb81c0260 con 0x56274cb011b0 2024-09-21T03:48:24.902 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:24.898+0000 7f6bc62a3740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f6bb81bedf0 con 0x56274cb09390 2024-09-21T03:48:24.903 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:24.898+0000 7f6bbe2cc640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56274cb2a070 0x56274ced4900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:48:24.903 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:24.898+0000 7f6bbe2cc640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56274cb2a070 0x56274ced4900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:48828/0 (socket says 172.21.15.39:48828) 2024-09-21T03:48:24.903 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:24.898+0000 7f6bbe2cc640 1 -- 172.21.15.39:0/265775659 learned_addr learned my addr 172.21.15.39:0/265775659 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:48:24.903 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:24.898+0000 7f6bbdacb640 1 --2- 172.21.15.39:0/265775659 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56274cb011b0 0x56274cedcec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:48:24.903 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:24.900+0000 7f6bbe2cc640 1 -- 172.21.15.39:0/265775659 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56274cb09390 msgr2=0x56274ced43c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:48:24.903 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:24.900+0000 7f6bbe2cc640 1 --2- 172.21.15.39:0/265775659 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56274cb09390 0x56274ced43c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:24.903 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:24.900+0000 7f6bbe2cc640 1 -- 172.21.15.39:0/265775659 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56274cb011b0 msgr2=0x56274cedcec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:48:24.903 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:24.900+0000 7f6bbe2cc640 1 --2- 172.21.15.39:0/265775659 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56274cb011b0 0x56274cedcec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:24.903 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:24.900+0000 7f6bbe2cc640 1 -- 172.21.15.39:0/265775659 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56274cecfb20 con 0x56274cb2a070 2024-09-21T03:48:24.904 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:24.901+0000 7f6bbe2cc640 1 --2- 172.21.15.39:0/265775659 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56274cb2a070 0x56274ced4900 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f6bb002ba20 tx=0x7f6bb002bbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ebe8977636c443b server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:48:24.905 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:24.902+0000 7f6ba37fe640 1 -- 172.21.15.39:0/265775659 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6bb005e070 con 0x56274cb2a070 2024-09-21T03:48:24.905 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:24.902+0000 7f6ba37fe640 1 -- 172.21.15.39:0/265775659 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6bb004c070 con 0x56274cb2a070 2024-09-21T03:48:24.905 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:24.902+0000 7f6ba37fe640 1 -- 172.21.15.39:0/265775659 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6bb0043070 con 0x56274cb2a070 2024-09-21T03:48:24.908 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:24.905+0000 7f6bc62a3740 1 -- 172.21.15.39:0/265775659 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56274cb2a070 msgr2=0x56274ced4900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:48:24.908 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:24.905+0000 7f6bc62a3740 1 --2- 172.21.15.39:0/265775659 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56274cb2a070 0x56274ced4900 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f6bb002ba20 tx=0x7f6bb002bbe0 comp rx=0 tx=0).stop 2024-09-21T03:48:24.908 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:24.905+0000 7f6bc62a3740 1 -- 172.21.15.39:0/265775659 shutdown_connections 2024-09-21T03:48:24.908 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:24.905+0000 7f6bc62a3740 1 --2- 172.21.15.39:0/265775659 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56274cb2a070 0x56274ced4900 unknown :-1 s=CLOSED pgs=626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:24.908 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:24.905+0000 7f6bc62a3740 1 --2- 172.21.15.39:0/265775659 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56274cb09390 0x56274ced43c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:24.908 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:24.905+0000 7f6bc62a3740 1 --2- 172.21.15.39:0/265775659 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56274cb011b0 0x56274cedcec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:24.909 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:24.905+0000 7f6bc62a3740 1 -- 172.21.15.39:0/265775659 >> 172.21.15.39:0/265775659 conn(0x56274cb1e460 msgr2=0x56274cedc7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:48:24.909 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:24.905+0000 7f6bc62a3740 1 -- 172.21.15.39:0/265775659 shutdown_connections 2024-09-21T03:48:24.909 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:24.905+0000 7f6bc62a3740 1 -- 172.21.15.39:0/265775659 wait complete. 2024-09-21T03:48:24.909 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:24.906+0000 7f6bc62a3740 1 -- start start 2024-09-21T03:48:24.909 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:24.906+0000 7f6bc62a3740 1 Processor -- start 2024-09-21T03:48:24.912 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:24.907+0000 7f6bc62a3740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56274cb011b0 0x56274cee0e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:48:24.912 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:24.907+0000 7f6bc62a3740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56274cb09390 0x56274cee1370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:48:24.912 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:24.907+0000 7f6bc62a3740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56274cb2a070 0x56274cee18b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:48:24.912 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:24.907+0000 7f6bc62a3740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f6bb81bfb90 con 0x56274cb2a070 2024-09-21T03:48:24.912 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:24.907+0000 7f6bc62a3740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f6bb81bf4c0 con 0x56274cb011b0 2024-09-21T03:48:24.912 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:24.907+0000 7f6bc62a3740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f6bb81bc700 con 0x56274cb09390 2024-09-21T03:48:24.912 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:24.907+0000 7f6bc62a3740 10 client.0 _ll_register_callbacks cb 0x7f6bb0040040 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:48:24.912 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:24.907+0000 7f6bbdacb640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56274cb011b0 0x56274cee0e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:48:24.912 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:24.908+0000 7f6bbe2cc640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56274cb2a070 0x56274cee18b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:48:24.912 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:24.908+0000 7f6bbdacb640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56274cb011b0 0x56274cee0e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.39:41146/0 (socket says 172.21.15.39:41146) 2024-09-21T03:48:24.912 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:24.908+0000 7f6bbdacb640 1 -- 172.21.15.39:0/1765067488 learned_addr learned my addr 172.21.15.39:0/1765067488 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:48:24.912 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:24.908+0000 7f6bbdacb640 1 -- 172.21.15.39:0/1765067488 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56274cb09390 msgr2=0x56274cee1370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:48:24.913 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:24.908+0000 7f6bbdacb640 1 --2- 172.21.15.39:0/1765067488 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56274cb09390 0x56274cee1370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:24.913 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:24.908+0000 7f6bbdacb640 1 -- 172.21.15.39:0/1765067488 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56274cb2a070 msgr2=0x56274cee18b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:48:24.913 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:24.908+0000 7f6bbdacb640 1 --2- 172.21.15.39:0/1765067488 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56274cb2a070 0x56274cee18b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:24.913 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:24.908+0000 7f6bbdacb640 1 -- 172.21.15.39:0/1765067488 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56274cf58a40 con 0x56274cb011b0 2024-09-21T03:48:24.913 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:24.908+0000 7f6bbe2cc640 1 --2- 172.21.15.39:0/1765067488 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56274cb2a070 0x56274cee18b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:48:24.913 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:24.908+0000 7f6bbdacb640 1 --2- 172.21.15.39:0/1765067488 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56274cb011b0 0x56274cee0e30 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x56274cecff40 tx=0x7f6ba80149c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a21a6a8a602b240 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:48:24.913 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:24.908+0000 7f6ba17fa640 10 client.0 ms_handle_connect on v2:172.21.15.150:3300/0 2024-09-21T03:48:24.913 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:24.909+0000 7f6ba17fa640 1 -- 172.21.15.39:0/1765067488 <== mon.2 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ba800d950 con 0x56274cb011b0 2024-09-21T03:48:24.913 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:24.909+0000 7f6ba17fa640 1 -- 172.21.15.39:0/1765067488 <== mon.2 v2:172.21.15.150:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8484+0+0 (secure 0 0 0) 0x7f6ba800daf0 con 0x56274cb011b0 2024-09-21T03:48:24.913 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:24.909+0000 7f6bc62a3740 1 -- 172.21.15.39:0/1765067488 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6bb004c070 con 0x56274cb011b0 2024-09-21T03:48:24.913 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:24.909+0000 7f6ba17fa640 1 -- 172.21.15.39:0/1765067488 <== mon.2 v2:172.21.15.150:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f6ba800dcf0 con 0x56274cb011b0 2024-09-21T03:48:24.913 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:24.909+0000 7f6bc62a3740 10 client.25652 fetch_fsmap learned FSMap version 29 2024-09-21T03:48:24.913 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:24.909+0000 7f6bc62a3740 1 -- 172.21.15.39:0/1765067488 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x56274cf59cc0 con 0x56274cb011b0 2024-09-21T03:48:24.914 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:24.910+0000 7f6ba17fa640 1 -- 172.21.15.39:0/1765067488 <== mon.2 v2:172.21.15.150:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6bb004c070 con 0x56274cb011b0 2024-09-21T03:48:24.914 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:24.910+0000 7f6ba17fa640 1 -- 172.21.15.39:0/1765067488 <== mon.2 v2:172.21.15.150:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f6ba8013070 con 0x56274cb011b0 2024-09-21T03:48:24.914 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:24.910+0000 7f6bc62a3740 10 client.25652 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T03:48:24.915 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:24.910+0000 7f6bc62a3740 10 client.25652 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-21T03:48:24.915 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:24.910+0000 7f6bc62a3740 20 client.25652 populate_metadata read hostname 'smithi039' 2024-09-21T03:48:24.915 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:24.910+0000 7f6bc62a3740 1 --2- 172.21.15.39:0/1765067488 >> [v2:172.21.15.161:6838/2190107270,v1:172.21.15.161:6839/2190107270] conn(0x56274cabdda0 0x56274cf5c0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:48:24.915 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:24.910+0000 7f6bc62a3740 4 client.25652 mds_command: new command op to 24667 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T03:48:24.915 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:24.910+0000 7f6bc62a3740 1 -- 172.21.15.39:0/1765067488 --> [v2:172.21.15.161:6838/2190107270,v1:172.21.15.161:6839/2190107270] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56274ce458a0 con 0x56274cabdda0 2024-09-21T03:48:24.915 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:24.911+0000 7f6bbd2ca640 1 --2- 172.21.15.39:0/1765067488 >> [v2:172.21.15.161:6838/2190107270,v1:172.21.15.161:6839/2190107270] conn(0x56274cabdda0 0x56274cf5c0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:48:24.916 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:24.913+0000 7f6bbd2ca640 1 --2- 172.21.15.39:0/1765067488 >> [v2:172.21.15.161:6838/2190107270,v1:172.21.15.161:6839/2190107270] conn(0x56274cabdda0 0x56274cf5c0b0 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=94bb55235aba2ade server_cookie=9f2032f5f9650be0 in_seq=0 out_seq=0 2024-09-21T03:48:24.916 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:24.914+0000 7f6ba17fa640 10 client.25652 ms_handle_connect on v2:172.21.15.161:6838/2190107270 2024-09-21T03:48:24.917 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:24.914+0000 7f6ba17fa640 1 -- 172.21.15.39:0/1765067488 <== mds.1 v2:172.21.15.161:6838/2190107270 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56274ce458a0 con 0x56274cabdda0 2024-09-21T03:48:24.917 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:24.914+0000 7f6ba17fa640 10 client.25652 handle_command_reply: tid=0 2024-09-21T03:48:24.917 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:24.914+0000 7f6bc62a3740 1 client.25652 shutdown 2024-09-21T03:48:24.918 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:24.915+0000 7f6bc62a3740 1 -- 172.21.15.39:0/1765067488 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56274cb011b0 msgr2=0x56274cee0e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:48:24.918 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:24.915+0000 7f6bc62a3740 1 --2- 172.21.15.39:0/1765067488 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56274cb011b0 0x56274cee0e30 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x56274cecff40 tx=0x7f6ba80149c0 comp rx=0 tx=0).stop 2024-09-21T03:48:24.918 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:24.916+0000 7f6bc62a3740 1 -- 172.21.15.39:0/1765067488 shutdown_connections 2024-09-21T03:48:24.918 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:24.916+0000 7f6bc62a3740 1 --2- 172.21.15.39:0/1765067488 >> [v2:172.21.15.161:6838/2190107270,v1:172.21.15.161:6839/2190107270] conn(0x56274cabdda0 0x56274cf5c0b0 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:24.919 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:24.916+0000 7f6bc62a3740 1 --2- 172.21.15.39:0/1765067488 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56274cb2a070 0x56274cee18b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:24.919 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:24.916+0000 7f6bc62a3740 1 --2- 172.21.15.39:0/1765067488 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56274cb09390 0x56274cee1370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:24.919 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:24.916+0000 7f6bc62a3740 1 --2- 172.21.15.39:0/1765067488 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56274cb011b0 0x56274cee0e30 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:24.919 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:24.916+0000 7f6bc62a3740 1 -- 172.21.15.39:0/1765067488 >> 172.21.15.39:0/1765067488 conn(0x56274cb1e460 msgr2=0x56274ced43c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:48:24.919 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:24.916+0000 7f6ba17fa640 0 client.25652 ms_handle_reset on v2:172.21.15.161:6838/2190107270 2024-09-21T03:48:24.919 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:24.916+0000 7f6bc62a3740 1 -- 172.21.15.39:0/1765067488 shutdown_connections 2024-09-21T03:48:24.919 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:24.916+0000 7f6bc62a3740 1 -- 172.21.15.39:0/1765067488 wait complete. 2024-09-21T03:48:24.920 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:24.916+0000 7f6bc62a3740 20 client.25652 trim_cache size 0 max 16384 2024-09-21T03:48:24.920 INFO:teuthology.orchestra.run.smithi039.stdout:[] 2024-09-21T03:48:24.921 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:24.917+0000 7f6bbfd2f640 1 -- 172.21.15.39:0/680507360 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f6ba40b6880 msgr2=0x7f6ba40b8d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:48:24.921 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:24.918+0000 7f6bbfd2f640 1 --2- 172.21.15.39:0/680507360 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f6ba40b6880 0x7f6ba40b8d90 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f6bb0035040 tx=0x7f6bb000d000 comp rx=0 tx=0).stop 2024-09-21T03:48:24.921 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:24.918+0000 7f6bbfd2f640 1 -- 172.21.15.39:0/680507360 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6bb810ad10 msgr2=0x7f6bb81a33f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:48:24.921 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:24.918+0000 7f6bbfd2f640 1 --2- 172.21.15.39:0/680507360 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6bb810ad10 0x7f6bb81a33f0 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f6bb400b450 tx=0x7f6bb400b910 comp rx=0 tx=0).stop 2024-09-21T03:48:24.921 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:24.918+0000 7f6bbfd2f640 1 -- 172.21.15.39:0/680507360 shutdown_connections 2024-09-21T03:48:24.921 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:24.918+0000 7f6bbfd2f640 1 --2- 172.21.15.39:0/680507360 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f6ba40b6880 0x7f6ba40b8d90 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:24.921 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:24.918+0000 7f6bbfd2f640 1 --2- 172.21.15.39:0/680507360 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6bb810f150 0x7f6bb81a8690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:24.921 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:24.918+0000 7f6bbfd2f640 1 --2- 172.21.15.39:0/680507360 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6bb810ad10 0x7f6bb81a33f0 unknown :-1 s=CLOSED pgs=623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:24.922 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:24.918+0000 7f6bbfd2f640 1 --2- 172.21.15.39:0/680507360 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6bb810a3c0 0x7f6bb81a2e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:24.922 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:24.918+0000 7f6bbfd2f640 1 -- 172.21.15.39:0/680507360 >> 172.21.15.39:0/680507360 conn(0x7f6bb80712f0 msgr2=0x7f6bb810c410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:48:24.922 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:24.918+0000 7f6bbfd2f640 1 -- 172.21.15.39:0/680507360 shutdown_connections 2024-09-21T03:48:24.922 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:24.918+0000 7f6bbfd2f640 1 -- 172.21.15.39:0/680507360 wait complete. 2024-09-21T03:48:25.173 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:48:25 smithi039 ceph-mon[31211]: pgmap v396: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 177 KiB/s rd, 639 MiB/s wr, 228 op/s 2024-09-21T03:48:25.173 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:48:25 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:48:25.467 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:48:25 smithi161 ceph-mon[34533]: pgmap v396: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 177 KiB/s rd, 639 MiB/s wr, 228 op/s 2024-09-21T03:48:25.468 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:48:25 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:48:25.474 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:48:25 smithi150 ceph-mon[34289]: pgmap v396: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 177 KiB/s rd, 639 MiB/s wr, 228 op/s 2024-09-21T03:48:25.475 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:48:25 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:48:26.338 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 51116d5c-77ca-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 damage ls 2024-09-21T03:48:27.428 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/51116d5c-77ca-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:48:27.834 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:48:27 smithi039 ceph-mon[31211]: pgmap v397: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 294 KiB/s rd, 655 MiB/s wr, 270 op/s 2024-09-21T03:48:27.834 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:48:27 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:48:27.834 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:48:27 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:48:27.834 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:48:27 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:48:27.974 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:48:27 smithi150 ceph-mon[34289]: pgmap v397: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 294 KiB/s rd, 655 MiB/s wr, 270 op/s 2024-09-21T03:48:27.975 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:48:27 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:48:27.975 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:48:27 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:48:27.975 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:48:27 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:48:28.218 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:48:27 smithi161 ceph-mon[34533]: pgmap v397: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 294 KiB/s rd, 655 MiB/s wr, 270 op/s 2024-09-21T03:48:28.218 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:48:27 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:48:28.218 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:48:27 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:48:28.218 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:48:27 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:48:28.302 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:28.299+0000 7f10b32d1640 1 -- 172.21.15.39:0/3179225532 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f10ac10fcf0 msgr2=0x7f10ac110100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:48:28.303 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:28.299+0000 7f10b32d1640 1 --2- 172.21.15.39:0/3179225532 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f10ac10fcf0 0x7f10ac110100 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f10a8007380 tx=0x7f10a802aff0 comp rx=0 tx=0).stop 2024-09-21T03:48:28.303 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:28.300+0000 7f10b32d1640 1 -- 172.21.15.39:0/3179225532 shutdown_connections 2024-09-21T03:48:28.303 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:28.300+0000 7f10b32d1640 1 --2- 172.21.15.39:0/3179225532 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f10ac110640 0x7f10ac116e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:28.304 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:28.300+0000 7f10b32d1640 1 --2- 172.21.15.39:0/3179225532 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f10ac10fcf0 0x7f10ac110100 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:28.304 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:28.300+0000 7f10b32d1640 1 --2- 172.21.15.39:0/3179225532 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f10ac10a650 0x7f10ac10aa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:28.304 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:28.300+0000 7f10b32d1640 1 -- 172.21.15.39:0/3179225532 >> 172.21.15.39:0/3179225532 conn(0x7f10ac06ee50 msgr2=0x7f10ac06f290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:48:28.306 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:28.303+0000 7f10b32d1640 1 -- 172.21.15.39:0/3179225532 shutdown_connections 2024-09-21T03:48:28.313 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:28.303+0000 7f10b32d1640 1 -- 172.21.15.39:0/3179225532 wait complete. 2024-09-21T03:48:28.313 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:28.305+0000 7f10b32d1640 1 Processor -- start 2024-09-21T03:48:28.313 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:28.305+0000 7f10b32d1640 1 -- start start 2024-09-21T03:48:28.313 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:28.305+0000 7f10b32d1640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f10ac10a650 0x7f10ac1a2070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:48:28.313 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:28.305+0000 7f10b32d1640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f10ac10fcf0 0x7f10ac1a0660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:48:28.314 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:28.305+0000 7f10b32d1640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f10ac110640 0x7f10ac1a0c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:48:28.314 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:28.305+0000 7f10b32d1640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f10ac119ec0 con 0x7f10ac110640 2024-09-21T03:48:28.314 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:28.305+0000 7f10b32d1640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f10ac119d40 con 0x7f10ac10a650 2024-09-21T03:48:28.314 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:28.305+0000 7f10b32d1640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f10ac11a040 con 0x7f10ac10fcf0 2024-09-21T03:48:28.314 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:28.308+0000 7f10b22cf640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f10ac10a650 0x7f10ac1a2070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:48:28.314 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:28.308+0000 7f10b1ace640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f10ac10fcf0 0x7f10ac1a0660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:48:28.314 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:28.308+0000 7f10b22cf640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f10ac10a650 0x7f10ac1a2070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.39:52934/0 (socket says 172.21.15.39:52934) 2024-09-21T03:48:28.314 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:28.308+0000 7f10b22cf640 1 -- 172.21.15.39:0/270319611 learned_addr learned my addr 172.21.15.39:0/270319611 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:48:28.314 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:28.310+0000 7f10b22cf640 1 -- 172.21.15.39:0/270319611 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f10ac10fcf0 msgr2=0x7f10ac1a0660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:48:28.314 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:28.310+0000 7f10b22cf640 1 --2- 172.21.15.39:0/270319611 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f10ac10fcf0 0x7f10ac1a0660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:28.314 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:28.310+0000 7f10b22cf640 1 -- 172.21.15.39:0/270319611 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f10ac110640 msgr2=0x7f10ac1a0c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:48:28.314 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:28.310+0000 7f10b22cf640 1 --2- 172.21.15.39:0/270319611 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f10ac110640 0x7f10ac1a0c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:28.315 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:28.310+0000 7f10b22cf640 1 -- 172.21.15.39:0/270319611 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f10ac1a1440 con 0x7f10ac10a650 2024-09-21T03:48:28.315 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:28.311+0000 7f10b22cf640 1 --2- 172.21.15.39:0/270319611 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f10ac10a650 0x7f10ac1a2070 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f10a400ed80 tx=0x7f10a400c6e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:48:28.318 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:28.312+0000 7f10a37fe640 1 -- 172.21.15.39:0/270319611 <== mon.2 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10a401c070 con 0x7f10ac10a650 2024-09-21T03:48:28.319 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:28.312+0000 7f10b32d1640 1 -- 172.21.15.39:0/270319611 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f10ac1a6c80 con 0x7f10ac10a650 2024-09-21T03:48:28.319 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:28.312+0000 7f10b32d1640 1 -- 172.21.15.39:0/270319611 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f10ac1a7190 con 0x7f10ac10a650 2024-09-21T03:48:28.320 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:28.317+0000 7f10baaa7740 1 Processor -- start 2024-09-21T03:48:28.322 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:28.318+0000 7f10baaa7740 1 -- start start 2024-09-21T03:48:28.322 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:28.318+0000 7f10baaa7740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5558d8de4070 0x5558d918bb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:48:28.322 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:28.319+0000 7f10baaa7740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5558d8dbb1b0 0x5558d918c090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:48:28.322 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:28.319+0000 7f10baaa7740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5558d8dc3390 0x5558d918c5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:48:28.322 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:28.319+0000 7f10baaa7740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x5558d9101ea0 con 0x5558d8dbb1b0 2024-09-21T03:48:28.322 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:28.319+0000 7f10baaa7740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x5558d90f12d0 con 0x5558d8de4070 2024-09-21T03:48:28.323 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:28.319+0000 7f10baaa7740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x5558d90f50e0 con 0x5558d8dc3390 2024-09-21T03:48:28.323 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:28.320+0000 7f10b2ad0640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5558d8de4070 0x5558d918bb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:48:28.323 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:28.320+0000 7f10b2ad0640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5558d8de4070 0x5558d918bb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.39:52936/0 (socket says 172.21.15.39:52936) 2024-09-21T03:48:28.323 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:28.320+0000 7f10b2ad0640 1 -- 172.21.15.39:0/2393601754 learned_addr learned my addr 172.21.15.39:0/2393601754 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:48:28.337 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:28.320+0000 7f10b22cf640 1 --2- 172.21.15.39:0/2393601754 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5558d8dbb1b0 0x5558d918c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:48:28.337 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:28.323+0000 7f10b22cf640 1 -- 172.21.15.39:0/2393601754 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5558d8dc3390 msgr2=0x5558d918c5d0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-21T03:48:28.337 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:28.323+0000 7f10b22cf640 1 --2- 172.21.15.39:0/2393601754 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5558d8dc3390 0x5558d918c5d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:28.337 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:28.323+0000 7f10b22cf640 1 -- 172.21.15.39:0/2393601754 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5558d8de4070 msgr2=0x5558d918bb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:48:28.337 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:28.323+0000 7f10b22cf640 1 --2- 172.21.15.39:0/2393601754 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5558d8de4070 0x5558d918bb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:28.337 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:28.323+0000 7f10b22cf640 1 -- 172.21.15.39:0/2393601754 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5558d8e4f650 con 0x5558d8dbb1b0 2024-09-21T03:48:28.337 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:28.323+0000 7f10b2ad0640 1 --2- 172.21.15.39:0/2393601754 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5558d8de4070 0x5558d918bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:48:28.337 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:28.324+0000 7f10a37fe640 1 -- 172.21.15.39:0/270319611 <== mon.2 v2:172.21.15.150:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f10a4017af0 con 0x7f10ac10a650 2024-09-21T03:48:28.338 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:28.324+0000 7f10a37fe640 1 -- 172.21.15.39:0/270319611 <== mon.2 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10a400fb70 con 0x7f10ac10a650 2024-09-21T03:48:28.338 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:28.324+0000 7f10a37fe640 1 -- 172.21.15.39:0/270319611 <== mon.2 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 27) ==== 154643+0+0 (secure 0 0 0) 0x7f10a401f070 con 0x7f10ac10a650 2024-09-21T03:48:28.338 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:28.326+0000 7f10b22cf640 1 --2- 172.21.15.39:0/2393601754 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5558d8dbb1b0 0x5558d918c090 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f10a40eb6b0 tx=0x7f10a40f0960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a7aa4c3e9c44978 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:48:28.338 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:28.326+0000 7f10a37fe640 1 --2- 172.21.15.39:0/270319611 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f10980b68b0 0x7f10980b8dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:48:28.338 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:28.327+0000 7f10b2ad0640 1 --2- 172.21.15.39:0/270319611 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f10980b68b0 0x7f10980b8dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:48:28.338 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:28.328+0000 7f10b2ad0640 1 --2- 172.21.15.39:0/270319611 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f10980b68b0 0x7f10980b8dc0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f109c005fd0 tx=0x7f109c00b380 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:48:28.338 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:28.328+0000 7f10a37fe640 1 -- 172.21.15.39:0/270319611 <== mon.2 v2:172.21.15.150:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8484+0+0 (secure 0 0 0) 0x7f10a40edf20 con 0x7f10ac10a650 2024-09-21T03:48:28.346 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:28.343+0000 7f1097fff640 1 -- 172.21.15.39:0/2393601754 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10a40ff070 con 0x5558d8dbb1b0 2024-09-21T03:48:28.346 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:28.343+0000 7f1097fff640 1 -- 172.21.15.39:0/2393601754 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f10a4017ca0 con 0x5558d8dbb1b0 2024-09-21T03:48:28.348 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:28.345+0000 7f10baaa7740 1 -- 172.21.15.39:0/2393601754 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5558d8dbb1b0 msgr2=0x5558d918c090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:48:28.348 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:28.345+0000 7f10baaa7740 1 --2- 172.21.15.39:0/2393601754 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5558d8dbb1b0 0x5558d918c090 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f10a40eb6b0 tx=0x7f10a40f0960 comp rx=0 tx=0).stop 2024-09-21T03:48:28.350 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:28.347+0000 7f10baaa7740 1 -- 172.21.15.39:0/2393601754 shutdown_connections 2024-09-21T03:48:28.350 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:28.347+0000 7f10baaa7740 1 --2- 172.21.15.39:0/2393601754 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5558d8dc3390 0x5558d918c5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:28.350 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:28.347+0000 7f10baaa7740 1 --2- 172.21.15.39:0/2393601754 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5558d8dbb1b0 0x5558d918c090 secure :-1 s=CLOSED pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f10a40eb6b0 tx=0x7f10a40f0960 comp rx=0 tx=0).stop 2024-09-21T03:48:28.351 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:28.347+0000 7f10baaa7740 1 --2- 172.21.15.39:0/2393601754 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5558d8de4070 0x5558d918bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:28.351 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:28.347+0000 7f10baaa7740 1 -- 172.21.15.39:0/2393601754 >> 172.21.15.39:0/2393601754 conn(0x5558d8dd8460 msgr2=0x5558d9189a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:48:28.351 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:28.349+0000 7f10baaa7740 1 -- 172.21.15.39:0/2393601754 shutdown_connections 2024-09-21T03:48:28.353 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:28.349+0000 7f10baaa7740 1 -- 172.21.15.39:0/2393601754 wait complete. 2024-09-21T03:48:28.353 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:28.350+0000 7f10baaa7740 1 -- start start 2024-09-21T03:48:28.353 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:28.350+0000 7f10baaa7740 1 Processor -- start 2024-09-21T03:48:28.355 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:28.351+0000 7f10baaa7740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5558d8dc3390 0x5558d91971d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:48:28.355 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:28.352+0000 7f10baaa7740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5558d8de4070 0x5558d9197710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:48:28.355 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:28.352+0000 7f10baaa7740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5558d8da29f0 0x5558d9197c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:48:28.355 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:28.352+0000 7f10b22cf640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5558d8dc3390 0x5558d91971d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:48:28.355 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:28.352+0000 7f10b22cf640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5558d8dc3390 0x5558d91971d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:41088/0 (socket says 172.21.15.39:41088) 2024-09-21T03:48:28.355 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:28.352+0000 7f10b22cf640 1 -- 172.21.15.39:0/3206030069 learned_addr learned my addr 172.21.15.39:0/3206030069 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:48:28.357 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:28.352+0000 7f10baaa7740 1 -- 172.21.15.39:0/3206030069 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x5558d8f52700 con 0x5558d8dc3390 2024-09-21T03:48:28.358 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:28.355+0000 7f10b2ad0640 1 --2- 172.21.15.39:0/3206030069 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5558d8da29f0 0x5558d9197c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:48:28.358 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:28.356+0000 7f10baaa7740 1 -- 172.21.15.39:0/3206030069 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x5558d908c160 con 0x5558d8da29f0 2024-09-21T03:48:28.359 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:28.356+0000 7f10baaa7740 1 -- 172.21.15.39:0/3206030069 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x5558d919a2a0 con 0x5558d8de4070 2024-09-21T03:48:28.359 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:28.356+0000 7f10baaa7740 10 client.0 _ll_register_callbacks cb 0x5558d90a59e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:48:28.360 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:28.358+0000 7f10b1ace640 1 --2- 172.21.15.39:0/3206030069 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5558d8de4070 0x5558d9197710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:48:28.361 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:28.358+0000 7f10b2ad0640 1 -- 172.21.15.39:0/3206030069 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5558d8de4070 msgr2=0x5558d9197710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:48:28.361 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:28.358+0000 7f10b2ad0640 1 --2- 172.21.15.39:0/3206030069 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5558d8de4070 0x5558d9197710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:28.361 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:28.358+0000 7f10b2ad0640 1 -- 172.21.15.39:0/3206030069 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5558d8dc3390 msgr2=0x5558d91971d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:48:28.362 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:28.358+0000 7f10b2ad0640 1 --2- 172.21.15.39:0/3206030069 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5558d8dc3390 0x5558d91971d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:28.362 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:28.358+0000 7f10b2ad0640 1 -- 172.21.15.39:0/3206030069 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5558d90b9c90 con 0x5558d8da29f0 2024-09-21T03:48:28.362 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:28.359+0000 7f10b2ad0640 1 --2- 172.21.15.39:0/3206030069 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5558d8da29f0 0x5558d9197c70 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f109c011010 tx=0x7f109c0046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9d6e44beef10093d server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:48:28.364 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:28.360+0000 7f1095ffb640 10 client.0 ms_handle_connect on v2:172.21.15.150:3300/0 2024-09-21T03:48:28.365 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:28.360+0000 7f1095ffb640 1 -- 172.21.15.39:0/3206030069 <== mon.2 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f109c022070 con 0x5558d8da29f0 2024-09-21T03:48:28.365 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:28.360+0000 7f10b1ace640 1 --2- 172.21.15.39:0/3206030069 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5558d8de4070 0x5558d9197710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:48:28.365 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:28.360+0000 7f10baaa7740 1 -- 172.21.15.39:0/3206030069 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5558d9103df0 con 0x5558d8da29f0 2024-09-21T03:48:28.365 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:28.360+0000 7f1095ffb640 1 -- 172.21.15.39:0/3206030069 <== mon.2 v2:172.21.15.150:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8484+0+0 (secure 0 0 0) 0x7f109c0035f0 con 0x5558d8da29f0 2024-09-21T03:48:28.365 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:28.362+0000 7f1095ffb640 1 -- 172.21.15.39:0/3206030069 <== mon.2 v2:172.21.15.150:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f109c02e050 con 0x5558d8da29f0 2024-09-21T03:48:28.365 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:28.363+0000 7f10baaa7740 10 client.25664 fetch_fsmap learned FSMap version 29 2024-09-21T03:48:28.366 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:28.363+0000 7f10baaa7740 1 -- 172.21.15.39:0/3206030069 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5558d91a60f0 con 0x5558d8da29f0 2024-09-21T03:48:28.371 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:28.368+0000 7f1095ffb640 1 -- 172.21.15.39:0/3206030069 <== mon.2 v2:172.21.15.150:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f109c003ca0 con 0x5558d8da29f0 2024-09-21T03:48:28.371 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:28.368+0000 7f1095ffb640 1 -- 172.21.15.39:0/3206030069 <== mon.2 v2:172.21.15.150:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f109c01ea60 con 0x5558d8da29f0 2024-09-21T03:48:28.373 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:28.369+0000 7f10baaa7740 10 client.25664 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T03:48:28.373 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:28.369+0000 7f10baaa7740 10 client.25664 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T03:48:28.373 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:28.370+0000 7f10baaa7740 20 client.25664 populate_metadata read hostname 'smithi039' 2024-09-21T03:48:28.373 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:28.370+0000 7f10baaa7740 1 --2- 172.21.15.39:0/3206030069 >> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] conn(0x5558d8d77da0 0x5558d91a9650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:48:28.373 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:28.370+0000 7f10baaa7740 4 client.25664 mds_command: new command op to 24695 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:48:28.373 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:28.370+0000 7f10baaa7740 1 -- 172.21.15.39:0/3206030069 --> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5558d9103df0 con 0x5558d8d77da0 2024-09-21T03:48:28.375 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:28.372+0000 7f10b22cf640 1 --2- 172.21.15.39:0/3206030069 >> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] conn(0x5558d8d77da0 0x5558d91a9650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:48:28.394 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:28.391+0000 7f1095ffb640 10 client.25664 ms_handle_connect on v2:172.21.15.150:6838/2949386907 2024-09-21T03:48:28.394 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:28.391+0000 7f10b22cf640 1 --2- 172.21.15.39:0/3206030069 >> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] conn(0x5558d8d77da0 0x5558d91a9650 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ce3aaba0a2cc21ec server_cookie=c8cecd245ec45885 in_seq=0 out_seq=0 2024-09-21T03:48:28.396 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:28.393+0000 7f1095ffb640 1 -- 172.21.15.39:0/3206030069 <== mds.0 v2:172.21.15.150:6838/2949386907 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5558d9103df0 con 0x5558d8d77da0 2024-09-21T03:48:28.396 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:28.393+0000 7f1095ffb640 10 client.25664 handle_command_reply: tid=0 2024-09-21T03:48:28.396 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:28.393+0000 7f10baaa7740 1 client.25664 shutdown 2024-09-21T03:48:28.398 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:28.395+0000 7f10baaa7740 1 -- 172.21.15.39:0/3206030069 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5558d8da29f0 msgr2=0x5558d9197c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:48:28.398 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:28.395+0000 7f10baaa7740 1 --2- 172.21.15.39:0/3206030069 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5558d8da29f0 0x5558d9197c70 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f109c011010 tx=0x7f109c0046c0 comp rx=0 tx=0).stop 2024-09-21T03:48:28.399 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:28.396+0000 7f10baaa7740 1 -- 172.21.15.39:0/3206030069 shutdown_connections 2024-09-21T03:48:28.399 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:28.396+0000 7f10baaa7740 1 --2- 172.21.15.39:0/3206030069 >> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] conn(0x5558d8d77da0 0x5558d91a9650 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:28.399 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:28.396+0000 7f10baaa7740 1 --2- 172.21.15.39:0/3206030069 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5558d8da29f0 0x5558d9197c70 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:28.399 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:28.396+0000 7f10baaa7740 1 --2- 172.21.15.39:0/3206030069 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5558d8de4070 0x5558d9197710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:28.399 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:28.396+0000 7f10baaa7740 1 --2- 172.21.15.39:0/3206030069 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5558d8dc3390 0x5558d91971d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:28.399 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:28.396+0000 7f10baaa7740 1 -- 172.21.15.39:0/3206030069 >> 172.21.15.39:0/3206030069 conn(0x5558d8dd8460 msgr2=0x5558d918cb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:48:28.400 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:28.397+0000 7f1095ffb640 0 client.25664 ms_handle_reset on v2:172.21.15.150:6838/2949386907 2024-09-21T03:48:28.400 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:28.397+0000 7f10baaa7740 1 -- 172.21.15.39:0/3206030069 shutdown_connections 2024-09-21T03:48:28.400 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:28.397+0000 7f10baaa7740 1 -- 172.21.15.39:0/3206030069 wait complete. 2024-09-21T03:48:28.400 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:28.397+0000 7f10baaa7740 20 client.25664 trim_cache size 0 max 16384 2024-09-21T03:48:28.434 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:28.431+0000 7f10baaa7740 1 Processor -- start 2024-09-21T03:48:28.435 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:28.431+0000 7f10baaa7740 1 -- start start 2024-09-21T03:48:28.436 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:28.432+0000 7f10baaa7740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5558d8da29f0 0x5558d9191c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:48:28.436 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:28.432+0000 7f10baaa7740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5558d8dc3390 0x5558d918d290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:48:28.437 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:28.432+0000 7f10baaa7740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5558d8de4070 0x5558d91943c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:48:28.437 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:28.432+0000 7f10baaa7740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f109000e8e0 con 0x5558d8da29f0 2024-09-21T03:48:28.437 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:28.432+0000 7f10baaa7740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f1090010420 con 0x5558d8de4070 2024-09-21T03:48:28.437 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:28.432+0000 7f10baaa7740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f109000efb0 con 0x5558d8dc3390 2024-09-21T03:48:28.437 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:28.433+0000 7f10b2ad0640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5558d8de4070 0x5558d91943c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:48:28.438 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:28.433+0000 7f10b1ace640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5558d8dc3390 0x5558d918d290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:48:28.438 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:28.433+0000 7f10b1ace640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5558d8dc3390 0x5558d918d290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.39:54666/0 (socket says 172.21.15.39:54666) 2024-09-21T03:48:28.438 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:28.433+0000 7f10b1ace640 1 -- 172.21.15.39:0/1498205688 learned_addr learned my addr 172.21.15.39:0/1498205688 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:48:28.438 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:28.434+0000 7f10b22cf640 1 --2- 172.21.15.39:0/1498205688 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5558d8da29f0 0x5558d9191c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:48:28.439 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:28.436+0000 7f10b2ad0640 1 -- 172.21.15.39:0/1498205688 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5558d8dc3390 msgr2=0x5558d918d290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:48:28.439 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:28.436+0000 7f10b2ad0640 1 --2- 172.21.15.39:0/1498205688 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5558d8dc3390 0x5558d918d290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:28.439 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:28.436+0000 7f10b2ad0640 1 -- 172.21.15.39:0/1498205688 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5558d8da29f0 msgr2=0x5558d9191c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:48:28.440 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:28.436+0000 7f10b2ad0640 1 --2- 172.21.15.39:0/1498205688 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5558d8da29f0 0x5558d9191c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:28.440 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:28.436+0000 7f10b2ad0640 1 -- 172.21.15.39:0/1498205688 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5558d90a7890 con 0x5558d8de4070 2024-09-21T03:48:28.442 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:28.439+0000 7f10b2ad0640 1 --2- 172.21.15.39:0/1498205688 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5558d8de4070 0x5558d91943c0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x5558d91981b0 tx=0x7f109c004500 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e9bae638b3d937b7 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:48:28.442 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:28.439+0000 7f10977fe640 1 -- 172.21.15.39:0/1498205688 <== mon.2 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f109c022040 con 0x5558d8de4070 2024-09-21T03:48:28.442 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:28.440+0000 7f10977fe640 1 -- 172.21.15.39:0/1498205688 <== mon.2 v2:172.21.15.150:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f109c036020 con 0x5558d8de4070 2024-09-21T03:48:28.446 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:28.442+0000 7f10baaa7740 1 -- 172.21.15.39:0/1498205688 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5558d8de4070 msgr2=0x5558d91943c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:48:28.446 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:28.442+0000 7f10baaa7740 1 --2- 172.21.15.39:0/1498205688 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5558d8de4070 0x5558d91943c0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x5558d91981b0 tx=0x7f109c004500 comp rx=0 tx=0).stop 2024-09-21T03:48:28.447 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:28.444+0000 7f10baaa7740 1 -- 172.21.15.39:0/1498205688 shutdown_connections 2024-09-21T03:48:28.447 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:28.444+0000 7f10baaa7740 1 --2- 172.21.15.39:0/1498205688 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5558d8de4070 0x5558d91943c0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:28.447 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:28.444+0000 7f10baaa7740 1 --2- 172.21.15.39:0/1498205688 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5558d8dc3390 0x5558d918d290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:28.447 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:28.444+0000 7f10baaa7740 1 --2- 172.21.15.39:0/1498205688 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5558d8da29f0 0x5558d9191c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:28.447 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:28.444+0000 7f10baaa7740 1 -- 172.21.15.39:0/1498205688 >> 172.21.15.39:0/1498205688 conn(0x5558d8dd8460 msgr2=0x5558d918cb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:48:28.456 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:28.450+0000 7f10baaa7740 1 -- 172.21.15.39:0/1498205688 shutdown_connections 2024-09-21T03:48:28.456 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:28.451+0000 7f10baaa7740 1 -- 172.21.15.39:0/1498205688 wait complete. 2024-09-21T03:48:28.456 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:28.451+0000 7f10baaa7740 1 -- start start 2024-09-21T03:48:28.456 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:28.452+0000 7f10baaa7740 1 Processor -- start 2024-09-21T03:48:28.456 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:28.452+0000 7f10baaa7740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5558d8da29f0 0x5558d919ce10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:48:28.456 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:28.452+0000 7f10baaa7740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5558d8dc3390 0x5558d919d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:48:28.456 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:28.452+0000 7f10baaa7740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5558d9210040 0x5558d9212460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:48:28.456 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:28.452+0000 7f10baaa7740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f109000fd50 con 0x5558d8da29f0 2024-09-21T03:48:28.456 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:28.452+0000 7f10baaa7740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f109000f680 con 0x5558d8dc3390 2024-09-21T03:48:28.456 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:28.452+0000 7f10baaa7740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f109000c8c0 con 0x5558d9210040 2024-09-21T03:48:28.456 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:28.453+0000 7f10baaa7740 10 client.0 _ll_register_callbacks cb 0x7f109c00b6c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:48:28.457 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:28.453+0000 7f10b22cf640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5558d8da29f0 0x5558d919ce10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:48:28.457 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:28.453+0000 7f10b2ad0640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5558d9210040 0x5558d9212460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:48:28.457 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:28.453+0000 7f10b2ad0640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5558d9210040 0x5558d9212460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.39:54682/0 (socket says 172.21.15.39:54682) 2024-09-21T03:48:28.457 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:28.453+0000 7f10b2ad0640 1 -- 172.21.15.39:0/133228807 learned_addr learned my addr 172.21.15.39:0/133228807 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:48:28.457 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:28.454+0000 7f10b1ace640 1 --2- 172.21.15.39:0/133228807 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5558d8dc3390 0x5558d919d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:48:28.458 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:28.454+0000 7f10b2ad0640 1 -- 172.21.15.39:0/133228807 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5558d8dc3390 msgr2=0x5558d919d3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:48:28.458 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:28.454+0000 7f10b2ad0640 1 --2- 172.21.15.39:0/133228807 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5558d8dc3390 0x5558d919d3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:28.458 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:28.454+0000 7f10b2ad0640 1 -- 172.21.15.39:0/133228807 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5558d8da29f0 msgr2=0x5558d919ce10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:48:28.458 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:28.454+0000 7f10b2ad0640 1 --2- 172.21.15.39:0/133228807 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5558d8da29f0 0x5558d919ce10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:28.458 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:28.454+0000 7f10b2ad0640 1 -- 172.21.15.39:0/133228807 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5558d9212ad0 con 0x5558d9210040 2024-09-21T03:48:28.459 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:28.456+0000 7f10b2ad0640 1 --2- 172.21.15.39:0/133228807 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5558d9210040 0x5558d9212460 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f109c005cd0 tx=0x7f109c00b7e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=62b6130d3462d641 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:48:28.459 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:28.456+0000 7f1095ffb640 10 client.0 ms_handle_connect on v2:172.21.15.161:3300/0 2024-09-21T03:48:28.460 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:28.457+0000 7f1095ffb640 1 -- 172.21.15.39:0/133228807 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f109c03f070 con 0x5558d9210040 2024-09-21T03:48:28.460 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:28.457+0000 7f10baaa7740 1 -- 172.21.15.39:0/133228807 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5558d90ff8a0 con 0x5558d9210040 2024-09-21T03:48:28.460 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:28.457+0000 7f1095ffb640 1 -- 172.21.15.39:0/133228807 <== mon.1 v2:172.21.15.161:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8484+0+0 (secure 0 0 0) 0x7f109c01ac90 con 0x5558d9210040 2024-09-21T03:48:28.460 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:28.457+0000 7f1095ffb640 1 -- 172.21.15.39:0/133228807 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f109c04b050 con 0x5558d9210040 2024-09-21T03:48:28.460 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:28.457+0000 7f10baaa7740 10 client.25675 fetch_fsmap learned FSMap version 29 2024-09-21T03:48:28.460 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:28.457+0000 7f10baaa7740 1 -- 172.21.15.39:0/133228807 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5558d9213de0 con 0x5558d9210040 2024-09-21T03:48:28.461 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:28.459+0000 7f1095ffb640 1 -- 172.21.15.39:0/133228807 <== mon.1 v2:172.21.15.161:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x5558d90ff8a0 con 0x5558d9210040 2024-09-21T03:48:28.463 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:28.460+0000 7f1095ffb640 1 -- 172.21.15.39:0/133228807 <== mon.1 v2:172.21.15.161:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f109c01ea30 con 0x5558d9210040 2024-09-21T03:48:28.463 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:28.461+0000 7f10baaa7740 10 client.25675 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T03:48:28.464 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:28.461+0000 7f10baaa7740 10 client.25675 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T03:48:28.464 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:28.461+0000 7f10baaa7740 20 client.25675 populate_metadata read hostname 'smithi039' 2024-09-21T03:48:28.464 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:28.461+0000 7f10baaa7740 1 --2- 172.21.15.39:0/133228807 >> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] conn(0x5558d92144a0 0x5558d92168c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:48:28.464 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:28.461+0000 7f10baaa7740 4 client.25675 mds_command: new command op to 24695 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T03:48:28.464 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:28.461+0000 7f10baaa7740 1 -- 172.21.15.39:0/133228807 --> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5558d90fccd0 con 0x5558d92144a0 2024-09-21T03:48:28.465 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:28.461+0000 7f10b22cf640 1 --2- 172.21.15.39:0/133228807 >> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] conn(0x5558d92144a0 0x5558d92168c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:48:28.466 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:28.463+0000 7f10b22cf640 1 --2- 172.21.15.39:0/133228807 >> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] conn(0x5558d92144a0 0x5558d92168c0 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=920e09a76701e247 server_cookie=79898ade03d38315 in_seq=0 out_seq=0 2024-09-21T03:48:28.466 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:28.463+0000 7f1095ffb640 10 client.25675 ms_handle_connect on v2:172.21.15.150:6838/2949386907 2024-09-21T03:48:28.467 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:28.463+0000 7f1095ffb640 1 -- 172.21.15.39:0/133228807 <== mds.0 v2:172.21.15.150:6838/2949386907 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5558d90fccd0 con 0x5558d92144a0 2024-09-21T03:48:28.467 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:28.463+0000 7f1095ffb640 10 client.25675 handle_command_reply: tid=0 2024-09-21T03:48:28.467 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:28.463+0000 7f10baaa7740 1 client.25675 shutdown 2024-09-21T03:48:28.469 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:28.466+0000 7f10baaa7740 1 -- 172.21.15.39:0/133228807 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5558d9210040 msgr2=0x5558d9212460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:48:28.469 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:28.466+0000 7f10baaa7740 1 --2- 172.21.15.39:0/133228807 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5558d9210040 0x5558d9212460 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f109c005cd0 tx=0x7f109c00b7e0 comp rx=0 tx=0).stop 2024-09-21T03:48:28.470 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:28.467+0000 7f10baaa7740 1 -- 172.21.15.39:0/133228807 shutdown_connections 2024-09-21T03:48:28.470 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:28.467+0000 7f10baaa7740 1 --2- 172.21.15.39:0/133228807 >> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] conn(0x5558d92144a0 0x5558d92168c0 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:28.471 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:28.467+0000 7f10baaa7740 1 --2- 172.21.15.39:0/133228807 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5558d9210040 0x5558d9212460 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:28.471 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:28.467+0000 7f10baaa7740 1 --2- 172.21.15.39:0/133228807 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5558d8dc3390 0x5558d919d3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:28.471 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:28.467+0000 7f10baaa7740 1 --2- 172.21.15.39:0/133228807 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5558d8da29f0 0x5558d919ce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:28.471 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:28.467+0000 7f10baaa7740 1 -- 172.21.15.39:0/133228807 >> 172.21.15.39:0/133228807 conn(0x5558d8dd8460 msgr2=0x5558d9198680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:48:28.471 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:28.467+0000 7f1095ffb640 0 client.25675 ms_handle_reset on v2:172.21.15.150:6838/2949386907 2024-09-21T03:48:28.471 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:28.468+0000 7f10baaa7740 1 -- 172.21.15.39:0/133228807 shutdown_connections 2024-09-21T03:48:28.471 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:28.468+0000 7f10baaa7740 1 -- 172.21.15.39:0/133228807 wait complete. 2024-09-21T03:48:28.471 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:28.468+0000 7f10baaa7740 20 client.25675 trim_cache size 0 max 16384 2024-09-21T03:48:28.472 INFO:teuthology.orchestra.run.smithi039.stdout:[] 2024-09-21T03:48:28.473 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:28.470+0000 7f10a17fa640 1 -- 172.21.15.39:0/270319611 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f10980b68b0 msgr2=0x7f10980b8dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:48:28.473 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:28.470+0000 7f10a17fa640 1 --2- 172.21.15.39:0/270319611 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f10980b68b0 0x7f10980b8dc0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f109c005fd0 tx=0x7f109c00b380 comp rx=0 tx=0).stop 2024-09-21T03:48:28.474 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:28.470+0000 7f10a17fa640 1 -- 172.21.15.39:0/270319611 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f10ac10a650 msgr2=0x7f10ac1a2070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:48:28.474 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:28.470+0000 7f10a17fa640 1 --2- 172.21.15.39:0/270319611 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f10ac10a650 0x7f10ac1a2070 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f10a400ed80 tx=0x7f10a400c6e0 comp rx=0 tx=0).stop 2024-09-21T03:48:28.474 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:28.470+0000 7f10a17fa640 1 -- 172.21.15.39:0/270319611 shutdown_connections 2024-09-21T03:48:28.474 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:28.470+0000 7f10a17fa640 1 --2- 172.21.15.39:0/270319611 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f10980b68b0 0x7f10980b8dc0 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:28.474 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:28.470+0000 7f10a17fa640 1 --2- 172.21.15.39:0/270319611 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f10ac110640 0x7f10ac1a0c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:28.474 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:28.471+0000 7f10a17fa640 1 --2- 172.21.15.39:0/270319611 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f10ac10fcf0 0x7f10ac1a0660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:28.474 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:28.471+0000 7f10a17fa640 1 --2- 172.21.15.39:0/270319611 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f10ac10a650 0x7f10ac1a2070 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:28.474 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:28.471+0000 7f10a17fa640 1 -- 172.21.15.39:0/270319611 >> 172.21.15.39:0/270319611 conn(0x7f10ac06ee50 msgr2=0x7f10ac10dd80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:48:28.475 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:28.471+0000 7f10a17fa640 1 -- 172.21.15.39:0/270319611 shutdown_connections 2024-09-21T03:48:28.475 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:28.471+0000 7f10a17fa640 1 -- 172.21.15.39:0/270319611 wait complete. 2024-09-21T03:48:29.584 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:48:29 smithi039 ceph-mon[31211]: pgmap v398: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 178 KiB/s rd, 453 MiB/s wr, 192 op/s 2024-09-21T03:48:29.718 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:48:29 smithi161 ceph-mon[34533]: pgmap v398: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 178 KiB/s rd, 453 MiB/s wr, 192 op/s 2024-09-21T03:48:29.724 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:48:29 smithi150 ceph-mon[34289]: pgmap v398: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 178 KiB/s rd, 453 MiB/s wr, 192 op/s 2024-09-21T03:48:30.349 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-21T03:48:30.350 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 51116d5c-77ca-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-21T03:48:30.661 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/51116d5c-77ca-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:48:31.186 INFO:tasks.workunit.client.0.smithi039.stdout: 2024-09-21T03:48:31.187 INFO:tasks.workunit.client.0.smithi039.stdout: Children see throughput for 1 rewriters = 774579.12 kB/sec 2024-09-21T03:48:31.187 INFO:tasks.workunit.client.0.smithi039.stdout: Parent sees throughput for 1 rewriters = 774565.37 kB/sec 2024-09-21T03:48:31.187 INFO:tasks.workunit.client.0.smithi039.stdout: Min throughput per process = 774579.12 kB/sec 2024-09-21T03:48:31.187 INFO:tasks.workunit.client.0.smithi039.stdout: Max throughput per process = 774579.12 kB/sec 2024-09-21T03:48:31.187 INFO:tasks.workunit.client.0.smithi039.stdout: Avg throughput per process = 774579.12 kB/sec 2024-09-21T03:48:31.187 INFO:tasks.workunit.client.0.smithi039.stdout: Min xfer = 10485760.00 kB 2024-09-21T03:48:31.560 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:31.413+0000 7fb090acb640 1 -- 172.21.15.39:0/3209617457 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb08c109e00 msgr2=0x7fb08c110630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:48:31.560 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:31.413+0000 7fb090acb640 1 --2- 172.21.15.39:0/3209617457 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb08c109e00 0x7fb08c110630 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7fb084009ca0 tx=0x7fb08402a8c0 comp rx=0 tx=0).stop 2024-09-21T03:48:31.560 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:31.413+0000 7fb090acb640 1 -- 172.21.15.39:0/3209617457 shutdown_connections 2024-09-21T03:48:31.561 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:31.413+0000 7fb090acb640 1 --2- 172.21.15.39:0/3209617457 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb08c109e00 0x7fb08c110630 unknown :-1 s=CLOSED pgs=628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:31.561 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:31.413+0000 7fb090acb640 1 --2- 172.21.15.39:0/3209617457 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb08c1094b0 0x7fb08c1098c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:31.561 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:31.413+0000 7fb090acb640 1 --2- 172.21.15.39:0/3209617457 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb08c103bc0 0x7fb08c103fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:31.561 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:31.413+0000 7fb090acb640 1 -- 172.21.15.39:0/3209617457 >> 172.21.15.39:0/3209617457 conn(0x7fb08c07b910 msgr2=0x7fb08c07bd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:48:31.561 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:31.413+0000 7fb090acb640 1 -- 172.21.15.39:0/3209617457 shutdown_connections 2024-09-21T03:48:31.561 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:31.413+0000 7fb090acb640 1 -- 172.21.15.39:0/3209617457 wait complete. 2024-09-21T03:48:31.561 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:31.414+0000 7fb090acb640 1 Processor -- start 2024-09-21T03:48:31.561 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:31.414+0000 7fb090acb640 1 -- start start 2024-09-21T03:48:31.561 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:31.414+0000 7fb090acb640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb08c103bc0 0x7fb08c19a4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:48:31.561 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:31.414+0000 7fb090acb640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb08c1094b0 0x7fb08c19aa90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:48:31.561 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:31.414+0000 7fb090acb640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb08c109e00 0x7fb08c19fd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:48:31.561 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:31.414+0000 7fb090acb640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fb08c113ac0 con 0x7fb08c103bc0 2024-09-21T03:48:31.561 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:31.414+0000 7fb090acb640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fb08c113940 con 0x7fb08c109e00 2024-09-21T03:48:31.562 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:31.414+0000 7fb090acb640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fb08c113c40 con 0x7fb08c1094b0 2024-09-21T03:48:31.562 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:31.414+0000 7fb08a59c640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb08c103bc0 0x7fb08c19a4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:48:31.562 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:31.414+0000 7fb08a59c640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb08c103bc0 0x7fb08c19a4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:41114/0 (socket says 172.21.15.39:41114) 2024-09-21T03:48:31.562 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:31.414+0000 7fb08a59c640 1 -- 172.21.15.39:0/446843168 learned_addr learned my addr 172.21.15.39:0/446843168 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:48:31.562 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:31.415+0000 7fb08ad9d640 1 --2- 172.21.15.39:0/446843168 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb08c109e00 0x7fb08c19fd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:48:31.562 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:31.415+0000 7fb089d9b640 1 --2- 172.21.15.39:0/446843168 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb08c1094b0 0x7fb08c19aa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:48:31.562 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:31.415+0000 7fb08a59c640 1 -- 172.21.15.39:0/446843168 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb08c1094b0 msgr2=0x7fb08c19aa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:48:31.562 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:31.415+0000 7fb08a59c640 1 --2- 172.21.15.39:0/446843168 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb08c1094b0 0x7fb08c19aa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:31.562 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:31.415+0000 7fb08a59c640 1 -- 172.21.15.39:0/446843168 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb08c109e00 msgr2=0x7fb08c19fd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:48:31.562 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:31.415+0000 7fb08a59c640 1 --2- 172.21.15.39:0/446843168 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb08c109e00 0x7fb08c19fd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:31.562 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:31.415+0000 7fb08a59c640 1 -- 172.21.15.39:0/446843168 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb08c1a0460 con 0x7fb08c103bc0 2024-09-21T03:48:31.562 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:31.415+0000 7fb08a59c640 1 --2- 172.21.15.39:0/446843168 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb08c103bc0 0x7fb08c19a4f0 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7fb07c00d950 tx=0x7fb07c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:48:31.562 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:31.415+0000 7fb0837fe640 1 -- 172.21.15.39:0/446843168 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb07c017070 con 0x7fb08c103bc0 2024-09-21T03:48:31.562 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:31.415+0000 7fb0837fe640 1 -- 172.21.15.39:0/446843168 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb07c009c30 con 0x7fb08c103bc0 2024-09-21T03:48:31.563 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:31.415+0000 7fb090acb640 1 -- 172.21.15.39:0/446843168 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb08c1a0750 con 0x7fb08c103bc0 2024-09-21T03:48:31.563 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:31.415+0000 7fb0837fe640 1 -- 172.21.15.39:0/446843168 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb07c00f620 con 0x7fb08c103bc0 2024-09-21T03:48:31.563 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:31.415+0000 7fb090acb640 1 -- 172.21.15.39:0/446843168 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb08c1a6eb0 con 0x7fb08c103bc0 2024-09-21T03:48:31.563 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:31.418+0000 7fb09703f740 1 Processor -- start 2024-09-21T03:48:31.563 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:31.418+0000 7fb09703f740 1 -- start start 2024-09-21T03:48:31.563 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:31.418+0000 7fb09703f740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560554144350 0x5605544eb430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:48:31.563 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:31.418+0000 7fb09703f740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56055411b490 0x5605544eb970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:48:31.563 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:31.418+0000 7fb09703f740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560554123670 0x5605544ebeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:48:31.563 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:31.418+0000 7fb09703f740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x560554461e60 con 0x56055411b490 2024-09-21T03:48:31.563 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:31.418+0000 7fb09703f740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x560554451290 con 0x560554144350 2024-09-21T03:48:31.563 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:31.418+0000 7fb09703f740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x5605544550a0 con 0x560554123670 2024-09-21T03:48:31.563 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:31.418+0000 7fb0837fe640 1 -- 172.21.15.39:0/446843168 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 27) ==== 154643+0+0 (secure 0 0 0) 0x7fb07c010460 con 0x7fb08c103bc0 2024-09-21T03:48:31.563 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:31.418+0000 7fb089d9b640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560554123670 0x5605544ebeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:48:31.563 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:31.418+0000 7fb089d9b640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560554123670 0x5605544ebeb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.39:54716/0 (socket says 172.21.15.39:54716) 2024-09-21T03:48:31.564 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:31.418+0000 7fb089d9b640 1 -- 172.21.15.39:0/1519657108 learned_addr learned my addr 172.21.15.39:0/1519657108 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:48:31.564 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:31.419+0000 7fb08ad9d640 1 --2- 172.21.15.39:0/1519657108 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560554144350 0x5605544eb430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:48:31.564 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:31.419+0000 7fb08a59c640 1 --2- 172.21.15.39:0/1519657108 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56055411b490 0x5605544eb970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:48:31.564 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:31.419+0000 7fb08ad9d640 1 -- 172.21.15.39:0/1519657108 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560554123670 msgr2=0x5605544ebeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:48:31.564 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:31.419+0000 7fb08ad9d640 1 --2- 172.21.15.39:0/1519657108 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560554123670 0x5605544ebeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:31.564 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:31.419+0000 7fb08ad9d640 1 -- 172.21.15.39:0/1519657108 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56055411b490 msgr2=0x5605544eb970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:48:31.564 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:31.419+0000 7fb08ad9d640 1 --2- 172.21.15.39:0/1519657108 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56055411b490 0x5605544eb970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:31.564 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:31.419+0000 7fb08ad9d640 1 -- 172.21.15.39:0/1519657108 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5605541af930 con 0x560554144350 2024-09-21T03:48:31.564 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:31.419+0000 7fb089d9b640 1 --2- 172.21.15.39:0/1519657108 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560554123670 0x5605544ebeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:48:31.564 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:31.419+0000 7fb08ad9d640 1 --2- 172.21.15.39:0/1519657108 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560554144350 0x5605544eb430 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fb084005e30 tx=0x7fb08402f4c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a33c41b4050ccea9 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:48:31.564 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:31.419+0000 7fb080ff9640 1 -- 172.21.15.39:0/1519657108 <== mon.2 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb084031cf0 con 0x560554144350 2024-09-21T03:48:31.564 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:31.420+0000 7fb080ff9640 1 -- 172.21.15.39:0/1519657108 <== mon.2 v2:172.21.15.150:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb084036030 con 0x560554144350 2024-09-21T03:48:31.564 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:31.420+0000 7fb080ff9640 1 -- 172.21.15.39:0/1519657108 <== mon.2 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0840042a0 con 0x560554144350 2024-09-21T03:48:31.565 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:31.420+0000 7fb0837fe640 1 --2- 172.21.15.39:0/446843168 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7fb0780b6680 0x7fb0780b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:48:31.565 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:31.420+0000 7fb08a59c640 1 --2- 172.21.15.39:0/446843168 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7fb0780b6680 0x7fb0780b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:48:31.565 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:31.420+0000 7fb0837fe640 1 -- 172.21.15.39:0/446843168 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8484+0+0 (secure 0 0 0) 0x7fb07c0e72c0 con 0x7fb08c103bc0 2024-09-21T03:48:31.565 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:31.420+0000 7fb09703f740 1 -- 172.21.15.39:0/1519657108 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560554144350 msgr2=0x5605544eb430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:48:31.565 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:31.420+0000 7fb08a59c640 1 --2- 172.21.15.39:0/446843168 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7fb0780b6680 0x7fb0780b8b90 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fb07c0e4ae0 tx=0x7fb07c0eae10 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:48:31.565 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:31.420+0000 7fb09703f740 1 --2- 172.21.15.39:0/1519657108 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560554144350 0x5605544eb430 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fb084005e30 tx=0x7fb08402f4c0 comp rx=0 tx=0).stop 2024-09-21T03:48:31.565 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:31.420+0000 7fb09703f740 1 -- 172.21.15.39:0/1519657108 shutdown_connections 2024-09-21T03:48:31.565 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:31.420+0000 7fb09703f740 1 --2- 172.21.15.39:0/1519657108 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560554123670 0x5605544ebeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:31.565 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:31.420+0000 7fb09703f740 1 --2- 172.21.15.39:0/1519657108 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56055411b490 0x5605544eb970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:31.565 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:31.420+0000 7fb09703f740 1 --2- 172.21.15.39:0/1519657108 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560554144350 0x5605544eb430 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:31.565 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:31.420+0000 7fb09703f740 1 -- 172.21.15.39:0/1519657108 >> 172.21.15.39:0/1519657108 conn(0x560554138740 msgr2=0x5605544e9320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:48:31.565 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:31.421+0000 7fb09703f740 1 -- 172.21.15.39:0/1519657108 shutdown_connections 2024-09-21T03:48:31.565 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:31.421+0000 7fb09703f740 1 -- 172.21.15.39:0/1519657108 wait complete. 2024-09-21T03:48:31.565 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:31.421+0000 7fb09703f740 1 -- start start 2024-09-21T03:48:31.566 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:31.421+0000 7fb09703f740 1 Processor -- start 2024-09-21T03:48:31.566 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:31.422+0000 7fb09703f740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56055411b490 0x5605544f7070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:48:31.566 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:31.422+0000 7fb09703f740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560554123670 0x5605544f75b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:48:31.566 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:31.422+0000 7fb09703f740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x560554144350 0x560554502d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:48:31.566 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:31.422+0000 7fb09703f740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x5605542b2890 con 0x560554144350 2024-09-21T03:48:31.566 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:31.422+0000 7fb09703f740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x5605543ebd40 con 0x560554123670 2024-09-21T03:48:31.566 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:31.422+0000 7fb09703f740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x5605544f9a30 con 0x56055411b490 2024-09-21T03:48:31.566 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:31.422+0000 7fb09703f740 10 client.0 _ll_register_callbacks cb 0x56055444fc80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:48:31.566 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:31.422+0000 7fb08ad9d640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x560554144350 0x560554502d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:48:31.566 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:31.422+0000 7fb08ad9d640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x560554144350 0x560554502d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:41130/0 (socket says 172.21.15.39:41130) 2024-09-21T03:48:31.566 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:31.422+0000 7fb08ad9d640 1 -- 172.21.15.39:0/3831508461 learned_addr learned my addr 172.21.15.39:0/3831508461 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:48:31.566 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:31.422+0000 7fb08ad9d640 1 --2- 172.21.15.39:0/3831508461 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56055411b490 0x5605544f7070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:48:31.566 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:31.422+0000 7fb089d9b640 1 --2- 172.21.15.39:0/3831508461 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560554123670 0x5605544f75b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:48:31.566 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:31.422+0000 7fb08ad9d640 1 -- 172.21.15.39:0/3831508461 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56055411b490 msgr2=0x5605544f7070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:48:31.567 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:31.422+0000 7fb08ad9d640 1 --2- 172.21.15.39:0/3831508461 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56055411b490 0x5605544f7070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:31.567 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:31.422+0000 7fb08ad9d640 1 -- 172.21.15.39:0/3831508461 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560554123670 msgr2=0x5605544f75b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:48:31.567 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:31.422+0000 7fb08ad9d640 1 --2- 172.21.15.39:0/3831508461 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560554123670 0x5605544f75b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:31.567 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:31.422+0000 7fb08ad9d640 1 -- 172.21.15.39:0/3831508461 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560554503290 con 0x560554144350 2024-09-21T03:48:31.567 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:31.422+0000 7fb08ad9d640 1 --2- 172.21.15.39:0/3831508461 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x560554144350 0x560554502d20 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7fb084003fd0 tx=0x7fb084005c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a96097f563b14865 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:48:31.567 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:31.422+0000 7fb071ffb640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-09-21T03:48:31.567 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:31.422+0000 7fb071ffb640 1 -- 172.21.15.39:0/3831508461 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb084004d40 con 0x560554144350 2024-09-21T03:48:31.567 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:31.423+0000 7fb09703f740 1 -- 172.21.15.39:0/3831508461 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb084036030 con 0x560554144350 2024-09-21T03:48:31.567 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:31.423+0000 7fb071ffb640 1 -- 172.21.15.39:0/3831508461 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8484+0+0 (secure 0 0 0) 0x7fb084004480 con 0x560554144350 2024-09-21T03:48:31.567 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:31.423+0000 7fb071ffb640 1 -- 172.21.15.39:0/3831508461 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fb084004680 con 0x560554144350 2024-09-21T03:48:31.567 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:31.423+0000 7fb09703f740 10 client.16128 fetch_fsmap learned FSMap version 29 2024-09-21T03:48:31.567 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:31.423+0000 7fb09703f740 1 -- 172.21.15.39:0/3831508461 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x560554505bc0 con 0x560554144350 2024-09-21T03:48:31.567 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:31.423+0000 7fb071ffb640 1 -- 172.21.15.39:0/3831508461 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb084036030 con 0x560554144350 2024-09-21T03:48:31.568 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:31.423+0000 7fb071ffb640 1 -- 172.21.15.39:0/3831508461 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fb08405b030 con 0x560554144350 2024-09-21T03:48:31.568 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:31.423+0000 7fb09703f740 10 client.16128 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T03:48:31.568 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:31.423+0000 7fb09703f740 10 client.16128 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T03:48:31.568 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:31.423+0000 7fb09703f740 20 client.16128 populate_metadata read hostname 'smithi039' 2024-09-21T03:48:31.568 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:31.423+0000 7fb09703f740 1 --2- 172.21.15.39:0/3831508461 >> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] conn(0x560554102cd0 0x560554508f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:48:31.568 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:31.423+0000 7fb09703f740 4 client.16128 mds_command: new command op to 24695 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:48:31.568 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:31.423+0000 7fb09703f740 1 -- 172.21.15.39:0/3831508461 --> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560554463db0 con 0x560554102cd0 2024-09-21T03:48:31.568 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:31.423+0000 7fb089d9b640 1 --2- 172.21.15.39:0/3831508461 >> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] conn(0x560554102cd0 0x560554508f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:48:31.568 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:31.424+0000 7fb089d9b640 1 --2- 172.21.15.39:0/3831508461 >> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] conn(0x560554102cd0 0x560554508f20 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=736b72225878641c server_cookie=1369d38cc3d9eba in_seq=0 out_seq=0 2024-09-21T03:48:31.568 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:31.424+0000 7fb071ffb640 10 client.16128 ms_handle_connect on v2:172.21.15.150:6838/2949386907 2024-09-21T03:48:31.568 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:31.425+0000 7fb071ffb640 1 -- 172.21.15.39:0/3831508461 <== mds.0 v2:172.21.15.150:6838/2949386907 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x560554463db0 con 0x560554102cd0 2024-09-21T03:48:31.568 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:31.425+0000 7fb071ffb640 10 client.16128 handle_command_reply: tid=0 2024-09-21T03:48:31.568 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:31.425+0000 7fb09703f740 1 client.16128 shutdown 2024-09-21T03:48:31.568 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:31.425+0000 7fb09703f740 1 -- 172.21.15.39:0/3831508461 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x560554144350 msgr2=0x560554502d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:48:31.568 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:31.425+0000 7fb09703f740 1 --2- 172.21.15.39:0/3831508461 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x560554144350 0x560554502d20 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7fb084003fd0 tx=0x7fb084005c30 comp rx=0 tx=0).stop 2024-09-21T03:48:31.569 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:31.425+0000 7fb09703f740 1 -- 172.21.15.39:0/3831508461 shutdown_connections 2024-09-21T03:48:31.569 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:31.425+0000 7fb09703f740 1 --2- 172.21.15.39:0/3831508461 >> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] conn(0x560554102cd0 0x560554508f20 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:31.569 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:31.425+0000 7fb09703f740 1 --2- 172.21.15.39:0/3831508461 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x560554144350 0x560554502d20 unknown :-1 s=CLOSED pgs=630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:31.569 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:31.425+0000 7fb09703f740 1 --2- 172.21.15.39:0/3831508461 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560554123670 0x5605544f75b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:31.569 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:31.425+0000 7fb071ffb640 0 client.16128 ms_handle_reset on v2:172.21.15.150:6838/2949386907 2024-09-21T03:48:31.569 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:31.425+0000 7fb09703f740 1 --2- 172.21.15.39:0/3831508461 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56055411b490 0x5605544f7070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:31.569 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:31.425+0000 7fb09703f740 1 -- 172.21.15.39:0/3831508461 >> 172.21.15.39:0/3831508461 conn(0x560554138740 msgr2=0x5605544ee610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:48:31.569 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:31.425+0000 7fb09703f740 1 -- 172.21.15.39:0/3831508461 shutdown_connections 2024-09-21T03:48:31.569 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:31.425+0000 7fb09703f740 1 -- 172.21.15.39:0/3831508461 wait complete. 2024-09-21T03:48:31.569 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:31.425+0000 7fb09703f740 20 client.16128 trim_cache size 0 max 16384 2024-09-21T03:48:31.569 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:31.442+0000 7fb09703f740 1 Processor -- start 2024-09-21T03:48:31.569 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:31.443+0000 7fb09703f740 1 -- start start 2024-09-21T03:48:31.569 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:31.443+0000 7fb09703f740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56055411b490 0x5605544f0540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:48:31.570 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:31.443+0000 7fb09703f740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560554123670 0x5605544f0a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:48:31.570 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:31.443+0000 7fb09703f740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560554144350 0x5605544f4060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:48:31.570 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:31.443+0000 7fb09703f740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fb08c1ad4f0 con 0x56055411b490 2024-09-21T03:48:31.570 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:31.443+0000 7fb09703f740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fb08c1af030 con 0x560554144350 2024-09-21T03:48:31.570 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:31.443+0000 7fb09703f740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fb08c1adbc0 con 0x560554123670 2024-09-21T03:48:31.570 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:31.443+0000 7fb08ad9d640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56055411b490 0x5605544f0540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:48:31.570 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:31.443+0000 7fb089d9b640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560554123670 0x5605544f0a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:48:31.570 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:31.443+0000 7fb089d9b640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560554123670 0x5605544f0a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.39:54742/0 (socket says 172.21.15.39:54742) 2024-09-21T03:48:31.570 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:31.443+0000 7fb089d9b640 1 -- 172.21.15.39:0/1836618992 learned_addr learned my addr 172.21.15.39:0/1836618992 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:48:31.570 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:31.443+0000 7fb089d9b640 1 -- 172.21.15.39:0/1836618992 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560554144350 msgr2=0x5605544f4060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:48:31.570 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:31.443+0000 7fb089d9b640 1 --2- 172.21.15.39:0/1836618992 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560554144350 0x5605544f4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:31.570 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:31.443+0000 7fb089d9b640 1 -- 172.21.15.39:0/1836618992 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56055411b490 msgr2=0x5605544f0540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:48:31.570 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:31.443+0000 7fb089d9b640 1 --2- 172.21.15.39:0/1836618992 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56055411b490 0x5605544f0540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:31.570 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:31.443+0000 7fb089d9b640 1 -- 172.21.15.39:0/1836618992 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5605544ee290 con 0x560554123670 2024-09-21T03:48:31.571 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:31.444+0000 7fb089d9b640 1 --2- 172.21.15.39:0/1836618992 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560554123670 0x5605544f0a80 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fb074015d00 tx=0x7fb0740096d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7d485e2d3f1e4a68 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:48:31.571 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:31.444+0000 7fb073fff640 1 -- 172.21.15.39:0/1836618992 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb074013a10 con 0x560554123670 2024-09-21T03:48:31.571 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:31.444+0000 7fb073fff640 1 -- 172.21.15.39:0/1836618992 <== mon.1 v2:172.21.15.161:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb074013bb0 con 0x560554123670 2024-09-21T03:48:31.571 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:31.444+0000 7fb073fff640 1 -- 172.21.15.39:0/1836618992 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0740047a0 con 0x560554123670 2024-09-21T03:48:31.571 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:31.444+0000 7fb09703f740 1 -- 172.21.15.39:0/1836618992 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560554123670 msgr2=0x5605544f0a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:48:31.571 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:31.444+0000 7fb09703f740 1 --2- 172.21.15.39:0/1836618992 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560554123670 0x5605544f0a80 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fb074015d00 tx=0x7fb0740096d0 comp rx=0 tx=0).stop 2024-09-21T03:48:31.571 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:31.444+0000 7fb09703f740 1 -- 172.21.15.39:0/1836618992 shutdown_connections 2024-09-21T03:48:31.571 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:31.444+0000 7fb09703f740 1 --2- 172.21.15.39:0/1836618992 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560554144350 0x5605544f4060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:31.571 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:31.444+0000 7fb09703f740 1 --2- 172.21.15.39:0/1836618992 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560554123670 0x5605544f0a80 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:31.571 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:31.444+0000 7fb09703f740 1 --2- 172.21.15.39:0/1836618992 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56055411b490 0x5605544f0540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:31.571 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:31.444+0000 7fb09703f740 1 -- 172.21.15.39:0/1836618992 >> 172.21.15.39:0/1836618992 conn(0x560554138740 msgr2=0x5605544f56f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:48:31.571 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:31.445+0000 7fb09703f740 1 -- 172.21.15.39:0/1836618992 shutdown_connections 2024-09-21T03:48:31.571 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:31.445+0000 7fb09703f740 1 -- 172.21.15.39:0/1836618992 wait complete. 2024-09-21T03:48:31.571 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:31.445+0000 7fb09703f740 1 -- start start 2024-09-21T03:48:31.572 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:31.445+0000 7fb09703f740 1 Processor -- start 2024-09-21T03:48:31.572 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:31.445+0000 7fb09703f740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56055411b490 0x5605544ff4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:48:31.572 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:31.445+0000 7fb09703f740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560554123670 0x5605544fb960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:48:31.572 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:31.445+0000 7fb09703f740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x560554144350 0x5605544fbee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:48:31.572 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:31.445+0000 7fb09703f740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fb08c1ae960 con 0x560554144350 2024-09-21T03:48:31.572 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:31.445+0000 7fb09703f740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fb08c1ae290 con 0x56055411b490 2024-09-21T03:48:31.572 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:31.445+0000 7fb09703f740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fb08c1ab4d0 con 0x560554123670 2024-09-21T03:48:31.572 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:31.445+0000 7fb09703f740 10 client.0 _ll_register_callbacks cb 0x7fb07c0eace0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:48:31.572 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:31.445+0000 7fb08ad9d640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x560554144350 0x5605544fbee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:48:31.572 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:31.445+0000 7fb08ad9d640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x560554144350 0x5605544fbee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:41150/0 (socket says 172.21.15.39:41150) 2024-09-21T03:48:31.572 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:31.445+0000 7fb08ad9d640 1 -- 172.21.15.39:0/1334022525 learned_addr learned my addr 172.21.15.39:0/1334022525 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:48:31.572 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:31.446+0000 7fb089d9b640 1 --2- 172.21.15.39:0/1334022525 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560554123670 0x5605544fb960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:48:31.572 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:31.446+0000 7fb08ad9d640 1 --2- 172.21.15.39:0/1334022525 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56055411b490 0x5605544ff4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:48:31.572 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:31.446+0000 7fb08ad9d640 1 -- 172.21.15.39:0/1334022525 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560554123670 msgr2=0x5605544fb960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:48:31.573 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:31.446+0000 7fb08ad9d640 1 --2- 172.21.15.39:0/1334022525 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560554123670 0x5605544fb960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:31.573 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:31.446+0000 7fb08ad9d640 1 -- 172.21.15.39:0/1334022525 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56055411b490 msgr2=0x5605544ff4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:48:31.573 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:31.446+0000 7fb08ad9d640 1 --2- 172.21.15.39:0/1334022525 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56055411b490 0x5605544ff4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:31.573 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:31.446+0000 7fb08ad9d640 1 -- 172.21.15.39:0/1334022525 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5605544fc670 con 0x560554144350 2024-09-21T03:48:31.573 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:31.446+0000 7fb08ad9d640 1 --2- 172.21.15.39:0/1334022525 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x560554144350 0x5605544fbee0 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7fb084005e30 tx=0x7fb084005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d611c6ccf9e1ca54 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:48:31.573 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:31.446+0000 7fb071ffb640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-09-21T03:48:31.573 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:31.446+0000 7fb071ffb640 1 -- 172.21.15.39:0/1334022525 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb084004820 con 0x560554144350 2024-09-21T03:48:31.573 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:31.446+0000 7fb071ffb640 1 -- 172.21.15.39:0/1334022525 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8484+0+0 (secure 0 0 0) 0x7fb084004b40 con 0x560554144350 2024-09-21T03:48:31.573 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:31.446+0000 7fb09703f740 1 -- 172.21.15.39:0/1334022525 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb074013bb0 con 0x560554144350 2024-09-21T03:48:31.573 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:31.446+0000 7fb071ffb640 1 -- 172.21.15.39:0/1334022525 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fb08406b050 con 0x560554144350 2024-09-21T03:48:31.573 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:31.446+0000 7fb09703f740 10 client.16134 fetch_fsmap learned FSMap version 29 2024-09-21T03:48:31.573 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:31.446+0000 7fb09703f740 1 -- 172.21.15.39:0/1334022525 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x560554574770 con 0x560554144350 2024-09-21T03:48:31.573 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:31.447+0000 7fb071ffb640 1 -- 172.21.15.39:0/1334022525 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb074013bb0 con 0x560554144350 2024-09-21T03:48:31.573 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:31.447+0000 7fb071ffb640 1 -- 172.21.15.39:0/1334022525 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fb084043070 con 0x560554144350 2024-09-21T03:48:31.574 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:31.447+0000 7fb09703f740 10 client.16134 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T03:48:31.574 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:31.447+0000 7fb09703f740 10 client.16134 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T03:48:31.574 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:31.447+0000 7fb09703f740 20 client.16134 populate_metadata read hostname 'smithi039' 2024-09-21T03:48:31.574 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:31.447+0000 7fb09703f740 1 --2- 172.21.15.39:0/1334022525 >> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] conn(0x5605540d8080 0x560554576e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:48:31.574 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:31.447+0000 7fb09703f740 4 client.16134 mds_command: new command op to 24695 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-21T03:48:31.574 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:31.447+0000 7fb09703f740 1 -- 172.21.15.39:0/1334022525 --> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x56055445f860 con 0x5605540d8080 2024-09-21T03:48:31.574 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:31.447+0000 7fb089d9b640 1 --2- 172.21.15.39:0/1334022525 >> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] conn(0x5605540d8080 0x560554576e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:48:31.574 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:31.448+0000 7fb071ffb640 10 client.16134 ms_handle_connect on v2:172.21.15.150:6838/2949386907 2024-09-21T03:48:31.574 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:31.448+0000 7fb089d9b640 1 --2- 172.21.15.39:0/1334022525 >> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] conn(0x5605540d8080 0x560554576e40 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=914cff9d6711dc9d server_cookie=3d9744f23602c813 in_seq=0 out_seq=0 2024-09-21T03:48:31.574 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:31.449+0000 7fb071ffb640 1 -- 172.21.15.39:0/1334022525 <== mds.0 v2:172.21.15.150:6838/2949386907 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x56055445f860 con 0x5605540d8080 2024-09-21T03:48:31.574 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:31.449+0000 7fb071ffb640 10 client.16134 handle_command_reply: tid=0 2024-09-21T03:48:31.574 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:31.449+0000 7fb09703f740 1 client.16134 shutdown 2024-09-21T03:48:31.574 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:31.449+0000 7fb09703f740 1 -- 172.21.15.39:0/1334022525 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x560554144350 msgr2=0x5605544fbee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:48:31.574 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:31.449+0000 7fb09703f740 1 --2- 172.21.15.39:0/1334022525 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x560554144350 0x5605544fbee0 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7fb084005e30 tx=0x7fb084005fd0 comp rx=0 tx=0).stop 2024-09-21T03:48:31.575 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:31.450+0000 7fb09703f740 1 -- 172.21.15.39:0/1334022525 shutdown_connections 2024-09-21T03:48:31.575 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:31.450+0000 7fb09703f740 1 --2- 172.21.15.39:0/1334022525 >> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] conn(0x5605540d8080 0x560554576e40 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:31.575 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:31.450+0000 7fb09703f740 1 --2- 172.21.15.39:0/1334022525 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x560554144350 0x5605544fbee0 unknown :-1 s=CLOSED pgs=631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:31.575 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:31.450+0000 7fb09703f740 1 --2- 172.21.15.39:0/1334022525 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560554123670 0x5605544fb960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:31.575 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:31.450+0000 7fb071ffb640 0 client.16134 ms_handle_reset on v2:172.21.15.150:6838/2949386907 2024-09-21T03:48:31.575 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:31.450+0000 7fb09703f740 1 --2- 172.21.15.39:0/1334022525 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56055411b490 0x5605544ff4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:31.575 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:31.450+0000 7fb09703f740 1 -- 172.21.15.39:0/1334022525 >> 172.21.15.39:0/1334022525 conn(0x560554138740 msgr2=0x5605544f5580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:48:31.575 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-09-21T03:48:31.575 INFO:teuthology.orchestra.run.smithi039.stdout: "return_code": 0, 2024-09-21T03:48:31.575 INFO:teuthology.orchestra.run.smithi039.stdout: "scrub_tag": "e28ad743-2b22-45b3-8799-a638c2883623", 2024-09-21T03:48:31.576 INFO:teuthology.orchestra.run.smithi039.stdout: "mode": "asynchronous" 2024-09-21T03:48:31.576 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-09-21T03:48:31.576 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:31.450+0000 7fb09703f740 1 -- 172.21.15.39:0/1334022525 shutdown_connections 2024-09-21T03:48:31.576 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:31.450+0000 7fb09703f740 1 -- 172.21.15.39:0/1334022525 wait complete. 2024-09-21T03:48:31.576 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:31.450+0000 7fb09703f740 20 client.16134 trim_cache size 0 max 16384 2024-09-21T03:48:31.576 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:31.450+0000 7fb090acb640 1 -- 172.21.15.39:0/446843168 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7fb0780b6680 msgr2=0x7fb0780b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:48:31.576 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:31.450+0000 7fb090acb640 1 --2- 172.21.15.39:0/446843168 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7fb0780b6680 0x7fb0780b8b90 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fb07c0e4ae0 tx=0x7fb07c0eae10 comp rx=0 tx=0).stop 2024-09-21T03:48:31.576 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:31.450+0000 7fb090acb640 1 -- 172.21.15.39:0/446843168 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb08c103bc0 msgr2=0x7fb08c19a4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:48:31.576 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:31.450+0000 7fb090acb640 1 --2- 172.21.15.39:0/446843168 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb08c103bc0 0x7fb08c19a4f0 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7fb07c00d950 tx=0x7fb07c00de10 comp rx=0 tx=0).stop 2024-09-21T03:48:31.576 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:31.450+0000 7fb090acb640 1 -- 172.21.15.39:0/446843168 shutdown_connections 2024-09-21T03:48:31.576 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:31.450+0000 7fb090acb640 1 --2- 172.21.15.39:0/446843168 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7fb0780b6680 0x7fb0780b8b90 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:31.576 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:31.450+0000 7fb090acb640 1 --2- 172.21.15.39:0/446843168 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb08c109e00 0x7fb08c19fd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:31.577 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:31.450+0000 7fb090acb640 1 --2- 172.21.15.39:0/446843168 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb08c1094b0 0x7fb08c19aa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:31.577 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:31.450+0000 7fb090acb640 1 --2- 172.21.15.39:0/446843168 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb08c103bc0 0x7fb08c19a4f0 unknown :-1 s=CLOSED pgs=629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:31.577 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:31.450+0000 7fb090acb640 1 -- 172.21.15.39:0/446843168 >> 172.21.15.39:0/446843168 conn(0x7fb08c07b910 msgr2=0x7fb08c107560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:48:31.577 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:31.451+0000 7fb090acb640 1 -- 172.21.15.39:0/446843168 shutdown_connections 2024-09-21T03:48:31.577 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:31.451+0000 7fb090acb640 1 -- 172.21.15.39:0/446843168 wait complete. 2024-09-21T03:48:31.583 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:48:31 smithi039 ceph-mon[31211]: pgmap v399: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 247 KiB/s rd, 675 MiB/s wr, 278 op/s 2024-09-21T03:48:31.717 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:48:31 smithi161 ceph-mon[34533]: pgmap v399: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 247 KiB/s rd, 675 MiB/s wr, 278 op/s 2024-09-21T03:48:31.724 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:48:31 smithi150 ceph-mon[34289]: pgmap v399: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 247 KiB/s rd, 675 MiB/s wr, 278 op/s 2024-09-21T03:48:32.335 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 51116d5c-77ca-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub status 2024-09-21T03:48:32.583 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:48:32 smithi039 ceph-mon[31211]: scrub queued for path: / 2024-09-21T03:48:32.584 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:48:32 smithi039 ceph-mon[31211]: scrub summary: idle+waiting paths [/] 2024-09-21T03:48:32.584 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:48:32 smithi039 ceph-mon[31211]: scrub summary: active paths [/] 2024-09-21T03:48:32.584 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:48:32 smithi039 ceph-mon[31211]: scrub summary: idle+waiting paths [/] 2024-09-21T03:48:32.599 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/51116d5c-77ca-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:48:32.718 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:48:32 smithi161 ceph-mon[34533]: scrub queued for path: / 2024-09-21T03:48:32.718 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:48:32 smithi161 ceph-mon[34533]: scrub summary: idle+waiting paths [/] 2024-09-21T03:48:32.718 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:48:32 smithi161 ceph-mon[34533]: scrub summary: active paths [/] 2024-09-21T03:48:32.718 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:48:32 smithi161 ceph-mon[34533]: scrub summary: idle+waiting paths [/] 2024-09-21T03:48:32.725 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:48:32 smithi150 ceph-mon[34289]: scrub queued for path: / 2024-09-21T03:48:32.725 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:48:32 smithi150 ceph-mon[34289]: scrub summary: idle+waiting paths [/] 2024-09-21T03:48:32.725 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:48:32 smithi150 ceph-mon[34289]: scrub summary: active paths [/] 2024-09-21T03:48:32.725 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:48:32 smithi150 ceph-mon[34289]: scrub summary: idle+waiting paths [/] 2024-09-21T03:48:33.078 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:33.061+0000 7f0d7b455640 1 -- 172.21.15.39:0/2595737357 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0d741083a0 msgr2=0x7f0d741087b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:48:33.078 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:33.061+0000 7f0d7b455640 1 --2- 172.21.15.39:0/2595737357 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0d741083a0 0x7f0d741087b0 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f0d68009900 tx=0x7f0d6802a410 comp rx=0 tx=0).stop 2024-09-21T03:48:33.078 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:33.061+0000 7f0d7b455640 1 -- 172.21.15.39:0/2595737357 shutdown_connections 2024-09-21T03:48:33.078 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:33.061+0000 7f0d7b455640 1 --2- 172.21.15.39:0/2595737357 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0d74108cf0 0x7f0d7410f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:33.078 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:33.061+0000 7f0d7b455640 1 --2- 172.21.15.39:0/2595737357 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0d741083a0 0x7f0d741087b0 unknown :-1 s=CLOSED pgs=632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:33.078 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:33.061+0000 7f0d7b455640 1 --2- 172.21.15.39:0/2595737357 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0d74102a60 0x7f0d74102e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:33.078 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:33.061+0000 7f0d7b455640 1 -- 172.21.15.39:0/2595737357 >> 172.21.15.39:0/2595737357 conn(0x7f0d74079730 msgr2=0x7f0d74079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:48:33.078 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:33.061+0000 7f0d7b455640 1 -- 172.21.15.39:0/2595737357 shutdown_connections 2024-09-21T03:48:33.078 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:33.061+0000 7f0d7b455640 1 -- 172.21.15.39:0/2595737357 wait complete. 2024-09-21T03:48:33.079 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:33.062+0000 7f0d7b455640 1 Processor -- start 2024-09-21T03:48:33.079 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:33.062+0000 7f0d7b455640 1 -- start start 2024-09-21T03:48:33.079 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:33.062+0000 7f0d7b455640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0d74102a60 0x7f0d74199390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:48:33.079 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:33.062+0000 7f0d7b455640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0d741083a0 0x7f0d74199930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:48:33.079 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:33.062+0000 7f0d7b455640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0d74108cf0 0x7f0d7419ebd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:48:33.079 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:33.062+0000 7f0d7b455640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f0d74112960 con 0x7f0d741083a0 2024-09-21T03:48:33.079 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:33.062+0000 7f0d7b455640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f0d741127e0 con 0x7f0d74102a60 2024-09-21T03:48:33.079 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:33.062+0000 7f0d7b455640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f0d74112ae0 con 0x7f0d74108cf0 2024-09-21T03:48:33.079 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:33.062+0000 7f0d789f0640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0d741083a0 0x7f0d74199930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:48:33.079 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:33.062+0000 7f0d789f0640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0d741083a0 0x7f0d74199930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:41170/0 (socket says 172.21.15.39:41170) 2024-09-21T03:48:33.079 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:33.062+0000 7f0d789f0640 1 -- 172.21.15.39:0/3035716755 learned_addr learned my addr 172.21.15.39:0/3035716755 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:48:33.079 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:33.063+0000 7f0d791f1640 1 --2- 172.21.15.39:0/3035716755 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0d74102a60 0x7f0d74199390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:48:33.079 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:33.063+0000 7f0d799f2640 1 --2- 172.21.15.39:0/3035716755 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0d74108cf0 0x7f0d7419ebd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:48:33.079 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:33.063+0000 7f0d789f0640 1 -- 172.21.15.39:0/3035716755 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0d74108cf0 msgr2=0x7f0d7419ebd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:48:33.080 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:33.063+0000 7f0d789f0640 1 --2- 172.21.15.39:0/3035716755 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0d74108cf0 0x7f0d7419ebd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:33.080 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:33.063+0000 7f0d789f0640 1 -- 172.21.15.39:0/3035716755 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0d74102a60 msgr2=0x7f0d74199390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:48:33.080 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:33.063+0000 7f0d789f0640 1 --2- 172.21.15.39:0/3035716755 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0d74102a60 0x7f0d74199390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:33.080 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:33.063+0000 7f0d789f0640 1 -- 172.21.15.39:0/3035716755 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0d7419f300 con 0x7f0d741083a0 2024-09-21T03:48:33.080 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:33.063+0000 7f0d799f2640 1 --2- 172.21.15.39:0/3035716755 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0d74108cf0 0x7f0d7419ebd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:48:33.080 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:33.063+0000 7f0d789f0640 1 --2- 172.21.15.39:0/3035716755 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0d741083a0 0x7f0d74199930 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f0d68032fd0 tx=0x7f0d68004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:48:33.080 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:33.063+0000 7f0d727fc640 1 -- 172.21.15.39:0/3035716755 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d6803f030 con 0x7f0d741083a0 2024-09-21T03:48:33.080 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:33.063+0000 7f0d7b455640 1 -- 172.21.15.39:0/3035716755 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0d7419f590 con 0x7f0d741083a0 2024-09-21T03:48:33.080 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:33.063+0000 7f0d727fc640 1 -- 172.21.15.39:0/3035716755 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0d68004490 con 0x7f0d741083a0 2024-09-21T03:48:33.080 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:33.063+0000 7f0d727fc640 1 -- 172.21.15.39:0/3035716755 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d68035070 con 0x7f0d741083a0 2024-09-21T03:48:33.080 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:33.063+0000 7f0d7b455640 1 -- 172.21.15.39:0/3035716755 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0d741a5d50 con 0x7f0d741083a0 2024-09-21T03:48:33.080 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:33.066+0000 7f0d819c9740 1 Processor -- start 2024-09-21T03:48:33.080 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:33.066+0000 7f0d819c9740 1 -- start start 2024-09-21T03:48:33.080 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:33.066+0000 7f0d727fc640 1 -- 172.21.15.39:0/3035716755 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 27) ==== 154643+0+0 (secure 0 0 0) 0x7f0d68042030 con 0x7f0d741083a0 2024-09-21T03:48:33.080 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:33.066+0000 7f0d819c9740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555af4dab070 0x555af5152650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:48:33.080 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:33.066+0000 7f0d819c9740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555af4d821b0 0x555af5152b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:48:33.080 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:33.067+0000 7f0d819c9740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x555af4d8a390 0x555af51530d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:48:33.081 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:33.067+0000 7f0d819c9740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x555af50c8e50 con 0x555af4d8a390 2024-09-21T03:48:33.081 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:33.067+0000 7f0d819c9740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x555af50b8280 con 0x555af4d821b0 2024-09-21T03:48:33.081 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:33.067+0000 7f0d819c9740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x555af50bc090 con 0x555af4dab070 2024-09-21T03:48:33.081 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:33.067+0000 7f0d789f0640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x555af4d8a390 0x555af51530d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:48:33.081 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:33.067+0000 7f0d789f0640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x555af4d8a390 0x555af51530d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:41178/0 (socket says 172.21.15.39:41178) 2024-09-21T03:48:33.081 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:33.067+0000 7f0d789f0640 1 -- 172.21.15.39:0/2704286162 learned_addr learned my addr 172.21.15.39:0/2704286162 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:48:33.081 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:33.067+0000 7f0d799f2640 1 --2- 172.21.15.39:0/2704286162 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555af4dab070 0x555af5152650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:48:33.081 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:33.067+0000 7f0d791f1640 1 --2- 172.21.15.39:0/2704286162 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555af4d821b0 0x555af5152b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:48:33.081 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:33.067+0000 7f0d789f0640 1 -- 172.21.15.39:0/2704286162 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555af4dab070 msgr2=0x555af5152650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:48:33.081 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:33.067+0000 7f0d789f0640 1 --2- 172.21.15.39:0/2704286162 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555af4dab070 0x555af5152650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:33.081 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:33.067+0000 7f0d789f0640 1 -- 172.21.15.39:0/2704286162 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555af4d821b0 msgr2=0x555af5152b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:48:33.081 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:33.067+0000 7f0d789f0640 1 --2- 172.21.15.39:0/2704286162 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555af4d821b0 0x555af5152b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:33.081 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:33.067+0000 7f0d789f0640 1 -- 172.21.15.39:0/2704286162 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555af4e16650 con 0x555af4d8a390 2024-09-21T03:48:33.081 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:33.067+0000 7f0d727fc640 1 --2- 172.21.15.39:0/3035716755 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f0d5c0b6680 0x7f0d5c0b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:48:33.081 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:33.067+0000 7f0d799f2640 1 --2- 172.21.15.39:0/3035716755 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f0d5c0b6680 0x7f0d5c0b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:48:33.082 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:33.067+0000 7f0d727fc640 1 -- 172.21.15.39:0/3035716755 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8484+0+0 (secure 0 0 0) 0x7f0d68107c10 con 0x7f0d741083a0 2024-09-21T03:48:33.082 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:33.068+0000 7f0d789f0640 1 --2- 172.21.15.39:0/2704286162 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x555af4d8a390 0x555af51530d0 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f0d681052f0 tx=0x7f0d6810bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=299642fce9b78217 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:48:33.082 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:33.068+0000 7f0d5b7fe640 1 -- 172.21.15.39:0/2704286162 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d68049030 con 0x555af4d8a390 2024-09-21T03:48:33.082 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:33.068+0000 7f0d5b7fe640 1 -- 172.21.15.39:0/2704286162 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0d680c9030 con 0x555af4d8a390 2024-09-21T03:48:33.082 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:33.068+0000 7f0d5b7fe640 1 -- 172.21.15.39:0/2704286162 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d680c9920 con 0x555af4d8a390 2024-09-21T03:48:33.082 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:33.069+0000 7f0d819c9740 1 -- 172.21.15.39:0/2704286162 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x555af4d8a390 msgr2=0x555af51530d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:48:33.082 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:33.069+0000 7f0d799f2640 1 --2- 172.21.15.39:0/3035716755 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f0d5c0b6680 0x7f0d5c0b8b90 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f0d64005fd0 tx=0x7f0d64009700 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:48:33.082 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:33.069+0000 7f0d819c9740 1 --2- 172.21.15.39:0/2704286162 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x555af4d8a390 0x555af51530d0 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f0d681052f0 tx=0x7f0d6810bee0 comp rx=0 tx=0).stop 2024-09-21T03:48:33.082 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:33.069+0000 7f0d819c9740 1 -- 172.21.15.39:0/2704286162 shutdown_connections 2024-09-21T03:48:33.082 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:33.069+0000 7f0d819c9740 1 --2- 172.21.15.39:0/2704286162 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x555af4d8a390 0x555af51530d0 unknown :-1 s=CLOSED pgs=634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:33.082 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:33.069+0000 7f0d819c9740 1 --2- 172.21.15.39:0/2704286162 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555af4d821b0 0x555af5152b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:33.082 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:33.069+0000 7f0d819c9740 1 --2- 172.21.15.39:0/2704286162 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555af4dab070 0x555af5152650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:33.082 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:33.069+0000 7f0d819c9740 1 -- 172.21.15.39:0/2704286162 >> 172.21.15.39:0/2704286162 conn(0x555af4d9f460 msgr2=0x555af5150540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:48:33.082 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:33.069+0000 7f0d819c9740 1 -- 172.21.15.39:0/2704286162 shutdown_connections 2024-09-21T03:48:33.082 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:33.069+0000 7f0d819c9740 1 -- 172.21.15.39:0/2704286162 wait complete. 2024-09-21T03:48:33.083 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:33.069+0000 7f0d819c9740 1 -- start start 2024-09-21T03:48:33.083 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:33.069+0000 7f0d819c9740 1 Processor -- start 2024-09-21T03:48:33.083 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:33.070+0000 7f0d819c9740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x555af4d821b0 0x555af515df40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:48:33.083 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:33.070+0000 7f0d819c9740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555af4d8a390 0x555af515e480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:48:33.083 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:33.070+0000 7f0d819c9740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555af4dab070 0x555af516e1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:48:33.083 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:33.070+0000 7f0d819c9740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x555af4f19700 con 0x555af4d821b0 2024-09-21T03:48:33.083 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:33.070+0000 7f0d819c9740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x555af5053110 con 0x555af4dab070 2024-09-21T03:48:33.083 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:33.070+0000 7f0d819c9740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x555af50c8e50 con 0x555af4d8a390 2024-09-21T03:48:33.083 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:33.070+0000 7f0d819c9740 10 client.0 _ll_register_callbacks cb 0x555af506c990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:48:33.083 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:33.070+0000 7f0d791f1640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x555af4d821b0 0x555af515df40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:48:33.083 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:33.070+0000 7f0d791f1640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x555af4d821b0 0x555af515df40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:41190/0 (socket says 172.21.15.39:41190) 2024-09-21T03:48:33.083 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:33.070+0000 7f0d791f1640 1 -- 172.21.15.39:0/979988200 learned_addr learned my addr 172.21.15.39:0/979988200 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:48:33.083 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:33.070+0000 7f0d789f0640 1 --2- 172.21.15.39:0/979988200 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555af4d8a390 0x555af515e480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:48:33.083 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:33.070+0000 7f0d799f2640 1 --2- 172.21.15.39:0/979988200 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555af4dab070 0x555af516e1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:48:33.083 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:33.070+0000 7f0d791f1640 1 -- 172.21.15.39:0/979988200 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555af4d8a390 msgr2=0x555af515e480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:48:33.084 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:33.070+0000 7f0d791f1640 1 --2- 172.21.15.39:0/979988200 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555af4d8a390 0x555af515e480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:33.084 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:33.070+0000 7f0d791f1640 1 -- 172.21.15.39:0/979988200 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555af4dab070 msgr2=0x555af516e1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:48:33.084 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:33.070+0000 7f0d791f1640 1 --2- 172.21.15.39:0/979988200 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555af4dab070 0x555af516e1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:33.084 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:33.070+0000 7f0d791f1640 1 -- 172.21.15.39:0/979988200 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555af5080c40 con 0x555af4d821b0 2024-09-21T03:48:33.084 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:33.070+0000 7f0d791f1640 1 --2- 172.21.15.39:0/979988200 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x555af4d821b0 0x555af515df40 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f0d6c010a50 tx=0x7f0d6c010f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=49d93bde026f3a85 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:48:33.084 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:33.070+0000 7f0d58ff9640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-09-21T03:48:33.084 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:33.071+0000 7f0d58ff9640 1 -- 172.21.15.39:0/979988200 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d6c00fbd0 con 0x555af4d821b0 2024-09-21T03:48:33.084 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:33.071+0000 7f0d58ff9640 1 -- 172.21.15.39:0/979988200 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8484+0+0 (secure 0 0 0) 0x7f0d6c00fd70 con 0x555af4d821b0 2024-09-21T03:48:33.084 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:33.071+0000 7f0d819c9740 1 -- 172.21.15.39:0/979988200 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0d680c9030 con 0x555af4d821b0 2024-09-21T03:48:33.084 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:33.071+0000 7f0d58ff9640 1 -- 172.21.15.39:0/979988200 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f0d6c004020 con 0x555af4d821b0 2024-09-21T03:48:33.084 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:33.071+0000 7f0d819c9740 10 client.16146 fetch_fsmap learned FSMap version 29 2024-09-21T03:48:33.084 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:33.071+0000 7f0d819c9740 1 -- 172.21.15.39:0/979988200 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x555af5170c30 con 0x555af4d821b0 2024-09-21T03:48:33.084 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:33.071+0000 7f0d58ff9640 1 -- 172.21.15.39:0/979988200 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0d680c9030 con 0x555af4d821b0 2024-09-21T03:48:33.084 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:33.071+0000 7f0d58ff9640 1 -- 172.21.15.39:0/979988200 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f0d6c020020 con 0x555af4d821b0 2024-09-21T03:48:33.084 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:33.071+0000 7f0d819c9740 10 client.16146 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T03:48:33.085 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:33.071+0000 7f0d819c9740 10 client.16146 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T03:48:33.085 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:33.071+0000 7f0d819c9740 20 client.16146 populate_metadata read hostname 'smithi039' 2024-09-21T03:48:33.085 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:33.071+0000 7f0d819c9740 1 --2- 172.21.15.39:0/979988200 >> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] conn(0x555af4d699f0 0x555af5174220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:48:33.085 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:33.071+0000 7f0d819c9740 4 client.16146 mds_command: new command op to 24695 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:48:33.085 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:33.071+0000 7f0d819c9740 1 -- 172.21.15.39:0/979988200 --> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555af50cada0 con 0x555af4d699f0 2024-09-21T03:48:33.085 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:33.072+0000 7f0d789f0640 1 --2- 172.21.15.39:0/979988200 >> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] conn(0x555af4d699f0 0x555af5174220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:48:33.085 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:33.072+0000 7f0d789f0640 1 --2- 172.21.15.39:0/979988200 >> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] conn(0x555af4d699f0 0x555af5174220 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=785fd08553685acf server_cookie=a0276189f24e3fb6 in_seq=0 out_seq=0 2024-09-21T03:48:33.085 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:33.072+0000 7f0d58ff9640 10 client.16146 ms_handle_connect on v2:172.21.15.150:6838/2949386907 2024-09-21T03:48:33.085 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:33.073+0000 7f0d58ff9640 1 -- 172.21.15.39:0/979988200 <== mds.0 v2:172.21.15.150:6838/2949386907 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x555af50cada0 con 0x555af4d699f0 2024-09-21T03:48:33.085 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:33.073+0000 7f0d58ff9640 10 client.16146 handle_command_reply: tid=0 2024-09-21T03:48:33.085 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:33.073+0000 7f0d819c9740 1 client.16146 shutdown 2024-09-21T03:48:33.085 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:33.073+0000 7f0d819c9740 1 -- 172.21.15.39:0/979988200 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x555af4d821b0 msgr2=0x555af515df40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:48:33.085 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:33.073+0000 7f0d819c9740 1 --2- 172.21.15.39:0/979988200 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x555af4d821b0 0x555af515df40 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f0d6c010a50 tx=0x7f0d6c010f10 comp rx=0 tx=0).stop 2024-09-21T03:48:33.085 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:33.073+0000 7f0d819c9740 1 -- 172.21.15.39:0/979988200 shutdown_connections 2024-09-21T03:48:33.085 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:33.073+0000 7f0d819c9740 1 --2- 172.21.15.39:0/979988200 >> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] conn(0x555af4d699f0 0x555af5174220 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:33.085 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:33.073+0000 7f0d819c9740 1 --2- 172.21.15.39:0/979988200 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555af4dab070 0x555af516e1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:33.086 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:33.073+0000 7f0d819c9740 1 --2- 172.21.15.39:0/979988200 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555af4d8a390 0x555af515e480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:33.086 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:33.073+0000 7f0d819c9740 1 --2- 172.21.15.39:0/979988200 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x555af4d821b0 0x555af515df40 unknown :-1 s=CLOSED pgs=635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:33.086 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:33.073+0000 7f0d58ff9640 0 client.16146 ms_handle_reset on v2:172.21.15.150:6838/2949386907 2024-09-21T03:48:33.086 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:33.073+0000 7f0d819c9740 1 -- 172.21.15.39:0/979988200 >> 172.21.15.39:0/979988200 conn(0x555af4d9f460 msgr2=0x555af5151600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:48:33.086 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:33.073+0000 7f0d819c9740 1 -- 172.21.15.39:0/979988200 shutdown_connections 2024-09-21T03:48:33.086 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:33.073+0000 7f0d819c9740 1 -- 172.21.15.39:0/979988200 wait complete. 2024-09-21T03:48:33.086 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:33.073+0000 7f0d819c9740 20 client.16146 trim_cache size 0 max 16384 2024-09-21T03:48:33.095 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:33.091+0000 7f0d819c9740 1 Processor -- start 2024-09-21T03:48:33.095 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:33.091+0000 7f0d819c9740 1 -- start start 2024-09-21T03:48:33.095 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:33.091+0000 7f0d819c9740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555af4d9f460 0x555af515bb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:48:33.095 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:33.091+0000 7f0d819c9740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555af4d821b0 0x555af516e1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:48:33.095 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:33.091+0000 7f0d819c9740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x555af4d8a390 0x555af5153d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:48:33.095 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:33.091+0000 7f0d819c9740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f0d741ad4a0 con 0x555af4d8a390 2024-09-21T03:48:33.095 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:33.091+0000 7f0d819c9740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f0d741aefe0 con 0x555af4d821b0 2024-09-21T03:48:33.095 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:33.091+0000 7f0d819c9740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f0d741adb70 con 0x555af4d9f460 2024-09-21T03:48:33.095 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:33.091+0000 7f0d799f2640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x555af4d8a390 0x555af5153d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:48:33.096 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:33.091+0000 7f0d799f2640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x555af4d8a390 0x555af5153d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:41198/0 (socket says 172.21.15.39:41198) 2024-09-21T03:48:33.096 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:33.091+0000 7f0d799f2640 1 -- 172.21.15.39:0/208563651 learned_addr learned my addr 172.21.15.39:0/208563651 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:48:33.096 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:33.092+0000 7f0d791f1640 1 --2- 172.21.15.39:0/208563651 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555af4d9f460 0x555af515bb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:48:33.096 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:33.092+0000 7f0d789f0640 1 --2- 172.21.15.39:0/208563651 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555af4d821b0 0x555af516e1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:48:33.096 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:33.092+0000 7f0d799f2640 1 -- 172.21.15.39:0/208563651 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555af4d9f460 msgr2=0x555af515bb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:48:33.096 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:33.092+0000 7f0d799f2640 1 --2- 172.21.15.39:0/208563651 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555af4d9f460 0x555af515bb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:33.096 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:33.092+0000 7f0d799f2640 1 -- 172.21.15.39:0/208563651 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555af4d821b0 msgr2=0x555af516e1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:48:33.096 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:33.092+0000 7f0d799f2640 1 --2- 172.21.15.39:0/208563651 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555af4d821b0 0x555af516e1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:33.096 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:33.092+0000 7f0d799f2640 1 -- 172.21.15.39:0/208563651 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555af514f770 con 0x555af4d8a390 2024-09-21T03:48:33.096 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:33.092+0000 7f0d789f0640 1 --2- 172.21.15.39:0/208563651 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555af4d821b0 0x555af516e1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:48:33.096 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:33.092+0000 7f0d791f1640 1 --2- 172.21.15.39:0/208563651 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555af4d9f460 0x555af515bb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:48:33.096 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:33.092+0000 7f0d799f2640 1 --2- 172.21.15.39:0/208563651 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x555af4d8a390 0x555af5153d90 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f0d64010620 tx=0x7f0d64015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb60e5b53f921af4 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:48:33.096 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:33.092+0000 7f0d5affd640 1 -- 172.21.15.39:0/208563651 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d64004b20 con 0x555af4d8a390 2024-09-21T03:48:33.096 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:33.092+0000 7f0d5affd640 1 -- 172.21.15.39:0/208563651 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0d64004cc0 con 0x555af4d8a390 2024-09-21T03:48:33.097 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:33.092+0000 7f0d5affd640 1 -- 172.21.15.39:0/208563651 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d6401e650 con 0x555af4d8a390 2024-09-21T03:48:33.100 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:33.093+0000 7f0d819c9740 1 -- 172.21.15.39:0/208563651 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x555af4d8a390 msgr2=0x555af5153d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:48:33.100 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:33.093+0000 7f0d819c9740 1 --2- 172.21.15.39:0/208563651 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x555af4d8a390 0x555af5153d90 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f0d64010620 tx=0x7f0d64015000 comp rx=0 tx=0).stop 2024-09-21T03:48:33.100 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:33.093+0000 7f0d819c9740 1 -- 172.21.15.39:0/208563651 shutdown_connections 2024-09-21T03:48:33.100 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:33.093+0000 7f0d819c9740 1 --2- 172.21.15.39:0/208563651 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x555af4d8a390 0x555af5153d90 unknown :-1 s=CLOSED pgs=636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:33.101 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:33.093+0000 7f0d819c9740 1 --2- 172.21.15.39:0/208563651 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555af4d821b0 0x555af516e1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:33.101 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:33.093+0000 7f0d819c9740 1 --2- 172.21.15.39:0/208563651 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555af4d9f460 0x555af515bb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:33.101 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:33.093+0000 7f0d819c9740 1 -- 172.21.15.39:0/208563651 >> 172.21.15.39:0/208563651 conn(0x555af5153250 msgr2=0x555af5153680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:48:33.101 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:33.093+0000 7f0d819c9740 1 -- 172.21.15.39:0/208563651 shutdown_connections 2024-09-21T03:48:33.101 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:33.093+0000 7f0d819c9740 1 -- 172.21.15.39:0/208563651 wait complete. 2024-09-21T03:48:33.101 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:33.093+0000 7f0d819c9740 1 -- start start 2024-09-21T03:48:33.101 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:33.093+0000 7f0d819c9740 1 Processor -- start 2024-09-21T03:48:33.101 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:33.094+0000 7f0d819c9740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555af4d821b0 0x555af5169f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:48:33.101 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:33.094+0000 7f0d819c9740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x555af4d8a390 0x555af516a480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:48:33.101 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:33.094+0000 7f0d819c9740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555af4d9f460 0x555af51ddc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:48:33.101 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:33.094+0000 7f0d819c9740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f0d741ae910 con 0x555af4d8a390 2024-09-21T03:48:33.101 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:33.094+0000 7f0d819c9740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f0d741ae240 con 0x555af4d821b0 2024-09-21T03:48:33.101 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:33.094+0000 7f0d819c9740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f0d741ab480 con 0x555af4d9f460 2024-09-21T03:48:33.101 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:33.094+0000 7f0d819c9740 10 client.0 _ll_register_callbacks cb 0x7f0d6c00a740 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:48:33.102 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:33.094+0000 7f0d789f0640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x555af4d8a390 0x555af516a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:48:33.102 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:33.094+0000 7f0d789f0640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x555af4d8a390 0x555af516a480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:41206/0 (socket says 172.21.15.39:41206) 2024-09-21T03:48:33.102 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:33.094+0000 7f0d789f0640 1 -- 172.21.15.39:0/956084421 learned_addr learned my addr 172.21.15.39:0/956084421 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:48:33.102 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:33.094+0000 7f0d799f2640 1 --2- 172.21.15.39:0/956084421 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555af4d9f460 0x555af51ddc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:48:33.102 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:33.094+0000 7f0d791f1640 1 --2- 172.21.15.39:0/956084421 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555af4d821b0 0x555af5169f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:48:33.102 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:33.094+0000 7f0d789f0640 1 -- 172.21.15.39:0/956084421 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555af4d9f460 msgr2=0x555af51ddc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:48:33.102 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:33.094+0000 7f0d789f0640 1 --2- 172.21.15.39:0/956084421 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555af4d9f460 0x555af51ddc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:33.102 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:33.094+0000 7f0d789f0640 1 -- 172.21.15.39:0/956084421 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555af4d821b0 msgr2=0x555af5169f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:48:33.102 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:33.094+0000 7f0d789f0640 1 --2- 172.21.15.39:0/956084421 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555af4d821b0 0x555af5169f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:33.102 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:33.094+0000 7f0d789f0640 1 -- 172.21.15.39:0/956084421 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555af516ad00 con 0x555af4d8a390 2024-09-21T03:48:33.102 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:33.095+0000 7f0d789f0640 1 --2- 172.21.15.39:0/956084421 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x555af4d8a390 0x555af516a480 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f0d68031440 tx=0x7f0d6802a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fffa22137dc1acd7 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:48:33.102 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:33.095+0000 7f0d58ff9640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-09-21T03:48:33.102 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:33.095+0000 7f0d58ff9640 1 -- 172.21.15.39:0/956084421 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d68049040 con 0x555af4d8a390 2024-09-21T03:48:33.102 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:33.095+0000 7f0d58ff9640 1 -- 172.21.15.39:0/956084421 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8484+0+0 (secure 0 0 0) 0x7f0d6810b940 con 0x555af4d8a390 2024-09-21T03:48:33.103 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:33.095+0000 7f0d819c9740 1 -- 172.21.15.39:0/956084421 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0d64004cc0 con 0x555af4d8a390 2024-09-21T03:48:33.103 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:33.095+0000 7f0d58ff9640 1 -- 172.21.15.39:0/956084421 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f0d680cd1f0 con 0x555af4d8a390 2024-09-21T03:48:33.103 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:33.095+0000 7f0d819c9740 10 client.16152 fetch_fsmap learned FSMap version 29 2024-09-21T03:48:33.103 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:33.095+0000 7f0d819c9740 1 -- 172.21.15.39:0/956084421 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x555af51df180 con 0x555af4d8a390 2024-09-21T03:48:33.103 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:33.095+0000 7f0d58ff9640 1 -- 172.21.15.39:0/956084421 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0d64004cc0 con 0x555af4d8a390 2024-09-21T03:48:33.103 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:33.095+0000 7f0d58ff9640 1 -- 172.21.15.39:0/956084421 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f0d68105600 con 0x555af4d8a390 2024-09-21T03:48:33.103 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:33.096+0000 7f0d819c9740 10 client.16152 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T03:48:33.103 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:33.096+0000 7f0d819c9740 10 client.16152 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T03:48:33.103 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:33.096+0000 7f0d819c9740 20 client.16152 populate_metadata read hostname 'smithi039' 2024-09-21T03:48:33.103 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:33.096+0000 7f0d819c9740 1 --2- 172.21.15.39:0/956084421 >> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] conn(0x555af4dab070 0x555af51e17b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:48:33.103 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:33.096+0000 7f0d819c9740 4 client.16152 mds_command: new command op to 24695 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-21T03:48:33.103 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:33.096+0000 7f0d819c9740 1 -- 172.21.15.39:0/956084421 --> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] -- command(tid 0: {"prefix": "scrub status"}) -- 0x555af50c6850 con 0x555af4dab070 2024-09-21T03:48:33.103 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:33.096+0000 7f0d791f1640 1 --2- 172.21.15.39:0/956084421 >> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] conn(0x555af4dab070 0x555af51e17b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:48:33.103 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:33.096+0000 7f0d791f1640 1 --2- 172.21.15.39:0/956084421 >> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] conn(0x555af4dab070 0x555af51e17b0 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e61f007a62384f72 server_cookie=fc169ca6f0b691fd in_seq=0 out_seq=0 2024-09-21T03:48:33.103 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:33.096+0000 7f0d58ff9640 10 client.16152 ms_handle_connect on v2:172.21.15.150:6838/2949386907 2024-09-21T03:48:33.104 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:33.097+0000 7f0d58ff9640 1 -- 172.21.15.39:0/956084421 <== mds.0 v2:172.21.15.150:6838/2949386907 1 ==== command_reply(tid 0: 0 ) ==== 8+0+1706 (crc 0 0 0) 0x555af50c6850 con 0x555af4dab070 2024-09-21T03:48:33.104 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:33.097+0000 7f0d58ff9640 10 client.16152 handle_command_reply: tid=0 2024-09-21T03:48:33.104 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:33.097+0000 7f0d819c9740 1 client.16152 shutdown 2024-09-21T03:48:33.104 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:33.097+0000 7f0d819c9740 1 -- 172.21.15.39:0/956084421 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x555af4d8a390 msgr2=0x555af516a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:48:33.104 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:33.097+0000 7f0d819c9740 1 --2- 172.21.15.39:0/956084421 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x555af4d8a390 0x555af516a480 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f0d68031440 tx=0x7f0d6802a410 comp rx=0 tx=0).stop 2024-09-21T03:48:33.104 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:33.097+0000 7f0d819c9740 1 -- 172.21.15.39:0/956084421 shutdown_connections 2024-09-21T03:48:33.104 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:33.097+0000 7f0d819c9740 1 --2- 172.21.15.39:0/956084421 >> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] conn(0x555af4dab070 0x555af51e17b0 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:33.104 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:33.097+0000 7f0d58ff9640 0 client.16152 ms_handle_reset on v2:172.21.15.150:6838/2949386907 2024-09-21T03:48:33.104 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:33.097+0000 7f0d819c9740 1 --2- 172.21.15.39:0/956084421 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555af4d9f460 0x555af51ddc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:33.104 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:33.097+0000 7f0d819c9740 1 --2- 172.21.15.39:0/956084421 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x555af4d8a390 0x555af516a480 unknown :-1 s=CLOSED pgs=637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:33.104 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:33.097+0000 7f0d819c9740 1 --2- 172.21.15.39:0/956084421 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555af4d821b0 0x555af5169f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:33.104 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:33.097+0000 7f0d819c9740 1 -- 172.21.15.39:0/956084421 >> 172.21.15.39:0/956084421 conn(0x555af5153250 msgr2=0x555af5161470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:48:33.104 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:33.097+0000 7f0d819c9740 1 -- 172.21.15.39:0/956084421 shutdown_connections 2024-09-21T03:48:33.104 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:33.097+0000 7f0d819c9740 1 -- 172.21.15.39:0/956084421 wait complete. 2024-09-21T03:48:33.105 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:33.097+0000 7f0d819c9740 20 client.16152 trim_cache size 0 max 16384 2024-09-21T03:48:33.105 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-09-21T03:48:33.105 INFO:teuthology.orchestra.run.smithi039.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-21T03:48:33.105 INFO:teuthology.orchestra.run.smithi039.stdout: "scrubs": { 2024-09-21T03:48:33.105 INFO:teuthology.orchestra.run.smithi039.stdout: "e28ad743-2b22-45b3-8799-a638c2883623": { 2024-09-21T03:48:33.105 INFO:teuthology.orchestra.run.smithi039.stdout: "path": "/", 2024-09-21T03:48:33.105 INFO:teuthology.orchestra.run.smithi039.stdout: "tag": "e28ad743-2b22-45b3-8799-a638c2883623", 2024-09-21T03:48:33.105 INFO:teuthology.orchestra.run.smithi039.stdout: "options": "recursive,force" 2024-09-21T03:48:33.105 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-21T03:48:33.105 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:48:33.106 INFO:teuthology.orchestra.run.smithi039.stdout: "scrub_stats": { 2024-09-21T03:48:33.106 INFO:teuthology.orchestra.run.smithi039.stdout: "e28ad743-2b22-45b3-8799-a638c2883623": { 2024-09-21T03:48:33.106 INFO:teuthology.orchestra.run.smithi039.stdout: "start_time": "2024-09-21T03:48:31.448821+0000", 2024-09-21T03:48:33.106 INFO:teuthology.orchestra.run.smithi039.stdout: "path": "/", 2024-09-21T03:48:33.106 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_started": 0, 2024-09-21T03:48:33.106 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_passed": 0, 2024-09-21T03:48:33.106 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_failed": 0, 2024-09-21T03:48:33.106 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_skipped": 0 2024-09-21T03:48:33.106 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:48:33.106 INFO:teuthology.orchestra.run.smithi039.stdout: "29411307-eb8d-4339-8d2b-47ebd32c0080": { 2024-09-21T03:48:33.106 INFO:teuthology.orchestra.run.smithi039.stdout: "start_time": "2024-09-21T03:47:38.092920+0000", 2024-09-21T03:48:33.106 INFO:teuthology.orchestra.run.smithi039.stdout: "path": "/", 2024-09-21T03:48:33.106 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_started": 0, 2024-09-21T03:48:33.106 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_passed": 0, 2024-09-21T03:48:33.106 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_failed": 0, 2024-09-21T03:48:33.107 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_skipped": 0 2024-09-21T03:48:33.107 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:48:33.107 INFO:teuthology.orchestra.run.smithi039.stdout: "c7e2b8e4-057d-435f-b2bd-b633de98035f": { 2024-09-21T03:48:33.107 INFO:teuthology.orchestra.run.smithi039.stdout: "start_time": "2024-09-21T03:47:19.400305+0000", 2024-09-21T03:48:33.107 INFO:teuthology.orchestra.run.smithi039.stdout: "path": "/", 2024-09-21T03:48:33.107 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_started": 0, 2024-09-21T03:48:33.107 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_passed": 0, 2024-09-21T03:48:33.107 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_failed": 0, 2024-09-21T03:48:33.107 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_skipped": 0 2024-09-21T03:48:33.107 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:48:33.107 INFO:teuthology.orchestra.run.smithi039.stdout: "10728de8-c49d-4edc-be2b-639249255e94": { 2024-09-21T03:48:33.107 INFO:teuthology.orchestra.run.smithi039.stdout: "start_time": "2024-09-21T03:47:03.965623+0000", 2024-09-21T03:48:33.107 INFO:teuthology.orchestra.run.smithi039.stdout: "path": "/", 2024-09-21T03:48:33.107 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_started": 0, 2024-09-21T03:48:33.107 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_passed": 0, 2024-09-21T03:48:33.108 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_failed": 0, 2024-09-21T03:48:33.108 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_skipped": 0 2024-09-21T03:48:33.108 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:48:33.108 INFO:teuthology.orchestra.run.smithi039.stdout: "02509e4f-3911-4983-b16f-df8ded0b9998": { 2024-09-21T03:48:33.108 INFO:teuthology.orchestra.run.smithi039.stdout: "start_time": "2024-09-21T03:46:17.889576+0000", 2024-09-21T03:48:33.108 INFO:teuthology.orchestra.run.smithi039.stdout: "path": "/", 2024-09-21T03:48:33.108 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_started": 0, 2024-09-21T03:48:33.108 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_passed": 0, 2024-09-21T03:48:33.108 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_failed": 0, 2024-09-21T03:48:33.108 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_skipped": 0 2024-09-21T03:48:33.108 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-21T03:48:33.108 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-21T03:48:33.108 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-09-21T03:48:33.109 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:33.098+0000 7f0d7b455640 1 -- 172.21.15.39:0/3035716755 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f0d5c0b6680 msgr2=0x7f0d5c0b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:48:33.109 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:33.098+0000 7f0d7b455640 1 --2- 172.21.15.39:0/3035716755 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f0d5c0b6680 0x7f0d5c0b8b90 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f0d64005fd0 tx=0x7f0d64009700 comp rx=0 tx=0).stop 2024-09-21T03:48:33.109 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:33.098+0000 7f0d7b455640 1 -- 172.21.15.39:0/3035716755 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0d741083a0 msgr2=0x7f0d74199930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:48:33.109 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:33.098+0000 7f0d7b455640 1 --2- 172.21.15.39:0/3035716755 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0d741083a0 0x7f0d74199930 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f0d68032fd0 tx=0x7f0d68004270 comp rx=0 tx=0).stop 2024-09-21T03:48:33.109 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:33.098+0000 7f0d7b455640 1 -- 172.21.15.39:0/3035716755 shutdown_connections 2024-09-21T03:48:33.109 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:33.098+0000 7f0d7b455640 1 --2- 172.21.15.39:0/3035716755 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f0d5c0b6680 0x7f0d5c0b8b90 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:33.109 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:33.098+0000 7f0d7b455640 1 --2- 172.21.15.39:0/3035716755 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0d74108cf0 0x7f0d7419ebd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:33.109 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:33.098+0000 7f0d7b455640 1 --2- 172.21.15.39:0/3035716755 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0d741083a0 0x7f0d74199930 unknown :-1 s=CLOSED pgs=633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:33.109 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:33.098+0000 7f0d7b455640 1 --2- 172.21.15.39:0/3035716755 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0d74102a60 0x7f0d74199390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:33.110 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:33.098+0000 7f0d7b455640 1 -- 172.21.15.39:0/3035716755 >> 172.21.15.39:0/3035716755 conn(0x7f0d74079730 msgr2=0x7f0d74104420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:48:33.110 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:33.098+0000 7f0d7b455640 1 -- 172.21.15.39:0/3035716755 shutdown_connections 2024-09-21T03:48:33.110 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:48:33.098+0000 7f0d7b455640 1 -- 172.21.15.39:0/3035716755 wait complete. 2024-09-21T03:48:33.717 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:48:33 smithi161 ceph-mon[34533]: pgmap v400: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 188 KiB/s rd, 682 MiB/s wr, 245 op/s 2024-09-21T03:48:33.724 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:48:33 smithi150 ceph-mon[34289]: pgmap v400: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 188 KiB/s rd, 682 MiB/s wr, 245 op/s 2024-09-21T03:48:33.833 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:48:33 smithi039 ceph-mon[31211]: pgmap v400: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 188 KiB/s rd, 682 MiB/s wr, 245 op/s 2024-09-21T03:48:33.950 INFO:tasks.cephfs.filesystem:scrub status for tag:e28ad743-2b22-45b3-8799-a638c2883623 - {'path': '/', 'tag': 'e28ad743-2b22-45b3-8799-a638c2883623', 'options': 'recursive,force'} 2024-09-21T03:48:34.570 INFO:tasks.workunit.client.0.smithi039.stdout: 2024-09-21T03:48:34.570 INFO:tasks.workunit.client.0.smithi039.stdout: Children see throughput for 1 readers = 9085929.00 kB/sec 2024-09-21T03:48:34.570 INFO:tasks.workunit.client.0.smithi039.stdout: Parent sees throughput for 1 readers = 9083787.84 kB/sec 2024-09-21T03:48:34.570 INFO:tasks.workunit.client.0.smithi039.stdout: Min throughput per process = 9085929.00 kB/sec 2024-09-21T03:48:34.571 INFO:tasks.workunit.client.0.smithi039.stdout: Max throughput per process = 9085929.00 kB/sec 2024-09-21T03:48:34.571 INFO:tasks.workunit.client.0.smithi039.stdout: Avg throughput per process = 9085929.00 kB/sec 2024-09-21T03:48:34.571 INFO:tasks.workunit.client.0.smithi039.stdout: Min xfer = 10485760.00 kB 2024-09-21T03:48:35.718 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:48:35 smithi161 ceph-mon[34533]: pgmap v401: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 206 KiB/s rd, 718 MiB/s wr, 254 op/s 2024-09-21T03:48:35.718 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:48:35 smithi161 ceph-mon[34533]: scrub summary: idle 2024-09-21T03:48:35.724 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:48:35 smithi150 ceph-mon[34289]: pgmap v401: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 206 KiB/s rd, 718 MiB/s wr, 254 op/s 2024-09-21T03:48:35.724 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:48:35 smithi150 ceph-mon[34289]: scrub summary: idle 2024-09-21T03:48:35.834 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:48:35 smithi039 ceph-mon[31211]: pgmap v401: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 206 KiB/s rd, 718 MiB/s wr, 254 op/s 2024-09-21T03:48:35.834 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:48:35 smithi039 ceph-mon[31211]: scrub summary: idle 2024-09-21T03:48:37.718 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:48:37 smithi161 ceph-mon[34533]: pgmap v402: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 207 KiB/s rd, 681 MiB/s wr, 245 op/s 2024-09-21T03:48:37.724 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:48:37 smithi150 ceph-mon[34289]: pgmap v402: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 207 KiB/s rd, 681 MiB/s wr, 245 op/s 2024-09-21T03:48:37.834 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:48:37 smithi039 ceph-mon[31211]: pgmap v402: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 207 KiB/s rd, 681 MiB/s wr, 245 op/s 2024-09-21T03:48:37.855 INFO:tasks.workunit.client.0.smithi039.stdout: 2024-09-21T03:48:37.855 INFO:tasks.workunit.client.0.smithi039.stdout: Children see throughput for 1 re-readers = 8927986.00 kB/sec 2024-09-21T03:48:37.855 INFO:tasks.workunit.client.0.smithi039.stdout: Parent sees throughput for 1 re-readers = 8926011.47 kB/sec 2024-09-21T03:48:37.855 INFO:tasks.workunit.client.0.smithi039.stdout: Min throughput per process = 8927986.00 kB/sec 2024-09-21T03:48:37.855 INFO:tasks.workunit.client.0.smithi039.stdout: Max throughput per process = 8927986.00 kB/sec 2024-09-21T03:48:37.855 INFO:tasks.workunit.client.0.smithi039.stdout: Avg throughput per process = 8927986.00 kB/sec 2024-09-21T03:48:37.855 INFO:tasks.workunit.client.0.smithi039.stdout: Min xfer = 10485760.00 kB 2024-09-21T03:48:38.717 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:48:38 smithi161 ceph-mon[34533]: pgmap v403: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 89 KiB/s rd, 487 MiB/s wr, 158 op/s 2024-09-21T03:48:38.725 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:48:38 smithi150 ceph-mon[34289]: pgmap v403: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 89 KiB/s rd, 487 MiB/s wr, 158 op/s 2024-09-21T03:48:38.834 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:48:38 smithi039 ceph-mon[31211]: pgmap v403: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 89 KiB/s rd, 487 MiB/s wr, 158 op/s 2024-09-21T03:48:40.968 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:48:40 smithi161 ceph-mon[34533]: pgmap v404: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 91 KiB/s rd, 487 MiB/s wr, 162 op/s 2024-09-21T03:48:40.974 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:48:40 smithi150 ceph-mon[34289]: pgmap v404: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 91 KiB/s rd, 487 MiB/s wr, 162 op/s 2024-09-21T03:48:41.084 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:48:40 smithi039 ceph-mon[31211]: pgmap v404: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 91 KiB/s rd, 487 MiB/s wr, 162 op/s 2024-09-21T03:48:42.968 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:48:42 smithi161 ceph-mon[34533]: pgmap v405: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 251 MiB/s wr, 73 op/s 2024-09-21T03:48:42.975 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:48:42 smithi150 ceph-mon[34289]: pgmap v405: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 251 MiB/s wr, 73 op/s 2024-09-21T03:48:43.084 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:48:42 smithi039 ceph-mon[31211]: pgmap v405: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 251 MiB/s wr, 73 op/s 2024-09-21T03:48:44.936 INFO:tasks.workunit.client.0.smithi039.stdout: 2024-09-21T03:48:44.936 INFO:tasks.workunit.client.0.smithi039.stdout: 2024-09-21T03:48:44.936 INFO:tasks.workunit.client.0.smithi039.stdout: 2024-09-21T03:48:44.936 INFO:tasks.workunit.client.0.smithi039.stdout:iozone test complete. 2024-09-21T03:48:44.937 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-09-21T03:48:44.937 DEBUG:teuthology.orchestra.run.smithi039:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-09-21T03:48:44.963 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:48:44 smithi039 ceph-mon[31211]: pgmap v406: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 152 MiB/s wr, 50 op/s 2024-09-21T03:48:44.967 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:48:44 smithi161 ceph-mon[34533]: pgmap v406: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 152 MiB/s wr, 50 op/s 2024-09-21T03:48:44.974 INFO:tasks.workunit:Stopping ['suites/iozone.sh'] on client.0... 2024-09-21T03:48:44.974 DEBUG:teuthology.orchestra.run.smithi039:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.0 /home/ubuntu/cephtest/clone.client.0 2024-09-21T03:48:44.974 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:48:44 smithi150 ceph-mon[34289]: pgmap v406: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 152 MiB/s wr, 50 op/s 2024-09-21T03:48:45.316 DEBUG:teuthology.parallel:result is None 2024-09-21T03:48:45.317 DEBUG:teuthology.orchestra.run.smithi039:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0 2024-09-21T03:48:45.355 INFO:tasks.workunit:Deleted dir /home/ubuntu/cephtest/mnt.0/client.0 2024-09-21T03:48:45.356 DEBUG:teuthology.run_tasks:Unwinding manager full_sequential_finally 2024-09-21T03:48:45.367 INFO:teuthology.task.full_sequential_finally:In full_sequential_finally, running task exec... 2024-09-21T03:48:45.367 INFO:teuthology.task.exec:Executing custom commands... 2024-09-21T03:48:45.368 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi039.front.sepia.ceph.com 2024-09-21T03:48:45.368 DEBUG:teuthology.orchestra.run.smithi039:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'date +%s > END_TIME' 2024-09-21T03:48:45.422 DEBUG:teuthology.orchestra.run.smithi039:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'START_TIME=$(cat START_TIME); END_TIME=$(cat END_TIME); DIFF_TIME=$((600-(END_TIME-START_TIME))); if [ "$DIFF_TIME" -gt 0 ]; then sleep "$DIFF_TIME"; fi' 2024-09-21T03:48:46.968 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:48:46 smithi161 ceph-mon[34533]: pgmap v407: 97 pgs: 97 active+clean; 6.0 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 KiB/s rd, 83 MiB/s wr, 120 op/s 2024-09-21T03:48:46.974 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:48:46 smithi150 ceph-mon[34289]: pgmap v407: 97 pgs: 97 active+clean; 6.0 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 KiB/s rd, 83 MiB/s wr, 120 op/s 2024-09-21T03:48:47.084 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:48:46 smithi039 ceph-mon[31211]: pgmap v407: 97 pgs: 97 active+clean; 6.0 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 KiB/s rd, 83 MiB/s wr, 120 op/s 2024-09-21T03:48:48.968 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:48:48 smithi161 ceph-mon[34533]: pgmap v408: 97 pgs: 97 active+clean; 4.9 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 2.4 KiB/s wr, 118 op/s 2024-09-21T03:48:48.974 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:48:48 smithi150 ceph-mon[34289]: pgmap v408: 97 pgs: 97 active+clean; 4.9 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 2.4 KiB/s wr, 118 op/s 2024-09-21T03:48:49.084 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:48:48 smithi039 ceph-mon[31211]: pgmap v408: 97 pgs: 97 active+clean; 4.9 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 2.4 KiB/s wr, 118 op/s 2024-09-21T03:48:50.968 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:48:50 smithi161 ceph-mon[34533]: pgmap v409: 97 pgs: 97 active+clean; 10 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 9.1 KiB/s rd, 3.9 KiB/s wr, 226 op/s 2024-09-21T03:48:50.974 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:48:50 smithi150 ceph-mon[34289]: pgmap v409: 97 pgs: 97 active+clean; 10 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 9.1 KiB/s rd, 3.9 KiB/s wr, 226 op/s 2024-09-21T03:48:51.084 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:48:50 smithi039 ceph-mon[31211]: pgmap v409: 97 pgs: 97 active+clean; 10 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 9.1 KiB/s rd, 3.9 KiB/s wr, 226 op/s 2024-09-21T03:48:52.834 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:48:52 smithi039 ceph-mon[31211]: pgmap v410: 97 pgs: 97 active+clean; 10 MiB data, 536 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 4.0 KiB/s wr, 224 op/s 2024-09-21T03:48:52.968 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:48:52 smithi161 ceph-mon[34533]: pgmap v410: 97 pgs: 97 active+clean; 10 MiB data, 536 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 4.0 KiB/s wr, 224 op/s 2024-09-21T03:48:52.974 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:48:52 smithi150 ceph-mon[34289]: pgmap v410: 97 pgs: 97 active+clean; 10 MiB data, 536 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 4.0 KiB/s wr, 224 op/s 2024-09-21T03:48:54.968 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:48:54 smithi161 ceph-mon[34533]: pgmap v411: 97 pgs: 97 active+clean; 10 MiB data, 536 MiB used, 1.0 TiB / 1.0 TiB avail; 9.1 KiB/s rd, 4.8 KiB/s wr, 224 op/s 2024-09-21T03:48:54.974 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:48:54 smithi150 ceph-mon[34289]: pgmap v411: 97 pgs: 97 active+clean; 10 MiB data, 536 MiB used, 1.0 TiB / 1.0 TiB avail; 9.1 KiB/s rd, 4.8 KiB/s wr, 224 op/s 2024-09-21T03:48:55.083 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:48:54 smithi039 ceph-mon[31211]: pgmap v411: 97 pgs: 97 active+clean; 10 MiB data, 536 MiB used, 1.0 TiB / 1.0 TiB avail; 9.1 KiB/s rd, 4.8 KiB/s wr, 224 op/s 2024-09-21T03:48:56.967 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:48:56 smithi161 ceph-mon[34533]: pgmap v412: 97 pgs: 97 active+clean; 10 MiB data, 536 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 2.7 KiB/s wr, 225 op/s 2024-09-21T03:48:56.974 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:48:56 smithi150 ceph-mon[34289]: pgmap v412: 97 pgs: 97 active+clean; 10 MiB data, 536 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 2.7 KiB/s wr, 225 op/s 2024-09-21T03:48:57.084 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:48:56 smithi039 ceph-mon[31211]: pgmap v412: 97 pgs: 97 active+clean; 10 MiB data, 536 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 2.7 KiB/s wr, 225 op/s 2024-09-21T03:48:58.968 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:48:58 smithi161 ceph-mon[34533]: pgmap v413: 97 pgs: 97 active+clean; 10 MiB data, 536 MiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 2.6 KiB/s wr, 137 op/s 2024-09-21T03:48:58.974 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:48:58 smithi150 ceph-mon[34289]: pgmap v413: 97 pgs: 97 active+clean; 10 MiB data, 536 MiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 2.6 KiB/s wr, 137 op/s 2024-09-21T03:48:59.083 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:48:58 smithi039 ceph-mon[31211]: pgmap v413: 97 pgs: 97 active+clean; 10 MiB data, 536 MiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 2.6 KiB/s wr, 137 op/s 2024-09-21T03:49:00.968 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:49:00 smithi161 ceph-mon[34533]: pgmap v414: 97 pgs: 97 active+clean; 10 MiB data, 536 MiB used, 1.0 TiB / 1.0 TiB avail; 8.5 KiB/s rd, 3.5 KiB/s wr, 117 op/s 2024-09-21T03:49:00.974 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:49:00 smithi150 ceph-mon[34289]: pgmap v414: 97 pgs: 97 active+clean; 10 MiB data, 536 MiB used, 1.0 TiB / 1.0 TiB avail; 8.5 KiB/s rd, 3.5 KiB/s wr, 117 op/s 2024-09-21T03:49:01.084 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:49:00 smithi039 ceph-mon[31211]: pgmap v414: 97 pgs: 97 active+clean; 10 MiB data, 536 MiB used, 1.0 TiB / 1.0 TiB avail; 8.5 KiB/s rd, 3.5 KiB/s wr, 117 op/s 2024-09-21T03:49:02.968 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:49:02 smithi161 ceph-mon[34533]: pgmap v415: 97 pgs: 97 active+clean; 10 MiB data, 536 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 2.0 KiB/s wr, 10 op/s 2024-09-21T03:49:02.975 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:49:02 smithi150 ceph-mon[34289]: pgmap v415: 97 pgs: 97 active+clean; 10 MiB data, 536 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 2.0 KiB/s wr, 10 op/s 2024-09-21T03:49:03.084 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:49:02 smithi039 ceph-mon[31211]: pgmap v415: 97 pgs: 97 active+clean; 10 MiB data, 536 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 2.0 KiB/s wr, 10 op/s 2024-09-21T03:49:03.951 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 51116d5c-77ca-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub status 2024-09-21T03:49:04.224 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/51116d5c-77ca-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:49:04.808 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:04.795+0000 7f05cc0a1640 1 -- 172.21.15.39:0/2597654456 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f05c41083a0 msgr2=0x7f05c41087b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:49:04.808 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:04.795+0000 7f05cc0a1640 1 --2- 172.21.15.39:0/2597654456 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f05c41083a0 0x7f05c41087b0 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f05b4009900 tx=0x7f05b402a410 comp rx=0 tx=0).stop 2024-09-21T03:49:04.808 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:04.795+0000 7f05cc0a1640 1 -- 172.21.15.39:0/2597654456 shutdown_connections 2024-09-21T03:49:04.808 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:04.795+0000 7f05cc0a1640 1 --2- 172.21.15.39:0/2597654456 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f05c4108cf0 0x7f05c410f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:04.808 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:04.795+0000 7f05cc0a1640 1 --2- 172.21.15.39:0/2597654456 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f05c41083a0 0x7f05c41087b0 unknown :-1 s=CLOSED pgs=638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:04.808 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:04.795+0000 7f05cc0a1640 1 --2- 172.21.15.39:0/2597654456 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f05c4102a60 0x7f05c4102e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:04.808 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:04.795+0000 7f05cc0a1640 1 -- 172.21.15.39:0/2597654456 >> 172.21.15.39:0/2597654456 conn(0x7f05c4079730 msgr2=0x7f05c4079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:49:04.808 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:04.795+0000 7f05cc0a1640 1 -- 172.21.15.39:0/2597654456 shutdown_connections 2024-09-21T03:49:04.808 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:04.795+0000 7f05cc0a1640 1 -- 172.21.15.39:0/2597654456 wait complete. 2024-09-21T03:49:04.808 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:04.796+0000 7f05cc0a1640 1 Processor -- start 2024-09-21T03:49:04.809 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:04.796+0000 7f05cc0a1640 1 -- start start 2024-09-21T03:49:04.809 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:04.796+0000 7f05cc0a1640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f05c4102a60 0x7f05c40ff670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:49:04.809 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:04.796+0000 7f05cc0a1640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f05c41083a0 0x7f05c40ffbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:49:04.809 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:04.796+0000 7f05cc0a1640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f05c4108cf0 0x7f05c41000f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:49:04.809 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:04.796+0000 7f05cc0a1640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f05c4112960 con 0x7f05c41083a0 2024-09-21T03:49:04.809 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:04.796+0000 7f05cc0a1640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f05c41127e0 con 0x7f05c4102a60 2024-09-21T03:49:04.809 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:04.796+0000 7f05cc0a1640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f05c4112ae0 con 0x7f05c4108cf0 2024-09-21T03:49:04.809 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:04.796+0000 7f05c963c640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f05c41083a0 0x7f05c40ffbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:49:04.809 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:04.796+0000 7f05c963c640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f05c41083a0 0x7f05c40ffbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:35092/0 (socket says 172.21.15.39:35092) 2024-09-21T03:49:04.809 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:04.796+0000 7f05c963c640 1 -- 172.21.15.39:0/4102612464 learned_addr learned my addr 172.21.15.39:0/4102612464 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:49:04.809 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:04.797+0000 7f05ca63e640 1 --2- 172.21.15.39:0/4102612464 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f05c4108cf0 0x7f05c41000f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:49:04.809 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:04.797+0000 7f05c9e3d640 1 --2- 172.21.15.39:0/4102612464 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f05c4102a60 0x7f05c40ff670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:49:04.809 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:04.797+0000 7f05c963c640 1 -- 172.21.15.39:0/4102612464 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f05c4108cf0 msgr2=0x7f05c41000f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:49:04.809 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:04.797+0000 7f05c963c640 1 --2- 172.21.15.39:0/4102612464 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f05c4108cf0 0x7f05c41000f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:04.810 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:04.797+0000 7f05c963c640 1 -- 172.21.15.39:0/4102612464 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f05c4102a60 msgr2=0x7f05c40ff670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:49:04.810 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:04.797+0000 7f05c963c640 1 --2- 172.21.15.39:0/4102612464 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f05c4102a60 0x7f05c40ff670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:04.810 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:04.797+0000 7f05c963c640 1 -- 172.21.15.39:0/4102612464 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f05c41a87a0 con 0x7f05c41083a0 2024-09-21T03:49:04.810 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:04.797+0000 7f05c963c640 1 --2- 172.21.15.39:0/4102612464 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f05c41083a0 0x7f05c40ffbb0 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f05b4006010 tx=0x7f05b4004060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:49:04.810 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:04.797+0000 7f05baffd640 1 -- 172.21.15.39:0/4102612464 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05b403f030 con 0x7f05c41083a0 2024-09-21T03:49:04.810 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:04.797+0000 7f05baffd640 1 -- 172.21.15.39:0/4102612464 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f05b4003aa0 con 0x7f05c41083a0 2024-09-21T03:49:04.810 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:04.797+0000 7f05cc0a1640 1 -- 172.21.15.39:0/4102612464 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f05c41a8a30 con 0x7f05c41083a0 2024-09-21T03:49:04.810 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:04.797+0000 7f05baffd640 1 -- 172.21.15.39:0/4102612464 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05b4035070 con 0x7f05c41083a0 2024-09-21T03:49:04.810 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:04.797+0000 7f05cc0a1640 1 -- 172.21.15.39:0/4102612464 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f05c41a8f10 con 0x7f05c41083a0 2024-09-21T03:49:04.810 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:04.800+0000 7f05baffd640 1 -- 172.21.15.39:0/4102612464 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 27) ==== 154643+0+0 (secure 0 0 0) 0x7f05b4042030 con 0x7f05c41083a0 2024-09-21T03:49:04.810 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:04.801+0000 7f05baffd640 1 --2- 172.21.15.39:0/4102612464 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f05b00b6680 0x7f05b00b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:49:04.810 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:04.801+0000 7f05c9e3d640 1 --2- 172.21.15.39:0/4102612464 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f05b00b6680 0x7f05b00b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:49:04.810 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:04.801+0000 7f05baffd640 1 -- 172.21.15.39:0/4102612464 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8484+0+0 (secure 0 0 0) 0x7f05b4107bf0 con 0x7f05c41083a0 2024-09-21T03:49:04.811 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:04.801+0000 7f05d2615740 1 Processor -- start 2024-09-21T03:49:04.811 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:04.802+0000 7f05d2615740 1 -- start start 2024-09-21T03:49:04.811 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:04.802+0000 7f05c9e3d640 1 --2- 172.21.15.39:0/4102612464 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f05b00b6680 0x7f05b00b8b90 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f05c41010c0 tx=0x7f05c000b5b0 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:49:04.811 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:04.802+0000 7f05d2615740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556b1b226070 0x556b1b5cd270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:49:04.811 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:04.802+0000 7f05d2615740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556b1b1fd1b0 0x556b1b5cd7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:49:04.811 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:04.802+0000 7f05d2615740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556b1b205390 0x556b1b5cdcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:49:04.811 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:04.802+0000 7f05d2615740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x556b1b543e50 con 0x556b1b205390 2024-09-21T03:49:04.811 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:04.802+0000 7f05d2615740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x556b1b533280 con 0x556b1b1fd1b0 2024-09-21T03:49:04.811 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:04.802+0000 7f05d2615740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x556b1b537090 con 0x556b1b226070 2024-09-21T03:49:04.811 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:04.802+0000 7f05ca63e640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556b1b205390 0x556b1b5cdcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:49:04.811 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:04.802+0000 7f05ca63e640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556b1b205390 0x556b1b5cdcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:35102/0 (socket says 172.21.15.39:35102) 2024-09-21T03:49:04.811 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:04.802+0000 7f05ca63e640 1 -- 172.21.15.39:0/3143688724 learned_addr learned my addr 172.21.15.39:0/3143688724 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:49:04.811 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:04.802+0000 7f05c9e3d640 1 --2- 172.21.15.39:0/3143688724 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556b1b226070 0x556b1b5cd270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:49:04.812 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:04.802+0000 7f05c963c640 1 --2- 172.21.15.39:0/3143688724 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556b1b1fd1b0 0x556b1b5cd7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:49:04.812 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:04.802+0000 7f05ca63e640 1 -- 172.21.15.39:0/3143688724 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556b1b226070 msgr2=0x556b1b5cd270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:49:04.812 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:04.802+0000 7f05ca63e640 1 --2- 172.21.15.39:0/3143688724 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556b1b226070 0x556b1b5cd270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:04.812 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:04.802+0000 7f05ca63e640 1 -- 172.21.15.39:0/3143688724 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556b1b1fd1b0 msgr2=0x556b1b5cd7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:49:04.812 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:04.802+0000 7f05ca63e640 1 --2- 172.21.15.39:0/3143688724 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556b1b1fd1b0 0x556b1b5cd7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:04.812 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:04.802+0000 7f05ca63e640 1 -- 172.21.15.39:0/3143688724 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556b1b291650 con 0x556b1b205390 2024-09-21T03:49:04.812 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:04.802+0000 7f05c963c640 1 --2- 172.21.15.39:0/3143688724 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556b1b1fd1b0 0x556b1b5cd7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:49:04.812 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:04.803+0000 7f05ca63e640 1 --2- 172.21.15.39:0/3143688724 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556b1b205390 0x556b1b5cdcf0 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f05bc009f50 tx=0x7f05bc0118d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e5f1e2063e9825bc server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:49:04.812 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:04.803+0000 7f05affff640 1 -- 172.21.15.39:0/3143688724 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05bc004d20 con 0x556b1b205390 2024-09-21T03:49:04.812 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:04.803+0000 7f05affff640 1 -- 172.21.15.39:0/3143688724 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f05bc004050 con 0x556b1b205390 2024-09-21T03:49:04.812 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:04.803+0000 7f05affff640 1 -- 172.21.15.39:0/3143688724 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05bc005280 con 0x556b1b205390 2024-09-21T03:49:04.812 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:04.804+0000 7f05d2615740 1 -- 172.21.15.39:0/3143688724 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556b1b205390 msgr2=0x556b1b5cdcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:49:04.812 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:04.804+0000 7f05d2615740 1 --2- 172.21.15.39:0/3143688724 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556b1b205390 0x556b1b5cdcf0 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f05bc009f50 tx=0x7f05bc0118d0 comp rx=0 tx=0).stop 2024-09-21T03:49:04.812 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:04.804+0000 7f05d2615740 1 -- 172.21.15.39:0/3143688724 shutdown_connections 2024-09-21T03:49:04.812 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:04.804+0000 7f05d2615740 1 --2- 172.21.15.39:0/3143688724 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556b1b205390 0x556b1b5cdcf0 unknown :-1 s=CLOSED pgs=640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:04.813 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:04.804+0000 7f05d2615740 1 --2- 172.21.15.39:0/3143688724 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556b1b1fd1b0 0x556b1b5cd7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:04.813 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:04.804+0000 7f05d2615740 1 --2- 172.21.15.39:0/3143688724 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556b1b226070 0x556b1b5cd270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:04.813 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:04.804+0000 7f05d2615740 1 -- 172.21.15.39:0/3143688724 >> 172.21.15.39:0/3143688724 conn(0x556b1b21a460 msgr2=0x556b1b5cb160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:49:04.813 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:04.804+0000 7f05d2615740 1 -- 172.21.15.39:0/3143688724 shutdown_connections 2024-09-21T03:49:04.813 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:04.804+0000 7f05d2615740 1 -- 172.21.15.39:0/3143688724 wait complete. 2024-09-21T03:49:04.813 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:04.805+0000 7f05d2615740 1 -- start start 2024-09-21T03:49:04.813 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:04.805+0000 7f05d2615740 1 Processor -- start 2024-09-21T03:49:04.813 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:04.806+0000 7f05d2615740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556b1b1fd1b0 0x556b1b5d8930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:49:04.813 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:04.806+0000 7f05d2615740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556b1b205390 0x556b1b5d8e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:49:04.813 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:04.806+0000 7f05d2615740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556b1b226070 0x556b1b5e4d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:49:04.813 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:04.806+0000 7f05d2615740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x556b1b394700 con 0x556b1b226070 2024-09-21T03:49:04.813 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:04.806+0000 7f05d2615740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x556b1b4ce110 con 0x556b1b205390 2024-09-21T03:49:04.813 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:04.806+0000 7f05d2615740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x556b1b5dba50 con 0x556b1b1fd1b0 2024-09-21T03:49:04.814 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:04.806+0000 7f05d2615740 10 client.0 _ll_register_callbacks cb 0x556b1b4e7990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:49:04.814 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:04.806+0000 7f05ca63e640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556b1b226070 0x556b1b5e4d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:49:04.814 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:04.806+0000 7f05c963c640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556b1b205390 0x556b1b5d8e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:49:04.814 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:04.806+0000 7f05c963c640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556b1b205390 0x556b1b5d8e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.39:40886/0 (socket says 172.21.15.39:40886) 2024-09-21T03:49:04.814 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:04.806+0000 7f05c963c640 1 -- 172.21.15.39:0/1385730307 learned_addr learned my addr 172.21.15.39:0/1385730307 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:49:04.814 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:04.806+0000 7f05c9e3d640 1 --2- 172.21.15.39:0/1385730307 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556b1b1fd1b0 0x556b1b5d8930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:49:04.816 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:04.806+0000 7f05c963c640 1 -- 172.21.15.39:0/1385730307 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556b1b1fd1b0 msgr2=0x556b1b5d8930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:49:04.816 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:04.806+0000 7f05c963c640 1 --2- 172.21.15.39:0/1385730307 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556b1b1fd1b0 0x556b1b5d8930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:04.816 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:04.806+0000 7f05c963c640 1 -- 172.21.15.39:0/1385730307 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556b1b226070 msgr2=0x556b1b5e4d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:49:04.816 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:04.806+0000 7f05c963c640 1 --2- 172.21.15.39:0/1385730307 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556b1b226070 0x556b1b5e4d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:04.816 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:04.806+0000 7f05c963c640 1 -- 172.21.15.39:0/1385730307 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556b1b4fbc40 con 0x556b1b205390 2024-09-21T03:49:04.816 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:04.807+0000 7f05c963c640 1 --2- 172.21.15.39:0/1385730307 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556b1b205390 0x556b1b5d8e70 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f05b40cac70 tx=0x7f05b40c98c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6d19e90e4bc181d9 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:49:04.817 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:04.807+0000 7f05ad7fa640 10 client.0 ms_handle_connect on v2:172.21.15.150:3300/0 2024-09-21T03:49:04.817 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:04.807+0000 7f05ad7fa640 1 -- 172.21.15.39:0/1385730307 <== mon.2 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05b4049030 con 0x556b1b205390 2024-09-21T03:49:04.817 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:04.807+0000 7f05ad7fa640 1 -- 172.21.15.39:0/1385730307 <== mon.2 v2:172.21.15.150:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8484+0+0 (secure 0 0 0) 0x7f05b40d6f90 con 0x556b1b205390 2024-09-21T03:49:04.817 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:04.807+0000 7f05d2615740 1 -- 172.21.15.39:0/1385730307 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f05bc004050 con 0x556b1b205390 2024-09-21T03:49:04.817 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:04.807+0000 7f05ad7fa640 1 -- 172.21.15.39:0/1385730307 <== mon.2 v2:172.21.15.150:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f05b40cd170 con 0x556b1b205390 2024-09-21T03:49:04.817 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:04.807+0000 7f05d2615740 10 client.25712 fetch_fsmap learned FSMap version 29 2024-09-21T03:49:04.817 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:04.807+0000 7f05d2615740 1 -- 172.21.15.39:0/1385730307 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x556b1b5e7790 con 0x556b1b205390 2024-09-21T03:49:04.817 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:04.807+0000 7f05ad7fa640 1 -- 172.21.15.39:0/1385730307 <== mon.2 v2:172.21.15.150:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f05bc004050 con 0x556b1b205390 2024-09-21T03:49:04.817 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:04.808+0000 7f05ad7fa640 1 -- 172.21.15.39:0/1385730307 <== mon.2 v2:172.21.15.150:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f05b403bb40 con 0x556b1b205390 2024-09-21T03:49:04.817 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:04.808+0000 7f05d2615740 10 client.25712 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T03:49:04.817 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:04.808+0000 7f05d2615740 10 client.25712 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T03:49:04.818 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:04.808+0000 7f05d2615740 20 client.25712 populate_metadata read hostname 'smithi039' 2024-09-21T03:49:04.818 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:04.808+0000 7f05d2615740 1 --2- 172.21.15.39:0/1385730307 >> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] conn(0x556b1b1e49f0 0x556b1b5ead50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:49:04.818 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:04.808+0000 7f05d2615740 4 client.25712 mds_command: new command op to 24695 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:49:04.818 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:04.808+0000 7f05d2615740 1 -- 172.21.15.39:0/1385730307 --> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556b1b545da0 con 0x556b1b1e49f0 2024-09-21T03:49:04.818 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:04.808+0000 7f05c9e3d640 1 --2- 172.21.15.39:0/1385730307 >> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] conn(0x556b1b1e49f0 0x556b1b5ead50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:49:04.818 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:04.809+0000 7f05ad7fa640 10 client.25712 ms_handle_connect on v2:172.21.15.150:6838/2949386907 2024-09-21T03:49:04.818 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:04.809+0000 7f05c9e3d640 1 --2- 172.21.15.39:0/1385730307 >> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] conn(0x556b1b1e49f0 0x556b1b5ead50 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=86d80bdf0888e152 server_cookie=4e38e676ba629896 in_seq=0 out_seq=0 2024-09-21T03:49:04.818 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:04.809+0000 7f05ad7fa640 1 -- 172.21.15.39:0/1385730307 <== mds.0 v2:172.21.15.150:6838/2949386907 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x556b1b545da0 con 0x556b1b1e49f0 2024-09-21T03:49:04.818 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:04.809+0000 7f05ad7fa640 10 client.25712 handle_command_reply: tid=0 2024-09-21T03:49:04.818 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:04.809+0000 7f05d2615740 1 client.25712 shutdown 2024-09-21T03:49:04.818 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:04.810+0000 7f05d2615740 1 -- 172.21.15.39:0/1385730307 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556b1b205390 msgr2=0x556b1b5d8e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:49:04.818 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:04.810+0000 7f05d2615740 1 --2- 172.21.15.39:0/1385730307 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556b1b205390 0x556b1b5d8e70 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f05b40cac70 tx=0x7f05b40c98c0 comp rx=0 tx=0).stop 2024-09-21T03:49:04.818 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:04.810+0000 7f05d2615740 1 -- 172.21.15.39:0/1385730307 shutdown_connections 2024-09-21T03:49:04.819 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:04.810+0000 7f05d2615740 1 --2- 172.21.15.39:0/1385730307 >> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] conn(0x556b1b1e49f0 0x556b1b5ead50 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:04.819 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:04.810+0000 7f05d2615740 1 --2- 172.21.15.39:0/1385730307 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556b1b226070 0x556b1b5e4d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:04.819 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:04.810+0000 7f05d2615740 1 --2- 172.21.15.39:0/1385730307 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556b1b205390 0x556b1b5d8e70 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:04.819 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:04.810+0000 7f05d2615740 1 --2- 172.21.15.39:0/1385730307 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556b1b1fd1b0 0x556b1b5d8930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:04.819 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:04.810+0000 7f05ad7fa640 0 client.25712 ms_handle_reset on v2:172.21.15.150:6838/2949386907 2024-09-21T03:49:04.819 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:04.810+0000 7f05d2615740 1 -- 172.21.15.39:0/1385730307 >> 172.21.15.39:0/1385730307 conn(0x556b1b21a460 msgr2=0x556b1b5d0400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:49:04.819 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:04.810+0000 7f05d2615740 1 -- 172.21.15.39:0/1385730307 shutdown_connections 2024-09-21T03:49:04.819 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:04.810+0000 7f05d2615740 1 -- 172.21.15.39:0/1385730307 wait complete. 2024-09-21T03:49:04.819 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:04.810+0000 7f05d2615740 20 client.25712 trim_cache size 0 max 16384 2024-09-21T03:49:04.830 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:04.828+0000 7f05d2615740 1 Processor -- start 2024-09-21T03:49:04.830 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:04.828+0000 7f05d2615740 1 -- start start 2024-09-21T03:49:04.830 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:04.828+0000 7f05d2615740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556b1b1fd1b0 0x556b1b5da810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:49:04.830 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:04.828+0000 7f05d2615740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556b1b205390 0x556b1b5ce9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:49:04.830 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:04.828+0000 7f05d2615740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556b1b226070 0x556b1b5e4c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:49:04.830 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:04.828+0000 7f05d2615740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f05c41b1830 con 0x556b1b205390 2024-09-21T03:49:04.831 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:04.828+0000 7f05d2615740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f05c41b3370 con 0x556b1b226070 2024-09-21T03:49:04.831 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:04.828+0000 7f05d2615740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f05c41b1f00 con 0x556b1b1fd1b0 2024-09-21T03:49:04.831 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:04.828+0000 7f05c963c640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556b1b205390 0x556b1b5ce9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:49:04.831 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:04.828+0000 7f05c963c640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556b1b205390 0x556b1b5ce9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:35118/0 (socket says 172.21.15.39:35118) 2024-09-21T03:49:04.831 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:04.828+0000 7f05c963c640 1 -- 172.21.15.39:0/3784032565 learned_addr learned my addr 172.21.15.39:0/3784032565 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:49:04.831 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:04.828+0000 7f05ca63e640 1 --2- 172.21.15.39:0/3784032565 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556b1b226070 0x556b1b5e4c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:49:04.831 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:04.828+0000 7f05c9e3d640 1 --2- 172.21.15.39:0/3784032565 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556b1b1fd1b0 0x556b1b5da810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:49:04.831 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:04.829+0000 7f05c963c640 1 -- 172.21.15.39:0/3784032565 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556b1b1fd1b0 msgr2=0x556b1b5da810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:49:04.831 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:04.829+0000 7f05c963c640 1 --2- 172.21.15.39:0/3784032565 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556b1b1fd1b0 0x556b1b5da810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:04.831 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:04.829+0000 7f05c963c640 1 -- 172.21.15.39:0/3784032565 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556b1b226070 msgr2=0x556b1b5e4c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:49:04.831 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:04.829+0000 7f05c963c640 1 --2- 172.21.15.39:0/3784032565 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556b1b226070 0x556b1b5e4c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:04.831 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:04.829+0000 7f05c963c640 1 -- 172.21.15.39:0/3784032565 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556b1b5db500 con 0x556b1b205390 2024-09-21T03:49:04.833 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:04.829+0000 7f05c963c640 1 --2- 172.21.15.39:0/3784032565 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556b1b205390 0x556b1b5ce9b0 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f05b4031440 tx=0x7f05b402a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=370011f3ef35c83f server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:49:04.833 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:04.829+0000 7f05af7fe640 1 -- 172.21.15.39:0/3784032565 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05b40e7030 con 0x556b1b205390 2024-09-21T03:49:04.833 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:04.829+0000 7f05af7fe640 1 -- 172.21.15.39:0/3784032565 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f05b403b460 con 0x556b1b205390 2024-09-21T03:49:04.833 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:04.829+0000 7f05af7fe640 1 -- 172.21.15.39:0/3784032565 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05b4049070 con 0x556b1b205390 2024-09-21T03:49:04.833 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:04.830+0000 7f05d2615740 1 -- 172.21.15.39:0/3784032565 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556b1b205390 msgr2=0x556b1b5ce9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:49:04.834 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:04.830+0000 7f05d2615740 1 --2- 172.21.15.39:0/3784032565 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556b1b205390 0x556b1b5ce9b0 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f05b4031440 tx=0x7f05b402a410 comp rx=0 tx=0).stop 2024-09-21T03:49:04.834 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:04.830+0000 7f05d2615740 1 -- 172.21.15.39:0/3784032565 shutdown_connections 2024-09-21T03:49:04.834 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:04.830+0000 7f05d2615740 1 --2- 172.21.15.39:0/3784032565 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556b1b226070 0x556b1b5e4c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:04.834 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:04.830+0000 7f05d2615740 1 --2- 172.21.15.39:0/3784032565 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556b1b205390 0x556b1b5ce9b0 unknown :-1 s=CLOSED pgs=641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:04.834 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:04.830+0000 7f05d2615740 1 --2- 172.21.15.39:0/3784032565 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556b1b1fd1b0 0x556b1b5da810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:04.834 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:04.830+0000 7f05d2615740 1 -- 172.21.15.39:0/3784032565 >> 172.21.15.39:0/3784032565 conn(0x556b1b21a460 msgr2=0x556b1b5ce2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:49:04.834 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:04.830+0000 7f05d2615740 1 -- 172.21.15.39:0/3784032565 shutdown_connections 2024-09-21T03:49:04.834 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:04.830+0000 7f05d2615740 1 -- 172.21.15.39:0/3784032565 wait complete. 2024-09-21T03:49:04.834 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:04.830+0000 7f05d2615740 1 -- start start 2024-09-21T03:49:04.834 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:04.830+0000 7f05d2615740 1 Processor -- start 2024-09-21T03:49:04.834 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:04.831+0000 7f05d2615740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556b1b1fd1b0 0x556b1b5e1680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:49:04.834 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:04.831+0000 7f05d2615740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556b1b205390 0x556b1b5dcb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:49:04.835 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:04.831+0000 7f05d2615740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556b1b226070 0x556b1b5dd290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:49:04.835 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:04.831+0000 7f05d2615740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f05c41b2ca0 con 0x556b1b205390 2024-09-21T03:49:04.835 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:04.831+0000 7f05d2615740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f05c41b25d0 con 0x556b1b1fd1b0 2024-09-21T03:49:04.835 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:04.831+0000 7f05d2615740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f05c41af810 con 0x556b1b226070 2024-09-21T03:49:04.835 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:04.831+0000 7f05d2615740 10 client.0 _ll_register_callbacks cb 0x7f05b4005ef0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:49:04.835 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:04.831+0000 7f05c963c640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556b1b205390 0x556b1b5dcb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:49:04.835 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:04.831+0000 7f05c963c640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556b1b205390 0x556b1b5dcb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:35132/0 (socket says 172.21.15.39:35132) 2024-09-21T03:49:04.835 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:04.831+0000 7f05c963c640 1 -- 172.21.15.39:0/2590954155 learned_addr learned my addr 172.21.15.39:0/2590954155 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:49:04.835 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:04.831+0000 7f05ca63e640 1 --2- 172.21.15.39:0/2590954155 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556b1b226070 0x556b1b5dd290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:49:04.835 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:04.831+0000 7f05c9e3d640 1 --2- 172.21.15.39:0/2590954155 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556b1b1fd1b0 0x556b1b5e1680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:49:04.835 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:04.831+0000 7f05c963c640 1 -- 172.21.15.39:0/2590954155 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556b1b226070 msgr2=0x556b1b5dd290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:49:04.835 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:04.831+0000 7f05c963c640 1 --2- 172.21.15.39:0/2590954155 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556b1b226070 0x556b1b5dd290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:04.835 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:04.831+0000 7f05c963c640 1 -- 172.21.15.39:0/2590954155 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556b1b1fd1b0 msgr2=0x556b1b5e1680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:49:04.835 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:04.831+0000 7f05c963c640 1 --2- 172.21.15.39:0/2590954155 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556b1b1fd1b0 0x556b1b5e1680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:04.836 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:04.832+0000 7f05c963c640 1 -- 172.21.15.39:0/2590954155 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556b1b654750 con 0x556b1b205390 2024-09-21T03:49:04.836 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:04.832+0000 7f05c963c640 1 --2- 172.21.15.39:0/2590954155 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556b1b205390 0x556b1b5dcb80 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7f05b4105880 tx=0x7f05b40045d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b219f53ec42648a server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:49:04.836 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:04.832+0000 7f05ad7fa640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-09-21T03:49:04.838 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:04.832+0000 7f05ad7fa640 1 -- 172.21.15.39:0/2590954155 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05b40e7070 con 0x556b1b205390 2024-09-21T03:49:04.838 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:04.832+0000 7f05ad7fa640 1 -- 172.21.15.39:0/2590954155 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8484+0+0 (secure 0 0 0) 0x7f05b40d6f90 con 0x556b1b205390 2024-09-21T03:49:04.838 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:04.832+0000 7f05d2615740 1 -- 172.21.15.39:0/2590954155 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x556b1b541850 con 0x556b1b205390 2024-09-21T03:49:04.838 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:04.832+0000 7f05ad7fa640 1 -- 172.21.15.39:0/2590954155 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f05b40af050 con 0x556b1b205390 2024-09-21T03:49:04.838 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:04.832+0000 7f05d2615740 10 client.16170 fetch_fsmap learned FSMap version 29 2024-09-21T03:49:04.838 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:04.832+0000 7f05d2615740 1 -- 172.21.15.39:0/2590954155 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x556b1b654750 con 0x556b1b205390 2024-09-21T03:49:04.838 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:04.832+0000 7f05ad7fa640 1 -- 172.21.15.39:0/2590954155 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x556b1b541850 con 0x556b1b205390 2024-09-21T03:49:04.838 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:04.833+0000 7f05ad7fa640 1 -- 172.21.15.39:0/2590954155 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f05b40e3bb0 con 0x556b1b205390 2024-09-21T03:49:04.839 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:04.833+0000 7f05d2615740 10 client.16170 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T03:49:04.839 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:04.833+0000 7f05d2615740 10 client.16170 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T03:49:04.839 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:04.833+0000 7f05d2615740 20 client.16170 populate_metadata read hostname 'smithi039' 2024-09-21T03:49:04.839 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:04.833+0000 7f05d2615740 1 --2- 172.21.15.39:0/2590954155 >> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] conn(0x556b1b1b9da0 0x556b1b657ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:49:04.839 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:04.833+0000 7f05d2615740 4 client.16170 mds_command: new command op to 24695 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-21T03:49:04.839 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:04.833+0000 7f05d2615740 1 -- 172.21.15.39:0/2590954155 --> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] -- command(tid 0: {"prefix": "scrub status"}) -- 0x556b1b53ec80 con 0x556b1b1b9da0 2024-09-21T03:49:04.839 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:04.833+0000 7f05c9e3d640 1 --2- 172.21.15.39:0/2590954155 >> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] conn(0x556b1b1b9da0 0x556b1b657ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:49:04.839 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:04.833+0000 7f05ad7fa640 10 client.16170 ms_handle_connect on v2:172.21.15.150:6838/2949386907 2024-09-21T03:49:04.839 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:04.833+0000 7f05c9e3d640 1 --2- 172.21.15.39:0/2590954155 >> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] conn(0x556b1b1b9da0 0x556b1b657ba0 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6b030ce87e7e6ab8 server_cookie=3452bdf87721cc4d in_seq=0 out_seq=0 2024-09-21T03:49:04.839 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:04.834+0000 7f05ad7fa640 1 -- 172.21.15.39:0/2590954155 <== mds.0 v2:172.21.15.150:6838/2949386907 1 ==== command_reply(tid 0: 0 ) ==== 8+0+1510 (crc 0 0 0) 0x556b1b53ec80 con 0x556b1b1b9da0 2024-09-21T03:49:04.839 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:04.834+0000 7f05ad7fa640 10 client.16170 handle_command_reply: tid=0 2024-09-21T03:49:04.839 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:04.834+0000 7f05d2615740 1 client.16170 shutdown 2024-09-21T03:49:04.839 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:04.834+0000 7f05d2615740 1 -- 172.21.15.39:0/2590954155 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556b1b205390 msgr2=0x556b1b5dcb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:49:04.839 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:04.834+0000 7f05d2615740 1 --2- 172.21.15.39:0/2590954155 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556b1b205390 0x556b1b5dcb80 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7f05b4105880 tx=0x7f05b40045d0 comp rx=0 tx=0).stop 2024-09-21T03:49:04.840 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:04.834+0000 7f05d2615740 1 -- 172.21.15.39:0/2590954155 shutdown_connections 2024-09-21T03:49:04.840 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:04.834+0000 7f05d2615740 1 --2- 172.21.15.39:0/2590954155 >> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] conn(0x556b1b1b9da0 0x556b1b657ba0 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:04.840 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:04.834+0000 7f05d2615740 1 --2- 172.21.15.39:0/2590954155 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556b1b226070 0x556b1b5dd290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:04.840 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:04.834+0000 7f05ad7fa640 0 client.16170 ms_handle_reset on v2:172.21.15.150:6838/2949386907 2024-09-21T03:49:04.840 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:04.834+0000 7f05d2615740 1 --2- 172.21.15.39:0/2590954155 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556b1b205390 0x556b1b5dcb80 unknown :-1 s=CLOSED pgs=642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:04.840 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:04.834+0000 7f05d2615740 1 --2- 172.21.15.39:0/2590954155 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556b1b1fd1b0 0x556b1b5e1680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:04.840 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:04.834+0000 7f05d2615740 1 -- 172.21.15.39:0/2590954155 >> 172.21.15.39:0/2590954155 conn(0x556b1b21a460 msgr2=0x556b1b5cec60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:49:04.840 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:04.834+0000 7f05d2615740 1 -- 172.21.15.39:0/2590954155 shutdown_connections 2024-09-21T03:49:04.840 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:04.834+0000 7f05d2615740 1 -- 172.21.15.39:0/2590954155 wait complete. 2024-09-21T03:49:04.840 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:04.834+0000 7f05d2615740 20 client.16170 trim_cache size 0 max 16384 2024-09-21T03:49:04.840 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-09-21T03:49:04.841 INFO:teuthology.orchestra.run.smithi039.stdout: "status": "no active scrubs running", 2024-09-21T03:49:04.841 INFO:teuthology.orchestra.run.smithi039.stdout: "scrubs": {}, 2024-09-21T03:49:04.841 INFO:teuthology.orchestra.run.smithi039.stdout: "scrub_stats": { 2024-09-21T03:49:04.841 INFO:teuthology.orchestra.run.smithi039.stdout: "e28ad743-2b22-45b3-8799-a638c2883623": { 2024-09-21T03:49:04.841 INFO:teuthology.orchestra.run.smithi039.stdout: "start_time": "2024-09-21T03:48:31.448821+0000", 2024-09-21T03:49:04.841 INFO:teuthology.orchestra.run.smithi039.stdout: "path": "/", 2024-09-21T03:49:04.841 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_started": 0, 2024-09-21T03:49:04.841 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_passed": 0, 2024-09-21T03:49:04.841 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_failed": 0, 2024-09-21T03:49:04.841 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_skipped": 0 2024-09-21T03:49:04.841 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:49:04.841 INFO:teuthology.orchestra.run.smithi039.stdout: "29411307-eb8d-4339-8d2b-47ebd32c0080": { 2024-09-21T03:49:04.841 INFO:teuthology.orchestra.run.smithi039.stdout: "start_time": "2024-09-21T03:47:38.092920+0000", 2024-09-21T03:49:04.841 INFO:teuthology.orchestra.run.smithi039.stdout: "path": "/", 2024-09-21T03:49:04.842 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_started": 0, 2024-09-21T03:49:04.842 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_passed": 0, 2024-09-21T03:49:04.842 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_failed": 0, 2024-09-21T03:49:04.842 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_skipped": 0 2024-09-21T03:49:04.842 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:49:04.842 INFO:teuthology.orchestra.run.smithi039.stdout: "c7e2b8e4-057d-435f-b2bd-b633de98035f": { 2024-09-21T03:49:04.842 INFO:teuthology.orchestra.run.smithi039.stdout: "start_time": "2024-09-21T03:47:19.400305+0000", 2024-09-21T03:49:04.842 INFO:teuthology.orchestra.run.smithi039.stdout: "path": "/", 2024-09-21T03:49:04.842 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_started": 0, 2024-09-21T03:49:04.842 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_passed": 0, 2024-09-21T03:49:04.842 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_failed": 0, 2024-09-21T03:49:04.842 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_skipped": 0 2024-09-21T03:49:04.842 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:49:04.842 INFO:teuthology.orchestra.run.smithi039.stdout: "10728de8-c49d-4edc-be2b-639249255e94": { 2024-09-21T03:49:04.843 INFO:teuthology.orchestra.run.smithi039.stdout: "start_time": "2024-09-21T03:47:03.965623+0000", 2024-09-21T03:49:04.843 INFO:teuthology.orchestra.run.smithi039.stdout: "path": "/", 2024-09-21T03:49:04.843 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_started": 0, 2024-09-21T03:49:04.843 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_passed": 0, 2024-09-21T03:49:04.843 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_failed": 0, 2024-09-21T03:49:04.843 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_skipped": 0 2024-09-21T03:49:04.843 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:49:04.843 INFO:teuthology.orchestra.run.smithi039.stdout: "02509e4f-3911-4983-b16f-df8ded0b9998": { 2024-09-21T03:49:04.843 INFO:teuthology.orchestra.run.smithi039.stdout: "start_time": "2024-09-21T03:46:17.889576+0000", 2024-09-21T03:49:04.843 INFO:teuthology.orchestra.run.smithi039.stdout: "path": "/", 2024-09-21T03:49:04.843 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_started": 0, 2024-09-21T03:49:04.844 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_passed": 0, 2024-09-21T03:49:04.844 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_failed": 0, 2024-09-21T03:49:04.844 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_skipped": 0 2024-09-21T03:49:04.844 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-21T03:49:04.844 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-21T03:49:04.844 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-09-21T03:49:04.844 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:04.835+0000 7f05cc0a1640 1 -- 172.21.15.39:0/4102612464 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f05b00b6680 msgr2=0x7f05b00b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:49:04.845 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:04.835+0000 7f05cc0a1640 1 --2- 172.21.15.39:0/4102612464 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f05b00b6680 0x7f05b00b8b90 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f05c41010c0 tx=0x7f05c000b5b0 comp rx=0 tx=0).stop 2024-09-21T03:49:04.845 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:04.835+0000 7f05cc0a1640 1 -- 172.21.15.39:0/4102612464 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f05c41083a0 msgr2=0x7f05c40ffbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:49:04.845 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:04.835+0000 7f05cc0a1640 1 --2- 172.21.15.39:0/4102612464 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f05c41083a0 0x7f05c40ffbb0 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f05b4006010 tx=0x7f05b4004060 comp rx=0 tx=0).stop 2024-09-21T03:49:04.845 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:04.835+0000 7f05cc0a1640 1 -- 172.21.15.39:0/4102612464 shutdown_connections 2024-09-21T03:49:04.845 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:04.835+0000 7f05cc0a1640 1 --2- 172.21.15.39:0/4102612464 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f05b00b6680 0x7f05b00b8b90 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:04.845 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:04.835+0000 7f05cc0a1640 1 --2- 172.21.15.39:0/4102612464 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f05c4108cf0 0x7f05c41000f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:04.845 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:04.835+0000 7f05cc0a1640 1 --2- 172.21.15.39:0/4102612464 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f05c41083a0 0x7f05c40ffbb0 unknown :-1 s=CLOSED pgs=639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:04.845 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:04.835+0000 7f05cc0a1640 1 --2- 172.21.15.39:0/4102612464 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f05c4102a60 0x7f05c40ff670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:04.845 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:04.835+0000 7f05cc0a1640 1 -- 172.21.15.39:0/4102612464 >> 172.21.15.39:0/4102612464 conn(0x7f05c4079730 msgr2=0x7f05c406b240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:49:04.845 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:04.835+0000 7f05cc0a1640 1 -- 172.21.15.39:0/4102612464 shutdown_connections 2024-09-21T03:49:04.845 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:04.835+0000 7f05cc0a1640 1 -- 172.21.15.39:0/4102612464 wait complete. 2024-09-21T03:49:04.968 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:49:04 smithi161 ceph-mon[34533]: pgmap v416: 97 pgs: 97 active+clean; 10 MiB data, 536 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 2.2 KiB/s wr, 10 op/s 2024-09-21T03:49:04.974 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:49:04 smithi150 ceph-mon[34289]: pgmap v416: 97 pgs: 97 active+clean; 10 MiB data, 536 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 2.2 KiB/s wr, 10 op/s 2024-09-21T03:49:04.976 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:49:04 smithi039 ceph-mon[31211]: pgmap v416: 97 pgs: 97 active+clean; 10 MiB data, 536 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 2.2 KiB/s wr, 10 op/s 2024-09-21T03:49:06.016 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-21T03:49:06.017 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 51116d5c-77ca-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T03:49:06.277 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/51116d5c-77ca-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:49:06.791 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:06.783+0000 7f7ff271e640 1 -- 172.21.15.39:0/2238093362 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7fec109df0 msgr2=0x7f7fec110620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:49:06.791 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:06.783+0000 7f7ff271e640 1 --2- 172.21.15.39:0/2238093362 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7fec109df0 0x7f7fec110620 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7f7fe4009ca0 tx=0x7f7fe402a8c0 comp rx=0 tx=0).stop 2024-09-21T03:49:06.792 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:06.783+0000 7f7ff271e640 1 -- 172.21.15.39:0/2238093362 shutdown_connections 2024-09-21T03:49:06.792 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:06.783+0000 7f7ff271e640 1 --2- 172.21.15.39:0/2238093362 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7fec109df0 0x7f7fec110620 unknown :-1 s=CLOSED pgs=643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:06.792 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:06.783+0000 7f7ff271e640 1 --2- 172.21.15.39:0/2238093362 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7fec1094a0 0x7f7fec1098b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:06.792 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:06.783+0000 7f7ff271e640 1 --2- 172.21.15.39:0/2238093362 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7fec103b60 0x7f7fec103f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:06.792 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:06.783+0000 7f7ff271e640 1 -- 172.21.15.39:0/2238093362 >> 172.21.15.39:0/2238093362 conn(0x7f7fec07b850 msgr2=0x7f7fec07bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:49:06.792 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:06.783+0000 7f7ff271e640 1 -- 172.21.15.39:0/2238093362 shutdown_connections 2024-09-21T03:49:06.792 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:06.783+0000 7f7ff271e640 1 -- 172.21.15.39:0/2238093362 wait complete. 2024-09-21T03:49:06.792 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:06.784+0000 7f7ff271e640 1 Processor -- start 2024-09-21T03:49:06.792 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:06.784+0000 7f7ff271e640 1 -- start start 2024-09-21T03:49:06.792 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:06.784+0000 7f7ff271e640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7fec103b60 0x7f7fec19c680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:49:06.792 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:06.784+0000 7f7ff271e640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7fec1094a0 0x7f7fec19cc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:49:06.792 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:06.784+0000 7f7ff271e640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7fec109df0 0x7f7fec1a1ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:49:06.793 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:06.784+0000 7f7ff271e640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f7fec113ab0 con 0x7f7fec103b60 2024-09-21T03:49:06.793 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:06.784+0000 7f7ff271e640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f7fec113930 con 0x7f7fec1094a0 2024-09-21T03:49:06.793 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:06.784+0000 7f7ff271e640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f7fec113c30 con 0x7f7fec109df0 2024-09-21T03:49:06.793 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:06.784+0000 7f7febfff640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7fec103b60 0x7f7fec19c680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:49:06.793 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:06.784+0000 7f7febfff640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7fec103b60 0x7f7fec19c680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:35146/0 (socket says 172.21.15.39:35146) 2024-09-21T03:49:06.793 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:06.784+0000 7f7febfff640 1 -- 172.21.15.39:0/731265241 learned_addr learned my addr 172.21.15.39:0/731265241 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:49:06.793 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:06.785+0000 7f7ff0cbb640 1 --2- 172.21.15.39:0/731265241 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7fec109df0 0x7f7fec1a1ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:49:06.793 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:06.785+0000 7f7feb7fe640 1 --2- 172.21.15.39:0/731265241 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7fec1094a0 0x7f7fec19cc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:49:06.793 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:06.785+0000 7f7febfff640 1 -- 172.21.15.39:0/731265241 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7fec109df0 msgr2=0x7f7fec1a1ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:49:06.793 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:06.785+0000 7f7febfff640 1 --2- 172.21.15.39:0/731265241 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7fec109df0 0x7f7fec1a1ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:06.793 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:06.785+0000 7f7febfff640 1 -- 172.21.15.39:0/731265241 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7fec1094a0 msgr2=0x7f7fec19cc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:49:06.793 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:06.785+0000 7f7febfff640 1 --2- 172.21.15.39:0/731265241 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7fec1094a0 0x7f7fec19cc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:06.793 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:06.785+0000 7f7febfff640 1 -- 172.21.15.39:0/731265241 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7fec1a25f0 con 0x7f7fec103b60 2024-09-21T03:49:06.793 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:06.785+0000 7f7febfff640 1 --2- 172.21.15.39:0/731265241 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7fec103b60 0x7f7fec19c680 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f7fdc00d950 tx=0x7f7fdc00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:49:06.793 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:06.785+0000 7f7fe97fa640 1 -- 172.21.15.39:0/731265241 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fdc017070 con 0x7f7fec103b60 2024-09-21T03:49:06.794 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:06.785+0000 7f7ff271e640 1 -- 172.21.15.39:0/731265241 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7fec1a28e0 con 0x7f7fec103b60 2024-09-21T03:49:06.794 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:06.785+0000 7f7fe97fa640 1 -- 172.21.15.39:0/731265241 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7fdc009c30 con 0x7f7fec103b60 2024-09-21T03:49:06.794 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:06.785+0000 7f7fe97fa640 1 -- 172.21.15.39:0/731265241 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fdc00f620 con 0x7f7fec103b60 2024-09-21T03:49:06.794 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:06.785+0000 7f7ff271e640 1 -- 172.21.15.39:0/731265241 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7fec1a9040 con 0x7f7fec103b60 2024-09-21T03:49:06.794 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:06.786+0000 7f7ff271e640 1 -- 172.21.15.39:0/731265241 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7fec103f70 con 0x7f7fec103b60 2024-09-21T03:49:06.794 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:06.788+0000 7f7fe97fa640 1 -- 172.21.15.39:0/731265241 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 27) ==== 154643+0+0 (secure 0 0 0) 0x7f7fdc005030 con 0x7f7fec103b60 2024-09-21T03:49:06.794 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:06.789+0000 7f7fe97fa640 1 --2- 172.21.15.39:0/731265241 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f7fd40b6680 0x7f7fd40b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:49:06.794 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:06.789+0000 7f7feb7fe640 1 --2- 172.21.15.39:0/731265241 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f7fd40b6680 0x7f7fd40b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:49:06.794 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:06.789+0000 7f7fe97fa640 1 -- 172.21.15.39:0/731265241 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8484+0+0 (secure 0 0 0) 0x7f7fdc0e73b0 con 0x7f7fec103b60 2024-09-21T03:49:06.794 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:06.789+0000 7f7feb7fe640 1 --2- 172.21.15.39:0/731265241 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f7fd40b6680 0x7f7fd40b8b90 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f7fec108fd0 tx=0x7f7fe0005f90 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:49:06.795 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:06.792+0000 7f7fe97fa640 1 -- 172.21.15.39:0/731265241 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f7fdc0ad920 con 0x7f7fec103b60 2024-09-21T03:49:06.968 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:49:06 smithi161 ceph-mon[34533]: pgmap v417: 97 pgs: 97 active+clean; 10 MiB data, 536 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 23 KiB/s wr, 27 op/s 2024-09-21T03:49:06.974 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:49:06 smithi150 ceph-mon[34289]: pgmap v417: 97 pgs: 97 active+clean; 10 MiB data, 536 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 23 KiB/s wr, 27 op/s 2024-09-21T03:49:07.021 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:07.019+0000 7f7ff271e640 1 -- 172.21.15.39:0/731265241 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f7fec19d7b0 con 0x7f7fec103b60 2024-09-21T03:49:07.022 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:49:06 smithi039 ceph-mon[31211]: pgmap v417: 97 pgs: 97 active+clean; 10 MiB data, 536 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 23 KiB/s wr, 27 op/s 2024-09-21T03:49:07.022 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:07.020+0000 7f7fe97fa640 1 -- 172.21.15.39:0/731265241 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11978 (secure 0 0 0) 0x7f7fdc009de0 con 0x7f7fec103b60 2024-09-21T03:49:07.022 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T03:49:07.022 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":29,"btime":"2024-09-21T03:46:08:982881+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14790,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6835/508995295","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":508995295},{"type":"v1","addr":"172.21.15.39:6835","nonce":508995295}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14802,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6837/3915912607","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":3915912607},{"type":"v1","addr":"172.21.15.39:6837","nonce":3915912607}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":29,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T03:44:15.009872+0000","modified":"2024-09-21T03:46:08.430113+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24695,"mds_1":24667,"mds_2":24659,"mds_3":24646,"mds_4":14868},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14856":{"gid":14856,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.39:6839/185509192","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6838","nonce":185509192},{"type":"v1","addr":"172.21.15.39:6839","nonce":185509192}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14868":{"gid":14868,"name":"a","rank":4,"incarnation":25,"state":"up:active","state_seq":7,"addr":"172.21.15.39:6841/4247190456","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6840","nonce":4247190456},{"type":"v1","addr":"172.21.15.39:6841","nonce":4247190456}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24599":{"gid":24599,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.150:6833/3114643217","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":3114643217},{"type":"v1","addr":"172.21.15.150:6833","nonce":3114643217}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24601":{"gid":24601,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.161:6833/2963990654","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":2963990654},{"type":"v1","addr":"172.21.15.161:6833","nonce":2963990654}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24632":{"gid":24632,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.150:6835/724594008","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6834","nonce":724594008},{"type":"v1","addr":"172.21.15.150:6835","nonce":724594008}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24640":{"gid":24640,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.161:6835/3747882110","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":3747882110},{"type":"v1","addr":"172.21.15.161:6835","nonce":3747882110}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24646":{"gid":24646,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":10,"addr":"172.21.15.161:6837/3451544597","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6836","nonce":3451544597},{"type":"v1","addr":"172.21.15.161:6837","nonce":3451544597}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24659":{"gid":24659,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.150:6837/4017043144","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6836","nonce":4017043144},{"type":"v1","addr":"172.21.15.150:6837","nonce":4017043144}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24667":{"gid":24667,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.161:6839/2190107270","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6838","nonce":2190107270},{"type":"v1","addr":"172.21.15.161:6839","nonce":2190107270}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24695":{"gid":24695,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.150:6839/2949386907","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6838","nonce":2949386907},{"type":"v1","addr":"172.21.15.150:6839","nonce":2949386907}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24695,"qdb_cluster":[24695,24667,24659,24646,14868]},"id":1}]} 2024-09-21T03:49:07.023 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 29 2024-09-21T03:49:07.025 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:07.023+0000 7f7ff271e640 1 -- 172.21.15.39:0/731265241 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f7fd40b6680 msgr2=0x7f7fd40b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:49:07.025 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:07.023+0000 7f7ff271e640 1 --2- 172.21.15.39:0/731265241 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f7fd40b6680 0x7f7fd40b8b90 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f7fec108fd0 tx=0x7f7fe0005f90 comp rx=0 tx=0).stop 2024-09-21T03:49:07.025 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:07.023+0000 7f7ff271e640 1 -- 172.21.15.39:0/731265241 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7fec103b60 msgr2=0x7f7fec19c680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:49:07.025 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:07.023+0000 7f7ff271e640 1 --2- 172.21.15.39:0/731265241 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7fec103b60 0x7f7fec19c680 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f7fdc00d950 tx=0x7f7fdc00de10 comp rx=0 tx=0).stop 2024-09-21T03:49:07.025 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:07.023+0000 7f7ff271e640 1 -- 172.21.15.39:0/731265241 shutdown_connections 2024-09-21T03:49:07.025 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:07.023+0000 7f7ff271e640 1 --2- 172.21.15.39:0/731265241 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f7fd40b6680 0x7f7fd40b8b90 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:07.025 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:07.023+0000 7f7ff271e640 1 --2- 172.21.15.39:0/731265241 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7fec109df0 0x7f7fec1a1ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:07.025 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:07.023+0000 7f7ff271e640 1 --2- 172.21.15.39:0/731265241 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7fec1094a0 0x7f7fec19cc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:07.026 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:07.023+0000 7f7ff271e640 1 --2- 172.21.15.39:0/731265241 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7fec103b60 0x7f7fec19c680 unknown :-1 s=CLOSED pgs=644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:07.026 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:07.023+0000 7f7ff271e640 1 -- 172.21.15.39:0/731265241 >> 172.21.15.39:0/731265241 conn(0x7f7fec07b850 msgr2=0x7f7fec1072b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:49:07.026 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:07.023+0000 7f7ff271e640 1 -- 172.21.15.39:0/731265241 shutdown_connections 2024-09-21T03:49:07.026 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:07.023+0000 7f7ff271e640 1 -- 172.21.15.39:0/731265241 wait complete. 2024-09-21T03:49:07.861 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 51116d5c-77ca-11ef-baf5-efdc52797490 -- ceph tell mds.1:4 damage ls 2024-09-21T03:49:07.968 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:49:07 smithi161 ceph-mon[34533]: from='client.? 172.21.15.39:0/731265241' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:49:07.974 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:49:07 smithi150 ceph-mon[34289]: from='client.? 172.21.15.39:0/731265241' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:49:08.084 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:49:07 smithi039 ceph-mon[31211]: from='client.? 172.21.15.39:0/731265241' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:49:08.121 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/51116d5c-77ca-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:49:08.600 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:08.585+0000 7f9d47bd0640 1 -- 172.21.15.39:0/1242790366 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9d4010deb0 msgr2=0x7f9d40110310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:49:08.600 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:08.585+0000 7f9d47bd0640 1 --2- 172.21.15.39:0/1242790366 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9d4010deb0 0x7f9d40110310 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f9d38009ca0 tx=0x7f9d3802a8e0 comp rx=0 tx=0).stop 2024-09-21T03:49:08.600 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:08.585+0000 7f9d47bd0640 1 -- 172.21.15.39:0/1242790366 shutdown_connections 2024-09-21T03:49:08.600 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:08.585+0000 7f9d47bd0640 1 --2- 172.21.15.39:0/1242790366 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9d4010deb0 0x7f9d40110310 unknown :-1 s=CLOSED pgs=645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:08.600 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:08.585+0000 7f9d47bd0640 1 --2- 172.21.15.39:0/1242790366 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9d40105250 0x7f9d4010d970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:08.600 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:08.585+0000 7f9d47bd0640 1 --2- 172.21.15.39:0/1242790366 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9d40104900 0x7f9d40104d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:08.600 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:08.585+0000 7f9d47bd0640 1 -- 172.21.15.39:0/1242790366 >> 172.21.15.39:0/1242790366 conn(0x7f9d40079720 msgr2=0x7f9d40079b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:49:08.600 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:08.585+0000 7f9d47bd0640 1 -- 172.21.15.39:0/1242790366 shutdown_connections 2024-09-21T03:49:08.601 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:08.585+0000 7f9d47bd0640 1 -- 172.21.15.39:0/1242790366 wait complete. 2024-09-21T03:49:08.601 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:08.585+0000 7f9d47bd0640 1 Processor -- start 2024-09-21T03:49:08.601 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:08.586+0000 7f9d47bd0640 1 -- start start 2024-09-21T03:49:08.601 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:08.586+0000 7f9d47bd0640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9d40104900 0x7f9d40102810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:49:08.601 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:08.586+0000 7f9d47bd0640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9d40105250 0x7f9d40102d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:49:08.601 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:08.586+0000 7f9d47bd0640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9d4010deb0 0x7f9d40103290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:49:08.601 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:08.586+0000 7f9d47bd0640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f9d40113ae0 con 0x7f9d40105250 2024-09-21T03:49:08.601 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:08.586+0000 7f9d47bd0640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f9d40113960 con 0x7f9d4010deb0 2024-09-21T03:49:08.601 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:08.586+0000 7f9d47bd0640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f9d40113c60 con 0x7f9d40104900 2024-09-21T03:49:08.601 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:08.586+0000 7f9d4516b640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9d40105250 0x7f9d40102d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:49:08.603 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:08.586+0000 7f9d4516b640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9d40105250 0x7f9d40102d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:33794/0 (socket says 172.21.15.39:33794) 2024-09-21T03:49:08.603 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:08.586+0000 7f9d4516b640 1 -- 172.21.15.39:0/882836174 learned_addr learned my addr 172.21.15.39:0/882836174 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:49:08.603 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:08.586+0000 7f9d4616d640 1 --2- 172.21.15.39:0/882836174 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9d4010deb0 0x7f9d40103290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:49:08.603 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:08.586+0000 7f9d4596c640 1 --2- 172.21.15.39:0/882836174 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9d40104900 0x7f9d40102810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:49:08.603 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:08.586+0000 7f9d4516b640 1 -- 172.21.15.39:0/882836174 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9d40104900 msgr2=0x7f9d40102810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:49:08.603 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:08.586+0000 7f9d4516b640 1 --2- 172.21.15.39:0/882836174 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9d40104900 0x7f9d40102810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:08.603 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:08.586+0000 7f9d4516b640 1 -- 172.21.15.39:0/882836174 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9d4010deb0 msgr2=0x7f9d40103290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:49:08.603 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:08.586+0000 7f9d4516b640 1 --2- 172.21.15.39:0/882836174 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9d4010deb0 0x7f9d40103290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:08.603 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:08.586+0000 7f9d4516b640 1 -- 172.21.15.39:0/882836174 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9d401a8730 con 0x7f9d40105250 2024-09-21T03:49:08.604 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:08.587+0000 7f9d4516b640 1 --2- 172.21.15.39:0/882836174 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9d40105250 0x7f9d40102d50 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f9d34006fd0 tx=0x7f9d3400b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:49:08.604 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:08.587+0000 7f9d32ffd640 1 -- 172.21.15.39:0/882836174 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d34011830 con 0x7f9d40105250 2024-09-21T03:49:08.604 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:08.587+0000 7f9d32ffd640 1 -- 172.21.15.39:0/882836174 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9d34016070 con 0x7f9d40105250 2024-09-21T03:49:08.604 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:08.587+0000 7f9d47bd0640 1 -- 172.21.15.39:0/882836174 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9d401a8a20 con 0x7f9d40105250 2024-09-21T03:49:08.604 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:08.587+0000 7f9d32ffd640 1 -- 172.21.15.39:0/882836174 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d34010e00 con 0x7f9d40105250 2024-09-21T03:49:08.604 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:08.587+0000 7f9d47bd0640 1 -- 172.21.15.39:0/882836174 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9d401a8fe0 con 0x7f9d40105250 2024-09-21T03:49:08.604 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:08.590+0000 7f9d4e144740 1 Processor -- start 2024-09-21T03:49:08.604 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:08.590+0000 7f9d4e144740 1 -- start start 2024-09-21T03:49:08.604 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:08.590+0000 7f9d32ffd640 1 -- 172.21.15.39:0/882836174 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 27) ==== 154643+0+0 (secure 0 0 0) 0x7f9d3401b460 con 0x7f9d40105250 2024-09-21T03:49:08.604 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:08.590+0000 7f9d4e144740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a699fae070 0x55a69a3552c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:49:08.604 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:08.590+0000 7f9d4e144740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a699f851b0 0x55a69a355800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:49:08.604 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:08.590+0000 7f9d4e144740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a699f8d390 0x55a69a355d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:49:08.604 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:08.590+0000 7f9d4e144740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x55a69a2cbea0 con 0x55a699f851b0 2024-09-21T03:49:08.605 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:08.590+0000 7f9d4e144740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x55a69a2bb2d0 con 0x55a699fae070 2024-09-21T03:49:08.605 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:08.590+0000 7f9d4e144740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55a69a2bf0e0 con 0x55a699f8d390 2024-09-21T03:49:08.605 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:08.590+0000 7f9d4596c640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a699f851b0 0x55a69a355800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:49:08.605 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:08.590+0000 7f9d4596c640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a699f851b0 0x55a69a355800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:33800/0 (socket says 172.21.15.39:33800) 2024-09-21T03:49:08.605 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:08.590+0000 7f9d4596c640 1 -- 172.21.15.39:0/3492863587 learned_addr learned my addr 172.21.15.39:0/3492863587 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:49:08.605 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:08.591+0000 7f9d4516b640 1 --2- 172.21.15.39:0/3492863587 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a699f8d390 0x55a69a355d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:49:08.605 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:08.591+0000 7f9d4616d640 1 --2- 172.21.15.39:0/3492863587 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a699fae070 0x55a69a3552c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:49:08.605 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:08.591+0000 7f9d4596c640 1 -- 172.21.15.39:0/3492863587 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a699f8d390 msgr2=0x55a69a355d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:49:08.605 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:08.591+0000 7f9d4596c640 1 --2- 172.21.15.39:0/3492863587 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a699f8d390 0x55a69a355d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:08.605 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:08.591+0000 7f9d4596c640 1 -- 172.21.15.39:0/3492863587 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a699fae070 msgr2=0x55a69a3552c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:49:08.605 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:08.591+0000 7f9d4596c640 1 --2- 172.21.15.39:0/3492863587 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a699fae070 0x55a69a3552c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:08.605 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:08.591+0000 7f9d4596c640 1 -- 172.21.15.39:0/3492863587 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a69a019650 con 0x55a699f851b0 2024-09-21T03:49:08.605 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:08.591+0000 7f9d32ffd640 1 --2- 172.21.15.39:0/882836174 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f9d2c0b6680 0x7f9d2c0b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:49:08.606 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:08.591+0000 7f9d4596c640 1 --2- 172.21.15.39:0/3492863587 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a699f851b0 0x55a69a355800 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f9d3c00bcb0 tx=0x7f9d3c0107a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=82a284396ca5b504 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:49:08.606 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:08.591+0000 7f9d4616d640 1 --2- 172.21.15.39:0/882836174 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f9d2c0b6680 0x7f9d2c0b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:49:08.606 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:08.591+0000 7f9d32ffd640 1 -- 172.21.15.39:0/882836174 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8484+0+0 (secure 0 0 0) 0x7f9d340e6bc0 con 0x7f9d40105250 2024-09-21T03:49:08.606 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:08.591+0000 7f9d2bfff640 1 -- 172.21.15.39:0/3492863587 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d3c01c480 con 0x55a699f851b0 2024-09-21T03:49:08.606 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:08.591+0000 7f9d2bfff640 1 -- 172.21.15.39:0/3492863587 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9d3c01cad0 con 0x55a699f851b0 2024-09-21T03:49:08.606 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:08.591+0000 7f9d2bfff640 1 -- 172.21.15.39:0/3492863587 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d3c011c30 con 0x55a699f851b0 2024-09-21T03:49:08.606 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:08.592+0000 7f9d4e144740 1 -- 172.21.15.39:0/3492863587 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a699f851b0 msgr2=0x55a69a355800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:49:08.606 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:08.592+0000 7f9d4616d640 1 --2- 172.21.15.39:0/882836174 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f9d2c0b6680 0x7f9d2c0b8b90 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f9d38006010 tx=0x7f9d38009800 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:49:08.606 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:08.592+0000 7f9d4e144740 1 --2- 172.21.15.39:0/3492863587 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a699f851b0 0x55a69a355800 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f9d3c00bcb0 tx=0x7f9d3c0107a0 comp rx=0 tx=0).stop 2024-09-21T03:49:08.606 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:08.592+0000 7f9d4e144740 1 -- 172.21.15.39:0/3492863587 shutdown_connections 2024-09-21T03:49:08.606 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:08.592+0000 7f9d4e144740 1 --2- 172.21.15.39:0/3492863587 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a699f8d390 0x55a69a355d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:08.606 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:08.592+0000 7f9d4e144740 1 --2- 172.21.15.39:0/3492863587 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a699f851b0 0x55a69a355800 unknown :-1 s=CLOSED pgs=647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:08.606 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:08.592+0000 7f9d4e144740 1 --2- 172.21.15.39:0/3492863587 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a699fae070 0x55a69a3552c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:08.607 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:08.592+0000 7f9d4e144740 1 -- 172.21.15.39:0/3492863587 >> 172.21.15.39:0/3492863587 conn(0x55a699fa2460 msgr2=0x55a69a3531b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:49:08.607 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:08.592+0000 7f9d4e144740 1 -- 172.21.15.39:0/3492863587 shutdown_connections 2024-09-21T03:49:08.607 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:08.592+0000 7f9d4e144740 1 -- 172.21.15.39:0/3492863587 wait complete. 2024-09-21T03:49:08.607 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:08.593+0000 7f9d4e144740 1 -- start start 2024-09-21T03:49:08.607 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:08.593+0000 7f9d4e144740 1 Processor -- start 2024-09-21T03:49:08.607 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:08.593+0000 7f9d4e144740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a699f851b0 0x55a69a360980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:49:08.607 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:08.593+0000 7f9d4e144740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a699f8d390 0x55a69a360ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:49:08.607 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:08.593+0000 7f9d4e144740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a699fae070 0x55a69a36cdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:49:08.607 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:08.594+0000 7f9d4e144740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x55a69a11c700 con 0x55a699fae070 2024-09-21T03:49:08.607 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:08.594+0000 7f9d4e144740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x55a69a256160 con 0x55a699f8d390 2024-09-21T03:49:08.607 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:08.594+0000 7f9d4e144740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55a69a363aa0 con 0x55a699f851b0 2024-09-21T03:49:08.607 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:08.594+0000 7f9d4e144740 10 client.0 _ll_register_callbacks cb 0x55a69a26f9e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:49:08.608 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:08.594+0000 7f9d4616d640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a699fae070 0x55a69a36cdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:49:08.608 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:08.594+0000 7f9d4616d640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a699fae070 0x55a69a36cdc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:33810/0 (socket says 172.21.15.39:33810) 2024-09-21T03:49:08.608 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:08.594+0000 7f9d4616d640 1 -- 172.21.15.39:0/1657038127 learned_addr learned my addr 172.21.15.39:0/1657038127 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:49:08.608 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:08.594+0000 7f9d4516b640 1 --2- 172.21.15.39:0/1657038127 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a699f8d390 0x55a69a360ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:49:08.608 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:08.594+0000 7f9d4596c640 1 --2- 172.21.15.39:0/1657038127 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a699f851b0 0x55a69a360980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:49:08.608 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:08.594+0000 7f9d4616d640 1 -- 172.21.15.39:0/1657038127 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a699f851b0 msgr2=0x55a69a360980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:49:08.608 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:08.594+0000 7f9d4616d640 1 --2- 172.21.15.39:0/1657038127 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a699f851b0 0x55a69a360980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:08.608 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:08.594+0000 7f9d4616d640 1 -- 172.21.15.39:0/1657038127 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a699f8d390 msgr2=0x55a69a360ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:49:08.608 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:08.594+0000 7f9d4616d640 1 --2- 172.21.15.39:0/1657038127 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a699f8d390 0x55a69a360ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:08.608 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:08.594+0000 7f9d4616d640 1 -- 172.21.15.39:0/1657038127 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a69a283c90 con 0x55a699fae070 2024-09-21T03:49:08.608 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:08.594+0000 7f9d4516b640 1 --2- 172.21.15.39:0/1657038127 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a699f8d390 0x55a69a360ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:49:08.608 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:08.594+0000 7f9d4616d640 1 --2- 172.21.15.39:0/1657038127 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a699fae070 0x55a69a36cdc0 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7f9d38031e80 tx=0x7f9d3802a8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cee1513c19b6940a server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:49:08.609 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:08.594+0000 7f9d297fa640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-09-21T03:49:08.609 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:08.594+0000 7f9d297fa640 1 -- 172.21.15.39:0/1657038127 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d38003d40 con 0x55a699fae070 2024-09-21T03:49:08.609 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:08.594+0000 7f9d297fa640 1 -- 172.21.15.39:0/1657038127 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8484+0+0 (secure 0 0 0) 0x7f9d380045c0 con 0x55a699fae070 2024-09-21T03:49:08.609 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:08.594+0000 7f9d4e144740 1 -- 172.21.15.39:0/1657038127 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9d3c01cad0 con 0x55a699fae070 2024-09-21T03:49:08.609 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:08.595+0000 7f9d297fa640 1 -- 172.21.15.39:0/1657038127 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f9d380047c0 con 0x55a699fae070 2024-09-21T03:49:08.609 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:08.595+0000 7f9d4e144740 10 client.16188 fetch_fsmap learned FSMap version 29 2024-09-21T03:49:08.609 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:08.595+0000 7f9d4e144740 1 -- 172.21.15.39:0/1657038127 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55a69a36f780 con 0x55a699fae070 2024-09-21T03:49:08.610 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:08.595+0000 7f9d297fa640 1 -- 172.21.15.39:0/1657038127 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9d3c01cad0 con 0x55a699fae070 2024-09-21T03:49:08.610 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:08.595+0000 7f9d297fa640 1 -- 172.21.15.39:0/1657038127 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f9d38059070 con 0x55a699fae070 2024-09-21T03:49:08.610 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:08.595+0000 7f9d4e144740 10 client.16188 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T03:49:08.610 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:08.595+0000 7f9d4e144740 10 client.16188 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.a 2024-09-21T03:49:08.610 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:08.595+0000 7f9d4e144740 20 client.16188 populate_metadata read hostname 'smithi039' 2024-09-21T03:49:08.610 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:08.595+0000 7f9d4e144740 1 --2- 172.21.15.39:0/1657038127 >> [v2:172.21.15.39:6840/4247190456,v1:172.21.15.39:6841/4247190456] conn(0x55a699f6c9f0 0x55a69a372ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:49:08.610 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:08.595+0000 7f9d4e144740 4 client.16188 mds_command: new command op to 14868 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:49:08.610 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:08.595+0000 7f9d4e144740 1 -- 172.21.15.39:0/1657038127 --> [v2:172.21.15.39:6840/4247190456,v1:172.21.15.39:6841/4247190456] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a69a2cddf0 con 0x55a699f6c9f0 2024-09-21T03:49:08.610 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:08.595+0000 7f9d4596c640 1 --2- 172.21.15.39:0/1657038127 >> [v2:172.21.15.39:6840/4247190456,v1:172.21.15.39:6841/4247190456] conn(0x55a699f6c9f0 0x55a69a372ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:49:08.610 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:08.596+0000 7f9d4596c640 1 --2- 172.21.15.39:0/1657038127 >> [v2:172.21.15.39:6840/4247190456,v1:172.21.15.39:6841/4247190456] conn(0x55a699f6c9f0 0x55a69a372ca0 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=c79860122b38586d server_cookie=40289841d1a6cfd7 in_seq=0 out_seq=0 2024-09-21T03:49:08.610 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:08.596+0000 7f9d297fa640 10 client.16188 ms_handle_connect on v2:172.21.15.39:6840/4247190456 2024-09-21T03:49:08.610 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:08.596+0000 7f9d297fa640 1 -- 172.21.15.39:0/1657038127 <== mds.4 v2:172.21.15.39:6840/4247190456 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55a69a2cddf0 con 0x55a699f6c9f0 2024-09-21T03:49:08.610 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:08.596+0000 7f9d297fa640 10 client.16188 handle_command_reply: tid=0 2024-09-21T03:49:08.611 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:08.596+0000 7f9d4e144740 1 client.16188 shutdown 2024-09-21T03:49:08.611 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:08.596+0000 7f9d4e144740 1 -- 172.21.15.39:0/1657038127 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a699fae070 msgr2=0x55a69a36cdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:49:08.611 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:08.596+0000 7f9d4e144740 1 --2- 172.21.15.39:0/1657038127 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a699fae070 0x55a69a36cdc0 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7f9d38031e80 tx=0x7f9d3802a8e0 comp rx=0 tx=0).stop 2024-09-21T03:49:08.611 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:08.597+0000 7f9d4e144740 1 -- 172.21.15.39:0/1657038127 shutdown_connections 2024-09-21T03:49:08.611 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:08.597+0000 7f9d4e144740 1 --2- 172.21.15.39:0/1657038127 >> [v2:172.21.15.39:6840/4247190456,v1:172.21.15.39:6841/4247190456] conn(0x55a699f6c9f0 0x55a69a372ca0 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:08.611 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:08.597+0000 7f9d4e144740 1 --2- 172.21.15.39:0/1657038127 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a699fae070 0x55a69a36cdc0 unknown :-1 s=CLOSED pgs=648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:08.611 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:08.597+0000 7f9d4e144740 1 --2- 172.21.15.39:0/1657038127 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a699f8d390 0x55a69a360ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:08.611 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:08.597+0000 7f9d297fa640 0 client.16188 ms_handle_reset on v2:172.21.15.39:6840/4247190456 2024-09-21T03:49:08.611 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:08.597+0000 7f9d4e144740 1 --2- 172.21.15.39:0/1657038127 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a699f851b0 0x55a69a360980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:08.611 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:08.597+0000 7f9d4e144740 1 -- 172.21.15.39:0/1657038127 >> 172.21.15.39:0/1657038127 conn(0x55a699fa2460 msgr2=0x55a69a358450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:49:08.611 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:08.597+0000 7f9d4e144740 1 -- 172.21.15.39:0/1657038127 shutdown_connections 2024-09-21T03:49:08.611 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:08.597+0000 7f9d4e144740 1 -- 172.21.15.39:0/1657038127 wait complete. 2024-09-21T03:49:08.611 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:08.597+0000 7f9d4e144740 20 client.16188 trim_cache size 0 max 16384 2024-09-21T03:49:08.613 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:08.610+0000 7f9d4e144740 1 Processor -- start 2024-09-21T03:49:08.613 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:08.610+0000 7f9d4e144740 1 -- start start 2024-09-21T03:49:08.613 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:08.611+0000 7f9d4e144740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a699f851b0 0x55a69a356cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:49:08.613 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:08.611+0000 7f9d4e144740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a699f8d390 0x55a69a3625b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:49:08.613 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:08.611+0000 7f9d4e144740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a699fae070 0x55a69a36ccc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:49:08.614 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:08.611+0000 7f9d4e144740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f9d401b19e0 con 0x55a699f8d390 2024-09-21T03:49:08.614 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:08.611+0000 7f9d4e144740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f9d401b3520 con 0x55a699f851b0 2024-09-21T03:49:08.614 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:08.611+0000 7f9d4e144740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f9d401b20b0 con 0x55a699fae070 2024-09-21T03:49:08.614 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:08.611+0000 7f9d4516b640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a699f8d390 0x55a69a3625b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:49:08.614 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:08.611+0000 7f9d4516b640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a699f8d390 0x55a69a3625b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:33820/0 (socket says 172.21.15.39:33820) 2024-09-21T03:49:08.614 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:08.611+0000 7f9d4516b640 1 -- 172.21.15.39:0/2808173007 learned_addr learned my addr 172.21.15.39:0/2808173007 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:49:08.614 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:08.611+0000 7f9d4596c640 1 --2- 172.21.15.39:0/2808173007 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a699f851b0 0x55a69a356cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:49:08.614 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:08.611+0000 7f9d4616d640 1 --2- 172.21.15.39:0/2808173007 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a699fae070 0x55a69a36ccc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:49:08.614 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:08.611+0000 7f9d4516b640 1 -- 172.21.15.39:0/2808173007 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a699fae070 msgr2=0x55a69a36ccc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:49:08.614 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:08.611+0000 7f9d4516b640 1 --2- 172.21.15.39:0/2808173007 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a699fae070 0x55a69a36ccc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:08.614 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:08.611+0000 7f9d4516b640 1 -- 172.21.15.39:0/2808173007 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a699f851b0 msgr2=0x55a69a356cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:49:08.614 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:08.611+0000 7f9d4516b640 1 --2- 172.21.15.39:0/2808173007 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a699f851b0 0x55a69a356cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:08.614 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:08.611+0000 7f9d4516b640 1 -- 172.21.15.39:0/2808173007 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a69a353f70 con 0x55a699f8d390 2024-09-21T03:49:08.615 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:08.611+0000 7f9d4516b640 1 --2- 172.21.15.39:0/2808173007 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a699f8d390 0x55a69a3625b0 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7f9d3401a000 tx=0x7f9d340ea320 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c0277885be3d287 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:49:08.615 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:08.612+0000 7f9d2b7fe640 1 -- 172.21.15.39:0/2808173007 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d340a88d0 con 0x55a699f8d390 2024-09-21T03:49:08.615 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:08.612+0000 7f9d2b7fe640 1 -- 172.21.15.39:0/2808173007 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9d340ac540 con 0x55a699f8d390 2024-09-21T03:49:08.615 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:08.612+0000 7f9d2b7fe640 1 -- 172.21.15.39:0/2808173007 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d340a81a0 con 0x55a699f8d390 2024-09-21T03:49:08.618 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:08.613+0000 7f9d4e144740 1 -- 172.21.15.39:0/2808173007 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a699f8d390 msgr2=0x55a69a3625b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:49:08.619 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:08.613+0000 7f9d4e144740 1 --2- 172.21.15.39:0/2808173007 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a699f8d390 0x55a69a3625b0 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7f9d3401a000 tx=0x7f9d340ea320 comp rx=0 tx=0).stop 2024-09-21T03:49:08.619 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:08.613+0000 7f9d4e144740 1 -- 172.21.15.39:0/2808173007 shutdown_connections 2024-09-21T03:49:08.619 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:08.613+0000 7f9d4e144740 1 --2- 172.21.15.39:0/2808173007 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a699fae070 0x55a69a36ccc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:08.619 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:08.613+0000 7f9d4e144740 1 --2- 172.21.15.39:0/2808173007 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a699f8d390 0x55a69a3625b0 unknown :-1 s=CLOSED pgs=649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:08.619 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:08.613+0000 7f9d4e144740 1 --2- 172.21.15.39:0/2808173007 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a699f851b0 0x55a69a356cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:08.619 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:08.613+0000 7f9d4e144740 1 -- 172.21.15.39:0/2808173007 >> 172.21.15.39:0/2808173007 conn(0x55a699fa2460 msgr2=0x55a69a361ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:49:08.619 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:08.613+0000 7f9d4e144740 1 -- 172.21.15.39:0/2808173007 shutdown_connections 2024-09-21T03:49:08.619 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:08.613+0000 7f9d4e144740 1 -- 172.21.15.39:0/2808173007 wait complete. 2024-09-21T03:49:08.619 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:08.613+0000 7f9d4e144740 1 -- start start 2024-09-21T03:49:08.619 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:08.613+0000 7f9d4e144740 1 Processor -- start 2024-09-21T03:49:08.619 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:08.614+0000 7f9d4e144740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a699f851b0 0x55a69a364f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:49:08.619 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:08.614+0000 7f9d4e144740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a699f8d390 0x55a69a3654b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:49:08.620 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:08.614+0000 7f9d4e144740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a699fae070 0x55a69a3dc660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:49:08.620 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:08.614+0000 7f9d4e144740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f9d401b2e50 con 0x55a699f851b0 2024-09-21T03:49:08.620 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:08.614+0000 7f9d4e144740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f9d401b2780 con 0x55a699f8d390 2024-09-21T03:49:08.620 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:08.614+0000 7f9d4e144740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f9d401af9c0 con 0x55a699fae070 2024-09-21T03:49:08.620 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:08.614+0000 7f9d4e144740 10 client.0 _ll_register_callbacks cb 0x7f9d38031d40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:49:08.620 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:08.614+0000 7f9d4596c640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a699f851b0 0x55a69a364f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:49:08.620 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:08.614+0000 7f9d4596c640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a699f851b0 0x55a69a364f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:33834/0 (socket says 172.21.15.39:33834) 2024-09-21T03:49:08.620 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:08.614+0000 7f9d4596c640 1 -- 172.21.15.39:0/1831108285 learned_addr learned my addr 172.21.15.39:0/1831108285 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:49:08.620 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:08.614+0000 7f9d4616d640 1 --2- 172.21.15.39:0/1831108285 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a699fae070 0x55a69a3dc660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:49:08.620 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:08.614+0000 7f9d4516b640 1 --2- 172.21.15.39:0/1831108285 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a699f8d390 0x55a69a3654b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:49:08.620 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:08.614+0000 7f9d4596c640 1 -- 172.21.15.39:0/1831108285 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a699fae070 msgr2=0x55a69a3dc660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:49:08.620 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:08.614+0000 7f9d4596c640 1 --2- 172.21.15.39:0/1831108285 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a699fae070 0x55a69a3dc660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:08.620 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:08.614+0000 7f9d4596c640 1 -- 172.21.15.39:0/1831108285 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a699f8d390 msgr2=0x55a69a3654b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:49:08.620 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:08.614+0000 7f9d4596c640 1 --2- 172.21.15.39:0/1831108285 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a699f8d390 0x55a69a3654b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:08.620 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:08.614+0000 7f9d4596c640 1 -- 172.21.15.39:0/1831108285 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a69a34e5d0 con 0x55a699f851b0 2024-09-21T03:49:08.621 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:08.614+0000 7f9d4516b640 1 --2- 172.21.15.39:0/1831108285 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a699f8d390 0x55a69a3654b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:49:08.621 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:08.614+0000 7f9d4596c640 1 --2- 172.21.15.39:0/1831108285 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a699f851b0 0x55a69a364f70 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x55a69a353f40 tx=0x7f9d3c00bf80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=adeff48911818a35 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:49:08.621 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:08.614+0000 7f9d297fa640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-09-21T03:49:08.621 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:08.615+0000 7f9d297fa640 1 -- 172.21.15.39:0/1831108285 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d3c005040 con 0x55a699f851b0 2024-09-21T03:49:08.621 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:08.615+0000 7f9d297fa640 1 -- 172.21.15.39:0/1831108285 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8484+0+0 (secure 0 0 0) 0x7f9d3c0051e0 con 0x55a699f851b0 2024-09-21T03:49:08.621 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:08.615+0000 7f9d4e144740 1 -- 172.21.15.39:0/1831108285 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9d340ac540 con 0x55a699f851b0 2024-09-21T03:49:08.621 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:08.615+0000 7f9d297fa640 1 -- 172.21.15.39:0/1831108285 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f9d3c0053e0 con 0x55a699f851b0 2024-09-21T03:49:08.621 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:08.615+0000 7f9d4e144740 10 client.16194 fetch_fsmap learned FSMap version 29 2024-09-21T03:49:08.621 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:08.615+0000 7f9d4e144740 1 -- 172.21.15.39:0/1831108285 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55a69a3dd990 con 0x55a699f851b0 2024-09-21T03:49:08.621 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:08.615+0000 7f9d297fa640 1 -- 172.21.15.39:0/1831108285 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9d340ac540 con 0x55a699f851b0 2024-09-21T03:49:08.621 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:08.615+0000 7f9d297fa640 1 -- 172.21.15.39:0/1831108285 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f9d3c036070 con 0x55a699f851b0 2024-09-21T03:49:08.621 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:08.615+0000 7f9d4e144740 10 client.16194 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T03:49:08.621 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:08.615+0000 7f9d4e144740 10 client.16194 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.a 2024-09-21T03:49:08.622 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:08.615+0000 7f9d4e144740 20 client.16194 populate_metadata read hostname 'smithi039' 2024-09-21T03:49:08.622 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:08.615+0000 7f9d4e144740 1 --2- 172.21.15.39:0/1831108285 >> [v2:172.21.15.39:6840/4247190456,v1:172.21.15.39:6841/4247190456] conn(0x55a699f41da0 0x55a69a3dffe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:49:08.622 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:08.615+0000 7f9d4e144740 4 client.16194 mds_command: new command op to 14868 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T03:49:08.622 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:08.615+0000 7f9d4e144740 1 -- 172.21.15.39:0/1831108285 --> [v2:172.21.15.39:6840/4247190456,v1:172.21.15.39:6841/4247190456] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55a69a2c98a0 con 0x55a699f41da0 2024-09-21T03:49:08.622 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:08.615+0000 7f9d4516b640 1 --2- 172.21.15.39:0/1831108285 >> [v2:172.21.15.39:6840/4247190456,v1:172.21.15.39:6841/4247190456] conn(0x55a699f41da0 0x55a69a3dffe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:49:08.622 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:08.616+0000 7f9d297fa640 10 client.16194 ms_handle_connect on v2:172.21.15.39:6840/4247190456 2024-09-21T03:49:08.622 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:08.616+0000 7f9d4516b640 1 --2- 172.21.15.39:0/1831108285 >> [v2:172.21.15.39:6840/4247190456,v1:172.21.15.39:6841/4247190456] conn(0x55a699f41da0 0x55a69a3dffe0 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=d15f1d82319b74d4 server_cookie=81716e13761fdacf in_seq=0 out_seq=0 2024-09-21T03:49:08.622 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:08.616+0000 7f9d297fa640 1 -- 172.21.15.39:0/1831108285 <== mds.4 v2:172.21.15.39:6840/4247190456 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55a69a2c98a0 con 0x55a699f41da0 2024-09-21T03:49:08.622 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:08.616+0000 7f9d297fa640 10 client.16194 handle_command_reply: tid=0 2024-09-21T03:49:08.622 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:08.616+0000 7f9d4e144740 1 client.16194 shutdown 2024-09-21T03:49:08.622 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:08.616+0000 7f9d4e144740 1 -- 172.21.15.39:0/1831108285 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a699f851b0 msgr2=0x55a69a364f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:49:08.622 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:08.616+0000 7f9d4e144740 1 --2- 172.21.15.39:0/1831108285 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a699f851b0 0x55a69a364f70 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x55a69a353f40 tx=0x7f9d3c00bf80 comp rx=0 tx=0).stop 2024-09-21T03:49:08.622 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:08.616+0000 7f9d4e144740 1 -- 172.21.15.39:0/1831108285 shutdown_connections 2024-09-21T03:49:08.622 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:08.616+0000 7f9d4e144740 1 --2- 172.21.15.39:0/1831108285 >> [v2:172.21.15.39:6840/4247190456,v1:172.21.15.39:6841/4247190456] conn(0x55a699f41da0 0x55a69a3dffe0 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:08.623 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:08.616+0000 7f9d4e144740 1 --2- 172.21.15.39:0/1831108285 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a699fae070 0x55a69a3dc660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:08.623 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:08.616+0000 7f9d4e144740 1 --2- 172.21.15.39:0/1831108285 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a699f8d390 0x55a69a3654b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:08.623 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:08.616+0000 7f9d4e144740 1 --2- 172.21.15.39:0/1831108285 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a699f851b0 0x55a69a364f70 unknown :-1 s=CLOSED pgs=650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:08.623 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:08.616+0000 7f9d297fa640 0 client.16194 ms_handle_reset on v2:172.21.15.39:6840/4247190456 2024-09-21T03:49:08.623 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:08.616+0000 7f9d4e144740 1 -- 172.21.15.39:0/1831108285 >> 172.21.15.39:0/1831108285 conn(0x55a699fa2460 msgr2=0x55a69a356a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:49:08.623 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:08.616+0000 7f9d4e144740 1 -- 172.21.15.39:0/1831108285 shutdown_connections 2024-09-21T03:49:08.623 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:08.616+0000 7f9d4e144740 1 -- 172.21.15.39:0/1831108285 wait complete. 2024-09-21T03:49:08.623 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:08.617+0000 7f9d4e144740 20 client.16194 trim_cache size 0 max 16384 2024-09-21T03:49:08.623 INFO:teuthology.orchestra.run.smithi039.stdout:[] 2024-09-21T03:49:08.624 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:08.617+0000 7f9d47bd0640 1 -- 172.21.15.39:0/882836174 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f9d2c0b6680 msgr2=0x7f9d2c0b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:49:08.624 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:08.617+0000 7f9d47bd0640 1 --2- 172.21.15.39:0/882836174 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f9d2c0b6680 0x7f9d2c0b8b90 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f9d38006010 tx=0x7f9d38009800 comp rx=0 tx=0).stop 2024-09-21T03:49:08.624 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:08.617+0000 7f9d47bd0640 1 -- 172.21.15.39:0/882836174 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9d40105250 msgr2=0x7f9d40102d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:49:08.624 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:08.617+0000 7f9d47bd0640 1 --2- 172.21.15.39:0/882836174 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9d40105250 0x7f9d40102d50 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f9d34006fd0 tx=0x7f9d3400b8e0 comp rx=0 tx=0).stop 2024-09-21T03:49:08.624 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:08.617+0000 7f9d47bd0640 1 -- 172.21.15.39:0/882836174 shutdown_connections 2024-09-21T03:49:08.624 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:08.617+0000 7f9d47bd0640 1 --2- 172.21.15.39:0/882836174 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f9d2c0b6680 0x7f9d2c0b8b90 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:08.624 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:08.617+0000 7f9d47bd0640 1 --2- 172.21.15.39:0/882836174 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9d4010deb0 0x7f9d40103290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:08.624 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:08.617+0000 7f9d47bd0640 1 --2- 172.21.15.39:0/882836174 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9d40105250 0x7f9d40102d50 unknown :-1 s=CLOSED pgs=646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:08.624 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:08.617+0000 7f9d47bd0640 1 --2- 172.21.15.39:0/882836174 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9d40104900 0x7f9d40102810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:08.624 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:08.617+0000 7f9d47bd0640 1 -- 172.21.15.39:0/882836174 >> 172.21.15.39:0/882836174 conn(0x7f9d40079720 msgr2=0x7f9d40100940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:49:08.625 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:08.617+0000 7f9d47bd0640 1 -- 172.21.15.39:0/882836174 shutdown_connections 2024-09-21T03:49:08.625 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:08.617+0000 7f9d47bd0640 1 -- 172.21.15.39:0/882836174 wait complete. 2024-09-21T03:49:08.833 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:49:08 smithi039 ceph-mon[31211]: pgmap v418: 97 pgs: 97 active+clean; 10 MiB data, 536 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 23 KiB/s wr, 23 op/s 2024-09-21T03:49:08.968 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:49:08 smithi161 ceph-mon[34533]: pgmap v418: 97 pgs: 97 active+clean; 10 MiB data, 536 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 23 KiB/s wr, 23 op/s 2024-09-21T03:49:08.975 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:49:08 smithi150 ceph-mon[34289]: pgmap v418: 97 pgs: 97 active+clean; 10 MiB data, 536 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 23 KiB/s wr, 23 op/s 2024-09-21T03:49:09.449 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 51116d5c-77ca-11ef-baf5-efdc52797490 -- ceph tell mds.1:3 damage ls 2024-09-21T03:49:09.715 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/51116d5c-77ca-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:49:10.186 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:10.183+0000 7f9f24a17640 1 -- 172.21.15.39:0/2543680002 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9f20108d70 msgr2=0x7f9f2010f5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:49:10.186 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:10.183+0000 7f9f24a17640 1 --2- 172.21.15.39:0/2543680002 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9f20108d70 0x7f9f2010f5a0 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7f9f18009ca0 tx=0x7f9f1802a8c0 comp rx=0 tx=0).stop 2024-09-21T03:49:10.186 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:10.184+0000 7f9f24a17640 1 -- 172.21.15.39:0/2543680002 shutdown_connections 2024-09-21T03:49:10.186 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:10.184+0000 7f9f24a17640 1 --2- 172.21.15.39:0/2543680002 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9f20108d70 0x7f9f2010f5a0 unknown :-1 s=CLOSED pgs=651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:10.186 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:10.184+0000 7f9f24a17640 1 --2- 172.21.15.39:0/2543680002 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9f20108420 0x7f9f20108830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:10.186 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:10.184+0000 7f9f24a17640 1 --2- 172.21.15.39:0/2543680002 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9f20102ae0 0x7f9f20102ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:10.186 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:10.184+0000 7f9f24a17640 1 -- 172.21.15.39:0/2543680002 >> 172.21.15.39:0/2543680002 conn(0x7f9f20079770 msgr2=0x7f9f20079b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:49:10.187 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:10.184+0000 7f9f24a17640 1 -- 172.21.15.39:0/2543680002 shutdown_connections 2024-09-21T03:49:10.187 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:10.184+0000 7f9f24a17640 1 -- 172.21.15.39:0/2543680002 wait complete. 2024-09-21T03:49:10.188 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:10.185+0000 7f9f24a17640 1 Processor -- start 2024-09-21T03:49:10.188 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:10.185+0000 7f9f24a17640 1 -- start start 2024-09-21T03:49:10.188 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:10.185+0000 7f9f24a17640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9f20102ae0 0x7f9f200ff670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:49:10.189 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:10.185+0000 7f9f24a17640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9f20108420 0x7f9f200ffbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:49:10.189 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:10.185+0000 7f9f24a17640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9f20108d70 0x7f9f201000f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:49:10.189 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:10.185+0000 7f9f24a17640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f9f20112a30 con 0x7f9f20108420 2024-09-21T03:49:10.189 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:10.185+0000 7f9f24a17640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f9f201128b0 con 0x7f9f20108d70 2024-09-21T03:49:10.189 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:10.185+0000 7f9f24a17640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f9f20112bb0 con 0x7f9f20102ae0 2024-09-21T03:49:10.189 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:10.185+0000 7f9f1dd9b640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9f20108420 0x7f9f200ffbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:49:10.189 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:10.185+0000 7f9f1dd9b640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9f20108420 0x7f9f200ffbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:33852/0 (socket says 172.21.15.39:33852) 2024-09-21T03:49:10.189 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:10.185+0000 7f9f1dd9b640 1 -- 172.21.15.39:0/1711008496 learned_addr learned my addr 172.21.15.39:0/1711008496 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:49:10.189 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:10.185+0000 7f9f1e59c640 1 --2- 172.21.15.39:0/1711008496 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9f20102ae0 0x7f9f200ff670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:49:10.189 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:10.185+0000 7f9f1ed9d640 1 --2- 172.21.15.39:0/1711008496 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9f20108d70 0x7f9f201000f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:49:10.189 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:10.185+0000 7f9f1dd9b640 1 -- 172.21.15.39:0/1711008496 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9f20102ae0 msgr2=0x7f9f200ff670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:49:10.189 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:10.185+0000 7f9f1dd9b640 1 --2- 172.21.15.39:0/1711008496 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9f20102ae0 0x7f9f200ff670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:10.189 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:10.185+0000 7f9f1dd9b640 1 -- 172.21.15.39:0/1711008496 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9f20108d70 msgr2=0x7f9f201000f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:49:10.190 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:10.185+0000 7f9f1dd9b640 1 --2- 172.21.15.39:0/1711008496 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9f20108d70 0x7f9f201000f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:10.190 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:10.186+0000 7f9f1dd9b640 1 -- 172.21.15.39:0/1711008496 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9f201a8770 con 0x7f9f20108420 2024-09-21T03:49:10.190 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:10.186+0000 7f9f1dd9b640 1 --2- 172.21.15.39:0/1711008496 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9f20108420 0x7f9f200ffbb0 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f9f1400e9f0 tx=0x7f9f1400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:49:10.190 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:10.186+0000 7f9f0f7fe640 1 -- 172.21.15.39:0/1711008496 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f1400ccc0 con 0x7f9f20108420 2024-09-21T03:49:10.190 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:10.186+0000 7f9f0f7fe640 1 -- 172.21.15.39:0/1711008496 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9f140108e0 con 0x7f9f20108420 2024-09-21T03:49:10.190 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:10.186+0000 7f9f24a17640 1 -- 172.21.15.39:0/1711008496 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9f201a8a60 con 0x7f9f20108420 2024-09-21T03:49:10.190 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:10.186+0000 7f9f0f7fe640 1 -- 172.21.15.39:0/1711008496 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f14018860 con 0x7f9f20108420 2024-09-21T03:49:10.190 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:10.186+0000 7f9f24a17640 1 -- 172.21.15.39:0/1711008496 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9f201a8ff0 con 0x7f9f20108420 2024-09-21T03:49:10.190 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:10.188+0000 7f9f2af8b740 1 Processor -- start 2024-09-21T03:49:10.191 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:10.189+0000 7f9f2af8b740 1 -- start start 2024-09-21T03:49:10.192 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:10.189+0000 7f9f2af8b740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56239f195070 0x56239f53c2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:49:10.192 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:10.189+0000 7f9f2af8b740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56239f16c1b0 0x56239f53c800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:49:10.192 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:10.189+0000 7f9f2af8b740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56239f174390 0x56239f53cd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:49:10.192 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:10.189+0000 7f9f2af8b740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x56239f4b2ea0 con 0x56239f174390 2024-09-21T03:49:10.192 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:10.189+0000 7f9f2af8b740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x56239f4a22d0 con 0x56239f195070 2024-09-21T03:49:10.193 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:10.189+0000 7f9f2af8b740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x56239f4a60e0 con 0x56239f16c1b0 2024-09-21T03:49:10.193 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:10.189+0000 7f9f0f7fe640 1 -- 172.21.15.39:0/1711008496 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 27) ==== 154643+0+0 (secure 0 0 0) 0x7f9f14018a00 con 0x7f9f20108420 2024-09-21T03:49:10.193 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:10.189+0000 7f9f1e59c640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56239f16c1b0 0x56239f53c800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:49:10.193 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:10.189+0000 7f9f1e59c640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56239f16c1b0 0x56239f53c800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.39:39812/0 (socket says 172.21.15.39:39812) 2024-09-21T03:49:10.193 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:10.189+0000 7f9f1e59c640 1 -- 172.21.15.39:0/791855518 learned_addr learned my addr 172.21.15.39:0/791855518 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:49:10.193 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:10.189+0000 7f9f1ed9d640 1 --2- 172.21.15.39:0/791855518 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56239f195070 0x56239f53c2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:49:10.193 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:10.190+0000 7f9f1dd9b640 1 --2- 172.21.15.39:0/791855518 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56239f174390 0x56239f53cd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:49:10.193 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:10.190+0000 7f9f1e59c640 1 -- 172.21.15.39:0/791855518 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56239f195070 msgr2=0x56239f53c2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:49:10.193 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:10.190+0000 7f9f1e59c640 1 --2- 172.21.15.39:0/791855518 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56239f195070 0x56239f53c2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:10.193 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:10.190+0000 7f9f1e59c640 1 -- 172.21.15.39:0/791855518 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56239f174390 msgr2=0x56239f53cd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:49:10.193 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:10.190+0000 7f9f1e59c640 1 --2- 172.21.15.39:0/791855518 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56239f174390 0x56239f53cd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:10.193 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:10.190+0000 7f9f1e59c640 1 -- 172.21.15.39:0/791855518 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56239f200650 con 0x56239f16c1b0 2024-09-21T03:49:10.193 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:10.190+0000 7f9f1e59c640 1 --2- 172.21.15.39:0/791855518 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56239f16c1b0 0x56239f53c800 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f9f10009ca0 tx=0x7f9f10011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ed0b6fe05f91fdae server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:49:10.194 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:10.190+0000 7f9f0cff9640 1 -- 172.21.15.39:0/791855518 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f10005220 con 0x56239f16c1b0 2024-09-21T03:49:10.194 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:10.190+0000 7f9f0cff9640 1 -- 172.21.15.39:0/791855518 <== mon.1 v2:172.21.15.161:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9f10004c50 con 0x56239f16c1b0 2024-09-21T03:49:10.194 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:10.190+0000 7f9f0cff9640 1 -- 172.21.15.39:0/791855518 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f10019690 con 0x56239f16c1b0 2024-09-21T03:49:10.194 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:10.190+0000 7f9f0f7fe640 1 --2- 172.21.15.39:0/1711008496 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f9f080b6700 0x7f9f080b8c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:49:10.194 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:10.191+0000 7f9f1ed9d640 1 --2- 172.21.15.39:0/1711008496 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f9f080b6700 0x7f9f080b8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:49:10.194 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:10.191+0000 7f9f0f7fe640 1 -- 172.21.15.39:0/1711008496 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8484+0+0 (secure 0 0 0) 0x7f9f140e7930 con 0x7f9f20108420 2024-09-21T03:49:10.198 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:10.191+0000 7f9f2af8b740 1 -- 172.21.15.39:0/791855518 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56239f16c1b0 msgr2=0x56239f53c800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:49:10.198 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:10.191+0000 7f9f1ed9d640 1 --2- 172.21.15.39:0/1711008496 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f9f080b6700 0x7f9f080b8c10 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f9f1802f500 tx=0x7f9f1802f310 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:49:10.198 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:10.191+0000 7f9f2af8b740 1 --2- 172.21.15.39:0/791855518 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56239f16c1b0 0x56239f53c800 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f9f10009ca0 tx=0x7f9f10011810 comp rx=0 tx=0).stop 2024-09-21T03:49:10.198 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:10.191+0000 7f9f2af8b740 1 -- 172.21.15.39:0/791855518 shutdown_connections 2024-09-21T03:49:10.198 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:10.191+0000 7f9f2af8b740 1 --2- 172.21.15.39:0/791855518 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56239f174390 0x56239f53cd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:10.198 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:10.191+0000 7f9f2af8b740 1 --2- 172.21.15.39:0/791855518 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56239f16c1b0 0x56239f53c800 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:10.198 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:10.191+0000 7f9f2af8b740 1 --2- 172.21.15.39:0/791855518 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56239f195070 0x56239f53c2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:10.198 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:10.191+0000 7f9f2af8b740 1 -- 172.21.15.39:0/791855518 >> 172.21.15.39:0/791855518 conn(0x56239f189460 msgr2=0x56239f53a1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:49:10.198 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:10.191+0000 7f9f2af8b740 1 -- 172.21.15.39:0/791855518 shutdown_connections 2024-09-21T03:49:10.198 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:10.192+0000 7f9f2af8b740 1 -- 172.21.15.39:0/791855518 wait complete. 2024-09-21T03:49:10.198 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:10.192+0000 7f9f2af8b740 1 -- start start 2024-09-21T03:49:10.198 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:10.192+0000 7f9f2af8b740 1 Processor -- start 2024-09-21T03:49:10.199 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:10.192+0000 7f9f2af8b740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56239f16c1b0 0x56239f547900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:49:10.199 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:10.192+0000 7f9f2af8b740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56239f174390 0x56239f547e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:49:10.199 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:10.192+0000 7f9f2af8b740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56239f195070 0x56239f5483c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:49:10.199 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:10.192+0000 7f9f2af8b740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x56239f303700 con 0x56239f174390 2024-09-21T03:49:10.199 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:10.192+0000 7f9f2af8b740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x56239f43d160 con 0x56239f195070 2024-09-21T03:49:10.199 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:10.192+0000 7f9f2af8b740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x56239f54aaa0 con 0x56239f16c1b0 2024-09-21T03:49:10.199 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:10.192+0000 7f9f2af8b740 10 client.0 _ll_register_callbacks cb 0x56239f4569e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:49:10.199 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:10.192+0000 7f9f1dd9b640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56239f174390 0x56239f547e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:49:10.199 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:10.192+0000 7f9f1dd9b640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56239f174390 0x56239f547e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:33868/0 (socket says 172.21.15.39:33868) 2024-09-21T03:49:10.199 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:10.192+0000 7f9f1dd9b640 1 -- 172.21.15.39:0/594150745 learned_addr learned my addr 172.21.15.39:0/594150745 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:49:10.199 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:10.193+0000 7f9f1e59c640 1 --2- 172.21.15.39:0/594150745 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56239f16c1b0 0x56239f547900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:49:10.199 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:10.193+0000 7f9f1ed9d640 1 --2- 172.21.15.39:0/594150745 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56239f195070 0x56239f5483c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:49:10.199 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:10.193+0000 7f9f1dd9b640 1 -- 172.21.15.39:0/594150745 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56239f16c1b0 msgr2=0x56239f547900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:49:10.199 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:10.193+0000 7f9f1dd9b640 1 --2- 172.21.15.39:0/594150745 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56239f16c1b0 0x56239f547900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:10.199 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:10.193+0000 7f9f1dd9b640 1 -- 172.21.15.39:0/594150745 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56239f195070 msgr2=0x56239f5483c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:49:10.200 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:10.193+0000 7f9f1dd9b640 1 --2- 172.21.15.39:0/594150745 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56239f195070 0x56239f5483c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:10.200 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:10.193+0000 7f9f1dd9b640 1 -- 172.21.15.39:0/594150745 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56239f46ac90 con 0x56239f174390 2024-09-21T03:49:10.200 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:10.193+0000 7f9f1dd9b640 1 --2- 172.21.15.39:0/594150745 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56239f174390 0x56239f547e40 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7f9f1400e9c0 tx=0x7f9f140ad6e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=43c06e758e275a8f server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:49:10.200 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:10.193+0000 7f9f05ffb640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-09-21T03:49:10.200 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:10.193+0000 7f9f05ffb640 1 -- 172.21.15.39:0/594150745 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f140ad360 con 0x56239f174390 2024-09-21T03:49:10.200 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:10.193+0000 7f9f05ffb640 1 -- 172.21.15.39:0/594150745 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8484+0+0 (secure 0 0 0) 0x7f9f140a91a0 con 0x56239f174390 2024-09-21T03:49:10.200 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:10.193+0000 7f9f2af8b740 1 -- 172.21.15.39:0/594150745 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9f10004c50 con 0x56239f174390 2024-09-21T03:49:10.200 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:10.193+0000 7f9f05ffb640 1 -- 172.21.15.39:0/594150745 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f9f140b2280 con 0x56239f174390 2024-09-21T03:49:10.200 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:10.193+0000 7f9f2af8b740 10 client.16206 fetch_fsmap learned FSMap version 29 2024-09-21T03:49:10.200 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:10.193+0000 7f9f2af8b740 1 -- 172.21.15.39:0/594150745 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x56239f556740 con 0x56239f174390 2024-09-21T03:49:10.200 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:10.194+0000 7f9f05ffb640 1 -- 172.21.15.39:0/594150745 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9f10004c50 con 0x56239f174390 2024-09-21T03:49:10.200 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:10.194+0000 7f9f05ffb640 1 -- 172.21.15.39:0/594150745 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f9f140ac520 con 0x56239f174390 2024-09-21T03:49:10.200 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:10.194+0000 7f9f2af8b740 10 client.16206 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T03:49:10.200 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:10.194+0000 7f9f2af8b740 10 client.16206 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-21T03:49:10.200 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:10.194+0000 7f9f2af8b740 20 client.16206 populate_metadata read hostname 'smithi039' 2024-09-21T03:49:10.201 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:10.194+0000 7f9f2af8b740 1 --2- 172.21.15.39:0/594150745 >> [v2:172.21.15.161:6836/3451544597,v1:172.21.15.161:6837/3451544597] conn(0x56239f1539f0 0x56239f559bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:49:10.201 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:10.194+0000 7f9f2af8b740 4 client.16206 mds_command: new command op to 24646 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:49:10.201 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:10.194+0000 7f9f2af8b740 1 -- 172.21.15.39:0/594150745 --> [v2:172.21.15.161:6836/3451544597,v1:172.21.15.161:6837/3451544597] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56239f4b4df0 con 0x56239f1539f0 2024-09-21T03:49:10.201 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:10.194+0000 7f9f1e59c640 1 --2- 172.21.15.39:0/594150745 >> [v2:172.21.15.161:6836/3451544597,v1:172.21.15.161:6837/3451544597] conn(0x56239f1539f0 0x56239f559bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:49:10.201 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:10.195+0000 7f9f1e59c640 1 --2- 172.21.15.39:0/594150745 >> [v2:172.21.15.161:6836/3451544597,v1:172.21.15.161:6837/3451544597] conn(0x56239f1539f0 0x56239f559bb0 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=698479dacaa7bd23 server_cookie=c4b911239147377 in_seq=0 out_seq=0 2024-09-21T03:49:10.201 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:10.195+0000 7f9f05ffb640 10 client.16206 ms_handle_connect on v2:172.21.15.161:6836/3451544597 2024-09-21T03:49:10.201 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:10.196+0000 7f9f05ffb640 1 -- 172.21.15.39:0/594150745 <== mds.3 v2:172.21.15.161:6836/3451544597 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x56239f4b4df0 con 0x56239f1539f0 2024-09-21T03:49:10.201 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:10.196+0000 7f9f05ffb640 10 client.16206 handle_command_reply: tid=0 2024-09-21T03:49:10.201 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:10.196+0000 7f9f2af8b740 1 client.16206 shutdown 2024-09-21T03:49:10.201 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:10.196+0000 7f9f2af8b740 1 -- 172.21.15.39:0/594150745 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56239f174390 msgr2=0x56239f547e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:49:10.201 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:10.196+0000 7f9f2af8b740 1 --2- 172.21.15.39:0/594150745 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56239f174390 0x56239f547e40 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7f9f1400e9c0 tx=0x7f9f140ad6e0 comp rx=0 tx=0).stop 2024-09-21T03:49:10.201 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:10.196+0000 7f9f2af8b740 1 -- 172.21.15.39:0/594150745 shutdown_connections 2024-09-21T03:49:10.201 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:10.196+0000 7f9f2af8b740 1 --2- 172.21.15.39:0/594150745 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56239f195070 0x56239f5483c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:10.201 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:10.196+0000 7f9f2af8b740 1 --2- 172.21.15.39:0/594150745 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56239f174390 0x56239f547e40 unknown :-1 s=CLOSED pgs=653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:10.201 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:10.196+0000 7f9f2af8b740 1 --2- 172.21.15.39:0/594150745 >> [v2:172.21.15.161:6836/3451544597,v1:172.21.15.161:6837/3451544597] conn(0x56239f1539f0 0x56239f559bb0 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:10.202 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:10.196+0000 7f9f2af8b740 1 --2- 172.21.15.39:0/594150745 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56239f16c1b0 0x56239f547900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:10.202 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:10.196+0000 7f9f05ffb640 0 client.16206 ms_handle_reset on v2:172.21.15.161:6836/3451544597 2024-09-21T03:49:10.202 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:10.196+0000 7f9f2af8b740 1 -- 172.21.15.39:0/594150745 >> 172.21.15.39:0/594150745 conn(0x56239f189460 msgr2=0x56239f53f4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:49:10.202 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:10.196+0000 7f9f2af8b740 1 -- 172.21.15.39:0/594150745 shutdown_connections 2024-09-21T03:49:10.202 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:10.196+0000 7f9f2af8b740 1 -- 172.21.15.39:0/594150745 wait complete. 2024-09-21T03:49:10.202 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:10.196+0000 7f9f2af8b740 20 client.16206 trim_cache size 0 max 16384 2024-09-21T03:49:10.212 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:10.209+0000 7f9f2af8b740 1 Processor -- start 2024-09-21T03:49:10.212 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:10.209+0000 7f9f2af8b740 1 -- start start 2024-09-21T03:49:10.212 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:10.209+0000 7f9f2af8b740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56239f16c1b0 0x56239f5430b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:49:10.212 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:10.209+0000 7f9f2af8b740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56239f174390 0x56239f549530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:49:10.212 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:10.209+0000 7f9f2af8b740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56239f195070 0x56239f544af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:49:10.212 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:10.209+0000 7f9f2af8b740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f9f201b1820 con 0x56239f16c1b0 2024-09-21T03:49:10.213 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:10.209+0000 7f9f2af8b740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f9f201b3360 con 0x56239f174390 2024-09-21T03:49:10.213 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:10.209+0000 7f9f2af8b740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f9f201b1ef0 con 0x56239f195070 2024-09-21T03:49:10.213 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:10.210+0000 7f9f1e59c640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56239f16c1b0 0x56239f5430b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:49:10.213 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:10.210+0000 7f9f1e59c640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56239f16c1b0 0x56239f5430b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:33874/0 (socket says 172.21.15.39:33874) 2024-09-21T03:49:10.213 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:10.210+0000 7f9f1e59c640 1 -- 172.21.15.39:0/1281904684 learned_addr learned my addr 172.21.15.39:0/1281904684 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:49:10.213 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:10.210+0000 7f9f1dd9b640 1 --2- 172.21.15.39:0/1281904684 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56239f174390 0x56239f549530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:49:10.213 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:10.210+0000 7f9f1ed9d640 1 --2- 172.21.15.39:0/1281904684 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56239f195070 0x56239f544af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:49:10.213 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:10.210+0000 7f9f1e59c640 1 -- 172.21.15.39:0/1281904684 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56239f195070 msgr2=0x56239f544af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:49:10.213 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:10.210+0000 7f9f1e59c640 1 --2- 172.21.15.39:0/1281904684 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56239f195070 0x56239f544af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:10.213 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:10.210+0000 7f9f1e59c640 1 -- 172.21.15.39:0/1281904684 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56239f174390 msgr2=0x56239f549530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:49:10.213 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:10.210+0000 7f9f1e59c640 1 --2- 172.21.15.39:0/1281904684 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56239f174390 0x56239f549530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:10.213 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:10.210+0000 7f9f1e59c640 1 -- 172.21.15.39:0/1281904684 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56239f53eb00 con 0x56239f16c1b0 2024-09-21T03:49:10.213 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:10.210+0000 7f9f1dd9b640 1 --2- 172.21.15.39:0/1281904684 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56239f174390 0x56239f549530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:49:10.213 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:10.210+0000 7f9f1e59c640 1 --2- 172.21.15.39:0/1281904684 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56239f16c1b0 0x56239f5430b0 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7f9f10009ca0 tx=0x7f9f10009ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d8e07dfb91646b8 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:49:10.213 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:10.210+0000 7f9f07fff640 1 -- 172.21.15.39:0/1281904684 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f10005650 con 0x56239f16c1b0 2024-09-21T03:49:10.214 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:10.210+0000 7f9f07fff640 1 -- 172.21.15.39:0/1281904684 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9f10009780 con 0x56239f16c1b0 2024-09-21T03:49:10.214 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:10.210+0000 7f9f07fff640 1 -- 172.21.15.39:0/1281904684 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f10003c80 con 0x56239f16c1b0 2024-09-21T03:49:10.217 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:10.211+0000 7f9f2af8b740 1 -- 172.21.15.39:0/1281904684 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56239f16c1b0 msgr2=0x56239f5430b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:49:10.218 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:10.211+0000 7f9f2af8b740 1 --2- 172.21.15.39:0/1281904684 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56239f16c1b0 0x56239f5430b0 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7f9f10009ca0 tx=0x7f9f10009ba0 comp rx=0 tx=0).stop 2024-09-21T03:49:10.218 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:10.211+0000 7f9f2af8b740 1 -- 172.21.15.39:0/1281904684 shutdown_connections 2024-09-21T03:49:10.218 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:10.211+0000 7f9f2af8b740 1 --2- 172.21.15.39:0/1281904684 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56239f195070 0x56239f544af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:10.218 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:10.211+0000 7f9f2af8b740 1 --2- 172.21.15.39:0/1281904684 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56239f174390 0x56239f549530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:10.218 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:10.211+0000 7f9f2af8b740 1 --2- 172.21.15.39:0/1281904684 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56239f16c1b0 0x56239f5430b0 unknown :-1 s=CLOSED pgs=654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:10.218 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:10.211+0000 7f9f2af8b740 1 -- 172.21.15.39:0/1281904684 >> 172.21.15.39:0/1281904684 conn(0x56239f189460 msgr2=0x56239f548e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:49:10.218 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:10.211+0000 7f9f2af8b740 1 -- 172.21.15.39:0/1281904684 shutdown_connections 2024-09-21T03:49:10.218 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:10.211+0000 7f9f2af8b740 1 -- 172.21.15.39:0/1281904684 wait complete. 2024-09-21T03:49:10.218 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:10.211+0000 7f9f2af8b740 1 -- start start 2024-09-21T03:49:10.218 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:10.212+0000 7f9f2af8b740 1 Processor -- start 2024-09-21T03:49:10.218 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:10.212+0000 7f9f2af8b740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56239f16c1b0 0x56239f550480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:49:10.218 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:10.212+0000 7f9f2af8b740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56239f174390 0x56239f54b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:49:10.218 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:10.212+0000 7f9f2af8b740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56239f195070 0x56239f54bec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:49:10.219 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:10.212+0000 7f9f2af8b740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f9f201b2c90 con 0x56239f16c1b0 2024-09-21T03:49:10.219 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:10.212+0000 7f9f2af8b740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f9f201b25c0 con 0x56239f195070 2024-09-21T03:49:10.219 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:10.212+0000 7f9f2af8b740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f9f201af800 con 0x56239f174390 2024-09-21T03:49:10.219 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:10.212+0000 7f9f2af8b740 10 client.0 _ll_register_callbacks cb 0x7f9f140a9ea0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:49:10.219 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:10.212+0000 7f9f1e59c640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56239f16c1b0 0x56239f550480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:49:10.219 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:10.212+0000 7f9f1e59c640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56239f16c1b0 0x56239f550480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:33888/0 (socket says 172.21.15.39:33888) 2024-09-21T03:49:10.219 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:10.212+0000 7f9f1e59c640 1 -- 172.21.15.39:0/4131321191 learned_addr learned my addr 172.21.15.39:0/4131321191 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:49:10.219 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:10.212+0000 7f9f1dd9b640 1 --2- 172.21.15.39:0/4131321191 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56239f174390 0x56239f54b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:49:10.219 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:10.212+0000 7f9f1ed9d640 1 --2- 172.21.15.39:0/4131321191 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56239f195070 0x56239f54bec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:49:10.219 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:10.212+0000 7f9f1e59c640 1 -- 172.21.15.39:0/4131321191 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56239f174390 msgr2=0x56239f54b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:49:10.219 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:10.212+0000 7f9f1e59c640 1 --2- 172.21.15.39:0/4131321191 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56239f174390 0x56239f54b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:10.219 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:10.212+0000 7f9f1e59c640 1 -- 172.21.15.39:0/4131321191 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56239f195070 msgr2=0x56239f54bec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:49:10.219 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:10.212+0000 7f9f1e59c640 1 --2- 172.21.15.39:0/4131321191 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56239f195070 0x56239f54bec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:10.220 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:10.212+0000 7f9f1e59c640 1 -- 172.21.15.39:0/4131321191 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56239f54c600 con 0x56239f16c1b0 2024-09-21T03:49:10.220 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:10.212+0000 7f9f1ed9d640 1 --2- 172.21.15.39:0/4131321191 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56239f195070 0x56239f54bec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:49:10.220 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:10.213+0000 7f9f1e59c640 1 --2- 172.21.15.39:0/4131321191 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56239f16c1b0 0x56239f550480 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7f9f100058c0 tx=0x7f9f10011e20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c1584eda140a272 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:49:10.220 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:10.213+0000 7f9f05ffb640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-09-21T03:49:10.220 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:10.213+0000 7f9f05ffb640 1 -- 172.21.15.39:0/4131321191 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f10005100 con 0x56239f16c1b0 2024-09-21T03:49:10.220 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:10.213+0000 7f9f05ffb640 1 -- 172.21.15.39:0/4131321191 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8484+0+0 (secure 0 0 0) 0x7f9f1002a070 con 0x56239f16c1b0 2024-09-21T03:49:10.220 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:10.213+0000 7f9f2af8b740 1 -- 172.21.15.39:0/4131321191 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9f10009780 con 0x56239f16c1b0 2024-09-21T03:49:10.220 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:10.213+0000 7f9f05ffb640 1 -- 172.21.15.39:0/4131321191 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f9f100052a0 con 0x56239f16c1b0 2024-09-21T03:49:10.220 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:10.213+0000 7f9f2af8b740 10 client.16212 fetch_fsmap learned FSMap version 29 2024-09-21T03:49:10.220 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:10.213+0000 7f9f2af8b740 1 -- 172.21.15.39:0/4131321191 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x56239f5c43b0 con 0x56239f16c1b0 2024-09-21T03:49:10.220 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:10.213+0000 7f9f05ffb640 1 -- 172.21.15.39:0/4131321191 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9f10009780 con 0x56239f16c1b0 2024-09-21T03:49:10.220 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:10.213+0000 7f9f05ffb640 1 -- 172.21.15.39:0/4131321191 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f9f1002da80 con 0x56239f16c1b0 2024-09-21T03:49:10.220 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:10.213+0000 7f9f2af8b740 10 client.16212 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T03:49:10.220 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:10.213+0000 7f9f2af8b740 10 client.16212 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-21T03:49:10.220 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:10.213+0000 7f9f2af8b740 20 client.16212 populate_metadata read hostname 'smithi039' 2024-09-21T03:49:10.221 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:10.214+0000 7f9f2af8b740 1 --2- 172.21.15.39:0/4131321191 >> [v2:172.21.15.161:6836/3451544597,v1:172.21.15.161:6837/3451544597] conn(0x56239f128da0 0x56239f5c6930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:49:10.221 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:10.214+0000 7f9f2af8b740 4 client.16212 mds_command: new command op to 24646 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T03:49:10.221 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:10.214+0000 7f9f2af8b740 1 -- 172.21.15.39:0/4131321191 --> [v2:172.21.15.161:6836/3451544597,v1:172.21.15.161:6837/3451544597] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56239f4b08a0 con 0x56239f128da0 2024-09-21T03:49:10.221 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:10.214+0000 7f9f1dd9b640 1 --2- 172.21.15.39:0/4131321191 >> [v2:172.21.15.161:6836/3451544597,v1:172.21.15.161:6837/3451544597] conn(0x56239f128da0 0x56239f5c6930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:49:10.221 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:10.214+0000 7f9f1dd9b640 1 --2- 172.21.15.39:0/4131321191 >> [v2:172.21.15.161:6836/3451544597,v1:172.21.15.161:6837/3451544597] conn(0x56239f128da0 0x56239f5c6930 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=a0b3d3d47963694f server_cookie=9f0a381b33acb921 in_seq=0 out_seq=0 2024-09-21T03:49:10.221 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:10.214+0000 7f9f05ffb640 10 client.16212 ms_handle_connect on v2:172.21.15.161:6836/3451544597 2024-09-21T03:49:10.221 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:10.214+0000 7f9f05ffb640 1 -- 172.21.15.39:0/4131321191 <== mds.3 v2:172.21.15.161:6836/3451544597 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56239f4b08a0 con 0x56239f128da0 2024-09-21T03:49:10.221 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:10.215+0000 7f9f05ffb640 10 client.16212 handle_command_reply: tid=0 2024-09-21T03:49:10.221 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:10.215+0000 7f9f2af8b740 1 client.16212 shutdown 2024-09-21T03:49:10.221 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:10.215+0000 7f9f2af8b740 1 -- 172.21.15.39:0/4131321191 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56239f16c1b0 msgr2=0x56239f550480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:49:10.221 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:10.215+0000 7f9f2af8b740 1 --2- 172.21.15.39:0/4131321191 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56239f16c1b0 0x56239f550480 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7f9f100058c0 tx=0x7f9f10011e20 comp rx=0 tx=0).stop 2024-09-21T03:49:10.221 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:10.215+0000 7f9f2af8b740 1 -- 172.21.15.39:0/4131321191 shutdown_connections 2024-09-21T03:49:10.221 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:10.215+0000 7f9f2af8b740 1 --2- 172.21.15.39:0/4131321191 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56239f195070 0x56239f54bec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:10.221 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:10.215+0000 7f9f2af8b740 1 --2- 172.21.15.39:0/4131321191 >> [v2:172.21.15.161:6836/3451544597,v1:172.21.15.161:6837/3451544597] conn(0x56239f128da0 0x56239f5c6930 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:10.221 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:10.215+0000 7f9f05ffb640 0 client.16212 ms_handle_reset on v2:172.21.15.161:6836/3451544597 2024-09-21T03:49:10.222 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:10.215+0000 7f9f2af8b740 1 --2- 172.21.15.39:0/4131321191 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56239f174390 0x56239f54b980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:10.222 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:10.215+0000 7f9f2af8b740 1 --2- 172.21.15.39:0/4131321191 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56239f16c1b0 0x56239f550480 unknown :-1 s=CLOSED pgs=655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:10.222 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:10.215+0000 7f9f2af8b740 1 -- 172.21.15.39:0/4131321191 >> 172.21.15.39:0/4131321191 conn(0x56239f189460 msgr2=0x56239f552c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:49:10.222 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:10.215+0000 7f9f2af8b740 1 -- 172.21.15.39:0/4131321191 shutdown_connections 2024-09-21T03:49:10.222 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:10.215+0000 7f9f2af8b740 1 -- 172.21.15.39:0/4131321191 wait complete. 2024-09-21T03:49:10.222 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:10.215+0000 7f9f2af8b740 20 client.16212 trim_cache size 0 max 16384 2024-09-21T03:49:10.222 INFO:teuthology.orchestra.run.smithi039.stdout:[] 2024-09-21T03:49:10.223 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:10.215+0000 7f9f24a17640 1 -- 172.21.15.39:0/1711008496 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f9f080b6700 msgr2=0x7f9f080b8c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:49:10.223 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:10.215+0000 7f9f24a17640 1 --2- 172.21.15.39:0/1711008496 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f9f080b6700 0x7f9f080b8c10 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f9f1802f500 tx=0x7f9f1802f310 comp rx=0 tx=0).stop 2024-09-21T03:49:10.223 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:10.215+0000 7f9f24a17640 1 -- 172.21.15.39:0/1711008496 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9f20108420 msgr2=0x7f9f200ffbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:49:10.223 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:10.215+0000 7f9f24a17640 1 --2- 172.21.15.39:0/1711008496 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9f20108420 0x7f9f200ffbb0 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f9f1400e9f0 tx=0x7f9f1400eeb0 comp rx=0 tx=0).stop 2024-09-21T03:49:10.223 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:10.216+0000 7f9f24a17640 1 -- 172.21.15.39:0/1711008496 shutdown_connections 2024-09-21T03:49:10.223 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:10.216+0000 7f9f24a17640 1 --2- 172.21.15.39:0/1711008496 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f9f080b6700 0x7f9f080b8c10 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:10.223 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:10.216+0000 7f9f24a17640 1 --2- 172.21.15.39:0/1711008496 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9f20108d70 0x7f9f201000f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:10.224 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:10.216+0000 7f9f24a17640 1 --2- 172.21.15.39:0/1711008496 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9f20108420 0x7f9f200ffbb0 unknown :-1 s=CLOSED pgs=652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:10.224 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:10.216+0000 7f9f24a17640 1 --2- 172.21.15.39:0/1711008496 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9f20102ae0 0x7f9f200ff670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:10.224 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:10.216+0000 7f9f24a17640 1 -- 172.21.15.39:0/1711008496 >> 172.21.15.39:0/1711008496 conn(0x7f9f20079770 msgr2=0x7f9f20101a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:49:10.224 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:10.216+0000 7f9f24a17640 1 -- 172.21.15.39:0/1711008496 shutdown_connections 2024-09-21T03:49:10.224 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:10.216+0000 7f9f24a17640 1 -- 172.21.15.39:0/1711008496 wait complete. 2024-09-21T03:49:10.834 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:49:10 smithi039 ceph-mon[31211]: pgmap v419: 97 pgs: 97 active+clean; 10 MiB data, 536 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 23 KiB/s wr, 26 op/s 2024-09-21T03:49:10.968 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:49:10 smithi161 ceph-mon[34533]: pgmap v419: 97 pgs: 97 active+clean; 10 MiB data, 536 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 23 KiB/s wr, 26 op/s 2024-09-21T03:49:10.974 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:49:10 smithi150 ceph-mon[34289]: pgmap v419: 97 pgs: 97 active+clean; 10 MiB data, 536 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 23 KiB/s wr, 26 op/s 2024-09-21T03:49:11.039 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 51116d5c-77ca-11ef-baf5-efdc52797490 -- ceph tell mds.1:2 damage ls 2024-09-21T03:49:11.297 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/51116d5c-77ca-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:49:11.797 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:11.794+0000 7fcba4f79640 1 -- 172.21.15.39:0/4050912090 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcba0109df0 msgr2=0x7fcba0110620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:49:11.797 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:11.794+0000 7fcba4f79640 1 --2- 172.21.15.39:0/4050912090 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcba0109df0 0x7fcba0110620 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7fcb90009ca0 tx=0x7fcb9002a8c0 comp rx=0 tx=0).stop 2024-09-21T03:49:11.797 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:11.795+0000 7fcba4f79640 1 -- 172.21.15.39:0/4050912090 shutdown_connections 2024-09-21T03:49:11.797 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:11.795+0000 7fcba4f79640 1 --2- 172.21.15.39:0/4050912090 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcba0109df0 0x7fcba0110620 unknown :-1 s=CLOSED pgs=656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:11.797 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:11.795+0000 7fcba4f79640 1 --2- 172.21.15.39:0/4050912090 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcba01094a0 0x7fcba01098b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:11.797 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:11.795+0000 7fcba4f79640 1 --2- 172.21.15.39:0/4050912090 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcba0103b60 0x7fcba0103f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:11.797 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:11.795+0000 7fcba4f79640 1 -- 172.21.15.39:0/4050912090 >> 172.21.15.39:0/4050912090 conn(0x7fcba007b850 msgr2=0x7fcba007bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:49:11.797 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:11.795+0000 7fcba4f79640 1 -- 172.21.15.39:0/4050912090 shutdown_connections 2024-09-21T03:49:11.798 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:11.795+0000 7fcba4f79640 1 -- 172.21.15.39:0/4050912090 wait complete. 2024-09-21T03:49:11.799 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:11.796+0000 7fcba4f79640 1 Processor -- start 2024-09-21T03:49:11.799 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:11.796+0000 7fcba4f79640 1 -- start start 2024-09-21T03:49:11.800 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:11.796+0000 7fcba4f79640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcba0103b60 0x7fcba019a440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:49:11.800 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:11.796+0000 7fcba4f79640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcba01094a0 0x7fcba019a9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:49:11.800 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:11.796+0000 7fcba4f79640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcba0109df0 0x7fcba019fc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:49:11.800 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:11.796+0000 7fcba4f79640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fcba0113ab0 con 0x7fcba01094a0 2024-09-21T03:49:11.800 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:11.796+0000 7fcba4f79640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fcba0113930 con 0x7fcba0109df0 2024-09-21T03:49:11.800 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:11.796+0000 7fcba4f79640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fcba0113c30 con 0x7fcba0103b60 2024-09-21T03:49:11.800 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:11.796+0000 7fcb9dd9b640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcba01094a0 0x7fcba019a9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:49:11.800 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:11.796+0000 7fcb9dd9b640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcba01094a0 0x7fcba019a9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:33912/0 (socket says 172.21.15.39:33912) 2024-09-21T03:49:11.800 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:11.796+0000 7fcb9dd9b640 1 -- 172.21.15.39:0/3610756401 learned_addr learned my addr 172.21.15.39:0/3610756401 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:49:11.800 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:11.796+0000 7fcb9e59c640 1 --2- 172.21.15.39:0/3610756401 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcba0103b60 0x7fcba019a440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:49:11.800 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:11.796+0000 7fcb9ed9d640 1 --2- 172.21.15.39:0/3610756401 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcba0109df0 0x7fcba019fc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:49:11.800 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:11.797+0000 7fcb9dd9b640 1 -- 172.21.15.39:0/3610756401 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcba0103b60 msgr2=0x7fcba019a440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:49:11.800 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:11.797+0000 7fcb9dd9b640 1 --2- 172.21.15.39:0/3610756401 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcba0103b60 0x7fcba019a440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:11.800 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:11.797+0000 7fcb9dd9b640 1 -- 172.21.15.39:0/3610756401 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcba0109df0 msgr2=0x7fcba019fc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:49:11.801 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:11.797+0000 7fcb9dd9b640 1 --2- 172.21.15.39:0/3610756401 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcba0109df0 0x7fcba019fc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:11.801 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:11.797+0000 7fcb9dd9b640 1 -- 172.21.15.39:0/3610756401 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcba01a03b0 con 0x7fcba01094a0 2024-09-21T03:49:11.801 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:11.797+0000 7fcb9ed9d640 1 --2- 172.21.15.39:0/3610756401 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcba0109df0 0x7fcba019fc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:49:11.801 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:11.797+0000 7fcb9dd9b640 1 --2- 172.21.15.39:0/3610756401 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcba01094a0 0x7fcba019a9e0 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7fcb9400d950 tx=0x7fcb9400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:49:11.801 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:11.797+0000 7fcb8f7fe640 1 -- 172.21.15.39:0/3610756401 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb94017070 con 0x7fcba01094a0 2024-09-21T03:49:11.801 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:11.797+0000 7fcb8f7fe640 1 -- 172.21.15.39:0/3610756401 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcb94009c30 con 0x7fcba01094a0 2024-09-21T03:49:11.801 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:11.797+0000 7fcba4f79640 1 -- 172.21.15.39:0/3610756401 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcba01a06a0 con 0x7fcba01094a0 2024-09-21T03:49:11.801 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:11.797+0000 7fcb8f7fe640 1 -- 172.21.15.39:0/3610756401 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb940047a0 con 0x7fcba01094a0 2024-09-21T03:49:11.801 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:11.797+0000 7fcba4f79640 1 -- 172.21.15.39:0/3610756401 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcba01a6e00 con 0x7fcba01094a0 2024-09-21T03:49:11.802 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:11.800+0000 7fcb8f7fe640 1 -- 172.21.15.39:0/3610756401 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 27) ==== 154643+0+0 (secure 0 0 0) 0x7fcb94004020 con 0x7fcba01094a0 2024-09-21T03:49:11.802 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:11.800+0000 7fcbab4ed740 1 Processor -- start 2024-09-21T03:49:11.802 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:11.800+0000 7fcbab4ed740 1 -- start start 2024-09-21T03:49:11.803 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:11.801+0000 7fcbab4ed740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a3cc488070 0x55a3cc82f2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:49:11.803 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:11.801+0000 7fcbab4ed740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a3cc45f1b0 0x55a3cc82f800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:49:11.803 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:11.801+0000 7fcbab4ed740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a3cc467390 0x55a3cc82fd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:49:11.803 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:11.801+0000 7fcbab4ed740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x55a3cc7a5ea0 con 0x55a3cc467390 2024-09-21T03:49:11.803 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:11.801+0000 7fcbab4ed740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x55a3cc7952d0 con 0x55a3cc488070 2024-09-21T03:49:11.803 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:11.801+0000 7fcbab4ed740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55a3cc7990e0 con 0x55a3cc45f1b0 2024-09-21T03:49:11.803 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:11.801+0000 7fcb8f7fe640 1 --2- 172.21.15.39:0/3610756401 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7fcb880b6700 0x7fcb880b8c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:49:11.803 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:11.801+0000 7fcb9e59c640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a3cc45f1b0 0x55a3cc82f800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:49:11.803 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:11.801+0000 7fcb9ed9d640 1 --2- 172.21.15.39:0/3610756401 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7fcb880b6700 0x7fcb880b8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:49:11.803 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:11.801+0000 7fcb9dd9b640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a3cc467390 0x55a3cc82fd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:49:11.804 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:11.801+0000 7fcb9e59c640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a3cc45f1b0 0x55a3cc82f800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.39:39890/0 (socket says 172.21.15.39:39890) 2024-09-21T03:49:11.804 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:11.801+0000 7fcb9e59c640 1 -- 172.21.15.39:0/1617137528 learned_addr learned my addr 172.21.15.39:0/1617137528 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:49:11.804 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:11.801+0000 7fcb9ed9d640 1 --2- 172.21.15.39:0/1617137528 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a3cc488070 0x55a3cc82f2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:49:11.804 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:11.801+0000 7fcb8f7fe640 1 -- 172.21.15.39:0/3610756401 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8484+0+0 (secure 0 0 0) 0x7fcb940e7340 con 0x7fcba01094a0 2024-09-21T03:49:11.805 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:11.801+0000 7fcb9dd9b640 1 -- 172.21.15.39:0/1617137528 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a3cc45f1b0 msgr2=0x55a3cc82f800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:49:11.805 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:11.801+0000 7fcb9dd9b640 1 --2- 172.21.15.39:0/1617137528 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a3cc45f1b0 0x55a3cc82f800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:11.805 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:11.801+0000 7fcb9dd9b640 1 -- 172.21.15.39:0/1617137528 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a3cc488070 msgr2=0x55a3cc82f2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:49:11.805 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:11.801+0000 7fcb9dd9b640 1 --2- 172.21.15.39:0/1617137528 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a3cc488070 0x55a3cc82f2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:11.805 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:11.801+0000 7fcb9dd9b640 1 -- 172.21.15.39:0/1617137528 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a3cc4f3650 con 0x55a3cc467390 2024-09-21T03:49:11.805 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:11.802+0000 7fcb9ed9d640 1 --2- 172.21.15.39:0/3610756401 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7fcb880b6700 0x7fcb880b8c10 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fcb90009a60 tx=0x7fcb90032cc0 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:49:11.806 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:11.802+0000 7fcb9dd9b640 1 --2- 172.21.15.39:0/1617137528 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a3cc467390 0x55a3cc82fd40 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7fcb940e4ab0 tx=0x7fcb94079e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d0da574899cabb2d server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:49:11.806 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:11.802+0000 7fcb8cff9640 1 -- 172.21.15.39:0/1617137528 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb940acf80 con 0x55a3cc467390 2024-09-21T03:49:11.806 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:11.802+0000 7fcb8cff9640 1 -- 172.21.15.39:0/1617137528 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcb940accc0 con 0x55a3cc467390 2024-09-21T03:49:11.806 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:11.802+0000 7fcb8cff9640 1 -- 172.21.15.39:0/1617137528 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb940ac920 con 0x55a3cc467390 2024-09-21T03:49:11.806 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:11.803+0000 7fcbab4ed740 1 -- 172.21.15.39:0/1617137528 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a3cc467390 msgr2=0x55a3cc82fd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:49:11.806 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:11.803+0000 7fcbab4ed740 1 --2- 172.21.15.39:0/1617137528 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a3cc467390 0x55a3cc82fd40 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7fcb940e4ab0 tx=0x7fcb94079e60 comp rx=0 tx=0).stop 2024-09-21T03:49:11.806 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:11.803+0000 7fcbab4ed740 1 -- 172.21.15.39:0/1617137528 shutdown_connections 2024-09-21T03:49:11.806 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:11.803+0000 7fcbab4ed740 1 --2- 172.21.15.39:0/1617137528 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a3cc467390 0x55a3cc82fd40 unknown :-1 s=CLOSED pgs=658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:11.806 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:11.803+0000 7fcbab4ed740 1 --2- 172.21.15.39:0/1617137528 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a3cc45f1b0 0x55a3cc82f800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:11.806 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:11.803+0000 7fcbab4ed740 1 --2- 172.21.15.39:0/1617137528 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a3cc488070 0x55a3cc82f2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:11.806 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:11.803+0000 7fcbab4ed740 1 -- 172.21.15.39:0/1617137528 >> 172.21.15.39:0/1617137528 conn(0x55a3cc47c460 msgr2=0x55a3cc82d1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:49:11.806 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:11.803+0000 7fcbab4ed740 1 -- 172.21.15.39:0/1617137528 shutdown_connections 2024-09-21T03:49:11.806 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:11.803+0000 7fcbab4ed740 1 -- 172.21.15.39:0/1617137528 wait complete. 2024-09-21T03:49:11.807 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:11.803+0000 7fcbab4ed740 1 -- start start 2024-09-21T03:49:11.807 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:11.803+0000 7fcbab4ed740 1 Processor -- start 2024-09-21T03:49:11.809 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:11.804+0000 7fcbab4ed740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a3cc45f1b0 0x55a3cc83a900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:49:11.809 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:11.804+0000 7fcbab4ed740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a3cc467390 0x55a3cc83ae40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:49:11.810 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:11.804+0000 7fcbab4ed740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a3cc488070 0x55a3cc83b3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:49:11.810 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:11.804+0000 7fcbab4ed740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x55a3cc5f6700 con 0x55a3cc488070 2024-09-21T03:49:11.810 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:11.804+0000 7fcbab4ed740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x55a3cc730160 con 0x55a3cc467390 2024-09-21T03:49:11.810 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:11.804+0000 7fcbab4ed740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55a3cc83d9e0 con 0x55a3cc45f1b0 2024-09-21T03:49:11.810 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:11.804+0000 7fcbab4ed740 10 client.0 _ll_register_callbacks cb 0x55a3cc7499e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:49:11.810 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:11.804+0000 7fcb9ed9d640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a3cc488070 0x55a3cc83b3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:49:11.810 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:11.804+0000 7fcb9e59c640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a3cc45f1b0 0x55a3cc83a900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:49:11.810 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:11.804+0000 7fcb9ed9d640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a3cc488070 0x55a3cc83b3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:33928/0 (socket says 172.21.15.39:33928) 2024-09-21T03:49:11.810 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:11.804+0000 7fcb9ed9d640 1 -- 172.21.15.39:0/643482871 learned_addr learned my addr 172.21.15.39:0/643482871 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:49:11.810 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:11.804+0000 7fcb9dd9b640 1 --2- 172.21.15.39:0/643482871 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a3cc467390 0x55a3cc83ae40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:49:11.810 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:11.804+0000 7fcb9ed9d640 1 -- 172.21.15.39:0/643482871 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a3cc45f1b0 msgr2=0x55a3cc83a900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:49:11.810 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:11.804+0000 7fcb9ed9d640 1 --2- 172.21.15.39:0/643482871 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a3cc45f1b0 0x55a3cc83a900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:11.810 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:11.804+0000 7fcb9ed9d640 1 -- 172.21.15.39:0/643482871 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a3cc467390 msgr2=0x55a3cc83ae40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:49:11.811 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:11.804+0000 7fcb9ed9d640 1 --2- 172.21.15.39:0/643482871 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a3cc467390 0x55a3cc83ae40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:11.811 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:11.804+0000 7fcb9ed9d640 1 -- 172.21.15.39:0/643482871 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a3cc75dc90 con 0x55a3cc488070 2024-09-21T03:49:11.811 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:11.804+0000 7fcb9e59c640 1 --2- 172.21.15.39:0/643482871 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a3cc45f1b0 0x55a3cc83a900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:49:11.811 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:11.805+0000 7fcb9ed9d640 1 --2- 172.21.15.39:0/643482871 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a3cc488070 0x55a3cc83b3c0 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7fcb90035000 tx=0x7fcb90048000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c5383367448feb8f server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:49:11.811 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:11.805+0000 7fcb85ffb640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-09-21T03:49:11.811 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:11.805+0000 7fcb85ffb640 1 -- 172.21.15.39:0/643482871 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb90050070 con 0x55a3cc488070 2024-09-21T03:49:11.811 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:11.805+0000 7fcb85ffb640 1 -- 172.21.15.39:0/643482871 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8484+0+0 (secure 0 0 0) 0x7fcb90031af0 con 0x55a3cc488070 2024-09-21T03:49:11.811 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:11.805+0000 7fcbab4ed740 1 -- 172.21.15.39:0/643482871 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcb940accc0 con 0x55a3cc488070 2024-09-21T03:49:11.811 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:11.805+0000 7fcb85ffb640 1 -- 172.21.15.39:0/643482871 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fcb90031cf0 con 0x55a3cc488070 2024-09-21T03:49:11.811 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:11.805+0000 7fcbab4ed740 10 client.16224 fetch_fsmap learned FSMap version 29 2024-09-21T03:49:11.811 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:11.805+0000 7fcbab4ed740 1 -- 172.21.15.39:0/643482871 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55a3cc849680 con 0x55a3cc488070 2024-09-21T03:49:11.811 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:11.805+0000 7fcb85ffb640 1 -- 172.21.15.39:0/643482871 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcb940accc0 con 0x55a3cc488070 2024-09-21T03:49:11.811 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:11.805+0000 7fcb85ffb640 1 -- 172.21.15.39:0/643482871 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fcb9002f780 con 0x55a3cc488070 2024-09-21T03:49:11.811 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:11.806+0000 7fcbab4ed740 10 client.16224 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T03:49:11.812 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:11.806+0000 7fcbab4ed740 10 client.16224 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-21T03:49:11.812 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:11.806+0000 7fcbab4ed740 20 client.16224 populate_metadata read hostname 'smithi039' 2024-09-21T03:49:11.812 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:11.806+0000 7fcbab4ed740 1 --2- 172.21.15.39:0/643482871 >> [v2:172.21.15.150:6836/4017043144,v1:172.21.15.150:6837/4017043144] conn(0x55a3cc4469f0 0x55a3cc84cdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:49:11.812 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:11.806+0000 7fcbab4ed740 4 client.16224 mds_command: new command op to 24659 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:49:11.812 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:11.806+0000 7fcbab4ed740 1 -- 172.21.15.39:0/643482871 --> [v2:172.21.15.150:6836/4017043144,v1:172.21.15.150:6837/4017043144] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a3cc7a7df0 con 0x55a3cc4469f0 2024-09-21T03:49:11.812 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:11.806+0000 7fcb9e59c640 1 --2- 172.21.15.39:0/643482871 >> [v2:172.21.15.150:6836/4017043144,v1:172.21.15.150:6837/4017043144] conn(0x55a3cc4469f0 0x55a3cc84cdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:49:11.812 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:11.806+0000 7fcb9e59c640 1 --2- 172.21.15.39:0/643482871 >> [v2:172.21.15.150:6836/4017043144,v1:172.21.15.150:6837/4017043144] conn(0x55a3cc4469f0 0x55a3cc84cdc0 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=c56e257aef08218f server_cookie=fb92fde8706f32e5 in_seq=0 out_seq=0 2024-09-21T03:49:11.812 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:11.806+0000 7fcb85ffb640 10 client.16224 ms_handle_connect on v2:172.21.15.150:6836/4017043144 2024-09-21T03:49:11.812 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:11.807+0000 7fcb85ffb640 1 -- 172.21.15.39:0/643482871 <== mds.2 v2:172.21.15.150:6836/4017043144 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55a3cc7a7df0 con 0x55a3cc4469f0 2024-09-21T03:49:11.812 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:11.807+0000 7fcb85ffb640 10 client.16224 handle_command_reply: tid=0 2024-09-21T03:49:11.812 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:11.807+0000 7fcbab4ed740 1 client.16224 shutdown 2024-09-21T03:49:11.812 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:11.807+0000 7fcbab4ed740 1 -- 172.21.15.39:0/643482871 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a3cc488070 msgr2=0x55a3cc83b3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:49:11.812 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:11.807+0000 7fcbab4ed740 1 --2- 172.21.15.39:0/643482871 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a3cc488070 0x55a3cc83b3c0 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7fcb90035000 tx=0x7fcb90048000 comp rx=0 tx=0).stop 2024-09-21T03:49:11.812 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:11.807+0000 7fcbab4ed740 1 -- 172.21.15.39:0/643482871 shutdown_connections 2024-09-21T03:49:11.812 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:11.807+0000 7fcbab4ed740 1 --2- 172.21.15.39:0/643482871 >> [v2:172.21.15.150:6836/4017043144,v1:172.21.15.150:6837/4017043144] conn(0x55a3cc4469f0 0x55a3cc84cdc0 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:11.813 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:11.807+0000 7fcbab4ed740 1 --2- 172.21.15.39:0/643482871 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a3cc488070 0x55a3cc83b3c0 unknown :-1 s=CLOSED pgs=659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:11.813 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:11.807+0000 7fcbab4ed740 1 --2- 172.21.15.39:0/643482871 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a3cc467390 0x55a3cc83ae40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:11.813 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:11.807+0000 7fcbab4ed740 1 --2- 172.21.15.39:0/643482871 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a3cc45f1b0 0x55a3cc83a900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:11.813 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:11.807+0000 7fcbab4ed740 1 -- 172.21.15.39:0/643482871 >> 172.21.15.39:0/643482871 conn(0x55a3cc47c460 msgr2=0x55a3cc8302f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:49:11.813 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:11.807+0000 7fcb85ffb640 0 client.16224 ms_handle_reset on v2:172.21.15.150:6836/4017043144 2024-09-21T03:49:11.813 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:11.807+0000 7fcbab4ed740 1 -- 172.21.15.39:0/643482871 shutdown_connections 2024-09-21T03:49:11.813 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:11.808+0000 7fcbab4ed740 1 -- 172.21.15.39:0/643482871 wait complete. 2024-09-21T03:49:11.813 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:11.808+0000 7fcbab4ed740 20 client.16224 trim_cache size 0 max 16384 2024-09-21T03:49:11.823 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:11.821+0000 7fcbab4ed740 1 Processor -- start 2024-09-21T03:49:11.824 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:11.821+0000 7fcbab4ed740 1 -- start start 2024-09-21T03:49:11.824 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:11.821+0000 7fcbab4ed740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a3cc45f1b0 0x55a3cc837af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:49:11.824 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:11.821+0000 7fcbab4ed740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a3cc467390 0x55a3cc838030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:49:11.824 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:11.821+0000 7fcbab4ed740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a3cc488070 0x55a3cc8302f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:49:11.824 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:11.821+0000 7fcbab4ed740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fcba01af790 con 0x55a3cc467390 2024-09-21T03:49:11.824 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:11.821+0000 7fcbab4ed740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fcba01b12d0 con 0x55a3cc488070 2024-09-21T03:49:11.824 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:11.821+0000 7fcbab4ed740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fcba01afe60 con 0x55a3cc45f1b0 2024-09-21T03:49:11.824 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:11.821+0000 7fcb9dd9b640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a3cc467390 0x55a3cc838030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:49:11.824 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:11.821+0000 7fcb9dd9b640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a3cc467390 0x55a3cc838030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:33938/0 (socket says 172.21.15.39:33938) 2024-09-21T03:49:11.824 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:11.821+0000 7fcb9dd9b640 1 -- 172.21.15.39:0/528433060 learned_addr learned my addr 172.21.15.39:0/528433060 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:49:11.824 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:11.821+0000 7fcb9e59c640 1 --2- 172.21.15.39:0/528433060 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a3cc45f1b0 0x55a3cc837af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:49:11.824 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:11.821+0000 7fcb9ed9d640 1 --2- 172.21.15.39:0/528433060 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a3cc488070 0x55a3cc8302f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:49:11.824 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:11.821+0000 7fcb9dd9b640 1 -- 172.21.15.39:0/528433060 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a3cc45f1b0 msgr2=0x55a3cc837af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:49:11.825 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:11.821+0000 7fcb9dd9b640 1 --2- 172.21.15.39:0/528433060 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a3cc45f1b0 0x55a3cc837af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:11.825 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:11.821+0000 7fcb9dd9b640 1 -- 172.21.15.39:0/528433060 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a3cc488070 msgr2=0x55a3cc8302f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:49:11.825 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:11.821+0000 7fcb9dd9b640 1 --2- 172.21.15.39:0/528433060 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a3cc488070 0x55a3cc8302f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:11.825 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:11.821+0000 7fcb9dd9b640 1 -- 172.21.15.39:0/528433060 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a3cc82d2c0 con 0x55a3cc467390 2024-09-21T03:49:11.825 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:11.821+0000 7fcb9ed9d640 1 --2- 172.21.15.39:0/528433060 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a3cc488070 0x55a3cc8302f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:49:11.825 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:11.822+0000 7fcb9dd9b640 1 --2- 172.21.15.39:0/528433060 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a3cc467390 0x55a3cc838030 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7fcb940e7310 tx=0x7fcb9401d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=749c120d925be042 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:49:11.825 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:11.822+0000 7fcb87fff640 1 -- 172.21.15.39:0/528433060 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb9405c070 con 0x55a3cc467390 2024-09-21T03:49:11.825 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:11.822+0000 7fcb87fff640 1 -- 172.21.15.39:0/528433060 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcb940accc0 con 0x55a3cc467390 2024-09-21T03:49:11.825 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:11.822+0000 7fcb87fff640 1 -- 172.21.15.39:0/528433060 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb940acb00 con 0x55a3cc467390 2024-09-21T03:49:11.829 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:11.823+0000 7fcbab4ed740 1 -- 172.21.15.39:0/528433060 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a3cc467390 msgr2=0x55a3cc838030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:49:11.829 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:11.823+0000 7fcbab4ed740 1 --2- 172.21.15.39:0/528433060 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a3cc467390 0x55a3cc838030 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7fcb940e7310 tx=0x7fcb9401d000 comp rx=0 tx=0).stop 2024-09-21T03:49:11.829 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:11.823+0000 7fcbab4ed740 1 -- 172.21.15.39:0/528433060 shutdown_connections 2024-09-21T03:49:11.829 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:11.823+0000 7fcbab4ed740 1 --2- 172.21.15.39:0/528433060 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a3cc488070 0x55a3cc8302f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:11.829 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:11.823+0000 7fcbab4ed740 1 --2- 172.21.15.39:0/528433060 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a3cc467390 0x55a3cc838030 unknown :-1 s=CLOSED pgs=660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:11.829 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:11.823+0000 7fcbab4ed740 1 --2- 172.21.15.39:0/528433060 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a3cc45f1b0 0x55a3cc837af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:11.829 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:11.823+0000 7fcbab4ed740 1 -- 172.21.15.39:0/528433060 >> 172.21.15.39:0/528433060 conn(0x55a3cc47c460 msgr2=0x55a3cc839180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:49:11.829 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:11.823+0000 7fcbab4ed740 1 -- 172.21.15.39:0/528433060 shutdown_connections 2024-09-21T03:49:11.829 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:11.823+0000 7fcbab4ed740 1 -- 172.21.15.39:0/528433060 wait complete. 2024-09-21T03:49:11.829 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:11.823+0000 7fcbab4ed740 1 -- start start 2024-09-21T03:49:11.829 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:11.823+0000 7fcbab4ed740 1 Processor -- start 2024-09-21T03:49:11.829 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:11.823+0000 7fcbab4ed740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a3cc45f1b0 0x55a3cc83e020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:49:11.829 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:11.823+0000 7fcbab4ed740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a3cc467390 0x55a3cc83e5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:49:11.830 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:11.823+0000 7fcbab4ed740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a3cc488070 0x55a3cc8b9bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:49:11.830 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:11.823+0000 7fcbab4ed740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fcba01b0c00 con 0x55a3cc467390 2024-09-21T03:49:11.830 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:11.824+0000 7fcbab4ed740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fcba01b0530 con 0x55a3cc45f1b0 2024-09-21T03:49:11.830 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:11.824+0000 7fcbab4ed740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fcba01ad770 con 0x55a3cc488070 2024-09-21T03:49:11.830 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:11.824+0000 7fcbab4ed740 10 client.0 _ll_register_callbacks cb 0x7fcb9002f310 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:49:11.830 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:11.824+0000 7fcb9dd9b640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a3cc467390 0x55a3cc83e5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:49:11.830 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:11.824+0000 7fcb9dd9b640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a3cc467390 0x55a3cc83e5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:33948/0 (socket says 172.21.15.39:33948) 2024-09-21T03:49:11.830 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:11.824+0000 7fcb9dd9b640 1 -- 172.21.15.39:0/2155545533 learned_addr learned my addr 172.21.15.39:0/2155545533 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:49:11.830 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:11.824+0000 7fcb9ed9d640 1 --2- 172.21.15.39:0/2155545533 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a3cc488070 0x55a3cc8b9bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:49:11.830 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:11.824+0000 7fcb9e59c640 1 --2- 172.21.15.39:0/2155545533 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a3cc45f1b0 0x55a3cc83e020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:49:11.830 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:11.824+0000 7fcb9dd9b640 1 -- 172.21.15.39:0/2155545533 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a3cc488070 msgr2=0x55a3cc8b9bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:49:11.830 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:11.824+0000 7fcb9dd9b640 1 --2- 172.21.15.39:0/2155545533 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a3cc488070 0x55a3cc8b9bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:11.831 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:11.824+0000 7fcb9dd9b640 1 -- 172.21.15.39:0/2155545533 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a3cc45f1b0 msgr2=0x55a3cc83e020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:49:11.831 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:11.824+0000 7fcb9dd9b640 1 --2- 172.21.15.39:0/2155545533 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a3cc45f1b0 0x55a3cc83e020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:11.831 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:11.824+0000 7fcb9dd9b640 1 -- 172.21.15.39:0/2155545533 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a3cc8ba100 con 0x55a3cc467390 2024-09-21T03:49:11.831 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:11.824+0000 7fcb9dd9b640 1 --2- 172.21.15.39:0/2155545533 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a3cc467390 0x55a3cc83e5c0 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7fcb940e50b0 tx=0x7fcb940b6d40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3278337c99184e8e server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:49:11.831 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:11.824+0000 7fcb85ffb640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-09-21T03:49:11.831 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:11.824+0000 7fcb85ffb640 1 -- 172.21.15.39:0/2155545533 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb94098020 con 0x55a3cc467390 2024-09-21T03:49:11.831 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:11.824+0000 7fcb85ffb640 1 -- 172.21.15.39:0/2155545533 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8484+0+0 (secure 0 0 0) 0x7fcb940d3fd0 con 0x55a3cc467390 2024-09-21T03:49:11.831 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:11.824+0000 7fcbab4ed740 1 -- 172.21.15.39:0/2155545533 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcb940accc0 con 0x55a3cc467390 2024-09-21T03:49:11.831 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:11.825+0000 7fcb85ffb640 1 -- 172.21.15.39:0/2155545533 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fcb9405c040 con 0x55a3cc467390 2024-09-21T03:49:11.831 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:11.825+0000 7fcbab4ed740 10 client.16230 fetch_fsmap learned FSMap version 29 2024-09-21T03:49:11.831 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:11.825+0000 7fcbab4ed740 1 -- 172.21.15.39:0/2155545533 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55a3cc8bb280 con 0x55a3cc467390 2024-09-21T03:49:11.831 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:11.825+0000 7fcb85ffb640 1 -- 172.21.15.39:0/2155545533 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcb940accc0 con 0x55a3cc467390 2024-09-21T03:49:11.831 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:11.825+0000 7fcb85ffb640 1 -- 172.21.15.39:0/2155545533 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fcb940d44a0 con 0x55a3cc467390 2024-09-21T03:49:11.832 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:11.825+0000 7fcbab4ed740 10 client.16230 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T03:49:11.832 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:11.825+0000 7fcbab4ed740 10 client.16230 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-21T03:49:11.832 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:11.825+0000 7fcbab4ed740 20 client.16230 populate_metadata read hostname 'smithi039' 2024-09-21T03:49:11.832 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:11.825+0000 7fcbab4ed740 1 --2- 172.21.15.39:0/2155545533 >> [v2:172.21.15.150:6836/4017043144,v1:172.21.15.150:6837/4017043144] conn(0x55a3cc41bda0 0x55a3cc8bd580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:49:11.832 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:11.825+0000 7fcbab4ed740 4 client.16230 mds_command: new command op to 24659 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T03:49:11.832 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:11.825+0000 7fcbab4ed740 1 -- 172.21.15.39:0/2155545533 --> [v2:172.21.15.150:6836/4017043144,v1:172.21.15.150:6837/4017043144] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55a3cc7a38a0 con 0x55a3cc41bda0 2024-09-21T03:49:11.832 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:11.825+0000 7fcb9e59c640 1 --2- 172.21.15.39:0/2155545533 >> [v2:172.21.15.150:6836/4017043144,v1:172.21.15.150:6837/4017043144] conn(0x55a3cc41bda0 0x55a3cc8bd580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:49:11.832 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:11.826+0000 7fcb9e59c640 1 --2- 172.21.15.39:0/2155545533 >> [v2:172.21.15.150:6836/4017043144,v1:172.21.15.150:6837/4017043144] conn(0x55a3cc41bda0 0x55a3cc8bd580 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=5636f6cf8c30e583 server_cookie=c56e650a8fa0ef8a in_seq=0 out_seq=0 2024-09-21T03:49:11.832 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:11.826+0000 7fcb85ffb640 10 client.16230 ms_handle_connect on v2:172.21.15.150:6836/4017043144 2024-09-21T03:49:11.832 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:11.826+0000 7fcb85ffb640 1 -- 172.21.15.39:0/2155545533 <== mds.2 v2:172.21.15.150:6836/4017043144 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55a3cc7a38a0 con 0x55a3cc41bda0 2024-09-21T03:49:11.832 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:11.826+0000 7fcb85ffb640 10 client.16230 handle_command_reply: tid=0 2024-09-21T03:49:11.832 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:11.826+0000 7fcbab4ed740 1 client.16230 shutdown 2024-09-21T03:49:11.832 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:11.826+0000 7fcbab4ed740 1 -- 172.21.15.39:0/2155545533 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a3cc467390 msgr2=0x55a3cc83e5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:49:11.833 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:11.826+0000 7fcbab4ed740 1 --2- 172.21.15.39:0/2155545533 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a3cc467390 0x55a3cc83e5c0 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7fcb940e50b0 tx=0x7fcb940b6d40 comp rx=0 tx=0).stop 2024-09-21T03:49:11.833 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:11.826+0000 7fcbab4ed740 1 -- 172.21.15.39:0/2155545533 shutdown_connections 2024-09-21T03:49:11.833 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:11.826+0000 7fcbab4ed740 1 --2- 172.21.15.39:0/2155545533 >> [v2:172.21.15.150:6836/4017043144,v1:172.21.15.150:6837/4017043144] conn(0x55a3cc41bda0 0x55a3cc8bd580 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:11.833 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:11.826+0000 7fcbab4ed740 1 --2- 172.21.15.39:0/2155545533 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a3cc488070 0x55a3cc8b9bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:11.833 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:11.826+0000 7fcbab4ed740 1 --2- 172.21.15.39:0/2155545533 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a3cc467390 0x55a3cc83e5c0 unknown :-1 s=CLOSED pgs=661 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:11.833 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:11.826+0000 7fcbab4ed740 1 --2- 172.21.15.39:0/2155545533 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a3cc45f1b0 0x55a3cc83e020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:11.833 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:11.826+0000 7fcb85ffb640 0 client.16230 ms_handle_reset on v2:172.21.15.150:6836/4017043144 2024-09-21T03:49:11.833 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:11.826+0000 7fcbab4ed740 1 -- 172.21.15.39:0/2155545533 >> 172.21.15.39:0/2155545533 conn(0x55a3cc47c460 msgr2=0x55a3cc838d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:49:11.833 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:11.826+0000 7fcbab4ed740 1 -- 172.21.15.39:0/2155545533 shutdown_connections 2024-09-21T03:49:11.833 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:11.826+0000 7fcbab4ed740 1 -- 172.21.15.39:0/2155545533 wait complete. 2024-09-21T03:49:11.833 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:11.826+0000 7fcbab4ed740 20 client.16230 trim_cache size 0 max 16384 2024-09-21T03:49:11.833 INFO:teuthology.orchestra.run.smithi039.stdout:[] 2024-09-21T03:49:11.834 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:11.827+0000 7fcba4f79640 1 -- 172.21.15.39:0/3610756401 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7fcb880b6700 msgr2=0x7fcb880b8c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:49:11.834 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:11.827+0000 7fcba4f79640 1 --2- 172.21.15.39:0/3610756401 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7fcb880b6700 0x7fcb880b8c10 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fcb90009a60 tx=0x7fcb90032cc0 comp rx=0 tx=0).stop 2024-09-21T03:49:11.834 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:11.827+0000 7fcba4f79640 1 -- 172.21.15.39:0/3610756401 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcba01094a0 msgr2=0x7fcba019a9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:49:11.834 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:11.827+0000 7fcba4f79640 1 --2- 172.21.15.39:0/3610756401 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcba01094a0 0x7fcba019a9e0 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7fcb9400d950 tx=0x7fcb9400de10 comp rx=0 tx=0).stop 2024-09-21T03:49:11.834 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:11.827+0000 7fcba4f79640 1 -- 172.21.15.39:0/3610756401 shutdown_connections 2024-09-21T03:49:11.834 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:11.827+0000 7fcba4f79640 1 --2- 172.21.15.39:0/3610756401 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7fcb880b6700 0x7fcb880b8c10 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:11.835 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:11.827+0000 7fcba4f79640 1 --2- 172.21.15.39:0/3610756401 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcba0109df0 0x7fcba019fc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:11.835 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:11.827+0000 7fcba4f79640 1 --2- 172.21.15.39:0/3610756401 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcba01094a0 0x7fcba019a9e0 unknown :-1 s=CLOSED pgs=657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:11.835 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:11.827+0000 7fcba4f79640 1 --2- 172.21.15.39:0/3610756401 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcba0103b60 0x7fcba019a440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:11.835 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:11.827+0000 7fcba4f79640 1 -- 172.21.15.39:0/3610756401 >> 172.21.15.39:0/3610756401 conn(0x7fcba007b850 msgr2=0x7fcba0105470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:49:11.835 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:11.827+0000 7fcba4f79640 1 -- 172.21.15.39:0/3610756401 shutdown_connections 2024-09-21T03:49:11.835 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:11.827+0000 7fcba4f79640 1 -- 172.21.15.39:0/3610756401 wait complete. 2024-09-21T03:49:12.581 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 51116d5c-77ca-11ef-baf5-efdc52797490 -- ceph tell mds.1:1 damage ls 2024-09-21T03:49:12.834 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:49:12 smithi039 ceph-mon[31211]: pgmap v420: 97 pgs: 97 active+clean; 10 MiB data, 536 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 24 op/s 2024-09-21T03:49:12.842 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/51116d5c-77ca-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:49:12.968 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:49:12 smithi161 ceph-mon[34533]: pgmap v420: 97 pgs: 97 active+clean; 10 MiB data, 536 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 24 op/s 2024-09-21T03:49:12.974 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:49:12 smithi150 ceph-mon[34289]: pgmap v420: 97 pgs: 97 active+clean; 10 MiB data, 536 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 24 op/s 2024-09-21T03:49:13.332 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:13.329+0000 7f44b97b7640 1 -- 172.21.15.39:0/958830737 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f44b4102a80 msgr2=0x7f44b4102e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:49:13.332 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:13.329+0000 7f44b97b7640 1 --2- 172.21.15.39:0/958830737 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f44b4102a80 0x7f44b4102e90 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f44a4009900 tx=0x7f44a402a410 comp rx=0 tx=0).stop 2024-09-21T03:49:13.332 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:13.329+0000 7f44b97b7640 1 -- 172.21.15.39:0/958830737 shutdown_connections 2024-09-21T03:49:13.332 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:13.329+0000 7f44b97b7640 1 --2- 172.21.15.39:0/958830737 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f44b4108d10 0x7f44b410f540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:13.332 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:13.329+0000 7f44b97b7640 1 --2- 172.21.15.39:0/958830737 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f44b41083c0 0x7f44b41087d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:13.332 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:13.330+0000 7f44b97b7640 1 --2- 172.21.15.39:0/958830737 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f44b4102a80 0x7f44b4102e90 unknown :-1 s=CLOSED pgs=662 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:13.332 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:13.330+0000 7f44b97b7640 1 -- 172.21.15.39:0/958830737 >> 172.21.15.39:0/958830737 conn(0x7f44b4079740 msgr2=0x7f44b4079b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:49:13.332 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:13.330+0000 7f44b97b7640 1 -- 172.21.15.39:0/958830737 shutdown_connections 2024-09-21T03:49:13.332 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:13.330+0000 7f44b97b7640 1 -- 172.21.15.39:0/958830737 wait complete. 2024-09-21T03:49:13.332 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:13.330+0000 7f44b97b7640 1 Processor -- start 2024-09-21T03:49:13.334 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:13.330+0000 7f44b97b7640 1 -- start start 2024-09-21T03:49:13.334 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:13.331+0000 7f44b97b7640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f44b4102a80 0x7f44b419b230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:49:13.334 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:13.331+0000 7f44b97b7640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f44b41083c0 0x7f44b419b770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:49:13.334 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:13.331+0000 7f44b97b7640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f44b4108d10 0x7f44b419bcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:49:13.334 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:13.331+0000 7f44b97b7640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f44b4112980 con 0x7f44b41083c0 2024-09-21T03:49:13.334 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:13.331+0000 7f44b97b7640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f44b4112800 con 0x7f44b4102a80 2024-09-21T03:49:13.334 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:13.331+0000 7f44b97b7640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f44b4112b00 con 0x7f44b4108d10 2024-09-21T03:49:13.334 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:13.331+0000 7f44b27fc640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f44b41083c0 0x7f44b419b770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:49:13.335 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:13.331+0000 7f44b27fc640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f44b41083c0 0x7f44b419b770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:33962/0 (socket says 172.21.15.39:33962) 2024-09-21T03:49:13.335 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:13.331+0000 7f44b27fc640 1 -- 172.21.15.39:0/2969780173 learned_addr learned my addr 172.21.15.39:0/2969780173 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:49:13.335 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:13.331+0000 7f44b2ffd640 1 --2- 172.21.15.39:0/2969780173 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f44b4102a80 0x7f44b419b230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:49:13.335 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:13.331+0000 7f44b37fe640 1 --2- 172.21.15.39:0/2969780173 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f44b4108d10 0x7f44b419bcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:49:13.335 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:13.331+0000 7f44b27fc640 1 -- 172.21.15.39:0/2969780173 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f44b4108d10 msgr2=0x7f44b419bcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:49:13.335 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:13.331+0000 7f44b27fc640 1 --2- 172.21.15.39:0/2969780173 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f44b4108d10 0x7f44b419bcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:13.335 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:13.331+0000 7f44b27fc640 1 -- 172.21.15.39:0/2969780173 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f44b4102a80 msgr2=0x7f44b419b230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:49:13.335 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:13.331+0000 7f44b27fc640 1 --2- 172.21.15.39:0/2969780173 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f44b4102a80 0x7f44b419b230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:13.335 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:13.331+0000 7f44b27fc640 1 -- 172.21.15.39:0/2969780173 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f44b41a6540 con 0x7f44b41083c0 2024-09-21T03:49:13.335 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:13.332+0000 7f44b27fc640 1 --2- 172.21.15.39:0/2969780173 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f44b41083c0 0x7f44b419b770 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f44a800ea80 tx=0x7f44a800ef40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:49:13.335 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:13.332+0000 7f449bfff640 1 -- 172.21.15.39:0/2969780173 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44a800ce40 con 0x7f44b41083c0 2024-09-21T03:49:13.335 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:13.332+0000 7f449bfff640 1 -- 172.21.15.39:0/2969780173 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f44a8011920 con 0x7f44b41083c0 2024-09-21T03:49:13.335 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:13.332+0000 7f44b97b7640 1 -- 172.21.15.39:0/2969780173 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f44b41a6830 con 0x7f44b41083c0 2024-09-21T03:49:13.335 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:13.332+0000 7f449bfff640 1 -- 172.21.15.39:0/2969780173 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44a801aa20 con 0x7f44b41083c0 2024-09-21T03:49:13.335 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:13.332+0000 7f44b97b7640 1 -- 172.21.15.39:0/2969780173 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f44b41a6d70 con 0x7f44b41083c0 2024-09-21T03:49:13.337 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:13.335+0000 7f44bfd2b740 1 Processor -- start 2024-09-21T03:49:13.337 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:13.335+0000 7f44bfd2b740 1 -- start start 2024-09-21T03:49:13.338 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:13.335+0000 7f44bfd2b740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55be8f347070 0x55be8f6ee6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:49:13.338 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:13.335+0000 7f44bfd2b740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55be8f31e1b0 0x55be8f6eebe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:49:13.338 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:13.335+0000 7f44bfd2b740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55be8f326390 0x55be8f6ef120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:49:13.338 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:13.335+0000 7f44bfd2b740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x55be8f664ea0 con 0x55be8f326390 2024-09-21T03:49:13.338 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:13.335+0000 7f44bfd2b740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x55be8f6542d0 con 0x55be8f31e1b0 2024-09-21T03:49:13.338 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:13.335+0000 7f44bfd2b740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55be8f6580e0 con 0x55be8f347070 2024-09-21T03:49:13.338 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:13.336+0000 7f449bfff640 1 -- 172.21.15.39:0/2969780173 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 27) ==== 154643+0+0 (secure 0 0 0) 0x7f44a801abc0 con 0x7f44b41083c0 2024-09-21T03:49:13.338 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:13.336+0000 7f44b27fc640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55be8f326390 0x55be8f6ef120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:49:13.338 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:13.336+0000 7f44b37fe640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55be8f347070 0x55be8f6ee6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:49:13.339 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:13.336+0000 7f44b37fe640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55be8f347070 0x55be8f6ee6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.39:39956/0 (socket says 172.21.15.39:39956) 2024-09-21T03:49:13.339 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:13.336+0000 7f44b37fe640 1 -- 172.21.15.39:0/3294822986 learned_addr learned my addr 172.21.15.39:0/3294822986 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:49:13.339 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:13.336+0000 7f44b2ffd640 1 --2- 172.21.15.39:0/3294822986 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55be8f31e1b0 0x55be8f6eebe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:49:13.339 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:13.336+0000 7f44b27fc640 1 -- 172.21.15.39:0/3294822986 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55be8f347070 msgr2=0x55be8f6ee6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:49:13.339 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:13.336+0000 7f44b27fc640 1 --2- 172.21.15.39:0/3294822986 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55be8f347070 0x55be8f6ee6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:13.339 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:13.336+0000 7f44b27fc640 1 -- 172.21.15.39:0/3294822986 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55be8f31e1b0 msgr2=0x55be8f6eebe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:49:13.339 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:13.336+0000 7f44b27fc640 1 --2- 172.21.15.39:0/3294822986 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55be8f31e1b0 0x55be8f6eebe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:13.339 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:13.336+0000 7f44b27fc640 1 -- 172.21.15.39:0/3294822986 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55be8f3b2650 con 0x55be8f326390 2024-09-21T03:49:13.339 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:13.336+0000 7f44b37fe640 1 --2- 172.21.15.39:0/3294822986 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55be8f347070 0x55be8f6ee6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:49:13.339 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:13.336+0000 7f44b27fc640 1 --2- 172.21.15.39:0/3294822986 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55be8f326390 0x55be8f6ef120 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7f44a80e53a0 tx=0x7f44a80ebf00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=991d917e6405468 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:49:13.339 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:13.336+0000 7f449bfff640 1 --2- 172.21.15.39:0/2969780173 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f449c0b6680 0x7f449c0b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:49:13.342 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:13.337+0000 7f44b37fe640 1 --2- 172.21.15.39:0/2969780173 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f449c0b6680 0x7f449c0b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:49:13.342 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:13.337+0000 7f449bfff640 1 -- 172.21.15.39:0/2969780173 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8484+0+0 (secure 0 0 0) 0x7f44a80e7c30 con 0x7f44b41083c0 2024-09-21T03:49:13.342 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:13.337+0000 7f44997fa640 1 -- 172.21.15.39:0/3294822986 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44a80aa0f0 con 0x55be8f326390 2024-09-21T03:49:13.343 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:13.337+0000 7f44997fa640 1 -- 172.21.15.39:0/3294822986 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f44a80f5070 con 0x55be8f326390 2024-09-21T03:49:13.343 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:13.337+0000 7f44997fa640 1 -- 172.21.15.39:0/3294822986 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44a80ad210 con 0x55be8f326390 2024-09-21T03:49:13.343 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:13.338+0000 7f44bfd2b740 1 -- 172.21.15.39:0/3294822986 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55be8f326390 msgr2=0x55be8f6ef120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:49:13.343 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:13.338+0000 7f44b37fe640 1 --2- 172.21.15.39:0/2969780173 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f449c0b6680 0x7f449c0b8b90 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f44ac005bc0 tx=0x7f44ac00b2e0 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:49:13.343 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:13.338+0000 7f44bfd2b740 1 --2- 172.21.15.39:0/3294822986 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55be8f326390 0x55be8f6ef120 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7f44a80e53a0 tx=0x7f44a80ebf00 comp rx=0 tx=0).stop 2024-09-21T03:49:13.343 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:13.338+0000 7f44bfd2b740 1 -- 172.21.15.39:0/3294822986 shutdown_connections 2024-09-21T03:49:13.343 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:13.338+0000 7f44bfd2b740 1 --2- 172.21.15.39:0/3294822986 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55be8f326390 0x55be8f6ef120 unknown :-1 s=CLOSED pgs=664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:13.343 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:13.338+0000 7f44bfd2b740 1 --2- 172.21.15.39:0/3294822986 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55be8f31e1b0 0x55be8f6eebe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:13.343 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:13.338+0000 7f44bfd2b740 1 --2- 172.21.15.39:0/3294822986 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55be8f347070 0x55be8f6ee6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:13.343 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:13.338+0000 7f44bfd2b740 1 -- 172.21.15.39:0/3294822986 >> 172.21.15.39:0/3294822986 conn(0x55be8f33b460 msgr2=0x55be8f6ec590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:49:13.343 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:13.338+0000 7f44bfd2b740 1 -- 172.21.15.39:0/3294822986 shutdown_connections 2024-09-21T03:49:13.343 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:13.338+0000 7f44bfd2b740 1 -- 172.21.15.39:0/3294822986 wait complete. 2024-09-21T03:49:13.343 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:13.338+0000 7f44bfd2b740 1 -- start start 2024-09-21T03:49:13.343 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:13.338+0000 7f44bfd2b740 1 Processor -- start 2024-09-21T03:49:13.344 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:13.339+0000 7f44bfd2b740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55be8f31e1b0 0x55be8f6fe1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:49:13.344 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:13.339+0000 7f44bfd2b740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55be8f326390 0x55be8f6fe710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:49:13.344 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:13.339+0000 7f44bfd2b740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55be8f347070 0x55be8f70a510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:49:13.344 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:13.339+0000 7f44bfd2b740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x55be8f4b5700 con 0x55be8f347070 2024-09-21T03:49:13.344 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:13.339+0000 7f44bfd2b740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x55be8f5ef160 con 0x55be8f31e1b0 2024-09-21T03:49:13.344 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:13.339+0000 7f44bfd2b740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55be8f7011e0 con 0x55be8f326390 2024-09-21T03:49:13.344 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:13.339+0000 7f44bfd2b740 10 client.0 _ll_register_callbacks cb 0x55be8f6089e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:49:13.344 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:13.339+0000 7f44b37fe640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55be8f347070 0x55be8f70a510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:49:13.344 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:13.339+0000 7f44b2ffd640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55be8f31e1b0 0x55be8f6fe1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:49:13.344 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:13.339+0000 7f44b37fe640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55be8f347070 0x55be8f70a510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:33982/0 (socket says 172.21.15.39:33982) 2024-09-21T03:49:13.344 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:13.339+0000 7f44b37fe640 1 -- 172.21.15.39:0/3436989403 learned_addr learned my addr 172.21.15.39:0/3436989403 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:49:13.344 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:13.339+0000 7f44b27fc640 1 --2- 172.21.15.39:0/3436989403 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55be8f326390 0x55be8f6fe710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:49:13.344 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:13.339+0000 7f44b37fe640 1 -- 172.21.15.39:0/3436989403 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55be8f326390 msgr2=0x55be8f6fe710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:49:13.344 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:13.339+0000 7f44b37fe640 1 --2- 172.21.15.39:0/3436989403 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55be8f326390 0x55be8f6fe710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:13.344 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:13.339+0000 7f44b37fe640 1 -- 172.21.15.39:0/3436989403 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55be8f31e1b0 msgr2=0x55be8f6fe1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:49:13.344 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:13.339+0000 7f44b37fe640 1 --2- 172.21.15.39:0/3436989403 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55be8f31e1b0 0x55be8f6fe1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:13.345 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:13.339+0000 7f44b37fe640 1 -- 172.21.15.39:0/3436989403 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55be8f61cc90 con 0x55be8f347070 2024-09-21T03:49:13.345 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:13.339+0000 7f44b37fe640 1 --2- 172.21.15.39:0/3436989403 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55be8f347070 0x55be8f70a510 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7f44ac005880 tx=0x7f44ac004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee54424c024bd492 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:49:13.345 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:13.339+0000 7f4496ff5640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-09-21T03:49:13.345 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:13.340+0000 7f4496ff5640 1 -- 172.21.15.39:0/3436989403 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44ac02c060 con 0x55be8f347070 2024-09-21T03:49:13.345 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:13.340+0000 7f4496ff5640 1 -- 172.21.15.39:0/3436989403 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8484+0+0 (secure 0 0 0) 0x7f44ac017030 con 0x55be8f347070 2024-09-21T03:49:13.345 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:13.340+0000 7f44bfd2b740 1 -- 172.21.15.39:0/3436989403 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f44a80f5070 con 0x55be8f347070 2024-09-21T03:49:13.345 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:13.340+0000 7f4496ff5640 1 -- 172.21.15.39:0/3436989403 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f44ac003e90 con 0x55be8f347070 2024-09-21T03:49:13.345 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:13.340+0000 7f44bfd2b740 10 client.16242 fetch_fsmap learned FSMap version 29 2024-09-21T03:49:13.345 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:13.340+0000 7f44bfd2b740 1 -- 172.21.15.39:0/3436989403 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55be8f70cfc0 con 0x55be8f347070 2024-09-21T03:49:13.345 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:13.340+0000 7f4496ff5640 1 -- 172.21.15.39:0/3436989403 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f44a80f5070 con 0x55be8f347070 2024-09-21T03:49:13.345 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:13.340+0000 7f4496ff5640 1 -- 172.21.15.39:0/3436989403 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f44ac0114f0 con 0x55be8f347070 2024-09-21T03:49:13.345 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:13.340+0000 7f44bfd2b740 10 client.16242 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T03:49:13.345 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:13.340+0000 7f44bfd2b740 10 client.16242 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-21T03:49:13.345 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:13.340+0000 7f44bfd2b740 20 client.16242 populate_metadata read hostname 'smithi039' 2024-09-21T03:49:13.346 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:13.340+0000 7f44bfd2b740 1 --2- 172.21.15.39:0/3436989403 >> [v2:172.21.15.161:6838/2190107270,v1:172.21.15.161:6839/2190107270] conn(0x55be8f3059f0 0x55be8f7104d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:49:13.346 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:13.340+0000 7f44bfd2b740 4 client.16242 mds_command: new command op to 24667 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:49:13.346 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:13.340+0000 7f44bfd2b740 1 -- 172.21.15.39:0/3436989403 --> [v2:172.21.15.161:6838/2190107270,v1:172.21.15.161:6839/2190107270] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55be8f666df0 con 0x55be8f3059f0 2024-09-21T03:49:13.346 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:13.341+0000 7f44b2ffd640 1 --2- 172.21.15.39:0/3436989403 >> [v2:172.21.15.161:6838/2190107270,v1:172.21.15.161:6839/2190107270] conn(0x55be8f3059f0 0x55be8f7104d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:49:13.347 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:13.341+0000 7f44b2ffd640 1 --2- 172.21.15.39:0/3436989403 >> [v2:172.21.15.161:6838/2190107270,v1:172.21.15.161:6839/2190107270] conn(0x55be8f3059f0 0x55be8f7104d0 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=a35a069a49696b43 server_cookie=bd21d49dfb971edd in_seq=0 out_seq=0 2024-09-21T03:49:13.347 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:13.341+0000 7f4496ff5640 10 client.16242 ms_handle_connect on v2:172.21.15.161:6838/2190107270 2024-09-21T03:49:13.347 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:13.342+0000 7f4496ff5640 1 -- 172.21.15.39:0/3436989403 <== mds.1 v2:172.21.15.161:6838/2190107270 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55be8f666df0 con 0x55be8f3059f0 2024-09-21T03:49:13.347 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:13.342+0000 7f4496ff5640 10 client.16242 handle_command_reply: tid=0 2024-09-21T03:49:13.347 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:13.342+0000 7f44bfd2b740 1 client.16242 shutdown 2024-09-21T03:49:13.347 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:13.342+0000 7f44bfd2b740 1 -- 172.21.15.39:0/3436989403 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55be8f347070 msgr2=0x55be8f70a510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:49:13.347 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:13.342+0000 7f44bfd2b740 1 --2- 172.21.15.39:0/3436989403 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55be8f347070 0x55be8f70a510 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7f44ac005880 tx=0x7f44ac004730 comp rx=0 tx=0).stop 2024-09-21T03:49:13.347 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:13.342+0000 7f44bfd2b740 1 -- 172.21.15.39:0/3436989403 shutdown_connections 2024-09-21T03:49:13.347 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:13.342+0000 7f44bfd2b740 1 --2- 172.21.15.39:0/3436989403 >> [v2:172.21.15.161:6838/2190107270,v1:172.21.15.161:6839/2190107270] conn(0x55be8f3059f0 0x55be8f7104d0 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:13.347 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:13.342+0000 7f44bfd2b740 1 --2- 172.21.15.39:0/3436989403 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55be8f347070 0x55be8f70a510 unknown :-1 s=CLOSED pgs=665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:13.347 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:13.342+0000 7f44bfd2b740 1 --2- 172.21.15.39:0/3436989403 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55be8f326390 0x55be8f6fe710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:13.348 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:13.342+0000 7f44bfd2b740 1 --2- 172.21.15.39:0/3436989403 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55be8f31e1b0 0x55be8f6fe1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:13.348 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:13.342+0000 7f44bfd2b740 1 -- 172.21.15.39:0/3436989403 >> 172.21.15.39:0/3436989403 conn(0x55be8f33b460 msgr2=0x55be8f6eabb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:49:13.348 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:13.342+0000 7f4496ff5640 0 client.16242 ms_handle_reset on v2:172.21.15.161:6838/2190107270 2024-09-21T03:49:13.348 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:13.342+0000 7f44bfd2b740 1 -- 172.21.15.39:0/3436989403 shutdown_connections 2024-09-21T03:49:13.348 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:13.342+0000 7f44bfd2b740 1 -- 172.21.15.39:0/3436989403 wait complete. 2024-09-21T03:49:13.348 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:13.342+0000 7f44bfd2b740 20 client.16242 trim_cache size 0 max 16384 2024-09-21T03:49:13.358 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:13.356+0000 7f44bfd2b740 1 Processor -- start 2024-09-21T03:49:13.358 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:13.356+0000 7f44bfd2b740 1 -- start start 2024-09-21T03:49:13.359 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:13.356+0000 7f44bfd2b740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55be8f31e1b0 0x55be8f6f3840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:49:13.359 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:13.356+0000 7f44bfd2b740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55be8f326390 0x55be8f6f3d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:49:13.359 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:13.356+0000 7f44bfd2b740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55be8f347070 0x55be8f6fb3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:49:13.359 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:13.356+0000 7f44bfd2b740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f44b41ad4d0 con 0x55be8f326390 2024-09-21T03:49:13.359 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:13.356+0000 7f44bfd2b740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f44b41af010 con 0x55be8f31e1b0 2024-09-21T03:49:13.359 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:13.356+0000 7f44bfd2b740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f44b41adba0 con 0x55be8f347070 2024-09-21T03:49:13.359 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:13.356+0000 7f44b27fc640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55be8f326390 0x55be8f6f3d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:49:13.359 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:13.356+0000 7f44b27fc640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55be8f326390 0x55be8f6f3d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:33992/0 (socket says 172.21.15.39:33992) 2024-09-21T03:49:13.359 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:13.356+0000 7f44b27fc640 1 -- 172.21.15.39:0/3653080076 learned_addr learned my addr 172.21.15.39:0/3653080076 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:49:13.359 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:13.356+0000 7f44b2ffd640 1 --2- 172.21.15.39:0/3653080076 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55be8f31e1b0 0x55be8f6f3840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:49:13.359 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:13.356+0000 7f44b37fe640 1 --2- 172.21.15.39:0/3653080076 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55be8f347070 0x55be8f6fb3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:49:13.359 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:13.356+0000 7f44b27fc640 1 -- 172.21.15.39:0/3653080076 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55be8f347070 msgr2=0x55be8f6fb3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:49:13.359 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:13.356+0000 7f44b27fc640 1 --2- 172.21.15.39:0/3653080076 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55be8f347070 0x55be8f6fb3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:13.360 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:13.356+0000 7f44b27fc640 1 -- 172.21.15.39:0/3653080076 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55be8f31e1b0 msgr2=0x55be8f6f3840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:49:13.360 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:13.356+0000 7f44b27fc640 1 --2- 172.21.15.39:0/3653080076 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55be8f31e1b0 0x55be8f6f3840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:13.360 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:13.356+0000 7f44b27fc640 1 -- 172.21.15.39:0/3653080076 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55be8f6ef490 con 0x55be8f326390 2024-09-21T03:49:13.360 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:13.357+0000 7f44b27fc640 1 --2- 172.21.15.39:0/3653080076 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55be8f326390 0x55be8f6f3d80 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7f44a80ebfd0 tx=0x7f44a800c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5bc8fd2de1a91e0b server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:49:13.360 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:13.357+0000 7f4498ff9640 1 -- 172.21.15.39:0/3653080076 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44a80ad3f0 con 0x55be8f326390 2024-09-21T03:49:13.361 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:13.357+0000 7f4498ff9640 1 -- 172.21.15.39:0/3653080076 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f44a80f5040 con 0x55be8f326390 2024-09-21T03:49:13.361 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:13.357+0000 7f4498ff9640 1 -- 172.21.15.39:0/3653080076 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44a80ad870 con 0x55be8f326390 2024-09-21T03:49:13.365 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:13.358+0000 7f44bfd2b740 1 -- 172.21.15.39:0/3653080076 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55be8f326390 msgr2=0x55be8f6f3d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:49:13.365 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:13.358+0000 7f44bfd2b740 1 --2- 172.21.15.39:0/3653080076 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55be8f326390 0x55be8f6f3d80 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7f44a80ebfd0 tx=0x7f44a800c8b0 comp rx=0 tx=0).stop 2024-09-21T03:49:13.365 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:13.358+0000 7f44bfd2b740 1 -- 172.21.15.39:0/3653080076 shutdown_connections 2024-09-21T03:49:13.365 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:13.358+0000 7f44bfd2b740 1 --2- 172.21.15.39:0/3653080076 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55be8f347070 0x55be8f6fb3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:13.365 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:13.358+0000 7f44bfd2b740 1 --2- 172.21.15.39:0/3653080076 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55be8f326390 0x55be8f6f3d80 unknown :-1 s=CLOSED pgs=666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:13.365 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:13.358+0000 7f44bfd2b740 1 --2- 172.21.15.39:0/3653080076 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55be8f31e1b0 0x55be8f6f3840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:13.365 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:13.358+0000 7f44bfd2b740 1 -- 172.21.15.39:0/3653080076 >> 172.21.15.39:0/3653080076 conn(0x55be8f33b460 msgr2=0x55be8f6fca50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:49:13.365 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:13.358+0000 7f44bfd2b740 1 -- 172.21.15.39:0/3653080076 shutdown_connections 2024-09-21T03:49:13.365 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:13.358+0000 7f44bfd2b740 1 -- 172.21.15.39:0/3653080076 wait complete. 2024-09-21T03:49:13.365 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:13.358+0000 7f44bfd2b740 1 -- start start 2024-09-21T03:49:13.365 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:13.358+0000 7f44bfd2b740 1 Processor -- start 2024-09-21T03:49:13.365 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:13.358+0000 7f44bfd2b740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55be8f31e1b0 0x55be8f702720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:49:13.365 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:13.358+0000 7f44bfd2b740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55be8f326390 0x55be8f702c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:49:13.365 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:13.358+0000 7f44bfd2b740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55be8f347070 0x55be8f779ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:49:13.366 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:13.359+0000 7f44bfd2b740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f44b41ae940 con 0x55be8f326390 2024-09-21T03:49:13.366 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:13.359+0000 7f44bfd2b740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f44b41ae270 con 0x55be8f347070 2024-09-21T03:49:13.366 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:13.359+0000 7f44bfd2b740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f44b41ab4b0 con 0x55be8f31e1b0 2024-09-21T03:49:13.366 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:13.359+0000 7f44bfd2b740 10 client.0 _ll_register_callbacks cb 0x7f44ac00b4e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:49:13.366 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:13.359+0000 7f44b27fc640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55be8f326390 0x55be8f702c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:49:13.366 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:13.359+0000 7f44b27fc640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55be8f326390 0x55be8f702c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:34006/0 (socket says 172.21.15.39:34006) 2024-09-21T03:49:13.366 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:13.359+0000 7f44b27fc640 1 -- 172.21.15.39:0/3834104196 learned_addr learned my addr 172.21.15.39:0/3834104196 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:49:13.366 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:13.359+0000 7f44b2ffd640 1 --2- 172.21.15.39:0/3834104196 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55be8f31e1b0 0x55be8f702720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:49:13.366 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:13.359+0000 7f44b37fe640 1 --2- 172.21.15.39:0/3834104196 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55be8f347070 0x55be8f779ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:49:13.366 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:13.359+0000 7f44b27fc640 1 -- 172.21.15.39:0/3834104196 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55be8f31e1b0 msgr2=0x55be8f702720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:49:13.366 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:13.359+0000 7f44b27fc640 1 --2- 172.21.15.39:0/3834104196 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55be8f31e1b0 0x55be8f702720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:13.366 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:13.359+0000 7f44b27fc640 1 -- 172.21.15.39:0/3834104196 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55be8f347070 msgr2=0x55be8f779ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:49:13.366 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:13.359+0000 7f44b27fc640 1 --2- 172.21.15.39:0/3834104196 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55be8f347070 0x55be8f779ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:13.367 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:13.359+0000 7f44b27fc640 1 -- 172.21.15.39:0/3834104196 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55be8f7034e0 con 0x55be8f326390 2024-09-21T03:49:13.367 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:13.359+0000 7f44b27fc640 1 --2- 172.21.15.39:0/3834104196 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55be8f326390 0x55be8f702c60 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7f44a80aad10 tx=0x7f44a80ad620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dfd98f290929149a server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:49:13.367 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:13.359+0000 7f4496ff5640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-09-21T03:49:13.367 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:13.359+0000 7f4496ff5640 1 -- 172.21.15.39:0/3834104196 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44a80b2070 con 0x55be8f326390 2024-09-21T03:49:13.367 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:13.359+0000 7f4496ff5640 1 -- 172.21.15.39:0/3834104196 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8484+0+0 (secure 0 0 0) 0x7f44a8011ad0 con 0x55be8f326390 2024-09-21T03:49:13.367 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:13.359+0000 7f44bfd2b740 1 -- 172.21.15.39:0/3834104196 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f44a80f5040 con 0x55be8f326390 2024-09-21T03:49:13.367 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:13.360+0000 7f4496ff5640 1 -- 172.21.15.39:0/3834104196 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f44a80a9940 con 0x55be8f326390 2024-09-21T03:49:13.367 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:13.360+0000 7f44bfd2b740 10 client.16248 fetch_fsmap learned FSMap version 29 2024-09-21T03:49:13.367 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:13.360+0000 7f44bfd2b740 1 -- 172.21.15.39:0/3834104196 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55be8f77b430 con 0x55be8f326390 2024-09-21T03:49:13.367 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:13.360+0000 7f4496ff5640 1 -- 172.21.15.39:0/3834104196 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f44a80f5040 con 0x55be8f326390 2024-09-21T03:49:13.367 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:13.360+0000 7f4496ff5640 1 -- 172.21.15.39:0/3834104196 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f44a8022030 con 0x55be8f326390 2024-09-21T03:49:13.367 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:13.360+0000 7f44bfd2b740 10 client.16248 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T03:49:13.367 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:13.360+0000 7f44bfd2b740 10 client.16248 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-21T03:49:13.368 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:13.360+0000 7f44bfd2b740 20 client.16248 populate_metadata read hostname 'smithi039' 2024-09-21T03:49:13.368 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:13.360+0000 7f44bfd2b740 1 --2- 172.21.15.39:0/3834104196 >> [v2:172.21.15.161:6838/2190107270,v1:172.21.15.161:6839/2190107270] conn(0x55be8f2dada0 0x55be8f77d660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:49:13.368 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:13.360+0000 7f44bfd2b740 4 client.16248 mds_command: new command op to 24667 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T03:49:13.368 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:13.360+0000 7f44bfd2b740 1 -- 172.21.15.39:0/3834104196 --> [v2:172.21.15.161:6838/2190107270,v1:172.21.15.161:6839/2190107270] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55be8f6628a0 con 0x55be8f2dada0 2024-09-21T03:49:13.368 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:13.360+0000 7f44b2ffd640 1 --2- 172.21.15.39:0/3834104196 >> [v2:172.21.15.161:6838/2190107270,v1:172.21.15.161:6839/2190107270] conn(0x55be8f2dada0 0x55be8f77d660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:49:13.368 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:13.361+0000 7f44b2ffd640 1 --2- 172.21.15.39:0/3834104196 >> [v2:172.21.15.161:6838/2190107270,v1:172.21.15.161:6839/2190107270] conn(0x55be8f2dada0 0x55be8f77d660 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=d2b792ff594ca4fd server_cookie=db40da8a10973b47 in_seq=0 out_seq=0 2024-09-21T03:49:13.368 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:13.361+0000 7f4496ff5640 10 client.16248 ms_handle_connect on v2:172.21.15.161:6838/2190107270 2024-09-21T03:49:13.368 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:13.361+0000 7f4496ff5640 1 -- 172.21.15.39:0/3834104196 <== mds.1 v2:172.21.15.161:6838/2190107270 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55be8f6628a0 con 0x55be8f2dada0 2024-09-21T03:49:13.368 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:13.361+0000 7f4496ff5640 10 client.16248 handle_command_reply: tid=0 2024-09-21T03:49:13.368 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:13.361+0000 7f44bfd2b740 1 client.16248 shutdown 2024-09-21T03:49:13.368 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:13.361+0000 7f44bfd2b740 1 -- 172.21.15.39:0/3834104196 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55be8f326390 msgr2=0x55be8f702c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:49:13.368 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:13.361+0000 7f44bfd2b740 1 --2- 172.21.15.39:0/3834104196 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55be8f326390 0x55be8f702c60 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7f44a80aad10 tx=0x7f44a80ad620 comp rx=0 tx=0).stop 2024-09-21T03:49:13.368 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:13.361+0000 7f44bfd2b740 1 -- 172.21.15.39:0/3834104196 shutdown_connections 2024-09-21T03:49:13.368 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:13.361+0000 7f44bfd2b740 1 --2- 172.21.15.39:0/3834104196 >> [v2:172.21.15.161:6838/2190107270,v1:172.21.15.161:6839/2190107270] conn(0x55be8f2dada0 0x55be8f77d660 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:13.369 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:13.361+0000 7f44bfd2b740 1 --2- 172.21.15.39:0/3834104196 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55be8f347070 0x55be8f779ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:13.369 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:13.361+0000 7f44bfd2b740 1 --2- 172.21.15.39:0/3834104196 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55be8f326390 0x55be8f702c60 unknown :-1 s=CLOSED pgs=667 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:13.369 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:13.361+0000 7f44bfd2b740 1 --2- 172.21.15.39:0/3834104196 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55be8f31e1b0 0x55be8f702720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:13.369 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:13.361+0000 7f44bfd2b740 1 -- 172.21.15.39:0/3834104196 >> 172.21.15.39:0/3834104196 conn(0x55be8f33b460 msgr2=0x55be8f6ff680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:49:13.369 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:13.361+0000 7f4496ff5640 0 client.16248 ms_handle_reset on v2:172.21.15.161:6838/2190107270 2024-09-21T03:49:13.369 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:13.361+0000 7f44bfd2b740 1 -- 172.21.15.39:0/3834104196 shutdown_connections 2024-09-21T03:49:13.369 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:13.361+0000 7f44bfd2b740 1 -- 172.21.15.39:0/3834104196 wait complete. 2024-09-21T03:49:13.369 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:13.362+0000 7f44bfd2b740 20 client.16248 trim_cache size 0 max 16384 2024-09-21T03:49:13.369 INFO:teuthology.orchestra.run.smithi039.stdout:[] 2024-09-21T03:49:13.370 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:13.362+0000 7f44b97b7640 1 -- 172.21.15.39:0/2969780173 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f449c0b6680 msgr2=0x7f449c0b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:49:13.370 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:13.362+0000 7f44b97b7640 1 --2- 172.21.15.39:0/2969780173 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f449c0b6680 0x7f449c0b8b90 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f44ac005bc0 tx=0x7f44ac00b2e0 comp rx=0 tx=0).stop 2024-09-21T03:49:13.370 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:13.362+0000 7f44b97b7640 1 -- 172.21.15.39:0/2969780173 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f44b41083c0 msgr2=0x7f44b419b770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:49:13.370 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:13.362+0000 7f44b97b7640 1 --2- 172.21.15.39:0/2969780173 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f44b41083c0 0x7f44b419b770 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f44a800ea80 tx=0x7f44a800ef40 comp rx=0 tx=0).stop 2024-09-21T03:49:13.370 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:13.362+0000 7f44b97b7640 1 -- 172.21.15.39:0/2969780173 shutdown_connections 2024-09-21T03:49:13.370 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:13.362+0000 7f44b97b7640 1 --2- 172.21.15.39:0/2969780173 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f449c0b6680 0x7f449c0b8b90 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:13.371 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:13.362+0000 7f44b97b7640 1 --2- 172.21.15.39:0/2969780173 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f44b4108d10 0x7f44b419bcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:13.371 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:13.362+0000 7f44b97b7640 1 --2- 172.21.15.39:0/2969780173 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f44b41083c0 0x7f44b419b770 unknown :-1 s=CLOSED pgs=663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:13.371 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:13.362+0000 7f44b97b7640 1 --2- 172.21.15.39:0/2969780173 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f44b4102a80 0x7f44b419b230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:13.371 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:13.362+0000 7f44b97b7640 1 -- 172.21.15.39:0/2969780173 >> 172.21.15.39:0/2969780173 conn(0x7f44b4079740 msgr2=0x7f44b4101440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:49:13.371 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:13.362+0000 7f44b97b7640 1 -- 172.21.15.39:0/2969780173 shutdown_connections 2024-09-21T03:49:13.371 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:13.362+0000 7f44b97b7640 1 -- 172.21.15.39:0/2969780173 wait complete. 2024-09-21T03:49:14.164 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 51116d5c-77ca-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 damage ls 2024-09-21T03:49:14.424 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/51116d5c-77ca-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:49:14.834 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:49:14 smithi039 ceph-mon[31211]: pgmap v421: 97 pgs: 97 active+clean; 10 MiB data, 536 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 24 op/s 2024-09-21T03:49:14.968 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:49:14 smithi161 ceph-mon[34533]: pgmap v421: 97 pgs: 97 active+clean; 10 MiB data, 536 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 24 op/s 2024-09-21T03:49:14.969 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:14.925+0000 7f67e7159640 1 -- 172.21.15.39:0/3133264138 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f67e0104860 msgr2=0x7f67e0104c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:49:14.969 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:14.925+0000 7f67e7159640 1 --2- 172.21.15.39:0/3133264138 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f67e0104860 0x7f67e0104c70 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7f67d8009900 tx=0x7f67d802a410 comp rx=0 tx=0).stop 2024-09-21T03:49:14.969 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:14.925+0000 7f67e7159640 1 -- 172.21.15.39:0/3133264138 shutdown_connections 2024-09-21T03:49:14.969 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:14.925+0000 7f67e7159640 1 --2- 172.21.15.39:0/3133264138 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f67e010de10 0x7f67e01102f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:14.969 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:14.925+0000 7f67e7159640 1 --2- 172.21.15.39:0/3133264138 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f67e0105290 0x7f67e010d8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:14.969 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:14.925+0000 7f67e7159640 1 --2- 172.21.15.39:0/3133264138 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f67e0104860 0x7f67e0104c70 unknown :-1 s=CLOSED pgs=668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:14.969 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:14.925+0000 7f67e7159640 1 -- 172.21.15.39:0/3133264138 >> 172.21.15.39:0/3133264138 conn(0x7f67e0079730 msgr2=0x7f67e0079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:49:14.969 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:14.925+0000 7f67e7159640 1 -- 172.21.15.39:0/3133264138 shutdown_connections 2024-09-21T03:49:14.969 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:14.925+0000 7f67e7159640 1 -- 172.21.15.39:0/3133264138 wait complete. 2024-09-21T03:49:14.969 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:14.926+0000 7f67e7159640 1 Processor -- start 2024-09-21T03:49:14.969 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:14.926+0000 7f67e7159640 1 -- start start 2024-09-21T03:49:14.969 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:14.926+0000 7f67e7159640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f67e0104860 0x7f67e0102770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:49:14.970 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:14.926+0000 7f67e7159640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f67e0105290 0x7f67e0102cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:49:14.970 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:14.926+0000 7f67e7159640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f67e010de10 0x7f67e01031f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:49:14.970 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:14.926+0000 7f67e7159640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f67e0113a60 con 0x7f67e010de10 2024-09-21T03:49:14.970 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:14.926+0000 7f67e7159640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f67e01138e0 con 0x7f67e0104860 2024-09-21T03:49:14.970 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:14.926+0000 7f67e7159640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f67e0113be0 con 0x7f67e0105290 2024-09-21T03:49:14.970 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:14.926+0000 7f67e56f6640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f67e010de10 0x7f67e01031f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:49:14.970 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:14.926+0000 7f67e56f6640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f67e010de10 0x7f67e01031f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:34020/0 (socket says 172.21.15.39:34020) 2024-09-21T03:49:14.970 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:14.926+0000 7f67e56f6640 1 -- 172.21.15.39:0/2537366220 learned_addr learned my addr 172.21.15.39:0/2537366220 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:49:14.970 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:14.926+0000 7f67dffff640 1 --2- 172.21.15.39:0/2537366220 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f67e0105290 0x7f67e0102cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:49:14.970 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:14.927+0000 7f67e4ef5640 1 --2- 172.21.15.39:0/2537366220 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f67e0104860 0x7f67e0102770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:49:14.970 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:14.927+0000 7f67e56f6640 1 -- 172.21.15.39:0/2537366220 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f67e0105290 msgr2=0x7f67e0102cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:49:14.970 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:14.927+0000 7f67e56f6640 1 --2- 172.21.15.39:0/2537366220 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f67e0105290 0x7f67e0102cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:14.970 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:14.927+0000 7f67e56f6640 1 -- 172.21.15.39:0/2537366220 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f67e0104860 msgr2=0x7f67e0102770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:49:14.970 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:14.927+0000 7f67e56f6640 1 --2- 172.21.15.39:0/2537366220 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f67e0104860 0x7f67e0102770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:14.970 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:14.927+0000 7f67e56f6640 1 -- 172.21.15.39:0/2537366220 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f67e01a8730 con 0x7f67e010de10 2024-09-21T03:49:14.971 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:14.927+0000 7f67dffff640 1 --2- 172.21.15.39:0/2537366220 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f67e0105290 0x7f67e0102cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:49:14.971 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:14.927+0000 7f67e4ef5640 1 --2- 172.21.15.39:0/2537366220 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f67e0104860 0x7f67e0102770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:49:14.971 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:14.927+0000 7f67e56f6640 1 --2- 172.21.15.39:0/2537366220 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f67e010de10 0x7f67e01031f0 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7f67d0009fd0 tx=0x7f67d000b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:49:14.971 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:14.927+0000 7f67ddffb640 1 -- 172.21.15.39:0/2537366220 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67d0017070 con 0x7f67e010de10 2024-09-21T03:49:14.971 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:14.927+0000 7f67e7159640 1 -- 172.21.15.39:0/2537366220 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f67e01a8a20 con 0x7f67e010de10 2024-09-21T03:49:14.971 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:14.927+0000 7f67ddffb640 1 -- 172.21.15.39:0/2537366220 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f67d0003e20 con 0x7f67e010de10 2024-09-21T03:49:14.971 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:14.927+0000 7f67ddffb640 1 -- 172.21.15.39:0/2537366220 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67d0010460 con 0x7f67e010de10 2024-09-21T03:49:14.971 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:14.927+0000 7f67e7159640 1 -- 172.21.15.39:0/2537366220 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f67e01a8f10 con 0x7f67e010de10 2024-09-21T03:49:14.971 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:14.930+0000 7f67ed6cd740 1 Processor -- start 2024-09-21T03:49:14.971 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:14.930+0000 7f67ddffb640 1 -- 172.21.15.39:0/2537366220 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 27) ==== 154643+0+0 (secure 0 0 0) 0x7f67d0003970 con 0x7f67e010de10 2024-09-21T03:49:14.971 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:14.930+0000 7f67ed6cd740 1 -- start start 2024-09-21T03:49:14.971 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:14.930+0000 7f67ed6cd740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55fc69a3e070 0x55fc69de56a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:49:14.971 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:14.930+0000 7f67ed6cd740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55fc69a151b0 0x55fc69de5be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:49:14.971 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:14.930+0000 7f67ed6cd740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55fc69a1d390 0x55fc69de6120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:49:14.972 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:14.930+0000 7f67ed6cd740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x55fc69d5bea0 con 0x55fc69a3e070 2024-09-21T03:49:14.972 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:14.930+0000 7f67ed6cd740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x55fc69d4b2d0 con 0x55fc69a1d390 2024-09-21T03:49:14.972 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:14.930+0000 7f67ed6cd740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55fc69d4f0e0 con 0x55fc69a151b0 2024-09-21T03:49:14.972 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:14.930+0000 7f67dffff640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55fc69a3e070 0x55fc69de56a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:49:14.972 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:14.930+0000 7f67dffff640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55fc69a3e070 0x55fc69de56a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:34030/0 (socket says 172.21.15.39:34030) 2024-09-21T03:49:14.972 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:14.930+0000 7f67dffff640 1 -- 172.21.15.39:0/2816649841 learned_addr learned my addr 172.21.15.39:0/2816649841 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:49:14.972 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:14.930+0000 7f67e4ef5640 1 --2- 172.21.15.39:0/2816649841 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55fc69a151b0 0x55fc69de5be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:49:14.972 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:14.931+0000 7f67dffff640 1 -- 172.21.15.39:0/2816649841 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55fc69a151b0 msgr2=0x55fc69de5be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:49:14.972 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:14.931+0000 7f67dffff640 1 --2- 172.21.15.39:0/2816649841 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55fc69a151b0 0x55fc69de5be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:14.972 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:14.931+0000 7f67dffff640 1 -- 172.21.15.39:0/2816649841 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55fc69a1d390 msgr2=0x55fc69de6120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:49:14.972 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:14.931+0000 7f67dffff640 1 --2- 172.21.15.39:0/2816649841 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55fc69a1d390 0x55fc69de6120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:14.972 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:14.931+0000 7f67dffff640 1 -- 172.21.15.39:0/2816649841 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fc69aa9650 con 0x55fc69a3e070 2024-09-21T03:49:14.972 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:14.931+0000 7f67ddffb640 1 --2- 172.21.15.39:0/2537366220 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f67cc0b6680 0x7f67cc0b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:49:14.972 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:14.931+0000 7f67e4ef5640 1 --2- 172.21.15.39:0/2537366220 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f67cc0b6680 0x7f67cc0b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:49:14.972 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:14.931+0000 7f67ddffb640 1 -- 172.21.15.39:0/2537366220 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8484+0+0 (secure 0 0 0) 0x7f67d00e7520 con 0x7f67e010de10 2024-09-21T03:49:14.972 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:14.931+0000 7f67dffff640 1 --2- 172.21.15.39:0/2816649841 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55fc69a3e070 0x55fc69de56a0 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7f67c800bd50 tx=0x7f67c8011750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c700e3e07ae11d03 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:49:14.973 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:14.931+0000 7f67d6ffd640 1 -- 172.21.15.39:0/2816649841 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67c801a400 con 0x55fc69a3e070 2024-09-21T03:49:14.973 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:14.931+0000 7f67d6ffd640 1 -- 172.21.15.39:0/2816649841 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f67c801aa50 con 0x55fc69a3e070 2024-09-21T03:49:14.973 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:14.931+0000 7f67d6ffd640 1 -- 172.21.15.39:0/2816649841 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67c8012ad0 con 0x55fc69a3e070 2024-09-21T03:49:14.973 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:14.932+0000 7f67ed6cd740 1 -- 172.21.15.39:0/2816649841 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55fc69a3e070 msgr2=0x55fc69de56a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:49:14.973 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:14.932+0000 7f67e4ef5640 1 --2- 172.21.15.39:0/2537366220 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f67cc0b6680 0x7f67cc0b8b90 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f67d8031440 tx=0x7f67d8005ee0 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:49:14.973 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:14.932+0000 7f67ed6cd740 1 --2- 172.21.15.39:0/2816649841 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55fc69a3e070 0x55fc69de56a0 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7f67c800bd50 tx=0x7f67c8011750 comp rx=0 tx=0).stop 2024-09-21T03:49:14.973 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:14.932+0000 7f67ed6cd740 1 -- 172.21.15.39:0/2816649841 shutdown_connections 2024-09-21T03:49:14.973 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:14.932+0000 7f67ed6cd740 1 --2- 172.21.15.39:0/2816649841 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55fc69a1d390 0x55fc69de6120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:14.973 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:14.932+0000 7f67ed6cd740 1 --2- 172.21.15.39:0/2816649841 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55fc69a151b0 0x55fc69de5be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:14.973 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:14.932+0000 7f67ed6cd740 1 --2- 172.21.15.39:0/2816649841 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55fc69a3e070 0x55fc69de56a0 unknown :-1 s=CLOSED pgs=670 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:14.973 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:14.932+0000 7f67ed6cd740 1 -- 172.21.15.39:0/2816649841 >> 172.21.15.39:0/2816649841 conn(0x55fc69a32460 msgr2=0x55fc69de3590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:49:14.973 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:14.932+0000 7f67ed6cd740 1 -- 172.21.15.39:0/2816649841 shutdown_connections 2024-09-21T03:49:14.973 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:14.932+0000 7f67ed6cd740 1 -- 172.21.15.39:0/2816649841 wait complete. 2024-09-21T03:49:14.973 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:14.932+0000 7f67ed6cd740 1 -- start start 2024-09-21T03:49:14.973 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:14.933+0000 7f67ed6cd740 1 Processor -- start 2024-09-21T03:49:14.974 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:14.933+0000 7f67ed6cd740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55fc69a151b0 0x55fc69df0d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:49:14.974 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:14.933+0000 7f67ed6cd740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55fc69a1d390 0x55fc69df12a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:49:14.974 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:14.933+0000 7f67ed6cd740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55fc69a3e070 0x55fc69dfd170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:49:14.974 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:14.933+0000 7f67ed6cd740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x55fc69bac700 con 0x55fc69a1d390 2024-09-21T03:49:14.974 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:14.933+0000 7f67ed6cd740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x55fc69ce6160 con 0x55fc69a151b0 2024-09-21T03:49:14.974 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:14.933+0000 7f67ed6cd740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55fc69df3e80 con 0x55fc69a3e070 2024-09-21T03:49:14.974 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:14.933+0000 7f67dffff640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55fc69a1d390 0x55fc69df12a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:49:14.974 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:14.933+0000 7f67ed6cd740 10 client.0 _ll_register_callbacks cb 0x55fc69cff9e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:49:14.974 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:14.933+0000 7f67dffff640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55fc69a1d390 0x55fc69df12a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:34042/0 (socket says 172.21.15.39:34042) 2024-09-21T03:49:14.974 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:14.933+0000 7f67dffff640 1 -- 172.21.15.39:0/2235357828 learned_addr learned my addr 172.21.15.39:0/2235357828 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:49:14.974 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:14.933+0000 7f67e56f6640 1 --2- 172.21.15.39:0/2235357828 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55fc69a3e070 0x55fc69dfd170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:49:14.974 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:14.933+0000 7f67e4ef5640 1 --2- 172.21.15.39:0/2235357828 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55fc69a151b0 0x55fc69df0d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:49:14.974 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:14.934+0000 7f67dffff640 1 -- 172.21.15.39:0/2235357828 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55fc69a3e070 msgr2=0x55fc69dfd170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:49:14.974 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:14.934+0000 7f67dffff640 1 --2- 172.21.15.39:0/2235357828 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55fc69a3e070 0x55fc69dfd170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:14.975 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:14.934+0000 7f67dffff640 1 -- 172.21.15.39:0/2235357828 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55fc69a151b0 msgr2=0x55fc69df0d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:49:14.975 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:14.934+0000 7f67dffff640 1 --2- 172.21.15.39:0/2235357828 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55fc69a151b0 0x55fc69df0d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:14.975 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:14.934+0000 7f67dffff640 1 -- 172.21.15.39:0/2235357828 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fc69d13c90 con 0x55fc69a1d390 2024-09-21T03:49:14.975 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:14.934+0000 7f67e4ef5640 1 --2- 172.21.15.39:0/2235357828 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55fc69a151b0 0x55fc69df0d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:49:14.975 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:14.934+0000 7f67dffff640 1 --2- 172.21.15.39:0/2235357828 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55fc69a1d390 0x55fc69df12a0 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7f67c80126b0 tx=0x7f67c800be40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=37495f73870e6de5 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:49:14.975 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:14.934+0000 7f67bffff640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-09-21T03:49:14.975 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:14.934+0000 7f67bffff640 1 -- 172.21.15.39:0/2235357828 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67c801a750 con 0x55fc69a1d390 2024-09-21T03:49:14.975 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:14.934+0000 7f67bffff640 1 -- 172.21.15.39:0/2235357828 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8484+0+0 (secure 0 0 0) 0x7f67c8005030 con 0x55fc69a1d390 2024-09-21T03:49:14.975 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:14.934+0000 7f67ed6cd740 1 -- 172.21.15.39:0/2235357828 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f67c801aa50 con 0x55fc69a1d390 2024-09-21T03:49:14.975 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:14.934+0000 7f67bffff640 1 -- 172.21.15.39:0/2235357828 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f67c8005230 con 0x55fc69a1d390 2024-09-21T03:49:14.975 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:14.934+0000 7f67ed6cd740 10 client.16260 fetch_fsmap learned FSMap version 29 2024-09-21T03:49:14.975 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:14.934+0000 7f67ed6cd740 1 -- 172.21.15.39:0/2235357828 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55fc69dffb90 con 0x55fc69a1d390 2024-09-21T03:49:14.975 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:14.935+0000 7f67bffff640 1 -- 172.21.15.39:0/2235357828 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f67c801aa50 con 0x55fc69a1d390 2024-09-21T03:49:14.975 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:14.935+0000 7f67bffff640 1 -- 172.21.15.39:0/2235357828 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f67c802c020 con 0x55fc69a1d390 2024-09-21T03:49:14.975 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:14.935+0000 7f67ed6cd740 10 client.16260 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T03:49:14.976 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:14.935+0000 7f67ed6cd740 10 client.16260 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T03:49:14.976 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:14.935+0000 7f67ed6cd740 20 client.16260 populate_metadata read hostname 'smithi039' 2024-09-21T03:49:14.976 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:14.935+0000 7f67ed6cd740 1 --2- 172.21.15.39:0/2235357828 >> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] conn(0x55fc699fc9f0 0x55fc69e02ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:49:14.976 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:14.935+0000 7f67ed6cd740 4 client.16260 mds_command: new command op to 24695 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:49:14.976 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:14.935+0000 7f67ed6cd740 1 -- 172.21.15.39:0/2235357828 --> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fc69d5ddf0 con 0x55fc699fc9f0 2024-09-21T03:49:14.976 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:14.935+0000 7f67e4ef5640 1 --2- 172.21.15.39:0/2235357828 >> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] conn(0x55fc699fc9f0 0x55fc69e02ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:49:14.976 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:14.936+0000 7f67bffff640 10 client.16260 ms_handle_connect on v2:172.21.15.150:6838/2949386907 2024-09-21T03:49:14.976 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:14.936+0000 7f67e4ef5640 1 --2- 172.21.15.39:0/2235357828 >> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] conn(0x55fc699fc9f0 0x55fc69e02ef0 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4da3fb605b0a61a2 server_cookie=f6aa4de4f31a8259 in_seq=0 out_seq=0 2024-09-21T03:49:14.976 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:14.936+0000 7f67bffff640 1 -- 172.21.15.39:0/2235357828 <== mds.0 v2:172.21.15.150:6838/2949386907 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55fc69d5ddf0 con 0x55fc699fc9f0 2024-09-21T03:49:14.976 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:14.936+0000 7f67bffff640 10 client.16260 handle_command_reply: tid=0 2024-09-21T03:49:14.976 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:14.936+0000 7f67ed6cd740 1 client.16260 shutdown 2024-09-21T03:49:14.976 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:14.936+0000 7f67ed6cd740 1 -- 172.21.15.39:0/2235357828 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55fc69a1d390 msgr2=0x55fc69df12a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:49:14.976 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:14.936+0000 7f67ed6cd740 1 --2- 172.21.15.39:0/2235357828 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55fc69a1d390 0x55fc69df12a0 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7f67c80126b0 tx=0x7f67c800be40 comp rx=0 tx=0).stop 2024-09-21T03:49:14.976 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:14.937+0000 7f67ed6cd740 1 -- 172.21.15.39:0/2235357828 shutdown_connections 2024-09-21T03:49:14.976 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:14.937+0000 7f67ed6cd740 1 --2- 172.21.15.39:0/2235357828 >> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] conn(0x55fc699fc9f0 0x55fc69e02ef0 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:14.977 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:14.937+0000 7f67ed6cd740 1 --2- 172.21.15.39:0/2235357828 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55fc69a3e070 0x55fc69dfd170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:14.977 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:14.937+0000 7f67ed6cd740 1 --2- 172.21.15.39:0/2235357828 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55fc69a1d390 0x55fc69df12a0 unknown :-1 s=CLOSED pgs=671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:14.977 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:14.937+0000 7f67ed6cd740 1 --2- 172.21.15.39:0/2235357828 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55fc69a151b0 0x55fc69df0d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:14.977 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:14.937+0000 7f67bffff640 0 client.16260 ms_handle_reset on v2:172.21.15.150:6838/2949386907 2024-09-21T03:49:14.977 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:14.937+0000 7f67ed6cd740 1 -- 172.21.15.39:0/2235357828 >> 172.21.15.39:0/2235357828 conn(0x55fc69a32460 msgr2=0x55fc69de8850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:49:14.977 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:14.937+0000 7f67ed6cd740 1 -- 172.21.15.39:0/2235357828 shutdown_connections 2024-09-21T03:49:14.977 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:14.937+0000 7f67ed6cd740 1 -- 172.21.15.39:0/2235357828 wait complete. 2024-09-21T03:49:14.977 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:14.937+0000 7f67ed6cd740 20 client.16260 trim_cache size 0 max 16384 2024-09-21T03:49:14.977 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:14.950+0000 7f67ed6cd740 1 Processor -- start 2024-09-21T03:49:14.977 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:14.950+0000 7f67ed6cd740 1 -- start start 2024-09-21T03:49:14.977 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:14.950+0000 7f67ed6cd740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55fc69a151b0 0x55fc69df1180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:49:14.977 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:14.950+0000 7f67ed6cd740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55fc69a1d390 0x55fc69df2990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:49:14.977 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:14.950+0000 7f67ed6cd740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55fc69a3e070 0x55fc69dfd0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:49:14.977 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:14.950+0000 7f67ed6cd740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f67e01b18e0 con 0x55fc69a3e070 2024-09-21T03:49:14.977 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:14.950+0000 7f67ed6cd740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f67e01b3420 con 0x55fc69a1d390 2024-09-21T03:49:14.978 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:14.950+0000 7f67ed6cd740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f67e01b1fb0 con 0x55fc69a151b0 2024-09-21T03:49:14.978 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:14.950+0000 7f67e56f6640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55fc69a3e070 0x55fc69dfd0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:49:14.978 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:14.951+0000 7f67e56f6640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55fc69a3e070 0x55fc69dfd0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:34048/0 (socket says 172.21.15.39:34048) 2024-09-21T03:49:14.978 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:14.951+0000 7f67e56f6640 1 -- 172.21.15.39:0/2772565560 learned_addr learned my addr 172.21.15.39:0/2772565560 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:49:14.978 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:14.951+0000 7f67e4ef5640 1 --2- 172.21.15.39:0/2772565560 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55fc69a151b0 0x55fc69df1180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:49:14.978 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:14.951+0000 7f67dffff640 1 --2- 172.21.15.39:0/2772565560 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55fc69a1d390 0x55fc69df2990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:49:14.978 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:14.951+0000 7f67e56f6640 1 -- 172.21.15.39:0/2772565560 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55fc69a151b0 msgr2=0x55fc69df1180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:49:14.978 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:14.951+0000 7f67e56f6640 1 --2- 172.21.15.39:0/2772565560 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55fc69a151b0 0x55fc69df1180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:14.978 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:14.951+0000 7f67e56f6640 1 -- 172.21.15.39:0/2772565560 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55fc69a1d390 msgr2=0x55fc69df2990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:49:14.978 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:14.951+0000 7f67e56f6640 1 --2- 172.21.15.39:0/2772565560 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55fc69a1d390 0x55fc69df2990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:14.978 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:14.951+0000 7f67e56f6640 1 -- 172.21.15.39:0/2772565560 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fc69de7990 con 0x55fc69a3e070 2024-09-21T03:49:14.978 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:14.951+0000 7f67e56f6640 1 --2- 172.21.15.39:0/2772565560 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55fc69a3e070 0x55fc69dfd0e0 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7f67d000bf50 tx=0x7f67d00ace10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=57c76a5aeb12aff1 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:49:14.978 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:14.951+0000 7f67d67fc640 1 -- 172.21.15.39:0/2772565560 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67d0020040 con 0x55fc69a3e070 2024-09-21T03:49:14.978 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:14.951+0000 7f67d67fc640 1 -- 172.21.15.39:0/2772565560 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f67d00acea0 con 0x55fc69a3e070 2024-09-21T03:49:14.979 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:14.951+0000 7f67d67fc640 1 -- 172.21.15.39:0/2772565560 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67d00a8ce0 con 0x55fc69a3e070 2024-09-21T03:49:14.979 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:14.952+0000 7f67ed6cd740 1 -- 172.21.15.39:0/2772565560 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55fc69a3e070 msgr2=0x55fc69dfd0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:49:14.979 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:14.952+0000 7f67ed6cd740 1 --2- 172.21.15.39:0/2772565560 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55fc69a3e070 0x55fc69dfd0e0 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7f67d000bf50 tx=0x7f67d00ace10 comp rx=0 tx=0).stop 2024-09-21T03:49:14.979 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:14.952+0000 7f67ed6cd740 1 -- 172.21.15.39:0/2772565560 shutdown_connections 2024-09-21T03:49:14.979 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:14.952+0000 7f67ed6cd740 1 --2- 172.21.15.39:0/2772565560 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55fc69a3e070 0x55fc69dfd0e0 unknown :-1 s=CLOSED pgs=672 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:14.979 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:14.952+0000 7f67ed6cd740 1 --2- 172.21.15.39:0/2772565560 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55fc69a1d390 0x55fc69df2990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:14.979 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:14.952+0000 7f67ed6cd740 1 --2- 172.21.15.39:0/2772565560 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55fc69a151b0 0x55fc69df1180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:14.979 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:14.952+0000 7f67ed6cd740 1 -- 172.21.15.39:0/2772565560 >> 172.21.15.39:0/2772565560 conn(0x55fc69a32460 msgr2=0x55fc69df2280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:49:14.979 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:14.952+0000 7f67ed6cd740 1 -- 172.21.15.39:0/2772565560 shutdown_connections 2024-09-21T03:49:14.979 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:14.952+0000 7f67ed6cd740 1 -- 172.21.15.39:0/2772565560 wait complete. 2024-09-21T03:49:14.979 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:14.952+0000 7f67ed6cd740 1 -- start start 2024-09-21T03:49:14.979 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:14.953+0000 7f67ed6cd740 1 Processor -- start 2024-09-21T03:49:14.979 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:14.953+0000 7f67ed6cd740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55fc69a151b0 0x55fc69e046d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:49:14.979 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:14.953+0000 7f67ed6cd740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55fc69a1d390 0x55fc69e04c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:49:14.980 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:14.953+0000 7f67ed6cd740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55fc69a3e070 0x55fc69e6c670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:49:14.980 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:14.953+0000 7f67ed6cd740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f67e01b2d50 con 0x55fc69a3e070 2024-09-21T03:49:14.980 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:14.953+0000 7f67ed6cd740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f67e01b2680 con 0x55fc69a151b0 2024-09-21T03:49:14.980 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:14.953+0000 7f67ed6cd740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f67e01af8c0 con 0x55fc69a1d390 2024-09-21T03:49:14.980 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:14.953+0000 7f67ed6cd740 10 client.0 _ll_register_callbacks cb 0x7f67c800bc50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:49:14.980 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:14.953+0000 7f67e56f6640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55fc69a3e070 0x55fc69e6c670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:49:14.980 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:14.953+0000 7f67e56f6640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55fc69a3e070 0x55fc69e6c670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:34052/0 (socket says 172.21.15.39:34052) 2024-09-21T03:49:14.980 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:14.953+0000 7f67e56f6640 1 -- 172.21.15.39:0/2611239722 learned_addr learned my addr 172.21.15.39:0/2611239722 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:49:14.980 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:14.954+0000 7f67dffff640 1 --2- 172.21.15.39:0/2611239722 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55fc69a1d390 0x55fc69e04c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:49:14.980 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:14.954+0000 7f67e4ef5640 1 --2- 172.21.15.39:0/2611239722 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55fc69a151b0 0x55fc69e046d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:49:14.980 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:14.954+0000 7f67e56f6640 1 -- 172.21.15.39:0/2611239722 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55fc69a1d390 msgr2=0x55fc69e04c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:49:14.980 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:14.954+0000 7f67e56f6640 1 --2- 172.21.15.39:0/2611239722 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55fc69a1d390 0x55fc69e04c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:14.980 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:14.954+0000 7f67e56f6640 1 -- 172.21.15.39:0/2611239722 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55fc69a151b0 msgr2=0x55fc69e046d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:49:14.980 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:14.954+0000 7f67e56f6640 1 --2- 172.21.15.39:0/2611239722 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55fc69a151b0 0x55fc69e046d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:14.981 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:14.954+0000 7f67e56f6640 1 -- 172.21.15.39:0/2611239722 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fc69e6cc30 con 0x55fc69a3e070 2024-09-21T03:49:14.981 INFO:teuthology.orchestra.run.smithi039.stdout:[] 2024-09-21T03:49:14.981 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:14.954+0000 7f67dffff640 1 --2- 172.21.15.39:0/2611239722 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55fc69a1d390 0x55fc69e04c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:49:14.981 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:14.954+0000 7f67e56f6640 1 --2- 172.21.15.39:0/2611239722 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55fc69a3e070 0x55fc69e6c670 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7f67d00ad320 tx=0x7f67d00e51c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a1c49d495f548539 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:49:14.981 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:14.954+0000 7f67bffff640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-09-21T03:49:14.981 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:14.954+0000 7f67bffff640 1 -- 172.21.15.39:0/2611239722 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67d0014070 con 0x55fc69a3e070 2024-09-21T03:49:14.981 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:14.954+0000 7f67bffff640 1 -- 172.21.15.39:0/2611239722 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8484+0+0 (secure 0 0 0) 0x7f67d00e7040 con 0x55fc69a3e070 2024-09-21T03:49:14.981 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:14.954+0000 7f67ed6cd740 1 -- 172.21.15.39:0/2611239722 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f67d00acea0 con 0x55fc69a3e070 2024-09-21T03:49:14.981 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:14.954+0000 7f67bffff640 1 -- 172.21.15.39:0/2611239722 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f67d0020030 con 0x55fc69a3e070 2024-09-21T03:49:14.982 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:14.954+0000 7f67ed6cd740 10 client.16266 fetch_fsmap learned FSMap version 29 2024-09-21T03:49:14.982 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:14.954+0000 7f67ed6cd740 1 -- 172.21.15.39:0/2611239722 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55fc69e6deb0 con 0x55fc69a3e070 2024-09-21T03:49:14.982 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:14.954+0000 7f67bffff640 1 -- 172.21.15.39:0/2611239722 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f67d00acea0 con 0x55fc69a3e070 2024-09-21T03:49:14.982 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:14.955+0000 7f67bffff640 1 -- 172.21.15.39:0/2611239722 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f67d0094bf0 con 0x55fc69a3e070 2024-09-21T03:49:14.982 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:14.955+0000 7f67ed6cd740 10 client.16266 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T03:49:14.982 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:14.955+0000 7f67ed6cd740 10 client.16266 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T03:49:14.982 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:14.955+0000 7f67ed6cd740 20 client.16266 populate_metadata read hostname 'smithi039' 2024-09-21T03:49:14.982 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:14.955+0000 7f67ed6cd740 1 --2- 172.21.15.39:0/2611239722 >> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] conn(0x55fc699d1da0 0x55fc69e70660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:49:14.982 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:14.955+0000 7f67ed6cd740 4 client.16266 mds_command: new command op to 24695 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T03:49:14.982 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:14.955+0000 7f67ed6cd740 1 -- 172.21.15.39:0/2611239722 --> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55fc69d598a0 con 0x55fc699d1da0 2024-09-21T03:49:14.982 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:14.955+0000 7f67e4ef5640 1 --2- 172.21.15.39:0/2611239722 >> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] conn(0x55fc699d1da0 0x55fc69e70660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:49:14.982 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:14.955+0000 7f67e4ef5640 1 --2- 172.21.15.39:0/2611239722 >> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] conn(0x55fc699d1da0 0x55fc69e70660 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5afa2a4f4ff84d66 server_cookie=b77afac8f7e4227f in_seq=0 out_seq=0 2024-09-21T03:49:14.982 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:14.955+0000 7f67bffff640 10 client.16266 ms_handle_connect on v2:172.21.15.150:6838/2949386907 2024-09-21T03:49:14.982 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:14.956+0000 7f67bffff640 1 -- 172.21.15.39:0/2611239722 <== mds.0 v2:172.21.15.150:6838/2949386907 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55fc69d598a0 con 0x55fc699d1da0 2024-09-21T03:49:14.983 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:14.956+0000 7f67bffff640 10 client.16266 handle_command_reply: tid=0 2024-09-21T03:49:14.983 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:14.956+0000 7f67ed6cd740 1 client.16266 shutdown 2024-09-21T03:49:14.983 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:14.956+0000 7f67ed6cd740 1 -- 172.21.15.39:0/2611239722 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55fc69a3e070 msgr2=0x55fc69e6c670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:49:14.983 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:14.956+0000 7f67ed6cd740 1 --2- 172.21.15.39:0/2611239722 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55fc69a3e070 0x55fc69e6c670 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7f67d00ad320 tx=0x7f67d00e51c0 comp rx=0 tx=0).stop 2024-09-21T03:49:14.983 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:14.956+0000 7f67ed6cd740 1 -- 172.21.15.39:0/2611239722 shutdown_connections 2024-09-21T03:49:14.983 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:14.956+0000 7f67ed6cd740 1 --2- 172.21.15.39:0/2611239722 >> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] conn(0x55fc699d1da0 0x55fc69e70660 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:14.983 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:14.956+0000 7f67ed6cd740 1 --2- 172.21.15.39:0/2611239722 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55fc69a3e070 0x55fc69e6c670 unknown :-1 s=CLOSED pgs=673 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:14.983 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:14.956+0000 7f67ed6cd740 1 --2- 172.21.15.39:0/2611239722 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55fc69a1d390 0x55fc69e04c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:14.983 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:14.956+0000 7f67ed6cd740 1 --2- 172.21.15.39:0/2611239722 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55fc69a151b0 0x55fc69e046d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:14.983 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:14.956+0000 7f67ed6cd740 1 -- 172.21.15.39:0/2611239722 >> 172.21.15.39:0/2611239722 conn(0x55fc69a32460 msgr2=0x55fc69df2020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:49:14.983 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:14.956+0000 7f67bffff640 0 client.16266 ms_handle_reset on v2:172.21.15.150:6838/2949386907 2024-09-21T03:49:14.983 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:14.956+0000 7f67ed6cd740 1 -- 172.21.15.39:0/2611239722 shutdown_connections 2024-09-21T03:49:14.983 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:14.956+0000 7f67ed6cd740 1 -- 172.21.15.39:0/2611239722 wait complete. 2024-09-21T03:49:14.983 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:14.956+0000 7f67ed6cd740 20 client.16266 trim_cache size 0 max 16384 2024-09-21T03:49:14.984 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:14.957+0000 7f67e7159640 1 -- 172.21.15.39:0/2537366220 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f67cc0b6680 msgr2=0x7f67cc0b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:49:14.984 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:14.957+0000 7f67e7159640 1 --2- 172.21.15.39:0/2537366220 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f67cc0b6680 0x7f67cc0b8b90 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f67d8031440 tx=0x7f67d8005ee0 comp rx=0 tx=0).stop 2024-09-21T03:49:14.984 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:14.957+0000 7f67e7159640 1 -- 172.21.15.39:0/2537366220 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f67e010de10 msgr2=0x7f67e01031f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:49:14.984 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:14.957+0000 7f67e7159640 1 --2- 172.21.15.39:0/2537366220 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f67e010de10 0x7f67e01031f0 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7f67d0009fd0 tx=0x7f67d000b260 comp rx=0 tx=0).stop 2024-09-21T03:49:14.984 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:14.957+0000 7f67e7159640 1 -- 172.21.15.39:0/2537366220 shutdown_connections 2024-09-21T03:49:14.984 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:14.957+0000 7f67e7159640 1 --2- 172.21.15.39:0/2537366220 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f67cc0b6680 0x7f67cc0b8b90 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:14.984 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:14.957+0000 7f67e7159640 1 --2- 172.21.15.39:0/2537366220 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f67e010de10 0x7f67e01031f0 unknown :-1 s=CLOSED pgs=669 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:14.984 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:14.957+0000 7f67e7159640 1 --2- 172.21.15.39:0/2537366220 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f67e0105290 0x7f67e0102cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:14.984 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:14.957+0000 7f67e7159640 1 --2- 172.21.15.39:0/2537366220 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f67e0104860 0x7f67e0102770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:14.984 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:14.957+0000 7f67e7159640 1 -- 172.21.15.39:0/2537366220 >> 172.21.15.39:0/2537366220 conn(0x7f67e0079730 msgr2=0x7f67e01008f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:49:14.984 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:14.957+0000 7f67e7159640 1 -- 172.21.15.39:0/2537366220 shutdown_connections 2024-09-21T03:49:14.984 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:14.957+0000 7f67e7159640 1 -- 172.21.15.39:0/2537366220 wait complete. 2024-09-21T03:49:14.985 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:49:14 smithi150 ceph-mon[34289]: pgmap v421: 97 pgs: 97 active+clean; 10 MiB data, 536 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 24 op/s 2024-09-21T03:49:16.912 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-21T03:49:16.913 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 51116d5c-77ca-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-21T03:49:16.968 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:49:16 smithi161 ceph-mon[34533]: pgmap v422: 97 pgs: 97 active+clean; 10 MiB data, 536 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 22 KiB/s wr, 26 op/s 2024-09-21T03:49:16.974 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:49:16 smithi150 ceph-mon[34289]: pgmap v422: 97 pgs: 97 active+clean; 10 MiB data, 536 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 22 KiB/s wr, 26 op/s 2024-09-21T03:49:17.083 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:49:16 smithi039 ceph-mon[31211]: pgmap v422: 97 pgs: 97 active+clean; 10 MiB data, 536 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 22 KiB/s wr, 26 op/s 2024-09-21T03:49:17.186 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/51116d5c-77ca-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:49:17.695 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:17.679+0000 7f8481ecd640 1 -- 172.21.15.39:0/2893786773 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f847c1083a0 msgr2=0x7f847c1087b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:49:17.695 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:17.679+0000 7f8481ecd640 1 --2- 172.21.15.39:0/2893786773 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f847c1083a0 0x7f847c1087b0 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f8464009900 tx=0x7f846402a410 comp rx=0 tx=0).stop 2024-09-21T03:49:17.695 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:17.679+0000 7f8481ecd640 1 -- 172.21.15.39:0/2893786773 shutdown_connections 2024-09-21T03:49:17.695 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:17.679+0000 7f8481ecd640 1 --2- 172.21.15.39:0/2893786773 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f847c108cf0 0x7f847c10f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:17.695 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:17.679+0000 7f8481ecd640 1 --2- 172.21.15.39:0/2893786773 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f847c1083a0 0x7f847c1087b0 unknown :-1 s=CLOSED pgs=674 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:17.695 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:17.679+0000 7f8481ecd640 1 --2- 172.21.15.39:0/2893786773 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f847c102a60 0x7f847c102e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:17.695 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:17.679+0000 7f8481ecd640 1 -- 172.21.15.39:0/2893786773 >> 172.21.15.39:0/2893786773 conn(0x7f847c079730 msgr2=0x7f847c079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:49:17.695 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:17.679+0000 7f8481ecd640 1 -- 172.21.15.39:0/2893786773 shutdown_connections 2024-09-21T03:49:17.696 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:17.679+0000 7f8481ecd640 1 -- 172.21.15.39:0/2893786773 wait complete. 2024-09-21T03:49:17.696 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:17.680+0000 7f8481ecd640 1 Processor -- start 2024-09-21T03:49:17.696 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:17.680+0000 7f8481ecd640 1 -- start start 2024-09-21T03:49:17.696 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:17.680+0000 7f8481ecd640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f847c102a60 0x7f847c19c750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:49:17.696 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:17.680+0000 7f8481ecd640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f847c1083a0 0x7f847c19ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:49:17.696 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:17.680+0000 7f8481ecd640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f847c108cf0 0x7f847c1a1f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:49:17.696 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:17.680+0000 7f8481ecd640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f847c113a80 con 0x7f847c1083a0 2024-09-21T03:49:17.696 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:17.680+0000 7f8481ecd640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f847c113900 con 0x7f847c102a60 2024-09-21T03:49:17.696 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:17.680+0000 7f8481ecd640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f847c113c00 con 0x7f847c108cf0 2024-09-21T03:49:17.696 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:17.680+0000 7f847affd640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f847c1083a0 0x7f847c19ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:49:17.696 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:17.680+0000 7f847affd640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f847c1083a0 0x7f847c19ccf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:55498/0 (socket says 172.21.15.39:55498) 2024-09-21T03:49:17.696 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:17.680+0000 7f847affd640 1 -- 172.21.15.39:0/1379351619 learned_addr learned my addr 172.21.15.39:0/1379351619 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:49:17.696 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:17.681+0000 7f847bfff640 1 --2- 172.21.15.39:0/1379351619 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f847c108cf0 0x7f847c1a1f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:49:17.697 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:17.681+0000 7f847b7fe640 1 --2- 172.21.15.39:0/1379351619 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f847c102a60 0x7f847c19c750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:49:17.697 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:17.681+0000 7f847affd640 1 -- 172.21.15.39:0/1379351619 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f847c108cf0 msgr2=0x7f847c1a1f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:49:17.697 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:17.681+0000 7f847affd640 1 --2- 172.21.15.39:0/1379351619 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f847c108cf0 0x7f847c1a1f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:17.697 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:17.681+0000 7f847affd640 1 -- 172.21.15.39:0/1379351619 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f847c102a60 msgr2=0x7f847c19c750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:49:17.697 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:17.681+0000 7f847affd640 1 --2- 172.21.15.39:0/1379351619 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f847c102a60 0x7f847c19c750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:17.697 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:17.681+0000 7f847affd640 1 -- 172.21.15.39:0/1379351619 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f847c1a2600 con 0x7f847c1083a0 2024-09-21T03:49:17.697 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:17.681+0000 7f847bfff640 1 --2- 172.21.15.39:0/1379351619 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f847c108cf0 0x7f847c1a1f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:49:17.697 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:17.681+0000 7f847b7fe640 1 --2- 172.21.15.39:0/1379351619 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f847c102a60 0x7f847c19c750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:49:17.697 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:17.681+0000 7f847affd640 1 --2- 172.21.15.39:0/1379351619 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f847c1083a0 0x7f847c19ccf0 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7f8464032fd0 tx=0x7f8464004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:49:17.697 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:17.681+0000 7f8478ff9640 1 -- 172.21.15.39:0/1379351619 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f846403f030 con 0x7f847c1083a0 2024-09-21T03:49:17.697 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:17.681+0000 7f8481ecd640 1 -- 172.21.15.39:0/1379351619 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f847c1a2890 con 0x7f847c1083a0 2024-09-21T03:49:17.697 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:17.681+0000 7f8478ff9640 1 -- 172.21.15.39:0/1379351619 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8464004490 con 0x7f847c1083a0 2024-09-21T03:49:17.697 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:17.681+0000 7f8478ff9640 1 -- 172.21.15.39:0/1379351619 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8464035070 con 0x7f847c1083a0 2024-09-21T03:49:17.698 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:17.681+0000 7f8481ecd640 1 -- 172.21.15.39:0/1379351619 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f847c1a2d70 con 0x7f847c1083a0 2024-09-21T03:49:17.698 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:17.684+0000 7f8488441740 1 Processor -- start 2024-09-21T03:49:17.698 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:17.684+0000 7f8488441740 1 -- start start 2024-09-21T03:49:17.698 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:17.684+0000 7f8478ff9640 1 -- 172.21.15.39:0/1379351619 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 27) ==== 154643+0+0 (secure 0 0 0) 0x7f8464042030 con 0x7f847c1083a0 2024-09-21T03:49:17.698 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:17.684+0000 7f8488441740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f575561350 0x55f575908430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:49:17.698 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:17.684+0000 7f8488441740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f575538490 0x55f575908970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:49:17.698 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:17.684+0000 7f8488441740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55f575540670 0x55f575908eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:49:17.698 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:17.684+0000 7f8488441740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x55f57587ee60 con 0x55f575540670 2024-09-21T03:49:17.698 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:17.684+0000 7f8488441740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x55f57586e290 con 0x55f575538490 2024-09-21T03:49:17.698 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:17.684+0000 7f8488441740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55f5758720a0 con 0x55f575561350 2024-09-21T03:49:17.698 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:17.684+0000 7f847affd640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55f575540670 0x55f575908eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:49:17.698 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:17.684+0000 7f847affd640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55f575540670 0x55f575908eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:55510/0 (socket says 172.21.15.39:55510) 2024-09-21T03:49:17.698 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:17.684+0000 7f847affd640 1 -- 172.21.15.39:0/457690533 learned_addr learned my addr 172.21.15.39:0/457690533 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:49:17.699 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:17.685+0000 7f847bfff640 1 --2- 172.21.15.39:0/457690533 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f575561350 0x55f575908430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:49:17.699 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:17.685+0000 7f847b7fe640 1 --2- 172.21.15.39:0/457690533 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f575538490 0x55f575908970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:49:17.699 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:17.685+0000 7f847affd640 1 -- 172.21.15.39:0/457690533 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f575561350 msgr2=0x55f575908430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:49:17.699 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:17.685+0000 7f847affd640 1 --2- 172.21.15.39:0/457690533 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f575561350 0x55f575908430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:17.699 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:17.685+0000 7f847affd640 1 -- 172.21.15.39:0/457690533 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f575538490 msgr2=0x55f575908970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:49:17.699 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:17.685+0000 7f847affd640 1 --2- 172.21.15.39:0/457690533 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f575538490 0x55f575908970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:17.699 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:17.685+0000 7f8478ff9640 1 --2- 172.21.15.39:0/1379351619 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f84680b66b0 0x7f84680b8bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:49:17.699 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:17.685+0000 7f847affd640 1 -- 172.21.15.39:0/457690533 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f5755cc930 con 0x55f575540670 2024-09-21T03:49:17.699 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:17.685+0000 7f847bfff640 1 --2- 172.21.15.39:0/1379351619 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f84680b66b0 0x7f84680b8bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:49:17.699 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:17.685+0000 7f8478ff9640 1 -- 172.21.15.39:0/1379351619 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8484+0+0 (secure 0 0 0) 0x7f8464107c90 con 0x7f847c1083a0 2024-09-21T03:49:17.699 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:17.685+0000 7f847affd640 1 --2- 172.21.15.39:0/457690533 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55f575540670 0x55f575908eb0 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7f8464105370 tx=0x7f846410bf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=35ff0c501f0739cf server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:49:17.699 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:17.685+0000 7f8471ffb640 1 -- 172.21.15.39:0/457690533 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8464049070 con 0x55f575540670 2024-09-21T03:49:17.699 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:17.685+0000 7f8471ffb640 1 -- 172.21.15.39:0/457690533 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f84640cd610 con 0x55f575540670 2024-09-21T03:49:17.700 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:17.685+0000 7f8471ffb640 1 -- 172.21.15.39:0/457690533 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84640cd8d0 con 0x55f575540670 2024-09-21T03:49:17.700 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:17.686+0000 7f8488441740 1 -- 172.21.15.39:0/457690533 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55f575540670 msgr2=0x55f575908eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:49:17.700 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:17.686+0000 7f847bfff640 1 --2- 172.21.15.39:0/1379351619 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f84680b66b0 0x7f84680b8bc0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f847400cca0 tx=0x7f847400c430 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:49:17.700 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:17.686+0000 7f8488441740 1 --2- 172.21.15.39:0/457690533 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55f575540670 0x55f575908eb0 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7f8464105370 tx=0x7f846410bf60 comp rx=0 tx=0).stop 2024-09-21T03:49:17.700 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:17.686+0000 7f8488441740 1 -- 172.21.15.39:0/457690533 shutdown_connections 2024-09-21T03:49:17.700 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:17.686+0000 7f8488441740 1 --2- 172.21.15.39:0/457690533 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55f575540670 0x55f575908eb0 unknown :-1 s=CLOSED pgs=676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:17.700 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:17.686+0000 7f8488441740 1 --2- 172.21.15.39:0/457690533 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f575538490 0x55f575908970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:17.700 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:17.686+0000 7f8488441740 1 --2- 172.21.15.39:0/457690533 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f575561350 0x55f575908430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:17.700 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:17.686+0000 7f8488441740 1 -- 172.21.15.39:0/457690533 >> 172.21.15.39:0/457690533 conn(0x55f575555740 msgr2=0x55f575906320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:49:17.700 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:17.686+0000 7f8488441740 1 -- 172.21.15.39:0/457690533 shutdown_connections 2024-09-21T03:49:17.700 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:17.686+0000 7f8488441740 1 -- 172.21.15.39:0/457690533 wait complete. 2024-09-21T03:49:17.700 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:17.687+0000 7f8488441740 1 -- start start 2024-09-21T03:49:17.700 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:17.687+0000 7f8488441740 1 Processor -- start 2024-09-21T03:49:17.700 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:17.687+0000 7f8488441740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f575538490 0x55f575913ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:49:17.700 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:17.687+0000 7f8488441740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f575540670 0x55f575914430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:49:17.701 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:17.687+0000 7f8488441740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55f575561350 0x55f57591fc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:49:17.701 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:17.687+0000 7f8488441740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x55f5756cf890 con 0x55f575561350 2024-09-21T03:49:17.701 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:17.687+0000 7f8488441740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x55f575808d40 con 0x55f575538490 2024-09-21T03:49:17.701 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:17.687+0000 7f8488441740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55f5759169d0 con 0x55f575540670 2024-09-21T03:49:17.701 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:17.687+0000 7f8488441740 10 client.0 _ll_register_callbacks cb 0x55f57586cc80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:49:17.701 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:17.687+0000 7f847bfff640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55f575561350 0x55f57591fc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:49:17.701 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:17.688+0000 7f847bfff640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55f575561350 0x55f57591fc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:55512/0 (socket says 172.21.15.39:55512) 2024-09-21T03:49:17.701 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:17.688+0000 7f847bfff640 1 -- 172.21.15.39:0/304665327 learned_addr learned my addr 172.21.15.39:0/304665327 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:49:17.701 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:17.688+0000 7f847affd640 1 --2- 172.21.15.39:0/304665327 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f575540670 0x55f575914430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:49:17.701 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:17.688+0000 7f847b7fe640 1 --2- 172.21.15.39:0/304665327 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f575538490 0x55f575913ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:49:17.701 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:17.688+0000 7f847bfff640 1 -- 172.21.15.39:0/304665327 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f575540670 msgr2=0x55f575914430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:49:17.701 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:17.688+0000 7f847bfff640 1 --2- 172.21.15.39:0/304665327 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f575540670 0x55f575914430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:17.701 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:17.688+0000 7f847bfff640 1 -- 172.21.15.39:0/304665327 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f575538490 msgr2=0x55f575913ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:49:17.701 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:17.688+0000 7f847bfff640 1 --2- 172.21.15.39:0/304665327 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f575538490 0x55f575913ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:17.701 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:17.688+0000 7f847bfff640 1 -- 172.21.15.39:0/304665327 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f5759201c0 con 0x55f575561350 2024-09-21T03:49:17.702 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:17.688+0000 7f847bfff640 1 --2- 172.21.15.39:0/304665327 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55f575561350 0x55f57591fc80 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7f847400ed60 tx=0x7f84740046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c5d1a86777e35c7 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:49:17.702 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:17.688+0000 7f845affd640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-09-21T03:49:17.702 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:17.688+0000 7f845affd640 1 -- 172.21.15.39:0/304665327 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f847401cb20 con 0x55f575561350 2024-09-21T03:49:17.702 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:17.688+0000 7f845affd640 1 -- 172.21.15.39:0/304665327 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8484+0+0 (secure 0 0 0) 0x7f847401ccc0 con 0x55f575561350 2024-09-21T03:49:17.702 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:17.688+0000 7f8488441740 1 -- 172.21.15.39:0/304665327 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f84640cd610 con 0x55f575561350 2024-09-21T03:49:17.702 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:17.689+0000 7f845affd640 1 -- 172.21.15.39:0/304665327 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f8474004020 con 0x55f575561350 2024-09-21T03:49:17.702 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:17.689+0000 7f8488441740 10 client.16278 fetch_fsmap learned FSMap version 29 2024-09-21T03:49:17.702 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:17.689+0000 7f8488441740 1 -- 172.21.15.39:0/304665327 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55f575922ae0 con 0x55f575561350 2024-09-21T03:49:17.702 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:17.689+0000 7f845affd640 1 -- 172.21.15.39:0/304665327 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f84640cd610 con 0x55f575561350 2024-09-21T03:49:17.702 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:17.689+0000 7f845affd640 1 -- 172.21.15.39:0/304665327 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f847402f020 con 0x55f575561350 2024-09-21T03:49:17.702 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:17.689+0000 7f8488441740 10 client.16278 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T03:49:17.702 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:17.689+0000 7f8488441740 10 client.16278 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T03:49:17.702 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:17.689+0000 7f8488441740 20 client.16278 populate_metadata read hostname 'smithi039' 2024-09-21T03:49:17.702 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:17.689+0000 7f8488441740 1 --2- 172.21.15.39:0/304665327 >> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] conn(0x55f57551fcd0 0x55f575925f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:49:17.702 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:17.689+0000 7f8488441740 4 client.16278 mds_command: new command op to 24695 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:49:17.703 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:17.689+0000 7f8488441740 1 -- 172.21.15.39:0/304665327 --> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f575880db0 con 0x55f57551fcd0 2024-09-21T03:49:17.703 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:17.689+0000 7f847b7fe640 1 --2- 172.21.15.39:0/304665327 >> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] conn(0x55f57551fcd0 0x55f575925f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:49:17.703 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:17.690+0000 7f845affd640 10 client.16278 ms_handle_connect on v2:172.21.15.150:6838/2949386907 2024-09-21T03:49:17.703 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:17.690+0000 7f847b7fe640 1 --2- 172.21.15.39:0/304665327 >> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] conn(0x55f57551fcd0 0x55f575925f90 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f6994a1c393fd0d3 server_cookie=daeed97002529b6f in_seq=0 out_seq=0 2024-09-21T03:49:17.703 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:17.691+0000 7f845affd640 1 -- 172.21.15.39:0/304665327 <== mds.0 v2:172.21.15.150:6838/2949386907 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55f575880db0 con 0x55f57551fcd0 2024-09-21T03:49:17.703 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:17.691+0000 7f845affd640 10 client.16278 handle_command_reply: tid=0 2024-09-21T03:49:17.703 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:17.691+0000 7f8488441740 1 client.16278 shutdown 2024-09-21T03:49:17.703 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:17.691+0000 7f8488441740 1 -- 172.21.15.39:0/304665327 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55f575561350 msgr2=0x55f57591fc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:49:17.703 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:17.691+0000 7f8488441740 1 --2- 172.21.15.39:0/304665327 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55f575561350 0x55f57591fc80 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7f847400ed60 tx=0x7f84740046c0 comp rx=0 tx=0).stop 2024-09-21T03:49:17.703 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:17.691+0000 7f8488441740 1 -- 172.21.15.39:0/304665327 shutdown_connections 2024-09-21T03:49:17.703 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:17.691+0000 7f8488441740 1 --2- 172.21.15.39:0/304665327 >> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] conn(0x55f57551fcd0 0x55f575925f90 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:17.703 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:17.691+0000 7f8488441740 1 --2- 172.21.15.39:0/304665327 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55f575561350 0x55f57591fc80 unknown :-1 s=CLOSED pgs=677 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:17.704 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:17.691+0000 7f8488441740 1 --2- 172.21.15.39:0/304665327 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f575540670 0x55f575914430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:17.704 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:17.691+0000 7f8488441740 1 --2- 172.21.15.39:0/304665327 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f575538490 0x55f575913ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:17.704 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:17.691+0000 7f8488441740 1 -- 172.21.15.39:0/304665327 >> 172.21.15.39:0/304665327 conn(0x55f575555740 msgr2=0x55f57590b5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:49:17.704 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:17.691+0000 7f845affd640 0 client.16278 ms_handle_reset on v2:172.21.15.150:6838/2949386907 2024-09-21T03:49:17.704 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:17.691+0000 7f8488441740 1 -- 172.21.15.39:0/304665327 shutdown_connections 2024-09-21T03:49:17.704 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:17.691+0000 7f8488441740 1 -- 172.21.15.39:0/304665327 wait complete. 2024-09-21T03:49:17.704 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:17.691+0000 7f8488441740 20 client.16278 trim_cache size 0 max 16384 2024-09-21T03:49:17.712 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:17.709+0000 7f8488441740 1 Processor -- start 2024-09-21T03:49:17.712 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:17.709+0000 7f8488441740 1 -- start start 2024-09-21T03:49:17.712 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:17.709+0000 7f8488441740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f575538490 0x55f57590e550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:49:17.713 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:17.709+0000 7f8488441740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f575540670 0x55f575915920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:49:17.713 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:17.709+0000 7f8488441740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55f575561350 0x55f57592b790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:49:17.713 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:17.709+0000 7f8488441740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f847c1b1900 con 0x55f575561350 2024-09-21T03:49:17.713 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:17.709+0000 7f8488441740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f847c1b3440 con 0x55f575540670 2024-09-21T03:49:17.713 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:17.709+0000 7f8488441740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f847c1b1fd0 con 0x55f575538490 2024-09-21T03:49:17.713 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:17.710+0000 7f847bfff640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55f575561350 0x55f57592b790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:49:17.713 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:17.710+0000 7f847bfff640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55f575561350 0x55f57592b790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:55524/0 (socket says 172.21.15.39:55524) 2024-09-21T03:49:17.713 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:17.710+0000 7f847bfff640 1 -- 172.21.15.39:0/2701172829 learned_addr learned my addr 172.21.15.39:0/2701172829 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:49:17.713 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:17.710+0000 7f847b7fe640 1 --2- 172.21.15.39:0/2701172829 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f575538490 0x55f57590e550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:49:17.713 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:17.710+0000 7f847affd640 1 --2- 172.21.15.39:0/2701172829 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f575540670 0x55f575915920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:49:17.715 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:17.710+0000 7f847bfff640 1 -- 172.21.15.39:0/2701172829 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f575538490 msgr2=0x55f57590e550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:49:17.715 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:17.710+0000 7f847bfff640 1 --2- 172.21.15.39:0/2701172829 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f575538490 0x55f57590e550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:17.715 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:17.710+0000 7f847bfff640 1 -- 172.21.15.39:0/2701172829 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f575540670 msgr2=0x55f575915920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:49:17.715 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:17.710+0000 7f847bfff640 1 --2- 172.21.15.39:0/2701172829 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f575540670 0x55f575915920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:17.715 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:17.710+0000 7f847bfff640 1 -- 172.21.15.39:0/2701172829 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f5759166c0 con 0x55f575561350 2024-09-21T03:49:17.715 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:17.710+0000 7f847bfff640 1 --2- 172.21.15.39:0/2701172829 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55f575561350 0x55f57592b790 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7f8474003d80 tx=0x7f847400e750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2445906a1502a525 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:49:17.715 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:17.710+0000 7f84717fa640 1 -- 172.21.15.39:0/2701172829 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8474031610 con 0x55f575561350 2024-09-21T03:49:17.715 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:17.710+0000 7f84717fa640 1 -- 172.21.15.39:0/2701172829 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8474003aa0 con 0x55f575561350 2024-09-21T03:49:17.715 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:17.710+0000 7f84717fa640 1 -- 172.21.15.39:0/2701172829 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8474031d70 con 0x55f575561350 2024-09-21T03:49:17.716 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:17.711+0000 7f8488441740 1 -- 172.21.15.39:0/2701172829 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55f575561350 msgr2=0x55f57592b790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:49:17.716 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:17.711+0000 7f8488441740 1 --2- 172.21.15.39:0/2701172829 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55f575561350 0x55f57592b790 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7f8474003d80 tx=0x7f847400e750 comp rx=0 tx=0).stop 2024-09-21T03:49:17.716 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:17.711+0000 7f8488441740 1 -- 172.21.15.39:0/2701172829 shutdown_connections 2024-09-21T03:49:17.716 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:17.711+0000 7f8488441740 1 --2- 172.21.15.39:0/2701172829 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55f575561350 0x55f57592b790 unknown :-1 s=CLOSED pgs=678 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:17.716 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:17.711+0000 7f8488441740 1 --2- 172.21.15.39:0/2701172829 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f575540670 0x55f575915920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:17.716 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:17.711+0000 7f8488441740 1 --2- 172.21.15.39:0/2701172829 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f575538490 0x55f57590e550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:17.716 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:17.711+0000 7f8488441740 1 -- 172.21.15.39:0/2701172829 >> 172.21.15.39:0/2701172829 conn(0x55f575555740 msgr2=0x55f575915210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:49:17.716 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:17.711+0000 7f8488441740 1 -- 172.21.15.39:0/2701172829 shutdown_connections 2024-09-21T03:49:17.716 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:17.711+0000 7f8488441740 1 -- 172.21.15.39:0/2701172829 wait complete. 2024-09-21T03:49:17.716 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:17.712+0000 7f8488441740 1 -- start start 2024-09-21T03:49:17.716 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:17.712+0000 7f8488441740 1 Processor -- start 2024-09-21T03:49:17.718 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:17.712+0000 7f8488441740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f575538490 0x55f575917f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:49:17.718 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:17.713+0000 7f8488441740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f575540670 0x55f575918440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:49:17.718 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:17.713+0000 7f8488441740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55f575561350 0x55f57598f990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:49:17.718 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:17.713+0000 7f8488441740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f847c1b2d70 con 0x55f575561350 2024-09-21T03:49:17.718 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:17.713+0000 7f8488441740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f847c1b26a0 con 0x55f575540670 2024-09-21T03:49:17.718 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:17.713+0000 7f8488441740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f847c1af8e0 con 0x55f575538490 2024-09-21T03:49:17.718 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:17.713+0000 7f8488441740 10 client.0 _ll_register_callbacks cb 0x7f847400ea10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:49:17.718 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:17.713+0000 7f847bfff640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55f575561350 0x55f57598f990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:49:17.719 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:17.713+0000 7f847bfff640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55f575561350 0x55f57598f990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:55526/0 (socket says 172.21.15.39:55526) 2024-09-21T03:49:17.719 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:17.713+0000 7f847bfff640 1 -- 172.21.15.39:0/1489817920 learned_addr learned my addr 172.21.15.39:0/1489817920 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:49:17.719 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:17.713+0000 7f847affd640 1 --2- 172.21.15.39:0/1489817920 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f575540670 0x55f575918440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:49:17.719 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:17.713+0000 7f847b7fe640 1 --2- 172.21.15.39:0/1489817920 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f575538490 0x55f575917f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:49:17.719 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:17.713+0000 7f847bfff640 1 -- 172.21.15.39:0/1489817920 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f575538490 msgr2=0x55f575917f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:49:17.719 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:17.713+0000 7f847bfff640 1 --2- 172.21.15.39:0/1489817920 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f575538490 0x55f575917f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:17.719 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:17.713+0000 7f847bfff640 1 -- 172.21.15.39:0/1489817920 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f575540670 msgr2=0x55f575918440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:49:17.719 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:17.713+0000 7f847bfff640 1 --2- 172.21.15.39:0/1489817920 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f575540670 0x55f575918440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:17.719 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:17.713+0000 7f847bfff640 1 -- 172.21.15.39:0/1489817920 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f575918ce0 con 0x55f575561350 2024-09-21T03:49:17.719 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:17.713+0000 7f847b7fe640 1 --2- 172.21.15.39:0/1489817920 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f575538490 0x55f575917f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:49:17.719 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:17.713+0000 7f847affd640 1 --2- 172.21.15.39:0/1489817920 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f575540670 0x55f575918440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:49:17.719 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:17.713+0000 7f847bfff640 1 --2- 172.21.15.39:0/1489817920 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55f575561350 0x55f57598f990 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7f8474011dc0 tx=0x7f84740043f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=31c71a6fc615aabc server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:49:17.719 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:17.713+0000 7f845affd640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-09-21T03:49:17.719 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:17.713+0000 7f845affd640 1 -- 172.21.15.39:0/1489817920 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8474031860 con 0x55f575561350 2024-09-21T03:49:17.719 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:17.714+0000 7f845affd640 1 -- 172.21.15.39:0/1489817920 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8484+0+0 (secure 0 0 0) 0x7f8474031ab0 con 0x55f575561350 2024-09-21T03:49:17.719 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:17.714+0000 7f8488441740 1 -- 172.21.15.39:0/1489817920 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8474003aa0 con 0x55f575561350 2024-09-21T03:49:17.720 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:17.714+0000 7f845affd640 1 -- 172.21.15.39:0/1489817920 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f8474031cb0 con 0x55f575561350 2024-09-21T03:49:17.720 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:17.714+0000 7f8488441740 10 client.16284 fetch_fsmap learned FSMap version 29 2024-09-21T03:49:17.720 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:17.714+0000 7f8488441740 1 -- 172.21.15.39:0/1489817920 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55f575990f60 con 0x55f575561350 2024-09-21T03:49:17.720 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:17.717+0000 7f845affd640 1 -- 172.21.15.39:0/1489817920 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8474003aa0 con 0x55f575561350 2024-09-21T03:49:17.721 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:17.717+0000 7f845affd640 1 -- 172.21.15.39:0/1489817920 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f847402f040 con 0x55f575561350 2024-09-21T03:49:17.721 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:17.717+0000 7f8488441740 10 client.16284 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T03:49:17.721 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:17.717+0000 7f8488441740 10 client.16284 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T03:49:17.721 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:17.717+0000 7f8488441740 20 client.16284 populate_metadata read hostname 'smithi039' 2024-09-21T03:49:17.721 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:17.717+0000 7f8488441740 1 --2- 172.21.15.39:0/1489817920 >> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] conn(0x55f5754f5080 0x55f5759935a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:49:17.721 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:17.717+0000 7f8488441740 4 client.16284 mds_command: new command op to 24695 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-21T03:49:17.721 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:17.717+0000 7f8488441740 1 -- 172.21.15.39:0/1489817920 --> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55f57587c860 con 0x55f5754f5080 2024-09-21T03:49:17.721 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:17.717+0000 7f847b7fe640 1 --2- 172.21.15.39:0/1489817920 >> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] conn(0x55f5754f5080 0x55f5759935a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:49:17.721 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:17.718+0000 7f845affd640 10 client.16284 ms_handle_connect on v2:172.21.15.150:6838/2949386907 2024-09-21T03:49:17.721 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:17.718+0000 7f847b7fe640 1 --2- 172.21.15.39:0/1489817920 >> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] conn(0x55f5754f5080 0x55f5759935a0 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1bad9b74d6bc2cdd server_cookie=c6a13c5e6c832b0d in_seq=0 out_seq=0 2024-09-21T03:49:17.723 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:17.720+0000 7f845affd640 1 -- 172.21.15.39:0/1489817920 <== mds.0 v2:172.21.15.150:6838/2949386907 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55f57587c860 con 0x55f5754f5080 2024-09-21T03:49:17.723 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:17.720+0000 7f845affd640 10 client.16284 handle_command_reply: tid=0 2024-09-21T03:49:17.723 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:17.720+0000 7f8488441740 1 client.16284 shutdown 2024-09-21T03:49:17.723 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:17.720+0000 7f8488441740 1 -- 172.21.15.39:0/1489817920 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55f575561350 msgr2=0x55f57598f990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:49:17.723 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:17.720+0000 7f8488441740 1 --2- 172.21.15.39:0/1489817920 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55f575561350 0x55f57598f990 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7f8474011dc0 tx=0x7f84740043f0 comp rx=0 tx=0).stop 2024-09-21T03:49:17.723 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:17.720+0000 7f8488441740 1 -- 172.21.15.39:0/1489817920 shutdown_connections 2024-09-21T03:49:17.723 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:17.720+0000 7f8488441740 1 --2- 172.21.15.39:0/1489817920 >> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] conn(0x55f5754f5080 0x55f5759935a0 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:17.723 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:17.720+0000 7f8488441740 1 --2- 172.21.15.39:0/1489817920 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55f575561350 0x55f57598f990 unknown :-1 s=CLOSED pgs=679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:17.723 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:17.720+0000 7f8488441740 1 --2- 172.21.15.39:0/1489817920 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f575540670 0x55f575918440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:17.724 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:17.720+0000 7f845affd640 0 client.16284 ms_handle_reset on v2:172.21.15.150:6838/2949386907 2024-09-21T03:49:17.724 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:17.720+0000 7f8488441740 1 --2- 172.21.15.39:0/1489817920 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f575538490 0x55f575917f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:17.724 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:17.720+0000 7f8488441740 1 -- 172.21.15.39:0/1489817920 >> 172.21.15.39:0/1489817920 conn(0x55f575555740 msgr2=0x55f575915b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:49:17.724 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:17.720+0000 7f8488441740 1 -- 172.21.15.39:0/1489817920 shutdown_connections 2024-09-21T03:49:17.724 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:17.720+0000 7f8488441740 1 -- 172.21.15.39:0/1489817920 wait complete. 2024-09-21T03:49:17.724 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:17.720+0000 7f8488441740 20 client.16284 trim_cache size 0 max 16384 2024-09-21T03:49:17.724 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-09-21T03:49:17.724 INFO:teuthology.orchestra.run.smithi039.stdout: "return_code": 0, 2024-09-21T03:49:17.724 INFO:teuthology.orchestra.run.smithi039.stdout: "scrub_tag": "9860c2f4-b7b1-47aa-865a-9a1c4434a5f6", 2024-09-21T03:49:17.724 INFO:teuthology.orchestra.run.smithi039.stdout: "mode": "asynchronous" 2024-09-21T03:49:17.724 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-09-21T03:49:17.725 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:17.721+0000 7f8481ecd640 1 -- 172.21.15.39:0/1379351619 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f84680b66b0 msgr2=0x7f84680b8bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:49:17.725 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:17.721+0000 7f8481ecd640 1 --2- 172.21.15.39:0/1379351619 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f84680b66b0 0x7f84680b8bc0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f847400cca0 tx=0x7f847400c430 comp rx=0 tx=0).stop 2024-09-21T03:49:17.725 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:17.721+0000 7f8481ecd640 1 -- 172.21.15.39:0/1379351619 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f847c1083a0 msgr2=0x7f847c19ccf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:49:17.725 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:17.721+0000 7f8481ecd640 1 --2- 172.21.15.39:0/1379351619 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f847c1083a0 0x7f847c19ccf0 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7f8464032fd0 tx=0x7f8464004270 comp rx=0 tx=0).stop 2024-09-21T03:49:17.725 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:17.721+0000 7f8481ecd640 1 -- 172.21.15.39:0/1379351619 shutdown_connections 2024-09-21T03:49:17.725 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:17.721+0000 7f8481ecd640 1 --2- 172.21.15.39:0/1379351619 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f84680b66b0 0x7f84680b8bc0 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:17.725 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:17.721+0000 7f8481ecd640 1 --2- 172.21.15.39:0/1379351619 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f847c108cf0 0x7f847c1a1f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:17.725 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:17.721+0000 7f8481ecd640 1 --2- 172.21.15.39:0/1379351619 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f847c1083a0 0x7f847c19ccf0 unknown :-1 s=CLOSED pgs=675 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:17.726 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:17.721+0000 7f8481ecd640 1 --2- 172.21.15.39:0/1379351619 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f847c102a60 0x7f847c19c750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:17.726 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:17.721+0000 7f8481ecd640 1 -- 172.21.15.39:0/1379351619 >> 172.21.15.39:0/1379351619 conn(0x7f847c079730 msgr2=0x7f847c104270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:49:17.726 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:17.721+0000 7f8481ecd640 1 -- 172.21.15.39:0/1379351619 shutdown_connections 2024-09-21T03:49:17.726 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:17.721+0000 7f8481ecd640 1 -- 172.21.15.39:0/1379351619 wait complete. 2024-09-21T03:49:18.637 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 51116d5c-77ca-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub status 2024-09-21T03:49:18.895 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/51116d5c-77ca-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:49:18.975 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:49:18 smithi150 ceph-mon[34289]: pgmap v423: 97 pgs: 97 active+clean; 10 MiB data, 536 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 170 B/s wr, 9 op/s 2024-09-21T03:49:18.975 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:49:18 smithi150 ceph-mon[34289]: scrub queued for path: / 2024-09-21T03:49:18.975 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:49:18 smithi150 ceph-mon[34289]: scrub summary: idle+waiting paths [/] 2024-09-21T03:49:18.975 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:49:18 smithi150 ceph-mon[34289]: scrub summary: active paths [/] 2024-09-21T03:49:18.975 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:49:18 smithi150 ceph-mon[34289]: scrub summary: idle+waiting paths [/] 2024-09-21T03:49:19.009 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:49:18 smithi039 ceph-mon[31211]: pgmap v423: 97 pgs: 97 active+clean; 10 MiB data, 536 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 170 B/s wr, 9 op/s 2024-09-21T03:49:19.010 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:49:18 smithi039 ceph-mon[31211]: scrub queued for path: / 2024-09-21T03:49:19.010 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:49:18 smithi039 ceph-mon[31211]: scrub summary: idle+waiting paths [/] 2024-09-21T03:49:19.010 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:49:18 smithi039 ceph-mon[31211]: scrub summary: active paths [/] 2024-09-21T03:49:19.010 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:49:18 smithi039 ceph-mon[31211]: scrub summary: idle+waiting paths [/] 2024-09-21T03:49:19.218 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:49:18 smithi161 ceph-mon[34533]: pgmap v423: 97 pgs: 97 active+clean; 10 MiB data, 536 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 170 B/s wr, 9 op/s 2024-09-21T03:49:19.273 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:49:18 smithi161 ceph-mon[34533]: scrub queued for path: / 2024-09-21T03:49:19.274 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:49:18 smithi161 ceph-mon[34533]: scrub summary: idle+waiting paths [/] 2024-09-21T03:49:19.274 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:49:18 smithi161 ceph-mon[34533]: scrub summary: active paths [/] 2024-09-21T03:49:19.274 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:49:18 smithi161 ceph-mon[34533]: scrub summary: idle+waiting paths [/] 2024-09-21T03:49:19.363 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:19.345+0000 7f791b463640 1 -- 172.21.15.39:0/2178097076 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f79141083d0 msgr2=0x7f79141087e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:49:19.363 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:19.345+0000 7f791b463640 1 --2- 172.21.15.39:0/2178097076 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f79141083d0 0x7f79141087e0 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7f7908009900 tx=0x7f790802a410 comp rx=0 tx=0).stop 2024-09-21T03:49:19.363 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:19.345+0000 7f791b463640 1 -- 172.21.15.39:0/2178097076 shutdown_connections 2024-09-21T03:49:19.363 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:19.345+0000 7f791b463640 1 --2- 172.21.15.39:0/2178097076 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7914108d20 0x7f791410f550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:19.363 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:19.345+0000 7f791b463640 1 --2- 172.21.15.39:0/2178097076 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f79141083d0 0x7f79141087e0 unknown :-1 s=CLOSED pgs=680 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:19.363 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:19.345+0000 7f791b463640 1 --2- 172.21.15.39:0/2178097076 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7914102ae0 0x7f7914102ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:19.364 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:19.345+0000 7f791b463640 1 -- 172.21.15.39:0/2178097076 >> 172.21.15.39:0/2178097076 conn(0x7f7914079760 msgr2=0x7f7914079b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:49:19.364 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:19.345+0000 7f791b463640 1 -- 172.21.15.39:0/2178097076 shutdown_connections 2024-09-21T03:49:19.364 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:19.345+0000 7f791b463640 1 -- 172.21.15.39:0/2178097076 wait complete. 2024-09-21T03:49:19.364 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:19.346+0000 7f791b463640 1 Processor -- start 2024-09-21T03:49:19.364 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:19.346+0000 7f791b463640 1 -- start start 2024-09-21T03:49:19.364 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:19.346+0000 7f791b463640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7914102ae0 0x7f791419b280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:49:19.364 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:19.346+0000 7f791b463640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f79141083d0 0x7f791419b7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:49:19.364 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:19.346+0000 7f791b463640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7914108d20 0x7f791419bd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:49:19.364 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:19.346+0000 7f791b463640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f7914112990 con 0x7f7914108d20 2024-09-21T03:49:19.364 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:19.346+0000 7f791b463640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f7914112810 con 0x7f7914102ae0 2024-09-21T03:49:19.364 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:19.346+0000 7f791b463640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f7914112b10 con 0x7f79141083d0 2024-09-21T03:49:19.364 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:19.346+0000 7f7919a00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7914108d20 0x7f791419bd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:49:19.364 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:19.346+0000 7f7919a00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7914108d20 0x7f791419bd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:55544/0 (socket says 172.21.15.39:55544) 2024-09-21T03:49:19.365 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:19.347+0000 7f7919a00640 1 -- 172.21.15.39:0/2897831278 learned_addr learned my addr 172.21.15.39:0/2897831278 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:49:19.365 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:19.347+0000 7f79189fe640 1 --2- 172.21.15.39:0/2897831278 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f79141083d0 0x7f791419b7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:49:19.365 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:19.347+0000 7f79191ff640 1 --2- 172.21.15.39:0/2897831278 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7914102ae0 0x7f791419b280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:49:19.365 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:19.347+0000 7f7919a00640 1 -- 172.21.15.39:0/2897831278 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f79141083d0 msgr2=0x7f791419b7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:49:19.365 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:19.347+0000 7f7919a00640 1 --2- 172.21.15.39:0/2897831278 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f79141083d0 0x7f791419b7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:19.365 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:19.347+0000 7f7919a00640 1 -- 172.21.15.39:0/2897831278 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7914102ae0 msgr2=0x7f791419b280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:49:19.365 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:19.347+0000 7f7919a00640 1 --2- 172.21.15.39:0/2897831278 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7914102ae0 0x7f791419b280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:19.365 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:19.347+0000 7f7919a00640 1 -- 172.21.15.39:0/2897831278 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f79141a6590 con 0x7f7914108d20 2024-09-21T03:49:19.365 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:19.347+0000 7f79191ff640 1 --2- 172.21.15.39:0/2897831278 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7914102ae0 0x7f791419b280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:49:19.365 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:19.347+0000 7f7919a00640 1 --2- 172.21.15.39:0/2897831278 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7914108d20 0x7f791419bd00 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7f790c00d750 tx=0x7f790c00dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:49:19.365 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:19.347+0000 7f79127fc640 1 -- 172.21.15.39:0/2897831278 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f790c017070 con 0x7f7914108d20 2024-09-21T03:49:19.365 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:19.347+0000 7f79127fc640 1 -- 172.21.15.39:0/2897831278 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f790c0054d0 con 0x7f7914108d20 2024-09-21T03:49:19.366 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:19.347+0000 7f791b463640 1 -- 172.21.15.39:0/2897831278 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f79141a6880 con 0x7f7914108d20 2024-09-21T03:49:19.366 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:19.347+0000 7f79127fc640 1 -- 172.21.15.39:0/2897831278 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f790c004c60 con 0x7f7914108d20 2024-09-21T03:49:19.366 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:19.347+0000 7f791b463640 1 -- 172.21.15.39:0/2897831278 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f79141a6d40 con 0x7f7914108d20 2024-09-21T03:49:19.366 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:19.350+0000 7f79127fc640 1 -- 172.21.15.39:0/2897831278 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 27) ==== 154643+0+0 (secure 0 0 0) 0x7f790c009bd0 con 0x7f7914108d20 2024-09-21T03:49:19.366 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:19.350+0000 7f79219d7740 1 Processor -- start 2024-09-21T03:49:19.366 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:19.350+0000 7f79219d7740 1 -- start start 2024-09-21T03:49:19.366 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:19.351+0000 7f79219d7740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559c68338070 0x559c686e3680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:49:19.366 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:19.351+0000 7f79219d7740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x559c6830f1b0 0x559c686e3bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:49:19.366 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:19.351+0000 7f79219d7740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559c68317390 0x559c686e4100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:49:19.366 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:19.351+0000 7f79219d7740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x559c68655e50 con 0x559c68317390 2024-09-21T03:49:19.367 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:19.351+0000 7f79219d7740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x559c68645280 con 0x559c6830f1b0 2024-09-21T03:49:19.367 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:19.351+0000 7f79219d7740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x559c68649090 con 0x559c68338070 2024-09-21T03:49:19.367 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:19.351+0000 7f79189fe640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559c68317390 0x559c686e4100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:49:19.367 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:19.351+0000 7f79189fe640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559c68317390 0x559c686e4100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:55546/0 (socket says 172.21.15.39:55546) 2024-09-21T03:49:19.367 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:19.351+0000 7f79189fe640 1 -- 172.21.15.39:0/1672544170 learned_addr learned my addr 172.21.15.39:0/1672544170 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:49:19.367 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:19.351+0000 7f79189fe640 1 --2- 172.21.15.39:0/1672544170 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559c68338070 0x559c686e3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:49:19.367 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:19.351+0000 7f79191ff640 1 --2- 172.21.15.39:0/1672544170 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x559c6830f1b0 0x559c686e3bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:49:19.367 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:19.351+0000 7f79127fc640 1 --2- 172.21.15.39:0/2897831278 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f78fc0b6680 0x7f78fc0b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:49:19.367 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:19.351+0000 7f79189fe640 1 -- 172.21.15.39:0/1672544170 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559c68338070 msgr2=0x559c686e3680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:49:19.367 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:19.351+0000 7f79189fe640 1 --2- 172.21.15.39:0/1672544170 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559c68338070 0x559c686e3680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:19.367 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:19.351+0000 7f79189fe640 1 -- 172.21.15.39:0/1672544170 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x559c6830f1b0 msgr2=0x559c686e3bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:49:19.368 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:19.351+0000 7f79189fe640 1 --2- 172.21.15.39:0/1672544170 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x559c6830f1b0 0x559c686e3bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:19.368 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:19.351+0000 7f79189fe640 1 -- 172.21.15.39:0/1672544170 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559c683a3650 con 0x559c68317390 2024-09-21T03:49:19.368 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:19.351+0000 7f7919a00640 1 --2- 172.21.15.39:0/2897831278 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f78fc0b6680 0x7f78fc0b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:49:19.368 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:19.351+0000 7f79127fc640 1 -- 172.21.15.39:0/2897831278 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8484+0+0 (secure 0 0 0) 0x7f790c0e70a0 con 0x7f7914108d20 2024-09-21T03:49:19.368 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:19.351+0000 7f79189fe640 1 --2- 172.21.15.39:0/1672544170 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559c68317390 0x559c686e4100 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7f7908005f70 tx=0x7f7908005780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5540ff60e77c63a server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:49:19.368 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:19.352+0000 7f78fb7fe640 1 -- 172.21.15.39:0/1672544170 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7908035070 con 0x559c68317390 2024-09-21T03:49:19.368 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:19.352+0000 7f78fb7fe640 1 -- 172.21.15.39:0/1672544170 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f79080035f0 con 0x559c68317390 2024-09-21T03:49:19.368 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:19.352+0000 7f78fb7fe640 1 -- 172.21.15.39:0/1672544170 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79080044e0 con 0x559c68317390 2024-09-21T03:49:19.368 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:19.353+0000 7f79219d7740 1 -- 172.21.15.39:0/1672544170 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559c68317390 msgr2=0x559c686e4100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:49:19.368 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:19.353+0000 7f7919a00640 1 --2- 172.21.15.39:0/2897831278 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f78fc0b6680 0x7f78fc0b8b90 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f790c0e48c0 tx=0x7f790c078ac0 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:49:19.368 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:19.353+0000 7f79219d7740 1 --2- 172.21.15.39:0/1672544170 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559c68317390 0x559c686e4100 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7f7908005f70 tx=0x7f7908005780 comp rx=0 tx=0).stop 2024-09-21T03:49:19.368 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:19.353+0000 7f79219d7740 1 -- 172.21.15.39:0/1672544170 shutdown_connections 2024-09-21T03:49:19.369 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:19.353+0000 7f79219d7740 1 --2- 172.21.15.39:0/1672544170 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559c68317390 0x559c686e4100 unknown :-1 s=CLOSED pgs=682 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:19.369 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:19.353+0000 7f79219d7740 1 --2- 172.21.15.39:0/1672544170 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x559c6830f1b0 0x559c686e3bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:19.369 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:19.353+0000 7f79219d7740 1 --2- 172.21.15.39:0/1672544170 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559c68338070 0x559c686e3680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:19.369 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:19.353+0000 7f79219d7740 1 -- 172.21.15.39:0/1672544170 >> 172.21.15.39:0/1672544170 conn(0x559c6832c460 msgr2=0x559c686e1570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:49:19.369 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:19.353+0000 7f79219d7740 1 -- 172.21.15.39:0/1672544170 shutdown_connections 2024-09-21T03:49:19.369 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:19.353+0000 7f79219d7740 1 -- 172.21.15.39:0/1672544170 wait complete. 2024-09-21T03:49:19.369 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:19.353+0000 7f79219d7740 1 -- start start 2024-09-21T03:49:19.369 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:19.353+0000 7f79219d7740 1 Processor -- start 2024-09-21T03:49:19.369 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:19.354+0000 7f79219d7740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559c6830f1b0 0x559c686eeda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:49:19.369 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:19.354+0000 7f79219d7740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559c68317390 0x559c686ef2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:49:19.369 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:19.354+0000 7f79219d7740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x559c68338070 0x559c686fb0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:49:19.369 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:19.354+0000 7f79219d7740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x559c684a6700 con 0x559c68317390 2024-09-21T03:49:19.369 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:19.354+0000 7f79219d7740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x559c685e0110 con 0x559c68338070 2024-09-21T03:49:19.369 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:19.354+0000 7f79219d7740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x559c686f1e00 con 0x559c6830f1b0 2024-09-21T03:49:19.370 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:19.354+0000 7f79219d7740 10 client.0 _ll_register_callbacks cb 0x559c685f9990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:49:19.370 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:19.354+0000 7f79191ff640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559c68317390 0x559c686ef2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:49:19.370 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:19.354+0000 7f79191ff640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559c68317390 0x559c686ef2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:55560/0 (socket says 172.21.15.39:55560) 2024-09-21T03:49:19.370 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:19.354+0000 7f79191ff640 1 -- 172.21.15.39:0/2515473996 learned_addr learned my addr 172.21.15.39:0/2515473996 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:49:19.370 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:19.354+0000 7f79189fe640 1 --2- 172.21.15.39:0/2515473996 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559c6830f1b0 0x559c686eeda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:49:19.370 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:19.354+0000 7f79189fe640 1 --2- 172.21.15.39:0/2515473996 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x559c68338070 0x559c686fb0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:49:19.370 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:19.354+0000 7f79191ff640 1 -- 172.21.15.39:0/2515473996 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559c6830f1b0 msgr2=0x559c686eeda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:49:19.370 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:19.354+0000 7f79191ff640 1 --2- 172.21.15.39:0/2515473996 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559c6830f1b0 0x559c686eeda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:19.370 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:19.354+0000 7f79191ff640 1 -- 172.21.15.39:0/2515473996 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x559c68338070 msgr2=0x559c686fb0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:49:19.370 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:19.354+0000 7f79191ff640 1 --2- 172.21.15.39:0/2515473996 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x559c68338070 0x559c686fb0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:19.370 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:19.354+0000 7f79191ff640 1 -- 172.21.15.39:0/2515473996 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559c6860dc40 con 0x559c68317390 2024-09-21T03:49:19.370 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:19.355+0000 7f79191ff640 1 --2- 172.21.15.39:0/2515473996 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559c68317390 0x559c686ef2e0 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7f79040139b0 tx=0x7f7904013e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=86a8c9f14c230855 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:49:19.371 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:19.355+0000 7f78f8ff9640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-09-21T03:49:19.371 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:19.355+0000 7f78f8ff9640 1 -- 172.21.15.39:0/2515473996 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7904020060 con 0x559c68317390 2024-09-21T03:49:19.371 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:19.355+0000 7f78f8ff9640 1 -- 172.21.15.39:0/2515473996 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8484+0+0 (secure 0 0 0) 0x7f7904011e10 con 0x559c68317390 2024-09-21T03:49:19.371 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:19.355+0000 7f79219d7740 1 -- 172.21.15.39:0/2515473996 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f79080035f0 con 0x559c68317390 2024-09-21T03:49:19.371 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:19.355+0000 7f78f8ff9640 1 -- 172.21.15.39:0/2515473996 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f79040243f0 con 0x559c68317390 2024-09-21T03:49:19.371 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:19.355+0000 7f79219d7740 10 client.16296 fetch_fsmap learned FSMap version 29 2024-09-21T03:49:19.371 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:19.355+0000 7f79219d7740 1 -- 172.21.15.39:0/2515473996 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x559c686fdc20 con 0x559c68317390 2024-09-21T03:49:19.371 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:19.355+0000 7f78f8ff9640 1 -- 172.21.15.39:0/2515473996 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f79080035f0 con 0x559c68317390 2024-09-21T03:49:19.371 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:19.355+0000 7f78f8ff9640 1 -- 172.21.15.39:0/2515473996 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f790401a030 con 0x559c68317390 2024-09-21T03:49:19.371 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:19.355+0000 7f79219d7740 10 client.16296 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T03:49:19.371 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:19.355+0000 7f79219d7740 10 client.16296 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T03:49:19.371 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:19.355+0000 7f79219d7740 20 client.16296 populate_metadata read hostname 'smithi039' 2024-09-21T03:49:19.371 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:19.355+0000 7f79219d7740 1 --2- 172.21.15.39:0/2515473996 >> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] conn(0x559c682f69f0 0x559c68701170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:49:19.371 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:19.355+0000 7f79219d7740 4 client.16296 mds_command: new command op to 24695 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:49:19.371 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:19.356+0000 7f79219d7740 1 -- 172.21.15.39:0/2515473996 --> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559c68657da0 con 0x559c682f69f0 2024-09-21T03:49:19.372 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:19.356+0000 7f79189fe640 1 --2- 172.21.15.39:0/2515473996 >> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] conn(0x559c682f69f0 0x559c68701170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:49:19.372 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:19.356+0000 7f78f8ff9640 10 client.16296 ms_handle_connect on v2:172.21.15.150:6838/2949386907 2024-09-21T03:49:19.372 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:19.356+0000 7f79189fe640 1 --2- 172.21.15.39:0/2515473996 >> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] conn(0x559c682f69f0 0x559c68701170 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4d888948c7b91e95 server_cookie=44757e5ee9a22fa2 in_seq=0 out_seq=0 2024-09-21T03:49:19.372 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:19.357+0000 7f78f8ff9640 1 -- 172.21.15.39:0/2515473996 <== mds.0 v2:172.21.15.150:6838/2949386907 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x559c68657da0 con 0x559c682f69f0 2024-09-21T03:49:19.372 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:19.357+0000 7f78f8ff9640 10 client.16296 handle_command_reply: tid=0 2024-09-21T03:49:19.372 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:19.357+0000 7f79219d7740 1 client.16296 shutdown 2024-09-21T03:49:19.372 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:19.357+0000 7f79219d7740 1 -- 172.21.15.39:0/2515473996 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559c68317390 msgr2=0x559c686ef2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:49:19.372 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:19.357+0000 7f79219d7740 1 --2- 172.21.15.39:0/2515473996 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559c68317390 0x559c686ef2e0 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7f79040139b0 tx=0x7f7904013e70 comp rx=0 tx=0).stop 2024-09-21T03:49:19.372 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:19.357+0000 7f79219d7740 1 -- 172.21.15.39:0/2515473996 shutdown_connections 2024-09-21T03:49:19.372 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:19.357+0000 7f79219d7740 1 --2- 172.21.15.39:0/2515473996 >> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] conn(0x559c682f69f0 0x559c68701170 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:19.372 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:19.357+0000 7f79219d7740 1 --2- 172.21.15.39:0/2515473996 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x559c68338070 0x559c686fb0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:19.372 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:19.357+0000 7f79219d7740 1 --2- 172.21.15.39:0/2515473996 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559c68317390 0x559c686ef2e0 unknown :-1 s=CLOSED pgs=683 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:19.372 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:19.357+0000 7f78f8ff9640 0 client.16296 ms_handle_reset on v2:172.21.15.150:6838/2949386907 2024-09-21T03:49:19.372 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:19.357+0000 7f79219d7740 1 --2- 172.21.15.39:0/2515473996 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559c6830f1b0 0x559c686eeda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:19.372 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:19.357+0000 7f79219d7740 1 -- 172.21.15.39:0/2515473996 >> 172.21.15.39:0/2515473996 conn(0x559c6832c460 msgr2=0x559c686e6810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:49:19.373 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:19.358+0000 7f79219d7740 1 -- 172.21.15.39:0/2515473996 shutdown_connections 2024-09-21T03:49:19.373 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:19.358+0000 7f79219d7740 1 -- 172.21.15.39:0/2515473996 wait complete. 2024-09-21T03:49:19.373 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:19.358+0000 7f79219d7740 20 client.16296 trim_cache size 0 max 16384 2024-09-21T03:49:19.379 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:19.376+0000 7f79219d7740 1 Processor -- start 2024-09-21T03:49:19.380 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:19.376+0000 7f79219d7740 1 -- start start 2024-09-21T03:49:19.380 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:19.376+0000 7f79219d7740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559c6830f1b0 0x559c686ef260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:49:19.380 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:19.376+0000 7f79219d7740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559c68317390 0x559c686faff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:49:19.380 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:19.376+0000 7f79219d7740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x559c68338070 0x559c686ebf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:49:19.380 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:19.376+0000 7f79219d7740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f79141ad4b0 con 0x559c68317390 2024-09-21T03:49:19.380 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:19.376+0000 7f79219d7740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f79141aeff0 con 0x559c68338070 2024-09-21T03:49:19.380 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:19.376+0000 7f79219d7740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f79141adb80 con 0x559c6830f1b0 2024-09-21T03:49:19.380 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:19.376+0000 7f79191ff640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559c68317390 0x559c686faff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:49:19.380 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:19.376+0000 7f79191ff640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559c68317390 0x559c686faff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:55564/0 (socket says 172.21.15.39:55564) 2024-09-21T03:49:19.380 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:19.376+0000 7f79191ff640 1 -- 172.21.15.39:0/1427549436 learned_addr learned my addr 172.21.15.39:0/1427549436 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:49:19.380 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:19.376+0000 7f79189fe640 1 --2- 172.21.15.39:0/1427549436 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559c6830f1b0 0x559c686ef260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:49:19.381 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:19.376+0000 7f79189fe640 1 --2- 172.21.15.39:0/1427549436 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x559c68338070 0x559c686ebf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:49:19.381 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:19.376+0000 7f79191ff640 1 -- 172.21.15.39:0/1427549436 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559c6830f1b0 msgr2=0x559c686ef260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:49:19.381 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:19.376+0000 7f79191ff640 1 --2- 172.21.15.39:0/1427549436 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559c6830f1b0 0x559c686ef260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:19.381 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:19.376+0000 7f79191ff640 1 -- 172.21.15.39:0/1427549436 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x559c68338070 msgr2=0x559c686ebf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:49:19.381 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:19.376+0000 7f79191ff640 1 --2- 172.21.15.39:0/1427549436 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x559c68338070 0x559c686ebf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:19.381 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:19.376+0000 7f79191ff640 1 -- 172.21.15.39:0/1427549436 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559c686edd00 con 0x559c68317390 2024-09-21T03:49:19.381 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:19.377+0000 7f79191ff640 1 --2- 172.21.15.39:0/1427549436 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559c68317390 0x559c686faff0 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7f7904013980 tx=0x7f790400e510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c70f1e3ec6dc163d server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:49:19.381 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:19.377+0000 7f78faffd640 1 -- 172.21.15.39:0/1427549436 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7904032070 con 0x559c68317390 2024-09-21T03:49:19.381 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:19.377+0000 7f78faffd640 1 -- 172.21.15.39:0/1427549436 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f790400ea60 con 0x559c68317390 2024-09-21T03:49:19.382 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:19.377+0000 7f78faffd640 1 -- 172.21.15.39:0/1427549436 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7904024c60 con 0x559c68317390 2024-09-21T03:49:19.385 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:19.378+0000 7f79219d7740 1 -- 172.21.15.39:0/1427549436 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559c68317390 msgr2=0x559c686faff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:49:19.385 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:19.378+0000 7f79219d7740 1 --2- 172.21.15.39:0/1427549436 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559c68317390 0x559c686faff0 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7f7904013980 tx=0x7f790400e510 comp rx=0 tx=0).stop 2024-09-21T03:49:19.385 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:19.378+0000 7f79219d7740 1 -- 172.21.15.39:0/1427549436 shutdown_connections 2024-09-21T03:49:19.385 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:19.378+0000 7f79219d7740 1 --2- 172.21.15.39:0/1427549436 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x559c68338070 0x559c686ebf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:19.385 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:19.378+0000 7f79219d7740 1 --2- 172.21.15.39:0/1427549436 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559c68317390 0x559c686faff0 unknown :-1 s=CLOSED pgs=684 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:19.386 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:19.378+0000 7f79219d7740 1 --2- 172.21.15.39:0/1427549436 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559c6830f1b0 0x559c686ef260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:19.386 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:19.378+0000 7f79219d7740 1 -- 172.21.15.39:0/1427549436 >> 172.21.15.39:0/1427549436 conn(0x559c6832c460 msgr2=0x559c686eeb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:49:19.386 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:19.378+0000 7f79219d7740 1 -- 172.21.15.39:0/1427549436 shutdown_connections 2024-09-21T03:49:19.386 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:19.378+0000 7f79219d7740 1 -- 172.21.15.39:0/1427549436 wait complete. 2024-09-21T03:49:19.386 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:19.378+0000 7f79219d7740 1 -- start start 2024-09-21T03:49:19.386 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:19.378+0000 7f79219d7740 1 Processor -- start 2024-09-21T03:49:19.386 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:19.379+0000 7f79219d7740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x559c6830f1b0 0x559c686f7a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:49:19.386 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:19.379+0000 7f79219d7740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559c68317390 0x559c686f2f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:49:19.386 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:19.379+0000 7f79219d7740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559c68338070 0x559c686f36a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:49:19.386 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:19.379+0000 7f79219d7740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f79141ae920 con 0x559c68338070 2024-09-21T03:49:19.386 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:19.379+0000 7f79219d7740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f79141ae250 con 0x559c6830f1b0 2024-09-21T03:49:19.386 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:19.379+0000 7f79219d7740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f79141ab490 con 0x559c68317390 2024-09-21T03:49:19.386 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:19.379+0000 7f79219d7740 10 client.0 _ll_register_callbacks cb 0x7f79040135c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:49:19.386 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:19.379+0000 7f79189fe640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559c68338070 0x559c686f36a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:49:19.387 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:19.379+0000 7f79189fe640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559c68338070 0x559c686f36a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:55570/0 (socket says 172.21.15.39:55570) 2024-09-21T03:49:19.387 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:19.379+0000 7f79189fe640 1 -- 172.21.15.39:0/2585652215 learned_addr learned my addr 172.21.15.39:0/2585652215 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:49:19.387 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:19.379+0000 7f79189fe640 1 --2- 172.21.15.39:0/2585652215 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x559c6830f1b0 0x559c686f7a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:49:19.387 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:19.379+0000 7f79191ff640 1 --2- 172.21.15.39:0/2585652215 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559c68317390 0x559c686f2f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:49:19.387 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:19.379+0000 7f79189fe640 1 -- 172.21.15.39:0/2585652215 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559c68317390 msgr2=0x559c686f2f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:49:19.387 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:19.379+0000 7f79189fe640 1 --2- 172.21.15.39:0/2585652215 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559c68317390 0x559c686f2f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:19.387 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:19.379+0000 7f79189fe640 1 -- 172.21.15.39:0/2585652215 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x559c6830f1b0 msgr2=0x559c686f7a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:49:19.387 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:19.379+0000 7f79189fe640 1 --2- 172.21.15.39:0/2585652215 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x559c6830f1b0 0x559c686f7a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:19.387 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:19.379+0000 7f79189fe640 1 -- 172.21.15.39:0/2585652215 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559c6876aaf0 con 0x559c68338070 2024-09-21T03:49:19.387 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:19.379+0000 7f79189fe640 1 --2- 172.21.15.39:0/2585652215 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559c68338070 0x559c686f36a0 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7f7908009900 tx=0x7f790804c040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4738b4ae1ada93f3 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:49:19.387 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:19.379+0000 7f78f8ff9640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-09-21T03:49:19.387 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:19.380+0000 7f78f8ff9640 1 -- 172.21.15.39:0/2585652215 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f790805a070 con 0x559c68338070 2024-09-21T03:49:19.387 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:19.380+0000 7f78f8ff9640 1 -- 172.21.15.39:0/2585652215 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8484+0+0 (secure 0 0 0) 0x7f7908031bc0 con 0x559c68338070 2024-09-21T03:49:19.387 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:19.380+0000 7f79219d7740 1 -- 172.21.15.39:0/2585652215 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f790400ea60 con 0x559c68338070 2024-09-21T03:49:19.388 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:19.380+0000 7f78f8ff9640 1 -- 172.21.15.39:0/2585652215 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f7908035040 con 0x559c68338070 2024-09-21T03:49:19.388 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:19.380+0000 7f79219d7740 10 client.16302 fetch_fsmap learned FSMap version 29 2024-09-21T03:49:19.388 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:19.380+0000 7f79219d7740 1 -- 172.21.15.39:0/2585652215 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x559c6876bc50 con 0x559c68338070 2024-09-21T03:49:19.388 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:19.380+0000 7f78f8ff9640 1 -- 172.21.15.39:0/2585652215 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f790400ea60 con 0x559c68338070 2024-09-21T03:49:19.388 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:19.380+0000 7f78f8ff9640 1 -- 172.21.15.39:0/2585652215 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f7908060030 con 0x559c68338070 2024-09-21T03:49:19.388 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:19.380+0000 7f79219d7740 10 client.16302 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T03:49:19.388 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:19.380+0000 7f79219d7740 10 client.16302 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T03:49:19.388 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:19.380+0000 7f79219d7740 20 client.16302 populate_metadata read hostname 'smithi039' 2024-09-21T03:49:19.388 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:19.380+0000 7f79219d7740 1 --2- 172.21.15.39:0/2585652215 >> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] conn(0x559c682cbda0 0x559c6876de90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:49:19.388 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:19.380+0000 7f79219d7740 4 client.16302 mds_command: new command op to 24695 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-21T03:49:19.388 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:19.380+0000 7f79219d7740 1 -- 172.21.15.39:0/2585652215 --> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] -- command(tid 0: {"prefix": "scrub status"}) -- 0x559c68653850 con 0x559c682cbda0 2024-09-21T03:49:19.388 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:19.381+0000 7f79191ff640 1 --2- 172.21.15.39:0/2585652215 >> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] conn(0x559c682cbda0 0x559c6876de90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:49:19.389 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:19.381+0000 7f78f8ff9640 10 client.16302 ms_handle_connect on v2:172.21.15.150:6838/2949386907 2024-09-21T03:49:19.389 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:19.381+0000 7f79191ff640 1 --2- 172.21.15.39:0/2585652215 >> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] conn(0x559c682cbda0 0x559c6876de90 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=26c0174bff734b04 server_cookie=3336d1005acc637a in_seq=0 out_seq=0 2024-09-21T03:49:19.389 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:19.381+0000 7f78f8ff9640 1 -- 172.21.15.39:0/2585652215 <== mds.0 v2:172.21.15.150:6838/2949386907 1 ==== command_reply(tid 0: 0 ) ==== 8+0+1990 (crc 0 0 0) 0x559c68653850 con 0x559c682cbda0 2024-09-21T03:49:19.389 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:19.381+0000 7f78f8ff9640 10 client.16302 handle_command_reply: tid=0 2024-09-21T03:49:19.389 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:19.381+0000 7f79219d7740 1 client.16302 shutdown 2024-09-21T03:49:19.389 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:19.382+0000 7f79219d7740 1 -- 172.21.15.39:0/2585652215 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559c68338070 msgr2=0x559c686f36a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:49:19.389 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:19.382+0000 7f79219d7740 1 --2- 172.21.15.39:0/2585652215 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559c68338070 0x559c686f36a0 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7f7908009900 tx=0x7f790804c040 comp rx=0 tx=0).stop 2024-09-21T03:49:19.389 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:19.382+0000 7f79219d7740 1 -- 172.21.15.39:0/2585652215 shutdown_connections 2024-09-21T03:49:19.389 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:19.382+0000 7f79219d7740 1 --2- 172.21.15.39:0/2585652215 >> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] conn(0x559c682cbda0 0x559c6876de90 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:19.389 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:19.382+0000 7f79219d7740 1 --2- 172.21.15.39:0/2585652215 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559c68338070 0x559c686f36a0 unknown :-1 s=CLOSED pgs=685 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:19.389 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:19.382+0000 7f79219d7740 1 --2- 172.21.15.39:0/2585652215 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559c68317390 0x559c686f2f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:19.389 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:19.382+0000 7f79219d7740 1 --2- 172.21.15.39:0/2585652215 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x559c6830f1b0 0x559c686f7a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:19.389 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:19.382+0000 7f78f8ff9640 0 client.16302 ms_handle_reset on v2:172.21.15.150:6838/2949386907 2024-09-21T03:49:19.389 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:19.382+0000 7f79219d7740 1 -- 172.21.15.39:0/2585652215 >> 172.21.15.39:0/2585652215 conn(0x559c6832c460 msgr2=0x559c686e46b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:49:19.389 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:19.382+0000 7f79219d7740 1 -- 172.21.15.39:0/2585652215 shutdown_connections 2024-09-21T03:49:19.389 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:19.382+0000 7f79219d7740 1 -- 172.21.15.39:0/2585652215 wait complete. 2024-09-21T03:49:19.390 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:19.382+0000 7f79219d7740 20 client.16302 trim_cache size 0 max 16384 2024-09-21T03:49:19.390 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-09-21T03:49:19.390 INFO:teuthology.orchestra.run.smithi039.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-21T03:49:19.390 INFO:teuthology.orchestra.run.smithi039.stdout: "scrubs": { 2024-09-21T03:49:19.390 INFO:teuthology.orchestra.run.smithi039.stdout: "9860c2f4-b7b1-47aa-865a-9a1c4434a5f6": { 2024-09-21T03:49:19.390 INFO:teuthology.orchestra.run.smithi039.stdout: "path": "/", 2024-09-21T03:49:19.390 INFO:teuthology.orchestra.run.smithi039.stdout: "tag": "9860c2f4-b7b1-47aa-865a-9a1c4434a5f6", 2024-09-21T03:49:19.390 INFO:teuthology.orchestra.run.smithi039.stdout: "options": "recursive,force" 2024-09-21T03:49:19.390 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-21T03:49:19.391 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:49:19.391 INFO:teuthology.orchestra.run.smithi039.stdout: "scrub_stats": { 2024-09-21T03:49:19.391 INFO:teuthology.orchestra.run.smithi039.stdout: "e28ad743-2b22-45b3-8799-a638c2883623": { 2024-09-21T03:49:19.391 INFO:teuthology.orchestra.run.smithi039.stdout: "start_time": "2024-09-21T03:48:31.448821+0000", 2024-09-21T03:49:19.391 INFO:teuthology.orchestra.run.smithi039.stdout: "path": "/", 2024-09-21T03:49:19.391 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_started": 0, 2024-09-21T03:49:19.391 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_passed": 0, 2024-09-21T03:49:19.391 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_failed": 0, 2024-09-21T03:49:19.391 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_skipped": 0 2024-09-21T03:49:19.391 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:49:19.391 INFO:teuthology.orchestra.run.smithi039.stdout: "29411307-eb8d-4339-8d2b-47ebd32c0080": { 2024-09-21T03:49:19.391 INFO:teuthology.orchestra.run.smithi039.stdout: "start_time": "2024-09-21T03:47:38.092920+0000", 2024-09-21T03:49:19.391 INFO:teuthology.orchestra.run.smithi039.stdout: "path": "/", 2024-09-21T03:49:19.391 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_started": 0, 2024-09-21T03:49:19.392 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_passed": 0, 2024-09-21T03:49:19.392 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_failed": 0, 2024-09-21T03:49:19.392 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_skipped": 0 2024-09-21T03:49:19.392 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:49:19.392 INFO:teuthology.orchestra.run.smithi039.stdout: "9860c2f4-b7b1-47aa-865a-9a1c4434a5f6": { 2024-09-21T03:49:19.392 INFO:teuthology.orchestra.run.smithi039.stdout: "start_time": "2024-09-21T03:49:17.718867+0000", 2024-09-21T03:49:19.393 INFO:teuthology.orchestra.run.smithi039.stdout: "path": "/", 2024-09-21T03:49:19.393 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_started": 0, 2024-09-21T03:49:19.393 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_passed": 0, 2024-09-21T03:49:19.394 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_failed": 0, 2024-09-21T03:49:19.394 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_skipped": 0 2024-09-21T03:49:19.394 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:49:19.394 INFO:teuthology.orchestra.run.smithi039.stdout: "c7e2b8e4-057d-435f-b2bd-b633de98035f": { 2024-09-21T03:49:19.394 INFO:teuthology.orchestra.run.smithi039.stdout: "start_time": "2024-09-21T03:47:19.400305+0000", 2024-09-21T03:49:19.394 INFO:teuthology.orchestra.run.smithi039.stdout: "path": "/", 2024-09-21T03:49:19.394 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_started": 0, 2024-09-21T03:49:19.394 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_passed": 0, 2024-09-21T03:49:19.394 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_failed": 0, 2024-09-21T03:49:19.394 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_skipped": 0 2024-09-21T03:49:19.394 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:49:19.394 INFO:teuthology.orchestra.run.smithi039.stdout: "10728de8-c49d-4edc-be2b-639249255e94": { 2024-09-21T03:49:19.394 INFO:teuthology.orchestra.run.smithi039.stdout: "start_time": "2024-09-21T03:47:03.965623+0000", 2024-09-21T03:49:19.395 INFO:teuthology.orchestra.run.smithi039.stdout: "path": "/", 2024-09-21T03:49:19.395 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_started": 0, 2024-09-21T03:49:19.395 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_passed": 0, 2024-09-21T03:49:19.395 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_failed": 0, 2024-09-21T03:49:19.395 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_skipped": 0 2024-09-21T03:49:19.395 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:49:19.395 INFO:teuthology.orchestra.run.smithi039.stdout: "02509e4f-3911-4983-b16f-df8ded0b9998": { 2024-09-21T03:49:19.395 INFO:teuthology.orchestra.run.smithi039.stdout: "start_time": "2024-09-21T03:46:17.889576+0000", 2024-09-21T03:49:19.395 INFO:teuthology.orchestra.run.smithi039.stdout: "path": "/", 2024-09-21T03:49:19.395 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_started": 0, 2024-09-21T03:49:19.395 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_passed": 0, 2024-09-21T03:49:19.395 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_failed": 0, 2024-09-21T03:49:19.395 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_skipped": 0 2024-09-21T03:49:19.395 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-21T03:49:19.395 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-21T03:49:19.396 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-09-21T03:49:19.396 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:19.382+0000 7f791b463640 1 -- 172.21.15.39:0/2897831278 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f78fc0b6680 msgr2=0x7f78fc0b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:49:19.396 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:19.382+0000 7f791b463640 1 --2- 172.21.15.39:0/2897831278 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f78fc0b6680 0x7f78fc0b8b90 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f790c0e48c0 tx=0x7f790c078ac0 comp rx=0 tx=0).stop 2024-09-21T03:49:19.396 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:19.382+0000 7f791b463640 1 -- 172.21.15.39:0/2897831278 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7914108d20 msgr2=0x7f791419bd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:49:19.396 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:19.382+0000 7f791b463640 1 --2- 172.21.15.39:0/2897831278 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7914108d20 0x7f791419bd00 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7f790c00d750 tx=0x7f790c00dc10 comp rx=0 tx=0).stop 2024-09-21T03:49:19.396 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:19.383+0000 7f791b463640 1 -- 172.21.15.39:0/2897831278 shutdown_connections 2024-09-21T03:49:19.397 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:19.383+0000 7f791b463640 1 --2- 172.21.15.39:0/2897831278 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f78fc0b6680 0x7f78fc0b8b90 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:19.397 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:19.383+0000 7f791b463640 1 --2- 172.21.15.39:0/2897831278 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7914108d20 0x7f791419bd00 unknown :-1 s=CLOSED pgs=681 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:19.397 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:19.383+0000 7f791b463640 1 --2- 172.21.15.39:0/2897831278 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f79141083d0 0x7f791419b7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:19.397 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:19.383+0000 7f791b463640 1 --2- 172.21.15.39:0/2897831278 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7914102ae0 0x7f791419b280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:19.397 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:19.383+0000 7f791b463640 1 -- 172.21.15.39:0/2897831278 >> 172.21.15.39:0/2897831278 conn(0x7f7914079760 msgr2=0x7f79141044a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:49:19.397 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:19.383+0000 7f791b463640 1 -- 172.21.15.39:0/2897831278 shutdown_connections 2024-09-21T03:49:19.397 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:19.383+0000 7f791b463640 1 -- 172.21.15.39:0/2897831278 wait complete. 2024-09-21T03:49:20.191 INFO:tasks.cephfs.filesystem:scrub status for tag:9860c2f4-b7b1-47aa-865a-9a1c4434a5f6 - {'path': '/', 'tag': '9860c2f4-b7b1-47aa-865a-9a1c4434a5f6', 'options': 'recursive,force'} 2024-09-21T03:49:21.084 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:49:20 smithi039 ceph-mon[31211]: pgmap v424: 97 pgs: 97 active+clean; 10 MiB data, 536 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 170 B/s wr, 11 op/s 2024-09-21T03:49:21.084 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:49:20 smithi039 ceph-mon[31211]: scrub summary: idle 2024-09-21T03:49:21.218 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:49:20 smithi161 ceph-mon[34533]: pgmap v424: 97 pgs: 97 active+clean; 10 MiB data, 536 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 170 B/s wr, 11 op/s 2024-09-21T03:49:21.218 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:49:20 smithi161 ceph-mon[34533]: scrub summary: idle 2024-09-21T03:49:21.224 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:49:20 smithi150 ceph-mon[34289]: pgmap v424: 97 pgs: 97 active+clean; 10 MiB data, 536 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 170 B/s wr, 11 op/s 2024-09-21T03:49:21.225 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:49:20 smithi150 ceph-mon[34289]: scrub summary: idle 2024-09-21T03:49:23.084 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:49:22 smithi039 ceph-mon[31211]: pgmap v425: 97 pgs: 97 active+clean; 10 MiB data, 536 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 85 B/s wr, 9 op/s 2024-09-21T03:49:23.218 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:49:22 smithi161 ceph-mon[34533]: pgmap v425: 97 pgs: 97 active+clean; 10 MiB data, 536 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 85 B/s wr, 9 op/s 2024-09-21T03:49:23.224 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:49:22 smithi150 ceph-mon[34289]: pgmap v425: 97 pgs: 97 active+clean; 10 MiB data, 536 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 85 B/s wr, 9 op/s 2024-09-21T03:49:25.084 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:49:24 smithi039 ceph-mon[31211]: pgmap v426: 97 pgs: 97 active+clean; 10 MiB data, 536 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 85 B/s wr, 9 op/s 2024-09-21T03:49:25.218 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:49:24 smithi161 ceph-mon[34533]: pgmap v426: 97 pgs: 97 active+clean; 10 MiB data, 536 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 85 B/s wr, 9 op/s 2024-09-21T03:49:25.224 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:49:24 smithi150 ceph-mon[34289]: pgmap v426: 97 pgs: 97 active+clean; 10 MiB data, 536 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 85 B/s wr, 9 op/s 2024-09-21T03:49:27.084 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:49:26 smithi039 ceph-mon[31211]: pgmap v427: 97 pgs: 97 active+clean; 10 MiB data, 536 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 11 op/s 2024-09-21T03:49:27.218 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:49:26 smithi161 ceph-mon[34533]: pgmap v427: 97 pgs: 97 active+clean; 10 MiB data, 536 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 11 op/s 2024-09-21T03:49:27.224 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:49:26 smithi150 ceph-mon[34289]: pgmap v427: 97 pgs: 97 active+clean; 10 MiB data, 536 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 11 op/s 2024-09-21T03:49:28.084 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:49:27 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:49:28.217 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:49:27 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:49:28.224 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:49:27 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:49:29.084 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:49:28 smithi039 ceph-mon[31211]: pgmap v428: 97 pgs: 97 active+clean; 10 MiB data, 536 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 8 op/s 2024-09-21T03:49:29.084 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:49:28 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:49:29.084 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:49:28 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:49:29.084 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:49:28 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:49:29.218 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:49:28 smithi161 ceph-mon[34533]: pgmap v428: 97 pgs: 97 active+clean; 10 MiB data, 536 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 8 op/s 2024-09-21T03:49:29.218 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:49:28 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:49:29.218 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:49:28 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:49:29.218 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:49:28 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:49:29.225 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:49:28 smithi150 ceph-mon[34289]: pgmap v428: 97 pgs: 97 active+clean; 10 MiB data, 536 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 8 op/s 2024-09-21T03:49:29.225 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:49:28 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:49:29.225 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:49:28 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:49:29.225 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:49:28 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:49:31.084 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:49:30 smithi039 ceph-mon[31211]: pgmap v429: 97 pgs: 97 active+clean; 10 MiB data, 536 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 11 op/s 2024-09-21T03:49:31.217 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:49:30 smithi161 ceph-mon[34533]: pgmap v429: 97 pgs: 97 active+clean; 10 MiB data, 536 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 11 op/s 2024-09-21T03:49:31.225 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:49:30 smithi150 ceph-mon[34289]: pgmap v429: 97 pgs: 97 active+clean; 10 MiB data, 536 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 11 op/s 2024-09-21T03:49:33.084 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:49:32 smithi039 ceph-mon[31211]: pgmap v430: 97 pgs: 97 active+clean; 10 MiB data, 536 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 9 op/s 2024-09-21T03:49:33.217 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:49:32 smithi161 ceph-mon[34533]: pgmap v430: 97 pgs: 97 active+clean; 10 MiB data, 536 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 9 op/s 2024-09-21T03:49:33.225 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:49:32 smithi150 ceph-mon[34289]: pgmap v430: 97 pgs: 97 active+clean; 10 MiB data, 536 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 9 op/s 2024-09-21T03:49:35.084 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:49:34 smithi039 ceph-mon[31211]: pgmap v431: 97 pgs: 97 active+clean; 10 MiB data, 536 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-21T03:49:35.218 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:49:34 smithi161 ceph-mon[34533]: pgmap v431: 97 pgs: 97 active+clean; 10 MiB data, 536 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-21T03:49:35.225 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:49:34 smithi150 ceph-mon[34289]: pgmap v431: 97 pgs: 97 active+clean; 10 MiB data, 536 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-21T03:49:37.084 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:49:36 smithi039 ceph-mon[31211]: pgmap v432: 97 pgs: 97 active+clean; 10 MiB data, 536 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-09-21T03:49:37.217 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:49:36 smithi161 ceph-mon[34533]: pgmap v432: 97 pgs: 97 active+clean; 10 MiB data, 536 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-09-21T03:49:37.225 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:49:36 smithi150 ceph-mon[34289]: pgmap v432: 97 pgs: 97 active+clean; 10 MiB data, 536 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-09-21T03:49:39.084 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:49:38 smithi039 ceph-mon[31211]: pgmap v433: 97 pgs: 97 active+clean; 10 MiB data, 536 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-09-21T03:49:39.218 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:49:38 smithi161 ceph-mon[34533]: pgmap v433: 97 pgs: 97 active+clean; 10 MiB data, 536 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-09-21T03:49:39.224 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:49:38 smithi150 ceph-mon[34289]: pgmap v433: 97 pgs: 97 active+clean; 10 MiB data, 536 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-09-21T03:49:41.084 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:49:40 smithi039 ceph-mon[31211]: pgmap v434: 97 pgs: 97 active+clean; 10 MiB data, 536 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-09-21T03:49:41.217 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:49:40 smithi161 ceph-mon[34533]: pgmap v434: 97 pgs: 97 active+clean; 10 MiB data, 536 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-09-21T03:49:41.224 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:49:40 smithi150 ceph-mon[34289]: pgmap v434: 97 pgs: 97 active+clean; 10 MiB data, 536 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-09-21T03:49:43.083 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:49:42 smithi039 ceph-mon[31211]: pgmap v435: 97 pgs: 97 active+clean; 10 MiB data, 536 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-21T03:49:43.218 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:49:42 smithi161 ceph-mon[34533]: pgmap v435: 97 pgs: 97 active+clean; 10 MiB data, 536 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-21T03:49:43.224 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:49:42 smithi150 ceph-mon[34289]: pgmap v435: 97 pgs: 97 active+clean; 10 MiB data, 536 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-21T03:49:45.083 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:49:44 smithi039 ceph-mon[31211]: pgmap v436: 97 pgs: 97 active+clean; 10 MiB data, 536 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-09-21T03:49:45.217 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:49:44 smithi161 ceph-mon[34533]: pgmap v436: 97 pgs: 97 active+clean; 10 MiB data, 536 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-09-21T03:49:45.225 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:49:44 smithi150 ceph-mon[34289]: pgmap v436: 97 pgs: 97 active+clean; 10 MiB data, 536 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-09-21T03:49:47.084 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:49:46 smithi039 ceph-mon[31211]: pgmap v437: 97 pgs: 97 active+clean; 10 MiB data, 536 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-09-21T03:49:47.218 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:49:46 smithi161 ceph-mon[34533]: pgmap v437: 97 pgs: 97 active+clean; 10 MiB data, 536 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-09-21T03:49:47.225 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:49:46 smithi150 ceph-mon[34289]: pgmap v437: 97 pgs: 97 active+clean; 10 MiB data, 536 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-09-21T03:49:49.084 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:49:48 smithi039 ceph-mon[31211]: pgmap v438: 97 pgs: 97 active+clean; 10 MiB data, 536 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-09-21T03:49:49.217 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:49:48 smithi161 ceph-mon[34533]: pgmap v438: 97 pgs: 97 active+clean; 10 MiB data, 536 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-09-21T03:49:49.224 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:49:48 smithi150 ceph-mon[34289]: pgmap v438: 97 pgs: 97 active+clean; 10 MiB data, 536 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-09-21T03:49:50.192 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 51116d5c-77ca-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub status 2024-09-21T03:49:50.469 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/51116d5c-77ca-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:49:50.843 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:49:50 smithi039 ceph-mon[31211]: pgmap v439: 97 pgs: 97 active+clean; 10 MiB data, 536 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-09-21T03:49:51.061 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:51.057+0000 7f6a76f81640 1 -- 172.21.15.39:0/957399525 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6a70102ae0 msgr2=0x7f6a70102ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:49:51.061 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:51.057+0000 7f6a76f81640 1 --2- 172.21.15.39:0/957399525 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6a70102ae0 0x7f6a70102ef0 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7f6a60009900 tx=0x7f6a6002a410 comp rx=0 tx=0).stop 2024-09-21T03:49:51.062 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:51.057+0000 7f6a76f81640 1 -- 172.21.15.39:0/957399525 shutdown_connections 2024-09-21T03:49:51.062 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:51.057+0000 7f6a76f81640 1 --2- 172.21.15.39:0/957399525 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6a70108d20 0x7f6a7010f550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:51.062 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:51.057+0000 7f6a76f81640 1 --2- 172.21.15.39:0/957399525 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6a701083d0 0x7f6a701087e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:51.062 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:51.057+0000 7f6a76f81640 1 --2- 172.21.15.39:0/957399525 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6a70102ae0 0x7f6a70102ef0 unknown :-1 s=CLOSED pgs=686 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:51.062 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:51.057+0000 7f6a76f81640 1 -- 172.21.15.39:0/957399525 >> 172.21.15.39:0/957399525 conn(0x7f6a70079760 msgr2=0x7f6a70079b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:49:51.062 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:51.057+0000 7f6a76f81640 1 -- 172.21.15.39:0/957399525 shutdown_connections 2024-09-21T03:49:51.062 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:51.057+0000 7f6a76f81640 1 -- 172.21.15.39:0/957399525 wait complete. 2024-09-21T03:49:51.062 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:51.058+0000 7f6a76f81640 1 Processor -- start 2024-09-21T03:49:51.062 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:51.058+0000 7f6a76f81640 1 -- start start 2024-09-21T03:49:51.062 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:51.058+0000 7f6a76f81640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6a70102ae0 0x7f6a70104280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:49:51.062 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:51.058+0000 7f6a76f81640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6a701083d0 0x7f6a701047c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:49:51.062 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:51.058+0000 7f6a76f81640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6a70108d20 0x7f6a70104d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:49:51.062 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:51.058+0000 7f6a76f81640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f6a70112990 con 0x7f6a70108d20 2024-09-21T03:49:51.062 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:51.058+0000 7f6a76f81640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f6a70112810 con 0x7f6a701083d0 2024-09-21T03:49:51.063 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:51.058+0000 7f6a76f81640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f6a70112b10 con 0x7f6a70102ae0 2024-09-21T03:49:51.063 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:51.058+0000 7f6a7551e640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6a70108d20 0x7f6a70104d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:49:51.063 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:51.059+0000 7f6a7551e640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6a70108d20 0x7f6a70104d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:37610/0 (socket says 172.21.15.39:37610) 2024-09-21T03:49:51.063 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:51.059+0000 7f6a7551e640 1 -- 172.21.15.39:0/1672825690 learned_addr learned my addr 172.21.15.39:0/1672825690 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:49:51.063 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:51.059+0000 7f6a74d1d640 1 --2- 172.21.15.39:0/1672825690 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6a70102ae0 0x7f6a70104280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:49:51.063 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:51.059+0000 7f6a6ffff640 1 --2- 172.21.15.39:0/1672825690 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6a701083d0 0x7f6a701047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:49:51.063 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:51.059+0000 7f6a7551e640 1 -- 172.21.15.39:0/1672825690 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6a70102ae0 msgr2=0x7f6a70104280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:49:51.063 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:51.059+0000 7f6a7551e640 1 --2- 172.21.15.39:0/1672825690 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6a70102ae0 0x7f6a70104280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:51.063 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:51.059+0000 7f6a7551e640 1 -- 172.21.15.39:0/1672825690 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6a701083d0 msgr2=0x7f6a701047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:49:51.063 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:51.059+0000 7f6a7551e640 1 --2- 172.21.15.39:0/1672825690 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6a701083d0 0x7f6a701047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:51.063 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:51.059+0000 7f6a7551e640 1 -- 172.21.15.39:0/1672825690 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6a701a8770 con 0x7f6a70108d20 2024-09-21T03:49:51.063 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:51.059+0000 7f6a74d1d640 1 --2- 172.21.15.39:0/1672825690 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6a70102ae0 0x7f6a70104280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:49:51.063 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:51.059+0000 7f6a7551e640 1 --2- 172.21.15.39:0/1672825690 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6a70108d20 0x7f6a70104d00 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7f6a6800eb10 tx=0x7f6a6800c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:49:51.063 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:51.059+0000 7f6a6dffb640 1 -- 172.21.15.39:0/1672825690 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a68004b20 con 0x7f6a70108d20 2024-09-21T03:49:51.063 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:51.059+0000 7f6a76f81640 1 -- 172.21.15.39:0/1672825690 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6a701a8a60 con 0x7f6a70108d20 2024-09-21T03:49:51.064 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:51.059+0000 7f6a6dffb640 1 -- 172.21.15.39:0/1672825690 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6a68004cc0 con 0x7f6a70108d20 2024-09-21T03:49:51.064 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:51.059+0000 7f6a6dffb640 1 -- 172.21.15.39:0/1672825690 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a68010690 con 0x7f6a70108d20 2024-09-21T03:49:51.064 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:51.060+0000 7f6a76f81640 1 -- 172.21.15.39:0/1672825690 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6a701a8ff0 con 0x7f6a70108d20 2024-09-21T03:49:51.066 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:51.062+0000 7f6a7d4f5740 1 Processor -- start 2024-09-21T03:49:51.066 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:51.062+0000 7f6a6dffb640 1 -- 172.21.15.39:0/1672825690 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 27) ==== 154643+0+0 (secure 0 0 0) 0x7f6a68005020 con 0x7f6a70108d20 2024-09-21T03:49:51.066 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:51.063+0000 7f6a7d4f5740 1 -- start start 2024-09-21T03:49:51.066 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:51.063+0000 7f6a7d4f5740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x555fb50ec070 0x555fb5493270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:49:51.066 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:51.063+0000 7f6a7d4f5740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555fb50c31b0 0x555fb54937b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:49:51.066 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:51.063+0000 7f6a7d4f5740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555fb50cb390 0x555fb5493cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:49:51.066 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:51.063+0000 7f6a7d4f5740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x555fb5409e50 con 0x555fb50ec070 2024-09-21T03:49:51.066 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:51.063+0000 7f6a7d4f5740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x555fb53f9280 con 0x555fb50cb390 2024-09-21T03:49:51.066 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:51.063+0000 7f6a7d4f5740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x555fb53fd090 con 0x555fb50c31b0 2024-09-21T03:49:51.066 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:51.063+0000 7f6a6ffff640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x555fb50ec070 0x555fb5493270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:49:51.066 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:51.063+0000 7f6a6ffff640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x555fb50ec070 0x555fb5493270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:37612/0 (socket says 172.21.15.39:37612) 2024-09-21T03:49:51.066 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:51.063+0000 7f6a6ffff640 1 -- 172.21.15.39:0/2334067742 learned_addr learned my addr 172.21.15.39:0/2334067742 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:49:51.067 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:51.063+0000 7f6a74d1d640 1 --2- 172.21.15.39:0/2334067742 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555fb50c31b0 0x555fb54937b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:49:51.067 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:51.063+0000 7f6a6ffff640 1 --2- 172.21.15.39:0/2334067742 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555fb50cb390 0x555fb5493cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:49:51.067 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:51.063+0000 7f6a6ffff640 1 -- 172.21.15.39:0/2334067742 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555fb50c31b0 msgr2=0x555fb54937b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:49:51.067 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:51.063+0000 7f6a6ffff640 1 --2- 172.21.15.39:0/2334067742 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555fb50c31b0 0x555fb54937b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:51.067 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:51.063+0000 7f6a6ffff640 1 -- 172.21.15.39:0/2334067742 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555fb50cb390 msgr2=0x555fb5493cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:49:51.067 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:51.063+0000 7f6a6ffff640 1 --2- 172.21.15.39:0/2334067742 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555fb50cb390 0x555fb5493cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:51.067 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:51.063+0000 7f6a6ffff640 1 -- 172.21.15.39:0/2334067742 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555fb5157650 con 0x555fb50ec070 2024-09-21T03:49:51.067 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:51.063+0000 7f6a74d1d640 1 --2- 172.21.15.39:0/2334067742 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555fb50c31b0 0x555fb54937b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:49:51.067 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:51.063+0000 7f6a6dffb640 1 --2- 172.21.15.39:0/1672825690 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f6a580b6680 0x7f6a580b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:49:51.067 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:51.064+0000 7f6a74d1d640 1 --2- 172.21.15.39:0/1672825690 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f6a580b6680 0x7f6a580b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:49:51.067 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:51.064+0000 7f6a6ffff640 1 --2- 172.21.15.39:0/2334067742 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x555fb50ec070 0x555fb5493270 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7f6a6400bfd0 tx=0x7f6a6400b650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=30fbfc379ae80416 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:49:51.067 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:51.064+0000 7f6a6dffb640 1 -- 172.21.15.39:0/1672825690 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8484+0+0 (secure 0 0 0) 0x7f6a680e7d70 con 0x7f6a70108d20 2024-09-21T03:49:51.067 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:51.064+0000 7f6a56ffd640 1 -- 172.21.15.39:0/2334067742 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a64019030 con 0x555fb50ec070 2024-09-21T03:49:51.067 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:51.064+0000 7f6a56ffd640 1 -- 172.21.15.39:0/2334067742 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6a64011970 con 0x555fb50ec070 2024-09-21T03:49:51.067 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:51.064+0000 7f6a56ffd640 1 -- 172.21.15.39:0/2334067742 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a6401d6b0 con 0x555fb50ec070 2024-09-21T03:49:51.070 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:51.065+0000 7f6a7d4f5740 1 -- 172.21.15.39:0/2334067742 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x555fb50ec070 msgr2=0x555fb5493270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:49:51.070 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:51.065+0000 7f6a74d1d640 1 --2- 172.21.15.39:0/1672825690 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f6a580b6680 0x7f6a580b8b90 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f6a60000c00 tx=0x7f6a600058b0 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:49:51.070 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:51.065+0000 7f6a7d4f5740 1 --2- 172.21.15.39:0/2334067742 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x555fb50ec070 0x555fb5493270 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7f6a6400bfd0 tx=0x7f6a6400b650 comp rx=0 tx=0).stop 2024-09-21T03:49:51.070 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:51.065+0000 7f6a7d4f5740 1 -- 172.21.15.39:0/2334067742 shutdown_connections 2024-09-21T03:49:51.071 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:51.065+0000 7f6a7d4f5740 1 --2- 172.21.15.39:0/2334067742 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555fb50cb390 0x555fb5493cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:51.071 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:51.065+0000 7f6a7d4f5740 1 --2- 172.21.15.39:0/2334067742 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555fb50c31b0 0x555fb54937b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:51.071 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:51.065+0000 7f6a7d4f5740 1 --2- 172.21.15.39:0/2334067742 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x555fb50ec070 0x555fb5493270 unknown :-1 s=CLOSED pgs=688 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:51.071 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:51.065+0000 7f6a7d4f5740 1 -- 172.21.15.39:0/2334067742 >> 172.21.15.39:0/2334067742 conn(0x555fb50e0460 msgr2=0x555fb5491160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:49:51.071 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:51.065+0000 7f6a7d4f5740 1 -- 172.21.15.39:0/2334067742 shutdown_connections 2024-09-21T03:49:51.071 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:51.065+0000 7f6a7d4f5740 1 -- 172.21.15.39:0/2334067742 wait complete. 2024-09-21T03:49:51.071 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:51.065+0000 7f6a7d4f5740 1 -- start start 2024-09-21T03:49:51.071 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:51.066+0000 7f6a7d4f5740 1 Processor -- start 2024-09-21T03:49:51.071 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:51.066+0000 7f6a7d4f5740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555fb50c31b0 0x555fb549e8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:49:51.071 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:51.066+0000 7f6a7d4f5740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x555fb50cb390 0x555fb549edf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:49:51.071 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:51.066+0000 7f6a7d4f5740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555fb50ec070 0x555fb549f370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:49:51.071 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:51.066+0000 7f6a7d4f5740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x555fb525a700 con 0x555fb50cb390 2024-09-21T03:49:51.071 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:51.066+0000 7f6a7d4f5740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x555fb5394110 con 0x555fb50c31b0 2024-09-21T03:49:51.071 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:51.066+0000 7f6a7d4f5740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x555fb54a1a50 con 0x555fb50ec070 2024-09-21T03:49:51.072 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:51.066+0000 7f6a7d4f5740 10 client.0 _ll_register_callbacks cb 0x555fb53ad990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:49:51.072 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:51.066+0000 7f6a6ffff640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x555fb50cb390 0x555fb549edf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:49:51.072 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:51.066+0000 7f6a6ffff640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x555fb50cb390 0x555fb549edf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:37614/0 (socket says 172.21.15.39:37614) 2024-09-21T03:49:51.072 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:51.066+0000 7f6a6ffff640 1 -- 172.21.15.39:0/3965739102 learned_addr learned my addr 172.21.15.39:0/3965739102 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:49:51.072 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:51.066+0000 7f6a74d1d640 1 --2- 172.21.15.39:0/3965739102 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555fb50c31b0 0x555fb549e8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:49:51.072 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:51.066+0000 7f6a7551e640 1 --2- 172.21.15.39:0/3965739102 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555fb50ec070 0x555fb549f370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:49:51.072 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:51.067+0000 7f6a6ffff640 1 -- 172.21.15.39:0/3965739102 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555fb50ec070 msgr2=0x555fb549f370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:49:51.072 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:51.067+0000 7f6a6ffff640 1 --2- 172.21.15.39:0/3965739102 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555fb50ec070 0x555fb549f370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:51.072 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:51.067+0000 7f6a6ffff640 1 -- 172.21.15.39:0/3965739102 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555fb50c31b0 msgr2=0x555fb549e8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:49:51.072 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:51.067+0000 7f6a6ffff640 1 --2- 172.21.15.39:0/3965739102 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555fb50c31b0 0x555fb549e8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:51.072 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:51.067+0000 7f6a6ffff640 1 -- 172.21.15.39:0/3965739102 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555fb53c1c40 con 0x555fb50cb390 2024-09-21T03:49:51.072 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:51.067+0000 7f6a6ffff640 1 --2- 172.21.15.39:0/3965739102 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x555fb50cb390 0x555fb549edf0 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7f6a6400bfa0 tx=0x7f6a64011770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bbb1e7d44b118ea5 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:49:51.072 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:51.067+0000 7f6a4ffff640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-09-21T03:49:51.072 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:51.067+0000 7f6a4ffff640 1 -- 172.21.15.39:0/3965739102 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a64019040 con 0x555fb50cb390 2024-09-21T03:49:51.072 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:51.067+0000 7f6a4ffff640 1 -- 172.21.15.39:0/3965739102 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8484+0+0 (secure 0 0 0) 0x7f6a64011d20 con 0x555fb50cb390 2024-09-21T03:49:51.073 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:51.067+0000 7f6a7d4f5740 1 -- 172.21.15.39:0/3965739102 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6a64011970 con 0x555fb50cb390 2024-09-21T03:49:51.073 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:51.067+0000 7f6a4ffff640 1 -- 172.21.15.39:0/3965739102 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f6a64005040 con 0x555fb50cb390 2024-09-21T03:49:51.073 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:51.067+0000 7f6a7d4f5740 10 client.16314 fetch_fsmap learned FSMap version 29 2024-09-21T03:49:51.073 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:51.067+0000 7f6a7d4f5740 1 -- 172.21.15.39:0/3965739102 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x555fb54ad6f0 con 0x555fb50cb390 2024-09-21T03:49:51.073 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:51.068+0000 7f6a4ffff640 1 -- 172.21.15.39:0/3965739102 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6a64011970 con 0x555fb50cb390 2024-09-21T03:49:51.073 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:51.068+0000 7f6a4ffff640 1 -- 172.21.15.39:0/3965739102 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f6a64025020 con 0x555fb50cb390 2024-09-21T03:49:51.073 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:51.068+0000 7f6a7d4f5740 10 client.16314 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T03:49:51.073 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:51.068+0000 7f6a7d4f5740 10 client.16314 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T03:49:51.073 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:51.068+0000 7f6a7d4f5740 20 client.16314 populate_metadata read hostname 'smithi039' 2024-09-21T03:49:51.073 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:51.068+0000 7f6a7d4f5740 1 --2- 172.21.15.39:0/3965739102 >> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] conn(0x555fb50aa9f0 0x555fb54b0b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:49:51.073 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:51.068+0000 7f6a7d4f5740 4 client.16314 mds_command: new command op to 24695 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:49:51.073 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:51.068+0000 7f6a7d4f5740 1 -- 172.21.15.39:0/3965739102 --> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555fb540bda0 con 0x555fb50aa9f0 2024-09-21T03:49:51.073 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:51.068+0000 7f6a74d1d640 1 --2- 172.21.15.39:0/3965739102 >> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] conn(0x555fb50aa9f0 0x555fb54b0b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:49:51.073 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:51.069+0000 7f6a74d1d640 1 --2- 172.21.15.39:0/3965739102 >> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] conn(0x555fb50aa9f0 0x555fb54b0b00 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ace8e60d290de54f server_cookie=f3d0b8acc8d9b53d in_seq=0 out_seq=0 2024-09-21T03:49:51.073 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:51.069+0000 7f6a4ffff640 10 client.16314 ms_handle_connect on v2:172.21.15.150:6838/2949386907 2024-09-21T03:49:51.074 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:51.069+0000 7f6a4ffff640 1 -- 172.21.15.39:0/3965739102 <== mds.0 v2:172.21.15.150:6838/2949386907 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x555fb540bda0 con 0x555fb50aa9f0 2024-09-21T03:49:51.075 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:51.069+0000 7f6a4ffff640 10 client.16314 handle_command_reply: tid=0 2024-09-21T03:49:51.075 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:51.069+0000 7f6a7d4f5740 1 client.16314 shutdown 2024-09-21T03:49:51.075 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:51.069+0000 7f6a7d4f5740 1 -- 172.21.15.39:0/3965739102 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x555fb50cb390 msgr2=0x555fb549edf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:49:51.075 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:51.069+0000 7f6a7d4f5740 1 --2- 172.21.15.39:0/3965739102 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x555fb50cb390 0x555fb549edf0 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7f6a6400bfa0 tx=0x7f6a64011770 comp rx=0 tx=0).stop 2024-09-21T03:49:51.075 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:51.070+0000 7f6a7d4f5740 1 -- 172.21.15.39:0/3965739102 shutdown_connections 2024-09-21T03:49:51.075 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:51.070+0000 7f6a7d4f5740 1 --2- 172.21.15.39:0/3965739102 >> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] conn(0x555fb50aa9f0 0x555fb54b0b00 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:51.075 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:51.070+0000 7f6a7d4f5740 1 --2- 172.21.15.39:0/3965739102 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555fb50ec070 0x555fb549f370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:51.075 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:51.070+0000 7f6a7d4f5740 1 --2- 172.21.15.39:0/3965739102 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x555fb50cb390 0x555fb549edf0 unknown :-1 s=CLOSED pgs=689 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:51.075 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:51.070+0000 7f6a7d4f5740 1 --2- 172.21.15.39:0/3965739102 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555fb50c31b0 0x555fb549e8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:51.075 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:51.070+0000 7f6a4ffff640 0 client.16314 ms_handle_reset on v2:172.21.15.150:6838/2949386907 2024-09-21T03:49:51.075 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:51.070+0000 7f6a7d4f5740 1 -- 172.21.15.39:0/3965739102 >> 172.21.15.39:0/3965739102 conn(0x555fb50e0460 msgr2=0x555fb5496450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:49:51.075 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:51.070+0000 7f6a7d4f5740 1 -- 172.21.15.39:0/3965739102 shutdown_connections 2024-09-21T03:49:51.075 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:51.070+0000 7f6a7d4f5740 1 -- 172.21.15.39:0/3965739102 wait complete. 2024-09-21T03:49:51.075 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:51.070+0000 7f6a7d4f5740 20 client.16314 trim_cache size 0 max 16384 2024-09-21T03:49:51.090 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:51.087+0000 7f6a7d4f5740 1 Processor -- start 2024-09-21T03:49:51.090 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:51.087+0000 7f6a7d4f5740 1 -- start start 2024-09-21T03:49:51.090 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:51.088+0000 7f6a7d4f5740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x555fb50c31b0 0x555fb54a08a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:49:51.090 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:51.088+0000 7f6a7d4f5740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555fb50cb390 0x555fb5490710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:49:51.090 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:51.088+0000 7f6a7d4f5740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555fb50ec070 0x555fb5490c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:49:51.090 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:51.088+0000 7f6a7d4f5740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f6a701b1920 con 0x555fb50c31b0 2024-09-21T03:49:51.091 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:51.088+0000 7f6a7d4f5740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f6a701b3460 con 0x555fb50ec070 2024-09-21T03:49:51.091 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:51.088+0000 7f6a7d4f5740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f6a701b1ff0 con 0x555fb50cb390 2024-09-21T03:49:51.091 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:51.088+0000 7f6a74d1d640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x555fb50c31b0 0x555fb54a08a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:49:51.091 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:51.088+0000 7f6a74d1d640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x555fb50c31b0 0x555fb54a08a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:37628/0 (socket says 172.21.15.39:37628) 2024-09-21T03:49:51.091 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:51.088+0000 7f6a74d1d640 1 -- 172.21.15.39:0/2301361713 learned_addr learned my addr 172.21.15.39:0/2301361713 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:49:51.091 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:51.088+0000 7f6a6ffff640 1 --2- 172.21.15.39:0/2301361713 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555fb50cb390 0x555fb5490710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:49:51.091 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:51.088+0000 7f6a7551e640 1 --2- 172.21.15.39:0/2301361713 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555fb50ec070 0x555fb5490c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:49:51.091 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:51.088+0000 7f6a74d1d640 1 -- 172.21.15.39:0/2301361713 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555fb50cb390 msgr2=0x555fb5490710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:49:51.091 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:51.088+0000 7f6a74d1d640 1 --2- 172.21.15.39:0/2301361713 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555fb50cb390 0x555fb5490710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:51.091 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:51.088+0000 7f6a74d1d640 1 -- 172.21.15.39:0/2301361713 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555fb50ec070 msgr2=0x555fb5490c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:49:51.091 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:51.088+0000 7f6a74d1d640 1 --2- 172.21.15.39:0/2301361713 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555fb50ec070 0x555fb5490c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:51.091 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:51.088+0000 7f6a74d1d640 1 -- 172.21.15.39:0/2301361713 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555fb548b410 con 0x555fb50c31b0 2024-09-21T03:49:51.091 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:51.088+0000 7f6a74d1d640 1 --2- 172.21.15.39:0/2301361713 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x555fb50c31b0 0x555fb54a08a0 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7f6a6002a930 tx=0x7f6a6003d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=37c1be9bc28ba996 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:49:51.091 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:51.088+0000 7f6a567fc640 1 -- 172.21.15.39:0/2301361713 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a60055070 con 0x555fb50c31b0 2024-09-21T03:49:51.092 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:51.089+0000 7f6a567fc640 1 -- 172.21.15.39:0/2301361713 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6a60004300 con 0x555fb50c31b0 2024-09-21T03:49:51.092 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:51.089+0000 7f6a567fc640 1 -- 172.21.15.39:0/2301361713 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a60035050 con 0x555fb50c31b0 2024-09-21T03:49:51.095 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:51.089+0000 7f6a7d4f5740 1 -- 172.21.15.39:0/2301361713 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x555fb50c31b0 msgr2=0x555fb54a08a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:49:51.095 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:51.089+0000 7f6a7d4f5740 1 --2- 172.21.15.39:0/2301361713 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x555fb50c31b0 0x555fb54a08a0 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7f6a6002a930 tx=0x7f6a6003d000 comp rx=0 tx=0).stop 2024-09-21T03:49:51.095 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:51.089+0000 7f6a7d4f5740 1 -- 172.21.15.39:0/2301361713 shutdown_connections 2024-09-21T03:49:51.095 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:51.089+0000 7f6a7d4f5740 1 --2- 172.21.15.39:0/2301361713 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555fb50ec070 0x555fb5490c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:51.096 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:51.089+0000 7f6a7d4f5740 1 --2- 172.21.15.39:0/2301361713 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555fb50cb390 0x555fb5490710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:51.096 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:51.089+0000 7f6a7d4f5740 1 --2- 172.21.15.39:0/2301361713 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x555fb50c31b0 0x555fb54a08a0 unknown :-1 s=CLOSED pgs=690 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:51.096 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:51.089+0000 7f6a7d4f5740 1 -- 172.21.15.39:0/2301361713 >> 172.21.15.39:0/2301361713 conn(0x555fb50e0460 msgr2=0x555fb54aa5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:49:51.096 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:51.089+0000 7f6a7d4f5740 1 -- 172.21.15.39:0/2301361713 shutdown_connections 2024-09-21T03:49:51.096 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:51.089+0000 7f6a7d4f5740 1 -- 172.21.15.39:0/2301361713 wait complete. 2024-09-21T03:49:51.096 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:51.090+0000 7f6a7d4f5740 1 -- start start 2024-09-21T03:49:51.096 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:51.090+0000 7f6a7d4f5740 1 Processor -- start 2024-09-21T03:49:51.096 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:51.090+0000 7f6a7d4f5740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555fb50c31b0 0x555fb54a2e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:49:51.096 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:51.090+0000 7f6a7d4f5740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555fb50cb390 0x555fb54a33d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:49:51.096 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:51.090+0000 7f6a7d4f5740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x555fb50ec070 0x555fb54a3930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:49:51.096 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:51.090+0000 7f6a7d4f5740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f6a701b2d90 con 0x555fb50ec070 2024-09-21T03:49:51.096 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:51.090+0000 7f6a7d4f5740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f6a701b26c0 con 0x555fb50c31b0 2024-09-21T03:49:51.096 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:51.090+0000 7f6a7d4f5740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f6a701af900 con 0x555fb50cb390 2024-09-21T03:49:51.096 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:51.091+0000 7f6a7d4f5740 10 client.0 _ll_register_callbacks cb 0x7f6a64013400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:49:51.097 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:51.091+0000 7f6a7551e640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x555fb50ec070 0x555fb54a3930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:49:51.097 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:51.091+0000 7f6a7551e640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x555fb50ec070 0x555fb54a3930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:37644/0 (socket says 172.21.15.39:37644) 2024-09-21T03:49:51.097 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:51.091+0000 7f6a7551e640 1 -- 172.21.15.39:0/3470886131 learned_addr learned my addr 172.21.15.39:0/3470886131 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:49:51.097 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:51.091+0000 7f6a6ffff640 1 --2- 172.21.15.39:0/3470886131 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555fb50cb390 0x555fb54a33d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:49:51.097 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:51.091+0000 7f6a74d1d640 1 --2- 172.21.15.39:0/3470886131 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555fb50c31b0 0x555fb54a2e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:49:51.097 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:51.091+0000 7f6a7551e640 1 -- 172.21.15.39:0/3470886131 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555fb50cb390 msgr2=0x555fb54a33d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:49:51.097 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:51.091+0000 7f6a7551e640 1 --2- 172.21.15.39:0/3470886131 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555fb50cb390 0x555fb54a33d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:51.097 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:51.091+0000 7f6a7551e640 1 -- 172.21.15.39:0/3470886131 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555fb50c31b0 msgr2=0x555fb54a2e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:49:51.097 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:51.091+0000 7f6a7551e640 1 --2- 172.21.15.39:0/3470886131 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555fb50c31b0 0x555fb54a2e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:51.097 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:51.091+0000 7f6a7551e640 1 -- 172.21.15.39:0/3470886131 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555fb551a5e0 con 0x555fb50ec070 2024-09-21T03:49:51.097 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:51.091+0000 7f6a7551e640 1 --2- 172.21.15.39:0/3470886131 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x555fb50ec070 0x555fb54a3930 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7f6a680aa1b0 tx=0x7f6a680a9b10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=21cda49e00af15a5 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:49:51.097 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:51.091+0000 7f6a4ffff640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-09-21T03:49:51.097 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:51.091+0000 7f6a4ffff640 1 -- 172.21.15.39:0/3470886131 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a680d5030 con 0x555fb50ec070 2024-09-21T03:49:51.097 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:51.091+0000 7f6a4ffff640 1 -- 172.21.15.39:0/3470886131 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8484+0+0 (secure 0 0 0) 0x7f6a680b7170 con 0x555fb50ec070 2024-09-21T03:49:51.097 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:51.091+0000 7f6a7d4f5740 1 -- 172.21.15.39:0/3470886131 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6a60004300 con 0x555fb50ec070 2024-09-21T03:49:51.098 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:51.092+0000 7f6a4ffff640 1 -- 172.21.15.39:0/3470886131 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f6a680ad350 con 0x555fb50ec070 2024-09-21T03:49:51.098 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:51.092+0000 7f6a7d4f5740 10 client.16320 fetch_fsmap learned FSMap version 29 2024-09-21T03:49:51.098 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:51.092+0000 7f6a7d4f5740 1 -- 172.21.15.39:0/3470886131 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x555fb551b8d0 con 0x555fb50ec070 2024-09-21T03:49:51.098 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:51.092+0000 7f6a4ffff640 1 -- 172.21.15.39:0/3470886131 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6a60004300 con 0x555fb50ec070 2024-09-21T03:49:51.098 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:51.092+0000 7f6a4ffff640 1 -- 172.21.15.39:0/3470886131 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f6a680b2150 con 0x555fb50ec070 2024-09-21T03:49:51.098 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:51.092+0000 7f6a7d4f5740 10 client.16320 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T03:49:51.098 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:51.092+0000 7f6a7d4f5740 10 client.16320 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T03:49:51.098 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:51.092+0000 7f6a7d4f5740 20 client.16320 populate_metadata read hostname 'smithi039' 2024-09-21T03:49:51.098 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:51.092+0000 7f6a7d4f5740 1 --2- 172.21.15.39:0/3470886131 >> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] conn(0x555fb507fda0 0x555fb551e0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:49:51.098 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:51.092+0000 7f6a7d4f5740 4 client.16320 mds_command: new command op to 24695 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-21T03:49:51.098 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:51.092+0000 7f6a7d4f5740 1 -- 172.21.15.39:0/3470886131 --> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] -- command(tid 0: {"prefix": "scrub status"}) -- 0x555fb5407850 con 0x555fb507fda0 2024-09-21T03:49:51.098 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:51.092+0000 7f6a74d1d640 1 --2- 172.21.15.39:0/3470886131 >> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] conn(0x555fb507fda0 0x555fb551e0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:49:51.098 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:51.093+0000 7f6a74d1d640 1 --2- 172.21.15.39:0/3470886131 >> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] conn(0x555fb507fda0 0x555fb551e0e0 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bb91fab6eb80eb7d server_cookie=1483b881d57e3246 in_seq=0 out_seq=0 2024-09-21T03:49:51.098 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:51.093+0000 7f6a4ffff640 10 client.16320 ms_handle_connect on v2:172.21.15.150:6838/2949386907 2024-09-21T03:49:51.098 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:51.093+0000 7f6a4ffff640 1 -- 172.21.15.39:0/3470886131 <== mds.0 v2:172.21.15.150:6838/2949386907 1 ==== command_reply(tid 0: 0 ) ==== 8+0+1794 (crc 0 0 0) 0x555fb5407850 con 0x555fb507fda0 2024-09-21T03:49:51.099 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:51.093+0000 7f6a4ffff640 10 client.16320 handle_command_reply: tid=0 2024-09-21T03:49:51.099 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:51.093+0000 7f6a7d4f5740 1 client.16320 shutdown 2024-09-21T03:49:51.099 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:51.093+0000 7f6a7d4f5740 1 -- 172.21.15.39:0/3470886131 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x555fb50ec070 msgr2=0x555fb54a3930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:49:51.099 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:51.093+0000 7f6a7d4f5740 1 --2- 172.21.15.39:0/3470886131 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x555fb50ec070 0x555fb54a3930 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7f6a680aa1b0 tx=0x7f6a680a9b10 comp rx=0 tx=0).stop 2024-09-21T03:49:51.099 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:51.093+0000 7f6a7d4f5740 1 -- 172.21.15.39:0/3470886131 shutdown_connections 2024-09-21T03:49:51.099 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:51.093+0000 7f6a7d4f5740 1 --2- 172.21.15.39:0/3470886131 >> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] conn(0x555fb507fda0 0x555fb551e0e0 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:51.099 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:51.093+0000 7f6a7d4f5740 1 --2- 172.21.15.39:0/3470886131 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x555fb50ec070 0x555fb54a3930 unknown :-1 s=CLOSED pgs=691 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:51.099 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:51.093+0000 7f6a7d4f5740 1 --2- 172.21.15.39:0/3470886131 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555fb50cb390 0x555fb54a33d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:51.099 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:51.093+0000 7f6a7d4f5740 1 --2- 172.21.15.39:0/3470886131 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555fb50c31b0 0x555fb54a2e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:51.099 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:51.093+0000 7f6a7d4f5740 1 -- 172.21.15.39:0/3470886131 >> 172.21.15.39:0/3470886131 conn(0x555fb50e0460 msgr2=0x555fb54aa3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:49:51.099 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:51.093+0000 7f6a4ffff640 0 client.16320 ms_handle_reset on v2:172.21.15.150:6838/2949386907 2024-09-21T03:49:51.099 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:51.094+0000 7f6a7d4f5740 1 -- 172.21.15.39:0/3470886131 shutdown_connections 2024-09-21T03:49:51.099 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:51.094+0000 7f6a7d4f5740 1 -- 172.21.15.39:0/3470886131 wait complete. 2024-09-21T03:49:51.099 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:51.094+0000 7f6a7d4f5740 20 client.16320 trim_cache size 0 max 16384 2024-09-21T03:49:51.100 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-09-21T03:49:51.100 INFO:teuthology.orchestra.run.smithi039.stdout: "status": "no active scrubs running", 2024-09-21T03:49:51.100 INFO:teuthology.orchestra.run.smithi039.stdout: "scrubs": {}, 2024-09-21T03:49:51.100 INFO:teuthology.orchestra.run.smithi039.stdout: "scrub_stats": { 2024-09-21T03:49:51.100 INFO:teuthology.orchestra.run.smithi039.stdout: "e28ad743-2b22-45b3-8799-a638c2883623": { 2024-09-21T03:49:51.100 INFO:teuthology.orchestra.run.smithi039.stdout: "start_time": "2024-09-21T03:48:31.448821+0000", 2024-09-21T03:49:51.100 INFO:teuthology.orchestra.run.smithi039.stdout: "path": "/", 2024-09-21T03:49:51.100 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_started": 0, 2024-09-21T03:49:51.100 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_passed": 0, 2024-09-21T03:49:51.101 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_failed": 0, 2024-09-21T03:49:51.101 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_skipped": 0 2024-09-21T03:49:51.101 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:49:51.101 INFO:teuthology.orchestra.run.smithi039.stdout: "29411307-eb8d-4339-8d2b-47ebd32c0080": { 2024-09-21T03:49:51.101 INFO:teuthology.orchestra.run.smithi039.stdout: "start_time": "2024-09-21T03:47:38.092920+0000", 2024-09-21T03:49:51.101 INFO:teuthology.orchestra.run.smithi039.stdout: "path": "/", 2024-09-21T03:49:51.101 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_started": 0, 2024-09-21T03:49:51.101 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_passed": 0, 2024-09-21T03:49:51.101 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_failed": 0, 2024-09-21T03:49:51.101 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_skipped": 0 2024-09-21T03:49:51.101 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:49:51.101 INFO:teuthology.orchestra.run.smithi039.stdout: "9860c2f4-b7b1-47aa-865a-9a1c4434a5f6": { 2024-09-21T03:49:51.101 INFO:teuthology.orchestra.run.smithi039.stdout: "start_time": "2024-09-21T03:49:17.718867+0000", 2024-09-21T03:49:51.101 INFO:teuthology.orchestra.run.smithi039.stdout: "path": "/", 2024-09-21T03:49:51.101 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_started": 0, 2024-09-21T03:49:51.102 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_passed": 0, 2024-09-21T03:49:51.102 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_failed": 0, 2024-09-21T03:49:51.102 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_skipped": 0 2024-09-21T03:49:51.102 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:49:51.102 INFO:teuthology.orchestra.run.smithi039.stdout: "c7e2b8e4-057d-435f-b2bd-b633de98035f": { 2024-09-21T03:49:51.102 INFO:teuthology.orchestra.run.smithi039.stdout: "start_time": "2024-09-21T03:47:19.400305+0000", 2024-09-21T03:49:51.102 INFO:teuthology.orchestra.run.smithi039.stdout: "path": "/", 2024-09-21T03:49:51.102 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_started": 0, 2024-09-21T03:49:51.102 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_passed": 0, 2024-09-21T03:49:51.102 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_failed": 0, 2024-09-21T03:49:51.102 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_skipped": 0 2024-09-21T03:49:51.102 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:49:51.102 INFO:teuthology.orchestra.run.smithi039.stdout: "10728de8-c49d-4edc-be2b-639249255e94": { 2024-09-21T03:49:51.102 INFO:teuthology.orchestra.run.smithi039.stdout: "start_time": "2024-09-21T03:47:03.965623+0000", 2024-09-21T03:49:51.103 INFO:teuthology.orchestra.run.smithi039.stdout: "path": "/", 2024-09-21T03:49:51.103 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_started": 0, 2024-09-21T03:49:51.103 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_passed": 0, 2024-09-21T03:49:51.103 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_failed": 0, 2024-09-21T03:49:51.103 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_skipped": 0 2024-09-21T03:49:51.103 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:49:51.103 INFO:teuthology.orchestra.run.smithi039.stdout: "02509e4f-3911-4983-b16f-df8ded0b9998": { 2024-09-21T03:49:51.103 INFO:teuthology.orchestra.run.smithi039.stdout: "start_time": "2024-09-21T03:46:17.889576+0000", 2024-09-21T03:49:51.103 INFO:teuthology.orchestra.run.smithi039.stdout: "path": "/", 2024-09-21T03:49:51.103 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_started": 0, 2024-09-21T03:49:51.103 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_passed": 0, 2024-09-21T03:49:51.103 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_failed": 0, 2024-09-21T03:49:51.103 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_skipped": 0 2024-09-21T03:49:51.103 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-21T03:49:51.103 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-21T03:49:51.103 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-09-21T03:49:51.104 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:51.094+0000 7f6a76f81640 1 -- 172.21.15.39:0/1672825690 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f6a580b6680 msgr2=0x7f6a580b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:49:51.104 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:51.094+0000 7f6a76f81640 1 --2- 172.21.15.39:0/1672825690 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f6a580b6680 0x7f6a580b8b90 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f6a60000c00 tx=0x7f6a600058b0 comp rx=0 tx=0).stop 2024-09-21T03:49:51.104 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:51.094+0000 7f6a76f81640 1 -- 172.21.15.39:0/1672825690 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6a70108d20 msgr2=0x7f6a70104d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:49:51.104 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:51.094+0000 7f6a76f81640 1 --2- 172.21.15.39:0/1672825690 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6a70108d20 0x7f6a70104d00 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7f6a6800eb10 tx=0x7f6a6800c2d0 comp rx=0 tx=0).stop 2024-09-21T03:49:51.104 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:51.094+0000 7f6a76f81640 1 -- 172.21.15.39:0/1672825690 shutdown_connections 2024-09-21T03:49:51.104 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:51.094+0000 7f6a76f81640 1 --2- 172.21.15.39:0/1672825690 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f6a580b6680 0x7f6a580b8b90 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:51.104 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:51.094+0000 7f6a76f81640 1 --2- 172.21.15.39:0/1672825690 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6a70108d20 0x7f6a70104d00 unknown :-1 s=CLOSED pgs=687 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:51.104 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:51.094+0000 7f6a76f81640 1 --2- 172.21.15.39:0/1672825690 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6a701083d0 0x7f6a701047c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:51.104 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:51.094+0000 7f6a76f81640 1 --2- 172.21.15.39:0/1672825690 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6a70102ae0 0x7f6a70104280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:51.104 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:51.094+0000 7f6a76f81640 1 -- 172.21.15.39:0/1672825690 >> 172.21.15.39:0/1672825690 conn(0x7f6a70079760 msgr2=0x7f6a701019c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:49:51.105 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:51.095+0000 7f6a76f81640 1 -- 172.21.15.39:0/1672825690 shutdown_connections 2024-09-21T03:49:51.105 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:51.095+0000 7f6a76f81640 1 -- 172.21.15.39:0/1672825690 wait complete. 2024-09-21T03:49:51.218 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:49:50 smithi161 ceph-mon[34533]: pgmap v439: 97 pgs: 97 active+clean; 10 MiB data, 536 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-09-21T03:49:51.224 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:49:50 smithi150 ceph-mon[34289]: pgmap v439: 97 pgs: 97 active+clean; 10 MiB data, 536 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-09-21T03:49:52.154 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-21T03:49:52.155 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 51116d5c-77ca-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T03:49:52.419 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/51116d5c-77ca-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:49:52.947 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:52.929+0000 7fdfe0f98640 1 -- 172.21.15.39:0/120920780 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdfdc104a30 msgr2=0x7fdfdc104e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:49:52.947 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:52.929+0000 7fdfe0f98640 1 --2- 172.21.15.39:0/120920780 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdfdc104a30 0x7fdfdc104e40 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7fdfcc009900 tx=0x7fdfcc02a410 comp rx=0 tx=0).stop 2024-09-21T03:49:52.947 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:52.929+0000 7fdfe0f98640 1 -- 172.21.15.39:0/120920780 shutdown_connections 2024-09-21T03:49:52.947 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:52.929+0000 7fdfe0f98640 1 --2- 172.21.15.39:0/120920780 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdfdc10de60 0x7fdfdc110330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:52.947 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:52.929+0000 7fdfe0f98640 1 --2- 172.21.15.39:0/120920780 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdfdc105380 0x7fdfdc10d8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:52.947 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:52.929+0000 7fdfe0f98640 1 --2- 172.21.15.39:0/120920780 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdfdc104a30 0x7fdfdc104e40 unknown :-1 s=CLOSED pgs=692 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:52.947 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:52.929+0000 7fdfe0f98640 1 -- 172.21.15.39:0/120920780 >> 172.21.15.39:0/120920780 conn(0x7fdfdc079760 msgr2=0x7fdfdc079b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:49:52.947 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:52.929+0000 7fdfe0f98640 1 -- 172.21.15.39:0/120920780 shutdown_connections 2024-09-21T03:49:52.947 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:52.929+0000 7fdfe0f98640 1 -- 172.21.15.39:0/120920780 wait complete. 2024-09-21T03:49:52.947 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:52.930+0000 7fdfe0f98640 1 Processor -- start 2024-09-21T03:49:52.947 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:52.930+0000 7fdfe0f98640 1 -- start start 2024-09-21T03:49:52.948 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:52.930+0000 7fdfe0f98640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdfdc104a30 0x7fdfdc19a4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:49:52.948 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:52.930+0000 7fdfe0f98640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdfdc105380 0x7fdfdc19aa90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:49:52.948 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:52.930+0000 7fdfe0f98640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdfdc10de60 0x7fdfdc19fd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:49:52.948 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:52.930+0000 7fdfe0f98640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fdfdc113aa0 con 0x7fdfdc10de60 2024-09-21T03:49:52.948 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:52.930+0000 7fdfe0f98640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fdfdc113920 con 0x7fdfdc105380 2024-09-21T03:49:52.948 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:52.930+0000 7fdfe0f98640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fdfdc113c20 con 0x7fdfdc104a30 2024-09-21T03:49:52.948 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:52.930+0000 7fdfdad9d640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdfdc10de60 0x7fdfdc19fd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:49:52.948 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:52.930+0000 7fdfdad9d640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdfdc10de60 0x7fdfdc19fd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:37656/0 (socket says 172.21.15.39:37656) 2024-09-21T03:49:52.948 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:52.930+0000 7fdfdad9d640 1 -- 172.21.15.39:0/282451727 learned_addr learned my addr 172.21.15.39:0/282451727 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:49:52.948 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:52.931+0000 7fdfda59c640 1 --2- 172.21.15.39:0/282451727 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdfdc104a30 0x7fdfdc19a4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:49:52.948 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:52.931+0000 7fdfd9d9b640 1 --2- 172.21.15.39:0/282451727 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdfdc105380 0x7fdfdc19aa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:49:52.948 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:52.931+0000 7fdfdad9d640 1 -- 172.21.15.39:0/282451727 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdfdc104a30 msgr2=0x7fdfdc19a4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:49:52.948 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:52.931+0000 7fdfdad9d640 1 --2- 172.21.15.39:0/282451727 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdfdc104a30 0x7fdfdc19a4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:52.948 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:52.931+0000 7fdfdad9d640 1 -- 172.21.15.39:0/282451727 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdfdc105380 msgr2=0x7fdfdc19aa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:49:52.948 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:52.931+0000 7fdfdad9d640 1 --2- 172.21.15.39:0/282451727 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdfdc105380 0x7fdfdc19aa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:52.949 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:52.931+0000 7fdfdad9d640 1 -- 172.21.15.39:0/282451727 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdfdc1a0460 con 0x7fdfdc10de60 2024-09-21T03:49:52.949 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:52.931+0000 7fdfd9d9b640 1 --2- 172.21.15.39:0/282451727 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdfdc105380 0x7fdfdc19aa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:49:52.949 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:52.931+0000 7fdfdad9d640 1 --2- 172.21.15.39:0/282451727 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdfdc10de60 0x7fdfdc19fd30 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7fdfd400eaa0 tx=0x7fdfd400ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:49:52.949 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:52.931+0000 7fdfcb7fe640 1 -- 172.21.15.39:0/282451727 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfd400cdb0 con 0x7fdfdc10de60 2024-09-21T03:49:52.949 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:52.931+0000 7fdfe0f98640 1 -- 172.21.15.39:0/282451727 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdfdc1a0750 con 0x7fdfdc10de60 2024-09-21T03:49:52.949 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:52.931+0000 7fdfcb7fe640 1 -- 172.21.15.39:0/282451727 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdfd4005020 con 0x7fdfdc10de60 2024-09-21T03:49:52.949 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:52.931+0000 7fdfcb7fe640 1 -- 172.21.15.39:0/282451727 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfd40116d0 con 0x7fdfdc10de60 2024-09-21T03:49:52.949 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:52.931+0000 7fdfe0f98640 1 -- 172.21.15.39:0/282451727 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdfdc1a6eb0 con 0x7fdfdc10de60 2024-09-21T03:49:52.949 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:52.933+0000 7fdfe0f98640 1 -- 172.21.15.39:0/282451727 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdfdc06c1f0 con 0x7fdfdc10de60 2024-09-21T03:49:52.949 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:52.934+0000 7fdfcb7fe640 1 -- 172.21.15.39:0/282451727 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 27) ==== 154643+0+0 (secure 0 0 0) 0x7fdfd40051d0 con 0x7fdfdc10de60 2024-09-21T03:49:52.949 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:52.935+0000 7fdfcb7fe640 1 --2- 172.21.15.39:0/282451727 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7fdfc40b6680 0x7fdfc40b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:49:52.949 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:52.935+0000 7fdfda59c640 1 --2- 172.21.15.39:0/282451727 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7fdfc40b6680 0x7fdfc40b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:49:52.949 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:52.935+0000 7fdfcb7fe640 1 -- 172.21.15.39:0/282451727 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8484+0+0 (secure 0 0 0) 0x7fdfd40e79b0 con 0x7fdfdc10de60 2024-09-21T03:49:52.949 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:52.935+0000 7fdfda59c640 1 --2- 172.21.15.39:0/282451727 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7fdfc40b6680 0x7fdfc40b8b90 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fdfcc000c00 tx=0x7fdfcc031400 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:49:52.949 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:52.939+0000 7fdfcb7fe640 1 -- 172.21.15.39:0/282451727 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fdfd4016030 con 0x7fdfdc10de60 2024-09-21T03:49:53.083 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:49:52 smithi039 ceph-mon[31211]: pgmap v440: 97 pgs: 97 active+clean; 10 MiB data, 536 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-21T03:49:53.167 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:53.165+0000 7fdfe0f98640 1 -- 172.21.15.39:0/282451727 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fdfdc02e640 con 0x7fdfdc10de60 2024-09-21T03:49:53.168 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:53.166+0000 7fdfcb7fe640 1 -- 172.21.15.39:0/282451727 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11978 (secure 0 0 0) 0x7fdfd40adf20 con 0x7fdfdc10de60 2024-09-21T03:49:53.168 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T03:49:53.168 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":29,"btime":"2024-09-21T03:46:08:982881+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14790,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6835/508995295","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":508995295},{"type":"v1","addr":"172.21.15.39:6835","nonce":508995295}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14802,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6837/3915912607","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":3915912607},{"type":"v1","addr":"172.21.15.39:6837","nonce":3915912607}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":29,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T03:44:15.009872+0000","modified":"2024-09-21T03:46:08.430113+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24695,"mds_1":24667,"mds_2":24659,"mds_3":24646,"mds_4":14868},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14856":{"gid":14856,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.39:6839/185509192","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6838","nonce":185509192},{"type":"v1","addr":"172.21.15.39:6839","nonce":185509192}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14868":{"gid":14868,"name":"a","rank":4,"incarnation":25,"state":"up:active","state_seq":7,"addr":"172.21.15.39:6841/4247190456","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6840","nonce":4247190456},{"type":"v1","addr":"172.21.15.39:6841","nonce":4247190456}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24599":{"gid":24599,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.150:6833/3114643217","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":3114643217},{"type":"v1","addr":"172.21.15.150:6833","nonce":3114643217}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24601":{"gid":24601,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.161:6833/2963990654","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":2963990654},{"type":"v1","addr":"172.21.15.161:6833","nonce":2963990654}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24632":{"gid":24632,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.150:6835/724594008","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6834","nonce":724594008},{"type":"v1","addr":"172.21.15.150:6835","nonce":724594008}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24640":{"gid":24640,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.161:6835/3747882110","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":3747882110},{"type":"v1","addr":"172.21.15.161:6835","nonce":3747882110}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24646":{"gid":24646,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":10,"addr":"172.21.15.161:6837/3451544597","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6836","nonce":3451544597},{"type":"v1","addr":"172.21.15.161:6837","nonce":3451544597}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24659":{"gid":24659,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.150:6837/4017043144","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6836","nonce":4017043144},{"type":"v1","addr":"172.21.15.150:6837","nonce":4017043144}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24667":{"gid":24667,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.161:6839/2190107270","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6838","nonce":2190107270},{"type":"v1","addr":"172.21.15.161:6839","nonce":2190107270}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24695":{"gid":24695,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.150:6839/2949386907","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6838","nonce":2949386907},{"type":"v1","addr":"172.21.15.150:6839","nonce":2949386907}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24695,"qdb_cluster":[24695,24667,24659,24646,14868]},"id":1}]} 2024-09-21T03:49:53.168 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 29 2024-09-21T03:49:53.170 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:53.168+0000 7fdfe0f98640 1 -- 172.21.15.39:0/282451727 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7fdfc40b6680 msgr2=0x7fdfc40b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:49:53.171 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:53.169+0000 7fdfe0f98640 1 --2- 172.21.15.39:0/282451727 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7fdfc40b6680 0x7fdfc40b8b90 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fdfcc000c00 tx=0x7fdfcc031400 comp rx=0 tx=0).stop 2024-09-21T03:49:53.171 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:53.169+0000 7fdfe0f98640 1 -- 172.21.15.39:0/282451727 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdfdc10de60 msgr2=0x7fdfdc19fd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:49:53.171 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:53.169+0000 7fdfe0f98640 1 --2- 172.21.15.39:0/282451727 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdfdc10de60 0x7fdfdc19fd30 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7fdfd400eaa0 tx=0x7fdfd400ef60 comp rx=0 tx=0).stop 2024-09-21T03:49:53.171 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:53.169+0000 7fdfe0f98640 1 -- 172.21.15.39:0/282451727 shutdown_connections 2024-09-21T03:49:53.171 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:53.169+0000 7fdfe0f98640 1 --2- 172.21.15.39:0/282451727 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7fdfc40b6680 0x7fdfc40b8b90 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:53.171 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:53.169+0000 7fdfe0f98640 1 --2- 172.21.15.39:0/282451727 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdfdc10de60 0x7fdfdc19fd30 unknown :-1 s=CLOSED pgs=693 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:53.171 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:53.169+0000 7fdfe0f98640 1 --2- 172.21.15.39:0/282451727 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdfdc105380 0x7fdfdc19aa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:53.171 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:53.169+0000 7fdfe0f98640 1 --2- 172.21.15.39:0/282451727 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdfdc104a30 0x7fdfdc19a4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:53.171 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:53.169+0000 7fdfe0f98640 1 -- 172.21.15.39:0/282451727 >> 172.21.15.39:0/282451727 conn(0x7fdfdc079760 msgr2=0x7fdfdc100890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:49:53.171 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:53.169+0000 7fdfe0f98640 1 -- 172.21.15.39:0/282451727 shutdown_connections 2024-09-21T03:49:53.171 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:53.169+0000 7fdfe0f98640 1 -- 172.21.15.39:0/282451727 wait complete. 2024-09-21T03:49:53.218 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:49:52 smithi161 ceph-mon[34533]: pgmap v440: 97 pgs: 97 active+clean; 10 MiB data, 536 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-21T03:49:53.224 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:49:52 smithi150 ceph-mon[34289]: pgmap v440: 97 pgs: 97 active+clean; 10 MiB data, 536 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-21T03:49:53.863 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 51116d5c-77ca-11ef-baf5-efdc52797490 -- ceph tell mds.1:4 damage ls 2024-09-21T03:49:54.084 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:49:53 smithi039 ceph-mon[31211]: from='client.? 172.21.15.39:0/282451727' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:49:54.120 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/51116d5c-77ca-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:49:54.217 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:49:53 smithi161 ceph-mon[34533]: from='client.? 172.21.15.39:0/282451727' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:49:54.224 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:49:53 smithi150 ceph-mon[34289]: from='client.? 172.21.15.39:0/282451727' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:49:54.599 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:54.597+0000 7f2a213c6640 1 -- 172.21.15.39:0/701749226 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2a1c104860 msgr2=0x7f2a1c104c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:49:54.600 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:54.597+0000 7f2a213c6640 1 --2- 172.21.15.39:0/701749226 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2a1c104860 0x7f2a1c104c70 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7f2a0c009900 tx=0x7f2a0c02a410 comp rx=0 tx=0).stop 2024-09-21T03:49:54.600 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:54.597+0000 7f2a213c6640 1 -- 172.21.15.39:0/701749226 shutdown_connections 2024-09-21T03:49:54.600 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:54.597+0000 7f2a213c6640 1 --2- 172.21.15.39:0/701749226 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2a1c10de10 0x7f2a1c1102f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:54.600 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:54.597+0000 7f2a213c6640 1 --2- 172.21.15.39:0/701749226 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2a1c105290 0x7f2a1c10d8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:54.600 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:54.597+0000 7f2a213c6640 1 --2- 172.21.15.39:0/701749226 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2a1c104860 0x7f2a1c104c70 unknown :-1 s=CLOSED pgs=694 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:54.600 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:54.597+0000 7f2a213c6640 1 -- 172.21.15.39:0/701749226 >> 172.21.15.39:0/701749226 conn(0x7f2a1c079730 msgr2=0x7f2a1c079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:49:54.600 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:54.597+0000 7f2a213c6640 1 -- 172.21.15.39:0/701749226 shutdown_connections 2024-09-21T03:49:54.600 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:54.597+0000 7f2a213c6640 1 -- 172.21.15.39:0/701749226 wait complete. 2024-09-21T03:49:54.600 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:54.598+0000 7f2a213c6640 1 Processor -- start 2024-09-21T03:49:54.600 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:54.598+0000 7f2a213c6640 1 -- start start 2024-09-21T03:49:54.601 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:54.598+0000 7f2a213c6640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2a1c104860 0x7f2a1c19a470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:49:54.602 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:54.598+0000 7f2a213c6640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2a1c105290 0x7f2a1c19aa10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:49:54.602 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:54.598+0000 7f2a213c6640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2a1c10de10 0x7f2a1c19fcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:49:54.602 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:54.598+0000 7f2a213c6640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f2a1c113a80 con 0x7f2a1c10de10 2024-09-21T03:49:54.602 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:54.598+0000 7f2a213c6640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f2a1c113900 con 0x7f2a1c105290 2024-09-21T03:49:54.602 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:54.598+0000 7f2a213c6640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f2a1c113c00 con 0x7f2a1c104860 2024-09-21T03:49:54.602 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:54.598+0000 7f2a1b7fe640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2a1c10de10 0x7f2a1c19fcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:49:54.602 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:54.598+0000 7f2a1b7fe640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2a1c10de10 0x7f2a1c19fcb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:37682/0 (socket says 172.21.15.39:37682) 2024-09-21T03:49:54.602 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:54.598+0000 7f2a1b7fe640 1 -- 172.21.15.39:0/2649605228 learned_addr learned my addr 172.21.15.39:0/2649605228 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:49:54.602 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:54.598+0000 7f2a1affd640 1 --2- 172.21.15.39:0/2649605228 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2a1c104860 0x7f2a1c19a470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:49:54.602 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:54.598+0000 7f2a1a7fc640 1 --2- 172.21.15.39:0/2649605228 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2a1c105290 0x7f2a1c19aa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:49:54.602 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:54.599+0000 7f2a1b7fe640 1 -- 172.21.15.39:0/2649605228 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2a1c104860 msgr2=0x7f2a1c19a470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:49:54.602 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:54.599+0000 7f2a1b7fe640 1 --2- 172.21.15.39:0/2649605228 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2a1c104860 0x7f2a1c19a470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:54.602 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:54.599+0000 7f2a1b7fe640 1 -- 172.21.15.39:0/2649605228 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2a1c105290 msgr2=0x7f2a1c19aa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:49:54.603 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:54.599+0000 7f2a1b7fe640 1 --2- 172.21.15.39:0/2649605228 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2a1c105290 0x7f2a1c19aa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:54.603 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:54.599+0000 7f2a1b7fe640 1 -- 172.21.15.39:0/2649605228 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2a1c1a0320 con 0x7f2a1c10de10 2024-09-21T03:49:54.603 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:54.599+0000 7f2a1b7fe640 1 --2- 172.21.15.39:0/2649605228 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2a1c10de10 0x7f2a1c19fcb0 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7f2a1400cce0 tx=0x7f2a1400a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:49:54.603 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:54.599+0000 7f2a03fff640 1 -- 172.21.15.39:0/2649605228 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a1400f450 con 0x7f2a1c10de10 2024-09-21T03:49:54.603 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:54.599+0000 7f2a03fff640 1 -- 172.21.15.39:0/2649605228 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2a1400faa0 con 0x7f2a1c10de10 2024-09-21T03:49:54.603 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:54.599+0000 7f2a213c6640 1 -- 172.21.15.39:0/2649605228 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2a1c1a0610 con 0x7f2a1c10de10 2024-09-21T03:49:54.603 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:54.599+0000 7f2a03fff640 1 -- 172.21.15.39:0/2649605228 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a14018b70 con 0x7f2a1c10de10 2024-09-21T03:49:54.603 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:54.599+0000 7f2a213c6640 1 -- 172.21.15.39:0/2649605228 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2a1c1a0b00 con 0x7f2a1c10de10 2024-09-21T03:49:54.604 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:54.602+0000 7f2a2793a740 1 Processor -- start 2024-09-21T03:49:54.604 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:54.602+0000 7f2a03fff640 1 -- 172.21.15.39:0/2649605228 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 27) ==== 154643+0+0 (secure 0 0 0) 0x7f2a14018d10 con 0x7f2a1c10de10 2024-09-21T03:49:54.604 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:54.602+0000 7f2a2793a740 1 -- start start 2024-09-21T03:49:54.605 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:54.603+0000 7f2a2793a740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a42c5f8070 0x55a42c99f6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:49:54.605 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:54.603+0000 7f2a2793a740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a42c5cf1b0 0x55a42c99fbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:49:54.605 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:54.603+0000 7f2a2793a740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a42c5d7390 0x55a42c9a0120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:49:54.605 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:54.603+0000 7f2a2793a740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x55a42c915ea0 con 0x55a42c5cf1b0 2024-09-21T03:49:54.605 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:54.603+0000 7f2a2793a740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x55a42c9052d0 con 0x55a42c5d7390 2024-09-21T03:49:54.605 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:54.603+0000 7f2a2793a740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55a42c9090e0 con 0x55a42c5f8070 2024-09-21T03:49:54.605 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:54.603+0000 7f2a1affd640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a42c5cf1b0 0x55a42c99fbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:49:54.605 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:54.603+0000 7f2a1affd640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a42c5cf1b0 0x55a42c99fbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:37686/0 (socket says 172.21.15.39:37686) 2024-09-21T03:49:54.606 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:54.603+0000 7f2a1affd640 1 -- 172.21.15.39:0/1543833586 learned_addr learned my addr 172.21.15.39:0/1543833586 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:49:54.606 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:54.603+0000 7f2a1a7fc640 1 --2- 172.21.15.39:0/1543833586 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a42c5f8070 0x55a42c99f6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:49:54.606 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:54.603+0000 7f2a1a7fc640 1 --2- 172.21.15.39:0/1543833586 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a42c5d7390 0x55a42c9a0120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:49:54.606 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:54.603+0000 7f2a1affd640 1 -- 172.21.15.39:0/1543833586 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a42c5f8070 msgr2=0x55a42c99f6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:49:54.606 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:54.603+0000 7f2a1affd640 1 --2- 172.21.15.39:0/1543833586 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a42c5f8070 0x55a42c99f6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:54.606 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:54.603+0000 7f2a1affd640 1 -- 172.21.15.39:0/1543833586 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a42c5d7390 msgr2=0x55a42c9a0120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:49:54.606 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:54.603+0000 7f2a1affd640 1 --2- 172.21.15.39:0/1543833586 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a42c5d7390 0x55a42c9a0120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:54.606 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:54.603+0000 7f2a1affd640 1 -- 172.21.15.39:0/1543833586 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a42c663650 con 0x55a42c5cf1b0 2024-09-21T03:49:54.606 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:54.603+0000 7f2a1a7fc640 1 --2- 172.21.15.39:0/1543833586 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a42c5f8070 0x55a42c99f6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:49:54.606 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:54.603+0000 7f2a03fff640 1 --2- 172.21.15.39:0/2649605228 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f2a040b66b0 0x7f2a040b8bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:49:54.606 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:54.603+0000 7f2a1a7fc640 1 --2- 172.21.15.39:0/2649605228 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f2a040b66b0 0x7f2a040b8bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:49:54.606 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:54.604+0000 7f2a03fff640 1 -- 172.21.15.39:0/2649605228 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8484+0+0 (secure 0 0 0) 0x7f2a140e7aa0 con 0x7f2a1c10de10 2024-09-21T03:49:54.606 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:54.604+0000 7f2a1affd640 1 --2- 172.21.15.39:0/1543833586 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a42c5cf1b0 0x55a42c99fbe0 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7f2a0c009900 tx=0x7f2a0c032e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7fad53c999fc619a server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:49:54.609 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:54.604+0000 7f2a017fa640 1 -- 172.21.15.39:0/1543833586 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a0c040070 con 0x55a42c5cf1b0 2024-09-21T03:49:54.610 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:54.604+0000 7f2a017fa640 1 -- 172.21.15.39:0/1543833586 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2a0c0316a0 con 0x55a42c5cf1b0 2024-09-21T03:49:54.610 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:54.604+0000 7f2a017fa640 1 -- 172.21.15.39:0/1543833586 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a0c035040 con 0x55a42c5cf1b0 2024-09-21T03:49:54.610 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:54.605+0000 7f2a2793a740 1 -- 172.21.15.39:0/1543833586 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a42c5cf1b0 msgr2=0x55a42c99fbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:49:54.610 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:54.605+0000 7f2a2793a740 1 --2- 172.21.15.39:0/1543833586 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a42c5cf1b0 0x55a42c99fbe0 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7f2a0c009900 tx=0x7f2a0c032e60 comp rx=0 tx=0).stop 2024-09-21T03:49:54.610 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:54.605+0000 7f2a1a7fc640 1 --2- 172.21.15.39:0/2649605228 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f2a040b66b0 0x7f2a040b8bc0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f2a10005d00 tx=0x7f2a1000b390 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:49:54.610 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:54.605+0000 7f2a2793a740 1 -- 172.21.15.39:0/1543833586 shutdown_connections 2024-09-21T03:49:54.610 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:54.605+0000 7f2a2793a740 1 --2- 172.21.15.39:0/1543833586 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a42c5d7390 0x55a42c9a0120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:54.610 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:54.605+0000 7f2a2793a740 1 --2- 172.21.15.39:0/1543833586 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a42c5cf1b0 0x55a42c99fbe0 unknown :-1 s=CLOSED pgs=696 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:54.610 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:54.605+0000 7f2a2793a740 1 --2- 172.21.15.39:0/1543833586 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a42c5f8070 0x55a42c99f6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:54.610 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:54.605+0000 7f2a2793a740 1 -- 172.21.15.39:0/1543833586 >> 172.21.15.39:0/1543833586 conn(0x55a42c5ec460 msgr2=0x55a42c99d590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:49:54.610 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:54.605+0000 7f2a2793a740 1 -- 172.21.15.39:0/1543833586 shutdown_connections 2024-09-21T03:49:54.610 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:54.605+0000 7f2a2793a740 1 -- 172.21.15.39:0/1543833586 wait complete. 2024-09-21T03:49:54.611 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:54.605+0000 7f2a2793a740 1 -- start start 2024-09-21T03:49:54.611 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:54.605+0000 7f2a2793a740 1 Processor -- start 2024-09-21T03:49:54.611 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:54.606+0000 7f2a2793a740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a42c5cf1b0 0x55a42c9aacd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:49:54.611 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:54.606+0000 7f2a2793a740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a42c5d7390 0x55a42c9ab210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:49:54.611 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:54.606+0000 7f2a2793a740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a42c5f8070 0x55a42c9ab750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:49:54.611 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:54.606+0000 7f2a2793a740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x55a42c766700 con 0x55a42c5f8070 2024-09-21T03:49:54.611 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:54.606+0000 7f2a2793a740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x55a42c8a0160 con 0x55a42c5d7390 2024-09-21T03:49:54.611 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:54.606+0000 7f2a2793a740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55a42c9ade20 con 0x55a42c5cf1b0 2024-09-21T03:49:54.611 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:54.606+0000 7f2a2793a740 10 client.0 _ll_register_callbacks cb 0x55a42c8b99e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:49:54.611 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:54.606+0000 7f2a1b7fe640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a42c5f8070 0x55a42c9ab750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:49:54.611 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:54.606+0000 7f2a1b7fe640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a42c5f8070 0x55a42c9ab750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:37690/0 (socket says 172.21.15.39:37690) 2024-09-21T03:49:54.611 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:54.606+0000 7f2a1b7fe640 1 -- 172.21.15.39:0/3503429748 learned_addr learned my addr 172.21.15.39:0/3503429748 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:49:54.611 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:54.606+0000 7f2a1affd640 1 --2- 172.21.15.39:0/3503429748 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a42c5cf1b0 0x55a42c9aacd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:49:54.611 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:54.606+0000 7f2a1a7fc640 1 --2- 172.21.15.39:0/3503429748 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a42c5d7390 0x55a42c9ab210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:49:54.612 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:54.606+0000 7f2a1b7fe640 1 -- 172.21.15.39:0/3503429748 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a42c5cf1b0 msgr2=0x55a42c9aacd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:49:54.612 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:54.606+0000 7f2a1b7fe640 1 --2- 172.21.15.39:0/3503429748 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a42c5cf1b0 0x55a42c9aacd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:54.612 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:54.606+0000 7f2a1b7fe640 1 -- 172.21.15.39:0/3503429748 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a42c5d7390 msgr2=0x55a42c9ab210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:49:54.612 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:54.606+0000 7f2a1b7fe640 1 --2- 172.21.15.39:0/3503429748 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a42c5d7390 0x55a42c9ab210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:54.612 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:54.606+0000 7f2a1b7fe640 1 -- 172.21.15.39:0/3503429748 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a42c8cdc90 con 0x55a42c5f8070 2024-09-21T03:49:54.612 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:54.606+0000 7f2a1a7fc640 1 --2- 172.21.15.39:0/3503429748 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a42c5d7390 0x55a42c9ab210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:49:54.612 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:54.606+0000 7f2a1b7fe640 1 --2- 172.21.15.39:0/3503429748 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a42c5f8070 0x55a42c9ab750 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7f2a140e52c0 tx=0x7f2a140ad7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb46e7aaeb0f4c50 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:49:54.612 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:54.606+0000 7f29feff5640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-09-21T03:49:54.612 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:54.606+0000 7f29feff5640 1 -- 172.21.15.39:0/3503429748 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a140b21d0 con 0x55a42c5f8070 2024-09-21T03:49:54.612 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:54.607+0000 7f29feff5640 1 -- 172.21.15.39:0/3503429748 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8484+0+0 (secure 0 0 0) 0x7f2a140acb70 con 0x55a42c5f8070 2024-09-21T03:49:54.612 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:54.607+0000 7f2a2793a740 1 -- 172.21.15.39:0/3503429748 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2a0c0316a0 con 0x55a42c5f8070 2024-09-21T03:49:54.612 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:54.607+0000 7f29feff5640 1 -- 172.21.15.39:0/3503429748 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f2a140ad420 con 0x55a42c5f8070 2024-09-21T03:49:54.612 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:54.607+0000 7f2a2793a740 10 client.16338 fetch_fsmap learned FSMap version 29 2024-09-21T03:49:54.613 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:54.607+0000 7f2a2793a740 1 -- 172.21.15.39:0/3503429748 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55a42c9b9a80 con 0x55a42c5f8070 2024-09-21T03:49:54.613 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:54.607+0000 7f29feff5640 1 -- 172.21.15.39:0/3503429748 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2a0c0316a0 con 0x55a42c5f8070 2024-09-21T03:49:54.613 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:54.607+0000 7f29feff5640 1 -- 172.21.15.39:0/3503429748 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f2a140c3ac0 con 0x55a42c5f8070 2024-09-21T03:49:54.613 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:54.607+0000 7f2a2793a740 10 client.16338 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T03:49:54.613 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:54.607+0000 7f2a2793a740 10 client.16338 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.a 2024-09-21T03:49:54.613 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:54.607+0000 7f2a2793a740 20 client.16338 populate_metadata read hostname 'smithi039' 2024-09-21T03:49:54.613 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:54.607+0000 7f2a2793a740 1 --2- 172.21.15.39:0/3503429748 >> [v2:172.21.15.39:6840/4247190456,v1:172.21.15.39:6841/4247190456] conn(0x55a42c5b69f0 0x55a42c9bcef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:49:54.613 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:54.607+0000 7f2a2793a740 4 client.16338 mds_command: new command op to 14868 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:49:54.613 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:54.607+0000 7f2a2793a740 1 -- 172.21.15.39:0/3503429748 --> [v2:172.21.15.39:6840/4247190456,v1:172.21.15.39:6841/4247190456] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a42c917df0 con 0x55a42c5b69f0 2024-09-21T03:49:54.613 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:54.607+0000 7f2a1affd640 1 --2- 172.21.15.39:0/3503429748 >> [v2:172.21.15.39:6840/4247190456,v1:172.21.15.39:6841/4247190456] conn(0x55a42c5b69f0 0x55a42c9bcef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:49:54.613 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:54.608+0000 7f2a1affd640 1 --2- 172.21.15.39:0/3503429748 >> [v2:172.21.15.39:6840/4247190456,v1:172.21.15.39:6841/4247190456] conn(0x55a42c5b69f0 0x55a42c9bcef0 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=a5e22364ab7d5fe5 server_cookie=7301c09824113226 in_seq=0 out_seq=0 2024-09-21T03:49:54.613 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:54.608+0000 7f29feff5640 10 client.16338 ms_handle_connect on v2:172.21.15.39:6840/4247190456 2024-09-21T03:49:54.614 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:54.608+0000 7f29feff5640 1 -- 172.21.15.39:0/3503429748 <== mds.4 v2:172.21.15.39:6840/4247190456 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55a42c917df0 con 0x55a42c5b69f0 2024-09-21T03:49:54.614 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:54.608+0000 7f29feff5640 10 client.16338 handle_command_reply: tid=0 2024-09-21T03:49:54.614 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:54.608+0000 7f2a2793a740 1 client.16338 shutdown 2024-09-21T03:49:54.614 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:54.609+0000 7f2a2793a740 1 -- 172.21.15.39:0/3503429748 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a42c5f8070 msgr2=0x55a42c9ab750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:49:54.614 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:54.609+0000 7f2a2793a740 1 --2- 172.21.15.39:0/3503429748 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a42c5f8070 0x55a42c9ab750 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7f2a140e52c0 tx=0x7f2a140ad7a0 comp rx=0 tx=0).stop 2024-09-21T03:49:54.615 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:54.609+0000 7f2a2793a740 1 -- 172.21.15.39:0/3503429748 shutdown_connections 2024-09-21T03:49:54.615 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:54.609+0000 7f2a2793a740 1 --2- 172.21.15.39:0/3503429748 >> [v2:172.21.15.39:6840/4247190456,v1:172.21.15.39:6841/4247190456] conn(0x55a42c5b69f0 0x55a42c9bcef0 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:54.615 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:54.609+0000 7f2a2793a740 1 --2- 172.21.15.39:0/3503429748 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a42c5f8070 0x55a42c9ab750 unknown :-1 s=CLOSED pgs=697 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:54.615 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:54.609+0000 7f2a2793a740 1 --2- 172.21.15.39:0/3503429748 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a42c5d7390 0x55a42c9ab210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:54.615 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:54.609+0000 7f2a2793a740 1 --2- 172.21.15.39:0/3503429748 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a42c5cf1b0 0x55a42c9aacd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:54.615 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:54.609+0000 7f2a2793a740 1 -- 172.21.15.39:0/3503429748 >> 172.21.15.39:0/3503429748 conn(0x55a42c5ec460 msgr2=0x55a42c9a2830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:49:54.615 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:54.609+0000 7f29feff5640 0 client.16338 ms_handle_reset on v2:172.21.15.39:6840/4247190456 2024-09-21T03:49:54.615 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:54.609+0000 7f2a2793a740 1 -- 172.21.15.39:0/3503429748 shutdown_connections 2024-09-21T03:49:54.615 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:54.609+0000 7f2a2793a740 1 -- 172.21.15.39:0/3503429748 wait complete. 2024-09-21T03:49:54.615 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:54.609+0000 7f2a2793a740 20 client.16338 trim_cache size 0 max 16384 2024-09-21T03:49:54.625 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:54.622+0000 7f2a2793a740 1 Processor -- start 2024-09-21T03:49:54.625 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:54.622+0000 7f2a2793a740 1 -- start start 2024-09-21T03:49:54.625 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:54.622+0000 7f2a2793a740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a42c5cf1b0 0x55a42c9a0de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:49:54.625 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:54.622+0000 7f2a2793a740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a42c5d7390 0x55a42c9b3480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:49:54.625 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:54.622+0000 7f2a2793a740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a42c5f8070 0x55a42c9b39c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:49:54.626 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:54.623+0000 7f2a2793a740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f2a1c1af630 con 0x55a42c5d7390 2024-09-21T03:49:54.626 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:54.623+0000 7f2a2793a740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f2a1c1b1170 con 0x55a42c5cf1b0 2024-09-21T03:49:54.626 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:54.623+0000 7f2a2793a740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f2a1c1afd00 con 0x55a42c5f8070 2024-09-21T03:49:54.626 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:54.623+0000 7f2a1a7fc640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a42c5d7390 0x55a42c9b3480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:49:54.626 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:54.623+0000 7f2a1a7fc640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a42c5d7390 0x55a42c9b3480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:37702/0 (socket says 172.21.15.39:37702) 2024-09-21T03:49:54.626 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:54.623+0000 7f2a1a7fc640 1 -- 172.21.15.39:0/3227201445 learned_addr learned my addr 172.21.15.39:0/3227201445 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:49:54.626 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:54.623+0000 7f2a1b7fe640 1 --2- 172.21.15.39:0/3227201445 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a42c5f8070 0x55a42c9b39c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:49:54.626 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:54.623+0000 7f2a1a7fc640 1 -- 172.21.15.39:0/3227201445 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a42c5f8070 msgr2=0x55a42c9b39c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:49:54.626 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:54.623+0000 7f2a1a7fc640 1 --2- 172.21.15.39:0/3227201445 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a42c5f8070 0x55a42c9b39c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:54.626 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:54.623+0000 7f2a1a7fc640 1 -- 172.21.15.39:0/3227201445 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a42c5cf1b0 msgr2=0x55a42c9a0de0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-21T03:49:54.626 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:54.623+0000 7f2a1a7fc640 1 --2- 172.21.15.39:0/3227201445 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a42c5cf1b0 0x55a42c9a0de0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:54.626 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:54.623+0000 7f2a1a7fc640 1 -- 172.21.15.39:0/3227201445 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a42c9a2500 con 0x55a42c5d7390 2024-09-21T03:49:54.626 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:54.623+0000 7f2a1b7fe640 1 --2- 172.21.15.39:0/3227201445 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a42c5f8070 0x55a42c9b39c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:49:54.626 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:54.623+0000 7f2a1a7fc640 1 --2- 172.21.15.39:0/3227201445 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a42c5d7390 0x55a42c9b3480 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7f2a10003f00 tx=0x7f2a10014040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=122b5441eca4755e server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:49:54.626 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:54.623+0000 7f2a00ff9640 1 -- 172.21.15.39:0/3227201445 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a1001d450 con 0x55a42c5d7390 2024-09-21T03:49:54.627 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:54.623+0000 7f2a00ff9640 1 -- 172.21.15.39:0/3227201445 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2a1001daa0 con 0x55a42c5d7390 2024-09-21T03:49:54.627 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:54.623+0000 7f2a00ff9640 1 -- 172.21.15.39:0/3227201445 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a10012ac0 con 0x55a42c5d7390 2024-09-21T03:49:54.630 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:54.624+0000 7f2a2793a740 1 -- 172.21.15.39:0/3227201445 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a42c5d7390 msgr2=0x55a42c9b3480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:49:54.630 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:54.624+0000 7f2a2793a740 1 --2- 172.21.15.39:0/3227201445 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a42c5d7390 0x55a42c9b3480 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7f2a10003f00 tx=0x7f2a10014040 comp rx=0 tx=0).stop 2024-09-21T03:49:54.630 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:54.624+0000 7f2a2793a740 1 -- 172.21.15.39:0/3227201445 shutdown_connections 2024-09-21T03:49:54.630 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:54.624+0000 7f2a2793a740 1 --2- 172.21.15.39:0/3227201445 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a42c5f8070 0x55a42c9b39c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:54.630 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:54.624+0000 7f2a2793a740 1 --2- 172.21.15.39:0/3227201445 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a42c5d7390 0x55a42c9b3480 unknown :-1 s=CLOSED pgs=698 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:54.630 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:54.624+0000 7f2a2793a740 1 --2- 172.21.15.39:0/3227201445 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a42c5cf1b0 0x55a42c9a0de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:54.630 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:54.624+0000 7f2a2793a740 1 -- 172.21.15.39:0/3227201445 >> 172.21.15.39:0/3227201445 conn(0x55a42c5ec460 msgr2=0x55a42c9a06d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:49:54.630 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:54.624+0000 7f2a2793a740 1 -- 172.21.15.39:0/3227201445 shutdown_connections 2024-09-21T03:49:54.630 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:54.624+0000 7f2a2793a740 1 -- 172.21.15.39:0/3227201445 wait complete. 2024-09-21T03:49:54.631 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:54.625+0000 7f2a2793a740 1 -- start start 2024-09-21T03:49:54.631 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:54.625+0000 7f2a2793a740 1 Processor -- start 2024-09-21T03:49:54.631 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:54.625+0000 7f2a2793a740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a42c5cf1b0 0x55a42ca1fa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:49:54.631 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:54.625+0000 7f2a2793a740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a42c5d7390 0x55a42ca20070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:49:54.631 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:54.625+0000 7f2a2793a740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a42c5f8070 0x55a42ca24950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:49:54.631 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:54.625+0000 7f2a2793a740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f2a1c1b0aa0 con 0x55a42c5cf1b0 2024-09-21T03:49:54.631 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:54.625+0000 7f2a2793a740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f2a1c1b03d0 con 0x55a42c5d7390 2024-09-21T03:49:54.631 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:54.625+0000 7f2a2793a740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f2a1c1ad610 con 0x55a42c5f8070 2024-09-21T03:49:54.631 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:54.625+0000 7f2a2793a740 10 client.0 _ll_register_callbacks cb 0x7f2a1400af30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:49:54.631 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:54.625+0000 7f2a1affd640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a42c5cf1b0 0x55a42ca1fa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:49:54.631 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:54.625+0000 7f2a1affd640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a42c5cf1b0 0x55a42ca1fa90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:37716/0 (socket says 172.21.15.39:37716) 2024-09-21T03:49:54.631 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:54.625+0000 7f2a1affd640 1 -- 172.21.15.39:0/2370542495 learned_addr learned my addr 172.21.15.39:0/2370542495 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:49:54.631 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:54.625+0000 7f2a1a7fc640 1 --2- 172.21.15.39:0/2370542495 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a42c5d7390 0x55a42ca20070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:49:54.632 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:54.625+0000 7f2a1b7fe640 1 --2- 172.21.15.39:0/2370542495 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a42c5f8070 0x55a42ca24950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:49:54.632 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:54.626+0000 7f2a1affd640 1 -- 172.21.15.39:0/2370542495 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a42c5f8070 msgr2=0x55a42ca24950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:49:54.632 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:54.626+0000 7f2a1affd640 1 --2- 172.21.15.39:0/2370542495 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a42c5f8070 0x55a42ca24950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:54.632 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:54.626+0000 7f2a1affd640 1 -- 172.21.15.39:0/2370542495 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a42c5d7390 msgr2=0x55a42ca20070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:49:54.632 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:54.626+0000 7f2a1affd640 1 --2- 172.21.15.39:0/2370542495 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a42c5d7390 0x55a42ca20070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:54.632 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:54.626+0000 7f2a1affd640 1 -- 172.21.15.39:0/2370542495 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a42ca24fc0 con 0x55a42c5cf1b0 2024-09-21T03:49:54.632 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:54.626+0000 7f2a1b7fe640 1 --2- 172.21.15.39:0/2370542495 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a42c5f8070 0x55a42ca24950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:49:54.632 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:54.626+0000 7f2a1a7fc640 1 --2- 172.21.15.39:0/2370542495 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a42c5d7390 0x55a42ca20070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:49:54.632 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:54.626+0000 7f2a1affd640 1 --2- 172.21.15.39:0/2370542495 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a42c5cf1b0 0x55a42ca1fa90 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7f2a0c0053b0 tx=0x7f2a0c005340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=119800845e344780 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:49:54.632 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:54.626+0000 7f29feff5640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-09-21T03:49:54.632 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:54.626+0000 7f29feff5640 1 -- 172.21.15.39:0/2370542495 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a0c035040 con 0x55a42c5cf1b0 2024-09-21T03:49:54.632 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:54.626+0000 7f29feff5640 1 -- 172.21.15.39:0/2370542495 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8484+0+0 (secure 0 0 0) 0x7f2a0c031b10 con 0x55a42c5cf1b0 2024-09-21T03:49:54.632 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:54.626+0000 7f2a2793a740 1 -- 172.21.15.39:0/2370542495 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2a1001daa0 con 0x55a42c5cf1b0 2024-09-21T03:49:54.632 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:54.626+0000 7f29feff5640 1 -- 172.21.15.39:0/2370542495 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f2a0c040040 con 0x55a42c5cf1b0 2024-09-21T03:49:54.633 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:54.626+0000 7f2a2793a740 10 client.16344 fetch_fsmap learned FSMap version 29 2024-09-21T03:49:54.633 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:54.626+0000 7f2a2793a740 1 -- 172.21.15.39:0/2370542495 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55a42ca2c330 con 0x55a42c5cf1b0 2024-09-21T03:49:54.633 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:54.627+0000 7f29feff5640 1 -- 172.21.15.39:0/2370542495 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2a1001daa0 con 0x55a42c5cf1b0 2024-09-21T03:49:54.633 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:54.627+0000 7f29feff5640 1 -- 172.21.15.39:0/2370542495 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f2a0c05d020 con 0x55a42c5cf1b0 2024-09-21T03:49:54.633 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:54.627+0000 7f2a2793a740 10 client.16344 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T03:49:54.633 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:54.627+0000 7f2a2793a740 10 client.16344 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.a 2024-09-21T03:49:54.633 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:54.627+0000 7f2a2793a740 20 client.16344 populate_metadata read hostname 'smithi039' 2024-09-21T03:49:54.633 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:54.627+0000 7f2a2793a740 1 --2- 172.21.15.39:0/2370542495 >> [v2:172.21.15.39:6840/4247190456,v1:172.21.15.39:6841/4247190456] conn(0x55a42c58bda0 0x55a42ca2e740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:49:54.633 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:54.627+0000 7f2a2793a740 4 client.16344 mds_command: new command op to 14868 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T03:49:54.633 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:54.627+0000 7f2a2793a740 1 -- 172.21.15.39:0/2370542495 --> [v2:172.21.15.39:6840/4247190456,v1:172.21.15.39:6841/4247190456] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55a42c9138a0 con 0x55a42c58bda0 2024-09-21T03:49:54.633 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:54.627+0000 7f2a1a7fc640 1 --2- 172.21.15.39:0/2370542495 >> [v2:172.21.15.39:6840/4247190456,v1:172.21.15.39:6841/4247190456] conn(0x55a42c58bda0 0x55a42ca2e740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:49:54.633 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:54.627+0000 7f2a1a7fc640 1 --2- 172.21.15.39:0/2370542495 >> [v2:172.21.15.39:6840/4247190456,v1:172.21.15.39:6841/4247190456] conn(0x55a42c58bda0 0x55a42ca2e740 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=3c9b2c0a6c64dc00 server_cookie=9f3be934afa9f733 in_seq=0 out_seq=0 2024-09-21T03:49:54.633 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:54.627+0000 7f29feff5640 10 client.16344 ms_handle_connect on v2:172.21.15.39:6840/4247190456 2024-09-21T03:49:54.633 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:54.628+0000 7f29feff5640 1 -- 172.21.15.39:0/2370542495 <== mds.4 v2:172.21.15.39:6840/4247190456 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55a42c9138a0 con 0x55a42c58bda0 2024-09-21T03:49:54.634 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:54.628+0000 7f29feff5640 10 client.16344 handle_command_reply: tid=0 2024-09-21T03:49:54.634 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:54.628+0000 7f2a2793a740 1 client.16344 shutdown 2024-09-21T03:49:54.634 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:54.628+0000 7f2a2793a740 1 -- 172.21.15.39:0/2370542495 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a42c5cf1b0 msgr2=0x55a42ca1fa90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:49:54.634 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:54.628+0000 7f2a2793a740 1 --2- 172.21.15.39:0/2370542495 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a42c5cf1b0 0x55a42ca1fa90 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7f2a0c0053b0 tx=0x7f2a0c005340 comp rx=0 tx=0).stop 2024-09-21T03:49:54.634 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:54.628+0000 7f2a2793a740 1 -- 172.21.15.39:0/2370542495 shutdown_connections 2024-09-21T03:49:54.634 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:54.628+0000 7f2a2793a740 1 --2- 172.21.15.39:0/2370542495 >> [v2:172.21.15.39:6840/4247190456,v1:172.21.15.39:6841/4247190456] conn(0x55a42c58bda0 0x55a42ca2e740 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:54.634 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:54.628+0000 7f2a2793a740 1 --2- 172.21.15.39:0/2370542495 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a42c5f8070 0x55a42ca24950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:54.634 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:54.628+0000 7f2a2793a740 1 --2- 172.21.15.39:0/2370542495 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a42c5d7390 0x55a42ca20070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:54.634 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:54.628+0000 7f2a2793a740 1 --2- 172.21.15.39:0/2370542495 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a42c5cf1b0 0x55a42ca1fa90 unknown :-1 s=CLOSED pgs=699 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:54.634 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:54.628+0000 7f2a2793a740 1 -- 172.21.15.39:0/2370542495 >> 172.21.15.39:0/2370542495 conn(0x55a42c5ec460 msgr2=0x55a42c9b67c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:49:54.634 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:54.628+0000 7f29feff5640 0 client.16344 ms_handle_reset on v2:172.21.15.39:6840/4247190456 2024-09-21T03:49:54.634 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:54.628+0000 7f2a2793a740 1 -- 172.21.15.39:0/2370542495 shutdown_connections 2024-09-21T03:49:54.634 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:54.628+0000 7f2a2793a740 1 -- 172.21.15.39:0/2370542495 wait complete. 2024-09-21T03:49:54.634 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:54.628+0000 7f2a2793a740 20 client.16344 trim_cache size 0 max 16384 2024-09-21T03:49:54.635 INFO:teuthology.orchestra.run.smithi039.stdout:[] 2024-09-21T03:49:54.635 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:54.629+0000 7f2a213c6640 1 -- 172.21.15.39:0/2649605228 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f2a040b66b0 msgr2=0x7f2a040b8bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:49:54.635 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:54.629+0000 7f2a213c6640 1 --2- 172.21.15.39:0/2649605228 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f2a040b66b0 0x7f2a040b8bc0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f2a10005d00 tx=0x7f2a1000b390 comp rx=0 tx=0).stop 2024-09-21T03:49:54.635 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:54.629+0000 7f2a213c6640 1 -- 172.21.15.39:0/2649605228 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2a1c10de10 msgr2=0x7f2a1c19fcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:49:54.636 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:54.629+0000 7f2a213c6640 1 --2- 172.21.15.39:0/2649605228 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2a1c10de10 0x7f2a1c19fcb0 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7f2a1400cce0 tx=0x7f2a1400a620 comp rx=0 tx=0).stop 2024-09-21T03:49:54.636 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:54.629+0000 7f2a213c6640 1 -- 172.21.15.39:0/2649605228 shutdown_connections 2024-09-21T03:49:54.636 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:54.629+0000 7f2a213c6640 1 --2- 172.21.15.39:0/2649605228 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f2a040b66b0 0x7f2a040b8bc0 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:54.636 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:54.629+0000 7f2a213c6640 1 --2- 172.21.15.39:0/2649605228 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2a1c10de10 0x7f2a1c19fcb0 unknown :-1 s=CLOSED pgs=695 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:54.636 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:54.629+0000 7f2a213c6640 1 --2- 172.21.15.39:0/2649605228 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2a1c105290 0x7f2a1c19aa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:54.636 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:54.629+0000 7f2a213c6640 1 --2- 172.21.15.39:0/2649605228 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2a1c104860 0x7f2a1c19a470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:54.636 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:54.629+0000 7f2a213c6640 1 -- 172.21.15.39:0/2649605228 >> 172.21.15.39:0/2649605228 conn(0x7f2a1c079730 msgr2=0x7f2a1c100890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:49:54.636 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:54.629+0000 7f2a213c6640 1 -- 172.21.15.39:0/2649605228 shutdown_connections 2024-09-21T03:49:54.636 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:54.629+0000 7f2a213c6640 1 -- 172.21.15.39:0/2649605228 wait complete. 2024-09-21T03:49:55.084 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:49:54 smithi039 ceph-mon[31211]: pgmap v441: 97 pgs: 97 active+clean; 10 MiB data, 536 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-09-21T03:49:55.218 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:49:54 smithi161 ceph-mon[34533]: pgmap v441: 97 pgs: 97 active+clean; 10 MiB data, 536 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-09-21T03:49:55.224 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:49:54 smithi150 ceph-mon[34289]: pgmap v441: 97 pgs: 97 active+clean; 10 MiB data, 536 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-09-21T03:49:55.423 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 51116d5c-77ca-11ef-baf5-efdc52797490 -- ceph tell mds.1:3 damage ls 2024-09-21T03:49:55.682 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/51116d5c-77ca-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:49:56.173 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:56.158+0000 7f6790ebb640 1 -- 172.21.15.39:0/647378568 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f678c103b60 msgr2=0x7f678c103f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:49:56.173 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:56.158+0000 7f6790ebb640 1 --2- 172.21.15.39:0/647378568 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f678c103b60 0x7f678c103f70 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7f677c009900 tx=0x7f677c02a410 comp rx=0 tx=0).stop 2024-09-21T03:49:56.173 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:56.158+0000 7f6790ebb640 1 -- 172.21.15.39:0/647378568 shutdown_connections 2024-09-21T03:49:56.173 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:56.158+0000 7f6790ebb640 1 --2- 172.21.15.39:0/647378568 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f678c109df0 0x7f678c110620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:56.173 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:56.158+0000 7f6790ebb640 1 --2- 172.21.15.39:0/647378568 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f678c1094a0 0x7f678c1098b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:56.173 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:56.158+0000 7f6790ebb640 1 --2- 172.21.15.39:0/647378568 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f678c103b60 0x7f678c103f70 unknown :-1 s=CLOSED pgs=700 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:56.173 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:56.158+0000 7f6790ebb640 1 -- 172.21.15.39:0/647378568 >> 172.21.15.39:0/647378568 conn(0x7f678c07b850 msgr2=0x7f678c07bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:49:56.173 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:56.158+0000 7f6790ebb640 1 -- 172.21.15.39:0/647378568 shutdown_connections 2024-09-21T03:49:56.173 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:56.158+0000 7f6790ebb640 1 -- 172.21.15.39:0/647378568 wait complete. 2024-09-21T03:49:56.173 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:56.159+0000 7f6790ebb640 1 Processor -- start 2024-09-21T03:49:56.174 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:56.159+0000 7f6790ebb640 1 -- start start 2024-09-21T03:49:56.174 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:56.159+0000 7f6790ebb640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f678c103b60 0x7f678c19a4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:49:56.174 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:56.159+0000 7f6790ebb640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f678c1094a0 0x7f678c19aa40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:49:56.174 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:56.159+0000 7f6790ebb640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f678c109df0 0x7f678c19fce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:49:56.174 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:56.159+0000 7f6790ebb640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f678c113ab0 con 0x7f678c1094a0 2024-09-21T03:49:56.174 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:56.159+0000 7f6790ebb640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f678c113930 con 0x7f678c109df0 2024-09-21T03:49:56.174 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:56.159+0000 7f6790ebb640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f678c113c30 con 0x7f678c103b60 2024-09-21T03:49:56.174 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:56.160+0000 7f6789d9b640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f678c1094a0 0x7f678c19aa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:49:56.174 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:56.160+0000 7f6789d9b640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f678c1094a0 0x7f678c19aa40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:37738/0 (socket says 172.21.15.39:37738) 2024-09-21T03:49:56.174 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:56.160+0000 7f678a59c640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f678c103b60 0x7f678c19a4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:49:56.174 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:56.160+0000 7f6789d9b640 1 -- 172.21.15.39:0/3554839910 learned_addr learned my addr 172.21.15.39:0/3554839910 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:49:56.174 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:56.160+0000 7f678ad9d640 1 --2- 172.21.15.39:0/3554839910 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f678c109df0 0x7f678c19fce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:49:56.174 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:56.160+0000 7f6789d9b640 1 -- 172.21.15.39:0/3554839910 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f678c103b60 msgr2=0x7f678c19a4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:49:56.175 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:56.160+0000 7f6789d9b640 1 --2- 172.21.15.39:0/3554839910 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f678c103b60 0x7f678c19a4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:56.175 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:56.160+0000 7f6789d9b640 1 -- 172.21.15.39:0/3554839910 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f678c109df0 msgr2=0x7f678c19fce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:49:56.175 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:56.160+0000 7f6789d9b640 1 --2- 172.21.15.39:0/3554839910 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f678c109df0 0x7f678c19fce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:56.175 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:56.160+0000 7f6789d9b640 1 -- 172.21.15.39:0/3554839910 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f678c1a0410 con 0x7f678c1094a0 2024-09-21T03:49:56.175 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:56.160+0000 7f678ad9d640 1 --2- 172.21.15.39:0/3554839910 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f678c109df0 0x7f678c19fce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:49:56.175 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:56.160+0000 7f6789d9b640 1 --2- 172.21.15.39:0/3554839910 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f678c1094a0 0x7f678c19aa40 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7f678000d750 tx=0x7f678000dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:49:56.175 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:56.160+0000 7f677b7fe640 1 -- 172.21.15.39:0/3554839910 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6780017070 con 0x7f678c1094a0 2024-09-21T03:49:56.175 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:56.160+0000 7f677b7fe640 1 -- 172.21.15.39:0/3554839910 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f67800054d0 con 0x7f678c1094a0 2024-09-21T03:49:56.175 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:56.160+0000 7f6790ebb640 1 -- 172.21.15.39:0/3554839910 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f678c1a0700 con 0x7f678c1094a0 2024-09-21T03:49:56.175 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:56.160+0000 7f677b7fe640 1 -- 172.21.15.39:0/3554839910 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6780004c30 con 0x7f678c1094a0 2024-09-21T03:49:56.175 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:56.161+0000 7f6790ebb640 1 -- 172.21.15.39:0/3554839910 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f678c1a6e60 con 0x7f678c1094a0 2024-09-21T03:49:56.175 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:56.163+0000 7f679742f740 1 Processor -- start 2024-09-21T03:49:56.175 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:56.164+0000 7f679742f740 1 -- start start 2024-09-21T03:49:56.176 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:56.164+0000 7f677b7fe640 1 -- 172.21.15.39:0/3554839910 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 27) ==== 154643+0+0 (secure 0 0 0) 0x7f6780004dd0 con 0x7f678c1094a0 2024-09-21T03:49:56.176 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:56.164+0000 7f679742f740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a1701f8070 0x55a17059f2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:49:56.176 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:56.164+0000 7f679742f740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a1701cf1b0 0x55a17059f800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:49:56.176 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:56.164+0000 7f679742f740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a1701d7390 0x55a17059fd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:49:56.176 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:56.164+0000 7f679742f740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x55a170515ea0 con 0x55a1701f8070 2024-09-21T03:49:56.176 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:56.164+0000 7f679742f740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x55a1705052d0 con 0x55a1701d7390 2024-09-21T03:49:56.176 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:56.164+0000 7f679742f740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55a1705090e0 con 0x55a1701cf1b0 2024-09-21T03:49:56.176 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:56.164+0000 7f678ad9d640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a1701f8070 0x55a17059f2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:49:56.176 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:56.164+0000 7f678ad9d640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a1701f8070 0x55a17059f2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:37744/0 (socket says 172.21.15.39:37744) 2024-09-21T03:49:56.176 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:56.164+0000 7f678ad9d640 1 -- 172.21.15.39:0/1299470205 learned_addr learned my addr 172.21.15.39:0/1299470205 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:49:56.176 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:56.164+0000 7f678a59c640 1 --2- 172.21.15.39:0/1299470205 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a1701cf1b0 0x55a17059f800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:49:56.176 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:56.164+0000 7f6789d9b640 1 --2- 172.21.15.39:0/1299470205 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a1701d7390 0x55a17059fd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:49:56.176 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:56.164+0000 7f678ad9d640 1 -- 172.21.15.39:0/1299470205 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a1701cf1b0 msgr2=0x55a17059f800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:49:56.176 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:56.164+0000 7f678ad9d640 1 --2- 172.21.15.39:0/1299470205 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a1701cf1b0 0x55a17059f800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:56.176 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:56.164+0000 7f678ad9d640 1 -- 172.21.15.39:0/1299470205 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a1701d7390 msgr2=0x55a17059fd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:49:56.177 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:56.164+0000 7f678ad9d640 1 --2- 172.21.15.39:0/1299470205 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a1701d7390 0x55a17059fd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:56.177 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:56.164+0000 7f678ad9d640 1 -- 172.21.15.39:0/1299470205 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a170263650 con 0x55a1701f8070 2024-09-21T03:49:56.177 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:56.165+0000 7f678ad9d640 1 --2- 172.21.15.39:0/1299470205 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a1701f8070 0x55a17059f2c0 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7f6784009ed0 tx=0x7f678400f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f8b669d4bce667d3 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:49:56.177 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:56.165+0000 7f6778ff9640 1 -- 172.21.15.39:0/1299470205 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6784004d20 con 0x55a1701f8070 2024-09-21T03:49:56.177 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:56.165+0000 7f6778ff9640 1 -- 172.21.15.39:0/1299470205 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f67840054d0 con 0x55a1701f8070 2024-09-21T03:49:56.177 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:56.165+0000 7f6778ff9640 1 -- 172.21.15.39:0/1299470205 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6784011840 con 0x55a1701f8070 2024-09-21T03:49:56.177 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:56.165+0000 7f677b7fe640 1 --2- 172.21.15.39:0/3554839910 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f67740b6700 0x7f67740b8c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:49:56.177 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:56.165+0000 7f678a59c640 1 --2- 172.21.15.39:0/3554839910 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f67740b6700 0x7f67740b8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:49:56.177 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:56.165+0000 7f677b7fe640 1 -- 172.21.15.39:0/3554839910 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8484+0+0 (secure 0 0 0) 0x7f67800e70a0 con 0x7f678c1094a0 2024-09-21T03:49:56.177 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:56.166+0000 7f679742f740 1 -- 172.21.15.39:0/1299470205 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a1701f8070 msgr2=0x55a17059f2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:49:56.177 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:56.166+0000 7f678a59c640 1 --2- 172.21.15.39:0/3554839910 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f67740b6700 0x7f67740b8c10 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f677c031440 tx=0x7f677c009610 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:49:56.177 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:56.166+0000 7f679742f740 1 --2- 172.21.15.39:0/1299470205 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a1701f8070 0x55a17059f2c0 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7f6784009ed0 tx=0x7f678400f730 comp rx=0 tx=0).stop 2024-09-21T03:49:56.177 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:56.166+0000 7f679742f740 1 -- 172.21.15.39:0/1299470205 shutdown_connections 2024-09-21T03:49:56.177 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:56.166+0000 7f679742f740 1 --2- 172.21.15.39:0/1299470205 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a1701d7390 0x55a17059fd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:56.177 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:56.166+0000 7f679742f740 1 --2- 172.21.15.39:0/1299470205 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a1701cf1b0 0x55a17059f800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:56.178 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:56.166+0000 7f679742f740 1 --2- 172.21.15.39:0/1299470205 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a1701f8070 0x55a17059f2c0 unknown :-1 s=CLOSED pgs=702 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:56.178 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:56.166+0000 7f679742f740 1 -- 172.21.15.39:0/1299470205 >> 172.21.15.39:0/1299470205 conn(0x55a1701ec460 msgr2=0x55a17059d1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:49:56.178 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:56.166+0000 7f679742f740 1 -- 172.21.15.39:0/1299470205 shutdown_connections 2024-09-21T03:49:56.178 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:56.166+0000 7f679742f740 1 -- 172.21.15.39:0/1299470205 wait complete. 2024-09-21T03:49:56.178 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:56.166+0000 7f679742f740 1 -- start start 2024-09-21T03:49:56.178 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:56.166+0000 7f679742f740 1 Processor -- start 2024-09-21T03:49:56.178 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:56.167+0000 7f679742f740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a1701cf1b0 0x55a1705aece0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:49:56.178 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:56.167+0000 7f679742f740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a1701d7390 0x55a1705af220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:49:56.178 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:56.167+0000 7f679742f740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a1701f8070 0x55a1705af780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:49:56.178 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:56.167+0000 7f679742f740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x55a170366700 con 0x55a1701cf1b0 2024-09-21T03:49:56.178 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:56.167+0000 7f679742f740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x55a1704a0160 con 0x55a1701f8070 2024-09-21T03:49:56.178 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:56.167+0000 7f679742f740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55a1705b1dd0 con 0x55a1701d7390 2024-09-21T03:49:56.178 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:56.167+0000 7f679742f740 10 client.0 _ll_register_callbacks cb 0x55a1704b99e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:49:56.179 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:56.167+0000 7f678a59c640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a1701cf1b0 0x55a1705aece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:49:56.179 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:56.167+0000 7f678a59c640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a1701cf1b0 0x55a1705aece0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:37746/0 (socket says 172.21.15.39:37746) 2024-09-21T03:49:56.179 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:56.167+0000 7f678a59c640 1 -- 172.21.15.39:0/2906396569 learned_addr learned my addr 172.21.15.39:0/2906396569 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:49:56.179 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:56.167+0000 7f6789d9b640 1 --2- 172.21.15.39:0/2906396569 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a1701d7390 0x55a1705af220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:49:56.179 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:56.167+0000 7f678ad9d640 1 --2- 172.21.15.39:0/2906396569 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a1701f8070 0x55a1705af780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:49:56.179 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:56.167+0000 7f678a59c640 1 -- 172.21.15.39:0/2906396569 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a1701d7390 msgr2=0x55a1705af220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:49:56.179 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:56.167+0000 7f678a59c640 1 --2- 172.21.15.39:0/2906396569 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a1701d7390 0x55a1705af220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:56.179 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:56.167+0000 7f678a59c640 1 -- 172.21.15.39:0/2906396569 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a1701f8070 msgr2=0x55a1705af780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:49:56.179 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:56.167+0000 7f678a59c640 1 --2- 172.21.15.39:0/2906396569 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a1701f8070 0x55a1705af780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:56.179 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:56.167+0000 7f678a59c640 1 -- 172.21.15.39:0/2906396569 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a1704cdc90 con 0x55a1701cf1b0 2024-09-21T03:49:56.179 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:56.167+0000 7f6789d9b640 1 --2- 172.21.15.39:0/2906396569 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a1701d7390 0x55a1705af220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:49:56.179 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:56.168+0000 7f678a59c640 1 --2- 172.21.15.39:0/2906396569 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a1701cf1b0 0x55a1705aece0 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7f677c033000 tx=0x7f677c009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b0cd47f3272ae7c9 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:49:56.179 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:56.168+0000 7f6771ffb640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-09-21T03:49:56.179 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:56.168+0000 7f6771ffb640 1 -- 172.21.15.39:0/2906396569 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f677c04d070 con 0x55a1701cf1b0 2024-09-21T03:49:56.180 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:56.168+0000 7f6771ffb640 1 -- 172.21.15.39:0/2906396569 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8484+0+0 (secure 0 0 0) 0x7f677c005030 con 0x55a1701cf1b0 2024-09-21T03:49:56.180 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:56.168+0000 7f679742f740 1 -- 172.21.15.39:0/2906396569 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f67840054d0 con 0x55a1701cf1b0 2024-09-21T03:49:56.180 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:56.168+0000 7f6771ffb640 1 -- 172.21.15.39:0/2906396569 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f677c059050 con 0x55a1701cf1b0 2024-09-21T03:49:56.180 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:56.168+0000 7f679742f740 10 client.16356 fetch_fsmap learned FSMap version 29 2024-09-21T03:49:56.180 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:56.168+0000 7f679742f740 1 -- 172.21.15.39:0/2906396569 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55a1705bdac0 con 0x55a1701cf1b0 2024-09-21T03:49:56.180 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:56.168+0000 7f6771ffb640 1 -- 172.21.15.39:0/2906396569 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f67840054d0 con 0x55a1701cf1b0 2024-09-21T03:49:56.180 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:56.168+0000 7f6771ffb640 1 -- 172.21.15.39:0/2906396569 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f677c0035f0 con 0x55a1701cf1b0 2024-09-21T03:49:56.180 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:56.168+0000 7f679742f740 10 client.16356 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T03:49:56.180 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:56.168+0000 7f679742f740 10 client.16356 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-21T03:49:56.180 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:56.168+0000 7f679742f740 20 client.16356 populate_metadata read hostname 'smithi039' 2024-09-21T03:49:56.180 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:56.168+0000 7f679742f740 1 --2- 172.21.15.39:0/2906396569 >> [v2:172.21.15.161:6836/3451544597,v1:172.21.15.161:6837/3451544597] conn(0x55a1701b69f0 0x55a1705c0f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:49:56.180 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:56.168+0000 7f679742f740 4 client.16356 mds_command: new command op to 24646 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:49:56.180 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:56.168+0000 7f679742f740 1 -- 172.21.15.39:0/2906396569 --> [v2:172.21.15.161:6836/3451544597,v1:172.21.15.161:6837/3451544597] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a170517df0 con 0x55a1701b69f0 2024-09-21T03:49:56.180 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:56.169+0000 7f6789d9b640 1 --2- 172.21.15.39:0/2906396569 >> [v2:172.21.15.161:6836/3451544597,v1:172.21.15.161:6837/3451544597] conn(0x55a1701b69f0 0x55a1705c0f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:49:56.181 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:56.169+0000 7f6771ffb640 10 client.16356 ms_handle_connect on v2:172.21.15.161:6836/3451544597 2024-09-21T03:49:56.181 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:56.169+0000 7f6789d9b640 1 --2- 172.21.15.39:0/2906396569 >> [v2:172.21.15.161:6836/3451544597,v1:172.21.15.161:6837/3451544597] conn(0x55a1701b69f0 0x55a1705c0f80 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=5bc09e22169ac230 server_cookie=5ff8dc7796c28bbb in_seq=0 out_seq=0 2024-09-21T03:49:56.181 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:56.170+0000 7f6771ffb640 1 -- 172.21.15.39:0/2906396569 <== mds.3 v2:172.21.15.161:6836/3451544597 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55a170517df0 con 0x55a1701b69f0 2024-09-21T03:49:56.181 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:56.170+0000 7f6771ffb640 10 client.16356 handle_command_reply: tid=0 2024-09-21T03:49:56.181 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:56.170+0000 7f679742f740 1 client.16356 shutdown 2024-09-21T03:49:56.181 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:56.170+0000 7f679742f740 1 -- 172.21.15.39:0/2906396569 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a1701cf1b0 msgr2=0x55a1705aece0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:49:56.181 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:56.170+0000 7f679742f740 1 --2- 172.21.15.39:0/2906396569 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a1701cf1b0 0x55a1705aece0 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7f677c033000 tx=0x7f677c009260 comp rx=0 tx=0).stop 2024-09-21T03:49:56.181 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:56.170+0000 7f679742f740 1 -- 172.21.15.39:0/2906396569 shutdown_connections 2024-09-21T03:49:56.181 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:56.170+0000 7f679742f740 1 --2- 172.21.15.39:0/2906396569 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a1701f8070 0x55a1705af780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:56.181 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:56.170+0000 7f679742f740 1 --2- 172.21.15.39:0/2906396569 >> [v2:172.21.15.161:6836/3451544597,v1:172.21.15.161:6837/3451544597] conn(0x55a1701b69f0 0x55a1705c0f80 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:56.181 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:56.170+0000 7f679742f740 1 --2- 172.21.15.39:0/2906396569 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a1701d7390 0x55a1705af220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:56.181 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:56.170+0000 7f679742f740 1 --2- 172.21.15.39:0/2906396569 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a1701cf1b0 0x55a1705aece0 unknown :-1 s=CLOSED pgs=703 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:56.181 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:56.170+0000 7f679742f740 1 -- 172.21.15.39:0/2906396569 >> 172.21.15.39:0/2906396569 conn(0x55a1701ec460 msgr2=0x55a1705a2410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:49:56.182 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:56.170+0000 7f6771ffb640 0 client.16356 ms_handle_reset on v2:172.21.15.161:6836/3451544597 2024-09-21T03:49:56.182 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:56.170+0000 7f679742f740 1 -- 172.21.15.39:0/2906396569 shutdown_connections 2024-09-21T03:49:56.182 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:56.170+0000 7f679742f740 1 -- 172.21.15.39:0/2906396569 wait complete. 2024-09-21T03:49:56.182 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:56.171+0000 7f679742f740 20 client.16356 trim_cache size 0 max 16384 2024-09-21T03:49:56.186 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:56.184+0000 7f679742f740 1 Processor -- start 2024-09-21T03:49:56.187 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:56.184+0000 7f679742f740 1 -- start start 2024-09-21T03:49:56.187 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:56.184+0000 7f679742f740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a1701cf1b0 0x55a1705aa490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:49:56.187 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:56.184+0000 7f679742f740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a1701d7390 0x55a1705b0910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:49:56.187 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:56.184+0000 7f679742f740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a1701f8070 0x55a1705abed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:49:56.187 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:56.184+0000 7f679742f740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f678c1ad520 con 0x55a1701cf1b0 2024-09-21T03:49:56.187 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:56.184+0000 7f679742f740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f678c1af060 con 0x55a1701f8070 2024-09-21T03:49:56.187 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:56.184+0000 7f679742f740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f678c1adbf0 con 0x55a1701d7390 2024-09-21T03:49:56.187 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:56.184+0000 7f678a59c640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a1701cf1b0 0x55a1705aa490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:49:56.187 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:56.184+0000 7f678a59c640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a1701cf1b0 0x55a1705aa490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:37758/0 (socket says 172.21.15.39:37758) 2024-09-21T03:49:56.187 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:56.184+0000 7f678a59c640 1 -- 172.21.15.39:0/1458073017 learned_addr learned my addr 172.21.15.39:0/1458073017 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:49:56.187 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:56.184+0000 7f6789d9b640 1 --2- 172.21.15.39:0/1458073017 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a1701d7390 0x55a1705b0910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:49:56.187 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:56.184+0000 7f678ad9d640 1 --2- 172.21.15.39:0/1458073017 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a1701f8070 0x55a1705abed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:49:56.187 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:56.184+0000 7f678a59c640 1 -- 172.21.15.39:0/1458073017 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a1701d7390 msgr2=0x55a1705b0910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:49:56.188 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:56.184+0000 7f678a59c640 1 --2- 172.21.15.39:0/1458073017 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a1701d7390 0x55a1705b0910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:56.188 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:56.185+0000 7f678a59c640 1 -- 172.21.15.39:0/1458073017 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a1701f8070 msgr2=0x55a1705abed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:49:56.188 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:56.185+0000 7f678a59c640 1 --2- 172.21.15.39:0/1458073017 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a1701f8070 0x55a1705abed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:56.188 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:56.185+0000 7f678a59c640 1 -- 172.21.15.39:0/1458073017 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a1705a3370 con 0x55a1701cf1b0 2024-09-21T03:49:56.188 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:56.185+0000 7f678a59c640 1 --2- 172.21.15.39:0/1458073017 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a1701cf1b0 0x55a1705aa490 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7f677c0039c0 tx=0x7f677c02a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a28ee86e1e45419 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:49:56.188 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:56.185+0000 7f6773fff640 1 -- 172.21.15.39:0/1458073017 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f677c04d070 con 0x55a1701cf1b0 2024-09-21T03:49:56.188 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:56.185+0000 7f6773fff640 1 -- 172.21.15.39:0/1458073017 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f677c0054e0 con 0x55a1701cf1b0 2024-09-21T03:49:56.188 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:56.185+0000 7f6773fff640 1 -- 172.21.15.39:0/1458073017 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f677c035040 con 0x55a1701cf1b0 2024-09-21T03:49:56.191 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:56.186+0000 7f679742f740 1 -- 172.21.15.39:0/1458073017 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a1701cf1b0 msgr2=0x55a1705aa490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:49:56.191 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:56.186+0000 7f679742f740 1 --2- 172.21.15.39:0/1458073017 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a1701cf1b0 0x55a1705aa490 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7f677c0039c0 tx=0x7f677c02a410 comp rx=0 tx=0).stop 2024-09-21T03:49:56.192 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:56.186+0000 7f679742f740 1 -- 172.21.15.39:0/1458073017 shutdown_connections 2024-09-21T03:49:56.192 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:56.186+0000 7f679742f740 1 --2- 172.21.15.39:0/1458073017 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a1701f8070 0x55a1705abed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:56.192 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:56.186+0000 7f679742f740 1 --2- 172.21.15.39:0/1458073017 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a1701d7390 0x55a1705b0910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:56.192 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:56.186+0000 7f679742f740 1 --2- 172.21.15.39:0/1458073017 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a1701cf1b0 0x55a1705aa490 unknown :-1 s=CLOSED pgs=704 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:56.192 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:56.186+0000 7f679742f740 1 -- 172.21.15.39:0/1458073017 >> 172.21.15.39:0/1458073017 conn(0x55a1701ec460 msgr2=0x55a1705b0200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:49:56.192 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:56.186+0000 7f679742f740 1 -- 172.21.15.39:0/1458073017 shutdown_connections 2024-09-21T03:49:56.192 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:56.186+0000 7f679742f740 1 -- 172.21.15.39:0/1458073017 wait complete. 2024-09-21T03:49:56.192 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:56.186+0000 7f679742f740 1 -- start start 2024-09-21T03:49:56.192 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:56.186+0000 7f679742f740 1 Processor -- start 2024-09-21T03:49:56.192 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:56.187+0000 7f679742f740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a1701cf1b0 0x55a1705b75d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:49:56.192 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:56.187+0000 7f679742f740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a1701d7390 0x55a1705b2a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:49:56.192 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:56.187+0000 7f679742f740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a1701f8070 0x55a1705b2fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:49:56.193 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:56.187+0000 7f679742f740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f678c1ae990 con 0x55a1701d7390 2024-09-21T03:49:56.193 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:56.187+0000 7f679742f740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f678c1ae2c0 con 0x55a1701cf1b0 2024-09-21T03:49:56.193 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:56.187+0000 7f679742f740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f678c1ab500 con 0x55a1701f8070 2024-09-21T03:49:56.193 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:56.187+0000 7f679742f740 10 client.0 _ll_register_callbacks cb 0x7f677c005db0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:49:56.193 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:56.187+0000 7f6789d9b640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a1701d7390 0x55a1705b2a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:49:56.193 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:56.187+0000 7f6789d9b640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a1701d7390 0x55a1705b2a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:37766/0 (socket says 172.21.15.39:37766) 2024-09-21T03:49:56.193 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:56.187+0000 7f6789d9b640 1 -- 172.21.15.39:0/511350084 learned_addr learned my addr 172.21.15.39:0/511350084 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:49:56.193 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:56.187+0000 7f678a59c640 1 --2- 172.21.15.39:0/511350084 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a1701cf1b0 0x55a1705b75d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:49:56.193 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:56.187+0000 7f678ad9d640 1 --2- 172.21.15.39:0/511350084 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a1701f8070 0x55a1705b2fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:49:56.193 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:56.187+0000 7f6789d9b640 1 -- 172.21.15.39:0/511350084 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a1701f8070 msgr2=0x55a1705b2fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:49:56.193 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:56.187+0000 7f6789d9b640 1 --2- 172.21.15.39:0/511350084 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a1701f8070 0x55a1705b2fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:56.193 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:56.187+0000 7f6789d9b640 1 -- 172.21.15.39:0/511350084 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a1701cf1b0 msgr2=0x55a1705b75d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:49:56.193 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:56.187+0000 7f6789d9b640 1 --2- 172.21.15.39:0/511350084 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a1701cf1b0 0x55a1705b75d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:56.193 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:56.187+0000 7f6789d9b640 1 -- 172.21.15.39:0/511350084 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a1705b3500 con 0x55a1701d7390 2024-09-21T03:49:56.194 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:56.187+0000 7f6789d9b640 1 --2- 172.21.15.39:0/511350084 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a1701d7390 0x55a1705b2a80 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7f67800e4e10 tx=0x7f67800d0040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e6b52d9f66c4457 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:49:56.194 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:56.187+0000 7f6771ffb640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-09-21T03:49:56.194 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:56.187+0000 7f6771ffb640 1 -- 172.21.15.39:0/511350084 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6780020040 con 0x55a1701d7390 2024-09-21T03:49:56.194 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:56.187+0000 7f6771ffb640 1 -- 172.21.15.39:0/511350084 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8484+0+0 (secure 0 0 0) 0x7f67800e4a90 con 0x55a1701d7390 2024-09-21T03:49:56.194 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:56.188+0000 7f679742f740 1 -- 172.21.15.39:0/511350084 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f677c0054e0 con 0x55a1701d7390 2024-09-21T03:49:56.194 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:56.188+0000 7f6771ffb640 1 -- 172.21.15.39:0/511350084 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f67800b19f0 con 0x55a1701d7390 2024-09-21T03:49:56.194 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:56.188+0000 7f679742f740 10 client.16362 fetch_fsmap learned FSMap version 29 2024-09-21T03:49:56.194 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:56.188+0000 7f679742f740 1 -- 172.21.15.39:0/511350084 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55a17062b630 con 0x55a1701d7390 2024-09-21T03:49:56.194 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:56.188+0000 7f6771ffb640 1 -- 172.21.15.39:0/511350084 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f677c0054e0 con 0x55a1701d7390 2024-09-21T03:49:56.194 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:56.188+0000 7f6771ffb640 1 -- 172.21.15.39:0/511350084 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f6780092ce0 con 0x55a1701d7390 2024-09-21T03:49:56.194 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:56.188+0000 7f679742f740 10 client.16362 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T03:49:56.194 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:56.188+0000 7f679742f740 10 client.16362 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-21T03:49:56.194 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:56.188+0000 7f679742f740 20 client.16362 populate_metadata read hostname 'smithi039' 2024-09-21T03:49:56.195 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:56.188+0000 7f679742f740 1 --2- 172.21.15.39:0/511350084 >> [v2:172.21.15.161:6836/3451544597,v1:172.21.15.161:6837/3451544597] conn(0x55a17018bda0 0x55a17062ddf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:49:56.195 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:56.188+0000 7f679742f740 4 client.16362 mds_command: new command op to 24646 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T03:49:56.195 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:56.188+0000 7f679742f740 1 -- 172.21.15.39:0/511350084 --> [v2:172.21.15.161:6836/3451544597,v1:172.21.15.161:6837/3451544597] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55a17059d2a0 con 0x55a17018bda0 2024-09-21T03:49:56.195 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:56.188+0000 7f678a59c640 1 --2- 172.21.15.39:0/511350084 >> [v2:172.21.15.161:6836/3451544597,v1:172.21.15.161:6837/3451544597] conn(0x55a17018bda0 0x55a17062ddf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:49:56.195 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:56.189+0000 7f6771ffb640 10 client.16362 ms_handle_connect on v2:172.21.15.161:6836/3451544597 2024-09-21T03:49:56.195 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:56.189+0000 7f678a59c640 1 --2- 172.21.15.39:0/511350084 >> [v2:172.21.15.161:6836/3451544597,v1:172.21.15.161:6837/3451544597] conn(0x55a17018bda0 0x55a17062ddf0 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=3567ec6b1e05d202 server_cookie=b076a7ebba3ccec5 in_seq=0 out_seq=0 2024-09-21T03:49:56.195 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:56.189+0000 7f6771ffb640 1 -- 172.21.15.39:0/511350084 <== mds.3 v2:172.21.15.161:6836/3451544597 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55a17059d2a0 con 0x55a17018bda0 2024-09-21T03:49:56.195 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:56.189+0000 7f6771ffb640 10 client.16362 handle_command_reply: tid=0 2024-09-21T03:49:56.195 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:56.189+0000 7f679742f740 1 client.16362 shutdown 2024-09-21T03:49:56.195 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:56.189+0000 7f679742f740 1 -- 172.21.15.39:0/511350084 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a1701d7390 msgr2=0x55a1705b2a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:49:56.195 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:56.189+0000 7f679742f740 1 --2- 172.21.15.39:0/511350084 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a1701d7390 0x55a1705b2a80 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7f67800e4e10 tx=0x7f67800d0040 comp rx=0 tx=0).stop 2024-09-21T03:49:56.195 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:56.189+0000 7f679742f740 1 -- 172.21.15.39:0/511350084 shutdown_connections 2024-09-21T03:49:56.195 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:56.189+0000 7f679742f740 1 --2- 172.21.15.39:0/511350084 >> [v2:172.21.15.161:6836/3451544597,v1:172.21.15.161:6837/3451544597] conn(0x55a17018bda0 0x55a17062ddf0 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:56.195 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:56.189+0000 7f679742f740 1 --2- 172.21.15.39:0/511350084 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a1701f8070 0x55a1705b2fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:56.196 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:56.189+0000 7f679742f740 1 --2- 172.21.15.39:0/511350084 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a1701d7390 0x55a1705b2a80 unknown :-1 s=CLOSED pgs=705 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:56.196 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:56.189+0000 7f679742f740 1 --2- 172.21.15.39:0/511350084 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a1701cf1b0 0x55a1705b75d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:56.196 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:56.189+0000 7f679742f740 1 -- 172.21.15.39:0/511350084 >> 172.21.15.39:0/511350084 conn(0x55a1701ec460 msgr2=0x55a1705a02f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:49:56.196 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:56.189+0000 7f6771ffb640 0 client.16362 ms_handle_reset on v2:172.21.15.161:6836/3451544597 2024-09-21T03:49:56.196 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:56.190+0000 7f679742f740 1 -- 172.21.15.39:0/511350084 shutdown_connections 2024-09-21T03:49:56.196 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:56.190+0000 7f679742f740 1 -- 172.21.15.39:0/511350084 wait complete. 2024-09-21T03:49:56.196 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:56.190+0000 7f679742f740 20 client.16362 trim_cache size 0 max 16384 2024-09-21T03:49:56.196 INFO:teuthology.orchestra.run.smithi039.stdout:[] 2024-09-21T03:49:56.197 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:56.190+0000 7f6790ebb640 1 -- 172.21.15.39:0/3554839910 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f67740b6700 msgr2=0x7f67740b8c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:49:56.197 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:56.190+0000 7f6790ebb640 1 --2- 172.21.15.39:0/3554839910 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f67740b6700 0x7f67740b8c10 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f677c031440 tx=0x7f677c009610 comp rx=0 tx=0).stop 2024-09-21T03:49:56.197 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:56.190+0000 7f6790ebb640 1 -- 172.21.15.39:0/3554839910 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f678c1094a0 msgr2=0x7f678c19aa40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:49:56.197 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:56.190+0000 7f6790ebb640 1 --2- 172.21.15.39:0/3554839910 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f678c1094a0 0x7f678c19aa40 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7f678000d750 tx=0x7f678000dc10 comp rx=0 tx=0).stop 2024-09-21T03:49:56.197 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:56.190+0000 7f6790ebb640 1 -- 172.21.15.39:0/3554839910 shutdown_connections 2024-09-21T03:49:56.197 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:56.190+0000 7f6790ebb640 1 --2- 172.21.15.39:0/3554839910 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f67740b6700 0x7f67740b8c10 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:56.197 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:56.190+0000 7f6790ebb640 1 --2- 172.21.15.39:0/3554839910 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f678c109df0 0x7f678c19fce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:56.197 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:56.190+0000 7f6790ebb640 1 --2- 172.21.15.39:0/3554839910 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f678c1094a0 0x7f678c19aa40 unknown :-1 s=CLOSED pgs=701 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:56.197 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:56.190+0000 7f6790ebb640 1 --2- 172.21.15.39:0/3554839910 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f678c103b60 0x7f678c19a4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:56.197 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:56.190+0000 7f6790ebb640 1 -- 172.21.15.39:0/3554839910 >> 172.21.15.39:0/3554839910 conn(0x7f678c07b850 msgr2=0x7f678c107530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:49:56.197 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:56.190+0000 7f6790ebb640 1 -- 172.21.15.39:0/3554839910 shutdown_connections 2024-09-21T03:49:56.198 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:56.190+0000 7f6790ebb640 1 -- 172.21.15.39:0/3554839910 wait complete. 2024-09-21T03:49:57.084 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:49:56 smithi039 ceph-mon[31211]: pgmap v442: 97 pgs: 97 active+clean; 10 MiB data, 536 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 10 op/s 2024-09-21T03:49:57.119 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 51116d5c-77ca-11ef-baf5-efdc52797490 -- ceph tell mds.1:2 damage ls 2024-09-21T03:49:57.218 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:49:56 smithi161 ceph-mon[34533]: pgmap v442: 97 pgs: 97 active+clean; 10 MiB data, 536 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 10 op/s 2024-09-21T03:49:57.225 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:49:56 smithi150 ceph-mon[34289]: pgmap v442: 97 pgs: 97 active+clean; 10 MiB data, 536 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 10 op/s 2024-09-21T03:49:57.380 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/51116d5c-77ca-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:49:57.859 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:57.857+0000 7f18a5dc9640 1 -- 172.21.15.39:0/1093204650 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f18a01094a0 msgr2=0x7f18a01098b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:49:57.859 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:57.857+0000 7f18a5dc9640 1 --2- 172.21.15.39:0/1093204650 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f18a01094a0 0x7f18a01098b0 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7f1894009900 tx=0x7f189402a410 comp rx=0 tx=0).stop 2024-09-21T03:49:57.859 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:57.857+0000 7f18a5dc9640 1 -- 172.21.15.39:0/1093204650 shutdown_connections 2024-09-21T03:49:57.859 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:57.857+0000 7f18a5dc9640 1 --2- 172.21.15.39:0/1093204650 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f18a0109df0 0x7f18a0110620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:57.859 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:57.857+0000 7f18a5dc9640 1 --2- 172.21.15.39:0/1093204650 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f18a01094a0 0x7f18a01098b0 unknown :-1 s=CLOSED pgs=706 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:57.859 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:57.857+0000 7f18a5dc9640 1 --2- 172.21.15.39:0/1093204650 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f18a0103b60 0x7f18a0103f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:57.859 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:57.857+0000 7f18a5dc9640 1 -- 172.21.15.39:0/1093204650 >> 172.21.15.39:0/1093204650 conn(0x7f18a007b850 msgr2=0x7f18a007bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:49:57.859 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:57.857+0000 7f18a5dc9640 1 -- 172.21.15.39:0/1093204650 shutdown_connections 2024-09-21T03:49:57.860 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:57.857+0000 7f18a5dc9640 1 -- 172.21.15.39:0/1093204650 wait complete. 2024-09-21T03:49:57.861 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:57.858+0000 7f18a5dc9640 1 Processor -- start 2024-09-21T03:49:57.861 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:57.858+0000 7f18a5dc9640 1 -- start start 2024-09-21T03:49:57.861 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:57.858+0000 7f18a5dc9640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f18a0103b60 0x7f18a019a600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:49:57.861 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:57.858+0000 7f18a5dc9640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f18a01094a0 0x7f18a019aba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:49:57.862 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:57.858+0000 7f18a5dc9640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f18a0109df0 0x7f18a019fe40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:49:57.862 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:57.858+0000 7f18a5dc9640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f18a0113ad0 con 0x7f18a0103b60 2024-09-21T03:49:57.862 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:57.858+0000 7f18a5dc9640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f18a0113950 con 0x7f18a01094a0 2024-09-21T03:49:57.862 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:57.858+0000 7f18a5dc9640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f18a0113c50 con 0x7f18a0109df0 2024-09-21T03:49:57.862 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:57.858+0000 7f189f7fe640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f18a0103b60 0x7f18a019a600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:49:57.862 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:57.858+0000 7f189f7fe640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f18a0103b60 0x7f18a019a600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:60046/0 (socket says 172.21.15.39:60046) 2024-09-21T03:49:57.862 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:57.858+0000 7f189f7fe640 1 -- 172.21.15.39:0/3171189076 learned_addr learned my addr 172.21.15.39:0/3171189076 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:49:57.862 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:57.858+0000 7f189effd640 1 --2- 172.21.15.39:0/3171189076 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f18a01094a0 0x7f18a019aba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:49:57.862 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:57.859+0000 7f189ffff640 1 --2- 172.21.15.39:0/3171189076 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f18a0109df0 0x7f18a019fe40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:49:57.862 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:57.859+0000 7f189f7fe640 1 -- 172.21.15.39:0/3171189076 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f18a0109df0 msgr2=0x7f18a019fe40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:49:57.862 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:57.859+0000 7f189f7fe640 1 --2- 172.21.15.39:0/3171189076 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f18a0109df0 0x7f18a019fe40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:57.862 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:57.859+0000 7f189f7fe640 1 -- 172.21.15.39:0/3171189076 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f18a01094a0 msgr2=0x7f18a019aba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:49:57.862 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:57.859+0000 7f189f7fe640 1 --2- 172.21.15.39:0/3171189076 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f18a01094a0 0x7f18a019aba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:57.862 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:57.859+0000 7f189f7fe640 1 -- 172.21.15.39:0/3171189076 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f18a01a0570 con 0x7f18a0103b60 2024-09-21T03:49:57.863 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:57.859+0000 7f189f7fe640 1 --2- 172.21.15.39:0/3171189076 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f18a0103b60 0x7f18a019a600 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7f189000c890 tx=0x7f189000cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:49:57.863 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:57.859+0000 7f189cff9640 1 -- 172.21.15.39:0/3171189076 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1890004220 con 0x7f18a0103b60 2024-09-21T03:49:57.863 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:57.859+0000 7f18a5dc9640 1 -- 172.21.15.39:0/3171189076 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f18a01a0860 con 0x7f18a0103b60 2024-09-21T03:49:57.863 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:57.859+0000 7f189cff9640 1 -- 172.21.15.39:0/3171189076 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f18900043c0 con 0x7f18a0103b60 2024-09-21T03:49:57.863 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:57.859+0000 7f189cff9640 1 -- 172.21.15.39:0/3171189076 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1890004a00 con 0x7f18a0103b60 2024-09-21T03:49:57.863 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:57.859+0000 7f18a5dc9640 1 -- 172.21.15.39:0/3171189076 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f18a01a6fc0 con 0x7f18a0103b60 2024-09-21T03:49:57.865 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:57.862+0000 7f18ac33d740 1 Processor -- start 2024-09-21T03:49:57.865 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:57.862+0000 7f18ac33d740 1 -- start start 2024-09-21T03:49:57.865 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:57.862+0000 7f18ac33d740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55f3ff4b8070 0x55f3ff85f2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:49:57.865 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:57.862+0000 7f18ac33d740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f3ff48f1b0 0x55f3ff85f800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:49:57.866 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:57.862+0000 7f18ac33d740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f3ff497390 0x55f3ff85fd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:49:57.866 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:57.862+0000 7f18ac33d740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x55f3ff7d5ea0 con 0x55f3ff4b8070 2024-09-21T03:49:57.866 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:57.862+0000 7f18ac33d740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x55f3ff7c52d0 con 0x55f3ff497390 2024-09-21T03:49:57.866 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:57.862+0000 7f18ac33d740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55f3ff7c90e0 con 0x55f3ff48f1b0 2024-09-21T03:49:57.866 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:57.862+0000 7f189cff9640 1 -- 172.21.15.39:0/3171189076 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 27) ==== 154643+0+0 (secure 0 0 0) 0x7f1890004ba0 con 0x7f18a0103b60 2024-09-21T03:49:57.866 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:57.862+0000 7f189ffff640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55f3ff4b8070 0x55f3ff85f2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:49:57.866 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:57.862+0000 7f189ffff640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55f3ff4b8070 0x55f3ff85f2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:60054/0 (socket says 172.21.15.39:60054) 2024-09-21T03:49:57.866 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:57.862+0000 7f189ffff640 1 -- 172.21.15.39:0/3875841662 learned_addr learned my addr 172.21.15.39:0/3875841662 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:49:57.866 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:57.862+0000 7f189effd640 1 --2- 172.21.15.39:0/3875841662 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f3ff497390 0x55f3ff85fd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:49:57.866 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:57.863+0000 7f189f7fe640 1 --2- 172.21.15.39:0/3875841662 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f3ff48f1b0 0x55f3ff85f800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:49:57.866 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:57.863+0000 7f189ffff640 1 -- 172.21.15.39:0/3875841662 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f3ff48f1b0 msgr2=0x55f3ff85f800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:49:57.867 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:57.863+0000 7f189ffff640 1 --2- 172.21.15.39:0/3875841662 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f3ff48f1b0 0x55f3ff85f800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:57.867 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:57.863+0000 7f189ffff640 1 -- 172.21.15.39:0/3875841662 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f3ff497390 msgr2=0x55f3ff85fd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:49:57.867 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:57.863+0000 7f189ffff640 1 --2- 172.21.15.39:0/3875841662 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f3ff497390 0x55f3ff85fd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:57.867 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:57.863+0000 7f189ffff640 1 -- 172.21.15.39:0/3875841662 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f3ff523650 con 0x55f3ff4b8070 2024-09-21T03:49:57.867 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:57.863+0000 7f189ffff640 1 --2- 172.21.15.39:0/3875841662 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55f3ff4b8070 0x55f3ff85f2c0 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7f1898009f20 tx=0x7f189800c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3039fafc1c32359c server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:49:57.867 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:57.863+0000 7f1885ffb640 1 -- 172.21.15.39:0/3875841662 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f189801a070 con 0x55f3ff4b8070 2024-09-21T03:49:57.867 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:57.863+0000 7f1885ffb640 1 -- 172.21.15.39:0/3875841662 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f18980158c0 con 0x55f3ff4b8070 2024-09-21T03:49:57.867 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:57.863+0000 7f1885ffb640 1 -- 172.21.15.39:0/3875841662 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f189800a7c0 con 0x55f3ff4b8070 2024-09-21T03:49:57.867 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:57.863+0000 7f189cff9640 1 --2- 172.21.15.39:0/3171189076 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f18880b6680 0x7f18880b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:49:57.867 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:57.864+0000 7f189f7fe640 1 --2- 172.21.15.39:0/3171189076 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f18880b6680 0x7f18880b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:49:57.867 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:57.864+0000 7f189cff9640 1 -- 172.21.15.39:0/3171189076 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8484+0+0 (secure 0 0 0) 0x7f18900e74e0 con 0x7f18a0103b60 2024-09-21T03:49:57.872 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:57.864+0000 7f18ac33d740 1 -- 172.21.15.39:0/3875841662 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55f3ff4b8070 msgr2=0x55f3ff85f2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:49:57.872 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:57.864+0000 7f18ac33d740 1 --2- 172.21.15.39:0/3875841662 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55f3ff4b8070 0x55f3ff85f2c0 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7f1898009f20 tx=0x7f189800c8b0 comp rx=0 tx=0).stop 2024-09-21T03:49:57.872 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:57.864+0000 7f189f7fe640 1 --2- 172.21.15.39:0/3171189076 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f18880b6680 0x7f18880b8b90 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f18900e4d00 tx=0x7f1890012000 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:49:57.872 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:57.864+0000 7f18ac33d740 1 -- 172.21.15.39:0/3875841662 shutdown_connections 2024-09-21T03:49:57.872 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:57.864+0000 7f18ac33d740 1 --2- 172.21.15.39:0/3875841662 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f3ff497390 0x55f3ff85fd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:57.872 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:57.864+0000 7f18ac33d740 1 --2- 172.21.15.39:0/3875841662 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f3ff48f1b0 0x55f3ff85f800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:57.872 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:57.864+0000 7f18ac33d740 1 --2- 172.21.15.39:0/3875841662 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55f3ff4b8070 0x55f3ff85f2c0 unknown :-1 s=CLOSED pgs=708 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:57.872 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:57.864+0000 7f18ac33d740 1 -- 172.21.15.39:0/3875841662 >> 172.21.15.39:0/3875841662 conn(0x55f3ff4ac460 msgr2=0x55f3ff85d1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:49:57.872 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:57.864+0000 7f18ac33d740 1 -- 172.21.15.39:0/3875841662 shutdown_connections 2024-09-21T03:49:57.872 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:57.864+0000 7f18ac33d740 1 -- 172.21.15.39:0/3875841662 wait complete. 2024-09-21T03:49:57.873 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:57.865+0000 7f18ac33d740 1 -- start start 2024-09-21T03:49:57.873 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:57.865+0000 7f18ac33d740 1 Processor -- start 2024-09-21T03:49:57.873 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:57.865+0000 7f18ac33d740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f3ff48f1b0 0x55f3ff86edc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:49:57.873 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:57.865+0000 7f18ac33d740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55f3ff497390 0x55f3ff86f300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:49:57.873 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:57.865+0000 7f18ac33d740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f3ff4b8070 0x55f3ff87b130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:49:57.873 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:57.865+0000 7f18ac33d740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x55f3ff626700 con 0x55f3ff497390 2024-09-21T03:49:57.873 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:57.865+0000 7f18ac33d740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x55f3ff760160 con 0x55f3ff4b8070 2024-09-21T03:49:57.873 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:57.865+0000 7f18ac33d740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55f3ff871e00 con 0x55f3ff48f1b0 2024-09-21T03:49:57.873 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:57.865+0000 7f18ac33d740 10 client.0 _ll_register_callbacks cb 0x55f3ff7799e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:49:57.873 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:57.865+0000 7f189effd640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55f3ff497390 0x55f3ff86f300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:49:57.873 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:57.866+0000 7f189effd640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55f3ff497390 0x55f3ff86f300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:60060/0 (socket says 172.21.15.39:60060) 2024-09-21T03:49:57.873 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:57.866+0000 7f189effd640 1 -- 172.21.15.39:0/3179960567 learned_addr learned my addr 172.21.15.39:0/3179960567 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:49:57.873 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:57.866+0000 7f189ffff640 1 --2- 172.21.15.39:0/3179960567 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f3ff4b8070 0x55f3ff87b130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:49:57.873 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:57.866+0000 7f189ffff640 1 --2- 172.21.15.39:0/3179960567 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f3ff48f1b0 0x55f3ff86edc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:49:57.873 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:57.866+0000 7f189effd640 1 -- 172.21.15.39:0/3179960567 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f3ff48f1b0 msgr2=0x55f3ff86edc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:49:57.874 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:57.866+0000 7f189effd640 1 --2- 172.21.15.39:0/3179960567 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f3ff48f1b0 0x55f3ff86edc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:57.874 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:57.866+0000 7f189effd640 1 -- 172.21.15.39:0/3179960567 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f3ff4b8070 msgr2=0x55f3ff87b130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:49:57.874 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:57.866+0000 7f189effd640 1 --2- 172.21.15.39:0/3179960567 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f3ff4b8070 0x55f3ff87b130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:57.874 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:57.866+0000 7f189effd640 1 -- 172.21.15.39:0/3179960567 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f3ff78dc90 con 0x55f3ff497390 2024-09-21T03:49:57.874 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:57.866+0000 7f189ffff640 1 --2- 172.21.15.39:0/3179960567 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f3ff4b8070 0x55f3ff87b130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:49:57.874 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:57.866+0000 7f189effd640 1 --2- 172.21.15.39:0/3179960567 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55f3ff497390 0x55f3ff86f300 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7f1894009900 tx=0x7f18940316d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=47cd01fda9489c06 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:49:57.874 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:57.866+0000 7f187effd640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-09-21T03:49:57.874 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:57.866+0000 7f187effd640 1 -- 172.21.15.39:0/3179960567 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1894035040 con 0x55f3ff497390 2024-09-21T03:49:57.874 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:57.866+0000 7f187effd640 1 -- 172.21.15.39:0/3179960567 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8484+0+0 (secure 0 0 0) 0x7f1894004040 con 0x55f3ff497390 2024-09-21T03:49:57.874 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:57.866+0000 7f18ac33d740 1 -- 172.21.15.39:0/3179960567 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f18980158c0 con 0x55f3ff497390 2024-09-21T03:49:57.874 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:57.867+0000 7f187effd640 1 -- 172.21.15.39:0/3179960567 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f1894004240 con 0x55f3ff497390 2024-09-21T03:49:57.874 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:57.867+0000 7f18ac33d740 10 client.16374 fetch_fsmap learned FSMap version 29 2024-09-21T03:49:57.874 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:57.867+0000 7f18ac33d740 1 -- 172.21.15.39:0/3179960567 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55f3ff87db50 con 0x55f3ff497390 2024-09-21T03:49:57.874 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:57.867+0000 7f187effd640 1 -- 172.21.15.39:0/3179960567 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f18980158c0 con 0x55f3ff497390 2024-09-21T03:49:57.874 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:57.867+0000 7f187effd640 1 -- 172.21.15.39:0/3179960567 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f189404d020 con 0x55f3ff497390 2024-09-21T03:49:57.875 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:57.867+0000 7f18ac33d740 10 client.16374 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T03:49:57.875 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:57.867+0000 7f18ac33d740 10 client.16374 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-21T03:49:57.875 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:57.867+0000 7f18ac33d740 20 client.16374 populate_metadata read hostname 'smithi039' 2024-09-21T03:49:57.875 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:57.867+0000 7f18ac33d740 1 --2- 172.21.15.39:0/3179960567 >> [v2:172.21.15.150:6836/4017043144,v1:172.21.15.150:6837/4017043144] conn(0x55f3ff4769f0 0x55f3ff881030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:49:57.875 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:57.867+0000 7f18ac33d740 4 client.16374 mds_command: new command op to 24659 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:49:57.875 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:57.867+0000 7f18ac33d740 1 -- 172.21.15.39:0/3179960567 --> [v2:172.21.15.150:6836/4017043144,v1:172.21.15.150:6837/4017043144] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f3ff7d7df0 con 0x55f3ff4769f0 2024-09-21T03:49:57.875 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:57.867+0000 7f189ffff640 1 --2- 172.21.15.39:0/3179960567 >> [v2:172.21.15.150:6836/4017043144,v1:172.21.15.150:6837/4017043144] conn(0x55f3ff4769f0 0x55f3ff881030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:49:57.875 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:57.868+0000 7f189ffff640 1 --2- 172.21.15.39:0/3179960567 >> [v2:172.21.15.150:6836/4017043144,v1:172.21.15.150:6837/4017043144] conn(0x55f3ff4769f0 0x55f3ff881030 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=c5136cb1683f58cd server_cookie=e9bcae589dc570f9 in_seq=0 out_seq=0 2024-09-21T03:49:57.875 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:57.868+0000 7f187effd640 10 client.16374 ms_handle_connect on v2:172.21.15.150:6836/4017043144 2024-09-21T03:49:57.875 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:57.869+0000 7f187effd640 1 -- 172.21.15.39:0/3179960567 <== mds.2 v2:172.21.15.150:6836/4017043144 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55f3ff7d7df0 con 0x55f3ff4769f0 2024-09-21T03:49:57.875 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:57.869+0000 7f187effd640 10 client.16374 handle_command_reply: tid=0 2024-09-21T03:49:57.875 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:57.869+0000 7f18ac33d740 1 client.16374 shutdown 2024-09-21T03:49:57.875 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:57.869+0000 7f18ac33d740 1 -- 172.21.15.39:0/3179960567 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55f3ff497390 msgr2=0x55f3ff86f300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:49:57.875 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:57.869+0000 7f18ac33d740 1 --2- 172.21.15.39:0/3179960567 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55f3ff497390 0x55f3ff86f300 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7f1894009900 tx=0x7f18940316d0 comp rx=0 tx=0).stop 2024-09-21T03:49:57.875 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:57.869+0000 7f18ac33d740 1 -- 172.21.15.39:0/3179960567 shutdown_connections 2024-09-21T03:49:57.876 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:57.869+0000 7f18ac33d740 1 --2- 172.21.15.39:0/3179960567 >> [v2:172.21.15.150:6836/4017043144,v1:172.21.15.150:6837/4017043144] conn(0x55f3ff4769f0 0x55f3ff881030 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:57.876 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:57.869+0000 7f18ac33d740 1 --2- 172.21.15.39:0/3179960567 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f3ff4b8070 0x55f3ff87b130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:57.876 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:57.869+0000 7f18ac33d740 1 --2- 172.21.15.39:0/3179960567 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55f3ff497390 0x55f3ff86f300 unknown :-1 s=CLOSED pgs=709 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:57.876 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:57.869+0000 7f18ac33d740 1 --2- 172.21.15.39:0/3179960567 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f3ff48f1b0 0x55f3ff86edc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:57.876 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:57.869+0000 7f18ac33d740 1 -- 172.21.15.39:0/3179960567 >> 172.21.15.39:0/3179960567 conn(0x55f3ff4ac460 msgr2=0x55f3ff85b750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:49:57.876 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:57.869+0000 7f187effd640 0 client.16374 ms_handle_reset on v2:172.21.15.150:6836/4017043144 2024-09-21T03:49:57.876 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:57.869+0000 7f18ac33d740 1 -- 172.21.15.39:0/3179960567 shutdown_connections 2024-09-21T03:49:57.876 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:57.869+0000 7f18ac33d740 1 -- 172.21.15.39:0/3179960567 wait complete. 2024-09-21T03:49:57.876 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:57.869+0000 7f18ac33d740 20 client.16374 trim_cache size 0 max 16384 2024-09-21T03:49:57.885 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:57.882+0000 7f18ac33d740 1 Processor -- start 2024-09-21T03:49:57.886 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:57.883+0000 7f18ac33d740 1 -- start start 2024-09-21T03:49:57.886 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:57.883+0000 7f18ac33d740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f3ff48f1b0 0x55f3ff864b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:49:57.886 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:57.883+0000 7f18ac33d740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55f3ff497390 0x55f3ff87b030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:49:57.886 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:57.883+0000 7f18ac33d740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f3ff4b8070 0x55f3ff86bfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:49:57.886 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:57.883+0000 7f18ac33d740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f18a01ad6c0 con 0x55f3ff497390 2024-09-21T03:49:57.886 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:57.883+0000 7f18ac33d740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f18a01af200 con 0x55f3ff48f1b0 2024-09-21T03:49:57.886 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:57.883+0000 7f18ac33d740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f18a01add90 con 0x55f3ff4b8070 2024-09-21T03:49:57.886 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:57.883+0000 7f189effd640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55f3ff497390 0x55f3ff87b030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:49:57.886 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:57.883+0000 7f189effd640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55f3ff497390 0x55f3ff87b030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:60074/0 (socket says 172.21.15.39:60074) 2024-09-21T03:49:57.886 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:57.883+0000 7f189effd640 1 -- 172.21.15.39:0/69906764 learned_addr learned my addr 172.21.15.39:0/69906764 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:49:57.886 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:57.883+0000 7f189ffff640 1 --2- 172.21.15.39:0/69906764 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f3ff4b8070 0x55f3ff86bfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:49:57.887 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:57.883+0000 7f189ffff640 1 --2- 172.21.15.39:0/69906764 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f3ff48f1b0 0x55f3ff864b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:49:57.887 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:57.883+0000 7f189effd640 1 -- 172.21.15.39:0/69906764 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f3ff4b8070 msgr2=0x55f3ff86bfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:49:57.887 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:57.883+0000 7f189effd640 1 --2- 172.21.15.39:0/69906764 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f3ff4b8070 0x55f3ff86bfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:57.887 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:57.883+0000 7f189effd640 1 -- 172.21.15.39:0/69906764 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f3ff48f1b0 msgr2=0x55f3ff864b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:49:57.887 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:57.883+0000 7f189effd640 1 --2- 172.21.15.39:0/69906764 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f3ff48f1b0 0x55f3ff864b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:57.887 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:57.883+0000 7f189effd640 1 -- 172.21.15.39:0/69906764 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f3ff86fc50 con 0x55f3ff497390 2024-09-21T03:49:57.887 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:57.883+0000 7f189ffff640 1 --2- 172.21.15.39:0/69906764 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f3ff4b8070 0x55f3ff86bfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:49:57.887 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:57.883+0000 7f189effd640 1 --2- 172.21.15.39:0/69906764 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55f3ff497390 0x55f3ff87b030 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7f189402ab90 tx=0x7f189402a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=41c5aeb72b8d6ff6 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:49:57.887 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:57.884+0000 7f18857fa640 1 -- 172.21.15.39:0/69906764 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1894035070 con 0x55f3ff497390 2024-09-21T03:49:57.887 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:57.884+0000 7f18857fa640 1 -- 172.21.15.39:0/69906764 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1894054070 con 0x55f3ff497390 2024-09-21T03:49:57.887 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:57.884+0000 7f18857fa640 1 -- 172.21.15.39:0/69906764 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f189403d450 con 0x55f3ff497390 2024-09-21T03:49:57.891 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:57.884+0000 7f18ac33d740 1 -- 172.21.15.39:0/69906764 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55f3ff497390 msgr2=0x55f3ff87b030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:49:57.892 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:57.884+0000 7f18ac33d740 1 --2- 172.21.15.39:0/69906764 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55f3ff497390 0x55f3ff87b030 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7f189402ab90 tx=0x7f189402a410 comp rx=0 tx=0).stop 2024-09-21T03:49:57.892 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:57.884+0000 7f18ac33d740 1 -- 172.21.15.39:0/69906764 shutdown_connections 2024-09-21T03:49:57.892 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:57.884+0000 7f18ac33d740 1 --2- 172.21.15.39:0/69906764 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f3ff4b8070 0x55f3ff86bfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:57.892 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:57.884+0000 7f18ac33d740 1 --2- 172.21.15.39:0/69906764 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55f3ff497390 0x55f3ff87b030 unknown :-1 s=CLOSED pgs=710 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:57.892 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:57.884+0000 7f18ac33d740 1 --2- 172.21.15.39:0/69906764 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f3ff48f1b0 0x55f3ff864b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:57.892 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:57.884+0000 7f18ac33d740 1 -- 172.21.15.39:0/69906764 >> 172.21.15.39:0/69906764 conn(0x55f3ff4ac460 msgr2=0x55f3ff864460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:49:57.892 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:57.885+0000 7f18ac33d740 1 -- 172.21.15.39:0/69906764 shutdown_connections 2024-09-21T03:49:57.892 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:57.885+0000 7f18ac33d740 1 -- 172.21.15.39:0/69906764 wait complete. 2024-09-21T03:49:57.892 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:57.885+0000 7f18ac33d740 1 -- start start 2024-09-21T03:49:57.892 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:57.885+0000 7f18ac33d740 1 Processor -- start 2024-09-21T03:49:57.892 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:57.885+0000 7f18ac33d740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f3ff48f1b0 0x55f3ff8776b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:49:57.892 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:57.885+0000 7f18ac33d740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55f3ff497390 0x55f3ff872b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:49:57.892 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:57.885+0000 7f18ac33d740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f3ff4b8070 0x55f3ff8730a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:49:57.893 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:57.885+0000 7f18ac33d740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f18a01aeb30 con 0x55f3ff497390 2024-09-21T03:49:57.893 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:57.885+0000 7f18ac33d740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f18a01ae460 con 0x55f3ff4b8070 2024-09-21T03:49:57.893 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:57.886+0000 7f18ac33d740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f18a01ab6a0 con 0x55f3ff48f1b0 2024-09-21T03:49:57.893 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:57.886+0000 7f18ac33d740 10 client.0 _ll_register_callbacks cb 0x7f1894031b10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:49:57.893 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:57.886+0000 7f189effd640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55f3ff497390 0x55f3ff872b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:49:57.893 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:57.886+0000 7f189effd640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55f3ff497390 0x55f3ff872b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:60088/0 (socket says 172.21.15.39:60088) 2024-09-21T03:49:57.893 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:57.886+0000 7f189effd640 1 -- 172.21.15.39:0/387034175 learned_addr learned my addr 172.21.15.39:0/387034175 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:49:57.893 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:57.886+0000 7f189ffff640 1 --2- 172.21.15.39:0/387034175 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f3ff48f1b0 0x55f3ff8776b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:49:57.893 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:57.886+0000 7f189ffff640 1 --2- 172.21.15.39:0/387034175 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f3ff4b8070 0x55f3ff8730a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:49:57.893 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:57.886+0000 7f189effd640 1 -- 172.21.15.39:0/387034175 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f3ff48f1b0 msgr2=0x55f3ff8776b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:49:57.893 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:57.886+0000 7f189effd640 1 --2- 172.21.15.39:0/387034175 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f3ff48f1b0 0x55f3ff8776b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:57.893 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:57.886+0000 7f189effd640 1 -- 172.21.15.39:0/387034175 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f3ff4b8070 msgr2=0x55f3ff8730a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:49:57.893 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:57.886+0000 7f189effd640 1 --2- 172.21.15.39:0/387034175 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f3ff4b8070 0x55f3ff8730a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:57.893 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:57.886+0000 7f189effd640 1 -- 172.21.15.39:0/387034175 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f3ff873660 con 0x55f3ff497390 2024-09-21T03:49:57.893 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:57.886+0000 7f189effd640 1 --2- 172.21.15.39:0/387034175 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55f3ff497390 0x55f3ff872b60 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7f18940092a0 tx=0x7f18940469e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dbf6ce6960fb191d server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:49:57.894 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:57.886+0000 7f187effd640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-09-21T03:49:57.894 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:57.886+0000 7f187effd640 1 -- 172.21.15.39:0/387034175 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1894035040 con 0x55f3ff497390 2024-09-21T03:49:57.894 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:57.886+0000 7f18ac33d740 1 -- 172.21.15.39:0/387034175 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1894054070 con 0x55f3ff497390 2024-09-21T03:49:57.894 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:57.886+0000 7f187effd640 1 -- 172.21.15.39:0/387034175 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8484+0+0 (secure 0 0 0) 0x7f18940318e0 con 0x55f3ff497390 2024-09-21T03:49:57.894 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:57.887+0000 7f187effd640 1 -- 172.21.15.39:0/387034175 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f189404fb50 con 0x55f3ff497390 2024-09-21T03:49:57.894 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:57.887+0000 7f18ac33d740 10 client.16380 fetch_fsmap learned FSMap version 29 2024-09-21T03:49:57.894 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:57.887+0000 7f18ac33d740 1 -- 172.21.15.39:0/387034175 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55f3ff8eb7b0 con 0x55f3ff497390 2024-09-21T03:49:57.894 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:57.887+0000 7f187effd640 1 -- 172.21.15.39:0/387034175 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1894054070 con 0x55f3ff497390 2024-09-21T03:49:57.894 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:57.887+0000 7f187effd640 1 -- 172.21.15.39:0/387034175 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f189404d040 con 0x55f3ff497390 2024-09-21T03:49:57.894 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:57.887+0000 7f18ac33d740 10 client.16380 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T03:49:57.894 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:57.887+0000 7f18ac33d740 10 client.16380 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-21T03:49:57.894 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:57.887+0000 7f18ac33d740 20 client.16380 populate_metadata read hostname 'smithi039' 2024-09-21T03:49:57.894 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:57.887+0000 7f18ac33d740 1 --2- 172.21.15.39:0/387034175 >> [v2:172.21.15.150:6836/4017043144,v1:172.21.15.150:6837/4017043144] conn(0x55f3ff44bda0 0x55f3ff8edec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:49:57.894 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:57.887+0000 7f18ac33d740 4 client.16380 mds_command: new command op to 24659 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T03:49:57.895 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:57.887+0000 7f18ac33d740 1 -- 172.21.15.39:0/387034175 --> [v2:172.21.15.150:6836/4017043144,v1:172.21.15.150:6837/4017043144] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55f3ff7d38a0 con 0x55f3ff44bda0 2024-09-21T03:49:57.895 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:57.887+0000 7f189ffff640 1 --2- 172.21.15.39:0/387034175 >> [v2:172.21.15.150:6836/4017043144,v1:172.21.15.150:6837/4017043144] conn(0x55f3ff44bda0 0x55f3ff8edec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:49:57.895 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:57.888+0000 7f187effd640 10 client.16380 ms_handle_connect on v2:172.21.15.150:6836/4017043144 2024-09-21T03:49:57.895 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:57.888+0000 7f189ffff640 1 --2- 172.21.15.39:0/387034175 >> [v2:172.21.15.150:6836/4017043144,v1:172.21.15.150:6837/4017043144] conn(0x55f3ff44bda0 0x55f3ff8edec0 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=bfac564c17fce96a server_cookie=3be5dfd771febb88 in_seq=0 out_seq=0 2024-09-21T03:49:57.895 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:57.888+0000 7f187effd640 1 -- 172.21.15.39:0/387034175 <== mds.2 v2:172.21.15.150:6836/4017043144 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55f3ff7d38a0 con 0x55f3ff44bda0 2024-09-21T03:49:57.895 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:57.888+0000 7f187effd640 10 client.16380 handle_command_reply: tid=0 2024-09-21T03:49:57.895 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:57.888+0000 7f18ac33d740 1 client.16380 shutdown 2024-09-21T03:49:57.895 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:57.888+0000 7f18ac33d740 1 -- 172.21.15.39:0/387034175 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55f3ff497390 msgr2=0x55f3ff872b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:49:57.895 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:57.888+0000 7f18ac33d740 1 --2- 172.21.15.39:0/387034175 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55f3ff497390 0x55f3ff872b60 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7f18940092a0 tx=0x7f18940469e0 comp rx=0 tx=0).stop 2024-09-21T03:49:57.895 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:57.888+0000 7f18ac33d740 1 -- 172.21.15.39:0/387034175 shutdown_connections 2024-09-21T03:49:57.895 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:57.888+0000 7f18ac33d740 1 --2- 172.21.15.39:0/387034175 >> [v2:172.21.15.150:6836/4017043144,v1:172.21.15.150:6837/4017043144] conn(0x55f3ff44bda0 0x55f3ff8edec0 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:57.895 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:57.888+0000 7f18ac33d740 1 --2- 172.21.15.39:0/387034175 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f3ff4b8070 0x55f3ff8730a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:57.895 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:57.889+0000 7f187effd640 0 client.16380 ms_handle_reset on v2:172.21.15.150:6836/4017043144 2024-09-21T03:49:57.895 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:57.889+0000 7f18ac33d740 1 --2- 172.21.15.39:0/387034175 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55f3ff497390 0x55f3ff872b60 unknown :-1 s=CLOSED pgs=711 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:57.896 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:57.889+0000 7f18ac33d740 1 --2- 172.21.15.39:0/387034175 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f3ff48f1b0 0x55f3ff8776b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:57.896 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:57.889+0000 7f18ac33d740 1 -- 172.21.15.39:0/387034175 >> 172.21.15.39:0/387034175 conn(0x55f3ff4ac460 msgr2=0x55f3ff864bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:49:57.896 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:57.889+0000 7f18ac33d740 1 -- 172.21.15.39:0/387034175 shutdown_connections 2024-09-21T03:49:57.896 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:57.889+0000 7f18ac33d740 1 -- 172.21.15.39:0/387034175 wait complete. 2024-09-21T03:49:57.896 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:57.889+0000 7f18ac33d740 20 client.16380 trim_cache size 0 max 16384 2024-09-21T03:49:57.896 INFO:teuthology.orchestra.run.smithi039.stdout:[] 2024-09-21T03:49:57.897 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:57.889+0000 7f18a5dc9640 1 -- 172.21.15.39:0/3171189076 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f18880b6680 msgr2=0x7f18880b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:49:57.897 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:57.889+0000 7f18a5dc9640 1 --2- 172.21.15.39:0/3171189076 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f18880b6680 0x7f18880b8b90 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f18900e4d00 tx=0x7f1890012000 comp rx=0 tx=0).stop 2024-09-21T03:49:57.897 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:57.889+0000 7f18a5dc9640 1 -- 172.21.15.39:0/3171189076 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f18a0103b60 msgr2=0x7f18a019a600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:49:57.897 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:57.889+0000 7f18a5dc9640 1 --2- 172.21.15.39:0/3171189076 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f18a0103b60 0x7f18a019a600 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7f189000c890 tx=0x7f189000cd50 comp rx=0 tx=0).stop 2024-09-21T03:49:57.897 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:57.889+0000 7f18a5dc9640 1 -- 172.21.15.39:0/3171189076 shutdown_connections 2024-09-21T03:49:57.897 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:57.889+0000 7f18a5dc9640 1 --2- 172.21.15.39:0/3171189076 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f18880b6680 0x7f18880b8b90 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:57.897 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:57.889+0000 7f18a5dc9640 1 --2- 172.21.15.39:0/3171189076 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f18a0109df0 0x7f18a019fe40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:57.897 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:57.889+0000 7f18a5dc9640 1 --2- 172.21.15.39:0/3171189076 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f18a01094a0 0x7f18a019aba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:57.897 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:57.889+0000 7f18a5dc9640 1 --2- 172.21.15.39:0/3171189076 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f18a0103b60 0x7f18a019a600 unknown :-1 s=CLOSED pgs=707 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:57.897 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:57.889+0000 7f18a5dc9640 1 -- 172.21.15.39:0/3171189076 >> 172.21.15.39:0/3171189076 conn(0x7f18a007b850 msgr2=0x7f18a0107600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:49:57.897 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:57.890+0000 7f18a5dc9640 1 -- 172.21.15.39:0/3171189076 shutdown_connections 2024-09-21T03:49:57.897 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:57.890+0000 7f18a5dc9640 1 -- 172.21.15.39:0/3171189076 wait complete. 2024-09-21T03:49:58.690 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 51116d5c-77ca-11ef-baf5-efdc52797490 -- ceph tell mds.1:1 damage ls 2024-09-21T03:49:58.951 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/51116d5c-77ca-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:49:59.072 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:49:58 smithi039 ceph-mon[31211]: pgmap v443: 97 pgs: 97 active+clean; 10 MiB data, 536 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-09-21T03:49:59.218 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:49:58 smithi161 ceph-mon[34533]: pgmap v443: 97 pgs: 97 active+clean; 10 MiB data, 536 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-09-21T03:49:59.225 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:49:58 smithi150 ceph-mon[34289]: pgmap v443: 97 pgs: 97 active+clean; 10 MiB data, 536 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-09-21T03:49:59.431 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:59.406+0000 7f3d69254640 1 -- 172.21.15.39:0/851961622 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3d641083a0 msgr2=0x7f3d641087b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:49:59.431 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:59.406+0000 7f3d69254640 1 --2- 172.21.15.39:0/851961622 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3d641083a0 0x7f3d641087b0 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7f3d4c009900 tx=0x7f3d4c02a410 comp rx=0 tx=0).stop 2024-09-21T03:49:59.432 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:59.406+0000 7f3d69254640 1 -- 172.21.15.39:0/851961622 shutdown_connections 2024-09-21T03:49:59.432 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:59.406+0000 7f3d69254640 1 --2- 172.21.15.39:0/851961622 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3d64108cf0 0x7f3d6410f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:59.432 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:59.406+0000 7f3d69254640 1 --2- 172.21.15.39:0/851961622 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3d641083a0 0x7f3d641087b0 unknown :-1 s=CLOSED pgs=712 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:59.432 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:59.406+0000 7f3d69254640 1 --2- 172.21.15.39:0/851961622 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3d64102a60 0x7f3d64102e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:59.432 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:59.406+0000 7f3d69254640 1 -- 172.21.15.39:0/851961622 >> 172.21.15.39:0/851961622 conn(0x7f3d64079730 msgr2=0x7f3d64079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:49:59.432 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:59.406+0000 7f3d69254640 1 -- 172.21.15.39:0/851961622 shutdown_connections 2024-09-21T03:49:59.432 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:59.406+0000 7f3d69254640 1 -- 172.21.15.39:0/851961622 wait complete. 2024-09-21T03:49:59.432 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:59.407+0000 7f3d69254640 1 Processor -- start 2024-09-21T03:49:59.432 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:59.407+0000 7f3d69254640 1 -- start start 2024-09-21T03:49:59.432 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:59.408+0000 7f3d69254640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3d64102a60 0x7f3d6419b2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:49:59.432 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:59.408+0000 7f3d69254640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3d641083a0 0x7f3d6419b830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:49:59.432 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:59.408+0000 7f3d69254640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3d64108cf0 0x7f3d6419bd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:49:59.432 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:59.408+0000 7f3d69254640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f3d64112960 con 0x7f3d641083a0 2024-09-21T03:49:59.432 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:59.408+0000 7f3d69254640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f3d641127e0 con 0x7f3d64108cf0 2024-09-21T03:49:59.433 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:59.408+0000 7f3d69254640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f3d64112ae0 con 0x7f3d64102a60 2024-09-21T03:49:59.433 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:59.408+0000 7f3d6259c640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3d641083a0 0x7f3d6419b830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:49:59.433 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:59.408+0000 7f3d6259c640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3d641083a0 0x7f3d6419b830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:60098/0 (socket says 172.21.15.39:60098) 2024-09-21T03:49:59.433 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:59.408+0000 7f3d6259c640 1 -- 172.21.15.39:0/2814574962 learned_addr learned my addr 172.21.15.39:0/2814574962 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:49:59.433 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:59.408+0000 7f3d62d9d640 1 --2- 172.21.15.39:0/2814574962 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3d64102a60 0x7f3d6419b2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:49:59.433 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:59.408+0000 7f3d6359e640 1 --2- 172.21.15.39:0/2814574962 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3d64108cf0 0x7f3d6419bd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:49:59.433 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:59.408+0000 7f3d6259c640 1 -- 172.21.15.39:0/2814574962 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3d64102a60 msgr2=0x7f3d6419b2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:49:59.433 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:59.408+0000 7f3d6259c640 1 --2- 172.21.15.39:0/2814574962 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3d64102a60 0x7f3d6419b2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:59.433 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:59.408+0000 7f3d6259c640 1 -- 172.21.15.39:0/2814574962 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3d64108cf0 msgr2=0x7f3d6419bd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:49:59.433 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:59.408+0000 7f3d6259c640 1 --2- 172.21.15.39:0/2814574962 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3d64108cf0 0x7f3d6419bd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:59.433 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:59.408+0000 7f3d6259c640 1 -- 172.21.15.39:0/2814574962 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3d641a6570 con 0x7f3d641083a0 2024-09-21T03:49:59.433 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:59.408+0000 7f3d6359e640 1 --2- 172.21.15.39:0/2814574962 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3d64108cf0 0x7f3d6419bd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:49:59.433 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:59.408+0000 7f3d6259c640 1 --2- 172.21.15.39:0/2814574962 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3d641083a0 0x7f3d6419b830 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7f3d4c032fd0 tx=0x7f3d4c004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:49:59.433 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:59.408+0000 7f3d5bfff640 1 -- 172.21.15.39:0/2814574962 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d4c03f030 con 0x7f3d641083a0 2024-09-21T03:49:59.434 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:59.409+0000 7f3d69254640 1 -- 172.21.15.39:0/2814574962 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3d641a6800 con 0x7f3d641083a0 2024-09-21T03:49:59.434 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:59.409+0000 7f3d5bfff640 1 -- 172.21.15.39:0/2814574962 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3d4c004490 con 0x7f3d641083a0 2024-09-21T03:49:59.434 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:59.409+0000 7f3d5bfff640 1 -- 172.21.15.39:0/2814574962 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d4c035070 con 0x7f3d641083a0 2024-09-21T03:49:59.434 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:59.409+0000 7f3d69254640 1 -- 172.21.15.39:0/2814574962 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3d641a6ce0 con 0x7f3d641083a0 2024-09-21T03:49:59.434 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:59.412+0000 7f3d6f7c8740 1 Processor -- start 2024-09-21T03:49:59.434 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:59.412+0000 7f3d6f7c8740 1 -- start start 2024-09-21T03:49:59.434 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:59.412+0000 7f3d6f7c8740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55906faa5070 0x55906fe4c6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:49:59.434 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:59.412+0000 7f3d6f7c8740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55906fa7c1b0 0x55906fe4cbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:49:59.434 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:59.412+0000 7f3d6f7c8740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55906fa84390 0x55906fe4d120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:49:59.434 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:59.412+0000 7f3d6f7c8740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x55906fdc2ea0 con 0x55906fa7c1b0 2024-09-21T03:49:59.434 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:59.412+0000 7f3d6f7c8740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x55906fdb22d0 con 0x55906faa5070 2024-09-21T03:49:59.434 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:59.412+0000 7f3d6f7c8740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55906fdb60e0 con 0x55906fa84390 2024-09-21T03:49:59.434 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:59.412+0000 7f3d5bfff640 1 -- 172.21.15.39:0/2814574962 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 27) ==== 154643+0+0 (secure 0 0 0) 0x7f3d4c042030 con 0x7f3d641083a0 2024-09-21T03:49:59.435 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:59.412+0000 7f3d62d9d640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55906fa7c1b0 0x55906fe4cbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:49:59.435 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:59.412+0000 7f3d62d9d640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55906fa7c1b0 0x55906fe4cbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:60110/0 (socket says 172.21.15.39:60110) 2024-09-21T03:49:59.435 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:59.413+0000 7f3d62d9d640 1 -- 172.21.15.39:0/3169390113 learned_addr learned my addr 172.21.15.39:0/3169390113 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:49:59.435 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:59.413+0000 7f3d6359e640 1 --2- 172.21.15.39:0/3169390113 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55906faa5070 0x55906fe4c6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:49:59.435 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:59.413+0000 7f3d62d9d640 1 -- 172.21.15.39:0/3169390113 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55906fa84390 msgr2=0x55906fe4d120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:49:59.435 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:59.413+0000 7f3d6259c640 1 --2- 172.21.15.39:0/3169390113 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55906fa84390 0x55906fe4d120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:49:59.435 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:59.413+0000 7f3d62d9d640 1 --2- 172.21.15.39:0/3169390113 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55906fa84390 0x55906fe4d120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:59.435 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:59.413+0000 7f3d62d9d640 1 -- 172.21.15.39:0/3169390113 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55906faa5070 msgr2=0x55906fe4c6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:49:59.435 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:59.413+0000 7f3d62d9d640 1 --2- 172.21.15.39:0/3169390113 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55906faa5070 0x55906fe4c6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:59.435 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:59.413+0000 7f3d62d9d640 1 -- 172.21.15.39:0/3169390113 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55906fb10650 con 0x55906fa7c1b0 2024-09-21T03:49:59.435 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:59.413+0000 7f3d6259c640 1 --2- 172.21.15.39:0/3169390113 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55906fa84390 0x55906fe4d120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:49:59.435 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:59.413+0000 7f3d6359e640 1 --2- 172.21.15.39:0/3169390113 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55906faa5070 0x55906fe4c6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:49:59.435 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:59.413+0000 7f3d62d9d640 1 --2- 172.21.15.39:0/3169390113 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55906fa7c1b0 0x55906fe4cbe0 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7f3d54009f50 tx=0x7f3d54010800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=178ad8bcdb0bb76f server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:49:59.435 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:59.413+0000 7f3d597fa640 1 -- 172.21.15.39:0/3169390113 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d54016070 con 0x55906fa7c1b0 2024-09-21T03:49:59.436 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:59.413+0000 7f3d597fa640 1 -- 172.21.15.39:0/3169390113 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3d5401bb10 con 0x55906fa7c1b0 2024-09-21T03:49:59.436 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:59.413+0000 7f3d597fa640 1 -- 172.21.15.39:0/3169390113 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d54012b70 con 0x55906fa7c1b0 2024-09-21T03:49:59.436 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:59.414+0000 7f3d5bfff640 1 --2- 172.21.15.39:0/2814574962 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f3d500b6680 0x7f3d500b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:49:59.436 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:59.414+0000 7f3d6359e640 1 --2- 172.21.15.39:0/2814574962 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f3d500b6680 0x7f3d500b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:49:59.436 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:59.414+0000 7f3d5bfff640 1 -- 172.21.15.39:0/2814574962 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8484+0+0 (secure 0 0 0) 0x7f3d4c107d30 con 0x7f3d641083a0 2024-09-21T03:49:59.436 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:59.414+0000 7f3d6359e640 1 --2- 172.21.15.39:0/2814574962 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f3d500b6680 0x7f3d500b8b90 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f3d6406bf40 tx=0x7f3d5c00b370 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:49:59.436 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:59.414+0000 7f3d6f7c8740 1 -- 172.21.15.39:0/3169390113 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55906fa7c1b0 msgr2=0x55906fe4cbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:49:59.436 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:59.414+0000 7f3d6f7c8740 1 --2- 172.21.15.39:0/3169390113 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55906fa7c1b0 0x55906fe4cbe0 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7f3d54009f50 tx=0x7f3d54010800 comp rx=0 tx=0).stop 2024-09-21T03:49:59.436 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:59.414+0000 7f3d6f7c8740 1 -- 172.21.15.39:0/3169390113 shutdown_connections 2024-09-21T03:49:59.436 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:59.414+0000 7f3d6f7c8740 1 --2- 172.21.15.39:0/3169390113 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55906fa84390 0x55906fe4d120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:59.436 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:59.414+0000 7f3d6f7c8740 1 --2- 172.21.15.39:0/3169390113 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55906fa7c1b0 0x55906fe4cbe0 unknown :-1 s=CLOSED pgs=714 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:59.436 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:59.414+0000 7f3d6f7c8740 1 --2- 172.21.15.39:0/3169390113 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55906faa5070 0x55906fe4c6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:59.436 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:59.414+0000 7f3d6f7c8740 1 -- 172.21.15.39:0/3169390113 >> 172.21.15.39:0/3169390113 conn(0x55906fa99460 msgr2=0x55906fe4a590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:49:59.436 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:59.415+0000 7f3d6f7c8740 1 -- 172.21.15.39:0/3169390113 shutdown_connections 2024-09-21T03:49:59.437 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:59.415+0000 7f3d6f7c8740 1 -- 172.21.15.39:0/3169390113 wait complete. 2024-09-21T03:49:59.437 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:59.415+0000 7f3d6f7c8740 1 -- start start 2024-09-21T03:49:59.437 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:59.415+0000 7f3d6f7c8740 1 Processor -- start 2024-09-21T03:49:59.437 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:59.415+0000 7f3d6f7c8740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55906fa7c1b0 0x55906fe57d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:49:59.437 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:59.415+0000 7f3d6f7c8740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55906fa84390 0x55906fe582b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:49:59.437 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:59.415+0000 7f3d6f7c8740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55906faa5070 0x55906fe641d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:49:59.437 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:59.415+0000 7f3d6f7c8740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x55906fc13700 con 0x55906faa5070 2024-09-21T03:49:59.437 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:59.415+0000 7f3d6f7c8740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x55906fd4d160 con 0x55906fa7c1b0 2024-09-21T03:49:59.437 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:59.415+0000 7f3d6f7c8740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55906fe5ae80 con 0x55906fa84390 2024-09-21T03:49:59.437 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:59.415+0000 7f3d6f7c8740 10 client.0 _ll_register_callbacks cb 0x55906fd669e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:49:59.437 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:59.415+0000 7f3d6359e640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55906faa5070 0x55906fe641d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:49:59.437 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:59.416+0000 7f3d6359e640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55906faa5070 0x55906fe641d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:60120/0 (socket says 172.21.15.39:60120) 2024-09-21T03:49:59.437 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:59.416+0000 7f3d6359e640 1 -- 172.21.15.39:0/393975689 learned_addr learned my addr 172.21.15.39:0/393975689 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:49:59.437 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:59.416+0000 7f3d6259c640 1 --2- 172.21.15.39:0/393975689 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55906fa84390 0x55906fe582b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:49:59.438 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:59.416+0000 7f3d62d9d640 1 --2- 172.21.15.39:0/393975689 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55906fa7c1b0 0x55906fe57d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:49:59.438 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:59.416+0000 7f3d6359e640 1 -- 172.21.15.39:0/393975689 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55906fa84390 msgr2=0x55906fe582b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:49:59.438 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:59.416+0000 7f3d6359e640 1 --2- 172.21.15.39:0/393975689 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55906fa84390 0x55906fe582b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:59.438 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:59.416+0000 7f3d6359e640 1 -- 172.21.15.39:0/393975689 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55906fa7c1b0 msgr2=0x55906fe57d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:49:59.438 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:59.416+0000 7f3d6359e640 1 --2- 172.21.15.39:0/393975689 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55906fa7c1b0 0x55906fe57d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:59.438 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:59.416+0000 7f3d6359e640 1 -- 172.21.15.39:0/393975689 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55906fd7ac90 con 0x55906faa5070 2024-09-21T03:49:59.438 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:59.416+0000 7f3d62d9d640 1 --2- 172.21.15.39:0/393975689 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55906fa7c1b0 0x55906fe57d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:49:59.438 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:59.416+0000 7f3d6359e640 1 --2- 172.21.15.39:0/393975689 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55906faa5070 0x55906fe641d0 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7f3d5c01bb40 tx=0x7f3d5c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cdd10344f31aa48d server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:49:59.438 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:59.416+0000 7f3d4a7fc640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-09-21T03:49:59.438 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:59.416+0000 7f3d4a7fc640 1 -- 172.21.15.39:0/393975689 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d5c00ed40 con 0x55906faa5070 2024-09-21T03:49:59.439 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:59.416+0000 7f3d4a7fc640 1 -- 172.21.15.39:0/393975689 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8484+0+0 (secure 0 0 0) 0x7f3d5c005020 con 0x55906faa5070 2024-09-21T03:49:59.439 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:59.416+0000 7f3d6f7c8740 1 -- 172.21.15.39:0/393975689 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3d5401bb10 con 0x55906faa5070 2024-09-21T03:49:59.439 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:59.416+0000 7f3d4a7fc640 1 -- 172.21.15.39:0/393975689 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f3d5c00ba00 con 0x55906faa5070 2024-09-21T03:49:59.439 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:59.416+0000 7f3d6f7c8740 10 client.16392 fetch_fsmap learned FSMap version 29 2024-09-21T03:49:59.439 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:59.416+0000 7f3d6f7c8740 1 -- 172.21.15.39:0/393975689 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55906fe66b80 con 0x55906faa5070 2024-09-21T03:49:59.439 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:59.417+0000 7f3d4a7fc640 1 -- 172.21.15.39:0/393975689 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3d5401bb10 con 0x55906faa5070 2024-09-21T03:49:59.439 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:59.417+0000 7f3d4a7fc640 1 -- 172.21.15.39:0/393975689 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f3d5c02d020 con 0x55906faa5070 2024-09-21T03:49:59.439 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:59.417+0000 7f3d6f7c8740 10 client.16392 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T03:49:59.439 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:59.417+0000 7f3d6f7c8740 10 client.16392 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-21T03:49:59.439 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:59.417+0000 7f3d6f7c8740 20 client.16392 populate_metadata read hostname 'smithi039' 2024-09-21T03:49:59.439 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:59.417+0000 7f3d6f7c8740 1 --2- 172.21.15.39:0/393975689 >> [v2:172.21.15.161:6838/2190107270,v1:172.21.15.161:6839/2190107270] conn(0x55906fa639f0 0x55906fe6a0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:49:59.439 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:59.417+0000 7f3d6f7c8740 4 client.16392 mds_command: new command op to 24667 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:49:59.440 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:59.417+0000 7f3d6f7c8740 1 -- 172.21.15.39:0/393975689 --> [v2:172.21.15.161:6838/2190107270,v1:172.21.15.161:6839/2190107270] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55906fdc4df0 con 0x55906fa639f0 2024-09-21T03:49:59.440 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:59.417+0000 7f3d62d9d640 1 --2- 172.21.15.39:0/393975689 >> [v2:172.21.15.161:6838/2190107270,v1:172.21.15.161:6839/2190107270] conn(0x55906fa639f0 0x55906fe6a0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:49:59.440 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:59.418+0000 7f3d62d9d640 1 --2- 172.21.15.39:0/393975689 >> [v2:172.21.15.161:6838/2190107270,v1:172.21.15.161:6839/2190107270] conn(0x55906fa639f0 0x55906fe6a0d0 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=cf9175756f2a79de server_cookie=aeb599e96e43ba65 in_seq=0 out_seq=0 2024-09-21T03:49:59.440 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:59.418+0000 7f3d4a7fc640 10 client.16392 ms_handle_connect on v2:172.21.15.161:6838/2190107270 2024-09-21T03:49:59.440 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:59.418+0000 7f3d4a7fc640 1 -- 172.21.15.39:0/393975689 <== mds.1 v2:172.21.15.161:6838/2190107270 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55906fdc4df0 con 0x55906fa639f0 2024-09-21T03:49:59.440 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:59.419+0000 7f3d4a7fc640 10 client.16392 handle_command_reply: tid=0 2024-09-21T03:49:59.440 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:59.419+0000 7f3d6f7c8740 1 client.16392 shutdown 2024-09-21T03:49:59.440 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:59.419+0000 7f3d6f7c8740 1 -- 172.21.15.39:0/393975689 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55906faa5070 msgr2=0x55906fe641d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:49:59.440 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:59.419+0000 7f3d6f7c8740 1 --2- 172.21.15.39:0/393975689 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55906faa5070 0x55906fe641d0 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7f3d5c01bb40 tx=0x7f3d5c0046c0 comp rx=0 tx=0).stop 2024-09-21T03:49:59.440 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:59.419+0000 7f3d6f7c8740 1 -- 172.21.15.39:0/393975689 shutdown_connections 2024-09-21T03:49:59.440 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:59.419+0000 7f3d6f7c8740 1 --2- 172.21.15.39:0/393975689 >> [v2:172.21.15.161:6838/2190107270,v1:172.21.15.161:6839/2190107270] conn(0x55906fa639f0 0x55906fe6a0d0 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:59.441 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:59.419+0000 7f3d6f7c8740 1 --2- 172.21.15.39:0/393975689 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55906faa5070 0x55906fe641d0 unknown :-1 s=CLOSED pgs=715 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:59.441 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:59.419+0000 7f3d6f7c8740 1 --2- 172.21.15.39:0/393975689 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55906fa84390 0x55906fe582b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:59.441 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:59.419+0000 7f3d4a7fc640 0 client.16392 ms_handle_reset on v2:172.21.15.161:6838/2190107270 2024-09-21T03:49:59.441 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:59.419+0000 7f3d6f7c8740 1 --2- 172.21.15.39:0/393975689 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55906fa7c1b0 0x55906fe57d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:59.441 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:59.419+0000 7f3d6f7c8740 1 -- 172.21.15.39:0/393975689 >> 172.21.15.39:0/393975689 conn(0x55906fa99460 msgr2=0x55906fe4f830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:49:59.441 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:59.419+0000 7f3d6f7c8740 1 -- 172.21.15.39:0/393975689 shutdown_connections 2024-09-21T03:49:59.441 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:59.419+0000 7f3d6f7c8740 1 -- 172.21.15.39:0/393975689 wait complete. 2024-09-21T03:49:59.441 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:59.419+0000 7f3d6f7c8740 20 client.16392 trim_cache size 0 max 16384 2024-09-21T03:49:59.445 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:59.432+0000 7f3d6f7c8740 1 Processor -- start 2024-09-21T03:49:59.446 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:59.432+0000 7f3d6f7c8740 1 -- start start 2024-09-21T03:49:59.446 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:59.433+0000 7f3d6f7c8740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55906fa7c1b0 0x55906fe4e1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:49:59.446 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:59.433+0000 7f3d6f7c8740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55906fa84390 0x55906fe59c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:49:59.446 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:59.433+0000 7f3d6f7c8740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55906faa5070 0x55906fe640d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:49:59.446 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:59.433+0000 7f3d6f7c8740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f3d641af730 con 0x55906fa7c1b0 2024-09-21T03:49:59.446 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:59.433+0000 7f3d6f7c8740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f3d641b1270 con 0x55906fa84390 2024-09-21T03:49:59.446 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:59.433+0000 7f3d6f7c8740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f3d641afe00 con 0x55906faa5070 2024-09-21T03:49:59.446 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:59.433+0000 7f3d62d9d640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55906fa7c1b0 0x55906fe4e1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:49:59.446 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:59.433+0000 7f3d62d9d640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55906fa7c1b0 0x55906fe4e1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:60126/0 (socket says 172.21.15.39:60126) 2024-09-21T03:49:59.446 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:59.433+0000 7f3d62d9d640 1 -- 172.21.15.39:0/1911739926 learned_addr learned my addr 172.21.15.39:0/1911739926 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:49:59.446 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:59.433+0000 7f3d6359e640 1 --2- 172.21.15.39:0/1911739926 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55906faa5070 0x55906fe640d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:49:59.446 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:59.433+0000 7f3d6259c640 1 --2- 172.21.15.39:0/1911739926 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55906fa84390 0x55906fe59c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:49:59.447 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:59.433+0000 7f3d62d9d640 1 -- 172.21.15.39:0/1911739926 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55906faa5070 msgr2=0x55906fe640d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:49:59.447 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:59.433+0000 7f3d62d9d640 1 --2- 172.21.15.39:0/1911739926 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55906faa5070 0x55906fe640d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:59.447 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:59.433+0000 7f3d62d9d640 1 -- 172.21.15.39:0/1911739926 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55906fa84390 msgr2=0x55906fe59c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:49:59.447 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:59.433+0000 7f3d62d9d640 1 --2- 172.21.15.39:0/1911739926 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55906fa84390 0x55906fe59c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:59.447 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:59.433+0000 7f3d62d9d640 1 -- 172.21.15.39:0/1911739926 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55906fe4ae80 con 0x55906fa7c1b0 2024-09-21T03:49:59.447 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:59.433+0000 7f3d6359e640 1 --2- 172.21.15.39:0/1911739926 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55906faa5070 0x55906fe640d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:49:59.447 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:59.433+0000 7f3d62d9d640 1 --2- 172.21.15.39:0/1911739926 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55906fa7c1b0 0x55906fe4e1a0 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7f3d54009f50 tx=0x7f3d540126f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c0007fd9c1aae64 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:49:59.447 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:59.434+0000 7f3d58ff9640 1 -- 172.21.15.39:0/1911739926 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d5402b040 con 0x55906fa7c1b0 2024-09-21T03:49:59.447 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:59.434+0000 7f3d58ff9640 1 -- 172.21.15.39:0/1911739926 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3d54003e20 con 0x55906fa7c1b0 2024-09-21T03:49:59.447 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:59.434+0000 7f3d58ff9640 1 -- 172.21.15.39:0/1911739926 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d54016040 con 0x55906fa7c1b0 2024-09-21T03:49:59.447 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:59.434+0000 7f3d6f7c8740 1 -- 172.21.15.39:0/1911739926 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55906fa7c1b0 msgr2=0x55906fe4e1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:49:59.447 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:59.434+0000 7f3d6f7c8740 1 --2- 172.21.15.39:0/1911739926 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55906fa7c1b0 0x55906fe4e1a0 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7f3d54009f50 tx=0x7f3d540126f0 comp rx=0 tx=0).stop 2024-09-21T03:49:59.447 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:59.435+0000 7f3d6f7c8740 1 -- 172.21.15.39:0/1911739926 shutdown_connections 2024-09-21T03:49:59.447 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:59.435+0000 7f3d6f7c8740 1 --2- 172.21.15.39:0/1911739926 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55906faa5070 0x55906fe640d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:59.448 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:59.435+0000 7f3d6f7c8740 1 --2- 172.21.15.39:0/1911739926 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55906fa84390 0x55906fe59c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:59.448 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:59.435+0000 7f3d6f7c8740 1 --2- 172.21.15.39:0/1911739926 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55906fa7c1b0 0x55906fe4e1a0 unknown :-1 s=CLOSED pgs=716 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:59.448 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:59.435+0000 7f3d6f7c8740 1 -- 172.21.15.39:0/1911739926 >> 172.21.15.39:0/1911739926 conn(0x55906fa99460 msgr2=0x55906fe517b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:49:59.448 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:59.435+0000 7f3d6f7c8740 1 -- 172.21.15.39:0/1911739926 shutdown_connections 2024-09-21T03:49:59.448 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:59.435+0000 7f3d6f7c8740 1 -- 172.21.15.39:0/1911739926 wait complete. 2024-09-21T03:49:59.448 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:59.435+0000 7f3d6f7c8740 1 -- start start 2024-09-21T03:49:59.448 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:59.435+0000 7f3d6f7c8740 1 Processor -- start 2024-09-21T03:49:59.448 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:59.435+0000 7f3d6f7c8740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55906fa7c1b0 0x55906fe5c3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:49:59.448 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:59.435+0000 7f3d6f7c8740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55906fa84390 0x55906fe5c8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:49:59.448 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:59.435+0000 7f3d6f7c8740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55906faa5070 0x55906fed3ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:49:59.448 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:59.435+0000 7f3d6f7c8740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f3d641b0ba0 con 0x55906fa7c1b0 2024-09-21T03:49:59.448 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:59.435+0000 7f3d6f7c8740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f3d641b04d0 con 0x55906faa5070 2024-09-21T03:49:59.448 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:59.435+0000 7f3d6f7c8740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f3d641ad710 con 0x55906fa84390 2024-09-21T03:49:59.448 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:59.436+0000 7f3d6f7c8740 10 client.0 _ll_register_callbacks cb 0x7f3d5c00be70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:49:59.449 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:59.436+0000 7f3d62d9d640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55906fa7c1b0 0x55906fe5c3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:49:59.449 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:59.436+0000 7f3d62d9d640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55906fa7c1b0 0x55906fe5c3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:60136/0 (socket says 172.21.15.39:60136) 2024-09-21T03:49:59.449 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:59.436+0000 7f3d62d9d640 1 -- 172.21.15.39:0/3205758405 learned_addr learned my addr 172.21.15.39:0/3205758405 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:49:59.449 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:59.436+0000 7f3d6259c640 1 --2- 172.21.15.39:0/3205758405 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55906fa84390 0x55906fe5c8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:49:59.449 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:59.436+0000 7f3d6359e640 1 --2- 172.21.15.39:0/3205758405 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55906faa5070 0x55906fed3ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:49:59.449 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:59.436+0000 7f3d62d9d640 1 -- 172.21.15.39:0/3205758405 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55906fa84390 msgr2=0x55906fe5c8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:49:59.449 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:59.436+0000 7f3d62d9d640 1 --2- 172.21.15.39:0/3205758405 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55906fa84390 0x55906fe5c8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:59.449 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:59.436+0000 7f3d62d9d640 1 -- 172.21.15.39:0/3205758405 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55906faa5070 msgr2=0x55906fed3ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:49:59.449 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:59.436+0000 7f3d62d9d640 1 --2- 172.21.15.39:0/3205758405 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55906faa5070 0x55906fed3ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:59.449 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:59.436+0000 7f3d62d9d640 1 -- 172.21.15.39:0/3205758405 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55906fe5d100 con 0x55906fa7c1b0 2024-09-21T03:49:59.449 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:59.436+0000 7f3d62d9d640 1 --2- 172.21.15.39:0/3205758405 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55906fa7c1b0 0x55906fe5c3a0 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7f3d54003a70 tx=0x7f3d54012680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ec4949a1cdf7c61 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:49:59.449 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:59.436+0000 7f3d4a7fc640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-09-21T03:49:59.449 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:59.436+0000 7f3d4a7fc640 1 -- 172.21.15.39:0/3205758405 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d5402b070 con 0x55906fa7c1b0 2024-09-21T03:49:59.449 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:59.436+0000 7f3d4a7fc640 1 -- 172.21.15.39:0/3205758405 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8484+0+0 (secure 0 0 0) 0x7f3d54003c10 con 0x55906fa7c1b0 2024-09-21T03:49:59.450 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:59.436+0000 7f3d6f7c8740 1 -- 172.21.15.39:0/3205758405 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3d54003e20 con 0x55906fa7c1b0 2024-09-21T03:49:59.450 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:59.437+0000 7f3d4a7fc640 1 -- 172.21.15.39:0/3205758405 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f3d54016050 con 0x55906fa7c1b0 2024-09-21T03:49:59.450 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:59.437+0000 7f3d6f7c8740 10 client.16398 fetch_fsmap learned FSMap version 29 2024-09-21T03:49:59.450 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:59.437+0000 7f3d6f7c8740 1 -- 172.21.15.39:0/3205758405 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55906fed4fe0 con 0x55906fa7c1b0 2024-09-21T03:49:59.450 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:59.437+0000 7f3d4a7fc640 1 -- 172.21.15.39:0/3205758405 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3d54003e20 con 0x55906fa7c1b0 2024-09-21T03:49:59.450 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:59.437+0000 7f3d4a7fc640 1 -- 172.21.15.39:0/3205758405 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f3d54040030 con 0x55906fa7c1b0 2024-09-21T03:49:59.450 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:59.437+0000 7f3d6f7c8740 10 client.16398 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T03:49:59.450 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:59.437+0000 7f3d6f7c8740 10 client.16398 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-21T03:49:59.450 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:59.437+0000 7f3d6f7c8740 20 client.16398 populate_metadata read hostname 'smithi039' 2024-09-21T03:49:59.450 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:59.437+0000 7f3d6f7c8740 1 --2- 172.21.15.39:0/3205758405 >> [v2:172.21.15.161:6838/2190107270,v1:172.21.15.161:6839/2190107270] conn(0x55906fa38da0 0x55906fed75f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:49:59.450 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:59.437+0000 7f3d6f7c8740 4 client.16398 mds_command: new command op to 24667 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T03:49:59.451 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:59.437+0000 7f3d6f7c8740 1 -- 172.21.15.39:0/3205758405 --> [v2:172.21.15.161:6838/2190107270,v1:172.21.15.161:6839/2190107270] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55906fdc08a0 con 0x55906fa38da0 2024-09-21T03:49:59.451 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:59.437+0000 7f3d6259c640 1 --2- 172.21.15.39:0/3205758405 >> [v2:172.21.15.161:6838/2190107270,v1:172.21.15.161:6839/2190107270] conn(0x55906fa38da0 0x55906fed75f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:49:59.451 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:59.438+0000 7f3d4a7fc640 10 client.16398 ms_handle_connect on v2:172.21.15.161:6838/2190107270 2024-09-21T03:49:59.451 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:59.438+0000 7f3d6259c640 1 --2- 172.21.15.39:0/3205758405 >> [v2:172.21.15.161:6838/2190107270,v1:172.21.15.161:6839/2190107270] conn(0x55906fa38da0 0x55906fed75f0 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=7b9582babb1f4211 server_cookie=b2cac6da72620f3f in_seq=0 out_seq=0 2024-09-21T03:49:59.451 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:59.438+0000 7f3d4a7fc640 1 -- 172.21.15.39:0/3205758405 <== mds.1 v2:172.21.15.161:6838/2190107270 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55906fdc08a0 con 0x55906fa38da0 2024-09-21T03:49:59.451 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:59.438+0000 7f3d4a7fc640 10 client.16398 handle_command_reply: tid=0 2024-09-21T03:49:59.451 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:59.438+0000 7f3d6f7c8740 1 client.16398 shutdown 2024-09-21T03:49:59.451 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:59.438+0000 7f3d6f7c8740 1 -- 172.21.15.39:0/3205758405 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55906fa7c1b0 msgr2=0x55906fe5c3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:49:59.451 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:59.438+0000 7f3d6f7c8740 1 --2- 172.21.15.39:0/3205758405 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55906fa7c1b0 0x55906fe5c3a0 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7f3d54003a70 tx=0x7f3d54012680 comp rx=0 tx=0).stop 2024-09-21T03:49:59.451 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:59.438+0000 7f3d6f7c8740 1 -- 172.21.15.39:0/3205758405 shutdown_connections 2024-09-21T03:49:59.451 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:59.438+0000 7f3d6f7c8740 1 --2- 172.21.15.39:0/3205758405 >> [v2:172.21.15.161:6838/2190107270,v1:172.21.15.161:6839/2190107270] conn(0x55906fa38da0 0x55906fed75f0 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:59.451 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:59.438+0000 7f3d6f7c8740 1 --2- 172.21.15.39:0/3205758405 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55906faa5070 0x55906fed3ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:59.451 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:59.438+0000 7f3d6f7c8740 1 --2- 172.21.15.39:0/3205758405 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55906fa84390 0x55906fe5c8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:59.451 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:59.438+0000 7f3d6f7c8740 1 --2- 172.21.15.39:0/3205758405 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55906fa7c1b0 0x55906fe5c3a0 unknown :-1 s=CLOSED pgs=717 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:59.451 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:59.438+0000 7f3d6f7c8740 1 -- 172.21.15.39:0/3205758405 >> 172.21.15.39:0/3205758405 conn(0x55906fa99460 msgr2=0x55906fe599b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:49:59.452 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:59.438+0000 7f3d4a7fc640 0 client.16398 ms_handle_reset on v2:172.21.15.161:6838/2190107270 2024-09-21T03:49:59.452 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:59.438+0000 7f3d6f7c8740 1 -- 172.21.15.39:0/3205758405 shutdown_connections 2024-09-21T03:49:59.452 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:59.438+0000 7f3d6f7c8740 1 -- 172.21.15.39:0/3205758405 wait complete. 2024-09-21T03:49:59.452 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:59.439+0000 7f3d6f7c8740 20 client.16398 trim_cache size 0 max 16384 2024-09-21T03:49:59.452 INFO:teuthology.orchestra.run.smithi039.stdout:[] 2024-09-21T03:49:59.453 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:59.439+0000 7f3d69254640 1 -- 172.21.15.39:0/2814574962 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f3d500b6680 msgr2=0x7f3d500b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:49:59.453 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:59.439+0000 7f3d69254640 1 --2- 172.21.15.39:0/2814574962 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f3d500b6680 0x7f3d500b8b90 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f3d6406bf40 tx=0x7f3d5c00b370 comp rx=0 tx=0).stop 2024-09-21T03:49:59.453 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:59.439+0000 7f3d69254640 1 -- 172.21.15.39:0/2814574962 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3d641083a0 msgr2=0x7f3d6419b830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:49:59.453 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:59.439+0000 7f3d69254640 1 --2- 172.21.15.39:0/2814574962 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3d641083a0 0x7f3d6419b830 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7f3d4c032fd0 tx=0x7f3d4c004270 comp rx=0 tx=0).stop 2024-09-21T03:49:59.453 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:59.439+0000 7f3d69254640 1 -- 172.21.15.39:0/2814574962 shutdown_connections 2024-09-21T03:49:59.453 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:59.439+0000 7f3d69254640 1 --2- 172.21.15.39:0/2814574962 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f3d500b6680 0x7f3d500b8b90 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:59.453 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:59.439+0000 7f3d69254640 1 --2- 172.21.15.39:0/2814574962 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3d64108cf0 0x7f3d6419bd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:59.453 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:59.439+0000 7f3d69254640 1 --2- 172.21.15.39:0/2814574962 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3d641083a0 0x7f3d6419b830 unknown :-1 s=CLOSED pgs=713 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:59.453 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:59.439+0000 7f3d69254640 1 --2- 172.21.15.39:0/2814574962 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3d64102a60 0x7f3d6419b2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:59.453 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:59.439+0000 7f3d69254640 1 -- 172.21.15.39:0/2814574962 >> 172.21.15.39:0/2814574962 conn(0x7f3d64079730 msgr2=0x7f3d64104520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:49:59.454 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:59.439+0000 7f3d69254640 1 -- 172.21.15.39:0/2814574962 shutdown_connections 2024-09-21T03:49:59.454 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:49:59.439+0000 7f3d69254640 1 -- 172.21.15.39:0/2814574962 wait complete. 2024-09-21T03:50:00.338 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 51116d5c-77ca-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 damage ls 2024-09-21T03:50:00.616 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/51116d5c-77ca-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:50:01.008 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:50:00 smithi039 ceph-mon[31211]: pgmap v444: 97 pgs: 97 active+clean; 10 MiB data, 536 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-09-21T03:50:01.008 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:50:00 smithi039 ceph-mon[31211]: overall HEALTH_OK 2024-09-21T03:50:01.125 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:01.102+0000 7f650c761640 1 -- 172.21.15.39:0/3120200953 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f65041094f0 msgr2=0x7f6504109900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:50:01.125 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:01.102+0000 7f650c761640 1 --2- 172.21.15.39:0/3120200953 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f65041094f0 0x7f6504109900 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7f64f8009900 tx=0x7f64f802a3f0 comp rx=0 tx=0).stop 2024-09-21T03:50:01.125 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:01.102+0000 7f650c761640 1 -- 172.21.15.39:0/3120200953 shutdown_connections 2024-09-21T03:50:01.125 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:01.102+0000 7f650c761640 1 --2- 172.21.15.39:0/3120200953 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6504109e40 0x7f6504110670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:01.125 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:01.102+0000 7f650c761640 1 --2- 172.21.15.39:0/3120200953 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f65041094f0 0x7f6504109900 unknown :-1 s=CLOSED pgs=718 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:01.125 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:01.102+0000 7f650c761640 1 --2- 172.21.15.39:0/3120200953 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6504103bb0 0x7f6504103fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:01.126 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:01.102+0000 7f650c761640 1 -- 172.21.15.39:0/3120200953 >> 172.21.15.39:0/3120200953 conn(0x7f65040ff600 msgr2=0x7f6504101a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:50:01.126 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:01.102+0000 7f650c761640 1 -- 172.21.15.39:0/3120200953 shutdown_connections 2024-09-21T03:50:01.126 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:01.102+0000 7f650c761640 1 -- 172.21.15.39:0/3120200953 wait complete. 2024-09-21T03:50:01.126 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:01.103+0000 7f650c761640 1 Processor -- start 2024-09-21T03:50:01.126 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:01.103+0000 7f650c761640 1 -- start start 2024-09-21T03:50:01.126 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:01.103+0000 7f650c761640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6504103bb0 0x7f6504075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:50:01.126 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:01.103+0000 7f650c761640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f65041094f0 0x7f6504075c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:50:01.126 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:01.103+0000 7f650c761640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6504109e40 0x7f6504076190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:50:01.126 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:01.103+0000 7f650c761640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f6504113b00 con 0x7f65041094f0 2024-09-21T03:50:01.126 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:01.103+0000 7f650c761640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f6504113980 con 0x7f6504103bb0 2024-09-21T03:50:01.126 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:01.103+0000 7f650c761640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f6504113c80 con 0x7f6504109e40 2024-09-21T03:50:01.126 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:01.103+0000 7f6509cfc640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f65041094f0 0x7f6504075c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:50:01.127 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:01.103+0000 7f6509cfc640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f65041094f0 0x7f6504075c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:60160/0 (socket says 172.21.15.39:60160) 2024-09-21T03:50:01.127 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:01.103+0000 7f6509cfc640 1 -- 172.21.15.39:0/248200745 learned_addr learned my addr 172.21.15.39:0/248200745 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:50:01.127 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:01.103+0000 7f650acfe640 1 --2- 172.21.15.39:0/248200745 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6504109e40 0x7f6504076190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:50:01.127 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:01.104+0000 7f650a4fd640 1 --2- 172.21.15.39:0/248200745 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6504103bb0 0x7f6504075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:50:01.127 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:01.104+0000 7f6509cfc640 1 -- 172.21.15.39:0/248200745 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6504109e40 msgr2=0x7f6504076190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:50:01.127 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:01.104+0000 7f6509cfc640 1 --2- 172.21.15.39:0/248200745 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6504109e40 0x7f6504076190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:01.127 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:01.104+0000 7f6509cfc640 1 -- 172.21.15.39:0/248200745 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6504103bb0 msgr2=0x7f6504075710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:50:01.127 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:01.104+0000 7f6509cfc640 1 --2- 172.21.15.39:0/248200745 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6504103bb0 0x7f6504075710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:01.127 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:01.104+0000 7f6509cfc640 1 -- 172.21.15.39:0/248200745 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f65041a8740 con 0x7f65041094f0 2024-09-21T03:50:01.127 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:01.104+0000 7f6509cfc640 1 --2- 172.21.15.39:0/248200745 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f65041094f0 0x7f6504075c50 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7f64f802a910 tx=0x7f64f8004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:50:01.127 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:01.104+0000 7f65037fe640 1 -- 172.21.15.39:0/248200745 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64f803f030 con 0x7f65041094f0 2024-09-21T03:50:01.127 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:01.104+0000 7f65037fe640 1 -- 172.21.15.39:0/248200745 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f64f8003aa0 con 0x7f65041094f0 2024-09-21T03:50:01.128 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:01.104+0000 7f650c761640 1 -- 172.21.15.39:0/248200745 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f65041a89d0 con 0x7f65041094f0 2024-09-21T03:50:01.128 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:01.104+0000 7f65037fe640 1 -- 172.21.15.39:0/248200745 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64f8035070 con 0x7f65041094f0 2024-09-21T03:50:01.128 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:01.104+0000 7f650c761640 1 -- 172.21.15.39:0/248200745 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f65041a8eb0 con 0x7f65041094f0 2024-09-21T03:50:01.128 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:01.107+0000 7f6512cd5740 1 Processor -- start 2024-09-21T03:50:01.128 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:01.107+0000 7f6512cd5740 1 -- start start 2024-09-21T03:50:01.128 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:01.107+0000 7f65037fe640 1 -- 172.21.15.39:0/248200745 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 27) ==== 154643+0+0 (secure 0 0 0) 0x7f64f8042030 con 0x7f65041094f0 2024-09-21T03:50:01.128 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:01.107+0000 7f6512cd5740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562b33b90070 0x562b33f376a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:50:01.128 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:01.107+0000 7f6512cd5740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562b33b671b0 0x562b33f37be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:50:01.128 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:01.107+0000 7f6512cd5740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562b33b6f390 0x562b33f38120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:50:01.128 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:01.107+0000 7f6512cd5740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x562b33eadea0 con 0x562b33b6f390 2024-09-21T03:50:01.128 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:01.107+0000 7f6512cd5740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x562b33e9d2d0 con 0x562b33b671b0 2024-09-21T03:50:01.128 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:01.107+0000 7f6512cd5740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x562b33ea10e0 con 0x562b33b90070 2024-09-21T03:50:01.128 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:01.107+0000 7f6509cfc640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562b33b6f390 0x562b33f38120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:50:01.128 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:01.108+0000 7f6509cfc640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562b33b6f390 0x562b33f38120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:60170/0 (socket says 172.21.15.39:60170) 2024-09-21T03:50:01.129 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:01.108+0000 7f6509cfc640 1 -- 172.21.15.39:0/1034792955 learned_addr learned my addr 172.21.15.39:0/1034792955 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:50:01.129 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:01.108+0000 7f650acfe640 1 --2- 172.21.15.39:0/1034792955 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562b33b90070 0x562b33f376a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:50:01.129 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:01.108+0000 7f650a4fd640 1 --2- 172.21.15.39:0/1034792955 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562b33b671b0 0x562b33f37be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:50:01.129 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:01.108+0000 7f6509cfc640 1 -- 172.21.15.39:0/1034792955 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562b33b90070 msgr2=0x562b33f376a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:50:01.129 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:01.108+0000 7f6509cfc640 1 --2- 172.21.15.39:0/1034792955 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562b33b90070 0x562b33f376a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:01.129 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:01.108+0000 7f6509cfc640 1 -- 172.21.15.39:0/1034792955 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562b33b671b0 msgr2=0x562b33f37be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:50:01.129 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:01.108+0000 7f6509cfc640 1 --2- 172.21.15.39:0/1034792955 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562b33b671b0 0x562b33f37be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:01.129 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:01.108+0000 7f6509cfc640 1 -- 172.21.15.39:0/1034792955 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562b33bfb650 con 0x562b33b6f390 2024-09-21T03:50:01.129 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:01.108+0000 7f65037fe640 1 --2- 172.21.15.39:0/248200745 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f64ec0b6680 0x7f64ec0b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:50:01.129 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:01.108+0000 7f650acfe640 1 --2- 172.21.15.39:0/248200745 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f64ec0b6680 0x7f64ec0b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:50:01.129 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:01.108+0000 7f6509cfc640 1 --2- 172.21.15.39:0/1034792955 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562b33b6f390 0x562b33f38120 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7f64f8105420 tx=0x7f64f810bf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=79a20861133073 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:50:01.129 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:01.108+0000 7f65037fe640 1 -- 172.21.15.39:0/248200745 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8484+0+0 (secure 0 0 0) 0x7f64f8107d40 con 0x7f65041094f0 2024-09-21T03:50:01.130 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:01.109+0000 7f6500ff9640 1 -- 172.21.15.39:0/1034792955 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64f80ca200 con 0x562b33b6f390 2024-09-21T03:50:01.130 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:01.109+0000 7f6500ff9640 1 -- 172.21.15.39:0/1034792955 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f64f803b460 con 0x562b33b6f390 2024-09-21T03:50:01.130 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:01.109+0000 7f6500ff9640 1 -- 172.21.15.39:0/1034792955 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64f80cd320 con 0x562b33b6f390 2024-09-21T03:50:01.130 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:01.110+0000 7f6512cd5740 1 -- 172.21.15.39:0/1034792955 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562b33b6f390 msgr2=0x562b33f38120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:50:01.130 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:01.110+0000 7f650acfe640 1 --2- 172.21.15.39:0/248200745 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f64ec0b6680 0x7f64ec0b8b90 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f64f4005fd0 tx=0x7f64f4009600 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:50:01.130 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:01.110+0000 7f6512cd5740 1 --2- 172.21.15.39:0/1034792955 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562b33b6f390 0x562b33f38120 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7f64f8105420 tx=0x7f64f810bf90 comp rx=0 tx=0).stop 2024-09-21T03:50:01.130 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:01.110+0000 7f6512cd5740 1 -- 172.21.15.39:0/1034792955 shutdown_connections 2024-09-21T03:50:01.130 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:01.110+0000 7f6512cd5740 1 --2- 172.21.15.39:0/1034792955 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562b33b6f390 0x562b33f38120 unknown :-1 s=CLOSED pgs=720 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:01.130 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:01.110+0000 7f6512cd5740 1 --2- 172.21.15.39:0/1034792955 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562b33b671b0 0x562b33f37be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:01.130 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:01.110+0000 7f6512cd5740 1 --2- 172.21.15.39:0/1034792955 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562b33b90070 0x562b33f376a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:01.130 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:01.110+0000 7f6512cd5740 1 -- 172.21.15.39:0/1034792955 >> 172.21.15.39:0/1034792955 conn(0x562b33b84460 msgr2=0x562b33f35590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:50:01.130 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:01.110+0000 7f6512cd5740 1 -- 172.21.15.39:0/1034792955 shutdown_connections 2024-09-21T03:50:01.130 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:01.110+0000 7f6512cd5740 1 -- 172.21.15.39:0/1034792955 wait complete. 2024-09-21T03:50:01.131 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:01.110+0000 7f6512cd5740 1 -- start start 2024-09-21T03:50:01.131 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:01.110+0000 7f6512cd5740 1 Processor -- start 2024-09-21T03:50:01.131 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:01.111+0000 7f6512cd5740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562b33b671b0 0x562b33f42d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:50:01.131 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:01.111+0000 7f6512cd5740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562b33b6f390 0x562b33f43270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:50:01.131 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:01.111+0000 7f6512cd5740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562b33b90070 0x562b33f4eea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:50:01.131 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:01.111+0000 7f6512cd5740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x562b33cfe700 con 0x562b33b671b0 2024-09-21T03:50:01.131 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:01.111+0000 7f6512cd5740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x562b33e38160 con 0x562b33b90070 2024-09-21T03:50:01.131 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:01.111+0000 7f6512cd5740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x562b33f45c40 con 0x562b33b6f390 2024-09-21T03:50:01.131 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:01.111+0000 7f6512cd5740 10 client.0 _ll_register_callbacks cb 0x562b33e519e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:50:01.131 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:01.111+0000 7f650a4fd640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562b33b671b0 0x562b33f42d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:50:01.131 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:01.111+0000 7f650a4fd640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562b33b671b0 0x562b33f42d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:60186/0 (socket says 172.21.15.39:60186) 2024-09-21T03:50:01.131 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:01.111+0000 7f650a4fd640 1 -- 172.21.15.39:0/837301536 learned_addr learned my addr 172.21.15.39:0/837301536 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:50:01.131 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:01.111+0000 7f6509cfc640 1 --2- 172.21.15.39:0/837301536 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562b33b6f390 0x562b33f43270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:50:01.131 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:01.111+0000 7f650acfe640 1 --2- 172.21.15.39:0/837301536 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562b33b90070 0x562b33f4eea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:50:01.132 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:01.111+0000 7f650a4fd640 1 -- 172.21.15.39:0/837301536 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562b33b6f390 msgr2=0x562b33f43270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:50:01.132 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:01.111+0000 7f650a4fd640 1 --2- 172.21.15.39:0/837301536 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562b33b6f390 0x562b33f43270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:01.132 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:01.111+0000 7f650a4fd640 1 -- 172.21.15.39:0/837301536 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562b33b90070 msgr2=0x562b33f4eea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:50:01.132 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:01.111+0000 7f650a4fd640 1 --2- 172.21.15.39:0/837301536 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562b33b90070 0x562b33f4eea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:01.132 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:01.111+0000 7f650a4fd640 1 -- 172.21.15.39:0/837301536 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562b33e65c90 con 0x562b33b671b0 2024-09-21T03:50:01.132 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:01.111+0000 7f650acfe640 1 --2- 172.21.15.39:0/837301536 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562b33b90070 0x562b33f4eea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:50:01.132 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:01.111+0000 7f6509cfc640 1 --2- 172.21.15.39:0/837301536 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562b33b6f390 0x562b33f43270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:50:01.132 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:01.112+0000 7f650a4fd640 1 --2- 172.21.15.39:0/837301536 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562b33b671b0 0x562b33f42d30 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7f64fc010820 tx=0x7f64fc010ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=17897a7027654fd1 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:50:01.132 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:01.112+0000 7f64e9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-09-21T03:50:01.132 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:01.112+0000 7f64e9ffb640 1 -- 172.21.15.39:0/837301536 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64fc00fb90 con 0x562b33b671b0 2024-09-21T03:50:01.132 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:01.112+0000 7f64e9ffb640 1 -- 172.21.15.39:0/837301536 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8484+0+0 (secure 0 0 0) 0x7f64fc00fd30 con 0x562b33b671b0 2024-09-21T03:50:01.132 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:01.112+0000 7f6512cd5740 1 -- 172.21.15.39:0/837301536 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x562b33eafdf0 con 0x562b33b671b0 2024-09-21T03:50:01.132 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:01.112+0000 7f64e9ffb640 1 -- 172.21.15.39:0/837301536 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f64fc005020 con 0x562b33b671b0 2024-09-21T03:50:01.132 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:01.112+0000 7f6512cd5740 10 client.16410 fetch_fsmap learned FSMap version 29 2024-09-21T03:50:01.133 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:01.112+0000 7f6512cd5740 1 -- 172.21.15.39:0/837301536 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x562b33f518f0 con 0x562b33b671b0 2024-09-21T03:50:01.133 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:01.112+0000 7f64e9ffb640 1 -- 172.21.15.39:0/837301536 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x562b33eafdf0 con 0x562b33b671b0 2024-09-21T03:50:01.133 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:01.112+0000 7f64e9ffb640 1 -- 172.21.15.39:0/837301536 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f64fc020020 con 0x562b33b671b0 2024-09-21T03:50:01.133 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:01.112+0000 7f6512cd5740 10 client.16410 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T03:50:01.133 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:01.112+0000 7f6512cd5740 10 client.16410 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T03:50:01.133 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:01.112+0000 7f6512cd5740 20 client.16410 populate_metadata read hostname 'smithi039' 2024-09-21T03:50:01.133 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:01.112+0000 7f6512cd5740 1 --2- 172.21.15.39:0/837301536 >> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] conn(0x562b33b4e9f0 0x562b33f54ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:50:01.133 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:01.112+0000 7f6512cd5740 4 client.16410 mds_command: new command op to 24695 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:50:01.133 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:01.112+0000 7f6512cd5740 1 -- 172.21.15.39:0/837301536 --> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562b33eab8a0 con 0x562b33b4e9f0 2024-09-21T03:50:01.133 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:01.113+0000 7f6509cfc640 1 --2- 172.21.15.39:0/837301536 >> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] conn(0x562b33b4e9f0 0x562b33f54ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:50:01.133 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:01.113+0000 7f64e9ffb640 10 client.16410 ms_handle_connect on v2:172.21.15.150:6838/2949386907 2024-09-21T03:50:01.133 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:01.113+0000 7f6509cfc640 1 --2- 172.21.15.39:0/837301536 >> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] conn(0x562b33b4e9f0 0x562b33f54ec0 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8c818f0969a790ea server_cookie=4868cf03903bf262 in_seq=0 out_seq=0 2024-09-21T03:50:01.133 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:01.114+0000 7f64e9ffb640 1 -- 172.21.15.39:0/837301536 <== mds.0 v2:172.21.15.150:6838/2949386907 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x562b33eab8a0 con 0x562b33b4e9f0 2024-09-21T03:50:01.133 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:01.114+0000 7f64e9ffb640 10 client.16410 handle_command_reply: tid=0 2024-09-21T03:50:01.134 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:01.114+0000 7f6512cd5740 1 client.16410 shutdown 2024-09-21T03:50:01.134 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:01.114+0000 7f6512cd5740 1 -- 172.21.15.39:0/837301536 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562b33b671b0 msgr2=0x562b33f42d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:50:01.134 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:01.114+0000 7f6512cd5740 1 --2- 172.21.15.39:0/837301536 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562b33b671b0 0x562b33f42d30 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7f64fc010820 tx=0x7f64fc010ce0 comp rx=0 tx=0).stop 2024-09-21T03:50:01.134 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:01.114+0000 7f6512cd5740 1 -- 172.21.15.39:0/837301536 shutdown_connections 2024-09-21T03:50:01.134 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:01.114+0000 7f6512cd5740 1 --2- 172.21.15.39:0/837301536 >> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] conn(0x562b33b4e9f0 0x562b33f54ec0 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:01.134 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:01.114+0000 7f6512cd5740 1 --2- 172.21.15.39:0/837301536 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562b33b90070 0x562b33f4eea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:01.134 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:01.114+0000 7f6512cd5740 1 --2- 172.21.15.39:0/837301536 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562b33b6f390 0x562b33f43270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:01.134 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:01.114+0000 7f64e9ffb640 0 client.16410 ms_handle_reset on v2:172.21.15.150:6838/2949386907 2024-09-21T03:50:01.134 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:01.114+0000 7f6512cd5740 1 --2- 172.21.15.39:0/837301536 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562b33b671b0 0x562b33f42d30 unknown :-1 s=CLOSED pgs=721 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:01.134 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:01.114+0000 7f6512cd5740 1 -- 172.21.15.39:0/837301536 >> 172.21.15.39:0/837301536 conn(0x562b33b84460 msgr2=0x562b33f3a830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:50:01.135 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:01.114+0000 7f6512cd5740 1 -- 172.21.15.39:0/837301536 shutdown_connections 2024-09-21T03:50:01.135 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:01.114+0000 7f6512cd5740 1 -- 172.21.15.39:0/837301536 wait complete. 2024-09-21T03:50:01.135 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:01.114+0000 7f6512cd5740 20 client.16410 trim_cache size 0 max 16384 2024-09-21T03:50:01.139 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:01.128+0000 7f6512cd5740 1 Processor -- start 2024-09-21T03:50:01.140 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:01.128+0000 7f6512cd5740 1 -- start start 2024-09-21T03:50:01.140 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:01.128+0000 7f6512cd5740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562b33b4e9f0 0x562b33f431f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:50:01.140 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:01.128+0000 7f6512cd5740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562b33b671b0 0x562b33f4eda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:50:01.140 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:01.128+0000 7f6512cd5740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562b33b6f390 0x562b33f34bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:50:01.140 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:01.128+0000 7f6512cd5740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f65041b1910 con 0x562b33b671b0 2024-09-21T03:50:01.140 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:01.128+0000 7f6512cd5740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f65041b3450 con 0x562b33b4e9f0 2024-09-21T03:50:01.140 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:01.128+0000 7f6512cd5740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f65041b1fe0 con 0x562b33b6f390 2024-09-21T03:50:01.140 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:01.128+0000 7f6509cfc640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562b33b671b0 0x562b33f4eda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:50:01.140 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:01.128+0000 7f6509cfc640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562b33b671b0 0x562b33f4eda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:60200/0 (socket says 172.21.15.39:60200) 2024-09-21T03:50:01.140 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:01.128+0000 7f6509cfc640 1 -- 172.21.15.39:0/751714380 learned_addr learned my addr 172.21.15.39:0/751714380 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:50:01.141 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:01.129+0000 7f650a4fd640 1 --2- 172.21.15.39:0/751714380 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562b33b4e9f0 0x562b33f431f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:50:01.141 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:01.129+0000 7f650acfe640 1 --2- 172.21.15.39:0/751714380 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562b33b6f390 0x562b33f34bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:50:01.141 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:01.129+0000 7f6509cfc640 1 -- 172.21.15.39:0/751714380 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562b33b6f390 msgr2=0x562b33f34bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:50:01.141 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:01.129+0000 7f6509cfc640 1 --2- 172.21.15.39:0/751714380 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562b33b6f390 0x562b33f34bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:01.141 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:01.129+0000 7f6509cfc640 1 -- 172.21.15.39:0/751714380 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562b33b4e9f0 msgr2=0x562b33f431f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:50:01.141 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:01.129+0000 7f6509cfc640 1 --2- 172.21.15.39:0/751714380 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562b33b4e9f0 0x562b33f431f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:01.141 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:01.129+0000 7f6509cfc640 1 -- 172.21.15.39:0/751714380 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562b33f41230 con 0x562b33b671b0 2024-09-21T03:50:01.141 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:01.129+0000 7f650acfe640 1 --2- 172.21.15.39:0/751714380 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562b33b6f390 0x562b33f34bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:50:01.141 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:01.129+0000 7f650a4fd640 1 --2- 172.21.15.39:0/751714380 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562b33b4e9f0 0x562b33f431f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:50:01.141 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:01.129+0000 7f6509cfc640 1 --2- 172.21.15.39:0/751714380 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562b33b671b0 0x562b33f4eda0 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7f64f8031440 tx=0x7f64f8009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e1ab1aa1ca36462 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:50:01.141 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:01.129+0000 7f64ebfff640 1 -- 172.21.15.39:0/751714380 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64f8049040 con 0x562b33b671b0 2024-09-21T03:50:01.141 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:01.129+0000 7f64ebfff640 1 -- 172.21.15.39:0/751714380 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f64f810b660 con 0x562b33b671b0 2024-09-21T03:50:01.141 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:01.129+0000 7f64ebfff640 1 -- 172.21.15.39:0/751714380 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64f80cd500 con 0x562b33b671b0 2024-09-21T03:50:01.141 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:01.130+0000 7f6512cd5740 1 -- 172.21.15.39:0/751714380 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562b33b671b0 msgr2=0x562b33f4eda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:50:01.141 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:01.130+0000 7f6512cd5740 1 --2- 172.21.15.39:0/751714380 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562b33b671b0 0x562b33f4eda0 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7f64f8031440 tx=0x7f64f8009260 comp rx=0 tx=0).stop 2024-09-21T03:50:01.142 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:01.130+0000 7f6512cd5740 1 -- 172.21.15.39:0/751714380 shutdown_connections 2024-09-21T03:50:01.142 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:01.130+0000 7f6512cd5740 1 --2- 172.21.15.39:0/751714380 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562b33b6f390 0x562b33f34bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:01.142 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:01.130+0000 7f6512cd5740 1 --2- 172.21.15.39:0/751714380 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562b33b671b0 0x562b33f4eda0 unknown :-1 s=CLOSED pgs=722 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:01.142 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:01.130+0000 7f6512cd5740 1 --2- 172.21.15.39:0/751714380 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562b33b4e9f0 0x562b33f431f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:01.142 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:01.130+0000 7f6512cd5740 1 -- 172.21.15.39:0/751714380 >> 172.21.15.39:0/751714380 conn(0x562b33b84460 msgr2=0x562b33f42ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:50:01.142 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:01.130+0000 7f6512cd5740 1 -- 172.21.15.39:0/751714380 shutdown_connections 2024-09-21T03:50:01.142 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:01.130+0000 7f6512cd5740 1 -- 172.21.15.39:0/751714380 wait complete. 2024-09-21T03:50:01.142 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:01.130+0000 7f6512cd5740 1 -- start start 2024-09-21T03:50:01.142 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:01.130+0000 7f6512cd5740 1 Processor -- start 2024-09-21T03:50:01.142 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:01.131+0000 7f6512cd5740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562b33b4e9f0 0x562b33f4b750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:50:01.142 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:01.131+0000 7f6512cd5740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562b33b671b0 0x562b33f46c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:50:01.142 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:01.131+0000 7f6512cd5740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562b33b6f390 0x562b33f472a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:50:01.142 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:01.131+0000 7f6512cd5740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f65041b2d80 con 0x562b33b6f390 2024-09-21T03:50:01.143 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:01.131+0000 7f6512cd5740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f65041b26b0 con 0x562b33b4e9f0 2024-09-21T03:50:01.143 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:01.131+0000 7f6512cd5740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f65041af8f0 con 0x562b33b671b0 2024-09-21T03:50:01.143 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:01.131+0000 7f6512cd5740 10 client.0 _ll_register_callbacks cb 0x7f64fc00a720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:50:01.143 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:01.131+0000 7f650acfe640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562b33b6f390 0x562b33f472a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:50:01.143 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:01.131+0000 7f650acfe640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562b33b6f390 0x562b33f472a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:60210/0 (socket says 172.21.15.39:60210) 2024-09-21T03:50:01.143 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:01.131+0000 7f650acfe640 1 -- 172.21.15.39:0/1546328107 learned_addr learned my addr 172.21.15.39:0/1546328107 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:50:01.143 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:01.131+0000 7f6509cfc640 1 --2- 172.21.15.39:0/1546328107 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562b33b671b0 0x562b33f46c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:50:01.143 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:01.131+0000 7f650a4fd640 1 --2- 172.21.15.39:0/1546328107 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562b33b4e9f0 0x562b33f4b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:50:01.143 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:01.131+0000 7f650acfe640 1 -- 172.21.15.39:0/1546328107 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562b33b671b0 msgr2=0x562b33f46c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:50:01.143 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:01.131+0000 7f650acfe640 1 --2- 172.21.15.39:0/1546328107 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562b33b671b0 0x562b33f46c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:01.144 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:01.131+0000 7f650acfe640 1 -- 172.21.15.39:0/1546328107 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562b33b4e9f0 msgr2=0x562b33f4b750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:50:01.144 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:01.131+0000 7f650acfe640 1 --2- 172.21.15.39:0/1546328107 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562b33b4e9f0 0x562b33f4b750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:01.144 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:01.131+0000 7f650acfe640 1 -- 172.21.15.39:0/1546328107 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562b33f47a30 con 0x562b33b6f390 2024-09-21T03:50:01.144 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:01.131+0000 7f650acfe640 1 --2- 172.21.15.39:0/1546328107 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562b33b6f390 0x562b33f472a0 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7f64f4011e90 tx=0x7f64f4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b788febaa7f61c82 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:50:01.144 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:01.131+0000 7f64e9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-09-21T03:50:01.144 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:01.132+0000 7f64e9ffb640 1 -- 172.21.15.39:0/1546328107 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64f401f810 con 0x562b33b6f390 2024-09-21T03:50:01.144 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:01.132+0000 7f64e9ffb640 1 -- 172.21.15.39:0/1546328107 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8484+0+0 (secure 0 0 0) 0x7f64f401f9b0 con 0x562b33b6f390 2024-09-21T03:50:01.144 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:01.132+0000 7f6512cd5740 1 -- 172.21.15.39:0/1546328107 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f64f810b660 con 0x562b33b6f390 2024-09-21T03:50:01.144 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:01.132+0000 7f64e9ffb640 1 -- 172.21.15.39:0/1546328107 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f64f401fbb0 con 0x562b33b6f390 2024-09-21T03:50:01.144 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:01.132+0000 7f6512cd5740 10 client.16416 fetch_fsmap learned FSMap version 29 2024-09-21T03:50:01.144 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:01.132+0000 7f6512cd5740 1 -- 172.21.15.39:0/1546328107 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x562b33fbf8e0 con 0x562b33b6f390 2024-09-21T03:50:01.144 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:01.132+0000 7f64e9ffb640 1 -- 172.21.15.39:0/1546328107 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f64f810b660 con 0x562b33b6f390 2024-09-21T03:50:01.144 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:01.132+0000 7f64e9ffb640 1 -- 172.21.15.39:0/1546328107 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f64f4032070 con 0x562b33b6f390 2024-09-21T03:50:01.144 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:01.132+0000 7f6512cd5740 10 client.16416 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T03:50:01.145 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:01.132+0000 7f6512cd5740 10 client.16416 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T03:50:01.145 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:01.132+0000 7f6512cd5740 20 client.16416 populate_metadata read hostname 'smithi039' 2024-09-21T03:50:01.145 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:01.132+0000 7f6512cd5740 1 --2- 172.21.15.39:0/1546328107 >> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] conn(0x562b33b90070 0x562b33fc2150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:50:01.145 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:01.132+0000 7f6512cd5740 4 client.16416 mds_command: new command op to 24695 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T03:50:01.145 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:01.132+0000 7f6512cd5740 1 -- 172.21.15.39:0/1546328107 --> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] -- command(tid 0: {"prefix": "damage ls"}) -- 0x562b33f554b0 con 0x562b33b90070 2024-09-21T03:50:01.145 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:01.133+0000 7f650a4fd640 1 --2- 172.21.15.39:0/1546328107 >> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] conn(0x562b33b90070 0x562b33fc2150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:50:01.145 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:01.133+0000 7f650a4fd640 1 --2- 172.21.15.39:0/1546328107 >> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] conn(0x562b33b90070 0x562b33fc2150 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3a8221cc00e9e848 server_cookie=8f943088d5ae816c in_seq=0 out_seq=0 2024-09-21T03:50:01.145 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:01.133+0000 7f64e9ffb640 10 client.16416 ms_handle_connect on v2:172.21.15.150:6838/2949386907 2024-09-21T03:50:01.145 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:01.133+0000 7f64e9ffb640 1 -- 172.21.15.39:0/1546328107 <== mds.0 v2:172.21.15.150:6838/2949386907 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x562b33f554b0 con 0x562b33b90070 2024-09-21T03:50:01.145 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:01.133+0000 7f64e9ffb640 10 client.16416 handle_command_reply: tid=0 2024-09-21T03:50:01.145 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:01.133+0000 7f6512cd5740 1 client.16416 shutdown 2024-09-21T03:50:01.145 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:01.134+0000 7f6512cd5740 1 -- 172.21.15.39:0/1546328107 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562b33b6f390 msgr2=0x562b33f472a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:50:01.145 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:01.134+0000 7f6512cd5740 1 --2- 172.21.15.39:0/1546328107 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562b33b6f390 0x562b33f472a0 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7f64f4011e90 tx=0x7f64f4004a40 comp rx=0 tx=0).stop 2024-09-21T03:50:01.146 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:01.134+0000 7f6512cd5740 1 -- 172.21.15.39:0/1546328107 shutdown_connections 2024-09-21T03:50:01.146 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:01.134+0000 7f6512cd5740 1 --2- 172.21.15.39:0/1546328107 >> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] conn(0x562b33b90070 0x562b33fc2150 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:01.146 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:01.134+0000 7f6512cd5740 1 --2- 172.21.15.39:0/1546328107 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562b33b6f390 0x562b33f472a0 unknown :-1 s=CLOSED pgs=723 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:01.146 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:01.134+0000 7f6512cd5740 1 --2- 172.21.15.39:0/1546328107 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562b33b671b0 0x562b33f46c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:01.146 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:01.134+0000 7f64e9ffb640 0 client.16416 ms_handle_reset on v2:172.21.15.150:6838/2949386907 2024-09-21T03:50:01.146 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:01.134+0000 7f6512cd5740 1 --2- 172.21.15.39:0/1546328107 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562b33b4e9f0 0x562b33f4b750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:01.146 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:01.134+0000 7f6512cd5740 1 -- 172.21.15.39:0/1546328107 >> 172.21.15.39:0/1546328107 conn(0x562b33b84460 msgr2=0x562b33f386d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:50:01.146 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:01.134+0000 7f6512cd5740 1 -- 172.21.15.39:0/1546328107 shutdown_connections 2024-09-21T03:50:01.146 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:01.134+0000 7f6512cd5740 1 -- 172.21.15.39:0/1546328107 wait complete. 2024-09-21T03:50:01.146 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:01.134+0000 7f6512cd5740 20 client.16416 trim_cache size 0 max 16384 2024-09-21T03:50:01.146 INFO:teuthology.orchestra.run.smithi039.stdout:[] 2024-09-21T03:50:01.147 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:01.134+0000 7f650c761640 1 -- 172.21.15.39:0/248200745 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f64ec0b6680 msgr2=0x7f64ec0b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:50:01.147 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:01.134+0000 7f650c761640 1 --2- 172.21.15.39:0/248200745 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f64ec0b6680 0x7f64ec0b8b90 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f64f4005fd0 tx=0x7f64f4009600 comp rx=0 tx=0).stop 2024-09-21T03:50:01.147 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:01.134+0000 7f650c761640 1 -- 172.21.15.39:0/248200745 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f65041094f0 msgr2=0x7f6504075c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:50:01.147 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:01.134+0000 7f650c761640 1 --2- 172.21.15.39:0/248200745 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f65041094f0 0x7f6504075c50 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7f64f802a910 tx=0x7f64f8004270 comp rx=0 tx=0).stop 2024-09-21T03:50:01.148 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:01.135+0000 7f650c761640 1 -- 172.21.15.39:0/248200745 shutdown_connections 2024-09-21T03:50:01.148 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:01.135+0000 7f650c761640 1 --2- 172.21.15.39:0/248200745 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f64ec0b6680 0x7f64ec0b8b90 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:01.148 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:01.135+0000 7f650c761640 1 --2- 172.21.15.39:0/248200745 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6504109e40 0x7f6504076190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:01.148 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:01.135+0000 7f650c761640 1 --2- 172.21.15.39:0/248200745 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f65041094f0 0x7f6504075c50 unknown :-1 s=CLOSED pgs=719 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:01.148 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:01.135+0000 7f650c761640 1 --2- 172.21.15.39:0/248200745 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6504103bb0 0x7f6504075710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:01.148 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:01.135+0000 7f650c761640 1 -- 172.21.15.39:0/248200745 >> 172.21.15.39:0/248200745 conn(0x7f65040ff600 msgr2=0x7f6504101a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:50:01.148 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:01.135+0000 7f650c761640 1 -- 172.21.15.39:0/248200745 shutdown_connections 2024-09-21T03:50:01.148 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:01.135+0000 7f650c761640 1 -- 172.21.15.39:0/248200745 wait complete. 2024-09-21T03:50:01.218 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:50:00 smithi161 ceph-mon[34533]: pgmap v444: 97 pgs: 97 active+clean; 10 MiB data, 536 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-09-21T03:50:01.218 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:50:00 smithi161 ceph-mon[34533]: overall HEALTH_OK 2024-09-21T03:50:01.225 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:50:00 smithi150 ceph-mon[34289]: pgmap v444: 97 pgs: 97 active+clean; 10 MiB data, 536 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-09-21T03:50:01.225 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:50:00 smithi150 ceph-mon[34289]: overall HEALTH_OK 2024-09-21T03:50:02.908 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-21T03:50:02.908 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 51116d5c-77ca-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-21T03:50:03.083 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:50:02 smithi039 ceph-mon[31211]: pgmap v445: 97 pgs: 97 active+clean; 10 MiB data, 536 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-21T03:50:03.180 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/51116d5c-77ca-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:50:03.218 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:50:02 smithi161 ceph-mon[34533]: pgmap v445: 97 pgs: 97 active+clean; 10 MiB data, 536 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-21T03:50:03.224 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:50:02 smithi150 ceph-mon[34289]: pgmap v445: 97 pgs: 97 active+clean; 10 MiB data, 536 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-21T03:50:03.725 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:03.677+0000 7f3ab69af640 1 -- 172.21.15.39:0/3617878292 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3ab01083a0 msgr2=0x7f3ab01087b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:50:03.725 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:03.677+0000 7f3ab69af640 1 --2- 172.21.15.39:0/3617878292 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3ab01083a0 0x7f3ab01087b0 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7f3a98009900 tx=0x7f3a9802a3f0 comp rx=0 tx=0).stop 2024-09-21T03:50:03.725 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:03.677+0000 7f3ab69af640 1 -- 172.21.15.39:0/3617878292 shutdown_connections 2024-09-21T03:50:03.725 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:03.677+0000 7f3ab69af640 1 --2- 172.21.15.39:0/3617878292 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3ab0108cf0 0x7f3ab010f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:03.726 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:03.677+0000 7f3ab69af640 1 --2- 172.21.15.39:0/3617878292 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3ab01083a0 0x7f3ab01087b0 unknown :-1 s=CLOSED pgs=724 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:03.726 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:03.677+0000 7f3ab69af640 1 --2- 172.21.15.39:0/3617878292 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3ab0102a60 0x7f3ab0102e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:03.726 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:03.677+0000 7f3ab69af640 1 -- 172.21.15.39:0/3617878292 >> 172.21.15.39:0/3617878292 conn(0x7f3ab0079730 msgr2=0x7f3ab0079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:50:03.726 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:03.677+0000 7f3ab69af640 1 -- 172.21.15.39:0/3617878292 shutdown_connections 2024-09-21T03:50:03.726 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:03.677+0000 7f3ab69af640 1 -- 172.21.15.39:0/3617878292 wait complete. 2024-09-21T03:50:03.726 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:03.678+0000 7f3ab69af640 1 Processor -- start 2024-09-21T03:50:03.726 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:03.678+0000 7f3ab69af640 1 -- start start 2024-09-21T03:50:03.726 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:03.678+0000 7f3ab69af640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3ab0102a60 0x7f3ab019a530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:50:03.726 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:03.678+0000 7f3ab69af640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3ab01083a0 0x7f3ab019aad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:50:03.726 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:03.678+0000 7f3ab69af640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3ab0108cf0 0x7f3ab019fd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:50:03.726 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:03.678+0000 7f3ab69af640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f3ab0113a80 con 0x7f3ab0102a60 2024-09-21T03:50:03.726 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:03.678+0000 7f3ab69af640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f3ab0113900 con 0x7f3ab0108cf0 2024-09-21T03:50:03.726 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:03.678+0000 7f3ab69af640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f3ab0113c00 con 0x7f3ab01083a0 2024-09-21T03:50:03.726 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:03.678+0000 7f3aaffff640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3ab0102a60 0x7f3ab019a530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:50:03.727 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:03.679+0000 7f3aaffff640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3ab0102a60 0x7f3ab019a530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:60222/0 (socket says 172.21.15.39:60222) 2024-09-21T03:50:03.727 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:03.679+0000 7f3aaffff640 1 -- 172.21.15.39:0/2480336464 learned_addr learned my addr 172.21.15.39:0/2480336464 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:50:03.727 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:03.679+0000 7f3aaf7fe640 1 --2- 172.21.15.39:0/2480336464 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3ab01083a0 0x7f3ab019aad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:50:03.727 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:03.679+0000 7f3ab4f4c640 1 --2- 172.21.15.39:0/2480336464 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3ab0108cf0 0x7f3ab019fd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:50:03.727 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:03.679+0000 7f3aaffff640 1 -- 172.21.15.39:0/2480336464 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3ab01083a0 msgr2=0x7f3ab019aad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:50:03.727 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:03.679+0000 7f3aaffff640 1 --2- 172.21.15.39:0/2480336464 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3ab01083a0 0x7f3ab019aad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:03.727 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:03.679+0000 7f3aaffff640 1 -- 172.21.15.39:0/2480336464 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3ab0108cf0 msgr2=0x7f3ab019fd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:50:03.727 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:03.679+0000 7f3aaffff640 1 --2- 172.21.15.39:0/2480336464 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3ab0108cf0 0x7f3ab019fd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:03.727 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:03.679+0000 7f3aaffff640 1 -- 172.21.15.39:0/2480336464 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3ab01a03e0 con 0x7f3ab0102a60 2024-09-21T03:50:03.727 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:03.679+0000 7f3aaffff640 1 --2- 172.21.15.39:0/2480336464 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3ab0102a60 0x7f3ab019a530 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7f3aa000d770 tx=0x7f3aa000dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:50:03.727 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:03.679+0000 7f3aad7fa640 1 -- 172.21.15.39:0/2480336464 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3aa0017070 con 0x7f3ab0102a60 2024-09-21T03:50:03.727 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:03.679+0000 7f3aad7fa640 1 -- 172.21.15.39:0/2480336464 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3aa00054d0 con 0x7f3ab0102a60 2024-09-21T03:50:03.727 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:03.679+0000 7f3ab69af640 1 -- 172.21.15.39:0/2480336464 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3ab01a06d0 con 0x7f3ab0102a60 2024-09-21T03:50:03.728 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:03.679+0000 7f3aad7fa640 1 -- 172.21.15.39:0/2480336464 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3aa0004c30 con 0x7f3ab0102a60 2024-09-21T03:50:03.728 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:03.679+0000 7f3ab69af640 1 -- 172.21.15.39:0/2480336464 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3ab01a0b90 con 0x7f3ab0102a60 2024-09-21T03:50:03.728 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:03.682+0000 7f3abcf23740 1 Processor -- start 2024-09-21T03:50:03.728 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:03.682+0000 7f3abcf23740 1 -- start start 2024-09-21T03:50:03.728 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:03.682+0000 7f3aad7fa640 1 -- 172.21.15.39:0/2480336464 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 27) ==== 154643+0+0 (secure 0 0 0) 0x7f3aa0004dd0 con 0x7f3ab0102a60 2024-09-21T03:50:03.728 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:03.683+0000 7f3abcf23740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564bc8f40350 0x564bc92e7430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:50:03.728 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:03.683+0000 7f3abcf23740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564bc8f17490 0x564bc92e7970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:50:03.728 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:03.683+0000 7f3abcf23740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564bc8f1f670 0x564bc92e7eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:50:03.728 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:03.683+0000 7f3abcf23740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x564bc925de60 con 0x564bc8f17490 2024-09-21T03:50:03.728 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:03.683+0000 7f3abcf23740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x564bc924d290 con 0x564bc8f40350 2024-09-21T03:50:03.728 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:03.683+0000 7f3abcf23740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x564bc92510a0 con 0x564bc8f1f670 2024-09-21T03:50:03.728 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:03.683+0000 7f3aaffff640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564bc8f17490 0x564bc92e7970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:50:03.728 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:03.683+0000 7f3aaffff640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564bc8f17490 0x564bc92e7970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:60230/0 (socket says 172.21.15.39:60230) 2024-09-21T03:50:03.728 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:03.683+0000 7f3aaffff640 1 -- 172.21.15.39:0/2568304449 learned_addr learned my addr 172.21.15.39:0/2568304449 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:50:03.728 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:03.683+0000 7f3aaf7fe640 1 --2- 172.21.15.39:0/2568304449 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564bc8f1f670 0x564bc92e7eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:50:03.729 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:03.683+0000 7f3ab4f4c640 1 --2- 172.21.15.39:0/2568304449 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564bc8f40350 0x564bc92e7430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:50:03.729 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:03.683+0000 7f3aaffff640 1 -- 172.21.15.39:0/2568304449 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564bc8f1f670 msgr2=0x564bc92e7eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:50:03.729 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:03.683+0000 7f3aaffff640 1 --2- 172.21.15.39:0/2568304449 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564bc8f1f670 0x564bc92e7eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:03.729 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:03.683+0000 7f3aaffff640 1 -- 172.21.15.39:0/2568304449 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564bc8f40350 msgr2=0x564bc92e7430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:50:03.729 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:03.683+0000 7f3aaffff640 1 --2- 172.21.15.39:0/2568304449 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564bc8f40350 0x564bc92e7430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:03.729 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:03.683+0000 7f3aaffff640 1 -- 172.21.15.39:0/2568304449 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564bc8fab930 con 0x564bc8f17490 2024-09-21T03:50:03.729 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:03.683+0000 7f3aad7fa640 1 --2- 172.21.15.39:0/2480336464 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f3a9c0b66b0 0x7f3a9c0b8bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:50:03.729 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:03.684+0000 7f3aaffff640 1 --2- 172.21.15.39:0/2568304449 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564bc8f17490 0x564bc92e7970 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7f3aa00e48c0 tx=0x7f3aa00ef910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=98e2e2e8cbb53ab8 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:50:03.729 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:03.684+0000 7f3ab4f4c640 1 --2- 172.21.15.39:0/2480336464 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f3a9c0b66b0 0x7f3a9c0b8bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:50:03.729 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:03.684+0000 7f3aa67fc640 1 -- 172.21.15.39:0/2568304449 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3aa00a96a0 con 0x564bc8f17490 2024-09-21T03:50:03.729 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:03.684+0000 7f3aad7fa640 1 -- 172.21.15.39:0/2480336464 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8484+0+0 (secure 0 0 0) 0x7f3aa00e71e0 con 0x7f3ab0102a60 2024-09-21T03:50:03.729 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:03.684+0000 7f3aa67fc640 1 -- 172.21.15.39:0/2568304449 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3aa00acb60 con 0x564bc8f17490 2024-09-21T03:50:03.729 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:03.684+0000 7f3aa67fc640 1 -- 172.21.15.39:0/2568304449 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3aa0022300 con 0x564bc8f17490 2024-09-21T03:50:03.730 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:03.685+0000 7f3abcf23740 1 -- 172.21.15.39:0/2568304449 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564bc8f17490 msgr2=0x564bc92e7970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:50:03.730 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:03.685+0000 7f3abcf23740 1 --2- 172.21.15.39:0/2568304449 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564bc8f17490 0x564bc92e7970 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7f3aa00e48c0 tx=0x7f3aa00ef910 comp rx=0 tx=0).stop 2024-09-21T03:50:03.730 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:03.685+0000 7f3ab4f4c640 1 --2- 172.21.15.39:0/2480336464 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f3a9c0b66b0 0x7f3a9c0b8bc0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f3aa8007e50 tx=0x7f3aa8007600 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:50:03.730 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:03.685+0000 7f3abcf23740 1 -- 172.21.15.39:0/2568304449 shutdown_connections 2024-09-21T03:50:03.730 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:03.685+0000 7f3abcf23740 1 --2- 172.21.15.39:0/2568304449 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564bc8f1f670 0x564bc92e7eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:03.730 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:03.685+0000 7f3abcf23740 1 --2- 172.21.15.39:0/2568304449 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564bc8f17490 0x564bc92e7970 unknown :-1 s=CLOSED pgs=726 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:03.730 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:03.685+0000 7f3abcf23740 1 --2- 172.21.15.39:0/2568304449 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564bc8f40350 0x564bc92e7430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:03.730 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:03.685+0000 7f3abcf23740 1 -- 172.21.15.39:0/2568304449 >> 172.21.15.39:0/2568304449 conn(0x564bc8f34740 msgr2=0x564bc92e5320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:50:03.730 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:03.685+0000 7f3abcf23740 1 -- 172.21.15.39:0/2568304449 shutdown_connections 2024-09-21T03:50:03.730 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:03.685+0000 7f3abcf23740 1 -- 172.21.15.39:0/2568304449 wait complete. 2024-09-21T03:50:03.730 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:03.685+0000 7f3abcf23740 1 -- start start 2024-09-21T03:50:03.730 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:03.686+0000 7f3abcf23740 1 Processor -- start 2024-09-21T03:50:03.730 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:03.686+0000 7f3abcf23740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564bc8f17490 0x564bc92f3090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:50:03.730 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:03.686+0000 7f3abcf23740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564bc8f1f670 0x564bc92f3610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:50:03.731 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:03.686+0000 7f3abcf23740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564bc8f40350 0x564bc92fef00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:50:03.731 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:03.686+0000 7f3abcf23740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x564bc90ae890 con 0x564bc8f17490 2024-09-21T03:50:03.731 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:03.686+0000 7f3abcf23740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x564bc91e7d40 con 0x564bc8f40350 2024-09-21T03:50:03.731 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:03.686+0000 7f3abcf23740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x564bc92f5c10 con 0x564bc8f1f670 2024-09-21T03:50:03.731 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:03.686+0000 7f3abcf23740 10 client.0 _ll_register_callbacks cb 0x564bc924bc80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:50:03.731 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:03.686+0000 7f3aaffff640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564bc8f17490 0x564bc92f3090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:50:03.731 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:03.686+0000 7f3aaffff640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564bc8f17490 0x564bc92f3090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:60236/0 (socket says 172.21.15.39:60236) 2024-09-21T03:50:03.731 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:03.686+0000 7f3aaffff640 1 -- 172.21.15.39:0/1716786656 learned_addr learned my addr 172.21.15.39:0/1716786656 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:50:03.731 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:03.686+0000 7f3aaf7fe640 1 --2- 172.21.15.39:0/1716786656 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564bc8f1f670 0x564bc92f3610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:50:03.731 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:03.686+0000 7f3ab4f4c640 1 --2- 172.21.15.39:0/1716786656 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564bc8f40350 0x564bc92fef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:50:03.731 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:03.686+0000 7f3aaffff640 1 -- 172.21.15.39:0/1716786656 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564bc8f1f670 msgr2=0x564bc92f3610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:50:03.731 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:03.686+0000 7f3aaffff640 1 --2- 172.21.15.39:0/1716786656 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564bc8f1f670 0x564bc92f3610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:03.731 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:03.686+0000 7f3aaffff640 1 -- 172.21.15.39:0/1716786656 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564bc8f40350 msgr2=0x564bc92fef00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:50:03.731 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:03.686+0000 7f3aaffff640 1 --2- 172.21.15.39:0/1716786656 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564bc8f40350 0x564bc92fef00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:03.732 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:03.686+0000 7f3aaffff640 1 -- 172.21.15.39:0/1716786656 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564bc92ff510 con 0x564bc8f17490 2024-09-21T03:50:03.732 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:03.686+0000 7f3ab4f4c640 1 --2- 172.21.15.39:0/1716786656 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564bc8f40350 0x564bc92fef00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:50:03.732 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:03.687+0000 7f3aaffff640 1 --2- 172.21.15.39:0/1716786656 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564bc8f17490 0x564bc92f3090 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7f3aa00a9620 tx=0x7f3aa0009e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b7837f56562edbef server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:50:03.732 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:03.687+0000 7f3a8f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-09-21T03:50:03.732 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:03.687+0000 7f3a8f7fe640 1 -- 172.21.15.39:0/1716786656 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3aa0020070 con 0x564bc8f17490 2024-09-21T03:50:03.732 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:03.687+0000 7f3a8f7fe640 1 -- 172.21.15.39:0/1716786656 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8484+0+0 (secure 0 0 0) 0x7f3aa00052c0 con 0x564bc8f17490 2024-09-21T03:50:03.732 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:03.687+0000 7f3abcf23740 1 -- 172.21.15.39:0/1716786656 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3aa00acb60 con 0x564bc8f17490 2024-09-21T03:50:03.732 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:03.687+0000 7f3a8f7fe640 1 -- 172.21.15.39:0/1716786656 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f3aa00104b0 con 0x564bc8f17490 2024-09-21T03:50:03.732 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:03.687+0000 7f3abcf23740 10 client.16428 fetch_fsmap learned FSMap version 29 2024-09-21T03:50:03.732 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:03.687+0000 7f3abcf23740 1 -- 172.21.15.39:0/1716786656 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x564bc9301e40 con 0x564bc8f17490 2024-09-21T03:50:03.732 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:03.687+0000 7f3a8f7fe640 1 -- 172.21.15.39:0/1716786656 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3aa00acb60 con 0x564bc8f17490 2024-09-21T03:50:03.732 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:03.687+0000 7f3a8f7fe640 1 -- 172.21.15.39:0/1716786656 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f3aa00b65e0 con 0x564bc8f17490 2024-09-21T03:50:03.732 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:03.687+0000 7f3abcf23740 10 client.16428 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T03:50:03.732 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:03.687+0000 7f3abcf23740 10 client.16428 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T03:50:03.732 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:03.687+0000 7f3abcf23740 20 client.16428 populate_metadata read hostname 'smithi039' 2024-09-21T03:50:03.732 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:03.687+0000 7f3abcf23740 1 --2- 172.21.15.39:0/1716786656 >> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] conn(0x564bc8efecd0 0x564bc93051a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:50:03.733 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:03.687+0000 7f3abcf23740 4 client.16428 mds_command: new command op to 24695 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:50:03.733 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:03.687+0000 7f3abcf23740 1 -- 172.21.15.39:0/1716786656 --> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564bc925fdb0 con 0x564bc8efecd0 2024-09-21T03:50:03.733 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:03.688+0000 7f3aaf7fe640 1 --2- 172.21.15.39:0/1716786656 >> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] conn(0x564bc8efecd0 0x564bc93051a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:50:03.733 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:03.688+0000 7f3aaf7fe640 1 --2- 172.21.15.39:0/1716786656 >> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] conn(0x564bc8efecd0 0x564bc93051a0 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f9600fd7bde3ee5f server_cookie=d628393bfcedab81 in_seq=0 out_seq=0 2024-09-21T03:50:03.733 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:03.688+0000 7f3a8f7fe640 10 client.16428 ms_handle_connect on v2:172.21.15.150:6838/2949386907 2024-09-21T03:50:03.733 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:03.689+0000 7f3a8f7fe640 1 -- 172.21.15.39:0/1716786656 <== mds.0 v2:172.21.15.150:6838/2949386907 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x564bc925fdb0 con 0x564bc8efecd0 2024-09-21T03:50:03.733 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:03.689+0000 7f3a8f7fe640 10 client.16428 handle_command_reply: tid=0 2024-09-21T03:50:03.733 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:03.689+0000 7f3abcf23740 1 client.16428 shutdown 2024-09-21T03:50:03.733 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:03.689+0000 7f3abcf23740 1 -- 172.21.15.39:0/1716786656 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564bc8f17490 msgr2=0x564bc92f3090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:50:03.733 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:03.689+0000 7f3abcf23740 1 --2- 172.21.15.39:0/1716786656 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564bc8f17490 0x564bc92f3090 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7f3aa00a9620 tx=0x7f3aa0009e90 comp rx=0 tx=0).stop 2024-09-21T03:50:03.733 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:03.689+0000 7f3abcf23740 1 -- 172.21.15.39:0/1716786656 shutdown_connections 2024-09-21T03:50:03.733 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:03.689+0000 7f3abcf23740 1 --2- 172.21.15.39:0/1716786656 >> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] conn(0x564bc8efecd0 0x564bc93051a0 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:03.733 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:03.689+0000 7f3abcf23740 1 --2- 172.21.15.39:0/1716786656 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564bc8f40350 0x564bc92fef00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:03.733 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:03.689+0000 7f3abcf23740 1 --2- 172.21.15.39:0/1716786656 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564bc8f1f670 0x564bc92f3610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:03.733 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:03.689+0000 7f3abcf23740 1 --2- 172.21.15.39:0/1716786656 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564bc8f17490 0x564bc92f3090 unknown :-1 s=CLOSED pgs=727 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:03.733 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:03.689+0000 7f3abcf23740 1 -- 172.21.15.39:0/1716786656 >> 172.21.15.39:0/1716786656 conn(0x564bc8f34740 msgr2=0x564bc92e8460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:50:03.734 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:03.689+0000 7f3a8f7fe640 0 client.16428 ms_handle_reset on v2:172.21.15.150:6838/2949386907 2024-09-21T03:50:03.734 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:03.689+0000 7f3abcf23740 1 -- 172.21.15.39:0/1716786656 shutdown_connections 2024-09-21T03:50:03.734 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:03.689+0000 7f3abcf23740 1 -- 172.21.15.39:0/1716786656 wait complete. 2024-09-21T03:50:03.734 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:03.689+0000 7f3abcf23740 20 client.16428 trim_cache size 0 max 16384 2024-09-21T03:50:03.734 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:03.707+0000 7f3abcf23740 1 Processor -- start 2024-09-21T03:50:03.734 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:03.707+0000 7f3abcf23740 1 -- start start 2024-09-21T03:50:03.734 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:03.707+0000 7f3abcf23740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564bc8f17490 0x564bc92ee640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:50:03.734 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:03.707+0000 7f3abcf23740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564bc8f1f670 0x564bc92f2720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:50:03.734 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:03.707+0000 7f3abcf23740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564bc8f40350 0x564bc92e8b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:50:03.734 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:03.707+0000 7f3abcf23740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f3ab01af640 con 0x564bc8f1f670 2024-09-21T03:50:03.734 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:03.707+0000 7f3abcf23740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f3ab01b1180 con 0x564bc8f40350 2024-09-21T03:50:03.734 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:03.707+0000 7f3abcf23740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f3ab01afd10 con 0x564bc8f17490 2024-09-21T03:50:03.734 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:03.708+0000 7f3aaf7fe640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564bc8f1f670 0x564bc92f2720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:50:03.735 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:03.708+0000 7f3aaf7fe640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564bc8f1f670 0x564bc92f2720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:60244/0 (socket says 172.21.15.39:60244) 2024-09-21T03:50:03.735 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:03.708+0000 7f3aaf7fe640 1 -- 172.21.15.39:0/153337466 learned_addr learned my addr 172.21.15.39:0/153337466 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:50:03.735 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:03.708+0000 7f3aaffff640 1 --2- 172.21.15.39:0/153337466 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564bc8f17490 0x564bc92ee640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:50:03.735 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:03.708+0000 7f3aaf7fe640 1 -- 172.21.15.39:0/153337466 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564bc8f17490 msgr2=0x564bc92ee640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:50:03.735 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:03.708+0000 7f3aaf7fe640 1 --2- 172.21.15.39:0/153337466 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564bc8f17490 0x564bc92ee640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:03.735 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:03.708+0000 7f3aaf7fe640 1 -- 172.21.15.39:0/153337466 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564bc8f40350 msgr2=0x564bc92e8b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:50:03.735 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:03.708+0000 7f3aaf7fe640 1 --2- 172.21.15.39:0/153337466 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564bc8f40350 0x564bc92e8b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:03.735 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:03.708+0000 7f3aaf7fe640 1 -- 172.21.15.39:0/153337466 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564bc92df590 con 0x564bc8f1f670 2024-09-21T03:50:03.735 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:03.708+0000 7f3aaffff640 1 --2- 172.21.15.39:0/153337466 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564bc8f17490 0x564bc92ee640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:50:03.735 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:03.708+0000 7f3aaf7fe640 1 --2- 172.21.15.39:0/153337466 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564bc8f1f670 0x564bc92f2720 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7f3ab006aaa0 tx=0x7f3a98031400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f05a623bba68a060 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:50:03.735 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:03.708+0000 7f3aa5ffb640 1 -- 172.21.15.39:0/153337466 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a98035040 con 0x564bc8f1f670 2024-09-21T03:50:03.735 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:03.708+0000 7f3aa5ffb640 1 -- 172.21.15.39:0/153337466 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3a98032920 con 0x564bc8f1f670 2024-09-21T03:50:03.735 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:03.708+0000 7f3aa5ffb640 1 -- 172.21.15.39:0/153337466 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a980049b0 con 0x564bc8f1f670 2024-09-21T03:50:03.736 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:03.709+0000 7f3abcf23740 1 -- 172.21.15.39:0/153337466 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564bc8f1f670 msgr2=0x564bc92f2720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:50:03.736 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:03.709+0000 7f3abcf23740 1 --2- 172.21.15.39:0/153337466 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564bc8f1f670 0x564bc92f2720 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7f3ab006aaa0 tx=0x7f3a98031400 comp rx=0 tx=0).stop 2024-09-21T03:50:03.736 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:03.709+0000 7f3abcf23740 1 -- 172.21.15.39:0/153337466 shutdown_connections 2024-09-21T03:50:03.736 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:03.709+0000 7f3abcf23740 1 --2- 172.21.15.39:0/153337466 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564bc8f40350 0x564bc92e8b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:03.736 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:03.709+0000 7f3abcf23740 1 --2- 172.21.15.39:0/153337466 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564bc8f1f670 0x564bc92f2720 unknown :-1 s=CLOSED pgs=728 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:03.736 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:03.709+0000 7f3abcf23740 1 --2- 172.21.15.39:0/153337466 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564bc8f17490 0x564bc92ee640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:03.736 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:03.709+0000 7f3abcf23740 1 -- 172.21.15.39:0/153337466 >> 172.21.15.39:0/153337466 conn(0x564bc8f34740 msgr2=0x564bc92e8460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:50:03.736 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:03.709+0000 7f3abcf23740 1 -- 172.21.15.39:0/153337466 shutdown_connections 2024-09-21T03:50:03.736 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:03.709+0000 7f3abcf23740 1 -- 172.21.15.39:0/153337466 wait complete. 2024-09-21T03:50:03.736 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:03.709+0000 7f3abcf23740 1 -- start start 2024-09-21T03:50:03.736 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:03.710+0000 7f3abcf23740 1 Processor -- start 2024-09-21T03:50:03.736 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:03.710+0000 7f3abcf23740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564bc8f17490 0x564bc93069f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:50:03.736 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:03.710+0000 7f3abcf23740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564bc8f1f670 0x564bc9306f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:50:03.736 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:03.710+0000 7f3abcf23740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564bc8f40350 0x564bc936e920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:50:03.736 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:03.710+0000 7f3abcf23740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f3ab01b0ab0 con 0x564bc8f17490 2024-09-21T03:50:03.737 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:03.710+0000 7f3abcf23740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f3ab01b03e0 con 0x564bc8f40350 2024-09-21T03:50:03.737 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:03.710+0000 7f3abcf23740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f3ab01ad620 con 0x564bc8f1f670 2024-09-21T03:50:03.737 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:03.710+0000 7f3abcf23740 10 client.0 _ll_register_callbacks cb 0x7f3aa80074a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:50:03.737 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:03.710+0000 7f3aaffff640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564bc8f17490 0x564bc93069f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:50:03.737 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:03.710+0000 7f3aaffff640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564bc8f17490 0x564bc93069f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:60254/0 (socket says 172.21.15.39:60254) 2024-09-21T03:50:03.737 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:03.710+0000 7f3aaffff640 1 -- 172.21.15.39:0/1290749466 learned_addr learned my addr 172.21.15.39:0/1290749466 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:50:03.737 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:03.710+0000 7f3aaf7fe640 1 --2- 172.21.15.39:0/1290749466 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564bc8f1f670 0x564bc9306f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:50:03.737 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:03.710+0000 7f3ab4f4c640 1 --2- 172.21.15.39:0/1290749466 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564bc8f40350 0x564bc936e920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:50:03.737 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:03.710+0000 7f3aaffff640 1 -- 172.21.15.39:0/1290749466 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564bc8f1f670 msgr2=0x564bc9306f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:50:03.737 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:03.710+0000 7f3aaffff640 1 --2- 172.21.15.39:0/1290749466 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564bc8f1f670 0x564bc9306f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:03.737 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:03.710+0000 7f3aaffff640 1 -- 172.21.15.39:0/1290749466 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564bc8f40350 msgr2=0x564bc936e920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:50:03.737 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:03.710+0000 7f3aaffff640 1 --2- 172.21.15.39:0/1290749466 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564bc8f40350 0x564bc936e920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:03.737 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:03.710+0000 7f3aaffff640 1 -- 172.21.15.39:0/1290749466 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564bc936ef60 con 0x564bc8f17490 2024-09-21T03:50:03.738 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:03.710+0000 7f3aaf7fe640 1 --2- 172.21.15.39:0/1290749466 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564bc8f1f670 0x564bc9306f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:50:03.738 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:03.711+0000 7f3aaffff640 1 --2- 172.21.15.39:0/1290749466 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564bc8f17490 0x564bc93069f0 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7f3aa000d740 tx=0x7f3aa00ef910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c37bc36bd6c9d894 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:50:03.738 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:03.711+0000 7f3a8f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-09-21T03:50:03.738 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:03.711+0000 7f3a8f7fe640 1 -- 172.21.15.39:0/1290749466 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3aa0020070 con 0x564bc8f17490 2024-09-21T03:50:03.738 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:03.711+0000 7f3a8f7fe640 1 -- 172.21.15.39:0/1290749466 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8484+0+0 (secure 0 0 0) 0x7f3aa00052c0 con 0x564bc8f17490 2024-09-21T03:50:03.738 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:03.711+0000 7f3abcf23740 1 -- 172.21.15.39:0/1290749466 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3a98032920 con 0x564bc8f17490 2024-09-21T03:50:03.738 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:03.711+0000 7f3a8f7fe640 1 -- 172.21.15.39:0/1290749466 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f3aa001cb20 con 0x564bc8f17490 2024-09-21T03:50:03.738 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:03.711+0000 7f3abcf23740 10 client.16434 fetch_fsmap learned FSMap version 29 2024-09-21T03:50:03.738 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:03.711+0000 7f3abcf23740 1 -- 172.21.15.39:0/1290749466 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x564bc9370220 con 0x564bc8f17490 2024-09-21T03:50:03.738 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:03.711+0000 7f3a8f7fe640 1 -- 172.21.15.39:0/1290749466 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3a98032920 con 0x564bc8f17490 2024-09-21T03:50:03.738 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:03.711+0000 7f3a8f7fe640 1 -- 172.21.15.39:0/1290749466 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f3aa00ac2b0 con 0x564bc8f17490 2024-09-21T03:50:03.738 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:03.711+0000 7f3abcf23740 10 client.16434 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T03:50:03.738 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:03.711+0000 7f3abcf23740 10 client.16434 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T03:50:03.739 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:03.711+0000 7f3abcf23740 20 client.16434 populate_metadata read hostname 'smithi039' 2024-09-21T03:50:03.739 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:03.711+0000 7f3abcf23740 1 --2- 172.21.15.39:0/1290749466 >> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] conn(0x564bc8ed4080 0x564bc9372560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:50:03.739 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:03.711+0000 7f3abcf23740 4 client.16434 mds_command: new command op to 24695 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-21T03:50:03.739 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:03.711+0000 7f3abcf23740 1 -- 172.21.15.39:0/1290749466 --> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x564bc925b860 con 0x564bc8ed4080 2024-09-21T03:50:03.739 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:03.712+0000 7f3aaf7fe640 1 --2- 172.21.15.39:0/1290749466 >> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] conn(0x564bc8ed4080 0x564bc9372560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:50:03.739 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:03.712+0000 7f3a8f7fe640 10 client.16434 ms_handle_connect on v2:172.21.15.150:6838/2949386907 2024-09-21T03:50:03.739 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:03.712+0000 7f3aaf7fe640 1 --2- 172.21.15.39:0/1290749466 >> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] conn(0x564bc8ed4080 0x564bc9372560 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=48b4a491f46f9961 server_cookie=3dd8adbc418a70ba in_seq=0 out_seq=0 2024-09-21T03:50:03.739 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:03.714+0000 7f3a8f7fe640 1 -- 172.21.15.39:0/1290749466 <== mds.0 v2:172.21.15.150:6838/2949386907 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x564bc925b860 con 0x564bc8ed4080 2024-09-21T03:50:03.739 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:03.714+0000 7f3a8f7fe640 10 client.16434 handle_command_reply: tid=0 2024-09-21T03:50:03.739 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:03.714+0000 7f3abcf23740 1 client.16434 shutdown 2024-09-21T03:50:03.739 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:03.714+0000 7f3abcf23740 1 -- 172.21.15.39:0/1290749466 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564bc8f17490 msgr2=0x564bc93069f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:50:03.739 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:03.714+0000 7f3abcf23740 1 --2- 172.21.15.39:0/1290749466 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564bc8f17490 0x564bc93069f0 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7f3aa000d740 tx=0x7f3aa00ef910 comp rx=0 tx=0).stop 2024-09-21T03:50:03.739 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:03.714+0000 7f3abcf23740 1 -- 172.21.15.39:0/1290749466 shutdown_connections 2024-09-21T03:50:03.740 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:03.714+0000 7f3abcf23740 1 --2- 172.21.15.39:0/1290749466 >> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] conn(0x564bc8ed4080 0x564bc9372560 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:03.740 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:03.714+0000 7f3abcf23740 1 --2- 172.21.15.39:0/1290749466 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564bc8f40350 0x564bc936e920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:03.740 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:03.714+0000 7f3abcf23740 1 --2- 172.21.15.39:0/1290749466 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564bc8f1f670 0x564bc9306f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:03.740 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:03.714+0000 7f3abcf23740 1 --2- 172.21.15.39:0/1290749466 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564bc8f17490 0x564bc93069f0 unknown :-1 s=CLOSED pgs=729 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:03.740 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:03.714+0000 7f3abcf23740 1 -- 172.21.15.39:0/1290749466 >> 172.21.15.39:0/1290749466 conn(0x564bc8f34740 msgr2=0x564bc92f4150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:50:03.740 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:03.714+0000 7f3a8f7fe640 0 client.16434 ms_handle_reset on v2:172.21.15.150:6838/2949386907 2024-09-21T03:50:03.740 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-09-21T03:50:03.740 INFO:teuthology.orchestra.run.smithi039.stdout: "return_code": 0, 2024-09-21T03:50:03.740 INFO:teuthology.orchestra.run.smithi039.stdout: "scrub_tag": "6e760f1f-9048-4f92-bd66-d708fbef75b3", 2024-09-21T03:50:03.740 INFO:teuthology.orchestra.run.smithi039.stdout: "mode": "asynchronous" 2024-09-21T03:50:03.741 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-09-21T03:50:03.741 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:03.714+0000 7f3abcf23740 1 -- 172.21.15.39:0/1290749466 shutdown_connections 2024-09-21T03:50:03.741 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:03.714+0000 7f3abcf23740 1 -- 172.21.15.39:0/1290749466 wait complete. 2024-09-21T03:50:03.741 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:03.714+0000 7f3abcf23740 20 client.16434 trim_cache size 0 max 16384 2024-09-21T03:50:03.741 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:03.715+0000 7f3ab69af640 1 -- 172.21.15.39:0/2480336464 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f3a9c0b66b0 msgr2=0x7f3a9c0b8bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:50:03.741 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:03.715+0000 7f3ab69af640 1 --2- 172.21.15.39:0/2480336464 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f3a9c0b66b0 0x7f3a9c0b8bc0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f3aa8007e50 tx=0x7f3aa8007600 comp rx=0 tx=0).stop 2024-09-21T03:50:03.741 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:03.715+0000 7f3ab69af640 1 -- 172.21.15.39:0/2480336464 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3ab0102a60 msgr2=0x7f3ab019a530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:50:03.741 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:03.715+0000 7f3ab69af640 1 --2- 172.21.15.39:0/2480336464 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3ab0102a60 0x7f3ab019a530 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7f3aa000d770 tx=0x7f3aa000dc30 comp rx=0 tx=0).stop 2024-09-21T03:50:03.741 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:03.715+0000 7f3ab69af640 1 -- 172.21.15.39:0/2480336464 shutdown_connections 2024-09-21T03:50:03.741 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:03.715+0000 7f3ab69af640 1 --2- 172.21.15.39:0/2480336464 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f3a9c0b66b0 0x7f3a9c0b8bc0 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:03.741 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:03.715+0000 7f3ab69af640 1 --2- 172.21.15.39:0/2480336464 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3ab0108cf0 0x7f3ab019fd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:03.742 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:03.715+0000 7f3ab69af640 1 --2- 172.21.15.39:0/2480336464 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3ab01083a0 0x7f3ab019aad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:03.742 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:03.715+0000 7f3ab69af640 1 --2- 172.21.15.39:0/2480336464 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3ab0102a60 0x7f3ab019a530 unknown :-1 s=CLOSED pgs=725 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:03.742 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:03.715+0000 7f3ab69af640 1 -- 172.21.15.39:0/2480336464 >> 172.21.15.39:0/2480336464 conn(0x7f3ab0079730 msgr2=0x7f3ab0104510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:50:03.742 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:03.715+0000 7f3ab69af640 1 -- 172.21.15.39:0/2480336464 shutdown_connections 2024-09-21T03:50:03.742 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:03.715+0000 7f3ab69af640 1 -- 172.21.15.39:0/2480336464 wait complete. 2024-09-21T03:50:04.633 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 51116d5c-77ca-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub status 2024-09-21T03:50:04.891 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/51116d5c-77ca-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:50:05.012 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:50:04 smithi039 ceph-mon[31211]: pgmap v446: 97 pgs: 97 active+clean; 10 MiB data, 536 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 426 B/s wr, 10 op/s 2024-09-21T03:50:05.012 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:50:04 smithi039 ceph-mon[31211]: scrub queued for path: / 2024-09-21T03:50:05.012 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:50:04 smithi039 ceph-mon[31211]: scrub summary: idle+waiting paths [/] 2024-09-21T03:50:05.012 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:50:04 smithi039 ceph-mon[31211]: scrub summary: active paths [/] 2024-09-21T03:50:05.012 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:50:04 smithi039 ceph-mon[31211]: scrub summary: idle+waiting paths [/] 2024-09-21T03:50:05.218 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:50:04 smithi161 ceph-mon[34533]: pgmap v446: 97 pgs: 97 active+clean; 10 MiB data, 536 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 426 B/s wr, 10 op/s 2024-09-21T03:50:05.218 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:50:04 smithi161 ceph-mon[34533]: scrub queued for path: / 2024-09-21T03:50:05.218 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:50:04 smithi161 ceph-mon[34533]: scrub summary: idle+waiting paths [/] 2024-09-21T03:50:05.218 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:50:04 smithi161 ceph-mon[34533]: scrub summary: active paths [/] 2024-09-21T03:50:05.218 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:50:04 smithi161 ceph-mon[34533]: scrub summary: idle+waiting paths [/] 2024-09-21T03:50:05.224 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:50:04 smithi150 ceph-mon[34289]: pgmap v446: 97 pgs: 97 active+clean; 10 MiB data, 536 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 426 B/s wr, 10 op/s 2024-09-21T03:50:05.225 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:50:04 smithi150 ceph-mon[34289]: scrub queued for path: / 2024-09-21T03:50:05.225 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:50:04 smithi150 ceph-mon[34289]: scrub summary: idle+waiting paths [/] 2024-09-21T03:50:05.225 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:50:04 smithi150 ceph-mon[34289]: scrub summary: active paths [/] 2024-09-21T03:50:05.225 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:50:04 smithi150 ceph-mon[34289]: scrub summary: idle+waiting paths [/] 2024-09-21T03:50:05.376 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:05.366+0000 7f0879119640 1 -- 172.21.15.39:0/3182676955 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f08741094a0 msgr2=0x7f08741098b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:50:05.376 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:05.366+0000 7f0879119640 1 --2- 172.21.15.39:0/3182676955 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f08741094a0 0x7f08741098b0 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7f085c009900 tx=0x7f085c02a410 comp rx=0 tx=0).stop 2024-09-21T03:50:05.376 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:05.366+0000 7f0879119640 1 -- 172.21.15.39:0/3182676955 shutdown_connections 2024-09-21T03:50:05.376 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:05.366+0000 7f0879119640 1 --2- 172.21.15.39:0/3182676955 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0874109df0 0x7f0874110620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:05.376 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:05.366+0000 7f0879119640 1 --2- 172.21.15.39:0/3182676955 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f08741094a0 0x7f08741098b0 unknown :-1 s=CLOSED pgs=730 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:05.376 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:05.366+0000 7f0879119640 1 --2- 172.21.15.39:0/3182676955 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0874103b60 0x7f0874103f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:05.377 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:05.366+0000 7f0879119640 1 -- 172.21.15.39:0/3182676955 >> 172.21.15.39:0/3182676955 conn(0x7f087407b850 msgr2=0x7f087407bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:50:05.377 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:05.366+0000 7f0879119640 1 -- 172.21.15.39:0/3182676955 shutdown_connections 2024-09-21T03:50:05.377 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:05.366+0000 7f0879119640 1 -- 172.21.15.39:0/3182676955 wait complete. 2024-09-21T03:50:05.377 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:05.367+0000 7f0879119640 1 Processor -- start 2024-09-21T03:50:05.377 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:05.367+0000 7f0879119640 1 -- start start 2024-09-21T03:50:05.377 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:05.367+0000 7f0879119640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0874103b60 0x7f087419a580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:50:05.377 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:05.367+0000 7f0879119640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f08741094a0 0x7f087419ab20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:50:05.377 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:05.367+0000 7f0879119640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0874109df0 0x7f087419fdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:50:05.377 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:05.367+0000 7f0879119640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f0874113ab0 con 0x7f0874109df0 2024-09-21T03:50:05.377 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:05.367+0000 7f0879119640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f0874113930 con 0x7f0874103b60 2024-09-21T03:50:05.377 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:05.367+0000 7f0879119640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f0874113c30 con 0x7f08741094a0 2024-09-21T03:50:05.377 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:05.368+0000 7f087359e640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0874109df0 0x7f087419fdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:50:05.377 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:05.368+0000 7f087359e640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0874109df0 0x7f087419fdc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:60266/0 (socket says 172.21.15.39:60266) 2024-09-21T03:50:05.377 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:05.368+0000 7f087359e640 1 -- 172.21.15.39:0/1581074133 learned_addr learned my addr 172.21.15.39:0/1581074133 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:50:05.378 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:05.368+0000 7f087259c640 1 --2- 172.21.15.39:0/1581074133 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f08741094a0 0x7f087419ab20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:50:05.378 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:05.368+0000 7f0872d9d640 1 --2- 172.21.15.39:0/1581074133 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0874103b60 0x7f087419a580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:50:05.378 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:05.368+0000 7f087359e640 1 -- 172.21.15.39:0/1581074133 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f08741094a0 msgr2=0x7f087419ab20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:50:05.378 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:05.368+0000 7f087359e640 1 --2- 172.21.15.39:0/1581074133 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f08741094a0 0x7f087419ab20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:05.378 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:05.368+0000 7f087359e640 1 -- 172.21.15.39:0/1581074133 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0874103b60 msgr2=0x7f087419a580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:50:05.378 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:05.368+0000 7f087359e640 1 --2- 172.21.15.39:0/1581074133 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0874103b60 0x7f087419a580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:05.378 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:05.368+0000 7f087359e640 1 -- 172.21.15.39:0/1581074133 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f08741a04f0 con 0x7f0874109df0 2024-09-21T03:50:05.378 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:05.368+0000 7f0872d9d640 1 --2- 172.21.15.39:0/1581074133 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0874103b60 0x7f087419a580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:50:05.378 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:05.368+0000 7f087359e640 1 --2- 172.21.15.39:0/1581074133 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0874109df0 0x7f087419fdc0 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7f086c00d750 tx=0x7f086c00dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:50:05.378 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:05.368+0000 7f086bfff640 1 -- 172.21.15.39:0/1581074133 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f086c017070 con 0x7f0874109df0 2024-09-21T03:50:05.378 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:05.368+0000 7f0879119640 1 -- 172.21.15.39:0/1581074133 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f08741a07e0 con 0x7f0874109df0 2024-09-21T03:50:05.378 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:05.368+0000 7f086bfff640 1 -- 172.21.15.39:0/1581074133 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f086c0054d0 con 0x7f0874109df0 2024-09-21T03:50:05.378 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:05.368+0000 7f086bfff640 1 -- 172.21.15.39:0/1581074133 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f086c004c60 con 0x7f0874109df0 2024-09-21T03:50:05.378 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:05.369+0000 7f0879119640 1 -- 172.21.15.39:0/1581074133 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f08741a6f40 con 0x7f0874109df0 2024-09-21T03:50:05.379 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:05.371+0000 7f087f68d740 1 Processor -- start 2024-09-21T03:50:05.379 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:05.371+0000 7f086bfff640 1 -- 172.21.15.39:0/1581074133 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 27) ==== 154643+0+0 (secure 0 0 0) 0x7f086c009bd0 con 0x7f0874109df0 2024-09-21T03:50:05.379 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:05.371+0000 7f087f68d740 1 -- start start 2024-09-21T03:50:05.379 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:05.372+0000 7f087f68d740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b6df3e9070 0x55b6df790650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:50:05.379 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:05.372+0000 7f087f68d740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b6df3c01b0 0x55b6df790b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:50:05.379 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:05.372+0000 7f087f68d740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b6df3c8390 0x55b6df7910d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:50:05.379 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:05.372+0000 7f087f68d740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x55b6df706e50 con 0x55b6df3c01b0 2024-09-21T03:50:05.379 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:05.372+0000 7f087f68d740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x55b6df6f6280 con 0x55b6df3e9070 2024-09-21T03:50:05.379 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:05.372+0000 7f087f68d740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55b6df6fa090 con 0x55b6df3c8390 2024-09-21T03:50:05.379 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:05.372+0000 7f0872d9d640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b6df3c01b0 0x55b6df790b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:50:05.379 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:05.372+0000 7f0872d9d640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b6df3c01b0 0x55b6df790b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:60276/0 (socket says 172.21.15.39:60276) 2024-09-21T03:50:05.379 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:05.372+0000 7f0872d9d640 1 -- 172.21.15.39:0/1902944485 learned_addr learned my addr 172.21.15.39:0/1902944485 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:50:05.379 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:05.372+0000 7f087259c640 1 --2- 172.21.15.39:0/1902944485 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b6df3c8390 0x55b6df7910d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:50:05.379 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:05.372+0000 7f087259c640 1 --2- 172.21.15.39:0/1902944485 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b6df3e9070 0x55b6df790650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:50:05.379 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:05.372+0000 7f0872d9d640 1 -- 172.21.15.39:0/1902944485 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b6df3c8390 msgr2=0x55b6df7910d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:50:05.380 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:05.372+0000 7f0872d9d640 1 --2- 172.21.15.39:0/1902944485 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b6df3c8390 0x55b6df7910d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:05.380 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:05.372+0000 7f0872d9d640 1 -- 172.21.15.39:0/1902944485 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b6df3e9070 msgr2=0x55b6df790650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:50:05.380 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:05.372+0000 7f0872d9d640 1 --2- 172.21.15.39:0/1902944485 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b6df3e9070 0x55b6df790650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:05.380 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:05.372+0000 7f0872d9d640 1 -- 172.21.15.39:0/1902944485 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b6df454650 con 0x55b6df3c01b0 2024-09-21T03:50:05.380 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:05.373+0000 7f086bfff640 1 --2- 172.21.15.39:0/1581074133 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f08600baa90 0x7f08600bcfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:50:05.380 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:05.373+0000 7f087259c640 1 --2- 172.21.15.39:0/1581074133 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f08600baa90 0x7f08600bcfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:50:05.380 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:05.373+0000 7f0872d9d640 1 --2- 172.21.15.39:0/1902944485 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b6df3c01b0 0x55b6df790b90 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7f08640104a0 tx=0x7f0864010960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=208b4eacb187a80b server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:50:05.380 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:05.373+0000 7f086bfff640 1 -- 172.21.15.39:0/1581074133 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8484+0+0 (secure 0 0 0) 0x7f086c0e7050 con 0x7f0874109df0 2024-09-21T03:50:05.380 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:05.373+0000 7f08697fa640 1 -- 172.21.15.39:0/1902944485 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0864011760 con 0x55b6df3c01b0 2024-09-21T03:50:05.380 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:05.373+0000 7f08697fa640 1 -- 172.21.15.39:0/1902944485 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0864011db0 con 0x55b6df3c01b0 2024-09-21T03:50:05.380 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:05.373+0000 7f08697fa640 1 -- 172.21.15.39:0/1902944485 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f086400ee70 con 0x55b6df3c01b0 2024-09-21T03:50:05.383 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:05.374+0000 7f087f68d740 1 -- 172.21.15.39:0/1902944485 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b6df3c01b0 msgr2=0x55b6df790b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:50:05.383 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:05.374+0000 7f087259c640 1 --2- 172.21.15.39:0/1581074133 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f08600baa90 0x7f08600bcfa0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f085c009230 tx=0x7f085c031400 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:50:05.383 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:05.374+0000 7f087f68d740 1 --2- 172.21.15.39:0/1902944485 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b6df3c01b0 0x55b6df790b90 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7f08640104a0 tx=0x7f0864010960 comp rx=0 tx=0).stop 2024-09-21T03:50:05.383 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:05.374+0000 7f087f68d740 1 -- 172.21.15.39:0/1902944485 shutdown_connections 2024-09-21T03:50:05.383 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:05.374+0000 7f087f68d740 1 --2- 172.21.15.39:0/1902944485 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b6df3c8390 0x55b6df7910d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:05.383 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:05.374+0000 7f087f68d740 1 --2- 172.21.15.39:0/1902944485 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b6df3c01b0 0x55b6df790b90 unknown :-1 s=CLOSED pgs=732 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:05.384 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:05.374+0000 7f087f68d740 1 --2- 172.21.15.39:0/1902944485 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b6df3e9070 0x55b6df790650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:05.384 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:05.374+0000 7f087f68d740 1 -- 172.21.15.39:0/1902944485 >> 172.21.15.39:0/1902944485 conn(0x55b6df3dd460 msgr2=0x55b6df78e540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:50:05.384 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:05.374+0000 7f087f68d740 1 -- 172.21.15.39:0/1902944485 shutdown_connections 2024-09-21T03:50:05.384 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:05.374+0000 7f087f68d740 1 -- 172.21.15.39:0/1902944485 wait complete. 2024-09-21T03:50:05.384 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:05.375+0000 7f087f68d740 1 -- start start 2024-09-21T03:50:05.384 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:05.375+0000 7f087f68d740 1 Processor -- start 2024-09-21T03:50:05.384 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:05.375+0000 7f087f68d740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b6df3c01b0 0x55b6df79bd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:50:05.384 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:05.375+0000 7f087f68d740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b6df3c8390 0x55b6df79c2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:50:05.384 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:05.375+0000 7f087f68d740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b6df3e9070 0x55b6df7a8120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:50:05.384 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:05.375+0000 7f087f68d740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x55b6df557700 con 0x55b6df3c01b0 2024-09-21T03:50:05.384 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:05.375+0000 7f087f68d740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x55b6df691110 con 0x55b6df3c8390 2024-09-21T03:50:05.384 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:05.375+0000 7f087f68d740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55b6df79ee30 con 0x55b6df3e9070 2024-09-21T03:50:05.384 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:05.375+0000 7f087f68d740 10 client.0 _ll_register_callbacks cb 0x55b6df6aa990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:50:05.384 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:05.375+0000 7f0872d9d640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b6df3c01b0 0x55b6df79bd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:50:05.385 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:05.375+0000 7f0872d9d640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b6df3c01b0 0x55b6df79bd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:60284/0 (socket says 172.21.15.39:60284) 2024-09-21T03:50:05.385 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:05.375+0000 7f0872d9d640 1 -- 172.21.15.39:0/898613079 learned_addr learned my addr 172.21.15.39:0/898613079 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:50:05.385 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:05.375+0000 7f087259c640 1 --2- 172.21.15.39:0/898613079 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b6df3c8390 0x55b6df79c2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:50:05.385 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:05.375+0000 7f087359e640 1 --2- 172.21.15.39:0/898613079 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b6df3e9070 0x55b6df7a8120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:50:05.385 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:05.376+0000 7f0872d9d640 1 -- 172.21.15.39:0/898613079 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b6df3e9070 msgr2=0x55b6df7a8120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:50:05.385 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:05.376+0000 7f0872d9d640 1 --2- 172.21.15.39:0/898613079 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b6df3e9070 0x55b6df7a8120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:05.385 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:05.376+0000 7f0872d9d640 1 -- 172.21.15.39:0/898613079 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b6df3c8390 msgr2=0x55b6df79c2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:50:05.385 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:05.376+0000 7f0872d9d640 1 --2- 172.21.15.39:0/898613079 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b6df3c8390 0x55b6df79c2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:05.385 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:05.376+0000 7f0872d9d640 1 -- 172.21.15.39:0/898613079 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b6df6bec40 con 0x55b6df3c01b0 2024-09-21T03:50:05.385 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:05.376+0000 7f087359e640 1 --2- 172.21.15.39:0/898613079 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b6df3e9070 0x55b6df7a8120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:50:05.385 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:05.376+0000 7f087259c640 1 --2- 172.21.15.39:0/898613079 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b6df3c8390 0x55b6df79c2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:50:05.385 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:05.376+0000 7f0872d9d640 1 --2- 172.21.15.39:0/898613079 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b6df3c01b0 0x55b6df79bd70 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7f0864010fd0 tx=0x7f086400e3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=67066de6921a6a0d server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:50:05.385 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:05.376+0000 7f085a7fc640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-09-21T03:50:05.386 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:05.376+0000 7f085a7fc640 1 -- 172.21.15.39:0/898613079 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f086402a060 con 0x55b6df3c01b0 2024-09-21T03:50:05.386 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:05.376+0000 7f085a7fc640 1 -- 172.21.15.39:0/898613079 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8484+0+0 (secure 0 0 0) 0x7f0864004a40 con 0x55b6df3c01b0 2024-09-21T03:50:05.386 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:05.376+0000 7f087f68d740 1 -- 172.21.15.39:0/898613079 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0864011db0 con 0x55b6df3c01b0 2024-09-21T03:50:05.386 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:05.376+0000 7f085a7fc640 1 -- 172.21.15.39:0/898613079 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f0864004c40 con 0x55b6df3c01b0 2024-09-21T03:50:05.386 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:05.376+0000 7f087f68d740 10 client.16446 fetch_fsmap learned FSMap version 29 2024-09-21T03:50:05.386 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:05.376+0000 7f087f68d740 1 -- 172.21.15.39:0/898613079 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55b6df7aab40 con 0x55b6df3c01b0 2024-09-21T03:50:05.386 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:05.376+0000 7f085a7fc640 1 -- 172.21.15.39:0/898613079 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0864011db0 con 0x55b6df3c01b0 2024-09-21T03:50:05.386 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:05.377+0000 7f085a7fc640 1 -- 172.21.15.39:0/898613079 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f08640051e0 con 0x55b6df3c01b0 2024-09-21T03:50:05.386 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:05.377+0000 7f087f68d740 10 client.16446 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T03:50:05.386 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:05.377+0000 7f087f68d740 10 client.16446 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T03:50:05.386 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:05.377+0000 7f087f68d740 20 client.16446 populate_metadata read hostname 'smithi039' 2024-09-21T03:50:05.386 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:05.377+0000 7f087f68d740 1 --2- 172.21.15.39:0/898613079 >> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] conn(0x55b6df3a79f0 0x55b6df7adf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:50:05.386 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:05.377+0000 7f087f68d740 4 client.16446 mds_command: new command op to 24695 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:50:05.387 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:05.377+0000 7f087f68d740 1 -- 172.21.15.39:0/898613079 --> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b6df708da0 con 0x55b6df3a79f0 2024-09-21T03:50:05.387 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:05.377+0000 7f087259c640 1 --2- 172.21.15.39:0/898613079 >> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] conn(0x55b6df3a79f0 0x55b6df7adf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:50:05.387 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:05.377+0000 7f085a7fc640 10 client.16446 ms_handle_connect on v2:172.21.15.150:6838/2949386907 2024-09-21T03:50:05.387 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:05.377+0000 7f087259c640 1 --2- 172.21.15.39:0/898613079 >> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] conn(0x55b6df3a79f0 0x55b6df7adf50 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2aa6322fdfb9c0ec server_cookie=f298593515b3b34c in_seq=0 out_seq=0 2024-09-21T03:50:05.387 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:05.378+0000 7f085a7fc640 1 -- 172.21.15.39:0/898613079 <== mds.0 v2:172.21.15.150:6838/2949386907 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55b6df708da0 con 0x55b6df3a79f0 2024-09-21T03:50:05.387 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:05.378+0000 7f085a7fc640 10 client.16446 handle_command_reply: tid=0 2024-09-21T03:50:05.387 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:05.378+0000 7f087f68d740 1 client.16446 shutdown 2024-09-21T03:50:05.387 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:05.378+0000 7f087f68d740 1 -- 172.21.15.39:0/898613079 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b6df3c01b0 msgr2=0x55b6df79bd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:50:05.387 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:05.378+0000 7f087f68d740 1 --2- 172.21.15.39:0/898613079 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b6df3c01b0 0x55b6df79bd70 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7f0864010fd0 tx=0x7f086400e3c0 comp rx=0 tx=0).stop 2024-09-21T03:50:05.387 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:05.379+0000 7f087f68d740 1 -- 172.21.15.39:0/898613079 shutdown_connections 2024-09-21T03:50:05.387 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:05.379+0000 7f087f68d740 1 --2- 172.21.15.39:0/898613079 >> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] conn(0x55b6df3a79f0 0x55b6df7adf50 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:05.387 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:05.379+0000 7f087f68d740 1 --2- 172.21.15.39:0/898613079 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b6df3e9070 0x55b6df7a8120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:05.387 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:05.379+0000 7f087f68d740 1 --2- 172.21.15.39:0/898613079 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b6df3c8390 0x55b6df79c2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:05.388 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:05.379+0000 7f087f68d740 1 --2- 172.21.15.39:0/898613079 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b6df3c01b0 0x55b6df79bd70 unknown :-1 s=CLOSED pgs=733 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:05.388 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:05.379+0000 7f085a7fc640 0 client.16446 ms_handle_reset on v2:172.21.15.150:6838/2949386907 2024-09-21T03:50:05.388 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:05.379+0000 7f087f68d740 1 -- 172.21.15.39:0/898613079 >> 172.21.15.39:0/898613079 conn(0x55b6df3dd460 msgr2=0x55b6df793830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:50:05.388 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:05.379+0000 7f087f68d740 1 -- 172.21.15.39:0/898613079 shutdown_connections 2024-09-21T03:50:05.388 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:05.379+0000 7f087f68d740 1 -- 172.21.15.39:0/898613079 wait complete. 2024-09-21T03:50:05.388 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:05.379+0000 7f087f68d740 20 client.16446 trim_cache size 0 max 16384 2024-09-21T03:50:05.399 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:05.396+0000 7f087f68d740 1 Processor -- start 2024-09-21T03:50:05.399 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:05.396+0000 7f087f68d740 1 -- start start 2024-09-21T03:50:05.400 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:05.397+0000 7f087f68d740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b6df3c01b0 0x55b6df79ac60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:50:05.400 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:05.397+0000 7f087f68d740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b6df3c8390 0x55b6df793e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:50:05.400 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:05.397+0000 7f087f68d740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b6df3e9070 0x55b6df798f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:50:05.400 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:05.397+0000 7f087f68d740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f08741ad640 con 0x55b6df3e9070 2024-09-21T03:50:05.400 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:05.397+0000 7f087f68d740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f08741af180 con 0x55b6df3c8390 2024-09-21T03:50:05.400 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:05.397+0000 7f087f68d740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f08741add10 con 0x55b6df3c01b0 2024-09-21T03:50:05.400 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:05.397+0000 7f087359e640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b6df3e9070 0x55b6df798f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:50:05.401 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:05.397+0000 7f087359e640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b6df3e9070 0x55b6df798f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:60286/0 (socket says 172.21.15.39:60286) 2024-09-21T03:50:05.401 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:05.397+0000 7f087359e640 1 -- 172.21.15.39:0/2735037334 learned_addr learned my addr 172.21.15.39:0/2735037334 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:50:05.401 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:05.397+0000 7f0872d9d640 1 --2- 172.21.15.39:0/2735037334 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b6df3c01b0 0x55b6df79ac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:50:05.401 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:05.397+0000 7f087259c640 1 --2- 172.21.15.39:0/2735037334 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b6df3c8390 0x55b6df793e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:50:05.401 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:05.397+0000 7f087359e640 1 -- 172.21.15.39:0/2735037334 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b6df3c01b0 msgr2=0x55b6df79ac60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:50:05.401 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:05.397+0000 7f087359e640 1 --2- 172.21.15.39:0/2735037334 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b6df3c01b0 0x55b6df79ac60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:05.401 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:05.397+0000 7f087359e640 1 -- 172.21.15.39:0/2735037334 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b6df3c8390 msgr2=0x55b6df793e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:50:05.401 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:05.397+0000 7f087359e640 1 --2- 172.21.15.39:0/2735037334 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b6df3c8390 0x55b6df793e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:05.401 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:05.397+0000 7f087359e640 1 -- 172.21.15.39:0/2735037334 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b6df7934b0 con 0x55b6df3e9070 2024-09-21T03:50:05.401 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:05.397+0000 7f087359e640 1 --2- 172.21.15.39:0/2735037334 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b6df3e9070 0x55b6df798f60 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7f086c0ace50 tx=0x7f086c0a8e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=90c371605b85876e server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:50:05.401 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:05.397+0000 7f0868ff9640 1 -- 172.21.15.39:0/2735037334 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f086c020040 con 0x55b6df3e9070 2024-09-21T03:50:05.401 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:05.398+0000 7f0868ff9640 1 -- 172.21.15.39:0/2735037334 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f086c0ad5c0 con 0x55b6df3e9070 2024-09-21T03:50:05.401 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:05.398+0000 7f0868ff9640 1 -- 172.21.15.39:0/2735037334 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f086c094300 con 0x55b6df3e9070 2024-09-21T03:50:05.404 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:05.398+0000 7f087f68d740 1 -- 172.21.15.39:0/2735037334 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b6df3e9070 msgr2=0x55b6df798f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:50:05.404 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:05.398+0000 7f087f68d740 1 --2- 172.21.15.39:0/2735037334 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b6df3e9070 0x55b6df798f60 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7f086c0ace50 tx=0x7f086c0a8e90 comp rx=0 tx=0).stop 2024-09-21T03:50:05.404 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:05.398+0000 7f087f68d740 1 -- 172.21.15.39:0/2735037334 shutdown_connections 2024-09-21T03:50:05.404 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:05.398+0000 7f087f68d740 1 --2- 172.21.15.39:0/2735037334 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b6df3e9070 0x55b6df798f60 unknown :-1 s=CLOSED pgs=734 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:05.404 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:05.398+0000 7f087f68d740 1 --2- 172.21.15.39:0/2735037334 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b6df3c8390 0x55b6df793e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:05.404 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:05.398+0000 7f087f68d740 1 --2- 172.21.15.39:0/2735037334 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b6df3c01b0 0x55b6df79ac60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:05.404 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:05.398+0000 7f087f68d740 1 -- 172.21.15.39:0/2735037334 >> 172.21.15.39:0/2735037334 conn(0x55b6df3dd460 msgr2=0x55b6df793750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:50:05.404 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:05.398+0000 7f087f68d740 1 -- 172.21.15.39:0/2735037334 shutdown_connections 2024-09-21T03:50:05.405 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:05.398+0000 7f087f68d740 1 -- 172.21.15.39:0/2735037334 wait complete. 2024-09-21T03:50:05.405 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:05.399+0000 7f087f68d740 1 -- start start 2024-09-21T03:50:05.405 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:05.399+0000 7f087f68d740 1 Processor -- start 2024-09-21T03:50:05.405 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:05.399+0000 7f087f68d740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b6df3c01b0 0x55b6df7b1950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:50:05.405 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:05.399+0000 7f087f68d740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b6df3c8390 0x55b6df7aee40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:50:05.405 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:05.399+0000 7f087f68d740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b6df3e9070 0x55b6df7af550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:50:05.405 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:05.399+0000 7f087f68d740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f08741aeab0 con 0x55b6df3e9070 2024-09-21T03:50:05.405 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:05.399+0000 7f087f68d740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f08741ae3e0 con 0x55b6df3c01b0 2024-09-21T03:50:05.405 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:05.399+0000 7f087f68d740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f08741ab620 con 0x55b6df3c8390 2024-09-21T03:50:05.405 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:05.399+0000 7f087f68d740 10 client.0 _ll_register_callbacks cb 0x7f085c032c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:50:05.405 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:05.399+0000 7f087359e640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b6df3e9070 0x55b6df7af550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:50:05.405 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:05.399+0000 7f087359e640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b6df3e9070 0x55b6df7af550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:60290/0 (socket says 172.21.15.39:60290) 2024-09-21T03:50:05.405 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:05.399+0000 7f087359e640 1 -- 172.21.15.39:0/1419177807 learned_addr learned my addr 172.21.15.39:0/1419177807 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:50:05.405 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:05.399+0000 7f0872d9d640 1 --2- 172.21.15.39:0/1419177807 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b6df3c01b0 0x55b6df7b1950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:50:05.406 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:05.400+0000 7f087259c640 1 --2- 172.21.15.39:0/1419177807 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b6df3c8390 0x55b6df7aee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:50:05.406 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:05.400+0000 7f087359e640 1 -- 172.21.15.39:0/1419177807 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b6df3c8390 msgr2=0x55b6df7aee40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:50:05.406 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:05.400+0000 7f087359e640 1 --2- 172.21.15.39:0/1419177807 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b6df3c8390 0x55b6df7aee40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:05.406 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:05.400+0000 7f087359e640 1 -- 172.21.15.39:0/1419177807 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b6df3c01b0 msgr2=0x55b6df7b1950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:50:05.406 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:05.400+0000 7f087359e640 1 --2- 172.21.15.39:0/1419177807 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b6df3c01b0 0x55b6df7b1950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:05.406 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:05.400+0000 7f087359e640 1 -- 172.21.15.39:0/1419177807 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b6df8176d0 con 0x55b6df3e9070 2024-09-21T03:50:05.406 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:05.400+0000 7f087359e640 1 --2- 172.21.15.39:0/1419177807 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b6df3e9070 0x55b6df7af550 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7f086c009610 tx=0x7f086c0e4730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f4a3f84c9f82faa server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:50:05.406 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:05.400+0000 7f085a7fc640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-09-21T03:50:05.406 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:05.400+0000 7f085a7fc640 1 -- 172.21.15.39:0/1419177807 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f086c0d4020 con 0x55b6df3e9070 2024-09-21T03:50:05.406 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:05.400+0000 7f085a7fc640 1 -- 172.21.15.39:0/1419177807 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8484+0+0 (secure 0 0 0) 0x7f086c0940c0 con 0x55b6df3e9070 2024-09-21T03:50:05.406 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:05.400+0000 7f087f68d740 1 -- 172.21.15.39:0/1419177807 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f086c0ad5c0 con 0x55b6df3e9070 2024-09-21T03:50:05.406 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:05.400+0000 7f085a7fc640 1 -- 172.21.15.39:0/1419177807 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f086c020070 con 0x55b6df3e9070 2024-09-21T03:50:05.406 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:05.400+0000 7f087f68d740 10 client.16452 fetch_fsmap learned FSMap version 29 2024-09-21T03:50:05.406 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:05.400+0000 7f087f68d740 1 -- 172.21.15.39:0/1419177807 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55b6df818810 con 0x55b6df3e9070 2024-09-21T03:50:05.407 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:05.400+0000 7f085a7fc640 1 -- 172.21.15.39:0/1419177807 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f086c0ad5c0 con 0x55b6df3e9070 2024-09-21T03:50:05.407 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:05.401+0000 7f085a7fc640 1 -- 172.21.15.39:0/1419177807 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f086c0d41c0 con 0x55b6df3e9070 2024-09-21T03:50:05.407 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:05.401+0000 7f087f68d740 10 client.16452 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T03:50:05.407 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:05.401+0000 7f087f68d740 10 client.16452 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T03:50:05.407 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:05.401+0000 7f087f68d740 20 client.16452 populate_metadata read hostname 'smithi039' 2024-09-21T03:50:05.407 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:05.401+0000 7f087f68d740 1 --2- 172.21.15.39:0/1419177807 >> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] conn(0x55b6df37cda0 0x55b6df81aef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:50:05.407 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:05.401+0000 7f087f68d740 4 client.16452 mds_command: new command op to 24695 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-21T03:50:05.407 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:05.401+0000 7f087f68d740 1 -- 172.21.15.39:0/1419177807 --> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55b6df704850 con 0x55b6df37cda0 2024-09-21T03:50:05.407 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:05.401+0000 7f0872d9d640 1 --2- 172.21.15.39:0/1419177807 >> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] conn(0x55b6df37cda0 0x55b6df81aef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:50:05.407 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:05.401+0000 7f085a7fc640 10 client.16452 ms_handle_connect on v2:172.21.15.150:6838/2949386907 2024-09-21T03:50:05.407 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:05.401+0000 7f0872d9d640 1 --2- 172.21.15.39:0/1419177807 >> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] conn(0x55b6df37cda0 0x55b6df81aef0 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=45039910ca5efda8 server_cookie=2f36d0ea32c0d55e in_seq=0 out_seq=0 2024-09-21T03:50:05.408 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:05.402+0000 7f085a7fc640 1 -- 172.21.15.39:0/1419177807 <== mds.0 v2:172.21.15.150:6838/2949386907 1 ==== command_reply(tid 0: 0 ) ==== 8+0+2078 (crc 0 0 0) 0x55b6df704850 con 0x55b6df37cda0 2024-09-21T03:50:05.408 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:05.402+0000 7f085a7fc640 10 client.16452 handle_command_reply: tid=0 2024-09-21T03:50:05.408 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:05.402+0000 7f087f68d740 1 client.16452 shutdown 2024-09-21T03:50:05.408 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:05.402+0000 7f087f68d740 1 -- 172.21.15.39:0/1419177807 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b6df3e9070 msgr2=0x55b6df7af550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:50:05.409 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:05.402+0000 7f087f68d740 1 --2- 172.21.15.39:0/1419177807 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b6df3e9070 0x55b6df7af550 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7f086c009610 tx=0x7f086c0e4730 comp rx=0 tx=0).stop 2024-09-21T03:50:05.409 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:05.402+0000 7f087f68d740 1 -- 172.21.15.39:0/1419177807 shutdown_connections 2024-09-21T03:50:05.409 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:05.402+0000 7f087f68d740 1 --2- 172.21.15.39:0/1419177807 >> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] conn(0x55b6df37cda0 0x55b6df81aef0 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:05.409 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:05.402+0000 7f087f68d740 1 --2- 172.21.15.39:0/1419177807 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b6df3e9070 0x55b6df7af550 unknown :-1 s=CLOSED pgs=735 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:05.409 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:05.402+0000 7f087f68d740 1 --2- 172.21.15.39:0/1419177807 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b6df3c8390 0x55b6df7aee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:05.409 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:05.402+0000 7f087f68d740 1 --2- 172.21.15.39:0/1419177807 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b6df3c01b0 0x55b6df7b1950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:05.409 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:05.402+0000 7f087f68d740 1 -- 172.21.15.39:0/1419177807 >> 172.21.15.39:0/1419177807 conn(0x55b6df3dd460 msgr2=0x55b6df791680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:50:05.409 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:05.402+0000 7f085a7fc640 0 client.16452 ms_handle_reset on v2:172.21.15.150:6838/2949386907 2024-09-21T03:50:05.409 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:05.402+0000 7f087f68d740 1 -- 172.21.15.39:0/1419177807 shutdown_connections 2024-09-21T03:50:05.409 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:05.402+0000 7f087f68d740 1 -- 172.21.15.39:0/1419177807 wait complete. 2024-09-21T03:50:05.409 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:05.402+0000 7f087f68d740 20 client.16452 trim_cache size 0 max 16384 2024-09-21T03:50:05.409 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-09-21T03:50:05.410 INFO:teuthology.orchestra.run.smithi039.stdout: "status": "no active scrubs running", 2024-09-21T03:50:05.410 INFO:teuthology.orchestra.run.smithi039.stdout: "scrubs": {}, 2024-09-21T03:50:05.410 INFO:teuthology.orchestra.run.smithi039.stdout: "scrub_stats": { 2024-09-21T03:50:05.410 INFO:teuthology.orchestra.run.smithi039.stdout: "6e760f1f-9048-4f92-bd66-d708fbef75b3": { 2024-09-21T03:50:05.410 INFO:teuthology.orchestra.run.smithi039.stdout: "start_time": "2024-09-21T03:50:03.711918+0000", 2024-09-21T03:50:05.410 INFO:teuthology.orchestra.run.smithi039.stdout: "path": "/", 2024-09-21T03:50:05.410 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_started": 0, 2024-09-21T03:50:05.410 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_passed": 0, 2024-09-21T03:50:05.410 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_failed": 0, 2024-09-21T03:50:05.410 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_skipped": 0 2024-09-21T03:50:05.410 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:50:05.410 INFO:teuthology.orchestra.run.smithi039.stdout: "e28ad743-2b22-45b3-8799-a638c2883623": { 2024-09-21T03:50:05.410 INFO:teuthology.orchestra.run.smithi039.stdout: "start_time": "2024-09-21T03:48:31.448821+0000", 2024-09-21T03:50:05.411 INFO:teuthology.orchestra.run.smithi039.stdout: "path": "/", 2024-09-21T03:50:05.411 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_started": 0, 2024-09-21T03:50:05.411 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_passed": 0, 2024-09-21T03:50:05.411 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_failed": 0, 2024-09-21T03:50:05.411 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_skipped": 0 2024-09-21T03:50:05.411 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:50:05.411 INFO:teuthology.orchestra.run.smithi039.stdout: "29411307-eb8d-4339-8d2b-47ebd32c0080": { 2024-09-21T03:50:05.411 INFO:teuthology.orchestra.run.smithi039.stdout: "start_time": "2024-09-21T03:47:38.092920+0000", 2024-09-21T03:50:05.411 INFO:teuthology.orchestra.run.smithi039.stdout: "path": "/", 2024-09-21T03:50:05.411 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_started": 0, 2024-09-21T03:50:05.411 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_passed": 0, 2024-09-21T03:50:05.411 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_failed": 0, 2024-09-21T03:50:05.411 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_skipped": 0 2024-09-21T03:50:05.411 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:50:05.412 INFO:teuthology.orchestra.run.smithi039.stdout: "9860c2f4-b7b1-47aa-865a-9a1c4434a5f6": { 2024-09-21T03:50:05.412 INFO:teuthology.orchestra.run.smithi039.stdout: "start_time": "2024-09-21T03:49:17.718867+0000", 2024-09-21T03:50:05.412 INFO:teuthology.orchestra.run.smithi039.stdout: "path": "/", 2024-09-21T03:50:05.412 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_started": 0, 2024-09-21T03:50:05.412 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_passed": 0, 2024-09-21T03:50:05.412 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_failed": 0, 2024-09-21T03:50:05.412 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_skipped": 0 2024-09-21T03:50:05.412 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:50:05.412 INFO:teuthology.orchestra.run.smithi039.stdout: "c7e2b8e4-057d-435f-b2bd-b633de98035f": { 2024-09-21T03:50:05.412 INFO:teuthology.orchestra.run.smithi039.stdout: "start_time": "2024-09-21T03:47:19.400305+0000", 2024-09-21T03:50:05.412 INFO:teuthology.orchestra.run.smithi039.stdout: "path": "/", 2024-09-21T03:50:05.412 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_started": 0, 2024-09-21T03:50:05.412 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_passed": 0, 2024-09-21T03:50:05.412 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_failed": 0, 2024-09-21T03:50:05.412 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_skipped": 0 2024-09-21T03:50:05.413 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:50:05.413 INFO:teuthology.orchestra.run.smithi039.stdout: "10728de8-c49d-4edc-be2b-639249255e94": { 2024-09-21T03:50:05.413 INFO:teuthology.orchestra.run.smithi039.stdout: "start_time": "2024-09-21T03:47:03.965623+0000", 2024-09-21T03:50:05.413 INFO:teuthology.orchestra.run.smithi039.stdout: "path": "/", 2024-09-21T03:50:05.413 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_started": 0, 2024-09-21T03:50:05.413 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_passed": 0, 2024-09-21T03:50:05.413 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_failed": 0, 2024-09-21T03:50:05.413 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_skipped": 0 2024-09-21T03:50:05.413 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:50:05.413 INFO:teuthology.orchestra.run.smithi039.stdout: "02509e4f-3911-4983-b16f-df8ded0b9998": { 2024-09-21T03:50:05.413 INFO:teuthology.orchestra.run.smithi039.stdout: "start_time": "2024-09-21T03:46:17.889576+0000", 2024-09-21T03:50:05.413 INFO:teuthology.orchestra.run.smithi039.stdout: "path": "/", 2024-09-21T03:50:05.413 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_started": 0, 2024-09-21T03:50:05.413 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_passed": 0, 2024-09-21T03:50:05.413 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_failed": 0, 2024-09-21T03:50:05.414 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_skipped": 0 2024-09-21T03:50:05.414 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-21T03:50:05.414 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-21T03:50:05.414 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-09-21T03:50:05.414 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:05.403+0000 7f0879119640 1 -- 172.21.15.39:0/1581074133 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f08600baa90 msgr2=0x7f08600bcfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:50:05.414 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:05.403+0000 7f0879119640 1 --2- 172.21.15.39:0/1581074133 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f08600baa90 0x7f08600bcfa0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f085c009230 tx=0x7f085c031400 comp rx=0 tx=0).stop 2024-09-21T03:50:05.415 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:05.403+0000 7f0879119640 1 -- 172.21.15.39:0/1581074133 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0874109df0 msgr2=0x7f087419fdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:50:05.415 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:05.403+0000 7f0879119640 1 --2- 172.21.15.39:0/1581074133 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0874109df0 0x7f087419fdc0 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7f086c00d750 tx=0x7f086c00dc10 comp rx=0 tx=0).stop 2024-09-21T03:50:05.415 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:05.403+0000 7f0879119640 1 -- 172.21.15.39:0/1581074133 shutdown_connections 2024-09-21T03:50:05.415 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:05.403+0000 7f0879119640 1 --2- 172.21.15.39:0/1581074133 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f08600baa90 0x7f08600bcfa0 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:05.415 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:05.403+0000 7f0879119640 1 --2- 172.21.15.39:0/1581074133 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0874109df0 0x7f087419fdc0 unknown :-1 s=CLOSED pgs=731 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:05.415 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:05.403+0000 7f0879119640 1 --2- 172.21.15.39:0/1581074133 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f08741094a0 0x7f087419ab20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:05.415 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:05.403+0000 7f0879119640 1 --2- 172.21.15.39:0/1581074133 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0874103b60 0x7f087419a580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:05.415 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:05.403+0000 7f0879119640 1 -- 172.21.15.39:0/1581074133 >> 172.21.15.39:0/1581074133 conn(0x7f087407b850 msgr2=0x7f0874105580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:50:05.415 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:05.403+0000 7f0879119640 1 -- 172.21.15.39:0/1581074133 shutdown_connections 2024-09-21T03:50:05.415 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:05.403+0000 7f0879119640 1 -- 172.21.15.39:0/1581074133 wait complete. 2024-09-21T03:50:06.037 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:50:05 smithi039 ceph-mon[31211]: scrub summary: idle 2024-09-21T03:50:06.066 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-21T03:50:06.067 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 51116d5c-77ca-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T03:50:06.218 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:50:05 smithi161 ceph-mon[34533]: scrub summary: idle 2024-09-21T03:50:06.224 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:50:05 smithi150 ceph-mon[34289]: scrub summary: idle 2024-09-21T03:50:06.325 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/51116d5c-77ca-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:50:06.803 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:06.800+0000 7f1f23da0640 1 -- 172.21.15.39:0/2412332493 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1f1c1049b0 msgr2=0x7f1f1c104dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:50:06.803 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:06.800+0000 7f1f23da0640 1 --2- 172.21.15.39:0/2412332493 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1f1c1049b0 0x7f1f1c104dc0 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7f1f18009900 tx=0x7f1f1802a410 comp rx=0 tx=0).stop 2024-09-21T03:50:06.803 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:06.800+0000 7f1f23da0640 1 -- 172.21.15.39:0/2412332493 shutdown_connections 2024-09-21T03:50:06.803 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:06.800+0000 7f1f23da0640 1 --2- 172.21.15.39:0/2412332493 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1f1c10de10 0x7f1f1c110300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:06.803 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:06.800+0000 7f1f23da0640 1 --2- 172.21.15.39:0/2412332493 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1f1c105300 0x7f1f1c10d850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:06.804 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:06.800+0000 7f1f23da0640 1 --2- 172.21.15.39:0/2412332493 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1f1c1049b0 0x7f1f1c104dc0 unknown :-1 s=CLOSED pgs=736 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:06.804 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:06.800+0000 7f1f23da0640 1 -- 172.21.15.39:0/2412332493 >> 172.21.15.39:0/2412332493 conn(0x7f1f1c079730 msgr2=0x7f1f1c079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:50:06.804 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:06.800+0000 7f1f23da0640 1 -- 172.21.15.39:0/2412332493 shutdown_connections 2024-09-21T03:50:06.804 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:06.800+0000 7f1f23da0640 1 -- 172.21.15.39:0/2412332493 wait complete. 2024-09-21T03:50:06.805 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:06.801+0000 7f1f23da0640 1 Processor -- start 2024-09-21T03:50:06.805 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:06.801+0000 7f1f23da0640 1 -- start start 2024-09-21T03:50:06.805 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:06.801+0000 7f1f23da0640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1f1c1049b0 0x7f1f1c1026f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:50:06.806 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:06.801+0000 7f1f23da0640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1f1c105300 0x7f1f1c102c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:50:06.806 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:06.801+0000 7f1f23da0640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1f1c10de10 0x7f1f1c103170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:50:06.806 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:06.801+0000 7f1f23da0640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f1f1c113a70 con 0x7f1f1c105300 2024-09-21T03:50:06.806 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:06.801+0000 7f1f23da0640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f1f1c1138f0 con 0x7f1f1c10de10 2024-09-21T03:50:06.806 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:06.801+0000 7f1f23da0640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f1f1c113bf0 con 0x7f1f1c1049b0 2024-09-21T03:50:06.806 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:06.801+0000 7f1f2133b640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1f1c105300 0x7f1f1c102c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:50:06.806 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:06.801+0000 7f1f2133b640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1f1c105300 0x7f1f1c102c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:60306/0 (socket says 172.21.15.39:60306) 2024-09-21T03:50:06.806 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:06.801+0000 7f1f2133b640 1 -- 172.21.15.39:0/2846651676 learned_addr learned my addr 172.21.15.39:0/2846651676 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:50:06.806 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:06.801+0000 7f1f21b3c640 1 --2- 172.21.15.39:0/2846651676 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1f1c1049b0 0x7f1f1c1026f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:50:06.806 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:06.801+0000 7f1f2233d640 1 --2- 172.21.15.39:0/2846651676 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1f1c10de10 0x7f1f1c103170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:50:06.806 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:06.802+0000 7f1f2133b640 1 -- 172.21.15.39:0/2846651676 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1f1c1049b0 msgr2=0x7f1f1c1026f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:50:06.806 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:06.802+0000 7f1f2133b640 1 --2- 172.21.15.39:0/2846651676 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1f1c1049b0 0x7f1f1c1026f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:06.806 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:06.802+0000 7f1f2133b640 1 -- 172.21.15.39:0/2846651676 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1f1c10de10 msgr2=0x7f1f1c103170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:50:06.807 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:06.802+0000 7f1f2133b640 1 --2- 172.21.15.39:0/2846651676 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1f1c10de10 0x7f1f1c103170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:06.807 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:06.802+0000 7f1f2133b640 1 -- 172.21.15.39:0/2846651676 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1f1c1a8730 con 0x7f1f1c105300 2024-09-21T03:50:06.807 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:06.802+0000 7f1f2133b640 1 --2- 172.21.15.39:0/2846651676 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1f1c105300 0x7f1f1c102c30 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7f1f0c00c8c0 tx=0x7f1f0c00cd80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:50:06.807 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:06.802+0000 7f1f12ffd640 1 -- 172.21.15.39:0/2846651676 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f0c007bb0 con 0x7f1f1c105300 2024-09-21T03:50:06.807 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:06.802+0000 7f1f12ffd640 1 -- 172.21.15.39:0/2846651676 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1f0c007d50 con 0x7f1f1c105300 2024-09-21T03:50:06.807 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:06.802+0000 7f1f23da0640 1 -- 172.21.15.39:0/2846651676 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1f1c1a8a20 con 0x7f1f1c105300 2024-09-21T03:50:06.807 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:06.802+0000 7f1f12ffd640 1 -- 172.21.15.39:0/2846651676 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f0c0187c0 con 0x7f1f1c105300 2024-09-21T03:50:06.807 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:06.802+0000 7f1f23da0640 1 -- 172.21.15.39:0/2846651676 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1f1c1a8f60 con 0x7f1f1c105300 2024-09-21T03:50:06.808 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:06.803+0000 7f1f23da0640 1 -- 172.21.15.39:0/2846651676 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1f1c104dc0 con 0x7f1f1c105300 2024-09-21T03:50:06.808 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:06.805+0000 7f1f12ffd640 1 -- 172.21.15.39:0/2846651676 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 27) ==== 154643+0+0 (secure 0 0 0) 0x7f1f0c00fa20 con 0x7f1f1c105300 2024-09-21T03:50:06.808 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:06.805+0000 7f1f12ffd640 1 --2- 172.21.15.39:0/2846651676 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f1f080b6680 0x7f1f080b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:50:06.808 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:06.806+0000 7f1f21b3c640 1 --2- 172.21.15.39:0/2846651676 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f1f080b6680 0x7f1f080b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:50:06.808 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:06.806+0000 7f1f12ffd640 1 -- 172.21.15.39:0/2846651676 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8484+0+0 (secure 0 0 0) 0x7f1f0c0e7b90 con 0x7f1f1c105300 2024-09-21T03:50:06.809 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:06.806+0000 7f1f21b3c640 1 --2- 172.21.15.39:0/2846651676 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f1f080b6680 0x7f1f080b8b90 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f1f18009230 tx=0x7f1f18031400 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:50:06.811 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:06.809+0000 7f1f12ffd640 1 -- 172.21.15.39:0/2846651676 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f1f0c014030 con 0x7f1f1c105300 2024-09-21T03:50:07.038 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:07.035+0000 7f1f23da0640 1 -- 172.21.15.39:0/2846651676 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1f1c1039f0 con 0x7f1f1c105300 2024-09-21T03:50:07.038 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:50:06 smithi039 ceph-mon[31211]: pgmap v447: 97 pgs: 97 active+clean; 10 MiB data, 536 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 25 op/s 2024-09-21T03:50:07.038 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:07.036+0000 7f1f12ffd640 1 -- 172.21.15.39:0/2846651676 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11978 (secure 0 0 0) 0x7f1f0c0ae190 con 0x7f1f1c105300 2024-09-21T03:50:07.039 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T03:50:07.039 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":29,"btime":"2024-09-21T03:46:08:982881+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14790,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6835/508995295","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":508995295},{"type":"v1","addr":"172.21.15.39:6835","nonce":508995295}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14802,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6837/3915912607","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":3915912607},{"type":"v1","addr":"172.21.15.39:6837","nonce":3915912607}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":29,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T03:44:15.009872+0000","modified":"2024-09-21T03:46:08.430113+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24695,"mds_1":24667,"mds_2":24659,"mds_3":24646,"mds_4":14868},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14856":{"gid":14856,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.39:6839/185509192","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6838","nonce":185509192},{"type":"v1","addr":"172.21.15.39:6839","nonce":185509192}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14868":{"gid":14868,"name":"a","rank":4,"incarnation":25,"state":"up:active","state_seq":7,"addr":"172.21.15.39:6841/4247190456","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6840","nonce":4247190456},{"type":"v1","addr":"172.21.15.39:6841","nonce":4247190456}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24599":{"gid":24599,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.150:6833/3114643217","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":3114643217},{"type":"v1","addr":"172.21.15.150:6833","nonce":3114643217}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24601":{"gid":24601,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.161:6833/2963990654","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":2963990654},{"type":"v1","addr":"172.21.15.161:6833","nonce":2963990654}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24632":{"gid":24632,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.150:6835/724594008","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6834","nonce":724594008},{"type":"v1","addr":"172.21.15.150:6835","nonce":724594008}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24640":{"gid":24640,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.161:6835/3747882110","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":3747882110},{"type":"v1","addr":"172.21.15.161:6835","nonce":3747882110}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24646":{"gid":24646,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":10,"addr":"172.21.15.161:6837/3451544597","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6836","nonce":3451544597},{"type":"v1","addr":"172.21.15.161:6837","nonce":3451544597}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24659":{"gid":24659,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.150:6837/4017043144","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6836","nonce":4017043144},{"type":"v1","addr":"172.21.15.150:6837","nonce":4017043144}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24667":{"gid":24667,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.161:6839/2190107270","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6838","nonce":2190107270},{"type":"v1","addr":"172.21.15.161:6839","nonce":2190107270}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24695":{"gid":24695,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.150:6839/2949386907","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6838","nonce":2949386907},{"type":"v1","addr":"172.21.15.150:6839","nonce":2949386907}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24695,"qdb_cluster":[24695,24667,24659,24646,14868]},"id":1}]} 2024-09-21T03:50:07.039 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 29 2024-09-21T03:50:07.041 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:07.038+0000 7f1f23da0640 1 -- 172.21.15.39:0/2846651676 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f1f080b6680 msgr2=0x7f1f080b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:50:07.041 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:07.038+0000 7f1f23da0640 1 --2- 172.21.15.39:0/2846651676 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f1f080b6680 0x7f1f080b8b90 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f1f18009230 tx=0x7f1f18031400 comp rx=0 tx=0).stop 2024-09-21T03:50:07.041 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:07.038+0000 7f1f23da0640 1 -- 172.21.15.39:0/2846651676 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1f1c105300 msgr2=0x7f1f1c102c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:50:07.041 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:07.038+0000 7f1f23da0640 1 --2- 172.21.15.39:0/2846651676 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1f1c105300 0x7f1f1c102c30 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7f1f0c00c8c0 tx=0x7f1f0c00cd80 comp rx=0 tx=0).stop 2024-09-21T03:50:07.041 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:07.038+0000 7f1f23da0640 1 -- 172.21.15.39:0/2846651676 shutdown_connections 2024-09-21T03:50:07.041 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:07.038+0000 7f1f23da0640 1 --2- 172.21.15.39:0/2846651676 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f1f080b6680 0x7f1f080b8b90 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:07.041 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:07.038+0000 7f1f23da0640 1 --2- 172.21.15.39:0/2846651676 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1f1c10de10 0x7f1f1c103170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:07.041 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:07.038+0000 7f1f23da0640 1 --2- 172.21.15.39:0/2846651676 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1f1c105300 0x7f1f1c102c30 unknown :-1 s=CLOSED pgs=737 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:07.042 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:07.038+0000 7f1f23da0640 1 --2- 172.21.15.39:0/2846651676 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1f1c1049b0 0x7f1f1c1026f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:07.042 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:07.038+0000 7f1f23da0640 1 -- 172.21.15.39:0/2846651676 >> 172.21.15.39:0/2846651676 conn(0x7f1f1c079730 msgr2=0x7f1f1c0ff790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:50:07.042 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:07.038+0000 7f1f23da0640 1 -- 172.21.15.39:0/2846651676 shutdown_connections 2024-09-21T03:50:07.042 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:07.038+0000 7f1f23da0640 1 -- 172.21.15.39:0/2846651676 wait complete. 2024-09-21T03:50:07.218 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:50:06 smithi161 ceph-mon[34533]: pgmap v447: 97 pgs: 97 active+clean; 10 MiB data, 536 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 25 op/s 2024-09-21T03:50:07.224 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:50:06 smithi150 ceph-mon[34289]: pgmap v447: 97 pgs: 97 active+clean; 10 MiB data, 536 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 25 op/s 2024-09-21T03:50:07.828 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 51116d5c-77ca-11ef-baf5-efdc52797490 -- ceph tell mds.1:4 damage ls 2024-09-21T03:50:08.084 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:50:07 smithi039 ceph-mon[31211]: from='client.? 172.21.15.39:0/2846651676' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:50:08.089 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/51116d5c-77ca-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:50:08.218 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:50:07 smithi161 ceph-mon[34533]: from='client.? 172.21.15.39:0/2846651676' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:50:08.224 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:50:07 smithi150 ceph-mon[34289]: from='client.? 172.21.15.39:0/2846651676' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:50:08.580 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:08.577+0000 7fef169dd640 1 -- 172.21.15.39:0/1950102178 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fef101084c0 msgr2=0x7fef1010ecf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:50:08.580 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:08.577+0000 7fef169dd640 1 --2- 172.21.15.39:0/1950102178 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fef101084c0 0x7fef1010ecf0 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7fef08009ca0 tx=0x7fef0802a8c0 comp rx=0 tx=0).stop 2024-09-21T03:50:08.580 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:08.577+0000 7fef169dd640 1 -- 172.21.15.39:0/1950102178 shutdown_connections 2024-09-21T03:50:08.581 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:08.577+0000 7fef169dd640 1 --2- 172.21.15.39:0/1950102178 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fef101084c0 0x7fef1010ecf0 unknown :-1 s=CLOSED pgs=738 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:08.581 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:08.577+0000 7fef169dd640 1 --2- 172.21.15.39:0/1950102178 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fef10107b70 0x7fef10107f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:08.581 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:08.577+0000 7fef169dd640 1 --2- 172.21.15.39:0/1950102178 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fef10102230 0x7fef10102640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:08.581 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:08.577+0000 7fef169dd640 1 -- 172.21.15.39:0/1950102178 >> 172.21.15.39:0/1950102178 conn(0x7fef10079e40 msgr2=0x7fef1007a250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:50:08.581 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:08.577+0000 7fef169dd640 1 -- 172.21.15.39:0/1950102178 shutdown_connections 2024-09-21T03:50:08.581 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:08.577+0000 7fef169dd640 1 -- 172.21.15.39:0/1950102178 wait complete. 2024-09-21T03:50:08.594 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:08.578+0000 7fef169dd640 1 Processor -- start 2024-09-21T03:50:08.594 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:08.578+0000 7fef169dd640 1 -- start start 2024-09-21T03:50:08.594 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:08.578+0000 7fef169dd640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fef10102230 0x7fef100fee70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:50:08.594 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:08.578+0000 7fef169dd640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fef10107b70 0x7fef100ff3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:50:08.594 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:08.578+0000 7fef169dd640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fef101084c0 0x7fef100ff8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:50:08.595 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:08.578+0000 7fef169dd640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fef10111900 con 0x7fef10107b70 2024-09-21T03:50:08.595 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:08.578+0000 7fef169dd640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fef10111780 con 0x7fef10102230 2024-09-21T03:50:08.595 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:08.578+0000 7fef169dd640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fef10111a80 con 0x7fef101084c0 2024-09-21T03:50:08.595 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:08.578+0000 7fef0f7fe640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fef10107b70 0x7fef100ff3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:50:08.595 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:08.578+0000 7fef0f7fe640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fef10107b70 0x7fef100ff3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:56596/0 (socket says 172.21.15.39:56596) 2024-09-21T03:50:08.595 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:08.578+0000 7fef0f7fe640 1 -- 172.21.15.39:0/497547884 learned_addr learned my addr 172.21.15.39:0/497547884 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:50:08.596 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:08.579+0000 7fef0ffff640 1 --2- 172.21.15.39:0/497547884 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fef10102230 0x7fef100fee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:50:08.596 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:08.579+0000 7fef14f7a640 1 --2- 172.21.15.39:0/497547884 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fef101084c0 0x7fef100ff8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:50:08.596 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:08.579+0000 7fef0f7fe640 1 -- 172.21.15.39:0/497547884 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fef101084c0 msgr2=0x7fef100ff8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:50:08.596 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:08.579+0000 7fef0f7fe640 1 --2- 172.21.15.39:0/497547884 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fef101084c0 0x7fef100ff8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:08.596 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:08.579+0000 7fef0f7fe640 1 -- 172.21.15.39:0/497547884 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fef10102230 msgr2=0x7fef100fee70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:50:08.596 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:08.579+0000 7fef0f7fe640 1 --2- 172.21.15.39:0/497547884 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fef10102230 0x7fef100fee70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:08.597 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:08.579+0000 7fef0f7fe640 1 -- 172.21.15.39:0/497547884 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fef101a8870 con 0x7fef10107b70 2024-09-21T03:50:08.597 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:08.579+0000 7fef0f7fe640 1 --2- 172.21.15.39:0/497547884 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fef10107b70 0x7fef100ff3b0 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7fef0400c970 tx=0x7fef0400ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:50:08.597 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:08.579+0000 7fef0d7fa640 1 -- 172.21.15.39:0/497547884 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef04007bb0 con 0x7fef10107b70 2024-09-21T03:50:08.597 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:08.579+0000 7fef0d7fa640 1 -- 172.21.15.39:0/497547884 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fef04007d50 con 0x7fef10107b70 2024-09-21T03:50:08.597 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:08.579+0000 7fef169dd640 1 -- 172.21.15.39:0/497547884 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fef101a8b60 con 0x7fef10107b70 2024-09-21T03:50:08.597 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:08.579+0000 7fef0d7fa640 1 -- 172.21.15.39:0/497547884 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef0400f450 con 0x7fef10107b70 2024-09-21T03:50:08.597 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:08.579+0000 7fef169dd640 1 -- 172.21.15.39:0/497547884 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fef101a90a0 con 0x7fef10107b70 2024-09-21T03:50:08.598 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:08.582+0000 7fef1cf51740 1 Processor -- start 2024-09-21T03:50:08.598 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:08.582+0000 7fef1cf51740 1 -- start start 2024-09-21T03:50:08.598 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:08.582+0000 7fef1cf51740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5631cad92070 0x5631cb1396a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:50:08.598 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:08.582+0000 7fef1cf51740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5631cad691b0 0x5631cb139be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:50:08.598 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:08.582+0000 7fef1cf51740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5631cad71390 0x5631cb13a120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:50:08.598 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:08.582+0000 7fef1cf51740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x5631cb0afea0 con 0x5631cad92070 2024-09-21T03:50:08.599 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:08.582+0000 7fef1cf51740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x5631cb09f2d0 con 0x5631cad691b0 2024-09-21T03:50:08.599 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:08.582+0000 7fef1cf51740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x5631cb0a30e0 con 0x5631cad71390 2024-09-21T03:50:08.599 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:08.582+0000 7fef14f7a640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5631cad92070 0x5631cb1396a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:50:08.599 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:08.582+0000 7fef14f7a640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5631cad92070 0x5631cb1396a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:56606/0 (socket says 172.21.15.39:56606) 2024-09-21T03:50:08.599 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:08.582+0000 7fef14f7a640 1 -- 172.21.15.39:0/1239339622 learned_addr learned my addr 172.21.15.39:0/1239339622 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:50:08.599 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:08.583+0000 7fef0d7fa640 1 -- 172.21.15.39:0/497547884 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 27) ==== 154643+0+0 (secure 0 0 0) 0x7fef04005020 con 0x7fef10107b70 2024-09-21T03:50:08.600 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:08.583+0000 7fef0ffff640 1 --2- 172.21.15.39:0/1239339622 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5631cad691b0 0x5631cb139be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:50:08.600 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:08.583+0000 7fef14f7a640 1 -- 172.21.15.39:0/1239339622 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5631cad71390 msgr2=0x5631cb13a120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:50:08.600 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:08.583+0000 7fef14f7a640 1 --2- 172.21.15.39:0/1239339622 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5631cad71390 0x5631cb13a120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:08.600 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:08.583+0000 7fef14f7a640 1 -- 172.21.15.39:0/1239339622 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5631cad691b0 msgr2=0x5631cb139be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:50:08.600 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:08.583+0000 7fef14f7a640 1 --2- 172.21.15.39:0/1239339622 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5631cad691b0 0x5631cb139be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:08.600 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:08.583+0000 7fef14f7a640 1 -- 172.21.15.39:0/1239339622 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5631cadfd650 con 0x5631cad92070 2024-09-21T03:50:08.601 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:08.583+0000 7fef14f7a640 1 --2- 172.21.15.39:0/1239339622 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5631cad92070 0x5631cb1396a0 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7fef08009920 tx=0x7fef08009950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=76ac071c555fe91 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:50:08.601 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:08.583+0000 7feef67fc640 1 -- 172.21.15.39:0/1239339622 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef08031e30 con 0x5631cad92070 2024-09-21T03:50:08.601 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:08.583+0000 7feef67fc640 1 -- 172.21.15.39:0/1239339622 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fef08036030 con 0x5631cad92070 2024-09-21T03:50:08.601 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:08.583+0000 7feef67fc640 1 -- 172.21.15.39:0/1239339622 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef080042a0 con 0x5631cad92070 2024-09-21T03:50:08.601 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:08.583+0000 7fef0d7fa640 1 --2- 172.21.15.39:0/497547884 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7feef80b6660 0x7feef80b8b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:50:08.601 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:08.584+0000 7fef0ffff640 1 --2- 172.21.15.39:0/497547884 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7feef80b6660 0x7feef80b8b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:50:08.601 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:08.584+0000 7fef0d7fa640 1 -- 172.21.15.39:0/497547884 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8484+0+0 (secure 0 0 0) 0x7fef040e7310 con 0x7fef10107b70 2024-09-21T03:50:08.602 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:08.584+0000 7fef1cf51740 1 -- 172.21.15.39:0/1239339622 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5631cad92070 msgr2=0x5631cb1396a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:50:08.602 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:08.584+0000 7fef0ffff640 1 --2- 172.21.15.39:0/497547884 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7feef80b6660 0x7feef80b8b70 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x5631cb138320 tx=0x7fef0000b390 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:50:08.602 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:08.584+0000 7fef1cf51740 1 --2- 172.21.15.39:0/1239339622 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5631cad92070 0x5631cb1396a0 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7fef08009920 tx=0x7fef08009950 comp rx=0 tx=0).stop 2024-09-21T03:50:08.602 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:08.584+0000 7fef1cf51740 1 -- 172.21.15.39:0/1239339622 shutdown_connections 2024-09-21T03:50:08.602 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:08.584+0000 7fef1cf51740 1 --2- 172.21.15.39:0/1239339622 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5631cad71390 0x5631cb13a120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:08.602 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:08.584+0000 7fef1cf51740 1 --2- 172.21.15.39:0/1239339622 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5631cad691b0 0x5631cb139be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:08.603 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:08.584+0000 7fef1cf51740 1 --2- 172.21.15.39:0/1239339622 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5631cad92070 0x5631cb1396a0 unknown :-1 s=CLOSED pgs=740 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:08.603 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:08.584+0000 7fef1cf51740 1 -- 172.21.15.39:0/1239339622 >> 172.21.15.39:0/1239339622 conn(0x5631cad86460 msgr2=0x5631cb137590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:50:08.603 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:08.584+0000 7fef1cf51740 1 -- 172.21.15.39:0/1239339622 shutdown_connections 2024-09-21T03:50:08.603 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:08.584+0000 7fef1cf51740 1 -- 172.21.15.39:0/1239339622 wait complete. 2024-09-21T03:50:08.603 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:08.585+0000 7fef1cf51740 1 -- start start 2024-09-21T03:50:08.603 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:08.585+0000 7fef1cf51740 1 Processor -- start 2024-09-21T03:50:08.604 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:08.585+0000 7fef1cf51740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5631cad691b0 0x5631cb144e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:50:08.604 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:08.585+0000 7fef1cf51740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5631cad71390 0x5631cb145390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:50:08.604 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:08.585+0000 7fef1cf51740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5631cad92070 0x5631cb150fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:50:08.604 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:08.585+0000 7fef1cf51740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x5631caf00700 con 0x5631cad691b0 2024-09-21T03:50:08.604 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:08.585+0000 7fef1cf51740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x5631cb03a160 con 0x5631cad71390 2024-09-21T03:50:08.604 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:08.585+0000 7fef1cf51740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x5631cb147ca0 con 0x5631cad92070 2024-09-21T03:50:08.605 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:08.586+0000 7fef1cf51740 10 client.0 _ll_register_callbacks cb 0x5631cb0539e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:50:08.605 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:08.586+0000 7fef0ffff640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5631cad691b0 0x5631cb144e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:50:08.605 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:08.586+0000 7fef0ffff640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5631cad691b0 0x5631cb144e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:56612/0 (socket says 172.21.15.39:56612) 2024-09-21T03:50:08.605 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:08.586+0000 7fef0ffff640 1 -- 172.21.15.39:0/1791704601 learned_addr learned my addr 172.21.15.39:0/1791704601 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:50:08.605 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:08.586+0000 7fef0f7fe640 1 --2- 172.21.15.39:0/1791704601 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5631cad71390 0x5631cb145390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:50:08.605 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:08.586+0000 7fef14f7a640 1 --2- 172.21.15.39:0/1791704601 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5631cad92070 0x5631cb150fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:50:08.605 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:08.586+0000 7fef0ffff640 1 -- 172.21.15.39:0/1791704601 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5631cad92070 msgr2=0x5631cb150fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:50:08.606 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:08.586+0000 7fef0ffff640 1 --2- 172.21.15.39:0/1791704601 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5631cad92070 0x5631cb150fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:08.606 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:08.586+0000 7fef0ffff640 1 -- 172.21.15.39:0/1791704601 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5631cad71390 msgr2=0x5631cb145390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:50:08.606 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:08.586+0000 7fef0ffff640 1 --2- 172.21.15.39:0/1791704601 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5631cad71390 0x5631cb145390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:08.606 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:08.586+0000 7fef0ffff640 1 -- 172.21.15.39:0/1791704601 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5631cb067c90 con 0x5631cad691b0 2024-09-21T03:50:08.606 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:08.586+0000 7fef14f7a640 1 --2- 172.21.15.39:0/1791704601 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5631cad92070 0x5631cb150fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:50:08.607 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:08.586+0000 7fef0ffff640 1 --2- 172.21.15.39:0/1791704601 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5631cad691b0 0x5631cb144e50 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7fef000106a0 tx=0x7fef000046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=238b045569676753 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:50:08.607 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:08.586+0000 7feeef7fe640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-09-21T03:50:08.607 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:08.586+0000 7feeef7fe640 1 -- 172.21.15.39:0/1791704601 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef00005220 con 0x5631cad691b0 2024-09-21T03:50:08.607 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:08.586+0000 7feeef7fe640 1 -- 172.21.15.39:0/1791704601 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8484+0+0 (secure 0 0 0) 0x7fef000053c0 con 0x5631cad691b0 2024-09-21T03:50:08.607 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:08.586+0000 7fef1cf51740 1 -- 172.21.15.39:0/1791704601 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fef08036030 con 0x5631cad691b0 2024-09-21T03:50:08.607 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:08.587+0000 7feeef7fe640 1 -- 172.21.15.39:0/1791704601 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fef000055c0 con 0x5631cad691b0 2024-09-21T03:50:08.607 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:08.587+0000 7fef1cf51740 10 client.16470 fetch_fsmap learned FSMap version 29 2024-09-21T03:50:08.608 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:08.587+0000 7fef1cf51740 1 -- 172.21.15.39:0/1791704601 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5631cb153ae0 con 0x5631cad691b0 2024-09-21T03:50:08.608 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:08.587+0000 7feeef7fe640 1 -- 172.21.15.39:0/1791704601 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fef08036030 con 0x5631cad691b0 2024-09-21T03:50:08.608 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:08.587+0000 7feeef7fe640 1 -- 172.21.15.39:0/1791704601 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fef0002e020 con 0x5631cad691b0 2024-09-21T03:50:08.608 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:08.587+0000 7fef1cf51740 10 client.16470 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T03:50:08.608 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:08.587+0000 7fef1cf51740 10 client.16470 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.a 2024-09-21T03:50:08.608 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:08.587+0000 7fef1cf51740 20 client.16470 populate_metadata read hostname 'smithi039' 2024-09-21T03:50:08.609 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:08.587+0000 7fef1cf51740 1 --2- 172.21.15.39:0/1791704601 >> [v2:172.21.15.39:6840/4247190456,v1:172.21.15.39:6841/4247190456] conn(0x5631cad509f0 0x5631cb156f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:50:08.609 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:08.587+0000 7fef1cf51740 4 client.16470 mds_command: new command op to 14868 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:50:08.609 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:08.587+0000 7fef1cf51740 1 -- 172.21.15.39:0/1791704601 --> [v2:172.21.15.39:6840/4247190456,v1:172.21.15.39:6841/4247190456] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5631cb0b1df0 con 0x5631cad509f0 2024-09-21T03:50:08.609 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:08.587+0000 7fef0f7fe640 1 --2- 172.21.15.39:0/1791704601 >> [v2:172.21.15.39:6840/4247190456,v1:172.21.15.39:6841/4247190456] conn(0x5631cad509f0 0x5631cb156f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:50:08.609 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:08.588+0000 7feeef7fe640 10 client.16470 ms_handle_connect on v2:172.21.15.39:6840/4247190456 2024-09-21T03:50:08.609 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:08.588+0000 7fef0f7fe640 1 --2- 172.21.15.39:0/1791704601 >> [v2:172.21.15.39:6840/4247190456,v1:172.21.15.39:6841/4247190456] conn(0x5631cad509f0 0x5631cb156f90 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=28efa63277528f63 server_cookie=1a2f947ceabd3fa5 in_seq=0 out_seq=0 2024-09-21T03:50:08.609 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:08.588+0000 7feeef7fe640 1 -- 172.21.15.39:0/1791704601 <== mds.4 v2:172.21.15.39:6840/4247190456 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5631cb0b1df0 con 0x5631cad509f0 2024-09-21T03:50:08.610 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:08.588+0000 7feeef7fe640 10 client.16470 handle_command_reply: tid=0 2024-09-21T03:50:08.610 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:08.588+0000 7fef1cf51740 1 client.16470 shutdown 2024-09-21T03:50:08.610 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:08.589+0000 7fef1cf51740 1 -- 172.21.15.39:0/1791704601 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5631cad691b0 msgr2=0x5631cb144e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:50:08.610 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:08.589+0000 7fef1cf51740 1 --2- 172.21.15.39:0/1791704601 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5631cad691b0 0x5631cb144e50 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7fef000106a0 tx=0x7fef000046c0 comp rx=0 tx=0).stop 2024-09-21T03:50:08.610 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:08.589+0000 7fef1cf51740 1 -- 172.21.15.39:0/1791704601 shutdown_connections 2024-09-21T03:50:08.610 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:08.589+0000 7fef1cf51740 1 --2- 172.21.15.39:0/1791704601 >> [v2:172.21.15.39:6840/4247190456,v1:172.21.15.39:6841/4247190456] conn(0x5631cad509f0 0x5631cb156f90 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:08.611 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:08.589+0000 7fef1cf51740 1 --2- 172.21.15.39:0/1791704601 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5631cad92070 0x5631cb150fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:08.611 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:08.589+0000 7fef1cf51740 1 --2- 172.21.15.39:0/1791704601 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5631cad71390 0x5631cb145390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:08.611 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:08.589+0000 7fef1cf51740 1 --2- 172.21.15.39:0/1791704601 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5631cad691b0 0x5631cb144e50 unknown :-1 s=CLOSED pgs=741 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:08.611 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:08.589+0000 7feeef7fe640 0 client.16470 ms_handle_reset on v2:172.21.15.39:6840/4247190456 2024-09-21T03:50:08.611 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:08.589+0000 7fef1cf51740 1 -- 172.21.15.39:0/1791704601 >> 172.21.15.39:0/1791704601 conn(0x5631cad86460 msgr2=0x5631cb13c850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:50:08.611 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:08.589+0000 7fef1cf51740 1 -- 172.21.15.39:0/1791704601 shutdown_connections 2024-09-21T03:50:08.612 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:08.589+0000 7fef1cf51740 1 -- 172.21.15.39:0/1791704601 wait complete. 2024-09-21T03:50:08.612 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:08.589+0000 7fef1cf51740 20 client.16470 trim_cache size 0 max 16384 2024-09-21T03:50:08.620 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:08.602+0000 7fef1cf51740 1 Processor -- start 2024-09-21T03:50:08.621 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:08.602+0000 7fef1cf51740 1 -- start start 2024-09-21T03:50:08.621 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:08.603+0000 7fef1cf51740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5631cad691b0 0x5631cb13ceb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:50:08.621 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:08.603+0000 7fef1cf51740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5631cad71390 0x5631cb150ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:50:08.621 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:08.603+0000 7fef1cf51740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5631cad92070 0x5631cb142040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:50:08.621 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:08.603+0000 7fef1cf51740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fef101b19a0 con 0x5631cad691b0 2024-09-21T03:50:08.622 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:08.603+0000 7fef1cf51740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fef101b34e0 con 0x5631cad92070 2024-09-21T03:50:08.622 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:08.603+0000 7fef1cf51740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fef101b2070 con 0x5631cad71390 2024-09-21T03:50:08.622 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:08.603+0000 7fef0ffff640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5631cad691b0 0x5631cb13ceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:50:08.622 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:08.603+0000 7fef0ffff640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5631cad691b0 0x5631cb13ceb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:56628/0 (socket says 172.21.15.39:56628) 2024-09-21T03:50:08.622 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:08.603+0000 7fef0ffff640 1 -- 172.21.15.39:0/2206950043 learned_addr learned my addr 172.21.15.39:0/2206950043 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:50:08.622 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:08.603+0000 7fef0f7fe640 1 --2- 172.21.15.39:0/2206950043 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5631cad71390 0x5631cb150ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:50:08.623 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:08.603+0000 7fef0ffff640 1 -- 172.21.15.39:0/2206950043 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5631cad71390 msgr2=0x5631cb150ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:50:08.623 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:08.603+0000 7fef14f7a640 1 --2- 172.21.15.39:0/2206950043 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5631cad92070 0x5631cb142040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:50:08.623 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:08.603+0000 7fef0ffff640 1 --2- 172.21.15.39:0/2206950043 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5631cad71390 0x5631cb150ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:08.623 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:08.603+0000 7fef0ffff640 1 -- 172.21.15.39:0/2206950043 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5631cad92070 msgr2=0x5631cb142040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:50:08.623 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:08.603+0000 7fef0ffff640 1 --2- 172.21.15.39:0/2206950043 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5631cad92070 0x5631cb142040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:08.623 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:08.603+0000 7fef0ffff640 1 -- 172.21.15.39:0/2206950043 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5631cb13fab0 con 0x5631cad691b0 2024-09-21T03:50:08.624 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:08.603+0000 7fef0f7fe640 1 --2- 172.21.15.39:0/2206950043 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5631cad71390 0x5631cb150ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:50:08.624 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:08.603+0000 7fef0ffff640 1 --2- 172.21.15.39:0/2206950043 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5631cad691b0 0x5631cb13ceb0 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7fef00010670 tx=0x7fef00012f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff326f2d937c8534 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:50:08.624 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:08.604+0000 7feef5ffb640 1 -- 172.21.15.39:0/2206950043 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef00036040 con 0x5631cad691b0 2024-09-21T03:50:08.624 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:08.604+0000 7feef5ffb640 1 -- 172.21.15.39:0/2206950043 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fef00012d00 con 0x5631cad691b0 2024-09-21T03:50:08.624 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:08.604+0000 7feef5ffb640 1 -- 172.21.15.39:0/2206950043 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef00030460 con 0x5631cad691b0 2024-09-21T03:50:08.624 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:08.604+0000 7fef1cf51740 1 -- 172.21.15.39:0/2206950043 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5631cad691b0 msgr2=0x5631cb13ceb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:50:08.625 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:08.604+0000 7fef1cf51740 1 --2- 172.21.15.39:0/2206950043 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5631cad691b0 0x5631cb13ceb0 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7fef00010670 tx=0x7fef00012f90 comp rx=0 tx=0).stop 2024-09-21T03:50:08.625 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:08.604+0000 7fef1cf51740 1 -- 172.21.15.39:0/2206950043 shutdown_connections 2024-09-21T03:50:08.625 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:08.604+0000 7fef1cf51740 1 --2- 172.21.15.39:0/2206950043 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5631cad92070 0x5631cb142040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:08.625 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:08.604+0000 7fef1cf51740 1 --2- 172.21.15.39:0/2206950043 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5631cad71390 0x5631cb150ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:08.625 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:08.604+0000 7fef1cf51740 1 --2- 172.21.15.39:0/2206950043 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5631cad691b0 0x5631cb13ceb0 unknown :-1 s=CLOSED pgs=742 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:08.625 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:08.604+0000 7fef1cf51740 1 -- 172.21.15.39:0/2206950043 >> 172.21.15.39:0/2206950043 conn(0x5631cad86460 msgr2=0x5631cb13c7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:50:08.626 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:08.604+0000 7fef1cf51740 1 -- 172.21.15.39:0/2206950043 shutdown_connections 2024-09-21T03:50:08.626 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:08.605+0000 7fef1cf51740 1 -- 172.21.15.39:0/2206950043 wait complete. 2024-09-21T03:50:08.626 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:08.605+0000 7fef1cf51740 1 -- start start 2024-09-21T03:50:08.626 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:08.605+0000 7fef1cf51740 1 Processor -- start 2024-09-21T03:50:08.626 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:08.605+0000 7fef1cf51740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5631cad691b0 0x5631cb14d8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:50:08.626 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:08.605+0000 7fef1cf51740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5631cad71390 0x5631cb148e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:50:08.627 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:08.605+0000 7fef1cf51740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5631cad92070 0x5631cb149520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:50:08.627 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:08.605+0000 7fef1cf51740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fef101b2e10 con 0x5631cad691b0 2024-09-21T03:50:08.627 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:08.605+0000 7fef1cf51740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fef101b2740 con 0x5631cad71390 2024-09-21T03:50:08.627 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:08.605+0000 7fef1cf51740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fef101af980 con 0x5631cad92070 2024-09-21T03:50:08.627 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:08.605+0000 7fef1cf51740 10 client.0 _ll_register_callbacks cb 0x7fef0000b9b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:50:08.628 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:08.606+0000 7fef0ffff640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5631cad691b0 0x5631cb14d8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:50:08.628 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:08.606+0000 7fef0ffff640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5631cad691b0 0x5631cb14d8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:56638/0 (socket says 172.21.15.39:56638) 2024-09-21T03:50:08.628 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:08.606+0000 7fef0ffff640 1 -- 172.21.15.39:0/1970265482 learned_addr learned my addr 172.21.15.39:0/1970265482 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:50:08.628 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:08.606+0000 7fef0f7fe640 1 --2- 172.21.15.39:0/1970265482 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5631cad71390 0x5631cb148e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:50:08.628 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:08.606+0000 7fef14f7a640 1 --2- 172.21.15.39:0/1970265482 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5631cad92070 0x5631cb149520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:50:08.628 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:08.606+0000 7fef0ffff640 1 -- 172.21.15.39:0/1970265482 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5631cad92070 msgr2=0x5631cb149520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:50:08.629 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:08.606+0000 7fef0ffff640 1 --2- 172.21.15.39:0/1970265482 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5631cad92070 0x5631cb149520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:08.629 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:08.606+0000 7fef0ffff640 1 -- 172.21.15.39:0/1970265482 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5631cad71390 msgr2=0x5631cb148e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:50:08.629 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:08.606+0000 7fef0ffff640 1 --2- 172.21.15.39:0/1970265482 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5631cad71390 0x5631cb148e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:08.629 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:08.606+0000 7fef0ffff640 1 -- 172.21.15.39:0/1970265482 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5631cb1c0950 con 0x5631cad691b0 2024-09-21T03:50:08.629 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:08.606+0000 7fef0f7fe640 1 --2- 172.21.15.39:0/1970265482 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5631cad71390 0x5631cb148e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:50:08.629 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:08.606+0000 7fef0ffff640 1 --2- 172.21.15.39:0/1970265482 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5631cad691b0 0x5631cb14d8f0 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7fef0000fe80 tx=0x7fef0000be10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=67ee250f67cf57d5 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:50:08.630 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:08.606+0000 7feeef7fe640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-09-21T03:50:08.630 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:08.606+0000 7feeef7fe640 1 -- 172.21.15.39:0/1970265482 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef00036070 con 0x5631cad691b0 2024-09-21T03:50:08.630 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:08.606+0000 7fef1cf51740 1 -- 172.21.15.39:0/1970265482 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fef00012d00 con 0x5631cad691b0 2024-09-21T03:50:08.630 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:08.606+0000 7feeef7fe640 1 -- 172.21.15.39:0/1970265482 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8484+0+0 (secure 0 0 0) 0x7fef0003b470 con 0x5631cad691b0 2024-09-21T03:50:08.630 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:08.607+0000 7feeef7fe640 1 -- 172.21.15.39:0/1970265482 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fef0004b050 con 0x5631cad691b0 2024-09-21T03:50:08.630 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:08.607+0000 7fef1cf51740 10 client.16476 fetch_fsmap learned FSMap version 29 2024-09-21T03:50:08.631 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:08.607+0000 7fef1cf51740 1 -- 172.21.15.39:0/1970265482 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5631cb1c19f0 con 0x5631cad691b0 2024-09-21T03:50:08.631 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:08.607+0000 7feeef7fe640 1 -- 172.21.15.39:0/1970265482 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fef00012d00 con 0x5631cad691b0 2024-09-21T03:50:08.631 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:08.607+0000 7feeef7fe640 1 -- 172.21.15.39:0/1970265482 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fef0002e040 con 0x5631cad691b0 2024-09-21T03:50:08.631 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:08.607+0000 7fef1cf51740 10 client.16476 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T03:50:08.631 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:08.607+0000 7fef1cf51740 10 client.16476 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.a 2024-09-21T03:50:08.631 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:08.607+0000 7fef1cf51740 20 client.16476 populate_metadata read hostname 'smithi039' 2024-09-21T03:50:08.632 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:08.607+0000 7fef1cf51740 1 --2- 172.21.15.39:0/1970265482 >> [v2:172.21.15.39:6840/4247190456,v1:172.21.15.39:6841/4247190456] conn(0x5631cad25da0 0x5631cb1c40c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:50:08.632 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:08.607+0000 7fef1cf51740 4 client.16476 mds_command: new command op to 14868 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T03:50:08.632 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:08.607+0000 7fef1cf51740 1 -- 172.21.15.39:0/1970265482 --> [v2:172.21.15.39:6840/4247190456,v1:172.21.15.39:6841/4247190456] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5631cb0ad8a0 con 0x5631cad25da0 2024-09-21T03:50:08.632 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:08.607+0000 7fef0f7fe640 1 --2- 172.21.15.39:0/1970265482 >> [v2:172.21.15.39:6840/4247190456,v1:172.21.15.39:6841/4247190456] conn(0x5631cad25da0 0x5631cb1c40c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:50:08.632 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:08.608+0000 7feeef7fe640 10 client.16476 ms_handle_connect on v2:172.21.15.39:6840/4247190456 2024-09-21T03:50:08.632 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:08.608+0000 7fef0f7fe640 1 --2- 172.21.15.39:0/1970265482 >> [v2:172.21.15.39:6840/4247190456,v1:172.21.15.39:6841/4247190456] conn(0x5631cad25da0 0x5631cb1c40c0 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=348d76341461803c server_cookie=f398f31b2ea0e2cf in_seq=0 out_seq=0 2024-09-21T03:50:08.633 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:08.608+0000 7feeef7fe640 1 -- 172.21.15.39:0/1970265482 <== mds.4 v2:172.21.15.39:6840/4247190456 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5631cb0ad8a0 con 0x5631cad25da0 2024-09-21T03:50:08.633 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:08.608+0000 7feeef7fe640 10 client.16476 handle_command_reply: tid=0 2024-09-21T03:50:08.633 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:08.608+0000 7fef1cf51740 1 client.16476 shutdown 2024-09-21T03:50:08.633 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:08.608+0000 7fef1cf51740 1 -- 172.21.15.39:0/1970265482 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5631cad691b0 msgr2=0x5631cb14d8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:50:08.633 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:08.608+0000 7fef1cf51740 1 --2- 172.21.15.39:0/1970265482 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5631cad691b0 0x5631cb14d8f0 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7fef0000fe80 tx=0x7fef0000be10 comp rx=0 tx=0).stop 2024-09-21T03:50:08.633 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:08.608+0000 7fef1cf51740 1 -- 172.21.15.39:0/1970265482 shutdown_connections 2024-09-21T03:50:08.633 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:08.608+0000 7fef1cf51740 1 --2- 172.21.15.39:0/1970265482 >> [v2:172.21.15.39:6840/4247190456,v1:172.21.15.39:6841/4247190456] conn(0x5631cad25da0 0x5631cb1c40c0 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:08.634 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:08.608+0000 7fef1cf51740 1 --2- 172.21.15.39:0/1970265482 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5631cad92070 0x5631cb149520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:08.634 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:08.608+0000 7feeef7fe640 0 client.16476 ms_handle_reset on v2:172.21.15.39:6840/4247190456 2024-09-21T03:50:08.634 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:08.608+0000 7fef1cf51740 1 --2- 172.21.15.39:0/1970265482 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5631cad71390 0x5631cb148e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:08.634 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:08.608+0000 7fef1cf51740 1 --2- 172.21.15.39:0/1970265482 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5631cad691b0 0x5631cb14d8f0 unknown :-1 s=CLOSED pgs=743 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:08.634 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:08.608+0000 7fef1cf51740 1 -- 172.21.15.39:0/1970265482 >> 172.21.15.39:0/1970265482 conn(0x5631cad86460 msgr2=0x5631cb13a6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:50:08.634 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:08.608+0000 7fef1cf51740 1 -- 172.21.15.39:0/1970265482 shutdown_connections 2024-09-21T03:50:08.634 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:08.608+0000 7fef1cf51740 1 -- 172.21.15.39:0/1970265482 wait complete. 2024-09-21T03:50:08.634 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:08.608+0000 7fef1cf51740 20 client.16476 trim_cache size 0 max 16384 2024-09-21T03:50:08.634 INFO:teuthology.orchestra.run.smithi039.stdout:[] 2024-09-21T03:50:08.635 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:08.609+0000 7fef169dd640 1 -- 172.21.15.39:0/497547884 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7feef80b6660 msgr2=0x7feef80b8b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:50:08.635 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:08.609+0000 7fef169dd640 1 --2- 172.21.15.39:0/497547884 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7feef80b6660 0x7feef80b8b70 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x5631cb138320 tx=0x7fef0000b390 comp rx=0 tx=0).stop 2024-09-21T03:50:08.635 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:08.609+0000 7fef169dd640 1 -- 172.21.15.39:0/497547884 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fef10107b70 msgr2=0x7fef100ff3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:50:08.635 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:08.609+0000 7fef169dd640 1 --2- 172.21.15.39:0/497547884 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fef10107b70 0x7fef100ff3b0 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7fef0400c970 tx=0x7fef0400ce30 comp rx=0 tx=0).stop 2024-09-21T03:50:08.636 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:08.609+0000 7fef169dd640 1 -- 172.21.15.39:0/497547884 shutdown_connections 2024-09-21T03:50:08.636 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:08.609+0000 7fef169dd640 1 --2- 172.21.15.39:0/497547884 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7feef80b6660 0x7feef80b8b70 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:08.636 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:08.609+0000 7fef169dd640 1 --2- 172.21.15.39:0/497547884 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fef101084c0 0x7fef100ff8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:08.636 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:08.609+0000 7fef169dd640 1 --2- 172.21.15.39:0/497547884 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fef10107b70 0x7fef100ff3b0 unknown :-1 s=CLOSED pgs=739 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:08.636 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:08.609+0000 7fef169dd640 1 --2- 172.21.15.39:0/497547884 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fef10102230 0x7fef100fee70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:08.636 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:08.609+0000 7fef169dd640 1 -- 172.21.15.39:0/497547884 >> 172.21.15.39:0/497547884 conn(0x7fef10079e40 msgr2=0x7fef10103e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:50:08.636 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:08.609+0000 7fef169dd640 1 -- 172.21.15.39:0/497547884 shutdown_connections 2024-09-21T03:50:08.636 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:08.609+0000 7fef169dd640 1 -- 172.21.15.39:0/497547884 wait complete. 2024-09-21T03:50:09.218 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:50:08 smithi161 ceph-mon[34533]: pgmap v448: 97 pgs: 97 active+clean; 10 MiB data, 536 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-09-21T03:50:09.224 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:50:08 smithi150 ceph-mon[34289]: pgmap v448: 97 pgs: 97 active+clean; 10 MiB data, 536 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-09-21T03:50:09.243 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:50:08 smithi039 ceph-mon[31211]: pgmap v448: 97 pgs: 97 active+clean; 10 MiB data, 536 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-09-21T03:50:09.539 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 51116d5c-77ca-11ef-baf5-efdc52797490 -- ceph tell mds.1:3 damage ls 2024-09-21T03:50:09.800 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/51116d5c-77ca-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:50:10.271 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:10.268+0000 7f86046ed640 1 -- 172.21.15.39:0/539867815 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f85fc105320 msgr2=0x7f85fc10d870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:50:10.271 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:10.268+0000 7f86046ed640 1 --2- 172.21.15.39:0/539867815 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f85fc105320 0x7f85fc10d870 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7f85ec009900 tx=0x7f85ec02a3f0 comp rx=0 tx=0).stop 2024-09-21T03:50:10.271 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:10.268+0000 7f86046ed640 1 -- 172.21.15.39:0/539867815 shutdown_connections 2024-09-21T03:50:10.271 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:10.268+0000 7f86046ed640 1 --2- 172.21.15.39:0/539867815 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f85fc10de30 0x7f85fc110320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:10.271 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:10.268+0000 7f86046ed640 1 --2- 172.21.15.39:0/539867815 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f85fc105320 0x7f85fc10d870 unknown :-1 s=CLOSED pgs=744 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:10.272 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:10.268+0000 7f86046ed640 1 --2- 172.21.15.39:0/539867815 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f85fc1049d0 0x7f85fc104de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:10.272 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:10.268+0000 7f86046ed640 1 -- 172.21.15.39:0/539867815 >> 172.21.15.39:0/539867815 conn(0x7f85fc079740 msgr2=0x7f85fc079b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:50:10.272 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:10.268+0000 7f86046ed640 1 -- 172.21.15.39:0/539867815 shutdown_connections 2024-09-21T03:50:10.272 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:10.268+0000 7f86046ed640 1 -- 172.21.15.39:0/539867815 wait complete. 2024-09-21T03:50:10.272 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:10.268+0000 7f86046ed640 1 Processor -- start 2024-09-21T03:50:10.272 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:10.269+0000 7f86046ed640 1 -- start start 2024-09-21T03:50:10.273 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:10.269+0000 7f86046ed640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f85fc1049d0 0x7f85fc19a4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:50:10.273 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:10.269+0000 7f86046ed640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f85fc105320 0x7f85fc19aa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:50:10.274 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:10.269+0000 7f86046ed640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f85fc10de30 0x7f85fc19fd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:50:10.274 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:10.269+0000 7f86046ed640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f85fc113a90 con 0x7f85fc105320 2024-09-21T03:50:10.274 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:10.269+0000 7f86046ed640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f85fc113910 con 0x7f85fc10de30 2024-09-21T03:50:10.274 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:10.269+0000 7f86046ed640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f85fc113c10 con 0x7f85fc1049d0 2024-09-21T03:50:10.274 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:10.269+0000 7f8601c88640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f85fc105320 0x7f85fc19aa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:50:10.274 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:10.269+0000 7f8601c88640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f85fc105320 0x7f85fc19aa60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:56654/0 (socket says 172.21.15.39:56654) 2024-09-21T03:50:10.274 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:10.269+0000 7f8601c88640 1 -- 172.21.15.39:0/3736085481 learned_addr learned my addr 172.21.15.39:0/3736085481 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:50:10.274 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:10.269+0000 7f8602c8a640 1 --2- 172.21.15.39:0/3736085481 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f85fc10de30 0x7f85fc19fd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:50:10.274 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:10.269+0000 7f8602489640 1 --2- 172.21.15.39:0/3736085481 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f85fc1049d0 0x7f85fc19a4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:50:10.274 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:10.269+0000 7f8601c88640 1 -- 172.21.15.39:0/3736085481 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f85fc1049d0 msgr2=0x7f85fc19a4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:50:10.274 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:10.269+0000 7f8601c88640 1 --2- 172.21.15.39:0/3736085481 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f85fc1049d0 0x7f85fc19a4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:10.274 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:10.269+0000 7f8601c88640 1 -- 172.21.15.39:0/3736085481 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f85fc10de30 msgr2=0x7f85fc19fd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:50:10.274 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:10.269+0000 7f8601c88640 1 --2- 172.21.15.39:0/3736085481 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f85fc10de30 0x7f85fc19fd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:10.275 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:10.269+0000 7f8601c88640 1 -- 172.21.15.39:0/3736085481 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f85fc1a0370 con 0x7f85fc105320 2024-09-21T03:50:10.275 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:10.270+0000 7f8601c88640 1 --2- 172.21.15.39:0/3736085481 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f85fc105320 0x7f85fc19aa60 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7f85ec02a910 tx=0x7f85ec004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:50:10.275 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:10.270+0000 7f85f37fe640 1 -- 172.21.15.39:0/3736085481 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85ec03f030 con 0x7f85fc105320 2024-09-21T03:50:10.275 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:10.270+0000 7f85f37fe640 1 -- 172.21.15.39:0/3736085481 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f85ec003aa0 con 0x7f85fc105320 2024-09-21T03:50:10.275 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:10.270+0000 7f86046ed640 1 -- 172.21.15.39:0/3736085481 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f85fc1a0600 con 0x7f85fc105320 2024-09-21T03:50:10.275 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:10.270+0000 7f85f37fe640 1 -- 172.21.15.39:0/3736085481 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85ec035070 con 0x7f85fc105320 2024-09-21T03:50:10.275 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:10.270+0000 7f86046ed640 1 -- 172.21.15.39:0/3736085481 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f85fc1a0ae0 con 0x7f85fc105320 2024-09-21T03:50:10.277 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:10.272+0000 7f860ac61740 1 Processor -- start 2024-09-21T03:50:10.277 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:10.273+0000 7f860ac61740 1 -- start start 2024-09-21T03:50:10.277 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:10.273+0000 7f85f37fe640 1 -- 172.21.15.39:0/3736085481 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 27) ==== 154643+0+0 (secure 0 0 0) 0x7f85ec042030 con 0x7f85fc105320 2024-09-21T03:50:10.277 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:10.273+0000 7f860ac61740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e95c45f070 0x55e95c8066a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:50:10.277 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:10.273+0000 7f860ac61740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55e95c4361b0 0x55e95c806be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:50:10.277 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:10.273+0000 7f860ac61740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e95c43e390 0x55e95c807120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:50:10.277 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:10.273+0000 7f860ac61740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x55e95c77cea0 con 0x55e95c4361b0 2024-09-21T03:50:10.277 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:10.273+0000 7f860ac61740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x55e95c76c2d0 con 0x55e95c45f070 2024-09-21T03:50:10.277 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:10.273+0000 7f860ac61740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55e95c7700e0 con 0x55e95c43e390 2024-09-21T03:50:10.277 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:10.273+0000 7f8602489640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55e95c4361b0 0x55e95c806be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:50:10.277 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:10.273+0000 7f8602c8a640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e95c45f070 0x55e95c8066a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:50:10.277 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:10.273+0000 7f8602489640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55e95c4361b0 0x55e95c806be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:56658/0 (socket says 172.21.15.39:56658) 2024-09-21T03:50:10.278 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:10.273+0000 7f8602c8a640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e95c45f070 0x55e95c8066a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.39:50502/0 (socket says 172.21.15.39:50502) 2024-09-21T03:50:10.278 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:10.273+0000 7f8602489640 1 -- 172.21.15.39:0/152960596 learned_addr learned my addr 172.21.15.39:0/152960596 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:50:10.278 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:10.273+0000 7f8601c88640 1 --2- 172.21.15.39:0/152960596 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e95c43e390 0x55e95c807120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:50:10.278 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:10.274+0000 7f8602c8a640 1 -- 172.21.15.39:0/152960596 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e95c43e390 msgr2=0x55e95c807120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:50:10.278 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:10.274+0000 7f8602c8a640 1 --2- 172.21.15.39:0/152960596 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e95c43e390 0x55e95c807120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:10.278 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:10.274+0000 7f8602c8a640 1 -- 172.21.15.39:0/152960596 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55e95c4361b0 msgr2=0x55e95c806be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:50:10.278 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:10.274+0000 7f8602c8a640 1 --2- 172.21.15.39:0/152960596 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55e95c4361b0 0x55e95c806be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:10.278 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:10.274+0000 7f8602c8a640 1 -- 172.21.15.39:0/152960596 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e95c4ca650 con 0x55e95c45f070 2024-09-21T03:50:10.278 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:10.274+0000 7f8602c8a640 1 --2- 172.21.15.39:0/152960596 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e95c45f070 0x55e95c8066a0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f85f4007e40 tx=0x7f85f400f790 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d8f733fa3711a4f4 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:50:10.278 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:10.274+0000 7f85f37fe640 1 --2- 172.21.15.39:0/3736085481 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f85e80b66b0 0x7f85e80b8bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:50:10.278 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:10.274+0000 7f8602489640 1 --2- 172.21.15.39:0/3736085481 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f85e80b66b0 0x7f85e80b8bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:50:10.278 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:10.274+0000 7f85f0ff9640 1 -- 172.21.15.39:0/152960596 <== mon.2 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85f4003b70 con 0x55e95c45f070 2024-09-21T03:50:10.279 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:10.274+0000 7f85f37fe640 1 -- 172.21.15.39:0/3736085481 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8484+0+0 (secure 0 0 0) 0x7f85ec107800 con 0x7f85fc105320 2024-09-21T03:50:10.279 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:10.274+0000 7f85f0ff9640 1 -- 172.21.15.39:0/152960596 <== mon.2 v2:172.21.15.150:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f85f4003d10 con 0x55e95c45f070 2024-09-21T03:50:10.279 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:10.274+0000 7f85f0ff9640 1 -- 172.21.15.39:0/152960596 <== mon.2 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85f4004490 con 0x55e95c45f070 2024-09-21T03:50:10.282 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:10.275+0000 7f860ac61740 1 -- 172.21.15.39:0/152960596 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e95c45f070 msgr2=0x55e95c8066a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:50:10.282 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:10.275+0000 7f8602489640 1 --2- 172.21.15.39:0/3736085481 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f85e80b66b0 0x7f85e80b8bc0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x55e95c805320 tx=0x7f85f800b390 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:50:10.282 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:10.275+0000 7f860ac61740 1 --2- 172.21.15.39:0/152960596 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e95c45f070 0x55e95c8066a0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f85f4007e40 tx=0x7f85f400f790 comp rx=0 tx=0).stop 2024-09-21T03:50:10.282 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:10.275+0000 7f860ac61740 1 -- 172.21.15.39:0/152960596 shutdown_connections 2024-09-21T03:50:10.282 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:10.275+0000 7f860ac61740 1 --2- 172.21.15.39:0/152960596 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e95c43e390 0x55e95c807120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:10.282 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:10.275+0000 7f860ac61740 1 --2- 172.21.15.39:0/152960596 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55e95c4361b0 0x55e95c806be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:10.282 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:10.275+0000 7f860ac61740 1 --2- 172.21.15.39:0/152960596 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e95c45f070 0x55e95c8066a0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:10.282 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:10.275+0000 7f860ac61740 1 -- 172.21.15.39:0/152960596 >> 172.21.15.39:0/152960596 conn(0x55e95c453460 msgr2=0x55e95c804590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:50:10.282 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:10.275+0000 7f860ac61740 1 -- 172.21.15.39:0/152960596 shutdown_connections 2024-09-21T03:50:10.282 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:10.275+0000 7f860ac61740 1 -- 172.21.15.39:0/152960596 wait complete. 2024-09-21T03:50:10.282 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:10.276+0000 7f860ac61740 1 -- start start 2024-09-21T03:50:10.282 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:10.276+0000 7f860ac61740 1 Processor -- start 2024-09-21T03:50:10.283 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:10.276+0000 7f860ac61740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55e95c4361b0 0x55e95c811e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:50:10.283 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:10.276+0000 7f860ac61740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e95c43e390 0x55e95c812340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:50:10.283 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:10.276+0000 7f860ac61740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e95c45f070 0x55e95c81e1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:50:10.283 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:10.276+0000 7f860ac61740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x55e95c5cd700 con 0x55e95c4361b0 2024-09-21T03:50:10.283 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:10.276+0000 7f860ac61740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x55e95c707160 con 0x55e95c45f070 2024-09-21T03:50:10.283 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:10.276+0000 7f860ac61740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55e95c814e80 con 0x55e95c43e390 2024-09-21T03:50:10.283 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:10.276+0000 7f860ac61740 10 client.0 _ll_register_callbacks cb 0x55e95c7209e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:50:10.283 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:10.276+0000 7f8602489640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55e95c4361b0 0x55e95c811e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:50:10.283 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:10.276+0000 7f8602489640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55e95c4361b0 0x55e95c811e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:56662/0 (socket says 172.21.15.39:56662) 2024-09-21T03:50:10.283 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:10.276+0000 7f8602489640 1 -- 172.21.15.39:0/3582258255 learned_addr learned my addr 172.21.15.39:0/3582258255 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:50:10.283 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:10.277+0000 7f8601c88640 1 --2- 172.21.15.39:0/3582258255 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e95c43e390 0x55e95c812340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:50:10.283 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:10.277+0000 7f8602c8a640 1 --2- 172.21.15.39:0/3582258255 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e95c45f070 0x55e95c81e1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:50:10.283 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:10.277+0000 7f8602489640 1 -- 172.21.15.39:0/3582258255 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e95c43e390 msgr2=0x55e95c812340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:50:10.283 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:10.277+0000 7f8602489640 1 --2- 172.21.15.39:0/3582258255 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e95c43e390 0x55e95c812340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:10.284 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:10.277+0000 7f8602489640 1 -- 172.21.15.39:0/3582258255 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e95c45f070 msgr2=0x55e95c81e1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:50:10.284 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:10.277+0000 7f8602489640 1 --2- 172.21.15.39:0/3582258255 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e95c45f070 0x55e95c81e1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:10.284 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:10.277+0000 7f8602489640 1 -- 172.21.15.39:0/3582258255 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e95c734c90 con 0x55e95c4361b0 2024-09-21T03:50:10.284 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:10.277+0000 7f8601c88640 1 --2- 172.21.15.39:0/3582258255 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e95c43e390 0x55e95c812340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:50:10.284 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:10.277+0000 7f8602489640 1 --2- 172.21.15.39:0/3582258255 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55e95c4361b0 0x55e95c811e00 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7f85f801f5a0 tx=0x7f85f8004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=23b9121507c0a103 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:50:10.284 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:10.277+0000 7f85e5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-09-21T03:50:10.284 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:10.277+0000 7f85e5ffb640 1 -- 172.21.15.39:0/3582258255 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85f802c060 con 0x55e95c4361b0 2024-09-21T03:50:10.284 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:10.277+0000 7f860ac61740 1 -- 172.21.15.39:0/3582258255 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f85f4003d10 con 0x55e95c4361b0 2024-09-21T03:50:10.284 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:10.277+0000 7f85e5ffb640 1 -- 172.21.15.39:0/3582258255 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8484+0+0 (secure 0 0 0) 0x7f85f8017030 con 0x55e95c4361b0 2024-09-21T03:50:10.284 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:10.277+0000 7f85e5ffb640 1 -- 172.21.15.39:0/3582258255 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f85f8010700 con 0x55e95c4361b0 2024-09-21T03:50:10.284 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:10.277+0000 7f860ac61740 10 client.16488 fetch_fsmap learned FSMap version 29 2024-09-21T03:50:10.284 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:10.277+0000 7f860ac61740 1 -- 172.21.15.39:0/3582258255 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55e95c820be0 con 0x55e95c4361b0 2024-09-21T03:50:10.284 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:10.278+0000 7f85e5ffb640 1 -- 172.21.15.39:0/3582258255 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f85f4003d10 con 0x55e95c4361b0 2024-09-21T03:50:10.284 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:10.278+0000 7f85e5ffb640 1 -- 172.21.15.39:0/3582258255 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f85f801ca00 con 0x55e95c4361b0 2024-09-21T03:50:10.284 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:10.278+0000 7f860ac61740 10 client.16488 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T03:50:10.284 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:10.278+0000 7f860ac61740 10 client.16488 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-21T03:50:10.285 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:10.278+0000 7f860ac61740 20 client.16488 populate_metadata read hostname 'smithi039' 2024-09-21T03:50:10.285 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:10.278+0000 7f860ac61740 1 --2- 172.21.15.39:0/3582258255 >> [v2:172.21.15.161:6836/3451544597,v1:172.21.15.161:6837/3451544597] conn(0x55e95c41d9f0 0x55e95c8240f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:50:10.285 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:10.278+0000 7f860ac61740 4 client.16488 mds_command: new command op to 24646 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:50:10.285 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:10.278+0000 7f860ac61740 1 -- 172.21.15.39:0/3582258255 --> [v2:172.21.15.161:6836/3451544597,v1:172.21.15.161:6837/3451544597] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e95c77edf0 con 0x55e95c41d9f0 2024-09-21T03:50:10.285 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:10.278+0000 7f8601c88640 1 --2- 172.21.15.39:0/3582258255 >> [v2:172.21.15.161:6836/3451544597,v1:172.21.15.161:6837/3451544597] conn(0x55e95c41d9f0 0x55e95c8240f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:50:10.285 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:10.279+0000 7f85e5ffb640 10 client.16488 ms_handle_connect on v2:172.21.15.161:6836/3451544597 2024-09-21T03:50:10.285 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:10.279+0000 7f8601c88640 1 --2- 172.21.15.39:0/3582258255 >> [v2:172.21.15.161:6836/3451544597,v1:172.21.15.161:6837/3451544597] conn(0x55e95c41d9f0 0x55e95c8240f0 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=31acefae5aa44a32 server_cookie=eef3f9df6b676059 in_seq=0 out_seq=0 2024-09-21T03:50:10.286 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:10.279+0000 7f85e5ffb640 1 -- 172.21.15.39:0/3582258255 <== mds.3 v2:172.21.15.161:6836/3451544597 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55e95c77edf0 con 0x55e95c41d9f0 2024-09-21T03:50:10.286 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:10.279+0000 7f85e5ffb640 10 client.16488 handle_command_reply: tid=0 2024-09-21T03:50:10.286 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:10.280+0000 7f860ac61740 1 client.16488 shutdown 2024-09-21T03:50:10.286 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:10.280+0000 7f860ac61740 1 -- 172.21.15.39:0/3582258255 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55e95c4361b0 msgr2=0x55e95c811e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:50:10.286 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:10.280+0000 7f860ac61740 1 --2- 172.21.15.39:0/3582258255 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55e95c4361b0 0x55e95c811e00 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7f85f801f5a0 tx=0x7f85f8004730 comp rx=0 tx=0).stop 2024-09-21T03:50:10.286 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:10.280+0000 7f860ac61740 1 -- 172.21.15.39:0/3582258255 shutdown_connections 2024-09-21T03:50:10.286 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:10.280+0000 7f860ac61740 1 --2- 172.21.15.39:0/3582258255 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e95c45f070 0x55e95c81e1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:10.286 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:10.280+0000 7f860ac61740 1 --2- 172.21.15.39:0/3582258255 >> [v2:172.21.15.161:6836/3451544597,v1:172.21.15.161:6837/3451544597] conn(0x55e95c41d9f0 0x55e95c8240f0 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:10.286 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:10.280+0000 7f85e5ffb640 0 client.16488 ms_handle_reset on v2:172.21.15.161:6836/3451544597 2024-09-21T03:50:10.286 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:10.280+0000 7f860ac61740 1 --2- 172.21.15.39:0/3582258255 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e95c43e390 0x55e95c812340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:10.287 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:10.280+0000 7f860ac61740 1 --2- 172.21.15.39:0/3582258255 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55e95c4361b0 0x55e95c811e00 unknown :-1 s=CLOSED pgs=746 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:10.287 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:10.280+0000 7f860ac61740 1 -- 172.21.15.39:0/3582258255 >> 172.21.15.39:0/3582258255 conn(0x55e95c453460 msgr2=0x55e95c809830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:50:10.287 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:10.280+0000 7f860ac61740 1 -- 172.21.15.39:0/3582258255 shutdown_connections 2024-09-21T03:50:10.287 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:10.280+0000 7f860ac61740 1 -- 172.21.15.39:0/3582258255 wait complete. 2024-09-21T03:50:10.287 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:10.280+0000 7f860ac61740 20 client.16488 trim_cache size 0 max 16384 2024-09-21T03:50:10.297 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:10.293+0000 7f860ac61740 1 Processor -- start 2024-09-21T03:50:10.297 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:10.293+0000 7f860ac61740 1 -- start start 2024-09-21T03:50:10.297 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:10.294+0000 7f860ac61740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e95c41d9f0 0x55e95c8122c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:50:10.297 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:10.294+0000 7f860ac61740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55e95c4361b0 0x55e95c81e0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:50:10.297 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:10.294+0000 7f860ac61740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e95c43e390 0x55e95c803bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:50:10.298 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:10.294+0000 7f860ac61740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f85fc1af6d0 con 0x55e95c4361b0 2024-09-21T03:50:10.298 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:10.294+0000 7f860ac61740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f85fc1b1210 con 0x55e95c43e390 2024-09-21T03:50:10.298 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:10.294+0000 7f860ac61740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f85fc1afda0 con 0x55e95c41d9f0 2024-09-21T03:50:10.298 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:10.294+0000 7f8601c88640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55e95c4361b0 0x55e95c81e0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:50:10.298 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:10.294+0000 7f8601c88640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55e95c4361b0 0x55e95c81e0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:56666/0 (socket says 172.21.15.39:56666) 2024-09-21T03:50:10.298 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:10.294+0000 7f8601c88640 1 -- 172.21.15.39:0/35829918 learned_addr learned my addr 172.21.15.39:0/35829918 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:50:10.298 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:10.294+0000 7f8602489640 1 --2- 172.21.15.39:0/35829918 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e95c41d9f0 0x55e95c8122c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:50:10.298 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:10.294+0000 7f8602c8a640 1 --2- 172.21.15.39:0/35829918 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e95c43e390 0x55e95c803bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:50:10.298 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:10.294+0000 7f8601c88640 1 -- 172.21.15.39:0/35829918 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e95c41d9f0 msgr2=0x55e95c8122c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:50:10.298 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:10.294+0000 7f8601c88640 1 --2- 172.21.15.39:0/35829918 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e95c41d9f0 0x55e95c8122c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:10.298 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:10.294+0000 7f8601c88640 1 -- 172.21.15.39:0/35829918 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e95c43e390 msgr2=0x55e95c803bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:50:10.298 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:10.294+0000 7f8601c88640 1 --2- 172.21.15.39:0/35829918 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e95c43e390 0x55e95c803bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:10.298 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:10.294+0000 7f8601c88640 1 -- 172.21.15.39:0/35829918 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e95c810300 con 0x55e95c4361b0 2024-09-21T03:50:10.298 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:10.294+0000 7f8601c88640 1 --2- 172.21.15.39:0/35829918 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55e95c4361b0 0x55e95c81e0a0 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7f85ec0cd420 tx=0x7f85ec003f50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee3431869071eada server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:50:10.299 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:10.294+0000 7f85e7fff640 1 -- 172.21.15.39:0/35829918 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85ec049040 con 0x55e95c4361b0 2024-09-21T03:50:10.299 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:10.295+0000 7f85e7fff640 1 -- 172.21.15.39:0/35829918 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f85ec111040 con 0x55e95c4361b0 2024-09-21T03:50:10.299 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:10.295+0000 7f85e7fff640 1 -- 172.21.15.39:0/35829918 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85ec110040 con 0x55e95c4361b0 2024-09-21T03:50:10.302 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:10.295+0000 7f860ac61740 1 -- 172.21.15.39:0/35829918 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55e95c4361b0 msgr2=0x55e95c81e0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:50:10.303 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:10.295+0000 7f860ac61740 1 --2- 172.21.15.39:0/35829918 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55e95c4361b0 0x55e95c81e0a0 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7f85ec0cd420 tx=0x7f85ec003f50 comp rx=0 tx=0).stop 2024-09-21T03:50:10.303 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:10.295+0000 7f860ac61740 1 -- 172.21.15.39:0/35829918 shutdown_connections 2024-09-21T03:50:10.303 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:10.295+0000 7f860ac61740 1 --2- 172.21.15.39:0/35829918 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e95c43e390 0x55e95c803bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:10.303 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:10.295+0000 7f860ac61740 1 --2- 172.21.15.39:0/35829918 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55e95c4361b0 0x55e95c81e0a0 unknown :-1 s=CLOSED pgs=747 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:10.303 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:10.295+0000 7f860ac61740 1 --2- 172.21.15.39:0/35829918 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e95c41d9f0 0x55e95c8122c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:10.303 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:10.295+0000 7f860ac61740 1 -- 172.21.15.39:0/35829918 >> 172.21.15.39:0/35829918 conn(0x55e95c453460 msgr2=0x55e95c811bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:50:10.303 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:10.295+0000 7f860ac61740 1 -- 172.21.15.39:0/35829918 shutdown_connections 2024-09-21T03:50:10.303 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:10.295+0000 7f860ac61740 1 -- 172.21.15.39:0/35829918 wait complete. 2024-09-21T03:50:10.303 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:10.296+0000 7f860ac61740 1 -- start start 2024-09-21T03:50:10.303 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:10.296+0000 7f860ac61740 1 Processor -- start 2024-09-21T03:50:10.303 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:10.296+0000 7f860ac61740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55e95c41d9f0 0x55e95c816060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:50:10.303 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:10.296+0000 7f860ac61740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e95c4361b0 0x55e95c8165a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:50:10.303 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:10.296+0000 7f860ac61740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e95c43e390 0x55e95c816ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:50:10.304 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:10.296+0000 7f860ac61740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f85fc1b0b40 con 0x55e95c41d9f0 2024-09-21T03:50:10.304 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:10.296+0000 7f860ac61740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f85fc1b0470 con 0x55e95c4361b0 2024-09-21T03:50:10.304 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:10.296+0000 7f860ac61740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f85fc1ad6b0 con 0x55e95c43e390 2024-09-21T03:50:10.304 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:10.296+0000 7f860ac61740 10 client.0 _ll_register_callbacks cb 0x7f85f800b820 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:50:10.304 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:10.296+0000 7f8602489640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55e95c41d9f0 0x55e95c816060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:50:10.304 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:10.296+0000 7f8602489640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55e95c41d9f0 0x55e95c816060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:56672/0 (socket says 172.21.15.39:56672) 2024-09-21T03:50:10.304 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:10.296+0000 7f8602489640 1 -- 172.21.15.39:0/2054250676 learned_addr learned my addr 172.21.15.39:0/2054250676 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:50:10.304 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:10.296+0000 7f8601c88640 1 --2- 172.21.15.39:0/2054250676 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e95c4361b0 0x55e95c8165a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:50:10.304 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:10.296+0000 7f8602c8a640 1 --2- 172.21.15.39:0/2054250676 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e95c43e390 0x55e95c816ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:50:10.304 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:10.297+0000 7f8602489640 1 -- 172.21.15.39:0/2054250676 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e95c43e390 msgr2=0x55e95c816ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:50:10.304 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:10.297+0000 7f8602489640 1 --2- 172.21.15.39:0/2054250676 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e95c43e390 0x55e95c816ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:10.304 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:10.297+0000 7f8602489640 1 -- 172.21.15.39:0/2054250676 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e95c4361b0 msgr2=0x55e95c8165a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:50:10.304 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:10.297+0000 7f8602489640 1 --2- 172.21.15.39:0/2054250676 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e95c4361b0 0x55e95c8165a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:10.305 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:10.297+0000 7f8602489640 1 -- 172.21.15.39:0/2054250676 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e95c88db00 con 0x55e95c41d9f0 2024-09-21T03:50:10.305 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:10.297+0000 7f8602489640 1 --2- 172.21.15.39:0/2054250676 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55e95c41d9f0 0x55e95c816060 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7f85f80059f0 tx=0x7f85f8010b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bae7fc6c3d3d49c6 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:50:10.305 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:10.297+0000 7f85e5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-09-21T03:50:10.305 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:10.297+0000 7f85e5ffb640 1 -- 172.21.15.39:0/2054250676 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85f8005240 con 0x55e95c41d9f0 2024-09-21T03:50:10.305 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:10.297+0000 7f85e5ffb640 1 -- 172.21.15.39:0/2054250676 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8484+0+0 (secure 0 0 0) 0x7f85f80053e0 con 0x55e95c41d9f0 2024-09-21T03:50:10.305 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:10.297+0000 7f860ac61740 1 -- 172.21.15.39:0/2054250676 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f85ec111040 con 0x55e95c41d9f0 2024-09-21T03:50:10.305 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:10.297+0000 7f85e5ffb640 1 -- 172.21.15.39:0/2054250676 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f85f80055e0 con 0x55e95c41d9f0 2024-09-21T03:50:10.305 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:10.297+0000 7f860ac61740 10 client.16494 fetch_fsmap learned FSMap version 29 2024-09-21T03:50:10.305 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:10.297+0000 7f860ac61740 1 -- 172.21.15.39:0/2054250676 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55e95c88ed40 con 0x55e95c41d9f0 2024-09-21T03:50:10.305 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:10.298+0000 7f85e5ffb640 1 -- 172.21.15.39:0/2054250676 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f85ec111040 con 0x55e95c41d9f0 2024-09-21T03:50:10.305 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:10.298+0000 7f85e5ffb640 1 -- 172.21.15.39:0/2054250676 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f85f8040070 con 0x55e95c41d9f0 2024-09-21T03:50:10.305 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:10.298+0000 7f860ac61740 10 client.16494 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T03:50:10.305 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:10.298+0000 7f860ac61740 10 client.16494 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-21T03:50:10.305 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:10.298+0000 7f860ac61740 20 client.16494 populate_metadata read hostname 'smithi039' 2024-09-21T03:50:10.306 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:10.298+0000 7f860ac61740 1 --2- 172.21.15.39:0/2054250676 >> [v2:172.21.15.161:6836/3451544597,v1:172.21.15.161:6837/3451544597] conn(0x55e95c45f070 0x55e95c891580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:50:10.306 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:10.298+0000 7f860ac61740 4 client.16494 mds_command: new command op to 24646 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T03:50:10.306 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:10.298+0000 7f860ac61740 1 -- 172.21.15.39:0/2054250676 --> [v2:172.21.15.161:6836/3451544597,v1:172.21.15.161:6837/3451544597] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55e95c77a8a0 con 0x55e95c45f070 2024-09-21T03:50:10.306 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:10.298+0000 7f8601c88640 1 --2- 172.21.15.39:0/2054250676 >> [v2:172.21.15.161:6836/3451544597,v1:172.21.15.161:6837/3451544597] conn(0x55e95c45f070 0x55e95c891580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:50:10.306 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:10.298+0000 7f8601c88640 1 --2- 172.21.15.39:0/2054250676 >> [v2:172.21.15.161:6836/3451544597,v1:172.21.15.161:6837/3451544597] conn(0x55e95c45f070 0x55e95c891580 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=f86819c7134c1976 server_cookie=6ff58e0304a850f4 in_seq=0 out_seq=0 2024-09-21T03:50:10.306 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:10.298+0000 7f85e5ffb640 10 client.16494 ms_handle_connect on v2:172.21.15.161:6836/3451544597 2024-09-21T03:50:10.306 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:10.299+0000 7f85e5ffb640 1 -- 172.21.15.39:0/2054250676 <== mds.3 v2:172.21.15.161:6836/3451544597 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55e95c77a8a0 con 0x55e95c45f070 2024-09-21T03:50:10.306 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:10.299+0000 7f85e5ffb640 10 client.16494 handle_command_reply: tid=0 2024-09-21T03:50:10.306 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:10.299+0000 7f860ac61740 1 client.16494 shutdown 2024-09-21T03:50:10.306 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:10.299+0000 7f860ac61740 1 -- 172.21.15.39:0/2054250676 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55e95c41d9f0 msgr2=0x55e95c816060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:50:10.306 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:10.299+0000 7f860ac61740 1 --2- 172.21.15.39:0/2054250676 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55e95c41d9f0 0x55e95c816060 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7f85f80059f0 tx=0x7f85f8010b50 comp rx=0 tx=0).stop 2024-09-21T03:50:10.306 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:10.299+0000 7f860ac61740 1 -- 172.21.15.39:0/2054250676 shutdown_connections 2024-09-21T03:50:10.306 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:10.299+0000 7f860ac61740 1 --2- 172.21.15.39:0/2054250676 >> [v2:172.21.15.161:6836/3451544597,v1:172.21.15.161:6837/3451544597] conn(0x55e95c45f070 0x55e95c891580 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:10.306 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:10.299+0000 7f860ac61740 1 --2- 172.21.15.39:0/2054250676 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e95c43e390 0x55e95c816ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:10.306 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:10.299+0000 7f860ac61740 1 --2- 172.21.15.39:0/2054250676 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e95c4361b0 0x55e95c8165a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:10.307 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:10.299+0000 7f85e5ffb640 0 client.16494 ms_handle_reset on v2:172.21.15.161:6836/3451544597 2024-09-21T03:50:10.307 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:10.299+0000 7f860ac61740 1 --2- 172.21.15.39:0/2054250676 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55e95c41d9f0 0x55e95c816060 unknown :-1 s=CLOSED pgs=748 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:10.307 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:10.299+0000 7f860ac61740 1 -- 172.21.15.39:0/2054250676 >> 172.21.15.39:0/2054250676 conn(0x55e95c453460 msgr2=0x55e95c8076d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:50:10.307 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:10.299+0000 7f860ac61740 1 -- 172.21.15.39:0/2054250676 shutdown_connections 2024-09-21T03:50:10.307 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:10.299+0000 7f860ac61740 1 -- 172.21.15.39:0/2054250676 wait complete. 2024-09-21T03:50:10.307 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:10.299+0000 7f860ac61740 20 client.16494 trim_cache size 0 max 16384 2024-09-21T03:50:10.307 INFO:teuthology.orchestra.run.smithi039.stdout:[] 2024-09-21T03:50:10.308 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:10.299+0000 7f86046ed640 1 -- 172.21.15.39:0/3736085481 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f85e80b66b0 msgr2=0x7f85e80b8bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:50:10.308 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:10.299+0000 7f86046ed640 1 --2- 172.21.15.39:0/3736085481 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f85e80b66b0 0x7f85e80b8bc0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x55e95c805320 tx=0x7f85f800b390 comp rx=0 tx=0).stop 2024-09-21T03:50:10.308 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:10.300+0000 7f86046ed640 1 -- 172.21.15.39:0/3736085481 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f85fc105320 msgr2=0x7f85fc19aa60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:50:10.308 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:10.300+0000 7f86046ed640 1 --2- 172.21.15.39:0/3736085481 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f85fc105320 0x7f85fc19aa60 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7f85ec02a910 tx=0x7f85ec004270 comp rx=0 tx=0).stop 2024-09-21T03:50:10.308 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:10.300+0000 7f86046ed640 1 -- 172.21.15.39:0/3736085481 shutdown_connections 2024-09-21T03:50:10.308 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:10.300+0000 7f86046ed640 1 --2- 172.21.15.39:0/3736085481 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f85e80b66b0 0x7f85e80b8bc0 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:10.308 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:10.300+0000 7f86046ed640 1 --2- 172.21.15.39:0/3736085481 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f85fc10de30 0x7f85fc19fd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:10.308 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:10.300+0000 7f86046ed640 1 --2- 172.21.15.39:0/3736085481 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f85fc105320 0x7f85fc19aa60 unknown :-1 s=CLOSED pgs=745 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:10.308 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:10.300+0000 7f86046ed640 1 --2- 172.21.15.39:0/3736085481 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f85fc1049d0 0x7f85fc19a4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:10.308 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:10.300+0000 7f86046ed640 1 -- 172.21.15.39:0/3736085481 >> 172.21.15.39:0/3736085481 conn(0x7f85fc079740 msgr2=0x7f85fc1008a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:50:10.309 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:10.300+0000 7f86046ed640 1 -- 172.21.15.39:0/3736085481 shutdown_connections 2024-09-21T03:50:10.309 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:10.300+0000 7f86046ed640 1 -- 172.21.15.39:0/3736085481 wait complete. 2024-09-21T03:50:11.015 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:50:10 smithi039 ceph-mon[31211]: pgmap v449: 97 pgs: 97 active+clean; 10 MiB data, 536 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 22 KiB/s wr, 26 op/s 2024-09-21T03:50:11.044 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 51116d5c-77ca-11ef-baf5-efdc52797490 -- ceph tell mds.1:2 damage ls 2024-09-21T03:50:11.218 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:50:10 smithi161 ceph-mon[34533]: pgmap v449: 97 pgs: 97 active+clean; 10 MiB data, 536 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 22 KiB/s wr, 26 op/s 2024-09-21T03:50:11.224 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:50:10 smithi150 ceph-mon[34289]: pgmap v449: 97 pgs: 97 active+clean; 10 MiB data, 536 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 22 KiB/s wr, 26 op/s 2024-09-21T03:50:11.300 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/51116d5c-77ca-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:50:11.795 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:11.785+0000 7f703d4d4640 1 -- 172.21.15.39:0/2950649042 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7038108cf0 msgr2=0x7f703810f520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:50:11.795 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:11.785+0000 7f703d4d4640 1 --2- 172.21.15.39:0/2950649042 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7038108cf0 0x7f703810f520 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7f7030009900 tx=0x7f703002a410 comp rx=0 tx=0).stop 2024-09-21T03:50:11.795 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:11.785+0000 7f703d4d4640 1 -- 172.21.15.39:0/2950649042 shutdown_connections 2024-09-21T03:50:11.795 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:11.785+0000 7f703d4d4640 1 --2- 172.21.15.39:0/2950649042 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7038108cf0 0x7f703810f520 unknown :-1 s=CLOSED pgs=749 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:11.795 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:11.785+0000 7f703d4d4640 1 --2- 172.21.15.39:0/2950649042 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f70381083a0 0x7f70381087b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:11.795 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:11.785+0000 7f703d4d4640 1 --2- 172.21.15.39:0/2950649042 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7038102a60 0x7f7038102e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:11.795 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:11.785+0000 7f703d4d4640 1 -- 172.21.15.39:0/2950649042 >> 172.21.15.39:0/2950649042 conn(0x7f7038079730 msgr2=0x7f7038079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:50:11.795 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:11.785+0000 7f703d4d4640 1 -- 172.21.15.39:0/2950649042 shutdown_connections 2024-09-21T03:50:11.795 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:11.785+0000 7f703d4d4640 1 -- 172.21.15.39:0/2950649042 wait complete. 2024-09-21T03:50:11.796 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:11.786+0000 7f703d4d4640 1 Processor -- start 2024-09-21T03:50:11.796 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:11.786+0000 7f703d4d4640 1 -- start start 2024-09-21T03:50:11.796 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:11.786+0000 7f703d4d4640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7038102a60 0x7f703819b290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:50:11.796 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:11.786+0000 7f703d4d4640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f70381083a0 0x7f703819b7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:50:11.796 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:11.786+0000 7f703d4d4640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7038108cf0 0x7f703819bd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:50:11.796 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:11.786+0000 7f703d4d4640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f7038112960 con 0x7f7038102a60 2024-09-21T03:50:11.796 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:11.786+0000 7f703d4d4640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f70381127e0 con 0x7f7038108cf0 2024-09-21T03:50:11.796 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:11.786+0000 7f703d4d4640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f7038112ae0 con 0x7f70381083a0 2024-09-21T03:50:11.796 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:11.786+0000 7f7036ffd640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7038102a60 0x7f703819b290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:50:11.796 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:11.787+0000 7f7036ffd640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7038102a60 0x7f703819b290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:56688/0 (socket says 172.21.15.39:56688) 2024-09-21T03:50:11.796 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:11.787+0000 7f7036ffd640 1 -- 172.21.15.39:0/2142356031 learned_addr learned my addr 172.21.15.39:0/2142356031 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:50:11.796 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:11.787+0000 7f70367fc640 1 --2- 172.21.15.39:0/2142356031 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f70381083a0 0x7f703819b7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:50:11.797 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:11.787+0000 7f7036ffd640 1 -- 172.21.15.39:0/2142356031 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f70381083a0 msgr2=0x7f703819b7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:50:11.797 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:11.787+0000 7f70377fe640 1 --2- 172.21.15.39:0/2142356031 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7038108cf0 0x7f703819bd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:50:11.797 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:11.787+0000 7f7036ffd640 1 --2- 172.21.15.39:0/2142356031 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f70381083a0 0x7f703819b7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:11.797 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:11.787+0000 7f7036ffd640 1 -- 172.21.15.39:0/2142356031 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7038108cf0 msgr2=0x7f703819bd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:50:11.797 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:11.787+0000 7f7036ffd640 1 --2- 172.21.15.39:0/2142356031 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7038108cf0 0x7f703819bd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:11.797 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:11.787+0000 7f7036ffd640 1 -- 172.21.15.39:0/2142356031 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f70381a6520 con 0x7f7038102a60 2024-09-21T03:50:11.797 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:11.787+0000 7f7036ffd640 1 --2- 172.21.15.39:0/2142356031 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7038102a60 0x7f703819b290 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7f702800ea80 tx=0x7f702800ef40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:50:11.797 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:11.787+0000 7f702ffff640 1 -- 172.21.15.39:0/2142356031 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f702800ce40 con 0x7f7038102a60 2024-09-21T03:50:11.797 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:11.787+0000 7f703d4d4640 1 -- 172.21.15.39:0/2142356031 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f70381a6810 con 0x7f7038102a60 2024-09-21T03:50:11.797 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:11.787+0000 7f702ffff640 1 -- 172.21.15.39:0/2142356031 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7028005020 con 0x7f7038102a60 2024-09-21T03:50:11.797 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:11.787+0000 7f702ffff640 1 -- 172.21.15.39:0/2142356031 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70280116d0 con 0x7f7038102a60 2024-09-21T03:50:11.797 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:11.787+0000 7f703d4d4640 1 -- 172.21.15.39:0/2142356031 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f70381a6d00 con 0x7f7038102a60 2024-09-21T03:50:11.797 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:11.790+0000 7f702ffff640 1 -- 172.21.15.39:0/2142356031 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 27) ==== 154643+0+0 (secure 0 0 0) 0x7f70280054d0 con 0x7f7038102a60 2024-09-21T03:50:11.797 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:11.790+0000 7f7043a48740 1 Processor -- start 2024-09-21T03:50:11.797 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:11.790+0000 7f7043a48740 1 -- start start 2024-09-21T03:50:11.798 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:11.791+0000 7f7043a48740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d485944070 0x55d485ceb6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:50:11.798 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:11.791+0000 7f7043a48740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d48591b1b0 0x55d485cebbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:50:11.798 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:11.791+0000 7f7043a48740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d485923390 0x55d485cec120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:50:11.798 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:11.791+0000 7f7043a48740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x55d485c61ea0 con 0x55d485944070 2024-09-21T03:50:11.798 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:11.791+0000 7f7043a48740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x55d485c512d0 con 0x55d48591b1b0 2024-09-21T03:50:11.798 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:11.791+0000 7f7043a48740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55d485c550e0 con 0x55d485923390 2024-09-21T03:50:11.798 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:11.791+0000 7f70377fe640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d485944070 0x55d485ceb6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:50:11.798 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:11.791+0000 7f70377fe640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d485944070 0x55d485ceb6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:56692/0 (socket says 172.21.15.39:56692) 2024-09-21T03:50:11.798 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:11.791+0000 7f70377fe640 1 -- 172.21.15.39:0/1186233041 learned_addr learned my addr 172.21.15.39:0/1186233041 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:50:11.798 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:11.791+0000 7f70367fc640 1 --2- 172.21.15.39:0/1186233041 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d485923390 0x55d485cec120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:50:11.798 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:11.791+0000 7f7036ffd640 1 --2- 172.21.15.39:0/1186233041 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d48591b1b0 0x55d485cebbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:50:11.798 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:11.791+0000 7f702ffff640 1 --2- 172.21.15.39:0/2142356031 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f70240b66b0 0x7f70240b8bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:50:11.798 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:11.791+0000 7f70377fe640 1 -- 172.21.15.39:0/1186233041 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d485923390 msgr2=0x55d485cec120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:50:11.799 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:11.791+0000 7f70377fe640 1 --2- 172.21.15.39:0/1186233041 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d485923390 0x55d485cec120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:11.799 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:11.791+0000 7f70377fe640 1 -- 172.21.15.39:0/1186233041 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d48591b1b0 msgr2=0x55d485cebbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:50:11.799 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:11.791+0000 7f70377fe640 1 --2- 172.21.15.39:0/1186233041 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d48591b1b0 0x55d485cebbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:11.799 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:11.791+0000 7f70377fe640 1 -- 172.21.15.39:0/1186233041 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d4859af650 con 0x55d485944070 2024-09-21T03:50:11.799 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:11.791+0000 7f70367fc640 1 --2- 172.21.15.39:0/1186233041 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d485923390 0x55d485cec120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:50:11.799 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:11.791+0000 7f70377fe640 1 --2- 172.21.15.39:0/2142356031 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f70240b66b0 0x7f70240b8bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:50:11.799 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:11.792+0000 7f702ffff640 1 -- 172.21.15.39:0/2142356031 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8484+0+0 (secure 0 0 0) 0x7f70280e7a40 con 0x7f7038102a60 2024-09-21T03:50:11.799 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:11.792+0000 7f70377fe640 1 --2- 172.21.15.39:0/1186233041 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d485944070 0x55d485ceb6a0 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7f7030005940 tx=0x7f7030032d40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=524b0ca9b12dc081 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:50:11.799 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:11.792+0000 7f702d7fa640 1 -- 172.21.15.39:0/1186233041 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7030035070 con 0x55d485944070 2024-09-21T03:50:11.799 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:11.792+0000 7f702d7fa640 1 -- 172.21.15.39:0/1186233041 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f70300054f0 con 0x55d485944070 2024-09-21T03:50:11.799 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:11.792+0000 7f702d7fa640 1 -- 172.21.15.39:0/1186233041 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7030004930 con 0x55d485944070 2024-09-21T03:50:11.799 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:11.792+0000 7f70377fe640 1 --2- 172.21.15.39:0/2142356031 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f70240b66b0 0x7f70240b8bc0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f7030005f70 tx=0x7f7030045000 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:50:11.803 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:11.793+0000 7f7043a48740 1 -- 172.21.15.39:0/1186233041 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d485944070 msgr2=0x55d485ceb6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:50:11.803 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:11.793+0000 7f7043a48740 1 --2- 172.21.15.39:0/1186233041 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d485944070 0x55d485ceb6a0 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7f7030005940 tx=0x7f7030032d40 comp rx=0 tx=0).stop 2024-09-21T03:50:11.803 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:11.793+0000 7f7043a48740 1 -- 172.21.15.39:0/1186233041 shutdown_connections 2024-09-21T03:50:11.803 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:11.793+0000 7f7043a48740 1 --2- 172.21.15.39:0/1186233041 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d485923390 0x55d485cec120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:11.803 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:11.793+0000 7f7043a48740 1 --2- 172.21.15.39:0/1186233041 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d48591b1b0 0x55d485cebbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:11.803 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:11.793+0000 7f7043a48740 1 --2- 172.21.15.39:0/1186233041 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d485944070 0x55d485ceb6a0 unknown :-1 s=CLOSED pgs=751 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:11.804 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:11.793+0000 7f7043a48740 1 -- 172.21.15.39:0/1186233041 >> 172.21.15.39:0/1186233041 conn(0x55d485938460 msgr2=0x55d485ce9590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:50:11.804 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:11.793+0000 7f7043a48740 1 -- 172.21.15.39:0/1186233041 shutdown_connections 2024-09-21T03:50:11.804 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:11.793+0000 7f7043a48740 1 -- 172.21.15.39:0/1186233041 wait complete. 2024-09-21T03:50:11.804 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:11.794+0000 7f7043a48740 1 -- start start 2024-09-21T03:50:11.804 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:11.794+0000 7f7043a48740 1 Processor -- start 2024-09-21T03:50:11.804 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:11.794+0000 7f7043a48740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d48591b1b0 0x55d485cf6d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:50:11.804 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:11.794+0000 7f7043a48740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d485923390 0x55d485cf7260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:50:11.804 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:11.794+0000 7f7043a48740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d485944070 0x55d485d03170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:50:11.804 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:11.794+0000 7f7043a48740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x55d485ab2700 con 0x55d485944070 2024-09-21T03:50:11.804 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:11.794+0000 7f7043a48740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x55d485bec160 con 0x55d485923390 2024-09-21T03:50:11.804 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:11.794+0000 7f7043a48740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55d485cf9e80 con 0x55d48591b1b0 2024-09-21T03:50:11.804 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:11.794+0000 7f7043a48740 10 client.0 _ll_register_callbacks cb 0x55d485c059e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:50:11.804 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:11.794+0000 7f70377fe640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d485944070 0x55d485d03170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:50:11.805 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:11.794+0000 7f70377fe640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d485944070 0x55d485d03170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:56698/0 (socket says 172.21.15.39:56698) 2024-09-21T03:50:11.805 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:11.794+0000 7f70377fe640 1 -- 172.21.15.39:0/1212736741 learned_addr learned my addr 172.21.15.39:0/1212736741 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:50:11.805 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:11.794+0000 7f7036ffd640 1 --2- 172.21.15.39:0/1212736741 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d48591b1b0 0x55d485cf6d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:50:11.805 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:11.794+0000 7f70367fc640 1 --2- 172.21.15.39:0/1212736741 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d485923390 0x55d485cf7260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:50:11.805 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:11.795+0000 7f70377fe640 1 -- 172.21.15.39:0/1212736741 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d48591b1b0 msgr2=0x55d485cf6d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:50:11.805 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:11.795+0000 7f70377fe640 1 --2- 172.21.15.39:0/1212736741 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d48591b1b0 0x55d485cf6d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:11.805 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:11.795+0000 7f70377fe640 1 -- 172.21.15.39:0/1212736741 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d485923390 msgr2=0x55d485cf7260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:50:11.805 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:11.795+0000 7f70377fe640 1 --2- 172.21.15.39:0/1212736741 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d485923390 0x55d485cf7260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:11.805 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:11.795+0000 7f70377fe640 1 -- 172.21.15.39:0/1212736741 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d485c19c90 con 0x55d485944070 2024-09-21T03:50:11.805 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:11.795+0000 7f70367fc640 1 --2- 172.21.15.39:0/1212736741 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d485923390 0x55d485cf7260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:50:11.805 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:11.795+0000 7f70377fe640 1 --2- 172.21.15.39:0/1212736741 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d485944070 0x55d485d03170 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7f7030004960 tx=0x7f7030004990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5375c3ebe34dc26d server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:50:11.805 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:11.795+0000 7f701e7fc640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-09-21T03:50:11.805 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:11.795+0000 7f701e7fc640 1 -- 172.21.15.39:0/1212736741 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7030035040 con 0x55d485944070 2024-09-21T03:50:11.805 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:11.795+0000 7f7043a48740 1 -- 172.21.15.39:0/1212736741 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f70300054f0 con 0x55d485944070 2024-09-21T03:50:11.805 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:11.795+0000 7f701e7fc640 1 -- 172.21.15.39:0/1212736741 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8484+0+0 (secure 0 0 0) 0x7f7030005050 con 0x55d485944070 2024-09-21T03:50:11.806 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:11.795+0000 7f701e7fc640 1 -- 172.21.15.39:0/1212736741 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f7030005250 con 0x55d485944070 2024-09-21T03:50:11.806 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:11.795+0000 7f7043a48740 10 client.16506 fetch_fsmap learned FSMap version 29 2024-09-21T03:50:11.806 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:11.795+0000 7f7043a48740 1 -- 172.21.15.39:0/1212736741 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55d485d05b50 con 0x55d485944070 2024-09-21T03:50:11.806 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:11.796+0000 7f701e7fc640 1 -- 172.21.15.39:0/1212736741 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f70300054f0 con 0x55d485944070 2024-09-21T03:50:11.806 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:11.796+0000 7f701e7fc640 1 -- 172.21.15.39:0/1212736741 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f7030046460 con 0x55d485944070 2024-09-21T03:50:11.806 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:11.796+0000 7f7043a48740 10 client.16506 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T03:50:11.806 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:11.796+0000 7f7043a48740 10 client.16506 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-21T03:50:11.806 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:11.796+0000 7f7043a48740 20 client.16506 populate_metadata read hostname 'smithi039' 2024-09-21T03:50:11.806 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:11.796+0000 7f7043a48740 1 --2- 172.21.15.39:0/1212736741 >> [v2:172.21.15.150:6836/4017043144,v1:172.21.15.150:6837/4017043144] conn(0x55d4859029f0 0x55d485d08eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:50:11.806 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:11.796+0000 7f7043a48740 4 client.16506 mds_command: new command op to 24659 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:50:11.806 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:11.796+0000 7f7043a48740 1 -- 172.21.15.39:0/1212736741 --> [v2:172.21.15.150:6836/4017043144,v1:172.21.15.150:6837/4017043144] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d485c63df0 con 0x55d4859029f0 2024-09-21T03:50:11.806 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:11.796+0000 7f7036ffd640 1 --2- 172.21.15.39:0/1212736741 >> [v2:172.21.15.150:6836/4017043144,v1:172.21.15.150:6837/4017043144] conn(0x55d4859029f0 0x55d485d08eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:50:11.806 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:11.797+0000 7f7036ffd640 1 --2- 172.21.15.39:0/1212736741 >> [v2:172.21.15.150:6836/4017043144,v1:172.21.15.150:6837/4017043144] conn(0x55d4859029f0 0x55d485d08eb0 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=3ea9e11f8b4f36d4 server_cookie=d9460fe8a86843b6 in_seq=0 out_seq=0 2024-09-21T03:50:11.806 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:11.797+0000 7f701e7fc640 10 client.16506 ms_handle_connect on v2:172.21.15.150:6836/4017043144 2024-09-21T03:50:11.806 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:11.797+0000 7f701e7fc640 1 -- 172.21.15.39:0/1212736741 <== mds.2 v2:172.21.15.150:6836/4017043144 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55d485c63df0 con 0x55d4859029f0 2024-09-21T03:50:11.807 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:11.797+0000 7f701e7fc640 10 client.16506 handle_command_reply: tid=0 2024-09-21T03:50:11.807 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:11.797+0000 7f7043a48740 1 client.16506 shutdown 2024-09-21T03:50:11.807 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:11.798+0000 7f7043a48740 1 -- 172.21.15.39:0/1212736741 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d485944070 msgr2=0x55d485d03170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:50:11.807 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:11.798+0000 7f7043a48740 1 --2- 172.21.15.39:0/1212736741 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d485944070 0x55d485d03170 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7f7030004960 tx=0x7f7030004990 comp rx=0 tx=0).stop 2024-09-21T03:50:11.807 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:11.798+0000 7f7043a48740 1 -- 172.21.15.39:0/1212736741 shutdown_connections 2024-09-21T03:50:11.807 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:11.798+0000 7f7043a48740 1 --2- 172.21.15.39:0/1212736741 >> [v2:172.21.15.150:6836/4017043144,v1:172.21.15.150:6837/4017043144] conn(0x55d4859029f0 0x55d485d08eb0 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:11.807 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:11.798+0000 7f7043a48740 1 --2- 172.21.15.39:0/1212736741 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d485944070 0x55d485d03170 unknown :-1 s=CLOSED pgs=752 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:11.807 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:11.798+0000 7f7043a48740 1 --2- 172.21.15.39:0/1212736741 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d485923390 0x55d485cf7260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:11.807 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:11.798+0000 7f7043a48740 1 --2- 172.21.15.39:0/1212736741 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d48591b1b0 0x55d485cf6d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:11.807 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:11.798+0000 7f7043a48740 1 -- 172.21.15.39:0/1212736741 >> 172.21.15.39:0/1212736741 conn(0x55d485938460 msgr2=0x55d485cee880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:50:11.807 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:11.798+0000 7f701e7fc640 0 client.16506 ms_handle_reset on v2:172.21.15.150:6836/4017043144 2024-09-21T03:50:11.807 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:11.798+0000 7f7043a48740 1 -- 172.21.15.39:0/1212736741 shutdown_connections 2024-09-21T03:50:11.807 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:11.798+0000 7f7043a48740 1 -- 172.21.15.39:0/1212736741 wait complete. 2024-09-21T03:50:11.807 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:11.798+0000 7f7043a48740 20 client.16506 trim_cache size 0 max 16384 2024-09-21T03:50:11.814 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:11.811+0000 7f7043a48740 1 Processor -- start 2024-09-21T03:50:11.814 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:11.811+0000 7f7043a48740 1 -- start start 2024-09-21T03:50:11.815 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:11.811+0000 7f7043a48740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d48591b1b0 0x55d485ced1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:50:11.815 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:11.811+0000 7f7043a48740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d485923390 0x55d485cf8c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:50:11.815 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:11.811+0000 7f7043a48740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d485944070 0x55d485ceeeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:50:11.815 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:11.811+0000 7f7043a48740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f70381ad4f0 con 0x55d48591b1b0 2024-09-21T03:50:11.815 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:11.811+0000 7f7043a48740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f70381af030 con 0x55d485923390 2024-09-21T03:50:11.815 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:11.811+0000 7f7043a48740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f70381adbc0 con 0x55d485944070 2024-09-21T03:50:11.815 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:11.812+0000 7f7036ffd640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d48591b1b0 0x55d485ced1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:50:11.815 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:11.812+0000 7f7036ffd640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d48591b1b0 0x55d485ced1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:56708/0 (socket says 172.21.15.39:56708) 2024-09-21T03:50:11.816 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:11.812+0000 7f7036ffd640 1 -- 172.21.15.39:0/30502760 learned_addr learned my addr 172.21.15.39:0/30502760 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:50:11.816 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:11.812+0000 7f70367fc640 1 --2- 172.21.15.39:0/30502760 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d485923390 0x55d485cf8c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:50:11.816 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:11.812+0000 7f7036ffd640 1 -- 172.21.15.39:0/30502760 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d485944070 msgr2=0x55d485ceeeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:50:11.816 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:11.812+0000 7f70377fe640 1 --2- 172.21.15.39:0/30502760 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d485944070 0x55d485ceeeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:50:11.816 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:11.812+0000 7f7036ffd640 1 --2- 172.21.15.39:0/30502760 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d485944070 0x55d485ceeeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:11.816 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:11.812+0000 7f7036ffd640 1 -- 172.21.15.39:0/30502760 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d485923390 msgr2=0x55d485cf8c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:50:11.816 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:11.812+0000 7f7036ffd640 1 --2- 172.21.15.39:0/30502760 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d485923390 0x55d485cf8c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:11.816 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:11.812+0000 7f7036ffd640 1 -- 172.21.15.39:0/30502760 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d485cedbc0 con 0x55d48591b1b0 2024-09-21T03:50:11.816 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:11.812+0000 7f70367fc640 1 --2- 172.21.15.39:0/30502760 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d485923390 0x55d485cf8c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:50:11.816 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:11.812+0000 7f70377fe640 1 --2- 172.21.15.39:0/30502760 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d485944070 0x55d485ceeeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:50:11.816 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:11.812+0000 7f7036ffd640 1 --2- 172.21.15.39:0/30502760 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d48591b1b0 0x55d485ced1b0 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7f70280ad840 tx=0x7f70280d1040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=886cbd0f14327d0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:50:11.816 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:11.812+0000 7f702cff9640 1 -- 172.21.15.39:0/30502760 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f702801f040 con 0x55d48591b1b0 2024-09-21T03:50:11.816 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:11.812+0000 7f702cff9640 1 -- 172.21.15.39:0/30502760 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f70280adfb0 con 0x55d48591b1b0 2024-09-21T03:50:11.816 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:11.812+0000 7f702cff9640 1 -- 172.21.15.39:0/30502760 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70280b3b20 con 0x55d48591b1b0 2024-09-21T03:50:11.819 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:11.813+0000 7f7043a48740 1 -- 172.21.15.39:0/30502760 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d48591b1b0 msgr2=0x55d485ced1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:50:11.819 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:11.813+0000 7f7043a48740 1 --2- 172.21.15.39:0/30502760 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d48591b1b0 0x55d485ced1b0 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7f70280ad840 tx=0x7f70280d1040 comp rx=0 tx=0).stop 2024-09-21T03:50:11.819 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:11.813+0000 7f7043a48740 1 -- 172.21.15.39:0/30502760 shutdown_connections 2024-09-21T03:50:11.819 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:11.813+0000 7f7043a48740 1 --2- 172.21.15.39:0/30502760 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d485944070 0x55d485ceeeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:11.820 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:11.813+0000 7f7043a48740 1 --2- 172.21.15.39:0/30502760 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d485923390 0x55d485cf8c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:11.820 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:11.813+0000 7f7043a48740 1 --2- 172.21.15.39:0/30502760 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d48591b1b0 0x55d485ced1b0 unknown :-1 s=CLOSED pgs=753 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:11.820 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:11.813+0000 7f7043a48740 1 -- 172.21.15.39:0/30502760 >> 172.21.15.39:0/30502760 conn(0x55d485938460 msgr2=0x55d485cee7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:50:11.820 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:11.813+0000 7f7043a48740 1 -- 172.21.15.39:0/30502760 shutdown_connections 2024-09-21T03:50:11.820 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:11.813+0000 7f7043a48740 1 -- 172.21.15.39:0/30502760 wait complete. 2024-09-21T03:50:11.820 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:11.814+0000 7f7043a48740 1 -- start start 2024-09-21T03:50:11.820 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:11.814+0000 7f7043a48740 1 Processor -- start 2024-09-21T03:50:11.820 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:11.814+0000 7f7043a48740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d48591b1b0 0x55d485d09e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:50:11.820 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:11.814+0000 7f7043a48740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d485923390 0x55d485d0a350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:50:11.820 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:11.814+0000 7f7043a48740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d485944070 0x55d485d0a890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:50:11.820 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:11.814+0000 7f7043a48740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f70381ae960 con 0x55d485944070 2024-09-21T03:50:11.820 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:11.814+0000 7f7043a48740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f70381ae290 con 0x55d485923390 2024-09-21T03:50:11.820 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:11.814+0000 7f7043a48740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f70381ab4d0 con 0x55d48591b1b0 2024-09-21T03:50:11.820 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:11.814+0000 7f7043a48740 10 client.0 _ll_register_callbacks cb 0x7f7030032bf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:50:11.820 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:11.814+0000 7f70377fe640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d485944070 0x55d485d0a890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:50:11.821 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:11.814+0000 7f70377fe640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d485944070 0x55d485d0a890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:56722/0 (socket says 172.21.15.39:56722) 2024-09-21T03:50:11.821 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:11.814+0000 7f70377fe640 1 -- 172.21.15.39:0/518798876 learned_addr learned my addr 172.21.15.39:0/518798876 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:50:11.821 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:11.815+0000 7f70367fc640 1 --2- 172.21.15.39:0/518798876 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d485923390 0x55d485d0a350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:50:11.821 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:11.815+0000 7f7036ffd640 1 --2- 172.21.15.39:0/518798876 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d48591b1b0 0x55d485d09e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:50:11.821 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:11.815+0000 7f70377fe640 1 -- 172.21.15.39:0/518798876 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d48591b1b0 msgr2=0x55d485d09e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:50:11.821 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:11.815+0000 7f70377fe640 1 --2- 172.21.15.39:0/518798876 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d48591b1b0 0x55d485d09e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:11.821 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:11.815+0000 7f70377fe640 1 -- 172.21.15.39:0/518798876 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d485923390 msgr2=0x55d485d0a350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:50:11.821 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:11.815+0000 7f70377fe640 1 --2- 172.21.15.39:0/518798876 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d485923390 0x55d485d0a350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:11.821 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:11.815+0000 7f70377fe640 1 -- 172.21.15.39:0/518798876 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d485d72630 con 0x55d485944070 2024-09-21T03:50:11.821 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:11.815+0000 7f7036ffd640 1 --2- 172.21.15.39:0/518798876 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d48591b1b0 0x55d485d09e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:50:11.821 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:11.815+0000 7f70377fe640 1 --2- 172.21.15.39:0/518798876 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d485944070 0x55d485d0a890 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7f7030006010 tx=0x7f7030005df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e34a3af5afa466be server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:50:11.821 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:11.815+0000 7f701e7fc640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-09-21T03:50:11.821 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:11.815+0000 7f701e7fc640 1 -- 172.21.15.39:0/518798876 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7030035070 con 0x55d485944070 2024-09-21T03:50:11.821 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:11.815+0000 7f701e7fc640 1 -- 172.21.15.39:0/518798876 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8484+0+0 (secure 0 0 0) 0x7f70300053f0 con 0x55d485944070 2024-09-21T03:50:11.821 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:11.815+0000 7f7043a48740 1 -- 172.21.15.39:0/518798876 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f70280adfb0 con 0x55d485944070 2024-09-21T03:50:11.822 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:11.815+0000 7f701e7fc640 1 -- 172.21.15.39:0/518798876 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f70300055f0 con 0x55d485944070 2024-09-21T03:50:11.822 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:11.815+0000 7f7043a48740 10 client.16512 fetch_fsmap learned FSMap version 29 2024-09-21T03:50:11.822 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:11.815+0000 7f7043a48740 1 -- 172.21.15.39:0/518798876 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55d485d73720 con 0x55d485944070 2024-09-21T03:50:11.822 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:11.816+0000 7f701e7fc640 1 -- 172.21.15.39:0/518798876 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f70280adfb0 con 0x55d485944070 2024-09-21T03:50:11.822 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:11.816+0000 7f701e7fc640 1 -- 172.21.15.39:0/518798876 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f703003d5f0 con 0x55d485944070 2024-09-21T03:50:11.822 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:11.816+0000 7f7043a48740 10 client.16512 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T03:50:11.822 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:11.816+0000 7f7043a48740 10 client.16512 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-21T03:50:11.822 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:11.816+0000 7f7043a48740 20 client.16512 populate_metadata read hostname 'smithi039' 2024-09-21T03:50:11.822 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:11.816+0000 7f7043a48740 1 --2- 172.21.15.39:0/518798876 >> [v2:172.21.15.150:6836/4017043144,v1:172.21.15.150:6837/4017043144] conn(0x55d4858d7da0 0x55d485d75f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:50:11.822 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:11.816+0000 7f7043a48740 4 client.16512 mds_command: new command op to 24659 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T03:50:11.822 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:11.816+0000 7f7043a48740 1 -- 172.21.15.39:0/518798876 --> [v2:172.21.15.150:6836/4017043144,v1:172.21.15.150:6837/4017043144] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55d485c5f8a0 con 0x55d4858d7da0 2024-09-21T03:50:11.822 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:11.816+0000 7f7036ffd640 1 --2- 172.21.15.39:0/518798876 >> [v2:172.21.15.150:6836/4017043144,v1:172.21.15.150:6837/4017043144] conn(0x55d4858d7da0 0x55d485d75f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:50:11.822 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:11.817+0000 7f7036ffd640 1 --2- 172.21.15.39:0/518798876 >> [v2:172.21.15.150:6836/4017043144,v1:172.21.15.150:6837/4017043144] conn(0x55d4858d7da0 0x55d485d75f10 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=14675bacc9ca5d97 server_cookie=2346d5a371ccd199 in_seq=0 out_seq=0 2024-09-21T03:50:11.822 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:11.817+0000 7f701e7fc640 10 client.16512 ms_handle_connect on v2:172.21.15.150:6836/4017043144 2024-09-21T03:50:11.824 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:11.817+0000 7f701e7fc640 1 -- 172.21.15.39:0/518798876 <== mds.2 v2:172.21.15.150:6836/4017043144 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55d485c5f8a0 con 0x55d4858d7da0 2024-09-21T03:50:11.824 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:11.817+0000 7f701e7fc640 10 client.16512 handle_command_reply: tid=0 2024-09-21T03:50:11.824 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:11.817+0000 7f7043a48740 1 client.16512 shutdown 2024-09-21T03:50:11.824 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:11.817+0000 7f7043a48740 1 -- 172.21.15.39:0/518798876 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d485944070 msgr2=0x55d485d0a890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:50:11.824 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:11.817+0000 7f7043a48740 1 --2- 172.21.15.39:0/518798876 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d485944070 0x55d485d0a890 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7f7030006010 tx=0x7f7030005df0 comp rx=0 tx=0).stop 2024-09-21T03:50:11.824 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:11.817+0000 7f7043a48740 1 -- 172.21.15.39:0/518798876 shutdown_connections 2024-09-21T03:50:11.824 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:11.817+0000 7f7043a48740 1 --2- 172.21.15.39:0/518798876 >> [v2:172.21.15.150:6836/4017043144,v1:172.21.15.150:6837/4017043144] conn(0x55d4858d7da0 0x55d485d75f10 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:11.824 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:11.817+0000 7f7043a48740 1 --2- 172.21.15.39:0/518798876 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d485944070 0x55d485d0a890 unknown :-1 s=CLOSED pgs=754 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:11.824 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:11.817+0000 7f7043a48740 1 --2- 172.21.15.39:0/518798876 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d485923390 0x55d485d0a350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:11.824 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:11.817+0000 7f7043a48740 1 --2- 172.21.15.39:0/518798876 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d48591b1b0 0x55d485d09e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:11.824 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:11.817+0000 7f7043a48740 1 -- 172.21.15.39:0/518798876 >> 172.21.15.39:0/518798876 conn(0x55d485938460 msgr2=0x55d485cef160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:50:11.824 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:11.817+0000 7f701e7fc640 0 client.16512 ms_handle_reset on v2:172.21.15.150:6836/4017043144 2024-09-21T03:50:11.824 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:11.817+0000 7f7043a48740 1 -- 172.21.15.39:0/518798876 shutdown_connections 2024-09-21T03:50:11.824 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:11.817+0000 7f7043a48740 1 -- 172.21.15.39:0/518798876 wait complete. 2024-09-21T03:50:11.825 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:11.817+0000 7f7043a48740 20 client.16512 trim_cache size 0 max 16384 2024-09-21T03:50:11.825 INFO:teuthology.orchestra.run.smithi039.stdout:[] 2024-09-21T03:50:11.825 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:11.818+0000 7f703d4d4640 1 -- 172.21.15.39:0/2142356031 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f70240b66b0 msgr2=0x7f70240b8bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:50:11.826 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:11.818+0000 7f703d4d4640 1 --2- 172.21.15.39:0/2142356031 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f70240b66b0 0x7f70240b8bc0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f7030005f70 tx=0x7f7030045000 comp rx=0 tx=0).stop 2024-09-21T03:50:11.826 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:11.818+0000 7f703d4d4640 1 -- 172.21.15.39:0/2142356031 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7038102a60 msgr2=0x7f703819b290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:50:11.826 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:11.818+0000 7f703d4d4640 1 --2- 172.21.15.39:0/2142356031 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7038102a60 0x7f703819b290 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7f702800ea80 tx=0x7f702800ef40 comp rx=0 tx=0).stop 2024-09-21T03:50:11.826 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:11.818+0000 7f703d4d4640 1 -- 172.21.15.39:0/2142356031 shutdown_connections 2024-09-21T03:50:11.826 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:11.818+0000 7f703d4d4640 1 --2- 172.21.15.39:0/2142356031 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f70240b66b0 0x7f70240b8bc0 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:11.826 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:11.818+0000 7f703d4d4640 1 --2- 172.21.15.39:0/2142356031 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7038108cf0 0x7f703819bd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:11.826 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:11.818+0000 7f703d4d4640 1 --2- 172.21.15.39:0/2142356031 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f70381083a0 0x7f703819b7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:11.826 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:11.818+0000 7f703d4d4640 1 --2- 172.21.15.39:0/2142356031 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7038102a60 0x7f703819b290 unknown :-1 s=CLOSED pgs=750 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:11.826 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:11.818+0000 7f703d4d4640 1 -- 172.21.15.39:0/2142356031 >> 172.21.15.39:0/2142356031 conn(0x7f7038079730 msgr2=0x7f7038104460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:50:11.826 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:11.818+0000 7f703d4d4640 1 -- 172.21.15.39:0/2142356031 shutdown_connections 2024-09-21T03:50:11.826 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:11.818+0000 7f703d4d4640 1 -- 172.21.15.39:0/2142356031 wait complete. 2024-09-21T03:50:12.678 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 51116d5c-77ca-11ef-baf5-efdc52797490 -- ceph tell mds.1:1 damage ls 2024-09-21T03:50:12.937 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/51116d5c-77ca-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:50:13.200 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:50:12 smithi039 ceph-mon[31211]: pgmap v450: 97 pgs: 97 active+clean; 10 MiB data, 536 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 24 op/s 2024-09-21T03:50:13.218 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:50:12 smithi161 ceph-mon[34533]: pgmap v450: 97 pgs: 97 active+clean; 10 MiB data, 536 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 24 op/s 2024-09-21T03:50:13.224 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:50:12 smithi150 ceph-mon[34289]: pgmap v450: 97 pgs: 97 active+clean; 10 MiB data, 536 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 24 op/s 2024-09-21T03:50:13.426 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:13.422+0000 7f79172c9640 1 -- 172.21.15.39:0/4033418352 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f79101083a0 msgr2=0x7f79101087b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:50:13.426 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:13.422+0000 7f79172c9640 1 --2- 172.21.15.39:0/4033418352 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f79101083a0 0x7f79101087b0 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7f7904009900 tx=0x7f790402a410 comp rx=0 tx=0).stop 2024-09-21T03:50:13.426 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:13.423+0000 7f79172c9640 1 -- 172.21.15.39:0/4033418352 shutdown_connections 2024-09-21T03:50:13.426 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:13.423+0000 7f79172c9640 1 --2- 172.21.15.39:0/4033418352 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7910108cf0 0x7f791010f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:13.427 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:13.423+0000 7f79172c9640 1 --2- 172.21.15.39:0/4033418352 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f79101083a0 0x7f79101087b0 unknown :-1 s=CLOSED pgs=755 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:13.427 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:13.423+0000 7f79172c9640 1 --2- 172.21.15.39:0/4033418352 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7910102a60 0x7f7910102e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:13.427 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:13.423+0000 7f79172c9640 1 -- 172.21.15.39:0/4033418352 >> 172.21.15.39:0/4033418352 conn(0x7f7910079730 msgr2=0x7f7910079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:50:13.427 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:13.423+0000 7f79172c9640 1 -- 172.21.15.39:0/4033418352 shutdown_connections 2024-09-21T03:50:13.427 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:13.423+0000 7f79172c9640 1 -- 172.21.15.39:0/4033418352 wait complete. 2024-09-21T03:50:13.427 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:13.423+0000 7f79172c9640 1 Processor -- start 2024-09-21T03:50:13.427 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:13.423+0000 7f79172c9640 1 -- start start 2024-09-21T03:50:13.428 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:13.424+0000 7f79172c9640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7910102a60 0x7f791019c740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:50:13.428 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:13.424+0000 7f79172c9640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f79101083a0 0x7f791019cce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:50:13.429 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:13.424+0000 7f79172c9640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7910108cf0 0x7f79101a1f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:50:13.429 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:13.424+0000 7f79172c9640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f7910112960 con 0x7f7910108cf0 2024-09-21T03:50:13.429 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:13.424+0000 7f79172c9640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f79101127e0 con 0x7f79101083a0 2024-09-21T03:50:13.429 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:13.424+0000 7f79172c9640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f7910112ae0 con 0x7f7910102a60 2024-09-21T03:50:13.429 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:13.424+0000 7f7915866640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7910108cf0 0x7f79101a1f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:50:13.429 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:13.424+0000 7f7915866640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7910108cf0 0x7f79101a1f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:56750/0 (socket says 172.21.15.39:56750) 2024-09-21T03:50:13.429 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:13.424+0000 7f7914864640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f79101083a0 0x7f791019cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:50:13.429 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:13.424+0000 7f7915866640 1 -- 172.21.15.39:0/2884190966 learned_addr learned my addr 172.21.15.39:0/2884190966 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:50:13.429 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:13.424+0000 7f7915065640 1 --2- 172.21.15.39:0/2884190966 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7910102a60 0x7f791019c740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:50:13.429 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:13.424+0000 7f7915866640 1 -- 172.21.15.39:0/2884190966 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7910102a60 msgr2=0x7f791019c740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:50:13.429 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:13.424+0000 7f7915866640 1 --2- 172.21.15.39:0/2884190966 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7910102a60 0x7f791019c740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:13.429 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:13.424+0000 7f7915866640 1 -- 172.21.15.39:0/2884190966 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f79101083a0 msgr2=0x7f791019cce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:50:13.429 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:13.424+0000 7f7915866640 1 --2- 172.21.15.39:0/2884190966 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f79101083a0 0x7f791019cce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:13.430 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:13.424+0000 7f7915866640 1 -- 172.21.15.39:0/2884190966 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f79101a25a0 con 0x7f7910108cf0 2024-09-21T03:50:13.430 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:13.425+0000 7f7915866640 1 --2- 172.21.15.39:0/2884190966 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7910108cf0 0x7f79101a1f30 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7f790800cce0 tx=0x7f790800a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:50:13.430 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:13.425+0000 7f79027fc640 1 -- 172.21.15.39:0/2884190966 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f790800f450 con 0x7f7910108cf0 2024-09-21T03:50:13.430 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:13.425+0000 7f79027fc640 1 -- 172.21.15.39:0/2884190966 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f790800faa0 con 0x7f7910108cf0 2024-09-21T03:50:13.430 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:13.425+0000 7f79172c9640 1 -- 172.21.15.39:0/2884190966 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f79101a2890 con 0x7f7910108cf0 2024-09-21T03:50:13.430 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:13.425+0000 7f79027fc640 1 -- 172.21.15.39:0/2884190966 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7908018b70 con 0x7f7910108cf0 2024-09-21T03:50:13.430 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:13.426+0000 7f79172c9640 1 -- 172.21.15.39:0/2884190966 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f79101a2d50 con 0x7f7910108cf0 2024-09-21T03:50:13.431 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:13.429+0000 7f79027fc640 1 -- 172.21.15.39:0/2884190966 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 27) ==== 154643+0+0 (secure 0 0 0) 0x7f7908018d10 con 0x7f7910108cf0 2024-09-21T03:50:13.432 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:13.429+0000 7f791d83d740 1 Processor -- start 2024-09-21T03:50:13.432 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:13.429+0000 7f791d83d740 1 -- start start 2024-09-21T03:50:13.433 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:13.429+0000 7f791d83d740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562ca8795070 0x562ca8b3c6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:50:13.433 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:13.429+0000 7f791d83d740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562ca876c1b0 0x562ca8b3cbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:50:13.434 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:13.429+0000 7f791d83d740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562ca8774390 0x562ca8b3d120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:50:13.434 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:13.429+0000 7f791d83d740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x562ca8ab2ea0 con 0x562ca8795070 2024-09-21T03:50:13.434 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:13.429+0000 7f791d83d740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x562ca8aa22d0 con 0x562ca876c1b0 2024-09-21T03:50:13.434 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:13.429+0000 7f791d83d740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x562ca8aa60e0 con 0x562ca8774390 2024-09-21T03:50:13.434 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:13.430+0000 7f7914864640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562ca8795070 0x562ca8b3c6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:50:13.434 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:13.430+0000 7f79027fc640 1 --2- 172.21.15.39:0/2884190966 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f78fc0b66b0 0x7f78fc0b8bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:50:13.434 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:13.430+0000 7f7914864640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562ca8795070 0x562ca8b3c6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:56754/0 (socket says 172.21.15.39:56754) 2024-09-21T03:50:13.434 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:13.430+0000 7f7914864640 1 -- 172.21.15.39:0/677875002 learned_addr learned my addr 172.21.15.39:0/677875002 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:50:13.434 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:13.430+0000 7f7915866640 1 --2- 172.21.15.39:0/2884190966 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f78fc0b66b0 0x7f78fc0b8bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:50:13.434 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:13.430+0000 7f7915065640 1 --2- 172.21.15.39:0/677875002 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562ca876c1b0 0x562ca8b3cbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:50:13.434 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:13.430+0000 7f7914864640 1 --2- 172.21.15.39:0/677875002 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562ca8774390 0x562ca8b3d120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:50:13.434 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:13.430+0000 7f79027fc640 1 -- 172.21.15.39:0/2884190966 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8484+0+0 (secure 0 0 0) 0x7f79080e79f0 con 0x7f7910108cf0 2024-09-21T03:50:13.434 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:13.430+0000 7f7915065640 1 -- 172.21.15.39:0/677875002 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562ca8774390 msgr2=0x562ca8b3d120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:50:13.435 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:13.430+0000 7f7915065640 1 --2- 172.21.15.39:0/677875002 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562ca8774390 0x562ca8b3d120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:13.435 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:13.430+0000 7f7915065640 1 -- 172.21.15.39:0/677875002 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562ca8795070 msgr2=0x562ca8b3c6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:50:13.435 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:13.430+0000 7f7915065640 1 --2- 172.21.15.39:0/677875002 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562ca8795070 0x562ca8b3c6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:13.435 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:13.430+0000 7f7915065640 1 -- 172.21.15.39:0/677875002 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562ca8800650 con 0x562ca876c1b0 2024-09-21T03:50:13.435 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:13.430+0000 7f7914864640 1 --2- 172.21.15.39:0/677875002 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562ca8795070 0x562ca8b3c6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:50:13.435 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:13.430+0000 7f7915866640 1 --2- 172.21.15.39:0/2884190966 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f78fc0b66b0 0x7f78fc0b8bc0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f79080e5760 tx=0x7f7908071df0 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:50:13.435 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:13.431+0000 7f7915065640 1 --2- 172.21.15.39:0/677875002 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562ca876c1b0 0x562ca8b3cbe0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f790c009c00 tx=0x7f790c011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3caf527741e1be97 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:50:13.435 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:13.431+0000 7f78fb7fe640 1 -- 172.21.15.39:0/677875002 <== mon.2 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f790c004230 con 0x562ca876c1b0 2024-09-21T03:50:13.435 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:13.431+0000 7f78fb7fe640 1 -- 172.21.15.39:0/677875002 <== mon.2 v2:172.21.15.150:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f790c0043d0 con 0x562ca876c1b0 2024-09-21T03:50:13.435 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:13.431+0000 7f78fb7fe640 1 -- 172.21.15.39:0/677875002 <== mon.2 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f790c004c60 con 0x562ca876c1b0 2024-09-21T03:50:13.438 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:13.432+0000 7f791d83d740 1 -- 172.21.15.39:0/677875002 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562ca876c1b0 msgr2=0x562ca8b3cbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:50:13.438 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:13.432+0000 7f791d83d740 1 --2- 172.21.15.39:0/677875002 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562ca876c1b0 0x562ca8b3cbe0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f790c009c00 tx=0x7f790c011810 comp rx=0 tx=0).stop 2024-09-21T03:50:13.438 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:13.432+0000 7f791d83d740 1 -- 172.21.15.39:0/677875002 shutdown_connections 2024-09-21T03:50:13.438 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:13.432+0000 7f791d83d740 1 --2- 172.21.15.39:0/677875002 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562ca8774390 0x562ca8b3d120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:13.438 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:13.432+0000 7f791d83d740 1 --2- 172.21.15.39:0/677875002 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562ca876c1b0 0x562ca8b3cbe0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:13.438 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:13.432+0000 7f791d83d740 1 --2- 172.21.15.39:0/677875002 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562ca8795070 0x562ca8b3c6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:13.438 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:13.432+0000 7f791d83d740 1 -- 172.21.15.39:0/677875002 >> 172.21.15.39:0/677875002 conn(0x562ca8789460 msgr2=0x562ca8b3a590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:50:13.438 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:13.432+0000 7f791d83d740 1 -- 172.21.15.39:0/677875002 shutdown_connections 2024-09-21T03:50:13.439 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:13.432+0000 7f791d83d740 1 -- 172.21.15.39:0/677875002 wait complete. 2024-09-21T03:50:13.439 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:13.432+0000 7f791d83d740 1 -- start start 2024-09-21T03:50:13.439 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:13.432+0000 7f791d83d740 1 Processor -- start 2024-09-21T03:50:13.439 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:13.433+0000 7f791d83d740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562ca876c1b0 0x562ca8b47d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:50:13.439 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:13.433+0000 7f791d83d740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562ca8774390 0x562ca8b482c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:50:13.439 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:13.433+0000 7f791d83d740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562ca8795070 0x562ca8b48800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:50:13.439 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:13.433+0000 7f791d83d740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x562ca8903700 con 0x562ca876c1b0 2024-09-21T03:50:13.439 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:13.433+0000 7f791d83d740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x562ca8a3d160 con 0x562ca8774390 2024-09-21T03:50:13.439 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:13.433+0000 7f791d83d740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x562ca8b4ae80 con 0x562ca8795070 2024-09-21T03:50:13.439 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:13.433+0000 7f791d83d740 10 client.0 _ll_register_callbacks cb 0x562ca8a569e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:50:13.439 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:13.433+0000 7f7914864640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562ca876c1b0 0x562ca8b47d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:50:13.439 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:13.433+0000 7f7914864640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562ca876c1b0 0x562ca8b47d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:56762/0 (socket says 172.21.15.39:56762) 2024-09-21T03:50:13.439 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:13.433+0000 7f7914864640 1 -- 172.21.15.39:0/565731806 learned_addr learned my addr 172.21.15.39:0/565731806 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:50:13.439 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:13.433+0000 7f7914864640 1 --2- 172.21.15.39:0/565731806 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562ca8795070 0x562ca8b48800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:50:13.440 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:13.433+0000 7f7915065640 1 --2- 172.21.15.39:0/565731806 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562ca8774390 0x562ca8b482c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:50:13.440 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:13.434+0000 7f7914864640 1 -- 172.21.15.39:0/565731806 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562ca8795070 msgr2=0x562ca8b48800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:50:13.440 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:13.434+0000 7f7914864640 1 --2- 172.21.15.39:0/565731806 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562ca8795070 0x562ca8b48800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:13.440 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:13.434+0000 7f7914864640 1 -- 172.21.15.39:0/565731806 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562ca8774390 msgr2=0x562ca8b482c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:50:13.440 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:13.434+0000 7f7914864640 1 --2- 172.21.15.39:0/565731806 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562ca8774390 0x562ca8b482c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:13.440 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:13.434+0000 7f7914864640 1 -- 172.21.15.39:0/565731806 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562ca8a6ac90 con 0x562ca876c1b0 2024-09-21T03:50:13.440 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:13.434+0000 7f7915065640 1 --2- 172.21.15.39:0/565731806 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562ca8774390 0x562ca8b482c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:50:13.440 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:13.434+0000 7f7914864640 1 --2- 172.21.15.39:0/565731806 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562ca876c1b0 0x562ca8b47d80 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7f7904009230 tx=0x7f7904031400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bab5049319ad99d8 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:50:13.440 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:13.434+0000 7f78f8ff9640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-09-21T03:50:13.440 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:13.434+0000 7f78f8ff9640 1 -- 172.21.15.39:0/565731806 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7904035040 con 0x562ca876c1b0 2024-09-21T03:50:13.440 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:13.434+0000 7f78f8ff9640 1 -- 172.21.15.39:0/565731806 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8484+0+0 (secure 0 0 0) 0x7f79040053a0 con 0x562ca876c1b0 2024-09-21T03:50:13.440 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:13.434+0000 7f791d83d740 1 -- 172.21.15.39:0/565731806 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f790c0043d0 con 0x562ca876c1b0 2024-09-21T03:50:13.440 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:13.434+0000 7f78f8ff9640 1 -- 172.21.15.39:0/565731806 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f79040055a0 con 0x562ca876c1b0 2024-09-21T03:50:13.440 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:13.434+0000 7f791d83d740 10 client.16524 fetch_fsmap learned FSMap version 29 2024-09-21T03:50:13.441 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:13.434+0000 7f791d83d740 1 -- 172.21.15.39:0/565731806 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x562ca8b56b00 con 0x562ca876c1b0 2024-09-21T03:50:13.441 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:13.435+0000 7f78f8ff9640 1 -- 172.21.15.39:0/565731806 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f790c0043d0 con 0x562ca876c1b0 2024-09-21T03:50:13.441 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:13.435+0000 7f78f8ff9640 1 -- 172.21.15.39:0/565731806 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f790404a030 con 0x562ca876c1b0 2024-09-21T03:50:13.441 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:13.435+0000 7f791d83d740 10 client.16524 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T03:50:13.441 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:13.435+0000 7f791d83d740 10 client.16524 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-21T03:50:13.441 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:13.435+0000 7f791d83d740 20 client.16524 populate_metadata read hostname 'smithi039' 2024-09-21T03:50:13.441 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:13.435+0000 7f791d83d740 1 --2- 172.21.15.39:0/565731806 >> [v2:172.21.15.161:6838/2190107270,v1:172.21.15.161:6839/2190107270] conn(0x562ca87539f0 0x562ca8b59fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:50:13.441 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:13.435+0000 7f791d83d740 4 client.16524 mds_command: new command op to 24667 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:50:13.441 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:13.435+0000 7f791d83d740 1 -- 172.21.15.39:0/565731806 --> [v2:172.21.15.161:6838/2190107270,v1:172.21.15.161:6839/2190107270] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562ca8ab4df0 con 0x562ca87539f0 2024-09-21T03:50:13.442 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:13.436+0000 7f7915065640 1 --2- 172.21.15.39:0/565731806 >> [v2:172.21.15.161:6838/2190107270,v1:172.21.15.161:6839/2190107270] conn(0x562ca87539f0 0x562ca8b59fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:50:13.442 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:13.438+0000 7f78f8ff9640 10 client.16524 ms_handle_connect on v2:172.21.15.161:6838/2190107270 2024-09-21T03:50:13.442 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:13.438+0000 7f7915065640 1 --2- 172.21.15.39:0/565731806 >> [v2:172.21.15.161:6838/2190107270,v1:172.21.15.161:6839/2190107270] conn(0x562ca87539f0 0x562ca8b59fd0 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=8e03f09511422749 server_cookie=c8f9b610e05cb28e in_seq=0 out_seq=0 2024-09-21T03:50:13.443 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:13.439+0000 7f78f8ff9640 1 -- 172.21.15.39:0/565731806 <== mds.1 v2:172.21.15.161:6838/2190107270 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x562ca8ab4df0 con 0x562ca87539f0 2024-09-21T03:50:13.443 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:13.439+0000 7f78f8ff9640 10 client.16524 handle_command_reply: tid=0 2024-09-21T03:50:13.443 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:13.439+0000 7f791d83d740 1 client.16524 shutdown 2024-09-21T03:50:13.443 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:13.439+0000 7f791d83d740 1 -- 172.21.15.39:0/565731806 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562ca876c1b0 msgr2=0x562ca8b47d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:50:13.443 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:13.439+0000 7f791d83d740 1 --2- 172.21.15.39:0/565731806 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562ca876c1b0 0x562ca8b47d80 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7f7904009230 tx=0x7f7904031400 comp rx=0 tx=0).stop 2024-09-21T03:50:13.443 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:13.439+0000 7f791d83d740 1 -- 172.21.15.39:0/565731806 shutdown_connections 2024-09-21T03:50:13.443 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:13.439+0000 7f791d83d740 1 --2- 172.21.15.39:0/565731806 >> [v2:172.21.15.161:6838/2190107270,v1:172.21.15.161:6839/2190107270] conn(0x562ca87539f0 0x562ca8b59fd0 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:13.443 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:13.439+0000 7f791d83d740 1 --2- 172.21.15.39:0/565731806 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562ca8795070 0x562ca8b48800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:13.443 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:13.439+0000 7f791d83d740 1 --2- 172.21.15.39:0/565731806 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562ca8774390 0x562ca8b482c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:13.443 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:13.439+0000 7f78f8ff9640 0 client.16524 ms_handle_reset on v2:172.21.15.161:6838/2190107270 2024-09-21T03:50:13.443 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:13.439+0000 7f791d83d740 1 --2- 172.21.15.39:0/565731806 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562ca876c1b0 0x562ca8b47d80 unknown :-1 s=CLOSED pgs=757 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:13.443 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:13.439+0000 7f791d83d740 1 -- 172.21.15.39:0/565731806 >> 172.21.15.39:0/565731806 conn(0x562ca8789460 msgr2=0x562ca8b3f880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:50:13.443 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:13.439+0000 7f791d83d740 1 -- 172.21.15.39:0/565731806 shutdown_connections 2024-09-21T03:50:13.444 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:13.439+0000 7f791d83d740 1 -- 172.21.15.39:0/565731806 wait complete. 2024-09-21T03:50:13.444 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:13.439+0000 7f791d83d740 20 client.16524 trim_cache size 0 max 16384 2024-09-21T03:50:13.467 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:13.464+0000 7f791d83d740 1 Processor -- start 2024-09-21T03:50:13.467 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:13.464+0000 7f791d83d740 1 -- start start 2024-09-21T03:50:13.468 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:13.465+0000 7f791d83d740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562ca876c1b0 0x562ca8b3e090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:50:13.468 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:13.465+0000 7f791d83d740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562ca8774390 0x562ca8b499b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:50:13.468 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:13.465+0000 7f791d83d740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562ca8795070 0x562ca8b39bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:50:13.468 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:13.465+0000 7f791d83d740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f79101b1800 con 0x562ca8774390 2024-09-21T03:50:13.468 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:13.465+0000 7f791d83d740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f79101b3340 con 0x562ca876c1b0 2024-09-21T03:50:13.468 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:13.465+0000 7f791d83d740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f79101b1ed0 con 0x562ca8795070 2024-09-21T03:50:13.468 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:13.465+0000 7f7915065640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562ca8774390 0x562ca8b499b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:50:13.468 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:13.465+0000 7f7914864640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562ca876c1b0 0x562ca8b3e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:50:13.469 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:13.465+0000 7f7915065640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562ca8774390 0x562ca8b499b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:56778/0 (socket says 172.21.15.39:56778) 2024-09-21T03:50:13.469 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:13.465+0000 7f7915065640 1 -- 172.21.15.39:0/1654811011 learned_addr learned my addr 172.21.15.39:0/1654811011 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:50:13.469 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:13.465+0000 7f7914864640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562ca876c1b0 0x562ca8b3e090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.39:50640/0 (socket says 172.21.15.39:50640) 2024-09-21T03:50:13.469 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:13.465+0000 7f7914864640 1 --2- 172.21.15.39:0/1654811011 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562ca8795070 0x562ca8b39bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:50:13.469 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:13.466+0000 7f7914864640 1 -- 172.21.15.39:0/1654811011 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562ca8795070 msgr2=0x562ca8b39bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:50:13.470 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:13.466+0000 7f7914864640 1 --2- 172.21.15.39:0/1654811011 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562ca8795070 0x562ca8b39bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:13.470 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:13.466+0000 7f7914864640 1 -- 172.21.15.39:0/1654811011 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562ca8774390 msgr2=0x562ca8b499b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:50:13.470 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:13.466+0000 7f7914864640 1 --2- 172.21.15.39:0/1654811011 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562ca8774390 0x562ca8b499b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:13.470 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:13.466+0000 7f7914864640 1 -- 172.21.15.39:0/1654811011 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562ca8b3e990 con 0x562ca876c1b0 2024-09-21T03:50:13.470 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:13.466+0000 7f7915065640 1 --2- 172.21.15.39:0/1654811011 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562ca8774390 0x562ca8b499b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:50:13.470 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:13.466+0000 7f7914864640 1 --2- 172.21.15.39:0/1654811011 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562ca876c1b0 0x562ca8b3e090 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f7904004e50 tx=0x7f7904009260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=253ef679fc56e75 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:50:13.470 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:13.466+0000 7f78faffd640 1 -- 172.21.15.39:0/1654811011 <== mon.2 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7904035070 con 0x562ca876c1b0 2024-09-21T03:50:13.470 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:13.466+0000 7f78faffd640 1 -- 172.21.15.39:0/1654811011 <== mon.2 v2:172.21.15.150:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f79040053a0 con 0x562ca876c1b0 2024-09-21T03:50:13.470 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:13.466+0000 7f78faffd640 1 -- 172.21.15.39:0/1654811011 <== mon.2 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7904004ae0 con 0x562ca876c1b0 2024-09-21T03:50:13.471 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:13.468+0000 7f791d83d740 1 -- 172.21.15.39:0/1654811011 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562ca876c1b0 msgr2=0x562ca8b3e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:50:13.471 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:13.468+0000 7f791d83d740 1 --2- 172.21.15.39:0/1654811011 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562ca876c1b0 0x562ca8b3e090 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f7904004e50 tx=0x7f7904009260 comp rx=0 tx=0).stop 2024-09-21T03:50:13.471 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:13.468+0000 7f791d83d740 1 -- 172.21.15.39:0/1654811011 shutdown_connections 2024-09-21T03:50:13.471 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:13.468+0000 7f791d83d740 1 --2- 172.21.15.39:0/1654811011 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562ca8795070 0x562ca8b39bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:13.471 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:13.468+0000 7f791d83d740 1 --2- 172.21.15.39:0/1654811011 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562ca8774390 0x562ca8b499b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:13.471 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:13.468+0000 7f791d83d740 1 --2- 172.21.15.39:0/1654811011 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562ca876c1b0 0x562ca8b3e090 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:13.471 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:13.468+0000 7f791d83d740 1 -- 172.21.15.39:0/1654811011 >> 172.21.15.39:0/1654811011 conn(0x562ca8789460 msgr2=0x562ca8b492a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:50:13.471 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:13.468+0000 7f791d83d740 1 -- 172.21.15.39:0/1654811011 shutdown_connections 2024-09-21T03:50:13.471 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:13.468+0000 7f791d83d740 1 -- 172.21.15.39:0/1654811011 wait complete. 2024-09-21T03:50:13.472 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:13.468+0000 7f791d83d740 1 -- start start 2024-09-21T03:50:13.472 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:13.468+0000 7f791d83d740 1 Processor -- start 2024-09-21T03:50:13.472 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:13.469+0000 7f791d83d740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562ca876c1b0 0x562ca8b50a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:50:13.472 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:13.469+0000 7f791d83d740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562ca8774390 0x562ca8b4bf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:50:13.473 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:13.469+0000 7f791d83d740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562ca8795070 0x562ca8b4c630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:50:13.473 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:13.469+0000 7f791d83d740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f79101b2c70 con 0x562ca8795070 2024-09-21T03:50:13.473 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:13.469+0000 7f791d83d740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f79101b25a0 con 0x562ca876c1b0 2024-09-21T03:50:13.473 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:13.469+0000 7f791d83d740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f79101af7e0 con 0x562ca8774390 2024-09-21T03:50:13.473 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:13.469+0000 7f791d83d740 10 client.0 _ll_register_callbacks cb 0x7f79040314a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:50:13.473 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:13.469+0000 7f7914864640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562ca876c1b0 0x562ca8b50a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:50:13.473 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:13.469+0000 7f7914864640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562ca876c1b0 0x562ca8b50a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.39:50646/0 (socket says 172.21.15.39:50646) 2024-09-21T03:50:13.473 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:13.469+0000 7f7914864640 1 -- 172.21.15.39:0/3923205945 learned_addr learned my addr 172.21.15.39:0/3923205945 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:50:13.473 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:13.469+0000 7f7915065640 1 --2- 172.21.15.39:0/3923205945 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562ca8774390 0x562ca8b4bf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:50:13.473 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:13.470+0000 7f7914864640 1 --2- 172.21.15.39:0/3923205945 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562ca8795070 0x562ca8b4c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:50:13.476 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:13.470+0000 7f7915065640 1 -- 172.21.15.39:0/3923205945 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562ca876c1b0 msgr2=0x562ca8b50a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:50:13.476 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:13.470+0000 7f7915065640 1 --2- 172.21.15.39:0/3923205945 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562ca876c1b0 0x562ca8b50a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:13.476 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:13.470+0000 7f7915065640 1 -- 172.21.15.39:0/3923205945 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562ca8795070 msgr2=0x562ca8b4c630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:50:13.476 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:13.470+0000 7f7915065640 1 --2- 172.21.15.39:0/3923205945 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562ca8795070 0x562ca8b4c630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:13.476 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:13.470+0000 7f7915065640 1 -- 172.21.15.39:0/3923205945 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562ca8bc3990 con 0x562ca8774390 2024-09-21T03:50:13.476 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:13.470+0000 7f7914864640 1 --2- 172.21.15.39:0/3923205945 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562ca876c1b0 0x562ca8b50a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:50:13.476 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:13.470+0000 7f7915065640 1 --2- 172.21.15.39:0/3923205945 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562ca8774390 0x562ca8b4bf40 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f790c009910 tx=0x7f790c009620 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a06decf6200b12dc server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:50:13.476 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:13.470+0000 7f78f8ff9640 10 client.0 ms_handle_connect on v2:172.21.15.161:3300/0 2024-09-21T03:50:13.476 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:13.470+0000 7f78f8ff9640 1 -- 172.21.15.39:0/3923205945 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f790c026650 con 0x562ca8774390 2024-09-21T03:50:13.477 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:13.471+0000 7f78f8ff9640 1 -- 172.21.15.39:0/3923205945 <== mon.1 v2:172.21.15.161:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8484+0+0 (secure 0 0 0) 0x7f790c01e070 con 0x562ca8774390 2024-09-21T03:50:13.477 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:13.471+0000 7f791d83d740 1 -- 172.21.15.39:0/3923205945 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f79040053a0 con 0x562ca8774390 2024-09-21T03:50:13.477 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:13.471+0000 7f78f8ff9640 1 -- 172.21.15.39:0/3923205945 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f790c03b050 con 0x562ca8774390 2024-09-21T03:50:13.477 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:13.471+0000 7f791d83d740 10 client.26050 fetch_fsmap learned FSMap version 29 2024-09-21T03:50:13.477 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:13.471+0000 7f791d83d740 1 -- 172.21.15.39:0/3923205945 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x562ca8bc4a80 con 0x562ca8774390 2024-09-21T03:50:13.477 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:13.471+0000 7f78f8ff9640 1 -- 172.21.15.39:0/3923205945 <== mon.1 v2:172.21.15.161:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f790c030020 con 0x562ca8774390 2024-09-21T03:50:13.477 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:13.471+0000 7f78f8ff9640 1 -- 172.21.15.39:0/3923205945 <== mon.1 v2:172.21.15.161:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f790c024040 con 0x562ca8774390 2024-09-21T03:50:13.477 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:13.471+0000 7f791d83d740 10 client.26050 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T03:50:13.477 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:13.471+0000 7f791d83d740 10 client.26050 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-21T03:50:13.477 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:13.471+0000 7f791d83d740 20 client.26050 populate_metadata read hostname 'smithi039' 2024-09-21T03:50:13.477 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:13.471+0000 7f791d83d740 1 --2- 172.21.15.39:0/3923205945 >> [v2:172.21.15.161:6838/2190107270,v1:172.21.15.161:6839/2190107270] conn(0x562ca8728da0 0x562ca8bc7100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:50:13.477 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:13.471+0000 7f791d83d740 4 client.26050 mds_command: new command op to 24667 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T03:50:13.477 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:13.471+0000 7f791d83d740 1 -- 172.21.15.39:0/3923205945 --> [v2:172.21.15.161:6838/2190107270,v1:172.21.15.161:6839/2190107270] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7f79040053a0 con 0x562ca8728da0 2024-09-21T03:50:13.477 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:13.472+0000 7f7914864640 1 --2- 172.21.15.39:0/3923205945 >> [v2:172.21.15.161:6838/2190107270,v1:172.21.15.161:6839/2190107270] conn(0x562ca8728da0 0x562ca8bc7100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:50:13.478 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:13.472+0000 7f78f8ff9640 10 client.26050 ms_handle_connect on v2:172.21.15.161:6838/2190107270 2024-09-21T03:50:13.478 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:13.472+0000 7f7914864640 1 --2- 172.21.15.39:0/3923205945 >> [v2:172.21.15.161:6838/2190107270,v1:172.21.15.161:6839/2190107270] conn(0x562ca8728da0 0x562ca8bc7100 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=8864158423f31b9a server_cookie=2d40e99fcc5119f8 in_seq=0 out_seq=0 2024-09-21T03:50:13.478 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:13.472+0000 7f78f8ff9640 1 -- 172.21.15.39:0/3923205945 <== mds.1 v2:172.21.15.161:6838/2190107270 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7f79040053a0 con 0x562ca8728da0 2024-09-21T03:50:13.478 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:13.472+0000 7f78f8ff9640 10 client.26050 handle_command_reply: tid=0 2024-09-21T03:50:13.478 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:13.472+0000 7f791d83d740 1 client.26050 shutdown 2024-09-21T03:50:13.478 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:13.473+0000 7f791d83d740 1 -- 172.21.15.39:0/3923205945 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562ca8774390 msgr2=0x562ca8b4bf40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:50:13.478 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:13.473+0000 7f791d83d740 1 --2- 172.21.15.39:0/3923205945 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562ca8774390 0x562ca8b4bf40 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f790c009910 tx=0x7f790c009620 comp rx=0 tx=0).stop 2024-09-21T03:50:13.478 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:13.473+0000 7f791d83d740 1 -- 172.21.15.39:0/3923205945 shutdown_connections 2024-09-21T03:50:13.478 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:13.473+0000 7f791d83d740 1 --2- 172.21.15.39:0/3923205945 >> [v2:172.21.15.161:6838/2190107270,v1:172.21.15.161:6839/2190107270] conn(0x562ca8728da0 0x562ca8bc7100 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:13.478 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:13.473+0000 7f791d83d740 1 --2- 172.21.15.39:0/3923205945 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562ca8795070 0x562ca8b4c630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:13.478 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:13.473+0000 7f791d83d740 1 --2- 172.21.15.39:0/3923205945 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562ca8774390 0x562ca8b4bf40 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:13.478 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:13.473+0000 7f791d83d740 1 --2- 172.21.15.39:0/3923205945 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562ca876c1b0 0x562ca8b50a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:13.478 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:13.473+0000 7f791d83d740 1 -- 172.21.15.39:0/3923205945 >> 172.21.15.39:0/3923205945 conn(0x562ca8789460 msgr2=0x562ca8b3df50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:50:13.478 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:13.473+0000 7f78f8ff9640 0 client.26050 ms_handle_reset on v2:172.21.15.161:6838/2190107270 2024-09-21T03:50:13.479 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:13.473+0000 7f791d83d740 1 -- 172.21.15.39:0/3923205945 shutdown_connections 2024-09-21T03:50:13.479 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:13.473+0000 7f791d83d740 1 -- 172.21.15.39:0/3923205945 wait complete. 2024-09-21T03:50:13.479 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:13.473+0000 7f791d83d740 20 client.26050 trim_cache size 0 max 16384 2024-09-21T03:50:13.479 INFO:teuthology.orchestra.run.smithi039.stdout:[] 2024-09-21T03:50:13.480 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:13.474+0000 7f79172c9640 1 -- 172.21.15.39:0/2884190966 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f78fc0b66b0 msgr2=0x7f78fc0b8bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:50:13.480 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:13.474+0000 7f79172c9640 1 --2- 172.21.15.39:0/2884190966 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f78fc0b66b0 0x7f78fc0b8bc0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f79080e5760 tx=0x7f7908071df0 comp rx=0 tx=0).stop 2024-09-21T03:50:13.480 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:13.474+0000 7f79172c9640 1 -- 172.21.15.39:0/2884190966 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7910108cf0 msgr2=0x7f79101a1f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:50:13.480 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:13.474+0000 7f79172c9640 1 --2- 172.21.15.39:0/2884190966 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7910108cf0 0x7f79101a1f30 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7f790800cce0 tx=0x7f790800a620 comp rx=0 tx=0).stop 2024-09-21T03:50:13.480 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:13.474+0000 7f79172c9640 1 -- 172.21.15.39:0/2884190966 shutdown_connections 2024-09-21T03:50:13.480 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:13.474+0000 7f79172c9640 1 --2- 172.21.15.39:0/2884190966 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f78fc0b66b0 0x7f78fc0b8bc0 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:13.480 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:13.474+0000 7f79172c9640 1 --2- 172.21.15.39:0/2884190966 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7910108cf0 0x7f79101a1f30 unknown :-1 s=CLOSED pgs=756 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:13.480 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:13.474+0000 7f79172c9640 1 --2- 172.21.15.39:0/2884190966 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f79101083a0 0x7f791019cce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:13.480 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:13.474+0000 7f79172c9640 1 --2- 172.21.15.39:0/2884190966 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7910102a60 0x7f791019c740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:13.480 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:13.474+0000 7f79172c9640 1 -- 172.21.15.39:0/2884190966 >> 172.21.15.39:0/2884190966 conn(0x7f7910079730 msgr2=0x7f79101071a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:50:13.480 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:13.474+0000 7f79172c9640 1 -- 172.21.15.39:0/2884190966 shutdown_connections 2024-09-21T03:50:13.481 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:13.474+0000 7f79172c9640 1 -- 172.21.15.39:0/2884190966 wait complete. 2024-09-21T03:50:14.313 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 51116d5c-77ca-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 damage ls 2024-09-21T03:50:14.582 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/51116d5c-77ca-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:50:15.109 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:15.104+0000 7fe18f97b640 1 -- 172.21.15.39:0/1224623304 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe188108d00 msgr2=0x7fe18810f530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:50:15.110 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:15.104+0000 7fe18f97b640 1 --2- 172.21.15.39:0/1224623304 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe188108d00 0x7fe18810f530 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7fe180009ca0 tx=0x7fe18002a8c0 comp rx=0 tx=0).stop 2024-09-21T03:50:15.110 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:15.104+0000 7fe18f97b640 1 -- 172.21.15.39:0/1224623304 shutdown_connections 2024-09-21T03:50:15.110 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:15.104+0000 7fe18f97b640 1 --2- 172.21.15.39:0/1224623304 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe188108d00 0x7fe18810f530 unknown :-1 s=CLOSED pgs=758 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:15.110 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:15.104+0000 7fe18f97b640 1 --2- 172.21.15.39:0/1224623304 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe1881083b0 0x7fe1881087c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:15.110 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:15.104+0000 7fe18f97b640 1 --2- 172.21.15.39:0/1224623304 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe188102a70 0x7fe188102e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:15.110 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:15.104+0000 7fe18f97b640 1 -- 172.21.15.39:0/1224623304 >> 172.21.15.39:0/1224623304 conn(0x7fe188079710 msgr2=0x7fe188079b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:50:15.110 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:15.105+0000 7fe18f97b640 1 -- 172.21.15.39:0/1224623304 shutdown_connections 2024-09-21T03:50:15.110 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:15.105+0000 7fe18f97b640 1 -- 172.21.15.39:0/1224623304 wait complete. 2024-09-21T03:50:15.110 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:15.105+0000 7fe18f97b640 1 Processor -- start 2024-09-21T03:50:15.110 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:15.106+0000 7fe18f97b640 1 -- start start 2024-09-21T03:50:15.110 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:15.106+0000 7fe18f97b640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe188102a70 0x7fe18819b370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:50:15.111 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:15.106+0000 7fe18f97b640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe1881083b0 0x7fe18819b8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:50:15.111 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:15.106+0000 7fe18f97b640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe188108d00 0x7fe18819bdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:50:15.111 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:15.106+0000 7fe18f97b640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fe1881129c0 con 0x7fe1881083b0 2024-09-21T03:50:15.111 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:15.106+0000 7fe18f97b640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fe188112840 con 0x7fe188102a70 2024-09-21T03:50:15.111 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:15.106+0000 7fe18f97b640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fe188112b40 con 0x7fe188108d00 2024-09-21T03:50:15.111 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:15.106+0000 7fe18cf16640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe1881083b0 0x7fe18819b8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:50:15.111 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:15.106+0000 7fe18cf16640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe1881083b0 0x7fe18819b8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:56802/0 (socket says 172.21.15.39:56802) 2024-09-21T03:50:15.111 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:15.106+0000 7fe18cf16640 1 -- 172.21.15.39:0/1926784328 learned_addr learned my addr 172.21.15.39:0/1926784328 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:50:15.111 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:15.106+0000 7fe18d717640 1 --2- 172.21.15.39:0/1926784328 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe188102a70 0x7fe18819b370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:50:15.111 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:15.106+0000 7fe18df18640 1 --2- 172.21.15.39:0/1926784328 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe188108d00 0x7fe18819bdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:50:15.111 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:15.106+0000 7fe18cf16640 1 -- 172.21.15.39:0/1926784328 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe188108d00 msgr2=0x7fe18819bdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:50:15.111 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:15.106+0000 7fe18cf16640 1 --2- 172.21.15.39:0/1926784328 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe188108d00 0x7fe18819bdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:15.111 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:15.106+0000 7fe18cf16640 1 -- 172.21.15.39:0/1926784328 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe188102a70 msgr2=0x7fe18819b370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:50:15.111 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:15.106+0000 7fe18cf16640 1 --2- 172.21.15.39:0/1926784328 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe188102a70 0x7fe18819b370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:15.111 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:15.106+0000 7fe18cf16640 1 -- 172.21.15.39:0/1926784328 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe1881a6600 con 0x7fe1881083b0 2024-09-21T03:50:15.112 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:15.107+0000 7fe18cf16640 1 --2- 172.21.15.39:0/1926784328 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe1881083b0 0x7fe18819b8b0 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7fe17800e9c0 tx=0x7fe17800ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:50:15.112 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:15.107+0000 7fe17e7fc640 1 -- 172.21.15.39:0/1926784328 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe17800ccc0 con 0x7fe1881083b0 2024-09-21T03:50:15.112 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:15.107+0000 7fe18f97b640 1 -- 172.21.15.39:0/1926784328 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe1881a68f0 con 0x7fe1881083b0 2024-09-21T03:50:15.112 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:15.107+0000 7fe17e7fc640 1 -- 172.21.15.39:0/1926784328 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe1780054d0 con 0x7fe1881083b0 2024-09-21T03:50:15.112 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:15.107+0000 7fe17e7fc640 1 -- 172.21.15.39:0/1926784328 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe178010430 con 0x7fe1881083b0 2024-09-21T03:50:15.112 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:15.107+0000 7fe18f97b640 1 -- 172.21.15.39:0/1926784328 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe1881a6de0 con 0x7fe1881083b0 2024-09-21T03:50:15.112 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:15.110+0000 7fe17e7fc640 1 -- 172.21.15.39:0/1926784328 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 27) ==== 154643+0+0 (secure 0 0 0) 0x7fe178005020 con 0x7fe1881083b0 2024-09-21T03:50:15.113 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:50:14 smithi039 ceph-mon[31211]: pgmap v451: 97 pgs: 97 active+clean; 10 MiB data, 536 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 25 op/s 2024-09-21T03:50:15.114 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:15.110+0000 7fe195eef740 1 Processor -- start 2024-09-21T03:50:15.114 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:15.110+0000 7fe195eef740 1 -- start start 2024-09-21T03:50:15.114 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:15.110+0000 7fe195eef740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b429481070 0x55b4298286a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:50:15.115 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:15.110+0000 7fe195eef740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b4294581b0 0x55b429828be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:50:15.115 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:15.110+0000 7fe195eef740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b429460390 0x55b429829120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:50:15.115 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:15.110+0000 7fe195eef740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x55b42979eea0 con 0x55b4294581b0 2024-09-21T03:50:15.115 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:15.110+0000 7fe195eef740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x55b42978e2d0 con 0x55b429481070 2024-09-21T03:50:15.115 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:15.110+0000 7fe195eef740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55b4297920e0 con 0x55b429460390 2024-09-21T03:50:15.115 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:15.111+0000 7fe18d717640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b4294581b0 0x55b429828be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:50:15.115 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:15.111+0000 7fe18d717640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b4294581b0 0x55b429828be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:56818/0 (socket says 172.21.15.39:56818) 2024-09-21T03:50:15.115 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:15.111+0000 7fe18d717640 1 -- 172.21.15.39:0/2073762912 learned_addr learned my addr 172.21.15.39:0/2073762912 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:50:15.115 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:15.111+0000 7fe18df18640 1 --2- 172.21.15.39:0/2073762912 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b429481070 0x55b4298286a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:50:15.115 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:15.111+0000 7fe18cf16640 1 --2- 172.21.15.39:0/2073762912 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b429460390 0x55b429829120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:50:15.115 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:15.111+0000 7fe18d717640 1 -- 172.21.15.39:0/2073762912 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b429460390 msgr2=0x55b429829120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:50:15.115 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:15.111+0000 7fe18d717640 1 --2- 172.21.15.39:0/2073762912 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b429460390 0x55b429829120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:15.115 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:15.111+0000 7fe18d717640 1 -- 172.21.15.39:0/2073762912 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b429481070 msgr2=0x55b4298286a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:50:15.115 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:15.111+0000 7fe18d717640 1 --2- 172.21.15.39:0/2073762912 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b429481070 0x55b4298286a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:15.116 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:15.111+0000 7fe17e7fc640 1 --2- 172.21.15.39:0/1926784328 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7fe1740b66b0 0x7fe1740b8bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:50:15.116 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:15.111+0000 7fe18d717640 1 -- 172.21.15.39:0/2073762912 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b4294ec650 con 0x55b4294581b0 2024-09-21T03:50:15.116 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:15.111+0000 7fe18df18640 1 --2- 172.21.15.39:0/2073762912 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b429481070 0x55b4298286a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:50:15.116 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:15.111+0000 7fe18df18640 1 --2- 172.21.15.39:0/1926784328 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7fe1740b66b0 0x7fe1740b8bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:50:15.116 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:15.111+0000 7fe17e7fc640 1 -- 172.21.15.39:0/1926784328 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8484+0+0 (secure 0 0 0) 0x7fe1780e7980 con 0x7fe1881083b0 2024-09-21T03:50:15.116 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:15.111+0000 7fe18d717640 1 --2- 172.21.15.39:0/2073762912 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b4294581b0 0x55b429828be0 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7fe184009c90 tx=0x7fe184011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=96cf8537c92cdf5 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:50:15.116 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:15.112+0000 7fe1737fe640 1 -- 172.21.15.39:0/2073762912 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe184005220 con 0x55b4294581b0 2024-09-21T03:50:15.116 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:15.112+0000 7fe1737fe640 1 -- 172.21.15.39:0/2073762912 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe184004c50 con 0x55b4294581b0 2024-09-21T03:50:15.116 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:15.112+0000 7fe1737fe640 1 -- 172.21.15.39:0/2073762912 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe184019690 con 0x55b4294581b0 2024-09-21T03:50:15.119 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:15.113+0000 7fe195eef740 1 -- 172.21.15.39:0/2073762912 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b4294581b0 msgr2=0x55b429828be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:50:15.119 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:15.113+0000 7fe195eef740 1 --2- 172.21.15.39:0/2073762912 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b4294581b0 0x55b429828be0 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7fe184009c90 tx=0x7fe184011810 comp rx=0 tx=0).stop 2024-09-21T03:50:15.119 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:15.113+0000 7fe18df18640 1 --2- 172.21.15.39:0/1926784328 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7fe1740b66b0 0x7fe1740b8bc0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fe180009a60 tx=0x7fe18002f470 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:50:15.119 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:15.113+0000 7fe195eef740 1 -- 172.21.15.39:0/2073762912 shutdown_connections 2024-09-21T03:50:15.119 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:15.113+0000 7fe195eef740 1 --2- 172.21.15.39:0/2073762912 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b429460390 0x55b429829120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:15.119 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:15.113+0000 7fe195eef740 1 --2- 172.21.15.39:0/2073762912 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b4294581b0 0x55b429828be0 unknown :-1 s=CLOSED pgs=760 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:15.119 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:15.113+0000 7fe195eef740 1 --2- 172.21.15.39:0/2073762912 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b429481070 0x55b4298286a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:15.119 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:15.113+0000 7fe195eef740 1 -- 172.21.15.39:0/2073762912 >> 172.21.15.39:0/2073762912 conn(0x55b429475460 msgr2=0x55b429826590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:50:15.119 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:15.113+0000 7fe195eef740 1 -- 172.21.15.39:0/2073762912 shutdown_connections 2024-09-21T03:50:15.119 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:15.113+0000 7fe195eef740 1 -- 172.21.15.39:0/2073762912 wait complete. 2024-09-21T03:50:15.119 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:15.113+0000 7fe195eef740 1 -- start start 2024-09-21T03:50:15.119 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:15.113+0000 7fe195eef740 1 Processor -- start 2024-09-21T03:50:15.120 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:15.114+0000 7fe195eef740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b4294581b0 0x55b429833d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:50:15.120 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:15.114+0000 7fe195eef740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b429460390 0x55b429834240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:50:15.120 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:15.114+0000 7fe195eef740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b429481070 0x55b429834780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:50:15.120 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:15.114+0000 7fe195eef740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x55b4295ef700 con 0x55b429460390 2024-09-21T03:50:15.120 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:15.114+0000 7fe195eef740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x55b429729160 con 0x55b429481070 2024-09-21T03:50:15.120 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:15.114+0000 7fe195eef740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55b429836e80 con 0x55b4294581b0 2024-09-21T03:50:15.120 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:15.114+0000 7fe195eef740 10 client.0 _ll_register_callbacks cb 0x55b4297429e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:50:15.120 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:15.114+0000 7fe18cf16640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b429460390 0x55b429834240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:50:15.120 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:15.114+0000 7fe18cf16640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b429460390 0x55b429834240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:56820/0 (socket says 172.21.15.39:56820) 2024-09-21T03:50:15.120 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:15.114+0000 7fe18cf16640 1 -- 172.21.15.39:0/4270557027 learned_addr learned my addr 172.21.15.39:0/4270557027 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:50:15.120 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:15.114+0000 7fe18df18640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b429481070 0x55b429834780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:50:15.120 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:15.114+0000 7fe18d717640 1 --2- 172.21.15.39:0/4270557027 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b4294581b0 0x55b429833d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:50:15.120 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:15.114+0000 7fe18df18640 1 -- 172.21.15.39:0/4270557027 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b4294581b0 msgr2=0x55b429833d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:50:15.120 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:15.114+0000 7fe18df18640 1 --2- 172.21.15.39:0/4270557027 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b4294581b0 0x55b429833d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:15.121 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:15.114+0000 7fe18df18640 1 -- 172.21.15.39:0/4270557027 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b429460390 msgr2=0x55b429834240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:50:15.121 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:15.115+0000 7fe18df18640 1 --2- 172.21.15.39:0/4270557027 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b429460390 0x55b429834240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:15.121 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:15.115+0000 7fe18df18640 1 -- 172.21.15.39:0/4270557027 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b429756c90 con 0x55b429481070 2024-09-21T03:50:15.121 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:15.115+0000 7fe18cf16640 1 --2- 172.21.15.39:0/4270557027 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b429460390 0x55b429834240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:50:15.121 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:15.115+0000 7fe18df18640 1 --2- 172.21.15.39:0/4270557027 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b429481070 0x55b429834780 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fe180006010 tx=0x7fe18002a8c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b608050238e48222 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:50:15.121 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:15.115+0000 7fe170ff9640 10 client.0 ms_handle_connect on v2:172.21.15.150:3300/0 2024-09-21T03:50:15.121 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:15.115+0000 7fe170ff9640 1 -- 172.21.15.39:0/4270557027 <== mon.2 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe180003c40 con 0x55b429481070 2024-09-21T03:50:15.121 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:15.115+0000 7fe195eef740 1 -- 172.21.15.39:0/4270557027 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe184004c50 con 0x55b429481070 2024-09-21T03:50:15.121 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:15.115+0000 7fe170ff9640 1 -- 172.21.15.39:0/4270557027 <== mon.2 v2:172.21.15.150:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8484+0+0 (secure 0 0 0) 0x7fe180003de0 con 0x55b429481070 2024-09-21T03:50:15.121 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:15.115+0000 7fe170ff9640 1 -- 172.21.15.39:0/4270557027 <== mon.2 v2:172.21.15.150:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fe180040820 con 0x55b429481070 2024-09-21T03:50:15.121 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:15.115+0000 7fe195eef740 10 client.26003 fetch_fsmap learned FSMap version 29 2024-09-21T03:50:15.121 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:15.115+0000 7fe195eef740 1 -- 172.21.15.39:0/4270557027 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55b429842ad0 con 0x55b429481070 2024-09-21T03:50:15.121 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:15.116+0000 7fe170ff9640 1 -- 172.21.15.39:0/4270557027 <== mon.2 v2:172.21.15.150:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe184004c50 con 0x55b429481070 2024-09-21T03:50:15.121 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:15.116+0000 7fe170ff9640 1 -- 172.21.15.39:0/4270557027 <== mon.2 v2:172.21.15.150:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fe18005e070 con 0x55b429481070 2024-09-21T03:50:15.121 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:15.116+0000 7fe195eef740 10 client.26003 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T03:50:15.121 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:15.116+0000 7fe195eef740 10 client.26003 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T03:50:15.122 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:15.116+0000 7fe195eef740 20 client.26003 populate_metadata read hostname 'smithi039' 2024-09-21T03:50:15.122 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:15.116+0000 7fe195eef740 1 --2- 172.21.15.39:0/4270557027 >> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] conn(0x55b42943f9f0 0x55b429845f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:50:15.122 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:15.116+0000 7fe195eef740 4 client.26003 mds_command: new command op to 24695 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:50:15.122 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:15.116+0000 7fe195eef740 1 -- 172.21.15.39:0/4270557027 --> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b4297a0df0 con 0x55b42943f9f0 2024-09-21T03:50:15.122 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:15.116+0000 7fe18d717640 1 --2- 172.21.15.39:0/4270557027 >> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] conn(0x55b42943f9f0 0x55b429845f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:50:15.123 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:15.117+0000 7fe170ff9640 10 client.26003 ms_handle_connect on v2:172.21.15.150:6838/2949386907 2024-09-21T03:50:15.123 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:15.117+0000 7fe18d717640 1 --2- 172.21.15.39:0/4270557027 >> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] conn(0x55b42943f9f0 0x55b429845f70 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2ca1e1293d03c1ee server_cookie=ab5089fa3be58d2e in_seq=0 out_seq=0 2024-09-21T03:50:15.123 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:15.117+0000 7fe170ff9640 1 -- 172.21.15.39:0/4270557027 <== mds.0 v2:172.21.15.150:6838/2949386907 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55b4297a0df0 con 0x55b42943f9f0 2024-09-21T03:50:15.123 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:15.117+0000 7fe170ff9640 10 client.26003 handle_command_reply: tid=0 2024-09-21T03:50:15.123 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:15.117+0000 7fe195eef740 1 client.26003 shutdown 2024-09-21T03:50:15.123 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:15.118+0000 7fe195eef740 1 -- 172.21.15.39:0/4270557027 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b429481070 msgr2=0x55b429834780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:50:15.123 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:15.118+0000 7fe195eef740 1 --2- 172.21.15.39:0/4270557027 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b429481070 0x55b429834780 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fe180006010 tx=0x7fe18002a8c0 comp rx=0 tx=0).stop 2024-09-21T03:50:15.123 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:15.118+0000 7fe195eef740 1 -- 172.21.15.39:0/4270557027 shutdown_connections 2024-09-21T03:50:15.123 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:15.118+0000 7fe195eef740 1 --2- 172.21.15.39:0/4270557027 >> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] conn(0x55b42943f9f0 0x55b429845f70 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:15.123 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:15.118+0000 7fe195eef740 1 --2- 172.21.15.39:0/4270557027 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b429481070 0x55b429834780 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:15.124 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:15.118+0000 7fe195eef740 1 --2- 172.21.15.39:0/4270557027 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b429460390 0x55b429834240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:15.124 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:15.118+0000 7fe170ff9640 0 client.26003 ms_handle_reset on v2:172.21.15.150:6838/2949386907 2024-09-21T03:50:15.124 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:15.118+0000 7fe195eef740 1 --2- 172.21.15.39:0/4270557027 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b4294581b0 0x55b429833d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:15.124 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:15.118+0000 7fe195eef740 1 -- 172.21.15.39:0/4270557027 >> 172.21.15.39:0/4270557027 conn(0x55b429475460 msgr2=0x55b42982b880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:50:15.124 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:15.118+0000 7fe195eef740 1 -- 172.21.15.39:0/4270557027 shutdown_connections 2024-09-21T03:50:15.124 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:15.118+0000 7fe195eef740 1 -- 172.21.15.39:0/4270557027 wait complete. 2024-09-21T03:50:15.124 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:15.118+0000 7fe195eef740 20 client.26003 trim_cache size 0 max 16384 2024-09-21T03:50:15.135 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:15.132+0000 7fe195eef740 1 Processor -- start 2024-09-21T03:50:15.135 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:15.132+0000 7fe195eef740 1 -- start start 2024-09-21T03:50:15.135 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:15.132+0000 7fe195eef740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b4294581b0 0x55b42982a1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:50:15.136 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:15.132+0000 7fe195eef740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b429460390 0x55b429835be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:50:15.136 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:15.132+0000 7fe195eef740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b429481070 0x55b42983f770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:50:15.136 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:15.132+0000 7fe195eef740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fe1881ad4e0 con 0x55b429460390 2024-09-21T03:50:15.136 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:15.132+0000 7fe195eef740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fe1881af020 con 0x55b4294581b0 2024-09-21T03:50:15.136 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:15.132+0000 7fe195eef740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fe1881adbb0 con 0x55b429481070 2024-09-21T03:50:15.136 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:15.132+0000 7fe18cf16640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b429460390 0x55b429835be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:50:15.136 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:15.132+0000 7fe18cf16640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b429460390 0x55b429835be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:56824/0 (socket says 172.21.15.39:56824) 2024-09-21T03:50:15.136 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:15.132+0000 7fe18cf16640 1 -- 172.21.15.39:0/678984777 learned_addr learned my addr 172.21.15.39:0/678984777 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:50:15.136 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:15.132+0000 7fe18d717640 1 --2- 172.21.15.39:0/678984777 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b4294581b0 0x55b42982a1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:50:15.136 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:15.132+0000 7fe18df18640 1 --2- 172.21.15.39:0/678984777 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b429481070 0x55b42983f770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:50:15.136 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:15.132+0000 7fe18cf16640 1 -- 172.21.15.39:0/678984777 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b429481070 msgr2=0x55b42983f770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:50:15.136 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:15.132+0000 7fe18cf16640 1 --2- 172.21.15.39:0/678984777 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b429481070 0x55b42983f770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:15.136 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:15.132+0000 7fe18cf16640 1 -- 172.21.15.39:0/678984777 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b4294581b0 msgr2=0x55b42982a1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:50:15.137 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:15.132+0000 7fe18cf16640 1 --2- 172.21.15.39:0/678984777 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b4294581b0 0x55b42982a1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:15.137 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:15.132+0000 7fe18cf16640 1 -- 172.21.15.39:0/678984777 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b429836440 con 0x55b429460390 2024-09-21T03:50:15.137 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:15.133+0000 7fe18cf16640 1 --2- 172.21.15.39:0/678984777 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b429460390 0x55b429835be0 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7fe1780e56f0 tx=0x7fe178004a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d1d4c88f8a26cfb4 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:50:15.137 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:15.133+0000 7fe172ffd640 1 -- 172.21.15.39:0/678984777 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1780acf60 con 0x55b429460390 2024-09-21T03:50:15.137 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:15.133+0000 7fe172ffd640 1 -- 172.21.15.39:0/678984777 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe178005310 con 0x55b429460390 2024-09-21T03:50:15.137 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:15.133+0000 7fe172ffd640 1 -- 172.21.15.39:0/678984777 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe17800ce60 con 0x55b429460390 2024-09-21T03:50:15.140 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:15.134+0000 7fe195eef740 1 -- 172.21.15.39:0/678984777 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b429460390 msgr2=0x55b429835be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:50:15.140 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:15.134+0000 7fe195eef740 1 --2- 172.21.15.39:0/678984777 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b429460390 0x55b429835be0 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7fe1780e56f0 tx=0x7fe178004a20 comp rx=0 tx=0).stop 2024-09-21T03:50:15.140 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:15.134+0000 7fe195eef740 1 -- 172.21.15.39:0/678984777 shutdown_connections 2024-09-21T03:50:15.141 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:15.134+0000 7fe195eef740 1 --2- 172.21.15.39:0/678984777 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b429481070 0x55b42983f770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:15.141 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:15.134+0000 7fe195eef740 1 --2- 172.21.15.39:0/678984777 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b429460390 0x55b429835be0 unknown :-1 s=CLOSED pgs=761 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:15.141 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:15.134+0000 7fe195eef740 1 --2- 172.21.15.39:0/678984777 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b4294581b0 0x55b42982a1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:15.141 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:15.134+0000 7fe195eef740 1 -- 172.21.15.39:0/678984777 >> 172.21.15.39:0/678984777 conn(0x55b429475460 msgr2=0x55b42983f060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:50:15.141 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:15.134+0000 7fe195eef740 1 -- 172.21.15.39:0/678984777 shutdown_connections 2024-09-21T03:50:15.141 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:15.134+0000 7fe195eef740 1 -- 172.21.15.39:0/678984777 wait complete. 2024-09-21T03:50:15.141 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:15.134+0000 7fe195eef740 1 -- start start 2024-09-21T03:50:15.141 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:15.134+0000 7fe195eef740 1 Processor -- start 2024-09-21T03:50:15.141 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:15.135+0000 7fe195eef740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b4294581b0 0x55b429838420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:50:15.141 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:15.135+0000 7fe195eef740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b429460390 0x55b429838960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:50:15.141 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:15.135+0000 7fe195eef740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b429481070 0x55b4298af970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:50:15.141 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:15.135+0000 7fe195eef740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fe1881ae950 con 0x55b4294581b0 2024-09-21T03:50:15.141 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:15.135+0000 7fe195eef740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fe1881ae280 con 0x55b429460390 2024-09-21T03:50:15.141 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:15.135+0000 7fe195eef740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fe1881ab4c0 con 0x55b429481070 2024-09-21T03:50:15.142 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:15.135+0000 7fe195eef740 10 client.0 _ll_register_callbacks cb 0x7fe180009b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:50:15.142 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:15.135+0000 7fe18d717640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b4294581b0 0x55b429838420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:50:15.142 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:15.135+0000 7fe18d717640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b4294581b0 0x55b429838420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:56834/0 (socket says 172.21.15.39:56834) 2024-09-21T03:50:15.142 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:15.135+0000 7fe18d717640 1 -- 172.21.15.39:0/1983827836 learned_addr learned my addr 172.21.15.39:0/1983827836 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:50:15.142 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:15.135+0000 7fe18df18640 1 --2- 172.21.15.39:0/1983827836 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b429481070 0x55b4298af970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:50:15.142 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:15.135+0000 7fe18cf16640 1 --2- 172.21.15.39:0/1983827836 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b429460390 0x55b429838960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:50:15.142 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:15.135+0000 7fe18d717640 1 -- 172.21.15.39:0/1983827836 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b429481070 msgr2=0x55b4298af970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:50:15.142 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:15.135+0000 7fe18d717640 1 --2- 172.21.15.39:0/1983827836 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b429481070 0x55b4298af970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:15.142 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:15.135+0000 7fe18d717640 1 -- 172.21.15.39:0/1983827836 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b429460390 msgr2=0x55b429838960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:50:15.142 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:15.135+0000 7fe18d717640 1 --2- 172.21.15.39:0/1983827836 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b429460390 0x55b429838960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:15.142 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:15.135+0000 7fe18d717640 1 -- 172.21.15.39:0/1983827836 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b4298afeb0 con 0x55b4294581b0 2024-09-21T03:50:15.142 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:15.135+0000 7fe18d717640 1 --2- 172.21.15.39:0/1983827836 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b4294581b0 0x55b429838420 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7fe184000c00 tx=0x7fe184005650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=92887a25b1f6e130 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:50:15.142 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:15.135+0000 7fe170ff9640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-09-21T03:50:15.142 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:15.136+0000 7fe170ff9640 1 -- 172.21.15.39:0/1983827836 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe184011d10 con 0x55b4294581b0 2024-09-21T03:50:15.143 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:15.136+0000 7fe170ff9640 1 -- 172.21.15.39:0/1983827836 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8484+0+0 (secure 0 0 0) 0x7fe1840035f0 con 0x55b4294581b0 2024-09-21T03:50:15.143 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:15.136+0000 7fe195eef740 1 -- 172.21.15.39:0/1983827836 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55b42979c8a0 con 0x55b4294581b0 2024-09-21T03:50:15.143 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:15.136+0000 7fe170ff9640 1 -- 172.21.15.39:0/1983827836 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fe1840037f0 con 0x55b4294581b0 2024-09-21T03:50:15.143 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:15.136+0000 7fe195eef740 10 client.16548 fetch_fsmap learned FSMap version 29 2024-09-21T03:50:15.143 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:15.136+0000 7fe195eef740 1 -- 172.21.15.39:0/1983827836 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55b4298b0f70 con 0x55b4294581b0 2024-09-21T03:50:15.143 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:15.136+0000 7fe170ff9640 1 -- 172.21.15.39:0/1983827836 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55b42979c8a0 con 0x55b4294581b0 2024-09-21T03:50:15.143 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:15.136+0000 7fe170ff9640 1 -- 172.21.15.39:0/1983827836 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fe1840197d0 con 0x55b4294581b0 2024-09-21T03:50:15.143 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:15.136+0000 7fe195eef740 10 client.16548 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T03:50:15.143 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:15.136+0000 7fe195eef740 10 client.16548 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T03:50:15.143 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:15.136+0000 7fe195eef740 20 client.16548 populate_metadata read hostname 'smithi039' 2024-09-21T03:50:15.143 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:15.136+0000 7fe195eef740 1 --2- 172.21.15.39:0/1983827836 >> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] conn(0x55b429414da0 0x55b4298b3540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:50:15.143 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:15.136+0000 7fe195eef740 4 client.16548 mds_command: new command op to 24695 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T03:50:15.143 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:15.136+0000 7fe195eef740 1 -- 172.21.15.39:0/1983827836 --> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55b429799cd0 con 0x55b429414da0 2024-09-21T03:50:15.144 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:15.137+0000 7fe18cf16640 1 --2- 172.21.15.39:0/1983827836 >> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] conn(0x55b429414da0 0x55b4298b3540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:50:15.144 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:15.137+0000 7fe18cf16640 1 --2- 172.21.15.39:0/1983827836 >> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] conn(0x55b429414da0 0x55b4298b3540 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c077f17e03429e56 server_cookie=79ffdc45b1af439f in_seq=0 out_seq=0 2024-09-21T03:50:15.144 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:15.137+0000 7fe170ff9640 10 client.16548 ms_handle_connect on v2:172.21.15.150:6838/2949386907 2024-09-21T03:50:15.144 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:15.137+0000 7fe170ff9640 1 -- 172.21.15.39:0/1983827836 <== mds.0 v2:172.21.15.150:6838/2949386907 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55b429799cd0 con 0x55b429414da0 2024-09-21T03:50:15.144 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:15.137+0000 7fe170ff9640 10 client.16548 handle_command_reply: tid=0 2024-09-21T03:50:15.144 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:15.137+0000 7fe195eef740 1 client.16548 shutdown 2024-09-21T03:50:15.144 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:15.137+0000 7fe195eef740 1 -- 172.21.15.39:0/1983827836 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b4294581b0 msgr2=0x55b429838420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:50:15.144 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:15.137+0000 7fe195eef740 1 --2- 172.21.15.39:0/1983827836 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b4294581b0 0x55b429838420 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7fe184000c00 tx=0x7fe184005650 comp rx=0 tx=0).stop 2024-09-21T03:50:15.144 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:15.137+0000 7fe195eef740 1 -- 172.21.15.39:0/1983827836 shutdown_connections 2024-09-21T03:50:15.144 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:15.137+0000 7fe195eef740 1 --2- 172.21.15.39:0/1983827836 >> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] conn(0x55b429414da0 0x55b4298b3540 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:15.144 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:15.137+0000 7fe195eef740 1 --2- 172.21.15.39:0/1983827836 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b429481070 0x55b4298af970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:15.144 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:15.137+0000 7fe195eef740 1 --2- 172.21.15.39:0/1983827836 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b429460390 0x55b429838960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:15.144 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:15.137+0000 7fe195eef740 1 --2- 172.21.15.39:0/1983827836 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b4294581b0 0x55b429838420 unknown :-1 s=CLOSED pgs=762 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:15.144 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:15.137+0000 7fe195eef740 1 -- 172.21.15.39:0/1983827836 >> 172.21.15.39:0/1983827836 conn(0x55b429475460 msgr2=0x55b42983fa20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:50:15.144 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:15.137+0000 7fe170ff9640 0 client.16548 ms_handle_reset on v2:172.21.15.150:6838/2949386907 2024-09-21T03:50:15.145 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:15.138+0000 7fe195eef740 1 -- 172.21.15.39:0/1983827836 shutdown_connections 2024-09-21T03:50:15.145 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:15.138+0000 7fe195eef740 1 -- 172.21.15.39:0/1983827836 wait complete. 2024-09-21T03:50:15.145 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:15.138+0000 7fe195eef740 20 client.16548 trim_cache size 0 max 16384 2024-09-21T03:50:15.145 INFO:teuthology.orchestra.run.smithi039.stdout:[] 2024-09-21T03:50:15.146 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:15.138+0000 7fe18f97b640 1 -- 172.21.15.39:0/1926784328 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7fe1740b66b0 msgr2=0x7fe1740b8bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:50:15.146 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:15.138+0000 7fe18f97b640 1 --2- 172.21.15.39:0/1926784328 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7fe1740b66b0 0x7fe1740b8bc0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fe180009a60 tx=0x7fe18002f470 comp rx=0 tx=0).stop 2024-09-21T03:50:15.146 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:15.138+0000 7fe18f97b640 1 -- 172.21.15.39:0/1926784328 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe1881083b0 msgr2=0x7fe18819b8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:50:15.146 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:15.138+0000 7fe18f97b640 1 --2- 172.21.15.39:0/1926784328 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe1881083b0 0x7fe18819b8b0 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7fe17800e9c0 tx=0x7fe17800ee80 comp rx=0 tx=0).stop 2024-09-21T03:50:15.146 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:15.138+0000 7fe18f97b640 1 -- 172.21.15.39:0/1926784328 shutdown_connections 2024-09-21T03:50:15.146 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:15.138+0000 7fe18f97b640 1 --2- 172.21.15.39:0/1926784328 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7fe1740b66b0 0x7fe1740b8bc0 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:15.146 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:15.138+0000 7fe18f97b640 1 --2- 172.21.15.39:0/1926784328 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe188108d00 0x7fe18819bdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:15.146 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:15.138+0000 7fe18f97b640 1 --2- 172.21.15.39:0/1926784328 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe1881083b0 0x7fe18819b8b0 unknown :-1 s=CLOSED pgs=759 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:15.146 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:15.138+0000 7fe18f97b640 1 --2- 172.21.15.39:0/1926784328 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe188102a70 0x7fe18819b370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:15.146 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:15.138+0000 7fe18f97b640 1 -- 172.21.15.39:0/1926784328 >> 172.21.15.39:0/1926784328 conn(0x7fe188079710 msgr2=0x7fe1881064b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:50:15.146 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:15.138+0000 7fe18f97b640 1 -- 172.21.15.39:0/1926784328 shutdown_connections 2024-09-21T03:50:15.147 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:15.139+0000 7fe18f97b640 1 -- 172.21.15.39:0/1926784328 wait complete. 2024-09-21T03:50:15.218 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:50:14 smithi161 ceph-mon[34533]: pgmap v451: 97 pgs: 97 active+clean; 10 MiB data, 536 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 25 op/s 2024-09-21T03:50:15.224 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:50:14 smithi150 ceph-mon[34289]: pgmap v451: 97 pgs: 97 active+clean; 10 MiB data, 536 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 25 op/s 2024-09-21T03:50:16.939 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-21T03:50:16.940 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 51116d5c-77ca-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-21T03:50:17.214 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/51116d5c-77ca-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:50:17.217 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:50:16 smithi161 ceph-mon[34533]: pgmap v452: 97 pgs: 97 active+clean; 10 MiB data, 536 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 25 op/s 2024-09-21T03:50:17.225 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:50:16 smithi150 ceph-mon[34289]: pgmap v452: 97 pgs: 97 active+clean; 10 MiB data, 536 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 25 op/s 2024-09-21T03:50:17.334 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:50:16 smithi039 ceph-mon[31211]: pgmap v452: 97 pgs: 97 active+clean; 10 MiB data, 536 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 25 op/s 2024-09-21T03:50:17.705 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:17.689+0000 7ffb59f1b640 1 -- 172.21.15.39:0/3366268266 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ffb541083d0 msgr2=0x7ffb541087e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:50:17.705 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:17.689+0000 7ffb59f1b640 1 --2- 172.21.15.39:0/3366268266 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ffb541083d0 0x7ffb541087e0 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7ffb3c009900 tx=0x7ffb3c02a410 comp rx=0 tx=0).stop 2024-09-21T03:50:17.705 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:17.690+0000 7ffb59f1b640 1 -- 172.21.15.39:0/3366268266 shutdown_connections 2024-09-21T03:50:17.705 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:17.690+0000 7ffb59f1b640 1 --2- 172.21.15.39:0/3366268266 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ffb54108d20 0x7ffb5410f550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:17.705 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:17.690+0000 7ffb59f1b640 1 --2- 172.21.15.39:0/3366268266 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ffb541083d0 0x7ffb541087e0 unknown :-1 s=CLOSED pgs=763 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:17.705 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:17.690+0000 7ffb59f1b640 1 --2- 172.21.15.39:0/3366268266 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ffb54102ae0 0x7ffb54102ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:17.705 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:17.690+0000 7ffb59f1b640 1 -- 172.21.15.39:0/3366268266 >> 172.21.15.39:0/3366268266 conn(0x7ffb54079760 msgr2=0x7ffb54079b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:50:17.705 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:17.690+0000 7ffb59f1b640 1 -- 172.21.15.39:0/3366268266 shutdown_connections 2024-09-21T03:50:17.705 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:17.690+0000 7ffb59f1b640 1 -- 172.21.15.39:0/3366268266 wait complete. 2024-09-21T03:50:17.705 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:17.690+0000 7ffb59f1b640 1 Processor -- start 2024-09-21T03:50:17.705 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:17.690+0000 7ffb59f1b640 1 -- start start 2024-09-21T03:50:17.706 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:17.691+0000 7ffb59f1b640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ffb54102ae0 0x7ffb5419a520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:50:17.706 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:17.691+0000 7ffb59f1b640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ffb541083d0 0x7ffb5419aac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:50:17.706 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:17.691+0000 7ffb59f1b640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ffb54108d20 0x7ffb5419fd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:50:17.706 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:17.691+0000 7ffb59f1b640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7ffb54113af0 con 0x7ffb54108d20 2024-09-21T03:50:17.706 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:17.691+0000 7ffb59f1b640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7ffb54113970 con 0x7ffb54102ae0 2024-09-21T03:50:17.706 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:17.691+0000 7ffb59f1b640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7ffb54113c70 con 0x7ffb541083d0 2024-09-21T03:50:17.706 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:17.691+0000 7ffb53fff640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ffb54108d20 0x7ffb5419fd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:50:17.706 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:17.691+0000 7ffb537fe640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ffb54102ae0 0x7ffb5419a520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:50:17.706 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:17.691+0000 7ffb53fff640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ffb54108d20 0x7ffb5419fd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:44720/0 (socket says 172.21.15.39:44720) 2024-09-21T03:50:17.706 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:17.691+0000 7ffb53fff640 1 -- 172.21.15.39:0/1414293968 learned_addr learned my addr 172.21.15.39:0/1414293968 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:50:17.706 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:17.691+0000 7ffb52ffd640 1 --2- 172.21.15.39:0/1414293968 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ffb541083d0 0x7ffb5419aac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:50:17.706 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:17.691+0000 7ffb53fff640 1 -- 172.21.15.39:0/1414293968 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ffb541083d0 msgr2=0x7ffb5419aac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:50:17.706 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:17.691+0000 7ffb53fff640 1 --2- 172.21.15.39:0/1414293968 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ffb541083d0 0x7ffb5419aac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:17.707 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:17.691+0000 7ffb53fff640 1 -- 172.21.15.39:0/1414293968 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ffb54102ae0 msgr2=0x7ffb5419a520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:50:17.707 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:17.691+0000 7ffb53fff640 1 --2- 172.21.15.39:0/1414293968 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ffb54102ae0 0x7ffb5419a520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:17.707 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:17.691+0000 7ffb53fff640 1 -- 172.21.15.39:0/1414293968 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffb541a0490 con 0x7ffb54108d20 2024-09-21T03:50:17.707 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:17.692+0000 7ffb53fff640 1 --2- 172.21.15.39:0/1414293968 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ffb54108d20 0x7ffb5419fd60 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7ffb4c00b970 tx=0x7ffb4c00be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:50:17.707 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:17.692+0000 7ffb50ff9640 1 -- 172.21.15.39:0/1414293968 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb4c019070 con 0x7ffb54108d20 2024-09-21T03:50:17.707 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:17.692+0000 7ffb59f1b640 1 -- 172.21.15.39:0/1414293968 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffb541a0780 con 0x7ffb54108d20 2024-09-21T03:50:17.708 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:17.692+0000 7ffb50ff9640 1 -- 172.21.15.39:0/1414293968 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ffb4c014440 con 0x7ffb54108d20 2024-09-21T03:50:17.708 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:17.692+0000 7ffb50ff9640 1 -- 172.21.15.39:0/1414293968 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb4c0075d0 con 0x7ffb54108d20 2024-09-21T03:50:17.708 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:17.692+0000 7ffb59f1b640 1 -- 172.21.15.39:0/1414293968 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffb541a6ee0 con 0x7ffb54108d20 2024-09-21T03:50:17.708 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:17.694+0000 7ffb50ff9640 1 -- 172.21.15.39:0/1414293968 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 27) ==== 154643+0+0 (secure 0 0 0) 0x7ffb4c01c070 con 0x7ffb54108d20 2024-09-21T03:50:17.708 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:17.695+0000 7ffb6048f740 1 Processor -- start 2024-09-21T03:50:17.708 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:17.695+0000 7ffb6048f740 1 -- start start 2024-09-21T03:50:17.709 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:17.695+0000 7ffb6048f740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555639b1b350 0x555639ec2430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:50:17.709 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:17.695+0000 7ffb6048f740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x555639af2490 0x555639ec2970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:50:17.709 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:17.695+0000 7ffb6048f740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555639afa670 0x555639ec2eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:50:17.709 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:17.695+0000 7ffb6048f740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x555639e38e60 con 0x555639af2490 2024-09-21T03:50:17.709 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:17.695+0000 7ffb6048f740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x555639e28290 con 0x555639afa670 2024-09-21T03:50:17.709 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:17.695+0000 7ffb6048f740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x555639e2c0a0 con 0x555639b1b350 2024-09-21T03:50:17.709 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:17.695+0000 7ffb537fe640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x555639af2490 0x555639ec2970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:50:17.709 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:17.695+0000 7ffb537fe640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x555639af2490 0x555639ec2970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:44732/0 (socket says 172.21.15.39:44732) 2024-09-21T03:50:17.709 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:17.695+0000 7ffb537fe640 1 -- 172.21.15.39:0/1477409212 learned_addr learned my addr 172.21.15.39:0/1477409212 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:50:17.709 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:17.695+0000 7ffb52ffd640 1 --2- 172.21.15.39:0/1477409212 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555639afa670 0x555639ec2eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:50:17.709 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:17.695+0000 7ffb52ffd640 1 --2- 172.21.15.39:0/1477409212 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555639b1b350 0x555639ec2430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:50:17.709 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:17.695+0000 7ffb537fe640 1 -- 172.21.15.39:0/1477409212 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555639b1b350 msgr2=0x555639ec2430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:50:17.709 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:17.695+0000 7ffb537fe640 1 --2- 172.21.15.39:0/1477409212 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555639b1b350 0x555639ec2430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:17.709 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:17.695+0000 7ffb537fe640 1 -- 172.21.15.39:0/1477409212 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555639afa670 msgr2=0x555639ec2eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:50:17.710 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:17.695+0000 7ffb537fe640 1 --2- 172.21.15.39:0/1477409212 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555639afa670 0x555639ec2eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:17.710 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:17.695+0000 7ffb537fe640 1 -- 172.21.15.39:0/1477409212 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555639b86930 con 0x555639af2490 2024-09-21T03:50:17.710 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:17.695+0000 7ffb52ffd640 1 --2- 172.21.15.39:0/1477409212 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555639b1b350 0x555639ec2430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:50:17.710 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:17.695+0000 7ffb50ff9640 1 --2- 172.21.15.39:0/1414293968 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7ffb400b6680 0x7ffb400b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:50:17.710 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:17.696+0000 7ffb52ffd640 1 --2- 172.21.15.39:0/1414293968 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7ffb400b6680 0x7ffb400b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:50:17.710 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:17.696+0000 7ffb537fe640 1 --2- 172.21.15.39:0/1477409212 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x555639af2490 0x555639ec2970 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7ffb4400b760 tx=0x7ffb4400bc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=426a3bc80f4dea89 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:50:17.710 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:17.696+0000 7ffb50ff9640 1 -- 172.21.15.39:0/1414293968 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8484+0+0 (secure 0 0 0) 0x7ffb4c0e6c00 con 0x7ffb54108d20 2024-09-21T03:50:17.710 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:17.696+0000 7ffb49ffb640 1 -- 172.21.15.39:0/1477409212 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb44003dd0 con 0x555639af2490 2024-09-21T03:50:17.710 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:17.696+0000 7ffb49ffb640 1 -- 172.21.15.39:0/1477409212 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ffb44015070 con 0x555639af2490 2024-09-21T03:50:17.710 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:17.696+0000 7ffb49ffb640 1 -- 172.21.15.39:0/1477409212 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb44004d90 con 0x555639af2490 2024-09-21T03:50:17.710 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:17.697+0000 7ffb6048f740 1 -- 172.21.15.39:0/1477409212 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x555639af2490 msgr2=0x555639ec2970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:50:17.710 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:17.697+0000 7ffb52ffd640 1 --2- 172.21.15.39:0/1414293968 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7ffb400b6680 0x7ffb400b8b90 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7ffb3c000c00 tx=0x7ffb3c005780 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:50:17.710 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:17.697+0000 7ffb6048f740 1 --2- 172.21.15.39:0/1477409212 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x555639af2490 0x555639ec2970 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7ffb4400b760 tx=0x7ffb4400bc20 comp rx=0 tx=0).stop 2024-09-21T03:50:17.711 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:17.697+0000 7ffb6048f740 1 -- 172.21.15.39:0/1477409212 shutdown_connections 2024-09-21T03:50:17.711 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:17.697+0000 7ffb6048f740 1 --2- 172.21.15.39:0/1477409212 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555639afa670 0x555639ec2eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:17.711 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:17.697+0000 7ffb6048f740 1 --2- 172.21.15.39:0/1477409212 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x555639af2490 0x555639ec2970 unknown :-1 s=CLOSED pgs=765 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:17.711 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:17.697+0000 7ffb6048f740 1 --2- 172.21.15.39:0/1477409212 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555639b1b350 0x555639ec2430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:17.711 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:17.697+0000 7ffb6048f740 1 -- 172.21.15.39:0/1477409212 >> 172.21.15.39:0/1477409212 conn(0x555639b0f740 msgr2=0x555639ec0320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:50:17.711 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:17.697+0000 7ffb6048f740 1 -- 172.21.15.39:0/1477409212 shutdown_connections 2024-09-21T03:50:17.711 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:17.697+0000 7ffb6048f740 1 -- 172.21.15.39:0/1477409212 wait complete. 2024-09-21T03:50:17.711 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:17.697+0000 7ffb6048f740 1 -- start start 2024-09-21T03:50:17.711 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:17.698+0000 7ffb6048f740 1 Processor -- start 2024-09-21T03:50:17.711 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:17.698+0000 7ffb6048f740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555639af2490 0x555639ece000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:50:17.711 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:17.698+0000 7ffb6048f740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555639afa670 0x555639ece540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:50:17.711 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:17.698+0000 7ffb6048f740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x555639b1b350 0x555639ed9f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:50:17.711 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:17.698+0000 7ffb6048f740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x555639c89890 con 0x555639b1b350 2024-09-21T03:50:17.711 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:17.698+0000 7ffb6048f740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x555639dc2d40 con 0x555639af2490 2024-09-21T03:50:17.711 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:17.698+0000 7ffb6048f740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x555639ed0c10 con 0x555639afa670 2024-09-21T03:50:17.712 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:17.698+0000 7ffb6048f740 10 client.0 _ll_register_callbacks cb 0x555639e26c80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:50:17.712 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:17.698+0000 7ffb53fff640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x555639b1b350 0x555639ed9f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:50:17.712 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:17.698+0000 7ffb53fff640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x555639b1b350 0x555639ed9f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:44740/0 (socket says 172.21.15.39:44740) 2024-09-21T03:50:17.712 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:17.698+0000 7ffb53fff640 1 -- 172.21.15.39:0/291970826 learned_addr learned my addr 172.21.15.39:0/291970826 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:50:17.712 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:17.698+0000 7ffb537fe640 1 --2- 172.21.15.39:0/291970826 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555639af2490 0x555639ece000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:50:17.712 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:17.698+0000 7ffb52ffd640 1 --2- 172.21.15.39:0/291970826 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555639afa670 0x555639ece540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:50:17.712 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:17.699+0000 7ffb53fff640 1 -- 172.21.15.39:0/291970826 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555639afa670 msgr2=0x555639ece540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:50:17.712 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:17.699+0000 7ffb53fff640 1 --2- 172.21.15.39:0/291970826 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555639afa670 0x555639ece540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:17.712 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:17.699+0000 7ffb53fff640 1 -- 172.21.15.39:0/291970826 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555639af2490 msgr2=0x555639ece000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:50:17.712 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:17.699+0000 7ffb53fff640 1 --2- 172.21.15.39:0/291970826 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555639af2490 0x555639ece000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:17.712 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:17.699+0000 7ffb53fff640 1 -- 172.21.15.39:0/291970826 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555639eda4c0 con 0x555639b1b350 2024-09-21T03:50:17.712 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:17.699+0000 7ffb53fff640 1 --2- 172.21.15.39:0/291970826 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x555639b1b350 0x555639ed9f30 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7ffb4c0a90f0 tx=0x7ffb4c0ac350 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2dfbb54e988a46a8 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:50:17.712 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:17.699+0000 7ffb32ffd640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-09-21T03:50:17.712 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:17.699+0000 7ffb32ffd640 1 -- 172.21.15.39:0/291970826 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb4c0b1550 con 0x555639b1b350 2024-09-21T03:50:17.713 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:17.699+0000 7ffb32ffd640 1 -- 172.21.15.39:0/291970826 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8484+0+0 (secure 0 0 0) 0x7ffb4c0b11f0 con 0x555639b1b350 2024-09-21T03:50:17.713 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:17.699+0000 7ffb6048f740 1 -- 172.21.15.39:0/291970826 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ffb44015070 con 0x555639b1b350 2024-09-21T03:50:17.713 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:17.699+0000 7ffb32ffd640 1 -- 172.21.15.39:0/291970826 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7ffb4c0b6490 con 0x555639b1b350 2024-09-21T03:50:17.713 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:17.699+0000 7ffb6048f740 10 client.16560 fetch_fsmap learned FSMap version 29 2024-09-21T03:50:17.713 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:17.699+0000 7ffb6048f740 1 -- 172.21.15.39:0/291970826 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x555639edcdd0 con 0x555639b1b350 2024-09-21T03:50:17.713 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:17.699+0000 7ffb32ffd640 1 -- 172.21.15.39:0/291970826 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ffb44015070 con 0x555639b1b350 2024-09-21T03:50:17.713 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:17.699+0000 7ffb32ffd640 1 -- 172.21.15.39:0/291970826 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7ffb4c0a8470 con 0x555639b1b350 2024-09-21T03:50:17.713 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:17.700+0000 7ffb6048f740 10 client.16560 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T03:50:17.713 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:17.700+0000 7ffb6048f740 10 client.16560 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T03:50:17.713 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:17.700+0000 7ffb6048f740 20 client.16560 populate_metadata read hostname 'smithi039' 2024-09-21T03:50:17.713 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:17.700+0000 7ffb6048f740 1 --2- 172.21.15.39:0/291970826 >> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] conn(0x555639ad9cd0 0x555639ee0300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:50:17.713 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:17.700+0000 7ffb6048f740 4 client.16560 mds_command: new command op to 24695 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:50:17.713 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:17.700+0000 7ffb6048f740 1 -- 172.21.15.39:0/291970826 --> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555639e3adb0 con 0x555639ad9cd0 2024-09-21T03:50:17.713 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:17.700+0000 7ffb537fe640 1 --2- 172.21.15.39:0/291970826 >> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] conn(0x555639ad9cd0 0x555639ee0300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:50:17.714 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:17.700+0000 7ffb537fe640 1 --2- 172.21.15.39:0/291970826 >> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] conn(0x555639ad9cd0 0x555639ee0300 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=27bbfe38af6f11e3 server_cookie=55e4e53475757d61 in_seq=0 out_seq=0 2024-09-21T03:50:17.714 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:17.700+0000 7ffb32ffd640 10 client.16560 ms_handle_connect on v2:172.21.15.150:6838/2949386907 2024-09-21T03:50:17.714 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:17.701+0000 7ffb32ffd640 1 -- 172.21.15.39:0/291970826 <== mds.0 v2:172.21.15.150:6838/2949386907 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x555639e3adb0 con 0x555639ad9cd0 2024-09-21T03:50:17.714 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:17.701+0000 7ffb32ffd640 10 client.16560 handle_command_reply: tid=0 2024-09-21T03:50:17.714 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:17.701+0000 7ffb6048f740 1 client.16560 shutdown 2024-09-21T03:50:17.714 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:17.701+0000 7ffb6048f740 1 -- 172.21.15.39:0/291970826 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x555639b1b350 msgr2=0x555639ed9f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:50:17.714 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:17.701+0000 7ffb6048f740 1 --2- 172.21.15.39:0/291970826 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x555639b1b350 0x555639ed9f30 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7ffb4c0a90f0 tx=0x7ffb4c0ac350 comp rx=0 tx=0).stop 2024-09-21T03:50:17.714 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:17.701+0000 7ffb6048f740 1 -- 172.21.15.39:0/291970826 shutdown_connections 2024-09-21T03:50:17.714 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:17.701+0000 7ffb6048f740 1 --2- 172.21.15.39:0/291970826 >> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] conn(0x555639ad9cd0 0x555639ee0300 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:17.714 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:17.701+0000 7ffb6048f740 1 --2- 172.21.15.39:0/291970826 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x555639b1b350 0x555639ed9f30 unknown :-1 s=CLOSED pgs=766 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:17.714 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:17.701+0000 7ffb32ffd640 0 client.16560 ms_handle_reset on v2:172.21.15.150:6838/2949386907 2024-09-21T03:50:17.714 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:17.701+0000 7ffb6048f740 1 --2- 172.21.15.39:0/291970826 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555639afa670 0x555639ece540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:17.714 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:17.701+0000 7ffb6048f740 1 --2- 172.21.15.39:0/291970826 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555639af2490 0x555639ece000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:17.714 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:17.701+0000 7ffb6048f740 1 -- 172.21.15.39:0/291970826 >> 172.21.15.39:0/291970826 conn(0x555639b0f740 msgr2=0x555639ec55c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:50:17.714 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:17.702+0000 7ffb6048f740 1 -- 172.21.15.39:0/291970826 shutdown_connections 2024-09-21T03:50:17.715 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:17.702+0000 7ffb6048f740 1 -- 172.21.15.39:0/291970826 wait complete. 2024-09-21T03:50:17.715 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:17.702+0000 7ffb6048f740 20 client.16560 trim_cache size 0 max 16384 2024-09-21T03:50:17.723 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:17.719+0000 7ffb6048f740 1 Processor -- start 2024-09-21T03:50:17.723 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:17.719+0000 7ffb6048f740 1 -- start start 2024-09-21T03:50:17.723 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:17.720+0000 7ffb6048f740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555639ad9cd0 0x555639ec8550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:50:17.723 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:17.720+0000 7ffb6048f740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x555639af2490 0x555639ecfa30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:50:17.723 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:17.720+0000 7ffb6048f740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555639afa670 0x555639ecaff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:50:17.724 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:17.720+0000 7ffb6048f740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7ffb541af600 con 0x555639af2490 2024-09-21T03:50:17.724 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:17.720+0000 7ffb6048f740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7ffb541b1140 con 0x555639afa670 2024-09-21T03:50:17.724 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:17.720+0000 7ffb6048f740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7ffb541afcd0 con 0x555639ad9cd0 2024-09-21T03:50:17.724 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:17.720+0000 7ffb52ffd640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x555639af2490 0x555639ecfa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:50:17.724 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:17.720+0000 7ffb52ffd640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x555639af2490 0x555639ecfa30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:44752/0 (socket says 172.21.15.39:44752) 2024-09-21T03:50:17.724 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:17.720+0000 7ffb52ffd640 1 -- 172.21.15.39:0/1063491381 learned_addr learned my addr 172.21.15.39:0/1063491381 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:50:17.724 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:17.720+0000 7ffb537fe640 1 --2- 172.21.15.39:0/1063491381 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555639ad9cd0 0x555639ec8550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:50:17.724 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:17.720+0000 7ffb53fff640 1 --2- 172.21.15.39:0/1063491381 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555639afa670 0x555639ecaff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:50:17.724 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:17.720+0000 7ffb52ffd640 1 -- 172.21.15.39:0/1063491381 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555639ad9cd0 msgr2=0x555639ec8550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:50:17.724 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:17.720+0000 7ffb52ffd640 1 --2- 172.21.15.39:0/1063491381 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555639ad9cd0 0x555639ec8550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:17.724 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:17.720+0000 7ffb52ffd640 1 -- 172.21.15.39:0/1063491381 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555639afa670 msgr2=0x555639ecaff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:50:17.724 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:17.720+0000 7ffb52ffd640 1 --2- 172.21.15.39:0/1063491381 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555639afa670 0x555639ecaff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:17.724 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:17.720+0000 7ffb52ffd640 1 -- 172.21.15.39:0/1063491381 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555639ecc300 con 0x555639af2490 2024-09-21T03:50:17.724 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:17.720+0000 7ffb537fe640 1 --2- 172.21.15.39:0/1063491381 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555639ad9cd0 0x555639ec8550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:50:17.724 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:17.720+0000 7ffb53fff640 1 --2- 172.21.15.39:0/1063491381 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555639afa670 0x555639ecaff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:50:17.725 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:17.720+0000 7ffb52ffd640 1 --2- 172.21.15.39:0/1063491381 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x555639af2490 0x555639ecfa30 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7ffb3c033000 tx=0x7ffb3c03f040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=60183b0aad391b59 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:50:17.725 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:17.721+0000 7ffb497fa640 1 -- 172.21.15.39:0/1063491381 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb3c035070 con 0x555639af2490 2024-09-21T03:50:17.725 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:17.721+0000 7ffb497fa640 1 -- 172.21.15.39:0/1063491381 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ffb3c0035f0 con 0x555639af2490 2024-09-21T03:50:17.725 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:17.721+0000 7ffb497fa640 1 -- 172.21.15.39:0/1063491381 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb3c03d6b0 con 0x555639af2490 2024-09-21T03:50:17.728 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:17.721+0000 7ffb6048f740 1 -- 172.21.15.39:0/1063491381 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x555639af2490 msgr2=0x555639ecfa30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:50:17.728 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:17.721+0000 7ffb6048f740 1 --2- 172.21.15.39:0/1063491381 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x555639af2490 0x555639ecfa30 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7ffb3c033000 tx=0x7ffb3c03f040 comp rx=0 tx=0).stop 2024-09-21T03:50:17.728 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:17.721+0000 7ffb6048f740 1 -- 172.21.15.39:0/1063491381 shutdown_connections 2024-09-21T03:50:17.728 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:17.721+0000 7ffb6048f740 1 --2- 172.21.15.39:0/1063491381 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555639afa670 0x555639ecaff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:17.728 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:17.721+0000 7ffb6048f740 1 --2- 172.21.15.39:0/1063491381 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x555639af2490 0x555639ecfa30 unknown :-1 s=CLOSED pgs=767 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:17.728 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:17.721+0000 7ffb6048f740 1 --2- 172.21.15.39:0/1063491381 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555639ad9cd0 0x555639ec8550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:17.728 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:17.721+0000 7ffb6048f740 1 -- 172.21.15.39:0/1063491381 >> 172.21.15.39:0/1063491381 conn(0x555639b0f740 msgr2=0x555639ecf320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:50:17.728 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:17.722+0000 7ffb6048f740 1 -- 172.21.15.39:0/1063491381 shutdown_connections 2024-09-21T03:50:17.729 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:17.722+0000 7ffb6048f740 1 -- 172.21.15.39:0/1063491381 wait complete. 2024-09-21T03:50:17.729 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:17.722+0000 7ffb6048f740 1 -- start start 2024-09-21T03:50:17.729 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:17.722+0000 7ffb6048f740 1 Processor -- start 2024-09-21T03:50:17.729 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:17.722+0000 7ffb6048f740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x555639ad9cd0 0x555639ed6330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:50:17.729 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:17.722+0000 7ffb6048f740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555639af2490 0x555639ed27f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:50:17.729 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:17.723+0000 7ffb6048f740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555639afa670 0x555639ed2d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:50:17.729 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:17.723+0000 7ffb6048f740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7ffb541b0a70 con 0x555639ad9cd0 2024-09-21T03:50:17.729 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:17.723+0000 7ffb6048f740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7ffb541b03a0 con 0x555639af2490 2024-09-21T03:50:17.729 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:17.723+0000 7ffb6048f740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7ffb541ad5e0 con 0x555639afa670 2024-09-21T03:50:17.729 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:17.723+0000 7ffb6048f740 10 client.0 _ll_register_callbacks cb 0x7ffb3c009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:50:17.729 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:17.723+0000 7ffb537fe640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x555639ad9cd0 0x555639ed6330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:50:17.729 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:17.723+0000 7ffb537fe640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x555639ad9cd0 0x555639ed6330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:44762/0 (socket says 172.21.15.39:44762) 2024-09-21T03:50:17.730 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:17.723+0000 7ffb537fe640 1 -- 172.21.15.39:0/650491242 learned_addr learned my addr 172.21.15.39:0/650491242 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:50:17.730 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:17.723+0000 7ffb52ffd640 1 --2- 172.21.15.39:0/650491242 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555639af2490 0x555639ed27f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:50:17.730 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:17.723+0000 7ffb53fff640 1 --2- 172.21.15.39:0/650491242 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555639afa670 0x555639ed2d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:50:17.730 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:17.723+0000 7ffb537fe640 1 -- 172.21.15.39:0/650491242 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555639afa670 msgr2=0x555639ed2d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:50:17.730 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:17.723+0000 7ffb537fe640 1 --2- 172.21.15.39:0/650491242 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555639afa670 0x555639ed2d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:17.730 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:17.723+0000 7ffb537fe640 1 -- 172.21.15.39:0/650491242 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555639af2490 msgr2=0x555639ed27f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:50:17.730 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:17.723+0000 7ffb537fe640 1 --2- 172.21.15.39:0/650491242 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555639af2490 0x555639ed27f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:17.730 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:17.723+0000 7ffb537fe640 1 -- 172.21.15.39:0/650491242 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555639ed3270 con 0x555639ad9cd0 2024-09-21T03:50:17.730 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:17.723+0000 7ffb53fff640 1 --2- 172.21.15.39:0/650491242 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555639afa670 0x555639ed2d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:50:17.730 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:17.723+0000 7ffb537fe640 1 --2- 172.21.15.39:0/650491242 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x555639ad9cd0 0x555639ed6330 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7ffb4400b760 tx=0x7ffb440048d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6967d43f99a18d38 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:50:17.730 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:17.723+0000 7ffb32ffd640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-09-21T03:50:17.730 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:17.723+0000 7ffb32ffd640 1 -- 172.21.15.39:0/650491242 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb4402c460 con 0x555639ad9cd0 2024-09-21T03:50:17.730 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:17.724+0000 7ffb32ffd640 1 -- 172.21.15.39:0/650491242 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8484+0+0 (secure 0 0 0) 0x7ffb44011440 con 0x555639ad9cd0 2024-09-21T03:50:17.730 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:17.724+0000 7ffb6048f740 1 -- 172.21.15.39:0/650491242 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ffb3c0035f0 con 0x555639ad9cd0 2024-09-21T03:50:17.731 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:17.724+0000 7ffb32ffd640 1 -- 172.21.15.39:0/650491242 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7ffb44011640 con 0x555639ad9cd0 2024-09-21T03:50:17.731 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:17.724+0000 7ffb6048f740 10 client.16566 fetch_fsmap learned FSMap version 29 2024-09-21T03:50:17.731 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:17.724+0000 7ffb6048f740 1 -- 172.21.15.39:0/650491242 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x555639f4b8e0 con 0x555639ad9cd0 2024-09-21T03:50:17.731 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:17.724+0000 7ffb32ffd640 1 -- 172.21.15.39:0/650491242 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ffb3c0035f0 con 0x555639ad9cd0 2024-09-21T03:50:17.731 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:17.724+0000 7ffb32ffd640 1 -- 172.21.15.39:0/650491242 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7ffb44033020 con 0x555639ad9cd0 2024-09-21T03:50:17.731 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:17.724+0000 7ffb6048f740 10 client.16566 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T03:50:17.731 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:17.724+0000 7ffb6048f740 10 client.16566 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T03:50:17.731 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:17.724+0000 7ffb6048f740 20 client.16566 populate_metadata read hostname 'smithi039' 2024-09-21T03:50:17.731 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:17.724+0000 7ffb6048f740 1 --2- 172.21.15.39:0/650491242 >> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] conn(0x555639b1b350 0x555639f4dda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:50:17.731 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:17.724+0000 7ffb6048f740 4 client.16566 mds_command: new command op to 24695 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-21T03:50:17.731 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:17.724+0000 7ffb6048f740 1 -- 172.21.15.39:0/650491242 --> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x555639e36860 con 0x555639b1b350 2024-09-21T03:50:17.731 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:17.724+0000 7ffb52ffd640 1 --2- 172.21.15.39:0/650491242 >> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] conn(0x555639b1b350 0x555639f4dda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:50:17.731 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:17.725+0000 7ffb52ffd640 1 --2- 172.21.15.39:0/650491242 >> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] conn(0x555639b1b350 0x555639f4dda0 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=588fa6729bb41609 server_cookie=23e35657b4e663e7 in_seq=0 out_seq=0 2024-09-21T03:50:17.731 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:17.725+0000 7ffb32ffd640 10 client.16566 ms_handle_connect on v2:172.21.15.150:6838/2949386907 2024-09-21T03:50:17.733 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:17.726+0000 7ffb32ffd640 1 -- 172.21.15.39:0/650491242 <== mds.0 v2:172.21.15.150:6838/2949386907 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x555639e36860 con 0x555639b1b350 2024-09-21T03:50:17.733 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:17.726+0000 7ffb32ffd640 10 client.16566 handle_command_reply: tid=0 2024-09-21T03:50:17.733 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:17.727+0000 7ffb6048f740 1 client.16566 shutdown 2024-09-21T03:50:17.733 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:17.727+0000 7ffb6048f740 1 -- 172.21.15.39:0/650491242 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x555639ad9cd0 msgr2=0x555639ed6330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:50:17.733 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:17.727+0000 7ffb6048f740 1 --2- 172.21.15.39:0/650491242 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x555639ad9cd0 0x555639ed6330 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7ffb4400b760 tx=0x7ffb440048d0 comp rx=0 tx=0).stop 2024-09-21T03:50:17.733 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:17.727+0000 7ffb6048f740 1 -- 172.21.15.39:0/650491242 shutdown_connections 2024-09-21T03:50:17.733 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:17.727+0000 7ffb6048f740 1 --2- 172.21.15.39:0/650491242 >> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] conn(0x555639b1b350 0x555639f4dda0 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:17.733 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:17.727+0000 7ffb6048f740 1 --2- 172.21.15.39:0/650491242 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555639afa670 0x555639ed2d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:17.733 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:17.727+0000 7ffb6048f740 1 --2- 172.21.15.39:0/650491242 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555639af2490 0x555639ed27f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:17.733 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:17.727+0000 7ffb6048f740 1 --2- 172.21.15.39:0/650491242 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x555639ad9cd0 0x555639ed6330 unknown :-1 s=CLOSED pgs=768 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:17.733 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:17.727+0000 7ffb6048f740 1 -- 172.21.15.39:0/650491242 >> 172.21.15.39:0/650491242 conn(0x555639b0f740 msgr2=0x555639ecfc70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:50:17.733 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:17.727+0000 7ffb32ffd640 0 client.16566 ms_handle_reset on v2:172.21.15.150:6838/2949386907 2024-09-21T03:50:17.733 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:17.727+0000 7ffb6048f740 1 -- 172.21.15.39:0/650491242 shutdown_connections 2024-09-21T03:50:17.733 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:17.727+0000 7ffb6048f740 1 -- 172.21.15.39:0/650491242 wait complete. 2024-09-21T03:50:17.734 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:17.727+0000 7ffb6048f740 20 client.16566 trim_cache size 0 max 16384 2024-09-21T03:50:17.734 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-09-21T03:50:17.734 INFO:teuthology.orchestra.run.smithi039.stdout: "return_code": 0, 2024-09-21T03:50:17.734 INFO:teuthology.orchestra.run.smithi039.stdout: "scrub_tag": "74ab917e-ecea-4778-a1c3-c184214fb24c", 2024-09-21T03:50:17.734 INFO:teuthology.orchestra.run.smithi039.stdout: "mode": "asynchronous" 2024-09-21T03:50:17.734 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-09-21T03:50:17.735 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:17.728+0000 7ffb59f1b640 1 -- 172.21.15.39:0/1414293968 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7ffb400b6680 msgr2=0x7ffb400b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:50:17.735 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:17.728+0000 7ffb59f1b640 1 --2- 172.21.15.39:0/1414293968 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7ffb400b6680 0x7ffb400b8b90 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7ffb3c000c00 tx=0x7ffb3c005780 comp rx=0 tx=0).stop 2024-09-21T03:50:17.735 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:17.728+0000 7ffb59f1b640 1 -- 172.21.15.39:0/1414293968 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ffb54108d20 msgr2=0x7ffb5419fd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:50:17.735 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:17.728+0000 7ffb59f1b640 1 --2- 172.21.15.39:0/1414293968 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ffb54108d20 0x7ffb5419fd60 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7ffb4c00b970 tx=0x7ffb4c00be30 comp rx=0 tx=0).stop 2024-09-21T03:50:17.735 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:17.728+0000 7ffb59f1b640 1 -- 172.21.15.39:0/1414293968 shutdown_connections 2024-09-21T03:50:17.735 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:17.728+0000 7ffb59f1b640 1 --2- 172.21.15.39:0/1414293968 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7ffb400b6680 0x7ffb400b8b90 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:17.735 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:17.728+0000 7ffb59f1b640 1 --2- 172.21.15.39:0/1414293968 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ffb54108d20 0x7ffb5419fd60 unknown :-1 s=CLOSED pgs=764 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:17.735 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:17.728+0000 7ffb59f1b640 1 --2- 172.21.15.39:0/1414293968 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ffb541083d0 0x7ffb5419aac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:17.735 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:17.728+0000 7ffb59f1b640 1 --2- 172.21.15.39:0/1414293968 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ffb54102ae0 0x7ffb5419a520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:17.735 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:17.728+0000 7ffb59f1b640 1 -- 172.21.15.39:0/1414293968 >> 172.21.15.39:0/1414293968 conn(0x7ffb54079760 msgr2=0x7ffb54104460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:50:17.735 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:17.728+0000 7ffb59f1b640 1 -- 172.21.15.39:0/1414293968 shutdown_connections 2024-09-21T03:50:17.735 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:17.728+0000 7ffb59f1b640 1 -- 172.21.15.39:0/1414293968 wait complete. 2024-09-21T03:50:18.586 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 51116d5c-77ca-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub status 2024-09-21T03:50:18.848 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/51116d5c-77ca-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:50:19.218 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:50:18 smithi161 ceph-mon[34533]: pgmap v453: 97 pgs: 97 active+clean; 10 MiB data, 536 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 170 B/s wr, 9 op/s 2024-09-21T03:50:19.218 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:50:18 smithi161 ceph-mon[34533]: scrub queued for path: / 2024-09-21T03:50:19.218 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:50:18 smithi161 ceph-mon[34533]: scrub summary: idle+waiting paths [/] 2024-09-21T03:50:19.218 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:50:18 smithi161 ceph-mon[34533]: scrub summary: active paths [/] 2024-09-21T03:50:19.218 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:50:18 smithi161 ceph-mon[34533]: scrub summary: idle+waiting paths [/] 2024-09-21T03:50:19.224 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:50:18 smithi150 ceph-mon[34289]: pgmap v453: 97 pgs: 97 active+clean; 10 MiB data, 536 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 170 B/s wr, 9 op/s 2024-09-21T03:50:19.224 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:50:18 smithi150 ceph-mon[34289]: scrub queued for path: / 2024-09-21T03:50:19.225 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:50:18 smithi150 ceph-mon[34289]: scrub summary: idle+waiting paths [/] 2024-09-21T03:50:19.225 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:50:18 smithi150 ceph-mon[34289]: scrub summary: active paths [/] 2024-09-21T03:50:19.225 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:50:18 smithi150 ceph-mon[34289]: scrub summary: idle+waiting paths [/] 2024-09-21T03:50:19.246 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:50:18 smithi039 ceph-mon[31211]: pgmap v453: 97 pgs: 97 active+clean; 10 MiB data, 536 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 170 B/s wr, 9 op/s 2024-09-21T03:50:19.246 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:50:18 smithi039 ceph-mon[31211]: scrub queued for path: / 2024-09-21T03:50:19.246 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:50:18 smithi039 ceph-mon[31211]: scrub summary: idle+waiting paths [/] 2024-09-21T03:50:19.246 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:50:18 smithi039 ceph-mon[31211]: scrub summary: active paths [/] 2024-09-21T03:50:19.247 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:50:18 smithi039 ceph-mon[31211]: scrub summary: idle+waiting paths [/] 2024-09-21T03:50:19.347 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:19.338+0000 7fde953f7640 1 -- 172.21.15.39:0/3932091236 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fde90108da0 msgr2=0x7fde9010f5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:50:19.347 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:19.338+0000 7fde953f7640 1 --2- 172.21.15.39:0/3932091236 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fde90108da0 0x7fde9010f5d0 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7fde88009900 tx=0x7fde8802a410 comp rx=0 tx=0).stop 2024-09-21T03:50:19.348 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:19.338+0000 7fde953f7640 1 -- 172.21.15.39:0/3932091236 shutdown_connections 2024-09-21T03:50:19.348 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:19.338+0000 7fde953f7640 1 --2- 172.21.15.39:0/3932091236 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fde90108da0 0x7fde9010f5d0 unknown :-1 s=CLOSED pgs=769 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:19.348 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:19.338+0000 7fde953f7640 1 --2- 172.21.15.39:0/3932091236 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fde90108450 0x7fde90108860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:19.348 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:19.338+0000 7fde953f7640 1 --2- 172.21.15.39:0/3932091236 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fde90102b10 0x7fde90102f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:19.348 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:19.338+0000 7fde953f7640 1 -- 172.21.15.39:0/3932091236 >> 172.21.15.39:0/3932091236 conn(0x7fde900796d0 msgr2=0x7fde90079ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:50:19.348 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:19.338+0000 7fde953f7640 1 -- 172.21.15.39:0/3932091236 shutdown_connections 2024-09-21T03:50:19.348 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:19.338+0000 7fde953f7640 1 -- 172.21.15.39:0/3932091236 wait complete. 2024-09-21T03:50:19.348 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:19.339+0000 7fde953f7640 1 Processor -- start 2024-09-21T03:50:19.348 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:19.339+0000 7fde953f7640 1 -- start start 2024-09-21T03:50:19.348 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:19.339+0000 7fde953f7640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fde90102b10 0x7fde9019b310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:50:19.348 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:19.339+0000 7fde953f7640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fde90108450 0x7fde9019b850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:50:19.348 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:19.339+0000 7fde953f7640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fde90108da0 0x7fde9019bd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:50:19.349 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:19.339+0000 7fde953f7640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fde90112a60 con 0x7fde90102b10 2024-09-21T03:50:19.349 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:19.339+0000 7fde953f7640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fde901128e0 con 0x7fde90108450 2024-09-21T03:50:19.349 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:19.339+0000 7fde953f7640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fde90112be0 con 0x7fde90108da0 2024-09-21T03:50:19.349 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:19.339+0000 7fde8effd640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fde90102b10 0x7fde9019b310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:50:19.349 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:19.339+0000 7fde8effd640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fde90102b10 0x7fde9019b310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:44782/0 (socket says 172.21.15.39:44782) 2024-09-21T03:50:19.349 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:19.339+0000 7fde8effd640 1 -- 172.21.15.39:0/2648285129 learned_addr learned my addr 172.21.15.39:0/2648285129 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:50:19.349 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:19.339+0000 7fde8e7fc640 1 --2- 172.21.15.39:0/2648285129 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fde90108450 0x7fde9019b850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:50:19.349 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:19.339+0000 7fde8f7fe640 1 --2- 172.21.15.39:0/2648285129 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fde90108da0 0x7fde9019bd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:50:19.349 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:19.340+0000 7fde8effd640 1 -- 172.21.15.39:0/2648285129 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fde90108da0 msgr2=0x7fde9019bd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:50:19.349 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:19.340+0000 7fde8effd640 1 --2- 172.21.15.39:0/2648285129 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fde90108da0 0x7fde9019bd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:19.349 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:19.340+0000 7fde8effd640 1 -- 172.21.15.39:0/2648285129 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fde90108450 msgr2=0x7fde9019b850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:50:19.349 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:19.340+0000 7fde8effd640 1 --2- 172.21.15.39:0/2648285129 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fde90108450 0x7fde9019b850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:19.349 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:19.340+0000 7fde8effd640 1 -- 172.21.15.39:0/2648285129 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fde901a65a0 con 0x7fde90102b10 2024-09-21T03:50:19.350 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:19.340+0000 7fde8e7fc640 1 --2- 172.21.15.39:0/2648285129 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fde90108450 0x7fde9019b850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:50:19.350 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:19.340+0000 7fde8effd640 1 --2- 172.21.15.39:0/2648285129 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fde90102b10 0x7fde9019b310 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7fde8000db80 tx=0x7fde8000b630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:50:19.350 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:19.340+0000 7fde87fff640 1 -- 172.21.15.39:0/2648285129 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde80017070 con 0x7fde90102b10 2024-09-21T03:50:19.350 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:19.340+0000 7fde953f7640 1 -- 172.21.15.39:0/2648285129 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fde901a6890 con 0x7fde90102b10 2024-09-21T03:50:19.350 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:19.340+0000 7fde87fff640 1 -- 172.21.15.39:0/2648285129 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fde80003b70 con 0x7fde90102b10 2024-09-21T03:50:19.350 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:19.340+0000 7fde87fff640 1 -- 172.21.15.39:0/2648285129 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde800106c0 con 0x7fde90102b10 2024-09-21T03:50:19.350 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:19.340+0000 7fde953f7640 1 -- 172.21.15.39:0/2648285129 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fde901a6dd0 con 0x7fde90102b10 2024-09-21T03:50:19.350 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:19.343+0000 7fde9b96b740 1 Processor -- start 2024-09-21T03:50:19.350 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:19.343+0000 7fde9b96b740 1 -- start start 2024-09-21T03:50:19.350 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:19.343+0000 7fde87fff640 1 -- 172.21.15.39:0/2648285129 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 27) ==== 154643+0+0 (secure 0 0 0) 0x7fde80004020 con 0x7fde90102b10 2024-09-21T03:50:19.350 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:19.343+0000 7fde9b96b740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564640495070 0x56464083c650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:50:19.350 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:19.343+0000 7fde9b96b740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56464046c1b0 0x56464083cb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:50:19.350 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:19.343+0000 7fde9b96b740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564640474390 0x56464083d0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:50:19.350 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:19.344+0000 7fde9b96b740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x5646407b2e50 con 0x564640474390 2024-09-21T03:50:19.351 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:19.344+0000 7fde9b96b740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x5646407a2280 con 0x564640495070 2024-09-21T03:50:19.351 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:19.344+0000 7fde9b96b740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x5646407a6090 con 0x56464046c1b0 2024-09-21T03:50:19.351 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:19.344+0000 7fde8effd640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56464046c1b0 0x56464083cb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:50:19.351 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:19.344+0000 7fde8e7fc640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564640474390 0x56464083d0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:50:19.351 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:19.344+0000 7fde8e7fc640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564640474390 0x56464083d0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:44794/0 (socket says 172.21.15.39:44794) 2024-09-21T03:50:19.351 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:19.344+0000 7fde8e7fc640 1 -- 172.21.15.39:0/3755191127 learned_addr learned my addr 172.21.15.39:0/3755191127 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:50:19.351 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:19.344+0000 7fde8f7fe640 1 --2- 172.21.15.39:0/3755191127 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564640495070 0x56464083c650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:50:19.351 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:19.344+0000 7fde8e7fc640 1 -- 172.21.15.39:0/3755191127 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56464046c1b0 msgr2=0x56464083cb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:50:19.351 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:19.344+0000 7fde8e7fc640 1 --2- 172.21.15.39:0/3755191127 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56464046c1b0 0x56464083cb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:19.351 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:19.344+0000 7fde8e7fc640 1 -- 172.21.15.39:0/3755191127 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564640495070 msgr2=0x56464083c650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:50:19.351 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:19.344+0000 7fde8e7fc640 1 --2- 172.21.15.39:0/3755191127 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564640495070 0x56464083c650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:19.351 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:19.344+0000 7fde8e7fc640 1 -- 172.21.15.39:0/3755191127 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564640500650 con 0x564640474390 2024-09-21T03:50:19.351 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:19.344+0000 7fde87fff640 1 --2- 172.21.15.39:0/2648285129 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7fde7c0b66b0 0x7fde7c0b8bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:50:19.351 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:19.344+0000 7fde8e7fc640 1 --2- 172.21.15.39:0/3755191127 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564640474390 0x56464083d0d0 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7fde7800bc60 tx=0x7fde7800f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3bf4d5189dd22507 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:50:19.352 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:19.345+0000 7fde8f7fe640 1 --2- 172.21.15.39:0/2648285129 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7fde7c0b66b0 0x7fde7c0b8bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:50:19.352 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:19.345+0000 7fde857fa640 1 -- 172.21.15.39:0/3755191127 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde78015070 con 0x564640474390 2024-09-21T03:50:19.352 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:19.345+0000 7fde87fff640 1 -- 172.21.15.39:0/2648285129 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8484+0+0 (secure 0 0 0) 0x7fde800e75c0 con 0x7fde90102b10 2024-09-21T03:50:19.352 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:19.345+0000 7fde857fa640 1 -- 172.21.15.39:0/3755191127 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fde780054d0 con 0x564640474390 2024-09-21T03:50:19.352 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:19.345+0000 7fde857fa640 1 -- 172.21.15.39:0/3755191127 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde78011440 con 0x564640474390 2024-09-21T03:50:19.355 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:19.346+0000 7fde9b96b740 1 -- 172.21.15.39:0/3755191127 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564640474390 msgr2=0x56464083d0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:50:19.355 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:19.346+0000 7fde8f7fe640 1 --2- 172.21.15.39:0/2648285129 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7fde7c0b66b0 0x7fde7c0b8bc0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fde88005ab0 tx=0x7fde88005a40 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:50:19.355 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:19.346+0000 7fde9b96b740 1 --2- 172.21.15.39:0/3755191127 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564640474390 0x56464083d0d0 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7fde7800bc60 tx=0x7fde7800f730 comp rx=0 tx=0).stop 2024-09-21T03:50:19.355 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:19.346+0000 7fde9b96b740 1 -- 172.21.15.39:0/3755191127 shutdown_connections 2024-09-21T03:50:19.356 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:19.346+0000 7fde9b96b740 1 --2- 172.21.15.39:0/3755191127 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564640474390 0x56464083d0d0 unknown :-1 s=CLOSED pgs=771 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:19.356 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:19.346+0000 7fde9b96b740 1 --2- 172.21.15.39:0/3755191127 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56464046c1b0 0x56464083cb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:19.356 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:19.346+0000 7fde9b96b740 1 --2- 172.21.15.39:0/3755191127 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564640495070 0x56464083c650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:19.356 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:19.346+0000 7fde9b96b740 1 -- 172.21.15.39:0/3755191127 >> 172.21.15.39:0/3755191127 conn(0x564640489460 msgr2=0x56464083a540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:50:19.356 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:19.346+0000 7fde9b96b740 1 -- 172.21.15.39:0/3755191127 shutdown_connections 2024-09-21T03:50:19.356 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:19.346+0000 7fde9b96b740 1 -- 172.21.15.39:0/3755191127 wait complete. 2024-09-21T03:50:19.356 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:19.346+0000 7fde9b96b740 1 -- start start 2024-09-21T03:50:19.356 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:19.347+0000 7fde9b96b740 1 Processor -- start 2024-09-21T03:50:19.356 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:19.347+0000 7fde9b96b740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56464046c1b0 0x56464084c090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:50:19.356 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:19.347+0000 7fde9b96b740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564640474390 0x56464084c5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:50:19.356 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:19.347+0000 7fde9b96b740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564640495070 0x56464084cb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:50:19.356 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:19.347+0000 7fde9b96b740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x564640603700 con 0x56464046c1b0 2024-09-21T03:50:19.356 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:19.347+0000 7fde9b96b740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x56464073d110 con 0x564640474390 2024-09-21T03:50:19.356 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:19.347+0000 7fde9b96b740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x56464084f190 con 0x564640495070 2024-09-21T03:50:19.357 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:19.347+0000 7fde9b96b740 10 client.0 _ll_register_callbacks cb 0x564640756990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:50:19.357 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:19.347+0000 7fde8effd640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56464046c1b0 0x56464084c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:50:19.357 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:19.347+0000 7fde8effd640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56464046c1b0 0x56464084c090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:44802/0 (socket says 172.21.15.39:44802) 2024-09-21T03:50:19.357 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:19.347+0000 7fde8effd640 1 -- 172.21.15.39:0/855436487 learned_addr learned my addr 172.21.15.39:0/855436487 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:50:19.357 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:19.347+0000 7fde8e7fc640 1 --2- 172.21.15.39:0/855436487 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564640474390 0x56464084c5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:50:19.357 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:19.347+0000 7fde8f7fe640 1 --2- 172.21.15.39:0/855436487 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564640495070 0x56464084cb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:50:19.357 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:19.347+0000 7fde8effd640 1 -- 172.21.15.39:0/855436487 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564640495070 msgr2=0x56464084cb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:50:19.357 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:19.347+0000 7fde8effd640 1 --2- 172.21.15.39:0/855436487 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564640495070 0x56464084cb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:19.357 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:19.347+0000 7fde8effd640 1 -- 172.21.15.39:0/855436487 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564640474390 msgr2=0x56464084c5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:50:19.357 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:19.347+0000 7fde8effd640 1 --2- 172.21.15.39:0/855436487 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564640474390 0x56464084c5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:19.357 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:19.347+0000 7fde8effd640 1 -- 172.21.15.39:0/855436487 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56464076ac40 con 0x56464046c1b0 2024-09-21T03:50:19.357 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:19.347+0000 7fde8e7fc640 1 --2- 172.21.15.39:0/855436487 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564640474390 0x56464084c5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:50:19.357 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:19.348+0000 7fde8effd640 1 --2- 172.21.15.39:0/855436487 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56464046c1b0 0x56464084c090 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7fde800e4d90 tx=0x7fde80004850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=93ecab8c5e34df64 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:50:19.357 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:19.348+0000 7fde767fc640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-09-21T03:50:19.358 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:19.348+0000 7fde767fc640 1 -- 172.21.15.39:0/855436487 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde800ad200 con 0x56464046c1b0 2024-09-21T03:50:19.358 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:19.348+0000 7fde767fc640 1 -- 172.21.15.39:0/855436487 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8484+0+0 (secure 0 0 0) 0x7fde800ac690 con 0x56464046c1b0 2024-09-21T03:50:19.358 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:19.348+0000 7fde9b96b740 1 -- 172.21.15.39:0/855436487 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fde780054d0 con 0x56464046c1b0 2024-09-21T03:50:19.358 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:19.348+0000 7fde767fc640 1 -- 172.21.15.39:0/855436487 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fde8000be60 con 0x56464046c1b0 2024-09-21T03:50:19.358 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:19.348+0000 7fde9b96b740 10 client.16578 fetch_fsmap learned FSMap version 29 2024-09-21T03:50:19.358 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:19.348+0000 7fde9b96b740 1 -- 172.21.15.39:0/855436487 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x56464085ae50 con 0x56464046c1b0 2024-09-21T03:50:19.358 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:19.348+0000 7fde767fc640 1 -- 172.21.15.39:0/855436487 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fde780054d0 con 0x56464046c1b0 2024-09-21T03:50:19.358 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:19.348+0000 7fde767fc640 1 -- 172.21.15.39:0/855436487 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fde80094b70 con 0x56464046c1b0 2024-09-21T03:50:19.358 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:19.348+0000 7fde9b96b740 10 client.16578 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T03:50:19.358 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:19.348+0000 7fde9b96b740 10 client.16578 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T03:50:19.358 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:19.348+0000 7fde9b96b740 20 client.16578 populate_metadata read hostname 'smithi039' 2024-09-21T03:50:19.358 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:19.348+0000 7fde9b96b740 1 --2- 172.21.15.39:0/855436487 >> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] conn(0x5646404539f0 0x56464085e2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:50:19.358 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:19.348+0000 7fde9b96b740 4 client.16578 mds_command: new command op to 24695 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:50:19.359 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:19.348+0000 7fde9b96b740 1 -- 172.21.15.39:0/855436487 --> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5646407b4da0 con 0x5646404539f0 2024-09-21T03:50:19.359 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:19.349+0000 7fde8e7fc640 1 --2- 172.21.15.39:0/855436487 >> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] conn(0x5646404539f0 0x56464085e2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:50:19.359 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:19.349+0000 7fde8e7fc640 1 --2- 172.21.15.39:0/855436487 >> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] conn(0x5646404539f0 0x56464085e2d0 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=909bf3e71ae615a6 server_cookie=2e192741b25c9004 in_seq=0 out_seq=0 2024-09-21T03:50:19.359 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:19.349+0000 7fde767fc640 10 client.16578 ms_handle_connect on v2:172.21.15.150:6838/2949386907 2024-09-21T03:50:19.359 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:19.350+0000 7fde767fc640 1 -- 172.21.15.39:0/855436487 <== mds.0 v2:172.21.15.150:6838/2949386907 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5646407b4da0 con 0x5646404539f0 2024-09-21T03:50:19.359 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:19.350+0000 7fde767fc640 10 client.16578 handle_command_reply: tid=0 2024-09-21T03:50:19.359 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:19.350+0000 7fde9b96b740 1 client.16578 shutdown 2024-09-21T03:50:19.359 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:19.350+0000 7fde9b96b740 1 -- 172.21.15.39:0/855436487 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56464046c1b0 msgr2=0x56464084c090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:50:19.359 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:19.350+0000 7fde9b96b740 1 --2- 172.21.15.39:0/855436487 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56464046c1b0 0x56464084c090 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7fde800e4d90 tx=0x7fde80004850 comp rx=0 tx=0).stop 2024-09-21T03:50:19.359 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:19.350+0000 7fde9b96b740 1 -- 172.21.15.39:0/855436487 shutdown_connections 2024-09-21T03:50:19.359 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:19.350+0000 7fde9b96b740 1 --2- 172.21.15.39:0/855436487 >> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] conn(0x5646404539f0 0x56464085e2d0 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:19.359 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:19.350+0000 7fde9b96b740 1 --2- 172.21.15.39:0/855436487 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564640495070 0x56464084cb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:19.359 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:19.350+0000 7fde9b96b740 1 --2- 172.21.15.39:0/855436487 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564640474390 0x56464084c5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:19.359 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:19.350+0000 7fde767fc640 0 client.16578 ms_handle_reset on v2:172.21.15.150:6838/2949386907 2024-09-21T03:50:19.360 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:19.350+0000 7fde9b96b740 1 --2- 172.21.15.39:0/855436487 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56464046c1b0 0x56464084c090 unknown :-1 s=CLOSED pgs=772 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:19.360 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:19.350+0000 7fde9b96b740 1 -- 172.21.15.39:0/855436487 >> 172.21.15.39:0/855436487 conn(0x564640489460 msgr2=0x564640838b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:50:19.360 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:19.350+0000 7fde9b96b740 1 -- 172.21.15.39:0/855436487 shutdown_connections 2024-09-21T03:50:19.360 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:19.350+0000 7fde9b96b740 1 -- 172.21.15.39:0/855436487 wait complete. 2024-09-21T03:50:19.360 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:19.351+0000 7fde9b96b740 20 client.16578 trim_cache size 0 max 16384 2024-09-21T03:50:19.371 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:19.368+0000 7fde9b96b740 1 Processor -- start 2024-09-21T03:50:19.371 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:19.368+0000 7fde9b96b740 1 -- start start 2024-09-21T03:50:19.372 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:19.368+0000 7fde9b96b740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5646404539f0 0x564640839fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:50:19.372 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:19.368+0000 7fde9b96b740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56464046c1b0 0x56464084dcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:50:19.372 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:19.368+0000 7fde9b96b740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564640474390 0x564640849280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:50:19.372 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:19.368+0000 7fde9b96b740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fde901ad510 con 0x56464046c1b0 2024-09-21T03:50:19.372 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:19.368+0000 7fde9b96b740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fde901af050 con 0x5646404539f0 2024-09-21T03:50:19.372 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:19.368+0000 7fde9b96b740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fde901adbe0 con 0x564640474390 2024-09-21T03:50:19.373 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:19.369+0000 7fde8e7fc640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56464046c1b0 0x56464084dcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:50:19.373 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:19.369+0000 7fde8e7fc640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56464046c1b0 0x56464084dcc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:44816/0 (socket says 172.21.15.39:44816) 2024-09-21T03:50:19.373 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:19.369+0000 7fde8e7fc640 1 -- 172.21.15.39:0/1714868437 learned_addr learned my addr 172.21.15.39:0/1714868437 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:50:19.373 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:19.369+0000 7fde8effd640 1 --2- 172.21.15.39:0/1714868437 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5646404539f0 0x564640839fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:50:19.373 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:19.369+0000 7fde8e7fc640 1 -- 172.21.15.39:0/1714868437 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564640474390 msgr2=0x564640849280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:50:19.373 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:19.369+0000 7fde8f7fe640 1 --2- 172.21.15.39:0/1714868437 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564640474390 0x564640849280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:50:19.373 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:19.369+0000 7fde8e7fc640 1 --2- 172.21.15.39:0/1714868437 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564640474390 0x564640849280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:19.373 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:19.369+0000 7fde8e7fc640 1 -- 172.21.15.39:0/1714868437 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5646404539f0 msgr2=0x564640839fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:50:19.373 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:19.369+0000 7fde8e7fc640 1 --2- 172.21.15.39:0/1714868437 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5646404539f0 0x564640839fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:19.373 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:19.369+0000 7fde8e7fc640 1 -- 172.21.15.39:0/1714868437 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56464084aff0 con 0x56464046c1b0 2024-09-21T03:50:19.373 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:19.369+0000 7fde8effd640 1 --2- 172.21.15.39:0/1714868437 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5646404539f0 0x564640839fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:50:19.373 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:19.369+0000 7fde8e7fc640 1 --2- 172.21.15.39:0/1714868437 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56464046c1b0 0x56464084dcc0 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7fde780057c0 tx=0x7fde78004a70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c4b51c0409b37f6b server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:50:19.373 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:19.369+0000 7fde84ff9640 1 -- 172.21.15.39:0/1714868437 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde78031070 con 0x56464046c1b0 2024-09-21T03:50:19.373 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:19.369+0000 7fde84ff9640 1 -- 172.21.15.39:0/1714868437 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fde7800b3a0 con 0x56464046c1b0 2024-09-21T03:50:19.374 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:19.369+0000 7fde84ff9640 1 -- 172.21.15.39:0/1714868437 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde78015040 con 0x56464046c1b0 2024-09-21T03:50:19.377 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:19.370+0000 7fde9b96b740 1 -- 172.21.15.39:0/1714868437 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56464046c1b0 msgr2=0x56464084dcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:50:19.377 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:19.370+0000 7fde9b96b740 1 --2- 172.21.15.39:0/1714868437 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56464046c1b0 0x56464084dcc0 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7fde780057c0 tx=0x7fde78004a70 comp rx=0 tx=0).stop 2024-09-21T03:50:19.377 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:19.370+0000 7fde9b96b740 1 -- 172.21.15.39:0/1714868437 shutdown_connections 2024-09-21T03:50:19.377 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:19.370+0000 7fde9b96b740 1 --2- 172.21.15.39:0/1714868437 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564640474390 0x564640849280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:19.377 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:19.370+0000 7fde9b96b740 1 --2- 172.21.15.39:0/1714868437 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56464046c1b0 0x56464084dcc0 unknown :-1 s=CLOSED pgs=773 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:19.378 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:19.370+0000 7fde9b96b740 1 --2- 172.21.15.39:0/1714868437 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5646404539f0 0x564640839fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:19.378 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:19.370+0000 7fde9b96b740 1 -- 172.21.15.39:0/1714868437 >> 172.21.15.39:0/1714868437 conn(0x564640489460 msgr2=0x56464084d5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:50:19.378 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:19.370+0000 7fde9b96b740 1 -- 172.21.15.39:0/1714868437 shutdown_connections 2024-09-21T03:50:19.378 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:19.370+0000 7fde9b96b740 1 -- 172.21.15.39:0/1714868437 wait complete. 2024-09-21T03:50:19.378 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:19.371+0000 7fde9b96b740 1 -- start start 2024-09-21T03:50:19.378 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:19.371+0000 7fde9b96b740 1 Processor -- start 2024-09-21T03:50:19.378 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:19.371+0000 7fde9b96b740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5646404539f0 0x5646408508a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:50:19.378 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:19.371+0000 7fde9b96b740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56464046c1b0 0x564640850e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:50:19.378 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:19.371+0000 7fde9b96b740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564640474390 0x5646408c7c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:50:19.378 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:19.371+0000 7fde9b96b740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fde901ae980 con 0x5646404539f0 2024-09-21T03:50:19.378 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:19.371+0000 7fde9b96b740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fde901ae2b0 con 0x56464046c1b0 2024-09-21T03:50:19.378 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:19.371+0000 7fde9b96b740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fde901ab4f0 con 0x564640474390 2024-09-21T03:50:19.378 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:19.371+0000 7fde9b96b740 10 client.0 _ll_register_callbacks cb 0x7fde800099c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:50:19.379 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:19.371+0000 7fde8effd640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5646404539f0 0x5646408508a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:50:19.379 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:19.371+0000 7fde8effd640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5646404539f0 0x5646408508a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:44824/0 (socket says 172.21.15.39:44824) 2024-09-21T03:50:19.379 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:19.371+0000 7fde8effd640 1 -- 172.21.15.39:0/90147863 learned_addr learned my addr 172.21.15.39:0/90147863 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:50:19.379 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:19.371+0000 7fde8e7fc640 1 --2- 172.21.15.39:0/90147863 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56464046c1b0 0x564640850e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:50:19.379 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:19.371+0000 7fde8f7fe640 1 --2- 172.21.15.39:0/90147863 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564640474390 0x5646408c7c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:50:19.379 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:19.372+0000 7fde8effd640 1 -- 172.21.15.39:0/90147863 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564640474390 msgr2=0x5646408c7c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:50:19.379 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:19.372+0000 7fde8effd640 1 --2- 172.21.15.39:0/90147863 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564640474390 0x5646408c7c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:19.379 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:19.372+0000 7fde8effd640 1 -- 172.21.15.39:0/90147863 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56464046c1b0 msgr2=0x564640850e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:50:19.379 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:19.372+0000 7fde8effd640 1 --2- 172.21.15.39:0/90147863 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56464046c1b0 0x564640850e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:19.379 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:19.372+0000 7fde8effd640 1 -- 172.21.15.39:0/90147863 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5646408c81d0 con 0x5646404539f0 2024-09-21T03:50:19.379 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:19.372+0000 7fde8f7fe640 1 --2- 172.21.15.39:0/90147863 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564640474390 0x5646408c7c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:50:19.379 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:19.372+0000 7fde8effd640 1 --2- 172.21.15.39:0/90147863 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5646404539f0 0x5646408508a0 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7fde800ad3c0 tx=0x7fde800ac8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c0c259efac36e3f1 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:50:19.379 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:19.372+0000 7fde767fc640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-09-21T03:50:19.379 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:19.372+0000 7fde767fc640 1 -- 172.21.15.39:0/90147863 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde800d4050 con 0x5646404539f0 2024-09-21T03:50:19.379 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:19.372+0000 7fde767fc640 1 -- 172.21.15.39:0/90147863 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8484+0+0 (secure 0 0 0) 0x7fde80058dc0 con 0x5646404539f0 2024-09-21T03:50:19.379 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:19.372+0000 7fde9b96b740 1 -- 172.21.15.39:0/90147863 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fde7800b3a0 con 0x5646404539f0 2024-09-21T03:50:19.380 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:19.372+0000 7fde767fc640 1 -- 172.21.15.39:0/90147863 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fde800acba0 con 0x5646404539f0 2024-09-21T03:50:19.380 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:19.372+0000 7fde9b96b740 10 client.16584 fetch_fsmap learned FSMap version 29 2024-09-21T03:50:19.380 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:19.372+0000 7fde9b96b740 1 -- 172.21.15.39:0/90147863 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5646408c92d0 con 0x5646404539f0 2024-09-21T03:50:19.380 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:19.372+0000 7fde767fc640 1 -- 172.21.15.39:0/90147863 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fde7800b3a0 con 0x5646404539f0 2024-09-21T03:50:19.380 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:19.373+0000 7fde767fc640 1 -- 172.21.15.39:0/90147863 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fde800a8d80 con 0x5646404539f0 2024-09-21T03:50:19.380 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:19.373+0000 7fde9b96b740 10 client.16584 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T03:50:19.380 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:19.373+0000 7fde9b96b740 10 client.16584 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T03:50:19.380 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:19.373+0000 7fde9b96b740 20 client.16584 populate_metadata read hostname 'smithi039' 2024-09-21T03:50:19.380 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:19.373+0000 7fde9b96b740 1 --2- 172.21.15.39:0/90147863 >> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] conn(0x564640495070 0x5646408cb900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:50:19.380 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:19.373+0000 7fde9b96b740 4 client.16584 mds_command: new command op to 24695 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-21T03:50:19.380 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:19.373+0000 7fde9b96b740 1 -- 172.21.15.39:0/90147863 --> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5646407b0850 con 0x564640495070 2024-09-21T03:50:19.380 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:19.373+0000 7fde8e7fc640 1 --2- 172.21.15.39:0/90147863 >> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] conn(0x564640495070 0x5646408cb900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:50:19.380 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:19.373+0000 7fde767fc640 10 client.16584 ms_handle_connect on v2:172.21.15.150:6838/2949386907 2024-09-21T03:50:19.380 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:19.373+0000 7fde8e7fc640 1 --2- 172.21.15.39:0/90147863 >> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] conn(0x564640495070 0x5646408cb900 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=36a5b7de18fe7b86 server_cookie=5d117903d7fb6d38 in_seq=0 out_seq=0 2024-09-21T03:50:19.380 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:19.374+0000 7fde767fc640 1 -- 172.21.15.39:0/90147863 <== mds.0 v2:172.21.15.150:6838/2949386907 1 ==== command_reply(tid 0: 0 ) ==== 8+0+2558 (crc 0 0 0) 0x5646407b0850 con 0x564640495070 2024-09-21T03:50:19.380 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:19.374+0000 7fde767fc640 10 client.16584 handle_command_reply: tid=0 2024-09-21T03:50:19.381 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:19.374+0000 7fde9b96b740 1 client.16584 shutdown 2024-09-21T03:50:19.381 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:19.374+0000 7fde9b96b740 1 -- 172.21.15.39:0/90147863 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5646404539f0 msgr2=0x5646408508a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:50:19.381 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:19.374+0000 7fde9b96b740 1 --2- 172.21.15.39:0/90147863 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5646404539f0 0x5646408508a0 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7fde800ad3c0 tx=0x7fde800ac8c0 comp rx=0 tx=0).stop 2024-09-21T03:50:19.381 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:19.374+0000 7fde9b96b740 1 -- 172.21.15.39:0/90147863 shutdown_connections 2024-09-21T03:50:19.381 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:19.374+0000 7fde9b96b740 1 --2- 172.21.15.39:0/90147863 >> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] conn(0x564640495070 0x5646408cb900 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:19.381 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:19.374+0000 7fde9b96b740 1 --2- 172.21.15.39:0/90147863 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564640474390 0x5646408c7c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:19.381 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:19.374+0000 7fde9b96b740 1 --2- 172.21.15.39:0/90147863 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56464046c1b0 0x564640850e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:19.381 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:19.374+0000 7fde9b96b740 1 --2- 172.21.15.39:0/90147863 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5646404539f0 0x5646408508a0 unknown :-1 s=CLOSED pgs=774 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:19.381 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:19.374+0000 7fde9b96b740 1 -- 172.21.15.39:0/90147863 >> 172.21.15.39:0/90147863 conn(0x564640489460 msgr2=0x5646408417f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:50:19.381 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:19.374+0000 7fde767fc640 0 client.16584 ms_handle_reset on v2:172.21.15.150:6838/2949386907 2024-09-21T03:50:19.381 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:19.374+0000 7fde9b96b740 1 -- 172.21.15.39:0/90147863 shutdown_connections 2024-09-21T03:50:19.381 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:19.374+0000 7fde9b96b740 1 -- 172.21.15.39:0/90147863 wait complete. 2024-09-21T03:50:19.381 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:19.374+0000 7fde9b96b740 20 client.16584 trim_cache size 0 max 16384 2024-09-21T03:50:19.382 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-09-21T03:50:19.382 INFO:teuthology.orchestra.run.smithi039.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-21T03:50:19.382 INFO:teuthology.orchestra.run.smithi039.stdout: "scrubs": { 2024-09-21T03:50:19.382 INFO:teuthology.orchestra.run.smithi039.stdout: "74ab917e-ecea-4778-a1c3-c184214fb24c": { 2024-09-21T03:50:19.382 INFO:teuthology.orchestra.run.smithi039.stdout: "path": "/", 2024-09-21T03:50:19.382 INFO:teuthology.orchestra.run.smithi039.stdout: "tag": "74ab917e-ecea-4778-a1c3-c184214fb24c", 2024-09-21T03:50:19.382 INFO:teuthology.orchestra.run.smithi039.stdout: "options": "recursive,force" 2024-09-21T03:50:19.382 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-21T03:50:19.382 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:50:19.382 INFO:teuthology.orchestra.run.smithi039.stdout: "scrub_stats": { 2024-09-21T03:50:19.382 INFO:teuthology.orchestra.run.smithi039.stdout: "74ab917e-ecea-4778-a1c3-c184214fb24c": { 2024-09-21T03:50:19.382 INFO:teuthology.orchestra.run.smithi039.stdout: "start_time": "2024-09-21T03:50:17.726630+0000", 2024-09-21T03:50:19.383 INFO:teuthology.orchestra.run.smithi039.stdout: "path": "/", 2024-09-21T03:50:19.383 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_started": 0, 2024-09-21T03:50:19.383 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_passed": 0, 2024-09-21T03:50:19.383 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_failed": 0, 2024-09-21T03:50:19.383 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_skipped": 0 2024-09-21T03:50:19.383 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:50:19.383 INFO:teuthology.orchestra.run.smithi039.stdout: "6e760f1f-9048-4f92-bd66-d708fbef75b3": { 2024-09-21T03:50:19.383 INFO:teuthology.orchestra.run.smithi039.stdout: "start_time": "2024-09-21T03:50:03.711918+0000", 2024-09-21T03:50:19.383 INFO:teuthology.orchestra.run.smithi039.stdout: "path": "/", 2024-09-21T03:50:19.383 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_started": 0, 2024-09-21T03:50:19.383 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_passed": 0, 2024-09-21T03:50:19.383 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_failed": 0, 2024-09-21T03:50:19.383 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_skipped": 0 2024-09-21T03:50:19.383 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:50:19.384 INFO:teuthology.orchestra.run.smithi039.stdout: "e28ad743-2b22-45b3-8799-a638c2883623": { 2024-09-21T03:50:19.384 INFO:teuthology.orchestra.run.smithi039.stdout: "start_time": "2024-09-21T03:48:31.448821+0000", 2024-09-21T03:50:19.384 INFO:teuthology.orchestra.run.smithi039.stdout: "path": "/", 2024-09-21T03:50:19.384 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_started": 0, 2024-09-21T03:50:19.384 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_passed": 0, 2024-09-21T03:50:19.384 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_failed": 0, 2024-09-21T03:50:19.384 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_skipped": 0 2024-09-21T03:50:19.384 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:50:19.384 INFO:teuthology.orchestra.run.smithi039.stdout: "29411307-eb8d-4339-8d2b-47ebd32c0080": { 2024-09-21T03:50:19.384 INFO:teuthology.orchestra.run.smithi039.stdout: "start_time": "2024-09-21T03:47:38.092920+0000", 2024-09-21T03:50:19.384 INFO:teuthology.orchestra.run.smithi039.stdout: "path": "/", 2024-09-21T03:50:19.384 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_started": 0, 2024-09-21T03:50:19.384 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_passed": 0, 2024-09-21T03:50:19.384 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_failed": 0, 2024-09-21T03:50:19.384 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_skipped": 0 2024-09-21T03:50:19.384 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:50:19.385 INFO:teuthology.orchestra.run.smithi039.stdout: "9860c2f4-b7b1-47aa-865a-9a1c4434a5f6": { 2024-09-21T03:50:19.385 INFO:teuthology.orchestra.run.smithi039.stdout: "start_time": "2024-09-21T03:49:17.718867+0000", 2024-09-21T03:50:19.385 INFO:teuthology.orchestra.run.smithi039.stdout: "path": "/", 2024-09-21T03:50:19.385 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_started": 0, 2024-09-21T03:50:19.385 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_passed": 0, 2024-09-21T03:50:19.385 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_failed": 0, 2024-09-21T03:50:19.385 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_skipped": 0 2024-09-21T03:50:19.385 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:50:19.385 INFO:teuthology.orchestra.run.smithi039.stdout: "c7e2b8e4-057d-435f-b2bd-b633de98035f": { 2024-09-21T03:50:19.385 INFO:teuthology.orchestra.run.smithi039.stdout: "start_time": "2024-09-21T03:47:19.400305+0000", 2024-09-21T03:50:19.385 INFO:teuthology.orchestra.run.smithi039.stdout: "path": "/", 2024-09-21T03:50:19.385 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_started": 0, 2024-09-21T03:50:19.385 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_passed": 0, 2024-09-21T03:50:19.385 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_failed": 0, 2024-09-21T03:50:19.385 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_skipped": 0 2024-09-21T03:50:19.385 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:50:19.386 INFO:teuthology.orchestra.run.smithi039.stdout: "10728de8-c49d-4edc-be2b-639249255e94": { 2024-09-21T03:50:19.386 INFO:teuthology.orchestra.run.smithi039.stdout: "start_time": "2024-09-21T03:47:03.965623+0000", 2024-09-21T03:50:19.386 INFO:teuthology.orchestra.run.smithi039.stdout: "path": "/", 2024-09-21T03:50:19.386 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_started": 0, 2024-09-21T03:50:19.386 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_passed": 0, 2024-09-21T03:50:19.386 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_failed": 0, 2024-09-21T03:50:19.386 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_skipped": 0 2024-09-21T03:50:19.386 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:50:19.386 INFO:teuthology.orchestra.run.smithi039.stdout: "02509e4f-3911-4983-b16f-df8ded0b9998": { 2024-09-21T03:50:19.386 INFO:teuthology.orchestra.run.smithi039.stdout: "start_time": "2024-09-21T03:46:17.889576+0000", 2024-09-21T03:50:19.386 INFO:teuthology.orchestra.run.smithi039.stdout: "path": "/", 2024-09-21T03:50:19.386 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_started": 0, 2024-09-21T03:50:19.386 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_passed": 0, 2024-09-21T03:50:19.387 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_failed": 0, 2024-09-21T03:50:19.387 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_skipped": 0 2024-09-21T03:50:19.387 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-21T03:50:19.387 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-21T03:50:19.387 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-09-21T03:50:19.387 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:19.375+0000 7fde953f7640 1 -- 172.21.15.39:0/2648285129 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7fde7c0b66b0 msgr2=0x7fde7c0b8bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:50:19.387 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:19.375+0000 7fde953f7640 1 --2- 172.21.15.39:0/2648285129 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7fde7c0b66b0 0x7fde7c0b8bc0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fde88005ab0 tx=0x7fde88005a40 comp rx=0 tx=0).stop 2024-09-21T03:50:19.387 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:19.375+0000 7fde953f7640 1 -- 172.21.15.39:0/2648285129 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fde90102b10 msgr2=0x7fde9019b310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:50:19.388 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:19.375+0000 7fde953f7640 1 --2- 172.21.15.39:0/2648285129 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fde90102b10 0x7fde9019b310 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7fde8000db80 tx=0x7fde8000b630 comp rx=0 tx=0).stop 2024-09-21T03:50:19.388 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:19.375+0000 7fde953f7640 1 -- 172.21.15.39:0/2648285129 shutdown_connections 2024-09-21T03:50:19.388 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:19.375+0000 7fde953f7640 1 --2- 172.21.15.39:0/2648285129 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7fde7c0b66b0 0x7fde7c0b8bc0 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:19.388 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:19.375+0000 7fde953f7640 1 --2- 172.21.15.39:0/2648285129 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fde90108da0 0x7fde9019bd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:19.388 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:19.375+0000 7fde953f7640 1 --2- 172.21.15.39:0/2648285129 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fde90108450 0x7fde9019b850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:19.388 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:19.375+0000 7fde953f7640 1 --2- 172.21.15.39:0/2648285129 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fde90102b10 0x7fde9019b310 unknown :-1 s=CLOSED pgs=770 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:19.388 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:19.375+0000 7fde953f7640 1 -- 172.21.15.39:0/2648285129 >> 172.21.15.39:0/2648285129 conn(0x7fde900796d0 msgr2=0x7fde90106440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:50:19.388 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:19.375+0000 7fde953f7640 1 -- 172.21.15.39:0/2648285129 shutdown_connections 2024-09-21T03:50:19.388 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:19.375+0000 7fde953f7640 1 -- 172.21.15.39:0/2648285129 wait complete. 2024-09-21T03:50:20.218 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:50:19 smithi161 ceph-mon[34533]: scrub summary: idle 2024-09-21T03:50:20.224 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:50:19 smithi150 ceph-mon[34289]: scrub summary: idle 2024-09-21T03:50:20.260 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:50:19 smithi039 ceph-mon[31211]: scrub summary: idle 2024-09-21T03:50:20.294 INFO:tasks.cephfs.filesystem:scrub status for tag:74ab917e-ecea-4778-a1c3-c184214fb24c - {'path': '/', 'tag': '74ab917e-ecea-4778-a1c3-c184214fb24c', 'options': 'recursive,force'} 2024-09-21T03:50:21.218 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:50:20 smithi161 ceph-mon[34533]: pgmap v454: 97 pgs: 97 active+clean; 10 MiB data, 536 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 170 B/s wr, 11 op/s 2024-09-21T03:50:21.225 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:50:20 smithi150 ceph-mon[34289]: pgmap v454: 97 pgs: 97 active+clean; 10 MiB data, 536 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 170 B/s wr, 11 op/s 2024-09-21T03:50:21.334 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:50:20 smithi039 ceph-mon[31211]: pgmap v454: 97 pgs: 97 active+clean; 10 MiB data, 536 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 170 B/s wr, 11 op/s 2024-09-21T03:50:23.218 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:50:22 smithi161 ceph-mon[34533]: pgmap v455: 97 pgs: 97 active+clean; 10 MiB data, 536 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 85 B/s wr, 9 op/s 2024-09-21T03:50:23.224 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:50:22 smithi150 ceph-mon[34289]: pgmap v455: 97 pgs: 97 active+clean; 10 MiB data, 536 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 85 B/s wr, 9 op/s 2024-09-21T03:50:23.334 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:50:22 smithi039 ceph-mon[31211]: pgmap v455: 97 pgs: 97 active+clean; 10 MiB data, 536 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 85 B/s wr, 9 op/s 2024-09-21T03:50:25.218 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:50:24 smithi161 ceph-mon[34533]: pgmap v456: 97 pgs: 97 active+clean; 10 MiB data, 536 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 85 B/s wr, 10 op/s 2024-09-21T03:50:25.224 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:50:24 smithi150 ceph-mon[34289]: pgmap v456: 97 pgs: 97 active+clean; 10 MiB data, 536 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 85 B/s wr, 10 op/s 2024-09-21T03:50:25.334 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:50:24 smithi039 ceph-mon[31211]: pgmap v456: 97 pgs: 97 active+clean; 10 MiB data, 536 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 85 B/s wr, 10 op/s 2024-09-21T03:50:27.218 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:50:26 smithi161 ceph-mon[34533]: pgmap v457: 97 pgs: 97 active+clean; 10 MiB data, 536 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 10 op/s 2024-09-21T03:50:27.224 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:50:26 smithi150 ceph-mon[34289]: pgmap v457: 97 pgs: 97 active+clean; 10 MiB data, 536 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 10 op/s 2024-09-21T03:50:27.334 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:50:26 smithi039 ceph-mon[31211]: pgmap v457: 97 pgs: 97 active+clean; 10 MiB data, 536 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 10 op/s 2024-09-21T03:50:28.218 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:50:27 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:50:28.225 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:50:27 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:50:28.333 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:50:27 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:50:29.218 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:50:28 smithi161 ceph-mon[34533]: pgmap v458: 97 pgs: 97 active+clean; 10 MiB data, 536 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 8 op/s 2024-09-21T03:50:29.218 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:50:28 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:50:29.218 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:50:28 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:50:29.218 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:50:28 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:50:29.224 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:50:28 smithi150 ceph-mon[34289]: pgmap v458: 97 pgs: 97 active+clean; 10 MiB data, 536 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 8 op/s 2024-09-21T03:50:29.224 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:50:28 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:50:29.225 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:50:28 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:50:29.225 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:50:28 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:50:29.334 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:50:28 smithi039 ceph-mon[31211]: pgmap v458: 97 pgs: 97 active+clean; 10 MiB data, 536 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 8 op/s 2024-09-21T03:50:29.334 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:50:28 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:50:29.334 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:50:28 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:50:29.334 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:50:28 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:50:31.218 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:50:30 smithi161 ceph-mon[34533]: pgmap v459: 97 pgs: 97 active+clean; 10 MiB data, 536 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 11 op/s 2024-09-21T03:50:31.224 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:50:30 smithi150 ceph-mon[34289]: pgmap v459: 97 pgs: 97 active+clean; 10 MiB data, 536 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 11 op/s 2024-09-21T03:50:31.334 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:50:30 smithi039 ceph-mon[31211]: pgmap v459: 97 pgs: 97 active+clean; 10 MiB data, 536 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 11 op/s 2024-09-21T03:50:33.218 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:50:32 smithi161 ceph-mon[34533]: pgmap v460: 97 pgs: 97 active+clean; 10 MiB data, 536 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 9 op/s 2024-09-21T03:50:33.224 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:50:32 smithi150 ceph-mon[34289]: pgmap v460: 97 pgs: 97 active+clean; 10 MiB data, 536 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 9 op/s 2024-09-21T03:50:33.334 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:50:32 smithi039 ceph-mon[31211]: pgmap v460: 97 pgs: 97 active+clean; 10 MiB data, 536 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 9 op/s 2024-09-21T03:50:35.218 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:50:34 smithi161 ceph-mon[34533]: pgmap v461: 97 pgs: 97 active+clean; 10 MiB data, 536 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-09-21T03:50:35.224 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:50:34 smithi150 ceph-mon[34289]: pgmap v461: 97 pgs: 97 active+clean; 10 MiB data, 536 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-09-21T03:50:35.334 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:50:34 smithi039 ceph-mon[31211]: pgmap v461: 97 pgs: 97 active+clean; 10 MiB data, 536 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-09-21T03:50:37.218 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:50:36 smithi161 ceph-mon[34533]: pgmap v462: 97 pgs: 97 active+clean; 10 MiB data, 536 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 10 op/s 2024-09-21T03:50:37.224 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:50:36 smithi150 ceph-mon[34289]: pgmap v462: 97 pgs: 97 active+clean; 10 MiB data, 536 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 10 op/s 2024-09-21T03:50:37.334 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:50:36 smithi039 ceph-mon[31211]: pgmap v462: 97 pgs: 97 active+clean; 10 MiB data, 536 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 10 op/s 2024-09-21T03:50:39.218 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:50:38 smithi161 ceph-mon[34533]: pgmap v463: 97 pgs: 97 active+clean; 10 MiB data, 536 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-09-21T03:50:39.224 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:50:38 smithi150 ceph-mon[34289]: pgmap v463: 97 pgs: 97 active+clean; 10 MiB data, 536 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-09-21T03:50:39.334 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:50:38 smithi039 ceph-mon[31211]: pgmap v463: 97 pgs: 97 active+clean; 10 MiB data, 536 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-09-21T03:50:41.218 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:50:40 smithi161 ceph-mon[34533]: pgmap v464: 97 pgs: 97 active+clean; 10 MiB data, 536 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-09-21T03:50:41.224 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:50:40 smithi150 ceph-mon[34289]: pgmap v464: 97 pgs: 97 active+clean; 10 MiB data, 536 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-09-21T03:50:41.334 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:50:40 smithi039 ceph-mon[31211]: pgmap v464: 97 pgs: 97 active+clean; 10 MiB data, 536 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-09-21T03:50:43.333 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:50:42 smithi039 ceph-mon[31211]: pgmap v465: 97 pgs: 97 active+clean; 10 MiB data, 536 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-21T03:50:43.468 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:50:42 smithi161 ceph-mon[34533]: pgmap v465: 97 pgs: 97 active+clean; 10 MiB data, 536 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-21T03:50:43.474 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:50:42 smithi150 ceph-mon[34289]: pgmap v465: 97 pgs: 97 active+clean; 10 MiB data, 536 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-21T03:50:45.277 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:50:44 smithi150 ceph-mon[34289]: pgmap v466: 97 pgs: 97 active+clean; 10 MiB data, 536 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-09-21T03:50:45.334 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:50:44 smithi039 ceph-mon[31211]: pgmap v466: 97 pgs: 97 active+clean; 10 MiB data, 536 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-09-21T03:50:45.468 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:50:44 smithi161 ceph-mon[34533]: pgmap v466: 97 pgs: 97 active+clean; 10 MiB data, 536 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-09-21T03:50:47.334 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:50:46 smithi039 ceph-mon[31211]: pgmap v467: 97 pgs: 97 active+clean; 10 MiB data, 536 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 10 op/s 2024-09-21T03:50:47.468 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:50:46 smithi161 ceph-mon[34533]: pgmap v467: 97 pgs: 97 active+clean; 10 MiB data, 536 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 10 op/s 2024-09-21T03:50:47.474 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:50:46 smithi150 ceph-mon[34289]: pgmap v467: 97 pgs: 97 active+clean; 10 MiB data, 536 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 10 op/s 2024-09-21T03:50:49.334 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:50:48 smithi039 ceph-mon[31211]: pgmap v468: 97 pgs: 97 active+clean; 10 MiB data, 536 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-09-21T03:50:49.468 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:50:48 smithi161 ceph-mon[34533]: pgmap v468: 97 pgs: 97 active+clean; 10 MiB data, 536 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-09-21T03:50:49.474 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:50:48 smithi150 ceph-mon[34289]: pgmap v468: 97 pgs: 97 active+clean; 10 MiB data, 536 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-09-21T03:50:50.294 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 51116d5c-77ca-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub status 2024-09-21T03:50:50.569 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/51116d5c-77ca-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:50:51.244 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:50:50 smithi039 ceph-mon[31211]: pgmap v469: 97 pgs: 97 active+clean; 10 MiB data, 536 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-09-21T03:50:51.246 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:51.241+0000 7fd001752640 1 -- 172.21.15.39:0/2889802142 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcffc10de60 msgr2=0x7fcffc110300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:50:51.246 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:51.241+0000 7fd001752640 1 --2- 172.21.15.39:0/2889802142 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcffc10de60 0x7fcffc110300 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7fcff4009ca0 tx=0x7fcff402a8c0 comp rx=0 tx=0).stop 2024-09-21T03:50:51.246 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:51.241+0000 7fd001752640 1 -- 172.21.15.39:0/2889802142 shutdown_connections 2024-09-21T03:50:51.247 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:51.241+0000 7fd001752640 1 --2- 172.21.15.39:0/2889802142 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcffc10de60 0x7fcffc110300 unknown :-1 s=CLOSED pgs=775 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:51.247 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:51.241+0000 7fd001752640 1 --2- 172.21.15.39:0/2889802142 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcffc1052e0 0x7fcffc10d920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:51.247 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:51.241+0000 7fd001752640 1 --2- 172.21.15.39:0/2889802142 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcffc1048b0 0x7fcffc104cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:51.247 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:51.241+0000 7fd001752640 1 -- 172.21.15.39:0/2889802142 >> 172.21.15.39:0/2889802142 conn(0x7fcffc079730 msgr2=0x7fcffc079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:50:51.247 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:51.241+0000 7fd001752640 1 -- 172.21.15.39:0/2889802142 shutdown_connections 2024-09-21T03:50:51.247 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:51.241+0000 7fd001752640 1 -- 172.21.15.39:0/2889802142 wait complete. 2024-09-21T03:50:51.247 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:51.242+0000 7fd001752640 1 Processor -- start 2024-09-21T03:50:51.247 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:51.242+0000 7fd001752640 1 -- start start 2024-09-21T03:50:51.247 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:51.242+0000 7fd001752640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcffc1048b0 0x7fcffc1a1750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:50:51.247 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:51.242+0000 7fd001752640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcffc1052e0 0x7fcffc1a1c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:50:51.247 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:51.242+0000 7fd001752640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcffc10de60 0x7fcffc1ac7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:50:51.247 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:51.242+0000 7fd001752640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fcffc113a70 con 0x7fcffc1052e0 2024-09-21T03:50:51.248 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:51.242+0000 7fd001752640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fcffc1138f0 con 0x7fcffc1048b0 2024-09-21T03:50:51.248 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:51.242+0000 7fd001752640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fcffc113bf0 con 0x7fcffc10de60 2024-09-21T03:50:51.248 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:51.243+0000 7fcffa7fc640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcffc1052e0 0x7fcffc1a1c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:50:51.248 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:51.243+0000 7fcffa7fc640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcffc1052e0 0x7fcffc1a1c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:53264/0 (socket says 172.21.15.39:53264) 2024-09-21T03:50:51.248 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:51.243+0000 7fcffa7fc640 1 -- 172.21.15.39:0/2307678454 learned_addr learned my addr 172.21.15.39:0/2307678454 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:50:51.248 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:51.243+0000 7fcffaffd640 1 --2- 172.21.15.39:0/2307678454 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcffc1048b0 0x7fcffc1a1750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:50:51.248 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:51.243+0000 7fcffb7fe640 1 --2- 172.21.15.39:0/2307678454 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcffc10de60 0x7fcffc1ac7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:50:51.248 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:51.243+0000 7fcffa7fc640 1 -- 172.21.15.39:0/2307678454 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcffc10de60 msgr2=0x7fcffc1ac7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:50:51.248 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:51.243+0000 7fcffa7fc640 1 --2- 172.21.15.39:0/2307678454 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcffc10de60 0x7fcffc1ac7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:51.248 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:51.243+0000 7fcffa7fc640 1 -- 172.21.15.39:0/2307678454 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcffc1048b0 msgr2=0x7fcffc1a1750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:50:51.248 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:51.243+0000 7fcffa7fc640 1 --2- 172.21.15.39:0/2307678454 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcffc1048b0 0x7fcffc1a1750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:51.248 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:51.243+0000 7fcffa7fc640 1 -- 172.21.15.39:0/2307678454 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcffc1acd10 con 0x7fcffc1052e0 2024-09-21T03:50:51.249 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:51.243+0000 7fcffaffd640 1 --2- 172.21.15.39:0/2307678454 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcffc1048b0 0x7fcffc1a1750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:50:51.249 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:51.243+0000 7fcffa7fc640 1 --2- 172.21.15.39:0/2307678454 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcffc1052e0 0x7fcffc1a1c90 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7fcff000e9f0 tx=0x7fcff000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:50:51.249 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:51.243+0000 7fcfe3fff640 1 -- 172.21.15.39:0/2307678454 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcff000ccc0 con 0x7fcffc1052e0 2024-09-21T03:50:51.249 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:51.244+0000 7fd001752640 1 -- 172.21.15.39:0/2307678454 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcffc1acee0 con 0x7fcffc1052e0 2024-09-21T03:50:51.249 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:51.244+0000 7fcfe3fff640 1 -- 172.21.15.39:0/2307678454 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcff00054d0 con 0x7fcffc1052e0 2024-09-21T03:50:51.249 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:51.244+0000 7fcfe3fff640 1 -- 172.21.15.39:0/2307678454 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcff0010430 con 0x7fcffc1052e0 2024-09-21T03:50:51.249 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:51.244+0000 7fd001752640 1 -- 172.21.15.39:0/2307678454 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcffc1ad340 con 0x7fcffc1052e0 2024-09-21T03:50:51.251 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:51.247+0000 7fd007cc6740 1 Processor -- start 2024-09-21T03:50:51.252 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:51.247+0000 7fd007cc6740 1 -- start start 2024-09-21T03:50:51.252 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:51.247+0000 7fd007cc6740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559140589070 0x559140930650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:50:51.252 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:51.247+0000 7fd007cc6740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5591405601b0 0x559140930b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:50:51.252 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:51.247+0000 7fd007cc6740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559140568390 0x5591409310d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:50:51.252 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:51.247+0000 7fd007cc6740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x5591408a6e50 con 0x559140589070 2024-09-21T03:50:51.252 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:51.247+0000 7fd007cc6740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x559140896280 con 0x5591405601b0 2024-09-21T03:50:51.252 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:51.247+0000 7fd007cc6740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55914089a090 con 0x559140568390 2024-09-21T03:50:51.252 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:51.247+0000 7fcffb7fe640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559140589070 0x559140930650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:50:51.252 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:51.248+0000 7fcffb7fe640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559140589070 0x559140930650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:53266/0 (socket says 172.21.15.39:53266) 2024-09-21T03:50:51.252 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:51.248+0000 7fcffb7fe640 1 -- 172.21.15.39:0/2885060640 learned_addr learned my addr 172.21.15.39:0/2885060640 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:50:51.252 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:51.248+0000 7fcffaffd640 1 --2- 172.21.15.39:0/2885060640 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5591405601b0 0x559140930b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:50:51.253 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:51.248+0000 7fcfe3fff640 1 -- 172.21.15.39:0/2307678454 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 27) ==== 154643+0+0 (secure 0 0 0) 0x7fcff0005020 con 0x7fcffc1052e0 2024-09-21T03:50:51.253 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:51.248+0000 7fcffb7fe640 1 -- 172.21.15.39:0/2885060640 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559140568390 msgr2=0x5591409310d0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-21T03:50:51.253 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:51.248+0000 7fcffb7fe640 1 --2- 172.21.15.39:0/2885060640 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559140568390 0x5591409310d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:51.253 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:51.248+0000 7fcffb7fe640 1 -- 172.21.15.39:0/2885060640 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5591405601b0 msgr2=0x559140930b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:50:51.253 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:51.248+0000 7fcffb7fe640 1 --2- 172.21.15.39:0/2885060640 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5591405601b0 0x559140930b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:51.253 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:51.248+0000 7fcffb7fe640 1 -- 172.21.15.39:0/2885060640 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5591405f4650 con 0x559140589070 2024-09-21T03:50:51.253 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:51.248+0000 7fcffaffd640 1 --2- 172.21.15.39:0/2885060640 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5591405601b0 0x559140930b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:50:51.253 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:51.248+0000 7fcffb7fe640 1 --2- 172.21.15.39:0/2885060640 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559140589070 0x559140930650 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7fcff402f4c0 tx=0x7fcff4009800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=24fad4988b4c26d0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:50:51.253 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:51.248+0000 7fcfe17fa640 1 -- 172.21.15.39:0/2885060640 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcff4031cf0 con 0x559140589070 2024-09-21T03:50:51.253 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:51.249+0000 7fcfe17fa640 1 -- 172.21.15.39:0/2885060640 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcff4036030 con 0x559140589070 2024-09-21T03:50:51.253 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:51.249+0000 7fcfe17fa640 1 -- 172.21.15.39:0/2885060640 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcff40042a0 con 0x559140589070 2024-09-21T03:50:51.253 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:51.249+0000 7fcfe3fff640 1 --2- 172.21.15.39:0/2307678454 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7fcfe40b6680 0x7fcfe40b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:50:51.253 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:51.249+0000 7fcffaffd640 1 --2- 172.21.15.39:0/2307678454 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7fcfe40b6680 0x7fcfe40b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:50:51.254 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:51.249+0000 7fcfe3fff640 1 -- 172.21.15.39:0/2307678454 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8484+0+0 (secure 0 0 0) 0x7fcff00e7a20 con 0x7fcffc1052e0 2024-09-21T03:50:51.254 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:51.250+0000 7fd007cc6740 1 -- 172.21.15.39:0/2885060640 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559140589070 msgr2=0x559140930650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:50:51.254 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:51.250+0000 7fcffaffd640 1 --2- 172.21.15.39:0/2307678454 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7fcfe40b6680 0x7fcfe40b8b90 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x55914092f2d0 tx=0x7fcfec00b390 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:50:51.254 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:51.250+0000 7fd007cc6740 1 --2- 172.21.15.39:0/2885060640 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559140589070 0x559140930650 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7fcff402f4c0 tx=0x7fcff4009800 comp rx=0 tx=0).stop 2024-09-21T03:50:51.254 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:51.250+0000 7fd007cc6740 1 -- 172.21.15.39:0/2885060640 shutdown_connections 2024-09-21T03:50:51.254 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:51.250+0000 7fd007cc6740 1 --2- 172.21.15.39:0/2885060640 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559140568390 0x5591409310d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:51.254 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:51.250+0000 7fd007cc6740 1 --2- 172.21.15.39:0/2885060640 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5591405601b0 0x559140930b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:51.254 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:51.250+0000 7fd007cc6740 1 --2- 172.21.15.39:0/2885060640 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559140589070 0x559140930650 unknown :-1 s=CLOSED pgs=777 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:51.254 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:51.250+0000 7fd007cc6740 1 -- 172.21.15.39:0/2885060640 >> 172.21.15.39:0/2885060640 conn(0x55914057d460 msgr2=0x55914092e540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:50:51.254 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:51.250+0000 7fd007cc6740 1 -- 172.21.15.39:0/2885060640 shutdown_connections 2024-09-21T03:50:51.254 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:51.250+0000 7fd007cc6740 1 -- 172.21.15.39:0/2885060640 wait complete. 2024-09-21T03:50:51.258 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:51.250+0000 7fd007cc6740 1 -- start start 2024-09-21T03:50:51.258 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:51.250+0000 7fd007cc6740 1 Processor -- start 2024-09-21T03:50:51.258 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:51.251+0000 7fd007cc6740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5591405601b0 0x55914093bd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:50:51.258 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:51.251+0000 7fd007cc6740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559140568390 0x55914093c260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:50:51.258 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:51.251+0000 7fd007cc6740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x559140589070 0x559140947fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:50:51.258 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:51.251+0000 7fd007cc6740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x5591406f7700 con 0x5591405601b0 2024-09-21T03:50:51.258 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:51.251+0000 7fd007cc6740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x559140831110 con 0x559140589070 2024-09-21T03:50:51.258 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:51.251+0000 7fd007cc6740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55914093ec50 con 0x559140568390 2024-09-21T03:50:51.258 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:51.251+0000 7fd007cc6740 10 client.0 _ll_register_callbacks cb 0x55914084a990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:50:51.258 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:51.251+0000 7fcffaffd640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5591405601b0 0x55914093bd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:50:51.259 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:51.251+0000 7fcffaffd640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5591405601b0 0x55914093bd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:53282/0 (socket says 172.21.15.39:53282) 2024-09-21T03:50:51.259 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:51.251+0000 7fcffaffd640 1 -- 172.21.15.39:0/4092624597 learned_addr learned my addr 172.21.15.39:0/4092624597 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:50:51.259 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:51.251+0000 7fcffa7fc640 1 --2- 172.21.15.39:0/4092624597 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559140568390 0x55914093c260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:50:51.259 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:51.251+0000 7fcffb7fe640 1 --2- 172.21.15.39:0/4092624597 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x559140589070 0x559140947fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:50:51.259 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:51.251+0000 7fcffaffd640 1 -- 172.21.15.39:0/4092624597 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559140568390 msgr2=0x55914093c260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:50:51.259 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:51.251+0000 7fcffaffd640 1 --2- 172.21.15.39:0/4092624597 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559140568390 0x55914093c260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:51.259 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:51.251+0000 7fcffaffd640 1 -- 172.21.15.39:0/4092624597 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x559140589070 msgr2=0x559140947fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:50:51.259 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:51.251+0000 7fcffaffd640 1 --2- 172.21.15.39:0/4092624597 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x559140589070 0x559140947fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:51.259 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:51.251+0000 7fcffaffd640 1 -- 172.21.15.39:0/4092624597 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55914085ec40 con 0x5591405601b0 2024-09-21T03:50:51.259 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:51.251+0000 7fcffa7fc640 1 --2- 172.21.15.39:0/4092624597 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559140568390 0x55914093c260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:50:51.259 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:51.251+0000 7fcffb7fe640 1 --2- 172.21.15.39:0/4092624597 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x559140589070 0x559140947fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:50:51.259 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:51.252+0000 7fcffaffd640 1 --2- 172.21.15.39:0/4092624597 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5591405601b0 0x55914093bd20 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7fcfec0042e0 tx=0x7fcfec0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6cbcf30c1effe24 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:50:51.259 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:51.252+0000 7fcfdeff5640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-09-21T03:50:51.259 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:51.252+0000 7fcfdeff5640 1 -- 172.21.15.39:0/4092624597 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfec0105d0 con 0x5591405601b0 2024-09-21T03:50:51.260 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:51.252+0000 7fd007cc6740 1 -- 172.21.15.39:0/4092624597 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcff4036030 con 0x5591405601b0 2024-09-21T03:50:51.260 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:51.252+0000 7fcfdeff5640 1 -- 172.21.15.39:0/4092624597 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8484+0+0 (secure 0 0 0) 0x7fcfec010770 con 0x5591405601b0 2024-09-21T03:50:51.260 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:51.252+0000 7fcfdeff5640 1 -- 172.21.15.39:0/4092624597 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fcfec010970 con 0x5591405601b0 2024-09-21T03:50:51.260 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:51.252+0000 7fd007cc6740 10 client.16596 fetch_fsmap learned FSMap version 29 2024-09-21T03:50:51.260 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:51.252+0000 7fd007cc6740 1 -- 172.21.15.39:0/4092624597 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55914094aa30 con 0x5591405601b0 2024-09-21T03:50:51.260 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:51.252+0000 7fcfdeff5640 1 -- 172.21.15.39:0/4092624597 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcff4036030 con 0x5591405601b0 2024-09-21T03:50:51.260 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:51.253+0000 7fcfdeff5640 1 -- 172.21.15.39:0/4092624597 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fcfec02e070 con 0x5591405601b0 2024-09-21T03:50:51.260 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:51.253+0000 7fd007cc6740 10 client.16596 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T03:50:51.260 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:51.253+0000 7fd007cc6740 10 client.16596 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T03:50:51.260 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:51.253+0000 7fd007cc6740 20 client.16596 populate_metadata read hostname 'smithi039' 2024-09-21T03:50:51.260 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:51.253+0000 7fd007cc6740 1 --2- 172.21.15.39:0/4092624597 >> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] conn(0x5591405479f0 0x55914094df50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:50:51.260 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:51.253+0000 7fd007cc6740 4 client.16596 mds_command: new command op to 24695 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:50:51.260 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:51.253+0000 7fd007cc6740 1 -- 172.21.15.39:0/4092624597 --> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5591408a8da0 con 0x5591405479f0 2024-09-21T03:50:51.260 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:51.253+0000 7fcffa7fc640 1 --2- 172.21.15.39:0/4092624597 >> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] conn(0x5591405479f0 0x55914094df50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:50:51.261 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:51.253+0000 7fcfdeff5640 10 client.16596 ms_handle_connect on v2:172.21.15.150:6838/2949386907 2024-09-21T03:50:51.261 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:51.253+0000 7fcffa7fc640 1 --2- 172.21.15.39:0/4092624597 >> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] conn(0x5591405479f0 0x55914094df50 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=146d9942fd1becf9 server_cookie=eabc457c31fe4cc2 in_seq=0 out_seq=0 2024-09-21T03:50:51.261 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:51.254+0000 7fcfdeff5640 1 -- 172.21.15.39:0/4092624597 <== mds.0 v2:172.21.15.150:6838/2949386907 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5591408a8da0 con 0x5591405479f0 2024-09-21T03:50:51.261 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:51.254+0000 7fcfdeff5640 10 client.16596 handle_command_reply: tid=0 2024-09-21T03:50:51.261 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:51.254+0000 7fd007cc6740 1 client.16596 shutdown 2024-09-21T03:50:51.261 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:51.254+0000 7fd007cc6740 1 -- 172.21.15.39:0/4092624597 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5591405601b0 msgr2=0x55914093bd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:50:51.261 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:51.254+0000 7fd007cc6740 1 --2- 172.21.15.39:0/4092624597 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5591405601b0 0x55914093bd20 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7fcfec0042e0 tx=0x7fcfec0046c0 comp rx=0 tx=0).stop 2024-09-21T03:50:51.261 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:51.254+0000 7fd007cc6740 1 -- 172.21.15.39:0/4092624597 shutdown_connections 2024-09-21T03:50:51.261 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:51.254+0000 7fd007cc6740 1 --2- 172.21.15.39:0/4092624597 >> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] conn(0x5591405479f0 0x55914094df50 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:51.261 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:51.255+0000 7fd007cc6740 1 --2- 172.21.15.39:0/4092624597 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x559140589070 0x559140947fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:51.261 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:51.255+0000 7fd007cc6740 1 --2- 172.21.15.39:0/4092624597 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559140568390 0x55914093c260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:51.261 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:51.255+0000 7fd007cc6740 1 --2- 172.21.15.39:0/4092624597 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5591405601b0 0x55914093bd20 unknown :-1 s=CLOSED pgs=778 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:51.261 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:51.255+0000 7fd007cc6740 1 -- 172.21.15.39:0/4092624597 >> 172.21.15.39:0/4092624597 conn(0x55914057d460 msgr2=0x559140933800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:50:51.261 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:51.255+0000 7fcfdeff5640 0 client.16596 ms_handle_reset on v2:172.21.15.150:6838/2949386907 2024-09-21T03:50:51.262 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:51.255+0000 7fd007cc6740 1 -- 172.21.15.39:0/4092624597 shutdown_connections 2024-09-21T03:50:51.262 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:51.255+0000 7fd007cc6740 1 -- 172.21.15.39:0/4092624597 wait complete. 2024-09-21T03:50:51.262 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:51.255+0000 7fd007cc6740 20 client.16596 trim_cache size 0 max 16384 2024-09-21T03:50:51.275 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:51.272+0000 7fd007cc6740 1 Processor -- start 2024-09-21T03:50:51.275 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:51.272+0000 7fd007cc6740 1 -- start start 2024-09-21T03:50:51.275 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:51.272+0000 7fd007cc6740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5591405601b0 0x55914093dc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:50:51.275 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:51.272+0000 7fd007cc6740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559140568390 0x559140931680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:50:51.276 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:51.272+0000 7fd007cc6740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559140589070 0x559140947ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:50:51.276 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:51.272+0000 7fd007cc6740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fcffc1b5cc0 con 0x559140589070 2024-09-21T03:50:51.276 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:51.272+0000 7fd007cc6740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fcffc1b7800 con 0x5591405601b0 2024-09-21T03:50:51.276 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:51.272+0000 7fd007cc6740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fcffc1b6390 con 0x559140568390 2024-09-21T03:50:51.276 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:51.273+0000 7fcffb7fe640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559140589070 0x559140947ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:50:51.276 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:51.273+0000 7fcffb7fe640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559140589070 0x559140947ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:53294/0 (socket says 172.21.15.39:53294) 2024-09-21T03:50:51.276 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:51.273+0000 7fcffb7fe640 1 -- 172.21.15.39:0/4140014494 learned_addr learned my addr 172.21.15.39:0/4140014494 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:50:51.276 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:51.273+0000 7fcffa7fc640 1 --2- 172.21.15.39:0/4140014494 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559140568390 0x559140931680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:50:51.276 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:51.273+0000 7fcffaffd640 1 --2- 172.21.15.39:0/4140014494 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5591405601b0 0x55914093dc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:50:51.276 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:51.273+0000 7fcffb7fe640 1 -- 172.21.15.39:0/4140014494 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559140568390 msgr2=0x559140931680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:50:51.276 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:51.273+0000 7fcffb7fe640 1 --2- 172.21.15.39:0/4140014494 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559140568390 0x559140931680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:51.276 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:51.273+0000 7fcffb7fe640 1 -- 172.21.15.39:0/4140014494 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5591405601b0 msgr2=0x55914093dc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:50:51.276 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:51.273+0000 7fcffb7fe640 1 --2- 172.21.15.39:0/4140014494 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5591405601b0 0x55914093dc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:51.276 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:51.273+0000 7fcffb7fe640 1 -- 172.21.15.39:0/4140014494 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559140932940 con 0x559140589070 2024-09-21T03:50:51.277 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:51.273+0000 7fcffaffd640 1 --2- 172.21.15.39:0/4140014494 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5591405601b0 0x55914093dc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:50:51.277 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:51.273+0000 7fcffb7fe640 1 --2- 172.21.15.39:0/4140014494 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559140589070 0x559140947ea0 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7fcff4005e30 tx=0x7fcff4000f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa2aec69476625be server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:50:51.277 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:51.273+0000 7fcfe0ff9640 1 -- 172.21.15.39:0/4140014494 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcff4031af0 con 0x559140589070 2024-09-21T03:50:51.277 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:51.273+0000 7fcfe0ff9640 1 -- 172.21.15.39:0/4140014494 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcff4036070 con 0x559140589070 2024-09-21T03:50:51.277 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:51.273+0000 7fcfe0ff9640 1 -- 172.21.15.39:0/4140014494 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcff4003df0 con 0x559140589070 2024-09-21T03:50:51.280 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:51.274+0000 7fd007cc6740 1 -- 172.21.15.39:0/4140014494 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559140589070 msgr2=0x559140947ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:50:51.281 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:51.274+0000 7fd007cc6740 1 --2- 172.21.15.39:0/4140014494 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559140589070 0x559140947ea0 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7fcff4005e30 tx=0x7fcff4000f40 comp rx=0 tx=0).stop 2024-09-21T03:50:51.281 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:51.274+0000 7fd007cc6740 1 -- 172.21.15.39:0/4140014494 shutdown_connections 2024-09-21T03:50:51.281 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:51.274+0000 7fd007cc6740 1 --2- 172.21.15.39:0/4140014494 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559140589070 0x559140947ea0 unknown :-1 s=CLOSED pgs=779 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:51.281 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:51.274+0000 7fd007cc6740 1 --2- 172.21.15.39:0/4140014494 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559140568390 0x559140931680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:51.281 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:51.274+0000 7fd007cc6740 1 --2- 172.21.15.39:0/4140014494 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5591405601b0 0x55914093dc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:51.281 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:51.274+0000 7fd007cc6740 1 -- 172.21.15.39:0/4140014494 >> 172.21.15.39:0/4140014494 conn(0x55914057d460 msgr2=0x559140935760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:50:51.281 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:51.274+0000 7fd007cc6740 1 -- 172.21.15.39:0/4140014494 shutdown_connections 2024-09-21T03:50:51.281 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:51.274+0000 7fd007cc6740 1 -- 172.21.15.39:0/4140014494 wait complete. 2024-09-21T03:50:51.281 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:51.275+0000 7fd007cc6740 1 -- start start 2024-09-21T03:50:51.281 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:51.275+0000 7fd007cc6740 1 Processor -- start 2024-09-21T03:50:51.281 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:51.275+0000 7fd007cc6740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5591405601b0 0x5591409401c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:50:51.281 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:51.275+0000 7fd007cc6740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559140568390 0x559140940700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:50:51.281 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:51.275+0000 7fd007cc6740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559140589070 0x5591409b7950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:50:51.282 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:51.275+0000 7fd007cc6740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fcffc1b7130 con 0x559140589070 2024-09-21T03:50:51.282 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:51.275+0000 7fd007cc6740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fcffc1b6a60 con 0x5591405601b0 2024-09-21T03:50:51.282 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:51.275+0000 7fd007cc6740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fcffc1b3ca0 con 0x559140568390 2024-09-21T03:50:51.282 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:51.275+0000 7fcffb7fe640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559140589070 0x5591409b7950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:50:51.282 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:51.275+0000 7fcffb7fe640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559140589070 0x5591409b7950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:53306/0 (socket says 172.21.15.39:53306) 2024-09-21T03:50:51.282 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:51.275+0000 7fcffb7fe640 1 -- 172.21.15.39:0/2785066444 learned_addr learned my addr 172.21.15.39:0/2785066444 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:50:51.282 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:51.275+0000 7fd007cc6740 10 client.0 _ll_register_callbacks cb 0x7fcfec00be60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:50:51.282 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:51.276+0000 7fcffaffd640 1 --2- 172.21.15.39:0/2785066444 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5591405601b0 0x5591409401c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:50:51.282 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:51.276+0000 7fcffa7fc640 1 --2- 172.21.15.39:0/2785066444 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559140568390 0x559140940700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:50:51.282 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:51.276+0000 7fcffb7fe640 1 -- 172.21.15.39:0/2785066444 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559140568390 msgr2=0x559140940700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:50:51.282 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:51.276+0000 7fcffb7fe640 1 --2- 172.21.15.39:0/2785066444 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559140568390 0x559140940700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:51.282 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:51.276+0000 7fcffb7fe640 1 -- 172.21.15.39:0/2785066444 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5591405601b0 msgr2=0x5591409401c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:50:51.282 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:51.276+0000 7fcffb7fe640 1 --2- 172.21.15.39:0/2785066444 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5591405601b0 0x5591409401c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:51.282 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:51.276+0000 7fcffb7fe640 1 -- 172.21.15.39:0/2785066444 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559140940f80 con 0x559140589070 2024-09-21T03:50:51.283 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:51.276+0000 7fcffb7fe640 1 --2- 172.21.15.39:0/2785066444 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559140589070 0x5591409b7950 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7fcff4031d20 tx=0x7fcff4031d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec05f84947cae456 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:50:51.283 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:51.276+0000 7fcfdeff5640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-09-21T03:50:51.283 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:51.276+0000 7fcfdeff5640 1 -- 172.21.15.39:0/2785066444 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcff405b070 con 0x559140589070 2024-09-21T03:50:51.283 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:51.276+0000 7fcfdeff5640 1 -- 172.21.15.39:0/2785066444 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8484+0+0 (secure 0 0 0) 0x7fcff4031860 con 0x559140589070 2024-09-21T03:50:51.283 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:51.276+0000 7fd007cc6740 1 -- 172.21.15.39:0/2785066444 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcff4036070 con 0x559140589070 2024-09-21T03:50:51.283 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:51.277+0000 7fcfdeff5640 1 -- 172.21.15.39:0/2785066444 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fcff4003970 con 0x559140589070 2024-09-21T03:50:51.283 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:51.277+0000 7fd007cc6740 10 client.16602 fetch_fsmap learned FSMap version 29 2024-09-21T03:50:51.283 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:51.277+0000 7fd007cc6740 1 -- 172.21.15.39:0/2785066444 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5591409b8e50 con 0x559140589070 2024-09-21T03:50:51.283 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:51.277+0000 7fcfdeff5640 1 -- 172.21.15.39:0/2785066444 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcff4036070 con 0x559140589070 2024-09-21T03:50:51.283 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:51.277+0000 7fcfdeff5640 1 -- 172.21.15.39:0/2785066444 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fcff4043070 con 0x559140589070 2024-09-21T03:50:51.283 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:51.277+0000 7fd007cc6740 10 client.16602 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T03:50:51.283 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:51.277+0000 7fd007cc6740 10 client.16602 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T03:50:51.283 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:51.277+0000 7fd007cc6740 20 client.16602 populate_metadata read hostname 'smithi039' 2024-09-21T03:50:51.283 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:51.277+0000 7fd007cc6740 1 --2- 172.21.15.39:0/2785066444 >> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] conn(0x55914051cda0 0x5591409bb290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:50:51.284 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:51.277+0000 7fd007cc6740 4 client.16602 mds_command: new command op to 24695 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-21T03:50:51.284 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:51.277+0000 7fd007cc6740 1 -- 172.21.15.39:0/2785066444 --> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5591408a4850 con 0x55914051cda0 2024-09-21T03:50:51.284 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:51.277+0000 7fcffaffd640 1 --2- 172.21.15.39:0/2785066444 >> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] conn(0x55914051cda0 0x5591409bb290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:50:51.284 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:51.278+0000 7fcffaffd640 1 --2- 172.21.15.39:0/2785066444 >> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] conn(0x55914051cda0 0x5591409bb290 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=564c8ac39d238621 server_cookie=5a69ee2dd781c6a3 in_seq=0 out_seq=0 2024-09-21T03:50:51.284 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:51.278+0000 7fcfdeff5640 10 client.16602 ms_handle_connect on v2:172.21.15.150:6838/2949386907 2024-09-21T03:50:51.284 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:51.278+0000 7fcfdeff5640 1 -- 172.21.15.39:0/2785066444 <== mds.0 v2:172.21.15.150:6838/2949386907 1 ==== command_reply(tid 0: 0 ) ==== 8+0+2362 (crc 0 0 0) 0x5591408a4850 con 0x55914051cda0 2024-09-21T03:50:51.284 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:51.278+0000 7fcfdeff5640 10 client.16602 handle_command_reply: tid=0 2024-09-21T03:50:51.284 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:51.278+0000 7fd007cc6740 1 client.16602 shutdown 2024-09-21T03:50:51.284 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:51.278+0000 7fd007cc6740 1 -- 172.21.15.39:0/2785066444 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559140589070 msgr2=0x5591409b7950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:50:51.284 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:51.278+0000 7fd007cc6740 1 --2- 172.21.15.39:0/2785066444 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559140589070 0x5591409b7950 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7fcff4031d20 tx=0x7fcff4031d50 comp rx=0 tx=0).stop 2024-09-21T03:50:51.284 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:51.278+0000 7fd007cc6740 1 -- 172.21.15.39:0/2785066444 shutdown_connections 2024-09-21T03:50:51.284 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:51.278+0000 7fd007cc6740 1 --2- 172.21.15.39:0/2785066444 >> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] conn(0x55914051cda0 0x5591409bb290 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:51.284 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:51.279+0000 7fd007cc6740 1 --2- 172.21.15.39:0/2785066444 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559140589070 0x5591409b7950 unknown :-1 s=CLOSED pgs=780 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:51.285 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:51.279+0000 7fcfdeff5640 0 client.16602 ms_handle_reset on v2:172.21.15.150:6838/2949386907 2024-09-21T03:50:51.285 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:51.279+0000 7fd007cc6740 1 --2- 172.21.15.39:0/2785066444 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559140568390 0x559140940700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:51.285 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:51.279+0000 7fd007cc6740 1 --2- 172.21.15.39:0/2785066444 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5591405601b0 0x5591409401c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:51.285 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:51.279+0000 7fd007cc6740 1 -- 172.21.15.39:0/2785066444 >> 172.21.15.39:0/2785066444 conn(0x55914057d460 msgr2=0x559140935760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:50:51.285 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:51.279+0000 7fd007cc6740 1 -- 172.21.15.39:0/2785066444 shutdown_connections 2024-09-21T03:50:51.285 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:51.279+0000 7fd007cc6740 1 -- 172.21.15.39:0/2785066444 wait complete. 2024-09-21T03:50:51.285 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:51.279+0000 7fd007cc6740 20 client.16602 trim_cache size 0 max 16384 2024-09-21T03:50:51.285 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-09-21T03:50:51.285 INFO:teuthology.orchestra.run.smithi039.stdout: "status": "no active scrubs running", 2024-09-21T03:50:51.285 INFO:teuthology.orchestra.run.smithi039.stdout: "scrubs": {}, 2024-09-21T03:50:51.286 INFO:teuthology.orchestra.run.smithi039.stdout: "scrub_stats": { 2024-09-21T03:50:51.286 INFO:teuthology.orchestra.run.smithi039.stdout: "74ab917e-ecea-4778-a1c3-c184214fb24c": { 2024-09-21T03:50:51.286 INFO:teuthology.orchestra.run.smithi039.stdout: "start_time": "2024-09-21T03:50:17.726630+0000", 2024-09-21T03:50:51.286 INFO:teuthology.orchestra.run.smithi039.stdout: "path": "/", 2024-09-21T03:50:51.286 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_started": 0, 2024-09-21T03:50:51.286 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_passed": 0, 2024-09-21T03:50:51.286 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_failed": 0, 2024-09-21T03:50:51.286 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_skipped": 0 2024-09-21T03:50:51.286 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:50:51.286 INFO:teuthology.orchestra.run.smithi039.stdout: "6e760f1f-9048-4f92-bd66-d708fbef75b3": { 2024-09-21T03:50:51.286 INFO:teuthology.orchestra.run.smithi039.stdout: "start_time": "2024-09-21T03:50:03.711918+0000", 2024-09-21T03:50:51.286 INFO:teuthology.orchestra.run.smithi039.stdout: "path": "/", 2024-09-21T03:50:51.286 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_started": 0, 2024-09-21T03:50:51.286 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_passed": 0, 2024-09-21T03:50:51.287 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_failed": 0, 2024-09-21T03:50:51.287 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_skipped": 0 2024-09-21T03:50:51.287 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:50:51.287 INFO:teuthology.orchestra.run.smithi039.stdout: "e28ad743-2b22-45b3-8799-a638c2883623": { 2024-09-21T03:50:51.287 INFO:teuthology.orchestra.run.smithi039.stdout: "start_time": "2024-09-21T03:48:31.448821+0000", 2024-09-21T03:50:51.287 INFO:teuthology.orchestra.run.smithi039.stdout: "path": "/", 2024-09-21T03:50:51.287 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_started": 0, 2024-09-21T03:50:51.287 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_passed": 0, 2024-09-21T03:50:51.287 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_failed": 0, 2024-09-21T03:50:51.287 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_skipped": 0 2024-09-21T03:50:51.287 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:50:51.287 INFO:teuthology.orchestra.run.smithi039.stdout: "29411307-eb8d-4339-8d2b-47ebd32c0080": { 2024-09-21T03:50:51.287 INFO:teuthology.orchestra.run.smithi039.stdout: "start_time": "2024-09-21T03:47:38.092920+0000", 2024-09-21T03:50:51.287 INFO:teuthology.orchestra.run.smithi039.stdout: "path": "/", 2024-09-21T03:50:51.288 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_started": 0, 2024-09-21T03:50:51.288 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_passed": 0, 2024-09-21T03:50:51.288 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_failed": 0, 2024-09-21T03:50:51.288 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_skipped": 0 2024-09-21T03:50:51.288 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:50:51.288 INFO:teuthology.orchestra.run.smithi039.stdout: "9860c2f4-b7b1-47aa-865a-9a1c4434a5f6": { 2024-09-21T03:50:51.288 INFO:teuthology.orchestra.run.smithi039.stdout: "start_time": "2024-09-21T03:49:17.718867+0000", 2024-09-21T03:50:51.288 INFO:teuthology.orchestra.run.smithi039.stdout: "path": "/", 2024-09-21T03:50:51.288 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_started": 0, 2024-09-21T03:50:51.288 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_passed": 0, 2024-09-21T03:50:51.288 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_failed": 0, 2024-09-21T03:50:51.288 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_skipped": 0 2024-09-21T03:50:51.288 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:50:51.288 INFO:teuthology.orchestra.run.smithi039.stdout: "c7e2b8e4-057d-435f-b2bd-b633de98035f": { 2024-09-21T03:50:51.288 INFO:teuthology.orchestra.run.smithi039.stdout: "start_time": "2024-09-21T03:47:19.400305+0000", 2024-09-21T03:50:51.289 INFO:teuthology.orchestra.run.smithi039.stdout: "path": "/", 2024-09-21T03:50:51.289 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_started": 0, 2024-09-21T03:50:51.289 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_passed": 0, 2024-09-21T03:50:51.289 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_failed": 0, 2024-09-21T03:50:51.289 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_skipped": 0 2024-09-21T03:50:51.289 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:50:51.289 INFO:teuthology.orchestra.run.smithi039.stdout: "10728de8-c49d-4edc-be2b-639249255e94": { 2024-09-21T03:50:51.289 INFO:teuthology.orchestra.run.smithi039.stdout: "start_time": "2024-09-21T03:47:03.965623+0000", 2024-09-21T03:50:51.289 INFO:teuthology.orchestra.run.smithi039.stdout: "path": "/", 2024-09-21T03:50:51.289 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_started": 0, 2024-09-21T03:50:51.289 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_passed": 0, 2024-09-21T03:50:51.289 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_failed": 0, 2024-09-21T03:50:51.289 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_skipped": 0 2024-09-21T03:50:51.289 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:50:51.289 INFO:teuthology.orchestra.run.smithi039.stdout: "02509e4f-3911-4983-b16f-df8ded0b9998": { 2024-09-21T03:50:51.290 INFO:teuthology.orchestra.run.smithi039.stdout: "start_time": "2024-09-21T03:46:17.889576+0000", 2024-09-21T03:50:51.290 INFO:teuthology.orchestra.run.smithi039.stdout: "path": "/", 2024-09-21T03:50:51.290 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_started": 0, 2024-09-21T03:50:51.290 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_passed": 0, 2024-09-21T03:50:51.290 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_failed": 0, 2024-09-21T03:50:51.290 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_skipped": 0 2024-09-21T03:50:51.290 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-21T03:50:51.290 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-21T03:50:51.290 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-09-21T03:50:51.291 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:51.279+0000 7fd001752640 1 -- 172.21.15.39:0/2307678454 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7fcfe40b6680 msgr2=0x7fcfe40b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:50:51.291 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:51.279+0000 7fd001752640 1 --2- 172.21.15.39:0/2307678454 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7fcfe40b6680 0x7fcfe40b8b90 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x55914092f2d0 tx=0x7fcfec00b390 comp rx=0 tx=0).stop 2024-09-21T03:50:51.291 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:51.279+0000 7fd001752640 1 -- 172.21.15.39:0/2307678454 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcffc1052e0 msgr2=0x7fcffc1a1c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:50:51.291 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:51.279+0000 7fd001752640 1 --2- 172.21.15.39:0/2307678454 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcffc1052e0 0x7fcffc1a1c90 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7fcff000e9f0 tx=0x7fcff000eeb0 comp rx=0 tx=0).stop 2024-09-21T03:50:51.291 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:51.279+0000 7fd001752640 1 -- 172.21.15.39:0/2307678454 shutdown_connections 2024-09-21T03:50:51.291 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:51.279+0000 7fd001752640 1 --2- 172.21.15.39:0/2307678454 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7fcfe40b6680 0x7fcfe40b8b90 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:51.291 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:51.279+0000 7fd001752640 1 --2- 172.21.15.39:0/2307678454 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcffc10de60 0x7fcffc1ac7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:51.291 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:51.279+0000 7fd001752640 1 --2- 172.21.15.39:0/2307678454 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcffc1052e0 0x7fcffc1a1c90 unknown :-1 s=CLOSED pgs=776 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:51.291 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:51.279+0000 7fd001752640 1 --2- 172.21.15.39:0/2307678454 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcffc1048b0 0x7fcffc1a1750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:51.291 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:51.279+0000 7fd001752640 1 -- 172.21.15.39:0/2307678454 >> 172.21.15.39:0/2307678454 conn(0x7fcffc079730 msgr2=0x7fcffc1068f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:50:51.291 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:51.280+0000 7fd001752640 1 -- 172.21.15.39:0/2307678454 shutdown_connections 2024-09-21T03:50:51.292 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:51.280+0000 7fd001752640 1 -- 172.21.15.39:0/2307678454 wait complete. 2024-09-21T03:50:51.468 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:50:50 smithi161 ceph-mon[34533]: pgmap v469: 97 pgs: 97 active+clean; 10 MiB data, 536 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-09-21T03:50:51.474 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:50:50 smithi150 ceph-mon[34289]: pgmap v469: 97 pgs: 97 active+clean; 10 MiB data, 536 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-09-21T03:50:52.211 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-21T03:50:52.212 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 51116d5c-77ca-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T03:50:52.476 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/51116d5c-77ca-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:50:53.119 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:53.116+0000 7f4c03084640 1 -- 172.21.15.39:0/1898927036 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4bfc1083c0 msgr2=0x7f4bfc1087d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:50:53.119 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:53.116+0000 7f4c03084640 1 --2- 172.21.15.39:0/1898927036 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4bfc1083c0 0x7f4bfc1087d0 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7f4bf4009ca0 tx=0x7f4bf402a8c0 comp rx=0 tx=0).stop 2024-09-21T03:50:53.119 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:53.116+0000 7f4c03084640 1 -- 172.21.15.39:0/1898927036 shutdown_connections 2024-09-21T03:50:53.119 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:53.116+0000 7f4c03084640 1 --2- 172.21.15.39:0/1898927036 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4bfc108d10 0x7f4bfc10f540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:53.119 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:53.116+0000 7f4c03084640 1 --2- 172.21.15.39:0/1898927036 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4bfc1083c0 0x7f4bfc1087d0 unknown :-1 s=CLOSED pgs=781 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:53.119 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:53.116+0000 7f4c03084640 1 --2- 172.21.15.39:0/1898927036 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4bfc102a80 0x7f4bfc102e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:53.119 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:53.116+0000 7f4c03084640 1 -- 172.21.15.39:0/1898927036 >> 172.21.15.39:0/1898927036 conn(0x7f4bfc079740 msgr2=0x7f4bfc079b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:50:53.119 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:53.116+0000 7f4c03084640 1 -- 172.21.15.39:0/1898927036 shutdown_connections 2024-09-21T03:50:53.119 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:53.116+0000 7f4c03084640 1 -- 172.21.15.39:0/1898927036 wait complete. 2024-09-21T03:50:53.119 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:53.117+0000 7f4c03084640 1 Processor -- start 2024-09-21T03:50:53.119 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:53.117+0000 7f4c03084640 1 -- start start 2024-09-21T03:50:53.121 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:53.117+0000 7f4c03084640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4bfc102a80 0x7f4bfc19a580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:50:53.121 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:53.117+0000 7f4c03084640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4bfc1083c0 0x7f4bfc19ab20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:50:53.121 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:53.117+0000 7f4c03084640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4bfc108d10 0x7f4bfc19fdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:50:53.121 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:53.117+0000 7f4c03084640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f4bfc112980 con 0x7f4bfc1083c0 2024-09-21T03:50:53.121 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:53.117+0000 7f4c03084640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f4bfc112800 con 0x7f4bfc108d10 2024-09-21T03:50:53.121 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:53.117+0000 7f4c03084640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f4bfc112b00 con 0x7f4bfc102a80 2024-09-21T03:50:53.121 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:53.117+0000 7f4bfbfff640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4bfc1083c0 0x7f4bfc19ab20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:50:53.121 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:53.118+0000 7f4bfbfff640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4bfc1083c0 0x7f4bfc19ab20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:53336/0 (socket says 172.21.15.39:53336) 2024-09-21T03:50:53.121 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:53.118+0000 7f4bfbfff640 1 -- 172.21.15.39:0/752540157 learned_addr learned my addr 172.21.15.39:0/752540157 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:50:53.121 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:53.118+0000 7f4c00e20640 1 --2- 172.21.15.39:0/752540157 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4bfc102a80 0x7f4bfc19a580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:50:53.122 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:53.118+0000 7f4c01621640 1 --2- 172.21.15.39:0/752540157 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4bfc108d10 0x7f4bfc19fdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:50:53.122 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:53.118+0000 7f4bfbfff640 1 -- 172.21.15.39:0/752540157 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4bfc102a80 msgr2=0x7f4bfc19a580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:50:53.122 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:53.118+0000 7f4bfbfff640 1 --2- 172.21.15.39:0/752540157 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4bfc102a80 0x7f4bfc19a580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:53.122 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:53.118+0000 7f4bfbfff640 1 -- 172.21.15.39:0/752540157 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4bfc108d10 msgr2=0x7f4bfc19fdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:50:53.122 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:53.118+0000 7f4bfbfff640 1 --2- 172.21.15.39:0/752540157 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4bfc108d10 0x7f4bfc19fdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:53.122 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:53.118+0000 7f4bfbfff640 1 -- 172.21.15.39:0/752540157 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4bfc1a04f0 con 0x7f4bfc1083c0 2024-09-21T03:50:53.122 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:53.118+0000 7f4bfbfff640 1 --2- 172.21.15.39:0/752540157 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4bfc1083c0 0x7f4bfc19ab20 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7f4bf4006010 tx=0x7f4bf40319f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:50:53.122 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:53.118+0000 7f4bf9ffb640 1 -- 172.21.15.39:0/752540157 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4bf4003bf0 con 0x7f4bfc1083c0 2024-09-21T03:50:53.122 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:53.118+0000 7f4c03084640 1 -- 172.21.15.39:0/752540157 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4bfc1a0780 con 0x7f4bfc1083c0 2024-09-21T03:50:53.122 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:53.118+0000 7f4bf9ffb640 1 -- 172.21.15.39:0/752540157 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4bf4003d90 con 0x7f4bfc1083c0 2024-09-21T03:50:53.123 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:53.118+0000 7f4bf9ffb640 1 -- 172.21.15.39:0/752540157 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4bf402fb60 con 0x7f4bfc1083c0 2024-09-21T03:50:53.123 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:53.119+0000 7f4c03084640 1 -- 172.21.15.39:0/752540157 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4bfc1a6f40 con 0x7f4bfc1083c0 2024-09-21T03:50:53.123 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:53.120+0000 7f4c03084640 1 -- 172.21.15.39:0/752540157 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4bfc102e90 con 0x7f4bfc1083c0 2024-09-21T03:50:53.123 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:53.121+0000 7f4bf9ffb640 1 -- 172.21.15.39:0/752540157 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 27) ==== 154643+0+0 (secure 0 0 0) 0x7f4bf402fd00 con 0x7f4bfc1083c0 2024-09-21T03:50:53.124 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:53.122+0000 7f4bf9ffb640 1 --2- 172.21.15.39:0/752540157 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f4be80b6680 0x7f4be80b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:50:53.124 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:53.122+0000 7f4c00e20640 1 --2- 172.21.15.39:0/752540157 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f4be80b6680 0x7f4be80b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:50:53.124 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:53.122+0000 7f4bf9ffb640 1 -- 172.21.15.39:0/752540157 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8484+0+0 (secure 0 0 0) 0x7f4bf4108bb0 con 0x7f4bfc1083c0 2024-09-21T03:50:53.124 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:53.123+0000 7f4c00e20640 1 --2- 172.21.15.39:0/752540157 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f4be80b6680 0x7f4be80b8b90 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f4be400bbe0 tx=0x7f4be400b390 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:50:53.127 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:53.125+0000 7f4bf9ffb640 1 -- 172.21.15.39:0/752540157 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f4bf40cf120 con 0x7f4bfc1083c0 2024-09-21T03:50:53.334 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:50:52 smithi039 ceph-mon[31211]: pgmap v470: 97 pgs: 97 active+clean; 10 MiB data, 536 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 9 op/s 2024-09-21T03:50:53.352 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:53.350+0000 7f4c03084640 1 -- 172.21.15.39:0/752540157 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f4bfc1a7110 con 0x7f4bfc1083c0 2024-09-21T03:50:53.353 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:53.351+0000 7f4bf9ffb640 1 -- 172.21.15.39:0/752540157 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11978 (secure 0 0 0) 0x7f4bf40d3500 con 0x7f4bfc1083c0 2024-09-21T03:50:53.353 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T03:50:53.353 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":29,"btime":"2024-09-21T03:46:08:982881+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14790,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6835/508995295","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":508995295},{"type":"v1","addr":"172.21.15.39:6835","nonce":508995295}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14802,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6837/3915912607","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":3915912607},{"type":"v1","addr":"172.21.15.39:6837","nonce":3915912607}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":29,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T03:44:15.009872+0000","modified":"2024-09-21T03:46:08.430113+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24695,"mds_1":24667,"mds_2":24659,"mds_3":24646,"mds_4":14868},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14856":{"gid":14856,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.39:6839/185509192","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6838","nonce":185509192},{"type":"v1","addr":"172.21.15.39:6839","nonce":185509192}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14868":{"gid":14868,"name":"a","rank":4,"incarnation":25,"state":"up:active","state_seq":7,"addr":"172.21.15.39:6841/4247190456","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6840","nonce":4247190456},{"type":"v1","addr":"172.21.15.39:6841","nonce":4247190456}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24599":{"gid":24599,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.150:6833/3114643217","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":3114643217},{"type":"v1","addr":"172.21.15.150:6833","nonce":3114643217}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24601":{"gid":24601,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.161:6833/2963990654","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":2963990654},{"type":"v1","addr":"172.21.15.161:6833","nonce":2963990654}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24632":{"gid":24632,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.150:6835/724594008","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6834","nonce":724594008},{"type":"v1","addr":"172.21.15.150:6835","nonce":724594008}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24640":{"gid":24640,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.161:6835/3747882110","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":3747882110},{"type":"v1","addr":"172.21.15.161:6835","nonce":3747882110}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24646":{"gid":24646,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":10,"addr":"172.21.15.161:6837/3451544597","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6836","nonce":3451544597},{"type":"v1","addr":"172.21.15.161:6837","nonce":3451544597}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24659":{"gid":24659,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.150:6837/4017043144","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6836","nonce":4017043144},{"type":"v1","addr":"172.21.15.150:6837","nonce":4017043144}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24667":{"gid":24667,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.161:6839/2190107270","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6838","nonce":2190107270},{"type":"v1","addr":"172.21.15.161:6839","nonce":2190107270}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24695":{"gid":24695,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.150:6839/2949386907","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6838","nonce":2949386907},{"type":"v1","addr":"172.21.15.150:6839","nonce":2949386907}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24695,"qdb_cluster":[24695,24667,24659,24646,14868]},"id":1}]} 2024-09-21T03:50:53.354 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 29 2024-09-21T03:50:53.356 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:53.354+0000 7f4c03084640 1 -- 172.21.15.39:0/752540157 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f4be80b6680 msgr2=0x7f4be80b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:50:53.356 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:53.354+0000 7f4c03084640 1 --2- 172.21.15.39:0/752540157 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f4be80b6680 0x7f4be80b8b90 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f4be400bbe0 tx=0x7f4be400b390 comp rx=0 tx=0).stop 2024-09-21T03:50:53.356 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:53.354+0000 7f4c03084640 1 -- 172.21.15.39:0/752540157 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4bfc1083c0 msgr2=0x7f4bfc19ab20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:50:53.356 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:53.354+0000 7f4c03084640 1 --2- 172.21.15.39:0/752540157 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4bfc1083c0 0x7f4bfc19ab20 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7f4bf4006010 tx=0x7f4bf40319f0 comp rx=0 tx=0).stop 2024-09-21T03:50:53.356 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:53.354+0000 7f4c03084640 1 -- 172.21.15.39:0/752540157 shutdown_connections 2024-09-21T03:50:53.356 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:53.354+0000 7f4c03084640 1 --2- 172.21.15.39:0/752540157 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f4be80b6680 0x7f4be80b8b90 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:53.356 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:53.354+0000 7f4c03084640 1 --2- 172.21.15.39:0/752540157 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4bfc108d10 0x7f4bfc19fdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:53.356 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:53.354+0000 7f4c03084640 1 --2- 172.21.15.39:0/752540157 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4bfc1083c0 0x7f4bfc19ab20 unknown :-1 s=CLOSED pgs=782 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:53.356 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:53.354+0000 7f4c03084640 1 --2- 172.21.15.39:0/752540157 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4bfc102a80 0x7f4bfc19a580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:53.356 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:53.354+0000 7f4c03084640 1 -- 172.21.15.39:0/752540157 >> 172.21.15.39:0/752540157 conn(0x7f4bfc079740 msgr2=0x7f4bfc101a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:50:53.356 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:53.354+0000 7f4c03084640 1 -- 172.21.15.39:0/752540157 shutdown_connections 2024-09-21T03:50:53.356 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:53.354+0000 7f4c03084640 1 -- 172.21.15.39:0/752540157 wait complete. 2024-09-21T03:50:53.468 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:50:52 smithi161 ceph-mon[34533]: pgmap v470: 97 pgs: 97 active+clean; 10 MiB data, 536 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 9 op/s 2024-09-21T03:50:53.475 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:50:52 smithi150 ceph-mon[34289]: pgmap v470: 97 pgs: 97 active+clean; 10 MiB data, 536 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 9 op/s 2024-09-21T03:50:54.184 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 51116d5c-77ca-11ef-baf5-efdc52797490 -- ceph tell mds.1:4 damage ls 2024-09-21T03:50:54.333 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:50:54 smithi039 ceph-mon[31211]: from='client.? 172.21.15.39:0/752540157' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:50:54.445 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/51116d5c-77ca-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:50:54.467 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:50:54 smithi161 ceph-mon[34533]: from='client.? 172.21.15.39:0/752540157' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:50:54.474 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:50:54 smithi150 ceph-mon[34289]: from='client.? 172.21.15.39:0/752540157' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:50:54.973 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:54.967+0000 7f38b9138640 1 -- 172.21.15.39:0/2772692495 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f38b4102fa0 msgr2=0x7f38b410fcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:50:54.974 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:54.967+0000 7f38b9138640 1 --2- 172.21.15.39:0/2772692495 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f38b4102fa0 0x7f38b410fcc0 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7f389c009900 tx=0x7f389c02a480 comp rx=0 tx=0).stop 2024-09-21T03:50:54.974 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:54.967+0000 7f38b9138640 1 -- 172.21.15.39:0/2772692495 shutdown_connections 2024-09-21T03:50:54.974 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:54.967+0000 7f38b9138640 1 --2- 172.21.15.39:0/2772692495 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f38b4110200 0x7f38b4112620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:54.974 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:54.967+0000 7f38b9138640 1 --2- 172.21.15.39:0/2772692495 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f38b4102fa0 0x7f38b410fcc0 unknown :-1 s=CLOSED pgs=783 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:54.974 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:54.967+0000 7f38b9138640 1 --2- 172.21.15.39:0/2772692495 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f38b4102650 0x7f38b4102a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:54.974 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:54.967+0000 7f38b9138640 1 -- 172.21.15.39:0/2772692495 >> 172.21.15.39:0/2772692495 conn(0x7f38b4079730 msgr2=0x7f38b4079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:50:54.974 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:54.967+0000 7f38b9138640 1 -- 172.21.15.39:0/2772692495 shutdown_connections 2024-09-21T03:50:54.974 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:54.967+0000 7f38b9138640 1 -- 172.21.15.39:0/2772692495 wait complete. 2024-09-21T03:50:54.974 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:54.968+0000 7f38b9138640 1 Processor -- start 2024-09-21T03:50:54.974 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:54.968+0000 7f38b9138640 1 -- start start 2024-09-21T03:50:54.974 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:54.968+0000 7f38b9138640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f38b4102650 0x7f38b419c620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:50:54.974 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:54.968+0000 7f38b9138640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f38b4102fa0 0x7f38b419cbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:50:54.975 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:54.968+0000 7f38b9138640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f38b4110200 0x7f38b41a1e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:50:54.975 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:54.968+0000 7f38b9138640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f38b4115c70 con 0x7f38b4110200 2024-09-21T03:50:54.975 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:54.968+0000 7f38b9138640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f38b4115af0 con 0x7f38b4102650 2024-09-21T03:50:54.975 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:54.968+0000 7f38b9138640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f38b4115df0 con 0x7f38b4102fa0 2024-09-21T03:50:54.975 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:54.968+0000 7f38b359e640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f38b4110200 0x7f38b41a1e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:50:54.975 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:54.969+0000 7f38b359e640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f38b4110200 0x7f38b41a1e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:53366/0 (socket says 172.21.15.39:53366) 2024-09-21T03:50:54.975 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:54.969+0000 7f38b359e640 1 -- 172.21.15.39:0/54189948 learned_addr learned my addr 172.21.15.39:0/54189948 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:50:54.975 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:54.969+0000 7f38b259c640 1 --2- 172.21.15.39:0/54189948 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f38b4102fa0 0x7f38b419cbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:50:54.975 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:54.969+0000 7f38b2d9d640 1 --2- 172.21.15.39:0/54189948 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f38b4102650 0x7f38b419c620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:50:54.975 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:54.969+0000 7f38b359e640 1 -- 172.21.15.39:0/54189948 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f38b4102fa0 msgr2=0x7f38b419cbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:50:54.975 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:54.969+0000 7f38b359e640 1 --2- 172.21.15.39:0/54189948 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f38b4102fa0 0x7f38b419cbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:54.975 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:54.969+0000 7f38b359e640 1 -- 172.21.15.39:0/54189948 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f38b4102650 msgr2=0x7f38b419c620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:50:54.975 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:54.969+0000 7f38b359e640 1 --2- 172.21.15.39:0/54189948 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f38b4102650 0x7f38b419c620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:54.975 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:54.969+0000 7f38b359e640 1 -- 172.21.15.39:0/54189948 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f38b41a24d0 con 0x7f38b4110200 2024-09-21T03:50:54.976 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:54.969+0000 7f38b359e640 1 --2- 172.21.15.39:0/54189948 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f38b4110200 0x7f38b41a1e60 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7f38a400eaa0 tx=0x7f38a400ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:50:54.976 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:54.969+0000 7f38abfff640 1 -- 172.21.15.39:0/54189948 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38a400cdb0 con 0x7f38b4110200 2024-09-21T03:50:54.976 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:54.969+0000 7f38b9138640 1 -- 172.21.15.39:0/54189948 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f38b41a27c0 con 0x7f38b4110200 2024-09-21T03:50:54.976 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:54.969+0000 7f38abfff640 1 -- 172.21.15.39:0/54189948 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f38a4005020 con 0x7f38b4110200 2024-09-21T03:50:54.976 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:54.969+0000 7f38abfff640 1 -- 172.21.15.39:0/54189948 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38a40116d0 con 0x7f38b4110200 2024-09-21T03:50:54.976 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:54.969+0000 7f38b9138640 1 -- 172.21.15.39:0/54189948 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f38b41a8fe0 con 0x7f38b4110200 2024-09-21T03:50:54.978 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:54.973+0000 7f38bf6ac740 1 Processor -- start 2024-09-21T03:50:54.978 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:54.973+0000 7f38abfff640 1 -- 172.21.15.39:0/54189948 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 27) ==== 154643+0+0 (secure 0 0 0) 0x7f38a40054a0 con 0x7f38b4110200 2024-09-21T03:50:54.978 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:54.973+0000 7f38bf6ac740 1 -- start start 2024-09-21T03:50:54.978 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:54.973+0000 7f38bf6ac740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556b0a626070 0x556b0a9cd6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:50:54.978 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:54.973+0000 7f38bf6ac740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556b0a5fd1b0 0x556b0a9cdbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:50:54.978 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:54.974+0000 7f38bf6ac740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556b0a605390 0x556b0a9ce120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:50:54.978 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:54.974+0000 7f38bf6ac740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x556b0a943ea0 con 0x556b0a605390 2024-09-21T03:50:54.978 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:54.974+0000 7f38bf6ac740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x556b0a9332d0 con 0x556b0a626070 2024-09-21T03:50:54.978 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:54.974+0000 7f38bf6ac740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x556b0a9370e0 con 0x556b0a5fd1b0 2024-09-21T03:50:54.978 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:54.974+0000 7f38b259c640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556b0a605390 0x556b0a9ce120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:50:54.978 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:54.974+0000 7f38b259c640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556b0a605390 0x556b0a9ce120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:53376/0 (socket says 172.21.15.39:53376) 2024-09-21T03:50:54.978 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:54.974+0000 7f38b259c640 1 -- 172.21.15.39:0/4169905093 learned_addr learned my addr 172.21.15.39:0/4169905093 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:50:54.979 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:54.974+0000 7f38b2d9d640 1 --2- 172.21.15.39:0/4169905093 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556b0a5fd1b0 0x556b0a9cdbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:50:54.979 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:54.974+0000 7f38b259c640 1 --2- 172.21.15.39:0/4169905093 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556b0a626070 0x556b0a9cd6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:50:54.979 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:54.974+0000 7f38b259c640 1 -- 172.21.15.39:0/4169905093 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556b0a5fd1b0 msgr2=0x556b0a9cdbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:50:54.979 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:54.974+0000 7f38b259c640 1 --2- 172.21.15.39:0/4169905093 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556b0a5fd1b0 0x556b0a9cdbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:54.979 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:54.974+0000 7f38b259c640 1 -- 172.21.15.39:0/4169905093 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556b0a626070 msgr2=0x556b0a9cd6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:50:54.979 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:54.974+0000 7f38b259c640 1 --2- 172.21.15.39:0/4169905093 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556b0a626070 0x556b0a9cd6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:54.979 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:54.974+0000 7f38b259c640 1 -- 172.21.15.39:0/4169905093 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556b0a691650 con 0x556b0a605390 2024-09-21T03:50:54.979 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:54.974+0000 7f38b259c640 1 --2- 172.21.15.39:0/4169905093 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556b0a605390 0x556b0a9ce120 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7f389c02a9a0 tx=0x7f389c0058b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=22ccca855ad54070 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:50:54.979 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:54.974+0000 7f38abfff640 1 --2- 172.21.15.39:0/54189948 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f38a00b66b0 0x7f38a00b8bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:50:54.979 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:54.975+0000 7f38a97fa640 1 -- 172.21.15.39:0/4169905093 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f389c043020 con 0x556b0a605390 2024-09-21T03:50:54.979 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:54.975+0000 7f38b2d9d640 1 --2- 172.21.15.39:0/54189948 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f38a00b66b0 0x7f38a00b8bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:50:54.979 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:54.975+0000 7f38a97fa640 1 -- 172.21.15.39:0/4169905093 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f389c004030 con 0x556b0a605390 2024-09-21T03:50:54.979 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:54.975+0000 7f38a97fa640 1 -- 172.21.15.39:0/4169905093 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f389c035040 con 0x556b0a605390 2024-09-21T03:50:54.979 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:54.975+0000 7f38abfff640 1 -- 172.21.15.39:0/54189948 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8484+0+0 (secure 0 0 0) 0x7f38a40e7920 con 0x7f38b4110200 2024-09-21T03:50:54.983 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:54.976+0000 7f38bf6ac740 1 -- 172.21.15.39:0/4169905093 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556b0a605390 msgr2=0x556b0a9ce120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:50:54.983 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:54.976+0000 7f38b2d9d640 1 --2- 172.21.15.39:0/54189948 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f38a00b66b0 0x7f38a00b8bc0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x556b0a9cc320 tx=0x7f38ac00c4b0 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:50:54.983 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:54.976+0000 7f38bf6ac740 1 --2- 172.21.15.39:0/4169905093 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556b0a605390 0x556b0a9ce120 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7f389c02a9a0 tx=0x7f389c0058b0 comp rx=0 tx=0).stop 2024-09-21T03:50:54.983 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:54.976+0000 7f38bf6ac740 1 -- 172.21.15.39:0/4169905093 shutdown_connections 2024-09-21T03:50:54.983 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:54.976+0000 7f38bf6ac740 1 --2- 172.21.15.39:0/4169905093 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556b0a605390 0x556b0a9ce120 unknown :-1 s=CLOSED pgs=785 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:54.983 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:54.976+0000 7f38bf6ac740 1 --2- 172.21.15.39:0/4169905093 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556b0a5fd1b0 0x556b0a9cdbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:54.983 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:54.976+0000 7f38bf6ac740 1 --2- 172.21.15.39:0/4169905093 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556b0a626070 0x556b0a9cd6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:54.983 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:54.976+0000 7f38bf6ac740 1 -- 172.21.15.39:0/4169905093 >> 172.21.15.39:0/4169905093 conn(0x556b0a61a460 msgr2=0x556b0a9cb590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:50:54.983 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:54.976+0000 7f38bf6ac740 1 -- 172.21.15.39:0/4169905093 shutdown_connections 2024-09-21T03:50:54.984 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:54.976+0000 7f38bf6ac740 1 -- 172.21.15.39:0/4169905093 wait complete. 2024-09-21T03:50:54.984 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:54.976+0000 7f38bf6ac740 1 -- start start 2024-09-21T03:50:54.984 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:54.976+0000 7f38bf6ac740 1 Processor -- start 2024-09-21T03:50:54.984 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:54.977+0000 7f38bf6ac740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556b0a5fd1b0 0x556b0a9d8d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:50:54.984 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:54.977+0000 7f38bf6ac740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556b0a605390 0x556b0a9d9250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:50:54.984 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:54.977+0000 7f38bf6ac740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556b0a626070 0x556b0a9e5140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:50:54.984 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:54.977+0000 7f38bf6ac740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x556b0a794700 con 0x556b0a5fd1b0 2024-09-21T03:50:54.984 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:54.977+0000 7f38bf6ac740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x556b0a8ce160 con 0x556b0a605390 2024-09-21T03:50:54.984 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:54.977+0000 7f38bf6ac740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x556b0a9dbe20 con 0x556b0a626070 2024-09-21T03:50:54.984 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:54.977+0000 7f38bf6ac740 10 client.0 _ll_register_callbacks cb 0x556b0a8e79e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:50:54.984 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:54.977+0000 7f38b2d9d640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556b0a5fd1b0 0x556b0a9d8d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:50:54.984 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:54.977+0000 7f38b2d9d640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556b0a5fd1b0 0x556b0a9d8d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:53390/0 (socket says 172.21.15.39:53390) 2024-09-21T03:50:54.984 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:54.977+0000 7f38b2d9d640 1 -- 172.21.15.39:0/2688029191 learned_addr learned my addr 172.21.15.39:0/2688029191 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:50:54.984 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:54.977+0000 7f38b259c640 1 --2- 172.21.15.39:0/2688029191 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556b0a605390 0x556b0a9d9250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:50:54.984 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:54.977+0000 7f38b359e640 1 --2- 172.21.15.39:0/2688029191 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556b0a626070 0x556b0a9e5140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:50:54.985 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:54.977+0000 7f38b2d9d640 1 -- 172.21.15.39:0/2688029191 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556b0a626070 msgr2=0x556b0a9e5140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:50:54.985 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:54.977+0000 7f38b2d9d640 1 --2- 172.21.15.39:0/2688029191 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556b0a626070 0x556b0a9e5140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:54.985 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:54.977+0000 7f38b2d9d640 1 -- 172.21.15.39:0/2688029191 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556b0a605390 msgr2=0x556b0a9d9250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:50:54.985 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:54.977+0000 7f38b2d9d640 1 --2- 172.21.15.39:0/2688029191 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556b0a605390 0x556b0a9d9250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:54.985 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:54.977+0000 7f38b2d9d640 1 -- 172.21.15.39:0/2688029191 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556b0a8fbc90 con 0x556b0a5fd1b0 2024-09-21T03:50:54.985 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:54.977+0000 7f38b2d9d640 1 --2- 172.21.15.39:0/2688029191 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556b0a5fd1b0 0x556b0a9d8d10 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7f38ac005e10 tx=0x7f38ac0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=66e62d3307abdb89 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:50:54.985 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:54.977+0000 7f389a7fc640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-09-21T03:50:54.985 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:54.978+0000 7f389a7fc640 1 -- 172.21.15.39:0/2688029191 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38ac029060 con 0x556b0a5fd1b0 2024-09-21T03:50:54.985 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:54.978+0000 7f38bf6ac740 1 -- 172.21.15.39:0/2688029191 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f389c004030 con 0x556b0a5fd1b0 2024-09-21T03:50:54.985 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:54.978+0000 7f389a7fc640 1 -- 172.21.15.39:0/2688029191 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8484+0+0 (secure 0 0 0) 0x7f38ac01c900 con 0x556b0a5fd1b0 2024-09-21T03:50:54.985 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:54.978+0000 7f389a7fc640 1 -- 172.21.15.39:0/2688029191 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f38ac01cb00 con 0x556b0a5fd1b0 2024-09-21T03:50:54.985 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:54.978+0000 7f38bf6ac740 10 client.16620 fetch_fsmap learned FSMap version 29 2024-09-21T03:50:54.985 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:54.978+0000 7f38bf6ac740 1 -- 172.21.15.39:0/2688029191 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x556b0a9e7bd0 con 0x556b0a5fd1b0 2024-09-21T03:50:54.985 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:54.978+0000 7f389a7fc640 1 -- 172.21.15.39:0/2688029191 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f389c004030 con 0x556b0a5fd1b0 2024-09-21T03:50:54.985 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:54.978+0000 7f389a7fc640 1 -- 172.21.15.39:0/2688029191 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f38ac018040 con 0x556b0a5fd1b0 2024-09-21T03:50:54.985 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:54.978+0000 7f38bf6ac740 10 client.16620 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T03:50:54.986 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:54.978+0000 7f38bf6ac740 10 client.16620 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.a 2024-09-21T03:50:54.986 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:54.978+0000 7f38bf6ac740 20 client.16620 populate_metadata read hostname 'smithi039' 2024-09-21T03:50:54.986 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:54.978+0000 7f38bf6ac740 1 --2- 172.21.15.39:0/2688029191 >> [v2:172.21.15.39:6840/4247190456,v1:172.21.15.39:6841/4247190456] conn(0x556b0a5e49f0 0x556b0a9eb080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:50:54.986 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:54.978+0000 7f38bf6ac740 4 client.16620 mds_command: new command op to 14868 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:50:54.986 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:54.978+0000 7f38bf6ac740 1 -- 172.21.15.39:0/2688029191 --> [v2:172.21.15.39:6840/4247190456,v1:172.21.15.39:6841/4247190456] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556b0a945df0 con 0x556b0a5e49f0 2024-09-21T03:50:54.986 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:54.978+0000 7f38b259c640 1 --2- 172.21.15.39:0/2688029191 >> [v2:172.21.15.39:6840/4247190456,v1:172.21.15.39:6841/4247190456] conn(0x556b0a5e49f0 0x556b0a9eb080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:50:54.986 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:54.979+0000 7f389a7fc640 10 client.16620 ms_handle_connect on v2:172.21.15.39:6840/4247190456 2024-09-21T03:50:54.986 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:54.979+0000 7f38b259c640 1 --2- 172.21.15.39:0/2688029191 >> [v2:172.21.15.39:6840/4247190456,v1:172.21.15.39:6841/4247190456] conn(0x556b0a5e49f0 0x556b0a9eb080 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=77bf3b3ef89a2049 server_cookie=b44bb07ed934c7d7 in_seq=0 out_seq=0 2024-09-21T03:50:54.986 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:54.979+0000 7f389a7fc640 1 -- 172.21.15.39:0/2688029191 <== mds.4 v2:172.21.15.39:6840/4247190456 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x556b0a945df0 con 0x556b0a5e49f0 2024-09-21T03:50:54.986 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:54.980+0000 7f389a7fc640 10 client.16620 handle_command_reply: tid=0 2024-09-21T03:50:54.986 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:54.980+0000 7f38bf6ac740 1 client.16620 shutdown 2024-09-21T03:50:54.986 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:54.980+0000 7f38bf6ac740 1 -- 172.21.15.39:0/2688029191 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556b0a5fd1b0 msgr2=0x556b0a9d8d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:50:54.986 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:54.980+0000 7f38bf6ac740 1 --2- 172.21.15.39:0/2688029191 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556b0a5fd1b0 0x556b0a9d8d10 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7f38ac005e10 tx=0x7f38ac0046c0 comp rx=0 tx=0).stop 2024-09-21T03:50:54.986 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:54.980+0000 7f38bf6ac740 1 -- 172.21.15.39:0/2688029191 shutdown_connections 2024-09-21T03:50:54.987 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:54.980+0000 7f38bf6ac740 1 --2- 172.21.15.39:0/2688029191 >> [v2:172.21.15.39:6840/4247190456,v1:172.21.15.39:6841/4247190456] conn(0x556b0a5e49f0 0x556b0a9eb080 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:54.987 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:54.980+0000 7f38bf6ac740 1 --2- 172.21.15.39:0/2688029191 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556b0a626070 0x556b0a9e5140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:54.987 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:54.980+0000 7f38bf6ac740 1 --2- 172.21.15.39:0/2688029191 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556b0a605390 0x556b0a9d9250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:54.987 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:54.980+0000 7f389a7fc640 0 client.16620 ms_handle_reset on v2:172.21.15.39:6840/4247190456 2024-09-21T03:50:54.987 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:54.980+0000 7f38bf6ac740 1 --2- 172.21.15.39:0/2688029191 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556b0a5fd1b0 0x556b0a9d8d10 unknown :-1 s=CLOSED pgs=786 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:54.987 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:54.980+0000 7f38bf6ac740 1 -- 172.21.15.39:0/2688029191 >> 172.21.15.39:0/2688029191 conn(0x556b0a61a460 msgr2=0x556b0a9d0830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:50:54.987 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:54.980+0000 7f38bf6ac740 1 -- 172.21.15.39:0/2688029191 shutdown_connections 2024-09-21T03:50:54.987 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:54.980+0000 7f38bf6ac740 1 -- 172.21.15.39:0/2688029191 wait complete. 2024-09-21T03:50:54.987 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:54.980+0000 7f38bf6ac740 20 client.16620 trim_cache size 0 max 16384 2024-09-21T03:50:54.996 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:54.993+0000 7f38bf6ac740 1 Processor -- start 2024-09-21T03:50:54.996 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:54.993+0000 7f38bf6ac740 1 -- start start 2024-09-21T03:50:54.999 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:54.994+0000 7f38bf6ac740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556b0a5fd1b0 0x556b0a9d7c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:50:54.999 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:54.994+0000 7f38bf6ac740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556b0a605390 0x556b0a9da940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:50:54.999 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:54.994+0000 7f38bf6ac740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556b0a626070 0x556b0a9e5040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:50:54.999 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:54.994+0000 7f38bf6ac740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f38b41b1830 con 0x556b0a5fd1b0 2024-09-21T03:50:54.999 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:54.994+0000 7f38bf6ac740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f38b41b3370 con 0x556b0a605390 2024-09-21T03:50:54.999 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:54.994+0000 7f38bf6ac740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f38b41b1f00 con 0x556b0a626070 2024-09-21T03:50:54.999 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:54.994+0000 7f38b2d9d640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556b0a5fd1b0 0x556b0a9d7c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:50:54.999 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:54.994+0000 7f38b2d9d640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556b0a5fd1b0 0x556b0a9d7c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:53398/0 (socket says 172.21.15.39:53398) 2024-09-21T03:50:54.999 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:54.994+0000 7f38b2d9d640 1 -- 172.21.15.39:0/2057850654 learned_addr learned my addr 172.21.15.39:0/2057850654 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:50:55.000 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:54.994+0000 7f38b2d9d640 1 -- 172.21.15.39:0/2057850654 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556b0a626070 msgr2=0x556b0a9e5040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:50:55.000 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:54.994+0000 7f38b359e640 1 --2- 172.21.15.39:0/2057850654 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556b0a626070 0x556b0a9e5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:50:55.000 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:54.994+0000 7f38b2d9d640 1 --2- 172.21.15.39:0/2057850654 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556b0a626070 0x556b0a9e5040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:55.000 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:54.994+0000 7f38b2d9d640 1 -- 172.21.15.39:0/2057850654 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556b0a605390 msgr2=0x556b0a9da940 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-21T03:50:55.000 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:54.994+0000 7f38b2d9d640 1 --2- 172.21.15.39:0/2057850654 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556b0a605390 0x556b0a9da940 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:55.000 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:54.994+0000 7f38b2d9d640 1 -- 172.21.15.39:0/2057850654 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556b0a9cc350 con 0x556b0a5fd1b0 2024-09-21T03:50:55.000 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:54.994+0000 7f38b359e640 1 --2- 172.21.15.39:0/2057850654 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556b0a626070 0x556b0a9e5040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:50:55.000 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:54.994+0000 7f38b2d9d640 1 --2- 172.21.15.39:0/2057850654 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556b0a5fd1b0 0x556b0a9d7c00 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7f38ac004430 tx=0x7f38ac000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dfae02c3b3ba8e96 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:50:55.000 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:54.995+0000 7f38a8ff9640 1 -- 172.21.15.39:0/2057850654 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38ac019730 con 0x556b0a5fd1b0 2024-09-21T03:50:55.000 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:54.995+0000 7f38a8ff9640 1 -- 172.21.15.39:0/2057850654 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f38ac019db0 con 0x556b0a5fd1b0 2024-09-21T03:50:55.000 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:54.995+0000 7f38a8ff9640 1 -- 172.21.15.39:0/2057850654 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38ac004990 con 0x556b0a5fd1b0 2024-09-21T03:50:55.000 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:54.996+0000 7f38bf6ac740 1 -- 172.21.15.39:0/2057850654 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556b0a5fd1b0 msgr2=0x556b0a9d7c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:50:55.000 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:54.996+0000 7f38bf6ac740 1 --2- 172.21.15.39:0/2057850654 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556b0a5fd1b0 0x556b0a9d7c00 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7f38ac004430 tx=0x7f38ac000bc0 comp rx=0 tx=0).stop 2024-09-21T03:50:55.001 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:54.996+0000 7f38bf6ac740 1 -- 172.21.15.39:0/2057850654 shutdown_connections 2024-09-21T03:50:55.001 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:54.996+0000 7f38bf6ac740 1 --2- 172.21.15.39:0/2057850654 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556b0a626070 0x556b0a9e5040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:55.001 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:54.996+0000 7f38bf6ac740 1 --2- 172.21.15.39:0/2057850654 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556b0a605390 0x556b0a9da940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:55.001 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:54.996+0000 7f38bf6ac740 1 --2- 172.21.15.39:0/2057850654 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556b0a5fd1b0 0x556b0a9d7c00 unknown :-1 s=CLOSED pgs=787 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:55.001 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:54.996+0000 7f38bf6ac740 1 -- 172.21.15.39:0/2057850654 >> 172.21.15.39:0/2057850654 conn(0x556b0a61a460 msgr2=0x556b0a9da230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:50:55.001 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:54.996+0000 7f38bf6ac740 1 -- 172.21.15.39:0/2057850654 shutdown_connections 2024-09-21T03:50:55.001 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:54.996+0000 7f38bf6ac740 1 -- 172.21.15.39:0/2057850654 wait complete. 2024-09-21T03:50:55.001 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:54.996+0000 7f38bf6ac740 1 -- start start 2024-09-21T03:50:55.001 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:54.996+0000 7f38bf6ac740 1 Processor -- start 2024-09-21T03:50:55.004 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:54.996+0000 7f38bf6ac740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556b0a5fd1b0 0x556b0a9dd360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:50:55.004 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:54.997+0000 7f38bf6ac740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556b0a605390 0x556b0a9dd8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:50:55.004 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:54.997+0000 7f38bf6ac740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556b0a626070 0x556b0aa54a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:50:55.004 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:54.997+0000 7f38bf6ac740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f38b41b2ca0 con 0x556b0a5fd1b0 2024-09-21T03:50:55.004 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:54.997+0000 7f38bf6ac740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f38b41b25d0 con 0x556b0a626070 2024-09-21T03:50:55.004 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:54.997+0000 7f38bf6ac740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f38b41af810 con 0x556b0a605390 2024-09-21T03:50:55.005 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:54.997+0000 7f38bf6ac740 10 client.0 _ll_register_callbacks cb 0x7f38ac004340 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:50:55.005 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:54.997+0000 7f38b2d9d640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556b0a5fd1b0 0x556b0a9dd360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:50:55.005 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:54.997+0000 7f38b2d9d640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556b0a5fd1b0 0x556b0a9dd360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:53410/0 (socket says 172.21.15.39:53410) 2024-09-21T03:50:55.005 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:54.997+0000 7f38b2d9d640 1 -- 172.21.15.39:0/4056331661 learned_addr learned my addr 172.21.15.39:0/4056331661 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:50:55.005 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:54.997+0000 7f38b259c640 1 --2- 172.21.15.39:0/4056331661 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556b0a605390 0x556b0a9dd8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:50:55.005 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:54.997+0000 7f38b2d9d640 1 -- 172.21.15.39:0/4056331661 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556b0a605390 msgr2=0x556b0a9dd8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:50:55.005 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:54.997+0000 7f38b359e640 1 --2- 172.21.15.39:0/4056331661 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556b0a626070 0x556b0aa54a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:50:55.005 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:54.997+0000 7f38b2d9d640 1 --2- 172.21.15.39:0/4056331661 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556b0a605390 0x556b0a9dd8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:55.005 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:54.997+0000 7f38b2d9d640 1 -- 172.21.15.39:0/4056331661 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556b0a626070 msgr2=0x556b0aa54a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:50:55.005 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:54.997+0000 7f38b2d9d640 1 --2- 172.21.15.39:0/4056331661 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556b0a626070 0x556b0aa54a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:55.005 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:54.997+0000 7f38b2d9d640 1 -- 172.21.15.39:0/4056331661 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556b0a9de120 con 0x556b0a5fd1b0 2024-09-21T03:50:55.005 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:54.997+0000 7f38b359e640 1 --2- 172.21.15.39:0/4056331661 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556b0a626070 0x556b0aa54a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:50:55.005 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:54.997+0000 7f38b2d9d640 1 --2- 172.21.15.39:0/4056331661 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556b0a5fd1b0 0x556b0a9dd360 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7f38ac005e10 tx=0x7f38ac019bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac016218ed9be3da server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:50:55.006 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:54.997+0000 7f389a7fc640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-09-21T03:50:55.006 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:54.997+0000 7f389a7fc640 1 -- 172.21.15.39:0/4056331661 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38ac043060 con 0x556b0a5fd1b0 2024-09-21T03:50:55.006 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:54.997+0000 7f389a7fc640 1 -- 172.21.15.39:0/4056331661 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8484+0+0 (secure 0 0 0) 0x7f38ac01a870 con 0x556b0a5fd1b0 2024-09-21T03:50:55.006 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:54.997+0000 7f38bf6ac740 1 -- 172.21.15.39:0/4056331661 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f38ac019db0 con 0x556b0a5fd1b0 2024-09-21T03:50:55.006 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:54.998+0000 7f389a7fc640 1 -- 172.21.15.39:0/4056331661 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f38ac029040 con 0x556b0a5fd1b0 2024-09-21T03:50:55.006 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:54.998+0000 7f38bf6ac740 10 client.16626 fetch_fsmap learned FSMap version 29 2024-09-21T03:50:55.006 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:54.998+0000 7f38bf6ac740 1 -- 172.21.15.39:0/4056331661 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x556b0aa55f70 con 0x556b0a5fd1b0 2024-09-21T03:50:55.006 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:54.998+0000 7f389a7fc640 1 -- 172.21.15.39:0/4056331661 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f38ac019db0 con 0x556b0a5fd1b0 2024-09-21T03:50:55.006 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:54.998+0000 7f389a7fc640 1 -- 172.21.15.39:0/4056331661 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f38ac018040 con 0x556b0a5fd1b0 2024-09-21T03:50:55.006 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:54.998+0000 7f38bf6ac740 10 client.16626 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T03:50:55.006 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:54.998+0000 7f38bf6ac740 10 client.16626 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.a 2024-09-21T03:50:55.006 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:54.998+0000 7f38bf6ac740 20 client.16626 populate_metadata read hostname 'smithi039' 2024-09-21T03:50:55.006 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:54.998+0000 7f38bf6ac740 1 --2- 172.21.15.39:0/4056331661 >> [v2:172.21.15.39:6840/4247190456,v1:172.21.15.39:6841/4247190456] conn(0x556b0a5b9da0 0x556b0aa58680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:50:55.006 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:54.998+0000 7f38bf6ac740 4 client.16626 mds_command: new command op to 14868 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T03:50:55.007 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:54.998+0000 7f38bf6ac740 1 -- 172.21.15.39:0/4056331661 --> [v2:172.21.15.39:6840/4247190456,v1:172.21.15.39:6841/4247190456] -- command(tid 0: {"prefix": "damage ls"}) -- 0x556b0a9d48c0 con 0x556b0a5b9da0 2024-09-21T03:50:55.007 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:54.998+0000 7f38b259c640 1 --2- 172.21.15.39:0/4056331661 >> [v2:172.21.15.39:6840/4247190456,v1:172.21.15.39:6841/4247190456] conn(0x556b0a5b9da0 0x556b0aa58680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:50:55.007 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:54.999+0000 7f389a7fc640 10 client.16626 ms_handle_connect on v2:172.21.15.39:6840/4247190456 2024-09-21T03:50:55.007 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:54.999+0000 7f38b259c640 1 --2- 172.21.15.39:0/4056331661 >> [v2:172.21.15.39:6840/4247190456,v1:172.21.15.39:6841/4247190456] conn(0x556b0a5b9da0 0x556b0aa58680 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=7fcd841c1c049114 server_cookie=debe84ba61014261 in_seq=0 out_seq=0 2024-09-21T03:50:55.007 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:54.999+0000 7f389a7fc640 1 -- 172.21.15.39:0/4056331661 <== mds.4 v2:172.21.15.39:6840/4247190456 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x556b0a9d48c0 con 0x556b0a5b9da0 2024-09-21T03:50:55.007 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:54.999+0000 7f389a7fc640 10 client.16626 handle_command_reply: tid=0 2024-09-21T03:50:55.007 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:54.999+0000 7f38bf6ac740 1 client.16626 shutdown 2024-09-21T03:50:55.007 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:54.999+0000 7f38bf6ac740 1 -- 172.21.15.39:0/4056331661 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556b0a5fd1b0 msgr2=0x556b0a9dd360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:50:55.007 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:54.999+0000 7f38bf6ac740 1 --2- 172.21.15.39:0/4056331661 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556b0a5fd1b0 0x556b0a9dd360 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7f38ac005e10 tx=0x7f38ac019bc0 comp rx=0 tx=0).stop 2024-09-21T03:50:55.007 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:54.999+0000 7f38bf6ac740 1 -- 172.21.15.39:0/4056331661 shutdown_connections 2024-09-21T03:50:55.007 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:54.999+0000 7f38bf6ac740 1 --2- 172.21.15.39:0/4056331661 >> [v2:172.21.15.39:6840/4247190456,v1:172.21.15.39:6841/4247190456] conn(0x556b0a5b9da0 0x556b0aa58680 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:55.007 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:54.999+0000 7f38bf6ac740 1 --2- 172.21.15.39:0/4056331661 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556b0a626070 0x556b0aa54a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:55.007 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:54.999+0000 7f38bf6ac740 1 --2- 172.21.15.39:0/4056331661 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556b0a605390 0x556b0a9dd8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:55.007 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:54.999+0000 7f38bf6ac740 1 --2- 172.21.15.39:0/4056331661 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556b0a5fd1b0 0x556b0a9dd360 unknown :-1 s=CLOSED pgs=788 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:55.008 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:54.999+0000 7f389a7fc640 0 client.16626 ms_handle_reset on v2:172.21.15.39:6840/4247190456 2024-09-21T03:50:55.008 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:54.999+0000 7f38bf6ac740 1 -- 172.21.15.39:0/4056331661 >> 172.21.15.39:0/4056331661 conn(0x556b0a61a460 msgr2=0x556b0a9ce6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:50:55.008 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:54.999+0000 7f38bf6ac740 1 -- 172.21.15.39:0/4056331661 shutdown_connections 2024-09-21T03:50:55.008 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:54.999+0000 7f38bf6ac740 1 -- 172.21.15.39:0/4056331661 wait complete. 2024-09-21T03:50:55.008 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:54.999+0000 7f38bf6ac740 20 client.16626 trim_cache size 0 max 16384 2024-09-21T03:50:55.008 INFO:teuthology.orchestra.run.smithi039.stdout:[] 2024-09-21T03:50:55.009 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:55.000+0000 7f38b9138640 1 -- 172.21.15.39:0/54189948 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f38a00b66b0 msgr2=0x7f38a00b8bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:50:55.009 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:55.000+0000 7f38b9138640 1 --2- 172.21.15.39:0/54189948 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f38a00b66b0 0x7f38a00b8bc0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x556b0a9cc320 tx=0x7f38ac00c4b0 comp rx=0 tx=0).stop 2024-09-21T03:50:55.009 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:55.000+0000 7f38b9138640 1 -- 172.21.15.39:0/54189948 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f38b4110200 msgr2=0x7f38b41a1e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:50:55.009 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:55.000+0000 7f38b9138640 1 --2- 172.21.15.39:0/54189948 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f38b4110200 0x7f38b41a1e60 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7f38a400eaa0 tx=0x7f38a400ef60 comp rx=0 tx=0).stop 2024-09-21T03:50:55.009 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:55.000+0000 7f38b9138640 1 -- 172.21.15.39:0/54189948 shutdown_connections 2024-09-21T03:50:55.009 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:55.000+0000 7f38b9138640 1 --2- 172.21.15.39:0/54189948 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f38a00b66b0 0x7f38a00b8bc0 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:55.009 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:55.000+0000 7f38b9138640 1 --2- 172.21.15.39:0/54189948 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f38b4110200 0x7f38b41a1e60 unknown :-1 s=CLOSED pgs=784 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:55.009 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:55.000+0000 7f38b9138640 1 --2- 172.21.15.39:0/54189948 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f38b4102fa0 0x7f38b419cbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:55.009 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:55.000+0000 7f38b9138640 1 --2- 172.21.15.39:0/54189948 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f38b4102650 0x7f38b419c620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:55.009 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:55.000+0000 7f38b9138640 1 -- 172.21.15.39:0/54189948 >> 172.21.15.39:0/54189948 conn(0x7f38b4079730 msgr2=0x7f38b4108cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:50:55.009 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:55.000+0000 7f38b9138640 1 -- 172.21.15.39:0/54189948 shutdown_connections 2024-09-21T03:50:55.010 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:55.000+0000 7f38b9138640 1 -- 172.21.15.39:0/54189948 wait complete. 2024-09-21T03:50:55.302 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:50:55 smithi039 ceph-mon[31211]: pgmap v471: 97 pgs: 97 active+clean; 10 MiB data, 536 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 10 op/s 2024-09-21T03:50:55.468 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:50:55 smithi161 ceph-mon[34533]: pgmap v471: 97 pgs: 97 active+clean; 10 MiB data, 536 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 10 op/s 2024-09-21T03:50:55.474 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:50:55 smithi150 ceph-mon[34289]: pgmap v471: 97 pgs: 97 active+clean; 10 MiB data, 536 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 10 op/s 2024-09-21T03:50:55.835 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 51116d5c-77ca-11ef-baf5-efdc52797490 -- ceph tell mds.1:3 damage ls 2024-09-21T03:50:56.094 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/51116d5c-77ca-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:50:56.592 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:56.583+0000 7f6894ce0640 1 -- 172.21.15.39:0/1250811273 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6890109e40 msgr2=0x7f6890110670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:50:56.592 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:56.583+0000 7f6894ce0640 1 --2- 172.21.15.39:0/1250811273 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6890109e40 0x7f6890110670 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7f6880009ca0 tx=0x7f688002a8e0 comp rx=0 tx=0).stop 2024-09-21T03:50:56.592 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:56.583+0000 7f6894ce0640 1 -- 172.21.15.39:0/1250811273 shutdown_connections 2024-09-21T03:50:56.592 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:56.583+0000 7f6894ce0640 1 --2- 172.21.15.39:0/1250811273 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6890109e40 0x7f6890110670 unknown :-1 s=CLOSED pgs=789 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:56.593 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:56.583+0000 7f6894ce0640 1 --2- 172.21.15.39:0/1250811273 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f68901094f0 0x7f6890109900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:56.593 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:56.583+0000 7f6894ce0640 1 --2- 172.21.15.39:0/1250811273 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6890103bb0 0x7f6890103fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:56.593 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:56.583+0000 7f6894ce0640 1 -- 172.21.15.39:0/1250811273 >> 172.21.15.39:0/1250811273 conn(0x7f68900ff600 msgr2=0x7f6890101a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:50:56.593 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:56.583+0000 7f6894ce0640 1 -- 172.21.15.39:0/1250811273 shutdown_connections 2024-09-21T03:50:56.593 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:56.583+0000 7f6894ce0640 1 -- 172.21.15.39:0/1250811273 wait complete. 2024-09-21T03:50:56.593 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:56.584+0000 7f6894ce0640 1 Processor -- start 2024-09-21T03:50:56.593 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:56.584+0000 7f6894ce0640 1 -- start start 2024-09-21T03:50:56.593 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:56.584+0000 7f6894ce0640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6890103bb0 0x7f689019a4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:50:56.593 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:56.584+0000 7f6894ce0640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f68901094f0 0x7f689019aa70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:50:56.593 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:56.584+0000 7f6894ce0640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6890109e40 0x7f689019fd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:50:56.593 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:56.584+0000 7f6894ce0640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f6890113b80 con 0x7f68901094f0 2024-09-21T03:50:56.593 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:56.584+0000 7f6894ce0640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f6890113a00 con 0x7f6890103bb0 2024-09-21T03:50:56.593 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:56.584+0000 7f6894ce0640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f6890113d00 con 0x7f6890109e40 2024-09-21T03:50:56.594 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:56.584+0000 7f688dd9b640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f68901094f0 0x7f689019aa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:50:56.594 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:56.584+0000 7f688dd9b640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f68901094f0 0x7f689019aa70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:53428/0 (socket says 172.21.15.39:53428) 2024-09-21T03:50:56.594 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:56.584+0000 7f688dd9b640 1 -- 172.21.15.39:0/2952090237 learned_addr learned my addr 172.21.15.39:0/2952090237 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:50:56.594 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:56.584+0000 7f688e59c640 1 --2- 172.21.15.39:0/2952090237 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6890103bb0 0x7f689019a4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:50:56.594 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:56.584+0000 7f688ed9d640 1 --2- 172.21.15.39:0/2952090237 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6890109e40 0x7f689019fd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:50:56.594 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:56.585+0000 7f688dd9b640 1 -- 172.21.15.39:0/2952090237 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6890109e40 msgr2=0x7f689019fd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:50:56.594 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:56.585+0000 7f688dd9b640 1 --2- 172.21.15.39:0/2952090237 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6890109e40 0x7f689019fd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:56.594 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:56.585+0000 7f688dd9b640 1 -- 172.21.15.39:0/2952090237 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6890103bb0 msgr2=0x7f689019a4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:50:56.594 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:56.585+0000 7f688dd9b640 1 --2- 172.21.15.39:0/2952090237 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6890103bb0 0x7f689019a4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:56.594 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:56.585+0000 7f688dd9b640 1 -- 172.21.15.39:0/2952090237 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f68901a0380 con 0x7f68901094f0 2024-09-21T03:50:56.594 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:56.585+0000 7f688dd9b640 1 --2- 172.21.15.39:0/2952090237 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f68901094f0 0x7f689019aa70 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7f687800c8c0 tx=0x7f687800cd80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:50:56.594 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:56.585+0000 7f68877fe640 1 -- 172.21.15.39:0/2952090237 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6878007bb0 con 0x7f68901094f0 2024-09-21T03:50:56.594 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:56.585+0000 7f68877fe640 1 -- 172.21.15.39:0/2952090237 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6878007d50 con 0x7f68901094f0 2024-09-21T03:50:56.595 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:56.585+0000 7f6894ce0640 1 -- 172.21.15.39:0/2952090237 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f68901a0670 con 0x7f68901094f0 2024-09-21T03:50:56.595 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:56.585+0000 7f68877fe640 1 -- 172.21.15.39:0/2952090237 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68780187c0 con 0x7f68901094f0 2024-09-21T03:50:56.595 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:56.585+0000 7f6894ce0640 1 -- 172.21.15.39:0/2952090237 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f68901a6e90 con 0x7f68901094f0 2024-09-21T03:50:56.595 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:56.588+0000 7f689b254740 1 Processor -- start 2024-09-21T03:50:56.595 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:56.588+0000 7f689b254740 1 -- start start 2024-09-21T03:50:56.595 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:56.588+0000 7f68877fe640 1 -- 172.21.15.39:0/2952090237 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 27) ==== 154643+0+0 (secure 0 0 0) 0x7f687800fa20 con 0x7f68901094f0 2024-09-21T03:50:56.595 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:56.588+0000 7f689b254740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564c71511070 0x564c718b86a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:50:56.595 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:56.588+0000 7f689b254740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564c714e81b0 0x564c718b8be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:50:56.595 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:56.588+0000 7f689b254740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564c714f0390 0x564c718b9120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:50:56.595 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:56.588+0000 7f689b254740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x564c7182eea0 con 0x564c714f0390 2024-09-21T03:50:56.595 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:56.588+0000 7f689b254740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x564c7181e2d0 con 0x564c714e81b0 2024-09-21T03:50:56.595 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:56.588+0000 7f689b254740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x564c718220e0 con 0x564c71511070 2024-09-21T03:50:56.595 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:56.588+0000 7f688dd9b640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564c714f0390 0x564c718b9120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:50:56.595 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:56.588+0000 7f688ed9d640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564c71511070 0x564c718b86a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:50:56.596 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:56.588+0000 7f688dd9b640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564c714f0390 0x564c718b9120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:53432/0 (socket says 172.21.15.39:53432) 2024-09-21T03:50:56.596 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:56.588+0000 7f688dd9b640 1 -- 172.21.15.39:0/3968391682 learned_addr learned my addr 172.21.15.39:0/3968391682 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:50:56.596 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:56.588+0000 7f688e59c640 1 --2- 172.21.15.39:0/3968391682 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564c714e81b0 0x564c718b8be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:50:56.596 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:56.589+0000 7f688e59c640 1 -- 172.21.15.39:0/3968391682 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564c71511070 msgr2=0x564c718b86a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:50:56.596 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:56.589+0000 7f688e59c640 1 --2- 172.21.15.39:0/3968391682 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564c71511070 0x564c718b86a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:56.596 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:56.589+0000 7f688e59c640 1 -- 172.21.15.39:0/3968391682 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564c714f0390 msgr2=0x564c718b9120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:50:56.596 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:56.589+0000 7f688e59c640 1 --2- 172.21.15.39:0/3968391682 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564c714f0390 0x564c718b9120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:56.596 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:56.589+0000 7f688e59c640 1 -- 172.21.15.39:0/3968391682 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564c7157c650 con 0x564c714e81b0 2024-09-21T03:50:56.596 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:56.589+0000 7f688dd9b640 1 --2- 172.21.15.39:0/3968391682 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564c714f0390 0x564c718b9120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:50:56.596 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:56.589+0000 7f688e59c640 1 --2- 172.21.15.39:0/3968391682 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564c714e81b0 0x564c718b8be0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f6888009980 tx=0x7f6888009e40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d167a0b38f33b5ac server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:50:56.596 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:56.589+0000 7f68877fe640 1 --2- 172.21.15.39:0/2952090237 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f687c0b66b0 0x7f687c0b8bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:50:56.596 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:56.589+0000 7f6884ff9640 1 -- 172.21.15.39:0/3968391682 <== mon.2 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6888017070 con 0x564c714e81b0 2024-09-21T03:50:56.596 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:56.590+0000 7f6884ff9640 1 -- 172.21.15.39:0/3968391682 <== mon.2 v2:172.21.15.150:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6888013470 con 0x564c714e81b0 2024-09-21T03:50:56.596 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:56.590+0000 7f6884ff9640 1 -- 172.21.15.39:0/3968391682 <== mon.2 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6888012630 con 0x564c714e81b0 2024-09-21T03:50:56.600 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:56.590+0000 7f688ed9d640 1 --2- 172.21.15.39:0/2952090237 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f687c0b66b0 0x7f687c0b8bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:50:56.600 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:56.590+0000 7f68877fe640 1 -- 172.21.15.39:0/2952090237 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8484+0+0 (secure 0 0 0) 0x7f68780e77b0 con 0x7f68901094f0 2024-09-21T03:50:56.600 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:56.591+0000 7f689b254740 1 -- 172.21.15.39:0/3968391682 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564c714e81b0 msgr2=0x564c718b8be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:50:56.600 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:56.591+0000 7f688ed9d640 1 --2- 172.21.15.39:0/2952090237 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f687c0b66b0 0x7f687c0b8bc0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f6880005cc0 tx=0x7f6880005c30 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:50:56.600 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:56.591+0000 7f689b254740 1 --2- 172.21.15.39:0/3968391682 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564c714e81b0 0x564c718b8be0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f6888009980 tx=0x7f6888009e40 comp rx=0 tx=0).stop 2024-09-21T03:50:56.600 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:56.591+0000 7f689b254740 1 -- 172.21.15.39:0/3968391682 shutdown_connections 2024-09-21T03:50:56.601 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:56.591+0000 7f689b254740 1 --2- 172.21.15.39:0/3968391682 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564c714f0390 0x564c718b9120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:56.601 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:56.591+0000 7f689b254740 1 --2- 172.21.15.39:0/3968391682 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564c714e81b0 0x564c718b8be0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:56.601 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:56.591+0000 7f689b254740 1 --2- 172.21.15.39:0/3968391682 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564c71511070 0x564c718b86a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:56.601 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:56.591+0000 7f689b254740 1 -- 172.21.15.39:0/3968391682 >> 172.21.15.39:0/3968391682 conn(0x564c71505460 msgr2=0x564c718b6590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:50:56.601 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:56.591+0000 7f689b254740 1 -- 172.21.15.39:0/3968391682 shutdown_connections 2024-09-21T03:50:56.601 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:56.591+0000 7f689b254740 1 -- 172.21.15.39:0/3968391682 wait complete. 2024-09-21T03:50:56.601 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:56.591+0000 7f689b254740 1 -- start start 2024-09-21T03:50:56.601 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:56.591+0000 7f689b254740 1 Processor -- start 2024-09-21T03:50:56.601 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:56.592+0000 7f689b254740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564c714e81b0 0x564c718c3ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:50:56.601 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:56.592+0000 7f689b254740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564c714f0390 0x564c718c41e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:50:56.601 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:56.592+0000 7f689b254740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564c71511070 0x564c718c4720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:50:56.601 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:56.592+0000 7f689b254740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x564c7167f700 con 0x564c714f0390 2024-09-21T03:50:56.602 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:56.592+0000 7f689b254740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x564c717b9160 con 0x564c714e81b0 2024-09-21T03:50:56.602 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:56.592+0000 7f689b254740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x564c718cb290 con 0x564c71511070 2024-09-21T03:50:56.602 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:56.592+0000 7f689b254740 10 client.0 _ll_register_callbacks cb 0x564c717d29e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:50:56.602 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:56.592+0000 7f688dd9b640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564c714f0390 0x564c718c41e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:50:56.602 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:56.592+0000 7f688dd9b640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564c714f0390 0x564c718c41e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:53438/0 (socket says 172.21.15.39:53438) 2024-09-21T03:50:56.602 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:56.592+0000 7f688dd9b640 1 -- 172.21.15.39:0/444279737 learned_addr learned my addr 172.21.15.39:0/444279737 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:50:56.602 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:56.592+0000 7f688e59c640 1 --2- 172.21.15.39:0/444279737 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564c714e81b0 0x564c718c3ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:50:56.602 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:56.592+0000 7f688ed9d640 1 --2- 172.21.15.39:0/444279737 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564c71511070 0x564c718c4720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:50:56.602 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:56.592+0000 7f688dd9b640 1 -- 172.21.15.39:0/444279737 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564c71511070 msgr2=0x564c718c4720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:50:56.602 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:56.592+0000 7f688dd9b640 1 --2- 172.21.15.39:0/444279737 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564c71511070 0x564c718c4720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:56.602 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:56.592+0000 7f688dd9b640 1 -- 172.21.15.39:0/444279737 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564c714e81b0 msgr2=0x564c718c3ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:50:56.602 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:56.592+0000 7f688dd9b640 1 --2- 172.21.15.39:0/444279737 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564c714e81b0 0x564c718c3ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:56.602 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:56.592+0000 7f688dd9b640 1 -- 172.21.15.39:0/444279737 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564c717e6c90 con 0x564c714f0390 2024-09-21T03:50:56.603 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:56.592+0000 7f688dd9b640 1 --2- 172.21.15.39:0/444279737 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564c714f0390 0x564c718c41e0 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7f68780ad5b0 tx=0x7f68780b6e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a29ecd1244f1c8e server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:50:56.603 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:56.592+0000 7f6875ffb640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-09-21T03:50:56.603 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:56.593+0000 7f6875ffb640 1 -- 172.21.15.39:0/444279737 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68780a9c70 con 0x564c714f0390 2024-09-21T03:50:56.603 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:56.593+0000 7f6875ffb640 1 -- 172.21.15.39:0/444279737 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8484+0+0 (secure 0 0 0) 0x7f68780a8f70 con 0x564c714f0390 2024-09-21T03:50:56.603 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:56.593+0000 7f689b254740 1 -- 172.21.15.39:0/444279737 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6888013470 con 0x564c714f0390 2024-09-21T03:50:56.603 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:56.593+0000 7f6875ffb640 1 -- 172.21.15.39:0/444279737 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f68780acd90 con 0x564c714f0390 2024-09-21T03:50:56.603 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:56.593+0000 7f689b254740 10 client.16638 fetch_fsmap learned FSMap version 29 2024-09-21T03:50:56.603 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:56.593+0000 7f689b254740 1 -- 172.21.15.39:0/444279737 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x564c718d6f10 con 0x564c714f0390 2024-09-21T03:50:56.603 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:56.593+0000 7f6875ffb640 1 -- 172.21.15.39:0/444279737 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6888013470 con 0x564c714f0390 2024-09-21T03:50:56.603 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:56.593+0000 7f6875ffb640 1 -- 172.21.15.39:0/444279737 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f68780a9170 con 0x564c714f0390 2024-09-21T03:50:56.603 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:56.593+0000 7f689b254740 10 client.16638 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T03:50:56.603 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:56.593+0000 7f689b254740 10 client.16638 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-21T03:50:56.603 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:56.593+0000 7f689b254740 20 client.16638 populate_metadata read hostname 'smithi039' 2024-09-21T03:50:56.603 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:56.593+0000 7f689b254740 1 --2- 172.21.15.39:0/444279737 >> [v2:172.21.15.161:6836/3451544597,v1:172.21.15.161:6837/3451544597] conn(0x564c714cf9f0 0x564c718da3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:50:56.604 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:56.593+0000 7f689b254740 4 client.16638 mds_command: new command op to 24646 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:50:56.604 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:56.593+0000 7f689b254740 1 -- 172.21.15.39:0/444279737 --> [v2:172.21.15.161:6836/3451544597,v1:172.21.15.161:6837/3451544597] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564c71830df0 con 0x564c714cf9f0 2024-09-21T03:50:56.604 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:56.594+0000 7f688e59c640 1 --2- 172.21.15.39:0/444279737 >> [v2:172.21.15.161:6836/3451544597,v1:172.21.15.161:6837/3451544597] conn(0x564c714cf9f0 0x564c718da3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:50:56.604 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:56.594+0000 7f6875ffb640 10 client.16638 ms_handle_connect on v2:172.21.15.161:6836/3451544597 2024-09-21T03:50:56.604 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:56.594+0000 7f688e59c640 1 --2- 172.21.15.39:0/444279737 >> [v2:172.21.15.161:6836/3451544597,v1:172.21.15.161:6837/3451544597] conn(0x564c714cf9f0 0x564c718da3e0 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=b7ff8f0e76168f9e server_cookie=cd186768c5fbc092 in_seq=0 out_seq=0 2024-09-21T03:50:56.604 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:56.595+0000 7f6875ffb640 1 -- 172.21.15.39:0/444279737 <== mds.3 v2:172.21.15.161:6836/3451544597 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x564c71830df0 con 0x564c714cf9f0 2024-09-21T03:50:56.604 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:56.595+0000 7f6875ffb640 10 client.16638 handle_command_reply: tid=0 2024-09-21T03:50:56.604 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:56.595+0000 7f689b254740 1 client.16638 shutdown 2024-09-21T03:50:56.604 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:56.595+0000 7f689b254740 1 -- 172.21.15.39:0/444279737 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564c714f0390 msgr2=0x564c718c41e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:50:56.604 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:56.595+0000 7f689b254740 1 --2- 172.21.15.39:0/444279737 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564c714f0390 0x564c718c41e0 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7f68780ad5b0 tx=0x7f68780b6e30 comp rx=0 tx=0).stop 2024-09-21T03:50:56.604 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:56.595+0000 7f689b254740 1 -- 172.21.15.39:0/444279737 shutdown_connections 2024-09-21T03:50:56.604 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:56.595+0000 7f689b254740 1 --2- 172.21.15.39:0/444279737 >> [v2:172.21.15.161:6836/3451544597,v1:172.21.15.161:6837/3451544597] conn(0x564c714cf9f0 0x564c718da3e0 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:56.604 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:56.595+0000 7f689b254740 1 --2- 172.21.15.39:0/444279737 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564c71511070 0x564c718c4720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:56.604 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:56.595+0000 7f689b254740 1 --2- 172.21.15.39:0/444279737 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564c714f0390 0x564c718c41e0 unknown :-1 s=CLOSED pgs=791 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:56.605 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:56.595+0000 7f689b254740 1 --2- 172.21.15.39:0/444279737 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564c714e81b0 0x564c718c3ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:56.605 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:56.595+0000 7f689b254740 1 -- 172.21.15.39:0/444279737 >> 172.21.15.39:0/444279737 conn(0x564c71505460 msgr2=0x564c718bb830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:50:56.605 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:56.595+0000 7f6875ffb640 0 client.16638 ms_handle_reset on v2:172.21.15.161:6836/3451544597 2024-09-21T03:50:56.605 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:56.595+0000 7f689b254740 1 -- 172.21.15.39:0/444279737 shutdown_connections 2024-09-21T03:50:56.605 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:56.595+0000 7f689b254740 1 -- 172.21.15.39:0/444279737 wait complete. 2024-09-21T03:50:56.605 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:56.595+0000 7f689b254740 20 client.16638 trim_cache size 0 max 16384 2024-09-21T03:50:56.612 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:56.609+0000 7f689b254740 1 Processor -- start 2024-09-21T03:50:56.612 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:56.609+0000 7f689b254740 1 -- start start 2024-09-21T03:50:56.612 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:56.609+0000 7f689b254740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564c714e81b0 0x564c718c58d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:50:56.612 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:56.609+0000 7f689b254740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564c714f0390 0x564c718dfae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:50:56.613 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:56.609+0000 7f689b254740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564c71511070 0x564c718e0020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:50:56.613 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:56.609+0000 7f689b254740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f68901ad4f0 con 0x564c714f0390 2024-09-21T03:50:56.613 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:56.609+0000 7f689b254740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f68901af030 con 0x564c714e81b0 2024-09-21T03:50:56.613 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:56.609+0000 7f689b254740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f68901adbc0 con 0x564c71511070 2024-09-21T03:50:56.613 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:56.609+0000 7f688dd9b640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564c714f0390 0x564c718dfae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:50:56.613 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:56.609+0000 7f688dd9b640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564c714f0390 0x564c718dfae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:53448/0 (socket says 172.21.15.39:53448) 2024-09-21T03:50:56.613 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:56.609+0000 7f688dd9b640 1 -- 172.21.15.39:0/139595690 learned_addr learned my addr 172.21.15.39:0/139595690 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:50:56.613 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:56.609+0000 7f688ed9d640 1 --2- 172.21.15.39:0/139595690 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564c71511070 0x564c718e0020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:50:56.613 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:56.609+0000 7f688e59c640 1 --2- 172.21.15.39:0/139595690 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564c714e81b0 0x564c718c58d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:50:56.613 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:56.609+0000 7f688dd9b640 1 -- 172.21.15.39:0/139595690 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564c71511070 msgr2=0x564c718e0020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:50:56.613 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:56.609+0000 7f688dd9b640 1 --2- 172.21.15.39:0/139595690 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564c71511070 0x564c718e0020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:56.613 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:56.609+0000 7f688dd9b640 1 -- 172.21.15.39:0/139595690 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564c714e81b0 msgr2=0x564c718c58d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:50:56.613 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:56.609+0000 7f688dd9b640 1 --2- 172.21.15.39:0/139595690 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564c714e81b0 0x564c718c58d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:56.613 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:56.609+0000 7f688dd9b640 1 -- 172.21.15.39:0/139595690 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564c718c4a60 con 0x564c714f0390 2024-09-21T03:50:56.614 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:56.609+0000 7f688e59c640 1 --2- 172.21.15.39:0/139595690 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564c714e81b0 0x564c718c58d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:50:56.614 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:56.610+0000 7f688dd9b640 1 --2- 172.21.15.39:0/139595690 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564c714f0390 0x564c718dfae0 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7f68780e5520 tx=0x7f68780b32d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aaa0fb71b3367f85 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:50:56.614 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:56.610+0000 7f6877fff640 1 -- 172.21.15.39:0/139595690 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68780acf70 con 0x564c714f0390 2024-09-21T03:50:56.614 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:56.610+0000 7f6877fff640 1 -- 172.21.15.39:0/139595690 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f68780ad130 con 0x564c714f0390 2024-09-21T03:50:56.614 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:56.610+0000 7f6877fff640 1 -- 172.21.15.39:0/139595690 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68780ad3f0 con 0x564c714f0390 2024-09-21T03:50:56.617 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:56.610+0000 7f689b254740 1 -- 172.21.15.39:0/139595690 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564c714f0390 msgr2=0x564c718dfae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:50:56.617 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:56.610+0000 7f689b254740 1 --2- 172.21.15.39:0/139595690 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564c714f0390 0x564c718dfae0 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7f68780e5520 tx=0x7f68780b32d0 comp rx=0 tx=0).stop 2024-09-21T03:50:56.617 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:56.611+0000 7f689b254740 1 -- 172.21.15.39:0/139595690 shutdown_connections 2024-09-21T03:50:56.618 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:56.611+0000 7f689b254740 1 --2- 172.21.15.39:0/139595690 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564c71511070 0x564c718e0020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:56.618 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:56.611+0000 7f689b254740 1 --2- 172.21.15.39:0/139595690 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564c714f0390 0x564c718dfae0 unknown :-1 s=CLOSED pgs=792 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:56.618 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:56.611+0000 7f689b254740 1 --2- 172.21.15.39:0/139595690 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564c714e81b0 0x564c718c58d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:56.618 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:56.611+0000 7f689b254740 1 -- 172.21.15.39:0/139595690 >> 172.21.15.39:0/139595690 conn(0x564c71505460 msgr2=0x564c718c51c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:50:56.618 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:56.611+0000 7f689b254740 1 -- 172.21.15.39:0/139595690 shutdown_connections 2024-09-21T03:50:56.618 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:56.611+0000 7f689b254740 1 -- 172.21.15.39:0/139595690 wait complete. 2024-09-21T03:50:56.618 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:56.611+0000 7f689b254740 1 -- start start 2024-09-21T03:50:56.618 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:56.611+0000 7f689b254740 1 Processor -- start 2024-09-21T03:50:56.618 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:56.611+0000 7f689b254740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564c714e81b0 0x564c718cc430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:50:56.618 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:56.611+0000 7f689b254740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564c714f0390 0x564c718cc970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:50:56.618 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:56.611+0000 7f689b254740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564c71511070 0x564c718cceb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:50:56.618 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:56.611+0000 7f689b254740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f68901ae960 con 0x564c71511070 2024-09-21T03:50:56.618 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:56.611+0000 7f689b254740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f68901ae290 con 0x564c714e81b0 2024-09-21T03:50:56.619 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:56.611+0000 7f689b254740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f68901ab4d0 con 0x564c714f0390 2024-09-21T03:50:56.619 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:56.612+0000 7f689b254740 10 client.0 _ll_register_callbacks cb 0x7f687800c810 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:50:56.619 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:56.612+0000 7f688ed9d640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564c71511070 0x564c718cceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:50:56.619 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:56.612+0000 7f688ed9d640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564c71511070 0x564c718cceb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:53460/0 (socket says 172.21.15.39:53460) 2024-09-21T03:50:56.619 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:56.612+0000 7f688ed9d640 1 -- 172.21.15.39:0/1066480739 learned_addr learned my addr 172.21.15.39:0/1066480739 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:50:56.619 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:56.612+0000 7f688dd9b640 1 --2- 172.21.15.39:0/1066480739 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564c714f0390 0x564c718cc970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:50:56.619 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:56.612+0000 7f688e59c640 1 --2- 172.21.15.39:0/1066480739 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564c714e81b0 0x564c718cc430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:50:56.619 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:56.612+0000 7f688ed9d640 1 -- 172.21.15.39:0/1066480739 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564c714f0390 msgr2=0x564c718cc970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:50:56.619 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:56.612+0000 7f688ed9d640 1 --2- 172.21.15.39:0/1066480739 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564c714f0390 0x564c718cc970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:56.619 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:56.612+0000 7f688ed9d640 1 -- 172.21.15.39:0/1066480739 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564c714e81b0 msgr2=0x564c718cc430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:50:56.619 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:56.612+0000 7f688ed9d640 1 --2- 172.21.15.39:0/1066480739 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564c714e81b0 0x564c718cc430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:56.619 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:56.612+0000 7f688ed9d640 1 -- 172.21.15.39:0/1066480739 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564c71943ce0 con 0x564c71511070 2024-09-21T03:50:56.619 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:56.612+0000 7f688ed9d640 1 --2- 172.21.15.39:0/1066480739 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564c71511070 0x564c718cceb0 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7f6880031e80 tx=0x7f688002a8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c0be74df2cf8e08 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:50:56.619 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:56.612+0000 7f6875ffb640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-09-21T03:50:56.620 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:56.612+0000 7f6875ffb640 1 -- 172.21.15.39:0/1066480739 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6880004750 con 0x564c71511070 2024-09-21T03:50:56.620 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:56.612+0000 7f6875ffb640 1 -- 172.21.15.39:0/1066480739 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8484+0+0 (secure 0 0 0) 0x7f6880041030 con 0x564c71511070 2024-09-21T03:50:56.620 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:56.612+0000 7f689b254740 1 -- 172.21.15.39:0/1066480739 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f68780ad130 con 0x564c71511070 2024-09-21T03:50:56.620 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:56.613+0000 7f6875ffb640 1 -- 172.21.15.39:0/1066480739 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f68800305e0 con 0x564c71511070 2024-09-21T03:50:56.620 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:56.613+0000 7f689b254740 10 client.16644 fetch_fsmap learned FSMap version 29 2024-09-21T03:50:56.620 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:56.613+0000 7f689b254740 1 -- 172.21.15.39:0/1066480739 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x564c71944e10 con 0x564c71511070 2024-09-21T03:50:56.620 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:56.613+0000 7f6875ffb640 1 -- 172.21.15.39:0/1066480739 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f68780ad130 con 0x564c71511070 2024-09-21T03:50:56.620 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:56.613+0000 7f6875ffb640 1 -- 172.21.15.39:0/1066480739 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f688005a070 con 0x564c71511070 2024-09-21T03:50:56.620 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:56.613+0000 7f689b254740 10 client.16644 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T03:50:56.620 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:56.613+0000 7f689b254740 10 client.16644 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-21T03:50:56.620 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:56.613+0000 7f689b254740 20 client.16644 populate_metadata read hostname 'smithi039' 2024-09-21T03:50:56.620 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:56.613+0000 7f689b254740 1 --2- 172.21.15.39:0/1066480739 >> [v2:172.21.15.161:6836/3451544597,v1:172.21.15.161:6837/3451544597] conn(0x564c714a4da0 0x564c71947680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:50:56.620 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:56.613+0000 7f689b254740 4 client.16644 mds_command: new command op to 24646 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T03:50:56.620 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:56.613+0000 7f689b254740 1 -- 172.21.15.39:0/1066480739 --> [v2:172.21.15.161:6836/3451544597,v1:172.21.15.161:6837/3451544597] -- command(tid 0: {"prefix": "damage ls"}) -- 0x564c718b19e0 con 0x564c714a4da0 2024-09-21T03:50:56.620 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:56.613+0000 7f688e59c640 1 --2- 172.21.15.39:0/1066480739 >> [v2:172.21.15.161:6836/3451544597,v1:172.21.15.161:6837/3451544597] conn(0x564c714a4da0 0x564c71947680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:50:56.621 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:56.614+0000 7f688e59c640 1 --2- 172.21.15.39:0/1066480739 >> [v2:172.21.15.161:6836/3451544597,v1:172.21.15.161:6837/3451544597] conn(0x564c714a4da0 0x564c71947680 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=3b0d0c8dbf330aeb server_cookie=196c593c4afc1cb8 in_seq=0 out_seq=0 2024-09-21T03:50:56.621 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:56.614+0000 7f6875ffb640 10 client.16644 ms_handle_connect on v2:172.21.15.161:6836/3451544597 2024-09-21T03:50:56.621 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:56.614+0000 7f6875ffb640 1 -- 172.21.15.39:0/1066480739 <== mds.3 v2:172.21.15.161:6836/3451544597 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x564c718b19e0 con 0x564c714a4da0 2024-09-21T03:50:56.621 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:56.614+0000 7f6875ffb640 10 client.16644 handle_command_reply: tid=0 2024-09-21T03:50:56.621 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:56.614+0000 7f689b254740 1 client.16644 shutdown 2024-09-21T03:50:56.621 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:56.614+0000 7f689b254740 1 -- 172.21.15.39:0/1066480739 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564c71511070 msgr2=0x564c718cceb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:50:56.621 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:56.614+0000 7f689b254740 1 --2- 172.21.15.39:0/1066480739 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564c71511070 0x564c718cceb0 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7f6880031e80 tx=0x7f688002a8e0 comp rx=0 tx=0).stop 2024-09-21T03:50:56.621 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:56.614+0000 7f689b254740 1 -- 172.21.15.39:0/1066480739 shutdown_connections 2024-09-21T03:50:56.621 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:56.614+0000 7f689b254740 1 --2- 172.21.15.39:0/1066480739 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564c71511070 0x564c718cceb0 unknown :-1 s=CLOSED pgs=793 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:56.621 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:56.614+0000 7f689b254740 1 --2- 172.21.15.39:0/1066480739 >> [v2:172.21.15.161:6836/3451544597,v1:172.21.15.161:6837/3451544597] conn(0x564c714a4da0 0x564c71947680 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:56.621 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:56.614+0000 7f689b254740 1 --2- 172.21.15.39:0/1066480739 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564c714f0390 0x564c718cc970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:56.621 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:56.614+0000 7f689b254740 1 --2- 172.21.15.39:0/1066480739 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564c714e81b0 0x564c718cc430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:56.621 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:56.614+0000 7f6875ffb640 0 client.16644 ms_handle_reset on v2:172.21.15.161:6836/3451544597 2024-09-21T03:50:56.621 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:56.614+0000 7f689b254740 1 -- 172.21.15.39:0/1066480739 >> 172.21.15.39:0/1066480739 conn(0x564c71505460 msgr2=0x564c718b96b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:50:56.622 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:56.614+0000 7f689b254740 1 -- 172.21.15.39:0/1066480739 shutdown_connections 2024-09-21T03:50:56.622 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:56.615+0000 7f689b254740 1 -- 172.21.15.39:0/1066480739 wait complete. 2024-09-21T03:50:56.622 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:56.615+0000 7f689b254740 20 client.16644 trim_cache size 0 max 16384 2024-09-21T03:50:56.622 INFO:teuthology.orchestra.run.smithi039.stdout:[] 2024-09-21T03:50:56.623 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:56.615+0000 7f6894ce0640 1 -- 172.21.15.39:0/2952090237 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f687c0b66b0 msgr2=0x7f687c0b8bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:50:56.623 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:56.615+0000 7f6894ce0640 1 --2- 172.21.15.39:0/2952090237 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f687c0b66b0 0x7f687c0b8bc0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f6880005cc0 tx=0x7f6880005c30 comp rx=0 tx=0).stop 2024-09-21T03:50:56.623 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:56.615+0000 7f6894ce0640 1 -- 172.21.15.39:0/2952090237 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f68901094f0 msgr2=0x7f689019aa70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:50:56.623 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:56.615+0000 7f6894ce0640 1 --2- 172.21.15.39:0/2952090237 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f68901094f0 0x7f689019aa70 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7f687800c8c0 tx=0x7f687800cd80 comp rx=0 tx=0).stop 2024-09-21T03:50:56.623 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:56.615+0000 7f6894ce0640 1 -- 172.21.15.39:0/2952090237 shutdown_connections 2024-09-21T03:50:56.623 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:56.615+0000 7f6894ce0640 1 --2- 172.21.15.39:0/2952090237 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f687c0b66b0 0x7f687c0b8bc0 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:56.623 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:56.615+0000 7f6894ce0640 1 --2- 172.21.15.39:0/2952090237 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6890109e40 0x7f689019fd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:56.623 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:56.615+0000 7f6894ce0640 1 --2- 172.21.15.39:0/2952090237 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f68901094f0 0x7f689019aa70 unknown :-1 s=CLOSED pgs=790 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:56.623 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:56.615+0000 7f6894ce0640 1 --2- 172.21.15.39:0/2952090237 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6890103bb0 0x7f689019a4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:56.623 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:56.615+0000 7f6894ce0640 1 -- 172.21.15.39:0/2952090237 >> 172.21.15.39:0/2952090237 conn(0x7f68900ff600 msgr2=0x7f6890107b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:50:56.623 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:56.615+0000 7f6894ce0640 1 -- 172.21.15.39:0/2952090237 shutdown_connections 2024-09-21T03:50:56.623 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:56.615+0000 7f6894ce0640 1 -- 172.21.15.39:0/2952090237 wait complete. 2024-09-21T03:50:57.334 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:50:57 smithi039 ceph-mon[31211]: pgmap v472: 97 pgs: 97 active+clean; 10 MiB data, 536 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 10 op/s 2024-09-21T03:50:57.468 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:50:57 smithi161 ceph-mon[34533]: pgmap v472: 97 pgs: 97 active+clean; 10 MiB data, 536 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 10 op/s 2024-09-21T03:50:57.470 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 51116d5c-77ca-11ef-baf5-efdc52797490 -- ceph tell mds.1:2 damage ls 2024-09-21T03:50:57.474 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:50:57 smithi150 ceph-mon[34289]: pgmap v472: 97 pgs: 97 active+clean; 10 MiB data, 536 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 10 op/s 2024-09-21T03:50:57.731 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/51116d5c-77ca-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:50:58.211 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:58.208+0000 7fccd5893640 1 -- 172.21.15.39:0/4221525014 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fccd01049c0 msgr2=0x7fccd0104dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:50:58.212 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:58.208+0000 7fccd5893640 1 --2- 172.21.15.39:0/4221525014 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fccd01049c0 0x7fccd0104dd0 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7fccc0009900 tx=0x7fccc002a410 comp rx=0 tx=0).stop 2024-09-21T03:50:58.212 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:58.209+0000 7fccd5893640 1 -- 172.21.15.39:0/4221525014 shutdown_connections 2024-09-21T03:50:58.212 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:58.209+0000 7fccd5893640 1 --2- 172.21.15.39:0/4221525014 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fccd010de20 0x7fccd0110310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:58.212 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:58.209+0000 7fccd5893640 1 --2- 172.21.15.39:0/4221525014 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fccd0105310 0x7fccd010d860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:58.212 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:58.209+0000 7fccd5893640 1 --2- 172.21.15.39:0/4221525014 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fccd01049c0 0x7fccd0104dd0 unknown :-1 s=CLOSED pgs=794 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:58.212 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:58.209+0000 7fccd5893640 1 -- 172.21.15.39:0/4221525014 >> 172.21.15.39:0/4221525014 conn(0x7fccd0079730 msgr2=0x7fccd0079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:50:58.212 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:58.209+0000 7fccd5893640 1 -- 172.21.15.39:0/4221525014 shutdown_connections 2024-09-21T03:50:58.212 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:58.209+0000 7fccd5893640 1 -- 172.21.15.39:0/4221525014 wait complete. 2024-09-21T03:50:58.214 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:58.209+0000 7fccd5893640 1 Processor -- start 2024-09-21T03:50:58.214 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:58.210+0000 7fccd5893640 1 -- start start 2024-09-21T03:50:58.214 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:58.210+0000 7fccd5893640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fccd01049c0 0x7fccd019a670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:50:58.214 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:58.210+0000 7fccd5893640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fccd0105310 0x7fccd019ac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:50:58.214 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:58.210+0000 7fccd5893640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fccd010de20 0x7fccd019feb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:50:58.215 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:58.210+0000 7fccd5893640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fccd0113b60 con 0x7fccd01049c0 2024-09-21T03:50:58.215 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:58.210+0000 7fccd5893640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fccd01139e0 con 0x7fccd0105310 2024-09-21T03:50:58.215 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:58.210+0000 7fccd5893640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fccd0113ce0 con 0x7fccd010de20 2024-09-21T03:50:58.215 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:58.210+0000 7fccceffd640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fccd01049c0 0x7fccd019a670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:50:58.215 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:58.210+0000 7fccceffd640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fccd01049c0 0x7fccd019a670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:57616/0 (socket says 172.21.15.39:57616) 2024-09-21T03:50:58.215 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:58.210+0000 7fccceffd640 1 -- 172.21.15.39:0/2671641562 learned_addr learned my addr 172.21.15.39:0/2671641562 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:50:58.215 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:58.210+0000 7fccc7fff640 1 --2- 172.21.15.39:0/2671641562 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fccd0105310 0x7fccd019ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:50:58.215 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:58.210+0000 7fcccf7fe640 1 --2- 172.21.15.39:0/2671641562 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fccd010de20 0x7fccd019feb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:50:58.215 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:58.210+0000 7fccceffd640 1 -- 172.21.15.39:0/2671641562 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fccd010de20 msgr2=0x7fccd019feb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:50:58.215 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:58.210+0000 7fccceffd640 1 --2- 172.21.15.39:0/2671641562 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fccd010de20 0x7fccd019feb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:58.215 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:58.210+0000 7fccceffd640 1 -- 172.21.15.39:0/2671641562 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fccd0105310 msgr2=0x7fccd019ac10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:50:58.215 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:58.210+0000 7fccceffd640 1 --2- 172.21.15.39:0/2671641562 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fccd0105310 0x7fccd019ac10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:58.215 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:58.210+0000 7fccceffd640 1 -- 172.21.15.39:0/2671641562 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fccd01a05e0 con 0x7fccd01049c0 2024-09-21T03:50:58.216 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:58.210+0000 7fcccf7fe640 1 --2- 172.21.15.39:0/2671641562 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fccd010de20 0x7fccd019feb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:50:58.216 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:58.211+0000 7fccceffd640 1 --2- 172.21.15.39:0/2671641562 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fccd01049c0 0x7fccd019a670 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7fccc0031a90 tx=0x7fccc0004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:50:58.216 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:58.211+0000 7fccd4891640 1 -- 172.21.15.39:0/2671641562 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccc003f030 con 0x7fccd01049c0 2024-09-21T03:50:58.216 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:58.211+0000 7fccd4891640 1 -- 172.21.15.39:0/2671641562 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fccc0003cb0 con 0x7fccd01049c0 2024-09-21T03:50:58.216 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:58.211+0000 7fccd5893640 1 -- 172.21.15.39:0/2671641562 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fccd01a0870 con 0x7fccd01049c0 2024-09-21T03:50:58.216 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:58.211+0000 7fccd4891640 1 -- 172.21.15.39:0/2671641562 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccc0035070 con 0x7fccd01049c0 2024-09-21T03:50:58.216 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:58.211+0000 7fccd5893640 1 -- 172.21.15.39:0/2671641562 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fccd01a7030 con 0x7fccd01049c0 2024-09-21T03:50:58.217 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:58.214+0000 7fccd4891640 1 -- 172.21.15.39:0/2671641562 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 27) ==== 154643+0+0 (secure 0 0 0) 0x7fccc0042030 con 0x7fccd01049c0 2024-09-21T03:50:58.217 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:58.214+0000 7fccdbe07740 1 Processor -- start 2024-09-21T03:50:58.217 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:58.214+0000 7fccdbe07740 1 -- start start 2024-09-21T03:50:58.218 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:58.214+0000 7fccdbe07740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e45b3ca070 0x55e45b7756d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:50:58.218 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:58.214+0000 7fccdbe07740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55e45b3a11b0 0x55e45b775c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:50:58.218 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:58.215+0000 7fccdbe07740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e45b3a9390 0x55e45b776150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:50:58.218 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:58.215+0000 7fccdbe07740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x55e45b6e7ea0 con 0x55e45b3a11b0 2024-09-21T03:50:58.218 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:58.215+0000 7fccdbe07740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x55e45b6d72d0 con 0x55e45b3a9390 2024-09-21T03:50:58.218 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:58.215+0000 7fccdbe07740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55e45b6db0e0 con 0x55e45b3ca070 2024-09-21T03:50:58.218 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:58.215+0000 7fccceffd640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55e45b3a11b0 0x55e45b775c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:50:58.218 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:58.215+0000 7fccceffd640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55e45b3a11b0 0x55e45b775c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:57628/0 (socket says 172.21.15.39:57628) 2024-09-21T03:50:58.219 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:58.215+0000 7fccceffd640 1 -- 172.21.15.39:0/966720547 learned_addr learned my addr 172.21.15.39:0/966720547 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:50:58.219 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:58.215+0000 7fcccf7fe640 1 --2- 172.21.15.39:0/966720547 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e45b3ca070 0x55e45b7756d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:50:58.219 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:58.215+0000 7fccc7fff640 1 --2- 172.21.15.39:0/966720547 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e45b3a9390 0x55e45b776150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:50:58.219 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:58.215+0000 7fccd4891640 1 --2- 172.21.15.39:0/2671641562 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7fccbc0b6680 0x7fccbc0b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:50:58.219 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:58.215+0000 7fcccf7fe640 1 --2- 172.21.15.39:0/2671641562 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7fccbc0b6680 0x7fccbc0b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:50:58.219 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:58.215+0000 7fccceffd640 1 -- 172.21.15.39:0/966720547 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e45b3ca070 msgr2=0x55e45b7756d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:50:58.219 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:58.215+0000 7fccceffd640 1 --2- 172.21.15.39:0/966720547 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e45b3ca070 0x55e45b7756d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:58.219 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:58.215+0000 7fccceffd640 1 -- 172.21.15.39:0/966720547 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e45b3a9390 msgr2=0x55e45b776150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:50:58.219 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:58.215+0000 7fccceffd640 1 --2- 172.21.15.39:0/966720547 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e45b3a9390 0x55e45b776150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:58.219 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:58.215+0000 7fccceffd640 1 -- 172.21.15.39:0/966720547 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e45b435650 con 0x55e45b3a11b0 2024-09-21T03:50:58.223 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:58.215+0000 7fccd4891640 1 -- 172.21.15.39:0/2671641562 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8484+0+0 (secure 0 0 0) 0x7fccc0107d00 con 0x7fccd01049c0 2024-09-21T03:50:58.223 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:58.216+0000 7fccceffd640 1 --2- 172.21.15.39:0/966720547 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55e45b3a11b0 0x55e45b775c10 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7fccc01053e0 tx=0x7fccc0041d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=30ff1ba9b4e6710d server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:50:58.223 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:58.216+0000 7fcccf7fe640 1 --2- 172.21.15.39:0/2671641562 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7fccbc0b6680 0x7fccbc0b8b90 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fccc8009fd0 tx=0x7fccc8004a40 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:50:58.223 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:58.216+0000 7fccc5ffb640 1 -- 172.21.15.39:0/966720547 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccc00cd940 con 0x55e45b3a11b0 2024-09-21T03:50:58.223 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:58.216+0000 7fccc5ffb640 1 -- 172.21.15.39:0/966720547 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fccc00cd680 con 0x55e45b3a11b0 2024-09-21T03:50:58.223 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:58.216+0000 7fccc5ffb640 1 -- 172.21.15.39:0/966720547 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccc0092cf0 con 0x55e45b3a11b0 2024-09-21T03:50:58.223 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:58.217+0000 7fccdbe07740 1 -- 172.21.15.39:0/966720547 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55e45b3a11b0 msgr2=0x55e45b775c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:50:58.223 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:58.217+0000 7fccdbe07740 1 --2- 172.21.15.39:0/966720547 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55e45b3a11b0 0x55e45b775c10 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7fccc01053e0 tx=0x7fccc0041d10 comp rx=0 tx=0).stop 2024-09-21T03:50:58.223 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:58.217+0000 7fccdbe07740 1 -- 172.21.15.39:0/966720547 shutdown_connections 2024-09-21T03:50:58.224 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:58.217+0000 7fccdbe07740 1 --2- 172.21.15.39:0/966720547 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e45b3a9390 0x55e45b776150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:58.224 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:58.217+0000 7fccdbe07740 1 --2- 172.21.15.39:0/966720547 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55e45b3a11b0 0x55e45b775c10 unknown :-1 s=CLOSED pgs=796 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:58.224 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:58.217+0000 7fccdbe07740 1 --2- 172.21.15.39:0/966720547 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e45b3ca070 0x55e45b7756d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:58.224 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:58.217+0000 7fccdbe07740 1 -- 172.21.15.39:0/966720547 >> 172.21.15.39:0/966720547 conn(0x55e45b3be460 msgr2=0x55e45b7735c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:50:58.224 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:58.217+0000 7fccdbe07740 1 -- 172.21.15.39:0/966720547 shutdown_connections 2024-09-21T03:50:58.224 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:58.217+0000 7fccdbe07740 1 -- 172.21.15.39:0/966720547 wait complete. 2024-09-21T03:50:58.224 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:58.217+0000 7fccdbe07740 1 -- start start 2024-09-21T03:50:58.224 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:58.218+0000 7fccdbe07740 1 Processor -- start 2024-09-21T03:50:58.224 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:58.218+0000 7fccdbe07740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e45b3a11b0 0x55e45b780da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:50:58.224 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:58.218+0000 7fccdbe07740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55e45b3a9390 0x55e45b7812e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:50:58.224 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:58.218+0000 7fccdbe07740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e45b3ca070 0x55e45b78ce10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:50:58.224 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:58.218+0000 7fccdbe07740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x55e45b538700 con 0x55e45b3a9390 2024-09-21T03:50:58.224 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:58.218+0000 7fccdbe07740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x55e45b672160 con 0x55e45b3a11b0 2024-09-21T03:50:58.224 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:58.218+0000 7fccdbe07740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55e45b6e7ea0 con 0x55e45b3ca070 2024-09-21T03:50:58.225 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:58.218+0000 7fccdbe07740 10 client.0 _ll_register_callbacks cb 0x55e45b68b9e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:50:58.225 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:58.218+0000 7fccc7fff640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55e45b3a9390 0x55e45b7812e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:50:58.225 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:58.218+0000 7fccc7fff640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55e45b3a9390 0x55e45b7812e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:57636/0 (socket says 172.21.15.39:57636) 2024-09-21T03:50:58.225 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:58.218+0000 7fccc7fff640 1 -- 172.21.15.39:0/2360635378 learned_addr learned my addr 172.21.15.39:0/2360635378 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:50:58.225 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:58.218+0000 7fccceffd640 1 --2- 172.21.15.39:0/2360635378 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e45b3a11b0 0x55e45b780da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:50:58.225 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:58.218+0000 7fcccf7fe640 1 --2- 172.21.15.39:0/2360635378 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e45b3ca070 0x55e45b78ce10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:50:58.225 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:58.218+0000 7fccc7fff640 1 -- 172.21.15.39:0/2360635378 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e45b3ca070 msgr2=0x55e45b78ce10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:50:58.225 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:58.218+0000 7fccc7fff640 1 --2- 172.21.15.39:0/2360635378 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e45b3ca070 0x55e45b78ce10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:58.225 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:58.218+0000 7fccc7fff640 1 -- 172.21.15.39:0/2360635378 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e45b3a11b0 msgr2=0x55e45b780da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:50:58.225 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:58.219+0000 7fccc7fff640 1 --2- 172.21.15.39:0/2360635378 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e45b3a11b0 0x55e45b780da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:58.225 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:58.219+0000 7fccc7fff640 1 -- 172.21.15.39:0/2360635378 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e45b69fc90 con 0x55e45b3a9390 2024-09-21T03:50:58.225 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:58.219+0000 7fccc7fff640 1 --2- 172.21.15.39:0/2360635378 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55e45b3a9390 0x55e45b7812e0 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7fccb8011570 tx=0x7fccb8011a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=47a68f98f1a21d52 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:50:58.225 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:58.219+0000 7fccaeffd640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-09-21T03:50:58.225 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:58.219+0000 7fccaeffd640 1 -- 172.21.15.39:0/2360635378 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccb8013670 con 0x55e45b3a9390 2024-09-21T03:50:58.225 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:58.219+0000 7fccaeffd640 1 -- 172.21.15.39:0/2360635378 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8484+0+0 (secure 0 0 0) 0x7fccb8013810 con 0x55e45b3a9390 2024-09-21T03:50:58.226 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:58.219+0000 7fccdbe07740 1 -- 172.21.15.39:0/2360635378 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fccc00cd680 con 0x55e45b3a9390 2024-09-21T03:50:58.226 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:58.219+0000 7fccaeffd640 1 -- 172.21.15.39:0/2360635378 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fccb8013a50 con 0x55e45b3a9390 2024-09-21T03:50:58.226 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:58.219+0000 7fccdbe07740 10 client.16656 fetch_fsmap learned FSMap version 29 2024-09-21T03:50:58.226 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:58.219+0000 7fccdbe07740 1 -- 172.21.15.39:0/2360635378 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55e45b78f850 con 0x55e45b3a9390 2024-09-21T03:50:58.226 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:58.219+0000 7fccaeffd640 1 -- 172.21.15.39:0/2360635378 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fccc00cd680 con 0x55e45b3a9390 2024-09-21T03:50:58.226 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:58.220+0000 7fccaeffd640 1 -- 172.21.15.39:0/2360635378 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fccb801d400 con 0x55e45b3a9390 2024-09-21T03:50:58.226 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:58.220+0000 7fccdbe07740 10 client.16656 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T03:50:58.226 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:58.220+0000 7fccdbe07740 10 client.16656 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-21T03:50:58.226 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:58.220+0000 7fccdbe07740 20 client.16656 populate_metadata read hostname 'smithi039' 2024-09-21T03:50:58.226 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:58.220+0000 7fccdbe07740 1 --2- 172.21.15.39:0/2360635378 >> [v2:172.21.15.150:6836/4017043144,v1:172.21.15.150:6837/4017043144] conn(0x55e45b3889f0 0x55e45b792d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:50:58.226 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:58.220+0000 7fccdbe07740 4 client.16656 mds_command: new command op to 24659 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:50:58.226 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:58.220+0000 7fccdbe07740 1 -- 172.21.15.39:0/2360635378 --> [v2:172.21.15.150:6836/4017043144,v1:172.21.15.150:6837/4017043144] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e45b6e9df0 con 0x55e45b3889f0 2024-09-21T03:50:58.226 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:58.220+0000 7fccceffd640 1 --2- 172.21.15.39:0/2360635378 >> [v2:172.21.15.150:6836/4017043144,v1:172.21.15.150:6837/4017043144] conn(0x55e45b3889f0 0x55e45b792d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:50:58.226 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:58.220+0000 7fccceffd640 1 --2- 172.21.15.39:0/2360635378 >> [v2:172.21.15.150:6836/4017043144,v1:172.21.15.150:6837/4017043144] conn(0x55e45b3889f0 0x55e45b792d80 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=bcde0476b478fe52 server_cookie=91790722fba559e1 in_seq=0 out_seq=0 2024-09-21T03:50:58.227 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:58.220+0000 7fccaeffd640 10 client.16656 ms_handle_connect on v2:172.21.15.150:6836/4017043144 2024-09-21T03:50:58.227 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:58.221+0000 7fccaeffd640 1 -- 172.21.15.39:0/2360635378 <== mds.2 v2:172.21.15.150:6836/4017043144 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55e45b6e9df0 con 0x55e45b3889f0 2024-09-21T03:50:58.228 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:58.221+0000 7fccaeffd640 10 client.16656 handle_command_reply: tid=0 2024-09-21T03:50:58.228 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:58.221+0000 7fccdbe07740 1 client.16656 shutdown 2024-09-21T03:50:58.228 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:58.221+0000 7fccdbe07740 1 -- 172.21.15.39:0/2360635378 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55e45b3a9390 msgr2=0x55e45b7812e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:50:58.228 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:58.221+0000 7fccdbe07740 1 --2- 172.21.15.39:0/2360635378 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55e45b3a9390 0x55e45b7812e0 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7fccb8011570 tx=0x7fccb8011a30 comp rx=0 tx=0).stop 2024-09-21T03:50:58.228 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:58.221+0000 7fccdbe07740 1 -- 172.21.15.39:0/2360635378 shutdown_connections 2024-09-21T03:50:58.228 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:58.221+0000 7fccdbe07740 1 --2- 172.21.15.39:0/2360635378 >> [v2:172.21.15.150:6836/4017043144,v1:172.21.15.150:6837/4017043144] conn(0x55e45b3889f0 0x55e45b792d80 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:58.228 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:58.221+0000 7fccdbe07740 1 --2- 172.21.15.39:0/2360635378 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e45b3ca070 0x55e45b78ce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:58.228 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:58.221+0000 7fccdbe07740 1 --2- 172.21.15.39:0/2360635378 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55e45b3a9390 0x55e45b7812e0 unknown :-1 s=CLOSED pgs=797 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:58.228 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:58.221+0000 7fccaeffd640 0 client.16656 ms_handle_reset on v2:172.21.15.150:6836/4017043144 2024-09-21T03:50:58.228 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:58.221+0000 7fccdbe07740 1 --2- 172.21.15.39:0/2360635378 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e45b3a11b0 0x55e45b780da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:58.228 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:58.221+0000 7fccdbe07740 1 -- 172.21.15.39:0/2360635378 >> 172.21.15.39:0/2360635378 conn(0x55e45b3be460 msgr2=0x55e45b7770c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:50:58.228 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:58.222+0000 7fccdbe07740 1 -- 172.21.15.39:0/2360635378 shutdown_connections 2024-09-21T03:50:58.228 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:58.222+0000 7fccdbe07740 1 -- 172.21.15.39:0/2360635378 wait complete. 2024-09-21T03:50:58.229 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:58.222+0000 7fccdbe07740 20 client.16656 trim_cache size 0 max 16384 2024-09-21T03:50:58.239 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:58.235+0000 7fccdbe07740 1 Processor -- start 2024-09-21T03:50:58.239 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:58.235+0000 7fccdbe07740 1 -- start start 2024-09-21T03:50:58.239 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:58.235+0000 7fccdbe07740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55e45b3a11b0 0x55e45b78cd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:50:58.239 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:58.235+0000 7fccdbe07740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e45b3a9390 0x55e45b772f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:50:58.239 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:58.235+0000 7fccdbe07740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e45b3ca070 0x55e45b773460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:50:58.239 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:58.235+0000 7fccdbe07740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fccd01ad650 con 0x55e45b3a11b0 2024-09-21T03:50:58.239 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:58.235+0000 7fccdbe07740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fccd01af190 con 0x55e45b3ca070 2024-09-21T03:50:58.239 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:58.235+0000 7fccdbe07740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fccd01add20 con 0x55e45b3a9390 2024-09-21T03:50:58.239 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:58.235+0000 7fccceffd640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55e45b3a11b0 0x55e45b78cd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:50:58.239 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:58.235+0000 7fccceffd640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55e45b3a11b0 0x55e45b78cd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:57646/0 (socket says 172.21.15.39:57646) 2024-09-21T03:50:58.240 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:58.235+0000 7fccceffd640 1 -- 172.21.15.39:0/1814735616 learned_addr learned my addr 172.21.15.39:0/1814735616 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:50:58.240 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:58.235+0000 7fccc7fff640 1 --2- 172.21.15.39:0/1814735616 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e45b3a9390 0x55e45b772f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:50:58.240 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:58.235+0000 7fccceffd640 1 -- 172.21.15.39:0/1814735616 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e45b3a9390 msgr2=0x55e45b772f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:50:58.240 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:58.235+0000 7fccceffd640 1 --2- 172.21.15.39:0/1814735616 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e45b3a9390 0x55e45b772f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:58.240 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:58.235+0000 7fccceffd640 1 -- 172.21.15.39:0/1814735616 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e45b3ca070 msgr2=0x55e45b773460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:50:58.240 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:58.235+0000 7fcccf7fe640 1 --2- 172.21.15.39:0/1814735616 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e45b3ca070 0x55e45b773460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:50:58.240 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:58.236+0000 7fccceffd640 1 --2- 172.21.15.39:0/1814735616 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e45b3ca070 0x55e45b773460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:58.240 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:58.236+0000 7fccceffd640 1 -- 172.21.15.39:0/1814735616 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e45b7779c0 con 0x55e45b3a11b0 2024-09-21T03:50:58.240 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:58.236+0000 7fccc7fff640 1 --2- 172.21.15.39:0/1814735616 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e45b3a9390 0x55e45b772f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:50:58.240 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:58.236+0000 7fcccf7fe640 1 --2- 172.21.15.39:0/1814735616 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e45b3ca070 0x55e45b773460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:50:58.240 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:58.236+0000 7fccceffd640 1 --2- 172.21.15.39:0/1814735616 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55e45b3a11b0 0x55e45b78cd10 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7fccc0000c00 tx=0x7fccc002a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1580ec4bb02f89ae server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:50:58.240 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:58.236+0000 7fccc57fa640 1 -- 172.21.15.39:0/1814735616 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccc00f5020 con 0x55e45b3a11b0 2024-09-21T03:50:58.240 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:58.236+0000 7fccc57fa640 1 -- 172.21.15.39:0/1814735616 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fccc00cd680 con 0x55e45b3a11b0 2024-09-21T03:50:58.241 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:58.236+0000 7fccc57fa640 1 -- 172.21.15.39:0/1814735616 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccc00cd2e0 con 0x55e45b3a11b0 2024-09-21T03:50:58.245 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:58.237+0000 7fccdbe07740 1 -- 172.21.15.39:0/1814735616 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55e45b3a11b0 msgr2=0x55e45b78cd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:50:58.245 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:58.237+0000 7fccdbe07740 1 --2- 172.21.15.39:0/1814735616 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55e45b3a11b0 0x55e45b78cd10 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7fccc0000c00 tx=0x7fccc002a410 comp rx=0 tx=0).stop 2024-09-21T03:50:58.245 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:58.237+0000 7fccdbe07740 1 -- 172.21.15.39:0/1814735616 shutdown_connections 2024-09-21T03:50:58.245 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:58.237+0000 7fccdbe07740 1 --2- 172.21.15.39:0/1814735616 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e45b3ca070 0x55e45b773460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:58.245 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:58.237+0000 7fccdbe07740 1 --2- 172.21.15.39:0/1814735616 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e45b3a9390 0x55e45b772f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:58.245 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:58.237+0000 7fccdbe07740 1 --2- 172.21.15.39:0/1814735616 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55e45b3a11b0 0x55e45b78cd10 unknown :-1 s=CLOSED pgs=798 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:58.245 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:58.237+0000 7fccdbe07740 1 -- 172.21.15.39:0/1814735616 >> 172.21.15.39:0/1814735616 conn(0x55e45b3be460 msgr2=0x55e45b77f620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:50:58.245 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:58.237+0000 7fccdbe07740 1 -- 172.21.15.39:0/1814735616 shutdown_connections 2024-09-21T03:50:58.246 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:58.237+0000 7fccdbe07740 1 -- 172.21.15.39:0/1814735616 wait complete. 2024-09-21T03:50:58.246 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:58.237+0000 7fccdbe07740 1 -- start start 2024-09-21T03:50:58.246 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:58.237+0000 7fccdbe07740 1 Processor -- start 2024-09-21T03:50:58.246 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:58.238+0000 7fccdbe07740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e45b3a11b0 0x55e45b789290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:50:58.246 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:58.238+0000 7fccdbe07740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55e45b3a9390 0x55e45b7857a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:50:58.246 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:58.238+0000 7fccdbe07740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e45b3ca070 0x55e45b785ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:50:58.246 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:58.238+0000 7fccdbe07740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fccd01aeac0 con 0x55e45b3a9390 2024-09-21T03:50:58.246 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:58.238+0000 7fccdbe07740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fccd01ae3f0 con 0x55e45b3ca070 2024-09-21T03:50:58.246 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:58.238+0000 7fccdbe07740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fccd01ab630 con 0x55e45b3a11b0 2024-09-21T03:50:58.246 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:58.238+0000 7fccdbe07740 10 client.0 _ll_register_callbacks cb 0x7fccb80094f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:50:58.246 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:58.238+0000 7fccc7fff640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55e45b3a9390 0x55e45b7857a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:50:58.246 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:58.238+0000 7fccc7fff640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55e45b3a9390 0x55e45b7857a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:57658/0 (socket says 172.21.15.39:57658) 2024-09-21T03:50:58.246 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:58.238+0000 7fccc7fff640 1 -- 172.21.15.39:0/580741597 learned_addr learned my addr 172.21.15.39:0/580741597 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:50:58.246 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:58.238+0000 7fcccf7fe640 1 --2- 172.21.15.39:0/580741597 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e45b3ca070 0x55e45b785ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:50:58.247 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:58.238+0000 7fccceffd640 1 --2- 172.21.15.39:0/580741597 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e45b3a11b0 0x55e45b789290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:50:58.247 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:58.238+0000 7fccc7fff640 1 -- 172.21.15.39:0/580741597 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e45b3a11b0 msgr2=0x55e45b789290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:50:58.247 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:58.238+0000 7fccc7fff640 1 --2- 172.21.15.39:0/580741597 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e45b3a11b0 0x55e45b789290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:58.247 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:58.238+0000 7fccc7fff640 1 -- 172.21.15.39:0/580741597 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e45b3ca070 msgr2=0x55e45b785ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:50:58.247 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:58.238+0000 7fccc7fff640 1 --2- 172.21.15.39:0/580741597 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e45b3ca070 0x55e45b785ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:58.247 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:58.238+0000 7fccc7fff640 1 -- 172.21.15.39:0/580741597 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e45b786220 con 0x55e45b3a9390 2024-09-21T03:50:58.247 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:58.238+0000 7fccceffd640 1 --2- 172.21.15.39:0/580741597 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e45b3a11b0 0x55e45b789290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:50:58.247 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:58.239+0000 7fccc7fff640 1 --2- 172.21.15.39:0/580741597 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55e45b3a9390 0x55e45b7857a0 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7fccb8000c00 tx=0x7fccb801ee90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a9a0cda251bf403 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:50:58.247 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:58.239+0000 7fccaeffd640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-09-21T03:50:58.247 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:58.239+0000 7fccaeffd640 1 -- 172.21.15.39:0/580741597 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccb802a070 con 0x55e45b3a9390 2024-09-21T03:50:58.247 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:58.239+0000 7fccaeffd640 1 -- 172.21.15.39:0/580741597 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8484+0+0 (secure 0 0 0) 0x7fccb8013bf0 con 0x55e45b3a9390 2024-09-21T03:50:58.247 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:58.239+0000 7fccdbe07740 1 -- 172.21.15.39:0/580741597 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fccc00cd680 con 0x55e45b3a9390 2024-09-21T03:50:58.247 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:58.239+0000 7fccaeffd640 1 -- 172.21.15.39:0/580741597 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fccb801dc40 con 0x55e45b3a9390 2024-09-21T03:50:58.247 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:58.239+0000 7fccdbe07740 10 client.16662 fetch_fsmap learned FSMap version 29 2024-09-21T03:50:58.248 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:58.239+0000 7fccdbe07740 1 -- 172.21.15.39:0/580741597 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55e45b7fe2d0 con 0x55e45b3a9390 2024-09-21T03:50:58.248 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:58.239+0000 7fccaeffd640 1 -- 172.21.15.39:0/580741597 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fccc00cd680 con 0x55e45b3a9390 2024-09-21T03:50:58.248 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:58.239+0000 7fccaeffd640 1 -- 172.21.15.39:0/580741597 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fccb80037b0 con 0x55e45b3a9390 2024-09-21T03:50:58.248 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:58.239+0000 7fccdbe07740 10 client.16662 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T03:50:58.248 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:58.239+0000 7fccdbe07740 10 client.16662 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-21T03:50:58.248 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:58.240+0000 7fccdbe07740 20 client.16662 populate_metadata read hostname 'smithi039' 2024-09-21T03:50:58.248 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:58.240+0000 7fccdbe07740 1 --2- 172.21.15.39:0/580741597 >> [v2:172.21.15.150:6836/4017043144,v1:172.21.15.150:6837/4017043144] conn(0x55e45b35dda0 0x55e45b800ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:50:58.248 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:58.240+0000 7fccdbe07740 4 client.16662 mds_command: new command op to 24659 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T03:50:58.248 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:58.240+0000 7fccdbe07740 1 -- 172.21.15.39:0/580741597 --> [v2:172.21.15.150:6836/4017043144,v1:172.21.15.150:6837/4017043144] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55e45b6e58a0 con 0x55e45b35dda0 2024-09-21T03:50:58.248 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:58.240+0000 7fccceffd640 1 --2- 172.21.15.39:0/580741597 >> [v2:172.21.15.150:6836/4017043144,v1:172.21.15.150:6837/4017043144] conn(0x55e45b35dda0 0x55e45b800ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:50:58.248 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:58.240+0000 7fccceffd640 1 --2- 172.21.15.39:0/580741597 >> [v2:172.21.15.150:6836/4017043144,v1:172.21.15.150:6837/4017043144] conn(0x55e45b35dda0 0x55e45b800ac0 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=3fabf6cf670623e server_cookie=5ad5f276bcec5f2c in_seq=0 out_seq=0 2024-09-21T03:50:58.248 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:58.240+0000 7fccaeffd640 10 client.16662 ms_handle_connect on v2:172.21.15.150:6836/4017043144 2024-09-21T03:50:58.249 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:58.241+0000 7fccaeffd640 1 -- 172.21.15.39:0/580741597 <== mds.2 v2:172.21.15.150:6836/4017043144 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55e45b6e58a0 con 0x55e45b35dda0 2024-09-21T03:50:58.249 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:58.241+0000 7fccaeffd640 10 client.16662 handle_command_reply: tid=0 2024-09-21T03:50:58.249 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:58.241+0000 7fccdbe07740 1 client.16662 shutdown 2024-09-21T03:50:58.249 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:58.241+0000 7fccdbe07740 1 -- 172.21.15.39:0/580741597 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55e45b3a9390 msgr2=0x55e45b7857a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:50:58.249 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:58.241+0000 7fccdbe07740 1 --2- 172.21.15.39:0/580741597 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55e45b3a9390 0x55e45b7857a0 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7fccb8000c00 tx=0x7fccb801ee90 comp rx=0 tx=0).stop 2024-09-21T03:50:58.249 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:58.241+0000 7fccdbe07740 1 -- 172.21.15.39:0/580741597 shutdown_connections 2024-09-21T03:50:58.249 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:58.241+0000 7fccdbe07740 1 --2- 172.21.15.39:0/580741597 >> [v2:172.21.15.150:6836/4017043144,v1:172.21.15.150:6837/4017043144] conn(0x55e45b35dda0 0x55e45b800ac0 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:58.249 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:58.241+0000 7fccdbe07740 1 --2- 172.21.15.39:0/580741597 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e45b3ca070 0x55e45b785ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:58.249 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:58.241+0000 7fccdbe07740 1 --2- 172.21.15.39:0/580741597 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55e45b3a9390 0x55e45b7857a0 unknown :-1 s=CLOSED pgs=799 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:58.249 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:58.241+0000 7fccaeffd640 0 client.16662 ms_handle_reset on v2:172.21.15.150:6836/4017043144 2024-09-21T03:50:58.249 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:58.241+0000 7fccdbe07740 1 --2- 172.21.15.39:0/580741597 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e45b3a11b0 0x55e45b789290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:58.249 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:58.241+0000 7fccdbe07740 1 -- 172.21.15.39:0/580741597 >> 172.21.15.39:0/580741597 conn(0x55e45b3be460 msgr2=0x55e45b772f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:50:58.249 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:58.241+0000 7fccdbe07740 1 -- 172.21.15.39:0/580741597 shutdown_connections 2024-09-21T03:50:58.249 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:58.241+0000 7fccdbe07740 1 -- 172.21.15.39:0/580741597 wait complete. 2024-09-21T03:50:58.249 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:58.241+0000 7fccdbe07740 20 client.16662 trim_cache size 0 max 16384 2024-09-21T03:50:58.250 INFO:teuthology.orchestra.run.smithi039.stdout:[] 2024-09-21T03:50:58.251 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:58.242+0000 7fccd5893640 1 -- 172.21.15.39:0/2671641562 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7fccbc0b6680 msgr2=0x7fccbc0b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:50:58.251 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:58.242+0000 7fccd5893640 1 --2- 172.21.15.39:0/2671641562 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7fccbc0b6680 0x7fccbc0b8b90 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fccc8009fd0 tx=0x7fccc8004a40 comp rx=0 tx=0).stop 2024-09-21T03:50:58.251 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:58.242+0000 7fccd5893640 1 -- 172.21.15.39:0/2671641562 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fccd01049c0 msgr2=0x7fccd019a670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:50:58.251 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:58.242+0000 7fccd5893640 1 --2- 172.21.15.39:0/2671641562 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fccd01049c0 0x7fccd019a670 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7fccc0031a90 tx=0x7fccc0004270 comp rx=0 tx=0).stop 2024-09-21T03:50:58.251 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:58.242+0000 7fccd5893640 1 -- 172.21.15.39:0/2671641562 shutdown_connections 2024-09-21T03:50:58.251 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:58.242+0000 7fccd5893640 1 --2- 172.21.15.39:0/2671641562 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7fccbc0b6680 0x7fccbc0b8b90 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:58.251 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:58.242+0000 7fccd5893640 1 --2- 172.21.15.39:0/2671641562 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fccd010de20 0x7fccd019feb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:58.251 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:58.242+0000 7fccd5893640 1 --2- 172.21.15.39:0/2671641562 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fccd0105310 0x7fccd019ac10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:58.251 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:58.242+0000 7fccd5893640 1 --2- 172.21.15.39:0/2671641562 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fccd01049c0 0x7fccd019a670 unknown :-1 s=CLOSED pgs=795 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:58.251 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:58.242+0000 7fccd5893640 1 -- 172.21.15.39:0/2671641562 >> 172.21.15.39:0/2671641562 conn(0x7fccd0079730 msgr2=0x7fccd00ff7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:50:58.251 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:58.242+0000 7fccd5893640 1 -- 172.21.15.39:0/2671641562 shutdown_connections 2024-09-21T03:50:58.251 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:58.242+0000 7fccd5893640 1 -- 172.21.15.39:0/2671641562 wait complete. 2024-09-21T03:50:59.126 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 51116d5c-77ca-11ef-baf5-efdc52797490 -- ceph tell mds.1:1 damage ls 2024-09-21T03:50:59.334 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:50:59 smithi039 ceph-mon[31211]: pgmap v473: 97 pgs: 97 active+clean; 10 MiB data, 536 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 8 op/s 2024-09-21T03:50:59.383 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/51116d5c-77ca-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:50:59.468 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:50:59 smithi161 ceph-mon[34533]: pgmap v473: 97 pgs: 97 active+clean; 10 MiB data, 536 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 8 op/s 2024-09-21T03:50:59.475 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:50:59 smithi150 ceph-mon[34289]: pgmap v473: 97 pgs: 97 active+clean; 10 MiB data, 536 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 8 op/s 2024-09-21T03:50:59.892 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:59.875+0000 7f93bc6ba640 1 -- 172.21.15.39:0/3858681998 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f93b4110200 msgr2=0x7f93b4112620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:50:59.892 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:59.875+0000 7f93bc6ba640 1 --2- 172.21.15.39:0/3858681998 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f93b4110200 0x7f93b4112620 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7f93ac009ca0 tx=0x7f93ac02a950 comp rx=0 tx=0).stop 2024-09-21T03:50:59.892 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:59.875+0000 7f93bc6ba640 1 -- 172.21.15.39:0/3858681998 shutdown_connections 2024-09-21T03:50:59.892 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:59.875+0000 7f93bc6ba640 1 --2- 172.21.15.39:0/3858681998 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f93b4110200 0x7f93b4112620 unknown :-1 s=CLOSED pgs=800 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:59.892 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:59.875+0000 7f93bc6ba640 1 --2- 172.21.15.39:0/3858681998 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f93b4102fa0 0x7f93b410fcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:59.893 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:59.875+0000 7f93bc6ba640 1 --2- 172.21.15.39:0/3858681998 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f93b4102650 0x7f93b4102a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:59.893 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:59.875+0000 7f93bc6ba640 1 -- 172.21.15.39:0/3858681998 >> 172.21.15.39:0/3858681998 conn(0x7f93b4079730 msgr2=0x7f93b4079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:50:59.893 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:59.875+0000 7f93bc6ba640 1 -- 172.21.15.39:0/3858681998 shutdown_connections 2024-09-21T03:50:59.893 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:59.875+0000 7f93bc6ba640 1 -- 172.21.15.39:0/3858681998 wait complete. 2024-09-21T03:50:59.893 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:59.876+0000 7f93bc6ba640 1 Processor -- start 2024-09-21T03:50:59.893 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:59.876+0000 7f93bc6ba640 1 -- start start 2024-09-21T03:50:59.893 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:59.876+0000 7f93bc6ba640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f93b4102650 0x7f93b419c6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:50:59.893 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:59.876+0000 7f93bc6ba640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f93b4102fa0 0x7f93b419cc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:50:59.893 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:59.876+0000 7f93bc6ba640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f93b4110200 0x7f93b41a1f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:50:59.893 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:59.876+0000 7f93bc6ba640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f93b4115cb0 con 0x7f93b4102fa0 2024-09-21T03:50:59.894 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:59.876+0000 7f93bc6ba640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f93b4115b30 con 0x7f93b4102650 2024-09-21T03:50:59.894 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:59.876+0000 7f93bc6ba640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f93b4115e30 con 0x7f93b4110200 2024-09-21T03:50:59.894 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:59.877+0000 7f93b9c55640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f93b4102fa0 0x7f93b419cc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:50:59.894 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:59.877+0000 7f93b9c55640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f93b4102fa0 0x7f93b419cc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:57670/0 (socket says 172.21.15.39:57670) 2024-09-21T03:50:59.894 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:59.877+0000 7f93b9c55640 1 -- 172.21.15.39:0/552503313 learned_addr learned my addr 172.21.15.39:0/552503313 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:50:59.894 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:59.877+0000 7f93bac57640 1 --2- 172.21.15.39:0/552503313 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f93b4110200 0x7f93b41a1f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:50:59.894 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:59.877+0000 7f93ba456640 1 --2- 172.21.15.39:0/552503313 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f93b4102650 0x7f93b419c6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:50:59.894 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:59.877+0000 7f93b9c55640 1 -- 172.21.15.39:0/552503313 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f93b4110200 msgr2=0x7f93b41a1f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:50:59.894 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:59.877+0000 7f93b9c55640 1 --2- 172.21.15.39:0/552503313 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f93b4110200 0x7f93b41a1f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:59.894 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:59.877+0000 7f93b9c55640 1 -- 172.21.15.39:0/552503313 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f93b4102650 msgr2=0x7f93b419c6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:50:59.894 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:59.877+0000 7f93b9c55640 1 --2- 172.21.15.39:0/552503313 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f93b4102650 0x7f93b419c6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:59.895 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:59.877+0000 7f93b9c55640 1 -- 172.21.15.39:0/552503313 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f93b41a2650 con 0x7f93b4102fa0 2024-09-21T03:50:59.895 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:59.877+0000 7f93ba456640 1 --2- 172.21.15.39:0/552503313 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f93b4102650 0x7f93b419c6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:50:59.895 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:59.877+0000 7f93b9c55640 1 --2- 172.21.15.39:0/552503313 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f93b4102fa0 0x7f93b419cc80 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7f93a400c970 tx=0x7f93a400ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:50:59.895 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:59.877+0000 7f93ab7fe640 1 -- 172.21.15.39:0/552503313 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93a4007bb0 con 0x7f93b4102fa0 2024-09-21T03:50:59.895 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:59.878+0000 7f93bc6ba640 1 -- 172.21.15.39:0/552503313 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f93b41a2940 con 0x7f93b4102fa0 2024-09-21T03:50:59.895 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:59.878+0000 7f93ab7fe640 1 -- 172.21.15.39:0/552503313 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f93a4007d50 con 0x7f93b4102fa0 2024-09-21T03:50:59.895 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:59.878+0000 7f93ab7fe640 1 -- 172.21.15.39:0/552503313 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93a400f450 con 0x7f93b4102fa0 2024-09-21T03:50:59.895 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:59.878+0000 7f93bc6ba640 1 -- 172.21.15.39:0/552503313 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f93b41a90a0 con 0x7f93b4102fa0 2024-09-21T03:50:59.895 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:59.880+0000 7f93c2c2e740 1 Processor -- start 2024-09-21T03:50:59.895 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:59.880+0000 7f93c2c2e740 1 -- start start 2024-09-21T03:50:59.895 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:59.881+0000 7f93c2c2e740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b73255d070 0x55b7329046a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:50:59.895 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:59.881+0000 7f93c2c2e740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b7325341b0 0x55b732904be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:50:59.895 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:59.881+0000 7f93c2c2e740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b73253c390 0x55b732905120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:50:59.895 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:59.881+0000 7f93c2c2e740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x55b73287aea0 con 0x55b73253c390 2024-09-21T03:50:59.895 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:59.881+0000 7f93c2c2e740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x55b73286a2d0 con 0x55b73255d070 2024-09-21T03:50:59.896 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:59.881+0000 7f93c2c2e740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55b73286e0e0 con 0x55b7325341b0 2024-09-21T03:50:59.896 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:59.881+0000 7f93ab7fe640 1 -- 172.21.15.39:0/552503313 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 27) ==== 154643+0+0 (secure 0 0 0) 0x7f93a4004f20 con 0x7f93b4102fa0 2024-09-21T03:50:59.896 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:59.881+0000 7f93ba456640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b7325341b0 0x55b732904be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:50:59.896 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:59.881+0000 7f93b9c55640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b73253c390 0x55b732905120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:50:59.896 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:59.881+0000 7f93ba456640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b7325341b0 0x55b732904be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.39:58582/0 (socket says 172.21.15.39:58582) 2024-09-21T03:50:59.896 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:59.881+0000 7f93ba456640 1 -- 172.21.15.39:0/1294954624 learned_addr learned my addr 172.21.15.39:0/1294954624 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:50:59.896 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:59.881+0000 7f93ba456640 1 -- 172.21.15.39:0/1294954624 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b73255d070 msgr2=0x55b7329046a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:50:59.896 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:59.881+0000 7f93ba456640 1 --2- 172.21.15.39:0/1294954624 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b73255d070 0x55b7329046a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:59.896 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:59.881+0000 7f93ba456640 1 -- 172.21.15.39:0/1294954624 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b73253c390 msgr2=0x55b732905120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:50:59.896 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:59.881+0000 7f93ba456640 1 --2- 172.21.15.39:0/1294954624 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b73253c390 0x55b732905120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:59.896 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:59.881+0000 7f93ba456640 1 -- 172.21.15.39:0/1294954624 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b7325c8650 con 0x55b7325341b0 2024-09-21T03:50:59.896 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:59.881+0000 7f93b9c55640 1 --2- 172.21.15.39:0/1294954624 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b73253c390 0x55b732905120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:50:59.896 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:59.882+0000 7f93ba456640 1 --2- 172.21.15.39:0/1294954624 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b7325341b0 0x55b732904be0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f93b0007850 tx=0x7f93b0007d10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=95f4f212b0b017db server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:50:59.897 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:59.882+0000 7f93ab7fe640 1 --2- 172.21.15.39:0/552503313 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f93a00b6680 0x7f93a00b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:50:59.897 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:59.882+0000 7f93bac57640 1 --2- 172.21.15.39:0/552503313 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f93a00b6680 0x7f93a00b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:50:59.897 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:59.882+0000 7f93a8ff9640 1 -- 172.21.15.39:0/1294954624 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93b000d950 con 0x55b7325341b0 2024-09-21T03:50:59.897 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:59.882+0000 7f93ab7fe640 1 -- 172.21.15.39:0/552503313 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8484+0+0 (secure 0 0 0) 0x7f93a40e7410 con 0x7f93b4102fa0 2024-09-21T03:50:59.897 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:59.882+0000 7f93a8ff9640 1 -- 172.21.15.39:0/1294954624 <== mon.1 v2:172.21.15.161:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f93b0018070 con 0x55b7325341b0 2024-09-21T03:50:59.897 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:59.882+0000 7f93a8ff9640 1 -- 172.21.15.39:0/1294954624 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93b0013de0 con 0x55b7325341b0 2024-09-21T03:50:59.897 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:59.883+0000 7f93c2c2e740 1 -- 172.21.15.39:0/1294954624 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b7325341b0 msgr2=0x55b732904be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:50:59.897 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:59.883+0000 7f93bac57640 1 --2- 172.21.15.39:0/552503313 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f93a00b6680 0x7f93a00b8b90 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f93ac0098d0 tx=0x7f93ac009980 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:50:59.897 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:59.883+0000 7f93c2c2e740 1 --2- 172.21.15.39:0/1294954624 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b7325341b0 0x55b732904be0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f93b0007850 tx=0x7f93b0007d10 comp rx=0 tx=0).stop 2024-09-21T03:50:59.897 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:59.883+0000 7f93c2c2e740 1 -- 172.21.15.39:0/1294954624 shutdown_connections 2024-09-21T03:50:59.897 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:59.883+0000 7f93c2c2e740 1 --2- 172.21.15.39:0/1294954624 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b73253c390 0x55b732905120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:59.897 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:59.883+0000 7f93c2c2e740 1 --2- 172.21.15.39:0/1294954624 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b7325341b0 0x55b732904be0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:59.897 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:59.883+0000 7f93c2c2e740 1 --2- 172.21.15.39:0/1294954624 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b73255d070 0x55b7329046a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:59.897 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:59.883+0000 7f93c2c2e740 1 -- 172.21.15.39:0/1294954624 >> 172.21.15.39:0/1294954624 conn(0x55b732551460 msgr2=0x55b732902590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:50:59.897 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:59.883+0000 7f93c2c2e740 1 -- 172.21.15.39:0/1294954624 shutdown_connections 2024-09-21T03:50:59.898 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:59.883+0000 7f93c2c2e740 1 -- 172.21.15.39:0/1294954624 wait complete. 2024-09-21T03:50:59.898 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:59.884+0000 7f93c2c2e740 1 -- start start 2024-09-21T03:50:59.898 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:59.884+0000 7f93c2c2e740 1 Processor -- start 2024-09-21T03:50:59.898 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:59.884+0000 7f93c2c2e740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b7325341b0 0x55b73290fcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:50:59.898 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:59.884+0000 7f93c2c2e740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b73253c390 0x55b7329101f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:50:59.898 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:59.884+0000 7f93c2c2e740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b73255d070 0x55b732910750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:50:59.898 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:59.884+0000 7f93c2c2e740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x55b7326cb700 con 0x55b73253c390 2024-09-21T03:50:59.898 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:59.884+0000 7f93c2c2e740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x55b732805160 con 0x55b7325341b0 2024-09-21T03:50:59.898 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:59.884+0000 7f93c2c2e740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55b732912e80 con 0x55b73255d070 2024-09-21T03:50:59.898 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:59.884+0000 7f93c2c2e740 10 client.0 _ll_register_callbacks cb 0x55b73281e9e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:50:59.898 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:59.884+0000 7f93b9c55640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b73253c390 0x55b7329101f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:50:59.898 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:59.884+0000 7f93b9c55640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b73253c390 0x55b7329101f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:57694/0 (socket says 172.21.15.39:57694) 2024-09-21T03:50:59.898 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:59.884+0000 7f93b9c55640 1 -- 172.21.15.39:0/3741248772 learned_addr learned my addr 172.21.15.39:0/3741248772 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:50:59.899 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:59.884+0000 7f93ba456640 1 --2- 172.21.15.39:0/3741248772 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b7325341b0 0x55b73290fcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:50:59.899 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:59.884+0000 7f93bac57640 1 --2- 172.21.15.39:0/3741248772 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b73255d070 0x55b732910750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:50:59.899 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:59.884+0000 7f93b9c55640 1 -- 172.21.15.39:0/3741248772 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b73255d070 msgr2=0x55b732910750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:50:59.899 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:59.884+0000 7f93b9c55640 1 --2- 172.21.15.39:0/3741248772 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b73255d070 0x55b732910750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:59.899 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:59.885+0000 7f93b9c55640 1 -- 172.21.15.39:0/3741248772 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b7325341b0 msgr2=0x55b73290fcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:50:59.899 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:59.885+0000 7f93b9c55640 1 --2- 172.21.15.39:0/3741248772 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b7325341b0 0x55b73290fcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:59.899 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:59.885+0000 7f93b9c55640 1 -- 172.21.15.39:0/3741248772 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b732832c90 con 0x55b73253c390 2024-09-21T03:50:59.899 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:59.885+0000 7f93bac57640 1 --2- 172.21.15.39:0/3741248772 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b73255d070 0x55b732910750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:50:59.899 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:59.885+0000 7f93b9c55640 1 --2- 172.21.15.39:0/3741248772 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b73253c390 0x55b7329101f0 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7f93a40e4be0 tx=0x7f93a400cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b1e827c7399d825 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:50:59.899 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:59.885+0000 7f939dffb640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-09-21T03:50:59.899 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:59.885+0000 7f939dffb640 1 -- 172.21.15.39:0/3741248772 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93a401d040 con 0x55b73253c390 2024-09-21T03:50:59.899 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:59.885+0000 7f939dffb640 1 -- 172.21.15.39:0/3741248772 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8484+0+0 (secure 0 0 0) 0x7f93a40a8dd0 con 0x55b73253c390 2024-09-21T03:50:59.899 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:59.885+0000 7f93c2c2e740 1 -- 172.21.15.39:0/3741248772 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f93b0018070 con 0x55b73253c390 2024-09-21T03:50:59.899 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:59.885+0000 7f939dffb640 1 -- 172.21.15.39:0/3741248772 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f93a40a9120 con 0x55b73253c390 2024-09-21T03:50:59.900 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:59.885+0000 7f93c2c2e740 10 client.16674 fetch_fsmap learned FSMap version 29 2024-09-21T03:50:59.900 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:59.885+0000 7f93c2c2e740 1 -- 172.21.15.39:0/3741248772 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55b73291eb30 con 0x55b73253c390 2024-09-21T03:50:59.900 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:59.885+0000 7f939dffb640 1 -- 172.21.15.39:0/3741248772 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f93b0018070 con 0x55b73253c390 2024-09-21T03:50:59.900 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:59.885+0000 7f939dffb640 1 -- 172.21.15.39:0/3741248772 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f93a40b2ef0 con 0x55b73253c390 2024-09-21T03:50:59.900 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:59.885+0000 7f93c2c2e740 10 client.16674 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T03:50:59.900 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:59.885+0000 7f93c2c2e740 10 client.16674 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-21T03:50:59.900 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:59.886+0000 7f93c2c2e740 20 client.16674 populate_metadata read hostname 'smithi039' 2024-09-21T03:50:59.900 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:59.886+0000 7f93c2c2e740 1 --2- 172.21.15.39:0/3741248772 >> [v2:172.21.15.161:6838/2190107270,v1:172.21.15.161:6839/2190107270] conn(0x55b73251b9f0 0x55b732921fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:50:59.900 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:59.886+0000 7f93c2c2e740 4 client.16674 mds_command: new command op to 24667 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:50:59.900 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:59.886+0000 7f93c2c2e740 1 -- 172.21.15.39:0/3741248772 --> [v2:172.21.15.161:6838/2190107270,v1:172.21.15.161:6839/2190107270] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b73287cdf0 con 0x55b73251b9f0 2024-09-21T03:50:59.900 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:59.886+0000 7f93ba456640 1 --2- 172.21.15.39:0/3741248772 >> [v2:172.21.15.161:6838/2190107270,v1:172.21.15.161:6839/2190107270] conn(0x55b73251b9f0 0x55b732921fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:50:59.900 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:59.886+0000 7f93ba456640 1 --2- 172.21.15.39:0/3741248772 >> [v2:172.21.15.161:6838/2190107270,v1:172.21.15.161:6839/2190107270] conn(0x55b73251b9f0 0x55b732921fa0 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=1eacb754bfbddeb0 server_cookie=fd46d4947545465c in_seq=0 out_seq=0 2024-09-21T03:50:59.900 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:59.886+0000 7f939dffb640 10 client.16674 ms_handle_connect on v2:172.21.15.161:6838/2190107270 2024-09-21T03:50:59.900 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:59.887+0000 7f939dffb640 1 -- 172.21.15.39:0/3741248772 <== mds.1 v2:172.21.15.161:6838/2190107270 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55b73287cdf0 con 0x55b73251b9f0 2024-09-21T03:50:59.900 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:59.887+0000 7f939dffb640 10 client.16674 handle_command_reply: tid=0 2024-09-21T03:50:59.901 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:59.887+0000 7f93c2c2e740 1 client.16674 shutdown 2024-09-21T03:50:59.901 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:59.887+0000 7f93c2c2e740 1 -- 172.21.15.39:0/3741248772 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b73253c390 msgr2=0x55b7329101f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:50:59.901 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:59.887+0000 7f93c2c2e740 1 --2- 172.21.15.39:0/3741248772 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b73253c390 0x55b7329101f0 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7f93a40e4be0 tx=0x7f93a400cf60 comp rx=0 tx=0).stop 2024-09-21T03:50:59.901 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:59.887+0000 7f93c2c2e740 1 -- 172.21.15.39:0/3741248772 shutdown_connections 2024-09-21T03:50:59.901 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:59.887+0000 7f93c2c2e740 1 --2- 172.21.15.39:0/3741248772 >> [v2:172.21.15.161:6838/2190107270,v1:172.21.15.161:6839/2190107270] conn(0x55b73251b9f0 0x55b732921fa0 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:59.901 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:59.887+0000 7f93c2c2e740 1 --2- 172.21.15.39:0/3741248772 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b73255d070 0x55b732910750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:59.901 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:59.887+0000 7f93c2c2e740 1 --2- 172.21.15.39:0/3741248772 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b73253c390 0x55b7329101f0 unknown :-1 s=CLOSED pgs=802 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:59.901 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:59.887+0000 7f939dffb640 0 client.16674 ms_handle_reset on v2:172.21.15.161:6838/2190107270 2024-09-21T03:50:59.901 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:59.887+0000 7f93c2c2e740 1 --2- 172.21.15.39:0/3741248772 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b7325341b0 0x55b73290fcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:59.901 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:59.887+0000 7f93c2c2e740 1 -- 172.21.15.39:0/3741248772 >> 172.21.15.39:0/3741248772 conn(0x55b732551460 msgr2=0x55b732907830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:50:59.901 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:59.887+0000 7f93c2c2e740 1 -- 172.21.15.39:0/3741248772 shutdown_connections 2024-09-21T03:50:59.901 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:59.887+0000 7f93c2c2e740 1 -- 172.21.15.39:0/3741248772 wait complete. 2024-09-21T03:50:59.901 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:59.888+0000 7f93c2c2e740 20 client.16674 trim_cache size 0 max 16384 2024-09-21T03:50:59.904 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:59.901+0000 7f93c2c2e740 1 Processor -- start 2024-09-21T03:50:59.904 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:59.901+0000 7f93c2c2e740 1 -- start start 2024-09-21T03:50:59.904 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:59.901+0000 7f93c2c2e740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b7325341b0 0x55b732906090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:50:59.904 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:59.901+0000 7f93c2c2e740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b73253c390 0x55b7329118e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:50:59.904 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:59.901+0000 7f93c2c2e740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b73255d070 0x55b732901bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:50:59.905 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:59.901+0000 7f93c2c2e740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f93b41b1980 con 0x55b73255d070 2024-09-21T03:50:59.905 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:59.901+0000 7f93c2c2e740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f93b41b34c0 con 0x55b73253c390 2024-09-21T03:50:59.905 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:59.901+0000 7f93c2c2e740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f93b41b2050 con 0x55b7325341b0 2024-09-21T03:50:59.910 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:59.902+0000 7f93bac57640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b73255d070 0x55b732901bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:50:59.910 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:59.902+0000 7f93bac57640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b73255d070 0x55b732901bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:57696/0 (socket says 172.21.15.39:57696) 2024-09-21T03:50:59.910 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:59.902+0000 7f93bac57640 1 -- 172.21.15.39:0/1528081816 learned_addr learned my addr 172.21.15.39:0/1528081816 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:50:59.910 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:59.902+0000 7f93bac57640 1 -- 172.21.15.39:0/1528081816 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b7325341b0 msgr2=0x55b732906090 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-21T03:50:59.910 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:59.902+0000 7f93bac57640 1 --2- 172.21.15.39:0/1528081816 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b7325341b0 0x55b732906090 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:59.910 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:59.902+0000 7f93bac57640 1 -- 172.21.15.39:0/1528081816 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b73253c390 msgr2=0x55b7329118e0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-21T03:50:59.910 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:59.902+0000 7f93bac57640 1 --2- 172.21.15.39:0/1528081816 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b73253c390 0x55b7329118e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:59.910 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:59.902+0000 7f93bac57640 1 -- 172.21.15.39:0/1528081816 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b73291c0a0 con 0x55b73255d070 2024-09-21T03:50:59.911 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:59.902+0000 7f93bac57640 1 --2- 172.21.15.39:0/1528081816 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b73255d070 0x55b732901bc0 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7f93ac009ca0 tx=0x7f93ac03f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b6a50f5b42a5b6b server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:50:59.911 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:59.902+0000 7f939ffff640 1 -- 172.21.15.39:0/1528081816 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93ac034040 con 0x55b73255d070 2024-09-21T03:50:59.911 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:59.902+0000 7f939ffff640 1 -- 172.21.15.39:0/1528081816 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f93ac004040 con 0x55b73255d070 2024-09-21T03:50:59.911 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:59.902+0000 7f939ffff640 1 -- 172.21.15.39:0/1528081816 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93ac030680 con 0x55b73255d070 2024-09-21T03:50:59.911 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:59.903+0000 7f93c2c2e740 1 -- 172.21.15.39:0/1528081816 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b73255d070 msgr2=0x55b732901bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:50:59.911 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:59.903+0000 7f93c2c2e740 1 --2- 172.21.15.39:0/1528081816 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b73255d070 0x55b732901bc0 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7f93ac009ca0 tx=0x7f93ac03f000 comp rx=0 tx=0).stop 2024-09-21T03:50:59.911 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:59.903+0000 7f93c2c2e740 1 -- 172.21.15.39:0/1528081816 shutdown_connections 2024-09-21T03:50:59.911 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:59.903+0000 7f93c2c2e740 1 --2- 172.21.15.39:0/1528081816 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b73255d070 0x55b732901bc0 unknown :-1 s=CLOSED pgs=803 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:59.911 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:59.903+0000 7f93c2c2e740 1 --2- 172.21.15.39:0/1528081816 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b73253c390 0x55b7329118e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:59.911 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:59.903+0000 7f93c2c2e740 1 --2- 172.21.15.39:0/1528081816 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b7325341b0 0x55b732906090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:59.911 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:59.903+0000 7f93c2c2e740 1 -- 172.21.15.39:0/1528081816 >> 172.21.15.39:0/1528081816 conn(0x55b732551460 msgr2=0x55b7329111d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:50:59.911 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:59.903+0000 7f93c2c2e740 1 -- 172.21.15.39:0/1528081816 shutdown_connections 2024-09-21T03:50:59.912 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:59.903+0000 7f93c2c2e740 1 -- 172.21.15.39:0/1528081816 wait complete. 2024-09-21T03:50:59.912 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:59.904+0000 7f93c2c2e740 1 -- start start 2024-09-21T03:50:59.912 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:59.904+0000 7f93c2c2e740 1 Processor -- start 2024-09-21T03:50:59.912 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:59.904+0000 7f93c2c2e740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b7325341b0 0x55b7329189f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:50:59.912 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:59.904+0000 7f93c2c2e740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b73253c390 0x55b732913ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:50:59.912 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:59.904+0000 7f93c2c2e740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b73255d070 0x55b7329144f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:50:59.912 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:59.904+0000 7f93c2c2e740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f93b41b2df0 con 0x55b7325341b0 2024-09-21T03:50:59.912 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:59.904+0000 7f93c2c2e740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f93b41b2720 con 0x55b73253c390 2024-09-21T03:50:59.912 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:59.904+0000 7f93c2c2e740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f93b41af960 con 0x55b73255d070 2024-09-21T03:50:59.912 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:59.904+0000 7f93c2c2e740 10 client.0 _ll_register_callbacks cb 0x7f93a400c8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:50:59.912 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:59.904+0000 7f93ba456640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b7325341b0 0x55b7329189f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:50:59.913 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:59.904+0000 7f93ba456640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b7325341b0 0x55b7329189f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:57698/0 (socket says 172.21.15.39:57698) 2024-09-21T03:50:59.913 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:59.904+0000 7f93ba456640 1 -- 172.21.15.39:0/731383477 learned_addr learned my addr 172.21.15.39:0/731383477 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:50:59.913 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:59.904+0000 7f93b9c55640 1 --2- 172.21.15.39:0/731383477 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b73253c390 0x55b732913ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:50:59.913 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:59.904+0000 7f93bac57640 1 --2- 172.21.15.39:0/731383477 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b73255d070 0x55b7329144f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:50:59.913 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:59.904+0000 7f93ba456640 1 -- 172.21.15.39:0/731383477 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b73255d070 msgr2=0x55b7329144f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:50:59.913 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:59.904+0000 7f93ba456640 1 --2- 172.21.15.39:0/731383477 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b73255d070 0x55b7329144f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:59.913 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:59.904+0000 7f93ba456640 1 -- 172.21.15.39:0/731383477 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b73253c390 msgr2=0x55b732913ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:50:59.913 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:59.904+0000 7f93ba456640 1 --2- 172.21.15.39:0/731383477 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b73253c390 0x55b732913ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:59.913 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:59.904+0000 7f93ba456640 1 -- 172.21.15.39:0/731383477 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b732914c80 con 0x55b7325341b0 2024-09-21T03:50:59.913 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:59.905+0000 7f93bac57640 1 --2- 172.21.15.39:0/731383477 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b73255d070 0x55b7329144f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:50:59.913 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:59.905+0000 7f93b9c55640 1 --2- 172.21.15.39:0/731383477 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b73253c390 0x55b732913ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:50:59.913 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:59.905+0000 7f93ba456640 1 --2- 172.21.15.39:0/731383477 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b7325341b0 0x55b7329189f0 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x55b732903320 tx=0x7f93b0006f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f7c67665adf83bc7 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:50:59.913 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:59.905+0000 7f939dffb640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-09-21T03:50:59.914 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:59.905+0000 7f939dffb640 1 -- 172.21.15.39:0/731383477 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93b000d950 con 0x55b7325341b0 2024-09-21T03:50:59.914 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:59.905+0000 7f93c2c2e740 1 -- 172.21.15.39:0/731383477 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f93ac004040 con 0x55b7325341b0 2024-09-21T03:50:59.914 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:59.905+0000 7f939dffb640 1 -- 172.21.15.39:0/731383477 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8484+0+0 (secure 0 0 0) 0x7f93b000daf0 con 0x55b7325341b0 2024-09-21T03:50:59.914 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:59.905+0000 7f939dffb640 1 -- 172.21.15.39:0/731383477 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f93b000dcf0 con 0x55b7325341b0 2024-09-21T03:50:59.914 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:59.905+0000 7f93c2c2e740 10 client.16680 fetch_fsmap learned FSMap version 29 2024-09-21T03:50:59.914 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:59.905+0000 7f93c2c2e740 1 -- 172.21.15.39:0/731383477 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55b73298ca40 con 0x55b7325341b0 2024-09-21T03:50:59.914 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:59.905+0000 7f939dffb640 1 -- 172.21.15.39:0/731383477 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f93ac004040 con 0x55b7325341b0 2024-09-21T03:50:59.914 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:59.905+0000 7f939dffb640 1 -- 172.21.15.39:0/731383477 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f93b0028850 con 0x55b7325341b0 2024-09-21T03:50:59.914 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:59.906+0000 7f93c2c2e740 10 client.16680 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T03:50:59.914 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:59.906+0000 7f93c2c2e740 10 client.16680 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-21T03:50:59.914 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:59.906+0000 7f93c2c2e740 20 client.16680 populate_metadata read hostname 'smithi039' 2024-09-21T03:50:59.914 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:59.906+0000 7f93c2c2e740 1 --2- 172.21.15.39:0/731383477 >> [v2:172.21.15.161:6838/2190107270,v1:172.21.15.161:6839/2190107270] conn(0x55b7324f0da0 0x55b73298efd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:50:59.915 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:59.906+0000 7f93c2c2e740 4 client.16680 mds_command: new command op to 24667 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T03:50:59.915 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:59.906+0000 7f93c2c2e740 1 -- 172.21.15.39:0/731383477 --> [v2:172.21.15.161:6838/2190107270,v1:172.21.15.161:6839/2190107270] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55b7328788a0 con 0x55b7324f0da0 2024-09-21T03:50:59.915 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:59.906+0000 7f93b9c55640 1 --2- 172.21.15.39:0/731383477 >> [v2:172.21.15.161:6838/2190107270,v1:172.21.15.161:6839/2190107270] conn(0x55b7324f0da0 0x55b73298efd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:50:59.915 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:59.906+0000 7f939dffb640 10 client.16680 ms_handle_connect on v2:172.21.15.161:6838/2190107270 2024-09-21T03:50:59.915 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:59.906+0000 7f93b9c55640 1 --2- 172.21.15.39:0/731383477 >> [v2:172.21.15.161:6838/2190107270,v1:172.21.15.161:6839/2190107270] conn(0x55b7324f0da0 0x55b73298efd0 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=bb401771ec88ea37 server_cookie=205c0d5ff5746098 in_seq=0 out_seq=0 2024-09-21T03:50:59.915 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:59.907+0000 7f939dffb640 1 -- 172.21.15.39:0/731383477 <== mds.1 v2:172.21.15.161:6838/2190107270 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55b7328788a0 con 0x55b7324f0da0 2024-09-21T03:50:59.915 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:59.907+0000 7f939dffb640 10 client.16680 handle_command_reply: tid=0 2024-09-21T03:50:59.915 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:59.907+0000 7f93c2c2e740 1 client.16680 shutdown 2024-09-21T03:50:59.915 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:59.907+0000 7f93c2c2e740 1 -- 172.21.15.39:0/731383477 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b7325341b0 msgr2=0x55b7329189f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:50:59.915 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:59.907+0000 7f93c2c2e740 1 --2- 172.21.15.39:0/731383477 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b7325341b0 0x55b7329189f0 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x55b732903320 tx=0x7f93b0006f70 comp rx=0 tx=0).stop 2024-09-21T03:50:59.915 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:59.907+0000 7f93c2c2e740 1 -- 172.21.15.39:0/731383477 shutdown_connections 2024-09-21T03:50:59.915 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:59.907+0000 7f93c2c2e740 1 --2- 172.21.15.39:0/731383477 >> [v2:172.21.15.161:6838/2190107270,v1:172.21.15.161:6839/2190107270] conn(0x55b7324f0da0 0x55b73298efd0 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:59.916 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:59.907+0000 7f93c2c2e740 1 --2- 172.21.15.39:0/731383477 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b73255d070 0x55b7329144f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:59.916 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:59.907+0000 7f93c2c2e740 1 --2- 172.21.15.39:0/731383477 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b73253c390 0x55b732913ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:59.916 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:59.907+0000 7f93c2c2e740 1 --2- 172.21.15.39:0/731383477 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b7325341b0 0x55b7329189f0 unknown :-1 s=CLOSED pgs=804 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:59.916 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:59.907+0000 7f939dffb640 0 client.16680 ms_handle_reset on v2:172.21.15.161:6838/2190107270 2024-09-21T03:50:59.916 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:59.907+0000 7f93c2c2e740 1 -- 172.21.15.39:0/731383477 >> 172.21.15.39:0/731383477 conn(0x55b732551460 msgr2=0x55b732905e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:50:59.916 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:59.907+0000 7f93c2c2e740 1 -- 172.21.15.39:0/731383477 shutdown_connections 2024-09-21T03:50:59.916 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:59.907+0000 7f93c2c2e740 1 -- 172.21.15.39:0/731383477 wait complete. 2024-09-21T03:50:59.916 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:59.907+0000 7f93c2c2e740 20 client.16680 trim_cache size 0 max 16384 2024-09-21T03:50:59.916 INFO:teuthology.orchestra.run.smithi039.stdout:[] 2024-09-21T03:50:59.917 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:59.907+0000 7f93bc6ba640 1 -- 172.21.15.39:0/552503313 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f93a00b6680 msgr2=0x7f93a00b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:50:59.917 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:59.907+0000 7f93bc6ba640 1 --2- 172.21.15.39:0/552503313 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f93a00b6680 0x7f93a00b8b90 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f93ac0098d0 tx=0x7f93ac009980 comp rx=0 tx=0).stop 2024-09-21T03:50:59.917 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:59.907+0000 7f93bc6ba640 1 -- 172.21.15.39:0/552503313 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f93b4102fa0 msgr2=0x7f93b419cc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:50:59.917 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:59.907+0000 7f93bc6ba640 1 --2- 172.21.15.39:0/552503313 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f93b4102fa0 0x7f93b419cc80 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7f93a400c970 tx=0x7f93a400ce30 comp rx=0 tx=0).stop 2024-09-21T03:50:59.917 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:59.908+0000 7f93bc6ba640 1 -- 172.21.15.39:0/552503313 shutdown_connections 2024-09-21T03:50:59.917 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:59.908+0000 7f93bc6ba640 1 --2- 172.21.15.39:0/552503313 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f93a00b6680 0x7f93a00b8b90 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:59.917 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:59.908+0000 7f93bc6ba640 1 --2- 172.21.15.39:0/552503313 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f93b4110200 0x7f93b41a1f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:59.918 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:59.908+0000 7f93bc6ba640 1 --2- 172.21.15.39:0/552503313 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f93b4102fa0 0x7f93b419cc80 unknown :-1 s=CLOSED pgs=801 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:59.918 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:59.908+0000 7f93bc6ba640 1 --2- 172.21.15.39:0/552503313 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f93b4102650 0x7f93b419c6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:59.918 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:59.908+0000 7f93bc6ba640 1 -- 172.21.15.39:0/552503313 >> 172.21.15.39:0/552503313 conn(0x7f93b4079730 msgr2=0x7f93b4109d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:50:59.918 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:59.908+0000 7f93bc6ba640 1 -- 172.21.15.39:0/552503313 shutdown_connections 2024-09-21T03:50:59.918 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:50:59.908+0000 7f93bc6ba640 1 -- 172.21.15.39:0/552503313 wait complete. 2024-09-21T03:51:00.740 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 51116d5c-77ca-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 damage ls 2024-09-21T03:51:00.997 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/51116d5c-77ca-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:51:01.277 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:51:01 smithi039 ceph-mon[31211]: pgmap v474: 97 pgs: 97 active+clean; 10 MiB data, 536 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 11 op/s 2024-09-21T03:51:01.468 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:51:01 smithi161 ceph-mon[34533]: pgmap v474: 97 pgs: 97 active+clean; 10 MiB data, 536 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 11 op/s 2024-09-21T03:51:01.475 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:51:01 smithi150 ceph-mon[34289]: pgmap v474: 97 pgs: 97 active+clean; 10 MiB data, 536 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 11 op/s 2024-09-21T03:51:01.476 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:01.469+0000 7fad173e3640 1 -- 172.21.15.39:0/3648647638 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fad101051c0 msgr2=0x7fad101055d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:51:01.476 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:01.469+0000 7fad173e3640 1 --2- 172.21.15.39:0/3648647638 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fad101051c0 0x7fad101055d0 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7fad04009900 tx=0x7fad0402a410 comp rx=0 tx=0).stop 2024-09-21T03:51:01.476 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:01.469+0000 7fad173e3640 1 -- 172.21.15.39:0/3648647638 shutdown_connections 2024-09-21T03:51:01.477 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:01.469+0000 7fad173e3640 1 --2- 172.21.15.39:0/3648647638 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fad10105b10 0x7fad1010f580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:01.477 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:01.469+0000 7fad173e3640 1 --2- 172.21.15.39:0/3648647638 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fad101051c0 0x7fad101055d0 unknown :-1 s=CLOSED pgs=805 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:01.477 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:01.469+0000 7fad173e3640 1 --2- 172.21.15.39:0/3648647638 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fad101031b0 0x7fad101035c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:01.477 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:01.469+0000 7fad173e3640 1 -- 172.21.15.39:0/3648647638 >> 172.21.15.39:0/3648647638 conn(0x7fad10079f90 msgr2=0x7fad1007a3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:51:01.477 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:01.469+0000 7fad173e3640 1 -- 172.21.15.39:0/3648647638 shutdown_connections 2024-09-21T03:51:01.477 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:01.469+0000 7fad173e3640 1 -- 172.21.15.39:0/3648647638 wait complete. 2024-09-21T03:51:01.477 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:01.470+0000 7fad173e3640 1 Processor -- start 2024-09-21T03:51:01.477 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:01.470+0000 7fad173e3640 1 -- start start 2024-09-21T03:51:01.478 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:01.470+0000 7fad173e3640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fad101031b0 0x7fad1019a590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:51:01.478 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:01.470+0000 7fad173e3640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fad101051c0 0x7fad1019ab30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:51:01.478 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:01.470+0000 7fad173e3640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fad10105b10 0x7fad1019fdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:51:01.478 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:01.470+0000 7fad173e3640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fad10112a10 con 0x7fad10105b10 2024-09-21T03:51:01.478 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:01.470+0000 7fad173e3640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fad10112890 con 0x7fad101051c0 2024-09-21T03:51:01.478 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:01.470+0000 7fad173e3640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fad10112b90 con 0x7fad101031b0 2024-09-21T03:51:01.478 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:01.470+0000 7fad15980640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fad10105b10 0x7fad1019fdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:51:01.478 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:01.470+0000 7fad15980640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fad10105b10 0x7fad1019fdd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:57714/0 (socket says 172.21.15.39:57714) 2024-09-21T03:51:01.479 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:01.470+0000 7fad15980640 1 -- 172.21.15.39:0/90826495 learned_addr learned my addr 172.21.15.39:0/90826495 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:51:01.479 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:01.470+0000 7fad1517f640 1 --2- 172.21.15.39:0/90826495 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fad101031b0 0x7fad1019a590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:51:01.479 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:01.470+0000 7fad1497e640 1 --2- 172.21.15.39:0/90826495 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fad101051c0 0x7fad1019ab30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:51:01.479 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:01.471+0000 7fad15980640 1 -- 172.21.15.39:0/90826495 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fad101031b0 msgr2=0x7fad1019a590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:51:01.479 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:01.471+0000 7fad15980640 1 --2- 172.21.15.39:0/90826495 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fad101031b0 0x7fad1019a590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:01.479 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:01.471+0000 7fad15980640 1 -- 172.21.15.39:0/90826495 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fad101051c0 msgr2=0x7fad1019ab30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:51:01.479 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:01.471+0000 7fad15980640 1 --2- 172.21.15.39:0/90826495 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fad101051c0 0x7fad1019ab30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:01.480 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:01.471+0000 7fad15980640 1 -- 172.21.15.39:0/90826495 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fad101a0500 con 0x7fad10105b10 2024-09-21T03:51:01.480 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:01.471+0000 7fad15980640 1 --2- 172.21.15.39:0/90826495 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fad10105b10 0x7fad1019fdd0 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7fad0800d750 tx=0x7fad0800dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:51:01.480 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:01.471+0000 7fad0e7fc640 1 -- 172.21.15.39:0/90826495 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad08017070 con 0x7fad10105b10 2024-09-21T03:51:01.480 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:01.471+0000 7fad173e3640 1 -- 172.21.15.39:0/90826495 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fad101a07f0 con 0x7fad10105b10 2024-09-21T03:51:01.480 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:01.471+0000 7fad0e7fc640 1 -- 172.21.15.39:0/90826495 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fad080054d0 con 0x7fad10105b10 2024-09-21T03:51:01.480 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:01.471+0000 7fad0e7fc640 1 -- 172.21.15.39:0/90826495 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad08004c60 con 0x7fad10105b10 2024-09-21T03:51:01.480 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:01.471+0000 7fad173e3640 1 -- 172.21.15.39:0/90826495 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fad101a6f50 con 0x7fad10105b10 2024-09-21T03:51:01.488 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:01.474+0000 7fad1d957740 1 Processor -- start 2024-09-21T03:51:01.489 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:01.474+0000 7fad0e7fc640 1 -- 172.21.15.39:0/90826495 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 27) ==== 154643+0+0 (secure 0 0 0) 0x7fad08009bd0 con 0x7fad10105b10 2024-09-21T03:51:01.489 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:01.474+0000 7fad1d957740 1 -- start start 2024-09-21T03:51:01.489 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:01.474+0000 7fad1d957740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c5ec2e8070 0x55c5ec68f6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:51:01.489 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:01.475+0000 7fad1d957740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c5ec2bf1b0 0x55c5ec68fbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:51:01.489 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:01.475+0000 7fad1d957740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c5ec2c7390 0x55c5ec690120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:51:01.489 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:01.475+0000 7fad1d957740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x55c5ec605ea0 con 0x55c5ec2e8070 2024-09-21T03:51:01.489 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:01.475+0000 7fad1d957740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x55c5ec5f52d0 con 0x55c5ec2c7390 2024-09-21T03:51:01.489 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:01.475+0000 7fad1d957740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55c5ec5f90e0 con 0x55c5ec2bf1b0 2024-09-21T03:51:01.490 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:01.475+0000 7fad1497e640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c5ec2e8070 0x55c5ec68f6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:51:01.490 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:01.475+0000 7fad1497e640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c5ec2e8070 0x55c5ec68f6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:57720/0 (socket says 172.21.15.39:57720) 2024-09-21T03:51:01.490 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:01.475+0000 7fad1497e640 1 -- 172.21.15.39:0/845839321 learned_addr learned my addr 172.21.15.39:0/845839321 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:51:01.490 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:01.475+0000 7fad1517f640 1 --2- 172.21.15.39:0/845839321 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c5ec2bf1b0 0x55c5ec68fbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:51:01.490 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:01.475+0000 7fad1497e640 1 --2- 172.21.15.39:0/845839321 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c5ec2c7390 0x55c5ec690120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:51:01.490 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:01.475+0000 7fad1497e640 1 -- 172.21.15.39:0/845839321 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c5ec2bf1b0 msgr2=0x55c5ec68fbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:51:01.490 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:01.475+0000 7fad1497e640 1 --2- 172.21.15.39:0/845839321 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c5ec2bf1b0 0x55c5ec68fbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:01.490 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:01.475+0000 7fad1497e640 1 -- 172.21.15.39:0/845839321 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c5ec2c7390 msgr2=0x55c5ec690120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:51:01.490 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:01.475+0000 7fad1497e640 1 --2- 172.21.15.39:0/845839321 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c5ec2c7390 0x55c5ec690120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:01.490 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:01.475+0000 7fad1497e640 1 -- 172.21.15.39:0/845839321 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c5ec353650 con 0x55c5ec2e8070 2024-09-21T03:51:01.490 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:01.475+0000 7fad0e7fc640 1 --2- 172.21.15.39:0/90826495 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7facf80b6680 0x7facf80b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:51:01.490 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:01.475+0000 7fad1497e640 1 --2- 172.21.15.39:0/845839321 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c5ec2e8070 0x55c5ec68f6a0 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7fad10075a60 tx=0x7fad04000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a8b0b1312dcabd7 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:51:01.490 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:01.475+0000 7fad1517f640 1 --2- 172.21.15.39:0/90826495 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7facf80b6680 0x7facf80b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:51:01.491 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:01.475+0000 7fad0e7fc640 1 -- 172.21.15.39:0/90826495 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8484+0+0 (secure 0 0 0) 0x7fad080e6ff0 con 0x7fad10105b10 2024-09-21T03:51:01.491 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:01.476+0000 7facf77fe640 1 -- 172.21.15.39:0/845839321 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad04044020 con 0x55c5ec2e8070 2024-09-21T03:51:01.491 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:01.476+0000 7facf77fe640 1 -- 172.21.15.39:0/845839321 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fad04031820 con 0x55c5ec2e8070 2024-09-21T03:51:01.491 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:01.476+0000 7facf77fe640 1 -- 172.21.15.39:0/845839321 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad04035040 con 0x55c5ec2e8070 2024-09-21T03:51:01.491 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:01.477+0000 7fad1d957740 1 -- 172.21.15.39:0/845839321 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c5ec2e8070 msgr2=0x55c5ec68f6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:51:01.491 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:01.477+0000 7fad1517f640 1 --2- 172.21.15.39:0/90826495 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7facf80b6680 0x7facf80b8b90 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x55c5ec68e320 tx=0x7fad0000a590 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:51:01.491 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:01.477+0000 7fad1d957740 1 --2- 172.21.15.39:0/845839321 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c5ec2e8070 0x55c5ec68f6a0 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7fad10075a60 tx=0x7fad04000bc0 comp rx=0 tx=0).stop 2024-09-21T03:51:01.491 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:01.477+0000 7fad1d957740 1 -- 172.21.15.39:0/845839321 shutdown_connections 2024-09-21T03:51:01.491 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:01.477+0000 7fad1d957740 1 --2- 172.21.15.39:0/845839321 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c5ec2c7390 0x55c5ec690120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:01.491 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:01.477+0000 7fad1d957740 1 --2- 172.21.15.39:0/845839321 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c5ec2bf1b0 0x55c5ec68fbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:01.491 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:01.477+0000 7fad1d957740 1 --2- 172.21.15.39:0/845839321 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c5ec2e8070 0x55c5ec68f6a0 unknown :-1 s=CLOSED pgs=807 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:01.491 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:01.477+0000 7fad1d957740 1 -- 172.21.15.39:0/845839321 >> 172.21.15.39:0/845839321 conn(0x55c5ec2dc460 msgr2=0x55c5ec68d590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:51:01.491 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:01.477+0000 7fad1d957740 1 -- 172.21.15.39:0/845839321 shutdown_connections 2024-09-21T03:51:01.491 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:01.477+0000 7fad1d957740 1 -- 172.21.15.39:0/845839321 wait complete. 2024-09-21T03:51:01.492 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:01.477+0000 7fad1d957740 1 -- start start 2024-09-21T03:51:01.492 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:01.477+0000 7fad1d957740 1 Processor -- start 2024-09-21T03:51:01.492 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:01.478+0000 7fad1d957740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c5ec2bf1b0 0x55c5ec69ae70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:51:01.492 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:01.478+0000 7fad1d957740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c5ec2c7390 0x55c5ec69b3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:51:01.492 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:01.478+0000 7fad1d957740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c5ec2e8070 0x55c5ec6a7140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:51:01.492 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:01.478+0000 7fad1d957740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x55c5ec456700 con 0x55c5ec2c7390 2024-09-21T03:51:01.493 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:01.478+0000 7fad1d957740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x55c5ec590160 con 0x55c5ec2e8070 2024-09-21T03:51:01.493 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:01.478+0000 7fad1d957740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55c5ec69de20 con 0x55c5ec2bf1b0 2024-09-21T03:51:01.493 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:01.478+0000 7fad1d957740 10 client.0 _ll_register_callbacks cb 0x55c5ec5a99e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:51:01.493 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:01.478+0000 7fad1497e640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c5ec2c7390 0x55c5ec69b3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:51:01.493 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:01.478+0000 7fad1497e640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c5ec2c7390 0x55c5ec69b3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:57734/0 (socket says 172.21.15.39:57734) 2024-09-21T03:51:01.494 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:01.478+0000 7fad1497e640 1 -- 172.21.15.39:0/339780219 learned_addr learned my addr 172.21.15.39:0/339780219 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:51:01.494 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:01.478+0000 7fad1517f640 1 --2- 172.21.15.39:0/339780219 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c5ec2bf1b0 0x55c5ec69ae70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:51:01.494 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:01.478+0000 7fad15980640 1 --2- 172.21.15.39:0/339780219 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c5ec2e8070 0x55c5ec6a7140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:51:01.494 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:01.478+0000 7fad1497e640 1 -- 172.21.15.39:0/339780219 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c5ec2bf1b0 msgr2=0x55c5ec69ae70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:51:01.494 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:01.478+0000 7fad1497e640 1 --2- 172.21.15.39:0/339780219 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c5ec2bf1b0 0x55c5ec69ae70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:01.494 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:01.479+0000 7fad1497e640 1 -- 172.21.15.39:0/339780219 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c5ec2e8070 msgr2=0x55c5ec6a7140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:51:01.494 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:01.479+0000 7fad1497e640 1 --2- 172.21.15.39:0/339780219 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c5ec2e8070 0x55c5ec6a7140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:01.494 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:01.479+0000 7fad1497e640 1 -- 172.21.15.39:0/339780219 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c5ec5bdc90 con 0x55c5ec2c7390 2024-09-21T03:51:01.494 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:01.479+0000 7fad1497e640 1 --2- 172.21.15.39:0/339780219 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c5ec2c7390 0x55c5ec69b3b0 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7fad0402a930 tx=0x7fad04031400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e3593c9a61d20dda server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:51:01.494 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:01.479+0000 7facf4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-09-21T03:51:01.494 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:01.479+0000 7facf4ff9640 1 -- 172.21.15.39:0/339780219 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad04044030 con 0x55c5ec2c7390 2024-09-21T03:51:01.494 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:01.479+0000 7facf4ff9640 1 -- 172.21.15.39:0/339780219 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8484+0+0 (secure 0 0 0) 0x7fad04049630 con 0x55c5ec2c7390 2024-09-21T03:51:01.494 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:01.479+0000 7fad1d957740 1 -- 172.21.15.39:0/339780219 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fad04031820 con 0x55c5ec2c7390 2024-09-21T03:51:01.494 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:01.479+0000 7facf4ff9640 1 -- 172.21.15.39:0/339780219 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fad04035040 con 0x55c5ec2c7390 2024-09-21T03:51:01.495 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:01.479+0000 7fad1d957740 10 client.16692 fetch_fsmap learned FSMap version 29 2024-09-21T03:51:01.495 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:01.479+0000 7fad1d957740 1 -- 172.21.15.39:0/339780219 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55c5ec6a9c20 con 0x55c5ec2c7390 2024-09-21T03:51:01.495 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:01.479+0000 7facf4ff9640 1 -- 172.21.15.39:0/339780219 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fad04031820 con 0x55c5ec2c7390 2024-09-21T03:51:01.495 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:01.480+0000 7facf4ff9640 1 -- 172.21.15.39:0/339780219 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fad04058020 con 0x55c5ec2c7390 2024-09-21T03:51:01.495 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:01.480+0000 7fad1d957740 10 client.16692 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T03:51:01.495 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:01.480+0000 7fad1d957740 10 client.16692 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T03:51:01.495 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:01.480+0000 7fad1d957740 20 client.16692 populate_metadata read hostname 'smithi039' 2024-09-21T03:51:01.495 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:01.480+0000 7fad1d957740 1 --2- 172.21.15.39:0/339780219 >> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] conn(0x55c5ec2a69f0 0x55c5ec6acf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:51:01.495 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:01.480+0000 7fad1d957740 4 client.16692 mds_command: new command op to 24695 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:51:01.495 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:01.480+0000 7fad1d957740 1 -- 172.21.15.39:0/339780219 --> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c5ec607df0 con 0x55c5ec2a69f0 2024-09-21T03:51:01.495 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:01.480+0000 7fad1517f640 1 --2- 172.21.15.39:0/339780219 >> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] conn(0x55c5ec2a69f0 0x55c5ec6acf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:51:01.495 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:01.480+0000 7fad1517f640 1 --2- 172.21.15.39:0/339780219 >> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] conn(0x55c5ec2a69f0 0x55c5ec6acf80 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=47eae6f8fe197f58 server_cookie=187456de1a1510e8 in_seq=0 out_seq=0 2024-09-21T03:51:01.495 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:01.480+0000 7facf4ff9640 10 client.16692 ms_handle_connect on v2:172.21.15.150:6838/2949386907 2024-09-21T03:51:01.496 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:01.481+0000 7facf4ff9640 1 -- 172.21.15.39:0/339780219 <== mds.0 v2:172.21.15.150:6838/2949386907 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55c5ec607df0 con 0x55c5ec2a69f0 2024-09-21T03:51:01.496 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:01.481+0000 7facf4ff9640 10 client.16692 handle_command_reply: tid=0 2024-09-21T03:51:01.496 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:01.481+0000 7fad1d957740 1 client.16692 shutdown 2024-09-21T03:51:01.496 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:01.481+0000 7fad1d957740 1 -- 172.21.15.39:0/339780219 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c5ec2c7390 msgr2=0x55c5ec69b3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:51:01.496 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:01.481+0000 7fad1d957740 1 --2- 172.21.15.39:0/339780219 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c5ec2c7390 0x55c5ec69b3b0 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7fad0402a930 tx=0x7fad04031400 comp rx=0 tx=0).stop 2024-09-21T03:51:01.496 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:01.481+0000 7fad1d957740 1 -- 172.21.15.39:0/339780219 shutdown_connections 2024-09-21T03:51:01.496 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:01.481+0000 7fad1d957740 1 --2- 172.21.15.39:0/339780219 >> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] conn(0x55c5ec2a69f0 0x55c5ec6acf80 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:01.496 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:01.481+0000 7fad1d957740 1 --2- 172.21.15.39:0/339780219 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c5ec2e8070 0x55c5ec6a7140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:01.496 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:01.481+0000 7fad1d957740 1 --2- 172.21.15.39:0/339780219 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c5ec2c7390 0x55c5ec69b3b0 unknown :-1 s=CLOSED pgs=808 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:01.496 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:01.481+0000 7fad1d957740 1 --2- 172.21.15.39:0/339780219 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c5ec2bf1b0 0x55c5ec69ae70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:01.496 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:01.481+0000 7fad1d957740 1 -- 172.21.15.39:0/339780219 >> 172.21.15.39:0/339780219 conn(0x55c5ec2dc460 msgr2=0x55c5ec692880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:51:01.496 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:01.481+0000 7facf4ff9640 0 client.16692 ms_handle_reset on v2:172.21.15.150:6838/2949386907 2024-09-21T03:51:01.496 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:01.482+0000 7fad1d957740 1 -- 172.21.15.39:0/339780219 shutdown_connections 2024-09-21T03:51:01.496 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:01.482+0000 7fad1d957740 1 -- 172.21.15.39:0/339780219 wait complete. 2024-09-21T03:51:01.497 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:01.482+0000 7fad1d957740 20 client.16692 trim_cache size 0 max 16384 2024-09-21T03:51:01.499 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:01.495+0000 7fad1d957740 1 Processor -- start 2024-09-21T03:51:01.499 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:01.495+0000 7fad1d957740 1 -- start start 2024-09-21T03:51:01.499 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:01.495+0000 7fad1d957740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c5ec2bf1b0 0x55c5ec692eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:51:01.499 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:01.495+0000 7fad1d957740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c5ec2c7390 0x55c5ec6a70b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:51:01.499 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:01.495+0000 7fad1d957740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c5ec2e8070 0x55c5ec698060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:51:01.499 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:01.495+0000 7fad1d957740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fad101af6d0 con 0x55c5ec2bf1b0 2024-09-21T03:51:01.499 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:01.495+0000 7fad1d957740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fad101b1210 con 0x55c5ec2c7390 2024-09-21T03:51:01.499 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:01.495+0000 7fad1d957740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fad101afda0 con 0x55c5ec2e8070 2024-09-21T03:51:01.500 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:01.495+0000 7fad1517f640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c5ec2bf1b0 0x55c5ec692eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:51:01.500 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:01.496+0000 7fad1517f640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c5ec2bf1b0 0x55c5ec692eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:57740/0 (socket says 172.21.15.39:57740) 2024-09-21T03:51:01.500 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:01.496+0000 7fad1517f640 1 -- 172.21.15.39:0/3065338167 learned_addr learned my addr 172.21.15.39:0/3065338167 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:51:01.500 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:01.496+0000 7fad1497e640 1 --2- 172.21.15.39:0/3065338167 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c5ec2c7390 0x55c5ec6a70b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:51:01.500 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:01.496+0000 7fad15980640 1 --2- 172.21.15.39:0/3065338167 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c5ec2e8070 0x55c5ec698060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:51:01.500 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:01.496+0000 7fad1517f640 1 -- 172.21.15.39:0/3065338167 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c5ec2e8070 msgr2=0x55c5ec698060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:51:01.500 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:01.496+0000 7fad1517f640 1 --2- 172.21.15.39:0/3065338167 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c5ec2e8070 0x55c5ec698060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:01.500 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:01.496+0000 7fad1517f640 1 -- 172.21.15.39:0/3065338167 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c5ec2c7390 msgr2=0x55c5ec6a70b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:51:01.500 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:01.496+0000 7fad1517f640 1 --2- 172.21.15.39:0/3065338167 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c5ec2c7390 0x55c5ec6a70b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:01.500 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:01.496+0000 7fad1517f640 1 -- 172.21.15.39:0/3065338167 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c5ec2a6c20 con 0x55c5ec2bf1b0 2024-09-21T03:51:01.500 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:01.496+0000 7fad15980640 1 --2- 172.21.15.39:0/3065338167 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c5ec2e8070 0x55c5ec698060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:51:01.500 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:01.496+0000 7fad1517f640 1 --2- 172.21.15.39:0/3065338167 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c5ec2bf1b0 0x55c5ec692eb0 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7fad0000a300 tx=0x7fad00017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be898313f1d5f06a server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:51:01.500 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:01.496+0000 7facf6ffd640 1 -- 172.21.15.39:0/3065338167 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad00004b20 con 0x55c5ec2bf1b0 2024-09-21T03:51:01.500 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:01.496+0000 7facf6ffd640 1 -- 172.21.15.39:0/3065338167 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fad00003e20 con 0x55c5ec2bf1b0 2024-09-21T03:51:01.501 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:01.496+0000 7facf6ffd640 1 -- 172.21.15.39:0/3065338167 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad00004790 con 0x55c5ec2bf1b0 2024-09-21T03:51:01.504 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:01.497+0000 7fad1d957740 1 -- 172.21.15.39:0/3065338167 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c5ec2bf1b0 msgr2=0x55c5ec692eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:51:01.504 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:01.497+0000 7fad1d957740 1 --2- 172.21.15.39:0/3065338167 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c5ec2bf1b0 0x55c5ec692eb0 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7fad0000a300 tx=0x7fad00017000 comp rx=0 tx=0).stop 2024-09-21T03:51:01.504 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:01.497+0000 7fad1d957740 1 -- 172.21.15.39:0/3065338167 shutdown_connections 2024-09-21T03:51:01.505 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:01.497+0000 7fad1d957740 1 --2- 172.21.15.39:0/3065338167 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c5ec2e8070 0x55c5ec698060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:01.505 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:01.497+0000 7fad1d957740 1 --2- 172.21.15.39:0/3065338167 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c5ec2c7390 0x55c5ec6a70b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:01.505 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:01.497+0000 7fad1d957740 1 --2- 172.21.15.39:0/3065338167 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c5ec2bf1b0 0x55c5ec692eb0 unknown :-1 s=CLOSED pgs=809 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:01.505 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:01.497+0000 7fad1d957740 1 -- 172.21.15.39:0/3065338167 >> 172.21.15.39:0/3065338167 conn(0x55c5ec2dc460 msgr2=0x55c5ec6927a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:51:01.505 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:01.497+0000 7fad1d957740 1 -- 172.21.15.39:0/3065338167 shutdown_connections 2024-09-21T03:51:01.505 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:01.497+0000 7fad1d957740 1 -- 172.21.15.39:0/3065338167 wait complete. 2024-09-21T03:51:01.505 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:01.498+0000 7fad1d957740 1 -- start start 2024-09-21T03:51:01.505 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:01.498+0000 7fad1d957740 1 Processor -- start 2024-09-21T03:51:01.505 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:01.498+0000 7fad1d957740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c5ec2bf1b0 0x55c5ec6b0880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:51:01.505 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:01.498+0000 7fad1d957740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c5ec2c7390 0x55c5ec6adda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:51:01.505 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:01.498+0000 7fad1d957740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c5ec2e8070 0x55c5ec6ae3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:51:01.505 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:01.498+0000 7fad1d957740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fad101b0b40 con 0x55c5ec2c7390 2024-09-21T03:51:01.505 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:01.498+0000 7fad1d957740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fad101b0470 con 0x55c5ec2bf1b0 2024-09-21T03:51:01.506 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:01.498+0000 7fad1d957740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fad101ad6b0 con 0x55c5ec2e8070 2024-09-21T03:51:01.506 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:01.498+0000 7fad1d957740 10 client.0 _ll_register_callbacks cb 0x7fad0000a250 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:51:01.506 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:01.498+0000 7fad1497e640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c5ec2c7390 0x55c5ec6adda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:51:01.506 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:01.498+0000 7fad1497e640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c5ec2c7390 0x55c5ec6adda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:57744/0 (socket says 172.21.15.39:57744) 2024-09-21T03:51:01.506 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:01.498+0000 7fad1497e640 1 -- 172.21.15.39:0/4249199560 learned_addr learned my addr 172.21.15.39:0/4249199560 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:51:01.506 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:01.499+0000 7fad15980640 1 --2- 172.21.15.39:0/4249199560 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c5ec2e8070 0x55c5ec6ae3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:51:01.506 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:01.499+0000 7fad1517f640 1 --2- 172.21.15.39:0/4249199560 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c5ec2bf1b0 0x55c5ec6b0880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:51:01.506 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:01.499+0000 7fad1497e640 1 -- 172.21.15.39:0/4249199560 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c5ec2e8070 msgr2=0x55c5ec6ae3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:51:01.506 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:01.499+0000 7fad1497e640 1 --2- 172.21.15.39:0/4249199560 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c5ec2e8070 0x55c5ec6ae3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:01.506 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:01.499+0000 7fad1497e640 1 -- 172.21.15.39:0/4249199560 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c5ec2bf1b0 msgr2=0x55c5ec6b0880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:51:01.506 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:01.499+0000 7fad1497e640 1 --2- 172.21.15.39:0/4249199560 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c5ec2bf1b0 0x55c5ec6b0880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:01.506 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:01.499+0000 7fad1497e640 1 -- 172.21.15.39:0/4249199560 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c5ec6aeb60 con 0x55c5ec2c7390 2024-09-21T03:51:01.506 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:01.499+0000 7fad15980640 1 --2- 172.21.15.39:0/4249199560 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c5ec2e8070 0x55c5ec6ae3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:51:01.506 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:01.499+0000 7fad1497e640 1 --2- 172.21.15.39:0/4249199560 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c5ec2c7390 0x55c5ec6adda0 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7fad0405a9a0 tx=0x7fad04032fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d528a50b6a5db55 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:51:01.507 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:01.499+0000 7facf4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-09-21T03:51:01.507 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:01.499+0000 7facf4ff9640 1 -- 172.21.15.39:0/4249199560 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad04044040 con 0x55c5ec2c7390 2024-09-21T03:51:01.507 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:01.499+0000 7facf4ff9640 1 -- 172.21.15.39:0/4249199560 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8484+0+0 (secure 0 0 0) 0x7fad04004220 con 0x55c5ec2c7390 2024-09-21T03:51:01.507 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:01.499+0000 7fad1d957740 1 -- 172.21.15.39:0/4249199560 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fad00003e20 con 0x55c5ec2c7390 2024-09-21T03:51:01.507 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:01.499+0000 7facf4ff9640 1 -- 172.21.15.39:0/4249199560 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fad04069050 con 0x55c5ec2c7390 2024-09-21T03:51:01.507 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:01.499+0000 7fad1d957740 10 client.16698 fetch_fsmap learned FSMap version 29 2024-09-21T03:51:01.507 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:01.499+0000 7fad1d957740 1 -- 172.21.15.39:0/4249199560 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55c5ec717740 con 0x55c5ec2c7390 2024-09-21T03:51:01.507 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:01.500+0000 7facf4ff9640 1 -- 172.21.15.39:0/4249199560 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fad00003e20 con 0x55c5ec2c7390 2024-09-21T03:51:01.507 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:01.500+0000 7facf4ff9640 1 -- 172.21.15.39:0/4249199560 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fad04058040 con 0x55c5ec2c7390 2024-09-21T03:51:01.507 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:01.500+0000 7fad1d957740 10 client.16698 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T03:51:01.507 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:01.500+0000 7fad1d957740 10 client.16698 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T03:51:01.507 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:01.500+0000 7fad1d957740 20 client.16698 populate_metadata read hostname 'smithi039' 2024-09-21T03:51:01.507 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:01.500+0000 7fad1d957740 1 --2- 172.21.15.39:0/4249199560 >> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] conn(0x55c5ec27bda0 0x55c5ec719e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:51:01.507 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:01.500+0000 7fad1d957740 4 client.16698 mds_command: new command op to 24695 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T03:51:01.507 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:01.500+0000 7fad1d957740 1 -- 172.21.15.39:0/4249199560 --> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55c5ec6038a0 con 0x55c5ec27bda0 2024-09-21T03:51:01.508 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:01.500+0000 7fad1517f640 1 --2- 172.21.15.39:0/4249199560 >> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] conn(0x55c5ec27bda0 0x55c5ec719e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:51:01.508 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:01.501+0000 7facf4ff9640 10 client.16698 ms_handle_connect on v2:172.21.15.150:6838/2949386907 2024-09-21T03:51:01.508 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:01.501+0000 7fad1517f640 1 --2- 172.21.15.39:0/4249199560 >> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] conn(0x55c5ec27bda0 0x55c5ec719e10 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=eba5c99c741b2097 server_cookie=c082f9152b551507 in_seq=0 out_seq=0 2024-09-21T03:51:01.508 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:01.501+0000 7facf4ff9640 1 -- 172.21.15.39:0/4249199560 <== mds.0 v2:172.21.15.150:6838/2949386907 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55c5ec6038a0 con 0x55c5ec27bda0 2024-09-21T03:51:01.508 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:01.501+0000 7facf4ff9640 10 client.16698 handle_command_reply: tid=0 2024-09-21T03:51:01.508 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:01.501+0000 7fad1d957740 1 client.16698 shutdown 2024-09-21T03:51:01.508 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:01.501+0000 7fad1d957740 1 -- 172.21.15.39:0/4249199560 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c5ec2c7390 msgr2=0x55c5ec6adda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:51:01.508 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:01.501+0000 7fad1d957740 1 --2- 172.21.15.39:0/4249199560 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c5ec2c7390 0x55c5ec6adda0 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7fad0405a9a0 tx=0x7fad04032fb0 comp rx=0 tx=0).stop 2024-09-21T03:51:01.508 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:01.501+0000 7fad1d957740 1 -- 172.21.15.39:0/4249199560 shutdown_connections 2024-09-21T03:51:01.508 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:01.501+0000 7fad1d957740 1 --2- 172.21.15.39:0/4249199560 >> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] conn(0x55c5ec27bda0 0x55c5ec719e10 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:01.508 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:01.501+0000 7fad1d957740 1 --2- 172.21.15.39:0/4249199560 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c5ec2e8070 0x55c5ec6ae3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:01.508 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:01.501+0000 7fad1d957740 1 --2- 172.21.15.39:0/4249199560 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c5ec2c7390 0x55c5ec6adda0 unknown :-1 s=CLOSED pgs=810 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:01.508 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:01.501+0000 7fad1d957740 1 --2- 172.21.15.39:0/4249199560 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c5ec2bf1b0 0x55c5ec6b0880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:01.508 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:01.501+0000 7fad1d957740 1 -- 172.21.15.39:0/4249199560 >> 172.21.15.39:0/4249199560 conn(0x55c5ec2dc460 msgr2=0x55c5ec690610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:51:01.509 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:01.501+0000 7facf4ff9640 0 client.16698 ms_handle_reset on v2:172.21.15.150:6838/2949386907 2024-09-21T03:51:01.509 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:01.501+0000 7fad1d957740 1 -- 172.21.15.39:0/4249199560 shutdown_connections 2024-09-21T03:51:01.509 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:01.501+0000 7fad1d957740 1 -- 172.21.15.39:0/4249199560 wait complete. 2024-09-21T03:51:01.509 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:01.502+0000 7fad1d957740 20 client.16698 trim_cache size 0 max 16384 2024-09-21T03:51:01.509 INFO:teuthology.orchestra.run.smithi039.stdout:[] 2024-09-21T03:51:01.510 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:01.502+0000 7fad173e3640 1 -- 172.21.15.39:0/90826495 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7facf80b6680 msgr2=0x7facf80b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:51:01.510 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:01.502+0000 7fad173e3640 1 --2- 172.21.15.39:0/90826495 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7facf80b6680 0x7facf80b8b90 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x55c5ec68e320 tx=0x7fad0000a590 comp rx=0 tx=0).stop 2024-09-21T03:51:01.510 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:01.502+0000 7fad173e3640 1 -- 172.21.15.39:0/90826495 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fad10105b10 msgr2=0x7fad1019fdd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:51:01.510 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:01.502+0000 7fad173e3640 1 --2- 172.21.15.39:0/90826495 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fad10105b10 0x7fad1019fdd0 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7fad0800d750 tx=0x7fad0800dc10 comp rx=0 tx=0).stop 2024-09-21T03:51:01.510 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:01.502+0000 7fad173e3640 1 -- 172.21.15.39:0/90826495 shutdown_connections 2024-09-21T03:51:01.510 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:01.502+0000 7fad173e3640 1 --2- 172.21.15.39:0/90826495 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7facf80b6680 0x7facf80b8b90 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:01.510 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:01.502+0000 7fad173e3640 1 --2- 172.21.15.39:0/90826495 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fad10105b10 0x7fad1019fdd0 unknown :-1 s=CLOSED pgs=806 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:01.510 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:01.502+0000 7fad173e3640 1 --2- 172.21.15.39:0/90826495 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fad101051c0 0x7fad1019ab30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:01.511 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:01.502+0000 7fad173e3640 1 --2- 172.21.15.39:0/90826495 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fad101031b0 0x7fad1019a590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:01.511 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:01.502+0000 7fad173e3640 1 -- 172.21.15.39:0/90826495 >> 172.21.15.39:0/90826495 conn(0x7fad10079f90 msgr2=0x7fad10101320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:51:01.511 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:01.502+0000 7fad173e3640 1 -- 172.21.15.39:0/90826495 shutdown_connections 2024-09-21T03:51:01.511 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:01.502+0000 7fad173e3640 1 -- 172.21.15.39:0/90826495 wait complete. 2024-09-21T03:51:03.334 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:51:03 smithi039 ceph-mon[31211]: pgmap v475: 97 pgs: 97 active+clean; 10 MiB data, 536 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 9 op/s 2024-09-21T03:51:03.388 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-21T03:51:03.389 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 51116d5c-77ca-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-21T03:51:03.468 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:51:03 smithi161 ceph-mon[34533]: pgmap v475: 97 pgs: 97 active+clean; 10 MiB data, 536 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 9 op/s 2024-09-21T03:51:03.474 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:51:03 smithi150 ceph-mon[34289]: pgmap v475: 97 pgs: 97 active+clean; 10 MiB data, 536 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 9 op/s 2024-09-21T03:51:03.661 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/51116d5c-77ca-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:51:04.177 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:04.171+0000 7f7b8397f640 1 -- 172.21.15.39:0/3221855312 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7b7c1094a0 msgr2=0x7f7b7c1098b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:51:04.177 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:04.171+0000 7f7b8397f640 1 --2- 172.21.15.39:0/3221855312 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7b7c1094a0 0x7f7b7c1098b0 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7f7b6c009900 tx=0x7f7b6c02a410 comp rx=0 tx=0).stop 2024-09-21T03:51:04.177 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:04.171+0000 7f7b8397f640 1 -- 172.21.15.39:0/3221855312 shutdown_connections 2024-09-21T03:51:04.177 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:04.171+0000 7f7b8397f640 1 --2- 172.21.15.39:0/3221855312 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7b7c109df0 0x7f7b7c110620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:04.177 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:04.171+0000 7f7b8397f640 1 --2- 172.21.15.39:0/3221855312 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7b7c1094a0 0x7f7b7c1098b0 unknown :-1 s=CLOSED pgs=811 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:04.177 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:04.171+0000 7f7b8397f640 1 --2- 172.21.15.39:0/3221855312 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7b7c103b60 0x7f7b7c103f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:04.178 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:04.171+0000 7f7b8397f640 1 -- 172.21.15.39:0/3221855312 >> 172.21.15.39:0/3221855312 conn(0x7f7b7c07b850 msgr2=0x7f7b7c07bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:51:04.178 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:04.171+0000 7f7b8397f640 1 -- 172.21.15.39:0/3221855312 shutdown_connections 2024-09-21T03:51:04.178 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:04.171+0000 7f7b8397f640 1 -- 172.21.15.39:0/3221855312 wait complete. 2024-09-21T03:51:04.178 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:04.172+0000 7f7b8397f640 1 Processor -- start 2024-09-21T03:51:04.178 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:04.172+0000 7f7b8397f640 1 -- start start 2024-09-21T03:51:04.178 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:04.172+0000 7f7b8397f640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7b7c103b60 0x7f7b7c19a4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:51:04.178 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:04.172+0000 7f7b8397f640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7b7c1094a0 0x7f7b7c19aa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:51:04.178 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:04.172+0000 7f7b8397f640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7b7c109df0 0x7f7b7c19fd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:51:04.178 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:04.172+0000 7f7b8397f640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f7b7c113ab0 con 0x7f7b7c103b60 2024-09-21T03:51:04.178 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:04.172+0000 7f7b8397f640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f7b7c113930 con 0x7f7b7c109df0 2024-09-21T03:51:04.178 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:04.172+0000 7f7b8397f640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f7b7c113c30 con 0x7f7b7c1094a0 2024-09-21T03:51:04.178 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:04.172+0000 7f7b8171b640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7b7c103b60 0x7f7b7c19a4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:51:04.178 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:04.172+0000 7f7b8171b640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7b7c103b60 0x7f7b7c19a4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:57762/0 (socket says 172.21.15.39:57762) 2024-09-21T03:51:04.178 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:04.172+0000 7f7b8171b640 1 -- 172.21.15.39:0/3914268566 learned_addr learned my addr 172.21.15.39:0/3914268566 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:51:04.179 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:04.172+0000 7f7b80f1a640 1 --2- 172.21.15.39:0/3914268566 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7b7c1094a0 0x7f7b7c19aa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:51:04.179 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:04.173+0000 7f7b81f1c640 1 --2- 172.21.15.39:0/3914268566 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7b7c109df0 0x7f7b7c19fd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:51:04.179 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:04.173+0000 7f7b8171b640 1 -- 172.21.15.39:0/3914268566 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7b7c1094a0 msgr2=0x7f7b7c19aa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:51:04.179 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:04.173+0000 7f7b8171b640 1 --2- 172.21.15.39:0/3914268566 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7b7c1094a0 0x7f7b7c19aa60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:04.179 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:04.173+0000 7f7b8171b640 1 -- 172.21.15.39:0/3914268566 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7b7c109df0 msgr2=0x7f7b7c19fd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:51:04.179 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:04.173+0000 7f7b8171b640 1 --2- 172.21.15.39:0/3914268566 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7b7c109df0 0x7f7b7c19fd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:04.179 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:04.173+0000 7f7b8171b640 1 -- 172.21.15.39:0/3914268566 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7b7c1a0430 con 0x7f7b7c103b60 2024-09-21T03:51:04.179 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:04.173+0000 7f7b8171b640 1 --2- 172.21.15.39:0/3914268566 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7b7c103b60 0x7f7b7c19a4c0 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7f7b7800e9f0 tx=0x7f7b7800edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:51:04.179 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:04.173+0000 7f7b727fc640 1 -- 172.21.15.39:0/3914268566 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b7800cd80 con 0x7f7b7c103b60 2024-09-21T03:51:04.179 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:04.173+0000 7f7b727fc640 1 -- 172.21.15.39:0/3914268566 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7b780054d0 con 0x7f7b7c103b60 2024-09-21T03:51:04.179 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:04.173+0000 7f7b8397f640 1 -- 172.21.15.39:0/3914268566 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7b7c1a0720 con 0x7f7b7c103b60 2024-09-21T03:51:04.179 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:04.173+0000 7f7b727fc640 1 -- 172.21.15.39:0/3914268566 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b78010690 con 0x7f7b7c103b60 2024-09-21T03:51:04.179 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:04.173+0000 7f7b8397f640 1 -- 172.21.15.39:0/3914268566 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7b7c1a6e80 con 0x7f7b7c103b60 2024-09-21T03:51:04.182 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:04.176+0000 7f7b89ef3740 1 Processor -- start 2024-09-21T03:51:04.182 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:04.176+0000 7f7b89ef3740 1 -- start start 2024-09-21T03:51:04.182 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:04.176+0000 7f7b727fc640 1 -- 172.21.15.39:0/3914268566 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 27) ==== 154643+0+0 (secure 0 0 0) 0x7f7b78004e90 con 0x7f7b7c103b60 2024-09-21T03:51:04.182 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:04.176+0000 7f7b89ef3740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c1d5a28350 0x55c1d5dcf430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:51:04.182 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:04.176+0000 7f7b89ef3740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c1d59ff490 0x55c1d5dcf970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:51:04.182 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:04.176+0000 7f7b89ef3740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c1d5a07670 0x55c1d5dcfeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:51:04.182 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:04.176+0000 7f7b89ef3740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x55c1d5d45e60 con 0x55c1d5a07670 2024-09-21T03:51:04.182 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:04.176+0000 7f7b89ef3740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x55c1d5d35290 con 0x55c1d59ff490 2024-09-21T03:51:04.183 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:04.176+0000 7f7b89ef3740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55c1d5d390a0 con 0x55c1d5a28350 2024-09-21T03:51:04.183 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:04.177+0000 7f7b80f1a640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c1d5a07670 0x55c1d5dcfeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:51:04.183 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:04.177+0000 7f7b80f1a640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c1d5a07670 0x55c1d5dcfeb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:57776/0 (socket says 172.21.15.39:57776) 2024-09-21T03:51:04.183 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:04.177+0000 7f7b80f1a640 1 -- 172.21.15.39:0/587183924 learned_addr learned my addr 172.21.15.39:0/587183924 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:51:04.183 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:04.177+0000 7f7b8171b640 1 --2- 172.21.15.39:0/587183924 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c1d59ff490 0x55c1d5dcf970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:51:04.183 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:04.177+0000 7f7b81f1c640 1 --2- 172.21.15.39:0/587183924 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c1d5a28350 0x55c1d5dcf430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:51:04.183 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:04.177+0000 7f7b80f1a640 1 -- 172.21.15.39:0/587183924 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c1d5a28350 msgr2=0x55c1d5dcf430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:51:04.183 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:04.177+0000 7f7b80f1a640 1 --2- 172.21.15.39:0/587183924 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c1d5a28350 0x55c1d5dcf430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:04.183 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:04.177+0000 7f7b80f1a640 1 -- 172.21.15.39:0/587183924 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c1d59ff490 msgr2=0x55c1d5dcf970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:51:04.183 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:04.177+0000 7f7b80f1a640 1 --2- 172.21.15.39:0/587183924 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c1d59ff490 0x55c1d5dcf970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:04.183 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:04.177+0000 7f7b80f1a640 1 -- 172.21.15.39:0/587183924 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c1d5a93930 con 0x55c1d5a07670 2024-09-21T03:51:04.183 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:04.177+0000 7f7b727fc640 1 --2- 172.21.15.39:0/3914268566 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f7b680b6700 0x7f7b680b8c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:51:04.183 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:04.177+0000 7f7b80f1a640 1 --2- 172.21.15.39:0/587183924 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c1d5a07670 0x55c1d5dcfeb0 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7f7b6c009900 tx=0x7f7b6c000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8831d5f8dee85a93 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:51:04.183 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:04.177+0000 7f7b81f1c640 1 --2- 172.21.15.39:0/3914268566 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f7b680b6700 0x7f7b680b8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:51:04.184 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:04.177+0000 7f7b727fc640 1 -- 172.21.15.39:0/3914268566 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8484+0+0 (secure 0 0 0) 0x7f7b780e7b20 con 0x7f7b7c103b60 2024-09-21T03:51:04.184 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:04.178+0000 7f7b677fe640 1 -- 172.21.15.39:0/587183924 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b6c040070 con 0x55c1d5a07670 2024-09-21T03:51:04.184 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:04.178+0000 7f7b677fe640 1 -- 172.21.15.39:0/587183924 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7b6c004030 con 0x55c1d5a07670 2024-09-21T03:51:04.184 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:04.178+0000 7f7b677fe640 1 -- 172.21.15.39:0/587183924 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b6c035040 con 0x55c1d5a07670 2024-09-21T03:51:04.184 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:04.179+0000 7f7b89ef3740 1 -- 172.21.15.39:0/587183924 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c1d5a07670 msgr2=0x55c1d5dcfeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:51:04.184 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:04.179+0000 7f7b81f1c640 1 --2- 172.21.15.39:0/3914268566 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f7b680b6700 0x7f7b680b8c10 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f7b74008e40 tx=0x7f7b740085f0 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:51:04.184 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:04.179+0000 7f7b89ef3740 1 --2- 172.21.15.39:0/587183924 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c1d5a07670 0x55c1d5dcfeb0 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7f7b6c009900 tx=0x7f7b6c000bc0 comp rx=0 tx=0).stop 2024-09-21T03:51:04.184 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:04.179+0000 7f7b89ef3740 1 -- 172.21.15.39:0/587183924 shutdown_connections 2024-09-21T03:51:04.184 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:04.179+0000 7f7b89ef3740 1 --2- 172.21.15.39:0/587183924 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c1d5a07670 0x55c1d5dcfeb0 unknown :-1 s=CLOSED pgs=813 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:04.184 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:04.179+0000 7f7b89ef3740 1 --2- 172.21.15.39:0/587183924 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c1d59ff490 0x55c1d5dcf970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:04.184 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:04.179+0000 7f7b89ef3740 1 --2- 172.21.15.39:0/587183924 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c1d5a28350 0x55c1d5dcf430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:04.184 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:04.179+0000 7f7b89ef3740 1 -- 172.21.15.39:0/587183924 >> 172.21.15.39:0/587183924 conn(0x55c1d5a1c740 msgr2=0x55c1d5dcd320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:51:04.184 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:04.179+0000 7f7b89ef3740 1 -- 172.21.15.39:0/587183924 shutdown_connections 2024-09-21T03:51:04.184 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:04.179+0000 7f7b89ef3740 1 -- 172.21.15.39:0/587183924 wait complete. 2024-09-21T03:51:04.185 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:04.179+0000 7f7b89ef3740 1 -- start start 2024-09-21T03:51:04.185 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:04.179+0000 7f7b89ef3740 1 Processor -- start 2024-09-21T03:51:04.187 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:04.180+0000 7f7b89ef3740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c1d59ff490 0x55c1d5ddf390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:51:04.187 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:04.180+0000 7f7b89ef3740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c1d5a07670 0x55c1d5ddf8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:51:04.187 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:04.180+0000 7f7b89ef3740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c1d5a28350 0x55c1d5deb270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:51:04.188 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:04.180+0000 7f7b89ef3740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x55c1d5b96890 con 0x55c1d5a28350 2024-09-21T03:51:04.188 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:04.180+0000 7f7b89ef3740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x55c1d5ccfd40 con 0x55c1d5a07670 2024-09-21T03:51:04.188 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:04.180+0000 7f7b89ef3740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55c1d5de1f10 con 0x55c1d59ff490 2024-09-21T03:51:04.188 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:04.180+0000 7f7b89ef3740 10 client.0 _ll_register_callbacks cb 0x55c1d5d33c80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:51:04.188 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:04.180+0000 7f7b81f1c640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c1d5a28350 0x55c1d5deb270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:51:04.188 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:04.180+0000 7f7b81f1c640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c1d5a28350 0x55c1d5deb270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:57790/0 (socket says 172.21.15.39:57790) 2024-09-21T03:51:04.188 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:04.180+0000 7f7b81f1c640 1 -- 172.21.15.39:0/3324588105 learned_addr learned my addr 172.21.15.39:0/3324588105 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:51:04.188 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:04.180+0000 7f7b80f1a640 1 --2- 172.21.15.39:0/3324588105 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c1d5a07670 0x55c1d5ddf8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:51:04.188 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:04.180+0000 7f7b8171b640 1 --2- 172.21.15.39:0/3324588105 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c1d59ff490 0x55c1d5ddf390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:51:04.188 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:04.181+0000 7f7b81f1c640 1 -- 172.21.15.39:0/3324588105 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c1d59ff490 msgr2=0x55c1d5ddf390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:51:04.188 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:04.181+0000 7f7b81f1c640 1 --2- 172.21.15.39:0/3324588105 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c1d59ff490 0x55c1d5ddf390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:04.188 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:04.181+0000 7f7b81f1c640 1 -- 172.21.15.39:0/3324588105 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c1d5a07670 msgr2=0x55c1d5ddf8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:51:04.188 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:04.181+0000 7f7b81f1c640 1 --2- 172.21.15.39:0/3324588105 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c1d5a07670 0x55c1d5ddf8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:04.188 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:04.181+0000 7f7b81f1c640 1 -- 172.21.15.39:0/3324588105 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c1d5deb7b0 con 0x55c1d5a28350 2024-09-21T03:51:04.188 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:04.181+0000 7f7b81f1c640 1 --2- 172.21.15.39:0/3324588105 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c1d5a28350 0x55c1d5deb270 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7f7b7400edb0 tx=0x7f7b74004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e14f9e055ee3f5b1 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:51:04.189 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:04.181+0000 7f7b64ff9640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-09-21T03:51:04.189 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:04.181+0000 7f7b64ff9640 1 -- 172.21.15.39:0/3324588105 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b7401cd70 con 0x55c1d5a28350 2024-09-21T03:51:04.189 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:04.181+0000 7f7b89ef3740 1 -- 172.21.15.39:0/3324588105 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7b6c004030 con 0x55c1d5a28350 2024-09-21T03:51:04.189 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:04.181+0000 7f7b64ff9640 1 -- 172.21.15.39:0/3324588105 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8484+0+0 (secure 0 0 0) 0x7f7b74027430 con 0x55c1d5a28350 2024-09-21T03:51:04.189 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:04.181+0000 7f7b64ff9640 1 -- 172.21.15.39:0/3324588105 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f7b74027630 con 0x55c1d5a28350 2024-09-21T03:51:04.189 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:04.181+0000 7f7b89ef3740 10 client.16710 fetch_fsmap learned FSMap version 29 2024-09-21T03:51:04.189 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:04.181+0000 7f7b89ef3740 1 -- 172.21.15.39:0/3324588105 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55c1d5dee170 con 0x55c1d5a28350 2024-09-21T03:51:04.189 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:04.181+0000 7f7b64ff9640 1 -- 172.21.15.39:0/3324588105 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7b6c004030 con 0x55c1d5a28350 2024-09-21T03:51:04.189 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:04.182+0000 7f7b64ff9640 1 -- 172.21.15.39:0/3324588105 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f7b7401d8b0 con 0x55c1d5a28350 2024-09-21T03:51:04.189 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:04.182+0000 7f7b89ef3740 10 client.16710 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T03:51:04.189 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:04.182+0000 7f7b89ef3740 10 client.16710 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T03:51:04.189 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:04.182+0000 7f7b89ef3740 20 client.16710 populate_metadata read hostname 'smithi039' 2024-09-21T03:51:04.189 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:04.182+0000 7f7b89ef3740 1 --2- 172.21.15.39:0/3324588105 >> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] conn(0x55c1d59e6cd0 0x55c1d5df1640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:51:04.189 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:04.182+0000 7f7b89ef3740 4 client.16710 mds_command: new command op to 24695 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:51:04.189 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:04.182+0000 7f7b89ef3740 1 -- 172.21.15.39:0/3324588105 --> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c1d5d47db0 con 0x55c1d59e6cd0 2024-09-21T03:51:04.190 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:04.182+0000 7f7b8171b640 1 --2- 172.21.15.39:0/3324588105 >> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] conn(0x55c1d59e6cd0 0x55c1d5df1640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:51:04.190 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:04.183+0000 7f7b64ff9640 10 client.16710 ms_handle_connect on v2:172.21.15.150:6838/2949386907 2024-09-21T03:51:04.190 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:04.183+0000 7f7b8171b640 1 --2- 172.21.15.39:0/3324588105 >> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] conn(0x55c1d59e6cd0 0x55c1d5df1640 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bead9292f0306845 server_cookie=916f191063f310fe in_seq=0 out_seq=0 2024-09-21T03:51:04.190 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:04.183+0000 7f7b64ff9640 1 -- 172.21.15.39:0/3324588105 <== mds.0 v2:172.21.15.150:6838/2949386907 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55c1d5d47db0 con 0x55c1d59e6cd0 2024-09-21T03:51:04.190 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:04.183+0000 7f7b64ff9640 10 client.16710 handle_command_reply: tid=0 2024-09-21T03:51:04.190 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:04.183+0000 7f7b89ef3740 1 client.16710 shutdown 2024-09-21T03:51:04.190 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:04.183+0000 7f7b89ef3740 1 -- 172.21.15.39:0/3324588105 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c1d5a28350 msgr2=0x55c1d5deb270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:51:04.190 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:04.183+0000 7f7b89ef3740 1 --2- 172.21.15.39:0/3324588105 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c1d5a28350 0x55c1d5deb270 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7f7b7400edb0 tx=0x7f7b74004a40 comp rx=0 tx=0).stop 2024-09-21T03:51:04.190 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:04.184+0000 7f7b89ef3740 1 -- 172.21.15.39:0/3324588105 shutdown_connections 2024-09-21T03:51:04.190 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:04.184+0000 7f7b89ef3740 1 --2- 172.21.15.39:0/3324588105 >> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] conn(0x55c1d59e6cd0 0x55c1d5df1640 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:04.190 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:04.184+0000 7f7b89ef3740 1 --2- 172.21.15.39:0/3324588105 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c1d5a28350 0x55c1d5deb270 unknown :-1 s=CLOSED pgs=814 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:04.190 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:04.184+0000 7f7b89ef3740 1 --2- 172.21.15.39:0/3324588105 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c1d5a07670 0x55c1d5ddf8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:04.190 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:04.184+0000 7f7b64ff9640 0 client.16710 ms_handle_reset on v2:172.21.15.150:6838/2949386907 2024-09-21T03:51:04.190 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:04.184+0000 7f7b89ef3740 1 --2- 172.21.15.39:0/3324588105 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c1d59ff490 0x55c1d5ddf390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:04.191 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:04.184+0000 7f7b89ef3740 1 -- 172.21.15.39:0/3324588105 >> 172.21.15.39:0/3324588105 conn(0x55c1d5a1c740 msgr2=0x55c1d5dd0e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:51:04.191 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:04.184+0000 7f7b89ef3740 1 -- 172.21.15.39:0/3324588105 shutdown_connections 2024-09-21T03:51:04.191 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:04.184+0000 7f7b89ef3740 1 -- 172.21.15.39:0/3324588105 wait complete. 2024-09-21T03:51:04.191 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:04.184+0000 7f7b89ef3740 20 client.16710 trim_cache size 0 max 16384 2024-09-21T03:51:04.205 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:04.202+0000 7f7b89ef3740 1 Processor -- start 2024-09-21T03:51:04.206 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:04.202+0000 7f7b89ef3740 1 -- start start 2024-09-21T03:51:04.206 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:04.202+0000 7f7b89ef3740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c1d59e6cd0 0x55c1d5deb170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:51:04.206 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:04.202+0000 7f7b89ef3740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c1d59ff490 0x55c1d5ddda10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:51:04.206 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:04.202+0000 7f7b89ef3740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c1d5a07670 0x55c1d5dddf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:51:04.206 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:04.202+0000 7f7b89ef3740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f7b7c1ad400 con 0x55c1d59ff490 2024-09-21T03:51:04.206 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:04.202+0000 7f7b89ef3740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f7b7c1aef40 con 0x55c1d59e6cd0 2024-09-21T03:51:04.206 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:04.202+0000 7f7b89ef3740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f7b7c1adad0 con 0x55c1d5a07670 2024-09-21T03:51:04.206 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:04.202+0000 7f7b80f1a640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c1d59ff490 0x55c1d5ddda10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:51:04.206 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:04.202+0000 7f7b80f1a640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c1d59ff490 0x55c1d5ddda10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:57806/0 (socket says 172.21.15.39:57806) 2024-09-21T03:51:04.206 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:04.202+0000 7f7b80f1a640 1 -- 172.21.15.39:0/521194745 learned_addr learned my addr 172.21.15.39:0/521194745 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:51:04.206 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:04.202+0000 7f7b8171b640 1 --2- 172.21.15.39:0/521194745 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c1d59e6cd0 0x55c1d5deb170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:51:04.206 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:04.202+0000 7f7b81f1c640 1 --2- 172.21.15.39:0/521194745 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c1d5a07670 0x55c1d5dddf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:51:04.206 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:04.202+0000 7f7b80f1a640 1 -- 172.21.15.39:0/521194745 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c1d5a07670 msgr2=0x55c1d5dddf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:51:04.207 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:04.202+0000 7f7b80f1a640 1 --2- 172.21.15.39:0/521194745 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c1d5a07670 0x55c1d5dddf50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:04.207 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:04.202+0000 7f7b80f1a640 1 -- 172.21.15.39:0/521194745 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c1d59e6cd0 msgr2=0x55c1d5deb170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:51:04.207 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:04.203+0000 7f7b80f1a640 1 --2- 172.21.15.39:0/521194745 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c1d59e6cd0 0x55c1d5deb170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:04.207 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:04.203+0000 7f7b80f1a640 1 -- 172.21.15.39:0/521194745 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c1d5dd21f0 con 0x55c1d59ff490 2024-09-21T03:51:04.207 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:04.203+0000 7f7b81f1c640 1 --2- 172.21.15.39:0/521194745 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c1d5a07670 0x55c1d5dddf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:51:04.207 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:04.203+0000 7f7b8171b640 1 --2- 172.21.15.39:0/521194745 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c1d59e6cd0 0x55c1d5deb170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:51:04.207 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:04.203+0000 7f7b80f1a640 1 --2- 172.21.15.39:0/521194745 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c1d59ff490 0x55c1d5ddda10 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7f7b6c0051f0 tx=0x7f7b6c005220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2050af6aa11927a server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:51:04.207 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:04.203+0000 7f7b66ffd640 1 -- 172.21.15.39:0/521194745 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b6c053070 con 0x55c1d59ff490 2024-09-21T03:51:04.207 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:04.203+0000 7f7b66ffd640 1 -- 172.21.15.39:0/521194745 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7b6c032e60 con 0x55c1d59ff490 2024-09-21T03:51:04.207 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:04.203+0000 7f7b66ffd640 1 -- 172.21.15.39:0/521194745 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b6c040040 con 0x55c1d59ff490 2024-09-21T03:51:04.210 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:04.204+0000 7f7b89ef3740 1 -- 172.21.15.39:0/521194745 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c1d59ff490 msgr2=0x55c1d5ddda10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:51:04.210 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:04.204+0000 7f7b89ef3740 1 --2- 172.21.15.39:0/521194745 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c1d59ff490 0x55c1d5ddda10 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7f7b6c0051f0 tx=0x7f7b6c005220 comp rx=0 tx=0).stop 2024-09-21T03:51:04.210 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:04.204+0000 7f7b89ef3740 1 -- 172.21.15.39:0/521194745 shutdown_connections 2024-09-21T03:51:04.210 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:04.204+0000 7f7b89ef3740 1 --2- 172.21.15.39:0/521194745 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c1d5a07670 0x55c1d5dddf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:04.210 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:04.204+0000 7f7b89ef3740 1 --2- 172.21.15.39:0/521194745 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c1d59ff490 0x55c1d5ddda10 unknown :-1 s=CLOSED pgs=815 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:04.210 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:04.204+0000 7f7b89ef3740 1 --2- 172.21.15.39:0/521194745 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c1d59e6cd0 0x55c1d5deb170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:04.210 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:04.204+0000 7f7b89ef3740 1 -- 172.21.15.39:0/521194745 >> 172.21.15.39:0/521194745 conn(0x55c1d5a1c740 msgr2=0x55c1d5dda2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:51:04.210 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:04.204+0000 7f7b89ef3740 1 -- 172.21.15.39:0/521194745 shutdown_connections 2024-09-21T03:51:04.210 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:04.204+0000 7f7b89ef3740 1 -- 172.21.15.39:0/521194745 wait complete. 2024-09-21T03:51:04.210 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:04.204+0000 7f7b89ef3740 1 -- start start 2024-09-21T03:51:04.210 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:04.204+0000 7f7b89ef3740 1 Processor -- start 2024-09-21T03:51:04.210 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:04.205+0000 7f7b89ef3740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c1d59e6cd0 0x55c1d5de7860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:51:04.211 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:04.205+0000 7f7b89ef3740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c1d59ff490 0x55c1d5de3d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:51:04.211 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:04.205+0000 7f7b89ef3740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c1d5a07670 0x55c1d5de4260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:51:04.211 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:04.205+0000 7f7b89ef3740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f7b7c1ae870 con 0x55c1d59ff490 2024-09-21T03:51:04.211 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:04.205+0000 7f7b89ef3740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f7b7c1ae1a0 con 0x55c1d59e6cd0 2024-09-21T03:51:04.211 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:04.205+0000 7f7b89ef3740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f7b7c1ab3e0 con 0x55c1d5a07670 2024-09-21T03:51:04.211 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:04.205+0000 7f7b89ef3740 10 client.0 _ll_register_callbacks cb 0x7f7b7400ea30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:51:04.211 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:04.205+0000 7f7b80f1a640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c1d59ff490 0x55c1d5de3d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:51:04.211 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:04.205+0000 7f7b80f1a640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c1d59ff490 0x55c1d5de3d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:57820/0 (socket says 172.21.15.39:57820) 2024-09-21T03:51:04.211 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:04.205+0000 7f7b80f1a640 1 -- 172.21.15.39:0/905454891 learned_addr learned my addr 172.21.15.39:0/905454891 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:51:04.211 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:04.205+0000 7f7b81f1c640 1 --2- 172.21.15.39:0/905454891 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c1d5a07670 0x55c1d5de4260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:51:04.211 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:04.205+0000 7f7b8171b640 1 --2- 172.21.15.39:0/905454891 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c1d59e6cd0 0x55c1d5de7860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:51:04.211 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:04.205+0000 7f7b80f1a640 1 -- 172.21.15.39:0/905454891 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c1d5a07670 msgr2=0x55c1d5de4260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:51:04.211 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:04.205+0000 7f7b80f1a640 1 --2- 172.21.15.39:0/905454891 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c1d5a07670 0x55c1d5de4260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:04.211 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:04.205+0000 7f7b80f1a640 1 -- 172.21.15.39:0/905454891 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c1d59e6cd0 msgr2=0x55c1d5de7860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:51:04.212 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:04.205+0000 7f7b80f1a640 1 --2- 172.21.15.39:0/905454891 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c1d59e6cd0 0x55c1d5de7860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:04.212 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:04.205+0000 7f7b80f1a640 1 -- 172.21.15.39:0/905454891 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c1d5de48a0 con 0x55c1d59ff490 2024-09-21T03:51:04.212 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:04.205+0000 7f7b8171b640 1 --2- 172.21.15.39:0/905454891 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c1d59e6cd0 0x55c1d5de7860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:51:04.212 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:04.205+0000 7f7b80f1a640 1 --2- 172.21.15.39:0/905454891 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c1d59ff490 0x55c1d5de3d20 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7f7b6c051040 tx=0x7f7b6c005540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=65821d0aec7ab68c server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:51:04.212 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:04.205+0000 7f7b64ff9640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-09-21T03:51:04.212 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:04.206+0000 7f7b64ff9640 1 -- 172.21.15.39:0/905454891 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b6c040070 con 0x55c1d59ff490 2024-09-21T03:51:04.212 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:04.206+0000 7f7b89ef3740 1 -- 172.21.15.39:0/905454891 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7b6c032e60 con 0x55c1d59ff490 2024-09-21T03:51:04.212 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:04.206+0000 7f7b64ff9640 1 -- 172.21.15.39:0/905454891 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8484+0+0 (secure 0 0 0) 0x7f7b6c0316b0 con 0x55c1d59ff490 2024-09-21T03:51:04.212 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:04.206+0000 7f7b64ff9640 1 -- 172.21.15.39:0/905454891 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f7b6c053070 con 0x55c1d59ff490 2024-09-21T03:51:04.212 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:04.206+0000 7f7b89ef3740 10 client.16716 fetch_fsmap learned FSMap version 29 2024-09-21T03:51:04.212 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:04.206+0000 7f7b89ef3740 1 -- 172.21.15.39:0/905454891 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55c1d5e5ce00 con 0x55c1d59ff490 2024-09-21T03:51:04.212 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:04.206+0000 7f7b64ff9640 1 -- 172.21.15.39:0/905454891 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7b6c032e60 con 0x55c1d59ff490 2024-09-21T03:51:04.212 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:04.206+0000 7f7b64ff9640 1 -- 172.21.15.39:0/905454891 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f7b6c003930 con 0x55c1d59ff490 2024-09-21T03:51:04.212 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:04.206+0000 7f7b89ef3740 10 client.16716 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T03:51:04.213 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:04.206+0000 7f7b89ef3740 10 client.16716 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T03:51:04.213 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:04.206+0000 7f7b89ef3740 20 client.16716 populate_metadata read hostname 'smithi039' 2024-09-21T03:51:04.213 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:04.206+0000 7f7b89ef3740 1 --2- 172.21.15.39:0/905454891 >> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] conn(0x55c1d5a28350 0x55c1d5e5f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:51:04.213 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:04.206+0000 7f7b89ef3740 4 client.16716 mds_command: new command op to 24695 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-21T03:51:04.213 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:04.206+0000 7f7b89ef3740 1 -- 172.21.15.39:0/905454891 --> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55c1d5d43860 con 0x55c1d5a28350 2024-09-21T03:51:04.213 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:04.207+0000 7f7b8171b640 1 --2- 172.21.15.39:0/905454891 >> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] conn(0x55c1d5a28350 0x55c1d5e5f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:51:04.213 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:04.207+0000 7f7b8171b640 1 --2- 172.21.15.39:0/905454891 >> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] conn(0x55c1d5a28350 0x55c1d5e5f0d0 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f9a3818dd492ef40 server_cookie=8735b5f911a75b36 in_seq=0 out_seq=0 2024-09-21T03:51:04.213 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:04.207+0000 7f7b64ff9640 10 client.16716 ms_handle_connect on v2:172.21.15.150:6838/2949386907 2024-09-21T03:51:04.214 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:04.209+0000 7f7b64ff9640 1 -- 172.21.15.39:0/905454891 <== mds.0 v2:172.21.15.150:6838/2949386907 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55c1d5d43860 con 0x55c1d5a28350 2024-09-21T03:51:04.214 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:04.209+0000 7f7b64ff9640 10 client.16716 handle_command_reply: tid=0 2024-09-21T03:51:04.214 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:04.209+0000 7f7b89ef3740 1 client.16716 shutdown 2024-09-21T03:51:04.214 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:04.209+0000 7f7b89ef3740 1 -- 172.21.15.39:0/905454891 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c1d59ff490 msgr2=0x55c1d5de3d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:51:04.214 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:04.209+0000 7f7b89ef3740 1 --2- 172.21.15.39:0/905454891 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c1d59ff490 0x55c1d5de3d20 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7f7b6c051040 tx=0x7f7b6c005540 comp rx=0 tx=0).stop 2024-09-21T03:51:04.215 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:04.209+0000 7f7b89ef3740 1 -- 172.21.15.39:0/905454891 shutdown_connections 2024-09-21T03:51:04.215 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:04.209+0000 7f7b89ef3740 1 --2- 172.21.15.39:0/905454891 >> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] conn(0x55c1d5a28350 0x55c1d5e5f0d0 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:04.215 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:04.209+0000 7f7b89ef3740 1 --2- 172.21.15.39:0/905454891 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c1d5a07670 0x55c1d5de4260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:04.215 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:04.209+0000 7f7b89ef3740 1 --2- 172.21.15.39:0/905454891 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c1d59ff490 0x55c1d5de3d20 unknown :-1 s=CLOSED pgs=816 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:04.215 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:04.209+0000 7f7b89ef3740 1 --2- 172.21.15.39:0/905454891 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c1d59e6cd0 0x55c1d5de7860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:04.215 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:04.209+0000 7f7b89ef3740 1 -- 172.21.15.39:0/905454891 >> 172.21.15.39:0/905454891 conn(0x55c1d5a1c740 msgr2=0x55c1d5de0640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:51:04.215 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:04.209+0000 7f7b64ff9640 0 client.16716 ms_handle_reset on v2:172.21.15.150:6838/2949386907 2024-09-21T03:51:04.215 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:04.209+0000 7f7b89ef3740 1 -- 172.21.15.39:0/905454891 shutdown_connections 2024-09-21T03:51:04.215 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:04.209+0000 7f7b89ef3740 1 -- 172.21.15.39:0/905454891 wait complete. 2024-09-21T03:51:04.215 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:04.209+0000 7f7b89ef3740 20 client.16716 trim_cache size 0 max 16384 2024-09-21T03:51:04.215 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-09-21T03:51:04.216 INFO:teuthology.orchestra.run.smithi039.stdout: "return_code": 0, 2024-09-21T03:51:04.216 INFO:teuthology.orchestra.run.smithi039.stdout: "scrub_tag": "160750d8-401f-4810-866d-e63e7e4bc3e0", 2024-09-21T03:51:04.216 INFO:teuthology.orchestra.run.smithi039.stdout: "mode": "asynchronous" 2024-09-21T03:51:04.216 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-09-21T03:51:04.216 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:04.210+0000 7f7b8397f640 1 -- 172.21.15.39:0/3914268566 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f7b680b6700 msgr2=0x7f7b680b8c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:51:04.216 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:04.210+0000 7f7b8397f640 1 --2- 172.21.15.39:0/3914268566 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f7b680b6700 0x7f7b680b8c10 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f7b74008e40 tx=0x7f7b740085f0 comp rx=0 tx=0).stop 2024-09-21T03:51:04.217 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:04.210+0000 7f7b8397f640 1 -- 172.21.15.39:0/3914268566 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7b7c103b60 msgr2=0x7f7b7c19a4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:51:04.217 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:04.210+0000 7f7b8397f640 1 --2- 172.21.15.39:0/3914268566 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7b7c103b60 0x7f7b7c19a4c0 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7f7b7800e9f0 tx=0x7f7b7800edf0 comp rx=0 tx=0).stop 2024-09-21T03:51:04.217 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:04.210+0000 7f7b8397f640 1 -- 172.21.15.39:0/3914268566 shutdown_connections 2024-09-21T03:51:04.217 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:04.210+0000 7f7b8397f640 1 --2- 172.21.15.39:0/3914268566 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f7b680b6700 0x7f7b680b8c10 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:04.217 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:04.210+0000 7f7b8397f640 1 --2- 172.21.15.39:0/3914268566 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7b7c109df0 0x7f7b7c19fd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:04.217 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:04.210+0000 7f7b8397f640 1 --2- 172.21.15.39:0/3914268566 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7b7c1094a0 0x7f7b7c19aa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:04.217 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:04.210+0000 7f7b8397f640 1 --2- 172.21.15.39:0/3914268566 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7b7c103b60 0x7f7b7c19a4c0 unknown :-1 s=CLOSED pgs=812 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:04.217 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:04.210+0000 7f7b8397f640 1 -- 172.21.15.39:0/3914268566 >> 172.21.15.39:0/3914268566 conn(0x7f7b7c07b850 msgr2=0x7f7b7c105520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:51:04.217 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:04.210+0000 7f7b8397f640 1 -- 172.21.15.39:0/3914268566 shutdown_connections 2024-09-21T03:51:04.217 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:04.210+0000 7f7b8397f640 1 -- 172.21.15.39:0/3914268566 wait complete. 2024-09-21T03:51:05.122 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 51116d5c-77ca-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub status 2024-09-21T03:51:05.334 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:51:05 smithi039 ceph-mon[31211]: pgmap v476: 97 pgs: 97 active+clean; 10 MiB data, 520 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 426 B/s wr, 10 op/s 2024-09-21T03:51:05.334 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:51:05 smithi039 ceph-mon[31211]: scrub queued for path: / 2024-09-21T03:51:05.334 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:51:05 smithi039 ceph-mon[31211]: scrub summary: idle+waiting paths [/] 2024-09-21T03:51:05.334 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:51:05 smithi039 ceph-mon[31211]: scrub summary: active paths [/] 2024-09-21T03:51:05.334 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:51:05 smithi039 ceph-mon[31211]: scrub summary: idle+waiting paths [/] 2024-09-21T03:51:05.334 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:51:05 smithi039 ceph-mon[31211]: scrub summary: idle 2024-09-21T03:51:05.381 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/51116d5c-77ca-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:51:05.468 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:51:05 smithi161 ceph-mon[34533]: pgmap v476: 97 pgs: 97 active+clean; 10 MiB data, 520 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 426 B/s wr, 10 op/s 2024-09-21T03:51:05.468 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:51:05 smithi161 ceph-mon[34533]: scrub queued for path: / 2024-09-21T03:51:05.468 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:51:05 smithi161 ceph-mon[34533]: scrub summary: idle+waiting paths [/] 2024-09-21T03:51:05.468 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:51:05 smithi161 ceph-mon[34533]: scrub summary: active paths [/] 2024-09-21T03:51:05.468 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:51:05 smithi161 ceph-mon[34533]: scrub summary: idle+waiting paths [/] 2024-09-21T03:51:05.468 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:51:05 smithi161 ceph-mon[34533]: scrub summary: idle 2024-09-21T03:51:05.474 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:51:05 smithi150 ceph-mon[34289]: pgmap v476: 97 pgs: 97 active+clean; 10 MiB data, 520 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 426 B/s wr, 10 op/s 2024-09-21T03:51:05.475 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:51:05 smithi150 ceph-mon[34289]: scrub queued for path: / 2024-09-21T03:51:05.475 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:51:05 smithi150 ceph-mon[34289]: scrub summary: idle+waiting paths [/] 2024-09-21T03:51:05.475 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:51:05 smithi150 ceph-mon[34289]: scrub summary: active paths [/] 2024-09-21T03:51:05.475 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:51:05 smithi150 ceph-mon[34289]: scrub summary: idle+waiting paths [/] 2024-09-21T03:51:05.475 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:51:05 smithi150 ceph-mon[34289]: scrub summary: idle 2024-09-21T03:51:05.890 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:05.887+0000 7fb6f3a12640 1 -- 172.21.15.39:0/2558016805 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb6ec10ddf0 msgr2=0x7fb6ec110300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:51:05.890 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:05.887+0000 7fb6f3a12640 1 --2- 172.21.15.39:0/2558016805 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb6ec10ddf0 0x7fb6ec110300 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7fb6e4009ca0 tx=0x7fb6e402a8c0 comp rx=0 tx=0).stop 2024-09-21T03:51:05.890 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:05.887+0000 7fb6f3a12640 1 -- 172.21.15.39:0/2558016805 shutdown_connections 2024-09-21T03:51:05.891 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:05.887+0000 7fb6f3a12640 1 --2- 172.21.15.39:0/2558016805 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb6ec10ddf0 0x7fb6ec110300 unknown :-1 s=CLOSED pgs=817 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:05.891 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:05.887+0000 7fb6f3a12640 1 --2- 172.21.15.39:0/2558016805 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb6ec105270 0x7fb6ec10d8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:05.891 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:05.887+0000 7fb6f3a12640 1 --2- 172.21.15.39:0/2558016805 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb6ec104840 0x7fb6ec104c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:05.891 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:05.887+0000 7fb6f3a12640 1 -- 172.21.15.39:0/2558016805 >> 172.21.15.39:0/2558016805 conn(0x7fb6ec079730 msgr2=0x7fb6ec079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:51:05.891 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:05.887+0000 7fb6f3a12640 1 -- 172.21.15.39:0/2558016805 shutdown_connections 2024-09-21T03:51:05.891 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:05.887+0000 7fb6f3a12640 1 -- 172.21.15.39:0/2558016805 wait complete. 2024-09-21T03:51:05.892 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:05.888+0000 7fb6f3a12640 1 Processor -- start 2024-09-21T03:51:05.892 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:05.888+0000 7fb6f3a12640 1 -- start start 2024-09-21T03:51:05.892 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:05.888+0000 7fb6f3a12640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb6ec104840 0x7fb6ec19a460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:51:05.893 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:05.889+0000 7fb6f3a12640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb6ec105270 0x7fb6ec19aa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:51:05.893 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:05.889+0000 7fb6f3a12640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb6ec10ddf0 0x7fb6ec19fca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:51:05.893 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:05.889+0000 7fb6f3a12640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fb6ec113a90 con 0x7fb6ec105270 2024-09-21T03:51:05.893 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:05.889+0000 7fb6f3a12640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fb6ec113910 con 0x7fb6ec10ddf0 2024-09-21T03:51:05.893 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:05.889+0000 7fb6f3a12640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fb6ec113c10 con 0x7fb6ec104840 2024-09-21T03:51:05.893 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:05.889+0000 7fb6f0fad640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb6ec105270 0x7fb6ec19aa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:51:05.893 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:05.889+0000 7fb6f0fad640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb6ec105270 0x7fb6ec19aa00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:57840/0 (socket says 172.21.15.39:57840) 2024-09-21T03:51:05.893 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:05.889+0000 7fb6f0fad640 1 -- 172.21.15.39:0/2217597550 learned_addr learned my addr 172.21.15.39:0/2217597550 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:51:05.893 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:05.889+0000 7fb6f17ae640 1 --2- 172.21.15.39:0/2217597550 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb6ec104840 0x7fb6ec19a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:51:05.893 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:05.889+0000 7fb6f1faf640 1 --2- 172.21.15.39:0/2217597550 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb6ec10ddf0 0x7fb6ec19fca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:51:05.893 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:05.889+0000 7fb6f0fad640 1 -- 172.21.15.39:0/2217597550 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb6ec104840 msgr2=0x7fb6ec19a460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:51:05.893 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:05.889+0000 7fb6f0fad640 1 --2- 172.21.15.39:0/2217597550 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb6ec104840 0x7fb6ec19a460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:05.893 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:05.889+0000 7fb6f0fad640 1 -- 172.21.15.39:0/2217597550 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb6ec10ddf0 msgr2=0x7fb6ec19fca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:51:05.893 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:05.889+0000 7fb6f0fad640 1 --2- 172.21.15.39:0/2217597550 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb6ec10ddf0 0x7fb6ec19fca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:05.894 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:05.889+0000 7fb6f0fad640 1 -- 172.21.15.39:0/2217597550 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb6ec1a03d0 con 0x7fb6ec105270 2024-09-21T03:51:05.894 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:05.889+0000 7fb6f1faf640 1 --2- 172.21.15.39:0/2217597550 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb6ec10ddf0 0x7fb6ec19fca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:51:05.894 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:05.889+0000 7fb6f0fad640 1 --2- 172.21.15.39:0/2217597550 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb6ec105270 0x7fb6ec19aa00 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7fb6e000d950 tx=0x7fb6e000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:51:05.894 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:05.889+0000 7fb6de7fc640 1 -- 172.21.15.39:0/2217597550 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6e0017070 con 0x7fb6ec105270 2024-09-21T03:51:05.894 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:05.889+0000 7fb6f3a12640 1 -- 172.21.15.39:0/2217597550 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb6ec1a06c0 con 0x7fb6ec105270 2024-09-21T03:51:05.894 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:05.890+0000 7fb6de7fc640 1 -- 172.21.15.39:0/2217597550 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb6e0009c30 con 0x7fb6ec105270 2024-09-21T03:51:05.894 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:05.890+0000 7fb6de7fc640 1 -- 172.21.15.39:0/2217597550 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6e00047a0 con 0x7fb6ec105270 2024-09-21T03:51:05.894 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:05.890+0000 7fb6f3a12640 1 -- 172.21.15.39:0/2217597550 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb6ec1a6e20 con 0x7fb6ec105270 2024-09-21T03:51:05.895 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:05.893+0000 7fb6f9f86740 1 Processor -- start 2024-09-21T03:51:05.896 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:05.893+0000 7fb6f9f86740 1 -- start start 2024-09-21T03:51:05.897 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:05.893+0000 7fb6f9f86740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561e88e00070 0x561e891a7650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:51:05.897 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:05.893+0000 7fb6f9f86740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561e88dd71b0 0x561e891a7b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:51:05.898 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:05.893+0000 7fb6f9f86740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561e88ddf390 0x561e891a80d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:51:05.898 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:05.893+0000 7fb6f9f86740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x561e8911de50 con 0x561e88e00070 2024-09-21T03:51:05.898 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:05.893+0000 7fb6f9f86740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x561e8910d280 con 0x561e88dd71b0 2024-09-21T03:51:05.898 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:05.893+0000 7fb6f9f86740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x561e89111090 con 0x561e88ddf390 2024-09-21T03:51:05.898 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:05.893+0000 7fb6de7fc640 1 -- 172.21.15.39:0/2217597550 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 27) ==== 154643+0+0 (secure 0 0 0) 0x7fb6e0004020 con 0x7fb6ec105270 2024-09-21T03:51:05.898 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:05.893+0000 7fb6f1faf640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561e88e00070 0x561e891a7650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:51:05.898 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:05.894+0000 7fb6f1faf640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561e88e00070 0x561e891a7650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:57848/0 (socket says 172.21.15.39:57848) 2024-09-21T03:51:05.898 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:05.894+0000 7fb6f1faf640 1 -- 172.21.15.39:0/29105209 learned_addr learned my addr 172.21.15.39:0/29105209 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:51:05.898 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:05.894+0000 7fb6f17ae640 1 --2- 172.21.15.39:0/29105209 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561e88dd71b0 0x561e891a7b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:51:05.898 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:05.894+0000 7fb6f0fad640 1 --2- 172.21.15.39:0/29105209 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561e88ddf390 0x561e891a80d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:51:05.898 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:05.894+0000 7fb6f1faf640 1 -- 172.21.15.39:0/29105209 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561e88ddf390 msgr2=0x561e891a80d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:51:05.898 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:05.894+0000 7fb6f1faf640 1 --2- 172.21.15.39:0/29105209 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561e88ddf390 0x561e891a80d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:05.898 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:05.894+0000 7fb6f1faf640 1 -- 172.21.15.39:0/29105209 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561e88dd71b0 msgr2=0x561e891a7b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:51:05.899 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:05.894+0000 7fb6f1faf640 1 --2- 172.21.15.39:0/29105209 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561e88dd71b0 0x561e891a7b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:05.899 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:05.894+0000 7fb6f1faf640 1 -- 172.21.15.39:0/29105209 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561e88e6b650 con 0x561e88e00070 2024-09-21T03:51:05.899 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:05.894+0000 7fb6f0fad640 1 --2- 172.21.15.39:0/29105209 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561e88ddf390 0x561e891a80d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:51:05.899 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:05.894+0000 7fb6f1faf640 1 --2- 172.21.15.39:0/29105209 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561e88e00070 0x561e891a7650 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7fb6e4006010 tx=0x7fb6e4009800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de8121d765a88384 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:51:05.899 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:05.894+0000 7fb6de7fc640 1 --2- 172.21.15.39:0/2217597550 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7fb6d80b6680 0x7fb6d80b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:51:05.899 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:05.894+0000 7fb6d77fe640 1 -- 172.21.15.39:0/29105209 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6e4031cf0 con 0x561e88e00070 2024-09-21T03:51:05.899 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:05.895+0000 7fb6d77fe640 1 -- 172.21.15.39:0/29105209 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb6e4036070 con 0x561e88e00070 2024-09-21T03:51:05.899 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:05.895+0000 7fb6d77fe640 1 -- 172.21.15.39:0/29105209 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6e40042a0 con 0x561e88e00070 2024-09-21T03:51:05.899 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:05.895+0000 7fb6f17ae640 1 --2- 172.21.15.39:0/2217597550 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7fb6d80b6680 0x7fb6d80b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:51:05.899 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:05.895+0000 7fb6de7fc640 1 -- 172.21.15.39:0/2217597550 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8484+0+0 (secure 0 0 0) 0x7fb6e00e7480 con 0x7fb6ec105270 2024-09-21T03:51:05.902 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:05.896+0000 7fb6f9f86740 1 -- 172.21.15.39:0/29105209 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561e88e00070 msgr2=0x561e891a7650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:51:05.902 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:05.896+0000 7fb6f17ae640 1 --2- 172.21.15.39:0/2217597550 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7fb6d80b6680 0x7fb6d80b8b90 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x561e891a62d0 tx=0x7fb6e8005f90 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:51:05.902 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:05.896+0000 7fb6f9f86740 1 --2- 172.21.15.39:0/29105209 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561e88e00070 0x561e891a7650 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7fb6e4006010 tx=0x7fb6e4009800 comp rx=0 tx=0).stop 2024-09-21T03:51:05.902 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:05.896+0000 7fb6f9f86740 1 -- 172.21.15.39:0/29105209 shutdown_connections 2024-09-21T03:51:05.902 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:05.896+0000 7fb6f9f86740 1 --2- 172.21.15.39:0/29105209 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561e88ddf390 0x561e891a80d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:05.902 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:05.896+0000 7fb6f9f86740 1 --2- 172.21.15.39:0/29105209 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561e88dd71b0 0x561e891a7b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:05.902 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:05.896+0000 7fb6f9f86740 1 --2- 172.21.15.39:0/29105209 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561e88e00070 0x561e891a7650 unknown :-1 s=CLOSED pgs=819 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:05.902 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:05.896+0000 7fb6f9f86740 1 -- 172.21.15.39:0/29105209 >> 172.21.15.39:0/29105209 conn(0x561e88df4460 msgr2=0x561e891a5540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:51:05.902 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:05.896+0000 7fb6f9f86740 1 -- 172.21.15.39:0/29105209 shutdown_connections 2024-09-21T03:51:05.903 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:05.896+0000 7fb6f9f86740 1 -- 172.21.15.39:0/29105209 wait complete. 2024-09-21T03:51:05.903 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:05.896+0000 7fb6f9f86740 1 -- start start 2024-09-21T03:51:05.903 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:05.896+0000 7fb6f9f86740 1 Processor -- start 2024-09-21T03:51:05.903 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:05.897+0000 7fb6f9f86740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561e88dd71b0 0x561e891b2c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:51:05.903 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:05.897+0000 7fb6f9f86740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561e88ddf390 0x561e891b31c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:51:05.903 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:05.897+0000 7fb6f9f86740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561e88e00070 0x561e891b3700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:51:05.903 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:05.897+0000 7fb6f9f86740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x561e88f6e700 con 0x561e88dd71b0 2024-09-21T03:51:05.903 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:05.897+0000 7fb6f9f86740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x561e890a8110 con 0x561e88e00070 2024-09-21T03:51:05.903 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:05.897+0000 7fb6f9f86740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x561e891b5dd0 con 0x561e88ddf390 2024-09-21T03:51:05.903 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:05.897+0000 7fb6f9f86740 10 client.0 _ll_register_callbacks cb 0x561e890c1990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:51:05.903 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:05.897+0000 7fb6f17ae640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561e88dd71b0 0x561e891b2c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:51:05.903 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:05.897+0000 7fb6f17ae640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561e88dd71b0 0x561e891b2c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:57856/0 (socket says 172.21.15.39:57856) 2024-09-21T03:51:05.903 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:05.897+0000 7fb6f17ae640 1 -- 172.21.15.39:0/575408708 learned_addr learned my addr 172.21.15.39:0/575408708 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:51:05.903 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:05.897+0000 7fb6f0fad640 1 --2- 172.21.15.39:0/575408708 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561e88ddf390 0x561e891b31c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:51:05.903 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:05.897+0000 7fb6f1faf640 1 --2- 172.21.15.39:0/575408708 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561e88e00070 0x561e891b3700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:51:05.904 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:05.897+0000 7fb6f17ae640 1 -- 172.21.15.39:0/575408708 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561e88ddf390 msgr2=0x561e891b31c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:51:05.904 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:05.897+0000 7fb6f17ae640 1 --2- 172.21.15.39:0/575408708 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561e88ddf390 0x561e891b31c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:05.904 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:05.897+0000 7fb6f17ae640 1 -- 172.21.15.39:0/575408708 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561e88e00070 msgr2=0x561e891b3700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:51:05.904 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:05.897+0000 7fb6f17ae640 1 --2- 172.21.15.39:0/575408708 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561e88e00070 0x561e891b3700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:05.904 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:05.897+0000 7fb6f17ae640 1 -- 172.21.15.39:0/575408708 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561e890d5c40 con 0x561e88dd71b0 2024-09-21T03:51:05.904 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:05.897+0000 7fb6f1faf640 1 --2- 172.21.15.39:0/575408708 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561e88e00070 0x561e891b3700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:51:05.904 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:05.898+0000 7fb6f17ae640 1 --2- 172.21.15.39:0/575408708 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561e88dd71b0 0x561e891b2c80 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7fb6e8013010 tx=0x7fb6e8004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8aad480341f2b3a8 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:51:05.904 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:05.898+0000 7fb6d4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-09-21T03:51:05.904 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:05.898+0000 7fb6d4ff9640 1 -- 172.21.15.39:0/575408708 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6e8004020 con 0x561e88dd71b0 2024-09-21T03:51:05.904 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:05.898+0000 7fb6f9f86740 1 -- 172.21.15.39:0/575408708 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb6e4036070 con 0x561e88dd71b0 2024-09-21T03:51:05.904 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:05.898+0000 7fb6d4ff9640 1 -- 172.21.15.39:0/575408708 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8484+0+0 (secure 0 0 0) 0x7fb6e8018030 con 0x561e88dd71b0 2024-09-21T03:51:05.904 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:05.898+0000 7fb6d4ff9640 1 -- 172.21.15.39:0/575408708 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fb6e80041c0 con 0x561e88dd71b0 2024-09-21T03:51:05.904 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:05.898+0000 7fb6f9f86740 10 client.16728 fetch_fsmap learned FSMap version 29 2024-09-21T03:51:05.904 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:05.898+0000 7fb6f9f86740 1 -- 172.21.15.39:0/575408708 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x561e891c1a50 con 0x561e88dd71b0 2024-09-21T03:51:05.904 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:05.898+0000 7fb6d4ff9640 1 -- 172.21.15.39:0/575408708 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb6e4036070 con 0x561e88dd71b0 2024-09-21T03:51:05.905 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:05.898+0000 7fb6d4ff9640 1 -- 172.21.15.39:0/575408708 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fb6e802f020 con 0x561e88dd71b0 2024-09-21T03:51:05.905 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:05.898+0000 7fb6f9f86740 10 client.16728 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T03:51:05.905 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:05.898+0000 7fb6f9f86740 10 client.16728 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T03:51:05.905 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:05.898+0000 7fb6f9f86740 20 client.16728 populate_metadata read hostname 'smithi039' 2024-09-21T03:51:05.905 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:05.898+0000 7fb6f9f86740 1 --2- 172.21.15.39:0/575408708 >> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] conn(0x561e88dbe9f0 0x561e891c4f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:51:05.905 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:05.898+0000 7fb6f9f86740 4 client.16728 mds_command: new command op to 24695 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:51:05.905 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:05.898+0000 7fb6f9f86740 1 -- 172.21.15.39:0/575408708 --> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561e8911fda0 con 0x561e88dbe9f0 2024-09-21T03:51:05.905 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:05.899+0000 7fb6f0fad640 1 --2- 172.21.15.39:0/575408708 >> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] conn(0x561e88dbe9f0 0x561e891c4f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:51:05.905 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:05.899+0000 7fb6f0fad640 1 --2- 172.21.15.39:0/575408708 >> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] conn(0x561e88dbe9f0 0x561e891c4f60 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2fdcc0a4cc64d317 server_cookie=e076b6df21e8b1d7 in_seq=0 out_seq=0 2024-09-21T03:51:05.905 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:05.899+0000 7fb6d4ff9640 10 client.16728 ms_handle_connect on v2:172.21.15.150:6838/2949386907 2024-09-21T03:51:05.906 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:05.900+0000 7fb6d4ff9640 1 -- 172.21.15.39:0/575408708 <== mds.0 v2:172.21.15.150:6838/2949386907 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x561e8911fda0 con 0x561e88dbe9f0 2024-09-21T03:51:05.906 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:05.900+0000 7fb6d4ff9640 10 client.16728 handle_command_reply: tid=0 2024-09-21T03:51:05.906 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:05.900+0000 7fb6f9f86740 1 client.16728 shutdown 2024-09-21T03:51:05.906 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:05.900+0000 7fb6f9f86740 1 -- 172.21.15.39:0/575408708 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561e88dd71b0 msgr2=0x561e891b2c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:51:05.907 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:05.900+0000 7fb6f9f86740 1 --2- 172.21.15.39:0/575408708 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561e88dd71b0 0x561e891b2c80 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7fb6e8013010 tx=0x7fb6e8004730 comp rx=0 tx=0).stop 2024-09-21T03:51:05.907 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:05.900+0000 7fb6f9f86740 1 -- 172.21.15.39:0/575408708 shutdown_connections 2024-09-21T03:51:05.907 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:05.900+0000 7fb6f9f86740 1 --2- 172.21.15.39:0/575408708 >> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] conn(0x561e88dbe9f0 0x561e891c4f60 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:05.907 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:05.900+0000 7fb6f9f86740 1 --2- 172.21.15.39:0/575408708 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561e88e00070 0x561e891b3700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:05.907 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:05.900+0000 7fb6f9f86740 1 --2- 172.21.15.39:0/575408708 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561e88ddf390 0x561e891b31c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:05.907 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:05.900+0000 7fb6d4ff9640 0 client.16728 ms_handle_reset on v2:172.21.15.150:6838/2949386907 2024-09-21T03:51:05.907 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:05.900+0000 7fb6f9f86740 1 --2- 172.21.15.39:0/575408708 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561e88dd71b0 0x561e891b2c80 unknown :-1 s=CLOSED pgs=820 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:05.907 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:05.900+0000 7fb6f9f86740 1 -- 172.21.15.39:0/575408708 >> 172.21.15.39:0/575408708 conn(0x561e88df4460 msgr2=0x561e891aa830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:51:05.907 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:05.900+0000 7fb6f9f86740 1 -- 172.21.15.39:0/575408708 shutdown_connections 2024-09-21T03:51:05.907 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:05.900+0000 7fb6f9f86740 1 -- 172.21.15.39:0/575408708 wait complete. 2024-09-21T03:51:05.907 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:05.901+0000 7fb6f9f86740 20 client.16728 trim_cache size 0 max 16384 2024-09-21T03:51:05.921 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:05.918+0000 7fb6f9f86740 1 Processor -- start 2024-09-21T03:51:05.922 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:05.918+0000 7fb6f9f86740 1 -- start start 2024-09-21T03:51:05.922 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:05.918+0000 7fb6f9f86740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561e88dd71b0 0x561e891a9040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:51:05.922 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:05.918+0000 7fb6f9f86740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561e88ddf390 0x561e891b48b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:51:05.922 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:05.918+0000 7fb6f9f86740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561e88e00070 0x561e891afe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:51:05.922 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:05.918+0000 7fb6f9f86740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fb6ec1af780 con 0x561e88e00070 2024-09-21T03:51:05.922 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:05.918+0000 7fb6f9f86740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fb6ec1b12c0 con 0x561e88dd71b0 2024-09-21T03:51:05.922 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:05.918+0000 7fb6f9f86740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fb6ec1afe50 con 0x561e88ddf390 2024-09-21T03:51:05.922 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:05.918+0000 7fb6f1faf640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561e88e00070 0x561e891afe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:51:05.922 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:05.918+0000 7fb6f1faf640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561e88e00070 0x561e891afe70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:57872/0 (socket says 172.21.15.39:57872) 2024-09-21T03:51:05.922 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:05.918+0000 7fb6f1faf640 1 -- 172.21.15.39:0/3631302836 learned_addr learned my addr 172.21.15.39:0/3631302836 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:51:05.922 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:05.919+0000 7fb6f17ae640 1 --2- 172.21.15.39:0/3631302836 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561e88dd71b0 0x561e891a9040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:51:05.922 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:05.919+0000 7fb6f0fad640 1 --2- 172.21.15.39:0/3631302836 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561e88ddf390 0x561e891b48b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:51:05.922 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:05.919+0000 7fb6f1faf640 1 -- 172.21.15.39:0/3631302836 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561e88ddf390 msgr2=0x561e891b48b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:51:05.922 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:05.919+0000 7fb6f1faf640 1 --2- 172.21.15.39:0/3631302836 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561e88ddf390 0x561e891b48b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:05.923 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:05.919+0000 7fb6f1faf640 1 -- 172.21.15.39:0/3631302836 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561e88dd71b0 msgr2=0x561e891a9040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:51:05.923 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:05.919+0000 7fb6f1faf640 1 --2- 172.21.15.39:0/3631302836 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561e88dd71b0 0x561e891a9040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:05.923 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:05.919+0000 7fb6f1faf640 1 -- 172.21.15.39:0/3631302836 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561e891c5320 con 0x561e88e00070 2024-09-21T03:51:05.923 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:05.919+0000 7fb6f1faf640 1 --2- 172.21.15.39:0/3631302836 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561e88e00070 0x561e891afe70 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7fb6e4009870 tx=0x7fb6e4009540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=659b6306e298a3d8 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:51:05.925 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:05.919+0000 7fb6d6ffd640 1 -- 172.21.15.39:0/3631302836 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6e4052070 con 0x561e88e00070 2024-09-21T03:51:05.925 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:05.919+0000 7fb6d6ffd640 1 -- 172.21.15.39:0/3631302836 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb6e4036070 con 0x561e88e00070 2024-09-21T03:51:05.925 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:05.919+0000 7fb6d6ffd640 1 -- 172.21.15.39:0/3631302836 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6e4003d90 con 0x561e88e00070 2024-09-21T03:51:05.925 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:05.920+0000 7fb6f9f86740 1 -- 172.21.15.39:0/3631302836 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561e88e00070 msgr2=0x561e891afe70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:51:05.925 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:05.920+0000 7fb6f9f86740 1 --2- 172.21.15.39:0/3631302836 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561e88e00070 0x561e891afe70 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7fb6e4009870 tx=0x7fb6e4009540 comp rx=0 tx=0).stop 2024-09-21T03:51:05.925 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:05.920+0000 7fb6f9f86740 1 -- 172.21.15.39:0/3631302836 shutdown_connections 2024-09-21T03:51:05.925 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:05.920+0000 7fb6f9f86740 1 --2- 172.21.15.39:0/3631302836 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561e88e00070 0x561e891afe70 unknown :-1 s=CLOSED pgs=821 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:05.925 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:05.920+0000 7fb6f9f86740 1 --2- 172.21.15.39:0/3631302836 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561e88ddf390 0x561e891b48b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:05.926 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:05.920+0000 7fb6f9f86740 1 --2- 172.21.15.39:0/3631302836 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561e88dd71b0 0x561e891a9040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:05.926 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:05.920+0000 7fb6f9f86740 1 -- 172.21.15.39:0/3631302836 >> 172.21.15.39:0/3631302836 conn(0x561e88df4460 msgr2=0x561e891b41a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:51:05.926 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:05.920+0000 7fb6f9f86740 1 -- 172.21.15.39:0/3631302836 shutdown_connections 2024-09-21T03:51:05.926 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:05.920+0000 7fb6f9f86740 1 -- 172.21.15.39:0/3631302836 wait complete. 2024-09-21T03:51:05.926 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:05.920+0000 7fb6f9f86740 1 -- start start 2024-09-21T03:51:05.926 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:05.921+0000 7fb6f9f86740 1 Processor -- start 2024-09-21T03:51:05.926 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:05.921+0000 7fb6f9f86740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561e88dd71b0 0x561e891b9110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:51:05.926 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:05.921+0000 7fb6f9f86740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561e88ddf390 0x561e891b9650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:51:05.926 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:05.921+0000 7fb6f9f86740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561e88e00070 0x561e891b9b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:51:05.926 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:05.921+0000 7fb6f9f86740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fb6ec1b0bf0 con 0x561e88e00070 2024-09-21T03:51:05.926 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:05.921+0000 7fb6f9f86740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fb6ec1b0520 con 0x561e88dd71b0 2024-09-21T03:51:05.926 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:05.921+0000 7fb6f9f86740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fb6ec1ad760 con 0x561e88ddf390 2024-09-21T03:51:05.926 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:05.921+0000 7fb6f9f86740 10 client.0 _ll_register_callbacks cb 0x7fb6e80099e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:51:05.926 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:05.921+0000 7fb6f1faf640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561e88e00070 0x561e891b9b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:51:05.927 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:05.921+0000 7fb6f0fad640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561e88ddf390 0x561e891b9650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:51:05.927 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:05.921+0000 7fb6f1faf640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561e88e00070 0x561e891b9b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:57880/0 (socket says 172.21.15.39:57880) 2024-09-21T03:51:05.927 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:05.921+0000 7fb6f1faf640 1 -- 172.21.15.39:0/2301800459 learned_addr learned my addr 172.21.15.39:0/2301800459 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:51:05.927 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:05.921+0000 7fb6f17ae640 1 --2- 172.21.15.39:0/2301800459 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561e88dd71b0 0x561e891b9110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:51:05.927 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:05.922+0000 7fb6f1faf640 1 -- 172.21.15.39:0/2301800459 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561e88ddf390 msgr2=0x561e891b9650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:51:05.927 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:05.922+0000 7fb6f1faf640 1 --2- 172.21.15.39:0/2301800459 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561e88ddf390 0x561e891b9650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:05.927 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:05.922+0000 7fb6f1faf640 1 -- 172.21.15.39:0/2301800459 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561e88dd71b0 msgr2=0x561e891b9110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:51:05.927 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:05.922+0000 7fb6f1faf640 1 --2- 172.21.15.39:0/2301800459 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561e88dd71b0 0x561e891b9110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:05.927 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:05.922+0000 7fb6f1faf640 1 -- 172.21.15.39:0/2301800459 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561e8922ea40 con 0x561e88e00070 2024-09-21T03:51:05.927 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:05.922+0000 7fb6f1faf640 1 --2- 172.21.15.39:0/2301800459 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561e88e00070 0x561e891b9b90 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7fb6e4005e00 tx=0x7fb6e4009800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=68aeca633418dd3d server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:51:05.927 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:05.922+0000 7fb6d4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-09-21T03:51:05.927 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:05.922+0000 7fb6d4ff9640 1 -- 172.21.15.39:0/2301800459 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6e4052040 con 0x561e88e00070 2024-09-21T03:51:05.927 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:05.922+0000 7fb6d4ff9640 1 -- 172.21.15.39:0/2301800459 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8484+0+0 (secure 0 0 0) 0x7fb6e4031650 con 0x561e88e00070 2024-09-21T03:51:05.927 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:05.922+0000 7fb6f9f86740 1 -- 172.21.15.39:0/2301800459 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb6e4036070 con 0x561e88e00070 2024-09-21T03:51:05.927 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:05.922+0000 7fb6d4ff9640 1 -- 172.21.15.39:0/2301800459 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fb6e4031df0 con 0x561e88e00070 2024-09-21T03:51:05.928 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:05.922+0000 7fb6f9f86740 10 client.16734 fetch_fsmap learned FSMap version 29 2024-09-21T03:51:05.928 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:05.922+0000 7fb6f9f86740 1 -- 172.21.15.39:0/2301800459 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x561e8922fcd0 con 0x561e88e00070 2024-09-21T03:51:05.928 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:05.923+0000 7fb6d4ff9640 1 -- 172.21.15.39:0/2301800459 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb6e4036070 con 0x561e88e00070 2024-09-21T03:51:05.929 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:05.923+0000 7fb6d4ff9640 1 -- 172.21.15.39:0/2301800459 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fb6e405d070 con 0x561e88e00070 2024-09-21T03:51:05.929 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:05.923+0000 7fb6f9f86740 10 client.16734 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T03:51:05.929 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:05.923+0000 7fb6f9f86740 10 client.16734 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T03:51:05.929 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:05.923+0000 7fb6f9f86740 20 client.16734 populate_metadata read hostname 'smithi039' 2024-09-21T03:51:05.929 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:05.923+0000 7fb6f9f86740 1 --2- 172.21.15.39:0/2301800459 >> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] conn(0x561e88d93da0 0x561e89231f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:51:05.929 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:05.923+0000 7fb6f9f86740 4 client.16734 mds_command: new command op to 24695 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-21T03:51:05.930 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:05.923+0000 7fb6f9f86740 1 -- 172.21.15.39:0/2301800459 --> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] -- command(tid 0: {"prefix": "scrub status"}) -- 0x561e8911b850 con 0x561e88d93da0 2024-09-21T03:51:05.930 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:05.923+0000 7fb6f17ae640 1 --2- 172.21.15.39:0/2301800459 >> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] conn(0x561e88d93da0 0x561e89231f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:51:05.930 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:05.924+0000 7fb6f17ae640 1 --2- 172.21.15.39:0/2301800459 >> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] conn(0x561e88d93da0 0x561e89231f00 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=146cae32048e470e server_cookie=8458f7eec89909bd in_seq=0 out_seq=0 2024-09-21T03:51:05.930 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:05.924+0000 7fb6d4ff9640 10 client.16734 ms_handle_connect on v2:172.21.15.150:6838/2949386907 2024-09-21T03:51:05.930 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:05.924+0000 7fb6d4ff9640 1 -- 172.21.15.39:0/2301800459 <== mds.0 v2:172.21.15.150:6838/2949386907 1 ==== command_reply(tid 0: 0 ) ==== 8+0+2646 (crc 0 0 0) 0x561e8911b850 con 0x561e88d93da0 2024-09-21T03:51:05.930 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:05.924+0000 7fb6d4ff9640 10 client.16734 handle_command_reply: tid=0 2024-09-21T03:51:05.930 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:05.924+0000 7fb6f9f86740 1 client.16734 shutdown 2024-09-21T03:51:05.930 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:05.924+0000 7fb6f9f86740 1 -- 172.21.15.39:0/2301800459 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561e88e00070 msgr2=0x561e891b9b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:51:05.930 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:05.924+0000 7fb6f9f86740 1 --2- 172.21.15.39:0/2301800459 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561e88e00070 0x561e891b9b90 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7fb6e4005e00 tx=0x7fb6e4009800 comp rx=0 tx=0).stop 2024-09-21T03:51:05.930 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:05.924+0000 7fb6f9f86740 1 -- 172.21.15.39:0/2301800459 shutdown_connections 2024-09-21T03:51:05.930 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:05.924+0000 7fb6f9f86740 1 --2- 172.21.15.39:0/2301800459 >> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] conn(0x561e88d93da0 0x561e89231f00 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:05.930 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:05.924+0000 7fb6f9f86740 1 --2- 172.21.15.39:0/2301800459 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561e88e00070 0x561e891b9b90 unknown :-1 s=CLOSED pgs=822 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:05.930 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:05.924+0000 7fb6f9f86740 1 --2- 172.21.15.39:0/2301800459 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561e88ddf390 0x561e891b9650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:05.930 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:05.924+0000 7fb6f9f86740 1 --2- 172.21.15.39:0/2301800459 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561e88dd71b0 0x561e891b9110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:05.931 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:05.924+0000 7fb6f9f86740 1 -- 172.21.15.39:0/2301800459 >> 172.21.15.39:0/2301800459 conn(0x561e88df4460 msgr2=0x561e891aaee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:51:05.931 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:05.924+0000 7fb6d4ff9640 0 client.16734 ms_handle_reset on v2:172.21.15.150:6838/2949386907 2024-09-21T03:51:05.931 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:05.924+0000 7fb6f9f86740 1 -- 172.21.15.39:0/2301800459 shutdown_connections 2024-09-21T03:51:05.931 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:05.924+0000 7fb6f9f86740 1 -- 172.21.15.39:0/2301800459 wait complete. 2024-09-21T03:51:05.931 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:05.924+0000 7fb6f9f86740 20 client.16734 trim_cache size 0 max 16384 2024-09-21T03:51:05.931 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-09-21T03:51:05.931 INFO:teuthology.orchestra.run.smithi039.stdout: "status": "no active scrubs running", 2024-09-21T03:51:05.931 INFO:teuthology.orchestra.run.smithi039.stdout: "scrubs": {}, 2024-09-21T03:51:05.931 INFO:teuthology.orchestra.run.smithi039.stdout: "scrub_stats": { 2024-09-21T03:51:05.931 INFO:teuthology.orchestra.run.smithi039.stdout: "160750d8-401f-4810-866d-e63e7e4bc3e0": { 2024-09-21T03:51:05.931 INFO:teuthology.orchestra.run.smithi039.stdout: "start_time": "2024-09-21T03:51:04.208673+0000", 2024-09-21T03:51:05.931 INFO:teuthology.orchestra.run.smithi039.stdout: "path": "/", 2024-09-21T03:51:05.932 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_started": 0, 2024-09-21T03:51:05.932 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_passed": 0, 2024-09-21T03:51:05.932 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_failed": 0, 2024-09-21T03:51:05.932 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_skipped": 0 2024-09-21T03:51:05.932 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:51:05.932 INFO:teuthology.orchestra.run.smithi039.stdout: "74ab917e-ecea-4778-a1c3-c184214fb24c": { 2024-09-21T03:51:05.932 INFO:teuthology.orchestra.run.smithi039.stdout: "start_time": "2024-09-21T03:50:17.726630+0000", 2024-09-21T03:51:05.932 INFO:teuthology.orchestra.run.smithi039.stdout: "path": "/", 2024-09-21T03:51:05.932 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_started": 0, 2024-09-21T03:51:05.932 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_passed": 0, 2024-09-21T03:51:05.932 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_failed": 0, 2024-09-21T03:51:05.932 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_skipped": 0 2024-09-21T03:51:05.932 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:51:05.932 INFO:teuthology.orchestra.run.smithi039.stdout: "6e760f1f-9048-4f92-bd66-d708fbef75b3": { 2024-09-21T03:51:05.933 INFO:teuthology.orchestra.run.smithi039.stdout: "start_time": "2024-09-21T03:50:03.711918+0000", 2024-09-21T03:51:05.933 INFO:teuthology.orchestra.run.smithi039.stdout: "path": "/", 2024-09-21T03:51:05.933 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_started": 0, 2024-09-21T03:51:05.933 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_passed": 0, 2024-09-21T03:51:05.933 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_failed": 0, 2024-09-21T03:51:05.933 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_skipped": 0 2024-09-21T03:51:05.933 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:51:05.933 INFO:teuthology.orchestra.run.smithi039.stdout: "e28ad743-2b22-45b3-8799-a638c2883623": { 2024-09-21T03:51:05.933 INFO:teuthology.orchestra.run.smithi039.stdout: "start_time": "2024-09-21T03:48:31.448821+0000", 2024-09-21T03:51:05.933 INFO:teuthology.orchestra.run.smithi039.stdout: "path": "/", 2024-09-21T03:51:05.933 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_started": 0, 2024-09-21T03:51:05.933 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_passed": 0, 2024-09-21T03:51:05.933 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_failed": 0, 2024-09-21T03:51:05.933 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_skipped": 0 2024-09-21T03:51:05.933 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:51:05.934 INFO:teuthology.orchestra.run.smithi039.stdout: "29411307-eb8d-4339-8d2b-47ebd32c0080": { 2024-09-21T03:51:05.934 INFO:teuthology.orchestra.run.smithi039.stdout: "start_time": "2024-09-21T03:47:38.092920+0000", 2024-09-21T03:51:05.934 INFO:teuthology.orchestra.run.smithi039.stdout: "path": "/", 2024-09-21T03:51:05.934 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_started": 0, 2024-09-21T03:51:05.934 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_passed": 0, 2024-09-21T03:51:05.934 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_failed": 0, 2024-09-21T03:51:05.934 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_skipped": 0 2024-09-21T03:51:05.934 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:51:05.934 INFO:teuthology.orchestra.run.smithi039.stdout: "9860c2f4-b7b1-47aa-865a-9a1c4434a5f6": { 2024-09-21T03:51:05.934 INFO:teuthology.orchestra.run.smithi039.stdout: "start_time": "2024-09-21T03:49:17.718867+0000", 2024-09-21T03:51:05.934 INFO:teuthology.orchestra.run.smithi039.stdout: "path": "/", 2024-09-21T03:51:05.934 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_started": 0, 2024-09-21T03:51:05.934 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_passed": 0, 2024-09-21T03:51:05.934 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_failed": 0, 2024-09-21T03:51:05.934 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_skipped": 0 2024-09-21T03:51:05.935 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:51:05.935 INFO:teuthology.orchestra.run.smithi039.stdout: "c7e2b8e4-057d-435f-b2bd-b633de98035f": { 2024-09-21T03:51:05.935 INFO:teuthology.orchestra.run.smithi039.stdout: "start_time": "2024-09-21T03:47:19.400305+0000", 2024-09-21T03:51:05.935 INFO:teuthology.orchestra.run.smithi039.stdout: "path": "/", 2024-09-21T03:51:05.935 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_started": 0, 2024-09-21T03:51:05.935 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_passed": 0, 2024-09-21T03:51:05.935 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_failed": 0, 2024-09-21T03:51:05.935 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_skipped": 0 2024-09-21T03:51:05.935 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:51:05.935 INFO:teuthology.orchestra.run.smithi039.stdout: "10728de8-c49d-4edc-be2b-639249255e94": { 2024-09-21T03:51:05.935 INFO:teuthology.orchestra.run.smithi039.stdout: "start_time": "2024-09-21T03:47:03.965623+0000", 2024-09-21T03:51:05.935 INFO:teuthology.orchestra.run.smithi039.stdout: "path": "/", 2024-09-21T03:51:05.935 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_started": 0, 2024-09-21T03:51:05.935 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_passed": 0, 2024-09-21T03:51:05.935 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_failed": 0, 2024-09-21T03:51:05.936 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_skipped": 0 2024-09-21T03:51:05.936 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:51:05.936 INFO:teuthology.orchestra.run.smithi039.stdout: "02509e4f-3911-4983-b16f-df8ded0b9998": { 2024-09-21T03:51:05.936 INFO:teuthology.orchestra.run.smithi039.stdout: "start_time": "2024-09-21T03:46:17.889576+0000", 2024-09-21T03:51:05.936 INFO:teuthology.orchestra.run.smithi039.stdout: "path": "/", 2024-09-21T03:51:05.936 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_started": 0, 2024-09-21T03:51:05.936 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_passed": 0, 2024-09-21T03:51:05.936 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_failed": 0, 2024-09-21T03:51:05.936 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_skipped": 0 2024-09-21T03:51:05.936 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-21T03:51:05.936 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-21T03:51:05.936 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-09-21T03:51:05.937 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:05.925+0000 7fb6f3a12640 1 -- 172.21.15.39:0/2217597550 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7fb6d80b6680 msgr2=0x7fb6d80b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:51:05.937 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:05.925+0000 7fb6f3a12640 1 --2- 172.21.15.39:0/2217597550 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7fb6d80b6680 0x7fb6d80b8b90 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x561e891a62d0 tx=0x7fb6e8005f90 comp rx=0 tx=0).stop 2024-09-21T03:51:05.937 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:05.925+0000 7fb6f3a12640 1 -- 172.21.15.39:0/2217597550 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb6ec105270 msgr2=0x7fb6ec19aa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:51:05.937 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:05.925+0000 7fb6f3a12640 1 --2- 172.21.15.39:0/2217597550 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb6ec105270 0x7fb6ec19aa00 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7fb6e000d950 tx=0x7fb6e000de10 comp rx=0 tx=0).stop 2024-09-21T03:51:05.937 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:05.925+0000 7fb6f3a12640 1 -- 172.21.15.39:0/2217597550 shutdown_connections 2024-09-21T03:51:05.937 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:05.925+0000 7fb6f3a12640 1 --2- 172.21.15.39:0/2217597550 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7fb6d80b6680 0x7fb6d80b8b90 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:05.937 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:05.925+0000 7fb6f3a12640 1 --2- 172.21.15.39:0/2217597550 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb6ec10ddf0 0x7fb6ec19fca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:05.937 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:05.925+0000 7fb6f3a12640 1 --2- 172.21.15.39:0/2217597550 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb6ec105270 0x7fb6ec19aa00 unknown :-1 s=CLOSED pgs=818 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:05.937 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:05.925+0000 7fb6f3a12640 1 --2- 172.21.15.39:0/2217597550 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb6ec104840 0x7fb6ec19a460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:05.938 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:05.925+0000 7fb6f3a12640 1 -- 172.21.15.39:0/2217597550 >> 172.21.15.39:0/2217597550 conn(0x7fb6ec079730 msgr2=0x7fb6ec107970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:51:05.938 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:05.925+0000 7fb6f3a12640 1 -- 172.21.15.39:0/2217597550 shutdown_connections 2024-09-21T03:51:05.938 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:05.925+0000 7fb6f3a12640 1 -- 172.21.15.39:0/2217597550 wait complete. 2024-09-21T03:51:06.772 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-21T03:51:06.772 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 51116d5c-77ca-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T03:51:07.034 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/51116d5c-77ca-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:51:07.297 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:51:07 smithi039 ceph-mon[31211]: pgmap v477: 97 pgs: 97 active+clean; 10 MiB data, 520 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 25 op/s 2024-09-21T03:51:07.468 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:51:07 smithi161 ceph-mon[34533]: pgmap v477: 97 pgs: 97 active+clean; 10 MiB data, 520 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 25 op/s 2024-09-21T03:51:07.474 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:51:07 smithi150 ceph-mon[34289]: pgmap v477: 97 pgs: 97 active+clean; 10 MiB data, 520 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 25 op/s 2024-09-21T03:51:07.520 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:07.509+0000 7fbdc4687640 1 -- 172.21.15.39:0/1913989422 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbdbc1083a0 msgr2=0x7fbdbc1087b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:51:07.520 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:07.509+0000 7fbdc4687640 1 --2- 172.21.15.39:0/1913989422 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbdbc1083a0 0x7fbdbc1087b0 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7fbdb0009ca0 tx=0x7fbdb002a8c0 comp rx=0 tx=0).stop 2024-09-21T03:51:07.520 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:07.509+0000 7fbdc4687640 1 -- 172.21.15.39:0/1913989422 shutdown_connections 2024-09-21T03:51:07.520 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:07.509+0000 7fbdc4687640 1 --2- 172.21.15.39:0/1913989422 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbdbc108cf0 0x7fbdbc10f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:07.520 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:07.509+0000 7fbdc4687640 1 --2- 172.21.15.39:0/1913989422 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbdbc1083a0 0x7fbdbc1087b0 unknown :-1 s=CLOSED pgs=823 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:07.520 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:07.509+0000 7fbdc4687640 1 --2- 172.21.15.39:0/1913989422 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbdbc102a60 0x7fbdbc102e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:07.520 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:07.509+0000 7fbdc4687640 1 -- 172.21.15.39:0/1913989422 >> 172.21.15.39:0/1913989422 conn(0x7fbdbc079730 msgr2=0x7fbdbc079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:51:07.521 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:07.509+0000 7fbdc4687640 1 -- 172.21.15.39:0/1913989422 shutdown_connections 2024-09-21T03:51:07.521 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:07.509+0000 7fbdc4687640 1 -- 172.21.15.39:0/1913989422 wait complete. 2024-09-21T03:51:07.521 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:07.510+0000 7fbdc4687640 1 Processor -- start 2024-09-21T03:51:07.521 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:07.510+0000 7fbdc4687640 1 -- start start 2024-09-21T03:51:07.521 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:07.510+0000 7fbdc4687640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbdbc102a60 0x7fbdbc19a4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:51:07.521 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:07.510+0000 7fbdc4687640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbdbc1083a0 0x7fbdbc19aa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:51:07.521 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:07.510+0000 7fbdc4687640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbdbc108cf0 0x7fbdbc19fd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:51:07.521 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:07.510+0000 7fbdc4687640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fbdbc112960 con 0x7fbdbc102a60 2024-09-21T03:51:07.521 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:07.510+0000 7fbdc4687640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fbdbc1127e0 con 0x7fbdbc108cf0 2024-09-21T03:51:07.521 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:07.510+0000 7fbdc4687640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fbdbc112ae0 con 0x7fbdbc1083a0 2024-09-21T03:51:07.521 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:07.510+0000 7fbdc2423640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbdbc102a60 0x7fbdbc19a4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:51:07.521 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:07.510+0000 7fbdc2423640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbdbc102a60 0x7fbdbc19a4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:60350/0 (socket says 172.21.15.39:60350) 2024-09-21T03:51:07.521 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:07.510+0000 7fbdc2423640 1 -- 172.21.15.39:0/3433736509 learned_addr learned my addr 172.21.15.39:0/3433736509 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:51:07.521 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:07.511+0000 7fbdc1c22640 1 --2- 172.21.15.39:0/3433736509 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbdbc1083a0 0x7fbdbc19aa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:51:07.521 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:07.511+0000 7fbdc2c24640 1 --2- 172.21.15.39:0/3433736509 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbdbc108cf0 0x7fbdbc19fd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:51:07.522 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:07.511+0000 7fbdc2423640 1 -- 172.21.15.39:0/3433736509 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbdbc1083a0 msgr2=0x7fbdbc19aa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:51:07.522 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:07.511+0000 7fbdc2423640 1 --2- 172.21.15.39:0/3433736509 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbdbc1083a0 0x7fbdbc19aa60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:07.522 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:07.511+0000 7fbdc2423640 1 -- 172.21.15.39:0/3433736509 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbdbc108cf0 msgr2=0x7fbdbc19fd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:51:07.522 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:07.511+0000 7fbdc2423640 1 --2- 172.21.15.39:0/3433736509 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbdbc108cf0 0x7fbdbc19fd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:07.522 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:07.511+0000 7fbdc2423640 1 -- 172.21.15.39:0/3433736509 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbdbc1a0430 con 0x7fbdbc102a60 2024-09-21T03:51:07.522 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:07.511+0000 7fbdc1c22640 1 --2- 172.21.15.39:0/3433736509 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbdbc1083a0 0x7fbdbc19aa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:51:07.522 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:07.511+0000 7fbdc2423640 1 --2- 172.21.15.39:0/3433736509 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbdbc102a60 0x7fbdbc19a4c0 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7fbdb400d770 tx=0x7fbdb400dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:51:07.522 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:07.511+0000 7fbdbb7fe640 1 -- 172.21.15.39:0/3433736509 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbdb4017070 con 0x7fbdbc102a60 2024-09-21T03:51:07.522 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:07.511+0000 7fbdbb7fe640 1 -- 172.21.15.39:0/3433736509 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbdb40054d0 con 0x7fbdbc102a60 2024-09-21T03:51:07.522 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:07.511+0000 7fbdc4687640 1 -- 172.21.15.39:0/3433736509 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbdbc1a0720 con 0x7fbdbc102a60 2024-09-21T03:51:07.522 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:07.511+0000 7fbdbb7fe640 1 -- 172.21.15.39:0/3433736509 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbdb4004c30 con 0x7fbdbc102a60 2024-09-21T03:51:07.522 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:07.511+0000 7fbdc4687640 1 -- 172.21.15.39:0/3433736509 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbdbc1a6e80 con 0x7fbdbc102a60 2024-09-21T03:51:07.522 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:07.512+0000 7fbdc4687640 1 -- 172.21.15.39:0/3433736509 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbdbc102e70 con 0x7fbdbc102a60 2024-09-21T03:51:07.522 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:07.514+0000 7fbdbb7fe640 1 -- 172.21.15.39:0/3433736509 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 27) ==== 154643+0+0 (secure 0 0 0) 0x7fbdb4004dd0 con 0x7fbdbc102a60 2024-09-21T03:51:07.523 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:07.515+0000 7fbdbb7fe640 1 --2- 172.21.15.39:0/3433736509 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7fbda40b6700 0x7fbda40b8c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:51:07.523 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:07.515+0000 7fbdc1c22640 1 --2- 172.21.15.39:0/3433736509 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7fbda40b6700 0x7fbda40b8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:51:07.523 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:07.515+0000 7fbdbb7fe640 1 -- 172.21.15.39:0/3433736509 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8484+0+0 (secure 0 0 0) 0x7fbdb40e75b0 con 0x7fbdbc102a60 2024-09-21T03:51:07.523 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:07.515+0000 7fbdc1c22640 1 --2- 172.21.15.39:0/3433736509 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7fbda40b6700 0x7fbda40b8c10 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7fbdb0006010 tx=0x7fbdb0000f40 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:51:07.523 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:07.518+0000 7fbdbb7fe640 1 -- 172.21.15.39:0/3433736509 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fbdb40adba0 con 0x7fbdbc102a60 2024-09-21T03:51:07.749 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:07.746+0000 7fbdc4687640 1 -- 172.21.15.39:0/3433736509 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fbdbc02e640 con 0x7fbdbc102a60 2024-09-21T03:51:07.749 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:07.746+0000 7fbdbb7fe640 1 -- 172.21.15.39:0/3433736509 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11978 (secure 0 0 0) 0x7fbdb40b1f80 con 0x7fbdbc102a60 2024-09-21T03:51:07.749 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T03:51:07.750 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":29,"btime":"2024-09-21T03:46:08:982881+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14790,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6835/508995295","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":508995295},{"type":"v1","addr":"172.21.15.39:6835","nonce":508995295}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14802,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6837/3915912607","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":3915912607},{"type":"v1","addr":"172.21.15.39:6837","nonce":3915912607}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":29,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T03:44:15.009872+0000","modified":"2024-09-21T03:46:08.430113+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24695,"mds_1":24667,"mds_2":24659,"mds_3":24646,"mds_4":14868},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14856":{"gid":14856,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.39:6839/185509192","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6838","nonce":185509192},{"type":"v1","addr":"172.21.15.39:6839","nonce":185509192}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14868":{"gid":14868,"name":"a","rank":4,"incarnation":25,"state":"up:active","state_seq":7,"addr":"172.21.15.39:6841/4247190456","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6840","nonce":4247190456},{"type":"v1","addr":"172.21.15.39:6841","nonce":4247190456}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24599":{"gid":24599,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.150:6833/3114643217","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":3114643217},{"type":"v1","addr":"172.21.15.150:6833","nonce":3114643217}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24601":{"gid":24601,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.161:6833/2963990654","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":2963990654},{"type":"v1","addr":"172.21.15.161:6833","nonce":2963990654}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24632":{"gid":24632,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.150:6835/724594008","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6834","nonce":724594008},{"type":"v1","addr":"172.21.15.150:6835","nonce":724594008}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24640":{"gid":24640,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.161:6835/3747882110","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":3747882110},{"type":"v1","addr":"172.21.15.161:6835","nonce":3747882110}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24646":{"gid":24646,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":10,"addr":"172.21.15.161:6837/3451544597","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6836","nonce":3451544597},{"type":"v1","addr":"172.21.15.161:6837","nonce":3451544597}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24659":{"gid":24659,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.150:6837/4017043144","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6836","nonce":4017043144},{"type":"v1","addr":"172.21.15.150:6837","nonce":4017043144}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24667":{"gid":24667,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.161:6839/2190107270","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6838","nonce":2190107270},{"type":"v1","addr":"172.21.15.161:6839","nonce":2190107270}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24695":{"gid":24695,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.150:6839/2949386907","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6838","nonce":2949386907},{"type":"v1","addr":"172.21.15.150:6839","nonce":2949386907}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24695,"qdb_cluster":[24695,24667,24659,24646,14868]},"id":1}]} 2024-09-21T03:51:07.750 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 29 2024-09-21T03:51:07.753 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:07.749+0000 7fbdc4687640 1 -- 172.21.15.39:0/3433736509 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7fbda40b6700 msgr2=0x7fbda40b8c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:51:07.753 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:07.749+0000 7fbdc4687640 1 --2- 172.21.15.39:0/3433736509 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7fbda40b6700 0x7fbda40b8c10 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7fbdb0006010 tx=0x7fbdb0000f40 comp rx=0 tx=0).stop 2024-09-21T03:51:07.753 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:07.749+0000 7fbdc4687640 1 -- 172.21.15.39:0/3433736509 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbdbc102a60 msgr2=0x7fbdbc19a4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:51:07.753 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:07.749+0000 7fbdc4687640 1 --2- 172.21.15.39:0/3433736509 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbdbc102a60 0x7fbdbc19a4c0 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7fbdb400d770 tx=0x7fbdb400dc30 comp rx=0 tx=0).stop 2024-09-21T03:51:07.753 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:07.749+0000 7fbdc4687640 1 -- 172.21.15.39:0/3433736509 shutdown_connections 2024-09-21T03:51:07.753 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:07.749+0000 7fbdc4687640 1 --2- 172.21.15.39:0/3433736509 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7fbda40b6700 0x7fbda40b8c10 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:07.753 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:07.749+0000 7fbdc4687640 1 --2- 172.21.15.39:0/3433736509 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbdbc108cf0 0x7fbdbc19fd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:07.753 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:07.749+0000 7fbdc4687640 1 --2- 172.21.15.39:0/3433736509 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbdbc1083a0 0x7fbdbc19aa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:07.753 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:07.749+0000 7fbdc4687640 1 --2- 172.21.15.39:0/3433736509 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbdbc102a60 0x7fbdbc19a4c0 unknown :-1 s=CLOSED pgs=824 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:07.753 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:07.749+0000 7fbdc4687640 1 -- 172.21.15.39:0/3433736509 >> 172.21.15.39:0/3433736509 conn(0x7fbdbc079730 msgr2=0x7fbdbc06b290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:51:07.753 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:07.750+0000 7fbdc4687640 1 -- 172.21.15.39:0/3433736509 shutdown_connections 2024-09-21T03:51:07.753 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:07.750+0000 7fbdc4687640 1 -- 172.21.15.39:0/3433736509 wait complete. 2024-09-21T03:51:08.334 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:51:08 smithi039 ceph-mon[31211]: from='client.? 172.21.15.39:0/3433736509' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:51:08.468 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:51:08 smithi161 ceph-mon[34533]: from='client.? 172.21.15.39:0/3433736509' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:51:08.474 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:51:08 smithi150 ceph-mon[34289]: from='client.? 172.21.15.39:0/3433736509' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:51:08.497 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 51116d5c-77ca-11ef-baf5-efdc52797490 -- ceph tell mds.1:4 damage ls 2024-09-21T03:51:08.755 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/51116d5c-77ca-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:51:09.161 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:51:09 smithi039 ceph-mon[31211]: pgmap v478: 97 pgs: 97 active+clean; 10 MiB data, 520 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-09-21T03:51:09.270 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:09.252+0000 7f3eabb8a640 1 -- 172.21.15.39:0/911842145 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3ea410dec0 msgr2=0x7f3ea41103b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:51:09.271 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:09.252+0000 7f3eabb8a640 1 --2- 172.21.15.39:0/911842145 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3ea410dec0 0x7f3ea41103b0 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7f3e9c009ca0 tx=0x7f3e9c02a8c0 comp rx=0 tx=0).stop 2024-09-21T03:51:09.271 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:09.252+0000 7f3eabb8a640 1 -- 172.21.15.39:0/911842145 shutdown_connections 2024-09-21T03:51:09.271 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:09.252+0000 7f3eabb8a640 1 --2- 172.21.15.39:0/911842145 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3ea410dec0 0x7f3ea41103b0 unknown :-1 s=CLOSED pgs=825 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:09.271 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:09.252+0000 7f3eabb8a640 1 --2- 172.21.15.39:0/911842145 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3ea41053b0 0x7f3ea410d900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:09.271 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:09.252+0000 7f3eabb8a640 1 --2- 172.21.15.39:0/911842145 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3ea4104a60 0x7f3ea4104e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:09.271 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:09.252+0000 7f3eabb8a640 1 -- 172.21.15.39:0/911842145 >> 172.21.15.39:0/911842145 conn(0x7f3ea40796d0 msgr2=0x7f3ea4079ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:51:09.271 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:09.252+0000 7f3eabb8a640 1 -- 172.21.15.39:0/911842145 shutdown_connections 2024-09-21T03:51:09.271 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:09.252+0000 7f3eabb8a640 1 -- 172.21.15.39:0/911842145 wait complete. 2024-09-21T03:51:09.271 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:09.252+0000 7f3eabb8a640 1 Processor -- start 2024-09-21T03:51:09.271 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:09.253+0000 7f3eabb8a640 1 -- start start 2024-09-21T03:51:09.271 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:09.253+0000 7f3eabb8a640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3ea4104a60 0x7f3ea41027a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:51:09.271 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:09.253+0000 7f3eabb8a640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3ea41053b0 0x7f3ea4102ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:51:09.272 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:09.253+0000 7f3eabb8a640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3ea410dec0 0x7f3ea4103220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:51:09.272 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:09.253+0000 7f3eabb8a640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f3ea4113b20 con 0x7f3ea4104a60 2024-09-21T03:51:09.272 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:09.253+0000 7f3eabb8a640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f3ea41139a0 con 0x7f3ea410dec0 2024-09-21T03:51:09.272 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:09.253+0000 7f3eabb8a640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f3ea4113ca0 con 0x7f3ea41053b0 2024-09-21T03:51:09.272 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:09.253+0000 7f3ea9926640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3ea4104a60 0x7f3ea41027a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:51:09.272 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:09.253+0000 7f3ea9926640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3ea4104a60 0x7f3ea41027a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:60376/0 (socket says 172.21.15.39:60376) 2024-09-21T03:51:09.272 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:09.253+0000 7f3ea9926640 1 -- 172.21.15.39:0/2593477323 learned_addr learned my addr 172.21.15.39:0/2593477323 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:51:09.272 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:09.253+0000 7f3ea9125640 1 --2- 172.21.15.39:0/2593477323 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3ea41053b0 0x7f3ea4102ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:51:09.272 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:09.253+0000 7f3eaa127640 1 --2- 172.21.15.39:0/2593477323 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3ea410dec0 0x7f3ea4103220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:51:09.272 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:09.253+0000 7f3ea9926640 1 -- 172.21.15.39:0/2593477323 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3ea41053b0 msgr2=0x7f3ea4102ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:51:09.272 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:09.253+0000 7f3ea9926640 1 --2- 172.21.15.39:0/2593477323 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3ea41053b0 0x7f3ea4102ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:09.272 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:09.253+0000 7f3ea9926640 1 -- 172.21.15.39:0/2593477323 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3ea410dec0 msgr2=0x7f3ea4103220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:51:09.273 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:09.253+0000 7f3ea9926640 1 --2- 172.21.15.39:0/2593477323 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3ea410dec0 0x7f3ea4103220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:09.273 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:09.253+0000 7f3ea9926640 1 -- 172.21.15.39:0/2593477323 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3ea41a86e0 con 0x7f3ea4104a60 2024-09-21T03:51:09.273 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:09.254+0000 7f3ea9926640 1 --2- 172.21.15.39:0/2593477323 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3ea4104a60 0x7f3ea41027a0 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7f3ea000c920 tx=0x7f3ea000cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:51:09.273 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:09.254+0000 7f3e9affd640 1 -- 172.21.15.39:0/2593477323 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ea0007c70 con 0x7f3ea4104a60 2024-09-21T03:51:09.273 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:09.254+0000 7f3e9affd640 1 -- 172.21.15.39:0/2593477323 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3ea0007e10 con 0x7f3ea4104a60 2024-09-21T03:51:09.273 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:09.254+0000 7f3eabb8a640 1 -- 172.21.15.39:0/2593477323 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3ea41a89d0 con 0x7f3ea4104a60 2024-09-21T03:51:09.273 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:09.254+0000 7f3e9affd640 1 -- 172.21.15.39:0/2593477323 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ea0018820 con 0x7f3ea4104a60 2024-09-21T03:51:09.273 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:09.254+0000 7f3eabb8a640 1 -- 172.21.15.39:0/2593477323 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3ea41a8f10 con 0x7f3ea4104a60 2024-09-21T03:51:09.273 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:09.256+0000 7f3eb20fe740 1 Processor -- start 2024-09-21T03:51:09.273 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:09.257+0000 7f3eb20fe740 1 -- start start 2024-09-21T03:51:09.273 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:09.257+0000 7f3e9affd640 1 -- 172.21.15.39:0/2593477323 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 27) ==== 154643+0+0 (secure 0 0 0) 0x7f3ea000f450 con 0x7f3ea4104a60 2024-09-21T03:51:09.273 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:09.257+0000 7f3eb20fe740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555f2ed4e070 0x555f2f0f52c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:51:09.274 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:09.257+0000 7f3eb20fe740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555f2ed251b0 0x555f2f0f5800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:51:09.274 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:09.257+0000 7f3eb20fe740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x555f2ed2d390 0x555f2f0f5d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:51:09.274 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:09.257+0000 7f3eb20fe740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x555f2f06bea0 con 0x555f2ed2d390 2024-09-21T03:51:09.274 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:09.257+0000 7f3eb20fe740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x555f2f05b2d0 con 0x555f2ed251b0 2024-09-21T03:51:09.274 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:09.257+0000 7f3eb20fe740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x555f2f05f0e0 con 0x555f2ed4e070 2024-09-21T03:51:09.274 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:09.257+0000 7f3eaa127640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555f2ed4e070 0x555f2f0f52c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:51:09.274 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:09.257+0000 7f3ea9125640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x555f2ed2d390 0x555f2f0f5d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:51:09.274 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:09.257+0000 7f3ea9125640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x555f2ed2d390 0x555f2f0f5d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:60390/0 (socket says 172.21.15.39:60390) 2024-09-21T03:51:09.274 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:09.257+0000 7f3eaa127640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555f2ed4e070 0x555f2f0f52c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.39:39038/0 (socket says 172.21.15.39:39038) 2024-09-21T03:51:09.274 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:09.257+0000 7f3ea9125640 1 -- 172.21.15.39:0/3003412132 learned_addr learned my addr 172.21.15.39:0/3003412132 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:51:09.274 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:09.257+0000 7f3ea9926640 1 --2- 172.21.15.39:0/3003412132 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555f2ed251b0 0x555f2f0f5800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:51:09.275 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:09.257+0000 7f3eaa127640 1 -- 172.21.15.39:0/3003412132 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555f2ed251b0 msgr2=0x555f2f0f5800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:51:09.275 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:09.257+0000 7f3eaa127640 1 --2- 172.21.15.39:0/3003412132 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555f2ed251b0 0x555f2f0f5800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:09.275 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:09.257+0000 7f3eaa127640 1 -- 172.21.15.39:0/3003412132 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x555f2ed2d390 msgr2=0x555f2f0f5d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:51:09.275 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:09.257+0000 7f3eaa127640 1 --2- 172.21.15.39:0/3003412132 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x555f2ed2d390 0x555f2f0f5d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:09.275 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:09.257+0000 7f3eaa127640 1 -- 172.21.15.39:0/3003412132 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555f2edb9650 con 0x555f2ed4e070 2024-09-21T03:51:09.275 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:09.257+0000 7f3ea9125640 1 --2- 172.21.15.39:0/3003412132 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x555f2ed2d390 0x555f2f0f5d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:51:09.275 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:09.258+0000 7f3eaa127640 1 --2- 172.21.15.39:0/3003412132 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555f2ed4e070 0x555f2f0f52c0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f3e9c006010 tx=0x7f3e9c02f4c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9aa6695fdc08ffed server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:51:09.275 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:09.258+0000 7f3e9affd640 1 --2- 172.21.15.39:0/2593477323 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f3e900b6680 0x7f3e900b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:51:09.275 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:09.258+0000 7f3e8ffff640 1 -- 172.21.15.39:0/3003412132 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e9c031df0 con 0x555f2ed4e070 2024-09-21T03:51:09.275 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:09.258+0000 7f3ea9926640 1 --2- 172.21.15.39:0/2593477323 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f3e900b6680 0x7f3e900b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:51:09.275 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:09.258+0000 7f3e8ffff640 1 -- 172.21.15.39:0/3003412132 <== mon.1 v2:172.21.15.161:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3e9c036030 con 0x555f2ed4e070 2024-09-21T03:51:09.275 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:09.258+0000 7f3e8ffff640 1 -- 172.21.15.39:0/3003412132 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e9c004280 con 0x555f2ed4e070 2024-09-21T03:51:09.276 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:09.258+0000 7f3e9affd640 1 -- 172.21.15.39:0/2593477323 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8484+0+0 (secure 0 0 0) 0x7f3ea00e77a0 con 0x7f3ea4104a60 2024-09-21T03:51:09.276 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:09.259+0000 7f3eb20fe740 1 -- 172.21.15.39:0/3003412132 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555f2ed4e070 msgr2=0x555f2f0f52c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:51:09.276 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:09.259+0000 7f3ea9926640 1 --2- 172.21.15.39:0/2593477323 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f3e900b6680 0x7f3e900b8b90 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f3ea00e4fc0 tx=0x7f3ea00ed550 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:51:09.276 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:09.259+0000 7f3eb20fe740 1 --2- 172.21.15.39:0/3003412132 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555f2ed4e070 0x555f2f0f52c0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f3e9c006010 tx=0x7f3e9c02f4c0 comp rx=0 tx=0).stop 2024-09-21T03:51:09.276 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:09.259+0000 7f3eb20fe740 1 -- 172.21.15.39:0/3003412132 shutdown_connections 2024-09-21T03:51:09.276 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:09.259+0000 7f3eb20fe740 1 --2- 172.21.15.39:0/3003412132 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x555f2ed2d390 0x555f2f0f5d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:09.276 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:09.259+0000 7f3eb20fe740 1 --2- 172.21.15.39:0/3003412132 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555f2ed251b0 0x555f2f0f5800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:09.276 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:09.259+0000 7f3eb20fe740 1 --2- 172.21.15.39:0/3003412132 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555f2ed4e070 0x555f2f0f52c0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:09.276 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:09.259+0000 7f3eb20fe740 1 -- 172.21.15.39:0/3003412132 >> 172.21.15.39:0/3003412132 conn(0x555f2ed42460 msgr2=0x555f2f0f31b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:51:09.276 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:09.259+0000 7f3eb20fe740 1 -- 172.21.15.39:0/3003412132 shutdown_connections 2024-09-21T03:51:09.276 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:09.259+0000 7f3eb20fe740 1 -- 172.21.15.39:0/3003412132 wait complete. 2024-09-21T03:51:09.277 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:09.260+0000 7f3eb20fe740 1 -- start start 2024-09-21T03:51:09.277 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:09.260+0000 7f3eb20fe740 1 Processor -- start 2024-09-21T03:51:09.277 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:09.260+0000 7f3eb20fe740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x555f2ed251b0 0x555f2f100910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:51:09.277 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:09.260+0000 7f3eb20fe740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555f2ed2d390 0x555f2f100e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:51:09.277 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:09.260+0000 7f3eb20fe740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555f2ed4e070 0x555f2f10cbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:51:09.277 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:09.260+0000 7f3eb20fe740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x555f2eebc700 con 0x555f2ed251b0 2024-09-21T03:51:09.277 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:09.260+0000 7f3eb20fe740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x555f2eff6160 con 0x555f2ed2d390 2024-09-21T03:51:09.277 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:09.260+0000 7f3eb20fe740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x555f2f1038c0 con 0x555f2ed4e070 2024-09-21T03:51:09.277 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:09.260+0000 7f3eb20fe740 10 client.0 _ll_register_callbacks cb 0x555f2f00f9e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:51:09.277 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:09.260+0000 7f3eaa127640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x555f2ed251b0 0x555f2f100910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:51:09.278 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:09.260+0000 7f3eaa127640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x555f2ed251b0 0x555f2f100910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:60394/0 (socket says 172.21.15.39:60394) 2024-09-21T03:51:09.278 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:09.260+0000 7f3eaa127640 1 -- 172.21.15.39:0/323625222 learned_addr learned my addr 172.21.15.39:0/323625222 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:51:09.278 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:09.261+0000 7f3ea9125640 1 --2- 172.21.15.39:0/323625222 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555f2ed2d390 0x555f2f100e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:51:09.278 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:09.261+0000 7f3eaa127640 1 --2- 172.21.15.39:0/323625222 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555f2ed4e070 0x555f2f10cbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:51:09.278 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:09.261+0000 7f3eaa127640 1 -- 172.21.15.39:0/323625222 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555f2ed4e070 msgr2=0x555f2f10cbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:51:09.278 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:09.261+0000 7f3eaa127640 1 --2- 172.21.15.39:0/323625222 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555f2ed4e070 0x555f2f10cbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:09.278 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:09.261+0000 7f3eaa127640 1 -- 172.21.15.39:0/323625222 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555f2ed2d390 msgr2=0x555f2f100e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:51:09.278 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:09.261+0000 7f3eaa127640 1 --2- 172.21.15.39:0/323625222 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555f2ed2d390 0x555f2f100e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:09.278 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:09.261+0000 7f3eaa127640 1 -- 172.21.15.39:0/323625222 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555f2f023c90 con 0x555f2ed251b0 2024-09-21T03:51:09.278 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:09.261+0000 7f3ea9125640 1 --2- 172.21.15.39:0/323625222 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555f2ed2d390 0x555f2f100e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:51:09.278 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:09.261+0000 7f3eaa127640 1 --2- 172.21.15.39:0/323625222 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x555f2ed251b0 0x555f2f100910 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7f3e9c031780 tx=0x7f3e9c0095c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c3f2bfe5dc614252 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:51:09.278 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:09.261+0000 7f3e8d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-09-21T03:51:09.279 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:09.261+0000 7f3e8d7fa640 1 -- 172.21.15.39:0/323625222 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e9c058060 con 0x555f2ed251b0 2024-09-21T03:51:09.279 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:09.261+0000 7f3e8d7fa640 1 -- 172.21.15.39:0/323625222 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8484+0+0 (secure 0 0 0) 0x7f3e9c02fa70 con 0x555f2ed251b0 2024-09-21T03:51:09.279 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:09.261+0000 7f3eb20fe740 1 -- 172.21.15.39:0/323625222 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3e9c036030 con 0x555f2ed251b0 2024-09-21T03:51:09.279 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:09.261+0000 7f3e8d7fa640 1 -- 172.21.15.39:0/323625222 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f3e9c02fc70 con 0x555f2ed251b0 2024-09-21T03:51:09.279 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:09.261+0000 7f3eb20fe740 10 client.16752 fetch_fsmap learned FSMap version 29 2024-09-21T03:51:09.279 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:09.261+0000 7f3eb20fe740 1 -- 172.21.15.39:0/323625222 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x555f2f10f570 con 0x555f2ed251b0 2024-09-21T03:51:09.279 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:09.262+0000 7f3e8d7fa640 1 -- 172.21.15.39:0/323625222 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3e9c036030 con 0x555f2ed251b0 2024-09-21T03:51:09.279 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:09.262+0000 7f3e8d7fa640 1 -- 172.21.15.39:0/323625222 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f3e9c043070 con 0x555f2ed251b0 2024-09-21T03:51:09.279 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:09.262+0000 7f3eb20fe740 10 client.16752 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T03:51:09.279 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:09.262+0000 7f3eb20fe740 10 client.16752 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.a 2024-09-21T03:51:09.280 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:09.262+0000 7f3eb20fe740 20 client.16752 populate_metadata read hostname 'smithi039' 2024-09-21T03:51:09.280 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:09.262+0000 7f3eb20fe740 1 --2- 172.21.15.39:0/323625222 >> [v2:172.21.15.39:6840/4247190456,v1:172.21.15.39:6841/4247190456] conn(0x555f2ed0c9f0 0x555f2f1128d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:51:09.280 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:09.262+0000 7f3eb20fe740 4 client.16752 mds_command: new command op to 14868 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:51:09.280 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:09.262+0000 7f3eb20fe740 1 -- 172.21.15.39:0/323625222 --> [v2:172.21.15.39:6840/4247190456,v1:172.21.15.39:6841/4247190456] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555f2f06ddf0 con 0x555f2ed0c9f0 2024-09-21T03:51:09.280 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:09.262+0000 7f3ea9125640 1 --2- 172.21.15.39:0/323625222 >> [v2:172.21.15.39:6840/4247190456,v1:172.21.15.39:6841/4247190456] conn(0x555f2ed0c9f0 0x555f2f1128d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:51:09.280 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:09.262+0000 7f3e8d7fa640 10 client.16752 ms_handle_connect on v2:172.21.15.39:6840/4247190456 2024-09-21T03:51:09.280 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:09.262+0000 7f3ea9125640 1 --2- 172.21.15.39:0/323625222 >> [v2:172.21.15.39:6840/4247190456,v1:172.21.15.39:6841/4247190456] conn(0x555f2ed0c9f0 0x555f2f1128d0 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=d26b9b18637bc6e9 server_cookie=ede23508848a741a in_seq=0 out_seq=0 2024-09-21T03:51:09.280 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:09.263+0000 7f3e8d7fa640 1 -- 172.21.15.39:0/323625222 <== mds.4 v2:172.21.15.39:6840/4247190456 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x555f2f06ddf0 con 0x555f2ed0c9f0 2024-09-21T03:51:09.280 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:09.263+0000 7f3e8d7fa640 10 client.16752 handle_command_reply: tid=0 2024-09-21T03:51:09.280 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:09.263+0000 7f3eb20fe740 1 client.16752 shutdown 2024-09-21T03:51:09.281 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:09.263+0000 7f3eb20fe740 1 -- 172.21.15.39:0/323625222 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x555f2ed251b0 msgr2=0x555f2f100910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:51:09.281 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:09.263+0000 7f3eb20fe740 1 --2- 172.21.15.39:0/323625222 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x555f2ed251b0 0x555f2f100910 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7f3e9c031780 tx=0x7f3e9c0095c0 comp rx=0 tx=0).stop 2024-09-21T03:51:09.281 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:09.263+0000 7f3eb20fe740 1 -- 172.21.15.39:0/323625222 shutdown_connections 2024-09-21T03:51:09.281 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:09.263+0000 7f3eb20fe740 1 --2- 172.21.15.39:0/323625222 >> [v2:172.21.15.39:6840/4247190456,v1:172.21.15.39:6841/4247190456] conn(0x555f2ed0c9f0 0x555f2f1128d0 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:09.281 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:09.263+0000 7f3eb20fe740 1 --2- 172.21.15.39:0/323625222 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555f2ed4e070 0x555f2f10cbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:09.281 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:09.263+0000 7f3eb20fe740 1 --2- 172.21.15.39:0/323625222 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555f2ed2d390 0x555f2f100e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:09.281 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:09.263+0000 7f3e8d7fa640 0 client.16752 ms_handle_reset on v2:172.21.15.39:6840/4247190456 2024-09-21T03:51:09.281 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:09.263+0000 7f3eb20fe740 1 --2- 172.21.15.39:0/323625222 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x555f2ed251b0 0x555f2f100910 unknown :-1 s=CLOSED pgs=827 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:09.281 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:09.263+0000 7f3eb20fe740 1 -- 172.21.15.39:0/323625222 >> 172.21.15.39:0/323625222 conn(0x555f2ed42460 msgr2=0x555f2f0f8450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:51:09.281 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:09.263+0000 7f3eb20fe740 1 -- 172.21.15.39:0/323625222 shutdown_connections 2024-09-21T03:51:09.282 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:09.263+0000 7f3eb20fe740 1 -- 172.21.15.39:0/323625222 wait complete. 2024-09-21T03:51:09.282 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:09.264+0000 7f3eb20fe740 20 client.16752 trim_cache size 0 max 16384 2024-09-21T03:51:09.287 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:09.277+0000 7f3eb20fe740 1 Processor -- start 2024-09-21T03:51:09.287 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:09.277+0000 7f3eb20fe740 1 -- start start 2024-09-21T03:51:09.287 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:09.277+0000 7f3eb20fe740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555f2ed251b0 0x555f2f0f6a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:51:09.287 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:09.277+0000 7f3eb20fe740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x555f2ed2d390 0x555f2f0f2760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:51:09.287 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:09.277+0000 7f3eb20fe740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555f2ed4e070 0x555f2f0f2ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:51:09.287 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:09.277+0000 7f3eb20fe740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f3ea41b16f0 con 0x555f2ed2d390 2024-09-21T03:51:09.287 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:09.277+0000 7f3eb20fe740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f3ea41b3230 con 0x555f2ed4e070 2024-09-21T03:51:09.288 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:09.277+0000 7f3eb20fe740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f3ea41b1dc0 con 0x555f2ed251b0 2024-09-21T03:51:09.288 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:09.277+0000 7f3ea9125640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x555f2ed2d390 0x555f2f0f2760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:51:09.288 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:09.277+0000 7f3ea9125640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x555f2ed2d390 0x555f2f0f2760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:60402/0 (socket says 172.21.15.39:60402) 2024-09-21T03:51:09.288 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:09.277+0000 7f3ea9125640 1 -- 172.21.15.39:0/2827978204 learned_addr learned my addr 172.21.15.39:0/2827978204 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:51:09.288 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:09.277+0000 7f3eaa127640 1 --2- 172.21.15.39:0/2827978204 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555f2ed251b0 0x555f2f0f6a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:51:09.288 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:09.278+0000 7f3ea9125640 1 -- 172.21.15.39:0/2827978204 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555f2ed251b0 msgr2=0x555f2f0f6a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:51:09.288 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:09.278+0000 7f3eaa127640 1 --2- 172.21.15.39:0/2827978204 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555f2ed4e070 0x555f2f0f2ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:51:09.288 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:09.278+0000 7f3ea9125640 1 --2- 172.21.15.39:0/2827978204 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555f2ed251b0 0x555f2f0f6a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:09.288 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:09.278+0000 7f3ea9125640 1 -- 172.21.15.39:0/2827978204 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555f2ed4e070 msgr2=0x555f2f0f2ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:51:09.288 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:09.278+0000 7f3ea9125640 1 --2- 172.21.15.39:0/2827978204 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555f2ed4e070 0x555f2f0f2ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:09.288 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:09.278+0000 7f3ea9125640 1 -- 172.21.15.39:0/2827978204 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555f2f0fee10 con 0x555f2ed2d390 2024-09-21T03:51:09.289 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:09.278+0000 7f3ea9125640 1 --2- 172.21.15.39:0/2827978204 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x555f2ed2d390 0x555f2f0f2760 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x555f2f112e10 tx=0x7f3e940108c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=12ddc90671fac462 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:51:09.289 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:09.278+0000 7f3e8f7fe640 1 -- 172.21.15.39:0/2827978204 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e94004220 con 0x555f2ed2d390 2024-09-21T03:51:09.289 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:09.278+0000 7f3e8f7fe640 1 -- 172.21.15.39:0/2827978204 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3e940043c0 con 0x555f2ed2d390 2024-09-21T03:51:09.289 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:09.278+0000 7f3e8f7fe640 1 -- 172.21.15.39:0/2827978204 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e9401ea10 con 0x555f2ed2d390 2024-09-21T03:51:09.289 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:09.279+0000 7f3eb20fe740 1 -- 172.21.15.39:0/2827978204 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x555f2ed2d390 msgr2=0x555f2f0f2760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:51:09.289 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:09.279+0000 7f3eb20fe740 1 --2- 172.21.15.39:0/2827978204 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x555f2ed2d390 0x555f2f0f2760 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x555f2f112e10 tx=0x7f3e940108c0 comp rx=0 tx=0).stop 2024-09-21T03:51:09.289 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:09.279+0000 7f3eb20fe740 1 -- 172.21.15.39:0/2827978204 shutdown_connections 2024-09-21T03:51:09.289 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:09.279+0000 7f3eb20fe740 1 --2- 172.21.15.39:0/2827978204 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555f2ed4e070 0x555f2f0f2ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:09.289 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:09.279+0000 7f3eb20fe740 1 --2- 172.21.15.39:0/2827978204 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x555f2ed2d390 0x555f2f0f2760 unknown :-1 s=CLOSED pgs=828 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:09.289 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:09.279+0000 7f3eb20fe740 1 --2- 172.21.15.39:0/2827978204 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555f2ed251b0 0x555f2f0f6a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:09.289 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:09.279+0000 7f3eb20fe740 1 -- 172.21.15.39:0/2827978204 >> 172.21.15.39:0/2827978204 conn(0x555f2ed42460 msgr2=0x555f2f0f62f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:51:09.289 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:09.279+0000 7f3eb20fe740 1 -- 172.21.15.39:0/2827978204 shutdown_connections 2024-09-21T03:51:09.289 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:09.279+0000 7f3eb20fe740 1 -- 172.21.15.39:0/2827978204 wait complete. 2024-09-21T03:51:09.290 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:09.279+0000 7f3eb20fe740 1 -- start start 2024-09-21T03:51:09.290 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:09.280+0000 7f3eb20fe740 1 Processor -- start 2024-09-21T03:51:09.290 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:09.280+0000 7f3eb20fe740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555f2ed251b0 0x555f2f109390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:51:09.290 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:09.280+0000 7f3eb20fe740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555f2ed2d390 0x555f2f104890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:51:09.290 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:09.280+0000 7f3eb20fe740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x555f2ed4e070 0x555f2f104e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:51:09.290 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:09.280+0000 7f3eb20fe740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f3ea41b2b60 con 0x555f2ed4e070 2024-09-21T03:51:09.290 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:09.280+0000 7f3eb20fe740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f3ea41b2490 con 0x555f2ed251b0 2024-09-21T03:51:09.290 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:09.280+0000 7f3eb20fe740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f3ea41af6d0 con 0x555f2ed2d390 2024-09-21T03:51:09.290 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:09.280+0000 7f3eb20fe740 10 client.0 _ll_register_callbacks cb 0x7f3e9400a590 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:51:09.290 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:09.280+0000 7f3eaa127640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x555f2ed4e070 0x555f2f104e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:51:09.290 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:09.280+0000 7f3eaa127640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x555f2ed4e070 0x555f2f104e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:60410/0 (socket says 172.21.15.39:60410) 2024-09-21T03:51:09.290 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:09.280+0000 7f3eaa127640 1 -- 172.21.15.39:0/3087862842 learned_addr learned my addr 172.21.15.39:0/3087862842 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:51:09.291 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:09.280+0000 7f3eaa127640 1 --2- 172.21.15.39:0/3087862842 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555f2ed251b0 0x555f2f109390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:51:09.291 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:09.280+0000 7f3ea9125640 1 --2- 172.21.15.39:0/3087862842 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555f2ed2d390 0x555f2f104890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:51:09.291 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:09.281+0000 7f3eaa127640 1 -- 172.21.15.39:0/3087862842 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555f2ed2d390 msgr2=0x555f2f104890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:51:09.291 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:09.281+0000 7f3eaa127640 1 --2- 172.21.15.39:0/3087862842 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555f2ed2d390 0x555f2f104890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:09.291 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:09.281+0000 7f3eaa127640 1 -- 172.21.15.39:0/3087862842 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555f2ed251b0 msgr2=0x555f2f109390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:51:09.291 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:09.281+0000 7f3eaa127640 1 --2- 172.21.15.39:0/3087862842 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555f2ed251b0 0x555f2f109390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:09.291 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:09.281+0000 7f3eaa127640 1 -- 172.21.15.39:0/3087862842 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555f2f1055f0 con 0x555f2ed4e070 2024-09-21T03:51:09.291 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:09.281+0000 7f3eaa127640 1 --2- 172.21.15.39:0/3087862842 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x555f2ed4e070 0x555f2f104e90 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7f3e9c009420 tx=0x7f3e9c03e040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a5e1b400f25d42a2 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:51:09.291 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:09.281+0000 7f3e8d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-09-21T03:51:09.291 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:09.281+0000 7f3e8d7fa640 1 -- 172.21.15.39:0/3087862842 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e9c030890 con 0x555f2ed4e070 2024-09-21T03:51:09.291 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:09.281+0000 7f3e8d7fa640 1 -- 172.21.15.39:0/3087862842 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8484+0+0 (secure 0 0 0) 0x7f3e9c02f870 con 0x555f2ed4e070 2024-09-21T03:51:09.292 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:09.281+0000 7f3eb20fe740 1 -- 172.21.15.39:0/3087862842 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3e940043c0 con 0x555f2ed4e070 2024-09-21T03:51:09.292 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:09.281+0000 7f3e8d7fa640 1 -- 172.21.15.39:0/3087862842 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f3e9c02fa70 con 0x555f2ed4e070 2024-09-21T03:51:09.292 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:09.281+0000 7f3eb20fe740 10 client.16758 fetch_fsmap learned FSMap version 29 2024-09-21T03:51:09.292 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:09.281+0000 7f3eb20fe740 1 -- 172.21.15.39:0/3087862842 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x555f2f17d040 con 0x555f2ed4e070 2024-09-21T03:51:09.292 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:09.281+0000 7f3e8d7fa640 1 -- 172.21.15.39:0/3087862842 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3e940043c0 con 0x555f2ed4e070 2024-09-21T03:51:09.292 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:09.282+0000 7f3e8d7fa640 1 -- 172.21.15.39:0/3087862842 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f3e9c06f020 con 0x555f2ed4e070 2024-09-21T03:51:09.292 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:09.282+0000 7f3eb20fe740 10 client.16758 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T03:51:09.292 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:09.282+0000 7f3eb20fe740 10 client.16758 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.a 2024-09-21T03:51:09.292 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:09.282+0000 7f3eb20fe740 20 client.16758 populate_metadata read hostname 'smithi039' 2024-09-21T03:51:09.292 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:09.282+0000 7f3eb20fe740 1 --2- 172.21.15.39:0/3087862842 >> [v2:172.21.15.39:6840/4247190456,v1:172.21.15.39:6841/4247190456] conn(0x555f2ece1da0 0x555f2f17f690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:51:09.292 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:09.282+0000 7f3eb20fe740 4 client.16758 mds_command: new command op to 14868 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T03:51:09.292 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:09.282+0000 7f3eb20fe740 1 -- 172.21.15.39:0/3087862842 --> [v2:172.21.15.39:6840/4247190456,v1:172.21.15.39:6841/4247190456] -- command(tid 0: {"prefix": "damage ls"}) -- 0x555f2f0698a0 con 0x555f2ece1da0 2024-09-21T03:51:09.293 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:09.282+0000 7f3ea9125640 1 --2- 172.21.15.39:0/3087862842 >> [v2:172.21.15.39:6840/4247190456,v1:172.21.15.39:6841/4247190456] conn(0x555f2ece1da0 0x555f2f17f690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:51:09.293 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:09.282+0000 7f3ea9125640 1 --2- 172.21.15.39:0/3087862842 >> [v2:172.21.15.39:6840/4247190456,v1:172.21.15.39:6841/4247190456] conn(0x555f2ece1da0 0x555f2f17f690 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=23528442ef88ea40 server_cookie=125634cf69f1c70 in_seq=0 out_seq=0 2024-09-21T03:51:09.293 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:09.282+0000 7f3e8d7fa640 10 client.16758 ms_handle_connect on v2:172.21.15.39:6840/4247190456 2024-09-21T03:51:09.293 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:09.282+0000 7f3e8d7fa640 1 -- 172.21.15.39:0/3087862842 <== mds.4 v2:172.21.15.39:6840/4247190456 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x555f2f0698a0 con 0x555f2ece1da0 2024-09-21T03:51:09.293 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:09.282+0000 7f3e8d7fa640 10 client.16758 handle_command_reply: tid=0 2024-09-21T03:51:09.293 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:09.283+0000 7f3eb20fe740 1 client.16758 shutdown 2024-09-21T03:51:09.293 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:09.283+0000 7f3eb20fe740 1 -- 172.21.15.39:0/3087862842 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x555f2ed4e070 msgr2=0x555f2f104e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:51:09.293 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:09.283+0000 7f3eb20fe740 1 --2- 172.21.15.39:0/3087862842 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x555f2ed4e070 0x555f2f104e90 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7f3e9c009420 tx=0x7f3e9c03e040 comp rx=0 tx=0).stop 2024-09-21T03:51:09.293 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:09.283+0000 7f3eb20fe740 1 -- 172.21.15.39:0/3087862842 shutdown_connections 2024-09-21T03:51:09.293 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:09.283+0000 7f3eb20fe740 1 --2- 172.21.15.39:0/3087862842 >> [v2:172.21.15.39:6840/4247190456,v1:172.21.15.39:6841/4247190456] conn(0x555f2ece1da0 0x555f2f17f690 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:09.293 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:09.283+0000 7f3eb20fe740 1 --2- 172.21.15.39:0/3087862842 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x555f2ed4e070 0x555f2f104e90 unknown :-1 s=CLOSED pgs=829 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:09.293 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:09.283+0000 7f3e8d7fa640 0 client.16758 ms_handle_reset on v2:172.21.15.39:6840/4247190456 2024-09-21T03:51:09.293 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:09.283+0000 7f3eb20fe740 1 --2- 172.21.15.39:0/3087862842 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555f2ed2d390 0x555f2f104890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:09.294 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:09.283+0000 7f3eb20fe740 1 --2- 172.21.15.39:0/3087862842 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555f2ed251b0 0x555f2f109390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:09.294 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:09.283+0000 7f3eb20fe740 1 -- 172.21.15.39:0/3087862842 >> 172.21.15.39:0/3087862842 conn(0x555f2ed42460 msgr2=0x555f2f0f6cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:51:09.294 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:09.283+0000 7f3eb20fe740 1 -- 172.21.15.39:0/3087862842 shutdown_connections 2024-09-21T03:51:09.294 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:09.283+0000 7f3eb20fe740 1 -- 172.21.15.39:0/3087862842 wait complete. 2024-09-21T03:51:09.294 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:09.283+0000 7f3eb20fe740 20 client.16758 trim_cache size 0 max 16384 2024-09-21T03:51:09.294 INFO:teuthology.orchestra.run.smithi039.stdout:[] 2024-09-21T03:51:09.295 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:09.284+0000 7f3eabb8a640 1 -- 172.21.15.39:0/2593477323 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f3e900b6680 msgr2=0x7f3e900b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:51:09.295 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:09.284+0000 7f3eabb8a640 1 --2- 172.21.15.39:0/2593477323 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f3e900b6680 0x7f3e900b8b90 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f3ea00e4fc0 tx=0x7f3ea00ed550 comp rx=0 tx=0).stop 2024-09-21T03:51:09.295 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:09.284+0000 7f3eabb8a640 1 -- 172.21.15.39:0/2593477323 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3ea4104a60 msgr2=0x7f3ea41027a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:51:09.295 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:09.284+0000 7f3eabb8a640 1 --2- 172.21.15.39:0/2593477323 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3ea4104a60 0x7f3ea41027a0 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7f3ea000c920 tx=0x7f3ea000cde0 comp rx=0 tx=0).stop 2024-09-21T03:51:09.295 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:09.284+0000 7f3eabb8a640 1 -- 172.21.15.39:0/2593477323 shutdown_connections 2024-09-21T03:51:09.295 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:09.284+0000 7f3eabb8a640 1 --2- 172.21.15.39:0/2593477323 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f3e900b6680 0x7f3e900b8b90 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:09.296 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:09.284+0000 7f3eabb8a640 1 --2- 172.21.15.39:0/2593477323 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3ea410dec0 0x7f3ea4103220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:09.296 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:09.284+0000 7f3eabb8a640 1 --2- 172.21.15.39:0/2593477323 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3ea41053b0 0x7f3ea4102ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:09.296 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:09.284+0000 7f3eabb8a640 1 --2- 172.21.15.39:0/2593477323 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3ea4104a60 0x7f3ea41027a0 unknown :-1 s=CLOSED pgs=826 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:09.296 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:09.284+0000 7f3eabb8a640 1 -- 172.21.15.39:0/2593477323 >> 172.21.15.39:0/2593477323 conn(0x7f3ea40796d0 msgr2=0x7f3ea40ff570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:51:09.296 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:09.284+0000 7f3eabb8a640 1 -- 172.21.15.39:0/2593477323 shutdown_connections 2024-09-21T03:51:09.296 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:09.284+0000 7f3eabb8a640 1 -- 172.21.15.39:0/2593477323 wait complete. 2024-09-21T03:51:09.468 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:51:09 smithi161 ceph-mon[34533]: pgmap v478: 97 pgs: 97 active+clean; 10 MiB data, 520 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-09-21T03:51:09.475 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:51:09 smithi150 ceph-mon[34289]: pgmap v478: 97 pgs: 97 active+clean; 10 MiB data, 520 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-09-21T03:51:10.096 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 51116d5c-77ca-11ef-baf5-efdc52797490 -- ceph tell mds.1:3 damage ls 2024-09-21T03:51:10.354 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/51116d5c-77ca-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:51:10.882 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:10.879+0000 7f2e6efed640 1 -- 172.21.15.39:0/4129704616 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2e68108d10 msgr2=0x7f2e6810f540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:51:10.882 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:10.879+0000 7f2e6efed640 1 --2- 172.21.15.39:0/4129704616 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2e68108d10 0x7f2e6810f540 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7f2e60009ca0 tx=0x7f2e6002a8c0 comp rx=0 tx=0).stop 2024-09-21T03:51:10.883 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:10.879+0000 7f2e6efed640 1 -- 172.21.15.39:0/4129704616 shutdown_connections 2024-09-21T03:51:10.883 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:10.879+0000 7f2e6efed640 1 --2- 172.21.15.39:0/4129704616 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2e68108d10 0x7f2e6810f540 unknown :-1 s=CLOSED pgs=830 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:10.883 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:10.879+0000 7f2e6efed640 1 --2- 172.21.15.39:0/4129704616 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2e681083c0 0x7f2e681087d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:10.883 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:10.879+0000 7f2e6efed640 1 --2- 172.21.15.39:0/4129704616 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2e68102a80 0x7f2e68102e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:10.883 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:10.879+0000 7f2e6efed640 1 -- 172.21.15.39:0/4129704616 >> 172.21.15.39:0/4129704616 conn(0x7f2e68079740 msgr2=0x7f2e68079b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:51:10.883 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:10.879+0000 7f2e6efed640 1 -- 172.21.15.39:0/4129704616 shutdown_connections 2024-09-21T03:51:10.883 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:10.879+0000 7f2e6efed640 1 -- 172.21.15.39:0/4129704616 wait complete. 2024-09-21T03:51:10.885 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:10.880+0000 7f2e6efed640 1 Processor -- start 2024-09-21T03:51:10.885 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:10.880+0000 7f2e6efed640 1 -- start start 2024-09-21T03:51:10.885 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:10.880+0000 7f2e6efed640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2e68102a80 0x7f2e6810ffd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:51:10.885 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:10.880+0000 7f2e6efed640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2e681083c0 0x7f2e68110510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:51:10.886 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:10.880+0000 7f2e6efed640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2e68108d10 0x7f2e68110a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:51:10.886 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:10.880+0000 7f2e6efed640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f2e68113aa0 con 0x7f2e68108d10 2024-09-21T03:51:10.886 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:10.880+0000 7f2e6efed640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f2e68113920 con 0x7f2e681083c0 2024-09-21T03:51:10.886 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:10.880+0000 7f2e6efed640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f2e68113c20 con 0x7f2e68102a80 2024-09-21T03:51:10.886 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:10.881+0000 7f2e6d58a640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2e68108d10 0x7f2e68110a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:51:10.886 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:10.881+0000 7f2e6d58a640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2e68108d10 0x7f2e68110a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:60426/0 (socket says 172.21.15.39:60426) 2024-09-21T03:51:10.886 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:10.881+0000 7f2e6d58a640 1 -- 172.21.15.39:0/4028104036 learned_addr learned my addr 172.21.15.39:0/4028104036 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:51:10.886 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:10.881+0000 7f2e6cd89640 1 --2- 172.21.15.39:0/4028104036 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2e68102a80 0x7f2e6810ffd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:51:10.886 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:10.881+0000 7f2e67fff640 1 --2- 172.21.15.39:0/4028104036 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2e681083c0 0x7f2e68110510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:51:10.886 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:10.881+0000 7f2e6d58a640 1 -- 172.21.15.39:0/4028104036 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2e68102a80 msgr2=0x7f2e6810ffd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:51:10.886 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:10.881+0000 7f2e6d58a640 1 --2- 172.21.15.39:0/4028104036 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2e68102a80 0x7f2e6810ffd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:10.887 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:10.881+0000 7f2e6d58a640 1 -- 172.21.15.39:0/4028104036 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2e681083c0 msgr2=0x7f2e68110510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:51:10.887 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:10.881+0000 7f2e6d58a640 1 --2- 172.21.15.39:0/4028104036 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2e681083c0 0x7f2e68110510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:10.887 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:10.881+0000 7f2e6d58a640 1 -- 172.21.15.39:0/4028104036 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2e681a87d0 con 0x7f2e68108d10 2024-09-21T03:51:10.887 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:10.881+0000 7f2e67fff640 1 --2- 172.21.15.39:0/4028104036 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2e681083c0 0x7f2e68110510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:51:10.887 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:10.881+0000 7f2e6d58a640 1 --2- 172.21.15.39:0/4028104036 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2e68108d10 0x7f2e68110a50 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7f2e600098d0 tx=0x7f2e600317b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:51:10.887 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:10.881+0000 7f2e65ffb640 1 -- 172.21.15.39:0/4028104036 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e60003b70 con 0x7f2e68108d10 2024-09-21T03:51:10.887 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:10.881+0000 7f2e6efed640 1 -- 172.21.15.39:0/4028104036 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2e681a8a60 con 0x7f2e68108d10 2024-09-21T03:51:10.887 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:10.881+0000 7f2e65ffb640 1 -- 172.21.15.39:0/4028104036 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2e60003d10 con 0x7f2e68108d10 2024-09-21T03:51:10.887 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:10.881+0000 7f2e65ffb640 1 -- 172.21.15.39:0/4028104036 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e6002fa20 con 0x7f2e68108d10 2024-09-21T03:51:10.887 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:10.882+0000 7f2e6efed640 1 -- 172.21.15.39:0/4028104036 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2e681a8ff0 con 0x7f2e68108d10 2024-09-21T03:51:10.889 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:10.884+0000 7f2e75561740 1 Processor -- start 2024-09-21T03:51:10.889 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:10.885+0000 7f2e65ffb640 1 -- 172.21.15.39:0/4028104036 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 27) ==== 154643+0+0 (secure 0 0 0) 0x7f2e6002fc40 con 0x7f2e68108d10 2024-09-21T03:51:10.889 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:10.885+0000 7f2e75561740 1 -- start start 2024-09-21T03:51:10.890 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:10.885+0000 7f2e75561740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ff10445070 0x55ff107ec6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:51:10.890 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:10.885+0000 7f2e75561740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ff1041c1b0 0x55ff107ecbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:51:10.890 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:10.885+0000 7f2e75561740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ff10424390 0x55ff107ed120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:51:10.890 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:10.885+0000 7f2e75561740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x55ff10762ea0 con 0x55ff1041c1b0 2024-09-21T03:51:10.890 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:10.885+0000 7f2e75561740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x55ff107522d0 con 0x55ff10445070 2024-09-21T03:51:10.890 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:10.885+0000 7f2e75561740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55ff107560e0 con 0x55ff10424390 2024-09-21T03:51:10.890 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:10.885+0000 7f2e6cd89640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ff1041c1b0 0x55ff107ecbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:51:10.890 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:10.885+0000 7f2e67fff640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ff10445070 0x55ff107ec6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:51:10.890 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:10.885+0000 7f2e6cd89640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ff1041c1b0 0x55ff107ecbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:60434/0 (socket says 172.21.15.39:60434) 2024-09-21T03:51:10.891 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:10.885+0000 7f2e67fff640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ff10445070 0x55ff107ec6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.39:60006/0 (socket says 172.21.15.39:60006) 2024-09-21T03:51:10.891 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:10.886+0000 7f2e6cd89640 1 -- 172.21.15.39:0/715271616 learned_addr learned my addr 172.21.15.39:0/715271616 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:51:10.891 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:10.886+0000 7f2e67fff640 1 --2- 172.21.15.39:0/715271616 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ff10424390 0x55ff107ed120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:51:10.891 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:10.886+0000 7f2e6cd89640 1 -- 172.21.15.39:0/715271616 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ff10424390 msgr2=0x55ff107ed120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:51:10.891 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:10.886+0000 7f2e6cd89640 1 --2- 172.21.15.39:0/715271616 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ff10424390 0x55ff107ed120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:10.891 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:10.886+0000 7f2e6cd89640 1 -- 172.21.15.39:0/715271616 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ff10445070 msgr2=0x55ff107ec6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:51:10.891 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:10.886+0000 7f2e6cd89640 1 --2- 172.21.15.39:0/715271616 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ff10445070 0x55ff107ec6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:10.891 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:10.886+0000 7f2e6cd89640 1 -- 172.21.15.39:0/715271616 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ff104b0650 con 0x55ff1041c1b0 2024-09-21T03:51:10.891 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:10.886+0000 7f2e65ffb640 1 --2- 172.21.15.39:0/4028104036 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f2e580b6700 0x7f2e580b8c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:51:10.891 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:10.886+0000 7f2e6cd89640 1 --2- 172.21.15.39:0/715271616 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ff1041c1b0 0x55ff107ecbe0 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7f2e5c009c20 tx=0x7f2e5c0117b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b959ba2efe38a9e6 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:51:10.891 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:10.886+0000 7f2e67fff640 1 --2- 172.21.15.39:0/4028104036 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f2e580b6700 0x7f2e580b8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:51:10.892 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:10.886+0000 7f2e65ffb640 1 -- 172.21.15.39:0/4028104036 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8484+0+0 (secure 0 0 0) 0x7f2e60108a30 con 0x7f2e68108d10 2024-09-21T03:51:10.892 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:10.886+0000 7f2e52ffd640 1 -- 172.21.15.39:0/715271616 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e5c012450 con 0x55ff1041c1b0 2024-09-21T03:51:10.892 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:10.887+0000 7f2e52ffd640 1 -- 172.21.15.39:0/715271616 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2e5c012aa0 con 0x55ff1041c1b0 2024-09-21T03:51:10.892 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:10.887+0000 7f2e52ffd640 1 -- 172.21.15.39:0/715271616 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e5c00fa10 con 0x55ff1041c1b0 2024-09-21T03:51:10.895 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:10.887+0000 7f2e75561740 1 -- 172.21.15.39:0/715271616 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ff1041c1b0 msgr2=0x55ff107ecbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:51:10.895 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:10.887+0000 7f2e67fff640 1 --2- 172.21.15.39:0/4028104036 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f2e580b6700 0x7f2e580b8c10 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f2e54005e00 tx=0x7f2e5400b420 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:51:10.895 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:10.887+0000 7f2e75561740 1 --2- 172.21.15.39:0/715271616 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ff1041c1b0 0x55ff107ecbe0 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7f2e5c009c20 tx=0x7f2e5c0117b0 comp rx=0 tx=0).stop 2024-09-21T03:51:10.895 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:10.888+0000 7f2e75561740 1 -- 172.21.15.39:0/715271616 shutdown_connections 2024-09-21T03:51:10.896 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:10.888+0000 7f2e75561740 1 --2- 172.21.15.39:0/715271616 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ff10424390 0x55ff107ed120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:10.896 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:10.888+0000 7f2e75561740 1 --2- 172.21.15.39:0/715271616 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ff1041c1b0 0x55ff107ecbe0 unknown :-1 s=CLOSED pgs=832 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:10.896 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:10.888+0000 7f2e75561740 1 --2- 172.21.15.39:0/715271616 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ff10445070 0x55ff107ec6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:10.896 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:10.888+0000 7f2e75561740 1 -- 172.21.15.39:0/715271616 >> 172.21.15.39:0/715271616 conn(0x55ff10439460 msgr2=0x55ff107ea590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:51:10.896 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:10.888+0000 7f2e75561740 1 -- 172.21.15.39:0/715271616 shutdown_connections 2024-09-21T03:51:10.896 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:10.888+0000 7f2e75561740 1 -- 172.21.15.39:0/715271616 wait complete. 2024-09-21T03:51:10.896 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:10.888+0000 7f2e75561740 1 -- start start 2024-09-21T03:51:10.896 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:10.888+0000 7f2e75561740 1 Processor -- start 2024-09-21T03:51:10.896 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:10.889+0000 7f2e75561740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ff1041c1b0 0x55ff107f7d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:51:10.896 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:10.889+0000 7f2e75561740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ff10424390 0x55ff107f8260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:51:10.896 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:10.889+0000 7f2e75561740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ff10445070 0x55ff107f87c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:51:10.896 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:10.889+0000 7f2e75561740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x55ff105b3700 con 0x55ff10424390 2024-09-21T03:51:10.897 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:10.889+0000 7f2e75561740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x55ff106ed160 con 0x55ff1041c1b0 2024-09-21T03:51:10.897 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:10.889+0000 7f2e75561740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55ff107fae80 con 0x55ff10445070 2024-09-21T03:51:10.897 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:10.889+0000 7f2e75561740 10 client.0 _ll_register_callbacks cb 0x55ff107069e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:51:10.897 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:10.889+0000 7f2e67fff640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ff10424390 0x55ff107f8260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:51:10.897 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:10.889+0000 7f2e67fff640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ff10424390 0x55ff107f8260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:60448/0 (socket says 172.21.15.39:60448) 2024-09-21T03:51:10.897 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:10.889+0000 7f2e67fff640 1 -- 172.21.15.39:0/186005316 learned_addr learned my addr 172.21.15.39:0/186005316 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:51:10.897 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:10.889+0000 7f2e6d58a640 1 --2- 172.21.15.39:0/186005316 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ff10445070 0x55ff107f87c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:51:10.897 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:10.889+0000 7f2e6cd89640 1 --2- 172.21.15.39:0/186005316 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ff1041c1b0 0x55ff107f7d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:51:10.897 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:10.889+0000 7f2e67fff640 1 -- 172.21.15.39:0/186005316 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ff10445070 msgr2=0x55ff107f87c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:51:10.897 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:10.889+0000 7f2e67fff640 1 --2- 172.21.15.39:0/186005316 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ff10445070 0x55ff107f87c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:10.897 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:10.889+0000 7f2e67fff640 1 -- 172.21.15.39:0/186005316 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ff1041c1b0 msgr2=0x55ff107f7d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:51:10.897 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:10.889+0000 7f2e67fff640 1 --2- 172.21.15.39:0/186005316 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ff1041c1b0 0x55ff107f7d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:10.897 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:10.889+0000 7f2e67fff640 1 -- 172.21.15.39:0/186005316 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ff1071ac90 con 0x55ff10424390 2024-09-21T03:51:10.897 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:10.889+0000 7f2e67fff640 1 --2- 172.21.15.39:0/186005316 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ff10424390 0x55ff107f8260 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7f2e54004520 tx=0x7f2e54004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d07019e01da8810b server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:51:10.897 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:10.889+0000 7f2e47fff640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-09-21T03:51:10.898 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:10.890+0000 7f2e47fff640 1 -- 172.21.15.39:0/186005316 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e54012860 con 0x55ff10424390 2024-09-21T03:51:10.898 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:10.890+0000 7f2e47fff640 1 -- 172.21.15.39:0/186005316 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8484+0+0 (secure 0 0 0) 0x7f2e54012a00 con 0x55ff10424390 2024-09-21T03:51:10.898 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:10.890+0000 7f2e75561740 1 -- 172.21.15.39:0/186005316 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2e5c012aa0 con 0x55ff10424390 2024-09-21T03:51:10.898 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:10.890+0000 7f2e47fff640 1 -- 172.21.15.39:0/186005316 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f2e54012c00 con 0x55ff10424390 2024-09-21T03:51:10.898 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:10.890+0000 7f2e75561740 10 client.16770 fetch_fsmap learned FSMap version 29 2024-09-21T03:51:10.898 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:10.890+0000 7f2e75561740 1 -- 172.21.15.39:0/186005316 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55ff10806b30 con 0x55ff10424390 2024-09-21T03:51:10.898 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:10.890+0000 7f2e47fff640 1 -- 172.21.15.39:0/186005316 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2e5c012aa0 con 0x55ff10424390 2024-09-21T03:51:10.898 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:10.890+0000 7f2e47fff640 1 -- 172.21.15.39:0/186005316 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f2e54030020 con 0x55ff10424390 2024-09-21T03:51:10.898 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:10.890+0000 7f2e75561740 10 client.16770 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T03:51:10.898 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:10.890+0000 7f2e75561740 10 client.16770 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-21T03:51:10.898 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:10.890+0000 7f2e75561740 20 client.16770 populate_metadata read hostname 'smithi039' 2024-09-21T03:51:10.898 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:10.890+0000 7f2e75561740 1 --2- 172.21.15.39:0/186005316 >> [v2:172.21.15.161:6836/3451544597,v1:172.21.15.161:6837/3451544597] conn(0x55ff104039f0 0x55ff10809fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:51:10.898 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:10.890+0000 7f2e75561740 4 client.16770 mds_command: new command op to 24646 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:51:10.898 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:10.890+0000 7f2e75561740 1 -- 172.21.15.39:0/186005316 --> [v2:172.21.15.161:6836/3451544597,v1:172.21.15.161:6837/3451544597] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ff10764df0 con 0x55ff104039f0 2024-09-21T03:51:10.899 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:10.891+0000 7f2e6cd89640 1 --2- 172.21.15.39:0/186005316 >> [v2:172.21.15.161:6836/3451544597,v1:172.21.15.161:6837/3451544597] conn(0x55ff104039f0 0x55ff10809fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:51:10.899 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:10.891+0000 7f2e6cd89640 1 --2- 172.21.15.39:0/186005316 >> [v2:172.21.15.161:6836/3451544597,v1:172.21.15.161:6837/3451544597] conn(0x55ff104039f0 0x55ff10809fe0 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=2cab5d741a5c10a6 server_cookie=d69d5ed89109a0b6 in_seq=0 out_seq=0 2024-09-21T03:51:10.899 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:10.891+0000 7f2e47fff640 10 client.16770 ms_handle_connect on v2:172.21.15.161:6836/3451544597 2024-09-21T03:51:10.899 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:10.892+0000 7f2e47fff640 1 -- 172.21.15.39:0/186005316 <== mds.3 v2:172.21.15.161:6836/3451544597 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55ff10764df0 con 0x55ff104039f0 2024-09-21T03:51:10.899 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:10.892+0000 7f2e47fff640 10 client.16770 handle_command_reply: tid=0 2024-09-21T03:51:10.899 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:10.892+0000 7f2e75561740 1 client.16770 shutdown 2024-09-21T03:51:10.899 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:10.892+0000 7f2e75561740 1 -- 172.21.15.39:0/186005316 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ff10424390 msgr2=0x55ff107f8260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:51:10.899 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:10.892+0000 7f2e75561740 1 --2- 172.21.15.39:0/186005316 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ff10424390 0x55ff107f8260 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7f2e54004520 tx=0x7f2e54004730 comp rx=0 tx=0).stop 2024-09-21T03:51:10.899 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:10.892+0000 7f2e75561740 1 -- 172.21.15.39:0/186005316 shutdown_connections 2024-09-21T03:51:10.899 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:10.892+0000 7f2e75561740 1 --2- 172.21.15.39:0/186005316 >> [v2:172.21.15.161:6836/3451544597,v1:172.21.15.161:6837/3451544597] conn(0x55ff104039f0 0x55ff10809fe0 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:10.899 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:10.892+0000 7f2e75561740 1 --2- 172.21.15.39:0/186005316 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ff10445070 0x55ff107f87c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:10.899 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:10.892+0000 7f2e75561740 1 --2- 172.21.15.39:0/186005316 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ff10424390 0x55ff107f8260 unknown :-1 s=CLOSED pgs=833 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:10.899 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:10.892+0000 7f2e75561740 1 --2- 172.21.15.39:0/186005316 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ff1041c1b0 0x55ff107f7d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:10.900 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:10.892+0000 7f2e75561740 1 -- 172.21.15.39:0/186005316 >> 172.21.15.39:0/186005316 conn(0x55ff10439460 msgr2=0x55ff107ef830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:51:10.900 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:10.892+0000 7f2e47fff640 0 client.16770 ms_handle_reset on v2:172.21.15.161:6836/3451544597 2024-09-21T03:51:10.900 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:10.893+0000 7f2e75561740 1 -- 172.21.15.39:0/186005316 shutdown_connections 2024-09-21T03:51:10.900 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:10.893+0000 7f2e75561740 1 -- 172.21.15.39:0/186005316 wait complete. 2024-09-21T03:51:10.900 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:10.893+0000 7f2e75561740 20 client.16770 trim_cache size 0 max 16384 2024-09-21T03:51:10.911 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:10.906+0000 7f2e75561740 1 Processor -- start 2024-09-21T03:51:10.911 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:10.906+0000 7f2e75561740 1 -- start start 2024-09-21T03:51:10.911 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:10.906+0000 7f2e75561740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ff1041c1b0 0x55ff107f81e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:51:10.911 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:10.906+0000 7f2e75561740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ff10424390 0x55ff107e9bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:51:10.911 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:10.906+0000 7f2e75561740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ff10445070 0x55ff107ea100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:51:10.911 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:10.906+0000 7f2e75561740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f2e681b1930 con 0x55ff10424390 2024-09-21T03:51:10.911 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:10.906+0000 7f2e75561740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f2e681b3470 con 0x55ff1041c1b0 2024-09-21T03:51:10.912 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:10.906+0000 7f2e75561740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f2e681b2000 con 0x55ff10445070 2024-09-21T03:51:10.912 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:10.906+0000 7f2e67fff640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ff10424390 0x55ff107e9bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:51:10.912 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:10.906+0000 7f2e67fff640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ff10424390 0x55ff107e9bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:60464/0 (socket says 172.21.15.39:60464) 2024-09-21T03:51:10.912 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:10.906+0000 7f2e67fff640 1 -- 172.21.15.39:0/2036120211 learned_addr learned my addr 172.21.15.39:0/2036120211 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:51:10.912 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:10.907+0000 7f2e6d58a640 1 --2- 172.21.15.39:0/2036120211 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ff10445070 0x55ff107ea100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:51:10.912 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:10.907+0000 7f2e6cd89640 1 --2- 172.21.15.39:0/2036120211 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ff1041c1b0 0x55ff107f81e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:51:10.912 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:10.907+0000 7f2e67fff640 1 -- 172.21.15.39:0/2036120211 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ff10445070 msgr2=0x55ff107ea100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:51:10.912 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:10.907+0000 7f2e67fff640 1 --2- 172.21.15.39:0/2036120211 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ff10445070 0x55ff107ea100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:10.912 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:10.907+0000 7f2e67fff640 1 -- 172.21.15.39:0/2036120211 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ff1041c1b0 msgr2=0x55ff107f81e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:51:10.912 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:10.907+0000 7f2e67fff640 1 --2- 172.21.15.39:0/2036120211 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ff1041c1b0 0x55ff107f81e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:10.912 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:10.907+0000 7f2e67fff640 1 -- 172.21.15.39:0/2036120211 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ff108040a0 con 0x55ff10424390 2024-09-21T03:51:10.912 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:10.907+0000 7f2e6d58a640 1 --2- 172.21.15.39:0/2036120211 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ff10445070 0x55ff107ea100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:51:10.912 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:10.907+0000 7f2e6cd89640 1 --2- 172.21.15.39:0/2036120211 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ff1041c1b0 0x55ff107f81e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:51:10.913 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:10.907+0000 7f2e67fff640 1 --2- 172.21.15.39:0/2036120211 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ff10424390 0x55ff107e9bc0 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7f2e54003ec0 tx=0x7f2e54005bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fea6ac8fe67f68f8 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:51:10.913 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:10.907+0000 7f2e527fc640 1 -- 172.21.15.39:0/2036120211 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e54014ea0 con 0x55ff10424390 2024-09-21T03:51:10.913 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:10.907+0000 7f2e527fc640 1 -- 172.21.15.39:0/2036120211 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2e5400f9a0 con 0x55ff10424390 2024-09-21T03:51:10.913 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:10.907+0000 7f2e527fc640 1 -- 172.21.15.39:0/2036120211 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e54032820 con 0x55ff10424390 2024-09-21T03:51:10.913 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:10.908+0000 7f2e75561740 1 -- 172.21.15.39:0/2036120211 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ff10424390 msgr2=0x55ff107e9bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:51:10.913 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:10.908+0000 7f2e75561740 1 --2- 172.21.15.39:0/2036120211 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ff10424390 0x55ff107e9bc0 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7f2e54003ec0 tx=0x7f2e54005bf0 comp rx=0 tx=0).stop 2024-09-21T03:51:10.913 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:10.908+0000 7f2e75561740 1 -- 172.21.15.39:0/2036120211 shutdown_connections 2024-09-21T03:51:10.913 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:10.908+0000 7f2e75561740 1 --2- 172.21.15.39:0/2036120211 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ff10445070 0x55ff107ea100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:10.913 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:10.908+0000 7f2e75561740 1 --2- 172.21.15.39:0/2036120211 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ff10424390 0x55ff107e9bc0 unknown :-1 s=CLOSED pgs=834 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:10.913 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:10.908+0000 7f2e75561740 1 --2- 172.21.15.39:0/2036120211 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ff1041c1b0 0x55ff107f81e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:10.914 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:10.908+0000 7f2e75561740 1 -- 172.21.15.39:0/2036120211 >> 172.21.15.39:0/2036120211 conn(0x55ff10439460 msgr2=0x55ff107f7ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:51:10.914 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:10.908+0000 7f2e75561740 1 -- 172.21.15.39:0/2036120211 shutdown_connections 2024-09-21T03:51:10.914 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:10.908+0000 7f2e75561740 1 -- 172.21.15.39:0/2036120211 wait complete. 2024-09-21T03:51:10.914 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:10.908+0000 7f2e75561740 1 -- start start 2024-09-21T03:51:10.914 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:10.909+0000 7f2e75561740 1 Processor -- start 2024-09-21T03:51:10.917 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:10.909+0000 7f2e75561740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ff1041c1b0 0x55ff107fc210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:51:10.917 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:10.909+0000 7f2e75561740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ff10424390 0x55ff107fc750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:51:10.917 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:10.909+0000 7f2e75561740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ff10445070 0x55ff107fcc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:51:10.917 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:10.909+0000 7f2e75561740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f2e681b2da0 con 0x55ff1041c1b0 2024-09-21T03:51:10.917 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:10.909+0000 7f2e75561740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f2e681b26d0 con 0x55ff10424390 2024-09-21T03:51:10.917 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:10.909+0000 7f2e75561740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f2e681af910 con 0x55ff10445070 2024-09-21T03:51:10.917 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:10.909+0000 7f2e75561740 10 client.0 _ll_register_callbacks cb 0x7f2e54003f70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:51:10.917 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:10.909+0000 7f2e6cd89640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ff1041c1b0 0x55ff107fc210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:51:10.918 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:10.909+0000 7f2e6cd89640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ff1041c1b0 0x55ff107fc210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:60470/0 (socket says 172.21.15.39:60470) 2024-09-21T03:51:10.918 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:10.909+0000 7f2e6cd89640 1 -- 172.21.15.39:0/1179534161 learned_addr learned my addr 172.21.15.39:0/1179534161 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:51:10.918 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:10.909+0000 7f2e67fff640 1 --2- 172.21.15.39:0/1179534161 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ff10424390 0x55ff107fc750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:51:10.918 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:10.909+0000 7f2e6d58a640 1 --2- 172.21.15.39:0/1179534161 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ff10445070 0x55ff107fcc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:51:10.918 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:10.910+0000 7f2e6cd89640 1 -- 172.21.15.39:0/1179534161 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ff10445070 msgr2=0x55ff107fcc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:51:10.918 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:10.910+0000 7f2e6cd89640 1 --2- 172.21.15.39:0/1179534161 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ff10445070 0x55ff107fcc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:10.918 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:10.910+0000 7f2e6cd89640 1 -- 172.21.15.39:0/1179534161 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ff10424390 msgr2=0x55ff107fc750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:51:10.918 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:10.910+0000 7f2e6cd89640 1 --2- 172.21.15.39:0/1179534161 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ff10424390 0x55ff107fc750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:10.918 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:10.910+0000 7f2e6cd89640 1 -- 172.21.15.39:0/1179534161 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ff10873b10 con 0x55ff1041c1b0 2024-09-21T03:51:10.918 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:10.910+0000 7f2e6cd89640 1 --2- 172.21.15.39:0/1179534161 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ff1041c1b0 0x55ff107fc210 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7f2e5c00f5c0 tx=0x7f2e5c009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e4be7ef99a242b3 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:51:10.918 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:10.910+0000 7f2e47fff640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-09-21T03:51:10.918 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:10.910+0000 7f2e47fff640 1 -- 172.21.15.39:0/1179534161 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e5c005240 con 0x55ff1041c1b0 2024-09-21T03:51:10.918 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:10.910+0000 7f2e47fff640 1 -- 172.21.15.39:0/1179534161 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8484+0+0 (secure 0 0 0) 0x7f2e5c0053e0 con 0x55ff1041c1b0 2024-09-21T03:51:10.918 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:10.910+0000 7f2e75561740 1 -- 172.21.15.39:0/1179534161 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2e5400f9a0 con 0x55ff1041c1b0 2024-09-21T03:51:10.919 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:10.910+0000 7f2e47fff640 1 -- 172.21.15.39:0/1179534161 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f2e5c0055e0 con 0x55ff1041c1b0 2024-09-21T03:51:10.919 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:10.910+0000 7f2e75561740 10 client.16776 fetch_fsmap learned FSMap version 29 2024-09-21T03:51:10.919 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:10.910+0000 7f2e75561740 1 -- 172.21.15.39:0/1179534161 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55ff10874d00 con 0x55ff1041c1b0 2024-09-21T03:51:10.919 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:10.910+0000 7f2e47fff640 1 -- 172.21.15.39:0/1179534161 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2e5400f9a0 con 0x55ff1041c1b0 2024-09-21T03:51:10.919 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:10.911+0000 7f2e47fff640 1 -- 172.21.15.39:0/1179534161 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f2e5c036070 con 0x55ff1041c1b0 2024-09-21T03:51:10.919 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:10.911+0000 7f2e75561740 10 client.16776 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T03:51:10.919 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:10.911+0000 7f2e75561740 10 client.16776 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-21T03:51:10.919 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:10.911+0000 7f2e75561740 20 client.16776 populate_metadata read hostname 'smithi039' 2024-09-21T03:51:10.919 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:10.911+0000 7f2e75561740 1 --2- 172.21.15.39:0/1179534161 >> [v2:172.21.15.161:6836/3451544597,v1:172.21.15.161:6837/3451544597] conn(0x55ff103d8da0 0x55ff10877360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:51:10.919 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:10.911+0000 7f2e75561740 4 client.16776 mds_command: new command op to 24646 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T03:51:10.919 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:10.911+0000 7f2e75561740 1 -- 172.21.15.39:0/1179534161 --> [v2:172.21.15.161:6836/3451544597,v1:172.21.15.161:6837/3451544597] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55ff107608a0 con 0x55ff103d8da0 2024-09-21T03:51:10.919 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:10.911+0000 7f2e67fff640 1 --2- 172.21.15.39:0/1179534161 >> [v2:172.21.15.161:6836/3451544597,v1:172.21.15.161:6837/3451544597] conn(0x55ff103d8da0 0x55ff10877360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:51:10.920 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:10.911+0000 7f2e47fff640 10 client.16776 ms_handle_connect on v2:172.21.15.161:6836/3451544597 2024-09-21T03:51:10.920 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:10.911+0000 7f2e67fff640 1 --2- 172.21.15.39:0/1179534161 >> [v2:172.21.15.161:6836/3451544597,v1:172.21.15.161:6837/3451544597] conn(0x55ff103d8da0 0x55ff10877360 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=d2a807ed60a43bb9 server_cookie=e9a2b9aece11b9ba in_seq=0 out_seq=0 2024-09-21T03:51:10.920 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:10.912+0000 7f2e47fff640 1 -- 172.21.15.39:0/1179534161 <== mds.3 v2:172.21.15.161:6836/3451544597 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55ff107608a0 con 0x55ff103d8da0 2024-09-21T03:51:10.920 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:10.912+0000 7f2e47fff640 10 client.16776 handle_command_reply: tid=0 2024-09-21T03:51:10.920 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:10.912+0000 7f2e75561740 1 client.16776 shutdown 2024-09-21T03:51:10.920 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:10.912+0000 7f2e75561740 1 -- 172.21.15.39:0/1179534161 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ff1041c1b0 msgr2=0x55ff107fc210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:51:10.920 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:10.912+0000 7f2e75561740 1 --2- 172.21.15.39:0/1179534161 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ff1041c1b0 0x55ff107fc210 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7f2e5c00f5c0 tx=0x7f2e5c009f90 comp rx=0 tx=0).stop 2024-09-21T03:51:10.920 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:10.912+0000 7f2e75561740 1 -- 172.21.15.39:0/1179534161 shutdown_connections 2024-09-21T03:51:10.920 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:10.912+0000 7f2e75561740 1 --2- 172.21.15.39:0/1179534161 >> [v2:172.21.15.161:6836/3451544597,v1:172.21.15.161:6837/3451544597] conn(0x55ff103d8da0 0x55ff10877360 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:10.920 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:10.912+0000 7f2e75561740 1 --2- 172.21.15.39:0/1179534161 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ff10445070 0x55ff107fcc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:10.920 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:10.912+0000 7f2e75561740 1 --2- 172.21.15.39:0/1179534161 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ff10424390 0x55ff107fc750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:10.920 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:10.912+0000 7f2e75561740 1 --2- 172.21.15.39:0/1179534161 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ff1041c1b0 0x55ff107fc210 unknown :-1 s=CLOSED pgs=835 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:10.921 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:10.912+0000 7f2e75561740 1 -- 172.21.15.39:0/1179534161 >> 172.21.15.39:0/1179534161 conn(0x55ff10439460 msgr2=0x55ff10803090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:51:10.921 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:10.912+0000 7f2e47fff640 0 client.16776 ms_handle_reset on v2:172.21.15.161:6836/3451544597 2024-09-21T03:51:10.921 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:10.912+0000 7f2e75561740 1 -- 172.21.15.39:0/1179534161 shutdown_connections 2024-09-21T03:51:10.921 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:10.912+0000 7f2e75561740 1 -- 172.21.15.39:0/1179534161 wait complete. 2024-09-21T03:51:10.921 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:10.912+0000 7f2e75561740 20 client.16776 trim_cache size 0 max 16384 2024-09-21T03:51:10.921 INFO:teuthology.orchestra.run.smithi039.stdout:[] 2024-09-21T03:51:10.922 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:10.913+0000 7f2e6efed640 1 -- 172.21.15.39:0/4028104036 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f2e580b6700 msgr2=0x7f2e580b8c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:51:10.922 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:10.913+0000 7f2e6efed640 1 --2- 172.21.15.39:0/4028104036 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f2e580b6700 0x7f2e580b8c10 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f2e54005e00 tx=0x7f2e5400b420 comp rx=0 tx=0).stop 2024-09-21T03:51:10.922 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:10.913+0000 7f2e6efed640 1 -- 172.21.15.39:0/4028104036 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2e68108d10 msgr2=0x7f2e68110a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:51:10.923 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:10.913+0000 7f2e6efed640 1 --2- 172.21.15.39:0/4028104036 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2e68108d10 0x7f2e68110a50 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7f2e600098d0 tx=0x7f2e600317b0 comp rx=0 tx=0).stop 2024-09-21T03:51:10.923 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:10.913+0000 7f2e6efed640 1 -- 172.21.15.39:0/4028104036 shutdown_connections 2024-09-21T03:51:10.923 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:10.913+0000 7f2e6efed640 1 --2- 172.21.15.39:0/4028104036 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f2e580b6700 0x7f2e580b8c10 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:10.923 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:10.913+0000 7f2e6efed640 1 --2- 172.21.15.39:0/4028104036 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2e68108d10 0x7f2e68110a50 unknown :-1 s=CLOSED pgs=831 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:10.923 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:10.913+0000 7f2e6efed640 1 --2- 172.21.15.39:0/4028104036 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2e681083c0 0x7f2e68110510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:10.923 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:10.913+0000 7f2e6efed640 1 --2- 172.21.15.39:0/4028104036 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2e68102a80 0x7f2e6810ffd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:10.923 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:10.913+0000 7f2e6efed640 1 -- 172.21.15.39:0/4028104036 >> 172.21.15.39:0/4028104036 conn(0x7f2e68079740 msgr2=0x7f2e68104430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:51:10.923 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:10.913+0000 7f2e6efed640 1 -- 172.21.15.39:0/4028104036 shutdown_connections 2024-09-21T03:51:10.923 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:10.913+0000 7f2e6efed640 1 -- 172.21.15.39:0/4028104036 wait complete. 2024-09-21T03:51:11.334 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:51:11 smithi039 ceph-mon[31211]: pgmap v479: 97 pgs: 97 active+clean; 10 MiB data, 520 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 22 KiB/s wr, 26 op/s 2024-09-21T03:51:11.468 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:51:11 smithi161 ceph-mon[34533]: pgmap v479: 97 pgs: 97 active+clean; 10 MiB data, 520 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 22 KiB/s wr, 26 op/s 2024-09-21T03:51:11.474 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:51:11 smithi150 ceph-mon[34289]: pgmap v479: 97 pgs: 97 active+clean; 10 MiB data, 520 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 22 KiB/s wr, 26 op/s 2024-09-21T03:51:11.793 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 51116d5c-77ca-11ef-baf5-efdc52797490 -- ceph tell mds.1:2 damage ls 2024-09-21T03:51:12.058 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/51116d5c-77ca-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:51:12.575 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:12.572+0000 7ff5e9a0d640 1 -- 172.21.15.39:0/2593093683 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff5e41049b0 msgr2=0x7ff5e4104dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:51:12.575 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:12.572+0000 7ff5e9a0d640 1 --2- 172.21.15.39:0/2593093683 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff5e41049b0 0x7ff5e4104dc0 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7ff5d4009ca0 tx=0x7ff5d402a8c0 comp rx=0 tx=0).stop 2024-09-21T03:51:12.575 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:12.572+0000 7ff5e9a0d640 1 -- 172.21.15.39:0/2593093683 shutdown_connections 2024-09-21T03:51:12.575 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:12.572+0000 7ff5e9a0d640 1 --2- 172.21.15.39:0/2593093683 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff5e410de10 0x7ff5e4110300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:12.575 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:12.572+0000 7ff5e9a0d640 1 --2- 172.21.15.39:0/2593093683 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff5e4105300 0x7ff5e410d850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:12.576 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:12.572+0000 7ff5e9a0d640 1 --2- 172.21.15.39:0/2593093683 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff5e41049b0 0x7ff5e4104dc0 unknown :-1 s=CLOSED pgs=836 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:12.576 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:12.572+0000 7ff5e9a0d640 1 -- 172.21.15.39:0/2593093683 >> 172.21.15.39:0/2593093683 conn(0x7ff5e4079730 msgr2=0x7ff5e4079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:51:12.576 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:12.572+0000 7ff5e9a0d640 1 -- 172.21.15.39:0/2593093683 shutdown_connections 2024-09-21T03:51:12.576 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:12.572+0000 7ff5e9a0d640 1 -- 172.21.15.39:0/2593093683 wait complete. 2024-09-21T03:51:12.578 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:12.573+0000 7ff5e9a0d640 1 Processor -- start 2024-09-21T03:51:12.578 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:12.573+0000 7ff5e9a0d640 1 -- start start 2024-09-21T03:51:12.578 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:12.573+0000 7ff5e9a0d640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff5e41049b0 0x7ff5e419a4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:51:12.578 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:12.573+0000 7ff5e9a0d640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff5e4105300 0x7ff5e419aa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:51:12.578 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:12.573+0000 7ff5e9a0d640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff5e410de10 0x7ff5e419fd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:51:12.578 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:12.573+0000 7ff5e9a0d640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7ff5e4113a90 con 0x7ff5e41049b0 2024-09-21T03:51:12.578 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:12.573+0000 7ff5e9a0d640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7ff5e4113910 con 0x7ff5e4105300 2024-09-21T03:51:12.578 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:12.573+0000 7ff5e9a0d640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7ff5e4113c10 con 0x7ff5e410de10 2024-09-21T03:51:12.578 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:12.573+0000 7ff5e2ffd640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff5e41049b0 0x7ff5e419a4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:51:12.578 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:12.573+0000 7ff5e2ffd640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff5e41049b0 0x7ff5e419a4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:60480/0 (socket says 172.21.15.39:60480) 2024-09-21T03:51:12.579 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:12.574+0000 7ff5e2ffd640 1 -- 172.21.15.39:0/2773450273 learned_addr learned my addr 172.21.15.39:0/2773450273 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:51:12.579 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:12.574+0000 7ff5e37fe640 1 --2- 172.21.15.39:0/2773450273 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff5e410de10 0x7ff5e419fd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:51:12.579 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:12.574+0000 7ff5e27fc640 1 --2- 172.21.15.39:0/2773450273 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff5e4105300 0x7ff5e419aa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:51:12.579 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:12.574+0000 7ff5e2ffd640 1 -- 172.21.15.39:0/2773450273 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff5e410de10 msgr2=0x7ff5e419fd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:51:12.579 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:12.574+0000 7ff5e2ffd640 1 --2- 172.21.15.39:0/2773450273 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff5e410de10 0x7ff5e419fd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:12.579 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:12.574+0000 7ff5e2ffd640 1 -- 172.21.15.39:0/2773450273 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff5e4105300 msgr2=0x7ff5e419aa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:51:12.579 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:12.574+0000 7ff5e2ffd640 1 --2- 172.21.15.39:0/2773450273 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff5e4105300 0x7ff5e419aa60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:12.579 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:12.574+0000 7ff5e2ffd640 1 -- 172.21.15.39:0/2773450273 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff5e41a0430 con 0x7ff5e41049b0 2024-09-21T03:51:12.579 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:12.574+0000 7ff5e2ffd640 1 --2- 172.21.15.39:0/2773450273 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff5e41049b0 0x7ff5e419a4c0 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7ff5d402ade0 tx=0x7ff5d4031770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:51:12.579 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:12.574+0000 7ff5e8a0b640 1 -- 172.21.15.39:0/2773450273 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5d4003b70 con 0x7ff5e41049b0 2024-09-21T03:51:12.579 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:12.574+0000 7ff5e8a0b640 1 -- 172.21.15.39:0/2773450273 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff5d4003d10 con 0x7ff5e41049b0 2024-09-21T03:51:12.580 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:12.574+0000 7ff5e9a0d640 1 -- 172.21.15.39:0/2773450273 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff5e41a06c0 con 0x7ff5e41049b0 2024-09-21T03:51:12.580 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:12.574+0000 7ff5e8a0b640 1 -- 172.21.15.39:0/2773450273 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5d402fa50 con 0x7ff5e41049b0 2024-09-21T03:51:12.580 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:12.574+0000 7ff5e9a0d640 1 -- 172.21.15.39:0/2773450273 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff5e41a6e80 con 0x7ff5e41049b0 2024-09-21T03:51:12.582 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:12.577+0000 7ff5eff81740 1 Processor -- start 2024-09-21T03:51:12.582 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:12.577+0000 7ff5eff81740 1 -- start start 2024-09-21T03:51:12.582 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:12.577+0000 7ff5eff81740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561ec775b070 0x561ec7b022c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:51:12.582 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:12.577+0000 7ff5eff81740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561ec77321b0 0x561ec7b02800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:51:12.582 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:12.577+0000 7ff5eff81740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561ec773a390 0x561ec7b02d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:51:12.582 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:12.577+0000 7ff5eff81740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x561ec7a78ea0 con 0x561ec77321b0 2024-09-21T03:51:12.582 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:12.577+0000 7ff5eff81740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x561ec7a682d0 con 0x561ec775b070 2024-09-21T03:51:12.582 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:12.577+0000 7ff5eff81740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x561ec7a6c0e0 con 0x561ec773a390 2024-09-21T03:51:12.582 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:12.577+0000 7ff5e8a0b640 1 -- 172.21.15.39:0/2773450273 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 27) ==== 154643+0+0 (secure 0 0 0) 0x7ff5d402fc30 con 0x7ff5e41049b0 2024-09-21T03:51:12.582 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:12.577+0000 7ff5e2ffd640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561ec77321b0 0x561ec7b02800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:51:12.583 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:12.578+0000 7ff5e2ffd640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561ec77321b0 0x561ec7b02800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:60484/0 (socket says 172.21.15.39:60484) 2024-09-21T03:51:12.583 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:12.578+0000 7ff5e2ffd640 1 -- 172.21.15.39:0/818148186 learned_addr learned my addr 172.21.15.39:0/818148186 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:51:12.583 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:12.578+0000 7ff5e37fe640 1 --2- 172.21.15.39:0/818148186 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561ec775b070 0x561ec7b022c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:51:12.583 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:12.578+0000 7ff5e27fc640 1 --2- 172.21.15.39:0/818148186 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561ec773a390 0x561ec7b02d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:51:12.583 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:12.578+0000 7ff5e2ffd640 1 -- 172.21.15.39:0/818148186 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561ec773a390 msgr2=0x561ec7b02d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:51:12.583 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:12.578+0000 7ff5e2ffd640 1 --2- 172.21.15.39:0/818148186 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561ec773a390 0x561ec7b02d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:12.583 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:12.578+0000 7ff5e2ffd640 1 -- 172.21.15.39:0/818148186 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561ec775b070 msgr2=0x561ec7b022c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:51:12.583 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:12.578+0000 7ff5e2ffd640 1 --2- 172.21.15.39:0/818148186 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561ec775b070 0x561ec7b022c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:12.583 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:12.578+0000 7ff5e2ffd640 1 -- 172.21.15.39:0/818148186 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561ec77c6650 con 0x561ec77321b0 2024-09-21T03:51:12.583 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:12.578+0000 7ff5e27fc640 1 --2- 172.21.15.39:0/818148186 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561ec773a390 0x561ec7b02d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:51:12.583 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:12.578+0000 7ff5e2ffd640 1 --2- 172.21.15.39:0/818148186 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561ec77321b0 0x561ec7b02800 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7ff5d4105f70 tx=0x7ff5d410eb20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=13a24111b4335aff server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:51:12.583 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:12.578+0000 7ff5e8a0b640 1 --2- 172.21.15.39:0/2773450273 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7ff5cc0b6680 0x7ff5cc0b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:51:12.583 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:12.578+0000 7ff5c9ffb640 1 -- 172.21.15.39:0/818148186 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5d40cdea0 con 0x561ec77321b0 2024-09-21T03:51:12.584 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:12.578+0000 7ff5e37fe640 1 --2- 172.21.15.39:0/2773450273 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7ff5cc0b6680 0x7ff5cc0b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:51:12.584 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:12.579+0000 7ff5c9ffb640 1 -- 172.21.15.39:0/818148186 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff5d4118020 con 0x561ec77321b0 2024-09-21T03:51:12.584 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:12.579+0000 7ff5c9ffb640 1 -- 172.21.15.39:0/818148186 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5d40ce080 con 0x561ec77321b0 2024-09-21T03:51:12.584 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:12.579+0000 7ff5e8a0b640 1 -- 172.21.15.39:0/2773450273 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8484+0+0 (secure 0 0 0) 0x7ff5d4108920 con 0x7ff5e41049b0 2024-09-21T03:51:12.588 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:12.579+0000 7ff5eff81740 1 -- 172.21.15.39:0/818148186 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561ec77321b0 msgr2=0x561ec7b02800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:51:12.588 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:12.579+0000 7ff5e37fe640 1 --2- 172.21.15.39:0/2773450273 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7ff5cc0b6680 0x7ff5cc0b8b90 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7ff5dc009da0 tx=0x7ff5dc005f90 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:51:12.588 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:12.579+0000 7ff5eff81740 1 --2- 172.21.15.39:0/818148186 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561ec77321b0 0x561ec7b02800 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7ff5d4105f70 tx=0x7ff5d410eb20 comp rx=0 tx=0).stop 2024-09-21T03:51:12.588 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:12.579+0000 7ff5eff81740 1 -- 172.21.15.39:0/818148186 shutdown_connections 2024-09-21T03:51:12.588 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:12.580+0000 7ff5eff81740 1 --2- 172.21.15.39:0/818148186 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561ec773a390 0x561ec7b02d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:12.588 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:12.580+0000 7ff5eff81740 1 --2- 172.21.15.39:0/818148186 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561ec77321b0 0x561ec7b02800 unknown :-1 s=CLOSED pgs=838 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:12.588 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:12.580+0000 7ff5eff81740 1 --2- 172.21.15.39:0/818148186 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561ec775b070 0x561ec7b022c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:12.588 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:12.580+0000 7ff5eff81740 1 -- 172.21.15.39:0/818148186 >> 172.21.15.39:0/818148186 conn(0x561ec774f460 msgr2=0x561ec7b001b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:51:12.588 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:12.580+0000 7ff5eff81740 1 -- 172.21.15.39:0/818148186 shutdown_connections 2024-09-21T03:51:12.588 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:12.580+0000 7ff5eff81740 1 -- 172.21.15.39:0/818148186 wait complete. 2024-09-21T03:51:12.589 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:12.580+0000 7ff5eff81740 1 -- start start 2024-09-21T03:51:12.589 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:12.580+0000 7ff5eff81740 1 Processor -- start 2024-09-21T03:51:12.589 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:12.581+0000 7ff5eff81740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561ec77321b0 0x561ec7b0d980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:51:12.589 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:12.581+0000 7ff5eff81740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561ec773a390 0x561ec7b0dec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:51:12.589 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:12.581+0000 7ff5eff81740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561ec775b070 0x561ec7b19940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:51:12.589 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:12.581+0000 7ff5eff81740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x561ec78c9700 con 0x561ec773a390 2024-09-21T03:51:12.589 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:12.581+0000 7ff5eff81740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x561ec7a03160 con 0x561ec775b070 2024-09-21T03:51:12.589 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:12.581+0000 7ff5eff81740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x561ec7a78ea0 con 0x561ec77321b0 2024-09-21T03:51:12.589 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:12.581+0000 7ff5eff81740 10 client.0 _ll_register_callbacks cb 0x561ec7a1c9e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:51:12.589 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:12.581+0000 7ff5e27fc640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561ec773a390 0x561ec7b0dec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:51:12.589 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:12.581+0000 7ff5e27fc640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561ec773a390 0x561ec7b0dec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:60494/0 (socket says 172.21.15.39:60494) 2024-09-21T03:51:12.589 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:12.581+0000 7ff5e27fc640 1 -- 172.21.15.39:0/397631079 learned_addr learned my addr 172.21.15.39:0/397631079 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:51:12.589 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:12.581+0000 7ff5e2ffd640 1 --2- 172.21.15.39:0/397631079 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561ec77321b0 0x561ec7b0d980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:51:12.589 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:12.581+0000 7ff5e37fe640 1 --2- 172.21.15.39:0/397631079 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561ec775b070 0x561ec7b19940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:51:12.590 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:12.581+0000 7ff5e27fc640 1 -- 172.21.15.39:0/397631079 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561ec77321b0 msgr2=0x561ec7b0d980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:51:12.590 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:12.581+0000 7ff5e27fc640 1 --2- 172.21.15.39:0/397631079 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561ec77321b0 0x561ec7b0d980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:12.590 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:12.581+0000 7ff5e27fc640 1 -- 172.21.15.39:0/397631079 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561ec775b070 msgr2=0x561ec7b19940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:51:12.590 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:12.581+0000 7ff5e27fc640 1 --2- 172.21.15.39:0/397631079 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561ec775b070 0x561ec7b19940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:12.590 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:12.581+0000 7ff5e27fc640 1 -- 172.21.15.39:0/397631079 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561ec7a30c90 con 0x561ec773a390 2024-09-21T03:51:12.590 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:12.581+0000 7ff5e27fc640 1 --2- 172.21.15.39:0/397631079 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561ec773a390 0x561ec7b0dec0 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7ff5d800b7f0 tx=0x7ff5d800bcb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fefd845b7006eca4 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:51:12.590 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:12.581+0000 7ff5c2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-09-21T03:51:12.590 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:12.581+0000 7ff5c2ffd640 1 -- 172.21.15.39:0/397631079 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5d801b440 con 0x561ec773a390 2024-09-21T03:51:12.590 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:12.582+0000 7ff5c2ffd640 1 -- 172.21.15.39:0/397631079 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8484+0+0 (secure 0 0 0) 0x7ff5d801b5e0 con 0x561ec773a390 2024-09-21T03:51:12.590 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:12.582+0000 7ff5eff81740 1 -- 172.21.15.39:0/397631079 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff5d4118020 con 0x561ec773a390 2024-09-21T03:51:12.590 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:12.582+0000 7ff5c2ffd640 1 -- 172.21.15.39:0/397631079 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7ff5d801b7e0 con 0x561ec773a390 2024-09-21T03:51:12.590 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:12.582+0000 7ff5eff81740 10 client.16788 fetch_fsmap learned FSMap version 29 2024-09-21T03:51:12.590 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:12.582+0000 7ff5eff81740 1 -- 172.21.15.39:0/397631079 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x561ec7b1c360 con 0x561ec773a390 2024-09-21T03:51:12.590 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:12.582+0000 7ff5c2ffd640 1 -- 172.21.15.39:0/397631079 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff5d4118020 con 0x561ec773a390 2024-09-21T03:51:12.591 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:12.582+0000 7ff5c2ffd640 1 -- 172.21.15.39:0/397631079 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7ff5d8022020 con 0x561ec773a390 2024-09-21T03:51:12.591 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:12.582+0000 7ff5eff81740 10 client.16788 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T03:51:12.591 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:12.582+0000 7ff5eff81740 10 client.16788 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-21T03:51:12.591 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:12.582+0000 7ff5eff81740 20 client.16788 populate_metadata read hostname 'smithi039' 2024-09-21T03:51:12.591 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:12.582+0000 7ff5eff81740 1 --2- 172.21.15.39:0/397631079 >> [v2:172.21.15.150:6836/4017043144,v1:172.21.15.150:6837/4017043144] conn(0x561ec77199f0 0x561ec7b1f800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:51:12.591 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:12.582+0000 7ff5eff81740 4 client.16788 mds_command: new command op to 24659 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:51:12.591 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:12.582+0000 7ff5eff81740 1 -- 172.21.15.39:0/397631079 --> [v2:172.21.15.150:6836/4017043144,v1:172.21.15.150:6837/4017043144] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561ec7a7adf0 con 0x561ec77199f0 2024-09-21T03:51:12.591 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:12.583+0000 7ff5e2ffd640 1 --2- 172.21.15.39:0/397631079 >> [v2:172.21.15.150:6836/4017043144,v1:172.21.15.150:6837/4017043144] conn(0x561ec77199f0 0x561ec7b1f800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:51:12.591 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:12.583+0000 7ff5e2ffd640 1 --2- 172.21.15.39:0/397631079 >> [v2:172.21.15.150:6836/4017043144,v1:172.21.15.150:6837/4017043144] conn(0x561ec77199f0 0x561ec7b1f800 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=92e822d540efb473 server_cookie=3765a2f505c535f1 in_seq=0 out_seq=0 2024-09-21T03:51:12.591 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:12.583+0000 7ff5c2ffd640 10 client.16788 ms_handle_connect on v2:172.21.15.150:6836/4017043144 2024-09-21T03:51:12.591 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:12.584+0000 7ff5c2ffd640 1 -- 172.21.15.39:0/397631079 <== mds.2 v2:172.21.15.150:6836/4017043144 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x561ec7a7adf0 con 0x561ec77199f0 2024-09-21T03:51:12.591 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:12.584+0000 7ff5c2ffd640 10 client.16788 handle_command_reply: tid=0 2024-09-21T03:51:12.591 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:12.584+0000 7ff5eff81740 1 client.16788 shutdown 2024-09-21T03:51:12.592 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:12.584+0000 7ff5eff81740 1 -- 172.21.15.39:0/397631079 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561ec773a390 msgr2=0x561ec7b0dec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:51:12.592 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:12.584+0000 7ff5eff81740 1 --2- 172.21.15.39:0/397631079 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561ec773a390 0x561ec7b0dec0 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7ff5d800b7f0 tx=0x7ff5d800bcb0 comp rx=0 tx=0).stop 2024-09-21T03:51:12.592 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:12.584+0000 7ff5eff81740 1 -- 172.21.15.39:0/397631079 shutdown_connections 2024-09-21T03:51:12.592 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:12.584+0000 7ff5eff81740 1 --2- 172.21.15.39:0/397631079 >> [v2:172.21.15.150:6836/4017043144,v1:172.21.15.150:6837/4017043144] conn(0x561ec77199f0 0x561ec7b1f800 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:12.592 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:12.584+0000 7ff5eff81740 1 --2- 172.21.15.39:0/397631079 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561ec775b070 0x561ec7b19940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:12.592 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:12.584+0000 7ff5eff81740 1 --2- 172.21.15.39:0/397631079 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561ec773a390 0x561ec7b0dec0 unknown :-1 s=CLOSED pgs=839 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:12.592 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:12.584+0000 7ff5eff81740 1 --2- 172.21.15.39:0/397631079 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561ec77321b0 0x561ec7b0d980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:12.592 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:12.584+0000 7ff5eff81740 1 -- 172.21.15.39:0/397631079 >> 172.21.15.39:0/397631079 conn(0x561ec774f460 msgr2=0x561ec7b05450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:51:12.592 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:12.584+0000 7ff5c2ffd640 0 client.16788 ms_handle_reset on v2:172.21.15.150:6836/4017043144 2024-09-21T03:51:12.592 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:12.584+0000 7ff5eff81740 1 -- 172.21.15.39:0/397631079 shutdown_connections 2024-09-21T03:51:12.592 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:12.584+0000 7ff5eff81740 1 -- 172.21.15.39:0/397631079 wait complete. 2024-09-21T03:51:12.592 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:12.584+0000 7ff5eff81740 20 client.16788 trim_cache size 0 max 16384 2024-09-21T03:51:12.601 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:12.597+0000 7ff5eff81740 1 Processor -- start 2024-09-21T03:51:12.601 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:12.598+0000 7ff5eff81740 1 -- start start 2024-09-21T03:51:12.602 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:12.598+0000 7ff5eff81740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561ec77321b0 0x561ec7b03cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:51:12.602 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:12.598+0000 7ff5eff81740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561ec773a390 0x561ec7b0f5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:51:12.602 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:12.598+0000 7ff5eff81740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561ec775b070 0x561ec7b19840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:51:12.602 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:12.598+0000 7ff5eff81740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7ff5e41b19b0 con 0x561ec77321b0 2024-09-21T03:51:12.602 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:12.598+0000 7ff5eff81740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7ff5e41b34f0 con 0x561ec773a390 2024-09-21T03:51:12.602 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:12.598+0000 7ff5eff81740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7ff5e41b2080 con 0x561ec775b070 2024-09-21T03:51:12.602 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:12.598+0000 7ff5e2ffd640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561ec77321b0 0x561ec7b03cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:51:12.602 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:12.598+0000 7ff5e2ffd640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561ec77321b0 0x561ec7b03cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:60500/0 (socket says 172.21.15.39:60500) 2024-09-21T03:51:12.602 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:12.598+0000 7ff5e2ffd640 1 -- 172.21.15.39:0/2686276921 learned_addr learned my addr 172.21.15.39:0/2686276921 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:51:12.602 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:12.598+0000 7ff5e27fc640 1 --2- 172.21.15.39:0/2686276921 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561ec773a390 0x561ec7b0f5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:51:12.602 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:12.598+0000 7ff5e37fe640 1 --2- 172.21.15.39:0/2686276921 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561ec775b070 0x561ec7b19840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:51:12.603 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:12.598+0000 7ff5e2ffd640 1 -- 172.21.15.39:0/2686276921 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561ec775b070 msgr2=0x561ec7b19840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:51:12.603 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:12.598+0000 7ff5e2ffd640 1 --2- 172.21.15.39:0/2686276921 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561ec775b070 0x561ec7b19840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:12.603 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:12.598+0000 7ff5e2ffd640 1 -- 172.21.15.39:0/2686276921 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561ec773a390 msgr2=0x561ec7b0f5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:51:12.603 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:12.598+0000 7ff5e2ffd640 1 --2- 172.21.15.39:0/2686276921 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561ec773a390 0x561ec7b0f5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:12.603 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:12.598+0000 7ff5e2ffd640 1 -- 172.21.15.39:0/2686276921 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561ec7b045b0 con 0x561ec77321b0 2024-09-21T03:51:12.603 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:12.598+0000 7ff5e27fc640 1 --2- 172.21.15.39:0/2686276921 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561ec773a390 0x561ec7b0f5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:51:12.603 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:12.598+0000 7ff5e2ffd640 1 --2- 172.21.15.39:0/2686276921 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561ec77321b0 0x561ec7b03cb0 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7ff5d40cad00 tx=0x7ff5d4000f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e048a91b038d6b00 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:51:12.603 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:12.599+0000 7ff5c97fa640 1 -- 172.21.15.39:0/2686276921 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5d4117040 con 0x561ec77321b0 2024-09-21T03:51:12.603 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:12.599+0000 7ff5c97fa640 1 -- 172.21.15.39:0/2686276921 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff5d4036040 con 0x561ec77321b0 2024-09-21T03:51:12.603 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:12.599+0000 7ff5c97fa640 1 -- 172.21.15.39:0/2686276921 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5d40ce270 con 0x561ec77321b0 2024-09-21T03:51:12.607 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:12.599+0000 7ff5eff81740 1 -- 172.21.15.39:0/2686276921 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561ec77321b0 msgr2=0x561ec7b03cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:51:12.607 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:12.599+0000 7ff5eff81740 1 --2- 172.21.15.39:0/2686276921 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561ec77321b0 0x561ec7b03cb0 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7ff5d40cad00 tx=0x7ff5d4000f40 comp rx=0 tx=0).stop 2024-09-21T03:51:12.607 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:12.600+0000 7ff5eff81740 1 -- 172.21.15.39:0/2686276921 shutdown_connections 2024-09-21T03:51:12.607 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:12.600+0000 7ff5eff81740 1 --2- 172.21.15.39:0/2686276921 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561ec775b070 0x561ec7b19840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:12.607 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:12.600+0000 7ff5eff81740 1 --2- 172.21.15.39:0/2686276921 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561ec773a390 0x561ec7b0f5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:12.607 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:12.600+0000 7ff5eff81740 1 --2- 172.21.15.39:0/2686276921 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561ec77321b0 0x561ec7b03cb0 unknown :-1 s=CLOSED pgs=840 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:12.607 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:12.600+0000 7ff5eff81740 1 -- 172.21.15.39:0/2686276921 >> 172.21.15.39:0/2686276921 conn(0x561ec774f460 msgr2=0x561ec7b0eea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:51:12.608 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:12.600+0000 7ff5eff81740 1 -- 172.21.15.39:0/2686276921 shutdown_connections 2024-09-21T03:51:12.608 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:12.600+0000 7ff5eff81740 1 -- 172.21.15.39:0/2686276921 wait complete. 2024-09-21T03:51:12.608 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:12.600+0000 7ff5eff81740 1 -- start start 2024-09-21T03:51:12.608 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:12.600+0000 7ff5eff81740 1 Processor -- start 2024-09-21T03:51:12.608 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:12.600+0000 7ff5eff81740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561ec77321b0 0x561ec7b118a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:51:12.608 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:12.600+0000 7ff5eff81740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561ec773a390 0x561ec7b11de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:51:12.608 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:12.601+0000 7ff5eff81740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561ec775b070 0x561ec7b12320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:51:12.608 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:12.601+0000 7ff5eff81740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7ff5e41b2e20 con 0x561ec773a390 2024-09-21T03:51:12.608 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:12.601+0000 7ff5eff81740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7ff5e41b2750 con 0x561ec77321b0 2024-09-21T03:51:12.608 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:12.601+0000 7ff5eff81740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7ff5e41af990 con 0x561ec775b070 2024-09-21T03:51:12.608 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:12.601+0000 7ff5eff81740 10 client.0 _ll_register_callbacks cb 0x7ff5d8005c50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:51:12.608 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:12.601+0000 7ff5e27fc640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561ec773a390 0x561ec7b11de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:51:12.608 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:12.601+0000 7ff5e27fc640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561ec773a390 0x561ec7b11de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:60516/0 (socket says 172.21.15.39:60516) 2024-09-21T03:51:12.609 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:12.601+0000 7ff5e27fc640 1 -- 172.21.15.39:0/232034685 learned_addr learned my addr 172.21.15.39:0/232034685 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:51:12.609 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:12.601+0000 7ff5e37fe640 1 --2- 172.21.15.39:0/232034685 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561ec775b070 0x561ec7b12320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:51:12.609 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:12.601+0000 7ff5e2ffd640 1 --2- 172.21.15.39:0/232034685 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561ec77321b0 0x561ec7b118a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:51:12.609 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:12.601+0000 7ff5e27fc640 1 -- 172.21.15.39:0/232034685 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561ec775b070 msgr2=0x561ec7b12320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:51:12.609 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:12.601+0000 7ff5e27fc640 1 --2- 172.21.15.39:0/232034685 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561ec775b070 0x561ec7b12320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:12.609 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:12.601+0000 7ff5e27fc640 1 -- 172.21.15.39:0/232034685 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561ec77321b0 msgr2=0x561ec7b118a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:51:12.609 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:12.601+0000 7ff5e27fc640 1 --2- 172.21.15.39:0/232034685 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561ec77321b0 0x561ec7b118a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:12.609 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:12.601+0000 7ff5e27fc640 1 -- 172.21.15.39:0/232034685 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561ec7b891f0 con 0x561ec773a390 2024-09-21T03:51:12.609 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:12.601+0000 7ff5e27fc640 1 --2- 172.21.15.39:0/232034685 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561ec773a390 0x561ec7b11de0 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7ff5d800b7f0 tx=0x7ff5d8024ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=46e52273bede5f34 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:51:12.609 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:12.601+0000 7ff5c2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-09-21T03:51:12.609 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:12.601+0000 7ff5c2ffd640 1 -- 172.21.15.39:0/232034685 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5d8024460 con 0x561ec773a390 2024-09-21T03:51:12.609 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:12.601+0000 7ff5c2ffd640 1 -- 172.21.15.39:0/232034685 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8484+0+0 (secure 0 0 0) 0x7ff5d8024600 con 0x561ec773a390 2024-09-21T03:51:12.609 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:12.601+0000 7ff5eff81740 1 -- 172.21.15.39:0/232034685 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff5d4036040 con 0x561ec773a390 2024-09-21T03:51:12.609 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:12.602+0000 7ff5c2ffd640 1 -- 172.21.15.39:0/232034685 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7ff5d8024800 con 0x561ec773a390 2024-09-21T03:51:12.610 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:12.602+0000 7ff5eff81740 10 client.16794 fetch_fsmap learned FSMap version 29 2024-09-21T03:51:12.610 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:12.602+0000 7ff5eff81740 1 -- 172.21.15.39:0/232034685 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x561ec7b8a470 con 0x561ec773a390 2024-09-21T03:51:12.610 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:12.602+0000 7ff5c2ffd640 1 -- 172.21.15.39:0/232034685 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff5d4036040 con 0x561ec773a390 2024-09-21T03:51:12.610 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:12.602+0000 7ff5c2ffd640 1 -- 172.21.15.39:0/232034685 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7ff5d8022040 con 0x561ec773a390 2024-09-21T03:51:12.610 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:12.602+0000 7ff5eff81740 10 client.16794 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T03:51:12.610 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:12.602+0000 7ff5eff81740 10 client.16794 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-21T03:51:12.610 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:12.602+0000 7ff5eff81740 20 client.16794 populate_metadata read hostname 'smithi039' 2024-09-21T03:51:12.610 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:12.602+0000 7ff5eff81740 1 --2- 172.21.15.39:0/232034685 >> [v2:172.21.15.150:6836/4017043144,v1:172.21.15.150:6837/4017043144] conn(0x561ec76eeda0 0x561ec7b8cbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:51:12.610 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:12.602+0000 7ff5eff81740 4 client.16794 mds_command: new command op to 24659 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T03:51:12.610 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:12.602+0000 7ff5eff81740 1 -- 172.21.15.39:0/232034685 --> [v2:172.21.15.150:6836/4017043144,v1:172.21.15.150:6837/4017043144] -- command(tid 0: {"prefix": "damage ls"}) -- 0x561ec7a768a0 con 0x561ec76eeda0 2024-09-21T03:51:12.610 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:12.603+0000 7ff5e2ffd640 1 --2- 172.21.15.39:0/232034685 >> [v2:172.21.15.150:6836/4017043144,v1:172.21.15.150:6837/4017043144] conn(0x561ec76eeda0 0x561ec7b8cbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:51:12.610 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:12.603+0000 7ff5c2ffd640 10 client.16794 ms_handle_connect on v2:172.21.15.150:6836/4017043144 2024-09-21T03:51:12.610 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:12.603+0000 7ff5e2ffd640 1 --2- 172.21.15.39:0/232034685 >> [v2:172.21.15.150:6836/4017043144,v1:172.21.15.150:6837/4017043144] conn(0x561ec76eeda0 0x561ec7b8cbf0 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=aeed08205bafbb84 server_cookie=665a22d935e4cef5 in_seq=0 out_seq=0 2024-09-21T03:51:12.611 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:12.603+0000 7ff5c2ffd640 1 -- 172.21.15.39:0/232034685 <== mds.2 v2:172.21.15.150:6836/4017043144 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x561ec7a768a0 con 0x561ec76eeda0 2024-09-21T03:51:12.611 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:12.603+0000 7ff5c2ffd640 10 client.16794 handle_command_reply: tid=0 2024-09-21T03:51:12.611 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:12.603+0000 7ff5eff81740 1 client.16794 shutdown 2024-09-21T03:51:12.611 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:12.603+0000 7ff5eff81740 1 -- 172.21.15.39:0/232034685 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561ec773a390 msgr2=0x561ec7b11de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:51:12.611 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:12.603+0000 7ff5eff81740 1 --2- 172.21.15.39:0/232034685 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561ec773a390 0x561ec7b11de0 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7ff5d800b7f0 tx=0x7ff5d8024ef0 comp rx=0 tx=0).stop 2024-09-21T03:51:12.611 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:12.604+0000 7ff5eff81740 1 -- 172.21.15.39:0/232034685 shutdown_connections 2024-09-21T03:51:12.611 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:12.604+0000 7ff5eff81740 1 --2- 172.21.15.39:0/232034685 >> [v2:172.21.15.150:6836/4017043144,v1:172.21.15.150:6837/4017043144] conn(0x561ec76eeda0 0x561ec7b8cbf0 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:12.611 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:12.604+0000 7ff5eff81740 1 --2- 172.21.15.39:0/232034685 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561ec775b070 0x561ec7b12320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:12.611 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:12.604+0000 7ff5eff81740 1 --2- 172.21.15.39:0/232034685 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561ec773a390 0x561ec7b11de0 unknown :-1 s=CLOSED pgs=841 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:12.611 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:12.604+0000 7ff5eff81740 1 --2- 172.21.15.39:0/232034685 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561ec77321b0 0x561ec7b118a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:12.611 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:12.604+0000 7ff5eff81740 1 -- 172.21.15.39:0/232034685 >> 172.21.15.39:0/232034685 conn(0x561ec774f460 msgr2=0x561ec7b03a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:51:12.611 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:12.604+0000 7ff5c2ffd640 0 client.16794 ms_handle_reset on v2:172.21.15.150:6836/4017043144 2024-09-21T03:51:12.612 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:12.604+0000 7ff5eff81740 1 -- 172.21.15.39:0/232034685 shutdown_connections 2024-09-21T03:51:12.612 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:12.604+0000 7ff5eff81740 1 -- 172.21.15.39:0/232034685 wait complete. 2024-09-21T03:51:12.612 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:12.604+0000 7ff5eff81740 20 client.16794 trim_cache size 0 max 16384 2024-09-21T03:51:12.612 INFO:teuthology.orchestra.run.smithi039.stdout:[] 2024-09-21T03:51:12.613 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:12.604+0000 7ff5e9a0d640 1 -- 172.21.15.39:0/2773450273 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7ff5cc0b6680 msgr2=0x7ff5cc0b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:51:12.613 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:12.604+0000 7ff5e9a0d640 1 --2- 172.21.15.39:0/2773450273 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7ff5cc0b6680 0x7ff5cc0b8b90 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7ff5dc009da0 tx=0x7ff5dc005f90 comp rx=0 tx=0).stop 2024-09-21T03:51:12.613 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:12.604+0000 7ff5e9a0d640 1 -- 172.21.15.39:0/2773450273 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff5e41049b0 msgr2=0x7ff5e419a4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:51:12.613 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:12.604+0000 7ff5e9a0d640 1 --2- 172.21.15.39:0/2773450273 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff5e41049b0 0x7ff5e419a4c0 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7ff5d402ade0 tx=0x7ff5d4031770 comp rx=0 tx=0).stop 2024-09-21T03:51:12.613 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:12.604+0000 7ff5e9a0d640 1 -- 172.21.15.39:0/2773450273 shutdown_connections 2024-09-21T03:51:12.613 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:12.604+0000 7ff5e9a0d640 1 --2- 172.21.15.39:0/2773450273 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7ff5cc0b6680 0x7ff5cc0b8b90 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:12.613 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:12.605+0000 7ff5e9a0d640 1 --2- 172.21.15.39:0/2773450273 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff5e410de10 0x7ff5e419fd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:12.613 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:12.605+0000 7ff5e9a0d640 1 --2- 172.21.15.39:0/2773450273 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff5e4105300 0x7ff5e419aa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:12.613 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:12.605+0000 7ff5e9a0d640 1 --2- 172.21.15.39:0/2773450273 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff5e41049b0 0x7ff5e419a4c0 unknown :-1 s=CLOSED pgs=837 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:12.613 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:12.605+0000 7ff5e9a0d640 1 -- 172.21.15.39:0/2773450273 >> 172.21.15.39:0/2773450273 conn(0x7ff5e4079730 msgr2=0x7ff5e4100770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:51:12.613 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:12.605+0000 7ff5e9a0d640 1 -- 172.21.15.39:0/2773450273 shutdown_connections 2024-09-21T03:51:12.613 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:12.605+0000 7ff5e9a0d640 1 -- 172.21.15.39:0/2773450273 wait complete. 2024-09-21T03:51:13.302 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:51:13 smithi039 ceph-mon[31211]: pgmap v480: 97 pgs: 97 active+clean; 10 MiB data, 520 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 24 op/s 2024-09-21T03:51:13.330 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 51116d5c-77ca-11ef-baf5-efdc52797490 -- ceph tell mds.1:1 damage ls 2024-09-21T03:51:13.468 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:51:13 smithi161 ceph-mon[34533]: pgmap v480: 97 pgs: 97 active+clean; 10 MiB data, 520 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 24 op/s 2024-09-21T03:51:13.474 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:51:13 smithi150 ceph-mon[34289]: pgmap v480: 97 pgs: 97 active+clean; 10 MiB data, 520 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 24 op/s 2024-09-21T03:51:13.589 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/51116d5c-77ca-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:51:14.262 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:14.254+0000 7fdd5af21640 1 -- 172.21.15.39:0/1770648901 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdd541049b0 msgr2=0x7fdd54104dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:51:14.262 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:14.254+0000 7fdd5af21640 1 --2- 172.21.15.39:0/1770648901 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdd541049b0 0x7fdd54104dc0 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7fdd48009900 tx=0x7fdd4802a410 comp rx=0 tx=0).stop 2024-09-21T03:51:14.262 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:14.254+0000 7fdd5af21640 1 -- 172.21.15.39:0/1770648901 shutdown_connections 2024-09-21T03:51:14.262 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:14.254+0000 7fdd5af21640 1 --2- 172.21.15.39:0/1770648901 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdd5410de10 0x7fdd54110300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:14.262 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:14.254+0000 7fdd5af21640 1 --2- 172.21.15.39:0/1770648901 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdd54105300 0x7fdd5410d850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:14.262 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:14.254+0000 7fdd5af21640 1 --2- 172.21.15.39:0/1770648901 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdd541049b0 0x7fdd54104dc0 unknown :-1 s=CLOSED pgs=842 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:14.262 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:14.254+0000 7fdd5af21640 1 -- 172.21.15.39:0/1770648901 >> 172.21.15.39:0/1770648901 conn(0x7fdd54079730 msgr2=0x7fdd54079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:51:14.263 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:14.254+0000 7fdd5af21640 1 -- 172.21.15.39:0/1770648901 shutdown_connections 2024-09-21T03:51:14.263 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:14.254+0000 7fdd5af21640 1 -- 172.21.15.39:0/1770648901 wait complete. 2024-09-21T03:51:14.263 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:14.255+0000 7fdd5af21640 1 Processor -- start 2024-09-21T03:51:14.263 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:14.255+0000 7fdd5af21640 1 -- start start 2024-09-21T03:51:14.263 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:14.255+0000 7fdd5af21640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdd541049b0 0x7fdd5419a4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:51:14.263 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:14.255+0000 7fdd5af21640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdd54105300 0x7fdd5419aa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:51:14.263 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:14.255+0000 7fdd5af21640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdd5410de10 0x7fdd5419fd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:51:14.263 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:14.255+0000 7fdd5af21640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fdd54113a70 con 0x7fdd541049b0 2024-09-21T03:51:14.263 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:14.255+0000 7fdd5af21640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fdd541138f0 con 0x7fdd54105300 2024-09-21T03:51:14.263 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:14.255+0000 7fdd5af21640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fdd54113bf0 con 0x7fdd5410de10 2024-09-21T03:51:14.264 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:14.255+0000 7fdd58cbd640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdd541049b0 0x7fdd5419a4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:51:14.264 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:14.256+0000 7fdd58cbd640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdd541049b0 0x7fdd5419a4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:60536/0 (socket says 172.21.15.39:60536) 2024-09-21T03:51:14.264 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:14.256+0000 7fdd58cbd640 1 -- 172.21.15.39:0/2196291557 learned_addr learned my addr 172.21.15.39:0/2196291557 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:51:14.264 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:14.256+0000 7fdd594be640 1 --2- 172.21.15.39:0/2196291557 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdd5410de10 0x7fdd5419fd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:51:14.264 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:14.256+0000 7fdd53fff640 1 --2- 172.21.15.39:0/2196291557 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdd54105300 0x7fdd5419aa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:51:14.264 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:14.256+0000 7fdd58cbd640 1 -- 172.21.15.39:0/2196291557 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdd5410de10 msgr2=0x7fdd5419fd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:51:14.264 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:14.256+0000 7fdd58cbd640 1 --2- 172.21.15.39:0/2196291557 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdd5410de10 0x7fdd5419fd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:14.264 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:14.256+0000 7fdd58cbd640 1 -- 172.21.15.39:0/2196291557 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdd54105300 msgr2=0x7fdd5419aa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:51:14.264 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:14.256+0000 7fdd58cbd640 1 --2- 172.21.15.39:0/2196291557 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdd54105300 0x7fdd5419aa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:14.264 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:14.256+0000 7fdd58cbd640 1 -- 172.21.15.39:0/2196291557 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdd541a0450 con 0x7fdd541049b0 2024-09-21T03:51:14.264 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:14.256+0000 7fdd53fff640 1 --2- 172.21.15.39:0/2196291557 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdd54105300 0x7fdd5419aa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:51:14.264 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:14.256+0000 7fdd594be640 1 --2- 172.21.15.39:0/2196291557 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdd5410de10 0x7fdd5419fd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:51:14.264 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:14.256+0000 7fdd58cbd640 1 --2- 172.21.15.39:0/2196291557 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdd541049b0 0x7fdd5419a4e0 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7fdd48032fd0 tx=0x7fdd48004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:51:14.265 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:14.256+0000 7fdd51ffb640 1 -- 172.21.15.39:0/2196291557 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd4803f030 con 0x7fdd541049b0 2024-09-21T03:51:14.265 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:14.256+0000 7fdd51ffb640 1 -- 172.21.15.39:0/2196291557 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdd48004490 con 0x7fdd541049b0 2024-09-21T03:51:14.265 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:14.256+0000 7fdd5af21640 1 -- 172.21.15.39:0/2196291557 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdd541a06e0 con 0x7fdd541049b0 2024-09-21T03:51:14.265 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:14.256+0000 7fdd51ffb640 1 -- 172.21.15.39:0/2196291557 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd48035070 con 0x7fdd541049b0 2024-09-21T03:51:14.265 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:14.257+0000 7fdd5af21640 1 -- 172.21.15.39:0/2196291557 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdd541a6ea0 con 0x7fdd541049b0 2024-09-21T03:51:14.270 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:14.260+0000 7fdd61495740 1 Processor -- start 2024-09-21T03:51:14.270 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:14.260+0000 7fdd61495740 1 -- start start 2024-09-21T03:51:14.270 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:14.260+0000 7fdd51ffb640 1 -- 172.21.15.39:0/2196291557 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 27) ==== 154643+0+0 (secure 0 0 0) 0x7fdd48042030 con 0x7fdd541049b0 2024-09-21T03:51:14.270 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:14.260+0000 7fdd61495740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55bd7215c070 0x55bd725032c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:51:14.270 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:14.260+0000 7fdd61495740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55bd721331b0 0x55bd72503800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:51:14.270 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:14.260+0000 7fdd61495740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55bd7213b390 0x55bd72503d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:51:14.270 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:14.260+0000 7fdd61495740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x55bd72479ea0 con 0x55bd721331b0 2024-09-21T03:51:14.270 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:14.260+0000 7fdd61495740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x55bd724692d0 con 0x55bd7215c070 2024-09-21T03:51:14.270 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:14.260+0000 7fdd61495740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55bd7246d0e0 con 0x55bd7213b390 2024-09-21T03:51:14.270 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:14.260+0000 7fdd58cbd640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55bd721331b0 0x55bd72503800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:51:14.270 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:14.261+0000 7fdd58cbd640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55bd721331b0 0x55bd72503800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:60538/0 (socket says 172.21.15.39:60538) 2024-09-21T03:51:14.271 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:14.261+0000 7fdd58cbd640 1 -- 172.21.15.39:0/3824799064 learned_addr learned my addr 172.21.15.39:0/3824799064 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:51:14.271 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:14.261+0000 7fdd594be640 1 --2- 172.21.15.39:0/3824799064 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55bd7215c070 0x55bd725032c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:51:14.271 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:14.261+0000 7fdd58cbd640 1 -- 172.21.15.39:0/3824799064 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55bd7213b390 msgr2=0x55bd72503d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:51:14.271 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:14.261+0000 7fdd58cbd640 1 --2- 172.21.15.39:0/3824799064 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55bd7213b390 0x55bd72503d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:14.271 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:14.261+0000 7fdd58cbd640 1 -- 172.21.15.39:0/3824799064 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55bd7215c070 msgr2=0x55bd725032c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:51:14.271 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:14.261+0000 7fdd58cbd640 1 --2- 172.21.15.39:0/3824799064 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55bd7215c070 0x55bd725032c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:14.271 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:14.261+0000 7fdd58cbd640 1 -- 172.21.15.39:0/3824799064 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bd721c7650 con 0x55bd721331b0 2024-09-21T03:51:14.271 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:14.261+0000 7fdd51ffb640 1 --2- 172.21.15.39:0/2196291557 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7fdd440b6680 0x7fdd440b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:51:14.271 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:14.261+0000 7fdd58cbd640 1 --2- 172.21.15.39:0/3824799064 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55bd721331b0 0x55bd72503800 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7fdd481053c0 tx=0x7fdd48110450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=15e2e47d6e829ce1 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:51:14.271 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:14.262+0000 7fdd51ffb640 1 -- 172.21.15.39:0/2196291557 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8484+0+0 (secure 0 0 0) 0x7fdd48107ce0 con 0x7fdd541049b0 2024-09-21T03:51:14.271 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:14.262+0000 7fdd594be640 1 --2- 172.21.15.39:0/2196291557 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7fdd440b6680 0x7fdd440b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:51:14.271 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:14.262+0000 7fdd3effd640 1 -- 172.21.15.39:0/3824799064 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd480ca1a0 con 0x55bd721331b0 2024-09-21T03:51:14.271 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:14.262+0000 7fdd3effd640 1 -- 172.21.15.39:0/3824799064 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdd480cd660 con 0x55bd721331b0 2024-09-21T03:51:14.271 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:14.262+0000 7fdd3effd640 1 -- 172.21.15.39:0/3824799064 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd480cd2c0 con 0x55bd721331b0 2024-09-21T03:51:14.272 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:14.263+0000 7fdd61495740 1 -- 172.21.15.39:0/3824799064 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55bd721331b0 msgr2=0x55bd72503800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:51:14.272 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:14.263+0000 7fdd61495740 1 --2- 172.21.15.39:0/3824799064 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55bd721331b0 0x55bd72503800 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7fdd481053c0 tx=0x7fdd48110450 comp rx=0 tx=0).stop 2024-09-21T03:51:14.272 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:14.263+0000 7fdd594be640 1 --2- 172.21.15.39:0/2196291557 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7fdd440b6680 0x7fdd440b8b90 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fdd4c005fd0 tx=0x7fdd4c0098a0 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:51:14.272 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:14.263+0000 7fdd61495740 1 -- 172.21.15.39:0/3824799064 shutdown_connections 2024-09-21T03:51:14.272 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:14.263+0000 7fdd61495740 1 --2- 172.21.15.39:0/3824799064 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55bd7213b390 0x55bd72503d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:14.272 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:14.263+0000 7fdd61495740 1 --2- 172.21.15.39:0/3824799064 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55bd721331b0 0x55bd72503800 unknown :-1 s=CLOSED pgs=844 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:14.272 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:14.263+0000 7fdd61495740 1 --2- 172.21.15.39:0/3824799064 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55bd7215c070 0x55bd725032c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:14.272 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:14.263+0000 7fdd61495740 1 -- 172.21.15.39:0/3824799064 >> 172.21.15.39:0/3824799064 conn(0x55bd72150460 msgr2=0x55bd725011b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:51:14.272 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:14.263+0000 7fdd61495740 1 -- 172.21.15.39:0/3824799064 shutdown_connections 2024-09-21T03:51:14.272 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:14.264+0000 7fdd61495740 1 -- 172.21.15.39:0/3824799064 wait complete. 2024-09-21T03:51:14.272 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:14.264+0000 7fdd61495740 1 -- start start 2024-09-21T03:51:14.272 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:14.264+0000 7fdd61495740 1 Processor -- start 2024-09-21T03:51:14.272 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:14.265+0000 7fdd61495740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55bd721331b0 0x55bd7250e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:51:14.272 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:14.265+0000 7fdd61495740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55bd7213b390 0x55bd7250eee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:51:14.273 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:14.265+0000 7fdd61495740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55bd7215c070 0x55bd7251eed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:51:14.273 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:14.265+0000 7fdd61495740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x55bd722ca700 con 0x55bd7213b390 2024-09-21T03:51:14.273 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:14.265+0000 7fdd61495740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x55bd72404160 con 0x55bd721331b0 2024-09-21T03:51:14.273 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:14.265+0000 7fdd61495740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55bd72515c70 con 0x55bd7215c070 2024-09-21T03:51:14.273 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:14.265+0000 7fdd53fff640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55bd7213b390 0x55bd7250eee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:51:14.273 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:14.265+0000 7fdd53fff640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55bd7213b390 0x55bd7250eee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:60544/0 (socket says 172.21.15.39:60544) 2024-09-21T03:51:14.273 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:14.265+0000 7fdd53fff640 1 -- 172.21.15.39:0/1612549087 learned_addr learned my addr 172.21.15.39:0/1612549087 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:51:14.273 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:14.265+0000 7fdd61495740 10 client.0 _ll_register_callbacks cb 0x55bd7241d9e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:51:14.273 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:14.265+0000 7fdd594be640 1 --2- 172.21.15.39:0/1612549087 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55bd7215c070 0x55bd7251eed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:51:14.273 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:14.265+0000 7fdd58cbd640 1 --2- 172.21.15.39:0/1612549087 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55bd721331b0 0x55bd7250e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:51:14.273 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:14.265+0000 7fdd53fff640 1 -- 172.21.15.39:0/1612549087 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55bd7215c070 msgr2=0x55bd7251eed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:51:14.273 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:14.265+0000 7fdd53fff640 1 --2- 172.21.15.39:0/1612549087 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55bd7215c070 0x55bd7251eed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:14.273 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:14.265+0000 7fdd53fff640 1 -- 172.21.15.39:0/1612549087 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55bd721331b0 msgr2=0x55bd7250e9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:51:14.274 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:14.265+0000 7fdd53fff640 1 --2- 172.21.15.39:0/1612549087 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55bd721331b0 0x55bd7250e9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:14.274 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:14.265+0000 7fdd53fff640 1 -- 172.21.15.39:0/1612549087 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bd72431c90 con 0x55bd7213b390 2024-09-21T03:51:14.274 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:14.266+0000 7fdd53fff640 1 --2- 172.21.15.39:0/1612549087 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55bd7213b390 0x55bd7250eee0 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7fdd4000e320 tx=0x7fdd4000e7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff751d9c46057ce9 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:51:14.274 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:14.266+0000 7fdd33fff640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-09-21T03:51:14.274 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:14.266+0000 7fdd33fff640 1 -- 172.21.15.39:0/1612549087 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd40016070 con 0x55bd7213b390 2024-09-21T03:51:14.274 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:14.266+0000 7fdd33fff640 1 -- 172.21.15.39:0/1612549087 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8484+0+0 (secure 0 0 0) 0x7fdd400108a0 con 0x55bd7213b390 2024-09-21T03:51:14.274 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:14.266+0000 7fdd61495740 1 -- 172.21.15.39:0/1612549087 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdd480cd660 con 0x55bd7213b390 2024-09-21T03:51:14.274 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:14.266+0000 7fdd33fff640 1 -- 172.21.15.39:0/1612549087 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fdd40010aa0 con 0x55bd7213b390 2024-09-21T03:51:14.274 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:14.266+0000 7fdd61495740 10 client.16806 fetch_fsmap learned FSMap version 29 2024-09-21T03:51:14.274 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:14.266+0000 7fdd61495740 1 -- 172.21.15.39:0/1612549087 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55bd72521910 con 0x55bd7213b390 2024-09-21T03:51:14.274 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:14.267+0000 7fdd33fff640 1 -- 172.21.15.39:0/1612549087 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdd480cd660 con 0x55bd7213b390 2024-09-21T03:51:14.275 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:14.267+0000 7fdd33fff640 1 -- 172.21.15.39:0/1612549087 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fdd4000f040 con 0x55bd7213b390 2024-09-21T03:51:14.275 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:14.267+0000 7fdd61495740 10 client.16806 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T03:51:14.275 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:14.267+0000 7fdd61495740 10 client.16806 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-21T03:51:14.275 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:14.267+0000 7fdd61495740 20 client.16806 populate_metadata read hostname 'smithi039' 2024-09-21T03:51:14.275 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:14.267+0000 7fdd61495740 1 --2- 172.21.15.39:0/1612549087 >> [v2:172.21.15.161:6838/2190107270,v1:172.21.15.161:6839/2190107270] conn(0x55bd7211a9f0 0x55bd72524da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:51:14.275 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:14.267+0000 7fdd61495740 4 client.16806 mds_command: new command op to 24667 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:51:14.275 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:14.267+0000 7fdd61495740 1 -- 172.21.15.39:0/1612549087 --> [v2:172.21.15.161:6838/2190107270,v1:172.21.15.161:6839/2190107270] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bd7247bdf0 con 0x55bd7211a9f0 2024-09-21T03:51:14.275 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:14.267+0000 7fdd58cbd640 1 --2- 172.21.15.39:0/1612549087 >> [v2:172.21.15.161:6838/2190107270,v1:172.21.15.161:6839/2190107270] conn(0x55bd7211a9f0 0x55bd72524da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:51:14.276 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:14.268+0000 7fdd33fff640 10 client.16806 ms_handle_connect on v2:172.21.15.161:6838/2190107270 2024-09-21T03:51:14.276 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:14.268+0000 7fdd58cbd640 1 --2- 172.21.15.39:0/1612549087 >> [v2:172.21.15.161:6838/2190107270,v1:172.21.15.161:6839/2190107270] conn(0x55bd7211a9f0 0x55bd72524da0 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=77949c739d0616ca server_cookie=1f73262183cccdbe in_seq=0 out_seq=0 2024-09-21T03:51:14.276 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:14.268+0000 7fdd33fff640 1 -- 172.21.15.39:0/1612549087 <== mds.1 v2:172.21.15.161:6838/2190107270 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55bd7247bdf0 con 0x55bd7211a9f0 2024-09-21T03:51:14.277 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:14.268+0000 7fdd33fff640 10 client.16806 handle_command_reply: tid=0 2024-09-21T03:51:14.277 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:14.268+0000 7fdd61495740 1 client.16806 shutdown 2024-09-21T03:51:14.277 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:14.269+0000 7fdd61495740 1 -- 172.21.15.39:0/1612549087 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55bd7213b390 msgr2=0x55bd7250eee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:51:14.277 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:14.269+0000 7fdd61495740 1 --2- 172.21.15.39:0/1612549087 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55bd7213b390 0x55bd7250eee0 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7fdd4000e320 tx=0x7fdd4000e7e0 comp rx=0 tx=0).stop 2024-09-21T03:51:14.277 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:14.269+0000 7fdd61495740 1 -- 172.21.15.39:0/1612549087 shutdown_connections 2024-09-21T03:51:14.277 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:14.269+0000 7fdd61495740 1 --2- 172.21.15.39:0/1612549087 >> [v2:172.21.15.161:6838/2190107270,v1:172.21.15.161:6839/2190107270] conn(0x55bd7211a9f0 0x55bd72524da0 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:14.277 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:14.269+0000 7fdd61495740 1 --2- 172.21.15.39:0/1612549087 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55bd7215c070 0x55bd7251eed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:14.277 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:14.269+0000 7fdd61495740 1 --2- 172.21.15.39:0/1612549087 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55bd7213b390 0x55bd7250eee0 unknown :-1 s=CLOSED pgs=845 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:14.277 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:14.269+0000 7fdd33fff640 0 client.16806 ms_handle_reset on v2:172.21.15.161:6838/2190107270 2024-09-21T03:51:14.277 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:14.269+0000 7fdd61495740 1 --2- 172.21.15.39:0/1612549087 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55bd721331b0 0x55bd7250e9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:14.277 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:14.269+0000 7fdd61495740 1 -- 172.21.15.39:0/1612549087 >> 172.21.15.39:0/1612549087 conn(0x55bd72150460 msgr2=0x55bd725042f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:51:14.277 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:14.269+0000 7fdd61495740 1 -- 172.21.15.39:0/1612549087 shutdown_connections 2024-09-21T03:51:14.277 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:14.269+0000 7fdd61495740 1 -- 172.21.15.39:0/1612549087 wait complete. 2024-09-21T03:51:14.278 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:14.269+0000 7fdd61495740 20 client.16806 trim_cache size 0 max 16384 2024-09-21T03:51:14.286 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:14.282+0000 7fdd61495740 1 Processor -- start 2024-09-21T03:51:14.286 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:14.282+0000 7fdd61495740 1 -- start start 2024-09-21T03:51:14.286 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:14.283+0000 7fdd61495740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55bd721331b0 0x55bd72504960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:51:14.286 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:14.283+0000 7fdd61495740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55bd7213b390 0x55bd7250ee60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:51:14.286 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:14.283+0000 7fdd61495740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55bd7215c070 0x55bd7251ed90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:51:14.286 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:14.283+0000 7fdd61495740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fdd541ad510 con 0x55bd721331b0 2024-09-21T03:51:14.287 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:14.283+0000 7fdd61495740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fdd541af050 con 0x55bd7213b390 2024-09-21T03:51:14.287 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:14.283+0000 7fdd61495740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fdd541adbe0 con 0x55bd7215c070 2024-09-21T03:51:14.287 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:14.283+0000 7fdd58cbd640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55bd721331b0 0x55bd72504960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:51:14.287 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:14.283+0000 7fdd58cbd640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55bd721331b0 0x55bd72504960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:60552/0 (socket says 172.21.15.39:60552) 2024-09-21T03:51:14.287 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:14.283+0000 7fdd58cbd640 1 -- 172.21.15.39:0/3516491089 learned_addr learned my addr 172.21.15.39:0/3516491089 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:51:14.287 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:14.283+0000 7fdd594be640 1 --2- 172.21.15.39:0/3516491089 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55bd7215c070 0x55bd7251ed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:51:14.287 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:14.283+0000 7fdd53fff640 1 --2- 172.21.15.39:0/3516491089 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55bd7213b390 0x55bd7250ee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:51:14.287 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:14.283+0000 7fdd58cbd640 1 -- 172.21.15.39:0/3516491089 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55bd7215c070 msgr2=0x55bd7251ed90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:51:14.287 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:14.283+0000 7fdd58cbd640 1 --2- 172.21.15.39:0/3516491089 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55bd7215c070 0x55bd7251ed90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:14.287 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:14.283+0000 7fdd58cbd640 1 -- 172.21.15.39:0/3516491089 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55bd7213b390 msgr2=0x55bd7250ee60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:51:14.287 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:14.283+0000 7fdd58cbd640 1 --2- 172.21.15.39:0/3516491089 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55bd7213b390 0x55bd7250ee60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:14.287 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:14.283+0000 7fdd58cbd640 1 -- 172.21.15.39:0/3516491089 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bd725114d0 con 0x55bd721331b0 2024-09-21T03:51:14.287 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:14.283+0000 7fdd53fff640 1 --2- 172.21.15.39:0/3516491089 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55bd7213b390 0x55bd7250ee60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:51:14.288 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:14.284+0000 7fdd58cbd640 1 --2- 172.21.15.39:0/3516491089 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55bd721331b0 0x55bd72504960 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7fdd48032af0 tx=0x7fdd48009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c9af3562f4fa76fc server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:51:14.290 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:14.284+0000 7fdd3e7fc640 1 -- 172.21.15.39:0/3516491089 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd48049040 con 0x55bd721331b0 2024-09-21T03:51:14.290 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:14.284+0000 7fdd3e7fc640 1 -- 172.21.15.39:0/3516491089 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdd480cd660 con 0x55bd721331b0 2024-09-21T03:51:14.290 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:14.284+0000 7fdd3e7fc640 1 -- 172.21.15.39:0/3516491089 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd480cd4a0 con 0x55bd721331b0 2024-09-21T03:51:14.290 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:14.285+0000 7fdd61495740 1 -- 172.21.15.39:0/3516491089 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55bd721331b0 msgr2=0x55bd72504960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:51:14.290 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:14.285+0000 7fdd61495740 1 --2- 172.21.15.39:0/3516491089 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55bd721331b0 0x55bd72504960 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7fdd48032af0 tx=0x7fdd48009260 comp rx=0 tx=0).stop 2024-09-21T03:51:14.290 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:14.285+0000 7fdd61495740 1 -- 172.21.15.39:0/3516491089 shutdown_connections 2024-09-21T03:51:14.291 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:14.285+0000 7fdd61495740 1 --2- 172.21.15.39:0/3516491089 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55bd7215c070 0x55bd7251ed90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:14.291 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:14.285+0000 7fdd61495740 1 --2- 172.21.15.39:0/3516491089 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55bd7213b390 0x55bd7250ee60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:14.291 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:14.285+0000 7fdd61495740 1 --2- 172.21.15.39:0/3516491089 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55bd721331b0 0x55bd72504960 unknown :-1 s=CLOSED pgs=846 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:14.291 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:14.285+0000 7fdd61495740 1 -- 172.21.15.39:0/3516491089 >> 172.21.15.39:0/3516491089 conn(0x55bd72150460 msgr2=0x55bd7250e750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:51:14.291 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:14.285+0000 7fdd61495740 1 -- 172.21.15.39:0/3516491089 shutdown_connections 2024-09-21T03:51:14.291 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:14.285+0000 7fdd61495740 1 -- 172.21.15.39:0/3516491089 wait complete. 2024-09-21T03:51:14.291 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:14.285+0000 7fdd61495740 1 -- start start 2024-09-21T03:51:14.291 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:14.285+0000 7fdd61495740 1 Processor -- start 2024-09-21T03:51:14.291 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:14.286+0000 7fdd61495740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55bd721331b0 0x55bd7251b390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:51:14.291 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:14.286+0000 7fdd61495740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55bd7213b390 0x55bd72516840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:51:14.291 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:14.286+0000 7fdd61495740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55bd7215c070 0x55bd72516d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:51:14.291 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:14.286+0000 7fdd61495740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fdd541ae980 con 0x55bd7215c070 2024-09-21T03:51:14.291 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:14.286+0000 7fdd61495740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fdd541ae2b0 con 0x55bd7213b390 2024-09-21T03:51:14.292 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:14.286+0000 7fdd61495740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fdd541ab4f0 con 0x55bd721331b0 2024-09-21T03:51:14.292 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:14.286+0000 7fdd61495740 10 client.0 _ll_register_callbacks cb 0x7fdd400092a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:51:14.292 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:14.286+0000 7fdd594be640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55bd7215c070 0x55bd72516d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:51:14.292 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:14.286+0000 7fdd594be640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55bd7215c070 0x55bd72516d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:60568/0 (socket says 172.21.15.39:60568) 2024-09-21T03:51:14.292 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:14.286+0000 7fdd594be640 1 -- 172.21.15.39:0/2218334068 learned_addr learned my addr 172.21.15.39:0/2218334068 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:51:14.292 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:14.286+0000 7fdd53fff640 1 --2- 172.21.15.39:0/2218334068 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55bd7213b390 0x55bd72516840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:51:14.292 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:14.286+0000 7fdd58cbd640 1 --2- 172.21.15.39:0/2218334068 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55bd721331b0 0x55bd7251b390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:51:14.292 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:14.286+0000 7fdd594be640 1 -- 172.21.15.39:0/2218334068 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55bd721331b0 msgr2=0x55bd7251b390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:51:14.292 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:14.286+0000 7fdd594be640 1 --2- 172.21.15.39:0/2218334068 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55bd721331b0 0x55bd7251b390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:14.292 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:14.286+0000 7fdd594be640 1 -- 172.21.15.39:0/2218334068 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55bd7213b390 msgr2=0x55bd72516840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:51:14.292 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:14.286+0000 7fdd594be640 1 --2- 172.21.15.39:0/2218334068 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55bd7213b390 0x55bd72516840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:14.292 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:14.286+0000 7fdd594be640 1 -- 172.21.15.39:0/2218334068 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bd725172c0 con 0x55bd7215c070 2024-09-21T03:51:14.292 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:14.287+0000 7fdd53fff640 1 --2- 172.21.15.39:0/2218334068 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55bd7213b390 0x55bd72516840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:51:14.292 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:14.287+0000 7fdd594be640 1 --2- 172.21.15.39:0/2218334068 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55bd7215c070 0x55bd72516d80 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7fdd4c010d20 tx=0x7fdd4c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=819886e68324567 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:51:14.293 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:14.287+0000 7fdd33fff640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-09-21T03:51:14.293 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:14.287+0000 7fdd33fff640 1 -- 172.21.15.39:0/2218334068 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd4c02e060 con 0x55bd7215c070 2024-09-21T03:51:14.293 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:14.287+0000 7fdd33fff640 1 -- 172.21.15.39:0/2218334068 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8484+0+0 (secure 0 0 0) 0x7fdd4c01fcb0 con 0x55bd7215c070 2024-09-21T03:51:14.293 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:14.287+0000 7fdd61495740 1 -- 172.21.15.39:0/2218334068 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdd480cd660 con 0x55bd7215c070 2024-09-21T03:51:14.293 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:14.287+0000 7fdd33fff640 1 -- 172.21.15.39:0/2218334068 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fdd4c01c400 con 0x55bd7215c070 2024-09-21T03:51:14.293 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:14.287+0000 7fdd61495740 10 client.16812 fetch_fsmap learned FSMap version 29 2024-09-21T03:51:14.293 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:14.287+0000 7fdd61495740 1 -- 172.21.15.39:0/2218334068 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55bd7258f450 con 0x55bd7215c070 2024-09-21T03:51:14.293 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:14.287+0000 7fdd33fff640 1 -- 172.21.15.39:0/2218334068 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdd480cd660 con 0x55bd7215c070 2024-09-21T03:51:14.294 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:14.288+0000 7fdd33fff640 1 -- 172.21.15.39:0/2218334068 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fdd4c028070 con 0x55bd7215c070 2024-09-21T03:51:14.294 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:14.288+0000 7fdd61495740 10 client.16812 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T03:51:14.294 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:14.288+0000 7fdd61495740 10 client.16812 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-21T03:51:14.294 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:14.288+0000 7fdd61495740 20 client.16812 populate_metadata read hostname 'smithi039' 2024-09-21T03:51:14.295 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:14.288+0000 7fdd61495740 1 --2- 172.21.15.39:0/2218334068 >> [v2:172.21.15.161:6838/2190107270,v1:172.21.15.161:6839/2190107270] conn(0x55bd720efda0 0x55bd72591c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:51:14.295 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:14.288+0000 7fdd61495740 4 client.16812 mds_command: new command op to 24667 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T03:51:14.295 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:14.288+0000 7fdd61495740 1 -- 172.21.15.39:0/2218334068 --> [v2:172.21.15.161:6838/2190107270,v1:172.21.15.161:6839/2190107270] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55bd724778a0 con 0x55bd720efda0 2024-09-21T03:51:14.295 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:14.288+0000 7fdd58cbd640 1 --2- 172.21.15.39:0/2218334068 >> [v2:172.21.15.161:6838/2190107270,v1:172.21.15.161:6839/2190107270] conn(0x55bd720efda0 0x55bd72591c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:51:14.295 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:14.288+0000 7fdd33fff640 10 client.16812 ms_handle_connect on v2:172.21.15.161:6838/2190107270 2024-09-21T03:51:14.295 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:14.288+0000 7fdd58cbd640 1 --2- 172.21.15.39:0/2218334068 >> [v2:172.21.15.161:6838/2190107270,v1:172.21.15.161:6839/2190107270] conn(0x55bd720efda0 0x55bd72591c70 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=d0f569d87c1dc8e9 server_cookie=9017ea6de3e26d25 in_seq=0 out_seq=0 2024-09-21T03:51:14.295 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:14.289+0000 7fdd33fff640 1 -- 172.21.15.39:0/2218334068 <== mds.1 v2:172.21.15.161:6838/2190107270 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55bd724778a0 con 0x55bd720efda0 2024-09-21T03:51:14.295 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:14.289+0000 7fdd33fff640 10 client.16812 handle_command_reply: tid=0 2024-09-21T03:51:14.295 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:14.289+0000 7fdd61495740 1 client.16812 shutdown 2024-09-21T03:51:14.295 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:14.289+0000 7fdd61495740 1 -- 172.21.15.39:0/2218334068 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55bd7215c070 msgr2=0x55bd72516d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:51:14.295 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:14.289+0000 7fdd61495740 1 --2- 172.21.15.39:0/2218334068 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55bd7215c070 0x55bd72516d80 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7fdd4c010d20 tx=0x7fdd4c004a40 comp rx=0 tx=0).stop 2024-09-21T03:51:14.295 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:14.289+0000 7fdd61495740 1 -- 172.21.15.39:0/2218334068 shutdown_connections 2024-09-21T03:51:14.295 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:14.289+0000 7fdd61495740 1 --2- 172.21.15.39:0/2218334068 >> [v2:172.21.15.161:6838/2190107270,v1:172.21.15.161:6839/2190107270] conn(0x55bd720efda0 0x55bd72591c70 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:14.295 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:14.289+0000 7fdd61495740 1 --2- 172.21.15.39:0/2218334068 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55bd7215c070 0x55bd72516d80 unknown :-1 s=CLOSED pgs=847 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:14.296 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:14.289+0000 7fdd61495740 1 --2- 172.21.15.39:0/2218334068 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55bd7213b390 0x55bd72516840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:14.296 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:14.289+0000 7fdd61495740 1 --2- 172.21.15.39:0/2218334068 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55bd721331b0 0x55bd7251b390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:14.296 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:14.289+0000 7fdd33fff640 0 client.16812 ms_handle_reset on v2:172.21.15.161:6838/2190107270 2024-09-21T03:51:14.296 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:14.289+0000 7fdd61495740 1 -- 172.21.15.39:0/2218334068 >> 172.21.15.39:0/2218334068 conn(0x55bd72150460 msgr2=0x55bd7250cee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:51:14.296 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:14.289+0000 7fdd61495740 1 -- 172.21.15.39:0/2218334068 shutdown_connections 2024-09-21T03:51:14.296 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:14.289+0000 7fdd61495740 1 -- 172.21.15.39:0/2218334068 wait complete. 2024-09-21T03:51:14.296 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:14.289+0000 7fdd61495740 20 client.16812 trim_cache size 0 max 16384 2024-09-21T03:51:14.296 INFO:teuthology.orchestra.run.smithi039.stdout:[] 2024-09-21T03:51:14.297 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:14.290+0000 7fdd5af21640 1 -- 172.21.15.39:0/2196291557 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7fdd440b6680 msgr2=0x7fdd440b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:51:14.297 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:14.290+0000 7fdd5af21640 1 --2- 172.21.15.39:0/2196291557 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7fdd440b6680 0x7fdd440b8b90 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fdd4c005fd0 tx=0x7fdd4c0098a0 comp rx=0 tx=0).stop 2024-09-21T03:51:14.297 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:14.290+0000 7fdd5af21640 1 -- 172.21.15.39:0/2196291557 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdd541049b0 msgr2=0x7fdd5419a4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:51:14.297 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:14.290+0000 7fdd5af21640 1 --2- 172.21.15.39:0/2196291557 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdd541049b0 0x7fdd5419a4e0 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7fdd48032fd0 tx=0x7fdd48004270 comp rx=0 tx=0).stop 2024-09-21T03:51:14.297 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:14.290+0000 7fdd5af21640 1 -- 172.21.15.39:0/2196291557 shutdown_connections 2024-09-21T03:51:14.297 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:14.290+0000 7fdd5af21640 1 --2- 172.21.15.39:0/2196291557 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7fdd440b6680 0x7fdd440b8b90 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:14.297 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:14.290+0000 7fdd5af21640 1 --2- 172.21.15.39:0/2196291557 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdd5410de10 0x7fdd5419fd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:14.297 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:14.290+0000 7fdd5af21640 1 --2- 172.21.15.39:0/2196291557 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdd54105300 0x7fdd5419aa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:14.298 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:14.290+0000 7fdd5af21640 1 --2- 172.21.15.39:0/2196291557 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdd541049b0 0x7fdd5419a4e0 unknown :-1 s=CLOSED pgs=843 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:14.298 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:14.290+0000 7fdd5af21640 1 -- 172.21.15.39:0/2196291557 >> 172.21.15.39:0/2196291557 conn(0x7fdd54079730 msgr2=0x7fdd54100880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:51:14.298 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:14.290+0000 7fdd5af21640 1 -- 172.21.15.39:0/2196291557 shutdown_connections 2024-09-21T03:51:14.298 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:14.290+0000 7fdd5af21640 1 -- 172.21.15.39:0/2196291557 wait complete. 2024-09-21T03:51:15.104 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 51116d5c-77ca-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 damage ls 2024-09-21T03:51:15.334 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:51:15 smithi039 ceph-mon[31211]: pgmap v481: 97 pgs: 97 active+clean; 10 MiB data, 520 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 25 op/s 2024-09-21T03:51:15.365 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/51116d5c-77ca-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:51:15.468 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:51:15 smithi161 ceph-mon[34533]: pgmap v481: 97 pgs: 97 active+clean; 10 MiB data, 520 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 25 op/s 2024-09-21T03:51:15.475 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:51:15 smithi150 ceph-mon[34289]: pgmap v481: 97 pgs: 97 active+clean; 10 MiB data, 520 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 25 op/s 2024-09-21T03:51:15.896 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:15.871+0000 7f55a5bdf640 1 -- 172.21.15.39:0/1791642740 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f55a01094a0 msgr2=0x7f55a01098b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:51:15.896 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:15.871+0000 7f55a5bdf640 1 --2- 172.21.15.39:0/1791642740 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f55a01094a0 0x7f55a01098b0 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7f5594009900 tx=0x7f559402a410 comp rx=0 tx=0).stop 2024-09-21T03:51:15.897 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:15.871+0000 7f55a5bdf640 1 -- 172.21.15.39:0/1791642740 shutdown_connections 2024-09-21T03:51:15.897 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:15.871+0000 7f55a5bdf640 1 --2- 172.21.15.39:0/1791642740 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f55a0109df0 0x7f55a0110620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:15.897 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:15.871+0000 7f55a5bdf640 1 --2- 172.21.15.39:0/1791642740 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f55a01094a0 0x7f55a01098b0 unknown :-1 s=CLOSED pgs=848 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:15.897 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:15.872+0000 7f55a5bdf640 1 --2- 172.21.15.39:0/1791642740 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f55a0103b60 0x7f55a0103f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:15.897 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:15.872+0000 7f55a5bdf640 1 -- 172.21.15.39:0/1791642740 >> 172.21.15.39:0/1791642740 conn(0x7f55a007b850 msgr2=0x7f55a007bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:51:15.897 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:15.872+0000 7f55a5bdf640 1 -- 172.21.15.39:0/1791642740 shutdown_connections 2024-09-21T03:51:15.897 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:15.872+0000 7f55a5bdf640 1 -- 172.21.15.39:0/1791642740 wait complete. 2024-09-21T03:51:15.897 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:15.872+0000 7f55a5bdf640 1 Processor -- start 2024-09-21T03:51:15.897 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:15.872+0000 7f55a5bdf640 1 -- start start 2024-09-21T03:51:15.898 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:15.873+0000 7f55a5bdf640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f55a0103b60 0x7f55a019a560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:51:15.898 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:15.873+0000 7f55a5bdf640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f55a01094a0 0x7f55a019ab00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:51:15.898 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:15.873+0000 7f55a5bdf640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f55a0109df0 0x7f55a019fda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:51:15.898 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:15.873+0000 7f55a5bdf640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f55a0113ab0 con 0x7f55a01094a0 2024-09-21T03:51:15.898 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:15.873+0000 7f55a5bdf640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f55a0113930 con 0x7f55a0103b60 2024-09-21T03:51:15.898 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:15.873+0000 7f55a5bdf640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f55a0113c30 con 0x7f55a0109df0 2024-09-21T03:51:15.898 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:15.873+0000 7f559ffff640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f55a0109df0 0x7f55a019fda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:51:15.898 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:15.873+0000 7f559effd640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f55a01094a0 0x7f55a019ab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:51:15.898 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:15.873+0000 7f559ffff640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f55a0109df0 0x7f55a019fda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.39:39236/0 (socket says 172.21.15.39:39236) 2024-09-21T03:51:15.898 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:15.873+0000 7f559effd640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f55a01094a0 0x7f55a019ab00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:60596/0 (socket says 172.21.15.39:60596) 2024-09-21T03:51:15.898 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:15.873+0000 7f559ffff640 1 -- 172.21.15.39:0/2754062349 learned_addr learned my addr 172.21.15.39:0/2754062349 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:51:15.899 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:15.873+0000 7f559f7fe640 1 --2- 172.21.15.39:0/2754062349 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f55a0103b60 0x7f55a019a560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:51:15.899 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:15.873+0000 7f559effd640 1 -- 172.21.15.39:0/2754062349 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f55a0109df0 msgr2=0x7f55a019fda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:51:15.899 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:15.873+0000 7f559effd640 1 --2- 172.21.15.39:0/2754062349 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f55a0109df0 0x7f55a019fda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:15.899 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:15.873+0000 7f559effd640 1 -- 172.21.15.39:0/2754062349 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f55a0103b60 msgr2=0x7f55a019a560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:51:15.899 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:15.873+0000 7f559effd640 1 --2- 172.21.15.39:0/2754062349 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f55a0103b60 0x7f55a019a560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:15.899 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:15.873+0000 7f559effd640 1 -- 172.21.15.39:0/2754062349 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f55a01a04d0 con 0x7f55a01094a0 2024-09-21T03:51:15.899 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:15.873+0000 7f559ffff640 1 --2- 172.21.15.39:0/2754062349 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f55a0109df0 0x7f55a019fda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:51:15.899 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:15.874+0000 7f559effd640 1 --2- 172.21.15.39:0/2754062349 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f55a01094a0 0x7f55a019ab00 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7f559402a930 tx=0x7f5594004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:51:15.900 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:15.874+0000 7f559cff9640 1 -- 172.21.15.39:0/2754062349 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f559403f070 con 0x7f55a01094a0 2024-09-21T03:51:15.900 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:15.874+0000 7f55a5bdf640 1 -- 172.21.15.39:0/2754062349 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f55a01a0760 con 0x7f55a01094a0 2024-09-21T03:51:15.900 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:15.874+0000 7f559cff9640 1 -- 172.21.15.39:0/2754062349 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5594003ca0 con 0x7f55a01094a0 2024-09-21T03:51:15.900 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:15.874+0000 7f559cff9640 1 -- 172.21.15.39:0/2754062349 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5594035070 con 0x7f55a01094a0 2024-09-21T03:51:15.900 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:15.874+0000 7f55a5bdf640 1 -- 172.21.15.39:0/2754062349 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f55a01a6f20 con 0x7f55a01094a0 2024-09-21T03:51:15.900 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:15.877+0000 7f55ac153740 1 Processor -- start 2024-09-21T03:51:15.900 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:15.877+0000 7f55ac153740 1 -- start start 2024-09-21T03:51:15.900 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:15.878+0000 7f55ac153740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556088929070 0x556088cd06a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:51:15.900 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:15.878+0000 7f55ac153740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5560889001b0 0x556088cd0be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:51:15.900 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:15.878+0000 7f55ac153740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556088908390 0x556088cd1120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:51:15.900 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:15.878+0000 7f55ac153740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x556088c46ea0 con 0x556088908390 2024-09-21T03:51:15.901 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:15.878+0000 7f55ac153740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x556088c362d0 con 0x556088929070 2024-09-21T03:51:15.901 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:15.878+0000 7f55ac153740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x556088c3a0e0 con 0x5560889001b0 2024-09-21T03:51:15.901 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:15.878+0000 7f559cff9640 1 -- 172.21.15.39:0/2754062349 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 27) ==== 154643+0+0 (secure 0 0 0) 0x7f559403b4f0 con 0x7f55a01094a0 2024-09-21T03:51:15.901 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:15.878+0000 7f559f7fe640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5560889001b0 0x556088cd0be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:51:15.901 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:15.878+0000 7f559effd640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556088908390 0x556088cd1120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:51:15.901 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:15.878+0000 7f559f7fe640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5560889001b0 0x556088cd0be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.39:39240/0 (socket says 172.21.15.39:39240) 2024-09-21T03:51:15.901 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:15.878+0000 7f559f7fe640 1 -- 172.21.15.39:0/2327508047 learned_addr learned my addr 172.21.15.39:0/2327508047 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:51:15.901 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:15.878+0000 7f559ffff640 1 --2- 172.21.15.39:0/2327508047 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556088929070 0x556088cd06a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:51:15.902 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:15.878+0000 7f559f7fe640 1 -- 172.21.15.39:0/2327508047 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556088929070 msgr2=0x556088cd06a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:51:15.902 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:15.878+0000 7f559f7fe640 1 --2- 172.21.15.39:0/2327508047 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556088929070 0x556088cd06a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:15.902 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:15.878+0000 7f559f7fe640 1 -- 172.21.15.39:0/2327508047 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556088908390 msgr2=0x556088cd1120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:51:15.902 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:15.878+0000 7f559f7fe640 1 --2- 172.21.15.39:0/2327508047 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556088908390 0x556088cd1120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:15.902 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:15.878+0000 7f559f7fe640 1 -- 172.21.15.39:0/2327508047 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556088994650 con 0x5560889001b0 2024-09-21T03:51:15.902 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:15.879+0000 7f559f7fe640 1 --2- 172.21.15.39:0/2327508047 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5560889001b0 0x556088cd0be0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f559000ac20 tx=0x7f559000f790 comp rx=0 tx=0).ready entity=mon.1 client_cookie=48491b6a4e2a7b62 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:51:15.902 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:15.879+0000 7f559cff9640 1 --2- 172.21.15.39:0/2754062349 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f55880b6680 0x7f55880b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:51:15.902 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:15.879+0000 7f5585ffb640 1 -- 172.21.15.39:0/2327508047 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f559001d070 con 0x5560889001b0 2024-09-21T03:51:15.902 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:15.879+0000 7f5585ffb640 1 -- 172.21.15.39:0/2327508047 <== mon.1 v2:172.21.15.161:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5590018af0 con 0x5560889001b0 2024-09-21T03:51:15.903 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:15.879+0000 7f5585ffb640 1 -- 172.21.15.39:0/2327508047 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5590010ab0 con 0x5560889001b0 2024-09-21T03:51:15.903 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:15.879+0000 7f559ffff640 1 --2- 172.21.15.39:0/2754062349 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f55880b6680 0x7f55880b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:51:15.903 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:15.879+0000 7f559cff9640 1 -- 172.21.15.39:0/2754062349 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8484+0+0 (secure 0 0 0) 0x7f5594107ee0 con 0x7f55a01094a0 2024-09-21T03:51:15.903 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:15.880+0000 7f55ac153740 1 -- 172.21.15.39:0/2327508047 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5560889001b0 msgr2=0x556088cd0be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:51:15.903 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:15.880+0000 7f559ffff640 1 --2- 172.21.15.39:0/2754062349 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f55880b6680 0x7f55880b8b90 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f5598009270 tx=0x7f5598009680 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:51:15.903 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:15.880+0000 7f55ac153740 1 --2- 172.21.15.39:0/2327508047 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5560889001b0 0x556088cd0be0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f559000ac20 tx=0x7f559000f790 comp rx=0 tx=0).stop 2024-09-21T03:51:15.903 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:15.880+0000 7f55ac153740 1 -- 172.21.15.39:0/2327508047 shutdown_connections 2024-09-21T03:51:15.903 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:15.880+0000 7f55ac153740 1 --2- 172.21.15.39:0/2327508047 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556088908390 0x556088cd1120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:15.903 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:15.880+0000 7f55ac153740 1 --2- 172.21.15.39:0/2327508047 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5560889001b0 0x556088cd0be0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:15.903 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:15.880+0000 7f55ac153740 1 --2- 172.21.15.39:0/2327508047 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556088929070 0x556088cd06a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:15.903 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:15.880+0000 7f55ac153740 1 -- 172.21.15.39:0/2327508047 >> 172.21.15.39:0/2327508047 conn(0x55608891d460 msgr2=0x556088cce590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:51:15.904 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:15.880+0000 7f55ac153740 1 -- 172.21.15.39:0/2327508047 shutdown_connections 2024-09-21T03:51:15.904 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:15.880+0000 7f55ac153740 1 -- 172.21.15.39:0/2327508047 wait complete. 2024-09-21T03:51:15.904 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:15.880+0000 7f55ac153740 1 -- start start 2024-09-21T03:51:15.904 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:15.881+0000 7f55ac153740 1 Processor -- start 2024-09-21T03:51:15.904 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:15.881+0000 7f55ac153740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5560889001b0 0x556088cdbd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:51:15.904 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:15.881+0000 7f55ac153740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556088908390 0x556088cdc2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:51:15.904 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:15.881+0000 7f55ac153740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556088929070 0x556088ce8170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:51:15.904 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:15.881+0000 7f55ac153740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x556088a97700 con 0x556088908390 2024-09-21T03:51:15.904 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:15.881+0000 7f55ac153740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x556088bd1160 con 0x556088929070 2024-09-21T03:51:15.904 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:15.881+0000 7f55ac153740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x556088cdee80 con 0x5560889001b0 2024-09-21T03:51:15.904 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:15.881+0000 7f55ac153740 10 client.0 _ll_register_callbacks cb 0x556088bea9e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:51:15.905 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:15.881+0000 7f559effd640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556088908390 0x556088cdc2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:51:15.905 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:15.881+0000 7f559effd640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556088908390 0x556088cdc2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:60614/0 (socket says 172.21.15.39:60614) 2024-09-21T03:51:15.905 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:15.881+0000 7f559effd640 1 -- 172.21.15.39:0/3678970005 learned_addr learned my addr 172.21.15.39:0/3678970005 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:51:15.905 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:15.881+0000 7f559ffff640 1 --2- 172.21.15.39:0/3678970005 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556088929070 0x556088ce8170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:51:15.905 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:15.881+0000 7f559f7fe640 1 --2- 172.21.15.39:0/3678970005 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5560889001b0 0x556088cdbd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:51:15.905 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:15.882+0000 7f559effd640 1 -- 172.21.15.39:0/3678970005 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5560889001b0 msgr2=0x556088cdbd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:51:15.905 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:15.882+0000 7f559effd640 1 --2- 172.21.15.39:0/3678970005 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5560889001b0 0x556088cdbd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:15.906 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:15.882+0000 7f559effd640 1 -- 172.21.15.39:0/3678970005 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556088929070 msgr2=0x556088ce8170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:51:15.906 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:15.882+0000 7f559effd640 1 --2- 172.21.15.39:0/3678970005 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556088929070 0x556088ce8170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:15.906 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:15.882+0000 7f559effd640 1 -- 172.21.15.39:0/3678970005 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556088bfec90 con 0x556088908390 2024-09-21T03:51:15.906 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:15.882+0000 7f559effd640 1 --2- 172.21.15.39:0/3678970005 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556088908390 0x556088cdc2c0 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7f55941056b0 tx=0x7f559403b460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c728c7dc7012f804 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:51:15.906 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:15.882+0000 7f557effd640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-09-21T03:51:15.906 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:15.882+0000 7f557effd640 1 -- 172.21.15.39:0/3678970005 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5594048070 con 0x556088908390 2024-09-21T03:51:15.906 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:15.882+0000 7f557effd640 1 -- 172.21.15.39:0/3678970005 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8484+0+0 (secure 0 0 0) 0x7f559410bde0 con 0x556088908390 2024-09-21T03:51:15.907 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:15.882+0000 7f55ac153740 1 -- 172.21.15.39:0/3678970005 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5590018af0 con 0x556088908390 2024-09-21T03:51:15.907 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:15.882+0000 7f557effd640 1 -- 172.21.15.39:0/3678970005 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f55940ca450 con 0x556088908390 2024-09-21T03:51:15.907 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:15.882+0000 7f55ac153740 10 client.16824 fetch_fsmap learned FSMap version 29 2024-09-21T03:51:15.907 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:15.882+0000 7f55ac153740 1 -- 172.21.15.39:0/3678970005 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x556088ceabc0 con 0x556088908390 2024-09-21T03:51:15.907 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:15.882+0000 7f557effd640 1 -- 172.21.15.39:0/3678970005 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5590018af0 con 0x556088908390 2024-09-21T03:51:15.907 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:15.882+0000 7f557effd640 1 -- 172.21.15.39:0/3678970005 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f55940d72e0 con 0x556088908390 2024-09-21T03:51:15.907 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:15.882+0000 7f55ac153740 10 client.16824 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T03:51:15.908 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:15.882+0000 7f55ac153740 10 client.16824 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T03:51:15.908 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:15.882+0000 7f55ac153740 20 client.16824 populate_metadata read hostname 'smithi039' 2024-09-21T03:51:15.908 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:15.882+0000 7f55ac153740 1 --2- 172.21.15.39:0/3678970005 >> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] conn(0x5560888e79f0 0x556088cee140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:51:15.908 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:15.882+0000 7f55ac153740 4 client.16824 mds_command: new command op to 24695 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:51:15.908 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:15.883+0000 7f55ac153740 1 -- 172.21.15.39:0/3678970005 --> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556088c48df0 con 0x5560888e79f0 2024-09-21T03:51:15.908 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:15.883+0000 7f559f7fe640 1 --2- 172.21.15.39:0/3678970005 >> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] conn(0x5560888e79f0 0x556088cee140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:51:15.908 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:15.883+0000 7f559f7fe640 1 --2- 172.21.15.39:0/3678970005 >> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] conn(0x5560888e79f0 0x556088cee140 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=75375d3bb6b867d3 server_cookie=12409268d19661ba in_seq=0 out_seq=0 2024-09-21T03:51:15.908 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:15.883+0000 7f557effd640 10 client.16824 ms_handle_connect on v2:172.21.15.150:6838/2949386907 2024-09-21T03:51:15.908 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:15.884+0000 7f557effd640 1 -- 172.21.15.39:0/3678970005 <== mds.0 v2:172.21.15.150:6838/2949386907 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x556088c48df0 con 0x5560888e79f0 2024-09-21T03:51:15.909 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:15.884+0000 7f557effd640 10 client.16824 handle_command_reply: tid=0 2024-09-21T03:51:15.909 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:15.884+0000 7f55ac153740 1 client.16824 shutdown 2024-09-21T03:51:15.909 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:15.884+0000 7f55ac153740 1 -- 172.21.15.39:0/3678970005 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556088908390 msgr2=0x556088cdc2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:51:15.909 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:15.884+0000 7f55ac153740 1 --2- 172.21.15.39:0/3678970005 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556088908390 0x556088cdc2c0 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7f55941056b0 tx=0x7f559403b460 comp rx=0 tx=0).stop 2024-09-21T03:51:15.909 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:15.884+0000 7f55ac153740 1 -- 172.21.15.39:0/3678970005 shutdown_connections 2024-09-21T03:51:15.909 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:15.884+0000 7f55ac153740 1 --2- 172.21.15.39:0/3678970005 >> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] conn(0x5560888e79f0 0x556088cee140 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:15.909 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:15.884+0000 7f55ac153740 1 --2- 172.21.15.39:0/3678970005 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556088929070 0x556088ce8170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:15.909 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:15.884+0000 7f557effd640 0 client.16824 ms_handle_reset on v2:172.21.15.150:6838/2949386907 2024-09-21T03:51:15.909 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:15.884+0000 7f55ac153740 1 --2- 172.21.15.39:0/3678970005 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556088908390 0x556088cdc2c0 unknown :-1 s=CLOSED pgs=850 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:15.909 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:15.884+0000 7f55ac153740 1 --2- 172.21.15.39:0/3678970005 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5560889001b0 0x556088cdbd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:15.909 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:15.884+0000 7f55ac153740 1 -- 172.21.15.39:0/3678970005 >> 172.21.15.39:0/3678970005 conn(0x55608891d460 msgr2=0x556088cd3880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:51:15.910 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:15.884+0000 7f55ac153740 1 -- 172.21.15.39:0/3678970005 shutdown_connections 2024-09-21T03:51:15.910 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:15.884+0000 7f55ac153740 1 -- 172.21.15.39:0/3678970005 wait complete. 2024-09-21T03:51:15.910 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:15.885+0000 7f55ac153740 20 client.16824 trim_cache size 0 max 16384 2024-09-21T03:51:15.917 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:15.902+0000 7f55ac153740 1 Processor -- start 2024-09-21T03:51:15.917 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:15.902+0000 7f55ac153740 1 -- start start 2024-09-21T03:51:15.918 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:15.903+0000 7f55ac153740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5560889001b0 0x556088cdc240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:51:15.918 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:15.903+0000 7f55ac153740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556088908390 0x556088ce8070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:51:15.918 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:15.903+0000 7f55ac153740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556088929070 0x556088cd8f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:51:15.918 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:15.903+0000 7f55ac153740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f55a01af760 con 0x5560889001b0 2024-09-21T03:51:15.918 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:15.903+0000 7f55ac153740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f55a01b12a0 con 0x556088929070 2024-09-21T03:51:15.918 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:15.903+0000 7f55ac153740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f55a01afe30 con 0x556088908390 2024-09-21T03:51:15.918 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:15.903+0000 7f559f7fe640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5560889001b0 0x556088cdc240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:51:15.919 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:15.903+0000 7f559f7fe640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5560889001b0 0x556088cdc240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:60630/0 (socket says 172.21.15.39:60630) 2024-09-21T03:51:15.919 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:15.903+0000 7f559f7fe640 1 -- 172.21.15.39:0/4094444165 learned_addr learned my addr 172.21.15.39:0/4094444165 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:51:15.919 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:15.903+0000 7f559effd640 1 --2- 172.21.15.39:0/4094444165 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556088908390 0x556088ce8070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:51:15.919 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:15.903+0000 7f559ffff640 1 --2- 172.21.15.39:0/4094444165 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556088929070 0x556088cd8f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:51:15.919 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:15.903+0000 7f559f7fe640 1 -- 172.21.15.39:0/4094444165 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556088908390 msgr2=0x556088ce8070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:51:15.919 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:15.903+0000 7f559f7fe640 1 --2- 172.21.15.39:0/4094444165 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556088908390 0x556088ce8070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:15.919 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:15.903+0000 7f559f7fe640 1 -- 172.21.15.39:0/4094444165 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556088929070 msgr2=0x556088cd8f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:51:15.919 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:15.903+0000 7f559f7fe640 1 --2- 172.21.15.39:0/4094444165 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556088929070 0x556088cd8f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:15.919 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:15.903+0000 7f559f7fe640 1 -- 172.21.15.39:0/4094444165 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556088cda280 con 0x5560889001b0 2024-09-21T03:51:15.920 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:15.903+0000 7f559effd640 1 --2- 172.21.15.39:0/4094444165 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556088908390 0x556088ce8070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:51:15.920 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:15.904+0000 7f559f7fe640 1 --2- 172.21.15.39:0/4094444165 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5560889001b0 0x556088cdc240 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7f5590010620 tx=0x7f5590005900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=17c9752af731397b server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:51:15.920 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:15.904+0000 7f55857fa640 1 -- 172.21.15.39:0/4094444165 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f559002f020 con 0x5560889001b0 2024-09-21T03:51:15.920 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:15.904+0000 7f55857fa640 1 -- 172.21.15.39:0/4094444165 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f559001e040 con 0x5560889001b0 2024-09-21T03:51:15.920 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:15.904+0000 7f55857fa640 1 -- 172.21.15.39:0/4094444165 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f559001d040 con 0x5560889001b0 2024-09-21T03:51:15.920 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:15.905+0000 7f55ac153740 1 -- 172.21.15.39:0/4094444165 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5560889001b0 msgr2=0x556088cdc240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:51:15.920 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:15.905+0000 7f55ac153740 1 --2- 172.21.15.39:0/4094444165 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5560889001b0 0x556088cdc240 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7f5590010620 tx=0x7f5590005900 comp rx=0 tx=0).stop 2024-09-21T03:51:15.920 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:15.905+0000 7f55ac153740 1 -- 172.21.15.39:0/4094444165 shutdown_connections 2024-09-21T03:51:15.920 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:15.905+0000 7f55ac153740 1 --2- 172.21.15.39:0/4094444165 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556088929070 0x556088cd8f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:15.920 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:15.905+0000 7f55ac153740 1 --2- 172.21.15.39:0/4094444165 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556088908390 0x556088ce8070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:15.921 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:15.905+0000 7f55ac153740 1 --2- 172.21.15.39:0/4094444165 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5560889001b0 0x556088cdc240 unknown :-1 s=CLOSED pgs=851 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:15.921 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:15.905+0000 7f55ac153740 1 -- 172.21.15.39:0/4094444165 >> 172.21.15.39:0/4094444165 conn(0x55608891d460 msgr2=0x556088cdbb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:51:15.921 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:15.905+0000 7f55ac153740 1 -- 172.21.15.39:0/4094444165 shutdown_connections 2024-09-21T03:51:15.921 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:15.905+0000 7f55ac153740 1 -- 172.21.15.39:0/4094444165 wait complete. 2024-09-21T03:51:15.921 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:15.905+0000 7f55ac153740 1 -- start start 2024-09-21T03:51:15.921 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:15.905+0000 7f55ac153740 1 Processor -- start 2024-09-21T03:51:15.921 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:15.906+0000 7f55ac153740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5560889001b0 0x556088ce40d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:51:15.921 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:15.906+0000 7f55ac153740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556088908390 0x556088ce4610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:51:15.921 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:15.906+0000 7f55ac153740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556088929070 0x556088d51c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:51:15.922 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:15.906+0000 7f55ac153740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f55a01b0bd0 con 0x556088929070 2024-09-21T03:51:15.922 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:15.906+0000 7f55ac153740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f55a01b0500 con 0x556088908390 2024-09-21T03:51:15.922 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:15.906+0000 7f55ac153740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f55a01ad740 con 0x5560889001b0 2024-09-21T03:51:15.922 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:15.906+0000 7f55ac153740 10 client.0 _ll_register_callbacks cb 0x7f5594005ef0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:51:15.922 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:15.906+0000 7f559ffff640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556088929070 0x556088d51c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:51:15.922 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:15.906+0000 7f559ffff640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556088929070 0x556088d51c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:60644/0 (socket says 172.21.15.39:60644) 2024-09-21T03:51:15.922 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:15.906+0000 7f559ffff640 1 -- 172.21.15.39:0/413906534 learned_addr learned my addr 172.21.15.39:0/413906534 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:51:15.922 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:15.906+0000 7f559f7fe640 1 --2- 172.21.15.39:0/413906534 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5560889001b0 0x556088ce40d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:51:15.923 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:15.906+0000 7f559effd640 1 --2- 172.21.15.39:0/413906534 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556088908390 0x556088ce4610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:51:15.923 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:15.906+0000 7f559ffff640 1 -- 172.21.15.39:0/413906534 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5560889001b0 msgr2=0x556088ce40d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:51:15.923 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:15.906+0000 7f559ffff640 1 --2- 172.21.15.39:0/413906534 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5560889001b0 0x556088ce40d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:15.923 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:15.906+0000 7f559ffff640 1 -- 172.21.15.39:0/413906534 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556088908390 msgr2=0x556088ce4610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:51:15.923 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:15.906+0000 7f559ffff640 1 --2- 172.21.15.39:0/413906534 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556088908390 0x556088ce4610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:15.924 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:15.906+0000 7f559ffff640 1 -- 172.21.15.39:0/413906534 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556088ce4d00 con 0x556088929070 2024-09-21T03:51:15.924 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:15.906+0000 7f559f7fe640 1 --2- 172.21.15.39:0/413906534 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5560889001b0 0x556088ce40d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:51:15.924 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:15.907+0000 7f559ffff640 1 --2- 172.21.15.39:0/413906534 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556088929070 0x556088d51c30 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7f5598009950 tx=0x7f5598004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8410765799534ecd server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:51:15.924 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:15.907+0000 7f557effd640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-09-21T03:51:15.924 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:15.907+0000 7f557effd640 1 -- 172.21.15.39:0/413906534 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f559802f060 con 0x556088929070 2024-09-21T03:51:15.924 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:15.907+0000 7f557effd640 1 -- 172.21.15.39:0/413906534 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8484+0+0 (secure 0 0 0) 0x7f559801d400 con 0x556088929070 2024-09-21T03:51:15.924 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:15.907+0000 7f55ac153740 1 -- 172.21.15.39:0/413906534 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f559001e040 con 0x556088929070 2024-09-21T03:51:15.924 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:15.907+0000 7f557effd640 1 -- 172.21.15.39:0/413906534 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f559801d600 con 0x556088929070 2024-09-21T03:51:15.924 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:15.907+0000 7f55ac153740 10 client.16830 fetch_fsmap learned FSMap version 29 2024-09-21T03:51:15.925 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:15.907+0000 7f55ac153740 1 -- 172.21.15.39:0/413906534 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x556088d5d3b0 con 0x556088929070 2024-09-21T03:51:15.925 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:15.907+0000 7f557effd640 1 -- 172.21.15.39:0/413906534 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f559001e040 con 0x556088929070 2024-09-21T03:51:15.925 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:15.907+0000 7f557effd640 1 -- 172.21.15.39:0/413906534 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f5598029070 con 0x556088929070 2024-09-21T03:51:15.925 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:15.907+0000 7f55ac153740 10 client.16830 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T03:51:15.925 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:15.907+0000 7f55ac153740 10 client.16830 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T03:51:15.925 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:15.907+0000 7f55ac153740 20 client.16830 populate_metadata read hostname 'smithi039' 2024-09-21T03:51:15.925 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:15.907+0000 7f55ac153740 1 --2- 172.21.15.39:0/413906534 >> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] conn(0x5560888bcda0 0x556088d5fbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:51:15.925 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:15.907+0000 7f55ac153740 4 client.16830 mds_command: new command op to 24695 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T03:51:15.925 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:15.907+0000 7f55ac153740 1 -- 172.21.15.39:0/413906534 --> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] -- command(tid 0: {"prefix": "damage ls"}) -- 0x556088c448a0 con 0x5560888bcda0 2024-09-21T03:51:15.926 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:15.908+0000 7f559f7fe640 1 --2- 172.21.15.39:0/413906534 >> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] conn(0x5560888bcda0 0x556088d5fbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:51:15.926 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:15.909+0000 7f557effd640 10 client.16830 ms_handle_connect on v2:172.21.15.150:6838/2949386907 2024-09-21T03:51:15.926 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:15.909+0000 7f559f7fe640 1 --2- 172.21.15.39:0/413906534 >> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] conn(0x5560888bcda0 0x556088d5fbd0 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5f0f5c0c568c6938 server_cookie=62501c30e996a49b in_seq=0 out_seq=0 2024-09-21T03:51:15.926 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:15.909+0000 7f557effd640 1 -- 172.21.15.39:0/413906534 <== mds.0 v2:172.21.15.150:6838/2949386907 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x556088c448a0 con 0x5560888bcda0 2024-09-21T03:51:15.926 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:15.909+0000 7f557effd640 10 client.16830 handle_command_reply: tid=0 2024-09-21T03:51:15.926 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:15.909+0000 7f55ac153740 1 client.16830 shutdown 2024-09-21T03:51:15.926 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:15.909+0000 7f55ac153740 1 -- 172.21.15.39:0/413906534 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556088929070 msgr2=0x556088d51c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:51:15.926 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:15.909+0000 7f55ac153740 1 --2- 172.21.15.39:0/413906534 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556088929070 0x556088d51c30 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7f5598009950 tx=0x7f5598004ab0 comp rx=0 tx=0).stop 2024-09-21T03:51:15.926 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:15.909+0000 7f55ac153740 1 -- 172.21.15.39:0/413906534 shutdown_connections 2024-09-21T03:51:15.927 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:15.909+0000 7f55ac153740 1 --2- 172.21.15.39:0/413906534 >> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] conn(0x5560888bcda0 0x556088d5fbd0 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:15.927 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:15.909+0000 7f55ac153740 1 --2- 172.21.15.39:0/413906534 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556088929070 0x556088d51c30 unknown :-1 s=CLOSED pgs=852 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:15.927 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:15.909+0000 7f55ac153740 1 --2- 172.21.15.39:0/413906534 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556088908390 0x556088ce4610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:15.927 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:15.909+0000 7f55ac153740 1 --2- 172.21.15.39:0/413906534 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5560889001b0 0x556088ce40d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:15.927 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:15.909+0000 7f55ac153740 1 -- 172.21.15.39:0/413906534 >> 172.21.15.39:0/413906534 conn(0x55608891d460 msgr2=0x556088cd37a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:51:15.927 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:15.909+0000 7f557effd640 0 client.16830 ms_handle_reset on v2:172.21.15.150:6838/2949386907 2024-09-21T03:51:15.927 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:15.909+0000 7f55ac153740 1 -- 172.21.15.39:0/413906534 shutdown_connections 2024-09-21T03:51:15.927 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:15.909+0000 7f55ac153740 1 -- 172.21.15.39:0/413906534 wait complete. 2024-09-21T03:51:15.928 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:15.910+0000 7f55ac153740 20 client.16830 trim_cache size 0 max 16384 2024-09-21T03:51:15.928 INFO:teuthology.orchestra.run.smithi039.stdout:[] 2024-09-21T03:51:15.929 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:15.910+0000 7f55a5bdf640 1 -- 172.21.15.39:0/2754062349 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f55880b6680 msgr2=0x7f55880b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:51:15.929 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:15.910+0000 7f55a5bdf640 1 --2- 172.21.15.39:0/2754062349 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f55880b6680 0x7f55880b8b90 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f5598009270 tx=0x7f5598009680 comp rx=0 tx=0).stop 2024-09-21T03:51:15.929 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:15.910+0000 7f55a5bdf640 1 -- 172.21.15.39:0/2754062349 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f55a01094a0 msgr2=0x7f55a019ab00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:51:15.930 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:15.910+0000 7f55a5bdf640 1 --2- 172.21.15.39:0/2754062349 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f55a01094a0 0x7f55a019ab00 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7f559402a930 tx=0x7f5594004270 comp rx=0 tx=0).stop 2024-09-21T03:51:15.930 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:15.910+0000 7f55a5bdf640 1 -- 172.21.15.39:0/2754062349 shutdown_connections 2024-09-21T03:51:15.930 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:15.910+0000 7f55a5bdf640 1 --2- 172.21.15.39:0/2754062349 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f55880b6680 0x7f55880b8b90 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:15.930 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:15.910+0000 7f55a5bdf640 1 --2- 172.21.15.39:0/2754062349 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f55a0109df0 0x7f55a019fda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:15.930 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:15.910+0000 7f55a5bdf640 1 --2- 172.21.15.39:0/2754062349 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f55a01094a0 0x7f55a019ab00 unknown :-1 s=CLOSED pgs=849 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:15.930 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:15.910+0000 7f55a5bdf640 1 --2- 172.21.15.39:0/2754062349 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f55a0103b60 0x7f55a019a560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:15.930 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:15.910+0000 7f55a5bdf640 1 -- 172.21.15.39:0/2754062349 >> 172.21.15.39:0/2754062349 conn(0x7f55a007b850 msgr2=0x7f55a0105560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:51:15.930 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:15.910+0000 7f55a5bdf640 1 -- 172.21.15.39:0/2754062349 shutdown_connections 2024-09-21T03:51:15.931 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:15.910+0000 7f55a5bdf640 1 -- 172.21.15.39:0/2754062349 wait complete. 2024-09-21T03:51:17.334 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:51:17 smithi039 ceph-mon[31211]: pgmap v482: 97 pgs: 97 active+clean; 10 MiB data, 520 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 25 op/s 2024-09-21T03:51:17.468 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:51:17 smithi161 ceph-mon[34533]: pgmap v482: 97 pgs: 97 active+clean; 10 MiB data, 520 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 25 op/s 2024-09-21T03:51:17.475 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:51:17 smithi150 ceph-mon[34289]: pgmap v482: 97 pgs: 97 active+clean; 10 MiB data, 520 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 25 op/s 2024-09-21T03:51:17.723 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-21T03:51:17.724 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 51116d5c-77ca-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-21T03:51:17.994 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/51116d5c-77ca-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:51:18.529 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:18.524+0000 7f4eb2d6c640 1 -- 172.21.15.39:0/4164751814 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4eac108d10 msgr2=0x7f4eac10f540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:51:18.529 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:18.524+0000 7f4eb2d6c640 1 --2- 172.21.15.39:0/4164751814 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4eac108d10 0x7f4eac10f540 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7f4e9c009900 tx=0x7f4e9c02a410 comp rx=0 tx=0).stop 2024-09-21T03:51:18.529 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:18.524+0000 7f4eb2d6c640 1 -- 172.21.15.39:0/4164751814 shutdown_connections 2024-09-21T03:51:18.529 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:18.524+0000 7f4eb2d6c640 1 --2- 172.21.15.39:0/4164751814 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4eac108d10 0x7f4eac10f540 unknown :-1 s=CLOSED pgs=853 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:18.529 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:18.524+0000 7f4eb2d6c640 1 --2- 172.21.15.39:0/4164751814 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4eac1083c0 0x7f4eac1087d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:18.529 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:18.524+0000 7f4eb2d6c640 1 --2- 172.21.15.39:0/4164751814 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4eac102a80 0x7f4eac102e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:18.529 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:18.524+0000 7f4eb2d6c640 1 -- 172.21.15.39:0/4164751814 >> 172.21.15.39:0/4164751814 conn(0x7f4eac079740 msgr2=0x7f4eac079b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:51:18.529 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:18.524+0000 7f4eb2d6c640 1 -- 172.21.15.39:0/4164751814 shutdown_connections 2024-09-21T03:51:18.529 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:18.524+0000 7f4eb2d6c640 1 -- 172.21.15.39:0/4164751814 wait complete. 2024-09-21T03:51:18.530 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:18.525+0000 7f4eb2d6c640 1 Processor -- start 2024-09-21T03:51:18.530 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:18.525+0000 7f4eb2d6c640 1 -- start start 2024-09-21T03:51:18.530 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:18.525+0000 7f4eb2d6c640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4eac102a80 0x7f4eac0ff690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:51:18.530 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:18.525+0000 7f4eb2d6c640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4eac1083c0 0x7f4eac0ffbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:51:18.530 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:18.525+0000 7f4eb2d6c640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4eac108d10 0x7f4eac100110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:51:18.530 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:18.525+0000 7f4eb2d6c640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f4eac112980 con 0x7f4eac102a80 2024-09-21T03:51:18.530 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:18.525+0000 7f4eb2d6c640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f4eac112800 con 0x7f4eac108d10 2024-09-21T03:51:18.530 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:18.525+0000 7f4eb2d6c640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f4eac112b00 con 0x7f4eac1083c0 2024-09-21T03:51:18.530 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:18.525+0000 7f4eb0b08640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4eac102a80 0x7f4eac0ff690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:51:18.530 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:18.525+0000 7f4eb0b08640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4eac102a80 0x7f4eac0ff690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:36280/0 (socket says 172.21.15.39:36280) 2024-09-21T03:51:18.530 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:18.525+0000 7f4eb0b08640 1 -- 172.21.15.39:0/4226592613 learned_addr learned my addr 172.21.15.39:0/4226592613 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:51:18.530 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:18.525+0000 7f4eabfff640 1 --2- 172.21.15.39:0/4226592613 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4eac1083c0 0x7f4eac0ffbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:51:18.530 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:18.525+0000 7f4eb1309640 1 --2- 172.21.15.39:0/4226592613 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4eac108d10 0x7f4eac100110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:51:18.531 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:18.526+0000 7f4eb0b08640 1 -- 172.21.15.39:0/4226592613 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4eac1083c0 msgr2=0x7f4eac0ffbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:51:18.531 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:18.526+0000 7f4eb0b08640 1 --2- 172.21.15.39:0/4226592613 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4eac1083c0 0x7f4eac0ffbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:18.531 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:18.526+0000 7f4eb0b08640 1 -- 172.21.15.39:0/4226592613 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4eac108d10 msgr2=0x7f4eac100110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:51:18.531 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:18.526+0000 7f4eb0b08640 1 --2- 172.21.15.39:0/4226592613 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4eac108d10 0x7f4eac100110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:18.531 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:18.526+0000 7f4eb0b08640 1 -- 172.21.15.39:0/4226592613 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4eac1a8710 con 0x7f4eac102a80 2024-09-21T03:51:18.531 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:18.526+0000 7f4eb0b08640 1 --2- 172.21.15.39:0/4226592613 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4eac102a80 0x7f4eac0ff690 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7f4ea400eab0 tx=0x7f4ea400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:51:18.531 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:18.526+0000 7f4ea9ffb640 1 -- 172.21.15.39:0/4226592613 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ea400cd80 con 0x7f4eac102a80 2024-09-21T03:51:18.531 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:18.526+0000 7f4eb2d6c640 1 -- 172.21.15.39:0/4226592613 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4eac1a8a00 con 0x7f4eac102a80 2024-09-21T03:51:18.531 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:18.526+0000 7f4ea9ffb640 1 -- 172.21.15.39:0/4226592613 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4ea4011920 con 0x7f4eac102a80 2024-09-21T03:51:18.531 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:18.526+0000 7f4ea9ffb640 1 -- 172.21.15.39:0/4226592613 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ea4010930 con 0x7f4eac102a80 2024-09-21T03:51:18.531 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:18.526+0000 7f4eb2d6c640 1 -- 172.21.15.39:0/4226592613 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4eac1a8f40 con 0x7f4eac102a80 2024-09-21T03:51:18.532 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:18.529+0000 7f4eb92e0740 1 Processor -- start 2024-09-21T03:51:18.532 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:18.529+0000 7f4eb92e0740 1 -- start start 2024-09-21T03:51:18.533 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:18.529+0000 7f4ea9ffb640 1 -- 172.21.15.39:0/4226592613 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 27) ==== 154643+0+0 (secure 0 0 0) 0x7f4ea4011470 con 0x7f4eac102a80 2024-09-21T03:51:18.533 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:18.529+0000 7f4eb92e0740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564daac06350 0x564daafad430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:51:18.533 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:18.529+0000 7f4eb92e0740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564daabdd490 0x564daafad970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:51:18.533 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:18.529+0000 7f4eb92e0740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564daabe5670 0x564daafadeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:51:18.533 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:18.529+0000 7f4eb92e0740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x564daaf23e60 con 0x564daabdd490 2024-09-21T03:51:18.533 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:18.529+0000 7f4eb92e0740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x564daaf13290 con 0x564daabe5670 2024-09-21T03:51:18.533 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:18.529+0000 7f4eb92e0740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x564daaf170a0 con 0x564daac06350 2024-09-21T03:51:18.533 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:18.529+0000 7f4eb0b08640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564daabdd490 0x564daafad970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:51:18.533 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:18.529+0000 7f4eb0b08640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564daabdd490 0x564daafad970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:36292/0 (socket says 172.21.15.39:36292) 2024-09-21T03:51:18.533 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:18.529+0000 7f4eb0b08640 1 -- 172.21.15.39:0/4088077831 learned_addr learned my addr 172.21.15.39:0/4088077831 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:51:18.533 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:18.529+0000 7f4eabfff640 1 --2- 172.21.15.39:0/4088077831 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564daabe5670 0x564daafadeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:51:18.533 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:18.529+0000 7f4eb1309640 1 --2- 172.21.15.39:0/4088077831 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564daac06350 0x564daafad430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:51:18.533 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:18.530+0000 7f4eb0b08640 1 -- 172.21.15.39:0/4088077831 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564daac06350 msgr2=0x564daafad430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:51:18.534 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:18.530+0000 7f4eb0b08640 1 --2- 172.21.15.39:0/4088077831 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564daac06350 0x564daafad430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:18.534 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:18.530+0000 7f4eb0b08640 1 -- 172.21.15.39:0/4088077831 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564daabe5670 msgr2=0x564daafadeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:51:18.534 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:18.530+0000 7f4eb0b08640 1 --2- 172.21.15.39:0/4088077831 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564daabe5670 0x564daafadeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:18.534 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:18.530+0000 7f4eb0b08640 1 -- 172.21.15.39:0/4088077831 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564daac71930 con 0x564daabdd490 2024-09-21T03:51:18.534 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:18.530+0000 7f4eb0b08640 1 --2- 172.21.15.39:0/4088077831 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564daabdd490 0x564daafad970 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7f4ea40e5330 tx=0x7f4ea40ebf20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bff959df17142b37 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:51:18.537 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:18.530+0000 7f4ea9ffb640 1 --2- 172.21.15.39:0/4226592613 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f4e940b6680 0x7f4e940b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:51:18.537 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:18.530+0000 7f4e92ffd640 1 -- 172.21.15.39:0/4088077831 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ea40fa470 con 0x564daabdd490 2024-09-21T03:51:18.538 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:18.530+0000 7f4e92ffd640 1 -- 172.21.15.39:0/4088077831 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4ea40f5070 con 0x564daabdd490 2024-09-21T03:51:18.538 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:18.530+0000 7f4e92ffd640 1 -- 172.21.15.39:0/4088077831 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ea4093670 con 0x564daabdd490 2024-09-21T03:51:18.538 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:18.530+0000 7f4eb1309640 1 --2- 172.21.15.39:0/4226592613 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f4e940b6680 0x7f4e940b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:51:18.538 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:18.530+0000 7f4ea9ffb640 1 -- 172.21.15.39:0/4226592613 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8484+0+0 (secure 0 0 0) 0x7f4ea40e7c50 con 0x7f4eac102a80 2024-09-21T03:51:18.538 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:18.531+0000 7f4eb92e0740 1 -- 172.21.15.39:0/4088077831 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564daabdd490 msgr2=0x564daafad970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:51:18.538 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:18.531+0000 7f4eb1309640 1 --2- 172.21.15.39:0/4226592613 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f4e940b6680 0x7f4e940b8b90 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f4e9c006010 tx=0x7f4e9c0058d0 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:51:18.538 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:18.531+0000 7f4eb92e0740 1 --2- 172.21.15.39:0/4088077831 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564daabdd490 0x564daafad970 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7f4ea40e5330 tx=0x7f4ea40ebf20 comp rx=0 tx=0).stop 2024-09-21T03:51:18.538 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:18.531+0000 7f4eb92e0740 1 -- 172.21.15.39:0/4088077831 shutdown_connections 2024-09-21T03:51:18.538 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:18.531+0000 7f4eb92e0740 1 --2- 172.21.15.39:0/4088077831 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564daabe5670 0x564daafadeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:18.538 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:18.531+0000 7f4eb92e0740 1 --2- 172.21.15.39:0/4088077831 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564daabdd490 0x564daafad970 unknown :-1 s=CLOSED pgs=855 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:18.538 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:18.531+0000 7f4eb92e0740 1 --2- 172.21.15.39:0/4088077831 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564daac06350 0x564daafad430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:18.538 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:18.531+0000 7f4eb92e0740 1 -- 172.21.15.39:0/4088077831 >> 172.21.15.39:0/4088077831 conn(0x564daabfa740 msgr2=0x564daafab320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:51:18.538 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:18.531+0000 7f4eb92e0740 1 -- 172.21.15.39:0/4088077831 shutdown_connections 2024-09-21T03:51:18.539 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:18.531+0000 7f4eb92e0740 1 -- 172.21.15.39:0/4088077831 wait complete. 2024-09-21T03:51:18.539 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:18.532+0000 7f4eb92e0740 1 -- start start 2024-09-21T03:51:18.539 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:18.532+0000 7f4eb92e0740 1 Processor -- start 2024-09-21T03:51:18.539 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:18.532+0000 7f4eb92e0740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564daabdd490 0x564daafb8fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:51:18.539 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:18.532+0000 7f4eb92e0740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564daabe5670 0x564daafb9510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:51:18.539 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:18.532+0000 7f4eb92e0740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564daac06350 0x564daafc4f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:51:18.539 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:18.532+0000 7f4eb92e0740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x564daad74890 con 0x564daabe5670 2024-09-21T03:51:18.539 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:18.532+0000 7f4eb92e0740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x564daaeadd40 con 0x564daabdd490 2024-09-21T03:51:18.539 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:18.532+0000 7f4eb92e0740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x564daafbbc10 con 0x564daac06350 2024-09-21T03:51:18.539 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:18.532+0000 7f4eb92e0740 10 client.0 _ll_register_callbacks cb 0x564daaf11c80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:51:18.539 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:18.532+0000 7f4eabfff640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564daabe5670 0x564daafb9510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:51:18.539 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:18.533+0000 7f4eabfff640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564daabe5670 0x564daafb9510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:36302/0 (socket says 172.21.15.39:36302) 2024-09-21T03:51:18.539 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:18.533+0000 7f4eabfff640 1 -- 172.21.15.39:0/2876487883 learned_addr learned my addr 172.21.15.39:0/2876487883 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:51:18.539 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:18.533+0000 7f4eb1309640 1 --2- 172.21.15.39:0/2876487883 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564daac06350 0x564daafc4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:51:18.539 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:18.533+0000 7f4eb0b08640 1 --2- 172.21.15.39:0/2876487883 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564daabdd490 0x564daafb8fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:51:18.540 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:18.533+0000 7f4eabfff640 1 -- 172.21.15.39:0/2876487883 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564daac06350 msgr2=0x564daafc4f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:51:18.540 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:18.533+0000 7f4eabfff640 1 --2- 172.21.15.39:0/2876487883 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564daac06350 0x564daafc4f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:18.540 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:18.533+0000 7f4eabfff640 1 -- 172.21.15.39:0/2876487883 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564daabdd490 msgr2=0x564daafb8fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:51:18.540 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:18.533+0000 7f4eabfff640 1 --2- 172.21.15.39:0/2876487883 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564daabdd490 0x564daafb8fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:18.540 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:18.533+0000 7f4eabfff640 1 -- 172.21.15.39:0/2876487883 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564daafc54c0 con 0x564daabe5670 2024-09-21T03:51:18.540 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:18.533+0000 7f4eb0b08640 1 --2- 172.21.15.39:0/2876487883 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564daabdd490 0x564daafb8fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:51:18.540 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:18.533+0000 7f4eabfff640 1 --2- 172.21.15.39:0/2876487883 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564daabe5670 0x564daafb9510 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7f4ea0010850 tx=0x7f4ea0010c50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff0d1599ebd9e445 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:51:18.540 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:18.533+0000 7f4e8bfff640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-09-21T03:51:18.540 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:18.533+0000 7f4e8bfff640 1 -- 172.21.15.39:0/2876487883 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ea000fb90 con 0x564daabe5670 2024-09-21T03:51:18.540 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:18.533+0000 7f4e8bfff640 1 -- 172.21.15.39:0/2876487883 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8484+0+0 (secure 0 0 0) 0x7f4ea000fd30 con 0x564daabe5670 2024-09-21T03:51:18.540 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:18.533+0000 7f4eb92e0740 1 -- 172.21.15.39:0/2876487883 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4ea40f5070 con 0x564daabe5670 2024-09-21T03:51:18.540 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:18.534+0000 7f4e8bfff640 1 -- 172.21.15.39:0/2876487883 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f4ea0005020 con 0x564daabe5670 2024-09-21T03:51:18.540 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:18.534+0000 7f4eb92e0740 10 client.16842 fetch_fsmap learned FSMap version 29 2024-09-21T03:51:18.540 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:18.534+0000 7f4eb92e0740 1 -- 172.21.15.39:0/2876487883 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x564daafc7e50 con 0x564daabe5670 2024-09-21T03:51:18.540 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:18.534+0000 7f4e8bfff640 1 -- 172.21.15.39:0/2876487883 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4ea40f5070 con 0x564daabe5670 2024-09-21T03:51:18.541 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:18.534+0000 7f4e8bfff640 1 -- 172.21.15.39:0/2876487883 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f4ea0020020 con 0x564daabe5670 2024-09-21T03:51:18.541 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:18.534+0000 7f4eb92e0740 10 client.16842 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T03:51:18.541 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:18.534+0000 7f4eb92e0740 10 client.16842 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T03:51:18.541 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:18.534+0000 7f4eb92e0740 20 client.16842 populate_metadata read hostname 'smithi039' 2024-09-21T03:51:18.541 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:18.534+0000 7f4eb92e0740 1 --2- 172.21.15.39:0/2876487883 >> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] conn(0x564daabc4cd0 0x564daafcb400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:51:18.541 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:18.534+0000 7f4eb92e0740 4 client.16842 mds_command: new command op to 24695 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:51:18.541 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:18.534+0000 7f4eb92e0740 1 -- 172.21.15.39:0/2876487883 --> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564daaf25db0 con 0x564daabc4cd0 2024-09-21T03:51:18.541 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:18.534+0000 7f4eb0b08640 1 --2- 172.21.15.39:0/2876487883 >> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] conn(0x564daabc4cd0 0x564daafcb400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:51:18.541 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:18.535+0000 7f4eb0b08640 1 --2- 172.21.15.39:0/2876487883 >> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] conn(0x564daabc4cd0 0x564daafcb400 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e57108f72a46b9e2 server_cookie=3b9853aaacd970dc in_seq=0 out_seq=0 2024-09-21T03:51:18.541 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:18.535+0000 7f4e8bfff640 10 client.16842 ms_handle_connect on v2:172.21.15.150:6838/2949386907 2024-09-21T03:51:18.542 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:18.536+0000 7f4e8bfff640 1 -- 172.21.15.39:0/2876487883 <== mds.0 v2:172.21.15.150:6838/2949386907 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x564daaf25db0 con 0x564daabc4cd0 2024-09-21T03:51:18.542 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:18.536+0000 7f4e8bfff640 10 client.16842 handle_command_reply: tid=0 2024-09-21T03:51:18.542 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:18.536+0000 7f4eb92e0740 1 client.16842 shutdown 2024-09-21T03:51:18.542 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:18.536+0000 7f4eb92e0740 1 -- 172.21.15.39:0/2876487883 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564daabe5670 msgr2=0x564daafb9510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:51:18.542 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:18.536+0000 7f4eb92e0740 1 --2- 172.21.15.39:0/2876487883 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564daabe5670 0x564daafb9510 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7f4ea0010850 tx=0x7f4ea0010c50 comp rx=0 tx=0).stop 2024-09-21T03:51:18.542 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:18.536+0000 7f4eb92e0740 1 -- 172.21.15.39:0/2876487883 shutdown_connections 2024-09-21T03:51:18.543 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:18.536+0000 7f4eb92e0740 1 --2- 172.21.15.39:0/2876487883 >> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] conn(0x564daabc4cd0 0x564daafcb400 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:18.543 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:18.536+0000 7f4eb92e0740 1 --2- 172.21.15.39:0/2876487883 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564daac06350 0x564daafc4f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:18.543 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:18.536+0000 7f4eb92e0740 1 --2- 172.21.15.39:0/2876487883 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564daabe5670 0x564daafb9510 unknown :-1 s=CLOSED pgs=856 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:18.543 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:18.536+0000 7f4eb92e0740 1 --2- 172.21.15.39:0/2876487883 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564daabdd490 0x564daafb8fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:18.543 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:18.536+0000 7f4eb92e0740 1 -- 172.21.15.39:0/2876487883 >> 172.21.15.39:0/2876487883 conn(0x564daabfa740 msgr2=0x564daafb05c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:51:18.543 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:18.536+0000 7f4e8bfff640 0 client.16842 ms_handle_reset on v2:172.21.15.150:6838/2949386907 2024-09-21T03:51:18.543 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:18.536+0000 7f4eb92e0740 1 -- 172.21.15.39:0/2876487883 shutdown_connections 2024-09-21T03:51:18.543 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:18.536+0000 7f4eb92e0740 1 -- 172.21.15.39:0/2876487883 wait complete. 2024-09-21T03:51:18.543 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:18.536+0000 7f4eb92e0740 20 client.16842 trim_cache size 0 max 16384 2024-09-21T03:51:18.558 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:18.554+0000 7f4eb92e0740 1 Processor -- start 2024-09-21T03:51:18.558 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:18.554+0000 7f4eb92e0740 1 -- start start 2024-09-21T03:51:18.558 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:18.554+0000 7f4eb92e0740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564daabdd490 0x564daafb4580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:51:18.558 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:18.554+0000 7f4eb92e0740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564daabe5670 0x564daafc4e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:51:18.558 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:18.554+0000 7f4eb92e0740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564daac06350 0x564daafaee20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:51:18.558 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:18.554+0000 7f4eb92e0740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f4eac1b18f0 con 0x564daac06350 2024-09-21T03:51:18.558 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:18.554+0000 7f4eb92e0740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f4eac1b3430 con 0x564daabe5670 2024-09-21T03:51:18.558 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:18.554+0000 7f4eb92e0740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f4eac1b1fc0 con 0x564daabdd490 2024-09-21T03:51:18.558 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:18.554+0000 7f4eb1309640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564daac06350 0x564daafaee20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:51:18.558 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:18.554+0000 7f4eb1309640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564daac06350 0x564daafaee20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:36306/0 (socket says 172.21.15.39:36306) 2024-09-21T03:51:18.558 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:18.554+0000 7f4eb1309640 1 -- 172.21.15.39:0/242909171 learned_addr learned my addr 172.21.15.39:0/242909171 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:51:18.558 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:18.554+0000 7f4eb0b08640 1 --2- 172.21.15.39:0/242909171 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564daabdd490 0x564daafb4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:51:18.559 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:18.554+0000 7f4eabfff640 1 --2- 172.21.15.39:0/242909171 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564daabe5670 0x564daafc4e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:51:18.559 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:18.554+0000 7f4eb1309640 1 -- 172.21.15.39:0/242909171 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564daabdd490 msgr2=0x564daafb4580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:51:18.559 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:18.554+0000 7f4eb1309640 1 --2- 172.21.15.39:0/242909171 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564daabdd490 0x564daafb4580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:18.559 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:18.554+0000 7f4eb1309640 1 -- 172.21.15.39:0/242909171 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564daabe5670 msgr2=0x564daafc4e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:51:18.559 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:18.554+0000 7f4eb1309640 1 --2- 172.21.15.39:0/242909171 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564daabe5670 0x564daafc4e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:18.559 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:18.554+0000 7f4eb1309640 1 -- 172.21.15.39:0/242909171 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564daafba090 con 0x564daac06350 2024-09-21T03:51:18.559 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:18.555+0000 7f4eb1309640 1 --2- 172.21.15.39:0/242909171 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564daac06350 0x564daafaee20 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7f4e9c009900 tx=0x7f4e9c03f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce0f677572686c52 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:51:18.559 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:18.555+0000 7f4e927fc640 1 -- 172.21.15.39:0/242909171 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e9c035040 con 0x564daac06350 2024-09-21T03:51:18.559 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:18.555+0000 7f4e927fc640 1 -- 172.21.15.39:0/242909171 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4e9c04e070 con 0x564daac06350 2024-09-21T03:51:18.559 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:18.555+0000 7f4e927fc640 1 -- 172.21.15.39:0/242909171 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e9c004e40 con 0x564daac06350 2024-09-21T03:51:18.562 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:18.556+0000 7f4eb92e0740 1 -- 172.21.15.39:0/242909171 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564daac06350 msgr2=0x564daafaee20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:51:18.562 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:18.556+0000 7f4eb92e0740 1 --2- 172.21.15.39:0/242909171 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564daac06350 0x564daafaee20 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7f4e9c009900 tx=0x7f4e9c03f000 comp rx=0 tx=0).stop 2024-09-21T03:51:18.562 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:18.556+0000 7f4eb92e0740 1 -- 172.21.15.39:0/242909171 shutdown_connections 2024-09-21T03:51:18.562 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:18.556+0000 7f4eb92e0740 1 --2- 172.21.15.39:0/242909171 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564daac06350 0x564daafaee20 unknown :-1 s=CLOSED pgs=857 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:18.562 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:18.556+0000 7f4eb92e0740 1 --2- 172.21.15.39:0/242909171 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564daabe5670 0x564daafc4e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:18.562 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:18.556+0000 7f4eb92e0740 1 --2- 172.21.15.39:0/242909171 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564daabdd490 0x564daafb4580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:18.562 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:18.556+0000 7f4eb92e0740 1 -- 172.21.15.39:0/242909171 >> 172.21.15.39:0/242909171 conn(0x564daabfa740 msgr2=0x564daafba2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:51:18.562 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:18.556+0000 7f4eb92e0740 1 -- 172.21.15.39:0/242909171 shutdown_connections 2024-09-21T03:51:18.563 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:18.556+0000 7f4eb92e0740 1 -- 172.21.15.39:0/242909171 wait complete. 2024-09-21T03:51:18.563 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:18.556+0000 7f4eb92e0740 1 -- start start 2024-09-21T03:51:18.563 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:18.556+0000 7f4eb92e0740 1 Processor -- start 2024-09-21T03:51:18.563 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:18.557+0000 7f4eb92e0740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564daabdd490 0x564daafbcd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:51:18.563 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:18.557+0000 7f4eb92e0740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564daabe5670 0x564daafbd2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:51:18.563 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:18.557+0000 7f4eb92e0740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564daac06350 0x564daafbd810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:51:18.563 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:18.557+0000 7f4eb92e0740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f4eac1b2d60 con 0x564daac06350 2024-09-21T03:51:18.563 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:18.557+0000 7f4eb92e0740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f4eac1b2690 con 0x564daabe5670 2024-09-21T03:51:18.563 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:18.557+0000 7f4eb92e0740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f4eac1af8d0 con 0x564daabdd490 2024-09-21T03:51:18.563 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:18.557+0000 7f4eb92e0740 10 client.0 _ll_register_callbacks cb 0x7f4ea000a700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:51:18.563 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:18.557+0000 7f4eb1309640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564daac06350 0x564daafbd810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:51:18.563 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:18.557+0000 7f4eb1309640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564daac06350 0x564daafbd810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:36314/0 (socket says 172.21.15.39:36314) 2024-09-21T03:51:18.563 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:18.557+0000 7f4eb1309640 1 -- 172.21.15.39:0/4093904376 learned_addr learned my addr 172.21.15.39:0/4093904376 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:51:18.563 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:18.557+0000 7f4eabfff640 1 --2- 172.21.15.39:0/4093904376 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564daabe5670 0x564daafbd2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:51:18.564 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:18.557+0000 7f4eb0b08640 1 --2- 172.21.15.39:0/4093904376 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564daabdd490 0x564daafbcd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:51:18.564 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:18.557+0000 7f4eb1309640 1 -- 172.21.15.39:0/4093904376 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564daabdd490 msgr2=0x564daafbcd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:51:18.564 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:18.557+0000 7f4eb1309640 1 --2- 172.21.15.39:0/4093904376 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564daabdd490 0x564daafbcd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:18.564 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:18.557+0000 7f4eb1309640 1 -- 172.21.15.39:0/4093904376 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564daabe5670 msgr2=0x564daafbd2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:51:18.564 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:18.557+0000 7f4eb1309640 1 --2- 172.21.15.39:0/4093904376 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564daabe5670 0x564daafbd2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:18.564 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:18.557+0000 7f4eb1309640 1 -- 172.21.15.39:0/4093904376 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564dab034dc0 con 0x564daac06350 2024-09-21T03:51:18.564 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:18.558+0000 7f4eb1309640 1 --2- 172.21.15.39:0/4093904376 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564daac06350 0x564daafbd810 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7f4e9c005940 tx=0x7f4e9c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=41638b355703a5fd server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:51:18.564 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:18.558+0000 7f4e8bfff640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-09-21T03:51:18.564 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:18.558+0000 7f4e8bfff640 1 -- 172.21.15.39:0/4093904376 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e9c05f020 con 0x564daac06350 2024-09-21T03:51:18.564 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:18.558+0000 7f4e8bfff640 1 -- 172.21.15.39:0/4093904376 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8484+0+0 (secure 0 0 0) 0x7f4e9c003a00 con 0x564daac06350 2024-09-21T03:51:18.564 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:18.558+0000 7f4eb92e0740 1 -- 172.21.15.39:0/4093904376 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4e9c04e070 con 0x564daac06350 2024-09-21T03:51:18.564 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:18.558+0000 7f4e8bfff640 1 -- 172.21.15.39:0/4093904376 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f4e9c06a050 con 0x564daac06350 2024-09-21T03:51:18.564 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:18.558+0000 7f4eb92e0740 10 client.16848 fetch_fsmap learned FSMap version 29 2024-09-21T03:51:18.564 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:18.558+0000 7f4eb92e0740 1 -- 172.21.15.39:0/4093904376 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x564dab035fe0 con 0x564daac06350 2024-09-21T03:51:18.564 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:18.558+0000 7f4e8bfff640 1 -- 172.21.15.39:0/4093904376 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4e9c04e070 con 0x564daac06350 2024-09-21T03:51:18.565 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:18.558+0000 7f4e8bfff640 1 -- 172.21.15.39:0/4093904376 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f4e9c03d450 con 0x564daac06350 2024-09-21T03:51:18.565 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:18.559+0000 7f4eb92e0740 10 client.16848 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T03:51:18.565 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:18.559+0000 7f4eb92e0740 10 client.16848 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T03:51:18.565 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:18.559+0000 7f4eb92e0740 20 client.16848 populate_metadata read hostname 'smithi039' 2024-09-21T03:51:18.565 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:18.559+0000 7f4eb92e0740 1 --2- 172.21.15.39:0/4093904376 >> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] conn(0x564daab9a080 0x564dab038770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:51:18.565 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:18.559+0000 7f4eb92e0740 4 client.16848 mds_command: new command op to 24695 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-21T03:51:18.565 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:18.559+0000 7f4eb92e0740 1 -- 172.21.15.39:0/4093904376 --> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x564daaf21860 con 0x564daab9a080 2024-09-21T03:51:18.565 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:18.559+0000 7f4eb0b08640 1 --2- 172.21.15.39:0/4093904376 >> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] conn(0x564daab9a080 0x564dab038770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:51:18.565 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:18.559+0000 7f4eb0b08640 1 --2- 172.21.15.39:0/4093904376 >> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] conn(0x564daab9a080 0x564dab038770 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=481e0db0b9b29717 server_cookie=59523fcbe86d3730 in_seq=0 out_seq=0 2024-09-21T03:51:18.565 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:18.559+0000 7f4e8bfff640 10 client.16848 ms_handle_connect on v2:172.21.15.150:6838/2949386907 2024-09-21T03:51:18.566 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:18.561+0000 7f4e8bfff640 1 -- 172.21.15.39:0/4093904376 <== mds.0 v2:172.21.15.150:6838/2949386907 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x564daaf21860 con 0x564daab9a080 2024-09-21T03:51:18.566 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:18.561+0000 7f4e8bfff640 10 client.16848 handle_command_reply: tid=0 2024-09-21T03:51:18.566 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:18.561+0000 7f4eb92e0740 1 client.16848 shutdown 2024-09-21T03:51:18.566 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:18.561+0000 7f4eb92e0740 1 -- 172.21.15.39:0/4093904376 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564daac06350 msgr2=0x564daafbd810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:51:18.566 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:18.561+0000 7f4eb92e0740 1 --2- 172.21.15.39:0/4093904376 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564daac06350 0x564daafbd810 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7f4e9c005940 tx=0x7f4e9c0046c0 comp rx=0 tx=0).stop 2024-09-21T03:51:18.566 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:18.561+0000 7f4eb92e0740 1 -- 172.21.15.39:0/4093904376 shutdown_connections 2024-09-21T03:51:18.566 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:18.561+0000 7f4eb92e0740 1 --2- 172.21.15.39:0/4093904376 >> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] conn(0x564daab9a080 0x564dab038770 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:18.567 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:18.561+0000 7f4eb92e0740 1 --2- 172.21.15.39:0/4093904376 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564daac06350 0x564daafbd810 unknown :-1 s=CLOSED pgs=858 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:18.567 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:18.561+0000 7f4eb92e0740 1 --2- 172.21.15.39:0/4093904376 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564daabe5670 0x564daafbd2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:18.567 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:18.561+0000 7f4eb92e0740 1 --2- 172.21.15.39:0/4093904376 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564daabdd490 0x564daafbcd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:18.567 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:18.561+0000 7f4eb92e0740 1 -- 172.21.15.39:0/4093904376 >> 172.21.15.39:0/4093904376 conn(0x564daabfa740 msgr2=0x564daafaebf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:51:18.567 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:18.561+0000 7f4e8bfff640 0 client.16848 ms_handle_reset on v2:172.21.15.150:6838/2949386907 2024-09-21T03:51:18.567 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:18.562+0000 7f4eb92e0740 1 -- 172.21.15.39:0/4093904376 shutdown_connections 2024-09-21T03:51:18.567 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:18.562+0000 7f4eb92e0740 1 -- 172.21.15.39:0/4093904376 wait complete. 2024-09-21T03:51:18.567 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:18.562+0000 7f4eb92e0740 20 client.16848 trim_cache size 0 max 16384 2024-09-21T03:51:18.567 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-09-21T03:51:18.567 INFO:teuthology.orchestra.run.smithi039.stdout: "return_code": 0, 2024-09-21T03:51:18.567 INFO:teuthology.orchestra.run.smithi039.stdout: "scrub_tag": "ae1fc4cf-5458-47d0-afcb-282b136dba15", 2024-09-21T03:51:18.568 INFO:teuthology.orchestra.run.smithi039.stdout: "mode": "asynchronous" 2024-09-21T03:51:18.568 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-09-21T03:51:18.568 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:18.562+0000 7f4eb2d6c640 1 -- 172.21.15.39:0/4226592613 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f4e940b6680 msgr2=0x7f4e940b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:51:18.568 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:18.562+0000 7f4eb2d6c640 1 --2- 172.21.15.39:0/4226592613 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f4e940b6680 0x7f4e940b8b90 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f4e9c006010 tx=0x7f4e9c0058d0 comp rx=0 tx=0).stop 2024-09-21T03:51:18.568 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:18.562+0000 7f4eb2d6c640 1 -- 172.21.15.39:0/4226592613 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4eac102a80 msgr2=0x7f4eac0ff690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:51:18.568 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:18.562+0000 7f4eb2d6c640 1 --2- 172.21.15.39:0/4226592613 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4eac102a80 0x7f4eac0ff690 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7f4ea400eab0 tx=0x7f4ea400eeb0 comp rx=0 tx=0).stop 2024-09-21T03:51:18.568 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:18.562+0000 7f4eb2d6c640 1 -- 172.21.15.39:0/4226592613 shutdown_connections 2024-09-21T03:51:18.569 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:18.562+0000 7f4eb2d6c640 1 --2- 172.21.15.39:0/4226592613 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f4e940b6680 0x7f4e940b8b90 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:18.569 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:18.562+0000 7f4eb2d6c640 1 --2- 172.21.15.39:0/4226592613 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4eac108d10 0x7f4eac100110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:18.569 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:18.562+0000 7f4eb2d6c640 1 --2- 172.21.15.39:0/4226592613 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4eac1083c0 0x7f4eac0ffbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:18.569 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:18.562+0000 7f4eb2d6c640 1 --2- 172.21.15.39:0/4226592613 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4eac102a80 0x7f4eac0ff690 unknown :-1 s=CLOSED pgs=854 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:18.569 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:18.562+0000 7f4eb2d6c640 1 -- 172.21.15.39:0/4226592613 >> 172.21.15.39:0/4226592613 conn(0x7f4eac079740 msgr2=0x7f4eac1019c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:51:18.569 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:18.563+0000 7f4eb2d6c640 1 -- 172.21.15.39:0/4226592613 shutdown_connections 2024-09-21T03:51:18.569 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:18.563+0000 7f4eb2d6c640 1 -- 172.21.15.39:0/4226592613 wait complete. 2024-09-21T03:51:19.334 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:51:19 smithi039 ceph-mon[31211]: pgmap v483: 97 pgs: 97 active+clean; 10 MiB data, 520 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 170 B/s wr, 9 op/s 2024-09-21T03:51:19.334 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:51:19 smithi039 ceph-mon[31211]: scrub queued for path: / 2024-09-21T03:51:19.334 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:51:19 smithi039 ceph-mon[31211]: scrub summary: idle+waiting paths [/] 2024-09-21T03:51:19.334 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:51:19 smithi039 ceph-mon[31211]: scrub summary: active paths [/] 2024-09-21T03:51:19.334 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:51:19 smithi039 ceph-mon[31211]: scrub summary: idle+waiting paths [/] 2024-09-21T03:51:19.421 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 51116d5c-77ca-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub status 2024-09-21T03:51:19.468 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:51:19 smithi161 ceph-mon[34533]: pgmap v483: 97 pgs: 97 active+clean; 10 MiB data, 520 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 170 B/s wr, 9 op/s 2024-09-21T03:51:19.468 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:51:19 smithi161 ceph-mon[34533]: scrub queued for path: / 2024-09-21T03:51:19.468 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:51:19 smithi161 ceph-mon[34533]: scrub summary: idle+waiting paths [/] 2024-09-21T03:51:19.468 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:51:19 smithi161 ceph-mon[34533]: scrub summary: active paths [/] 2024-09-21T03:51:19.468 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:51:19 smithi161 ceph-mon[34533]: scrub summary: idle+waiting paths [/] 2024-09-21T03:51:19.475 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:51:19 smithi150 ceph-mon[34289]: pgmap v483: 97 pgs: 97 active+clean; 10 MiB data, 520 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 170 B/s wr, 9 op/s 2024-09-21T03:51:19.475 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:51:19 smithi150 ceph-mon[34289]: scrub queued for path: / 2024-09-21T03:51:19.475 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:51:19 smithi150 ceph-mon[34289]: scrub summary: idle+waiting paths [/] 2024-09-21T03:51:19.475 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:51:19 smithi150 ceph-mon[34289]: scrub summary: active paths [/] 2024-09-21T03:51:19.475 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:51:19 smithi150 ceph-mon[34289]: scrub summary: idle+waiting paths [/] 2024-09-21T03:51:19.680 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/51116d5c-77ca-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:51:20.181 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:20.166+0000 7fa3d903d640 1 -- 172.21.15.39:0/2304411318 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa3d4103b60 msgr2=0x7fa3d4103f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:51:20.181 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:20.166+0000 7fa3d903d640 1 --2- 172.21.15.39:0/2304411318 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa3d4103b60 0x7fa3d4103f70 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7fa3c4009900 tx=0x7fa3c402a410 comp rx=0 tx=0).stop 2024-09-21T03:51:20.182 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:20.166+0000 7fa3d903d640 1 -- 172.21.15.39:0/2304411318 shutdown_connections 2024-09-21T03:51:20.182 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:20.166+0000 7fa3d903d640 1 --2- 172.21.15.39:0/2304411318 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa3d4109df0 0x7fa3d4110620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:20.182 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:20.166+0000 7fa3d903d640 1 --2- 172.21.15.39:0/2304411318 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa3d41094a0 0x7fa3d41098b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:20.182 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:20.166+0000 7fa3d903d640 1 --2- 172.21.15.39:0/2304411318 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa3d4103b60 0x7fa3d4103f70 unknown :-1 s=CLOSED pgs=859 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:20.182 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:20.166+0000 7fa3d903d640 1 -- 172.21.15.39:0/2304411318 >> 172.21.15.39:0/2304411318 conn(0x7fa3d407b850 msgr2=0x7fa3d407bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:51:20.182 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:20.166+0000 7fa3d903d640 1 -- 172.21.15.39:0/2304411318 shutdown_connections 2024-09-21T03:51:20.182 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:20.166+0000 7fa3d903d640 1 -- 172.21.15.39:0/2304411318 wait complete. 2024-09-21T03:51:20.182 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:20.168+0000 7fa3d903d640 1 Processor -- start 2024-09-21T03:51:20.182 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:20.168+0000 7fa3d903d640 1 -- start start 2024-09-21T03:51:20.182 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:20.168+0000 7fa3d903d640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa3d4103b60 0x7fa3d419a460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:51:20.182 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:20.168+0000 7fa3d903d640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa3d41094a0 0x7fa3d419aa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:51:20.183 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:20.168+0000 7fa3d903d640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa3d4109df0 0x7fa3d419fca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:51:20.183 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:20.168+0000 7fa3d903d640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fa3d4113ab0 con 0x7fa3d4103b60 2024-09-21T03:51:20.183 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:20.168+0000 7fa3d903d640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fa3d4113930 con 0x7fa3d41094a0 2024-09-21T03:51:20.183 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:20.168+0000 7fa3d903d640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fa3d4113c30 con 0x7fa3d4109df0 2024-09-21T03:51:20.183 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:20.168+0000 7fa3d2d9d640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa3d4103b60 0x7fa3d419a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:51:20.183 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:20.168+0000 7fa3d2d9d640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa3d4103b60 0x7fa3d419a460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:36328/0 (socket says 172.21.15.39:36328) 2024-09-21T03:51:20.183 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:20.168+0000 7fa3d2d9d640 1 -- 172.21.15.39:0/205326556 learned_addr learned my addr 172.21.15.39:0/205326556 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:51:20.183 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:20.169+0000 7fa3d259c640 1 --2- 172.21.15.39:0/205326556 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa3d41094a0 0x7fa3d419aa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:51:20.183 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:20.169+0000 7fa3d359e640 1 --2- 172.21.15.39:0/205326556 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa3d4109df0 0x7fa3d419fca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:51:20.183 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:20.169+0000 7fa3d2d9d640 1 -- 172.21.15.39:0/205326556 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa3d4109df0 msgr2=0x7fa3d419fca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:51:20.183 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:20.169+0000 7fa3d2d9d640 1 --2- 172.21.15.39:0/205326556 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa3d4109df0 0x7fa3d419fca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:20.183 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:20.169+0000 7fa3d2d9d640 1 -- 172.21.15.39:0/205326556 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa3d41094a0 msgr2=0x7fa3d419aa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:51:20.183 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:20.169+0000 7fa3d2d9d640 1 --2- 172.21.15.39:0/205326556 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa3d41094a0 0x7fa3d419aa00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:20.183 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:20.169+0000 7fa3d2d9d640 1 -- 172.21.15.39:0/205326556 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa3d41a03d0 con 0x7fa3d4103b60 2024-09-21T03:51:20.184 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:20.169+0000 7fa3d359e640 1 --2- 172.21.15.39:0/205326556 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa3d4109df0 0x7fa3d419fca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:51:20.184 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:20.169+0000 7fa3d2d9d640 1 --2- 172.21.15.39:0/205326556 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa3d4103b60 0x7fa3d419a460 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7fa3c4005940 tx=0x7fa3c40042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:51:20.184 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:20.169+0000 7fa3cbfff640 1 -- 172.21.15.39:0/205326556 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3c403f030 con 0x7fa3d4103b60 2024-09-21T03:51:20.184 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:20.169+0000 7fa3d903d640 1 -- 172.21.15.39:0/205326556 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa3d41a0660 con 0x7fa3d4103b60 2024-09-21T03:51:20.184 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:20.169+0000 7fa3cbfff640 1 -- 172.21.15.39:0/205326556 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa3c4003cb0 con 0x7fa3d4103b60 2024-09-21T03:51:20.184 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:20.169+0000 7fa3cbfff640 1 -- 172.21.15.39:0/205326556 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3c4035070 con 0x7fa3d4103b60 2024-09-21T03:51:20.184 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:20.169+0000 7fa3d903d640 1 -- 172.21.15.39:0/205326556 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa3d41a6e20 con 0x7fa3d4103b60 2024-09-21T03:51:20.184 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:20.172+0000 7fa3df5b1740 1 Processor -- start 2024-09-21T03:51:20.184 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:20.172+0000 7fa3df5b1740 1 -- start start 2024-09-21T03:51:20.184 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:20.172+0000 7fa3cbfff640 1 -- 172.21.15.39:0/205326556 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 27) ==== 154643+0+0 (secure 0 0 0) 0x7fa3c4042030 con 0x7fa3d4103b60 2024-09-21T03:51:20.184 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:20.172+0000 7fa3df5b1740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a89befe070 0x55a89c2a5650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:51:20.184 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:20.172+0000 7fa3df5b1740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a89bed51b0 0x55a89c2a5b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:51:20.184 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:20.172+0000 7fa3df5b1740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a89bedd390 0x55a89c2a60d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:51:20.185 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:20.172+0000 7fa3df5b1740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x55a89c21be50 con 0x55a89befe070 2024-09-21T03:51:20.185 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:20.172+0000 7fa3df5b1740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x55a89c20b280 con 0x55a89bed51b0 2024-09-21T03:51:20.185 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:20.173+0000 7fa3df5b1740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55a89c20f090 con 0x55a89bedd390 2024-09-21T03:51:20.185 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:20.173+0000 7fa3d359e640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a89befe070 0x55a89c2a5650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:51:20.185 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:20.173+0000 7fa3d359e640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a89befe070 0x55a89c2a5650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:36332/0 (socket says 172.21.15.39:36332) 2024-09-21T03:51:20.185 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:20.173+0000 7fa3d359e640 1 -- 172.21.15.39:0/602845949 learned_addr learned my addr 172.21.15.39:0/602845949 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:51:20.185 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:20.173+0000 7fa3d259c640 1 --2- 172.21.15.39:0/602845949 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a89bedd390 0x55a89c2a60d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:51:20.185 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:20.173+0000 7fa3d2d9d640 1 --2- 172.21.15.39:0/602845949 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a89bed51b0 0x55a89c2a5b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:51:20.185 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:20.173+0000 7fa3d359e640 1 -- 172.21.15.39:0/602845949 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a89bedd390 msgr2=0x55a89c2a60d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:51:20.185 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:20.173+0000 7fa3d359e640 1 --2- 172.21.15.39:0/602845949 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a89bedd390 0x55a89c2a60d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:20.185 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:20.173+0000 7fa3d359e640 1 -- 172.21.15.39:0/602845949 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a89bed51b0 msgr2=0x55a89c2a5b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:51:20.185 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:20.173+0000 7fa3d359e640 1 --2- 172.21.15.39:0/602845949 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a89bed51b0 0x55a89c2a5b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:20.186 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:20.173+0000 7fa3d359e640 1 -- 172.21.15.39:0/602845949 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a89bf69650 con 0x55a89befe070 2024-09-21T03:51:20.186 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:20.173+0000 7fa3d359e640 1 --2- 172.21.15.39:0/602845949 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a89befe070 0x55a89c2a5650 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7fa3cc008dd0 tx=0x7fa3cc00d710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d6112cb6aaf5a2d server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:51:20.186 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:20.173+0000 7fa3c97fa640 1 -- 172.21.15.39:0/602845949 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3cc00f830 con 0x55a89befe070 2024-09-21T03:51:20.186 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:20.173+0000 7fa3c97fa640 1 -- 172.21.15.39:0/602845949 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa3cc00de20 con 0x55a89befe070 2024-09-21T03:51:20.186 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:20.173+0000 7fa3c97fa640 1 -- 172.21.15.39:0/602845949 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3cc00ecf0 con 0x55a89befe070 2024-09-21T03:51:20.186 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:20.174+0000 7fa3cbfff640 1 --2- 172.21.15.39:0/205326556 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7fa3c00b6680 0x7fa3c00b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:51:20.186 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:20.174+0000 7fa3d2d9d640 1 --2- 172.21.15.39:0/205326556 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7fa3c00b6680 0x7fa3c00b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:51:20.186 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:20.174+0000 7fa3cbfff640 1 -- 172.21.15.39:0/205326556 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8484+0+0 (secure 0 0 0) 0x7fa3c4108060 con 0x7fa3d4103b60 2024-09-21T03:51:20.186 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:20.174+0000 7fa3df5b1740 1 -- 172.21.15.39:0/602845949 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a89befe070 msgr2=0x55a89c2a5650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:51:20.186 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:20.174+0000 7fa3d2d9d640 1 --2- 172.21.15.39:0/205326556 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7fa3c00b6680 0x7fa3c00b8b90 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fa3c4105880 tx=0x7fa3c4032f60 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:51:20.186 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:20.174+0000 7fa3df5b1740 1 --2- 172.21.15.39:0/602845949 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a89befe070 0x55a89c2a5650 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7fa3cc008dd0 tx=0x7fa3cc00d710 comp rx=0 tx=0).stop 2024-09-21T03:51:20.186 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:20.175+0000 7fa3df5b1740 1 -- 172.21.15.39:0/602845949 shutdown_connections 2024-09-21T03:51:20.186 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:20.175+0000 7fa3df5b1740 1 --2- 172.21.15.39:0/602845949 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a89bedd390 0x55a89c2a60d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:20.187 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:20.175+0000 7fa3df5b1740 1 --2- 172.21.15.39:0/602845949 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a89bed51b0 0x55a89c2a5b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:20.187 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:20.175+0000 7fa3df5b1740 1 --2- 172.21.15.39:0/602845949 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a89befe070 0x55a89c2a5650 unknown :-1 s=CLOSED pgs=861 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:20.187 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:20.175+0000 7fa3df5b1740 1 -- 172.21.15.39:0/602845949 >> 172.21.15.39:0/602845949 conn(0x55a89bef2460 msgr2=0x55a89c2a3540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:51:20.187 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:20.175+0000 7fa3df5b1740 1 -- 172.21.15.39:0/602845949 shutdown_connections 2024-09-21T03:51:20.187 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:20.175+0000 7fa3df5b1740 1 -- 172.21.15.39:0/602845949 wait complete. 2024-09-21T03:51:20.187 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:20.175+0000 7fa3df5b1740 1 -- start start 2024-09-21T03:51:20.187 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:20.175+0000 7fa3df5b1740 1 Processor -- start 2024-09-21T03:51:20.187 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:20.175+0000 7fa3df5b1740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a89bed51b0 0x55a89c2b0d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:51:20.187 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:20.175+0000 7fa3df5b1740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a89bedd390 0x55a89c2b1250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:51:20.187 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:20.175+0000 7fa3df5b1740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a89befe070 0x55a89c2bd120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:51:20.187 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:20.175+0000 7fa3df5b1740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x55a89c06c700 con 0x55a89bed51b0 2024-09-21T03:51:20.187 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:20.175+0000 7fa3df5b1740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x55a89c1a6110 con 0x55a89bedd390 2024-09-21T03:51:20.187 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:20.175+0000 7fa3df5b1740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55a89c2b3e30 con 0x55a89befe070 2024-09-21T03:51:20.188 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:20.176+0000 7fa3df5b1740 10 client.0 _ll_register_callbacks cb 0x55a89c1bf990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:51:20.188 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:20.176+0000 7fa3d359e640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a89bed51b0 0x55a89c2b0d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:51:20.188 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:20.176+0000 7fa3d359e640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a89bed51b0 0x55a89c2b0d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:36342/0 (socket says 172.21.15.39:36342) 2024-09-21T03:51:20.188 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:20.176+0000 7fa3d359e640 1 -- 172.21.15.39:0/1562654426 learned_addr learned my addr 172.21.15.39:0/1562654426 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:51:20.188 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:20.176+0000 7fa3d259c640 1 --2- 172.21.15.39:0/1562654426 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a89bedd390 0x55a89c2b1250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:51:20.188 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:20.176+0000 7fa3d359e640 1 -- 172.21.15.39:0/1562654426 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a89befe070 msgr2=0x55a89c2bd120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:51:20.188 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:20.176+0000 7fa3d359e640 1 --2- 172.21.15.39:0/1562654426 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a89befe070 0x55a89c2bd120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:20.188 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:20.176+0000 7fa3d359e640 1 -- 172.21.15.39:0/1562654426 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a89bedd390 msgr2=0x55a89c2b1250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:51:20.188 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:20.176+0000 7fa3d359e640 1 --2- 172.21.15.39:0/1562654426 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a89bedd390 0x55a89c2b1250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:20.188 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:20.176+0000 7fa3d359e640 1 -- 172.21.15.39:0/1562654426 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a89c1d3c40 con 0x55a89bed51b0 2024-09-21T03:51:20.188 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:20.176+0000 7fa3d259c640 1 --2- 172.21.15.39:0/1562654426 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a89bedd390 0x55a89c2b1250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:51:20.188 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:20.176+0000 7fa3d359e640 1 --2- 172.21.15.39:0/1562654426 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a89bed51b0 0x55a89c2b0d10 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7fa3cc000f80 tx=0x7fa3cc00ec60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b2220fe1ebfc013d server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:51:20.188 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:20.176+0000 7fa3ba7fc640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-09-21T03:51:20.189 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:20.176+0000 7fa3ba7fc640 1 -- 172.21.15.39:0/1562654426 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3cc028060 con 0x55a89bed51b0 2024-09-21T03:51:20.189 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:20.176+0000 7fa3ba7fc640 1 -- 172.21.15.39:0/1562654426 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8484+0+0 (secure 0 0 0) 0x7fa3cc00f630 con 0x55a89bed51b0 2024-09-21T03:51:20.189 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:20.176+0000 7fa3df5b1740 1 -- 172.21.15.39:0/1562654426 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa3cc00de20 con 0x55a89bed51b0 2024-09-21T03:51:20.189 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:20.177+0000 7fa3ba7fc640 1 -- 172.21.15.39:0/1562654426 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fa3cc00f830 con 0x55a89bed51b0 2024-09-21T03:51:20.189 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:20.177+0000 7fa3df5b1740 10 client.16860 fetch_fsmap learned FSMap version 29 2024-09-21T03:51:20.189 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:20.177+0000 7fa3df5b1740 1 -- 172.21.15.39:0/1562654426 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55a89c2bfae0 con 0x55a89bed51b0 2024-09-21T03:51:20.189 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:20.177+0000 7fa3ba7fc640 1 -- 172.21.15.39:0/1562654426 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa3cc00de20 con 0x55a89bed51b0 2024-09-21T03:51:20.189 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:20.177+0000 7fa3ba7fc640 1 -- 172.21.15.39:0/1562654426 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fa3cc023a40 con 0x55a89bed51b0 2024-09-21T03:51:20.189 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:20.177+0000 7fa3df5b1740 10 client.16860 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T03:51:20.189 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:20.177+0000 7fa3df5b1740 10 client.16860 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T03:51:20.189 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:20.177+0000 7fa3df5b1740 20 client.16860 populate_metadata read hostname 'smithi039' 2024-09-21T03:51:20.189 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:20.177+0000 7fa3df5b1740 1 --2- 172.21.15.39:0/1562654426 >> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] conn(0x55a89bebc9f0 0x55a89c2c2f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:51:20.189 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:20.177+0000 7fa3df5b1740 4 client.16860 mds_command: new command op to 24695 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:51:20.190 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:20.177+0000 7fa3df5b1740 1 -- 172.21.15.39:0/1562654426 --> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a89c21dda0 con 0x55a89bebc9f0 2024-09-21T03:51:20.190 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:20.177+0000 7fa3d259c640 1 --2- 172.21.15.39:0/1562654426 >> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] conn(0x55a89bebc9f0 0x55a89c2c2f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:51:20.190 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:20.178+0000 7fa3d259c640 1 --2- 172.21.15.39:0/1562654426 >> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] conn(0x55a89bebc9f0 0x55a89c2c2f50 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=58e5c31bc5431be1 server_cookie=d403d5ab3f75f00a in_seq=0 out_seq=0 2024-09-21T03:51:20.190 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:20.178+0000 7fa3ba7fc640 10 client.16860 ms_handle_connect on v2:172.21.15.150:6838/2949386907 2024-09-21T03:51:20.190 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:20.179+0000 7fa3ba7fc640 1 -- 172.21.15.39:0/1562654426 <== mds.0 v2:172.21.15.150:6838/2949386907 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55a89c21dda0 con 0x55a89bebc9f0 2024-09-21T03:51:20.190 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:20.179+0000 7fa3ba7fc640 10 client.16860 handle_command_reply: tid=0 2024-09-21T03:51:20.190 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:20.179+0000 7fa3df5b1740 1 client.16860 shutdown 2024-09-21T03:51:20.190 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:20.179+0000 7fa3df5b1740 1 -- 172.21.15.39:0/1562654426 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a89bed51b0 msgr2=0x55a89c2b0d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:51:20.190 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:20.179+0000 7fa3df5b1740 1 --2- 172.21.15.39:0/1562654426 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a89bed51b0 0x55a89c2b0d10 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7fa3cc000f80 tx=0x7fa3cc00ec60 comp rx=0 tx=0).stop 2024-09-21T03:51:20.190 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:20.179+0000 7fa3df5b1740 1 -- 172.21.15.39:0/1562654426 shutdown_connections 2024-09-21T03:51:20.190 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:20.179+0000 7fa3df5b1740 1 --2- 172.21.15.39:0/1562654426 >> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] conn(0x55a89bebc9f0 0x55a89c2c2f50 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:20.190 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:20.179+0000 7fa3df5b1740 1 --2- 172.21.15.39:0/1562654426 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a89befe070 0x55a89c2bd120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:20.190 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:20.179+0000 7fa3df5b1740 1 --2- 172.21.15.39:0/1562654426 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a89bedd390 0x55a89c2b1250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:20.191 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:20.179+0000 7fa3ba7fc640 0 client.16860 ms_handle_reset on v2:172.21.15.150:6838/2949386907 2024-09-21T03:51:20.191 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:20.179+0000 7fa3df5b1740 1 --2- 172.21.15.39:0/1562654426 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a89bed51b0 0x55a89c2b0d10 unknown :-1 s=CLOSED pgs=862 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:20.191 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:20.179+0000 7fa3df5b1740 1 -- 172.21.15.39:0/1562654426 >> 172.21.15.39:0/1562654426 conn(0x55a89bef2460 msgr2=0x55a89c2a8830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:51:20.191 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:20.179+0000 7fa3df5b1740 1 -- 172.21.15.39:0/1562654426 shutdown_connections 2024-09-21T03:51:20.191 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:20.179+0000 7fa3df5b1740 1 -- 172.21.15.39:0/1562654426 wait complete. 2024-09-21T03:51:20.191 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:20.179+0000 7fa3df5b1740 20 client.16860 trim_cache size 0 max 16384 2024-09-21T03:51:20.198 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:20.197+0000 7fa3df5b1740 1 Processor -- start 2024-09-21T03:51:20.198 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:20.197+0000 7fa3df5b1740 1 -- start start 2024-09-21T03:51:20.200 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:20.197+0000 7fa3df5b1740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a89bebc9f0 0x55a89c2a6d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:51:20.200 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:20.197+0000 7fa3df5b1740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a89bed51b0 0x55a89c2a2b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:51:20.200 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:20.197+0000 7fa3df5b1740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a89bedd390 0x55a89c2a30b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:51:20.200 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:20.197+0000 7fa3df5b1740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fa3d41af6f0 con 0x55a89bebc9f0 2024-09-21T03:51:20.200 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:20.197+0000 7fa3df5b1740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fa3d41b1230 con 0x55a89bed51b0 2024-09-21T03:51:20.200 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:20.197+0000 7fa3df5b1740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fa3d41afdc0 con 0x55a89bedd390 2024-09-21T03:51:20.200 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:20.197+0000 7fa3d359e640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a89bebc9f0 0x55a89c2a6d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:51:20.200 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:20.197+0000 7fa3d359e640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a89bebc9f0 0x55a89c2a6d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:36350/0 (socket says 172.21.15.39:36350) 2024-09-21T03:51:20.200 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:20.197+0000 7fa3d359e640 1 -- 172.21.15.39:0/1081457250 learned_addr learned my addr 172.21.15.39:0/1081457250 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:51:20.201 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:20.197+0000 7fa3d259c640 1 --2- 172.21.15.39:0/1081457250 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a89bed51b0 0x55a89c2a2b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:51:20.201 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:20.198+0000 7fa3d359e640 1 -- 172.21.15.39:0/1081457250 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a89bedd390 msgr2=0x55a89c2a30b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:51:20.201 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:20.198+0000 7fa3d359e640 1 --2- 172.21.15.39:0/1081457250 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a89bedd390 0x55a89c2a30b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:20.201 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:20.198+0000 7fa3d359e640 1 -- 172.21.15.39:0/1081457250 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a89bed51b0 msgr2=0x55a89c2a2b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:51:20.201 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:20.198+0000 7fa3d359e640 1 --2- 172.21.15.39:0/1081457250 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a89bed51b0 0x55a89c2a2b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:20.201 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:20.198+0000 7fa3d359e640 1 -- 172.21.15.39:0/1081457250 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a89c2a7b90 con 0x55a89bebc9f0 2024-09-21T03:51:20.201 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:20.198+0000 7fa3d259c640 1 --2- 172.21.15.39:0/1081457250 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a89bed51b0 0x55a89c2a2b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:51:20.201 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:20.198+0000 7fa3d359e640 1 --2- 172.21.15.39:0/1081457250 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a89bebc9f0 0x55a89c2a6d90 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x7fa3cc00d750 tx=0x7fa3cc0085b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a8b4538f0af90f3 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:51:20.201 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:20.198+0000 7fa3c8ff9640 1 -- 172.21.15.39:0/1081457250 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3cc028070 con 0x55a89bebc9f0 2024-09-21T03:51:20.201 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:20.198+0000 7fa3c8ff9640 1 -- 172.21.15.39:0/1081457250 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa3cc008950 con 0x55a89bebc9f0 2024-09-21T03:51:20.201 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:20.198+0000 7fa3c8ff9640 1 -- 172.21.15.39:0/1081457250 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3cc02cb00 con 0x55a89bebc9f0 2024-09-21T03:51:20.205 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:20.199+0000 7fa3df5b1740 1 -- 172.21.15.39:0/1081457250 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a89bebc9f0 msgr2=0x55a89c2a6d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:51:20.205 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:20.199+0000 7fa3df5b1740 1 --2- 172.21.15.39:0/1081457250 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a89bebc9f0 0x55a89c2a6d90 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x7fa3cc00d750 tx=0x7fa3cc0085b0 comp rx=0 tx=0).stop 2024-09-21T03:51:20.205 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:20.199+0000 7fa3df5b1740 1 -- 172.21.15.39:0/1081457250 shutdown_connections 2024-09-21T03:51:20.205 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:20.199+0000 7fa3df5b1740 1 --2- 172.21.15.39:0/1081457250 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a89bedd390 0x55a89c2a30b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:20.205 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:20.199+0000 7fa3df5b1740 1 --2- 172.21.15.39:0/1081457250 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a89bed51b0 0x55a89c2a2b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:20.205 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:20.199+0000 7fa3df5b1740 1 --2- 172.21.15.39:0/1081457250 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a89bebc9f0 0x55a89c2a6d90 unknown :-1 s=CLOSED pgs=863 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:20.205 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:20.199+0000 7fa3df5b1740 1 -- 172.21.15.39:0/1081457250 >> 172.21.15.39:0/1081457250 conn(0x55a89bef2460 msgr2=0x55a89c2a6680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:51:20.206 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:20.199+0000 7fa3df5b1740 1 -- 172.21.15.39:0/1081457250 shutdown_connections 2024-09-21T03:51:20.206 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:20.199+0000 7fa3df5b1740 1 -- 172.21.15.39:0/1081457250 wait complete. 2024-09-21T03:51:20.206 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:20.199+0000 7fa3df5b1740 1 -- start start 2024-09-21T03:51:20.206 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:20.199+0000 7fa3df5b1740 1 Processor -- start 2024-09-21T03:51:20.206 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:20.200+0000 7fa3df5b1740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a89bebc9f0 0x55a89c2c6800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:51:20.206 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:20.200+0000 7fa3df5b1740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a89bed51b0 0x55a89c2c3cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:51:20.206 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:20.200+0000 7fa3df5b1740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a89bedd390 0x55a89c2c42d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:51:20.206 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:20.200+0000 7fa3df5b1740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fa3d41b0b60 con 0x55a89bebc9f0 2024-09-21T03:51:20.206 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:20.200+0000 7fa3df5b1740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fa3d41b0490 con 0x55a89bedd390 2024-09-21T03:51:20.206 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:20.200+0000 7fa3df5b1740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fa3d41ad6d0 con 0x55a89bed51b0 2024-09-21T03:51:20.206 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:20.200+0000 7fa3df5b1740 10 client.0 _ll_register_callbacks cb 0x7fa3bc00b3a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:51:20.206 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:20.200+0000 7fa3d359e640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a89bebc9f0 0x55a89c2c6800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:51:20.207 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:20.200+0000 7fa3d359e640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a89bebc9f0 0x55a89c2c6800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:36358/0 (socket says 172.21.15.39:36358) 2024-09-21T03:51:20.207 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:20.200+0000 7fa3d359e640 1 -- 172.21.15.39:0/676873869 learned_addr learned my addr 172.21.15.39:0/676873869 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:51:20.207 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:20.200+0000 7fa3d259c640 1 --2- 172.21.15.39:0/676873869 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a89bed51b0 0x55a89c2c3cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:51:20.207 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:20.200+0000 7fa3d359e640 1 -- 172.21.15.39:0/676873869 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a89bed51b0 msgr2=0x55a89c2c3cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:51:20.207 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:20.200+0000 7fa3d359e640 1 --2- 172.21.15.39:0/676873869 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a89bed51b0 0x55a89c2c3cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:20.207 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:20.200+0000 7fa3d359e640 1 -- 172.21.15.39:0/676873869 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a89bedd390 msgr2=0x55a89c2c42d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:51:20.207 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:20.200+0000 7fa3d359e640 1 --2- 172.21.15.39:0/676873869 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a89bedd390 0x55a89c2c42d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:20.207 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:20.200+0000 7fa3d359e640 1 -- 172.21.15.39:0/676873869 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a89c2c4a30 con 0x55a89bebc9f0 2024-09-21T03:51:20.207 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:20.200+0000 7fa3d359e640 1 --2- 172.21.15.39:0/676873869 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a89bebc9f0 0x55a89c2c6800 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7fa3cc023ee0 tx=0x7fa3cc00e9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bcf7a48158c6a0f5 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:51:20.207 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:20.200+0000 7fa3ba7fc640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-09-21T03:51:20.207 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:20.201+0000 7fa3ba7fc640 1 -- 172.21.15.39:0/676873869 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3cc03b020 con 0x55a89bebc9f0 2024-09-21T03:51:20.207 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:20.201+0000 7fa3ba7fc640 1 -- 172.21.15.39:0/676873869 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8484+0+0 (secure 0 0 0) 0x7fa3cc00fbd0 con 0x55a89bebc9f0 2024-09-21T03:51:20.208 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:20.201+0000 7fa3df5b1740 1 -- 172.21.15.39:0/676873869 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa3cc008950 con 0x55a89bebc9f0 2024-09-21T03:51:20.208 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:20.201+0000 7fa3ba7fc640 1 -- 172.21.15.39:0/676873869 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fa3cc028050 con 0x55a89bebc9f0 2024-09-21T03:51:20.208 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:20.201+0000 7fa3df5b1740 10 client.16866 fetch_fsmap learned FSMap version 29 2024-09-21T03:51:20.208 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:20.201+0000 7fa3df5b1740 1 -- 172.21.15.39:0/676873869 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55a89c32d6b0 con 0x55a89bebc9f0 2024-09-21T03:51:20.208 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:20.201+0000 7fa3ba7fc640 1 -- 172.21.15.39:0/676873869 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa3cc008950 con 0x55a89bebc9f0 2024-09-21T03:51:20.208 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:20.201+0000 7fa3ba7fc640 1 -- 172.21.15.39:0/676873869 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fa3cc023710 con 0x55a89bebc9f0 2024-09-21T03:51:20.208 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:20.201+0000 7fa3df5b1740 10 client.16866 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T03:51:20.208 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:20.201+0000 7fa3df5b1740 10 client.16866 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T03:51:20.208 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:20.201+0000 7fa3df5b1740 20 client.16866 populate_metadata read hostname 'smithi039' 2024-09-21T03:51:20.208 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:20.201+0000 7fa3df5b1740 1 --2- 172.21.15.39:0/676873869 >> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] conn(0x55a89befe070 0x55a89c32fdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:51:20.208 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:20.201+0000 7fa3df5b1740 4 client.16866 mds_command: new command op to 24695 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-21T03:51:20.208 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:20.201+0000 7fa3df5b1740 1 -- 172.21.15.39:0/676873869 --> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55a89c219850 con 0x55a89befe070 2024-09-21T03:51:20.208 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:20.202+0000 7fa3d259c640 1 --2- 172.21.15.39:0/676873869 >> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] conn(0x55a89befe070 0x55a89c32fdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:51:20.209 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:20.202+0000 7fa3ba7fc640 10 client.16866 ms_handle_connect on v2:172.21.15.150:6838/2949386907 2024-09-21T03:51:20.209 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:20.202+0000 7fa3d259c640 1 --2- 172.21.15.39:0/676873869 >> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] conn(0x55a89befe070 0x55a89c32fdb0 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=37786f4ba9c4cc2b server_cookie=e170920d454a9823 in_seq=0 out_seq=0 2024-09-21T03:51:20.209 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:20.203+0000 7fa3ba7fc640 1 -- 172.21.15.39:0/676873869 <== mds.0 v2:172.21.15.150:6838/2949386907 1 ==== command_reply(tid 0: 0 ) ==== 8+0+2930 (crc 0 0 0) 0x55a89c219850 con 0x55a89befe070 2024-09-21T03:51:20.209 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:20.203+0000 7fa3ba7fc640 10 client.16866 handle_command_reply: tid=0 2024-09-21T03:51:20.209 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:20.203+0000 7fa3df5b1740 1 client.16866 shutdown 2024-09-21T03:51:20.209 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:20.203+0000 7fa3df5b1740 1 -- 172.21.15.39:0/676873869 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a89bebc9f0 msgr2=0x55a89c2c6800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:51:20.209 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:20.203+0000 7fa3df5b1740 1 --2- 172.21.15.39:0/676873869 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a89bebc9f0 0x55a89c2c6800 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7fa3cc023ee0 tx=0x7fa3cc00e9c0 comp rx=0 tx=0).stop 2024-09-21T03:51:20.209 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:20.203+0000 7fa3df5b1740 1 -- 172.21.15.39:0/676873869 shutdown_connections 2024-09-21T03:51:20.209 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:20.203+0000 7fa3df5b1740 1 --2- 172.21.15.39:0/676873869 >> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] conn(0x55a89befe070 0x55a89c32fdb0 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:20.209 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:20.203+0000 7fa3df5b1740 1 --2- 172.21.15.39:0/676873869 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a89bedd390 0x55a89c2c42d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:20.209 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:20.203+0000 7fa3df5b1740 1 --2- 172.21.15.39:0/676873869 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a89bed51b0 0x55a89c2c3cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:20.209 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:20.203+0000 7fa3df5b1740 1 --2- 172.21.15.39:0/676873869 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a89bebc9f0 0x55a89c2c6800 unknown :-1 s=CLOSED pgs=864 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:20.209 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:20.203+0000 7fa3ba7fc640 0 client.16866 ms_handle_reset on v2:172.21.15.150:6838/2949386907 2024-09-21T03:51:20.209 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:20.203+0000 7fa3df5b1740 1 -- 172.21.15.39:0/676873869 >> 172.21.15.39:0/676873869 conn(0x55a89bef2460 msgr2=0x55a89c2a6f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:51:20.209 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:20.203+0000 7fa3df5b1740 1 -- 172.21.15.39:0/676873869 shutdown_connections 2024-09-21T03:51:20.210 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:20.203+0000 7fa3df5b1740 1 -- 172.21.15.39:0/676873869 wait complete. 2024-09-21T03:51:20.210 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:20.203+0000 7fa3df5b1740 20 client.16866 trim_cache size 0 max 16384 2024-09-21T03:51:20.210 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-09-21T03:51:20.210 INFO:teuthology.orchestra.run.smithi039.stdout: "status": "no active scrubs running", 2024-09-21T03:51:20.210 INFO:teuthology.orchestra.run.smithi039.stdout: "scrubs": {}, 2024-09-21T03:51:20.210 INFO:teuthology.orchestra.run.smithi039.stdout: "scrub_stats": { 2024-09-21T03:51:20.210 INFO:teuthology.orchestra.run.smithi039.stdout: "ae1fc4cf-5458-47d0-afcb-282b136dba15": { 2024-09-21T03:51:20.211 INFO:teuthology.orchestra.run.smithi039.stdout: "start_time": "2024-09-21T03:51:18.560378+0000", 2024-09-21T03:51:20.211 INFO:teuthology.orchestra.run.smithi039.stdout: "path": "/", 2024-09-21T03:51:20.211 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_started": 0, 2024-09-21T03:51:20.211 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_passed": 0, 2024-09-21T03:51:20.211 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_failed": 0, 2024-09-21T03:51:20.211 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_skipped": 0 2024-09-21T03:51:20.211 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:51:20.211 INFO:teuthology.orchestra.run.smithi039.stdout: "160750d8-401f-4810-866d-e63e7e4bc3e0": { 2024-09-21T03:51:20.211 INFO:teuthology.orchestra.run.smithi039.stdout: "start_time": "2024-09-21T03:51:04.208673+0000", 2024-09-21T03:51:20.211 INFO:teuthology.orchestra.run.smithi039.stdout: "path": "/", 2024-09-21T03:51:20.211 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_started": 0, 2024-09-21T03:51:20.211 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_passed": 0, 2024-09-21T03:51:20.211 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_failed": 0, 2024-09-21T03:51:20.211 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_skipped": 0 2024-09-21T03:51:20.212 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:51:20.212 INFO:teuthology.orchestra.run.smithi039.stdout: "74ab917e-ecea-4778-a1c3-c184214fb24c": { 2024-09-21T03:51:20.212 INFO:teuthology.orchestra.run.smithi039.stdout: "start_time": "2024-09-21T03:50:17.726630+0000", 2024-09-21T03:51:20.212 INFO:teuthology.orchestra.run.smithi039.stdout: "path": "/", 2024-09-21T03:51:20.212 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_started": 0, 2024-09-21T03:51:20.212 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_passed": 0, 2024-09-21T03:51:20.212 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_failed": 0, 2024-09-21T03:51:20.212 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_skipped": 0 2024-09-21T03:51:20.212 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:51:20.212 INFO:teuthology.orchestra.run.smithi039.stdout: "6e760f1f-9048-4f92-bd66-d708fbef75b3": { 2024-09-21T03:51:20.213 INFO:teuthology.orchestra.run.smithi039.stdout: "start_time": "2024-09-21T03:50:03.711918+0000", 2024-09-21T03:51:20.213 INFO:teuthology.orchestra.run.smithi039.stdout: "path": "/", 2024-09-21T03:51:20.213 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_started": 0, 2024-09-21T03:51:20.213 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_passed": 0, 2024-09-21T03:51:20.213 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_failed": 0, 2024-09-21T03:51:20.213 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_skipped": 0 2024-09-21T03:51:20.213 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:51:20.213 INFO:teuthology.orchestra.run.smithi039.stdout: "e28ad743-2b22-45b3-8799-a638c2883623": { 2024-09-21T03:51:20.213 INFO:teuthology.orchestra.run.smithi039.stdout: "start_time": "2024-09-21T03:48:31.448821+0000", 2024-09-21T03:51:20.213 INFO:teuthology.orchestra.run.smithi039.stdout: "path": "/", 2024-09-21T03:51:20.213 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_started": 0, 2024-09-21T03:51:20.213 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_passed": 0, 2024-09-21T03:51:20.213 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_failed": 0, 2024-09-21T03:51:20.213 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_skipped": 0 2024-09-21T03:51:20.214 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:51:20.214 INFO:teuthology.orchestra.run.smithi039.stdout: "29411307-eb8d-4339-8d2b-47ebd32c0080": { 2024-09-21T03:51:20.214 INFO:teuthology.orchestra.run.smithi039.stdout: "start_time": "2024-09-21T03:47:38.092920+0000", 2024-09-21T03:51:20.214 INFO:teuthology.orchestra.run.smithi039.stdout: "path": "/", 2024-09-21T03:51:20.214 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_started": 0, 2024-09-21T03:51:20.214 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_passed": 0, 2024-09-21T03:51:20.214 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_failed": 0, 2024-09-21T03:51:20.214 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_skipped": 0 2024-09-21T03:51:20.214 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:51:20.214 INFO:teuthology.orchestra.run.smithi039.stdout: "9860c2f4-b7b1-47aa-865a-9a1c4434a5f6": { 2024-09-21T03:51:20.214 INFO:teuthology.orchestra.run.smithi039.stdout: "start_time": "2024-09-21T03:49:17.718867+0000", 2024-09-21T03:51:20.214 INFO:teuthology.orchestra.run.smithi039.stdout: "path": "/", 2024-09-21T03:51:20.214 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_started": 0, 2024-09-21T03:51:20.215 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_passed": 0, 2024-09-21T03:51:20.215 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_failed": 0, 2024-09-21T03:51:20.215 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_skipped": 0 2024-09-21T03:51:20.215 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:51:20.215 INFO:teuthology.orchestra.run.smithi039.stdout: "c7e2b8e4-057d-435f-b2bd-b633de98035f": { 2024-09-21T03:51:20.215 INFO:teuthology.orchestra.run.smithi039.stdout: "start_time": "2024-09-21T03:47:19.400305+0000", 2024-09-21T03:51:20.215 INFO:teuthology.orchestra.run.smithi039.stdout: "path": "/", 2024-09-21T03:51:20.215 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_started": 0, 2024-09-21T03:51:20.215 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_passed": 0, 2024-09-21T03:51:20.215 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_failed": 0, 2024-09-21T03:51:20.215 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_skipped": 0 2024-09-21T03:51:20.215 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:51:20.215 INFO:teuthology.orchestra.run.smithi039.stdout: "10728de8-c49d-4edc-be2b-639249255e94": { 2024-09-21T03:51:20.215 INFO:teuthology.orchestra.run.smithi039.stdout: "start_time": "2024-09-21T03:47:03.965623+0000", 2024-09-21T03:51:20.216 INFO:teuthology.orchestra.run.smithi039.stdout: "path": "/", 2024-09-21T03:51:20.216 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_started": 0, 2024-09-21T03:51:20.216 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_passed": 0, 2024-09-21T03:51:20.216 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_failed": 0, 2024-09-21T03:51:20.216 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_skipped": 0 2024-09-21T03:51:20.216 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:51:20.216 INFO:teuthology.orchestra.run.smithi039.stdout: "02509e4f-3911-4983-b16f-df8ded0b9998": { 2024-09-21T03:51:20.216 INFO:teuthology.orchestra.run.smithi039.stdout: "start_time": "2024-09-21T03:46:17.889576+0000", 2024-09-21T03:51:20.216 INFO:teuthology.orchestra.run.smithi039.stdout: "path": "/", 2024-09-21T03:51:20.216 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_started": 0, 2024-09-21T03:51:20.216 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_passed": 0, 2024-09-21T03:51:20.216 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_failed": 0, 2024-09-21T03:51:20.216 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_skipped": 0 2024-09-21T03:51:20.217 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-21T03:51:20.217 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-21T03:51:20.217 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-09-21T03:51:20.217 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:20.204+0000 7fa3d903d640 1 -- 172.21.15.39:0/205326556 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7fa3c00b6680 msgr2=0x7fa3c00b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:51:20.217 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:20.204+0000 7fa3d903d640 1 --2- 172.21.15.39:0/205326556 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7fa3c00b6680 0x7fa3c00b8b90 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fa3c4105880 tx=0x7fa3c4032f60 comp rx=0 tx=0).stop 2024-09-21T03:51:20.217 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:20.204+0000 7fa3d903d640 1 -- 172.21.15.39:0/205326556 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa3d4103b60 msgr2=0x7fa3d419a460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:51:20.218 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:20.204+0000 7fa3d903d640 1 --2- 172.21.15.39:0/205326556 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa3d4103b60 0x7fa3d419a460 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7fa3c4005940 tx=0x7fa3c40042f0 comp rx=0 tx=0).stop 2024-09-21T03:51:20.218 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:20.204+0000 7fa3d903d640 1 -- 172.21.15.39:0/205326556 shutdown_connections 2024-09-21T03:51:20.218 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:20.204+0000 7fa3d903d640 1 --2- 172.21.15.39:0/205326556 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7fa3c00b6680 0x7fa3c00b8b90 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:20.218 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:20.204+0000 7fa3d903d640 1 --2- 172.21.15.39:0/205326556 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa3d4109df0 0x7fa3d419fca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:20.218 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:20.204+0000 7fa3d903d640 1 --2- 172.21.15.39:0/205326556 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa3d41094a0 0x7fa3d419aa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:20.218 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:20.204+0000 7fa3d903d640 1 --2- 172.21.15.39:0/205326556 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa3d4103b60 0x7fa3d419a460 unknown :-1 s=CLOSED pgs=860 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:20.218 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:20.204+0000 7fa3d903d640 1 -- 172.21.15.39:0/205326556 >> 172.21.15.39:0/205326556 conn(0x7fa3d407b850 msgr2=0x7fa3d4107530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:51:20.218 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:20.204+0000 7fa3d903d640 1 -- 172.21.15.39:0/205326556 shutdown_connections 2024-09-21T03:51:20.218 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:20.204+0000 7fa3d903d640 1 -- 172.21.15.39:0/205326556 wait complete. 2024-09-21T03:51:20.334 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:51:20 smithi039 ceph-mon[31211]: scrub summary: idle 2024-09-21T03:51:20.468 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:51:20 smithi161 ceph-mon[34533]: scrub summary: idle 2024-09-21T03:51:20.475 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:51:20 smithi150 ceph-mon[34289]: scrub summary: idle 2024-09-21T03:51:20.990 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-21T03:51:20.990 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 51116d5c-77ca-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T03:51:21.249 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/51116d5c-77ca-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:51:21.334 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:51:21 smithi039 ceph-mon[31211]: pgmap v484: 97 pgs: 97 active+clean; 10 MiB data, 520 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 170 B/s wr, 11 op/s 2024-09-21T03:51:21.467 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:51:21 smithi161 ceph-mon[34533]: pgmap v484: 97 pgs: 97 active+clean; 10 MiB data, 520 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 170 B/s wr, 11 op/s 2024-09-21T03:51:21.474 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:51:21 smithi150 ceph-mon[34289]: pgmap v484: 97 pgs: 97 active+clean; 10 MiB data, 520 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 170 B/s wr, 11 op/s 2024-09-21T03:51:21.789 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:21.755+0000 7fe7b6504640 1 -- 172.21.15.39:0/2187024698 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe7b01048c0 msgr2=0x7fe7b0104cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:51:21.790 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:21.755+0000 7fe7b6504640 1 --2- 172.21.15.39:0/2187024698 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe7b01048c0 0x7fe7b0104cd0 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7fe7a0009900 tx=0x7fe7a002a410 comp rx=0 tx=0).stop 2024-09-21T03:51:21.790 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:21.755+0000 7fe7b6504640 1 -- 172.21.15.39:0/2187024698 shutdown_connections 2024-09-21T03:51:21.790 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:21.755+0000 7fe7b6504640 1 --2- 172.21.15.39:0/2187024698 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe7b010de70 0x7fe7b0110340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:21.790 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:21.755+0000 7fe7b6504640 1 --2- 172.21.15.39:0/2187024698 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe7b01052a0 0x7fe7b010d930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:21.790 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:21.755+0000 7fe7b6504640 1 --2- 172.21.15.39:0/2187024698 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe7b01048c0 0x7fe7b0104cd0 unknown :-1 s=CLOSED pgs=865 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:21.790 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:21.755+0000 7fe7b6504640 1 -- 172.21.15.39:0/2187024698 >> 172.21.15.39:0/2187024698 conn(0x7fe7b0079760 msgr2=0x7fe7b0079b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:51:21.790 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:21.755+0000 7fe7b6504640 1 -- 172.21.15.39:0/2187024698 shutdown_connections 2024-09-21T03:51:21.790 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:21.755+0000 7fe7b6504640 1 -- 172.21.15.39:0/2187024698 wait complete. 2024-09-21T03:51:21.790 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:21.756+0000 7fe7b6504640 1 Processor -- start 2024-09-21T03:51:21.790 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:21.756+0000 7fe7b6504640 1 -- start start 2024-09-21T03:51:21.791 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:21.756+0000 7fe7b6504640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe7b01048c0 0x7fe7b019a440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:51:21.791 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:21.756+0000 7fe7b6504640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe7b01052a0 0x7fe7b019a9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:51:21.791 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:21.756+0000 7fe7b6504640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe7b010de70 0x7fe7b019fc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:51:21.791 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:21.756+0000 7fe7b6504640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fe7b0113ab0 con 0x7fe7b01048c0 2024-09-21T03:51:21.791 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:21.756+0000 7fe7b6504640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fe7b0113930 con 0x7fe7b010de70 2024-09-21T03:51:21.791 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:21.756+0000 7fe7b6504640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fe7b0113c30 con 0x7fe7b01052a0 2024-09-21T03:51:21.791 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:21.756+0000 7fe7affff640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe7b01048c0 0x7fe7b019a440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:51:21.791 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:21.756+0000 7fe7affff640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe7b01048c0 0x7fe7b019a440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:36372/0 (socket says 172.21.15.39:36372) 2024-09-21T03:51:21.791 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:21.756+0000 7fe7affff640 1 -- 172.21.15.39:0/2152445912 learned_addr learned my addr 172.21.15.39:0/2152445912 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:51:21.791 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:21.757+0000 7fe7a7fff640 1 --2- 172.21.15.39:0/2152445912 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe7b01052a0 0x7fe7b019a9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:51:21.791 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:21.757+0000 7fe7b4aa1640 1 --2- 172.21.15.39:0/2152445912 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe7b010de70 0x7fe7b019fc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:51:21.791 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:21.757+0000 7fe7affff640 1 -- 172.21.15.39:0/2152445912 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe7b01052a0 msgr2=0x7fe7b019a9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:51:21.791 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:21.757+0000 7fe7affff640 1 --2- 172.21.15.39:0/2152445912 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe7b01052a0 0x7fe7b019a9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:21.792 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:21.757+0000 7fe7affff640 1 -- 172.21.15.39:0/2152445912 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe7b010de70 msgr2=0x7fe7b019fc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:51:21.792 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:21.757+0000 7fe7affff640 1 --2- 172.21.15.39:0/2152445912 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe7b010de70 0x7fe7b019fc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:21.792 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:21.757+0000 7fe7affff640 1 -- 172.21.15.39:0/2152445912 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe7b01a02f0 con 0x7fe7b01048c0 2024-09-21T03:51:21.792 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:21.757+0000 7fe7affff640 1 --2- 172.21.15.39:0/2152445912 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe7b01048c0 0x7fe7b019a440 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7fe7a0031a90 tx=0x7fe7a0004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:51:21.792 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:21.757+0000 7fe7adffb640 1 -- 172.21.15.39:0/2152445912 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7a003f030 con 0x7fe7b01048c0 2024-09-21T03:51:21.792 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:21.757+0000 7fe7adffb640 1 -- 172.21.15.39:0/2152445912 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe7a0004490 con 0x7fe7b01048c0 2024-09-21T03:51:21.792 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:21.757+0000 7fe7b6504640 1 -- 172.21.15.39:0/2152445912 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe7b01a0580 con 0x7fe7b01048c0 2024-09-21T03:51:21.792 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:21.757+0000 7fe7adffb640 1 -- 172.21.15.39:0/2152445912 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7a0035070 con 0x7fe7b01048c0 2024-09-21T03:51:21.792 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:21.757+0000 7fe7b6504640 1 -- 172.21.15.39:0/2152445912 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe7b01a0a60 con 0x7fe7b01048c0 2024-09-21T03:51:21.792 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:21.759+0000 7fe7b6504640 1 -- 172.21.15.39:0/2152445912 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe7b006c190 con 0x7fe7b01048c0 2024-09-21T03:51:21.792 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:21.760+0000 7fe7adffb640 1 -- 172.21.15.39:0/2152445912 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 27) ==== 154643+0+0 (secure 0 0 0) 0x7fe7a0042030 con 0x7fe7b01048c0 2024-09-21T03:51:21.792 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:21.761+0000 7fe7adffb640 1 --2- 172.21.15.39:0/2152445912 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7fe79c0b66b0 0x7fe79c0b8bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:51:21.792 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:21.761+0000 7fe7a7fff640 1 --2- 172.21.15.39:0/2152445912 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7fe79c0b66b0 0x7fe79c0b8bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:51:21.793 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:21.761+0000 7fe7adffb640 1 -- 172.21.15.39:0/2152445912 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8484+0+0 (secure 0 0 0) 0x7fe7a0107cc0 con 0x7fe7b01048c0 2024-09-21T03:51:21.793 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:21.761+0000 7fe7a7fff640 1 --2- 172.21.15.39:0/2152445912 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7fe79c0b66b0 0x7fe79c0b8bc0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fe7b0102470 tx=0x7fe79800a2d0 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:51:21.793 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:21.765+0000 7fe7adffb640 1 -- 172.21.15.39:0/2152445912 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fe7a00ce230 con 0x7fe7b01048c0 2024-09-21T03:51:21.994 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:21.992+0000 7fe7b6504640 1 -- 172.21.15.39:0/2152445912 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe7b019b570 con 0x7fe7b01048c0 2024-09-21T03:51:21.995 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:21.993+0000 7fe7adffb640 1 -- 172.21.15.39:0/2152445912 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11978 (secure 0 0 0) 0x7fe7b019b570 con 0x7fe7b01048c0 2024-09-21T03:51:21.995 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T03:51:21.995 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":29,"btime":"2024-09-21T03:46:08:982881+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14790,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6835/508995295","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":508995295},{"type":"v1","addr":"172.21.15.39:6835","nonce":508995295}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14802,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6837/3915912607","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":3915912607},{"type":"v1","addr":"172.21.15.39:6837","nonce":3915912607}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":29,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T03:44:15.009872+0000","modified":"2024-09-21T03:46:08.430113+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24695,"mds_1":24667,"mds_2":24659,"mds_3":24646,"mds_4":14868},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14856":{"gid":14856,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.39:6839/185509192","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6838","nonce":185509192},{"type":"v1","addr":"172.21.15.39:6839","nonce":185509192}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14868":{"gid":14868,"name":"a","rank":4,"incarnation":25,"state":"up:active","state_seq":7,"addr":"172.21.15.39:6841/4247190456","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6840","nonce":4247190456},{"type":"v1","addr":"172.21.15.39:6841","nonce":4247190456}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24599":{"gid":24599,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.150:6833/3114643217","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":3114643217},{"type":"v1","addr":"172.21.15.150:6833","nonce":3114643217}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24601":{"gid":24601,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.161:6833/2963990654","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":2963990654},{"type":"v1","addr":"172.21.15.161:6833","nonce":2963990654}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24632":{"gid":24632,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.150:6835/724594008","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6834","nonce":724594008},{"type":"v1","addr":"172.21.15.150:6835","nonce":724594008}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24640":{"gid":24640,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.161:6835/3747882110","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":3747882110},{"type":"v1","addr":"172.21.15.161:6835","nonce":3747882110}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24646":{"gid":24646,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":10,"addr":"172.21.15.161:6837/3451544597","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6836","nonce":3451544597},{"type":"v1","addr":"172.21.15.161:6837","nonce":3451544597}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24659":{"gid":24659,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.150:6837/4017043144","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6836","nonce":4017043144},{"type":"v1","addr":"172.21.15.150:6837","nonce":4017043144}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24667":{"gid":24667,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.161:6839/2190107270","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6838","nonce":2190107270},{"type":"v1","addr":"172.21.15.161:6839","nonce":2190107270}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24695":{"gid":24695,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.150:6839/2949386907","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6838","nonce":2949386907},{"type":"v1","addr":"172.21.15.150:6839","nonce":2949386907}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24695,"qdb_cluster":[24695,24667,24659,24646,14868]},"id":1}]} 2024-09-21T03:51:21.996 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 29 2024-09-21T03:51:21.997 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:21.995+0000 7fe7b6504640 1 -- 172.21.15.39:0/2152445912 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7fe79c0b66b0 msgr2=0x7fe79c0b8bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:51:21.998 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:21.995+0000 7fe7b6504640 1 --2- 172.21.15.39:0/2152445912 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7fe79c0b66b0 0x7fe79c0b8bc0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fe7b0102470 tx=0x7fe79800a2d0 comp rx=0 tx=0).stop 2024-09-21T03:51:21.998 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:21.995+0000 7fe7b6504640 1 -- 172.21.15.39:0/2152445912 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe7b01048c0 msgr2=0x7fe7b019a440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:51:21.998 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:21.995+0000 7fe7b6504640 1 --2- 172.21.15.39:0/2152445912 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe7b01048c0 0x7fe7b019a440 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7fe7a0031a90 tx=0x7fe7a0004270 comp rx=0 tx=0).stop 2024-09-21T03:51:21.998 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:21.996+0000 7fe7b6504640 1 -- 172.21.15.39:0/2152445912 shutdown_connections 2024-09-21T03:51:21.998 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:21.996+0000 7fe7b6504640 1 --2- 172.21.15.39:0/2152445912 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7fe79c0b66b0 0x7fe79c0b8bc0 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:21.998 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:21.996+0000 7fe7b6504640 1 --2- 172.21.15.39:0/2152445912 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe7b010de70 0x7fe7b019fc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:21.998 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:21.996+0000 7fe7b6504640 1 --2- 172.21.15.39:0/2152445912 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe7b01052a0 0x7fe7b019a9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:21.998 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:21.996+0000 7fe7b6504640 1 --2- 172.21.15.39:0/2152445912 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe7b01048c0 0x7fe7b019a440 unknown :-1 s=CLOSED pgs=866 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:21.998 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:21.996+0000 7fe7b6504640 1 -- 172.21.15.39:0/2152445912 >> 172.21.15.39:0/2152445912 conn(0x7fe7b0079760 msgr2=0x7fe7b01007f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:51:21.998 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:21.996+0000 7fe7b6504640 1 -- 172.21.15.39:0/2152445912 shutdown_connections 2024-09-21T03:51:21.998 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:21.996+0000 7fe7b6504640 1 -- 172.21.15.39:0/2152445912 wait complete. 2024-09-21T03:51:22.334 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:51:22 smithi039 ceph-mon[31211]: from='client.? 172.21.15.39:0/2152445912' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:51:22.468 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:51:22 smithi161 ceph-mon[34533]: from='client.? 172.21.15.39:0/2152445912' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:51:22.474 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:51:22 smithi150 ceph-mon[34289]: from='client.? 172.21.15.39:0/2152445912' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:51:22.723 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 51116d5c-77ca-11ef-baf5-efdc52797490 -- ceph tell mds.1:4 damage ls 2024-09-21T03:51:22.980 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/51116d5c-77ca-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:51:23.364 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:51:23 smithi039 ceph-mon[31211]: pgmap v485: 97 pgs: 97 active+clean; 10 MiB data, 520 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 85 B/s wr, 9 op/s 2024-09-21T03:51:23.467 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:51:23 smithi161 ceph-mon[34533]: pgmap v485: 97 pgs: 97 active+clean; 10 MiB data, 520 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 85 B/s wr, 9 op/s 2024-09-21T03:51:23.474 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:51:23 smithi150 ceph-mon[34289]: pgmap v485: 97 pgs: 97 active+clean; 10 MiB data, 520 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 85 B/s wr, 9 op/s 2024-09-21T03:51:23.481 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:23.457+0000 7f65b4e2b640 1 -- 172.21.15.39:0/2151610364 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f65b0109e00 msgr2=0x7f65b0110630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:51:23.481 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:23.457+0000 7f65b4e2b640 1 --2- 172.21.15.39:0/2151610364 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f65b0109e00 0x7f65b0110630 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7f65a8009ca0 tx=0x7f65a802a8c0 comp rx=0 tx=0).stop 2024-09-21T03:51:23.481 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:23.457+0000 7f65b4e2b640 1 -- 172.21.15.39:0/2151610364 shutdown_connections 2024-09-21T03:51:23.481 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:23.457+0000 7f65b4e2b640 1 --2- 172.21.15.39:0/2151610364 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f65b0109e00 0x7f65b0110630 unknown :-1 s=CLOSED pgs=867 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:23.481 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:23.457+0000 7f65b4e2b640 1 --2- 172.21.15.39:0/2151610364 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f65b01094b0 0x7f65b01098c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:23.481 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:23.457+0000 7f65b4e2b640 1 --2- 172.21.15.39:0/2151610364 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f65b0103b70 0x7f65b0103f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:23.481 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:23.457+0000 7f65b4e2b640 1 -- 172.21.15.39:0/2151610364 >> 172.21.15.39:0/2151610364 conn(0x7f65b00ff5e0 msgr2=0x7f65b0101a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:51:23.481 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:23.457+0000 7f65b4e2b640 1 -- 172.21.15.39:0/2151610364 shutdown_connections 2024-09-21T03:51:23.481 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:23.457+0000 7f65b4e2b640 1 -- 172.21.15.39:0/2151610364 wait complete. 2024-09-21T03:51:23.481 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:23.458+0000 7f65b4e2b640 1 Processor -- start 2024-09-21T03:51:23.481 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:23.458+0000 7f65b4e2b640 1 -- start start 2024-09-21T03:51:23.482 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:23.458+0000 7f65b4e2b640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f65b0103b70 0x7f65b019a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:51:23.482 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:23.458+0000 7f65b4e2b640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f65b01094b0 0x7f65b019a9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:51:23.482 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:23.458+0000 7f65b4e2b640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f65b0109e00 0x7f65b019fc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:51:23.482 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:23.458+0000 7f65b4e2b640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f65b0113ac0 con 0x7f65b0103b70 2024-09-21T03:51:23.482 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:23.458+0000 7f65b4e2b640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f65b0113940 con 0x7f65b0109e00 2024-09-21T03:51:23.482 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:23.458+0000 7f65b4e2b640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f65b0113c40 con 0x7f65b01094b0 2024-09-21T03:51:23.482 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:23.458+0000 7f65ae59c640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f65b0103b70 0x7f65b019a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:51:23.482 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:23.458+0000 7f65ae59c640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f65b0103b70 0x7f65b019a430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:36388/0 (socket says 172.21.15.39:36388) 2024-09-21T03:51:23.482 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:23.458+0000 7f65ae59c640 1 -- 172.21.15.39:0/84085728 learned_addr learned my addr 172.21.15.39:0/84085728 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:51:23.482 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:23.458+0000 7f65aed9d640 1 --2- 172.21.15.39:0/84085728 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f65b0109e00 0x7f65b019fc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:51:23.482 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:23.458+0000 7f65add9b640 1 --2- 172.21.15.39:0/84085728 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f65b01094b0 0x7f65b019a9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:51:23.482 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:23.458+0000 7f65ae59c640 1 -- 172.21.15.39:0/84085728 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f65b01094b0 msgr2=0x7f65b019a9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:51:23.482 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:23.459+0000 7f65ae59c640 1 --2- 172.21.15.39:0/84085728 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f65b01094b0 0x7f65b019a9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:23.482 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:23.459+0000 7f65ae59c640 1 -- 172.21.15.39:0/84085728 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f65b0109e00 msgr2=0x7f65b019fc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:51:23.483 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:23.459+0000 7f65ae59c640 1 --2- 172.21.15.39:0/84085728 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f65b0109e00 0x7f65b019fc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:23.483 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:23.459+0000 7f65ae59c640 1 -- 172.21.15.39:0/84085728 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f65b01a03a0 con 0x7f65b0103b70 2024-09-21T03:51:23.483 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:23.459+0000 7f65ae59c640 1 --2- 172.21.15.39:0/84085728 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f65b0103b70 0x7f65b019a430 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7f65a000d950 tx=0x7f65a000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:51:23.483 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:23.459+0000 7f659f7fe640 1 -- 172.21.15.39:0/84085728 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65a0017030 con 0x7f65b0103b70 2024-09-21T03:51:23.483 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:23.459+0000 7f659f7fe640 1 -- 172.21.15.39:0/84085728 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f65a0004020 con 0x7f65b0103b70 2024-09-21T03:51:23.483 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:23.459+0000 7f65b4e2b640 1 -- 172.21.15.39:0/84085728 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f65b01a0690 con 0x7f65b0103b70 2024-09-21T03:51:23.483 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:23.459+0000 7f659f7fe640 1 -- 172.21.15.39:0/84085728 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65a00047a0 con 0x7f65b0103b70 2024-09-21T03:51:23.483 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:23.459+0000 7f65b4e2b640 1 -- 172.21.15.39:0/84085728 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f65b01a6df0 con 0x7f65b0103b70 2024-09-21T03:51:23.483 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:23.462+0000 7f65bb39f740 1 Processor -- start 2024-09-21T03:51:23.483 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:23.462+0000 7f65bb39f740 1 -- start start 2024-09-21T03:51:23.483 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:23.462+0000 7f659f7fe640 1 -- 172.21.15.39:0/84085728 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 27) ==== 154643+0+0 (secure 0 0 0) 0x7f65a0009a30 con 0x7f65b0103b70 2024-09-21T03:51:23.483 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:23.462+0000 7f65bb39f740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d82d4cb070 0x55d82d8726a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:51:23.483 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:23.462+0000 7f65bb39f740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d82d4a21b0 0x55d82d872be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:51:23.483 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:23.462+0000 7f65bb39f740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d82d4aa390 0x55d82d873120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:51:23.484 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:23.462+0000 7f65bb39f740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x55d82d7e8ea0 con 0x55d82d4cb070 2024-09-21T03:51:23.484 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:23.462+0000 7f65bb39f740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x55d82d7d82d0 con 0x55d82d4aa390 2024-09-21T03:51:23.484 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:23.462+0000 7f65bb39f740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55d82d7dc0e0 con 0x55d82d4a21b0 2024-09-21T03:51:23.484 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:23.462+0000 7f65aed9d640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d82d4cb070 0x55d82d8726a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:51:23.484 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:23.462+0000 7f65aed9d640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d82d4cb070 0x55d82d8726a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:36402/0 (socket says 172.21.15.39:36402) 2024-09-21T03:51:23.484 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:23.462+0000 7f65aed9d640 1 -- 172.21.15.39:0/3244808155 learned_addr learned my addr 172.21.15.39:0/3244808155 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:51:23.484 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:23.463+0000 7f65ae59c640 1 --2- 172.21.15.39:0/3244808155 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d82d4a21b0 0x55d82d872be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:51:23.484 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:23.463+0000 7f65add9b640 1 --2- 172.21.15.39:0/3244808155 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d82d4aa390 0x55d82d873120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:51:23.484 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:23.463+0000 7f65aed9d640 1 -- 172.21.15.39:0/3244808155 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d82d4a21b0 msgr2=0x55d82d872be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:51:23.484 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:23.463+0000 7f65aed9d640 1 --2- 172.21.15.39:0/3244808155 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d82d4a21b0 0x55d82d872be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:23.484 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:23.463+0000 7f65aed9d640 1 -- 172.21.15.39:0/3244808155 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d82d4aa390 msgr2=0x55d82d873120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:51:23.484 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:23.463+0000 7f65aed9d640 1 --2- 172.21.15.39:0/3244808155 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d82d4aa390 0x55d82d873120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:23.484 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:23.463+0000 7f65aed9d640 1 -- 172.21.15.39:0/3244808155 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d82d536650 con 0x55d82d4cb070 2024-09-21T03:51:23.484 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:23.463+0000 7f65add9b640 1 --2- 172.21.15.39:0/3244808155 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d82d4aa390 0x55d82d873120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:51:23.485 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:23.463+0000 7f659f7fe640 1 --2- 172.21.15.39:0/84085728 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f65980b6700 0x7f65980b8c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:51:23.485 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:23.463+0000 7f65aed9d640 1 --2- 172.21.15.39:0/3244808155 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d82d4cb070 0x55d82d8726a0 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7f65a8009920 tx=0x7f65a8009950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a497d69d46cf243 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:51:23.485 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:23.463+0000 7f65ae59c640 1 --2- 172.21.15.39:0/84085728 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f65980b6700 0x7f65980b8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:51:23.485 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:23.463+0000 7f659cff9640 1 -- 172.21.15.39:0/3244808155 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65a8031e30 con 0x55d82d4cb070 2024-09-21T03:51:23.485 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:23.463+0000 7f659f7fe640 1 -- 172.21.15.39:0/84085728 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8484+0+0 (secure 0 0 0) 0x7f65a00e70e0 con 0x7f65b0103b70 2024-09-21T03:51:23.485 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:23.463+0000 7f659cff9640 1 -- 172.21.15.39:0/3244808155 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f65a8036030 con 0x55d82d4cb070 2024-09-21T03:51:23.485 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:23.463+0000 7f659cff9640 1 -- 172.21.15.39:0/3244808155 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65a8004260 con 0x55d82d4cb070 2024-09-21T03:51:23.485 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:23.464+0000 7f65bb39f740 1 -- 172.21.15.39:0/3244808155 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d82d4cb070 msgr2=0x55d82d8726a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:51:23.485 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:23.464+0000 7f65ae59c640 1 --2- 172.21.15.39:0/84085728 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f65980b6700 0x7f65980b8c10 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f65a00e4900 tx=0x7f65a0058680 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:51:23.485 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:23.464+0000 7f65bb39f740 1 --2- 172.21.15.39:0/3244808155 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d82d4cb070 0x55d82d8726a0 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7f65a8009920 tx=0x7f65a8009950 comp rx=0 tx=0).stop 2024-09-21T03:51:23.485 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:23.464+0000 7f65bb39f740 1 -- 172.21.15.39:0/3244808155 shutdown_connections 2024-09-21T03:51:23.485 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:23.464+0000 7f65bb39f740 1 --2- 172.21.15.39:0/3244808155 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d82d4aa390 0x55d82d873120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:23.485 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:23.464+0000 7f65bb39f740 1 --2- 172.21.15.39:0/3244808155 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d82d4a21b0 0x55d82d872be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:23.485 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:23.464+0000 7f65bb39f740 1 --2- 172.21.15.39:0/3244808155 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d82d4cb070 0x55d82d8726a0 unknown :-1 s=CLOSED pgs=869 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:23.486 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:23.464+0000 7f65bb39f740 1 -- 172.21.15.39:0/3244808155 >> 172.21.15.39:0/3244808155 conn(0x55d82d4bf460 msgr2=0x55d82d870590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:51:23.486 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:23.464+0000 7f65bb39f740 1 -- 172.21.15.39:0/3244808155 shutdown_connections 2024-09-21T03:51:23.486 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:23.464+0000 7f65bb39f740 1 -- 172.21.15.39:0/3244808155 wait complete. 2024-09-21T03:51:23.486 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:23.465+0000 7f65bb39f740 1 -- start start 2024-09-21T03:51:23.486 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:23.465+0000 7f65bb39f740 1 Processor -- start 2024-09-21T03:51:23.486 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:23.465+0000 7f65bb39f740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d82d4a21b0 0x55d82d87dd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:51:23.486 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:23.465+0000 7f65bb39f740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d82d4aa390 0x55d82d87e250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:51:23.486 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:23.465+0000 7f65bb39f740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d82d4cb070 0x55d82d889f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:51:23.486 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:23.465+0000 7f65bb39f740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x55d82d639700 con 0x55d82d4aa390 2024-09-21T03:51:23.486 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:23.465+0000 7f65bb39f740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x55d82d773160 con 0x55d82d4cb070 2024-09-21T03:51:23.486 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:23.465+0000 7f65bb39f740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55d82d880ca0 con 0x55d82d4a21b0 2024-09-21T03:51:23.486 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:23.465+0000 7f65bb39f740 10 client.0 _ll_register_callbacks cb 0x55d82d78c9e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:51:23.486 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:23.466+0000 7f65add9b640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d82d4aa390 0x55d82d87e250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:51:23.486 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:23.466+0000 7f65add9b640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d82d4aa390 0x55d82d87e250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:36406/0 (socket says 172.21.15.39:36406) 2024-09-21T03:51:23.487 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:23.466+0000 7f65add9b640 1 -- 172.21.15.39:0/1806832207 learned_addr learned my addr 172.21.15.39:0/1806832207 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:51:23.487 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:23.466+0000 7f65aed9d640 1 --2- 172.21.15.39:0/1806832207 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d82d4a21b0 0x55d82d87dd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:51:23.487 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:23.466+0000 7f65aed9d640 1 --2- 172.21.15.39:0/1806832207 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d82d4cb070 0x55d82d889f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:51:23.487 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:23.466+0000 7f65add9b640 1 -- 172.21.15.39:0/1806832207 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d82d4a21b0 msgr2=0x55d82d87dd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:51:23.487 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:23.466+0000 7f65add9b640 1 --2- 172.21.15.39:0/1806832207 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d82d4a21b0 0x55d82d87dd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:23.487 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:23.466+0000 7f65add9b640 1 -- 172.21.15.39:0/1806832207 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d82d4cb070 msgr2=0x55d82d889f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:51:23.487 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:23.466+0000 7f65add9b640 1 --2- 172.21.15.39:0/1806832207 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d82d4cb070 0x55d82d889f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:23.487 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:23.466+0000 7f65add9b640 1 -- 172.21.15.39:0/1806832207 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d82d7a0c90 con 0x55d82d4aa390 2024-09-21T03:51:23.487 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:23.466+0000 7f65add9b640 1 --2- 172.21.15.39:0/1806832207 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d82d4aa390 0x55d82d87e250 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7f65a4011660 tx=0x7f65a400af90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3fec032e3fda7bcf server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:51:23.487 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:23.466+0000 7f6595ffb640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-09-21T03:51:23.487 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:23.466+0000 7f6595ffb640 1 -- 172.21.15.39:0/1806832207 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65a401f060 con 0x55d82d4aa390 2024-09-21T03:51:23.487 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:23.466+0000 7f6595ffb640 1 -- 172.21.15.39:0/1806832207 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8484+0+0 (secure 0 0 0) 0x7f65a400fa70 con 0x55d82d4aa390 2024-09-21T03:51:23.487 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:23.466+0000 7f65bb39f740 1 -- 172.21.15.39:0/1806832207 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f65a8036030 con 0x55d82d4aa390 2024-09-21T03:51:23.487 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:23.467+0000 7f6595ffb640 1 -- 172.21.15.39:0/1806832207 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f65a400fc70 con 0x55d82d4aa390 2024-09-21T03:51:23.487 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:23.467+0000 7f65bb39f740 10 client.16884 fetch_fsmap learned FSMap version 29 2024-09-21T03:51:23.488 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:23.467+0000 7f65bb39f740 1 -- 172.21.15.39:0/1806832207 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55d82d88c940 con 0x55d82d4aa390 2024-09-21T03:51:23.488 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:23.467+0000 7f6595ffb640 1 -- 172.21.15.39:0/1806832207 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f65a8036030 con 0x55d82d4aa390 2024-09-21T03:51:23.488 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:23.467+0000 7f6595ffb640 1 -- 172.21.15.39:0/1806832207 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f65a4022390 con 0x55d82d4aa390 2024-09-21T03:51:23.488 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:23.467+0000 7f65bb39f740 10 client.16884 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T03:51:23.488 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:23.467+0000 7f65bb39f740 10 client.16884 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.a 2024-09-21T03:51:23.488 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:23.467+0000 7f65bb39f740 20 client.16884 populate_metadata read hostname 'smithi039' 2024-09-21T03:51:23.488 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:23.467+0000 7f65bb39f740 1 --2- 172.21.15.39:0/1806832207 >> [v2:172.21.15.39:6840/4247190456,v1:172.21.15.39:6841/4247190456] conn(0x55d82d4899f0 0x55d82d88fe90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:51:23.488 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:23.467+0000 7f65bb39f740 4 client.16884 mds_command: new command op to 14868 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:51:23.488 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:23.467+0000 7f65bb39f740 1 -- 172.21.15.39:0/1806832207 --> [v2:172.21.15.39:6840/4247190456,v1:172.21.15.39:6841/4247190456] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d82d7eadf0 con 0x55d82d4899f0 2024-09-21T03:51:23.488 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:23.467+0000 7f65aed9d640 1 --2- 172.21.15.39:0/1806832207 >> [v2:172.21.15.39:6840/4247190456,v1:172.21.15.39:6841/4247190456] conn(0x55d82d4899f0 0x55d82d88fe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:51:23.488 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:23.468+0000 7f6595ffb640 10 client.16884 ms_handle_connect on v2:172.21.15.39:6840/4247190456 2024-09-21T03:51:23.488 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:23.468+0000 7f65aed9d640 1 --2- 172.21.15.39:0/1806832207 >> [v2:172.21.15.39:6840/4247190456,v1:172.21.15.39:6841/4247190456] conn(0x55d82d4899f0 0x55d82d88fe90 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=f605b8e58b48b155 server_cookie=b45197b9603c8e46 in_seq=0 out_seq=0 2024-09-21T03:51:23.488 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:23.468+0000 7f6595ffb640 1 -- 172.21.15.39:0/1806832207 <== mds.4 v2:172.21.15.39:6840/4247190456 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55d82d7eadf0 con 0x55d82d4899f0 2024-09-21T03:51:23.488 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:23.468+0000 7f6595ffb640 10 client.16884 handle_command_reply: tid=0 2024-09-21T03:51:23.488 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:23.468+0000 7f65bb39f740 1 client.16884 shutdown 2024-09-21T03:51:23.489 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:23.468+0000 7f65bb39f740 1 -- 172.21.15.39:0/1806832207 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d82d4aa390 msgr2=0x55d82d87e250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:51:23.489 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:23.468+0000 7f65bb39f740 1 --2- 172.21.15.39:0/1806832207 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d82d4aa390 0x55d82d87e250 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7f65a4011660 tx=0x7f65a400af90 comp rx=0 tx=0).stop 2024-09-21T03:51:23.489 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:23.469+0000 7f65bb39f740 1 -- 172.21.15.39:0/1806832207 shutdown_connections 2024-09-21T03:51:23.489 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:23.469+0000 7f65bb39f740 1 --2- 172.21.15.39:0/1806832207 >> [v2:172.21.15.39:6840/4247190456,v1:172.21.15.39:6841/4247190456] conn(0x55d82d4899f0 0x55d82d88fe90 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:23.489 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:23.469+0000 7f65bb39f740 1 --2- 172.21.15.39:0/1806832207 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d82d4cb070 0x55d82d889f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:23.489 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:23.469+0000 7f65bb39f740 1 --2- 172.21.15.39:0/1806832207 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d82d4aa390 0x55d82d87e250 unknown :-1 s=CLOSED pgs=870 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:23.489 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:23.469+0000 7f6595ffb640 0 client.16884 ms_handle_reset on v2:172.21.15.39:6840/4247190456 2024-09-21T03:51:23.489 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:23.469+0000 7f65bb39f740 1 --2- 172.21.15.39:0/1806832207 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d82d4a21b0 0x55d82d87dd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:23.489 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:23.469+0000 7f65bb39f740 1 -- 172.21.15.39:0/1806832207 >> 172.21.15.39:0/1806832207 conn(0x55d82d4bf460 msgr2=0x55d82d875880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:51:23.489 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:23.469+0000 7f65bb39f740 1 -- 172.21.15.39:0/1806832207 shutdown_connections 2024-09-21T03:51:23.489 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:23.469+0000 7f65bb39f740 1 -- 172.21.15.39:0/1806832207 wait complete. 2024-09-21T03:51:23.489 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:23.469+0000 7f65bb39f740 20 client.16884 trim_cache size 0 max 16384 2024-09-21T03:51:23.494 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:23.482+0000 7f65bb39f740 1 Processor -- start 2024-09-21T03:51:23.494 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:23.482+0000 7f65bb39f740 1 -- start start 2024-09-21T03:51:23.494 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:23.482+0000 7f65bb39f740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d82d4a21b0 0x55d82d874090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:51:23.494 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:23.482+0000 7f65bb39f740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d82d4aa390 0x55d82d87f940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:51:23.494 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:23.482+0000 7f65bb39f740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d82d4cb070 0x55d82d889e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:51:23.494 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:23.482+0000 7f65bb39f740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f65b01af6f0 con 0x55d82d4aa390 2024-09-21T03:51:23.494 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:23.482+0000 7f65bb39f740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f65b01b1230 con 0x55d82d4a21b0 2024-09-21T03:51:23.494 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:23.482+0000 7f65bb39f740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f65b01afdc0 con 0x55d82d4cb070 2024-09-21T03:51:23.495 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:23.483+0000 7f65add9b640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d82d4aa390 0x55d82d87f940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:51:23.495 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:23.483+0000 7f65add9b640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d82d4aa390 0x55d82d87f940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:36408/0 (socket says 172.21.15.39:36408) 2024-09-21T03:51:23.495 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:23.483+0000 7f65add9b640 1 -- 172.21.15.39:0/2886483156 learned_addr learned my addr 172.21.15.39:0/2886483156 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:51:23.495 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:23.483+0000 7f65aed9d640 1 --2- 172.21.15.39:0/2886483156 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d82d4cb070 0x55d82d889e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:51:23.495 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:23.483+0000 7f65aed9d640 1 --2- 172.21.15.39:0/2886483156 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d82d4a21b0 0x55d82d874090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:51:23.495 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:23.483+0000 7f65add9b640 1 -- 172.21.15.39:0/2886483156 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d82d4cb070 msgr2=0x55d82d889e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:51:23.495 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:23.483+0000 7f65add9b640 1 --2- 172.21.15.39:0/2886483156 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d82d4cb070 0x55d82d889e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:23.495 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:23.483+0000 7f65add9b640 1 -- 172.21.15.39:0/2886483156 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d82d4a21b0 msgr2=0x55d82d874090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:51:23.495 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:23.483+0000 7f65add9b640 1 --2- 172.21.15.39:0/2886483156 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d82d4a21b0 0x55d82d874090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:23.495 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:23.483+0000 7f65add9b640 1 -- 172.21.15.39:0/2886483156 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d82d87c210 con 0x55d82d4aa390 2024-09-21T03:51:23.495 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:23.483+0000 7f65add9b640 1 --2- 172.21.15.39:0/2886483156 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d82d4aa390 0x55d82d87f940 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7f65a400a470 tx=0x7f65a400a4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e34fd1e98d8cb12 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:51:23.495 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:23.483+0000 7f6597fff640 1 -- 172.21.15.39:0/2886483156 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65a400fe10 con 0x55d82d4aa390 2024-09-21T03:51:23.495 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:23.483+0000 7f6597fff640 1 -- 172.21.15.39:0/2886483156 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f65a4022390 con 0x55d82d4aa390 2024-09-21T03:51:23.495 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:23.483+0000 7f6597fff640 1 -- 172.21.15.39:0/2886483156 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65a4004330 con 0x55d82d4aa390 2024-09-21T03:51:23.496 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:23.484+0000 7f65bb39f740 1 -- 172.21.15.39:0/2886483156 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d82d4aa390 msgr2=0x55d82d87f940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:51:23.496 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:23.484+0000 7f65bb39f740 1 --2- 172.21.15.39:0/2886483156 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d82d4aa390 0x55d82d87f940 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7f65a400a470 tx=0x7f65a400a4a0 comp rx=0 tx=0).stop 2024-09-21T03:51:23.496 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:23.484+0000 7f65bb39f740 1 -- 172.21.15.39:0/2886483156 shutdown_connections 2024-09-21T03:51:23.496 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:23.484+0000 7f65bb39f740 1 --2- 172.21.15.39:0/2886483156 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d82d4cb070 0x55d82d889e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:23.496 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:23.484+0000 7f65bb39f740 1 --2- 172.21.15.39:0/2886483156 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d82d4aa390 0x55d82d87f940 unknown :-1 s=CLOSED pgs=871 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:23.496 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:23.484+0000 7f65bb39f740 1 --2- 172.21.15.39:0/2886483156 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d82d4a21b0 0x55d82d874090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:23.496 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:23.484+0000 7f65bb39f740 1 -- 172.21.15.39:0/2886483156 >> 172.21.15.39:0/2886483156 conn(0x55d82d4bf460 msgr2=0x55d82d87f230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:51:23.496 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:23.484+0000 7f65bb39f740 1 -- 172.21.15.39:0/2886483156 shutdown_connections 2024-09-21T03:51:23.496 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:23.484+0000 7f65bb39f740 1 -- 172.21.15.39:0/2886483156 wait complete. 2024-09-21T03:51:23.496 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:23.485+0000 7f65bb39f740 1 -- start start 2024-09-21T03:51:23.496 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:23.485+0000 7f65bb39f740 1 Processor -- start 2024-09-21T03:51:23.496 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:23.485+0000 7f65bb39f740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d82d4a21b0 0x55d82d882050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:51:23.496 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:23.485+0000 7f65bb39f740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d82d4aa390 0x55d82d882590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:51:23.496 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:23.485+0000 7f65bb39f740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d82d4cb070 0x55d82d882ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:51:23.497 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:23.485+0000 7f65bb39f740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f65b01b0b60 con 0x55d82d4cb070 2024-09-21T03:51:23.497 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:23.485+0000 7f65bb39f740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f65b01b0490 con 0x55d82d4aa390 2024-09-21T03:51:23.497 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:23.485+0000 7f65bb39f740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f65b01ad6d0 con 0x55d82d4a21b0 2024-09-21T03:51:23.497 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:23.485+0000 7f65bb39f740 10 client.0 _ll_register_callbacks cb 0x7f65a400a590 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:51:23.497 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:23.485+0000 7f65aed9d640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d82d4cb070 0x55d82d882ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:51:23.497 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:23.485+0000 7f65aed9d640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d82d4cb070 0x55d82d882ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:36424/0 (socket says 172.21.15.39:36424) 2024-09-21T03:51:23.497 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:23.485+0000 7f65aed9d640 1 -- 172.21.15.39:0/3888403773 learned_addr learned my addr 172.21.15.39:0/3888403773 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:51:23.497 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:23.485+0000 7f65add9b640 1 --2- 172.21.15.39:0/3888403773 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d82d4aa390 0x55d82d882590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:51:23.497 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:23.485+0000 7f65aed9d640 1 --2- 172.21.15.39:0/3888403773 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d82d4a21b0 0x55d82d882050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:51:23.497 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:23.486+0000 7f65aed9d640 1 -- 172.21.15.39:0/3888403773 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d82d4a21b0 msgr2=0x55d82d882050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:51:23.497 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:23.486+0000 7f65aed9d640 1 --2- 172.21.15.39:0/3888403773 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d82d4a21b0 0x55d82d882050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:23.497 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:23.486+0000 7f65aed9d640 1 -- 172.21.15.39:0/3888403773 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d82d4aa390 msgr2=0x55d82d882590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:51:23.497 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:23.486+0000 7f65aed9d640 1 --2- 172.21.15.39:0/3888403773 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d82d4aa390 0x55d82d882590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:23.497 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:23.486+0000 7f65aed9d640 1 -- 172.21.15.39:0/3888403773 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d82d8f99b0 con 0x55d82d4cb070 2024-09-21T03:51:23.498 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:23.486+0000 7f65aed9d640 1 --2- 172.21.15.39:0/3888403773 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d82d4cb070 0x55d82d882ad0 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7f65a8005d50 tx=0x7f65a803e040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8dd1345d8c47588a server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:51:23.498 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:23.486+0000 7f6595ffb640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-09-21T03:51:23.498 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:23.486+0000 7f6595ffb640 1 -- 172.21.15.39:0/3888403773 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65a8004b40 con 0x55d82d4cb070 2024-09-21T03:51:23.498 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:23.486+0000 7f6595ffb640 1 -- 172.21.15.39:0/3888403773 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8484+0+0 (secure 0 0 0) 0x7f65a8004ce0 con 0x55d82d4cb070 2024-09-21T03:51:23.498 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:23.486+0000 7f65bb39f740 1 -- 172.21.15.39:0/3888403773 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f65a4022390 con 0x55d82d4cb070 2024-09-21T03:51:23.498 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:23.486+0000 7f6595ffb640 1 -- 172.21.15.39:0/3888403773 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f65a8031450 con 0x55d82d4cb070 2024-09-21T03:51:23.498 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:23.486+0000 7f65bb39f740 10 client.16890 fetch_fsmap learned FSMap version 29 2024-09-21T03:51:23.498 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:23.486+0000 7f65bb39f740 1 -- 172.21.15.39:0/3888403773 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55d82d8fac30 con 0x55d82d4cb070 2024-09-21T03:51:23.498 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:23.486+0000 7f6595ffb640 1 -- 172.21.15.39:0/3888403773 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f65a4022390 con 0x55d82d4cb070 2024-09-21T03:51:23.498 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:23.487+0000 7f6595ffb640 1 -- 172.21.15.39:0/3888403773 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f65a8043070 con 0x55d82d4cb070 2024-09-21T03:51:23.498 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:23.487+0000 7f65bb39f740 10 client.16890 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T03:51:23.498 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:23.487+0000 7f65bb39f740 10 client.16890 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.a 2024-09-21T03:51:23.498 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:23.487+0000 7f65bb39f740 20 client.16890 populate_metadata read hostname 'smithi039' 2024-09-21T03:51:23.499 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:23.487+0000 7f65bb39f740 1 --2- 172.21.15.39:0/3888403773 >> [v2:172.21.15.39:6840/4247190456,v1:172.21.15.39:6841/4247190456] conn(0x55d82d45eda0 0x55d82d8fd0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:51:23.499 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:23.487+0000 7f65bb39f740 4 client.16890 mds_command: new command op to 14868 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T03:51:23.499 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:23.487+0000 7f65bb39f740 1 -- 172.21.15.39:0/3888403773 --> [v2:172.21.15.39:6840/4247190456,v1:172.21.15.39:6841/4247190456] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55d82d7e68a0 con 0x55d82d45eda0 2024-09-21T03:51:23.499 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:23.487+0000 7f65add9b640 1 --2- 172.21.15.39:0/3888403773 >> [v2:172.21.15.39:6840/4247190456,v1:172.21.15.39:6841/4247190456] conn(0x55d82d45eda0 0x55d82d8fd0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:51:23.499 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:23.487+0000 7f6595ffb640 10 client.16890 ms_handle_connect on v2:172.21.15.39:6840/4247190456 2024-09-21T03:51:23.499 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:23.487+0000 7f65add9b640 1 --2- 172.21.15.39:0/3888403773 >> [v2:172.21.15.39:6840/4247190456,v1:172.21.15.39:6841/4247190456] conn(0x55d82d45eda0 0x55d82d8fd0e0 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=3551ed2cd60d5000 server_cookie=3bd3639577110a61 in_seq=0 out_seq=0 2024-09-21T03:51:23.499 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:23.487+0000 7f6595ffb640 1 -- 172.21.15.39:0/3888403773 <== mds.4 v2:172.21.15.39:6840/4247190456 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55d82d7e68a0 con 0x55d82d45eda0 2024-09-21T03:51:23.499 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:23.487+0000 7f6595ffb640 10 client.16890 handle_command_reply: tid=0 2024-09-21T03:51:23.499 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:23.487+0000 7f65bb39f740 1 client.16890 shutdown 2024-09-21T03:51:23.499 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:23.488+0000 7f65bb39f740 1 -- 172.21.15.39:0/3888403773 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d82d4cb070 msgr2=0x55d82d882ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:51:23.499 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:23.488+0000 7f65bb39f740 1 --2- 172.21.15.39:0/3888403773 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d82d4cb070 0x55d82d882ad0 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7f65a8005d50 tx=0x7f65a803e040 comp rx=0 tx=0).stop 2024-09-21T03:51:23.499 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:23.488+0000 7f65bb39f740 1 -- 172.21.15.39:0/3888403773 shutdown_connections 2024-09-21T03:51:23.499 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:23.488+0000 7f65bb39f740 1 --2- 172.21.15.39:0/3888403773 >> [v2:172.21.15.39:6840/4247190456,v1:172.21.15.39:6841/4247190456] conn(0x55d82d45eda0 0x55d82d8fd0e0 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:23.500 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:23.488+0000 7f65bb39f740 1 --2- 172.21.15.39:0/3888403773 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d82d4cb070 0x55d82d882ad0 unknown :-1 s=CLOSED pgs=872 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:23.500 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:23.488+0000 7f65bb39f740 1 --2- 172.21.15.39:0/3888403773 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d82d4aa390 0x55d82d882590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:23.500 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:23.488+0000 7f65bb39f740 1 --2- 172.21.15.39:0/3888403773 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d82d4a21b0 0x55d82d882050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:23.500 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:23.488+0000 7f6595ffb640 0 client.16890 ms_handle_reset on v2:172.21.15.39:6840/4247190456 2024-09-21T03:51:23.500 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:23.488+0000 7f65bb39f740 1 -- 172.21.15.39:0/3888403773 >> 172.21.15.39:0/3888403773 conn(0x55d82d4bf460 msgr2=0x55d82d873e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:51:23.500 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:23.488+0000 7f65bb39f740 1 -- 172.21.15.39:0/3888403773 shutdown_connections 2024-09-21T03:51:23.500 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:23.488+0000 7f65bb39f740 1 -- 172.21.15.39:0/3888403773 wait complete. 2024-09-21T03:51:23.500 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:23.488+0000 7f65bb39f740 20 client.16890 trim_cache size 0 max 16384 2024-09-21T03:51:23.500 INFO:teuthology.orchestra.run.smithi039.stdout:[] 2024-09-21T03:51:23.501 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:23.488+0000 7f65b4e2b640 1 -- 172.21.15.39:0/84085728 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f65980b6700 msgr2=0x7f65980b8c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:51:23.501 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:23.488+0000 7f65b4e2b640 1 --2- 172.21.15.39:0/84085728 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f65980b6700 0x7f65980b8c10 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f65a00e4900 tx=0x7f65a0058680 comp rx=0 tx=0).stop 2024-09-21T03:51:23.501 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:23.488+0000 7f65b4e2b640 1 -- 172.21.15.39:0/84085728 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f65b0103b70 msgr2=0x7f65b019a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:51:23.501 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:23.488+0000 7f65b4e2b640 1 --2- 172.21.15.39:0/84085728 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f65b0103b70 0x7f65b019a430 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7f65a000d950 tx=0x7f65a000de10 comp rx=0 tx=0).stop 2024-09-21T03:51:23.501 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:23.489+0000 7f65b4e2b640 1 -- 172.21.15.39:0/84085728 shutdown_connections 2024-09-21T03:51:23.501 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:23.489+0000 7f65b4e2b640 1 --2- 172.21.15.39:0/84085728 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f65980b6700 0x7f65980b8c10 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:23.501 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:23.489+0000 7f65b4e2b640 1 --2- 172.21.15.39:0/84085728 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f65b0109e00 0x7f65b019fc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:23.501 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:23.489+0000 7f65b4e2b640 1 --2- 172.21.15.39:0/84085728 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f65b01094b0 0x7f65b019a9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:23.502 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:23.489+0000 7f65b4e2b640 1 --2- 172.21.15.39:0/84085728 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f65b0103b70 0x7f65b019a430 unknown :-1 s=CLOSED pgs=868 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:23.502 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:23.489+0000 7f65b4e2b640 1 -- 172.21.15.39:0/84085728 >> 172.21.15.39:0/84085728 conn(0x7f65b00ff5e0 msgr2=0x7f65b0101a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:51:23.502 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:23.489+0000 7f65b4e2b640 1 -- 172.21.15.39:0/84085728 shutdown_connections 2024-09-21T03:51:23.502 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:23.489+0000 7f65b4e2b640 1 -- 172.21.15.39:0/84085728 wait complete. 2024-09-21T03:51:24.240 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 51116d5c-77ca-11ef-baf5-efdc52797490 -- ceph tell mds.1:3 damage ls 2024-09-21T03:51:24.498 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/51116d5c-77ca-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:51:24.985 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:24.983+0000 7fb065e7d640 1 -- 172.21.15.39:0/896404245 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb060103b90 msgr2=0x7fb060103fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:51:24.985 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:24.983+0000 7fb065e7d640 1 --2- 172.21.15.39:0/896404245 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb060103b90 0x7fb060103fa0 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7fb050009900 tx=0x7fb05002a410 comp rx=0 tx=0).stop 2024-09-21T03:51:24.985 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:24.983+0000 7fb065e7d640 1 -- 172.21.15.39:0/896404245 shutdown_connections 2024-09-21T03:51:24.985 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:24.983+0000 7fb065e7d640 1 --2- 172.21.15.39:0/896404245 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb060109e20 0x7fb060110650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:24.985 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:24.983+0000 7fb065e7d640 1 --2- 172.21.15.39:0/896404245 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb0601094d0 0x7fb0601098e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:24.985 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:24.983+0000 7fb065e7d640 1 --2- 172.21.15.39:0/896404245 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb060103b90 0x7fb060103fa0 unknown :-1 s=CLOSED pgs=873 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:24.985 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:24.983+0000 7fb065e7d640 1 -- 172.21.15.39:0/896404245 >> 172.21.15.39:0/896404245 conn(0x7fb0600ff5e0 msgr2=0x7fb060101a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:51:24.986 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:24.983+0000 7fb065e7d640 1 -- 172.21.15.39:0/896404245 shutdown_connections 2024-09-21T03:51:24.986 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:24.983+0000 7fb065e7d640 1 -- 172.21.15.39:0/896404245 wait complete. 2024-09-21T03:51:24.987 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:24.984+0000 7fb065e7d640 1 Processor -- start 2024-09-21T03:51:24.987 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:24.984+0000 7fb065e7d640 1 -- start start 2024-09-21T03:51:24.987 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:24.984+0000 7fb065e7d640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb060103b90 0x7fb060075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:51:24.987 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:24.984+0000 7fb065e7d640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb0601094d0 0x7fb060075c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:51:24.987 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:24.984+0000 7fb065e7d640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb060109e20 0x7fb060076190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:51:24.987 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:24.984+0000 7fb065e7d640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fb060113ae0 con 0x7fb0601094d0 2024-09-21T03:51:24.988 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:24.984+0000 7fb065e7d640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fb060113960 con 0x7fb060103b90 2024-09-21T03:51:24.988 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:24.984+0000 7fb065e7d640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fb060113c60 con 0x7fb060109e20 2024-09-21T03:51:24.988 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:24.984+0000 7fb05effd640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb0601094d0 0x7fb060075c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:51:24.988 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:24.984+0000 7fb05effd640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb0601094d0 0x7fb060075c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:36442/0 (socket says 172.21.15.39:36442) 2024-09-21T03:51:24.988 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:24.984+0000 7fb05effd640 1 -- 172.21.15.39:0/152435365 learned_addr learned my addr 172.21.15.39:0/152435365 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:51:24.988 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:24.985+0000 7fb05f7fe640 1 --2- 172.21.15.39:0/152435365 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb060103b90 0x7fb060075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:51:24.988 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:24.985+0000 7fb05ffff640 1 --2- 172.21.15.39:0/152435365 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb060109e20 0x7fb060076190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:51:24.988 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:24.985+0000 7fb05effd640 1 -- 172.21.15.39:0/152435365 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb060109e20 msgr2=0x7fb060076190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:51:24.988 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:24.985+0000 7fb05effd640 1 --2- 172.21.15.39:0/152435365 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb060109e20 0x7fb060076190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:24.988 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:24.985+0000 7fb05effd640 1 -- 172.21.15.39:0/152435365 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb060103b90 msgr2=0x7fb060075710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:51:24.988 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:24.985+0000 7fb05effd640 1 --2- 172.21.15.39:0/152435365 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb060103b90 0x7fb060075710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:24.988 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:24.985+0000 7fb05effd640 1 -- 172.21.15.39:0/152435365 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb0601a86c0 con 0x7fb0601094d0 2024-09-21T03:51:24.988 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:24.985+0000 7fb05effd640 1 --2- 172.21.15.39:0/152435365 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb0601094d0 0x7fb060075c50 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7fb05400d950 tx=0x7fb05400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:51:24.988 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:24.985+0000 7fb05cff9640 1 -- 172.21.15.39:0/152435365 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb054017070 con 0x7fb0601094d0 2024-09-21T03:51:24.989 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:24.985+0000 7fb05cff9640 1 -- 172.21.15.39:0/152435365 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb054009c30 con 0x7fb0601094d0 2024-09-21T03:51:24.989 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:24.985+0000 7fb065e7d640 1 -- 172.21.15.39:0/152435365 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb0601a89b0 con 0x7fb0601094d0 2024-09-21T03:51:24.989 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:24.985+0000 7fb05cff9640 1 -- 172.21.15.39:0/152435365 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0540047a0 con 0x7fb0601094d0 2024-09-21T03:51:24.989 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:24.985+0000 7fb065e7d640 1 -- 172.21.15.39:0/152435365 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb0601a8ef0 con 0x7fb0601094d0 2024-09-21T03:51:24.990 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:24.988+0000 7fb06c3f1740 1 Processor -- start 2024-09-21T03:51:24.990 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:24.988+0000 7fb06c3f1740 1 -- start start 2024-09-21T03:51:24.991 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:24.989+0000 7fb06c3f1740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561cca784070 0x561ccab2b6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:51:24.991 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:24.989+0000 7fb06c3f1740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561cca75b1b0 0x561ccab2bbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:51:24.991 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:24.989+0000 7fb06c3f1740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561cca763390 0x561ccab2c120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:51:24.991 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:24.989+0000 7fb06c3f1740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x561ccaaa1ea0 con 0x561cca763390 2024-09-21T03:51:24.991 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:24.989+0000 7fb06c3f1740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x561ccaa912d0 con 0x561cca75b1b0 2024-09-21T03:51:24.992 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:24.989+0000 7fb06c3f1740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x561ccaa950e0 con 0x561cca784070 2024-09-21T03:51:24.992 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:24.989+0000 7fb05cff9640 1 -- 172.21.15.39:0/152435365 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 27) ==== 154643+0+0 (secure 0 0 0) 0x7fb054004020 con 0x7fb0601094d0 2024-09-21T03:51:24.992 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:24.989+0000 7fb05effd640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561cca763390 0x561ccab2c120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:51:24.992 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:24.989+0000 7fb05ffff640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561cca784070 0x561ccab2b6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:51:24.992 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:24.989+0000 7fb05ffff640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561cca784070 0x561ccab2b6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.39:55416/0 (socket says 172.21.15.39:55416) 2024-09-21T03:51:24.992 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:24.989+0000 7fb05ffff640 1 -- 172.21.15.39:0/613776330 learned_addr learned my addr 172.21.15.39:0/613776330 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:51:24.992 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:24.989+0000 7fb05f7fe640 1 --2- 172.21.15.39:0/613776330 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561cca75b1b0 0x561ccab2bbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:51:24.992 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:24.989+0000 7fb05effd640 1 -- 172.21.15.39:0/613776330 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561cca784070 msgr2=0x561ccab2b6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:51:24.992 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:24.989+0000 7fb05effd640 1 --2- 172.21.15.39:0/613776330 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561cca784070 0x561ccab2b6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:24.992 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:24.989+0000 7fb05effd640 1 -- 172.21.15.39:0/613776330 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561cca75b1b0 msgr2=0x561ccab2bbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:51:24.992 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:24.989+0000 7fb05effd640 1 --2- 172.21.15.39:0/613776330 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561cca75b1b0 0x561ccab2bbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:24.992 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:24.989+0000 7fb05effd640 1 -- 172.21.15.39:0/613776330 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561cca7ef650 con 0x561cca763390 2024-09-21T03:51:24.992 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:24.990+0000 7fb05effd640 1 --2- 172.21.15.39:0/613776330 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561cca763390 0x561ccab2c120 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x7fb0540e4b40 tx=0x7fb0540eda50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a68bed3dbef5aad server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:51:24.992 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:24.990+0000 7fb05cff9640 1 --2- 172.21.15.39:0/152435365 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7fb0480b6680 0x7fb0480b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:51:24.996 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:24.990+0000 7fb05ffff640 1 --2- 172.21.15.39:0/152435365 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7fb0480b6680 0x7fb0480b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:51:24.996 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:24.990+0000 7fb05cff9640 1 -- 172.21.15.39:0/152435365 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8484+0+0 (secure 0 0 0) 0x7fb0540e73d0 con 0x7fb0601094d0 2024-09-21T03:51:24.996 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:24.990+0000 7fb045ffb640 1 -- 172.21.15.39:0/613776330 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0540fa070 con 0x561cca763390 2024-09-21T03:51:24.996 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:24.990+0000 7fb045ffb640 1 -- 172.21.15.39:0/613776330 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb0540acd50 con 0x561cca763390 2024-09-21T03:51:24.996 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:24.990+0000 7fb045ffb640 1 -- 172.21.15.39:0/613776330 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0540a9890 con 0x561cca763390 2024-09-21T03:51:24.996 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:24.991+0000 7fb06c3f1740 1 -- 172.21.15.39:0/613776330 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561cca763390 msgr2=0x561ccab2c120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:51:24.996 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:24.991+0000 7fb05ffff640 1 --2- 172.21.15.39:0/152435365 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7fb0480b6680 0x7fb0480b8b90 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fb058007f10 tx=0x7fb0580076c0 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:51:24.996 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:24.991+0000 7fb06c3f1740 1 --2- 172.21.15.39:0/613776330 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561cca763390 0x561ccab2c120 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x7fb0540e4b40 tx=0x7fb0540eda50 comp rx=0 tx=0).stop 2024-09-21T03:51:24.996 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:24.991+0000 7fb06c3f1740 1 -- 172.21.15.39:0/613776330 shutdown_connections 2024-09-21T03:51:24.996 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:24.991+0000 7fb06c3f1740 1 --2- 172.21.15.39:0/613776330 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561cca763390 0x561ccab2c120 unknown :-1 s=CLOSED pgs=875 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:24.996 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:24.991+0000 7fb06c3f1740 1 --2- 172.21.15.39:0/613776330 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561cca75b1b0 0x561ccab2bbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:24.996 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:24.991+0000 7fb06c3f1740 1 --2- 172.21.15.39:0/613776330 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561cca784070 0x561ccab2b6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:24.997 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:24.991+0000 7fb06c3f1740 1 -- 172.21.15.39:0/613776330 >> 172.21.15.39:0/613776330 conn(0x561cca778460 msgr2=0x561ccab29590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:51:24.997 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:24.991+0000 7fb06c3f1740 1 -- 172.21.15.39:0/613776330 shutdown_connections 2024-09-21T03:51:24.997 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:24.991+0000 7fb06c3f1740 1 -- 172.21.15.39:0/613776330 wait complete. 2024-09-21T03:51:24.997 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:24.992+0000 7fb06c3f1740 1 -- start start 2024-09-21T03:51:24.997 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:24.992+0000 7fb06c3f1740 1 Processor -- start 2024-09-21T03:51:24.997 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:24.992+0000 7fb06c3f1740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561cca75b1b0 0x561ccab36e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:51:24.997 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:24.992+0000 7fb06c3f1740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561cca763390 0x561ccab37340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:51:24.997 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:24.992+0000 7fb06c3f1740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561cca784070 0x561ccab43170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:51:24.997 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:24.992+0000 7fb06c3f1740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x561cca8f2700 con 0x561cca763390 2024-09-21T03:51:24.997 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:24.992+0000 7fb06c3f1740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x561ccaa2c160 con 0x561cca784070 2024-09-21T03:51:24.997 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:24.992+0000 7fb06c3f1740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x561ccab39e80 con 0x561cca75b1b0 2024-09-21T03:51:24.997 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:24.992+0000 7fb06c3f1740 10 client.0 _ll_register_callbacks cb 0x561ccaa459e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:51:24.997 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:24.992+0000 7fb05effd640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561cca763390 0x561ccab37340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:51:24.997 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:24.992+0000 7fb05effd640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561cca763390 0x561ccab37340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:36458/0 (socket says 172.21.15.39:36458) 2024-09-21T03:51:24.997 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:24.992+0000 7fb05effd640 1 -- 172.21.15.39:0/2162216532 learned_addr learned my addr 172.21.15.39:0/2162216532 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:51:24.998 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:24.992+0000 7fb05f7fe640 1 --2- 172.21.15.39:0/2162216532 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561cca75b1b0 0x561ccab36e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:51:24.998 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:24.993+0000 7fb05ffff640 1 --2- 172.21.15.39:0/2162216532 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561cca784070 0x561ccab43170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:51:24.998 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:24.993+0000 7fb05effd640 1 -- 172.21.15.39:0/2162216532 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561cca75b1b0 msgr2=0x561ccab36e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:51:24.998 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:24.993+0000 7fb05effd640 1 --2- 172.21.15.39:0/2162216532 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561cca75b1b0 0x561ccab36e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:24.998 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:24.993+0000 7fb05effd640 1 -- 172.21.15.39:0/2162216532 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561cca784070 msgr2=0x561ccab43170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:51:24.998 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:24.993+0000 7fb05effd640 1 --2- 172.21.15.39:0/2162216532 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561cca784070 0x561ccab43170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:24.998 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:24.993+0000 7fb05effd640 1 -- 172.21.15.39:0/2162216532 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561ccaa59c90 con 0x561cca763390 2024-09-21T03:51:24.998 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:24.993+0000 7fb05effd640 1 --2- 172.21.15.39:0/2162216532 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561cca763390 0x561ccab37340 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x7fb0540e4bf0 tx=0x7fb0540046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9aab5d17e62e9b22 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:51:24.998 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:24.993+0000 7fb03effd640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-09-21T03:51:24.998 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:24.993+0000 7fb03effd640 1 -- 172.21.15.39:0/2162216532 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0540ac9b0 con 0x561cca763390 2024-09-21T03:51:24.998 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:24.993+0000 7fb03effd640 1 -- 172.21.15.39:0/2162216532 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8484+0+0 (secure 0 0 0) 0x7fb0540b67d0 con 0x561cca763390 2024-09-21T03:51:24.998 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:24.993+0000 7fb06c3f1740 1 -- 172.21.15.39:0/2162216532 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb0540acd50 con 0x561cca763390 2024-09-21T03:51:24.998 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:24.993+0000 7fb03effd640 1 -- 172.21.15.39:0/2162216532 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fb0540acb90 con 0x561cca763390 2024-09-21T03:51:24.998 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:24.993+0000 7fb06c3f1740 10 client.16902 fetch_fsmap learned FSMap version 29 2024-09-21T03:51:24.999 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:24.993+0000 7fb06c3f1740 1 -- 172.21.15.39:0/2162216532 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x561ccab45c30 con 0x561cca763390 2024-09-21T03:51:24.999 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:24.993+0000 7fb03effd640 1 -- 172.21.15.39:0/2162216532 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb0540acd50 con 0x561cca763390 2024-09-21T03:51:24.999 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:24.994+0000 7fb03effd640 1 -- 172.21.15.39:0/2162216532 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fb0540d0bc0 con 0x561cca763390 2024-09-21T03:51:24.999 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:24.994+0000 7fb06c3f1740 10 client.16902 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T03:51:24.999 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:24.994+0000 7fb06c3f1740 10 client.16902 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-21T03:51:24.999 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:24.994+0000 7fb06c3f1740 20 client.16902 populate_metadata read hostname 'smithi039' 2024-09-21T03:51:24.999 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:24.994+0000 7fb06c3f1740 1 --2- 172.21.15.39:0/2162216532 >> [v2:172.21.15.161:6836/3451544597,v1:172.21.15.161:6837/3451544597] conn(0x561cca7429f0 0x561ccab48f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:51:24.999 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:24.994+0000 7fb06c3f1740 4 client.16902 mds_command: new command op to 24646 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:51:24.999 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:24.994+0000 7fb06c3f1740 1 -- 172.21.15.39:0/2162216532 --> [v2:172.21.15.161:6836/3451544597,v1:172.21.15.161:6837/3451544597] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561ccaaa3df0 con 0x561cca7429f0 2024-09-21T03:51:24.999 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:24.994+0000 7fb05f7fe640 1 --2- 172.21.15.39:0/2162216532 >> [v2:172.21.15.161:6836/3451544597,v1:172.21.15.161:6837/3451544597] conn(0x561cca7429f0 0x561ccab48f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:51:25.000 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:24.995+0000 7fb05f7fe640 1 --2- 172.21.15.39:0/2162216532 >> [v2:172.21.15.161:6836/3451544597,v1:172.21.15.161:6837/3451544597] conn(0x561cca7429f0 0x561ccab48f90 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=44408bd7c238ff66 server_cookie=cfe9a6fa29ff9805 in_seq=0 out_seq=0 2024-09-21T03:51:25.000 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:24.995+0000 7fb03effd640 10 client.16902 ms_handle_connect on v2:172.21.15.161:6836/3451544597 2024-09-21T03:51:25.000 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:24.995+0000 7fb03effd640 1 -- 172.21.15.39:0/2162216532 <== mds.3 v2:172.21.15.161:6836/3451544597 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x561ccaaa3df0 con 0x561cca7429f0 2024-09-21T03:51:25.000 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:24.995+0000 7fb03effd640 10 client.16902 handle_command_reply: tid=0 2024-09-21T03:51:25.000 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:24.995+0000 7fb06c3f1740 1 client.16902 shutdown 2024-09-21T03:51:25.001 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:24.995+0000 7fb06c3f1740 1 -- 172.21.15.39:0/2162216532 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561cca763390 msgr2=0x561ccab37340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:51:25.001 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:24.995+0000 7fb06c3f1740 1 --2- 172.21.15.39:0/2162216532 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561cca763390 0x561ccab37340 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x7fb0540e4bf0 tx=0x7fb0540046c0 comp rx=0 tx=0).stop 2024-09-21T03:51:25.001 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:24.996+0000 7fb06c3f1740 1 -- 172.21.15.39:0/2162216532 shutdown_connections 2024-09-21T03:51:25.001 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:24.996+0000 7fb06c3f1740 1 --2- 172.21.15.39:0/2162216532 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561cca784070 0x561ccab43170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:25.001 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:24.996+0000 7fb06c3f1740 1 --2- 172.21.15.39:0/2162216532 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561cca763390 0x561ccab37340 unknown :-1 s=CLOSED pgs=876 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:25.001 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:24.996+0000 7fb06c3f1740 1 --2- 172.21.15.39:0/2162216532 >> [v2:172.21.15.161:6836/3451544597,v1:172.21.15.161:6837/3451544597] conn(0x561cca7429f0 0x561ccab48f90 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:25.001 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:24.996+0000 7fb06c3f1740 1 --2- 172.21.15.39:0/2162216532 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561cca75b1b0 0x561ccab36e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:25.001 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:24.996+0000 7fb03effd640 0 client.16902 ms_handle_reset on v2:172.21.15.161:6836/3451544597 2024-09-21T03:51:25.001 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:24.996+0000 7fb06c3f1740 1 -- 172.21.15.39:0/2162216532 >> 172.21.15.39:0/2162216532 conn(0x561cca778460 msgr2=0x561ccab2c6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:51:25.001 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:24.996+0000 7fb06c3f1740 1 -- 172.21.15.39:0/2162216532 shutdown_connections 2024-09-21T03:51:25.001 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:24.996+0000 7fb06c3f1740 1 -- 172.21.15.39:0/2162216532 wait complete. 2024-09-21T03:51:25.001 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:24.996+0000 7fb06c3f1740 20 client.16902 trim_cache size 0 max 16384 2024-09-21T03:51:25.011 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:25.009+0000 7fb06c3f1740 1 Processor -- start 2024-09-21T03:51:25.012 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:25.009+0000 7fb06c3f1740 1 -- start start 2024-09-21T03:51:25.012 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:25.009+0000 7fb06c3f1740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561cca7429f0 0x561ccab38ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:51:25.012 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:25.009+0000 7fb06c3f1740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561cca75b1b0 0x561ccab430e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:51:25.012 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:25.009+0000 7fb06c3f1740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561cca763390 0x561ccab30ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:51:25.012 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:25.009+0000 7fb06c3f1740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fb0601b1810 con 0x561cca763390 2024-09-21T03:51:25.012 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:25.009+0000 7fb06c3f1740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fb0601b3350 con 0x561cca75b1b0 2024-09-21T03:51:25.012 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:25.009+0000 7fb06c3f1740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fb0601b1ee0 con 0x561cca7429f0 2024-09-21T03:51:25.012 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:25.009+0000 7fb05ffff640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561cca763390 0x561ccab30ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:51:25.012 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:25.009+0000 7fb05ffff640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561cca763390 0x561ccab30ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:36466/0 (socket says 172.21.15.39:36466) 2024-09-21T03:51:25.012 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:25.010+0000 7fb05ffff640 1 -- 172.21.15.39:0/3734072193 learned_addr learned my addr 172.21.15.39:0/3734072193 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:51:25.012 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:25.010+0000 7fb05f7fe640 1 --2- 172.21.15.39:0/3734072193 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561cca7429f0 0x561ccab38ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:51:25.012 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:25.010+0000 7fb05effd640 1 --2- 172.21.15.39:0/3734072193 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561cca75b1b0 0x561ccab430e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:51:25.012 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:25.010+0000 7fb05ffff640 1 -- 172.21.15.39:0/3734072193 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561cca7429f0 msgr2=0x561ccab38ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:51:25.012 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:25.010+0000 7fb05ffff640 1 --2- 172.21.15.39:0/3734072193 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561cca7429f0 0x561ccab38ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:25.013 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:25.010+0000 7fb05ffff640 1 -- 172.21.15.39:0/3734072193 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561cca75b1b0 msgr2=0x561ccab430e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:51:25.013 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:25.010+0000 7fb05ffff640 1 --2- 172.21.15.39:0/3734072193 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561cca75b1b0 0x561ccab430e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:25.013 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:25.010+0000 7fb05ffff640 1 -- 172.21.15.39:0/3734072193 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561ccab2d990 con 0x561cca763390 2024-09-21T03:51:25.013 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:25.010+0000 7fb05effd640 1 --2- 172.21.15.39:0/3734072193 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561cca75b1b0 0x561ccab430e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:51:25.013 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:25.010+0000 7fb05ffff640 1 --2- 172.21.15.39:0/3734072193 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561cca763390 0x561ccab30ec0 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7fb058003b00 tx=0x7fb058014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c4cd83f40e3dc7 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:51:25.015 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:25.010+0000 7fb0457fa640 1 -- 172.21.15.39:0/3734072193 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb05801f630 con 0x561cca763390 2024-09-21T03:51:25.015 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:25.010+0000 7fb0457fa640 1 -- 172.21.15.39:0/3734072193 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb058023070 con 0x561cca763390 2024-09-21T03:51:25.015 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:25.010+0000 7fb0457fa640 1 -- 172.21.15.39:0/3734072193 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb058027a40 con 0x561cca763390 2024-09-21T03:51:25.015 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:25.011+0000 7fb06c3f1740 1 -- 172.21.15.39:0/3734072193 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561cca763390 msgr2=0x561ccab30ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:51:25.015 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:25.011+0000 7fb06c3f1740 1 --2- 172.21.15.39:0/3734072193 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561cca763390 0x561ccab30ec0 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7fb058003b00 tx=0x7fb058014000 comp rx=0 tx=0).stop 2024-09-21T03:51:25.015 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:25.011+0000 7fb06c3f1740 1 -- 172.21.15.39:0/3734072193 shutdown_connections 2024-09-21T03:51:25.015 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:25.011+0000 7fb06c3f1740 1 --2- 172.21.15.39:0/3734072193 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561cca763390 0x561ccab30ec0 unknown :-1 s=CLOSED pgs=877 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:25.015 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:25.011+0000 7fb06c3f1740 1 --2- 172.21.15.39:0/3734072193 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561cca75b1b0 0x561ccab430e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:25.016 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:25.011+0000 7fb06c3f1740 1 --2- 172.21.15.39:0/3734072193 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561cca7429f0 0x561ccab38ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:25.016 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:25.011+0000 7fb06c3f1740 1 -- 172.21.15.39:0/3734072193 >> 172.21.15.39:0/3734072193 conn(0x561cca778460 msgr2=0x561ccab307b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:51:25.016 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:25.011+0000 7fb06c3f1740 1 -- 172.21.15.39:0/3734072193 shutdown_connections 2024-09-21T03:51:25.016 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:25.011+0000 7fb06c3f1740 1 -- 172.21.15.39:0/3734072193 wait complete. 2024-09-21T03:51:25.017 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:25.012+0000 7fb06c3f1740 1 -- start start 2024-09-21T03:51:25.017 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:25.012+0000 7fb06c3f1740 1 Processor -- start 2024-09-21T03:51:25.017 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:25.012+0000 7fb06c3f1740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561cca7429f0 0x561ccab4bfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:51:25.018 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:25.012+0000 7fb06c3f1740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561cca75b1b0 0x561ccab4c5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:51:25.018 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:25.012+0000 7fb06c3f1740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561cca763390 0x561ccaba8aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:51:25.018 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:25.012+0000 7fb06c3f1740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fb0601b2c80 con 0x561cca7429f0 2024-09-21T03:51:25.018 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:25.012+0000 7fb06c3f1740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fb0601b25b0 con 0x561cca763390 2024-09-21T03:51:25.018 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:25.012+0000 7fb06c3f1740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fb0601af7f0 con 0x561cca75b1b0 2024-09-21T03:51:25.018 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:25.012+0000 7fb06c3f1740 10 client.0 _ll_register_callbacks cb 0x7fb0540f8070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:51:25.018 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:25.012+0000 7fb05f7fe640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561cca7429f0 0x561ccab4bfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:51:25.018 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:25.012+0000 7fb05f7fe640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561cca7429f0 0x561ccab4bfd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:36478/0 (socket says 172.21.15.39:36478) 2024-09-21T03:51:25.018 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:25.012+0000 7fb05f7fe640 1 -- 172.21.15.39:0/958715567 learned_addr learned my addr 172.21.15.39:0/958715567 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:51:25.018 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:25.012+0000 7fb05effd640 1 --2- 172.21.15.39:0/958715567 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561cca75b1b0 0x561ccab4c5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:51:25.018 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:25.013+0000 7fb05ffff640 1 --2- 172.21.15.39:0/958715567 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561cca763390 0x561ccaba8aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:51:25.018 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:25.013+0000 7fb05f7fe640 1 -- 172.21.15.39:0/958715567 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561cca75b1b0 msgr2=0x561ccab4c5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:51:25.018 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:25.013+0000 7fb05f7fe640 1 --2- 172.21.15.39:0/958715567 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561cca75b1b0 0x561ccab4c5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:25.018 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:25.013+0000 7fb05f7fe640 1 -- 172.21.15.39:0/958715567 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561cca763390 msgr2=0x561ccaba8aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:51:25.019 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:25.013+0000 7fb05f7fe640 1 --2- 172.21.15.39:0/958715567 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561cca763390 0x561ccaba8aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:25.019 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:25.013+0000 7fb05f7fe640 1 -- 172.21.15.39:0/958715567 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561ccaba9110 con 0x561cca7429f0 2024-09-21T03:51:25.019 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:25.013+0000 7fb05f7fe640 1 --2- 172.21.15.39:0/958715567 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561cca7429f0 0x561ccab4bfd0 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 crypto rx=0x7fb05002a930 tx=0x7fb050004180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be02f0016dacf507 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:51:25.019 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:25.013+0000 7fb03effd640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-09-21T03:51:25.019 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:25.013+0000 7fb03effd640 1 -- 172.21.15.39:0/958715567 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb050035040 con 0x561cca7429f0 2024-09-21T03:51:25.019 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:25.013+0000 7fb03effd640 1 -- 172.21.15.39:0/958715567 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8484+0+0 (secure 0 0 0) 0x7fb050005240 con 0x561cca7429f0 2024-09-21T03:51:25.019 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:25.013+0000 7fb06c3f1740 1 -- 172.21.15.39:0/958715567 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb058023070 con 0x561cca7429f0 2024-09-21T03:51:25.021 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:25.013+0000 7fb03effd640 1 -- 172.21.15.39:0/958715567 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fb050055050 con 0x561cca7429f0 2024-09-21T03:51:25.021 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:25.013+0000 7fb06c3f1740 10 client.16908 fetch_fsmap learned FSMap version 29 2024-09-21T03:51:25.021 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:25.013+0000 7fb06c3f1740 1 -- 172.21.15.39:0/958715567 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x561ccabb3710 con 0x561cca7429f0 2024-09-21T03:51:25.021 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:25.014+0000 7fb03effd640 1 -- 172.21.15.39:0/958715567 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb058023070 con 0x561cca7429f0 2024-09-21T03:51:25.021 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:25.014+0000 7fb03effd640 1 -- 172.21.15.39:0/958715567 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fb0500035f0 con 0x561cca7429f0 2024-09-21T03:51:25.021 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:25.014+0000 7fb06c3f1740 10 client.16908 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T03:51:25.021 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:25.014+0000 7fb06c3f1740 10 client.16908 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-21T03:51:25.021 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:25.014+0000 7fb06c3f1740 20 client.16908 populate_metadata read hostname 'smithi039' 2024-09-21T03:51:25.022 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:25.014+0000 7fb06c3f1740 1 --2- 172.21.15.39:0/958715567 >> [v2:172.21.15.161:6836/3451544597,v1:172.21.15.161:6837/3451544597] conn(0x561cca784070 0x561ccabb5ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:51:25.022 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:25.014+0000 7fb06c3f1740 4 client.16908 mds_command: new command op to 24646 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T03:51:25.022 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:25.014+0000 7fb06c3f1740 1 -- 172.21.15.39:0/958715567 --> [v2:172.21.15.161:6836/3451544597,v1:172.21.15.161:6837/3451544597] -- command(tid 0: {"prefix": "damage ls"}) -- 0x561ccaa9f8a0 con 0x561cca784070 2024-09-21T03:51:25.022 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:25.014+0000 7fb05effd640 1 --2- 172.21.15.39:0/958715567 >> [v2:172.21.15.161:6836/3451544597,v1:172.21.15.161:6837/3451544597] conn(0x561cca784070 0x561ccabb5ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:51:25.022 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:25.015+0000 7fb03effd640 10 client.16908 ms_handle_connect on v2:172.21.15.161:6836/3451544597 2024-09-21T03:51:25.022 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:25.015+0000 7fb05effd640 1 --2- 172.21.15.39:0/958715567 >> [v2:172.21.15.161:6836/3451544597,v1:172.21.15.161:6837/3451544597] conn(0x561cca784070 0x561ccabb5ef0 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=2be5bdfec0a2d764 server_cookie=56a47833b50eba81 in_seq=0 out_seq=0 2024-09-21T03:51:25.022 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:25.015+0000 7fb03effd640 1 -- 172.21.15.39:0/958715567 <== mds.3 v2:172.21.15.161:6836/3451544597 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x561ccaa9f8a0 con 0x561cca784070 2024-09-21T03:51:25.022 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:25.015+0000 7fb03effd640 10 client.16908 handle_command_reply: tid=0 2024-09-21T03:51:25.022 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:25.015+0000 7fb06c3f1740 1 client.16908 shutdown 2024-09-21T03:51:25.022 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:25.015+0000 7fb06c3f1740 1 -- 172.21.15.39:0/958715567 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561cca7429f0 msgr2=0x561ccab4bfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:51:25.022 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:25.015+0000 7fb06c3f1740 1 --2- 172.21.15.39:0/958715567 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561cca7429f0 0x561ccab4bfd0 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 crypto rx=0x7fb05002a930 tx=0x7fb050004180 comp rx=0 tx=0).stop 2024-09-21T03:51:25.022 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:25.015+0000 7fb06c3f1740 1 -- 172.21.15.39:0/958715567 shutdown_connections 2024-09-21T03:51:25.022 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:25.015+0000 7fb06c3f1740 1 --2- 172.21.15.39:0/958715567 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561cca763390 0x561ccaba8aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:25.022 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:25.015+0000 7fb06c3f1740 1 --2- 172.21.15.39:0/958715567 >> [v2:172.21.15.161:6836/3451544597,v1:172.21.15.161:6837/3451544597] conn(0x561cca784070 0x561ccabb5ef0 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:25.023 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:25.015+0000 7fb06c3f1740 1 --2- 172.21.15.39:0/958715567 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561cca75b1b0 0x561ccab4c5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:25.023 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:25.015+0000 7fb06c3f1740 1 --2- 172.21.15.39:0/958715567 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561cca7429f0 0x561ccab4bfd0 unknown :-1 s=CLOSED pgs=878 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:25.023 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:25.015+0000 7fb06c3f1740 1 -- 172.21.15.39:0/958715567 >> 172.21.15.39:0/958715567 conn(0x561cca778460 msgr2=0x561ccab38a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:51:25.023 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:25.015+0000 7fb03effd640 0 client.16908 ms_handle_reset on v2:172.21.15.161:6836/3451544597 2024-09-21T03:51:25.023 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:25.015+0000 7fb06c3f1740 1 -- 172.21.15.39:0/958715567 shutdown_connections 2024-09-21T03:51:25.023 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:25.015+0000 7fb06c3f1740 1 -- 172.21.15.39:0/958715567 wait complete. 2024-09-21T03:51:25.023 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:25.015+0000 7fb06c3f1740 20 client.16908 trim_cache size 0 max 16384 2024-09-21T03:51:25.023 INFO:teuthology.orchestra.run.smithi039.stdout:[] 2024-09-21T03:51:25.024 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:25.016+0000 7fb065e7d640 1 -- 172.21.15.39:0/152435365 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7fb0480b6680 msgr2=0x7fb0480b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:51:25.024 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:25.016+0000 7fb065e7d640 1 --2- 172.21.15.39:0/152435365 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7fb0480b6680 0x7fb0480b8b90 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fb058007f10 tx=0x7fb0580076c0 comp rx=0 tx=0).stop 2024-09-21T03:51:25.024 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:25.016+0000 7fb065e7d640 1 -- 172.21.15.39:0/152435365 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb0601094d0 msgr2=0x7fb060075c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:51:25.024 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:25.016+0000 7fb065e7d640 1 --2- 172.21.15.39:0/152435365 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb0601094d0 0x7fb060075c50 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7fb05400d950 tx=0x7fb05400de10 comp rx=0 tx=0).stop 2024-09-21T03:51:25.024 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:25.016+0000 7fb065e7d640 1 -- 172.21.15.39:0/152435365 shutdown_connections 2024-09-21T03:51:25.024 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:25.016+0000 7fb065e7d640 1 --2- 172.21.15.39:0/152435365 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7fb0480b6680 0x7fb0480b8b90 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:25.024 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:25.016+0000 7fb065e7d640 1 --2- 172.21.15.39:0/152435365 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb060109e20 0x7fb060076190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:25.025 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:25.016+0000 7fb065e7d640 1 --2- 172.21.15.39:0/152435365 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb0601094d0 0x7fb060075c50 unknown :-1 s=CLOSED pgs=874 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:25.025 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:25.016+0000 7fb065e7d640 1 --2- 172.21.15.39:0/152435365 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb060103b90 0x7fb060075710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:25.025 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:25.016+0000 7fb065e7d640 1 -- 172.21.15.39:0/152435365 >> 172.21.15.39:0/152435365 conn(0x7fb0600ff5e0 msgr2=0x7fb060101a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:51:25.025 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:25.016+0000 7fb065e7d640 1 -- 172.21.15.39:0/152435365 shutdown_connections 2024-09-21T03:51:25.025 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:25.016+0000 7fb065e7d640 1 -- 172.21.15.39:0/152435365 wait complete. 2024-09-21T03:51:25.334 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:51:25 smithi039 ceph-mon[31211]: pgmap v486: 97 pgs: 97 active+clean; 10 MiB data, 520 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 85 B/s wr, 10 op/s 2024-09-21T03:51:25.468 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:51:25 smithi161 ceph-mon[34533]: pgmap v486: 97 pgs: 97 active+clean; 10 MiB data, 520 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 85 B/s wr, 10 op/s 2024-09-21T03:51:25.475 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:51:25 smithi150 ceph-mon[34289]: pgmap v486: 97 pgs: 97 active+clean; 10 MiB data, 520 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 85 B/s wr, 10 op/s 2024-09-21T03:51:25.783 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 51116d5c-77ca-11ef-baf5-efdc52797490 -- ceph tell mds.1:2 damage ls 2024-09-21T03:51:26.042 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/51116d5c-77ca-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:51:26.516 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:26.508+0000 7f4f80111640 1 -- 172.21.15.39:0/3879561003 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4f781083a0 msgr2=0x7f4f781087b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:51:26.516 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:26.508+0000 7f4f80111640 1 --2- 172.21.15.39:0/3879561003 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4f781083a0 0x7f4f781087b0 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7f4f6c009900 tx=0x7f4f6c02a3f0 comp rx=0 tx=0).stop 2024-09-21T03:51:26.516 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:26.508+0000 7f4f80111640 1 -- 172.21.15.39:0/3879561003 shutdown_connections 2024-09-21T03:51:26.516 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:26.508+0000 7f4f80111640 1 --2- 172.21.15.39:0/3879561003 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4f78108cf0 0x7f4f7810f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:26.517 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:26.508+0000 7f4f80111640 1 --2- 172.21.15.39:0/3879561003 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4f781083a0 0x7f4f781087b0 unknown :-1 s=CLOSED pgs=879 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:26.517 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:26.508+0000 7f4f80111640 1 --2- 172.21.15.39:0/3879561003 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4f78102a60 0x7f4f78102e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:26.517 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:26.508+0000 7f4f80111640 1 -- 172.21.15.39:0/3879561003 >> 172.21.15.39:0/3879561003 conn(0x7f4f78079730 msgr2=0x7f4f78079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:51:26.517 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:26.508+0000 7f4f80111640 1 -- 172.21.15.39:0/3879561003 shutdown_connections 2024-09-21T03:51:26.517 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:26.508+0000 7f4f80111640 1 -- 172.21.15.39:0/3879561003 wait complete. 2024-09-21T03:51:26.517 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:26.509+0000 7f4f80111640 1 Processor -- start 2024-09-21T03:51:26.517 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:26.509+0000 7f4f80111640 1 -- start start 2024-09-21T03:51:26.517 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:26.509+0000 7f4f80111640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4f78102a60 0x7f4f78199410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:51:26.517 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:26.509+0000 7f4f80111640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4f781083a0 0x7f4f781999b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:51:26.517 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:26.509+0000 7f4f80111640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4f78108cf0 0x7f4f7819ec50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:51:26.517 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:26.509+0000 7f4f80111640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f4f78112960 con 0x7f4f781083a0 2024-09-21T03:51:26.517 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:26.509+0000 7f4f80111640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f4f781127e0 con 0x7f4f78108cf0 2024-09-21T03:51:26.517 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:26.509+0000 7f4f80111640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f4f78112ae0 con 0x7f4f78102a60 2024-09-21T03:51:26.517 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:26.509+0000 7f4f7d6ac640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4f781083a0 0x7f4f781999b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:51:26.518 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:26.509+0000 7f4f7d6ac640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4f781083a0 0x7f4f781999b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:36494/0 (socket says 172.21.15.39:36494) 2024-09-21T03:51:26.518 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:26.509+0000 7f4f7d6ac640 1 -- 172.21.15.39:0/672208421 learned_addr learned my addr 172.21.15.39:0/672208421 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:51:26.518 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:26.510+0000 7f4f7dead640 1 --2- 172.21.15.39:0/672208421 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4f78102a60 0x7f4f78199410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:51:26.518 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:26.510+0000 7f4f7e6ae640 1 --2- 172.21.15.39:0/672208421 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4f78108cf0 0x7f4f7819ec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:51:26.518 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:26.510+0000 7f4f7d6ac640 1 -- 172.21.15.39:0/672208421 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4f78102a60 msgr2=0x7f4f78199410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:51:26.518 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:26.510+0000 7f4f7d6ac640 1 --2- 172.21.15.39:0/672208421 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4f78102a60 0x7f4f78199410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:26.518 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:26.510+0000 7f4f7d6ac640 1 -- 172.21.15.39:0/672208421 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4f78108cf0 msgr2=0x7f4f7819ec50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:51:26.518 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:26.510+0000 7f4f7d6ac640 1 --2- 172.21.15.39:0/672208421 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4f78108cf0 0x7f4f7819ec50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:26.518 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:26.510+0000 7f4f7d6ac640 1 -- 172.21.15.39:0/672208421 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4f7819f2c0 con 0x7f4f781083a0 2024-09-21T03:51:26.518 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:26.510+0000 7f4f7dead640 1 --2- 172.21.15.39:0/672208421 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4f78102a60 0x7f4f78199410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:51:26.518 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:26.510+0000 7f4f7d6ac640 1 --2- 172.21.15.39:0/672208421 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4f781083a0 0x7f4f781999b0 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x7f4f6c032fd0 tx=0x7f4f6c004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:51:26.518 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:26.510+0000 7f4f76ffd640 1 -- 172.21.15.39:0/672208421 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f6c046070 con 0x7f4f781083a0 2024-09-21T03:51:26.518 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:26.510+0000 7f4f80111640 1 -- 172.21.15.39:0/672208421 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4f7819f550 con 0x7f4f781083a0 2024-09-21T03:51:26.519 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:26.510+0000 7f4f76ffd640 1 -- 172.21.15.39:0/672208421 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4f6c003aa0 con 0x7f4f781083a0 2024-09-21T03:51:26.519 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:26.510+0000 7f4f76ffd640 1 -- 172.21.15.39:0/672208421 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f6c035050 con 0x7f4f781083a0 2024-09-21T03:51:26.519 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:26.510+0000 7f4f80111640 1 -- 172.21.15.39:0/672208421 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4f7819fa80 con 0x7f4f781083a0 2024-09-21T03:51:26.519 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:26.513+0000 7f4f86685740 1 Processor -- start 2024-09-21T03:51:26.519 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:26.513+0000 7f4f86685740 1 -- start start 2024-09-21T03:51:26.519 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:26.513+0000 7f4f86685740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5611027c9070 0x561102b702c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:51:26.519 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:26.513+0000 7f4f86685740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5611027a01b0 0x561102b70800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:51:26.519 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:26.513+0000 7f4f76ffd640 1 -- 172.21.15.39:0/672208421 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 27) ==== 154643+0+0 (secure 0 0 0) 0x7f4f6c049070 con 0x7f4f781083a0 2024-09-21T03:51:26.519 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:26.513+0000 7f4f86685740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5611027a8390 0x561102b70d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:51:26.519 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:26.513+0000 7f4f86685740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x561102ae6ea0 con 0x5611027a8390 2024-09-21T03:51:26.519 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:26.513+0000 7f4f86685740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x561102ad62d0 con 0x5611027a01b0 2024-09-21T03:51:26.519 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:26.513+0000 7f4f86685740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x561102ada0e0 con 0x5611027c9070 2024-09-21T03:51:26.519 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:26.514+0000 7f4f7d6ac640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5611027a8390 0x561102b70d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:51:26.519 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:26.514+0000 7f4f7d6ac640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5611027a8390 0x561102b70d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:36498/0 (socket says 172.21.15.39:36498) 2024-09-21T03:51:26.520 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:26.514+0000 7f4f7d6ac640 1 -- 172.21.15.39:0/3654564621 learned_addr learned my addr 172.21.15.39:0/3654564621 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:51:26.520 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:26.514+0000 7f4f7e6ae640 1 --2- 172.21.15.39:0/3654564621 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5611027c9070 0x561102b702c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:51:26.520 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:26.514+0000 7f4f7dead640 1 --2- 172.21.15.39:0/3654564621 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5611027a01b0 0x561102b70800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:51:26.520 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:26.514+0000 7f4f7d6ac640 1 -- 172.21.15.39:0/3654564621 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5611027c9070 msgr2=0x561102b702c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:51:26.520 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:26.514+0000 7f4f7d6ac640 1 --2- 172.21.15.39:0/3654564621 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5611027c9070 0x561102b702c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:26.520 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:26.514+0000 7f4f7d6ac640 1 -- 172.21.15.39:0/3654564621 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5611027a01b0 msgr2=0x561102b70800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:51:26.520 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:26.514+0000 7f4f7d6ac640 1 --2- 172.21.15.39:0/3654564621 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5611027a01b0 0x561102b70800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:26.520 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:26.514+0000 7f4f7d6ac640 1 -- 172.21.15.39:0/3654564621 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561102834650 con 0x5611027a8390 2024-09-21T03:51:26.520 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:26.514+0000 7f4f7d6ac640 1 --2- 172.21.15.39:0/3654564621 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5611027a8390 0x561102b70d40 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7f4f6c106940 tx=0x7f4f6c111600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=badaae8a6a8a2064 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:51:26.524 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:26.515+0000 7f4f5ffff640 1 -- 172.21.15.39:0/3654564621 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f6c10b2e0 con 0x5611027a8390 2024-09-21T03:51:26.524 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:26.515+0000 7f4f5ffff640 1 -- 172.21.15.39:0/3654564621 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4f6c116030 con 0x5611027a8390 2024-09-21T03:51:26.524 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:26.515+0000 7f4f5ffff640 1 -- 172.21.15.39:0/3654564621 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f6c11b8e0 con 0x5611027a8390 2024-09-21T03:51:26.524 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:26.515+0000 7f4f76ffd640 1 --2- 172.21.15.39:0/672208421 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f4f600b66b0 0x7f4f600b8bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:51:26.524 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:26.515+0000 7f4f7e6ae640 1 --2- 172.21.15.39:0/672208421 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f4f600b66b0 0x7f4f600b8bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:51:26.524 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:26.515+0000 7f4f76ffd640 1 -- 172.21.15.39:0/672208421 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8484+0+0 (secure 0 0 0) 0x7f4f6c109260 con 0x7f4f781083a0 2024-09-21T03:51:26.524 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:26.516+0000 7f4f86685740 1 -- 172.21.15.39:0/3654564621 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5611027a8390 msgr2=0x561102b70d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:51:26.524 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:26.516+0000 7f4f7e6ae640 1 --2- 172.21.15.39:0/672208421 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f4f600b66b0 0x7f4f600b8bc0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f4f68005cc0 tx=0x7f4f6800b580 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:51:26.524 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:26.516+0000 7f4f86685740 1 --2- 172.21.15.39:0/3654564621 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5611027a8390 0x561102b70d40 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7f4f6c106940 tx=0x7f4f6c111600 comp rx=0 tx=0).stop 2024-09-21T03:51:26.524 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:26.516+0000 7f4f86685740 1 -- 172.21.15.39:0/3654564621 shutdown_connections 2024-09-21T03:51:26.524 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:26.516+0000 7f4f86685740 1 --2- 172.21.15.39:0/3654564621 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5611027a8390 0x561102b70d40 unknown :-1 s=CLOSED pgs=881 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:26.524 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:26.516+0000 7f4f86685740 1 --2- 172.21.15.39:0/3654564621 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5611027a01b0 0x561102b70800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:26.525 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:26.516+0000 7f4f86685740 1 --2- 172.21.15.39:0/3654564621 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5611027c9070 0x561102b702c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:26.525 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:26.516+0000 7f4f86685740 1 -- 172.21.15.39:0/3654564621 >> 172.21.15.39:0/3654564621 conn(0x5611027bd460 msgr2=0x561102b6e1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:51:26.525 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:26.516+0000 7f4f86685740 1 -- 172.21.15.39:0/3654564621 shutdown_connections 2024-09-21T03:51:26.525 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:26.516+0000 7f4f86685740 1 -- 172.21.15.39:0/3654564621 wait complete. 2024-09-21T03:51:26.525 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:26.517+0000 7f4f86685740 1 -- start start 2024-09-21T03:51:26.525 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:26.517+0000 7f4f86685740 1 Processor -- start 2024-09-21T03:51:26.525 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:26.517+0000 7f4f86685740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5611027a01b0 0x561102b7b930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:51:26.525 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:26.517+0000 7f4f86685740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5611027a8390 0x561102b7be70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:51:26.525 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:26.517+0000 7f4f86685740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5611027c9070 0x561102b87ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:51:26.525 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:26.517+0000 7f4f86685740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x561102937700 con 0x5611027a01b0 2024-09-21T03:51:26.525 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:26.517+0000 7f4f86685740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x561102a71160 con 0x5611027a8390 2024-09-21T03:51:26.525 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:26.517+0000 7f4f86685740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x561102b7e860 con 0x5611027c9070 2024-09-21T03:51:26.525 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:26.518+0000 7f4f7dead640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5611027a01b0 0x561102b7b930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:51:26.525 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:26.518+0000 7f4f86685740 10 client.0 _ll_register_callbacks cb 0x561102a8a9e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:51:26.525 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:26.518+0000 7f4f7dead640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5611027a01b0 0x561102b7b930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:36500/0 (socket says 172.21.15.39:36500) 2024-09-21T03:51:26.526 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:26.518+0000 7f4f7dead640 1 -- 172.21.15.39:0/1286760747 learned_addr learned my addr 172.21.15.39:0/1286760747 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:51:26.526 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:26.518+0000 7f4f7d6ac640 1 --2- 172.21.15.39:0/1286760747 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5611027a8390 0x561102b7be70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:51:26.526 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:26.518+0000 7f4f7e6ae640 1 --2- 172.21.15.39:0/1286760747 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5611027c9070 0x561102b87ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:51:26.526 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:26.518+0000 7f4f7dead640 1 -- 172.21.15.39:0/1286760747 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5611027c9070 msgr2=0x561102b87ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:51:26.526 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:26.518+0000 7f4f7dead640 1 --2- 172.21.15.39:0/1286760747 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5611027c9070 0x561102b87ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:26.526 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:26.518+0000 7f4f7dead640 1 -- 172.21.15.39:0/1286760747 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5611027a8390 msgr2=0x561102b7be70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:51:26.526 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:26.518+0000 7f4f7dead640 1 --2- 172.21.15.39:0/1286760747 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5611027a8390 0x561102b7be70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:26.526 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:26.518+0000 7f4f7dead640 1 -- 172.21.15.39:0/1286760747 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561102a9ec90 con 0x5611027a01b0 2024-09-21T03:51:26.526 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:26.518+0000 7f4f7dead640 1 --2- 172.21.15.39:0/1286760747 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5611027a01b0 0x561102b7b930 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x7f4f70009e90 tx=0x7f4f700117b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ad476939a1976f6 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:51:26.526 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:26.518+0000 7f4f5d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-09-21T03:51:26.526 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:26.518+0000 7f4f5d7fa640 1 -- 172.21.15.39:0/1286760747 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f70012860 con 0x5611027a01b0 2024-09-21T03:51:26.526 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:26.518+0000 7f4f5d7fa640 1 -- 172.21.15.39:0/1286760747 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8484+0+0 (secure 0 0 0) 0x7f4f70012a00 con 0x5611027a01b0 2024-09-21T03:51:26.526 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:26.518+0000 7f4f86685740 1 -- 172.21.15.39:0/1286760747 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4f6c116030 con 0x5611027a01b0 2024-09-21T03:51:26.526 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:26.519+0000 7f4f5d7fa640 1 -- 172.21.15.39:0/1286760747 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f4f70012c00 con 0x5611027a01b0 2024-09-21T03:51:26.527 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:26.519+0000 7f4f86685740 10 client.16920 fetch_fsmap learned FSMap version 29 2024-09-21T03:51:26.527 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:26.519+0000 7f4f86685740 1 -- 172.21.15.39:0/1286760747 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x561102b8a500 con 0x5611027a01b0 2024-09-21T03:51:26.527 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:26.519+0000 7f4f5d7fa640 1 -- 172.21.15.39:0/1286760747 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4f6c116030 con 0x5611027a01b0 2024-09-21T03:51:26.527 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:26.519+0000 7f4f5d7fa640 1 -- 172.21.15.39:0/1286760747 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f4f70021030 con 0x5611027a01b0 2024-09-21T03:51:26.527 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:26.519+0000 7f4f86685740 10 client.16920 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T03:51:26.527 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:26.519+0000 7f4f86685740 10 client.16920 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-21T03:51:26.527 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:26.519+0000 7f4f86685740 20 client.16920 populate_metadata read hostname 'smithi039' 2024-09-21T03:51:26.527 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:26.519+0000 7f4f86685740 1 --2- 172.21.15.39:0/1286760747 >> [v2:172.21.15.150:6836/4017043144,v1:172.21.15.150:6837/4017043144] conn(0x5611027879f0 0x561102b8da60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:51:26.527 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:26.519+0000 7f4f86685740 4 client.16920 mds_command: new command op to 24659 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:51:26.527 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:26.519+0000 7f4f86685740 1 -- 172.21.15.39:0/1286760747 --> [v2:172.21.15.150:6836/4017043144,v1:172.21.15.150:6837/4017043144] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561102ae8df0 con 0x5611027879f0 2024-09-21T03:51:26.527 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:26.520+0000 7f4f7d6ac640 1 --2- 172.21.15.39:0/1286760747 >> [v2:172.21.15.150:6836/4017043144,v1:172.21.15.150:6837/4017043144] conn(0x5611027879f0 0x561102b8da60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:51:26.527 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:26.520+0000 7f4f7d6ac640 1 --2- 172.21.15.39:0/1286760747 >> [v2:172.21.15.150:6836/4017043144,v1:172.21.15.150:6837/4017043144] conn(0x5611027879f0 0x561102b8da60 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=89be901fd4d82070 server_cookie=da11ff423ed640e6 in_seq=0 out_seq=0 2024-09-21T03:51:26.527 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:26.520+0000 7f4f5d7fa640 10 client.16920 ms_handle_connect on v2:172.21.15.150:6836/4017043144 2024-09-21T03:51:26.527 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:26.521+0000 7f4f5d7fa640 1 -- 172.21.15.39:0/1286760747 <== mds.2 v2:172.21.15.150:6836/4017043144 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x561102ae8df0 con 0x5611027879f0 2024-09-21T03:51:26.528 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:26.521+0000 7f4f5d7fa640 10 client.16920 handle_command_reply: tid=0 2024-09-21T03:51:26.528 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:26.521+0000 7f4f86685740 1 client.16920 shutdown 2024-09-21T03:51:26.528 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:26.521+0000 7f4f86685740 1 -- 172.21.15.39:0/1286760747 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5611027a01b0 msgr2=0x561102b7b930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:51:26.528 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:26.521+0000 7f4f86685740 1 --2- 172.21.15.39:0/1286760747 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5611027a01b0 0x561102b7b930 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x7f4f70009e90 tx=0x7f4f700117b0 comp rx=0 tx=0).stop 2024-09-21T03:51:26.528 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:26.521+0000 7f4f86685740 1 -- 172.21.15.39:0/1286760747 shutdown_connections 2024-09-21T03:51:26.528 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:26.521+0000 7f4f86685740 1 --2- 172.21.15.39:0/1286760747 >> [v2:172.21.15.150:6836/4017043144,v1:172.21.15.150:6837/4017043144] conn(0x5611027879f0 0x561102b8da60 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:26.528 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:26.521+0000 7f4f86685740 1 --2- 172.21.15.39:0/1286760747 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5611027c9070 0x561102b87ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:26.528 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:26.521+0000 7f4f5d7fa640 0 client.16920 ms_handle_reset on v2:172.21.15.150:6836/4017043144 2024-09-21T03:51:26.528 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:26.521+0000 7f4f86685740 1 --2- 172.21.15.39:0/1286760747 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5611027a8390 0x561102b7be70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:26.528 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:26.521+0000 7f4f86685740 1 --2- 172.21.15.39:0/1286760747 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5611027a01b0 0x561102b7b930 unknown :-1 s=CLOSED pgs=882 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:26.528 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:26.521+0000 7f4f86685740 1 -- 172.21.15.39:0/1286760747 >> 172.21.15.39:0/1286760747 conn(0x5611027bd460 msgr2=0x561102b73450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:51:26.528 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:26.521+0000 7f4f86685740 1 -- 172.21.15.39:0/1286760747 shutdown_connections 2024-09-21T03:51:26.528 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:26.521+0000 7f4f86685740 1 -- 172.21.15.39:0/1286760747 wait complete. 2024-09-21T03:51:26.528 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:26.521+0000 7f4f86685740 20 client.16920 trim_cache size 0 max 16384 2024-09-21T03:51:26.537 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:26.534+0000 7f4f86685740 1 Processor -- start 2024-09-21T03:51:26.537 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:26.534+0000 7f4f86685740 1 -- start start 2024-09-21T03:51:26.537 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:26.535+0000 7f4f86685740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5611027a01b0 0x561102b71a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:51:26.537 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:26.535+0000 7f4f86685740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5611027a8390 0x561102b6d760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:51:26.537 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:26.535+0000 7f4f86685740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5611027c9070 0x561102b6dca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:51:26.537 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:26.535+0000 7f4f86685740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f4f781ad4f0 con 0x5611027c9070 2024-09-21T03:51:26.538 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:26.535+0000 7f4f86685740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f4f781af030 con 0x5611027a8390 2024-09-21T03:51:26.538 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:26.535+0000 7f4f86685740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f4f781adbc0 con 0x5611027a01b0 2024-09-21T03:51:26.538 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:26.535+0000 7f4f7e6ae640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5611027c9070 0x561102b6dca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:51:26.538 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:26.535+0000 7f4f7e6ae640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5611027c9070 0x561102b6dca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:36502/0 (socket says 172.21.15.39:36502) 2024-09-21T03:51:26.538 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:26.535+0000 7f4f7e6ae640 1 -- 172.21.15.39:0/1102084522 learned_addr learned my addr 172.21.15.39:0/1102084522 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:51:26.538 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:26.535+0000 7f4f7d6ac640 1 --2- 172.21.15.39:0/1102084522 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5611027a8390 0x561102b6d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:51:26.538 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:26.535+0000 7f4f7dead640 1 --2- 172.21.15.39:0/1102084522 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5611027a01b0 0x561102b71a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:51:26.538 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:26.535+0000 7f4f7e6ae640 1 -- 172.21.15.39:0/1102084522 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5611027a01b0 msgr2=0x561102b71a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:51:26.538 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:26.535+0000 7f4f7e6ae640 1 --2- 172.21.15.39:0/1102084522 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5611027a01b0 0x561102b71a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:26.538 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:26.535+0000 7f4f7e6ae640 1 -- 172.21.15.39:0/1102084522 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5611027a8390 msgr2=0x561102b6d760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:51:26.538 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:26.535+0000 7f4f7e6ae640 1 --2- 172.21.15.39:0/1102084522 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5611027a8390 0x561102b6d760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:26.538 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:26.535+0000 7f4f7e6ae640 1 -- 172.21.15.39:0/1102084522 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561102b7c880 con 0x5611027c9070 2024-09-21T03:51:26.538 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:26.535+0000 7f4f7d6ac640 1 --2- 172.21.15.39:0/1102084522 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5611027a8390 0x561102b6d760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:51:26.538 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:26.535+0000 7f4f7dead640 1 --2- 172.21.15.39:0/1102084522 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5611027a01b0 0x561102b71a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:51:26.539 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:26.535+0000 7f4f7e6ae640 1 --2- 172.21.15.39:0/1102084522 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5611027c9070 0x561102b6dca0 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x7f4f6801d560 tx=0x7f4f68013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e54dcd1a98759681 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:51:26.539 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:26.536+0000 7f4f5f7fe640 1 -- 172.21.15.39:0/1102084522 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f6801e9c0 con 0x5611027c9070 2024-09-21T03:51:26.539 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:26.536+0000 7f4f5f7fe640 1 -- 172.21.15.39:0/1102084522 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4f68022070 con 0x5611027c9070 2024-09-21T03:51:26.539 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:26.536+0000 7f4f5f7fe640 1 -- 172.21.15.39:0/1102084522 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f680193b0 con 0x5611027c9070 2024-09-21T03:51:26.542 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:26.536+0000 7f4f86685740 1 -- 172.21.15.39:0/1102084522 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5611027c9070 msgr2=0x561102b6dca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:51:26.542 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:26.536+0000 7f4f86685740 1 --2- 172.21.15.39:0/1102084522 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5611027c9070 0x561102b6dca0 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x7f4f6801d560 tx=0x7f4f68013000 comp rx=0 tx=0).stop 2024-09-21T03:51:26.542 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:26.536+0000 7f4f86685740 1 -- 172.21.15.39:0/1102084522 shutdown_connections 2024-09-21T03:51:26.542 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:26.536+0000 7f4f86685740 1 --2- 172.21.15.39:0/1102084522 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5611027c9070 0x561102b6dca0 unknown :-1 s=CLOSED pgs=883 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:26.542 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:26.536+0000 7f4f86685740 1 --2- 172.21.15.39:0/1102084522 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5611027a8390 0x561102b6d760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:26.542 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:26.536+0000 7f4f86685740 1 --2- 172.21.15.39:0/1102084522 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5611027a01b0 0x561102b71a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:26.542 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:26.536+0000 7f4f86685740 1 -- 172.21.15.39:0/1102084522 >> 172.21.15.39:0/1102084522 conn(0x5611027bd460 msgr2=0x561102b712f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:51:26.542 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:26.537+0000 7f4f86685740 1 -- 172.21.15.39:0/1102084522 shutdown_connections 2024-09-21T03:51:26.542 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:26.537+0000 7f4f86685740 1 -- 172.21.15.39:0/1102084522 wait complete. 2024-09-21T03:51:26.542 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:26.537+0000 7f4f86685740 1 -- start start 2024-09-21T03:51:26.543 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:26.537+0000 7f4f86685740 1 Processor -- start 2024-09-21T03:51:26.543 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:26.537+0000 7f4f86685740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5611027a01b0 0x561102b84440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:51:26.543 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:26.537+0000 7f4f86685740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5611027a8390 0x561102b7f950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:51:26.543 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:26.537+0000 7f4f86685740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5611027c9070 0x561102b80060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:51:26.543 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:26.538+0000 7f4f86685740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f4f781ae960 con 0x5611027a01b0 2024-09-21T03:51:26.543 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:26.538+0000 7f4f86685740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f4f781ae290 con 0x5611027c9070 2024-09-21T03:51:26.543 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:26.538+0000 7f4f86685740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f4f781ab4d0 con 0x5611027a8390 2024-09-21T03:51:26.543 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:26.538+0000 7f4f86685740 10 client.0 _ll_register_callbacks cb 0x7f4f70009800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:51:26.543 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:26.538+0000 7f4f7dead640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5611027a01b0 0x561102b84440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:51:26.543 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:26.538+0000 7f4f7dead640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5611027a01b0 0x561102b84440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:36514/0 (socket says 172.21.15.39:36514) 2024-09-21T03:51:26.543 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:26.538+0000 7f4f7dead640 1 -- 172.21.15.39:0/1550289301 learned_addr learned my addr 172.21.15.39:0/1550289301 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:51:26.543 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:26.538+0000 7f4f7d6ac640 1 --2- 172.21.15.39:0/1550289301 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5611027a8390 0x561102b7f950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:51:26.543 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:26.538+0000 7f4f7e6ae640 1 --2- 172.21.15.39:0/1550289301 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5611027c9070 0x561102b80060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:51:26.543 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:26.538+0000 7f4f7dead640 1 -- 172.21.15.39:0/1550289301 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5611027a8390 msgr2=0x561102b7f950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:51:26.544 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:26.538+0000 7f4f7dead640 1 --2- 172.21.15.39:0/1550289301 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5611027a8390 0x561102b7f950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:26.544 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:26.538+0000 7f4f7dead640 1 -- 172.21.15.39:0/1550289301 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5611027c9070 msgr2=0x561102b80060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:51:26.544 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:26.538+0000 7f4f7dead640 1 --2- 172.21.15.39:0/1550289301 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5611027c9070 0x561102b80060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:26.544 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:26.538+0000 7f4f7dead640 1 -- 172.21.15.39:0/1550289301 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561102bf7460 con 0x5611027a01b0 2024-09-21T03:51:26.544 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:26.538+0000 7f4f7e6ae640 1 --2- 172.21.15.39:0/1550289301 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5611027c9070 0x561102b80060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:51:26.544 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:26.538+0000 7f4f7dead640 1 --2- 172.21.15.39:0/1550289301 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5611027a01b0 0x561102b84440 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 crypto rx=0x7f4f70009e90 tx=0x7f4f70005c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a9e093aa067d6895 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:51:26.544 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:26.538+0000 7f4f5d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-09-21T03:51:26.544 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:26.538+0000 7f4f5d7fa640 1 -- 172.21.15.39:0/1550289301 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f70030020 con 0x5611027a01b0 2024-09-21T03:51:26.544 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:26.539+0000 7f4f5d7fa640 1 -- 172.21.15.39:0/1550289301 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8484+0+0 (secure 0 0 0) 0x7f4f70004020 con 0x5611027a01b0 2024-09-21T03:51:26.544 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:26.539+0000 7f4f86685740 1 -- 172.21.15.39:0/1550289301 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4f68022070 con 0x5611027a01b0 2024-09-21T03:51:26.544 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:26.539+0000 7f4f5d7fa640 1 -- 172.21.15.39:0/1550289301 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f4f70004220 con 0x5611027a01b0 2024-09-21T03:51:26.544 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:26.539+0000 7f4f86685740 10 client.16926 fetch_fsmap learned FSMap version 29 2024-09-21T03:51:26.544 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:26.539+0000 7f4f86685740 1 -- 172.21.15.39:0/1550289301 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x561102bf85e0 con 0x5611027a01b0 2024-09-21T03:51:26.544 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:26.539+0000 7f4f5d7fa640 1 -- 172.21.15.39:0/1550289301 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4f68022070 con 0x5611027a01b0 2024-09-21T03:51:26.545 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:26.539+0000 7f4f5d7fa640 1 -- 172.21.15.39:0/1550289301 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f4f7000f040 con 0x5611027a01b0 2024-09-21T03:51:26.545 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:26.539+0000 7f4f86685740 10 client.16926 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T03:51:26.545 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:26.539+0000 7f4f86685740 10 client.16926 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-21T03:51:26.545 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:26.539+0000 7f4f86685740 20 client.16926 populate_metadata read hostname 'smithi039' 2024-09-21T03:51:26.545 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:26.539+0000 7f4f86685740 1 --2- 172.21.15.39:0/1550289301 >> [v2:172.21.15.150:6836/4017043144,v1:172.21.15.150:6837/4017043144] conn(0x56110275cda0 0x561102bfae50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:51:26.545 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:26.539+0000 7f4f86685740 4 client.16926 mds_command: new command op to 24659 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T03:51:26.545 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:26.539+0000 7f4f86685740 1 -- 172.21.15.39:0/1550289301 --> [v2:172.21.15.150:6836/4017043144,v1:172.21.15.150:6837/4017043144] -- command(tid 0: {"prefix": "damage ls"}) -- 0x561102ae48a0 con 0x56110275cda0 2024-09-21T03:51:26.545 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:26.540+0000 7f4f7d6ac640 1 --2- 172.21.15.39:0/1550289301 >> [v2:172.21.15.150:6836/4017043144,v1:172.21.15.150:6837/4017043144] conn(0x56110275cda0 0x561102bfae50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:51:26.545 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:26.540+0000 7f4f7d6ac640 1 --2- 172.21.15.39:0/1550289301 >> [v2:172.21.15.150:6836/4017043144,v1:172.21.15.150:6837/4017043144] conn(0x56110275cda0 0x561102bfae50 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=7699793ad167fdf4 server_cookie=61a8594460594612 in_seq=0 out_seq=0 2024-09-21T03:51:26.545 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:26.540+0000 7f4f5d7fa640 10 client.16926 ms_handle_connect on v2:172.21.15.150:6836/4017043144 2024-09-21T03:51:26.545 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:26.540+0000 7f4f5d7fa640 1 -- 172.21.15.39:0/1550289301 <== mds.2 v2:172.21.15.150:6836/4017043144 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x561102ae48a0 con 0x56110275cda0 2024-09-21T03:51:26.545 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:26.540+0000 7f4f5d7fa640 10 client.16926 handle_command_reply: tid=0 2024-09-21T03:51:26.545 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:26.540+0000 7f4f86685740 1 client.16926 shutdown 2024-09-21T03:51:26.545 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:26.540+0000 7f4f86685740 1 -- 172.21.15.39:0/1550289301 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5611027a01b0 msgr2=0x561102b84440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:51:26.546 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:26.540+0000 7f4f86685740 1 --2- 172.21.15.39:0/1550289301 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5611027a01b0 0x561102b84440 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 crypto rx=0x7f4f70009e90 tx=0x7f4f70005c10 comp rx=0 tx=0).stop 2024-09-21T03:51:26.546 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:26.540+0000 7f4f86685740 1 -- 172.21.15.39:0/1550289301 shutdown_connections 2024-09-21T03:51:26.546 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:26.540+0000 7f4f86685740 1 --2- 172.21.15.39:0/1550289301 >> [v2:172.21.15.150:6836/4017043144,v1:172.21.15.150:6837/4017043144] conn(0x56110275cda0 0x561102bfae50 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:26.546 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:26.540+0000 7f4f86685740 1 --2- 172.21.15.39:0/1550289301 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5611027c9070 0x561102b80060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:26.546 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:26.540+0000 7f4f86685740 1 --2- 172.21.15.39:0/1550289301 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5611027a8390 0x561102b7f950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:26.546 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:26.540+0000 7f4f86685740 1 --2- 172.21.15.39:0/1550289301 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5611027a01b0 0x561102b84440 unknown :-1 s=CLOSED pgs=884 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:26.547 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:26.540+0000 7f4f5d7fa640 0 client.16926 ms_handle_reset on v2:172.21.15.150:6836/4017043144 2024-09-21T03:51:26.547 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:26.540+0000 7f4f86685740 1 -- 172.21.15.39:0/1550289301 >> 172.21.15.39:0/1550289301 conn(0x5611027bd460 msgr2=0x561102b7be70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:51:26.547 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:26.540+0000 7f4f86685740 1 -- 172.21.15.39:0/1550289301 shutdown_connections 2024-09-21T03:51:26.547 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:26.541+0000 7f4f86685740 1 -- 172.21.15.39:0/1550289301 wait complete. 2024-09-21T03:51:26.547 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:26.541+0000 7f4f86685740 20 client.16926 trim_cache size 0 max 16384 2024-09-21T03:51:26.547 INFO:teuthology.orchestra.run.smithi039.stdout:[] 2024-09-21T03:51:26.548 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:26.541+0000 7f4f80111640 1 -- 172.21.15.39:0/672208421 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f4f600b66b0 msgr2=0x7f4f600b8bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:51:26.548 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:26.541+0000 7f4f80111640 1 --2- 172.21.15.39:0/672208421 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f4f600b66b0 0x7f4f600b8bc0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f4f68005cc0 tx=0x7f4f6800b580 comp rx=0 tx=0).stop 2024-09-21T03:51:26.548 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:26.541+0000 7f4f80111640 1 -- 172.21.15.39:0/672208421 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4f781083a0 msgr2=0x7f4f781999b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:51:26.548 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:26.541+0000 7f4f80111640 1 --2- 172.21.15.39:0/672208421 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4f781083a0 0x7f4f781999b0 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x7f4f6c032fd0 tx=0x7f4f6c004270 comp rx=0 tx=0).stop 2024-09-21T03:51:26.548 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:26.541+0000 7f4f80111640 1 -- 172.21.15.39:0/672208421 shutdown_connections 2024-09-21T03:51:26.548 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:26.541+0000 7f4f80111640 1 --2- 172.21.15.39:0/672208421 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f4f600b66b0 0x7f4f600b8bc0 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:26.548 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:26.541+0000 7f4f80111640 1 --2- 172.21.15.39:0/672208421 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4f78108cf0 0x7f4f7819ec50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:26.548 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:26.541+0000 7f4f80111640 1 --2- 172.21.15.39:0/672208421 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4f781083a0 0x7f4f781999b0 unknown :-1 s=CLOSED pgs=880 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:26.548 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:26.541+0000 7f4f80111640 1 --2- 172.21.15.39:0/672208421 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4f78102a60 0x7f4f78199410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:26.548 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:26.541+0000 7f4f80111640 1 -- 172.21.15.39:0/672208421 >> 172.21.15.39:0/672208421 conn(0x7f4f78079730 msgr2=0x7f4f781044d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:51:26.549 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:26.541+0000 7f4f80111640 1 -- 172.21.15.39:0/672208421 shutdown_connections 2024-09-21T03:51:26.549 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:26.541+0000 7f4f80111640 1 -- 172.21.15.39:0/672208421 wait complete. 2024-09-21T03:51:27.240 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:51:27 smithi039 ceph-mon[31211]: pgmap v487: 97 pgs: 97 active+clean; 10 MiB data, 520 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 10 op/s 2024-09-21T03:51:27.265 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 51116d5c-77ca-11ef-baf5-efdc52797490 -- ceph tell mds.1:1 damage ls 2024-09-21T03:51:27.468 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:51:27 smithi161 ceph-mon[34533]: pgmap v487: 97 pgs: 97 active+clean; 10 MiB data, 520 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 10 op/s 2024-09-21T03:51:27.474 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:51:27 smithi150 ceph-mon[34289]: pgmap v487: 97 pgs: 97 active+clean; 10 MiB data, 520 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 10 op/s 2024-09-21T03:51:27.523 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/51116d5c-77ca-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:51:28.009 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:28.007+0000 7f61d6dfc640 1 -- 172.21.15.39:0/2848437630 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f61d0109eb0 msgr2=0x7f61d01106e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:51:28.009 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:28.007+0000 7f61d6dfc640 1 --2- 172.21.15.39:0/2848437630 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f61d0109eb0 0x7f61d01106e0 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x7f61c8009ca0 tx=0x7f61c802a8c0 comp rx=0 tx=0).stop 2024-09-21T03:51:28.009 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:28.007+0000 7f61d6dfc640 1 -- 172.21.15.39:0/2848437630 shutdown_connections 2024-09-21T03:51:28.009 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:28.007+0000 7f61d6dfc640 1 --2- 172.21.15.39:0/2848437630 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f61d0109eb0 0x7f61d01106e0 unknown :-1 s=CLOSED pgs=885 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:28.009 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:28.007+0000 7f61d6dfc640 1 --2- 172.21.15.39:0/2848437630 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f61d0109560 0x7f61d0109970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:28.009 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:28.007+0000 7f61d6dfc640 1 --2- 172.21.15.39:0/2848437630 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f61d0102080 0x7f61d0102490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:28.009 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:28.007+0000 7f61d6dfc640 1 -- 172.21.15.39:0/2848437630 >> 172.21.15.39:0/2848437630 conn(0x7f61d0079ec0 msgr2=0x7f61d007a2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:51:28.009 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:28.007+0000 7f61d6dfc640 1 -- 172.21.15.39:0/2848437630 shutdown_connections 2024-09-21T03:51:28.009 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:28.007+0000 7f61d6dfc640 1 -- 172.21.15.39:0/2848437630 wait complete. 2024-09-21T03:51:28.011 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:28.007+0000 7f61d6dfc640 1 Processor -- start 2024-09-21T03:51:28.011 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:28.007+0000 7f61d6dfc640 1 -- start start 2024-09-21T03:51:28.011 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:28.008+0000 7f61d6dfc640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f61d0102080 0x7f61d0103020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:51:28.011 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:28.008+0000 7f61d6dfc640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f61d0109560 0x7f61d0103560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:51:28.011 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:28.008+0000 7f61d6dfc640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f61d0109eb0 0x7f61d0103aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:51:28.011 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:28.008+0000 7f61d6dfc640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f61d0113b70 con 0x7f61d0102080 2024-09-21T03:51:28.011 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:28.008+0000 7f61d6dfc640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f61d01139f0 con 0x7f61d0109560 2024-09-21T03:51:28.011 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:28.008+0000 7f61d6dfc640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f61d0113cf0 con 0x7f61d0109eb0 2024-09-21T03:51:28.012 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:28.008+0000 7f61d4b98640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f61d0102080 0x7f61d0103020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:51:28.012 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:28.008+0000 7f61d4b98640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f61d0102080 0x7f61d0103020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:40252/0 (socket says 172.21.15.39:40252) 2024-09-21T03:51:28.012 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:28.008+0000 7f61d4b98640 1 -- 172.21.15.39:0/3173261245 learned_addr learned my addr 172.21.15.39:0/3173261245 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:51:28.012 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:28.008+0000 7f61d5399640 1 --2- 172.21.15.39:0/3173261245 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f61d0109eb0 0x7f61d0103aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:51:28.012 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:28.008+0000 7f61cffff640 1 --2- 172.21.15.39:0/3173261245 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f61d0109560 0x7f61d0103560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:51:28.012 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:28.008+0000 7f61d4b98640 1 -- 172.21.15.39:0/3173261245 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f61d0109eb0 msgr2=0x7f61d0103aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:51:28.012 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:28.008+0000 7f61d4b98640 1 --2- 172.21.15.39:0/3173261245 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f61d0109eb0 0x7f61d0103aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:28.012 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:28.008+0000 7f61d4b98640 1 -- 172.21.15.39:0/3173261245 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f61d0109560 msgr2=0x7f61d0103560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:51:28.012 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:28.008+0000 7f61d4b98640 1 --2- 172.21.15.39:0/3173261245 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f61d0109560 0x7f61d0103560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:28.012 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:28.008+0000 7f61d4b98640 1 -- 172.21.15.39:0/3173261245 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f61d01a87c0 con 0x7f61d0102080 2024-09-21T03:51:28.012 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:28.008+0000 7f61d4b98640 1 --2- 172.21.15.39:0/3173261245 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f61d0102080 0x7f61d0103020 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 crypto rx=0x7f61c400c890 tx=0x7f61c400cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:51:28.012 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:28.009+0000 7f61cdffb640 1 -- 172.21.15.39:0/3173261245 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61c4004220 con 0x7f61d0102080 2024-09-21T03:51:28.012 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:28.009+0000 7f61d6dfc640 1 -- 172.21.15.39:0/3173261245 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f61d01a8ab0 con 0x7f61d0102080 2024-09-21T03:51:28.013 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:28.009+0000 7f61cdffb640 1 -- 172.21.15.39:0/3173261245 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f61c40043c0 con 0x7f61d0102080 2024-09-21T03:51:28.013 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:28.009+0000 7f61cdffb640 1 -- 172.21.15.39:0/3173261245 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61c4004a00 con 0x7f61d0102080 2024-09-21T03:51:28.013 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:28.009+0000 7f61d6dfc640 1 -- 172.21.15.39:0/3173261245 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f61d01a8ff0 con 0x7f61d0102080 2024-09-21T03:51:28.014 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:28.011+0000 7f61dd370740 1 Processor -- start 2024-09-21T03:51:28.014 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:28.011+0000 7f61dd370740 1 -- start start 2024-09-21T03:51:28.014 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:28.012+0000 7f61dd370740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x560953ffb070 0x5609543a22c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:51:28.014 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:28.012+0000 7f61cdffb640 1 -- 172.21.15.39:0/3173261245 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 27) ==== 154643+0+0 (secure 0 0 0) 0x7f61c40079b0 con 0x7f61d0102080 2024-09-21T03:51:28.014 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:28.012+0000 7f61dd370740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560953fd21b0 0x5609543a2800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:51:28.015 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:28.012+0000 7f61dd370740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560953fda390 0x5609543a2d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:51:28.015 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:28.012+0000 7f61dd370740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x560954318ea0 con 0x560953ffb070 2024-09-21T03:51:28.015 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:28.012+0000 7f61dd370740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x5609543082d0 con 0x560953fda390 2024-09-21T03:51:28.015 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:28.012+0000 7f61dd370740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x56095430c0e0 con 0x560953fd21b0 2024-09-21T03:51:28.015 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:28.012+0000 7f61d5399640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x560953ffb070 0x5609543a22c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:51:28.015 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:28.012+0000 7f61d5399640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x560953ffb070 0x5609543a22c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:40256/0 (socket says 172.21.15.39:40256) 2024-09-21T03:51:28.015 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:28.012+0000 7f61d5399640 1 -- 172.21.15.39:0/2291727715 learned_addr learned my addr 172.21.15.39:0/2291727715 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:51:28.015 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:28.012+0000 7f61cffff640 1 --2- 172.21.15.39:0/2291727715 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560953fda390 0x5609543a2d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:51:28.015 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:28.012+0000 7f61d4b98640 1 --2- 172.21.15.39:0/2291727715 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560953fd21b0 0x5609543a2800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:51:28.015 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:28.012+0000 7f61d5399640 1 -- 172.21.15.39:0/2291727715 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560953fd21b0 msgr2=0x5609543a2800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:51:28.015 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:28.012+0000 7f61d5399640 1 --2- 172.21.15.39:0/2291727715 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560953fd21b0 0x5609543a2800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:28.015 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:28.012+0000 7f61d5399640 1 -- 172.21.15.39:0/2291727715 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560953fda390 msgr2=0x5609543a2d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:51:28.015 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:28.012+0000 7f61d5399640 1 --2- 172.21.15.39:0/2291727715 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560953fda390 0x5609543a2d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:28.015 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:28.012+0000 7f61d5399640 1 -- 172.21.15.39:0/2291727715 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560954066650 con 0x560953ffb070 2024-09-21T03:51:28.016 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:28.012+0000 7f61d4b98640 1 --2- 172.21.15.39:0/2291727715 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560953fd21b0 0x5609543a2800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:51:28.016 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:28.013+0000 7f61d5399640 1 --2- 172.21.15.39:0/2291727715 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x560953ffb070 0x5609543a22c0 secure :-1 s=READY pgs=887 cs=0 l=1 rev1=1 crypto rx=0x7f61c8009920 tx=0x7f61c8009950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=74234941a0b170ce server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:51:28.019 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:28.013+0000 7f61baffd640 1 -- 172.21.15.39:0/2291727715 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61c8031e30 con 0x560953ffb070 2024-09-21T03:51:28.019 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:28.013+0000 7f61cdffb640 1 --2- 172.21.15.39:0/3173261245 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f61c00b6680 0x7f61c00b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:51:28.019 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:28.013+0000 7f61baffd640 1 -- 172.21.15.39:0/2291727715 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f61c8036030 con 0x560953ffb070 2024-09-21T03:51:28.019 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:28.013+0000 7f61baffd640 1 -- 172.21.15.39:0/2291727715 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61c80042a0 con 0x560953ffb070 2024-09-21T03:51:28.019 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:28.013+0000 7f61d4b98640 1 --2- 172.21.15.39:0/3173261245 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f61c00b6680 0x7f61c00b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:51:28.019 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:28.013+0000 7f61cdffb640 1 -- 172.21.15.39:0/3173261245 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8484+0+0 (secure 0 0 0) 0x7f61c40e73d0 con 0x7f61d0102080 2024-09-21T03:51:28.019 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:28.014+0000 7f61dd370740 1 -- 172.21.15.39:0/2291727715 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x560953ffb070 msgr2=0x5609543a22c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:51:28.019 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:28.014+0000 7f61d4b98640 1 --2- 172.21.15.39:0/3173261245 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f61c00b6680 0x7f61c00b8b90 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f61c40e4bf0 tx=0x7f61c40e9040 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:51:28.019 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:28.014+0000 7f61dd370740 1 --2- 172.21.15.39:0/2291727715 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x560953ffb070 0x5609543a22c0 secure :-1 s=READY pgs=887 cs=0 l=1 rev1=1 crypto rx=0x7f61c8009920 tx=0x7f61c8009950 comp rx=0 tx=0).stop 2024-09-21T03:51:28.020 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:28.014+0000 7f61dd370740 1 -- 172.21.15.39:0/2291727715 shutdown_connections 2024-09-21T03:51:28.020 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:28.014+0000 7f61dd370740 1 --2- 172.21.15.39:0/2291727715 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560953fda390 0x5609543a2d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:28.020 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:28.014+0000 7f61dd370740 1 --2- 172.21.15.39:0/2291727715 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560953fd21b0 0x5609543a2800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:28.020 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:28.014+0000 7f61dd370740 1 --2- 172.21.15.39:0/2291727715 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x560953ffb070 0x5609543a22c0 unknown :-1 s=CLOSED pgs=887 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:28.020 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:28.014+0000 7f61dd370740 1 -- 172.21.15.39:0/2291727715 >> 172.21.15.39:0/2291727715 conn(0x560953fef460 msgr2=0x5609543a01b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:51:28.020 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:28.014+0000 7f61dd370740 1 -- 172.21.15.39:0/2291727715 shutdown_connections 2024-09-21T03:51:28.020 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:28.014+0000 7f61dd370740 1 -- 172.21.15.39:0/2291727715 wait complete. 2024-09-21T03:51:28.020 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:28.014+0000 7f61dd370740 1 -- start start 2024-09-21T03:51:28.020 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:28.014+0000 7f61dd370740 1 Processor -- start 2024-09-21T03:51:28.020 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:28.015+0000 7f61dd370740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560953fd21b0 0x5609543ad8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:51:28.020 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:28.015+0000 7f61dd370740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x560953fda390 0x5609543ade00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:51:28.020 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:28.015+0000 7f61dd370740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560953ffb070 0x5609543ae340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:51:28.020 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:28.015+0000 7f61dd370740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x560954169700 con 0x560953fda390 2024-09-21T03:51:28.021 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:28.015+0000 7f61dd370740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x5609542a3160 con 0x560953fd21b0 2024-09-21T03:51:28.021 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:28.015+0000 7f61dd370740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x5609543b08c0 con 0x560953ffb070 2024-09-21T03:51:28.021 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:28.015+0000 7f61dd370740 10 client.0 _ll_register_callbacks cb 0x5609542bc9e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:51:28.021 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:28.015+0000 7f61cffff640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x560953fda390 0x5609543ade00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:51:28.021 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:28.015+0000 7f61cffff640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x560953fda390 0x5609543ade00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:40272/0 (socket says 172.21.15.39:40272) 2024-09-21T03:51:28.021 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:28.015+0000 7f61cffff640 1 -- 172.21.15.39:0/2071573623 learned_addr learned my addr 172.21.15.39:0/2071573623 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:51:28.021 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:28.015+0000 7f61d5399640 1 --2- 172.21.15.39:0/2071573623 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560953ffb070 0x5609543ae340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:51:28.021 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:28.015+0000 7f61d5399640 1 --2- 172.21.15.39:0/2071573623 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560953fd21b0 0x5609543ad8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:51:28.021 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:28.016+0000 7f61cffff640 1 -- 172.21.15.39:0/2071573623 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560953ffb070 msgr2=0x5609543ae340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:51:28.021 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:28.016+0000 7f61cffff640 1 --2- 172.21.15.39:0/2071573623 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560953ffb070 0x5609543ae340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:28.021 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:28.016+0000 7f61cffff640 1 -- 172.21.15.39:0/2071573623 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560953fd21b0 msgr2=0x5609543ad8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:51:28.021 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:28.016+0000 7f61cffff640 1 --2- 172.21.15.39:0/2071573623 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560953fd21b0 0x5609543ad8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:28.021 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:28.016+0000 7f61cffff640 1 -- 172.21.15.39:0/2071573623 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5609542d0c90 con 0x560953fda390 2024-09-21T03:51:28.022 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:28.016+0000 7f61cffff640 1 --2- 172.21.15.39:0/2071573623 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x560953fda390 0x5609543ade00 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 crypto rx=0x7f61bc00b990 tx=0x7f61bc00be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cac7c0f3557dc161 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:51:28.022 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:28.016+0000 7f61affff640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-09-21T03:51:28.022 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:28.016+0000 7f61affff640 1 -- 172.21.15.39:0/2071573623 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61bc005020 con 0x560953fda390 2024-09-21T03:51:28.022 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:28.016+0000 7f61affff640 1 -- 172.21.15.39:0/2071573623 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8484+0+0 (secure 0 0 0) 0x7f61bc0051c0 con 0x560953fda390 2024-09-21T03:51:28.022 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:28.016+0000 7f61dd370740 1 -- 172.21.15.39:0/2071573623 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f61c8036030 con 0x560953fda390 2024-09-21T03:51:28.022 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:28.016+0000 7f61affff640 1 -- 172.21.15.39:0/2071573623 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f61bc0053c0 con 0x560953fda390 2024-09-21T03:51:28.022 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:28.016+0000 7f61dd370740 10 client.16938 fetch_fsmap learned FSMap version 29 2024-09-21T03:51:28.022 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:28.016+0000 7f61dd370740 1 -- 172.21.15.39:0/2071573623 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5609543bc570 con 0x560953fda390 2024-09-21T03:51:28.022 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:28.017+0000 7f61affff640 1 -- 172.21.15.39:0/2071573623 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f61c8036030 con 0x560953fda390 2024-09-21T03:51:28.022 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:28.017+0000 7f61affff640 1 -- 172.21.15.39:0/2071573623 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f61bc022020 con 0x560953fda390 2024-09-21T03:51:28.022 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:28.017+0000 7f61dd370740 10 client.16938 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T03:51:28.022 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:28.017+0000 7f61dd370740 10 client.16938 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-21T03:51:28.022 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:28.017+0000 7f61dd370740 20 client.16938 populate_metadata read hostname 'smithi039' 2024-09-21T03:51:28.022 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:28.017+0000 7f61dd370740 1 --2- 172.21.15.39:0/2071573623 >> [v2:172.21.15.161:6838/2190107270,v1:172.21.15.161:6839/2190107270] conn(0x560953fb99f0 0x5609543bf9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:51:28.022 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:28.017+0000 7f61dd370740 4 client.16938 mds_command: new command op to 24667 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:51:28.023 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:28.017+0000 7f61dd370740 1 -- 172.21.15.39:0/2071573623 --> [v2:172.21.15.161:6838/2190107270,v1:172.21.15.161:6839/2190107270] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56095431adf0 con 0x560953fb99f0 2024-09-21T03:51:28.023 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:28.017+0000 7f61d5399640 1 --2- 172.21.15.39:0/2071573623 >> [v2:172.21.15.161:6838/2190107270,v1:172.21.15.161:6839/2190107270] conn(0x560953fb99f0 0x5609543bf9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:51:28.024 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:28.018+0000 7f61affff640 10 client.16938 ms_handle_connect on v2:172.21.15.161:6838/2190107270 2024-09-21T03:51:28.024 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:28.018+0000 7f61d5399640 1 --2- 172.21.15.39:0/2071573623 >> [v2:172.21.15.161:6838/2190107270,v1:172.21.15.161:6839/2190107270] conn(0x560953fb99f0 0x5609543bf9f0 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=d22003de9f3f61d7 server_cookie=26eb8653c5d239db in_seq=0 out_seq=0 2024-09-21T03:51:28.024 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:28.018+0000 7f61affff640 1 -- 172.21.15.39:0/2071573623 <== mds.1 v2:172.21.15.161:6838/2190107270 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x56095431adf0 con 0x560953fb99f0 2024-09-21T03:51:28.024 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:28.018+0000 7f61affff640 10 client.16938 handle_command_reply: tid=0 2024-09-21T03:51:28.024 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:28.019+0000 7f61dd370740 1 client.16938 shutdown 2024-09-21T03:51:28.024 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:28.019+0000 7f61dd370740 1 -- 172.21.15.39:0/2071573623 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x560953fda390 msgr2=0x5609543ade00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:51:28.024 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:28.019+0000 7f61dd370740 1 --2- 172.21.15.39:0/2071573623 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x560953fda390 0x5609543ade00 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 crypto rx=0x7f61bc00b990 tx=0x7f61bc00be50 comp rx=0 tx=0).stop 2024-09-21T03:51:28.024 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:28.019+0000 7f61dd370740 1 -- 172.21.15.39:0/2071573623 shutdown_connections 2024-09-21T03:51:28.024 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:28.019+0000 7f61dd370740 1 --2- 172.21.15.39:0/2071573623 >> [v2:172.21.15.161:6838/2190107270,v1:172.21.15.161:6839/2190107270] conn(0x560953fb99f0 0x5609543bf9f0 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:28.024 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:28.019+0000 7f61dd370740 1 --2- 172.21.15.39:0/2071573623 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560953ffb070 0x5609543ae340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:28.024 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:28.019+0000 7f61affff640 0 client.16938 ms_handle_reset on v2:172.21.15.161:6838/2190107270 2024-09-21T03:51:28.025 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:28.019+0000 7f61dd370740 1 --2- 172.21.15.39:0/2071573623 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x560953fda390 0x5609543ade00 unknown :-1 s=CLOSED pgs=888 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:28.025 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:28.019+0000 7f61dd370740 1 --2- 172.21.15.39:0/2071573623 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560953fd21b0 0x5609543ad8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:28.025 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:28.019+0000 7f61dd370740 1 -- 172.21.15.39:0/2071573623 >> 172.21.15.39:0/2071573623 conn(0x560953fef460 msgr2=0x5609543a54a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:51:28.025 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:28.019+0000 7f61dd370740 1 -- 172.21.15.39:0/2071573623 shutdown_connections 2024-09-21T03:51:28.025 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:28.019+0000 7f61dd370740 1 -- 172.21.15.39:0/2071573623 wait complete. 2024-09-21T03:51:28.025 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:28.019+0000 7f61dd370740 20 client.16938 trim_cache size 0 max 16384 2024-09-21T03:51:28.035 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:28.032+0000 7f61dd370740 1 Processor -- start 2024-09-21T03:51:28.035 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:28.032+0000 7f61dd370740 1 -- start start 2024-09-21T03:51:28.035 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:28.032+0000 7f61dd370740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560953fb99f0 0x5609543af7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:51:28.035 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:28.032+0000 7f61dd370740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560953fd21b0 0x5609543a32f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:51:28.035 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:28.032+0000 7f61dd370740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x560953fda390 0x5609543b91b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:51:28.035 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:28.032+0000 7f61dd370740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f61d01b18d0 con 0x560953fda390 2024-09-21T03:51:28.035 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:28.032+0000 7f61dd370740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f61d01b3410 con 0x560953fb99f0 2024-09-21T03:51:28.035 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:28.032+0000 7f61dd370740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f61d01b1fa0 con 0x560953fd21b0 2024-09-21T03:51:28.036 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:28.033+0000 7f61d5399640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x560953fda390 0x5609543b91b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:51:28.036 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:28.033+0000 7f61d5399640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x560953fda390 0x5609543b91b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:40274/0 (socket says 172.21.15.39:40274) 2024-09-21T03:51:28.036 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:28.033+0000 7f61d5399640 1 -- 172.21.15.39:0/53741021 learned_addr learned my addr 172.21.15.39:0/53741021 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:51:28.036 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:28.033+0000 7f61d5399640 1 --2- 172.21.15.39:0/53741021 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560953fb99f0 0x5609543af7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:51:28.036 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:28.033+0000 7f61cffff640 1 --2- 172.21.15.39:0/53741021 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560953fd21b0 0x5609543a32f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:51:28.036 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:28.033+0000 7f61d5399640 1 -- 172.21.15.39:0/53741021 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560953fd21b0 msgr2=0x5609543a32f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:51:28.036 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:28.033+0000 7f61d5399640 1 --2- 172.21.15.39:0/53741021 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560953fd21b0 0x5609543a32f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:28.036 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:28.033+0000 7f61d5399640 1 -- 172.21.15.39:0/53741021 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560953fb99f0 msgr2=0x5609543af7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:51:28.036 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:28.033+0000 7f61d5399640 1 --2- 172.21.15.39:0/53741021 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560953fb99f0 0x5609543af7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:28.036 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:28.033+0000 7f61d5399640 1 -- 172.21.15.39:0/53741021 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5609543aeb80 con 0x560953fda390 2024-09-21T03:51:28.036 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:28.033+0000 7f61d5399640 1 --2- 172.21.15.39:0/53741021 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x560953fda390 0x5609543b91b0 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 crypto rx=0x7f61c8009bf0 tx=0x7f61c802a8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3655bd1f949084a3 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:51:28.036 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:28.033+0000 7f61ba7fc640 1 -- 172.21.15.39:0/53741021 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61c8060020 con 0x560953fda390 2024-09-21T03:51:28.036 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:28.033+0000 7f61ba7fc640 1 -- 172.21.15.39:0/53741021 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f61c8036050 con 0x560953fda390 2024-09-21T03:51:28.036 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:28.033+0000 7f61ba7fc640 1 -- 172.21.15.39:0/53741021 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61c804a040 con 0x560953fda390 2024-09-21T03:51:28.040 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:28.034+0000 7f61dd370740 1 -- 172.21.15.39:0/53741021 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x560953fda390 msgr2=0x5609543b91b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:51:28.040 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:28.034+0000 7f61dd370740 1 --2- 172.21.15.39:0/53741021 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x560953fda390 0x5609543b91b0 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 crypto rx=0x7f61c8009bf0 tx=0x7f61c802a8c0 comp rx=0 tx=0).stop 2024-09-21T03:51:28.040 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:28.034+0000 7f61dd370740 1 -- 172.21.15.39:0/53741021 shutdown_connections 2024-09-21T03:51:28.040 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:28.034+0000 7f61dd370740 1 --2- 172.21.15.39:0/53741021 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x560953fda390 0x5609543b91b0 unknown :-1 s=CLOSED pgs=889 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:28.040 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:28.034+0000 7f61dd370740 1 --2- 172.21.15.39:0/53741021 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560953fd21b0 0x5609543a32f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:28.041 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:28.034+0000 7f61dd370740 1 --2- 172.21.15.39:0/53741021 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560953fb99f0 0x5609543af7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:28.041 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:28.034+0000 7f61dd370740 1 -- 172.21.15.39:0/53741021 >> 172.21.15.39:0/53741021 conn(0x560953fef460 msgr2=0x5609543b8aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:51:28.041 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:28.034+0000 7f61dd370740 1 -- 172.21.15.39:0/53741021 shutdown_connections 2024-09-21T03:51:28.041 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:28.034+0000 7f61dd370740 1 -- 172.21.15.39:0/53741021 wait complete. 2024-09-21T03:51:28.041 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:28.035+0000 7f61dd370740 1 -- start start 2024-09-21T03:51:28.041 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:28.035+0000 7f61dd370740 1 Processor -- start 2024-09-21T03:51:28.041 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:28.035+0000 7f61dd370740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x560953fb99f0 0x5609543b1b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:51:28.041 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:28.035+0000 7f61dd370740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560953fd21b0 0x5609543b20a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:51:28.041 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:28.035+0000 7f61dd370740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560953fda390 0x5609543b25e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:51:28.041 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:28.035+0000 7f61dd370740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f61d01b2d40 con 0x560953fb99f0 2024-09-21T03:51:28.041 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:28.035+0000 7f61dd370740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f61d01b2670 con 0x560953fda390 2024-09-21T03:51:28.041 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:28.035+0000 7f61dd370740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f61d01af8b0 con 0x560953fd21b0 2024-09-21T03:51:28.041 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:28.035+0000 7f61dd370740 10 client.0 _ll_register_callbacks cb 0x7f61bc0042b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:51:28.041 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:28.035+0000 7f61d5399640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x560953fb99f0 0x5609543b1b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:51:28.041 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:28.035+0000 7f61d5399640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x560953fb99f0 0x5609543b1b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:40278/0 (socket says 172.21.15.39:40278) 2024-09-21T03:51:28.041 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:28.035+0000 7f61d5399640 1 -- 172.21.15.39:0/2097199040 learned_addr learned my addr 172.21.15.39:0/2097199040 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:51:28.042 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:28.035+0000 7f61cffff640 1 --2- 172.21.15.39:0/2097199040 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560953fd21b0 0x5609543b20a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:51:28.042 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:28.036+0000 7f61d5399640 1 --2- 172.21.15.39:0/2097199040 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560953fda390 0x5609543b25e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:51:28.042 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:28.036+0000 7f61d5399640 1 -- 172.21.15.39:0/2097199040 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560953fd21b0 msgr2=0x5609543b20a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:51:28.042 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:28.036+0000 7f61d5399640 1 --2- 172.21.15.39:0/2097199040 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560953fd21b0 0x5609543b20a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:28.042 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:28.036+0000 7f61d5399640 1 -- 172.21.15.39:0/2097199040 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560953fda390 msgr2=0x5609543b25e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:51:28.042 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:28.036+0000 7f61d5399640 1 --2- 172.21.15.39:0/2097199040 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560953fda390 0x5609543b25e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:28.042 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:28.036+0000 7f61d5399640 1 -- 172.21.15.39:0/2097199040 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560954429430 con 0x560953fb99f0 2024-09-21T03:51:28.042 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:28.036+0000 7f61d5399640 1 --2- 172.21.15.39:0/2097199040 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x560953fb99f0 0x5609543b1b60 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 crypto rx=0x7f61c8009ca0 tx=0x7f61c8003db0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aaad1f6ded231c0f server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:51:28.042 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:28.036+0000 7f61affff640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-09-21T03:51:28.042 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:28.036+0000 7f61affff640 1 -- 172.21.15.39:0/2097199040 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61c8060040 con 0x560953fb99f0 2024-09-21T03:51:28.042 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:28.036+0000 7f61affff640 1 -- 172.21.15.39:0/2097199040 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8484+0+0 (secure 0 0 0) 0x7f61c8003b30 con 0x560953fb99f0 2024-09-21T03:51:28.042 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:28.036+0000 7f61dd370740 1 -- 172.21.15.39:0/2097199040 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f61c8036050 con 0x560953fb99f0 2024-09-21T03:51:28.042 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:28.036+0000 7f61affff640 1 -- 172.21.15.39:0/2097199040 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f61c806d050 con 0x560953fb99f0 2024-09-21T03:51:28.042 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:28.036+0000 7f61dd370740 10 client.16944 fetch_fsmap learned FSMap version 29 2024-09-21T03:51:28.042 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:28.036+0000 7f61dd370740 1 -- 172.21.15.39:0/2097199040 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x56095442a6b0 con 0x560953fb99f0 2024-09-21T03:51:28.043 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:28.037+0000 7f61affff640 1 -- 172.21.15.39:0/2097199040 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f61c8036050 con 0x560953fb99f0 2024-09-21T03:51:28.043 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:28.037+0000 7f61affff640 1 -- 172.21.15.39:0/2097199040 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f61c8065040 con 0x560953fb99f0 2024-09-21T03:51:28.043 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:28.037+0000 7f61dd370740 10 client.16944 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T03:51:28.043 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:28.037+0000 7f61dd370740 10 client.16944 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-21T03:51:28.043 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:28.037+0000 7f61dd370740 20 client.16944 populate_metadata read hostname 'smithi039' 2024-09-21T03:51:28.043 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:28.037+0000 7f61dd370740 1 --2- 172.21.15.39:0/2097199040 >> [v2:172.21.15.161:6838/2190107270,v1:172.21.15.161:6839/2190107270] conn(0x560953ffb070 0x56095442ca00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:51:28.043 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:28.037+0000 7f61dd370740 4 client.16944 mds_command: new command op to 24667 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T03:51:28.043 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:28.037+0000 7f61dd370740 1 -- 172.21.15.39:0/2097199040 --> [v2:172.21.15.161:6838/2190107270,v1:172.21.15.161:6839/2190107270] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5609543168a0 con 0x560953ffb070 2024-09-21T03:51:28.043 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:28.037+0000 7f61cffff640 1 --2- 172.21.15.39:0/2097199040 >> [v2:172.21.15.161:6838/2190107270,v1:172.21.15.161:6839/2190107270] conn(0x560953ffb070 0x56095442ca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:51:28.043 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:28.037+0000 7f61cffff640 1 --2- 172.21.15.39:0/2097199040 >> [v2:172.21.15.161:6838/2190107270,v1:172.21.15.161:6839/2190107270] conn(0x560953ffb070 0x56095442ca00 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=a69bed4b7382f26a server_cookie=4fb5a1a5fc3a0458 in_seq=0 out_seq=0 2024-09-21T03:51:28.043 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:28.037+0000 7f61affff640 10 client.16944 ms_handle_connect on v2:172.21.15.161:6838/2190107270 2024-09-21T03:51:28.043 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:28.038+0000 7f61affff640 1 -- 172.21.15.39:0/2097199040 <== mds.1 v2:172.21.15.161:6838/2190107270 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5609543168a0 con 0x560953ffb070 2024-09-21T03:51:28.043 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:28.038+0000 7f61affff640 10 client.16944 handle_command_reply: tid=0 2024-09-21T03:51:28.043 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:28.038+0000 7f61dd370740 1 client.16944 shutdown 2024-09-21T03:51:28.043 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:28.038+0000 7f61dd370740 1 -- 172.21.15.39:0/2097199040 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x560953fb99f0 msgr2=0x5609543b1b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:51:28.043 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:28.038+0000 7f61dd370740 1 --2- 172.21.15.39:0/2097199040 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x560953fb99f0 0x5609543b1b60 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 crypto rx=0x7f61c8009ca0 tx=0x7f61c8003db0 comp rx=0 tx=0).stop 2024-09-21T03:51:28.044 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:28.038+0000 7f61dd370740 1 -- 172.21.15.39:0/2097199040 shutdown_connections 2024-09-21T03:51:28.044 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:28.038+0000 7f61dd370740 1 --2- 172.21.15.39:0/2097199040 >> [v2:172.21.15.161:6838/2190107270,v1:172.21.15.161:6839/2190107270] conn(0x560953ffb070 0x56095442ca00 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:28.044 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:28.038+0000 7f61dd370740 1 --2- 172.21.15.39:0/2097199040 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560953fda390 0x5609543b25e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:28.044 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:28.038+0000 7f61dd370740 1 --2- 172.21.15.39:0/2097199040 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560953fd21b0 0x5609543b20a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:28.044 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:28.038+0000 7f61dd370740 1 --2- 172.21.15.39:0/2097199040 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x560953fb99f0 0x5609543b1b60 unknown :-1 s=CLOSED pgs=890 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:28.044 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:28.038+0000 7f61affff640 0 client.16944 ms_handle_reset on v2:172.21.15.161:6838/2190107270 2024-09-21T03:51:28.044 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:28.038+0000 7f61dd370740 1 -- 172.21.15.39:0/2097199040 >> 172.21.15.39:0/2097199040 conn(0x560953fef460 msgr2=0x5609543b9460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:51:28.044 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:28.038+0000 7f61dd370740 1 -- 172.21.15.39:0/2097199040 shutdown_connections 2024-09-21T03:51:28.044 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:28.038+0000 7f61dd370740 1 -- 172.21.15.39:0/2097199040 wait complete. 2024-09-21T03:51:28.044 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:28.038+0000 7f61dd370740 20 client.16944 trim_cache size 0 max 16384 2024-09-21T03:51:28.044 INFO:teuthology.orchestra.run.smithi039.stdout:[] 2024-09-21T03:51:28.045 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:28.039+0000 7f61d6dfc640 1 -- 172.21.15.39:0/3173261245 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f61c00b6680 msgr2=0x7f61c00b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:51:28.045 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:28.039+0000 7f61d6dfc640 1 --2- 172.21.15.39:0/3173261245 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f61c00b6680 0x7f61c00b8b90 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f61c40e4bf0 tx=0x7f61c40e9040 comp rx=0 tx=0).stop 2024-09-21T03:51:28.045 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:28.039+0000 7f61d6dfc640 1 -- 172.21.15.39:0/3173261245 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f61d0102080 msgr2=0x7f61d0103020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:51:28.045 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:28.039+0000 7f61d6dfc640 1 --2- 172.21.15.39:0/3173261245 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f61d0102080 0x7f61d0103020 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 crypto rx=0x7f61c400c890 tx=0x7f61c400cd50 comp rx=0 tx=0).stop 2024-09-21T03:51:28.045 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:28.039+0000 7f61d6dfc640 1 -- 172.21.15.39:0/3173261245 shutdown_connections 2024-09-21T03:51:28.045 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:28.039+0000 7f61d6dfc640 1 --2- 172.21.15.39:0/3173261245 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f61c00b6680 0x7f61c00b8b90 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:28.046 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:28.039+0000 7f61d6dfc640 1 --2- 172.21.15.39:0/3173261245 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f61d0109eb0 0x7f61d0103aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:28.046 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:28.039+0000 7f61d6dfc640 1 --2- 172.21.15.39:0/3173261245 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f61d0109560 0x7f61d0103560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:28.046 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:28.039+0000 7f61d6dfc640 1 --2- 172.21.15.39:0/3173261245 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f61d0102080 0x7f61d0103020 unknown :-1 s=CLOSED pgs=886 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:28.046 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:28.039+0000 7f61d6dfc640 1 -- 172.21.15.39:0/3173261245 >> 172.21.15.39:0/3173261245 conn(0x7f61d0079ec0 msgr2=0x7f61d0107550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:51:28.046 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:28.039+0000 7f61d6dfc640 1 -- 172.21.15.39:0/3173261245 shutdown_connections 2024-09-21T03:51:28.046 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:28.039+0000 7f61d6dfc640 1 -- 172.21.15.39:0/3173261245 wait complete. 2024-09-21T03:51:28.859 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 51116d5c-77ca-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 damage ls 2024-09-21T03:51:29.128 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/51116d5c-77ca-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:51:29.447 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:51:29 smithi039 ceph-mon[31211]: pgmap v488: 97 pgs: 97 active+clean; 10 MiB data, 520 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 8 op/s 2024-09-21T03:51:29.447 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:51:29 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:51:29.447 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:51:29 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:51:29.447 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:51:29 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:51:29.447 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:51:29 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:51:29.468 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:51:29 smithi161 ceph-mon[34533]: pgmap v488: 97 pgs: 97 active+clean; 10 MiB data, 520 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 8 op/s 2024-09-21T03:51:29.468 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:51:29 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:51:29.468 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:51:29 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:51:29.468 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:51:29 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:51:29.468 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:51:29 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:51:29.475 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:51:29 smithi150 ceph-mon[34289]: pgmap v488: 97 pgs: 97 active+clean; 10 MiB data, 520 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 8 op/s 2024-09-21T03:51:29.475 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:51:29 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:51:29.475 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:51:29 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:51:29.475 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:51:29 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:51:29.475 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:51:29 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:51:29.687 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:29.678+0000 7f898f1eb640 1 -- 172.21.15.39:0/2366791509 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8988104840 msgr2=0x7f8988104c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:51:29.687 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:29.678+0000 7f898f1eb640 1 --2- 172.21.15.39:0/2366791509 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8988104840 0x7f8988104c50 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 crypto rx=0x7f8980009900 tx=0x7f898002a410 comp rx=0 tx=0).stop 2024-09-21T03:51:29.687 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:29.678+0000 7f898f1eb640 1 -- 172.21.15.39:0/2366791509 shutdown_connections 2024-09-21T03:51:29.687 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:29.678+0000 7f898f1eb640 1 --2- 172.21.15.39:0/2366791509 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f898810ddf0 0x7f8988110300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:29.687 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:29.678+0000 7f898f1eb640 1 --2- 172.21.15.39:0/2366791509 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8988105270 0x7f898810d8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:29.687 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:29.678+0000 7f898f1eb640 1 --2- 172.21.15.39:0/2366791509 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8988104840 0x7f8988104c50 unknown :-1 s=CLOSED pgs=891 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:29.688 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:29.678+0000 7f898f1eb640 1 -- 172.21.15.39:0/2366791509 >> 172.21.15.39:0/2366791509 conn(0x7f8988079730 msgr2=0x7f8988079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:51:29.688 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:29.678+0000 7f898f1eb640 1 -- 172.21.15.39:0/2366791509 shutdown_connections 2024-09-21T03:51:29.688 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:29.678+0000 7f898f1eb640 1 -- 172.21.15.39:0/2366791509 wait complete. 2024-09-21T03:51:29.688 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:29.679+0000 7f898f1eb640 1 Processor -- start 2024-09-21T03:51:29.688 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:29.679+0000 7f898f1eb640 1 -- start start 2024-09-21T03:51:29.688 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:29.679+0000 7f898f1eb640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8988104840 0x7f898819a5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:51:29.688 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:29.679+0000 7f898f1eb640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8988105270 0x7f898819ab40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:51:29.688 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:29.679+0000 7f898f1eb640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f898810ddf0 0x7f898819fde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:51:29.688 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:29.679+0000 7f898f1eb640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f8988113a70 con 0x7f898810ddf0 2024-09-21T03:51:29.688 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:29.679+0000 7f898f1eb640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f89881138f0 con 0x7f8988105270 2024-09-21T03:51:29.688 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:29.679+0000 7f898f1eb640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f8988113bf0 con 0x7f8988104840 2024-09-21T03:51:29.688 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:29.679+0000 7f898d788640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f898810ddf0 0x7f898819fde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:51:29.688 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:29.679+0000 7f898d788640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f898810ddf0 0x7f898819fde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:40292/0 (socket says 172.21.15.39:40292) 2024-09-21T03:51:29.688 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:29.679+0000 7f898d788640 1 -- 172.21.15.39:0/1511812597 learned_addr learned my addr 172.21.15.39:0/1511812597 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:51:29.689 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:29.680+0000 7f898cf87640 1 --2- 172.21.15.39:0/1511812597 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8988104840 0x7f898819a5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:51:29.689 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:29.680+0000 7f8987fff640 1 --2- 172.21.15.39:0/1511812597 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8988105270 0x7f898819ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:51:29.689 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:29.680+0000 7f898d788640 1 -- 172.21.15.39:0/1511812597 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8988104840 msgr2=0x7f898819a5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:51:29.689 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:29.680+0000 7f898d788640 1 --2- 172.21.15.39:0/1511812597 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8988104840 0x7f898819a5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:29.689 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:29.680+0000 7f898d788640 1 -- 172.21.15.39:0/1511812597 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8988105270 msgr2=0x7f898819ab40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:51:29.689 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:29.680+0000 7f898d788640 1 --2- 172.21.15.39:0/1511812597 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8988105270 0x7f898819ab40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:29.689 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:29.680+0000 7f898d788640 1 -- 172.21.15.39:0/1511812597 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f89881a0510 con 0x7f898810ddf0 2024-09-21T03:51:29.689 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:29.680+0000 7f8987fff640 1 --2- 172.21.15.39:0/1511812597 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8988105270 0x7f898819ab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:51:29.689 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:29.680+0000 7f898d788640 1 --2- 172.21.15.39:0/1511812597 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f898810ddf0 0x7f898819fde0 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 crypto rx=0x7f897800eaa0 tx=0x7f897800ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:51:29.689 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:29.680+0000 7f8985ffb640 1 -- 172.21.15.39:0/1511812597 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f897800cdb0 con 0x7f898810ddf0 2024-09-21T03:51:29.689 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:29.680+0000 7f898f1eb640 1 -- 172.21.15.39:0/1511812597 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f89881a0800 con 0x7f898810ddf0 2024-09-21T03:51:29.689 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:29.680+0000 7f8985ffb640 1 -- 172.21.15.39:0/1511812597 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8978005020 con 0x7f898810ddf0 2024-09-21T03:51:29.689 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:29.680+0000 7f8985ffb640 1 -- 172.21.15.39:0/1511812597 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89780116d0 con 0x7f898810ddf0 2024-09-21T03:51:29.690 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:29.680+0000 7f898f1eb640 1 -- 172.21.15.39:0/1511812597 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f89881a6f60 con 0x7f898810ddf0 2024-09-21T03:51:29.690 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:29.683+0000 7f899575f740 1 Processor -- start 2024-09-21T03:51:29.690 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:29.683+0000 7f8985ffb640 1 -- 172.21.15.39:0/1511812597 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 27) ==== 154643+0+0 (secure 0 0 0) 0x7f89780051d0 con 0x7f898810ddf0 2024-09-21T03:51:29.690 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:29.683+0000 7f899575f740 1 -- start start 2024-09-21T03:51:29.690 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:29.684+0000 7f899575f740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560840a9a070 0x560840e412c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:51:29.690 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:29.684+0000 7f899575f740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x560840a711b0 0x560840e41800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:51:29.690 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:29.684+0000 7f899575f740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560840a79390 0x560840e41d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:51:29.690 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:29.684+0000 7f899575f740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x560840db7ea0 con 0x560840a711b0 2024-09-21T03:51:29.690 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:29.684+0000 7f899575f740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x560840da72d0 con 0x560840a79390 2024-09-21T03:51:29.690 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:29.684+0000 7f899575f740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x560840dab0e0 con 0x560840a9a070 2024-09-21T03:51:29.690 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:29.684+0000 7f898cf87640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x560840a711b0 0x560840e41800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:51:29.690 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:29.684+0000 7f898cf87640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x560840a711b0 0x560840e41800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:40306/0 (socket says 172.21.15.39:40306) 2024-09-21T03:51:29.690 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:29.684+0000 7f898cf87640 1 -- 172.21.15.39:0/1966151332 learned_addr learned my addr 172.21.15.39:0/1966151332 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:51:29.691 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:29.684+0000 7f8987fff640 1 --2- 172.21.15.39:0/1966151332 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560840a79390 0x560840e41d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:51:29.691 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:29.684+0000 7f8987fff640 1 --2- 172.21.15.39:0/1966151332 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560840a9a070 0x560840e412c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:51:29.691 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:29.684+0000 7f8985ffb640 1 --2- 172.21.15.39:0/1511812597 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f89740b6680 0x7f89740b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:51:29.691 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:29.684+0000 7f898cf87640 1 -- 172.21.15.39:0/1966151332 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560840a9a070 msgr2=0x560840e412c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:51:29.691 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:29.684+0000 7f898cf87640 1 --2- 172.21.15.39:0/1966151332 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560840a9a070 0x560840e412c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:29.691 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:29.684+0000 7f898cf87640 1 -- 172.21.15.39:0/1966151332 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560840a79390 msgr2=0x560840e41d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:51:29.691 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:29.684+0000 7f898cf87640 1 --2- 172.21.15.39:0/1966151332 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560840a79390 0x560840e41d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:29.691 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:29.684+0000 7f898cf87640 1 -- 172.21.15.39:0/1966151332 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560840b05650 con 0x560840a711b0 2024-09-21T03:51:29.691 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:29.684+0000 7f8987fff640 1 --2- 172.21.15.39:0/1966151332 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560840a9a070 0x560840e412c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:51:29.691 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:29.684+0000 7f898d788640 1 --2- 172.21.15.39:0/1511812597 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f89740b6680 0x7f89740b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:51:29.691 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:29.685+0000 7f898cf87640 1 --2- 172.21.15.39:0/1966151332 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x560840a711b0 0x560840e41800 secure :-1 s=READY pgs=893 cs=0 l=1 rev1=1 crypto rx=0x7f898806bc00 tx=0x7f8980032df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce3fcf51458bc476 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:51:29.691 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:29.685+0000 7f8985ffb640 1 -- 172.21.15.39:0/1511812597 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8484+0+0 (secure 0 0 0) 0x7f89780e7880 con 0x7f898810ddf0 2024-09-21T03:51:29.692 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:29.685+0000 7f897effd640 1 -- 172.21.15.39:0/1966151332 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8980040070 con 0x560840a711b0 2024-09-21T03:51:29.692 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:29.685+0000 7f897effd640 1 -- 172.21.15.39:0/1966151332 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8980005db0 con 0x560840a711b0 2024-09-21T03:51:29.692 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:29.685+0000 7f897effd640 1 -- 172.21.15.39:0/1966151332 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8980035040 con 0x560840a711b0 2024-09-21T03:51:29.695 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:29.686+0000 7f899575f740 1 -- 172.21.15.39:0/1966151332 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x560840a711b0 msgr2=0x560840e41800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:51:29.695 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:29.686+0000 7f899575f740 1 --2- 172.21.15.39:0/1966151332 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x560840a711b0 0x560840e41800 secure :-1 s=READY pgs=893 cs=0 l=1 rev1=1 crypto rx=0x7f898806bc00 tx=0x7f8980032df0 comp rx=0 tx=0).stop 2024-09-21T03:51:29.695 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:29.686+0000 7f898d788640 1 --2- 172.21.15.39:0/1511812597 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f89740b6680 0x7f89740b8b90 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f897800cfd0 tx=0x7f8978076210 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:51:29.695 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:29.686+0000 7f899575f740 1 -- 172.21.15.39:0/1966151332 shutdown_connections 2024-09-21T03:51:29.696 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:29.686+0000 7f899575f740 1 --2- 172.21.15.39:0/1966151332 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560840a79390 0x560840e41d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:29.696 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:29.686+0000 7f899575f740 1 --2- 172.21.15.39:0/1966151332 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x560840a711b0 0x560840e41800 unknown :-1 s=CLOSED pgs=893 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:29.696 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:29.686+0000 7f899575f740 1 --2- 172.21.15.39:0/1966151332 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560840a9a070 0x560840e412c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:29.696 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:29.686+0000 7f899575f740 1 -- 172.21.15.39:0/1966151332 >> 172.21.15.39:0/1966151332 conn(0x560840a8e460 msgr2=0x560840e3f1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:51:29.696 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:29.686+0000 7f899575f740 1 -- 172.21.15.39:0/1966151332 shutdown_connections 2024-09-21T03:51:29.696 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:29.686+0000 7f899575f740 1 -- 172.21.15.39:0/1966151332 wait complete. 2024-09-21T03:51:29.696 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:29.686+0000 7f899575f740 1 -- start start 2024-09-21T03:51:29.696 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:29.686+0000 7f899575f740 1 Processor -- start 2024-09-21T03:51:29.696 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:29.687+0000 7f899575f740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560840a711b0 0x560840e4c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:51:29.696 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:29.687+0000 7f899575f740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x560840a79390 0x560840e4cf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:51:29.696 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:29.687+0000 7f899575f740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560840a9a070 0x560840e58d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:51:29.696 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:29.687+0000 7f899575f740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x560840c08700 con 0x560840a79390 2024-09-21T03:51:29.696 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:29.687+0000 7f899575f740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x560840d42160 con 0x560840a711b0 2024-09-21T03:51:29.696 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:29.687+0000 7f899575f740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x560840e4fa40 con 0x560840a9a070 2024-09-21T03:51:29.697 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:29.687+0000 7f899575f740 10 client.0 _ll_register_callbacks cb 0x560840d5b9e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:51:29.697 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:29.687+0000 7f898cf87640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x560840a79390 0x560840e4cf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:51:29.697 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:29.687+0000 7f898cf87640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x560840a79390 0x560840e4cf10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:40320/0 (socket says 172.21.15.39:40320) 2024-09-21T03:51:29.697 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:29.687+0000 7f898cf87640 1 -- 172.21.15.39:0/3077854703 learned_addr learned my addr 172.21.15.39:0/3077854703 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:51:29.697 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:29.687+0000 7f8987fff640 1 --2- 172.21.15.39:0/3077854703 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560840a9a070 0x560840e58d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:51:29.697 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:29.687+0000 7f8987fff640 1 --2- 172.21.15.39:0/3077854703 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560840a711b0 0x560840e4c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:51:29.697 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:29.688+0000 7f898cf87640 1 -- 172.21.15.39:0/3077854703 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560840a9a070 msgr2=0x560840e58d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:51:29.697 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:29.688+0000 7f898cf87640 1 --2- 172.21.15.39:0/3077854703 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560840a9a070 0x560840e58d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:29.697 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:29.688+0000 7f898cf87640 1 -- 172.21.15.39:0/3077854703 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560840a711b0 msgr2=0x560840e4c9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:51:29.697 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:29.688+0000 7f898cf87640 1 --2- 172.21.15.39:0/3077854703 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560840a711b0 0x560840e4c9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:29.697 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:29.688+0000 7f898cf87640 1 -- 172.21.15.39:0/3077854703 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560840d6fc90 con 0x560840a79390 2024-09-21T03:51:29.697 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:29.688+0000 7f898cf87640 1 --2- 172.21.15.39:0/3077854703 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x560840a79390 0x560840e4cf10 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 crypto rx=0x7f898002ab90 tx=0x7f89800051c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6996380c641c4daa server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:51:29.697 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:29.688+0000 7f8967fff640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-09-21T03:51:29.697 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:29.688+0000 7f8967fff640 1 -- 172.21.15.39:0/3077854703 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8980035070 con 0x560840a79390 2024-09-21T03:51:29.698 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:29.688+0000 7f8967fff640 1 -- 172.21.15.39:0/3077854703 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8484+0+0 (secure 0 0 0) 0x7f8980004030 con 0x560840a79390 2024-09-21T03:51:29.698 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:29.688+0000 7f899575f740 1 -- 172.21.15.39:0/3077854703 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8980005db0 con 0x560840a79390 2024-09-21T03:51:29.698 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:29.688+0000 7f8967fff640 1 -- 172.21.15.39:0/3077854703 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f8980040040 con 0x560840a79390 2024-09-21T03:51:29.698 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:29.689+0000 7f899575f740 10 client.16956 fetch_fsmap learned FSMap version 29 2024-09-21T03:51:29.698 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:29.689+0000 7f899575f740 1 -- 172.21.15.39:0/3077854703 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x560840e5b7b0 con 0x560840a79390 2024-09-21T03:51:29.698 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:29.689+0000 7f8967fff640 1 -- 172.21.15.39:0/3077854703 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8980005db0 con 0x560840a79390 2024-09-21T03:51:29.698 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:29.689+0000 7f8967fff640 1 -- 172.21.15.39:0/3077854703 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f8980059020 con 0x560840a79390 2024-09-21T03:51:29.698 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:29.689+0000 7f899575f740 10 client.16956 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T03:51:29.698 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:29.689+0000 7f899575f740 10 client.16956 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T03:51:29.698 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:29.689+0000 7f899575f740 20 client.16956 populate_metadata read hostname 'smithi039' 2024-09-21T03:51:29.698 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:29.689+0000 7f899575f740 1 --2- 172.21.15.39:0/3077854703 >> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] conn(0x560840a589f0 0x560840e5eb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:51:29.698 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:29.689+0000 7f899575f740 4 client.16956 mds_command: new command op to 24695 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:51:29.698 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:29.689+0000 7f899575f740 1 -- 172.21.15.39:0/3077854703 --> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560840db9df0 con 0x560840a589f0 2024-09-21T03:51:29.698 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:29.689+0000 7f8987fff640 1 --2- 172.21.15.39:0/3077854703 >> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] conn(0x560840a589f0 0x560840e5eb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:51:29.699 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:29.690+0000 7f8987fff640 1 --2- 172.21.15.39:0/3077854703 >> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] conn(0x560840a589f0 0x560840e5eb10 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=94cb3e24038ec693 server_cookie=8021ac97c50fb944 in_seq=0 out_seq=0 2024-09-21T03:51:29.699 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:29.690+0000 7f8967fff640 10 client.16956 ms_handle_connect on v2:172.21.15.150:6838/2949386907 2024-09-21T03:51:29.699 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:29.691+0000 7f8967fff640 1 -- 172.21.15.39:0/3077854703 <== mds.0 v2:172.21.15.150:6838/2949386907 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x560840db9df0 con 0x560840a589f0 2024-09-21T03:51:29.699 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:29.691+0000 7f8967fff640 10 client.16956 handle_command_reply: tid=0 2024-09-21T03:51:29.699 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:29.691+0000 7f899575f740 1 client.16956 shutdown 2024-09-21T03:51:29.699 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:29.691+0000 7f899575f740 1 -- 172.21.15.39:0/3077854703 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x560840a79390 msgr2=0x560840e4cf10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:51:29.699 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:29.691+0000 7f899575f740 1 --2- 172.21.15.39:0/3077854703 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x560840a79390 0x560840e4cf10 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 crypto rx=0x7f898002ab90 tx=0x7f89800051c0 comp rx=0 tx=0).stop 2024-09-21T03:51:29.699 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:29.691+0000 7f899575f740 1 -- 172.21.15.39:0/3077854703 shutdown_connections 2024-09-21T03:51:29.699 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:29.691+0000 7f899575f740 1 --2- 172.21.15.39:0/3077854703 >> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] conn(0x560840a589f0 0x560840e5eb10 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:29.699 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:29.691+0000 7f899575f740 1 --2- 172.21.15.39:0/3077854703 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560840a9a070 0x560840e58d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:29.699 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:29.691+0000 7f899575f740 1 --2- 172.21.15.39:0/3077854703 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x560840a79390 0x560840e4cf10 unknown :-1 s=CLOSED pgs=894 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:29.699 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:29.691+0000 7f899575f740 1 --2- 172.21.15.39:0/3077854703 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560840a711b0 0x560840e4c9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:29.699 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:29.691+0000 7f899575f740 1 -- 172.21.15.39:0/3077854703 >> 172.21.15.39:0/3077854703 conn(0x560840a8e460 msgr2=0x560840e44450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:51:29.699 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:29.691+0000 7f8967fff640 0 client.16956 ms_handle_reset on v2:172.21.15.150:6838/2949386907 2024-09-21T03:51:29.700 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:29.691+0000 7f899575f740 1 -- 172.21.15.39:0/3077854703 shutdown_connections 2024-09-21T03:51:29.700 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:29.691+0000 7f899575f740 1 -- 172.21.15.39:0/3077854703 wait complete. 2024-09-21T03:51:29.700 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:29.691+0000 7f899575f740 20 client.16956 trim_cache size 0 max 16384 2024-09-21T03:51:29.707 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:29.704+0000 7f899575f740 1 Processor -- start 2024-09-21T03:51:29.707 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:29.704+0000 7f899575f740 1 -- start start 2024-09-21T03:51:29.708 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:29.705+0000 7f899575f740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560840a711b0 0x560840e4ce90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:51:29.708 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:29.705+0000 7f899575f740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x560840a79390 0x560840e443c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:51:29.708 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:29.705+0000 7f899575f740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560840a9a070 0x560840e44900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:51:29.708 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:29.705+0000 7f899575f740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f89881ad4e0 con 0x560840a79390 2024-09-21T03:51:29.708 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:29.705+0000 7f899575f740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f89881af020 con 0x560840a711b0 2024-09-21T03:51:29.708 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:29.705+0000 7f899575f740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f89881adbb0 con 0x560840a9a070 2024-09-21T03:51:29.708 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:29.705+0000 7f898cf87640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x560840a79390 0x560840e443c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:51:29.708 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:29.705+0000 7f898cf87640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x560840a79390 0x560840e443c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:40324/0 (socket says 172.21.15.39:40324) 2024-09-21T03:51:29.708 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:29.705+0000 7f898cf87640 1 -- 172.21.15.39:0/420334433 learned_addr learned my addr 172.21.15.39:0/420334433 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:51:29.708 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:29.705+0000 7f8987fff640 1 --2- 172.21.15.39:0/420334433 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560840a9a070 0x560840e44900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:51:29.708 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:29.705+0000 7f898cf87640 1 -- 172.21.15.39:0/420334433 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560840a9a070 msgr2=0x560840e44900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:51:29.709 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:29.705+0000 7f898cf87640 1 --2- 172.21.15.39:0/420334433 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560840a9a070 0x560840e44900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:29.709 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:29.705+0000 7f898cf87640 1 -- 172.21.15.39:0/420334433 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560840a711b0 msgr2=0x560840e4ce90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:51:29.709 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:29.705+0000 7f898cf87640 1 --2- 172.21.15.39:0/420334433 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560840a711b0 0x560840e4ce90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:29.709 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:29.705+0000 7f898cf87640 1 -- 172.21.15.39:0/420334433 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560840e4dc90 con 0x560840a79390 2024-09-21T03:51:29.709 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:29.705+0000 7f8987fff640 1 --2- 172.21.15.39:0/420334433 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560840a9a070 0x560840e44900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:51:29.709 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:29.705+0000 7f898cf87640 1 --2- 172.21.15.39:0/420334433 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x560840a79390 0x560840e443c0 secure :-1 s=READY pgs=895 cs=0 l=1 rev1=1 crypto rx=0x7f898002a930 tx=0x7f8980031400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f49115776ff0280b server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:51:29.709 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:29.706+0000 7f897e7fc640 1 -- 172.21.15.39:0/420334433 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8980035020 con 0x560840a79390 2024-09-21T03:51:29.709 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:29.706+0000 7f897e7fc640 1 -- 172.21.15.39:0/420334433 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f898004d030 con 0x560840a79390 2024-09-21T03:51:29.709 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:29.706+0000 7f897e7fc640 1 -- 172.21.15.39:0/420334433 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8980004260 con 0x560840a79390 2024-09-21T03:51:29.711 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:29.707+0000 7f899575f740 1 -- 172.21.15.39:0/420334433 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x560840a79390 msgr2=0x560840e443c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:51:29.712 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:29.707+0000 7f899575f740 1 --2- 172.21.15.39:0/420334433 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x560840a79390 0x560840e443c0 secure :-1 s=READY pgs=895 cs=0 l=1 rev1=1 crypto rx=0x7f898002a930 tx=0x7f8980031400 comp rx=0 tx=0).stop 2024-09-21T03:51:29.712 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:29.707+0000 7f899575f740 1 -- 172.21.15.39:0/420334433 shutdown_connections 2024-09-21T03:51:29.712 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:29.707+0000 7f899575f740 1 --2- 172.21.15.39:0/420334433 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560840a9a070 0x560840e44900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:29.712 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:29.707+0000 7f899575f740 1 --2- 172.21.15.39:0/420334433 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x560840a79390 0x560840e443c0 unknown :-1 s=CLOSED pgs=895 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:29.712 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:29.707+0000 7f899575f740 1 --2- 172.21.15.39:0/420334433 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560840a711b0 0x560840e4ce90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:29.712 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:29.707+0000 7f899575f740 1 -- 172.21.15.39:0/420334433 >> 172.21.15.39:0/420334433 conn(0x560840a8e460 msgr2=0x560840e4c780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:51:29.712 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:29.707+0000 7f899575f740 1 -- 172.21.15.39:0/420334433 shutdown_connections 2024-09-21T03:51:29.712 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:29.707+0000 7f899575f740 1 -- 172.21.15.39:0/420334433 wait complete. 2024-09-21T03:51:29.712 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:29.707+0000 7f899575f740 1 -- start start 2024-09-21T03:51:29.712 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:29.707+0000 7f899575f740 1 Processor -- start 2024-09-21T03:51:29.712 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:29.708+0000 7f899575f740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560840a711b0 0x560840e5fac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:51:29.712 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:29.708+0000 7f899575f740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x560840a79390 0x560840e60000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:51:29.713 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:29.708+0000 7f899575f740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560840a9a070 0x560840e60540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:51:29.713 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:29.708+0000 7f899575f740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f89881ae950 con 0x560840a79390 2024-09-21T03:51:29.713 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:29.708+0000 7f899575f740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f89881ae280 con 0x560840a711b0 2024-09-21T03:51:29.713 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:29.708+0000 7f899575f740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f89881ab4e0 con 0x560840a9a070 2024-09-21T03:51:29.713 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:29.708+0000 7f899575f740 10 client.0 _ll_register_callbacks cb 0x7f897000ba20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:51:29.713 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:29.708+0000 7f898cf87640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x560840a79390 0x560840e60000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:51:29.713 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:29.708+0000 7f898cf87640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x560840a79390 0x560840e60000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:40340/0 (socket says 172.21.15.39:40340) 2024-09-21T03:51:29.713 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:29.708+0000 7f898cf87640 1 -- 172.21.15.39:0/2406423760 learned_addr learned my addr 172.21.15.39:0/2406423760 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:51:29.713 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:29.708+0000 7f8987fff640 1 --2- 172.21.15.39:0/2406423760 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560840a711b0 0x560840e5fac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:51:29.713 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:29.708+0000 7f8987fff640 1 --2- 172.21.15.39:0/2406423760 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560840a9a070 0x560840e60540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:51:29.713 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:29.708+0000 7f898cf87640 1 -- 172.21.15.39:0/2406423760 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560840a9a070 msgr2=0x560840e60540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:51:29.713 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:29.708+0000 7f898cf87640 1 --2- 172.21.15.39:0/2406423760 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560840a9a070 0x560840e60540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:29.713 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:29.708+0000 7f898cf87640 1 -- 172.21.15.39:0/2406423760 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560840a711b0 msgr2=0x560840e5fac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:51:29.713 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:29.708+0000 7f898cf87640 1 --2- 172.21.15.39:0/2406423760 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560840a711b0 0x560840e5fac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:29.714 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:29.708+0000 7f898cf87640 1 -- 172.21.15.39:0/2406423760 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560840ec8290 con 0x560840a79390 2024-09-21T03:51:29.714 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:29.708+0000 7f8987fff640 1 --2- 172.21.15.39:0/2406423760 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560840a9a070 0x560840e60540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:51:29.714 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:29.709+0000 7f898cf87640 1 --2- 172.21.15.39:0/2406423760 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x560840a79390 0x560840e60000 secure :-1 s=READY pgs=896 cs=0 l=1 rev1=1 crypto rx=0x7f898002a450 tx=0x7f8980005950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d7a5aea99e48a72f server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:51:29.714 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:29.709+0000 7f8967fff640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-09-21T03:51:29.714 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:29.709+0000 7f8967fff640 1 -- 172.21.15.39:0/2406423760 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8980035050 con 0x560840a79390 2024-09-21T03:51:29.714 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:29.709+0000 7f8967fff640 1 -- 172.21.15.39:0/2406423760 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8484+0+0 (secure 0 0 0) 0x7f8980004210 con 0x560840a79390 2024-09-21T03:51:29.714 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:29.709+0000 7f899575f740 1 -- 172.21.15.39:0/2406423760 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f898004d030 con 0x560840a79390 2024-09-21T03:51:29.714 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:29.709+0000 7f8967fff640 1 -- 172.21.15.39:0/2406423760 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f898006d050 con 0x560840a79390 2024-09-21T03:51:29.714 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:29.709+0000 7f899575f740 10 client.16962 fetch_fsmap learned FSMap version 29 2024-09-21T03:51:29.714 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:29.709+0000 7f899575f740 1 -- 172.21.15.39:0/2406423760 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x560840ec93b0 con 0x560840a79390 2024-09-21T03:51:29.715 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:29.709+0000 7f8967fff640 1 -- 172.21.15.39:0/2406423760 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f898004d030 con 0x560840a79390 2024-09-21T03:51:29.715 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:29.710+0000 7f8967fff640 1 -- 172.21.15.39:0/2406423760 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f8980059070 con 0x560840a79390 2024-09-21T03:51:29.715 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:29.710+0000 7f899575f740 10 client.16962 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T03:51:29.715 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:29.710+0000 7f899575f740 10 client.16962 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T03:51:29.715 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:29.710+0000 7f899575f740 20 client.16962 populate_metadata read hostname 'smithi039' 2024-09-21T03:51:29.715 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:29.710+0000 7f899575f740 1 --2- 172.21.15.39:0/2406423760 >> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] conn(0x560840a2dda0 0x560840ecb7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:51:29.715 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:29.710+0000 7f899575f740 4 client.16962 mds_command: new command op to 24695 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T03:51:29.715 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:29.710+0000 7f899575f740 1 -- 172.21.15.39:0/2406423760 --> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] -- command(tid 0: {"prefix": "damage ls"}) -- 0x560840db58a0 con 0x560840a2dda0 2024-09-21T03:51:29.715 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:29.710+0000 7f8987fff640 1 --2- 172.21.15.39:0/2406423760 >> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] conn(0x560840a2dda0 0x560840ecb7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:51:29.716 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:29.711+0000 7f8967fff640 10 client.16962 ms_handle_connect on v2:172.21.15.150:6838/2949386907 2024-09-21T03:51:29.717 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:29.711+0000 7f8987fff640 1 --2- 172.21.15.39:0/2406423760 >> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] conn(0x560840a2dda0 0x560840ecb7a0 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=39379f4d0099313f server_cookie=20219202776bded5 in_seq=0 out_seq=0 2024-09-21T03:51:29.717 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:29.711+0000 7f8967fff640 1 -- 172.21.15.39:0/2406423760 <== mds.0 v2:172.21.15.150:6838/2949386907 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x560840db58a0 con 0x560840a2dda0 2024-09-21T03:51:29.717 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:29.711+0000 7f8967fff640 10 client.16962 handle_command_reply: tid=0 2024-09-21T03:51:29.717 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:29.711+0000 7f899575f740 1 client.16962 shutdown 2024-09-21T03:51:29.717 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:29.711+0000 7f899575f740 1 -- 172.21.15.39:0/2406423760 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x560840a79390 msgr2=0x560840e60000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:51:29.717 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:29.711+0000 7f899575f740 1 --2- 172.21.15.39:0/2406423760 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x560840a79390 0x560840e60000 secure :-1 s=READY pgs=896 cs=0 l=1 rev1=1 crypto rx=0x7f898002a450 tx=0x7f8980005950 comp rx=0 tx=0).stop 2024-09-21T03:51:29.717 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:29.711+0000 7f899575f740 1 -- 172.21.15.39:0/2406423760 shutdown_connections 2024-09-21T03:51:29.717 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:29.711+0000 7f899575f740 1 --2- 172.21.15.39:0/2406423760 >> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] conn(0x560840a2dda0 0x560840ecb7a0 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:29.717 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:29.711+0000 7f899575f740 1 --2- 172.21.15.39:0/2406423760 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560840a9a070 0x560840e60540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:29.717 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:29.711+0000 7f899575f740 1 --2- 172.21.15.39:0/2406423760 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x560840a79390 0x560840e60000 unknown :-1 s=CLOSED pgs=896 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:29.717 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:29.711+0000 7f899575f740 1 --2- 172.21.15.39:0/2406423760 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560840a711b0 0x560840e5fac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:29.717 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:29.711+0000 7f8967fff640 0 client.16962 ms_handle_reset on v2:172.21.15.150:6838/2949386907 2024-09-21T03:51:29.717 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:29.711+0000 7f899575f740 1 -- 172.21.15.39:0/2406423760 >> 172.21.15.39:0/2406423760 conn(0x560840a8e460 msgr2=0x560840e422f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:51:29.717 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:29.711+0000 7f899575f740 1 -- 172.21.15.39:0/2406423760 shutdown_connections 2024-09-21T03:51:29.718 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:29.712+0000 7f899575f740 1 -- 172.21.15.39:0/2406423760 wait complete. 2024-09-21T03:51:29.718 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:29.712+0000 7f899575f740 20 client.16962 trim_cache size 0 max 16384 2024-09-21T03:51:29.718 INFO:teuthology.orchestra.run.smithi039.stdout:[] 2024-09-21T03:51:29.718 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:29.712+0000 7f898f1eb640 1 -- 172.21.15.39:0/1511812597 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f89740b6680 msgr2=0x7f89740b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:51:29.719 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:29.712+0000 7f898f1eb640 1 --2- 172.21.15.39:0/1511812597 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f89740b6680 0x7f89740b8b90 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f897800cfd0 tx=0x7f8978076210 comp rx=0 tx=0).stop 2024-09-21T03:51:29.719 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:29.712+0000 7f898f1eb640 1 -- 172.21.15.39:0/1511812597 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f898810ddf0 msgr2=0x7f898819fde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:51:29.719 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:29.712+0000 7f898f1eb640 1 --2- 172.21.15.39:0/1511812597 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f898810ddf0 0x7f898819fde0 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 crypto rx=0x7f897800eaa0 tx=0x7f897800ef60 comp rx=0 tx=0).stop 2024-09-21T03:51:29.719 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:29.712+0000 7f898f1eb640 1 -- 172.21.15.39:0/1511812597 shutdown_connections 2024-09-21T03:51:29.719 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:29.712+0000 7f898f1eb640 1 --2- 172.21.15.39:0/1511812597 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f89740b6680 0x7f89740b8b90 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:29.719 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:29.713+0000 7f898f1eb640 1 --2- 172.21.15.39:0/1511812597 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f898810ddf0 0x7f898819fde0 unknown :-1 s=CLOSED pgs=892 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:29.719 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:29.713+0000 7f898f1eb640 1 --2- 172.21.15.39:0/1511812597 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8988105270 0x7f898819ab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:29.719 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:29.713+0000 7f898f1eb640 1 --2- 172.21.15.39:0/1511812597 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8988104840 0x7f898819a5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:29.719 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:29.713+0000 7f898f1eb640 1 -- 172.21.15.39:0/1511812597 >> 172.21.15.39:0/1511812597 conn(0x7f8988079730 msgr2=0x7f8988100880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:51:29.719 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:29.713+0000 7f898f1eb640 1 -- 172.21.15.39:0/1511812597 shutdown_connections 2024-09-21T03:51:29.719 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:29.713+0000 7f898f1eb640 1 -- 172.21.15.39:0/1511812597 wait complete. 2024-09-21T03:51:31.420 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-21T03:51:31.420 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 51116d5c-77ca-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-21T03:51:31.449 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:51:31 smithi039 ceph-mon[31211]: pgmap v489: 97 pgs: 97 active+clean; 10 MiB data, 520 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 11 op/s 2024-09-21T03:51:31.468 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:51:31 smithi161 ceph-mon[34533]: pgmap v489: 97 pgs: 97 active+clean; 10 MiB data, 520 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 11 op/s 2024-09-21T03:51:31.475 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:51:31 smithi150 ceph-mon[34289]: pgmap v489: 97 pgs: 97 active+clean; 10 MiB data, 520 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 11 op/s 2024-09-21T03:51:31.695 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/51116d5c-77ca-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:51:32.202 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:32.198+0000 7f336952e640 1 -- 172.21.15.39:0/3598409787 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3364109e00 msgr2=0x7f3364110630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:51:32.202 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:32.198+0000 7f336952e640 1 --2- 172.21.15.39:0/3598409787 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3364109e00 0x7f3364110630 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 crypto rx=0x7f3354009ca0 tx=0x7f335402a8c0 comp rx=0 tx=0).stop 2024-09-21T03:51:32.202 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:32.198+0000 7f336952e640 1 -- 172.21.15.39:0/3598409787 shutdown_connections 2024-09-21T03:51:32.203 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:32.198+0000 7f336952e640 1 --2- 172.21.15.39:0/3598409787 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3364109e00 0x7f3364110630 unknown :-1 s=CLOSED pgs=897 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:32.203 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:32.198+0000 7f336952e640 1 --2- 172.21.15.39:0/3598409787 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f33641094b0 0x7f33641098c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:32.203 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:32.198+0000 7f336952e640 1 --2- 172.21.15.39:0/3598409787 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3364101f70 0x7f3364102380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:32.203 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:32.198+0000 7f336952e640 1 -- 172.21.15.39:0/3598409787 >> 172.21.15.39:0/3598409787 conn(0x7f3364077c60 msgr2=0x7f33640780a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:51:32.203 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:32.198+0000 7f336952e640 1 -- 172.21.15.39:0/3598409787 shutdown_connections 2024-09-21T03:51:32.203 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:32.198+0000 7f336952e640 1 -- 172.21.15.39:0/3598409787 wait complete. 2024-09-21T03:51:32.203 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:32.199+0000 7f336952e640 1 Processor -- start 2024-09-21T03:51:32.203 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:32.199+0000 7f336952e640 1 -- start start 2024-09-21T03:51:32.203 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:32.200+0000 7f336952e640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3364101f70 0x7f336419a490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:51:32.203 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:32.200+0000 7f336952e640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f33641094b0 0x7f336419aa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:51:32.203 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:32.200+0000 7f336952e640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3364109e00 0x7f336419fcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:51:32.203 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:32.200+0000 7f336952e640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f3364113ac0 con 0x7f3364101f70 2024-09-21T03:51:32.203 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:32.200+0000 7f336952e640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f3364113940 con 0x7f3364109e00 2024-09-21T03:51:32.203 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:32.200+0000 7f336952e640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f3364113c40 con 0x7f33641094b0 2024-09-21T03:51:32.204 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:32.200+0000 7f3362ffd640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3364101f70 0x7f336419a490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:51:32.204 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:32.200+0000 7f3362ffd640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3364101f70 0x7f336419a490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:40354/0 (socket says 172.21.15.39:40354) 2024-09-21T03:51:32.204 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:32.200+0000 7f3362ffd640 1 -- 172.21.15.39:0/395646684 learned_addr learned my addr 172.21.15.39:0/395646684 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:51:32.204 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:32.200+0000 7f33627fc640 1 --2- 172.21.15.39:0/395646684 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f33641094b0 0x7f336419aa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:51:32.204 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:32.200+0000 7f33637fe640 1 --2- 172.21.15.39:0/395646684 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3364109e00 0x7f336419fcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:51:32.205 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:32.200+0000 7f3362ffd640 1 -- 172.21.15.39:0/395646684 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f33641094b0 msgr2=0x7f336419aa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:51:32.205 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:32.200+0000 7f3362ffd640 1 --2- 172.21.15.39:0/395646684 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f33641094b0 0x7f336419aa30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:32.205 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:32.200+0000 7f3362ffd640 1 -- 172.21.15.39:0/395646684 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3364109e00 msgr2=0x7f336419fcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:51:32.205 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:32.200+0000 7f3362ffd640 1 --2- 172.21.15.39:0/395646684 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3364109e00 0x7f336419fcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:32.205 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:32.200+0000 7f3362ffd640 1 -- 172.21.15.39:0/395646684 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f33641a0400 con 0x7f3364101f70 2024-09-21T03:51:32.205 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:32.200+0000 7f33637fe640 1 --2- 172.21.15.39:0/395646684 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3364109e00 0x7f336419fcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:51:32.205 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:32.201+0000 7f3362ffd640 1 --2- 172.21.15.39:0/395646684 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3364101f70 0x7f336419a490 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 crypto rx=0x7f335c00d770 tx=0x7f335c00dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:51:32.205 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:32.201+0000 7f335bfff640 1 -- 172.21.15.39:0/395646684 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f335c017070 con 0x7f3364101f70 2024-09-21T03:51:32.205 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:32.201+0000 7f335bfff640 1 -- 172.21.15.39:0/395646684 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f335c0054d0 con 0x7f3364101f70 2024-09-21T03:51:32.205 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:32.201+0000 7f336952e640 1 -- 172.21.15.39:0/395646684 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f33641a06f0 con 0x7f3364101f70 2024-09-21T03:51:32.205 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:32.201+0000 7f335bfff640 1 -- 172.21.15.39:0/395646684 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f335c004c30 con 0x7f3364101f70 2024-09-21T03:51:32.205 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:32.201+0000 7f336952e640 1 -- 172.21.15.39:0/395646684 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f33641a6e50 con 0x7f3364101f70 2024-09-21T03:51:32.208 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:32.204+0000 7f335bfff640 1 -- 172.21.15.39:0/395646684 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 27) ==== 154643+0+0 (secure 0 0 0) 0x7f335c010460 con 0x7f3364101f70 2024-09-21T03:51:32.208 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:32.204+0000 7f336faa2740 1 Processor -- start 2024-09-21T03:51:32.208 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:32.204+0000 7f336faa2740 1 -- start start 2024-09-21T03:51:32.208 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:32.205+0000 7f336faa2740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56043b524350 0x56043b8cb430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:51:32.208 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:32.205+0000 7f336faa2740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56043b4fb490 0x56043b8cb970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:51:32.208 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:32.205+0000 7f336faa2740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56043b503670 0x56043b8cbeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:51:32.208 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:32.205+0000 7f336faa2740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x56043b841e60 con 0x56043b524350 2024-09-21T03:51:32.208 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:32.205+0000 7f336faa2740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x56043b831290 con 0x56043b503670 2024-09-21T03:51:32.208 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:32.205+0000 7f336faa2740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x56043b8350a0 con 0x56043b4fb490 2024-09-21T03:51:32.209 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:32.205+0000 7f33637fe640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56043b524350 0x56043b8cb430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:51:32.209 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:32.205+0000 7f33637fe640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56043b524350 0x56043b8cb430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:40360/0 (socket says 172.21.15.39:40360) 2024-09-21T03:51:32.209 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:32.205+0000 7f33637fe640 1 -- 172.21.15.39:0/1014104626 learned_addr learned my addr 172.21.15.39:0/1014104626 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:51:32.209 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:32.205+0000 7f3362ffd640 1 --2- 172.21.15.39:0/1014104626 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56043b4fb490 0x56043b8cb970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:51:32.209 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:32.205+0000 7f335bfff640 1 --2- 172.21.15.39:0/395646684 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f33500b6680 0x7f33500b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:51:32.209 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:32.205+0000 7f33627fc640 1 --2- 172.21.15.39:0/1014104626 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56043b503670 0x56043b8cbeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:51:32.209 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:32.205+0000 7f33637fe640 1 -- 172.21.15.39:0/1014104626 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56043b4fb490 msgr2=0x56043b8cb970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:51:32.209 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:32.205+0000 7f33637fe640 1 --2- 172.21.15.39:0/1014104626 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56043b4fb490 0x56043b8cb970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:32.209 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:32.205+0000 7f33637fe640 1 -- 172.21.15.39:0/1014104626 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56043b503670 msgr2=0x56043b8cbeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:51:32.209 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:32.205+0000 7f33637fe640 1 --2- 172.21.15.39:0/1014104626 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56043b503670 0x56043b8cbeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:32.209 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:32.205+0000 7f33637fe640 1 -- 172.21.15.39:0/1014104626 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56043b58f930 con 0x56043b524350 2024-09-21T03:51:32.209 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:32.206+0000 7f335bfff640 1 -- 172.21.15.39:0/395646684 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8484+0+0 (secure 0 0 0) 0x7f335c0e7270 con 0x7f3364101f70 2024-09-21T03:51:32.210 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:32.206+0000 7f33637fe640 1 --2- 172.21.15.39:0/395646684 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f33500b6680 0x7f33500b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:51:32.210 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:32.206+0000 7f33637fe640 1 --2- 172.21.15.39:0/1014104626 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56043b524350 0x56043b8cb430 secure :-1 s=READY pgs=899 cs=0 l=1 rev1=1 crypto rx=0x7f3354005cc0 tx=0x7f3354031b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe36c00280189f1e server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:51:32.210 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:32.206+0000 7f33597fa640 1 -- 172.21.15.39:0/1014104626 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3354003b30 con 0x56043b524350 2024-09-21T03:51:32.210 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:32.206+0000 7f33597fa640 1 -- 172.21.15.39:0/1014104626 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3354036040 con 0x56043b524350 2024-09-21T03:51:32.210 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:32.206+0000 7f33597fa640 1 -- 172.21.15.39:0/1014104626 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f335402f8a0 con 0x56043b524350 2024-09-21T03:51:32.210 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:32.206+0000 7f33637fe640 1 --2- 172.21.15.39:0/395646684 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f33500b6680 0x7f33500b8b90 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f335404c000 tx=0x7f3354049000 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:51:32.214 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:32.207+0000 7f336faa2740 1 -- 172.21.15.39:0/1014104626 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56043b524350 msgr2=0x56043b8cb430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:51:32.214 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:32.207+0000 7f336faa2740 1 --2- 172.21.15.39:0/1014104626 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56043b524350 0x56043b8cb430 secure :-1 s=READY pgs=899 cs=0 l=1 rev1=1 crypto rx=0x7f3354005cc0 tx=0x7f3354031b70 comp rx=0 tx=0).stop 2024-09-21T03:51:32.214 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:32.207+0000 7f336faa2740 1 -- 172.21.15.39:0/1014104626 shutdown_connections 2024-09-21T03:51:32.214 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:32.207+0000 7f336faa2740 1 --2- 172.21.15.39:0/1014104626 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56043b503670 0x56043b8cbeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:32.215 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:32.207+0000 7f336faa2740 1 --2- 172.21.15.39:0/1014104626 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56043b4fb490 0x56043b8cb970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:32.215 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:32.207+0000 7f336faa2740 1 --2- 172.21.15.39:0/1014104626 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56043b524350 0x56043b8cb430 unknown :-1 s=CLOSED pgs=899 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:32.215 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:32.207+0000 7f336faa2740 1 -- 172.21.15.39:0/1014104626 >> 172.21.15.39:0/1014104626 conn(0x56043b518740 msgr2=0x56043b8c9320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:51:32.215 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:32.207+0000 7f336faa2740 1 -- 172.21.15.39:0/1014104626 shutdown_connections 2024-09-21T03:51:32.215 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:32.207+0000 7f336faa2740 1 -- 172.21.15.39:0/1014104626 wait complete. 2024-09-21T03:51:32.215 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:32.208+0000 7f336faa2740 1 -- start start 2024-09-21T03:51:32.215 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:32.208+0000 7f336faa2740 1 Processor -- start 2024-09-21T03:51:32.215 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:32.208+0000 7f336faa2740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56043b4fb490 0x56043b8db3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:51:32.215 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:32.208+0000 7f336faa2740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56043b503670 0x56043b8db8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:51:32.215 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:32.208+0000 7f336faa2740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56043b524350 0x56043b8e7120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:51:32.215 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:32.208+0000 7f336faa2740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x56043b692890 con 0x56043b4fb490 2024-09-21T03:51:32.215 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:32.208+0000 7f336faa2740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x56043b7cbd40 con 0x56043b524350 2024-09-21T03:51:32.216 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:32.208+0000 7f336faa2740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x56043b8dddf0 con 0x56043b503670 2024-09-21T03:51:32.216 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:32.208+0000 7f336faa2740 10 client.0 _ll_register_callbacks cb 0x56043b82fc80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:51:32.216 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:32.208+0000 7f3362ffd640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56043b4fb490 0x56043b8db3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:51:32.216 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:32.208+0000 7f3362ffd640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56043b4fb490 0x56043b8db3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:40372/0 (socket says 172.21.15.39:40372) 2024-09-21T03:51:32.216 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:32.208+0000 7f3362ffd640 1 -- 172.21.15.39:0/2388324902 learned_addr learned my addr 172.21.15.39:0/2388324902 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:51:32.216 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:32.208+0000 7f33627fc640 1 --2- 172.21.15.39:0/2388324902 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56043b503670 0x56043b8db8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:51:32.216 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:32.208+0000 7f33637fe640 1 --2- 172.21.15.39:0/2388324902 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56043b524350 0x56043b8e7120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:51:32.216 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:32.208+0000 7f3362ffd640 1 -- 172.21.15.39:0/2388324902 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56043b503670 msgr2=0x56043b8db8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:51:32.216 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:32.208+0000 7f3362ffd640 1 --2- 172.21.15.39:0/2388324902 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56043b503670 0x56043b8db8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:32.216 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:32.209+0000 7f3362ffd640 1 -- 172.21.15.39:0/2388324902 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56043b524350 msgr2=0x56043b8e7120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:51:32.216 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:32.209+0000 7f3362ffd640 1 --2- 172.21.15.39:0/2388324902 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56043b524350 0x56043b8e7120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:32.216 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:32.209+0000 7f3362ffd640 1 -- 172.21.15.39:0/2388324902 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56043b8e7660 con 0x56043b4fb490 2024-09-21T03:51:32.216 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:32.209+0000 7f33637fe640 1 --2- 172.21.15.39:0/2388324902 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56043b524350 0x56043b8e7120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:51:32.217 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:32.209+0000 7f3362ffd640 1 --2- 172.21.15.39:0/2388324902 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56043b4fb490 0x56043b8db3a0 secure :-1 s=READY pgs=900 cs=0 l=1 rev1=1 crypto rx=0x7f335c0a96b0 tx=0x7f335c0e44a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=97e2174499452ea3 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:51:32.217 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:32.209+0000 7f334a7fc640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-09-21T03:51:32.217 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:32.209+0000 7f334a7fc640 1 -- 172.21.15.39:0/2388324902 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f335c020040 con 0x56043b4fb490 2024-09-21T03:51:32.217 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:32.209+0000 7f334a7fc640 1 -- 172.21.15.39:0/2388324902 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8484+0+0 (secure 0 0 0) 0x7f335c0e4c60 con 0x56043b4fb490 2024-09-21T03:51:32.217 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:32.209+0000 7f336faa2740 1 -- 172.21.15.39:0/2388324902 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3354036040 con 0x56043b4fb490 2024-09-21T03:51:32.217 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:32.209+0000 7f334a7fc640 1 -- 172.21.15.39:0/2388324902 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f335c0aca30 con 0x56043b4fb490 2024-09-21T03:51:32.217 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:32.209+0000 7f336faa2740 10 client.16974 fetch_fsmap learned FSMap version 29 2024-09-21T03:51:32.217 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:32.209+0000 7f336faa2740 1 -- 172.21.15.39:0/2388324902 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x56043b8e9fe0 con 0x56043b4fb490 2024-09-21T03:51:32.217 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:32.209+0000 7f334a7fc640 1 -- 172.21.15.39:0/2388324902 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3354036040 con 0x56043b4fb490 2024-09-21T03:51:32.217 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:32.209+0000 7f334a7fc640 1 -- 172.21.15.39:0/2388324902 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f335c0a9350 con 0x56043b4fb490 2024-09-21T03:51:32.217 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:32.210+0000 7f336faa2740 10 client.16974 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T03:51:32.217 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:32.210+0000 7f336faa2740 10 client.16974 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T03:51:32.217 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:32.210+0000 7f336faa2740 20 client.16974 populate_metadata read hostname 'smithi039' 2024-09-21T03:51:32.218 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:32.210+0000 7f336faa2740 1 --2- 172.21.15.39:0/2388324902 >> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] conn(0x56043b4e2cd0 0x56043b8ed4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:51:32.218 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:32.210+0000 7f336faa2740 4 client.16974 mds_command: new command op to 24695 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:51:32.218 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:32.210+0000 7f336faa2740 1 -- 172.21.15.39:0/2388324902 --> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56043b843db0 con 0x56043b4e2cd0 2024-09-21T03:51:32.218 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:32.210+0000 7f33627fc640 1 --2- 172.21.15.39:0/2388324902 >> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] conn(0x56043b4e2cd0 0x56043b8ed4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:51:32.218 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:32.210+0000 7f334a7fc640 10 client.16974 ms_handle_connect on v2:172.21.15.150:6838/2949386907 2024-09-21T03:51:32.218 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:32.210+0000 7f33627fc640 1 --2- 172.21.15.39:0/2388324902 >> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] conn(0x56043b4e2cd0 0x56043b8ed4b0 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dadf0007e17d8b4f server_cookie=a2fb116a967f3ca7 in_seq=0 out_seq=0 2024-09-21T03:51:32.218 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:32.211+0000 7f334a7fc640 1 -- 172.21.15.39:0/2388324902 <== mds.0 v2:172.21.15.150:6838/2949386907 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x56043b843db0 con 0x56043b4e2cd0 2024-09-21T03:51:32.218 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:32.211+0000 7f334a7fc640 10 client.16974 handle_command_reply: tid=0 2024-09-21T03:51:32.218 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:32.211+0000 7f336faa2740 1 client.16974 shutdown 2024-09-21T03:51:32.218 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:32.211+0000 7f336faa2740 1 -- 172.21.15.39:0/2388324902 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56043b4fb490 msgr2=0x56043b8db3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:51:32.218 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:32.211+0000 7f336faa2740 1 --2- 172.21.15.39:0/2388324902 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56043b4fb490 0x56043b8db3a0 secure :-1 s=READY pgs=900 cs=0 l=1 rev1=1 crypto rx=0x7f335c0a96b0 tx=0x7f335c0e44a0 comp rx=0 tx=0).stop 2024-09-21T03:51:32.218 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:32.212+0000 7f336faa2740 1 -- 172.21.15.39:0/2388324902 shutdown_connections 2024-09-21T03:51:32.219 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:32.212+0000 7f336faa2740 1 --2- 172.21.15.39:0/2388324902 >> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] conn(0x56043b4e2cd0 0x56043b8ed4b0 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:32.219 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:32.212+0000 7f336faa2740 1 --2- 172.21.15.39:0/2388324902 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56043b524350 0x56043b8e7120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:32.219 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:32.212+0000 7f336faa2740 1 --2- 172.21.15.39:0/2388324902 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56043b503670 0x56043b8db8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:32.219 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:32.212+0000 7f336faa2740 1 --2- 172.21.15.39:0/2388324902 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56043b4fb490 0x56043b8db3a0 unknown :-1 s=CLOSED pgs=900 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:32.219 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:32.212+0000 7f334a7fc640 0 client.16974 ms_handle_reset on v2:172.21.15.150:6838/2949386907 2024-09-21T03:51:32.219 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:32.212+0000 7f336faa2740 1 -- 172.21.15.39:0/2388324902 >> 172.21.15.39:0/2388324902 conn(0x56043b518740 msgr2=0x56043b8c7850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:51:32.219 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:32.212+0000 7f336faa2740 1 -- 172.21.15.39:0/2388324902 shutdown_connections 2024-09-21T03:51:32.219 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:32.212+0000 7f336faa2740 1 -- 172.21.15.39:0/2388324902 wait complete. 2024-09-21T03:51:32.219 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:32.212+0000 7f336faa2740 20 client.16974 trim_cache size 0 max 16384 2024-09-21T03:51:32.231 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:32.229+0000 7f336faa2740 1 Processor -- start 2024-09-21T03:51:32.231 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:32.229+0000 7f336faa2740 1 -- start start 2024-09-21T03:51:32.232 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:32.230+0000 7f336faa2740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56043b4fb490 0x56043b8e7020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:51:32.233 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:32.230+0000 7f336faa2740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56043b503670 0x56043b8c7850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:51:32.233 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:32.230+0000 7f336faa2740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56043b524350 0x56043b8c7d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:51:32.233 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:32.230+0000 7f336faa2740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f33641ad4b0 con 0x56043b4fb490 2024-09-21T03:51:32.233 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:32.230+0000 7f336faa2740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f33641aeff0 con 0x56043b524350 2024-09-21T03:51:32.233 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:32.230+0000 7f336faa2740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f33641adb80 con 0x56043b503670 2024-09-21T03:51:32.233 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:32.230+0000 7f3362ffd640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56043b4fb490 0x56043b8e7020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:51:32.233 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:32.230+0000 7f3362ffd640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56043b4fb490 0x56043b8e7020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:40386/0 (socket says 172.21.15.39:40386) 2024-09-21T03:51:32.233 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:32.230+0000 7f3362ffd640 1 -- 172.21.15.39:0/1226042476 learned_addr learned my addr 172.21.15.39:0/1226042476 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:51:32.233 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:32.230+0000 7f33627fc640 1 --2- 172.21.15.39:0/1226042476 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56043b503670 0x56043b8c7850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:51:32.233 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:32.230+0000 7f33637fe640 1 --2- 172.21.15.39:0/1226042476 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56043b524350 0x56043b8c7d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:51:32.233 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:32.230+0000 7f3362ffd640 1 -- 172.21.15.39:0/1226042476 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56043b503670 msgr2=0x56043b8c7850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:51:32.233 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:32.230+0000 7f3362ffd640 1 --2- 172.21.15.39:0/1226042476 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56043b503670 0x56043b8c7850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:32.233 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:32.230+0000 7f3362ffd640 1 -- 172.21.15.39:0/1226042476 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56043b524350 msgr2=0x56043b8c7d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:51:32.234 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:32.230+0000 7f3362ffd640 1 --2- 172.21.15.39:0/1226042476 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56043b524350 0x56043b8c7d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:32.234 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:32.230+0000 7f3362ffd640 1 -- 172.21.15.39:0/1226042476 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56043b8ce250 con 0x56043b4fb490 2024-09-21T03:51:32.234 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:32.230+0000 7f33637fe640 1 --2- 172.21.15.39:0/1226042476 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56043b524350 0x56043b8c7d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:51:32.234 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:32.230+0000 7f33627fc640 1 --2- 172.21.15.39:0/1226042476 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56043b503670 0x56043b8c7850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:51:32.234 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:32.231+0000 7f3362ffd640 1 --2- 172.21.15.39:0/1226042476 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56043b4fb490 0x56043b8e7020 secure :-1 s=READY pgs=901 cs=0 l=1 rev1=1 crypto rx=0x7f335c0aa350 tx=0x7f335c0eaab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e774f10f3f2c1ae3 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:51:32.234 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:32.231+0000 7f3358ff9640 1 -- 172.21.15.39:0/1226042476 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f335c020070 con 0x56043b4fb490 2024-09-21T03:51:32.234 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:32.231+0000 7f3358ff9640 1 -- 172.21.15.39:0/1226042476 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f335c0acbf0 con 0x56043b4fb490 2024-09-21T03:51:32.234 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:32.231+0000 7f3358ff9640 1 -- 172.21.15.39:0/1226042476 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f335c0ac850 con 0x56043b4fb490 2024-09-21T03:51:32.237 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:32.232+0000 7f336faa2740 1 -- 172.21.15.39:0/1226042476 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56043b4fb490 msgr2=0x56043b8e7020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:51:32.237 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:32.232+0000 7f336faa2740 1 --2- 172.21.15.39:0/1226042476 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56043b4fb490 0x56043b8e7020 secure :-1 s=READY pgs=901 cs=0 l=1 rev1=1 crypto rx=0x7f335c0aa350 tx=0x7f335c0eaab0 comp rx=0 tx=0).stop 2024-09-21T03:51:32.237 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:32.232+0000 7f336faa2740 1 -- 172.21.15.39:0/1226042476 shutdown_connections 2024-09-21T03:51:32.238 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:32.232+0000 7f336faa2740 1 --2- 172.21.15.39:0/1226042476 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56043b524350 0x56043b8c7d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:32.238 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:32.232+0000 7f336faa2740 1 --2- 172.21.15.39:0/1226042476 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56043b503670 0x56043b8c7850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:32.238 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:32.232+0000 7f336faa2740 1 --2- 172.21.15.39:0/1226042476 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56043b4fb490 0x56043b8e7020 unknown :-1 s=CLOSED pgs=901 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:32.238 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:32.232+0000 7f336faa2740 1 -- 172.21.15.39:0/1226042476 >> 172.21.15.39:0/1226042476 conn(0x56043b518740 msgr2=0x56043b8d9a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:51:32.238 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:32.232+0000 7f336faa2740 1 -- 172.21.15.39:0/1226042476 shutdown_connections 2024-09-21T03:51:32.238 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:32.232+0000 7f336faa2740 1 -- 172.21.15.39:0/1226042476 wait complete. 2024-09-21T03:51:32.238 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:32.232+0000 7f336faa2740 1 -- start start 2024-09-21T03:51:32.238 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:32.232+0000 7f336faa2740 1 Processor -- start 2024-09-21T03:51:32.238 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:32.233+0000 7f336faa2740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56043b4fb490 0x56043b8e3090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:51:32.238 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:32.233+0000 7f336faa2740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56043b503670 0x56043b8e35d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:51:32.238 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:32.233+0000 7f336faa2740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56043b524350 0x56043b8de040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:51:32.239 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:32.233+0000 7f336faa2740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f33641ae920 con 0x56043b503670 2024-09-21T03:51:32.239 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:32.233+0000 7f336faa2740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f33641ae250 con 0x56043b524350 2024-09-21T03:51:32.239 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:32.233+0000 7f336faa2740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f33641ab490 con 0x56043b4fb490 2024-09-21T03:51:32.239 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:32.233+0000 7f336faa2740 10 client.0 _ll_register_callbacks cb 0x7f335c00d6c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:51:32.239 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:32.233+0000 7f33627fc640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56043b503670 0x56043b8e35d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:51:32.239 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:32.233+0000 7f33627fc640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56043b503670 0x56043b8e35d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:40402/0 (socket says 172.21.15.39:40402) 2024-09-21T03:51:32.239 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:32.233+0000 7f33627fc640 1 -- 172.21.15.39:0/3846396078 learned_addr learned my addr 172.21.15.39:0/3846396078 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:51:32.239 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:32.233+0000 7f3362ffd640 1 --2- 172.21.15.39:0/3846396078 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56043b4fb490 0x56043b8e3090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:51:32.239 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:32.233+0000 7f33627fc640 1 -- 172.21.15.39:0/3846396078 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56043b4fb490 msgr2=0x56043b8e3090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:51:32.239 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:32.233+0000 7f33637fe640 1 --2- 172.21.15.39:0/3846396078 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56043b524350 0x56043b8de040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:51:32.239 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:32.233+0000 7f33627fc640 1 --2- 172.21.15.39:0/3846396078 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56043b4fb490 0x56043b8e3090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:32.239 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:32.233+0000 7f33627fc640 1 -- 172.21.15.39:0/3846396078 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56043b524350 msgr2=0x56043b8de040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:51:32.240 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:32.233+0000 7f33627fc640 1 --2- 172.21.15.39:0/3846396078 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56043b524350 0x56043b8de040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:32.240 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:32.233+0000 7f33627fc640 1 -- 172.21.15.39:0/3846396078 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56043b8e3cc0 con 0x56043b503670 2024-09-21T03:51:32.240 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:32.233+0000 7f3362ffd640 1 --2- 172.21.15.39:0/3846396078 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56043b4fb490 0x56043b8e3090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:51:32.240 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:32.233+0000 7f33637fe640 1 --2- 172.21.15.39:0/3846396078 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56043b524350 0x56043b8de040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:51:32.240 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:32.233+0000 7f33627fc640 1 --2- 172.21.15.39:0/3846396078 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56043b503670 0x56043b8e35d0 secure :-1 s=READY pgs=902 cs=0 l=1 rev1=1 crypto rx=0x7f3364075820 tx=0x7f334c011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c2097b8470f597ad server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:51:32.240 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:32.233+0000 7f334a7fc640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-09-21T03:51:32.240 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:32.233+0000 7f334a7fc640 1 -- 172.21.15.39:0/3846396078 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f334c01d740 con 0x56043b503670 2024-09-21T03:51:32.240 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:32.233+0000 7f334a7fc640 1 -- 172.21.15.39:0/3846396078 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8484+0+0 (secure 0 0 0) 0x7f334c01d8e0 con 0x56043b503670 2024-09-21T03:51:32.240 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:32.234+0000 7f336faa2740 1 -- 172.21.15.39:0/3846396078 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f335c0acbf0 con 0x56043b503670 2024-09-21T03:51:32.240 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:32.234+0000 7f334a7fc640 1 -- 172.21.15.39:0/3846396078 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f334c01dae0 con 0x56043b503670 2024-09-21T03:51:32.240 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:32.234+0000 7f336faa2740 10 client.16980 fetch_fsmap learned FSMap version 29 2024-09-21T03:51:32.240 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:32.234+0000 7f336faa2740 1 -- 172.21.15.39:0/3846396078 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x56043b958260 con 0x56043b503670 2024-09-21T03:51:32.241 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:32.234+0000 7f334a7fc640 1 -- 172.21.15.39:0/3846396078 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f335c0acbf0 con 0x56043b503670 2024-09-21T03:51:32.241 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:32.234+0000 7f334a7fc640 1 -- 172.21.15.39:0/3846396078 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f334c02b020 con 0x56043b503670 2024-09-21T03:51:32.241 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:32.234+0000 7f336faa2740 10 client.16980 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T03:51:32.241 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:32.234+0000 7f336faa2740 10 client.16980 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T03:51:32.241 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:32.234+0000 7f336faa2740 20 client.16980 populate_metadata read hostname 'smithi039' 2024-09-21T03:51:32.241 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:32.234+0000 7f336faa2740 1 --2- 172.21.15.39:0/3846396078 >> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] conn(0x56043b4b8080 0x56043b95aa00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:51:32.241 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:32.234+0000 7f336faa2740 4 client.16980 mds_command: new command op to 24695 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-21T03:51:32.241 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:32.234+0000 7f336faa2740 1 -- 172.21.15.39:0/3846396078 --> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x56043b83f860 con 0x56043b4b8080 2024-09-21T03:51:32.241 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:32.235+0000 7f3362ffd640 1 --2- 172.21.15.39:0/3846396078 >> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] conn(0x56043b4b8080 0x56043b95aa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:51:32.241 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:32.235+0000 7f3362ffd640 1 --2- 172.21.15.39:0/3846396078 >> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] conn(0x56043b4b8080 0x56043b95aa00 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=78a576ec0bb0e837 server_cookie=b7abb0c17316e606 in_seq=0 out_seq=0 2024-09-21T03:51:32.241 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:32.235+0000 7f334a7fc640 10 client.16980 ms_handle_connect on v2:172.21.15.150:6838/2949386907 2024-09-21T03:51:32.242 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:32.237+0000 7f334a7fc640 1 -- 172.21.15.39:0/3846396078 <== mds.0 v2:172.21.15.150:6838/2949386907 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x56043b83f860 con 0x56043b4b8080 2024-09-21T03:51:32.243 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:32.237+0000 7f334a7fc640 10 client.16980 handle_command_reply: tid=0 2024-09-21T03:51:32.243 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:32.237+0000 7f336faa2740 1 client.16980 shutdown 2024-09-21T03:51:32.243 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:32.237+0000 7f336faa2740 1 -- 172.21.15.39:0/3846396078 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56043b503670 msgr2=0x56043b8e35d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:51:32.243 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:32.237+0000 7f336faa2740 1 --2- 172.21.15.39:0/3846396078 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56043b503670 0x56043b8e35d0 secure :-1 s=READY pgs=902 cs=0 l=1 rev1=1 crypto rx=0x7f3364075820 tx=0x7f334c011810 comp rx=0 tx=0).stop 2024-09-21T03:51:32.243 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:32.237+0000 7f336faa2740 1 -- 172.21.15.39:0/3846396078 shutdown_connections 2024-09-21T03:51:32.243 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:32.237+0000 7f336faa2740 1 --2- 172.21.15.39:0/3846396078 >> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] conn(0x56043b4b8080 0x56043b95aa00 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:32.243 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:32.237+0000 7f336faa2740 1 --2- 172.21.15.39:0/3846396078 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56043b524350 0x56043b8de040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:32.243 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:32.237+0000 7f336faa2740 1 --2- 172.21.15.39:0/3846396078 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56043b503670 0x56043b8e35d0 unknown :-1 s=CLOSED pgs=902 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:32.243 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:32.237+0000 7f334a7fc640 0 client.16980 ms_handle_reset on v2:172.21.15.150:6838/2949386907 2024-09-21T03:51:32.243 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:32.237+0000 7f336faa2740 1 --2- 172.21.15.39:0/3846396078 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56043b4fb490 0x56043b8e3090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:32.243 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:32.237+0000 7f336faa2740 1 -- 172.21.15.39:0/3846396078 >> 172.21.15.39:0/3846396078 conn(0x56043b518740 msgr2=0x56043b8cc440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:51:32.244 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:32.237+0000 7f336faa2740 1 -- 172.21.15.39:0/3846396078 shutdown_connections 2024-09-21T03:51:32.244 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:32.237+0000 7f336faa2740 1 -- 172.21.15.39:0/3846396078 wait complete. 2024-09-21T03:51:32.244 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:32.237+0000 7f336faa2740 20 client.16980 trim_cache size 0 max 16384 2024-09-21T03:51:32.244 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-09-21T03:51:32.244 INFO:teuthology.orchestra.run.smithi039.stdout: "return_code": 0, 2024-09-21T03:51:32.244 INFO:teuthology.orchestra.run.smithi039.stdout: "scrub_tag": "ede73e6e-3fd0-4a7f-b5bb-f68946f57eea", 2024-09-21T03:51:32.244 INFO:teuthology.orchestra.run.smithi039.stdout: "mode": "asynchronous" 2024-09-21T03:51:32.244 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-09-21T03:51:32.245 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:32.238+0000 7f336952e640 1 -- 172.21.15.39:0/395646684 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f33500b6680 msgr2=0x7f33500b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:51:32.245 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:32.238+0000 7f336952e640 1 --2- 172.21.15.39:0/395646684 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f33500b6680 0x7f33500b8b90 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f335404c000 tx=0x7f3354049000 comp rx=0 tx=0).stop 2024-09-21T03:51:32.245 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:32.238+0000 7f336952e640 1 -- 172.21.15.39:0/395646684 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3364101f70 msgr2=0x7f336419a490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:51:32.245 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:32.238+0000 7f336952e640 1 --2- 172.21.15.39:0/395646684 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3364101f70 0x7f336419a490 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 crypto rx=0x7f335c00d770 tx=0x7f335c00dc30 comp rx=0 tx=0).stop 2024-09-21T03:51:32.245 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:32.238+0000 7f336952e640 1 -- 172.21.15.39:0/395646684 shutdown_connections 2024-09-21T03:51:32.245 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:32.238+0000 7f336952e640 1 --2- 172.21.15.39:0/395646684 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f33500b6680 0x7f33500b8b90 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:32.246 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:32.238+0000 7f336952e640 1 --2- 172.21.15.39:0/395646684 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3364109e00 0x7f336419fcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:32.246 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:32.238+0000 7f336952e640 1 --2- 172.21.15.39:0/395646684 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f33641094b0 0x7f336419aa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:32.246 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:32.238+0000 7f336952e640 1 --2- 172.21.15.39:0/395646684 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3364101f70 0x7f336419a490 unknown :-1 s=CLOSED pgs=898 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:32.246 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:32.238+0000 7f336952e640 1 -- 172.21.15.39:0/395646684 >> 172.21.15.39:0/395646684 conn(0x7f3364077c60 msgr2=0x7f3364105480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:51:32.246 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:32.238+0000 7f336952e640 1 -- 172.21.15.39:0/395646684 shutdown_connections 2024-09-21T03:51:32.246 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:32.238+0000 7f336952e640 1 -- 172.21.15.39:0/395646684 wait complete. 2024-09-21T03:51:33.027 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 51116d5c-77ca-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub status 2024-09-21T03:51:33.326 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/51116d5c-77ca-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:51:33.333 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:51:33 smithi039 ceph-mon[31211]: pgmap v490: 97 pgs: 97 active+clean; 10 MiB data, 520 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 9 op/s 2024-09-21T03:51:33.334 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:51:33 smithi039 ceph-mon[31211]: scrub queued for path: / 2024-09-21T03:51:33.334 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:51:33 smithi039 ceph-mon[31211]: scrub summary: idle+waiting paths [/] 2024-09-21T03:51:33.334 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:51:33 smithi039 ceph-mon[31211]: scrub summary: active paths [/] 2024-09-21T03:51:33.334 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:51:33 smithi039 ceph-mon[31211]: scrub summary: idle+waiting paths [/] 2024-09-21T03:51:33.468 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:51:33 smithi161 ceph-mon[34533]: pgmap v490: 97 pgs: 97 active+clean; 10 MiB data, 520 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 9 op/s 2024-09-21T03:51:33.468 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:51:33 smithi161 ceph-mon[34533]: scrub queued for path: / 2024-09-21T03:51:33.468 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:51:33 smithi161 ceph-mon[34533]: scrub summary: idle+waiting paths [/] 2024-09-21T03:51:33.468 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:51:33 smithi161 ceph-mon[34533]: scrub summary: active paths [/] 2024-09-21T03:51:33.468 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:51:33 smithi161 ceph-mon[34533]: scrub summary: idle+waiting paths [/] 2024-09-21T03:51:33.475 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:51:33 smithi150 ceph-mon[34289]: pgmap v490: 97 pgs: 97 active+clean; 10 MiB data, 520 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 9 op/s 2024-09-21T03:51:33.475 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:51:33 smithi150 ceph-mon[34289]: scrub queued for path: / 2024-09-21T03:51:33.475 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:51:33 smithi150 ceph-mon[34289]: scrub summary: idle+waiting paths [/] 2024-09-21T03:51:33.475 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:51:33 smithi150 ceph-mon[34289]: scrub summary: active paths [/] 2024-09-21T03:51:33.475 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:51:33 smithi150 ceph-mon[34289]: scrub summary: idle+waiting paths [/] 2024-09-21T03:51:33.811 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:33.795+0000 7f849c054640 1 -- 172.21.15.39:0/1351344457 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8494103b90 msgr2=0x7f8494103fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:51:33.812 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:33.795+0000 7f849c054640 1 --2- 172.21.15.39:0/1351344457 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8494103b90 0x7f8494103fa0 secure :-1 s=READY pgs=903 cs=0 l=1 rev1=1 crypto rx=0x7f8490009900 tx=0x7f849002a410 comp rx=0 tx=0).stop 2024-09-21T03:51:33.812 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:33.795+0000 7f849c054640 1 -- 172.21.15.39:0/1351344457 shutdown_connections 2024-09-21T03:51:33.812 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:33.795+0000 7f849c054640 1 --2- 172.21.15.39:0/1351344457 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8494109e20 0x7f8494110650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:33.812 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:33.795+0000 7f849c054640 1 --2- 172.21.15.39:0/1351344457 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f84941094d0 0x7f84941098e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:33.812 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:33.795+0000 7f849c054640 1 --2- 172.21.15.39:0/1351344457 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8494103b90 0x7f8494103fa0 unknown :-1 s=CLOSED pgs=903 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:33.812 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:33.795+0000 7f849c054640 1 -- 172.21.15.39:0/1351344457 >> 172.21.15.39:0/1351344457 conn(0x7f84940ff600 msgr2=0x7f8494101a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:51:33.812 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:33.795+0000 7f849c054640 1 -- 172.21.15.39:0/1351344457 shutdown_connections 2024-09-21T03:51:33.812 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:33.795+0000 7f849c054640 1 -- 172.21.15.39:0/1351344457 wait complete. 2024-09-21T03:51:33.812 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:33.796+0000 7f849c054640 1 Processor -- start 2024-09-21T03:51:33.812 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:33.796+0000 7f849c054640 1 -- start start 2024-09-21T03:51:33.812 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:33.796+0000 7f849c054640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8494103b90 0x7f8494075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:51:33.812 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:33.796+0000 7f849c054640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f84941094d0 0x7f8494075c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:51:33.812 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:33.796+0000 7f849c054640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8494109e20 0x7f8494076190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:51:33.813 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:33.796+0000 7f849c054640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f8494113ae0 con 0x7f84941094d0 2024-09-21T03:51:33.813 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:33.796+0000 7f849c054640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f8494113960 con 0x7f8494103b90 2024-09-21T03:51:33.813 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:33.796+0000 7f849c054640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f8494113c60 con 0x7f8494109e20 2024-09-21T03:51:33.813 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:33.796+0000 7f84995ef640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f84941094d0 0x7f8494075c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:51:33.813 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:33.796+0000 7f84995ef640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f84941094d0 0x7f8494075c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:40414/0 (socket says 172.21.15.39:40414) 2024-09-21T03:51:33.813 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:33.796+0000 7f84995ef640 1 -- 172.21.15.39:0/2683243707 learned_addr learned my addr 172.21.15.39:0/2683243707 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:51:33.813 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:33.796+0000 7f8499df0640 1 --2- 172.21.15.39:0/2683243707 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8494103b90 0x7f8494075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:51:33.813 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:33.796+0000 7f849a5f1640 1 --2- 172.21.15.39:0/2683243707 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8494109e20 0x7f8494076190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:51:33.813 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:33.797+0000 7f84995ef640 1 -- 172.21.15.39:0/2683243707 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8494109e20 msgr2=0x7f8494076190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:51:33.813 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:33.797+0000 7f84995ef640 1 --2- 172.21.15.39:0/2683243707 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8494109e20 0x7f8494076190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:33.813 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:33.797+0000 7f84995ef640 1 -- 172.21.15.39:0/2683243707 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8494103b90 msgr2=0x7f8494075710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:51:33.813 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:33.797+0000 7f84995ef640 1 --2- 172.21.15.39:0/2683243707 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8494103b90 0x7f8494075710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:33.813 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:33.797+0000 7f84995ef640 1 -- 172.21.15.39:0/2683243707 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f84941a8790 con 0x7f84941094d0 2024-09-21T03:51:33.813 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:33.797+0000 7f8499df0640 1 --2- 172.21.15.39:0/2683243707 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8494103b90 0x7f8494075710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:51:33.813 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:33.797+0000 7f84995ef640 1 --2- 172.21.15.39:0/2683243707 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f84941094d0 0x7f8494075c50 secure :-1 s=READY pgs=904 cs=0 l=1 rev1=1 crypto rx=0x7f848400d740 tx=0x7f848400dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:51:33.814 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:33.797+0000 7f848affd640 1 -- 172.21.15.39:0/2683243707 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8484017070 con 0x7f84941094d0 2024-09-21T03:51:33.814 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:33.797+0000 7f848affd640 1 -- 172.21.15.39:0/2683243707 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f848400de50 con 0x7f84941094d0 2024-09-21T03:51:33.814 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:33.797+0000 7f849c054640 1 -- 172.21.15.39:0/2683243707 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f84941a8a80 con 0x7f84941094d0 2024-09-21T03:51:33.814 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:33.797+0000 7f848affd640 1 -- 172.21.15.39:0/2683243707 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f848400f620 con 0x7f84941094d0 2024-09-21T03:51:33.814 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:33.797+0000 7f849c054640 1 -- 172.21.15.39:0/2683243707 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f84941a8fc0 con 0x7f84941094d0 2024-09-21T03:51:33.814 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:33.800+0000 7f84a25c8740 1 Processor -- start 2024-09-21T03:51:33.814 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:33.800+0000 7f84a25c8740 1 -- start start 2024-09-21T03:51:33.814 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:33.800+0000 7f848affd640 1 -- 172.21.15.39:0/2683243707 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 27) ==== 154643+0+0 (secure 0 0 0) 0x7f848401a070 con 0x7f84941094d0 2024-09-21T03:51:33.814 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:33.800+0000 7f84a25c8740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557c500f7070 0x557c5049e650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:51:33.814 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:33.800+0000 7f84a25c8740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557c500ce1b0 0x557c5049eb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:51:33.814 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:33.800+0000 7f84a25c8740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557c500d6390 0x557c5049f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:51:33.814 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:33.800+0000 7f84a25c8740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x557c50414e50 con 0x557c500ce1b0 2024-09-21T03:51:33.814 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:33.800+0000 7f84a25c8740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x557c50404280 con 0x557c500f7070 2024-09-21T03:51:33.815 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:33.800+0000 7f84a25c8740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x557c50408090 con 0x557c500d6390 2024-09-21T03:51:33.815 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:33.800+0000 7f8499df0640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557c500ce1b0 0x557c5049eb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:51:33.815 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:33.800+0000 7f8499df0640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557c500ce1b0 0x557c5049eb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:40424/0 (socket says 172.21.15.39:40424) 2024-09-21T03:51:33.815 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:33.800+0000 7f8499df0640 1 -- 172.21.15.39:0/3858465680 learned_addr learned my addr 172.21.15.39:0/3858465680 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:51:33.815 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:33.801+0000 7f849a5f1640 1 --2- 172.21.15.39:0/3858465680 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557c500f7070 0x557c5049e650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:51:33.815 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:33.801+0000 7f84995ef640 1 --2- 172.21.15.39:0/3858465680 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557c500d6390 0x557c5049f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:51:33.815 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:33.801+0000 7f8499df0640 1 -- 172.21.15.39:0/3858465680 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557c500d6390 msgr2=0x557c5049f0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:51:33.815 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:33.801+0000 7f8499df0640 1 --2- 172.21.15.39:0/3858465680 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557c500d6390 0x557c5049f0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:33.815 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:33.801+0000 7f8499df0640 1 -- 172.21.15.39:0/3858465680 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557c500f7070 msgr2=0x557c5049e650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:51:33.815 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:33.801+0000 7f8499df0640 1 --2- 172.21.15.39:0/3858465680 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557c500f7070 0x557c5049e650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:33.815 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:33.801+0000 7f8499df0640 1 -- 172.21.15.39:0/3858465680 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557c50162650 con 0x557c500ce1b0 2024-09-21T03:51:33.815 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:33.801+0000 7f848affd640 1 --2- 172.21.15.39:0/2683243707 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f84800b6680 0x7f84800b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:51:33.815 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:33.801+0000 7f8499df0640 1 --2- 172.21.15.39:0/3858465680 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557c500ce1b0 0x557c5049eb90 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 crypto rx=0x7f849002a930 tx=0x7f8490032e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e36e12d69178f0ed server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:51:33.816 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:33.801+0000 7f849a5f1640 1 --2- 172.21.15.39:0/2683243707 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f84800b6680 0x7f84800b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:51:33.816 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:33.801+0000 7f848affd640 1 -- 172.21.15.39:0/2683243707 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8484+0+0 (secure 0 0 0) 0x7f84840e71b0 con 0x7f84941094d0 2024-09-21T03:51:33.816 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:33.801+0000 7f847ffff640 1 -- 172.21.15.39:0/3858465680 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f849003f070 con 0x557c500ce1b0 2024-09-21T03:51:33.816 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:33.801+0000 7f847ffff640 1 -- 172.21.15.39:0/3858465680 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f84900316a0 con 0x557c500ce1b0 2024-09-21T03:51:33.816 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:33.801+0000 7f847ffff640 1 -- 172.21.15.39:0/3858465680 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8490035070 con 0x557c500ce1b0 2024-09-21T03:51:33.816 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:33.802+0000 7f84a25c8740 1 -- 172.21.15.39:0/3858465680 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557c500ce1b0 msgr2=0x557c5049eb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:51:33.816 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:33.802+0000 7f84a25c8740 1 --2- 172.21.15.39:0/3858465680 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557c500ce1b0 0x557c5049eb90 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 crypto rx=0x7f849002a930 tx=0x7f8490032e70 comp rx=0 tx=0).stop 2024-09-21T03:51:33.816 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:33.802+0000 7f849a5f1640 1 --2- 172.21.15.39:0/2683243707 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f84800b6680 0x7f84800b8b90 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f848c008e40 tx=0x7f848c0085f0 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:51:33.816 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:33.802+0000 7f84a25c8740 1 -- 172.21.15.39:0/3858465680 shutdown_connections 2024-09-21T03:51:33.816 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:33.802+0000 7f84a25c8740 1 --2- 172.21.15.39:0/3858465680 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557c500d6390 0x557c5049f0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:33.816 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:33.802+0000 7f84a25c8740 1 --2- 172.21.15.39:0/3858465680 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557c500ce1b0 0x557c5049eb90 unknown :-1 s=CLOSED pgs=905 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:33.816 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:33.802+0000 7f84a25c8740 1 --2- 172.21.15.39:0/3858465680 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557c500f7070 0x557c5049e650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:33.816 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:33.802+0000 7f84a25c8740 1 -- 172.21.15.39:0/3858465680 >> 172.21.15.39:0/3858465680 conn(0x557c500eb460 msgr2=0x557c5049c540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:51:33.816 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:33.802+0000 7f84a25c8740 1 -- 172.21.15.39:0/3858465680 shutdown_connections 2024-09-21T03:51:33.817 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:33.803+0000 7f84a25c8740 1 -- 172.21.15.39:0/3858465680 wait complete. 2024-09-21T03:51:33.817 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:33.803+0000 7f84a25c8740 1 -- start start 2024-09-21T03:51:33.817 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:33.803+0000 7f84a25c8740 1 Processor -- start 2024-09-21T03:51:33.817 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:33.803+0000 7f84a25c8740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557c500ce1b0 0x557c504a9cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:51:33.817 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:33.804+0000 7f84a25c8740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557c500d6390 0x557c504aa230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:51:33.817 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:33.804+0000 7f84a25c8740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557c500f7070 0x557c504aa7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:51:33.817 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:33.804+0000 7f84a25c8740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x557c50265700 con 0x557c500d6390 2024-09-21T03:51:33.817 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:33.804+0000 7f84a25c8740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x557c5039f110 con 0x557c500f7070 2024-09-21T03:51:33.817 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:33.804+0000 7f84a25c8740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x557c504acdd0 con 0x557c500ce1b0 2024-09-21T03:51:33.817 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:33.804+0000 7f84a25c8740 10 client.0 _ll_register_callbacks cb 0x557c503b8990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:51:33.817 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:33.804+0000 7f84995ef640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557c500d6390 0x557c504aa230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:51:33.817 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:33.804+0000 7f84995ef640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557c500d6390 0x557c504aa230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:40432/0 (socket says 172.21.15.39:40432) 2024-09-21T03:51:33.817 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:33.804+0000 7f84995ef640 1 -- 172.21.15.39:0/2250696023 learned_addr learned my addr 172.21.15.39:0/2250696023 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:51:33.817 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:33.804+0000 7f8499df0640 1 --2- 172.21.15.39:0/2250696023 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557c500ce1b0 0x557c504a9cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:51:33.817 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:33.804+0000 7f849a5f1640 1 --2- 172.21.15.39:0/2250696023 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557c500f7070 0x557c504aa7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:51:33.818 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:33.804+0000 7f84995ef640 1 -- 172.21.15.39:0/2250696023 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557c500ce1b0 msgr2=0x557c504a9cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:51:33.818 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:33.804+0000 7f84995ef640 1 --2- 172.21.15.39:0/2250696023 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557c500ce1b0 0x557c504a9cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:33.818 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:33.804+0000 7f84995ef640 1 -- 172.21.15.39:0/2250696023 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557c500f7070 msgr2=0x557c504aa7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:51:33.818 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:33.804+0000 7f84995ef640 1 --2- 172.21.15.39:0/2250696023 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557c500f7070 0x557c504aa7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:33.818 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:33.804+0000 7f84995ef640 1 -- 172.21.15.39:0/2250696023 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557c503ccc40 con 0x557c500d6390 2024-09-21T03:51:33.818 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:33.804+0000 7f84995ef640 1 --2- 172.21.15.39:0/2250696023 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557c500d6390 0x557c504aa230 secure :-1 s=READY pgs=906 cs=0 l=1 rev1=1 crypto rx=0x7f84840e4980 tx=0x7f84840a97a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=12b710ec1693fd11 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:51:33.818 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:33.804+0000 7f847d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-09-21T03:51:33.818 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:33.804+0000 7f847d7fa640 1 -- 172.21.15.39:0/2250696023 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8484021070 con 0x557c500d6390 2024-09-21T03:51:33.818 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:33.804+0000 7f847d7fa640 1 -- 172.21.15.39:0/2250696023 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8484+0+0 (secure 0 0 0) 0x7f84840a8970 con 0x557c500d6390 2024-09-21T03:51:33.818 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:33.804+0000 7f84a25c8740 1 -- 172.21.15.39:0/2250696023 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f84900316a0 con 0x557c500d6390 2024-09-21T03:51:33.818 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:33.805+0000 7f847d7fa640 1 -- 172.21.15.39:0/2250696023 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f84840ac970 con 0x557c500d6390 2024-09-21T03:51:33.818 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:33.805+0000 7f84a25c8740 10 client.16992 fetch_fsmap learned FSMap version 29 2024-09-21T03:51:33.819 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:33.805+0000 7f84a25c8740 1 -- 172.21.15.39:0/2250696023 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x557c504b8aa0 con 0x557c500d6390 2024-09-21T03:51:33.819 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:33.805+0000 7f847d7fa640 1 -- 172.21.15.39:0/2250696023 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f84900316a0 con 0x557c500d6390 2024-09-21T03:51:33.819 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:33.805+0000 7f847d7fa640 1 -- 172.21.15.39:0/2250696023 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f84840a8b70 con 0x557c500d6390 2024-09-21T03:51:33.819 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:33.805+0000 7f84a25c8740 10 client.16992 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T03:51:33.819 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:33.805+0000 7f84a25c8740 10 client.16992 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T03:51:33.819 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:33.805+0000 7f84a25c8740 20 client.16992 populate_metadata read hostname 'smithi039' 2024-09-21T03:51:33.819 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:33.805+0000 7f84a25c8740 1 --2- 172.21.15.39:0/2250696023 >> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] conn(0x557c500b59f0 0x557c504bbf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:51:33.819 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:33.805+0000 7f84a25c8740 4 client.16992 mds_command: new command op to 24695 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:51:33.819 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:33.805+0000 7f84a25c8740 1 -- 172.21.15.39:0/2250696023 --> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557c50416da0 con 0x557c500b59f0 2024-09-21T03:51:33.819 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:33.805+0000 7f8499df0640 1 --2- 172.21.15.39:0/2250696023 >> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] conn(0x557c500b59f0 0x557c504bbf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:51:33.819 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:33.806+0000 7f847d7fa640 10 client.16992 ms_handle_connect on v2:172.21.15.150:6838/2949386907 2024-09-21T03:51:33.819 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:33.806+0000 7f8499df0640 1 --2- 172.21.15.39:0/2250696023 >> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] conn(0x557c500b59f0 0x557c504bbf50 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dd0255dd14a09db8 server_cookie=3445d7b6d993b8e in_seq=0 out_seq=0 2024-09-21T03:51:33.819 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:33.807+0000 7f847d7fa640 1 -- 172.21.15.39:0/2250696023 <== mds.0 v2:172.21.15.150:6838/2949386907 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x557c50416da0 con 0x557c500b59f0 2024-09-21T03:51:33.819 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:33.807+0000 7f847d7fa640 10 client.16992 handle_command_reply: tid=0 2024-09-21T03:51:33.819 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:33.807+0000 7f84a25c8740 1 client.16992 shutdown 2024-09-21T03:51:33.820 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:33.807+0000 7f84a25c8740 1 -- 172.21.15.39:0/2250696023 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557c500d6390 msgr2=0x557c504aa230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:51:33.820 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:33.807+0000 7f84a25c8740 1 --2- 172.21.15.39:0/2250696023 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557c500d6390 0x557c504aa230 secure :-1 s=READY pgs=906 cs=0 l=1 rev1=1 crypto rx=0x7f84840e4980 tx=0x7f84840a97a0 comp rx=0 tx=0).stop 2024-09-21T03:51:33.820 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:33.807+0000 7f84a25c8740 1 -- 172.21.15.39:0/2250696023 shutdown_connections 2024-09-21T03:51:33.820 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:33.807+0000 7f84a25c8740 1 --2- 172.21.15.39:0/2250696023 >> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] conn(0x557c500b59f0 0x557c504bbf50 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:33.820 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:33.807+0000 7f84a25c8740 1 --2- 172.21.15.39:0/2250696023 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557c500f7070 0x557c504aa7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:33.820 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:33.807+0000 7f84a25c8740 1 --2- 172.21.15.39:0/2250696023 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557c500d6390 0x557c504aa230 unknown :-1 s=CLOSED pgs=906 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:33.820 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:33.807+0000 7f84a25c8740 1 --2- 172.21.15.39:0/2250696023 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557c500ce1b0 0x557c504a9cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:33.820 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:33.807+0000 7f847d7fa640 0 client.16992 ms_handle_reset on v2:172.21.15.150:6838/2949386907 2024-09-21T03:51:33.820 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:33.807+0000 7f84a25c8740 1 -- 172.21.15.39:0/2250696023 >> 172.21.15.39:0/2250696023 conn(0x557c500eb460 msgr2=0x557c504a17e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:51:33.820 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:33.807+0000 7f84a25c8740 1 -- 172.21.15.39:0/2250696023 shutdown_connections 2024-09-21T03:51:33.820 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:33.807+0000 7f84a25c8740 1 -- 172.21.15.39:0/2250696023 wait complete. 2024-09-21T03:51:33.820 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:33.807+0000 7f84a25c8740 20 client.16992 trim_cache size 0 max 16384 2024-09-21T03:51:33.827 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:33.824+0000 7f84a25c8740 1 Processor -- start 2024-09-21T03:51:33.827 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:33.825+0000 7f84a25c8740 1 -- start start 2024-09-21T03:51:33.828 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:33.825+0000 7f84a25c8740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557c500ce1b0 0x557c504a0040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:51:33.828 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:33.825+0000 7f84a25c8740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557c500d6390 0x557c504ab920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:51:33.828 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:33.825+0000 7f84a25c8740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557c500f7070 0x557c504a6ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:51:33.828 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:33.825+0000 7f84a25c8740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f84941b17f0 con 0x557c500f7070 2024-09-21T03:51:33.828 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:33.825+0000 7f84a25c8740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f84941b3330 con 0x557c500d6390 2024-09-21T03:51:33.828 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:33.825+0000 7f84a25c8740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f84941b1ec0 con 0x557c500ce1b0 2024-09-21T03:51:33.828 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:33.825+0000 7f849a5f1640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557c500f7070 0x557c504a6ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:51:33.828 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:33.825+0000 7f849a5f1640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557c500f7070 0x557c504a6ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:40434/0 (socket says 172.21.15.39:40434) 2024-09-21T03:51:33.828 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:33.825+0000 7f849a5f1640 1 -- 172.21.15.39:0/2385391059 learned_addr learned my addr 172.21.15.39:0/2385391059 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:51:33.828 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:33.825+0000 7f8499df0640 1 --2- 172.21.15.39:0/2385391059 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557c500ce1b0 0x557c504a0040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:51:33.828 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:33.825+0000 7f84995ef640 1 --2- 172.21.15.39:0/2385391059 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557c500d6390 0x557c504ab920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:51:33.828 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:33.825+0000 7f849a5f1640 1 -- 172.21.15.39:0/2385391059 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557c500ce1b0 msgr2=0x557c504a0040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:51:33.828 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:33.825+0000 7f849a5f1640 1 --2- 172.21.15.39:0/2385391059 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557c500ce1b0 0x557c504a0040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:33.829 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:33.825+0000 7f849a5f1640 1 -- 172.21.15.39:0/2385391059 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557c500d6390 msgr2=0x557c504ab920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:51:33.829 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:33.825+0000 7f849a5f1640 1 --2- 172.21.15.39:0/2385391059 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557c500d6390 0x557c504ab920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:33.829 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:33.825+0000 7f849a5f1640 1 -- 172.21.15.39:0/2385391059 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557c504a14b0 con 0x557c500f7070 2024-09-21T03:51:33.829 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:33.826+0000 7f849a5f1640 1 --2- 172.21.15.39:0/2385391059 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557c500f7070 0x557c504a6ee0 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 crypto rx=0x7f848c008c20 tx=0x7f848c014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=afc60919a5e96f5c server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:51:33.829 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:33.826+0000 7f847f7fe640 1 -- 172.21.15.39:0/2385391059 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f848c01c840 con 0x557c500f7070 2024-09-21T03:51:33.829 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:33.826+0000 7f847f7fe640 1 -- 172.21.15.39:0/2385391059 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f848c01ce90 con 0x557c500f7070 2024-09-21T03:51:33.829 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:33.826+0000 7f847f7fe640 1 -- 172.21.15.39:0/2385391059 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f848c00e3b0 con 0x557c500f7070 2024-09-21T03:51:33.833 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:33.827+0000 7f84a25c8740 1 -- 172.21.15.39:0/2385391059 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557c500f7070 msgr2=0x557c504a6ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:51:33.833 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:33.827+0000 7f84a25c8740 1 --2- 172.21.15.39:0/2385391059 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557c500f7070 0x557c504a6ee0 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 crypto rx=0x7f848c008c20 tx=0x7f848c014000 comp rx=0 tx=0).stop 2024-09-21T03:51:33.833 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:33.827+0000 7f84a25c8740 1 -- 172.21.15.39:0/2385391059 shutdown_connections 2024-09-21T03:51:33.833 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:33.827+0000 7f84a25c8740 1 --2- 172.21.15.39:0/2385391059 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557c500f7070 0x557c504a6ee0 unknown :-1 s=CLOSED pgs=907 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:33.834 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:33.827+0000 7f84a25c8740 1 --2- 172.21.15.39:0/2385391059 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557c500d6390 0x557c504ab920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:33.834 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:33.827+0000 7f84a25c8740 1 --2- 172.21.15.39:0/2385391059 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557c500ce1b0 0x557c504a0040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:33.834 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:33.827+0000 7f84a25c8740 1 -- 172.21.15.39:0/2385391059 >> 172.21.15.39:0/2385391059 conn(0x557c500eb460 msgr2=0x557c504ab210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:51:33.834 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:33.827+0000 7f84a25c8740 1 -- 172.21.15.39:0/2385391059 shutdown_connections 2024-09-21T03:51:33.834 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:33.827+0000 7f84a25c8740 1 -- 172.21.15.39:0/2385391059 wait complete. 2024-09-21T03:51:33.834 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:33.827+0000 7f84a25c8740 1 -- start start 2024-09-21T03:51:33.834 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:33.827+0000 7f84a25c8740 1 Processor -- start 2024-09-21T03:51:33.834 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:33.828+0000 7f84a25c8740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557c500ce1b0 0x557c504b2490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:51:33.834 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:33.828+0000 7f84a25c8740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557c500d6390 0x557c504b2a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:51:33.834 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:33.828+0000 7f84a25c8740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557c500f7070 0x557c504adad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:51:33.834 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:33.828+0000 7f84a25c8740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f84941b2c60 con 0x557c500d6390 2024-09-21T03:51:33.834 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:33.828+0000 7f84a25c8740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f84941b2590 con 0x557c500ce1b0 2024-09-21T03:51:33.834 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:33.828+0000 7f84a25c8740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f84941af7d0 con 0x557c500f7070 2024-09-21T03:51:33.835 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:33.828+0000 7f84a25c8740 10 client.0 _ll_register_callbacks cb 0x7f8484009f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:51:33.835 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:33.828+0000 7f84995ef640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557c500d6390 0x557c504b2a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:51:33.835 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:33.828+0000 7f84995ef640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557c500d6390 0x557c504b2a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:40450/0 (socket says 172.21.15.39:40450) 2024-09-21T03:51:33.835 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:33.828+0000 7f84995ef640 1 -- 172.21.15.39:0/3070967219 learned_addr learned my addr 172.21.15.39:0/3070967219 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:51:33.835 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:33.828+0000 7f849a5f1640 1 --2- 172.21.15.39:0/3070967219 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557c500f7070 0x557c504adad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:51:33.835 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:33.828+0000 7f8499df0640 1 --2- 172.21.15.39:0/3070967219 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557c500ce1b0 0x557c504b2490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:51:33.835 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:33.828+0000 7f84995ef640 1 -- 172.21.15.39:0/3070967219 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557c500f7070 msgr2=0x557c504adad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:51:33.835 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:33.828+0000 7f84995ef640 1 --2- 172.21.15.39:0/3070967219 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557c500f7070 0x557c504adad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:33.835 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:33.828+0000 7f84995ef640 1 -- 172.21.15.39:0/3070967219 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557c500ce1b0 msgr2=0x557c504b2490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:51:33.835 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:33.828+0000 7f84995ef640 1 --2- 172.21.15.39:0/3070967219 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557c500ce1b0 0x557c504b2490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:33.835 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:33.828+0000 7f84995ef640 1 -- 172.21.15.39:0/3070967219 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557c504ae260 con 0x557c500d6390 2024-09-21T03:51:33.835 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:33.828+0000 7f849a5f1640 1 --2- 172.21.15.39:0/3070967219 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557c500f7070 0x557c504adad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:51:33.835 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:33.828+0000 7f8499df0640 1 --2- 172.21.15.39:0/3070967219 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557c500ce1b0 0x557c504b2490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:51:33.835 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:33.828+0000 7f84995ef640 1 --2- 172.21.15.39:0/3070967219 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557c500d6390 0x557c504b2a50 secure :-1 s=READY pgs=908 cs=0 l=1 rev1=1 crypto rx=0x7f8484010c20 tx=0x7f84840ac1d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7dc5980ef7ef6a97 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:51:33.836 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:33.828+0000 7f847d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-09-21T03:51:33.836 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:33.828+0000 7f847d7fa640 1 -- 172.21.15.39:0/3070967219 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8484021040 con 0x557c500d6390 2024-09-21T03:51:33.836 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:33.829+0000 7f847d7fa640 1 -- 172.21.15.39:0/3070967219 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8484+0+0 (secure 0 0 0) 0x7f848405c1f0 con 0x557c500d6390 2024-09-21T03:51:33.836 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:33.829+0000 7f84a25c8740 1 -- 172.21.15.39:0/3070967219 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f848c01ce90 con 0x557c500d6390 2024-09-21T03:51:33.836 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:33.829+0000 7f847d7fa640 1 -- 172.21.15.39:0/3070967219 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f84840ac790 con 0x557c500d6390 2024-09-21T03:51:33.836 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:33.829+0000 7f84a25c8740 10 client.16998 fetch_fsmap learned FSMap version 29 2024-09-21T03:51:33.836 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:33.829+0000 7f84a25c8740 1 -- 172.21.15.39:0/3070967219 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x557c50526510 con 0x557c500d6390 2024-09-21T03:51:33.836 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:33.829+0000 7f847d7fa640 1 -- 172.21.15.39:0/3070967219 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f848c01ce90 con 0x557c500d6390 2024-09-21T03:51:33.836 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:33.829+0000 7f847d7fa640 1 -- 172.21.15.39:0/3070967219 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f84840b2c90 con 0x557c500d6390 2024-09-21T03:51:33.836 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:33.829+0000 7f84a25c8740 10 client.16998 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T03:51:33.836 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:33.829+0000 7f84a25c8740 10 client.16998 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T03:51:33.836 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:33.829+0000 7f84a25c8740 20 client.16998 populate_metadata read hostname 'smithi039' 2024-09-21T03:51:33.836 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:33.829+0000 7f84a25c8740 1 --2- 172.21.15.39:0/3070967219 >> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] conn(0x557c5008ada0 0x557c50528bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:51:33.836 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:33.829+0000 7f84a25c8740 4 client.16998 mds_command: new command op to 24695 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-21T03:51:33.837 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:33.829+0000 7f84a25c8740 1 -- 172.21.15.39:0/3070967219 --> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] -- command(tid 0: {"prefix": "scrub status"}) -- 0x557c50412850 con 0x557c5008ada0 2024-09-21T03:51:33.837 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:33.829+0000 7f8499df0640 1 --2- 172.21.15.39:0/3070967219 >> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] conn(0x557c5008ada0 0x557c50528bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:51:33.837 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:33.830+0000 7f8499df0640 1 --2- 172.21.15.39:0/3070967219 >> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] conn(0x557c5008ada0 0x557c50528bf0 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1dfbb65df9153194 server_cookie=d982adec3ec9bb77 in_seq=0 out_seq=0 2024-09-21T03:51:33.837 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:33.830+0000 7f847d7fa640 10 client.16998 ms_handle_connect on v2:172.21.15.150:6838/2949386907 2024-09-21T03:51:33.837 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:33.830+0000 7f847d7fa640 1 -- 172.21.15.39:0/3070967219 <== mds.0 v2:172.21.15.150:6838/2949386907 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3410 (crc 0 0 0) 0x557c50412850 con 0x557c5008ada0 2024-09-21T03:51:33.837 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:33.830+0000 7f847d7fa640 10 client.16998 handle_command_reply: tid=0 2024-09-21T03:51:33.837 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:33.830+0000 7f84a25c8740 1 client.16998 shutdown 2024-09-21T03:51:33.837 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:33.830+0000 7f84a25c8740 1 -- 172.21.15.39:0/3070967219 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557c500d6390 msgr2=0x557c504b2a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:51:33.837 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:33.830+0000 7f84a25c8740 1 --2- 172.21.15.39:0/3070967219 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557c500d6390 0x557c504b2a50 secure :-1 s=READY pgs=908 cs=0 l=1 rev1=1 crypto rx=0x7f8484010c20 tx=0x7f84840ac1d0 comp rx=0 tx=0).stop 2024-09-21T03:51:33.837 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:33.830+0000 7f84a25c8740 1 -- 172.21.15.39:0/3070967219 shutdown_connections 2024-09-21T03:51:33.837 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:33.830+0000 7f84a25c8740 1 --2- 172.21.15.39:0/3070967219 >> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] conn(0x557c5008ada0 0x557c50528bf0 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:33.837 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:33.830+0000 7f84a25c8740 1 --2- 172.21.15.39:0/3070967219 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557c500f7070 0x557c504adad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:33.837 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:33.830+0000 7f847d7fa640 0 client.16998 ms_handle_reset on v2:172.21.15.150:6838/2949386907 2024-09-21T03:51:33.837 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:33.830+0000 7f84a25c8740 1 --2- 172.21.15.39:0/3070967219 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557c500d6390 0x557c504b2a50 unknown :-1 s=CLOSED pgs=908 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:33.837 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:33.830+0000 7f84a25c8740 1 --2- 172.21.15.39:0/3070967219 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557c500ce1b0 0x557c504b2490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:33.837 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:33.830+0000 7f84a25c8740 1 -- 172.21.15.39:0/3070967219 >> 172.21.15.39:0/3070967219 conn(0x557c500eb460 msgr2=0x557c5049fe10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:51:33.838 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:33.830+0000 7f84a25c8740 1 -- 172.21.15.39:0/3070967219 shutdown_connections 2024-09-21T03:51:33.838 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:33.830+0000 7f84a25c8740 1 -- 172.21.15.39:0/3070967219 wait complete. 2024-09-21T03:51:33.838 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:33.831+0000 7f84a25c8740 20 client.16998 trim_cache size 0 max 16384 2024-09-21T03:51:33.838 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-09-21T03:51:33.838 INFO:teuthology.orchestra.run.smithi039.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-21T03:51:33.838 INFO:teuthology.orchestra.run.smithi039.stdout: "scrubs": { 2024-09-21T03:51:33.838 INFO:teuthology.orchestra.run.smithi039.stdout: "ede73e6e-3fd0-4a7f-b5bb-f68946f57eea": { 2024-09-21T03:51:33.838 INFO:teuthology.orchestra.run.smithi039.stdout: "path": "/", 2024-09-21T03:51:33.838 INFO:teuthology.orchestra.run.smithi039.stdout: "tag": "ede73e6e-3fd0-4a7f-b5bb-f68946f57eea", 2024-09-21T03:51:33.838 INFO:teuthology.orchestra.run.smithi039.stdout: "options": "recursive,force" 2024-09-21T03:51:33.838 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-21T03:51:33.839 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:51:33.839 INFO:teuthology.orchestra.run.smithi039.stdout: "scrub_stats": { 2024-09-21T03:51:33.839 INFO:teuthology.orchestra.run.smithi039.stdout: "ae1fc4cf-5458-47d0-afcb-282b136dba15": { 2024-09-21T03:51:33.839 INFO:teuthology.orchestra.run.smithi039.stdout: "start_time": "2024-09-21T03:51:18.560378+0000", 2024-09-21T03:51:33.839 INFO:teuthology.orchestra.run.smithi039.stdout: "path": "/", 2024-09-21T03:51:33.839 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_started": 0, 2024-09-21T03:51:33.839 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_passed": 0, 2024-09-21T03:51:33.839 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_failed": 0, 2024-09-21T03:51:33.839 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_skipped": 0 2024-09-21T03:51:33.839 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:51:33.839 INFO:teuthology.orchestra.run.smithi039.stdout: "160750d8-401f-4810-866d-e63e7e4bc3e0": { 2024-09-21T03:51:33.839 INFO:teuthology.orchestra.run.smithi039.stdout: "start_time": "2024-09-21T03:51:04.208673+0000", 2024-09-21T03:51:33.839 INFO:teuthology.orchestra.run.smithi039.stdout: "path": "/", 2024-09-21T03:51:33.839 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_started": 0, 2024-09-21T03:51:33.839 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_passed": 0, 2024-09-21T03:51:33.840 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_failed": 0, 2024-09-21T03:51:33.840 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_skipped": 0 2024-09-21T03:51:33.840 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:51:33.840 INFO:teuthology.orchestra.run.smithi039.stdout: "74ab917e-ecea-4778-a1c3-c184214fb24c": { 2024-09-21T03:51:33.840 INFO:teuthology.orchestra.run.smithi039.stdout: "start_time": "2024-09-21T03:50:17.726630+0000", 2024-09-21T03:51:33.840 INFO:teuthology.orchestra.run.smithi039.stdout: "path": "/", 2024-09-21T03:51:33.840 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_started": 0, 2024-09-21T03:51:33.840 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_passed": 0, 2024-09-21T03:51:33.840 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_failed": 0, 2024-09-21T03:51:33.840 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_skipped": 0 2024-09-21T03:51:33.840 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:51:33.840 INFO:teuthology.orchestra.run.smithi039.stdout: "6e760f1f-9048-4f92-bd66-d708fbef75b3": { 2024-09-21T03:51:33.840 INFO:teuthology.orchestra.run.smithi039.stdout: "start_time": "2024-09-21T03:50:03.711918+0000", 2024-09-21T03:51:33.840 INFO:teuthology.orchestra.run.smithi039.stdout: "path": "/", 2024-09-21T03:51:33.840 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_started": 0, 2024-09-21T03:51:33.841 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_passed": 0, 2024-09-21T03:51:33.841 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_failed": 0, 2024-09-21T03:51:33.841 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_skipped": 0 2024-09-21T03:51:33.841 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:51:33.841 INFO:teuthology.orchestra.run.smithi039.stdout: "e28ad743-2b22-45b3-8799-a638c2883623": { 2024-09-21T03:51:33.841 INFO:teuthology.orchestra.run.smithi039.stdout: "start_time": "2024-09-21T03:48:31.448821+0000", 2024-09-21T03:51:33.841 INFO:teuthology.orchestra.run.smithi039.stdout: "path": "/", 2024-09-21T03:51:33.841 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_started": 0, 2024-09-21T03:51:33.841 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_passed": 0, 2024-09-21T03:51:33.841 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_failed": 0, 2024-09-21T03:51:33.841 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_skipped": 0 2024-09-21T03:51:33.841 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:51:33.841 INFO:teuthology.orchestra.run.smithi039.stdout: "ede73e6e-3fd0-4a7f-b5bb-f68946f57eea": { 2024-09-21T03:51:33.841 INFO:teuthology.orchestra.run.smithi039.stdout: "start_time": "2024-09-21T03:51:32.235096+0000", 2024-09-21T03:51:33.841 INFO:teuthology.orchestra.run.smithi039.stdout: "path": "/", 2024-09-21T03:51:33.842 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_started": 0, 2024-09-21T03:51:33.842 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_passed": 0, 2024-09-21T03:51:33.842 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_failed": 0, 2024-09-21T03:51:33.842 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_skipped": 0 2024-09-21T03:51:33.842 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:51:33.842 INFO:teuthology.orchestra.run.smithi039.stdout: "29411307-eb8d-4339-8d2b-47ebd32c0080": { 2024-09-21T03:51:33.842 INFO:teuthology.orchestra.run.smithi039.stdout: "start_time": "2024-09-21T03:47:38.092920+0000", 2024-09-21T03:51:33.842 INFO:teuthology.orchestra.run.smithi039.stdout: "path": "/", 2024-09-21T03:51:33.842 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_started": 0, 2024-09-21T03:51:33.842 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_passed": 0, 2024-09-21T03:51:33.842 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_failed": 0, 2024-09-21T03:51:33.842 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_skipped": 0 2024-09-21T03:51:33.842 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:51:33.842 INFO:teuthology.orchestra.run.smithi039.stdout: "9860c2f4-b7b1-47aa-865a-9a1c4434a5f6": { 2024-09-21T03:51:33.842 INFO:teuthology.orchestra.run.smithi039.stdout: "start_time": "2024-09-21T03:49:17.718867+0000", 2024-09-21T03:51:33.843 INFO:teuthology.orchestra.run.smithi039.stdout: "path": "/", 2024-09-21T03:51:33.843 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_started": 0, 2024-09-21T03:51:33.843 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_passed": 0, 2024-09-21T03:51:33.843 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_failed": 0, 2024-09-21T03:51:33.843 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_skipped": 0 2024-09-21T03:51:33.843 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:51:33.843 INFO:teuthology.orchestra.run.smithi039.stdout: "c7e2b8e4-057d-435f-b2bd-b633de98035f": { 2024-09-21T03:51:33.843 INFO:teuthology.orchestra.run.smithi039.stdout: "start_time": "2024-09-21T03:47:19.400305+0000", 2024-09-21T03:51:33.843 INFO:teuthology.orchestra.run.smithi039.stdout: "path": "/", 2024-09-21T03:51:33.843 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_started": 0, 2024-09-21T03:51:33.843 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_passed": 0, 2024-09-21T03:51:33.843 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_failed": 0, 2024-09-21T03:51:33.843 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_skipped": 0 2024-09-21T03:51:33.843 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:51:33.843 INFO:teuthology.orchestra.run.smithi039.stdout: "10728de8-c49d-4edc-be2b-639249255e94": { 2024-09-21T03:51:33.844 INFO:teuthology.orchestra.run.smithi039.stdout: "start_time": "2024-09-21T03:47:03.965623+0000", 2024-09-21T03:51:33.844 INFO:teuthology.orchestra.run.smithi039.stdout: "path": "/", 2024-09-21T03:51:33.844 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_started": 0, 2024-09-21T03:51:33.844 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_passed": 0, 2024-09-21T03:51:33.844 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_failed": 0, 2024-09-21T03:51:33.844 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_skipped": 0 2024-09-21T03:51:33.844 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:51:33.844 INFO:teuthology.orchestra.run.smithi039.stdout: "02509e4f-3911-4983-b16f-df8ded0b9998": { 2024-09-21T03:51:33.844 INFO:teuthology.orchestra.run.smithi039.stdout: "start_time": "2024-09-21T03:46:17.889576+0000", 2024-09-21T03:51:33.844 INFO:teuthology.orchestra.run.smithi039.stdout: "path": "/", 2024-09-21T03:51:33.844 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_started": 0, 2024-09-21T03:51:33.844 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_passed": 0, 2024-09-21T03:51:33.844 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_failed": 0, 2024-09-21T03:51:33.844 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_skipped": 0 2024-09-21T03:51:33.844 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-21T03:51:33.845 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-21T03:51:33.845 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-09-21T03:51:33.845 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:33.831+0000 7f849c054640 1 -- 172.21.15.39:0/2683243707 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f84800b6680 msgr2=0x7f84800b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:51:33.845 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:33.831+0000 7f849c054640 1 --2- 172.21.15.39:0/2683243707 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f84800b6680 0x7f84800b8b90 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f848c008e40 tx=0x7f848c0085f0 comp rx=0 tx=0).stop 2024-09-21T03:51:33.845 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:33.831+0000 7f849c054640 1 -- 172.21.15.39:0/2683243707 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f84941094d0 msgr2=0x7f8494075c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:51:33.846 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:33.831+0000 7f849c054640 1 --2- 172.21.15.39:0/2683243707 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f84941094d0 0x7f8494075c50 secure :-1 s=READY pgs=904 cs=0 l=1 rev1=1 crypto rx=0x7f848400d740 tx=0x7f848400dc00 comp rx=0 tx=0).stop 2024-09-21T03:51:33.846 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:33.831+0000 7f849c054640 1 -- 172.21.15.39:0/2683243707 shutdown_connections 2024-09-21T03:51:33.846 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:33.831+0000 7f849c054640 1 --2- 172.21.15.39:0/2683243707 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f84800b6680 0x7f84800b8b90 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:33.846 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:33.831+0000 7f849c054640 1 --2- 172.21.15.39:0/2683243707 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8494109e20 0x7f8494076190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:33.846 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:33.831+0000 7f849c054640 1 --2- 172.21.15.39:0/2683243707 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f84941094d0 0x7f8494075c50 unknown :-1 s=CLOSED pgs=904 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:33.846 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:33.831+0000 7f849c054640 1 --2- 172.21.15.39:0/2683243707 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8494103b90 0x7f8494075710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:33.846 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:33.831+0000 7f849c054640 1 -- 172.21.15.39:0/2683243707 >> 172.21.15.39:0/2683243707 conn(0x7f84940ff600 msgr2=0x7f8494101a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:51:33.846 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:33.831+0000 7f849c054640 1 -- 172.21.15.39:0/2683243707 shutdown_connections 2024-09-21T03:51:33.846 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:51:33.831+0000 7f849c054640 1 -- 172.21.15.39:0/2683243707 wait complete. 2024-09-21T03:51:34.636 INFO:tasks.cephfs.filesystem:scrub status for tag:ede73e6e-3fd0-4a7f-b5bb-f68946f57eea - {'path': '/', 'tag': 'ede73e6e-3fd0-4a7f-b5bb-f68946f57eea', 'options': 'recursive,force'} 2024-09-21T03:51:35.468 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:51:35 smithi161 ceph-mon[34533]: pgmap v491: 97 pgs: 97 active+clean; 10 MiB data, 520 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 10 op/s 2024-09-21T03:51:35.468 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:51:35 smithi161 ceph-mon[34533]: scrub summary: idle 2024-09-21T03:51:35.474 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:51:35 smithi150 ceph-mon[34289]: pgmap v491: 97 pgs: 97 active+clean; 10 MiB data, 520 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 10 op/s 2024-09-21T03:51:35.475 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:51:35 smithi150 ceph-mon[34289]: scrub summary: idle 2024-09-21T03:51:35.584 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:51:35 smithi039 ceph-mon[31211]: pgmap v491: 97 pgs: 97 active+clean; 10 MiB data, 520 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 10 op/s 2024-09-21T03:51:35.584 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:51:35 smithi039 ceph-mon[31211]: scrub summary: idle 2024-09-21T03:51:37.468 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:51:37 smithi161 ceph-mon[34533]: pgmap v492: 97 pgs: 97 active+clean; 10 MiB data, 520 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 10 op/s 2024-09-21T03:51:37.474 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:51:37 smithi150 ceph-mon[34289]: pgmap v492: 97 pgs: 97 active+clean; 10 MiB data, 520 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 10 op/s 2024-09-21T03:51:37.584 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:51:37 smithi039 ceph-mon[31211]: pgmap v492: 97 pgs: 97 active+clean; 10 MiB data, 520 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 10 op/s 2024-09-21T03:51:39.468 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:51:39 smithi161 ceph-mon[34533]: pgmap v493: 97 pgs: 97 active+clean; 10 MiB data, 520 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 8 op/s 2024-09-21T03:51:39.475 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:51:39 smithi150 ceph-mon[34289]: pgmap v493: 97 pgs: 97 active+clean; 10 MiB data, 520 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 8 op/s 2024-09-21T03:51:39.584 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:51:39 smithi039 ceph-mon[31211]: pgmap v493: 97 pgs: 97 active+clean; 10 MiB data, 520 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 8 op/s 2024-09-21T03:51:41.468 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:51:41 smithi161 ceph-mon[34533]: pgmap v494: 97 pgs: 97 active+clean; 10 MiB data, 520 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 11 op/s 2024-09-21T03:51:41.474 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:51:41 smithi150 ceph-mon[34289]: pgmap v494: 97 pgs: 97 active+clean; 10 MiB data, 520 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 11 op/s 2024-09-21T03:51:41.584 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:51:41 smithi039 ceph-mon[31211]: pgmap v494: 97 pgs: 97 active+clean; 10 MiB data, 520 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 11 op/s 2024-09-21T03:51:43.400 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:51:43 smithi039 ceph-mon[31211]: pgmap v495: 97 pgs: 97 active+clean; 10 MiB data, 520 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 9 op/s 2024-09-21T03:51:43.468 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:51:43 smithi161 ceph-mon[34533]: pgmap v495: 97 pgs: 97 active+clean; 10 MiB data, 520 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 9 op/s 2024-09-21T03:51:43.475 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:51:43 smithi150 ceph-mon[34289]: pgmap v495: 97 pgs: 97 active+clean; 10 MiB data, 520 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 9 op/s 2024-09-21T03:51:45.468 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:51:45 smithi161 ceph-mon[34533]: pgmap v496: 97 pgs: 97 active+clean; 10 MiB data, 520 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 10 op/s 2024-09-21T03:51:45.475 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:51:45 smithi150 ceph-mon[34289]: pgmap v496: 97 pgs: 97 active+clean; 10 MiB data, 520 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 10 op/s 2024-09-21T03:51:45.584 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:51:45 smithi039 ceph-mon[31211]: pgmap v496: 97 pgs: 97 active+clean; 10 MiB data, 520 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 10 op/s 2024-09-21T03:51:47.468 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:51:47 smithi161 ceph-mon[34533]: pgmap v497: 97 pgs: 97 active+clean; 10 MiB data, 520 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 10 op/s 2024-09-21T03:51:47.475 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:51:47 smithi150 ceph-mon[34289]: pgmap v497: 97 pgs: 97 active+clean; 10 MiB data, 520 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 10 op/s 2024-09-21T03:51:47.584 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:51:47 smithi039 ceph-mon[31211]: pgmap v497: 97 pgs: 97 active+clean; 10 MiB data, 520 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 10 op/s 2024-09-21T03:51:49.468 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:51:49 smithi161 ceph-mon[34533]: pgmap v498: 97 pgs: 97 active+clean; 10 MiB data, 520 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-09-21T03:51:49.475 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:51:49 smithi150 ceph-mon[34289]: pgmap v498: 97 pgs: 97 active+clean; 10 MiB data, 520 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-09-21T03:51:49.584 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:51:49 smithi039 ceph-mon[31211]: pgmap v498: 97 pgs: 97 active+clean; 10 MiB data, 520 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-09-21T03:51:51.468 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:51:51 smithi161 ceph-mon[34533]: pgmap v499: 97 pgs: 97 active+clean; 10 MiB data, 520 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-09-21T03:51:51.475 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:51:51 smithi150 ceph-mon[34289]: pgmap v499: 97 pgs: 97 active+clean; 10 MiB data, 520 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-09-21T03:51:51.584 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:51:51 smithi039 ceph-mon[31211]: pgmap v499: 97 pgs: 97 active+clean; 10 MiB data, 520 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-09-21T03:51:53.468 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:51:53 smithi161 ceph-mon[34533]: pgmap v500: 97 pgs: 97 active+clean; 10 MiB data, 520 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-21T03:51:53.474 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:51:53 smithi150 ceph-mon[34289]: pgmap v500: 97 pgs: 97 active+clean; 10 MiB data, 520 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-21T03:51:53.584 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:51:53 smithi039 ceph-mon[31211]: pgmap v500: 97 pgs: 97 active+clean; 10 MiB data, 520 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-21T03:51:55.468 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:51:55 smithi161 ceph-mon[34533]: pgmap v501: 97 pgs: 97 active+clean; 10 MiB data, 520 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-09-21T03:51:55.474 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:51:55 smithi150 ceph-mon[34289]: pgmap v501: 97 pgs: 97 active+clean; 10 MiB data, 520 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-09-21T03:51:55.584 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:51:55 smithi039 ceph-mon[31211]: pgmap v501: 97 pgs: 97 active+clean; 10 MiB data, 520 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-09-21T03:51:57.468 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:51:57 smithi161 ceph-mon[34533]: pgmap v502: 97 pgs: 97 active+clean; 10 MiB data, 520 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 10 op/s 2024-09-21T03:51:57.475 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:51:57 smithi150 ceph-mon[34289]: pgmap v502: 97 pgs: 97 active+clean; 10 MiB data, 520 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 10 op/s 2024-09-21T03:51:57.584 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:51:57 smithi039 ceph-mon[31211]: pgmap v502: 97 pgs: 97 active+clean; 10 MiB data, 520 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 10 op/s 2024-09-21T03:51:59.468 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:51:59 smithi161 ceph-mon[34533]: pgmap v503: 97 pgs: 97 active+clean; 10 MiB data, 520 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-09-21T03:51:59.474 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:51:59 smithi150 ceph-mon[34289]: pgmap v503: 97 pgs: 97 active+clean; 10 MiB data, 520 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-09-21T03:51:59.584 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:51:59 smithi039 ceph-mon[31211]: pgmap v503: 97 pgs: 97 active+clean; 10 MiB data, 520 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-09-21T03:52:01.468 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:52:01 smithi161 ceph-mon[34533]: pgmap v504: 97 pgs: 97 active+clean; 10 MiB data, 520 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-09-21T03:52:01.474 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:52:01 smithi150 ceph-mon[34289]: pgmap v504: 97 pgs: 97 active+clean; 10 MiB data, 520 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-09-21T03:52:01.584 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:52:01 smithi039 ceph-mon[31211]: pgmap v504: 97 pgs: 97 active+clean; 10 MiB data, 520 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-09-21T03:52:03.469 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:52:03 smithi161 ceph-mon[34533]: pgmap v505: 97 pgs: 97 active+clean; 10 MiB data, 520 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-21T03:52:03.607 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:52:03 smithi039 ceph-mon[31211]: pgmap v505: 97 pgs: 97 active+clean; 10 MiB data, 520 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-21T03:52:03.607 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:52:03 smithi150 ceph-mon[34289]: pgmap v505: 97 pgs: 97 active+clean; 10 MiB data, 520 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-21T03:52:04.637 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 51116d5c-77ca-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub status 2024-09-21T03:52:04.913 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/51116d5c-77ca-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:52:05.334 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:52:05 smithi039 ceph-mon[31211]: pgmap v506: 97 pgs: 97 active+clean; 10 MiB data, 520 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 426 B/s wr, 10 op/s 2024-09-21T03:52:05.467 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:52:05 smithi161 ceph-mon[34533]: pgmap v506: 97 pgs: 97 active+clean; 10 MiB data, 520 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 426 B/s wr, 10 op/s 2024-09-21T03:52:05.474 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:52:05 smithi150 ceph-mon[34289]: pgmap v506: 97 pgs: 97 active+clean; 10 MiB data, 520 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 426 B/s wr, 10 op/s 2024-09-21T03:52:05.536 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:05.506+0000 7fc358cbe640 1 -- 172.21.15.39:0/1538655065 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc354105300 msgr2=0x7fc35410d850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:52:05.536 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:05.506+0000 7fc358cbe640 1 --2- 172.21.15.39:0/1538655065 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc354105300 0x7fc35410d850 secure :-1 s=READY pgs=909 cs=0 l=1 rev1=1 crypto rx=0x7fc33c009900 tx=0x7fc33c02a410 comp rx=0 tx=0).stop 2024-09-21T03:52:05.536 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:05.506+0000 7fc358cbe640 1 -- 172.21.15.39:0/1538655065 shutdown_connections 2024-09-21T03:52:05.536 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:05.506+0000 7fc358cbe640 1 --2- 172.21.15.39:0/1538655065 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc35410de10 0x7fc354110300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:05.536 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:05.506+0000 7fc358cbe640 1 --2- 172.21.15.39:0/1538655065 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc354105300 0x7fc35410d850 unknown :-1 s=CLOSED pgs=909 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:05.536 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:05.506+0000 7fc358cbe640 1 --2- 172.21.15.39:0/1538655065 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc3541049b0 0x7fc354104dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:05.537 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:05.506+0000 7fc358cbe640 1 -- 172.21.15.39:0/1538655065 >> 172.21.15.39:0/1538655065 conn(0x7fc354079730 msgr2=0x7fc354079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:52:05.537 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:05.506+0000 7fc358cbe640 1 -- 172.21.15.39:0/1538655065 shutdown_connections 2024-09-21T03:52:05.537 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:05.506+0000 7fc358cbe640 1 -- 172.21.15.39:0/1538655065 wait complete. 2024-09-21T03:52:05.537 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:05.507+0000 7fc358cbe640 1 Processor -- start 2024-09-21T03:52:05.537 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:05.507+0000 7fc358cbe640 1 -- start start 2024-09-21T03:52:05.537 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:05.508+0000 7fc358cbe640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc3541049b0 0x7fc35419a540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:52:05.537 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:05.508+0000 7fc358cbe640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc354105300 0x7fc35419aae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:52:05.537 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:05.508+0000 7fc358cbe640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc35410de10 0x7fc35419fd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:52:05.537 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:05.508+0000 7fc358cbe640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fc354113a70 con 0x7fc354105300 2024-09-21T03:52:05.537 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:05.508+0000 7fc358cbe640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fc3541138f0 con 0x7fc35410de10 2024-09-21T03:52:05.537 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:05.508+0000 7fc358cbe640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fc354113bf0 con 0x7fc3541049b0 2024-09-21T03:52:05.537 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:05.508+0000 7fc351d9b640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc354105300 0x7fc35419aae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:52:05.537 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:05.508+0000 7fc351d9b640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc354105300 0x7fc35419aae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:60802/0 (socket says 172.21.15.39:60802) 2024-09-21T03:52:05.537 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:05.508+0000 7fc351d9b640 1 -- 172.21.15.39:0/721333096 learned_addr learned my addr 172.21.15.39:0/721333096 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:52:05.538 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:05.508+0000 7fc35259c640 1 --2- 172.21.15.39:0/721333096 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc3541049b0 0x7fc35419a540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:52:05.538 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:05.508+0000 7fc352d9d640 1 --2- 172.21.15.39:0/721333096 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc35410de10 0x7fc35419fd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:52:05.538 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:05.508+0000 7fc351d9b640 1 -- 172.21.15.39:0/721333096 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc3541049b0 msgr2=0x7fc35419a540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:52:05.538 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:05.508+0000 7fc351d9b640 1 --2- 172.21.15.39:0/721333096 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc3541049b0 0x7fc35419a540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:05.538 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:05.508+0000 7fc351d9b640 1 -- 172.21.15.39:0/721333096 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc35410de10 msgr2=0x7fc35419fd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:52:05.538 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:05.508+0000 7fc351d9b640 1 --2- 172.21.15.39:0/721333096 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc35410de10 0x7fc35419fd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:05.538 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:05.508+0000 7fc351d9b640 1 -- 172.21.15.39:0/721333096 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc3541a04b0 con 0x7fc354105300 2024-09-21T03:52:05.538 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:05.509+0000 7fc351d9b640 1 --2- 172.21.15.39:0/721333096 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc354105300 0x7fc35419aae0 secure :-1 s=READY pgs=910 cs=0 l=1 rev1=1 crypto rx=0x7fc33c02a930 tx=0x7fc33c004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:52:05.538 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:05.509+0000 7fc34b7fe640 1 -- 172.21.15.39:0/721333096 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc33c03f030 con 0x7fc354105300 2024-09-21T03:52:05.538 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:05.509+0000 7fc358cbe640 1 -- 172.21.15.39:0/721333096 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc3541a0740 con 0x7fc354105300 2024-09-21T03:52:05.538 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:05.509+0000 7fc34b7fe640 1 -- 172.21.15.39:0/721333096 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc33c003aa0 con 0x7fc354105300 2024-09-21T03:52:05.538 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:05.509+0000 7fc34b7fe640 1 -- 172.21.15.39:0/721333096 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc33c035070 con 0x7fc354105300 2024-09-21T03:52:05.538 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:05.509+0000 7fc358cbe640 1 -- 172.21.15.39:0/721333096 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc3541a6f00 con 0x7fc354105300 2024-09-21T03:52:05.538 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:05.511+0000 7fc35f232740 1 Processor -- start 2024-09-21T03:52:05.539 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:05.512+0000 7fc35f232740 1 -- start start 2024-09-21T03:52:05.539 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:05.512+0000 7fc35f232740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a658559070 0x55a658900650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:52:05.539 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:05.512+0000 7fc35f232740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a6585301b0 0x55a658900b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:52:05.539 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:05.512+0000 7fc35f232740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a658538390 0x55a6589010d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:52:05.539 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:05.512+0000 7fc35f232740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x55a658876e50 con 0x55a658559070 2024-09-21T03:52:05.539 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:05.512+0000 7fc35f232740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x55a658866280 con 0x55a6585301b0 2024-09-21T03:52:05.539 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:05.512+0000 7fc35f232740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55a65886a090 con 0x55a658538390 2024-09-21T03:52:05.539 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:05.512+0000 7fc34b7fe640 1 -- 172.21.15.39:0/721333096 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 27) ==== 154643+0+0 (secure 0 0 0) 0x7fc33c042030 con 0x7fc354105300 2024-09-21T03:52:05.539 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:05.512+0000 7fc352d9d640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a658559070 0x55a658900650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:52:05.539 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:05.512+0000 7fc352d9d640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a658559070 0x55a658900650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:60806/0 (socket says 172.21.15.39:60806) 2024-09-21T03:52:05.540 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:05.512+0000 7fc352d9d640 1 -- 172.21.15.39:0/1419058403 learned_addr learned my addr 172.21.15.39:0/1419058403 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:52:05.540 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:05.512+0000 7fc35259c640 1 --2- 172.21.15.39:0/1419058403 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a6585301b0 0x55a658900b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:52:05.540 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:05.512+0000 7fc351d9b640 1 --2- 172.21.15.39:0/1419058403 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a658538390 0x55a6589010d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:52:05.540 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:05.512+0000 7fc352d9d640 1 -- 172.21.15.39:0/1419058403 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a658538390 msgr2=0x55a6589010d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:52:05.540 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:05.512+0000 7fc352d9d640 1 --2- 172.21.15.39:0/1419058403 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a658538390 0x55a6589010d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:05.540 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:05.512+0000 7fc352d9d640 1 -- 172.21.15.39:0/1419058403 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a6585301b0 msgr2=0x55a658900b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:52:05.540 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:05.512+0000 7fc352d9d640 1 --2- 172.21.15.39:0/1419058403 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a6585301b0 0x55a658900b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:05.540 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:05.512+0000 7fc352d9d640 1 -- 172.21.15.39:0/1419058403 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a6585c4650 con 0x55a658559070 2024-09-21T03:52:05.540 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:05.512+0000 7fc351d9b640 1 --2- 172.21.15.39:0/1419058403 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a658538390 0x55a6589010d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:52:05.540 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:05.513+0000 7fc352d9d640 1 --2- 172.21.15.39:0/1419058403 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a658559070 0x55a658900650 secure :-1 s=READY pgs=911 cs=0 l=1 rev1=1 crypto rx=0x7fc344009f70 tx=0x7fc344010800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=14f2fb6164dfe5d3 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:52:05.541 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:05.513+0000 7fc348ff9640 1 -- 172.21.15.39:0/1419058403 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc344016070 con 0x55a658559070 2024-09-21T03:52:05.541 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:05.513+0000 7fc348ff9640 1 -- 172.21.15.39:0/1419058403 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc344004d20 con 0x55a658559070 2024-09-21T03:52:05.541 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:05.513+0000 7fc348ff9640 1 -- 172.21.15.39:0/1419058403 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3440127d0 con 0x55a658559070 2024-09-21T03:52:05.541 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:05.513+0000 7fc34b7fe640 1 --2- 172.21.15.39:0/721333096 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7fc3400b6680 0x7fc3400b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:52:05.541 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:05.513+0000 7fc35259c640 1 --2- 172.21.15.39:0/721333096 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7fc3400b6680 0x7fc3400b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:52:05.541 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:05.513+0000 7fc34b7fe640 1 -- 172.21.15.39:0/721333096 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8484+0+0 (secure 0 0 0) 0x7fc33c107d70 con 0x7fc354105300 2024-09-21T03:52:05.541 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:05.514+0000 7fc35f232740 1 -- 172.21.15.39:0/1419058403 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a658559070 msgr2=0x55a658900650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:52:05.541 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:05.514+0000 7fc35259c640 1 --2- 172.21.15.39:0/721333096 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7fc3400b6680 0x7fc3400b8b90 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x55a6588ff2d0 tx=0x7fc34c00b410 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:52:05.541 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:05.514+0000 7fc35f232740 1 --2- 172.21.15.39:0/1419058403 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a658559070 0x55a658900650 secure :-1 s=READY pgs=911 cs=0 l=1 rev1=1 crypto rx=0x7fc344009f70 tx=0x7fc344010800 comp rx=0 tx=0).stop 2024-09-21T03:52:05.541 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:05.514+0000 7fc35f232740 1 -- 172.21.15.39:0/1419058403 shutdown_connections 2024-09-21T03:52:05.541 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:05.514+0000 7fc35f232740 1 --2- 172.21.15.39:0/1419058403 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a658538390 0x55a6589010d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:05.541 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:05.514+0000 7fc35f232740 1 --2- 172.21.15.39:0/1419058403 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a6585301b0 0x55a658900b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:05.541 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:05.514+0000 7fc35f232740 1 --2- 172.21.15.39:0/1419058403 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a658559070 0x55a658900650 unknown :-1 s=CLOSED pgs=911 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:05.541 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:05.514+0000 7fc35f232740 1 -- 172.21.15.39:0/1419058403 >> 172.21.15.39:0/1419058403 conn(0x55a65854d460 msgr2=0x55a6588fe540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:52:05.541 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:05.514+0000 7fc35f232740 1 -- 172.21.15.39:0/1419058403 shutdown_connections 2024-09-21T03:52:05.542 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:05.514+0000 7fc35f232740 1 -- 172.21.15.39:0/1419058403 wait complete. 2024-09-21T03:52:05.542 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:05.514+0000 7fc35f232740 1 -- start start 2024-09-21T03:52:05.542 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:05.515+0000 7fc35f232740 1 Processor -- start 2024-09-21T03:52:05.542 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:05.515+0000 7fc35f232740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a6585301b0 0x55a658910150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:52:05.542 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:05.515+0000 7fc35f232740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a658538390 0x55a658910690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:52:05.542 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:05.515+0000 7fc35f232740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a658559070 0x55a65891c4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:52:05.542 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:05.515+0000 7fc35f232740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x55a6586c7700 con 0x55a6585301b0 2024-09-21T03:52:05.542 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:05.515+0000 7fc35f232740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x55a658801110 con 0x55a658538390 2024-09-21T03:52:05.542 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:05.515+0000 7fc35f232740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55a658913190 con 0x55a658559070 2024-09-21T03:52:05.542 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:05.515+0000 7fc35f232740 10 client.0 _ll_register_callbacks cb 0x55a65881a990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:52:05.542 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:05.515+0000 7fc35259c640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a6585301b0 0x55a658910150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:52:05.542 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:05.515+0000 7fc35259c640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a6585301b0 0x55a658910150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:60812/0 (socket says 172.21.15.39:60812) 2024-09-21T03:52:05.542 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:05.515+0000 7fc35259c640 1 -- 172.21.15.39:0/502317461 learned_addr learned my addr 172.21.15.39:0/502317461 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:52:05.543 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:05.515+0000 7fc351d9b640 1 --2- 172.21.15.39:0/502317461 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a658538390 0x55a658910690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:52:05.543 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:05.515+0000 7fc352d9d640 1 --2- 172.21.15.39:0/502317461 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a658559070 0x55a65891c4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:52:05.543 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:05.515+0000 7fc35259c640 1 -- 172.21.15.39:0/502317461 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a658559070 msgr2=0x55a65891c4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:52:05.543 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:05.515+0000 7fc35259c640 1 --2- 172.21.15.39:0/502317461 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a658559070 0x55a65891c4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:05.543 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:05.515+0000 7fc35259c640 1 -- 172.21.15.39:0/502317461 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a658538390 msgr2=0x55a658910690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:52:05.543 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:05.515+0000 7fc35259c640 1 --2- 172.21.15.39:0/502317461 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a658538390 0x55a658910690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:05.543 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:05.515+0000 7fc35259c640 1 -- 172.21.15.39:0/502317461 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a65882ec40 con 0x55a6585301b0 2024-09-21T03:52:05.543 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:05.515+0000 7fc352d9d640 1 --2- 172.21.15.39:0/502317461 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a658559070 0x55a65891c4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:52:05.543 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:05.516+0000 7fc35259c640 1 --2- 172.21.15.39:0/502317461 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a6585301b0 0x55a658910150 secure :-1 s=READY pgs=912 cs=0 l=1 rev1=1 crypto rx=0x7fc34c003d00 tx=0x7fc34c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d633c6066968dd8 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:52:05.543 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:05.516+0000 7fc339ffb640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-09-21T03:52:05.543 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:05.516+0000 7fc339ffb640 1 -- 172.21.15.39:0/502317461 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc34c02c060 con 0x55a6585301b0 2024-09-21T03:52:05.543 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:05.516+0000 7fc339ffb640 1 -- 172.21.15.39:0/502317461 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8484+0+0 (secure 0 0 0) 0x7fc34c017030 con 0x55a6585301b0 2024-09-21T03:52:05.543 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:05.516+0000 7fc35f232740 1 -- 172.21.15.39:0/502317461 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc344004d20 con 0x55a6585301b0 2024-09-21T03:52:05.543 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:05.516+0000 7fc339ffb640 1 -- 172.21.15.39:0/502317461 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fc34c027410 con 0x55a6585301b0 2024-09-21T03:52:05.543 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:05.516+0000 7fc35f232740 10 client.17010 fetch_fsmap learned FSMap version 29 2024-09-21T03:52:05.544 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:05.516+0000 7fc35f232740 1 -- 172.21.15.39:0/502317461 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55a65891eee0 con 0x55a6585301b0 2024-09-21T03:52:05.544 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:05.516+0000 7fc339ffb640 1 -- 172.21.15.39:0/502317461 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc344004d20 con 0x55a6585301b0 2024-09-21T03:52:05.544 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:05.516+0000 7fc339ffb640 1 -- 172.21.15.39:0/502317461 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fc34c01c3f0 con 0x55a6585301b0 2024-09-21T03:52:05.544 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:05.516+0000 7fc35f232740 10 client.17010 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T03:52:05.544 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:05.516+0000 7fc35f232740 10 client.17010 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T03:52:05.544 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:05.516+0000 7fc35f232740 20 client.17010 populate_metadata read hostname 'smithi039' 2024-09-21T03:52:05.544 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:05.517+0000 7fc35f232740 1 --2- 172.21.15.39:0/502317461 >> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] conn(0x55a6585179f0 0x55a658922410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:52:05.544 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:05.517+0000 7fc35f232740 4 client.17010 mds_command: new command op to 24695 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:52:05.544 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:05.517+0000 7fc35f232740 1 -- 172.21.15.39:0/502317461 --> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a658878da0 con 0x55a6585179f0 2024-09-21T03:52:05.544 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:05.517+0000 7fc351d9b640 1 --2- 172.21.15.39:0/502317461 >> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] conn(0x55a6585179f0 0x55a658922410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:52:05.544 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:05.517+0000 7fc351d9b640 1 --2- 172.21.15.39:0/502317461 >> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] conn(0x55a6585179f0 0x55a658922410 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3dd231966e231bb7 server_cookie=83cf5f6437c411a7 in_seq=0 out_seq=0 2024-09-21T03:52:05.544 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:05.517+0000 7fc339ffb640 10 client.17010 ms_handle_connect on v2:172.21.15.150:6838/2949386907 2024-09-21T03:52:05.544 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:05.518+0000 7fc339ffb640 1 -- 172.21.15.39:0/502317461 <== mds.0 v2:172.21.15.150:6838/2949386907 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55a658878da0 con 0x55a6585179f0 2024-09-21T03:52:05.544 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:05.518+0000 7fc339ffb640 10 client.17010 handle_command_reply: tid=0 2024-09-21T03:52:05.544 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:05.518+0000 7fc35f232740 1 client.17010 shutdown 2024-09-21T03:52:05.545 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:05.518+0000 7fc35f232740 1 -- 172.21.15.39:0/502317461 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a6585301b0 msgr2=0x55a658910150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:52:05.545 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:05.518+0000 7fc35f232740 1 --2- 172.21.15.39:0/502317461 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a6585301b0 0x55a658910150 secure :-1 s=READY pgs=912 cs=0 l=1 rev1=1 crypto rx=0x7fc34c003d00 tx=0x7fc34c0046c0 comp rx=0 tx=0).stop 2024-09-21T03:52:05.545 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:05.518+0000 7fc35f232740 1 -- 172.21.15.39:0/502317461 shutdown_connections 2024-09-21T03:52:05.545 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:05.518+0000 7fc35f232740 1 --2- 172.21.15.39:0/502317461 >> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] conn(0x55a6585179f0 0x55a658922410 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:05.545 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:05.518+0000 7fc35f232740 1 --2- 172.21.15.39:0/502317461 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a658559070 0x55a65891c4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:05.545 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:05.518+0000 7fc35f232740 1 --2- 172.21.15.39:0/502317461 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a658538390 0x55a658910690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:05.545 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:05.518+0000 7fc339ffb640 0 client.17010 ms_handle_reset on v2:172.21.15.150:6838/2949386907 2024-09-21T03:52:05.545 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:05.518+0000 7fc35f232740 1 --2- 172.21.15.39:0/502317461 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a6585301b0 0x55a658910150 unknown :-1 s=CLOSED pgs=912 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:05.545 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:05.518+0000 7fc35f232740 1 -- 172.21.15.39:0/502317461 >> 172.21.15.39:0/502317461 conn(0x55a65854d460 msgr2=0x55a658902040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:52:05.545 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:05.519+0000 7fc35f232740 1 -- 172.21.15.39:0/502317461 shutdown_connections 2024-09-21T03:52:05.546 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:05.519+0000 7fc35f232740 1 -- 172.21.15.39:0/502317461 wait complete. 2024-09-21T03:52:05.546 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:05.519+0000 7fc35f232740 20 client.17010 trim_cache size 0 max 16384 2024-09-21T03:52:05.550 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:05.537+0000 7fc35f232740 1 Processor -- start 2024-09-21T03:52:05.550 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:05.537+0000 7fc35f232740 1 -- start start 2024-09-21T03:52:05.550 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:05.537+0000 7fc35f232740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a6585301b0 0x55a658910610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:52:05.550 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:05.537+0000 7fc35f232740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a658538390 0x55a6588f9280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:52:05.550 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:05.537+0000 7fc35f232740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a658559070 0x55a65890d340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:52:05.550 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:05.537+0000 7fc35f232740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fc3541ad5c0 con 0x55a658538390 2024-09-21T03:52:05.551 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:05.537+0000 7fc35f232740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fc3541af100 con 0x55a658559070 2024-09-21T03:52:05.551 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:05.537+0000 7fc35f232740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fc3541adc90 con 0x55a6585301b0 2024-09-21T03:52:05.551 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:05.537+0000 7fc351d9b640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a658538390 0x55a6588f9280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:52:05.551 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:05.537+0000 7fc351d9b640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a658538390 0x55a6588f9280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:60814/0 (socket says 172.21.15.39:60814) 2024-09-21T03:52:05.551 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:05.537+0000 7fc351d9b640 1 -- 172.21.15.39:0/3152716055 learned_addr learned my addr 172.21.15.39:0/3152716055 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:52:05.551 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:05.537+0000 7fc35259c640 1 --2- 172.21.15.39:0/3152716055 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a6585301b0 0x55a658910610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:52:05.551 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:05.537+0000 7fc352d9d640 1 --2- 172.21.15.39:0/3152716055 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a658559070 0x55a65890d340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:52:05.551 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:05.537+0000 7fc351d9b640 1 -- 172.21.15.39:0/3152716055 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a6585301b0 msgr2=0x55a658910610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:52:05.551 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:05.537+0000 7fc351d9b640 1 --2- 172.21.15.39:0/3152716055 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a6585301b0 0x55a658910610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:05.551 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:05.537+0000 7fc351d9b640 1 -- 172.21.15.39:0/3152716055 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a658559070 msgr2=0x55a65890d340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:52:05.551 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:05.537+0000 7fc351d9b640 1 --2- 172.21.15.39:0/3152716055 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a658559070 0x55a65890d340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:05.551 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:05.537+0000 7fc351d9b640 1 -- 172.21.15.39:0/3152716055 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a65890f0b0 con 0x55a658538390 2024-09-21T03:52:05.551 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:05.538+0000 7fc35259c640 1 --2- 172.21.15.39:0/3152716055 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a6585301b0 0x55a658910610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:52:05.551 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:05.538+0000 7fc351d9b640 1 --2- 172.21.15.39:0/3152716055 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a658538390 0x55a6588f9280 secure :-1 s=READY pgs=913 cs=0 l=1 rev1=1 crypto rx=0x7fc33c0cdc20 tx=0x7fc33c003f50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=13880bb94353a5cf server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:52:05.551 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:05.538+0000 7fc33bfff640 1 -- 172.21.15.39:0/3152716055 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc33c049040 con 0x55a658538390 2024-09-21T03:52:05.552 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:05.538+0000 7fc33bfff640 1 -- 172.21.15.39:0/3152716055 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc33c0b50f0 con 0x55a658538390 2024-09-21T03:52:05.552 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:05.538+0000 7fc33bfff640 1 -- 172.21.15.39:0/3152716055 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc33c003d70 con 0x55a658538390 2024-09-21T03:52:05.552 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:05.539+0000 7fc35f232740 1 -- 172.21.15.39:0/3152716055 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a658538390 msgr2=0x55a6588f9280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:52:05.552 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:05.539+0000 7fc35f232740 1 --2- 172.21.15.39:0/3152716055 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a658538390 0x55a6588f9280 secure :-1 s=READY pgs=913 cs=0 l=1 rev1=1 crypto rx=0x7fc33c0cdc20 tx=0x7fc33c003f50 comp rx=0 tx=0).stop 2024-09-21T03:52:05.552 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:05.539+0000 7fc35f232740 1 -- 172.21.15.39:0/3152716055 shutdown_connections 2024-09-21T03:52:05.552 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:05.539+0000 7fc35f232740 1 --2- 172.21.15.39:0/3152716055 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a658559070 0x55a65890d340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:05.552 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:05.539+0000 7fc35f232740 1 --2- 172.21.15.39:0/3152716055 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a658538390 0x55a6588f9280 unknown :-1 s=CLOSED pgs=913 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:05.552 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:05.539+0000 7fc35f232740 1 --2- 172.21.15.39:0/3152716055 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a6585301b0 0x55a658910610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:05.552 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:05.539+0000 7fc35f232740 1 -- 172.21.15.39:0/3152716055 >> 172.21.15.39:0/3152716055 conn(0x55a65854d460 msgr2=0x55a65890ff00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:52:05.552 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:05.539+0000 7fc35f232740 1 -- 172.21.15.39:0/3152716055 shutdown_connections 2024-09-21T03:52:05.552 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:05.539+0000 7fc35f232740 1 -- 172.21.15.39:0/3152716055 wait complete. 2024-09-21T03:52:05.552 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:05.539+0000 7fc35f232740 1 -- start start 2024-09-21T03:52:05.552 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:05.539+0000 7fc35f232740 1 Processor -- start 2024-09-21T03:52:05.552 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:05.540+0000 7fc35f232740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a6585301b0 0x55a658914700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:52:05.552 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:05.540+0000 7fc35f232740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a658538390 0x55a658914c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:52:05.552 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:05.540+0000 7fc35f232740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a658559070 0x55a65898be10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:52:05.553 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:05.540+0000 7fc35f232740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fc3541aea30 con 0x55a658538390 2024-09-21T03:52:05.553 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:05.540+0000 7fc35f232740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fc3541ae360 con 0x55a658559070 2024-09-21T03:52:05.553 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:05.540+0000 7fc35f232740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fc3541ab5a0 con 0x55a6585301b0 2024-09-21T03:52:05.553 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:05.540+0000 7fc35f232740 10 client.0 _ll_register_callbacks cb 0x7fc34c004430 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:52:05.553 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:05.540+0000 7fc351d9b640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a658538390 0x55a658914c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:52:05.553 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:05.540+0000 7fc351d9b640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a658538390 0x55a658914c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:60820/0 (socket says 172.21.15.39:60820) 2024-09-21T03:52:05.553 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:05.540+0000 7fc351d9b640 1 -- 172.21.15.39:0/3751640397 learned_addr learned my addr 172.21.15.39:0/3751640397 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:52:05.553 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:05.540+0000 7fc35259c640 1 --2- 172.21.15.39:0/3751640397 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a6585301b0 0x55a658914700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:52:05.553 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:05.540+0000 7fc351d9b640 1 -- 172.21.15.39:0/3751640397 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a6585301b0 msgr2=0x55a658914700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:52:05.553 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:05.540+0000 7fc351d9b640 1 --2- 172.21.15.39:0/3751640397 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a6585301b0 0x55a658914700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:05.553 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:05.540+0000 7fc351d9b640 1 -- 172.21.15.39:0/3751640397 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a658559070 msgr2=0x55a65898be10 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-21T03:52:05.553 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:05.540+0000 7fc351d9b640 1 --2- 172.21.15.39:0/3751640397 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a658559070 0x55a65898be10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:05.553 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:05.540+0000 7fc351d9b640 1 -- 172.21.15.39:0/3751640397 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a6589154c0 con 0x55a658538390 2024-09-21T03:52:05.553 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:05.540+0000 7fc351d9b640 1 --2- 172.21.15.39:0/3751640397 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a658538390 0x55a658914c40 secure :-1 s=READY pgs=914 cs=0 l=1 rev1=1 crypto rx=0x7fc33c0ca260 tx=0x7fc33c031a10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bee85b9d960f26b0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:52:05.553 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:05.540+0000 7fc339ffb640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-09-21T03:52:05.554 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:05.540+0000 7fc339ffb640 1 -- 172.21.15.39:0/3751640397 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc33c049040 con 0x55a658538390 2024-09-21T03:52:05.554 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:05.540+0000 7fc339ffb640 1 -- 172.21.15.39:0/3751640397 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8484+0+0 (secure 0 0 0) 0x7fc33c0c9f00 con 0x55a658538390 2024-09-21T03:52:05.554 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:05.540+0000 7fc35f232740 1 -- 172.21.15.39:0/3751640397 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55a658874850 con 0x55a658538390 2024-09-21T03:52:05.554 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:05.541+0000 7fc339ffb640 1 -- 172.21.15.39:0/3751640397 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fc33c0f51b0 con 0x55a658538390 2024-09-21T03:52:05.554 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:05.541+0000 7fc35f232740 10 client.17016 fetch_fsmap learned FSMap version 29 2024-09-21T03:52:05.554 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:05.541+0000 7fc35f232740 1 -- 172.21.15.39:0/3751640397 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55a65898d2e0 con 0x55a658538390 2024-09-21T03:52:05.554 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:05.541+0000 7fc339ffb640 1 -- 172.21.15.39:0/3751640397 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55a658874850 con 0x55a658538390 2024-09-21T03:52:05.554 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:05.541+0000 7fc339ffb640 1 -- 172.21.15.39:0/3751640397 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fc33c0d7170 con 0x55a658538390 2024-09-21T03:52:05.554 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:05.541+0000 7fc35f232740 10 client.17016 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T03:52:05.554 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:05.541+0000 7fc35f232740 10 client.17016 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T03:52:05.554 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:05.541+0000 7fc35f232740 20 client.17016 populate_metadata read hostname 'smithi039' 2024-09-21T03:52:05.554 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:05.541+0000 7fc35f232740 1 --2- 172.21.15.39:0/3751640397 >> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] conn(0x55a6584ecda0 0x55a65898fa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:52:05.554 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:05.541+0000 7fc35f232740 4 client.17016 mds_command: new command op to 24695 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-21T03:52:05.554 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:05.541+0000 7fc35f232740 1 -- 172.21.15.39:0/3751640397 --> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55a658871c80 con 0x55a6584ecda0 2024-09-21T03:52:05.554 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:05.541+0000 7fc35259c640 1 --2- 172.21.15.39:0/3751640397 >> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] conn(0x55a6584ecda0 0x55a65898fa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:52:05.554 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:05.542+0000 7fc35259c640 1 --2- 172.21.15.39:0/3751640397 >> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] conn(0x55a6584ecda0 0x55a65898fa10 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4d390c2a03b857f3 server_cookie=b57694bd22667cf5 in_seq=0 out_seq=0 2024-09-21T03:52:05.555 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:05.542+0000 7fc339ffb640 10 client.17016 ms_handle_connect on v2:172.21.15.150:6838/2949386907 2024-09-21T03:52:05.555 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:05.542+0000 7fc339ffb640 1 -- 172.21.15.39:0/3751640397 <== mds.0 v2:172.21.15.150:6838/2949386907 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3214 (crc 0 0 0) 0x55a658871c80 con 0x55a6584ecda0 2024-09-21T03:52:05.555 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:05.542+0000 7fc339ffb640 10 client.17016 handle_command_reply: tid=0 2024-09-21T03:52:05.555 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:05.542+0000 7fc35f232740 1 client.17016 shutdown 2024-09-21T03:52:05.555 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:05.542+0000 7fc35f232740 1 -- 172.21.15.39:0/3751640397 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a658538390 msgr2=0x55a658914c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:52:05.555 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:05.542+0000 7fc35f232740 1 --2- 172.21.15.39:0/3751640397 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a658538390 0x55a658914c40 secure :-1 s=READY pgs=914 cs=0 l=1 rev1=1 crypto rx=0x7fc33c0ca260 tx=0x7fc33c031a10 comp rx=0 tx=0).stop 2024-09-21T03:52:05.555 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:05.542+0000 7fc35f232740 1 -- 172.21.15.39:0/3751640397 shutdown_connections 2024-09-21T03:52:05.555 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:05.542+0000 7fc35f232740 1 --2- 172.21.15.39:0/3751640397 >> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] conn(0x55a6584ecda0 0x55a65898fa10 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:05.555 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:05.542+0000 7fc35f232740 1 --2- 172.21.15.39:0/3751640397 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a658559070 0x55a65898be10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:05.555 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:05.542+0000 7fc35f232740 1 --2- 172.21.15.39:0/3751640397 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a658538390 0x55a658914c40 unknown :-1 s=CLOSED pgs=914 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:05.555 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:05.542+0000 7fc35f232740 1 --2- 172.21.15.39:0/3751640397 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a6585301b0 0x55a658914700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:05.555 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:05.542+0000 7fc35f232740 1 -- 172.21.15.39:0/3751640397 >> 172.21.15.39:0/3751640397 conn(0x55a65854d460 msgr2=0x55a658901680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:52:05.555 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:05.542+0000 7fc339ffb640 0 client.17016 ms_handle_reset on v2:172.21.15.150:6838/2949386907 2024-09-21T03:52:05.556 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:05.543+0000 7fc35f232740 1 -- 172.21.15.39:0/3751640397 shutdown_connections 2024-09-21T03:52:05.556 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:05.543+0000 7fc35f232740 1 -- 172.21.15.39:0/3751640397 wait complete. 2024-09-21T03:52:05.556 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:05.543+0000 7fc35f232740 20 client.17016 trim_cache size 0 max 16384 2024-09-21T03:52:05.556 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-09-21T03:52:05.556 INFO:teuthology.orchestra.run.smithi039.stdout: "status": "no active scrubs running", 2024-09-21T03:52:05.556 INFO:teuthology.orchestra.run.smithi039.stdout: "scrubs": {}, 2024-09-21T03:52:05.556 INFO:teuthology.orchestra.run.smithi039.stdout: "scrub_stats": { 2024-09-21T03:52:05.556 INFO:teuthology.orchestra.run.smithi039.stdout: "ae1fc4cf-5458-47d0-afcb-282b136dba15": { 2024-09-21T03:52:05.557 INFO:teuthology.orchestra.run.smithi039.stdout: "start_time": "2024-09-21T03:51:18.560378+0000", 2024-09-21T03:52:05.557 INFO:teuthology.orchestra.run.smithi039.stdout: "path": "/", 2024-09-21T03:52:05.557 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_started": 0, 2024-09-21T03:52:05.557 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_passed": 0, 2024-09-21T03:52:05.557 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_failed": 0, 2024-09-21T03:52:05.557 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_skipped": 0 2024-09-21T03:52:05.557 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:52:05.557 INFO:teuthology.orchestra.run.smithi039.stdout: "160750d8-401f-4810-866d-e63e7e4bc3e0": { 2024-09-21T03:52:05.557 INFO:teuthology.orchestra.run.smithi039.stdout: "start_time": "2024-09-21T03:51:04.208673+0000", 2024-09-21T03:52:05.557 INFO:teuthology.orchestra.run.smithi039.stdout: "path": "/", 2024-09-21T03:52:05.557 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_started": 0, 2024-09-21T03:52:05.557 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_passed": 0, 2024-09-21T03:52:05.557 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_failed": 0, 2024-09-21T03:52:05.557 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_skipped": 0 2024-09-21T03:52:05.558 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:52:05.558 INFO:teuthology.orchestra.run.smithi039.stdout: "74ab917e-ecea-4778-a1c3-c184214fb24c": { 2024-09-21T03:52:05.558 INFO:teuthology.orchestra.run.smithi039.stdout: "start_time": "2024-09-21T03:50:17.726630+0000", 2024-09-21T03:52:05.558 INFO:teuthology.orchestra.run.smithi039.stdout: "path": "/", 2024-09-21T03:52:05.558 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_started": 0, 2024-09-21T03:52:05.558 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_passed": 0, 2024-09-21T03:52:05.558 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_failed": 0, 2024-09-21T03:52:05.558 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_skipped": 0 2024-09-21T03:52:05.558 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:52:05.558 INFO:teuthology.orchestra.run.smithi039.stdout: "6e760f1f-9048-4f92-bd66-d708fbef75b3": { 2024-09-21T03:52:05.558 INFO:teuthology.orchestra.run.smithi039.stdout: "start_time": "2024-09-21T03:50:03.711918+0000", 2024-09-21T03:52:05.558 INFO:teuthology.orchestra.run.smithi039.stdout: "path": "/", 2024-09-21T03:52:05.558 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_started": 0, 2024-09-21T03:52:05.558 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_passed": 0, 2024-09-21T03:52:05.559 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_failed": 0, 2024-09-21T03:52:05.559 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_skipped": 0 2024-09-21T03:52:05.559 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:52:05.559 INFO:teuthology.orchestra.run.smithi039.stdout: "e28ad743-2b22-45b3-8799-a638c2883623": { 2024-09-21T03:52:05.559 INFO:teuthology.orchestra.run.smithi039.stdout: "start_time": "2024-09-21T03:48:31.448821+0000", 2024-09-21T03:52:05.559 INFO:teuthology.orchestra.run.smithi039.stdout: "path": "/", 2024-09-21T03:52:05.559 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_started": 0, 2024-09-21T03:52:05.559 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_passed": 0, 2024-09-21T03:52:05.559 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_failed": 0, 2024-09-21T03:52:05.560 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_skipped": 0 2024-09-21T03:52:05.560 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:52:05.560 INFO:teuthology.orchestra.run.smithi039.stdout: "ede73e6e-3fd0-4a7f-b5bb-f68946f57eea": { 2024-09-21T03:52:05.560 INFO:teuthology.orchestra.run.smithi039.stdout: "start_time": "2024-09-21T03:51:32.235096+0000", 2024-09-21T03:52:05.560 INFO:teuthology.orchestra.run.smithi039.stdout: "path": "/", 2024-09-21T03:52:05.560 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_started": 0, 2024-09-21T03:52:05.560 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_passed": 0, 2024-09-21T03:52:05.560 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_failed": 0, 2024-09-21T03:52:05.560 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_skipped": 0 2024-09-21T03:52:05.560 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:52:05.560 INFO:teuthology.orchestra.run.smithi039.stdout: "29411307-eb8d-4339-8d2b-47ebd32c0080": { 2024-09-21T03:52:05.560 INFO:teuthology.orchestra.run.smithi039.stdout: "start_time": "2024-09-21T03:47:38.092920+0000", 2024-09-21T03:52:05.560 INFO:teuthology.orchestra.run.smithi039.stdout: "path": "/", 2024-09-21T03:52:05.561 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_started": 0, 2024-09-21T03:52:05.561 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_passed": 0, 2024-09-21T03:52:05.561 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_failed": 0, 2024-09-21T03:52:05.561 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_skipped": 0 2024-09-21T03:52:05.561 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:52:05.561 INFO:teuthology.orchestra.run.smithi039.stdout: "9860c2f4-b7b1-47aa-865a-9a1c4434a5f6": { 2024-09-21T03:52:05.561 INFO:teuthology.orchestra.run.smithi039.stdout: "start_time": "2024-09-21T03:49:17.718867+0000", 2024-09-21T03:52:05.561 INFO:teuthology.orchestra.run.smithi039.stdout: "path": "/", 2024-09-21T03:52:05.561 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_started": 0, 2024-09-21T03:52:05.561 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_passed": 0, 2024-09-21T03:52:05.561 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_failed": 0, 2024-09-21T03:52:05.561 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_skipped": 0 2024-09-21T03:52:05.561 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:52:05.561 INFO:teuthology.orchestra.run.smithi039.stdout: "c7e2b8e4-057d-435f-b2bd-b633de98035f": { 2024-09-21T03:52:05.561 INFO:teuthology.orchestra.run.smithi039.stdout: "start_time": "2024-09-21T03:47:19.400305+0000", 2024-09-21T03:52:05.562 INFO:teuthology.orchestra.run.smithi039.stdout: "path": "/", 2024-09-21T03:52:05.562 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_started": 0, 2024-09-21T03:52:05.562 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_passed": 0, 2024-09-21T03:52:05.562 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_failed": 0, 2024-09-21T03:52:05.562 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_skipped": 0 2024-09-21T03:52:05.562 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:52:05.562 INFO:teuthology.orchestra.run.smithi039.stdout: "10728de8-c49d-4edc-be2b-639249255e94": { 2024-09-21T03:52:05.562 INFO:teuthology.orchestra.run.smithi039.stdout: "start_time": "2024-09-21T03:47:03.965623+0000", 2024-09-21T03:52:05.562 INFO:teuthology.orchestra.run.smithi039.stdout: "path": "/", 2024-09-21T03:52:05.562 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_started": 0, 2024-09-21T03:52:05.562 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_passed": 0, 2024-09-21T03:52:05.562 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_failed": 0, 2024-09-21T03:52:05.562 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_skipped": 0 2024-09-21T03:52:05.562 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:52:05.563 INFO:teuthology.orchestra.run.smithi039.stdout: "02509e4f-3911-4983-b16f-df8ded0b9998": { 2024-09-21T03:52:05.563 INFO:teuthology.orchestra.run.smithi039.stdout: "start_time": "2024-09-21T03:46:17.889576+0000", 2024-09-21T03:52:05.563 INFO:teuthology.orchestra.run.smithi039.stdout: "path": "/", 2024-09-21T03:52:05.563 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_started": 0, 2024-09-21T03:52:05.563 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_passed": 0, 2024-09-21T03:52:05.563 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_failed": 0, 2024-09-21T03:52:05.563 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_skipped": 0 2024-09-21T03:52:05.563 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-21T03:52:05.563 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-21T03:52:05.563 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-09-21T03:52:05.564 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:05.543+0000 7fc358cbe640 1 -- 172.21.15.39:0/721333096 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7fc3400b6680 msgr2=0x7fc3400b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:52:05.564 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:05.543+0000 7fc358cbe640 1 --2- 172.21.15.39:0/721333096 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7fc3400b6680 0x7fc3400b8b90 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x55a6588ff2d0 tx=0x7fc34c00b410 comp rx=0 tx=0).stop 2024-09-21T03:52:05.564 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:05.543+0000 7fc358cbe640 1 -- 172.21.15.39:0/721333096 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc354105300 msgr2=0x7fc35419aae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:52:05.564 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:05.543+0000 7fc358cbe640 1 --2- 172.21.15.39:0/721333096 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc354105300 0x7fc35419aae0 secure :-1 s=READY pgs=910 cs=0 l=1 rev1=1 crypto rx=0x7fc33c02a930 tx=0x7fc33c004270 comp rx=0 tx=0).stop 2024-09-21T03:52:05.564 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:05.543+0000 7fc358cbe640 1 -- 172.21.15.39:0/721333096 shutdown_connections 2024-09-21T03:52:05.564 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:05.543+0000 7fc358cbe640 1 --2- 172.21.15.39:0/721333096 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7fc3400b6680 0x7fc3400b8b90 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:05.564 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:05.543+0000 7fc358cbe640 1 --2- 172.21.15.39:0/721333096 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc35410de10 0x7fc35419fd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:05.565 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:05.543+0000 7fc358cbe640 1 --2- 172.21.15.39:0/721333096 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc354105300 0x7fc35419aae0 unknown :-1 s=CLOSED pgs=910 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:05.565 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:05.543+0000 7fc358cbe640 1 --2- 172.21.15.39:0/721333096 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc3541049b0 0x7fc35419a540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:05.565 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:05.543+0000 7fc358cbe640 1 -- 172.21.15.39:0/721333096 >> 172.21.15.39:0/721333096 conn(0x7fc354079730 msgr2=0x7fc354100850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:52:05.565 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:05.543+0000 7fc358cbe640 1 -- 172.21.15.39:0/721333096 shutdown_connections 2024-09-21T03:52:05.565 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:05.543+0000 7fc358cbe640 1 -- 172.21.15.39:0/721333096 wait complete. 2024-09-21T03:52:06.448 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-21T03:52:06.448 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 51116d5c-77ca-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T03:52:06.717 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/51116d5c-77ca-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:52:07.366 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:07.357+0000 7fb0d6aa8640 1 -- 172.21.15.39:0/1528978782 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb0d0102a80 msgr2=0x7fb0d0102e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:52:07.366 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:07.357+0000 7fb0d6aa8640 1 --2- 172.21.15.39:0/1528978782 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb0d0102a80 0x7fb0d0102e90 secure :-1 s=READY pgs=915 cs=0 l=1 rev1=1 crypto rx=0x7fb0cc009900 tx=0x7fb0cc02a410 comp rx=0 tx=0).stop 2024-09-21T03:52:07.367 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:07.357+0000 7fb0d6aa8640 1 -- 172.21.15.39:0/1528978782 shutdown_connections 2024-09-21T03:52:07.367 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:07.357+0000 7fb0d6aa8640 1 --2- 172.21.15.39:0/1528978782 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb0d0108d10 0x7fb0d010f540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:07.367 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:07.357+0000 7fb0d6aa8640 1 --2- 172.21.15.39:0/1528978782 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb0d01083c0 0x7fb0d01087d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:07.367 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:07.357+0000 7fb0d6aa8640 1 --2- 172.21.15.39:0/1528978782 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb0d0102a80 0x7fb0d0102e90 unknown :-1 s=CLOSED pgs=915 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:07.367 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:07.357+0000 7fb0d6aa8640 1 -- 172.21.15.39:0/1528978782 >> 172.21.15.39:0/1528978782 conn(0x7fb0d0079740 msgr2=0x7fb0d0079b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:52:07.367 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:07.357+0000 7fb0d6aa8640 1 -- 172.21.15.39:0/1528978782 shutdown_connections 2024-09-21T03:52:07.367 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:07.357+0000 7fb0d6aa8640 1 -- 172.21.15.39:0/1528978782 wait complete. 2024-09-21T03:52:07.367 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:07.358+0000 7fb0d6aa8640 1 Processor -- start 2024-09-21T03:52:07.367 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:07.358+0000 7fb0d6aa8640 1 -- start start 2024-09-21T03:52:07.367 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:07.358+0000 7fb0d6aa8640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb0d0102a80 0x7fb0d019b310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:52:07.367 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:07.358+0000 7fb0d6aa8640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb0d01083c0 0x7fb0d019b850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:52:07.367 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:07.358+0000 7fb0d6aa8640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb0d0108d10 0x7fb0d019bd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:52:07.367 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:07.358+0000 7fb0d6aa8640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fb0d0112980 con 0x7fb0d01083c0 2024-09-21T03:52:07.368 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:07.358+0000 7fb0d6aa8640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fb0d0112800 con 0x7fb0d0108d10 2024-09-21T03:52:07.368 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:07.358+0000 7fb0d6aa8640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fb0d0112b00 con 0x7fb0d0102a80 2024-09-21T03:52:07.368 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:07.358+0000 7fb0c7fff640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb0d01083c0 0x7fb0d019b850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:52:07.368 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:07.358+0000 7fb0c7fff640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb0d01083c0 0x7fb0d019b850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:59778/0 (socket says 172.21.15.39:59778) 2024-09-21T03:52:07.368 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:07.358+0000 7fb0c7fff640 1 -- 172.21.15.39:0/1539777465 learned_addr learned my addr 172.21.15.39:0/1539777465 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:52:07.368 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:07.359+0000 7fb0d4844640 1 --2- 172.21.15.39:0/1539777465 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb0d0102a80 0x7fb0d019b310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:52:07.368 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:07.359+0000 7fb0d5045640 1 --2- 172.21.15.39:0/1539777465 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb0d0108d10 0x7fb0d019bd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:52:07.368 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:07.359+0000 7fb0c7fff640 1 -- 172.21.15.39:0/1539777465 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb0d0102a80 msgr2=0x7fb0d019b310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:52:07.368 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:07.359+0000 7fb0c7fff640 1 --2- 172.21.15.39:0/1539777465 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb0d0102a80 0x7fb0d019b310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:07.368 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:07.359+0000 7fb0c7fff640 1 -- 172.21.15.39:0/1539777465 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb0d0108d10 msgr2=0x7fb0d019bd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:52:07.368 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:07.359+0000 7fb0c7fff640 1 --2- 172.21.15.39:0/1539777465 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb0d0108d10 0x7fb0d019bd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:07.368 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:07.359+0000 7fb0c7fff640 1 -- 172.21.15.39:0/1539777465 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb0d01a65a0 con 0x7fb0d01083c0 2024-09-21T03:52:07.369 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:07.359+0000 7fb0d5045640 1 --2- 172.21.15.39:0/1539777465 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb0d0108d10 0x7fb0d019bd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:52:07.369 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:07.359+0000 7fb0c7fff640 1 --2- 172.21.15.39:0/1539777465 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb0d01083c0 0x7fb0d019b850 secure :-1 s=READY pgs=916 cs=0 l=1 rev1=1 crypto rx=0x7fb0c000e9f0 tx=0x7fb0c000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:52:07.369 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:07.359+0000 7fb0c5ffb640 1 -- 172.21.15.39:0/1539777465 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0c000ccc0 con 0x7fb0d01083c0 2024-09-21T03:52:07.369 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:07.359+0000 7fb0c5ffb640 1 -- 172.21.15.39:0/1539777465 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb0c00054d0 con 0x7fb0d01083c0 2024-09-21T03:52:07.369 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:07.359+0000 7fb0d6aa8640 1 -- 172.21.15.39:0/1539777465 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb0d01a6890 con 0x7fb0d01083c0 2024-09-21T03:52:07.369 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:07.359+0000 7fb0c5ffb640 1 -- 172.21.15.39:0/1539777465 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0c0010430 con 0x7fb0d01083c0 2024-09-21T03:52:07.369 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:07.359+0000 7fb0d6aa8640 1 -- 172.21.15.39:0/1539777465 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb0d01a6d80 con 0x7fb0d01083c0 2024-09-21T03:52:07.369 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:07.360+0000 7fb0d6aa8640 1 -- 172.21.15.39:0/1539777465 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb0d0103d00 con 0x7fb0d01083c0 2024-09-21T03:52:07.369 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:07.362+0000 7fb0c5ffb640 1 -- 172.21.15.39:0/1539777465 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 27) ==== 154643+0+0 (secure 0 0 0) 0x7fb0c00046c0 con 0x7fb0d01083c0 2024-09-21T03:52:07.369 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:07.363+0000 7fb0c5ffb640 1 --2- 172.21.15.39:0/1539777465 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7fb0b80b66b0 0x7fb0b80b8bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:52:07.369 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:07.363+0000 7fb0d4844640 1 --2- 172.21.15.39:0/1539777465 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7fb0b80b66b0 0x7fb0b80b8bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:52:07.369 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:07.363+0000 7fb0c5ffb640 1 -- 172.21.15.39:0/1539777465 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8484+0+0 (secure 0 0 0) 0x7fb0c00e7db0 con 0x7fb0d01083c0 2024-09-21T03:52:07.370 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:07.364+0000 7fb0d4844640 1 --2- 172.21.15.39:0/1539777465 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7fb0b80b66b0 0x7fb0b80b8bc0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fb0cc000c00 tx=0x7fb0cc031400 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:52:07.370 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:07.367+0000 7fb0c5ffb640 1 -- 172.21.15.39:0/1539777465 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fb0c0056710 con 0x7fb0d01083c0 2024-09-21T03:52:07.468 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:52:07 smithi161 ceph-mon[34533]: pgmap v507: 97 pgs: 97 active+clean; 10 MiB data, 520 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 25 op/s 2024-09-21T03:52:07.475 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:52:07 smithi150 ceph-mon[34289]: pgmap v507: 97 pgs: 97 active+clean; 10 MiB data, 520 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 25 op/s 2024-09-21T03:52:07.583 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:52:07 smithi039 ceph-mon[31211]: pgmap v507: 97 pgs: 97 active+clean; 10 MiB data, 520 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 25 op/s 2024-09-21T03:52:07.597 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:07.594+0000 7fb0d6aa8640 1 -- 172.21.15.39:0/1539777465 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb0d0102e90 con 0x7fb0d01083c0 2024-09-21T03:52:07.597 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:07.595+0000 7fb0c5ffb640 1 -- 172.21.15.39:0/1539777465 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11978 (secure 0 0 0) 0x7fb0d0102e90 con 0x7fb0d01083c0 2024-09-21T03:52:07.598 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T03:52:07.598 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":29,"btime":"2024-09-21T03:46:08:982881+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14790,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6835/508995295","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":508995295},{"type":"v1","addr":"172.21.15.39:6835","nonce":508995295}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14802,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6837/3915912607","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":3915912607},{"type":"v1","addr":"172.21.15.39:6837","nonce":3915912607}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":29,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T03:44:15.009872+0000","modified":"2024-09-21T03:46:08.430113+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24695,"mds_1":24667,"mds_2":24659,"mds_3":24646,"mds_4":14868},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14856":{"gid":14856,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.39:6839/185509192","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6838","nonce":185509192},{"type":"v1","addr":"172.21.15.39:6839","nonce":185509192}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14868":{"gid":14868,"name":"a","rank":4,"incarnation":25,"state":"up:active","state_seq":7,"addr":"172.21.15.39:6841/4247190456","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6840","nonce":4247190456},{"type":"v1","addr":"172.21.15.39:6841","nonce":4247190456}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24599":{"gid":24599,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.150:6833/3114643217","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":3114643217},{"type":"v1","addr":"172.21.15.150:6833","nonce":3114643217}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24601":{"gid":24601,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.161:6833/2963990654","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":2963990654},{"type":"v1","addr":"172.21.15.161:6833","nonce":2963990654}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24632":{"gid":24632,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.150:6835/724594008","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6834","nonce":724594008},{"type":"v1","addr":"172.21.15.150:6835","nonce":724594008}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24640":{"gid":24640,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.161:6835/3747882110","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":3747882110},{"type":"v1","addr":"172.21.15.161:6835","nonce":3747882110}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24646":{"gid":24646,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":10,"addr":"172.21.15.161:6837/3451544597","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6836","nonce":3451544597},{"type":"v1","addr":"172.21.15.161:6837","nonce":3451544597}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24659":{"gid":24659,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.150:6837/4017043144","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6836","nonce":4017043144},{"type":"v1","addr":"172.21.15.150:6837","nonce":4017043144}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24667":{"gid":24667,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.161:6839/2190107270","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6838","nonce":2190107270},{"type":"v1","addr":"172.21.15.161:6839","nonce":2190107270}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24695":{"gid":24695,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.150:6839/2949386907","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6838","nonce":2949386907},{"type":"v1","addr":"172.21.15.150:6839","nonce":2949386907}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24695,"qdb_cluster":[24695,24667,24659,24646,14868]},"id":1}]} 2024-09-21T03:52:07.598 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 29 2024-09-21T03:52:07.600 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:07.597+0000 7fb0d6aa8640 1 -- 172.21.15.39:0/1539777465 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7fb0b80b66b0 msgr2=0x7fb0b80b8bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:52:07.600 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:07.597+0000 7fb0d6aa8640 1 --2- 172.21.15.39:0/1539777465 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7fb0b80b66b0 0x7fb0b80b8bc0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fb0cc000c00 tx=0x7fb0cc031400 comp rx=0 tx=0).stop 2024-09-21T03:52:07.600 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:07.597+0000 7fb0d6aa8640 1 -- 172.21.15.39:0/1539777465 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb0d01083c0 msgr2=0x7fb0d019b850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:52:07.600 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:07.597+0000 7fb0d6aa8640 1 --2- 172.21.15.39:0/1539777465 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb0d01083c0 0x7fb0d019b850 secure :-1 s=READY pgs=916 cs=0 l=1 rev1=1 crypto rx=0x7fb0c000e9f0 tx=0x7fb0c000eeb0 comp rx=0 tx=0).stop 2024-09-21T03:52:07.600 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:07.597+0000 7fb0d6aa8640 1 -- 172.21.15.39:0/1539777465 shutdown_connections 2024-09-21T03:52:07.600 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:07.597+0000 7fb0d6aa8640 1 --2- 172.21.15.39:0/1539777465 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7fb0b80b66b0 0x7fb0b80b8bc0 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:07.600 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:07.597+0000 7fb0d6aa8640 1 --2- 172.21.15.39:0/1539777465 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb0d0108d10 0x7fb0d019bd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:07.600 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:07.597+0000 7fb0d6aa8640 1 --2- 172.21.15.39:0/1539777465 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb0d01083c0 0x7fb0d019b850 unknown :-1 s=CLOSED pgs=916 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:07.600 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:07.597+0000 7fb0d6aa8640 1 --2- 172.21.15.39:0/1539777465 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb0d0102a80 0x7fb0d019b310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:07.600 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:07.597+0000 7fb0d6aa8640 1 -- 172.21.15.39:0/1539777465 >> 172.21.15.39:0/1539777465 conn(0x7fb0d0079740 msgr2=0x7fb0d01014d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:52:07.601 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:07.597+0000 7fb0d6aa8640 1 -- 172.21.15.39:0/1539777465 shutdown_connections 2024-09-21T03:52:07.601 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:07.597+0000 7fb0d6aa8640 1 -- 172.21.15.39:0/1539777465 wait complete. 2024-09-21T03:52:08.336 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:52:08 smithi039 ceph-mon[31211]: from='client.? 172.21.15.39:0/1539777465' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:52:08.427 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 51116d5c-77ca-11ef-baf5-efdc52797490 -- ceph tell mds.1:4 damage ls 2024-09-21T03:52:08.468 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:52:08 smithi161 ceph-mon[34533]: from='client.? 172.21.15.39:0/1539777465' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:52:08.475 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:52:08 smithi150 ceph-mon[34289]: from='client.? 172.21.15.39:0/1539777465' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:52:08.689 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/51116d5c-77ca-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:52:09.229 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:09.226+0000 7f82958a5640 1 -- 172.21.15.39:0/3366156370 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f82901083a0 msgr2=0x7f82901087b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:52:09.229 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:09.226+0000 7f82958a5640 1 --2- 172.21.15.39:0/3366156370 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f82901083a0 0x7f82901087b0 secure :-1 s=READY pgs=917 cs=0 l=1 rev1=1 crypto rx=0x7f8278009900 tx=0x7f827802a410 comp rx=0 tx=0).stop 2024-09-21T03:52:09.229 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:09.226+0000 7f82958a5640 1 -- 172.21.15.39:0/3366156370 shutdown_connections 2024-09-21T03:52:09.230 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:09.226+0000 7f82958a5640 1 --2- 172.21.15.39:0/3366156370 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8290108cf0 0x7f829010f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:09.230 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:09.226+0000 7f82958a5640 1 --2- 172.21.15.39:0/3366156370 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f82901083a0 0x7f82901087b0 unknown :-1 s=CLOSED pgs=917 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:09.230 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:09.226+0000 7f82958a5640 1 --2- 172.21.15.39:0/3366156370 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8290102a60 0x7f8290102e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:09.230 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:09.226+0000 7f82958a5640 1 -- 172.21.15.39:0/3366156370 >> 172.21.15.39:0/3366156370 conn(0x7f8290079730 msgr2=0x7f8290079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:52:09.230 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:09.226+0000 7f82958a5640 1 -- 172.21.15.39:0/3366156370 shutdown_connections 2024-09-21T03:52:09.230 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:09.226+0000 7f82958a5640 1 -- 172.21.15.39:0/3366156370 wait complete. 2024-09-21T03:52:09.230 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:09.227+0000 7f82958a5640 1 Processor -- start 2024-09-21T03:52:09.231 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:09.227+0000 7f82958a5640 1 -- start start 2024-09-21T03:52:09.231 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:09.227+0000 7f82958a5640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8290102a60 0x7f829019b320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:52:09.231 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:09.227+0000 7f82958a5640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f82901083a0 0x7f829019b860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:52:09.232 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:09.227+0000 7f82958a5640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8290108cf0 0x7f829019bda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:52:09.232 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:09.227+0000 7f82958a5640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f82901129b0 con 0x7f8290102a60 2024-09-21T03:52:09.232 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:09.227+0000 7f82958a5640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f8290112830 con 0x7f8290108cf0 2024-09-21T03:52:09.232 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:09.227+0000 7f82958a5640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f8290112b30 con 0x7f82901083a0 2024-09-21T03:52:09.232 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:09.227+0000 7f828effd640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8290102a60 0x7f829019b320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:52:09.232 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:09.227+0000 7f828effd640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8290102a60 0x7f829019b320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:59790/0 (socket says 172.21.15.39:59790) 2024-09-21T03:52:09.232 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:09.227+0000 7f828effd640 1 -- 172.21.15.39:0/4189240614 learned_addr learned my addr 172.21.15.39:0/4189240614 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:52:09.232 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:09.227+0000 7f828f7fe640 1 --2- 172.21.15.39:0/4189240614 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8290108cf0 0x7f829019bda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:52:09.232 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:09.228+0000 7f828e7fc640 1 --2- 172.21.15.39:0/4189240614 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f82901083a0 0x7f829019b860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:52:09.232 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:09.228+0000 7f828effd640 1 -- 172.21.15.39:0/4189240614 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f82901083a0 msgr2=0x7f829019b860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:52:09.232 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:09.228+0000 7f828effd640 1 --2- 172.21.15.39:0/4189240614 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f82901083a0 0x7f829019b860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:09.232 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:09.228+0000 7f828effd640 1 -- 172.21.15.39:0/4189240614 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8290108cf0 msgr2=0x7f829019bda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:52:09.232 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:09.228+0000 7f828effd640 1 --2- 172.21.15.39:0/4189240614 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8290108cf0 0x7f829019bda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:09.232 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:09.228+0000 7f828effd640 1 -- 172.21.15.39:0/4189240614 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f82901a65b0 con 0x7f8290102a60 2024-09-21T03:52:09.232 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:09.228+0000 7f828effd640 1 --2- 172.21.15.39:0/4189240614 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8290102a60 0x7f829019b320 secure :-1 s=READY pgs=918 cs=0 l=1 rev1=1 crypto rx=0x7f828800d950 tx=0x7f828800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:52:09.233 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:09.228+0000 7f82948a3640 1 -- 172.21.15.39:0/4189240614 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8288017030 con 0x7f8290102a60 2024-09-21T03:52:09.233 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:09.228+0000 7f82948a3640 1 -- 172.21.15.39:0/4189240614 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8288004020 con 0x7f8290102a60 2024-09-21T03:52:09.233 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:09.228+0000 7f82958a5640 1 -- 172.21.15.39:0/4189240614 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f82901a68a0 con 0x7f8290102a60 2024-09-21T03:52:09.233 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:09.228+0000 7f82948a3640 1 -- 172.21.15.39:0/4189240614 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82880047a0 con 0x7f8290102a60 2024-09-21T03:52:09.233 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:09.228+0000 7f82958a5640 1 -- 172.21.15.39:0/4189240614 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f82901a6db0 con 0x7f8290102a60 2024-09-21T03:52:09.235 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:09.231+0000 7f829be19740 1 Processor -- start 2024-09-21T03:52:09.235 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:09.231+0000 7f829be19740 1 -- start start 2024-09-21T03:52:09.235 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:09.231+0000 7f82948a3640 1 -- 172.21.15.39:0/4189240614 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 27) ==== 154643+0+0 (secure 0 0 0) 0x7f8288009a30 con 0x7f8290102a60 2024-09-21T03:52:09.235 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:09.231+0000 7f829be19740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564042bc0070 0x564042f676a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:52:09.235 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:09.231+0000 7f829be19740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564042b971b0 0x564042f67be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:52:09.235 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:09.231+0000 7f829be19740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564042b9f390 0x564042f68120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:52:09.235 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:09.231+0000 7f829be19740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x564042eddea0 con 0x564042bc0070 2024-09-21T03:52:09.235 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:09.231+0000 7f829be19740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x564042ecd2d0 con 0x564042b971b0 2024-09-21T03:52:09.236 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:09.231+0000 7f829be19740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x564042ed10e0 con 0x564042b9f390 2024-09-21T03:52:09.236 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:09.231+0000 7f828f7fe640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564042bc0070 0x564042f676a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:52:09.236 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:09.231+0000 7f828f7fe640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564042bc0070 0x564042f676a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:59798/0 (socket says 172.21.15.39:59798) 2024-09-21T03:52:09.236 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:09.231+0000 7f828f7fe640 1 -- 172.21.15.39:0/2946409650 learned_addr learned my addr 172.21.15.39:0/2946409650 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:52:09.236 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:09.231+0000 7f828effd640 1 --2- 172.21.15.39:0/2946409650 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564042b971b0 0x564042f67be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:52:09.236 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:09.232+0000 7f828e7fc640 1 --2- 172.21.15.39:0/2946409650 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564042b9f390 0x564042f68120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:52:09.236 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:09.232+0000 7f828f7fe640 1 -- 172.21.15.39:0/2946409650 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564042b9f390 msgr2=0x564042f68120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:52:09.236 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:09.232+0000 7f828f7fe640 1 --2- 172.21.15.39:0/2946409650 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564042b9f390 0x564042f68120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:09.236 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:09.232+0000 7f828f7fe640 1 -- 172.21.15.39:0/2946409650 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564042b971b0 msgr2=0x564042f67be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:52:09.236 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:09.232+0000 7f828f7fe640 1 --2- 172.21.15.39:0/2946409650 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564042b971b0 0x564042f67be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:09.236 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:09.232+0000 7f828f7fe640 1 -- 172.21.15.39:0/2946409650 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564042c2b650 con 0x564042bc0070 2024-09-21T03:52:09.236 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:09.232+0000 7f828f7fe640 1 --2- 172.21.15.39:0/2946409650 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564042bc0070 0x564042f676a0 secure :-1 s=READY pgs=919 cs=0 l=1 rev1=1 crypto rx=0x7f8280009f50 tx=0x7f82800118d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=64ec29cd05a9ac06 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:52:09.236 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:09.232+0000 7f82948a3640 1 --2- 172.21.15.39:0/4189240614 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f827c0b66b0 0x7f827c0b8bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:52:09.236 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:09.232+0000 7f8285ffb640 1 -- 172.21.15.39:0/2946409650 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8280004d20 con 0x564042bc0070 2024-09-21T03:52:09.236 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:09.232+0000 7f828effd640 1 --2- 172.21.15.39:0/4189240614 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f827c0b66b0 0x7f827c0b8bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:52:09.237 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:09.232+0000 7f8285ffb640 1 -- 172.21.15.39:0/2946409650 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8280004050 con 0x564042bc0070 2024-09-21T03:52:09.237 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:09.232+0000 7f8285ffb640 1 -- 172.21.15.39:0/2946409650 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8280005280 con 0x564042bc0070 2024-09-21T03:52:09.237 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:09.232+0000 7f82948a3640 1 -- 172.21.15.39:0/4189240614 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8484+0+0 (secure 0 0 0) 0x7f82880e7080 con 0x7f8290102a60 2024-09-21T03:52:09.241 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:09.233+0000 7f829be19740 1 -- 172.21.15.39:0/2946409650 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564042bc0070 msgr2=0x564042f676a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:52:09.241 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:09.233+0000 7f829be19740 1 --2- 172.21.15.39:0/2946409650 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564042bc0070 0x564042f676a0 secure :-1 s=READY pgs=919 cs=0 l=1 rev1=1 crypto rx=0x7f8280009f50 tx=0x7f82800118d0 comp rx=0 tx=0).stop 2024-09-21T03:52:09.241 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:09.233+0000 7f828effd640 1 --2- 172.21.15.39:0/4189240614 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f827c0b66b0 0x7f827c0b8bc0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f82880e48a0 tx=0x7f82880eabd0 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:52:09.241 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:09.233+0000 7f829be19740 1 -- 172.21.15.39:0/2946409650 shutdown_connections 2024-09-21T03:52:09.241 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:09.233+0000 7f829be19740 1 --2- 172.21.15.39:0/2946409650 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564042b9f390 0x564042f68120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:09.241 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:09.233+0000 7f829be19740 1 --2- 172.21.15.39:0/2946409650 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564042b971b0 0x564042f67be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:09.241 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:09.233+0000 7f829be19740 1 --2- 172.21.15.39:0/2946409650 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564042bc0070 0x564042f676a0 unknown :-1 s=CLOSED pgs=919 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:09.241 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:09.233+0000 7f829be19740 1 -- 172.21.15.39:0/2946409650 >> 172.21.15.39:0/2946409650 conn(0x564042bb4460 msgr2=0x564042f65590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:52:09.241 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:09.233+0000 7f829be19740 1 -- 172.21.15.39:0/2946409650 shutdown_connections 2024-09-21T03:52:09.241 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:09.233+0000 7f829be19740 1 -- 172.21.15.39:0/2946409650 wait complete. 2024-09-21T03:52:09.241 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:09.234+0000 7f829be19740 1 -- start start 2024-09-21T03:52:09.241 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:09.234+0000 7f829be19740 1 Processor -- start 2024-09-21T03:52:09.242 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:09.234+0000 7f829be19740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564042b971b0 0x564042f72e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:52:09.242 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:09.234+0000 7f829be19740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564042b9f390 0x564042f73340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:52:09.242 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:09.234+0000 7f829be19740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564042bc0070 0x564042f7f170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:52:09.242 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:09.234+0000 7f829be19740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x564042d2e700 con 0x564042b971b0 2024-09-21T03:52:09.242 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:09.234+0000 7f829be19740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x564042e68160 con 0x564042b9f390 2024-09-21T03:52:09.242 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:09.234+0000 7f829be19740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x564042f75e80 con 0x564042bc0070 2024-09-21T03:52:09.242 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:09.234+0000 7f829be19740 10 client.0 _ll_register_callbacks cb 0x564042e819e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:52:09.242 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:09.235+0000 7f828f7fe640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564042b971b0 0x564042f72e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:52:09.242 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:09.235+0000 7f828f7fe640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564042b971b0 0x564042f72e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:59804/0 (socket says 172.21.15.39:59804) 2024-09-21T03:52:09.242 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:09.235+0000 7f828f7fe640 1 -- 172.21.15.39:0/3521849815 learned_addr learned my addr 172.21.15.39:0/3521849815 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:52:09.242 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:09.235+0000 7f828f7fe640 1 --2- 172.21.15.39:0/3521849815 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564042bc0070 0x564042f7f170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:52:09.242 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:09.235+0000 7f828e7fc640 1 --2- 172.21.15.39:0/3521849815 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564042b9f390 0x564042f73340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:52:09.242 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:09.235+0000 7f828f7fe640 1 -- 172.21.15.39:0/3521849815 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564042bc0070 msgr2=0x564042f7f170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:52:09.242 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:09.235+0000 7f828f7fe640 1 --2- 172.21.15.39:0/3521849815 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564042bc0070 0x564042f7f170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:09.243 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:09.235+0000 7f828f7fe640 1 -- 172.21.15.39:0/3521849815 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564042b9f390 msgr2=0x564042f73340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:52:09.243 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:09.235+0000 7f828f7fe640 1 --2- 172.21.15.39:0/3521849815 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564042b9f390 0x564042f73340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:09.243 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:09.235+0000 7f828f7fe640 1 -- 172.21.15.39:0/3521849815 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564042e95c90 con 0x564042b971b0 2024-09-21T03:52:09.243 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:09.235+0000 7f828e7fc640 1 --2- 172.21.15.39:0/3521849815 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564042b9f390 0x564042f73340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:52:09.243 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:09.235+0000 7f828f7fe640 1 --2- 172.21.15.39:0/3521849815 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564042b971b0 0x564042f72e00 secure :-1 s=READY pgs=920 cs=0 l=1 rev1=1 crypto rx=0x7f8280003fd0 tx=0x7f8280005bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4fea89da1e57bf96 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:52:09.243 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:09.235+0000 7f826effd640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-09-21T03:52:09.243 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:09.235+0000 7f826effd640 1 -- 172.21.15.39:0/3521849815 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f828002b030 con 0x564042b971b0 2024-09-21T03:52:09.243 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:09.235+0000 7f826effd640 1 -- 172.21.15.39:0/3521849815 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8484+0+0 (secure 0 0 0) 0x7f8280004c10 con 0x564042b971b0 2024-09-21T03:52:09.243 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:09.235+0000 7f829be19740 1 -- 172.21.15.39:0/3521849815 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8280004050 con 0x564042b971b0 2024-09-21T03:52:09.243 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:09.236+0000 7f826effd640 1 -- 172.21.15.39:0/3521849815 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f8280035050 con 0x564042b971b0 2024-09-21T03:52:09.243 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:09.236+0000 7f829be19740 10 client.17034 fetch_fsmap learned FSMap version 29 2024-09-21T03:52:09.243 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:09.236+0000 7f829be19740 1 -- 172.21.15.39:0/3521849815 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x564042f81bf0 con 0x564042b971b0 2024-09-21T03:52:09.243 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:09.236+0000 7f826effd640 1 -- 172.21.15.39:0/3521849815 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8280004050 con 0x564042b971b0 2024-09-21T03:52:09.243 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:09.236+0000 7f826effd640 1 -- 172.21.15.39:0/3521849815 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f8280027470 con 0x564042b971b0 2024-09-21T03:52:09.244 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:09.236+0000 7f829be19740 10 client.17034 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T03:52:09.244 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:09.236+0000 7f829be19740 10 client.17034 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.a 2024-09-21T03:52:09.244 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:09.236+0000 7f829be19740 20 client.17034 populate_metadata read hostname 'smithi039' 2024-09-21T03:52:09.244 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:09.236+0000 7f829be19740 1 --2- 172.21.15.39:0/3521849815 >> [v2:172.21.15.39:6840/4247190456,v1:172.21.15.39:6841/4247190456] conn(0x564042b7e9f0 0x564042f85060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:52:09.244 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:09.236+0000 7f829be19740 4 client.17034 mds_command: new command op to 14868 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:52:09.244 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:09.236+0000 7f829be19740 1 -- 172.21.15.39:0/3521849815 --> [v2:172.21.15.39:6840/4247190456,v1:172.21.15.39:6841/4247190456] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564042edfdf0 con 0x564042b7e9f0 2024-09-21T03:52:09.244 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:09.236+0000 7f828e7fc640 1 --2- 172.21.15.39:0/3521849815 >> [v2:172.21.15.39:6840/4247190456,v1:172.21.15.39:6841/4247190456] conn(0x564042b7e9f0 0x564042f85060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:52:09.244 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:09.237+0000 7f826effd640 10 client.17034 ms_handle_connect on v2:172.21.15.39:6840/4247190456 2024-09-21T03:52:09.244 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:09.237+0000 7f828e7fc640 1 --2- 172.21.15.39:0/3521849815 >> [v2:172.21.15.39:6840/4247190456,v1:172.21.15.39:6841/4247190456] conn(0x564042b7e9f0 0x564042f85060 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=3a35949f3e8b539d server_cookie=8ae26c4102dae7cb in_seq=0 out_seq=0 2024-09-21T03:52:09.244 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:09.237+0000 7f826effd640 1 -- 172.21.15.39:0/3521849815 <== mds.4 v2:172.21.15.39:6840/4247190456 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x564042edfdf0 con 0x564042b7e9f0 2024-09-21T03:52:09.244 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:09.237+0000 7f826effd640 10 client.17034 handle_command_reply: tid=0 2024-09-21T03:52:09.244 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:09.237+0000 7f829be19740 1 client.17034 shutdown 2024-09-21T03:52:09.244 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:09.238+0000 7f829be19740 1 -- 172.21.15.39:0/3521849815 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564042b971b0 msgr2=0x564042f72e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:52:09.244 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:09.238+0000 7f829be19740 1 --2- 172.21.15.39:0/3521849815 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564042b971b0 0x564042f72e00 secure :-1 s=READY pgs=920 cs=0 l=1 rev1=1 crypto rx=0x7f8280003fd0 tx=0x7f8280005bd0 comp rx=0 tx=0).stop 2024-09-21T03:52:09.244 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:09.238+0000 7f829be19740 1 -- 172.21.15.39:0/3521849815 shutdown_connections 2024-09-21T03:52:09.245 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:09.238+0000 7f829be19740 1 --2- 172.21.15.39:0/3521849815 >> [v2:172.21.15.39:6840/4247190456,v1:172.21.15.39:6841/4247190456] conn(0x564042b7e9f0 0x564042f85060 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:09.245 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:09.238+0000 7f826effd640 0 client.17034 ms_handle_reset on v2:172.21.15.39:6840/4247190456 2024-09-21T03:52:09.245 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:09.238+0000 7f829be19740 1 --2- 172.21.15.39:0/3521849815 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564042bc0070 0x564042f7f170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:09.245 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:09.238+0000 7f829be19740 1 --2- 172.21.15.39:0/3521849815 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564042b9f390 0x564042f73340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:09.245 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:09.238+0000 7f829be19740 1 --2- 172.21.15.39:0/3521849815 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564042b971b0 0x564042f72e00 unknown :-1 s=CLOSED pgs=920 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:09.245 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:09.238+0000 7f829be19740 1 -- 172.21.15.39:0/3521849815 >> 172.21.15.39:0/3521849815 conn(0x564042bb4460 msgr2=0x564042f6a880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:52:09.245 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:09.238+0000 7f829be19740 1 -- 172.21.15.39:0/3521849815 shutdown_connections 2024-09-21T03:52:09.245 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:09.238+0000 7f829be19740 1 -- 172.21.15.39:0/3521849815 wait complete. 2024-09-21T03:52:09.245 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:09.238+0000 7f829be19740 20 client.17034 trim_cache size 0 max 16384 2024-09-21T03:52:09.255 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:09.251+0000 7f829be19740 1 Processor -- start 2024-09-21T03:52:09.255 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:09.251+0000 7f829be19740 1 -- start start 2024-09-21T03:52:09.255 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:09.251+0000 7f829be19740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564042b7e9f0 0x564042f68de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:52:09.255 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:09.251+0000 7f829be19740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564042b971b0 0x564042f6fff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:52:09.255 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:09.251+0000 7f829be19740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564042b9f390 0x564042f70530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:52:09.255 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:09.251+0000 7f829be19740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f82901af840 con 0x564042b9f390 2024-09-21T03:52:09.255 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:09.251+0000 7f829be19740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f82901b1380 con 0x564042b971b0 2024-09-21T03:52:09.255 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:09.251+0000 7f829be19740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f82901aff10 con 0x564042b7e9f0 2024-09-21T03:52:09.255 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:09.251+0000 7f828f7fe640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564042b9f390 0x564042f70530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:52:09.256 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:09.251+0000 7f828f7fe640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564042b9f390 0x564042f70530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:59820/0 (socket says 172.21.15.39:59820) 2024-09-21T03:52:09.256 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:09.251+0000 7f828f7fe640 1 -- 172.21.15.39:0/4072393316 learned_addr learned my addr 172.21.15.39:0/4072393316 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:52:09.256 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:09.252+0000 7f828e7fc640 1 --2- 172.21.15.39:0/4072393316 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564042b971b0 0x564042f6fff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:52:09.256 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:09.252+0000 7f828f7fe640 1 --2- 172.21.15.39:0/4072393316 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564042b7e9f0 0x564042f68de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:52:09.256 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:09.252+0000 7f828f7fe640 1 -- 172.21.15.39:0/4072393316 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564042b7e9f0 msgr2=0x564042f68de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:52:09.256 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:09.252+0000 7f828f7fe640 1 --2- 172.21.15.39:0/4072393316 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564042b7e9f0 0x564042f68de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:09.256 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:09.252+0000 7f828f7fe640 1 -- 172.21.15.39:0/4072393316 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564042b971b0 msgr2=0x564042f6fff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:52:09.256 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:09.252+0000 7f828f7fe640 1 --2- 172.21.15.39:0/4072393316 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564042b971b0 0x564042f6fff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:09.256 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:09.252+0000 7f828f7fe640 1 -- 172.21.15.39:0/4072393316 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564042f6a500 con 0x564042b9f390 2024-09-21T03:52:09.256 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:09.252+0000 7f828f7fe640 1 --2- 172.21.15.39:0/4072393316 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564042b9f390 0x564042f70530 secure :-1 s=READY pgs=921 cs=0 l=1 rev1=1 crypto rx=0x7f8280027890 tx=0x7f8280009500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b99a18ba813560fb server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:52:09.256 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:09.252+0000 7f82857fa640 1 -- 172.21.15.39:0/4072393316 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f828002b070 con 0x564042b9f390 2024-09-21T03:52:09.256 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:09.252+0000 7f82857fa640 1 -- 172.21.15.39:0/4072393316 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f828003f070 con 0x564042b9f390 2024-09-21T03:52:09.256 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:09.252+0000 7f82857fa640 1 -- 172.21.15.39:0/4072393316 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8280027a80 con 0x564042b9f390 2024-09-21T03:52:09.260 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:09.253+0000 7f829be19740 1 -- 172.21.15.39:0/4072393316 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564042b9f390 msgr2=0x564042f70530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:52:09.260 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:09.253+0000 7f829be19740 1 --2- 172.21.15.39:0/4072393316 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564042b9f390 0x564042f70530 secure :-1 s=READY pgs=921 cs=0 l=1 rev1=1 crypto rx=0x7f8280027890 tx=0x7f8280009500 comp rx=0 tx=0).stop 2024-09-21T03:52:09.260 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:09.253+0000 7f829be19740 1 -- 172.21.15.39:0/4072393316 shutdown_connections 2024-09-21T03:52:09.260 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:09.253+0000 7f829be19740 1 --2- 172.21.15.39:0/4072393316 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564042b9f390 0x564042f70530 unknown :-1 s=CLOSED pgs=921 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:09.260 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:09.253+0000 7f829be19740 1 --2- 172.21.15.39:0/4072393316 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564042b971b0 0x564042f6fff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:09.260 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:09.253+0000 7f829be19740 1 --2- 172.21.15.39:0/4072393316 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564042b7e9f0 0x564042f68de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:09.260 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:09.253+0000 7f829be19740 1 -- 172.21.15.39:0/4072393316 >> 172.21.15.39:0/4072393316 conn(0x564042bb4460 msgr2=0x564042f686d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:52:09.260 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:09.253+0000 7f829be19740 1 -- 172.21.15.39:0/4072393316 shutdown_connections 2024-09-21T03:52:09.260 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:09.253+0000 7f829be19740 1 -- 172.21.15.39:0/4072393316 wait complete. 2024-09-21T03:52:09.260 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:09.253+0000 7f829be19740 1 -- start start 2024-09-21T03:52:09.261 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:09.254+0000 7f829be19740 1 Processor -- start 2024-09-21T03:52:09.261 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:09.254+0000 7f829be19740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564042b7e9f0 0x564042f78a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:52:09.261 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:09.254+0000 7f829be19740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564042b971b0 0x564042f78fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:52:09.261 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:09.254+0000 7f829be19740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564042b9f390 0x564042fe78d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:52:09.261 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:09.254+0000 7f829be19740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f82901b0cb0 con 0x564042b7e9f0 2024-09-21T03:52:09.261 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:09.254+0000 7f829be19740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f82901b05e0 con 0x564042b9f390 2024-09-21T03:52:09.261 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:09.254+0000 7f829be19740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f82901ad820 con 0x564042b971b0 2024-09-21T03:52:09.261 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:09.254+0000 7f829be19740 10 client.0 _ll_register_callbacks cb 0x7f82880eaaa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:52:09.261 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:09.254+0000 7f828f7fe640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564042b7e9f0 0x564042f78a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:52:09.261 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:09.254+0000 7f828f7fe640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564042b7e9f0 0x564042f78a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:59830/0 (socket says 172.21.15.39:59830) 2024-09-21T03:52:09.261 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:09.254+0000 7f828f7fe640 1 -- 172.21.15.39:0/707426685 learned_addr learned my addr 172.21.15.39:0/707426685 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:52:09.261 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:09.254+0000 7f828e7fc640 1 --2- 172.21.15.39:0/707426685 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564042b971b0 0x564042f78fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:52:09.261 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:09.254+0000 7f828f7fe640 1 -- 172.21.15.39:0/707426685 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564042b971b0 msgr2=0x564042f78fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:52:09.261 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:09.254+0000 7f828f7fe640 1 --2- 172.21.15.39:0/707426685 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564042b971b0 0x564042f78fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:09.261 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:09.254+0000 7f828f7fe640 1 -- 172.21.15.39:0/707426685 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564042b9f390 msgr2=0x564042fe78d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:52:09.261 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:09.254+0000 7f828f7fe640 1 --2- 172.21.15.39:0/707426685 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564042b9f390 0x564042fe78d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:09.262 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:09.255+0000 7f828f7fe640 1 -- 172.21.15.39:0/707426685 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564042fe7f40 con 0x564042b7e9f0 2024-09-21T03:52:09.262 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:09.255+0000 7f828f7fe640 1 --2- 172.21.15.39:0/707426685 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564042b7e9f0 0x564042f78a20 secure :-1 s=READY pgs=922 cs=0 l=1 rev1=1 crypto rx=0x7f8280027930 tx=0x7f8280004200 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c9033aeec23d950 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:52:09.262 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:09.255+0000 7f826effd640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-09-21T03:52:09.262 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:09.255+0000 7f826effd640 1 -- 172.21.15.39:0/707426685 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8280048020 con 0x564042b7e9f0 2024-09-21T03:52:09.262 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:09.255+0000 7f826effd640 1 -- 172.21.15.39:0/707426685 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8484+0+0 (secure 0 0 0) 0x7f82800044e0 con 0x564042b7e9f0 2024-09-21T03:52:09.262 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:09.255+0000 7f829be19740 1 -- 172.21.15.39:0/707426685 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f828003f070 con 0x564042b7e9f0 2024-09-21T03:52:09.262 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:09.255+0000 7f826effd640 1 -- 172.21.15.39:0/707426685 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f828002b040 con 0x564042b7e9f0 2024-09-21T03:52:09.262 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:09.255+0000 7f829be19740 10 client.17040 fetch_fsmap learned FSMap version 29 2024-09-21T03:52:09.262 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:09.255+0000 7f829be19740 1 -- 172.21.15.39:0/707426685 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x564042fef260 con 0x564042b7e9f0 2024-09-21T03:52:09.262 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:09.255+0000 7f826effd640 1 -- 172.21.15.39:0/707426685 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f828003f070 con 0x564042b7e9f0 2024-09-21T03:52:09.262 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:09.256+0000 7f826effd640 1 -- 172.21.15.39:0/707426685 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f828000fad0 con 0x564042b7e9f0 2024-09-21T03:52:09.262 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:09.256+0000 7f829be19740 10 client.17040 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T03:52:09.262 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:09.256+0000 7f829be19740 10 client.17040 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.a 2024-09-21T03:52:09.262 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:09.256+0000 7f829be19740 20 client.17040 populate_metadata read hostname 'smithi039' 2024-09-21T03:52:09.262 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:09.256+0000 7f829be19740 1 --2- 172.21.15.39:0/707426685 >> [v2:172.21.15.39:6840/4247190456,v1:172.21.15.39:6841/4247190456] conn(0x564042bc0070 0x564042ff18b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:52:09.263 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:09.256+0000 7f829be19740 4 client.17040 mds_command: new command op to 14868 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T03:52:09.263 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:09.256+0000 7f829be19740 1 -- 172.21.15.39:0/707426685 --> [v2:172.21.15.39:6840/4247190456,v1:172.21.15.39:6841/4247190456] -- command(tid 0: {"prefix": "damage ls"}) -- 0x564042edb8a0 con 0x564042bc0070 2024-09-21T03:52:09.263 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:09.256+0000 7f828e7fc640 1 --2- 172.21.15.39:0/707426685 >> [v2:172.21.15.39:6840/4247190456,v1:172.21.15.39:6841/4247190456] conn(0x564042bc0070 0x564042ff18b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:52:09.263 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:09.256+0000 7f826effd640 10 client.17040 ms_handle_connect on v2:172.21.15.39:6840/4247190456 2024-09-21T03:52:09.263 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:09.256+0000 7f828e7fc640 1 --2- 172.21.15.39:0/707426685 >> [v2:172.21.15.39:6840/4247190456,v1:172.21.15.39:6841/4247190456] conn(0x564042bc0070 0x564042ff18b0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=b4ad54c66c7edb66 server_cookie=d89468808c63654a in_seq=0 out_seq=0 2024-09-21T03:52:09.263 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:09.256+0000 7f826effd640 1 -- 172.21.15.39:0/707426685 <== mds.4 v2:172.21.15.39:6840/4247190456 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x564042edb8a0 con 0x564042bc0070 2024-09-21T03:52:09.263 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:09.256+0000 7f826effd640 10 client.17040 handle_command_reply: tid=0 2024-09-21T03:52:09.263 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:09.256+0000 7f829be19740 1 client.17040 shutdown 2024-09-21T03:52:09.263 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:09.257+0000 7f829be19740 1 -- 172.21.15.39:0/707426685 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564042b7e9f0 msgr2=0x564042f78a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:52:09.263 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:09.257+0000 7f829be19740 1 --2- 172.21.15.39:0/707426685 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564042b7e9f0 0x564042f78a20 secure :-1 s=READY pgs=922 cs=0 l=1 rev1=1 crypto rx=0x7f8280027930 tx=0x7f8280004200 comp rx=0 tx=0).stop 2024-09-21T03:52:09.263 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:09.257+0000 7f829be19740 1 -- 172.21.15.39:0/707426685 shutdown_connections 2024-09-21T03:52:09.263 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:09.257+0000 7f829be19740 1 --2- 172.21.15.39:0/707426685 >> [v2:172.21.15.39:6840/4247190456,v1:172.21.15.39:6841/4247190456] conn(0x564042bc0070 0x564042ff18b0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:09.263 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:09.257+0000 7f829be19740 1 --2- 172.21.15.39:0/707426685 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564042b9f390 0x564042fe78d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:09.263 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:09.257+0000 7f829be19740 1 --2- 172.21.15.39:0/707426685 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564042b971b0 0x564042f78fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:09.263 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:09.257+0000 7f829be19740 1 --2- 172.21.15.39:0/707426685 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564042b7e9f0 0x564042f78a20 unknown :-1 s=CLOSED pgs=922 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:09.264 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:09.257+0000 7f829be19740 1 -- 172.21.15.39:0/707426685 >> 172.21.15.39:0/707426685 conn(0x564042bb4460 msgr2=0x564042f68600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:52:09.264 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:09.257+0000 7f826effd640 0 client.17040 ms_handle_reset on v2:172.21.15.39:6840/4247190456 2024-09-21T03:52:09.264 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:09.257+0000 7f829be19740 1 -- 172.21.15.39:0/707426685 shutdown_connections 2024-09-21T03:52:09.264 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:09.257+0000 7f829be19740 1 -- 172.21.15.39:0/707426685 wait complete. 2024-09-21T03:52:09.264 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:09.257+0000 7f829be19740 20 client.17040 trim_cache size 0 max 16384 2024-09-21T03:52:09.264 INFO:teuthology.orchestra.run.smithi039.stdout:[] 2024-09-21T03:52:09.265 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:09.257+0000 7f82958a5640 1 -- 172.21.15.39:0/4189240614 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f827c0b66b0 msgr2=0x7f827c0b8bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:52:09.265 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:09.257+0000 7f82958a5640 1 --2- 172.21.15.39:0/4189240614 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f827c0b66b0 0x7f827c0b8bc0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f82880e48a0 tx=0x7f82880eabd0 comp rx=0 tx=0).stop 2024-09-21T03:52:09.265 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:09.257+0000 7f82958a5640 1 -- 172.21.15.39:0/4189240614 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8290102a60 msgr2=0x7f829019b320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:52:09.265 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:09.257+0000 7f82958a5640 1 --2- 172.21.15.39:0/4189240614 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8290102a60 0x7f829019b320 secure :-1 s=READY pgs=918 cs=0 l=1 rev1=1 crypto rx=0x7f828800d950 tx=0x7f828800de10 comp rx=0 tx=0).stop 2024-09-21T03:52:09.265 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:09.258+0000 7f82958a5640 1 -- 172.21.15.39:0/4189240614 shutdown_connections 2024-09-21T03:52:09.265 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:09.258+0000 7f82958a5640 1 --2- 172.21.15.39:0/4189240614 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f827c0b66b0 0x7f827c0b8bc0 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:09.265 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:09.258+0000 7f82958a5640 1 --2- 172.21.15.39:0/4189240614 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8290108cf0 0x7f829019bda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:09.265 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:09.258+0000 7f82958a5640 1 --2- 172.21.15.39:0/4189240614 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f82901083a0 0x7f829019b860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:09.265 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:09.258+0000 7f82958a5640 1 --2- 172.21.15.39:0/4189240614 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8290102a60 0x7f829019b320 unknown :-1 s=CLOSED pgs=918 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:09.265 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:09.258+0000 7f82958a5640 1 -- 172.21.15.39:0/4189240614 >> 172.21.15.39:0/4189240614 conn(0x7f8290079730 msgr2=0x7f8290104470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:52:09.265 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:09.258+0000 7f82958a5640 1 -- 172.21.15.39:0/4189240614 shutdown_connections 2024-09-21T03:52:09.265 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:09.258+0000 7f82958a5640 1 -- 172.21.15.39:0/4189240614 wait complete. 2024-09-21T03:52:09.468 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:52:09 smithi161 ceph-mon[34533]: pgmap v508: 97 pgs: 97 active+clean; 10 MiB data, 520 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-09-21T03:52:09.475 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:52:09 smithi150 ceph-mon[34289]: pgmap v508: 97 pgs: 97 active+clean; 10 MiB data, 520 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-09-21T03:52:09.584 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:52:09 smithi039 ceph-mon[31211]: pgmap v508: 97 pgs: 97 active+clean; 10 MiB data, 520 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-09-21T03:52:10.237 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 51116d5c-77ca-11ef-baf5-efdc52797490 -- ceph tell mds.1:3 damage ls 2024-09-21T03:52:10.499 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/51116d5c-77ca-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:52:11.000 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:10.993+0000 7fdfd6aec640 1 -- 172.21.15.39:0/1343108798 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdfd0103b90 msgr2=0x7fdfd0103fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:52:11.000 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:10.993+0000 7fdfd6aec640 1 --2- 172.21.15.39:0/1343108798 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdfd0103b90 0x7fdfd0103fa0 secure :-1 s=READY pgs=923 cs=0 l=1 rev1=1 crypto rx=0x7fdfc0009900 tx=0x7fdfc002a3f0 comp rx=0 tx=0).stop 2024-09-21T03:52:11.001 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:10.993+0000 7fdfd6aec640 1 -- 172.21.15.39:0/1343108798 shutdown_connections 2024-09-21T03:52:11.001 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:10.993+0000 7fdfd6aec640 1 --2- 172.21.15.39:0/1343108798 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdfd0109e20 0x7fdfd0110650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:11.001 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:10.993+0000 7fdfd6aec640 1 --2- 172.21.15.39:0/1343108798 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdfd01094d0 0x7fdfd01098e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:11.001 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:10.993+0000 7fdfd6aec640 1 --2- 172.21.15.39:0/1343108798 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdfd0103b90 0x7fdfd0103fa0 unknown :-1 s=CLOSED pgs=923 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:11.001 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:10.993+0000 7fdfd6aec640 1 -- 172.21.15.39:0/1343108798 >> 172.21.15.39:0/1343108798 conn(0x7fdfd00ff600 msgr2=0x7fdfd0101a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:52:11.001 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:10.993+0000 7fdfd6aec640 1 -- 172.21.15.39:0/1343108798 shutdown_connections 2024-09-21T03:52:11.001 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:10.993+0000 7fdfd6aec640 1 -- 172.21.15.39:0/1343108798 wait complete. 2024-09-21T03:52:11.001 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:10.994+0000 7fdfd6aec640 1 Processor -- start 2024-09-21T03:52:11.001 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:10.994+0000 7fdfd6aec640 1 -- start start 2024-09-21T03:52:11.001 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:10.994+0000 7fdfd6aec640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdfd0103b90 0x7fdfd019a530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:52:11.001 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:10.994+0000 7fdfd6aec640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdfd01094d0 0x7fdfd019aad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:52:11.001 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:10.994+0000 7fdfd6aec640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdfd0109e20 0x7fdfd019fd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:52:11.001 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:10.994+0000 7fdfd6aec640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fdfd0113ae0 con 0x7fdfd01094d0 2024-09-21T03:52:11.002 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:10.994+0000 7fdfd6aec640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fdfd0113960 con 0x7fdfd0103b90 2024-09-21T03:52:11.002 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:10.994+0000 7fdfd6aec640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fdfd0113c60 con 0x7fdfd0109e20 2024-09-21T03:52:11.002 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:10.994+0000 7fdfcffff640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdfd01094d0 0x7fdfd019aad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:52:11.002 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:10.994+0000 7fdfcffff640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdfd01094d0 0x7fdfd019aad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:59844/0 (socket says 172.21.15.39:59844) 2024-09-21T03:52:11.002 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:10.994+0000 7fdfcffff640 1 -- 172.21.15.39:0/4282081589 learned_addr learned my addr 172.21.15.39:0/4282081589 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:52:11.002 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:10.994+0000 7fdfd4888640 1 --2- 172.21.15.39:0/4282081589 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdfd0103b90 0x7fdfd019a530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:52:11.002 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:10.994+0000 7fdfd5089640 1 --2- 172.21.15.39:0/4282081589 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdfd0109e20 0x7fdfd019fd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:52:11.002 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:10.994+0000 7fdfcffff640 1 -- 172.21.15.39:0/4282081589 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdfd0109e20 msgr2=0x7fdfd019fd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:52:11.002 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:10.994+0000 7fdfcffff640 1 --2- 172.21.15.39:0/4282081589 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdfd0109e20 0x7fdfd019fd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:11.002 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:10.994+0000 7fdfcffff640 1 -- 172.21.15.39:0/4282081589 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdfd0103b90 msgr2=0x7fdfd019a530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:52:11.002 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:10.994+0000 7fdfcffff640 1 --2- 172.21.15.39:0/4282081589 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdfd0103b90 0x7fdfd019a530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:11.002 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:10.994+0000 7fdfcffff640 1 -- 172.21.15.39:0/4282081589 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdfd01a03e0 con 0x7fdfd01094d0 2024-09-21T03:52:11.003 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:10.995+0000 7fdfd4888640 1 --2- 172.21.15.39:0/4282081589 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdfd0103b90 0x7fdfd019a530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:52:11.003 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:10.995+0000 7fdfcffff640 1 --2- 172.21.15.39:0/4282081589 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdfd01094d0 0x7fdfd019aad0 secure :-1 s=READY pgs=924 cs=0 l=1 rev1=1 crypto rx=0x7fdfc400c8c0 tx=0x7fdfc400cd80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:52:11.003 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:10.995+0000 7fdfcdffb640 1 -- 172.21.15.39:0/4282081589 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfc4007bb0 con 0x7fdfd01094d0 2024-09-21T03:52:11.003 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:10.995+0000 7fdfd6aec640 1 -- 172.21.15.39:0/4282081589 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdfd01a06d0 con 0x7fdfd01094d0 2024-09-21T03:52:11.003 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:10.995+0000 7fdfcdffb640 1 -- 172.21.15.39:0/4282081589 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdfc4007d50 con 0x7fdfd01094d0 2024-09-21T03:52:11.003 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:10.995+0000 7fdfcdffb640 1 -- 172.21.15.39:0/4282081589 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfc40187c0 con 0x7fdfd01094d0 2024-09-21T03:52:11.003 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:10.995+0000 7fdfd6aec640 1 -- 172.21.15.39:0/4282081589 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdfd01a6ef0 con 0x7fdfd01094d0 2024-09-21T03:52:11.003 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:10.998+0000 7fdfcdffb640 1 -- 172.21.15.39:0/4282081589 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 27) ==== 154643+0+0 (secure 0 0 0) 0x7fdfc400fa20 con 0x7fdfd01094d0 2024-09-21T03:52:11.003 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:10.998+0000 7fdfdd060740 1 Processor -- start 2024-09-21T03:52:11.003 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:10.998+0000 7fdfdd060740 1 -- start start 2024-09-21T03:52:11.006 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:10.998+0000 7fdfdd060740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ca53e6f070 0x55ca542162c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:52:11.007 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:10.998+0000 7fdfdd060740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ca53e461b0 0x55ca54216800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:52:11.007 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:10.998+0000 7fdfdd060740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ca53e4e390 0x55ca54216d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:52:11.007 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:10.998+0000 7fdfdd060740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x55ca5418cea0 con 0x55ca53e461b0 2024-09-21T03:52:11.007 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:10.998+0000 7fdfdd060740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x55ca5417c2d0 con 0x55ca53e6f070 2024-09-21T03:52:11.007 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:10.998+0000 7fdfdd060740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55ca541800e0 con 0x55ca53e4e390 2024-09-21T03:52:11.007 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:10.999+0000 7fdfcdffb640 1 --2- 172.21.15.39:0/4282081589 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7fdfb80beec0 0x7fdfb80c13d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:52:11.007 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:10.999+0000 7fdfd5089640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ca53e6f070 0x55ca542162c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:52:11.007 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:10.999+0000 7fdfd5089640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ca53e6f070 0x55ca542162c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.39:41538/0 (socket says 172.21.15.39:41538) 2024-09-21T03:52:11.007 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:10.999+0000 7fdfd5089640 1 -- 172.21.15.39:0/1023666459 learned_addr learned my addr 172.21.15.39:0/1023666459 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:52:11.007 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:10.999+0000 7fdfcdffb640 1 -- 172.21.15.39:0/4282081589 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8484+0+0 (secure 0 0 0) 0x7fdfc40e7870 con 0x7fdfd01094d0 2024-09-21T03:52:11.007 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:10.999+0000 7fdfcffff640 1 --2- 172.21.15.39:0/1023666459 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ca53e4e390 0x55ca54216d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:52:11.007 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:10.999+0000 7fdfd4888640 1 --2- 172.21.15.39:0/1023666459 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ca53e461b0 0x55ca54216800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:52:11.007 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:10.999+0000 7fdfd5089640 1 -- 172.21.15.39:0/1023666459 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ca53e4e390 msgr2=0x55ca54216d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:52:11.007 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:10.999+0000 7fdfd5089640 1 --2- 172.21.15.39:0/1023666459 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ca53e4e390 0x55ca54216d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:11.008 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:10.999+0000 7fdfd5089640 1 -- 172.21.15.39:0/1023666459 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ca53e461b0 msgr2=0x55ca54216800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:52:11.008 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:10.999+0000 7fdfd5089640 1 --2- 172.21.15.39:0/1023666459 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ca53e461b0 0x55ca54216800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:11.008 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:10.999+0000 7fdfd5089640 1 -- 172.21.15.39:0/1023666459 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ca53eda650 con 0x55ca53e6f070 2024-09-21T03:52:11.008 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:10.999+0000 7fdfcffff640 1 --2- 172.21.15.39:0/1023666459 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ca53e4e390 0x55ca54216d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:52:11.008 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:10.999+0000 7fdfd4888640 1 --2- 172.21.15.39:0/1023666459 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ca53e461b0 0x55ca54216800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:52:11.008 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:10.999+0000 7fdfd5089640 1 --2- 172.21.15.39:0/4282081589 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7fdfb80beec0 0x7fdfb80c13d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:52:11.008 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:10.999+0000 7fdfd5089640 1 --2- 172.21.15.39:0/1023666459 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ca53e6f070 0x55ca542162c0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fdfc8008e60 tx=0x7fdfc8010740 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9a2f611ad0654c81 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:52:11.008 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:11.000+0000 7fdfb6ffd640 1 -- 172.21.15.39:0/1023666459 <== mon.2 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfc8022070 con 0x55ca53e6f070 2024-09-21T03:52:11.008 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:11.000+0000 7fdfb6ffd640 1 -- 172.21.15.39:0/1023666459 <== mon.2 v2:172.21.15.150:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdfc801c470 con 0x55ca53e6f070 2024-09-21T03:52:11.008 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:11.000+0000 7fdfb6ffd640 1 -- 172.21.15.39:0/1023666459 <== mon.2 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfc801a620 con 0x55ca53e6f070 2024-09-21T03:52:11.008 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:11.001+0000 7fdfdd060740 1 -- 172.21.15.39:0/1023666459 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ca53e6f070 msgr2=0x55ca542162c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:52:11.008 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:11.001+0000 7fdfd5089640 1 --2- 172.21.15.39:0/4282081589 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7fdfb80beec0 0x7fdfb80c13d0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fdfc801f000 tx=0x7fdfc8020000 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:52:11.008 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:11.001+0000 7fdfdd060740 1 --2- 172.21.15.39:0/1023666459 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ca53e6f070 0x55ca542162c0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fdfc8008e60 tx=0x7fdfc8010740 comp rx=0 tx=0).stop 2024-09-21T03:52:11.008 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:11.001+0000 7fdfdd060740 1 -- 172.21.15.39:0/1023666459 shutdown_connections 2024-09-21T03:52:11.009 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:11.001+0000 7fdfdd060740 1 --2- 172.21.15.39:0/1023666459 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ca53e4e390 0x55ca54216d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:11.009 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:11.001+0000 7fdfdd060740 1 --2- 172.21.15.39:0/1023666459 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ca53e461b0 0x55ca54216800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:11.009 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:11.001+0000 7fdfdd060740 1 --2- 172.21.15.39:0/1023666459 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ca53e6f070 0x55ca542162c0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:11.009 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:11.001+0000 7fdfdd060740 1 -- 172.21.15.39:0/1023666459 >> 172.21.15.39:0/1023666459 conn(0x55ca53e63460 msgr2=0x55ca542141b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:52:11.009 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:11.001+0000 7fdfdd060740 1 -- 172.21.15.39:0/1023666459 shutdown_connections 2024-09-21T03:52:11.009 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:11.001+0000 7fdfdd060740 1 -- 172.21.15.39:0/1023666459 wait complete. 2024-09-21T03:52:11.009 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:11.001+0000 7fdfdd060740 1 -- start start 2024-09-21T03:52:11.009 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:11.001+0000 7fdfdd060740 1 Processor -- start 2024-09-21T03:52:11.009 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:11.002+0000 7fdfdd060740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ca53e461b0 0x55ca54221ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:52:11.009 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:11.002+0000 7fdfdd060740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ca53e4e390 0x55ca54222480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:52:11.009 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:11.002+0000 7fdfdd060740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ca53e6f070 0x55ca54227670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:52:11.009 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:11.002+0000 7fdfdd060740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x55ca53fdd700 con 0x55ca53e6f070 2024-09-21T03:52:11.009 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:11.002+0000 7fdfdd060740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x55ca54117160 con 0x55ca53e461b0 2024-09-21T03:52:11.009 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:11.002+0000 7fdfdd060740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55ca5421c830 con 0x55ca53e4e390 2024-09-21T03:52:11.010 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:11.002+0000 7fdfdd060740 10 client.0 _ll_register_callbacks cb 0x55ca541309e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:52:11.010 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:11.002+0000 7fdfd5089640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ca53e6f070 0x55ca54227670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:52:11.010 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:11.002+0000 7fdfd4888640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ca53e461b0 0x55ca54221ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:52:11.010 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:11.002+0000 7fdfd5089640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ca53e6f070 0x55ca54227670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:59858/0 (socket says 172.21.15.39:59858) 2024-09-21T03:52:11.010 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:11.002+0000 7fdfcffff640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ca53e4e390 0x55ca54222480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:52:11.010 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:11.002+0000 7fdfd5089640 1 -- 172.21.15.39:0/1658922262 learned_addr learned my addr 172.21.15.39:0/1658922262 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:52:11.010 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:11.002+0000 7fdfd5089640 1 -- 172.21.15.39:0/1658922262 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ca53e4e390 msgr2=0x55ca54222480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:52:11.010 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:11.002+0000 7fdfd5089640 1 --2- 172.21.15.39:0/1658922262 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ca53e4e390 0x55ca54222480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:11.010 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:11.002+0000 7fdfd5089640 1 -- 172.21.15.39:0/1658922262 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ca53e461b0 msgr2=0x55ca54221ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:52:11.010 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:11.002+0000 7fdfd5089640 1 --2- 172.21.15.39:0/1658922262 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ca53e461b0 0x55ca54221ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:11.010 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:11.002+0000 7fdfd5089640 1 -- 172.21.15.39:0/1658922262 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ca54144c90 con 0x55ca53e6f070 2024-09-21T03:52:11.010 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:11.002+0000 7fdfd4888640 1 --2- 172.21.15.39:0/1658922262 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ca53e461b0 0x55ca54221ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:52:11.010 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:11.003+0000 7fdfd5089640 1 --2- 172.21.15.39:0/1658922262 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ca53e6f070 0x55ca54227670 secure :-1 s=READY pgs=925 cs=0 l=1 rev1=1 crypto rx=0x7fdfc801a6c0 tx=0x7fdfc801a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc3177ce569eaf22 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:52:11.010 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:11.003+0000 7fdfaffff640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-09-21T03:52:11.011 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:11.003+0000 7fdfaffff640 1 -- 172.21.15.39:0/1658922262 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfc8022070 con 0x55ca53e6f070 2024-09-21T03:52:11.011 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:11.003+0000 7fdfaffff640 1 -- 172.21.15.39:0/1658922262 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8484+0+0 (secure 0 0 0) 0x7fdfc801a940 con 0x55ca53e6f070 2024-09-21T03:52:11.011 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:11.003+0000 7fdfdd060740 1 -- 172.21.15.39:0/1658922262 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdfc801c470 con 0x55ca53e6f070 2024-09-21T03:52:11.011 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:11.003+0000 7fdfaffff640 1 -- 172.21.15.39:0/1658922262 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fdfc801ab40 con 0x55ca53e6f070 2024-09-21T03:52:11.011 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:11.003+0000 7fdfdd060740 10 client.17049 fetch_fsmap learned FSMap version 29 2024-09-21T03:52:11.011 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:11.003+0000 7fdfdd060740 1 -- 172.21.15.39:0/1658922262 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55ca54230660 con 0x55ca53e6f070 2024-09-21T03:52:11.011 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:11.003+0000 7fdfaffff640 1 -- 172.21.15.39:0/1658922262 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdfc801c470 con 0x55ca53e6f070 2024-09-21T03:52:11.011 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:11.004+0000 7fdfaffff640 1 -- 172.21.15.39:0/1658922262 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fdfc8038020 con 0x55ca53e6f070 2024-09-21T03:52:11.011 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:11.004+0000 7fdfdd060740 10 client.17049 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T03:52:11.011 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:11.004+0000 7fdfdd060740 10 client.17049 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-21T03:52:11.011 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:11.004+0000 7fdfdd060740 20 client.17049 populate_metadata read hostname 'smithi039' 2024-09-21T03:52:11.011 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:11.004+0000 7fdfdd060740 1 --2- 172.21.15.39:0/1658922262 >> [v2:172.21.15.161:6836/3451544597,v1:172.21.15.161:6837/3451544597] conn(0x55ca53e2d9f0 0x55ca542339c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:52:11.011 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:11.004+0000 7fdfdd060740 4 client.17049 mds_command: new command op to 24646 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:52:11.012 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:11.004+0000 7fdfdd060740 1 -- 172.21.15.39:0/1658922262 --> [v2:172.21.15.161:6836/3451544597,v1:172.21.15.161:6837/3451544597] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ca5418edf0 con 0x55ca53e2d9f0 2024-09-21T03:52:11.012 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:11.004+0000 7fdfd4888640 1 --2- 172.21.15.39:0/1658922262 >> [v2:172.21.15.161:6836/3451544597,v1:172.21.15.161:6837/3451544597] conn(0x55ca53e2d9f0 0x55ca542339c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:52:11.012 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:11.004+0000 7fdfaffff640 10 client.17049 ms_handle_connect on v2:172.21.15.161:6836/3451544597 2024-09-21T03:52:11.012 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:11.004+0000 7fdfd4888640 1 --2- 172.21.15.39:0/1658922262 >> [v2:172.21.15.161:6836/3451544597,v1:172.21.15.161:6837/3451544597] conn(0x55ca53e2d9f0 0x55ca542339c0 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=cf479f0e111b8927 server_cookie=59f2ac732e6656c0 in_seq=0 out_seq=0 2024-09-21T03:52:11.013 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:11.005+0000 7fdfaffff640 1 -- 172.21.15.39:0/1658922262 <== mds.3 v2:172.21.15.161:6836/3451544597 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55ca5418edf0 con 0x55ca53e2d9f0 2024-09-21T03:52:11.013 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:11.005+0000 7fdfaffff640 10 client.17049 handle_command_reply: tid=0 2024-09-21T03:52:11.013 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:11.005+0000 7fdfdd060740 1 client.17049 shutdown 2024-09-21T03:52:11.013 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:11.005+0000 7fdfdd060740 1 -- 172.21.15.39:0/1658922262 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ca53e6f070 msgr2=0x55ca54227670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:52:11.013 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:11.005+0000 7fdfdd060740 1 --2- 172.21.15.39:0/1658922262 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ca53e6f070 0x55ca54227670 secure :-1 s=READY pgs=925 cs=0 l=1 rev1=1 crypto rx=0x7fdfc801a6c0 tx=0x7fdfc801a6f0 comp rx=0 tx=0).stop 2024-09-21T03:52:11.013 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:11.005+0000 7fdfdd060740 1 -- 172.21.15.39:0/1658922262 shutdown_connections 2024-09-21T03:52:11.013 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:11.005+0000 7fdfdd060740 1 --2- 172.21.15.39:0/1658922262 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ca53e6f070 0x55ca54227670 unknown :-1 s=CLOSED pgs=925 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:11.013 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:11.005+0000 7fdfdd060740 1 --2- 172.21.15.39:0/1658922262 >> [v2:172.21.15.161:6836/3451544597,v1:172.21.15.161:6837/3451544597] conn(0x55ca53e2d9f0 0x55ca542339c0 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:11.013 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:11.005+0000 7fdfdd060740 1 --2- 172.21.15.39:0/1658922262 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ca53e4e390 0x55ca54222480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:11.013 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:11.005+0000 7fdfaffff640 0 client.17049 ms_handle_reset on v2:172.21.15.161:6836/3451544597 2024-09-21T03:52:11.013 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:11.005+0000 7fdfdd060740 1 --2- 172.21.15.39:0/1658922262 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ca53e461b0 0x55ca54221ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:11.014 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:11.005+0000 7fdfdd060740 1 -- 172.21.15.39:0/1658922262 >> 172.21.15.39:0/1658922262 conn(0x55ca53e63460 msgr2=0x55ca54212750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:52:11.014 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:11.006+0000 7fdfdd060740 1 -- 172.21.15.39:0/1658922262 shutdown_connections 2024-09-21T03:52:11.014 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:11.006+0000 7fdfdd060740 1 -- 172.21.15.39:0/1658922262 wait complete. 2024-09-21T03:52:11.014 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:11.006+0000 7fdfdd060740 20 client.17049 trim_cache size 0 max 16384 2024-09-21T03:52:11.021 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:11.019+0000 7fdfdd060740 1 Processor -- start 2024-09-21T03:52:11.021 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:11.019+0000 7fdfdd060740 1 -- start start 2024-09-21T03:52:11.022 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:11.019+0000 7fdfdd060740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ca53e461b0 0x55ca542142a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:52:11.022 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:11.019+0000 7fdfdd060740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ca53e4e390 0x55ca54226c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:52:11.022 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:11.019+0000 7fdfdd060740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ca53e6f070 0x55ca542172f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:52:11.022 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:11.019+0000 7fdfdd060740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fdfd01ad520 con 0x55ca53e4e390 2024-09-21T03:52:11.022 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:11.019+0000 7fdfdd060740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fdfd01af060 con 0x55ca53e461b0 2024-09-21T03:52:11.022 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:11.019+0000 7fdfdd060740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fdfd01adbf0 con 0x55ca53e6f070 2024-09-21T03:52:11.022 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:11.019+0000 7fdfcffff640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ca53e4e390 0x55ca54226c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:52:11.023 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:11.019+0000 7fdfcffff640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ca53e4e390 0x55ca54226c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:59874/0 (socket says 172.21.15.39:59874) 2024-09-21T03:52:11.023 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:11.019+0000 7fdfcffff640 1 -- 172.21.15.39:0/4216293774 learned_addr learned my addr 172.21.15.39:0/4216293774 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:52:11.023 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:11.020+0000 7fdfd4888640 1 --2- 172.21.15.39:0/4216293774 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ca53e461b0 0x55ca542142a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:52:11.023 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:11.020+0000 7fdfd5089640 1 --2- 172.21.15.39:0/4216293774 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ca53e6f070 0x55ca542172f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:52:11.023 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:11.020+0000 7fdfcffff640 1 -- 172.21.15.39:0/4216293774 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ca53e6f070 msgr2=0x55ca542172f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:52:11.023 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:11.020+0000 7fdfcffff640 1 --2- 172.21.15.39:0/4216293774 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ca53e6f070 0x55ca542172f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:11.023 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:11.020+0000 7fdfcffff640 1 -- 172.21.15.39:0/4216293774 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ca53e461b0 msgr2=0x55ca542142a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:52:11.023 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:11.020+0000 7fdfcffff640 1 --2- 172.21.15.39:0/4216293774 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ca53e461b0 0x55ca542142a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:11.023 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:11.020+0000 7fdfcffff640 1 -- 172.21.15.39:0/4216293774 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ca542203e0 con 0x55ca53e4e390 2024-09-21T03:52:11.023 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:11.020+0000 7fdfcffff640 1 --2- 172.21.15.39:0/4216293774 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ca53e4e390 0x55ca54226c50 secure :-1 s=READY pgs=926 cs=0 l=1 rev1=1 crypto rx=0x7fdfc4000c00 tx=0x7fdfc40a9d90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5cd41278dac2a7cb server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:52:11.023 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:11.020+0000 7fdfb67fc640 1 -- 172.21.15.39:0/4216293774 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfc4085640 con 0x55ca53e4e390 2024-09-21T03:52:11.023 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:11.020+0000 7fdfb67fc640 1 -- 172.21.15.39:0/4216293774 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdfc40b21a0 con 0x55ca53e4e390 2024-09-21T03:52:11.023 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:11.020+0000 7fdfb67fc640 1 -- 172.21.15.39:0/4216293774 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfc40b6c50 con 0x55ca53e4e390 2024-09-21T03:52:11.026 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:11.021+0000 7fdfdd060740 1 -- 172.21.15.39:0/4216293774 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ca53e4e390 msgr2=0x55ca54226c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:52:11.027 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:11.021+0000 7fdfdd060740 1 --2- 172.21.15.39:0/4216293774 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ca53e4e390 0x55ca54226c50 secure :-1 s=READY pgs=926 cs=0 l=1 rev1=1 crypto rx=0x7fdfc4000c00 tx=0x7fdfc40a9d90 comp rx=0 tx=0).stop 2024-09-21T03:52:11.027 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:11.021+0000 7fdfdd060740 1 -- 172.21.15.39:0/4216293774 shutdown_connections 2024-09-21T03:52:11.027 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:11.021+0000 7fdfdd060740 1 --2- 172.21.15.39:0/4216293774 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ca53e6f070 0x55ca542172f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:11.027 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:11.021+0000 7fdfdd060740 1 --2- 172.21.15.39:0/4216293774 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ca53e4e390 0x55ca54226c50 unknown :-1 s=CLOSED pgs=926 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:11.027 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:11.021+0000 7fdfdd060740 1 --2- 172.21.15.39:0/4216293774 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ca53e461b0 0x55ca542142a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:11.027 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:11.021+0000 7fdfdd060740 1 -- 172.21.15.39:0/4216293774 >> 172.21.15.39:0/4216293774 conn(0x55ca53e63460 msgr2=0x55ca54226540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:52:11.027 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:11.021+0000 7fdfdd060740 1 -- 172.21.15.39:0/4216293774 shutdown_connections 2024-09-21T03:52:11.027 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:11.021+0000 7fdfdd060740 1 -- 172.21.15.39:0/4216293774 wait complete. 2024-09-21T03:52:11.027 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:11.021+0000 7fdfdd060740 1 -- start start 2024-09-21T03:52:11.027 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:11.021+0000 7fdfdd060740 1 Processor -- start 2024-09-21T03:52:11.027 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:11.022+0000 7fdfdd060740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ca53e461b0 0x55ca54239a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:52:11.027 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:11.022+0000 7fdfdd060740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ca53e4e390 0x55ca54228bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:52:11.027 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:11.022+0000 7fdfdd060740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ca53e6f070 0x55ca542292d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:52:11.027 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:11.022+0000 7fdfdd060740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fdfd01ae990 con 0x55ca53e6f070 2024-09-21T03:52:11.028 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:11.022+0000 7fdfdd060740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fdfd01ae2c0 con 0x55ca53e4e390 2024-09-21T03:52:11.028 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:11.022+0000 7fdfdd060740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fdfd01ab520 con 0x55ca53e461b0 2024-09-21T03:52:11.028 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:11.022+0000 7fdfdd060740 10 client.0 _ll_register_callbacks cb 0x7fdfc8020070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:52:11.028 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:11.022+0000 7fdfd5089640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ca53e6f070 0x55ca542292d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:52:11.028 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:11.022+0000 7fdfd5089640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ca53e6f070 0x55ca542292d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:59880/0 (socket says 172.21.15.39:59880) 2024-09-21T03:52:11.028 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:11.022+0000 7fdfd5089640 1 -- 172.21.15.39:0/2644989791 learned_addr learned my addr 172.21.15.39:0/2644989791 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:52:11.028 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:11.022+0000 7fdfd4888640 1 --2- 172.21.15.39:0/2644989791 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ca53e461b0 0x55ca54239a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:52:11.028 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:11.022+0000 7fdfcffff640 1 --2- 172.21.15.39:0/2644989791 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ca53e4e390 0x55ca54228bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:52:11.028 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:11.022+0000 7fdfd5089640 1 -- 172.21.15.39:0/2644989791 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ca53e461b0 msgr2=0x55ca54239a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:52:11.028 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:11.022+0000 7fdfd5089640 1 --2- 172.21.15.39:0/2644989791 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ca53e461b0 0x55ca54239a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:11.028 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:11.022+0000 7fdfd5089640 1 -- 172.21.15.39:0/2644989791 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ca53e4e390 msgr2=0x55ca54228bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:52:11.028 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:11.022+0000 7fdfd5089640 1 --2- 172.21.15.39:0/2644989791 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ca53e4e390 0x55ca54228bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:11.028 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:11.022+0000 7fdfd5089640 1 -- 172.21.15.39:0/2644989791 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ca5422dc10 con 0x55ca53e6f070 2024-09-21T03:52:11.028 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:11.023+0000 7fdfd5089640 1 --2- 172.21.15.39:0/2644989791 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ca53e6f070 0x55ca542292d0 secure :-1 s=READY pgs=927 cs=0 l=1 rev1=1 crypto rx=0x7fdfc8011010 tx=0x7fdfc8010cc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=445a79c50e784ea8 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:52:11.028 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:11.023+0000 7fdfaffff640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-09-21T03:52:11.029 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:11.023+0000 7fdfaffff640 1 -- 172.21.15.39:0/2644989791 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfc8022070 con 0x55ca53e6f070 2024-09-21T03:52:11.029 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:11.023+0000 7fdfaffff640 1 -- 172.21.15.39:0/2644989791 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8484+0+0 (secure 0 0 0) 0x7fdfc801c470 con 0x55ca53e6f070 2024-09-21T03:52:11.029 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:11.023+0000 7fdfdd060740 1 -- 172.21.15.39:0/2644989791 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdfc40b21a0 con 0x55ca53e6f070 2024-09-21T03:52:11.029 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:11.023+0000 7fdfaffff640 1 -- 172.21.15.39:0/2644989791 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fdfc80084a0 con 0x55ca53e6f070 2024-09-21T03:52:11.029 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:11.023+0000 7fdfdd060740 10 client.17055 fetch_fsmap learned FSMap version 29 2024-09-21T03:52:11.029 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:11.023+0000 7fdfdd060740 1 -- 172.21.15.39:0/2644989791 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55ca5429e3f0 con 0x55ca53e6f070 2024-09-21T03:52:11.029 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:11.023+0000 7fdfaffff640 1 -- 172.21.15.39:0/2644989791 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdfc40b21a0 con 0x55ca53e6f070 2024-09-21T03:52:11.029 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:11.023+0000 7fdfaffff640 1 -- 172.21.15.39:0/2644989791 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fdfc8041070 con 0x55ca53e6f070 2024-09-21T03:52:11.029 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:11.023+0000 7fdfdd060740 10 client.17055 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T03:52:11.029 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:11.023+0000 7fdfdd060740 10 client.17055 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-21T03:52:11.029 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:11.023+0000 7fdfdd060740 20 client.17055 populate_metadata read hostname 'smithi039' 2024-09-21T03:52:11.029 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:11.023+0000 7fdfdd060740 1 --2- 172.21.15.39:0/2644989791 >> [v2:172.21.15.161:6836/3451544597,v1:172.21.15.161:6837/3451544597] conn(0x55ca53e02da0 0x55ca542a0bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:52:11.029 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:11.023+0000 7fdfdd060740 4 client.17055 mds_command: new command op to 24646 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T03:52:11.029 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:11.023+0000 7fdfdd060740 1 -- 172.21.15.39:0/2644989791 --> [v2:172.21.15.161:6836/3451544597,v1:172.21.15.161:6837/3451544597] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55ca5418a8a0 con 0x55ca53e02da0 2024-09-21T03:52:11.029 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:11.024+0000 7fdfd4888640 1 --2- 172.21.15.39:0/2644989791 >> [v2:172.21.15.161:6836/3451544597,v1:172.21.15.161:6837/3451544597] conn(0x55ca53e02da0 0x55ca542a0bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:52:11.030 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:11.024+0000 7fdfd4888640 1 --2- 172.21.15.39:0/2644989791 >> [v2:172.21.15.161:6836/3451544597,v1:172.21.15.161:6837/3451544597] conn(0x55ca53e02da0 0x55ca542a0bb0 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=a93f3944c6975618 server_cookie=4eeceda18e17acbd in_seq=0 out_seq=0 2024-09-21T03:52:11.030 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:11.024+0000 7fdfaffff640 10 client.17055 ms_handle_connect on v2:172.21.15.161:6836/3451544597 2024-09-21T03:52:11.030 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:11.024+0000 7fdfaffff640 1 -- 172.21.15.39:0/2644989791 <== mds.3 v2:172.21.15.161:6836/3451544597 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55ca5418a8a0 con 0x55ca53e02da0 2024-09-21T03:52:11.030 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:11.024+0000 7fdfaffff640 10 client.17055 handle_command_reply: tid=0 2024-09-21T03:52:11.030 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:11.024+0000 7fdfdd060740 1 client.17055 shutdown 2024-09-21T03:52:11.030 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:11.024+0000 7fdfdd060740 1 -- 172.21.15.39:0/2644989791 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ca53e6f070 msgr2=0x55ca542292d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:52:11.030 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:11.024+0000 7fdfdd060740 1 --2- 172.21.15.39:0/2644989791 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ca53e6f070 0x55ca542292d0 secure :-1 s=READY pgs=927 cs=0 l=1 rev1=1 crypto rx=0x7fdfc8011010 tx=0x7fdfc8010cc0 comp rx=0 tx=0).stop 2024-09-21T03:52:11.030 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:11.025+0000 7fdfdd060740 1 -- 172.21.15.39:0/2644989791 shutdown_connections 2024-09-21T03:52:11.030 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:11.025+0000 7fdfdd060740 1 --2- 172.21.15.39:0/2644989791 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ca53e6f070 0x55ca542292d0 unknown :-1 s=CLOSED pgs=927 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:11.030 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:11.025+0000 7fdfdd060740 1 --2- 172.21.15.39:0/2644989791 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ca53e4e390 0x55ca54228bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:11.030 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:11.025+0000 7fdfdd060740 1 --2- 172.21.15.39:0/2644989791 >> [v2:172.21.15.161:6836/3451544597,v1:172.21.15.161:6837/3451544597] conn(0x55ca53e02da0 0x55ca542a0bb0 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:11.030 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:11.025+0000 7fdfdd060740 1 --2- 172.21.15.39:0/2644989791 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ca53e461b0 0x55ca54239a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:11.030 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:11.025+0000 7fdfaffff640 0 client.17055 ms_handle_reset on v2:172.21.15.161:6836/3451544597 2024-09-21T03:52:11.030 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:11.025+0000 7fdfdd060740 1 -- 172.21.15.39:0/2644989791 >> 172.21.15.39:0/2644989791 conn(0x55ca53e63460 msgr2=0x55ca542244a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:52:11.031 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:11.025+0000 7fdfdd060740 1 -- 172.21.15.39:0/2644989791 shutdown_connections 2024-09-21T03:52:11.031 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:11.025+0000 7fdfdd060740 1 -- 172.21.15.39:0/2644989791 wait complete. 2024-09-21T03:52:11.031 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:11.025+0000 7fdfdd060740 20 client.17055 trim_cache size 0 max 16384 2024-09-21T03:52:11.031 INFO:teuthology.orchestra.run.smithi039.stdout:[] 2024-09-21T03:52:11.031 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:11.025+0000 7fdfd6aec640 1 -- 172.21.15.39:0/4282081589 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7fdfb80beec0 msgr2=0x7fdfb80c13d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:52:11.032 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:11.025+0000 7fdfd6aec640 1 --2- 172.21.15.39:0/4282081589 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7fdfb80beec0 0x7fdfb80c13d0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fdfc801f000 tx=0x7fdfc8020000 comp rx=0 tx=0).stop 2024-09-21T03:52:11.032 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:11.025+0000 7fdfd6aec640 1 -- 172.21.15.39:0/4282081589 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdfd01094d0 msgr2=0x7fdfd019aad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:52:11.032 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:11.025+0000 7fdfd6aec640 1 --2- 172.21.15.39:0/4282081589 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdfd01094d0 0x7fdfd019aad0 secure :-1 s=READY pgs=924 cs=0 l=1 rev1=1 crypto rx=0x7fdfc400c8c0 tx=0x7fdfc400cd80 comp rx=0 tx=0).stop 2024-09-21T03:52:11.032 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:11.026+0000 7fdfd6aec640 1 -- 172.21.15.39:0/4282081589 shutdown_connections 2024-09-21T03:52:11.032 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:11.026+0000 7fdfd6aec640 1 --2- 172.21.15.39:0/4282081589 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7fdfb80beec0 0x7fdfb80c13d0 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:11.032 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:11.026+0000 7fdfd6aec640 1 --2- 172.21.15.39:0/4282081589 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdfd0109e20 0x7fdfd019fd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:11.032 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:11.026+0000 7fdfd6aec640 1 --2- 172.21.15.39:0/4282081589 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdfd01094d0 0x7fdfd019aad0 unknown :-1 s=CLOSED pgs=924 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:11.032 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:11.026+0000 7fdfd6aec640 1 --2- 172.21.15.39:0/4282081589 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdfd0103b90 0x7fdfd019a530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:11.032 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:11.026+0000 7fdfd6aec640 1 -- 172.21.15.39:0/4282081589 >> 172.21.15.39:0/4282081589 conn(0x7fdfd00ff600 msgr2=0x7fdfd0101a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:52:11.032 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:11.026+0000 7fdfd6aec640 1 -- 172.21.15.39:0/4282081589 shutdown_connections 2024-09-21T03:52:11.032 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:11.026+0000 7fdfd6aec640 1 -- 172.21.15.39:0/4282081589 wait complete. 2024-09-21T03:52:11.468 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:52:11 smithi161 ceph-mon[34533]: pgmap v509: 97 pgs: 97 active+clean; 10 MiB data, 520 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 22 KiB/s wr, 26 op/s 2024-09-21T03:52:11.474 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:52:11 smithi150 ceph-mon[34289]: pgmap v509: 97 pgs: 97 active+clean; 10 MiB data, 520 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 22 KiB/s wr, 26 op/s 2024-09-21T03:52:11.584 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:52:11 smithi039 ceph-mon[31211]: pgmap v509: 97 pgs: 97 active+clean; 10 MiB data, 520 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 22 KiB/s wr, 26 op/s 2024-09-21T03:52:11.819 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 51116d5c-77ca-11ef-baf5-efdc52797490 -- ceph tell mds.1:2 damage ls 2024-09-21T03:52:12.074 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/51116d5c-77ca-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:52:12.585 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:12.543+0000 7fd3b93b2640 1 -- 172.21.15.39:0/727192916 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd3b41094f0 msgr2=0x7fd3b4109900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:52:12.585 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:12.543+0000 7fd3b93b2640 1 --2- 172.21.15.39:0/727192916 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd3b41094f0 0x7fd3b4109900 secure :-1 s=READY pgs=928 cs=0 l=1 rev1=1 crypto rx=0x7fd3a8009900 tx=0x7fd3a802a410 comp rx=0 tx=0).stop 2024-09-21T03:52:12.585 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:12.544+0000 7fd3b93b2640 1 -- 172.21.15.39:0/727192916 shutdown_connections 2024-09-21T03:52:12.585 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:12.544+0000 7fd3b93b2640 1 --2- 172.21.15.39:0/727192916 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd3b4109e40 0x7fd3b4110670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:12.585 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:12.544+0000 7fd3b93b2640 1 --2- 172.21.15.39:0/727192916 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd3b41094f0 0x7fd3b4109900 unknown :-1 s=CLOSED pgs=928 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:12.585 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:12.544+0000 7fd3b93b2640 1 --2- 172.21.15.39:0/727192916 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd3b4103bb0 0x7fd3b4103fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:12.585 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:12.544+0000 7fd3b93b2640 1 -- 172.21.15.39:0/727192916 >> 172.21.15.39:0/727192916 conn(0x7fd3b40ff600 msgr2=0x7fd3b4101a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:52:12.585 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:12.544+0000 7fd3b93b2640 1 -- 172.21.15.39:0/727192916 shutdown_connections 2024-09-21T03:52:12.586 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:12.544+0000 7fd3b93b2640 1 -- 172.21.15.39:0/727192916 wait complete. 2024-09-21T03:52:12.586 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:12.544+0000 7fd3b93b2640 1 Processor -- start 2024-09-21T03:52:12.586 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:12.544+0000 7fd3b93b2640 1 -- start start 2024-09-21T03:52:12.586 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:12.545+0000 7fd3b93b2640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd3b4103bb0 0x7fd3b4075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:52:12.586 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:12.545+0000 7fd3b93b2640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd3b41094f0 0x7fd3b4075c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:52:12.586 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:12.545+0000 7fd3b93b2640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd3b4109e40 0x7fd3b4076190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:52:12.586 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:12.545+0000 7fd3b93b2640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fd3b4113b00 con 0x7fd3b4109e40 2024-09-21T03:52:12.586 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:12.545+0000 7fd3b93b2640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fd3b4113980 con 0x7fd3b4103bb0 2024-09-21T03:52:12.586 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:12.545+0000 7fd3b93b2640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fd3b4113c80 con 0x7fd3b41094f0 2024-09-21T03:52:12.586 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:12.545+0000 7fd3b37fe640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd3b4109e40 0x7fd3b4076190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:52:12.586 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:12.545+0000 7fd3b37fe640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd3b4109e40 0x7fd3b4076190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:59898/0 (socket says 172.21.15.39:59898) 2024-09-21T03:52:12.586 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:12.545+0000 7fd3b37fe640 1 -- 172.21.15.39:0/2273098140 learned_addr learned my addr 172.21.15.39:0/2273098140 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:52:12.586 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:12.545+0000 7fd3b27fc640 1 --2- 172.21.15.39:0/2273098140 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd3b41094f0 0x7fd3b4075c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:52:12.587 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:12.545+0000 7fd3b2ffd640 1 --2- 172.21.15.39:0/2273098140 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd3b4103bb0 0x7fd3b4075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:52:12.587 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:12.545+0000 7fd3b37fe640 1 -- 172.21.15.39:0/2273098140 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd3b41094f0 msgr2=0x7fd3b4075c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:52:12.587 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:12.545+0000 7fd3b37fe640 1 --2- 172.21.15.39:0/2273098140 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd3b41094f0 0x7fd3b4075c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:12.587 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:12.545+0000 7fd3b37fe640 1 -- 172.21.15.39:0/2273098140 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd3b4103bb0 msgr2=0x7fd3b4075710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:52:12.587 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:12.545+0000 7fd3b37fe640 1 --2- 172.21.15.39:0/2273098140 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd3b4103bb0 0x7fd3b4075710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:12.587 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:12.545+0000 7fd3b37fe640 1 -- 172.21.15.39:0/2273098140 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd3b41a87f0 con 0x7fd3b4109e40 2024-09-21T03:52:12.587 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:12.546+0000 7fd3b37fe640 1 --2- 172.21.15.39:0/2273098140 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd3b4109e40 0x7fd3b4076190 secure :-1 s=READY pgs=929 cs=0 l=1 rev1=1 crypto rx=0x7fd3ac00eb10 tx=0x7fd3ac00c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:52:12.587 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:12.546+0000 7fd39bfff640 1 -- 172.21.15.39:0/2273098140 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3ac004b20 con 0x7fd3b4109e40 2024-09-21T03:52:12.587 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:12.546+0000 7fd39bfff640 1 -- 172.21.15.39:0/2273098140 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd3ac004cc0 con 0x7fd3b4109e40 2024-09-21T03:52:12.587 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:12.546+0000 7fd3b93b2640 1 -- 172.21.15.39:0/2273098140 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd3b41a8ae0 con 0x7fd3b4109e40 2024-09-21T03:52:12.587 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:12.546+0000 7fd39bfff640 1 -- 172.21.15.39:0/2273098140 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3ac010670 con 0x7fd3b4109e40 2024-09-21T03:52:12.587 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:12.546+0000 7fd3b93b2640 1 -- 172.21.15.39:0/2273098140 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd3b41a8ff0 con 0x7fd3b4109e40 2024-09-21T03:52:12.587 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:12.548+0000 7fd3bf926740 1 Processor -- start 2024-09-21T03:52:12.588 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:12.548+0000 7fd39bfff640 1 -- 172.21.15.39:0/2273098140 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 27) ==== 154643+0+0 (secure 0 0 0) 0x7fd3ac005020 con 0x7fd3b4109e40 2024-09-21T03:52:12.588 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:12.549+0000 7fd3bf926740 1 -- start start 2024-09-21T03:52:12.588 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:12.549+0000 7fd3bf926740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560ef5fe1070 0x560ef63882c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:52:12.588 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:12.549+0000 7fd3bf926740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560ef5fb81b0 0x560ef6388800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:52:12.588 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:12.549+0000 7fd3bf926740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x560ef5fc0390 0x560ef6388d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:52:12.588 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:12.549+0000 7fd3bf926740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x560ef62feea0 con 0x560ef5fc0390 2024-09-21T03:52:12.588 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:12.549+0000 7fd3bf926740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x560ef62ee2d0 con 0x560ef5fe1070 2024-09-21T03:52:12.588 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:12.549+0000 7fd3bf926740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x560ef62f20e0 con 0x560ef5fb81b0 2024-09-21T03:52:12.588 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:12.549+0000 7fd3b27fc640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x560ef5fc0390 0x560ef6388d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:52:12.588 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:12.549+0000 7fd3b27fc640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x560ef5fc0390 0x560ef6388d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:59908/0 (socket says 172.21.15.39:59908) 2024-09-21T03:52:12.588 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:12.549+0000 7fd3b27fc640 1 -- 172.21.15.39:0/4118475727 learned_addr learned my addr 172.21.15.39:0/4118475727 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:52:12.588 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:12.549+0000 7fd3b2ffd640 1 --2- 172.21.15.39:0/4118475727 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560ef5fb81b0 0x560ef6388800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:52:12.588 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:12.549+0000 7fd3b27fc640 1 --2- 172.21.15.39:0/4118475727 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560ef5fe1070 0x560ef63882c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:52:12.588 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:12.549+0000 7fd39bfff640 1 --2- 172.21.15.39:0/2273098140 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7fd39c0b6680 0x7fd39c0b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:52:12.589 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:12.550+0000 7fd3b2ffd640 1 -- 172.21.15.39:0/4118475727 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560ef5fe1070 msgr2=0x560ef63882c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:52:12.589 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:12.550+0000 7fd3b2ffd640 1 --2- 172.21.15.39:0/4118475727 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560ef5fe1070 0x560ef63882c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:12.589 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:12.550+0000 7fd3b2ffd640 1 -- 172.21.15.39:0/4118475727 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x560ef5fc0390 msgr2=0x560ef6388d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:52:12.589 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:12.550+0000 7fd3b2ffd640 1 --2- 172.21.15.39:0/4118475727 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x560ef5fc0390 0x560ef6388d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:12.589 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:12.550+0000 7fd3b2ffd640 1 -- 172.21.15.39:0/4118475727 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560ef604c650 con 0x560ef5fb81b0 2024-09-21T03:52:12.589 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:12.550+0000 7fd3b27fc640 1 --2- 172.21.15.39:0/4118475727 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x560ef5fc0390 0x560ef6388d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:52:12.589 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:12.550+0000 7fd3b37fe640 1 --2- 172.21.15.39:0/2273098140 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7fd39c0b6680 0x7fd39c0b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:52:12.589 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:12.550+0000 7fd39bfff640 1 -- 172.21.15.39:0/2273098140 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8484+0+0 (secure 0 0 0) 0x7fd3ac0e7c20 con 0x7fd3b4109e40 2024-09-21T03:52:12.589 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:12.550+0000 7fd3b2ffd640 1 --2- 172.21.15.39:0/4118475727 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560ef5fb81b0 0x560ef6388800 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fd3a400bbc0 tx=0x7fd3a40107a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e8596b996bec6757 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:52:12.589 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:12.550+0000 7fd3997fa640 1 -- 172.21.15.39:0/4118475727 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3a4004030 con 0x560ef5fb81b0 2024-09-21T03:52:12.589 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:12.550+0000 7fd3997fa640 1 -- 172.21.15.39:0/4118475727 <== mon.1 v2:172.21.15.161:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd3a40041d0 con 0x560ef5fb81b0 2024-09-21T03:52:12.589 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:12.550+0000 7fd3997fa640 1 -- 172.21.15.39:0/4118475727 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3a40116a0 con 0x560ef5fb81b0 2024-09-21T03:52:12.589 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:12.551+0000 7fd3bf926740 1 -- 172.21.15.39:0/4118475727 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560ef5fb81b0 msgr2=0x560ef6388800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:52:12.589 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:12.551+0000 7fd3b37fe640 1 --2- 172.21.15.39:0/2273098140 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7fd39c0b6680 0x7fd39c0b8b90 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fd3ac0aa060 tx=0x7fd3ac095040 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:52:12.590 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:12.551+0000 7fd3bf926740 1 --2- 172.21.15.39:0/4118475727 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560ef5fb81b0 0x560ef6388800 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fd3a400bbc0 tx=0x7fd3a40107a0 comp rx=0 tx=0).stop 2024-09-21T03:52:12.590 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:12.551+0000 7fd3bf926740 1 -- 172.21.15.39:0/4118475727 shutdown_connections 2024-09-21T03:52:12.590 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:12.551+0000 7fd3bf926740 1 --2- 172.21.15.39:0/4118475727 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x560ef5fc0390 0x560ef6388d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:12.590 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:12.551+0000 7fd3bf926740 1 --2- 172.21.15.39:0/4118475727 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560ef5fb81b0 0x560ef6388800 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:12.590 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:12.551+0000 7fd3bf926740 1 --2- 172.21.15.39:0/4118475727 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560ef5fe1070 0x560ef63882c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:12.590 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:12.551+0000 7fd3bf926740 1 -- 172.21.15.39:0/4118475727 >> 172.21.15.39:0/4118475727 conn(0x560ef5fd5460 msgr2=0x560ef63861b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:52:12.590 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:12.551+0000 7fd3bf926740 1 -- 172.21.15.39:0/4118475727 shutdown_connections 2024-09-21T03:52:12.590 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:12.551+0000 7fd3bf926740 1 -- 172.21.15.39:0/4118475727 wait complete. 2024-09-21T03:52:12.590 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:12.551+0000 7fd3bf926740 1 -- start start 2024-09-21T03:52:12.590 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:12.552+0000 7fd3bf926740 1 Processor -- start 2024-09-21T03:52:12.590 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:12.552+0000 7fd3bf926740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560ef5fb81b0 0x560ef6393930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:52:12.590 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:12.552+0000 7fd3bf926740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x560ef5fc0390 0x560ef6393e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:52:12.590 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:12.552+0000 7fd3bf926740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560ef5fe1070 0x560ef639fdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:52:12.590 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:12.552+0000 7fd3bf926740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x560ef614f700 con 0x560ef5fc0390 2024-09-21T03:52:12.591 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:12.552+0000 7fd3bf926740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x560ef6289160 con 0x560ef5fe1070 2024-09-21T03:52:12.591 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:12.552+0000 7fd3bf926740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x560ef6396aa0 con 0x560ef5fb81b0 2024-09-21T03:52:12.591 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:12.552+0000 7fd3bf926740 10 client.0 _ll_register_callbacks cb 0x560ef62a29e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:52:12.591 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:12.552+0000 7fd3b2ffd640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x560ef5fc0390 0x560ef6393e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:52:12.591 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:12.552+0000 7fd3b2ffd640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x560ef5fc0390 0x560ef6393e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:59920/0 (socket says 172.21.15.39:59920) 2024-09-21T03:52:12.591 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:12.552+0000 7fd3b2ffd640 1 -- 172.21.15.39:0/2975215630 learned_addr learned my addr 172.21.15.39:0/2975215630 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:52:12.591 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:12.552+0000 7fd3b27fc640 1 --2- 172.21.15.39:0/2975215630 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560ef5fb81b0 0x560ef6393930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:52:12.591 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:12.552+0000 7fd3b27fc640 1 --2- 172.21.15.39:0/2975215630 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560ef5fe1070 0x560ef639fdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:52:12.591 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:12.552+0000 7fd3b2ffd640 1 -- 172.21.15.39:0/2975215630 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560ef5fb81b0 msgr2=0x560ef6393930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:52:12.591 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:12.552+0000 7fd3b2ffd640 1 --2- 172.21.15.39:0/2975215630 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560ef5fb81b0 0x560ef6393930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:12.591 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:12.552+0000 7fd3b2ffd640 1 -- 172.21.15.39:0/2975215630 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560ef5fe1070 msgr2=0x560ef639fdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:52:12.591 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:12.552+0000 7fd3b2ffd640 1 --2- 172.21.15.39:0/2975215630 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560ef5fe1070 0x560ef639fdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:12.591 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:12.552+0000 7fd3b2ffd640 1 -- 172.21.15.39:0/2975215630 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560ef62b6c90 con 0x560ef5fc0390 2024-09-21T03:52:12.592 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:12.553+0000 7fd3b2ffd640 1 --2- 172.21.15.39:0/2975215630 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x560ef5fc0390 0x560ef6393e70 secure :-1 s=READY pgs=930 cs=0 l=1 rev1=1 crypto rx=0x7fd3a4005890 tx=0x7fd3a40045b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a78352e69f6ea37 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:52:12.592 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:12.553+0000 7fd396ff5640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-09-21T03:52:12.592 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:12.553+0000 7fd396ff5640 1 -- 172.21.15.39:0/2975215630 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3a402a060 con 0x560ef5fc0390 2024-09-21T03:52:12.592 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:12.553+0000 7fd396ff5640 1 -- 172.21.15.39:0/2975215630 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8484+0+0 (secure 0 0 0) 0x7fd3a4003df0 con 0x560ef5fc0390 2024-09-21T03:52:12.592 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:12.553+0000 7fd3bf926740 1 -- 172.21.15.39:0/2975215630 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd3a40041d0 con 0x560ef5fc0390 2024-09-21T03:52:12.592 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:12.553+0000 7fd396ff5640 1 -- 172.21.15.39:0/2975215630 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fd3a4003ff0 con 0x560ef5fc0390 2024-09-21T03:52:12.592 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:12.553+0000 7fd3bf926740 10 client.17067 fetch_fsmap learned FSMap version 29 2024-09-21T03:52:12.592 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:12.553+0000 7fd3bf926740 1 -- 172.21.15.39:0/2975215630 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x560ef63a2780 con 0x560ef5fc0390 2024-09-21T03:52:12.592 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:12.553+0000 7fd396ff5640 1 -- 172.21.15.39:0/2975215630 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd3a40041d0 con 0x560ef5fc0390 2024-09-21T03:52:12.592 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:12.553+0000 7fd396ff5640 1 -- 172.21.15.39:0/2975215630 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fd3a40046c0 con 0x560ef5fc0390 2024-09-21T03:52:12.592 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:12.553+0000 7fd3bf926740 10 client.17067 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T03:52:12.592 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:12.553+0000 7fd3bf926740 10 client.17067 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-21T03:52:12.592 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:12.553+0000 7fd3bf926740 20 client.17067 populate_metadata read hostname 'smithi039' 2024-09-21T03:52:12.592 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:12.554+0000 7fd3bf926740 1 --2- 172.21.15.39:0/2975215630 >> [v2:172.21.15.150:6836/4017043144,v1:172.21.15.150:6837/4017043144] conn(0x560ef5f9f9f0 0x560ef63a5b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:52:12.593 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:12.554+0000 7fd3bf926740 4 client.17067 mds_command: new command op to 24659 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:52:12.593 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:12.554+0000 7fd3bf926740 1 -- 172.21.15.39:0/2975215630 --> [v2:172.21.15.150:6836/4017043144,v1:172.21.15.150:6837/4017043144] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560ef6300df0 con 0x560ef5f9f9f0 2024-09-21T03:52:12.593 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:12.554+0000 7fd3b27fc640 1 --2- 172.21.15.39:0/2975215630 >> [v2:172.21.15.150:6836/4017043144,v1:172.21.15.150:6837/4017043144] conn(0x560ef5f9f9f0 0x560ef63a5b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:52:12.593 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:12.554+0000 7fd3b27fc640 1 --2- 172.21.15.39:0/2975215630 >> [v2:172.21.15.150:6836/4017043144,v1:172.21.15.150:6837/4017043144] conn(0x560ef5f9f9f0 0x560ef63a5b90 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=cbbe7344da0d026d server_cookie=eedef0402165f9a8 in_seq=0 out_seq=0 2024-09-21T03:52:12.593 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:12.554+0000 7fd396ff5640 10 client.17067 ms_handle_connect on v2:172.21.15.150:6836/4017043144 2024-09-21T03:52:12.593 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:12.555+0000 7fd396ff5640 1 -- 172.21.15.39:0/2975215630 <== mds.2 v2:172.21.15.150:6836/4017043144 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x560ef6300df0 con 0x560ef5f9f9f0 2024-09-21T03:52:12.593 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:12.555+0000 7fd396ff5640 10 client.17067 handle_command_reply: tid=0 2024-09-21T03:52:12.593 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:12.555+0000 7fd3bf926740 1 client.17067 shutdown 2024-09-21T03:52:12.593 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:12.555+0000 7fd3bf926740 1 -- 172.21.15.39:0/2975215630 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x560ef5fc0390 msgr2=0x560ef6393e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:52:12.593 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:12.555+0000 7fd3bf926740 1 --2- 172.21.15.39:0/2975215630 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x560ef5fc0390 0x560ef6393e70 secure :-1 s=READY pgs=930 cs=0 l=1 rev1=1 crypto rx=0x7fd3a4005890 tx=0x7fd3a40045b0 comp rx=0 tx=0).stop 2024-09-21T03:52:12.593 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:12.555+0000 7fd3bf926740 1 -- 172.21.15.39:0/2975215630 shutdown_connections 2024-09-21T03:52:12.593 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:12.555+0000 7fd3bf926740 1 --2- 172.21.15.39:0/2975215630 >> [v2:172.21.15.150:6836/4017043144,v1:172.21.15.150:6837/4017043144] conn(0x560ef5f9f9f0 0x560ef63a5b90 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:12.593 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:12.555+0000 7fd3bf926740 1 --2- 172.21.15.39:0/2975215630 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560ef5fe1070 0x560ef639fdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:12.593 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:12.555+0000 7fd3bf926740 1 --2- 172.21.15.39:0/2975215630 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x560ef5fc0390 0x560ef6393e70 unknown :-1 s=CLOSED pgs=930 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:12.594 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:12.555+0000 7fd396ff5640 0 client.17067 ms_handle_reset on v2:172.21.15.150:6836/4017043144 2024-09-21T03:52:12.594 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:12.555+0000 7fd3bf926740 1 --2- 172.21.15.39:0/2975215630 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560ef5fb81b0 0x560ef6393930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:12.594 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:12.555+0000 7fd3bf926740 1 -- 172.21.15.39:0/2975215630 >> 172.21.15.39:0/2975215630 conn(0x560ef5fd5460 msgr2=0x560ef638b450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:52:12.594 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:12.555+0000 7fd3bf926740 1 -- 172.21.15.39:0/2975215630 shutdown_connections 2024-09-21T03:52:12.594 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:12.555+0000 7fd3bf926740 1 -- 172.21.15.39:0/2975215630 wait complete. 2024-09-21T03:52:12.594 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:12.555+0000 7fd3bf926740 20 client.17067 trim_cache size 0 max 16384 2024-09-21T03:52:12.594 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:12.568+0000 7fd3bf926740 1 Processor -- start 2024-09-21T03:52:12.594 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:12.568+0000 7fd3bf926740 1 -- start start 2024-09-21T03:52:12.594 INFO:teuthology.orchestra.run.smithi039.stdout:[] 2024-09-21T03:52:12.595 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:12.569+0000 7fd3bf926740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560ef5fb81b0 0x560ef6395560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:52:12.595 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:12.569+0000 7fd3bf926740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x560ef5fc0390 0x560ef639fd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:52:12.595 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:12.569+0000 7fd3bf926740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560ef5fe1070 0x560ef6390b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:52:12.595 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:12.569+0000 7fd3bf926740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fd3b41b1a10 con 0x560ef5fc0390 2024-09-21T03:52:12.595 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:12.569+0000 7fd3bf926740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fd3b41b3550 con 0x560ef5fb81b0 2024-09-21T03:52:12.595 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:12.569+0000 7fd3bf926740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fd3b41b20e0 con 0x560ef5fe1070 2024-09-21T03:52:12.595 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:12.569+0000 7fd3b2ffd640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x560ef5fc0390 0x560ef639fd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:52:12.595 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:12.569+0000 7fd3b2ffd640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x560ef5fc0390 0x560ef639fd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:59928/0 (socket says 172.21.15.39:59928) 2024-09-21T03:52:12.595 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:12.569+0000 7fd3b2ffd640 1 -- 172.21.15.39:0/942815770 learned_addr learned my addr 172.21.15.39:0/942815770 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:52:12.596 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:12.569+0000 7fd3b27fc640 1 --2- 172.21.15.39:0/942815770 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560ef5fe1070 0x560ef6390b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:52:12.596 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:12.569+0000 7fd3b27fc640 1 --2- 172.21.15.39:0/942815770 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560ef5fb81b0 0x560ef6395560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:52:12.596 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:12.569+0000 7fd3b2ffd640 1 -- 172.21.15.39:0/942815770 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560ef5fe1070 msgr2=0x560ef6390b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:52:12.596 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:12.569+0000 7fd3b2ffd640 1 --2- 172.21.15.39:0/942815770 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560ef5fe1070 0x560ef6390b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:12.596 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:12.569+0000 7fd3b2ffd640 1 -- 172.21.15.39:0/942815770 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560ef5fb81b0 msgr2=0x560ef6395560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:52:12.596 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:12.569+0000 7fd3b2ffd640 1 --2- 172.21.15.39:0/942815770 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560ef5fb81b0 0x560ef6395560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:12.596 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:12.569+0000 7fd3b2ffd640 1 -- 172.21.15.39:0/942815770 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560ef6396610 con 0x560ef5fc0390 2024-09-21T03:52:12.596 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:12.569+0000 7fd3b27fc640 1 --2- 172.21.15.39:0/942815770 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560ef5fb81b0 0x560ef6395560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:52:12.596 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:12.569+0000 7fd3b2ffd640 1 --2- 172.21.15.39:0/942815770 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x560ef5fc0390 0x560ef639fd30 secure :-1 s=READY pgs=931 cs=0 l=1 rev1=1 crypto rx=0x7fd3a40107e0 tx=0x7fd3a4005900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed5253f5bac6ab08 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:52:12.596 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:12.570+0000 7fd398ff9640 1 -- 172.21.15.39:0/942815770 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3a401f070 con 0x560ef5fc0390 2024-09-21T03:52:12.596 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:12.570+0000 7fd398ff9640 1 -- 172.21.15.39:0/942815770 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd3a4004190 con 0x560ef5fc0390 2024-09-21T03:52:12.596 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:12.570+0000 7fd398ff9640 1 -- 172.21.15.39:0/942815770 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3a402a070 con 0x560ef5fc0390 2024-09-21T03:52:12.596 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:12.570+0000 7fd3bf926740 1 -- 172.21.15.39:0/942815770 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x560ef5fc0390 msgr2=0x560ef639fd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:52:12.596 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:12.570+0000 7fd3bf926740 1 --2- 172.21.15.39:0/942815770 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x560ef5fc0390 0x560ef639fd30 secure :-1 s=READY pgs=931 cs=0 l=1 rev1=1 crypto rx=0x7fd3a40107e0 tx=0x7fd3a4005900 comp rx=0 tx=0).stop 2024-09-21T03:52:12.596 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:12.570+0000 7fd3bf926740 1 -- 172.21.15.39:0/942815770 shutdown_connections 2024-09-21T03:52:12.597 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:12.570+0000 7fd3bf926740 1 --2- 172.21.15.39:0/942815770 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560ef5fe1070 0x560ef6390b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:12.597 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:12.570+0000 7fd3bf926740 1 --2- 172.21.15.39:0/942815770 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x560ef5fc0390 0x560ef639fd30 unknown :-1 s=CLOSED pgs=931 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:12.597 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:12.570+0000 7fd3bf926740 1 --2- 172.21.15.39:0/942815770 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560ef5fb81b0 0x560ef6395560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:12.597 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:12.570+0000 7fd3bf926740 1 -- 172.21.15.39:0/942815770 >> 172.21.15.39:0/942815770 conn(0x560ef5fd5460 msgr2=0x560ef6394e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:52:12.597 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:12.570+0000 7fd3bf926740 1 -- 172.21.15.39:0/942815770 shutdown_connections 2024-09-21T03:52:12.597 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:12.570+0000 7fd3bf926740 1 -- 172.21.15.39:0/942815770 wait complete. 2024-09-21T03:52:12.597 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:12.571+0000 7fd3bf926740 1 -- start start 2024-09-21T03:52:12.597 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:12.571+0000 7fd3bf926740 1 Processor -- start 2024-09-21T03:52:12.597 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:12.571+0000 7fd3bf926740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560ef5fb81b0 0x560ef63a7270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:52:12.597 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:12.571+0000 7fd3bf926740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560ef5fc0390 0x560ef63a77b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:52:12.597 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:12.571+0000 7fd3bf926740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x560ef5fe1070 0x560ef640f310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:52:12.597 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:12.571+0000 7fd3bf926740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fd3b41b2e80 con 0x560ef5fe1070 2024-09-21T03:52:12.597 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:12.571+0000 7fd3bf926740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fd3b41b27b0 con 0x560ef5fb81b0 2024-09-21T03:52:12.597 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:12.571+0000 7fd3bf926740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fd3b41af9f0 con 0x560ef5fc0390 2024-09-21T03:52:12.597 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:12.571+0000 7fd3bf926740 10 client.0 _ll_register_callbacks cb 0x7fd3a8009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:52:12.598 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:12.571+0000 7fd3b27fc640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x560ef5fe1070 0x560ef640f310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:52:12.598 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:12.571+0000 7fd3b27fc640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x560ef5fe1070 0x560ef640f310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:59936/0 (socket says 172.21.15.39:59936) 2024-09-21T03:52:12.598 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:12.571+0000 7fd3b27fc640 1 -- 172.21.15.39:0/3948232778 learned_addr learned my addr 172.21.15.39:0/3948232778 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:52:12.598 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:12.571+0000 7fd3b27fc640 1 --2- 172.21.15.39:0/3948232778 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560ef5fb81b0 0x560ef63a7270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:52:12.598 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:12.572+0000 7fd3b2ffd640 1 --2- 172.21.15.39:0/3948232778 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560ef5fc0390 0x560ef63a77b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:52:12.598 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:12.572+0000 7fd3b27fc640 1 -- 172.21.15.39:0/3948232778 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560ef5fc0390 msgr2=0x560ef63a77b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:52:12.598 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:12.572+0000 7fd3b27fc640 1 --2- 172.21.15.39:0/3948232778 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560ef5fc0390 0x560ef63a77b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:12.598 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:12.572+0000 7fd3b27fc640 1 -- 172.21.15.39:0/3948232778 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560ef5fb81b0 msgr2=0x560ef63a7270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:52:12.598 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:12.572+0000 7fd3b27fc640 1 --2- 172.21.15.39:0/3948232778 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560ef5fb81b0 0x560ef63a7270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:12.598 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:12.572+0000 7fd3b27fc640 1 -- 172.21.15.39:0/3948232778 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560ef640f850 con 0x560ef5fe1070 2024-09-21T03:52:12.598 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:12.572+0000 7fd3b27fc640 1 --2- 172.21.15.39:0/3948232778 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x560ef5fe1070 0x560ef640f310 secure :-1 s=READY pgs=932 cs=0 l=1 rev1=1 crypto rx=0x560ef63a2990 tx=0x7fd3a400b890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=deed9386832ded89 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:52:12.598 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:12.572+0000 7fd396ff5640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-09-21T03:52:12.598 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:12.572+0000 7fd396ff5640 1 -- 172.21.15.39:0/3948232778 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3a8035040 con 0x560ef5fe1070 2024-09-21T03:52:12.599 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:12.572+0000 7fd396ff5640 1 -- 172.21.15.39:0/3948232778 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8484+0+0 (secure 0 0 0) 0x7fd3a8005050 con 0x560ef5fe1070 2024-09-21T03:52:12.599 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:12.572+0000 7fd3bf926740 1 -- 172.21.15.39:0/3948232778 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd3a4004190 con 0x560ef5fe1070 2024-09-21T03:52:12.599 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:12.572+0000 7fd396ff5640 1 -- 172.21.15.39:0/3948232778 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fd3a8005250 con 0x560ef5fe1070 2024-09-21T03:52:12.599 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:12.572+0000 7fd3bf926740 10 client.17073 fetch_fsmap learned FSMap version 29 2024-09-21T03:52:12.599 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:12.572+0000 7fd3bf926740 1 -- 172.21.15.39:0/3948232778 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x560ef6410a40 con 0x560ef5fe1070 2024-09-21T03:52:12.599 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:12.572+0000 7fd396ff5640 1 -- 172.21.15.39:0/3948232778 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd3a4004190 con 0x560ef5fe1070 2024-09-21T03:52:12.599 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:12.573+0000 7fd396ff5640 1 -- 172.21.15.39:0/3948232778 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fd3a80035f0 con 0x560ef5fe1070 2024-09-21T03:52:12.599 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:12.573+0000 7fd3bf926740 10 client.17073 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T03:52:12.599 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:12.573+0000 7fd3bf926740 10 client.17073 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-21T03:52:12.599 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:12.573+0000 7fd3bf926740 20 client.17073 populate_metadata read hostname 'smithi039' 2024-09-21T03:52:12.599 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:12.573+0000 7fd3bf926740 1 --2- 172.21.15.39:0/3948232778 >> [v2:172.21.15.150:6836/4017043144,v1:172.21.15.150:6837/4017043144] conn(0x560ef5f74da0 0x560ef64131e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:52:12.599 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:12.573+0000 7fd3bf926740 4 client.17073 mds_command: new command op to 24659 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T03:52:12.599 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:12.573+0000 7fd3bf926740 1 -- 172.21.15.39:0/3948232778 --> [v2:172.21.15.150:6836/4017043144,v1:172.21.15.150:6837/4017043144] -- command(tid 0: {"prefix": "damage ls"}) -- 0x560ef62fc8a0 con 0x560ef5f74da0 2024-09-21T03:52:12.599 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:12.573+0000 7fd3b2ffd640 1 --2- 172.21.15.39:0/3948232778 >> [v2:172.21.15.150:6836/4017043144,v1:172.21.15.150:6837/4017043144] conn(0x560ef5f74da0 0x560ef64131e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:52:12.599 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:12.573+0000 7fd396ff5640 10 client.17073 ms_handle_connect on v2:172.21.15.150:6836/4017043144 2024-09-21T03:52:12.600 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:12.573+0000 7fd3b2ffd640 1 --2- 172.21.15.39:0/3948232778 >> [v2:172.21.15.150:6836/4017043144,v1:172.21.15.150:6837/4017043144] conn(0x560ef5f74da0 0x560ef64131e0 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=56f51eeb11ae90b9 server_cookie=dd88693757494445 in_seq=0 out_seq=0 2024-09-21T03:52:12.600 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:12.574+0000 7fd396ff5640 1 -- 172.21.15.39:0/3948232778 <== mds.2 v2:172.21.15.150:6836/4017043144 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x560ef62fc8a0 con 0x560ef5f74da0 2024-09-21T03:52:12.600 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:12.574+0000 7fd396ff5640 10 client.17073 handle_command_reply: tid=0 2024-09-21T03:52:12.600 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:12.574+0000 7fd3bf926740 1 client.17073 shutdown 2024-09-21T03:52:12.600 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:12.574+0000 7fd3bf926740 1 -- 172.21.15.39:0/3948232778 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x560ef5fe1070 msgr2=0x560ef640f310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:52:12.600 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:12.574+0000 7fd3bf926740 1 --2- 172.21.15.39:0/3948232778 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x560ef5fe1070 0x560ef640f310 secure :-1 s=READY pgs=932 cs=0 l=1 rev1=1 crypto rx=0x560ef63a2990 tx=0x7fd3a400b890 comp rx=0 tx=0).stop 2024-09-21T03:52:12.600 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:12.574+0000 7fd3bf926740 1 -- 172.21.15.39:0/3948232778 shutdown_connections 2024-09-21T03:52:12.600 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:12.574+0000 7fd3bf926740 1 --2- 172.21.15.39:0/3948232778 >> [v2:172.21.15.150:6836/4017043144,v1:172.21.15.150:6837/4017043144] conn(0x560ef5f74da0 0x560ef64131e0 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:12.600 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:12.574+0000 7fd3bf926740 1 --2- 172.21.15.39:0/3948232778 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x560ef5fe1070 0x560ef640f310 unknown :-1 s=CLOSED pgs=932 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:12.600 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:12.574+0000 7fd3bf926740 1 --2- 172.21.15.39:0/3948232778 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560ef5fc0390 0x560ef63a77b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:12.600 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:12.574+0000 7fd3bf926740 1 --2- 172.21.15.39:0/3948232778 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560ef5fb81b0 0x560ef63a7270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:12.600 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:12.574+0000 7fd3bf926740 1 -- 172.21.15.39:0/3948232778 >> 172.21.15.39:0/3948232778 conn(0x560ef5fd5460 msgr2=0x560ef6394c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:52:12.600 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:12.574+0000 7fd396ff5640 0 client.17073 ms_handle_reset on v2:172.21.15.150:6836/4017043144 2024-09-21T03:52:12.600 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:12.574+0000 7fd3bf926740 1 -- 172.21.15.39:0/3948232778 shutdown_connections 2024-09-21T03:52:12.600 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:12.574+0000 7fd3bf926740 1 -- 172.21.15.39:0/3948232778 wait complete. 2024-09-21T03:52:12.601 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:12.574+0000 7fd3bf926740 20 client.17073 trim_cache size 0 max 16384 2024-09-21T03:52:12.601 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:12.574+0000 7fd3b93b2640 1 -- 172.21.15.39:0/2273098140 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7fd39c0b6680 msgr2=0x7fd39c0b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:52:12.601 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:12.574+0000 7fd3b93b2640 1 --2- 172.21.15.39:0/2273098140 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7fd39c0b6680 0x7fd39c0b8b90 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fd3ac0aa060 tx=0x7fd3ac095040 comp rx=0 tx=0).stop 2024-09-21T03:52:12.601 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:12.574+0000 7fd3b93b2640 1 -- 172.21.15.39:0/2273098140 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd3b4109e40 msgr2=0x7fd3b4076190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:52:12.601 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:12.574+0000 7fd3b93b2640 1 --2- 172.21.15.39:0/2273098140 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd3b4109e40 0x7fd3b4076190 secure :-1 s=READY pgs=929 cs=0 l=1 rev1=1 crypto rx=0x7fd3ac00eb10 tx=0x7fd3ac00c2d0 comp rx=0 tx=0).stop 2024-09-21T03:52:12.601 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:12.574+0000 7fd3b93b2640 1 -- 172.21.15.39:0/2273098140 shutdown_connections 2024-09-21T03:52:12.601 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:12.574+0000 7fd3b93b2640 1 --2- 172.21.15.39:0/2273098140 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7fd39c0b6680 0x7fd39c0b8b90 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:12.601 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:12.574+0000 7fd3b93b2640 1 --2- 172.21.15.39:0/2273098140 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd3b4109e40 0x7fd3b4076190 unknown :-1 s=CLOSED pgs=929 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:12.601 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:12.574+0000 7fd3b93b2640 1 --2- 172.21.15.39:0/2273098140 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd3b41094f0 0x7fd3b4075c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:12.601 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:12.574+0000 7fd3b93b2640 1 --2- 172.21.15.39:0/2273098140 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd3b4103bb0 0x7fd3b4075710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:12.601 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:12.575+0000 7fd3b93b2640 1 -- 172.21.15.39:0/2273098140 >> 172.21.15.39:0/2273098140 conn(0x7fd3b40ff600 msgr2=0x7fd3b4101a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:52:12.601 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:12.575+0000 7fd3b93b2640 1 -- 172.21.15.39:0/2273098140 shutdown_connections 2024-09-21T03:52:12.601 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:12.575+0000 7fd3b93b2640 1 -- 172.21.15.39:0/2273098140 wait complete. 2024-09-21T03:52:13.404 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 51116d5c-77ca-11ef-baf5-efdc52797490 -- ceph tell mds.1:1 damage ls 2024-09-21T03:52:13.468 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:52:13 smithi161 ceph-mon[34533]: pgmap v510: 97 pgs: 97 active+clean; 10 MiB data, 520 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 24 op/s 2024-09-21T03:52:13.475 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:52:13 smithi150 ceph-mon[34289]: pgmap v510: 97 pgs: 97 active+clean; 10 MiB data, 520 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 24 op/s 2024-09-21T03:52:13.583 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:52:13 smithi039 ceph-mon[31211]: pgmap v510: 97 pgs: 97 active+clean; 10 MiB data, 520 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 24 op/s 2024-09-21T03:52:13.664 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/51116d5c-77ca-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:52:14.240 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:14.237+0000 7fe132bda640 1 -- 172.21.15.39:0/1929529107 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe12c108cf0 msgr2=0x7fe12c10f520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:52:14.240 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:14.237+0000 7fe132bda640 1 --2- 172.21.15.39:0/1929529107 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe12c108cf0 0x7fe12c10f520 secure :-1 s=READY pgs=933 cs=0 l=1 rev1=1 crypto rx=0x7fe124009ca0 tx=0x7fe12402a8c0 comp rx=0 tx=0).stop 2024-09-21T03:52:14.240 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:14.237+0000 7fe132bda640 1 -- 172.21.15.39:0/1929529107 shutdown_connections 2024-09-21T03:52:14.240 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:14.237+0000 7fe132bda640 1 --2- 172.21.15.39:0/1929529107 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe12c108cf0 0x7fe12c10f520 unknown :-1 s=CLOSED pgs=933 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:14.241 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:14.237+0000 7fe132bda640 1 --2- 172.21.15.39:0/1929529107 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe12c1083a0 0x7fe12c1087b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:14.241 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:14.237+0000 7fe132bda640 1 --2- 172.21.15.39:0/1929529107 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe12c102a60 0x7fe12c102e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:14.241 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:14.237+0000 7fe132bda640 1 -- 172.21.15.39:0/1929529107 >> 172.21.15.39:0/1929529107 conn(0x7fe12c079730 msgr2=0x7fe12c079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:52:14.241 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:14.237+0000 7fe132bda640 1 -- 172.21.15.39:0/1929529107 shutdown_connections 2024-09-21T03:52:14.241 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:14.237+0000 7fe132bda640 1 -- 172.21.15.39:0/1929529107 wait complete. 2024-09-21T03:52:14.241 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:14.238+0000 7fe132bda640 1 Processor -- start 2024-09-21T03:52:14.241 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:14.238+0000 7fe132bda640 1 -- start start 2024-09-21T03:52:14.241 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:14.238+0000 7fe132bda640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe12c102a60 0x7fe12c19b270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:52:14.241 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:14.238+0000 7fe132bda640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe12c1083a0 0x7fe12c19b7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:52:14.241 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:14.238+0000 7fe132bda640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe12c108cf0 0x7fe12c19bcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:52:14.241 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:14.238+0000 7fe132bda640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fe12c112960 con 0x7fe12c1083a0 2024-09-21T03:52:14.241 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:14.238+0000 7fe132bda640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fe12c1127e0 con 0x7fe12c102a60 2024-09-21T03:52:14.241 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:14.238+0000 7fe132bda640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fe12c112ae0 con 0x7fe12c108cf0 2024-09-21T03:52:14.241 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:14.238+0000 7fe12bfff640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe12c1083a0 0x7fe12c19b7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:52:14.241 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:14.238+0000 7fe12bfff640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe12c1083a0 0x7fe12c19b7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:59944/0 (socket says 172.21.15.39:59944) 2024-09-21T03:52:14.242 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:14.238+0000 7fe12bfff640 1 -- 172.21.15.39:0/2770667270 learned_addr learned my addr 172.21.15.39:0/2770667270 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:52:14.242 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:14.238+0000 7fe130976640 1 --2- 172.21.15.39:0/2770667270 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe12c102a60 0x7fe12c19b270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:52:14.242 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:14.238+0000 7fe131177640 1 --2- 172.21.15.39:0/2770667270 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe12c108cf0 0x7fe12c19bcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:52:14.242 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:14.238+0000 7fe12bfff640 1 -- 172.21.15.39:0/2770667270 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe12c108cf0 msgr2=0x7fe12c19bcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:52:14.242 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:14.238+0000 7fe12bfff640 1 --2- 172.21.15.39:0/2770667270 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe12c108cf0 0x7fe12c19bcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:14.242 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:14.238+0000 7fe12bfff640 1 -- 172.21.15.39:0/2770667270 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe12c102a60 msgr2=0x7fe12c19b270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:52:14.242 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:14.239+0000 7fe12bfff640 1 --2- 172.21.15.39:0/2770667270 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe12c102a60 0x7fe12c19b270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:14.242 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:14.239+0000 7fe12bfff640 1 -- 172.21.15.39:0/2770667270 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe12c1a6530 con 0x7fe12c1083a0 2024-09-21T03:52:14.243 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:14.239+0000 7fe12bfff640 1 --2- 172.21.15.39:0/2770667270 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe12c1083a0 0x7fe12c19b7b0 secure :-1 s=READY pgs=934 cs=0 l=1 rev1=1 crypto rx=0x7fe11800d770 tx=0x7fe11800dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:52:14.243 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:14.239+0000 7fe129ffb640 1 -- 172.21.15.39:0/2770667270 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe118017070 con 0x7fe12c1083a0 2024-09-21T03:52:14.243 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:14.239+0000 7fe129ffb640 1 -- 172.21.15.39:0/2770667270 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe118009c30 con 0x7fe12c1083a0 2024-09-21T03:52:14.243 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:14.239+0000 7fe132bda640 1 -- 172.21.15.39:0/2770667270 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe12c1a6820 con 0x7fe12c1083a0 2024-09-21T03:52:14.243 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:14.239+0000 7fe129ffb640 1 -- 172.21.15.39:0/2770667270 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe11800f620 con 0x7fe12c1083a0 2024-09-21T03:52:14.243 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:14.239+0000 7fe132bda640 1 -- 172.21.15.39:0/2770667270 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe12c1a6d60 con 0x7fe12c1083a0 2024-09-21T03:52:14.244 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:14.242+0000 7fe13914e740 1 Processor -- start 2024-09-21T03:52:14.244 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:14.242+0000 7fe13914e740 1 -- start start 2024-09-21T03:52:14.244 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:14.242+0000 7fe13914e740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55eda1370070 0x55eda17176a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:52:14.245 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:14.242+0000 7fe13914e740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55eda13471b0 0x55eda1717be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:52:14.245 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:14.242+0000 7fe13914e740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55eda134f390 0x55eda1718120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:52:14.245 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:14.242+0000 7fe13914e740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x55eda168dea0 con 0x55eda1370070 2024-09-21T03:52:14.245 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:14.242+0000 7fe13914e740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x55eda167d2d0 con 0x55eda13471b0 2024-09-21T03:52:14.245 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:14.242+0000 7fe13914e740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55eda16810e0 con 0x55eda134f390 2024-09-21T03:52:14.245 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:14.242+0000 7fe131177640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55eda1370070 0x55eda17176a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:52:14.245 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:14.242+0000 7fe131177640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55eda1370070 0x55eda17176a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:59948/0 (socket says 172.21.15.39:59948) 2024-09-21T03:52:14.245 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:14.242+0000 7fe131177640 1 -- 172.21.15.39:0/3173815018 learned_addr learned my addr 172.21.15.39:0/3173815018 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:52:14.245 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:14.242+0000 7fe129ffb640 1 -- 172.21.15.39:0/2770667270 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 27) ==== 154643+0+0 (secure 0 0 0) 0x7fe11801a070 con 0x7fe12c1083a0 2024-09-21T03:52:14.245 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:14.243+0000 7fe130976640 1 --2- 172.21.15.39:0/3173815018 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55eda13471b0 0x55eda1717be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:52:14.245 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:14.243+0000 7fe131177640 1 -- 172.21.15.39:0/3173815018 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55eda134f390 msgr2=0x55eda1718120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:52:14.245 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:14.243+0000 7fe131177640 1 --2- 172.21.15.39:0/3173815018 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55eda134f390 0x55eda1718120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:14.245 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:14.243+0000 7fe131177640 1 -- 172.21.15.39:0/3173815018 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55eda13471b0 msgr2=0x55eda1717be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:52:14.245 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:14.243+0000 7fe131177640 1 --2- 172.21.15.39:0/3173815018 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55eda13471b0 0x55eda1717be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:14.246 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:14.243+0000 7fe131177640 1 -- 172.21.15.39:0/3173815018 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55eda13db650 con 0x55eda1370070 2024-09-21T03:52:14.246 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:14.243+0000 7fe131177640 1 --2- 172.21.15.39:0/3173815018 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55eda1370070 0x55eda17176a0 secure :-1 s=READY pgs=935 cs=0 l=1 rev1=1 crypto rx=0x7fe1240098b0 tx=0x7fe1240098e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=91d980f6f5e00d8c server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:52:14.249 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:14.243+0000 7fe116ffd640 1 -- 172.21.15.39:0/3173815018 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe124031cf0 con 0x55eda1370070 2024-09-21T03:52:14.249 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:14.243+0000 7fe116ffd640 1 -- 172.21.15.39:0/3173815018 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe124036030 con 0x55eda1370070 2024-09-21T03:52:14.249 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:14.243+0000 7fe116ffd640 1 -- 172.21.15.39:0/3173815018 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1240042a0 con 0x55eda1370070 2024-09-21T03:52:14.249 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:14.243+0000 7fe129ffb640 1 --2- 172.21.15.39:0/2770667270 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7fe11c0b6680 0x7fe11c0b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:52:14.249 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:14.243+0000 7fe130976640 1 --2- 172.21.15.39:0/2770667270 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7fe11c0b6680 0x7fe11c0b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:52:14.250 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:14.244+0000 7fe129ffb640 1 -- 172.21.15.39:0/2770667270 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8484+0+0 (secure 0 0 0) 0x7fe1180e7120 con 0x7fe12c1083a0 2024-09-21T03:52:14.250 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:14.244+0000 7fe13914e740 1 -- 172.21.15.39:0/3173815018 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55eda1370070 msgr2=0x55eda17176a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:52:14.250 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:14.244+0000 7fe130976640 1 --2- 172.21.15.39:0/2770667270 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7fe11c0b6680 0x7fe11c0b8b90 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x55eda1716320 tx=0x7fe12000b390 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:52:14.250 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:14.244+0000 7fe13914e740 1 --2- 172.21.15.39:0/3173815018 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55eda1370070 0x55eda17176a0 secure :-1 s=READY pgs=935 cs=0 l=1 rev1=1 crypto rx=0x7fe1240098b0 tx=0x7fe1240098e0 comp rx=0 tx=0).stop 2024-09-21T03:52:14.250 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:14.244+0000 7fe13914e740 1 -- 172.21.15.39:0/3173815018 shutdown_connections 2024-09-21T03:52:14.250 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:14.244+0000 7fe13914e740 1 --2- 172.21.15.39:0/3173815018 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55eda134f390 0x55eda1718120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:14.250 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:14.244+0000 7fe13914e740 1 --2- 172.21.15.39:0/3173815018 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55eda13471b0 0x55eda1717be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:14.250 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:14.244+0000 7fe13914e740 1 --2- 172.21.15.39:0/3173815018 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55eda1370070 0x55eda17176a0 unknown :-1 s=CLOSED pgs=935 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:14.250 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:14.244+0000 7fe13914e740 1 -- 172.21.15.39:0/3173815018 >> 172.21.15.39:0/3173815018 conn(0x55eda1364460 msgr2=0x55eda1715590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:52:14.250 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:14.244+0000 7fe13914e740 1 -- 172.21.15.39:0/3173815018 shutdown_connections 2024-09-21T03:52:14.250 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:14.244+0000 7fe13914e740 1 -- 172.21.15.39:0/3173815018 wait complete. 2024-09-21T03:52:14.250 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:14.245+0000 7fe13914e740 1 -- start start 2024-09-21T03:52:14.250 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:14.245+0000 7fe13914e740 1 Processor -- start 2024-09-21T03:52:14.250 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:14.245+0000 7fe13914e740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55eda13471b0 0x55eda1722db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:52:14.251 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:14.245+0000 7fe13914e740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55eda134f390 0x55eda17232f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:52:14.251 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:14.245+0000 7fe13914e740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55eda1370070 0x55eda172efc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:52:14.251 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:14.245+0000 7fe13914e740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x55eda14de700 con 0x55eda134f390 2024-09-21T03:52:14.251 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:14.245+0000 7fe13914e740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x55eda1618160 con 0x55eda1370070 2024-09-21T03:52:14.251 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:14.245+0000 7fe13914e740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55eda1725ca0 con 0x55eda13471b0 2024-09-21T03:52:14.251 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:14.246+0000 7fe13914e740 10 client.0 _ll_register_callbacks cb 0x55eda16319e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:52:14.251 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:14.246+0000 7fe12bfff640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55eda134f390 0x55eda17232f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:52:14.251 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:14.246+0000 7fe12bfff640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55eda134f390 0x55eda17232f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:59964/0 (socket says 172.21.15.39:59964) 2024-09-21T03:52:14.251 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:14.246+0000 7fe12bfff640 1 -- 172.21.15.39:0/1995539205 learned_addr learned my addr 172.21.15.39:0/1995539205 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:52:14.251 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:14.246+0000 7fe130976640 1 --2- 172.21.15.39:0/1995539205 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55eda13471b0 0x55eda1722db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:52:14.251 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:14.246+0000 7fe131177640 1 --2- 172.21.15.39:0/1995539205 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55eda1370070 0x55eda172efc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:52:14.251 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:14.246+0000 7fe12bfff640 1 -- 172.21.15.39:0/1995539205 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55eda13471b0 msgr2=0x55eda1722db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:52:14.251 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:14.246+0000 7fe12bfff640 1 --2- 172.21.15.39:0/1995539205 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55eda13471b0 0x55eda1722db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:14.251 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:14.246+0000 7fe12bfff640 1 -- 172.21.15.39:0/1995539205 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55eda1370070 msgr2=0x55eda172efc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:52:14.252 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:14.246+0000 7fe12bfff640 1 --2- 172.21.15.39:0/1995539205 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55eda1370070 0x55eda172efc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:14.252 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:14.246+0000 7fe12bfff640 1 -- 172.21.15.39:0/1995539205 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55eda1645c90 con 0x55eda134f390 2024-09-21T03:52:14.252 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:14.246+0000 7fe12bfff640 1 --2- 172.21.15.39:0/1995539205 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55eda134f390 0x55eda17232f0 secure :-1 s=READY pgs=936 cs=0 l=1 rev1=1 crypto rx=0x7fe1180acfd0 tx=0x7fe1180acf40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a5096b3d069bed1c server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:52:14.252 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:14.246+0000 7fe10bfff640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-09-21T03:52:14.252 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:14.246+0000 7fe10bfff640 1 -- 172.21.15.39:0/1995539205 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe118021030 con 0x55eda134f390 2024-09-21T03:52:14.252 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:14.246+0000 7fe10bfff640 1 -- 172.21.15.39:0/1995539205 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8484+0+0 (secure 0 0 0) 0x7fe1180e92d0 con 0x55eda134f390 2024-09-21T03:52:14.252 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:14.246+0000 7fe13914e740 1 -- 172.21.15.39:0/1995539205 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe124036030 con 0x55eda134f390 2024-09-21T03:52:14.252 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:14.247+0000 7fe10bfff640 1 -- 172.21.15.39:0/1995539205 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fe1180acb50 con 0x55eda134f390 2024-09-21T03:52:14.252 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:14.247+0000 7fe13914e740 10 client.17085 fetch_fsmap learned FSMap version 29 2024-09-21T03:52:14.252 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:14.247+0000 7fe13914e740 1 -- 172.21.15.39:0/1995539205 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55eda17319e0 con 0x55eda134f390 2024-09-21T03:52:14.252 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:14.247+0000 7fe10bfff640 1 -- 172.21.15.39:0/1995539205 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe124036030 con 0x55eda134f390 2024-09-21T03:52:14.252 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:14.247+0000 7fe10bfff640 1 -- 172.21.15.39:0/1995539205 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fe118098200 con 0x55eda134f390 2024-09-21T03:52:14.252 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:14.247+0000 7fe13914e740 10 client.17085 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T03:52:14.253 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:14.247+0000 7fe13914e740 10 client.17085 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-21T03:52:14.253 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:14.247+0000 7fe13914e740 20 client.17085 populate_metadata read hostname 'smithi039' 2024-09-21T03:52:14.253 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:14.247+0000 7fe13914e740 1 --2- 172.21.15.39:0/1995539205 >> [v2:172.21.15.161:6838/2190107270,v1:172.21.15.161:6839/2190107270] conn(0x55eda132e9f0 0x55eda1734f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:52:14.253 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:14.247+0000 7fe13914e740 4 client.17085 mds_command: new command op to 24667 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:52:14.253 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:14.247+0000 7fe13914e740 1 -- 172.21.15.39:0/1995539205 --> [v2:172.21.15.161:6838/2190107270,v1:172.21.15.161:6839/2190107270] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55eda168fdf0 con 0x55eda132e9f0 2024-09-21T03:52:14.253 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:14.248+0000 7fe130976640 1 --2- 172.21.15.39:0/1995539205 >> [v2:172.21.15.161:6838/2190107270,v1:172.21.15.161:6839/2190107270] conn(0x55eda132e9f0 0x55eda1734f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:52:14.254 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:14.248+0000 7fe10bfff640 10 client.17085 ms_handle_connect on v2:172.21.15.161:6838/2190107270 2024-09-21T03:52:14.254 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:14.248+0000 7fe130976640 1 --2- 172.21.15.39:0/1995539205 >> [v2:172.21.15.161:6838/2190107270,v1:172.21.15.161:6839/2190107270] conn(0x55eda132e9f0 0x55eda1734f00 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=d734d059bbe5b541 server_cookie=a4e581f19bd8daa4 in_seq=0 out_seq=0 2024-09-21T03:52:14.254 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:14.249+0000 7fe10bfff640 1 -- 172.21.15.39:0/1995539205 <== mds.1 v2:172.21.15.161:6838/2190107270 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55eda168fdf0 con 0x55eda132e9f0 2024-09-21T03:52:14.254 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:14.249+0000 7fe10bfff640 10 client.17085 handle_command_reply: tid=0 2024-09-21T03:52:14.254 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:14.249+0000 7fe13914e740 1 client.17085 shutdown 2024-09-21T03:52:14.254 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:14.249+0000 7fe13914e740 1 -- 172.21.15.39:0/1995539205 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55eda134f390 msgr2=0x55eda17232f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:52:14.255 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:14.249+0000 7fe13914e740 1 --2- 172.21.15.39:0/1995539205 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55eda134f390 0x55eda17232f0 secure :-1 s=READY pgs=936 cs=0 l=1 rev1=1 crypto rx=0x7fe1180acfd0 tx=0x7fe1180acf40 comp rx=0 tx=0).stop 2024-09-21T03:52:14.255 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:14.249+0000 7fe13914e740 1 -- 172.21.15.39:0/1995539205 shutdown_connections 2024-09-21T03:52:14.255 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:14.249+0000 7fe13914e740 1 --2- 172.21.15.39:0/1995539205 >> [v2:172.21.15.161:6838/2190107270,v1:172.21.15.161:6839/2190107270] conn(0x55eda132e9f0 0x55eda1734f00 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:14.255 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:14.249+0000 7fe13914e740 1 --2- 172.21.15.39:0/1995539205 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55eda1370070 0x55eda172efc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:14.255 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:14.249+0000 7fe13914e740 1 --2- 172.21.15.39:0/1995539205 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55eda134f390 0x55eda17232f0 unknown :-1 s=CLOSED pgs=936 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:14.255 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:14.249+0000 7fe10bfff640 0 client.17085 ms_handle_reset on v2:172.21.15.161:6838/2190107270 2024-09-21T03:52:14.255 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:14.249+0000 7fe13914e740 1 --2- 172.21.15.39:0/1995539205 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55eda13471b0 0x55eda1722db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:14.255 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:14.249+0000 7fe13914e740 1 -- 172.21.15.39:0/1995539205 >> 172.21.15.39:0/1995539205 conn(0x55eda1364460 msgr2=0x55eda171a880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:52:14.255 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:14.250+0000 7fe13914e740 1 -- 172.21.15.39:0/1995539205 shutdown_connections 2024-09-21T03:52:14.255 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:14.250+0000 7fe13914e740 1 -- 172.21.15.39:0/1995539205 wait complete. 2024-09-21T03:52:14.256 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:14.250+0000 7fe13914e740 20 client.17085 trim_cache size 0 max 16384 2024-09-21T03:52:14.273 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:14.263+0000 7fe13914e740 1 Processor -- start 2024-09-21T03:52:14.273 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:14.263+0000 7fe13914e740 1 -- start start 2024-09-21T03:52:14.273 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:14.263+0000 7fe13914e740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55eda13471b0 0x55eda17242d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:52:14.273 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:14.263+0000 7fe13914e740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55eda134f390 0x55eda1714bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:52:14.273 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:14.263+0000 7fe13914e740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55eda1370070 0x55eda1715100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:52:14.274 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:14.263+0000 7fe13914e740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fe12c1af6a0 con 0x55eda13471b0 2024-09-21T03:52:14.274 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:14.263+0000 7fe13914e740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fe12c1b11e0 con 0x55eda134f390 2024-09-21T03:52:14.274 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:14.263+0000 7fe13914e740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fe12c1afd70 con 0x55eda1370070 2024-09-21T03:52:14.274 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:14.263+0000 7fe130976640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55eda13471b0 0x55eda17242d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:52:14.274 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:14.263+0000 7fe130976640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55eda13471b0 0x55eda17242d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:59974/0 (socket says 172.21.15.39:59974) 2024-09-21T03:52:14.274 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:14.263+0000 7fe130976640 1 -- 172.21.15.39:0/3015105414 learned_addr learned my addr 172.21.15.39:0/3015105414 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:52:14.275 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:14.264+0000 7fe130976640 1 -- 172.21.15.39:0/3015105414 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55eda1370070 msgr2=0x55eda1715100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:52:14.275 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:14.264+0000 7fe130976640 1 --2- 172.21.15.39:0/3015105414 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55eda1370070 0x55eda1715100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:14.275 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:14.264+0000 7fe130976640 1 -- 172.21.15.39:0/3015105414 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55eda134f390 msgr2=0x55eda1714bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:52:14.275 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:14.264+0000 7fe130976640 1 --2- 172.21.15.39:0/3015105414 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55eda134f390 0x55eda1714bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:14.275 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:14.264+0000 7fe130976640 1 -- 172.21.15.39:0/3015105414 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55eda1725180 con 0x55eda13471b0 2024-09-21T03:52:14.275 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:14.264+0000 7fe130976640 1 --2- 172.21.15.39:0/3015105414 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55eda13471b0 0x55eda17242d0 secure :-1 s=READY pgs=937 cs=0 l=1 rev1=1 crypto rx=0x55eda1719fc0 tx=0x7fe120015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=900cc418ac0b97e2 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:52:14.276 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:14.264+0000 7fe1167fc640 1 -- 172.21.15.39:0/3015105414 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe120020070 con 0x55eda13471b0 2024-09-21T03:52:14.276 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:14.264+0000 7fe1167fc640 1 -- 172.21.15.39:0/3015105414 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe120028860 con 0x55eda13471b0 2024-09-21T03:52:14.276 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:14.264+0000 7fe1167fc640 1 -- 172.21.15.39:0/3015105414 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe120027810 con 0x55eda13471b0 2024-09-21T03:52:14.276 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:14.265+0000 7fe13914e740 1 -- 172.21.15.39:0/3015105414 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55eda13471b0 msgr2=0x55eda17242d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:52:14.276 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:14.265+0000 7fe13914e740 1 --2- 172.21.15.39:0/3015105414 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55eda13471b0 0x55eda17242d0 secure :-1 s=READY pgs=937 cs=0 l=1 rev1=1 crypto rx=0x55eda1719fc0 tx=0x7fe120015000 comp rx=0 tx=0).stop 2024-09-21T03:52:14.276 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:14.265+0000 7fe13914e740 1 -- 172.21.15.39:0/3015105414 shutdown_connections 2024-09-21T03:52:14.276 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:14.265+0000 7fe13914e740 1 --2- 172.21.15.39:0/3015105414 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55eda1370070 0x55eda1715100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:14.277 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:14.265+0000 7fe13914e740 1 --2- 172.21.15.39:0/3015105414 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55eda134f390 0x55eda1714bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:14.277 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:14.265+0000 7fe13914e740 1 --2- 172.21.15.39:0/3015105414 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55eda13471b0 0x55eda17242d0 unknown :-1 s=CLOSED pgs=937 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:14.277 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:14.265+0000 7fe13914e740 1 -- 172.21.15.39:0/3015105414 >> 172.21.15.39:0/3015105414 conn(0x55eda1364460 msgr2=0x55eda171b160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:52:14.277 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:14.265+0000 7fe13914e740 1 -- 172.21.15.39:0/3015105414 shutdown_connections 2024-09-21T03:52:14.277 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:14.265+0000 7fe13914e740 1 -- 172.21.15.39:0/3015105414 wait complete. 2024-09-21T03:52:14.277 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:14.265+0000 7fe13914e740 1 -- start start 2024-09-21T03:52:14.278 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:14.265+0000 7fe13914e740 1 Processor -- start 2024-09-21T03:52:14.278 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:14.266+0000 7fe13914e740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55eda13471b0 0x55eda1727370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:52:14.278 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:14.266+0000 7fe13914e740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55eda134f390 0x55eda1727950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:52:14.278 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:14.266+0000 7fe13914e740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55eda1370070 0x55eda179e830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:52:14.278 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:14.266+0000 7fe13914e740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fe12c1b0b10 con 0x55eda1370070 2024-09-21T03:52:14.278 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:14.266+0000 7fe13914e740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fe12c1b0440 con 0x55eda13471b0 2024-09-21T03:52:14.278 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:14.266+0000 7fe13914e740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fe12c1ad680 con 0x55eda134f390 2024-09-21T03:52:14.279 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:14.266+0000 7fe13914e740 10 client.0 _ll_register_callbacks cb 0x7fe1180a8f70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:52:14.279 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:14.266+0000 7fe131177640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55eda1370070 0x55eda179e830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:52:14.279 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:14.266+0000 7fe130976640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55eda13471b0 0x55eda1727370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:52:14.279 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:14.266+0000 7fe131177640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55eda1370070 0x55eda179e830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:59990/0 (socket says 172.21.15.39:59990) 2024-09-21T03:52:14.279 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:14.266+0000 7fe131177640 1 -- 172.21.15.39:0/2617749365 learned_addr learned my addr 172.21.15.39:0/2617749365 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:52:14.279 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:14.266+0000 7fe12bfff640 1 --2- 172.21.15.39:0/2617749365 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55eda134f390 0x55eda1727950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:52:14.280 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:14.266+0000 7fe131177640 1 -- 172.21.15.39:0/2617749365 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55eda134f390 msgr2=0x55eda1727950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:52:14.280 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:14.266+0000 7fe131177640 1 --2- 172.21.15.39:0/2617749365 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55eda134f390 0x55eda1727950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:14.280 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:14.266+0000 7fe131177640 1 -- 172.21.15.39:0/2617749365 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55eda13471b0 msgr2=0x55eda1727370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:52:14.280 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:14.266+0000 7fe131177640 1 --2- 172.21.15.39:0/2617749365 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55eda13471b0 0x55eda1727370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:14.280 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:14.266+0000 7fe131177640 1 -- 172.21.15.39:0/2617749365 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55eda17236a0 con 0x55eda1370070 2024-09-21T03:52:14.280 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:14.267+0000 7fe131177640 1 --2- 172.21.15.39:0/2617749365 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55eda1370070 0x55eda179e830 secure :-1 s=READY pgs=938 cs=0 l=1 rev1=1 crypto rx=0x7fe12402ade0 tx=0x7fe124031660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=75907863e95fb9be server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:52:14.280 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:14.267+0000 7fe10bfff640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-09-21T03:52:14.281 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:14.267+0000 7fe10bfff640 1 -- 172.21.15.39:0/2617749365 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe12404a040 con 0x55eda1370070 2024-09-21T03:52:14.281 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:14.267+0000 7fe10bfff640 1 -- 172.21.15.39:0/2617749365 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8484+0+0 (secure 0 0 0) 0x7fe124003970 con 0x55eda1370070 2024-09-21T03:52:14.281 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:14.267+0000 7fe13914e740 1 -- 172.21.15.39:0/2617749365 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe120028860 con 0x55eda1370070 2024-09-21T03:52:14.281 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:14.267+0000 7fe10bfff640 1 -- 172.21.15.39:0/2617749365 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fe124009360 con 0x55eda1370070 2024-09-21T03:52:14.281 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:14.267+0000 7fe13914e740 10 client.17091 fetch_fsmap learned FSMap version 29 2024-09-21T03:52:14.281 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:14.267+0000 7fe13914e740 1 -- 172.21.15.39:0/2617749365 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55eda179fd80 con 0x55eda1370070 2024-09-21T03:52:14.282 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:14.267+0000 7fe10bfff640 1 -- 172.21.15.39:0/2617749365 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe120028860 con 0x55eda1370070 2024-09-21T03:52:14.282 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:14.267+0000 7fe10bfff640 1 -- 172.21.15.39:0/2617749365 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fe12404d030 con 0x55eda1370070 2024-09-21T03:52:14.282 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:14.267+0000 7fe13914e740 10 client.17091 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T03:52:14.282 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:14.267+0000 7fe13914e740 10 client.17091 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-21T03:52:14.282 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:14.267+0000 7fe13914e740 20 client.17091 populate_metadata read hostname 'smithi039' 2024-09-21T03:52:14.282 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:14.267+0000 7fe13914e740 1 --2- 172.21.15.39:0/2617749365 >> [v2:172.21.15.161:6838/2190107270,v1:172.21.15.161:6839/2190107270] conn(0x55eda1303da0 0x55eda17a2360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:52:14.282 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:14.267+0000 7fe13914e740 4 client.17091 mds_command: new command op to 24667 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T03:52:14.283 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:14.268+0000 7fe13914e740 1 -- 172.21.15.39:0/2617749365 --> [v2:172.21.15.161:6838/2190107270,v1:172.21.15.161:6839/2190107270] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55eda168b8a0 con 0x55eda1303da0 2024-09-21T03:52:14.283 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:14.268+0000 7fe130976640 1 --2- 172.21.15.39:0/2617749365 >> [v2:172.21.15.161:6838/2190107270,v1:172.21.15.161:6839/2190107270] conn(0x55eda1303da0 0x55eda17a2360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:52:14.283 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:14.268+0000 7fe130976640 1 --2- 172.21.15.39:0/2617749365 >> [v2:172.21.15.161:6838/2190107270,v1:172.21.15.161:6839/2190107270] conn(0x55eda1303da0 0x55eda17a2360 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=312b8f95011175ce server_cookie=2cf35b5d0619b14a in_seq=0 out_seq=0 2024-09-21T03:52:14.283 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:14.268+0000 7fe10bfff640 10 client.17091 ms_handle_connect on v2:172.21.15.161:6838/2190107270 2024-09-21T03:52:14.283 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:14.268+0000 7fe10bfff640 1 -- 172.21.15.39:0/2617749365 <== mds.1 v2:172.21.15.161:6838/2190107270 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55eda168b8a0 con 0x55eda1303da0 2024-09-21T03:52:14.283 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:14.268+0000 7fe10bfff640 10 client.17091 handle_command_reply: tid=0 2024-09-21T03:52:14.284 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:14.268+0000 7fe13914e740 1 client.17091 shutdown 2024-09-21T03:52:14.284 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:14.269+0000 7fe13914e740 1 -- 172.21.15.39:0/2617749365 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55eda1370070 msgr2=0x55eda179e830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:52:14.284 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:14.269+0000 7fe13914e740 1 --2- 172.21.15.39:0/2617749365 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55eda1370070 0x55eda179e830 secure :-1 s=READY pgs=938 cs=0 l=1 rev1=1 crypto rx=0x7fe12402ade0 tx=0x7fe124031660 comp rx=0 tx=0).stop 2024-09-21T03:52:14.284 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:14.269+0000 7fe13914e740 1 -- 172.21.15.39:0/2617749365 shutdown_connections 2024-09-21T03:52:14.284 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:14.269+0000 7fe13914e740 1 --2- 172.21.15.39:0/2617749365 >> [v2:172.21.15.161:6838/2190107270,v1:172.21.15.161:6839/2190107270] conn(0x55eda1303da0 0x55eda17a2360 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:14.284 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:14.269+0000 7fe13914e740 1 --2- 172.21.15.39:0/2617749365 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55eda1370070 0x55eda179e830 unknown :-1 s=CLOSED pgs=938 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:14.284 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:14.269+0000 7fe13914e740 1 --2- 172.21.15.39:0/2617749365 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55eda134f390 0x55eda1727950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:14.285 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:14.269+0000 7fe13914e740 1 --2- 172.21.15.39:0/2617749365 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55eda13471b0 0x55eda1727370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:14.285 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:14.269+0000 7fe13914e740 1 -- 172.21.15.39:0/2617749365 >> 172.21.15.39:0/2617749365 conn(0x55eda1364460 msgr2=0x55eda1724260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:52:14.285 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:14.269+0000 7fe10bfff640 0 client.17091 ms_handle_reset on v2:172.21.15.161:6838/2190107270 2024-09-21T03:52:14.285 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:14.269+0000 7fe13914e740 1 -- 172.21.15.39:0/2617749365 shutdown_connections 2024-09-21T03:52:14.285 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:14.269+0000 7fe13914e740 1 -- 172.21.15.39:0/2617749365 wait complete. 2024-09-21T03:52:14.285 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:14.269+0000 7fe13914e740 20 client.17091 trim_cache size 0 max 16384 2024-09-21T03:52:14.286 INFO:teuthology.orchestra.run.smithi039.stdout:[] 2024-09-21T03:52:14.287 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:14.269+0000 7fe132bda640 1 -- 172.21.15.39:0/2770667270 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7fe11c0b6680 msgr2=0x7fe11c0b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:52:14.287 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:14.270+0000 7fe132bda640 1 --2- 172.21.15.39:0/2770667270 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7fe11c0b6680 0x7fe11c0b8b90 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x55eda1716320 tx=0x7fe12000b390 comp rx=0 tx=0).stop 2024-09-21T03:52:14.287 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:14.270+0000 7fe132bda640 1 -- 172.21.15.39:0/2770667270 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe12c1083a0 msgr2=0x7fe12c19b7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:52:14.287 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:14.270+0000 7fe132bda640 1 --2- 172.21.15.39:0/2770667270 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe12c1083a0 0x7fe12c19b7b0 secure :-1 s=READY pgs=934 cs=0 l=1 rev1=1 crypto rx=0x7fe11800d770 tx=0x7fe11800dc30 comp rx=0 tx=0).stop 2024-09-21T03:52:14.288 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:14.270+0000 7fe132bda640 1 -- 172.21.15.39:0/2770667270 shutdown_connections 2024-09-21T03:52:14.288 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:14.270+0000 7fe132bda640 1 --2- 172.21.15.39:0/2770667270 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7fe11c0b6680 0x7fe11c0b8b90 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:14.288 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:14.270+0000 7fe132bda640 1 --2- 172.21.15.39:0/2770667270 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe12c108cf0 0x7fe12c19bcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:14.288 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:14.270+0000 7fe132bda640 1 --2- 172.21.15.39:0/2770667270 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe12c1083a0 0x7fe12c19b7b0 unknown :-1 s=CLOSED pgs=934 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:14.288 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:14.270+0000 7fe132bda640 1 --2- 172.21.15.39:0/2770667270 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe12c102a60 0x7fe12c19b270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:14.288 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:14.270+0000 7fe132bda640 1 -- 172.21.15.39:0/2770667270 >> 172.21.15.39:0/2770667270 conn(0x7fe12c079730 msgr2=0x7fe12c104460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:52:14.288 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:14.270+0000 7fe132bda640 1 -- 172.21.15.39:0/2770667270 shutdown_connections 2024-09-21T03:52:14.288 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:14.270+0000 7fe132bda640 1 -- 172.21.15.39:0/2770667270 wait complete. 2024-09-21T03:52:15.113 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 51116d5c-77ca-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 damage ls 2024-09-21T03:52:15.334 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:52:15 smithi039 ceph-mon[31211]: pgmap v511: 97 pgs: 97 active+clean; 10 MiB data, 520 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 22 KiB/s wr, 25 op/s 2024-09-21T03:52:15.376 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/51116d5c-77ca-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:52:15.475 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:52:15 smithi150 ceph-mon[34289]: pgmap v511: 97 pgs: 97 active+clean; 10 MiB data, 520 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 22 KiB/s wr, 25 op/s 2024-09-21T03:52:15.718 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:52:15 smithi161 ceph-mon[34533]: pgmap v511: 97 pgs: 97 active+clean; 10 MiB data, 520 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 22 KiB/s wr, 25 op/s 2024-09-21T03:52:15.885 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:15.881+0000 7f21ec1f0640 1 -- 172.21.15.39:0/2450274443 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f21e410bff0 msgr2=0x7f21e410c400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:52:15.885 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:15.881+0000 7f21ec1f0640 1 --2- 172.21.15.39:0/2450274443 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f21e410bff0 0x7f21e410c400 secure :-1 s=READY pgs=939 cs=0 l=1 rev1=1 crypto rx=0x7f21e0009900 tx=0x7f21e002a410 comp rx=0 tx=0).stop 2024-09-21T03:52:15.885 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:15.881+0000 7f21ec1f0640 1 -- 172.21.15.39:0/2450274443 shutdown_connections 2024-09-21T03:52:15.885 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:15.881+0000 7f21ec1f0640 1 --2- 172.21.15.39:0/2450274443 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f21e4115720 0x7f21e4117b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:15.885 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:15.881+0000 7f21ec1f0640 1 --2- 172.21.15.39:0/2450274443 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f21e4112dc0 0x7f21e41151e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:15.885 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:15.881+0000 7f21ec1f0640 1 --2- 172.21.15.39:0/2450274443 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f21e410bff0 0x7f21e410c400 unknown :-1 s=CLOSED pgs=939 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:15.885 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:15.882+0000 7f21ec1f0640 1 -- 172.21.15.39:0/2450274443 >> 172.21.15.39:0/2450274443 conn(0x7f21e4070ec0 msgr2=0x7f21e40712d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:52:15.885 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:15.882+0000 7f21ec1f0640 1 -- 172.21.15.39:0/2450274443 shutdown_connections 2024-09-21T03:52:15.886 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:15.882+0000 7f21ec1f0640 1 -- 172.21.15.39:0/2450274443 wait complete. 2024-09-21T03:52:15.886 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:15.882+0000 7f21ec1f0640 1 Processor -- start 2024-09-21T03:52:15.886 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:15.882+0000 7f21ec1f0640 1 -- start start 2024-09-21T03:52:15.886 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:15.883+0000 7f21ec1f0640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f21e410bff0 0x7f21e41a4ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:52:15.886 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:15.883+0000 7f21ec1f0640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f21e4112dc0 0x7f21e41a5440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:52:15.886 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:15.883+0000 7f21ec1f0640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f21e4115720 0x7f21e41aa6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:52:15.886 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:15.883+0000 7f21ec1f0640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f21e411b1f0 con 0x7f21e4115720 2024-09-21T03:52:15.886 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:15.883+0000 7f21ec1f0640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f21e411b070 con 0x7f21e410bff0 2024-09-21T03:52:15.886 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:15.883+0000 7f21ec1f0640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f21e411b370 con 0x7f21e4112dc0 2024-09-21T03:52:15.886 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:15.883+0000 7f21ea78d640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f21e4115720 0x7f21e41aa6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:52:15.886 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:15.883+0000 7f21ea78d640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f21e4115720 0x7f21e41aa6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:60004/0 (socket says 172.21.15.39:60004) 2024-09-21T03:52:15.887 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:15.883+0000 7f21ea78d640 1 -- 172.21.15.39:0/3559694743 learned_addr learned my addr 172.21.15.39:0/3559694743 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:52:15.887 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:15.883+0000 7f21e978b640 1 --2- 172.21.15.39:0/3559694743 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f21e4112dc0 0x7f21e41a5440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:52:15.887 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:15.883+0000 7f21e9f8c640 1 --2- 172.21.15.39:0/3559694743 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f21e410bff0 0x7f21e41a4ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:52:15.887 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:15.883+0000 7f21ea78d640 1 -- 172.21.15.39:0/3559694743 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f21e4112dc0 msgr2=0x7f21e41a5440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:52:15.887 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:15.883+0000 7f21ea78d640 1 --2- 172.21.15.39:0/3559694743 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f21e4112dc0 0x7f21e41a5440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:15.887 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:15.883+0000 7f21ea78d640 1 -- 172.21.15.39:0/3559694743 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f21e410bff0 msgr2=0x7f21e41a4ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:52:15.887 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:15.883+0000 7f21ea78d640 1 --2- 172.21.15.39:0/3559694743 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f21e410bff0 0x7f21e41a4ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:15.887 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:15.883+0000 7f21ea78d640 1 -- 172.21.15.39:0/3559694743 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f21e41aae10 con 0x7f21e4115720 2024-09-21T03:52:15.887 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:15.883+0000 7f21ea78d640 1 --2- 172.21.15.39:0/3559694743 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f21e4115720 0x7f21e41aa6e0 secure :-1 s=READY pgs=940 cs=0 l=1 rev1=1 crypto rx=0x7f21dc00eb10 tx=0x7f21dc00c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:52:15.888 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:15.884+0000 7f21daffd640 1 -- 172.21.15.39:0/3559694743 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21dc004b20 con 0x7f21e4115720 2024-09-21T03:52:15.888 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:15.884+0000 7f21daffd640 1 -- 172.21.15.39:0/3559694743 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f21dc004cc0 con 0x7f21e4115720 2024-09-21T03:52:15.888 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:15.884+0000 7f21daffd640 1 -- 172.21.15.39:0/3559694743 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21dc010670 con 0x7f21e4115720 2024-09-21T03:52:15.888 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:15.884+0000 7f21ec1f0640 1 -- 172.21.15.39:0/3559694743 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f21e41ab100 con 0x7f21e4115720 2024-09-21T03:52:15.888 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:15.884+0000 7f21ec1f0640 1 -- 172.21.15.39:0/3559694743 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f21e41b1860 con 0x7f21e4115720 2024-09-21T03:52:15.890 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:15.886+0000 7f21daffd640 1 -- 172.21.15.39:0/3559694743 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 27) ==== 154643+0+0 (secure 0 0 0) 0x7f21dc005020 con 0x7f21e4115720 2024-09-21T03:52:15.890 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:15.886+0000 7f21f2764740 1 Processor -- start 2024-09-21T03:52:15.890 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:15.887+0000 7f21f2764740 1 -- start start 2024-09-21T03:52:15.890 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:15.887+0000 7f21f2764740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x560e402eb070 0x560e406926a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:52:15.890 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:15.887+0000 7f21f2764740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560e402c21b0 0x560e40692be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:52:15.890 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:15.887+0000 7f21f2764740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560e402ca390 0x560e40693120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:52:15.890 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:15.887+0000 7f21f2764740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x560e40608ea0 con 0x560e402eb070 2024-09-21T03:52:15.890 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:15.887+0000 7f21f2764740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x560e405f82d0 con 0x560e402ca390 2024-09-21T03:52:15.890 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:15.887+0000 7f21f2764740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x560e405fc0e0 con 0x560e402c21b0 2024-09-21T03:52:15.890 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:15.887+0000 7f21e978b640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x560e402eb070 0x560e406926a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:52:15.890 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:15.887+0000 7f21e978b640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x560e402eb070 0x560e406926a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:60012/0 (socket says 172.21.15.39:60012) 2024-09-21T03:52:15.891 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:15.887+0000 7f21e978b640 1 -- 172.21.15.39:0/490934264 learned_addr learned my addr 172.21.15.39:0/490934264 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:52:15.891 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:15.887+0000 7f21e9f8c640 1 --2- 172.21.15.39:0/490934264 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560e402c21b0 0x560e40692be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:52:15.891 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:15.887+0000 7f21e978b640 1 --2- 172.21.15.39:0/490934264 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560e402ca390 0x560e40693120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:52:15.891 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:15.888+0000 7f21e978b640 1 -- 172.21.15.39:0/490934264 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560e402c21b0 msgr2=0x560e40692be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:52:15.891 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:15.888+0000 7f21e978b640 1 --2- 172.21.15.39:0/490934264 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560e402c21b0 0x560e40692be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:15.891 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:15.888+0000 7f21e978b640 1 -- 172.21.15.39:0/490934264 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560e402ca390 msgr2=0x560e40693120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:52:15.891 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:15.888+0000 7f21e978b640 1 --2- 172.21.15.39:0/490934264 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560e402ca390 0x560e40693120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:15.891 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:15.888+0000 7f21e978b640 1 -- 172.21.15.39:0/490934264 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560e40356650 con 0x560e402eb070 2024-09-21T03:52:15.891 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:15.888+0000 7f21daffd640 1 --2- 172.21.15.39:0/3559694743 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f21d00b6700 0x7f21d00b8c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:52:15.891 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:15.888+0000 7f21e978b640 1 --2- 172.21.15.39:0/490934264 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x560e402eb070 0x560e406926a0 secure :-1 s=READY pgs=941 cs=0 l=1 rev1=1 crypto rx=0x7f21d400bea0 tx=0x7f21d400b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a2d594e05d0801d server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:52:15.891 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:15.888+0000 7f21e9f8c640 1 --2- 172.21.15.39:0/3559694743 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f21d00b6700 0x7f21d00b8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:52:15.891 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:15.888+0000 7f21daffd640 1 -- 172.21.15.39:0/3559694743 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8484+0+0 (secure 0 0 0) 0x7f21dc0e7cb0 con 0x7f21e4115720 2024-09-21T03:52:15.891 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:15.888+0000 7f21cffff640 1 -- 172.21.15.39:0/490934264 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21d4019070 con 0x560e402eb070 2024-09-21T03:52:15.891 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:15.888+0000 7f21cffff640 1 -- 172.21.15.39:0/490934264 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f21d40054d0 con 0x560e402eb070 2024-09-21T03:52:15.891 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:15.888+0000 7f21cffff640 1 -- 172.21.15.39:0/490934264 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21d4004b90 con 0x560e402eb070 2024-09-21T03:52:15.894 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:15.889+0000 7f21f2764740 1 -- 172.21.15.39:0/490934264 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x560e402eb070 msgr2=0x560e406926a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:52:15.894 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:15.889+0000 7f21f2764740 1 --2- 172.21.15.39:0/490934264 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x560e402eb070 0x560e406926a0 secure :-1 s=READY pgs=941 cs=0 l=1 rev1=1 crypto rx=0x7f21d400bea0 tx=0x7f21d400b8e0 comp rx=0 tx=0).stop 2024-09-21T03:52:15.894 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:15.889+0000 7f21e9f8c640 1 --2- 172.21.15.39:0/3559694743 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f21d00b6700 0x7f21d00b8c10 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f21e0031440 tx=0x7f21e0009680 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:52:15.894 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:15.889+0000 7f21f2764740 1 -- 172.21.15.39:0/490934264 shutdown_connections 2024-09-21T03:52:15.895 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:15.889+0000 7f21f2764740 1 --2- 172.21.15.39:0/490934264 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560e402ca390 0x560e40693120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:15.895 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:15.889+0000 7f21f2764740 1 --2- 172.21.15.39:0/490934264 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560e402c21b0 0x560e40692be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:15.895 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:15.889+0000 7f21f2764740 1 --2- 172.21.15.39:0/490934264 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x560e402eb070 0x560e406926a0 unknown :-1 s=CLOSED pgs=941 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:15.895 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:15.889+0000 7f21f2764740 1 -- 172.21.15.39:0/490934264 >> 172.21.15.39:0/490934264 conn(0x560e402df460 msgr2=0x560e40690590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:52:15.895 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:15.889+0000 7f21f2764740 1 -- 172.21.15.39:0/490934264 shutdown_connections 2024-09-21T03:52:15.895 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:15.889+0000 7f21f2764740 1 -- 172.21.15.39:0/490934264 wait complete. 2024-09-21T03:52:15.895 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:15.890+0000 7f21f2764740 1 -- start start 2024-09-21T03:52:15.895 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:15.890+0000 7f21f2764740 1 Processor -- start 2024-09-21T03:52:15.895 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:15.890+0000 7f21f2764740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x560e402c21b0 0x560e4069dd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:52:15.895 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:15.890+0000 7f21f2764740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560e402ca390 0x560e4069e260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:52:15.895 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:15.890+0000 7f21f2764740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560e402eb070 0x560e4069e7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:52:15.895 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:15.890+0000 7f21f2764740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x560e40459700 con 0x560e402c21b0 2024-09-21T03:52:15.895 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:15.890+0000 7f21f2764740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x560e40593160 con 0x560e402eb070 2024-09-21T03:52:15.895 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:15.890+0000 7f21f2764740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x560e406a0e80 con 0x560e402ca390 2024-09-21T03:52:15.895 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:15.890+0000 7f21e9f8c640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x560e402c21b0 0x560e4069dd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:52:15.896 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:15.890+0000 7f21f2764740 10 client.0 _ll_register_callbacks cb 0x560e405ac9e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:52:15.896 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:15.890+0000 7f21e9f8c640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x560e402c21b0 0x560e4069dd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:60018/0 (socket says 172.21.15.39:60018) 2024-09-21T03:52:15.896 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:15.890+0000 7f21e9f8c640 1 -- 172.21.15.39:0/106277515 learned_addr learned my addr 172.21.15.39:0/106277515 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:52:15.896 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:15.891+0000 7f21e978b640 1 --2- 172.21.15.39:0/106277515 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560e402ca390 0x560e4069e260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:52:15.896 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:15.891+0000 7f21ea78d640 1 --2- 172.21.15.39:0/106277515 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560e402eb070 0x560e4069e7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:52:15.896 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:15.891+0000 7f21e9f8c640 1 -- 172.21.15.39:0/106277515 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560e402ca390 msgr2=0x560e4069e260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:52:15.896 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:15.891+0000 7f21e9f8c640 1 --2- 172.21.15.39:0/106277515 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560e402ca390 0x560e4069e260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:15.896 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:15.891+0000 7f21e9f8c640 1 -- 172.21.15.39:0/106277515 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560e402eb070 msgr2=0x560e4069e7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:52:15.896 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:15.891+0000 7f21e9f8c640 1 --2- 172.21.15.39:0/106277515 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560e402eb070 0x560e4069e7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:15.896 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:15.891+0000 7f21e9f8c640 1 -- 172.21.15.39:0/106277515 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560e405c0c90 con 0x560e402c21b0 2024-09-21T03:52:15.896 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:15.891+0000 7f21ea78d640 1 --2- 172.21.15.39:0/106277515 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560e402eb070 0x560e4069e7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:52:15.896 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:15.891+0000 7f21e9f8c640 1 --2- 172.21.15.39:0/106277515 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x560e402c21b0 0x560e4069dd20 secure :-1 s=READY pgs=942 cs=0 l=1 rev1=1 crypto rx=0x7f21e0033000 tx=0x7f21e0009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7734089f9575df1f server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:52:15.896 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:15.891+0000 7f21cd7fa640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-09-21T03:52:15.896 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:15.891+0000 7f21cd7fa640 1 -- 172.21.15.39:0/106277515 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21e004b070 con 0x560e402c21b0 2024-09-21T03:52:15.897 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:15.891+0000 7f21cd7fa640 1 -- 172.21.15.39:0/106277515 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8484+0+0 (secure 0 0 0) 0x7f21e0005460 con 0x560e402c21b0 2024-09-21T03:52:15.897 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:15.891+0000 7f21f2764740 1 -- 172.21.15.39:0/106277515 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f21d40054d0 con 0x560e402c21b0 2024-09-21T03:52:15.897 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:15.891+0000 7f21cd7fa640 1 -- 172.21.15.39:0/106277515 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f21e0057050 con 0x560e402c21b0 2024-09-21T03:52:15.897 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:15.891+0000 7f21f2764740 10 client.17103 fetch_fsmap learned FSMap version 29 2024-09-21T03:52:15.897 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:15.891+0000 7f21f2764740 1 -- 172.21.15.39:0/106277515 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x560e406acb30 con 0x560e402c21b0 2024-09-21T03:52:15.897 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:15.892+0000 7f21cd7fa640 1 -- 172.21.15.39:0/106277515 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f21d40054d0 con 0x560e402c21b0 2024-09-21T03:52:15.897 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:15.892+0000 7f21cd7fa640 1 -- 172.21.15.39:0/106277515 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f21e00035f0 con 0x560e402c21b0 2024-09-21T03:52:15.897 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:15.892+0000 7f21f2764740 10 client.17103 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T03:52:15.897 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:15.892+0000 7f21f2764740 10 client.17103 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T03:52:15.897 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:15.892+0000 7f21f2764740 20 client.17103 populate_metadata read hostname 'smithi039' 2024-09-21T03:52:15.897 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:15.892+0000 7f21f2764740 1 --2- 172.21.15.39:0/106277515 >> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] conn(0x560e402a99f0 0x560e406b0010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:52:15.897 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:15.892+0000 7f21f2764740 4 client.17103 mds_command: new command op to 24695 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:52:15.897 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:15.892+0000 7f21f2764740 1 -- 172.21.15.39:0/106277515 --> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560e4060adf0 con 0x560e402a99f0 2024-09-21T03:52:15.897 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:15.892+0000 7f21e978b640 1 --2- 172.21.15.39:0/106277515 >> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] conn(0x560e402a99f0 0x560e406b0010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:52:15.897 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:15.893+0000 7f21cd7fa640 10 client.17103 ms_handle_connect on v2:172.21.15.150:6838/2949386907 2024-09-21T03:52:15.898 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:15.893+0000 7f21e978b640 1 --2- 172.21.15.39:0/106277515 >> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] conn(0x560e402a99f0 0x560e406b0010 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2d4fa009d7ea12ab server_cookie=482bfb93de7483c3 in_seq=0 out_seq=0 2024-09-21T03:52:15.898 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:15.894+0000 7f21cd7fa640 1 -- 172.21.15.39:0/106277515 <== mds.0 v2:172.21.15.150:6838/2949386907 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x560e4060adf0 con 0x560e402a99f0 2024-09-21T03:52:15.899 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:15.894+0000 7f21cd7fa640 10 client.17103 handle_command_reply: tid=0 2024-09-21T03:52:15.899 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:15.894+0000 7f21f2764740 1 client.17103 shutdown 2024-09-21T03:52:15.899 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:15.894+0000 7f21f2764740 1 -- 172.21.15.39:0/106277515 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x560e402c21b0 msgr2=0x560e4069dd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:52:15.899 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:15.894+0000 7f21f2764740 1 --2- 172.21.15.39:0/106277515 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x560e402c21b0 0x560e4069dd20 secure :-1 s=READY pgs=942 cs=0 l=1 rev1=1 crypto rx=0x7f21e0033000 tx=0x7f21e0009260 comp rx=0 tx=0).stop 2024-09-21T03:52:15.899 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:15.894+0000 7f21f2764740 1 -- 172.21.15.39:0/106277515 shutdown_connections 2024-09-21T03:52:15.899 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:15.894+0000 7f21f2764740 1 --2- 172.21.15.39:0/106277515 >> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] conn(0x560e402a99f0 0x560e406b0010 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:15.899 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:15.894+0000 7f21f2764740 1 --2- 172.21.15.39:0/106277515 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560e402eb070 0x560e4069e7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:15.899 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:15.894+0000 7f21f2764740 1 --2- 172.21.15.39:0/106277515 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560e402ca390 0x560e4069e260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:15.899 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:15.894+0000 7f21cd7fa640 0 client.17103 ms_handle_reset on v2:172.21.15.150:6838/2949386907 2024-09-21T03:52:15.899 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:15.894+0000 7f21f2764740 1 --2- 172.21.15.39:0/106277515 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x560e402c21b0 0x560e4069dd20 unknown :-1 s=CLOSED pgs=942 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:15.899 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:15.894+0000 7f21f2764740 1 -- 172.21.15.39:0/106277515 >> 172.21.15.39:0/106277515 conn(0x560e402df460 msgr2=0x560e40695850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:52:15.899 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:15.894+0000 7f21f2764740 1 -- 172.21.15.39:0/106277515 shutdown_connections 2024-09-21T03:52:15.899 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:15.894+0000 7f21f2764740 1 -- 172.21.15.39:0/106277515 wait complete. 2024-09-21T03:52:15.899 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:15.894+0000 7f21f2764740 20 client.17103 trim_cache size 0 max 16384 2024-09-21T03:52:15.910 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:15.907+0000 7f21f2764740 1 Processor -- start 2024-09-21T03:52:15.910 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:15.908+0000 7f21f2764740 1 -- start start 2024-09-21T03:52:15.910 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:15.908+0000 7f21f2764740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560e402c21b0 0x560e4069dad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:52:15.911 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:15.908+0000 7f21f2764740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560e402ca390 0x560e4069e010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:52:15.911 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:15.908+0000 7f21f2764740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x560e402eb070 0x560e4069af10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:52:15.911 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:15.908+0000 7f21f2764740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f21e41ba040 con 0x560e402eb070 2024-09-21T03:52:15.911 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:15.908+0000 7f21f2764740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f21e41bbb80 con 0x560e402c21b0 2024-09-21T03:52:15.911 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:15.908+0000 7f21f2764740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f21e41ba710 con 0x560e402ca390 2024-09-21T03:52:15.911 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:15.908+0000 7f21ea78d640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x560e402eb070 0x560e4069af10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:52:15.911 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:15.908+0000 7f21ea78d640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x560e402eb070 0x560e4069af10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:60032/0 (socket says 172.21.15.39:60032) 2024-09-21T03:52:15.911 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:15.908+0000 7f21ea78d640 1 -- 172.21.15.39:0/971632920 learned_addr learned my addr 172.21.15.39:0/971632920 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:52:15.911 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:15.908+0000 7f21e978b640 1 --2- 172.21.15.39:0/971632920 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560e402ca390 0x560e4069e010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:52:15.911 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:15.908+0000 7f21e9f8c640 1 --2- 172.21.15.39:0/971632920 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560e402c21b0 0x560e4069dad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:52:15.911 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:15.908+0000 7f21ea78d640 1 -- 172.21.15.39:0/971632920 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560e402ca390 msgr2=0x560e4069e010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:52:15.911 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:15.908+0000 7f21ea78d640 1 --2- 172.21.15.39:0/971632920 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560e402ca390 0x560e4069e010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:15.911 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:15.908+0000 7f21ea78d640 1 -- 172.21.15.39:0/971632920 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560e402c21b0 msgr2=0x560e4069dad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:52:15.912 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:15.908+0000 7f21ea78d640 1 --2- 172.21.15.39:0/971632920 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560e402c21b0 0x560e4069dad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:15.912 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:15.908+0000 7f21ea78d640 1 -- 172.21.15.39:0/971632920 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560e406aa0a0 con 0x560e402eb070 2024-09-21T03:52:15.912 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:15.909+0000 7f21ea78d640 1 --2- 172.21.15.39:0/971632920 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x560e402eb070 0x560e4069af10 secure :-1 s=READY pgs=943 cs=0 l=1 rev1=1 crypto rx=0x7f21dc0e5480 tx=0x7f21dc0a9a80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f713a6a879fa35c8 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:52:15.912 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:15.909+0000 7f21cf7fe640 1 -- 172.21.15.39:0/971632920 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21dc0b0e60 con 0x560e402eb070 2024-09-21T03:52:15.912 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:15.909+0000 7f21cf7fe640 1 -- 172.21.15.39:0/971632920 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f21dc0aee50 con 0x560e402eb070 2024-09-21T03:52:15.912 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:15.909+0000 7f21cf7fe640 1 -- 172.21.15.39:0/971632920 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21dc0b7540 con 0x560e402eb070 2024-09-21T03:52:15.915 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:15.909+0000 7f21f2764740 1 -- 172.21.15.39:0/971632920 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x560e402eb070 msgr2=0x560e4069af10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:52:15.916 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:15.909+0000 7f21f2764740 1 --2- 172.21.15.39:0/971632920 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x560e402eb070 0x560e4069af10 secure :-1 s=READY pgs=943 cs=0 l=1 rev1=1 crypto rx=0x7f21dc0e5480 tx=0x7f21dc0a9a80 comp rx=0 tx=0).stop 2024-09-21T03:52:15.916 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:15.909+0000 7f21f2764740 1 -- 172.21.15.39:0/971632920 shutdown_connections 2024-09-21T03:52:15.916 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:15.909+0000 7f21f2764740 1 --2- 172.21.15.39:0/971632920 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x560e402eb070 0x560e4069af10 unknown :-1 s=CLOSED pgs=943 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:15.916 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:15.909+0000 7f21f2764740 1 --2- 172.21.15.39:0/971632920 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560e402ca390 0x560e4069e010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:15.916 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:15.909+0000 7f21f2764740 1 --2- 172.21.15.39:0/971632920 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560e402c21b0 0x560e4069dad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:15.916 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:15.909+0000 7f21f2764740 1 -- 172.21.15.39:0/971632920 >> 172.21.15.39:0/971632920 conn(0x560e402df460 msgr2=0x560e4069c5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:52:15.916 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:15.910+0000 7f21f2764740 1 -- 172.21.15.39:0/971632920 shutdown_connections 2024-09-21T03:52:15.916 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:15.910+0000 7f21f2764740 1 -- 172.21.15.39:0/971632920 wait complete. 2024-09-21T03:52:15.916 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:15.910+0000 7f21f2764740 1 -- start start 2024-09-21T03:52:15.916 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:15.910+0000 7f21f2764740 1 Processor -- start 2024-09-21T03:52:15.916 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:15.911+0000 7f21f2764740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560e402c21b0 0x560e406a2120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:52:15.916 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:15.911+0000 7f21f2764740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x560e402ca390 0x560e406a2660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:52:15.917 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:15.911+0000 7f21f2764740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560e402eb070 0x560e406a2ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:52:15.917 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:15.911+0000 7f21f2764740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f21e41bb4b0 con 0x560e402ca390 2024-09-21T03:52:15.917 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:15.911+0000 7f21f2764740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f21e41bade0 con 0x560e402eb070 2024-09-21T03:52:15.917 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:15.911+0000 7f21f2764740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f21e41b8020 con 0x560e402c21b0 2024-09-21T03:52:15.917 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:15.911+0000 7f21e978b640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x560e402ca390 0x560e406a2660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:52:15.917 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:15.911+0000 7f21f2764740 10 client.0 _ll_register_callbacks cb 0x7f21dc0093a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:52:15.917 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:15.911+0000 7f21e978b640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x560e402ca390 0x560e406a2660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:60036/0 (socket says 172.21.15.39:60036) 2024-09-21T03:52:15.917 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:15.911+0000 7f21e978b640 1 -- 172.21.15.39:0/1656682811 learned_addr learned my addr 172.21.15.39:0/1656682811 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:52:15.917 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:15.911+0000 7f21e9f8c640 1 --2- 172.21.15.39:0/1656682811 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560e402c21b0 0x560e406a2120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:52:15.917 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:15.911+0000 7f21ea78d640 1 --2- 172.21.15.39:0/1656682811 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560e402eb070 0x560e406a2ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:52:15.917 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:15.911+0000 7f21e978b640 1 -- 172.21.15.39:0/1656682811 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560e402c21b0 msgr2=0x560e406a2120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:52:15.917 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:15.911+0000 7f21e978b640 1 --2- 172.21.15.39:0/1656682811 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560e402c21b0 0x560e406a2120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:15.917 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:15.911+0000 7f21e978b640 1 -- 172.21.15.39:0/1656682811 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560e402eb070 msgr2=0x560e406a2ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:52:15.917 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:15.911+0000 7f21e978b640 1 --2- 172.21.15.39:0/1656682811 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560e402eb070 0x560e406a2ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:15.917 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:15.911+0000 7f21e978b640 1 -- 172.21.15.39:0/1656682811 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560e40719a60 con 0x560e402ca390 2024-09-21T03:52:15.917 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:15.911+0000 7f21ea78d640 1 --2- 172.21.15.39:0/1656682811 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560e402eb070 0x560e406a2ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:52:15.918 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:15.911+0000 7f21e9f8c640 1 --2- 172.21.15.39:0/1656682811 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560e402c21b0 0x560e406a2120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:52:15.918 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:15.911+0000 7f21e978b640 1 --2- 172.21.15.39:0/1656682811 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x560e402ca390 0x560e406a2660 secure :-1 s=READY pgs=944 cs=0 l=1 rev1=1 crypto rx=0x7f21d400bea0 tx=0x7f21d4010f30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dba00979611f5b64 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:52:15.918 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:15.911+0000 7f21cd7fa640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-09-21T03:52:15.918 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:15.911+0000 7f21cd7fa640 1 -- 172.21.15.39:0/1656682811 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21d4019040 con 0x560e402ca390 2024-09-21T03:52:15.918 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:15.912+0000 7f21cd7fa640 1 -- 172.21.15.39:0/1656682811 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8484+0+0 (secure 0 0 0) 0x7f21d4028620 con 0x560e402ca390 2024-09-21T03:52:15.918 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:15.912+0000 7f21f2764740 1 -- 172.21.15.39:0/1656682811 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f21dc0aee50 con 0x560e402ca390 2024-09-21T03:52:15.918 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:15.912+0000 7f21cd7fa640 1 -- 172.21.15.39:0/1656682811 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f21d4028820 con 0x560e402ca390 2024-09-21T03:52:15.918 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:15.912+0000 7f21f2764740 10 client.17109 fetch_fsmap learned FSMap version 29 2024-09-21T03:52:15.918 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:15.912+0000 7f21f2764740 1 -- 172.21.15.39:0/1656682811 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x560e4071ace0 con 0x560e402ca390 2024-09-21T03:52:15.918 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:15.912+0000 7f21cd7fa640 1 -- 172.21.15.39:0/1656682811 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f21dc0aee50 con 0x560e402ca390 2024-09-21T03:52:15.918 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:15.912+0000 7f21cd7fa640 1 -- 172.21.15.39:0/1656682811 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f21d4036070 con 0x560e402ca390 2024-09-21T03:52:15.918 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:15.912+0000 7f21f2764740 10 client.17109 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T03:52:15.918 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:15.912+0000 7f21f2764740 10 client.17109 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T03:52:15.918 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:15.912+0000 7f21f2764740 20 client.17109 populate_metadata read hostname 'smithi039' 2024-09-21T03:52:15.919 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:15.912+0000 7f21f2764740 1 --2- 172.21.15.39:0/1656682811 >> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] conn(0x560e4027eda0 0x560e4071d300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:52:15.919 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:15.912+0000 7f21f2764740 4 client.17109 mds_command: new command op to 24695 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T03:52:15.919 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:15.912+0000 7f21f2764740 1 -- 172.21.15.39:0/1656682811 --> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] -- command(tid 0: {"prefix": "damage ls"}) -- 0x560e406068a0 con 0x560e4027eda0 2024-09-21T03:52:15.919 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:15.913+0000 7f21e9f8c640 1 --2- 172.21.15.39:0/1656682811 >> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] conn(0x560e4027eda0 0x560e4071d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:52:15.919 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:15.913+0000 7f21cd7fa640 10 client.17109 ms_handle_connect on v2:172.21.15.150:6838/2949386907 2024-09-21T03:52:15.919 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:15.913+0000 7f21e9f8c640 1 --2- 172.21.15.39:0/1656682811 >> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] conn(0x560e4027eda0 0x560e4071d300 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c6c83d21ed2d6e38 server_cookie=58b7c0be2cc0942b in_seq=0 out_seq=0 2024-09-21T03:52:15.919 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:15.913+0000 7f21cd7fa640 1 -- 172.21.15.39:0/1656682811 <== mds.0 v2:172.21.15.150:6838/2949386907 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x560e406068a0 con 0x560e4027eda0 2024-09-21T03:52:15.919 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:15.913+0000 7f21cd7fa640 10 client.17109 handle_command_reply: tid=0 2024-09-21T03:52:15.919 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:15.913+0000 7f21f2764740 1 client.17109 shutdown 2024-09-21T03:52:15.919 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:15.913+0000 7f21f2764740 1 -- 172.21.15.39:0/1656682811 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x560e402ca390 msgr2=0x560e406a2660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:52:15.919 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:15.913+0000 7f21f2764740 1 --2- 172.21.15.39:0/1656682811 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x560e402ca390 0x560e406a2660 secure :-1 s=READY pgs=944 cs=0 l=1 rev1=1 crypto rx=0x7f21d400bea0 tx=0x7f21d4010f30 comp rx=0 tx=0).stop 2024-09-21T03:52:15.919 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:15.914+0000 7f21f2764740 1 -- 172.21.15.39:0/1656682811 shutdown_connections 2024-09-21T03:52:15.919 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:15.914+0000 7f21f2764740 1 --2- 172.21.15.39:0/1656682811 >> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] conn(0x560e4027eda0 0x560e4071d300 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:15.920 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:15.914+0000 7f21f2764740 1 --2- 172.21.15.39:0/1656682811 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560e402eb070 0x560e406a2ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:15.920 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:15.914+0000 7f21f2764740 1 --2- 172.21.15.39:0/1656682811 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x560e402ca390 0x560e406a2660 unknown :-1 s=CLOSED pgs=944 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:15.920 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:15.914+0000 7f21f2764740 1 --2- 172.21.15.39:0/1656682811 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560e402c21b0 0x560e406a2120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:15.920 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:15.914+0000 7f21f2764740 1 -- 172.21.15.39:0/1656682811 >> 172.21.15.39:0/1656682811 conn(0x560e402df460 msgr2=0x560e4069dad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:52:15.920 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:15.914+0000 7f21cd7fa640 0 client.17109 ms_handle_reset on v2:172.21.15.150:6838/2949386907 2024-09-21T03:52:15.920 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:15.914+0000 7f21f2764740 1 -- 172.21.15.39:0/1656682811 shutdown_connections 2024-09-21T03:52:15.920 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:15.914+0000 7f21f2764740 1 -- 172.21.15.39:0/1656682811 wait complete. 2024-09-21T03:52:15.920 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:15.914+0000 7f21f2764740 20 client.17109 trim_cache size 0 max 16384 2024-09-21T03:52:15.920 INFO:teuthology.orchestra.run.smithi039.stdout:[] 2024-09-21T03:52:15.921 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:15.914+0000 7f21ec1f0640 1 -- 172.21.15.39:0/3559694743 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f21d00b6700 msgr2=0x7f21d00b8c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:52:15.921 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:15.914+0000 7f21ec1f0640 1 --2- 172.21.15.39:0/3559694743 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f21d00b6700 0x7f21d00b8c10 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f21e0031440 tx=0x7f21e0009680 comp rx=0 tx=0).stop 2024-09-21T03:52:15.921 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:15.914+0000 7f21ec1f0640 1 -- 172.21.15.39:0/3559694743 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f21e4115720 msgr2=0x7f21e41aa6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:52:15.921 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:15.914+0000 7f21ec1f0640 1 --2- 172.21.15.39:0/3559694743 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f21e4115720 0x7f21e41aa6e0 secure :-1 s=READY pgs=940 cs=0 l=1 rev1=1 crypto rx=0x7f21dc00eb10 tx=0x7f21dc00c2d0 comp rx=0 tx=0).stop 2024-09-21T03:52:15.921 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:15.915+0000 7f21ec1f0640 1 -- 172.21.15.39:0/3559694743 shutdown_connections 2024-09-21T03:52:15.921 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:15.915+0000 7f21ec1f0640 1 --2- 172.21.15.39:0/3559694743 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f21d00b6700 0x7f21d00b8c10 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:15.921 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:15.915+0000 7f21ec1f0640 1 --2- 172.21.15.39:0/3559694743 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f21e4115720 0x7f21e41aa6e0 unknown :-1 s=CLOSED pgs=940 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:15.921 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:15.915+0000 7f21ec1f0640 1 --2- 172.21.15.39:0/3559694743 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f21e4112dc0 0x7f21e41a5440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:15.922 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:15.915+0000 7f21ec1f0640 1 --2- 172.21.15.39:0/3559694743 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f21e410bff0 0x7f21e41a4ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:15.922 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:15.915+0000 7f21ec1f0640 1 -- 172.21.15.39:0/3559694743 >> 172.21.15.39:0/3559694743 conn(0x7f21e4070ec0 msgr2=0x7f21e4111180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:52:15.922 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:15.915+0000 7f21ec1f0640 1 -- 172.21.15.39:0/3559694743 shutdown_connections 2024-09-21T03:52:15.922 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:15.915+0000 7f21ec1f0640 1 -- 172.21.15.39:0/3559694743 wait complete. 2024-09-21T03:52:17.584 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:52:17 smithi039 ceph-mon[31211]: pgmap v512: 97 pgs: 97 active+clean; 10 MiB data, 520 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 25 op/s 2024-09-21T03:52:17.691 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-21T03:52:17.691 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 51116d5c-77ca-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-21T03:52:17.718 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:52:17 smithi161 ceph-mon[34533]: pgmap v512: 97 pgs: 97 active+clean; 10 MiB data, 520 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 25 op/s 2024-09-21T03:52:17.724 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:52:17 smithi150 ceph-mon[34289]: pgmap v512: 97 pgs: 97 active+clean; 10 MiB data, 520 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 25 op/s 2024-09-21T03:52:17.963 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/51116d5c-77ca-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:52:18.585 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:18.583+0000 7fb5bbbc9640 1 -- 172.21.15.39:0/3830167732 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb5b4103b80 msgr2=0x7fb5b4103f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:52:18.585 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:18.583+0000 7fb5bbbc9640 1 --2- 172.21.15.39:0/3830167732 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb5b4103b80 0x7fb5b4103f90 secure :-1 s=READY pgs=945 cs=0 l=1 rev1=1 crypto rx=0x7fb5b0009900 tx=0x7fb5b002a410 comp rx=0 tx=0).stop 2024-09-21T03:52:18.585 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:18.583+0000 7fb5bbbc9640 1 -- 172.21.15.39:0/3830167732 shutdown_connections 2024-09-21T03:52:18.585 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:18.583+0000 7fb5bbbc9640 1 --2- 172.21.15.39:0/3830167732 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb5b4109e10 0x7fb5b4110640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:18.585 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:18.583+0000 7fb5bbbc9640 1 --2- 172.21.15.39:0/3830167732 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb5b41094c0 0x7fb5b41098d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:18.585 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:18.583+0000 7fb5bbbc9640 1 --2- 172.21.15.39:0/3830167732 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb5b4103b80 0x7fb5b4103f90 unknown :-1 s=CLOSED pgs=945 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:18.585 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:18.583+0000 7fb5bbbc9640 1 -- 172.21.15.39:0/3830167732 >> 172.21.15.39:0/3830167732 conn(0x7fb5b40ff5d0 msgr2=0x7fb5b4101a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:52:18.585 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:18.583+0000 7fb5bbbc9640 1 -- 172.21.15.39:0/3830167732 shutdown_connections 2024-09-21T03:52:18.585 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:18.583+0000 7fb5bbbc9640 1 -- 172.21.15.39:0/3830167732 wait complete. 2024-09-21T03:52:18.587 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:18.584+0000 7fb5bbbc9640 1 Processor -- start 2024-09-21T03:52:18.587 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:18.584+0000 7fb5bbbc9640 1 -- start start 2024-09-21T03:52:18.587 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:18.584+0000 7fb5bbbc9640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb5b4103b80 0x7fb5b419c6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:52:18.587 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:18.584+0000 7fb5bbbc9640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb5b41094c0 0x7fb5b419cc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:52:18.588 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:18.584+0000 7fb5bbbc9640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb5b4109e10 0x7fb5b41a1f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:52:18.588 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:18.584+0000 7fb5bbbc9640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fb5b4113ad0 con 0x7fb5b4109e10 2024-09-21T03:52:18.588 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:18.584+0000 7fb5bbbc9640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fb5b4113950 con 0x7fb5b4103b80 2024-09-21T03:52:18.588 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:18.584+0000 7fb5bbbc9640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fb5b4113c50 con 0x7fb5b41094c0 2024-09-21T03:52:18.588 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:18.584+0000 7fb5ba166640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb5b4109e10 0x7fb5b41a1f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:52:18.588 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:18.584+0000 7fb5ba166640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb5b4109e10 0x7fb5b41a1f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:39832/0 (socket says 172.21.15.39:39832) 2024-09-21T03:52:18.588 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:18.584+0000 7fb5ba166640 1 -- 172.21.15.39:0/1953442232 learned_addr learned my addr 172.21.15.39:0/1953442232 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:52:18.588 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:18.584+0000 7fb5b9965640 1 --2- 172.21.15.39:0/1953442232 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb5b4103b80 0x7fb5b419c6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:52:18.588 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:18.584+0000 7fb5b9164640 1 --2- 172.21.15.39:0/1953442232 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb5b41094c0 0x7fb5b419cc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:52:18.588 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:18.584+0000 7fb5ba166640 1 -- 172.21.15.39:0/1953442232 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb5b41094c0 msgr2=0x7fb5b419cc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:52:18.588 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:18.584+0000 7fb5ba166640 1 --2- 172.21.15.39:0/1953442232 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb5b41094c0 0x7fb5b419cc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:18.588 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:18.584+0000 7fb5ba166640 1 -- 172.21.15.39:0/1953442232 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb5b4103b80 msgr2=0x7fb5b419c6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:52:18.588 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:18.584+0000 7fb5ba166640 1 --2- 172.21.15.39:0/1953442232 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb5b4103b80 0x7fb5b419c6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:18.588 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:18.585+0000 7fb5ba166640 1 -- 172.21.15.39:0/1953442232 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb5b41a2630 con 0x7fb5b4109e10 2024-09-21T03:52:18.588 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:18.585+0000 7fb5ba166640 1 --2- 172.21.15.39:0/1953442232 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb5b4109e10 0x7fb5b41a1f00 secure :-1 s=READY pgs=946 cs=0 l=1 rev1=1 crypto rx=0x7fb5ac00eaa0 tx=0x7fb5ac00ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:52:18.589 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:18.585+0000 7fb5aaffd640 1 -- 172.21.15.39:0/1953442232 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5ac00cdb0 con 0x7fb5b4109e10 2024-09-21T03:52:18.589 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:18.585+0000 7fb5aaffd640 1 -- 172.21.15.39:0/1953442232 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb5ac005020 con 0x7fb5b4109e10 2024-09-21T03:52:18.589 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:18.585+0000 7fb5bbbc9640 1 -- 172.21.15.39:0/1953442232 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb5b41a2920 con 0x7fb5b4109e10 2024-09-21T03:52:18.589 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:18.585+0000 7fb5aaffd640 1 -- 172.21.15.39:0/1953442232 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5ac0116d0 con 0x7fb5b4109e10 2024-09-21T03:52:18.589 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:18.585+0000 7fb5bbbc9640 1 -- 172.21.15.39:0/1953442232 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb5b41a9080 con 0x7fb5b4109e10 2024-09-21T03:52:18.589 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:18.588+0000 7fb5c213d740 1 Processor -- start 2024-09-21T03:52:18.589 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:18.588+0000 7fb5aaffd640 1 -- 172.21.15.39:0/1953442232 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 27) ==== 154643+0+0 (secure 0 0 0) 0x7fb5ac0051d0 con 0x7fb5b4109e10 2024-09-21T03:52:18.590 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:18.588+0000 7fb5c213d740 1 -- start start 2024-09-21T03:52:18.591 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:18.588+0000 7fb5c213d740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56298993b350 0x562989ce2430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:52:18.591 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:18.588+0000 7fb5c213d740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562989912490 0x562989ce2970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:52:18.592 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:18.588+0000 7fb5c213d740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56298991a670 0x562989ce2eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:52:18.592 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:18.588+0000 7fb5c213d740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x562989c58e60 con 0x56298993b350 2024-09-21T03:52:18.592 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:18.588+0000 7fb5c213d740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x562989c48290 con 0x562989912490 2024-09-21T03:52:18.592 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:18.588+0000 7fb5c213d740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x562989c4c0a0 con 0x56298991a670 2024-09-21T03:52:18.592 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:18.588+0000 7fb5b9164640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56298993b350 0x562989ce2430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:52:18.592 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:18.588+0000 7fb5b9164640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56298993b350 0x562989ce2430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:39844/0 (socket says 172.21.15.39:39844) 2024-09-21T03:52:18.592 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:18.588+0000 7fb5b9164640 1 -- 172.21.15.39:0/2528269687 learned_addr learned my addr 172.21.15.39:0/2528269687 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:52:18.592 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:18.589+0000 7fb5b9965640 1 --2- 172.21.15.39:0/2528269687 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562989912490 0x562989ce2970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:52:18.592 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:18.589+0000 7fb5b9164640 1 --2- 172.21.15.39:0/2528269687 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56298991a670 0x562989ce2eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:52:18.592 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:18.589+0000 7fb5b9164640 1 -- 172.21.15.39:0/2528269687 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56298991a670 msgr2=0x562989ce2eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:52:18.592 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:18.589+0000 7fb5b9164640 1 --2- 172.21.15.39:0/2528269687 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56298991a670 0x562989ce2eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:18.592 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:18.589+0000 7fb5b9164640 1 -- 172.21.15.39:0/2528269687 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562989912490 msgr2=0x562989ce2970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:52:18.592 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:18.589+0000 7fb5b9164640 1 --2- 172.21.15.39:0/2528269687 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562989912490 0x562989ce2970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:18.592 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:18.589+0000 7fb5b9164640 1 -- 172.21.15.39:0/2528269687 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5629899a6930 con 0x56298993b350 2024-09-21T03:52:18.592 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:18.589+0000 7fb5aaffd640 1 --2- 172.21.15.39:0/1953442232 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7fb5a00b6680 0x7fb5a00b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:52:18.593 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:18.589+0000 7fb5b9965640 1 --2- 172.21.15.39:0/1953442232 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7fb5a00b6680 0x7fb5a00b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:52:18.593 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:18.589+0000 7fb5b9164640 1 --2- 172.21.15.39:0/2528269687 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56298993b350 0x562989ce2430 secure :-1 s=READY pgs=947 cs=0 l=1 rev1=1 crypto rx=0x7fb5a400bb20 tx=0x7fb5a40041d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1dc0c067bc2d363c server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:52:18.593 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:18.589+0000 7fb5aaffd640 1 -- 172.21.15.39:0/1953442232 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8484+0+0 (secure 0 0 0) 0x7fb5ac0e7850 con 0x7fb5b4109e10 2024-09-21T03:52:18.593 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:18.589+0000 7fb59ffff640 1 -- 172.21.15.39:0/2528269687 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5a4005020 con 0x56298993b350 2024-09-21T03:52:18.593 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:18.589+0000 7fb59ffff640 1 -- 172.21.15.39:0/2528269687 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb5a4005670 con 0x56298993b350 2024-09-21T03:52:18.593 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:18.589+0000 7fb59ffff640 1 -- 172.21.15.39:0/2528269687 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5a40106c0 con 0x56298993b350 2024-09-21T03:52:18.596 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:18.590+0000 7fb5c213d740 1 -- 172.21.15.39:0/2528269687 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56298993b350 msgr2=0x562989ce2430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:52:18.596 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:18.590+0000 7fb5b9965640 1 --2- 172.21.15.39:0/1953442232 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7fb5a00b6680 0x7fb5a00b8b90 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fb5b0031440 tx=0x7fb5b0009610 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:52:18.596 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:18.590+0000 7fb5c213d740 1 --2- 172.21.15.39:0/2528269687 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56298993b350 0x562989ce2430 secure :-1 s=READY pgs=947 cs=0 l=1 rev1=1 crypto rx=0x7fb5a400bb20 tx=0x7fb5a40041d0 comp rx=0 tx=0).stop 2024-09-21T03:52:18.596 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:18.590+0000 7fb5c213d740 1 -- 172.21.15.39:0/2528269687 shutdown_connections 2024-09-21T03:52:18.596 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:18.590+0000 7fb5c213d740 1 --2- 172.21.15.39:0/2528269687 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56298991a670 0x562989ce2eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:18.596 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:18.590+0000 7fb5c213d740 1 --2- 172.21.15.39:0/2528269687 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562989912490 0x562989ce2970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:18.596 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:18.590+0000 7fb5c213d740 1 --2- 172.21.15.39:0/2528269687 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56298993b350 0x562989ce2430 unknown :-1 s=CLOSED pgs=947 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:18.596 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:18.590+0000 7fb5c213d740 1 -- 172.21.15.39:0/2528269687 >> 172.21.15.39:0/2528269687 conn(0x56298992f740 msgr2=0x562989ce0320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:52:18.596 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:18.591+0000 7fb5c213d740 1 -- 172.21.15.39:0/2528269687 shutdown_connections 2024-09-21T03:52:18.596 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:18.591+0000 7fb5c213d740 1 -- 172.21.15.39:0/2528269687 wait complete. 2024-09-21T03:52:18.597 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:18.591+0000 7fb5c213d740 1 -- start start 2024-09-21T03:52:18.597 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:18.591+0000 7fb5c213d740 1 Processor -- start 2024-09-21T03:52:18.597 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:18.592+0000 7fb5c213d740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562989912490 0x562989cee090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:52:18.597 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:18.592+0000 7fb5c213d740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56298991a670 0x562989cee610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:52:18.597 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:18.592+0000 7fb5c213d740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56298993b350 0x562989cf9ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:52:18.597 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:18.592+0000 7fb5c213d740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x562989aa9890 con 0x562989912490 2024-09-21T03:52:18.597 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:18.592+0000 7fb5c213d740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x562989be2d40 con 0x56298993b350 2024-09-21T03:52:18.597 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:18.592+0000 7fb5c213d740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x562989cf0bb0 con 0x56298991a670 2024-09-21T03:52:18.597 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:18.592+0000 7fb5c213d740 10 client.0 _ll_register_callbacks cb 0x562989c46c80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:52:18.597 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:18.592+0000 7fb5b9965640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562989912490 0x562989cee090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:52:18.597 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:18.592+0000 7fb5b9965640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562989912490 0x562989cee090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:39856/0 (socket says 172.21.15.39:39856) 2024-09-21T03:52:18.597 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:18.592+0000 7fb5b9965640 1 -- 172.21.15.39:0/2962059297 learned_addr learned my addr 172.21.15.39:0/2962059297 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:52:18.597 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:18.592+0000 7fb5b9164640 1 --2- 172.21.15.39:0/2962059297 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56298991a670 0x562989cee610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:52:18.597 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:18.592+0000 7fb5ba166640 1 --2- 172.21.15.39:0/2962059297 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56298993b350 0x562989cf9ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:52:18.598 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:18.592+0000 7fb5b9965640 1 -- 172.21.15.39:0/2962059297 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56298991a670 msgr2=0x562989cee610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:52:18.598 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:18.592+0000 7fb5b9965640 1 --2- 172.21.15.39:0/2962059297 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56298991a670 0x562989cee610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:18.598 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:18.592+0000 7fb5b9965640 1 -- 172.21.15.39:0/2962059297 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56298993b350 msgr2=0x562989cf9ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:52:18.598 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:18.592+0000 7fb5b9965640 1 --2- 172.21.15.39:0/2962059297 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56298993b350 0x562989cf9ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:18.598 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:18.592+0000 7fb5b9965640 1 -- 172.21.15.39:0/2962059297 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562989cfa4b0 con 0x562989912490 2024-09-21T03:52:18.598 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:18.592+0000 7fb5b9164640 1 --2- 172.21.15.39:0/2962059297 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56298991a670 0x562989cee610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:52:18.598 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:18.593+0000 7fb5b9965640 1 --2- 172.21.15.39:0/2962059297 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562989912490 0x562989cee090 secure :-1 s=READY pgs=948 cs=0 l=1 rev1=1 crypto rx=0x7fb5b0033000 tx=0x7fb5b0009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd1ee1bc987418e1 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:52:18.598 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:18.593+0000 7fb59d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-09-21T03:52:18.598 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:18.593+0000 7fb59d7fa640 1 -- 172.21.15.39:0/2962059297 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5b004d070 con 0x562989912490 2024-09-21T03:52:18.598 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:18.593+0000 7fb5c213d740 1 -- 172.21.15.39:0/2962059297 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb5a4005670 con 0x562989912490 2024-09-21T03:52:18.598 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:18.593+0000 7fb59d7fa640 1 -- 172.21.15.39:0/2962059297 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8484+0+0 (secure 0 0 0) 0x7fb5b0005030 con 0x562989912490 2024-09-21T03:52:18.598 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:18.593+0000 7fb59d7fa640 1 -- 172.21.15.39:0/2962059297 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fb5b0059050 con 0x562989912490 2024-09-21T03:52:18.598 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:18.593+0000 7fb5c213d740 10 client.17121 fetch_fsmap learned FSMap version 29 2024-09-21T03:52:18.598 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:18.593+0000 7fb5c213d740 1 -- 172.21.15.39:0/2962059297 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x562989cfce30 con 0x562989912490 2024-09-21T03:52:18.598 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:18.593+0000 7fb59d7fa640 1 -- 172.21.15.39:0/2962059297 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb5a4005670 con 0x562989912490 2024-09-21T03:52:18.599 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:18.594+0000 7fb59d7fa640 1 -- 172.21.15.39:0/2962059297 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fb5b00035f0 con 0x562989912490 2024-09-21T03:52:18.599 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:18.594+0000 7fb5c213d740 10 client.17121 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T03:52:18.599 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:18.594+0000 7fb5c213d740 10 client.17121 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T03:52:18.599 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:18.594+0000 7fb5c213d740 20 client.17121 populate_metadata read hostname 'smithi039' 2024-09-21T03:52:18.599 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:18.594+0000 7fb5c213d740 1 --2- 172.21.15.39:0/2962059297 >> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] conn(0x5629898f9cd0 0x562989d00320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:52:18.599 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:18.594+0000 7fb5c213d740 4 client.17121 mds_command: new command op to 24695 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:52:18.599 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:18.594+0000 7fb5c213d740 1 -- 172.21.15.39:0/2962059297 --> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562989c5adb0 con 0x5629898f9cd0 2024-09-21T03:52:18.599 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:18.594+0000 7fb5b9164640 1 --2- 172.21.15.39:0/2962059297 >> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] conn(0x5629898f9cd0 0x562989d00320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:52:18.601 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:18.595+0000 7fb5b9164640 1 --2- 172.21.15.39:0/2962059297 >> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] conn(0x5629898f9cd0 0x562989d00320 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1966109e12020360 server_cookie=a5e8583b5a52696c in_seq=0 out_seq=0 2024-09-21T03:52:18.601 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:18.595+0000 7fb59d7fa640 10 client.17121 ms_handle_connect on v2:172.21.15.150:6838/2949386907 2024-09-21T03:52:18.601 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:18.595+0000 7fb59d7fa640 1 -- 172.21.15.39:0/2962059297 <== mds.0 v2:172.21.15.150:6838/2949386907 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x562989c5adb0 con 0x5629898f9cd0 2024-09-21T03:52:18.601 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:18.595+0000 7fb59d7fa640 10 client.17121 handle_command_reply: tid=0 2024-09-21T03:52:18.601 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:18.596+0000 7fb5c213d740 1 client.17121 shutdown 2024-09-21T03:52:18.601 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:18.596+0000 7fb5c213d740 1 -- 172.21.15.39:0/2962059297 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562989912490 msgr2=0x562989cee090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:52:18.601 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:18.596+0000 7fb5c213d740 1 --2- 172.21.15.39:0/2962059297 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562989912490 0x562989cee090 secure :-1 s=READY pgs=948 cs=0 l=1 rev1=1 crypto rx=0x7fb5b0033000 tx=0x7fb5b0009260 comp rx=0 tx=0).stop 2024-09-21T03:52:18.601 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:18.596+0000 7fb5c213d740 1 -- 172.21.15.39:0/2962059297 shutdown_connections 2024-09-21T03:52:18.601 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:18.596+0000 7fb5c213d740 1 --2- 172.21.15.39:0/2962059297 >> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] conn(0x5629898f9cd0 0x562989d00320 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:18.601 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:18.596+0000 7fb5c213d740 1 --2- 172.21.15.39:0/2962059297 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56298993b350 0x562989cf9ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:18.601 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:18.596+0000 7fb5c213d740 1 --2- 172.21.15.39:0/2962059297 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56298991a670 0x562989cee610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:18.601 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:18.596+0000 7fb5c213d740 1 --2- 172.21.15.39:0/2962059297 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562989912490 0x562989cee090 unknown :-1 s=CLOSED pgs=948 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:18.601 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:18.596+0000 7fb59d7fa640 0 client.17121 ms_handle_reset on v2:172.21.15.150:6838/2949386907 2024-09-21T03:52:18.602 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:18.596+0000 7fb5c213d740 1 -- 172.21.15.39:0/2962059297 >> 172.21.15.39:0/2962059297 conn(0x56298992f740 msgr2=0x562989ce5610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:52:18.602 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:18.596+0000 7fb5c213d740 1 -- 172.21.15.39:0/2962059297 shutdown_connections 2024-09-21T03:52:18.602 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:18.596+0000 7fb5c213d740 1 -- 172.21.15.39:0/2962059297 wait complete. 2024-09-21T03:52:18.602 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:18.596+0000 7fb5c213d740 20 client.17121 trim_cache size 0 max 16384 2024-09-21T03:52:18.619 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:18.616+0000 7fb5c213d740 1 Processor -- start 2024-09-21T03:52:18.619 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:18.616+0000 7fb5c213d740 1 -- start start 2024-09-21T03:52:18.620 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:18.617+0000 7fb5c213d740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562989912490 0x562989cefac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:52:18.620 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:18.617+0000 7fb5c213d740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56298991a670 0x562989ce76b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:52:18.620 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:18.617+0000 7fb5c213d740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56298993b350 0x562989ce7bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:52:18.620 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:18.617+0000 7fb5c213d740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fb5b41b1880 con 0x56298991a670 2024-09-21T03:52:18.620 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:18.617+0000 7fb5c213d740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fb5b41b33c0 con 0x562989912490 2024-09-21T03:52:18.620 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:18.617+0000 7fb5c213d740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fb5b41b1f50 con 0x56298993b350 2024-09-21T03:52:18.620 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:18.617+0000 7fb5b9164640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56298991a670 0x562989ce76b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:52:18.620 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:18.617+0000 7fb5b9164640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56298991a670 0x562989ce76b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:39860/0 (socket says 172.21.15.39:39860) 2024-09-21T03:52:18.620 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:18.617+0000 7fb5b9164640 1 -- 172.21.15.39:0/2488547162 learned_addr learned my addr 172.21.15.39:0/2488547162 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:52:18.620 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:18.617+0000 7fb5ba166640 1 --2- 172.21.15.39:0/2488547162 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56298993b350 0x562989ce7bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:52:18.620 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:18.617+0000 7fb5b9965640 1 --2- 172.21.15.39:0/2488547162 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562989912490 0x562989cefac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:52:18.620 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:18.617+0000 7fb5b9164640 1 -- 172.21.15.39:0/2488547162 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56298993b350 msgr2=0x562989ce7bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:52:18.620 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:18.617+0000 7fb5b9164640 1 --2- 172.21.15.39:0/2488547162 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56298993b350 0x562989ce7bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:18.620 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:18.617+0000 7fb5b9164640 1 -- 172.21.15.39:0/2488547162 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562989912490 msgr2=0x562989cefac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:52:18.621 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:18.617+0000 7fb5b9164640 1 --2- 172.21.15.39:0/2488547162 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562989912490 0x562989cefac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:18.621 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:18.617+0000 7fb5b9164640 1 -- 172.21.15.39:0/2488547162 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562989ce0860 con 0x56298991a670 2024-09-21T03:52:18.621 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:18.617+0000 7fb5b9965640 1 --2- 172.21.15.39:0/2488547162 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562989912490 0x562989cefac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:52:18.621 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:18.617+0000 7fb5b9164640 1 --2- 172.21.15.39:0/2488547162 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56298991a670 0x562989ce76b0 secure :-1 s=READY pgs=949 cs=0 l=1 rev1=1 crypto rx=0x7fb5a400bb20 tx=0x7fb5a400b390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c12aa4a2a896151a server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:52:18.621 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:18.618+0000 7fb59f7fe640 1 -- 172.21.15.39:0/2488547162 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5a4004850 con 0x56298991a670 2024-09-21T03:52:18.621 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:18.618+0000 7fb59f7fe640 1 -- 172.21.15.39:0/2488547162 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb5a40049f0 con 0x56298991a670 2024-09-21T03:52:18.621 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:18.618+0000 7fb59f7fe640 1 -- 172.21.15.39:0/2488547162 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5a40106c0 con 0x56298991a670 2024-09-21T03:52:18.624 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:18.618+0000 7fb5c213d740 1 -- 172.21.15.39:0/2488547162 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56298991a670 msgr2=0x562989ce76b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:52:18.624 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:18.618+0000 7fb5c213d740 1 --2- 172.21.15.39:0/2488547162 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56298991a670 0x562989ce76b0 secure :-1 s=READY pgs=949 cs=0 l=1 rev1=1 crypto rx=0x7fb5a400bb20 tx=0x7fb5a400b390 comp rx=0 tx=0).stop 2024-09-21T03:52:18.624 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:18.619+0000 7fb5c213d740 1 -- 172.21.15.39:0/2488547162 shutdown_connections 2024-09-21T03:52:18.624 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:18.619+0000 7fb5c213d740 1 --2- 172.21.15.39:0/2488547162 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56298993b350 0x562989ce7bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:18.624 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:18.619+0000 7fb5c213d740 1 --2- 172.21.15.39:0/2488547162 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56298991a670 0x562989ce76b0 unknown :-1 s=CLOSED pgs=949 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:18.624 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:18.619+0000 7fb5c213d740 1 --2- 172.21.15.39:0/2488547162 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562989912490 0x562989cefac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:18.624 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:18.619+0000 7fb5c213d740 1 -- 172.21.15.39:0/2488547162 >> 172.21.15.39:0/2488547162 conn(0x56298992f740 msgr2=0x562989cef3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:52:18.624 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:18.619+0000 7fb5c213d740 1 -- 172.21.15.39:0/2488547162 shutdown_connections 2024-09-21T03:52:18.624 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:18.619+0000 7fb5c213d740 1 -- 172.21.15.39:0/2488547162 wait complete. 2024-09-21T03:52:18.624 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:18.619+0000 7fb5c213d740 1 -- start start 2024-09-21T03:52:18.624 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:18.619+0000 7fb5c213d740 1 Processor -- start 2024-09-21T03:52:18.624 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:18.620+0000 7fb5c213d740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562989912490 0x562989cf1ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:52:18.624 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:18.620+0000 7fb5c213d740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56298991a670 0x562989cf2530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:52:18.625 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:18.620+0000 7fb5c213d740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56298993b350 0x562989cf2a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:52:18.625 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:18.620+0000 7fb5c213d740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fb5b41b2cf0 con 0x56298991a670 2024-09-21T03:52:18.625 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:18.620+0000 7fb5c213d740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fb5b41b2620 con 0x562989912490 2024-09-21T03:52:18.625 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:18.620+0000 7fb5c213d740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fb5b41af860 con 0x56298993b350 2024-09-21T03:52:18.625 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:18.620+0000 7fb5c213d740 10 client.0 _ll_register_callbacks cb 0x7fb5b0005db0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:52:18.625 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:18.620+0000 7fb5b9164640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56298991a670 0x562989cf2530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:52:18.625 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:18.620+0000 7fb5b9164640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56298991a670 0x562989cf2530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:39864/0 (socket says 172.21.15.39:39864) 2024-09-21T03:52:18.625 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:18.620+0000 7fb5b9164640 1 -- 172.21.15.39:0/2401153573 learned_addr learned my addr 172.21.15.39:0/2401153573 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:52:18.625 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:18.620+0000 7fb5b9965640 1 --2- 172.21.15.39:0/2401153573 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562989912490 0x562989cf1ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:52:18.625 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:18.620+0000 7fb5ba166640 1 --2- 172.21.15.39:0/2401153573 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56298993b350 0x562989cf2a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:52:18.625 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:18.620+0000 7fb5b9164640 1 -- 172.21.15.39:0/2401153573 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56298993b350 msgr2=0x562989cf2a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:52:18.625 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:18.620+0000 7fb5b9164640 1 --2- 172.21.15.39:0/2401153573 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56298993b350 0x562989cf2a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:18.625 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:18.620+0000 7fb5b9164640 1 -- 172.21.15.39:0/2401153573 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562989912490 msgr2=0x562989cf1ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:52:18.625 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:18.620+0000 7fb5b9164640 1 --2- 172.21.15.39:0/2401153573 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562989912490 0x562989cf1ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:18.625 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:18.620+0000 7fb5b9164640 1 -- 172.21.15.39:0/2401153573 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562989d69f10 con 0x56298991a670 2024-09-21T03:52:18.626 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:18.620+0000 7fb5b9164640 1 --2- 172.21.15.39:0/2401153573 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56298991a670 0x562989cf2530 secure :-1 s=READY pgs=950 cs=0 l=1 rev1=1 crypto rx=0x7fb5a400b5f0 tx=0x7fb5a40040d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a110dae02efc3c0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:52:18.626 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:18.620+0000 7fb59d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-09-21T03:52:18.626 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:18.620+0000 7fb59d7fa640 1 -- 172.21.15.39:0/2401153573 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5a4040060 con 0x56298991a670 2024-09-21T03:52:18.626 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:18.621+0000 7fb59d7fa640 1 -- 172.21.15.39:0/2401153573 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8484+0+0 (secure 0 0 0) 0x7fb5a4004630 con 0x56298991a670 2024-09-21T03:52:18.626 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:18.621+0000 7fb5c213d740 1 -- 172.21.15.39:0/2401153573 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb5a40049f0 con 0x56298991a670 2024-09-21T03:52:18.626 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:18.621+0000 7fb59d7fa640 1 -- 172.21.15.39:0/2401153573 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fb5a4005710 con 0x56298991a670 2024-09-21T03:52:18.626 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:18.621+0000 7fb5c213d740 10 client.17127 fetch_fsmap learned FSMap version 29 2024-09-21T03:52:18.626 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:18.621+0000 7fb5c213d740 1 -- 172.21.15.39:0/2401153573 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x562989d6b1d0 con 0x56298991a670 2024-09-21T03:52:18.626 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:18.621+0000 7fb59d7fa640 1 -- 172.21.15.39:0/2401153573 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb5a40049f0 con 0x56298991a670 2024-09-21T03:52:18.626 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:18.621+0000 7fb59d7fa640 1 -- 172.21.15.39:0/2401153573 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fb5a4010a00 con 0x56298991a670 2024-09-21T03:52:18.626 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:18.621+0000 7fb5c213d740 10 client.17127 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T03:52:18.626 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:18.621+0000 7fb5c213d740 10 client.17127 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T03:52:18.626 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:18.621+0000 7fb5c213d740 20 client.17127 populate_metadata read hostname 'smithi039' 2024-09-21T03:52:18.626 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:18.621+0000 7fb5c213d740 1 --2- 172.21.15.39:0/2401153573 >> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] conn(0x5629898cf080 0x562989d6d6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:52:18.627 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:18.621+0000 7fb5c213d740 4 client.17127 mds_command: new command op to 24695 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-21T03:52:18.627 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:18.621+0000 7fb5c213d740 1 -- 172.21.15.39:0/2401153573 --> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x562989c56860 con 0x5629898cf080 2024-09-21T03:52:18.627 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:18.622+0000 7fb5b9965640 1 --2- 172.21.15.39:0/2401153573 >> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] conn(0x5629898cf080 0x562989d6d6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:52:18.627 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:18.622+0000 7fb59d7fa640 10 client.17127 ms_handle_connect on v2:172.21.15.150:6838/2949386907 2024-09-21T03:52:18.627 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:18.622+0000 7fb5b9965640 1 --2- 172.21.15.39:0/2401153573 >> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] conn(0x5629898cf080 0x562989d6d6b0 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2dd1e3992623c786 server_cookie=afa275c9673eea18 in_seq=0 out_seq=0 2024-09-21T03:52:18.628 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:18.624+0000 7fb59d7fa640 1 -- 172.21.15.39:0/2401153573 <== mds.0 v2:172.21.15.150:6838/2949386907 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x562989c56860 con 0x5629898cf080 2024-09-21T03:52:18.628 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:18.624+0000 7fb59d7fa640 10 client.17127 handle_command_reply: tid=0 2024-09-21T03:52:18.628 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:18.624+0000 7fb5c213d740 1 client.17127 shutdown 2024-09-21T03:52:18.628 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:18.624+0000 7fb5c213d740 1 -- 172.21.15.39:0/2401153573 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56298991a670 msgr2=0x562989cf2530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:52:18.628 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:18.624+0000 7fb5c213d740 1 --2- 172.21.15.39:0/2401153573 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56298991a670 0x562989cf2530 secure :-1 s=READY pgs=950 cs=0 l=1 rev1=1 crypto rx=0x7fb5a400b5f0 tx=0x7fb5a40040d0 comp rx=0 tx=0).stop 2024-09-21T03:52:18.628 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:18.624+0000 7fb5c213d740 1 -- 172.21.15.39:0/2401153573 shutdown_connections 2024-09-21T03:52:18.628 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:18.624+0000 7fb5c213d740 1 --2- 172.21.15.39:0/2401153573 >> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] conn(0x5629898cf080 0x562989d6d6b0 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:18.628 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:18.624+0000 7fb5c213d740 1 --2- 172.21.15.39:0/2401153573 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56298993b350 0x562989cf2a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:18.628 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:18.624+0000 7fb5c213d740 1 --2- 172.21.15.39:0/2401153573 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56298991a670 0x562989cf2530 unknown :-1 s=CLOSED pgs=950 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:18.629 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:18.624+0000 7fb5c213d740 1 --2- 172.21.15.39:0/2401153573 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562989912490 0x562989cf1ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:18.629 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:18.624+0000 7fb5c213d740 1 -- 172.21.15.39:0/2401153573 >> 172.21.15.39:0/2401153573 conn(0x56298992f740 msgr2=0x562989ce3460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:52:18.629 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:18.624+0000 7fb59d7fa640 0 client.17127 ms_handle_reset on v2:172.21.15.150:6838/2949386907 2024-09-21T03:52:18.629 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:18.624+0000 7fb5c213d740 1 -- 172.21.15.39:0/2401153573 shutdown_connections 2024-09-21T03:52:18.629 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:18.624+0000 7fb5c213d740 1 -- 172.21.15.39:0/2401153573 wait complete. 2024-09-21T03:52:18.629 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:18.624+0000 7fb5c213d740 20 client.17127 trim_cache size 0 max 16384 2024-09-21T03:52:18.629 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-09-21T03:52:18.629 INFO:teuthology.orchestra.run.smithi039.stdout: "return_code": 0, 2024-09-21T03:52:18.629 INFO:teuthology.orchestra.run.smithi039.stdout: "scrub_tag": "4bda6ee3-7237-4ee4-96ce-150ad50f26dc", 2024-09-21T03:52:18.629 INFO:teuthology.orchestra.run.smithi039.stdout: "mode": "asynchronous" 2024-09-21T03:52:18.629 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-09-21T03:52:18.630 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:18.625+0000 7fb5bbbc9640 1 -- 172.21.15.39:0/1953442232 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7fb5a00b6680 msgr2=0x7fb5a00b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:52:18.630 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:18.625+0000 7fb5bbbc9640 1 --2- 172.21.15.39:0/1953442232 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7fb5a00b6680 0x7fb5a00b8b90 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fb5b0031440 tx=0x7fb5b0009610 comp rx=0 tx=0).stop 2024-09-21T03:52:18.630 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:18.625+0000 7fb5bbbc9640 1 -- 172.21.15.39:0/1953442232 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb5b4109e10 msgr2=0x7fb5b41a1f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:52:18.630 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:18.625+0000 7fb5bbbc9640 1 --2- 172.21.15.39:0/1953442232 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb5b4109e10 0x7fb5b41a1f00 secure :-1 s=READY pgs=946 cs=0 l=1 rev1=1 crypto rx=0x7fb5ac00eaa0 tx=0x7fb5ac00ef60 comp rx=0 tx=0).stop 2024-09-21T03:52:18.630 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:18.625+0000 7fb5bbbc9640 1 -- 172.21.15.39:0/1953442232 shutdown_connections 2024-09-21T03:52:18.630 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:18.625+0000 7fb5bbbc9640 1 --2- 172.21.15.39:0/1953442232 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7fb5a00b6680 0x7fb5a00b8b90 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:18.630 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:18.625+0000 7fb5bbbc9640 1 --2- 172.21.15.39:0/1953442232 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb5b4109e10 0x7fb5b41a1f00 unknown :-1 s=CLOSED pgs=946 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:18.631 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:18.625+0000 7fb5bbbc9640 1 --2- 172.21.15.39:0/1953442232 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb5b41094c0 0x7fb5b419cc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:18.631 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:18.625+0000 7fb5bbbc9640 1 --2- 172.21.15.39:0/1953442232 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb5b4103b80 0x7fb5b419c6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:18.631 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:18.625+0000 7fb5bbbc9640 1 -- 172.21.15.39:0/1953442232 >> 172.21.15.39:0/1953442232 conn(0x7fb5b40ff5d0 msgr2=0x7fb5b4108b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:52:18.631 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:18.625+0000 7fb5bbbc9640 1 -- 172.21.15.39:0/1953442232 shutdown_connections 2024-09-21T03:52:18.631 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:18.625+0000 7fb5bbbc9640 1 -- 172.21.15.39:0/1953442232 wait complete. 2024-09-21T03:52:19.503 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:52:19 smithi039 ceph-mon[31211]: pgmap v513: 97 pgs: 97 active+clean; 10 MiB data, 520 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 170 B/s wr, 9 op/s 2024-09-21T03:52:19.504 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:52:19 smithi039 ceph-mon[31211]: scrub queued for path: / 2024-09-21T03:52:19.504 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:52:19 smithi039 ceph-mon[31211]: scrub summary: idle+waiting paths [/] 2024-09-21T03:52:19.504 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:52:19 smithi039 ceph-mon[31211]: scrub summary: active paths [/] 2024-09-21T03:52:19.504 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:52:19 smithi039 ceph-mon[31211]: scrub summary: idle+waiting paths [/] 2024-09-21T03:52:19.505 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 51116d5c-77ca-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub status 2024-09-21T03:52:19.718 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:52:19 smithi161 ceph-mon[34533]: pgmap v513: 97 pgs: 97 active+clean; 10 MiB data, 520 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 170 B/s wr, 9 op/s 2024-09-21T03:52:19.718 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:52:19 smithi161 ceph-mon[34533]: scrub queued for path: / 2024-09-21T03:52:19.718 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:52:19 smithi161 ceph-mon[34533]: scrub summary: idle+waiting paths [/] 2024-09-21T03:52:19.718 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:52:19 smithi161 ceph-mon[34533]: scrub summary: active paths [/] 2024-09-21T03:52:19.718 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:52:19 smithi161 ceph-mon[34533]: scrub summary: idle+waiting paths [/] 2024-09-21T03:52:19.725 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:52:19 smithi150 ceph-mon[34289]: pgmap v513: 97 pgs: 97 active+clean; 10 MiB data, 520 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 170 B/s wr, 9 op/s 2024-09-21T03:52:19.725 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:52:19 smithi150 ceph-mon[34289]: scrub queued for path: / 2024-09-21T03:52:19.725 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:52:19 smithi150 ceph-mon[34289]: scrub summary: idle+waiting paths [/] 2024-09-21T03:52:19.725 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:52:19 smithi150 ceph-mon[34289]: scrub summary: active paths [/] 2024-09-21T03:52:19.725 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:52:19 smithi150 ceph-mon[34289]: scrub summary: idle+waiting paths [/] 2024-09-21T03:52:19.768 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/51116d5c-77ca-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:52:20.272 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:20.268+0000 7f9f5fe41640 1 -- 172.21.15.39:0/4235214089 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9f5810de40 msgr2=0x7f9f58110330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:52:20.272 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:20.268+0000 7f9f5fe41640 1 --2- 172.21.15.39:0/4235214089 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9f5810de40 0x7f9f58110330 secure :-1 s=READY pgs=951 cs=0 l=1 rev1=1 crypto rx=0x7f9f48009ca0 tx=0x7f9f4802a8c0 comp rx=0 tx=0).stop 2024-09-21T03:52:20.272 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:20.268+0000 7f9f5fe41640 1 -- 172.21.15.39:0/4235214089 shutdown_connections 2024-09-21T03:52:20.272 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:20.268+0000 7f9f5fe41640 1 --2- 172.21.15.39:0/4235214089 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9f5810de40 0x7f9f58110330 unknown :-1 s=CLOSED pgs=951 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:20.272 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:20.268+0000 7f9f5fe41640 1 --2- 172.21.15.39:0/4235214089 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9f58105330 0x7f9f5810d880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:20.273 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:20.268+0000 7f9f5fe41640 1 --2- 172.21.15.39:0/4235214089 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9f581049e0 0x7f9f58104df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:20.273 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:20.268+0000 7f9f5fe41640 1 -- 172.21.15.39:0/4235214089 >> 172.21.15.39:0/4235214089 conn(0x7f9f58079730 msgr2=0x7f9f58079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:52:20.273 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:20.268+0000 7f9f5fe41640 1 -- 172.21.15.39:0/4235214089 shutdown_connections 2024-09-21T03:52:20.273 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:20.268+0000 7f9f5fe41640 1 -- 172.21.15.39:0/4235214089 wait complete. 2024-09-21T03:52:20.273 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:20.269+0000 7f9f5fe41640 1 Processor -- start 2024-09-21T03:52:20.273 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:20.269+0000 7f9f5fe41640 1 -- start start 2024-09-21T03:52:20.273 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:20.269+0000 7f9f5fe41640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9f581049e0 0x7f9f58102720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:52:20.273 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:20.269+0000 7f9f5fe41640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9f58105330 0x7f9f58102c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:52:20.273 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:20.269+0000 7f9f5fe41640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9f5810de40 0x7f9f581031a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:52:20.273 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:20.269+0000 7f9f5fe41640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f9f58113b10 con 0x7f9f5810de40 2024-09-21T03:52:20.273 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:20.269+0000 7f9f5fe41640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f9f58113990 con 0x7f9f581049e0 2024-09-21T03:52:20.273 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:20.270+0000 7f9f5fe41640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f9f58113c90 con 0x7f9f58105330 2024-09-21T03:52:20.273 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:20.270+0000 7f9f5e3de640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9f5810de40 0x7f9f581031a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:52:20.273 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:20.270+0000 7f9f5e3de640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9f5810de40 0x7f9f581031a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:39870/0 (socket says 172.21.15.39:39870) 2024-09-21T03:52:20.273 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:20.270+0000 7f9f5e3de640 1 -- 172.21.15.39:0/2660617463 learned_addr learned my addr 172.21.15.39:0/2660617463 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:52:20.274 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:20.270+0000 7f9f5d3dc640 1 --2- 172.21.15.39:0/2660617463 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9f58105330 0x7f9f58102c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:52:20.274 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:20.270+0000 7f9f5dbdd640 1 --2- 172.21.15.39:0/2660617463 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9f581049e0 0x7f9f58102720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:52:20.274 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:20.270+0000 7f9f5e3de640 1 -- 172.21.15.39:0/2660617463 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9f58105330 msgr2=0x7f9f58102c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:52:20.274 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:20.270+0000 7f9f5e3de640 1 --2- 172.21.15.39:0/2660617463 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9f58105330 0x7f9f58102c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:20.274 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:20.270+0000 7f9f5e3de640 1 -- 172.21.15.39:0/2660617463 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9f581049e0 msgr2=0x7f9f58102720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:52:20.274 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:20.270+0000 7f9f5e3de640 1 --2- 172.21.15.39:0/2660617463 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9f581049e0 0x7f9f58102720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:20.274 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:20.270+0000 7f9f5e3de640 1 -- 172.21.15.39:0/2660617463 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9f581a87a0 con 0x7f9f5810de40 2024-09-21T03:52:20.274 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:20.270+0000 7f9f5dbdd640 1 --2- 172.21.15.39:0/2660617463 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9f581049e0 0x7f9f58102720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:52:20.274 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:20.270+0000 7f9f5e3de640 1 --2- 172.21.15.39:0/2660617463 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9f5810de40 0x7f9f581031a0 secure :-1 s=READY pgs=952 cs=0 l=1 rev1=1 crypto rx=0x7f9f48009430 tx=0x7f9f480319f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:52:20.274 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:20.270+0000 7f9f56ffd640 1 -- 172.21.15.39:0/2660617463 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f48003c30 con 0x7f9f5810de40 2024-09-21T03:52:20.274 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:20.271+0000 7f9f5fe41640 1 -- 172.21.15.39:0/2660617463 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9f581a8a30 con 0x7f9f5810de40 2024-09-21T03:52:20.275 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:20.271+0000 7f9f56ffd640 1 -- 172.21.15.39:0/2660617463 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9f48003dd0 con 0x7f9f5810de40 2024-09-21T03:52:20.275 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:20.271+0000 7f9f56ffd640 1 -- 172.21.15.39:0/2660617463 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f4802faa0 con 0x7f9f5810de40 2024-09-21T03:52:20.275 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:20.271+0000 7f9f5fe41640 1 -- 172.21.15.39:0/2660617463 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9f581a8f70 con 0x7f9f5810de40 2024-09-21T03:52:20.275 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:20.273+0000 7f9f663b5740 1 Processor -- start 2024-09-21T03:52:20.275 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:20.274+0000 7f9f56ffd640 1 -- 172.21.15.39:0/2660617463 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 27) ==== 154643+0+0 (secure 0 0 0) 0x7f9f4802fc40 con 0x7f9f5810de40 2024-09-21T03:52:20.276 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:20.274+0000 7f9f663b5740 1 -- start start 2024-09-21T03:52:20.277 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:20.274+0000 7f9f663b5740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d5d4f4a070 0x55d5d52f1650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:52:20.277 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:20.274+0000 7f9f663b5740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d5d4f211b0 0x55d5d52f1b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:52:20.277 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:20.274+0000 7f9f663b5740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d5d4f29390 0x55d5d52f20d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:52:20.277 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:20.274+0000 7f9f663b5740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x55d5d5267e50 con 0x55d5d4f211b0 2024-09-21T03:52:20.277 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:20.274+0000 7f9f663b5740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x55d5d5257280 con 0x55d5d4f4a070 2024-09-21T03:52:20.277 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:20.274+0000 7f9f663b5740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55d5d525b090 con 0x55d5d4f29390 2024-09-21T03:52:20.277 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:20.274+0000 7f9f5dbdd640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d5d4f211b0 0x55d5d52f1b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:52:20.278 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:20.274+0000 7f9f5dbdd640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d5d4f211b0 0x55d5d52f1b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:39878/0 (socket says 172.21.15.39:39878) 2024-09-21T03:52:20.278 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:20.274+0000 7f9f5dbdd640 1 -- 172.21.15.39:0/1850674489 learned_addr learned my addr 172.21.15.39:0/1850674489 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:52:20.278 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:20.274+0000 7f9f5d3dc640 1 --2- 172.21.15.39:0/1850674489 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d5d4f4a070 0x55d5d52f1650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:52:20.278 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:20.275+0000 7f9f56ffd640 1 --2- 172.21.15.39:0/2660617463 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f9f400b6680 0x7f9f400b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:52:20.278 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:20.275+0000 7f9f5d3dc640 1 --2- 172.21.15.39:0/1850674489 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d5d4f29390 0x55d5d52f20d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:52:20.278 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:20.275+0000 7f9f5dbdd640 1 -- 172.21.15.39:0/1850674489 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d5d4f29390 msgr2=0x55d5d52f20d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:52:20.278 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:20.275+0000 7f9f5dbdd640 1 --2- 172.21.15.39:0/1850674489 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d5d4f29390 0x55d5d52f20d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:20.278 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:20.275+0000 7f9f5dbdd640 1 -- 172.21.15.39:0/1850674489 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d5d4f4a070 msgr2=0x55d5d52f1650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:52:20.278 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:20.275+0000 7f9f5dbdd640 1 --2- 172.21.15.39:0/1850674489 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d5d4f4a070 0x55d5d52f1650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:20.278 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:20.275+0000 7f9f5dbdd640 1 -- 172.21.15.39:0/1850674489 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d5d4fb5650 con 0x55d5d4f211b0 2024-09-21T03:52:20.278 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:20.275+0000 7f9f5e3de640 1 --2- 172.21.15.39:0/2660617463 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f9f400b6680 0x7f9f400b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:52:20.278 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:20.275+0000 7f9f56ffd640 1 -- 172.21.15.39:0/2660617463 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8484+0+0 (secure 0 0 0) 0x7f9f48108a60 con 0x7f9f5810de40 2024-09-21T03:52:20.278 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:20.275+0000 7f9f5dbdd640 1 --2- 172.21.15.39:0/1850674489 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d5d4f211b0 0x55d5d52f1b90 secure :-1 s=READY pgs=953 cs=0 l=1 rev1=1 crypto rx=0x7f9f5000bd30 tx=0x7f9f5000f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e8cfd20ba4925ac4 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:52:20.278 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:20.275+0000 7f9f3ffff640 1 -- 172.21.15.39:0/1850674489 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f50015070 con 0x55d5d4f211b0 2024-09-21T03:52:20.278 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:20.275+0000 7f9f3ffff640 1 -- 172.21.15.39:0/1850674489 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9f5000be20 con 0x55d5d4f211b0 2024-09-21T03:52:20.278 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:20.275+0000 7f9f3ffff640 1 -- 172.21.15.39:0/1850674489 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f500116a0 con 0x55d5d4f211b0 2024-09-21T03:52:20.281 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:20.276+0000 7f9f663b5740 1 -- 172.21.15.39:0/1850674489 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d5d4f211b0 msgr2=0x55d5d52f1b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:52:20.281 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:20.276+0000 7f9f663b5740 1 --2- 172.21.15.39:0/1850674489 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d5d4f211b0 0x55d5d52f1b90 secure :-1 s=READY pgs=953 cs=0 l=1 rev1=1 crypto rx=0x7f9f5000bd30 tx=0x7f9f5000f730 comp rx=0 tx=0).stop 2024-09-21T03:52:20.281 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:20.276+0000 7f9f5e3de640 1 --2- 172.21.15.39:0/2660617463 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f9f400b6680 0x7f9f400b8b90 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f9f481067d0 tx=0x7f9f4809b690 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:52:20.281 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:20.276+0000 7f9f663b5740 1 -- 172.21.15.39:0/1850674489 shutdown_connections 2024-09-21T03:52:20.282 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:20.276+0000 7f9f663b5740 1 --2- 172.21.15.39:0/1850674489 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d5d4f29390 0x55d5d52f20d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:20.282 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:20.276+0000 7f9f663b5740 1 --2- 172.21.15.39:0/1850674489 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d5d4f211b0 0x55d5d52f1b90 unknown :-1 s=CLOSED pgs=953 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:20.282 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:20.276+0000 7f9f663b5740 1 --2- 172.21.15.39:0/1850674489 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d5d4f4a070 0x55d5d52f1650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:20.282 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:20.276+0000 7f9f663b5740 1 -- 172.21.15.39:0/1850674489 >> 172.21.15.39:0/1850674489 conn(0x55d5d4f3e460 msgr2=0x55d5d52ef540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:52:20.282 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:20.276+0000 7f9f663b5740 1 -- 172.21.15.39:0/1850674489 shutdown_connections 2024-09-21T03:52:20.282 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:20.276+0000 7f9f663b5740 1 -- 172.21.15.39:0/1850674489 wait complete. 2024-09-21T03:52:20.282 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:20.277+0000 7f9f663b5740 1 -- start start 2024-09-21T03:52:20.282 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:20.277+0000 7f9f663b5740 1 Processor -- start 2024-09-21T03:52:20.282 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:20.277+0000 7f9f663b5740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d5d4f211b0 0x55d5d52fcd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:52:20.282 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:20.277+0000 7f9f663b5740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d5d4f29390 0x55d5d52fd2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:52:20.282 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:20.277+0000 7f9f663b5740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d5d4f4a070 0x55d5d5309150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:52:20.282 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:20.277+0000 7f9f663b5740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x55d5d50b8700 con 0x55d5d4f211b0 2024-09-21T03:52:20.282 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:20.277+0000 7f9f663b5740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x55d5d51f2110 con 0x55d5d4f4a070 2024-09-21T03:52:20.282 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:20.277+0000 7f9f663b5740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55d5d52ffe30 con 0x55d5d4f29390 2024-09-21T03:52:20.283 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:20.278+0000 7f9f663b5740 10 client.0 _ll_register_callbacks cb 0x55d5d520b990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:52:20.283 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:20.278+0000 7f9f5d3dc640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d5d4f211b0 0x55d5d52fcd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:52:20.283 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:20.278+0000 7f9f5d3dc640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d5d4f211b0 0x55d5d52fcd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:39884/0 (socket says 172.21.15.39:39884) 2024-09-21T03:52:20.283 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:20.278+0000 7f9f5d3dc640 1 -- 172.21.15.39:0/2635638756 learned_addr learned my addr 172.21.15.39:0/2635638756 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:52:20.283 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:20.278+0000 7f9f5dbdd640 1 --2- 172.21.15.39:0/2635638756 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d5d4f29390 0x55d5d52fd2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:52:20.283 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:20.278+0000 7f9f5d3dc640 1 --2- 172.21.15.39:0/2635638756 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d5d4f4a070 0x55d5d5309150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:52:20.283 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:20.278+0000 7f9f5d3dc640 1 -- 172.21.15.39:0/2635638756 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d5d4f29390 msgr2=0x55d5d52fd2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:52:20.283 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:20.278+0000 7f9f5d3dc640 1 --2- 172.21.15.39:0/2635638756 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d5d4f29390 0x55d5d52fd2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:20.283 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:20.278+0000 7f9f5d3dc640 1 -- 172.21.15.39:0/2635638756 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d5d4f4a070 msgr2=0x55d5d5309150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:52:20.283 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:20.278+0000 7f9f5d3dc640 1 --2- 172.21.15.39:0/2635638756 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d5d4f4a070 0x55d5d5309150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:20.283 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:20.278+0000 7f9f5d3dc640 1 -- 172.21.15.39:0/2635638756 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d5d521fc40 con 0x55d5d4f211b0 2024-09-21T03:52:20.283 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:20.278+0000 7f9f5d3dc640 1 --2- 172.21.15.39:0/2635638756 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d5d4f211b0 0x55d5d52fcd60 secure :-1 s=READY pgs=954 cs=0 l=1 rev1=1 crypto rx=0x7f9f4c004640 tx=0x7f9f4c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dcc8c92790fd1086 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:52:20.283 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:20.278+0000 7f9f3d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-09-21T03:52:20.283 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:20.278+0000 7f9f3d7fa640 1 -- 172.21.15.39:0/2635638756 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f4c0126c0 con 0x55d5d4f211b0 2024-09-21T03:52:20.283 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:20.278+0000 7f9f3d7fa640 1 -- 172.21.15.39:0/2635638756 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8484+0+0 (secure 0 0 0) 0x7f9f4c012860 con 0x55d5d4f211b0 2024-09-21T03:52:20.283 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:20.278+0000 7f9f663b5740 1 -- 172.21.15.39:0/2635638756 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9f5000be20 con 0x55d5d4f211b0 2024-09-21T03:52:20.284 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:20.279+0000 7f9f3d7fa640 1 -- 172.21.15.39:0/2635638756 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f9f4c012a60 con 0x55d5d4f211b0 2024-09-21T03:52:20.284 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:20.279+0000 7f9f663b5740 10 client.17139 fetch_fsmap learned FSMap version 29 2024-09-21T03:52:20.284 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:20.279+0000 7f9f663b5740 1 -- 172.21.15.39:0/2635638756 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55d5d530bbd0 con 0x55d5d4f211b0 2024-09-21T03:52:20.284 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:20.279+0000 7f9f3d7fa640 1 -- 172.21.15.39:0/2635638756 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9f5000be20 con 0x55d5d4f211b0 2024-09-21T03:52:20.284 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:20.279+0000 7f9f3d7fa640 1 -- 172.21.15.39:0/2635638756 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f9f4c029400 con 0x55d5d4f211b0 2024-09-21T03:52:20.284 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:20.279+0000 7f9f663b5740 10 client.17139 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T03:52:20.284 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:20.279+0000 7f9f663b5740 10 client.17139 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T03:52:20.284 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:20.279+0000 7f9f663b5740 20 client.17139 populate_metadata read hostname 'smithi039' 2024-09-21T03:52:20.284 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:20.279+0000 7f9f663b5740 1 --2- 172.21.15.39:0/2635638756 >> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] conn(0x55d5d4f089f0 0x55d5d530f010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:52:20.284 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:20.279+0000 7f9f663b5740 4 client.17139 mds_command: new command op to 24695 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:52:20.284 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:20.279+0000 7f9f663b5740 1 -- 172.21.15.39:0/2635638756 --> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d5d5269da0 con 0x55d5d4f089f0 2024-09-21T03:52:20.284 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:20.280+0000 7f9f5dbdd640 1 --2- 172.21.15.39:0/2635638756 >> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] conn(0x55d5d4f089f0 0x55d5d530f010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:52:20.286 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:20.280+0000 7f9f3d7fa640 10 client.17139 ms_handle_connect on v2:172.21.15.150:6838/2949386907 2024-09-21T03:52:20.286 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:20.280+0000 7f9f5dbdd640 1 --2- 172.21.15.39:0/2635638756 >> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] conn(0x55d5d4f089f0 0x55d5d530f010 crc :-1 s=READY pgs=111 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9d63cfc7923223b server_cookie=52784cfa662bcb1a in_seq=0 out_seq=0 2024-09-21T03:52:20.286 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:20.281+0000 7f9f3d7fa640 1 -- 172.21.15.39:0/2635638756 <== mds.0 v2:172.21.15.150:6838/2949386907 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55d5d5269da0 con 0x55d5d4f089f0 2024-09-21T03:52:20.286 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:20.281+0000 7f9f3d7fa640 10 client.17139 handle_command_reply: tid=0 2024-09-21T03:52:20.286 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:20.281+0000 7f9f663b5740 1 client.17139 shutdown 2024-09-21T03:52:20.286 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:20.281+0000 7f9f663b5740 1 -- 172.21.15.39:0/2635638756 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d5d4f211b0 msgr2=0x55d5d52fcd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:52:20.286 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:20.281+0000 7f9f663b5740 1 --2- 172.21.15.39:0/2635638756 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d5d4f211b0 0x55d5d52fcd60 secure :-1 s=READY pgs=954 cs=0 l=1 rev1=1 crypto rx=0x7f9f4c004640 tx=0x7f9f4c004730 comp rx=0 tx=0).stop 2024-09-21T03:52:20.286 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:20.281+0000 7f9f663b5740 1 -- 172.21.15.39:0/2635638756 shutdown_connections 2024-09-21T03:52:20.286 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:20.281+0000 7f9f663b5740 1 --2- 172.21.15.39:0/2635638756 >> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] conn(0x55d5d4f089f0 0x55d5d530f010 crc :-1 s=READY pgs=111 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:20.286 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:20.281+0000 7f9f663b5740 1 --2- 172.21.15.39:0/2635638756 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d5d4f4a070 0x55d5d5309150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:20.287 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:20.281+0000 7f9f663b5740 1 --2- 172.21.15.39:0/2635638756 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d5d4f29390 0x55d5d52fd2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:20.287 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:20.281+0000 7f9f663b5740 1 --2- 172.21.15.39:0/2635638756 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d5d4f211b0 0x55d5d52fcd60 unknown :-1 s=CLOSED pgs=954 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:20.287 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:20.281+0000 7f9f3d7fa640 0 client.17139 ms_handle_reset on v2:172.21.15.150:6838/2949386907 2024-09-21T03:52:20.287 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:20.281+0000 7f9f663b5740 1 -- 172.21.15.39:0/2635638756 >> 172.21.15.39:0/2635638756 conn(0x55d5d4f3e460 msgr2=0x55d5d52f47e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:52:20.287 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:20.281+0000 7f9f663b5740 1 -- 172.21.15.39:0/2635638756 shutdown_connections 2024-09-21T03:52:20.287 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:20.281+0000 7f9f663b5740 1 -- 172.21.15.39:0/2635638756 wait complete. 2024-09-21T03:52:20.287 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:20.281+0000 7f9f663b5740 20 client.17139 trim_cache size 0 max 16384 2024-09-21T03:52:20.301 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:20.299+0000 7f9f663b5740 1 Processor -- start 2024-09-21T03:52:20.302 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:20.299+0000 7f9f663b5740 1 -- start start 2024-09-21T03:52:20.302 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:20.299+0000 7f9f663b5740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d5d4f211b0 0x55d5d52fd220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:52:20.302 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:20.299+0000 7f9f663b5740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d5d4f29390 0x55d5d52f4750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:52:20.302 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:20.299+0000 7f9f663b5740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d5d4f4a070 0x55d5d52f4c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:52:20.302 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:20.299+0000 7f9f663b5740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f9f581b19b0 con 0x55d5d4f4a070 2024-09-21T03:52:20.302 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:20.299+0000 7f9f663b5740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f9f581b34f0 con 0x55d5d4f211b0 2024-09-21T03:52:20.302 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:20.299+0000 7f9f663b5740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f9f581b2080 con 0x55d5d4f29390 2024-09-21T03:52:20.302 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:20.299+0000 7f9f5d3dc640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d5d4f4a070 0x55d5d52f4c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:52:20.302 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:20.299+0000 7f9f5d3dc640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d5d4f4a070 0x55d5d52f4c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:39892/0 (socket says 172.21.15.39:39892) 2024-09-21T03:52:20.302 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:20.299+0000 7f9f5d3dc640 1 -- 172.21.15.39:0/1849076508 learned_addr learned my addr 172.21.15.39:0/1849076508 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:52:20.302 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:20.299+0000 7f9f5dbdd640 1 --2- 172.21.15.39:0/1849076508 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d5d4f29390 0x55d5d52f4750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:52:20.302 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:20.299+0000 7f9f5d3dc640 1 -- 172.21.15.39:0/1849076508 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d5d4f29390 msgr2=0x55d5d52f4750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:52:20.303 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:20.299+0000 7f9f5d3dc640 1 --2- 172.21.15.39:0/1849076508 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d5d4f29390 0x55d5d52f4750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:20.303 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:20.299+0000 7f9f5d3dc640 1 -- 172.21.15.39:0/1849076508 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d5d4f211b0 msgr2=0x55d5d52fd220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:52:20.303 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:20.299+0000 7f9f5d3dc640 1 --2- 172.21.15.39:0/1849076508 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d5d4f211b0 0x55d5d52fd220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:20.303 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:20.299+0000 7f9f5d3dc640 1 -- 172.21.15.39:0/1849076508 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d5d52f85f0 con 0x55d5d4f4a070 2024-09-21T03:52:20.303 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:20.299+0000 7f9f5dbdd640 1 --2- 172.21.15.39:0/1849076508 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d5d4f29390 0x55d5d52f4750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:52:20.303 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:20.300+0000 7f9f5d3dc640 1 --2- 172.21.15.39:0/1849076508 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d5d4f4a070 0x55d5d52f4c90 secure :-1 s=READY pgs=955 cs=0 l=1 rev1=1 crypto rx=0x7f9f4c001eb0 tx=0x7f9f4c00b670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b933bf0303c777e server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:52:20.303 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:20.300+0000 7f9f3f7fe640 1 -- 172.21.15.39:0/1849076508 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f4c012170 con 0x55d5d4f4a070 2024-09-21T03:52:20.303 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:20.300+0000 7f9f3f7fe640 1 -- 172.21.15.39:0/1849076508 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9f4c00b390 con 0x55d5d4f4a070 2024-09-21T03:52:20.303 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:20.300+0000 7f9f3f7fe640 1 -- 172.21.15.39:0/1849076508 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f4c029400 con 0x55d5d4f4a070 2024-09-21T03:52:20.306 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:20.301+0000 7f9f663b5740 1 -- 172.21.15.39:0/1849076508 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d5d4f4a070 msgr2=0x55d5d52f4c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:52:20.306 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:20.301+0000 7f9f663b5740 1 --2- 172.21.15.39:0/1849076508 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d5d4f4a070 0x55d5d52f4c90 secure :-1 s=READY pgs=955 cs=0 l=1 rev1=1 crypto rx=0x7f9f4c001eb0 tx=0x7f9f4c00b670 comp rx=0 tx=0).stop 2024-09-21T03:52:20.306 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:20.301+0000 7f9f663b5740 1 -- 172.21.15.39:0/1849076508 shutdown_connections 2024-09-21T03:52:20.306 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:20.301+0000 7f9f663b5740 1 --2- 172.21.15.39:0/1849076508 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d5d4f4a070 0x55d5d52f4c90 unknown :-1 s=CLOSED pgs=955 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:20.306 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:20.301+0000 7f9f663b5740 1 --2- 172.21.15.39:0/1849076508 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d5d4f29390 0x55d5d52f4750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:20.306 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:20.301+0000 7f9f663b5740 1 --2- 172.21.15.39:0/1849076508 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d5d4f211b0 0x55d5d52fd220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:20.306 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:20.301+0000 7f9f663b5740 1 -- 172.21.15.39:0/1849076508 >> 172.21.15.39:0/1849076508 conn(0x55d5d4f3e460 msgr2=0x55d5d52fcb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:52:20.306 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:20.301+0000 7f9f663b5740 1 -- 172.21.15.39:0/1849076508 shutdown_connections 2024-09-21T03:52:20.306 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:20.301+0000 7f9f663b5740 1 -- 172.21.15.39:0/1849076508 wait complete. 2024-09-21T03:52:20.306 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:20.301+0000 7f9f663b5740 1 -- start start 2024-09-21T03:52:20.306 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:20.301+0000 7f9f663b5740 1 Processor -- start 2024-09-21T03:52:20.306 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:20.302+0000 7f9f663b5740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d5d4f211b0 0x55d5d53012e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:52:20.307 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:20.302+0000 7f9f663b5740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d5d4f29390 0x55d5d5301820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:52:20.307 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:20.302+0000 7f9f663b5740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d5d4f4a070 0x55d5d53789d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:52:20.307 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:20.302+0000 7f9f663b5740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f9f581b2e20 con 0x55d5d4f4a070 2024-09-21T03:52:20.307 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:20.302+0000 7f9f663b5740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f9f581b2750 con 0x55d5d4f29390 2024-09-21T03:52:20.307 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:20.302+0000 7f9f663b5740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f9f581af990 con 0x55d5d4f211b0 2024-09-21T03:52:20.307 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:20.302+0000 7f9f663b5740 10 client.0 _ll_register_callbacks cb 0x7f9f4c00bcb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:52:20.307 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:20.302+0000 7f9f5d3dc640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d5d4f4a070 0x55d5d53789d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:52:20.307 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:20.302+0000 7f9f5d3dc640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d5d4f4a070 0x55d5d53789d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:39902/0 (socket says 172.21.15.39:39902) 2024-09-21T03:52:20.307 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:20.302+0000 7f9f5d3dc640 1 -- 172.21.15.39:0/2746520770 learned_addr learned my addr 172.21.15.39:0/2746520770 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:52:20.307 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:20.302+0000 7f9f5dbdd640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d5d4f29390 0x55d5d5301820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:52:20.307 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:20.302+0000 7f9f5d3dc640 1 --2- 172.21.15.39:0/2746520770 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d5d4f211b0 0x55d5d53012e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:52:20.307 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:20.302+0000 7f9f5d3dc640 1 -- 172.21.15.39:0/2746520770 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d5d4f211b0 msgr2=0x55d5d53012e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:52:20.307 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:20.302+0000 7f9f5d3dc640 1 --2- 172.21.15.39:0/2746520770 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d5d4f211b0 0x55d5d53012e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:20.307 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:20.302+0000 7f9f5d3dc640 1 -- 172.21.15.39:0/2746520770 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d5d4f29390 msgr2=0x55d5d5301820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:52:20.308 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:20.302+0000 7f9f5d3dc640 1 --2- 172.21.15.39:0/2746520770 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d5d4f29390 0x55d5d5301820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:20.308 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:20.302+0000 7f9f5d3dc640 1 -- 172.21.15.39:0/2746520770 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d5d5302010 con 0x55d5d4f4a070 2024-09-21T03:52:20.308 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:20.302+0000 7f9f5d3dc640 1 --2- 172.21.15.39:0/2746520770 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d5d4f4a070 0x55d5d53789d0 secure :-1 s=READY pgs=956 cs=0 l=1 rev1=1 crypto rx=0x7f9f4c012a30 tx=0x7f9f4c004150 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3596a007116e914a server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:52:20.308 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:20.302+0000 7f9f3d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-09-21T03:52:20.308 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:20.303+0000 7f9f3d7fa640 1 -- 172.21.15.39:0/2746520770 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f4c029610 con 0x55d5d4f4a070 2024-09-21T03:52:20.308 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:20.303+0000 7f9f3d7fa640 1 -- 172.21.15.39:0/2746520770 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8484+0+0 (secure 0 0 0) 0x7f9f4c005020 con 0x55d5d4f4a070 2024-09-21T03:52:20.308 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:20.303+0000 7f9f663b5740 1 -- 172.21.15.39:0/2746520770 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9f4c00b390 con 0x55d5d4f4a070 2024-09-21T03:52:20.308 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:20.303+0000 7f9f3d7fa640 1 -- 172.21.15.39:0/2746520770 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f9f4c005220 con 0x55d5d4f4a070 2024-09-21T03:52:20.308 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:20.303+0000 7f9f663b5740 10 client.17145 fetch_fsmap learned FSMap version 29 2024-09-21T03:52:20.308 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:20.303+0000 7f9f663b5740 1 -- 172.21.15.39:0/2746520770 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55d5d5379e40 con 0x55d5d4f4a070 2024-09-21T03:52:20.308 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:20.303+0000 7f9f3d7fa640 1 -- 172.21.15.39:0/2746520770 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9f4c00b390 con 0x55d5d4f4a070 2024-09-21T03:52:20.308 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:20.303+0000 7f9f3d7fa640 1 -- 172.21.15.39:0/2746520770 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f9f4c035480 con 0x55d5d4f4a070 2024-09-21T03:52:20.308 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:20.303+0000 7f9f663b5740 10 client.17145 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T03:52:20.308 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:20.303+0000 7f9f663b5740 10 client.17145 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T03:52:20.308 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:20.303+0000 7f9f663b5740 20 client.17145 populate_metadata read hostname 'smithi039' 2024-09-21T03:52:20.309 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:20.303+0000 7f9f663b5740 1 --2- 172.21.15.39:0/2746520770 >> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] conn(0x55d5d4eddda0 0x55d5d537c4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:52:20.309 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:20.303+0000 7f9f663b5740 4 client.17145 mds_command: new command op to 24695 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-21T03:52:20.309 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:20.303+0000 7f9f663b5740 1 -- 172.21.15.39:0/2746520770 --> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55d5d5265850 con 0x55d5d4eddda0 2024-09-21T03:52:20.309 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:20.304+0000 7f9f5dbdd640 1 --2- 172.21.15.39:0/2746520770 >> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] conn(0x55d5d4eddda0 0x55d5d537c4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:52:20.309 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:20.304+0000 7f9f5dbdd640 1 --2- 172.21.15.39:0/2746520770 >> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] conn(0x55d5d4eddda0 0x55d5d537c4b0 crc :-1 s=READY pgs=112 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3b647bd8966c97f3 server_cookie=7282f91e9f585c05 in_seq=0 out_seq=0 2024-09-21T03:52:20.309 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:20.304+0000 7f9f3d7fa640 10 client.17145 ms_handle_connect on v2:172.21.15.150:6838/2949386907 2024-09-21T03:52:20.310 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:20.304+0000 7f9f3d7fa640 1 -- 172.21.15.39:0/2746520770 <== mds.0 v2:172.21.15.150:6838/2949386907 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3498 (crc 0 0 0) 0x55d5d5265850 con 0x55d5d4eddda0 2024-09-21T03:52:20.310 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:20.304+0000 7f9f3d7fa640 10 client.17145 handle_command_reply: tid=0 2024-09-21T03:52:20.310 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:20.304+0000 7f9f663b5740 1 client.17145 shutdown 2024-09-21T03:52:20.310 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:20.305+0000 7f9f663b5740 1 -- 172.21.15.39:0/2746520770 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d5d4f4a070 msgr2=0x55d5d53789d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:52:20.310 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:20.305+0000 7f9f663b5740 1 --2- 172.21.15.39:0/2746520770 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d5d4f4a070 0x55d5d53789d0 secure :-1 s=READY pgs=956 cs=0 l=1 rev1=1 crypto rx=0x7f9f4c012a30 tx=0x7f9f4c004150 comp rx=0 tx=0).stop 2024-09-21T03:52:20.310 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:20.305+0000 7f9f663b5740 1 -- 172.21.15.39:0/2746520770 shutdown_connections 2024-09-21T03:52:20.310 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:20.305+0000 7f9f663b5740 1 --2- 172.21.15.39:0/2746520770 >> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] conn(0x55d5d4eddda0 0x55d5d537c4b0 crc :-1 s=READY pgs=112 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:20.311 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:20.305+0000 7f9f663b5740 1 --2- 172.21.15.39:0/2746520770 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d5d4f4a070 0x55d5d53789d0 unknown :-1 s=CLOSED pgs=956 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:20.311 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:20.305+0000 7f9f663b5740 1 --2- 172.21.15.39:0/2746520770 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d5d4f29390 0x55d5d5301820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:20.311 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:20.305+0000 7f9f663b5740 1 --2- 172.21.15.39:0/2746520770 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d5d4f211b0 0x55d5d53012e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:20.311 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:20.305+0000 7f9f3d7fa640 0 client.17145 ms_handle_reset on v2:172.21.15.150:6838/2949386907 2024-09-21T03:52:20.311 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:20.305+0000 7f9f663b5740 1 -- 172.21.15.39:0/2746520770 >> 172.21.15.39:0/2746520770 conn(0x55d5d4f3e460 msgr2=0x55d5d52f4750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:52:20.311 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:20.305+0000 7f9f663b5740 1 -- 172.21.15.39:0/2746520770 shutdown_connections 2024-09-21T03:52:20.311 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:20.305+0000 7f9f663b5740 1 -- 172.21.15.39:0/2746520770 wait complete. 2024-09-21T03:52:20.311 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:20.305+0000 7f9f663b5740 20 client.17145 trim_cache size 0 max 16384 2024-09-21T03:52:20.311 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-09-21T03:52:20.311 INFO:teuthology.orchestra.run.smithi039.stdout: "status": "no active scrubs running", 2024-09-21T03:52:20.311 INFO:teuthology.orchestra.run.smithi039.stdout: "scrubs": {}, 2024-09-21T03:52:20.312 INFO:teuthology.orchestra.run.smithi039.stdout: "scrub_stats": { 2024-09-21T03:52:20.312 INFO:teuthology.orchestra.run.smithi039.stdout: "4bda6ee3-7237-4ee4-96ce-150ad50f26dc": { 2024-09-21T03:52:20.312 INFO:teuthology.orchestra.run.smithi039.stdout: "start_time": "2024-09-21T03:52:18.622141+0000", 2024-09-21T03:52:20.312 INFO:teuthology.orchestra.run.smithi039.stdout: "path": "/", 2024-09-21T03:52:20.312 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_started": 0, 2024-09-21T03:52:20.312 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_passed": 0, 2024-09-21T03:52:20.312 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_failed": 0, 2024-09-21T03:52:20.312 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_skipped": 0 2024-09-21T03:52:20.312 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:52:20.312 INFO:teuthology.orchestra.run.smithi039.stdout: "ae1fc4cf-5458-47d0-afcb-282b136dba15": { 2024-09-21T03:52:20.312 INFO:teuthology.orchestra.run.smithi039.stdout: "start_time": "2024-09-21T03:51:18.560378+0000", 2024-09-21T03:52:20.312 INFO:teuthology.orchestra.run.smithi039.stdout: "path": "/", 2024-09-21T03:52:20.312 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_started": 0, 2024-09-21T03:52:20.312 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_passed": 0, 2024-09-21T03:52:20.312 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_failed": 0, 2024-09-21T03:52:20.313 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_skipped": 0 2024-09-21T03:52:20.313 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:52:20.313 INFO:teuthology.orchestra.run.smithi039.stdout: "160750d8-401f-4810-866d-e63e7e4bc3e0": { 2024-09-21T03:52:20.313 INFO:teuthology.orchestra.run.smithi039.stdout: "start_time": "2024-09-21T03:51:04.208673+0000", 2024-09-21T03:52:20.313 INFO:teuthology.orchestra.run.smithi039.stdout: "path": "/", 2024-09-21T03:52:20.313 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_started": 0, 2024-09-21T03:52:20.313 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_passed": 0, 2024-09-21T03:52:20.313 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_failed": 0, 2024-09-21T03:52:20.313 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_skipped": 0 2024-09-21T03:52:20.313 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:52:20.313 INFO:teuthology.orchestra.run.smithi039.stdout: "74ab917e-ecea-4778-a1c3-c184214fb24c": { 2024-09-21T03:52:20.313 INFO:teuthology.orchestra.run.smithi039.stdout: "start_time": "2024-09-21T03:50:17.726630+0000", 2024-09-21T03:52:20.313 INFO:teuthology.orchestra.run.smithi039.stdout: "path": "/", 2024-09-21T03:52:20.313 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_started": 0, 2024-09-21T03:52:20.313 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_passed": 0, 2024-09-21T03:52:20.314 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_failed": 0, 2024-09-21T03:52:20.314 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_skipped": 0 2024-09-21T03:52:20.314 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:52:20.314 INFO:teuthology.orchestra.run.smithi039.stdout: "6e760f1f-9048-4f92-bd66-d708fbef75b3": { 2024-09-21T03:52:20.314 INFO:teuthology.orchestra.run.smithi039.stdout: "start_time": "2024-09-21T03:50:03.711918+0000", 2024-09-21T03:52:20.314 INFO:teuthology.orchestra.run.smithi039.stdout: "path": "/", 2024-09-21T03:52:20.314 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_started": 0, 2024-09-21T03:52:20.314 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_passed": 0, 2024-09-21T03:52:20.314 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_failed": 0, 2024-09-21T03:52:20.314 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_skipped": 0 2024-09-21T03:52:20.314 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:52:20.314 INFO:teuthology.orchestra.run.smithi039.stdout: "e28ad743-2b22-45b3-8799-a638c2883623": { 2024-09-21T03:52:20.314 INFO:teuthology.orchestra.run.smithi039.stdout: "start_time": "2024-09-21T03:48:31.448821+0000", 2024-09-21T03:52:20.314 INFO:teuthology.orchestra.run.smithi039.stdout: "path": "/", 2024-09-21T03:52:20.315 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_started": 0, 2024-09-21T03:52:20.315 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_passed": 0, 2024-09-21T03:52:20.315 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_failed": 0, 2024-09-21T03:52:20.315 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_skipped": 0 2024-09-21T03:52:20.315 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:52:20.315 INFO:teuthology.orchestra.run.smithi039.stdout: "ede73e6e-3fd0-4a7f-b5bb-f68946f57eea": { 2024-09-21T03:52:20.315 INFO:teuthology.orchestra.run.smithi039.stdout: "start_time": "2024-09-21T03:51:32.235096+0000", 2024-09-21T03:52:20.315 INFO:teuthology.orchestra.run.smithi039.stdout: "path": "/", 2024-09-21T03:52:20.315 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_started": 0, 2024-09-21T03:52:20.315 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_passed": 0, 2024-09-21T03:52:20.315 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_failed": 0, 2024-09-21T03:52:20.315 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_skipped": 0 2024-09-21T03:52:20.315 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:52:20.315 INFO:teuthology.orchestra.run.smithi039.stdout: "29411307-eb8d-4339-8d2b-47ebd32c0080": { 2024-09-21T03:52:20.316 INFO:teuthology.orchestra.run.smithi039.stdout: "start_time": "2024-09-21T03:47:38.092920+0000", 2024-09-21T03:52:20.316 INFO:teuthology.orchestra.run.smithi039.stdout: "path": "/", 2024-09-21T03:52:20.316 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_started": 0, 2024-09-21T03:52:20.316 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_passed": 0, 2024-09-21T03:52:20.316 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_failed": 0, 2024-09-21T03:52:20.316 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_skipped": 0 2024-09-21T03:52:20.316 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:52:20.316 INFO:teuthology.orchestra.run.smithi039.stdout: "9860c2f4-b7b1-47aa-865a-9a1c4434a5f6": { 2024-09-21T03:52:20.316 INFO:teuthology.orchestra.run.smithi039.stdout: "start_time": "2024-09-21T03:49:17.718867+0000", 2024-09-21T03:52:20.316 INFO:teuthology.orchestra.run.smithi039.stdout: "path": "/", 2024-09-21T03:52:20.316 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_started": 0, 2024-09-21T03:52:20.316 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_passed": 0, 2024-09-21T03:52:20.316 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_failed": 0, 2024-09-21T03:52:20.316 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_skipped": 0 2024-09-21T03:52:20.317 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:52:20.317 INFO:teuthology.orchestra.run.smithi039.stdout: "c7e2b8e4-057d-435f-b2bd-b633de98035f": { 2024-09-21T03:52:20.317 INFO:teuthology.orchestra.run.smithi039.stdout: "start_time": "2024-09-21T03:47:19.400305+0000", 2024-09-21T03:52:20.317 INFO:teuthology.orchestra.run.smithi039.stdout: "path": "/", 2024-09-21T03:52:20.317 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_started": 0, 2024-09-21T03:52:20.317 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_passed": 0, 2024-09-21T03:52:20.317 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_failed": 0, 2024-09-21T03:52:20.317 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_skipped": 0 2024-09-21T03:52:20.317 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:52:20.317 INFO:teuthology.orchestra.run.smithi039.stdout: "10728de8-c49d-4edc-be2b-639249255e94": { 2024-09-21T03:52:20.317 INFO:teuthology.orchestra.run.smithi039.stdout: "start_time": "2024-09-21T03:47:03.965623+0000", 2024-09-21T03:52:20.317 INFO:teuthology.orchestra.run.smithi039.stdout: "path": "/", 2024-09-21T03:52:20.317 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_started": 0, 2024-09-21T03:52:20.317 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_passed": 0, 2024-09-21T03:52:20.317 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_failed": 0, 2024-09-21T03:52:20.318 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_skipped": 0 2024-09-21T03:52:20.318 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:52:20.318 INFO:teuthology.orchestra.run.smithi039.stdout: "02509e4f-3911-4983-b16f-df8ded0b9998": { 2024-09-21T03:52:20.318 INFO:teuthology.orchestra.run.smithi039.stdout: "start_time": "2024-09-21T03:46:17.889576+0000", 2024-09-21T03:52:20.318 INFO:teuthology.orchestra.run.smithi039.stdout: "path": "/", 2024-09-21T03:52:20.318 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_started": 0, 2024-09-21T03:52:20.318 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_passed": 0, 2024-09-21T03:52:20.318 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_failed": 0, 2024-09-21T03:52:20.318 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_skipped": 0 2024-09-21T03:52:20.318 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-21T03:52:20.318 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-21T03:52:20.318 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-09-21T03:52:20.319 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:20.305+0000 7f9f5fe41640 1 -- 172.21.15.39:0/2660617463 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f9f400b6680 msgr2=0x7f9f400b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:52:20.319 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:20.305+0000 7f9f5fe41640 1 --2- 172.21.15.39:0/2660617463 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f9f400b6680 0x7f9f400b8b90 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f9f481067d0 tx=0x7f9f4809b690 comp rx=0 tx=0).stop 2024-09-21T03:52:20.319 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:20.305+0000 7f9f5fe41640 1 -- 172.21.15.39:0/2660617463 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9f5810de40 msgr2=0x7f9f581031a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:52:20.319 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:20.305+0000 7f9f5fe41640 1 --2- 172.21.15.39:0/2660617463 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9f5810de40 0x7f9f581031a0 secure :-1 s=READY pgs=952 cs=0 l=1 rev1=1 crypto rx=0x7f9f48009430 tx=0x7f9f480319f0 comp rx=0 tx=0).stop 2024-09-21T03:52:20.319 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:20.306+0000 7f9f5fe41640 1 -- 172.21.15.39:0/2660617463 shutdown_connections 2024-09-21T03:52:20.319 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:20.306+0000 7f9f5fe41640 1 --2- 172.21.15.39:0/2660617463 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f9f400b6680 0x7f9f400b8b90 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:20.319 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:20.306+0000 7f9f5fe41640 1 --2- 172.21.15.39:0/2660617463 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9f5810de40 0x7f9f581031a0 unknown :-1 s=CLOSED pgs=952 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:20.319 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:20.306+0000 7f9f5fe41640 1 --2- 172.21.15.39:0/2660617463 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9f58105330 0x7f9f58102c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:20.319 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:20.306+0000 7f9f5fe41640 1 --2- 172.21.15.39:0/2660617463 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9f581049e0 0x7f9f58102720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:20.320 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:20.306+0000 7f9f5fe41640 1 -- 172.21.15.39:0/2660617463 >> 172.21.15.39:0/2660617463 conn(0x7f9f58079730 msgr2=0x7f9f580ff4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:52:20.320 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:20.306+0000 7f9f5fe41640 1 -- 172.21.15.39:0/2660617463 shutdown_connections 2024-09-21T03:52:20.320 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:20.306+0000 7f9f5fe41640 1 -- 172.21.15.39:0/2660617463 wait complete. 2024-09-21T03:52:20.584 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:52:20 smithi039 ceph-mon[31211]: scrub summary: idle 2024-09-21T03:52:20.718 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:52:20 smithi161 ceph-mon[34533]: scrub summary: idle 2024-09-21T03:52:20.724 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:52:20 smithi150 ceph-mon[34289]: scrub summary: idle 2024-09-21T03:52:21.040 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-21T03:52:21.040 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 51116d5c-77ca-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T03:52:21.296 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/51116d5c-77ca-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:52:21.334 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:52:21 smithi039 ceph-mon[31211]: pgmap v514: 97 pgs: 97 active+clean; 10 MiB data, 520 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 170 B/s wr, 11 op/s 2024-09-21T03:52:21.718 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:52:21 smithi161 ceph-mon[34533]: pgmap v514: 97 pgs: 97 active+clean; 10 MiB data, 520 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 170 B/s wr, 11 op/s 2024-09-21T03:52:21.725 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:52:21 smithi150 ceph-mon[34289]: pgmap v514: 97 pgs: 97 active+clean; 10 MiB data, 520 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 170 B/s wr, 11 op/s 2024-09-21T03:52:21.816 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:21.806+0000 7fbd34795640 1 -- 172.21.15.39:0/3030621357 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbd2c109e40 msgr2=0x7fbd2c110670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:52:21.816 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:21.806+0000 7fbd34795640 1 --2- 172.21.15.39:0/3030621357 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbd2c109e40 0x7fbd2c110670 secure :-1 s=READY pgs=957 cs=0 l=1 rev1=1 crypto rx=0x7fbd24009ca0 tx=0x7fbd2402a8c0 comp rx=0 tx=0).stop 2024-09-21T03:52:21.816 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:21.806+0000 7fbd34795640 1 -- 172.21.15.39:0/3030621357 shutdown_connections 2024-09-21T03:52:21.816 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:21.806+0000 7fbd34795640 1 --2- 172.21.15.39:0/3030621357 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbd2c109e40 0x7fbd2c110670 unknown :-1 s=CLOSED pgs=957 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:21.816 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:21.806+0000 7fbd34795640 1 --2- 172.21.15.39:0/3030621357 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbd2c1094f0 0x7fbd2c109900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:21.816 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:21.806+0000 7fbd34795640 1 --2- 172.21.15.39:0/3030621357 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbd2c103bb0 0x7fbd2c103fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:21.816 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:21.807+0000 7fbd34795640 1 -- 172.21.15.39:0/3030621357 >> 172.21.15.39:0/3030621357 conn(0x7fbd2c0ff600 msgr2=0x7fbd2c101a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:52:21.816 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:21.807+0000 7fbd34795640 1 -- 172.21.15.39:0/3030621357 shutdown_connections 2024-09-21T03:52:21.816 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:21.807+0000 7fbd34795640 1 -- 172.21.15.39:0/3030621357 wait complete. 2024-09-21T03:52:21.816 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:21.807+0000 7fbd34795640 1 Processor -- start 2024-09-21T03:52:21.817 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:21.807+0000 7fbd34795640 1 -- start start 2024-09-21T03:52:21.817 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:21.807+0000 7fbd34795640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbd2c103bb0 0x7fbd2c19a450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:52:21.817 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:21.808+0000 7fbd34795640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbd2c1094f0 0x7fbd2c19a9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:52:21.817 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:21.808+0000 7fbd34795640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbd2c109e40 0x7fbd2c19fc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:52:21.817 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:21.808+0000 7fbd34795640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fbd2c113b00 con 0x7fbd2c109e40 2024-09-21T03:52:21.817 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:21.808+0000 7fbd34795640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fbd2c113980 con 0x7fbd2c1094f0 2024-09-21T03:52:21.817 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:21.808+0000 7fbd34795640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fbd2c113c80 con 0x7fbd2c103bb0 2024-09-21T03:52:21.817 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:21.808+0000 7fbd32d32640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbd2c109e40 0x7fbd2c19fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:52:21.817 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:21.808+0000 7fbd32d32640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbd2c109e40 0x7fbd2c19fc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:39922/0 (socket says 172.21.15.39:39922) 2024-09-21T03:52:21.817 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:21.808+0000 7fbd32d32640 1 -- 172.21.15.39:0/2997722194 learned_addr learned my addr 172.21.15.39:0/2997722194 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:52:21.817 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:21.808+0000 7fbd31d30640 1 --2- 172.21.15.39:0/2997722194 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbd2c1094f0 0x7fbd2c19a9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:52:21.817 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:21.808+0000 7fbd32531640 1 --2- 172.21.15.39:0/2997722194 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbd2c103bb0 0x7fbd2c19a450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:52:21.817 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:21.808+0000 7fbd32d32640 1 -- 172.21.15.39:0/2997722194 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbd2c103bb0 msgr2=0x7fbd2c19a450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:52:21.818 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:21.808+0000 7fbd32d32640 1 --2- 172.21.15.39:0/2997722194 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbd2c103bb0 0x7fbd2c19a450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:21.818 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:21.808+0000 7fbd32d32640 1 -- 172.21.15.39:0/2997722194 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbd2c1094f0 msgr2=0x7fbd2c19a9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:52:21.818 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:21.808+0000 7fbd32d32640 1 --2- 172.21.15.39:0/2997722194 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbd2c1094f0 0x7fbd2c19a9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:21.818 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:21.808+0000 7fbd32d32640 1 -- 172.21.15.39:0/2997722194 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbd2c1a03c0 con 0x7fbd2c109e40 2024-09-21T03:52:21.818 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:21.808+0000 7fbd32d32640 1 --2- 172.21.15.39:0/2997722194 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbd2c109e40 0x7fbd2c19fc90 secure :-1 s=READY pgs=958 cs=0 l=1 rev1=1 crypto rx=0x7fbd24005cc0 tx=0x7fbd2402f380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:52:21.818 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:21.808+0000 7fbd237fe640 1 -- 172.21.15.39:0/2997722194 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd24003bf0 con 0x7fbd2c109e40 2024-09-21T03:52:21.818 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:21.809+0000 7fbd237fe640 1 -- 172.21.15.39:0/2997722194 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbd24003d90 con 0x7fbd2c109e40 2024-09-21T03:52:21.818 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:21.809+0000 7fbd34795640 1 -- 172.21.15.39:0/2997722194 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbd2c1a0650 con 0x7fbd2c109e40 2024-09-21T03:52:21.818 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:21.809+0000 7fbd237fe640 1 -- 172.21.15.39:0/2997722194 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd2402fbc0 con 0x7fbd2c109e40 2024-09-21T03:52:21.818 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:21.809+0000 7fbd34795640 1 -- 172.21.15.39:0/2997722194 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbd2c1a6e10 con 0x7fbd2c109e40 2024-09-21T03:52:21.818 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:21.810+0000 7fbd34795640 1 -- 172.21.15.39:0/2997722194 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbd2c103fc0 con 0x7fbd2c109e40 2024-09-21T03:52:21.818 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:21.811+0000 7fbd237fe640 1 -- 172.21.15.39:0/2997722194 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 27) ==== 154643+0+0 (secure 0 0 0) 0x7fbd2402fd60 con 0x7fbd2c109e40 2024-09-21T03:52:21.819 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:21.812+0000 7fbd237fe640 1 --2- 172.21.15.39:0/2997722194 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7fbd180b6680 0x7fbd180b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:52:21.819 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:21.812+0000 7fbd32531640 1 --2- 172.21.15.39:0/2997722194 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7fbd180b6680 0x7fbd180b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:52:21.819 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:21.812+0000 7fbd237fe640 1 -- 172.21.15.39:0/2997722194 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8484+0+0 (secure 0 0 0) 0x7fbd24108d90 con 0x7fbd2c109e40 2024-09-21T03:52:21.819 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:21.813+0000 7fbd32531640 1 --2- 172.21.15.39:0/2997722194 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7fbd180b6680 0x7fbd180b8b90 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fbd28005fd0 tx=0x7fbd28005850 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:52:21.819 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:21.815+0000 7fbd237fe640 1 -- 172.21.15.39:0/2997722194 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fbd240cf300 con 0x7fbd2c109e40 2024-09-21T03:52:22.044 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:22.042+0000 7fbd34795640 1 -- 172.21.15.39:0/2997722194 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fbd2c19b580 con 0x7fbd2c109e40 2024-09-21T03:52:22.045 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:22.043+0000 7fbd237fe640 1 -- 172.21.15.39:0/2997722194 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11978 (secure 0 0 0) 0x7fbd2c19b580 con 0x7fbd2c109e40 2024-09-21T03:52:22.045 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T03:52:22.046 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":29,"btime":"2024-09-21T03:46:08:982881+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14790,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6835/508995295","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":508995295},{"type":"v1","addr":"172.21.15.39:6835","nonce":508995295}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14802,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6837/3915912607","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":3915912607},{"type":"v1","addr":"172.21.15.39:6837","nonce":3915912607}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":29,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T03:44:15.009872+0000","modified":"2024-09-21T03:46:08.430113+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24695,"mds_1":24667,"mds_2":24659,"mds_3":24646,"mds_4":14868},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14856":{"gid":14856,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.39:6839/185509192","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6838","nonce":185509192},{"type":"v1","addr":"172.21.15.39:6839","nonce":185509192}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14868":{"gid":14868,"name":"a","rank":4,"incarnation":25,"state":"up:active","state_seq":7,"addr":"172.21.15.39:6841/4247190456","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6840","nonce":4247190456},{"type":"v1","addr":"172.21.15.39:6841","nonce":4247190456}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24599":{"gid":24599,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.150:6833/3114643217","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":3114643217},{"type":"v1","addr":"172.21.15.150:6833","nonce":3114643217}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24601":{"gid":24601,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.161:6833/2963990654","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":2963990654},{"type":"v1","addr":"172.21.15.161:6833","nonce":2963990654}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24632":{"gid":24632,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.150:6835/724594008","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6834","nonce":724594008},{"type":"v1","addr":"172.21.15.150:6835","nonce":724594008}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24640":{"gid":24640,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.161:6835/3747882110","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":3747882110},{"type":"v1","addr":"172.21.15.161:6835","nonce":3747882110}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24646":{"gid":24646,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":10,"addr":"172.21.15.161:6837/3451544597","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6836","nonce":3451544597},{"type":"v1","addr":"172.21.15.161:6837","nonce":3451544597}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24659":{"gid":24659,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.150:6837/4017043144","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6836","nonce":4017043144},{"type":"v1","addr":"172.21.15.150:6837","nonce":4017043144}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24667":{"gid":24667,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.161:6839/2190107270","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6838","nonce":2190107270},{"type":"v1","addr":"172.21.15.161:6839","nonce":2190107270}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24695":{"gid":24695,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.150:6839/2949386907","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6838","nonce":2949386907},{"type":"v1","addr":"172.21.15.150:6839","nonce":2949386907}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24695,"qdb_cluster":[24695,24667,24659,24646,14868]},"id":1}]} 2024-09-21T03:52:22.046 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 29 2024-09-21T03:52:22.048 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:22.046+0000 7fbd34795640 1 -- 172.21.15.39:0/2997722194 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7fbd180b6680 msgr2=0x7fbd180b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:52:22.048 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:22.046+0000 7fbd34795640 1 --2- 172.21.15.39:0/2997722194 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7fbd180b6680 0x7fbd180b8b90 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fbd28005fd0 tx=0x7fbd28005850 comp rx=0 tx=0).stop 2024-09-21T03:52:22.048 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:22.046+0000 7fbd34795640 1 -- 172.21.15.39:0/2997722194 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbd2c109e40 msgr2=0x7fbd2c19fc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:52:22.048 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:22.046+0000 7fbd34795640 1 --2- 172.21.15.39:0/2997722194 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbd2c109e40 0x7fbd2c19fc90 secure :-1 s=READY pgs=958 cs=0 l=1 rev1=1 crypto rx=0x7fbd24005cc0 tx=0x7fbd2402f380 comp rx=0 tx=0).stop 2024-09-21T03:52:22.048 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:22.046+0000 7fbd34795640 1 -- 172.21.15.39:0/2997722194 shutdown_connections 2024-09-21T03:52:22.048 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:22.046+0000 7fbd34795640 1 --2- 172.21.15.39:0/2997722194 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7fbd180b6680 0x7fbd180b8b90 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:22.048 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:22.046+0000 7fbd34795640 1 --2- 172.21.15.39:0/2997722194 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbd2c109e40 0x7fbd2c19fc90 unknown :-1 s=CLOSED pgs=958 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:22.048 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:22.046+0000 7fbd34795640 1 --2- 172.21.15.39:0/2997722194 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbd2c1094f0 0x7fbd2c19a9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:22.048 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:22.046+0000 7fbd34795640 1 --2- 172.21.15.39:0/2997722194 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbd2c103bb0 0x7fbd2c19a450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:22.048 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:22.046+0000 7fbd34795640 1 -- 172.21.15.39:0/2997722194 >> 172.21.15.39:0/2997722194 conn(0x7fbd2c0ff600 msgr2=0x7fbd2c108b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:52:22.048 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:22.046+0000 7fbd34795640 1 -- 172.21.15.39:0/2997722194 shutdown_connections 2024-09-21T03:52:22.048 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:22.046+0000 7fbd34795640 1 -- 172.21.15.39:0/2997722194 wait complete. 2024-09-21T03:52:22.334 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:52:22 smithi039 ceph-mon[31211]: from='client.? 172.21.15.39:0/2997722194' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:52:22.718 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:52:22 smithi161 ceph-mon[34533]: from='client.? 172.21.15.39:0/2997722194' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:52:22.724 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:52:22 smithi150 ceph-mon[34289]: from='client.? 172.21.15.39:0/2997722194' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:52:22.835 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 51116d5c-77ca-11ef-baf5-efdc52797490 -- ceph tell mds.1:4 damage ls 2024-09-21T03:52:23.092 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/51116d5c-77ca-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:52:23.503 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:52:23 smithi039 ceph-mon[31211]: pgmap v515: 97 pgs: 97 active+clean; 10 MiB data, 520 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 85 B/s wr, 9 op/s 2024-09-21T03:52:23.508 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:23.501+0000 7f38bef60640 1 -- 172.21.15.39:0/1018075675 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f38b81031f0 msgr2=0x7f38b8103600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:52:23.508 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:23.501+0000 7f38bef60640 1 --2- 172.21.15.39:0/1018075675 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f38b81031f0 0x7f38b8103600 secure :-1 s=READY pgs=959 cs=0 l=1 rev1=1 crypto rx=0x7f38ac009900 tx=0x7f38ac02a410 comp rx=0 tx=0).stop 2024-09-21T03:52:23.509 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:23.501+0000 7f38bef60640 1 -- 172.21.15.39:0/1018075675 shutdown_connections 2024-09-21T03:52:23.509 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:23.501+0000 7f38bef60640 1 --2- 172.21.15.39:0/1018075675 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f38b8105b10 0x7f38b810f5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:23.509 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:23.501+0000 7f38bef60640 1 --2- 172.21.15.39:0/1018075675 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f38b81051c0 0x7f38b81055d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:23.509 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:23.501+0000 7f38bef60640 1 --2- 172.21.15.39:0/1018075675 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f38b81031f0 0x7f38b8103600 unknown :-1 s=CLOSED pgs=959 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:23.509 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:23.501+0000 7f38bef60640 1 -- 172.21.15.39:0/1018075675 >> 172.21.15.39:0/1018075675 conn(0x7f38b8079e40 msgr2=0x7f38b807a250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:52:23.509 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:23.501+0000 7f38bef60640 1 -- 172.21.15.39:0/1018075675 shutdown_connections 2024-09-21T03:52:23.509 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:23.501+0000 7f38bef60640 1 -- 172.21.15.39:0/1018075675 wait complete. 2024-09-21T03:52:23.509 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:23.502+0000 7f38bef60640 1 Processor -- start 2024-09-21T03:52:23.510 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:23.502+0000 7f38bef60640 1 -- start start 2024-09-21T03:52:23.510 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:23.503+0000 7f38bef60640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f38b81031f0 0x7f38b8110080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:52:23.510 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:23.503+0000 7f38bef60640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f38b81051c0 0x7f38b81105c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:52:23.510 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:23.503+0000 7f38bef60640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f38b8105b10 0x7f38b8110b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:52:23.510 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:23.503+0000 7f38bef60640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f38b8113af0 con 0x7f38b81031f0 2024-09-21T03:52:23.510 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:23.503+0000 7f38bef60640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f38b8113970 con 0x7f38b81051c0 2024-09-21T03:52:23.510 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:23.503+0000 7f38bef60640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f38b8113c70 con 0x7f38b8105b10 2024-09-21T03:52:23.510 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:23.503+0000 7f38bccfc640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f38b81031f0 0x7f38b8110080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:52:23.511 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:23.503+0000 7f38bccfc640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f38b81031f0 0x7f38b8110080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:39950/0 (socket says 172.21.15.39:39950) 2024-09-21T03:52:23.511 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:23.503+0000 7f38bccfc640 1 -- 172.21.15.39:0/4066623220 learned_addr learned my addr 172.21.15.39:0/4066623220 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:52:23.511 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:23.503+0000 7f38bd4fd640 1 --2- 172.21.15.39:0/4066623220 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f38b8105b10 0x7f38b8110b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:52:23.511 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:23.503+0000 7f38b7fff640 1 --2- 172.21.15.39:0/4066623220 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f38b81051c0 0x7f38b81105c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:52:23.511 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:23.503+0000 7f38bccfc640 1 -- 172.21.15.39:0/4066623220 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f38b8105b10 msgr2=0x7f38b8110b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:52:23.511 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:23.503+0000 7f38bccfc640 1 --2- 172.21.15.39:0/4066623220 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f38b8105b10 0x7f38b8110b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:23.511 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:23.503+0000 7f38bccfc640 1 -- 172.21.15.39:0/4066623220 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f38b81051c0 msgr2=0x7f38b81105c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:52:23.511 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:23.503+0000 7f38bccfc640 1 --2- 172.21.15.39:0/4066623220 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f38b81051c0 0x7f38b81105c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:23.511 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:23.503+0000 7f38bccfc640 1 -- 172.21.15.39:0/4066623220 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f38b81a87a0 con 0x7f38b81031f0 2024-09-21T03:52:23.511 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:23.503+0000 7f38bccfc640 1 --2- 172.21.15.39:0/4066623220 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f38b81031f0 0x7f38b8110080 secure :-1 s=READY pgs=960 cs=0 l=1 rev1=1 crypto rx=0x7f38ac005940 tx=0x7f38ac0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:52:23.512 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:23.504+0000 7f38b5ffb640 1 -- 172.21.15.39:0/4066623220 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38ac03f030 con 0x7f38b81031f0 2024-09-21T03:52:23.512 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:23.504+0000 7f38bef60640 1 -- 172.21.15.39:0/4066623220 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f38b81a8a30 con 0x7f38b81031f0 2024-09-21T03:52:23.512 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:23.504+0000 7f38b5ffb640 1 -- 172.21.15.39:0/4066623220 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f38ac003cb0 con 0x7f38b81031f0 2024-09-21T03:52:23.512 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:23.504+0000 7f38b5ffb640 1 -- 172.21.15.39:0/4066623220 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38ac035070 con 0x7f38b81031f0 2024-09-21T03:52:23.512 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:23.504+0000 7f38bef60640 1 -- 172.21.15.39:0/4066623220 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f38b81a8f10 con 0x7f38b81031f0 2024-09-21T03:52:23.512 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:23.507+0000 7f38c54d4740 1 Processor -- start 2024-09-21T03:52:23.517 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:23.507+0000 7f38c54d4740 1 -- start start 2024-09-21T03:52:23.517 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:23.507+0000 7f38c54d4740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558a30190070 0x558a305376a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:52:23.517 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:23.507+0000 7f38c54d4740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558a301671b0 0x558a30537be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:52:23.517 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:23.507+0000 7f38c54d4740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558a3016f390 0x558a30538120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:52:23.517 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:23.507+0000 7f38c54d4740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x558a304adea0 con 0x558a30190070 2024-09-21T03:52:23.517 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:23.507+0000 7f38c54d4740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x558a3049d2d0 con 0x558a301671b0 2024-09-21T03:52:23.517 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:23.507+0000 7f38c54d4740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x558a304a10e0 con 0x558a3016f390 2024-09-21T03:52:23.518 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:23.507+0000 7f38bd4fd640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558a30190070 0x558a305376a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:52:23.518 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:23.507+0000 7f38bd4fd640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558a30190070 0x558a305376a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:39954/0 (socket says 172.21.15.39:39954) 2024-09-21T03:52:23.518 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:23.507+0000 7f38bd4fd640 1 -- 172.21.15.39:0/2585074491 learned_addr learned my addr 172.21.15.39:0/2585074491 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:52:23.518 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:23.507+0000 7f38b7fff640 1 --2- 172.21.15.39:0/2585074491 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558a3016f390 0x558a30538120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:52:23.518 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:23.508+0000 7f38bd4fd640 1 -- 172.21.15.39:0/2585074491 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558a3016f390 msgr2=0x558a30538120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:52:23.518 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:23.508+0000 7f38bd4fd640 1 --2- 172.21.15.39:0/2585074491 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558a3016f390 0x558a30538120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:23.518 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:23.508+0000 7f38bd4fd640 1 -- 172.21.15.39:0/2585074491 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558a301671b0 msgr2=0x558a30537be0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-21T03:52:23.518 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:23.508+0000 7f38bd4fd640 1 --2- 172.21.15.39:0/2585074491 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558a301671b0 0x558a30537be0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:23.518 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:23.508+0000 7f38bd4fd640 1 -- 172.21.15.39:0/2585074491 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558a301fb650 con 0x558a30190070 2024-09-21T03:52:23.518 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:23.508+0000 7f38bd4fd640 1 --2- 172.21.15.39:0/2585074491 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558a30190070 0x558a305376a0 secure :-1 s=READY pgs=961 cs=0 l=1 rev1=1 crypto rx=0x7f38b0009f70 tx=0x7f38b0010800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f93fabd8215b47e3 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:52:23.518 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:23.508+0000 7f38a2ffd640 1 -- 172.21.15.39:0/2585074491 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38b0016070 con 0x558a30190070 2024-09-21T03:52:23.518 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:23.508+0000 7f38a2ffd640 1 -- 172.21.15.39:0/2585074491 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f38b0004d20 con 0x558a30190070 2024-09-21T03:52:23.518 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:23.508+0000 7f38a2ffd640 1 -- 172.21.15.39:0/2585074491 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38b0012850 con 0x558a30190070 2024-09-21T03:52:23.519 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:23.508+0000 7f38b5ffb640 1 -- 172.21.15.39:0/4066623220 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 27) ==== 154643+0+0 (secure 0 0 0) 0x7f38ac042030 con 0x7f38b81031f0 2024-09-21T03:52:23.519 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:23.509+0000 7f38c54d4740 1 -- 172.21.15.39:0/2585074491 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558a30190070 msgr2=0x558a305376a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:52:23.519 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:23.509+0000 7f38c54d4740 1 --2- 172.21.15.39:0/2585074491 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558a30190070 0x558a305376a0 secure :-1 s=READY pgs=961 cs=0 l=1 rev1=1 crypto rx=0x7f38b0009f70 tx=0x7f38b0010800 comp rx=0 tx=0).stop 2024-09-21T03:52:23.519 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:23.509+0000 7f38c54d4740 1 -- 172.21.15.39:0/2585074491 shutdown_connections 2024-09-21T03:52:23.519 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:23.509+0000 7f38c54d4740 1 --2- 172.21.15.39:0/2585074491 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558a3016f390 0x558a30538120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:23.519 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:23.509+0000 7f38c54d4740 1 --2- 172.21.15.39:0/2585074491 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558a301671b0 0x558a30537be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:23.519 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:23.509+0000 7f38c54d4740 1 --2- 172.21.15.39:0/2585074491 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558a30190070 0x558a305376a0 unknown :-1 s=CLOSED pgs=961 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:23.519 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:23.509+0000 7f38c54d4740 1 -- 172.21.15.39:0/2585074491 >> 172.21.15.39:0/2585074491 conn(0x558a30184460 msgr2=0x558a30535590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:52:23.519 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:23.509+0000 7f38c54d4740 1 -- 172.21.15.39:0/2585074491 shutdown_connections 2024-09-21T03:52:23.519 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:23.509+0000 7f38c54d4740 1 -- 172.21.15.39:0/2585074491 wait complete. 2024-09-21T03:52:23.519 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:23.510+0000 7f38b5ffb640 1 --2- 172.21.15.39:0/4066623220 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f38a80b6610 0x7f38a80b8b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:52:23.519 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:23.510+0000 7f38c54d4740 1 -- start start 2024-09-21T03:52:23.519 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:23.510+0000 7f38bd4fd640 1 --2- 172.21.15.39:0/4066623220 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f38a80b6610 0x7f38a80b8b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:52:23.519 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:23.510+0000 7f38b5ffb640 1 -- 172.21.15.39:0/4066623220 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8484+0+0 (secure 0 0 0) 0x7f38ac107bf0 con 0x7f38b81031f0 2024-09-21T03:52:23.519 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:23.510+0000 7f38c54d4740 1 Processor -- start 2024-09-21T03:52:23.519 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:23.511+0000 7f38bd4fd640 1 --2- 172.21.15.39:0/4066623220 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f38a80b6610 0x7f38a80b8b20 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f38b0009d30 tx=0x7f38b0010fd0 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:52:23.520 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:23.511+0000 7f38c54d4740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558a301671b0 0x558a3054b5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:52:23.520 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:23.511+0000 7f38c54d4740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558a3016f390 0x558a3054bb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:52:23.520 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:23.511+0000 7f38c54d4740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558a30190070 0x558a30557970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:52:23.520 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:23.511+0000 7f38c54d4740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x558a302fe700 con 0x558a3016f390 2024-09-21T03:52:23.520 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:23.511+0000 7f38c54d4740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x558a30438160 con 0x558a30190070 2024-09-21T03:52:23.520 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:23.511+0000 7f38c54d4740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x558a3054e5b0 con 0x558a301671b0 2024-09-21T03:52:23.520 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:23.511+0000 7f38c54d4740 10 client.0 _ll_register_callbacks cb 0x558a304519e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:52:23.520 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:23.511+0000 7f38b7fff640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558a3016f390 0x558a3054bb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:52:23.520 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:23.511+0000 7f38b7fff640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558a3016f390 0x558a3054bb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:39970/0 (socket says 172.21.15.39:39970) 2024-09-21T03:52:23.520 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:23.511+0000 7f38b7fff640 1 -- 172.21.15.39:0/2917481049 learned_addr learned my addr 172.21.15.39:0/2917481049 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:52:23.520 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:23.511+0000 7f38bd4fd640 1 --2- 172.21.15.39:0/2917481049 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558a30190070 0x558a30557970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:52:23.520 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:23.511+0000 7f38bccfc640 1 --2- 172.21.15.39:0/2917481049 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558a301671b0 0x558a3054b5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:52:23.520 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:23.511+0000 7f38b7fff640 1 -- 172.21.15.39:0/2917481049 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558a301671b0 msgr2=0x558a3054b5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:52:23.520 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:23.511+0000 7f38b7fff640 1 --2- 172.21.15.39:0/2917481049 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558a301671b0 0x558a3054b5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:23.521 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:23.511+0000 7f38b7fff640 1 -- 172.21.15.39:0/2917481049 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558a30190070 msgr2=0x558a30557970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:52:23.521 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:23.511+0000 7f38b7fff640 1 --2- 172.21.15.39:0/2917481049 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558a30190070 0x558a30557970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:23.521 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:23.511+0000 7f38b7fff640 1 -- 172.21.15.39:0/2917481049 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558a30465c90 con 0x558a3016f390 2024-09-21T03:52:23.521 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:23.511+0000 7f38bccfc640 1 --2- 172.21.15.39:0/2917481049 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558a301671b0 0x558a3054b5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:52:23.521 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:23.511+0000 7f38bd4fd640 1 --2- 172.21.15.39:0/2917481049 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558a30190070 0x558a30557970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:52:23.521 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:23.511+0000 7f38b7fff640 1 --2- 172.21.15.39:0/2917481049 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558a3016f390 0x558a3054bb00 secure :-1 s=READY pgs=962 cs=0 l=1 rev1=1 crypto rx=0x7f38a400bf50 tx=0x7f38a40118d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=38a2f86f443d9c20 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:52:23.521 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:23.511+0000 7f3897fff640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-09-21T03:52:23.521 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:23.512+0000 7f3897fff640 1 -- 172.21.15.39:0/2917481049 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38a4004020 con 0x558a3016f390 2024-09-21T03:52:23.521 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:23.512+0000 7f3897fff640 1 -- 172.21.15.39:0/2917481049 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8484+0+0 (secure 0 0 0) 0x7f38a40041c0 con 0x558a3016f390 2024-09-21T03:52:23.521 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:23.512+0000 7f38c54d4740 1 -- 172.21.15.39:0/2917481049 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f38b0004d20 con 0x558a3016f390 2024-09-21T03:52:23.521 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:23.512+0000 7f3897fff640 1 -- 172.21.15.39:0/2917481049 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f38a40043c0 con 0x558a3016f390 2024-09-21T03:52:23.521 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:23.512+0000 7f38c54d4740 10 client.17163 fetch_fsmap learned FSMap version 29 2024-09-21T03:52:23.521 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:23.512+0000 7f38c54d4740 1 -- 172.21.15.39:0/2917481049 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x558a3055a3c0 con 0x558a3016f390 2024-09-21T03:52:23.521 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:23.512+0000 7f3897fff640 1 -- 172.21.15.39:0/2917481049 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f38b0004d20 con 0x558a3016f390 2024-09-21T03:52:23.521 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:23.512+0000 7f3897fff640 1 -- 172.21.15.39:0/2917481049 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f38a4021030 con 0x558a3016f390 2024-09-21T03:52:23.522 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:23.512+0000 7f38c54d4740 10 client.17163 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T03:52:23.522 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:23.512+0000 7f38c54d4740 10 client.17163 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.a 2024-09-21T03:52:23.522 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:23.512+0000 7f38c54d4740 20 client.17163 populate_metadata read hostname 'smithi039' 2024-09-21T03:52:23.522 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:23.513+0000 7f38c54d4740 1 --2- 172.21.15.39:0/2917481049 >> [v2:172.21.15.39:6840/4247190456,v1:172.21.15.39:6841/4247190456] conn(0x558a3014e9f0 0x558a3055d9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:52:23.522 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:23.513+0000 7f38c54d4740 4 client.17163 mds_command: new command op to 14868 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:52:23.522 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:23.513+0000 7f38c54d4740 1 -- 172.21.15.39:0/2917481049 --> [v2:172.21.15.39:6840/4247190456,v1:172.21.15.39:6841/4247190456] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558a304afdf0 con 0x558a3014e9f0 2024-09-21T03:52:23.522 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:23.513+0000 7f38bccfc640 1 --2- 172.21.15.39:0/2917481049 >> [v2:172.21.15.39:6840/4247190456,v1:172.21.15.39:6841/4247190456] conn(0x558a3014e9f0 0x558a3055d9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:52:23.522 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:23.513+0000 7f3897fff640 10 client.17163 ms_handle_connect on v2:172.21.15.39:6840/4247190456 2024-09-21T03:52:23.522 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:23.513+0000 7f38bccfc640 1 --2- 172.21.15.39:0/2917481049 >> [v2:172.21.15.39:6840/4247190456,v1:172.21.15.39:6841/4247190456] conn(0x558a3014e9f0 0x558a3055d9b0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=8811c7636e2de461 server_cookie=771ae22f2f8572a6 in_seq=0 out_seq=0 2024-09-21T03:52:23.522 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:23.514+0000 7f3897fff640 1 -- 172.21.15.39:0/2917481049 <== mds.4 v2:172.21.15.39:6840/4247190456 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x558a304afdf0 con 0x558a3014e9f0 2024-09-21T03:52:23.522 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:23.514+0000 7f3897fff640 10 client.17163 handle_command_reply: tid=0 2024-09-21T03:52:23.522 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:23.514+0000 7f38c54d4740 1 client.17163 shutdown 2024-09-21T03:52:23.522 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:23.514+0000 7f38c54d4740 1 -- 172.21.15.39:0/2917481049 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558a3016f390 msgr2=0x558a3054bb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:52:23.523 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:23.514+0000 7f38c54d4740 1 --2- 172.21.15.39:0/2917481049 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558a3016f390 0x558a3054bb00 secure :-1 s=READY pgs=962 cs=0 l=1 rev1=1 crypto rx=0x7f38a400bf50 tx=0x7f38a40118d0 comp rx=0 tx=0).stop 2024-09-21T03:52:23.523 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:23.514+0000 7f38c54d4740 1 -- 172.21.15.39:0/2917481049 shutdown_connections 2024-09-21T03:52:23.523 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:23.514+0000 7f38c54d4740 1 --2- 172.21.15.39:0/2917481049 >> [v2:172.21.15.39:6840/4247190456,v1:172.21.15.39:6841/4247190456] conn(0x558a3014e9f0 0x558a3055d9b0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:23.523 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:23.514+0000 7f38c54d4740 1 --2- 172.21.15.39:0/2917481049 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558a30190070 0x558a30557970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:23.523 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:23.514+0000 7f38c54d4740 1 --2- 172.21.15.39:0/2917481049 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558a3016f390 0x558a3054bb00 unknown :-1 s=CLOSED pgs=962 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:23.523 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:23.514+0000 7f38c54d4740 1 --2- 172.21.15.39:0/2917481049 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558a301671b0 0x558a3054b5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:23.523 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:23.514+0000 7f3897fff640 0 client.17163 ms_handle_reset on v2:172.21.15.39:6840/4247190456 2024-09-21T03:52:23.523 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:23.514+0000 7f38c54d4740 1 -- 172.21.15.39:0/2917481049 >> 172.21.15.39:0/2917481049 conn(0x558a30184460 msgr2=0x558a3053a800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:52:23.523 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:23.515+0000 7f38c54d4740 1 -- 172.21.15.39:0/2917481049 shutdown_connections 2024-09-21T03:52:23.523 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:23.515+0000 7f38c54d4740 1 -- 172.21.15.39:0/2917481049 wait complete. 2024-09-21T03:52:23.523 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:23.515+0000 7f38c54d4740 20 client.17163 trim_cache size 0 max 16384 2024-09-21T03:52:23.531 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:23.529+0000 7f38c54d4740 1 Processor -- start 2024-09-21T03:52:23.531 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:23.529+0000 7f38c54d4740 1 -- start start 2024-09-21T03:52:23.532 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:23.529+0000 7f38c54d4740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558a301671b0 0x558a3054d4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:52:23.532 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:23.529+0000 7f38c54d4740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558a3016f390 0x558a30538de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:52:23.532 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:23.529+0000 7f38c54d4740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558a30190070 0x558a30534bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:52:23.532 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:23.529+0000 7f38c54d4740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f38b81ad4b0 con 0x558a3016f390 2024-09-21T03:52:23.532 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:23.529+0000 7f38c54d4740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f38b81aeff0 con 0x558a301671b0 2024-09-21T03:52:23.532 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:23.529+0000 7f38c54d4740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f38b81adb80 con 0x558a30190070 2024-09-21T03:52:23.532 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:23.529+0000 7f38b7fff640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558a3016f390 0x558a30538de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:52:23.532 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:23.529+0000 7f38b7fff640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558a3016f390 0x558a30538de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:39984/0 (socket says 172.21.15.39:39984) 2024-09-21T03:52:23.532 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:23.529+0000 7f38b7fff640 1 -- 172.21.15.39:0/3732995022 learned_addr learned my addr 172.21.15.39:0/3732995022 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:52:23.532 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:23.529+0000 7f38bd4fd640 1 --2- 172.21.15.39:0/3732995022 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558a30190070 0x558a30534bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:52:23.532 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:23.529+0000 7f38bccfc640 1 --2- 172.21.15.39:0/3732995022 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558a301671b0 0x558a3054d4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:52:23.532 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:23.530+0000 7f38b7fff640 1 -- 172.21.15.39:0/3732995022 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558a30190070 msgr2=0x558a30534bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:52:23.533 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:23.530+0000 7f38b7fff640 1 --2- 172.21.15.39:0/3732995022 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558a30190070 0x558a30534bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:23.533 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:23.530+0000 7f38b7fff640 1 -- 172.21.15.39:0/3732995022 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558a301671b0 msgr2=0x558a3054d4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:52:23.533 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:23.530+0000 7f38b7fff640 1 --2- 172.21.15.39:0/3732995022 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558a301671b0 0x558a3054d4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:23.533 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:23.530+0000 7f38b7fff640 1 -- 172.21.15.39:0/3732995022 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558a30535f10 con 0x558a3016f390 2024-09-21T03:52:23.533 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:23.530+0000 7f38b7fff640 1 --2- 172.21.15.39:0/3732995022 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558a3016f390 0x558a30538de0 secure :-1 s=READY pgs=963 cs=0 l=1 rev1=1 crypto rx=0x7f38a4001eb0 tx=0x7f38a4005850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da83cb001afd6047 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:52:23.533 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:23.530+0000 7f38a27fc640 1 -- 172.21.15.39:0/3732995022 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38a4000e60 con 0x558a3016f390 2024-09-21T03:52:23.533 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:23.530+0000 7f38a27fc640 1 -- 172.21.15.39:0/3732995022 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f38a4004160 con 0x558a3016f390 2024-09-21T03:52:23.533 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:23.530+0000 7f38a27fc640 1 -- 172.21.15.39:0/3732995022 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38a4010700 con 0x558a3016f390 2024-09-21T03:52:23.536 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:23.531+0000 7f38c54d4740 1 -- 172.21.15.39:0/3732995022 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558a3016f390 msgr2=0x558a30538de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:52:23.536 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:23.531+0000 7f38c54d4740 1 --2- 172.21.15.39:0/3732995022 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558a3016f390 0x558a30538de0 secure :-1 s=READY pgs=963 cs=0 l=1 rev1=1 crypto rx=0x7f38a4001eb0 tx=0x7f38a4005850 comp rx=0 tx=0).stop 2024-09-21T03:52:23.536 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:23.531+0000 7f38c54d4740 1 -- 172.21.15.39:0/3732995022 shutdown_connections 2024-09-21T03:52:23.536 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:23.531+0000 7f38c54d4740 1 --2- 172.21.15.39:0/3732995022 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558a30190070 0x558a30534bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:23.536 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:23.531+0000 7f38c54d4740 1 --2- 172.21.15.39:0/3732995022 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558a3016f390 0x558a30538de0 unknown :-1 s=CLOSED pgs=963 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:23.536 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:23.531+0000 7f38c54d4740 1 --2- 172.21.15.39:0/3732995022 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558a301671b0 0x558a3054d4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:23.536 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:23.531+0000 7f38c54d4740 1 -- 172.21.15.39:0/3732995022 >> 172.21.15.39:0/3732995022 conn(0x558a30184460 msgr2=0x558a305386d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:52:23.536 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:23.531+0000 7f38c54d4740 1 -- 172.21.15.39:0/3732995022 shutdown_connections 2024-09-21T03:52:23.536 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:23.531+0000 7f38c54d4740 1 -- 172.21.15.39:0/3732995022 wait complete. 2024-09-21T03:52:23.537 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:23.531+0000 7f38c54d4740 1 -- start start 2024-09-21T03:52:23.537 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:23.531+0000 7f38c54d4740 1 Processor -- start 2024-09-21T03:52:23.537 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:23.532+0000 7f38c54d4740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558a301671b0 0x558a3054f980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:52:23.537 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:23.532+0000 7f38c54d4740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558a3016f390 0x558a3054fec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:52:23.537 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:23.532+0000 7f38c54d4740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558a30190070 0x558a30550400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:52:23.537 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:23.532+0000 7f38c54d4740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f38b81ae920 con 0x558a30190070 2024-09-21T03:52:23.537 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:23.532+0000 7f38c54d4740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f38b81ae250 con 0x558a3016f390 2024-09-21T03:52:23.537 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:23.532+0000 7f38c54d4740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f38b81ab490 con 0x558a301671b0 2024-09-21T03:52:23.537 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:23.532+0000 7f38c54d4740 10 client.0 _ll_register_callbacks cb 0x7f38a400bb90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:52:23.537 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:23.532+0000 7f38bd4fd640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558a30190070 0x558a30550400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:52:23.537 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:23.532+0000 7f38bd4fd640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558a30190070 0x558a30550400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:39994/0 (socket says 172.21.15.39:39994) 2024-09-21T03:52:23.537 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:23.532+0000 7f38bd4fd640 1 -- 172.21.15.39:0/3320048708 learned_addr learned my addr 172.21.15.39:0/3320048708 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:52:23.537 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:23.532+0000 7f38bccfc640 1 --2- 172.21.15.39:0/3320048708 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558a301671b0 0x558a3054f980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:52:23.537 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:23.532+0000 7f38b7fff640 1 --2- 172.21.15.39:0/3320048708 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558a3016f390 0x558a3054fec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:52:23.538 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:23.532+0000 7f38bd4fd640 1 -- 172.21.15.39:0/3320048708 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558a301671b0 msgr2=0x558a3054f980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:52:23.538 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:23.532+0000 7f38bd4fd640 1 --2- 172.21.15.39:0/3320048708 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558a301671b0 0x558a3054f980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:23.538 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:23.532+0000 7f38bd4fd640 1 -- 172.21.15.39:0/3320048708 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558a3016f390 msgr2=0x558a3054fec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:52:23.538 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:23.532+0000 7f38bd4fd640 1 --2- 172.21.15.39:0/3320048708 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558a3016f390 0x558a3054fec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:23.538 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:23.532+0000 7f38bd4fd640 1 -- 172.21.15.39:0/3320048708 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558a305c7480 con 0x558a30190070 2024-09-21T03:52:23.538 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:23.532+0000 7f38b7fff640 1 --2- 172.21.15.39:0/3320048708 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558a3016f390 0x558a3054fec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:52:23.538 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:23.533+0000 7f38bd4fd640 1 --2- 172.21.15.39:0/3320048708 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558a30190070 0x558a30550400 secure :-1 s=READY pgs=964 cs=0 l=1 rev1=1 crypto rx=0x7f38b0009f70 tx=0x7f38b00124f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c06bd812ced8157 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:52:23.538 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:23.533+0000 7f3897fff640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-09-21T03:52:23.538 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:23.533+0000 7f3897fff640 1 -- 172.21.15.39:0/3320048708 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38b0013dd0 con 0x558a30190070 2024-09-21T03:52:23.538 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:23.533+0000 7f3897fff640 1 -- 172.21.15.39:0/3320048708 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8484+0+0 (secure 0 0 0) 0x7f38b0021070 con 0x558a30190070 2024-09-21T03:52:23.538 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:23.533+0000 7f38c54d4740 1 -- 172.21.15.39:0/3320048708 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f38a4004160 con 0x558a30190070 2024-09-21T03:52:23.538 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:23.533+0000 7f3897fff640 1 -- 172.21.15.39:0/3320048708 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f38b0004030 con 0x558a30190070 2024-09-21T03:52:23.538 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:23.533+0000 7f38c54d4740 10 client.17169 fetch_fsmap learned FSMap version 29 2024-09-21T03:52:23.538 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:23.533+0000 7f38c54d4740 1 -- 172.21.15.39:0/3320048708 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x558a305c8700 con 0x558a30190070 2024-09-21T03:52:23.539 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:23.533+0000 7f3897fff640 1 -- 172.21.15.39:0/3320048708 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f38a4004160 con 0x558a30190070 2024-09-21T03:52:23.539 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:23.533+0000 7f3897fff640 1 -- 172.21.15.39:0/3320048708 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f38b0025040 con 0x558a30190070 2024-09-21T03:52:23.539 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:23.534+0000 7f38c54d4740 10 client.17169 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T03:52:23.539 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:23.534+0000 7f38c54d4740 10 client.17169 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.a 2024-09-21T03:52:23.539 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:23.534+0000 7f38c54d4740 20 client.17169 populate_metadata read hostname 'smithi039' 2024-09-21T03:52:23.539 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:23.534+0000 7f38c54d4740 1 --2- 172.21.15.39:0/3320048708 >> [v2:172.21.15.39:6840/4247190456,v1:172.21.15.39:6841/4247190456] conn(0x558a30123da0 0x558a305cad50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:52:23.539 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:23.534+0000 7f38c54d4740 4 client.17169 mds_command: new command op to 14868 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T03:52:23.539 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:23.534+0000 7f38c54d4740 1 -- 172.21.15.39:0/3320048708 --> [v2:172.21.15.39:6840/4247190456,v1:172.21.15.39:6841/4247190456] -- command(tid 0: {"prefix": "damage ls"}) -- 0x558a304ab8a0 con 0x558a30123da0 2024-09-21T03:52:23.539 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:23.534+0000 7f38bccfc640 1 --2- 172.21.15.39:0/3320048708 >> [v2:172.21.15.39:6840/4247190456,v1:172.21.15.39:6841/4247190456] conn(0x558a30123da0 0x558a305cad50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:52:23.539 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:23.534+0000 7f3897fff640 10 client.17169 ms_handle_connect on v2:172.21.15.39:6840/4247190456 2024-09-21T03:52:23.539 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:23.534+0000 7f38bccfc640 1 --2- 172.21.15.39:0/3320048708 >> [v2:172.21.15.39:6840/4247190456,v1:172.21.15.39:6841/4247190456] conn(0x558a30123da0 0x558a305cad50 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=bfe6ab79026f8fa8 server_cookie=65f269a88428059 in_seq=0 out_seq=0 2024-09-21T03:52:23.539 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:23.534+0000 7f3897fff640 1 -- 172.21.15.39:0/3320048708 <== mds.4 v2:172.21.15.39:6840/4247190456 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x558a304ab8a0 con 0x558a30123da0 2024-09-21T03:52:23.539 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:23.534+0000 7f3897fff640 10 client.17169 handle_command_reply: tid=0 2024-09-21T03:52:23.539 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:23.534+0000 7f38c54d4740 1 client.17169 shutdown 2024-09-21T03:52:23.540 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:23.535+0000 7f38c54d4740 1 -- 172.21.15.39:0/3320048708 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558a30190070 msgr2=0x558a30550400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:52:23.540 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:23.535+0000 7f38c54d4740 1 --2- 172.21.15.39:0/3320048708 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558a30190070 0x558a30550400 secure :-1 s=READY pgs=964 cs=0 l=1 rev1=1 crypto rx=0x7f38b0009f70 tx=0x7f38b00124f0 comp rx=0 tx=0).stop 2024-09-21T03:52:23.540 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:23.535+0000 7f38c54d4740 1 -- 172.21.15.39:0/3320048708 shutdown_connections 2024-09-21T03:52:23.540 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:23.535+0000 7f38c54d4740 1 --2- 172.21.15.39:0/3320048708 >> [v2:172.21.15.39:6840/4247190456,v1:172.21.15.39:6841/4247190456] conn(0x558a30123da0 0x558a305cad50 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:23.541 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:23.535+0000 7f38c54d4740 1 --2- 172.21.15.39:0/3320048708 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558a30190070 0x558a30550400 unknown :-1 s=CLOSED pgs=964 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:23.541 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:23.535+0000 7f38c54d4740 1 --2- 172.21.15.39:0/3320048708 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558a3016f390 0x558a3054fec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:23.541 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:23.535+0000 7f38c54d4740 1 --2- 172.21.15.39:0/3320048708 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558a301671b0 0x558a3054f980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:23.541 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:23.535+0000 7f38c54d4740 1 -- 172.21.15.39:0/3320048708 >> 172.21.15.39:0/3320048708 conn(0x558a30184460 msgr2=0x558a30538e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:52:23.541 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:23.535+0000 7f3897fff640 0 client.17169 ms_handle_reset on v2:172.21.15.39:6840/4247190456 2024-09-21T03:52:23.541 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:23.535+0000 7f38c54d4740 1 -- 172.21.15.39:0/3320048708 shutdown_connections 2024-09-21T03:52:23.541 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:23.535+0000 7f38c54d4740 1 -- 172.21.15.39:0/3320048708 wait complete. 2024-09-21T03:52:23.541 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:23.535+0000 7f38c54d4740 20 client.17169 trim_cache size 0 max 16384 2024-09-21T03:52:23.541 INFO:teuthology.orchestra.run.smithi039.stdout:[] 2024-09-21T03:52:23.542 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:23.536+0000 7f38bef60640 1 -- 172.21.15.39:0/4066623220 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f38a80b6610 msgr2=0x7f38a80b8b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:52:23.542 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:23.536+0000 7f38bef60640 1 --2- 172.21.15.39:0/4066623220 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f38a80b6610 0x7f38a80b8b20 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f38b0009d30 tx=0x7f38b0010fd0 comp rx=0 tx=0).stop 2024-09-21T03:52:23.542 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:23.536+0000 7f38bef60640 1 -- 172.21.15.39:0/4066623220 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f38b81031f0 msgr2=0x7f38b8110080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:52:23.542 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:23.536+0000 7f38bef60640 1 --2- 172.21.15.39:0/4066623220 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f38b81031f0 0x7f38b8110080 secure :-1 s=READY pgs=960 cs=0 l=1 rev1=1 crypto rx=0x7f38ac005940 tx=0x7f38ac0042f0 comp rx=0 tx=0).stop 2024-09-21T03:52:23.542 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:23.536+0000 7f38bef60640 1 -- 172.21.15.39:0/4066623220 shutdown_connections 2024-09-21T03:52:23.542 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:23.536+0000 7f38bef60640 1 --2- 172.21.15.39:0/4066623220 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f38a80b6610 0x7f38a80b8b20 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:23.542 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:23.536+0000 7f38bef60640 1 --2- 172.21.15.39:0/4066623220 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f38b8105b10 0x7f38b8110b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:23.542 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:23.536+0000 7f38bef60640 1 --2- 172.21.15.39:0/4066623220 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f38b81051c0 0x7f38b81105c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:23.543 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:23.536+0000 7f38bef60640 1 --2- 172.21.15.39:0/4066623220 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f38b81031f0 0x7f38b8110080 unknown :-1 s=CLOSED pgs=960 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:23.543 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:23.536+0000 7f38bef60640 1 -- 172.21.15.39:0/4066623220 >> 172.21.15.39:0/4066623220 conn(0x7f38b8079e40 msgr2=0x7f38b81063f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:52:23.543 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:23.536+0000 7f38bef60640 1 -- 172.21.15.39:0/4066623220 shutdown_connections 2024-09-21T03:52:23.543 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:23.536+0000 7f38bef60640 1 -- 172.21.15.39:0/4066623220 wait complete. 2024-09-21T03:52:23.718 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:52:23 smithi161 ceph-mon[34533]: pgmap v515: 97 pgs: 97 active+clean; 10 MiB data, 520 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 85 B/s wr, 9 op/s 2024-09-21T03:52:23.724 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:52:23 smithi150 ceph-mon[34289]: pgmap v515: 97 pgs: 97 active+clean; 10 MiB data, 520 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 85 B/s wr, 9 op/s 2024-09-21T03:52:24.384 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 51116d5c-77ca-11ef-baf5-efdc52797490 -- ceph tell mds.1:3 damage ls 2024-09-21T03:52:24.641 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/51116d5c-77ca-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:52:25.121 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:25.118+0000 7fdc3a3d6640 1 -- 172.21.15.39:0/2021482317 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdc34103b60 msgr2=0x7fdc34103f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:52:25.121 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:25.118+0000 7fdc3a3d6640 1 --2- 172.21.15.39:0/2021482317 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdc34103b60 0x7fdc34103f70 secure :-1 s=READY pgs=965 cs=0 l=1 rev1=1 crypto rx=0x7fdc24009900 tx=0x7fdc2402a410 comp rx=0 tx=0).stop 2024-09-21T03:52:25.121 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:25.118+0000 7fdc3a3d6640 1 -- 172.21.15.39:0/2021482317 shutdown_connections 2024-09-21T03:52:25.121 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:25.118+0000 7fdc3a3d6640 1 --2- 172.21.15.39:0/2021482317 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdc34109df0 0x7fdc34110620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:25.121 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:25.118+0000 7fdc3a3d6640 1 --2- 172.21.15.39:0/2021482317 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdc341094a0 0x7fdc341098b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:25.121 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:25.118+0000 7fdc3a3d6640 1 --2- 172.21.15.39:0/2021482317 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdc34103b60 0x7fdc34103f70 unknown :-1 s=CLOSED pgs=965 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:25.121 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:25.118+0000 7fdc3a3d6640 1 -- 172.21.15.39:0/2021482317 >> 172.21.15.39:0/2021482317 conn(0x7fdc3407b850 msgr2=0x7fdc3407bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:52:25.121 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:25.118+0000 7fdc3a3d6640 1 -- 172.21.15.39:0/2021482317 shutdown_connections 2024-09-21T03:52:25.122 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:25.118+0000 7fdc3a3d6640 1 -- 172.21.15.39:0/2021482317 wait complete. 2024-09-21T03:52:25.122 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:25.119+0000 7fdc3a3d6640 1 Processor -- start 2024-09-21T03:52:25.122 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:25.119+0000 7fdc3a3d6640 1 -- start start 2024-09-21T03:52:25.124 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:25.119+0000 7fdc3a3d6640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdc34103b60 0x7fdc3419a4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:52:25.124 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:25.119+0000 7fdc3a3d6640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdc341094a0 0x7fdc3419aa40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:52:25.124 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:25.119+0000 7fdc3a3d6640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdc34109df0 0x7fdc3419fce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:52:25.124 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:25.119+0000 7fdc3a3d6640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fdc34113ab0 con 0x7fdc341094a0 2024-09-21T03:52:25.124 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:25.119+0000 7fdc3a3d6640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fdc34113930 con 0x7fdc34103b60 2024-09-21T03:52:25.125 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:25.119+0000 7fdc3a3d6640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fdc34113c30 con 0x7fdc34109df0 2024-09-21T03:52:25.125 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:25.119+0000 7fdc337fe640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdc341094a0 0x7fdc3419aa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:52:25.125 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:25.119+0000 7fdc337fe640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdc341094a0 0x7fdc3419aa40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:39998/0 (socket says 172.21.15.39:39998) 2024-09-21T03:52:25.125 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:25.120+0000 7fdc337fe640 1 -- 172.21.15.39:0/3673777122 learned_addr learned my addr 172.21.15.39:0/3673777122 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:52:25.125 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:25.120+0000 7fdc38973640 1 --2- 172.21.15.39:0/3673777122 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdc34109df0 0x7fdc3419fce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:52:25.125 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:25.120+0000 7fdc33fff640 1 --2- 172.21.15.39:0/3673777122 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdc34103b60 0x7fdc3419a4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:52:25.126 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:25.120+0000 7fdc337fe640 1 -- 172.21.15.39:0/3673777122 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdc34109df0 msgr2=0x7fdc3419fce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:52:25.126 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:25.120+0000 7fdc337fe640 1 --2- 172.21.15.39:0/3673777122 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdc34109df0 0x7fdc3419fce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:25.126 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:25.120+0000 7fdc337fe640 1 -- 172.21.15.39:0/3673777122 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdc34103b60 msgr2=0x7fdc3419a4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:52:25.126 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:25.120+0000 7fdc337fe640 1 --2- 172.21.15.39:0/3673777122 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdc34103b60 0x7fdc3419a4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:25.127 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:25.120+0000 7fdc337fe640 1 -- 172.21.15.39:0/3673777122 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdc341a0410 con 0x7fdc341094a0 2024-09-21T03:52:25.127 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:25.120+0000 7fdc33fff640 1 --2- 172.21.15.39:0/3673777122 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdc34103b60 0x7fdc3419a4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:52:25.127 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:25.120+0000 7fdc337fe640 1 --2- 172.21.15.39:0/3673777122 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdc341094a0 0x7fdc3419aa40 secure :-1 s=READY pgs=966 cs=0 l=1 rev1=1 crypto rx=0x7fdc2800d740 tx=0x7fdc2800dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:52:25.127 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:25.120+0000 7fdc317fa640 1 -- 172.21.15.39:0/3673777122 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc28017070 con 0x7fdc341094a0 2024-09-21T03:52:25.127 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:25.120+0000 7fdc317fa640 1 -- 172.21.15.39:0/3673777122 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdc2800de50 con 0x7fdc341094a0 2024-09-21T03:52:25.127 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:25.120+0000 7fdc3a3d6640 1 -- 172.21.15.39:0/3673777122 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdc341a0700 con 0x7fdc341094a0 2024-09-21T03:52:25.128 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:25.120+0000 7fdc317fa640 1 -- 172.21.15.39:0/3673777122 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc2800f620 con 0x7fdc341094a0 2024-09-21T03:52:25.128 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:25.120+0000 7fdc3a3d6640 1 -- 172.21.15.39:0/3673777122 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdc341a6e60 con 0x7fdc341094a0 2024-09-21T03:52:25.137 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:25.123+0000 7fdc4094a740 1 Processor -- start 2024-09-21T03:52:25.137 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:25.123+0000 7fdc4094a740 1 -- start start 2024-09-21T03:52:25.137 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:25.123+0000 7fdc317fa640 1 -- 172.21.15.39:0/3673777122 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 27) ==== 154643+0+0 (secure 0 0 0) 0x7fdc2801a070 con 0x7fdc341094a0 2024-09-21T03:52:25.137 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:25.123+0000 7fdc4094a740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a6c5f8d070 0x55a6c63346a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:52:25.137 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:25.123+0000 7fdc4094a740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a6c5f641b0 0x55a6c6334be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:52:25.137 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:25.123+0000 7fdc4094a740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a6c5f6c390 0x55a6c6335120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:52:25.137 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:25.123+0000 7fdc4094a740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x55a6c62aaea0 con 0x55a6c5f6c390 2024-09-21T03:52:25.137 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:25.123+0000 7fdc4094a740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x55a6c629a2d0 con 0x55a6c5f641b0 2024-09-21T03:52:25.137 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:25.123+0000 7fdc4094a740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55a6c629e0e0 con 0x55a6c5f8d070 2024-09-21T03:52:25.137 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:25.123+0000 7fdc337fe640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a6c5f6c390 0x55a6c6335120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:52:25.137 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:25.124+0000 7fdc337fe640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a6c5f6c390 0x55a6c6335120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:40000/0 (socket says 172.21.15.39:40000) 2024-09-21T03:52:25.138 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:25.124+0000 7fdc337fe640 1 -- 172.21.15.39:0/4252067969 learned_addr learned my addr 172.21.15.39:0/4252067969 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:52:25.138 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:25.124+0000 7fdc38973640 1 --2- 172.21.15.39:0/4252067969 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a6c5f8d070 0x55a6c63346a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:52:25.138 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:25.124+0000 7fdc33fff640 1 --2- 172.21.15.39:0/4252067969 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a6c5f641b0 0x55a6c6334be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:52:25.138 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:25.124+0000 7fdc337fe640 1 -- 172.21.15.39:0/4252067969 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a6c5f8d070 msgr2=0x55a6c63346a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:52:25.138 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:25.124+0000 7fdc337fe640 1 --2- 172.21.15.39:0/4252067969 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a6c5f8d070 0x55a6c63346a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:25.138 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:25.124+0000 7fdc337fe640 1 -- 172.21.15.39:0/4252067969 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a6c5f641b0 msgr2=0x55a6c6334be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:52:25.138 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:25.124+0000 7fdc337fe640 1 --2- 172.21.15.39:0/4252067969 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a6c5f641b0 0x55a6c6334be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:25.138 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:25.124+0000 7fdc337fe640 1 -- 172.21.15.39:0/4252067969 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a6c5ff8650 con 0x55a6c5f6c390 2024-09-21T03:52:25.138 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:25.124+0000 7fdc33fff640 1 --2- 172.21.15.39:0/4252067969 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a6c5f641b0 0x55a6c6334be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:52:25.138 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:25.124+0000 7fdc317fa640 1 --2- 172.21.15.39:0/3673777122 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7fdc1c0b6680 0x7fdc1c0b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:52:25.138 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:25.124+0000 7fdc337fe640 1 --2- 172.21.15.39:0/4252067969 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a6c5f6c390 0x55a6c6335120 secure :-1 s=READY pgs=967 cs=0 l=1 rev1=1 crypto rx=0x7fdc280e4810 tx=0x7fdc280ef910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=45abf602a299b249 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:52:25.138 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:25.124+0000 7fdc38973640 1 --2- 172.21.15.39:0/3673777122 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7fdc1c0b6680 0x7fdc1c0b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:52:25.138 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:25.124+0000 7fdc317fa640 1 -- 172.21.15.39:0/3673777122 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8484+0+0 (secure 0 0 0) 0x7fdc280e7130 con 0x7fdc341094a0 2024-09-21T03:52:25.138 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:25.124+0000 7fdc1a7fc640 1 -- 172.21.15.39:0/4252067969 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc280a95f0 con 0x55a6c5f6c390 2024-09-21T03:52:25.139 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:25.125+0000 7fdc1a7fc640 1 -- 172.21.15.39:0/4252067969 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdc280acab0 con 0x55a6c5f6c390 2024-09-21T03:52:25.139 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:25.125+0000 7fdc1a7fc640 1 -- 172.21.15.39:0/4252067969 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc280ac710 con 0x55a6c5f6c390 2024-09-21T03:52:25.139 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:25.125+0000 7fdc4094a740 1 -- 172.21.15.39:0/4252067969 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a6c5f6c390 msgr2=0x55a6c6335120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:52:25.139 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:25.125+0000 7fdc38973640 1 --2- 172.21.15.39:0/3673777122 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7fdc1c0b6680 0x7fdc1c0b8b90 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fdc2c005fd0 tx=0x7fdc2c009600 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:52:25.139 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:25.125+0000 7fdc4094a740 1 --2- 172.21.15.39:0/4252067969 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a6c5f6c390 0x55a6c6335120 secure :-1 s=READY pgs=967 cs=0 l=1 rev1=1 crypto rx=0x7fdc280e4810 tx=0x7fdc280ef910 comp rx=0 tx=0).stop 2024-09-21T03:52:25.139 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:25.125+0000 7fdc4094a740 1 -- 172.21.15.39:0/4252067969 shutdown_connections 2024-09-21T03:52:25.139 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:25.125+0000 7fdc4094a740 1 --2- 172.21.15.39:0/4252067969 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a6c5f6c390 0x55a6c6335120 unknown :-1 s=CLOSED pgs=967 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:25.139 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:25.125+0000 7fdc4094a740 1 --2- 172.21.15.39:0/4252067969 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a6c5f641b0 0x55a6c6334be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:25.139 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:25.125+0000 7fdc4094a740 1 --2- 172.21.15.39:0/4252067969 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a6c5f8d070 0x55a6c63346a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:25.139 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:25.125+0000 7fdc4094a740 1 -- 172.21.15.39:0/4252067969 >> 172.21.15.39:0/4252067969 conn(0x55a6c5f81460 msgr2=0x55a6c6332590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:52:25.139 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:25.126+0000 7fdc4094a740 1 -- 172.21.15.39:0/4252067969 shutdown_connections 2024-09-21T03:52:25.139 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:25.126+0000 7fdc4094a740 1 -- 172.21.15.39:0/4252067969 wait complete. 2024-09-21T03:52:25.139 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:25.126+0000 7fdc4094a740 1 -- start start 2024-09-21T03:52:25.139 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:25.126+0000 7fdc4094a740 1 Processor -- start 2024-09-21T03:52:25.140 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:25.126+0000 7fdc4094a740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a6c5f641b0 0x55a6c633fdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:52:25.140 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:25.127+0000 7fdc4094a740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a6c5f6c390 0x55a6c6340300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:52:25.140 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:25.127+0000 7fdc4094a740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a6c5f8d070 0x55a6c634c1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:52:25.140 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:25.127+0000 7fdc4094a740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x55a6c60fb700 con 0x55a6c5f6c390 2024-09-21T03:52:25.140 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:25.127+0000 7fdc4094a740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x55a6c6235160 con 0x55a6c5f641b0 2024-09-21T03:52:25.140 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:25.127+0000 7fdc4094a740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55a6c6342e80 con 0x55a6c5f8d070 2024-09-21T03:52:25.140 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:25.127+0000 7fdc4094a740 10 client.0 _ll_register_callbacks cb 0x55a6c624e9e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:52:25.140 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:25.127+0000 7fdc337fe640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a6c5f6c390 0x55a6c6340300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:52:25.140 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:25.127+0000 7fdc337fe640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a6c5f6c390 0x55a6c6340300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:40010/0 (socket says 172.21.15.39:40010) 2024-09-21T03:52:25.140 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:25.127+0000 7fdc337fe640 1 -- 172.21.15.39:0/1148047949 learned_addr learned my addr 172.21.15.39:0/1148047949 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:52:25.140 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:25.127+0000 7fdc33fff640 1 --2- 172.21.15.39:0/1148047949 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a6c5f641b0 0x55a6c633fdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:52:25.140 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:25.127+0000 7fdc38973640 1 --2- 172.21.15.39:0/1148047949 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a6c5f8d070 0x55a6c634c1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:52:25.140 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:25.127+0000 7fdc337fe640 1 -- 172.21.15.39:0/1148047949 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a6c5f8d070 msgr2=0x55a6c634c1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:52:25.140 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:25.127+0000 7fdc337fe640 1 --2- 172.21.15.39:0/1148047949 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a6c5f8d070 0x55a6c634c1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:25.141 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:25.127+0000 7fdc337fe640 1 -- 172.21.15.39:0/1148047949 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a6c5f641b0 msgr2=0x55a6c633fdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:52:25.141 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:25.127+0000 7fdc337fe640 1 --2- 172.21.15.39:0/1148047949 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a6c5f641b0 0x55a6c633fdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:25.141 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:25.127+0000 7fdc337fe640 1 -- 172.21.15.39:0/1148047949 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a6c6262c90 con 0x55a6c5f6c390 2024-09-21T03:52:25.141 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:25.127+0000 7fdc38973640 1 --2- 172.21.15.39:0/1148047949 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a6c5f8d070 0x55a6c634c1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:52:25.141 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:25.127+0000 7fdc337fe640 1 --2- 172.21.15.39:0/1148047949 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a6c5f6c390 0x55a6c6340300 secure :-1 s=READY pgs=968 cs=0 l=1 rev1=1 crypto rx=0x7fdc280e4950 tx=0x7fdc28010c70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7096f5554b2f66f3 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:52:25.141 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:25.127+0000 7fdc137fe640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-09-21T03:52:25.141 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:25.127+0000 7fdc137fe640 1 -- 172.21.15.39:0/1148047949 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc28021070 con 0x55a6c5f6c390 2024-09-21T03:52:25.141 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:25.127+0000 7fdc137fe640 1 -- 172.21.15.39:0/1148047949 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8484+0+0 (secure 0 0 0) 0x7fdc280b6530 con 0x55a6c5f6c390 2024-09-21T03:52:25.141 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:25.127+0000 7fdc4094a740 1 -- 172.21.15.39:0/1148047949 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdc280acab0 con 0x55a6c5f6c390 2024-09-21T03:52:25.141 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:25.128+0000 7fdc137fe640 1 -- 172.21.15.39:0/1148047949 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fdc280ac8f0 con 0x55a6c5f6c390 2024-09-21T03:52:25.141 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:25.128+0000 7fdc4094a740 10 client.17181 fetch_fsmap learned FSMap version 29 2024-09-21T03:52:25.141 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:25.128+0000 7fdc4094a740 1 -- 172.21.15.39:0/1148047949 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55a6c634ec20 con 0x55a6c5f6c390 2024-09-21T03:52:25.141 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:25.128+0000 7fdc137fe640 1 -- 172.21.15.39:0/1148047949 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdc280acab0 con 0x55a6c5f6c390 2024-09-21T03:52:25.141 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:25.128+0000 7fdc137fe640 1 -- 172.21.15.39:0/1148047949 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fdc280940d0 con 0x55a6c5f6c390 2024-09-21T03:52:25.142 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:25.128+0000 7fdc4094a740 10 client.17181 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T03:52:25.142 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:25.128+0000 7fdc4094a740 10 client.17181 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-21T03:52:25.142 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:25.128+0000 7fdc4094a740 20 client.17181 populate_metadata read hostname 'smithi039' 2024-09-21T03:52:25.142 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:25.128+0000 7fdc4094a740 1 --2- 172.21.15.39:0/1148047949 >> [v2:172.21.15.161:6836/3451544597,v1:172.21.15.161:6837/3451544597] conn(0x55a6c5f4b9f0 0x55a6c6351f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:52:25.142 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:25.128+0000 7fdc4094a740 4 client.17181 mds_command: new command op to 24646 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:52:25.142 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:25.128+0000 7fdc4094a740 1 -- 172.21.15.39:0/1148047949 --> [v2:172.21.15.161:6836/3451544597,v1:172.21.15.161:6837/3451544597] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a6c62acdf0 con 0x55a6c5f4b9f0 2024-09-21T03:52:25.142 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:25.128+0000 7fdc33fff640 1 --2- 172.21.15.39:0/1148047949 >> [v2:172.21.15.161:6836/3451544597,v1:172.21.15.161:6837/3451544597] conn(0x55a6c5f4b9f0 0x55a6c6351f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:52:25.142 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:25.129+0000 7fdc33fff640 1 --2- 172.21.15.39:0/1148047949 >> [v2:172.21.15.161:6836/3451544597,v1:172.21.15.161:6837/3451544597] conn(0x55a6c5f4b9f0 0x55a6c6351f80 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=891403375d70acb2 server_cookie=8202bd1dfb935593 in_seq=0 out_seq=0 2024-09-21T03:52:25.142 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:25.129+0000 7fdc137fe640 10 client.17181 ms_handle_connect on v2:172.21.15.161:6836/3451544597 2024-09-21T03:52:25.142 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:25.130+0000 7fdc137fe640 1 -- 172.21.15.39:0/1148047949 <== mds.3 v2:172.21.15.161:6836/3451544597 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55a6c62acdf0 con 0x55a6c5f4b9f0 2024-09-21T03:52:25.142 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:25.130+0000 7fdc137fe640 10 client.17181 handle_command_reply: tid=0 2024-09-21T03:52:25.142 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:25.130+0000 7fdc4094a740 1 client.17181 shutdown 2024-09-21T03:52:25.142 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:25.130+0000 7fdc4094a740 1 -- 172.21.15.39:0/1148047949 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a6c5f6c390 msgr2=0x55a6c6340300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:52:25.142 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:25.130+0000 7fdc4094a740 1 --2- 172.21.15.39:0/1148047949 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a6c5f6c390 0x55a6c6340300 secure :-1 s=READY pgs=968 cs=0 l=1 rev1=1 crypto rx=0x7fdc280e4950 tx=0x7fdc28010c70 comp rx=0 tx=0).stop 2024-09-21T03:52:25.143 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:25.130+0000 7fdc4094a740 1 -- 172.21.15.39:0/1148047949 shutdown_connections 2024-09-21T03:52:25.143 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:25.130+0000 7fdc4094a740 1 --2- 172.21.15.39:0/1148047949 >> [v2:172.21.15.161:6836/3451544597,v1:172.21.15.161:6837/3451544597] conn(0x55a6c5f4b9f0 0x55a6c6351f80 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:25.143 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:25.130+0000 7fdc4094a740 1 --2- 172.21.15.39:0/1148047949 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a6c5f8d070 0x55a6c634c1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:25.143 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:25.130+0000 7fdc4094a740 1 --2- 172.21.15.39:0/1148047949 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a6c5f6c390 0x55a6c6340300 unknown :-1 s=CLOSED pgs=968 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:25.143 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:25.130+0000 7fdc4094a740 1 --2- 172.21.15.39:0/1148047949 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a6c5f641b0 0x55a6c633fdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:25.143 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:25.130+0000 7fdc137fe640 0 client.17181 ms_handle_reset on v2:172.21.15.161:6836/3451544597 2024-09-21T03:52:25.143 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:25.130+0000 7fdc4094a740 1 -- 172.21.15.39:0/1148047949 >> 172.21.15.39:0/1148047949 conn(0x55a6c5f81460 msgr2=0x55a6c6337830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:52:25.143 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:25.130+0000 7fdc4094a740 1 -- 172.21.15.39:0/1148047949 shutdown_connections 2024-09-21T03:52:25.143 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:25.130+0000 7fdc4094a740 1 -- 172.21.15.39:0/1148047949 wait complete. 2024-09-21T03:52:25.143 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:25.130+0000 7fdc4094a740 20 client.17181 trim_cache size 0 max 16384 2024-09-21T03:52:25.145 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:25.143+0000 7fdc4094a740 1 Processor -- start 2024-09-21T03:52:25.145 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:25.143+0000 7fdc4094a740 1 -- start start 2024-09-21T03:52:25.145 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:25.144+0000 7fdc4094a740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a6c5f641b0 0x55a6c6341ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:52:25.146 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:25.144+0000 7fdc4094a740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a6c5f6c390 0x55a6c6335de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:52:25.146 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:25.144+0000 7fdc4094a740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a6c5f8d070 0x55a6c634c110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:52:25.146 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:25.144+0000 7fdc4094a740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fdc341af810 con 0x55a6c5f6c390 2024-09-21T03:52:25.146 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:25.144+0000 7fdc4094a740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fdc341b1350 con 0x55a6c5f641b0 2024-09-21T03:52:25.146 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:25.144+0000 7fdc4094a740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fdc341afee0 con 0x55a6c5f8d070 2024-09-21T03:52:25.146 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:25.144+0000 7fdc337fe640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a6c5f6c390 0x55a6c6335de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:52:25.146 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:25.144+0000 7fdc337fe640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a6c5f6c390 0x55a6c6335de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:40016/0 (socket says 172.21.15.39:40016) 2024-09-21T03:52:25.146 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:25.144+0000 7fdc337fe640 1 -- 172.21.15.39:0/1884985065 learned_addr learned my addr 172.21.15.39:0/1884985065 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:52:25.146 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:25.144+0000 7fdc38973640 1 --2- 172.21.15.39:0/1884985065 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a6c5f8d070 0x55a6c634c110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:52:25.147 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:25.144+0000 7fdc337fe640 1 -- 172.21.15.39:0/1884985065 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a6c5f8d070 msgr2=0x55a6c634c110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:52:25.148 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:25.144+0000 7fdc33fff640 1 --2- 172.21.15.39:0/1884985065 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a6c5f641b0 0x55a6c6341ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:52:25.148 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:25.144+0000 7fdc337fe640 1 --2- 172.21.15.39:0/1884985065 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a6c5f8d070 0x55a6c634c110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:25.148 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:25.144+0000 7fdc337fe640 1 -- 172.21.15.39:0/1884985065 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a6c5f641b0 msgr2=0x55a6c6341ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:52:25.148 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:25.144+0000 7fdc337fe640 1 --2- 172.21.15.39:0/1884985065 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a6c5f641b0 0x55a6c6341ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:25.148 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:25.144+0000 7fdc337fe640 1 -- 172.21.15.39:0/1884985065 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a6c633e2c0 con 0x55a6c5f6c390 2024-09-21T03:52:25.148 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:25.144+0000 7fdc337fe640 1 --2- 172.21.15.39:0/1884985065 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a6c5f6c390 0x55a6c6335de0 secure :-1 s=READY pgs=969 cs=0 l=1 rev1=1 crypto rx=0x7fdc280aa210 tx=0x7fdc28005ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb93a9e848a4eae6 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:52:25.148 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:25.144+0000 7fdc19ffb640 1 -- 172.21.15.39:0/1884985065 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc28021070 con 0x55a6c5f6c390 2024-09-21T03:52:25.148 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:25.144+0000 7fdc19ffb640 1 -- 172.21.15.39:0/1884985065 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdc280acab0 con 0x55a6c5f6c390 2024-09-21T03:52:25.148 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:25.145+0000 7fdc19ffb640 1 -- 172.21.15.39:0/1884985065 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc280acd70 con 0x55a6c5f6c390 2024-09-21T03:52:25.148 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:25.145+0000 7fdc4094a740 1 -- 172.21.15.39:0/1884985065 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a6c5f6c390 msgr2=0x55a6c6335de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:52:25.148 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:25.145+0000 7fdc4094a740 1 --2- 172.21.15.39:0/1884985065 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a6c5f6c390 0x55a6c6335de0 secure :-1 s=READY pgs=969 cs=0 l=1 rev1=1 crypto rx=0x7fdc280aa210 tx=0x7fdc28005ce0 comp rx=0 tx=0).stop 2024-09-21T03:52:25.148 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:25.145+0000 7fdc4094a740 1 -- 172.21.15.39:0/1884985065 shutdown_connections 2024-09-21T03:52:25.148 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:25.145+0000 7fdc4094a740 1 --2- 172.21.15.39:0/1884985065 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a6c5f8d070 0x55a6c634c110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:25.148 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:25.145+0000 7fdc4094a740 1 --2- 172.21.15.39:0/1884985065 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a6c5f6c390 0x55a6c6335de0 unknown :-1 s=CLOSED pgs=969 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:25.149 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:25.145+0000 7fdc4094a740 1 --2- 172.21.15.39:0/1884985065 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a6c5f641b0 0x55a6c6341ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:25.149 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:25.145+0000 7fdc4094a740 1 -- 172.21.15.39:0/1884985065 >> 172.21.15.39:0/1884985065 conn(0x55a6c5f81460 msgr2=0x55a6c63356d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:52:25.149 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:25.145+0000 7fdc4094a740 1 -- 172.21.15.39:0/1884985065 shutdown_connections 2024-09-21T03:52:25.149 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:25.145+0000 7fdc4094a740 1 -- 172.21.15.39:0/1884985065 wait complete. 2024-09-21T03:52:25.149 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:25.146+0000 7fdc4094a740 1 -- start start 2024-09-21T03:52:25.149 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:25.146+0000 7fdc4094a740 1 Processor -- start 2024-09-21T03:52:25.152 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:25.146+0000 7fdc4094a740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a6c5f641b0 0x55a6c63557e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:52:25.152 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:25.146+0000 7fdc4094a740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a6c5f6c390 0x55a6c6352cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:52:25.152 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:25.146+0000 7fdc4094a740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a6c5f8d070 0x55a6c6353230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:52:25.152 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:25.146+0000 7fdc4094a740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fdc341b0c80 con 0x55a6c5f6c390 2024-09-21T03:52:25.152 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:25.146+0000 7fdc4094a740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fdc341b05b0 con 0x55a6c5f641b0 2024-09-21T03:52:25.152 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:25.146+0000 7fdc4094a740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fdc341ad7f0 con 0x55a6c5f8d070 2024-09-21T03:52:25.152 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:25.146+0000 7fdc4094a740 10 client.0 _ll_register_callbacks cb 0x7fdc280eaf50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:52:25.152 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:25.146+0000 7fdc337fe640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a6c5f6c390 0x55a6c6352cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:52:25.152 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:25.146+0000 7fdc337fe640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a6c5f6c390 0x55a6c6352cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:40026/0 (socket says 172.21.15.39:40026) 2024-09-21T03:52:25.152 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:25.146+0000 7fdc337fe640 1 -- 172.21.15.39:0/2277389470 learned_addr learned my addr 172.21.15.39:0/2277389470 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:52:25.153 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:25.146+0000 7fdc33fff640 1 --2- 172.21.15.39:0/2277389470 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a6c5f641b0 0x55a6c63557e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:52:25.153 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:25.147+0000 7fdc38973640 1 --2- 172.21.15.39:0/2277389470 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a6c5f8d070 0x55a6c6353230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:52:25.153 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:25.147+0000 7fdc337fe640 1 -- 172.21.15.39:0/2277389470 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a6c5f8d070 msgr2=0x55a6c6353230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:52:25.153 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:25.147+0000 7fdc337fe640 1 --2- 172.21.15.39:0/2277389470 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a6c5f8d070 0x55a6c6353230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:25.153 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:25.147+0000 7fdc337fe640 1 -- 172.21.15.39:0/2277389470 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a6c5f641b0 msgr2=0x55a6c63557e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:52:25.153 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:25.147+0000 7fdc337fe640 1 --2- 172.21.15.39:0/2277389470 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a6c5f641b0 0x55a6c63557e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:25.153 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:25.147+0000 7fdc337fe640 1 -- 172.21.15.39:0/2277389470 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a6c63539c0 con 0x55a6c5f6c390 2024-09-21T03:52:25.153 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:25.147+0000 7fdc337fe640 1 --2- 172.21.15.39:0/2277389470 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a6c5f6c390 0x55a6c6352cb0 secure :-1 s=READY pgs=970 cs=0 l=1 rev1=1 crypto rx=0x7fdc280a9570 tx=0x7fdc280b1c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=44bb3406a9bdaa7a server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:52:25.153 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:25.147+0000 7fdc137fe640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-09-21T03:52:25.153 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:25.147+0000 7fdc137fe640 1 -- 172.21.15.39:0/2277389470 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc28021050 con 0x55a6c5f6c390 2024-09-21T03:52:25.153 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:25.147+0000 7fdc137fe640 1 -- 172.21.15.39:0/2277389470 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8484+0+0 (secure 0 0 0) 0x7fdc280b6530 con 0x55a6c5f6c390 2024-09-21T03:52:25.153 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:25.147+0000 7fdc4094a740 1 -- 172.21.15.39:0/2277389470 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdc280acab0 con 0x55a6c5f6c390 2024-09-21T03:52:25.153 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:25.147+0000 7fdc137fe640 1 -- 172.21.15.39:0/2277389470 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fdc280a8e40 con 0x55a6c5f6c390 2024-09-21T03:52:25.153 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:25.147+0000 7fdc4094a740 10 client.17187 fetch_fsmap learned FSMap version 29 2024-09-21T03:52:25.154 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:25.147+0000 7fdc4094a740 1 -- 172.21.15.39:0/2277389470 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55a6c63539c0 con 0x55a6c5f6c390 2024-09-21T03:52:25.154 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:25.147+0000 7fdc137fe640 1 -- 172.21.15.39:0/2277389470 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdc280acab0 con 0x55a6c5f6c390 2024-09-21T03:52:25.154 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:25.148+0000 7fdc137fe640 1 -- 172.21.15.39:0/2277389470 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fdc280a88f0 con 0x55a6c5f6c390 2024-09-21T03:52:25.154 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:25.148+0000 7fdc4094a740 10 client.17187 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T03:52:25.154 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:25.148+0000 7fdc4094a740 10 client.17187 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-21T03:52:25.154 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:25.148+0000 7fdc4094a740 20 client.17187 populate_metadata read hostname 'smithi039' 2024-09-21T03:52:25.154 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:25.148+0000 7fdc4094a740 1 --2- 172.21.15.39:0/2277389470 >> [v2:172.21.15.161:6836/3451544597,v1:172.21.15.161:6837/3451544597] conn(0x55a6c5f20da0 0x55a6c63be890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:52:25.154 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:25.148+0000 7fdc4094a740 4 client.17187 mds_command: new command op to 24646 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T03:52:25.154 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:25.148+0000 7fdc4094a740 1 -- 172.21.15.39:0/2277389470 --> [v2:172.21.15.161:6836/3451544597,v1:172.21.15.161:6837/3451544597] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55a6c62a88a0 con 0x55a6c5f20da0 2024-09-21T03:52:25.154 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:25.148+0000 7fdc33fff640 1 --2- 172.21.15.39:0/2277389470 >> [v2:172.21.15.161:6836/3451544597,v1:172.21.15.161:6837/3451544597] conn(0x55a6c5f20da0 0x55a6c63be890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:52:25.154 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:25.148+0000 7fdc137fe640 10 client.17187 ms_handle_connect on v2:172.21.15.161:6836/3451544597 2024-09-21T03:52:25.154 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:25.148+0000 7fdc33fff640 1 --2- 172.21.15.39:0/2277389470 >> [v2:172.21.15.161:6836/3451544597,v1:172.21.15.161:6837/3451544597] conn(0x55a6c5f20da0 0x55a6c63be890 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=49fe25fd70e90646 server_cookie=3093f0b83ef2b3f6 in_seq=0 out_seq=0 2024-09-21T03:52:25.154 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:25.149+0000 7fdc137fe640 1 -- 172.21.15.39:0/2277389470 <== mds.3 v2:172.21.15.161:6836/3451544597 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55a6c62a88a0 con 0x55a6c5f20da0 2024-09-21T03:52:25.154 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:25.149+0000 7fdc137fe640 10 client.17187 handle_command_reply: tid=0 2024-09-21T03:52:25.155 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:25.149+0000 7fdc4094a740 1 client.17187 shutdown 2024-09-21T03:52:25.155 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:25.149+0000 7fdc4094a740 1 -- 172.21.15.39:0/2277389470 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a6c5f6c390 msgr2=0x55a6c6352cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:52:25.155 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:25.149+0000 7fdc4094a740 1 --2- 172.21.15.39:0/2277389470 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a6c5f6c390 0x55a6c6352cb0 secure :-1 s=READY pgs=970 cs=0 l=1 rev1=1 crypto rx=0x7fdc280a9570 tx=0x7fdc280b1c00 comp rx=0 tx=0).stop 2024-09-21T03:52:25.155 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:25.149+0000 7fdc4094a740 1 -- 172.21.15.39:0/2277389470 shutdown_connections 2024-09-21T03:52:25.155 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:25.149+0000 7fdc4094a740 1 --2- 172.21.15.39:0/2277389470 >> [v2:172.21.15.161:6836/3451544597,v1:172.21.15.161:6837/3451544597] conn(0x55a6c5f20da0 0x55a6c63be890 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:25.155 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:25.149+0000 7fdc4094a740 1 --2- 172.21.15.39:0/2277389470 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a6c5f8d070 0x55a6c6353230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:25.155 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:25.149+0000 7fdc4094a740 1 --2- 172.21.15.39:0/2277389470 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a6c5f6c390 0x55a6c6352cb0 unknown :-1 s=CLOSED pgs=970 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:25.155 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:25.149+0000 7fdc4094a740 1 --2- 172.21.15.39:0/2277389470 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a6c5f641b0 0x55a6c63557e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:25.155 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:25.149+0000 7fdc4094a740 1 -- 172.21.15.39:0/2277389470 >> 172.21.15.39:0/2277389470 conn(0x55a6c5f81460 msgr2=0x55a6c6341a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:52:25.155 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:25.149+0000 7fdc137fe640 0 client.17187 ms_handle_reset on v2:172.21.15.161:6836/3451544597 2024-09-21T03:52:25.155 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:25.149+0000 7fdc4094a740 1 -- 172.21.15.39:0/2277389470 shutdown_connections 2024-09-21T03:52:25.155 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:25.149+0000 7fdc4094a740 1 -- 172.21.15.39:0/2277389470 wait complete. 2024-09-21T03:52:25.155 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:25.149+0000 7fdc4094a740 20 client.17187 trim_cache size 0 max 16384 2024-09-21T03:52:25.156 INFO:teuthology.orchestra.run.smithi039.stdout:[] 2024-09-21T03:52:25.156 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:25.150+0000 7fdc3a3d6640 1 -- 172.21.15.39:0/3673777122 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7fdc1c0b6680 msgr2=0x7fdc1c0b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:52:25.156 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:25.150+0000 7fdc3a3d6640 1 --2- 172.21.15.39:0/3673777122 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7fdc1c0b6680 0x7fdc1c0b8b90 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fdc2c005fd0 tx=0x7fdc2c009600 comp rx=0 tx=0).stop 2024-09-21T03:52:25.156 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:25.150+0000 7fdc3a3d6640 1 -- 172.21.15.39:0/3673777122 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdc341094a0 msgr2=0x7fdc3419aa40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:52:25.156 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:25.150+0000 7fdc3a3d6640 1 --2- 172.21.15.39:0/3673777122 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdc341094a0 0x7fdc3419aa40 secure :-1 s=READY pgs=966 cs=0 l=1 rev1=1 crypto rx=0x7fdc2800d740 tx=0x7fdc2800dc00 comp rx=0 tx=0).stop 2024-09-21T03:52:25.156 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:25.150+0000 7fdc3a3d6640 1 -- 172.21.15.39:0/3673777122 shutdown_connections 2024-09-21T03:52:25.157 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:25.150+0000 7fdc3a3d6640 1 --2- 172.21.15.39:0/3673777122 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7fdc1c0b6680 0x7fdc1c0b8b90 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:25.157 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:25.150+0000 7fdc3a3d6640 1 --2- 172.21.15.39:0/3673777122 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdc34109df0 0x7fdc3419fce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:25.157 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:25.150+0000 7fdc3a3d6640 1 --2- 172.21.15.39:0/3673777122 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdc341094a0 0x7fdc3419aa40 unknown :-1 s=CLOSED pgs=966 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:25.157 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:25.150+0000 7fdc3a3d6640 1 --2- 172.21.15.39:0/3673777122 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdc34103b60 0x7fdc3419a4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:25.157 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:25.150+0000 7fdc3a3d6640 1 -- 172.21.15.39:0/3673777122 >> 172.21.15.39:0/3673777122 conn(0x7fdc3407b850 msgr2=0x7fdc34107530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:52:25.157 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:25.150+0000 7fdc3a3d6640 1 -- 172.21.15.39:0/3673777122 shutdown_connections 2024-09-21T03:52:25.157 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:25.150+0000 7fdc3a3d6640 1 -- 172.21.15.39:0/3673777122 wait complete. 2024-09-21T03:52:25.334 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:52:25 smithi039 ceph-mon[31211]: pgmap v516: 97 pgs: 97 active+clean; 10 MiB data, 520 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 85 B/s wr, 10 op/s 2024-09-21T03:52:25.718 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:52:25 smithi161 ceph-mon[34533]: pgmap v516: 97 pgs: 97 active+clean; 10 MiB data, 520 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 85 B/s wr, 10 op/s 2024-09-21T03:52:25.725 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:52:25 smithi150 ceph-mon[34289]: pgmap v516: 97 pgs: 97 active+clean; 10 MiB data, 520 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 85 B/s wr, 10 op/s 2024-09-21T03:52:25.972 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 51116d5c-77ca-11ef-baf5-efdc52797490 -- ceph tell mds.1:2 damage ls 2024-09-21T03:52:26.232 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/51116d5c-77ca-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:52:26.713 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:26.711+0000 7f6bb591b640 1 -- 172.21.15.39:0/2217913514 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6bb0102a90 msgr2=0x7f6bb0102ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:52:26.713 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:26.711+0000 7f6bb591b640 1 --2- 172.21.15.39:0/2217913514 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6bb0102a90 0x7f6bb0102ea0 secure :-1 s=READY pgs=971 cs=0 l=1 rev1=1 crypto rx=0x7f6ba8009900 tx=0x7f6ba802a410 comp rx=0 tx=0).stop 2024-09-21T03:52:26.713 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:26.711+0000 7f6bb591b640 1 -- 172.21.15.39:0/2217913514 shutdown_connections 2024-09-21T03:52:26.713 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:26.711+0000 7f6bb591b640 1 --2- 172.21.15.39:0/2217913514 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6bb0108d20 0x7f6bb010f550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:26.714 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:26.711+0000 7f6bb591b640 1 --2- 172.21.15.39:0/2217913514 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6bb01083d0 0x7f6bb01087e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:26.714 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:26.711+0000 7f6bb591b640 1 --2- 172.21.15.39:0/2217913514 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6bb0102a90 0x7f6bb0102ea0 unknown :-1 s=CLOSED pgs=971 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:26.714 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:26.711+0000 7f6bb591b640 1 -- 172.21.15.39:0/2217913514 >> 172.21.15.39:0/2217913514 conn(0x7f6bb0079730 msgr2=0x7f6bb0079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:52:26.714 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:26.711+0000 7f6bb591b640 1 -- 172.21.15.39:0/2217913514 shutdown_connections 2024-09-21T03:52:26.714 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:26.711+0000 7f6bb591b640 1 -- 172.21.15.39:0/2217913514 wait complete. 2024-09-21T03:52:26.716 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:26.712+0000 7f6bb591b640 1 Processor -- start 2024-09-21T03:52:26.716 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:26.712+0000 7f6bb591b640 1 -- start start 2024-09-21T03:52:26.716 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:26.712+0000 7f6bb591b640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6bb0102a90 0x7f6bb019a550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:52:26.716 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:26.712+0000 7f6bb591b640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6bb01083d0 0x7f6bb019aaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:52:26.716 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:26.712+0000 7f6bb591b640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6bb0108d20 0x7f6bb019fd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:52:26.716 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:26.712+0000 7f6bb591b640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f6bb0113b40 con 0x7f6bb0102a90 2024-09-21T03:52:26.716 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:26.712+0000 7f6bb591b640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f6bb01139c0 con 0x7f6bb01083d0 2024-09-21T03:52:26.716 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:26.712+0000 7f6bb591b640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f6bb0113cc0 con 0x7f6bb0108d20 2024-09-21T03:52:26.716 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:26.712+0000 7f6baeffd640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6bb0102a90 0x7f6bb019a550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:52:26.716 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:26.712+0000 7f6baeffd640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6bb0102a90 0x7f6bb019a550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:40050/0 (socket says 172.21.15.39:40050) 2024-09-21T03:52:26.716 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:26.712+0000 7f6baeffd640 1 -- 172.21.15.39:0/2026206324 learned_addr learned my addr 172.21.15.39:0/2026206324 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:52:26.716 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:26.712+0000 7f6baf7fe640 1 --2- 172.21.15.39:0/2026206324 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6bb0108d20 0x7f6bb019fd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:52:26.716 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:26.712+0000 7f6bae7fc640 1 --2- 172.21.15.39:0/2026206324 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6bb01083d0 0x7f6bb019aaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:52:26.717 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:26.713+0000 7f6baeffd640 1 -- 172.21.15.39:0/2026206324 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6bb0108d20 msgr2=0x7f6bb019fd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:52:26.717 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:26.713+0000 7f6baeffd640 1 --2- 172.21.15.39:0/2026206324 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6bb0108d20 0x7f6bb019fd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:26.717 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:26.713+0000 7f6baeffd640 1 -- 172.21.15.39:0/2026206324 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6bb01083d0 msgr2=0x7f6bb019aaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:52:26.717 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:26.713+0000 7f6baeffd640 1 --2- 172.21.15.39:0/2026206324 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6bb01083d0 0x7f6bb019aaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:26.717 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:26.713+0000 7f6baeffd640 1 -- 172.21.15.39:0/2026206324 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6bb01a04c0 con 0x7f6bb0102a90 2024-09-21T03:52:26.717 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:26.713+0000 7f6baeffd640 1 --2- 172.21.15.39:0/2026206324 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6bb0102a90 0x7f6bb019a550 secure :-1 s=READY pgs=972 cs=0 l=1 rev1=1 crypto rx=0x7f6ba8005940 tx=0x7f6ba80042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:52:26.717 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:26.713+0000 7f6bb4919640 1 -- 172.21.15.39:0/2026206324 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ba803f030 con 0x7f6bb0102a90 2024-09-21T03:52:26.717 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:26.713+0000 7f6bb4919640 1 -- 172.21.15.39:0/2026206324 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6ba8005d10 con 0x7f6bb0102a90 2024-09-21T03:52:26.717 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:26.713+0000 7f6bb591b640 1 -- 172.21.15.39:0/2026206324 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6bb01a0750 con 0x7f6bb0102a90 2024-09-21T03:52:26.717 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:26.713+0000 7f6bb4919640 1 -- 172.21.15.39:0/2026206324 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ba8035070 con 0x7f6bb0102a90 2024-09-21T03:52:26.717 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:26.713+0000 7f6bb591b640 1 -- 172.21.15.39:0/2026206324 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6bb01a6f10 con 0x7f6bb0102a90 2024-09-21T03:52:26.719 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:26.716+0000 7f6bbbe8f740 1 Processor -- start 2024-09-21T03:52:26.719 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:26.716+0000 7f6bbbe8f740 1 -- start start 2024-09-21T03:52:26.720 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:26.716+0000 7f6bb4919640 1 -- 172.21.15.39:0/2026206324 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 27) ==== 154643+0+0 (secure 0 0 0) 0x7f6ba8042030 con 0x7f6bb0102a90 2024-09-21T03:52:26.720 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:26.716+0000 7f6bbbe8f740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5572ebe21070 0x5572ec1c86a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:52:26.720 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:26.716+0000 7f6bbbe8f740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5572ebdf81b0 0x5572ec1c8be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:52:26.720 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:26.716+0000 7f6bbbe8f740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5572ebe00390 0x5572ec1c9120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:52:26.720 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:26.716+0000 7f6bbbe8f740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x5572ec13eea0 con 0x5572ebdf81b0 2024-09-21T03:52:26.720 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:26.716+0000 7f6bbbe8f740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x5572ec12e2d0 con 0x5572ebe00390 2024-09-21T03:52:26.720 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:26.716+0000 7f6bbbe8f740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x5572ec1320e0 con 0x5572ebe21070 2024-09-21T03:52:26.720 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:26.716+0000 7f6baeffd640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5572ebdf81b0 0x5572ec1c8be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:52:26.720 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:26.716+0000 7f6baeffd640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5572ebdf81b0 0x5572ec1c8be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:40052/0 (socket says 172.21.15.39:40052) 2024-09-21T03:52:26.720 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:26.716+0000 7f6baeffd640 1 -- 172.21.15.39:0/30742244 learned_addr learned my addr 172.21.15.39:0/30742244 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:52:26.720 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:26.717+0000 7f6baf7fe640 1 --2- 172.21.15.39:0/30742244 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5572ebe21070 0x5572ec1c86a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:52:26.720 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:26.717+0000 7f6bae7fc640 1 --2- 172.21.15.39:0/30742244 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5572ebe00390 0x5572ec1c9120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:52:26.720 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:26.717+0000 7f6baeffd640 1 -- 172.21.15.39:0/30742244 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5572ebe21070 msgr2=0x5572ec1c86a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:52:26.720 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:26.717+0000 7f6baeffd640 1 --2- 172.21.15.39:0/30742244 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5572ebe21070 0x5572ec1c86a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:26.721 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:26.717+0000 7f6baeffd640 1 -- 172.21.15.39:0/30742244 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5572ebe00390 msgr2=0x5572ec1c9120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:52:26.721 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:26.717+0000 7f6baeffd640 1 --2- 172.21.15.39:0/30742244 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5572ebe00390 0x5572ec1c9120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:26.721 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:26.717+0000 7f6baeffd640 1 -- 172.21.15.39:0/30742244 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5572ebe8c650 con 0x5572ebdf81b0 2024-09-21T03:52:26.721 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:26.717+0000 7f6bb4919640 1 --2- 172.21.15.39:0/2026206324 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f6b980b6680 0x7f6b980b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:52:26.721 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:26.717+0000 7f6baf7fe640 1 --2- 172.21.15.39:0/2026206324 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f6b980b6680 0x7f6b980b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:52:26.721 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:26.717+0000 7f6baeffd640 1 --2- 172.21.15.39:0/30742244 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5572ebdf81b0 0x5572ec1c8be0 secure :-1 s=READY pgs=973 cs=0 l=1 rev1=1 crypto rx=0x7f6ba8105270 tx=0x7f6ba810be60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=13b0273054f97b5e server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:52:26.721 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:26.717+0000 7f6bb4919640 1 -- 172.21.15.39:0/2026206324 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8484+0+0 (secure 0 0 0) 0x7f6ba8107b90 con 0x7f6bb0102a90 2024-09-21T03:52:26.721 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:26.717+0000 7f6b95ffb640 1 -- 172.21.15.39:0/30742244 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ba80ca050 con 0x5572ebdf81b0 2024-09-21T03:52:26.721 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:26.717+0000 7f6b95ffb640 1 -- 172.21.15.39:0/30742244 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6ba80c9160 con 0x5572ebdf81b0 2024-09-21T03:52:26.722 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:26.717+0000 7f6b95ffb640 1 -- 172.21.15.39:0/30742244 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ba80044e0 con 0x5572ebdf81b0 2024-09-21T03:52:26.725 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:26.718+0000 7f6bbbe8f740 1 -- 172.21.15.39:0/30742244 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5572ebdf81b0 msgr2=0x5572ec1c8be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:52:26.725 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:26.718+0000 7f6baf7fe640 1 --2- 172.21.15.39:0/2026206324 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f6b980b6680 0x7f6b980b8b90 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f6ba0005fd0 tx=0x7f6ba0009600 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:52:26.725 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:26.718+0000 7f6bbbe8f740 1 --2- 172.21.15.39:0/30742244 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5572ebdf81b0 0x5572ec1c8be0 secure :-1 s=READY pgs=973 cs=0 l=1 rev1=1 crypto rx=0x7f6ba8105270 tx=0x7f6ba810be60 comp rx=0 tx=0).stop 2024-09-21T03:52:26.725 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:26.718+0000 7f6bbbe8f740 1 -- 172.21.15.39:0/30742244 shutdown_connections 2024-09-21T03:52:26.725 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:26.718+0000 7f6bbbe8f740 1 --2- 172.21.15.39:0/30742244 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5572ebe00390 0x5572ec1c9120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:26.725 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:26.718+0000 7f6bbbe8f740 1 --2- 172.21.15.39:0/30742244 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5572ebdf81b0 0x5572ec1c8be0 unknown :-1 s=CLOSED pgs=973 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:26.725 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:26.718+0000 7f6bbbe8f740 1 --2- 172.21.15.39:0/30742244 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5572ebe21070 0x5572ec1c86a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:26.725 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:26.718+0000 7f6bbbe8f740 1 -- 172.21.15.39:0/30742244 >> 172.21.15.39:0/30742244 conn(0x5572ebe15460 msgr2=0x5572ec1c6590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:52:26.726 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:26.718+0000 7f6bbbe8f740 1 -- 172.21.15.39:0/30742244 shutdown_connections 2024-09-21T03:52:26.726 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:26.718+0000 7f6bbbe8f740 1 -- 172.21.15.39:0/30742244 wait complete. 2024-09-21T03:52:26.726 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:26.719+0000 7f6bbbe8f740 1 -- start start 2024-09-21T03:52:26.726 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:26.719+0000 7f6bbbe8f740 1 Processor -- start 2024-09-21T03:52:26.726 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:26.719+0000 7f6bbbe8f740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5572ebdf81b0 0x5572ec1d3d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:52:26.726 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:26.719+0000 7f6bbbe8f740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5572ebe00390 0x5572ec1d42a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:52:26.726 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:26.719+0000 7f6bbbe8f740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5572ebe21070 0x5572ec1dfde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:52:26.726 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:26.719+0000 7f6bbbe8f740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x5572ebf8f700 con 0x5572ebdf81b0 2024-09-21T03:52:26.726 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:26.719+0000 7f6bbbe8f740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x5572ec0c9160 con 0x5572ebe21070 2024-09-21T03:52:26.726 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:26.719+0000 7f6bbbe8f740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x5572ec13eea0 con 0x5572ebe00390 2024-09-21T03:52:26.726 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:26.720+0000 7f6baeffd640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5572ebdf81b0 0x5572ec1d3d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:52:26.726 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:26.720+0000 7f6bbbe8f740 10 client.0 _ll_register_callbacks cb 0x5572ec0e29e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:52:26.726 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:26.720+0000 7f6baeffd640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5572ebdf81b0 0x5572ec1d3d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:40056/0 (socket says 172.21.15.39:40056) 2024-09-21T03:52:26.727 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:26.720+0000 7f6baeffd640 1 -- 172.21.15.39:0/3493021980 learned_addr learned my addr 172.21.15.39:0/3493021980 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:52:26.727 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:26.720+0000 7f6baf7fe640 1 --2- 172.21.15.39:0/3493021980 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5572ebe21070 0x5572ec1dfde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:52:26.727 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:26.720+0000 7f6bae7fc640 1 --2- 172.21.15.39:0/3493021980 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5572ebe00390 0x5572ec1d42a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:52:26.727 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:26.720+0000 7f6baeffd640 1 -- 172.21.15.39:0/3493021980 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5572ebe00390 msgr2=0x5572ec1d42a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:52:26.727 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:26.720+0000 7f6baeffd640 1 --2- 172.21.15.39:0/3493021980 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5572ebe00390 0x5572ec1d42a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:26.727 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:26.720+0000 7f6baeffd640 1 -- 172.21.15.39:0/3493021980 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5572ebe21070 msgr2=0x5572ec1dfde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:52:26.727 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:26.720+0000 7f6baeffd640 1 --2- 172.21.15.39:0/3493021980 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5572ebe21070 0x5572ec1dfde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:26.727 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:26.720+0000 7f6baeffd640 1 -- 172.21.15.39:0/3493021980 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5572ec0f6c90 con 0x5572ebdf81b0 2024-09-21T03:52:26.727 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:26.720+0000 7f6baeffd640 1 --2- 172.21.15.39:0/3493021980 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5572ebdf81b0 0x5572ec1d3d60 secure :-1 s=READY pgs=974 cs=0 l=1 rev1=1 crypto rx=0x7f6ba8031440 tx=0x7f6ba8107ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7aa26f6491f66696 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:52:26.727 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:26.720+0000 7f6b8effd640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-09-21T03:52:26.728 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:26.720+0000 7f6b8effd640 1 -- 172.21.15.39:0/3493021980 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ba8049050 con 0x5572ebdf81b0 2024-09-21T03:52:26.728 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:26.720+0000 7f6b8effd640 1 -- 172.21.15.39:0/3493021980 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8484+0+0 (secure 0 0 0) 0x7f6ba810b8c0 con 0x5572ebdf81b0 2024-09-21T03:52:26.728 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:26.720+0000 7f6bbbe8f740 1 -- 172.21.15.39:0/3493021980 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6ba80c9160 con 0x5572ebdf81b0 2024-09-21T03:52:26.728 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:26.720+0000 7f6b8effd640 1 -- 172.21.15.39:0/3493021980 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f6ba80cd170 con 0x5572ebdf81b0 2024-09-21T03:52:26.728 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:26.721+0000 7f6bbbe8f740 10 client.17199 fetch_fsmap learned FSMap version 29 2024-09-21T03:52:26.728 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:26.721+0000 7f6bbbe8f740 1 -- 172.21.15.39:0/3493021980 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5572ec1e27a0 con 0x5572ebdf81b0 2024-09-21T03:52:26.728 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:26.721+0000 7f6b8effd640 1 -- 172.21.15.39:0/3493021980 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6ba80c9160 con 0x5572ebdf81b0 2024-09-21T03:52:26.728 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:26.721+0000 7f6b8effd640 1 -- 172.21.15.39:0/3493021980 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f6ba8105580 con 0x5572ebdf81b0 2024-09-21T03:52:26.728 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:26.721+0000 7f6bbbe8f740 10 client.17199 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T03:52:26.728 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:26.721+0000 7f6bbbe8f740 10 client.17199 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-21T03:52:26.728 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:26.721+0000 7f6bbbe8f740 20 client.17199 populate_metadata read hostname 'smithi039' 2024-09-21T03:52:26.728 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:26.721+0000 7f6bbbe8f740 1 --2- 172.21.15.39:0/3493021980 >> [v2:172.21.15.150:6836/4017043144,v1:172.21.15.150:6837/4017043144] conn(0x5572ebddf9f0 0x5572ec1e5b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:52:26.728 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:26.721+0000 7f6bbbe8f740 4 client.17199 mds_command: new command op to 24659 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:52:26.728 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:26.721+0000 7f6bbbe8f740 1 -- 172.21.15.39:0/3493021980 --> [v2:172.21.15.150:6836/4017043144,v1:172.21.15.150:6837/4017043144] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5572ec140df0 con 0x5572ebddf9f0 2024-09-21T03:52:26.728 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:26.721+0000 7f6bae7fc640 1 --2- 172.21.15.39:0/3493021980 >> [v2:172.21.15.150:6836/4017043144,v1:172.21.15.150:6837/4017043144] conn(0x5572ebddf9f0 0x5572ec1e5b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:52:26.729 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:26.722+0000 7f6bae7fc640 1 --2- 172.21.15.39:0/3493021980 >> [v2:172.21.15.150:6836/4017043144,v1:172.21.15.150:6837/4017043144] conn(0x5572ebddf9f0 0x5572ec1e5b00 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=daecc8d5733abf44 server_cookie=40f98d627b7d4d92 in_seq=0 out_seq=0 2024-09-21T03:52:26.729 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:26.722+0000 7f6b8effd640 10 client.17199 ms_handle_connect on v2:172.21.15.150:6836/4017043144 2024-09-21T03:52:26.729 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:26.723+0000 7f6b8effd640 1 -- 172.21.15.39:0/3493021980 <== mds.2 v2:172.21.15.150:6836/4017043144 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5572ec140df0 con 0x5572ebddf9f0 2024-09-21T03:52:26.729 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:26.723+0000 7f6b8effd640 10 client.17199 handle_command_reply: tid=0 2024-09-21T03:52:26.729 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:26.723+0000 7f6bbbe8f740 1 client.17199 shutdown 2024-09-21T03:52:26.729 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:26.723+0000 7f6bbbe8f740 1 -- 172.21.15.39:0/3493021980 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5572ebdf81b0 msgr2=0x5572ec1d3d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:52:26.729 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:26.723+0000 7f6bbbe8f740 1 --2- 172.21.15.39:0/3493021980 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5572ebdf81b0 0x5572ec1d3d60 secure :-1 s=READY pgs=974 cs=0 l=1 rev1=1 crypto rx=0x7f6ba8031440 tx=0x7f6ba8107ef0 comp rx=0 tx=0).stop 2024-09-21T03:52:26.729 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:26.723+0000 7f6bbbe8f740 1 -- 172.21.15.39:0/3493021980 shutdown_connections 2024-09-21T03:52:26.729 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:26.723+0000 7f6bbbe8f740 1 --2- 172.21.15.39:0/3493021980 >> [v2:172.21.15.150:6836/4017043144,v1:172.21.15.150:6837/4017043144] conn(0x5572ebddf9f0 0x5572ec1e5b00 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:26.729 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:26.723+0000 7f6bbbe8f740 1 --2- 172.21.15.39:0/3493021980 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5572ebe21070 0x5572ec1dfde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:26.729 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:26.723+0000 7f6bbbe8f740 1 --2- 172.21.15.39:0/3493021980 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5572ebe00390 0x5572ec1d42a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:26.729 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:26.723+0000 7f6bbbe8f740 1 --2- 172.21.15.39:0/3493021980 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5572ebdf81b0 0x5572ec1d3d60 unknown :-1 s=CLOSED pgs=974 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:26.730 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:26.723+0000 7f6b8effd640 0 client.17199 ms_handle_reset on v2:172.21.15.150:6836/4017043144 2024-09-21T03:52:26.730 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:26.723+0000 7f6bbbe8f740 1 -- 172.21.15.39:0/3493021980 >> 172.21.15.39:0/3493021980 conn(0x5572ebe15460 msgr2=0x5572ec1cb830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:52:26.730 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:26.723+0000 7f6bbbe8f740 1 -- 172.21.15.39:0/3493021980 shutdown_connections 2024-09-21T03:52:26.730 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:26.723+0000 7f6bbbe8f740 1 -- 172.21.15.39:0/3493021980 wait complete. 2024-09-21T03:52:26.730 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:26.723+0000 7f6bbbe8f740 20 client.17199 trim_cache size 0 max 16384 2024-09-21T03:52:26.739 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:26.736+0000 7f6bbbe8f740 1 Processor -- start 2024-09-21T03:52:26.739 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:26.736+0000 7f6bbbe8f740 1 -- start start 2024-09-21T03:52:26.739 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:26.737+0000 7f6bbbe8f740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5572ebdf81b0 0x5572ec1d5990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:52:26.740 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:26.737+0000 7f6bbbe8f740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5572ebe00390 0x5572ec1c5bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:52:26.740 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:26.737+0000 7f6bbbe8f740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5572ebe21070 0x5572ec1c6100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:52:26.740 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:26.737+0000 7f6bbbe8f740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f6bb01b1a40 con 0x5572ebe00390 2024-09-21T03:52:26.740 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:26.737+0000 7f6bbbe8f740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f6bb01b3580 con 0x5572ebe21070 2024-09-21T03:52:26.740 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:26.737+0000 7f6bbbe8f740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f6bb01b2110 con 0x5572ebdf81b0 2024-09-21T03:52:26.740 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:26.737+0000 7f6bae7fc640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5572ebe00390 0x5572ec1c5bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:52:26.740 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:26.737+0000 7f6bae7fc640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5572ebe00390 0x5572ec1c5bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:40064/0 (socket says 172.21.15.39:40064) 2024-09-21T03:52:26.740 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:26.737+0000 7f6bae7fc640 1 -- 172.21.15.39:0/1165112437 learned_addr learned my addr 172.21.15.39:0/1165112437 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:52:26.740 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:26.737+0000 7f6baf7fe640 1 --2- 172.21.15.39:0/1165112437 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5572ebe21070 0x5572ec1c6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:52:26.740 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:26.737+0000 7f6baeffd640 1 --2- 172.21.15.39:0/1165112437 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5572ebdf81b0 0x5572ec1d5990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:52:26.740 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:26.737+0000 7f6bae7fc640 1 -- 172.21.15.39:0/1165112437 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5572ebdf81b0 msgr2=0x5572ec1d5990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:52:26.740 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:26.737+0000 7f6bae7fc640 1 --2- 172.21.15.39:0/1165112437 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5572ebdf81b0 0x5572ec1d5990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:26.741 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:26.737+0000 7f6bae7fc640 1 -- 172.21.15.39:0/1165112437 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5572ebe21070 msgr2=0x5572ec1c6100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:52:26.741 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:26.737+0000 7f6bae7fc640 1 --2- 172.21.15.39:0/1165112437 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5572ebe21070 0x5572ec1c6100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:26.741 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:26.737+0000 7f6bae7fc640 1 -- 172.21.15.39:0/1165112437 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5572ec1d5020 con 0x5572ebe00390 2024-09-21T03:52:26.741 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:26.737+0000 7f6baeffd640 1 --2- 172.21.15.39:0/1165112437 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5572ebdf81b0 0x5572ec1d5990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:52:26.741 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:26.737+0000 7f6bae7fc640 1 --2- 172.21.15.39:0/1165112437 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5572ebe00390 0x5572ec1c5bc0 secure :-1 s=READY pgs=975 cs=0 l=1 rev1=1 crypto rx=0x7f6ba4009b00 tx=0x7f6ba4008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b71580e96d3188fc server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:52:26.741 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:26.737+0000 7f6b957fa640 1 -- 172.21.15.39:0/1165112437 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ba4015480 con 0x5572ebe00390 2024-09-21T03:52:26.741 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:26.738+0000 7f6b957fa640 1 -- 172.21.15.39:0/1165112437 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6ba4015ad0 con 0x5572ebe00390 2024-09-21T03:52:26.741 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:26.738+0000 7f6b957fa640 1 -- 172.21.15.39:0/1165112437 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ba40148e0 con 0x5572ebe00390 2024-09-21T03:52:26.745 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:26.738+0000 7f6bbbe8f740 1 -- 172.21.15.39:0/1165112437 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5572ebe00390 msgr2=0x5572ec1c5bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:52:26.745 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:26.738+0000 7f6bbbe8f740 1 --2- 172.21.15.39:0/1165112437 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5572ebe00390 0x5572ec1c5bc0 secure :-1 s=READY pgs=975 cs=0 l=1 rev1=1 crypto rx=0x7f6ba4009b00 tx=0x7f6ba4008040 comp rx=0 tx=0).stop 2024-09-21T03:52:26.745 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:26.738+0000 7f6bbbe8f740 1 -- 172.21.15.39:0/1165112437 shutdown_connections 2024-09-21T03:52:26.745 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:26.738+0000 7f6bbbe8f740 1 --2- 172.21.15.39:0/1165112437 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5572ebe21070 0x5572ec1c6100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:26.745 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:26.738+0000 7f6bbbe8f740 1 --2- 172.21.15.39:0/1165112437 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5572ebe00390 0x5572ec1c5bc0 unknown :-1 s=CLOSED pgs=975 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:26.745 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:26.738+0000 7f6bbbe8f740 1 --2- 172.21.15.39:0/1165112437 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5572ebdf81b0 0x5572ec1d5990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:26.745 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:26.738+0000 7f6bbbe8f740 1 -- 172.21.15.39:0/1165112437 >> 172.21.15.39:0/1165112437 conn(0x5572ebe15460 msgr2=0x5572ec1d5280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:52:26.745 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:26.738+0000 7f6bbbe8f740 1 -- 172.21.15.39:0/1165112437 shutdown_connections 2024-09-21T03:52:26.745 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:26.738+0000 7f6bbbe8f740 1 -- 172.21.15.39:0/1165112437 wait complete. 2024-09-21T03:52:26.745 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:26.739+0000 7f6bbbe8f740 1 -- start start 2024-09-21T03:52:26.745 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:26.739+0000 7f6bbbe8f740 1 Processor -- start 2024-09-21T03:52:26.746 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:26.739+0000 7f6bbbe8f740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5572ebdf81b0 0x5572ec1d7ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:52:26.746 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:26.739+0000 7f6bbbe8f740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5572ebe00390 0x5572ec1d81e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:52:26.746 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:26.739+0000 7f6bbbe8f740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5572ebe21070 0x5572ec1d8720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:52:26.746 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:26.739+0000 7f6bbbe8f740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f6bb01b2eb0 con 0x5572ebdf81b0 2024-09-21T03:52:26.746 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:26.739+0000 7f6bbbe8f740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f6bb01b27e0 con 0x5572ebe21070 2024-09-21T03:52:26.746 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:26.739+0000 7f6bbbe8f740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f6bb01afa20 con 0x5572ebe00390 2024-09-21T03:52:26.746 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:26.739+0000 7f6bbbe8f740 10 client.0 _ll_register_callbacks cb 0x7f6ba40093c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:52:26.746 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:26.739+0000 7f6baeffd640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5572ebdf81b0 0x5572ec1d7ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:52:26.746 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:26.739+0000 7f6baeffd640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5572ebdf81b0 0x5572ec1d7ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:40068/0 (socket says 172.21.15.39:40068) 2024-09-21T03:52:26.746 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:26.739+0000 7f6baeffd640 1 -- 172.21.15.39:0/3448170249 learned_addr learned my addr 172.21.15.39:0/3448170249 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:52:26.746 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:26.740+0000 7f6bae7fc640 1 --2- 172.21.15.39:0/3448170249 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5572ebe00390 0x5572ec1d81e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:52:26.746 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:26.740+0000 7f6baf7fe640 1 --2- 172.21.15.39:0/3448170249 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5572ebe21070 0x5572ec1d8720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:52:26.746 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:26.740+0000 7f6baeffd640 1 -- 172.21.15.39:0/3448170249 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5572ebe00390 msgr2=0x5572ec1d81e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:52:26.746 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:26.740+0000 7f6baeffd640 1 --2- 172.21.15.39:0/3448170249 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5572ebe00390 0x5572ec1d81e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:26.747 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:26.740+0000 7f6baeffd640 1 -- 172.21.15.39:0/3448170249 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5572ebe21070 msgr2=0x5572ec1d8720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:52:26.747 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:26.740+0000 7f6baeffd640 1 --2- 172.21.15.39:0/3448170249 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5572ebe21070 0x5572ec1d8720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:26.747 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:26.740+0000 7f6baeffd640 1 -- 172.21.15.39:0/3448170249 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5572ec24f420 con 0x5572ebdf81b0 2024-09-21T03:52:26.747 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:26.740+0000 7f6baf7fe640 1 --2- 172.21.15.39:0/3448170249 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5572ebe21070 0x5572ec1d8720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:52:26.747 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:26.740+0000 7f6baeffd640 1 --2- 172.21.15.39:0/3448170249 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5572ebdf81b0 0x5572ec1d7ca0 secure :-1 s=READY pgs=976 cs=0 l=1 rev1=1 crypto rx=0x7f6ba80c9fd0 tx=0x7f6ba8110470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d08d80d6b1ec26e server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:52:26.747 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:26.740+0000 7f6b8effd640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-09-21T03:52:26.747 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:26.740+0000 7f6b8effd640 1 -- 172.21.15.39:0/3448170249 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ba8049070 con 0x5572ebdf81b0 2024-09-21T03:52:26.747 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:26.740+0000 7f6b8effd640 1 -- 172.21.15.39:0/3448170249 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8484+0+0 (secure 0 0 0) 0x7f6ba80d6f90 con 0x5572ebdf81b0 2024-09-21T03:52:26.747 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:26.740+0000 7f6bbbe8f740 1 -- 172.21.15.39:0/3448170249 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6ba4015ad0 con 0x5572ebdf81b0 2024-09-21T03:52:26.747 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:26.740+0000 7f6b8effd640 1 -- 172.21.15.39:0/3448170249 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f6ba80cd350 con 0x5572ebdf81b0 2024-09-21T03:52:26.747 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:26.740+0000 7f6bbbe8f740 10 client.17205 fetch_fsmap learned FSMap version 29 2024-09-21T03:52:26.747 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:26.740+0000 7f6bbbe8f740 1 -- 172.21.15.39:0/3448170249 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5572ec24f420 con 0x5572ebdf81b0 2024-09-21T03:52:26.748 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:26.741+0000 7f6b8effd640 1 -- 172.21.15.39:0/3448170249 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6ba4015ad0 con 0x5572ebdf81b0 2024-09-21T03:52:26.748 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:26.741+0000 7f6b8effd640 1 -- 172.21.15.39:0/3448170249 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f6ba8110040 con 0x5572ebdf81b0 2024-09-21T03:52:26.748 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:26.741+0000 7f6bbbe8f740 10 client.17205 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T03:52:26.748 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:26.741+0000 7f6bbbe8f740 10 client.17205 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-21T03:52:26.748 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:26.741+0000 7f6bbbe8f740 20 client.17205 populate_metadata read hostname 'smithi039' 2024-09-21T03:52:26.748 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:26.741+0000 7f6bbbe8f740 1 --2- 172.21.15.39:0/3448170249 >> [v2:172.21.15.150:6836/4017043144,v1:172.21.15.150:6837/4017043144] conn(0x5572ebdb4da0 0x5572ec2529a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:52:26.748 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:26.741+0000 7f6bbbe8f740 4 client.17205 mds_command: new command op to 24659 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T03:52:26.748 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:26.741+0000 7f6bbbe8f740 1 -- 172.21.15.39:0/3448170249 --> [v2:172.21.15.150:6836/4017043144,v1:172.21.15.150:6837/4017043144] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5572ec13c8a0 con 0x5572ebdb4da0 2024-09-21T03:52:26.748 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:26.741+0000 7f6bae7fc640 1 --2- 172.21.15.39:0/3448170249 >> [v2:172.21.15.150:6836/4017043144,v1:172.21.15.150:6837/4017043144] conn(0x5572ebdb4da0 0x5572ec2529a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:52:26.748 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:26.742+0000 7f6bae7fc640 1 --2- 172.21.15.39:0/3448170249 >> [v2:172.21.15.150:6836/4017043144,v1:172.21.15.150:6837/4017043144] conn(0x5572ebdb4da0 0x5572ec2529a0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=e7f733f70d102a79 server_cookie=c21c704ea9b3f9ca in_seq=0 out_seq=0 2024-09-21T03:52:26.748 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:26.742+0000 7f6b8effd640 10 client.17205 ms_handle_connect on v2:172.21.15.150:6836/4017043144 2024-09-21T03:52:26.748 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:26.742+0000 7f6b8effd640 1 -- 172.21.15.39:0/3448170249 <== mds.2 v2:172.21.15.150:6836/4017043144 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5572ec13c8a0 con 0x5572ebdb4da0 2024-09-21T03:52:26.748 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:26.742+0000 7f6b8effd640 10 client.17205 handle_command_reply: tid=0 2024-09-21T03:52:26.749 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:26.742+0000 7f6bbbe8f740 1 client.17205 shutdown 2024-09-21T03:52:26.749 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:26.742+0000 7f6bbbe8f740 1 -- 172.21.15.39:0/3448170249 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5572ebdf81b0 msgr2=0x5572ec1d7ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:52:26.749 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:26.742+0000 7f6bbbe8f740 1 --2- 172.21.15.39:0/3448170249 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5572ebdf81b0 0x5572ec1d7ca0 secure :-1 s=READY pgs=976 cs=0 l=1 rev1=1 crypto rx=0x7f6ba80c9fd0 tx=0x7f6ba8110470 comp rx=0 tx=0).stop 2024-09-21T03:52:26.749 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:26.742+0000 7f6bbbe8f740 1 -- 172.21.15.39:0/3448170249 shutdown_connections 2024-09-21T03:52:26.749 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:26.742+0000 7f6bbbe8f740 1 --2- 172.21.15.39:0/3448170249 >> [v2:172.21.15.150:6836/4017043144,v1:172.21.15.150:6837/4017043144] conn(0x5572ebdb4da0 0x5572ec2529a0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:26.749 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:26.742+0000 7f6bbbe8f740 1 --2- 172.21.15.39:0/3448170249 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5572ebe21070 0x5572ec1d8720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:26.749 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:26.742+0000 7f6bbbe8f740 1 --2- 172.21.15.39:0/3448170249 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5572ebe00390 0x5572ec1d81e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:26.749 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:26.742+0000 7f6bbbe8f740 1 --2- 172.21.15.39:0/3448170249 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5572ebdf81b0 0x5572ec1d7ca0 unknown :-1 s=CLOSED pgs=976 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:26.749 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:26.742+0000 7f6bbbe8f740 1 -- 172.21.15.39:0/3448170249 >> 172.21.15.39:0/3448170249 conn(0x5572ebe15460 msgr2=0x5572ec1d5bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:52:26.749 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:26.742+0000 7f6b8effd640 0 client.17205 ms_handle_reset on v2:172.21.15.150:6836/4017043144 2024-09-21T03:52:26.749 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:26.742+0000 7f6bbbe8f740 1 -- 172.21.15.39:0/3448170249 shutdown_connections 2024-09-21T03:52:26.749 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:26.742+0000 7f6bbbe8f740 1 -- 172.21.15.39:0/3448170249 wait complete. 2024-09-21T03:52:26.749 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:26.742+0000 7f6bbbe8f740 20 client.17205 trim_cache size 0 max 16384 2024-09-21T03:52:26.750 INFO:teuthology.orchestra.run.smithi039.stdout:[] 2024-09-21T03:52:26.750 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:26.743+0000 7f6bb591b640 1 -- 172.21.15.39:0/2026206324 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f6b980b6680 msgr2=0x7f6b980b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:52:26.750 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:26.743+0000 7f6bb591b640 1 --2- 172.21.15.39:0/2026206324 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f6b980b6680 0x7f6b980b8b90 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f6ba0005fd0 tx=0x7f6ba0009600 comp rx=0 tx=0).stop 2024-09-21T03:52:26.750 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:26.743+0000 7f6bb591b640 1 -- 172.21.15.39:0/2026206324 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6bb0102a90 msgr2=0x7f6bb019a550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:52:26.751 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:26.743+0000 7f6bb591b640 1 --2- 172.21.15.39:0/2026206324 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6bb0102a90 0x7f6bb019a550 secure :-1 s=READY pgs=972 cs=0 l=1 rev1=1 crypto rx=0x7f6ba8005940 tx=0x7f6ba80042f0 comp rx=0 tx=0).stop 2024-09-21T03:52:26.751 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:26.743+0000 7f6bb591b640 1 -- 172.21.15.39:0/2026206324 shutdown_connections 2024-09-21T03:52:26.751 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:26.743+0000 7f6bb591b640 1 --2- 172.21.15.39:0/2026206324 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f6b980b6680 0x7f6b980b8b90 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:26.751 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:26.743+0000 7f6bb591b640 1 --2- 172.21.15.39:0/2026206324 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6bb0108d20 0x7f6bb019fd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:26.751 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:26.743+0000 7f6bb591b640 1 --2- 172.21.15.39:0/2026206324 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6bb01083d0 0x7f6bb019aaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:26.751 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:26.743+0000 7f6bb591b640 1 --2- 172.21.15.39:0/2026206324 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6bb0102a90 0x7f6bb019a550 unknown :-1 s=CLOSED pgs=972 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:26.751 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:26.743+0000 7f6bb591b640 1 -- 172.21.15.39:0/2026206324 >> 172.21.15.39:0/2026206324 conn(0x7f6bb0079730 msgr2=0x7f6bb0106460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:52:26.751 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:26.743+0000 7f6bb591b640 1 -- 172.21.15.39:0/2026206324 shutdown_connections 2024-09-21T03:52:26.751 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:26.743+0000 7f6bb591b640 1 -- 172.21.15.39:0/2026206324 wait complete. 2024-09-21T03:52:27.584 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:52:27 smithi039 ceph-mon[31211]: pgmap v517: 97 pgs: 97 active+clean; 10 MiB data, 520 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 10 op/s 2024-09-21T03:52:27.651 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 51116d5c-77ca-11ef-baf5-efdc52797490 -- ceph tell mds.1:1 damage ls 2024-09-21T03:52:27.718 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:52:27 smithi161 ceph-mon[34533]: pgmap v517: 97 pgs: 97 active+clean; 10 MiB data, 520 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 10 op/s 2024-09-21T03:52:27.724 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:52:27 smithi150 ceph-mon[34289]: pgmap v517: 97 pgs: 97 active+clean; 10 MiB data, 520 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 10 op/s 2024-09-21T03:52:27.907 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/51116d5c-77ca-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:52:28.391 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:28.389+0000 7f2b02186640 1 -- 172.21.15.39:0/1140948295 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2afc1094a0 msgr2=0x7f2afc1098b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:52:28.391 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:28.389+0000 7f2b02186640 1 --2- 172.21.15.39:0/1140948295 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2afc1094a0 0x7f2afc1098b0 secure :-1 s=READY pgs=977 cs=0 l=1 rev1=1 crypto rx=0x7f2ae4009900 tx=0x7f2ae402a410 comp rx=0 tx=0).stop 2024-09-21T03:52:28.391 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:28.389+0000 7f2b02186640 1 -- 172.21.15.39:0/1140948295 shutdown_connections 2024-09-21T03:52:28.391 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:28.389+0000 7f2b02186640 1 --2- 172.21.15.39:0/1140948295 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2afc109df0 0x7f2afc110620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:28.391 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:28.389+0000 7f2b02186640 1 --2- 172.21.15.39:0/1140948295 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2afc1094a0 0x7f2afc1098b0 unknown :-1 s=CLOSED pgs=977 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:28.391 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:28.389+0000 7f2b02186640 1 --2- 172.21.15.39:0/1140948295 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2afc103b60 0x7f2afc103f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:28.392 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:28.389+0000 7f2b02186640 1 -- 172.21.15.39:0/1140948295 >> 172.21.15.39:0/1140948295 conn(0x7f2afc07b850 msgr2=0x7f2afc07bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:52:28.392 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:28.389+0000 7f2b02186640 1 -- 172.21.15.39:0/1140948295 shutdown_connections 2024-09-21T03:52:28.392 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:28.389+0000 7f2b02186640 1 -- 172.21.15.39:0/1140948295 wait complete. 2024-09-21T03:52:28.393 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:28.390+0000 7f2b02186640 1 Processor -- start 2024-09-21T03:52:28.394 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:28.390+0000 7f2b02186640 1 -- start start 2024-09-21T03:52:28.394 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:28.390+0000 7f2b02186640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2afc103b60 0x7f2afc19a570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:52:28.394 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:28.390+0000 7f2b02186640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2afc1094a0 0x7f2afc19ab10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:52:28.394 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:28.390+0000 7f2b02186640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2afc109df0 0x7f2afc19fdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:52:28.394 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:28.390+0000 7f2b02186640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f2afc113b00 con 0x7f2afc103b60 2024-09-21T03:52:28.394 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:28.390+0000 7f2b02186640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f2afc113980 con 0x7f2afc109df0 2024-09-21T03:52:28.394 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:28.390+0000 7f2b02186640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f2afc113c80 con 0x7f2afc1094a0 2024-09-21T03:52:28.394 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:28.390+0000 7f2afb7fe640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2afc103b60 0x7f2afc19a570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:52:28.394 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:28.390+0000 7f2afb7fe640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2afc103b60 0x7f2afc19a570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:49784/0 (socket says 172.21.15.39:49784) 2024-09-21T03:52:28.394 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:28.390+0000 7f2afb7fe640 1 -- 172.21.15.39:0/3259272194 learned_addr learned my addr 172.21.15.39:0/3259272194 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:52:28.394 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:28.390+0000 7f2afaffd640 1 --2- 172.21.15.39:0/3259272194 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2afc1094a0 0x7f2afc19ab10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:52:28.394 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:28.391+0000 7f2afbfff640 1 --2- 172.21.15.39:0/3259272194 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2afc109df0 0x7f2afc19fdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:52:28.394 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:28.391+0000 7f2afb7fe640 1 -- 172.21.15.39:0/3259272194 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2afc1094a0 msgr2=0x7f2afc19ab10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:52:28.394 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:28.391+0000 7f2afb7fe640 1 --2- 172.21.15.39:0/3259272194 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2afc1094a0 0x7f2afc19ab10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:28.395 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:28.391+0000 7f2afb7fe640 1 -- 172.21.15.39:0/3259272194 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2afc109df0 msgr2=0x7f2afc19fdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:52:28.395 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:28.391+0000 7f2afb7fe640 1 --2- 172.21.15.39:0/3259272194 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2afc109df0 0x7f2afc19fdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:28.395 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:28.391+0000 7f2afb7fe640 1 -- 172.21.15.39:0/3259272194 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2afc1a0420 con 0x7f2afc103b60 2024-09-21T03:52:28.395 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:28.391+0000 7f2afbfff640 1 --2- 172.21.15.39:0/3259272194 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2afc109df0 0x7f2afc19fdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:52:28.395 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:28.391+0000 7f2afb7fe640 1 --2- 172.21.15.39:0/3259272194 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2afc103b60 0x7f2afc19a570 secure :-1 s=READY pgs=978 cs=0 l=1 rev1=1 crypto rx=0x7f2aec00d950 tx=0x7f2aec00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:52:28.395 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:28.391+0000 7f2af8ff9640 1 -- 172.21.15.39:0/3259272194 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2aec017070 con 0x7f2afc103b60 2024-09-21T03:52:28.395 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:28.391+0000 7f2af8ff9640 1 -- 172.21.15.39:0/3259272194 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2aec009c30 con 0x7f2afc103b60 2024-09-21T03:52:28.395 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:28.391+0000 7f2b02186640 1 -- 172.21.15.39:0/3259272194 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2afc1a0710 con 0x7f2afc103b60 2024-09-21T03:52:28.395 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:28.391+0000 7f2af8ff9640 1 -- 172.21.15.39:0/3259272194 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2aec00f620 con 0x7f2afc103b60 2024-09-21T03:52:28.395 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:28.391+0000 7f2b02186640 1 -- 172.21.15.39:0/3259272194 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2afc1a6f30 con 0x7f2afc103b60 2024-09-21T03:52:28.397 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:28.394+0000 7f2b086fa740 1 Processor -- start 2024-09-21T03:52:28.397 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:28.394+0000 7f2b086fa740 1 -- start start 2024-09-21T03:52:28.397 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:28.394+0000 7f2af8ff9640 1 -- 172.21.15.39:0/3259272194 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 27) ==== 154643+0+0 (secure 0 0 0) 0x7f2aec010460 con 0x7f2afc103b60 2024-09-21T03:52:28.398 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:28.394+0000 7f2b086fa740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55bd7caa8070 0x55bd7ce4f2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:52:28.398 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:28.394+0000 7f2b086fa740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55bd7ca7f1b0 0x55bd7ce4f800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:52:28.398 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:28.394+0000 7f2b086fa740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55bd7ca87390 0x55bd7ce4fd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:52:28.398 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:28.394+0000 7f2b086fa740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x55bd7cdc5ea0 con 0x55bd7ca7f1b0 2024-09-21T03:52:28.398 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:28.394+0000 7f2b086fa740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x55bd7cdb52d0 con 0x55bd7ca87390 2024-09-21T03:52:28.398 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:28.394+0000 7f2b086fa740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55bd7cdb90e0 con 0x55bd7caa8070 2024-09-21T03:52:28.398 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:28.394+0000 7f2afb7fe640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55bd7ca7f1b0 0x55bd7ce4f800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:52:28.398 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:28.394+0000 7f2afb7fe640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55bd7ca7f1b0 0x55bd7ce4f800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:49800/0 (socket says 172.21.15.39:49800) 2024-09-21T03:52:28.398 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:28.394+0000 7f2afb7fe640 1 -- 172.21.15.39:0/2435014376 learned_addr learned my addr 172.21.15.39:0/2435014376 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:52:28.398 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:28.394+0000 7f2afbfff640 1 --2- 172.21.15.39:0/2435014376 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55bd7caa8070 0x55bd7ce4f2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:52:28.398 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:28.395+0000 7f2afaffd640 1 --2- 172.21.15.39:0/2435014376 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55bd7ca87390 0x55bd7ce4fd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:52:28.398 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:28.395+0000 7f2afbfff640 1 -- 172.21.15.39:0/2435014376 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55bd7ca87390 msgr2=0x55bd7ce4fd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:52:28.399 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:28.395+0000 7f2afbfff640 1 --2- 172.21.15.39:0/2435014376 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55bd7ca87390 0x55bd7ce4fd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:28.399 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:28.395+0000 7f2afbfff640 1 -- 172.21.15.39:0/2435014376 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55bd7ca7f1b0 msgr2=0x55bd7ce4f800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:52:28.399 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:28.395+0000 7f2afbfff640 1 --2- 172.21.15.39:0/2435014376 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55bd7ca7f1b0 0x55bd7ce4f800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:28.399 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:28.395+0000 7f2afbfff640 1 -- 172.21.15.39:0/2435014376 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bd7cb13650 con 0x55bd7caa8070 2024-09-21T03:52:28.399 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:28.395+0000 7f2afaffd640 1 --2- 172.21.15.39:0/2435014376 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55bd7ca87390 0x55bd7ce4fd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:52:28.399 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:28.395+0000 7f2afbfff640 1 --2- 172.21.15.39:0/2435014376 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55bd7caa8070 0x55bd7ce4f2c0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f2af4007db0 tx=0x7f2af400e720 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c1118f97cb915ed2 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:52:28.399 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:28.395+0000 7f2af8ff9640 1 --2- 172.21.15.39:0/3259272194 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f2ae80b66b0 0x7f2ae80b8bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:52:28.399 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:28.395+0000 7f2afb7fe640 1 --2- 172.21.15.39:0/3259272194 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f2ae80b66b0 0x7f2ae80b8bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:52:28.399 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:28.395+0000 7f2af1ffb640 1 -- 172.21.15.39:0/2435014376 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2af4014070 con 0x55bd7caa8070 2024-09-21T03:52:28.399 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:28.396+0000 7f2af1ffb640 1 -- 172.21.15.39:0/2435014376 <== mon.1 v2:172.21.15.161:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2af4004d20 con 0x55bd7caa8070 2024-09-21T03:52:28.399 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:28.396+0000 7f2af1ffb640 1 -- 172.21.15.39:0/2435014376 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2af4010830 con 0x55bd7caa8070 2024-09-21T03:52:28.399 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:28.396+0000 7f2af8ff9640 1 -- 172.21.15.39:0/3259272194 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8484+0+0 (secure 0 0 0) 0x7f2aec0e7340 con 0x7f2afc103b60 2024-09-21T03:52:28.402 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:28.397+0000 7f2b086fa740 1 -- 172.21.15.39:0/2435014376 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55bd7caa8070 msgr2=0x55bd7ce4f2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:52:28.402 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:28.397+0000 7f2afb7fe640 1 --2- 172.21.15.39:0/3259272194 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f2ae80b66b0 0x7f2ae80b8bc0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f2aec0aceb0 tx=0x7f2aec0eae90 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:52:28.402 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:28.397+0000 7f2b086fa740 1 --2- 172.21.15.39:0/2435014376 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55bd7caa8070 0x55bd7ce4f2c0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f2af4007db0 tx=0x7f2af400e720 comp rx=0 tx=0).stop 2024-09-21T03:52:28.403 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:28.397+0000 7f2b086fa740 1 -- 172.21.15.39:0/2435014376 shutdown_connections 2024-09-21T03:52:28.403 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:28.397+0000 7f2b086fa740 1 --2- 172.21.15.39:0/2435014376 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55bd7ca87390 0x55bd7ce4fd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:28.403 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:28.397+0000 7f2b086fa740 1 --2- 172.21.15.39:0/2435014376 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55bd7ca7f1b0 0x55bd7ce4f800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:28.403 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:28.397+0000 7f2b086fa740 1 --2- 172.21.15.39:0/2435014376 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55bd7caa8070 0x55bd7ce4f2c0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:28.403 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:28.397+0000 7f2b086fa740 1 -- 172.21.15.39:0/2435014376 >> 172.21.15.39:0/2435014376 conn(0x55bd7ca9c460 msgr2=0x55bd7ce4d1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:52:28.403 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:28.397+0000 7f2b086fa740 1 -- 172.21.15.39:0/2435014376 shutdown_connections 2024-09-21T03:52:28.403 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:28.397+0000 7f2b086fa740 1 -- 172.21.15.39:0/2435014376 wait complete. 2024-09-21T03:52:28.403 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:28.397+0000 7f2b086fa740 1 -- start start 2024-09-21T03:52:28.403 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:28.397+0000 7f2b086fa740 1 Processor -- start 2024-09-21T03:52:28.403 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:28.398+0000 7f2b086fa740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55bd7ca7f1b0 0x55bd7ce5a990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:52:28.403 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:28.398+0000 7f2b086fa740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55bd7ca87390 0x55bd7ce5aed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:52:28.403 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:28.398+0000 7f2b086fa740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55bd7caa8070 0x55bd7ce66e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:52:28.403 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:28.398+0000 7f2b086fa740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x55bd7cc16700 con 0x55bd7ca7f1b0 2024-09-21T03:52:28.403 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:28.398+0000 7f2b086fa740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x55bd7cd50160 con 0x55bd7ca87390 2024-09-21T03:52:28.404 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:28.398+0000 7f2b086fa740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55bd7ce5daa0 con 0x55bd7caa8070 2024-09-21T03:52:28.404 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:28.398+0000 7f2b086fa740 10 client.0 _ll_register_callbacks cb 0x55bd7cd699e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:52:28.404 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:28.398+0000 7f2afbfff640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55bd7ca7f1b0 0x55bd7ce5a990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:52:28.404 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:28.398+0000 7f2afbfff640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55bd7ca7f1b0 0x55bd7ce5a990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:49802/0 (socket says 172.21.15.39:49802) 2024-09-21T03:52:28.404 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:28.398+0000 7f2afbfff640 1 -- 172.21.15.39:0/3421770066 learned_addr learned my addr 172.21.15.39:0/3421770066 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:52:28.404 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:28.398+0000 7f2afaffd640 1 --2- 172.21.15.39:0/3421770066 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55bd7ca87390 0x55bd7ce5aed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:52:28.404 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:28.398+0000 7f2afbfff640 1 --2- 172.21.15.39:0/3421770066 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55bd7caa8070 0x55bd7ce66e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:52:28.404 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:28.398+0000 7f2afbfff640 1 -- 172.21.15.39:0/3421770066 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55bd7caa8070 msgr2=0x55bd7ce66e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:52:28.404 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:28.398+0000 7f2afbfff640 1 --2- 172.21.15.39:0/3421770066 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55bd7caa8070 0x55bd7ce66e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:28.404 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:28.398+0000 7f2afbfff640 1 -- 172.21.15.39:0/3421770066 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55bd7ca87390 msgr2=0x55bd7ce5aed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:52:28.404 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:28.398+0000 7f2afbfff640 1 --2- 172.21.15.39:0/3421770066 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55bd7ca87390 0x55bd7ce5aed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:28.404 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:28.398+0000 7f2afbfff640 1 -- 172.21.15.39:0/3421770066 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bd7cd7dc90 con 0x55bd7ca7f1b0 2024-09-21T03:52:28.404 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:28.398+0000 7f2afbfff640 1 --2- 172.21.15.39:0/3421770066 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55bd7ca7f1b0 0x55bd7ce5a990 secure :-1 s=READY pgs=979 cs=0 l=1 rev1=1 crypto rx=0x7f2af4007510 tx=0x7f2af40074a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ff4a57d8d68c9dc server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:52:28.404 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:28.398+0000 7f2adaffd640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-09-21T03:52:28.404 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:28.399+0000 7f2adaffd640 1 -- 172.21.15.39:0/3421770066 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2af4014070 con 0x55bd7ca7f1b0 2024-09-21T03:52:28.404 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:28.399+0000 7f2adaffd640 1 -- 172.21.15.39:0/3421770066 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8484+0+0 (secure 0 0 0) 0x7f2af4020040 con 0x55bd7ca7f1b0 2024-09-21T03:52:28.405 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:28.399+0000 7f2b086fa740 1 -- 172.21.15.39:0/3421770066 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2af4004d20 con 0x55bd7ca7f1b0 2024-09-21T03:52:28.405 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:28.399+0000 7f2adaffd640 1 -- 172.21.15.39:0/3421770066 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f2af40055b0 con 0x55bd7ca7f1b0 2024-09-21T03:52:28.405 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:28.399+0000 7f2b086fa740 10 client.17217 fetch_fsmap learned FSMap version 29 2024-09-21T03:52:28.405 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:28.399+0000 7f2b086fa740 1 -- 172.21.15.39:0/3421770066 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55bd7ce69800 con 0x55bd7ca7f1b0 2024-09-21T03:52:28.405 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:28.399+0000 7f2adaffd640 1 -- 172.21.15.39:0/3421770066 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2af4004d20 con 0x55bd7ca7f1b0 2024-09-21T03:52:28.405 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:28.399+0000 7f2adaffd640 1 -- 172.21.15.39:0/3421770066 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f2af402e050 con 0x55bd7ca7f1b0 2024-09-21T03:52:28.405 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:28.399+0000 7f2b086fa740 10 client.17217 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T03:52:28.405 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:28.399+0000 7f2b086fa740 10 client.17217 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-21T03:52:28.405 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:28.399+0000 7f2b086fa740 20 client.17217 populate_metadata read hostname 'smithi039' 2024-09-21T03:52:28.405 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:28.399+0000 7f2b086fa740 1 --2- 172.21.15.39:0/3421770066 >> [v2:172.21.15.161:6838/2190107270,v1:172.21.15.161:6839/2190107270] conn(0x55bd7ca669f0 0x55bd7ce6cb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:52:28.405 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:28.399+0000 7f2b086fa740 4 client.17217 mds_command: new command op to 24667 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:52:28.405 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:28.399+0000 7f2b086fa740 1 -- 172.21.15.39:0/3421770066 --> [v2:172.21.15.161:6838/2190107270,v1:172.21.15.161:6839/2190107270] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bd7cdc7df0 con 0x55bd7ca669f0 2024-09-21T03:52:28.405 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:28.400+0000 7f2afaffd640 1 --2- 172.21.15.39:0/3421770066 >> [v2:172.21.15.161:6838/2190107270,v1:172.21.15.161:6839/2190107270] conn(0x55bd7ca669f0 0x55bd7ce6cb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:52:28.405 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:28.400+0000 7f2afaffd640 1 --2- 172.21.15.39:0/3421770066 >> [v2:172.21.15.161:6838/2190107270,v1:172.21.15.161:6839/2190107270] conn(0x55bd7ca669f0 0x55bd7ce6cb60 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=2591fce6109d7652 server_cookie=16d555dfc49ccbd3 in_seq=0 out_seq=0 2024-09-21T03:52:28.406 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:28.400+0000 7f2adaffd640 10 client.17217 ms_handle_connect on v2:172.21.15.161:6838/2190107270 2024-09-21T03:52:28.406 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:28.401+0000 7f2adaffd640 1 -- 172.21.15.39:0/3421770066 <== mds.1 v2:172.21.15.161:6838/2190107270 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55bd7cdc7df0 con 0x55bd7ca669f0 2024-09-21T03:52:28.407 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:28.401+0000 7f2adaffd640 10 client.17217 handle_command_reply: tid=0 2024-09-21T03:52:28.407 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:28.401+0000 7f2b086fa740 1 client.17217 shutdown 2024-09-21T03:52:28.407 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:28.401+0000 7f2b086fa740 1 -- 172.21.15.39:0/3421770066 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55bd7ca7f1b0 msgr2=0x55bd7ce5a990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:52:28.407 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:28.401+0000 7f2b086fa740 1 --2- 172.21.15.39:0/3421770066 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55bd7ca7f1b0 0x55bd7ce5a990 secure :-1 s=READY pgs=979 cs=0 l=1 rev1=1 crypto rx=0x7f2af4007510 tx=0x7f2af40074a0 comp rx=0 tx=0).stop 2024-09-21T03:52:28.407 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:28.401+0000 7f2b086fa740 1 -- 172.21.15.39:0/3421770066 shutdown_connections 2024-09-21T03:52:28.407 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:28.401+0000 7f2b086fa740 1 --2- 172.21.15.39:0/3421770066 >> [v2:172.21.15.161:6838/2190107270,v1:172.21.15.161:6839/2190107270] conn(0x55bd7ca669f0 0x55bd7ce6cb60 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:28.407 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:28.401+0000 7f2b086fa740 1 --2- 172.21.15.39:0/3421770066 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55bd7caa8070 0x55bd7ce66e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:28.407 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:28.401+0000 7f2b086fa740 1 --2- 172.21.15.39:0/3421770066 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55bd7ca87390 0x55bd7ce5aed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:28.407 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:28.401+0000 7f2b086fa740 1 --2- 172.21.15.39:0/3421770066 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55bd7ca7f1b0 0x55bd7ce5a990 unknown :-1 s=CLOSED pgs=979 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:28.407 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:28.401+0000 7f2b086fa740 1 -- 172.21.15.39:0/3421770066 >> 172.21.15.39:0/3421770066 conn(0x55bd7ca9c460 msgr2=0x55bd7ce52450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:52:28.407 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:28.401+0000 7f2adaffd640 0 client.17217 ms_handle_reset on v2:172.21.15.161:6838/2190107270 2024-09-21T03:52:28.407 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:28.401+0000 7f2b086fa740 1 -- 172.21.15.39:0/3421770066 shutdown_connections 2024-09-21T03:52:28.407 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:28.401+0000 7f2b086fa740 1 -- 172.21.15.39:0/3421770066 wait complete. 2024-09-21T03:52:28.407 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:28.402+0000 7f2b086fa740 20 client.17217 trim_cache size 0 max 16384 2024-09-21T03:52:28.417 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:28.415+0000 7f2b086fa740 1 Processor -- start 2024-09-21T03:52:28.418 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:28.415+0000 7f2b086fa740 1 -- start start 2024-09-21T03:52:28.418 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:28.415+0000 7f2b086fa740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55bd7ca7f1b0 0x55bd7ce5ae50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:52:28.418 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:28.415+0000 7f2b086fa740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55bd7ca87390 0x55bd7ce523c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:52:28.418 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:28.415+0000 7f2b086fa740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55bd7caa8070 0x55bd7ce52900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:52:28.418 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:28.415+0000 7f2b086fa740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f2afc1af820 con 0x55bd7ca87390 2024-09-21T03:52:28.418 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:28.415+0000 7f2b086fa740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f2afc1b1360 con 0x55bd7ca7f1b0 2024-09-21T03:52:28.418 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:28.415+0000 7f2b086fa740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f2afc1afef0 con 0x55bd7caa8070 2024-09-21T03:52:28.418 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:28.415+0000 7f2afaffd640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55bd7ca87390 0x55bd7ce523c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:52:28.418 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:28.415+0000 7f2afaffd640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55bd7ca87390 0x55bd7ce523c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:49804/0 (socket says 172.21.15.39:49804) 2024-09-21T03:52:28.418 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:28.415+0000 7f2afaffd640 1 -- 172.21.15.39:0/327292686 learned_addr learned my addr 172.21.15.39:0/327292686 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:52:28.418 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:28.415+0000 7f2afbfff640 1 --2- 172.21.15.39:0/327292686 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55bd7ca7f1b0 0x55bd7ce5ae50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:52:28.418 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:28.415+0000 7f2afbfff640 1 --2- 172.21.15.39:0/327292686 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55bd7caa8070 0x55bd7ce52900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:52:28.418 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:28.415+0000 7f2afaffd640 1 -- 172.21.15.39:0/327292686 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55bd7caa8070 msgr2=0x55bd7ce52900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:52:28.418 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:28.415+0000 7f2afaffd640 1 --2- 172.21.15.39:0/327292686 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55bd7caa8070 0x55bd7ce52900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:28.419 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:28.415+0000 7f2afaffd640 1 -- 172.21.15.39:0/327292686 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55bd7ca7f1b0 msgr2=0x55bd7ce5ae50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:52:28.419 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:28.415+0000 7f2afaffd640 1 --2- 172.21.15.39:0/327292686 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55bd7ca7f1b0 0x55bd7ce5ae50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:28.419 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:28.415+0000 7f2afaffd640 1 -- 172.21.15.39:0/327292686 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bd7ce5bc50 con 0x55bd7ca87390 2024-09-21T03:52:28.419 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:28.415+0000 7f2afbfff640 1 --2- 172.21.15.39:0/327292686 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55bd7caa8070 0x55bd7ce52900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:52:28.419 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:28.416+0000 7f2afaffd640 1 --2- 172.21.15.39:0/327292686 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55bd7ca87390 0x55bd7ce523c0 secure :-1 s=READY pgs=980 cs=0 l=1 rev1=1 crypto rx=0x7f2ae4000c00 tx=0x7f2ae4005a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=250e3cd87f410964 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:52:28.419 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:28.416+0000 7f2af17fa640 1 -- 172.21.15.39:0/327292686 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ae4049020 con 0x55bd7ca87390 2024-09-21T03:52:28.419 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:28.416+0000 7f2af17fa640 1 -- 172.21.15.39:0/327292686 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2ae4004750 con 0x55bd7ca87390 2024-09-21T03:52:28.419 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:28.416+0000 7f2af17fa640 1 -- 172.21.15.39:0/327292686 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ae4035040 con 0x55bd7ca87390 2024-09-21T03:52:28.423 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:28.417+0000 7f2b086fa740 1 -- 172.21.15.39:0/327292686 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55bd7ca87390 msgr2=0x55bd7ce523c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:52:28.423 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:28.417+0000 7f2b086fa740 1 --2- 172.21.15.39:0/327292686 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55bd7ca87390 0x55bd7ce523c0 secure :-1 s=READY pgs=980 cs=0 l=1 rev1=1 crypto rx=0x7f2ae4000c00 tx=0x7f2ae4005a20 comp rx=0 tx=0).stop 2024-09-21T03:52:28.423 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:28.417+0000 7f2b086fa740 1 -- 172.21.15.39:0/327292686 shutdown_connections 2024-09-21T03:52:28.423 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:28.417+0000 7f2b086fa740 1 --2- 172.21.15.39:0/327292686 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55bd7caa8070 0x55bd7ce52900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:28.423 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:28.417+0000 7f2b086fa740 1 --2- 172.21.15.39:0/327292686 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55bd7ca87390 0x55bd7ce523c0 unknown :-1 s=CLOSED pgs=980 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:28.423 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:28.417+0000 7f2b086fa740 1 --2- 172.21.15.39:0/327292686 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55bd7ca7f1b0 0x55bd7ce5ae50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:28.423 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:28.417+0000 7f2b086fa740 1 -- 172.21.15.39:0/327292686 >> 172.21.15.39:0/327292686 conn(0x55bd7ca9c460 msgr2=0x55bd7ce5a740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:52:28.423 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:28.417+0000 7f2b086fa740 1 -- 172.21.15.39:0/327292686 shutdown_connections 2024-09-21T03:52:28.423 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:28.417+0000 7f2b086fa740 1 -- 172.21.15.39:0/327292686 wait complete. 2024-09-21T03:52:28.423 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:28.417+0000 7f2b086fa740 1 -- start start 2024-09-21T03:52:28.423 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:28.417+0000 7f2b086fa740 1 Processor -- start 2024-09-21T03:52:28.424 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:28.418+0000 7f2b086fa740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55bd7ca7f1b0 0x55bd7ce70220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:52:28.424 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:28.418+0000 7f2b086fa740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55bd7ca87390 0x55bd7ce6d740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:52:28.424 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:28.418+0000 7f2b086fa740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55bd7caa8070 0x55bd7ce6dc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:52:28.424 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:28.418+0000 7f2b086fa740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f2afc1b0c90 con 0x55bd7caa8070 2024-09-21T03:52:28.424 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:28.418+0000 7f2b086fa740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f2afc1b05c0 con 0x55bd7ca7f1b0 2024-09-21T03:52:28.424 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:28.418+0000 7f2b086fa740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f2afc1ad800 con 0x55bd7ca87390 2024-09-21T03:52:28.424 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:28.418+0000 7f2b086fa740 10 client.0 _ll_register_callbacks cb 0x7f2aec0ead60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:52:28.424 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:28.418+0000 7f2afbfff640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55bd7caa8070 0x55bd7ce6dc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:52:28.424 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:28.418+0000 7f2afbfff640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55bd7caa8070 0x55bd7ce6dc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:49810/0 (socket says 172.21.15.39:49810) 2024-09-21T03:52:28.424 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:28.418+0000 7f2afbfff640 1 -- 172.21.15.39:0/3653948841 learned_addr learned my addr 172.21.15.39:0/3653948841 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:52:28.424 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:28.418+0000 7f2afbfff640 1 --2- 172.21.15.39:0/3653948841 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55bd7ca7f1b0 0x55bd7ce70220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:52:28.424 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:28.418+0000 7f2afaffd640 1 --2- 172.21.15.39:0/3653948841 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55bd7ca87390 0x55bd7ce6d740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:52:28.424 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:28.418+0000 7f2afbfff640 1 -- 172.21.15.39:0/3653948841 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55bd7ca87390 msgr2=0x55bd7ce6d740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:52:28.424 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:28.418+0000 7f2afbfff640 1 --2- 172.21.15.39:0/3653948841 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55bd7ca87390 0x55bd7ce6d740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:28.424 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:28.418+0000 7f2afbfff640 1 -- 172.21.15.39:0/3653948841 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55bd7ca7f1b0 msgr2=0x55bd7ce70220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:52:28.425 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:28.418+0000 7f2afbfff640 1 --2- 172.21.15.39:0/3653948841 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55bd7ca7f1b0 0x55bd7ce70220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:28.425 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:28.418+0000 7f2afbfff640 1 -- 172.21.15.39:0/3653948841 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bd7ce6e2c0 con 0x55bd7caa8070 2024-09-21T03:52:28.425 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:28.419+0000 7f2afbfff640 1 --2- 172.21.15.39:0/3653948841 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55bd7caa8070 0x55bd7ce6dc80 secure :-1 s=READY pgs=981 cs=0 l=1 rev1=1 crypto rx=0x7f2af4004720 tx=0x7f2af40052f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df75aed85080c9c9 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:52:28.425 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:28.419+0000 7f2adaffd640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-09-21T03:52:28.425 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:28.419+0000 7f2adaffd640 1 -- 172.21.15.39:0/3653948841 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2af4014040 con 0x55bd7caa8070 2024-09-21T03:52:28.425 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:28.419+0000 7f2adaffd640 1 -- 172.21.15.39:0/3653948841 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8484+0+0 (secure 0 0 0) 0x7f2af4020070 con 0x55bd7caa8070 2024-09-21T03:52:28.425 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:28.419+0000 7f2b086fa740 1 -- 172.21.15.39:0/3653948841 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2ae4004750 con 0x55bd7caa8070 2024-09-21T03:52:28.425 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:28.419+0000 7f2adaffd640 1 -- 172.21.15.39:0/3653948841 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f2af4041050 con 0x55bd7caa8070 2024-09-21T03:52:28.425 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:28.419+0000 7f2b086fa740 10 client.17223 fetch_fsmap learned FSMap version 29 2024-09-21T03:52:28.425 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:28.419+0000 7f2b086fa740 1 -- 172.21.15.39:0/3653948841 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55bd7ced6fb0 con 0x55bd7caa8070 2024-09-21T03:52:28.425 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:28.419+0000 7f2adaffd640 1 -- 172.21.15.39:0/3653948841 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2ae4004750 con 0x55bd7caa8070 2024-09-21T03:52:28.425 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:28.419+0000 7f2adaffd640 1 -- 172.21.15.39:0/3653948841 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f2af402e040 con 0x55bd7caa8070 2024-09-21T03:52:28.425 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:28.419+0000 7f2b086fa740 10 client.17223 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T03:52:28.425 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:28.419+0000 7f2b086fa740 10 client.17223 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-21T03:52:28.426 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:28.419+0000 7f2b086fa740 20 client.17223 populate_metadata read hostname 'smithi039' 2024-09-21T03:52:28.426 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:28.419+0000 7f2b086fa740 1 --2- 172.21.15.39:0/3653948841 >> [v2:172.21.15.161:6838/2190107270,v1:172.21.15.161:6839/2190107270] conn(0x55bd7ca3bda0 0x55bd7ced96e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:52:28.426 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:28.419+0000 7f2b086fa740 4 client.17223 mds_command: new command op to 24667 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T03:52:28.426 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:28.419+0000 7f2b086fa740 1 -- 172.21.15.39:0/3653948841 --> [v2:172.21.15.161:6838/2190107270,v1:172.21.15.161:6839/2190107270] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55bd7cdc38a0 con 0x55bd7ca3bda0 2024-09-21T03:52:28.426 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:28.420+0000 7f2afaffd640 1 --2- 172.21.15.39:0/3653948841 >> [v2:172.21.15.161:6838/2190107270,v1:172.21.15.161:6839/2190107270] conn(0x55bd7ca3bda0 0x55bd7ced96e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:52:28.426 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:28.420+0000 7f2afaffd640 1 --2- 172.21.15.39:0/3653948841 >> [v2:172.21.15.161:6838/2190107270,v1:172.21.15.161:6839/2190107270] conn(0x55bd7ca3bda0 0x55bd7ced96e0 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=83967cd58b63a8bc server_cookie=d544e5846760910d in_seq=0 out_seq=0 2024-09-21T03:52:28.426 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:28.420+0000 7f2adaffd640 10 client.17223 ms_handle_connect on v2:172.21.15.161:6838/2190107270 2024-09-21T03:52:28.426 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:28.420+0000 7f2adaffd640 1 -- 172.21.15.39:0/3653948841 <== mds.1 v2:172.21.15.161:6838/2190107270 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55bd7cdc38a0 con 0x55bd7ca3bda0 2024-09-21T03:52:28.426 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:28.420+0000 7f2adaffd640 10 client.17223 handle_command_reply: tid=0 2024-09-21T03:52:28.426 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:28.420+0000 7f2b086fa740 1 client.17223 shutdown 2024-09-21T03:52:28.426 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:28.421+0000 7f2b086fa740 1 -- 172.21.15.39:0/3653948841 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55bd7caa8070 msgr2=0x55bd7ce6dc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:52:28.426 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:28.421+0000 7f2b086fa740 1 --2- 172.21.15.39:0/3653948841 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55bd7caa8070 0x55bd7ce6dc80 secure :-1 s=READY pgs=981 cs=0 l=1 rev1=1 crypto rx=0x7f2af4004720 tx=0x7f2af40052f0 comp rx=0 tx=0).stop 2024-09-21T03:52:28.426 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:28.421+0000 7f2b086fa740 1 -- 172.21.15.39:0/3653948841 shutdown_connections 2024-09-21T03:52:28.426 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:28.421+0000 7f2b086fa740 1 --2- 172.21.15.39:0/3653948841 >> [v2:172.21.15.161:6838/2190107270,v1:172.21.15.161:6839/2190107270] conn(0x55bd7ca3bda0 0x55bd7ced96e0 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:28.427 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:28.421+0000 7f2b086fa740 1 --2- 172.21.15.39:0/3653948841 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55bd7caa8070 0x55bd7ce6dc80 unknown :-1 s=CLOSED pgs=981 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:28.427 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:28.421+0000 7f2b086fa740 1 --2- 172.21.15.39:0/3653948841 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55bd7ca87390 0x55bd7ce6d740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:28.427 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:28.421+0000 7f2b086fa740 1 --2- 172.21.15.39:0/3653948841 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55bd7ca7f1b0 0x55bd7ce70220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:28.427 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:28.421+0000 7f2b086fa740 1 -- 172.21.15.39:0/3653948841 >> 172.21.15.39:0/3653948841 conn(0x55bd7ca9c460 msgr2=0x55bd7ce523c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:52:28.427 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:28.421+0000 7f2adaffd640 0 client.17223 ms_handle_reset on v2:172.21.15.161:6838/2190107270 2024-09-21T03:52:28.427 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:28.421+0000 7f2b086fa740 1 -- 172.21.15.39:0/3653948841 shutdown_connections 2024-09-21T03:52:28.427 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:28.421+0000 7f2b086fa740 1 -- 172.21.15.39:0/3653948841 wait complete. 2024-09-21T03:52:28.427 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:28.421+0000 7f2b086fa740 20 client.17223 trim_cache size 0 max 16384 2024-09-21T03:52:28.427 INFO:teuthology.orchestra.run.smithi039.stdout:[] 2024-09-21T03:52:28.428 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:28.421+0000 7f2b02186640 1 -- 172.21.15.39:0/3259272194 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f2ae80b66b0 msgr2=0x7f2ae80b8bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:52:28.428 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:28.421+0000 7f2b02186640 1 --2- 172.21.15.39:0/3259272194 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f2ae80b66b0 0x7f2ae80b8bc0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f2aec0aceb0 tx=0x7f2aec0eae90 comp rx=0 tx=0).stop 2024-09-21T03:52:28.428 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:28.421+0000 7f2b02186640 1 -- 172.21.15.39:0/3259272194 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2afc103b60 msgr2=0x7f2afc19a570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:52:28.428 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:28.421+0000 7f2b02186640 1 --2- 172.21.15.39:0/3259272194 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2afc103b60 0x7f2afc19a570 secure :-1 s=READY pgs=978 cs=0 l=1 rev1=1 crypto rx=0x7f2aec00d950 tx=0x7f2aec00de10 comp rx=0 tx=0).stop 2024-09-21T03:52:28.428 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:28.422+0000 7f2b02186640 1 -- 172.21.15.39:0/3259272194 shutdown_connections 2024-09-21T03:52:28.428 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:28.422+0000 7f2b02186640 1 --2- 172.21.15.39:0/3259272194 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f2ae80b66b0 0x7f2ae80b8bc0 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:28.428 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:28.422+0000 7f2b02186640 1 --2- 172.21.15.39:0/3259272194 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2afc109df0 0x7f2afc19fdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:28.428 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:28.422+0000 7f2b02186640 1 --2- 172.21.15.39:0/3259272194 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2afc1094a0 0x7f2afc19ab10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:28.428 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:28.422+0000 7f2b02186640 1 --2- 172.21.15.39:0/3259272194 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2afc103b60 0x7f2afc19a570 unknown :-1 s=CLOSED pgs=978 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:28.428 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:28.422+0000 7f2b02186640 1 -- 172.21.15.39:0/3259272194 >> 172.21.15.39:0/3259272194 conn(0x7f2afc07b850 msgr2=0x7f2afc1055c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:52:28.428 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:28.422+0000 7f2b02186640 1 -- 172.21.15.39:0/3259272194 shutdown_connections 2024-09-21T03:52:28.429 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:28.422+0000 7f2b02186640 1 -- 172.21.15.39:0/3259272194 wait complete. 2024-09-21T03:52:29.062 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 51116d5c-77ca-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 damage ls 2024-09-21T03:52:29.325 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/51116d5c-77ca-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:52:29.334 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:52:29 smithi039 ceph-mon[31211]: pgmap v518: 97 pgs: 97 active+clean; 10 MiB data, 520 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 8 op/s 2024-09-21T03:52:29.334 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:52:29 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:52:29.718 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:52:29 smithi161 ceph-mon[34533]: pgmap v518: 97 pgs: 97 active+clean; 10 MiB data, 520 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 8 op/s 2024-09-21T03:52:29.718 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:52:29 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:52:29.724 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:52:29 smithi150 ceph-mon[34289]: pgmap v518: 97 pgs: 97 active+clean; 10 MiB data, 520 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 8 op/s 2024-09-21T03:52:29.725 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:52:29 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:52:29.799 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:29.797+0000 7f560d02f640 1 -- 172.21.15.39:0/37896696 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f56081084c0 msgr2=0x7f560810ecf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:52:29.800 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:29.797+0000 7f560d02f640 1 --2- 172.21.15.39:0/37896696 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f56081084c0 0x7f560810ecf0 secure :-1 s=READY pgs=982 cs=0 l=1 rev1=1 crypto rx=0x7f5600009ca0 tx=0x7f560002a8c0 comp rx=0 tx=0).stop 2024-09-21T03:52:29.800 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:29.797+0000 7f560d02f640 1 -- 172.21.15.39:0/37896696 shutdown_connections 2024-09-21T03:52:29.800 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:29.798+0000 7f560d02f640 1 --2- 172.21.15.39:0/37896696 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f56081084c0 0x7f560810ecf0 unknown :-1 s=CLOSED pgs=982 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:29.800 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:29.798+0000 7f560d02f640 1 --2- 172.21.15.39:0/37896696 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5608107b70 0x7f5608107f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:29.800 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:29.798+0000 7f560d02f640 1 --2- 172.21.15.39:0/37896696 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5608102230 0x7f5608102640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:29.800 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:29.798+0000 7f560d02f640 1 -- 172.21.15.39:0/37896696 >> 172.21.15.39:0/37896696 conn(0x7f5608079e40 msgr2=0x7f560807a250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:52:29.800 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:29.798+0000 7f560d02f640 1 -- 172.21.15.39:0/37896696 shutdown_connections 2024-09-21T03:52:29.800 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:29.798+0000 7f560d02f640 1 -- 172.21.15.39:0/37896696 wait complete. 2024-09-21T03:52:29.801 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:29.798+0000 7f560d02f640 1 Processor -- start 2024-09-21T03:52:29.801 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:29.799+0000 7f560d02f640 1 -- start start 2024-09-21T03:52:29.801 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:29.799+0000 7f560d02f640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5608102230 0x7f560819a420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:52:29.801 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:29.799+0000 7f560d02f640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5608107b70 0x7f560819a9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:52:29.801 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:29.799+0000 7f560d02f640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f56081084c0 0x7f560819fc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:52:29.801 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:29.799+0000 7f560d02f640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f5608112190 con 0x7f5608107b70 2024-09-21T03:52:29.801 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:29.799+0000 7f560d02f640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f5608112010 con 0x7f5608102230 2024-09-21T03:52:29.801 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:29.799+0000 7f560d02f640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f5608112310 con 0x7f56081084c0 2024-09-21T03:52:29.802 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:29.799+0000 7f560659c640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5608107b70 0x7f560819a9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:52:29.802 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:29.799+0000 7f5606d9d640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5608102230 0x7f560819a420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:52:29.802 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:29.799+0000 7f560659c640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5608107b70 0x7f560819a9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:49820/0 (socket says 172.21.15.39:49820) 2024-09-21T03:52:29.802 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:29.799+0000 7f560659c640 1 -- 172.21.15.39:0/3511466501 learned_addr learned my addr 172.21.15.39:0/3511466501 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:52:29.802 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:29.799+0000 7f560759e640 1 --2- 172.21.15.39:0/3511466501 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f56081084c0 0x7f560819fc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:52:29.802 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:29.799+0000 7f560659c640 1 -- 172.21.15.39:0/3511466501 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f56081084c0 msgr2=0x7f560819fc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:52:29.802 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:29.799+0000 7f560659c640 1 --2- 172.21.15.39:0/3511466501 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f56081084c0 0x7f560819fc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:29.802 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:29.799+0000 7f560659c640 1 -- 172.21.15.39:0/3511466501 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5608102230 msgr2=0x7f560819a420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:52:29.802 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:29.799+0000 7f560659c640 1 --2- 172.21.15.39:0/3511466501 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5608102230 0x7f560819a420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:29.802 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:29.799+0000 7f560659c640 1 -- 172.21.15.39:0/3511466501 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f56081a0390 con 0x7f5608107b70 2024-09-21T03:52:29.802 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:29.799+0000 7f5606d9d640 1 --2- 172.21.15.39:0/3511466501 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5608102230 0x7f560819a420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:52:29.803 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:29.800+0000 7f560659c640 1 --2- 172.21.15.39:0/3511466501 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5608107b70 0x7f560819a9c0 secure :-1 s=READY pgs=983 cs=0 l=1 rev1=1 crypto rx=0x7f55fc00d750 tx=0x7f55fc00dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:52:29.803 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:29.800+0000 7f55effff640 1 -- 172.21.15.39:0/3511466501 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55fc017070 con 0x7f5608107b70 2024-09-21T03:52:29.803 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:29.800+0000 7f560d02f640 1 -- 172.21.15.39:0/3511466501 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f56081a0680 con 0x7f5608107b70 2024-09-21T03:52:29.803 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:29.800+0000 7f55effff640 1 -- 172.21.15.39:0/3511466501 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f55fc0054d0 con 0x7f5608107b70 2024-09-21T03:52:29.803 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:29.800+0000 7f55effff640 1 -- 172.21.15.39:0/3511466501 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55fc004c30 con 0x7f5608107b70 2024-09-21T03:52:29.803 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:29.800+0000 7f560d02f640 1 -- 172.21.15.39:0/3511466501 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f56081a6de0 con 0x7f5608107b70 2024-09-21T03:52:29.805 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:29.803+0000 7f56135a3740 1 Processor -- start 2024-09-21T03:52:29.805 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:29.803+0000 7f56135a3740 1 -- start start 2024-09-21T03:52:29.805 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:29.803+0000 7f55effff640 1 -- 172.21.15.39:0/3511466501 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 27) ==== 154643+0+0 (secure 0 0 0) 0x7f55fc004dd0 con 0x7f5608107b70 2024-09-21T03:52:29.805 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:29.803+0000 7f56135a3740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5596efef8070 0x5596f029f680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:52:29.806 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:29.803+0000 7f56135a3740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5596efecf1b0 0x5596f029fbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:52:29.806 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:29.803+0000 7f56135a3740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5596efed7390 0x5596f02a0100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:52:29.806 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:29.803+0000 7f56135a3740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x5596f0215ea0 con 0x5596efef8070 2024-09-21T03:52:29.806 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:29.803+0000 7f56135a3740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x5596f02052d0 con 0x5596efecf1b0 2024-09-21T03:52:29.806 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:29.803+0000 7f56135a3740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x5596f02090e0 con 0x5596efed7390 2024-09-21T03:52:29.806 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:29.803+0000 7f560759e640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5596efef8070 0x5596f029f680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:52:29.806 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:29.803+0000 7f560759e640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5596efef8070 0x5596f029f680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:49826/0 (socket says 172.21.15.39:49826) 2024-09-21T03:52:29.806 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:29.803+0000 7f560759e640 1 -- 172.21.15.39:0/278631051 learned_addr learned my addr 172.21.15.39:0/278631051 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:52:29.806 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:29.803+0000 7f5606d9d640 1 --2- 172.21.15.39:0/278631051 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5596efecf1b0 0x5596f029fbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:52:29.806 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:29.803+0000 7f560659c640 1 --2- 172.21.15.39:0/278631051 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5596efed7390 0x5596f02a0100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:52:29.806 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:29.804+0000 7f560759e640 1 -- 172.21.15.39:0/278631051 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5596efed7390 msgr2=0x5596f02a0100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:52:29.806 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:29.804+0000 7f560759e640 1 --2- 172.21.15.39:0/278631051 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5596efed7390 0x5596f02a0100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:29.806 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:29.804+0000 7f560759e640 1 -- 172.21.15.39:0/278631051 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5596efecf1b0 msgr2=0x5596f029fbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:52:29.806 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:29.804+0000 7f560759e640 1 --2- 172.21.15.39:0/278631051 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5596efecf1b0 0x5596f029fbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:29.806 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:29.804+0000 7f560759e640 1 -- 172.21.15.39:0/278631051 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5596eff63650 con 0x5596efef8070 2024-09-21T03:52:29.809 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:29.804+0000 7f55effff640 1 --2- 172.21.15.39:0/3511466501 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f55f00b6680 0x7f55f00b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:52:29.809 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:29.804+0000 7f560759e640 1 --2- 172.21.15.39:0/278631051 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5596efef8070 0x5596f029f680 secure :-1 s=READY pgs=984 cs=0 l=1 rev1=1 crypto rx=0x7f56000098b0 tx=0x7f56000098e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc91a287888323 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:52:29.810 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:29.804+0000 7f5606d9d640 1 --2- 172.21.15.39:0/3511466501 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f55f00b6680 0x7f55f00b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:52:29.810 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:29.804+0000 7f55effff640 1 -- 172.21.15.39:0/3511466501 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8484+0+0 (secure 0 0 0) 0x7f55fc0e7070 con 0x7f5608107b70 2024-09-21T03:52:29.810 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:29.804+0000 7f55ed7fa640 1 -- 172.21.15.39:0/278631051 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5600031cf0 con 0x5596efef8070 2024-09-21T03:52:29.810 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:29.804+0000 7f55ed7fa640 1 -- 172.21.15.39:0/278631051 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5600036030 con 0x5596efef8070 2024-09-21T03:52:29.810 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:29.804+0000 7f55ed7fa640 1 -- 172.21.15.39:0/278631051 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5600004240 con 0x5596efef8070 2024-09-21T03:52:29.810 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:29.805+0000 7f56135a3740 1 -- 172.21.15.39:0/278631051 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5596efef8070 msgr2=0x5596f029f680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:52:29.810 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:29.805+0000 7f56135a3740 1 --2- 172.21.15.39:0/278631051 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5596efef8070 0x5596f029f680 secure :-1 s=READY pgs=984 cs=0 l=1 rev1=1 crypto rx=0x7f56000098b0 tx=0x7f56000098e0 comp rx=0 tx=0).stop 2024-09-21T03:52:29.810 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:29.805+0000 7f5606d9d640 1 --2- 172.21.15.39:0/3511466501 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f55f00b6680 0x7f55f00b8b90 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x5596f029e300 tx=0x7f55f800b390 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:52:29.810 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:29.805+0000 7f56135a3740 1 -- 172.21.15.39:0/278631051 shutdown_connections 2024-09-21T03:52:29.810 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:29.805+0000 7f56135a3740 1 --2- 172.21.15.39:0/278631051 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5596efed7390 0x5596f02a0100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:29.810 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:29.805+0000 7f56135a3740 1 --2- 172.21.15.39:0/278631051 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5596efecf1b0 0x5596f029fbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:29.810 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:29.805+0000 7f56135a3740 1 --2- 172.21.15.39:0/278631051 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5596efef8070 0x5596f029f680 unknown :-1 s=CLOSED pgs=984 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:29.810 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:29.805+0000 7f56135a3740 1 -- 172.21.15.39:0/278631051 >> 172.21.15.39:0/278631051 conn(0x5596efeec460 msgr2=0x5596f029d570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:52:29.811 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:29.805+0000 7f56135a3740 1 -- 172.21.15.39:0/278631051 shutdown_connections 2024-09-21T03:52:29.811 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:29.805+0000 7f56135a3740 1 -- 172.21.15.39:0/278631051 wait complete. 2024-09-21T03:52:29.811 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:29.806+0000 7f56135a3740 1 -- start start 2024-09-21T03:52:29.811 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:29.806+0000 7f56135a3740 1 Processor -- start 2024-09-21T03:52:29.811 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:29.806+0000 7f56135a3740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5596efecf1b0 0x5596f02aad90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:52:29.811 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:29.806+0000 7f56135a3740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5596efed7390 0x5596f02ab2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:52:29.811 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:29.806+0000 7f56135a3740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5596efef8070 0x5596f02b6fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:52:29.811 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:29.806+0000 7f56135a3740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x5596f0066700 con 0x5596efef8070 2024-09-21T03:52:29.811 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:29.806+0000 7f56135a3740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x5596f01a0160 con 0x5596efed7390 2024-09-21T03:52:29.811 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:29.806+0000 7f56135a3740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x5596f02adc80 con 0x5596efecf1b0 2024-09-21T03:52:29.811 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:29.806+0000 7f56135a3740 10 client.0 _ll_register_callbacks cb 0x5596f01b99e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:52:29.811 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:29.806+0000 7f560759e640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5596efef8070 0x5596f02b6fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:52:29.811 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:29.807+0000 7f560759e640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5596efef8070 0x5596f02b6fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:49834/0 (socket says 172.21.15.39:49834) 2024-09-21T03:52:29.811 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:29.807+0000 7f560759e640 1 -- 172.21.15.39:0/518674311 learned_addr learned my addr 172.21.15.39:0/518674311 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:52:29.812 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:29.807+0000 7f5606d9d640 1 --2- 172.21.15.39:0/518674311 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5596efecf1b0 0x5596f02aad90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:52:29.812 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:29.807+0000 7f560659c640 1 --2- 172.21.15.39:0/518674311 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5596efed7390 0x5596f02ab2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:52:29.812 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:29.807+0000 7f560759e640 1 -- 172.21.15.39:0/518674311 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5596efecf1b0 msgr2=0x5596f02aad90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:52:29.812 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:29.807+0000 7f560759e640 1 --2- 172.21.15.39:0/518674311 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5596efecf1b0 0x5596f02aad90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:29.812 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:29.807+0000 7f560759e640 1 -- 172.21.15.39:0/518674311 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5596efed7390 msgr2=0x5596f02ab2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:52:29.812 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:29.807+0000 7f560759e640 1 --2- 172.21.15.39:0/518674311 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5596efed7390 0x5596f02ab2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:29.812 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:29.807+0000 7f560759e640 1 -- 172.21.15.39:0/518674311 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5596f01cdc90 con 0x5596efef8070 2024-09-21T03:52:29.812 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:29.807+0000 7f560659c640 1 --2- 172.21.15.39:0/518674311 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5596efed7390 0x5596f02ab2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:52:29.812 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:29.807+0000 7f560759e640 1 --2- 172.21.15.39:0/518674311 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5596efef8070 0x5596f02b6fa0 secure :-1 s=READY pgs=985 cs=0 l=1 rev1=1 crypto rx=0x7f5600005d50 tx=0x7f5600031e20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6531b080adb076b0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:52:29.812 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:29.807+0000 7f55eaff5640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-09-21T03:52:29.812 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:29.807+0000 7f55eaff5640 1 -- 172.21.15.39:0/518674311 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f560004a040 con 0x5596efef8070 2024-09-21T03:52:29.812 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:29.807+0000 7f55eaff5640 1 -- 172.21.15.39:0/518674311 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8484+0+0 (secure 0 0 0) 0x7f5600031650 con 0x5596efef8070 2024-09-21T03:52:29.812 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:29.807+0000 7f56135a3740 1 -- 172.21.15.39:0/518674311 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5600036030 con 0x5596efef8070 2024-09-21T03:52:29.812 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:29.808+0000 7f55eaff5640 1 -- 172.21.15.39:0/518674311 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f560005e050 con 0x5596efef8070 2024-09-21T03:52:29.813 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:29.808+0000 7f56135a3740 10 client.17235 fetch_fsmap learned FSMap version 29 2024-09-21T03:52:29.813 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:29.808+0000 7f56135a3740 1 -- 172.21.15.39:0/518674311 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5596f02b99c0 con 0x5596efef8070 2024-09-21T03:52:29.813 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:29.808+0000 7f55eaff5640 1 -- 172.21.15.39:0/518674311 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5600036030 con 0x5596efef8070 2024-09-21T03:52:29.815 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:29.808+0000 7f55eaff5640 1 -- 172.21.15.39:0/518674311 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f560004d030 con 0x5596efef8070 2024-09-21T03:52:29.815 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:29.808+0000 7f56135a3740 10 client.17235 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T03:52:29.815 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:29.808+0000 7f56135a3740 10 client.17235 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T03:52:29.815 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:29.808+0000 7f56135a3740 20 client.17235 populate_metadata read hostname 'smithi039' 2024-09-21T03:52:29.815 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:29.808+0000 7f56135a3740 1 --2- 172.21.15.39:0/518674311 >> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] conn(0x5596efeb69f0 0x5596f02bcd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:52:29.815 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:29.808+0000 7f56135a3740 4 client.17235 mds_command: new command op to 24695 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:52:29.815 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:29.808+0000 7f56135a3740 1 -- 172.21.15.39:0/518674311 --> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5596f0217df0 con 0x5596efeb69f0 2024-09-21T03:52:29.815 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:29.808+0000 7f5606d9d640 1 --2- 172.21.15.39:0/518674311 >> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] conn(0x5596efeb69f0 0x5596f02bcd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:52:29.815 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:29.809+0000 7f5606d9d640 1 --2- 172.21.15.39:0/518674311 >> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] conn(0x5596efeb69f0 0x5596f02bcd20 crc :-1 s=READY pgs=113 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6511153cf4d04c4 server_cookie=9d84e5af89f5e1ff in_seq=0 out_seq=0 2024-09-21T03:52:29.815 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:29.809+0000 7f55eaff5640 10 client.17235 ms_handle_connect on v2:172.21.15.150:6838/2949386907 2024-09-21T03:52:29.815 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:29.809+0000 7f55eaff5640 1 -- 172.21.15.39:0/518674311 <== mds.0 v2:172.21.15.150:6838/2949386907 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5596f0217df0 con 0x5596efeb69f0 2024-09-21T03:52:29.815 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:29.809+0000 7f55eaff5640 10 client.17235 handle_command_reply: tid=0 2024-09-21T03:52:29.816 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:29.810+0000 7f56135a3740 1 client.17235 shutdown 2024-09-21T03:52:29.816 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:29.810+0000 7f56135a3740 1 -- 172.21.15.39:0/518674311 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5596efef8070 msgr2=0x5596f02b6fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:52:29.816 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:29.810+0000 7f56135a3740 1 --2- 172.21.15.39:0/518674311 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5596efef8070 0x5596f02b6fa0 secure :-1 s=READY pgs=985 cs=0 l=1 rev1=1 crypto rx=0x7f5600005d50 tx=0x7f5600031e20 comp rx=0 tx=0).stop 2024-09-21T03:52:29.816 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:29.810+0000 7f56135a3740 1 -- 172.21.15.39:0/518674311 shutdown_connections 2024-09-21T03:52:29.816 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:29.810+0000 7f56135a3740 1 --2- 172.21.15.39:0/518674311 >> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] conn(0x5596efeb69f0 0x5596f02bcd20 crc :-1 s=READY pgs=113 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:29.816 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:29.810+0000 7f56135a3740 1 --2- 172.21.15.39:0/518674311 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5596efef8070 0x5596f02b6fa0 unknown :-1 s=CLOSED pgs=985 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:29.816 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:29.810+0000 7f56135a3740 1 --2- 172.21.15.39:0/518674311 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5596efed7390 0x5596f02ab2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:29.816 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:29.810+0000 7f55eaff5640 0 client.17235 ms_handle_reset on v2:172.21.15.150:6838/2949386907 2024-09-21T03:52:29.816 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:29.810+0000 7f56135a3740 1 --2- 172.21.15.39:0/518674311 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5596efecf1b0 0x5596f02aad90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:29.816 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:29.810+0000 7f56135a3740 1 -- 172.21.15.39:0/518674311 >> 172.21.15.39:0/518674311 conn(0x5596efeec460 msgr2=0x5596f02a2860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:52:29.816 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:29.810+0000 7f56135a3740 1 -- 172.21.15.39:0/518674311 shutdown_connections 2024-09-21T03:52:29.816 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:29.810+0000 7f56135a3740 1 -- 172.21.15.39:0/518674311 wait complete. 2024-09-21T03:52:29.816 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:29.810+0000 7f56135a3740 20 client.17235 trim_cache size 0 max 16384 2024-09-21T03:52:29.826 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:29.823+0000 7f56135a3740 1 Processor -- start 2024-09-21T03:52:29.826 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:29.823+0000 7f56135a3740 1 -- start start 2024-09-21T03:52:29.826 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:29.823+0000 7f56135a3740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5596efeb69f0 0x5596f02ab250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:52:29.826 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:29.824+0000 7f56135a3740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5596efecf1b0 0x5596f02b6ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:52:29.827 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:29.824+0000 7f56135a3740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5596efed7390 0x5596f02a7f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:52:29.827 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:29.824+0000 7f56135a3740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f56081af5a0 con 0x5596efeb69f0 2024-09-21T03:52:29.827 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:29.824+0000 7f56135a3740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f56081b10e0 con 0x5596efed7390 2024-09-21T03:52:29.827 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:29.824+0000 7f56135a3740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f56081afc70 con 0x5596efecf1b0 2024-09-21T03:52:29.827 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:29.824+0000 7f5606d9d640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5596efeb69f0 0x5596f02ab250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:52:29.827 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:29.824+0000 7f5606d9d640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5596efeb69f0 0x5596f02ab250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:49848/0 (socket says 172.21.15.39:49848) 2024-09-21T03:52:29.827 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:29.824+0000 7f5606d9d640 1 -- 172.21.15.39:0/2933959223 learned_addr learned my addr 172.21.15.39:0/2933959223 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:52:29.827 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:29.824+0000 7f560659c640 1 --2- 172.21.15.39:0/2933959223 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5596efecf1b0 0x5596f02b6ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:52:29.827 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:29.824+0000 7f5606d9d640 1 -- 172.21.15.39:0/2933959223 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5596efecf1b0 msgr2=0x5596f02b6ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:52:29.827 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:29.824+0000 7f5606d9d640 1 --2- 172.21.15.39:0/2933959223 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5596efecf1b0 0x5596f02b6ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:29.827 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:29.824+0000 7f5606d9d640 1 -- 172.21.15.39:0/2933959223 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5596efed7390 msgr2=0x5596f02a7f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:52:29.827 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:29.824+0000 7f560759e640 1 --2- 172.21.15.39:0/2933959223 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5596efed7390 0x5596f02a7f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:52:29.828 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:29.824+0000 7f5606d9d640 1 --2- 172.21.15.39:0/2933959223 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5596efed7390 0x5596f02a7f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:29.828 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:29.824+0000 7f5606d9d640 1 -- 172.21.15.39:0/2933959223 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5596f02a1970 con 0x5596efeb69f0 2024-09-21T03:52:29.828 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:29.824+0000 7f560659c640 1 --2- 172.21.15.39:0/2933959223 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5596efecf1b0 0x5596f02b6ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:52:29.828 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:29.824+0000 7f560759e640 1 --2- 172.21.15.39:0/2933959223 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5596efed7390 0x5596f02a7f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:52:29.828 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:29.824+0000 7f5606d9d640 1 --2- 172.21.15.39:0/2933959223 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5596efeb69f0 0x5596f02ab250 secure :-1 s=READY pgs=986 cs=0 l=1 rev1=1 crypto rx=0x7f55f8010ca0 tx=0x7f55f8019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=51712400702203c8 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:52:29.828 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:29.824+0000 7f55ecff9640 1 -- 172.21.15.39:0/2933959223 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55f8010e50 con 0x5596efeb69f0 2024-09-21T03:52:29.828 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:29.825+0000 7f55ecff9640 1 -- 172.21.15.39:0/2933959223 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f55f801e9a0 con 0x5596efeb69f0 2024-09-21T03:52:29.828 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:29.825+0000 7f55ecff9640 1 -- 172.21.15.39:0/2933959223 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55f8003a10 con 0x5596efeb69f0 2024-09-21T03:52:29.831 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:29.825+0000 7f56135a3740 1 -- 172.21.15.39:0/2933959223 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5596efeb69f0 msgr2=0x5596f02ab250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:52:29.832 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:29.825+0000 7f56135a3740 1 --2- 172.21.15.39:0/2933959223 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5596efeb69f0 0x5596f02ab250 secure :-1 s=READY pgs=986 cs=0 l=1 rev1=1 crypto rx=0x7f55f8010ca0 tx=0x7f55f8019000 comp rx=0 tx=0).stop 2024-09-21T03:52:29.832 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:29.825+0000 7f56135a3740 1 -- 172.21.15.39:0/2933959223 shutdown_connections 2024-09-21T03:52:29.832 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:29.825+0000 7f56135a3740 1 --2- 172.21.15.39:0/2933959223 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5596efed7390 0x5596f02a7f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:29.832 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:29.825+0000 7f56135a3740 1 --2- 172.21.15.39:0/2933959223 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5596efecf1b0 0x5596f02b6ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:29.832 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:29.825+0000 7f56135a3740 1 --2- 172.21.15.39:0/2933959223 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5596efeb69f0 0x5596f02ab250 unknown :-1 s=CLOSED pgs=986 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:29.832 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:29.825+0000 7f56135a3740 1 -- 172.21.15.39:0/2933959223 >> 172.21.15.39:0/2933959223 conn(0x5596efeec460 msgr2=0x5596f02aab40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:52:29.832 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:29.825+0000 7f56135a3740 1 -- 172.21.15.39:0/2933959223 shutdown_connections 2024-09-21T03:52:29.832 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:29.825+0000 7f56135a3740 1 -- 172.21.15.39:0/2933959223 wait complete. 2024-09-21T03:52:29.832 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:29.826+0000 7f56135a3740 1 -- start start 2024-09-21T03:52:29.832 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:29.826+0000 7f56135a3740 1 Processor -- start 2024-09-21T03:52:29.832 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:29.826+0000 7f56135a3740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5596efeb69f0 0x5596f02af100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:52:29.832 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:29.826+0000 7f56135a3740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5596efecf1b0 0x5596f02af640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:52:29.832 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:29.826+0000 7f56135a3740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5596efed7390 0x5596f02afba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:52:29.833 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:29.826+0000 7f56135a3740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f56081b0a10 con 0x5596efeb69f0 2024-09-21T03:52:29.833 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:29.826+0000 7f56135a3740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f56081b0340 con 0x5596efed7390 2024-09-21T03:52:29.833 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:29.826+0000 7f56135a3740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f56081ad580 con 0x5596efecf1b0 2024-09-21T03:52:29.833 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:29.826+0000 7f56135a3740 10 client.0 _ll_register_callbacks cb 0x7f55f800bb20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:52:29.833 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:29.826+0000 7f5606d9d640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5596efeb69f0 0x5596f02af100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:52:29.833 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:29.826+0000 7f5606d9d640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5596efeb69f0 0x5596f02af100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:49860/0 (socket says 172.21.15.39:49860) 2024-09-21T03:52:29.833 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:29.826+0000 7f5606d9d640 1 -- 172.21.15.39:0/3581840621 learned_addr learned my addr 172.21.15.39:0/3581840621 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:52:29.833 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:29.827+0000 7f560759e640 1 --2- 172.21.15.39:0/3581840621 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5596efed7390 0x5596f02afba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:52:29.833 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:29.827+0000 7f560659c640 1 --2- 172.21.15.39:0/3581840621 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5596efecf1b0 0x5596f02af640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:52:29.833 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:29.827+0000 7f5606d9d640 1 -- 172.21.15.39:0/3581840621 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5596efecf1b0 msgr2=0x5596f02af640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:52:29.833 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:29.827+0000 7f5606d9d640 1 --2- 172.21.15.39:0/3581840621 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5596efecf1b0 0x5596f02af640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:29.833 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:29.827+0000 7f5606d9d640 1 -- 172.21.15.39:0/3581840621 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5596efed7390 msgr2=0x5596f02afba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:52:29.833 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:29.827+0000 7f5606d9d640 1 --2- 172.21.15.39:0/3581840621 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5596efed7390 0x5596f02afba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:29.833 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:29.827+0000 7f5606d9d640 1 -- 172.21.15.39:0/3581840621 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5596f0326750 con 0x5596efeb69f0 2024-09-21T03:52:29.833 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:29.827+0000 7f560759e640 1 --2- 172.21.15.39:0/3581840621 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5596efed7390 0x5596f02afba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:52:29.834 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:29.827+0000 7f5606d9d640 1 --2- 172.21.15.39:0/3581840621 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5596efeb69f0 0x5596f02af100 secure :-1 s=READY pgs=987 cs=0 l=1 rev1=1 crypto rx=0x7f55f801e7f0 tx=0x7f55f8010fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d0c0ff271a602a2 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:52:29.834 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:29.827+0000 7f55eaff5640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-09-21T03:52:29.834 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:29.827+0000 7f55eaff5640 1 -- 172.21.15.39:0/3581840621 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55f803e060 con 0x5596efeb69f0 2024-09-21T03:52:29.834 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:29.827+0000 7f56135a3740 1 -- 172.21.15.39:0/3581840621 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f55f801e9a0 con 0x5596efeb69f0 2024-09-21T03:52:29.834 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:29.827+0000 7f55eaff5640 1 -- 172.21.15.39:0/3581840621 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8484+0+0 (secure 0 0 0) 0x7f55f801eb50 con 0x5596efeb69f0 2024-09-21T03:52:29.834 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:29.827+0000 7f55eaff5640 1 -- 172.21.15.39:0/3581840621 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f55f801ed50 con 0x5596efeb69f0 2024-09-21T03:52:29.834 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:29.827+0000 7f56135a3740 10 client.17241 fetch_fsmap learned FSMap version 29 2024-09-21T03:52:29.834 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:29.827+0000 7f56135a3740 1 -- 172.21.15.39:0/3581840621 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5596f0327940 con 0x5596efeb69f0 2024-09-21T03:52:29.834 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:29.828+0000 7f55eaff5640 1 -- 172.21.15.39:0/3581840621 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f55f801e9a0 con 0x5596efeb69f0 2024-09-21T03:52:29.834 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:29.828+0000 7f55eaff5640 1 -- 172.21.15.39:0/3581840621 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f55f8003ce0 con 0x5596efeb69f0 2024-09-21T03:52:29.834 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:29.828+0000 7f56135a3740 10 client.17241 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T03:52:29.834 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:29.828+0000 7f56135a3740 10 client.17241 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T03:52:29.834 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:29.828+0000 7f56135a3740 20 client.17241 populate_metadata read hostname 'smithi039' 2024-09-21T03:52:29.834 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:29.828+0000 7f56135a3740 1 --2- 172.21.15.39:0/3581840621 >> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] conn(0x5596efef8070 0x5596f0329f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:52:29.835 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:29.828+0000 7f56135a3740 4 client.17241 mds_command: new command op to 24695 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T03:52:29.835 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:29.828+0000 7f56135a3740 1 -- 172.21.15.39:0/3581840621 --> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5596f02138a0 con 0x5596efef8070 2024-09-21T03:52:29.835 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:29.828+0000 7f560659c640 1 --2- 172.21.15.39:0/3581840621 >> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] conn(0x5596efef8070 0x5596f0329f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:52:29.835 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:29.829+0000 7f560659c640 1 --2- 172.21.15.39:0/3581840621 >> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] conn(0x5596efef8070 0x5596f0329f70 crc :-1 s=READY pgs=114 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ae2e20db742891ce server_cookie=f6f73e5b6864a869 in_seq=0 out_seq=0 2024-09-21T03:52:29.835 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:29.829+0000 7f55eaff5640 10 client.17241 ms_handle_connect on v2:172.21.15.150:6838/2949386907 2024-09-21T03:52:29.835 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:29.829+0000 7f55eaff5640 1 -- 172.21.15.39:0/3581840621 <== mds.0 v2:172.21.15.150:6838/2949386907 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5596f02138a0 con 0x5596efef8070 2024-09-21T03:52:29.835 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:29.829+0000 7f55eaff5640 10 client.17241 handle_command_reply: tid=0 2024-09-21T03:52:29.835 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:29.829+0000 7f56135a3740 1 client.17241 shutdown 2024-09-21T03:52:29.835 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:29.829+0000 7f56135a3740 1 -- 172.21.15.39:0/3581840621 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5596efeb69f0 msgr2=0x5596f02af100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:52:29.835 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:29.829+0000 7f56135a3740 1 --2- 172.21.15.39:0/3581840621 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5596efeb69f0 0x5596f02af100 secure :-1 s=READY pgs=987 cs=0 l=1 rev1=1 crypto rx=0x7f55f801e7f0 tx=0x7f55f8010fd0 comp rx=0 tx=0).stop 2024-09-21T03:52:29.835 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:29.829+0000 7f56135a3740 1 -- 172.21.15.39:0/3581840621 shutdown_connections 2024-09-21T03:52:29.835 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:29.829+0000 7f56135a3740 1 --2- 172.21.15.39:0/3581840621 >> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] conn(0x5596efef8070 0x5596f0329f70 crc :-1 s=READY pgs=114 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:29.835 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:29.829+0000 7f55eaff5640 0 client.17241 ms_handle_reset on v2:172.21.15.150:6838/2949386907 2024-09-21T03:52:29.835 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:29.829+0000 7f56135a3740 1 --2- 172.21.15.39:0/3581840621 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5596efed7390 0x5596f02afba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:29.835 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:29.829+0000 7f56135a3740 1 --2- 172.21.15.39:0/3581840621 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5596efecf1b0 0x5596f02af640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:29.836 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:29.829+0000 7f56135a3740 1 --2- 172.21.15.39:0/3581840621 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5596efeb69f0 0x5596f02af100 unknown :-1 s=CLOSED pgs=987 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:29.836 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:29.829+0000 7f56135a3740 1 -- 172.21.15.39:0/3581840621 >> 172.21.15.39:0/3581840621 conn(0x5596efeec460 msgr2=0x5596f02a9b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:52:29.836 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:29.829+0000 7f56135a3740 1 -- 172.21.15.39:0/3581840621 shutdown_connections 2024-09-21T03:52:29.836 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:29.829+0000 7f56135a3740 1 -- 172.21.15.39:0/3581840621 wait complete. 2024-09-21T03:52:29.836 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:29.829+0000 7f56135a3740 20 client.17241 trim_cache size 0 max 16384 2024-09-21T03:52:29.836 INFO:teuthology.orchestra.run.smithi039.stdout:[] 2024-09-21T03:52:29.837 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:29.830+0000 7f560d02f640 1 -- 172.21.15.39:0/3511466501 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f55f00b6680 msgr2=0x7f55f00b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:52:29.837 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:29.830+0000 7f560d02f640 1 --2- 172.21.15.39:0/3511466501 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f55f00b6680 0x7f55f00b8b90 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x5596f029e300 tx=0x7f55f800b390 comp rx=0 tx=0).stop 2024-09-21T03:52:29.837 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:29.830+0000 7f560d02f640 1 -- 172.21.15.39:0/3511466501 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5608107b70 msgr2=0x7f560819a9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:52:29.837 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:29.830+0000 7f560d02f640 1 --2- 172.21.15.39:0/3511466501 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5608107b70 0x7f560819a9c0 secure :-1 s=READY pgs=983 cs=0 l=1 rev1=1 crypto rx=0x7f55fc00d750 tx=0x7f55fc00dc10 comp rx=0 tx=0).stop 2024-09-21T03:52:29.837 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:29.830+0000 7f560d02f640 1 -- 172.21.15.39:0/3511466501 shutdown_connections 2024-09-21T03:52:29.837 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:29.830+0000 7f560d02f640 1 --2- 172.21.15.39:0/3511466501 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f55f00b6680 0x7f55f00b8b90 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:29.837 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:29.830+0000 7f560d02f640 1 --2- 172.21.15.39:0/3511466501 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f56081084c0 0x7f560819fc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:29.837 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:29.830+0000 7f560d02f640 1 --2- 172.21.15.39:0/3511466501 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5608107b70 0x7f560819a9c0 unknown :-1 s=CLOSED pgs=983 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:29.837 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:29.830+0000 7f560d02f640 1 --2- 172.21.15.39:0/3511466501 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5608102230 0x7f560819a420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:29.837 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:29.830+0000 7f560d02f640 1 -- 172.21.15.39:0/3511466501 >> 172.21.15.39:0/3511466501 conn(0x7f5608079e40 msgr2=0x7f5608100f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:52:29.838 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:29.830+0000 7f560d02f640 1 -- 172.21.15.39:0/3511466501 shutdown_connections 2024-09-21T03:52:29.838 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:29.830+0000 7f560d02f640 1 -- 172.21.15.39:0/3511466501 wait complete. 2024-09-21T03:52:30.537 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:52:30 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:52:30.537 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:52:30 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:52:30.537 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:52:30 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:52:30.718 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:52:30 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:52:30.718 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:52:30 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:52:30.718 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:52:30 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:52:30.725 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:52:30 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:52:30.725 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:52:30 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:52:30.725 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:52:30 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:52:31.539 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-21T03:52:31.539 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 51116d5c-77ca-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-21T03:52:31.568 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:52:31 smithi039 ceph-mon[31211]: pgmap v519: 97 pgs: 97 active+clean; 10 MiB data, 520 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 11 op/s 2024-09-21T03:52:31.718 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:52:31 smithi161 ceph-mon[34533]: pgmap v519: 97 pgs: 97 active+clean; 10 MiB data, 520 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 11 op/s 2024-09-21T03:52:31.724 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:52:31 smithi150 ceph-mon[34289]: pgmap v519: 97 pgs: 97 active+clean; 10 MiB data, 520 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 11 op/s 2024-09-21T03:52:31.812 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/51116d5c-77ca-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:52:32.336 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:32.333+0000 7f2e44173640 1 -- 172.21.15.39:0/1121779080 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2e3c104840 msgr2=0x7f2e3c104c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:52:32.336 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:32.333+0000 7f2e44173640 1 --2- 172.21.15.39:0/1121779080 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2e3c104840 0x7f2e3c104c50 secure :-1 s=READY pgs=988 cs=0 l=1 rev1=1 crypto rx=0x7f2e34009900 tx=0x7f2e3402a410 comp rx=0 tx=0).stop 2024-09-21T03:52:32.336 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:32.333+0000 7f2e44173640 1 -- 172.21.15.39:0/1121779080 shutdown_connections 2024-09-21T03:52:32.336 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:32.333+0000 7f2e44173640 1 --2- 172.21.15.39:0/1121779080 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2e3c10ddf0 0x7f2e3c110300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:32.336 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:32.333+0000 7f2e44173640 1 --2- 172.21.15.39:0/1121779080 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2e3c105270 0x7f2e3c10d8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:32.336 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:32.333+0000 7f2e44173640 1 --2- 172.21.15.39:0/1121779080 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2e3c104840 0x7f2e3c104c50 unknown :-1 s=CLOSED pgs=988 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:32.336 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:32.333+0000 7f2e44173640 1 -- 172.21.15.39:0/1121779080 >> 172.21.15.39:0/1121779080 conn(0x7f2e3c079730 msgr2=0x7f2e3c079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:52:32.336 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:32.333+0000 7f2e44173640 1 -- 172.21.15.39:0/1121779080 shutdown_connections 2024-09-21T03:52:32.336 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:32.333+0000 7f2e44173640 1 -- 172.21.15.39:0/1121779080 wait complete. 2024-09-21T03:52:32.336 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:32.334+0000 7f2e44173640 1 Processor -- start 2024-09-21T03:52:32.337 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:32.334+0000 7f2e44173640 1 -- start start 2024-09-21T03:52:32.339 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:32.334+0000 7f2e44173640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2e3c104840 0x7f2e3c19a580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:52:32.339 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:32.334+0000 7f2e44173640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2e3c105270 0x7f2e3c19ab20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:52:32.339 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:32.334+0000 7f2e44173640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2e3c10ddf0 0x7f2e3c19fdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:52:32.339 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:32.334+0000 7f2e44173640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f2e3c113a90 con 0x7f2e3c104840 2024-09-21T03:52:32.339 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:32.334+0000 7f2e44173640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f2e3c113910 con 0x7f2e3c10ddf0 2024-09-21T03:52:32.339 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:32.334+0000 7f2e44173640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f2e3c113c10 con 0x7f2e3c105270 2024-09-21T03:52:32.339 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:32.334+0000 7f2e41f0f640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2e3c104840 0x7f2e3c19a580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:52:32.340 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:32.334+0000 7f2e41f0f640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2e3c104840 0x7f2e3c19a580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:49866/0 (socket says 172.21.15.39:49866) 2024-09-21T03:52:32.340 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:32.334+0000 7f2e41f0f640 1 -- 172.21.15.39:0/2903781795 learned_addr learned my addr 172.21.15.39:0/2903781795 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:52:32.340 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:32.335+0000 7f2e4170e640 1 --2- 172.21.15.39:0/2903781795 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2e3c105270 0x7f2e3c19ab20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:52:32.340 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:32.335+0000 7f2e42710640 1 --2- 172.21.15.39:0/2903781795 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2e3c10ddf0 0x7f2e3c19fdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:52:32.340 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:32.335+0000 7f2e41f0f640 1 -- 172.21.15.39:0/2903781795 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2e3c105270 msgr2=0x7f2e3c19ab20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:52:32.340 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:32.335+0000 7f2e41f0f640 1 --2- 172.21.15.39:0/2903781795 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2e3c105270 0x7f2e3c19ab20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:32.340 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:32.335+0000 7f2e41f0f640 1 -- 172.21.15.39:0/2903781795 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2e3c10ddf0 msgr2=0x7f2e3c19fdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:52:32.340 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:32.335+0000 7f2e41f0f640 1 --2- 172.21.15.39:0/2903781795 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2e3c10ddf0 0x7f2e3c19fdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:32.341 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:32.335+0000 7f2e41f0f640 1 -- 172.21.15.39:0/2903781795 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2e3c1a04f0 con 0x7f2e3c104840 2024-09-21T03:52:32.341 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:32.335+0000 7f2e41f0f640 1 --2- 172.21.15.39:0/2903781795 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2e3c104840 0x7f2e3c19a580 secure :-1 s=READY pgs=989 cs=0 l=1 rev1=1 crypto rx=0x7f2e34032fd0 tx=0x7f2e34004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:52:32.341 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:32.335+0000 7f2e3affd640 1 -- 172.21.15.39:0/2903781795 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e3403f030 con 0x7f2e3c104840 2024-09-21T03:52:32.341 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:32.335+0000 7f2e3affd640 1 -- 172.21.15.39:0/2903781795 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2e34004490 con 0x7f2e3c104840 2024-09-21T03:52:32.341 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:32.335+0000 7f2e44173640 1 -- 172.21.15.39:0/2903781795 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2e3c1a0780 con 0x7f2e3c104840 2024-09-21T03:52:32.341 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:32.335+0000 7f2e3affd640 1 -- 172.21.15.39:0/2903781795 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e34035070 con 0x7f2e3c104840 2024-09-21T03:52:32.341 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:32.335+0000 7f2e44173640 1 -- 172.21.15.39:0/2903781795 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2e3c1a6f40 con 0x7f2e3c104840 2024-09-21T03:52:32.348 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:32.338+0000 7f2e3affd640 1 -- 172.21.15.39:0/2903781795 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 27) ==== 154643+0+0 (secure 0 0 0) 0x7f2e34042030 con 0x7f2e3c104840 2024-09-21T03:52:32.348 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:32.338+0000 7f2e4a6e7740 1 Processor -- start 2024-09-21T03:52:32.348 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:32.338+0000 7f2e4a6e7740 1 -- start start 2024-09-21T03:52:32.348 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:32.338+0000 7f2e4a6e7740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c78fe0d350 0x55c7901b4430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:52:32.348 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:32.338+0000 7f2e4a6e7740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c78fde4490 0x55c7901b4970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:52:32.348 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:32.339+0000 7f2e4a6e7740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c78fdec670 0x55c7901b4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:52:32.348 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:32.339+0000 7f2e4a6e7740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x55c79012ae60 con 0x55c78fde4490 2024-09-21T03:52:32.348 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:32.339+0000 7f2e4a6e7740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x55c79011a290 con 0x55c78fe0d350 2024-09-21T03:52:32.348 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:32.339+0000 7f2e4a6e7740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55c79011e0a0 con 0x55c78fdec670 2024-09-21T03:52:32.348 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:32.339+0000 7f2e41f0f640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c78fde4490 0x55c7901b4970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:52:32.348 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:32.339+0000 7f2e41f0f640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c78fde4490 0x55c7901b4970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:49876/0 (socket says 172.21.15.39:49876) 2024-09-21T03:52:32.348 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:32.339+0000 7f2e41f0f640 1 -- 172.21.15.39:0/2239565827 learned_addr learned my addr 172.21.15.39:0/2239565827 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:52:32.349 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:32.339+0000 7f2e42710640 1 --2- 172.21.15.39:0/2239565827 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c78fe0d350 0x55c7901b4430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:52:32.349 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:32.339+0000 7f2e4170e640 1 --2- 172.21.15.39:0/2239565827 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c78fdec670 0x55c7901b4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:52:32.349 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:32.339+0000 7f2e3affd640 1 --2- 172.21.15.39:0/2903781795 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f2e240b6700 0x7f2e240b8c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:52:32.349 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:32.339+0000 7f2e41f0f640 1 -- 172.21.15.39:0/2239565827 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c78fdec670 msgr2=0x55c7901b4eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:52:32.349 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:32.339+0000 7f2e41f0f640 1 --2- 172.21.15.39:0/2239565827 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c78fdec670 0x55c7901b4eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:32.349 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:32.339+0000 7f2e41f0f640 1 -- 172.21.15.39:0/2239565827 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c78fe0d350 msgr2=0x55c7901b4430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:52:32.349 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:32.339+0000 7f2e41f0f640 1 --2- 172.21.15.39:0/2239565827 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c78fe0d350 0x55c7901b4430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:32.349 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:32.339+0000 7f2e41f0f640 1 -- 172.21.15.39:0/2239565827 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c78fe78930 con 0x55c78fde4490 2024-09-21T03:52:32.349 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:32.339+0000 7f2e42710640 1 --2- 172.21.15.39:0/2903781795 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f2e240b6700 0x7f2e240b8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:52:32.349 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:32.340+0000 7f2e3affd640 1 -- 172.21.15.39:0/2903781795 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8484+0+0 (secure 0 0 0) 0x7f2e34107790 con 0x7f2e3c104840 2024-09-21T03:52:32.349 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:32.340+0000 7f2e41f0f640 1 --2- 172.21.15.39:0/2239565827 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c78fde4490 0x55c7901b4970 secure :-1 s=READY pgs=990 cs=0 l=1 rev1=1 crypto rx=0x7f2e34104e70 tx=0x7f2e340cd8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc749544a2194a0a server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:52:32.349 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:32.340+0000 7f2e23fff640 1 -- 172.21.15.39:0/2239565827 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e34049030 con 0x55c78fde4490 2024-09-21T03:52:32.349 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:32.340+0000 7f2e42710640 1 --2- 172.21.15.39:0/2903781795 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f2e240b6700 0x7f2e240b8c10 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f2e2c005fd0 tx=0x7f2e2c004a40 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:52:32.349 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:32.340+0000 7f2e23fff640 1 -- 172.21.15.39:0/2239565827 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2e340cd1c0 con 0x55c78fde4490 2024-09-21T03:52:32.350 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:32.340+0000 7f2e23fff640 1 -- 172.21.15.39:0/2239565827 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e340cd000 con 0x55c78fde4490 2024-09-21T03:52:32.350 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:32.341+0000 7f2e4a6e7740 1 -- 172.21.15.39:0/2239565827 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c78fde4490 msgr2=0x55c7901b4970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:52:32.350 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:32.341+0000 7f2e4a6e7740 1 --2- 172.21.15.39:0/2239565827 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c78fde4490 0x55c7901b4970 secure :-1 s=READY pgs=990 cs=0 l=1 rev1=1 crypto rx=0x7f2e34104e70 tx=0x7f2e340cd8a0 comp rx=0 tx=0).stop 2024-09-21T03:52:32.350 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:32.341+0000 7f2e4a6e7740 1 -- 172.21.15.39:0/2239565827 shutdown_connections 2024-09-21T03:52:32.350 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:32.341+0000 7f2e4a6e7740 1 --2- 172.21.15.39:0/2239565827 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c78fdec670 0x55c7901b4eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:32.350 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:32.341+0000 7f2e4a6e7740 1 --2- 172.21.15.39:0/2239565827 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c78fde4490 0x55c7901b4970 unknown :-1 s=CLOSED pgs=990 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:32.350 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:32.341+0000 7f2e4a6e7740 1 --2- 172.21.15.39:0/2239565827 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c78fe0d350 0x55c7901b4430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:32.350 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:32.341+0000 7f2e4a6e7740 1 -- 172.21.15.39:0/2239565827 >> 172.21.15.39:0/2239565827 conn(0x55c78fe01740 msgr2=0x55c7901b2320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:52:32.350 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:32.341+0000 7f2e4a6e7740 1 -- 172.21.15.39:0/2239565827 shutdown_connections 2024-09-21T03:52:32.350 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:32.341+0000 7f2e4a6e7740 1 -- 172.21.15.39:0/2239565827 wait complete. 2024-09-21T03:52:32.350 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:32.341+0000 7f2e4a6e7740 1 -- start start 2024-09-21T03:52:32.350 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:32.341+0000 7f2e4a6e7740 1 Processor -- start 2024-09-21T03:52:32.350 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:32.342+0000 7f2e4a6e7740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c78fde4490 0x55c7901c00b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:52:32.350 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:32.342+0000 7f2e4a6e7740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c78fdec670 0x55c7901c0650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:52:32.351 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:32.342+0000 7f2e4a6e7740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c78fe0d350 0x55c7901cbc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:52:32.351 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:32.342+0000 7f2e4a6e7740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x55c78ff7b890 con 0x55c78fde4490 2024-09-21T03:52:32.351 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:32.342+0000 7f2e4a6e7740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x55c7900b4d40 con 0x55c78fdec670 2024-09-21T03:52:32.351 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:32.342+0000 7f2e4a6e7740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55c7901c29d0 con 0x55c78fe0d350 2024-09-21T03:52:32.351 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:32.342+0000 7f2e4a6e7740 10 client.0 _ll_register_callbacks cb 0x55c790118c80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:52:32.351 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:32.342+0000 7f2e41f0f640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c78fde4490 0x55c7901c00b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:52:32.351 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:32.342+0000 7f2e41f0f640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c78fde4490 0x55c7901c00b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:49886/0 (socket says 172.21.15.39:49886) 2024-09-21T03:52:32.351 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:32.342+0000 7f2e41f0f640 1 -- 172.21.15.39:0/699688249 learned_addr learned my addr 172.21.15.39:0/699688249 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:52:32.351 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:32.342+0000 7f2e42710640 1 --2- 172.21.15.39:0/699688249 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c78fe0d350 0x55c7901cbc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:52:32.351 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:32.342+0000 7f2e4170e640 1 --2- 172.21.15.39:0/699688249 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c78fdec670 0x55c7901c0650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:52:32.351 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:32.342+0000 7f2e41f0f640 1 -- 172.21.15.39:0/699688249 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c78fe0d350 msgr2=0x55c7901cbc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:52:32.351 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:32.342+0000 7f2e41f0f640 1 --2- 172.21.15.39:0/699688249 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c78fe0d350 0x55c7901cbc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:32.351 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:32.342+0000 7f2e41f0f640 1 -- 172.21.15.39:0/699688249 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c78fdec670 msgr2=0x55c7901c0650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:52:32.352 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:32.342+0000 7f2e41f0f640 1 --2- 172.21.15.39:0/699688249 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c78fdec670 0x55c7901c0650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:32.352 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:32.342+0000 7f2e41f0f640 1 -- 172.21.15.39:0/699688249 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c7901cc270 con 0x55c78fde4490 2024-09-21T03:52:32.352 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:32.343+0000 7f2e41f0f640 1 --2- 172.21.15.39:0/699688249 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c78fde4490 0x55c7901c00b0 secure :-1 s=READY pgs=991 cs=0 l=1 rev1=1 crypto rx=0x7f2e340ca920 tx=0x7f2e3402a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df3c18cdc8140b05 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:52:32.352 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:32.343+0000 7f2e217fa640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-09-21T03:52:32.352 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:32.343+0000 7f2e217fa640 1 -- 172.21.15.39:0/699688249 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e340cce20 con 0x55c78fde4490 2024-09-21T03:52:32.352 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:32.343+0000 7f2e217fa640 1 -- 172.21.15.39:0/699688249 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8484+0+0 (secure 0 0 0) 0x7f2e340c9920 con 0x55c78fde4490 2024-09-21T03:52:32.352 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:32.343+0000 7f2e4a6e7740 1 -- 172.21.15.39:0/699688249 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2e340cd1c0 con 0x55c78fde4490 2024-09-21T03:52:32.352 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:32.343+0000 7f2e217fa640 1 -- 172.21.15.39:0/699688249 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f2e340cd000 con 0x55c78fde4490 2024-09-21T03:52:32.352 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:32.343+0000 7f2e4a6e7740 10 client.17253 fetch_fsmap learned FSMap version 29 2024-09-21T03:52:32.352 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:32.343+0000 7f2e4a6e7740 1 -- 172.21.15.39:0/699688249 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55c7901ceba0 con 0x55c78fde4490 2024-09-21T03:52:32.352 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:32.343+0000 7f2e217fa640 1 -- 172.21.15.39:0/699688249 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2e340cd1c0 con 0x55c78fde4490 2024-09-21T03:52:32.352 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:32.343+0000 7f2e217fa640 1 -- 172.21.15.39:0/699688249 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f2e3404c060 con 0x55c78fde4490 2024-09-21T03:52:32.352 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:32.344+0000 7f2e4a6e7740 10 client.17253 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T03:52:32.352 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:32.344+0000 7f2e4a6e7740 10 client.17253 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T03:52:32.352 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:32.344+0000 7f2e4a6e7740 20 client.17253 populate_metadata read hostname 'smithi039' 2024-09-21T03:52:32.353 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:32.344+0000 7f2e4a6e7740 1 --2- 172.21.15.39:0/699688249 >> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] conn(0x55c78fdcbcd0 0x55c7901d1f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:52:32.353 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:32.344+0000 7f2e4a6e7740 4 client.17253 mds_command: new command op to 24695 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:52:32.353 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:32.344+0000 7f2e4a6e7740 1 -- 172.21.15.39:0/699688249 --> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c79012cdb0 con 0x55c78fdcbcd0 2024-09-21T03:52:32.353 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:32.344+0000 7f2e4170e640 1 --2- 172.21.15.39:0/699688249 >> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] conn(0x55c78fdcbcd0 0x55c7901d1f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:52:32.353 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:32.344+0000 7f2e4170e640 1 --2- 172.21.15.39:0/699688249 >> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] conn(0x55c78fdcbcd0 0x55c7901d1f00 crc :-1 s=READY pgs=115 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8dba707a5f758fd5 server_cookie=676fddfa24eeca9c in_seq=0 out_seq=0 2024-09-21T03:52:32.353 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:32.344+0000 7f2e217fa640 10 client.17253 ms_handle_connect on v2:172.21.15.150:6838/2949386907 2024-09-21T03:52:32.353 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:32.345+0000 7f2e217fa640 1 -- 172.21.15.39:0/699688249 <== mds.0 v2:172.21.15.150:6838/2949386907 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55c79012cdb0 con 0x55c78fdcbcd0 2024-09-21T03:52:32.353 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:32.345+0000 7f2e217fa640 10 client.17253 handle_command_reply: tid=0 2024-09-21T03:52:32.353 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:32.345+0000 7f2e4a6e7740 1 client.17253 shutdown 2024-09-21T03:52:32.353 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:32.345+0000 7f2e4a6e7740 1 -- 172.21.15.39:0/699688249 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c78fde4490 msgr2=0x55c7901c00b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:52:32.353 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:32.345+0000 7f2e4a6e7740 1 --2- 172.21.15.39:0/699688249 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c78fde4490 0x55c7901c00b0 secure :-1 s=READY pgs=991 cs=0 l=1 rev1=1 crypto rx=0x7f2e340ca920 tx=0x7f2e3402a410 comp rx=0 tx=0).stop 2024-09-21T03:52:32.353 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:32.346+0000 7f2e4a6e7740 1 -- 172.21.15.39:0/699688249 shutdown_connections 2024-09-21T03:52:32.353 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:32.346+0000 7f2e4a6e7740 1 --2- 172.21.15.39:0/699688249 >> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] conn(0x55c78fdcbcd0 0x55c7901d1f00 crc :-1 s=READY pgs=115 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:32.353 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:32.346+0000 7f2e4a6e7740 1 --2- 172.21.15.39:0/699688249 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c78fe0d350 0x55c7901cbc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:32.354 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:32.346+0000 7f2e4a6e7740 1 --2- 172.21.15.39:0/699688249 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c78fdec670 0x55c7901c0650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:32.354 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:32.346+0000 7f2e4a6e7740 1 --2- 172.21.15.39:0/699688249 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c78fde4490 0x55c7901c00b0 unknown :-1 s=CLOSED pgs=991 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:32.354 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:32.346+0000 7f2e217fa640 0 client.17253 ms_handle_reset on v2:172.21.15.150:6838/2949386907 2024-09-21T03:52:32.354 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:32.346+0000 7f2e4a6e7740 1 -- 172.21.15.39:0/699688249 >> 172.21.15.39:0/699688249 conn(0x55c78fe01740 msgr2=0x55c7901b75c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:52:32.354 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:32.346+0000 7f2e4a6e7740 1 -- 172.21.15.39:0/699688249 shutdown_connections 2024-09-21T03:52:32.354 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:32.346+0000 7f2e4a6e7740 1 -- 172.21.15.39:0/699688249 wait complete. 2024-09-21T03:52:32.354 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:32.346+0000 7f2e4a6e7740 20 client.17253 trim_cache size 0 max 16384 2024-09-21T03:52:32.365 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:32.363+0000 7f2e4a6e7740 1 Processor -- start 2024-09-21T03:52:32.365 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:32.363+0000 7f2e4a6e7740 1 -- start start 2024-09-21T03:52:32.366 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:32.364+0000 7f2e4a6e7740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c78fde4490 0x55c7901cba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:52:32.366 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:32.364+0000 7f2e4a6e7740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c78fdec670 0x55c7901b1c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:52:32.367 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:32.364+0000 7f2e4a6e7740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c78fe0d350 0x55c7901b21c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:52:32.367 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:32.364+0000 7f2e4a6e7740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f2e3c1af880 con 0x55c78fe0d350 2024-09-21T03:52:32.367 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:32.364+0000 7f2e4a6e7740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f2e3c1b13c0 con 0x55c78fde4490 2024-09-21T03:52:32.367 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:32.364+0000 7f2e4a6e7740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f2e3c1aff50 con 0x55c78fdec670 2024-09-21T03:52:32.367 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:32.364+0000 7f2e42710640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c78fe0d350 0x55c7901b21c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:52:32.367 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:32.364+0000 7f2e42710640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c78fe0d350 0x55c7901b21c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:49892/0 (socket says 172.21.15.39:49892) 2024-09-21T03:52:32.367 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:32.364+0000 7f2e42710640 1 -- 172.21.15.39:0/2718006526 learned_addr learned my addr 172.21.15.39:0/2718006526 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:52:32.367 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:32.364+0000 7f2e4170e640 1 --2- 172.21.15.39:0/2718006526 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c78fdec670 0x55c7901b1c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:52:32.367 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:32.364+0000 7f2e41f0f640 1 --2- 172.21.15.39:0/2718006526 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c78fde4490 0x55c7901cba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:52:32.367 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:32.364+0000 7f2e42710640 1 -- 172.21.15.39:0/2718006526 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c78fdec670 msgr2=0x55c7901b1c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:52:32.367 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:32.364+0000 7f2e42710640 1 --2- 172.21.15.39:0/2718006526 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c78fdec670 0x55c7901b1c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:32.367 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:32.364+0000 7f2e42710640 1 -- 172.21.15.39:0/2718006526 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c78fde4490 msgr2=0x55c7901cba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:52:32.367 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:32.364+0000 7f2e42710640 1 --2- 172.21.15.39:0/2718006526 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c78fde4490 0x55c7901cba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:32.368 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:32.364+0000 7f2e42710640 1 -- 172.21.15.39:0/2718006526 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c7901bee10 con 0x55c78fe0d350 2024-09-21T03:52:32.368 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:32.364+0000 7f2e41f0f640 1 --2- 172.21.15.39:0/2718006526 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c78fde4490 0x55c7901cba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:52:32.368 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:32.365+0000 7f2e42710640 1 --2- 172.21.15.39:0/2718006526 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c78fe0d350 0x55c7901b21c0 secure :-1 s=READY pgs=992 cs=0 l=1 rev1=1 crypto rx=0x7f2e2c012600 tx=0x7f2e2c016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cda3246decd03538 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:52:32.368 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:32.365+0000 7f2e237fe640 1 -- 172.21.15.39:0/2718006526 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e2c021810 con 0x55c78fe0d350 2024-09-21T03:52:32.368 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:32.365+0000 7f2e237fe640 1 -- 172.21.15.39:0/2718006526 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2e2c010400 con 0x55c78fe0d350 2024-09-21T03:52:32.368 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:32.365+0000 7f2e237fe640 1 -- 172.21.15.39:0/2718006526 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e2c021e60 con 0x55c78fe0d350 2024-09-21T03:52:32.371 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:32.365+0000 7f2e4a6e7740 1 -- 172.21.15.39:0/2718006526 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c78fe0d350 msgr2=0x55c7901b21c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:52:32.371 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:32.365+0000 7f2e4a6e7740 1 --2- 172.21.15.39:0/2718006526 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c78fe0d350 0x55c7901b21c0 secure :-1 s=READY pgs=992 cs=0 l=1 rev1=1 crypto rx=0x7f2e2c012600 tx=0x7f2e2c016000 comp rx=0 tx=0).stop 2024-09-21T03:52:32.371 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:32.366+0000 7f2e4a6e7740 1 -- 172.21.15.39:0/2718006526 shutdown_connections 2024-09-21T03:52:32.371 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:32.366+0000 7f2e4a6e7740 1 --2- 172.21.15.39:0/2718006526 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c78fe0d350 0x55c7901b21c0 unknown :-1 s=CLOSED pgs=992 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:32.372 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:32.366+0000 7f2e4a6e7740 1 --2- 172.21.15.39:0/2718006526 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c78fdec670 0x55c7901b1c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:32.372 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:32.366+0000 7f2e4a6e7740 1 --2- 172.21.15.39:0/2718006526 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c78fde4490 0x55c7901cba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:32.372 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:32.366+0000 7f2e4a6e7740 1 -- 172.21.15.39:0/2718006526 >> 172.21.15.39:0/2718006526 conn(0x55c78fe01740 msgr2=0x55c7901bb060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:52:32.372 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:32.366+0000 7f2e4a6e7740 1 -- 172.21.15.39:0/2718006526 shutdown_connections 2024-09-21T03:52:32.372 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:32.366+0000 7f2e4a6e7740 1 -- 172.21.15.39:0/2718006526 wait complete. 2024-09-21T03:52:32.372 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:32.366+0000 7f2e4a6e7740 1 -- start start 2024-09-21T03:52:32.372 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:32.366+0000 7f2e4a6e7740 1 Processor -- start 2024-09-21T03:52:32.372 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:32.367+0000 7f2e4a6e7740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c78fde4490 0x55c7901c8450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:52:32.372 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:32.367+0000 7f2e4a6e7740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c78fdec670 0x55c7901c2a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:52:32.372 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:32.367+0000 7f2e4a6e7740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c78fe0d350 0x55c7901c3120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:52:32.372 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:32.367+0000 7f2e4a6e7740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f2e3c1b0cf0 con 0x55c78fde4490 2024-09-21T03:52:32.372 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:32.367+0000 7f2e4a6e7740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f2e3c1b0620 con 0x55c78fe0d350 2024-09-21T03:52:32.372 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:32.367+0000 7f2e4a6e7740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f2e3c1ad860 con 0x55c78fdec670 2024-09-21T03:52:32.372 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:32.367+0000 7f2e4a6e7740 10 client.0 _ll_register_callbacks cb 0x7f2e2c009a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:52:32.373 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:32.367+0000 7f2e41f0f640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c78fde4490 0x55c7901c8450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:52:32.373 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:32.367+0000 7f2e41f0f640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c78fde4490 0x55c7901c8450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:49908/0 (socket says 172.21.15.39:49908) 2024-09-21T03:52:32.373 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:32.367+0000 7f2e41f0f640 1 -- 172.21.15.39:0/1062519968 learned_addr learned my addr 172.21.15.39:0/1062519968 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:52:32.373 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:32.367+0000 7f2e4170e640 1 --2- 172.21.15.39:0/1062519968 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c78fdec670 0x55c7901c2a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:52:32.373 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:32.367+0000 7f2e42710640 1 --2- 172.21.15.39:0/1062519968 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c78fe0d350 0x55c7901c3120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:52:32.373 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:32.367+0000 7f2e41f0f640 1 -- 172.21.15.39:0/1062519968 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c78fdec670 msgr2=0x55c7901c2a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:52:32.373 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:32.367+0000 7f2e41f0f640 1 --2- 172.21.15.39:0/1062519968 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c78fdec670 0x55c7901c2a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:32.373 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:32.367+0000 7f2e41f0f640 1 -- 172.21.15.39:0/1062519968 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c78fe0d350 msgr2=0x55c7901c3120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:52:32.373 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:32.367+0000 7f2e41f0f640 1 --2- 172.21.15.39:0/1062519968 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c78fe0d350 0x55c7901c3120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:32.373 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:32.367+0000 7f2e41f0f640 1 -- 172.21.15.39:0/1062519968 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c79023c700 con 0x55c78fde4490 2024-09-21T03:52:32.373 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:32.367+0000 7f2e42710640 1 --2- 172.21.15.39:0/1062519968 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c78fe0d350 0x55c7901c3120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:52:32.373 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:32.367+0000 7f2e4170e640 1 --2- 172.21.15.39:0/1062519968 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c78fdec670 0x55c7901c2a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:52:32.373 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:32.367+0000 7f2e41f0f640 1 --2- 172.21.15.39:0/1062519968 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c78fde4490 0x55c7901c8450 secure :-1 s=READY pgs=993 cs=0 l=1 rev1=1 crypto rx=0x7f2e34105500 tx=0x7f2e34107ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6dea2ca39a8a4ac0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:52:32.373 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:32.367+0000 7f2e217fa640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-09-21T03:52:32.373 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:32.367+0000 7f2e217fa640 1 -- 172.21.15.39:0/1062519968 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e34049070 con 0x55c78fde4490 2024-09-21T03:52:32.374 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:32.367+0000 7f2e217fa640 1 -- 172.21.15.39:0/1062519968 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8484+0+0 (secure 0 0 0) 0x7f2e3410d620 con 0x55c78fde4490 2024-09-21T03:52:32.374 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:32.367+0000 7f2e4a6e7740 1 -- 172.21.15.39:0/1062519968 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2e2c010400 con 0x55c78fde4490 2024-09-21T03:52:32.374 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:32.368+0000 7f2e217fa640 1 -- 172.21.15.39:0/1062519968 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f2e340cce20 con 0x55c78fde4490 2024-09-21T03:52:32.374 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:32.368+0000 7f2e4a6e7740 10 client.17259 fetch_fsmap learned FSMap version 29 2024-09-21T03:52:32.374 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:32.368+0000 7f2e4a6e7740 1 -- 172.21.15.39:0/1062519968 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55c79023c700 con 0x55c78fde4490 2024-09-21T03:52:32.374 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:32.368+0000 7f2e217fa640 1 -- 172.21.15.39:0/1062519968 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2e2c010400 con 0x55c78fde4490 2024-09-21T03:52:32.374 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:32.368+0000 7f2e217fa640 1 -- 172.21.15.39:0/1062519968 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f2e340d6c00 con 0x55c78fde4490 2024-09-21T03:52:32.374 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:32.368+0000 7f2e4a6e7740 10 client.17259 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T03:52:32.374 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:32.368+0000 7f2e4a6e7740 10 client.17259 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T03:52:32.374 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:32.368+0000 7f2e4a6e7740 20 client.17259 populate_metadata read hostname 'smithi039' 2024-09-21T03:52:32.374 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:32.368+0000 7f2e4a6e7740 1 --2- 172.21.15.39:0/1062519968 >> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] conn(0x55c78fda1080 0x55c79023fc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:52:32.375 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:32.368+0000 7f2e4a6e7740 4 client.17259 mds_command: new command op to 24695 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-21T03:52:32.375 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:32.368+0000 7f2e4a6e7740 1 -- 172.21.15.39:0/1062519968 --> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55c790128860 con 0x55c78fda1080 2024-09-21T03:52:32.375 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:32.368+0000 7f2e4170e640 1 --2- 172.21.15.39:0/1062519968 >> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] conn(0x55c78fda1080 0x55c79023fc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:52:32.375 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:32.369+0000 7f2e4170e640 1 --2- 172.21.15.39:0/1062519968 >> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] conn(0x55c78fda1080 0x55c79023fc50 crc :-1 s=READY pgs=116 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=76f8e07bbed2df2 server_cookie=84a0a51f0af0eb8c in_seq=0 out_seq=0 2024-09-21T03:52:32.375 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:32.369+0000 7f2e217fa640 10 client.17259 ms_handle_connect on v2:172.21.15.150:6838/2949386907 2024-09-21T03:52:32.376 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:32.371+0000 7f2e217fa640 1 -- 172.21.15.39:0/1062519968 <== mds.0 v2:172.21.15.150:6838/2949386907 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55c790128860 con 0x55c78fda1080 2024-09-21T03:52:32.376 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:32.371+0000 7f2e217fa640 10 client.17259 handle_command_reply: tid=0 2024-09-21T03:52:32.376 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:32.371+0000 7f2e4a6e7740 1 client.17259 shutdown 2024-09-21T03:52:32.377 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:32.371+0000 7f2e4a6e7740 1 -- 172.21.15.39:0/1062519968 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c78fde4490 msgr2=0x55c7901c8450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:52:32.377 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:32.371+0000 7f2e4a6e7740 1 --2- 172.21.15.39:0/1062519968 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c78fde4490 0x55c7901c8450 secure :-1 s=READY pgs=993 cs=0 l=1 rev1=1 crypto rx=0x7f2e34105500 tx=0x7f2e34107ab0 comp rx=0 tx=0).stop 2024-09-21T03:52:32.377 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:32.371+0000 7f2e4a6e7740 1 -- 172.21.15.39:0/1062519968 shutdown_connections 2024-09-21T03:52:32.377 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:32.371+0000 7f2e4a6e7740 1 --2- 172.21.15.39:0/1062519968 >> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] conn(0x55c78fda1080 0x55c79023fc50 crc :-1 s=READY pgs=116 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:32.377 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:32.371+0000 7f2e4a6e7740 1 --2- 172.21.15.39:0/1062519968 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c78fe0d350 0x55c7901c3120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:32.377 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:32.371+0000 7f2e4a6e7740 1 --2- 172.21.15.39:0/1062519968 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c78fdec670 0x55c7901c2a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:32.377 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:32.371+0000 7f2e4a6e7740 1 --2- 172.21.15.39:0/1062519968 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c78fde4490 0x55c7901c8450 unknown :-1 s=CLOSED pgs=993 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:32.377 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:32.371+0000 7f2e4a6e7740 1 -- 172.21.15.39:0/1062519968 >> 172.21.15.39:0/1062519968 conn(0x55c78fe01740 msgr2=0x55c7901bb060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:52:32.377 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:32.371+0000 7f2e217fa640 0 client.17259 ms_handle_reset on v2:172.21.15.150:6838/2949386907 2024-09-21T03:52:32.377 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:32.371+0000 7f2e4a6e7740 1 -- 172.21.15.39:0/1062519968 shutdown_connections 2024-09-21T03:52:32.377 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:32.371+0000 7f2e4a6e7740 1 -- 172.21.15.39:0/1062519968 wait complete. 2024-09-21T03:52:32.377 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:32.371+0000 7f2e4a6e7740 20 client.17259 trim_cache size 0 max 16384 2024-09-21T03:52:32.377 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-09-21T03:52:32.378 INFO:teuthology.orchestra.run.smithi039.stdout: "return_code": 0, 2024-09-21T03:52:32.378 INFO:teuthology.orchestra.run.smithi039.stdout: "scrub_tag": "c7c700f3-2cd8-41ac-8381-b57dd1b37b02", 2024-09-21T03:52:32.378 INFO:teuthology.orchestra.run.smithi039.stdout: "mode": "asynchronous" 2024-09-21T03:52:32.378 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-09-21T03:52:32.378 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:32.372+0000 7f2e44173640 1 -- 172.21.15.39:0/2903781795 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f2e240b6700 msgr2=0x7f2e240b8c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:52:32.378 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:32.372+0000 7f2e44173640 1 --2- 172.21.15.39:0/2903781795 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f2e240b6700 0x7f2e240b8c10 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f2e2c005fd0 tx=0x7f2e2c004a40 comp rx=0 tx=0).stop 2024-09-21T03:52:32.379 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:32.372+0000 7f2e44173640 1 -- 172.21.15.39:0/2903781795 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2e3c104840 msgr2=0x7f2e3c19a580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:52:32.379 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:32.372+0000 7f2e44173640 1 --2- 172.21.15.39:0/2903781795 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2e3c104840 0x7f2e3c19a580 secure :-1 s=READY pgs=989 cs=0 l=1 rev1=1 crypto rx=0x7f2e34032fd0 tx=0x7f2e34004270 comp rx=0 tx=0).stop 2024-09-21T03:52:32.379 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:32.372+0000 7f2e44173640 1 -- 172.21.15.39:0/2903781795 shutdown_connections 2024-09-21T03:52:32.379 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:32.372+0000 7f2e44173640 1 --2- 172.21.15.39:0/2903781795 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f2e240b6700 0x7f2e240b8c10 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:32.379 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:32.372+0000 7f2e44173640 1 --2- 172.21.15.39:0/2903781795 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2e3c10ddf0 0x7f2e3c19fdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:32.379 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:32.372+0000 7f2e44173640 1 --2- 172.21.15.39:0/2903781795 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2e3c105270 0x7f2e3c19ab20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:32.379 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:32.372+0000 7f2e44173640 1 --2- 172.21.15.39:0/2903781795 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2e3c104840 0x7f2e3c19a580 unknown :-1 s=CLOSED pgs=989 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:32.379 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:32.372+0000 7f2e44173640 1 -- 172.21.15.39:0/2903781795 >> 172.21.15.39:0/2903781795 conn(0x7f2e3c079730 msgr2=0x7f2e3c107a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:52:32.379 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:32.372+0000 7f2e44173640 1 -- 172.21.15.39:0/2903781795 shutdown_connections 2024-09-21T03:52:32.379 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:32.372+0000 7f2e44173640 1 -- 172.21.15.39:0/2903781795 wait complete. 2024-09-21T03:52:33.219 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 51116d5c-77ca-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub status 2024-09-21T03:52:33.523 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/51116d5c-77ca-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:52:33.584 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:52:33 smithi039 ceph-mon[31211]: pgmap v520: 97 pgs: 97 active+clean; 10 MiB data, 520 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 9 op/s 2024-09-21T03:52:33.584 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:52:33 smithi039 ceph-mon[31211]: scrub queued for path: / 2024-09-21T03:52:33.584 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:52:33 smithi039 ceph-mon[31211]: scrub summary: idle+waiting paths [/] 2024-09-21T03:52:33.584 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:52:33 smithi039 ceph-mon[31211]: scrub summary: active paths [/] 2024-09-21T03:52:33.584 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:52:33 smithi039 ceph-mon[31211]: scrub summary: idle+waiting paths [/] 2024-09-21T03:52:33.718 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:52:33 smithi161 ceph-mon[34533]: pgmap v520: 97 pgs: 97 active+clean; 10 MiB data, 520 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 9 op/s 2024-09-21T03:52:33.718 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:52:33 smithi161 ceph-mon[34533]: scrub queued for path: / 2024-09-21T03:52:33.718 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:52:33 smithi161 ceph-mon[34533]: scrub summary: idle+waiting paths [/] 2024-09-21T03:52:33.719 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:52:33 smithi161 ceph-mon[34533]: scrub summary: active paths [/] 2024-09-21T03:52:33.719 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:52:33 smithi161 ceph-mon[34533]: scrub summary: idle+waiting paths [/] 2024-09-21T03:52:33.725 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:52:33 smithi150 ceph-mon[34289]: pgmap v520: 97 pgs: 97 active+clean; 10 MiB data, 520 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 9 op/s 2024-09-21T03:52:33.725 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:52:33 smithi150 ceph-mon[34289]: scrub queued for path: / 2024-09-21T03:52:33.725 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:52:33 smithi150 ceph-mon[34289]: scrub summary: idle+waiting paths [/] 2024-09-21T03:52:33.725 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:52:33 smithi150 ceph-mon[34289]: scrub summary: active paths [/] 2024-09-21T03:52:33.725 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:52:33 smithi150 ceph-mon[34289]: scrub summary: idle+waiting paths [/] 2024-09-21T03:52:34.035 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:34.033+0000 7fb76ad0a640 1 -- 172.21.15.39:0/3159375185 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb764102100 msgr2=0x7fb764102510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:52:34.119 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:34.033+0000 7fb76ad0a640 1 --2- 172.21.15.39:0/3159375185 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb764102100 0x7fb764102510 secure :-1 s=READY pgs=994 cs=0 l=1 rev1=1 crypto rx=0x7fb754009ca0 tx=0x7fb75402a8e0 comp rx=0 tx=0).stop 2024-09-21T03:52:34.119 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:34.033+0000 7fb76ad0a640 1 -- 172.21.15.39:0/3159375185 shutdown_connections 2024-09-21T03:52:34.119 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:34.033+0000 7fb76ad0a640 1 --2- 172.21.15.39:0/3159375185 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb764107c50 0x7fb764110670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:34.119 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:34.033+0000 7fb76ad0a640 1 --2- 172.21.15.39:0/3159375185 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb764107300 0x7fb764107710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:34.119 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:34.033+0000 7fb76ad0a640 1 --2- 172.21.15.39:0/3159375185 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb764102100 0x7fb764102510 unknown :-1 s=CLOSED pgs=994 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:34.120 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:34.033+0000 7fb76ad0a640 1 -- 172.21.15.39:0/3159375185 >> 172.21.15.39:0/3159375185 conn(0x7fb764079e40 msgr2=0x7fb76407a250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:52:34.120 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:34.033+0000 7fb76ad0a640 1 -- 172.21.15.39:0/3159375185 shutdown_connections 2024-09-21T03:52:34.120 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:34.033+0000 7fb76ad0a640 1 -- 172.21.15.39:0/3159375185 wait complete. 2024-09-21T03:52:34.134 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:34.034+0000 7fb76ad0a640 1 Processor -- start 2024-09-21T03:52:34.134 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:34.034+0000 7fb76ad0a640 1 -- start start 2024-09-21T03:52:34.134 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:34.034+0000 7fb76ad0a640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb764102100 0x7fb76419a550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:52:34.134 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:34.034+0000 7fb76ad0a640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb764107300 0x7fb76419aaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:52:34.135 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:34.034+0000 7fb76ad0a640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb764107c50 0x7fb76419fd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:52:34.135 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:34.034+0000 7fb76ad0a640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fb764113b80 con 0x7fb764107c50 2024-09-21T03:52:34.135 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:34.034+0000 7fb76ad0a640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fb764113a00 con 0x7fb764107300 2024-09-21T03:52:34.135 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:34.034+0000 7fb76ad0a640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fb764113d00 con 0x7fb764102100 2024-09-21T03:52:34.135 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:34.034+0000 7fb7692a7640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb764107c50 0x7fb76419fd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:52:34.135 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:34.034+0000 7fb7692a7640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb764107c50 0x7fb76419fd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:49936/0 (socket says 172.21.15.39:49936) 2024-09-21T03:52:34.135 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:34.034+0000 7fb7692a7640 1 -- 172.21.15.39:0/3808838003 learned_addr learned my addr 172.21.15.39:0/3808838003 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:52:34.135 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:34.034+0000 7fb768aa6640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb764102100 0x7fb76419a550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:52:34.135 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:34.034+0000 7fb763fff640 1 --2- 172.21.15.39:0/3808838003 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb764107300 0x7fb76419aaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:52:34.135 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:34.034+0000 7fb7692a7640 1 -- 172.21.15.39:0/3808838003 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb764102100 msgr2=0x7fb76419a550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:52:34.135 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:34.034+0000 7fb7692a7640 1 --2- 172.21.15.39:0/3808838003 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb764102100 0x7fb76419a550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:34.135 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:34.034+0000 7fb7692a7640 1 -- 172.21.15.39:0/3808838003 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb764107300 msgr2=0x7fb76419aaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:52:34.135 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:34.034+0000 7fb7692a7640 1 --2- 172.21.15.39:0/3808838003 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb764107300 0x7fb76419aaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:34.135 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:34.034+0000 7fb7692a7640 1 -- 172.21.15.39:0/3808838003 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb7641a04c0 con 0x7fb764107c50 2024-09-21T03:52:34.136 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:34.035+0000 7fb7692a7640 1 --2- 172.21.15.39:0/3808838003 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb764107c50 0x7fb76419fd90 secure :-1 s=READY pgs=995 cs=0 l=1 rev1=1 crypto rx=0x7fb75c00b3f0 tx=0x7fb75c00b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:52:34.136 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:34.035+0000 7fb761ffb640 1 -- 172.21.15.39:0/3808838003 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb75c015020 con 0x7fb764107c50 2024-09-21T03:52:34.136 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:34.035+0000 7fb761ffb640 1 -- 172.21.15.39:0/3808838003 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb75c011ce0 con 0x7fb764107c50 2024-09-21T03:52:34.136 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:34.035+0000 7fb76ad0a640 1 -- 172.21.15.39:0/3808838003 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb7641a07b0 con 0x7fb764107c50 2024-09-21T03:52:34.136 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:34.035+0000 7fb761ffb640 1 -- 172.21.15.39:0/3808838003 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb75c010da0 con 0x7fb764107c50 2024-09-21T03:52:34.136 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:34.035+0000 7fb76ad0a640 1 -- 172.21.15.39:0/3808838003 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb7641a6f10 con 0x7fb764107c50 2024-09-21T03:52:34.136 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:34.038+0000 7fb761ffb640 1 -- 172.21.15.39:0/3808838003 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 27) ==== 154643+0+0 (secure 0 0 0) 0x7fb75c018020 con 0x7fb764107c50 2024-09-21T03:52:34.136 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:34.038+0000 7fb77127e740 1 Processor -- start 2024-09-21T03:52:34.136 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:34.038+0000 7fb77127e740 1 -- start start 2024-09-21T03:52:34.136 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:34.038+0000 7fb77127e740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555c2a300070 0x555c2a6a7270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:52:34.136 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:34.038+0000 7fb77127e740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x555c2a2d71b0 0x555c2a6a77b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:52:34.136 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:34.038+0000 7fb77127e740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555c2a2df390 0x555c2a6a7cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:52:34.136 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:34.038+0000 7fb77127e740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x555c2a61de50 con 0x555c2a2d71b0 2024-09-21T03:52:34.136 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:34.038+0000 7fb77127e740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x555c2a60d280 con 0x555c2a300070 2024-09-21T03:52:34.136 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:34.038+0000 7fb77127e740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x555c2a611090 con 0x555c2a2df390 2024-09-21T03:52:34.137 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:34.038+0000 7fb768aa6640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x555c2a2d71b0 0x555c2a6a77b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:52:34.137 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:34.038+0000 7fb768aa6640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x555c2a2d71b0 0x555c2a6a77b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:49948/0 (socket says 172.21.15.39:49948) 2024-09-21T03:52:34.137 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:34.038+0000 7fb768aa6640 1 -- 172.21.15.39:0/275458080 learned_addr learned my addr 172.21.15.39:0/275458080 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:52:34.137 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:34.038+0000 7fb763fff640 1 --2- 172.21.15.39:0/275458080 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555c2a300070 0x555c2a6a7270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:52:34.137 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:34.038+0000 7fb763fff640 1 --2- 172.21.15.39:0/275458080 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555c2a2df390 0x555c2a6a7cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:52:34.137 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:34.039+0000 7fb761ffb640 1 --2- 172.21.15.39:0/3808838003 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7fb7500b6700 0x7fb7500b8c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:52:34.137 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:34.039+0000 7fb768aa6640 1 -- 172.21.15.39:0/275458080 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555c2a2df390 msgr2=0x555c2a6a7cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:52:34.137 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:34.039+0000 7fb768aa6640 1 --2- 172.21.15.39:0/275458080 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555c2a2df390 0x555c2a6a7cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:34.137 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:34.039+0000 7fb768aa6640 1 -- 172.21.15.39:0/275458080 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555c2a300070 msgr2=0x555c2a6a7270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:52:34.137 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:34.039+0000 7fb768aa6640 1 --2- 172.21.15.39:0/275458080 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555c2a300070 0x555c2a6a7270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:34.137 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:34.039+0000 7fb768aa6640 1 -- 172.21.15.39:0/275458080 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555c2a36b650 con 0x555c2a2d71b0 2024-09-21T03:52:34.137 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:34.039+0000 7fb7692a7640 1 --2- 172.21.15.39:0/3808838003 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7fb7500b6700 0x7fb7500b8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:52:34.137 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:34.039+0000 7fb761ffb640 1 -- 172.21.15.39:0/3808838003 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8484+0+0 (secure 0 0 0) 0x7fb75c0e6e30 con 0x7fb764107c50 2024-09-21T03:52:34.137 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:34.039+0000 7fb768aa6640 1 --2- 172.21.15.39:0/275458080 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x555c2a2d71b0 0x555c2a6a77b0 secure :-1 s=READY pgs=996 cs=0 l=1 rev1=1 crypto rx=0x7fb754035000 tx=0x7fb754031520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e246eaf032bc739d server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:52:34.138 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:34.039+0000 7fb75affd640 1 -- 172.21.15.39:0/275458080 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb754031ad0 con 0x555c2a2d71b0 2024-09-21T03:52:34.138 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:34.039+0000 7fb75affd640 1 -- 172.21.15.39:0/275458080 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb754036030 con 0x555c2a2d71b0 2024-09-21T03:52:34.138 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:34.039+0000 7fb75affd640 1 -- 172.21.15.39:0/275458080 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb754004590 con 0x555c2a2d71b0 2024-09-21T03:52:34.138 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:34.040+0000 7fb77127e740 1 -- 172.21.15.39:0/275458080 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x555c2a2d71b0 msgr2=0x555c2a6a77b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:52:34.138 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:34.040+0000 7fb7692a7640 1 --2- 172.21.15.39:0/3808838003 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7fb7500b6700 0x7fb7500b8c10 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fb75c0e4ba0 tx=0x7fb75c0a1500 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:52:34.138 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:34.040+0000 7fb77127e740 1 --2- 172.21.15.39:0/275458080 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x555c2a2d71b0 0x555c2a6a77b0 secure :-1 s=READY pgs=996 cs=0 l=1 rev1=1 crypto rx=0x7fb754035000 tx=0x7fb754031520 comp rx=0 tx=0).stop 2024-09-21T03:52:34.138 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:34.040+0000 7fb77127e740 1 -- 172.21.15.39:0/275458080 shutdown_connections 2024-09-21T03:52:34.138 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:34.040+0000 7fb77127e740 1 --2- 172.21.15.39:0/275458080 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555c2a2df390 0x555c2a6a7cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:34.138 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:34.040+0000 7fb77127e740 1 --2- 172.21.15.39:0/275458080 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x555c2a2d71b0 0x555c2a6a77b0 unknown :-1 s=CLOSED pgs=996 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:34.138 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:34.040+0000 7fb77127e740 1 --2- 172.21.15.39:0/275458080 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555c2a300070 0x555c2a6a7270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:34.138 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:34.040+0000 7fb77127e740 1 -- 172.21.15.39:0/275458080 >> 172.21.15.39:0/275458080 conn(0x555c2a2f4460 msgr2=0x555c2a6a5160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:52:34.138 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:34.040+0000 7fb77127e740 1 -- 172.21.15.39:0/275458080 shutdown_connections 2024-09-21T03:52:34.139 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:34.040+0000 7fb77127e740 1 -- 172.21.15.39:0/275458080 wait complete. 2024-09-21T03:52:34.139 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:34.041+0000 7fb77127e740 1 -- start start 2024-09-21T03:52:34.139 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:34.041+0000 7fb77127e740 1 Processor -- start 2024-09-21T03:52:34.139 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:34.041+0000 7fb77127e740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555c2a2d71b0 0x555c2a6b6cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:52:34.139 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:34.041+0000 7fb77127e740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x555c2a2df390 0x555c2a6b7230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:52:34.139 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:34.041+0000 7fb77127e740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555c2a300070 0x555c2a6b77b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:52:34.139 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:34.041+0000 7fb77127e740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x555c2a46e700 con 0x555c2a2df390 2024-09-21T03:52:34.139 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:34.041+0000 7fb77127e740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x555c2a5a8110 con 0x555c2a2d71b0 2024-09-21T03:52:34.139 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:34.041+0000 7fb77127e740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x555c2a6b9d50 con 0x555c2a300070 2024-09-21T03:52:34.139 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:34.042+0000 7fb77127e740 10 client.0 _ll_register_callbacks cb 0x555c2a5c1990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:52:34.139 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:34.042+0000 7fb768aa6640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x555c2a2df390 0x555c2a6b7230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:52:34.139 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:34.042+0000 7fb768aa6640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x555c2a2df390 0x555c2a6b7230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:49962/0 (socket says 172.21.15.39:49962) 2024-09-21T03:52:34.139 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:34.042+0000 7fb768aa6640 1 -- 172.21.15.39:0/1098649687 learned_addr learned my addr 172.21.15.39:0/1098649687 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:52:34.139 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:34.042+0000 7fb763fff640 1 --2- 172.21.15.39:0/1098649687 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555c2a300070 0x555c2a6b77b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:52:34.139 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:34.042+0000 7fb763fff640 1 --2- 172.21.15.39:0/1098649687 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555c2a2d71b0 0x555c2a6b6cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:52:34.140 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:34.042+0000 7fb768aa6640 1 -- 172.21.15.39:0/1098649687 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555c2a300070 msgr2=0x555c2a6b77b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:52:34.140 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:34.042+0000 7fb768aa6640 1 --2- 172.21.15.39:0/1098649687 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555c2a300070 0x555c2a6b77b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:34.140 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:34.042+0000 7fb768aa6640 1 -- 172.21.15.39:0/1098649687 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555c2a2d71b0 msgr2=0x555c2a6b6cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:52:34.140 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:34.042+0000 7fb768aa6640 1 --2- 172.21.15.39:0/1098649687 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555c2a2d71b0 0x555c2a6b6cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:34.140 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:34.042+0000 7fb768aa6640 1 -- 172.21.15.39:0/1098649687 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555c2a5d5c40 con 0x555c2a2df390 2024-09-21T03:52:34.140 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:34.042+0000 7fb763fff640 1 --2- 172.21.15.39:0/1098649687 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555c2a2d71b0 0x555c2a6b6cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:52:34.140 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:34.042+0000 7fb768aa6640 1 --2- 172.21.15.39:0/1098649687 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x555c2a2df390 0x555c2a6b7230 secure :-1 s=READY pgs=997 cs=0 l=1 rev1=1 crypto rx=0x7fb75402ae00 tx=0x7fb75402a8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=80d7cb792b820bae server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:52:34.140 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:34.042+0000 7fb743fff640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-09-21T03:52:34.140 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:34.042+0000 7fb743fff640 1 -- 172.21.15.39:0/1098649687 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb754049040 con 0x555c2a2df390 2024-09-21T03:52:34.140 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:34.042+0000 7fb743fff640 1 -- 172.21.15.39:0/1098649687 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8484+0+0 (secure 0 0 0) 0x7fb754004720 con 0x555c2a2df390 2024-09-21T03:52:34.140 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:34.042+0000 7fb77127e740 1 -- 172.21.15.39:0/1098649687 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb754036030 con 0x555c2a2df390 2024-09-21T03:52:34.140 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:34.043+0000 7fb743fff640 1 -- 172.21.15.39:0/1098649687 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fb75405c050 con 0x555c2a2df390 2024-09-21T03:52:34.140 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:34.043+0000 7fb77127e740 10 client.17271 fetch_fsmap learned FSMap version 29 2024-09-21T03:52:34.141 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:34.043+0000 7fb77127e740 1 -- 172.21.15.39:0/1098649687 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x555c2a6c5a60 con 0x555c2a2df390 2024-09-21T03:52:34.141 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:34.043+0000 7fb743fff640 1 -- 172.21.15.39:0/1098649687 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb754036030 con 0x555c2a2df390 2024-09-21T03:52:34.141 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:34.043+0000 7fb743fff640 1 -- 172.21.15.39:0/1098649687 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fb75404c030 con 0x555c2a2df390 2024-09-21T03:52:34.141 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:34.043+0000 7fb77127e740 10 client.17271 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T03:52:34.141 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:34.043+0000 7fb77127e740 10 client.17271 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T03:52:34.141 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:34.043+0000 7fb77127e740 20 client.17271 populate_metadata read hostname 'smithi039' 2024-09-21T03:52:34.141 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:34.043+0000 7fb77127e740 1 --2- 172.21.15.39:0/1098649687 >> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] conn(0x555c2a2be9f0 0x555c2a6c8dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:52:34.141 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:34.043+0000 7fb77127e740 4 client.17271 mds_command: new command op to 24695 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:52:34.141 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:34.043+0000 7fb77127e740 1 -- 172.21.15.39:0/1098649687 --> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555c2a61fda0 con 0x555c2a2be9f0 2024-09-21T03:52:34.141 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:34.043+0000 7fb763fff640 1 --2- 172.21.15.39:0/1098649687 >> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] conn(0x555c2a2be9f0 0x555c2a6c8dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:52:34.141 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:34.044+0000 7fb763fff640 1 --2- 172.21.15.39:0/1098649687 >> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] conn(0x555c2a2be9f0 0x555c2a6c8dc0 crc :-1 s=READY pgs=117 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b1accde54952b848 server_cookie=3aff74dc46a55d0e in_seq=0 out_seq=0 2024-09-21T03:52:34.141 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:34.044+0000 7fb743fff640 10 client.17271 ms_handle_connect on v2:172.21.15.150:6838/2949386907 2024-09-21T03:52:34.141 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:34.045+0000 7fb743fff640 1 -- 172.21.15.39:0/1098649687 <== mds.0 v2:172.21.15.150:6838/2949386907 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x555c2a61fda0 con 0x555c2a2be9f0 2024-09-21T03:52:34.142 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:34.045+0000 7fb743fff640 10 client.17271 handle_command_reply: tid=0 2024-09-21T03:52:34.142 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:34.045+0000 7fb77127e740 1 client.17271 shutdown 2024-09-21T03:52:34.142 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:34.045+0000 7fb77127e740 1 -- 172.21.15.39:0/1098649687 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x555c2a2df390 msgr2=0x555c2a6b7230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:52:34.142 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:34.045+0000 7fb77127e740 1 --2- 172.21.15.39:0/1098649687 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x555c2a2df390 0x555c2a6b7230 secure :-1 s=READY pgs=997 cs=0 l=1 rev1=1 crypto rx=0x7fb75402ae00 tx=0x7fb75402a8e0 comp rx=0 tx=0).stop 2024-09-21T03:52:34.142 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:34.045+0000 7fb77127e740 1 -- 172.21.15.39:0/1098649687 shutdown_connections 2024-09-21T03:52:34.142 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:34.045+0000 7fb77127e740 1 --2- 172.21.15.39:0/1098649687 >> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] conn(0x555c2a2be9f0 0x555c2a6c8dc0 crc :-1 s=READY pgs=117 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:34.142 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:34.045+0000 7fb77127e740 1 --2- 172.21.15.39:0/1098649687 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555c2a300070 0x555c2a6b77b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:34.142 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:34.045+0000 7fb77127e740 1 --2- 172.21.15.39:0/1098649687 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x555c2a2df390 0x555c2a6b7230 unknown :-1 s=CLOSED pgs=997 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:34.142 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:34.045+0000 7fb77127e740 1 --2- 172.21.15.39:0/1098649687 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555c2a2d71b0 0x555c2a6b6cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:34.142 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:34.045+0000 7fb77127e740 1 -- 172.21.15.39:0/1098649687 >> 172.21.15.39:0/1098649687 conn(0x555c2a2f4460 msgr2=0x555c2a6a8c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:52:34.142 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:34.045+0000 7fb743fff640 0 client.17271 ms_handle_reset on v2:172.21.15.150:6838/2949386907 2024-09-21T03:52:34.142 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:34.045+0000 7fb77127e740 1 -- 172.21.15.39:0/1098649687 shutdown_connections 2024-09-21T03:52:34.143 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:34.045+0000 7fb77127e740 1 -- 172.21.15.39:0/1098649687 wait complete. 2024-09-21T03:52:34.143 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:34.045+0000 7fb77127e740 20 client.17271 trim_cache size 0 max 16384 2024-09-21T03:52:34.143 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:34.063+0000 7fb77127e740 1 Processor -- start 2024-09-21T03:52:34.143 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:34.063+0000 7fb77127e740 1 -- start start 2024-09-21T03:52:34.143 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:34.063+0000 7fb77127e740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x555c2a2d71b0 0x555c2a6b24a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:52:34.143 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:34.063+0000 7fb77127e740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555c2a2df390 0x555c2a6b8bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:52:34.143 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:34.063+0000 7fb77127e740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555c2a300070 0x555c2a6a89b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:52:34.143 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:34.063+0000 7fb77127e740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fb7641ad470 con 0x555c2a2d71b0 2024-09-21T03:52:34.143 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:34.063+0000 7fb77127e740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fb7641aefb0 con 0x555c2a2df390 2024-09-21T03:52:34.143 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:34.063+0000 7fb77127e740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fb7641adb40 con 0x555c2a300070 2024-09-21T03:52:34.143 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:34.063+0000 7fb763fff640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x555c2a2d71b0 0x555c2a6b24a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:52:34.143 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:34.063+0000 7fb763fff640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x555c2a2d71b0 0x555c2a6b24a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:49976/0 (socket says 172.21.15.39:49976) 2024-09-21T03:52:34.144 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:34.063+0000 7fb763fff640 1 -- 172.21.15.39:0/2081989878 learned_addr learned my addr 172.21.15.39:0/2081989878 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:52:34.144 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:34.063+0000 7fb768aa6640 1 --2- 172.21.15.39:0/2081989878 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555c2a2df390 0x555c2a6b8bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:52:34.144 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:34.063+0000 7fb763fff640 1 --2- 172.21.15.39:0/2081989878 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555c2a300070 0x555c2a6a89b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:52:34.144 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:34.063+0000 7fb763fff640 1 -- 172.21.15.39:0/2081989878 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555c2a300070 msgr2=0x555c2a6a89b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:52:34.144 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:34.063+0000 7fb763fff640 1 --2- 172.21.15.39:0/2081989878 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555c2a300070 0x555c2a6a89b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:34.144 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:34.063+0000 7fb763fff640 1 -- 172.21.15.39:0/2081989878 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555c2a2df390 msgr2=0x555c2a6b8bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:52:34.144 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:34.063+0000 7fb763fff640 1 --2- 172.21.15.39:0/2081989878 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555c2a2df390 0x555c2a6b8bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:34.144 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:34.063+0000 7fb763fff640 1 -- 172.21.15.39:0/2081989878 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555c2a6b7d90 con 0x555c2a2d71b0 2024-09-21T03:52:34.144 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:34.064+0000 7fb763fff640 1 --2- 172.21.15.39:0/2081989878 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x555c2a2d71b0 0x555c2a6b24a0 secure :-1 s=READY pgs=998 cs=0 l=1 rev1=1 crypto rx=0x7fb74c00e3d0 tx=0x7fb74c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=382f0a8c4fffaf0b server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:52:34.144 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:34.064+0000 7fb75a7fc640 1 -- 172.21.15.39:0/2081989878 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb74c02d070 con 0x555c2a2d71b0 2024-09-21T03:52:34.144 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:34.064+0000 7fb75a7fc640 1 -- 172.21.15.39:0/2081989878 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb74c0288d0 con 0x555c2a2d71b0 2024-09-21T03:52:34.145 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:34.064+0000 7fb75a7fc640 1 -- 172.21.15.39:0/2081989878 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb74c010630 con 0x555c2a2d71b0 2024-09-21T03:52:34.145 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:34.065+0000 7fb77127e740 1 -- 172.21.15.39:0/2081989878 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x555c2a2d71b0 msgr2=0x555c2a6b24a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:52:34.145 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:34.065+0000 7fb77127e740 1 --2- 172.21.15.39:0/2081989878 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x555c2a2d71b0 0x555c2a6b24a0 secure :-1 s=READY pgs=998 cs=0 l=1 rev1=1 crypto rx=0x7fb74c00e3d0 tx=0x7fb74c0046c0 comp rx=0 tx=0).stop 2024-09-21T03:52:34.145 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:34.065+0000 7fb77127e740 1 -- 172.21.15.39:0/2081989878 shutdown_connections 2024-09-21T03:52:34.145 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:34.065+0000 7fb77127e740 1 --2- 172.21.15.39:0/2081989878 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555c2a300070 0x555c2a6a89b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:34.145 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:34.065+0000 7fb77127e740 1 --2- 172.21.15.39:0/2081989878 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555c2a2df390 0x555c2a6b8bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:34.145 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:34.065+0000 7fb77127e740 1 --2- 172.21.15.39:0/2081989878 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x555c2a2d71b0 0x555c2a6b24a0 unknown :-1 s=CLOSED pgs=998 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:34.145 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:34.065+0000 7fb77127e740 1 -- 172.21.15.39:0/2081989878 >> 172.21.15.39:0/2081989878 conn(0x555c2a2f4460 msgr2=0x555c2a6a82a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:52:34.145 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:34.065+0000 7fb77127e740 1 -- 172.21.15.39:0/2081989878 shutdown_connections 2024-09-21T03:52:34.145 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:34.065+0000 7fb77127e740 1 -- 172.21.15.39:0/2081989878 wait complete. 2024-09-21T03:52:34.145 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:34.065+0000 7fb77127e740 1 -- start start 2024-09-21T03:52:34.145 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:34.065+0000 7fb77127e740 1 Processor -- start 2024-09-21T03:52:34.145 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:34.066+0000 7fb77127e740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x555c2a2d71b0 0x555c2a6cbc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:52:34.145 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:34.066+0000 7fb77127e740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555c2a2df390 0x555c2a6cc230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:52:34.145 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:34.066+0000 7fb77127e740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555c2a300070 0x555c2a728770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:52:34.146 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:34.066+0000 7fb77127e740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fb7641ae8e0 con 0x555c2a2d71b0 2024-09-21T03:52:34.146 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:34.066+0000 7fb77127e740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fb7641ae210 con 0x555c2a300070 2024-09-21T03:52:34.146 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:34.066+0000 7fb77127e740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fb7641ab450 con 0x555c2a2df390 2024-09-21T03:52:34.146 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:34.066+0000 7fb77127e740 10 client.0 _ll_register_callbacks cb 0x7fb74c00e950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:52:34.146 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:34.066+0000 7fb763fff640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x555c2a2d71b0 0x555c2a6cbc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:52:34.146 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:34.066+0000 7fb763fff640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x555c2a2d71b0 0x555c2a6cbc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:49984/0 (socket says 172.21.15.39:49984) 2024-09-21T03:52:34.146 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:34.066+0000 7fb763fff640 1 -- 172.21.15.39:0/2804982924 learned_addr learned my addr 172.21.15.39:0/2804982924 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:52:34.146 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:34.066+0000 7fb768aa6640 1 --2- 172.21.15.39:0/2804982924 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555c2a2df390 0x555c2a6cc230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:52:34.146 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:34.066+0000 7fb763fff640 1 -- 172.21.15.39:0/2804982924 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555c2a2df390 msgr2=0x555c2a6cc230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:52:34.146 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:34.066+0000 7fb763fff640 1 --2- 172.21.15.39:0/2804982924 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555c2a2df390 0x555c2a6cc230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:34.146 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:34.066+0000 7fb763fff640 1 -- 172.21.15.39:0/2804982924 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555c2a300070 msgr2=0x555c2a728770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:52:34.147 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:34.066+0000 7fb763fff640 1 --2- 172.21.15.39:0/2804982924 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555c2a300070 0x555c2a728770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:34.147 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:34.066+0000 7fb763fff640 1 -- 172.21.15.39:0/2804982924 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555c2a728de0 con 0x555c2a2d71b0 2024-09-21T03:52:34.147 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:34.066+0000 7fb763fff640 1 --2- 172.21.15.39:0/2804982924 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x555c2a2d71b0 0x555c2a6cbc50 secure :-1 s=READY pgs=999 cs=0 l=1 rev1=1 crypto rx=0x7fb74c006010 tx=0x7fb74c00e850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c315877e68d73b server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:52:34.147 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:34.066+0000 7fb743fff640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-09-21T03:52:34.147 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:34.066+0000 7fb743fff640 1 -- 172.21.15.39:0/2804982924 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb74c036020 con 0x555c2a2d71b0 2024-09-21T03:52:34.147 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:34.066+0000 7fb743fff640 1 -- 172.21.15.39:0/2804982924 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8484+0+0 (secure 0 0 0) 0x7fb74c028420 con 0x555c2a2d71b0 2024-09-21T03:52:34.147 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:34.067+0000 7fb77127e740 1 -- 172.21.15.39:0/2804982924 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb74c0288d0 con 0x555c2a2d71b0 2024-09-21T03:52:34.148 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:34.067+0000 7fb743fff640 1 -- 172.21.15.39:0/2804982924 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fb74c02d040 con 0x555c2a2d71b0 2024-09-21T03:52:34.148 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:34.067+0000 7fb77127e740 10 client.17277 fetch_fsmap learned FSMap version 29 2024-09-21T03:52:34.148 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:34.067+0000 7fb77127e740 1 -- 172.21.15.39:0/2804982924 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x555c2a733420 con 0x555c2a2d71b0 2024-09-21T03:52:34.148 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:34.067+0000 7fb743fff640 1 -- 172.21.15.39:0/2804982924 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb74c0288d0 con 0x555c2a2d71b0 2024-09-21T03:52:34.148 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:34.067+0000 7fb743fff640 1 -- 172.21.15.39:0/2804982924 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fb74c00f040 con 0x555c2a2d71b0 2024-09-21T03:52:34.148 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:34.067+0000 7fb77127e740 10 client.17277 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T03:52:34.149 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:34.067+0000 7fb77127e740 10 client.17277 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T03:52:34.149 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:34.067+0000 7fb77127e740 20 client.17277 populate_metadata read hostname 'smithi039' 2024-09-21T03:52:34.149 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:34.067+0000 7fb77127e740 1 --2- 172.21.15.39:0/2804982924 >> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] conn(0x555c2a293da0 0x555c2a735b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:52:34.149 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:34.067+0000 7fb77127e740 4 client.17277 mds_command: new command op to 24695 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-21T03:52:34.149 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:34.067+0000 7fb77127e740 1 -- 172.21.15.39:0/2804982924 --> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] -- command(tid 0: {"prefix": "scrub status"}) -- 0x555c2a61b850 con 0x555c2a293da0 2024-09-21T03:52:34.149 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:34.067+0000 7fb768aa6640 1 --2- 172.21.15.39:0/2804982924 >> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] conn(0x555c2a293da0 0x555c2a735b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:52:34.149 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:34.068+0000 7fb768aa6640 1 --2- 172.21.15.39:0/2804982924 >> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] conn(0x555c2a293da0 0x555c2a735b90 crc :-1 s=READY pgs=118 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5d194edd5cad8a9f server_cookie=6617cf6db208af1e in_seq=0 out_seq=0 2024-09-21T03:52:34.149 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:34.068+0000 7fb743fff640 10 client.17277 ms_handle_connect on v2:172.21.15.150:6838/2949386907 2024-09-21T03:52:34.149 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:34.068+0000 7fb743fff640 1 -- 172.21.15.39:0/2804982924 <== mds.0 v2:172.21.15.150:6838/2949386907 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3978 (crc 0 0 0) 0x555c2a61b850 con 0x555c2a293da0 2024-09-21T03:52:34.149 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:34.068+0000 7fb743fff640 10 client.17277 handle_command_reply: tid=0 2024-09-21T03:52:34.149 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:34.068+0000 7fb77127e740 1 client.17277 shutdown 2024-09-21T03:52:34.150 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:34.068+0000 7fb77127e740 1 -- 172.21.15.39:0/2804982924 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x555c2a2d71b0 msgr2=0x555c2a6cbc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:52:34.150 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:34.068+0000 7fb77127e740 1 --2- 172.21.15.39:0/2804982924 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x555c2a2d71b0 0x555c2a6cbc50 secure :-1 s=READY pgs=999 cs=0 l=1 rev1=1 crypto rx=0x7fb74c006010 tx=0x7fb74c00e850 comp rx=0 tx=0).stop 2024-09-21T03:52:34.150 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:34.068+0000 7fb77127e740 1 -- 172.21.15.39:0/2804982924 shutdown_connections 2024-09-21T03:52:34.150 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:34.068+0000 7fb77127e740 1 --2- 172.21.15.39:0/2804982924 >> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] conn(0x555c2a293da0 0x555c2a735b90 crc :-1 s=READY pgs=118 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:34.150 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:34.068+0000 7fb77127e740 1 --2- 172.21.15.39:0/2804982924 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555c2a300070 0x555c2a728770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:34.150 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:34.068+0000 7fb743fff640 0 client.17277 ms_handle_reset on v2:172.21.15.150:6838/2949386907 2024-09-21T03:52:34.150 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:34.068+0000 7fb77127e740 1 --2- 172.21.15.39:0/2804982924 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555c2a2df390 0x555c2a6cc230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:34.150 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:34.068+0000 7fb77127e740 1 --2- 172.21.15.39:0/2804982924 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x555c2a2d71b0 0x555c2a6cbc50 unknown :-1 s=CLOSED pgs=999 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:34.150 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:34.068+0000 7fb77127e740 1 -- 172.21.15.39:0/2804982924 >> 172.21.15.39:0/2804982924 conn(0x555c2a2f4460 msgr2=0x555c2a6a8c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:52:34.150 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:34.069+0000 7fb77127e740 1 -- 172.21.15.39:0/2804982924 shutdown_connections 2024-09-21T03:52:34.150 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:34.069+0000 7fb77127e740 1 -- 172.21.15.39:0/2804982924 wait complete. 2024-09-21T03:52:34.150 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:34.069+0000 7fb77127e740 20 client.17277 trim_cache size 0 max 16384 2024-09-21T03:52:34.151 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-09-21T03:52:34.151 INFO:teuthology.orchestra.run.smithi039.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-21T03:52:34.151 INFO:teuthology.orchestra.run.smithi039.stdout: "scrubs": { 2024-09-21T03:52:34.151 INFO:teuthology.orchestra.run.smithi039.stdout: "c7c700f3-2cd8-41ac-8381-b57dd1b37b02": { 2024-09-21T03:52:34.151 INFO:teuthology.orchestra.run.smithi039.stdout: "path": "/", 2024-09-21T03:52:34.151 INFO:teuthology.orchestra.run.smithi039.stdout: "tag": "c7c700f3-2cd8-41ac-8381-b57dd1b37b02", 2024-09-21T03:52:34.151 INFO:teuthology.orchestra.run.smithi039.stdout: "options": "recursive,force" 2024-09-21T03:52:34.151 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-21T03:52:34.151 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:52:34.151 INFO:teuthology.orchestra.run.smithi039.stdout: "scrub_stats": { 2024-09-21T03:52:34.151 INFO:teuthology.orchestra.run.smithi039.stdout: "4bda6ee3-7237-4ee4-96ce-150ad50f26dc": { 2024-09-21T03:52:34.151 INFO:teuthology.orchestra.run.smithi039.stdout: "start_time": "2024-09-21T03:52:18.622141+0000", 2024-09-21T03:52:34.151 INFO:teuthology.orchestra.run.smithi039.stdout: "path": "/", 2024-09-21T03:52:34.152 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_started": 0, 2024-09-21T03:52:34.152 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_passed": 0, 2024-09-21T03:52:34.152 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_failed": 0, 2024-09-21T03:52:34.152 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_skipped": 0 2024-09-21T03:52:34.152 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:52:34.152 INFO:teuthology.orchestra.run.smithi039.stdout: "ae1fc4cf-5458-47d0-afcb-282b136dba15": { 2024-09-21T03:52:34.152 INFO:teuthology.orchestra.run.smithi039.stdout: "start_time": "2024-09-21T03:51:18.560378+0000", 2024-09-21T03:52:34.152 INFO:teuthology.orchestra.run.smithi039.stdout: "path": "/", 2024-09-21T03:52:34.152 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_started": 0, 2024-09-21T03:52:34.152 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_passed": 0, 2024-09-21T03:52:34.152 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_failed": 0, 2024-09-21T03:52:34.152 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_skipped": 0 2024-09-21T03:52:34.152 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:52:34.152 INFO:teuthology.orchestra.run.smithi039.stdout: "160750d8-401f-4810-866d-e63e7e4bc3e0": { 2024-09-21T03:52:34.153 INFO:teuthology.orchestra.run.smithi039.stdout: "start_time": "2024-09-21T03:51:04.208673+0000", 2024-09-21T03:52:34.153 INFO:teuthology.orchestra.run.smithi039.stdout: "path": "/", 2024-09-21T03:52:34.153 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_started": 0, 2024-09-21T03:52:34.153 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_passed": 0, 2024-09-21T03:52:34.153 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_failed": 0, 2024-09-21T03:52:34.153 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_skipped": 0 2024-09-21T03:52:34.153 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:52:34.153 INFO:teuthology.orchestra.run.smithi039.stdout: "74ab917e-ecea-4778-a1c3-c184214fb24c": { 2024-09-21T03:52:34.153 INFO:teuthology.orchestra.run.smithi039.stdout: "start_time": "2024-09-21T03:50:17.726630+0000", 2024-09-21T03:52:34.153 INFO:teuthology.orchestra.run.smithi039.stdout: "path": "/", 2024-09-21T03:52:34.153 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_started": 0, 2024-09-21T03:52:34.154 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_passed": 0, 2024-09-21T03:52:34.154 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_failed": 0, 2024-09-21T03:52:34.154 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_skipped": 0 2024-09-21T03:52:34.154 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:52:34.154 INFO:teuthology.orchestra.run.smithi039.stdout: "6e760f1f-9048-4f92-bd66-d708fbef75b3": { 2024-09-21T03:52:34.154 INFO:teuthology.orchestra.run.smithi039.stdout: "start_time": "2024-09-21T03:50:03.711918+0000", 2024-09-21T03:52:34.154 INFO:teuthology.orchestra.run.smithi039.stdout: "path": "/", 2024-09-21T03:52:34.154 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_started": 0, 2024-09-21T03:52:34.154 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_passed": 0, 2024-09-21T03:52:34.154 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_failed": 0, 2024-09-21T03:52:34.154 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_skipped": 0 2024-09-21T03:52:34.154 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:52:34.154 INFO:teuthology.orchestra.run.smithi039.stdout: "e28ad743-2b22-45b3-8799-a638c2883623": { 2024-09-21T03:52:34.154 INFO:teuthology.orchestra.run.smithi039.stdout: "start_time": "2024-09-21T03:48:31.448821+0000", 2024-09-21T03:52:34.155 INFO:teuthology.orchestra.run.smithi039.stdout: "path": "/", 2024-09-21T03:52:34.155 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_started": 0, 2024-09-21T03:52:34.155 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_passed": 0, 2024-09-21T03:52:34.155 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_failed": 0, 2024-09-21T03:52:34.155 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_skipped": 0 2024-09-21T03:52:34.155 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:52:34.155 INFO:teuthology.orchestra.run.smithi039.stdout: "ede73e6e-3fd0-4a7f-b5bb-f68946f57eea": { 2024-09-21T03:52:34.155 INFO:teuthology.orchestra.run.smithi039.stdout: "start_time": "2024-09-21T03:51:32.235096+0000", 2024-09-21T03:52:34.155 INFO:teuthology.orchestra.run.smithi039.stdout: "path": "/", 2024-09-21T03:52:34.155 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_started": 0, 2024-09-21T03:52:34.155 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_passed": 0, 2024-09-21T03:52:34.155 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_failed": 0, 2024-09-21T03:52:34.155 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_skipped": 0 2024-09-21T03:52:34.156 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:52:34.156 INFO:teuthology.orchestra.run.smithi039.stdout: "29411307-eb8d-4339-8d2b-47ebd32c0080": { 2024-09-21T03:52:34.156 INFO:teuthology.orchestra.run.smithi039.stdout: "start_time": "2024-09-21T03:47:38.092920+0000", 2024-09-21T03:52:34.156 INFO:teuthology.orchestra.run.smithi039.stdout: "path": "/", 2024-09-21T03:52:34.156 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_started": 0, 2024-09-21T03:52:34.156 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_passed": 0, 2024-09-21T03:52:34.156 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_failed": 0, 2024-09-21T03:52:34.156 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_skipped": 0 2024-09-21T03:52:34.156 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:52:34.156 INFO:teuthology.orchestra.run.smithi039.stdout: "c7c700f3-2cd8-41ac-8381-b57dd1b37b02": { 2024-09-21T03:52:34.156 INFO:teuthology.orchestra.run.smithi039.stdout: "start_time": "2024-09-21T03:52:32.368858+0000", 2024-09-21T03:52:34.156 INFO:teuthology.orchestra.run.smithi039.stdout: "path": "/", 2024-09-21T03:52:34.156 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_started": 0, 2024-09-21T03:52:34.156 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_passed": 0, 2024-09-21T03:52:34.157 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_failed": 0, 2024-09-21T03:52:34.157 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_skipped": 0 2024-09-21T03:52:34.157 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:52:34.157 INFO:teuthology.orchestra.run.smithi039.stdout: "9860c2f4-b7b1-47aa-865a-9a1c4434a5f6": { 2024-09-21T03:52:34.157 INFO:teuthology.orchestra.run.smithi039.stdout: "start_time": "2024-09-21T03:49:17.718867+0000", 2024-09-21T03:52:34.157 INFO:teuthology.orchestra.run.smithi039.stdout: "path": "/", 2024-09-21T03:52:34.157 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_started": 0, 2024-09-21T03:52:34.157 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_passed": 0, 2024-09-21T03:52:34.157 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_failed": 0, 2024-09-21T03:52:34.157 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_skipped": 0 2024-09-21T03:52:34.157 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:52:34.157 INFO:teuthology.orchestra.run.smithi039.stdout: "c7e2b8e4-057d-435f-b2bd-b633de98035f": { 2024-09-21T03:52:34.157 INFO:teuthology.orchestra.run.smithi039.stdout: "start_time": "2024-09-21T03:47:19.400305+0000", 2024-09-21T03:52:34.157 INFO:teuthology.orchestra.run.smithi039.stdout: "path": "/", 2024-09-21T03:52:34.157 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_started": 0, 2024-09-21T03:52:34.158 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_passed": 0, 2024-09-21T03:52:34.158 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_failed": 0, 2024-09-21T03:52:34.158 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_skipped": 0 2024-09-21T03:52:34.158 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:52:34.158 INFO:teuthology.orchestra.run.smithi039.stdout: "10728de8-c49d-4edc-be2b-639249255e94": { 2024-09-21T03:52:34.158 INFO:teuthology.orchestra.run.smithi039.stdout: "start_time": "2024-09-21T03:47:03.965623+0000", 2024-09-21T03:52:34.158 INFO:teuthology.orchestra.run.smithi039.stdout: "path": "/", 2024-09-21T03:52:34.158 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_started": 0, 2024-09-21T03:52:34.158 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_passed": 0, 2024-09-21T03:52:34.158 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_failed": 0, 2024-09-21T03:52:34.158 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_skipped": 0 2024-09-21T03:52:34.158 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:52:34.158 INFO:teuthology.orchestra.run.smithi039.stdout: "02509e4f-3911-4983-b16f-df8ded0b9998": { 2024-09-21T03:52:34.158 INFO:teuthology.orchestra.run.smithi039.stdout: "start_time": "2024-09-21T03:46:17.889576+0000", 2024-09-21T03:52:34.159 INFO:teuthology.orchestra.run.smithi039.stdout: "path": "/", 2024-09-21T03:52:34.159 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_started": 0, 2024-09-21T03:52:34.159 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_passed": 0, 2024-09-21T03:52:34.159 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_failed": 0, 2024-09-21T03:52:34.159 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_skipped": 0 2024-09-21T03:52:34.159 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-21T03:52:34.159 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-21T03:52:34.159 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-09-21T03:52:34.159 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:34.069+0000 7fb76ad0a640 1 -- 172.21.15.39:0/3808838003 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7fb7500b6700 msgr2=0x7fb7500b8c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:52:34.160 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:34.069+0000 7fb76ad0a640 1 --2- 172.21.15.39:0/3808838003 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7fb7500b6700 0x7fb7500b8c10 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fb75c0e4ba0 tx=0x7fb75c0a1500 comp rx=0 tx=0).stop 2024-09-21T03:52:34.160 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:34.069+0000 7fb76ad0a640 1 -- 172.21.15.39:0/3808838003 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb764107c50 msgr2=0x7fb76419fd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:52:34.160 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:34.069+0000 7fb76ad0a640 1 --2- 172.21.15.39:0/3808838003 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb764107c50 0x7fb76419fd90 secure :-1 s=READY pgs=995 cs=0 l=1 rev1=1 crypto rx=0x7fb75c00b3f0 tx=0x7fb75c00b8b0 comp rx=0 tx=0).stop 2024-09-21T03:52:34.160 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:34.069+0000 7fb76ad0a640 1 -- 172.21.15.39:0/3808838003 shutdown_connections 2024-09-21T03:52:34.160 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:34.069+0000 7fb76ad0a640 1 --2- 172.21.15.39:0/3808838003 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7fb7500b6700 0x7fb7500b8c10 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:34.160 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:34.069+0000 7fb76ad0a640 1 --2- 172.21.15.39:0/3808838003 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb764107c50 0x7fb76419fd90 unknown :-1 s=CLOSED pgs=995 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:34.160 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:34.069+0000 7fb76ad0a640 1 --2- 172.21.15.39:0/3808838003 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb764107300 0x7fb76419aaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:34.160 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:34.069+0000 7fb76ad0a640 1 --2- 172.21.15.39:0/3808838003 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb764102100 0x7fb76419a550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:34.160 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:34.069+0000 7fb76ad0a640 1 -- 172.21.15.39:0/3808838003 >> 172.21.15.39:0/3808838003 conn(0x7fb764079e40 msgr2=0x7fb764106080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:52:34.160 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:34.069+0000 7fb76ad0a640 1 -- 172.21.15.39:0/3808838003 shutdown_connections 2024-09-21T03:52:34.160 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:52:34.069+0000 7fb76ad0a640 1 -- 172.21.15.39:0/3808838003 wait complete. 2024-09-21T03:52:34.968 INFO:tasks.cephfs.filesystem:scrub status for tag:c7c700f3-2cd8-41ac-8381-b57dd1b37b02 - {'path': '/', 'tag': 'c7c700f3-2cd8-41ac-8381-b57dd1b37b02', 'options': 'recursive,force'} 2024-09-21T03:52:35.584 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:52:35 smithi039 ceph-mon[31211]: pgmap v521: 97 pgs: 97 active+clean; 10 MiB data, 520 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 10 op/s 2024-09-21T03:52:35.584 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:52:35 smithi039 ceph-mon[31211]: scrub summary: idle 2024-09-21T03:52:35.718 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:52:35 smithi161 ceph-mon[34533]: pgmap v521: 97 pgs: 97 active+clean; 10 MiB data, 520 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 10 op/s 2024-09-21T03:52:35.718 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:52:35 smithi161 ceph-mon[34533]: scrub summary: idle 2024-09-21T03:52:35.724 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:52:35 smithi150 ceph-mon[34289]: pgmap v521: 97 pgs: 97 active+clean; 10 MiB data, 520 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 10 op/s 2024-09-21T03:52:35.725 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:52:35 smithi150 ceph-mon[34289]: scrub summary: idle 2024-09-21T03:52:37.584 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:52:37 smithi039 ceph-mon[31211]: pgmap v522: 97 pgs: 97 active+clean; 10 MiB data, 520 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 10 op/s 2024-09-21T03:52:37.718 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:52:37 smithi161 ceph-mon[34533]: pgmap v522: 97 pgs: 97 active+clean; 10 MiB data, 520 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 10 op/s 2024-09-21T03:52:37.724 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:52:37 smithi150 ceph-mon[34289]: pgmap v522: 97 pgs: 97 active+clean; 10 MiB data, 520 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 10 op/s 2024-09-21T03:52:39.584 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:52:39 smithi039 ceph-mon[31211]: pgmap v523: 97 pgs: 97 active+clean; 10 MiB data, 520 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 8 op/s 2024-09-21T03:52:39.718 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:52:39 smithi161 ceph-mon[34533]: pgmap v523: 97 pgs: 97 active+clean; 10 MiB data, 520 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 8 op/s 2024-09-21T03:52:39.725 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:52:39 smithi150 ceph-mon[34289]: pgmap v523: 97 pgs: 97 active+clean; 10 MiB data, 520 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 8 op/s 2024-09-21T03:52:41.584 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:52:41 smithi039 ceph-mon[31211]: pgmap v524: 97 pgs: 97 active+clean; 10 MiB data, 520 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 11 op/s 2024-09-21T03:52:41.718 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:52:41 smithi161 ceph-mon[34533]: pgmap v524: 97 pgs: 97 active+clean; 10 MiB data, 520 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 11 op/s 2024-09-21T03:52:41.724 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:52:41 smithi150 ceph-mon[34289]: pgmap v524: 97 pgs: 97 active+clean; 10 MiB data, 520 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 11 op/s 2024-09-21T03:52:43.584 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:52:43 smithi039 ceph-mon[31211]: pgmap v525: 97 pgs: 97 active+clean; 10 MiB data, 520 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 9 op/s 2024-09-21T03:52:43.718 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:52:43 smithi161 ceph-mon[34533]: pgmap v525: 97 pgs: 97 active+clean; 10 MiB data, 520 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 9 op/s 2024-09-21T03:52:43.724 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:52:43 smithi150 ceph-mon[34289]: pgmap v525: 97 pgs: 97 active+clean; 10 MiB data, 520 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 9 op/s 2024-09-21T03:52:45.584 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:52:45 smithi039 ceph-mon[31211]: pgmap v526: 97 pgs: 97 active+clean; 10 MiB data, 520 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 10 op/s 2024-09-21T03:52:45.718 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:52:45 smithi161 ceph-mon[34533]: pgmap v526: 97 pgs: 97 active+clean; 10 MiB data, 520 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 10 op/s 2024-09-21T03:52:45.724 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:52:45 smithi150 ceph-mon[34289]: pgmap v526: 97 pgs: 97 active+clean; 10 MiB data, 520 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 10 op/s 2024-09-21T03:52:47.584 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:52:47 smithi039 ceph-mon[31211]: pgmap v527: 97 pgs: 97 active+clean; 10 MiB data, 520 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 10 op/s 2024-09-21T03:52:47.718 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:52:47 smithi161 ceph-mon[34533]: pgmap v527: 97 pgs: 97 active+clean; 10 MiB data, 520 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 10 op/s 2024-09-21T03:52:47.724 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:52:47 smithi150 ceph-mon[34289]: pgmap v527: 97 pgs: 97 active+clean; 10 MiB data, 520 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 10 op/s 2024-09-21T03:52:49.584 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:52:49 smithi039 ceph-mon[31211]: pgmap v528: 97 pgs: 97 active+clean; 10 MiB data, 520 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-09-21T03:52:49.718 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:52:49 smithi161 ceph-mon[34533]: pgmap v528: 97 pgs: 97 active+clean; 10 MiB data, 520 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-09-21T03:52:49.724 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:52:49 smithi150 ceph-mon[34289]: pgmap v528: 97 pgs: 97 active+clean; 10 MiB data, 520 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-09-21T03:52:51.584 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:52:51 smithi039 ceph-mon[31211]: pgmap v529: 97 pgs: 97 active+clean; 10 MiB data, 520 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-09-21T03:52:51.717 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:52:51 smithi161 ceph-mon[34533]: pgmap v529: 97 pgs: 97 active+clean; 10 MiB data, 520 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-09-21T03:52:51.725 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:52:51 smithi150 ceph-mon[34289]: pgmap v529: 97 pgs: 97 active+clean; 10 MiB data, 520 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-09-21T03:52:53.584 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:52:53 smithi039 ceph-mon[31211]: pgmap v530: 97 pgs: 97 active+clean; 10 MiB data, 520 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-21T03:52:53.718 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:52:53 smithi161 ceph-mon[34533]: pgmap v530: 97 pgs: 97 active+clean; 10 MiB data, 520 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-21T03:52:53.725 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:52:53 smithi150 ceph-mon[34289]: pgmap v530: 97 pgs: 97 active+clean; 10 MiB data, 520 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-21T03:52:55.718 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:52:55 smithi161 ceph-mon[34533]: pgmap v531: 97 pgs: 97 active+clean; 10 MiB data, 520 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-09-21T03:52:55.724 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:52:55 smithi150 ceph-mon[34289]: pgmap v531: 97 pgs: 97 active+clean; 10 MiB data, 520 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-09-21T03:52:55.834 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:52:55 smithi039 ceph-mon[31211]: pgmap v531: 97 pgs: 97 active+clean; 10 MiB data, 520 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-09-21T03:52:57.718 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:52:57 smithi161 ceph-mon[34533]: pgmap v532: 97 pgs: 97 active+clean; 10 MiB data, 520 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 10 op/s 2024-09-21T03:52:57.725 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:52:57 smithi150 ceph-mon[34289]: pgmap v532: 97 pgs: 97 active+clean; 10 MiB data, 520 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 10 op/s 2024-09-21T03:52:57.833 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:52:57 smithi039 ceph-mon[31211]: pgmap v532: 97 pgs: 97 active+clean; 10 MiB data, 520 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 10 op/s 2024-09-21T03:52:59.718 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:52:59 smithi161 ceph-mon[34533]: pgmap v533: 97 pgs: 97 active+clean; 10 MiB data, 520 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-09-21T03:52:59.725 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:52:59 smithi150 ceph-mon[34289]: pgmap v533: 97 pgs: 97 active+clean; 10 MiB data, 520 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-09-21T03:52:59.834 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:52:59 smithi039 ceph-mon[31211]: pgmap v533: 97 pgs: 97 active+clean; 10 MiB data, 520 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-09-21T03:53:01.718 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:53:01 smithi161 ceph-mon[34533]: pgmap v534: 97 pgs: 97 active+clean; 10 MiB data, 520 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-09-21T03:53:01.724 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:53:01 smithi150 ceph-mon[34289]: pgmap v534: 97 pgs: 97 active+clean; 10 MiB data, 520 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-09-21T03:53:01.834 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:53:01 smithi039 ceph-mon[31211]: pgmap v534: 97 pgs: 97 active+clean; 10 MiB data, 520 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-09-21T03:53:03.718 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:53:03 smithi161 ceph-mon[34533]: pgmap v535: 97 pgs: 97 active+clean; 10 MiB data, 520 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-21T03:53:03.725 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:53:03 smithi150 ceph-mon[34289]: pgmap v535: 97 pgs: 97 active+clean; 10 MiB data, 520 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-21T03:53:03.833 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:53:03 smithi039 ceph-mon[31211]: pgmap v535: 97 pgs: 97 active+clean; 10 MiB data, 520 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-21T03:53:04.970 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 51116d5c-77ca-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub status 2024-09-21T03:53:05.244 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/51116d5c-77ca-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:53:05.718 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:53:05 smithi161 ceph-mon[34533]: pgmap v536: 97 pgs: 97 active+clean; 10 MiB data, 520 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 767 B/s wr, 10 op/s 2024-09-21T03:53:05.724 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:53:05 smithi150 ceph-mon[34289]: pgmap v536: 97 pgs: 97 active+clean; 10 MiB data, 520 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 767 B/s wr, 10 op/s 2024-09-21T03:53:05.816 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:53:05 smithi039 ceph-mon[31211]: pgmap v536: 97 pgs: 97 active+clean; 10 MiB data, 520 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 767 B/s wr, 10 op/s 2024-09-21T03:53:05.912 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:05.909+0000 7f3131d24640 1 -- 172.21.15.39:0/3449692831 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f312c109df0 msgr2=0x7f312c110620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:53:05.912 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:05.909+0000 7f3131d24640 1 --2- 172.21.15.39:0/3449692831 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f312c109df0 0x7f312c110620 secure :-1 s=READY pgs=1000 cs=0 l=1 rev1=1 crypto rx=0x7f3124009ca0 tx=0x7f312402a8c0 comp rx=0 tx=0).stop 2024-09-21T03:53:05.912 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:05.909+0000 7f3131d24640 1 -- 172.21.15.39:0/3449692831 shutdown_connections 2024-09-21T03:53:05.912 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:05.909+0000 7f3131d24640 1 --2- 172.21.15.39:0/3449692831 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f312c109df0 0x7f312c110620 unknown :-1 s=CLOSED pgs=1000 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:05.912 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:05.909+0000 7f3131d24640 1 --2- 172.21.15.39:0/3449692831 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f312c1094a0 0x7f312c1098b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:05.912 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:05.909+0000 7f3131d24640 1 --2- 172.21.15.39:0/3449692831 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f312c103b60 0x7f312c103f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:05.912 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:05.909+0000 7f3131d24640 1 -- 172.21.15.39:0/3449692831 >> 172.21.15.39:0/3449692831 conn(0x7f312c07b850 msgr2=0x7f312c07bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:53:05.912 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:05.909+0000 7f3131d24640 1 -- 172.21.15.39:0/3449692831 shutdown_connections 2024-09-21T03:53:05.912 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:05.909+0000 7f3131d24640 1 -- 172.21.15.39:0/3449692831 wait complete. 2024-09-21T03:53:05.913 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:05.910+0000 7f3131d24640 1 Processor -- start 2024-09-21T03:53:05.913 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:05.910+0000 7f3131d24640 1 -- start start 2024-09-21T03:53:05.914 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:05.910+0000 7f3131d24640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f312c103b60 0x7f312c19a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:53:05.914 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:05.910+0000 7f3131d24640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f312c1094a0 0x7f312c19aa20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:53:05.914 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:05.910+0000 7f3131d24640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f312c109df0 0x7f312c19fcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:53:05.914 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:05.910+0000 7f3131d24640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f312c113ab0 con 0x7f312c109df0 2024-09-21T03:53:05.914 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:05.910+0000 7f3131d24640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f312c113930 con 0x7f312c1094a0 2024-09-21T03:53:05.914 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:05.910+0000 7f3131d24640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f312c113c30 con 0x7f312c103b60 2024-09-21T03:53:05.914 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:05.911+0000 7f312bfff640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f312c109df0 0x7f312c19fcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:53:05.914 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:05.911+0000 7f312bfff640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f312c109df0 0x7f312c19fcc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:49270/0 (socket says 172.21.15.39:49270) 2024-09-21T03:53:05.915 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:05.911+0000 7f312bfff640 1 -- 172.21.15.39:0/916762225 learned_addr learned my addr 172.21.15.39:0/916762225 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:53:05.915 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:05.911+0000 7f312b7fe640 1 --2- 172.21.15.39:0/916762225 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f312c103b60 0x7f312c19a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:53:05.915 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:05.911+0000 7f312affd640 1 --2- 172.21.15.39:0/916762225 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f312c1094a0 0x7f312c19aa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:53:05.915 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:05.911+0000 7f312bfff640 1 -- 172.21.15.39:0/916762225 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f312c103b60 msgr2=0x7f312c19a480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:53:05.915 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:05.911+0000 7f312bfff640 1 --2- 172.21.15.39:0/916762225 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f312c103b60 0x7f312c19a480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:05.915 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:05.911+0000 7f312bfff640 1 -- 172.21.15.39:0/916762225 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f312c1094a0 msgr2=0x7f312c19aa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:53:05.915 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:05.911+0000 7f312bfff640 1 --2- 172.21.15.39:0/916762225 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f312c1094a0 0x7f312c19aa20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:05.915 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:05.911+0000 7f312bfff640 1 -- 172.21.15.39:0/916762225 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f312c1a0330 con 0x7f312c109df0 2024-09-21T03:53:05.915 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:05.911+0000 7f312b7fe640 1 --2- 172.21.15.39:0/916762225 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f312c103b60 0x7f312c19a480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:53:05.915 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:05.911+0000 7f312bfff640 1 --2- 172.21.15.39:0/916762225 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f312c109df0 0x7f312c19fcc0 secure :-1 s=READY pgs=1001 cs=0 l=1 rev1=1 crypto rx=0x7f3124005cc0 tx=0x7f312402f380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:53:05.915 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:05.911+0000 7f3128ff9640 1 -- 172.21.15.39:0/916762225 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3124003bf0 con 0x7f312c109df0 2024-09-21T03:53:05.915 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:05.911+0000 7f3128ff9640 1 -- 172.21.15.39:0/916762225 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3124003d90 con 0x7f312c109df0 2024-09-21T03:53:05.916 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:05.911+0000 7f3131d24640 1 -- 172.21.15.39:0/916762225 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f312c1a05c0 con 0x7f312c109df0 2024-09-21T03:53:05.916 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:05.912+0000 7f3128ff9640 1 -- 172.21.15.39:0/916762225 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f312402fbc0 con 0x7f312c109df0 2024-09-21T03:53:05.916 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:05.912+0000 7f3131d24640 1 -- 172.21.15.39:0/916762225 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f312c1a0b00 con 0x7f312c109df0 2024-09-21T03:53:05.917 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:05.914+0000 7f3138298740 1 Processor -- start 2024-09-21T03:53:05.917 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:05.914+0000 7f3128ff9640 1 -- 172.21.15.39:0/916762225 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 27) ==== 154643+0+0 (secure 0 0 0) 0x7f312402fd60 con 0x7f312c109df0 2024-09-21T03:53:05.917 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:05.915+0000 7f3138298740 1 -- start start 2024-09-21T03:53:05.917 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:05.915+0000 7f3138298740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5577444d2070 0x557744879650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:53:05.917 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:05.915+0000 7f3138298740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5577444a91b0 0x557744879b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:53:05.917 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:05.915+0000 7f3138298740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5577444b1390 0x55774487a0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:53:05.917 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:05.915+0000 7f3138298740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x5577447efe50 con 0x5577444d2070 2024-09-21T03:53:05.917 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:05.915+0000 7f3138298740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x5577447df280 con 0x5577444a91b0 2024-09-21T03:53:05.917 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:05.915+0000 7f3138298740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x5577447e3090 con 0x5577444b1390 2024-09-21T03:53:05.917 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:05.915+0000 7f312affd640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5577444d2070 0x557744879650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:53:05.917 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:05.915+0000 7f312affd640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5577444d2070 0x557744879650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:49276/0 (socket says 172.21.15.39:49276) 2024-09-21T03:53:05.918 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:05.915+0000 7f312affd640 1 -- 172.21.15.39:0/3787480897 learned_addr learned my addr 172.21.15.39:0/3787480897 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:53:05.918 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:05.915+0000 7f312b7fe640 1 --2- 172.21.15.39:0/3787480897 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5577444a91b0 0x557744879b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:53:05.918 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:05.915+0000 7f312affd640 1 --2- 172.21.15.39:0/3787480897 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5577444b1390 0x55774487a0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:53:05.919 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:05.915+0000 7f312affd640 1 -- 172.21.15.39:0/3787480897 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5577444b1390 msgr2=0x55774487a0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:53:05.919 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:05.915+0000 7f312affd640 1 --2- 172.21.15.39:0/3787480897 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5577444b1390 0x55774487a0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:05.919 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:05.915+0000 7f312affd640 1 -- 172.21.15.39:0/3787480897 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5577444a91b0 msgr2=0x557744879b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:53:05.919 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:05.915+0000 7f312affd640 1 --2- 172.21.15.39:0/3787480897 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5577444a91b0 0x557744879b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:05.920 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:05.916+0000 7f312affd640 1 -- 172.21.15.39:0/3787480897 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55774453d650 con 0x5577444d2070 2024-09-21T03:53:05.920 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:05.916+0000 7f3128ff9640 1 --2- 172.21.15.39:0/916762225 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f31140b66b0 0x7f31140b8bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:53:05.920 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:05.916+0000 7f312affd640 1 --2- 172.21.15.39:0/3787480897 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5577444d2070 0x557744879650 secure :-1 s=READY pgs=1002 cs=0 l=1 rev1=1 crypto rx=0x7f3120012880 tx=0x7f312000c5c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6713d1cbf017baf2 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:53:05.920 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:05.916+0000 7f312b7fe640 1 --2- 172.21.15.39:0/916762225 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f31140b66b0 0x7f31140b8bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:53:05.920 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:05.916+0000 7f3111ffb640 1 -- 172.21.15.39:0/3787480897 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3120004260 con 0x5577444d2070 2024-09-21T03:53:05.920 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:05.916+0000 7f3111ffb640 1 -- 172.21.15.39:0/3787480897 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3120004400 con 0x5577444d2070 2024-09-21T03:53:05.920 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:05.916+0000 7f3111ffb640 1 -- 172.21.15.39:0/3787480897 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3120003ac0 con 0x5577444d2070 2024-09-21T03:53:05.920 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:05.916+0000 7f3128ff9640 1 -- 172.21.15.39:0/916762225 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8484+0+0 (secure 0 0 0) 0x7f3124108d30 con 0x7f312c109df0 2024-09-21T03:53:05.920 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:05.917+0000 7f3138298740 1 -- 172.21.15.39:0/3787480897 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5577444d2070 msgr2=0x557744879650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:53:05.920 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:05.917+0000 7f312b7fe640 1 --2- 172.21.15.39:0/916762225 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f31140b66b0 0x7f31140b8bc0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f311c005fd0 tx=0x7f311c00b5a0 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:53:05.920 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:05.917+0000 7f3138298740 1 --2- 172.21.15.39:0/3787480897 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5577444d2070 0x557744879650 secure :-1 s=READY pgs=1002 cs=0 l=1 rev1=1 crypto rx=0x7f3120012880 tx=0x7f312000c5c0 comp rx=0 tx=0).stop 2024-09-21T03:53:05.920 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:05.917+0000 7f3138298740 1 -- 172.21.15.39:0/3787480897 shutdown_connections 2024-09-21T03:53:05.920 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:05.917+0000 7f3138298740 1 --2- 172.21.15.39:0/3787480897 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5577444b1390 0x55774487a0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:05.920 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:05.917+0000 7f3138298740 1 --2- 172.21.15.39:0/3787480897 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5577444a91b0 0x557744879b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:05.920 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:05.917+0000 7f3138298740 1 --2- 172.21.15.39:0/3787480897 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5577444d2070 0x557744879650 unknown :-1 s=CLOSED pgs=1002 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:05.921 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:05.917+0000 7f3138298740 1 -- 172.21.15.39:0/3787480897 >> 172.21.15.39:0/3787480897 conn(0x5577444c6460 msgr2=0x557744877540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:53:05.921 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:05.917+0000 7f3138298740 1 -- 172.21.15.39:0/3787480897 shutdown_connections 2024-09-21T03:53:05.921 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:05.917+0000 7f3138298740 1 -- 172.21.15.39:0/3787480897 wait complete. 2024-09-21T03:53:05.921 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:05.917+0000 7f3138298740 1 -- start start 2024-09-21T03:53:05.921 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:05.918+0000 7f3138298740 1 Processor -- start 2024-09-21T03:53:05.923 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:05.918+0000 7f3138298740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5577444a91b0 0x557744889160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:53:05.923 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:05.918+0000 7f3138298740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5577444b1390 0x5577448896a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:53:05.923 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:05.918+0000 7f3138298740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5577444d2070 0x5577448954c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:53:05.923 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:05.918+0000 7f3138298740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x557744640700 con 0x5577444a91b0 2024-09-21T03:53:05.923 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:05.918+0000 7f3138298740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x55774477a110 con 0x5577444b1390 2024-09-21T03:53:05.923 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:05.918+0000 7f3138298740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55774488c160 con 0x5577444d2070 2024-09-21T03:53:05.923 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:05.918+0000 7f3138298740 10 client.0 _ll_register_callbacks cb 0x557744793990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:53:05.924 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:05.918+0000 7f312b7fe640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5577444a91b0 0x557744889160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:53:05.924 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:05.918+0000 7f312b7fe640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5577444a91b0 0x557744889160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:49282/0 (socket says 172.21.15.39:49282) 2024-09-21T03:53:05.924 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:05.918+0000 7f312b7fe640 1 -- 172.21.15.39:0/2568350085 learned_addr learned my addr 172.21.15.39:0/2568350085 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:53:05.924 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:05.918+0000 7f312bfff640 1 --2- 172.21.15.39:0/2568350085 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5577444d2070 0x5577448954c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:53:05.924 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:05.918+0000 7f312affd640 1 --2- 172.21.15.39:0/2568350085 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5577444b1390 0x5577448896a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:53:05.924 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:05.919+0000 7f312b7fe640 1 -- 172.21.15.39:0/2568350085 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5577444d2070 msgr2=0x5577448954c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:53:05.924 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:05.919+0000 7f312b7fe640 1 --2- 172.21.15.39:0/2568350085 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5577444d2070 0x5577448954c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:05.924 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:05.919+0000 7f312b7fe640 1 -- 172.21.15.39:0/2568350085 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5577444b1390 msgr2=0x5577448896a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:53:05.924 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:05.919+0000 7f312b7fe640 1 --2- 172.21.15.39:0/2568350085 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5577444b1390 0x5577448896a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:05.924 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:05.919+0000 7f312b7fe640 1 -- 172.21.15.39:0/2568350085 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5577447a7c40 con 0x5577444a91b0 2024-09-21T03:53:05.924 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:05.919+0000 7f312bfff640 1 --2- 172.21.15.39:0/2568350085 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5577444d2070 0x5577448954c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:53:05.924 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:05.919+0000 7f312b7fe640 1 --2- 172.21.15.39:0/2568350085 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5577444a91b0 0x557744889160 secure :-1 s=READY pgs=1003 cs=0 l=1 rev1=1 crypto rx=0x7f311c00bbf0 tx=0x7f311c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f867e79af9ac852 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:53:05.924 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:05.919+0000 7f310affd640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-09-21T03:53:05.924 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:05.919+0000 7f310affd640 1 -- 172.21.15.39:0/2568350085 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f311c01de60 con 0x5577444a91b0 2024-09-21T03:53:05.924 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:05.919+0000 7f310affd640 1 -- 172.21.15.39:0/2568350085 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8484+0+0 (secure 0 0 0) 0x7f311c01e820 con 0x5577444a91b0 2024-09-21T03:53:05.925 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:05.919+0000 7f3138298740 1 -- 172.21.15.39:0/2568350085 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3120004400 con 0x5577444a91b0 2024-09-21T03:53:05.925 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:05.919+0000 7f310affd640 1 -- 172.21.15.39:0/2568350085 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f311c01ea20 con 0x5577444a91b0 2024-09-21T03:53:05.925 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:05.919+0000 7f3138298740 10 client.17289 fetch_fsmap learned FSMap version 29 2024-09-21T03:53:05.925 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:05.919+0000 7f3138298740 1 -- 172.21.15.39:0/2568350085 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x557744897fa0 con 0x5577444a91b0 2024-09-21T03:53:05.925 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:05.920+0000 7f310affd640 1 -- 172.21.15.39:0/2568350085 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3120004400 con 0x5577444a91b0 2024-09-21T03:53:05.925 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:05.920+0000 7f310affd640 1 -- 172.21.15.39:0/2568350085 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f311c01b390 con 0x5577444a91b0 2024-09-21T03:53:05.925 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:05.920+0000 7f3138298740 10 client.17289 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T03:53:05.925 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:05.920+0000 7f3138298740 10 client.17289 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T03:53:05.925 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:05.920+0000 7f3138298740 20 client.17289 populate_metadata read hostname 'smithi039' 2024-09-21T03:53:05.925 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:05.920+0000 7f3138298740 1 --2- 172.21.15.39:0/2568350085 >> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] conn(0x5577444909f0 0x55774489b400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:53:05.925 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:05.920+0000 7f3138298740 4 client.17289 mds_command: new command op to 24695 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:53:05.925 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:05.920+0000 7f3138298740 1 -- 172.21.15.39:0/2568350085 --> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5577447f1da0 con 0x5577444909f0 2024-09-21T03:53:05.925 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:05.920+0000 7f312affd640 1 --2- 172.21.15.39:0/2568350085 >> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] conn(0x5577444909f0 0x55774489b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:53:05.926 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:05.921+0000 7f312affd640 1 --2- 172.21.15.39:0/2568350085 >> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] conn(0x5577444909f0 0x55774489b400 crc :-1 s=READY pgs=119 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cabcafd528883304 server_cookie=4834b48cd7a166c6 in_seq=0 out_seq=0 2024-09-21T03:53:05.926 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:05.921+0000 7f310affd640 10 client.17289 ms_handle_connect on v2:172.21.15.150:6838/2949386907 2024-09-21T03:53:05.927 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:05.922+0000 7f310affd640 1 -- 172.21.15.39:0/2568350085 <== mds.0 v2:172.21.15.150:6838/2949386907 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5577447f1da0 con 0x5577444909f0 2024-09-21T03:53:05.927 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:05.922+0000 7f310affd640 10 client.17289 handle_command_reply: tid=0 2024-09-21T03:53:05.927 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:05.922+0000 7f3138298740 1 client.17289 shutdown 2024-09-21T03:53:05.927 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:05.922+0000 7f3138298740 1 -- 172.21.15.39:0/2568350085 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5577444a91b0 msgr2=0x557744889160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:53:05.927 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:05.922+0000 7f3138298740 1 --2- 172.21.15.39:0/2568350085 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5577444a91b0 0x557744889160 secure :-1 s=READY pgs=1003 cs=0 l=1 rev1=1 crypto rx=0x7f311c00bbf0 tx=0x7f311c0046c0 comp rx=0 tx=0).stop 2024-09-21T03:53:05.927 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:05.922+0000 7f3138298740 1 -- 172.21.15.39:0/2568350085 shutdown_connections 2024-09-21T03:53:05.927 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:05.922+0000 7f3138298740 1 --2- 172.21.15.39:0/2568350085 >> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] conn(0x5577444909f0 0x55774489b400 crc :-1 s=READY pgs=119 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:05.927 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:05.922+0000 7f310affd640 0 client.17289 ms_handle_reset on v2:172.21.15.150:6838/2949386907 2024-09-21T03:53:05.927 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:05.922+0000 7f3138298740 1 --2- 172.21.15.39:0/2568350085 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5577444d2070 0x5577448954c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:05.927 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:05.922+0000 7f3138298740 1 --2- 172.21.15.39:0/2568350085 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5577444b1390 0x5577448896a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:05.927 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:05.922+0000 7f3138298740 1 --2- 172.21.15.39:0/2568350085 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5577444a91b0 0x557744889160 unknown :-1 s=CLOSED pgs=1003 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:05.927 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:05.922+0000 7f3138298740 1 -- 172.21.15.39:0/2568350085 >> 172.21.15.39:0/2568350085 conn(0x5577444c6460 msgr2=0x55774487c7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:53:05.927 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:05.922+0000 7f3138298740 1 -- 172.21.15.39:0/2568350085 shutdown_connections 2024-09-21T03:53:05.927 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:05.922+0000 7f3138298740 1 -- 172.21.15.39:0/2568350085 wait complete. 2024-09-21T03:53:05.928 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:05.922+0000 7f3138298740 20 client.17289 trim_cache size 0 max 16384 2024-09-21T03:53:05.942 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:05.940+0000 7f3138298740 1 Processor -- start 2024-09-21T03:53:05.942 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:05.940+0000 7f3138298740 1 -- start start 2024-09-21T03:53:05.942 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:05.940+0000 7f3138298740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5577444909f0 0x557744888050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:53:05.942 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:05.940+0000 7f3138298740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5577444a91b0 0x55774488ad90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:53:05.942 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:05.940+0000 7f3138298740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5577444b1390 0x557744876b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:53:05.943 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:05.940+0000 7f3138298740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f312c1ad440 con 0x5577444a91b0 2024-09-21T03:53:05.943 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:05.940+0000 7f3138298740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f312c1aef80 con 0x5577444909f0 2024-09-21T03:53:05.943 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:05.940+0000 7f3138298740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f312c1adb10 con 0x5577444b1390 2024-09-21T03:53:05.943 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:05.940+0000 7f312affd640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5577444a91b0 0x55774488ad90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:53:05.943 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:05.940+0000 7f312affd640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5577444a91b0 0x55774488ad90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:49288/0 (socket says 172.21.15.39:49288) 2024-09-21T03:53:05.943 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:05.940+0000 7f312affd640 1 -- 172.21.15.39:0/2220538207 learned_addr learned my addr 172.21.15.39:0/2220538207 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:53:05.943 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:05.940+0000 7f312b7fe640 1 --2- 172.21.15.39:0/2220538207 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5577444909f0 0x557744888050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:53:05.943 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:05.940+0000 7f312bfff640 1 --2- 172.21.15.39:0/2220538207 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5577444b1390 0x557744876b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:53:05.943 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:05.941+0000 7f312affd640 1 -- 172.21.15.39:0/2220538207 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5577444b1390 msgr2=0x557744876b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:53:05.943 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:05.941+0000 7f312affd640 1 --2- 172.21.15.39:0/2220538207 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5577444b1390 0x557744876b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:05.943 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:05.941+0000 7f312affd640 1 -- 172.21.15.39:0/2220538207 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5577444909f0 msgr2=0x557744888050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:53:05.943 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:05.941+0000 7f312affd640 1 --2- 172.21.15.39:0/2220538207 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5577444909f0 0x557744888050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:05.943 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:05.941+0000 7f312affd640 1 -- 172.21.15.39:0/2220538207 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557744887660 con 0x5577444a91b0 2024-09-21T03:53:05.945 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:05.941+0000 7f312affd640 1 --2- 172.21.15.39:0/2220538207 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5577444a91b0 0x55774488ad90 secure :-1 s=READY pgs=1004 cs=0 l=1 rev1=1 crypto rx=0x7f3120003770 tx=0x7f31200037a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=74a92822c8714307 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:53:05.945 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:05.941+0000 7f31117fa640 1 -- 172.21.15.39:0/2220538207 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f312000c330 con 0x5577444a91b0 2024-09-21T03:53:05.945 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:05.941+0000 7f31117fa640 1 -- 172.21.15.39:0/2220538207 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3120005360 con 0x5577444a91b0 2024-09-21T03:53:05.945 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:05.941+0000 7f31117fa640 1 -- 172.21.15.39:0/2220538207 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31200109e0 con 0x5577444a91b0 2024-09-21T03:53:05.946 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:05.942+0000 7f3138298740 1 -- 172.21.15.39:0/2220538207 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5577444a91b0 msgr2=0x55774488ad90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:53:05.946 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:05.942+0000 7f3138298740 1 --2- 172.21.15.39:0/2220538207 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5577444a91b0 0x55774488ad90 secure :-1 s=READY pgs=1004 cs=0 l=1 rev1=1 crypto rx=0x7f3120003770 tx=0x7f31200037a0 comp rx=0 tx=0).stop 2024-09-21T03:53:05.946 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:05.942+0000 7f3138298740 1 -- 172.21.15.39:0/2220538207 shutdown_connections 2024-09-21T03:53:05.946 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:05.942+0000 7f3138298740 1 --2- 172.21.15.39:0/2220538207 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5577444b1390 0x557744876b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:05.946 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:05.942+0000 7f3138298740 1 --2- 172.21.15.39:0/2220538207 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5577444a91b0 0x55774488ad90 unknown :-1 s=CLOSED pgs=1004 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:05.946 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:05.942+0000 7f3138298740 1 --2- 172.21.15.39:0/2220538207 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5577444909f0 0x557744888050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:05.946 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:05.942+0000 7f3138298740 1 -- 172.21.15.39:0/2220538207 >> 172.21.15.39:0/2220538207 conn(0x5577444c6460 msgr2=0x55774488a680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:53:05.946 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:05.942+0000 7f3138298740 1 -- 172.21.15.39:0/2220538207 shutdown_connections 2024-09-21T03:53:05.946 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:05.942+0000 7f3138298740 1 -- 172.21.15.39:0/2220538207 wait complete. 2024-09-21T03:53:05.946 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:05.942+0000 7f3138298740 1 -- start start 2024-09-21T03:53:05.947 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:05.943+0000 7f3138298740 1 Processor -- start 2024-09-21T03:53:05.947 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:05.943+0000 7f3138298740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5577444909f0 0x55774488d5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:53:05.947 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:05.943+0000 7f3138298740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5577444a91b0 0x55774488db00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:53:05.947 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:05.943+0000 7f3138298740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5577444b1390 0x55774488e040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:53:05.947 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:05.943+0000 7f3138298740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f312c1ae8b0 con 0x5577444b1390 2024-09-21T03:53:05.947 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:05.943+0000 7f3138298740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f312c1ae1e0 con 0x5577444909f0 2024-09-21T03:53:05.947 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:05.943+0000 7f3138298740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f312c1ab420 con 0x5577444a91b0 2024-09-21T03:53:05.947 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:05.943+0000 7f3138298740 10 client.0 _ll_register_callbacks cb 0x7f311c00b820 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:53:05.947 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:05.943+0000 7f312bfff640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5577444b1390 0x55774488e040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:53:05.947 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:05.943+0000 7f312b7fe640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5577444909f0 0x55774488d5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:53:05.947 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:05.943+0000 7f312bfff640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5577444b1390 0x55774488e040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:49294/0 (socket says 172.21.15.39:49294) 2024-09-21T03:53:05.947 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:05.943+0000 7f312affd640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5577444a91b0 0x55774488db00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:53:05.947 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:05.943+0000 7f312bfff640 1 -- 172.21.15.39:0/170281556 learned_addr learned my addr 172.21.15.39:0/170281556 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:53:05.947 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:05.943+0000 7f312bfff640 1 -- 172.21.15.39:0/170281556 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5577444a91b0 msgr2=0x55774488db00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:53:05.947 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:05.943+0000 7f312bfff640 1 --2- 172.21.15.39:0/170281556 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5577444a91b0 0x55774488db00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:05.947 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:05.943+0000 7f312bfff640 1 -- 172.21.15.39:0/170281556 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5577444909f0 msgr2=0x55774488d5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:53:05.948 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:05.943+0000 7f312bfff640 1 --2- 172.21.15.39:0/170281556 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5577444909f0 0x55774488d5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:05.948 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:05.943+0000 7f312bfff640 1 -- 172.21.15.39:0/170281556 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557744904fc0 con 0x5577444b1390 2024-09-21T03:53:05.948 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:05.944+0000 7f312bfff640 1 --2- 172.21.15.39:0/170281556 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5577444b1390 0x55774488e040 secure :-1 s=READY pgs=1005 cs=0 l=1 rev1=1 crypto rx=0x7f31240cb170 tx=0x7f31240ce3d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c3529fa5bdab717 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:53:05.948 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:05.944+0000 7f310affd640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-09-21T03:53:05.950 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:05.944+0000 7f310affd640 1 -- 172.21.15.39:0/170281556 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31240cf2a0 con 0x5577444b1390 2024-09-21T03:53:05.950 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:05.944+0000 7f310affd640 1 -- 172.21.15.39:0/170281556 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8484+0+0 (secure 0 0 0) 0x7f3124004610 con 0x5577444b1390 2024-09-21T03:53:05.950 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:05.944+0000 7f3138298740 1 -- 172.21.15.39:0/170281556 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3120005360 con 0x5577444b1390 2024-09-21T03:53:05.950 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:05.944+0000 7f310affd640 1 -- 172.21.15.39:0/170281556 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f31240d3680 con 0x5577444b1390 2024-09-21T03:53:05.950 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:05.944+0000 7f3138298740 10 client.17295 fetch_fsmap learned FSMap version 29 2024-09-21T03:53:05.950 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:05.944+0000 7f3138298740 1 -- 172.21.15.39:0/170281556 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x557744906270 con 0x5577444b1390 2024-09-21T03:53:05.950 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:05.944+0000 7f310affd640 1 -- 172.21.15.39:0/170281556 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3120005360 con 0x5577444b1390 2024-09-21T03:53:05.950 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:05.944+0000 7f310affd640 1 -- 172.21.15.39:0/170281556 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f31240d3120 con 0x5577444b1390 2024-09-21T03:53:05.950 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:05.944+0000 7f3138298740 10 client.17295 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T03:53:05.950 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:05.945+0000 7f3138298740 10 client.17295 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T03:53:05.950 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:05.945+0000 7f3138298740 20 client.17295 populate_metadata read hostname 'smithi039' 2024-09-21T03:53:05.950 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:05.945+0000 7f3138298740 1 --2- 172.21.15.39:0/170281556 >> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] conn(0x5577444d2070 0x557744908a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:53:05.950 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:05.945+0000 7f3138298740 4 client.17295 mds_command: new command op to 24695 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-21T03:53:05.950 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:05.945+0000 7f3138298740 1 -- 172.21.15.39:0/170281556 --> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5577447ed850 con 0x5577444d2070 2024-09-21T03:53:05.951 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:05.945+0000 7f312b7fe640 1 --2- 172.21.15.39:0/170281556 >> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] conn(0x5577444d2070 0x557744908a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:53:05.951 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:05.945+0000 7f310affd640 10 client.17295 ms_handle_connect on v2:172.21.15.150:6838/2949386907 2024-09-21T03:53:05.951 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:05.945+0000 7f312b7fe640 1 --2- 172.21.15.39:0/170281556 >> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] conn(0x5577444d2070 0x557744908a50 crc :-1 s=READY pgs=120 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d9a0162ed3a3e64e server_cookie=16696f106e4fe4e9 in_seq=0 out_seq=0 2024-09-21T03:53:05.951 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:05.946+0000 7f310affd640 1 -- 172.21.15.39:0/170281556 <== mds.0 v2:172.21.15.150:6838/2949386907 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3782 (crc 0 0 0) 0x5577447ed850 con 0x5577444d2070 2024-09-21T03:53:05.951 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:05.946+0000 7f310affd640 10 client.17295 handle_command_reply: tid=0 2024-09-21T03:53:05.951 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:05.946+0000 7f3138298740 1 client.17295 shutdown 2024-09-21T03:53:05.951 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:05.946+0000 7f3138298740 1 -- 172.21.15.39:0/170281556 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5577444b1390 msgr2=0x55774488e040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:53:05.951 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:05.946+0000 7f3138298740 1 --2- 172.21.15.39:0/170281556 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5577444b1390 0x55774488e040 secure :-1 s=READY pgs=1005 cs=0 l=1 rev1=1 crypto rx=0x7f31240cb170 tx=0x7f31240ce3d0 comp rx=0 tx=0).stop 2024-09-21T03:53:05.951 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:05.946+0000 7f3138298740 1 -- 172.21.15.39:0/170281556 shutdown_connections 2024-09-21T03:53:05.951 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:05.946+0000 7f3138298740 1 --2- 172.21.15.39:0/170281556 >> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] conn(0x5577444d2070 0x557744908a50 crc :-1 s=READY pgs=120 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:05.951 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:05.946+0000 7f3138298740 1 --2- 172.21.15.39:0/170281556 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5577444b1390 0x55774488e040 unknown :-1 s=CLOSED pgs=1005 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:05.951 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:05.946+0000 7f3138298740 1 --2- 172.21.15.39:0/170281556 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5577444a91b0 0x55774488db00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:05.951 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:05.946+0000 7f3138298740 1 --2- 172.21.15.39:0/170281556 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5577444909f0 0x55774488d5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:05.951 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:05.946+0000 7f3138298740 1 -- 172.21.15.39:0/170281556 >> 172.21.15.39:0/170281556 conn(0x5577444c6460 msgr2=0x55774487a680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:53:05.951 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:05.946+0000 7f310affd640 0 client.17295 ms_handle_reset on v2:172.21.15.150:6838/2949386907 2024-09-21T03:53:05.951 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:05.946+0000 7f3138298740 1 -- 172.21.15.39:0/170281556 shutdown_connections 2024-09-21T03:53:05.952 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:05.946+0000 7f3138298740 1 -- 172.21.15.39:0/170281556 wait complete. 2024-09-21T03:53:05.952 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:05.946+0000 7f3138298740 20 client.17295 trim_cache size 0 max 16384 2024-09-21T03:53:05.952 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-09-21T03:53:05.952 INFO:teuthology.orchestra.run.smithi039.stdout: "status": "no active scrubs running", 2024-09-21T03:53:05.952 INFO:teuthology.orchestra.run.smithi039.stdout: "scrubs": {}, 2024-09-21T03:53:05.952 INFO:teuthology.orchestra.run.smithi039.stdout: "scrub_stats": { 2024-09-21T03:53:05.952 INFO:teuthology.orchestra.run.smithi039.stdout: "4bda6ee3-7237-4ee4-96ce-150ad50f26dc": { 2024-09-21T03:53:05.952 INFO:teuthology.orchestra.run.smithi039.stdout: "start_time": "2024-09-21T03:52:18.622141+0000", 2024-09-21T03:53:05.952 INFO:teuthology.orchestra.run.smithi039.stdout: "path": "/", 2024-09-21T03:53:05.952 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_started": 0, 2024-09-21T03:53:05.952 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_passed": 0, 2024-09-21T03:53:05.952 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_failed": 0, 2024-09-21T03:53:05.953 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_skipped": 0 2024-09-21T03:53:05.953 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:53:05.953 INFO:teuthology.orchestra.run.smithi039.stdout: "ae1fc4cf-5458-47d0-afcb-282b136dba15": { 2024-09-21T03:53:05.953 INFO:teuthology.orchestra.run.smithi039.stdout: "start_time": "2024-09-21T03:51:18.560378+0000", 2024-09-21T03:53:05.953 INFO:teuthology.orchestra.run.smithi039.stdout: "path": "/", 2024-09-21T03:53:05.953 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_started": 0, 2024-09-21T03:53:05.953 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_passed": 0, 2024-09-21T03:53:05.953 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_failed": 0, 2024-09-21T03:53:05.953 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_skipped": 0 2024-09-21T03:53:05.953 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:53:05.953 INFO:teuthology.orchestra.run.smithi039.stdout: "160750d8-401f-4810-866d-e63e7e4bc3e0": { 2024-09-21T03:53:05.953 INFO:teuthology.orchestra.run.smithi039.stdout: "start_time": "2024-09-21T03:51:04.208673+0000", 2024-09-21T03:53:05.953 INFO:teuthology.orchestra.run.smithi039.stdout: "path": "/", 2024-09-21T03:53:05.953 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_started": 0, 2024-09-21T03:53:05.953 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_passed": 0, 2024-09-21T03:53:05.954 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_failed": 0, 2024-09-21T03:53:05.954 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_skipped": 0 2024-09-21T03:53:05.954 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:53:05.954 INFO:teuthology.orchestra.run.smithi039.stdout: "74ab917e-ecea-4778-a1c3-c184214fb24c": { 2024-09-21T03:53:05.954 INFO:teuthology.orchestra.run.smithi039.stdout: "start_time": "2024-09-21T03:50:17.726630+0000", 2024-09-21T03:53:05.954 INFO:teuthology.orchestra.run.smithi039.stdout: "path": "/", 2024-09-21T03:53:05.954 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_started": 0, 2024-09-21T03:53:05.954 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_passed": 0, 2024-09-21T03:53:05.954 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_failed": 0, 2024-09-21T03:53:05.954 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_skipped": 0 2024-09-21T03:53:05.954 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:53:05.954 INFO:teuthology.orchestra.run.smithi039.stdout: "6e760f1f-9048-4f92-bd66-d708fbef75b3": { 2024-09-21T03:53:05.954 INFO:teuthology.orchestra.run.smithi039.stdout: "start_time": "2024-09-21T03:50:03.711918+0000", 2024-09-21T03:53:05.954 INFO:teuthology.orchestra.run.smithi039.stdout: "path": "/", 2024-09-21T03:53:05.954 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_started": 0, 2024-09-21T03:53:05.954 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_passed": 0, 2024-09-21T03:53:05.955 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_failed": 0, 2024-09-21T03:53:05.955 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_skipped": 0 2024-09-21T03:53:05.955 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:53:05.955 INFO:teuthology.orchestra.run.smithi039.stdout: "e28ad743-2b22-45b3-8799-a638c2883623": { 2024-09-21T03:53:05.955 INFO:teuthology.orchestra.run.smithi039.stdout: "start_time": "2024-09-21T03:48:31.448821+0000", 2024-09-21T03:53:05.955 INFO:teuthology.orchestra.run.smithi039.stdout: "path": "/", 2024-09-21T03:53:05.955 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_started": 0, 2024-09-21T03:53:05.955 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_passed": 0, 2024-09-21T03:53:05.955 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_failed": 0, 2024-09-21T03:53:05.955 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_skipped": 0 2024-09-21T03:53:05.955 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:53:05.955 INFO:teuthology.orchestra.run.smithi039.stdout: "ede73e6e-3fd0-4a7f-b5bb-f68946f57eea": { 2024-09-21T03:53:05.955 INFO:teuthology.orchestra.run.smithi039.stdout: "start_time": "2024-09-21T03:51:32.235096+0000", 2024-09-21T03:53:05.955 INFO:teuthology.orchestra.run.smithi039.stdout: "path": "/", 2024-09-21T03:53:05.955 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_started": 0, 2024-09-21T03:53:05.955 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_passed": 0, 2024-09-21T03:53:05.956 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_failed": 0, 2024-09-21T03:53:05.956 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_skipped": 0 2024-09-21T03:53:05.956 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:53:05.956 INFO:teuthology.orchestra.run.smithi039.stdout: "29411307-eb8d-4339-8d2b-47ebd32c0080": { 2024-09-21T03:53:05.956 INFO:teuthology.orchestra.run.smithi039.stdout: "start_time": "2024-09-21T03:47:38.092920+0000", 2024-09-21T03:53:05.956 INFO:teuthology.orchestra.run.smithi039.stdout: "path": "/", 2024-09-21T03:53:05.956 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_started": 0, 2024-09-21T03:53:05.956 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_passed": 0, 2024-09-21T03:53:05.956 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_failed": 0, 2024-09-21T03:53:05.956 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_skipped": 0 2024-09-21T03:53:05.956 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:53:05.956 INFO:teuthology.orchestra.run.smithi039.stdout: "c7c700f3-2cd8-41ac-8381-b57dd1b37b02": { 2024-09-21T03:53:05.956 INFO:teuthology.orchestra.run.smithi039.stdout: "start_time": "2024-09-21T03:52:32.368858+0000", 2024-09-21T03:53:05.956 INFO:teuthology.orchestra.run.smithi039.stdout: "path": "/", 2024-09-21T03:53:05.957 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_started": 0, 2024-09-21T03:53:05.957 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_passed": 0, 2024-09-21T03:53:05.957 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_failed": 0, 2024-09-21T03:53:05.957 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_skipped": 0 2024-09-21T03:53:05.957 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:53:05.957 INFO:teuthology.orchestra.run.smithi039.stdout: "9860c2f4-b7b1-47aa-865a-9a1c4434a5f6": { 2024-09-21T03:53:05.957 INFO:teuthology.orchestra.run.smithi039.stdout: "start_time": "2024-09-21T03:49:17.718867+0000", 2024-09-21T03:53:05.957 INFO:teuthology.orchestra.run.smithi039.stdout: "path": "/", 2024-09-21T03:53:05.957 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_started": 0, 2024-09-21T03:53:05.957 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_passed": 0, 2024-09-21T03:53:05.957 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_failed": 0, 2024-09-21T03:53:05.957 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_skipped": 0 2024-09-21T03:53:05.957 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:53:05.957 INFO:teuthology.orchestra.run.smithi039.stdout: "c7e2b8e4-057d-435f-b2bd-b633de98035f": { 2024-09-21T03:53:05.957 INFO:teuthology.orchestra.run.smithi039.stdout: "start_time": "2024-09-21T03:47:19.400305+0000", 2024-09-21T03:53:05.958 INFO:teuthology.orchestra.run.smithi039.stdout: "path": "/", 2024-09-21T03:53:05.958 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_started": 0, 2024-09-21T03:53:05.958 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_passed": 0, 2024-09-21T03:53:05.958 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_failed": 0, 2024-09-21T03:53:05.958 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_skipped": 0 2024-09-21T03:53:05.958 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:53:05.958 INFO:teuthology.orchestra.run.smithi039.stdout: "10728de8-c49d-4edc-be2b-639249255e94": { 2024-09-21T03:53:05.958 INFO:teuthology.orchestra.run.smithi039.stdout: "start_time": "2024-09-21T03:47:03.965623+0000", 2024-09-21T03:53:05.958 INFO:teuthology.orchestra.run.smithi039.stdout: "path": "/", 2024-09-21T03:53:05.958 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_started": 0, 2024-09-21T03:53:05.958 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_passed": 0, 2024-09-21T03:53:05.958 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_failed": 0, 2024-09-21T03:53:05.958 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_skipped": 0 2024-09-21T03:53:05.958 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:53:05.958 INFO:teuthology.orchestra.run.smithi039.stdout: "02509e4f-3911-4983-b16f-df8ded0b9998": { 2024-09-21T03:53:05.958 INFO:teuthology.orchestra.run.smithi039.stdout: "start_time": "2024-09-21T03:46:17.889576+0000", 2024-09-21T03:53:05.959 INFO:teuthology.orchestra.run.smithi039.stdout: "path": "/", 2024-09-21T03:53:05.959 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_started": 0, 2024-09-21T03:53:05.959 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_passed": 0, 2024-09-21T03:53:05.959 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_failed": 0, 2024-09-21T03:53:05.959 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_skipped": 0 2024-09-21T03:53:05.959 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-21T03:53:05.959 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-21T03:53:05.959 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-09-21T03:53:05.960 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:05.947+0000 7f3131d24640 1 -- 172.21.15.39:0/916762225 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f31140b66b0 msgr2=0x7f31140b8bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:53:05.960 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:05.947+0000 7f3131d24640 1 --2- 172.21.15.39:0/916762225 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f31140b66b0 0x7f31140b8bc0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f311c005fd0 tx=0x7f311c00b5a0 comp rx=0 tx=0).stop 2024-09-21T03:53:05.960 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:05.947+0000 7f3131d24640 1 -- 172.21.15.39:0/916762225 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f312c109df0 msgr2=0x7f312c19fcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:53:05.960 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:05.947+0000 7f3131d24640 1 --2- 172.21.15.39:0/916762225 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f312c109df0 0x7f312c19fcc0 secure :-1 s=READY pgs=1001 cs=0 l=1 rev1=1 crypto rx=0x7f3124005cc0 tx=0x7f312402f380 comp rx=0 tx=0).stop 2024-09-21T03:53:05.960 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:05.947+0000 7f3131d24640 1 -- 172.21.15.39:0/916762225 shutdown_connections 2024-09-21T03:53:05.960 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:05.947+0000 7f3131d24640 1 --2- 172.21.15.39:0/916762225 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f31140b66b0 0x7f31140b8bc0 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:05.960 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:05.947+0000 7f3131d24640 1 --2- 172.21.15.39:0/916762225 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f312c109df0 0x7f312c19fcc0 unknown :-1 s=CLOSED pgs=1001 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:05.960 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:05.947+0000 7f3131d24640 1 --2- 172.21.15.39:0/916762225 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f312c1094a0 0x7f312c19aa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:05.960 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:05.947+0000 7f3131d24640 1 --2- 172.21.15.39:0/916762225 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f312c103b60 0x7f312c19a480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:05.960 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:05.947+0000 7f3131d24640 1 -- 172.21.15.39:0/916762225 >> 172.21.15.39:0/916762225 conn(0x7f312c07b850 msgr2=0x7f312c105470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:53:05.961 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:05.947+0000 7f3131d24640 1 -- 172.21.15.39:0/916762225 shutdown_connections 2024-09-21T03:53:05.961 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:05.947+0000 7f3131d24640 1 -- 172.21.15.39:0/916762225 wait complete. 2024-09-21T03:53:06.758 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-21T03:53:06.759 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 51116d5c-77ca-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T03:53:07.025 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/51116d5c-77ca-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:53:07.433 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:53:07 smithi039 ceph-mon[31211]: pgmap v537: 97 pgs: 97 active+clean; 10 MiB data, 520 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 44 KiB/s wr, 27 op/s 2024-09-21T03:53:07.533 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:07.525+0000 7f93ef2f0640 1 -- 172.21.15.39:0/3813504317 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f93e8110190 msgr2=0x7f93e81125b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:53:07.534 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:07.525+0000 7f93ef2f0640 1 --2- 172.21.15.39:0/3813504317 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f93e8110190 0x7f93e81125b0 secure :-1 s=READY pgs=1006 cs=0 l=1 rev1=1 crypto rx=0x7f93e0009ca0 tx=0x7f93e002a950 comp rx=0 tx=0).stop 2024-09-21T03:53:07.534 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:07.525+0000 7f93ef2f0640 1 -- 172.21.15.39:0/3813504317 shutdown_connections 2024-09-21T03:53:07.534 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:07.525+0000 7f93ef2f0640 1 --2- 172.21.15.39:0/3813504317 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f93e8110190 0x7f93e81125b0 unknown :-1 s=CLOSED pgs=1006 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:07.534 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:07.525+0000 7f93ef2f0640 1 --2- 172.21.15.39:0/3813504317 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f93e8102f30 0x7f93e810fc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:07.534 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:07.525+0000 7f93ef2f0640 1 --2- 172.21.15.39:0/3813504317 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f93e81025e0 0x7f93e81029f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:07.534 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:07.525+0000 7f93ef2f0640 1 -- 172.21.15.39:0/3813504317 >> 172.21.15.39:0/3813504317 conn(0x7f93e8079710 msgr2=0x7f93e8079b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:53:07.534 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:07.525+0000 7f93ef2f0640 1 -- 172.21.15.39:0/3813504317 shutdown_connections 2024-09-21T03:53:07.534 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:07.525+0000 7f93ef2f0640 1 -- 172.21.15.39:0/3813504317 wait complete. 2024-09-21T03:53:07.534 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:07.525+0000 7f93ef2f0640 1 Processor -- start 2024-09-21T03:53:07.534 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:07.526+0000 7f93ef2f0640 1 -- start start 2024-09-21T03:53:07.534 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:07.526+0000 7f93ef2f0640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f93e81025e0 0x7f93e819c5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:53:07.534 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:07.526+0000 7f93ef2f0640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f93e8102f30 0x7f93e819cb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:53:07.534 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:07.526+0000 7f93ef2f0640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f93e8110190 0x7f93e81a1e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:53:07.535 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:07.526+0000 7f93ef2f0640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f93e8115c80 con 0x7f93e8102f30 2024-09-21T03:53:07.535 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:07.526+0000 7f93ef2f0640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f93e8115b00 con 0x7f93e81025e0 2024-09-21T03:53:07.535 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:07.526+0000 7f93ef2f0640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f93e8115e00 con 0x7f93e8110190 2024-09-21T03:53:07.535 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:07.526+0000 7f93ec88b640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f93e8102f30 0x7f93e819cb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:53:07.535 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:07.526+0000 7f93ec88b640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f93e8102f30 0x7f93e819cb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:48558/0 (socket says 172.21.15.39:48558) 2024-09-21T03:53:07.535 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:07.526+0000 7f93ec88b640 1 -- 172.21.15.39:0/4020345291 learned_addr learned my addr 172.21.15.39:0/4020345291 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:53:07.535 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:07.526+0000 7f93ed08c640 1 --2- 172.21.15.39:0/4020345291 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f93e81025e0 0x7f93e819c5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:53:07.535 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:07.526+0000 7f93ed88d640 1 --2- 172.21.15.39:0/4020345291 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f93e8110190 0x7f93e81a1e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:53:07.535 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:07.526+0000 7f93ec88b640 1 -- 172.21.15.39:0/4020345291 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f93e8110190 msgr2=0x7f93e81a1e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:53:07.535 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:07.526+0000 7f93ec88b640 1 --2- 172.21.15.39:0/4020345291 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f93e8110190 0x7f93e81a1e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:07.535 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:07.526+0000 7f93ec88b640 1 -- 172.21.15.39:0/4020345291 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f93e81025e0 msgr2=0x7f93e819c5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:53:07.535 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:07.526+0000 7f93ec88b640 1 --2- 172.21.15.39:0/4020345291 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f93e81025e0 0x7f93e819c5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:07.535 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:07.526+0000 7f93ec88b640 1 -- 172.21.15.39:0/4020345291 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f93e81a2540 con 0x7f93e8102f30 2024-09-21T03:53:07.535 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:07.527+0000 7f93ec88b640 1 --2- 172.21.15.39:0/4020345291 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f93e8102f30 0x7f93e819cb70 secure :-1 s=READY pgs=1007 cs=0 l=1 rev1=1 crypto rx=0x7f93d800d950 tx=0x7f93d800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:53:07.536 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:07.527+0000 7f93de7fc640 1 -- 172.21.15.39:0/4020345291 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93d8017070 con 0x7f93e8102f30 2024-09-21T03:53:07.536 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:07.527+0000 7f93de7fc640 1 -- 172.21.15.39:0/4020345291 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f93d8009c30 con 0x7f93e8102f30 2024-09-21T03:53:07.536 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:07.527+0000 7f93ef2f0640 1 -- 172.21.15.39:0/4020345291 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f93e81a2830 con 0x7f93e8102f30 2024-09-21T03:53:07.536 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:07.527+0000 7f93de7fc640 1 -- 172.21.15.39:0/4020345291 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93d800f620 con 0x7f93e8102f30 2024-09-21T03:53:07.536 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:07.527+0000 7f93ef2f0640 1 -- 172.21.15.39:0/4020345291 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f93e81a8f90 con 0x7f93e8102f30 2024-09-21T03:53:07.536 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:07.528+0000 7f93ef2f0640 1 -- 172.21.15.39:0/4020345291 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f93e806c280 con 0x7f93e8102f30 2024-09-21T03:53:07.536 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:07.530+0000 7f93de7fc640 1 -- 172.21.15.39:0/4020345291 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 27) ==== 154643+0+0 (secure 0 0 0) 0x7f93d8005030 con 0x7f93e8102f30 2024-09-21T03:53:07.536 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:07.531+0000 7f93de7fc640 1 --2- 172.21.15.39:0/4020345291 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f93d40b6700 0x7f93d40b8c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:53:07.536 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:07.531+0000 7f93ed08c640 1 --2- 172.21.15.39:0/4020345291 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f93d40b6700 0x7f93d40b8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:53:07.536 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:07.531+0000 7f93de7fc640 1 -- 172.21.15.39:0/4020345291 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8484+0+0 (secure 0 0 0) 0x7f93d80e7280 con 0x7f93e8102f30 2024-09-21T03:53:07.536 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:07.532+0000 7f93ed08c640 1 --2- 172.21.15.39:0/4020345291 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f93d40b6700 0x7f93d40b8c10 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f93e4009bd0 tx=0x7f93e4005f90 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:53:07.537 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:07.534+0000 7f93de7fc640 1 -- 172.21.15.39:0/4020345291 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f93d80ad870 con 0x7f93e8102f30 2024-09-21T03:53:07.718 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:53:07 smithi161 ceph-mon[34533]: pgmap v537: 97 pgs: 97 active+clean; 10 MiB data, 520 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 44 KiB/s wr, 27 op/s 2024-09-21T03:53:07.725 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:53:07 smithi150 ceph-mon[34289]: pgmap v537: 97 pgs: 97 active+clean; 10 MiB data, 520 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 44 KiB/s wr, 27 op/s 2024-09-21T03:53:07.763 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:07.761+0000 7f93ef2f0640 1 -- 172.21.15.39:0/4020345291 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f93e819d700 con 0x7f93e8102f30 2024-09-21T03:53:07.764 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:07.762+0000 7f93de7fc640 1 -- 172.21.15.39:0/4020345291 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11978 (secure 0 0 0) 0x7f93d8009de0 con 0x7f93e8102f30 2024-09-21T03:53:07.764 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T03:53:07.764 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":29,"btime":"2024-09-21T03:46:08:982881+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14790,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6835/508995295","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":508995295},{"type":"v1","addr":"172.21.15.39:6835","nonce":508995295}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14802,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6837/3915912607","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":3915912607},{"type":"v1","addr":"172.21.15.39:6837","nonce":3915912607}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":29,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T03:44:15.009872+0000","modified":"2024-09-21T03:46:08.430113+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24695,"mds_1":24667,"mds_2":24659,"mds_3":24646,"mds_4":14868},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14856":{"gid":14856,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.39:6839/185509192","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6838","nonce":185509192},{"type":"v1","addr":"172.21.15.39:6839","nonce":185509192}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14868":{"gid":14868,"name":"a","rank":4,"incarnation":25,"state":"up:active","state_seq":7,"addr":"172.21.15.39:6841/4247190456","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6840","nonce":4247190456},{"type":"v1","addr":"172.21.15.39:6841","nonce":4247190456}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24599":{"gid":24599,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.150:6833/3114643217","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":3114643217},{"type":"v1","addr":"172.21.15.150:6833","nonce":3114643217}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24601":{"gid":24601,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.161:6833/2963990654","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":2963990654},{"type":"v1","addr":"172.21.15.161:6833","nonce":2963990654}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24632":{"gid":24632,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.150:6835/724594008","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6834","nonce":724594008},{"type":"v1","addr":"172.21.15.150:6835","nonce":724594008}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24640":{"gid":24640,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.161:6835/3747882110","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":3747882110},{"type":"v1","addr":"172.21.15.161:6835","nonce":3747882110}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24646":{"gid":24646,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":10,"addr":"172.21.15.161:6837/3451544597","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6836","nonce":3451544597},{"type":"v1","addr":"172.21.15.161:6837","nonce":3451544597}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24659":{"gid":24659,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.150:6837/4017043144","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6836","nonce":4017043144},{"type":"v1","addr":"172.21.15.150:6837","nonce":4017043144}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24667":{"gid":24667,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.161:6839/2190107270","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6838","nonce":2190107270},{"type":"v1","addr":"172.21.15.161:6839","nonce":2190107270}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24695":{"gid":24695,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.150:6839/2949386907","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6838","nonce":2949386907},{"type":"v1","addr":"172.21.15.150:6839","nonce":2949386907}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24695,"qdb_cluster":[24695,24667,24659,24646,14868]},"id":1}]} 2024-09-21T03:53:07.765 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 29 2024-09-21T03:53:07.766 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:07.764+0000 7f93ef2f0640 1 -- 172.21.15.39:0/4020345291 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f93d40b6700 msgr2=0x7f93d40b8c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:53:07.766 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:07.764+0000 7f93ef2f0640 1 --2- 172.21.15.39:0/4020345291 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f93d40b6700 0x7f93d40b8c10 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f93e4009bd0 tx=0x7f93e4005f90 comp rx=0 tx=0).stop 2024-09-21T03:53:07.767 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:07.764+0000 7f93ef2f0640 1 -- 172.21.15.39:0/4020345291 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f93e8102f30 msgr2=0x7f93e819cb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:53:07.767 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:07.764+0000 7f93ef2f0640 1 --2- 172.21.15.39:0/4020345291 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f93e8102f30 0x7f93e819cb70 secure :-1 s=READY pgs=1007 cs=0 l=1 rev1=1 crypto rx=0x7f93d800d950 tx=0x7f93d800de10 comp rx=0 tx=0).stop 2024-09-21T03:53:07.767 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:07.764+0000 7f93ef2f0640 1 -- 172.21.15.39:0/4020345291 shutdown_connections 2024-09-21T03:53:07.767 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:07.764+0000 7f93ef2f0640 1 --2- 172.21.15.39:0/4020345291 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f93d40b6700 0x7f93d40b8c10 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:07.767 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:07.764+0000 7f93ef2f0640 1 --2- 172.21.15.39:0/4020345291 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f93e8110190 0x7f93e81a1e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:07.767 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:07.764+0000 7f93ef2f0640 1 --2- 172.21.15.39:0/4020345291 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f93e8102f30 0x7f93e819cb70 unknown :-1 s=CLOSED pgs=1007 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:07.767 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:07.764+0000 7f93ef2f0640 1 --2- 172.21.15.39:0/4020345291 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f93e81025e0 0x7f93e819c5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:07.767 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:07.764+0000 7f93ef2f0640 1 -- 172.21.15.39:0/4020345291 >> 172.21.15.39:0/4020345291 conn(0x7f93e8079710 msgr2=0x7f93e8109ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:53:07.767 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:07.765+0000 7f93ef2f0640 1 -- 172.21.15.39:0/4020345291 shutdown_connections 2024-09-21T03:53:07.767 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:07.765+0000 7f93ef2f0640 1 -- 172.21.15.39:0/4020345291 wait complete. 2024-09-21T03:53:08.421 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:53:08 smithi039 ceph-mon[31211]: from='client.? 172.21.15.39:0/4020345291' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:53:08.451 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 51116d5c-77ca-11ef-baf5-efdc52797490 -- ceph tell mds.1:4 damage ls 2024-09-21T03:53:08.709 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/51116d5c-77ca-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:53:08.718 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:53:08 smithi161 ceph-mon[34533]: from='client.? 172.21.15.39:0/4020345291' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:53:08.725 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:53:08 smithi150 ceph-mon[34289]: from='client.? 172.21.15.39:0/4020345291' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:53:09.252 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:09.220+0000 7f6c455be640 1 -- 172.21.15.39:0/981455247 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6c40108cf0 msgr2=0x7f6c4010f520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:53:09.252 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:09.220+0000 7f6c455be640 1 --2- 172.21.15.39:0/981455247 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6c40108cf0 0x7f6c4010f520 secure :-1 s=READY pgs=1008 cs=0 l=1 rev1=1 crypto rx=0x7f6c30009ca0 tx=0x7f6c3002a8c0 comp rx=0 tx=0).stop 2024-09-21T03:53:09.252 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:09.220+0000 7f6c455be640 1 -- 172.21.15.39:0/981455247 shutdown_connections 2024-09-21T03:53:09.252 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:09.220+0000 7f6c455be640 1 --2- 172.21.15.39:0/981455247 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6c40108cf0 0x7f6c4010f520 unknown :-1 s=CLOSED pgs=1008 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:09.252 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:09.220+0000 7f6c455be640 1 --2- 172.21.15.39:0/981455247 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6c401083a0 0x7f6c401087b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:09.252 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:09.220+0000 7f6c455be640 1 --2- 172.21.15.39:0/981455247 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6c40102a60 0x7f6c40102e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:09.252 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:09.220+0000 7f6c455be640 1 -- 172.21.15.39:0/981455247 >> 172.21.15.39:0/981455247 conn(0x7f6c40079730 msgr2=0x7f6c40079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:53:09.252 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:09.220+0000 7f6c455be640 1 -- 172.21.15.39:0/981455247 shutdown_connections 2024-09-21T03:53:09.252 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:09.220+0000 7f6c455be640 1 -- 172.21.15.39:0/981455247 wait complete. 2024-09-21T03:53:09.253 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:09.221+0000 7f6c455be640 1 Processor -- start 2024-09-21T03:53:09.253 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:09.221+0000 7f6c455be640 1 -- start start 2024-09-21T03:53:09.253 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:09.221+0000 7f6c455be640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6c40102a60 0x7f6c401993b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:53:09.253 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:09.221+0000 7f6c455be640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6c401083a0 0x7f6c40199950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:53:09.253 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:09.221+0000 7f6c455be640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6c40108cf0 0x7f6c4019ebf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:53:09.253 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:09.221+0000 7f6c455be640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f6c40112960 con 0x7f6c401083a0 2024-09-21T03:53:09.253 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:09.221+0000 7f6c455be640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f6c401127e0 con 0x7f6c40108cf0 2024-09-21T03:53:09.253 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:09.221+0000 7f6c455be640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f6c40112ae0 con 0x7f6c40102a60 2024-09-21T03:53:09.253 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:09.221+0000 7f6c3e7fc640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6c401083a0 0x7f6c40199950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:53:09.253 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:09.222+0000 7f6c3e7fc640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6c401083a0 0x7f6c40199950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:48576/0 (socket says 172.21.15.39:48576) 2024-09-21T03:53:09.253 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:09.222+0000 7f6c3e7fc640 1 -- 172.21.15.39:0/3426973713 learned_addr learned my addr 172.21.15.39:0/3426973713 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:53:09.253 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:09.222+0000 7f6c3f7fe640 1 --2- 172.21.15.39:0/3426973713 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6c40108cf0 0x7f6c4019ebf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:53:09.253 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:09.222+0000 7f6c3effd640 1 --2- 172.21.15.39:0/3426973713 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6c40102a60 0x7f6c401993b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:53:09.253 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:09.222+0000 7f6c3e7fc640 1 -- 172.21.15.39:0/3426973713 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6c40102a60 msgr2=0x7f6c401993b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:53:09.254 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:09.222+0000 7f6c3e7fc640 1 --2- 172.21.15.39:0/3426973713 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6c40102a60 0x7f6c401993b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:09.254 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:09.222+0000 7f6c3e7fc640 1 -- 172.21.15.39:0/3426973713 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6c40108cf0 msgr2=0x7f6c4019ebf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:53:09.254 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:09.222+0000 7f6c3e7fc640 1 --2- 172.21.15.39:0/3426973713 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6c40108cf0 0x7f6c4019ebf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:09.254 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:09.222+0000 7f6c3e7fc640 1 -- 172.21.15.39:0/3426973713 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6c4019f320 con 0x7f6c401083a0 2024-09-21T03:53:09.254 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:09.222+0000 7f6c3e7fc640 1 --2- 172.21.15.39:0/3426973713 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6c401083a0 0x7f6c40199950 secure :-1 s=READY pgs=1009 cs=0 l=1 rev1=1 crypto rx=0x7f6c2800eb00 tx=0x7f6c28009ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:53:09.254 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:09.222+0000 7f6c37fff640 1 -- 172.21.15.39:0/3426973713 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c2800bc30 con 0x7f6c401083a0 2024-09-21T03:53:09.254 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:09.222+0000 7f6c37fff640 1 -- 172.21.15.39:0/3426973713 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6c2800bdd0 con 0x7f6c401083a0 2024-09-21T03:53:09.254 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:09.222+0000 7f6c455be640 1 -- 172.21.15.39:0/3426973713 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6c4019f610 con 0x7f6c401083a0 2024-09-21T03:53:09.254 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:09.222+0000 7f6c37fff640 1 -- 172.21.15.39:0/3426973713 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c28017410 con 0x7f6c401083a0 2024-09-21T03:53:09.254 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:09.222+0000 7f6c455be640 1 -- 172.21.15.39:0/3426973713 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6c4006ae60 con 0x7f6c401083a0 2024-09-21T03:53:09.254 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:09.225+0000 7f6c4bb32740 1 Processor -- start 2024-09-21T03:53:09.254 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:09.225+0000 7f6c4bb32740 1 -- start start 2024-09-21T03:53:09.254 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:09.225+0000 7f6c37fff640 1 -- 172.21.15.39:0/3426973713 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 27) ==== 154643+0+0 (secure 0 0 0) 0x7f6c28004f20 con 0x7f6c401083a0 2024-09-21T03:53:09.254 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:09.226+0000 7f6c4bb32740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56343b693070 0x56343ba3a2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:53:09.254 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:09.226+0000 7f6c4bb32740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56343b66a1b0 0x56343ba3a800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:53:09.255 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:09.226+0000 7f6c4bb32740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56343b672390 0x56343ba3ad40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:53:09.255 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:09.226+0000 7f6c4bb32740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x56343b9b0ea0 con 0x56343b66a1b0 2024-09-21T03:53:09.255 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:09.226+0000 7f6c4bb32740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x56343b9a02d0 con 0x56343b672390 2024-09-21T03:53:09.255 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:09.226+0000 7f6c4bb32740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x56343b9a40e0 con 0x56343b693070 2024-09-21T03:53:09.255 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:09.226+0000 7f6c3effd640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56343b66a1b0 0x56343ba3a800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:53:09.255 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:09.226+0000 7f6c3effd640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56343b66a1b0 0x56343ba3a800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:48578/0 (socket says 172.21.15.39:48578) 2024-09-21T03:53:09.255 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:09.226+0000 7f6c3effd640 1 -- 172.21.15.39:0/222855517 learned_addr learned my addr 172.21.15.39:0/222855517 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:53:09.255 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:09.226+0000 7f6c3e7fc640 1 --2- 172.21.15.39:0/222855517 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56343b672390 0x56343ba3ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:53:09.255 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:09.226+0000 7f6c3f7fe640 1 --2- 172.21.15.39:0/222855517 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56343b693070 0x56343ba3a2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:53:09.255 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:09.226+0000 7f6c3effd640 1 -- 172.21.15.39:0/222855517 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56343b693070 msgr2=0x56343ba3a2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:53:09.255 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:09.226+0000 7f6c3effd640 1 --2- 172.21.15.39:0/222855517 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56343b693070 0x56343ba3a2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:09.255 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:09.226+0000 7f6c3effd640 1 -- 172.21.15.39:0/222855517 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56343b672390 msgr2=0x56343ba3ad40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:53:09.255 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:09.226+0000 7f6c3effd640 1 --2- 172.21.15.39:0/222855517 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56343b672390 0x56343ba3ad40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:09.256 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:09.226+0000 7f6c3effd640 1 -- 172.21.15.39:0/222855517 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56343b6fe650 con 0x56343b66a1b0 2024-09-21T03:53:09.256 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:09.226+0000 7f6c37fff640 1 --2- 172.21.15.39:0/3426973713 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f6c2c0b6680 0x7f6c2c0b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:53:09.256 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:09.227+0000 7f6c3f7fe640 1 --2- 172.21.15.39:0/3426973713 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f6c2c0b6680 0x7f6c2c0b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:53:09.256 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:09.227+0000 7f6c3effd640 1 --2- 172.21.15.39:0/222855517 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56343b66a1b0 0x56343ba3a800 secure :-1 s=READY pgs=1010 cs=0 l=1 rev1=1 crypto rx=0x7f6c38009cf0 tx=0x7f6c38011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eeed07445b3fee84 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:53:09.256 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:09.227+0000 7f6c37fff640 1 -- 172.21.15.39:0/3426973713 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8484+0+0 (secure 0 0 0) 0x7f6c280e8f70 con 0x7f6c401083a0 2024-09-21T03:53:09.256 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:09.227+0000 7f6c357fa640 1 -- 172.21.15.39:0/222855517 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c38004230 con 0x56343b66a1b0 2024-09-21T03:53:09.256 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:09.227+0000 7f6c357fa640 1 -- 172.21.15.39:0/222855517 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6c380043d0 con 0x56343b66a1b0 2024-09-21T03:53:09.256 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:09.227+0000 7f6c357fa640 1 -- 172.21.15.39:0/222855517 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c38019820 con 0x56343b66a1b0 2024-09-21T03:53:09.256 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:09.228+0000 7f6c4bb32740 1 -- 172.21.15.39:0/222855517 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56343b66a1b0 msgr2=0x56343ba3a800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:53:09.256 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:09.228+0000 7f6c3f7fe640 1 --2- 172.21.15.39:0/3426973713 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f6c2c0b6680 0x7f6c2c0b8b90 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f6c3002f500 tx=0x7f6c3002f310 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:53:09.256 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:09.228+0000 7f6c4bb32740 1 --2- 172.21.15.39:0/222855517 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56343b66a1b0 0x56343ba3a800 secure :-1 s=READY pgs=1010 cs=0 l=1 rev1=1 crypto rx=0x7f6c38009cf0 tx=0x7f6c38011810 comp rx=0 tx=0).stop 2024-09-21T03:53:09.256 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:09.228+0000 7f6c4bb32740 1 -- 172.21.15.39:0/222855517 shutdown_connections 2024-09-21T03:53:09.256 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:09.228+0000 7f6c4bb32740 1 --2- 172.21.15.39:0/222855517 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56343b672390 0x56343ba3ad40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:09.256 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:09.228+0000 7f6c4bb32740 1 --2- 172.21.15.39:0/222855517 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56343b66a1b0 0x56343ba3a800 unknown :-1 s=CLOSED pgs=1010 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:09.256 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:09.228+0000 7f6c4bb32740 1 --2- 172.21.15.39:0/222855517 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56343b693070 0x56343ba3a2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:09.257 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:09.228+0000 7f6c4bb32740 1 -- 172.21.15.39:0/222855517 >> 172.21.15.39:0/222855517 conn(0x56343b687460 msgr2=0x56343ba381b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:53:09.257 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:09.228+0000 7f6c4bb32740 1 -- 172.21.15.39:0/222855517 shutdown_connections 2024-09-21T03:53:09.257 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:09.228+0000 7f6c4bb32740 1 -- 172.21.15.39:0/222855517 wait complete. 2024-09-21T03:53:09.257 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:09.228+0000 7f6c4bb32740 1 -- start start 2024-09-21T03:53:09.257 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:09.229+0000 7f6c4bb32740 1 Processor -- start 2024-09-21T03:53:09.257 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:09.229+0000 7f6c4bb32740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56343b66a1b0 0x56343ba458f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:53:09.257 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:09.229+0000 7f6c4bb32740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56343b672390 0x56343ba45e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:53:09.257 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:09.229+0000 7f6c4bb32740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56343b693070 0x56343ba463b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:53:09.257 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:09.229+0000 7f6c4bb32740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x56343b801700 con 0x56343b672390 2024-09-21T03:53:09.257 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:09.229+0000 7f6c4bb32740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x56343b93b160 con 0x56343b693070 2024-09-21T03:53:09.257 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:09.229+0000 7f6c4bb32740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x56343ba48aa0 con 0x56343b66a1b0 2024-09-21T03:53:09.257 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:09.229+0000 7f6c4bb32740 10 client.0 _ll_register_callbacks cb 0x56343b9549e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:53:09.257 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:09.229+0000 7f6c3e7fc640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56343b672390 0x56343ba45e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:53:09.257 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:09.229+0000 7f6c3e7fc640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56343b672390 0x56343ba45e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:48584/0 (socket says 172.21.15.39:48584) 2024-09-21T03:53:09.257 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:09.229+0000 7f6c3e7fc640 1 -- 172.21.15.39:0/4182651956 learned_addr learned my addr 172.21.15.39:0/4182651956 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:53:09.258 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:09.229+0000 7f6c3effd640 1 --2- 172.21.15.39:0/4182651956 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56343b66a1b0 0x56343ba458f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:53:09.258 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:09.229+0000 7f6c3f7fe640 1 --2- 172.21.15.39:0/4182651956 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56343b693070 0x56343ba463b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:53:09.258 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:09.229+0000 7f6c3e7fc640 1 -- 172.21.15.39:0/4182651956 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56343b66a1b0 msgr2=0x56343ba458f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:53:09.258 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:09.229+0000 7f6c3e7fc640 1 --2- 172.21.15.39:0/4182651956 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56343b66a1b0 0x56343ba458f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:09.258 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:09.229+0000 7f6c3e7fc640 1 -- 172.21.15.39:0/4182651956 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56343b693070 msgr2=0x56343ba463b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:53:09.258 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:09.229+0000 7f6c3e7fc640 1 --2- 172.21.15.39:0/4182651956 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56343b693070 0x56343ba463b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:09.258 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:09.229+0000 7f6c3e7fc640 1 -- 172.21.15.39:0/4182651956 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56343b968c90 con 0x56343b672390 2024-09-21T03:53:09.258 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:09.229+0000 7f6c3f7fe640 1 --2- 172.21.15.39:0/4182651956 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56343b693070 0x56343ba463b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:53:09.258 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:09.229+0000 7f6c3e7fc640 1 --2- 172.21.15.39:0/4182651956 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56343b672390 0x56343ba45e30 secure :-1 s=READY pgs=1011 cs=0 l=1 rev1=1 crypto rx=0x7f6c2800ead0 tx=0x7f6c280ec6b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ecd90a3e86feb6a6 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:53:09.258 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:09.229+0000 7f6c267fc640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-09-21T03:53:09.258 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:09.230+0000 7f6c267fc640 1 -- 172.21.15.39:0/4182651956 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c280aad30 con 0x56343b672390 2024-09-21T03:53:09.258 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:09.230+0000 7f6c4bb32740 1 -- 172.21.15.39:0/4182651956 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6c380043d0 con 0x56343b672390 2024-09-21T03:53:09.258 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:09.230+0000 7f6c267fc640 1 -- 172.21.15.39:0/4182651956 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8484+0+0 (secure 0 0 0) 0x7f6c280ecc70 con 0x56343b672390 2024-09-21T03:53:09.259 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:09.230+0000 7f6c267fc640 1 -- 172.21.15.39:0/4182651956 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f6c280ae600 con 0x56343b672390 2024-09-21T03:53:09.259 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:09.230+0000 7f6c4bb32740 10 client.17313 fetch_fsmap learned FSMap version 29 2024-09-21T03:53:09.259 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:09.230+0000 7f6c4bb32740 1 -- 172.21.15.39:0/4182651956 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x56343ba547a0 con 0x56343b672390 2024-09-21T03:53:09.259 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:09.230+0000 7f6c267fc640 1 -- 172.21.15.39:0/4182651956 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6c380043d0 con 0x56343b672390 2024-09-21T03:53:09.259 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:09.230+0000 7f6c267fc640 1 -- 172.21.15.39:0/4182651956 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f6c28005210 con 0x56343b672390 2024-09-21T03:53:09.259 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:09.230+0000 7f6c4bb32740 10 client.17313 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T03:53:09.259 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:09.230+0000 7f6c4bb32740 10 client.17313 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.a 2024-09-21T03:53:09.259 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:09.230+0000 7f6c4bb32740 20 client.17313 populate_metadata read hostname 'smithi039' 2024-09-21T03:53:09.259 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:09.230+0000 7f6c4bb32740 1 --2- 172.21.15.39:0/4182651956 >> [v2:172.21.15.39:6840/4247190456,v1:172.21.15.39:6841/4247190456] conn(0x56343b6519f0 0x56343ba57d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:53:09.259 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:09.230+0000 7f6c4bb32740 4 client.17313 mds_command: new command op to 14868 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:53:09.259 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:09.230+0000 7f6c4bb32740 1 -- 172.21.15.39:0/4182651956 --> [v2:172.21.15.39:6840/4247190456,v1:172.21.15.39:6841/4247190456] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56343b9b2df0 con 0x56343b6519f0 2024-09-21T03:53:09.259 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:09.230+0000 7f6c3effd640 1 --2- 172.21.15.39:0/4182651956 >> [v2:172.21.15.39:6840/4247190456,v1:172.21.15.39:6841/4247190456] conn(0x56343b6519f0 0x56343ba57d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:53:09.259 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:09.231+0000 7f6c267fc640 10 client.17313 ms_handle_connect on v2:172.21.15.39:6840/4247190456 2024-09-21T03:53:09.259 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:09.231+0000 7f6c3effd640 1 --2- 172.21.15.39:0/4182651956 >> [v2:172.21.15.39:6840/4247190456,v1:172.21.15.39:6841/4247190456] conn(0x56343b6519f0 0x56343ba57d00 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=b5f757f62b855899 server_cookie=12355efa9307f83d in_seq=0 out_seq=0 2024-09-21T03:53:09.259 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:09.231+0000 7f6c267fc640 1 -- 172.21.15.39:0/4182651956 <== mds.4 v2:172.21.15.39:6840/4247190456 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x56343b9b2df0 con 0x56343b6519f0 2024-09-21T03:53:09.260 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:09.231+0000 7f6c267fc640 10 client.17313 handle_command_reply: tid=0 2024-09-21T03:53:09.260 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:09.232+0000 7f6c4bb32740 1 client.17313 shutdown 2024-09-21T03:53:09.260 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:09.232+0000 7f6c4bb32740 1 -- 172.21.15.39:0/4182651956 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56343b672390 msgr2=0x56343ba45e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:53:09.260 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:09.232+0000 7f6c4bb32740 1 --2- 172.21.15.39:0/4182651956 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56343b672390 0x56343ba45e30 secure :-1 s=READY pgs=1011 cs=0 l=1 rev1=1 crypto rx=0x7f6c2800ead0 tx=0x7f6c280ec6b0 comp rx=0 tx=0).stop 2024-09-21T03:53:09.260 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:09.232+0000 7f6c4bb32740 1 -- 172.21.15.39:0/4182651956 shutdown_connections 2024-09-21T03:53:09.260 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:09.232+0000 7f6c4bb32740 1 --2- 172.21.15.39:0/4182651956 >> [v2:172.21.15.39:6840/4247190456,v1:172.21.15.39:6841/4247190456] conn(0x56343b6519f0 0x56343ba57d00 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:09.260 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:09.232+0000 7f6c4bb32740 1 --2- 172.21.15.39:0/4182651956 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56343b693070 0x56343ba463b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:09.260 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:09.232+0000 7f6c267fc640 0 client.17313 ms_handle_reset on v2:172.21.15.39:6840/4247190456 2024-09-21T03:53:09.260 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:09.232+0000 7f6c4bb32740 1 --2- 172.21.15.39:0/4182651956 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56343b672390 0x56343ba45e30 unknown :-1 s=CLOSED pgs=1011 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:09.260 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:09.232+0000 7f6c4bb32740 1 --2- 172.21.15.39:0/4182651956 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56343b66a1b0 0x56343ba458f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:09.260 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:09.232+0000 7f6c4bb32740 1 -- 172.21.15.39:0/4182651956 >> 172.21.15.39:0/4182651956 conn(0x56343b687460 msgr2=0x56343ba3d450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:53:09.260 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:09.232+0000 7f6c4bb32740 1 -- 172.21.15.39:0/4182651956 shutdown_connections 2024-09-21T03:53:09.260 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:09.232+0000 7f6c4bb32740 1 -- 172.21.15.39:0/4182651956 wait complete. 2024-09-21T03:53:09.260 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:09.232+0000 7f6c4bb32740 20 client.17313 trim_cache size 0 max 16384 2024-09-21T03:53:09.261 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:09.245+0000 7f6c4bb32740 1 Processor -- start 2024-09-21T03:53:09.261 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:09.245+0000 7f6c4bb32740 1 -- start start 2024-09-21T03:53:09.261 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:09.245+0000 7f6c4bb32740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56343b66a1b0 0x56343ba410d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:53:09.261 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:09.246+0000 7f6c4bb32740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56343b672390 0x56343ba3ba00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:53:09.261 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:09.246+0000 7f6c4bb32740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56343b693070 0x56343ba42ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:53:09.261 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:09.246+0000 7f6c4bb32740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f6c401ad540 con 0x56343b693070 2024-09-21T03:53:09.261 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:09.246+0000 7f6c4bb32740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f6c401af080 con 0x56343b672390 2024-09-21T03:53:09.261 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:09.246+0000 7f6c4bb32740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f6c401adc10 con 0x56343b66a1b0 2024-09-21T03:53:09.261 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:09.246+0000 7f6c3f7fe640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56343b693070 0x56343ba42ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:53:09.261 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:09.246+0000 7f6c3f7fe640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56343b693070 0x56343ba42ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:48586/0 (socket says 172.21.15.39:48586) 2024-09-21T03:53:09.261 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:09.246+0000 7f6c3f7fe640 1 -- 172.21.15.39:0/1237487411 learned_addr learned my addr 172.21.15.39:0/1237487411 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:53:09.261 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:09.246+0000 7f6c3f7fe640 1 -- 172.21.15.39:0/1237487411 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56343b66a1b0 msgr2=0x56343ba410d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:53:09.261 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:09.246+0000 7f6c3f7fe640 1 --2- 172.21.15.39:0/1237487411 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56343b66a1b0 0x56343ba410d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:09.261 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:09.246+0000 7f6c3f7fe640 1 -- 172.21.15.39:0/1237487411 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56343b672390 msgr2=0x56343ba3ba00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:53:09.262 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:09.246+0000 7f6c3f7fe640 1 --2- 172.21.15.39:0/1237487411 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56343b672390 0x56343ba3ba00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:09.262 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:09.246+0000 7f6c3f7fe640 1 -- 172.21.15.39:0/1237487411 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56343ba43df0 con 0x56343b693070 2024-09-21T03:53:09.262 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:09.246+0000 7f6c3f7fe640 1 --2- 172.21.15.39:0/1237487411 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56343b693070 0x56343ba42ae0 secure :-1 s=READY pgs=1012 cs=0 l=1 rev1=1 crypto rx=0x7f6c30009ca0 tx=0x7f6c30041000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5177baf451488257 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:53:09.262 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:09.246+0000 7f6c34ff9640 1 -- 172.21.15.39:0/1237487411 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c30003970 con 0x56343b693070 2024-09-21T03:53:09.262 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:09.246+0000 7f6c34ff9640 1 -- 172.21.15.39:0/1237487411 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6c30036050 con 0x56343b693070 2024-09-21T03:53:09.262 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:09.246+0000 7f6c34ff9640 1 -- 172.21.15.39:0/1237487411 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c30004810 con 0x56343b693070 2024-09-21T03:53:09.262 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:09.247+0000 7f6c4bb32740 1 -- 172.21.15.39:0/1237487411 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56343b693070 msgr2=0x56343ba42ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:53:09.262 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:09.247+0000 7f6c4bb32740 1 --2- 172.21.15.39:0/1237487411 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56343b693070 0x56343ba42ae0 secure :-1 s=READY pgs=1012 cs=0 l=1 rev1=1 crypto rx=0x7f6c30009ca0 tx=0x7f6c30041000 comp rx=0 tx=0).stop 2024-09-21T03:53:09.262 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:09.247+0000 7f6c4bb32740 1 -- 172.21.15.39:0/1237487411 shutdown_connections 2024-09-21T03:53:09.262 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:09.247+0000 7f6c4bb32740 1 --2- 172.21.15.39:0/1237487411 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56343b693070 0x56343ba42ae0 unknown :-1 s=CLOSED pgs=1012 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:09.262 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:09.247+0000 7f6c4bb32740 1 --2- 172.21.15.39:0/1237487411 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56343b672390 0x56343ba3ba00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:09.262 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:09.247+0000 7f6c4bb32740 1 --2- 172.21.15.39:0/1237487411 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56343b66a1b0 0x56343ba410d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:09.262 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:09.247+0000 7f6c4bb32740 1 -- 172.21.15.39:0/1237487411 >> 172.21.15.39:0/1237487411 conn(0x56343b687460 msgr2=0x56343ba3b2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:53:09.262 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:09.247+0000 7f6c4bb32740 1 -- 172.21.15.39:0/1237487411 shutdown_connections 2024-09-21T03:53:09.262 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:09.247+0000 7f6c4bb32740 1 -- 172.21.15.39:0/1237487411 wait complete. 2024-09-21T03:53:09.263 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:09.248+0000 7f6c4bb32740 1 -- start start 2024-09-21T03:53:09.263 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:09.248+0000 7f6c4bb32740 1 Processor -- start 2024-09-21T03:53:09.263 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:09.248+0000 7f6c4bb32740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56343b66a1b0 0x56343ba4e640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:53:09.263 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:09.248+0000 7f6c4bb32740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56343b672390 0x56343ba49af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:53:09.263 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:09.248+0000 7f6c4bb32740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56343b693070 0x56343ba4a190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:53:09.263 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:09.248+0000 7f6c4bb32740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f6c401ae9b0 con 0x56343b66a1b0 2024-09-21T03:53:09.263 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:09.248+0000 7f6c4bb32740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f6c401ae2e0 con 0x56343b672390 2024-09-21T03:53:09.263 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:09.248+0000 7f6c4bb32740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f6c401ab520 con 0x56343b693070 2024-09-21T03:53:09.263 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:09.248+0000 7f6c4bb32740 10 client.0 _ll_register_callbacks cb 0x7f6c28009f50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:53:09.263 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:09.248+0000 7f6c3effd640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56343b66a1b0 0x56343ba4e640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:53:09.263 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:09.248+0000 7f6c3effd640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56343b66a1b0 0x56343ba4e640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:48588/0 (socket says 172.21.15.39:48588) 2024-09-21T03:53:09.263 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:09.248+0000 7f6c3effd640 1 -- 172.21.15.39:0/1786848789 learned_addr learned my addr 172.21.15.39:0/1786848789 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:53:09.263 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:09.249+0000 7f6c3f7fe640 1 --2- 172.21.15.39:0/1786848789 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56343b693070 0x56343ba4a190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:53:09.263 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:09.249+0000 7f6c3e7fc640 1 --2- 172.21.15.39:0/1786848789 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56343b672390 0x56343ba49af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:53:09.263 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:09.249+0000 7f6c3effd640 1 -- 172.21.15.39:0/1786848789 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56343b693070 msgr2=0x56343ba4a190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:53:09.264 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:09.249+0000 7f6c3effd640 1 --2- 172.21.15.39:0/1786848789 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56343b693070 0x56343ba4a190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:09.264 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:09.249+0000 7f6c3effd640 1 -- 172.21.15.39:0/1786848789 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56343b672390 msgr2=0x56343ba49af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:53:09.264 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:09.249+0000 7f6c3effd640 1 --2- 172.21.15.39:0/1786848789 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56343b672390 0x56343ba49af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:09.264 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:09.249+0000 7f6c3effd640 1 -- 172.21.15.39:0/1786848789 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56343ba4a920 con 0x56343b66a1b0 2024-09-21T03:53:09.264 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:09.249+0000 7f6c3effd640 1 --2- 172.21.15.39:0/1786848789 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56343b66a1b0 0x56343ba4e640 secure :-1 s=READY pgs=1013 cs=0 l=1 rev1=1 crypto rx=0x56343ba38f40 tx=0x7f6c38009a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=931d62db90174865 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:53:09.264 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:09.249+0000 7f6c267fc640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-09-21T03:53:09.264 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:09.249+0000 7f6c267fc640 1 -- 172.21.15.39:0/1786848789 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c38031060 con 0x56343b66a1b0 2024-09-21T03:53:09.264 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:09.249+0000 7f6c267fc640 1 -- 172.21.15.39:0/1786848789 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8484+0+0 (secure 0 0 0) 0x7f6c38003a00 con 0x56343b66a1b0 2024-09-21T03:53:09.264 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:09.249+0000 7f6c4bb32740 1 -- 172.21.15.39:0/1786848789 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6c30036050 con 0x56343b66a1b0 2024-09-21T03:53:09.264 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:09.249+0000 7f6c267fc640 1 -- 172.21.15.39:0/1786848789 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f6c38011d10 con 0x56343b66a1b0 2024-09-21T03:53:09.264 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:09.249+0000 7f6c4bb32740 10 client.17319 fetch_fsmap learned FSMap version 29 2024-09-21T03:53:09.264 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:09.250+0000 7f6c4bb32740 1 -- 172.21.15.39:0/1786848789 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x56343bac2720 con 0x56343b66a1b0 2024-09-21T03:53:09.264 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:09.250+0000 7f6c267fc640 1 -- 172.21.15.39:0/1786848789 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6c30036050 con 0x56343b66a1b0 2024-09-21T03:53:09.264 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:09.250+0000 7f6c267fc640 1 -- 172.21.15.39:0/1786848789 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f6c38019800 con 0x56343b66a1b0 2024-09-21T03:53:09.264 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:09.250+0000 7f6c4bb32740 10 client.17319 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T03:53:09.265 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:09.250+0000 7f6c4bb32740 10 client.17319 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.a 2024-09-21T03:53:09.265 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:09.250+0000 7f6c4bb32740 20 client.17319 populate_metadata read hostname 'smithi039' 2024-09-21T03:53:09.265 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:09.250+0000 7f6c4bb32740 1 --2- 172.21.15.39:0/1786848789 >> [v2:172.21.15.39:6840/4247190456,v1:172.21.15.39:6841/4247190456] conn(0x56343b626da0 0x56343bac4d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:53:09.265 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:09.250+0000 7f6c4bb32740 4 client.17319 mds_command: new command op to 14868 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T03:53:09.265 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:09.250+0000 7f6c4bb32740 1 -- 172.21.15.39:0/1786848789 --> [v2:172.21.15.39:6840/4247190456,v1:172.21.15.39:6841/4247190456] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56343b9ae8a0 con 0x56343b626da0 2024-09-21T03:53:09.265 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:09.250+0000 7f6c3e7fc640 1 --2- 172.21.15.39:0/1786848789 >> [v2:172.21.15.39:6840/4247190456,v1:172.21.15.39:6841/4247190456] conn(0x56343b626da0 0x56343bac4d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:53:09.266 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:09.250+0000 7f6c267fc640 10 client.17319 ms_handle_connect on v2:172.21.15.39:6840/4247190456 2024-09-21T03:53:09.266 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:09.251+0000 7f6c3e7fc640 1 --2- 172.21.15.39:0/1786848789 >> [v2:172.21.15.39:6840/4247190456,v1:172.21.15.39:6841/4247190456] conn(0x56343b626da0 0x56343bac4d20 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=607209773bd04c92 server_cookie=5e3d4ee09d17574e in_seq=0 out_seq=0 2024-09-21T03:53:09.266 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:09.251+0000 7f6c267fc640 1 -- 172.21.15.39:0/1786848789 <== mds.4 v2:172.21.15.39:6840/4247190456 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56343b9ae8a0 con 0x56343b626da0 2024-09-21T03:53:09.267 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:09.251+0000 7f6c267fc640 10 client.17319 handle_command_reply: tid=0 2024-09-21T03:53:09.267 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:09.251+0000 7f6c4bb32740 1 client.17319 shutdown 2024-09-21T03:53:09.267 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:09.251+0000 7f6c4bb32740 1 -- 172.21.15.39:0/1786848789 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56343b66a1b0 msgr2=0x56343ba4e640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:53:09.267 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:09.251+0000 7f6c4bb32740 1 --2- 172.21.15.39:0/1786848789 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56343b66a1b0 0x56343ba4e640 secure :-1 s=READY pgs=1013 cs=0 l=1 rev1=1 crypto rx=0x56343ba38f40 tx=0x7f6c38009a30 comp rx=0 tx=0).stop 2024-09-21T03:53:09.267 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:09.251+0000 7f6c4bb32740 1 -- 172.21.15.39:0/1786848789 shutdown_connections 2024-09-21T03:53:09.267 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:09.251+0000 7f6c4bb32740 1 --2- 172.21.15.39:0/1786848789 >> [v2:172.21.15.39:6840/4247190456,v1:172.21.15.39:6841/4247190456] conn(0x56343b626da0 0x56343bac4d20 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:09.267 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:09.251+0000 7f6c4bb32740 1 --2- 172.21.15.39:0/1786848789 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56343b693070 0x56343ba4a190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:09.267 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:09.251+0000 7f6c4bb32740 1 --2- 172.21.15.39:0/1786848789 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56343b672390 0x56343ba49af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:09.267 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:09.251+0000 7f6c4bb32740 1 --2- 172.21.15.39:0/1786848789 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56343b66a1b0 0x56343ba4e640 unknown :-1 s=CLOSED pgs=1013 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:09.267 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:09.251+0000 7f6c4bb32740 1 -- 172.21.15.39:0/1786848789 >> 172.21.15.39:0/1786848789 conn(0x56343b687460 msgr2=0x56343ba50c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:53:09.267 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:09.251+0000 7f6c267fc640 0 client.17319 ms_handle_reset on v2:172.21.15.39:6840/4247190456 2024-09-21T03:53:09.267 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:09.251+0000 7f6c4bb32740 1 -- 172.21.15.39:0/1786848789 shutdown_connections 2024-09-21T03:53:09.267 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:09.251+0000 7f6c4bb32740 1 -- 172.21.15.39:0/1786848789 wait complete. 2024-09-21T03:53:09.267 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:09.251+0000 7f6c4bb32740 20 client.17319 trim_cache size 0 max 16384 2024-09-21T03:53:09.268 INFO:teuthology.orchestra.run.smithi039.stdout:[] 2024-09-21T03:53:09.268 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:09.252+0000 7f6c455be640 1 -- 172.21.15.39:0/3426973713 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f6c2c0b6680 msgr2=0x7f6c2c0b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:53:09.268 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:09.252+0000 7f6c455be640 1 --2- 172.21.15.39:0/3426973713 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f6c2c0b6680 0x7f6c2c0b8b90 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f6c3002f500 tx=0x7f6c3002f310 comp rx=0 tx=0).stop 2024-09-21T03:53:09.268 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:09.252+0000 7f6c455be640 1 -- 172.21.15.39:0/3426973713 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6c401083a0 msgr2=0x7f6c40199950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:53:09.269 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:09.252+0000 7f6c455be640 1 --2- 172.21.15.39:0/3426973713 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6c401083a0 0x7f6c40199950 secure :-1 s=READY pgs=1009 cs=0 l=1 rev1=1 crypto rx=0x7f6c2800eb00 tx=0x7f6c28009ee0 comp rx=0 tx=0).stop 2024-09-21T03:53:09.269 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:09.252+0000 7f6c455be640 1 -- 172.21.15.39:0/3426973713 shutdown_connections 2024-09-21T03:53:09.269 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:09.252+0000 7f6c455be640 1 --2- 172.21.15.39:0/3426973713 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f6c2c0b6680 0x7f6c2c0b8b90 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:09.269 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:09.252+0000 7f6c455be640 1 --2- 172.21.15.39:0/3426973713 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6c40108cf0 0x7f6c4019ebf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:09.269 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:09.252+0000 7f6c455be640 1 --2- 172.21.15.39:0/3426973713 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6c401083a0 0x7f6c40199950 unknown :-1 s=CLOSED pgs=1009 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:09.269 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:09.252+0000 7f6c455be640 1 --2- 172.21.15.39:0/3426973713 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6c40102a60 0x7f6c401993b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:09.269 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:09.252+0000 7f6c455be640 1 -- 172.21.15.39:0/3426973713 >> 172.21.15.39:0/3426973713 conn(0x7f6c40079730 msgr2=0x7f6c40104410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:53:09.269 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:09.252+0000 7f6c455be640 1 -- 172.21.15.39:0/3426973713 shutdown_connections 2024-09-21T03:53:09.269 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:09.252+0000 7f6c455be640 1 -- 172.21.15.39:0/3426973713 wait complete. 2024-09-21T03:53:09.584 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:53:09 smithi039 ceph-mon[31211]: pgmap v538: 97 pgs: 97 active+clean; 10 MiB data, 520 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 44 KiB/s wr, 25 op/s 2024-09-21T03:53:09.718 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:53:09 smithi161 ceph-mon[34533]: pgmap v538: 97 pgs: 97 active+clean; 10 MiB data, 520 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 44 KiB/s wr, 25 op/s 2024-09-21T03:53:09.725 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:53:09 smithi150 ceph-mon[34289]: pgmap v538: 97 pgs: 97 active+clean; 10 MiB data, 520 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 44 KiB/s wr, 25 op/s 2024-09-21T03:53:10.039 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 51116d5c-77ca-11ef-baf5-efdc52797490 -- ceph tell mds.1:3 damage ls 2024-09-21T03:53:10.301 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/51116d5c-77ca-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:53:10.794 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:10.778+0000 7f3b0d99f640 1 -- 172.21.15.39:0/259199079 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3b081094a0 msgr2=0x7f3b081098b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:53:10.794 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:10.778+0000 7f3b0d99f640 1 --2- 172.21.15.39:0/259199079 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3b081094a0 0x7f3b081098b0 secure :-1 s=READY pgs=1014 cs=0 l=1 rev1=1 crypto rx=0x7f3af0009900 tx=0x7f3af002a410 comp rx=0 tx=0).stop 2024-09-21T03:53:10.794 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:10.778+0000 7f3b0d99f640 1 -- 172.21.15.39:0/259199079 shutdown_connections 2024-09-21T03:53:10.794 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:10.778+0000 7f3b0d99f640 1 --2- 172.21.15.39:0/259199079 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3b08109df0 0x7f3b08110620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:10.794 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:10.778+0000 7f3b0d99f640 1 --2- 172.21.15.39:0/259199079 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3b081094a0 0x7f3b081098b0 unknown :-1 s=CLOSED pgs=1014 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:10.795 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:10.778+0000 7f3b0d99f640 1 --2- 172.21.15.39:0/259199079 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3b08103b60 0x7f3b08103f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:10.795 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:10.778+0000 7f3b0d99f640 1 -- 172.21.15.39:0/259199079 >> 172.21.15.39:0/259199079 conn(0x7f3b0807b850 msgr2=0x7f3b0807bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:53:10.795 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:10.778+0000 7f3b0d99f640 1 -- 172.21.15.39:0/259199079 shutdown_connections 2024-09-21T03:53:10.795 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:10.778+0000 7f3b0d99f640 1 -- 172.21.15.39:0/259199079 wait complete. 2024-09-21T03:53:10.795 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:10.779+0000 7f3b0d99f640 1 Processor -- start 2024-09-21T03:53:10.795 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:10.779+0000 7f3b0d99f640 1 -- start start 2024-09-21T03:53:10.795 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:10.779+0000 7f3b0d99f640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3b08103b60 0x7f3b0819a580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:53:10.795 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:10.779+0000 7f3b0d99f640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3b081094a0 0x7f3b0819ab20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:53:10.795 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:10.779+0000 7f3b0d99f640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3b08109df0 0x7f3b0819fdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:53:10.795 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:10.779+0000 7f3b0d99f640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f3b08113af0 con 0x7f3b08103b60 2024-09-21T03:53:10.795 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:10.779+0000 7f3b0d99f640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f3b08113970 con 0x7f3b08109df0 2024-09-21T03:53:10.795 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:10.779+0000 7f3b0d99f640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f3b08113c70 con 0x7f3b081094a0 2024-09-21T03:53:10.795 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:10.780+0000 7f3b06ffd640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3b08103b60 0x7f3b0819a580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:53:10.795 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:10.780+0000 7f3b06ffd640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3b08103b60 0x7f3b0819a580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:48606/0 (socket says 172.21.15.39:48606) 2024-09-21T03:53:10.796 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:10.780+0000 7f3b06ffd640 1 -- 172.21.15.39:0/2339125589 learned_addr learned my addr 172.21.15.39:0/2339125589 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:53:10.796 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:10.780+0000 7f3b067fc640 1 --2- 172.21.15.39:0/2339125589 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3b081094a0 0x7f3b0819ab20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:53:10.796 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:10.780+0000 7f3b077fe640 1 --2- 172.21.15.39:0/2339125589 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3b08109df0 0x7f3b0819fdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:53:10.796 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:10.780+0000 7f3b06ffd640 1 -- 172.21.15.39:0/2339125589 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3b081094a0 msgr2=0x7f3b0819ab20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:53:10.796 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:10.780+0000 7f3b06ffd640 1 --2- 172.21.15.39:0/2339125589 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3b081094a0 0x7f3b0819ab20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:10.796 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:10.780+0000 7f3b06ffd640 1 -- 172.21.15.39:0/2339125589 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3b08109df0 msgr2=0x7f3b0819fdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:53:10.796 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:10.780+0000 7f3b06ffd640 1 --2- 172.21.15.39:0/2339125589 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3b08109df0 0x7f3b0819fdc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:10.796 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:10.780+0000 7f3b06ffd640 1 -- 172.21.15.39:0/2339125589 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3b081a04f0 con 0x7f3b08103b60 2024-09-21T03:53:10.796 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:10.780+0000 7f3b067fc640 1 --2- 172.21.15.39:0/2339125589 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3b081094a0 0x7f3b0819ab20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:53:10.796 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:10.780+0000 7f3b06ffd640 1 --2- 172.21.15.39:0/2339125589 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3b08103b60 0x7f3b0819a580 secure :-1 s=READY pgs=1015 cs=0 l=1 rev1=1 crypto rx=0x7f3af800ba50 tx=0x7f3af800bf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:53:10.796 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:10.780+0000 7f3b0c99d640 1 -- 172.21.15.39:0/2339125589 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3af8019070 con 0x7f3b08103b60 2024-09-21T03:53:10.796 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:10.780+0000 7f3b0c99d640 1 -- 172.21.15.39:0/2339125589 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3af800cbe0 con 0x7f3b08103b60 2024-09-21T03:53:10.796 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:10.780+0000 7f3b0d99f640 1 -- 172.21.15.39:0/2339125589 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3b081a07e0 con 0x7f3b08103b60 2024-09-21T03:53:10.797 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:10.780+0000 7f3b0c99d640 1 -- 172.21.15.39:0/2339125589 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3af80075d0 con 0x7f3b08103b60 2024-09-21T03:53:10.797 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:10.780+0000 7f3b0d99f640 1 -- 172.21.15.39:0/2339125589 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3b081a6f40 con 0x7f3b08103b60 2024-09-21T03:53:10.797 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:10.783+0000 7f3b13f13740 1 Processor -- start 2024-09-21T03:53:10.797 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:10.784+0000 7f3b13f13740 1 -- start start 2024-09-21T03:53:10.797 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:10.784+0000 7f3b13f13740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562d931fc070 0x562d935a32c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:53:10.797 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:10.784+0000 7f3b13f13740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562d931d31b0 0x562d935a3800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:53:10.797 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:10.784+0000 7f3b13f13740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562d931db390 0x562d935a3d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:53:10.797 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:10.784+0000 7f3b13f13740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x562d93519ea0 con 0x562d931fc070 2024-09-21T03:53:10.797 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:10.784+0000 7f3b13f13740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x562d935092d0 con 0x562d931d31b0 2024-09-21T03:53:10.797 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:10.784+0000 7f3b13f13740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x562d9350d0e0 con 0x562d931db390 2024-09-21T03:53:10.797 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:10.784+0000 7f3b0c99d640 1 -- 172.21.15.39:0/2339125589 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 27) ==== 154643+0+0 (secure 0 0 0) 0x7f3af801c070 con 0x7f3b08103b60 2024-09-21T03:53:10.797 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:10.784+0000 7f3b077fe640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562d931fc070 0x562d935a32c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:53:10.797 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:10.784+0000 7f3b077fe640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562d931fc070 0x562d935a32c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:48612/0 (socket says 172.21.15.39:48612) 2024-09-21T03:53:10.797 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:10.784+0000 7f3b077fe640 1 -- 172.21.15.39:0/3527432708 learned_addr learned my addr 172.21.15.39:0/3527432708 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:53:10.797 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:10.784+0000 7f3b067fc640 1 --2- 172.21.15.39:0/3527432708 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562d931db390 0x562d935a3d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:53:10.798 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:10.785+0000 7f3b06ffd640 1 --2- 172.21.15.39:0/3527432708 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562d931d31b0 0x562d935a3800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:53:10.798 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:10.785+0000 7f3b077fe640 1 -- 172.21.15.39:0/3527432708 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562d931db390 msgr2=0x562d935a3d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:53:10.798 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:10.785+0000 7f3b077fe640 1 --2- 172.21.15.39:0/3527432708 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562d931db390 0x562d935a3d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:10.798 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:10.785+0000 7f3b077fe640 1 -- 172.21.15.39:0/3527432708 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562d931d31b0 msgr2=0x562d935a3800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:53:10.798 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:10.785+0000 7f3b077fe640 1 --2- 172.21.15.39:0/3527432708 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562d931d31b0 0x562d935a3800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:10.798 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:10.785+0000 7f3b077fe640 1 -- 172.21.15.39:0/3527432708 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562d93267650 con 0x562d931fc070 2024-09-21T03:53:10.798 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:10.785+0000 7f3b077fe640 1 --2- 172.21.15.39:0/3527432708 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562d931fc070 0x562d935a32c0 secure :-1 s=READY pgs=1016 cs=0 l=1 rev1=1 crypto rx=0x7f3b0000ba10 tx=0x7f3b0000bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a9fbd1bbff208fe5 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:53:10.798 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:10.785+0000 7f3afdffb640 1 -- 172.21.15.39:0/3527432708 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b0000de50 con 0x562d931fc070 2024-09-21T03:53:10.798 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:10.785+0000 7f3afdffb640 1 -- 172.21.15.39:0/3527432708 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3b000178d0 con 0x562d931fc070 2024-09-21T03:53:10.798 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:10.785+0000 7f3afdffb640 1 -- 172.21.15.39:0/3527432708 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b0000f950 con 0x562d931fc070 2024-09-21T03:53:10.798 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:10.785+0000 7f3b0c99d640 1 --2- 172.21.15.39:0/2339125589 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f3af40b6680 0x7f3af40b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:53:10.798 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:10.785+0000 7f3b06ffd640 1 --2- 172.21.15.39:0/2339125589 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f3af40b6680 0x7f3af40b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:53:10.798 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:10.786+0000 7f3b0c99d640 1 -- 172.21.15.39:0/2339125589 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8572+0+0 (secure 0 0 0) 0x7f3af80e6d00 con 0x7f3b08103b60 2024-09-21T03:53:10.799 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:10.786+0000 7f3b13f13740 1 -- 172.21.15.39:0/3527432708 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562d931fc070 msgr2=0x562d935a32c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:53:10.799 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:10.786+0000 7f3b06ffd640 1 --2- 172.21.15.39:0/2339125589 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f3af40b6680 0x7f3af40b8b90 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f3af80e44d0 tx=0x7f3af8094040 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:53:10.799 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:10.786+0000 7f3b13f13740 1 --2- 172.21.15.39:0/3527432708 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562d931fc070 0x562d935a32c0 secure :-1 s=READY pgs=1016 cs=0 l=1 rev1=1 crypto rx=0x7f3b0000ba10 tx=0x7f3b0000bed0 comp rx=0 tx=0).stop 2024-09-21T03:53:10.799 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:10.786+0000 7f3b13f13740 1 -- 172.21.15.39:0/3527432708 shutdown_connections 2024-09-21T03:53:10.799 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:10.786+0000 7f3b13f13740 1 --2- 172.21.15.39:0/3527432708 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562d931db390 0x562d935a3d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:10.799 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:10.786+0000 7f3b13f13740 1 --2- 172.21.15.39:0/3527432708 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562d931d31b0 0x562d935a3800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:10.799 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:10.786+0000 7f3b13f13740 1 --2- 172.21.15.39:0/3527432708 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562d931fc070 0x562d935a32c0 unknown :-1 s=CLOSED pgs=1016 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:10.799 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:10.786+0000 7f3b13f13740 1 -- 172.21.15.39:0/3527432708 >> 172.21.15.39:0/3527432708 conn(0x562d931f0460 msgr2=0x562d935a11b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:53:10.799 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:10.786+0000 7f3b13f13740 1 -- 172.21.15.39:0/3527432708 shutdown_connections 2024-09-21T03:53:10.799 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:10.786+0000 7f3b13f13740 1 -- 172.21.15.39:0/3527432708 wait complete. 2024-09-21T03:53:10.799 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:10.787+0000 7f3b13f13740 1 -- start start 2024-09-21T03:53:10.799 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:10.787+0000 7f3b13f13740 1 Processor -- start 2024-09-21T03:53:10.799 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:10.787+0000 7f3b13f13740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562d931d31b0 0x562d935ae9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:53:10.799 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:10.787+0000 7f3b13f13740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562d931db390 0x562d935aeee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:53:10.799 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:10.787+0000 7f3b13f13740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562d931fc070 0x562d935bad90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:53:10.800 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:10.787+0000 7f3b13f13740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x562d9336a700 con 0x562d931db390 2024-09-21T03:53:10.800 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:10.787+0000 7f3b13f13740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x562d934a4160 con 0x562d931d31b0 2024-09-21T03:53:10.800 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:10.787+0000 7f3b13f13740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x562d935b1aa0 con 0x562d931fc070 2024-09-21T03:53:10.800 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:10.787+0000 7f3b13f13740 10 client.0 _ll_register_callbacks cb 0x562d934bd9e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:53:10.800 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:10.787+0000 7f3b067fc640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562d931db390 0x562d935aeee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:53:10.800 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:10.787+0000 7f3b067fc640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562d931db390 0x562d935aeee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:48628/0 (socket says 172.21.15.39:48628) 2024-09-21T03:53:10.800 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:10.787+0000 7f3b067fc640 1 -- 172.21.15.39:0/3728781272 learned_addr learned my addr 172.21.15.39:0/3728781272 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:53:10.800 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:10.788+0000 7f3b077fe640 1 --2- 172.21.15.39:0/3728781272 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562d931d31b0 0x562d935ae9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:53:10.800 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:10.788+0000 7f3b077fe640 1 --2- 172.21.15.39:0/3728781272 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562d931fc070 0x562d935bad90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:53:10.800 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:10.788+0000 7f3b067fc640 1 -- 172.21.15.39:0/3728781272 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562d931fc070 msgr2=0x562d935bad90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:53:10.800 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:10.788+0000 7f3b067fc640 1 --2- 172.21.15.39:0/3728781272 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562d931fc070 0x562d935bad90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:10.800 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:10.788+0000 7f3b067fc640 1 -- 172.21.15.39:0/3728781272 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562d931d31b0 msgr2=0x562d935ae9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:53:10.801 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:10.788+0000 7f3b067fc640 1 --2- 172.21.15.39:0/3728781272 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562d931d31b0 0x562d935ae9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:10.801 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:10.788+0000 7f3b067fc640 1 -- 172.21.15.39:0/3728781272 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562d934d1c90 con 0x562d931db390 2024-09-21T03:53:10.801 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:10.788+0000 7f3b077fe640 1 --2- 172.21.15.39:0/3728781272 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562d931d31b0 0x562d935ae9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:53:10.801 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:10.788+0000 7f3b067fc640 1 --2- 172.21.15.39:0/3728781272 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562d931db390 0x562d935aeee0 secure :-1 s=READY pgs=1017 cs=0 l=1 rev1=1 crypto rx=0x7f3af0005ab0 tx=0x7f3af0031970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f377375e61e2a60 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:53:10.801 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:10.788+0000 7f3ae6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-09-21T03:53:10.801 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:10.788+0000 7f3ae6ffd640 1 -- 172.21.15.39:0/3728781272 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3af0035040 con 0x562d931db390 2024-09-21T03:53:10.801 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:10.788+0000 7f3ae6ffd640 1 -- 172.21.15.39:0/3728781272 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8572+0+0 (secure 0 0 0) 0x7f3af0004300 con 0x562d931db390 2024-09-21T03:53:10.801 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:10.788+0000 7f3b13f13740 1 -- 172.21.15.39:0/3728781272 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3b000178d0 con 0x562d931db390 2024-09-21T03:53:10.801 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:10.788+0000 7f3ae6ffd640 1 -- 172.21.15.39:0/3728781272 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f3af0005020 con 0x562d931db390 2024-09-21T03:53:10.801 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:10.788+0000 7f3b13f13740 10 client.17331 fetch_fsmap learned FSMap version 29 2024-09-21T03:53:10.801 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:10.788+0000 7f3b13f13740 1 -- 172.21.15.39:0/3728781272 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x562d935bd770 con 0x562d931db390 2024-09-21T03:53:10.802 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:10.789+0000 7f3ae6ffd640 1 -- 172.21.15.39:0/3728781272 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3b000178d0 con 0x562d931db390 2024-09-21T03:53:10.802 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:10.789+0000 7f3ae6ffd640 1 -- 172.21.15.39:0/3728781272 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f3af00035f0 con 0x562d931db390 2024-09-21T03:53:10.802 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:10.789+0000 7f3b13f13740 10 client.17331 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T03:53:10.802 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:10.789+0000 7f3b13f13740 10 client.17331 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-21T03:53:10.802 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:10.789+0000 7f3b13f13740 20 client.17331 populate_metadata read hostname 'smithi039' 2024-09-21T03:53:10.802 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:10.789+0000 7f3b13f13740 1 --2- 172.21.15.39:0/3728781272 >> [v2:172.21.15.161:6836/3451544597,v1:172.21.15.161:6837/3451544597] conn(0x562d931ba9f0 0x562d935c0c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:53:10.802 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:10.789+0000 7f3b13f13740 4 client.17331 mds_command: new command op to 24646 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:53:10.802 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:10.789+0000 7f3b13f13740 1 -- 172.21.15.39:0/3728781272 --> [v2:172.21.15.161:6836/3451544597,v1:172.21.15.161:6837/3451544597] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562d9351bdf0 con 0x562d931ba9f0 2024-09-21T03:53:10.802 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:10.789+0000 7f3b077fe640 1 --2- 172.21.15.39:0/3728781272 >> [v2:172.21.15.161:6836/3451544597,v1:172.21.15.161:6837/3451544597] conn(0x562d931ba9f0 0x562d935c0c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:53:10.802 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:10.790+0000 7f3b077fe640 1 --2- 172.21.15.39:0/3728781272 >> [v2:172.21.15.161:6836/3451544597,v1:172.21.15.161:6837/3451544597] conn(0x562d931ba9f0 0x562d935c0c80 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=3812949fb76d9311 server_cookie=5c72d8e17e2f7b1c in_seq=0 out_seq=0 2024-09-21T03:53:10.803 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:10.790+0000 7f3ae6ffd640 10 client.17331 ms_handle_connect on v2:172.21.15.161:6836/3451544597 2024-09-21T03:53:10.803 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:10.790+0000 7f3ae6ffd640 1 -- 172.21.15.39:0/3728781272 <== mds.3 v2:172.21.15.161:6836/3451544597 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x562d9351bdf0 con 0x562d931ba9f0 2024-09-21T03:53:10.803 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:10.790+0000 7f3ae6ffd640 10 client.17331 handle_command_reply: tid=0 2024-09-21T03:53:10.803 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:10.790+0000 7f3b13f13740 1 client.17331 shutdown 2024-09-21T03:53:10.803 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:10.791+0000 7f3b13f13740 1 -- 172.21.15.39:0/3728781272 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562d931db390 msgr2=0x562d935aeee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:53:10.803 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:10.791+0000 7f3b13f13740 1 --2- 172.21.15.39:0/3728781272 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562d931db390 0x562d935aeee0 secure :-1 s=READY pgs=1017 cs=0 l=1 rev1=1 crypto rx=0x7f3af0005ab0 tx=0x7f3af0031970 comp rx=0 tx=0).stop 2024-09-21T03:53:10.803 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:10.791+0000 7f3b13f13740 1 -- 172.21.15.39:0/3728781272 shutdown_connections 2024-09-21T03:53:10.803 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:10.791+0000 7f3b13f13740 1 --2- 172.21.15.39:0/3728781272 >> [v2:172.21.15.161:6836/3451544597,v1:172.21.15.161:6837/3451544597] conn(0x562d931ba9f0 0x562d935c0c80 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:10.803 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:10.791+0000 7f3b13f13740 1 --2- 172.21.15.39:0/3728781272 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562d931fc070 0x562d935bad90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:10.803 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:10.791+0000 7f3b13f13740 1 --2- 172.21.15.39:0/3728781272 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562d931db390 0x562d935aeee0 unknown :-1 s=CLOSED pgs=1017 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:10.803 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:10.791+0000 7f3ae6ffd640 0 client.17331 ms_handle_reset on v2:172.21.15.161:6836/3451544597 2024-09-21T03:53:10.803 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:10.791+0000 7f3b13f13740 1 --2- 172.21.15.39:0/3728781272 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562d931d31b0 0x562d935ae9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:10.803 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:10.791+0000 7f3b13f13740 1 -- 172.21.15.39:0/3728781272 >> 172.21.15.39:0/3728781272 conn(0x562d931f0460 msgr2=0x562d935a6470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:53:10.803 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:10.791+0000 7f3b13f13740 1 -- 172.21.15.39:0/3728781272 shutdown_connections 2024-09-21T03:53:10.804 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:10.791+0000 7f3b13f13740 1 -- 172.21.15.39:0/3728781272 wait complete. 2024-09-21T03:53:10.804 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:10.791+0000 7f3b13f13740 20 client.17331 trim_cache size 0 max 16384 2024-09-21T03:53:10.807 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:10.804+0000 7f3b13f13740 1 Processor -- start 2024-09-21T03:53:10.807 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:10.804+0000 7f3b13f13740 1 -- start start 2024-09-21T03:53:10.807 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:10.805+0000 7f3b13f13740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562d931d31b0 0x562d935b05d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:53:10.808 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:10.805+0000 7f3b13f13740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562d931db390 0x562d935bac90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:53:10.808 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:10.805+0000 7f3b13f13740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562d931fc070 0x562d935abb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:53:10.808 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:10.805+0000 7f3b13f13740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f3b081af870 con 0x562d931db390 2024-09-21T03:53:10.808 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:10.805+0000 7f3b13f13740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f3b081b13b0 con 0x562d931fc070 2024-09-21T03:53:10.808 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:10.805+0000 7f3b13f13740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f3b081aff40 con 0x562d931d31b0 2024-09-21T03:53:10.808 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:10.805+0000 7f3b067fc640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562d931db390 0x562d935bac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:53:10.808 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:10.805+0000 7f3b067fc640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562d931db390 0x562d935bac90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:48634/0 (socket says 172.21.15.39:48634) 2024-09-21T03:53:10.808 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:10.805+0000 7f3b067fc640 1 -- 172.21.15.39:0/2485819416 learned_addr learned my addr 172.21.15.39:0/2485819416 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:53:10.808 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:10.805+0000 7f3b077fe640 1 --2- 172.21.15.39:0/2485819416 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562d931d31b0 0x562d935b05d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:53:10.808 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:10.805+0000 7f3b077fe640 1 --2- 172.21.15.39:0/2485819416 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562d931fc070 0x562d935abb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:53:10.808 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:10.805+0000 7f3b067fc640 1 -- 172.21.15.39:0/2485819416 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562d931d31b0 msgr2=0x562d935b05d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:53:10.808 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:10.805+0000 7f3b067fc640 1 --2- 172.21.15.39:0/2485819416 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562d931d31b0 0x562d935b05d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:10.808 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:10.805+0000 7f3b067fc640 1 -- 172.21.15.39:0/2485819416 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562d931fc070 msgr2=0x562d935abb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:53:10.809 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:10.805+0000 7f3b067fc640 1 --2- 172.21.15.39:0/2485819416 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562d931fc070 0x562d935abb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:10.809 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:10.805+0000 7f3b067fc640 1 -- 172.21.15.39:0/2485819416 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562d935afc60 con 0x562d931db390 2024-09-21T03:53:10.809 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:10.806+0000 7f3b067fc640 1 --2- 172.21.15.39:0/2485819416 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562d931db390 0x562d935bac90 secure :-1 s=READY pgs=1018 cs=0 l=1 rev1=1 crypto rx=0x7f3af002a930 tx=0x7f3af0004f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c54fa054c812f0ce server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:53:10.809 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:10.806+0000 7f3afd7fa640 1 -- 172.21.15.39:0/2485819416 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3af0035030 con 0x562d931db390 2024-09-21T03:53:10.809 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:10.806+0000 7f3afd7fa640 1 -- 172.21.15.39:0/2485819416 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3af0003c30 con 0x562d931db390 2024-09-21T03:53:10.809 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:10.806+0000 7f3afd7fa640 1 -- 172.21.15.39:0/2485819416 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3af0051040 con 0x562d931db390 2024-09-21T03:53:10.813 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:10.807+0000 7f3b13f13740 1 -- 172.21.15.39:0/2485819416 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562d931db390 msgr2=0x562d935bac90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:53:10.813 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:10.807+0000 7f3b13f13740 1 --2- 172.21.15.39:0/2485819416 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562d931db390 0x562d935bac90 secure :-1 s=READY pgs=1018 cs=0 l=1 rev1=1 crypto rx=0x7f3af002a930 tx=0x7f3af0004f80 comp rx=0 tx=0).stop 2024-09-21T03:53:10.813 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:10.807+0000 7f3b13f13740 1 -- 172.21.15.39:0/2485819416 shutdown_connections 2024-09-21T03:53:10.813 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:10.807+0000 7f3b13f13740 1 --2- 172.21.15.39:0/2485819416 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562d931fc070 0x562d935abb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:10.813 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:10.807+0000 7f3b13f13740 1 --2- 172.21.15.39:0/2485819416 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562d931db390 0x562d935bac90 unknown :-1 s=CLOSED pgs=1018 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:10.813 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:10.807+0000 7f3b13f13740 1 --2- 172.21.15.39:0/2485819416 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562d931d31b0 0x562d935b05d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:10.813 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:10.807+0000 7f3b13f13740 1 -- 172.21.15.39:0/2485819416 >> 172.21.15.39:0/2485819416 conn(0x562d931f0460 msgr2=0x562d935afec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:53:10.813 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:10.807+0000 7f3b13f13740 1 -- 172.21.15.39:0/2485819416 shutdown_connections 2024-09-21T03:53:10.813 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:10.807+0000 7f3b13f13740 1 -- 172.21.15.39:0/2485819416 wait complete. 2024-09-21T03:53:10.813 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:10.807+0000 7f3b13f13740 1 -- start start 2024-09-21T03:53:10.813 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:10.807+0000 7f3b13f13740 1 Processor -- start 2024-09-21T03:53:10.814 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:10.808+0000 7f3b13f13740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562d931d31b0 0x562d935b72a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:53:10.814 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:10.808+0000 7f3b13f13740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562d931db390 0x562d935b37b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:53:10.814 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:10.808+0000 7f3b13f13740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562d931fc070 0x562d935b3cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:53:10.814 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:10.808+0000 7f3b13f13740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f3b081b0ce0 con 0x562d931fc070 2024-09-21T03:53:10.814 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:10.808+0000 7f3b13f13740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f3b081b0610 con 0x562d931db390 2024-09-21T03:53:10.814 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:10.808+0000 7f3b13f13740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f3b081ad870 con 0x562d931d31b0 2024-09-21T03:53:10.814 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:10.808+0000 7f3b13f13740 10 client.0 _ll_register_callbacks cb 0x7f3af0031b10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:53:10.814 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:10.808+0000 7f3b077fe640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562d931fc070 0x562d935b3cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:53:10.814 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:10.808+0000 7f3b077fe640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562d931fc070 0x562d935b3cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:48636/0 (socket says 172.21.15.39:48636) 2024-09-21T03:53:10.814 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:10.808+0000 7f3b077fe640 1 -- 172.21.15.39:0/3555511754 learned_addr learned my addr 172.21.15.39:0/3555511754 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:53:10.814 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:10.808+0000 7f3b077fe640 1 --2- 172.21.15.39:0/3555511754 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562d931d31b0 0x562d935b72a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:53:10.814 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:10.808+0000 7f3b067fc640 1 --2- 172.21.15.39:0/3555511754 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562d931db390 0x562d935b37b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:53:10.815 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:10.808+0000 7f3b077fe640 1 -- 172.21.15.39:0/3555511754 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562d931d31b0 msgr2=0x562d935b72a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:53:10.815 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:10.808+0000 7f3b077fe640 1 --2- 172.21.15.39:0/3555511754 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562d931d31b0 0x562d935b72a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:10.815 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:10.808+0000 7f3b077fe640 1 -- 172.21.15.39:0/3555511754 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562d931db390 msgr2=0x562d935b37b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:53:10.815 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:10.808+0000 7f3b077fe640 1 --2- 172.21.15.39:0/3555511754 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562d931db390 0x562d935b37b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:10.815 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:10.808+0000 7f3b077fe640 1 -- 172.21.15.39:0/3555511754 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562d935b4260 con 0x562d931fc070 2024-09-21T03:53:10.815 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:10.808+0000 7f3b077fe640 1 --2- 172.21.15.39:0/3555511754 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562d931fc070 0x562d935b3cf0 secure :-1 s=READY pgs=1019 cs=0 l=1 rev1=1 crypto rx=0x562d935bbe10 tx=0x7f3af0031570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=794d44d69a916e2c server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:53:10.815 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:10.808+0000 7f3ae6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-09-21T03:53:10.815 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:10.809+0000 7f3ae6ffd640 1 -- 172.21.15.39:0/3555511754 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b00005610 con 0x562d931fc070 2024-09-21T03:53:10.815 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:10.809+0000 7f3ae6ffd640 1 -- 172.21.15.39:0/3555511754 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8572+0+0 (secure 0 0 0) 0x7f3b00004680 con 0x562d931fc070 2024-09-21T03:53:10.815 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:10.809+0000 7f3b13f13740 1 -- 172.21.15.39:0/3555511754 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3af0003c30 con 0x562d931fc070 2024-09-21T03:53:10.815 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:10.809+0000 7f3ae6ffd640 1 -- 172.21.15.39:0/3555511754 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f3b00004880 con 0x562d931fc070 2024-09-21T03:53:10.815 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:10.809+0000 7f3b13f13740 10 client.17337 fetch_fsmap learned FSMap version 29 2024-09-21T03:53:10.815 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:10.809+0000 7f3b13f13740 1 -- 172.21.15.39:0/3555511754 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x562d9362c390 con 0x562d931fc070 2024-09-21T03:53:10.815 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:10.809+0000 7f3ae6ffd640 1 -- 172.21.15.39:0/3555511754 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3af0003c30 con 0x562d931fc070 2024-09-21T03:53:10.816 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:10.809+0000 7f3ae6ffd640 1 -- 172.21.15.39:0/3555511754 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f3b0000fc40 con 0x562d931fc070 2024-09-21T03:53:10.816 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:10.809+0000 7f3b13f13740 10 client.17337 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T03:53:10.816 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:10.809+0000 7f3b13f13740 10 client.17337 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-21T03:53:10.816 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:10.809+0000 7f3b13f13740 20 client.17337 populate_metadata read hostname 'smithi039' 2024-09-21T03:53:10.816 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:10.809+0000 7f3b13f13740 1 --2- 172.21.15.39:0/3555511754 >> [v2:172.21.15.161:6836/3451544597,v1:172.21.15.161:6837/3451544597] conn(0x562d9318fda0 0x562d9362ea90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:53:10.816 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:10.809+0000 7f3b13f13740 4 client.17337 mds_command: new command op to 24646 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T03:53:10.816 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:10.809+0000 7f3b13f13740 1 -- 172.21.15.39:0/3555511754 --> [v2:172.21.15.161:6836/3451544597,v1:172.21.15.161:6837/3451544597] -- command(tid 0: {"prefix": "damage ls"}) -- 0x562d935178a0 con 0x562d9318fda0 2024-09-21T03:53:10.816 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:10.810+0000 7f3b067fc640 1 --2- 172.21.15.39:0/3555511754 >> [v2:172.21.15.161:6836/3451544597,v1:172.21.15.161:6837/3451544597] conn(0x562d9318fda0 0x562d9362ea90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:53:10.816 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:10.810+0000 7f3b067fc640 1 --2- 172.21.15.39:0/3555511754 >> [v2:172.21.15.161:6836/3451544597,v1:172.21.15.161:6837/3451544597] conn(0x562d9318fda0 0x562d9362ea90 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=d8a64ddf39ef4fd3 server_cookie=a78dfe86984d1f70 in_seq=0 out_seq=0 2024-09-21T03:53:10.816 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:10.810+0000 7f3ae6ffd640 10 client.17337 ms_handle_connect on v2:172.21.15.161:6836/3451544597 2024-09-21T03:53:10.816 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:10.810+0000 7f3ae6ffd640 1 -- 172.21.15.39:0/3555511754 <== mds.3 v2:172.21.15.161:6836/3451544597 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x562d935178a0 con 0x562d9318fda0 2024-09-21T03:53:10.816 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:10.810+0000 7f3ae6ffd640 10 client.17337 handle_command_reply: tid=0 2024-09-21T03:53:10.816 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:10.810+0000 7f3b13f13740 1 client.17337 shutdown 2024-09-21T03:53:10.817 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:10.811+0000 7f3b13f13740 1 -- 172.21.15.39:0/3555511754 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562d931fc070 msgr2=0x562d935b3cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:53:10.817 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:10.811+0000 7f3b13f13740 1 --2- 172.21.15.39:0/3555511754 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562d931fc070 0x562d935b3cf0 secure :-1 s=READY pgs=1019 cs=0 l=1 rev1=1 crypto rx=0x562d935bbe10 tx=0x7f3af0031570 comp rx=0 tx=0).stop 2024-09-21T03:53:10.817 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:10.811+0000 7f3b13f13740 1 -- 172.21.15.39:0/3555511754 shutdown_connections 2024-09-21T03:53:10.817 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:10.811+0000 7f3b13f13740 1 --2- 172.21.15.39:0/3555511754 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562d931fc070 0x562d935b3cf0 unknown :-1 s=CLOSED pgs=1019 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:10.817 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:10.811+0000 7f3b13f13740 1 --2- 172.21.15.39:0/3555511754 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562d931db390 0x562d935b37b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:10.817 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:10.811+0000 7f3b13f13740 1 --2- 172.21.15.39:0/3555511754 >> [v2:172.21.15.161:6836/3451544597,v1:172.21.15.161:6837/3451544597] conn(0x562d9318fda0 0x562d9362ea90 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:10.817 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:10.811+0000 7f3ae6ffd640 0 client.17337 ms_handle_reset on v2:172.21.15.161:6836/3451544597 2024-09-21T03:53:10.817 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:10.811+0000 7f3b13f13740 1 --2- 172.21.15.39:0/3555511754 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562d931d31b0 0x562d935b72a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:10.817 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:10.811+0000 7f3b13f13740 1 -- 172.21.15.39:0/3555511754 >> 172.21.15.39:0/3555511754 conn(0x562d931f0460 msgr2=0x562d935b07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:53:10.817 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:10.811+0000 7f3b13f13740 1 -- 172.21.15.39:0/3555511754 shutdown_connections 2024-09-21T03:53:10.817 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:10.811+0000 7f3b13f13740 1 -- 172.21.15.39:0/3555511754 wait complete. 2024-09-21T03:53:10.817 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:10.811+0000 7f3b13f13740 20 client.17337 trim_cache size 0 max 16384 2024-09-21T03:53:10.818 INFO:teuthology.orchestra.run.smithi039.stdout:[] 2024-09-21T03:53:10.818 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:10.811+0000 7f3b0d99f640 1 -- 172.21.15.39:0/2339125589 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f3af40b6680 msgr2=0x7f3af40b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:53:10.818 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:10.811+0000 7f3b0d99f640 1 --2- 172.21.15.39:0/2339125589 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f3af40b6680 0x7f3af40b8b90 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f3af80e44d0 tx=0x7f3af8094040 comp rx=0 tx=0).stop 2024-09-21T03:53:10.818 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:10.811+0000 7f3b0d99f640 1 -- 172.21.15.39:0/2339125589 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3b08103b60 msgr2=0x7f3b0819a580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:53:10.818 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:10.811+0000 7f3b0d99f640 1 --2- 172.21.15.39:0/2339125589 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3b08103b60 0x7f3b0819a580 secure :-1 s=READY pgs=1015 cs=0 l=1 rev1=1 crypto rx=0x7f3af800ba50 tx=0x7f3af800bf10 comp rx=0 tx=0).stop 2024-09-21T03:53:10.819 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:10.812+0000 7f3b0d99f640 1 -- 172.21.15.39:0/2339125589 shutdown_connections 2024-09-21T03:53:10.819 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:10.812+0000 7f3b0d99f640 1 --2- 172.21.15.39:0/2339125589 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f3af40b6680 0x7f3af40b8b90 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:10.819 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:10.812+0000 7f3b0d99f640 1 --2- 172.21.15.39:0/2339125589 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3b08109df0 0x7f3b0819fdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:10.819 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:10.812+0000 7f3b0d99f640 1 --2- 172.21.15.39:0/2339125589 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3b081094a0 0x7f3b0819ab20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:10.819 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:10.812+0000 7f3b0d99f640 1 --2- 172.21.15.39:0/2339125589 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3b08103b60 0x7f3b0819a580 unknown :-1 s=CLOSED pgs=1015 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:10.819 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:10.812+0000 7f3b0d99f640 1 -- 172.21.15.39:0/2339125589 >> 172.21.15.39:0/2339125589 conn(0x7f3b0807b850 msgr2=0x7f3b081075e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:53:10.819 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:10.812+0000 7f3b0d99f640 1 -- 172.21.15.39:0/2339125589 shutdown_connections 2024-09-21T03:53:10.819 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:10.812+0000 7f3b0d99f640 1 -- 172.21.15.39:0/2339125589 wait complete. 2024-09-21T03:53:11.624 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:53:11 smithi039 ceph-mon[31211]: pgmap v539: 97 pgs: 97 active+clean; 10 MiB data, 520 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 44 KiB/s wr, 28 op/s 2024-09-21T03:53:11.624 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:53:11 smithi039 ceph-mon[31211]: osdmap e80: 12 total, 12 up, 12 in 2024-09-21T03:53:11.655 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 51116d5c-77ca-11ef-baf5-efdc52797490 -- ceph tell mds.1:2 damage ls 2024-09-21T03:53:11.718 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:53:11 smithi161 ceph-mon[34533]: pgmap v539: 97 pgs: 97 active+clean; 10 MiB data, 520 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 44 KiB/s wr, 28 op/s 2024-09-21T03:53:11.718 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:53:11 smithi161 ceph-mon[34533]: osdmap e80: 12 total, 12 up, 12 in 2024-09-21T03:53:11.724 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:53:11 smithi150 ceph-mon[34289]: pgmap v539: 97 pgs: 97 active+clean; 10 MiB data, 520 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 44 KiB/s wr, 28 op/s 2024-09-21T03:53:11.725 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:53:11 smithi150 ceph-mon[34289]: osdmap e80: 12 total, 12 up, 12 in 2024-09-21T03:53:11.916 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/51116d5c-77ca-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:53:12.394 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:12.378+0000 7f9c86c54640 1 -- 172.21.15.39:0/2557596481 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9c8010de20 msgr2=0x7f9c80110310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:53:12.395 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:12.378+0000 7f9c86c54640 1 --2- 172.21.15.39:0/2557596481 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9c8010de20 0x7f9c80110310 secure :-1 s=READY pgs=1020 cs=0 l=1 rev1=1 crypto rx=0x7f9c78009900 tx=0x7f9c7802a410 comp rx=0 tx=0).stop 2024-09-21T03:53:12.395 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:12.378+0000 7f9c86c54640 1 -- 172.21.15.39:0/2557596481 shutdown_connections 2024-09-21T03:53:12.395 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:12.378+0000 7f9c86c54640 1 --2- 172.21.15.39:0/2557596481 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9c8010de20 0x7f9c80110310 unknown :-1 s=CLOSED pgs=1020 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:12.395 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:12.378+0000 7f9c86c54640 1 --2- 172.21.15.39:0/2557596481 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9c80105340 0x7f9c8010d890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:12.395 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:12.378+0000 7f9c86c54640 1 --2- 172.21.15.39:0/2557596481 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9c801049f0 0x7f9c80104e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:12.395 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:12.378+0000 7f9c86c54640 1 -- 172.21.15.39:0/2557596481 >> 172.21.15.39:0/2557596481 conn(0x7f9c80079730 msgr2=0x7f9c80079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:53:12.395 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:12.378+0000 7f9c86c54640 1 -- 172.21.15.39:0/2557596481 shutdown_connections 2024-09-21T03:53:12.395 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:12.378+0000 7f9c86c54640 1 -- 172.21.15.39:0/2557596481 wait complete. 2024-09-21T03:53:12.395 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:12.379+0000 7f9c86c54640 1 Processor -- start 2024-09-21T03:53:12.395 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:12.379+0000 7f9c86c54640 1 -- start start 2024-09-21T03:53:12.395 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:12.380+0000 7f9c86c54640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9c801049f0 0x7f9c8019a3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:53:12.395 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:12.380+0000 7f9c86c54640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9c80105340 0x7f9c8019a990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:53:12.396 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:12.380+0000 7f9c86c54640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9c8010de20 0x7f9c8019fc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:53:12.396 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:12.380+0000 7f9c86c54640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f9c80113a80 con 0x7f9c8010de20 2024-09-21T03:53:12.396 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:12.380+0000 7f9c86c54640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f9c80113900 con 0x7f9c801049f0 2024-09-21T03:53:12.396 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:12.380+0000 7f9c86c54640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f9c80113c00 con 0x7f9c80105340 2024-09-21T03:53:12.396 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:12.380+0000 7f9c851f1640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9c8010de20 0x7f9c8019fc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:53:12.396 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:12.380+0000 7f9c851f1640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9c8010de20 0x7f9c8019fc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:48646/0 (socket says 172.21.15.39:48646) 2024-09-21T03:53:12.396 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:12.380+0000 7f9c851f1640 1 -- 172.21.15.39:0/3874215587 learned_addr learned my addr 172.21.15.39:0/3874215587 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:53:12.396 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:12.380+0000 7f9c849f0640 1 --2- 172.21.15.39:0/3874215587 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9c801049f0 0x7f9c8019a3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:53:12.396 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:12.380+0000 7f9c7ffff640 1 --2- 172.21.15.39:0/3874215587 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9c80105340 0x7f9c8019a990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:53:12.396 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:12.380+0000 7f9c851f1640 1 -- 172.21.15.39:0/3874215587 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9c80105340 msgr2=0x7f9c8019a990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:53:12.396 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:12.380+0000 7f9c851f1640 1 --2- 172.21.15.39:0/3874215587 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9c80105340 0x7f9c8019a990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:12.396 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:12.380+0000 7f9c851f1640 1 -- 172.21.15.39:0/3874215587 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9c801049f0 msgr2=0x7f9c8019a3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:53:12.396 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:12.380+0000 7f9c851f1640 1 --2- 172.21.15.39:0/3874215587 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9c801049f0 0x7f9c8019a3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:12.396 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:12.380+0000 7f9c851f1640 1 -- 172.21.15.39:0/3874215587 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9c801a0360 con 0x7f9c8010de20 2024-09-21T03:53:12.397 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:12.380+0000 7f9c851f1640 1 --2- 172.21.15.39:0/3874215587 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9c8010de20 0x7f9c8019fc30 secure :-1 s=READY pgs=1021 cs=0 l=1 rev1=1 crypto rx=0x7f9c78004770 tx=0x7f9c78004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:53:12.397 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:12.380+0000 7f9c7dffb640 1 -- 172.21.15.39:0/3874215587 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c7803f030 con 0x7f9c8010de20 2024-09-21T03:53:12.397 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:12.380+0000 7f9c7dffb640 1 -- 172.21.15.39:0/3874215587 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9c78003aa0 con 0x7f9c8010de20 2024-09-21T03:53:12.397 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:12.380+0000 7f9c86c54640 1 -- 172.21.15.39:0/3874215587 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9c801a05f0 con 0x7f9c8010de20 2024-09-21T03:53:12.397 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:12.381+0000 7f9c7dffb640 1 -- 172.21.15.39:0/3874215587 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c78035070 con 0x7f9c8010de20 2024-09-21T03:53:12.397 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:12.381+0000 7f9c86c54640 1 -- 172.21.15.39:0/3874215587 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9c801a6db0 con 0x7f9c8010de20 2024-09-21T03:53:12.397 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:12.383+0000 7f9c8d1c8740 1 Processor -- start 2024-09-21T03:53:12.397 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:12.383+0000 7f9c7dffb640 1 -- 172.21.15.39:0/3874215587 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 27) ==== 154643+0+0 (secure 0 0 0) 0x7f9c78042030 con 0x7f9c8010de20 2024-09-21T03:53:12.397 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:12.383+0000 7f9c8d1c8740 1 -- start start 2024-09-21T03:53:12.397 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:12.384+0000 7f9c8d1c8740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55aa9d574070 0x55aa9d91b6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:53:12.397 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:12.384+0000 7f9c8d1c8740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55aa9d54b1b0 0x55aa9d91bbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:53:12.397 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:12.384+0000 7f9c8d1c8740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55aa9d553390 0x55aa9d91c120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:53:12.397 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:12.384+0000 7f9c8d1c8740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x55aa9d891ea0 con 0x55aa9d54b1b0 2024-09-21T03:53:12.397 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:12.384+0000 7f9c8d1c8740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x55aa9d8812d0 con 0x55aa9d553390 2024-09-21T03:53:12.398 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:12.384+0000 7f9c8d1c8740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55aa9d8850e0 con 0x55aa9d574070 2024-09-21T03:53:12.398 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:12.384+0000 7f9c849f0640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55aa9d54b1b0 0x55aa9d91bbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:53:12.398 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:12.384+0000 7f9c849f0640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55aa9d54b1b0 0x55aa9d91bbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:48654/0 (socket says 172.21.15.39:48654) 2024-09-21T03:53:12.398 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:12.384+0000 7f9c849f0640 1 -- 172.21.15.39:0/2254094050 learned_addr learned my addr 172.21.15.39:0/2254094050 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:53:12.398 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:12.384+0000 7f9c7ffff640 1 --2- 172.21.15.39:0/2254094050 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55aa9d574070 0x55aa9d91b6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:53:12.398 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:12.384+0000 7f9c7ffff640 1 --2- 172.21.15.39:0/2254094050 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55aa9d553390 0x55aa9d91c120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:53:12.398 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:12.384+0000 7f9c7ffff640 1 -- 172.21.15.39:0/2254094050 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55aa9d553390 msgr2=0x55aa9d91c120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:53:12.398 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:12.384+0000 7f9c7ffff640 1 --2- 172.21.15.39:0/2254094050 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55aa9d553390 0x55aa9d91c120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:12.398 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:12.384+0000 7f9c7ffff640 1 -- 172.21.15.39:0/2254094050 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55aa9d54b1b0 msgr2=0x55aa9d91bbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:53:12.398 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:12.384+0000 7f9c7ffff640 1 --2- 172.21.15.39:0/2254094050 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55aa9d54b1b0 0x55aa9d91bbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:12.398 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:12.384+0000 7f9c7ffff640 1 -- 172.21.15.39:0/2254094050 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55aa9d5df650 con 0x55aa9d574070 2024-09-21T03:53:12.398 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:12.384+0000 7f9c849f0640 1 --2- 172.21.15.39:0/2254094050 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55aa9d54b1b0 0x55aa9d91bbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:53:12.399 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:12.385+0000 7f9c7dffb640 1 --2- 172.21.15.39:0/3874215587 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f9c6c0b6700 0x7f9c6c0b8c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:53:12.399 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:12.385+0000 7f9c7ffff640 1 --2- 172.21.15.39:0/2254094050 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55aa9d574070 0x55aa9d91b6a0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f9c8006aaa0 tx=0x7f9c70009710 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7cc24b8a7cd37dc0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:53:12.399 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:12.385+0000 7f9c849f0640 1 --2- 172.21.15.39:0/3874215587 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f9c6c0b6700 0x7f9c6c0b8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:53:12.399 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:12.385+0000 7f9c76ffd640 1 -- 172.21.15.39:0/2254094050 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c70010740 con 0x55aa9d574070 2024-09-21T03:53:12.399 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:12.385+0000 7f9c7dffb640 1 -- 172.21.15.39:0/3874215587 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8572+0+0 (secure 0 0 0) 0x7f9c78107c50 con 0x7f9c8010de20 2024-09-21T03:53:12.399 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:12.385+0000 7f9c76ffd640 1 -- 172.21.15.39:0/2254094050 <== mon.1 v2:172.21.15.161:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9c70010d90 con 0x55aa9d574070 2024-09-21T03:53:12.399 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:12.385+0000 7f9c76ffd640 1 -- 172.21.15.39:0/2254094050 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c70011c80 con 0x55aa9d574070 2024-09-21T03:53:12.399 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:12.386+0000 7f9c8d1c8740 1 -- 172.21.15.39:0/2254094050 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55aa9d574070 msgr2=0x55aa9d91b6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:53:12.399 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:12.386+0000 7f9c8d1c8740 1 --2- 172.21.15.39:0/2254094050 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55aa9d574070 0x55aa9d91b6a0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f9c8006aaa0 tx=0x7f9c70009710 comp rx=0 tx=0).stop 2024-09-21T03:53:12.399 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:12.386+0000 7f9c849f0640 1 --2- 172.21.15.39:0/3874215587 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f9c6c0b6700 0x7f9c6c0b8c10 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x55aa9d91a320 tx=0x7f9c68005f90 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:53:12.399 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:12.386+0000 7f9c8d1c8740 1 -- 172.21.15.39:0/2254094050 shutdown_connections 2024-09-21T03:53:12.399 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:12.386+0000 7f9c8d1c8740 1 --2- 172.21.15.39:0/2254094050 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55aa9d553390 0x55aa9d91c120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:12.399 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:12.386+0000 7f9c8d1c8740 1 --2- 172.21.15.39:0/2254094050 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55aa9d54b1b0 0x55aa9d91bbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:12.400 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:12.386+0000 7f9c8d1c8740 1 --2- 172.21.15.39:0/2254094050 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55aa9d574070 0x55aa9d91b6a0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:12.400 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:12.386+0000 7f9c8d1c8740 1 -- 172.21.15.39:0/2254094050 >> 172.21.15.39:0/2254094050 conn(0x55aa9d568460 msgr2=0x55aa9d919590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:53:12.400 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:12.386+0000 7f9c8d1c8740 1 -- 172.21.15.39:0/2254094050 shutdown_connections 2024-09-21T03:53:12.400 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:12.386+0000 7f9c8d1c8740 1 -- 172.21.15.39:0/2254094050 wait complete. 2024-09-21T03:53:12.400 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:12.386+0000 7f9c8d1c8740 1 -- start start 2024-09-21T03:53:12.400 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:12.386+0000 7f9c8d1c8740 1 Processor -- start 2024-09-21T03:53:12.400 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:12.387+0000 7f9c8d1c8740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55aa9d54b1b0 0x55aa9d926d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:53:12.400 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:12.387+0000 7f9c8d1c8740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55aa9d553390 0x55aa9d927280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:53:12.400 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:12.387+0000 7f9c8d1c8740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55aa9d574070 0x55aa9d927800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:53:12.400 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:12.387+0000 7f9c8d1c8740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x55aa9d6e2700 con 0x55aa9d54b1b0 2024-09-21T03:53:12.400 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:12.387+0000 7f9c8d1c8740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x55aa9d81c160 con 0x55aa9d553390 2024-09-21T03:53:12.400 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:12.387+0000 7f9c8d1c8740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55aa9d929e80 con 0x55aa9d574070 2024-09-21T03:53:12.400 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:12.387+0000 7f9c8d1c8740 10 client.0 _ll_register_callbacks cb 0x55aa9d8359e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:53:12.400 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:12.387+0000 7f9c849f0640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55aa9d54b1b0 0x55aa9d926d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:53:12.401 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:12.387+0000 7f9c849f0640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55aa9d54b1b0 0x55aa9d926d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:48656/0 (socket says 172.21.15.39:48656) 2024-09-21T03:53:12.401 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:12.387+0000 7f9c849f0640 1 -- 172.21.15.39:0/3380305713 learned_addr learned my addr 172.21.15.39:0/3380305713 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:53:12.401 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:12.387+0000 7f9c851f1640 1 --2- 172.21.15.39:0/3380305713 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55aa9d574070 0x55aa9d927800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:53:12.401 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:12.387+0000 7f9c7ffff640 1 --2- 172.21.15.39:0/3380305713 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55aa9d553390 0x55aa9d927280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:53:12.401 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:12.387+0000 7f9c849f0640 1 -- 172.21.15.39:0/3380305713 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55aa9d574070 msgr2=0x55aa9d927800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:53:12.401 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:12.387+0000 7f9c849f0640 1 --2- 172.21.15.39:0/3380305713 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55aa9d574070 0x55aa9d927800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:12.401 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:12.387+0000 7f9c849f0640 1 -- 172.21.15.39:0/3380305713 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55aa9d553390 msgr2=0x55aa9d927280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:53:12.401 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:12.387+0000 7f9c849f0640 1 --2- 172.21.15.39:0/3380305713 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55aa9d553390 0x55aa9d927280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:12.401 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:12.387+0000 7f9c849f0640 1 -- 172.21.15.39:0/3380305713 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55aa9d849c90 con 0x55aa9d54b1b0 2024-09-21T03:53:12.401 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:12.388+0000 7f9c849f0640 1 --2- 172.21.15.39:0/3380305713 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55aa9d54b1b0 0x55aa9d926d40 secure :-1 s=READY pgs=1022 cs=0 l=1 rev1=1 crypto rx=0x7f9c68020a10 tx=0x7f9c68004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c363043425ed97 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:53:12.401 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:12.388+0000 7f9c5ffff640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-09-21T03:53:12.401 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:12.388+0000 7f9c5ffff640 1 -- 172.21.15.39:0/3380305713 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c6802d060 con 0x55aa9d54b1b0 2024-09-21T03:53:12.401 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:12.388+0000 7f9c5ffff640 1 -- 172.21.15.39:0/3380305713 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8572+0+0 (secure 0 0 0) 0x7f9c68018030 con 0x55aa9d54b1b0 2024-09-21T03:53:12.402 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:12.388+0000 7f9c8d1c8740 1 -- 172.21.15.39:0/3380305713 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9c70010d90 con 0x55aa9d54b1b0 2024-09-21T03:53:12.402 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:12.388+0000 7f9c5ffff640 1 -- 172.21.15.39:0/3380305713 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f9c68009bf0 con 0x55aa9d54b1b0 2024-09-21T03:53:12.402 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:12.388+0000 7f9c8d1c8740 10 client.17349 fetch_fsmap learned FSMap version 29 2024-09-21T03:53:12.402 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:12.388+0000 7f9c8d1c8740 1 -- 172.21.15.39:0/3380305713 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55aa9d939fc0 con 0x55aa9d54b1b0 2024-09-21T03:53:12.402 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:12.388+0000 7f9c5ffff640 1 -- 172.21.15.39:0/3380305713 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9c70010d90 con 0x55aa9d54b1b0 2024-09-21T03:53:12.402 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:12.388+0000 7f9c5ffff640 1 -- 172.21.15.39:0/3380305713 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f9c6801dbf0 con 0x55aa9d54b1b0 2024-09-21T03:53:12.402 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:12.388+0000 7f9c8d1c8740 10 client.17349 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T03:53:12.402 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:12.388+0000 7f9c8d1c8740 10 client.17349 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-21T03:53:12.402 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:12.388+0000 7f9c8d1c8740 20 client.17349 populate_metadata read hostname 'smithi039' 2024-09-21T03:53:12.402 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:12.388+0000 7f9c8d1c8740 1 --2- 172.21.15.39:0/3380305713 >> [v2:172.21.15.150:6836/4017043144,v1:172.21.15.150:6837/4017043144] conn(0x55aa9d5329f0 0x55aa9d93d570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:53:12.402 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:12.388+0000 7f9c8d1c8740 4 client.17349 mds_command: new command op to 24659 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:53:12.402 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:12.388+0000 7f9c8d1c8740 1 -- 172.21.15.39:0/3380305713 --> [v2:172.21.15.150:6836/4017043144,v1:172.21.15.150:6837/4017043144] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55aa9d893df0 con 0x55aa9d5329f0 2024-09-21T03:53:12.402 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:12.389+0000 7f9c7ffff640 1 --2- 172.21.15.39:0/3380305713 >> [v2:172.21.15.150:6836/4017043144,v1:172.21.15.150:6837/4017043144] conn(0x55aa9d5329f0 0x55aa9d93d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:53:12.403 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:12.389+0000 7f9c5ffff640 10 client.17349 ms_handle_connect on v2:172.21.15.150:6836/4017043144 2024-09-21T03:53:12.403 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:12.389+0000 7f9c7ffff640 1 --2- 172.21.15.39:0/3380305713 >> [v2:172.21.15.150:6836/4017043144,v1:172.21.15.150:6837/4017043144] conn(0x55aa9d5329f0 0x55aa9d93d570 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=f8c04700cbedda95 server_cookie=93efafca6a3fd353 in_seq=0 out_seq=0 2024-09-21T03:53:12.403 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:12.390+0000 7f9c5ffff640 1 -- 172.21.15.39:0/3380305713 <== mds.2 v2:172.21.15.150:6836/4017043144 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55aa9d893df0 con 0x55aa9d5329f0 2024-09-21T03:53:12.403 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:12.390+0000 7f9c5ffff640 10 client.17349 handle_command_reply: tid=0 2024-09-21T03:53:12.403 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:12.390+0000 7f9c8d1c8740 1 client.17349 shutdown 2024-09-21T03:53:12.403 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:12.390+0000 7f9c8d1c8740 1 -- 172.21.15.39:0/3380305713 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55aa9d54b1b0 msgr2=0x55aa9d926d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:53:12.403 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:12.390+0000 7f9c8d1c8740 1 --2- 172.21.15.39:0/3380305713 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55aa9d54b1b0 0x55aa9d926d40 secure :-1 s=READY pgs=1022 cs=0 l=1 rev1=1 crypto rx=0x7f9c68020a10 tx=0x7f9c68004730 comp rx=0 tx=0).stop 2024-09-21T03:53:12.403 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:12.390+0000 7f9c8d1c8740 1 -- 172.21.15.39:0/3380305713 shutdown_connections 2024-09-21T03:53:12.403 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:12.390+0000 7f9c8d1c8740 1 --2- 172.21.15.39:0/3380305713 >> [v2:172.21.15.150:6836/4017043144,v1:172.21.15.150:6837/4017043144] conn(0x55aa9d5329f0 0x55aa9d93d570 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:12.403 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:12.390+0000 7f9c8d1c8740 1 --2- 172.21.15.39:0/3380305713 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55aa9d574070 0x55aa9d927800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:12.403 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:12.390+0000 7f9c8d1c8740 1 --2- 172.21.15.39:0/3380305713 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55aa9d553390 0x55aa9d927280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:12.404 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:12.390+0000 7f9c8d1c8740 1 --2- 172.21.15.39:0/3380305713 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55aa9d54b1b0 0x55aa9d926d40 unknown :-1 s=CLOSED pgs=1022 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:12.404 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:12.390+0000 7f9c5ffff640 0 client.17349 ms_handle_reset on v2:172.21.15.150:6836/4017043144 2024-09-21T03:53:12.404 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:12.390+0000 7f9c8d1c8740 1 -- 172.21.15.39:0/3380305713 >> 172.21.15.39:0/3380305713 conn(0x55aa9d568460 msgr2=0x55aa9d91e830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:53:12.404 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:12.390+0000 7f9c8d1c8740 1 -- 172.21.15.39:0/3380305713 shutdown_connections 2024-09-21T03:53:12.404 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:12.390+0000 7f9c8d1c8740 1 -- 172.21.15.39:0/3380305713 wait complete. 2024-09-21T03:53:12.404 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:12.390+0000 7f9c8d1c8740 20 client.17349 trim_cache size 0 max 16384 2024-09-21T03:53:12.405 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:12.404+0000 7f9c8d1c8740 1 Processor -- start 2024-09-21T03:53:12.405 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:12.404+0000 7f9c8d1c8740 1 -- start start 2024-09-21T03:53:12.407 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:12.404+0000 7f9c8d1c8740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55aa9d54b1b0 0x55aa9d91d090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:53:12.407 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:12.404+0000 7f9c8d1c8740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55aa9d553390 0x55aa9d928970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:53:12.407 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:12.404+0000 7f9c8d1c8740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55aa9d574070 0x55aa9d918bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:53:12.407 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:12.404+0000 7f9c8d1c8740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f9c801ad600 con 0x55aa9d54b1b0 2024-09-21T03:53:12.407 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:12.404+0000 7f9c8d1c8740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f9c801af140 con 0x55aa9d553390 2024-09-21T03:53:12.407 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:12.404+0000 7f9c8d1c8740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f9c801adcd0 con 0x55aa9d574070 2024-09-21T03:53:12.407 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:12.404+0000 7f9c849f0640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55aa9d54b1b0 0x55aa9d91d090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:53:12.407 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:12.404+0000 7f9c849f0640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55aa9d54b1b0 0x55aa9d91d090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:48660/0 (socket says 172.21.15.39:48660) 2024-09-21T03:53:12.407 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:12.404+0000 7f9c849f0640 1 -- 172.21.15.39:0/1740754557 learned_addr learned my addr 172.21.15.39:0/1740754557 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:53:12.407 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:12.404+0000 7f9c7ffff640 1 --2- 172.21.15.39:0/1740754557 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55aa9d553390 0x55aa9d928970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:53:12.408 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:12.405+0000 7f9c851f1640 1 --2- 172.21.15.39:0/1740754557 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55aa9d574070 0x55aa9d918bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:53:12.408 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:12.405+0000 7f9c849f0640 1 -- 172.21.15.39:0/1740754557 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55aa9d574070 msgr2=0x55aa9d918bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:53:12.408 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:12.405+0000 7f9c849f0640 1 --2- 172.21.15.39:0/1740754557 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55aa9d574070 0x55aa9d918bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:12.408 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:12.405+0000 7f9c849f0640 1 -- 172.21.15.39:0/1740754557 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55aa9d553390 msgr2=0x55aa9d928970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:53:12.408 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:12.405+0000 7f9c849f0640 1 --2- 172.21.15.39:0/1740754557 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55aa9d553390 0x55aa9d928970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:12.408 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:12.405+0000 7f9c849f0640 1 -- 172.21.15.39:0/1740754557 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55aa9d91e500 con 0x55aa9d54b1b0 2024-09-21T03:53:12.408 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:12.405+0000 7f9c849f0640 1 --2- 172.21.15.39:0/1740754557 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55aa9d54b1b0 0x55aa9d91d090 secure :-1 s=READY pgs=1023 cs=0 l=1 rev1=1 crypto rx=0x7f9c68011910 tx=0x7f9c68005910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=76e28b18caabc26b server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:53:12.408 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:12.405+0000 7f9c767fc640 1 -- 172.21.15.39:0/1740754557 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c68011d50 con 0x55aa9d54b1b0 2024-09-21T03:53:12.408 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:12.405+0000 7f9c767fc640 1 -- 172.21.15.39:0/1740754557 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9c6800fd00 con 0x55aa9d54b1b0 2024-09-21T03:53:12.408 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:12.405+0000 7f9c767fc640 1 -- 172.21.15.39:0/1740754557 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c68004800 con 0x55aa9d54b1b0 2024-09-21T03:53:12.411 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:12.406+0000 7f9c8d1c8740 1 -- 172.21.15.39:0/1740754557 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55aa9d54b1b0 msgr2=0x55aa9d91d090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:53:12.411 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:12.406+0000 7f9c8d1c8740 1 --2- 172.21.15.39:0/1740754557 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55aa9d54b1b0 0x55aa9d91d090 secure :-1 s=READY pgs=1023 cs=0 l=1 rev1=1 crypto rx=0x7f9c68011910 tx=0x7f9c68005910 comp rx=0 tx=0).stop 2024-09-21T03:53:12.411 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:12.406+0000 7f9c8d1c8740 1 -- 172.21.15.39:0/1740754557 shutdown_connections 2024-09-21T03:53:12.411 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:12.406+0000 7f9c8d1c8740 1 --2- 172.21.15.39:0/1740754557 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55aa9d574070 0x55aa9d918bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:12.411 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:12.406+0000 7f9c8d1c8740 1 --2- 172.21.15.39:0/1740754557 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55aa9d553390 0x55aa9d928970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:12.411 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:12.406+0000 7f9c8d1c8740 1 --2- 172.21.15.39:0/1740754557 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55aa9d54b1b0 0x55aa9d91d090 unknown :-1 s=CLOSED pgs=1023 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:12.411 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:12.406+0000 7f9c8d1c8740 1 -- 172.21.15.39:0/1740754557 >> 172.21.15.39:0/1740754557 conn(0x55aa9d568460 msgr2=0x55aa9d928260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:53:12.412 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:12.406+0000 7f9c8d1c8740 1 -- 172.21.15.39:0/1740754557 shutdown_connections 2024-09-21T03:53:12.412 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:12.406+0000 7f9c8d1c8740 1 -- 172.21.15.39:0/1740754557 wait complete. 2024-09-21T03:53:12.412 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:12.406+0000 7f9c8d1c8740 1 -- start start 2024-09-21T03:53:12.412 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:12.407+0000 7f9c8d1c8740 1 Processor -- start 2024-09-21T03:53:12.412 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:12.407+0000 7f9c8d1c8740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55aa9d54b1b0 0x55aa9d92b180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:53:12.412 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:12.407+0000 7f9c8d1c8740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55aa9d553390 0x55aa9d92b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:53:12.412 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:12.407+0000 7f9c8d1c8740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55aa9d574070 0x55aa9d92bc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:53:12.412 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:12.407+0000 7f9c8d1c8740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f9c801aea70 con 0x55aa9d574070 2024-09-21T03:53:12.412 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:12.407+0000 7f9c8d1c8740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f9c801ae3a0 con 0x55aa9d553390 2024-09-21T03:53:12.412 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:12.407+0000 7f9c8d1c8740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f9c801ab600 con 0x55aa9d54b1b0 2024-09-21T03:53:12.412 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:12.407+0000 7f9c8d1c8740 10 client.0 _ll_register_callbacks cb 0x7f9c68009530 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:53:12.412 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:12.407+0000 7f9c851f1640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55aa9d574070 0x55aa9d92bc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:53:12.412 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:12.407+0000 7f9c851f1640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55aa9d574070 0x55aa9d92bc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:48666/0 (socket says 172.21.15.39:48666) 2024-09-21T03:53:12.412 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:12.407+0000 7f9c851f1640 1 -- 172.21.15.39:0/2748779047 learned_addr learned my addr 172.21.15.39:0/2748779047 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:53:12.412 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:12.407+0000 7f9c849f0640 1 --2- 172.21.15.39:0/2748779047 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55aa9d54b1b0 0x55aa9d92b180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:53:12.412 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:12.407+0000 7f9c7ffff640 1 --2- 172.21.15.39:0/2748779047 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55aa9d553390 0x55aa9d92b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:53:12.413 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:12.408+0000 7f9c851f1640 1 -- 172.21.15.39:0/2748779047 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55aa9d54b1b0 msgr2=0x55aa9d92b180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:53:12.413 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:12.408+0000 7f9c851f1640 1 --2- 172.21.15.39:0/2748779047 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55aa9d54b1b0 0x55aa9d92b180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:12.413 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:12.408+0000 7f9c851f1640 1 -- 172.21.15.39:0/2748779047 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55aa9d553390 msgr2=0x55aa9d92b6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:53:12.413 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:12.408+0000 7f9c851f1640 1 --2- 172.21.15.39:0/2748779047 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55aa9d553390 0x55aa9d92b6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:12.413 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:12.408+0000 7f9c851f1640 1 -- 172.21.15.39:0/2748779047 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55aa9d9a6fe0 con 0x55aa9d574070 2024-09-21T03:53:12.413 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:12.408+0000 7f9c849f0640 1 --2- 172.21.15.39:0/2748779047 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55aa9d54b1b0 0x55aa9d92b180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:53:12.413 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:12.408+0000 7f9c851f1640 1 --2- 172.21.15.39:0/2748779047 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55aa9d574070 0x55aa9d92bc00 secure :-1 s=READY pgs=1024 cs=0 l=1 rev1=1 crypto rx=0x7f9c781053d0 tx=0x7f9c780c9a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aecc9d3dbb9eacc4 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:53:12.413 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:12.408+0000 7f9c5ffff640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-09-21T03:53:12.413 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:12.408+0000 7f9c5ffff640 1 -- 172.21.15.39:0/2748779047 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c78049040 con 0x55aa9d574070 2024-09-21T03:53:12.413 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:12.408+0000 7f9c5ffff640 1 -- 172.21.15.39:0/2748779047 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8572+0+0 (secure 0 0 0) 0x7f9c780d2110 con 0x55aa9d574070 2024-09-21T03:53:12.413 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:12.408+0000 7f9c8d1c8740 1 -- 172.21.15.39:0/2748779047 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9c6800fd00 con 0x55aa9d574070 2024-09-21T03:53:12.413 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:12.408+0000 7f9c5ffff640 1 -- 172.21.15.39:0/2748779047 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f9c780c9040 con 0x55aa9d574070 2024-09-21T03:53:12.413 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:12.408+0000 7f9c8d1c8740 10 client.17355 fetch_fsmap learned FSMap version 29 2024-09-21T03:53:12.413 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:12.408+0000 7f9c8d1c8740 1 -- 172.21.15.39:0/2748779047 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55aa9d9a82c0 con 0x55aa9d574070 2024-09-21T03:53:12.413 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:12.408+0000 7f9c5ffff640 1 -- 172.21.15.39:0/2748779047 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9c6800fd00 con 0x55aa9d574070 2024-09-21T03:53:12.414 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:12.408+0000 7f9c5ffff640 1 -- 172.21.15.39:0/2748779047 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f9c780d7000 con 0x55aa9d574070 2024-09-21T03:53:12.414 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:12.409+0000 7f9c8d1c8740 10 client.17355 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T03:53:12.414 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:12.409+0000 7f9c8d1c8740 10 client.17355 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-21T03:53:12.414 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:12.409+0000 7f9c8d1c8740 20 client.17355 populate_metadata read hostname 'smithi039' 2024-09-21T03:53:12.414 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:12.409+0000 7f9c8d1c8740 1 --2- 172.21.15.39:0/2748779047 >> [v2:172.21.15.150:6836/4017043144,v1:172.21.15.150:6837/4017043144] conn(0x55aa9d507da0 0x55aa9d9aaaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:53:12.414 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:12.409+0000 7f9c8d1c8740 4 client.17355 mds_command: new command op to 24659 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T03:53:12.414 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:12.409+0000 7f9c8d1c8740 1 -- 172.21.15.39:0/2748779047 --> [v2:172.21.15.150:6836/4017043144,v1:172.21.15.150:6837/4017043144] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55aa9d88f8a0 con 0x55aa9d507da0 2024-09-21T03:53:12.414 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:12.409+0000 7f9c849f0640 1 --2- 172.21.15.39:0/2748779047 >> [v2:172.21.15.150:6836/4017043144,v1:172.21.15.150:6837/4017043144] conn(0x55aa9d507da0 0x55aa9d9aaaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:53:12.414 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:12.409+0000 7f9c5ffff640 10 client.17355 ms_handle_connect on v2:172.21.15.150:6836/4017043144 2024-09-21T03:53:12.414 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:12.409+0000 7f9c849f0640 1 --2- 172.21.15.39:0/2748779047 >> [v2:172.21.15.150:6836/4017043144,v1:172.21.15.150:6837/4017043144] conn(0x55aa9d507da0 0x55aa9d9aaaa0 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=a8f86ccf77805cfe server_cookie=7209eead0c03423a in_seq=0 out_seq=0 2024-09-21T03:53:12.414 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:12.410+0000 7f9c5ffff640 1 -- 172.21.15.39:0/2748779047 <== mds.2 v2:172.21.15.150:6836/4017043144 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55aa9d88f8a0 con 0x55aa9d507da0 2024-09-21T03:53:12.414 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:12.410+0000 7f9c5ffff640 10 client.17355 handle_command_reply: tid=0 2024-09-21T03:53:12.414 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:12.410+0000 7f9c8d1c8740 1 client.17355 shutdown 2024-09-21T03:53:12.415 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:12.410+0000 7f9c8d1c8740 1 -- 172.21.15.39:0/2748779047 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55aa9d574070 msgr2=0x55aa9d92bc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:53:12.415 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:12.410+0000 7f9c8d1c8740 1 --2- 172.21.15.39:0/2748779047 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55aa9d574070 0x55aa9d92bc00 secure :-1 s=READY pgs=1024 cs=0 l=1 rev1=1 crypto rx=0x7f9c781053d0 tx=0x7f9c780c9a40 comp rx=0 tx=0).stop 2024-09-21T03:53:12.415 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:12.410+0000 7f9c8d1c8740 1 -- 172.21.15.39:0/2748779047 shutdown_connections 2024-09-21T03:53:12.415 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:12.410+0000 7f9c8d1c8740 1 --2- 172.21.15.39:0/2748779047 >> [v2:172.21.15.150:6836/4017043144,v1:172.21.15.150:6837/4017043144] conn(0x55aa9d507da0 0x55aa9d9aaaa0 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:12.416 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:12.410+0000 7f9c8d1c8740 1 --2- 172.21.15.39:0/2748779047 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55aa9d574070 0x55aa9d92bc00 unknown :-1 s=CLOSED pgs=1024 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:12.416 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:12.410+0000 7f9c8d1c8740 1 --2- 172.21.15.39:0/2748779047 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55aa9d553390 0x55aa9d92b6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:12.416 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:12.410+0000 7f9c8d1c8740 1 --2- 172.21.15.39:0/2748779047 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55aa9d54b1b0 0x55aa9d92b180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:12.416 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:12.410+0000 7f9c8d1c8740 1 -- 172.21.15.39:0/2748779047 >> 172.21.15.39:0/2748779047 conn(0x55aa9d568460 msgr2=0x55aa9d91ce60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:53:12.416 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:12.410+0000 7f9c5ffff640 0 client.17355 ms_handle_reset on v2:172.21.15.150:6836/4017043144 2024-09-21T03:53:12.416 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:12.410+0000 7f9c8d1c8740 1 -- 172.21.15.39:0/2748779047 shutdown_connections 2024-09-21T03:53:12.416 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:12.410+0000 7f9c8d1c8740 1 -- 172.21.15.39:0/2748779047 wait complete. 2024-09-21T03:53:12.416 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:12.410+0000 7f9c8d1c8740 20 client.17355 trim_cache size 0 max 16384 2024-09-21T03:53:12.416 INFO:teuthology.orchestra.run.smithi039.stdout:[] 2024-09-21T03:53:12.417 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:12.411+0000 7f9c86c54640 1 -- 172.21.15.39:0/3874215587 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f9c6c0b6700 msgr2=0x7f9c6c0b8c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:53:12.417 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:12.411+0000 7f9c86c54640 1 --2- 172.21.15.39:0/3874215587 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f9c6c0b6700 0x7f9c6c0b8c10 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x55aa9d91a320 tx=0x7f9c68005f90 comp rx=0 tx=0).stop 2024-09-21T03:53:12.417 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:12.411+0000 7f9c86c54640 1 -- 172.21.15.39:0/3874215587 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9c8010de20 msgr2=0x7f9c8019fc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:53:12.417 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:12.411+0000 7f9c86c54640 1 --2- 172.21.15.39:0/3874215587 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9c8010de20 0x7f9c8019fc30 secure :-1 s=READY pgs=1021 cs=0 l=1 rev1=1 crypto rx=0x7f9c78004770 tx=0x7f9c78004270 comp rx=0 tx=0).stop 2024-09-21T03:53:12.417 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:12.411+0000 7f9c86c54640 1 -- 172.21.15.39:0/3874215587 shutdown_connections 2024-09-21T03:53:12.417 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:12.411+0000 7f9c86c54640 1 --2- 172.21.15.39:0/3874215587 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f9c6c0b6700 0x7f9c6c0b8c10 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:12.417 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:12.411+0000 7f9c86c54640 1 --2- 172.21.15.39:0/3874215587 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9c8010de20 0x7f9c8019fc30 unknown :-1 s=CLOSED pgs=1021 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:12.417 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:12.411+0000 7f9c86c54640 1 --2- 172.21.15.39:0/3874215587 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9c80105340 0x7f9c8019a990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:12.417 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:12.411+0000 7f9c86c54640 1 --2- 172.21.15.39:0/3874215587 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9c801049f0 0x7f9c8019a3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:12.418 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:12.411+0000 7f9c86c54640 1 -- 172.21.15.39:0/3874215587 >> 172.21.15.39:0/3874215587 conn(0x7f9c80079730 msgr2=0x7f9c800ff550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:53:12.418 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:12.411+0000 7f9c86c54640 1 -- 172.21.15.39:0/3874215587 shutdown_connections 2024-09-21T03:53:12.418 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:12.411+0000 7f9c86c54640 1 -- 172.21.15.39:0/3874215587 wait complete. 2024-09-21T03:53:13.236 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 51116d5c-77ca-11ef-baf5-efdc52797490 -- ceph tell mds.1:1 damage ls 2024-09-21T03:53:13.503 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/51116d5c-77ca-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:53:13.584 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:53:13 smithi039 ceph-mon[31211]: pgmap v541: 97 pgs: 97 active+clean; 10 MiB data, 520 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 53 KiB/s wr, 30 op/s 2024-09-21T03:53:13.718 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:53:13 smithi161 ceph-mon[34533]: pgmap v541: 97 pgs: 97 active+clean; 10 MiB data, 520 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 53 KiB/s wr, 30 op/s 2024-09-21T03:53:13.725 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:53:13 smithi150 ceph-mon[34289]: pgmap v541: 97 pgs: 97 active+clean; 10 MiB data, 520 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 53 KiB/s wr, 30 op/s 2024-09-21T03:53:13.973 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:13.971+0000 7f12a5d37640 1 -- 172.21.15.39:0/2125425946 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f12a0108610 msgr2=0x7f12a0108a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:53:13.973 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:13.971+0000 7f12a5d37640 1 --2- 172.21.15.39:0/2125425946 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f12a0108610 0x7f12a0108a20 secure :-1 s=READY pgs=1025 cs=0 l=1 rev1=1 crypto rx=0x7f1294009900 tx=0x7f129402a410 comp rx=0 tx=0).stop 2024-09-21T03:53:13.973 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:13.971+0000 7f12a5d37640 1 -- 172.21.15.39:0/2125425946 shutdown_connections 2024-09-21T03:53:13.973 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:13.971+0000 7f12a5d37640 1 --2- 172.21.15.39:0/2125425946 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f12a0108f60 0x7f12a010f790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:13.973 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:13.971+0000 7f12a5d37640 1 --2- 172.21.15.39:0/2125425946 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f12a0108610 0x7f12a0108a20 unknown :-1 s=CLOSED pgs=1025 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:13.974 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:13.971+0000 7f12a5d37640 1 --2- 172.21.15.39:0/2125425946 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f12a01031d0 0x7f12a01035e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:13.974 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:13.971+0000 7f12a5d37640 1 -- 172.21.15.39:0/2125425946 >> 172.21.15.39:0/2125425946 conn(0x7f12a0079e40 msgr2=0x7f12a007a250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:53:13.974 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:13.971+0000 7f12a5d37640 1 -- 172.21.15.39:0/2125425946 shutdown_connections 2024-09-21T03:53:13.974 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:13.971+0000 7f12a5d37640 1 -- 172.21.15.39:0/2125425946 wait complete. 2024-09-21T03:53:13.976 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:13.972+0000 7f12a5d37640 1 Processor -- start 2024-09-21T03:53:13.976 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:13.972+0000 7f12a5d37640 1 -- start start 2024-09-21T03:53:13.976 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:13.972+0000 7f12a5d37640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f12a01031d0 0x7f12a019b4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:53:13.976 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:13.972+0000 7f12a5d37640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f12a0108610 0x7f12a019ba10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:53:13.976 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:13.972+0000 7f12a5d37640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f12a0108f60 0x7f12a019bf50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:53:13.976 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:13.972+0000 7f12a5d37640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f12a0112c20 con 0x7f12a0108610 2024-09-21T03:53:13.976 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:13.972+0000 7f12a5d37640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f12a0112aa0 con 0x7f12a01031d0 2024-09-21T03:53:13.976 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:13.972+0000 7f12a5d37640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f12a0112da0 con 0x7f12a0108f60 2024-09-21T03:53:13.976 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:13.972+0000 7f129effd640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f12a0108610 0x7f12a019ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:53:13.976 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:13.972+0000 7f129effd640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f12a0108610 0x7f12a019ba10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:48680/0 (socket says 172.21.15.39:48680) 2024-09-21T03:53:13.977 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:13.972+0000 7f129effd640 1 -- 172.21.15.39:0/3043746149 learned_addr learned my addr 172.21.15.39:0/3043746149 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:53:13.977 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:13.972+0000 7f129f7fe640 1 --2- 172.21.15.39:0/3043746149 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f12a01031d0 0x7f12a019b4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:53:13.977 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:13.972+0000 7f129ffff640 1 --2- 172.21.15.39:0/3043746149 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f12a0108f60 0x7f12a019bf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:53:13.977 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:13.973+0000 7f129effd640 1 -- 172.21.15.39:0/3043746149 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f12a0108f60 msgr2=0x7f12a019bf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:53:13.977 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:13.973+0000 7f129effd640 1 --2- 172.21.15.39:0/3043746149 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f12a0108f60 0x7f12a019bf50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:13.977 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:13.973+0000 7f129effd640 1 -- 172.21.15.39:0/3043746149 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f12a01031d0 msgr2=0x7f12a019b4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:53:13.977 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:13.973+0000 7f129effd640 1 --2- 172.21.15.39:0/3043746149 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f12a01031d0 0x7f12a019b4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:13.977 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:13.973+0000 7f129effd640 1 -- 172.21.15.39:0/3043746149 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f12a01a6700 con 0x7f12a0108610 2024-09-21T03:53:13.977 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:13.973+0000 7f129effd640 1 --2- 172.21.15.39:0/3043746149 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f12a0108610 0x7f12a019ba10 secure :-1 s=READY pgs=1026 cs=0 l=1 rev1=1 crypto rx=0x7f129402a930 tx=0x7f1294004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:53:13.977 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:13.973+0000 7f129cff9640 1 -- 172.21.15.39:0/3043746149 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f129403f030 con 0x7f12a0108610 2024-09-21T03:53:13.977 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:13.973+0000 7f12a5d37640 1 -- 172.21.15.39:0/3043746149 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f12a01a6990 con 0x7f12a0108610 2024-09-21T03:53:13.977 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:13.973+0000 7f129cff9640 1 -- 172.21.15.39:0/3043746149 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1294003cb0 con 0x7f12a0108610 2024-09-21T03:53:13.978 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:13.973+0000 7f129cff9640 1 -- 172.21.15.39:0/3043746149 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1294035070 con 0x7f12a0108610 2024-09-21T03:53:13.978 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:13.973+0000 7f12a5d37640 1 -- 172.21.15.39:0/3043746149 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f12a01a6ec0 con 0x7f12a0108610 2024-09-21T03:53:13.979 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:13.976+0000 7f12ac2ab740 1 Processor -- start 2024-09-21T03:53:13.980 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:13.976+0000 7f12ac2ab740 1 -- start start 2024-09-21T03:53:13.980 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:13.976+0000 7f12ac2ab740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5589a16c9070 0x5589a1a706a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:53:13.980 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:13.976+0000 7f12ac2ab740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5589a16a01b0 0x5589a1a70be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:53:13.980 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:13.976+0000 7f129cff9640 1 -- 172.21.15.39:0/3043746149 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 27) ==== 154643+0+0 (secure 0 0 0) 0x7f1294042030 con 0x7f12a0108610 2024-09-21T03:53:13.980 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:13.976+0000 7f12ac2ab740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5589a16a8390 0x5589a1a71120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:53:13.980 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:13.976+0000 7f12ac2ab740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x5589a19e6ea0 con 0x5589a16a8390 2024-09-21T03:53:13.980 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:13.976+0000 7f12ac2ab740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x5589a19d62d0 con 0x5589a16a01b0 2024-09-21T03:53:13.980 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:13.976+0000 7f12ac2ab740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x5589a19da0e0 con 0x5589a16c9070 2024-09-21T03:53:13.980 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:13.976+0000 7f129effd640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5589a16a8390 0x5589a1a71120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:53:13.980 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:13.976+0000 7f129ffff640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5589a16c9070 0x5589a1a706a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:53:13.981 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:13.976+0000 7f129f7fe640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5589a16a01b0 0x5589a1a70be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:53:13.981 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:13.976+0000 7f129effd640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5589a16a8390 0x5589a1a71120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:48684/0 (socket says 172.21.15.39:48684) 2024-09-21T03:53:13.981 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:13.976+0000 7f129f7fe640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5589a16a01b0 0x5589a1a70be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.39:49122/0 (socket says 172.21.15.39:49122) 2024-09-21T03:53:13.981 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:13.976+0000 7f129ffff640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5589a16c9070 0x5589a1a706a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.39:50432/0 (socket says 172.21.15.39:50432) 2024-09-21T03:53:13.981 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:13.976+0000 7f129effd640 1 -- 172.21.15.39:0/3841868694 learned_addr learned my addr 172.21.15.39:0/3841868694 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:53:13.981 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:13.977+0000 7f129effd640 1 -- 172.21.15.39:0/3841868694 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5589a16c9070 msgr2=0x5589a1a706a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:53:13.981 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:13.977+0000 7f129effd640 1 --2- 172.21.15.39:0/3841868694 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5589a16c9070 0x5589a1a706a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:13.981 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:13.977+0000 7f129effd640 1 -- 172.21.15.39:0/3841868694 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5589a16a01b0 msgr2=0x5589a1a70be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:53:13.981 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:13.977+0000 7f129effd640 1 --2- 172.21.15.39:0/3841868694 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5589a16a01b0 0x5589a1a70be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:13.981 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:13.977+0000 7f129effd640 1 -- 172.21.15.39:0/3841868694 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5589a1734650 con 0x5589a16a8390 2024-09-21T03:53:13.981 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:13.977+0000 7f129effd640 1 --2- 172.21.15.39:0/3841868694 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5589a16a8390 0x5589a1a71120 secure :-1 s=READY pgs=1027 cs=0 l=1 rev1=1 crypto rx=0x7f1294105170 tx=0x7f129410bcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1612fc6598a53e22 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:53:13.982 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:13.977+0000 7f129cff9640 1 --2- 172.21.15.39:0/3043746149 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f12880b6680 0x7f12880b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:53:13.982 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:13.977+0000 7f1285ffb640 1 -- 172.21.15.39:0/3841868694 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f129411a070 con 0x5589a16a8390 2024-09-21T03:53:13.982 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:13.977+0000 7f129ffff640 1 --2- 172.21.15.39:0/3043746149 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f12880b6680 0x7f12880b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:53:13.982 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:13.977+0000 7f1285ffb640 1 -- 172.21.15.39:0/3841868694 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f129409d920 con 0x5589a16a8390 2024-09-21T03:53:13.982 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:13.977+0000 7f1285ffb640 1 -- 172.21.15.39:0/3841868694 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12940c9f50 con 0x5589a16a8390 2024-09-21T03:53:13.982 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:13.977+0000 7f129cff9640 1 -- 172.21.15.39:0/3043746149 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8572+0+0 (secure 0 0 0) 0x7f1294107ae0 con 0x7f12a0108610 2024-09-21T03:53:13.986 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:13.978+0000 7f12ac2ab740 1 -- 172.21.15.39:0/3841868694 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5589a16a8390 msgr2=0x5589a1a71120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:53:13.986 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:13.978+0000 7f129ffff640 1 --2- 172.21.15.39:0/3043746149 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f12880b6680 0x7f12880b8b90 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f129800cbd0 tx=0x7f129800c380 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:53:13.986 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:13.978+0000 7f12ac2ab740 1 --2- 172.21.15.39:0/3841868694 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5589a16a8390 0x5589a1a71120 secure :-1 s=READY pgs=1027 cs=0 l=1 rev1=1 crypto rx=0x7f1294105170 tx=0x7f129410bcf0 comp rx=0 tx=0).stop 2024-09-21T03:53:13.986 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:13.978+0000 7f12ac2ab740 1 -- 172.21.15.39:0/3841868694 shutdown_connections 2024-09-21T03:53:13.986 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:13.978+0000 7f12ac2ab740 1 --2- 172.21.15.39:0/3841868694 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5589a16a8390 0x5589a1a71120 unknown :-1 s=CLOSED pgs=1027 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:13.986 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:13.978+0000 7f12ac2ab740 1 --2- 172.21.15.39:0/3841868694 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5589a16a01b0 0x5589a1a70be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:13.986 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:13.978+0000 7f12ac2ab740 1 --2- 172.21.15.39:0/3841868694 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5589a16c9070 0x5589a1a706a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:13.986 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:13.978+0000 7f12ac2ab740 1 -- 172.21.15.39:0/3841868694 >> 172.21.15.39:0/3841868694 conn(0x5589a16bd460 msgr2=0x5589a1a6e590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:53:13.986 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:13.978+0000 7f12ac2ab740 1 -- 172.21.15.39:0/3841868694 shutdown_connections 2024-09-21T03:53:13.987 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:13.978+0000 7f12ac2ab740 1 -- 172.21.15.39:0/3841868694 wait complete. 2024-09-21T03:53:13.987 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:13.979+0000 7f12ac2ab740 1 -- start start 2024-09-21T03:53:13.987 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:13.979+0000 7f12ac2ab740 1 Processor -- start 2024-09-21T03:53:13.987 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:13.979+0000 7f12ac2ab740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5589a16a01b0 0x5589a1a7bd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:53:13.987 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:13.979+0000 7f12ac2ab740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5589a16a8390 0x5589a1a7c240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:53:13.987 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:13.979+0000 7f12ac2ab740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5589a16c9070 0x5589a1a87ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:53:13.987 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:13.979+0000 7f12ac2ab740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x5589a1837700 con 0x5589a16c9070 2024-09-21T03:53:13.987 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:13.979+0000 7f12ac2ab740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x5589a1971160 con 0x5589a16a8390 2024-09-21T03:53:13.987 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:13.979+0000 7f12ac2ab740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x5589a1a7ec40 con 0x5589a16a01b0 2024-09-21T03:53:13.987 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:13.980+0000 7f12ac2ab740 10 client.0 _ll_register_callbacks cb 0x5589a198a9e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:53:13.987 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:13.980+0000 7f129ffff640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5589a16c9070 0x5589a1a87ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:53:13.987 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:13.980+0000 7f129ffff640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5589a16c9070 0x5589a1a87ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:48700/0 (socket says 172.21.15.39:48700) 2024-09-21T03:53:13.987 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:13.980+0000 7f129ffff640 1 -- 172.21.15.39:0/105705544 learned_addr learned my addr 172.21.15.39:0/105705544 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:53:13.987 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:13.980+0000 7f129effd640 1 --2- 172.21.15.39:0/105705544 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5589a16a8390 0x5589a1a7c240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:53:13.988 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:13.980+0000 7f129f7fe640 1 --2- 172.21.15.39:0/105705544 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5589a16a01b0 0x5589a1a7bd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:53:13.988 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:13.980+0000 7f129ffff640 1 -- 172.21.15.39:0/105705544 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5589a16a01b0 msgr2=0x5589a1a7bd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:53:13.988 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:13.980+0000 7f129ffff640 1 --2- 172.21.15.39:0/105705544 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5589a16a01b0 0x5589a1a7bd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:13.988 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:13.980+0000 7f129ffff640 1 -- 172.21.15.39:0/105705544 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5589a16a8390 msgr2=0x5589a1a7c240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:53:13.988 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:13.980+0000 7f129ffff640 1 --2- 172.21.15.39:0/105705544 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5589a16a8390 0x5589a1a7c240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:13.988 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:13.980+0000 7f129ffff640 1 -- 172.21.15.39:0/105705544 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5589a199ec90 con 0x5589a16c9070 2024-09-21T03:53:13.988 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:13.980+0000 7f129ffff640 1 --2- 172.21.15.39:0/105705544 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5589a16c9070 0x5589a1a87ef0 secure :-1 s=READY pgs=1028 cs=0 l=1 rev1=1 crypto rx=0x7f129800c630 tx=0x7f12980046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b7470518524970f server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:53:13.988 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:13.980+0000 7f127effd640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-09-21T03:53:13.988 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:13.980+0000 7f127effd640 1 -- 172.21.15.39:0/105705544 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1298029060 con 0x5589a16c9070 2024-09-21T03:53:13.988 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:13.980+0000 7f127effd640 1 -- 172.21.15.39:0/105705544 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8572+0+0 (secure 0 0 0) 0x7f129801a460 con 0x5589a16c9070 2024-09-21T03:53:13.988 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:13.980+0000 7f12ac2ab740 1 -- 172.21.15.39:0/105705544 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f129409d920 con 0x5589a16c9070 2024-09-21T03:53:13.988 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:13.981+0000 7f127effd640 1 -- 172.21.15.39:0/105705544 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f129801a660 con 0x5589a16c9070 2024-09-21T03:53:13.988 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:13.981+0000 7f12ac2ab740 10 client.17367 fetch_fsmap learned FSMap version 29 2024-09-21T03:53:13.988 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:13.981+0000 7f12ac2ab740 1 -- 172.21.15.39:0/105705544 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5589a1a8a910 con 0x5589a16c9070 2024-09-21T03:53:13.989 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:13.981+0000 7f127effd640 1 -- 172.21.15.39:0/105705544 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f129409d920 con 0x5589a16c9070 2024-09-21T03:53:13.989 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:13.981+0000 7f127effd640 1 -- 172.21.15.39:0/105705544 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f1298018040 con 0x5589a16c9070 2024-09-21T03:53:13.989 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:13.981+0000 7f12ac2ab740 10 client.17367 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T03:53:13.989 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:13.981+0000 7f12ac2ab740 10 client.17367 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-21T03:53:13.989 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:13.981+0000 7f12ac2ab740 20 client.17367 populate_metadata read hostname 'smithi039' 2024-09-21T03:53:13.989 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:13.981+0000 7f12ac2ab740 1 --2- 172.21.15.39:0/105705544 >> [v2:172.21.15.161:6838/2190107270,v1:172.21.15.161:6839/2190107270] conn(0x5589a16879f0 0x5589a1a8deb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:53:13.989 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:13.981+0000 7f12ac2ab740 4 client.17367 mds_command: new command op to 24667 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:53:13.989 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:13.981+0000 7f12ac2ab740 1 -- 172.21.15.39:0/105705544 --> [v2:172.21.15.161:6838/2190107270,v1:172.21.15.161:6839/2190107270] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5589a19e8df0 con 0x5589a16879f0 2024-09-21T03:53:13.989 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:13.982+0000 7f129f7fe640 1 --2- 172.21.15.39:0/105705544 >> [v2:172.21.15.161:6838/2190107270,v1:172.21.15.161:6839/2190107270] conn(0x5589a16879f0 0x5589a1a8deb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:53:13.989 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:13.982+0000 7f127effd640 10 client.17367 ms_handle_connect on v2:172.21.15.161:6838/2190107270 2024-09-21T03:53:13.989 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:13.982+0000 7f129f7fe640 1 --2- 172.21.15.39:0/105705544 >> [v2:172.21.15.161:6838/2190107270,v1:172.21.15.161:6839/2190107270] conn(0x5589a16879f0 0x5589a1a8deb0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=8076d057eef74f1b server_cookie=3402ca647660a733 in_seq=0 out_seq=0 2024-09-21T03:53:13.989 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:13.983+0000 7f127effd640 1 -- 172.21.15.39:0/105705544 <== mds.1 v2:172.21.15.161:6838/2190107270 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5589a19e8df0 con 0x5589a16879f0 2024-09-21T03:53:13.989 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:13.983+0000 7f127effd640 10 client.17367 handle_command_reply: tid=0 2024-09-21T03:53:13.989 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:13.983+0000 7f12ac2ab740 1 client.17367 shutdown 2024-09-21T03:53:13.989 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:13.983+0000 7f12ac2ab740 1 -- 172.21.15.39:0/105705544 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5589a16c9070 msgr2=0x5589a1a87ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:53:13.990 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:13.983+0000 7f12ac2ab740 1 --2- 172.21.15.39:0/105705544 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5589a16c9070 0x5589a1a87ef0 secure :-1 s=READY pgs=1028 cs=0 l=1 rev1=1 crypto rx=0x7f129800c630 tx=0x7f12980046c0 comp rx=0 tx=0).stop 2024-09-21T03:53:13.990 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:13.983+0000 7f12ac2ab740 1 -- 172.21.15.39:0/105705544 shutdown_connections 2024-09-21T03:53:13.990 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:13.983+0000 7f12ac2ab740 1 --2- 172.21.15.39:0/105705544 >> [v2:172.21.15.161:6838/2190107270,v1:172.21.15.161:6839/2190107270] conn(0x5589a16879f0 0x5589a1a8deb0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:13.990 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:13.983+0000 7f12ac2ab740 1 --2- 172.21.15.39:0/105705544 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5589a16c9070 0x5589a1a87ef0 unknown :-1 s=CLOSED pgs=1028 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:13.990 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:13.983+0000 7f12ac2ab740 1 --2- 172.21.15.39:0/105705544 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5589a16a8390 0x5589a1a7c240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:13.990 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:13.983+0000 7f127effd640 0 client.17367 ms_handle_reset on v2:172.21.15.161:6838/2190107270 2024-09-21T03:53:13.990 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:13.983+0000 7f12ac2ab740 1 --2- 172.21.15.39:0/105705544 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5589a16a01b0 0x5589a1a7bd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:13.990 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:13.983+0000 7f12ac2ab740 1 -- 172.21.15.39:0/105705544 >> 172.21.15.39:0/105705544 conn(0x5589a16bd460 msgr2=0x5589a1a73830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:53:13.990 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:13.984+0000 7f12ac2ab740 1 -- 172.21.15.39:0/105705544 shutdown_connections 2024-09-21T03:53:13.990 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:13.984+0000 7f12ac2ab740 1 -- 172.21.15.39:0/105705544 wait complete. 2024-09-21T03:53:13.990 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:13.984+0000 7f12ac2ab740 20 client.17367 trim_cache size 0 max 16384 2024-09-21T03:53:13.999 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:13.997+0000 7f12ac2ab740 1 Processor -- start 2024-09-21T03:53:13.999 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:13.997+0000 7f12ac2ab740 1 -- start start 2024-09-21T03:53:14.000 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:13.997+0000 7f12ac2ab740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5589a16a01b0 0x5589a1a7c1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:53:14.000 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:13.997+0000 7f12ac2ab740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5589a16a8390 0x5589a1a87e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:53:14.000 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:13.997+0000 7f12ac2ab740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5589a16c9070 0x5589a1a78ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:53:14.000 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:13.997+0000 7f12ac2ab740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f12a01af760 con 0x5589a16a8390 2024-09-21T03:53:14.000 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:13.997+0000 7f12ac2ab740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f12a01b12a0 con 0x5589a16c9070 2024-09-21T03:53:14.000 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:13.997+0000 7f12ac2ab740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f12a01afe30 con 0x5589a16a01b0 2024-09-21T03:53:14.000 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:13.997+0000 7f129effd640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5589a16a8390 0x5589a1a87e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:53:14.000 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:13.997+0000 7f129effd640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5589a16a8390 0x5589a1a87e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:48716/0 (socket says 172.21.15.39:48716) 2024-09-21T03:53:14.000 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:13.997+0000 7f129effd640 1 -- 172.21.15.39:0/3715267871 learned_addr learned my addr 172.21.15.39:0/3715267871 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:53:14.000 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:13.997+0000 7f129f7fe640 1 --2- 172.21.15.39:0/3715267871 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5589a16a01b0 0x5589a1a7c1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:53:14.000 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:13.997+0000 7f129ffff640 1 --2- 172.21.15.39:0/3715267871 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5589a16c9070 0x5589a1a78ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:53:14.000 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:13.997+0000 7f129effd640 1 -- 172.21.15.39:0/3715267871 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5589a16a01b0 msgr2=0x5589a1a7c1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:53:14.000 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:13.997+0000 7f129effd640 1 --2- 172.21.15.39:0/3715267871 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5589a16a01b0 0x5589a1a7c1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:14.000 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:13.997+0000 7f129effd640 1 -- 172.21.15.39:0/3715267871 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5589a16c9070 msgr2=0x5589a1a78ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:53:14.001 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:13.997+0000 7f129effd640 1 --2- 172.21.15.39:0/3715267871 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5589a16c9070 0x5589a1a78ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:14.001 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:13.997+0000 7f129effd640 1 -- 172.21.15.39:0/3715267871 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5589a1a699a0 con 0x5589a16a8390 2024-09-21T03:53:14.001 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:13.998+0000 7f129effd640 1 --2- 172.21.15.39:0/3715267871 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5589a16a8390 0x5589a1a87e30 secure :-1 s=READY pgs=1029 cs=0 l=1 rev1=1 crypto rx=0x7f12940cab70 tx=0x7f1294107f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a786f9c57ed6fc9f server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:53:14.001 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:13.998+0000 7f12857fa640 1 -- 172.21.15.39:0/3715267871 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f129407e2e0 con 0x5589a16a8390 2024-09-21T03:53:14.001 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:13.998+0000 7f12857fa640 1 -- 172.21.15.39:0/3715267871 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f129410b400 con 0x5589a16a8390 2024-09-21T03:53:14.001 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:13.998+0000 7f12857fa640 1 -- 172.21.15.39:0/3715267871 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f129407ba80 con 0x5589a16a8390 2024-09-21T03:53:14.004 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:13.999+0000 7f12ac2ab740 1 -- 172.21.15.39:0/3715267871 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5589a16a8390 msgr2=0x5589a1a87e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:53:14.004 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:13.999+0000 7f12ac2ab740 1 --2- 172.21.15.39:0/3715267871 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5589a16a8390 0x5589a1a87e30 secure :-1 s=READY pgs=1029 cs=0 l=1 rev1=1 crypto rx=0x7f12940cab70 tx=0x7f1294107f90 comp rx=0 tx=0).stop 2024-09-21T03:53:14.004 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:13.999+0000 7f12ac2ab740 1 -- 172.21.15.39:0/3715267871 shutdown_connections 2024-09-21T03:53:14.005 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:13.999+0000 7f12ac2ab740 1 --2- 172.21.15.39:0/3715267871 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5589a16c9070 0x5589a1a78ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:14.005 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:13.999+0000 7f12ac2ab740 1 --2- 172.21.15.39:0/3715267871 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5589a16a8390 0x5589a1a87e30 unknown :-1 s=CLOSED pgs=1029 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:14.005 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:13.999+0000 7f12ac2ab740 1 --2- 172.21.15.39:0/3715267871 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5589a16a01b0 0x5589a1a7c1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:14.005 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:13.999+0000 7f12ac2ab740 1 -- 172.21.15.39:0/3715267871 >> 172.21.15.39:0/3715267871 conn(0x5589a16bd460 msgr2=0x5589a1a7bab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:53:14.005 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:13.999+0000 7f12ac2ab740 1 -- 172.21.15.39:0/3715267871 shutdown_connections 2024-09-21T03:53:14.005 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:13.999+0000 7f12ac2ab740 1 -- 172.21.15.39:0/3715267871 wait complete. 2024-09-21T03:53:14.005 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:13.999+0000 7f12ac2ab740 1 -- start start 2024-09-21T03:53:14.005 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:13.999+0000 7f12ac2ab740 1 Processor -- start 2024-09-21T03:53:14.005 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:14.000+0000 7f12ac2ab740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5589a16a01b0 0x5589a1a801c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:53:14.005 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:14.000+0000 7f12ac2ab740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5589a16a8390 0x5589a1a80700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:53:14.005 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:14.000+0000 7f12ac2ab740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5589a16c9070 0x5589a1af78b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:53:14.005 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:14.000+0000 7f12ac2ab740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f12a01b0bd0 con 0x5589a16a8390 2024-09-21T03:53:14.005 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:14.000+0000 7f12ac2ab740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f12a01b0500 con 0x5589a16c9070 2024-09-21T03:53:14.006 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:14.000+0000 7f12ac2ab740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f12a01ad760 con 0x5589a16a01b0 2024-09-21T03:53:14.006 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:14.000+0000 7f12ac2ab740 10 client.0 _ll_register_callbacks cb 0x7f129800c820 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:53:14.006 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:14.000+0000 7f129effd640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5589a16a8390 0x5589a1a80700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:53:14.006 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:14.000+0000 7f129effd640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5589a16a8390 0x5589a1a80700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:48726/0 (socket says 172.21.15.39:48726) 2024-09-21T03:53:14.006 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:14.000+0000 7f129effd640 1 -- 172.21.15.39:0/802579735 learned_addr learned my addr 172.21.15.39:0/802579735 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:53:14.006 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:14.000+0000 7f129f7fe640 1 --2- 172.21.15.39:0/802579735 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5589a16a01b0 0x5589a1a801c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:53:14.006 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:14.000+0000 7f129ffff640 1 --2- 172.21.15.39:0/802579735 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5589a16c9070 0x5589a1af78b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:53:14.006 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:14.000+0000 7f129effd640 1 -- 172.21.15.39:0/802579735 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5589a16a01b0 msgr2=0x5589a1a801c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:53:14.006 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:14.000+0000 7f129effd640 1 --2- 172.21.15.39:0/802579735 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5589a16a01b0 0x5589a1a801c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:14.006 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:14.000+0000 7f129effd640 1 -- 172.21.15.39:0/802579735 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5589a16c9070 msgr2=0x5589a1af78b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:53:14.006 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:14.000+0000 7f129effd640 1 --2- 172.21.15.39:0/802579735 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5589a16c9070 0x5589a1af78b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:14.006 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:14.000+0000 7f129effd640 1 -- 172.21.15.39:0/802579735 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5589a1a80ff0 con 0x5589a16a8390 2024-09-21T03:53:14.006 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:14.000+0000 7f129f7fe640 1 --2- 172.21.15.39:0/802579735 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5589a16a01b0 0x5589a1a801c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:53:14.007 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:14.000+0000 7f129effd640 1 --2- 172.21.15.39:0/802579735 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5589a16a8390 0x5589a1a80700 secure :-1 s=READY pgs=1030 cs=0 l=1 rev1=1 crypto rx=0x7f12940cd890 tx=0x7f129402a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8aaddfcb8fca2d1e server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:53:14.007 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:14.000+0000 7f127effd640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-09-21T03:53:14.007 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:14.001+0000 7f127effd640 1 -- 172.21.15.39:0/802579735 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1294049070 con 0x5589a16a8390 2024-09-21T03:53:14.007 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:14.001+0000 7f127effd640 1 -- 172.21.15.39:0/802579735 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8572+0+0 (secure 0 0 0) 0x7f1294105480 con 0x5589a16a8390 2024-09-21T03:53:14.007 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:14.001+0000 7f12ac2ab740 1 -- 172.21.15.39:0/802579735 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f129410b400 con 0x5589a16a8390 2024-09-21T03:53:14.007 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:14.001+0000 7f127effd640 1 -- 172.21.15.39:0/802579735 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f129407bc80 con 0x5589a16a8390 2024-09-21T03:53:14.007 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:14.001+0000 7f12ac2ab740 10 client.17373 fetch_fsmap learned FSMap version 29 2024-09-21T03:53:14.007 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:14.001+0000 7f12ac2ab740 1 -- 172.21.15.39:0/802579735 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5589a1af8ea0 con 0x5589a16a8390 2024-09-21T03:53:14.007 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:14.001+0000 7f127effd640 1 -- 172.21.15.39:0/802579735 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f129410b400 con 0x5589a16a8390 2024-09-21T03:53:14.007 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:14.001+0000 7f127effd640 1 -- 172.21.15.39:0/802579735 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f129404c060 con 0x5589a16a8390 2024-09-21T03:53:14.007 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:14.001+0000 7f12ac2ab740 10 client.17373 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T03:53:14.008 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:14.001+0000 7f12ac2ab740 10 client.17373 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-21T03:53:14.008 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:14.001+0000 7f12ac2ab740 20 client.17373 populate_metadata read hostname 'smithi039' 2024-09-21T03:53:14.008 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:14.001+0000 7f12ac2ab740 1 --2- 172.21.15.39:0/802579735 >> [v2:172.21.15.161:6838/2190107270,v1:172.21.15.161:6839/2190107270] conn(0x5589a165cda0 0x5589a1afb180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:53:14.008 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:14.001+0000 7f12ac2ab740 4 client.17373 mds_command: new command op to 24667 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T03:53:14.008 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:14.001+0000 7f12ac2ab740 1 -- 172.21.15.39:0/802579735 --> [v2:172.21.15.161:6838/2190107270,v1:172.21.15.161:6839/2190107270] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5589a19e48a0 con 0x5589a165cda0 2024-09-21T03:53:14.008 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:14.002+0000 7f129f7fe640 1 --2- 172.21.15.39:0/802579735 >> [v2:172.21.15.161:6838/2190107270,v1:172.21.15.161:6839/2190107270] conn(0x5589a165cda0 0x5589a1afb180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:53:14.008 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:14.002+0000 7f129f7fe640 1 --2- 172.21.15.39:0/802579735 >> [v2:172.21.15.161:6838/2190107270,v1:172.21.15.161:6839/2190107270] conn(0x5589a165cda0 0x5589a1afb180 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=a56f253bb230ec95 server_cookie=dc34fc08444ca653 in_seq=0 out_seq=0 2024-09-21T03:53:14.008 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:14.002+0000 7f127effd640 10 client.17373 ms_handle_connect on v2:172.21.15.161:6838/2190107270 2024-09-21T03:53:14.008 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:14.002+0000 7f127effd640 1 -- 172.21.15.39:0/802579735 <== mds.1 v2:172.21.15.161:6838/2190107270 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5589a19e48a0 con 0x5589a165cda0 2024-09-21T03:53:14.008 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:14.002+0000 7f127effd640 10 client.17373 handle_command_reply: tid=0 2024-09-21T03:53:14.008 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:14.002+0000 7f12ac2ab740 1 client.17373 shutdown 2024-09-21T03:53:14.008 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:14.002+0000 7f12ac2ab740 1 -- 172.21.15.39:0/802579735 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5589a16a8390 msgr2=0x5589a1a80700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:53:14.008 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:14.002+0000 7f12ac2ab740 1 --2- 172.21.15.39:0/802579735 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5589a16a8390 0x5589a1a80700 secure :-1 s=READY pgs=1030 cs=0 l=1 rev1=1 crypto rx=0x7f12940cd890 tx=0x7f129402a410 comp rx=0 tx=0).stop 2024-09-21T03:53:14.008 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:14.002+0000 7f12ac2ab740 1 -- 172.21.15.39:0/802579735 shutdown_connections 2024-09-21T03:53:14.009 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:14.002+0000 7f12ac2ab740 1 --2- 172.21.15.39:0/802579735 >> [v2:172.21.15.161:6838/2190107270,v1:172.21.15.161:6839/2190107270] conn(0x5589a165cda0 0x5589a1afb180 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:14.009 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:14.002+0000 7f12ac2ab740 1 --2- 172.21.15.39:0/802579735 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5589a16c9070 0x5589a1af78b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:14.009 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:14.002+0000 7f12ac2ab740 1 --2- 172.21.15.39:0/802579735 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5589a16a8390 0x5589a1a80700 unknown :-1 s=CLOSED pgs=1030 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:14.009 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:14.002+0000 7f12ac2ab740 1 --2- 172.21.15.39:0/802579735 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5589a16a01b0 0x5589a1a801c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:14.009 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:14.002+0000 7f12ac2ab740 1 -- 172.21.15.39:0/802579735 >> 172.21.15.39:0/802579735 conn(0x5589a16bd460 msgr2=0x5589a1a716d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:53:14.009 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:14.002+0000 7f127effd640 0 client.17373 ms_handle_reset on v2:172.21.15.161:6838/2190107270 2024-09-21T03:53:14.009 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:14.003+0000 7f12ac2ab740 1 -- 172.21.15.39:0/802579735 shutdown_connections 2024-09-21T03:53:14.009 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:14.003+0000 7f12ac2ab740 1 -- 172.21.15.39:0/802579735 wait complete. 2024-09-21T03:53:14.009 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:14.003+0000 7f12ac2ab740 20 client.17373 trim_cache size 0 max 16384 2024-09-21T03:53:14.010 INFO:teuthology.orchestra.run.smithi039.stdout:[] 2024-09-21T03:53:14.010 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:14.003+0000 7f12a5d37640 1 -- 172.21.15.39:0/3043746149 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f12880b6680 msgr2=0x7f12880b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:53:14.010 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:14.003+0000 7f12a5d37640 1 --2- 172.21.15.39:0/3043746149 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f12880b6680 0x7f12880b8b90 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f129800cbd0 tx=0x7f129800c380 comp rx=0 tx=0).stop 2024-09-21T03:53:14.010 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:14.003+0000 7f12a5d37640 1 -- 172.21.15.39:0/3043746149 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f12a0108610 msgr2=0x7f12a019ba10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:53:14.011 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:14.003+0000 7f12a5d37640 1 --2- 172.21.15.39:0/3043746149 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f12a0108610 0x7f12a019ba10 secure :-1 s=READY pgs=1026 cs=0 l=1 rev1=1 crypto rx=0x7f129402a930 tx=0x7f1294004270 comp rx=0 tx=0).stop 2024-09-21T03:53:14.011 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:14.003+0000 7f12a5d37640 1 -- 172.21.15.39:0/3043746149 shutdown_connections 2024-09-21T03:53:14.011 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:14.003+0000 7f12a5d37640 1 --2- 172.21.15.39:0/3043746149 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f12880b6680 0x7f12880b8b90 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:14.011 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:14.003+0000 7f12a5d37640 1 --2- 172.21.15.39:0/3043746149 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f12a0108f60 0x7f12a019bf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:14.011 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:14.003+0000 7f12a5d37640 1 --2- 172.21.15.39:0/3043746149 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f12a0108610 0x7f12a019ba10 unknown :-1 s=CLOSED pgs=1026 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:14.011 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:14.003+0000 7f12a5d37640 1 --2- 172.21.15.39:0/3043746149 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f12a01031d0 0x7f12a019b4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:14.011 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:14.003+0000 7f12a5d37640 1 -- 172.21.15.39:0/3043746149 >> 172.21.15.39:0/3043746149 conn(0x7f12a0079e40 msgr2=0x7f12a01066a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:53:14.011 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:14.003+0000 7f12a5d37640 1 -- 172.21.15.39:0/3043746149 shutdown_connections 2024-09-21T03:53:14.011 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:14.004+0000 7f12a5d37640 1 -- 172.21.15.39:0/3043746149 wait complete. 2024-09-21T03:53:14.918 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 51116d5c-77ca-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 damage ls 2024-09-21T03:53:15.179 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/51116d5c-77ca-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:53:15.584 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:53:15 smithi039 ceph-mon[31211]: pgmap v542: 97 pgs: 97 active+clean; 10 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 53 KiB/s wr, 30 op/s 2024-09-21T03:53:15.685 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:15.683+0000 7fd5682cf640 1 -- 172.21.15.39:0/2154206160 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd560109e40 msgr2=0x7fd560110670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:53:15.685 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:15.683+0000 7fd5682cf640 1 --2- 172.21.15.39:0/2154206160 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd560109e40 0x7fd560110670 secure :-1 s=READY pgs=1031 cs=0 l=1 rev1=1 crypto rx=0x7fd558009ca0 tx=0x7fd55802a8c0 comp rx=0 tx=0).stop 2024-09-21T03:53:15.686 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:15.683+0000 7fd5682cf640 1 -- 172.21.15.39:0/2154206160 shutdown_connections 2024-09-21T03:53:15.686 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:15.683+0000 7fd5682cf640 1 --2- 172.21.15.39:0/2154206160 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd560109e40 0x7fd560110670 unknown :-1 s=CLOSED pgs=1031 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:15.686 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:15.683+0000 7fd5682cf640 1 --2- 172.21.15.39:0/2154206160 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd5601094f0 0x7fd560109900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:15.686 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:15.683+0000 7fd5682cf640 1 --2- 172.21.15.39:0/2154206160 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd560103bb0 0x7fd560103fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:15.686 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:15.683+0000 7fd5682cf640 1 -- 172.21.15.39:0/2154206160 >> 172.21.15.39:0/2154206160 conn(0x7fd5600ff600 msgr2=0x7fd560101a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:53:15.686 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:15.683+0000 7fd5682cf640 1 -- 172.21.15.39:0/2154206160 shutdown_connections 2024-09-21T03:53:15.686 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:15.683+0000 7fd5682cf640 1 -- 172.21.15.39:0/2154206160 wait complete. 2024-09-21T03:53:15.688 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:15.684+0000 7fd5682cf640 1 Processor -- start 2024-09-21T03:53:15.688 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:15.684+0000 7fd5682cf640 1 -- start start 2024-09-21T03:53:15.688 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:15.684+0000 7fd5682cf640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd560103bb0 0x7fd56019a470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:53:15.688 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:15.684+0000 7fd5682cf640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd5601094f0 0x7fd56019aa10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:53:15.688 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:15.684+0000 7fd5682cf640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd560109e40 0x7fd56019fcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:53:15.688 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:15.684+0000 7fd5682cf640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fd560113b00 con 0x7fd560103bb0 2024-09-21T03:53:15.688 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:15.684+0000 7fd5682cf640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fd560113980 con 0x7fd5601094f0 2024-09-21T03:53:15.688 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:15.684+0000 7fd5682cf640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fd560113c80 con 0x7fd560109e40 2024-09-21T03:53:15.689 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:15.684+0000 7fd56606b640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd560103bb0 0x7fd56019a470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:53:15.689 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:15.684+0000 7fd56606b640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd560103bb0 0x7fd56019a470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:48748/0 (socket says 172.21.15.39:48748) 2024-09-21T03:53:15.689 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:15.684+0000 7fd56606b640 1 -- 172.21.15.39:0/97221052 learned_addr learned my addr 172.21.15.39:0/97221052 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:53:15.689 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:15.685+0000 7fd56686c640 1 --2- 172.21.15.39:0/97221052 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd560109e40 0x7fd56019fcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:53:15.689 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:15.685+0000 7fd56586a640 1 --2- 172.21.15.39:0/97221052 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd5601094f0 0x7fd56019aa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:53:15.689 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:15.685+0000 7fd56606b640 1 -- 172.21.15.39:0/97221052 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd560109e40 msgr2=0x7fd56019fcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:53:15.689 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:15.685+0000 7fd56606b640 1 --2- 172.21.15.39:0/97221052 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd560109e40 0x7fd56019fcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:15.689 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:15.685+0000 7fd56606b640 1 -- 172.21.15.39:0/97221052 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd5601094f0 msgr2=0x7fd56019aa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:53:15.689 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:15.685+0000 7fd56606b640 1 --2- 172.21.15.39:0/97221052 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd5601094f0 0x7fd56019aa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:15.689 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:15.685+0000 7fd56606b640 1 -- 172.21.15.39:0/97221052 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd5601a03e0 con 0x7fd560103bb0 2024-09-21T03:53:15.690 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:15.685+0000 7fd56586a640 1 --2- 172.21.15.39:0/97221052 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd5601094f0 0x7fd56019aa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:53:15.690 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:15.685+0000 7fd56606b640 1 --2- 172.21.15.39:0/97221052 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd560103bb0 0x7fd56019a470 secure :-1 s=READY pgs=1032 cs=0 l=1 rev1=1 crypto rx=0x7fd55c00c890 tx=0x7fd55c00cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:53:15.690 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:15.685+0000 7fd5577fe640 1 -- 172.21.15.39:0/97221052 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd55c004220 con 0x7fd560103bb0 2024-09-21T03:53:15.690 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:15.685+0000 7fd5577fe640 1 -- 172.21.15.39:0/97221052 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd55c0043c0 con 0x7fd560103bb0 2024-09-21T03:53:15.690 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:15.685+0000 7fd5682cf640 1 -- 172.21.15.39:0/97221052 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd5601a06d0 con 0x7fd560103bb0 2024-09-21T03:53:15.690 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:15.685+0000 7fd5577fe640 1 -- 172.21.15.39:0/97221052 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd55c004a00 con 0x7fd560103bb0 2024-09-21T03:53:15.690 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:15.685+0000 7fd5682cf640 1 -- 172.21.15.39:0/97221052 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd5601a6e30 con 0x7fd560103bb0 2024-09-21T03:53:15.695 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:15.688+0000 7fd56e843740 1 Processor -- start 2024-09-21T03:53:15.696 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:15.688+0000 7fd56e843740 1 -- start start 2024-09-21T03:53:15.696 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:15.688+0000 7fd56e843740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558dd3e43070 0x558dd41ea6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:53:15.696 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:15.688+0000 7fd56e843740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558dd3e1a1b0 0x558dd41eabe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:53:15.696 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:15.688+0000 7fd56e843740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558dd3e22390 0x558dd41eb120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:53:15.696 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:15.688+0000 7fd56e843740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x558dd4160ea0 con 0x558dd3e43070 2024-09-21T03:53:15.696 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:15.688+0000 7fd56e843740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x558dd41502d0 con 0x558dd3e1a1b0 2024-09-21T03:53:15.696 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:15.688+0000 7fd56e843740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x558dd41540e0 con 0x558dd3e22390 2024-09-21T03:53:15.696 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:15.688+0000 7fd5577fe640 1 -- 172.21.15.39:0/97221052 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 27) ==== 154643+0+0 (secure 0 0 0) 0x7fd55c0079b0 con 0x7fd560103bb0 2024-09-21T03:53:15.696 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:15.689+0000 7fd56686c640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558dd3e43070 0x558dd41ea6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:53:15.696 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:15.689+0000 7fd56686c640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558dd3e43070 0x558dd41ea6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:48752/0 (socket says 172.21.15.39:48752) 2024-09-21T03:53:15.696 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:15.689+0000 7fd56686c640 1 -- 172.21.15.39:0/1925964394 learned_addr learned my addr 172.21.15.39:0/1925964394 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:53:15.696 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:15.689+0000 7fd56586a640 1 --2- 172.21.15.39:0/1925964394 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558dd3e22390 0x558dd41eb120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:53:15.696 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:15.689+0000 7fd56606b640 1 --2- 172.21.15.39:0/1925964394 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558dd3e1a1b0 0x558dd41eabe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:53:15.697 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:15.689+0000 7fd56686c640 1 -- 172.21.15.39:0/1925964394 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558dd3e22390 msgr2=0x558dd41eb120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:53:15.697 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:15.689+0000 7fd56686c640 1 --2- 172.21.15.39:0/1925964394 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558dd3e22390 0x558dd41eb120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:15.697 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:15.689+0000 7fd56686c640 1 -- 172.21.15.39:0/1925964394 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558dd3e1a1b0 msgr2=0x558dd41eabe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:53:15.697 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:15.689+0000 7fd56686c640 1 --2- 172.21.15.39:0/1925964394 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558dd3e1a1b0 0x558dd41eabe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:15.697 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:15.689+0000 7fd56686c640 1 -- 172.21.15.39:0/1925964394 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558dd3eae650 con 0x558dd3e43070 2024-09-21T03:53:15.697 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:15.689+0000 7fd56586a640 1 --2- 172.21.15.39:0/1925964394 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558dd3e22390 0x558dd41eb120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:53:15.697 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:15.689+0000 7fd56606b640 1 --2- 172.21.15.39:0/1925964394 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558dd3e1a1b0 0x558dd41eabe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:53:15.697 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:15.689+0000 7fd56686c640 1 --2- 172.21.15.39:0/1925964394 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558dd3e43070 0x558dd41ea6a0 secure :-1 s=READY pgs=1033 cs=0 l=1 rev1=1 crypto rx=0x7fd5580094d0 tx=0x7fd558031af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f6cc3b67d97ba9c server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:53:15.697 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:15.690+0000 7fd5577fe640 1 --2- 172.21.15.39:0/97221052 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7fd54c0b6680 0x7fd54c0b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:53:15.697 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:15.690+0000 7fd554d9a640 1 -- 172.21.15.39:0/1925964394 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd558031710 con 0x558dd3e43070 2024-09-21T03:53:15.697 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:15.690+0000 7fd554d9a640 1 -- 172.21.15.39:0/1925964394 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd558036070 con 0x558dd3e43070 2024-09-21T03:53:15.697 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:15.690+0000 7fd554d9a640 1 -- 172.21.15.39:0/1925964394 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5580047b0 con 0x558dd3e43070 2024-09-21T03:53:15.697 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:15.690+0000 7fd56606b640 1 --2- 172.21.15.39:0/97221052 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7fd54c0b6680 0x7fd54c0b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:53:15.697 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:15.690+0000 7fd5577fe640 1 -- 172.21.15.39:0/97221052 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8572+0+0 (secure 0 0 0) 0x7fd55c0e7530 con 0x7fd560103bb0 2024-09-21T03:53:15.698 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:15.691+0000 7fd56e843740 1 -- 172.21.15.39:0/1925964394 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558dd3e43070 msgr2=0x558dd41ea6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:53:15.698 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:15.691+0000 7fd56606b640 1 --2- 172.21.15.39:0/97221052 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7fd54c0b6680 0x7fd54c0b8b90 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7fd55c005fd0 tx=0x7fd55c000bc0 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:53:15.698 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:15.691+0000 7fd56e843740 1 --2- 172.21.15.39:0/1925964394 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558dd3e43070 0x558dd41ea6a0 secure :-1 s=READY pgs=1033 cs=0 l=1 rev1=1 crypto rx=0x7fd5580094d0 tx=0x7fd558031af0 comp rx=0 tx=0).stop 2024-09-21T03:53:15.698 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:15.691+0000 7fd56e843740 1 -- 172.21.15.39:0/1925964394 shutdown_connections 2024-09-21T03:53:15.698 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:15.691+0000 7fd56e843740 1 --2- 172.21.15.39:0/1925964394 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558dd3e22390 0x558dd41eb120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:15.698 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:15.691+0000 7fd56e843740 1 --2- 172.21.15.39:0/1925964394 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558dd3e1a1b0 0x558dd41eabe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:15.698 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:15.691+0000 7fd56e843740 1 --2- 172.21.15.39:0/1925964394 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558dd3e43070 0x558dd41ea6a0 unknown :-1 s=CLOSED pgs=1033 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:15.698 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:15.691+0000 7fd56e843740 1 -- 172.21.15.39:0/1925964394 >> 172.21.15.39:0/1925964394 conn(0x558dd3e37460 msgr2=0x558dd41e8590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:53:15.698 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:15.691+0000 7fd56e843740 1 -- 172.21.15.39:0/1925964394 shutdown_connections 2024-09-21T03:53:15.698 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:15.691+0000 7fd56e843740 1 -- 172.21.15.39:0/1925964394 wait complete. 2024-09-21T03:53:15.698 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:15.691+0000 7fd56e843740 1 -- start start 2024-09-21T03:53:15.698 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:15.691+0000 7fd56e843740 1 Processor -- start 2024-09-21T03:53:15.698 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:15.692+0000 7fd56e843740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558dd3e1a1b0 0x558dd41fa160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:53:15.699 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:15.692+0000 7fd56e843740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558dd3e22390 0x558dd41fa6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:53:15.699 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:15.692+0000 7fd56e843740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558dd3e43070 0x558dd4206330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:53:15.699 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:15.692+0000 7fd56e843740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x558dd3fb1700 con 0x558dd3e1a1b0 2024-09-21T03:53:15.699 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:15.692+0000 7fd56e843740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x558dd40eb160 con 0x558dd3e43070 2024-09-21T03:53:15.699 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:15.692+0000 7fd56e843740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x558dd41fcfd0 con 0x558dd3e22390 2024-09-21T03:53:15.699 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:15.692+0000 7fd56e843740 10 client.0 _ll_register_callbacks cb 0x558dd41049e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:53:15.699 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:15.692+0000 7fd56686c640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558dd3e1a1b0 0x558dd41fa160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:53:15.699 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:15.692+0000 7fd56686c640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558dd3e1a1b0 0x558dd41fa160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:48760/0 (socket says 172.21.15.39:48760) 2024-09-21T03:53:15.699 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:15.692+0000 7fd56686c640 1 -- 172.21.15.39:0/280143732 learned_addr learned my addr 172.21.15.39:0/280143732 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:53:15.699 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:15.692+0000 7fd56586a640 1 --2- 172.21.15.39:0/280143732 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558dd3e22390 0x558dd41fa6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:53:15.699 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:15.692+0000 7fd56686c640 1 --2- 172.21.15.39:0/280143732 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558dd3e43070 0x558dd4206330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:53:15.699 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:15.692+0000 7fd56686c640 1 -- 172.21.15.39:0/280143732 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558dd3e22390 msgr2=0x558dd41fa6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:53:15.699 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:15.692+0000 7fd56686c640 1 --2- 172.21.15.39:0/280143732 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558dd3e22390 0x558dd41fa6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:15.699 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:15.692+0000 7fd56686c640 1 -- 172.21.15.39:0/280143732 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558dd3e43070 msgr2=0x558dd4206330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:53:15.700 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:15.692+0000 7fd56686c640 1 --2- 172.21.15.39:0/280143732 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558dd3e43070 0x558dd4206330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:15.700 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:15.692+0000 7fd56686c640 1 -- 172.21.15.39:0/280143732 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558dd4118c90 con 0x558dd3e1a1b0 2024-09-21T03:53:15.700 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:15.692+0000 7fd56686c640 1 --2- 172.21.15.39:0/280143732 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558dd3e1a1b0 0x558dd41fa160 secure :-1 s=READY pgs=1034 cs=0 l=1 rev1=1 crypto rx=0x7fd558004490 tx=0x7fd5580042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f01c1f9b4c517de1 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:53:15.700 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:15.692+0000 7fd549ffb640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-09-21T03:53:15.700 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:15.693+0000 7fd549ffb640 1 -- 172.21.15.39:0/280143732 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd558003970 con 0x558dd3e1a1b0 2024-09-21T03:53:15.700 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:15.693+0000 7fd549ffb640 1 -- 172.21.15.39:0/280143732 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8572+0+0 (secure 0 0 0) 0x7fd558003b10 con 0x558dd3e1a1b0 2024-09-21T03:53:15.700 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:15.693+0000 7fd56e843740 1 -- 172.21.15.39:0/280143732 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd558036070 con 0x558dd3e1a1b0 2024-09-21T03:53:15.700 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:15.693+0000 7fd549ffb640 1 -- 172.21.15.39:0/280143732 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fd55805c050 con 0x558dd3e1a1b0 2024-09-21T03:53:15.700 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:15.693+0000 7fd56e843740 10 client.17385 fetch_fsmap learned FSMap version 29 2024-09-21T03:53:15.701 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:15.693+0000 7fd56e843740 1 -- 172.21.15.39:0/280143732 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x558dd4208d50 con 0x558dd3e1a1b0 2024-09-21T03:53:15.701 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:15.693+0000 7fd549ffb640 1 -- 172.21.15.39:0/280143732 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd558036070 con 0x558dd3e1a1b0 2024-09-21T03:53:15.701 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:15.693+0000 7fd549ffb640 1 -- 172.21.15.39:0/280143732 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fd558043020 con 0x558dd3e1a1b0 2024-09-21T03:53:15.701 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:15.693+0000 7fd56e843740 10 client.17385 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T03:53:15.701 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:15.693+0000 7fd56e843740 10 client.17385 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T03:53:15.701 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:15.693+0000 7fd56e843740 20 client.17385 populate_metadata read hostname 'smithi039' 2024-09-21T03:53:15.702 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:15.693+0000 7fd56e843740 1 --2- 172.21.15.39:0/280143732 >> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] conn(0x558dd3e019f0 0x558dd420c0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:53:15.702 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:15.693+0000 7fd56e843740 4 client.17385 mds_command: new command op to 24695 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:53:15.702 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:15.693+0000 7fd56e843740 1 -- 172.21.15.39:0/280143732 --> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558dd4162df0 con 0x558dd3e019f0 2024-09-21T03:53:15.702 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:15.694+0000 7fd56586a640 1 --2- 172.21.15.39:0/280143732 >> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] conn(0x558dd3e019f0 0x558dd420c0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:53:15.703 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:15.694+0000 7fd549ffb640 10 client.17385 ms_handle_connect on v2:172.21.15.150:6838/2949386907 2024-09-21T03:53:15.703 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:15.694+0000 7fd56586a640 1 --2- 172.21.15.39:0/280143732 >> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] conn(0x558dd3e019f0 0x558dd420c0d0 crc :-1 s=READY pgs=121 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=56c106d2da4b5194 server_cookie=ec14cdaee347c5e2 in_seq=0 out_seq=0 2024-09-21T03:53:15.703 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:15.695+0000 7fd549ffb640 1 -- 172.21.15.39:0/280143732 <== mds.0 v2:172.21.15.150:6838/2949386907 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x558dd4162df0 con 0x558dd3e019f0 2024-09-21T03:53:15.703 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:15.695+0000 7fd549ffb640 10 client.17385 handle_command_reply: tid=0 2024-09-21T03:53:15.703 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:15.695+0000 7fd56e843740 1 client.17385 shutdown 2024-09-21T03:53:15.703 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:15.695+0000 7fd56e843740 1 -- 172.21.15.39:0/280143732 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558dd3e1a1b0 msgr2=0x558dd41fa160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:53:15.703 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:15.695+0000 7fd56e843740 1 --2- 172.21.15.39:0/280143732 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558dd3e1a1b0 0x558dd41fa160 secure :-1 s=READY pgs=1034 cs=0 l=1 rev1=1 crypto rx=0x7fd558004490 tx=0x7fd5580042f0 comp rx=0 tx=0).stop 2024-09-21T03:53:15.703 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:15.695+0000 7fd56e843740 1 -- 172.21.15.39:0/280143732 shutdown_connections 2024-09-21T03:53:15.703 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:15.695+0000 7fd56e843740 1 --2- 172.21.15.39:0/280143732 >> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] conn(0x558dd3e019f0 0x558dd420c0d0 crc :-1 s=READY pgs=121 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:15.703 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:15.695+0000 7fd56e843740 1 --2- 172.21.15.39:0/280143732 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558dd3e43070 0x558dd4206330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:15.703 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:15.695+0000 7fd549ffb640 0 client.17385 ms_handle_reset on v2:172.21.15.150:6838/2949386907 2024-09-21T03:53:15.704 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:15.695+0000 7fd56e843740 1 --2- 172.21.15.39:0/280143732 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558dd3e22390 0x558dd41fa6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:15.704 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:15.695+0000 7fd56e843740 1 --2- 172.21.15.39:0/280143732 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558dd3e1a1b0 0x558dd41fa160 unknown :-1 s=CLOSED pgs=1034 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:15.704 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:15.695+0000 7fd56e843740 1 -- 172.21.15.39:0/280143732 >> 172.21.15.39:0/280143732 conn(0x558dd3e37460 msgr2=0x558dd41ed830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:53:15.704 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:15.695+0000 7fd56e843740 1 -- 172.21.15.39:0/280143732 shutdown_connections 2024-09-21T03:53:15.704 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:15.696+0000 7fd56e843740 1 -- 172.21.15.39:0/280143732 wait complete. 2024-09-21T03:53:15.704 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:15.696+0000 7fd56e843740 20 client.17385 trim_cache size 0 max 16384 2024-09-21T03:53:15.711 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:15.709+0000 7fd56e843740 1 Processor -- start 2024-09-21T03:53:15.711 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:15.709+0000 7fd56e843740 1 -- start start 2024-09-21T03:53:15.711 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:15.709+0000 7fd56e843740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558dd3e019f0 0x558dd41fc150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:53:15.711 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:15.709+0000 7fd56e843740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558dd3e1a1b0 0x558dd41e7bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:53:15.711 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:15.709+0000 7fd56e843740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558dd3e22390 0x558dd41e8100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:53:15.711 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:15.709+0000 7fd56e843740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fd5601ad5d0 con 0x558dd3e019f0 2024-09-21T03:53:15.711 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:15.709+0000 7fd56e843740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fd5601af110 con 0x558dd3e1a1b0 2024-09-21T03:53:15.711 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:15.709+0000 7fd56e843740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fd5601adca0 con 0x558dd3e22390 2024-09-21T03:53:15.711 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:15.709+0000 7fd56686c640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558dd3e019f0 0x558dd41fc150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:53:15.711 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:15.709+0000 7fd56686c640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558dd3e019f0 0x558dd41fc150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:48774/0 (socket says 172.21.15.39:48774) 2024-09-21T03:53:15.712 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:15.709+0000 7fd56686c640 1 -- 172.21.15.39:0/3969199898 learned_addr learned my addr 172.21.15.39:0/3969199898 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:53:15.712 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:15.709+0000 7fd56586a640 1 --2- 172.21.15.39:0/3969199898 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558dd3e1a1b0 0x558dd41e7bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:53:15.714 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:15.709+0000 7fd56686c640 1 -- 172.21.15.39:0/3969199898 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558dd3e22390 msgr2=0x558dd41e8100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:53:15.714 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:15.710+0000 7fd56686c640 1 --2- 172.21.15.39:0/3969199898 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558dd3e22390 0x558dd41e8100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:15.714 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:15.710+0000 7fd56686c640 1 -- 172.21.15.39:0/3969199898 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558dd3e1a1b0 msgr2=0x558dd41e7bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:53:15.714 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:15.710+0000 7fd56686c640 1 --2- 172.21.15.39:0/3969199898 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558dd3e1a1b0 0x558dd41e7bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:15.715 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:15.710+0000 7fd56686c640 1 -- 172.21.15.39:0/3969199898 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558dd41e39a0 con 0x558dd3e019f0 2024-09-21T03:53:15.715 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:15.710+0000 7fd56686c640 1 --2- 172.21.15.39:0/3969199898 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558dd3e019f0 0x558dd41fc150 secure :-1 s=READY pgs=1035 cs=0 l=1 rev1=1 crypto rx=0x7fd5580096c0 tx=0x7fd55802f3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9740d267646a2f1c server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:53:15.715 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:15.710+0000 7fd54bfff640 1 -- 172.21.15.39:0/3969199898 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5580305c0 con 0x558dd3e019f0 2024-09-21T03:53:15.715 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:15.710+0000 7fd54bfff640 1 -- 172.21.15.39:0/3969199898 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd558036040 con 0x558dd3e019f0 2024-09-21T03:53:15.715 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:15.710+0000 7fd54bfff640 1 -- 172.21.15.39:0/3969199898 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd55802f8c0 con 0x558dd3e019f0 2024-09-21T03:53:15.715 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:15.711+0000 7fd56e843740 1 -- 172.21.15.39:0/3969199898 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558dd3e019f0 msgr2=0x558dd41fc150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:53:15.715 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:15.711+0000 7fd56e843740 1 --2- 172.21.15.39:0/3969199898 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558dd3e019f0 0x558dd41fc150 secure :-1 s=READY pgs=1035 cs=0 l=1 rev1=1 crypto rx=0x7fd5580096c0 tx=0x7fd55802f3a0 comp rx=0 tx=0).stop 2024-09-21T03:53:15.715 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:15.711+0000 7fd56e843740 1 -- 172.21.15.39:0/3969199898 shutdown_connections 2024-09-21T03:53:15.715 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:15.711+0000 7fd56e843740 1 --2- 172.21.15.39:0/3969199898 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558dd3e22390 0x558dd41e8100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:15.715 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:15.711+0000 7fd56e843740 1 --2- 172.21.15.39:0/3969199898 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558dd3e1a1b0 0x558dd41e7bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:15.715 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:15.711+0000 7fd56e843740 1 --2- 172.21.15.39:0/3969199898 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558dd3e019f0 0x558dd41fc150 unknown :-1 s=CLOSED pgs=1035 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:15.715 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:15.711+0000 7fd56e843740 1 -- 172.21.15.39:0/3969199898 >> 172.21.15.39:0/3969199898 conn(0x558dd3e37460 msgr2=0x558dd41ee1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:53:15.715 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:15.711+0000 7fd56e843740 1 -- 172.21.15.39:0/3969199898 shutdown_connections 2024-09-21T03:53:15.715 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:15.711+0000 7fd56e843740 1 -- 172.21.15.39:0/3969199898 wait complete. 2024-09-21T03:53:15.716 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:15.711+0000 7fd56e843740 1 -- start start 2024-09-21T03:53:15.716 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:15.711+0000 7fd56e843740 1 Processor -- start 2024-09-21T03:53:15.716 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:15.711+0000 7fd56e843740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558dd3e019f0 0x558dd41fe220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:53:15.716 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:15.712+0000 7fd56e843740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558dd3e1a1b0 0x558dd41fe760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:53:15.716 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:15.712+0000 7fd56e843740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558dd3e22390 0x558dd41feca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:53:15.716 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:15.712+0000 7fd56e843740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fd5601aea40 con 0x558dd3e22390 2024-09-21T03:53:15.716 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:15.712+0000 7fd56e843740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fd5601ae370 con 0x558dd3e019f0 2024-09-21T03:53:15.716 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:15.712+0000 7fd56e843740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fd5601ab5d0 con 0x558dd3e1a1b0 2024-09-21T03:53:15.716 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:15.712+0000 7fd56e843740 10 client.0 _ll_register_callbacks cb 0x7fd55802f410 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:53:15.716 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:15.712+0000 7fd56686c640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558dd3e22390 0x558dd41feca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:53:15.716 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:15.712+0000 7fd56686c640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558dd3e22390 0x558dd41feca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:48776/0 (socket says 172.21.15.39:48776) 2024-09-21T03:53:15.716 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:15.712+0000 7fd56686c640 1 -- 172.21.15.39:0/3856498695 learned_addr learned my addr 172.21.15.39:0/3856498695 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:53:15.716 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:15.712+0000 7fd56686c640 1 --2- 172.21.15.39:0/3856498695 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558dd3e019f0 0x558dd41fe220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:53:15.716 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:15.712+0000 7fd56586a640 1 --2- 172.21.15.39:0/3856498695 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558dd3e1a1b0 0x558dd41fe760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:53:15.716 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:15.712+0000 7fd56686c640 1 -- 172.21.15.39:0/3856498695 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558dd3e1a1b0 msgr2=0x558dd41fe760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:53:15.717 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:15.712+0000 7fd56686c640 1 --2- 172.21.15.39:0/3856498695 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558dd3e1a1b0 0x558dd41fe760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:15.717 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:15.712+0000 7fd56686c640 1 -- 172.21.15.39:0/3856498695 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558dd3e019f0 msgr2=0x558dd41fe220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:53:15.717 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:15.712+0000 7fd56686c640 1 --2- 172.21.15.39:0/3856498695 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558dd3e019f0 0x558dd41fe220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:15.717 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:15.712+0000 7fd56686c640 1 -- 172.21.15.39:0/3856498695 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558dd4275a90 con 0x558dd3e22390 2024-09-21T03:53:15.717 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:15.712+0000 7fd56686c640 1 --2- 172.21.15.39:0/3856498695 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558dd3e22390 0x558dd41feca0 secure :-1 s=READY pgs=1036 cs=0 l=1 rev1=1 crypto rx=0x7fd558004500 tx=0x7fd5580317a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f286d0df17de485e server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:53:15.717 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:15.712+0000 7fd549ffb640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-09-21T03:53:15.717 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:15.712+0000 7fd549ffb640 1 -- 172.21.15.39:0/3856498695 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd558030c20 con 0x558dd3e22390 2024-09-21T03:53:15.717 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:15.712+0000 7fd549ffb640 1 -- 172.21.15.39:0/3856498695 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8572+0+0 (secure 0 0 0) 0x7fd558051070 con 0x558dd3e22390 2024-09-21T03:53:15.717 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:15.712+0000 7fd56e843740 1 -- 172.21.15.39:0/3856498695 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd558036040 con 0x558dd3e22390 2024-09-21T03:53:15.717 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:15.713+0000 7fd549ffb640 1 -- 172.21.15.39:0/3856498695 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fd558004b40 con 0x558dd3e22390 2024-09-21T03:53:15.717 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:15.713+0000 7fd56e843740 10 client.17391 fetch_fsmap learned FSMap version 29 2024-09-21T03:53:15.717 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:15.713+0000 7fd56e843740 1 -- 172.21.15.39:0/3856498695 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x558dd4276cc0 con 0x558dd3e22390 2024-09-21T03:53:15.718 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:53:15 smithi161 ceph-mon[34533]: pgmap v542: 97 pgs: 97 active+clean; 10 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 53 KiB/s wr, 30 op/s 2024-09-21T03:53:15.719 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:15.713+0000 7fd549ffb640 1 -- 172.21.15.39:0/3856498695 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd558036040 con 0x558dd3e22390 2024-09-21T03:53:15.719 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:15.713+0000 7fd549ffb640 1 -- 172.21.15.39:0/3856498695 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fd558043070 con 0x558dd3e22390 2024-09-21T03:53:15.719 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:15.713+0000 7fd56e843740 10 client.17391 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T03:53:15.719 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:15.713+0000 7fd56e843740 10 client.17391 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T03:53:15.720 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:15.713+0000 7fd56e843740 20 client.17391 populate_metadata read hostname 'smithi039' 2024-09-21T03:53:15.720 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:15.713+0000 7fd56e843740 1 --2- 172.21.15.39:0/3856498695 >> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] conn(0x558dd3e43070 0x558dd4279180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:53:15.720 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:15.713+0000 7fd56e843740 4 client.17391 mds_command: new command op to 24695 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T03:53:15.720 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:15.713+0000 7fd56e843740 1 -- 172.21.15.39:0/3856498695 --> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] -- command(tid 0: {"prefix": "damage ls"}) -- 0x558dd415e8a0 con 0x558dd3e43070 2024-09-21T03:53:15.720 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:15.714+0000 7fd56586a640 1 --2- 172.21.15.39:0/3856498695 >> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] conn(0x558dd3e43070 0x558dd4279180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:53:15.720 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:15.714+0000 7fd56586a640 1 --2- 172.21.15.39:0/3856498695 >> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] conn(0x558dd3e43070 0x558dd4279180 crc :-1 s=READY pgs=122 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5b3dbbac76e2e98d server_cookie=464182d154e21213 in_seq=0 out_seq=0 2024-09-21T03:53:15.720 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:15.714+0000 7fd549ffb640 10 client.17391 ms_handle_connect on v2:172.21.15.150:6838/2949386907 2024-09-21T03:53:15.720 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:15.714+0000 7fd549ffb640 1 -- 172.21.15.39:0/3856498695 <== mds.0 v2:172.21.15.150:6838/2949386907 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x558dd415e8a0 con 0x558dd3e43070 2024-09-21T03:53:15.720 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:15.714+0000 7fd549ffb640 10 client.17391 handle_command_reply: tid=0 2024-09-21T03:53:15.720 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:15.714+0000 7fd56e843740 1 client.17391 shutdown 2024-09-21T03:53:15.720 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:15.714+0000 7fd56e843740 1 -- 172.21.15.39:0/3856498695 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558dd3e22390 msgr2=0x558dd41feca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:53:15.720 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:15.714+0000 7fd56e843740 1 --2- 172.21.15.39:0/3856498695 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558dd3e22390 0x558dd41feca0 secure :-1 s=READY pgs=1036 cs=0 l=1 rev1=1 crypto rx=0x7fd558004500 tx=0x7fd5580317a0 comp rx=0 tx=0).stop 2024-09-21T03:53:15.720 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:15.714+0000 7fd56e843740 1 -- 172.21.15.39:0/3856498695 shutdown_connections 2024-09-21T03:53:15.720 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:15.714+0000 7fd56e843740 1 --2- 172.21.15.39:0/3856498695 >> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] conn(0x558dd3e43070 0x558dd4279180 crc :-1 s=READY pgs=122 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:15.721 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:15.714+0000 7fd56e843740 1 --2- 172.21.15.39:0/3856498695 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558dd3e22390 0x558dd41feca0 unknown :-1 s=CLOSED pgs=1036 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:15.721 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:15.714+0000 7fd56e843740 1 --2- 172.21.15.39:0/3856498695 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558dd3e1a1b0 0x558dd41fe760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:15.721 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:15.714+0000 7fd549ffb640 0 client.17391 ms_handle_reset on v2:172.21.15.150:6838/2949386907 2024-09-21T03:53:15.721 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:15.714+0000 7fd56e843740 1 --2- 172.21.15.39:0/3856498695 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558dd3e019f0 0x558dd41fe220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:15.721 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:15.715+0000 7fd56e843740 1 -- 172.21.15.39:0/3856498695 >> 172.21.15.39:0/3856498695 conn(0x558dd3e37460 msgr2=0x558dd41ee1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:53:15.721 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:15.715+0000 7fd56e843740 1 -- 172.21.15.39:0/3856498695 shutdown_connections 2024-09-21T03:53:15.721 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:15.715+0000 7fd56e843740 1 -- 172.21.15.39:0/3856498695 wait complete. 2024-09-21T03:53:15.721 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:15.715+0000 7fd56e843740 20 client.17391 trim_cache size 0 max 16384 2024-09-21T03:53:15.721 INFO:teuthology.orchestra.run.smithi039.stdout:[] 2024-09-21T03:53:15.722 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:15.715+0000 7fd5682cf640 1 -- 172.21.15.39:0/97221052 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7fd54c0b6680 msgr2=0x7fd54c0b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:53:15.722 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:15.715+0000 7fd5682cf640 1 --2- 172.21.15.39:0/97221052 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7fd54c0b6680 0x7fd54c0b8b90 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7fd55c005fd0 tx=0x7fd55c000bc0 comp rx=0 tx=0).stop 2024-09-21T03:53:15.722 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:15.715+0000 7fd5682cf640 1 -- 172.21.15.39:0/97221052 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd560103bb0 msgr2=0x7fd56019a470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:53:15.722 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:15.715+0000 7fd5682cf640 1 --2- 172.21.15.39:0/97221052 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd560103bb0 0x7fd56019a470 secure :-1 s=READY pgs=1032 cs=0 l=1 rev1=1 crypto rx=0x7fd55c00c890 tx=0x7fd55c00cd50 comp rx=0 tx=0).stop 2024-09-21T03:53:15.722 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:15.715+0000 7fd5682cf640 1 -- 172.21.15.39:0/97221052 shutdown_connections 2024-09-21T03:53:15.722 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:15.715+0000 7fd5682cf640 1 --2- 172.21.15.39:0/97221052 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7fd54c0b6680 0x7fd54c0b8b90 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:15.722 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:15.715+0000 7fd5682cf640 1 --2- 172.21.15.39:0/97221052 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd560109e40 0x7fd56019fcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:15.722 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:15.715+0000 7fd5682cf640 1 --2- 172.21.15.39:0/97221052 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd5601094f0 0x7fd56019aa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:15.722 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:15.715+0000 7fd5682cf640 1 --2- 172.21.15.39:0/97221052 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd560103bb0 0x7fd56019a470 unknown :-1 s=CLOSED pgs=1032 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:15.723 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:15.715+0000 7fd5682cf640 1 -- 172.21.15.39:0/97221052 >> 172.21.15.39:0/97221052 conn(0x7fd5600ff600 msgr2=0x7fd560101a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:53:15.723 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:15.715+0000 7fd5682cf640 1 -- 172.21.15.39:0/97221052 shutdown_connections 2024-09-21T03:53:15.723 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:15.715+0000 7fd5682cf640 1 -- 172.21.15.39:0/97221052 wait complete. 2024-09-21T03:53:15.724 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:53:15 smithi150 ceph-mon[34289]: pgmap v542: 97 pgs: 97 active+clean; 10 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 53 KiB/s wr, 30 op/s 2024-09-21T03:53:17.624 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-21T03:53:17.625 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 51116d5c-77ca-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-21T03:53:17.652 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:53:17 smithi039 ceph-mon[31211]: pgmap v543: 97 pgs: 97 active+clean; 10 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-09-21T03:53:17.718 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:53:17 smithi161 ceph-mon[34533]: pgmap v543: 97 pgs: 97 active+clean; 10 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-09-21T03:53:17.725 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:53:17 smithi150 ceph-mon[34289]: pgmap v543: 97 pgs: 97 active+clean; 10 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-09-21T03:53:17.893 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/51116d5c-77ca-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:53:18.408 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:18.406+0000 7ff9eb991640 1 -- 172.21.15.39:0/402153277 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff9e410de40 msgr2=0x7ff9e4110310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:53:18.408 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:18.406+0000 7ff9eb991640 1 --2- 172.21.15.39:0/402153277 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff9e410de40 0x7ff9e4110310 secure :-1 s=READY pgs=1037 cs=0 l=1 rev1=1 crypto rx=0x7ff9d4009ca0 tx=0x7ff9d402a8c0 comp rx=0 tx=0).stop 2024-09-21T03:53:18.409 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:18.406+0000 7ff9eb991640 1 -- 172.21.15.39:0/402153277 shutdown_connections 2024-09-21T03:53:18.409 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:18.406+0000 7ff9eb991640 1 --2- 172.21.15.39:0/402153277 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff9e410de40 0x7ff9e4110310 unknown :-1 s=CLOSED pgs=1037 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:18.409 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:18.406+0000 7ff9eb991640 1 --2- 172.21.15.39:0/402153277 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff9e41052c0 0x7ff9e410d900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:18.409 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:18.406+0000 7ff9eb991640 1 --2- 172.21.15.39:0/402153277 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff9e4104890 0x7ff9e4104ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:18.409 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:18.406+0000 7ff9eb991640 1 -- 172.21.15.39:0/402153277 >> 172.21.15.39:0/402153277 conn(0x7ff9e4079710 msgr2=0x7ff9e4079b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:53:18.409 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:18.406+0000 7ff9eb991640 1 -- 172.21.15.39:0/402153277 shutdown_connections 2024-09-21T03:53:18.409 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:18.406+0000 7ff9eb991640 1 -- 172.21.15.39:0/402153277 wait complete. 2024-09-21T03:53:18.411 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:18.407+0000 7ff9eb991640 1 Processor -- start 2024-09-21T03:53:18.411 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:18.407+0000 7ff9eb991640 1 -- start start 2024-09-21T03:53:18.411 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:18.407+0000 7ff9eb991640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff9e4104890 0x7ff9e419a450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:53:18.411 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:18.407+0000 7ff9eb991640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff9e41052c0 0x7ff9e419a9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:53:18.411 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:18.407+0000 7ff9eb991640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff9e410de40 0x7ff9e419fc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:53:18.411 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:18.407+0000 7ff9eb991640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7ff9e4113a80 con 0x7ff9e4104890 2024-09-21T03:53:18.411 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:18.407+0000 7ff9eb991640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7ff9e4113900 con 0x7ff9e41052c0 2024-09-21T03:53:18.411 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:18.407+0000 7ff9eb991640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7ff9e4113c00 con 0x7ff9e410de40 2024-09-21T03:53:18.411 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:18.408+0000 7ff9e972d640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff9e4104890 0x7ff9e419a450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:53:18.411 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:18.408+0000 7ff9e972d640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff9e4104890 0x7ff9e419a450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:57192/0 (socket says 172.21.15.39:57192) 2024-09-21T03:53:18.411 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:18.408+0000 7ff9e972d640 1 -- 172.21.15.39:0/162926316 learned_addr learned my addr 172.21.15.39:0/162926316 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:53:18.412 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:18.408+0000 7ff9e8f2c640 1 --2- 172.21.15.39:0/162926316 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff9e41052c0 0x7ff9e419a9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:53:18.412 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:18.408+0000 7ff9e9f2e640 1 --2- 172.21.15.39:0/162926316 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff9e410de40 0x7ff9e419fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:53:18.412 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:18.408+0000 7ff9e972d640 1 -- 172.21.15.39:0/162926316 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff9e410de40 msgr2=0x7ff9e419fc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:53:18.412 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:18.408+0000 7ff9e972d640 1 --2- 172.21.15.39:0/162926316 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff9e410de40 0x7ff9e419fc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:18.412 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:18.408+0000 7ff9e972d640 1 -- 172.21.15.39:0/162926316 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff9e41052c0 msgr2=0x7ff9e419a9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:53:18.412 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:18.408+0000 7ff9e972d640 1 --2- 172.21.15.39:0/162926316 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff9e41052c0 0x7ff9e419a9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:18.412 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:18.408+0000 7ff9e972d640 1 -- 172.21.15.39:0/162926316 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff9e41a03c0 con 0x7ff9e4104890 2024-09-21T03:53:18.412 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:18.408+0000 7ff9e8f2c640 1 --2- 172.21.15.39:0/162926316 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff9e41052c0 0x7ff9e419a9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:53:18.412 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:18.408+0000 7ff9e972d640 1 --2- 172.21.15.39:0/162926316 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff9e4104890 0x7ff9e419a450 secure :-1 s=READY pgs=1038 cs=0 l=1 rev1=1 crypto rx=0x7ff9dc00e9f0 tx=0x7ff9dc00edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:53:18.412 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:18.408+0000 7ff9e27fc640 1 -- 172.21.15.39:0/162926316 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9dc00cd80 con 0x7ff9e4104890 2024-09-21T03:53:18.412 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:18.408+0000 7ff9e27fc640 1 -- 172.21.15.39:0/162926316 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff9dc0054d0 con 0x7ff9e4104890 2024-09-21T03:53:18.412 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:18.408+0000 7ff9e27fc640 1 -- 172.21.15.39:0/162926316 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9dc010690 con 0x7ff9e4104890 2024-09-21T03:53:18.412 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:18.408+0000 7ff9eb991640 1 -- 172.21.15.39:0/162926316 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff9e41a06b0 con 0x7ff9e4104890 2024-09-21T03:53:18.412 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:18.408+0000 7ff9eb991640 1 -- 172.21.15.39:0/162926316 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff9e41a6e10 con 0x7ff9e4104890 2024-09-21T03:53:18.413 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:18.411+0000 7ff9f1f05740 1 Processor -- start 2024-09-21T03:53:18.413 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:18.411+0000 7ff9e27fc640 1 -- 172.21.15.39:0/162926316 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 27) ==== 154643+0+0 (secure 0 0 0) 0x7ff9dc004e90 con 0x7ff9e4104890 2024-09-21T03:53:18.413 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:18.411+0000 7ff9f1f05740 1 -- start start 2024-09-21T03:53:18.415 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:18.412+0000 7ff9f1f05740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55de388cf350 0x55de38c76430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:53:18.415 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:18.412+0000 7ff9f1f05740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55de388a6490 0x55de38c76970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:53:18.415 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:18.412+0000 7ff9f1f05740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55de388ae670 0x55de38c76eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:53:18.415 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:18.412+0000 7ff9f1f05740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x55de38bece60 con 0x55de388a6490 2024-09-21T03:53:18.415 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:18.412+0000 7ff9f1f05740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x55de38bdc290 con 0x55de388cf350 2024-09-21T03:53:18.415 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:18.412+0000 7ff9f1f05740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55de38be00a0 con 0x55de388ae670 2024-09-21T03:53:18.415 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:18.412+0000 7ff9e972d640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55de388a6490 0x55de38c76970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:53:18.415 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:18.412+0000 7ff9e8f2c640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55de388ae670 0x55de38c76eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:53:18.415 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:18.412+0000 7ff9e972d640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55de388a6490 0x55de38c76970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:57194/0 (socket says 172.21.15.39:57194) 2024-09-21T03:53:18.415 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:18.412+0000 7ff9e972d640 1 -- 172.21.15.39:0/851675068 learned_addr learned my addr 172.21.15.39:0/851675068 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:53:18.416 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:18.412+0000 7ff9e9f2e640 1 --2- 172.21.15.39:0/851675068 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55de388cf350 0x55de38c76430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:53:18.416 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:18.412+0000 7ff9e972d640 1 -- 172.21.15.39:0/851675068 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55de388ae670 msgr2=0x55de38c76eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:53:18.416 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:18.412+0000 7ff9e972d640 1 --2- 172.21.15.39:0/851675068 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55de388ae670 0x55de38c76eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:18.416 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:18.412+0000 7ff9e972d640 1 -- 172.21.15.39:0/851675068 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55de388cf350 msgr2=0x55de38c76430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:53:18.416 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:18.412+0000 7ff9e972d640 1 --2- 172.21.15.39:0/851675068 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55de388cf350 0x55de38c76430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:18.416 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:18.412+0000 7ff9e972d640 1 -- 172.21.15.39:0/851675068 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55de3893a930 con 0x55de388a6490 2024-09-21T03:53:18.416 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:18.413+0000 7ff9e27fc640 1 --2- 172.21.15.39:0/162926316 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7ff9cc0b6680 0x7ff9cc0b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:53:18.416 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:18.413+0000 7ff9e972d640 1 --2- 172.21.15.39:0/851675068 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55de388a6490 0x55de38c76970 secure :-1 s=READY pgs=1039 cs=0 l=1 rev1=1 crypto rx=0x7ff9dc0e5370 tx=0x7ff9dc0ebe20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=80d52bad52fa76b3 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:53:18.416 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:18.413+0000 7ff9e9f2e640 1 --2- 172.21.15.39:0/162926316 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7ff9cc0b6680 0x7ff9cc0b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:53:18.416 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:18.413+0000 7ff9cb7fe640 1 -- 172.21.15.39:0/851675068 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9dc0aa150 con 0x55de388a6490 2024-09-21T03:53:18.416 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:18.413+0000 7ff9cb7fe640 1 -- 172.21.15.39:0/851675068 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff9dc0047b0 con 0x55de388a6490 2024-09-21T03:53:18.416 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:18.413+0000 7ff9cb7fe640 1 -- 172.21.15.39:0/851675068 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9dc0b4b80 con 0x55de388a6490 2024-09-21T03:53:18.416 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:18.413+0000 7ff9e27fc640 1 -- 172.21.15.39:0/162926316 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8572+0+0 (secure 0 0 0) 0x7ff9dc0e7ce0 con 0x7ff9e4104890 2024-09-21T03:53:18.419 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:18.414+0000 7ff9f1f05740 1 -- 172.21.15.39:0/851675068 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55de388a6490 msgr2=0x55de38c76970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:53:18.419 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:18.414+0000 7ff9e9f2e640 1 --2- 172.21.15.39:0/162926316 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7ff9cc0b6680 0x7ff9cc0b8b90 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7ff9d4005e30 tx=0x7ff9d4005c30 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:53:18.419 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:18.414+0000 7ff9f1f05740 1 --2- 172.21.15.39:0/851675068 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55de388a6490 0x55de38c76970 secure :-1 s=READY pgs=1039 cs=0 l=1 rev1=1 crypto rx=0x7ff9dc0e5370 tx=0x7ff9dc0ebe20 comp rx=0 tx=0).stop 2024-09-21T03:53:18.419 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:18.414+0000 7ff9f1f05740 1 -- 172.21.15.39:0/851675068 shutdown_connections 2024-09-21T03:53:18.420 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:18.414+0000 7ff9f1f05740 1 --2- 172.21.15.39:0/851675068 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55de388ae670 0x55de38c76eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:18.420 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:18.414+0000 7ff9f1f05740 1 --2- 172.21.15.39:0/851675068 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55de388a6490 0x55de38c76970 unknown :-1 s=CLOSED pgs=1039 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:18.420 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:18.414+0000 7ff9f1f05740 1 --2- 172.21.15.39:0/851675068 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55de388cf350 0x55de38c76430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:18.420 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:18.414+0000 7ff9f1f05740 1 -- 172.21.15.39:0/851675068 >> 172.21.15.39:0/851675068 conn(0x55de388c3740 msgr2=0x55de38c74320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:53:18.420 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:18.414+0000 7ff9f1f05740 1 -- 172.21.15.39:0/851675068 shutdown_connections 2024-09-21T03:53:18.420 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:18.414+0000 7ff9f1f05740 1 -- 172.21.15.39:0/851675068 wait complete. 2024-09-21T03:53:18.420 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:18.414+0000 7ff9f1f05740 1 -- start start 2024-09-21T03:53:18.420 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:18.415+0000 7ff9f1f05740 1 Processor -- start 2024-09-21T03:53:18.420 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:18.415+0000 7ff9f1f05740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55de388a6490 0x55de38c81fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:53:18.420 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:18.415+0000 7ff9f1f05740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55de388ae670 0x55de38c824f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:53:18.420 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:18.415+0000 7ff9f1f05740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55de388cf350 0x55de38c92340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:53:18.420 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:18.415+0000 7ff9f1f05740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x55de38a3d890 con 0x55de388ae670 2024-09-21T03:53:18.420 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:18.415+0000 7ff9f1f05740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x55de38b76d40 con 0x55de388a6490 2024-09-21T03:53:18.420 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:18.415+0000 7ff9f1f05740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55de38c89020 con 0x55de388cf350 2024-09-21T03:53:18.421 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:18.415+0000 7ff9f1f05740 10 client.0 _ll_register_callbacks cb 0x55de38bdac80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:53:18.421 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:18.415+0000 7ff9e8f2c640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55de388ae670 0x55de38c824f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:53:18.421 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:18.415+0000 7ff9e8f2c640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55de388ae670 0x55de38c824f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:57198/0 (socket says 172.21.15.39:57198) 2024-09-21T03:53:18.421 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:18.415+0000 7ff9e8f2c640 1 -- 172.21.15.39:0/3544220305 learned_addr learned my addr 172.21.15.39:0/3544220305 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:53:18.421 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:18.415+0000 7ff9e972d640 1 --2- 172.21.15.39:0/3544220305 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55de388a6490 0x55de38c81fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:53:18.421 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:18.416+0000 7ff9e9f2e640 1 --2- 172.21.15.39:0/3544220305 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55de388cf350 0x55de38c92340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:53:18.421 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:18.416+0000 7ff9e8f2c640 1 -- 172.21.15.39:0/3544220305 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55de388cf350 msgr2=0x55de38c92340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:53:18.421 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:18.416+0000 7ff9e8f2c640 1 --2- 172.21.15.39:0/3544220305 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55de388cf350 0x55de38c92340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:18.421 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:18.416+0000 7ff9e8f2c640 1 -- 172.21.15.39:0/3544220305 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55de388a6490 msgr2=0x55de38c81fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:53:18.421 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:18.416+0000 7ff9e8f2c640 1 --2- 172.21.15.39:0/3544220305 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55de388a6490 0x55de38c81fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:18.421 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:18.416+0000 7ff9e8f2c640 1 -- 172.21.15.39:0/3544220305 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55de38c92880 con 0x55de388ae670 2024-09-21T03:53:18.421 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:18.416+0000 7ff9e9f2e640 1 --2- 172.21.15.39:0/3544220305 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55de388cf350 0x55de38c92340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:53:18.421 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:18.416+0000 7ff9e972d640 1 --2- 172.21.15.39:0/3544220305 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55de388a6490 0x55de38c81fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:53:18.421 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:18.416+0000 7ff9e8f2c640 1 --2- 172.21.15.39:0/3544220305 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55de388ae670 0x55de38c824f0 secure :-1 s=READY pgs=1040 cs=0 l=1 rev1=1 crypto rx=0x7ff9d800bf70 tx=0x7ff9d80128e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b2dd89fc91a8dd26 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:53:18.421 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:18.416+0000 7ff9c8ff9640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-09-21T03:53:18.421 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:18.416+0000 7ff9c8ff9640 1 -- 172.21.15.39:0/3544220305 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9d80035f0 con 0x55de388ae670 2024-09-21T03:53:18.422 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:18.416+0000 7ff9c8ff9640 1 -- 172.21.15.39:0/3544220305 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8572+0+0 (secure 0 0 0) 0x7ff9d8003790 con 0x55de388ae670 2024-09-21T03:53:18.422 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:18.416+0000 7ff9f1f05740 1 -- 172.21.15.39:0/3544220305 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff9dc0047b0 con 0x55de388ae670 2024-09-21T03:53:18.422 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:18.416+0000 7ff9c8ff9640 1 -- 172.21.15.39:0/3544220305 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7ff9d8003990 con 0x55de388ae670 2024-09-21T03:53:18.422 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:18.416+0000 7ff9f1f05740 10 client.17403 fetch_fsmap learned FSMap version 29 2024-09-21T03:53:18.422 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:18.416+0000 7ff9f1f05740 1 -- 172.21.15.39:0/3544220305 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55de38c951f0 con 0x55de388ae670 2024-09-21T03:53:18.422 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:18.417+0000 7ff9c8ff9640 1 -- 172.21.15.39:0/3544220305 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff9dc0047b0 con 0x55de388ae670 2024-09-21T03:53:18.422 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:18.417+0000 7ff9c8ff9640 1 -- 172.21.15.39:0/3544220305 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7ff9d8019070 con 0x55de388ae670 2024-09-21T03:53:18.422 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:18.417+0000 7ff9f1f05740 10 client.17403 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T03:53:18.422 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:18.417+0000 7ff9f1f05740 10 client.17403 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T03:53:18.422 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:18.417+0000 7ff9f1f05740 20 client.17403 populate_metadata read hostname 'smithi039' 2024-09-21T03:53:18.422 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:18.417+0000 7ff9f1f05740 1 --2- 172.21.15.39:0/3544220305 >> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] conn(0x55de3888dcd0 0x55de38c987c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:53:18.422 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:18.417+0000 7ff9f1f05740 4 client.17403 mds_command: new command op to 24695 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:53:18.422 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:18.417+0000 7ff9f1f05740 1 -- 172.21.15.39:0/3544220305 --> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55de38beedb0 con 0x55de3888dcd0 2024-09-21T03:53:18.422 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:18.417+0000 7ff9e972d640 1 --2- 172.21.15.39:0/3544220305 >> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] conn(0x55de3888dcd0 0x55de38c987c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:53:18.422 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:18.418+0000 7ff9e972d640 1 --2- 172.21.15.39:0/3544220305 >> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] conn(0x55de3888dcd0 0x55de38c987c0 crc :-1 s=READY pgs=123 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fa90de178f8a0c72 server_cookie=6b589b7e5f30646c in_seq=0 out_seq=0 2024-09-21T03:53:18.423 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:18.418+0000 7ff9c8ff9640 10 client.17403 ms_handle_connect on v2:172.21.15.150:6838/2949386907 2024-09-21T03:53:18.424 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:18.418+0000 7ff9c8ff9640 1 -- 172.21.15.39:0/3544220305 <== mds.0 v2:172.21.15.150:6838/2949386907 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55de38beedb0 con 0x55de3888dcd0 2024-09-21T03:53:18.424 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:18.418+0000 7ff9c8ff9640 10 client.17403 handle_command_reply: tid=0 2024-09-21T03:53:18.424 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:18.418+0000 7ff9f1f05740 1 client.17403 shutdown 2024-09-21T03:53:18.424 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:18.418+0000 7ff9f1f05740 1 -- 172.21.15.39:0/3544220305 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55de388ae670 msgr2=0x55de38c824f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:53:18.424 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:18.418+0000 7ff9f1f05740 1 --2- 172.21.15.39:0/3544220305 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55de388ae670 0x55de38c824f0 secure :-1 s=READY pgs=1040 cs=0 l=1 rev1=1 crypto rx=0x7ff9d800bf70 tx=0x7ff9d80128e0 comp rx=0 tx=0).stop 2024-09-21T03:53:18.424 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:18.419+0000 7ff9f1f05740 1 -- 172.21.15.39:0/3544220305 shutdown_connections 2024-09-21T03:53:18.424 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:18.419+0000 7ff9f1f05740 1 --2- 172.21.15.39:0/3544220305 >> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] conn(0x55de3888dcd0 0x55de38c987c0 crc :-1 s=READY pgs=123 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:18.424 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:18.419+0000 7ff9f1f05740 1 --2- 172.21.15.39:0/3544220305 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55de388cf350 0x55de38c92340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:18.424 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:18.419+0000 7ff9f1f05740 1 --2- 172.21.15.39:0/3544220305 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55de388ae670 0x55de38c824f0 unknown :-1 s=CLOSED pgs=1040 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:18.424 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:18.419+0000 7ff9f1f05740 1 --2- 172.21.15.39:0/3544220305 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55de388a6490 0x55de38c81fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:18.424 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:18.419+0000 7ff9f1f05740 1 -- 172.21.15.39:0/3544220305 >> 172.21.15.39:0/3544220305 conn(0x55de388c3740 msgr2=0x55de38c795c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:53:18.424 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:18.419+0000 7ff9c8ff9640 0 client.17403 ms_handle_reset on v2:172.21.15.150:6838/2949386907 2024-09-21T03:53:18.425 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:18.419+0000 7ff9f1f05740 1 -- 172.21.15.39:0/3544220305 shutdown_connections 2024-09-21T03:53:18.425 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:18.419+0000 7ff9f1f05740 1 -- 172.21.15.39:0/3544220305 wait complete. 2024-09-21T03:53:18.425 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:18.419+0000 7ff9f1f05740 20 client.17403 trim_cache size 0 max 16384 2024-09-21T03:53:18.440 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:18.437+0000 7ff9f1f05740 1 Processor -- start 2024-09-21T03:53:18.440 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:18.437+0000 7ff9f1f05740 1 -- start start 2024-09-21T03:53:18.440 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:18.437+0000 7ff9f1f05740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55de388a6490 0x55de38c73860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:53:18.440 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:18.437+0000 7ff9f1f05740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55de388ae670 0x55de38c73da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:53:18.440 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:18.437+0000 7ff9f1f05740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55de388cf350 0x55de38c7efa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:53:18.440 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:18.437+0000 7ff9f1f05740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7ff9e41ad640 con 0x55de388ae670 2024-09-21T03:53:18.440 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:18.437+0000 7ff9f1f05740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7ff9e41af180 con 0x55de388a6490 2024-09-21T03:53:18.440 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:18.437+0000 7ff9f1f05740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7ff9e41add10 con 0x55de388cf350 2024-09-21T03:53:18.440 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:18.437+0000 7ff9e8f2c640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55de388ae670 0x55de38c73da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:53:18.440 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:18.437+0000 7ff9e8f2c640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55de388ae670 0x55de38c73da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:57206/0 (socket says 172.21.15.39:57206) 2024-09-21T03:53:18.441 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:18.437+0000 7ff9e8f2c640 1 -- 172.21.15.39:0/1003481848 learned_addr learned my addr 172.21.15.39:0/1003481848 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:53:18.441 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:18.437+0000 7ff9e972d640 1 --2- 172.21.15.39:0/1003481848 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55de388a6490 0x55de38c73860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:53:18.441 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:18.437+0000 7ff9e9f2e640 1 --2- 172.21.15.39:0/1003481848 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55de388cf350 0x55de38c7efa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:53:18.441 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:18.438+0000 7ff9e8f2c640 1 -- 172.21.15.39:0/1003481848 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55de388cf350 msgr2=0x55de38c7efa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:53:18.441 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:18.438+0000 7ff9e8f2c640 1 --2- 172.21.15.39:0/1003481848 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55de388cf350 0x55de38c7efa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:18.441 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:18.438+0000 7ff9e8f2c640 1 -- 172.21.15.39:0/1003481848 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55de388a6490 msgr2=0x55de38c73860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:53:18.441 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:18.438+0000 7ff9e8f2c640 1 --2- 172.21.15.39:0/1003481848 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55de388a6490 0x55de38c73860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:18.441 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:18.438+0000 7ff9e8f2c640 1 -- 172.21.15.39:0/1003481848 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55de38c80d10 con 0x55de388ae670 2024-09-21T03:53:18.441 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:18.438+0000 7ff9e9f2e640 1 --2- 172.21.15.39:0/1003481848 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55de388cf350 0x55de38c7efa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:53:18.441 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:18.438+0000 7ff9e8f2c640 1 --2- 172.21.15.39:0/1003481848 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55de388ae670 0x55de38c73da0 secure :-1 s=READY pgs=1041 cs=0 l=1 rev1=1 crypto rx=0x7ff9d800bf40 tx=0x7ff9d8004510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=95c547aabeb59d9a server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:53:18.441 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:18.438+0000 7ff9caffd640 1 -- 172.21.15.39:0/1003481848 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9d8030070 con 0x55de388ae670 2024-09-21T03:53:18.441 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:18.438+0000 7ff9caffd640 1 -- 172.21.15.39:0/1003481848 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff9d8005040 con 0x55de388ae670 2024-09-21T03:53:18.441 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:18.438+0000 7ff9caffd640 1 -- 172.21.15.39:0/1003481848 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9d8010920 con 0x55de388ae670 2024-09-21T03:53:18.444 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:18.439+0000 7ff9f1f05740 1 -- 172.21.15.39:0/1003481848 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55de388ae670 msgr2=0x55de38c73da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:53:18.444 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:18.439+0000 7ff9f1f05740 1 --2- 172.21.15.39:0/1003481848 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55de388ae670 0x55de38c73da0 secure :-1 s=READY pgs=1041 cs=0 l=1 rev1=1 crypto rx=0x7ff9d800bf40 tx=0x7ff9d8004510 comp rx=0 tx=0).stop 2024-09-21T03:53:18.444 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:18.439+0000 7ff9f1f05740 1 -- 172.21.15.39:0/1003481848 shutdown_connections 2024-09-21T03:53:18.444 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:18.439+0000 7ff9f1f05740 1 --2- 172.21.15.39:0/1003481848 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55de388cf350 0x55de38c7efa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:18.444 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:18.439+0000 7ff9f1f05740 1 --2- 172.21.15.39:0/1003481848 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55de388ae670 0x55de38c73da0 unknown :-1 s=CLOSED pgs=1041 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:18.444 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:18.439+0000 7ff9f1f05740 1 --2- 172.21.15.39:0/1003481848 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55de388a6490 0x55de38c73860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:18.445 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:18.439+0000 7ff9f1f05740 1 -- 172.21.15.39:0/1003481848 >> 172.21.15.39:0/1003481848 conn(0x55de388c3740 msgr2=0x55de38c7cf60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:53:18.445 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:18.439+0000 7ff9f1f05740 1 -- 172.21.15.39:0/1003481848 shutdown_connections 2024-09-21T03:53:18.445 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:18.439+0000 7ff9f1f05740 1 -- 172.21.15.39:0/1003481848 wait complete. 2024-09-21T03:53:18.445 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:18.439+0000 7ff9f1f05740 1 -- start start 2024-09-21T03:53:18.445 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:18.439+0000 7ff9f1f05740 1 Processor -- start 2024-09-21T03:53:18.445 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:18.440+0000 7ff9f1f05740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55de388a6490 0x55de38c8ec60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:53:18.445 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:18.440+0000 7ff9f1f05740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55de388ae670 0x55de38c8b170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:53:18.445 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:18.440+0000 7ff9f1f05740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55de388cf350 0x55de38c8b880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:53:18.445 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:18.440+0000 7ff9f1f05740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7ff9e41aeab0 con 0x55de388ae670 2024-09-21T03:53:18.445 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:18.440+0000 7ff9f1f05740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7ff9e41ae3e0 con 0x55de388a6490 2024-09-21T03:53:18.445 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:18.440+0000 7ff9f1f05740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7ff9e41ab640 con 0x55de388cf350 2024-09-21T03:53:18.445 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:18.440+0000 7ff9f1f05740 10 client.0 _ll_register_callbacks cb 0x7ff9d800b8e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:53:18.445 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:18.440+0000 7ff9e8f2c640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55de388ae670 0x55de38c8b170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:53:18.445 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:18.440+0000 7ff9e8f2c640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55de388ae670 0x55de38c8b170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:57210/0 (socket says 172.21.15.39:57210) 2024-09-21T03:53:18.446 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:18.440+0000 7ff9e8f2c640 1 -- 172.21.15.39:0/589397203 learned_addr learned my addr 172.21.15.39:0/589397203 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:53:18.446 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:18.440+0000 7ff9e9f2e640 1 --2- 172.21.15.39:0/589397203 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55de388cf350 0x55de38c8b880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:53:18.446 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:18.440+0000 7ff9e972d640 1 --2- 172.21.15.39:0/589397203 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55de388a6490 0x55de38c8ec60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:53:18.446 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:18.440+0000 7ff9e8f2c640 1 -- 172.21.15.39:0/589397203 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55de388cf350 msgr2=0x55de38c8b880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:53:18.446 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:18.440+0000 7ff9e8f2c640 1 --2- 172.21.15.39:0/589397203 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55de388cf350 0x55de38c8b880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:18.446 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:18.440+0000 7ff9e8f2c640 1 -- 172.21.15.39:0/589397203 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55de388a6490 msgr2=0x55de38c8ec60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:53:18.446 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:18.440+0000 7ff9e8f2c640 1 --2- 172.21.15.39:0/589397203 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55de388a6490 0x55de38c8ec60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:18.446 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:18.440+0000 7ff9e8f2c640 1 -- 172.21.15.39:0/589397203 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55de38d031d0 con 0x55de388ae670 2024-09-21T03:53:18.446 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:18.441+0000 7ff9e8f2c640 1 --2- 172.21.15.39:0/589397203 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55de388ae670 0x55de38c8b170 secure :-1 s=READY pgs=1042 cs=0 l=1 rev1=1 crypto rx=0x7ff9d802e040 tx=0x7ff9d80038a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d074c6d40399696 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:53:18.446 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:18.441+0000 7ff9c8ff9640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-09-21T03:53:18.446 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:18.441+0000 7ff9c8ff9640 1 -- 172.21.15.39:0/589397203 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9d8030070 con 0x55de388ae670 2024-09-21T03:53:18.446 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:18.441+0000 7ff9c8ff9640 1 -- 172.21.15.39:0/589397203 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8572+0+0 (secure 0 0 0) 0x7ff9d80051f0 con 0x55de388ae670 2024-09-21T03:53:18.446 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:18.441+0000 7ff9f1f05740 1 -- 172.21.15.39:0/589397203 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff9d8005040 con 0x55de388ae670 2024-09-21T03:53:18.447 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:18.441+0000 7ff9c8ff9640 1 -- 172.21.15.39:0/589397203 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7ff9d8005450 con 0x55de388ae670 2024-09-21T03:53:18.447 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:18.441+0000 7ff9f1f05740 10 client.17409 fetch_fsmap learned FSMap version 29 2024-09-21T03:53:18.447 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:18.441+0000 7ff9f1f05740 1 -- 172.21.15.39:0/589397203 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55de38d04330 con 0x55de388ae670 2024-09-21T03:53:18.447 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:18.441+0000 7ff9c8ff9640 1 -- 172.21.15.39:0/589397203 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff9d8005040 con 0x55de388ae670 2024-09-21T03:53:18.447 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:18.441+0000 7ff9c8ff9640 1 -- 172.21.15.39:0/589397203 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7ff9d8019040 con 0x55de388ae670 2024-09-21T03:53:18.447 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:18.442+0000 7ff9f1f05740 10 client.17409 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T03:53:18.447 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:18.442+0000 7ff9f1f05740 10 client.17409 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T03:53:18.447 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:18.442+0000 7ff9f1f05740 20 client.17409 populate_metadata read hostname 'smithi039' 2024-09-21T03:53:18.447 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:18.442+0000 7ff9f1f05740 1 --2- 172.21.15.39:0/589397203 >> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] conn(0x55de38863080 0x55de38d06a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:53:18.447 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:18.442+0000 7ff9f1f05740 4 client.17409 mds_command: new command op to 24695 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-21T03:53:18.447 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:18.442+0000 7ff9f1f05740 1 -- 172.21.15.39:0/589397203 --> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55de38bea860 con 0x55de38863080 2024-09-21T03:53:18.447 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:18.442+0000 7ff9e972d640 1 --2- 172.21.15.39:0/589397203 >> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] conn(0x55de38863080 0x55de38d06a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:53:18.447 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:18.442+0000 7ff9c8ff9640 10 client.17409 ms_handle_connect on v2:172.21.15.150:6838/2949386907 2024-09-21T03:53:18.447 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:18.442+0000 7ff9e972d640 1 --2- 172.21.15.39:0/589397203 >> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] conn(0x55de38863080 0x55de38d06a40 crc :-1 s=READY pgs=124 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2a6dd55c77eeb10f server_cookie=9b09f2eb68312526 in_seq=0 out_seq=0 2024-09-21T03:53:18.448 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:18.445+0000 7ff9c8ff9640 1 -- 172.21.15.39:0/589397203 <== mds.0 v2:172.21.15.150:6838/2949386907 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55de38bea860 con 0x55de38863080 2024-09-21T03:53:18.448 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:18.445+0000 7ff9c8ff9640 10 client.17409 handle_command_reply: tid=0 2024-09-21T03:53:18.449 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:18.445+0000 7ff9f1f05740 1 client.17409 shutdown 2024-09-21T03:53:18.449 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:18.445+0000 7ff9f1f05740 1 -- 172.21.15.39:0/589397203 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55de388ae670 msgr2=0x55de38c8b170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:53:18.449 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:18.445+0000 7ff9f1f05740 1 --2- 172.21.15.39:0/589397203 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55de388ae670 0x55de38c8b170 secure :-1 s=READY pgs=1042 cs=0 l=1 rev1=1 crypto rx=0x7ff9d802e040 tx=0x7ff9d80038a0 comp rx=0 tx=0).stop 2024-09-21T03:53:18.449 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:18.445+0000 7ff9f1f05740 1 -- 172.21.15.39:0/589397203 shutdown_connections 2024-09-21T03:53:18.449 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:18.445+0000 7ff9f1f05740 1 --2- 172.21.15.39:0/589397203 >> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] conn(0x55de38863080 0x55de38d06a40 crc :-1 s=READY pgs=124 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:18.449 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:18.445+0000 7ff9f1f05740 1 --2- 172.21.15.39:0/589397203 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55de388cf350 0x55de38c8b880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:18.449 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:18.445+0000 7ff9f1f05740 1 --2- 172.21.15.39:0/589397203 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55de388ae670 0x55de38c8b170 unknown :-1 s=CLOSED pgs=1042 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:18.449 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:18.445+0000 7ff9c8ff9640 0 client.17409 ms_handle_reset on v2:172.21.15.150:6838/2949386907 2024-09-21T03:53:18.449 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:18.445+0000 7ff9f1f05740 1 --2- 172.21.15.39:0/589397203 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55de388a6490 0x55de38c8ec60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:18.449 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:18.445+0000 7ff9f1f05740 1 -- 172.21.15.39:0/589397203 >> 172.21.15.39:0/589397203 conn(0x55de388c3740 msgr2=0x55de38c7cf60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:53:18.449 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:18.445+0000 7ff9f1f05740 1 -- 172.21.15.39:0/589397203 shutdown_connections 2024-09-21T03:53:18.449 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:18.445+0000 7ff9f1f05740 1 -- 172.21.15.39:0/589397203 wait complete. 2024-09-21T03:53:18.449 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:18.445+0000 7ff9f1f05740 20 client.17409 trim_cache size 0 max 16384 2024-09-21T03:53:18.449 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-09-21T03:53:18.450 INFO:teuthology.orchestra.run.smithi039.stdout: "return_code": 0, 2024-09-21T03:53:18.450 INFO:teuthology.orchestra.run.smithi039.stdout: "scrub_tag": "ad87da79-0248-4e18-9694-8228cb95422f", 2024-09-21T03:53:18.450 INFO:teuthology.orchestra.run.smithi039.stdout: "mode": "asynchronous" 2024-09-21T03:53:18.450 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-09-21T03:53:18.450 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:18.446+0000 7ff9eb991640 1 -- 172.21.15.39:0/162926316 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7ff9cc0b6680 msgr2=0x7ff9cc0b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:53:18.450 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:18.446+0000 7ff9eb991640 1 --2- 172.21.15.39:0/162926316 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7ff9cc0b6680 0x7ff9cc0b8b90 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7ff9d4005e30 tx=0x7ff9d4005c30 comp rx=0 tx=0).stop 2024-09-21T03:53:18.450 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:18.446+0000 7ff9eb991640 1 -- 172.21.15.39:0/162926316 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff9e4104890 msgr2=0x7ff9e419a450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:53:18.451 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:18.446+0000 7ff9eb991640 1 --2- 172.21.15.39:0/162926316 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff9e4104890 0x7ff9e419a450 secure :-1 s=READY pgs=1038 cs=0 l=1 rev1=1 crypto rx=0x7ff9dc00e9f0 tx=0x7ff9dc00edf0 comp rx=0 tx=0).stop 2024-09-21T03:53:18.451 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:18.446+0000 7ff9eb991640 1 -- 172.21.15.39:0/162926316 shutdown_connections 2024-09-21T03:53:18.451 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:18.446+0000 7ff9eb991640 1 --2- 172.21.15.39:0/162926316 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7ff9cc0b6680 0x7ff9cc0b8b90 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:18.451 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:18.446+0000 7ff9eb991640 1 --2- 172.21.15.39:0/162926316 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff9e410de40 0x7ff9e419fc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:18.451 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:18.446+0000 7ff9eb991640 1 --2- 172.21.15.39:0/162926316 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff9e41052c0 0x7ff9e419a9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:18.451 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:18.446+0000 7ff9eb991640 1 --2- 172.21.15.39:0/162926316 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff9e4104890 0x7ff9e419a450 unknown :-1 s=CLOSED pgs=1038 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:18.451 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:18.446+0000 7ff9eb991640 1 -- 172.21.15.39:0/162926316 >> 172.21.15.39:0/162926316 conn(0x7ff9e4079710 msgr2=0x7ff9e41006d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:53:18.451 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:18.446+0000 7ff9eb991640 1 -- 172.21.15.39:0/162926316 shutdown_connections 2024-09-21T03:53:18.451 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:18.446+0000 7ff9eb991640 1 -- 172.21.15.39:0/162926316 wait complete. 2024-09-21T03:53:19.331 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 51116d5c-77ca-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub status 2024-09-21T03:53:19.584 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:53:19 smithi039 ceph-mon[31211]: pgmap v544: 97 pgs: 97 active+clean; 10 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-09-21T03:53:19.584 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:53:19 smithi039 ceph-mon[31211]: scrub queued for path: / 2024-09-21T03:53:19.584 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:53:19 smithi039 ceph-mon[31211]: scrub summary: idle+waiting paths [/] 2024-09-21T03:53:19.584 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:53:19 smithi039 ceph-mon[31211]: scrub summary: active paths [/] 2024-09-21T03:53:19.584 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:53:19 smithi039 ceph-mon[31211]: scrub summary: idle+waiting paths [/] 2024-09-21T03:53:19.637 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/51116d5c-77ca-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:53:19.718 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:53:19 smithi161 ceph-mon[34533]: pgmap v544: 97 pgs: 97 active+clean; 10 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-09-21T03:53:19.718 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:53:19 smithi161 ceph-mon[34533]: scrub queued for path: / 2024-09-21T03:53:19.718 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:53:19 smithi161 ceph-mon[34533]: scrub summary: idle+waiting paths [/] 2024-09-21T03:53:19.718 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:53:19 smithi161 ceph-mon[34533]: scrub summary: active paths [/] 2024-09-21T03:53:19.718 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:53:19 smithi161 ceph-mon[34533]: scrub summary: idle+waiting paths [/] 2024-09-21T03:53:19.725 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:53:19 smithi150 ceph-mon[34289]: pgmap v544: 97 pgs: 97 active+clean; 10 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-09-21T03:53:19.725 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:53:19 smithi150 ceph-mon[34289]: scrub queued for path: / 2024-09-21T03:53:19.725 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:53:19 smithi150 ceph-mon[34289]: scrub summary: idle+waiting paths [/] 2024-09-21T03:53:19.725 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:53:19 smithi150 ceph-mon[34289]: scrub summary: active paths [/] 2024-09-21T03:53:19.725 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:53:19 smithi150 ceph-mon[34289]: scrub summary: idle+waiting paths [/] 2024-09-21T03:53:20.133 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:20.120+0000 7f8a9a13f640 1 -- 172.21.15.39:0/1272510175 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8a94106c40 msgr2=0x7f8a941106b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:53:20.134 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:20.120+0000 7f8a9a13f640 1 --2- 172.21.15.39:0/1272510175 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8a94106c40 0x7f8a941106b0 secure :-1 s=READY pgs=1043 cs=0 l=1 rev1=1 crypto rx=0x7f8a8c009ca0 tx=0x7f8a8c02a8c0 comp rx=0 tx=0).stop 2024-09-21T03:53:20.134 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:20.120+0000 7f8a9a13f640 1 -- 172.21.15.39:0/1272510175 shutdown_connections 2024-09-21T03:53:20.134 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:20.120+0000 7f8a9a13f640 1 --2- 172.21.15.39:0/1272510175 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8a94106c40 0x7f8a941106b0 unknown :-1 s=CLOSED pgs=1043 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:20.134 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:20.120+0000 7f8a9a13f640 1 --2- 172.21.15.39:0/1272510175 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8a941062f0 0x7f8a94106700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:20.134 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:20.120+0000 7f8a9a13f640 1 --2- 172.21.15.39:0/1272510175 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8a941020a0 0x7f8a941024b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:20.134 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:20.120+0000 7f8a9a13f640 1 -- 172.21.15.39:0/1272510175 >> 172.21.15.39:0/1272510175 conn(0x7f8a94079f90 msgr2=0x7f8a9407a3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:53:20.134 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:20.120+0000 7f8a9a13f640 1 -- 172.21.15.39:0/1272510175 shutdown_connections 2024-09-21T03:53:20.134 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:20.120+0000 7f8a9a13f640 1 -- 172.21.15.39:0/1272510175 wait complete. 2024-09-21T03:53:20.134 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:20.121+0000 7f8a9a13f640 1 Processor -- start 2024-09-21T03:53:20.134 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:20.121+0000 7f8a9a13f640 1 -- start start 2024-09-21T03:53:20.134 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:20.121+0000 7f8a9a13f640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8a941020a0 0x7f8a94103040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:53:20.135 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:20.121+0000 7f8a9a13f640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8a941062f0 0x7f8a94103580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:53:20.135 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:20.121+0000 7f8a9a13f640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8a94106c40 0x7f8a94103ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:53:20.135 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:20.121+0000 7f8a9a13f640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f8a94113b10 con 0x7f8a94106c40 2024-09-21T03:53:20.135 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:20.121+0000 7f8a9a13f640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f8a94113990 con 0x7f8a941020a0 2024-09-21T03:53:20.135 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:20.121+0000 7f8a9a13f640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f8a94113c90 con 0x7f8a941062f0 2024-09-21T03:53:20.135 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:20.121+0000 7f8a93fff640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8a94106c40 0x7f8a94103ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:53:20.135 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:20.121+0000 7f8a93fff640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8a94106c40 0x7f8a94103ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:57224/0 (socket says 172.21.15.39:57224) 2024-09-21T03:53:20.135 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:20.121+0000 7f8a93fff640 1 -- 172.21.15.39:0/3268722324 learned_addr learned my addr 172.21.15.39:0/3268722324 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:53:20.135 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:20.121+0000 7f8a937fe640 1 --2- 172.21.15.39:0/3268722324 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8a941020a0 0x7f8a94103040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:53:20.135 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:20.122+0000 7f8a92ffd640 1 --2- 172.21.15.39:0/3268722324 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8a941062f0 0x7f8a94103580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:53:20.135 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:20.122+0000 7f8a93fff640 1 -- 172.21.15.39:0/3268722324 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8a941062f0 msgr2=0x7f8a94103580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:53:20.135 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:20.122+0000 7f8a93fff640 1 --2- 172.21.15.39:0/3268722324 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8a941062f0 0x7f8a94103580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:20.135 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:20.122+0000 7f8a93fff640 1 -- 172.21.15.39:0/3268722324 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8a941020a0 msgr2=0x7f8a94103040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:53:20.136 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:20.122+0000 7f8a93fff640 1 --2- 172.21.15.39:0/3268722324 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8a941020a0 0x7f8a94103040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:20.136 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:20.122+0000 7f8a93fff640 1 -- 172.21.15.39:0/3268722324 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8a941a8740 con 0x7f8a94106c40 2024-09-21T03:53:20.136 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:20.122+0000 7f8a93fff640 1 --2- 172.21.15.39:0/3268722324 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8a94106c40 0x7f8a94103ac0 secure :-1 s=READY pgs=1044 cs=0 l=1 rev1=1 crypto rx=0x7f8a8c0098d0 tx=0x7f8a8c031770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:53:20.136 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:20.122+0000 7f8a90ff9640 1 -- 172.21.15.39:0/3268722324 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a8c003b70 con 0x7f8a94106c40 2024-09-21T03:53:20.136 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:20.122+0000 7f8a9a13f640 1 -- 172.21.15.39:0/3268722324 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8a941a89d0 con 0x7f8a94106c40 2024-09-21T03:53:20.136 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:20.122+0000 7f8a90ff9640 1 -- 172.21.15.39:0/3268722324 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8a8c003d10 con 0x7f8a94106c40 2024-09-21T03:53:20.136 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:20.122+0000 7f8a90ff9640 1 -- 172.21.15.39:0/3268722324 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a8c02fa00 con 0x7f8a94106c40 2024-09-21T03:53:20.136 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:20.122+0000 7f8a9a13f640 1 -- 172.21.15.39:0/3268722324 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8a941a8f10 con 0x7f8a94106c40 2024-09-21T03:53:20.136 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:20.125+0000 7f8aa06b3740 1 Processor -- start 2024-09-21T03:53:20.136 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:20.125+0000 7f8a90ff9640 1 -- 172.21.15.39:0/3268722324 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 27) ==== 154643+0+0 (secure 0 0 0) 0x7f8a8c02fc20 con 0x7f8a94106c40 2024-09-21T03:53:20.136 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:20.125+0000 7f8aa06b3740 1 -- start start 2024-09-21T03:53:20.136 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:20.125+0000 7f8aa06b3740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55eb4fefc070 0x55eb502a3270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:53:20.136 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:20.125+0000 7f8aa06b3740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55eb4fed31b0 0x55eb502a37b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:53:20.136 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:20.125+0000 7f8aa06b3740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55eb4fedb390 0x55eb502a3cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:53:20.136 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:20.125+0000 7f8aa06b3740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x55eb50219e50 con 0x55eb4fedb390 2024-09-21T03:53:20.137 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:20.125+0000 7f8aa06b3740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x55eb50209280 con 0x55eb4fed31b0 2024-09-21T03:53:20.137 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:20.125+0000 7f8aa06b3740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55eb5020d090 con 0x55eb4fefc070 2024-09-21T03:53:20.137 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:20.126+0000 7f8a92ffd640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55eb4fedb390 0x55eb502a3cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:53:20.137 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:20.126+0000 7f8a92ffd640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55eb4fedb390 0x55eb502a3cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:57236/0 (socket says 172.21.15.39:57236) 2024-09-21T03:53:20.137 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:20.126+0000 7f8a92ffd640 1 -- 172.21.15.39:0/2840363769 learned_addr learned my addr 172.21.15.39:0/2840363769 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:53:20.137 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:20.126+0000 7f8a92ffd640 1 --2- 172.21.15.39:0/2840363769 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55eb4fefc070 0x55eb502a3270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:53:20.137 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:20.126+0000 7f8a937fe640 1 --2- 172.21.15.39:0/2840363769 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55eb4fed31b0 0x55eb502a37b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:53:20.137 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:20.126+0000 7f8a90ff9640 1 --2- 172.21.15.39:0/3268722324 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f8a7c0b6680 0x7f8a7c0b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:53:20.137 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:20.126+0000 7f8a92ffd640 1 -- 172.21.15.39:0/2840363769 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55eb4fefc070 msgr2=0x55eb502a3270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:53:20.137 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:20.126+0000 7f8a92ffd640 1 --2- 172.21.15.39:0/2840363769 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55eb4fefc070 0x55eb502a3270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:20.137 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:20.126+0000 7f8a92ffd640 1 -- 172.21.15.39:0/2840363769 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55eb4fed31b0 msgr2=0x55eb502a37b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:53:20.137 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:20.126+0000 7f8a92ffd640 1 --2- 172.21.15.39:0/2840363769 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55eb4fed31b0 0x55eb502a37b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:20.138 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:20.126+0000 7f8a92ffd640 1 -- 172.21.15.39:0/2840363769 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55eb4ff67650 con 0x55eb4fedb390 2024-09-21T03:53:20.138 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:20.126+0000 7f8a937fe640 1 --2- 172.21.15.39:0/2840363769 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55eb4fed31b0 0x55eb502a37b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:53:20.138 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:20.126+0000 7f8a93fff640 1 --2- 172.21.15.39:0/3268722324 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f8a7c0b6680 0x7f8a7c0b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:53:20.138 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:20.126+0000 7f8a90ff9640 1 -- 172.21.15.39:0/3268722324 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8528+0+0 (secure 0 0 0) 0x7f8a8c108a20 con 0x7f8a94106c40 2024-09-21T03:53:20.138 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:20.126+0000 7f8a92ffd640 1 --2- 172.21.15.39:0/2840363769 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55eb4fedb390 0x55eb502a3cf0 secure :-1 s=READY pgs=1045 cs=0 l=1 rev1=1 crypto rx=0x7f8a88014960 tx=0x7f8a8800c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f79e3eeb8df482b4 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:53:20.138 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:20.127+0000 7f8a79ffb640 1 -- 172.21.15.39:0/2840363769 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a880142d0 con 0x55eb4fedb390 2024-09-21T03:53:20.138 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:20.127+0000 7f8a79ffb640 1 -- 172.21.15.39:0/2840363769 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8a8800fc10 con 0x55eb4fedb390 2024-09-21T03:53:20.138 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:20.127+0000 7f8a93fff640 1 --2- 172.21.15.39:0/3268722324 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f8a7c0b6680 0x7f8a7c0b8b90 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f8a8c0cbae0 tx=0x7f8a8c0f2040 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:53:20.138 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:20.127+0000 7f8a79ffb640 1 -- 172.21.15.39:0/2840363769 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a88012960 con 0x55eb4fedb390 2024-09-21T03:53:20.138 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:20.127+0000 7f8aa06b3740 1 -- 172.21.15.39:0/2840363769 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55eb4fedb390 msgr2=0x55eb502a3cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:53:20.138 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:20.127+0000 7f8aa06b3740 1 --2- 172.21.15.39:0/2840363769 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55eb4fedb390 0x55eb502a3cf0 secure :-1 s=READY pgs=1045 cs=0 l=1 rev1=1 crypto rx=0x7f8a88014960 tx=0x7f8a8800c490 comp rx=0 tx=0).stop 2024-09-21T03:53:20.138 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:20.128+0000 7f8aa06b3740 1 -- 172.21.15.39:0/2840363769 shutdown_connections 2024-09-21T03:53:20.139 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:20.128+0000 7f8aa06b3740 1 --2- 172.21.15.39:0/2840363769 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55eb4fedb390 0x55eb502a3cf0 unknown :-1 s=CLOSED pgs=1045 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:20.139 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:20.128+0000 7f8aa06b3740 1 --2- 172.21.15.39:0/2840363769 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55eb4fed31b0 0x55eb502a37b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:20.139 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:20.128+0000 7f8aa06b3740 1 --2- 172.21.15.39:0/2840363769 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55eb4fefc070 0x55eb502a3270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:20.139 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:20.128+0000 7f8aa06b3740 1 -- 172.21.15.39:0/2840363769 >> 172.21.15.39:0/2840363769 conn(0x55eb4fef0460 msgr2=0x55eb502a1160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:53:20.139 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:20.128+0000 7f8aa06b3740 1 -- 172.21.15.39:0/2840363769 shutdown_connections 2024-09-21T03:53:20.139 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:20.128+0000 7f8aa06b3740 1 -- 172.21.15.39:0/2840363769 wait complete. 2024-09-21T03:53:20.139 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:20.128+0000 7f8aa06b3740 1 -- start start 2024-09-21T03:53:20.139 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:20.128+0000 7f8aa06b3740 1 Processor -- start 2024-09-21T03:53:20.139 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:20.129+0000 7f8aa06b3740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55eb4fed31b0 0x55eb502ae880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:53:20.139 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:20.129+0000 7f8aa06b3740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55eb4fedb390 0x55eb502aedc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:53:20.139 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:20.129+0000 7f8aa06b3740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55eb4fefc070 0x55eb502af300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:53:20.139 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:20.129+0000 7f8aa06b3740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x55eb5006a700 con 0x55eb4fed31b0 2024-09-21T03:53:20.140 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:20.129+0000 7f8aa06b3740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x55eb501a4110 con 0x55eb4fedb390 2024-09-21T03:53:20.140 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:20.129+0000 7f8aa06b3740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55eb502b1a50 con 0x55eb4fefc070 2024-09-21T03:53:20.140 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:20.129+0000 7f8aa06b3740 10 client.0 _ll_register_callbacks cb 0x55eb501bd990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:53:20.140 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:20.129+0000 7f8a92ffd640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55eb4fed31b0 0x55eb502ae880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:53:20.140 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:20.129+0000 7f8a92ffd640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55eb4fed31b0 0x55eb502ae880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:57248/0 (socket says 172.21.15.39:57248) 2024-09-21T03:53:20.140 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:20.129+0000 7f8a92ffd640 1 -- 172.21.15.39:0/1581962678 learned_addr learned my addr 172.21.15.39:0/1581962678 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:53:20.140 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:20.129+0000 7f8a937fe640 1 --2- 172.21.15.39:0/1581962678 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55eb4fedb390 0x55eb502aedc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:53:20.140 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:20.129+0000 7f8a92ffd640 1 --2- 172.21.15.39:0/1581962678 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55eb4fefc070 0x55eb502af300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:53:20.140 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:20.129+0000 7f8a92ffd640 1 -- 172.21.15.39:0/1581962678 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55eb4fefc070 msgr2=0x55eb502af300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:53:20.140 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:20.129+0000 7f8a92ffd640 1 --2- 172.21.15.39:0/1581962678 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55eb4fefc070 0x55eb502af300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:20.140 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:20.129+0000 7f8a92ffd640 1 -- 172.21.15.39:0/1581962678 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55eb4fedb390 msgr2=0x55eb502aedc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:53:20.140 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:20.129+0000 7f8a92ffd640 1 --2- 172.21.15.39:0/1581962678 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55eb4fedb390 0x55eb502aedc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:20.140 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:20.129+0000 7f8a92ffd640 1 -- 172.21.15.39:0/1581962678 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55eb501d1c40 con 0x55eb4fed31b0 2024-09-21T03:53:20.141 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:20.129+0000 7f8a92ffd640 1 --2- 172.21.15.39:0/1581962678 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55eb4fed31b0 0x55eb502ae880 secure :-1 s=READY pgs=1046 cs=0 l=1 rev1=1 crypto rx=0x7f8a8800c430 tx=0x7f8a8800cf80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c11846ae3d32ceda server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:53:20.141 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:20.129+0000 7f8a72ffd640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-09-21T03:53:20.141 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:20.129+0000 7f8a72ffd640 1 -- 172.21.15.39:0/1581962678 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a8800f960 con 0x55eb4fed31b0 2024-09-21T03:53:20.141 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:20.130+0000 7f8a72ffd640 1 -- 172.21.15.39:0/1581962678 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8528+0+0 (secure 0 0 0) 0x7f8a88004030 con 0x55eb4fed31b0 2024-09-21T03:53:20.141 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:20.130+0000 7f8aa06b3740 1 -- 172.21.15.39:0/1581962678 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8a8800fc10 con 0x55eb4fed31b0 2024-09-21T03:53:20.141 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:20.130+0000 7f8a72ffd640 1 -- 172.21.15.39:0/1581962678 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f8a88032050 con 0x55eb4fed31b0 2024-09-21T03:53:20.141 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:20.130+0000 7f8aa06b3740 10 client.17421 fetch_fsmap learned FSMap version 29 2024-09-21T03:53:20.141 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:20.130+0000 7f8aa06b3740 1 -- 172.21.15.39:0/1581962678 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55eb502bd6e0 con 0x55eb4fed31b0 2024-09-21T03:53:20.141 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:20.130+0000 7f8a72ffd640 1 -- 172.21.15.39:0/1581962678 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8a8800fc10 con 0x55eb4fed31b0 2024-09-21T03:53:20.141 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:20.130+0000 7f8a72ffd640 1 -- 172.21.15.39:0/1581962678 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f8a880046c0 con 0x55eb4fed31b0 2024-09-21T03:53:20.141 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:20.130+0000 7f8aa06b3740 10 client.17421 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T03:53:20.141 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:20.130+0000 7f8aa06b3740 10 client.17421 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T03:53:20.141 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:20.130+0000 7f8aa06b3740 20 client.17421 populate_metadata read hostname 'smithi039' 2024-09-21T03:53:20.141 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:20.130+0000 7f8aa06b3740 1 --2- 172.21.15.39:0/1581962678 >> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] conn(0x55eb4feba9f0 0x55eb502c0a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:53:20.141 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:20.130+0000 7f8aa06b3740 4 client.17421 mds_command: new command op to 24695 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:53:20.141 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:20.130+0000 7f8aa06b3740 1 -- 172.21.15.39:0/1581962678 --> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55eb5021bda0 con 0x55eb4feba9f0 2024-09-21T03:53:20.142 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:20.131+0000 7f8a937fe640 1 --2- 172.21.15.39:0/1581962678 >> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] conn(0x55eb4feba9f0 0x55eb502c0a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:53:20.142 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:20.131+0000 7f8a72ffd640 10 client.17421 ms_handle_connect on v2:172.21.15.150:6838/2949386907 2024-09-21T03:53:20.142 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:20.131+0000 7f8a937fe640 1 --2- 172.21.15.39:0/1581962678 >> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] conn(0x55eb4feba9f0 0x55eb502c0a40 crc :-1 s=READY pgs=125 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3d8e5a17a3411561 server_cookie=58a6705dcb9df6b4 in_seq=0 out_seq=0 2024-09-21T03:53:20.142 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:20.132+0000 7f8a72ffd640 1 -- 172.21.15.39:0/1581962678 <== mds.0 v2:172.21.15.150:6838/2949386907 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55eb5021bda0 con 0x55eb4feba9f0 2024-09-21T03:53:20.142 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:20.132+0000 7f8a72ffd640 10 client.17421 handle_command_reply: tid=0 2024-09-21T03:53:20.142 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:20.132+0000 7f8aa06b3740 1 client.17421 shutdown 2024-09-21T03:53:20.143 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:20.132+0000 7f8aa06b3740 1 -- 172.21.15.39:0/1581962678 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55eb4fed31b0 msgr2=0x55eb502ae880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:53:20.143 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:20.132+0000 7f8aa06b3740 1 --2- 172.21.15.39:0/1581962678 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55eb4fed31b0 0x55eb502ae880 secure :-1 s=READY pgs=1046 cs=0 l=1 rev1=1 crypto rx=0x7f8a8800c430 tx=0x7f8a8800cf80 comp rx=0 tx=0).stop 2024-09-21T03:53:20.143 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:20.132+0000 7f8aa06b3740 1 -- 172.21.15.39:0/1581962678 shutdown_connections 2024-09-21T03:53:20.143 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:20.132+0000 7f8aa06b3740 1 --2- 172.21.15.39:0/1581962678 >> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] conn(0x55eb4feba9f0 0x55eb502c0a40 crc :-1 s=READY pgs=125 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:20.143 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:20.132+0000 7f8aa06b3740 1 --2- 172.21.15.39:0/1581962678 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55eb4fefc070 0x55eb502af300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:20.143 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:20.132+0000 7f8aa06b3740 1 --2- 172.21.15.39:0/1581962678 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55eb4fedb390 0x55eb502aedc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:20.143 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:20.132+0000 7f8aa06b3740 1 --2- 172.21.15.39:0/1581962678 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55eb4fed31b0 0x55eb502ae880 unknown :-1 s=CLOSED pgs=1046 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:20.143 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:20.132+0000 7f8aa06b3740 1 -- 172.21.15.39:0/1581962678 >> 172.21.15.39:0/1581962678 conn(0x55eb4fef0460 msgr2=0x55eb502a6400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:53:20.143 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:20.132+0000 7f8a72ffd640 0 client.17421 ms_handle_reset on v2:172.21.15.150:6838/2949386907 2024-09-21T03:53:20.144 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:20.132+0000 7f8aa06b3740 1 -- 172.21.15.39:0/1581962678 shutdown_connections 2024-09-21T03:53:20.144 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:20.132+0000 7f8aa06b3740 1 -- 172.21.15.39:0/1581962678 wait complete. 2024-09-21T03:53:20.144 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:20.133+0000 7f8aa06b3740 20 client.17421 trim_cache size 0 max 16384 2024-09-21T03:53:20.153 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:20.150+0000 7f8aa06b3740 1 Processor -- start 2024-09-21T03:53:20.153 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:20.150+0000 7f8aa06b3740 1 -- start start 2024-09-21T03:53:20.153 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:20.150+0000 7f8aa06b3740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55eb4fed31b0 0x55eb502a4c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:53:20.153 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:20.150+0000 7f8aa06b3740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55eb4fedb390 0x55eb502afda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:53:20.153 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:20.150+0000 7f8aa06b3740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55eb4fefc070 0x55eb502ba3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:53:20.153 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:20.150+0000 7f8aa06b3740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f8a941b1900 con 0x55eb4fed31b0 2024-09-21T03:53:20.153 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:20.150+0000 7f8aa06b3740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f8a941b3440 con 0x55eb4fefc070 2024-09-21T03:53:20.153 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:20.150+0000 7f8aa06b3740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f8a941b1fd0 con 0x55eb4fedb390 2024-09-21T03:53:20.154 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:20.150+0000 7f8a92ffd640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55eb4fed31b0 0x55eb502a4c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:53:20.154 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:20.150+0000 7f8a92ffd640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55eb4fed31b0 0x55eb502a4c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:57260/0 (socket says 172.21.15.39:57260) 2024-09-21T03:53:20.154 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:20.150+0000 7f8a92ffd640 1 -- 172.21.15.39:0/499678633 learned_addr learned my addr 172.21.15.39:0/499678633 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:53:20.154 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:20.151+0000 7f8a937fe640 1 --2- 172.21.15.39:0/499678633 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55eb4fedb390 0x55eb502afda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:53:20.154 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:20.151+0000 7f8a92ffd640 1 --2- 172.21.15.39:0/499678633 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55eb4fefc070 0x55eb502ba3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:53:20.154 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:20.151+0000 7f8a92ffd640 1 -- 172.21.15.39:0/499678633 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55eb4fedb390 msgr2=0x55eb502afda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:53:20.154 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:20.151+0000 7f8a92ffd640 1 --2- 172.21.15.39:0/499678633 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55eb4fedb390 0x55eb502afda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:20.154 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:20.151+0000 7f8a92ffd640 1 -- 172.21.15.39:0/499678633 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55eb4fefc070 msgr2=0x55eb502ba3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:53:20.154 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:20.151+0000 7f8a92ffd640 1 --2- 172.21.15.39:0/499678633 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55eb4fefc070 0x55eb502ba3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:20.154 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:20.151+0000 7f8a92ffd640 1 -- 172.21.15.39:0/499678633 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55eb502b0c50 con 0x55eb4fed31b0 2024-09-21T03:53:20.154 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:20.151+0000 7f8a92ffd640 1 --2- 172.21.15.39:0/499678633 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55eb4fed31b0 0x55eb502a4c10 secure :-1 s=READY pgs=1047 cs=0 l=1 rev1=1 crypto rx=0x7f8a8800c7e0 tx=0x7f8a88005970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ada5507849f9aba4 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:53:20.154 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:20.151+0000 7f8a797fa640 1 -- 172.21.15.39:0/499678633 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a88004680 con 0x55eb4fed31b0 2024-09-21T03:53:20.154 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:20.151+0000 7f8a797fa640 1 -- 172.21.15.39:0/499678633 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8a8800fbe0 con 0x55eb4fed31b0 2024-09-21T03:53:20.155 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:20.151+0000 7f8a797fa640 1 -- 172.21.15.39:0/499678633 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a88012870 con 0x55eb4fed31b0 2024-09-21T03:53:20.158 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:20.152+0000 7f8aa06b3740 1 -- 172.21.15.39:0/499678633 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55eb4fed31b0 msgr2=0x55eb502a4c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:53:20.158 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:20.152+0000 7f8aa06b3740 1 --2- 172.21.15.39:0/499678633 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55eb4fed31b0 0x55eb502a4c10 secure :-1 s=READY pgs=1047 cs=0 l=1 rev1=1 crypto rx=0x7f8a8800c7e0 tx=0x7f8a88005970 comp rx=0 tx=0).stop 2024-09-21T03:53:20.158 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:20.152+0000 7f8aa06b3740 1 -- 172.21.15.39:0/499678633 shutdown_connections 2024-09-21T03:53:20.159 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:20.152+0000 7f8aa06b3740 1 --2- 172.21.15.39:0/499678633 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55eb4fefc070 0x55eb502ba3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:20.159 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:20.152+0000 7f8aa06b3740 1 --2- 172.21.15.39:0/499678633 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55eb4fedb390 0x55eb502afda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:20.159 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:20.152+0000 7f8aa06b3740 1 --2- 172.21.15.39:0/499678633 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55eb4fed31b0 0x55eb502a4c10 unknown :-1 s=CLOSED pgs=1047 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:20.159 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:20.152+0000 7f8aa06b3740 1 -- 172.21.15.39:0/499678633 >> 172.21.15.39:0/499678633 conn(0x55eb4fef0460 msgr2=0x55eb502b9cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:53:20.159 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:20.152+0000 7f8aa06b3740 1 -- 172.21.15.39:0/499678633 shutdown_connections 2024-09-21T03:53:20.159 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:20.152+0000 7f8aa06b3740 1 -- 172.21.15.39:0/499678633 wait complete. 2024-09-21T03:53:20.159 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:20.153+0000 7f8aa06b3740 1 -- start start 2024-09-21T03:53:20.159 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:20.153+0000 7f8aa06b3740 1 Processor -- start 2024-09-21T03:53:20.159 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:20.153+0000 7f8aa06b3740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55eb4fed31b0 0x55eb502b2500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:53:20.159 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:20.153+0000 7f8aa06b3740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55eb4fedb390 0x55eb502b2a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:53:20.159 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:20.153+0000 7f8aa06b3740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55eb4fefc070 0x55eb50329200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:53:20.159 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:20.153+0000 7f8aa06b3740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f8a941b2d70 con 0x55eb4fefc070 2024-09-21T03:53:20.159 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:20.153+0000 7f8aa06b3740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f8a941b26a0 con 0x55eb4fed31b0 2024-09-21T03:53:20.159 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:20.153+0000 7f8aa06b3740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f8a941af8e0 con 0x55eb4fedb390 2024-09-21T03:53:20.160 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:20.153+0000 7f8aa06b3740 10 client.0 _ll_register_callbacks cb 0x7f8a84009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:53:20.160 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:20.153+0000 7f8a92ffd640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55eb4fefc070 0x55eb50329200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:53:20.160 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:20.153+0000 7f8a92ffd640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55eb4fefc070 0x55eb50329200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:57268/0 (socket says 172.21.15.39:57268) 2024-09-21T03:53:20.160 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:20.153+0000 7f8a92ffd640 1 -- 172.21.15.39:0/3610974137 learned_addr learned my addr 172.21.15.39:0/3610974137 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:53:20.160 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:20.153+0000 7f8a92ffd640 1 --2- 172.21.15.39:0/3610974137 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55eb4fed31b0 0x55eb502b2500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:53:20.160 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:20.153+0000 7f8a937fe640 1 --2- 172.21.15.39:0/3610974137 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55eb4fedb390 0x55eb502b2a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:53:20.160 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:20.154+0000 7f8a92ffd640 1 -- 172.21.15.39:0/3610974137 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55eb4fedb390 msgr2=0x55eb502b2a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:53:20.160 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:20.154+0000 7f8a92ffd640 1 --2- 172.21.15.39:0/3610974137 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55eb4fedb390 0x55eb502b2a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:20.160 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:20.154+0000 7f8a92ffd640 1 -- 172.21.15.39:0/3610974137 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55eb4fed31b0 msgr2=0x55eb502b2500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:53:20.160 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:20.154+0000 7f8a92ffd640 1 --2- 172.21.15.39:0/3610974137 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55eb4fed31b0 0x55eb502b2500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:20.160 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:20.154+0000 7f8a92ffd640 1 -- 172.21.15.39:0/3610974137 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55eb50329810 con 0x55eb4fefc070 2024-09-21T03:53:20.160 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:20.154+0000 7f8a92ffd640 1 --2- 172.21.15.39:0/3610974137 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55eb4fefc070 0x55eb50329200 secure :-1 s=READY pgs=1048 cs=0 l=1 rev1=1 crypto rx=0x7f8a88014920 tx=0x7f8a88039040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d6a15c659cf0127c server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:53:20.160 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:20.154+0000 7f8a72ffd640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-09-21T03:53:20.160 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:20.154+0000 7f8a72ffd640 1 -- 172.21.15.39:0/3610974137 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a8804c060 con 0x55eb4fefc070 2024-09-21T03:53:20.161 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:20.154+0000 7f8a72ffd640 1 -- 172.21.15.39:0/3610974137 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8528+0+0 (secure 0 0 0) 0x7f8a8800f940 con 0x55eb4fefc070 2024-09-21T03:53:20.161 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:20.154+0000 7f8aa06b3740 1 -- 172.21.15.39:0/3610974137 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8a8800fbe0 con 0x55eb4fefc070 2024-09-21T03:53:20.161 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:20.154+0000 7f8a72ffd640 1 -- 172.21.15.39:0/3610974137 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f8a88005b90 con 0x55eb4fefc070 2024-09-21T03:53:20.161 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:20.154+0000 7f8aa06b3740 10 client.17427 fetch_fsmap learned FSMap version 29 2024-09-21T03:53:20.161 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:20.154+0000 7f8aa06b3740 1 -- 172.21.15.39:0/3610974137 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55eb5032aaa0 con 0x55eb4fefc070 2024-09-21T03:53:20.161 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:20.155+0000 7f8a72ffd640 1 -- 172.21.15.39:0/3610974137 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8a8800fbe0 con 0x55eb4fefc070 2024-09-21T03:53:20.161 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:20.155+0000 7f8a72ffd640 1 -- 172.21.15.39:0/3610974137 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f8a88045470 con 0x55eb4fefc070 2024-09-21T03:53:20.161 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:20.155+0000 7f8aa06b3740 10 client.17427 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T03:53:20.161 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:20.155+0000 7f8aa06b3740 10 client.17427 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T03:53:20.161 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:20.155+0000 7f8aa06b3740 20 client.17427 populate_metadata read hostname 'smithi039' 2024-09-21T03:53:20.161 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:20.155+0000 7f8aa06b3740 1 --2- 172.21.15.39:0/3610974137 >> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] conn(0x55eb4fe8fda0 0x55eb5032d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:53:20.161 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:20.155+0000 7f8aa06b3740 4 client.17427 mds_command: new command op to 24695 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-21T03:53:20.161 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:20.155+0000 7f8aa06b3740 1 -- 172.21.15.39:0/3610974137 --> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55eb50217850 con 0x55eb4fe8fda0 2024-09-21T03:53:20.162 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:20.155+0000 7f8a937fe640 1 --2- 172.21.15.39:0/3610974137 >> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] conn(0x55eb4fe8fda0 0x55eb5032d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:53:20.162 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:20.155+0000 7f8a72ffd640 10 client.17427 ms_handle_connect on v2:172.21.15.150:6838/2949386907 2024-09-21T03:53:20.162 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:20.155+0000 7f8a937fe640 1 --2- 172.21.15.39:0/3610974137 >> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] conn(0x55eb4fe8fda0 0x55eb5032d0c0 crc :-1 s=READY pgs=126 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=36001d1eaaf002a7 server_cookie=9031cb4ea9704fa4 in_seq=0 out_seq=0 2024-09-21T03:53:20.162 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:20.156+0000 7f8a72ffd640 1 -- 172.21.15.39:0/3610974137 <== mds.0 v2:172.21.15.150:6838/2949386907 1 ==== command_reply(tid 0: 0 ) ==== 8+0+4066 (crc 0 0 0) 0x55eb50217850 con 0x55eb4fe8fda0 2024-09-21T03:53:20.162 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:20.156+0000 7f8a72ffd640 10 client.17427 handle_command_reply: tid=0 2024-09-21T03:53:20.162 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:20.156+0000 7f8aa06b3740 1 client.17427 shutdown 2024-09-21T03:53:20.162 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:20.156+0000 7f8aa06b3740 1 -- 172.21.15.39:0/3610974137 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55eb4fefc070 msgr2=0x55eb50329200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:53:20.162 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:20.156+0000 7f8aa06b3740 1 --2- 172.21.15.39:0/3610974137 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55eb4fefc070 0x55eb50329200 secure :-1 s=READY pgs=1048 cs=0 l=1 rev1=1 crypto rx=0x7f8a88014920 tx=0x7f8a88039040 comp rx=0 tx=0).stop 2024-09-21T03:53:20.162 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:20.156+0000 7f8aa06b3740 1 -- 172.21.15.39:0/3610974137 shutdown_connections 2024-09-21T03:53:20.162 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:20.156+0000 7f8aa06b3740 1 --2- 172.21.15.39:0/3610974137 >> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] conn(0x55eb4fe8fda0 0x55eb5032d0c0 crc :-1 s=READY pgs=126 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:20.162 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:20.156+0000 7f8aa06b3740 1 --2- 172.21.15.39:0/3610974137 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55eb4fefc070 0x55eb50329200 unknown :-1 s=CLOSED pgs=1048 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:20.162 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:20.156+0000 7f8aa06b3740 1 --2- 172.21.15.39:0/3610974137 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55eb4fedb390 0x55eb502b2a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:20.162 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:20.156+0000 7f8aa06b3740 1 --2- 172.21.15.39:0/3610974137 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55eb4fed31b0 0x55eb502b2500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:20.162 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:20.156+0000 7f8aa06b3740 1 -- 172.21.15.39:0/3610974137 >> 172.21.15.39:0/3610974137 conn(0x55eb4fef0460 msgr2=0x55eb502a4b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:53:20.163 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:20.156+0000 7f8a72ffd640 0 client.17427 ms_handle_reset on v2:172.21.15.150:6838/2949386907 2024-09-21T03:53:20.163 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:20.156+0000 7f8aa06b3740 1 -- 172.21.15.39:0/3610974137 shutdown_connections 2024-09-21T03:53:20.163 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:20.156+0000 7f8aa06b3740 1 -- 172.21.15.39:0/3610974137 wait complete. 2024-09-21T03:53:20.163 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:20.156+0000 7f8aa06b3740 20 client.17427 trim_cache size 0 max 16384 2024-09-21T03:53:20.163 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-09-21T03:53:20.163 INFO:teuthology.orchestra.run.smithi039.stdout: "status": "no active scrubs running", 2024-09-21T03:53:20.163 INFO:teuthology.orchestra.run.smithi039.stdout: "scrubs": {}, 2024-09-21T03:53:20.163 INFO:teuthology.orchestra.run.smithi039.stdout: "scrub_stats": { 2024-09-21T03:53:20.163 INFO:teuthology.orchestra.run.smithi039.stdout: "ad87da79-0248-4e18-9694-8228cb95422f": { 2024-09-21T03:53:20.163 INFO:teuthology.orchestra.run.smithi039.stdout: "start_time": "2024-09-21T03:53:18.441907+0000", 2024-09-21T03:53:20.163 INFO:teuthology.orchestra.run.smithi039.stdout: "path": "/", 2024-09-21T03:53:20.164 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_started": 0, 2024-09-21T03:53:20.164 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_passed": 0, 2024-09-21T03:53:20.164 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_failed": 0, 2024-09-21T03:53:20.164 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_skipped": 0 2024-09-21T03:53:20.164 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:53:20.164 INFO:teuthology.orchestra.run.smithi039.stdout: "02509e4f-3911-4983-b16f-df8ded0b9998": { 2024-09-21T03:53:20.164 INFO:teuthology.orchestra.run.smithi039.stdout: "start_time": "2024-09-21T03:46:17.889576+0000", 2024-09-21T03:53:20.164 INFO:teuthology.orchestra.run.smithi039.stdout: "path": "/", 2024-09-21T03:53:20.164 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_started": 0, 2024-09-21T03:53:20.164 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_passed": 0, 2024-09-21T03:53:20.164 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_failed": 0, 2024-09-21T03:53:20.164 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_skipped": 0 2024-09-21T03:53:20.164 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:53:20.164 INFO:teuthology.orchestra.run.smithi039.stdout: "10728de8-c49d-4edc-be2b-639249255e94": { 2024-09-21T03:53:20.165 INFO:teuthology.orchestra.run.smithi039.stdout: "start_time": "2024-09-21T03:47:03.965623+0000", 2024-09-21T03:53:20.165 INFO:teuthology.orchestra.run.smithi039.stdout: "path": "/", 2024-09-21T03:53:20.165 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_started": 0, 2024-09-21T03:53:20.165 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_passed": 0, 2024-09-21T03:53:20.165 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_failed": 0, 2024-09-21T03:53:20.165 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_skipped": 0 2024-09-21T03:53:20.165 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:53:20.165 INFO:teuthology.orchestra.run.smithi039.stdout: "9860c2f4-b7b1-47aa-865a-9a1c4434a5f6": { 2024-09-21T03:53:20.165 INFO:teuthology.orchestra.run.smithi039.stdout: "start_time": "2024-09-21T03:49:17.718867+0000", 2024-09-21T03:53:20.165 INFO:teuthology.orchestra.run.smithi039.stdout: "path": "/", 2024-09-21T03:53:20.165 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_started": 0, 2024-09-21T03:53:20.165 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_passed": 0, 2024-09-21T03:53:20.165 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_failed": 0, 2024-09-21T03:53:20.165 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_skipped": 0 2024-09-21T03:53:20.166 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:53:20.166 INFO:teuthology.orchestra.run.smithi039.stdout: "c7c700f3-2cd8-41ac-8381-b57dd1b37b02": { 2024-09-21T03:53:20.166 INFO:teuthology.orchestra.run.smithi039.stdout: "start_time": "2024-09-21T03:52:32.368858+0000", 2024-09-21T03:53:20.166 INFO:teuthology.orchestra.run.smithi039.stdout: "path": "/", 2024-09-21T03:53:20.166 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_started": 0, 2024-09-21T03:53:20.166 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_passed": 0, 2024-09-21T03:53:20.166 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_failed": 0, 2024-09-21T03:53:20.166 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_skipped": 0 2024-09-21T03:53:20.166 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:53:20.166 INFO:teuthology.orchestra.run.smithi039.stdout: "29411307-eb8d-4339-8d2b-47ebd32c0080": { 2024-09-21T03:53:20.166 INFO:teuthology.orchestra.run.smithi039.stdout: "start_time": "2024-09-21T03:47:38.092920+0000", 2024-09-21T03:53:20.166 INFO:teuthology.orchestra.run.smithi039.stdout: "path": "/", 2024-09-21T03:53:20.166 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_started": 0, 2024-09-21T03:53:20.166 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_passed": 0, 2024-09-21T03:53:20.167 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_failed": 0, 2024-09-21T03:53:20.167 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_skipped": 0 2024-09-21T03:53:20.167 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:53:20.167 INFO:teuthology.orchestra.run.smithi039.stdout: "ede73e6e-3fd0-4a7f-b5bb-f68946f57eea": { 2024-09-21T03:53:20.167 INFO:teuthology.orchestra.run.smithi039.stdout: "start_time": "2024-09-21T03:51:32.235096+0000", 2024-09-21T03:53:20.167 INFO:teuthology.orchestra.run.smithi039.stdout: "path": "/", 2024-09-21T03:53:20.167 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_started": 0, 2024-09-21T03:53:20.167 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_passed": 0, 2024-09-21T03:53:20.167 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_failed": 0, 2024-09-21T03:53:20.167 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_skipped": 0 2024-09-21T03:53:20.167 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:53:20.168 INFO:teuthology.orchestra.run.smithi039.stdout: "e28ad743-2b22-45b3-8799-a638c2883623": { 2024-09-21T03:53:20.168 INFO:teuthology.orchestra.run.smithi039.stdout: "start_time": "2024-09-21T03:48:31.448821+0000", 2024-09-21T03:53:20.168 INFO:teuthology.orchestra.run.smithi039.stdout: "path": "/", 2024-09-21T03:53:20.168 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_started": 0, 2024-09-21T03:53:20.168 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_passed": 0, 2024-09-21T03:53:20.168 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_failed": 0, 2024-09-21T03:53:20.168 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_skipped": 0 2024-09-21T03:53:20.168 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:53:20.168 INFO:teuthology.orchestra.run.smithi039.stdout: "6e760f1f-9048-4f92-bd66-d708fbef75b3": { 2024-09-21T03:53:20.168 INFO:teuthology.orchestra.run.smithi039.stdout: "start_time": "2024-09-21T03:50:03.711918+0000", 2024-09-21T03:53:20.168 INFO:teuthology.orchestra.run.smithi039.stdout: "path": "/", 2024-09-21T03:53:20.168 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_started": 0, 2024-09-21T03:53:20.168 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_passed": 0, 2024-09-21T03:53:20.169 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_failed": 0, 2024-09-21T03:53:20.169 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_skipped": 0 2024-09-21T03:53:20.169 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:53:20.169 INFO:teuthology.orchestra.run.smithi039.stdout: "c7e2b8e4-057d-435f-b2bd-b633de98035f": { 2024-09-21T03:53:20.169 INFO:teuthology.orchestra.run.smithi039.stdout: "start_time": "2024-09-21T03:47:19.400305+0000", 2024-09-21T03:53:20.169 INFO:teuthology.orchestra.run.smithi039.stdout: "path": "/", 2024-09-21T03:53:20.169 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_started": 0, 2024-09-21T03:53:20.169 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_passed": 0, 2024-09-21T03:53:20.169 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_failed": 0, 2024-09-21T03:53:20.169 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_skipped": 0 2024-09-21T03:53:20.169 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:53:20.169 INFO:teuthology.orchestra.run.smithi039.stdout: "74ab917e-ecea-4778-a1c3-c184214fb24c": { 2024-09-21T03:53:20.169 INFO:teuthology.orchestra.run.smithi039.stdout: "start_time": "2024-09-21T03:50:17.726630+0000", 2024-09-21T03:53:20.169 INFO:teuthology.orchestra.run.smithi039.stdout: "path": "/", 2024-09-21T03:53:20.169 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_started": 0, 2024-09-21T03:53:20.170 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_passed": 0, 2024-09-21T03:53:20.170 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_failed": 0, 2024-09-21T03:53:20.170 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_skipped": 0 2024-09-21T03:53:20.170 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:53:20.170 INFO:teuthology.orchestra.run.smithi039.stdout: "160750d8-401f-4810-866d-e63e7e4bc3e0": { 2024-09-21T03:53:20.170 INFO:teuthology.orchestra.run.smithi039.stdout: "start_time": "2024-09-21T03:51:04.208673+0000", 2024-09-21T03:53:20.170 INFO:teuthology.orchestra.run.smithi039.stdout: "path": "/", 2024-09-21T03:53:20.170 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_started": 0, 2024-09-21T03:53:20.170 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_passed": 0, 2024-09-21T03:53:20.170 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_failed": 0, 2024-09-21T03:53:20.170 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_skipped": 0 2024-09-21T03:53:20.170 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:53:20.170 INFO:teuthology.orchestra.run.smithi039.stdout: "ae1fc4cf-5458-47d0-afcb-282b136dba15": { 2024-09-21T03:53:20.171 INFO:teuthology.orchestra.run.smithi039.stdout: "start_time": "2024-09-21T03:51:18.560378+0000", 2024-09-21T03:53:20.171 INFO:teuthology.orchestra.run.smithi039.stdout: "path": "/", 2024-09-21T03:53:20.171 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_started": 0, 2024-09-21T03:53:20.171 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_passed": 0, 2024-09-21T03:53:20.171 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_failed": 0, 2024-09-21T03:53:20.171 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_skipped": 0 2024-09-21T03:53:20.171 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:53:20.171 INFO:teuthology.orchestra.run.smithi039.stdout: "4bda6ee3-7237-4ee4-96ce-150ad50f26dc": { 2024-09-21T03:53:20.171 INFO:teuthology.orchestra.run.smithi039.stdout: "start_time": "2024-09-21T03:52:18.622141+0000", 2024-09-21T03:53:20.171 INFO:teuthology.orchestra.run.smithi039.stdout: "path": "/", 2024-09-21T03:53:20.171 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_started": 0, 2024-09-21T03:53:20.171 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_passed": 0, 2024-09-21T03:53:20.171 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_failed": 0, 2024-09-21T03:53:20.171 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_skipped": 0 2024-09-21T03:53:20.171 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-21T03:53:20.172 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-21T03:53:20.172 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-09-21T03:53:20.172 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:20.157+0000 7f8a9a13f640 1 -- 172.21.15.39:0/3268722324 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f8a7c0b6680 msgr2=0x7f8a7c0b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:53:20.172 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:20.157+0000 7f8a9a13f640 1 --2- 172.21.15.39:0/3268722324 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f8a7c0b6680 0x7f8a7c0b8b90 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f8a8c0cbae0 tx=0x7f8a8c0f2040 comp rx=0 tx=0).stop 2024-09-21T03:53:20.172 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:20.157+0000 7f8a9a13f640 1 -- 172.21.15.39:0/3268722324 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8a94106c40 msgr2=0x7f8a94103ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:53:20.173 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:20.157+0000 7f8a9a13f640 1 --2- 172.21.15.39:0/3268722324 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8a94106c40 0x7f8a94103ac0 secure :-1 s=READY pgs=1044 cs=0 l=1 rev1=1 crypto rx=0x7f8a8c0098d0 tx=0x7f8a8c031770 comp rx=0 tx=0).stop 2024-09-21T03:53:20.173 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:20.157+0000 7f8a9a13f640 1 -- 172.21.15.39:0/3268722324 shutdown_connections 2024-09-21T03:53:20.173 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:20.157+0000 7f8a9a13f640 1 --2- 172.21.15.39:0/3268722324 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f8a7c0b6680 0x7f8a7c0b8b90 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:20.173 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:20.157+0000 7f8a9a13f640 1 --2- 172.21.15.39:0/3268722324 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8a94106c40 0x7f8a94103ac0 unknown :-1 s=CLOSED pgs=1044 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:20.173 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:20.157+0000 7f8a9a13f640 1 --2- 172.21.15.39:0/3268722324 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8a941062f0 0x7f8a94103580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:20.173 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:20.157+0000 7f8a9a13f640 1 --2- 172.21.15.39:0/3268722324 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8a941020a0 0x7f8a94103040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:20.173 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:20.157+0000 7f8a9a13f640 1 -- 172.21.15.39:0/3268722324 >> 172.21.15.39:0/3268722324 conn(0x7f8a94079f90 msgr2=0x7f8a94105440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:53:20.173 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:20.157+0000 7f8a9a13f640 1 -- 172.21.15.39:0/3268722324 shutdown_connections 2024-09-21T03:53:20.173 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:20.157+0000 7f8a9a13f640 1 -- 172.21.15.39:0/3268722324 wait complete. 2024-09-21T03:53:20.938 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:53:20 smithi039 ceph-mon[31211]: pgmap v545: 97 pgs: 97 active+clean; 10 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 204 B/s wr, 10 op/s 2024-09-21T03:53:20.938 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:53:20 smithi039 ceph-mon[31211]: osdmap e81: 12 total, 12 up, 12 in 2024-09-21T03:53:20.938 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:53:20 smithi039 ceph-mon[31211]: scrub summary: idle 2024-09-21T03:53:20.972 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-21T03:53:20.972 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 51116d5c-77ca-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T03:53:21.218 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:53:20 smithi161 ceph-mon[34533]: pgmap v545: 97 pgs: 97 active+clean; 10 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 204 B/s wr, 10 op/s 2024-09-21T03:53:21.218 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:53:20 smithi161 ceph-mon[34533]: osdmap e81: 12 total, 12 up, 12 in 2024-09-21T03:53:21.218 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:53:20 smithi161 ceph-mon[34533]: scrub summary: idle 2024-09-21T03:53:21.225 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:53:20 smithi150 ceph-mon[34289]: pgmap v545: 97 pgs: 97 active+clean; 10 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 204 B/s wr, 10 op/s 2024-09-21T03:53:21.225 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:53:20 smithi150 ceph-mon[34289]: osdmap e81: 12 total, 12 up, 12 in 2024-09-21T03:53:21.225 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:53:20 smithi150 ceph-mon[34289]: scrub summary: idle 2024-09-21T03:53:21.235 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/51116d5c-77ca-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:53:21.726 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:21.724+0000 7f3053e4d640 1 -- 172.21.15.39:0/1455473730 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f304c109df0 msgr2=0x7f304c110620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:53:21.726 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:21.724+0000 7f3053e4d640 1 --2- 172.21.15.39:0/1455473730 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f304c109df0 0x7f304c110620 secure :-1 s=READY pgs=1049 cs=0 l=1 rev1=1 crypto rx=0x7f3044009ca0 tx=0x7f304402a8c0 comp rx=0 tx=0).stop 2024-09-21T03:53:21.726 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:21.724+0000 7f3053e4d640 1 -- 172.21.15.39:0/1455473730 shutdown_connections 2024-09-21T03:53:21.726 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:21.724+0000 7f3053e4d640 1 --2- 172.21.15.39:0/1455473730 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f304c109df0 0x7f304c110620 unknown :-1 s=CLOSED pgs=1049 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:21.726 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:21.724+0000 7f3053e4d640 1 --2- 172.21.15.39:0/1455473730 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f304c1094a0 0x7f304c1098b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:21.726 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:21.724+0000 7f3053e4d640 1 --2- 172.21.15.39:0/1455473730 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f304c103b60 0x7f304c103f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:21.726 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:21.724+0000 7f3053e4d640 1 -- 172.21.15.39:0/1455473730 >> 172.21.15.39:0/1455473730 conn(0x7f304c07b850 msgr2=0x7f304c07bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:53:21.726 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:21.724+0000 7f3053e4d640 1 -- 172.21.15.39:0/1455473730 shutdown_connections 2024-09-21T03:53:21.726 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:21.724+0000 7f3053e4d640 1 -- 172.21.15.39:0/1455473730 wait complete. 2024-09-21T03:53:21.728 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:21.725+0000 7f3053e4d640 1 Processor -- start 2024-09-21T03:53:21.728 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:21.725+0000 7f3053e4d640 1 -- start start 2024-09-21T03:53:21.728 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:21.725+0000 7f3053e4d640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f304c103b60 0x7f304c19a460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:53:21.728 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:21.725+0000 7f3053e4d640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f304c1094a0 0x7f304c19aa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:53:21.728 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:21.725+0000 7f3053e4d640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f304c109df0 0x7f304c19fca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:53:21.728 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:21.725+0000 7f3053e4d640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f304c113ab0 con 0x7f304c103b60 2024-09-21T03:53:21.729 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:21.725+0000 7f3053e4d640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f304c113930 con 0x7f304c109df0 2024-09-21T03:53:21.729 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:21.725+0000 7f3053e4d640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f304c113c30 con 0x7f304c1094a0 2024-09-21T03:53:21.729 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:21.725+0000 7f3051be9640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f304c103b60 0x7f304c19a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:53:21.729 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:21.725+0000 7f3051be9640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f304c103b60 0x7f304c19a460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:57276/0 (socket says 172.21.15.39:57276) 2024-09-21T03:53:21.729 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:21.725+0000 7f3051be9640 1 -- 172.21.15.39:0/413500155 learned_addr learned my addr 172.21.15.39:0/413500155 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:53:21.729 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:21.725+0000 7f30523ea640 1 --2- 172.21.15.39:0/413500155 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f304c109df0 0x7f304c19fca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:53:21.729 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:21.726+0000 7f30513e8640 1 --2- 172.21.15.39:0/413500155 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f304c1094a0 0x7f304c19aa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:53:21.729 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:21.726+0000 7f3051be9640 1 -- 172.21.15.39:0/413500155 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f304c1094a0 msgr2=0x7f304c19aa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:53:21.729 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:21.726+0000 7f3051be9640 1 --2- 172.21.15.39:0/413500155 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f304c1094a0 0x7f304c19aa00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:21.729 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:21.726+0000 7f3051be9640 1 -- 172.21.15.39:0/413500155 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f304c109df0 msgr2=0x7f304c19fca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:53:21.729 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:21.726+0000 7f3051be9640 1 --2- 172.21.15.39:0/413500155 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f304c109df0 0x7f304c19fca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:21.729 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:21.726+0000 7f3051be9640 1 -- 172.21.15.39:0/413500155 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f304c1a03d0 con 0x7f304c103b60 2024-09-21T03:53:21.729 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:21.726+0000 7f3051be9640 1 --2- 172.21.15.39:0/413500155 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f304c103b60 0x7f304c19a460 secure :-1 s=READY pgs=1050 cs=0 l=1 rev1=1 crypto rx=0x7f304800d950 tx=0x7f304800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:53:21.729 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:21.726+0000 7f3042ffd640 1 -- 172.21.15.39:0/413500155 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3048017070 con 0x7f304c103b60 2024-09-21T03:53:21.729 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:21.726+0000 7f3053e4d640 1 -- 172.21.15.39:0/413500155 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f304c1a06c0 con 0x7f304c103b60 2024-09-21T03:53:21.730 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:21.726+0000 7f3042ffd640 1 -- 172.21.15.39:0/413500155 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3048009c30 con 0x7f304c103b60 2024-09-21T03:53:21.730 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:21.726+0000 7f3042ffd640 1 -- 172.21.15.39:0/413500155 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f304800f620 con 0x7f304c103b60 2024-09-21T03:53:21.730 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:21.726+0000 7f3053e4d640 1 -- 172.21.15.39:0/413500155 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f304c1a6e20 con 0x7f304c103b60 2024-09-21T03:53:21.730 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:21.727+0000 7f3053e4d640 1 -- 172.21.15.39:0/413500155 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f304c103f70 con 0x7f304c103b60 2024-09-21T03:53:21.730 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:21.729+0000 7f3042ffd640 1 -- 172.21.15.39:0/413500155 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 27) ==== 154643+0+0 (secure 0 0 0) 0x7f3048005030 con 0x7f304c103b60 2024-09-21T03:53:21.731 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:21.729+0000 7f3042ffd640 1 --2- 172.21.15.39:0/413500155 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f30380b6680 0x7f30380b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:53:21.731 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:21.730+0000 7f30513e8640 1 --2- 172.21.15.39:0/413500155 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f30380b6680 0x7f30380b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:53:21.731 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:21.730+0000 7f3042ffd640 1 -- 172.21.15.39:0/413500155 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8528+0+0 (secure 0 0 0) 0x7f30480e7320 con 0x7f304c103b60 2024-09-21T03:53:21.732 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:21.730+0000 7f30513e8640 1 --2- 172.21.15.39:0/413500155 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f30380b6680 0x7f30380b8b90 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f303c005fd0 tx=0x7f303c005e20 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:53:21.734 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:21.733+0000 7f3042ffd640 1 -- 172.21.15.39:0/413500155 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f30480ad870 con 0x7f304c103b60 2024-09-21T03:53:21.963 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:21.961+0000 7f3053e4d640 1 -- 172.21.15.39:0/413500155 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f304c19b590 con 0x7f304c103b60 2024-09-21T03:53:21.964 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:21.962+0000 7f3042ffd640 1 -- 172.21.15.39:0/413500155 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11978 (secure 0 0 0) 0x7f3048009de0 con 0x7f304c103b60 2024-09-21T03:53:21.964 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T03:53:21.964 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":29,"btime":"2024-09-21T03:46:08:982881+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14790,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6835/508995295","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":508995295},{"type":"v1","addr":"172.21.15.39:6835","nonce":508995295}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14802,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6837/3915912607","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":3915912607},{"type":"v1","addr":"172.21.15.39:6837","nonce":3915912607}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":29,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T03:44:15.009872+0000","modified":"2024-09-21T03:46:08.430113+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24695,"mds_1":24667,"mds_2":24659,"mds_3":24646,"mds_4":14868},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14856":{"gid":14856,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.39:6839/185509192","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6838","nonce":185509192},{"type":"v1","addr":"172.21.15.39:6839","nonce":185509192}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14868":{"gid":14868,"name":"a","rank":4,"incarnation":25,"state":"up:active","state_seq":7,"addr":"172.21.15.39:6841/4247190456","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6840","nonce":4247190456},{"type":"v1","addr":"172.21.15.39:6841","nonce":4247190456}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24599":{"gid":24599,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.150:6833/3114643217","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":3114643217},{"type":"v1","addr":"172.21.15.150:6833","nonce":3114643217}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24601":{"gid":24601,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.161:6833/2963990654","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":2963990654},{"type":"v1","addr":"172.21.15.161:6833","nonce":2963990654}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24632":{"gid":24632,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.150:6835/724594008","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6834","nonce":724594008},{"type":"v1","addr":"172.21.15.150:6835","nonce":724594008}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24640":{"gid":24640,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.161:6835/3747882110","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":3747882110},{"type":"v1","addr":"172.21.15.161:6835","nonce":3747882110}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24646":{"gid":24646,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":10,"addr":"172.21.15.161:6837/3451544597","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6836","nonce":3451544597},{"type":"v1","addr":"172.21.15.161:6837","nonce":3451544597}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24659":{"gid":24659,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.150:6837/4017043144","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6836","nonce":4017043144},{"type":"v1","addr":"172.21.15.150:6837","nonce":4017043144}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24667":{"gid":24667,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.161:6839/2190107270","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6838","nonce":2190107270},{"type":"v1","addr":"172.21.15.161:6839","nonce":2190107270}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24695":{"gid":24695,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.150:6839/2949386907","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6838","nonce":2949386907},{"type":"v1","addr":"172.21.15.150:6839","nonce":2949386907}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24695,"qdb_cluster":[24695,24667,24659,24646,14868]},"id":1}]} 2024-09-21T03:53:21.964 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 29 2024-09-21T03:53:21.966 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:21.964+0000 7f3053e4d640 1 -- 172.21.15.39:0/413500155 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f30380b6680 msgr2=0x7f30380b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:53:21.966 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:21.964+0000 7f3053e4d640 1 --2- 172.21.15.39:0/413500155 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f30380b6680 0x7f30380b8b90 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f303c005fd0 tx=0x7f303c005e20 comp rx=0 tx=0).stop 2024-09-21T03:53:21.966 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:21.964+0000 7f3053e4d640 1 -- 172.21.15.39:0/413500155 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f304c103b60 msgr2=0x7f304c19a460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:53:21.966 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:21.964+0000 7f3053e4d640 1 --2- 172.21.15.39:0/413500155 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f304c103b60 0x7f304c19a460 secure :-1 s=READY pgs=1050 cs=0 l=1 rev1=1 crypto rx=0x7f304800d950 tx=0x7f304800de10 comp rx=0 tx=0).stop 2024-09-21T03:53:21.967 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:21.964+0000 7f3053e4d640 1 -- 172.21.15.39:0/413500155 shutdown_connections 2024-09-21T03:53:21.967 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:21.964+0000 7f3053e4d640 1 --2- 172.21.15.39:0/413500155 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f30380b6680 0x7f30380b8b90 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:21.967 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:21.964+0000 7f3053e4d640 1 --2- 172.21.15.39:0/413500155 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f304c109df0 0x7f304c19fca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:21.967 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:21.964+0000 7f3053e4d640 1 --2- 172.21.15.39:0/413500155 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f304c1094a0 0x7f304c19aa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:21.967 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:21.964+0000 7f3053e4d640 1 --2- 172.21.15.39:0/413500155 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f304c103b60 0x7f304c19a460 unknown :-1 s=CLOSED pgs=1050 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:21.967 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:21.964+0000 7f3053e4d640 1 -- 172.21.15.39:0/413500155 >> 172.21.15.39:0/413500155 conn(0x7f304c07b850 msgr2=0x7f304c1074f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:53:21.967 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:21.965+0000 7f3053e4d640 1 -- 172.21.15.39:0/413500155 shutdown_connections 2024-09-21T03:53:21.967 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:21.965+0000 7f3053e4d640 1 -- 172.21.15.39:0/413500155 wait complete. 2024-09-21T03:53:22.709 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 51116d5c-77ca-11ef-baf5-efdc52797490 -- ceph tell mds.1:4 damage ls 2024-09-21T03:53:22.969 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/51116d5c-77ca-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:53:23.084 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:53:22 smithi039 ceph-mon[31211]: pgmap v547: 97 pgs: 97 active+clean; 10 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 204 B/s wr, 10 op/s 2024-09-21T03:53:23.084 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:53:22 smithi039 ceph-mon[31211]: from='client.? 172.21.15.39:0/413500155' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:53:23.218 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:53:22 smithi161 ceph-mon[34533]: pgmap v547: 97 pgs: 97 active+clean; 10 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 204 B/s wr, 10 op/s 2024-09-21T03:53:23.218 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:53:22 smithi161 ceph-mon[34533]: from='client.? 172.21.15.39:0/413500155' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:53:23.225 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:53:22 smithi150 ceph-mon[34289]: pgmap v547: 97 pgs: 97 active+clean; 10 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 204 B/s wr, 10 op/s 2024-09-21T03:53:23.225 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:53:22 smithi150 ceph-mon[34289]: from='client.? 172.21.15.39:0/413500155' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:53:23.490 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:23.475+0000 7fbb28551640 1 -- 172.21.15.39:0/4147140515 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbb201083c0 msgr2=0x7fbb201087d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:53:23.491 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:23.475+0000 7fbb28551640 1 --2- 172.21.15.39:0/4147140515 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbb201083c0 0x7fbb201087d0 secure :-1 s=READY pgs=1051 cs=0 l=1 rev1=1 crypto rx=0x7fbb10009900 tx=0x7fbb1002a3f0 comp rx=0 tx=0).stop 2024-09-21T03:53:23.491 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:23.475+0000 7fbb28551640 1 -- 172.21.15.39:0/4147140515 shutdown_connections 2024-09-21T03:53:23.491 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:23.475+0000 7fbb28551640 1 --2- 172.21.15.39:0/4147140515 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbb20108d10 0x7fbb2010f540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:23.491 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:23.475+0000 7fbb28551640 1 --2- 172.21.15.39:0/4147140515 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbb201083c0 0x7fbb201087d0 unknown :-1 s=CLOSED pgs=1051 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:23.491 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:23.475+0000 7fbb28551640 1 --2- 172.21.15.39:0/4147140515 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbb20102a80 0x7fbb20102e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:23.491 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:23.475+0000 7fbb28551640 1 -- 172.21.15.39:0/4147140515 >> 172.21.15.39:0/4147140515 conn(0x7fbb20079740 msgr2=0x7fbb20079b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:53:23.491 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:23.475+0000 7fbb28551640 1 -- 172.21.15.39:0/4147140515 shutdown_connections 2024-09-21T03:53:23.491 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:23.475+0000 7fbb28551640 1 -- 172.21.15.39:0/4147140515 wait complete. 2024-09-21T03:53:23.491 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:23.476+0000 7fbb28551640 1 Processor -- start 2024-09-21T03:53:23.491 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:23.476+0000 7fbb28551640 1 -- start start 2024-09-21T03:53:23.491 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:23.476+0000 7fbb28551640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbb20102a80 0x7fbb2019a4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:53:23.491 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:23.476+0000 7fbb28551640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbb201083c0 0x7fbb2019aa40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:53:23.491 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:23.476+0000 7fbb28551640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbb20108d10 0x7fbb2019fce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:53:23.492 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:23.476+0000 7fbb28551640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fbb20112980 con 0x7fbb20108d10 2024-09-21T03:53:23.492 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:23.476+0000 7fbb28551640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fbb20112800 con 0x7fbb20102a80 2024-09-21T03:53:23.492 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:23.476+0000 7fbb28551640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fbb20112b00 con 0x7fbb201083c0 2024-09-21T03:53:23.492 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:23.476+0000 7fbb26aee640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbb20108d10 0x7fbb2019fce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:53:23.492 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:23.476+0000 7fbb26aee640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbb20108d10 0x7fbb2019fce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:57286/0 (socket says 172.21.15.39:57286) 2024-09-21T03:53:23.492 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:23.476+0000 7fbb26aee640 1 -- 172.21.15.39:0/2152271990 learned_addr learned my addr 172.21.15.39:0/2152271990 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:53:23.492 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:23.477+0000 7fbb262ed640 1 --2- 172.21.15.39:0/2152271990 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbb20102a80 0x7fbb2019a4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:53:23.492 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:23.477+0000 7fbb25aec640 1 --2- 172.21.15.39:0/2152271990 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbb201083c0 0x7fbb2019aa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:53:23.492 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:23.477+0000 7fbb26aee640 1 -- 172.21.15.39:0/2152271990 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbb201083c0 msgr2=0x7fbb2019aa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:53:23.492 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:23.477+0000 7fbb26aee640 1 --2- 172.21.15.39:0/2152271990 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbb201083c0 0x7fbb2019aa40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:23.492 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:23.477+0000 7fbb26aee640 1 -- 172.21.15.39:0/2152271990 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbb20102a80 msgr2=0x7fbb2019a4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:53:23.492 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:23.477+0000 7fbb26aee640 1 --2- 172.21.15.39:0/2152271990 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbb20102a80 0x7fbb2019a4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:23.492 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:23.477+0000 7fbb26aee640 1 -- 172.21.15.39:0/2152271990 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbb201a0350 con 0x7fbb20108d10 2024-09-21T03:53:23.492 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:23.477+0000 7fbb25aec640 1 --2- 172.21.15.39:0/2152271990 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbb201083c0 0x7fbb2019aa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:53:23.492 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:23.477+0000 7fbb26aee640 1 --2- 172.21.15.39:0/2152271990 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbb20108d10 0x7fbb2019fce0 secure :-1 s=READY pgs=1052 cs=0 l=1 rev1=1 crypto rx=0x7fbb1800d750 tx=0x7fbb1800dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:53:23.493 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:23.477+0000 7fbb177fe640 1 -- 172.21.15.39:0/2152271990 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb18017070 con 0x7fbb20108d10 2024-09-21T03:53:23.493 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:23.477+0000 7fbb177fe640 1 -- 172.21.15.39:0/2152271990 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbb180054d0 con 0x7fbb20108d10 2024-09-21T03:53:23.493 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:23.477+0000 7fbb28551640 1 -- 172.21.15.39:0/2152271990 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbb201a0640 con 0x7fbb20108d10 2024-09-21T03:53:23.493 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:23.477+0000 7fbb177fe640 1 -- 172.21.15.39:0/2152271990 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb18004c60 con 0x7fbb20108d10 2024-09-21T03:53:23.493 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:23.477+0000 7fbb28551640 1 -- 172.21.15.39:0/2152271990 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbb201a0b00 con 0x7fbb20108d10 2024-09-21T03:53:23.493 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:23.480+0000 7fbb2eac5740 1 Processor -- start 2024-09-21T03:53:23.493 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:23.480+0000 7fbb177fe640 1 -- 172.21.15.39:0/2152271990 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 27) ==== 154643+0+0 (secure 0 0 0) 0x7fbb18009bd0 con 0x7fbb20108d10 2024-09-21T03:53:23.493 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:23.480+0000 7fbb2eac5740 1 -- start start 2024-09-21T03:53:23.493 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:23.480+0000 7fbb2eac5740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561c62713070 0x561c62aba6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:53:23.493 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:23.480+0000 7fbb2eac5740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561c626ea1b0 0x561c62ababe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:53:23.493 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:23.480+0000 7fbb2eac5740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561c626f2390 0x561c62abb120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:53:23.493 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:23.480+0000 7fbb2eac5740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x561c62a30ea0 con 0x561c626ea1b0 2024-09-21T03:53:23.493 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:23.480+0000 7fbb2eac5740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x561c62a202d0 con 0x561c626f2390 2024-09-21T03:53:23.493 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:23.480+0000 7fbb2eac5740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x561c62a240e0 con 0x561c62713070 2024-09-21T03:53:23.493 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:23.481+0000 7fbb262ed640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561c626ea1b0 0x561c62ababe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:53:23.494 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:23.481+0000 7fbb262ed640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561c626ea1b0 0x561c62ababe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:57288/0 (socket says 172.21.15.39:57288) 2024-09-21T03:53:23.494 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:23.481+0000 7fbb262ed640 1 -- 172.21.15.39:0/1507047333 learned_addr learned my addr 172.21.15.39:0/1507047333 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:53:23.494 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:23.481+0000 7fbb25aec640 1 --2- 172.21.15.39:0/1507047333 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561c62713070 0x561c62aba6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:53:23.494 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:23.481+0000 7fbb25aec640 1 --2- 172.21.15.39:0/1507047333 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561c626f2390 0x561c62abb120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:53:23.494 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:23.481+0000 7fbb262ed640 1 -- 172.21.15.39:0/1507047333 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561c62713070 msgr2=0x561c62aba6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:53:23.494 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:23.481+0000 7fbb262ed640 1 --2- 172.21.15.39:0/1507047333 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561c62713070 0x561c62aba6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:23.494 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:23.481+0000 7fbb262ed640 1 -- 172.21.15.39:0/1507047333 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561c626f2390 msgr2=0x561c62abb120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:53:23.494 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:23.481+0000 7fbb262ed640 1 --2- 172.21.15.39:0/1507047333 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561c626f2390 0x561c62abb120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:23.494 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:23.481+0000 7fbb262ed640 1 -- 172.21.15.39:0/1507047333 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561c6277e650 con 0x561c626ea1b0 2024-09-21T03:53:23.494 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:23.481+0000 7fbb177fe640 1 --2- 172.21.15.39:0/2152271990 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7fbb0c0b66b0 0x7fbb0c0b8bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:53:23.494 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:23.481+0000 7fbb25aec640 1 --2- 172.21.15.39:0/2152271990 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7fbb0c0b66b0 0x7fbb0c0b8bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:53:23.494 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:23.481+0000 7fbb262ed640 1 --2- 172.21.15.39:0/1507047333 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561c626ea1b0 0x561c62ababe0 secure :-1 s=READY pgs=1053 cs=0 l=1 rev1=1 crypto rx=0x7fbb1c007fc0 tx=0x7fbb1c00f850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=45683bfde6d55d91 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:53:23.494 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:23.481+0000 7fbb177fe640 1 -- 172.21.15.39:0/2152271990 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8528+0+0 (secure 0 0 0) 0x7fbb180e70d0 con 0x7fbb20108d10 2024-09-21T03:53:23.494 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:23.481+0000 7fbb14ff9640 1 -- 172.21.15.39:0/1507047333 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb1c004d20 con 0x561c626ea1b0 2024-09-21T03:53:23.495 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:23.481+0000 7fbb14ff9640 1 -- 172.21.15.39:0/1507047333 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbb1c010bb0 con 0x561c626ea1b0 2024-09-21T03:53:23.495 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:23.481+0000 7fbb14ff9640 1 -- 172.21.15.39:0/1507047333 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb1c005700 con 0x561c626ea1b0 2024-09-21T03:53:23.495 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:23.482+0000 7fbb2eac5740 1 -- 172.21.15.39:0/1507047333 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561c626ea1b0 msgr2=0x561c62ababe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:53:23.495 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:23.482+0000 7fbb2eac5740 1 --2- 172.21.15.39:0/1507047333 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561c626ea1b0 0x561c62ababe0 secure :-1 s=READY pgs=1053 cs=0 l=1 rev1=1 crypto rx=0x7fbb1c007fc0 tx=0x7fbb1c00f850 comp rx=0 tx=0).stop 2024-09-21T03:53:23.495 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:23.482+0000 7fbb25aec640 1 --2- 172.21.15.39:0/2152271990 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7fbb0c0b66b0 0x7fbb0c0b8bc0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7fbb10000c00 tx=0x7fbb10031400 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:53:23.495 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:23.482+0000 7fbb2eac5740 1 -- 172.21.15.39:0/1507047333 shutdown_connections 2024-09-21T03:53:23.495 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:23.482+0000 7fbb2eac5740 1 --2- 172.21.15.39:0/1507047333 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561c626f2390 0x561c62abb120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:23.495 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:23.482+0000 7fbb2eac5740 1 --2- 172.21.15.39:0/1507047333 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561c626ea1b0 0x561c62ababe0 unknown :-1 s=CLOSED pgs=1053 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:23.495 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:23.482+0000 7fbb2eac5740 1 --2- 172.21.15.39:0/1507047333 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561c62713070 0x561c62aba6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:23.495 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:23.482+0000 7fbb2eac5740 1 -- 172.21.15.39:0/1507047333 >> 172.21.15.39:0/1507047333 conn(0x561c62707460 msgr2=0x561c62ab8590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:53:23.495 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:23.482+0000 7fbb2eac5740 1 -- 172.21.15.39:0/1507047333 shutdown_connections 2024-09-21T03:53:23.495 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:23.483+0000 7fbb2eac5740 1 -- 172.21.15.39:0/1507047333 wait complete. 2024-09-21T03:53:23.495 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:23.483+0000 7fbb2eac5740 1 -- start start 2024-09-21T03:53:23.495 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:23.483+0000 7fbb2eac5740 1 Processor -- start 2024-09-21T03:53:23.496 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:23.483+0000 7fbb2eac5740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561c626ea1b0 0x561c62aca0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:53:23.496 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:23.483+0000 7fbb2eac5740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561c626f2390 0x561c62aca620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:53:23.496 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:23.483+0000 7fbb2eac5740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561c62713070 0x561c62acaba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:53:23.496 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:23.483+0000 7fbb2eac5740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x561c62881700 con 0x561c62713070 2024-09-21T03:53:23.496 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:23.483+0000 7fbb2eac5740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x561c629bb160 con 0x561c626f2390 2024-09-21T03:53:23.496 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:23.483+0000 7fbb2eac5740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x561c62acd1b0 con 0x561c626ea1b0 2024-09-21T03:53:23.496 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:23.483+0000 7fbb2eac5740 10 client.0 _ll_register_callbacks cb 0x561c629d49e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:53:23.496 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:23.483+0000 7fbb26aee640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561c62713070 0x561c62acaba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:53:23.496 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:23.484+0000 7fbb26aee640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561c62713070 0x561c62acaba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:57300/0 (socket says 172.21.15.39:57300) 2024-09-21T03:53:23.496 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:23.484+0000 7fbb26aee640 1 -- 172.21.15.39:0/3847778189 learned_addr learned my addr 172.21.15.39:0/3847778189 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:53:23.496 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:23.484+0000 7fbb262ed640 1 --2- 172.21.15.39:0/3847778189 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561c626ea1b0 0x561c62aca0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:53:23.496 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:23.484+0000 7fbb25aec640 1 --2- 172.21.15.39:0/3847778189 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561c626f2390 0x561c62aca620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:53:23.496 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:23.484+0000 7fbb26aee640 1 -- 172.21.15.39:0/3847778189 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561c626ea1b0 msgr2=0x561c62aca0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:53:23.496 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:23.484+0000 7fbb26aee640 1 --2- 172.21.15.39:0/3847778189 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561c626ea1b0 0x561c62aca0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:23.497 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:23.484+0000 7fbb26aee640 1 -- 172.21.15.39:0/3847778189 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561c626f2390 msgr2=0x561c62aca620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:53:23.497 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:23.484+0000 7fbb26aee640 1 --2- 172.21.15.39:0/3847778189 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561c626f2390 0x561c62aca620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:23.497 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:23.484+0000 7fbb26aee640 1 -- 172.21.15.39:0/3847778189 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561c629e8c90 con 0x561c62713070 2024-09-21T03:53:23.497 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:23.484+0000 7fbb26aee640 1 --2- 172.21.15.39:0/3847778189 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561c62713070 0x561c62acaba0 secure :-1 s=READY pgs=1054 cs=0 l=1 rev1=1 crypto rx=0x7fbb180acc20 tx=0x7fbb180ac970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb0a5552daa8f00a server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:53:23.497 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:23.484+0000 7fbb09ffb640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-09-21T03:53:23.497 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:23.484+0000 7fbb09ffb640 1 -- 172.21.15.39:0/3847778189 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb18020070 con 0x561c62713070 2024-09-21T03:53:23.497 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:23.484+0000 7fbb09ffb640 1 -- 172.21.15.39:0/3847778189 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8528+0+0 (secure 0 0 0) 0x7fbb180a8870 con 0x561c62713070 2024-09-21T03:53:23.497 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:23.484+0000 7fbb2eac5740 1 -- 172.21.15.39:0/3847778189 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbb1c010bb0 con 0x561c62713070 2024-09-21T03:53:23.497 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:23.484+0000 7fbb09ffb640 1 -- 172.21.15.39:0/3847778189 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fbb180ad620 con 0x561c62713070 2024-09-21T03:53:23.497 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:23.484+0000 7fbb2eac5740 10 client.17445 fetch_fsmap learned FSMap version 29 2024-09-21T03:53:23.497 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:23.484+0000 7fbb2eac5740 1 -- 172.21.15.39:0/3847778189 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x561c62ad8ef0 con 0x561c62713070 2024-09-21T03:53:23.497 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:23.485+0000 7fbb09ffb640 1 -- 172.21.15.39:0/3847778189 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbb1c010bb0 con 0x561c62713070 2024-09-21T03:53:23.497 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:23.485+0000 7fbb09ffb640 1 -- 172.21.15.39:0/3847778189 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fbb180b0020 con 0x561c62713070 2024-09-21T03:53:23.497 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:23.485+0000 7fbb2eac5740 10 client.17445 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T03:53:23.498 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:23.485+0000 7fbb2eac5740 10 client.17445 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.a 2024-09-21T03:53:23.498 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:23.485+0000 7fbb2eac5740 20 client.17445 populate_metadata read hostname 'smithi039' 2024-09-21T03:53:23.498 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:23.485+0000 7fbb2eac5740 1 --2- 172.21.15.39:0/3847778189 >> [v2:172.21.15.39:6840/4247190456,v1:172.21.15.39:6841/4247190456] conn(0x561c626d19f0 0x561c62adc450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:53:23.498 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:23.485+0000 7fbb2eac5740 4 client.17445 mds_command: new command op to 14868 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:53:23.498 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:23.485+0000 7fbb2eac5740 1 -- 172.21.15.39:0/3847778189 --> [v2:172.21.15.39:6840/4247190456,v1:172.21.15.39:6841/4247190456] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561c62a32df0 con 0x561c626d19f0 2024-09-21T03:53:23.498 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:23.485+0000 7fbb262ed640 1 --2- 172.21.15.39:0/3847778189 >> [v2:172.21.15.39:6840/4247190456,v1:172.21.15.39:6841/4247190456] conn(0x561c626d19f0 0x561c62adc450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:53:23.498 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:23.485+0000 7fbb09ffb640 10 client.17445 ms_handle_connect on v2:172.21.15.39:6840/4247190456 2024-09-21T03:53:23.498 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:23.486+0000 7fbb262ed640 1 --2- 172.21.15.39:0/3847778189 >> [v2:172.21.15.39:6840/4247190456,v1:172.21.15.39:6841/4247190456] conn(0x561c626d19f0 0x561c62adc450 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=85c0cb4ed1c644ee server_cookie=1e72987df128ef4e in_seq=0 out_seq=0 2024-09-21T03:53:23.498 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:23.486+0000 7fbb09ffb640 1 -- 172.21.15.39:0/3847778189 <== mds.4 v2:172.21.15.39:6840/4247190456 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x561c62a32df0 con 0x561c626d19f0 2024-09-21T03:53:23.498 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:23.486+0000 7fbb09ffb640 10 client.17445 handle_command_reply: tid=0 2024-09-21T03:53:23.498 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:23.486+0000 7fbb2eac5740 1 client.17445 shutdown 2024-09-21T03:53:23.498 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:23.486+0000 7fbb2eac5740 1 -- 172.21.15.39:0/3847778189 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561c62713070 msgr2=0x561c62acaba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:53:23.498 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:23.486+0000 7fbb2eac5740 1 --2- 172.21.15.39:0/3847778189 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561c62713070 0x561c62acaba0 secure :-1 s=READY pgs=1054 cs=0 l=1 rev1=1 crypto rx=0x7fbb180acc20 tx=0x7fbb180ac970 comp rx=0 tx=0).stop 2024-09-21T03:53:23.498 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:23.486+0000 7fbb2eac5740 1 -- 172.21.15.39:0/3847778189 shutdown_connections 2024-09-21T03:53:23.499 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:23.486+0000 7fbb2eac5740 1 --2- 172.21.15.39:0/3847778189 >> [v2:172.21.15.39:6840/4247190456,v1:172.21.15.39:6841/4247190456] conn(0x561c626d19f0 0x561c62adc450 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:23.499 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:23.486+0000 7fbb2eac5740 1 --2- 172.21.15.39:0/3847778189 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561c62713070 0x561c62acaba0 unknown :-1 s=CLOSED pgs=1054 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:23.499 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:23.486+0000 7fbb2eac5740 1 --2- 172.21.15.39:0/3847778189 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561c626f2390 0x561c62aca620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:23.499 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:23.486+0000 7fbb2eac5740 1 --2- 172.21.15.39:0/3847778189 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561c626ea1b0 0x561c62aca0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:23.499 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:23.486+0000 7fbb09ffb640 0 client.17445 ms_handle_reset on v2:172.21.15.39:6840/4247190456 2024-09-21T03:53:23.499 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:23.486+0000 7fbb2eac5740 1 -- 172.21.15.39:0/3847778189 >> 172.21.15.39:0/3847778189 conn(0x561c62707460 msgr2=0x561c62abd830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:53:23.499 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:23.486+0000 7fbb2eac5740 1 -- 172.21.15.39:0/3847778189 shutdown_connections 2024-09-21T03:53:23.499 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:23.487+0000 7fbb2eac5740 1 -- 172.21.15.39:0/3847778189 wait complete. 2024-09-21T03:53:23.499 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:23.487+0000 7fbb2eac5740 20 client.17445 trim_cache size 0 max 16384 2024-09-21T03:53:23.502 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:23.500+0000 7fbb2eac5740 1 Processor -- start 2024-09-21T03:53:23.502 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:23.500+0000 7fbb2eac5740 1 -- start start 2024-09-21T03:53:23.502 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:23.500+0000 7fbb2eac5740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561c626ea1b0 0x561c62acc0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:53:23.502 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:23.500+0000 7fbb2eac5740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561c626f2390 0x561c62ac72d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:53:23.503 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:23.500+0000 7fbb2eac5740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561c62713070 0x561c62ac7810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:53:23.503 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:23.500+0000 7fbb2eac5740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fbb201ad4d0 con 0x561c626f2390 2024-09-21T03:53:23.503 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:23.500+0000 7fbb2eac5740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fbb201af010 con 0x561c62713070 2024-09-21T03:53:23.503 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:23.500+0000 7fbb2eac5740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fbb201adba0 con 0x561c626ea1b0 2024-09-21T03:53:23.503 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:23.500+0000 7fbb25aec640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561c626f2390 0x561c62ac72d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:53:23.503 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:23.500+0000 7fbb25aec640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561c626f2390 0x561c62ac72d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:57302/0 (socket says 172.21.15.39:57302) 2024-09-21T03:53:23.503 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:23.500+0000 7fbb25aec640 1 -- 172.21.15.39:0/4177133165 learned_addr learned my addr 172.21.15.39:0/4177133165 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:53:23.503 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:23.500+0000 7fbb262ed640 1 --2- 172.21.15.39:0/4177133165 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561c626ea1b0 0x561c62acc0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:53:23.503 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:23.501+0000 7fbb26aee640 1 --2- 172.21.15.39:0/4177133165 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561c62713070 0x561c62ac7810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:53:23.503 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:23.501+0000 7fbb25aec640 1 -- 172.21.15.39:0/4177133165 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561c626ea1b0 msgr2=0x561c62acc0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:53:23.503 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:23.501+0000 7fbb25aec640 1 --2- 172.21.15.39:0/4177133165 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561c626ea1b0 0x561c62acc0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:23.503 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:23.501+0000 7fbb25aec640 1 -- 172.21.15.39:0/4177133165 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561c62713070 msgr2=0x561c62ac7810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:53:23.503 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:23.501+0000 7fbb25aec640 1 --2- 172.21.15.39:0/4177133165 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561c62713070 0x561c62ac7810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:23.503 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:23.501+0000 7fbb25aec640 1 -- 172.21.15.39:0/4177133165 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561c62abd4a0 con 0x561c626f2390 2024-09-21T03:53:23.507 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:23.501+0000 7fbb25aec640 1 --2- 172.21.15.39:0/4177133165 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561c626f2390 0x561c62ac72d0 secure :-1 s=READY pgs=1055 cs=0 l=1 rev1=1 crypto rx=0x7fbb10009900 tx=0x7fbb1003f070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=792faf96eea4cb33 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:53:23.507 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:23.501+0000 7fbb0bfff640 1 -- 172.21.15.39:0/4177133165 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb10035040 con 0x561c626f2390 2024-09-21T03:53:23.507 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:23.501+0000 7fbb0bfff640 1 -- 172.21.15.39:0/4177133165 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbb1004f070 con 0x561c626f2390 2024-09-21T03:53:23.507 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:23.501+0000 7fbb0bfff640 1 -- 172.21.15.39:0/4177133165 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb10054930 con 0x561c626f2390 2024-09-21T03:53:23.507 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:23.502+0000 7fbb2eac5740 1 -- 172.21.15.39:0/4177133165 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561c626f2390 msgr2=0x561c62ac72d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:53:23.507 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:23.502+0000 7fbb2eac5740 1 --2- 172.21.15.39:0/4177133165 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561c626f2390 0x561c62ac72d0 secure :-1 s=READY pgs=1055 cs=0 l=1 rev1=1 crypto rx=0x7fbb10009900 tx=0x7fbb1003f070 comp rx=0 tx=0).stop 2024-09-21T03:53:23.507 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:23.502+0000 7fbb2eac5740 1 -- 172.21.15.39:0/4177133165 shutdown_connections 2024-09-21T03:53:23.507 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:23.502+0000 7fbb2eac5740 1 --2- 172.21.15.39:0/4177133165 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561c62713070 0x561c62ac7810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:23.507 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:23.502+0000 7fbb2eac5740 1 --2- 172.21.15.39:0/4177133165 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561c626f2390 0x561c62ac72d0 unknown :-1 s=CLOSED pgs=1055 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:23.507 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:23.502+0000 7fbb2eac5740 1 --2- 172.21.15.39:0/4177133165 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561c626ea1b0 0x561c62acc0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:23.507 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:23.502+0000 7fbb2eac5740 1 -- 172.21.15.39:0/4177133165 >> 172.21.15.39:0/4177133165 conn(0x561c62707460 msgr2=0x561c62abc090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:53:23.507 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:23.502+0000 7fbb2eac5740 1 -- 172.21.15.39:0/4177133165 shutdown_connections 2024-09-21T03:53:23.508 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:23.502+0000 7fbb2eac5740 1 -- 172.21.15.39:0/4177133165 wait complete. 2024-09-21T03:53:23.508 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:23.502+0000 7fbb2eac5740 1 -- start start 2024-09-21T03:53:23.508 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:23.502+0000 7fbb2eac5740 1 Processor -- start 2024-09-21T03:53:23.508 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:23.503+0000 7fbb2eac5740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561c626ea1b0 0x561c62ace460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:53:23.508 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:23.503+0000 7fbb2eac5740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561c626f2390 0x561c62ace9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:53:23.508 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:23.503+0000 7fbb2eac5740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561c62713070 0x561c62aceee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:53:23.508 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:23.503+0000 7fbb2eac5740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fbb201ae940 con 0x561c62713070 2024-09-21T03:53:23.508 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:23.503+0000 7fbb2eac5740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fbb201ae270 con 0x561c626f2390 2024-09-21T03:53:23.508 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:23.503+0000 7fbb2eac5740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fbb201ab4b0 con 0x561c626ea1b0 2024-09-21T03:53:23.508 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:23.503+0000 7fbb2eac5740 10 client.0 _ll_register_callbacks cb 0x7fbb180a9570 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:53:23.508 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:23.503+0000 7fbb26aee640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561c62713070 0x561c62aceee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:53:23.508 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:23.503+0000 7fbb26aee640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561c62713070 0x561c62aceee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:57310/0 (socket says 172.21.15.39:57310) 2024-09-21T03:53:23.508 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:23.503+0000 7fbb26aee640 1 -- 172.21.15.39:0/2352622083 learned_addr learned my addr 172.21.15.39:0/2352622083 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:53:23.508 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:23.503+0000 7fbb25aec640 1 --2- 172.21.15.39:0/2352622083 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561c626f2390 0x561c62ace9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:53:23.509 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:23.503+0000 7fbb262ed640 1 --2- 172.21.15.39:0/2352622083 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561c626ea1b0 0x561c62ace460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:53:23.509 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:23.503+0000 7fbb26aee640 1 -- 172.21.15.39:0/2352622083 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561c626ea1b0 msgr2=0x561c62ace460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:53:23.509 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:23.503+0000 7fbb26aee640 1 --2- 172.21.15.39:0/2352622083 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561c626ea1b0 0x561c62ace460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:23.509 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:23.503+0000 7fbb26aee640 1 -- 172.21.15.39:0/2352622083 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561c626f2390 msgr2=0x561c62ace9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:53:23.509 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:23.503+0000 7fbb26aee640 1 --2- 172.21.15.39:0/2352622083 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561c626f2390 0x561c62ace9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:23.509 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:23.503+0000 7fbb26aee640 1 -- 172.21.15.39:0/2352622083 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561c62b45e60 con 0x561c62713070 2024-09-21T03:53:23.509 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:23.503+0000 7fbb262ed640 1 --2- 172.21.15.39:0/2352622083 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561c626ea1b0 0x561c62ace460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:53:23.509 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:23.503+0000 7fbb25aec640 1 --2- 172.21.15.39:0/2352622083 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561c626f2390 0x561c62ace9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:53:23.509 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:23.504+0000 7fbb26aee640 1 --2- 172.21.15.39:0/2352622083 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561c62713070 0x561c62aceee0 secure :-1 s=READY pgs=1056 cs=0 l=1 rev1=1 crypto rx=0x7fbb180983a0 tx=0x7fbb180944d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=10d5bbfa3cfc696 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:53:23.509 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:23.504+0000 7fbb09ffb640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-09-21T03:53:23.509 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:23.504+0000 7fbb09ffb640 1 -- 172.21.15.39:0/2352622083 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb18020070 con 0x561c62713070 2024-09-21T03:53:23.509 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:23.504+0000 7fbb09ffb640 1 -- 172.21.15.39:0/2352622083 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8528+0+0 (secure 0 0 0) 0x7fbb180d4390 con 0x561c62713070 2024-09-21T03:53:23.509 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:23.504+0000 7fbb2eac5740 1 -- 172.21.15.39:0/2352622083 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbb1004f070 con 0x561c62713070 2024-09-21T03:53:23.509 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:23.504+0000 7fbb09ffb640 1 -- 172.21.15.39:0/2352622083 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fbb180abf90 con 0x561c62713070 2024-09-21T03:53:23.509 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:23.504+0000 7fbb2eac5740 10 client.17451 fetch_fsmap learned FSMap version 29 2024-09-21T03:53:23.510 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:23.504+0000 7fbb2eac5740 1 -- 172.21.15.39:0/2352622083 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x561c62b470e0 con 0x561c62713070 2024-09-21T03:53:23.510 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:23.504+0000 7fbb09ffb640 1 -- 172.21.15.39:0/2352622083 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbb1004f070 con 0x561c62713070 2024-09-21T03:53:23.510 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:23.504+0000 7fbb09ffb640 1 -- 172.21.15.39:0/2352622083 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fbb180b0070 con 0x561c62713070 2024-09-21T03:53:23.510 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:23.505+0000 7fbb2eac5740 10 client.17451 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T03:53:23.510 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:23.505+0000 7fbb2eac5740 10 client.17451 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.a 2024-09-21T03:53:23.510 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:23.505+0000 7fbb2eac5740 20 client.17451 populate_metadata read hostname 'smithi039' 2024-09-21T03:53:23.510 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:23.505+0000 7fbb2eac5740 1 --2- 172.21.15.39:0/2352622083 >> [v2:172.21.15.39:6840/4247190456,v1:172.21.15.39:6841/4247190456] conn(0x561c626a6da0 0x561c62b49810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:53:23.510 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:23.505+0000 7fbb2eac5740 4 client.17451 mds_command: new command op to 14868 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T03:53:23.510 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:23.505+0000 7fbb2eac5740 1 -- 172.21.15.39:0/2352622083 --> [v2:172.21.15.39:6840/4247190456,v1:172.21.15.39:6841/4247190456] -- command(tid 0: {"prefix": "damage ls"}) -- 0x561c62ab39e0 con 0x561c626a6da0 2024-09-21T03:53:23.510 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:23.505+0000 7fbb262ed640 1 --2- 172.21.15.39:0/2352622083 >> [v2:172.21.15.39:6840/4247190456,v1:172.21.15.39:6841/4247190456] conn(0x561c626a6da0 0x561c62b49810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:53:23.510 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:23.505+0000 7fbb09ffb640 10 client.17451 ms_handle_connect on v2:172.21.15.39:6840/4247190456 2024-09-21T03:53:23.510 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:23.505+0000 7fbb262ed640 1 --2- 172.21.15.39:0/2352622083 >> [v2:172.21.15.39:6840/4247190456,v1:172.21.15.39:6841/4247190456] conn(0x561c626a6da0 0x561c62b49810 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=19b710331c35656b server_cookie=1707648c70974f63 in_seq=0 out_seq=0 2024-09-21T03:53:23.511 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:23.505+0000 7fbb09ffb640 1 -- 172.21.15.39:0/2352622083 <== mds.4 v2:172.21.15.39:6840/4247190456 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x561c62ab39e0 con 0x561c626a6da0 2024-09-21T03:53:23.511 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:23.505+0000 7fbb09ffb640 10 client.17451 handle_command_reply: tid=0 2024-09-21T03:53:23.511 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:23.505+0000 7fbb2eac5740 1 client.17451 shutdown 2024-09-21T03:53:23.511 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:23.505+0000 7fbb2eac5740 1 -- 172.21.15.39:0/2352622083 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561c62713070 msgr2=0x561c62aceee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:53:23.511 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:23.505+0000 7fbb2eac5740 1 --2- 172.21.15.39:0/2352622083 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561c62713070 0x561c62aceee0 secure :-1 s=READY pgs=1056 cs=0 l=1 rev1=1 crypto rx=0x7fbb180983a0 tx=0x7fbb180944d0 comp rx=0 tx=0).stop 2024-09-21T03:53:23.512 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:23.505+0000 7fbb2eac5740 1 -- 172.21.15.39:0/2352622083 shutdown_connections 2024-09-21T03:53:23.512 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:23.506+0000 7fbb2eac5740 1 --2- 172.21.15.39:0/2352622083 >> [v2:172.21.15.39:6840/4247190456,v1:172.21.15.39:6841/4247190456] conn(0x561c626a6da0 0x561c62b49810 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:23.512 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:23.506+0000 7fbb2eac5740 1 --2- 172.21.15.39:0/2352622083 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561c62713070 0x561c62aceee0 unknown :-1 s=CLOSED pgs=1056 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:23.512 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:23.506+0000 7fbb2eac5740 1 --2- 172.21.15.39:0/2352622083 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561c626f2390 0x561c62ace9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:23.512 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:23.506+0000 7fbb09ffb640 0 client.17451 ms_handle_reset on v2:172.21.15.39:6840/4247190456 2024-09-21T03:53:23.512 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:23.506+0000 7fbb2eac5740 1 --2- 172.21.15.39:0/2352622083 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561c626ea1b0 0x561c62ace460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:23.512 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:23.506+0000 7fbb2eac5740 1 -- 172.21.15.39:0/2352622083 >> 172.21.15.39:0/2352622083 conn(0x561c62707460 msgr2=0x561c62abc090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:53:23.512 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:23.506+0000 7fbb2eac5740 1 -- 172.21.15.39:0/2352622083 shutdown_connections 2024-09-21T03:53:23.512 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:23.506+0000 7fbb2eac5740 1 -- 172.21.15.39:0/2352622083 wait complete. 2024-09-21T03:53:23.512 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:23.506+0000 7fbb2eac5740 20 client.17451 trim_cache size 0 max 16384 2024-09-21T03:53:23.512 INFO:teuthology.orchestra.run.smithi039.stdout:[] 2024-09-21T03:53:23.513 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:23.506+0000 7fbb28551640 1 -- 172.21.15.39:0/2152271990 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7fbb0c0b66b0 msgr2=0x7fbb0c0b8bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:53:23.513 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:23.506+0000 7fbb28551640 1 --2- 172.21.15.39:0/2152271990 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7fbb0c0b66b0 0x7fbb0c0b8bc0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7fbb10000c00 tx=0x7fbb10031400 comp rx=0 tx=0).stop 2024-09-21T03:53:23.513 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:23.506+0000 7fbb28551640 1 -- 172.21.15.39:0/2152271990 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbb20108d10 msgr2=0x7fbb2019fce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:53:23.513 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:23.506+0000 7fbb28551640 1 --2- 172.21.15.39:0/2152271990 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbb20108d10 0x7fbb2019fce0 secure :-1 s=READY pgs=1052 cs=0 l=1 rev1=1 crypto rx=0x7fbb1800d750 tx=0x7fbb1800dc10 comp rx=0 tx=0).stop 2024-09-21T03:53:23.513 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:23.506+0000 7fbb28551640 1 -- 172.21.15.39:0/2152271990 shutdown_connections 2024-09-21T03:53:23.513 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:23.506+0000 7fbb28551640 1 --2- 172.21.15.39:0/2152271990 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7fbb0c0b66b0 0x7fbb0c0b8bc0 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:23.514 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:23.506+0000 7fbb28551640 1 --2- 172.21.15.39:0/2152271990 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbb20108d10 0x7fbb2019fce0 unknown :-1 s=CLOSED pgs=1052 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:23.514 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:23.506+0000 7fbb28551640 1 --2- 172.21.15.39:0/2152271990 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbb201083c0 0x7fbb2019aa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:23.514 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:23.506+0000 7fbb28551640 1 --2- 172.21.15.39:0/2152271990 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbb20102a80 0x7fbb2019a4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:23.514 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:23.506+0000 7fbb28551640 1 -- 172.21.15.39:0/2152271990 >> 172.21.15.39:0/2152271990 conn(0x7fbb20079740 msgr2=0x7fbb20101a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:53:23.514 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:23.507+0000 7fbb28551640 1 -- 172.21.15.39:0/2152271990 shutdown_connections 2024-09-21T03:53:23.514 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:23.507+0000 7fbb28551640 1 -- 172.21.15.39:0/2152271990 wait complete. 2024-09-21T03:53:24.320 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 51116d5c-77ca-11ef-baf5-efdc52797490 -- ceph tell mds.1:3 damage ls 2024-09-21T03:53:24.582 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/51116d5c-77ca-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:53:24.983 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:53:24 smithi039 ceph-mon[31211]: pgmap v548: 97 pgs: 97 active+clean; 10 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 102 B/s wr, 10 op/s 2024-09-21T03:53:25.077 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:25.075+0000 7f50681e3640 1 -- 172.21.15.39:0/3900658546 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5060104a30 msgr2=0x7f5060104e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:53:25.077 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:25.075+0000 7f50681e3640 1 --2- 172.21.15.39:0/3900658546 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5060104a30 0x7f5060104e40 secure :-1 s=READY pgs=1057 cs=0 l=1 rev1=1 crypto rx=0x7f505c009900 tx=0x7f505c02a410 comp rx=0 tx=0).stop 2024-09-21T03:53:25.077 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:25.075+0000 7f50681e3640 1 -- 172.21.15.39:0/3900658546 shutdown_connections 2024-09-21T03:53:25.077 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:25.075+0000 7f50681e3640 1 --2- 172.21.15.39:0/3900658546 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f506010de90 0x7f5060110380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:25.077 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:25.075+0000 7f50681e3640 1 --2- 172.21.15.39:0/3900658546 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5060105380 0x7f506010d8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:25.078 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:25.075+0000 7f50681e3640 1 --2- 172.21.15.39:0/3900658546 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5060104a30 0x7f5060104e40 unknown :-1 s=CLOSED pgs=1057 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:25.078 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:25.075+0000 7f50681e3640 1 -- 172.21.15.39:0/3900658546 >> 172.21.15.39:0/3900658546 conn(0x7f5060079770 msgr2=0x7f5060079b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:53:25.078 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:25.075+0000 7f50681e3640 1 -- 172.21.15.39:0/3900658546 shutdown_connections 2024-09-21T03:53:25.078 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:25.075+0000 7f50681e3640 1 -- 172.21.15.39:0/3900658546 wait complete. 2024-09-21T03:53:25.079 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:25.076+0000 7f50681e3640 1 Processor -- start 2024-09-21T03:53:25.080 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:25.076+0000 7f50681e3640 1 -- start start 2024-09-21T03:53:25.080 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:25.076+0000 7f50681e3640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5060104a30 0x7f506019a5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:53:25.080 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:25.076+0000 7f50681e3640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5060105380 0x7f506019ab40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:53:25.080 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:25.076+0000 7f50681e3640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f506010de90 0x7f506019fde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:53:25.080 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:25.076+0000 7f50681e3640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f5060113b10 con 0x7f5060105380 2024-09-21T03:53:25.080 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:25.076+0000 7f50681e3640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f5060113990 con 0x7f5060104a30 2024-09-21T03:53:25.080 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:25.076+0000 7f50681e3640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f5060113c90 con 0x7f506010de90 2024-09-21T03:53:25.080 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:25.076+0000 7f506577e640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5060105380 0x7f506019ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:53:25.080 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:25.076+0000 7f506577e640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5060105380 0x7f506019ab40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:57314/0 (socket says 172.21.15.39:57314) 2024-09-21T03:53:25.080 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:25.076+0000 7f506577e640 1 -- 172.21.15.39:0/2338738562 learned_addr learned my addr 172.21.15.39:0/2338738562 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:53:25.080 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:25.076+0000 7f5065f7f640 1 --2- 172.21.15.39:0/2338738562 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5060104a30 0x7f506019a5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:53:25.080 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:25.077+0000 7f5066780640 1 --2- 172.21.15.39:0/2338738562 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f506010de90 0x7f506019fde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:53:25.080 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:25.077+0000 7f506577e640 1 -- 172.21.15.39:0/2338738562 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f506010de90 msgr2=0x7f506019fde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:53:25.080 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:25.077+0000 7f506577e640 1 --2- 172.21.15.39:0/2338738562 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f506010de90 0x7f506019fde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:25.081 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:25.077+0000 7f506577e640 1 -- 172.21.15.39:0/2338738562 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5060104a30 msgr2=0x7f506019a5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:53:25.081 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:25.077+0000 7f506577e640 1 --2- 172.21.15.39:0/2338738562 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5060104a30 0x7f506019a5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:25.081 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:25.077+0000 7f506577e640 1 -- 172.21.15.39:0/2338738562 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f50601a0510 con 0x7f5060105380 2024-09-21T03:53:25.081 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:25.077+0000 7f506577e640 1 --2- 172.21.15.39:0/2338738562 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5060105380 0x7f506019ab40 secure :-1 s=READY pgs=1058 cs=0 l=1 rev1=1 crypto rx=0x7f505000d750 tx=0x7f505000dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:53:25.081 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:25.077+0000 7f5056ffd640 1 -- 172.21.15.39:0/2338738562 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5050017070 con 0x7f5060105380 2024-09-21T03:53:25.081 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:25.077+0000 7f5056ffd640 1 -- 172.21.15.39:0/2338738562 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f50500054d0 con 0x7f5060105380 2024-09-21T03:53:25.081 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:25.077+0000 7f50681e3640 1 -- 172.21.15.39:0/2338738562 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f50601a0800 con 0x7f5060105380 2024-09-21T03:53:25.081 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:25.077+0000 7f5056ffd640 1 -- 172.21.15.39:0/2338738562 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5050004c30 con 0x7f5060105380 2024-09-21T03:53:25.081 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:25.077+0000 7f50681e3640 1 -- 172.21.15.39:0/2338738562 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f50601a6f60 con 0x7f5060105380 2024-09-21T03:53:25.081 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:25.080+0000 7f5056ffd640 1 -- 172.21.15.39:0/2338738562 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 27) ==== 154643+0+0 (secure 0 0 0) 0x7f5050004dd0 con 0x7f5060105380 2024-09-21T03:53:25.082 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:25.080+0000 7f506e757740 1 Processor -- start 2024-09-21T03:53:25.083 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:25.080+0000 7f506e757740 1 -- start start 2024-09-21T03:53:25.083 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:25.080+0000 7f506e757740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x559e06d4c070 0x559e070f32c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:53:25.083 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:25.081+0000 7f506e757740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559e06d231b0 0x559e070f3800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:53:25.083 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:25.081+0000 7f506e757740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559e06d2b390 0x559e070f3d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:53:25.083 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:25.081+0000 7f506e757740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x559e07069ea0 con 0x559e06d2b390 2024-09-21T03:53:25.083 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:25.081+0000 7f506e757740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x559e070592d0 con 0x559e06d4c070 2024-09-21T03:53:25.083 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:25.081+0000 7f506e757740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x559e0705d0e0 con 0x559e06d231b0 2024-09-21T03:53:25.083 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:25.081+0000 7f506577e640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559e06d2b390 0x559e070f3d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:53:25.083 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:25.081+0000 7f506577e640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559e06d2b390 0x559e070f3d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:57318/0 (socket says 172.21.15.39:57318) 2024-09-21T03:53:25.084 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:25.081+0000 7f506577e640 1 -- 172.21.15.39:0/1231361006 learned_addr learned my addr 172.21.15.39:0/1231361006 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:53:25.084 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:25.081+0000 7f5065f7f640 1 --2- 172.21.15.39:0/1231361006 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559e06d231b0 0x559e070f3800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:53:25.084 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:25.081+0000 7f5066780640 1 --2- 172.21.15.39:0/1231361006 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x559e06d4c070 0x559e070f32c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:53:25.084 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:25.081+0000 7f5056ffd640 1 --2- 172.21.15.39:0/2338738562 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f504c0b6680 0x7f504c0b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:53:25.084 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:25.081+0000 7f5066780640 1 --2- 172.21.15.39:0/2338738562 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f504c0b6680 0x7f504c0b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:53:25.084 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:25.081+0000 7f5056ffd640 1 -- 172.21.15.39:0/2338738562 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8528+0+0 (secure 0 0 0) 0x7f50500e7090 con 0x7f5060105380 2024-09-21T03:53:25.084 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:25.081+0000 7f506577e640 1 -- 172.21.15.39:0/1231361006 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559e06d231b0 msgr2=0x559e070f3800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:53:25.084 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:25.081+0000 7f506577e640 1 --2- 172.21.15.39:0/1231361006 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559e06d231b0 0x559e070f3800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:25.084 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:25.081+0000 7f506577e640 1 -- 172.21.15.39:0/1231361006 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x559e06d4c070 msgr2=0x559e070f32c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:53:25.084 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:25.081+0000 7f506577e640 1 --2- 172.21.15.39:0/1231361006 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x559e06d4c070 0x559e070f32c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:25.084 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:25.081+0000 7f506577e640 1 -- 172.21.15.39:0/1231361006 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559e06db7650 con 0x559e06d2b390 2024-09-21T03:53:25.087 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:25.082+0000 7f506577e640 1 --2- 172.21.15.39:0/1231361006 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559e06d2b390 0x559e070f3d40 secure :-1 s=READY pgs=1059 cs=0 l=1 rev1=1 crypto rx=0x7f50500e4720 tx=0x7f5050009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=532f05ec1ba0d624 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:53:25.087 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:25.082+0000 7f504bfff640 1 -- 172.21.15.39:0/1231361006 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5050020070 con 0x559e06d2b390 2024-09-21T03:53:25.087 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:25.082+0000 7f5066780640 1 --2- 172.21.15.39:0/2338738562 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f504c0b6680 0x7f504c0b8b90 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f5058005fd0 tx=0x7f5058004ab0 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:53:25.087 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:25.082+0000 7f504bfff640 1 -- 172.21.15.39:0/1231361006 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f50500acaa0 con 0x559e06d2b390 2024-09-21T03:53:25.088 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:25.082+0000 7f504bfff640 1 -- 172.21.15.39:0/1231361006 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50500b6900 con 0x559e06d2b390 2024-09-21T03:53:25.088 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:25.083+0000 7f506e757740 1 -- 172.21.15.39:0/1231361006 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559e06d2b390 msgr2=0x559e070f3d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:53:25.088 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:25.083+0000 7f506e757740 1 --2- 172.21.15.39:0/1231361006 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559e06d2b390 0x559e070f3d40 secure :-1 s=READY pgs=1059 cs=0 l=1 rev1=1 crypto rx=0x7f50500e4720 tx=0x7f5050009f90 comp rx=0 tx=0).stop 2024-09-21T03:53:25.088 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:25.083+0000 7f506e757740 1 -- 172.21.15.39:0/1231361006 shutdown_connections 2024-09-21T03:53:25.088 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:25.083+0000 7f506e757740 1 --2- 172.21.15.39:0/1231361006 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559e06d2b390 0x559e070f3d40 unknown :-1 s=CLOSED pgs=1059 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:25.088 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:25.083+0000 7f506e757740 1 --2- 172.21.15.39:0/1231361006 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559e06d231b0 0x559e070f3800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:25.088 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:25.083+0000 7f506e757740 1 --2- 172.21.15.39:0/1231361006 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x559e06d4c070 0x559e070f32c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:25.088 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:25.083+0000 7f506e757740 1 -- 172.21.15.39:0/1231361006 >> 172.21.15.39:0/1231361006 conn(0x559e06d40460 msgr2=0x559e070f11b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:53:25.088 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:25.083+0000 7f506e757740 1 -- 172.21.15.39:0/1231361006 shutdown_connections 2024-09-21T03:53:25.088 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:25.083+0000 7f506e757740 1 -- 172.21.15.39:0/1231361006 wait complete. 2024-09-21T03:53:25.088 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:25.083+0000 7f506e757740 1 -- start start 2024-09-21T03:53:25.088 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:25.083+0000 7f506e757740 1 Processor -- start 2024-09-21T03:53:25.088 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:25.084+0000 7f506e757740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559e06d231b0 0x559e070fe980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:53:25.088 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:25.084+0000 7f506e757740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559e06d2b390 0x559e070feec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:53:25.089 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:25.084+0000 7f506e757740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x559e06d4c070 0x559e0710acd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:53:25.089 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:25.084+0000 7f506e757740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x559e06eba700 con 0x559e06d2b390 2024-09-21T03:53:25.089 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:25.084+0000 7f506e757740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x559e06ff4160 con 0x559e06d4c070 2024-09-21T03:53:25.089 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:25.084+0000 7f506e757740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x559e071019e0 con 0x559e06d231b0 2024-09-21T03:53:25.089 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:25.084+0000 7f506e757740 10 client.0 _ll_register_callbacks cb 0x559e0700d9e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:53:25.089 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:25.084+0000 7f506577e640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559e06d2b390 0x559e070feec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:53:25.089 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:25.084+0000 7f506577e640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559e06d2b390 0x559e070feec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:57326/0 (socket says 172.21.15.39:57326) 2024-09-21T03:53:25.089 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:25.084+0000 7f506577e640 1 -- 172.21.15.39:0/2980262749 learned_addr learned my addr 172.21.15.39:0/2980262749 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:53:25.089 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:25.084+0000 7f5066780640 1 --2- 172.21.15.39:0/2980262749 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x559e06d4c070 0x559e0710acd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:53:25.089 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:25.084+0000 7f5065f7f640 1 --2- 172.21.15.39:0/2980262749 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559e06d231b0 0x559e070fe980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:53:25.089 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:25.084+0000 7f506577e640 1 -- 172.21.15.39:0/2980262749 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559e06d231b0 msgr2=0x559e070fe980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:53:25.089 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:25.084+0000 7f506577e640 1 --2- 172.21.15.39:0/2980262749 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559e06d231b0 0x559e070fe980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:25.089 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:25.084+0000 7f506577e640 1 -- 172.21.15.39:0/2980262749 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x559e06d4c070 msgr2=0x559e0710acd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:53:25.089 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:25.084+0000 7f506577e640 1 --2- 172.21.15.39:0/2980262749 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x559e06d4c070 0x559e0710acd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:25.090 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:25.084+0000 7f506577e640 1 -- 172.21.15.39:0/2980262749 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559e07021c90 con 0x559e06d2b390 2024-09-21T03:53:25.090 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:25.084+0000 7f5065f7f640 1 --2- 172.21.15.39:0/2980262749 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559e06d231b0 0x559e070fe980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:53:25.090 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:25.085+0000 7f506577e640 1 --2- 172.21.15.39:0/2980262749 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559e06d2b390 0x559e070feec0 secure :-1 s=READY pgs=1060 cs=0 l=1 rev1=1 crypto rx=0x7f50500e73c0 tx=0x7f50500ea800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=85940b9a5ebd7d13 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:53:25.090 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:25.085+0000 7f50497fa640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-09-21T03:53:25.090 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:25.085+0000 7f50497fa640 1 -- 172.21.15.39:0/2980262749 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5050020070 con 0x559e06d2b390 2024-09-21T03:53:25.090 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:25.085+0000 7f50497fa640 1 -- 172.21.15.39:0/2980262749 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8528+0+0 (secure 0 0 0) 0x7f50500ac3f0 con 0x559e06d2b390 2024-09-21T03:53:25.090 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:25.085+0000 7f506e757740 1 -- 172.21.15.39:0/2980262749 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f50500acaa0 con 0x559e06d2b390 2024-09-21T03:53:25.090 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:25.085+0000 7f50497fa640 1 -- 172.21.15.39:0/2980262749 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f50500acd60 con 0x559e06d2b390 2024-09-21T03:53:25.090 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:25.085+0000 7f506e757740 10 client.17463 fetch_fsmap learned FSMap version 29 2024-09-21T03:53:25.090 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:25.085+0000 7f506e757740 1 -- 172.21.15.39:0/2980262749 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x559e0710d6c0 con 0x559e06d2b390 2024-09-21T03:53:25.090 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:25.085+0000 7f50497fa640 1 -- 172.21.15.39:0/2980262749 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f50500acaa0 con 0x559e06d2b390 2024-09-21T03:53:25.090 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:25.085+0000 7f50497fa640 1 -- 172.21.15.39:0/2980262749 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f5050094270 con 0x559e06d2b390 2024-09-21T03:53:25.090 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:25.086+0000 7f506e757740 10 client.17463 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T03:53:25.090 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:25.086+0000 7f506e757740 10 client.17463 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-21T03:53:25.091 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:25.086+0000 7f506e757740 20 client.17463 populate_metadata read hostname 'smithi039' 2024-09-21T03:53:25.091 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:25.086+0000 7f506e757740 1 --2- 172.21.15.39:0/2980262749 >> [v2:172.21.15.161:6836/3451544597,v1:172.21.15.161:6837/3451544597] conn(0x559e06d0a9f0 0x559e07110a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:53:25.091 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:25.086+0000 7f506e757740 4 client.17463 mds_command: new command op to 24646 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:53:25.091 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:25.086+0000 7f506e757740 1 -- 172.21.15.39:0/2980262749 --> [v2:172.21.15.161:6836/3451544597,v1:172.21.15.161:6837/3451544597] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559e0706bdf0 con 0x559e06d0a9f0 2024-09-21T03:53:25.092 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:25.086+0000 7f5065f7f640 1 --2- 172.21.15.39:0/2980262749 >> [v2:172.21.15.161:6836/3451544597,v1:172.21.15.161:6837/3451544597] conn(0x559e06d0a9f0 0x559e07110a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:53:25.092 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:25.087+0000 7f5065f7f640 1 --2- 172.21.15.39:0/2980262749 >> [v2:172.21.15.161:6836/3451544597,v1:172.21.15.161:6837/3451544597] conn(0x559e06d0a9f0 0x559e07110a20 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=22baa6b18d41aabb server_cookie=d1fe3f8d4e072f4b in_seq=0 out_seq=0 2024-09-21T03:53:25.092 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:25.087+0000 7f50497fa640 10 client.17463 ms_handle_connect on v2:172.21.15.161:6836/3451544597 2024-09-21T03:53:25.092 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:25.087+0000 7f50497fa640 1 -- 172.21.15.39:0/2980262749 <== mds.3 v2:172.21.15.161:6836/3451544597 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x559e0706bdf0 con 0x559e06d0a9f0 2024-09-21T03:53:25.092 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:25.087+0000 7f50497fa640 10 client.17463 handle_command_reply: tid=0 2024-09-21T03:53:25.092 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:25.087+0000 7f506e757740 1 client.17463 shutdown 2024-09-21T03:53:25.092 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:25.087+0000 7f506e757740 1 -- 172.21.15.39:0/2980262749 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559e06d2b390 msgr2=0x559e070feec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:53:25.092 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:25.087+0000 7f506e757740 1 --2- 172.21.15.39:0/2980262749 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559e06d2b390 0x559e070feec0 secure :-1 s=READY pgs=1060 cs=0 l=1 rev1=1 crypto rx=0x7f50500e73c0 tx=0x7f50500ea800 comp rx=0 tx=0).stop 2024-09-21T03:53:25.093 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:25.088+0000 7f506e757740 1 -- 172.21.15.39:0/2980262749 shutdown_connections 2024-09-21T03:53:25.093 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:25.088+0000 7f506e757740 1 --2- 172.21.15.39:0/2980262749 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x559e06d4c070 0x559e0710acd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:25.093 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:25.088+0000 7f506e757740 1 --2- 172.21.15.39:0/2980262749 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559e06d2b390 0x559e070feec0 unknown :-1 s=CLOSED pgs=1060 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:25.093 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:25.088+0000 7f506e757740 1 --2- 172.21.15.39:0/2980262749 >> [v2:172.21.15.161:6836/3451544597,v1:172.21.15.161:6837/3451544597] conn(0x559e06d0a9f0 0x559e07110a20 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:25.093 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:25.088+0000 7f506e757740 1 --2- 172.21.15.39:0/2980262749 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559e06d231b0 0x559e070fe980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:25.093 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:25.088+0000 7f50497fa640 0 client.17463 ms_handle_reset on v2:172.21.15.161:6836/3451544597 2024-09-21T03:53:25.093 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:25.088+0000 7f506e757740 1 -- 172.21.15.39:0/2980262749 >> 172.21.15.39:0/2980262749 conn(0x559e06d40460 msgr2=0x559e070f64a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:53:25.093 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:25.088+0000 7f506e757740 1 -- 172.21.15.39:0/2980262749 shutdown_connections 2024-09-21T03:53:25.093 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:25.088+0000 7f506e757740 1 -- 172.21.15.39:0/2980262749 wait complete. 2024-09-21T03:53:25.093 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:25.088+0000 7f506e757740 20 client.17463 trim_cache size 0 max 16384 2024-09-21T03:53:25.104 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:25.101+0000 7f506e757740 1 Processor -- start 2024-09-21T03:53:25.104 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:25.101+0000 7f506e757740 1 -- start start 2024-09-21T03:53:25.104 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:25.101+0000 7f506e757740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559e06d231b0 0x559e070f4a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:53:25.104 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:25.101+0000 7f506e757740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x559e06d2b390 0x559e070f0760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:53:25.104 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:25.101+0000 7f506e757740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559e06d4c070 0x559e070f0ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:53:25.105 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:25.101+0000 7f506e757740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f50601af850 con 0x559e06d231b0 2024-09-21T03:53:25.105 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:25.101+0000 7f506e757740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f50601b1390 con 0x559e06d2b390 2024-09-21T03:53:25.105 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:25.101+0000 7f506e757740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f50601aff20 con 0x559e06d4c070 2024-09-21T03:53:25.105 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:25.102+0000 7f5065f7f640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559e06d231b0 0x559e070f4a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:53:25.105 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:25.102+0000 7f5065f7f640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559e06d231b0 0x559e070f4a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:57330/0 (socket says 172.21.15.39:57330) 2024-09-21T03:53:25.105 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:25.102+0000 7f5065f7f640 1 -- 172.21.15.39:0/895896844 learned_addr learned my addr 172.21.15.39:0/895896844 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:53:25.105 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:25.102+0000 7f5066780640 1 --2- 172.21.15.39:0/895896844 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559e06d4c070 0x559e070f0ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:53:25.105 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:25.102+0000 7f506577e640 1 --2- 172.21.15.39:0/895896844 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x559e06d2b390 0x559e070f0760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:53:25.105 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:25.102+0000 7f5065f7f640 1 -- 172.21.15.39:0/895896844 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559e06d4c070 msgr2=0x559e070f0ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:53:25.105 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:25.102+0000 7f5065f7f640 1 --2- 172.21.15.39:0/895896844 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559e06d4c070 0x559e070f0ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:25.105 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:25.102+0000 7f5065f7f640 1 -- 172.21.15.39:0/895896844 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x559e06d2b390 msgr2=0x559e070f0760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:53:25.105 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:25.102+0000 7f5065f7f640 1 --2- 172.21.15.39:0/895896844 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x559e06d2b390 0x559e070f0760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:25.105 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:25.102+0000 7f5065f7f640 1 -- 172.21.15.39:0/895896844 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559e070f55b0 con 0x559e06d231b0 2024-09-21T03:53:25.105 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:25.102+0000 7f506577e640 1 --2- 172.21.15.39:0/895896844 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x559e06d2b390 0x559e070f0760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:53:25.106 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:25.102+0000 7f5065f7f640 1 --2- 172.21.15.39:0/895896844 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559e06d231b0 0x559e070f4a00 secure :-1 s=READY pgs=1061 cs=0 l=1 rev1=1 crypto rx=0x7f505c02a930 tx=0x7f505c005db0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=314db83e05bfb1cb server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:53:25.106 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:25.102+0000 7f504b7fe640 1 -- 172.21.15.39:0/895896844 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f505c035040 con 0x559e06d231b0 2024-09-21T03:53:25.106 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:25.102+0000 7f504b7fe640 1 -- 172.21.15.39:0/895896844 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f505c0054f0 con 0x559e06d231b0 2024-09-21T03:53:25.106 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:25.102+0000 7f504b7fe640 1 -- 172.21.15.39:0/895896844 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f505c0035f0 con 0x559e06d231b0 2024-09-21T03:53:25.108 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:25.103+0000 7f506e757740 1 -- 172.21.15.39:0/895896844 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559e06d231b0 msgr2=0x559e070f4a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:53:25.108 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:25.103+0000 7f506e757740 1 --2- 172.21.15.39:0/895896844 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559e06d231b0 0x559e070f4a00 secure :-1 s=READY pgs=1061 cs=0 l=1 rev1=1 crypto rx=0x7f505c02a930 tx=0x7f505c005db0 comp rx=0 tx=0).stop 2024-09-21T03:53:25.108 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:25.103+0000 7f506e757740 1 -- 172.21.15.39:0/895896844 shutdown_connections 2024-09-21T03:53:25.108 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:25.103+0000 7f506e757740 1 --2- 172.21.15.39:0/895896844 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559e06d4c070 0x559e070f0ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:25.109 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:25.103+0000 7f506e757740 1 --2- 172.21.15.39:0/895896844 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x559e06d2b390 0x559e070f0760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:25.109 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:25.103+0000 7f506e757740 1 --2- 172.21.15.39:0/895896844 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559e06d231b0 0x559e070f4a00 unknown :-1 s=CLOSED pgs=1061 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:25.109 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:25.103+0000 7f506e757740 1 -- 172.21.15.39:0/895896844 >> 172.21.15.39:0/895896844 conn(0x559e06d40460 msgr2=0x559e070f42f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:53:25.109 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:25.103+0000 7f506e757740 1 -- 172.21.15.39:0/895896844 shutdown_connections 2024-09-21T03:53:25.109 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:25.103+0000 7f506e757740 1 -- 172.21.15.39:0/895896844 wait complete. 2024-09-21T03:53:25.109 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:25.104+0000 7f506e757740 1 -- start start 2024-09-21T03:53:25.109 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:25.104+0000 7f506e757740 1 Processor -- start 2024-09-21T03:53:25.109 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:25.104+0000 7f506e757740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559e06d231b0 0x559e07111de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:53:25.109 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:25.104+0000 7f506e757740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x559e06d2b390 0x559e07112320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:53:25.109 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:25.104+0000 7f506e757740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559e06d4c070 0x559e0717a1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:53:25.109 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:25.104+0000 7f506e757740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f50601b0cc0 con 0x559e06d4c070 2024-09-21T03:53:25.109 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:25.104+0000 7f506e757740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f50601b05f0 con 0x559e06d2b390 2024-09-21T03:53:25.109 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:25.104+0000 7f506e757740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f50601ad830 con 0x559e06d231b0 2024-09-21T03:53:25.109 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:25.104+0000 7f506e757740 10 client.0 _ll_register_callbacks cb 0x7f5058009a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:53:25.110 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:25.105+0000 7f5066780640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559e06d4c070 0x559e0717a1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:53:25.110 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:25.105+0000 7f5066780640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559e06d4c070 0x559e0717a1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:57338/0 (socket says 172.21.15.39:57338) 2024-09-21T03:53:25.110 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:25.105+0000 7f5066780640 1 -- 172.21.15.39:0/1950087386 learned_addr learned my addr 172.21.15.39:0/1950087386 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:53:25.110 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:25.105+0000 7f5065f7f640 1 --2- 172.21.15.39:0/1950087386 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559e06d231b0 0x559e07111de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:53:25.110 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:25.105+0000 7f506577e640 1 --2- 172.21.15.39:0/1950087386 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x559e06d2b390 0x559e07112320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:53:25.110 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:25.105+0000 7f5066780640 1 -- 172.21.15.39:0/1950087386 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559e06d231b0 msgr2=0x559e07111de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:53:25.110 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:25.105+0000 7f5066780640 1 --2- 172.21.15.39:0/1950087386 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559e06d231b0 0x559e07111de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:25.110 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:25.105+0000 7f5066780640 1 -- 172.21.15.39:0/1950087386 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x559e06d2b390 msgr2=0x559e07112320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:53:25.110 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:25.105+0000 7f5066780640 1 --2- 172.21.15.39:0/1950087386 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x559e06d2b390 0x559e07112320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:25.110 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:25.105+0000 7f5066780640 1 -- 172.21.15.39:0/1950087386 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559e0717a6e0 con 0x559e06d4c070 2024-09-21T03:53:25.110 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:25.105+0000 7f5066780640 1 --2- 172.21.15.39:0/1950087386 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559e06d4c070 0x559e0717a1a0 secure :-1 s=READY pgs=1062 cs=0 l=1 rev1=1 crypto rx=0x7f505800ebc0 tx=0x7f5058004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=182b40989e8f1006 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:53:25.110 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:25.105+0000 7f50497fa640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-09-21T03:53:25.110 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:25.105+0000 7f50497fa640 1 -- 172.21.15.39:0/1950087386 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5058021c40 con 0x559e06d4c070 2024-09-21T03:53:25.110 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:25.105+0000 7f50497fa640 1 -- 172.21.15.39:0/1950087386 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8528+0+0 (secure 0 0 0) 0x7f5058021de0 con 0x559e06d4c070 2024-09-21T03:53:25.110 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:25.105+0000 7f506e757740 1 -- 172.21.15.39:0/1950087386 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f505c0054f0 con 0x559e06d4c070 2024-09-21T03:53:25.111 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:25.106+0000 7f50497fa640 1 -- 172.21.15.39:0/1950087386 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f505802b420 con 0x559e06d4c070 2024-09-21T03:53:25.111 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:25.106+0000 7f506e757740 10 client.17469 fetch_fsmap learned FSMap version 29 2024-09-21T03:53:25.111 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:25.106+0000 7f506e757740 1 -- 172.21.15.39:0/1950087386 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x559e0717b780 con 0x559e06d4c070 2024-09-21T03:53:25.111 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:25.106+0000 7f50497fa640 1 -- 172.21.15.39:0/1950087386 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f505c0054f0 con 0x559e06d4c070 2024-09-21T03:53:25.111 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:25.106+0000 7f50497fa640 1 -- 172.21.15.39:0/1950087386 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f505801e040 con 0x559e06d4c070 2024-09-21T03:53:25.111 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:25.106+0000 7f506e757740 10 client.17469 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T03:53:25.111 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:25.106+0000 7f506e757740 10 client.17469 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-21T03:53:25.111 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:25.106+0000 7f506e757740 20 client.17469 populate_metadata read hostname 'smithi039' 2024-09-21T03:53:25.111 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:25.106+0000 7f506e757740 1 --2- 172.21.15.39:0/1950087386 >> [v2:172.21.15.161:6836/3451544597,v1:172.21.15.161:6837/3451544597] conn(0x559e06cdfda0 0x559e0717dfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:53:25.111 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:25.106+0000 7f506e757740 4 client.17469 mds_command: new command op to 24646 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T03:53:25.111 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:25.106+0000 7f506e757740 1 -- 172.21.15.39:0/1950087386 --> [v2:172.21.15.161:6836/3451544597,v1:172.21.15.161:6837/3451544597] -- command(tid 0: {"prefix": "damage ls"}) -- 0x559e070678a0 con 0x559e06cdfda0 2024-09-21T03:53:25.112 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:25.107+0000 7f5065f7f640 1 --2- 172.21.15.39:0/1950087386 >> [v2:172.21.15.161:6836/3451544597,v1:172.21.15.161:6837/3451544597] conn(0x559e06cdfda0 0x559e0717dfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:53:25.113 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:25.107+0000 7f50497fa640 10 client.17469 ms_handle_connect on v2:172.21.15.161:6836/3451544597 2024-09-21T03:53:25.113 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:25.107+0000 7f5065f7f640 1 --2- 172.21.15.39:0/1950087386 >> [v2:172.21.15.161:6836/3451544597,v1:172.21.15.161:6837/3451544597] conn(0x559e06cdfda0 0x559e0717dfc0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=f3e33b6847197203 server_cookie=d39fd5e776acb4fb in_seq=0 out_seq=0 2024-09-21T03:53:25.113 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:25.107+0000 7f50497fa640 1 -- 172.21.15.39:0/1950087386 <== mds.3 v2:172.21.15.161:6836/3451544597 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x559e070678a0 con 0x559e06cdfda0 2024-09-21T03:53:25.113 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:25.107+0000 7f50497fa640 10 client.17469 handle_command_reply: tid=0 2024-09-21T03:53:25.113 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:25.107+0000 7f506e757740 1 client.17469 shutdown 2024-09-21T03:53:25.114 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:25.108+0000 7f506e757740 1 -- 172.21.15.39:0/1950087386 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559e06d4c070 msgr2=0x559e0717a1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:53:25.114 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:25.108+0000 7f506e757740 1 --2- 172.21.15.39:0/1950087386 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559e06d4c070 0x559e0717a1a0 secure :-1 s=READY pgs=1062 cs=0 l=1 rev1=1 crypto rx=0x7f505800ebc0 tx=0x7f5058004a40 comp rx=0 tx=0).stop 2024-09-21T03:53:25.114 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:25.108+0000 7f506e757740 1 -- 172.21.15.39:0/1950087386 shutdown_connections 2024-09-21T03:53:25.114 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:25.108+0000 7f506e757740 1 --2- 172.21.15.39:0/1950087386 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559e06d4c070 0x559e0717a1a0 unknown :-1 s=CLOSED pgs=1062 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:25.114 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:25.108+0000 7f506e757740 1 --2- 172.21.15.39:0/1950087386 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x559e06d2b390 0x559e07112320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:25.114 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:25.108+0000 7f506e757740 1 --2- 172.21.15.39:0/1950087386 >> [v2:172.21.15.161:6836/3451544597,v1:172.21.15.161:6837/3451544597] conn(0x559e06cdfda0 0x559e0717dfc0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:25.114 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:25.108+0000 7f506e757740 1 --2- 172.21.15.39:0/1950087386 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559e06d231b0 0x559e07111de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:25.114 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:25.108+0000 7f506e757740 1 -- 172.21.15.39:0/1950087386 >> 172.21.15.39:0/1950087386 conn(0x559e06d40460 msgr2=0x559e070f6d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:53:25.114 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:25.108+0000 7f50497fa640 0 client.17469 ms_handle_reset on v2:172.21.15.161:6836/3451544597 2024-09-21T03:53:25.114 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:25.108+0000 7f506e757740 1 -- 172.21.15.39:0/1950087386 shutdown_connections 2024-09-21T03:53:25.114 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:25.108+0000 7f506e757740 1 -- 172.21.15.39:0/1950087386 wait complete. 2024-09-21T03:53:25.114 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:25.108+0000 7f506e757740 20 client.17469 trim_cache size 0 max 16384 2024-09-21T03:53:25.115 INFO:teuthology.orchestra.run.smithi039.stdout:[] 2024-09-21T03:53:25.115 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:25.108+0000 7f50681e3640 1 -- 172.21.15.39:0/2338738562 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f504c0b6680 msgr2=0x7f504c0b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:53:25.115 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:25.108+0000 7f50681e3640 1 --2- 172.21.15.39:0/2338738562 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f504c0b6680 0x7f504c0b8b90 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f5058005fd0 tx=0x7f5058004ab0 comp rx=0 tx=0).stop 2024-09-21T03:53:25.116 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:25.108+0000 7f50681e3640 1 -- 172.21.15.39:0/2338738562 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5060105380 msgr2=0x7f506019ab40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:53:25.116 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:25.108+0000 7f50681e3640 1 --2- 172.21.15.39:0/2338738562 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5060105380 0x7f506019ab40 secure :-1 s=READY pgs=1058 cs=0 l=1 rev1=1 crypto rx=0x7f505000d750 tx=0x7f505000dc10 comp rx=0 tx=0).stop 2024-09-21T03:53:25.116 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:25.108+0000 7f50681e3640 1 -- 172.21.15.39:0/2338738562 shutdown_connections 2024-09-21T03:53:25.116 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:25.108+0000 7f50681e3640 1 --2- 172.21.15.39:0/2338738562 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f504c0b6680 0x7f504c0b8b90 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:25.116 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:25.108+0000 7f50681e3640 1 --2- 172.21.15.39:0/2338738562 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f506010de90 0x7f506019fde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:25.116 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:25.108+0000 7f50681e3640 1 --2- 172.21.15.39:0/2338738562 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5060105380 0x7f506019ab40 unknown :-1 s=CLOSED pgs=1058 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:25.116 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:25.108+0000 7f50681e3640 1 --2- 172.21.15.39:0/2338738562 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5060104a30 0x7f506019a5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:25.116 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:25.108+0000 7f50681e3640 1 -- 172.21.15.39:0/2338738562 >> 172.21.15.39:0/2338738562 conn(0x7f5060079770 msgr2=0x7f5060100880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:53:25.116 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:25.109+0000 7f50681e3640 1 -- 172.21.15.39:0/2338738562 shutdown_connections 2024-09-21T03:53:25.116 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:25.109+0000 7f50681e3640 1 -- 172.21.15.39:0/2338738562 wait complete. 2024-09-21T03:53:25.218 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:53:24 smithi161 ceph-mon[34533]: pgmap v548: 97 pgs: 97 active+clean; 10 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 102 B/s wr, 10 op/s 2024-09-21T03:53:25.224 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:53:24 smithi150 ceph-mon[34289]: pgmap v548: 97 pgs: 97 active+clean; 10 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 102 B/s wr, 10 op/s 2024-09-21T03:53:25.931 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 51116d5c-77ca-11ef-baf5-efdc52797490 -- ceph tell mds.1:2 damage ls 2024-09-21T03:53:26.193 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/51116d5c-77ca-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:53:26.650 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:26.641+0000 7fbbee37e640 1 -- 172.21.15.39:0/2868233882 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbbe8108d10 msgr2=0x7fbbe810f540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:53:26.651 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:26.641+0000 7fbbee37e640 1 --2- 172.21.15.39:0/2868233882 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbbe8108d10 0x7fbbe810f540 secure :-1 s=READY pgs=1063 cs=0 l=1 rev1=1 crypto rx=0x7fbbe0009ca0 tx=0x7fbbe002a8c0 comp rx=0 tx=0).stop 2024-09-21T03:53:26.651 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:26.641+0000 7fbbee37e640 1 -- 172.21.15.39:0/2868233882 shutdown_connections 2024-09-21T03:53:26.651 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:26.641+0000 7fbbee37e640 1 --2- 172.21.15.39:0/2868233882 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbbe8108d10 0x7fbbe810f540 unknown :-1 s=CLOSED pgs=1063 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:26.651 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:26.641+0000 7fbbee37e640 1 --2- 172.21.15.39:0/2868233882 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbbe81083c0 0x7fbbe81087d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:26.651 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:26.641+0000 7fbbee37e640 1 --2- 172.21.15.39:0/2868233882 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbbe8102a80 0x7fbbe8102e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:26.651 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:26.641+0000 7fbbee37e640 1 -- 172.21.15.39:0/2868233882 >> 172.21.15.39:0/2868233882 conn(0x7fbbe8079740 msgr2=0x7fbbe8079b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:53:26.651 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:26.641+0000 7fbbee37e640 1 -- 172.21.15.39:0/2868233882 shutdown_connections 2024-09-21T03:53:26.651 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:26.641+0000 7fbbee37e640 1 -- 172.21.15.39:0/2868233882 wait complete. 2024-09-21T03:53:26.651 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:26.642+0000 7fbbee37e640 1 Processor -- start 2024-09-21T03:53:26.651 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:26.642+0000 7fbbee37e640 1 -- start start 2024-09-21T03:53:26.651 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:26.642+0000 7fbbee37e640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbbe8102a80 0x7fbbe819b290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:53:26.651 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:26.642+0000 7fbbee37e640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbbe81083c0 0x7fbbe819b7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:53:26.652 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:26.642+0000 7fbbee37e640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbbe8108d10 0x7fbbe819bd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:53:26.652 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:26.642+0000 7fbbee37e640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fbbe8112980 con 0x7fbbe8108d10 2024-09-21T03:53:26.652 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:26.642+0000 7fbbee37e640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fbbe8112800 con 0x7fbbe81083c0 2024-09-21T03:53:26.652 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:26.642+0000 7fbbee37e640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fbbe8112b00 con 0x7fbbe8102a80 2024-09-21T03:53:26.652 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:26.642+0000 7fbbec91b640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbbe8108d10 0x7fbbe819bd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:53:26.652 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:26.642+0000 7fbbec91b640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbbe8108d10 0x7fbbe819bd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:57348/0 (socket says 172.21.15.39:57348) 2024-09-21T03:53:26.652 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:26.642+0000 7fbbec91b640 1 -- 172.21.15.39:0/2636384879 learned_addr learned my addr 172.21.15.39:0/2636384879 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:53:26.652 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:26.642+0000 7fbbe77fe640 1 --2- 172.21.15.39:0/2636384879 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbbe81083c0 0x7fbbe819b7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:53:26.652 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:26.642+0000 7fbbe7fff640 1 --2- 172.21.15.39:0/2636384879 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbbe8102a80 0x7fbbe819b290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:53:26.652 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:26.643+0000 7fbbec91b640 1 -- 172.21.15.39:0/2636384879 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbbe8102a80 msgr2=0x7fbbe819b290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:53:26.652 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:26.643+0000 7fbbec91b640 1 --2- 172.21.15.39:0/2636384879 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbbe8102a80 0x7fbbe819b290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:26.652 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:26.643+0000 7fbbec91b640 1 -- 172.21.15.39:0/2636384879 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbbe81083c0 msgr2=0x7fbbe819b7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:53:26.652 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:26.643+0000 7fbbec91b640 1 --2- 172.21.15.39:0/2636384879 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbbe81083c0 0x7fbbe819b7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:26.652 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:26.643+0000 7fbbec91b640 1 -- 172.21.15.39:0/2636384879 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbbe81a6520 con 0x7fbbe8108d10 2024-09-21T03:53:26.653 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:26.643+0000 7fbbec91b640 1 --2- 172.21.15.39:0/2636384879 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbbe8108d10 0x7fbbe819bd10 secure :-1 s=READY pgs=1064 cs=0 l=1 rev1=1 crypto rx=0x7fbbe0005cc0 tx=0x7fbbe002f380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:53:26.653 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:26.643+0000 7fbbe57fa640 1 -- 172.21.15.39:0/2636384879 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbe0003bf0 con 0x7fbbe8108d10 2024-09-21T03:53:26.653 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:26.643+0000 7fbbe57fa640 1 -- 172.21.15.39:0/2636384879 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbbe0003d90 con 0x7fbbe8108d10 2024-09-21T03:53:26.653 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:26.643+0000 7fbbee37e640 1 -- 172.21.15.39:0/2636384879 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbbe81a67b0 con 0x7fbbe8108d10 2024-09-21T03:53:26.653 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:26.643+0000 7fbbe57fa640 1 -- 172.21.15.39:0/2636384879 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbe002fbc0 con 0x7fbbe8108d10 2024-09-21T03:53:26.653 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:26.643+0000 7fbbee37e640 1 -- 172.21.15.39:0/2636384879 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbbe81a6cf0 con 0x7fbbe8108d10 2024-09-21T03:53:26.653 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:26.646+0000 7fbbf48f2740 1 Processor -- start 2024-09-21T03:53:26.653 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:26.646+0000 7fbbe57fa640 1 -- 172.21.15.39:0/2636384879 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 27) ==== 154643+0+0 (secure 0 0 0) 0x7fbbe002fd60 con 0x7fbbe8108d10 2024-09-21T03:53:26.653 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:26.646+0000 7fbbf48f2740 1 -- start start 2024-09-21T03:53:26.653 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:26.646+0000 7fbbf48f2740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ae51500070 0x55ae518a76a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:53:26.653 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:26.646+0000 7fbbf48f2740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ae514d71b0 0x55ae518a7be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:53:26.653 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:26.646+0000 7fbbf48f2740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ae514df390 0x55ae518a8120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:53:26.653 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:26.646+0000 7fbbf48f2740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x55ae5181dea0 con 0x55ae514d71b0 2024-09-21T03:53:26.653 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:26.646+0000 7fbbf48f2740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x55ae5180d2d0 con 0x55ae514df390 2024-09-21T03:53:26.653 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:26.646+0000 7fbbf48f2740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55ae518110e0 con 0x55ae51500070 2024-09-21T03:53:26.654 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:26.647+0000 7fbbe7fff640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ae514d71b0 0x55ae518a7be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:53:26.654 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:26.647+0000 7fbbe7fff640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ae514d71b0 0x55ae518a7be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:57358/0 (socket says 172.21.15.39:57358) 2024-09-21T03:53:26.654 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:26.647+0000 7fbbe7fff640 1 -- 172.21.15.39:0/3363066163 learned_addr learned my addr 172.21.15.39:0/3363066163 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:53:26.654 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:26.647+0000 7fbbe77fe640 1 --2- 172.21.15.39:0/3363066163 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ae514df390 0x55ae518a8120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:53:26.654 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:26.647+0000 7fbbe77fe640 1 --2- 172.21.15.39:0/3363066163 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ae51500070 0x55ae518a76a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:53:26.654 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:26.647+0000 7fbbe57fa640 1 --2- 172.21.15.39:0/2636384879 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7fbbd00b66b0 0x7fbbd00b8bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:53:26.654 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:26.647+0000 7fbbe7fff640 1 -- 172.21.15.39:0/3363066163 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ae51500070 msgr2=0x55ae518a76a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:53:26.654 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:26.647+0000 7fbbe7fff640 1 --2- 172.21.15.39:0/3363066163 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ae51500070 0x55ae518a76a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:26.654 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:26.647+0000 7fbbe7fff640 1 -- 172.21.15.39:0/3363066163 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ae514df390 msgr2=0x55ae518a8120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:53:26.654 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:26.647+0000 7fbbe7fff640 1 --2- 172.21.15.39:0/3363066163 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ae514df390 0x55ae518a8120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:26.654 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:26.647+0000 7fbbe7fff640 1 -- 172.21.15.39:0/3363066163 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ae5156b650 con 0x55ae514d71b0 2024-09-21T03:53:26.654 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:26.647+0000 7fbbec91b640 1 --2- 172.21.15.39:0/2636384879 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7fbbd00b66b0 0x7fbbd00b8bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:53:26.654 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:26.647+0000 7fbbe7fff640 1 --2- 172.21.15.39:0/3363066163 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ae514d71b0 0x55ae518a7be0 secure :-1 s=READY pgs=1065 cs=0 l=1 rev1=1 crypto rx=0x7fbbd800bd10 tx=0x7fbbd80107a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a6b91a4669f66416 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:53:26.654 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:26.647+0000 7fbbe57fa640 1 -- 172.21.15.39:0/2636384879 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8528+0+0 (secure 0 0 0) 0x7fbbe0108970 con 0x7fbbe8108d10 2024-09-21T03:53:26.658 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:26.648+0000 7fbbce7fc640 1 -- 172.21.15.39:0/3363066163 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbd8005220 con 0x55ae514d71b0 2024-09-21T03:53:26.658 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:26.648+0000 7fbbce7fc640 1 -- 172.21.15.39:0/3363066163 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbbd8004c50 con 0x55ae514d71b0 2024-09-21T03:53:26.658 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:26.648+0000 7fbbce7fc640 1 -- 172.21.15.39:0/3363066163 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbd8011840 con 0x55ae514d71b0 2024-09-21T03:53:26.658 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:26.649+0000 7fbbf48f2740 1 -- 172.21.15.39:0/3363066163 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ae514d71b0 msgr2=0x55ae518a7be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:53:26.658 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:26.649+0000 7fbbec91b640 1 --2- 172.21.15.39:0/2636384879 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7fbbd00b66b0 0x7fbbd00b8bc0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7fbbe009a450 tx=0x7fbbe009bbb0 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:53:26.658 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:26.649+0000 7fbbf48f2740 1 --2- 172.21.15.39:0/3363066163 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ae514d71b0 0x55ae518a7be0 secure :-1 s=READY pgs=1065 cs=0 l=1 rev1=1 crypto rx=0x7fbbd800bd10 tx=0x7fbbd80107a0 comp rx=0 tx=0).stop 2024-09-21T03:53:26.658 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:26.649+0000 7fbbf48f2740 1 -- 172.21.15.39:0/3363066163 shutdown_connections 2024-09-21T03:53:26.658 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:26.649+0000 7fbbf48f2740 1 --2- 172.21.15.39:0/3363066163 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ae514df390 0x55ae518a8120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:26.658 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:26.649+0000 7fbbf48f2740 1 --2- 172.21.15.39:0/3363066163 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ae514d71b0 0x55ae518a7be0 unknown :-1 s=CLOSED pgs=1065 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:26.658 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:26.649+0000 7fbbf48f2740 1 --2- 172.21.15.39:0/3363066163 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ae51500070 0x55ae518a76a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:26.658 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:26.649+0000 7fbbf48f2740 1 -- 172.21.15.39:0/3363066163 >> 172.21.15.39:0/3363066163 conn(0x55ae514f4460 msgr2=0x55ae518a5590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:53:26.659 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:26.649+0000 7fbbf48f2740 1 -- 172.21.15.39:0/3363066163 shutdown_connections 2024-09-21T03:53:26.659 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:26.649+0000 7fbbf48f2740 1 -- 172.21.15.39:0/3363066163 wait complete. 2024-09-21T03:53:26.659 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:26.650+0000 7fbbf48f2740 1 -- start start 2024-09-21T03:53:26.659 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:26.650+0000 7fbbf48f2740 1 Processor -- start 2024-09-21T03:53:26.659 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:26.650+0000 7fbbf48f2740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ae514d71b0 0x55ae518b2d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:53:26.659 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:26.650+0000 7fbbf48f2740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ae514df390 0x55ae518b32b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:53:26.659 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:26.650+0000 7fbbf48f2740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ae51500070 0x55ae518bf170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:53:26.659 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:26.650+0000 7fbbf48f2740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x55ae5166e700 con 0x55ae514d71b0 2024-09-21T03:53:26.659 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:26.650+0000 7fbbf48f2740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x55ae517a8160 con 0x55ae51500070 2024-09-21T03:53:26.659 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:26.650+0000 7fbbf48f2740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55ae518b5e80 con 0x55ae514df390 2024-09-21T03:53:26.659 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:26.650+0000 7fbbf48f2740 10 client.0 _ll_register_callbacks cb 0x55ae517c19e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:53:26.659 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:26.651+0000 7fbbe77fe640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ae514d71b0 0x55ae518b2d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:53:26.659 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:26.651+0000 7fbbe77fe640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ae514d71b0 0x55ae518b2d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:57368/0 (socket says 172.21.15.39:57368) 2024-09-21T03:53:26.659 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:26.651+0000 7fbbe77fe640 1 -- 172.21.15.39:0/837546903 learned_addr learned my addr 172.21.15.39:0/837546903 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:53:26.660 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:26.651+0000 7fbbe7fff640 1 --2- 172.21.15.39:0/837546903 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ae514df390 0x55ae518b32b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:53:26.660 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:26.651+0000 7fbbe77fe640 1 -- 172.21.15.39:0/837546903 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ae514df390 msgr2=0x55ae518b32b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:53:26.660 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:26.651+0000 7fbbe77fe640 1 --2- 172.21.15.39:0/837546903 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ae514df390 0x55ae518b32b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:26.660 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:26.651+0000 7fbbe77fe640 1 -- 172.21.15.39:0/837546903 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ae51500070 msgr2=0x55ae518bf170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:53:26.660 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:26.651+0000 7fbbe77fe640 1 --2- 172.21.15.39:0/837546903 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ae51500070 0x55ae518bf170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:26.660 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:26.651+0000 7fbbe77fe640 1 -- 172.21.15.39:0/837546903 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ae517d5c90 con 0x55ae514d71b0 2024-09-21T03:53:26.660 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:26.651+0000 7fbbe77fe640 1 --2- 172.21.15.39:0/837546903 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ae514d71b0 0x55ae518b2d70 secure :-1 s=READY pgs=1066 cs=0 l=1 rev1=1 crypto rx=0x7fbbdc0124a0 tx=0x7fbbdc012960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce484ddaa0c0f67 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:53:26.660 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:26.651+0000 7fbbc77fe640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-09-21T03:53:26.660 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:26.651+0000 7fbbc77fe640 1 -- 172.21.15.39:0/837546903 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbdc004230 con 0x55ae514d71b0 2024-09-21T03:53:26.660 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:26.651+0000 7fbbc77fe640 1 -- 172.21.15.39:0/837546903 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8528+0+0 (secure 0 0 0) 0x7fbbdc0043d0 con 0x55ae514d71b0 2024-09-21T03:53:26.660 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:26.651+0000 7fbbf48f2740 1 -- 172.21.15.39:0/837546903 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbbd8004c50 con 0x55ae514d71b0 2024-09-21T03:53:26.660 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:26.651+0000 7fbbc77fe640 1 -- 172.21.15.39:0/837546903 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fbbdc0106a0 con 0x55ae514d71b0 2024-09-21T03:53:26.660 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:26.651+0000 7fbbf48f2740 10 client.17481 fetch_fsmap learned FSMap version 29 2024-09-21T03:53:26.660 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:26.651+0000 7fbbf48f2740 1 -- 172.21.15.39:0/837546903 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55ae518c1c00 con 0x55ae514d71b0 2024-09-21T03:53:26.660 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:26.652+0000 7fbbc77fe640 1 -- 172.21.15.39:0/837546903 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbbd8004c50 con 0x55ae514d71b0 2024-09-21T03:53:26.661 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:26.652+0000 7fbbc77fe640 1 -- 172.21.15.39:0/837546903 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fbbdc01b070 con 0x55ae514d71b0 2024-09-21T03:53:26.661 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:26.652+0000 7fbbf48f2740 10 client.17481 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T03:53:26.661 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:26.652+0000 7fbbf48f2740 10 client.17481 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-21T03:53:26.661 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:26.652+0000 7fbbf48f2740 20 client.17481 populate_metadata read hostname 'smithi039' 2024-09-21T03:53:26.661 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:26.652+0000 7fbbf48f2740 1 --2- 172.21.15.39:0/837546903 >> [v2:172.21.15.150:6836/4017043144,v1:172.21.15.150:6837/4017043144] conn(0x55ae514be9f0 0x55ae518c5110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:53:26.661 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:26.652+0000 7fbbf48f2740 4 client.17481 mds_command: new command op to 24659 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:53:26.661 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:26.652+0000 7fbbf48f2740 1 -- 172.21.15.39:0/837546903 --> [v2:172.21.15.150:6836/4017043144,v1:172.21.15.150:6837/4017043144] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ae5181fdf0 con 0x55ae514be9f0 2024-09-21T03:53:26.661 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:26.652+0000 7fbbe7fff640 1 --2- 172.21.15.39:0/837546903 >> [v2:172.21.15.150:6836/4017043144,v1:172.21.15.150:6837/4017043144] conn(0x55ae514be9f0 0x55ae518c5110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:53:26.661 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:26.653+0000 7fbbe7fff640 1 --2- 172.21.15.39:0/837546903 >> [v2:172.21.15.150:6836/4017043144,v1:172.21.15.150:6837/4017043144] conn(0x55ae514be9f0 0x55ae518c5110 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=c9406693211e3e6a server_cookie=dc68d3f2dac60fd1 in_seq=0 out_seq=0 2024-09-21T03:53:26.661 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:26.653+0000 7fbbc77fe640 10 client.17481 ms_handle_connect on v2:172.21.15.150:6836/4017043144 2024-09-21T03:53:26.661 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:26.654+0000 7fbbc77fe640 1 -- 172.21.15.39:0/837546903 <== mds.2 v2:172.21.15.150:6836/4017043144 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55ae5181fdf0 con 0x55ae514be9f0 2024-09-21T03:53:26.661 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:26.654+0000 7fbbc77fe640 10 client.17481 handle_command_reply: tid=0 2024-09-21T03:53:26.661 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:26.654+0000 7fbbf48f2740 1 client.17481 shutdown 2024-09-21T03:53:26.661 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:26.654+0000 7fbbf48f2740 1 -- 172.21.15.39:0/837546903 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ae514d71b0 msgr2=0x55ae518b2d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:53:26.661 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:26.654+0000 7fbbf48f2740 1 --2- 172.21.15.39:0/837546903 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ae514d71b0 0x55ae518b2d70 secure :-1 s=READY pgs=1066 cs=0 l=1 rev1=1 crypto rx=0x7fbbdc0124a0 tx=0x7fbbdc012960 comp rx=0 tx=0).stop 2024-09-21T03:53:26.662 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:26.654+0000 7fbbf48f2740 1 -- 172.21.15.39:0/837546903 shutdown_connections 2024-09-21T03:53:26.662 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:26.654+0000 7fbbf48f2740 1 --2- 172.21.15.39:0/837546903 >> [v2:172.21.15.150:6836/4017043144,v1:172.21.15.150:6837/4017043144] conn(0x55ae514be9f0 0x55ae518c5110 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:26.662 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:26.654+0000 7fbbf48f2740 1 --2- 172.21.15.39:0/837546903 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ae51500070 0x55ae518bf170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:26.662 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:26.654+0000 7fbbf48f2740 1 --2- 172.21.15.39:0/837546903 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ae514df390 0x55ae518b32b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:26.662 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:26.654+0000 7fbbf48f2740 1 --2- 172.21.15.39:0/837546903 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ae514d71b0 0x55ae518b2d70 unknown :-1 s=CLOSED pgs=1066 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:26.662 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:26.654+0000 7fbbf48f2740 1 -- 172.21.15.39:0/837546903 >> 172.21.15.39:0/837546903 conn(0x55ae514f4460 msgr2=0x55ae518aa830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:53:26.662 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:26.654+0000 7fbbc77fe640 0 client.17481 ms_handle_reset on v2:172.21.15.150:6836/4017043144 2024-09-21T03:53:26.662 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:26.654+0000 7fbbf48f2740 1 -- 172.21.15.39:0/837546903 shutdown_connections 2024-09-21T03:53:26.662 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:26.654+0000 7fbbf48f2740 1 -- 172.21.15.39:0/837546903 wait complete. 2024-09-21T03:53:26.662 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:26.654+0000 7fbbf48f2740 20 client.17481 trim_cache size 0 max 16384 2024-09-21T03:53:26.671 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:26.667+0000 7fbbf48f2740 1 Processor -- start 2024-09-21T03:53:26.671 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:26.667+0000 7fbbf48f2740 1 -- start start 2024-09-21T03:53:26.671 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:26.668+0000 7fbbf48f2740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ae514d71b0 0x55ae518b49a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:53:26.671 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:26.668+0000 7fbbf48f2740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ae514df390 0x55ae518a4bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:53:26.671 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:26.668+0000 7fbbf48f2740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ae51500070 0x55ae518a5100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:53:26.671 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:26.668+0000 7fbbf48f2740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fbbe81b1a20 con 0x55ae51500070 2024-09-21T03:53:26.672 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:26.668+0000 7fbbf48f2740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fbbe81b3560 con 0x55ae514df390 2024-09-21T03:53:26.672 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:26.668+0000 7fbbf48f2740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fbbe81b20f0 con 0x55ae514d71b0 2024-09-21T03:53:26.672 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:26.668+0000 7fbbe77fe640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ae51500070 0x55ae518a5100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:53:26.672 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:26.668+0000 7fbbe77fe640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ae51500070 0x55ae518a5100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:57374/0 (socket says 172.21.15.39:57374) 2024-09-21T03:53:26.672 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:26.668+0000 7fbbe77fe640 1 -- 172.21.15.39:0/710814385 learned_addr learned my addr 172.21.15.39:0/710814385 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:53:26.672 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:26.668+0000 7fbbe7fff640 1 --2- 172.21.15.39:0/710814385 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ae514df390 0x55ae518a4bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:53:26.672 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:26.668+0000 7fbbe77fe640 1 --2- 172.21.15.39:0/710814385 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ae514d71b0 0x55ae518b49a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:53:26.672 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:26.668+0000 7fbbe77fe640 1 -- 172.21.15.39:0/710814385 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ae514d71b0 msgr2=0x55ae518b49a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:53:26.672 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:26.668+0000 7fbbe77fe640 1 --2- 172.21.15.39:0/710814385 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ae514d71b0 0x55ae518b49a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:26.672 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:26.668+0000 7fbbe77fe640 1 -- 172.21.15.39:0/710814385 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ae514df390 msgr2=0x55ae518a4bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:53:26.672 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:26.668+0000 7fbbe77fe640 1 --2- 172.21.15.39:0/710814385 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ae514df390 0x55ae518a4bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:26.672 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:26.668+0000 7fbbe77fe640 1 -- 172.21.15.39:0/710814385 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ae518b1270 con 0x55ae51500070 2024-09-21T03:53:26.672 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:26.668+0000 7fbbe7fff640 1 --2- 172.21.15.39:0/710814385 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ae514df390 0x55ae518a4bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:53:26.672 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:26.669+0000 7fbbe77fe640 1 --2- 172.21.15.39:0/710814385 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ae51500070 0x55ae518a5100 secure :-1 s=READY pgs=1067 cs=0 l=1 rev1=1 crypto rx=0x7fbbdc001eb0 tx=0x7fbbdc00b390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=53b3873d901aecbb server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:53:26.675 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:26.669+0000 7fbbcdffb640 1 -- 172.21.15.39:0/710814385 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbdc0122f0 con 0x55ae51500070 2024-09-21T03:53:26.675 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:26.669+0000 7fbbcdffb640 1 -- 172.21.15.39:0/710814385 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbbdc003fa0 con 0x55ae51500070 2024-09-21T03:53:26.675 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:26.669+0000 7fbbcdffb640 1 -- 172.21.15.39:0/710814385 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbdc005620 con 0x55ae51500070 2024-09-21T03:53:26.675 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:26.670+0000 7fbbf48f2740 1 -- 172.21.15.39:0/710814385 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ae51500070 msgr2=0x55ae518a5100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:53:26.675 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:26.670+0000 7fbbf48f2740 1 --2- 172.21.15.39:0/710814385 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ae51500070 0x55ae518a5100 secure :-1 s=READY pgs=1067 cs=0 l=1 rev1=1 crypto rx=0x7fbbdc001eb0 tx=0x7fbbdc00b390 comp rx=0 tx=0).stop 2024-09-21T03:53:26.675 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:26.670+0000 7fbbf48f2740 1 -- 172.21.15.39:0/710814385 shutdown_connections 2024-09-21T03:53:26.675 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:26.670+0000 7fbbf48f2740 1 --2- 172.21.15.39:0/710814385 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ae51500070 0x55ae518a5100 unknown :-1 s=CLOSED pgs=1067 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:26.675 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:26.670+0000 7fbbf48f2740 1 --2- 172.21.15.39:0/710814385 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ae514df390 0x55ae518a4bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:26.675 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:26.670+0000 7fbbf48f2740 1 --2- 172.21.15.39:0/710814385 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ae514d71b0 0x55ae518b49a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:26.675 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:26.670+0000 7fbbf48f2740 1 -- 172.21.15.39:0/710814385 >> 172.21.15.39:0/710814385 conn(0x55ae514f4460 msgr2=0x55ae518b4290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:53:26.675 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:26.670+0000 7fbbf48f2740 1 -- 172.21.15.39:0/710814385 shutdown_connections 2024-09-21T03:53:26.675 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:26.670+0000 7fbbf48f2740 1 -- 172.21.15.39:0/710814385 wait complete. 2024-09-21T03:53:26.676 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:26.670+0000 7fbbf48f2740 1 -- start start 2024-09-21T03:53:26.676 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:26.670+0000 7fbbf48f2740 1 Processor -- start 2024-09-21T03:53:26.676 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:26.671+0000 7fbbf48f2740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ae514d71b0 0x55ae518b7060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:53:26.676 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:26.671+0000 7fbbf48f2740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ae514df390 0x55ae518b75a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:53:26.676 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:26.671+0000 7fbbf48f2740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ae51500070 0x55ae518b7ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:53:26.676 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:26.671+0000 7fbbf48f2740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fbbe81b2e90 con 0x55ae51500070 2024-09-21T03:53:26.676 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:26.671+0000 7fbbf48f2740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fbbe81b27c0 con 0x55ae514d71b0 2024-09-21T03:53:26.676 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:26.671+0000 7fbbf48f2740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fbbe81afa00 con 0x55ae514df390 2024-09-21T03:53:26.676 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:26.671+0000 7fbbf48f2740 10 client.0 _ll_register_callbacks cb 0x7fbbdc00b9e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:53:26.676 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:26.671+0000 7fbbe77fe640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ae51500070 0x55ae518b7ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:53:26.676 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:26.671+0000 7fbbe77fe640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ae51500070 0x55ae518b7ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:57386/0 (socket says 172.21.15.39:57386) 2024-09-21T03:53:26.676 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:26.671+0000 7fbbe77fe640 1 -- 172.21.15.39:0/3528710937 learned_addr learned my addr 172.21.15.39:0/3528710937 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:53:26.676 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:26.671+0000 7fbbe77fe640 1 --2- 172.21.15.39:0/3528710937 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ae514d71b0 0x55ae518b7060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:53:26.676 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:26.671+0000 7fbbe7fff640 1 --2- 172.21.15.39:0/3528710937 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ae514df390 0x55ae518b75a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:53:26.677 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:26.671+0000 7fbbe77fe640 1 -- 172.21.15.39:0/3528710937 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ae514df390 msgr2=0x55ae518b75a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:53:26.677 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:26.671+0000 7fbbe77fe640 1 --2- 172.21.15.39:0/3528710937 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ae514df390 0x55ae518b75a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:26.677 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:26.671+0000 7fbbe77fe640 1 -- 172.21.15.39:0/3528710937 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ae514d71b0 msgr2=0x55ae518b7060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:53:26.677 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:26.671+0000 7fbbe77fe640 1 --2- 172.21.15.39:0/3528710937 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ae514d71b0 0x55ae518b7060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:26.677 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:26.671+0000 7fbbe77fe640 1 -- 172.21.15.39:0/3528710937 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ae5192ead0 con 0x55ae51500070 2024-09-21T03:53:26.677 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:26.672+0000 7fbbe77fe640 1 --2- 172.21.15.39:0/3528710937 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ae51500070 0x55ae518b7ae0 secure :-1 s=READY pgs=1068 cs=0 l=1 rev1=1 crypto rx=0x7fbbdc00bbf0 tx=0x7fbbdc00b510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8cf332b3b140de82 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:53:26.677 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:26.672+0000 7fbbc77fe640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-09-21T03:53:26.677 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:26.672+0000 7fbbc77fe640 1 -- 172.21.15.39:0/3528710937 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbdc012e60 con 0x55ae51500070 2024-09-21T03:53:26.677 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:26.672+0000 7fbbc77fe640 1 -- 172.21.15.39:0/3528710937 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8528+0+0 (secure 0 0 0) 0x7fbbdc004360 con 0x55ae51500070 2024-09-21T03:53:26.677 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:26.672+0000 7fbbf48f2740 1 -- 172.21.15.39:0/3528710937 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbbdc003fa0 con 0x55ae51500070 2024-09-21T03:53:26.677 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:26.672+0000 7fbbc77fe640 1 -- 172.21.15.39:0/3528710937 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fbbdc005620 con 0x55ae51500070 2024-09-21T03:53:26.677 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:26.672+0000 7fbbf48f2740 10 client.17487 fetch_fsmap learned FSMap version 29 2024-09-21T03:53:26.677 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:26.672+0000 7fbbf48f2740 1 -- 172.21.15.39:0/3528710937 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55ae5192fd00 con 0x55ae51500070 2024-09-21T03:53:26.679 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:26.672+0000 7fbbc77fe640 1 -- 172.21.15.39:0/3528710937 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbbdc003fa0 con 0x55ae51500070 2024-09-21T03:53:26.679 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:26.673+0000 7fbbc77fe640 1 -- 172.21.15.39:0/3528710937 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fbbdc01b040 con 0x55ae51500070 2024-09-21T03:53:26.679 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:26.673+0000 7fbbf48f2740 10 client.17487 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T03:53:26.679 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:26.673+0000 7fbbf48f2740 10 client.17487 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-21T03:53:26.679 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:26.673+0000 7fbbf48f2740 20 client.17487 populate_metadata read hostname 'smithi039' 2024-09-21T03:53:26.679 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:26.673+0000 7fbbf48f2740 1 --2- 172.21.15.39:0/3528710937 >> [v2:172.21.15.150:6836/4017043144,v1:172.21.15.150:6837/4017043144] conn(0x55ae51493da0 0x55ae519322d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:53:26.679 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:26.673+0000 7fbbf48f2740 4 client.17487 mds_command: new command op to 24659 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T03:53:26.679 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:26.673+0000 7fbbf48f2740 1 -- 172.21.15.39:0/3528710937 --> [v2:172.21.15.150:6836/4017043144,v1:172.21.15.150:6837/4017043144] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55ae5181b8a0 con 0x55ae51493da0 2024-09-21T03:53:26.680 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:26.673+0000 7fbbe7fff640 1 --2- 172.21.15.39:0/3528710937 >> [v2:172.21.15.150:6836/4017043144,v1:172.21.15.150:6837/4017043144] conn(0x55ae51493da0 0x55ae519322d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:53:26.680 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:26.673+0000 7fbbc77fe640 10 client.17487 ms_handle_connect on v2:172.21.15.150:6836/4017043144 2024-09-21T03:53:26.680 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:26.673+0000 7fbbe7fff640 1 --2- 172.21.15.39:0/3528710937 >> [v2:172.21.15.150:6836/4017043144,v1:172.21.15.150:6837/4017043144] conn(0x55ae51493da0 0x55ae519322d0 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=c54e742c2e237398 server_cookie=701933b14cbe810a in_seq=0 out_seq=0 2024-09-21T03:53:26.680 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:26.674+0000 7fbbc77fe640 1 -- 172.21.15.39:0/3528710937 <== mds.2 v2:172.21.15.150:6836/4017043144 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55ae5181b8a0 con 0x55ae51493da0 2024-09-21T03:53:26.680 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:26.674+0000 7fbbc77fe640 10 client.17487 handle_command_reply: tid=0 2024-09-21T03:53:26.680 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:26.674+0000 7fbbf48f2740 1 client.17487 shutdown 2024-09-21T03:53:26.680 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:26.674+0000 7fbbf48f2740 1 -- 172.21.15.39:0/3528710937 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ae51500070 msgr2=0x55ae518b7ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:53:26.680 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:26.674+0000 7fbbf48f2740 1 --2- 172.21.15.39:0/3528710937 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ae51500070 0x55ae518b7ae0 secure :-1 s=READY pgs=1068 cs=0 l=1 rev1=1 crypto rx=0x7fbbdc00bbf0 tx=0x7fbbdc00b510 comp rx=0 tx=0).stop 2024-09-21T03:53:26.680 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:26.674+0000 7fbbf48f2740 1 -- 172.21.15.39:0/3528710937 shutdown_connections 2024-09-21T03:53:26.680 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:26.674+0000 7fbbf48f2740 1 --2- 172.21.15.39:0/3528710937 >> [v2:172.21.15.150:6836/4017043144,v1:172.21.15.150:6837/4017043144] conn(0x55ae51493da0 0x55ae519322d0 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:26.680 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:26.674+0000 7fbbf48f2740 1 --2- 172.21.15.39:0/3528710937 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ae51500070 0x55ae518b7ae0 unknown :-1 s=CLOSED pgs=1068 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:26.680 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:26.674+0000 7fbbf48f2740 1 --2- 172.21.15.39:0/3528710937 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ae514df390 0x55ae518b75a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:26.680 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:26.674+0000 7fbbf48f2740 1 --2- 172.21.15.39:0/3528710937 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ae514d71b0 0x55ae518b7060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:26.680 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:26.674+0000 7fbbf48f2740 1 -- 172.21.15.39:0/3528710937 >> 172.21.15.39:0/3528710937 conn(0x55ae514f4460 msgr2=0x55ae518b4b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:53:26.681 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:26.674+0000 7fbbc77fe640 0 client.17487 ms_handle_reset on v2:172.21.15.150:6836/4017043144 2024-09-21T03:53:26.681 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:26.674+0000 7fbbf48f2740 1 -- 172.21.15.39:0/3528710937 shutdown_connections 2024-09-21T03:53:26.681 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:26.674+0000 7fbbf48f2740 1 -- 172.21.15.39:0/3528710937 wait complete. 2024-09-21T03:53:26.681 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:26.674+0000 7fbbf48f2740 20 client.17487 trim_cache size 0 max 16384 2024-09-21T03:53:26.681 INFO:teuthology.orchestra.run.smithi039.stdout:[] 2024-09-21T03:53:26.682 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:26.675+0000 7fbbee37e640 1 -- 172.21.15.39:0/2636384879 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7fbbd00b66b0 msgr2=0x7fbbd00b8bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:53:26.682 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:26.675+0000 7fbbee37e640 1 --2- 172.21.15.39:0/2636384879 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7fbbd00b66b0 0x7fbbd00b8bc0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7fbbe009a450 tx=0x7fbbe009bbb0 comp rx=0 tx=0).stop 2024-09-21T03:53:26.682 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:26.675+0000 7fbbee37e640 1 -- 172.21.15.39:0/2636384879 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbbe8108d10 msgr2=0x7fbbe819bd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:53:26.682 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:26.675+0000 7fbbee37e640 1 --2- 172.21.15.39:0/2636384879 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbbe8108d10 0x7fbbe819bd10 secure :-1 s=READY pgs=1064 cs=0 l=1 rev1=1 crypto rx=0x7fbbe0005cc0 tx=0x7fbbe002f380 comp rx=0 tx=0).stop 2024-09-21T03:53:26.682 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:26.675+0000 7fbbee37e640 1 -- 172.21.15.39:0/2636384879 shutdown_connections 2024-09-21T03:53:26.682 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:26.675+0000 7fbbee37e640 1 --2- 172.21.15.39:0/2636384879 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7fbbd00b66b0 0x7fbbd00b8bc0 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:26.682 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:26.675+0000 7fbbee37e640 1 --2- 172.21.15.39:0/2636384879 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbbe8108d10 0x7fbbe819bd10 unknown :-1 s=CLOSED pgs=1064 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:26.682 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:26.675+0000 7fbbee37e640 1 --2- 172.21.15.39:0/2636384879 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbbe81083c0 0x7fbbe819b7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:26.682 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:26.675+0000 7fbbee37e640 1 --2- 172.21.15.39:0/2636384879 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbbe8102a80 0x7fbbe819b290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:26.682 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:26.675+0000 7fbbee37e640 1 -- 172.21.15.39:0/2636384879 >> 172.21.15.39:0/2636384879 conn(0x7fbbe8079740 msgr2=0x7fbbe8104430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:53:26.682 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:26.675+0000 7fbbee37e640 1 -- 172.21.15.39:0/2636384879 shutdown_connections 2024-09-21T03:53:26.682 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:26.675+0000 7fbbee37e640 1 -- 172.21.15.39:0/2636384879 wait complete. 2024-09-21T03:53:27.053 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:53:26 smithi039 ceph-mon[31211]: pgmap v549: 97 pgs: 97 active+clean; 10 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 102 B/s wr, 10 op/s 2024-09-21T03:53:27.218 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:53:26 smithi161 ceph-mon[34533]: pgmap v549: 97 pgs: 97 active+clean; 10 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 102 B/s wr, 10 op/s 2024-09-21T03:53:27.225 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:53:26 smithi150 ceph-mon[34289]: pgmap v549: 97 pgs: 97 active+clean; 10 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 102 B/s wr, 10 op/s 2024-09-21T03:53:27.466 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 51116d5c-77ca-11ef-baf5-efdc52797490 -- ceph tell mds.1:1 damage ls 2024-09-21T03:53:27.728 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/51116d5c-77ca-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:53:28.234 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:28.208+0000 7f874e745640 1 -- 172.21.15.39:0/2350028829 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f87481049b0 msgr2=0x7f8748104dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:53:28.234 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:28.208+0000 7f874e745640 1 --2- 172.21.15.39:0/2350028829 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f87481049b0 0x7f8748104dc0 secure :-1 s=READY pgs=1069 cs=0 l=1 rev1=1 crypto rx=0x7f8738009900 tx=0x7f873802a410 comp rx=0 tx=0).stop 2024-09-21T03:53:28.235 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:28.208+0000 7f874e745640 1 -- 172.21.15.39:0/2350028829 shutdown_connections 2024-09-21T03:53:28.235 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:28.208+0000 7f874e745640 1 --2- 172.21.15.39:0/2350028829 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f874810de10 0x7f8748110300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:28.235 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:28.208+0000 7f874e745640 1 --2- 172.21.15.39:0/2350028829 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8748105300 0x7f874810d850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:28.235 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:28.208+0000 7f874e745640 1 --2- 172.21.15.39:0/2350028829 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f87481049b0 0x7f8748104dc0 unknown :-1 s=CLOSED pgs=1069 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:28.235 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:28.208+0000 7f874e745640 1 -- 172.21.15.39:0/2350028829 >> 172.21.15.39:0/2350028829 conn(0x7f8748079730 msgr2=0x7f8748079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:53:28.235 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:28.208+0000 7f874e745640 1 -- 172.21.15.39:0/2350028829 shutdown_connections 2024-09-21T03:53:28.235 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:28.208+0000 7f874e745640 1 -- 172.21.15.39:0/2350028829 wait complete. 2024-09-21T03:53:28.235 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:28.209+0000 7f874e745640 1 Processor -- start 2024-09-21T03:53:28.235 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:28.209+0000 7f874e745640 1 -- start start 2024-09-21T03:53:28.235 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:28.209+0000 7f874e745640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f87481049b0 0x7f874819a5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:53:28.235 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:28.209+0000 7f874e745640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8748105300 0x7f874819ab40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:53:28.235 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:28.209+0000 7f874e745640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f874810de10 0x7f874819fde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:53:28.236 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:28.209+0000 7f874e745640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f8748113ad0 con 0x7f8748105300 2024-09-21T03:53:28.236 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:28.209+0000 7f874e745640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f8748113950 con 0x7f874810de10 2024-09-21T03:53:28.236 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:28.209+0000 7f874e745640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f8748113c50 con 0x7f87481049b0 2024-09-21T03:53:28.236 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:28.209+0000 7f87477fe640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8748105300 0x7f874819ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:53:28.236 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:28.209+0000 7f87477fe640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8748105300 0x7f874819ab40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:54510/0 (socket says 172.21.15.39:54510) 2024-09-21T03:53:28.236 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:28.209+0000 7f87477fe640 1 -- 172.21.15.39:0/2146947961 learned_addr learned my addr 172.21.15.39:0/2146947961 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:53:28.236 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:28.210+0000 7f874cce2640 1 --2- 172.21.15.39:0/2146947961 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f874810de10 0x7f874819fde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:53:28.236 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:28.210+0000 7f8747fff640 1 --2- 172.21.15.39:0/2146947961 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f87481049b0 0x7f874819a5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:53:28.236 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:28.210+0000 7f87477fe640 1 -- 172.21.15.39:0/2146947961 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f87481049b0 msgr2=0x7f874819a5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:53:28.236 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:28.210+0000 7f87477fe640 1 --2- 172.21.15.39:0/2146947961 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f87481049b0 0x7f874819a5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:28.236 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:28.210+0000 7f87477fe640 1 -- 172.21.15.39:0/2146947961 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f874810de10 msgr2=0x7f874819fde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:53:28.236 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:28.210+0000 7f87477fe640 1 --2- 172.21.15.39:0/2146947961 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f874810de10 0x7f874819fde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:28.236 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:28.210+0000 7f87477fe640 1 -- 172.21.15.39:0/2146947961 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f87481a0450 con 0x7f8748105300 2024-09-21T03:53:28.236 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:28.210+0000 7f874cce2640 1 --2- 172.21.15.39:0/2146947961 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f874810de10 0x7f874819fde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:53:28.237 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:28.210+0000 7f87477fe640 1 --2- 172.21.15.39:0/2146947961 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8748105300 0x7f874819ab40 secure :-1 s=READY pgs=1070 cs=0 l=1 rev1=1 crypto rx=0x7f873c006fd0 tx=0x7f873c00b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:53:28.237 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:28.210+0000 7f87457fa640 1 -- 172.21.15.39:0/2146947961 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f873c011830 con 0x7f8748105300 2024-09-21T03:53:28.237 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:28.210+0000 7f874e745640 1 -- 172.21.15.39:0/2146947961 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f87481a0740 con 0x7f8748105300 2024-09-21T03:53:28.237 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:28.210+0000 7f87457fa640 1 -- 172.21.15.39:0/2146947961 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f873c016070 con 0x7f8748105300 2024-09-21T03:53:28.237 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:28.210+0000 7f87457fa640 1 -- 172.21.15.39:0/2146947961 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f873c010e00 con 0x7f8748105300 2024-09-21T03:53:28.237 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:28.210+0000 7f874e745640 1 -- 172.21.15.39:0/2146947961 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f87481a6f60 con 0x7f8748105300 2024-09-21T03:53:28.237 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:28.213+0000 7f8754cb9740 1 Processor -- start 2024-09-21T03:53:28.237 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:28.213+0000 7f8754cb9740 1 -- start start 2024-09-21T03:53:28.237 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:28.213+0000 7f87457fa640 1 -- 172.21.15.39:0/2146947961 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 27) ==== 154643+0+0 (secure 0 0 0) 0x7f873c01b460 con 0x7f8748105300 2024-09-21T03:53:28.237 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:28.213+0000 7f8754cb9740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561fa1648070 0x561fa19ef6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:53:28.237 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:28.213+0000 7f8754cb9740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561fa161f1b0 0x561fa19efbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:53:28.237 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:28.213+0000 7f8754cb9740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561fa1627390 0x561fa19f0120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:53:28.237 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:28.213+0000 7f8754cb9740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x561fa1965ea0 con 0x561fa1648070 2024-09-21T03:53:28.237 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:28.213+0000 7f8754cb9740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x561fa19552d0 con 0x561fa161f1b0 2024-09-21T03:53:28.238 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:28.213+0000 7f8754cb9740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x561fa19590e0 con 0x561fa1627390 2024-09-21T03:53:28.238 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:28.213+0000 7f874cce2640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561fa1648070 0x561fa19ef6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:53:28.238 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:28.213+0000 7f874cce2640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561fa1648070 0x561fa19ef6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:54518/0 (socket says 172.21.15.39:54518) 2024-09-21T03:53:28.238 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:28.213+0000 7f874cce2640 1 -- 172.21.15.39:0/2074718623 learned_addr learned my addr 172.21.15.39:0/2074718623 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:53:28.238 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:28.213+0000 7f87477fe640 1 --2- 172.21.15.39:0/2074718623 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561fa1627390 0x561fa19f0120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:53:28.238 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:28.214+0000 7f8747fff640 1 --2- 172.21.15.39:0/2074718623 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561fa161f1b0 0x561fa19efbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:53:28.238 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:28.214+0000 7f874cce2640 1 -- 172.21.15.39:0/2074718623 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561fa1627390 msgr2=0x561fa19f0120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:53:28.238 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:28.214+0000 7f874cce2640 1 --2- 172.21.15.39:0/2074718623 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561fa1627390 0x561fa19f0120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:28.238 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:28.214+0000 7f874cce2640 1 -- 172.21.15.39:0/2074718623 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561fa161f1b0 msgr2=0x561fa19efbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:53:28.238 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:28.214+0000 7f874cce2640 1 --2- 172.21.15.39:0/2074718623 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561fa161f1b0 0x561fa19efbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:28.238 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:28.214+0000 7f874cce2640 1 -- 172.21.15.39:0/2074718623 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561fa16b3650 con 0x561fa1648070 2024-09-21T03:53:28.238 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:28.214+0000 7f874cce2640 1 --2- 172.21.15.39:0/2074718623 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561fa1648070 0x561fa19ef6a0 secure :-1 s=READY pgs=1071 cs=0 l=1 rev1=1 crypto rx=0x7f8740007f40 tx=0x7f874000e7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d3c5a47e5a51b1c8 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:53:28.238 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:28.214+0000 7f87457fa640 1 --2- 172.21.15.39:0/2146947961 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f87300b66b0 0x7f87300b8bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:53:28.238 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:28.214+0000 7f872e7fc640 1 -- 172.21.15.39:0/2074718623 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8740014070 con 0x561fa1648070 2024-09-21T03:53:28.239 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:28.214+0000 7f872e7fc640 1 -- 172.21.15.39:0/2074718623 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8740004d20 con 0x561fa1648070 2024-09-21T03:53:28.239 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:28.214+0000 7f872e7fc640 1 -- 172.21.15.39:0/2074718623 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8740005460 con 0x561fa1648070 2024-09-21T03:53:28.239 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:28.214+0000 7f8747fff640 1 --2- 172.21.15.39:0/2146947961 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f87300b66b0 0x7f87300b8bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:53:28.239 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:28.214+0000 7f87457fa640 1 -- 172.21.15.39:0/2146947961 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8528+0+0 (secure 0 0 0) 0x7f873c0e6cf0 con 0x7f8748105300 2024-09-21T03:53:28.239 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:28.215+0000 7f8754cb9740 1 -- 172.21.15.39:0/2074718623 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561fa1648070 msgr2=0x561fa19ef6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:53:28.239 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:28.215+0000 7f8754cb9740 1 --2- 172.21.15.39:0/2074718623 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561fa1648070 0x561fa19ef6a0 secure :-1 s=READY pgs=1071 cs=0 l=1 rev1=1 crypto rx=0x7f8740007f40 tx=0x7f874000e7e0 comp rx=0 tx=0).stop 2024-09-21T03:53:28.239 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:28.215+0000 7f8747fff640 1 --2- 172.21.15.39:0/2146947961 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f87300b66b0 0x7f87300b8bc0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f8738000c00 tx=0x7f8738009660 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:53:28.239 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:28.215+0000 7f8754cb9740 1 -- 172.21.15.39:0/2074718623 shutdown_connections 2024-09-21T03:53:28.239 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:28.215+0000 7f8754cb9740 1 --2- 172.21.15.39:0/2074718623 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561fa1627390 0x561fa19f0120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:28.239 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:28.215+0000 7f8754cb9740 1 --2- 172.21.15.39:0/2074718623 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561fa161f1b0 0x561fa19efbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:28.239 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:28.215+0000 7f8754cb9740 1 --2- 172.21.15.39:0/2074718623 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561fa1648070 0x561fa19ef6a0 unknown :-1 s=CLOSED pgs=1071 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:28.239 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:28.215+0000 7f8754cb9740 1 -- 172.21.15.39:0/2074718623 >> 172.21.15.39:0/2074718623 conn(0x561fa163c460 msgr2=0x561fa19ed590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:53:28.239 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:28.215+0000 7f8754cb9740 1 -- 172.21.15.39:0/2074718623 shutdown_connections 2024-09-21T03:53:28.239 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:28.215+0000 7f8754cb9740 1 -- 172.21.15.39:0/2074718623 wait complete. 2024-09-21T03:53:28.240 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:28.216+0000 7f8754cb9740 1 -- start start 2024-09-21T03:53:28.240 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:28.216+0000 7f8754cb9740 1 Processor -- start 2024-09-21T03:53:28.240 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:28.216+0000 7f8754cb9740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561fa161f1b0 0x561fa19fae00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:53:28.240 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:28.216+0000 7f8754cb9740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561fa1627390 0x561fa19fb340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:53:28.240 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:28.216+0000 7f8754cb9740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561fa1648070 0x561fa1a07170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:53:28.240 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:28.216+0000 7f8754cb9740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x561fa17b6700 con 0x561fa1648070 2024-09-21T03:53:28.240 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:28.216+0000 7f8754cb9740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x561fa18f0160 con 0x561fa161f1b0 2024-09-21T03:53:28.240 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:28.216+0000 7f8754cb9740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x561fa19fde80 con 0x561fa1627390 2024-09-21T03:53:28.240 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:28.216+0000 7f8754cb9740 10 client.0 _ll_register_callbacks cb 0x561fa19099e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:53:28.240 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:28.216+0000 7f874cce2640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561fa1648070 0x561fa1a07170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:53:28.240 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:28.216+0000 7f874cce2640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561fa1648070 0x561fa1a07170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:54520/0 (socket says 172.21.15.39:54520) 2024-09-21T03:53:28.240 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:28.216+0000 7f874cce2640 1 -- 172.21.15.39:0/181977843 learned_addr learned my addr 172.21.15.39:0/181977843 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:53:28.240 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:28.216+0000 7f87477fe640 1 --2- 172.21.15.39:0/181977843 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561fa1627390 0x561fa19fb340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:53:28.240 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:28.217+0000 7f8747fff640 1 --2- 172.21.15.39:0/181977843 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561fa161f1b0 0x561fa19fae00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:53:28.241 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:28.217+0000 7f874cce2640 1 -- 172.21.15.39:0/181977843 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561fa1627390 msgr2=0x561fa19fb340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:53:28.241 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:28.217+0000 7f874cce2640 1 --2- 172.21.15.39:0/181977843 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561fa1627390 0x561fa19fb340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:28.241 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:28.217+0000 7f874cce2640 1 -- 172.21.15.39:0/181977843 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561fa161f1b0 msgr2=0x561fa19fae00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:53:28.241 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:28.217+0000 7f874cce2640 1 --2- 172.21.15.39:0/181977843 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561fa161f1b0 0x561fa19fae00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:28.241 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:28.217+0000 7f874cce2640 1 -- 172.21.15.39:0/181977843 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561fa191dc90 con 0x561fa1648070 2024-09-21T03:53:28.241 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:28.217+0000 7f8747fff640 1 --2- 172.21.15.39:0/181977843 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561fa161f1b0 0x561fa19fae00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:53:28.241 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:28.217+0000 7f874cce2640 1 --2- 172.21.15.39:0/181977843 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561fa1648070 0x561fa1a07170 secure :-1 s=READY pgs=1072 cs=0 l=1 rev1=1 crypto rx=0x7f8740007f10 tx=0x7f8740007c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=93b3a4d51f49079e server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:53:28.241 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:28.217+0000 7f87277fe640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-09-21T03:53:28.241 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:28.217+0000 7f87277fe640 1 -- 172.21.15.39:0/181977843 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8740027060 con 0x561fa1648070 2024-09-21T03:53:28.241 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:28.217+0000 7f87277fe640 1 -- 172.21.15.39:0/181977843 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8528+0+0 (secure 0 0 0) 0x7f8740010640 con 0x561fa1648070 2024-09-21T03:53:28.241 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:28.217+0000 7f8754cb9740 1 -- 172.21.15.39:0/181977843 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8740004d20 con 0x561fa1648070 2024-09-21T03:53:28.241 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:28.217+0000 7f87277fe640 1 -- 172.21.15.39:0/181977843 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f8740014070 con 0x561fa1648070 2024-09-21T03:53:28.241 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:28.217+0000 7f8754cb9740 10 client.17499 fetch_fsmap learned FSMap version 29 2024-09-21T03:53:28.241 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:28.217+0000 7f8754cb9740 1 -- 172.21.15.39:0/181977843 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x561fa1a09c30 con 0x561fa1648070 2024-09-21T03:53:28.241 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:28.218+0000 7f87277fe640 1 -- 172.21.15.39:0/181977843 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8740004d20 con 0x561fa1648070 2024-09-21T03:53:28.242 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:28.218+0000 7f87277fe640 1 -- 172.21.15.39:0/181977843 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f87400051a0 con 0x561fa1648070 2024-09-21T03:53:28.242 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:28.218+0000 7f8754cb9740 10 client.17499 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T03:53:28.242 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:28.218+0000 7f8754cb9740 10 client.17499 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-21T03:53:28.242 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:28.218+0000 7f8754cb9740 20 client.17499 populate_metadata read hostname 'smithi039' 2024-09-21T03:53:28.242 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:28.218+0000 7f8754cb9740 1 --2- 172.21.15.39:0/181977843 >> [v2:172.21.15.161:6838/2190107270,v1:172.21.15.161:6839/2190107270] conn(0x561fa16069f0 0x561fa1a0d0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:53:28.242 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:28.218+0000 7f8754cb9740 4 client.17499 mds_command: new command op to 24667 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:53:28.242 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:28.218+0000 7f8754cb9740 1 -- 172.21.15.39:0/181977843 --> [v2:172.21.15.161:6838/2190107270,v1:172.21.15.161:6839/2190107270] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561fa1967df0 con 0x561fa16069f0 2024-09-21T03:53:28.242 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:28.218+0000 7f8747fff640 1 --2- 172.21.15.39:0/181977843 >> [v2:172.21.15.161:6838/2190107270,v1:172.21.15.161:6839/2190107270] conn(0x561fa16069f0 0x561fa1a0d0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:53:28.242 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:28.219+0000 7f8747fff640 1 --2- 172.21.15.39:0/181977843 >> [v2:172.21.15.161:6838/2190107270,v1:172.21.15.161:6839/2190107270] conn(0x561fa16069f0 0x561fa1a0d0a0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=16df05e7bc7dc62d server_cookie=3cfcd33d812e4a22 in_seq=0 out_seq=0 2024-09-21T03:53:28.242 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:28.219+0000 7f87277fe640 10 client.17499 ms_handle_connect on v2:172.21.15.161:6838/2190107270 2024-09-21T03:53:28.242 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:28.220+0000 7f87277fe640 1 -- 172.21.15.39:0/181977843 <== mds.1 v2:172.21.15.161:6838/2190107270 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x561fa1967df0 con 0x561fa16069f0 2024-09-21T03:53:28.242 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:28.220+0000 7f87277fe640 10 client.17499 handle_command_reply: tid=0 2024-09-21T03:53:28.242 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:28.220+0000 7f8754cb9740 1 client.17499 shutdown 2024-09-21T03:53:28.242 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:28.220+0000 7f8754cb9740 1 -- 172.21.15.39:0/181977843 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561fa1648070 msgr2=0x561fa1a07170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:53:28.242 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:28.220+0000 7f8754cb9740 1 --2- 172.21.15.39:0/181977843 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561fa1648070 0x561fa1a07170 secure :-1 s=READY pgs=1072 cs=0 l=1 rev1=1 crypto rx=0x7f8740007f10 tx=0x7f8740007c80 comp rx=0 tx=0).stop 2024-09-21T03:53:28.243 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:28.220+0000 7f8754cb9740 1 -- 172.21.15.39:0/181977843 shutdown_connections 2024-09-21T03:53:28.243 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:28.220+0000 7f8754cb9740 1 --2- 172.21.15.39:0/181977843 >> [v2:172.21.15.161:6838/2190107270,v1:172.21.15.161:6839/2190107270] conn(0x561fa16069f0 0x561fa1a0d0a0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:28.243 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:28.220+0000 7f8754cb9740 1 --2- 172.21.15.39:0/181977843 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561fa1648070 0x561fa1a07170 unknown :-1 s=CLOSED pgs=1072 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:28.243 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:28.220+0000 7f8754cb9740 1 --2- 172.21.15.39:0/181977843 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561fa1627390 0x561fa19fb340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:28.243 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:28.220+0000 7f8754cb9740 1 --2- 172.21.15.39:0/181977843 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561fa161f1b0 0x561fa19fae00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:28.243 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:28.220+0000 7f87277fe640 0 client.17499 ms_handle_reset on v2:172.21.15.161:6838/2190107270 2024-09-21T03:53:28.243 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:28.220+0000 7f8754cb9740 1 -- 172.21.15.39:0/181977843 >> 172.21.15.39:0/181977843 conn(0x561fa163c460 msgr2=0x561fa19f2880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:53:28.243 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:28.220+0000 7f8754cb9740 1 -- 172.21.15.39:0/181977843 shutdown_connections 2024-09-21T03:53:28.243 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:28.220+0000 7f8754cb9740 1 -- 172.21.15.39:0/181977843 wait complete. 2024-09-21T03:53:28.243 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:28.220+0000 7f8754cb9740 20 client.17499 trim_cache size 0 max 16384 2024-09-21T03:53:28.248 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:28.233+0000 7f8754cb9740 1 Processor -- start 2024-09-21T03:53:28.248 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:28.233+0000 7f8754cb9740 1 -- start start 2024-09-21T03:53:28.248 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:28.234+0000 7f8754cb9740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561fa161f1b0 0x561fa19f2eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:53:28.248 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:28.234+0000 7f8754cb9740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561fa1627390 0x561fa1a070e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:53:28.248 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:28.234+0000 7f8754cb9740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561fa1648070 0x561fa19fabb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:53:28.248 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:28.234+0000 7f8754cb9740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f87481af810 con 0x561fa161f1b0 2024-09-21T03:53:28.248 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:28.234+0000 7f8754cb9740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f87481b1350 con 0x561fa1627390 2024-09-21T03:53:28.249 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:28.234+0000 7f8754cb9740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f87481afee0 con 0x561fa1648070 2024-09-21T03:53:28.249 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:28.234+0000 7f8747fff640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561fa161f1b0 0x561fa19f2eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:53:28.249 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:28.234+0000 7f8747fff640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561fa161f1b0 0x561fa19f2eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:54522/0 (socket says 172.21.15.39:54522) 2024-09-21T03:53:28.249 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:28.234+0000 7f8747fff640 1 -- 172.21.15.39:0/486607202 learned_addr learned my addr 172.21.15.39:0/486607202 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:53:28.249 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:28.234+0000 7f874cce2640 1 --2- 172.21.15.39:0/486607202 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561fa1648070 0x561fa19fabb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:53:28.249 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:28.234+0000 7f87477fe640 1 --2- 172.21.15.39:0/486607202 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561fa1627390 0x561fa1a070e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:53:28.249 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:28.234+0000 7f8747fff640 1 -- 172.21.15.39:0/486607202 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561fa1648070 msgr2=0x561fa19fabb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:53:28.249 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:28.234+0000 7f8747fff640 1 --2- 172.21.15.39:0/486607202 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561fa1648070 0x561fa19fabb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:28.249 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:28.234+0000 7f8747fff640 1 -- 172.21.15.39:0/486607202 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561fa1627390 msgr2=0x561fa1a070e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:53:28.249 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:28.234+0000 7f8747fff640 1 --2- 172.21.15.39:0/486607202 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561fa1627390 0x561fa1a070e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:28.249 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:28.234+0000 7f8747fff640 1 -- 172.21.15.39:0/486607202 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561fa1606c20 con 0x561fa161f1b0 2024-09-21T03:53:28.249 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:28.234+0000 7f87477fe640 1 --2- 172.21.15.39:0/486607202 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561fa1627390 0x561fa1a070e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:53:28.249 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:28.234+0000 7f8747fff640 1 --2- 172.21.15.39:0/486607202 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561fa161f1b0 0x561fa19f2eb0 secure :-1 s=READY pgs=1073 cs=0 l=1 rev1=1 crypto rx=0x7f8738009900 tx=0x7f873803d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7bfe4ded7964d695 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:53:28.249 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:28.234+0000 7f872dffb640 1 -- 172.21.15.39:0/486607202 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8738055070 con 0x561fa161f1b0 2024-09-21T03:53:28.249 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:28.235+0000 7f872dffb640 1 -- 172.21.15.39:0/486607202 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f87380035f0 con 0x561fa161f1b0 2024-09-21T03:53:28.250 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:28.235+0000 7f872dffb640 1 -- 172.21.15.39:0/486607202 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8738035050 con 0x561fa161f1b0 2024-09-21T03:53:28.250 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:28.235+0000 7f8754cb9740 1 -- 172.21.15.39:0/486607202 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561fa161f1b0 msgr2=0x561fa19f2eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:53:28.250 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:28.235+0000 7f8754cb9740 1 --2- 172.21.15.39:0/486607202 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561fa161f1b0 0x561fa19f2eb0 secure :-1 s=READY pgs=1073 cs=0 l=1 rev1=1 crypto rx=0x7f8738009900 tx=0x7f873803d000 comp rx=0 tx=0).stop 2024-09-21T03:53:28.250 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:28.235+0000 7f8754cb9740 1 -- 172.21.15.39:0/486607202 shutdown_connections 2024-09-21T03:53:28.250 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:28.235+0000 7f8754cb9740 1 --2- 172.21.15.39:0/486607202 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561fa1648070 0x561fa19fabb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:28.250 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:28.235+0000 7f8754cb9740 1 --2- 172.21.15.39:0/486607202 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561fa1627390 0x561fa1a070e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:28.250 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:28.235+0000 7f8754cb9740 1 --2- 172.21.15.39:0/486607202 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561fa161f1b0 0x561fa19f2eb0 unknown :-1 s=CLOSED pgs=1073 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:28.250 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:28.235+0000 7f8754cb9740 1 -- 172.21.15.39:0/486607202 >> 172.21.15.39:0/486607202 conn(0x561fa163c460 msgr2=0x561fa19f27a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:53:28.250 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:28.235+0000 7f8754cb9740 1 -- 172.21.15.39:0/486607202 shutdown_connections 2024-09-21T03:53:28.250 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:28.235+0000 7f8754cb9740 1 -- 172.21.15.39:0/486607202 wait complete. 2024-09-21T03:53:28.250 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:28.236+0000 7f8754cb9740 1 -- start start 2024-09-21T03:53:28.250 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:28.236+0000 7f8754cb9740 1 Processor -- start 2024-09-21T03:53:28.250 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:28.236+0000 7f8754cb9740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561fa161f1b0 0x561fa1a0dab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:53:28.250 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:28.236+0000 7f8754cb9740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561fa1627390 0x561fa1a0e050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:53:28.250 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:28.236+0000 7f8754cb9740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561fa1648070 0x561fa1a73cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:53:28.251 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:28.237+0000 7f8754cb9740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f87481b0c80 con 0x561fa1627390 2024-09-21T03:53:28.251 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:28.237+0000 7f8754cb9740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f87481b05b0 con 0x561fa1648070 2024-09-21T03:53:28.251 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:28.237+0000 7f8754cb9740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f87481ad7f0 con 0x561fa161f1b0 2024-09-21T03:53:28.251 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:28.237+0000 7f8754cb9740 10 client.0 _ll_register_callbacks cb 0x7f8740007d90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:53:28.251 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:28.237+0000 7f87477fe640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561fa1627390 0x561fa1a0e050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:53:28.251 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:28.237+0000 7f87477fe640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561fa1627390 0x561fa1a0e050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:54534/0 (socket says 172.21.15.39:54534) 2024-09-21T03:53:28.251 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:28.237+0000 7f87477fe640 1 -- 172.21.15.39:0/773945307 learned_addr learned my addr 172.21.15.39:0/773945307 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:53:28.251 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:28.237+0000 7f874cce2640 1 --2- 172.21.15.39:0/773945307 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561fa1648070 0x561fa1a73cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:53:28.251 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:28.237+0000 7f8747fff640 1 --2- 172.21.15.39:0/773945307 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561fa161f1b0 0x561fa1a0dab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:53:28.251 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:28.237+0000 7f87477fe640 1 -- 172.21.15.39:0/773945307 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561fa161f1b0 msgr2=0x561fa1a0dab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:53:28.251 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:28.237+0000 7f87477fe640 1 --2- 172.21.15.39:0/773945307 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561fa161f1b0 0x561fa1a0dab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:28.251 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:28.237+0000 7f87477fe640 1 -- 172.21.15.39:0/773945307 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561fa1648070 msgr2=0x561fa1a73cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:53:28.251 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:28.237+0000 7f87477fe640 1 --2- 172.21.15.39:0/773945307 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561fa1648070 0x561fa1a73cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:28.251 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:28.237+0000 7f87477fe640 1 -- 172.21.15.39:0/773945307 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561fa1a74340 con 0x561fa1627390 2024-09-21T03:53:28.251 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:28.237+0000 7f8747fff640 1 --2- 172.21.15.39:0/773945307 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561fa161f1b0 0x561fa1a0dab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:53:28.251 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:28.237+0000 7f87477fe640 1 --2- 172.21.15.39:0/773945307 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561fa1627390 0x561fa1a0e050 secure :-1 s=READY pgs=1074 cs=0 l=1 rev1=1 crypto rx=0x7f873c0acad0 tx=0x7f873c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ebb6ba8aa46b628 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:53:28.252 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:28.237+0000 7f87277fe640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-09-21T03:53:28.252 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:28.237+0000 7f87277fe640 1 -- 172.21.15.39:0/773945307 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f873c0e48c0 con 0x561fa1627390 2024-09-21T03:53:28.252 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:28.237+0000 7f87277fe640 1 -- 172.21.15.39:0/773945307 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8528+0+0 (secure 0 0 0) 0x7f873c0e4b90 con 0x561fa1627390 2024-09-21T03:53:28.252 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:28.237+0000 7f8754cb9740 1 -- 172.21.15.39:0/773945307 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f87380035f0 con 0x561fa1627390 2024-09-21T03:53:28.252 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:28.238+0000 7f87277fe640 1 -- 172.21.15.39:0/773945307 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f873c0ac650 con 0x561fa1627390 2024-09-21T03:53:28.252 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:28.238+0000 7f8754cb9740 10 client.17505 fetch_fsmap learned FSMap version 29 2024-09-21T03:53:28.252 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:28.238+0000 7f8754cb9740 1 -- 172.21.15.39:0/773945307 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x561fa1a7b5f0 con 0x561fa1627390 2024-09-21T03:53:28.252 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:28.238+0000 7f87277fe640 1 -- 172.21.15.39:0/773945307 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f87380035f0 con 0x561fa1627390 2024-09-21T03:53:28.252 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:28.238+0000 7f87277fe640 1 -- 172.21.15.39:0/773945307 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f873c0b1220 con 0x561fa1627390 2024-09-21T03:53:28.252 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:28.238+0000 7f8754cb9740 10 client.17505 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T03:53:28.252 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:28.238+0000 7f8754cb9740 10 client.17505 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-21T03:53:28.252 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:28.238+0000 7f8754cb9740 20 client.17505 populate_metadata read hostname 'smithi039' 2024-09-21T03:53:28.252 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:28.238+0000 7f8754cb9740 1 --2- 172.21.15.39:0/773945307 >> [v2:172.21.15.161:6838/2190107270,v1:172.21.15.161:6839/2190107270] conn(0x561fa15dbda0 0x561fa1a7ddb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:53:28.252 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:28.238+0000 7f8754cb9740 4 client.17505 mds_command: new command op to 24667 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T03:53:28.252 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:28.238+0000 7f8754cb9740 1 -- 172.21.15.39:0/773945307 --> [v2:172.21.15.161:6838/2190107270,v1:172.21.15.161:6839/2190107270] -- command(tid 0: {"prefix": "damage ls"}) -- 0x561fa19638a0 con 0x561fa15dbda0 2024-09-21T03:53:28.253 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:28.238+0000 7f8747fff640 1 --2- 172.21.15.39:0/773945307 >> [v2:172.21.15.161:6838/2190107270,v1:172.21.15.161:6839/2190107270] conn(0x561fa15dbda0 0x561fa1a7ddb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:53:28.253 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:28.239+0000 7f8747fff640 1 --2- 172.21.15.39:0/773945307 >> [v2:172.21.15.161:6838/2190107270,v1:172.21.15.161:6839/2190107270] conn(0x561fa15dbda0 0x561fa1a7ddb0 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=4b2df7e5c1256c8d server_cookie=c95e791d8d4f4c4 in_seq=0 out_seq=0 2024-09-21T03:53:28.253 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:28.239+0000 7f87277fe640 10 client.17505 ms_handle_connect on v2:172.21.15.161:6838/2190107270 2024-09-21T03:53:28.253 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:28.239+0000 7f87277fe640 1 -- 172.21.15.39:0/773945307 <== mds.1 v2:172.21.15.161:6838/2190107270 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x561fa19638a0 con 0x561fa15dbda0 2024-09-21T03:53:28.253 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:28.239+0000 7f87277fe640 10 client.17505 handle_command_reply: tid=0 2024-09-21T03:53:28.253 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:28.239+0000 7f8754cb9740 1 client.17505 shutdown 2024-09-21T03:53:28.253 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:28.239+0000 7f8754cb9740 1 -- 172.21.15.39:0/773945307 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561fa1627390 msgr2=0x561fa1a0e050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:53:28.253 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:28.239+0000 7f8754cb9740 1 --2- 172.21.15.39:0/773945307 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561fa1627390 0x561fa1a0e050 secure :-1 s=READY pgs=1074 cs=0 l=1 rev1=1 crypto rx=0x7f873c0acad0 tx=0x7f873c0046c0 comp rx=0 tx=0).stop 2024-09-21T03:53:28.253 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:28.239+0000 7f8754cb9740 1 -- 172.21.15.39:0/773945307 shutdown_connections 2024-09-21T03:53:28.253 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:28.239+0000 7f8754cb9740 1 --2- 172.21.15.39:0/773945307 >> [v2:172.21.15.161:6838/2190107270,v1:172.21.15.161:6839/2190107270] conn(0x561fa15dbda0 0x561fa1a7ddb0 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:28.253 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:28.239+0000 7f8754cb9740 1 --2- 172.21.15.39:0/773945307 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561fa1648070 0x561fa1a73cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:28.253 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:28.239+0000 7f8754cb9740 1 --2- 172.21.15.39:0/773945307 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561fa1627390 0x561fa1a0e050 unknown :-1 s=CLOSED pgs=1074 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:28.253 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:28.239+0000 7f8754cb9740 1 --2- 172.21.15.39:0/773945307 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561fa161f1b0 0x561fa1a0dab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:28.253 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:28.239+0000 7f8754cb9740 1 -- 172.21.15.39:0/773945307 >> 172.21.15.39:0/773945307 conn(0x561fa163c460 msgr2=0x561fa19fabb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:53:28.253 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:28.240+0000 7f87277fe640 0 client.17505 ms_handle_reset on v2:172.21.15.161:6838/2190107270 2024-09-21T03:53:28.254 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:28.240+0000 7f8754cb9740 1 -- 172.21.15.39:0/773945307 shutdown_connections 2024-09-21T03:53:28.254 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:28.240+0000 7f8754cb9740 1 -- 172.21.15.39:0/773945307 wait complete. 2024-09-21T03:53:28.254 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:28.240+0000 7f8754cb9740 20 client.17505 trim_cache size 0 max 16384 2024-09-21T03:53:28.254 INFO:teuthology.orchestra.run.smithi039.stdout:[] 2024-09-21T03:53:28.255 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:28.240+0000 7f874e745640 1 -- 172.21.15.39:0/2146947961 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f87300b66b0 msgr2=0x7f87300b8bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:53:28.255 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:28.240+0000 7f874e745640 1 --2- 172.21.15.39:0/2146947961 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f87300b66b0 0x7f87300b8bc0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f8738000c00 tx=0x7f8738009660 comp rx=0 tx=0).stop 2024-09-21T03:53:28.255 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:28.240+0000 7f874e745640 1 -- 172.21.15.39:0/2146947961 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8748105300 msgr2=0x7f874819ab40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:53:28.255 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:28.240+0000 7f874e745640 1 --2- 172.21.15.39:0/2146947961 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8748105300 0x7f874819ab40 secure :-1 s=READY pgs=1070 cs=0 l=1 rev1=1 crypto rx=0x7f873c006fd0 tx=0x7f873c00b8e0 comp rx=0 tx=0).stop 2024-09-21T03:53:28.255 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:28.240+0000 7f874e745640 1 -- 172.21.15.39:0/2146947961 shutdown_connections 2024-09-21T03:53:28.255 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:28.240+0000 7f874e745640 1 --2- 172.21.15.39:0/2146947961 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f87300b66b0 0x7f87300b8bc0 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:28.255 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:28.240+0000 7f874e745640 1 --2- 172.21.15.39:0/2146947961 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f874810de10 0x7f874819fde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:28.255 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:28.240+0000 7f874e745640 1 --2- 172.21.15.39:0/2146947961 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8748105300 0x7f874819ab40 unknown :-1 s=CLOSED pgs=1070 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:28.255 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:28.240+0000 7f874e745640 1 --2- 172.21.15.39:0/2146947961 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f87481049b0 0x7f874819a5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:28.255 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:28.240+0000 7f874e745640 1 -- 172.21.15.39:0/2146947961 >> 172.21.15.39:0/2146947961 conn(0x7f8748079730 msgr2=0x7f87480ff7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:53:28.256 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:28.240+0000 7f874e745640 1 -- 172.21.15.39:0/2146947961 shutdown_connections 2024-09-21T03:53:28.256 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:28.240+0000 7f874e745640 1 -- 172.21.15.39:0/2146947961 wait complete. 2024-09-21T03:53:29.068 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 51116d5c-77ca-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 damage ls 2024-09-21T03:53:29.218 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:53:28 smithi161 ceph-mon[34533]: pgmap v550: 97 pgs: 97 active+clean; 10 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 102 B/s wr, 10 op/s 2024-09-21T03:53:29.224 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:53:28 smithi150 ceph-mon[34289]: pgmap v550: 97 pgs: 97 active+clean; 10 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 102 B/s wr, 10 op/s 2024-09-21T03:53:29.324 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/51116d5c-77ca-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:53:29.333 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:53:28 smithi039 ceph-mon[31211]: pgmap v550: 97 pgs: 97 active+clean; 10 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 102 B/s wr, 10 op/s 2024-09-21T03:53:29.799 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:29.795+0000 7f0cb46ab640 1 -- 172.21.15.39:0/3805043326 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0cac10de60 msgr2=0x7f0cac110330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:53:29.799 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:29.795+0000 7f0cb46ab640 1 --2- 172.21.15.39:0/3805043326 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0cac10de60 0x7f0cac110330 secure :-1 s=READY pgs=1075 cs=0 l=1 rev1=1 crypto rx=0x7f0ca4009ca0 tx=0x7f0ca402a8c0 comp rx=0 tx=0).stop 2024-09-21T03:53:29.799 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:29.795+0000 7f0cb46ab640 1 -- 172.21.15.39:0/3805043326 shutdown_connections 2024-09-21T03:53:29.799 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:29.795+0000 7f0cb46ab640 1 --2- 172.21.15.39:0/3805043326 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0cac10de60 0x7f0cac110330 unknown :-1 s=CLOSED pgs=1075 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:29.799 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:29.795+0000 7f0cb46ab640 1 --2- 172.21.15.39:0/3805043326 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0cac105380 0x7f0cac10d8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:29.799 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:29.795+0000 7f0cb46ab640 1 --2- 172.21.15.39:0/3805043326 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0cac104a30 0x7f0cac104e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:29.799 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:29.795+0000 7f0cb46ab640 1 -- 172.21.15.39:0/3805043326 >> 172.21.15.39:0/3805043326 conn(0x7f0cac0797b0 msgr2=0x7f0cac079bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:53:29.799 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:29.796+0000 7f0cb46ab640 1 -- 172.21.15.39:0/3805043326 shutdown_connections 2024-09-21T03:53:29.799 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:29.796+0000 7f0cb46ab640 1 -- 172.21.15.39:0/3805043326 wait complete. 2024-09-21T03:53:29.801 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:29.797+0000 7f0cb46ab640 1 Processor -- start 2024-09-21T03:53:29.801 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:29.797+0000 7f0cb46ab640 1 -- start start 2024-09-21T03:53:29.801 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:29.797+0000 7f0cb46ab640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0cac104a30 0x7f0cac19a470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:53:29.801 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:29.797+0000 7f0cb46ab640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0cac105380 0x7f0cac19aa10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:53:29.801 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:29.797+0000 7f0cb46ab640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0cac10de60 0x7f0cac19fcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:53:29.801 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:29.797+0000 7f0cb46ab640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f0cac113ac0 con 0x7f0cac10de60 2024-09-21T03:53:29.801 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:29.797+0000 7f0cb46ab640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f0cac113940 con 0x7f0cac104a30 2024-09-21T03:53:29.801 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:29.797+0000 7f0cb46ab640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f0cac113c40 con 0x7f0cac105380 2024-09-21T03:53:29.801 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:29.797+0000 7f0cb2c48640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0cac10de60 0x7f0cac19fcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:53:29.801 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:29.797+0000 7f0cb2c48640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0cac10de60 0x7f0cac19fcb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:54554/0 (socket says 172.21.15.39:54554) 2024-09-21T03:53:29.801 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:29.797+0000 7f0cb2c48640 1 -- 172.21.15.39:0/4239585893 learned_addr learned my addr 172.21.15.39:0/4239585893 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:53:29.802 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:29.797+0000 7f0cb1c46640 1 --2- 172.21.15.39:0/4239585893 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0cac105380 0x7f0cac19aa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:53:29.802 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:29.797+0000 7f0cb2447640 1 --2- 172.21.15.39:0/4239585893 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0cac104a30 0x7f0cac19a470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:53:29.802 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:29.798+0000 7f0cb2c48640 1 -- 172.21.15.39:0/4239585893 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0cac105380 msgr2=0x7f0cac19aa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:53:29.802 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:29.798+0000 7f0cb2c48640 1 --2- 172.21.15.39:0/4239585893 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0cac105380 0x7f0cac19aa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:29.802 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:29.798+0000 7f0cb2c48640 1 -- 172.21.15.39:0/4239585893 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0cac104a30 msgr2=0x7f0cac19a470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:53:29.802 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:29.798+0000 7f0cb2c48640 1 --2- 172.21.15.39:0/4239585893 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0cac104a30 0x7f0cac19a470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:29.802 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:29.798+0000 7f0cb2c48640 1 -- 172.21.15.39:0/4239585893 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0cac1a03e0 con 0x7f0cac10de60 2024-09-21T03:53:29.802 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:29.798+0000 7f0cb2447640 1 --2- 172.21.15.39:0/4239585893 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0cac104a30 0x7f0cac19a470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:53:29.802 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:29.798+0000 7f0cb2c48640 1 --2- 172.21.15.39:0/4239585893 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0cac10de60 0x7f0cac19fcb0 secure :-1 s=READY pgs=1076 cs=0 l=1 rev1=1 crypto rx=0x7f0ca4005cc0 tx=0x7f0ca4031700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:53:29.802 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:29.798+0000 7f0ca37fe640 1 -- 172.21.15.39:0/4239585893 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ca4003b70 con 0x7f0cac10de60 2024-09-21T03:53:29.803 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:29.798+0000 7f0ca37fe640 1 -- 172.21.15.39:0/4239585893 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0ca40318f0 con 0x7f0cac10de60 2024-09-21T03:53:29.803 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:29.798+0000 7f0cb46ab640 1 -- 172.21.15.39:0/4239585893 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0cac1a0670 con 0x7f0cac10de60 2024-09-21T03:53:29.803 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:29.798+0000 7f0ca37fe640 1 -- 172.21.15.39:0/4239585893 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ca402f950 con 0x7f0cac10de60 2024-09-21T03:53:29.803 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:29.798+0000 7f0cb46ab640 1 -- 172.21.15.39:0/4239585893 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0cac1a6e30 con 0x7f0cac10de60 2024-09-21T03:53:29.804 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:29.801+0000 7f0cbac1f740 1 Processor -- start 2024-09-21T03:53:29.804 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:29.801+0000 7f0ca37fe640 1 -- 172.21.15.39:0/4239585893 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 27) ==== 154643+0+0 (secure 0 0 0) 0x7f0ca402fb30 con 0x7f0cac10de60 2024-09-21T03:53:29.804 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:29.801+0000 7f0cbac1f740 1 -- start start 2024-09-21T03:53:29.805 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:29.802+0000 7f0cbac1f740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5560f824c070 0x5560f85f3290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:53:29.805 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:29.802+0000 7f0cbac1f740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5560f82231b0 0x5560f85f37d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:53:29.806 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:29.802+0000 7f0cbac1f740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5560f822b390 0x5560f85f3d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:53:29.806 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:29.802+0000 7f0cbac1f740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x5560f8569ea0 con 0x5560f82231b0 2024-09-21T03:53:29.806 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:29.802+0000 7f0cbac1f740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x5560f85592d0 con 0x5560f824c070 2024-09-21T03:53:29.806 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:29.802+0000 7f0cbac1f740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x5560f855d0e0 con 0x5560f822b390 2024-09-21T03:53:29.806 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:29.802+0000 7f0cb2447640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5560f82231b0 0x5560f85f37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:53:29.806 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:29.802+0000 7f0cb1c46640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5560f824c070 0x5560f85f3290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:53:29.806 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:29.802+0000 7f0cb2447640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5560f82231b0 0x5560f85f37d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:54566/0 (socket says 172.21.15.39:54566) 2024-09-21T03:53:29.806 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:29.802+0000 7f0cb2447640 1 -- 172.21.15.39:0/1311901297 learned_addr learned my addr 172.21.15.39:0/1311901297 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:53:29.806 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:29.802+0000 7f0cb1c46640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5560f824c070 0x5560f85f3290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.39:60526/0 (socket says 172.21.15.39:60526) 2024-09-21T03:53:29.806 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:29.802+0000 7f0cb1c46640 1 --2- 172.21.15.39:0/1311901297 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5560f822b390 0x5560f85f3d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:53:29.806 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:29.802+0000 7f0ca37fe640 1 --2- 172.21.15.39:0/4239585893 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f0c980b6680 0x7f0c980b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:53:29.806 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:29.802+0000 7f0cb1c46640 1 -- 172.21.15.39:0/1311901297 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5560f822b390 msgr2=0x5560f85f3d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:53:29.806 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:29.802+0000 7f0cb1c46640 1 --2- 172.21.15.39:0/1311901297 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5560f822b390 0x5560f85f3d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:29.807 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:29.802+0000 7f0cb1c46640 1 -- 172.21.15.39:0/1311901297 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5560f82231b0 msgr2=0x5560f85f37d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:53:29.807 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:29.802+0000 7f0cb1c46640 1 --2- 172.21.15.39:0/1311901297 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5560f82231b0 0x5560f85f37d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:29.807 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:29.802+0000 7f0cb1c46640 1 -- 172.21.15.39:0/1311901297 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5560f82b7650 con 0x5560f824c070 2024-09-21T03:53:29.807 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:29.803+0000 7f0ca37fe640 1 -- 172.21.15.39:0/4239585893 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8528+0+0 (secure 0 0 0) 0x7f0ca4108860 con 0x7f0cac10de60 2024-09-21T03:53:29.807 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:29.803+0000 7f0cb2c48640 1 --2- 172.21.15.39:0/4239585893 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f0c980b6680 0x7f0c980b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:53:29.807 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:29.803+0000 7f0cb1c46640 1 --2- 172.21.15.39:0/1311901297 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5560f824c070 0x5560f85f3290 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f0c9c009730 tx=0x7f0c9c0096c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=adb395d09b95bf22 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:53:29.809 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:29.803+0000 7f0ca0ff9640 1 -- 172.21.15.39:0/1311901297 <== mon.2 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c9c004050 con 0x5560f824c070 2024-09-21T03:53:29.809 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:29.803+0000 7f0ca0ff9640 1 -- 172.21.15.39:0/1311901297 <== mon.2 v2:172.21.15.150:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0c9c0054d0 con 0x5560f824c070 2024-09-21T03:53:29.809 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:29.803+0000 7f0ca0ff9640 1 -- 172.21.15.39:0/1311901297 <== mon.2 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c9c004c60 con 0x5560f824c070 2024-09-21T03:53:29.809 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:29.804+0000 7f0cbac1f740 1 -- 172.21.15.39:0/1311901297 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5560f824c070 msgr2=0x5560f85f3290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:53:29.809 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:29.804+0000 7f0cb2c48640 1 --2- 172.21.15.39:0/4239585893 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f0c980b6680 0x7f0c980b8b90 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f0ca4005e30 tx=0x7f0ca40972e0 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:53:29.809 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:29.804+0000 7f0cbac1f740 1 --2- 172.21.15.39:0/1311901297 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5560f824c070 0x5560f85f3290 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f0c9c009730 tx=0x7f0c9c0096c0 comp rx=0 tx=0).stop 2024-09-21T03:53:29.809 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:29.804+0000 7f0cbac1f740 1 -- 172.21.15.39:0/1311901297 shutdown_connections 2024-09-21T03:53:29.809 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:29.804+0000 7f0cbac1f740 1 --2- 172.21.15.39:0/1311901297 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5560f822b390 0x5560f85f3d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:29.810 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:29.804+0000 7f0cbac1f740 1 --2- 172.21.15.39:0/1311901297 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5560f82231b0 0x5560f85f37d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:29.810 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:29.804+0000 7f0cbac1f740 1 --2- 172.21.15.39:0/1311901297 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5560f824c070 0x5560f85f3290 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:29.810 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:29.804+0000 7f0cbac1f740 1 -- 172.21.15.39:0/1311901297 >> 172.21.15.39:0/1311901297 conn(0x5560f8240460 msgr2=0x5560f85f1180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:53:29.810 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:29.805+0000 7f0cbac1f740 1 -- 172.21.15.39:0/1311901297 shutdown_connections 2024-09-21T03:53:29.810 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:29.805+0000 7f0cbac1f740 1 -- 172.21.15.39:0/1311901297 wait complete. 2024-09-21T03:53:29.810 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:29.805+0000 7f0cbac1f740 1 -- start start 2024-09-21T03:53:29.810 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:29.805+0000 7f0cbac1f740 1 Processor -- start 2024-09-21T03:53:29.810 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:29.805+0000 7f0cbac1f740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5560f82231b0 0x5560f8602d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:53:29.810 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:29.805+0000 7f0cbac1f740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5560f822b390 0x5560f8603280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:53:29.810 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:29.805+0000 7f0cbac1f740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5560f824c070 0x5560f860f070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:53:29.810 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:29.805+0000 7f0cbac1f740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x5560f83ba700 con 0x5560f824c070 2024-09-21T03:53:29.810 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:29.805+0000 7f0cbac1f740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x5560f84f4160 con 0x5560f822b390 2024-09-21T03:53:29.810 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:29.805+0000 7f0cbac1f740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x5560f8605d10 con 0x5560f82231b0 2024-09-21T03:53:29.810 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:29.806+0000 7f0cb2447640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5560f822b390 0x5560f8603280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:53:29.811 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:29.806+0000 7f0cb2447640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5560f822b390 0x5560f8603280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.39:60532/0 (socket says 172.21.15.39:60532) 2024-09-21T03:53:29.811 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:29.806+0000 7f0cb2447640 1 -- 172.21.15.39:0/4168160563 learned_addr learned my addr 172.21.15.39:0/4168160563 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:53:29.811 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:29.806+0000 7f0cbac1f740 10 client.0 _ll_register_callbacks cb 0x5560f850d9e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:53:29.811 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:29.806+0000 7f0cb1c46640 1 --2- 172.21.15.39:0/4168160563 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5560f824c070 0x5560f860f070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:53:29.811 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:29.806+0000 7f0cb1c46640 1 --2- 172.21.15.39:0/4168160563 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5560f82231b0 0x5560f8602d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:53:29.811 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:29.806+0000 7f0cb2447640 1 -- 172.21.15.39:0/4168160563 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5560f82231b0 msgr2=0x5560f8602d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:53:29.811 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:29.806+0000 7f0cb2447640 1 --2- 172.21.15.39:0/4168160563 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5560f82231b0 0x5560f8602d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:29.811 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:29.806+0000 7f0cb2447640 1 -- 172.21.15.39:0/4168160563 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5560f824c070 msgr2=0x5560f860f070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:53:29.811 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:29.806+0000 7f0cb2447640 1 --2- 172.21.15.39:0/4168160563 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5560f824c070 0x5560f860f070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:29.811 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:29.806+0000 7f0cb2447640 1 -- 172.21.15.39:0/4168160563 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5560f8521c90 con 0x5560f822b390 2024-09-21T03:53:29.811 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:29.806+0000 7f0cb2447640 1 --2- 172.21.15.39:0/4168160563 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5560f822b390 0x5560f8603280 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f0ca8012d20 tx=0x7f0ca800f790 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f9f0c5f9f4ebfac7 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:53:29.811 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:29.806+0000 7f0c95ffb640 10 client.0 ms_handle_connect on v2:172.21.15.150:3300/0 2024-09-21T03:53:29.813 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:29.806+0000 7f0c95ffb640 1 -- 172.21.15.39:0/4168160563 <== mon.2 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ca8010430 con 0x5560f822b390 2024-09-21T03:53:29.813 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:29.807+0000 7f0c95ffb640 1 -- 172.21.15.39:0/4168160563 <== mon.2 v2:172.21.15.150:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8528+0+0 (secure 0 0 0) 0x7f0ca80105d0 con 0x5560f822b390 2024-09-21T03:53:29.814 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:29.807+0000 7f0cbac1f740 1 -- 172.21.15.39:0/4168160563 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0c9c0054d0 con 0x5560f822b390 2024-09-21T03:53:29.814 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:29.807+0000 7f0c95ffb640 1 -- 172.21.15.39:0/4168160563 <== mon.2 v2:172.21.15.150:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f0ca80107d0 con 0x5560f822b390 2024-09-21T03:53:29.814 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:29.807+0000 7f0cbac1f740 10 client.26759 fetch_fsmap learned FSMap version 29 2024-09-21T03:53:29.814 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:29.807+0000 7f0cbac1f740 1 -- 172.21.15.39:0/4168160563 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5560f8611a90 con 0x5560f822b390 2024-09-21T03:53:29.814 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:29.807+0000 7f0c95ffb640 1 -- 172.21.15.39:0/4168160563 <== mon.2 v2:172.21.15.150:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0ca8010e20 con 0x5560f822b390 2024-09-21T03:53:29.814 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:29.807+0000 7f0c95ffb640 1 -- 172.21.15.39:0/4168160563 <== mon.2 v2:172.21.15.150:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f0ca801c890 con 0x5560f822b390 2024-09-21T03:53:29.814 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:29.807+0000 7f0cbac1f740 10 client.26759 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T03:53:29.814 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:29.807+0000 7f0cbac1f740 10 client.26759 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T03:53:29.814 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:29.807+0000 7f0cbac1f740 20 client.26759 populate_metadata read hostname 'smithi039' 2024-09-21T03:53:29.814 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:29.807+0000 7f0cbac1f740 1 --2- 172.21.15.39:0/4168160563 >> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] conn(0x5560f820a9f0 0x5560f86150a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:53:29.814 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:29.807+0000 7f0cbac1f740 4 client.26759 mds_command: new command op to 24695 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:53:29.814 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:29.807+0000 7f0cbac1f740 1 -- 172.21.15.39:0/4168160563 --> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f0c9c0054d0 con 0x5560f820a9f0 2024-09-21T03:53:29.814 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:29.808+0000 7f0cb1c46640 1 --2- 172.21.15.39:0/4168160563 >> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] conn(0x5560f820a9f0 0x5560f86150a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:53:29.814 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:29.808+0000 7f0cb1c46640 1 --2- 172.21.15.39:0/4168160563 >> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] conn(0x5560f820a9f0 0x5560f86150a0 crc :-1 s=READY pgs=127 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=48e79918964b1bf6 server_cookie=f362b2e0dc14a3d4 in_seq=0 out_seq=0 2024-09-21T03:53:29.815 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:29.808+0000 7f0c95ffb640 10 client.26759 ms_handle_connect on v2:172.21.15.150:6838/2949386907 2024-09-21T03:53:29.815 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:29.809+0000 7f0c95ffb640 1 -- 172.21.15.39:0/4168160563 <== mds.0 v2:172.21.15.150:6838/2949386907 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x7f0c9c0054d0 con 0x5560f820a9f0 2024-09-21T03:53:29.815 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:29.809+0000 7f0c95ffb640 10 client.26759 handle_command_reply: tid=0 2024-09-21T03:53:29.815 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:29.809+0000 7f0cbac1f740 1 client.26759 shutdown 2024-09-21T03:53:29.815 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:29.809+0000 7f0cbac1f740 1 -- 172.21.15.39:0/4168160563 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5560f822b390 msgr2=0x5560f8603280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:53:29.815 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:29.809+0000 7f0cbac1f740 1 --2- 172.21.15.39:0/4168160563 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5560f822b390 0x5560f8603280 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f0ca8012d20 tx=0x7f0ca800f790 comp rx=0 tx=0).stop 2024-09-21T03:53:29.815 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:29.809+0000 7f0cbac1f740 1 -- 172.21.15.39:0/4168160563 shutdown_connections 2024-09-21T03:53:29.815 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:29.809+0000 7f0cbac1f740 1 --2- 172.21.15.39:0/4168160563 >> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] conn(0x5560f820a9f0 0x5560f86150a0 crc :-1 s=READY pgs=127 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:29.815 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:29.809+0000 7f0c95ffb640 0 client.26759 ms_handle_reset on v2:172.21.15.150:6838/2949386907 2024-09-21T03:53:29.815 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:29.809+0000 7f0cbac1f740 1 --2- 172.21.15.39:0/4168160563 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5560f824c070 0x5560f860f070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:29.815 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:29.809+0000 7f0cbac1f740 1 --2- 172.21.15.39:0/4168160563 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5560f822b390 0x5560f8603280 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:29.815 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:29.809+0000 7f0cbac1f740 1 --2- 172.21.15.39:0/4168160563 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5560f82231b0 0x5560f8602d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:29.815 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:29.809+0000 7f0cbac1f740 1 -- 172.21.15.39:0/4168160563 >> 172.21.15.39:0/4168160563 conn(0x5560f8240460 msgr2=0x5560f85ef720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:53:29.815 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:29.810+0000 7f0cbac1f740 1 -- 172.21.15.39:0/4168160563 shutdown_connections 2024-09-21T03:53:29.816 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:29.810+0000 7f0cbac1f740 1 -- 172.21.15.39:0/4168160563 wait complete. 2024-09-21T03:53:29.816 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:29.810+0000 7f0cbac1f740 20 client.26759 trim_cache size 0 max 16384 2024-09-21T03:53:29.826 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:29.823+0000 7f0cbac1f740 1 Processor -- start 2024-09-21T03:53:29.827 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:29.823+0000 7f0cbac1f740 1 -- start start 2024-09-21T03:53:29.827 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:29.824+0000 7f0cbac1f740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5560f820a9f0 0x5560f8604c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:53:29.827 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:29.824+0000 7f0cbac1f740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5560f82231b0 0x5560f85f49d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:53:29.827 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:29.824+0000 7f0cbac1f740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5560f822b390 0x5560f860ef70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:53:29.827 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:29.824+0000 7f0cbac1f740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f0cac1ad550 con 0x5560f820a9f0 2024-09-21T03:53:29.827 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:29.824+0000 7f0cbac1f740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f0cac1af090 con 0x5560f82231b0 2024-09-21T03:53:29.827 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:29.824+0000 7f0cbac1f740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f0cac1adc20 con 0x5560f822b390 2024-09-21T03:53:29.827 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:29.824+0000 7f0cb1c46640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5560f820a9f0 0x5560f8604c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:53:29.827 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:29.824+0000 7f0cb1c46640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5560f820a9f0 0x5560f8604c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:54588/0 (socket says 172.21.15.39:54588) 2024-09-21T03:53:29.827 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:29.824+0000 7f0cb1c46640 1 -- 172.21.15.39:0/1883362137 learned_addr learned my addr 172.21.15.39:0/1883362137 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:53:29.828 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:29.824+0000 7f0cb1c46640 1 -- 172.21.15.39:0/1883362137 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5560f822b390 msgr2=0x5560f860ef70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:53:29.828 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:29.824+0000 7f0cb1c46640 1 --2- 172.21.15.39:0/1883362137 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5560f822b390 0x5560f860ef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:29.828 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:29.824+0000 7f0cb1c46640 1 -- 172.21.15.39:0/1883362137 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5560f82231b0 msgr2=0x5560f85f49d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:53:29.828 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:29.824+0000 7f0cb1c46640 1 --2- 172.21.15.39:0/1883362137 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5560f82231b0 0x5560f85f49d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:29.828 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:29.824+0000 7f0cb1c46640 1 -- 172.21.15.39:0/1883362137 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5560f85eb440 con 0x5560f820a9f0 2024-09-21T03:53:29.828 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:29.824+0000 7f0cb1c46640 1 --2- 172.21.15.39:0/1883362137 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5560f820a9f0 0x5560f8604c20 secure :-1 s=READY pgs=1077 cs=0 l=1 rev1=1 crypto rx=0x7f0c9c011750 tx=0x7f0c9c0096c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=79e5bea5ccf9d344 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:53:29.828 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:29.825+0000 7f0c97fff640 1 -- 172.21.15.39:0/1883362137 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c9c0099f0 con 0x5560f820a9f0 2024-09-21T03:53:29.828 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:29.825+0000 7f0c97fff640 1 -- 172.21.15.39:0/1883362137 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0c9c00fe80 con 0x5560f820a9f0 2024-09-21T03:53:29.828 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:29.825+0000 7f0c97fff640 1 -- 172.21.15.39:0/1883362137 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c9c02ea40 con 0x5560f820a9f0 2024-09-21T03:53:29.829 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:29.826+0000 7f0cbac1f740 1 -- 172.21.15.39:0/1883362137 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5560f820a9f0 msgr2=0x5560f8604c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:53:29.829 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:29.826+0000 7f0cbac1f740 1 --2- 172.21.15.39:0/1883362137 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5560f820a9f0 0x5560f8604c20 secure :-1 s=READY pgs=1077 cs=0 l=1 rev1=1 crypto rx=0x7f0c9c011750 tx=0x7f0c9c0096c0 comp rx=0 tx=0).stop 2024-09-21T03:53:29.829 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:29.826+0000 7f0cbac1f740 1 -- 172.21.15.39:0/1883362137 shutdown_connections 2024-09-21T03:53:29.830 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:29.826+0000 7f0cbac1f740 1 --2- 172.21.15.39:0/1883362137 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5560f822b390 0x5560f860ef70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:29.830 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:29.826+0000 7f0cbac1f740 1 --2- 172.21.15.39:0/1883362137 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5560f82231b0 0x5560f85f49d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:29.830 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:29.826+0000 7f0cbac1f740 1 --2- 172.21.15.39:0/1883362137 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5560f820a9f0 0x5560f8604c20 unknown :-1 s=CLOSED pgs=1077 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:29.830 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:29.826+0000 7f0cbac1f740 1 -- 172.21.15.39:0/1883362137 >> 172.21.15.39:0/1883362137 conn(0x5560f8240460 msgr2=0x5560f85f42c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:53:29.830 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:29.826+0000 7f0cbac1f740 1 -- 172.21.15.39:0/1883362137 shutdown_connections 2024-09-21T03:53:29.830 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:29.826+0000 7f0cbac1f740 1 -- 172.21.15.39:0/1883362137 wait complete. 2024-09-21T03:53:29.830 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:29.827+0000 7f0cbac1f740 1 -- start start 2024-09-21T03:53:29.830 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:29.827+0000 7f0cbac1f740 1 Processor -- start 2024-09-21T03:53:29.836 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:29.827+0000 7f0cbac1f740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5560f820a9f0 0x5560f8607080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:53:29.836 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:29.827+0000 7f0cbac1f740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5560f82231b0 0x5560f86075c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:53:29.836 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:29.827+0000 7f0cbac1f740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5560f822b390 0x5560f8607b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:53:29.836 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:29.827+0000 7f0cbac1f740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f0cac1ae9c0 con 0x5560f822b390 2024-09-21T03:53:29.836 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:29.827+0000 7f0cbac1f740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f0cac1ae2f0 con 0x5560f82231b0 2024-09-21T03:53:29.836 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:29.827+0000 7f0cbac1f740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f0cac1ab530 con 0x5560f820a9f0 2024-09-21T03:53:29.836 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:29.827+0000 7f0cbac1f740 10 client.0 _ll_register_callbacks cb 0x7f0ca8009d40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:53:29.836 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:29.827+0000 7f0cb1c46640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5560f822b390 0x5560f8607b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:53:29.836 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:29.827+0000 7f0cb1c46640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5560f822b390 0x5560f8607b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:54596/0 (socket says 172.21.15.39:54596) 2024-09-21T03:53:29.836 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:29.827+0000 7f0cb1c46640 1 -- 172.21.15.39:0/3922595069 learned_addr learned my addr 172.21.15.39:0/3922595069 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:53:29.836 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:29.827+0000 7f0cb1c46640 1 --2- 172.21.15.39:0/3922595069 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5560f820a9f0 0x5560f8607080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:53:29.836 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:29.828+0000 7f0cb2447640 1 --2- 172.21.15.39:0/3922595069 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5560f82231b0 0x5560f86075c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:53:29.836 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:29.828+0000 7f0cb1c46640 1 -- 172.21.15.39:0/3922595069 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5560f820a9f0 msgr2=0x5560f8607080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:53:29.837 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:29.828+0000 7f0cb1c46640 1 --2- 172.21.15.39:0/3922595069 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5560f820a9f0 0x5560f8607080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:29.837 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:29.828+0000 7f0cb1c46640 1 -- 172.21.15.39:0/3922595069 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5560f82231b0 msgr2=0x5560f86075c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:53:29.837 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:29.828+0000 7f0cb1c46640 1 --2- 172.21.15.39:0/3922595069 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5560f82231b0 0x5560f86075c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:29.837 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:29.828+0000 7f0cb1c46640 1 -- 172.21.15.39:0/3922595069 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5560f867eb30 con 0x5560f822b390 2024-09-21T03:53:29.837 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:29.828+0000 7f0cb1c46640 1 --2- 172.21.15.39:0/3922595069 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5560f822b390 0x5560f8607b00 secure :-1 s=READY pgs=1078 cs=0 l=1 rev1=1 crypto rx=0x7f0c9c02e920 tx=0x7f0c9c004840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ce148dbe1686557 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:53:29.837 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:29.828+0000 7f0c95ffb640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-09-21T03:53:29.837 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:29.828+0000 7f0c95ffb640 1 -- 172.21.15.39:0/3922595069 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c9c00f9d0 con 0x5560f822b390 2024-09-21T03:53:29.837 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:29.828+0000 7f0c95ffb640 1 -- 172.21.15.39:0/3922595069 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8528+0+0 (secure 0 0 0) 0x7f0c9c00fb70 con 0x5560f822b390 2024-09-21T03:53:29.837 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:29.828+0000 7f0cbac1f740 1 -- 172.21.15.39:0/3922595069 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5560f856bdf0 con 0x5560f822b390 2024-09-21T03:53:29.837 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:29.828+0000 7f0c95ffb640 1 -- 172.21.15.39:0/3922595069 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f0c9c02e9c0 con 0x5560f822b390 2024-09-21T03:53:29.837 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:29.828+0000 7f0cbac1f740 10 client.17523 fetch_fsmap learned FSMap version 29 2024-09-21T03:53:29.837 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:29.828+0000 7f0cbac1f740 1 -- 172.21.15.39:0/3922595069 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5560f867fd90 con 0x5560f822b390 2024-09-21T03:53:29.837 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:29.829+0000 7f0c95ffb640 1 -- 172.21.15.39:0/3922595069 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x5560f856bdf0 con 0x5560f822b390 2024-09-21T03:53:29.838 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:29.829+0000 7f0c95ffb640 1 -- 172.21.15.39:0/3922595069 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f0c9c035a20 con 0x5560f822b390 2024-09-21T03:53:29.838 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:29.829+0000 7f0cbac1f740 10 client.17523 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T03:53:29.838 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:29.829+0000 7f0cbac1f740 10 client.17523 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T03:53:29.838 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:29.829+0000 7f0cbac1f740 20 client.17523 populate_metadata read hostname 'smithi039' 2024-09-21T03:53:29.838 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:29.829+0000 7f0cbac1f740 1 --2- 172.21.15.39:0/3922595069 >> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] conn(0x5560f824c070 0x5560f8682270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:53:29.838 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:29.829+0000 7f0cbac1f740 4 client.17523 mds_command: new command op to 24695 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T03:53:29.838 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:29.829+0000 7f0cbac1f740 1 -- 172.21.15.39:0/3922595069 --> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5560f85678a0 con 0x5560f824c070 2024-09-21T03:53:29.838 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:29.829+0000 7f0cb2447640 1 --2- 172.21.15.39:0/3922595069 >> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] conn(0x5560f824c070 0x5560f8682270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:53:29.838 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:29.830+0000 7f0c95ffb640 10 client.17523 ms_handle_connect on v2:172.21.15.150:6838/2949386907 2024-09-21T03:53:29.838 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:29.830+0000 7f0cb2447640 1 --2- 172.21.15.39:0/3922595069 >> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] conn(0x5560f824c070 0x5560f8682270 crc :-1 s=READY pgs=128 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5014f04c5d05f45b server_cookie=69894952c2361dd8 in_seq=0 out_seq=0 2024-09-21T03:53:29.838 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:29.830+0000 7f0c95ffb640 1 -- 172.21.15.39:0/3922595069 <== mds.0 v2:172.21.15.150:6838/2949386907 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5560f85678a0 con 0x5560f824c070 2024-09-21T03:53:29.838 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:29.830+0000 7f0c95ffb640 10 client.17523 handle_command_reply: tid=0 2024-09-21T03:53:29.838 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:29.830+0000 7f0cbac1f740 1 client.17523 shutdown 2024-09-21T03:53:29.839 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:29.830+0000 7f0cbac1f740 1 -- 172.21.15.39:0/3922595069 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5560f822b390 msgr2=0x5560f8607b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:53:29.839 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:29.830+0000 7f0cbac1f740 1 --2- 172.21.15.39:0/3922595069 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5560f822b390 0x5560f8607b00 secure :-1 s=READY pgs=1078 cs=0 l=1 rev1=1 crypto rx=0x7f0c9c02e920 tx=0x7f0c9c004840 comp rx=0 tx=0).stop 2024-09-21T03:53:29.839 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:29.830+0000 7f0cbac1f740 1 -- 172.21.15.39:0/3922595069 shutdown_connections 2024-09-21T03:53:29.839 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:29.830+0000 7f0cbac1f740 1 --2- 172.21.15.39:0/3922595069 >> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] conn(0x5560f824c070 0x5560f8682270 crc :-1 s=READY pgs=128 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:29.839 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:29.830+0000 7f0cbac1f740 1 --2- 172.21.15.39:0/3922595069 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5560f822b390 0x5560f8607b00 unknown :-1 s=CLOSED pgs=1078 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:29.839 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:29.830+0000 7f0cbac1f740 1 --2- 172.21.15.39:0/3922595069 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5560f82231b0 0x5560f86075c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:29.839 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:29.830+0000 7f0cbac1f740 1 --2- 172.21.15.39:0/3922595069 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5560f820a9f0 0x5560f8607080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:29.839 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:29.830+0000 7f0cbac1f740 1 -- 172.21.15.39:0/3922595069 >> 172.21.15.39:0/3922595069 conn(0x5560f8240460 msgr2=0x5560f86049c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:53:29.839 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:29.830+0000 7f0c95ffb640 0 client.17523 ms_handle_reset on v2:172.21.15.150:6838/2949386907 2024-09-21T03:53:29.839 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:29.830+0000 7f0cbac1f740 1 -- 172.21.15.39:0/3922595069 shutdown_connections 2024-09-21T03:53:29.839 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:29.830+0000 7f0cbac1f740 1 -- 172.21.15.39:0/3922595069 wait complete. 2024-09-21T03:53:29.839 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:29.830+0000 7f0cbac1f740 20 client.17523 trim_cache size 0 max 16384 2024-09-21T03:53:29.840 INFO:teuthology.orchestra.run.smithi039.stdout:[] 2024-09-21T03:53:29.840 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:29.831+0000 7f0cb46ab640 1 -- 172.21.15.39:0/4239585893 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f0c980b6680 msgr2=0x7f0c980b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:53:29.840 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:29.831+0000 7f0cb46ab640 1 --2- 172.21.15.39:0/4239585893 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f0c980b6680 0x7f0c980b8b90 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f0ca4005e30 tx=0x7f0ca40972e0 comp rx=0 tx=0).stop 2024-09-21T03:53:29.840 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:29.831+0000 7f0cb46ab640 1 -- 172.21.15.39:0/4239585893 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0cac10de60 msgr2=0x7f0cac19fcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:53:29.840 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:29.831+0000 7f0cb46ab640 1 --2- 172.21.15.39:0/4239585893 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0cac10de60 0x7f0cac19fcb0 secure :-1 s=READY pgs=1076 cs=0 l=1 rev1=1 crypto rx=0x7f0ca4005cc0 tx=0x7f0ca4031700 comp rx=0 tx=0).stop 2024-09-21T03:53:29.841 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:29.831+0000 7f0cb46ab640 1 -- 172.21.15.39:0/4239585893 shutdown_connections 2024-09-21T03:53:29.841 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:29.831+0000 7f0cb46ab640 1 --2- 172.21.15.39:0/4239585893 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f0c980b6680 0x7f0c980b8b90 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:29.841 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:29.831+0000 7f0cb46ab640 1 --2- 172.21.15.39:0/4239585893 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0cac10de60 0x7f0cac19fcb0 unknown :-1 s=CLOSED pgs=1076 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:29.841 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:29.831+0000 7f0cb46ab640 1 --2- 172.21.15.39:0/4239585893 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0cac105380 0x7f0cac19aa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:29.841 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:29.831+0000 7f0cb46ab640 1 --2- 172.21.15.39:0/4239585893 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0cac104a30 0x7f0cac19a470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:29.841 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:29.831+0000 7f0cb46ab640 1 -- 172.21.15.39:0/4239585893 >> 172.21.15.39:0/4239585893 conn(0x7f0cac0797b0 msgr2=0x7f0cac0ff550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:53:29.841 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:29.831+0000 7f0cb46ab640 1 -- 172.21.15.39:0/4239585893 shutdown_connections 2024-09-21T03:53:29.841 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:29.831+0000 7f0cb46ab640 1 -- 172.21.15.39:0/4239585893 wait complete. 2024-09-21T03:53:30.084 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:53:29 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:53:30.217 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:53:29 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:53:30.224 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:53:29 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:53:31.218 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:53:30 smithi161 ceph-mon[34533]: pgmap v551: 97 pgs: 97 active+clean; 10 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 10 op/s 2024-09-21T03:53:31.218 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:53:30 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-09-21T03:53:31.218 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:53:30 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-21T03:53:31.219 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:53:30 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-09-21T03:53:31.219 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:53:30 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-09-21T03:53:31.219 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:53:30 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"} : dispatch 2024-09-21T03:53:31.219 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:53:30 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-09-21T03:53:31.219 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:53:30 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"} : dispatch 2024-09-21T03:53:31.219 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:53:30 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"} : dispatch 2024-09-21T03:53:31.219 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:53:30 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"} : dispatch 2024-09-21T03:53:31.219 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:53:30 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"} : dispatch 2024-09-21T03:53:31.219 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:53:30 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"} : dispatch 2024-09-21T03:53:31.219 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:53:30 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"} : dispatch 2024-09-21T03:53:31.219 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:53:30 smithi161 ceph-mon[34533]: Adjusting osd_memory_target on smithi161 to 176.0M 2024-09-21T03:53:31.219 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:53:30 smithi161 ceph-mon[34533]: Adjusting osd_memory_target on smithi039 to 176.0M 2024-09-21T03:53:31.219 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:53:30 smithi161 ceph-mon[34533]: Adjusting osd_memory_target on smithi150 to 176.0M 2024-09-21T03:53:31.219 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:53:30 smithi161 ceph-mon[34533]: Unable to set osd_memory_target on smithi161 to 184592486: error parsing value: Value '184592486' is below minimum 939524096 2024-09-21T03:53:31.219 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:53:30 smithi161 ceph-mon[34533]: Unable to set osd_memory_target on smithi039 to 184583884: error parsing value: Value '184583884' is below minimum 939524096 2024-09-21T03:53:31.220 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:53:30 smithi161 ceph-mon[34533]: Unable to set osd_memory_target on smithi150 to 184590336: error parsing value: Value '184590336' is below minimum 939524096 2024-09-21T03:53:31.220 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:53:30 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:53:31.220 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:53:30 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:53:31.220 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:53:30 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:53:31.225 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:53:30 smithi150 ceph-mon[34289]: pgmap v551: 97 pgs: 97 active+clean; 10 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 10 op/s 2024-09-21T03:53:31.225 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:53:30 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-09-21T03:53:31.225 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:53:30 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-21T03:53:31.225 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:53:30 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-09-21T03:53:31.225 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:53:30 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-09-21T03:53:31.225 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:53:30 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"} : dispatch 2024-09-21T03:53:31.225 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:53:30 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-09-21T03:53:31.225 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:53:30 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"} : dispatch 2024-09-21T03:53:31.225 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:53:30 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"} : dispatch 2024-09-21T03:53:31.225 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:53:30 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"} : dispatch 2024-09-21T03:53:31.226 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:53:30 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"} : dispatch 2024-09-21T03:53:31.226 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:53:30 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"} : dispatch 2024-09-21T03:53:31.226 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:53:30 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"} : dispatch 2024-09-21T03:53:31.226 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:53:30 smithi150 ceph-mon[34289]: Adjusting osd_memory_target on smithi161 to 176.0M 2024-09-21T03:53:31.226 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:53:30 smithi150 ceph-mon[34289]: Adjusting osd_memory_target on smithi039 to 176.0M 2024-09-21T03:53:31.226 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:53:30 smithi150 ceph-mon[34289]: Adjusting osd_memory_target on smithi150 to 176.0M 2024-09-21T03:53:31.226 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:53:30 smithi150 ceph-mon[34289]: Unable to set osd_memory_target on smithi161 to 184592486: error parsing value: Value '184592486' is below minimum 939524096 2024-09-21T03:53:31.226 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:53:30 smithi150 ceph-mon[34289]: Unable to set osd_memory_target on smithi039 to 184583884: error parsing value: Value '184583884' is below minimum 939524096 2024-09-21T03:53:31.226 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:53:30 smithi150 ceph-mon[34289]: Unable to set osd_memory_target on smithi150 to 184590336: error parsing value: Value '184590336' is below minimum 939524096 2024-09-21T03:53:31.226 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:53:30 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:53:31.226 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:53:30 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:53:31.226 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:53:30 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:53:31.334 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:53:30 smithi039 ceph-mon[31211]: pgmap v551: 97 pgs: 97 active+clean; 10 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 10 op/s 2024-09-21T03:53:31.334 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:53:30 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-09-21T03:53:31.334 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:53:30 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-21T03:53:31.334 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:53:30 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-09-21T03:53:31.334 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:53:30 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-09-21T03:53:31.335 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:53:30 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"} : dispatch 2024-09-21T03:53:31.335 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:53:30 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-09-21T03:53:31.335 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:53:30 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"} : dispatch 2024-09-21T03:53:31.335 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:53:30 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"} : dispatch 2024-09-21T03:53:31.335 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:53:30 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"} : dispatch 2024-09-21T03:53:31.335 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:53:30 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"} : dispatch 2024-09-21T03:53:31.335 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:53:30 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"} : dispatch 2024-09-21T03:53:31.335 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:53:30 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"} : dispatch 2024-09-21T03:53:31.335 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:53:30 smithi039 ceph-mon[31211]: Adjusting osd_memory_target on smithi161 to 176.0M 2024-09-21T03:53:31.335 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:53:30 smithi039 ceph-mon[31211]: Adjusting osd_memory_target on smithi039 to 176.0M 2024-09-21T03:53:31.335 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:53:30 smithi039 ceph-mon[31211]: Adjusting osd_memory_target on smithi150 to 176.0M 2024-09-21T03:53:31.335 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:53:30 smithi039 ceph-mon[31211]: Unable to set osd_memory_target on smithi161 to 184592486: error parsing value: Value '184592486' is below minimum 939524096 2024-09-21T03:53:31.335 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:53:30 smithi039 ceph-mon[31211]: Unable to set osd_memory_target on smithi039 to 184583884: error parsing value: Value '184583884' is below minimum 939524096 2024-09-21T03:53:31.335 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:53:30 smithi039 ceph-mon[31211]: Unable to set osd_memory_target on smithi150 to 184590336: error parsing value: Value '184590336' is below minimum 939524096 2024-09-21T03:53:31.335 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:53:30 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:53:31.336 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:53:30 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:53:31.336 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:53:30 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:53:31.535 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-21T03:53:31.535 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 51116d5c-77ca-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-21T03:53:31.809 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/51116d5c-77ca-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:53:32.330 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:32.327+0000 7f6c49371640 1 -- 172.21.15.39:0/2656728476 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6c44108d10 msgr2=0x7f6c4410f540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:53:32.330 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:32.327+0000 7f6c49371640 1 --2- 172.21.15.39:0/2656728476 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6c44108d10 0x7f6c4410f540 secure :-1 s=READY pgs=1079 cs=0 l=1 rev1=1 crypto rx=0x7f6c3c009ca0 tx=0x7f6c3c02a8c0 comp rx=0 tx=0).stop 2024-09-21T03:53:32.330 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:32.327+0000 7f6c49371640 1 -- 172.21.15.39:0/2656728476 shutdown_connections 2024-09-21T03:53:32.330 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:32.327+0000 7f6c49371640 1 --2- 172.21.15.39:0/2656728476 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6c44108d10 0x7f6c4410f540 unknown :-1 s=CLOSED pgs=1079 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:32.331 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:32.327+0000 7f6c49371640 1 --2- 172.21.15.39:0/2656728476 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6c441083c0 0x7f6c441087d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:32.331 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:32.327+0000 7f6c49371640 1 --2- 172.21.15.39:0/2656728476 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6c44102a80 0x7f6c44102e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:32.331 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:32.327+0000 7f6c49371640 1 -- 172.21.15.39:0/2656728476 >> 172.21.15.39:0/2656728476 conn(0x7f6c44079740 msgr2=0x7f6c44079b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:53:32.331 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:32.327+0000 7f6c49371640 1 -- 172.21.15.39:0/2656728476 shutdown_connections 2024-09-21T03:53:32.331 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:32.327+0000 7f6c49371640 1 -- 172.21.15.39:0/2656728476 wait complete. 2024-09-21T03:53:32.333 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:32.328+0000 7f6c49371640 1 Processor -- start 2024-09-21T03:53:32.333 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:32.328+0000 7f6c49371640 1 -- start start 2024-09-21T03:53:32.333 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:32.328+0000 7f6c49371640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6c44102a80 0x7f6c4419b2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:53:32.333 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:32.328+0000 7f6c49371640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6c441083c0 0x7f6c4419b830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:53:32.333 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:32.328+0000 7f6c49371640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6c44108d10 0x7f6c4419bd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:53:32.333 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:32.328+0000 7f6c49371640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f6c44112980 con 0x7f6c44108d10 2024-09-21T03:53:32.333 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:32.328+0000 7f6c49371640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f6c44112800 con 0x7f6c441083c0 2024-09-21T03:53:32.333 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:32.328+0000 7f6c49371640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f6c44112b00 con 0x7f6c44102a80 2024-09-21T03:53:32.333 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:32.329+0000 7f6c437fe640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6c44108d10 0x7f6c4419bd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:53:32.333 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:32.329+0000 7f6c437fe640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6c44108d10 0x7f6c4419bd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:54616/0 (socket says 172.21.15.39:54616) 2024-09-21T03:53:32.333 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:32.329+0000 7f6c437fe640 1 -- 172.21.15.39:0/1830349513 learned_addr learned my addr 172.21.15.39:0/1830349513 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:53:32.334 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:32.329+0000 7f6c42ffd640 1 --2- 172.21.15.39:0/1830349513 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6c44102a80 0x7f6c4419b2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:53:32.334 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:32.329+0000 7f6c427fc640 1 --2- 172.21.15.39:0/1830349513 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6c441083c0 0x7f6c4419b830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:53:32.334 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:32.329+0000 7f6c437fe640 1 -- 172.21.15.39:0/1830349513 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6c44102a80 msgr2=0x7f6c4419b2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:53:32.334 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:32.329+0000 7f6c437fe640 1 --2- 172.21.15.39:0/1830349513 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6c44102a80 0x7f6c4419b2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:32.334 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:32.329+0000 7f6c437fe640 1 -- 172.21.15.39:0/1830349513 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6c441083c0 msgr2=0x7f6c4419b830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:53:32.334 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:32.329+0000 7f6c437fe640 1 --2- 172.21.15.39:0/1830349513 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6c441083c0 0x7f6c4419b830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:32.334 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:32.329+0000 7f6c437fe640 1 -- 172.21.15.39:0/1830349513 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6c441a6580 con 0x7f6c44108d10 2024-09-21T03:53:32.334 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:32.329+0000 7f6c437fe640 1 --2- 172.21.15.39:0/1830349513 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6c44108d10 0x7f6c4419bd70 secure :-1 s=READY pgs=1080 cs=0 l=1 rev1=1 crypto rx=0x7f6c3c005cc0 tx=0x7f6c3c02f380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:53:32.334 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:32.329+0000 7f6c2bfff640 1 -- 172.21.15.39:0/1830349513 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c3c003bf0 con 0x7f6c44108d10 2024-09-21T03:53:32.334 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:32.329+0000 7f6c49371640 1 -- 172.21.15.39:0/1830349513 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6c441a6810 con 0x7f6c44108d10 2024-09-21T03:53:32.334 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:32.329+0000 7f6c2bfff640 1 -- 172.21.15.39:0/1830349513 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6c3c003d90 con 0x7f6c44108d10 2024-09-21T03:53:32.334 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:32.329+0000 7f6c2bfff640 1 -- 172.21.15.39:0/1830349513 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c3c02fbc0 con 0x7f6c44108d10 2024-09-21T03:53:32.334 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:32.330+0000 7f6c49371640 1 -- 172.21.15.39:0/1830349513 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6c441a6da0 con 0x7f6c44108d10 2024-09-21T03:53:32.335 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:32.332+0000 7f6c4f8e5740 1 Processor -- start 2024-09-21T03:53:32.335 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:32.332+0000 7f6c2bfff640 1 -- 172.21.15.39:0/1830349513 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 27) ==== 154643+0+0 (secure 0 0 0) 0x7f6c3c02fd60 con 0x7f6c44108d10 2024-09-21T03:53:32.335 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:32.332+0000 7f6c4f8e5740 1 -- start start 2024-09-21T03:53:32.337 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:32.333+0000 7f6c4f8e5740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f7ad857350 0x55f7adbfe430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:53:32.337 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:32.333+0000 7f6c4f8e5740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55f7ad82e490 0x55f7adbfe970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:53:32.337 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:32.333+0000 7f6c4f8e5740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f7ad836670 0x55f7adbfeeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:53:32.337 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:32.333+0000 7f6c4f8e5740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x55f7adb74e60 con 0x55f7ad82e490 2024-09-21T03:53:32.337 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:32.333+0000 7f6c4f8e5740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x55f7adb64290 con 0x55f7ad836670 2024-09-21T03:53:32.337 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:32.333+0000 7f6c4f8e5740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55f7adb680a0 con 0x55f7ad857350 2024-09-21T03:53:32.337 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:32.333+0000 7f6c42ffd640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55f7ad82e490 0x55f7adbfe970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:53:32.337 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:32.333+0000 7f6c42ffd640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55f7ad82e490 0x55f7adbfe970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:54628/0 (socket says 172.21.15.39:54628) 2024-09-21T03:53:32.337 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:32.333+0000 7f6c42ffd640 1 -- 172.21.15.39:0/1510243577 learned_addr learned my addr 172.21.15.39:0/1510243577 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:53:32.338 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:32.333+0000 7f6c427fc640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f7ad857350 0x55f7adbfe430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:53:32.338 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:32.333+0000 7f6c427fc640 1 --2- 172.21.15.39:0/1510243577 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f7ad836670 0x55f7adbfeeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:53:32.338 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:32.333+0000 7f6c42ffd640 1 -- 172.21.15.39:0/1510243577 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f7ad857350 msgr2=0x55f7adbfe430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:53:32.338 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:32.333+0000 7f6c42ffd640 1 --2- 172.21.15.39:0/1510243577 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f7ad857350 0x55f7adbfe430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:32.338 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:32.333+0000 7f6c42ffd640 1 -- 172.21.15.39:0/1510243577 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f7ad836670 msgr2=0x55f7adbfeeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:53:32.338 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:32.333+0000 7f6c42ffd640 1 --2- 172.21.15.39:0/1510243577 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f7ad836670 0x55f7adbfeeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:32.338 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:32.333+0000 7f6c42ffd640 1 -- 172.21.15.39:0/1510243577 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f7ad8c2930 con 0x55f7ad82e490 2024-09-21T03:53:32.338 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:32.333+0000 7f6c427fc640 1 --2- 172.21.15.39:0/1510243577 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f7ad857350 0x55f7adbfe430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:53:32.338 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:32.334+0000 7f6c2bfff640 1 --2- 172.21.15.39:0/1830349513 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f6c2c0b66b0 0x7f6c2c0b8bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:53:32.338 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:32.334+0000 7f6c42ffd640 1 --2- 172.21.15.39:0/1510243577 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55f7ad82e490 0x55f7adbfe970 secure :-1 s=READY pgs=1081 cs=0 l=1 rev1=1 crypto rx=0x7f6c34004650 tx=0x7f6c3400bba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=557a46e7c7a29557 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:53:32.338 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:32.334+0000 7f6c427fc640 1 --2- 172.21.15.39:0/1830349513 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f6c2c0b66b0 0x7f6c2c0b8bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:53:32.338 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:32.334+0000 7f6c2bfff640 1 -- 172.21.15.39:0/1830349513 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8528+0+0 (secure 0 0 0) 0x7f6c3c108bf0 con 0x7f6c44108d10 2024-09-21T03:53:32.338 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:32.334+0000 7f6c297fa640 1 -- 172.21.15.39:0/1510243577 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c34003bd0 con 0x55f7ad82e490 2024-09-21T03:53:32.338 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:32.334+0000 7f6c297fa640 1 -- 172.21.15.39:0/1510243577 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6c34015070 con 0x55f7ad82e490 2024-09-21T03:53:32.339 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:32.334+0000 7f6c297fa640 1 -- 172.21.15.39:0/1510243577 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c34004c60 con 0x55f7ad82e490 2024-09-21T03:53:32.341 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:32.335+0000 7f6c4f8e5740 1 -- 172.21.15.39:0/1510243577 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55f7ad82e490 msgr2=0x55f7adbfe970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:53:32.341 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:32.335+0000 7f6c427fc640 1 --2- 172.21.15.39:0/1830349513 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f6c2c0b66b0 0x7f6c2c0b8bc0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f6c3800bfd0 tx=0x7f6c3800b410 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:53:32.341 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:32.335+0000 7f6c4f8e5740 1 --2- 172.21.15.39:0/1510243577 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55f7ad82e490 0x55f7adbfe970 secure :-1 s=READY pgs=1081 cs=0 l=1 rev1=1 crypto rx=0x7f6c34004650 tx=0x7f6c3400bba0 comp rx=0 tx=0).stop 2024-09-21T03:53:32.342 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:32.335+0000 7f6c4f8e5740 1 -- 172.21.15.39:0/1510243577 shutdown_connections 2024-09-21T03:53:32.342 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:32.335+0000 7f6c4f8e5740 1 --2- 172.21.15.39:0/1510243577 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f7ad836670 0x55f7adbfeeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:32.342 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:32.335+0000 7f6c4f8e5740 1 --2- 172.21.15.39:0/1510243577 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55f7ad82e490 0x55f7adbfe970 unknown :-1 s=CLOSED pgs=1081 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:32.342 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:32.335+0000 7f6c4f8e5740 1 --2- 172.21.15.39:0/1510243577 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f7ad857350 0x55f7adbfe430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:32.342 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:32.335+0000 7f6c4f8e5740 1 -- 172.21.15.39:0/1510243577 >> 172.21.15.39:0/1510243577 conn(0x55f7ad84b740 msgr2=0x55f7adbfc320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:53:32.342 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:32.335+0000 7f6c4f8e5740 1 -- 172.21.15.39:0/1510243577 shutdown_connections 2024-09-21T03:53:32.342 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:32.335+0000 7f6c4f8e5740 1 -- 172.21.15.39:0/1510243577 wait complete. 2024-09-21T03:53:32.342 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:32.335+0000 7f6c4f8e5740 1 -- start start 2024-09-21T03:53:32.342 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:32.335+0000 7f6c4f8e5740 1 Processor -- start 2024-09-21T03:53:32.342 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:32.336+0000 7f6c4f8e5740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55f7ad82e490 0x55f7adc0a000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:53:32.342 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:32.336+0000 7f6c4f8e5740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f7ad836670 0x55f7adc0a540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:53:32.342 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:32.336+0000 7f6c4f8e5740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f7ad857350 0x55f7adc1a340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:53:32.342 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:32.336+0000 7f6c4f8e5740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x55f7ad9c5890 con 0x55f7ad82e490 2024-09-21T03:53:32.342 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:32.336+0000 7f6c4f8e5740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x55f7adafed40 con 0x55f7ad836670 2024-09-21T03:53:32.342 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:32.336+0000 7f6c4f8e5740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55f7adc11020 con 0x55f7ad857350 2024-09-21T03:53:32.343 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:32.336+0000 7f6c4f8e5740 10 client.0 _ll_register_callbacks cb 0x55f7adb62c80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:53:32.343 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:32.336+0000 7f6c42ffd640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55f7ad82e490 0x55f7adc0a000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:53:32.343 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:32.336+0000 7f6c42ffd640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55f7ad82e490 0x55f7adc0a000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:54642/0 (socket says 172.21.15.39:54642) 2024-09-21T03:53:32.343 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:32.336+0000 7f6c42ffd640 1 -- 172.21.15.39:0/2891058691 learned_addr learned my addr 172.21.15.39:0/2891058691 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:53:32.343 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:32.336+0000 7f6c427fc640 1 --2- 172.21.15.39:0/2891058691 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f7ad836670 0x55f7adc0a540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:53:32.343 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:32.336+0000 7f6c437fe640 1 --2- 172.21.15.39:0/2891058691 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f7ad857350 0x55f7adc1a340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:53:32.343 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:32.336+0000 7f6c42ffd640 1 -- 172.21.15.39:0/2891058691 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f7ad857350 msgr2=0x55f7adc1a340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:53:32.343 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:32.336+0000 7f6c42ffd640 1 --2- 172.21.15.39:0/2891058691 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f7ad857350 0x55f7adc1a340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:32.343 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:32.336+0000 7f6c42ffd640 1 -- 172.21.15.39:0/2891058691 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f7ad836670 msgr2=0x55f7adc0a540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:53:32.343 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:32.336+0000 7f6c42ffd640 1 --2- 172.21.15.39:0/2891058691 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f7ad836670 0x55f7adc0a540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:32.343 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:32.336+0000 7f6c42ffd640 1 -- 172.21.15.39:0/2891058691 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f7adc1a8d0 con 0x55f7ad82e490 2024-09-21T03:53:32.343 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:32.337+0000 7f6c42ffd640 1 --2- 172.21.15.39:0/2891058691 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55f7ad82e490 0x55f7adc0a000 secure :-1 s=READY pgs=1082 cs=0 l=1 rev1=1 crypto rx=0x7f6c34006010 tx=0x7f6c34004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6129b13debc1602f server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:53:32.343 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:32.337+0000 7f6c26ff5640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-09-21T03:53:32.343 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:32.337+0000 7f6c26ff5640 1 -- 172.21.15.39:0/2891058691 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c34028060 con 0x55f7ad82e490 2024-09-21T03:53:32.344 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:32.337+0000 7f6c4f8e5740 1 -- 172.21.15.39:0/2891058691 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6c34015070 con 0x55f7ad82e490 2024-09-21T03:53:32.344 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:32.337+0000 7f6c26ff5640 1 -- 172.21.15.39:0/2891058691 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8528+0+0 (secure 0 0 0) 0x7f6c34011440 con 0x55f7ad82e490 2024-09-21T03:53:32.344 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:32.337+0000 7f6c26ff5640 1 -- 172.21.15.39:0/2891058691 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f6c34011640 con 0x55f7ad82e490 2024-09-21T03:53:32.344 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:32.337+0000 7f6c4f8e5740 10 client.17535 fetch_fsmap learned FSMap version 29 2024-09-21T03:53:32.344 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:32.337+0000 7f6c4f8e5740 1 -- 172.21.15.39:0/2891058691 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55f7adc1d1e0 con 0x55f7ad82e490 2024-09-21T03:53:32.344 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:32.337+0000 7f6c26ff5640 1 -- 172.21.15.39:0/2891058691 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6c34015070 con 0x55f7ad82e490 2024-09-21T03:53:32.344 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:32.337+0000 7f6c26ff5640 1 -- 172.21.15.39:0/2891058691 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f6c34022070 con 0x55f7ad82e490 2024-09-21T03:53:32.344 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:32.338+0000 7f6c4f8e5740 10 client.17535 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T03:53:32.344 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:32.338+0000 7f6c4f8e5740 10 client.17535 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T03:53:32.344 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:32.338+0000 7f6c4f8e5740 20 client.17535 populate_metadata read hostname 'smithi039' 2024-09-21T03:53:32.344 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:32.338+0000 7f6c4f8e5740 1 --2- 172.21.15.39:0/2891058691 >> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] conn(0x55f7ad815cd0 0x55f7adc205f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:53:32.344 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:32.338+0000 7f6c4f8e5740 4 client.17535 mds_command: new command op to 24695 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:53:32.344 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:32.338+0000 7f6c4f8e5740 1 -- 172.21.15.39:0/2891058691 --> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f7adb76db0 con 0x55f7ad815cd0 2024-09-21T03:53:32.344 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:32.338+0000 7f6c427fc640 1 --2- 172.21.15.39:0/2891058691 >> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] conn(0x55f7ad815cd0 0x55f7adc205f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:53:32.345 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:32.339+0000 7f6c427fc640 1 --2- 172.21.15.39:0/2891058691 >> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] conn(0x55f7ad815cd0 0x55f7adc205f0 crc :-1 s=READY pgs=129 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8375609f9061defe server_cookie=8e25acd2de564b58 in_seq=0 out_seq=0 2024-09-21T03:53:32.345 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:32.339+0000 7f6c26ff5640 10 client.17535 ms_handle_connect on v2:172.21.15.150:6838/2949386907 2024-09-21T03:53:32.345 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:32.339+0000 7f6c26ff5640 1 -- 172.21.15.39:0/2891058691 <== mds.0 v2:172.21.15.150:6838/2949386907 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55f7adb76db0 con 0x55f7ad815cd0 2024-09-21T03:53:32.345 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:32.339+0000 7f6c26ff5640 10 client.17535 handle_command_reply: tid=0 2024-09-21T03:53:32.346 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:32.339+0000 7f6c4f8e5740 1 client.17535 shutdown 2024-09-21T03:53:32.346 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:32.339+0000 7f6c4f8e5740 1 -- 172.21.15.39:0/2891058691 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55f7ad82e490 msgr2=0x55f7adc0a000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:53:32.346 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:32.339+0000 7f6c4f8e5740 1 --2- 172.21.15.39:0/2891058691 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55f7ad82e490 0x55f7adc0a000 secure :-1 s=READY pgs=1082 cs=0 l=1 rev1=1 crypto rx=0x7f6c34006010 tx=0x7f6c34004730 comp rx=0 tx=0).stop 2024-09-21T03:53:32.346 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:32.340+0000 7f6c4f8e5740 1 -- 172.21.15.39:0/2891058691 shutdown_connections 2024-09-21T03:53:32.346 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:32.340+0000 7f6c4f8e5740 1 --2- 172.21.15.39:0/2891058691 >> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] conn(0x55f7ad815cd0 0x55f7adc205f0 crc :-1 s=READY pgs=129 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:32.346 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:32.340+0000 7f6c4f8e5740 1 --2- 172.21.15.39:0/2891058691 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f7ad857350 0x55f7adc1a340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:32.346 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:32.340+0000 7f6c4f8e5740 1 --2- 172.21.15.39:0/2891058691 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f7ad836670 0x55f7adc0a540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:32.346 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:32.340+0000 7f6c4f8e5740 1 --2- 172.21.15.39:0/2891058691 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55f7ad82e490 0x55f7adc0a000 unknown :-1 s=CLOSED pgs=1082 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:32.346 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:32.340+0000 7f6c4f8e5740 1 -- 172.21.15.39:0/2891058691 >> 172.21.15.39:0/2891058691 conn(0x55f7ad84b740 msgr2=0x55f7adc015c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:53:32.346 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:32.340+0000 7f6c26ff5640 0 client.17535 ms_handle_reset on v2:172.21.15.150:6838/2949386907 2024-09-21T03:53:32.346 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:32.340+0000 7f6c4f8e5740 1 -- 172.21.15.39:0/2891058691 shutdown_connections 2024-09-21T03:53:32.346 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:32.340+0000 7f6c4f8e5740 1 -- 172.21.15.39:0/2891058691 wait complete. 2024-09-21T03:53:32.346 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:32.340+0000 7f6c4f8e5740 20 client.17535 trim_cache size 0 max 16384 2024-09-21T03:53:32.361 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:32.358+0000 7f6c4f8e5740 1 Processor -- start 2024-09-21T03:53:32.361 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:32.358+0000 7f6c4f8e5740 1 -- start start 2024-09-21T03:53:32.362 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:32.358+0000 7f6c4f8e5740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f7ad82e490 0x55f7adc1a2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:53:32.362 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:32.358+0000 7f6c4f8e5740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55f7ad836670 0x55f7adc04550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:53:32.362 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:32.358+0000 7f6c4f8e5740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f7ad857350 0x55f7adc06ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:53:32.362 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:32.358+0000 7f6c4f8e5740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f6c441ad570 con 0x55f7ad836670 2024-09-21T03:53:32.362 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:32.358+0000 7f6c4f8e5740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f6c441af0b0 con 0x55f7ad82e490 2024-09-21T03:53:32.362 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:32.358+0000 7f6c4f8e5740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f6c441adc40 con 0x55f7ad857350 2024-09-21T03:53:32.362 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:32.358+0000 7f6c427fc640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55f7ad836670 0x55f7adc04550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:53:32.362 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:32.358+0000 7f6c427fc640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55f7ad836670 0x55f7adc04550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:54656/0 (socket says 172.21.15.39:54656) 2024-09-21T03:53:32.362 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:32.358+0000 7f6c427fc640 1 -- 172.21.15.39:0/256262441 learned_addr learned my addr 172.21.15.39:0/256262441 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:53:32.362 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:32.358+0000 7f6c42ffd640 1 --2- 172.21.15.39:0/256262441 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f7ad82e490 0x55f7adc1a2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:53:32.362 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:32.358+0000 7f6c437fe640 1 --2- 172.21.15.39:0/256262441 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f7ad857350 0x55f7adc06ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:53:32.362 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:32.358+0000 7f6c427fc640 1 -- 172.21.15.39:0/256262441 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f7ad857350 msgr2=0x55f7adc06ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:53:32.362 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:32.358+0000 7f6c427fc640 1 --2- 172.21.15.39:0/256262441 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f7ad857350 0x55f7adc06ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:32.363 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:32.358+0000 7f6c427fc640 1 -- 172.21.15.39:0/256262441 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f7ad82e490 msgr2=0x55f7adc1a2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:53:32.363 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:32.358+0000 7f6c427fc640 1 --2- 172.21.15.39:0/256262441 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f7ad82e490 0x55f7adc1a2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:32.363 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:32.358+0000 7f6c427fc640 1 -- 172.21.15.39:0/256262441 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f7ad815f00 con 0x55f7ad836670 2024-09-21T03:53:32.363 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:32.358+0000 7f6c437fe640 1 --2- 172.21.15.39:0/256262441 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f7ad857350 0x55f7adc06ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:53:32.363 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:32.359+0000 7f6c427fc640 1 --2- 172.21.15.39:0/256262441 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55f7ad836670 0x55f7adc04550 secure :-1 s=READY pgs=1083 cs=0 l=1 rev1=1 crypto rx=0x7f6c38005d00 tx=0x7f6c38004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1be8b3e92554dc47 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:53:32.363 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:32.359+0000 7f6c28ff9640 1 -- 172.21.15.39:0/256262441 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c38005020 con 0x55f7ad836670 2024-09-21T03:53:32.363 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:32.359+0000 7f6c28ff9640 1 -- 172.21.15.39:0/256262441 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6c380047a0 con 0x55f7ad836670 2024-09-21T03:53:32.363 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:32.359+0000 7f6c28ff9640 1 -- 172.21.15.39:0/256262441 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c3802b6b0 con 0x55f7ad836670 2024-09-21T03:53:32.366 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:32.360+0000 7f6c4f8e5740 1 -- 172.21.15.39:0/256262441 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55f7ad836670 msgr2=0x55f7adc04550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:53:32.366 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:32.360+0000 7f6c4f8e5740 1 --2- 172.21.15.39:0/256262441 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55f7ad836670 0x55f7adc04550 secure :-1 s=READY pgs=1083 cs=0 l=1 rev1=1 crypto rx=0x7f6c38005d00 tx=0x7f6c38004730 comp rx=0 tx=0).stop 2024-09-21T03:53:32.366 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:32.360+0000 7f6c4f8e5740 1 -- 172.21.15.39:0/256262441 shutdown_connections 2024-09-21T03:53:32.366 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:32.360+0000 7f6c4f8e5740 1 --2- 172.21.15.39:0/256262441 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f7ad857350 0x55f7adc06ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:32.366 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:32.360+0000 7f6c4f8e5740 1 --2- 172.21.15.39:0/256262441 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55f7ad836670 0x55f7adc04550 unknown :-1 s=CLOSED pgs=1083 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:32.366 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:32.360+0000 7f6c4f8e5740 1 --2- 172.21.15.39:0/256262441 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f7ad82e490 0x55f7adc1a2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:32.366 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:32.360+0000 7f6c4f8e5740 1 -- 172.21.15.39:0/256262441 >> 172.21.15.39:0/256262441 conn(0x55f7ad84b740 msgr2=0x55f7adc08680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:53:32.367 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:32.360+0000 7f6c4f8e5740 1 -- 172.21.15.39:0/256262441 shutdown_connections 2024-09-21T03:53:32.367 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:32.360+0000 7f6c4f8e5740 1 -- 172.21.15.39:0/256262441 wait complete. 2024-09-21T03:53:32.367 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:32.360+0000 7f6c4f8e5740 1 -- start start 2024-09-21T03:53:32.367 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:32.360+0000 7f6c4f8e5740 1 Processor -- start 2024-09-21T03:53:32.367 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:32.361+0000 7f6c4f8e5740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f7ad82e490 0x55f7adc21730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:53:32.367 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:32.361+0000 7f6c4f8e5740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55f7ad836670 0x55f7adc21c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:53:32.367 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:32.361+0000 7f6c4f8e5740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f7ad857350 0x55f7adc221b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:53:32.367 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:32.361+0000 7f6c4f8e5740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f6c441ae9e0 con 0x55f7ad836670 2024-09-21T03:53:32.367 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:32.361+0000 7f6c4f8e5740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f6c441ae310 con 0x55f7ad82e490 2024-09-21T03:53:32.367 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:32.361+0000 7f6c4f8e5740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f6c441ab550 con 0x55f7ad857350 2024-09-21T03:53:32.367 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:32.361+0000 7f6c4f8e5740 10 client.0 _ll_register_callbacks cb 0x7f6c3800bde0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:53:32.367 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:32.361+0000 7f6c427fc640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55f7ad836670 0x55f7adc21c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:53:32.367 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:32.361+0000 7f6c427fc640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55f7ad836670 0x55f7adc21c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:54658/0 (socket says 172.21.15.39:54658) 2024-09-21T03:53:32.367 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:32.361+0000 7f6c427fc640 1 -- 172.21.15.39:0/3312891610 learned_addr learned my addr 172.21.15.39:0/3312891610 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:53:32.367 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:32.361+0000 7f6c42ffd640 1 --2- 172.21.15.39:0/3312891610 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f7ad82e490 0x55f7adc21730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:53:32.368 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:32.361+0000 7f6c437fe640 1 --2- 172.21.15.39:0/3312891610 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f7ad857350 0x55f7adc221b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:53:32.368 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:32.361+0000 7f6c427fc640 1 -- 172.21.15.39:0/3312891610 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f7ad857350 msgr2=0x55f7adc221b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:53:32.368 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:32.361+0000 7f6c427fc640 1 --2- 172.21.15.39:0/3312891610 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f7ad857350 0x55f7adc221b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:32.368 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:32.361+0000 7f6c427fc640 1 -- 172.21.15.39:0/3312891610 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f7ad82e490 msgr2=0x55f7adc21730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:53:32.368 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:32.361+0000 7f6c427fc640 1 --2- 172.21.15.39:0/3312891610 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f7ad82e490 0x55f7adc21730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:32.368 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:32.361+0000 7f6c427fc640 1 -- 172.21.15.39:0/3312891610 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f7adc89ea0 con 0x55f7ad836670 2024-09-21T03:53:32.368 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:32.361+0000 7f6c437fe640 1 --2- 172.21.15.39:0/3312891610 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f7ad857350 0x55f7adc221b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:53:32.368 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:32.361+0000 7f6c427fc640 1 --2- 172.21.15.39:0/3312891610 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55f7ad836670 0x55f7adc21c70 secure :-1 s=READY pgs=1084 cs=0 l=1 rev1=1 crypto rx=0x7f6c3800bb70 tx=0x7f6c3802b470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc9a38c10fa28d26 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:53:32.368 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:32.361+0000 7f6c26ff5640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-09-21T03:53:32.368 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:32.362+0000 7f6c26ff5640 1 -- 172.21.15.39:0/3312891610 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c3802b830 con 0x55f7ad836670 2024-09-21T03:53:32.368 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:32.362+0000 7f6c26ff5640 1 -- 172.21.15.39:0/3312891610 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8528+0+0 (secure 0 0 0) 0x7f6c3802b9d0 con 0x55f7ad836670 2024-09-21T03:53:32.368 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:32.362+0000 7f6c4f8e5740 1 -- 172.21.15.39:0/3312891610 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6c380047a0 con 0x55f7ad836670 2024-09-21T03:53:32.368 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:32.362+0000 7f6c26ff5640 1 -- 172.21.15.39:0/3312891610 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f6c3802bbd0 con 0x55f7ad836670 2024-09-21T03:53:32.368 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:32.362+0000 7f6c4f8e5740 10 client.17541 fetch_fsmap learned FSMap version 29 2024-09-21T03:53:32.369 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:32.362+0000 7f6c4f8e5740 1 -- 172.21.15.39:0/3312891610 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55f7adc8b140 con 0x55f7ad836670 2024-09-21T03:53:32.369 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:32.362+0000 7f6c26ff5640 1 -- 172.21.15.39:0/3312891610 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6c380047a0 con 0x55f7ad836670 2024-09-21T03:53:32.369 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:32.362+0000 7f6c26ff5640 1 -- 172.21.15.39:0/3312891610 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f6c3801d070 con 0x55f7ad836670 2024-09-21T03:53:32.369 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:32.362+0000 7f6c4f8e5740 10 client.17541 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T03:53:32.369 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:32.362+0000 7f6c4f8e5740 10 client.17541 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T03:53:32.369 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:32.362+0000 7f6c4f8e5740 20 client.17541 populate_metadata read hostname 'smithi039' 2024-09-21T03:53:32.369 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:32.362+0000 7f6c4f8e5740 1 --2- 172.21.15.39:0/3312891610 >> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] conn(0x55f7ad7eb080 0x55f7adc8d820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:53:32.369 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:32.362+0000 7f6c4f8e5740 4 client.17541 mds_command: new command op to 24695 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-21T03:53:32.369 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:32.362+0000 7f6c4f8e5740 1 -- 172.21.15.39:0/3312891610 --> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55f7adb72860 con 0x55f7ad7eb080 2024-09-21T03:53:32.369 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:32.363+0000 7f6c42ffd640 1 --2- 172.21.15.39:0/3312891610 >> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] conn(0x55f7ad7eb080 0x55f7adc8d820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:53:32.369 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:32.363+0000 7f6c26ff5640 10 client.17541 ms_handle_connect on v2:172.21.15.150:6838/2949386907 2024-09-21T03:53:32.369 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:32.363+0000 7f6c42ffd640 1 --2- 172.21.15.39:0/3312891610 >> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] conn(0x55f7ad7eb080 0x55f7adc8d820 crc :-1 s=READY pgs=130 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d734bfd7336e2808 server_cookie=76cd5b416d10a78a in_seq=0 out_seq=0 2024-09-21T03:53:32.370 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:32.365+0000 7f6c26ff5640 1 -- 172.21.15.39:0/3312891610 <== mds.0 v2:172.21.15.150:6838/2949386907 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55f7adb72860 con 0x55f7ad7eb080 2024-09-21T03:53:32.370 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:32.365+0000 7f6c26ff5640 10 client.17541 handle_command_reply: tid=0 2024-09-21T03:53:32.370 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:32.365+0000 7f6c4f8e5740 1 client.17541 shutdown 2024-09-21T03:53:32.370 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:32.365+0000 7f6c4f8e5740 1 -- 172.21.15.39:0/3312891610 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55f7ad836670 msgr2=0x55f7adc21c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:53:32.370 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:32.365+0000 7f6c4f8e5740 1 --2- 172.21.15.39:0/3312891610 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55f7ad836670 0x55f7adc21c70 secure :-1 s=READY pgs=1084 cs=0 l=1 rev1=1 crypto rx=0x7f6c3800bb70 tx=0x7f6c3802b470 comp rx=0 tx=0).stop 2024-09-21T03:53:32.371 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:32.365+0000 7f6c4f8e5740 1 -- 172.21.15.39:0/3312891610 shutdown_connections 2024-09-21T03:53:32.371 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:32.365+0000 7f6c4f8e5740 1 --2- 172.21.15.39:0/3312891610 >> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] conn(0x55f7ad7eb080 0x55f7adc8d820 crc :-1 s=READY pgs=130 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:32.371 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:32.365+0000 7f6c26ff5640 0 client.17541 ms_handle_reset on v2:172.21.15.150:6838/2949386907 2024-09-21T03:53:32.371 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:32.365+0000 7f6c4f8e5740 1 --2- 172.21.15.39:0/3312891610 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f7ad857350 0x55f7adc221b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:32.371 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:32.365+0000 7f6c4f8e5740 1 --2- 172.21.15.39:0/3312891610 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55f7ad836670 0x55f7adc21c70 unknown :-1 s=CLOSED pgs=1084 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:32.371 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:32.365+0000 7f6c4f8e5740 1 --2- 172.21.15.39:0/3312891610 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f7ad82e490 0x55f7adc21730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:32.371 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:32.365+0000 7f6c4f8e5740 1 -- 172.21.15.39:0/3312891610 >> 172.21.15.39:0/3312891610 conn(0x55f7ad84b740 msgr2=0x55f7adbff460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:53:32.371 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:32.365+0000 7f6c4f8e5740 1 -- 172.21.15.39:0/3312891610 shutdown_connections 2024-09-21T03:53:32.371 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:32.366+0000 7f6c4f8e5740 1 -- 172.21.15.39:0/3312891610 wait complete. 2024-09-21T03:53:32.371 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:32.366+0000 7f6c4f8e5740 20 client.17541 trim_cache size 0 max 16384 2024-09-21T03:53:32.371 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-09-21T03:53:32.371 INFO:teuthology.orchestra.run.smithi039.stdout: "return_code": 0, 2024-09-21T03:53:32.372 INFO:teuthology.orchestra.run.smithi039.stdout: "scrub_tag": "d9391d69-baac-453a-9d52-d046760c070d", 2024-09-21T03:53:32.372 INFO:teuthology.orchestra.run.smithi039.stdout: "mode": "asynchronous" 2024-09-21T03:53:32.372 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-09-21T03:53:32.372 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:32.366+0000 7f6c49371640 1 -- 172.21.15.39:0/1830349513 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f6c2c0b66b0 msgr2=0x7f6c2c0b8bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:53:32.372 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:32.366+0000 7f6c49371640 1 --2- 172.21.15.39:0/1830349513 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f6c2c0b66b0 0x7f6c2c0b8bc0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f6c3800bfd0 tx=0x7f6c3800b410 comp rx=0 tx=0).stop 2024-09-21T03:53:32.372 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:32.366+0000 7f6c49371640 1 -- 172.21.15.39:0/1830349513 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6c44108d10 msgr2=0x7f6c4419bd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:53:32.372 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:32.366+0000 7f6c49371640 1 --2- 172.21.15.39:0/1830349513 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6c44108d10 0x7f6c4419bd70 secure :-1 s=READY pgs=1080 cs=0 l=1 rev1=1 crypto rx=0x7f6c3c005cc0 tx=0x7f6c3c02f380 comp rx=0 tx=0).stop 2024-09-21T03:53:32.372 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:32.366+0000 7f6c49371640 1 -- 172.21.15.39:0/1830349513 shutdown_connections 2024-09-21T03:53:32.372 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:32.366+0000 7f6c49371640 1 --2- 172.21.15.39:0/1830349513 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f6c2c0b66b0 0x7f6c2c0b8bc0 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:32.372 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:32.366+0000 7f6c49371640 1 --2- 172.21.15.39:0/1830349513 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6c44108d10 0x7f6c4419bd70 unknown :-1 s=CLOSED pgs=1080 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:32.373 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:32.366+0000 7f6c49371640 1 --2- 172.21.15.39:0/1830349513 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6c441083c0 0x7f6c4419b830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:32.373 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:32.366+0000 7f6c49371640 1 --2- 172.21.15.39:0/1830349513 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6c44102a80 0x7f6c4419b2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:32.373 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:32.366+0000 7f6c49371640 1 -- 172.21.15.39:0/1830349513 >> 172.21.15.39:0/1830349513 conn(0x7f6c44079740 msgr2=0x7f6c441043d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:53:32.373 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:32.367+0000 7f6c49371640 1 -- 172.21.15.39:0/1830349513 shutdown_connections 2024-09-21T03:53:32.373 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:32.367+0000 7f6c49371640 1 -- 172.21.15.39:0/1830349513 wait complete. 2024-09-21T03:53:33.188 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:53:32 smithi039 ceph-mon[31211]: pgmap v552: 97 pgs: 97 active+clean; 10 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 9 op/s 2024-09-21T03:53:33.218 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 51116d5c-77ca-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub status 2024-09-21T03:53:33.219 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:53:32 smithi161 ceph-mon[34533]: pgmap v552: 97 pgs: 97 active+clean; 10 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 9 op/s 2024-09-21T03:53:33.224 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:53:32 smithi150 ceph-mon[34289]: pgmap v552: 97 pgs: 97 active+clean; 10 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 9 op/s 2024-09-21T03:53:33.475 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/51116d5c-77ca-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:53:33.987 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:33.984+0000 7f2b64ce7640 1 -- 172.21.15.39:0/1522423588 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2b60108cf0 msgr2=0x7f2b6010f520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:53:33.987 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:33.984+0000 7f2b64ce7640 1 --2- 172.21.15.39:0/1522423588 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2b60108cf0 0x7f2b6010f520 secure :-1 s=READY pgs=1085 cs=0 l=1 rev1=1 crypto rx=0x7f2b58009ca0 tx=0x7f2b5802a8c0 comp rx=0 tx=0).stop 2024-09-21T03:53:33.987 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:33.984+0000 7f2b64ce7640 1 -- 172.21.15.39:0/1522423588 shutdown_connections 2024-09-21T03:53:33.987 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:33.984+0000 7f2b64ce7640 1 --2- 172.21.15.39:0/1522423588 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2b60108cf0 0x7f2b6010f520 unknown :-1 s=CLOSED pgs=1085 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:33.987 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:33.984+0000 7f2b64ce7640 1 --2- 172.21.15.39:0/1522423588 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2b601083a0 0x7f2b601087b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:33.987 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:33.984+0000 7f2b64ce7640 1 --2- 172.21.15.39:0/1522423588 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2b60102a60 0x7f2b60102e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:33.987 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:33.984+0000 7f2b64ce7640 1 -- 172.21.15.39:0/1522423588 >> 172.21.15.39:0/1522423588 conn(0x7f2b60079730 msgr2=0x7f2b60079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:53:33.987 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:33.984+0000 7f2b64ce7640 1 -- 172.21.15.39:0/1522423588 shutdown_connections 2024-09-21T03:53:33.988 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:33.984+0000 7f2b64ce7640 1 -- 172.21.15.39:0/1522423588 wait complete. 2024-09-21T03:53:33.989 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:33.985+0000 7f2b64ce7640 1 Processor -- start 2024-09-21T03:53:33.989 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:33.985+0000 7f2b64ce7640 1 -- start start 2024-09-21T03:53:33.989 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:33.985+0000 7f2b64ce7640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2b60102a60 0x7f2b600ff670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:53:33.989 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:33.985+0000 7f2b64ce7640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2b601083a0 0x7f2b600ffbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:53:33.989 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:33.985+0000 7f2b64ce7640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2b60108cf0 0x7f2b601000f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:53:33.989 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:33.985+0000 7f2b64ce7640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f2b60112960 con 0x7f2b60108cf0 2024-09-21T03:53:33.989 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:33.985+0000 7f2b64ce7640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f2b601127e0 con 0x7f2b60102a60 2024-09-21T03:53:33.990 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:33.985+0000 7f2b64ce7640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f2b60112ae0 con 0x7f2b601083a0 2024-09-21T03:53:33.990 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:33.985+0000 7f2b5ed9d640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2b60108cf0 0x7f2b601000f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:53:33.990 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:33.985+0000 7f2b5ed9d640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2b60108cf0 0x7f2b601000f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:54684/0 (socket says 172.21.15.39:54684) 2024-09-21T03:53:33.990 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:33.985+0000 7f2b5ed9d640 1 -- 172.21.15.39:0/369378574 learned_addr learned my addr 172.21.15.39:0/369378574 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:53:33.990 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:33.985+0000 7f2b5e59c640 1 --2- 172.21.15.39:0/369378574 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2b60102a60 0x7f2b600ff670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:53:33.990 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:33.985+0000 7f2b5dd9b640 1 --2- 172.21.15.39:0/369378574 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2b601083a0 0x7f2b600ffbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:53:33.990 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:33.986+0000 7f2b5ed9d640 1 -- 172.21.15.39:0/369378574 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2b601083a0 msgr2=0x7f2b600ffbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:53:33.990 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:33.986+0000 7f2b5ed9d640 1 --2- 172.21.15.39:0/369378574 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2b601083a0 0x7f2b600ffbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:33.990 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:33.986+0000 7f2b5ed9d640 1 -- 172.21.15.39:0/369378574 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2b60102a60 msgr2=0x7f2b600ff670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:53:33.990 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:33.986+0000 7f2b5ed9d640 1 --2- 172.21.15.39:0/369378574 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2b60102a60 0x7f2b600ff670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:33.990 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:33.986+0000 7f2b5ed9d640 1 -- 172.21.15.39:0/369378574 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2b601a85f0 con 0x7f2b60108cf0 2024-09-21T03:53:33.990 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:33.986+0000 7f2b5dd9b640 1 --2- 172.21.15.39:0/369378574 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2b601083a0 0x7f2b600ffbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:53:33.990 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:33.986+0000 7f2b5ed9d640 1 --2- 172.21.15.39:0/369378574 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2b60108cf0 0x7f2b601000f0 secure :-1 s=READY pgs=1086 cs=0 l=1 rev1=1 crypto rx=0x7f2b58005cc0 tx=0x7f2b5802f470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:53:33.991 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:33.986+0000 7f2b4f7fe640 1 -- 172.21.15.39:0/369378574 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b58003cf0 con 0x7f2b60108cf0 2024-09-21T03:53:33.991 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:33.986+0000 7f2b64ce7640 1 -- 172.21.15.39:0/369378574 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2b601a8880 con 0x7f2b60108cf0 2024-09-21T03:53:33.991 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:33.986+0000 7f2b4f7fe640 1 -- 172.21.15.39:0/369378574 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2b580044f0 con 0x7f2b60108cf0 2024-09-21T03:53:33.991 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:33.986+0000 7f2b4f7fe640 1 -- 172.21.15.39:0/369378574 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b5802fc30 con 0x7f2b60108cf0 2024-09-21T03:53:33.991 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:33.986+0000 7f2b64ce7640 1 -- 172.21.15.39:0/369378574 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2b601a8e10 con 0x7f2b60108cf0 2024-09-21T03:53:33.992 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:33.989+0000 7f2b6b25b740 1 Processor -- start 2024-09-21T03:53:33.992 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:33.989+0000 7f2b4f7fe640 1 -- 172.21.15.39:0/369378574 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 27) ==== 154643+0+0 (secure 0 0 0) 0x7f2b580046a0 con 0x7f2b60108cf0 2024-09-21T03:53:33.992 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:33.989+0000 7f2b6b25b740 1 -- start start 2024-09-21T03:53:33.994 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:33.990+0000 7f2b6b25b740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5645a7907070 0x5645a7cae650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:53:33.994 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:33.990+0000 7f2b6b25b740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5645a78de1b0 0x5645a7caeb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:53:33.994 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:33.990+0000 7f2b6b25b740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5645a78e6390 0x5645a7caf0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:53:33.994 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:33.990+0000 7f2b6b25b740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x5645a7c24e50 con 0x5645a7907070 2024-09-21T03:53:33.994 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:33.990+0000 7f2b6b25b740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x5645a7c14280 con 0x5645a78de1b0 2024-09-21T03:53:33.994 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:33.990+0000 7f2b6b25b740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x5645a7c18090 con 0x5645a78e6390 2024-09-21T03:53:33.994 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:33.990+0000 7f2b5dd9b640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5645a7907070 0x5645a7cae650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:53:33.994 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:33.990+0000 7f2b5dd9b640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5645a7907070 0x5645a7cae650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:54696/0 (socket says 172.21.15.39:54696) 2024-09-21T03:53:33.994 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:33.990+0000 7f2b5dd9b640 1 -- 172.21.15.39:0/3328302749 learned_addr learned my addr 172.21.15.39:0/3328302749 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:53:33.994 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:33.990+0000 7f2b5e59c640 1 --2- 172.21.15.39:0/3328302749 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5645a78de1b0 0x5645a7caeb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:53:33.995 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:33.990+0000 7f2b5dd9b640 1 --2- 172.21.15.39:0/3328302749 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5645a78e6390 0x5645a7caf0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:53:33.995 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:33.990+0000 7f2b5dd9b640 1 -- 172.21.15.39:0/3328302749 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5645a78e6390 msgr2=0x5645a7caf0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:53:33.995 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:33.990+0000 7f2b5dd9b640 1 --2- 172.21.15.39:0/3328302749 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5645a78e6390 0x5645a7caf0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:33.995 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:33.990+0000 7f2b5dd9b640 1 -- 172.21.15.39:0/3328302749 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5645a78de1b0 msgr2=0x5645a7caeb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:53:33.995 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:33.990+0000 7f2b5dd9b640 1 --2- 172.21.15.39:0/3328302749 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5645a78de1b0 0x5645a7caeb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:33.995 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:33.991+0000 7f2b5dd9b640 1 -- 172.21.15.39:0/3328302749 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5645a7972650 con 0x5645a7907070 2024-09-21T03:53:33.995 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:33.991+0000 7f2b4f7fe640 1 --2- 172.21.15.39:0/369378574 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f2b480b6680 0x7f2b480b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:53:33.995 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:33.991+0000 7f2b5dd9b640 1 --2- 172.21.15.39:0/3328302749 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5645a7907070 0x5645a7cae650 secure :-1 s=READY pgs=1087 cs=0 l=1 rev1=1 crypto rx=0x7f2b540103a0 tx=0x7f2b54008580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db41c3a688243305 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:53:33.995 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:33.991+0000 7f2b5e59c640 1 --2- 172.21.15.39:0/369378574 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f2b480b6680 0x7f2b480b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:53:33.995 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:33.991+0000 7f2b4f7fe640 1 -- 172.21.15.39:0/369378574 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8528+0+0 (secure 0 0 0) 0x7f2b58108c20 con 0x7f2b60108cf0 2024-09-21T03:53:33.995 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:33.991+0000 7f2b4cff9640 1 -- 172.21.15.39:0/3328302749 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b54012660 con 0x5645a7907070 2024-09-21T03:53:33.995 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:33.991+0000 7f2b4cff9640 1 -- 172.21.15.39:0/3328302749 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2b54012cb0 con 0x5645a7907070 2024-09-21T03:53:33.995 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:33.991+0000 7f2b4cff9640 1 -- 172.21.15.39:0/3328302749 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b5400eb80 con 0x5645a7907070 2024-09-21T03:53:33.998 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:33.992+0000 7f2b6b25b740 1 -- 172.21.15.39:0/3328302749 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5645a7907070 msgr2=0x5645a7cae650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:53:33.998 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:33.992+0000 7f2b6b25b740 1 --2- 172.21.15.39:0/3328302749 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5645a7907070 0x5645a7cae650 secure :-1 s=READY pgs=1087 cs=0 l=1 rev1=1 crypto rx=0x7f2b540103a0 tx=0x7f2b54008580 comp rx=0 tx=0).stop 2024-09-21T03:53:33.998 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:33.992+0000 7f2b5e59c640 1 --2- 172.21.15.39:0/369378574 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f2b480b6680 0x7f2b480b8b90 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x5645a7cad2d0 tx=0x7f2b5000a2d0 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:53:33.998 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:33.992+0000 7f2b6b25b740 1 -- 172.21.15.39:0/3328302749 shutdown_connections 2024-09-21T03:53:33.998 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:33.992+0000 7f2b6b25b740 1 --2- 172.21.15.39:0/3328302749 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5645a78e6390 0x5645a7caf0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:33.998 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:33.992+0000 7f2b6b25b740 1 --2- 172.21.15.39:0/3328302749 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5645a78de1b0 0x5645a7caeb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:33.998 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:33.992+0000 7f2b6b25b740 1 --2- 172.21.15.39:0/3328302749 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5645a7907070 0x5645a7cae650 unknown :-1 s=CLOSED pgs=1087 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:33.998 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:33.992+0000 7f2b6b25b740 1 -- 172.21.15.39:0/3328302749 >> 172.21.15.39:0/3328302749 conn(0x5645a78fb460 msgr2=0x5645a7cac540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:53:33.998 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:33.992+0000 7f2b6b25b740 1 -- 172.21.15.39:0/3328302749 shutdown_connections 2024-09-21T03:53:33.998 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:33.992+0000 7f2b6b25b740 1 -- 172.21.15.39:0/3328302749 wait complete. 2024-09-21T03:53:33.998 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:33.992+0000 7f2b6b25b740 1 -- start start 2024-09-21T03:53:33.999 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:33.993+0000 7f2b6b25b740 1 Processor -- start 2024-09-21T03:53:33.999 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:33.993+0000 7f2b6b25b740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5645a78de1b0 0x5645a7cb9c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:53:33.999 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:33.993+0000 7f2b6b25b740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5645a78e6390 0x5645a7cba180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:53:33.999 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:33.993+0000 7f2b6b25b740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5645a7907070 0x5645a7cba6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:53:33.999 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:33.993+0000 7f2b6b25b740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x5645a7a75700 con 0x5645a7907070 2024-09-21T03:53:33.999 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:33.993+0000 7f2b6b25b740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x5645a7baf110 con 0x5645a78de1b0 2024-09-21T03:53:33.999 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:33.993+0000 7f2b6b25b740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x5645a7cbced0 con 0x5645a78e6390 2024-09-21T03:53:33.999 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:33.993+0000 7f2b6b25b740 10 client.0 _ll_register_callbacks cb 0x5645a7bc8990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:53:33.999 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:33.993+0000 7f2b5ed9d640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5645a7907070 0x5645a7cba6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:53:33.999 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:33.993+0000 7f2b5ed9d640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5645a7907070 0x5645a7cba6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:54704/0 (socket says 172.21.15.39:54704) 2024-09-21T03:53:33.999 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:33.993+0000 7f2b5ed9d640 1 -- 172.21.15.39:0/1619082069 learned_addr learned my addr 172.21.15.39:0/1619082069 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:53:33.999 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:33.993+0000 7f2b5ed9d640 1 -- 172.21.15.39:0/1619082069 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5645a78e6390 msgr2=0x5645a7cba180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:53:33.999 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:33.993+0000 7f2b5ed9d640 1 --2- 172.21.15.39:0/1619082069 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5645a78e6390 0x5645a7cba180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:34.000 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:33.993+0000 7f2b5ed9d640 1 -- 172.21.15.39:0/1619082069 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5645a78de1b0 msgr2=0x5645a7cb9c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:53:34.000 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:33.993+0000 7f2b5ed9d640 1 --2- 172.21.15.39:0/1619082069 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5645a78de1b0 0x5645a7cb9c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:34.000 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:33.993+0000 7f2b5ed9d640 1 -- 172.21.15.39:0/1619082069 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5645a7bdcc40 con 0x5645a7907070 2024-09-21T03:53:34.000 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:33.994+0000 7f2b5ed9d640 1 --2- 172.21.15.39:0/1619082069 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5645a7907070 0x5645a7cba6c0 secure :-1 s=READY pgs=1088 cs=0 l=1 rev1=1 crypto rx=0x7f2b58106310 tx=0x7f2b58030000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a141fbeee100910a server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:53:34.000 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:33.994+0000 7f2b45ffb640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-09-21T03:53:34.000 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:33.994+0000 7f2b45ffb640 1 -- 172.21.15.39:0/1619082069 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b580ca070 con 0x5645a7907070 2024-09-21T03:53:34.000 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:33.994+0000 7f2b45ffb640 1 -- 172.21.15.39:0/1619082069 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8528+0+0 (secure 0 0 0) 0x7f2b580d3170 con 0x5645a7907070 2024-09-21T03:53:34.000 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:33.994+0000 7f2b6b25b740 1 -- 172.21.15.39:0/1619082069 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2b54012cb0 con 0x5645a7907070 2024-09-21T03:53:34.000 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:33.994+0000 7f2b45ffb640 1 -- 172.21.15.39:0/1619082069 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f2b580e9050 con 0x5645a7907070 2024-09-21T03:53:34.000 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:33.994+0000 7f2b6b25b740 10 client.17553 fetch_fsmap learned FSMap version 29 2024-09-21T03:53:34.000 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:33.994+0000 7f2b6b25b740 1 -- 172.21.15.39:0/1619082069 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5645a7cc8ae0 con 0x5645a7907070 2024-09-21T03:53:34.000 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:33.994+0000 7f2b45ffb640 1 -- 172.21.15.39:0/1619082069 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2b54012cb0 con 0x5645a7907070 2024-09-21T03:53:34.001 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:33.994+0000 7f2b45ffb640 1 -- 172.21.15.39:0/1619082069 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f2b580d8000 con 0x5645a7907070 2024-09-21T03:53:34.001 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:33.994+0000 7f2b6b25b740 10 client.17553 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T03:53:34.001 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:33.994+0000 7f2b6b25b740 10 client.17553 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T03:53:34.001 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:33.994+0000 7f2b6b25b740 20 client.17553 populate_metadata read hostname 'smithi039' 2024-09-21T03:53:34.001 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:33.994+0000 7f2b6b25b740 1 --2- 172.21.15.39:0/1619082069 >> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] conn(0x5645a78c59f0 0x5645a7ccc0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:53:34.001 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:33.995+0000 7f2b6b25b740 4 client.17553 mds_command: new command op to 24695 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:53:34.001 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:33.995+0000 7f2b6b25b740 1 -- 172.21.15.39:0/1619082069 --> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5645a7c26da0 con 0x5645a78c59f0 2024-09-21T03:53:34.002 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:33.996+0000 7f2b5e59c640 1 --2- 172.21.15.39:0/1619082069 >> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] conn(0x5645a78c59f0 0x5645a7ccc0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:53:34.003 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:33.998+0000 7f2b5e59c640 1 --2- 172.21.15.39:0/1619082069 >> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] conn(0x5645a78c59f0 0x5645a7ccc0c0 crc :-1 s=READY pgs=131 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=57a04659d855c47f server_cookie=27d531f29c3033c1 in_seq=0 out_seq=0 2024-09-21T03:53:34.003 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:33.998+0000 7f2b45ffb640 10 client.17553 ms_handle_connect on v2:172.21.15.150:6838/2949386907 2024-09-21T03:53:34.003 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:33.998+0000 7f2b45ffb640 1 -- 172.21.15.39:0/1619082069 <== mds.0 v2:172.21.15.150:6838/2949386907 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5645a7c26da0 con 0x5645a78c59f0 2024-09-21T03:53:34.003 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:33.998+0000 7f2b45ffb640 10 client.17553 handle_command_reply: tid=0 2024-09-21T03:53:34.003 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:33.999+0000 7f2b6b25b740 1 client.17553 shutdown 2024-09-21T03:53:34.003 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:33.999+0000 7f2b6b25b740 1 -- 172.21.15.39:0/1619082069 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5645a7907070 msgr2=0x5645a7cba6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:53:34.003 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:33.999+0000 7f2b6b25b740 1 --2- 172.21.15.39:0/1619082069 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5645a7907070 0x5645a7cba6c0 secure :-1 s=READY pgs=1088 cs=0 l=1 rev1=1 crypto rx=0x7f2b58106310 tx=0x7f2b58030000 comp rx=0 tx=0).stop 2024-09-21T03:53:34.003 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:33.999+0000 7f2b6b25b740 1 -- 172.21.15.39:0/1619082069 shutdown_connections 2024-09-21T03:53:34.003 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:33.999+0000 7f2b6b25b740 1 --2- 172.21.15.39:0/1619082069 >> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] conn(0x5645a78c59f0 0x5645a7ccc0c0 crc :-1 s=READY pgs=131 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:34.003 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:33.999+0000 7f2b6b25b740 1 --2- 172.21.15.39:0/1619082069 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5645a7907070 0x5645a7cba6c0 unknown :-1 s=CLOSED pgs=1088 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:34.003 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:33.999+0000 7f2b6b25b740 1 --2- 172.21.15.39:0/1619082069 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5645a78e6390 0x5645a7cba180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:34.003 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:33.999+0000 7f2b6b25b740 1 --2- 172.21.15.39:0/1619082069 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5645a78de1b0 0x5645a7cb9c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:34.003 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:33.999+0000 7f2b45ffb640 0 client.17553 ms_handle_reset on v2:172.21.15.150:6838/2949386907 2024-09-21T03:53:34.003 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:33.999+0000 7f2b6b25b740 1 -- 172.21.15.39:0/1619082069 >> 172.21.15.39:0/1619082069 conn(0x5645a78fb460 msgr2=0x5645a7cb1830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:53:34.004 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:33.999+0000 7f2b6b25b740 1 -- 172.21.15.39:0/1619082069 shutdown_connections 2024-09-21T03:53:34.004 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:33.999+0000 7f2b6b25b740 1 -- 172.21.15.39:0/1619082069 wait complete. 2024-09-21T03:53:34.004 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:33.999+0000 7f2b6b25b740 20 client.17553 trim_cache size 0 max 16384 2024-09-21T03:53:34.020 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:34.017+0000 7f2b6b25b740 1 Processor -- start 2024-09-21T03:53:34.020 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:34.017+0000 7f2b6b25b740 1 -- start start 2024-09-21T03:53:34.021 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:34.017+0000 7f2b6b25b740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5645a78c59f0 0x5645a7cb5460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:53:34.021 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:34.017+0000 7f2b6b25b740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5645a78de1b0 0x5645a7cac1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:53:34.021 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:34.017+0000 7f2b6b25b740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5645a78e6390 0x5645a7cbb870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:53:34.021 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:34.017+0000 7f2b6b25b740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f2b601b1810 con 0x5645a78de1b0 2024-09-21T03:53:34.021 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:34.017+0000 7f2b6b25b740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f2b601b3350 con 0x5645a78c59f0 2024-09-21T03:53:34.021 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:34.017+0000 7f2b6b25b740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f2b601b1ee0 con 0x5645a78e6390 2024-09-21T03:53:34.021 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:34.017+0000 7f2b5dd9b640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5645a78de1b0 0x5645a7cac1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:53:34.021 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:34.017+0000 7f2b5dd9b640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5645a78de1b0 0x5645a7cac1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:54708/0 (socket says 172.21.15.39:54708) 2024-09-21T03:53:34.021 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:34.017+0000 7f2b5dd9b640 1 -- 172.21.15.39:0/252777659 learned_addr learned my addr 172.21.15.39:0/252777659 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:53:34.021 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:34.017+0000 7f2b5ed9d640 1 --2- 172.21.15.39:0/252777659 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5645a78e6390 0x5645a7cbb870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:53:34.021 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:34.017+0000 7f2b5e59c640 1 --2- 172.21.15.39:0/252777659 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5645a78c59f0 0x5645a7cb5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:53:34.021 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:34.017+0000 7f2b5dd9b640 1 -- 172.21.15.39:0/252777659 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5645a78e6390 msgr2=0x5645a7cbb870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:53:34.021 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:34.017+0000 7f2b5dd9b640 1 --2- 172.21.15.39:0/252777659 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5645a78e6390 0x5645a7cbb870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:34.022 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:34.017+0000 7f2b5dd9b640 1 -- 172.21.15.39:0/252777659 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5645a78c59f0 msgr2=0x5645a7cb5460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:53:34.022 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:34.017+0000 7f2b5dd9b640 1 --2- 172.21.15.39:0/252777659 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5645a78c59f0 0x5645a7cb5460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:34.022 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:34.017+0000 7f2b5dd9b640 1 -- 172.21.15.39:0/252777659 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5645a7cbaf00 con 0x5645a78de1b0 2024-09-21T03:53:34.022 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:34.017+0000 7f2b5ed9d640 1 --2- 172.21.15.39:0/252777659 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5645a78e6390 0x5645a7cbb870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:53:34.022 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:34.018+0000 7f2b5dd9b640 1 --2- 172.21.15.39:0/252777659 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5645a78de1b0 0x5645a7cac1e0 secure :-1 s=READY pgs=1089 cs=0 l=1 rev1=1 crypto rx=0x7f2b54005830 tx=0x7f2b54005f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=95632fe2702f23c0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:53:34.022 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:34.018+0000 7f2b47fff640 1 -- 172.21.15.39:0/252777659 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b54008c30 con 0x5645a78de1b0 2024-09-21T03:53:34.022 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:34.018+0000 7f2b47fff640 1 -- 172.21.15.39:0/252777659 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2b540054e0 con 0x5645a78de1b0 2024-09-21T03:53:34.022 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:34.018+0000 7f2b47fff640 1 -- 172.21.15.39:0/252777659 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b54004d30 con 0x5645a78de1b0 2024-09-21T03:53:34.026 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:34.018+0000 7f2b6b25b740 1 -- 172.21.15.39:0/252777659 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5645a78de1b0 msgr2=0x5645a7cac1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:53:34.026 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:34.019+0000 7f2b6b25b740 1 --2- 172.21.15.39:0/252777659 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5645a78de1b0 0x5645a7cac1e0 secure :-1 s=READY pgs=1089 cs=0 l=1 rev1=1 crypto rx=0x7f2b54005830 tx=0x7f2b54005f40 comp rx=0 tx=0).stop 2024-09-21T03:53:34.026 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:34.019+0000 7f2b6b25b740 1 -- 172.21.15.39:0/252777659 shutdown_connections 2024-09-21T03:53:34.026 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:34.019+0000 7f2b6b25b740 1 --2- 172.21.15.39:0/252777659 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5645a78e6390 0x5645a7cbb870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:34.026 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:34.019+0000 7f2b6b25b740 1 --2- 172.21.15.39:0/252777659 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5645a78de1b0 0x5645a7cac1e0 unknown :-1 s=CLOSED pgs=1089 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:34.026 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:34.019+0000 7f2b6b25b740 1 --2- 172.21.15.39:0/252777659 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5645a78c59f0 0x5645a7cb5460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:34.026 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:34.019+0000 7f2b6b25b740 1 -- 172.21.15.39:0/252777659 >> 172.21.15.39:0/252777659 conn(0x5645a78fb460 msgr2=0x5645a7cbb160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:53:34.026 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:34.019+0000 7f2b6b25b740 1 -- 172.21.15.39:0/252777659 shutdown_connections 2024-09-21T03:53:34.026 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:34.019+0000 7f2b6b25b740 1 -- 172.21.15.39:0/252777659 wait complete. 2024-09-21T03:53:34.026 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:34.019+0000 7f2b6b25b740 1 -- start start 2024-09-21T03:53:34.026 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:34.019+0000 7f2b6b25b740 1 Processor -- start 2024-09-21T03:53:34.026 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:34.019+0000 7f2b6b25b740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5645a78c59f0 0x5645a7cc2b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:53:34.027 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:34.019+0000 7f2b6b25b740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5645a78de1b0 0x5645a7cbe020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:53:34.027 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:34.019+0000 7f2b6b25b740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5645a78e6390 0x5645a7cbe730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:53:34.027 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:34.019+0000 7f2b6b25b740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f2b601b2c80 con 0x5645a78de1b0 2024-09-21T03:53:34.027 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:34.019+0000 7f2b6b25b740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f2b601b25b0 con 0x5645a78c59f0 2024-09-21T03:53:34.027 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:34.019+0000 7f2b6b25b740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f2b601af7f0 con 0x5645a78e6390 2024-09-21T03:53:34.027 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:34.019+0000 7f2b6b25b740 10 client.0 _ll_register_callbacks cb 0x7f2b580095c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:53:34.027 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:34.020+0000 7f2b5dd9b640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5645a78de1b0 0x5645a7cbe020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:53:34.027 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:34.020+0000 7f2b5dd9b640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5645a78de1b0 0x5645a7cbe020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:54724/0 (socket says 172.21.15.39:54724) 2024-09-21T03:53:34.027 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:34.020+0000 7f2b5dd9b640 1 -- 172.21.15.39:0/4067176697 learned_addr learned my addr 172.21.15.39:0/4067176697 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:53:34.027 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:34.020+0000 7f2b5e59c640 1 --2- 172.21.15.39:0/4067176697 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5645a78c59f0 0x5645a7cc2b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:53:34.027 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:34.020+0000 7f2b5ed9d640 1 --2- 172.21.15.39:0/4067176697 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5645a78e6390 0x5645a7cbe730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:53:34.027 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:34.020+0000 7f2b5dd9b640 1 -- 172.21.15.39:0/4067176697 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5645a78e6390 msgr2=0x5645a7cbe730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:53:34.027 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:34.020+0000 7f2b5dd9b640 1 --2- 172.21.15.39:0/4067176697 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5645a78e6390 0x5645a7cbe730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:34.027 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:34.020+0000 7f2b5dd9b640 1 -- 172.21.15.39:0/4067176697 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5645a78c59f0 msgr2=0x5645a7cc2b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:53:34.028 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:34.020+0000 7f2b5dd9b640 1 --2- 172.21.15.39:0/4067176697 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5645a78c59f0 0x5645a7cc2b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:34.028 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:34.020+0000 7f2b5dd9b640 1 -- 172.21.15.39:0/4067176697 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5645a7d35ac0 con 0x5645a78de1b0 2024-09-21T03:53:34.028 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:34.020+0000 7f2b5dd9b640 1 --2- 172.21.15.39:0/4067176697 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5645a78de1b0 0x5645a7cbe020 secure :-1 s=READY pgs=1090 cs=0 l=1 rev1=1 crypto rx=0x7f2b54005880 tx=0x7f2b540052f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5460fa24840a5933 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:53:34.028 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:34.020+0000 7f2b45ffb640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-09-21T03:53:34.028 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:34.020+0000 7f2b45ffb640 1 -- 172.21.15.39:0/4067176697 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b54034060 con 0x5645a78de1b0 2024-09-21T03:53:34.028 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:34.020+0000 7f2b45ffb640 1 -- 172.21.15.39:0/4067176697 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8528+0+0 (secure 0 0 0) 0x7f2b54023040 con 0x5645a78de1b0 2024-09-21T03:53:34.028 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:34.020+0000 7f2b6b25b740 1 -- 172.21.15.39:0/4067176697 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2b540054e0 con 0x5645a78de1b0 2024-09-21T03:53:34.028 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:34.021+0000 7f2b45ffb640 1 -- 172.21.15.39:0/4067176697 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f2b54005690 con 0x5645a78de1b0 2024-09-21T03:53:34.028 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:34.021+0000 7f2b6b25b740 10 client.17559 fetch_fsmap learned FSMap version 29 2024-09-21T03:53:34.028 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:34.021+0000 7f2b6b25b740 1 -- 172.21.15.39:0/4067176697 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5645a7d36c20 con 0x5645a78de1b0 2024-09-21T03:53:34.028 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:34.021+0000 7f2b45ffb640 1 -- 172.21.15.39:0/4067176697 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2b540054e0 con 0x5645a78de1b0 2024-09-21T03:53:34.028 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:34.021+0000 7f2b45ffb640 1 -- 172.21.15.39:0/4067176697 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f2b54027420 con 0x5645a78de1b0 2024-09-21T03:53:34.028 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:34.021+0000 7f2b6b25b740 10 client.17559 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T03:53:34.028 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:34.021+0000 7f2b6b25b740 10 client.17559 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T03:53:34.028 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:34.021+0000 7f2b6b25b740 20 client.17559 populate_metadata read hostname 'smithi039' 2024-09-21T03:53:34.029 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:34.021+0000 7f2b6b25b740 1 --2- 172.21.15.39:0/4067176697 >> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] conn(0x5645a7907070 0x5645a7d39110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:53:34.029 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:34.021+0000 7f2b6b25b740 4 client.17559 mds_command: new command op to 24695 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-21T03:53:34.029 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:34.021+0000 7f2b6b25b740 1 -- 172.21.15.39:0/4067176697 --> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5645a7c22850 con 0x5645a7907070 2024-09-21T03:53:34.029 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:34.021+0000 7f2b5e59c640 1 --2- 172.21.15.39:0/4067176697 >> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] conn(0x5645a7907070 0x5645a7d39110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:53:34.029 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:34.022+0000 7f2b45ffb640 10 client.17559 ms_handle_connect on v2:172.21.15.150:6838/2949386907 2024-09-21T03:53:34.029 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:34.022+0000 7f2b5e59c640 1 --2- 172.21.15.39:0/4067176697 >> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] conn(0x5645a7907070 0x5645a7d39110 crc :-1 s=READY pgs=132 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fa791085da772bcb server_cookie=24707f7499be1dda in_seq=0 out_seq=0 2024-09-21T03:53:34.029 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:34.022+0000 7f2b45ffb640 1 -- 172.21.15.39:0/4067176697 <== mds.0 v2:172.21.15.150:6838/2949386907 1 ==== command_reply(tid 0: 0 ) ==== 8+0+4546 (crc 0 0 0) 0x5645a7c22850 con 0x5645a7907070 2024-09-21T03:53:34.029 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:34.022+0000 7f2b45ffb640 10 client.17559 handle_command_reply: tid=0 2024-09-21T03:53:34.029 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:34.022+0000 7f2b6b25b740 1 client.17559 shutdown 2024-09-21T03:53:34.029 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:34.022+0000 7f2b6b25b740 1 -- 172.21.15.39:0/4067176697 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5645a78de1b0 msgr2=0x5645a7cbe020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:53:34.029 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:34.022+0000 7f2b6b25b740 1 --2- 172.21.15.39:0/4067176697 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5645a78de1b0 0x5645a7cbe020 secure :-1 s=READY pgs=1090 cs=0 l=1 rev1=1 crypto rx=0x7f2b54005880 tx=0x7f2b540052f0 comp rx=0 tx=0).stop 2024-09-21T03:53:34.029 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:34.022+0000 7f2b6b25b740 1 -- 172.21.15.39:0/4067176697 shutdown_connections 2024-09-21T03:53:34.029 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:34.022+0000 7f2b6b25b740 1 --2- 172.21.15.39:0/4067176697 >> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] conn(0x5645a7907070 0x5645a7d39110 crc :-1 s=READY pgs=132 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:34.029 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:34.022+0000 7f2b6b25b740 1 --2- 172.21.15.39:0/4067176697 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5645a78e6390 0x5645a7cbe730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:34.030 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:34.022+0000 7f2b6b25b740 1 --2- 172.21.15.39:0/4067176697 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5645a78de1b0 0x5645a7cbe020 unknown :-1 s=CLOSED pgs=1090 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:34.030 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:34.022+0000 7f2b45ffb640 0 client.17559 ms_handle_reset on v2:172.21.15.150:6838/2949386907 2024-09-21T03:53:34.030 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:34.022+0000 7f2b6b25b740 1 --2- 172.21.15.39:0/4067176697 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5645a78c59f0 0x5645a7cc2b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:34.030 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:34.022+0000 7f2b6b25b740 1 -- 172.21.15.39:0/4067176697 >> 172.21.15.39:0/4067176697 conn(0x5645a78fb460 msgr2=0x5645a7cb2110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:53:34.030 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:34.022+0000 7f2b6b25b740 1 -- 172.21.15.39:0/4067176697 shutdown_connections 2024-09-21T03:53:34.030 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:34.022+0000 7f2b6b25b740 1 -- 172.21.15.39:0/4067176697 wait complete. 2024-09-21T03:53:34.030 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:34.023+0000 7f2b6b25b740 20 client.17559 trim_cache size 0 max 16384 2024-09-21T03:53:34.030 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-09-21T03:53:34.030 INFO:teuthology.orchestra.run.smithi039.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-21T03:53:34.030 INFO:teuthology.orchestra.run.smithi039.stdout: "scrubs": { 2024-09-21T03:53:34.031 INFO:teuthology.orchestra.run.smithi039.stdout: "d9391d69-baac-453a-9d52-d046760c070d": { 2024-09-21T03:53:34.031 INFO:teuthology.orchestra.run.smithi039.stdout: "path": "/", 2024-09-21T03:53:34.031 INFO:teuthology.orchestra.run.smithi039.stdout: "tag": "d9391d69-baac-453a-9d52-d046760c070d", 2024-09-21T03:53:34.031 INFO:teuthology.orchestra.run.smithi039.stdout: "options": "recursive,force" 2024-09-21T03:53:34.031 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-21T03:53:34.031 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:53:34.031 INFO:teuthology.orchestra.run.smithi039.stdout: "scrub_stats": { 2024-09-21T03:53:34.031 INFO:teuthology.orchestra.run.smithi039.stdout: "ad87da79-0248-4e18-9694-8228cb95422f": { 2024-09-21T03:53:34.031 INFO:teuthology.orchestra.run.smithi039.stdout: "start_time": "2024-09-21T03:53:18.441907+0000", 2024-09-21T03:53:34.031 INFO:teuthology.orchestra.run.smithi039.stdout: "path": "/", 2024-09-21T03:53:34.031 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_started": 0, 2024-09-21T03:53:34.031 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_passed": 0, 2024-09-21T03:53:34.031 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_failed": 0, 2024-09-21T03:53:34.032 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_skipped": 0 2024-09-21T03:53:34.032 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:53:34.032 INFO:teuthology.orchestra.run.smithi039.stdout: "02509e4f-3911-4983-b16f-df8ded0b9998": { 2024-09-21T03:53:34.032 INFO:teuthology.orchestra.run.smithi039.stdout: "start_time": "2024-09-21T03:46:17.889576+0000", 2024-09-21T03:53:34.032 INFO:teuthology.orchestra.run.smithi039.stdout: "path": "/", 2024-09-21T03:53:34.032 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_started": 0, 2024-09-21T03:53:34.032 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_passed": 0, 2024-09-21T03:53:34.032 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_failed": 0, 2024-09-21T03:53:34.032 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_skipped": 0 2024-09-21T03:53:34.032 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:53:34.032 INFO:teuthology.orchestra.run.smithi039.stdout: "10728de8-c49d-4edc-be2b-639249255e94": { 2024-09-21T03:53:34.032 INFO:teuthology.orchestra.run.smithi039.stdout: "start_time": "2024-09-21T03:47:03.965623+0000", 2024-09-21T03:53:34.032 INFO:teuthology.orchestra.run.smithi039.stdout: "path": "/", 2024-09-21T03:53:34.032 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_started": 0, 2024-09-21T03:53:34.033 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_passed": 0, 2024-09-21T03:53:34.033 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_failed": 0, 2024-09-21T03:53:34.033 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_skipped": 0 2024-09-21T03:53:34.033 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:53:34.033 INFO:teuthology.orchestra.run.smithi039.stdout: "9860c2f4-b7b1-47aa-865a-9a1c4434a5f6": { 2024-09-21T03:53:34.033 INFO:teuthology.orchestra.run.smithi039.stdout: "start_time": "2024-09-21T03:49:17.718867+0000", 2024-09-21T03:53:34.033 INFO:teuthology.orchestra.run.smithi039.stdout: "path": "/", 2024-09-21T03:53:34.033 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_started": 0, 2024-09-21T03:53:34.033 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_passed": 0, 2024-09-21T03:53:34.033 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_failed": 0, 2024-09-21T03:53:34.033 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_skipped": 0 2024-09-21T03:53:34.033 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:53:34.033 INFO:teuthology.orchestra.run.smithi039.stdout: "d9391d69-baac-453a-9d52-d046760c070d": { 2024-09-21T03:53:34.033 INFO:teuthology.orchestra.run.smithi039.stdout: "start_time": "2024-09-21T03:53:32.364622+0000", 2024-09-21T03:53:34.034 INFO:teuthology.orchestra.run.smithi039.stdout: "path": "/", 2024-09-21T03:53:34.034 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_started": 0, 2024-09-21T03:53:34.034 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_passed": 0, 2024-09-21T03:53:34.034 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_failed": 0, 2024-09-21T03:53:34.034 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_skipped": 0 2024-09-21T03:53:34.034 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:53:34.034 INFO:teuthology.orchestra.run.smithi039.stdout: "c7c700f3-2cd8-41ac-8381-b57dd1b37b02": { 2024-09-21T03:53:34.034 INFO:teuthology.orchestra.run.smithi039.stdout: "start_time": "2024-09-21T03:52:32.368858+0000", 2024-09-21T03:53:34.034 INFO:teuthology.orchestra.run.smithi039.stdout: "path": "/", 2024-09-21T03:53:34.034 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_started": 0, 2024-09-21T03:53:34.034 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_passed": 0, 2024-09-21T03:53:34.034 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_failed": 0, 2024-09-21T03:53:34.034 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_skipped": 0 2024-09-21T03:53:34.034 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:53:34.035 INFO:teuthology.orchestra.run.smithi039.stdout: "29411307-eb8d-4339-8d2b-47ebd32c0080": { 2024-09-21T03:53:34.035 INFO:teuthology.orchestra.run.smithi039.stdout: "start_time": "2024-09-21T03:47:38.092920+0000", 2024-09-21T03:53:34.035 INFO:teuthology.orchestra.run.smithi039.stdout: "path": "/", 2024-09-21T03:53:34.035 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_started": 0, 2024-09-21T03:53:34.035 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_passed": 0, 2024-09-21T03:53:34.035 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_failed": 0, 2024-09-21T03:53:34.035 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_skipped": 0 2024-09-21T03:53:34.035 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:53:34.035 INFO:teuthology.orchestra.run.smithi039.stdout: "ede73e6e-3fd0-4a7f-b5bb-f68946f57eea": { 2024-09-21T03:53:34.035 INFO:teuthology.orchestra.run.smithi039.stdout: "start_time": "2024-09-21T03:51:32.235096+0000", 2024-09-21T03:53:34.035 INFO:teuthology.orchestra.run.smithi039.stdout: "path": "/", 2024-09-21T03:53:34.035 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_started": 0, 2024-09-21T03:53:34.035 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_passed": 0, 2024-09-21T03:53:34.035 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_failed": 0, 2024-09-21T03:53:34.036 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_skipped": 0 2024-09-21T03:53:34.036 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:53:34.036 INFO:teuthology.orchestra.run.smithi039.stdout: "e28ad743-2b22-45b3-8799-a638c2883623": { 2024-09-21T03:53:34.036 INFO:teuthology.orchestra.run.smithi039.stdout: "start_time": "2024-09-21T03:48:31.448821+0000", 2024-09-21T03:53:34.036 INFO:teuthology.orchestra.run.smithi039.stdout: "path": "/", 2024-09-21T03:53:34.036 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_started": 0, 2024-09-21T03:53:34.036 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_passed": 0, 2024-09-21T03:53:34.036 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_failed": 0, 2024-09-21T03:53:34.036 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_skipped": 0 2024-09-21T03:53:34.036 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:53:34.036 INFO:teuthology.orchestra.run.smithi039.stdout: "6e760f1f-9048-4f92-bd66-d708fbef75b3": { 2024-09-21T03:53:34.036 INFO:teuthology.orchestra.run.smithi039.stdout: "start_time": "2024-09-21T03:50:03.711918+0000", 2024-09-21T03:53:34.036 INFO:teuthology.orchestra.run.smithi039.stdout: "path": "/", 2024-09-21T03:53:34.036 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_started": 0, 2024-09-21T03:53:34.037 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_passed": 0, 2024-09-21T03:53:34.037 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_failed": 0, 2024-09-21T03:53:34.037 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_skipped": 0 2024-09-21T03:53:34.037 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:53:34.037 INFO:teuthology.orchestra.run.smithi039.stdout: "c7e2b8e4-057d-435f-b2bd-b633de98035f": { 2024-09-21T03:53:34.037 INFO:teuthology.orchestra.run.smithi039.stdout: "start_time": "2024-09-21T03:47:19.400305+0000", 2024-09-21T03:53:34.037 INFO:teuthology.orchestra.run.smithi039.stdout: "path": "/", 2024-09-21T03:53:34.037 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_started": 0, 2024-09-21T03:53:34.037 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_passed": 0, 2024-09-21T03:53:34.037 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_failed": 0, 2024-09-21T03:53:34.037 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_skipped": 0 2024-09-21T03:53:34.037 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:53:34.037 INFO:teuthology.orchestra.run.smithi039.stdout: "74ab917e-ecea-4778-a1c3-c184214fb24c": { 2024-09-21T03:53:34.037 INFO:teuthology.orchestra.run.smithi039.stdout: "start_time": "2024-09-21T03:50:17.726630+0000", 2024-09-21T03:53:34.038 INFO:teuthology.orchestra.run.smithi039.stdout: "path": "/", 2024-09-21T03:53:34.038 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_started": 0, 2024-09-21T03:53:34.038 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_passed": 0, 2024-09-21T03:53:34.038 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_failed": 0, 2024-09-21T03:53:34.038 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_skipped": 0 2024-09-21T03:53:34.038 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:53:34.038 INFO:teuthology.orchestra.run.smithi039.stdout: "160750d8-401f-4810-866d-e63e7e4bc3e0": { 2024-09-21T03:53:34.038 INFO:teuthology.orchestra.run.smithi039.stdout: "start_time": "2024-09-21T03:51:04.208673+0000", 2024-09-21T03:53:34.038 INFO:teuthology.orchestra.run.smithi039.stdout: "path": "/", 2024-09-21T03:53:34.038 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_started": 0, 2024-09-21T03:53:34.038 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_passed": 0, 2024-09-21T03:53:34.038 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_failed": 0, 2024-09-21T03:53:34.038 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_skipped": 0 2024-09-21T03:53:34.039 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:53:34.039 INFO:teuthology.orchestra.run.smithi039.stdout: "ae1fc4cf-5458-47d0-afcb-282b136dba15": { 2024-09-21T03:53:34.039 INFO:teuthology.orchestra.run.smithi039.stdout: "start_time": "2024-09-21T03:51:18.560378+0000", 2024-09-21T03:53:34.039 INFO:teuthology.orchestra.run.smithi039.stdout: "path": "/", 2024-09-21T03:53:34.039 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_started": 0, 2024-09-21T03:53:34.039 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_passed": 0, 2024-09-21T03:53:34.039 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_failed": 0, 2024-09-21T03:53:34.039 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_skipped": 0 2024-09-21T03:53:34.039 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:53:34.039 INFO:teuthology.orchestra.run.smithi039.stdout: "4bda6ee3-7237-4ee4-96ce-150ad50f26dc": { 2024-09-21T03:53:34.039 INFO:teuthology.orchestra.run.smithi039.stdout: "start_time": "2024-09-21T03:52:18.622141+0000", 2024-09-21T03:53:34.039 INFO:teuthology.orchestra.run.smithi039.stdout: "path": "/", 2024-09-21T03:53:34.039 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_started": 0, 2024-09-21T03:53:34.039 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_passed": 0, 2024-09-21T03:53:34.040 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_failed": 0, 2024-09-21T03:53:34.040 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_skipped": 0 2024-09-21T03:53:34.040 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-21T03:53:34.040 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-21T03:53:34.040 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-09-21T03:53:34.040 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:34.023+0000 7f2b64ce7640 1 -- 172.21.15.39:0/369378574 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f2b480b6680 msgr2=0x7f2b480b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:53:34.040 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:34.023+0000 7f2b64ce7640 1 --2- 172.21.15.39:0/369378574 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f2b480b6680 0x7f2b480b8b90 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x5645a7cad2d0 tx=0x7f2b5000a2d0 comp rx=0 tx=0).stop 2024-09-21T03:53:34.041 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:34.023+0000 7f2b64ce7640 1 -- 172.21.15.39:0/369378574 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2b60108cf0 msgr2=0x7f2b601000f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:53:34.041 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:34.023+0000 7f2b64ce7640 1 --2- 172.21.15.39:0/369378574 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2b60108cf0 0x7f2b601000f0 secure :-1 s=READY pgs=1086 cs=0 l=1 rev1=1 crypto rx=0x7f2b58005cc0 tx=0x7f2b5802f470 comp rx=0 tx=0).stop 2024-09-21T03:53:34.041 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:34.023+0000 7f2b64ce7640 1 -- 172.21.15.39:0/369378574 shutdown_connections 2024-09-21T03:53:34.041 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:34.023+0000 7f2b64ce7640 1 --2- 172.21.15.39:0/369378574 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f2b480b6680 0x7f2b480b8b90 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:34.041 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:34.023+0000 7f2b64ce7640 1 --2- 172.21.15.39:0/369378574 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2b60108cf0 0x7f2b601000f0 unknown :-1 s=CLOSED pgs=1086 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:34.041 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:34.023+0000 7f2b64ce7640 1 --2- 172.21.15.39:0/369378574 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2b601083a0 0x7f2b600ffbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:34.041 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:34.023+0000 7f2b64ce7640 1 --2- 172.21.15.39:0/369378574 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2b60102a60 0x7f2b600ff670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:34.041 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:34.023+0000 7f2b64ce7640 1 -- 172.21.15.39:0/369378574 >> 172.21.15.39:0/369378574 conn(0x7f2b60079730 msgr2=0x7f2b6006b190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:53:34.041 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:34.023+0000 7f2b64ce7640 1 -- 172.21.15.39:0/369378574 shutdown_connections 2024-09-21T03:53:34.041 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:53:34.023+0000 7f2b64ce7640 1 -- 172.21.15.39:0/369378574 wait complete. 2024-09-21T03:53:34.218 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:53:33 smithi161 ceph-mon[34533]: scrub queued for path: / 2024-09-21T03:53:34.218 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:53:33 smithi161 ceph-mon[34533]: scrub summary: idle+waiting paths [/] 2024-09-21T03:53:34.218 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:53:33 smithi161 ceph-mon[34533]: scrub summary: active paths [/] 2024-09-21T03:53:34.218 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:53:33 smithi161 ceph-mon[34533]: scrub summary: idle+waiting paths [/] 2024-09-21T03:53:34.225 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:53:33 smithi150 ceph-mon[34289]: scrub queued for path: / 2024-09-21T03:53:34.225 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:53:33 smithi150 ceph-mon[34289]: scrub summary: idle+waiting paths [/] 2024-09-21T03:53:34.225 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:53:33 smithi150 ceph-mon[34289]: scrub summary: active paths [/] 2024-09-21T03:53:34.225 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:53:33 smithi150 ceph-mon[34289]: scrub summary: idle+waiting paths [/] 2024-09-21T03:53:34.333 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:53:33 smithi039 ceph-mon[31211]: scrub queued for path: / 2024-09-21T03:53:34.334 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:53:33 smithi039 ceph-mon[31211]: scrub summary: idle+waiting paths [/] 2024-09-21T03:53:34.334 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:53:33 smithi039 ceph-mon[31211]: scrub summary: active paths [/] 2024-09-21T03:53:34.334 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:53:33 smithi039 ceph-mon[31211]: scrub summary: idle+waiting paths [/] 2024-09-21T03:53:34.957 INFO:tasks.cephfs.filesystem:scrub status for tag:d9391d69-baac-453a-9d52-d046760c070d - {'path': '/', 'tag': 'd9391d69-baac-453a-9d52-d046760c070d', 'options': 'recursive,force'} 2024-09-21T03:53:35.218 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:53:34 smithi161 ceph-mon[34533]: pgmap v553: 97 pgs: 97 active+clean; 10 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 10 op/s 2024-09-21T03:53:35.224 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:53:34 smithi150 ceph-mon[34289]: pgmap v553: 97 pgs: 97 active+clean; 10 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 10 op/s 2024-09-21T03:53:35.333 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:53:34 smithi039 ceph-mon[31211]: pgmap v553: 97 pgs: 97 active+clean; 10 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 10 op/s 2024-09-21T03:53:36.218 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:53:35 smithi161 ceph-mon[34533]: scrub summary: idle 2024-09-21T03:53:36.224 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:53:35 smithi150 ceph-mon[34289]: scrub summary: idle 2024-09-21T03:53:36.334 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:53:35 smithi039 ceph-mon[31211]: scrub summary: idle 2024-09-21T03:53:37.218 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:53:36 smithi161 ceph-mon[34533]: pgmap v554: 97 pgs: 97 active+clean; 10 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 10 op/s 2024-09-21T03:53:37.225 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:53:36 smithi150 ceph-mon[34289]: pgmap v554: 97 pgs: 97 active+clean; 10 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 10 op/s 2024-09-21T03:53:37.334 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:53:36 smithi039 ceph-mon[31211]: pgmap v554: 97 pgs: 97 active+clean; 10 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 10 op/s 2024-09-21T03:53:39.218 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:53:38 smithi161 ceph-mon[34533]: pgmap v555: 97 pgs: 97 active+clean; 10 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 8 op/s 2024-09-21T03:53:39.224 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:53:38 smithi150 ceph-mon[34289]: pgmap v555: 97 pgs: 97 active+clean; 10 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 8 op/s 2024-09-21T03:53:39.334 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:53:38 smithi039 ceph-mon[31211]: pgmap v555: 97 pgs: 97 active+clean; 10 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 8 op/s 2024-09-21T03:53:41.218 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:53:40 smithi161 ceph-mon[34533]: pgmap v556: 97 pgs: 97 active+clean; 10 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 11 op/s 2024-09-21T03:53:41.225 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:53:40 smithi150 ceph-mon[34289]: pgmap v556: 97 pgs: 97 active+clean; 10 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 11 op/s 2024-09-21T03:53:41.334 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:53:40 smithi039 ceph-mon[31211]: pgmap v556: 97 pgs: 97 active+clean; 10 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 11 op/s 2024-09-21T03:53:43.218 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:53:42 smithi161 ceph-mon[34533]: pgmap v557: 97 pgs: 97 active+clean; 10 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 9 op/s 2024-09-21T03:53:43.224 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:53:42 smithi150 ceph-mon[34289]: pgmap v557: 97 pgs: 97 active+clean; 10 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 9 op/s 2024-09-21T03:53:43.334 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:53:42 smithi039 ceph-mon[31211]: pgmap v557: 97 pgs: 97 active+clean; 10 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 9 op/s 2024-09-21T03:53:45.218 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:53:44 smithi161 ceph-mon[34533]: pgmap v558: 97 pgs: 97 active+clean; 10 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-09-21T03:53:45.225 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:53:44 smithi150 ceph-mon[34289]: pgmap v558: 97 pgs: 97 active+clean; 10 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-09-21T03:53:45.334 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:53:44 smithi039 ceph-mon[31211]: pgmap v558: 97 pgs: 97 active+clean; 10 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-09-21T03:53:47.218 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:53:46 smithi161 ceph-mon[34533]: pgmap v559: 97 pgs: 97 active+clean; 10 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 10 op/s 2024-09-21T03:53:47.225 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:53:46 smithi150 ceph-mon[34289]: pgmap v559: 97 pgs: 97 active+clean; 10 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 10 op/s 2024-09-21T03:53:47.334 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:53:46 smithi039 ceph-mon[31211]: pgmap v559: 97 pgs: 97 active+clean; 10 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 10 op/s 2024-09-21T03:53:49.218 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:53:48 smithi161 ceph-mon[34533]: pgmap v560: 97 pgs: 97 active+clean; 10 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 8 op/s 2024-09-21T03:53:49.224 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:53:48 smithi150 ceph-mon[34289]: pgmap v560: 97 pgs: 97 active+clean; 10 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 8 op/s 2024-09-21T03:53:49.334 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:53:48 smithi039 ceph-mon[31211]: pgmap v560: 97 pgs: 97 active+clean; 10 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 8 op/s 2024-09-21T03:53:51.218 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:53:50 smithi161 ceph-mon[34533]: pgmap v561: 97 pgs: 97 active+clean; 10 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 11 op/s 2024-09-21T03:53:51.224 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:53:50 smithi150 ceph-mon[34289]: pgmap v561: 97 pgs: 97 active+clean; 10 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 11 op/s 2024-09-21T03:53:51.334 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:53:50 smithi039 ceph-mon[31211]: pgmap v561: 97 pgs: 97 active+clean; 10 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 11 op/s 2024-09-21T03:53:53.218 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:53:52 smithi161 ceph-mon[34533]: pgmap v562: 97 pgs: 97 active+clean; 10 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 8 op/s 2024-09-21T03:53:53.225 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:53:52 smithi150 ceph-mon[34289]: pgmap v562: 97 pgs: 97 active+clean; 10 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 8 op/s 2024-09-21T03:53:53.334 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:53:52 smithi039 ceph-mon[31211]: pgmap v562: 97 pgs: 97 active+clean; 10 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 8 op/s 2024-09-21T03:53:55.218 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:53:54 smithi161 ceph-mon[34533]: pgmap v563: 97 pgs: 97 active+clean; 10 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 10 op/s 2024-09-21T03:53:55.224 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:53:54 smithi150 ceph-mon[34289]: pgmap v563: 97 pgs: 97 active+clean; 10 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 10 op/s 2024-09-21T03:53:55.334 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:53:54 smithi039 ceph-mon[31211]: pgmap v563: 97 pgs: 97 active+clean; 10 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 10 op/s 2024-09-21T03:53:57.218 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:53:56 smithi161 ceph-mon[34533]: pgmap v564: 97 pgs: 97 active+clean; 10 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 10 op/s 2024-09-21T03:53:57.225 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:53:56 smithi150 ceph-mon[34289]: pgmap v564: 97 pgs: 97 active+clean; 10 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 10 op/s 2024-09-21T03:53:57.333 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:53:56 smithi039 ceph-mon[31211]: pgmap v564: 97 pgs: 97 active+clean; 10 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 10 op/s 2024-09-21T03:53:59.218 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:53:58 smithi161 ceph-mon[34533]: pgmap v565: 97 pgs: 97 active+clean; 10 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-09-21T03:53:59.224 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:53:58 smithi150 ceph-mon[34289]: pgmap v565: 97 pgs: 97 active+clean; 10 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-09-21T03:53:59.334 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:53:58 smithi039 ceph-mon[31211]: pgmap v565: 97 pgs: 97 active+clean; 10 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-09-21T03:54:01.218 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:54:00 smithi161 ceph-mon[34533]: pgmap v566: 97 pgs: 97 active+clean; 10 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-09-21T03:54:01.224 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:54:00 smithi150 ceph-mon[34289]: pgmap v566: 97 pgs: 97 active+clean; 10 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-09-21T03:54:01.334 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:54:00 smithi039 ceph-mon[31211]: pgmap v566: 97 pgs: 97 active+clean; 10 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-09-21T03:54:03.218 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:54:02 smithi161 ceph-mon[34533]: pgmap v567: 97 pgs: 97 active+clean; 10 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-21T03:54:03.225 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:54:02 smithi150 ceph-mon[34289]: pgmap v567: 97 pgs: 97 active+clean; 10 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-21T03:54:03.333 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:54:02 smithi039 ceph-mon[31211]: pgmap v567: 97 pgs: 97 active+clean; 10 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-21T03:54:04.958 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 51116d5c-77ca-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub status 2024-09-21T03:54:05.218 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:54:04 smithi161 ceph-mon[34533]: pgmap v568: 97 pgs: 97 active+clean; 10 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 597 B/s wr, 10 op/s 2024-09-21T03:54:05.225 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:54:04 smithi150 ceph-mon[34289]: pgmap v568: 97 pgs: 97 active+clean; 10 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 597 B/s wr, 10 op/s 2024-09-21T03:54:05.225 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/51116d5c-77ca-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:54:05.334 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:54:04 smithi039 ceph-mon[31211]: pgmap v568: 97 pgs: 97 active+clean; 10 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 597 B/s wr, 10 op/s 2024-09-21T03:54:05.717 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:05.709+0000 7f95a1e6b640 1 -- 172.21.15.39:0/3841792725 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f959c1083a0 msgr2=0x7f959c1087b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:54:05.718 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:05.709+0000 7f95a1e6b640 1 --2- 172.21.15.39:0/3841792725 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f959c1083a0 0x7f959c1087b0 secure :-1 s=READY pgs=1091 cs=0 l=1 rev1=1 crypto rx=0x7f9584009900 tx=0x7f958402a410 comp rx=0 tx=0).stop 2024-09-21T03:54:05.718 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:05.709+0000 7f95a1e6b640 1 -- 172.21.15.39:0/3841792725 shutdown_connections 2024-09-21T03:54:05.718 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:05.709+0000 7f95a1e6b640 1 --2- 172.21.15.39:0/3841792725 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f959c108cf0 0x7f959c10f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:05.718 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:05.709+0000 7f95a1e6b640 1 --2- 172.21.15.39:0/3841792725 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f959c1083a0 0x7f959c1087b0 unknown :-1 s=CLOSED pgs=1091 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:05.718 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:05.709+0000 7f95a1e6b640 1 --2- 172.21.15.39:0/3841792725 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f959c102a60 0x7f959c102e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:05.718 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:05.709+0000 7f95a1e6b640 1 -- 172.21.15.39:0/3841792725 >> 172.21.15.39:0/3841792725 conn(0x7f959c079730 msgr2=0x7f959c079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:54:05.718 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:05.709+0000 7f95a1e6b640 1 -- 172.21.15.39:0/3841792725 shutdown_connections 2024-09-21T03:54:05.718 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:05.709+0000 7f95a1e6b640 1 -- 172.21.15.39:0/3841792725 wait complete. 2024-09-21T03:54:05.718 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:05.710+0000 7f95a1e6b640 1 Processor -- start 2024-09-21T03:54:05.718 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:05.710+0000 7f95a1e6b640 1 -- start start 2024-09-21T03:54:05.718 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:05.710+0000 7f95a1e6b640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f959c102a60 0x7f959c199330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:54:05.718 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:05.710+0000 7f95a1e6b640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f959c1083a0 0x7f959c1998d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:54:05.718 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:05.710+0000 7f95a1e6b640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f959c108cf0 0x7f959c19eb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:54:05.719 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:05.710+0000 7f95a1e6b640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f959c112960 con 0x7f959c108cf0 2024-09-21T03:54:05.719 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:05.710+0000 7f95a1e6b640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f959c1127e0 con 0x7f959c102a60 2024-09-21T03:54:05.719 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:05.710+0000 7f95a1e6b640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f959c112ae0 con 0x7f959c1083a0 2024-09-21T03:54:05.719 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:05.710+0000 7f959bfff640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f959c108cf0 0x7f959c19eb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:54:05.719 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:05.710+0000 7f959bfff640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f959c108cf0 0x7f959c19eb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:34882/0 (socket says 172.21.15.39:34882) 2024-09-21T03:54:05.719 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:05.710+0000 7f959bfff640 1 -- 172.21.15.39:0/1530932116 learned_addr learned my addr 172.21.15.39:0/1530932116 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:54:05.719 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:05.711+0000 7f959b7fe640 1 --2- 172.21.15.39:0/1530932116 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f959c102a60 0x7f959c199330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:54:05.719 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:05.711+0000 7f959affd640 1 --2- 172.21.15.39:0/1530932116 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f959c1083a0 0x7f959c1998d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:54:05.719 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:05.711+0000 7f959bfff640 1 -- 172.21.15.39:0/1530932116 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f959c1083a0 msgr2=0x7f959c1998d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:54:05.719 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:05.711+0000 7f959bfff640 1 --2- 172.21.15.39:0/1530932116 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f959c1083a0 0x7f959c1998d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:05.719 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:05.711+0000 7f959bfff640 1 -- 172.21.15.39:0/1530932116 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f959c102a60 msgr2=0x7f959c199330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:54:05.719 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:05.711+0000 7f959bfff640 1 --2- 172.21.15.39:0/1530932116 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f959c102a60 0x7f959c199330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:05.719 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:05.711+0000 7f959bfff640 1 -- 172.21.15.39:0/1530932116 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f959c19f1e0 con 0x7f959c108cf0 2024-09-21T03:54:05.720 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:05.711+0000 7f959bfff640 1 --2- 172.21.15.39:0/1530932116 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f959c108cf0 0x7f959c19eb70 secure :-1 s=READY pgs=1092 cs=0 l=1 rev1=1 crypto rx=0x7f959400c970 tx=0x7f959400ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:54:05.720 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:05.711+0000 7f9598ff9640 1 -- 172.21.15.39:0/1530932116 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9594007bb0 con 0x7f959c108cf0 2024-09-21T03:54:05.720 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:05.711+0000 7f9598ff9640 1 -- 172.21.15.39:0/1530932116 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9594007d50 con 0x7f959c108cf0 2024-09-21T03:54:05.720 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:05.711+0000 7f95a1e6b640 1 -- 172.21.15.39:0/1530932116 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f959c19f4d0 con 0x7f959c108cf0 2024-09-21T03:54:05.720 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:05.711+0000 7f9598ff9640 1 -- 172.21.15.39:0/1530932116 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9594018770 con 0x7f959c108cf0 2024-09-21T03:54:05.720 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:05.711+0000 7f95a1e6b640 1 -- 172.21.15.39:0/1530932116 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f959c19f990 con 0x7f959c108cf0 2024-09-21T03:54:05.720 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:05.714+0000 7f95a83df740 1 Processor -- start 2024-09-21T03:54:05.720 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:05.714+0000 7f9598ff9640 1 -- 172.21.15.39:0/1530932116 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 27) ==== 154643+0+0 (secure 0 0 0) 0x7f959400f990 con 0x7f959c108cf0 2024-09-21T03:54:05.720 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:05.714+0000 7f95a83df740 1 -- start start 2024-09-21T03:54:05.720 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:05.714+0000 7f95a83df740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5573a7814070 0x5573a7bbb650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:54:05.720 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:05.714+0000 7f95a83df740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5573a77eb1b0 0x5573a7bbbb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:54:05.720 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:05.714+0000 7f95a83df740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5573a77f3390 0x5573a7bbc0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:54:05.720 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:05.714+0000 7f95a83df740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x5573a7b31e50 con 0x5573a7814070 2024-09-21T03:54:05.720 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:05.714+0000 7f95a83df740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x5573a7b21280 con 0x5573a77f3390 2024-09-21T03:54:05.721 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:05.714+0000 7f95a83df740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x5573a7b25090 con 0x5573a77eb1b0 2024-09-21T03:54:05.721 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:05.715+0000 7f959affd640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5573a7814070 0x5573a7bbb650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:54:05.721 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:05.715+0000 7f959affd640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5573a7814070 0x5573a7bbb650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:34886/0 (socket says 172.21.15.39:34886) 2024-09-21T03:54:05.721 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:05.715+0000 7f959affd640 1 -- 172.21.15.39:0/2954534957 learned_addr learned my addr 172.21.15.39:0/2954534957 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:54:05.721 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:05.715+0000 7f959b7fe640 1 --2- 172.21.15.39:0/2954534957 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5573a77eb1b0 0x5573a7bbbb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:54:05.721 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:05.715+0000 7f959affd640 1 --2- 172.21.15.39:0/2954534957 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5573a77f3390 0x5573a7bbc0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:54:05.721 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:05.715+0000 7f9598ff9640 1 --2- 172.21.15.39:0/1530932116 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f95880b66b0 0x7f95880b8bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:54:05.721 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:05.715+0000 7f959affd640 1 -- 172.21.15.39:0/2954534957 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5573a77eb1b0 msgr2=0x5573a7bbbb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:54:05.721 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:05.715+0000 7f959affd640 1 --2- 172.21.15.39:0/2954534957 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5573a77eb1b0 0x5573a7bbbb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:05.721 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:05.715+0000 7f959affd640 1 -- 172.21.15.39:0/2954534957 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5573a77f3390 msgr2=0x5573a7bbc0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:54:05.721 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:05.715+0000 7f959affd640 1 --2- 172.21.15.39:0/2954534957 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5573a77f3390 0x5573a7bbc0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:05.721 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:05.715+0000 7f959affd640 1 -- 172.21.15.39:0/2954534957 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5573a787f650 con 0x5573a7814070 2024-09-21T03:54:05.721 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:05.715+0000 7f959b7fe640 1 --2- 172.21.15.39:0/2954534957 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5573a77eb1b0 0x5573a7bbbb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:54:05.722 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:05.715+0000 7f959bfff640 1 --2- 172.21.15.39:0/1530932116 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f95880b66b0 0x7f95880b8bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:54:05.722 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:05.715+0000 7f959affd640 1 --2- 172.21.15.39:0/2954534957 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5573a7814070 0x5573a7bbb650 secure :-1 s=READY pgs=1093 cs=0 l=1 rev1=1 crypto rx=0x7f9584009900 tx=0x7f9584031b10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7aa571039ca97d35 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:54:05.722 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:05.715+0000 7f9598ff9640 1 -- 172.21.15.39:0/1530932116 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8528+0+0 (secure 0 0 0) 0x7f95940e7700 con 0x7f959c108cf0 2024-09-21T03:54:05.722 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:05.716+0000 7f9591ffb640 1 -- 172.21.15.39:0/2954534957 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f958403f070 con 0x5573a7814070 2024-09-21T03:54:05.722 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:05.716+0000 7f9591ffb640 1 -- 172.21.15.39:0/2954534957 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9584031910 con 0x5573a7814070 2024-09-21T03:54:05.722 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:05.716+0000 7f9591ffb640 1 -- 172.21.15.39:0/2954534957 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9584035040 con 0x5573a7814070 2024-09-21T03:54:05.725 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:05.717+0000 7f95a83df740 1 -- 172.21.15.39:0/2954534957 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5573a7814070 msgr2=0x5573a7bbb650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:54:05.725 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:05.717+0000 7f959bfff640 1 --2- 172.21.15.39:0/1530932116 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f95880b66b0 0x7f95880b8bc0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f959400afd0 tx=0x7f9594071040 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:54:05.726 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:05.717+0000 7f95a83df740 1 --2- 172.21.15.39:0/2954534957 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5573a7814070 0x5573a7bbb650 secure :-1 s=READY pgs=1093 cs=0 l=1 rev1=1 crypto rx=0x7f9584009900 tx=0x7f9584031b10 comp rx=0 tx=0).stop 2024-09-21T03:54:05.726 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:05.717+0000 7f95a83df740 1 -- 172.21.15.39:0/2954534957 shutdown_connections 2024-09-21T03:54:05.726 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:05.717+0000 7f95a83df740 1 --2- 172.21.15.39:0/2954534957 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5573a77f3390 0x5573a7bbc0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:05.726 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:05.717+0000 7f95a83df740 1 --2- 172.21.15.39:0/2954534957 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5573a77eb1b0 0x5573a7bbbb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:05.726 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:05.717+0000 7f95a83df740 1 --2- 172.21.15.39:0/2954534957 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5573a7814070 0x5573a7bbb650 unknown :-1 s=CLOSED pgs=1093 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:05.726 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:05.717+0000 7f95a83df740 1 -- 172.21.15.39:0/2954534957 >> 172.21.15.39:0/2954534957 conn(0x5573a7808460 msgr2=0x5573a7bb9540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:54:05.726 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:05.717+0000 7f95a83df740 1 -- 172.21.15.39:0/2954534957 shutdown_connections 2024-09-21T03:54:05.726 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:05.717+0000 7f95a83df740 1 -- 172.21.15.39:0/2954534957 wait complete. 2024-09-21T03:54:05.726 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:05.717+0000 7f95a83df740 1 -- start start 2024-09-21T03:54:05.726 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:05.717+0000 7f95a83df740 1 Processor -- start 2024-09-21T03:54:05.726 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:05.718+0000 7f95a83df740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5573a77eb1b0 0x5573a7bc6d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:54:05.726 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:05.718+0000 7f95a83df740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5573a77f3390 0x5573a7bc7260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:54:05.726 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:05.718+0000 7f95a83df740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5573a7814070 0x5573a7bd30c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:54:05.727 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:05.718+0000 7f95a83df740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x5573a7982700 con 0x5573a77f3390 2024-09-21T03:54:05.727 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:05.718+0000 7f95a83df740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x5573a7abc110 con 0x5573a7814070 2024-09-21T03:54:05.727 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:05.718+0000 7f95a83df740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x5573a7bc9dd0 con 0x5573a77eb1b0 2024-09-21T03:54:05.727 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:05.718+0000 7f95a83df740 10 client.0 _ll_register_callbacks cb 0x5573a7ad5990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:54:05.727 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:05.718+0000 7f959b7fe640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5573a77f3390 0x5573a7bc7260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:54:05.727 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:05.718+0000 7f959b7fe640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5573a77f3390 0x5573a7bc7260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:34896/0 (socket says 172.21.15.39:34896) 2024-09-21T03:54:05.727 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:05.718+0000 7f959b7fe640 1 -- 172.21.15.39:0/1163934272 learned_addr learned my addr 172.21.15.39:0/1163934272 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:54:05.727 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:05.718+0000 7f959affd640 1 --2- 172.21.15.39:0/1163934272 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5573a7814070 0x5573a7bd30c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:54:05.727 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:05.718+0000 7f959affd640 1 --2- 172.21.15.39:0/1163934272 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5573a77eb1b0 0x5573a7bc6d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:54:05.727 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:05.718+0000 7f959b7fe640 1 -- 172.21.15.39:0/1163934272 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5573a77eb1b0 msgr2=0x5573a7bc6d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:54:05.727 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:05.718+0000 7f959b7fe640 1 --2- 172.21.15.39:0/1163934272 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5573a77eb1b0 0x5573a7bc6d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:05.727 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:05.718+0000 7f959b7fe640 1 -- 172.21.15.39:0/1163934272 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5573a7814070 msgr2=0x5573a7bd30c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:54:05.727 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:05.718+0000 7f959b7fe640 1 --2- 172.21.15.39:0/1163934272 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5573a7814070 0x5573a7bd30c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:05.727 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:05.718+0000 7f959b7fe640 1 -- 172.21.15.39:0/1163934272 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5573a7ae9c40 con 0x5573a77f3390 2024-09-21T03:54:05.728 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:05.718+0000 7f959b7fe640 1 --2- 172.21.15.39:0/1163934272 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5573a77f3390 0x5573a7bc7260 secure :-1 s=READY pgs=1094 cs=0 l=1 rev1=1 crypto rx=0x7f958c011ae0 tx=0x7f958c00f3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a6e96a8db101edcd server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:54:05.728 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:05.718+0000 7f957affd640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-09-21T03:54:05.728 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:05.719+0000 7f957affd640 1 -- 172.21.15.39:0/1163934272 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f958c004b20 con 0x5573a77f3390 2024-09-21T03:54:05.728 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:05.719+0000 7f957affd640 1 -- 172.21.15.39:0/1163934272 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8528+0+0 (secure 0 0 0) 0x7f958c004cc0 con 0x5573a77f3390 2024-09-21T03:54:05.728 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:05.719+0000 7f95a83df740 1 -- 172.21.15.39:0/1163934272 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9584031910 con 0x5573a77f3390 2024-09-21T03:54:05.728 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:05.719+0000 7f957affd640 1 -- 172.21.15.39:0/1163934272 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f958c005020 con 0x5573a77f3390 2024-09-21T03:54:05.728 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:05.719+0000 7f95a83df740 10 client.17571 fetch_fsmap learned FSMap version 29 2024-09-21T03:54:05.728 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:05.719+0000 7f95a83df740 1 -- 172.21.15.39:0/1163934272 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5573a7bd5b50 con 0x5573a77f3390 2024-09-21T03:54:05.728 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:05.719+0000 7f957affd640 1 -- 172.21.15.39:0/1163934272 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9584031910 con 0x5573a77f3390 2024-09-21T03:54:05.728 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:05.719+0000 7f957affd640 1 -- 172.21.15.39:0/1163934272 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f958c0235c0 con 0x5573a77f3390 2024-09-21T03:54:05.728 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:05.719+0000 7f95a83df740 10 client.17571 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T03:54:05.728 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:05.719+0000 7f95a83df740 10 client.17571 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T03:54:05.728 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:05.719+0000 7f95a83df740 20 client.17571 populate_metadata read hostname 'smithi039' 2024-09-21T03:54:05.728 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:05.719+0000 7f95a83df740 1 --2- 172.21.15.39:0/1163934272 >> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] conn(0x5573a77d29f0 0x5573a7bd9110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:54:05.729 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:05.719+0000 7f95a83df740 4 client.17571 mds_command: new command op to 24695 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:54:05.729 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:05.719+0000 7f95a83df740 1 -- 172.21.15.39:0/1163934272 --> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5573a7b33da0 con 0x5573a77d29f0 2024-09-21T03:54:05.729 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:05.720+0000 7f959affd640 1 --2- 172.21.15.39:0/1163934272 >> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] conn(0x5573a77d29f0 0x5573a7bd9110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:54:05.729 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:05.720+0000 7f957affd640 10 client.17571 ms_handle_connect on v2:172.21.15.150:6838/2949386907 2024-09-21T03:54:05.729 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:05.720+0000 7f959affd640 1 --2- 172.21.15.39:0/1163934272 >> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] conn(0x5573a77d29f0 0x5573a7bd9110 crc :-1 s=READY pgs=133 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ea8198291a56b988 server_cookie=5b25232d59bdd7be in_seq=0 out_seq=0 2024-09-21T03:54:05.729 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:05.721+0000 7f957affd640 1 -- 172.21.15.39:0/1163934272 <== mds.0 v2:172.21.15.150:6838/2949386907 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5573a7b33da0 con 0x5573a77d29f0 2024-09-21T03:54:05.729 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:05.721+0000 7f957affd640 10 client.17571 handle_command_reply: tid=0 2024-09-21T03:54:05.729 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:05.721+0000 7f95a83df740 1 client.17571 shutdown 2024-09-21T03:54:05.729 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:05.721+0000 7f95a83df740 1 -- 172.21.15.39:0/1163934272 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5573a77f3390 msgr2=0x5573a7bc7260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:54:05.729 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:05.721+0000 7f95a83df740 1 --2- 172.21.15.39:0/1163934272 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5573a77f3390 0x5573a7bc7260 secure :-1 s=READY pgs=1094 cs=0 l=1 rev1=1 crypto rx=0x7f958c011ae0 tx=0x7f958c00f3c0 comp rx=0 tx=0).stop 2024-09-21T03:54:05.729 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:05.721+0000 7f95a83df740 1 -- 172.21.15.39:0/1163934272 shutdown_connections 2024-09-21T03:54:05.729 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:05.721+0000 7f95a83df740 1 --2- 172.21.15.39:0/1163934272 >> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] conn(0x5573a77d29f0 0x5573a7bd9110 crc :-1 s=READY pgs=133 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:05.729 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:05.721+0000 7f95a83df740 1 --2- 172.21.15.39:0/1163934272 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5573a7814070 0x5573a7bd30c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:05.729 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:05.721+0000 7f95a83df740 1 --2- 172.21.15.39:0/1163934272 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5573a77f3390 0x5573a7bc7260 unknown :-1 s=CLOSED pgs=1094 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:05.729 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:05.721+0000 7f95a83df740 1 --2- 172.21.15.39:0/1163934272 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5573a77eb1b0 0x5573a7bc6d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:05.730 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:05.721+0000 7f95a83df740 1 -- 172.21.15.39:0/1163934272 >> 172.21.15.39:0/1163934272 conn(0x5573a7808460 msgr2=0x5573a7bbe830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:54:05.730 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:05.721+0000 7f957affd640 0 client.17571 ms_handle_reset on v2:172.21.15.150:6838/2949386907 2024-09-21T03:54:05.730 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:05.722+0000 7f95a83df740 1 -- 172.21.15.39:0/1163934272 shutdown_connections 2024-09-21T03:54:05.730 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:05.722+0000 7f95a83df740 1 -- 172.21.15.39:0/1163934272 wait complete. 2024-09-21T03:54:05.730 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:05.722+0000 7f95a83df740 20 client.17571 trim_cache size 0 max 16384 2024-09-21T03:54:05.742 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:05.739+0000 7f95a83df740 1 Processor -- start 2024-09-21T03:54:05.742 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:05.739+0000 7f95a83df740 1 -- start start 2024-09-21T03:54:05.742 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:05.739+0000 7f95a83df740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5573a77eb1b0 0x5573a7bc72a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:54:05.742 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:05.739+0000 7f95a83df740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5573a77f3390 0x5573a7bbee60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:54:05.742 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:05.739+0000 7f95a83df740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5573a7814070 0x5573a7bd2fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:54:05.742 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:05.739+0000 7f95a83df740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f959c1ad3e0 con 0x5573a7814070 2024-09-21T03:54:05.742 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:05.739+0000 7f95a83df740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f959c1aef20 con 0x5573a77eb1b0 2024-09-21T03:54:05.742 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:05.739+0000 7f95a83df740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f959c1adab0 con 0x5573a77f3390 2024-09-21T03:54:05.742 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:05.740+0000 7f959affd640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5573a7814070 0x5573a7bd2fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:54:05.743 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:05.740+0000 7f959affd640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5573a7814070 0x5573a7bd2fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:34902/0 (socket says 172.21.15.39:34902) 2024-09-21T03:54:05.743 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:05.740+0000 7f959affd640 1 -- 172.21.15.39:0/208715654 learned_addr learned my addr 172.21.15.39:0/208715654 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:54:05.743 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:05.740+0000 7f959affd640 1 --2- 172.21.15.39:0/208715654 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5573a77eb1b0 0x5573a7bc72a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:54:05.743 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:05.740+0000 7f959b7fe640 1 --2- 172.21.15.39:0/208715654 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5573a77f3390 0x5573a7bbee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:54:05.743 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:05.740+0000 7f959affd640 1 -- 172.21.15.39:0/208715654 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5573a77f3390 msgr2=0x5573a7bbee60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:54:05.743 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:05.740+0000 7f959affd640 1 --2- 172.21.15.39:0/208715654 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5573a77f3390 0x5573a7bbee60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:05.743 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:05.740+0000 7f959affd640 1 -- 172.21.15.39:0/208715654 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5573a77eb1b0 msgr2=0x5573a7bc72a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:54:05.743 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:05.740+0000 7f959affd640 1 --2- 172.21.15.39:0/208715654 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5573a77eb1b0 0x5573a7bc72a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:05.743 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:05.740+0000 7f959affd640 1 -- 172.21.15.39:0/208715654 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5573a77d2c20 con 0x5573a7814070 2024-09-21T03:54:05.743 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:05.740+0000 7f959b7fe640 1 --2- 172.21.15.39:0/208715654 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5573a77f3390 0x5573a7bbee60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:54:05.743 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:05.740+0000 7f959affd640 1 --2- 172.21.15.39:0/208715654 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5573a7814070 0x5573a7bd2fc0 secure :-1 s=READY pgs=1095 cs=0 l=1 rev1=1 crypto rx=0x7f9584005940 tx=0x7f9584031b10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb46657d884c523f server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:54:05.743 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:05.740+0000 7f95917fa640 1 -- 172.21.15.39:0/208715654 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f958405b070 con 0x5573a7814070 2024-09-21T03:54:05.743 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:05.740+0000 7f95917fa640 1 -- 172.21.15.39:0/208715654 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f95840318f0 con 0x5573a7814070 2024-09-21T03:54:05.743 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:05.740+0000 7f95917fa640 1 -- 172.21.15.39:0/208715654 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9584035040 con 0x5573a7814070 2024-09-21T03:54:05.747 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:05.741+0000 7f95a83df740 1 -- 172.21.15.39:0/208715654 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5573a7814070 msgr2=0x5573a7bd2fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:54:05.747 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:05.741+0000 7f95a83df740 1 --2- 172.21.15.39:0/208715654 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5573a7814070 0x5573a7bd2fc0 secure :-1 s=READY pgs=1095 cs=0 l=1 rev1=1 crypto rx=0x7f9584005940 tx=0x7f9584031b10 comp rx=0 tx=0).stop 2024-09-21T03:54:05.747 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:05.741+0000 7f95a83df740 1 -- 172.21.15.39:0/208715654 shutdown_connections 2024-09-21T03:54:05.748 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:05.741+0000 7f95a83df740 1 --2- 172.21.15.39:0/208715654 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5573a7814070 0x5573a7bd2fc0 unknown :-1 s=CLOSED pgs=1095 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:05.748 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:05.741+0000 7f95a83df740 1 --2- 172.21.15.39:0/208715654 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5573a77f3390 0x5573a7bbee60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:05.748 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:05.741+0000 7f95a83df740 1 --2- 172.21.15.39:0/208715654 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5573a77eb1b0 0x5573a7bc72a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:05.748 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:05.741+0000 7f95a83df740 1 -- 172.21.15.39:0/208715654 >> 172.21.15.39:0/208715654 conn(0x5573a7808460 msgr2=0x5573a7bbe750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:54:05.748 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:05.741+0000 7f95a83df740 1 -- 172.21.15.39:0/208715654 shutdown_connections 2024-09-21T03:54:05.748 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:05.741+0000 7f95a83df740 1 -- 172.21.15.39:0/208715654 wait complete. 2024-09-21T03:54:05.748 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:05.741+0000 7f95a83df740 1 -- start start 2024-09-21T03:54:05.748 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:05.742+0000 7f95a83df740 1 Processor -- start 2024-09-21T03:54:05.748 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:05.742+0000 7f95a83df740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5573a77eb1b0 0x5573a7bcbe20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:54:05.748 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:05.742+0000 7f95a83df740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5573a77f3390 0x5573a7bcc400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:54:05.748 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:05.742+0000 7f95a83df740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5573a7814070 0x5573a7c3ba90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:54:05.748 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:05.742+0000 7f95a83df740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f959c1ae850 con 0x5573a77f3390 2024-09-21T03:54:05.748 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:05.742+0000 7f95a83df740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f959c1ae180 con 0x5573a7814070 2024-09-21T03:54:05.749 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:05.742+0000 7f95a83df740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f959c1ab3c0 con 0x5573a77eb1b0 2024-09-21T03:54:05.749 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:05.742+0000 7f95a83df740 10 client.0 _ll_register_callbacks cb 0x7f958c007910 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:54:05.749 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:05.742+0000 7f959b7fe640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5573a77f3390 0x5573a7bcc400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:54:05.749 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:05.742+0000 7f959b7fe640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5573a77f3390 0x5573a7bcc400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:34906/0 (socket says 172.21.15.39:34906) 2024-09-21T03:54:05.749 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:05.742+0000 7f959b7fe640 1 -- 172.21.15.39:0/1500188750 learned_addr learned my addr 172.21.15.39:0/1500188750 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:54:05.749 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:05.742+0000 7f959affd640 1 --2- 172.21.15.39:0/1500188750 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5573a77eb1b0 0x5573a7bcbe20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:54:05.749 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:05.742+0000 7f959affd640 1 --2- 172.21.15.39:0/1500188750 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5573a7814070 0x5573a7c3ba90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:54:05.749 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:05.743+0000 7f959b7fe640 1 -- 172.21.15.39:0/1500188750 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5573a77eb1b0 msgr2=0x5573a7bcbe20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:54:05.749 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:05.743+0000 7f959b7fe640 1 --2- 172.21.15.39:0/1500188750 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5573a77eb1b0 0x5573a7bcbe20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:05.749 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:05.743+0000 7f959b7fe640 1 -- 172.21.15.39:0/1500188750 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5573a7814070 msgr2=0x5573a7c3ba90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:54:05.749 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:05.743+0000 7f959b7fe640 1 --2- 172.21.15.39:0/1500188750 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5573a7814070 0x5573a7c3ba90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:05.749 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:05.743+0000 7f959b7fe640 1 -- 172.21.15.39:0/1500188750 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5573a7c3c100 con 0x5573a77f3390 2024-09-21T03:54:05.749 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:05.743+0000 7f959b7fe640 1 --2- 172.21.15.39:0/1500188750 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5573a77f3390 0x5573a7bcc400 secure :-1 s=READY pgs=1096 cs=0 l=1 rev1=1 crypto rx=0x7f958c023c20 tx=0x7f958c0236b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=39102835c33fff71 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:54:05.749 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:05.743+0000 7f957affd640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-09-21T03:54:05.749 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:05.743+0000 7f957affd640 1 -- 172.21.15.39:0/1500188750 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f958c032060 con 0x5573a77f3390 2024-09-21T03:54:05.750 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:05.743+0000 7f957affd640 1 -- 172.21.15.39:0/1500188750 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8528+0+0 (secure 0 0 0) 0x7f958c004df0 con 0x5573a77f3390 2024-09-21T03:54:05.750 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:05.743+0000 7f95a83df740 1 -- 172.21.15.39:0/1500188750 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f95840318f0 con 0x5573a77f3390 2024-09-21T03:54:05.750 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:05.743+0000 7f957affd640 1 -- 172.21.15.39:0/1500188750 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f958c0051c0 con 0x5573a77f3390 2024-09-21T03:54:05.750 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:05.743+0000 7f95a83df740 10 client.17577 fetch_fsmap learned FSMap version 29 2024-09-21T03:54:05.750 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:05.743+0000 7f95a83df740 1 -- 172.21.15.39:0/1500188750 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5573a7c43380 con 0x5573a77f3390 2024-09-21T03:54:05.750 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:05.744+0000 7f957affd640 1 -- 172.21.15.39:0/1500188750 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f95840318f0 con 0x5573a77f3390 2024-09-21T03:54:05.750 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:05.744+0000 7f957affd640 1 -- 172.21.15.39:0/1500188750 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f958c0367e0 con 0x5573a77f3390 2024-09-21T03:54:05.750 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:05.744+0000 7f95a83df740 10 client.17577 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T03:54:05.750 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:05.744+0000 7f95a83df740 10 client.17577 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T03:54:05.750 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:05.744+0000 7f95a83df740 20 client.17577 populate_metadata read hostname 'smithi039' 2024-09-21T03:54:05.750 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:05.744+0000 7f95a83df740 1 --2- 172.21.15.39:0/1500188750 >> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] conn(0x5573a77a7da0 0x5573a7c45b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:54:05.750 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:05.744+0000 7f95a83df740 4 client.17577 mds_command: new command op to 24695 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-21T03:54:05.750 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:05.744+0000 7f95a83df740 1 -- 172.21.15.39:0/1500188750 --> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5573a7b2f850 con 0x5573a77a7da0 2024-09-21T03:54:05.750 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:05.744+0000 7f959affd640 1 --2- 172.21.15.39:0/1500188750 >> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] conn(0x5573a77a7da0 0x5573a7c45b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:54:05.750 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:05.745+0000 7f957affd640 10 client.17577 ms_handle_connect on v2:172.21.15.150:6838/2949386907 2024-09-21T03:54:05.751 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:05.745+0000 7f959affd640 1 --2- 172.21.15.39:0/1500188750 >> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] conn(0x5573a77a7da0 0x5573a7c45b40 crc :-1 s=READY pgs=134 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2a06dbfc288e3940 server_cookie=b1f1e1cf367af68d in_seq=0 out_seq=0 2024-09-21T03:54:05.751 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:05.745+0000 7f957affd640 1 -- 172.21.15.39:0/1500188750 <== mds.0 v2:172.21.15.150:6838/2949386907 1 ==== command_reply(tid 0: 0 ) ==== 8+0+4350 (crc 0 0 0) 0x5573a7b2f850 con 0x5573a77a7da0 2024-09-21T03:54:05.751 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:05.745+0000 7f957affd640 10 client.17577 handle_command_reply: tid=0 2024-09-21T03:54:05.751 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:05.745+0000 7f95a83df740 1 client.17577 shutdown 2024-09-21T03:54:05.751 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:05.745+0000 7f95a83df740 1 -- 172.21.15.39:0/1500188750 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5573a77f3390 msgr2=0x5573a7bcc400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:54:05.751 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:05.745+0000 7f95a83df740 1 --2- 172.21.15.39:0/1500188750 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5573a77f3390 0x5573a7bcc400 secure :-1 s=READY pgs=1096 cs=0 l=1 rev1=1 crypto rx=0x7f958c023c20 tx=0x7f958c0236b0 comp rx=0 tx=0).stop 2024-09-21T03:54:05.751 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:05.745+0000 7f95a83df740 1 -- 172.21.15.39:0/1500188750 shutdown_connections 2024-09-21T03:54:05.751 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:05.745+0000 7f95a83df740 1 --2- 172.21.15.39:0/1500188750 >> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] conn(0x5573a77a7da0 0x5573a7c45b40 crc :-1 s=READY pgs=134 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:05.751 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:05.745+0000 7f95a83df740 1 --2- 172.21.15.39:0/1500188750 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5573a7814070 0x5573a7c3ba90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:05.751 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:05.745+0000 7f95a83df740 1 --2- 172.21.15.39:0/1500188750 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5573a77f3390 0x5573a7bcc400 unknown :-1 s=CLOSED pgs=1096 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:05.751 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:05.745+0000 7f95a83df740 1 --2- 172.21.15.39:0/1500188750 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5573a77eb1b0 0x5573a7bcbe20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:05.751 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:05.745+0000 7f95a83df740 1 -- 172.21.15.39:0/1500188750 >> 172.21.15.39:0/1500188750 conn(0x5573a7808460 msgr2=0x5573a7bc8170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:54:05.751 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:05.745+0000 7f957affd640 0 client.17577 ms_handle_reset on v2:172.21.15.150:6838/2949386907 2024-09-21T03:54:05.751 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:05.745+0000 7f95a83df740 1 -- 172.21.15.39:0/1500188750 shutdown_connections 2024-09-21T03:54:05.752 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:05.745+0000 7f95a83df740 1 -- 172.21.15.39:0/1500188750 wait complete. 2024-09-21T03:54:05.752 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:05.746+0000 7f95a83df740 20 client.17577 trim_cache size 0 max 16384 2024-09-21T03:54:05.752 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-09-21T03:54:05.752 INFO:teuthology.orchestra.run.smithi039.stdout: "status": "no active scrubs running", 2024-09-21T03:54:05.752 INFO:teuthology.orchestra.run.smithi039.stdout: "scrubs": {}, 2024-09-21T03:54:05.752 INFO:teuthology.orchestra.run.smithi039.stdout: "scrub_stats": { 2024-09-21T03:54:05.752 INFO:teuthology.orchestra.run.smithi039.stdout: "ad87da79-0248-4e18-9694-8228cb95422f": { 2024-09-21T03:54:05.752 INFO:teuthology.orchestra.run.smithi039.stdout: "start_time": "2024-09-21T03:53:18.441907+0000", 2024-09-21T03:54:05.753 INFO:teuthology.orchestra.run.smithi039.stdout: "path": "/", 2024-09-21T03:54:05.753 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_started": 0, 2024-09-21T03:54:05.753 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_passed": 0, 2024-09-21T03:54:05.753 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_failed": 0, 2024-09-21T03:54:05.753 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_skipped": 0 2024-09-21T03:54:05.753 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:54:05.753 INFO:teuthology.orchestra.run.smithi039.stdout: "02509e4f-3911-4983-b16f-df8ded0b9998": { 2024-09-21T03:54:05.753 INFO:teuthology.orchestra.run.smithi039.stdout: "start_time": "2024-09-21T03:46:17.889576+0000", 2024-09-21T03:54:05.753 INFO:teuthology.orchestra.run.smithi039.stdout: "path": "/", 2024-09-21T03:54:05.753 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_started": 0, 2024-09-21T03:54:05.753 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_passed": 0, 2024-09-21T03:54:05.753 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_failed": 0, 2024-09-21T03:54:05.753 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_skipped": 0 2024-09-21T03:54:05.753 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:54:05.753 INFO:teuthology.orchestra.run.smithi039.stdout: "10728de8-c49d-4edc-be2b-639249255e94": { 2024-09-21T03:54:05.754 INFO:teuthology.orchestra.run.smithi039.stdout: "start_time": "2024-09-21T03:47:03.965623+0000", 2024-09-21T03:54:05.754 INFO:teuthology.orchestra.run.smithi039.stdout: "path": "/", 2024-09-21T03:54:05.754 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_started": 0, 2024-09-21T03:54:05.754 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_passed": 0, 2024-09-21T03:54:05.754 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_failed": 0, 2024-09-21T03:54:05.754 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_skipped": 0 2024-09-21T03:54:05.754 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:54:05.754 INFO:teuthology.orchestra.run.smithi039.stdout: "9860c2f4-b7b1-47aa-865a-9a1c4434a5f6": { 2024-09-21T03:54:05.754 INFO:teuthology.orchestra.run.smithi039.stdout: "start_time": "2024-09-21T03:49:17.718867+0000", 2024-09-21T03:54:05.754 INFO:teuthology.orchestra.run.smithi039.stdout: "path": "/", 2024-09-21T03:54:05.754 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_started": 0, 2024-09-21T03:54:05.754 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_passed": 0, 2024-09-21T03:54:05.754 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_failed": 0, 2024-09-21T03:54:05.754 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_skipped": 0 2024-09-21T03:54:05.755 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:54:05.755 INFO:teuthology.orchestra.run.smithi039.stdout: "d9391d69-baac-453a-9d52-d046760c070d": { 2024-09-21T03:54:05.755 INFO:teuthology.orchestra.run.smithi039.stdout: "start_time": "2024-09-21T03:53:32.364622+0000", 2024-09-21T03:54:05.755 INFO:teuthology.orchestra.run.smithi039.stdout: "path": "/", 2024-09-21T03:54:05.755 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_started": 0, 2024-09-21T03:54:05.755 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_passed": 0, 2024-09-21T03:54:05.755 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_failed": 0, 2024-09-21T03:54:05.755 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_skipped": 0 2024-09-21T03:54:05.755 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:54:05.755 INFO:teuthology.orchestra.run.smithi039.stdout: "c7c700f3-2cd8-41ac-8381-b57dd1b37b02": { 2024-09-21T03:54:05.755 INFO:teuthology.orchestra.run.smithi039.stdout: "start_time": "2024-09-21T03:52:32.368858+0000", 2024-09-21T03:54:05.755 INFO:teuthology.orchestra.run.smithi039.stdout: "path": "/", 2024-09-21T03:54:05.755 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_started": 0, 2024-09-21T03:54:05.755 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_passed": 0, 2024-09-21T03:54:05.755 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_failed": 0, 2024-09-21T03:54:05.755 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_skipped": 0 2024-09-21T03:54:05.756 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:54:05.756 INFO:teuthology.orchestra.run.smithi039.stdout: "29411307-eb8d-4339-8d2b-47ebd32c0080": { 2024-09-21T03:54:05.756 INFO:teuthology.orchestra.run.smithi039.stdout: "start_time": "2024-09-21T03:47:38.092920+0000", 2024-09-21T03:54:05.756 INFO:teuthology.orchestra.run.smithi039.stdout: "path": "/", 2024-09-21T03:54:05.756 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_started": 0, 2024-09-21T03:54:05.756 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_passed": 0, 2024-09-21T03:54:05.756 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_failed": 0, 2024-09-21T03:54:05.756 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_skipped": 0 2024-09-21T03:54:05.756 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:54:05.756 INFO:teuthology.orchestra.run.smithi039.stdout: "ede73e6e-3fd0-4a7f-b5bb-f68946f57eea": { 2024-09-21T03:54:05.756 INFO:teuthology.orchestra.run.smithi039.stdout: "start_time": "2024-09-21T03:51:32.235096+0000", 2024-09-21T03:54:05.756 INFO:teuthology.orchestra.run.smithi039.stdout: "path": "/", 2024-09-21T03:54:05.756 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_started": 0, 2024-09-21T03:54:05.756 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_passed": 0, 2024-09-21T03:54:05.757 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_failed": 0, 2024-09-21T03:54:05.757 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_skipped": 0 2024-09-21T03:54:05.757 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:54:05.757 INFO:teuthology.orchestra.run.smithi039.stdout: "e28ad743-2b22-45b3-8799-a638c2883623": { 2024-09-21T03:54:05.757 INFO:teuthology.orchestra.run.smithi039.stdout: "start_time": "2024-09-21T03:48:31.448821+0000", 2024-09-21T03:54:05.757 INFO:teuthology.orchestra.run.smithi039.stdout: "path": "/", 2024-09-21T03:54:05.757 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_started": 0, 2024-09-21T03:54:05.757 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_passed": 0, 2024-09-21T03:54:05.757 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_failed": 0, 2024-09-21T03:54:05.757 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_skipped": 0 2024-09-21T03:54:05.757 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:54:05.757 INFO:teuthology.orchestra.run.smithi039.stdout: "6e760f1f-9048-4f92-bd66-d708fbef75b3": { 2024-09-21T03:54:05.757 INFO:teuthology.orchestra.run.smithi039.stdout: "start_time": "2024-09-21T03:50:03.711918+0000", 2024-09-21T03:54:05.757 INFO:teuthology.orchestra.run.smithi039.stdout: "path": "/", 2024-09-21T03:54:05.757 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_started": 0, 2024-09-21T03:54:05.758 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_passed": 0, 2024-09-21T03:54:05.758 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_failed": 0, 2024-09-21T03:54:05.758 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_skipped": 0 2024-09-21T03:54:05.758 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:54:05.758 INFO:teuthology.orchestra.run.smithi039.stdout: "c7e2b8e4-057d-435f-b2bd-b633de98035f": { 2024-09-21T03:54:05.758 INFO:teuthology.orchestra.run.smithi039.stdout: "start_time": "2024-09-21T03:47:19.400305+0000", 2024-09-21T03:54:05.758 INFO:teuthology.orchestra.run.smithi039.stdout: "path": "/", 2024-09-21T03:54:05.758 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_started": 0, 2024-09-21T03:54:05.758 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_passed": 0, 2024-09-21T03:54:05.758 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_failed": 0, 2024-09-21T03:54:05.758 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_skipped": 0 2024-09-21T03:54:05.758 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:54:05.758 INFO:teuthology.orchestra.run.smithi039.stdout: "74ab917e-ecea-4778-a1c3-c184214fb24c": { 2024-09-21T03:54:05.758 INFO:teuthology.orchestra.run.smithi039.stdout: "start_time": "2024-09-21T03:50:17.726630+0000", 2024-09-21T03:54:05.759 INFO:teuthology.orchestra.run.smithi039.stdout: "path": "/", 2024-09-21T03:54:05.759 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_started": 0, 2024-09-21T03:54:05.759 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_passed": 0, 2024-09-21T03:54:05.759 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_failed": 0, 2024-09-21T03:54:05.759 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_skipped": 0 2024-09-21T03:54:05.759 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:54:05.759 INFO:teuthology.orchestra.run.smithi039.stdout: "160750d8-401f-4810-866d-e63e7e4bc3e0": { 2024-09-21T03:54:05.759 INFO:teuthology.orchestra.run.smithi039.stdout: "start_time": "2024-09-21T03:51:04.208673+0000", 2024-09-21T03:54:05.759 INFO:teuthology.orchestra.run.smithi039.stdout: "path": "/", 2024-09-21T03:54:05.759 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_started": 0, 2024-09-21T03:54:05.759 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_passed": 0, 2024-09-21T03:54:05.759 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_failed": 0, 2024-09-21T03:54:05.759 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_skipped": 0 2024-09-21T03:54:05.759 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:54:05.759 INFO:teuthology.orchestra.run.smithi039.stdout: "ae1fc4cf-5458-47d0-afcb-282b136dba15": { 2024-09-21T03:54:05.760 INFO:teuthology.orchestra.run.smithi039.stdout: "start_time": "2024-09-21T03:51:18.560378+0000", 2024-09-21T03:54:05.760 INFO:teuthology.orchestra.run.smithi039.stdout: "path": "/", 2024-09-21T03:54:05.760 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_started": 0, 2024-09-21T03:54:05.760 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_passed": 0, 2024-09-21T03:54:05.760 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_failed": 0, 2024-09-21T03:54:05.760 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_skipped": 0 2024-09-21T03:54:05.760 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:54:05.760 INFO:teuthology.orchestra.run.smithi039.stdout: "4bda6ee3-7237-4ee4-96ce-150ad50f26dc": { 2024-09-21T03:54:05.760 INFO:teuthology.orchestra.run.smithi039.stdout: "start_time": "2024-09-21T03:52:18.622141+0000", 2024-09-21T03:54:05.760 INFO:teuthology.orchestra.run.smithi039.stdout: "path": "/", 2024-09-21T03:54:05.760 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_started": 0, 2024-09-21T03:54:05.760 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_passed": 0, 2024-09-21T03:54:05.760 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_failed": 0, 2024-09-21T03:54:05.760 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_skipped": 0 2024-09-21T03:54:05.760 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-21T03:54:05.761 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-21T03:54:05.761 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-09-21T03:54:05.761 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:05.746+0000 7f95a1e6b640 1 -- 172.21.15.39:0/1530932116 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f95880b66b0 msgr2=0x7f95880b8bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:54:05.761 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:05.746+0000 7f95a1e6b640 1 --2- 172.21.15.39:0/1530932116 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f95880b66b0 0x7f95880b8bc0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f959400afd0 tx=0x7f9594071040 comp rx=0 tx=0).stop 2024-09-21T03:54:05.761 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:05.746+0000 7f95a1e6b640 1 -- 172.21.15.39:0/1530932116 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f959c108cf0 msgr2=0x7f959c19eb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:54:05.761 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:05.746+0000 7f95a1e6b640 1 --2- 172.21.15.39:0/1530932116 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f959c108cf0 0x7f959c19eb70 secure :-1 s=READY pgs=1092 cs=0 l=1 rev1=1 crypto rx=0x7f959400c970 tx=0x7f959400ce30 comp rx=0 tx=0).stop 2024-09-21T03:54:05.761 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:05.746+0000 7f95a1e6b640 1 -- 172.21.15.39:0/1530932116 shutdown_connections 2024-09-21T03:54:05.762 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:05.746+0000 7f95a1e6b640 1 --2- 172.21.15.39:0/1530932116 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f95880b66b0 0x7f95880b8bc0 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:05.762 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:05.746+0000 7f95a1e6b640 1 --2- 172.21.15.39:0/1530932116 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f959c108cf0 0x7f959c19eb70 unknown :-1 s=CLOSED pgs=1092 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:05.762 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:05.746+0000 7f95a1e6b640 1 --2- 172.21.15.39:0/1530932116 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f959c1083a0 0x7f959c1998d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:05.762 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:05.746+0000 7f95a1e6b640 1 --2- 172.21.15.39:0/1530932116 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f959c102a60 0x7f959c199330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:05.762 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:05.746+0000 7f95a1e6b640 1 -- 172.21.15.39:0/1530932116 >> 172.21.15.39:0/1530932116 conn(0x7f959c079730 msgr2=0x7f959c104420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:54:05.762 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:05.746+0000 7f95a1e6b640 1 -- 172.21.15.39:0/1530932116 shutdown_connections 2024-09-21T03:54:05.762 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:05.746+0000 7f95a1e6b640 1 -- 172.21.15.39:0/1530932116 wait complete. 2024-09-21T03:54:06.663 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-21T03:54:06.663 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 51116d5c-77ca-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T03:54:06.924 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/51116d5c-77ca-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:54:07.186 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:54:06 smithi039 ceph-mon[31211]: pgmap v569: 97 pgs: 97 active+clean; 10 MiB data, 525 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 44 KiB/s wr, 27 op/s 2024-09-21T03:54:07.218 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:54:06 smithi161 ceph-mon[34533]: pgmap v569: 97 pgs: 97 active+clean; 10 MiB data, 525 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 44 KiB/s wr, 27 op/s 2024-09-21T03:54:07.225 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:54:06 smithi150 ceph-mon[34289]: pgmap v569: 97 pgs: 97 active+clean; 10 MiB data, 525 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 44 KiB/s wr, 27 op/s 2024-09-21T03:54:07.410 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:07.408+0000 7f7c5dba0640 1 -- 172.21.15.39:0/2586476659 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7c58107a00 msgr2=0x7f7c58107ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:54:07.410 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:07.408+0000 7f7c5dba0640 1 --2- 172.21.15.39:0/2586476659 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7c58107a00 0x7f7c58107ee0 secure :-1 s=READY pgs=1097 cs=0 l=1 rev1=1 crypto rx=0x7f7c50009cd0 tx=0x7f7c5002a8e0 comp rx=0 tx=0).stop 2024-09-21T03:54:07.410 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:07.408+0000 7f7c5dba0640 1 -- 172.21.15.39:0/2586476659 shutdown_connections 2024-09-21T03:54:07.410 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:07.408+0000 7f7c5dba0640 1 --2- 172.21.15.39:0/2586476659 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7c58107a00 0x7f7c58107ee0 unknown :-1 s=CLOSED pgs=1097 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:07.410 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:07.408+0000 7f7c5dba0640 1 --2- 172.21.15.39:0/2586476659 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7c58102f10 0x7f7c58107440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:07.410 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:07.408+0000 7f7c5dba0640 1 --2- 172.21.15.39:0/2586476659 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7c581025c0 0x7f7c581029d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:07.410 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:07.408+0000 7f7c5dba0640 1 -- 172.21.15.39:0/2586476659 >> 172.21.15.39:0/2586476659 conn(0x7f7c580fe2d0 msgr2=0x7f7c58100720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:54:07.410 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:07.408+0000 7f7c5dba0640 1 -- 172.21.15.39:0/2586476659 shutdown_connections 2024-09-21T03:54:07.410 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:07.408+0000 7f7c5dba0640 1 -- 172.21.15.39:0/2586476659 wait complete. 2024-09-21T03:54:07.412 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:07.409+0000 7f7c5dba0640 1 Processor -- start 2024-09-21T03:54:07.412 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:07.409+0000 7f7c5dba0640 1 -- start start 2024-09-21T03:54:07.412 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:07.409+0000 7f7c5dba0640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7c581025c0 0x7f7c581a0880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:54:07.412 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:07.409+0000 7f7c5dba0640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7c58102f10 0x7f7c581a0e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:54:07.412 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:07.409+0000 7f7c5dba0640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7c58107a00 0x7f7c581a60c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:54:07.412 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:07.409+0000 7f7c5dba0640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f7c58119ef0 con 0x7f7c581025c0 2024-09-21T03:54:07.413 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:07.409+0000 7f7c5dba0640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f7c58119d70 con 0x7f7c58107a00 2024-09-21T03:54:07.413 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:07.409+0000 7f7c5dba0640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f7c5811a070 con 0x7f7c58102f10 2024-09-21T03:54:07.413 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:07.409+0000 7f7c5cb9e640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7c581025c0 0x7f7c581a0880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:54:07.413 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:07.409+0000 7f7c5cb9e640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7c581025c0 0x7f7c581a0880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:46012/0 (socket says 172.21.15.39:46012) 2024-09-21T03:54:07.413 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:07.409+0000 7f7c5cb9e640 1 -- 172.21.15.39:0/491313144 learned_addr learned my addr 172.21.15.39:0/491313144 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:54:07.413 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:07.409+0000 7f7c5d39f640 1 --2- 172.21.15.39:0/491313144 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7c58107a00 0x7f7c581a60c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:54:07.413 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:07.410+0000 7f7c57fff640 1 --2- 172.21.15.39:0/491313144 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7c58102f10 0x7f7c581a0e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:54:07.413 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:07.410+0000 7f7c5cb9e640 1 -- 172.21.15.39:0/491313144 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7c58102f10 msgr2=0x7f7c581a0e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:54:07.413 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:07.410+0000 7f7c5cb9e640 1 --2- 172.21.15.39:0/491313144 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7c58102f10 0x7f7c581a0e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:07.413 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:07.410+0000 7f7c5cb9e640 1 -- 172.21.15.39:0/491313144 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7c58107a00 msgr2=0x7f7c581a60c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:54:07.413 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:07.410+0000 7f7c5cb9e640 1 --2- 172.21.15.39:0/491313144 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7c58107a00 0x7f7c581a60c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:07.413 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:07.410+0000 7f7c5cb9e640 1 -- 172.21.15.39:0/491313144 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7c581a67f0 con 0x7f7c581025c0 2024-09-21T03:54:07.413 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:07.410+0000 7f7c57fff640 1 --2- 172.21.15.39:0/491313144 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7c58102f10 0x7f7c581a0e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:54:07.413 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:07.410+0000 7f7c5cb9e640 1 --2- 172.21.15.39:0/491313144 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7c581025c0 0x7f7c581a0880 secure :-1 s=READY pgs=1098 cs=0 l=1 rev1=1 crypto rx=0x7f7c4c00e9f0 tx=0x7f7c4c00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:54:07.414 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:07.410+0000 7f7c55ffb640 1 -- 172.21.15.39:0/491313144 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c4c00cd80 con 0x7f7c581025c0 2024-09-21T03:54:07.414 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:07.410+0000 7f7c55ffb640 1 -- 172.21.15.39:0/491313144 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7c4c0108e0 con 0x7f7c581025c0 2024-09-21T03:54:07.414 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:07.410+0000 7f7c5dba0640 1 -- 172.21.15.39:0/491313144 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7c581a6ae0 con 0x7f7c581025c0 2024-09-21T03:54:07.414 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:07.410+0000 7f7c55ffb640 1 -- 172.21.15.39:0/491313144 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c4c018980 con 0x7f7c581025c0 2024-09-21T03:54:07.414 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:07.410+0000 7f7c5dba0640 1 -- 172.21.15.39:0/491313144 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7c581ad240 con 0x7f7c581025c0 2024-09-21T03:54:07.414 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:07.411+0000 7f7c5dba0640 1 -- 172.21.15.39:0/491313144 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7c58107440 con 0x7f7c581025c0 2024-09-21T03:54:07.416 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:07.414+0000 7f7c55ffb640 1 -- 172.21.15.39:0/491313144 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 27) ==== 154643+0+0 (secure 0 0 0) 0x7f7c4c010430 con 0x7f7c581025c0 2024-09-21T03:54:07.417 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:07.415+0000 7f7c55ffb640 1 --2- 172.21.15.39:0/491313144 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f7c480b6680 0x7f7c480b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:54:07.417 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:07.415+0000 7f7c57fff640 1 --2- 172.21.15.39:0/491313144 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f7c480b6680 0x7f7c480b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:54:07.417 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:07.415+0000 7f7c55ffb640 1 -- 172.21.15.39:0/491313144 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8528+0+0 (secure 0 0 0) 0x7f7c4c0e7fb0 con 0x7f7c581025c0 2024-09-21T03:54:07.417 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:07.416+0000 7f7c57fff640 1 --2- 172.21.15.39:0/491313144 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f7c480b6680 0x7f7c480b8b90 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f7c58107f30 tx=0x7f7c44005f90 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:54:07.420 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:07.418+0000 7f7c55ffb640 1 -- 172.21.15.39:0/491313144 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f7c4c0ae500 con 0x7f7c581025c0 2024-09-21T03:54:07.648 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:07.646+0000 7f7c5dba0640 1 -- 172.21.15.39:0/491313144 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f7c581ad4d0 con 0x7f7c581025c0 2024-09-21T03:54:07.649 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:07.647+0000 7f7c55ffb640 1 -- 172.21.15.39:0/491313144 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11978 (secure 0 0 0) 0x7f7c4c0b28e0 con 0x7f7c581025c0 2024-09-21T03:54:07.649 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T03:54:07.649 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":29,"btime":"2024-09-21T03:46:08:982881+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14790,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6835/508995295","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":508995295},{"type":"v1","addr":"172.21.15.39:6835","nonce":508995295}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14802,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6837/3915912607","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":3915912607},{"type":"v1","addr":"172.21.15.39:6837","nonce":3915912607}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":29,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T03:44:15.009872+0000","modified":"2024-09-21T03:46:08.430113+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24695,"mds_1":24667,"mds_2":24659,"mds_3":24646,"mds_4":14868},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14856":{"gid":14856,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.39:6839/185509192","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6838","nonce":185509192},{"type":"v1","addr":"172.21.15.39:6839","nonce":185509192}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14868":{"gid":14868,"name":"a","rank":4,"incarnation":25,"state":"up:active","state_seq":7,"addr":"172.21.15.39:6841/4247190456","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6840","nonce":4247190456},{"type":"v1","addr":"172.21.15.39:6841","nonce":4247190456}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24599":{"gid":24599,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.150:6833/3114643217","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":3114643217},{"type":"v1","addr":"172.21.15.150:6833","nonce":3114643217}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24601":{"gid":24601,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.161:6833/2963990654","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":2963990654},{"type":"v1","addr":"172.21.15.161:6833","nonce":2963990654}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24632":{"gid":24632,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.150:6835/724594008","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6834","nonce":724594008},{"type":"v1","addr":"172.21.15.150:6835","nonce":724594008}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24640":{"gid":24640,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.161:6835/3747882110","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":3747882110},{"type":"v1","addr":"172.21.15.161:6835","nonce":3747882110}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24646":{"gid":24646,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":10,"addr":"172.21.15.161:6837/3451544597","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6836","nonce":3451544597},{"type":"v1","addr":"172.21.15.161:6837","nonce":3451544597}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24659":{"gid":24659,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.150:6837/4017043144","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6836","nonce":4017043144},{"type":"v1","addr":"172.21.15.150:6837","nonce":4017043144}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24667":{"gid":24667,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.161:6839/2190107270","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6838","nonce":2190107270},{"type":"v1","addr":"172.21.15.161:6839","nonce":2190107270}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24695":{"gid":24695,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.150:6839/2949386907","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6838","nonce":2949386907},{"type":"v1","addr":"172.21.15.150:6839","nonce":2949386907}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24695,"qdb_cluster":[24695,24667,24659,24646,14868]},"id":1}]} 2024-09-21T03:54:07.650 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 29 2024-09-21T03:54:07.651 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:07.649+0000 7f7c5dba0640 1 -- 172.21.15.39:0/491313144 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f7c480b6680 msgr2=0x7f7c480b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:54:07.651 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:07.649+0000 7f7c5dba0640 1 --2- 172.21.15.39:0/491313144 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f7c480b6680 0x7f7c480b8b90 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f7c58107f30 tx=0x7f7c44005f90 comp rx=0 tx=0).stop 2024-09-21T03:54:07.651 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:07.649+0000 7f7c5dba0640 1 -- 172.21.15.39:0/491313144 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7c581025c0 msgr2=0x7f7c581a0880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:54:07.651 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:07.649+0000 7f7c5dba0640 1 --2- 172.21.15.39:0/491313144 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7c581025c0 0x7f7c581a0880 secure :-1 s=READY pgs=1098 cs=0 l=1 rev1=1 crypto rx=0x7f7c4c00e9f0 tx=0x7f7c4c00eeb0 comp rx=0 tx=0).stop 2024-09-21T03:54:07.651 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:07.649+0000 7f7c5dba0640 1 -- 172.21.15.39:0/491313144 shutdown_connections 2024-09-21T03:54:07.652 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:07.649+0000 7f7c5dba0640 1 --2- 172.21.15.39:0/491313144 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f7c480b6680 0x7f7c480b8b90 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:07.652 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:07.649+0000 7f7c5dba0640 1 --2- 172.21.15.39:0/491313144 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7c58107a00 0x7f7c581a60c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:07.652 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:07.649+0000 7f7c5dba0640 1 --2- 172.21.15.39:0/491313144 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7c58102f10 0x7f7c581a0e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:07.652 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:07.649+0000 7f7c5dba0640 1 --2- 172.21.15.39:0/491313144 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7c581025c0 0x7f7c581a0880 unknown :-1 s=CLOSED pgs=1098 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:07.652 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:07.649+0000 7f7c5dba0640 1 -- 172.21.15.39:0/491313144 >> 172.21.15.39:0/491313144 conn(0x7f7c580fe2d0 msgr2=0x7f7c58105410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:54:07.652 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:07.649+0000 7f7c5dba0640 1 -- 172.21.15.39:0/491313144 shutdown_connections 2024-09-21T03:54:07.652 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:07.649+0000 7f7c5dba0640 1 -- 172.21.15.39:0/491313144 wait complete. 2024-09-21T03:54:08.218 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:54:07 smithi161 ceph-mon[34533]: from='client.? 172.21.15.39:0/491313144' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:54:08.225 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:54:07 smithi150 ceph-mon[34289]: from='client.? 172.21.15.39:0/491313144' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:54:08.329 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:54:07 smithi039 ceph-mon[31211]: from='client.? 172.21.15.39:0/491313144' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:54:08.360 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 51116d5c-77ca-11ef-baf5-efdc52797490 -- ceph tell mds.1:4 damage ls 2024-09-21T03:54:08.617 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/51116d5c-77ca-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:54:08.991 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:54:08 smithi039 ceph-mon[31211]: pgmap v570: 97 pgs: 97 active+clean; 10 MiB data, 525 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 44 KiB/s wr, 25 op/s 2024-09-21T03:54:09.085 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:09.083+0000 7efe8e5d5640 1 -- 172.21.15.39:0/1021151496 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efe881084d0 msgr2=0x7efe8810ed00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:54:09.086 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:09.083+0000 7efe8e5d5640 1 --2- 172.21.15.39:0/1021151496 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efe881084d0 0x7efe8810ed00 secure :-1 s=READY pgs=1099 cs=0 l=1 rev1=1 crypto rx=0x7efe80009ca0 tx=0x7efe8002a8c0 comp rx=0 tx=0).stop 2024-09-21T03:54:09.086 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:09.083+0000 7efe8e5d5640 1 -- 172.21.15.39:0/1021151496 shutdown_connections 2024-09-21T03:54:09.086 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:09.083+0000 7efe8e5d5640 1 --2- 172.21.15.39:0/1021151496 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efe881084d0 0x7efe8810ed00 unknown :-1 s=CLOSED pgs=1099 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:09.086 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:09.083+0000 7efe8e5d5640 1 --2- 172.21.15.39:0/1021151496 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7efe88107b80 0x7efe88107f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:09.086 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:09.083+0000 7efe8e5d5640 1 --2- 172.21.15.39:0/1021151496 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7efe88102240 0x7efe88102650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:09.086 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:09.083+0000 7efe8e5d5640 1 -- 172.21.15.39:0/1021151496 >> 172.21.15.39:0/1021151496 conn(0x7efe88079f90 msgr2=0x7efe8807a3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:54:09.086 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:09.083+0000 7efe8e5d5640 1 -- 172.21.15.39:0/1021151496 shutdown_connections 2024-09-21T03:54:09.086 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:09.083+0000 7efe8e5d5640 1 -- 172.21.15.39:0/1021151496 wait complete. 2024-09-21T03:54:09.086 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:09.084+0000 7efe8e5d5640 1 Processor -- start 2024-09-21T03:54:09.087 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:09.084+0000 7efe8e5d5640 1 -- start start 2024-09-21T03:54:09.088 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:09.084+0000 7efe8e5d5640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efe88102240 0x7efe8819b270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:54:09.088 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:09.084+0000 7efe8e5d5640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7efe88107b80 0x7efe8819b7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:54:09.088 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:09.084+0000 7efe8e5d5640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7efe881084d0 0x7efe8819bcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:54:09.088 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:09.084+0000 7efe8e5d5640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7efe88112a60 con 0x7efe88102240 2024-09-21T03:54:09.088 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:09.084+0000 7efe8e5d5640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7efe881128e0 con 0x7efe881084d0 2024-09-21T03:54:09.088 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:09.084+0000 7efe8e5d5640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7efe88112be0 con 0x7efe88107b80 2024-09-21T03:54:09.088 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:09.084+0000 7efe87fff640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efe88102240 0x7efe8819b270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:54:09.088 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:09.084+0000 7efe877fe640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7efe88107b80 0x7efe8819b7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:54:09.088 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:09.084+0000 7efe87fff640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efe88102240 0x7efe8819b270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:46028/0 (socket says 172.21.15.39:46028) 2024-09-21T03:54:09.089 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:09.084+0000 7efe87fff640 1 -- 172.21.15.39:0/3364131231 learned_addr learned my addr 172.21.15.39:0/3364131231 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:54:09.089 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:09.085+0000 7efe8cb72640 1 --2- 172.21.15.39:0/3364131231 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7efe881084d0 0x7efe8819bcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:54:09.089 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:09.085+0000 7efe87fff640 1 -- 172.21.15.39:0/3364131231 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7efe88107b80 msgr2=0x7efe8819b7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:54:09.089 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:09.085+0000 7efe87fff640 1 --2- 172.21.15.39:0/3364131231 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7efe88107b80 0x7efe8819b7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:09.089 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:09.085+0000 7efe87fff640 1 -- 172.21.15.39:0/3364131231 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7efe881084d0 msgr2=0x7efe8819bcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:54:09.089 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:09.085+0000 7efe87fff640 1 --2- 172.21.15.39:0/3364131231 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7efe881084d0 0x7efe8819bcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:09.089 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:09.085+0000 7efe87fff640 1 -- 172.21.15.39:0/3364131231 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efe881a6470 con 0x7efe88102240 2024-09-21T03:54:09.089 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:09.085+0000 7efe87fff640 1 --2- 172.21.15.39:0/3364131231 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efe88102240 0x7efe8819b270 secure :-1 s=READY pgs=1100 cs=0 l=1 rev1=1 crypto rx=0x7efe7800c920 tx=0x7efe7800cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:54:09.089 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:09.085+0000 7efe857fa640 1 -- 172.21.15.39:0/3364131231 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe78007c70 con 0x7efe88102240 2024-09-21T03:54:09.089 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:09.085+0000 7efe8e5d5640 1 -- 172.21.15.39:0/3364131231 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efe881a6760 con 0x7efe88102240 2024-09-21T03:54:09.089 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:09.085+0000 7efe857fa640 1 -- 172.21.15.39:0/3364131231 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efe78007e10 con 0x7efe88102240 2024-09-21T03:54:09.090 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:09.085+0000 7efe857fa640 1 -- 172.21.15.39:0/3364131231 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe78018820 con 0x7efe88102240 2024-09-21T03:54:09.090 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:09.085+0000 7efe8e5d5640 1 -- 172.21.15.39:0/3364131231 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efe881a6cf0 con 0x7efe88102240 2024-09-21T03:54:09.092 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:09.088+0000 7efe94b49740 1 Processor -- start 2024-09-21T03:54:09.092 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:09.088+0000 7efe94b49740 1 -- start start 2024-09-21T03:54:09.092 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:09.088+0000 7efe857fa640 1 -- 172.21.15.39:0/3364131231 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 27) ==== 154643+0+0 (secure 0 0 0) 0x7efe7800f450 con 0x7efe88102240 2024-09-21T03:54:09.092 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:09.088+0000 7efe94b49740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55875fee3070 0x55876028a2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:54:09.092 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:09.088+0000 7efe94b49740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55875feba1b0 0x55876028a800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:54:09.092 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:09.088+0000 7efe94b49740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55875fec2390 0x55876028ad40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:54:09.092 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:09.088+0000 7efe94b49740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x558760200ea0 con 0x55875feba1b0 2024-09-21T03:54:09.092 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:09.088+0000 7efe94b49740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x5587601f02d0 con 0x55875fee3070 2024-09-21T03:54:09.093 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:09.088+0000 7efe94b49740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x5587601f40e0 con 0x55875fec2390 2024-09-21T03:54:09.093 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:09.088+0000 7efe87fff640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55875feba1b0 0x55876028a800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:54:09.093 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:09.088+0000 7efe87fff640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55875feba1b0 0x55876028a800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:46030/0 (socket says 172.21.15.39:46030) 2024-09-21T03:54:09.093 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:09.088+0000 7efe87fff640 1 -- 172.21.15.39:0/190011250 learned_addr learned my addr 172.21.15.39:0/190011250 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:54:09.093 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:09.088+0000 7efe8cb72640 1 --2- 172.21.15.39:0/190011250 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55875fee3070 0x55876028a2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:54:09.093 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:09.088+0000 7efe877fe640 1 --2- 172.21.15.39:0/190011250 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55875fec2390 0x55876028ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:54:09.093 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:09.089+0000 7efe87fff640 1 -- 172.21.15.39:0/190011250 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55875fec2390 msgr2=0x55876028ad40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:54:09.093 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:09.089+0000 7efe87fff640 1 --2- 172.21.15.39:0/190011250 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55875fec2390 0x55876028ad40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:09.093 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:09.089+0000 7efe87fff640 1 -- 172.21.15.39:0/190011250 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55875fee3070 msgr2=0x55876028a2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:54:09.093 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:09.089+0000 7efe87fff640 1 --2- 172.21.15.39:0/190011250 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55875fee3070 0x55876028a2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:09.093 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:09.089+0000 7efe87fff640 1 -- 172.21.15.39:0/190011250 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55875ff4e650 con 0x55875feba1b0 2024-09-21T03:54:09.093 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:09.089+0000 7efe87fff640 1 --2- 172.21.15.39:0/190011250 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55875feba1b0 0x55876028a800 secure :-1 s=READY pgs=1101 cs=0 l=1 rev1=1 crypto rx=0x7efe780e4ee0 tx=0x7efe780edb20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ca3a8f686c766f8 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:54:09.094 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:09.089+0000 7efe857fa640 1 --2- 172.21.15.39:0/3364131231 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7efe700b66b0 0x7efe700b8bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:54:09.094 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:09.089+0000 7efe6e7fc640 1 -- 172.21.15.39:0/190011250 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe780f6070 con 0x55875feba1b0 2024-09-21T03:54:09.094 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:09.089+0000 7efe8cb72640 1 --2- 172.21.15.39:0/3364131231 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7efe700b66b0 0x7efe700b8bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:54:09.094 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:09.089+0000 7efe6e7fc640 1 -- 172.21.15.39:0/190011250 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efe780ba0d0 con 0x55875feba1b0 2024-09-21T03:54:09.094 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:09.089+0000 7efe6e7fc640 1 -- 172.21.15.39:0/190011250 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe780ace10 con 0x55875feba1b0 2024-09-21T03:54:09.094 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:09.090+0000 7efe857fa640 1 -- 172.21.15.39:0/3364131231 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8528+0+0 (secure 0 0 0) 0x7efe780e7850 con 0x7efe88102240 2024-09-21T03:54:09.094 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:09.090+0000 7efe94b49740 1 -- 172.21.15.39:0/190011250 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55875feba1b0 msgr2=0x55876028a800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:54:09.094 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:09.090+0000 7efe94b49740 1 --2- 172.21.15.39:0/190011250 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55875feba1b0 0x55876028a800 secure :-1 s=READY pgs=1101 cs=0 l=1 rev1=1 crypto rx=0x7efe780e4ee0 tx=0x7efe780edb20 comp rx=0 tx=0).stop 2024-09-21T03:54:09.094 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:09.090+0000 7efe8cb72640 1 --2- 172.21.15.39:0/3364131231 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7efe700b66b0 0x7efe700b8bc0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7efe800094e0 tx=0x7efe80009380 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:54:09.100 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:09.091+0000 7efe94b49740 1 -- 172.21.15.39:0/190011250 shutdown_connections 2024-09-21T03:54:09.100 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:09.091+0000 7efe94b49740 1 --2- 172.21.15.39:0/190011250 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55875fec2390 0x55876028ad40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:09.100 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:09.091+0000 7efe94b49740 1 --2- 172.21.15.39:0/190011250 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55875feba1b0 0x55876028a800 unknown :-1 s=CLOSED pgs=1101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:09.100 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:09.091+0000 7efe94b49740 1 --2- 172.21.15.39:0/190011250 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55875fee3070 0x55876028a2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:09.100 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:09.091+0000 7efe94b49740 1 -- 172.21.15.39:0/190011250 >> 172.21.15.39:0/190011250 conn(0x55875fed7460 msgr2=0x5587602881b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:54:09.100 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:09.091+0000 7efe94b49740 1 -- 172.21.15.39:0/190011250 shutdown_connections 2024-09-21T03:54:09.100 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:09.091+0000 7efe94b49740 1 -- 172.21.15.39:0/190011250 wait complete. 2024-09-21T03:54:09.100 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:09.091+0000 7efe94b49740 1 -- start start 2024-09-21T03:54:09.100 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:09.091+0000 7efe94b49740 1 Processor -- start 2024-09-21T03:54:09.100 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:09.092+0000 7efe94b49740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55875feba1b0 0x5587602959c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:54:09.101 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:09.092+0000 7efe94b49740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55875fec2390 0x558760295f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:54:09.101 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:09.092+0000 7efe94b49740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55875fee3070 0x5587602a61d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:54:09.101 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:09.092+0000 7efe94b49740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x558760051700 con 0x55875feba1b0 2024-09-21T03:54:09.101 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:09.092+0000 7efe94b49740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x55876018b160 con 0x55875fec2390 2024-09-21T03:54:09.101 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:09.092+0000 7efe94b49740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55876029ceb0 con 0x55875fee3070 2024-09-21T03:54:09.101 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:09.092+0000 7efe94b49740 10 client.0 _ll_register_callbacks cb 0x5587601a49e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:54:09.101 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:09.092+0000 7efe87fff640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55875feba1b0 0x5587602959c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:54:09.101 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:09.092+0000 7efe87fff640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55875feba1b0 0x5587602959c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:46032/0 (socket says 172.21.15.39:46032) 2024-09-21T03:54:09.101 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:09.092+0000 7efe87fff640 1 -- 172.21.15.39:0/2573665734 learned_addr learned my addr 172.21.15.39:0/2573665734 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:54:09.101 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:09.092+0000 7efe877fe640 1 --2- 172.21.15.39:0/2573665734 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55875fec2390 0x558760295f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:54:09.101 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:09.092+0000 7efe8cb72640 1 --2- 172.21.15.39:0/2573665734 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55875fee3070 0x5587602a61d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:54:09.101 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:09.092+0000 7efe87fff640 1 -- 172.21.15.39:0/2573665734 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55875fee3070 msgr2=0x5587602a61d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:54:09.101 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:09.092+0000 7efe87fff640 1 --2- 172.21.15.39:0/2573665734 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55875fee3070 0x5587602a61d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:09.101 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:09.092+0000 7efe87fff640 1 -- 172.21.15.39:0/2573665734 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55875fec2390 msgr2=0x558760295f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:54:09.102 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:09.092+0000 7efe87fff640 1 --2- 172.21.15.39:0/2573665734 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55875fec2390 0x558760295f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:09.102 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:09.092+0000 7efe87fff640 1 -- 172.21.15.39:0/2573665734 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5587601b8c90 con 0x55875feba1b0 2024-09-21T03:54:09.102 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:09.092+0000 7efe8cb72640 1 --2- 172.21.15.39:0/2573665734 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55875fee3070 0x5587602a61d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:54:09.102 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:09.092+0000 7efe87fff640 1 --2- 172.21.15.39:0/2573665734 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55875feba1b0 0x5587602959c0 secure :-1 s=READY pgs=1102 cs=0 l=1 rev1=1 crypto rx=0x7efe78007590 tx=0x7efe7800fcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b94ab095c7f6989d server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:54:09.102 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:09.092+0000 7efe677fe640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-09-21T03:54:09.102 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:09.093+0000 7efe677fe640 1 -- 172.21.15.39:0/2573665734 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe780f6070 con 0x55875feba1b0 2024-09-21T03:54:09.102 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:09.093+0000 7efe94b49740 1 -- 172.21.15.39:0/2573665734 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efe780ba0d0 con 0x55875feba1b0 2024-09-21T03:54:09.102 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:09.093+0000 7efe677fe640 1 -- 172.21.15.39:0/2573665734 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8528+0+0 (secure 0 0 0) 0x7efe780b6c30 con 0x55875feba1b0 2024-09-21T03:54:09.102 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:09.093+0000 7efe677fe640 1 -- 172.21.15.39:0/2573665734 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7efe780ace10 con 0x55875feba1b0 2024-09-21T03:54:09.102 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:09.093+0000 7efe94b49740 10 client.17595 fetch_fsmap learned FSMap version 29 2024-09-21T03:54:09.102 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:09.093+0000 7efe94b49740 1 -- 172.21.15.39:0/2573665734 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5587602a8bd0 con 0x55875feba1b0 2024-09-21T03:54:09.102 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:09.093+0000 7efe677fe640 1 -- 172.21.15.39:0/2573665734 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efe780ba0d0 con 0x55875feba1b0 2024-09-21T03:54:09.102 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:09.093+0000 7efe677fe640 1 -- 172.21.15.39:0/2573665734 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7efe780a8ff0 con 0x55875feba1b0 2024-09-21T03:54:09.103 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:09.093+0000 7efe94b49740 10 client.17595 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T03:54:09.103 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:09.093+0000 7efe94b49740 10 client.17595 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.a 2024-09-21T03:54:09.103 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:09.093+0000 7efe94b49740 20 client.17595 populate_metadata read hostname 'smithi039' 2024-09-21T03:54:09.103 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:09.093+0000 7efe94b49740 1 --2- 172.21.15.39:0/2573665734 >> [v2:172.21.15.39:6840/4247190456,v1:172.21.15.39:6841/4247190456] conn(0x55875fea19f0 0x5587602abf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:54:09.103 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:09.093+0000 7efe94b49740 4 client.17595 mds_command: new command op to 14868 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:54:09.103 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:09.093+0000 7efe94b49740 1 -- 172.21.15.39:0/2573665734 --> [v2:172.21.15.39:6840/4247190456,v1:172.21.15.39:6841/4247190456] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558760202df0 con 0x55875fea19f0 2024-09-21T03:54:09.103 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:09.094+0000 7efe877fe640 1 --2- 172.21.15.39:0/2573665734 >> [v2:172.21.15.39:6840/4247190456,v1:172.21.15.39:6841/4247190456] conn(0x55875fea19f0 0x5587602abf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:54:09.103 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:09.094+0000 7efe677fe640 10 client.17595 ms_handle_connect on v2:172.21.15.39:6840/4247190456 2024-09-21T03:54:09.103 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:09.094+0000 7efe877fe640 1 --2- 172.21.15.39:0/2573665734 >> [v2:172.21.15.39:6840/4247190456,v1:172.21.15.39:6841/4247190456] conn(0x55875fea19f0 0x5587602abf30 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=2252c320c5c6af66 server_cookie=4dd965a93fde344d in_seq=0 out_seq=0 2024-09-21T03:54:09.103 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:09.095+0000 7efe677fe640 1 -- 172.21.15.39:0/2573665734 <== mds.4 v2:172.21.15.39:6840/4247190456 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x558760202df0 con 0x55875fea19f0 2024-09-21T03:54:09.103 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:09.095+0000 7efe677fe640 10 client.17595 handle_command_reply: tid=0 2024-09-21T03:54:09.103 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:09.095+0000 7efe94b49740 1 client.17595 shutdown 2024-09-21T03:54:09.103 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:09.095+0000 7efe94b49740 1 -- 172.21.15.39:0/2573665734 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55875feba1b0 msgr2=0x5587602959c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:54:09.103 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:09.095+0000 7efe94b49740 1 --2- 172.21.15.39:0/2573665734 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55875feba1b0 0x5587602959c0 secure :-1 s=READY pgs=1102 cs=0 l=1 rev1=1 crypto rx=0x7efe78007590 tx=0x7efe7800fcc0 comp rx=0 tx=0).stop 2024-09-21T03:54:09.104 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:09.095+0000 7efe94b49740 1 -- 172.21.15.39:0/2573665734 shutdown_connections 2024-09-21T03:54:09.104 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:09.095+0000 7efe94b49740 1 --2- 172.21.15.39:0/2573665734 >> [v2:172.21.15.39:6840/4247190456,v1:172.21.15.39:6841/4247190456] conn(0x55875fea19f0 0x5587602abf30 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:09.104 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:09.095+0000 7efe94b49740 1 --2- 172.21.15.39:0/2573665734 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55875fee3070 0x5587602a61d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:09.104 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:09.095+0000 7efe94b49740 1 --2- 172.21.15.39:0/2573665734 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55875fec2390 0x558760295f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:09.104 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:09.095+0000 7efe94b49740 1 --2- 172.21.15.39:0/2573665734 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55875feba1b0 0x5587602959c0 unknown :-1 s=CLOSED pgs=1102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:09.104 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:09.095+0000 7efe677fe640 0 client.17595 ms_handle_reset on v2:172.21.15.39:6840/4247190456 2024-09-21T03:54:09.104 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:09.095+0000 7efe94b49740 1 -- 172.21.15.39:0/2573665734 >> 172.21.15.39:0/2573665734 conn(0x55875fed7460 msgr2=0x55876028b2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:54:09.104 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:09.095+0000 7efe94b49740 1 -- 172.21.15.39:0/2573665734 shutdown_connections 2024-09-21T03:54:09.104 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:09.095+0000 7efe94b49740 1 -- 172.21.15.39:0/2573665734 wait complete. 2024-09-21T03:54:09.104 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:09.096+0000 7efe94b49740 20 client.17595 trim_cache size 0 max 16384 2024-09-21T03:54:09.111 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:09.109+0000 7efe94b49740 1 Processor -- start 2024-09-21T03:54:09.111 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:09.109+0000 7efe94b49740 1 -- start start 2024-09-21T03:54:09.112 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:09.110+0000 7efe94b49740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55875fea19f0 0x5587602978a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:54:09.112 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:09.110+0000 7efe94b49740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55875feba1b0 0x5587602a6140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:54:09.112 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:09.110+0000 7efe94b49740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55875fec2390 0x55876028fae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:54:09.112 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:09.110+0000 7efe94b49740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7efe881ad540 con 0x55875feba1b0 2024-09-21T03:54:09.112 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:09.110+0000 7efe94b49740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7efe881af080 con 0x55875fec2390 2024-09-21T03:54:09.112 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:09.110+0000 7efe94b49740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7efe881adc10 con 0x55875fea19f0 2024-09-21T03:54:09.112 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:09.110+0000 7efe877fe640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55875feba1b0 0x5587602a6140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:54:09.112 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:09.110+0000 7efe877fe640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55875feba1b0 0x5587602a6140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:46040/0 (socket says 172.21.15.39:46040) 2024-09-21T03:54:09.112 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:09.110+0000 7efe877fe640 1 -- 172.21.15.39:0/3495576710 learned_addr learned my addr 172.21.15.39:0/3495576710 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:54:09.114 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:09.110+0000 7efe87fff640 1 --2- 172.21.15.39:0/3495576710 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55875fea19f0 0x5587602978a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:54:09.114 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:09.110+0000 7efe8cb72640 1 --2- 172.21.15.39:0/3495576710 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55875fec2390 0x55876028fae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:54:09.114 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:09.110+0000 7efe877fe640 1 -- 172.21.15.39:0/3495576710 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55875fea19f0 msgr2=0x5587602978a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:54:09.114 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:09.110+0000 7efe877fe640 1 --2- 172.21.15.39:0/3495576710 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55875fea19f0 0x5587602978a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:09.114 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:09.110+0000 7efe877fe640 1 -- 172.21.15.39:0/3495576710 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55875fec2390 msgr2=0x55876028fae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:54:09.114 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:09.110+0000 7efe877fe640 1 --2- 172.21.15.39:0/3495576710 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55875fec2390 0x55876028fae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:09.114 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:09.110+0000 7efe877fe640 1 -- 172.21.15.39:0/3495576710 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55876028be30 con 0x55875feba1b0 2024-09-21T03:54:09.114 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:09.110+0000 7efe877fe640 1 --2- 172.21.15.39:0/3495576710 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55875feba1b0 0x5587602a6140 secure :-1 s=READY pgs=1103 cs=0 l=1 rev1=1 crypto rx=0x7efe7c005fd0 tx=0x7efe7c00b390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf1fea60b42c8657 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:54:09.114 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:09.111+0000 7efe6dffb640 1 -- 172.21.15.39:0/3495576710 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe7c029070 con 0x55875feba1b0 2024-09-21T03:54:09.114 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:09.111+0000 7efe6dffb640 1 -- 172.21.15.39:0/3495576710 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efe7c024920 con 0x55875feba1b0 2024-09-21T03:54:09.115 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:09.111+0000 7efe6dffb640 1 -- 172.21.15.39:0/3495576710 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe7c011830 con 0x55875feba1b0 2024-09-21T03:54:09.115 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:09.111+0000 7efe94b49740 1 -- 172.21.15.39:0/3495576710 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55875feba1b0 msgr2=0x5587602a6140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:54:09.115 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:09.111+0000 7efe94b49740 1 --2- 172.21.15.39:0/3495576710 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55875feba1b0 0x5587602a6140 secure :-1 s=READY pgs=1103 cs=0 l=1 rev1=1 crypto rx=0x7efe7c005fd0 tx=0x7efe7c00b390 comp rx=0 tx=0).stop 2024-09-21T03:54:09.115 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:09.111+0000 7efe94b49740 1 -- 172.21.15.39:0/3495576710 shutdown_connections 2024-09-21T03:54:09.115 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:09.112+0000 7efe94b49740 1 --2- 172.21.15.39:0/3495576710 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55875fec2390 0x55876028fae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:09.115 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:09.112+0000 7efe94b49740 1 --2- 172.21.15.39:0/3495576710 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55875feba1b0 0x5587602a6140 unknown :-1 s=CLOSED pgs=1103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:09.115 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:09.112+0000 7efe94b49740 1 --2- 172.21.15.39:0/3495576710 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55875fea19f0 0x5587602978a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:09.115 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:09.112+0000 7efe94b49740 1 -- 172.21.15.39:0/3495576710 >> 172.21.15.39:0/3495576710 conn(0x55875fed7460 msgr2=0x55876028f3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:54:09.115 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:09.112+0000 7efe94b49740 1 -- 172.21.15.39:0/3495576710 shutdown_connections 2024-09-21T03:54:09.115 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:09.112+0000 7efe94b49740 1 -- 172.21.15.39:0/3495576710 wait complete. 2024-09-21T03:54:09.115 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:09.112+0000 7efe94b49740 1 -- start start 2024-09-21T03:54:09.115 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:09.112+0000 7efe94b49740 1 Processor -- start 2024-09-21T03:54:09.119 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:09.112+0000 7efe94b49740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55875fea19f0 0x5587602af050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:54:09.119 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:09.112+0000 7efe94b49740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55875feba1b0 0x5587602af630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:54:09.119 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:09.112+0000 7efe94b49740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55875fec2390 0x55876030bb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:54:09.119 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:09.113+0000 7efe94b49740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7efe881ae9b0 con 0x55875feba1b0 2024-09-21T03:54:09.119 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:09.113+0000 7efe94b49740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7efe881ae2e0 con 0x55875fea19f0 2024-09-21T03:54:09.119 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:09.113+0000 7efe94b49740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7efe881ab520 con 0x55875fec2390 2024-09-21T03:54:09.119 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:09.113+0000 7efe94b49740 10 client.0 _ll_register_callbacks cb 0x7efe80009b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:54:09.120 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:09.113+0000 7efe877fe640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55875feba1b0 0x5587602af630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:54:09.120 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:09.113+0000 7efe877fe640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55875feba1b0 0x5587602af630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:46046/0 (socket says 172.21.15.39:46046) 2024-09-21T03:54:09.120 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:09.113+0000 7efe877fe640 1 -- 172.21.15.39:0/1467530828 learned_addr learned my addr 172.21.15.39:0/1467530828 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:54:09.120 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:09.113+0000 7efe8cb72640 1 --2- 172.21.15.39:0/1467530828 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55875fec2390 0x55876030bb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:54:09.120 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:09.113+0000 7efe87fff640 1 --2- 172.21.15.39:0/1467530828 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55875fea19f0 0x5587602af050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:54:09.120 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:09.113+0000 7efe877fe640 1 -- 172.21.15.39:0/1467530828 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55875fec2390 msgr2=0x55876030bb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:54:09.120 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:09.113+0000 7efe877fe640 1 --2- 172.21.15.39:0/1467530828 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55875fec2390 0x55876030bb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:09.120 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:09.113+0000 7efe877fe640 1 -- 172.21.15.39:0/1467530828 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55875fea19f0 msgr2=0x5587602af050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:54:09.120 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:09.113+0000 7efe877fe640 1 --2- 172.21.15.39:0/1467530828 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55875fea19f0 0x5587602af050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:09.120 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:09.113+0000 7efe877fe640 1 -- 172.21.15.39:0/1467530828 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55876030c1b0 con 0x55875feba1b0 2024-09-21T03:54:09.120 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:09.113+0000 7efe87fff640 1 --2- 172.21.15.39:0/1467530828 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55875fea19f0 0x5587602af050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:54:09.120 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:09.113+0000 7efe877fe640 1 --2- 172.21.15.39:0/1467530828 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55875feba1b0 0x5587602af630 secure :-1 s=READY pgs=1104 cs=0 l=1 rev1=1 crypto rx=0x7efe7c00baa0 tx=0x7efe7c00bad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=444a1175a459bad server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:54:09.120 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:09.113+0000 7efe677fe640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-09-21T03:54:09.120 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:09.113+0000 7efe677fe640 1 -- 172.21.15.39:0/1467530828 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe7c032060 con 0x55875feba1b0 2024-09-21T03:54:09.121 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:09.114+0000 7efe677fe640 1 -- 172.21.15.39:0/1467530828 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8528+0+0 (secure 0 0 0) 0x7efe7c024670 con 0x55875feba1b0 2024-09-21T03:54:09.121 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:09.114+0000 7efe94b49740 1 -- 172.21.15.39:0/1467530828 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efe7c024920 con 0x55875feba1b0 2024-09-21T03:54:09.121 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:09.114+0000 7efe677fe640 1 -- 172.21.15.39:0/1467530828 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7efe7c029040 con 0x55875feba1b0 2024-09-21T03:54:09.121 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:09.114+0000 7efe94b49740 10 client.17601 fetch_fsmap learned FSMap version 29 2024-09-21T03:54:09.121 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:09.114+0000 7efe94b49740 1 -- 172.21.15.39:0/1467530828 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5587603167c0 con 0x55875feba1b0 2024-09-21T03:54:09.121 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:09.114+0000 7efe677fe640 1 -- 172.21.15.39:0/1467530828 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efe7c024920 con 0x55875feba1b0 2024-09-21T03:54:09.121 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:09.114+0000 7efe677fe640 1 -- 172.21.15.39:0/1467530828 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7efe7c02c070 con 0x55875feba1b0 2024-09-21T03:54:09.121 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:09.114+0000 7efe94b49740 10 client.17601 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T03:54:09.121 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:09.114+0000 7efe94b49740 10 client.17601 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.a 2024-09-21T03:54:09.121 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:09.114+0000 7efe94b49740 20 client.17601 populate_metadata read hostname 'smithi039' 2024-09-21T03:54:09.121 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:09.114+0000 7efe94b49740 1 --2- 172.21.15.39:0/1467530828 >> [v2:172.21.15.39:6840/4247190456,v1:172.21.15.39:6841/4247190456] conn(0x55875fee3070 0x558760318e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:54:09.121 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:09.114+0000 7efe94b49740 4 client.17601 mds_command: new command op to 14868 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T03:54:09.121 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:09.114+0000 7efe94b49740 1 -- 172.21.15.39:0/1467530828 --> [v2:172.21.15.39:6840/4247190456,v1:172.21.15.39:6841/4247190456] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5587601fe8a0 con 0x55875fee3070 2024-09-21T03:54:09.121 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:09.114+0000 7efe87fff640 1 --2- 172.21.15.39:0/1467530828 >> [v2:172.21.15.39:6840/4247190456,v1:172.21.15.39:6841/4247190456] conn(0x55875fee3070 0x558760318e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:54:09.122 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:09.115+0000 7efe677fe640 10 client.17601 ms_handle_connect on v2:172.21.15.39:6840/4247190456 2024-09-21T03:54:09.122 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:09.115+0000 7efe87fff640 1 --2- 172.21.15.39:0/1467530828 >> [v2:172.21.15.39:6840/4247190456,v1:172.21.15.39:6841/4247190456] conn(0x55875fee3070 0x558760318e50 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=1cb3f9e8e4e8d772 server_cookie=15a5d4229d7d5154 in_seq=0 out_seq=0 2024-09-21T03:54:09.122 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:09.115+0000 7efe677fe640 1 -- 172.21.15.39:0/1467530828 <== mds.4 v2:172.21.15.39:6840/4247190456 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5587601fe8a0 con 0x55875fee3070 2024-09-21T03:54:09.122 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:09.115+0000 7efe677fe640 10 client.17601 handle_command_reply: tid=0 2024-09-21T03:54:09.122 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:09.115+0000 7efe94b49740 1 client.17601 shutdown 2024-09-21T03:54:09.122 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:09.115+0000 7efe94b49740 1 -- 172.21.15.39:0/1467530828 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55875feba1b0 msgr2=0x5587602af630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:54:09.122 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:09.115+0000 7efe94b49740 1 --2- 172.21.15.39:0/1467530828 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55875feba1b0 0x5587602af630 secure :-1 s=READY pgs=1104 cs=0 l=1 rev1=1 crypto rx=0x7efe7c00baa0 tx=0x7efe7c00bad0 comp rx=0 tx=0).stop 2024-09-21T03:54:09.122 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:09.115+0000 7efe94b49740 1 -- 172.21.15.39:0/1467530828 shutdown_connections 2024-09-21T03:54:09.122 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:09.115+0000 7efe94b49740 1 --2- 172.21.15.39:0/1467530828 >> [v2:172.21.15.39:6840/4247190456,v1:172.21.15.39:6841/4247190456] conn(0x55875fee3070 0x558760318e50 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:09.122 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:09.115+0000 7efe677fe640 0 client.17601 ms_handle_reset on v2:172.21.15.39:6840/4247190456 2024-09-21T03:54:09.122 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:09.115+0000 7efe94b49740 1 --2- 172.21.15.39:0/1467530828 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55875fec2390 0x55876030bb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:09.122 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:09.115+0000 7efe94b49740 1 --2- 172.21.15.39:0/1467530828 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55875feba1b0 0x5587602af630 unknown :-1 s=CLOSED pgs=1104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:09.122 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:09.115+0000 7efe94b49740 1 --2- 172.21.15.39:0/1467530828 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55875fea19f0 0x5587602af050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:09.123 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:09.115+0000 7efe94b49740 1 -- 172.21.15.39:0/1467530828 >> 172.21.15.39:0/1467530828 conn(0x55875fed7460 msgr2=0x55876028f3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:54:09.123 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:09.115+0000 7efe94b49740 1 -- 172.21.15.39:0/1467530828 shutdown_connections 2024-09-21T03:54:09.123 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:09.116+0000 7efe94b49740 1 -- 172.21.15.39:0/1467530828 wait complete. 2024-09-21T03:54:09.123 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:09.116+0000 7efe94b49740 20 client.17601 trim_cache size 0 max 16384 2024-09-21T03:54:09.123 INFO:teuthology.orchestra.run.smithi039.stdout:[] 2024-09-21T03:54:09.124 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:09.116+0000 7efe8e5d5640 1 -- 172.21.15.39:0/3364131231 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7efe700b66b0 msgr2=0x7efe700b8bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:54:09.124 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:09.116+0000 7efe8e5d5640 1 --2- 172.21.15.39:0/3364131231 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7efe700b66b0 0x7efe700b8bc0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7efe800094e0 tx=0x7efe80009380 comp rx=0 tx=0).stop 2024-09-21T03:54:09.124 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:09.116+0000 7efe8e5d5640 1 -- 172.21.15.39:0/3364131231 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efe88102240 msgr2=0x7efe8819b270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:54:09.124 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:09.116+0000 7efe8e5d5640 1 --2- 172.21.15.39:0/3364131231 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efe88102240 0x7efe8819b270 secure :-1 s=READY pgs=1100 cs=0 l=1 rev1=1 crypto rx=0x7efe7800c920 tx=0x7efe7800cde0 comp rx=0 tx=0).stop 2024-09-21T03:54:09.124 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:09.116+0000 7efe8e5d5640 1 -- 172.21.15.39:0/3364131231 shutdown_connections 2024-09-21T03:54:09.124 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:09.116+0000 7efe8e5d5640 1 --2- 172.21.15.39:0/3364131231 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7efe700b66b0 0x7efe700b8bc0 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:09.124 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:09.116+0000 7efe8e5d5640 1 --2- 172.21.15.39:0/3364131231 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7efe881084d0 0x7efe8819bcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:09.124 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:09.116+0000 7efe8e5d5640 1 --2- 172.21.15.39:0/3364131231 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7efe88107b80 0x7efe8819b7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:09.124 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:09.116+0000 7efe8e5d5640 1 --2- 172.21.15.39:0/3364131231 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efe88102240 0x7efe8819b270 unknown :-1 s=CLOSED pgs=1100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:09.124 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:09.116+0000 7efe8e5d5640 1 -- 172.21.15.39:0/3364131231 >> 172.21.15.39:0/3364131231 conn(0x7efe88079f90 msgr2=0x7efe88105b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:54:09.124 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:09.116+0000 7efe8e5d5640 1 -- 172.21.15.39:0/3364131231 shutdown_connections 2024-09-21T03:54:09.124 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:09.116+0000 7efe8e5d5640 1 -- 172.21.15.39:0/3364131231 wait complete. 2024-09-21T03:54:09.218 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:54:08 smithi161 ceph-mon[34533]: pgmap v570: 97 pgs: 97 active+clean; 10 MiB data, 525 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 44 KiB/s wr, 25 op/s 2024-09-21T03:54:09.225 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:54:08 smithi150 ceph-mon[34289]: pgmap v570: 97 pgs: 97 active+clean; 10 MiB data, 525 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 44 KiB/s wr, 25 op/s 2024-09-21T03:54:09.799 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 51116d5c-77ca-11ef-baf5-efdc52797490 -- ceph tell mds.1:3 damage ls 2024-09-21T03:54:10.060 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/51116d5c-77ca-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:54:10.517 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:10.515+0000 7fe51fce3640 1 -- 172.21.15.39:0/3879096801 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe518109df0 msgr2=0x7fe518110620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:54:10.517 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:10.515+0000 7fe51fce3640 1 --2- 172.21.15.39:0/3879096801 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe518109df0 0x7fe518110620 secure :-1 s=READY pgs=1105 cs=0 l=1 rev1=1 crypto rx=0x7fe510009ca0 tx=0x7fe51002a8c0 comp rx=0 tx=0).stop 2024-09-21T03:54:10.517 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:10.515+0000 7fe51fce3640 1 -- 172.21.15.39:0/3879096801 shutdown_connections 2024-09-21T03:54:10.517 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:10.515+0000 7fe51fce3640 1 --2- 172.21.15.39:0/3879096801 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe518109df0 0x7fe518110620 unknown :-1 s=CLOSED pgs=1105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:10.517 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:10.515+0000 7fe51fce3640 1 --2- 172.21.15.39:0/3879096801 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe5181094a0 0x7fe5181098b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:10.517 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:10.515+0000 7fe51fce3640 1 --2- 172.21.15.39:0/3879096801 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe518103b60 0x7fe518103f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:10.517 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:10.515+0000 7fe51fce3640 1 -- 172.21.15.39:0/3879096801 >> 172.21.15.39:0/3879096801 conn(0x7fe51807b850 msgr2=0x7fe51807bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:54:10.517 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:10.515+0000 7fe51fce3640 1 -- 172.21.15.39:0/3879096801 shutdown_connections 2024-09-21T03:54:10.518 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:10.515+0000 7fe51fce3640 1 -- 172.21.15.39:0/3879096801 wait complete. 2024-09-21T03:54:10.519 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:10.516+0000 7fe51fce3640 1 Processor -- start 2024-09-21T03:54:10.519 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:10.516+0000 7fe51fce3640 1 -- start start 2024-09-21T03:54:10.519 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:10.516+0000 7fe51fce3640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe518103b60 0x7fe51819c6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:54:10.519 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:10.516+0000 7fe51fce3640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe5181094a0 0x7fe51819cc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:54:10.520 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:10.516+0000 7fe51fce3640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe518109df0 0x7fe5181a1ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:54:10.520 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:10.516+0000 7fe51fce3640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fe518113ab0 con 0x7fe518109df0 2024-09-21T03:54:10.520 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:10.516+0000 7fe51fce3640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fe518113930 con 0x7fe518103b60 2024-09-21T03:54:10.520 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:10.516+0000 7fe51fce3640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fe518113c30 con 0x7fe5181094a0 2024-09-21T03:54:10.520 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:10.516+0000 7fe51e280640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe518109df0 0x7fe5181a1ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:54:10.520 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:10.516+0000 7fe51e280640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe518109df0 0x7fe5181a1ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:46064/0 (socket says 172.21.15.39:46064) 2024-09-21T03:54:10.520 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:10.516+0000 7fe51e280640 1 -- 172.21.15.39:0/2278512885 learned_addr learned my addr 172.21.15.39:0/2278512885 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:54:10.520 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:10.516+0000 7fe51da7f640 1 --2- 172.21.15.39:0/2278512885 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe518103b60 0x7fe51819c6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:54:10.520 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:10.516+0000 7fe51d27e640 1 --2- 172.21.15.39:0/2278512885 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe5181094a0 0x7fe51819cc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:54:10.520 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:10.517+0000 7fe51e280640 1 -- 172.21.15.39:0/2278512885 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe5181094a0 msgr2=0x7fe51819cc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:54:10.520 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:10.517+0000 7fe51e280640 1 --2- 172.21.15.39:0/2278512885 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe5181094a0 0x7fe51819cc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:10.520 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:10.517+0000 7fe51e280640 1 -- 172.21.15.39:0/2278512885 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe518103b60 msgr2=0x7fe51819c6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:54:10.520 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:10.517+0000 7fe51e280640 1 --2- 172.21.15.39:0/2278512885 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe518103b60 0x7fe51819c6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:10.520 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:10.517+0000 7fe51e280640 1 -- 172.21.15.39:0/2278512885 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe5181a2610 con 0x7fe518109df0 2024-09-21T03:54:10.521 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:10.517+0000 7fe51d27e640 1 --2- 172.21.15.39:0/2278512885 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe5181094a0 0x7fe51819cc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:54:10.521 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:10.517+0000 7fe51da7f640 1 --2- 172.21.15.39:0/2278512885 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe518103b60 0x7fe51819c6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:54:10.521 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:10.517+0000 7fe51e280640 1 --2- 172.21.15.39:0/2278512885 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe518109df0 0x7fe5181a1ee0 secure :-1 s=READY pgs=1106 cs=0 l=1 rev1=1 crypto rx=0x7fe510005cc0 tx=0x7fe510031700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:54:10.521 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:10.517+0000 7fe50effd640 1 -- 172.21.15.39:0/2278512885 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe510003b70 con 0x7fe518109df0 2024-09-21T03:54:10.521 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:10.517+0000 7fe51fce3640 1 -- 172.21.15.39:0/2278512885 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe5181a28a0 con 0x7fe518109df0 2024-09-21T03:54:10.521 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:10.517+0000 7fe50effd640 1 -- 172.21.15.39:0/2278512885 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe5100318f0 con 0x7fe518109df0 2024-09-21T03:54:10.521 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:10.517+0000 7fe50effd640 1 -- 172.21.15.39:0/2278512885 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe51002f950 con 0x7fe518109df0 2024-09-21T03:54:10.521 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:10.517+0000 7fe51fce3640 1 -- 172.21.15.39:0/2278512885 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe5181a9060 con 0x7fe518109df0 2024-09-21T03:54:10.521 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:10.520+0000 7fe526257740 1 Processor -- start 2024-09-21T03:54:10.522 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:10.520+0000 7fe50effd640 1 -- 172.21.15.39:0/2278512885 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 27) ==== 154643+0+0 (secure 0 0 0) 0x7fe51002fb50 con 0x7fe518109df0 2024-09-21T03:54:10.522 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:10.520+0000 7fe526257740 1 -- start start 2024-09-21T03:54:10.524 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:10.521+0000 7fe526257740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e0cd792070 0x55e0cdb396a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:54:10.524 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:10.521+0000 7fe526257740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55e0cd7691b0 0x55e0cdb39be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:54:10.524 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:10.521+0000 7fe526257740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e0cd771390 0x55e0cdb3a120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:54:10.524 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:10.521+0000 7fe526257740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x55e0cdaafea0 con 0x55e0cd7691b0 2024-09-21T03:54:10.524 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:10.521+0000 7fe526257740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x55e0cda9f2d0 con 0x55e0cd771390 2024-09-21T03:54:10.524 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:10.521+0000 7fe526257740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55e0cdaa30e0 con 0x55e0cd792070 2024-09-21T03:54:10.524 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:10.521+0000 7fe51da7f640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55e0cd7691b0 0x55e0cdb39be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:54:10.524 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:10.521+0000 7fe51da7f640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55e0cd7691b0 0x55e0cdb39be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:46068/0 (socket says 172.21.15.39:46068) 2024-09-21T03:54:10.524 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:10.521+0000 7fe51da7f640 1 -- 172.21.15.39:0/4088771975 learned_addr learned my addr 172.21.15.39:0/4088771975 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:54:10.524 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:10.521+0000 7fe51d27e640 1 --2- 172.21.15.39:0/4088771975 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e0cd771390 0x55e0cdb3a120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:54:10.524 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:10.521+0000 7fe51d27e640 1 --2- 172.21.15.39:0/4088771975 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e0cd792070 0x55e0cdb396a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:54:10.524 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:10.521+0000 7fe50effd640 1 --2- 172.21.15.39:0/2278512885 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7fe5040b6680 0x7fe5040b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:54:10.525 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:10.521+0000 7fe51da7f640 1 -- 172.21.15.39:0/4088771975 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e0cd792070 msgr2=0x55e0cdb396a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:54:10.525 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:10.521+0000 7fe51da7f640 1 --2- 172.21.15.39:0/4088771975 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e0cd792070 0x55e0cdb396a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:10.525 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:10.521+0000 7fe51da7f640 1 -- 172.21.15.39:0/4088771975 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e0cd771390 msgr2=0x55e0cdb3a120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:54:10.525 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:10.521+0000 7fe51da7f640 1 --2- 172.21.15.39:0/4088771975 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e0cd771390 0x55e0cdb3a120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:10.525 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:10.521+0000 7fe51da7f640 1 -- 172.21.15.39:0/4088771975 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e0cd7fd650 con 0x55e0cd7691b0 2024-09-21T03:54:10.525 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:10.521+0000 7fe51d27e640 1 --2- 172.21.15.39:0/4088771975 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e0cd792070 0x55e0cdb396a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:54:10.525 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:10.521+0000 7fe51e280640 1 --2- 172.21.15.39:0/2278512885 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7fe5040b6680 0x7fe5040b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:54:10.525 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:10.522+0000 7fe50effd640 1 -- 172.21.15.39:0/2278512885 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8572+0+0 (secure 0 0 0) 0x7fe5101088d0 con 0x7fe518109df0 2024-09-21T03:54:10.525 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:10.522+0000 7fe51da7f640 1 --2- 172.21.15.39:0/4088771975 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55e0cd7691b0 0x55e0cdb39be0 secure :-1 s=READY pgs=1107 cs=0 l=1 rev1=1 crypto rx=0x7fe5140115f0 tx=0x7fe514011ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=33f8f0ade91f54e8 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:54:10.525 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:10.522+0000 7fe503fff640 1 -- 172.21.15.39:0/4088771975 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe51401e070 con 0x55e0cd7691b0 2024-09-21T03:54:10.525 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:10.522+0000 7fe503fff640 1 -- 172.21.15.39:0/4088771975 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe51400faf0 con 0x55e0cd7691b0 2024-09-21T03:54:10.525 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:10.522+0000 7fe503fff640 1 -- 172.21.15.39:0/4088771975 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe514019ac0 con 0x55e0cd7691b0 2024-09-21T03:54:10.528 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:10.523+0000 7fe526257740 1 -- 172.21.15.39:0/4088771975 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55e0cd7691b0 msgr2=0x55e0cdb39be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:54:10.528 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:10.523+0000 7fe51e280640 1 --2- 172.21.15.39:0/2278512885 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7fe5040b6680 0x7fe5040b8b90 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7fe5101060a0 tx=0x7fe510037e60 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:54:10.528 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:10.523+0000 7fe526257740 1 --2- 172.21.15.39:0/4088771975 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55e0cd7691b0 0x55e0cdb39be0 secure :-1 s=READY pgs=1107 cs=0 l=1 rev1=1 crypto rx=0x7fe5140115f0 tx=0x7fe514011ab0 comp rx=0 tx=0).stop 2024-09-21T03:54:10.528 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:10.523+0000 7fe526257740 1 -- 172.21.15.39:0/4088771975 shutdown_connections 2024-09-21T03:54:10.529 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:10.523+0000 7fe526257740 1 --2- 172.21.15.39:0/4088771975 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e0cd771390 0x55e0cdb3a120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:10.529 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:10.523+0000 7fe526257740 1 --2- 172.21.15.39:0/4088771975 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55e0cd7691b0 0x55e0cdb39be0 unknown :-1 s=CLOSED pgs=1107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:10.529 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:10.523+0000 7fe526257740 1 --2- 172.21.15.39:0/4088771975 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e0cd792070 0x55e0cdb396a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:10.529 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:10.523+0000 7fe526257740 1 -- 172.21.15.39:0/4088771975 >> 172.21.15.39:0/4088771975 conn(0x55e0cd786460 msgr2=0x55e0cdb37590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:54:10.529 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:10.523+0000 7fe526257740 1 -- 172.21.15.39:0/4088771975 shutdown_connections 2024-09-21T03:54:10.529 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:10.523+0000 7fe526257740 1 -- 172.21.15.39:0/4088771975 wait complete. 2024-09-21T03:54:10.529 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:10.523+0000 7fe526257740 1 -- start start 2024-09-21T03:54:10.529 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:10.523+0000 7fe526257740 1 Processor -- start 2024-09-21T03:54:10.529 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:10.524+0000 7fe526257740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55e0cd7691b0 0x55e0cdb44d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:54:10.529 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:10.524+0000 7fe526257740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e0cd771390 0x55e0cdb452d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:54:10.529 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:10.524+0000 7fe526257740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e0cd792070 0x55e0cdb511a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:54:10.529 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:10.524+0000 7fe526257740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x55e0cd900700 con 0x55e0cd7691b0 2024-09-21T03:54:10.529 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:10.524+0000 7fe526257740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x55e0cda3a160 con 0x55e0cd771390 2024-09-21T03:54:10.529 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:10.524+0000 7fe526257740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55e0cdb47e80 con 0x55e0cd792070 2024-09-21T03:54:10.530 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:10.524+0000 7fe51d27e640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55e0cd7691b0 0x55e0cdb44d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:54:10.530 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:10.524+0000 7fe526257740 10 client.0 _ll_register_callbacks cb 0x55e0cda539e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:54:10.530 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:10.524+0000 7fe51d27e640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55e0cd7691b0 0x55e0cdb44d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:46084/0 (socket says 172.21.15.39:46084) 2024-09-21T03:54:10.530 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:10.524+0000 7fe51d27e640 1 -- 172.21.15.39:0/2663392555 learned_addr learned my addr 172.21.15.39:0/2663392555 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:54:10.530 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:10.524+0000 7fe51da7f640 1 --2- 172.21.15.39:0/2663392555 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e0cd771390 0x55e0cdb452d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:54:10.530 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:10.524+0000 7fe51d27e640 1 --2- 172.21.15.39:0/2663392555 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e0cd792070 0x55e0cdb511a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:54:10.530 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:10.524+0000 7fe51d27e640 1 -- 172.21.15.39:0/2663392555 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e0cd792070 msgr2=0x55e0cdb511a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:54:10.530 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:10.524+0000 7fe51d27e640 1 --2- 172.21.15.39:0/2663392555 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e0cd792070 0x55e0cdb511a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:10.530 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:10.524+0000 7fe51d27e640 1 -- 172.21.15.39:0/2663392555 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e0cd771390 msgr2=0x55e0cdb452d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:54:10.530 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:10.524+0000 7fe51d27e640 1 --2- 172.21.15.39:0/2663392555 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e0cd771390 0x55e0cdb452d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:10.530 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:10.524+0000 7fe51d27e640 1 -- 172.21.15.39:0/2663392555 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e0cda67c90 con 0x55e0cd7691b0 2024-09-21T03:54:10.530 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:10.524+0000 7fe51da7f640 1 --2- 172.21.15.39:0/2663392555 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e0cd771390 0x55e0cdb452d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:54:10.530 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:10.525+0000 7fe51d27e640 1 --2- 172.21.15.39:0/2663392555 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55e0cd7691b0 0x55e0cdb44d90 secure :-1 s=READY pgs=1108 cs=0 l=1 rev1=1 crypto rx=0x7fe508009ed0 tx=0x7fe5080046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e4662108c870a25f server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:54:10.531 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:10.525+0000 7fe5017fa640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-09-21T03:54:10.531 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:10.525+0000 7fe5017fa640 1 -- 172.21.15.39:0/2663392555 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe508011e00 con 0x55e0cd7691b0 2024-09-21T03:54:10.531 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:10.525+0000 7fe526257740 1 -- 172.21.15.39:0/2663392555 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe51400faf0 con 0x55e0cd7691b0 2024-09-21T03:54:10.531 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:10.525+0000 7fe5017fa640 1 -- 172.21.15.39:0/2663392555 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8572+0+0 (secure 0 0 0) 0x7fe508009040 con 0x55e0cd7691b0 2024-09-21T03:54:10.531 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:10.525+0000 7fe5017fa640 1 -- 172.21.15.39:0/2663392555 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fe508004030 con 0x55e0cd7691b0 2024-09-21T03:54:10.531 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:10.525+0000 7fe526257740 10 client.17613 fetch_fsmap learned FSMap version 29 2024-09-21T03:54:10.531 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:10.525+0000 7fe526257740 1 -- 172.21.15.39:0/2663392555 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55e0cdb53c40 con 0x55e0cd7691b0 2024-09-21T03:54:10.531 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:10.525+0000 7fe5017fa640 1 -- 172.21.15.39:0/2663392555 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe51400faf0 con 0x55e0cd7691b0 2024-09-21T03:54:10.531 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:10.525+0000 7fe5017fa640 1 -- 172.21.15.39:0/2663392555 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fe50800f450 con 0x55e0cd7691b0 2024-09-21T03:54:10.531 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:10.525+0000 7fe526257740 10 client.17613 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T03:54:10.531 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:10.525+0000 7fe526257740 10 client.17613 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-21T03:54:10.531 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:10.526+0000 7fe526257740 20 client.17613 populate_metadata read hostname 'smithi039' 2024-09-21T03:54:10.531 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:10.526+0000 7fe526257740 1 --2- 172.21.15.39:0/2663392555 >> [v2:172.21.15.161:6836/3451544597,v1:172.21.15.161:6837/3451544597] conn(0x55e0cd7509f0 0x55e0cdb57060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:54:10.532 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:10.526+0000 7fe526257740 4 client.17613 mds_command: new command op to 24646 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:54:10.532 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:10.526+0000 7fe526257740 1 -- 172.21.15.39:0/2663392555 --> [v2:172.21.15.161:6836/3451544597,v1:172.21.15.161:6837/3451544597] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e0cdab1df0 con 0x55e0cd7509f0 2024-09-21T03:54:10.532 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:10.526+0000 7fe51da7f640 1 --2- 172.21.15.39:0/2663392555 >> [v2:172.21.15.161:6836/3451544597,v1:172.21.15.161:6837/3451544597] conn(0x55e0cd7509f0 0x55e0cdb57060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:54:10.532 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:10.526+0000 7fe5017fa640 10 client.17613 ms_handle_connect on v2:172.21.15.161:6836/3451544597 2024-09-21T03:54:10.532 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:10.526+0000 7fe51da7f640 1 --2- 172.21.15.39:0/2663392555 >> [v2:172.21.15.161:6836/3451544597,v1:172.21.15.161:6837/3451544597] conn(0x55e0cd7509f0 0x55e0cdb57060 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=81a6c185cf3d919f server_cookie=c22e482e5053bef7 in_seq=0 out_seq=0 2024-09-21T03:54:10.533 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:10.527+0000 7fe5017fa640 1 -- 172.21.15.39:0/2663392555 <== mds.3 v2:172.21.15.161:6836/3451544597 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55e0cdab1df0 con 0x55e0cd7509f0 2024-09-21T03:54:10.533 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:10.527+0000 7fe5017fa640 10 client.17613 handle_command_reply: tid=0 2024-09-21T03:54:10.533 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:10.527+0000 7fe526257740 1 client.17613 shutdown 2024-09-21T03:54:10.533 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:10.527+0000 7fe526257740 1 -- 172.21.15.39:0/2663392555 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55e0cd7691b0 msgr2=0x55e0cdb44d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:54:10.533 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:10.527+0000 7fe526257740 1 --2- 172.21.15.39:0/2663392555 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55e0cd7691b0 0x55e0cdb44d90 secure :-1 s=READY pgs=1108 cs=0 l=1 rev1=1 crypto rx=0x7fe508009ed0 tx=0x7fe5080046c0 comp rx=0 tx=0).stop 2024-09-21T03:54:10.533 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:10.527+0000 7fe526257740 1 -- 172.21.15.39:0/2663392555 shutdown_connections 2024-09-21T03:54:10.533 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:10.527+0000 7fe526257740 1 --2- 172.21.15.39:0/2663392555 >> [v2:172.21.15.161:6836/3451544597,v1:172.21.15.161:6837/3451544597] conn(0x55e0cd7509f0 0x55e0cdb57060 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:10.534 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:10.527+0000 7fe526257740 1 --2- 172.21.15.39:0/2663392555 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e0cd792070 0x55e0cdb511a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:10.534 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:10.527+0000 7fe526257740 1 --2- 172.21.15.39:0/2663392555 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e0cd771390 0x55e0cdb452d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:10.534 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:10.527+0000 7fe526257740 1 --2- 172.21.15.39:0/2663392555 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55e0cd7691b0 0x55e0cdb44d90 unknown :-1 s=CLOSED pgs=1108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:10.534 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:10.527+0000 7fe5017fa640 0 client.17613 ms_handle_reset on v2:172.21.15.161:6836/3451544597 2024-09-21T03:54:10.534 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:10.527+0000 7fe526257740 1 -- 172.21.15.39:0/2663392555 >> 172.21.15.39:0/2663392555 conn(0x55e0cd786460 msgr2=0x55e0cdb3c830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:54:10.534 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:10.527+0000 7fe526257740 1 -- 172.21.15.39:0/2663392555 shutdown_connections 2024-09-21T03:54:10.534 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:10.527+0000 7fe526257740 1 -- 172.21.15.39:0/2663392555 wait complete. 2024-09-21T03:54:10.534 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:10.528+0000 7fe526257740 20 client.17613 trim_cache size 0 max 16384 2024-09-21T03:54:10.543 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:10.541+0000 7fe526257740 1 Processor -- start 2024-09-21T03:54:10.543 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:10.541+0000 7fe526257740 1 -- start start 2024-09-21T03:54:10.543 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:10.541+0000 7fe526257740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e0cd7691b0 0x55e0cdb469c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:54:10.543 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:10.541+0000 7fe526257740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e0cd771390 0x55e0cdb36bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:54:10.543 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:10.541+0000 7fe526257740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55e0cd792070 0x55e0cdb37100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:54:10.544 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:10.541+0000 7fe526257740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fe5181b1ab0 con 0x55e0cd792070 2024-09-21T03:54:10.544 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:10.541+0000 7fe526257740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fe5181b35f0 con 0x55e0cd7691b0 2024-09-21T03:54:10.544 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:10.541+0000 7fe526257740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fe5181b2180 con 0x55e0cd771390 2024-09-21T03:54:10.544 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:10.541+0000 7fe51d27e640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55e0cd792070 0x55e0cdb37100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:54:10.544 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:10.541+0000 7fe51d27e640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55e0cd792070 0x55e0cdb37100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:46100/0 (socket says 172.21.15.39:46100) 2024-09-21T03:54:10.544 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:10.541+0000 7fe51d27e640 1 -- 172.21.15.39:0/684076467 learned_addr learned my addr 172.21.15.39:0/684076467 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:54:10.544 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:10.541+0000 7fe51da7f640 1 --2- 172.21.15.39:0/684076467 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e0cd771390 0x55e0cdb36bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:54:10.544 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:10.541+0000 7fe51d27e640 1 --2- 172.21.15.39:0/684076467 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e0cd7691b0 0x55e0cdb469c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:54:10.544 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:10.541+0000 7fe51d27e640 1 -- 172.21.15.39:0/684076467 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e0cd771390 msgr2=0x55e0cdb36bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:54:10.544 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:10.541+0000 7fe51d27e640 1 --2- 172.21.15.39:0/684076467 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e0cd771390 0x55e0cdb36bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:10.544 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:10.541+0000 7fe51d27e640 1 -- 172.21.15.39:0/684076467 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e0cd7691b0 msgr2=0x55e0cdb469c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:54:10.544 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:10.541+0000 7fe51d27e640 1 --2- 172.21.15.39:0/684076467 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e0cd7691b0 0x55e0cdb469c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:10.544 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:10.541+0000 7fe51d27e640 1 -- 172.21.15.39:0/684076467 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e0cdb43290 con 0x55e0cd792070 2024-09-21T03:54:10.544 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:10.542+0000 7fe51d27e640 1 --2- 172.21.15.39:0/684076467 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55e0cd792070 0x55e0cdb37100 secure :-1 s=READY pgs=1109 cs=0 l=1 rev1=1 crypto rx=0x7fe50800f820 tx=0x7fe508009600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=284651569ff3731d server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:54:10.547 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:10.542+0000 7fe5037fe640 1 -- 172.21.15.39:0/684076467 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe50800f450 con 0x55e0cd792070 2024-09-21T03:54:10.547 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:10.542+0000 7fe5037fe640 1 -- 172.21.15.39:0/684076467 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe50800f5f0 con 0x55e0cd792070 2024-09-21T03:54:10.547 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:10.542+0000 7fe5037fe640 1 -- 172.21.15.39:0/684076467 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe508005290 con 0x55e0cd792070 2024-09-21T03:54:10.547 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:10.543+0000 7fe526257740 1 -- 172.21.15.39:0/684076467 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55e0cd792070 msgr2=0x55e0cdb37100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:54:10.547 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:10.543+0000 7fe526257740 1 --2- 172.21.15.39:0/684076467 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55e0cd792070 0x55e0cdb37100 secure :-1 s=READY pgs=1109 cs=0 l=1 rev1=1 crypto rx=0x7fe50800f820 tx=0x7fe508009600 comp rx=0 tx=0).stop 2024-09-21T03:54:10.548 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:10.543+0000 7fe526257740 1 -- 172.21.15.39:0/684076467 shutdown_connections 2024-09-21T03:54:10.548 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:10.543+0000 7fe526257740 1 --2- 172.21.15.39:0/684076467 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55e0cd792070 0x55e0cdb37100 unknown :-1 s=CLOSED pgs=1109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:10.548 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:10.543+0000 7fe526257740 1 --2- 172.21.15.39:0/684076467 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e0cd771390 0x55e0cdb36bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:10.548 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:10.543+0000 7fe526257740 1 --2- 172.21.15.39:0/684076467 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e0cd7691b0 0x55e0cdb469c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:10.548 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:10.543+0000 7fe526257740 1 -- 172.21.15.39:0/684076467 >> 172.21.15.39:0/684076467 conn(0x55e0cd786460 msgr2=0x55e0cdb462b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:54:10.548 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:10.543+0000 7fe526257740 1 -- 172.21.15.39:0/684076467 shutdown_connections 2024-09-21T03:54:10.548 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:10.543+0000 7fe526257740 1 -- 172.21.15.39:0/684076467 wait complete. 2024-09-21T03:54:10.548 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:10.543+0000 7fe526257740 1 -- start start 2024-09-21T03:54:10.548 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:10.543+0000 7fe526257740 1 Processor -- start 2024-09-21T03:54:10.548 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:10.544+0000 7fe526257740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e0cd7691b0 0x55e0cdb49810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:54:10.548 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:10.544+0000 7fe526257740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55e0cd771390 0x55e0cdb49d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:54:10.548 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:10.544+0000 7fe526257740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e0cd792070 0x55e0cdbc0a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:54:10.548 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:10.544+0000 7fe526257740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fe5181b2f20 con 0x55e0cd771390 2024-09-21T03:54:10.548 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:10.544+0000 7fe526257740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fe5181b2850 con 0x55e0cd792070 2024-09-21T03:54:10.549 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:10.544+0000 7fe526257740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fe5181afab0 con 0x55e0cd7691b0 2024-09-21T03:54:10.549 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:10.544+0000 7fe526257740 10 client.0 _ll_register_callbacks cb 0x7fe508009c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:54:10.549 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:10.544+0000 7fe51da7f640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55e0cd771390 0x55e0cdb49d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:54:10.549 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:10.544+0000 7fe51da7f640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55e0cd771390 0x55e0cdb49d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:46116/0 (socket says 172.21.15.39:46116) 2024-09-21T03:54:10.549 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:10.544+0000 7fe51da7f640 1 -- 172.21.15.39:0/3385063148 learned_addr learned my addr 172.21.15.39:0/3385063148 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:54:10.549 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:10.544+0000 7fe51d27e640 1 --2- 172.21.15.39:0/3385063148 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e0cd7691b0 0x55e0cdb49810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:54:10.549 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:10.544+0000 7fe51d27e640 1 --2- 172.21.15.39:0/3385063148 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e0cd792070 0x55e0cdbc0a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:54:10.549 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:10.544+0000 7fe51da7f640 1 -- 172.21.15.39:0/3385063148 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e0cd7691b0 msgr2=0x55e0cdb49810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:54:10.549 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:10.544+0000 7fe51da7f640 1 --2- 172.21.15.39:0/3385063148 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e0cd7691b0 0x55e0cdb49810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:10.549 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:10.544+0000 7fe51da7f640 1 -- 172.21.15.39:0/3385063148 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e0cd792070 msgr2=0x55e0cdbc0a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:54:10.549 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:10.544+0000 7fe51da7f640 1 --2- 172.21.15.39:0/3385063148 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e0cd792070 0x55e0cdbc0a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:10.549 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:10.544+0000 7fe51da7f640 1 -- 172.21.15.39:0/3385063148 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e0cdbc0f60 con 0x55e0cd771390 2024-09-21T03:54:10.549 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:10.544+0000 7fe51d27e640 1 --2- 172.21.15.39:0/3385063148 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e0cd7691b0 0x55e0cdb49810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:54:10.549 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:10.545+0000 7fe51da7f640 1 --2- 172.21.15.39:0/3385063148 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55e0cd771390 0x55e0cdb49d50 secure :-1 s=READY pgs=1110 cs=0 l=1 rev1=1 crypto rx=0x55e0cdb38320 tx=0x7fe5140196a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c6dc3561aafbaf27 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:54:10.549 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:10.545+0000 7fe5017fa640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-09-21T03:54:10.550 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:10.545+0000 7fe5017fa640 1 -- 172.21.15.39:0/3385063148 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe51401e040 con 0x55e0cd771390 2024-09-21T03:54:10.550 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:10.545+0000 7fe5017fa640 1 -- 172.21.15.39:0/3385063148 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8572+0+0 (secure 0 0 0) 0x7fe5140050b0 con 0x55e0cd771390 2024-09-21T03:54:10.550 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:10.545+0000 7fe526257740 1 -- 172.21.15.39:0/3385063148 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe50800f5f0 con 0x55e0cd771390 2024-09-21T03:54:10.550 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:10.545+0000 7fe5017fa640 1 -- 172.21.15.39:0/3385063148 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fe5140052b0 con 0x55e0cd771390 2024-09-21T03:54:10.550 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:10.545+0000 7fe526257740 10 client.17619 fetch_fsmap learned FSMap version 29 2024-09-21T03:54:10.550 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:10.545+0000 7fe526257740 1 -- 172.21.15.39:0/3385063148 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55e0cdbc2190 con 0x55e0cd771390 2024-09-21T03:54:10.550 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:10.545+0000 7fe5017fa640 1 -- 172.21.15.39:0/3385063148 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe50800f5f0 con 0x55e0cd771390 2024-09-21T03:54:10.550 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:10.545+0000 7fe5017fa640 1 -- 172.21.15.39:0/3385063148 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fe51402b070 con 0x55e0cd771390 2024-09-21T03:54:10.550 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:10.546+0000 7fe526257740 10 client.17619 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T03:54:10.550 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:10.546+0000 7fe526257740 10 client.17619 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-21T03:54:10.550 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:10.546+0000 7fe526257740 20 client.17619 populate_metadata read hostname 'smithi039' 2024-09-21T03:54:10.550 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:10.546+0000 7fe526257740 1 --2- 172.21.15.39:0/3385063148 >> [v2:172.21.15.161:6836/3451544597,v1:172.21.15.161:6837/3451544597] conn(0x55e0cd725da0 0x55e0cdbc47f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:54:10.550 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:10.546+0000 7fe526257740 4 client.17619 mds_command: new command op to 24646 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T03:54:10.551 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:10.546+0000 7fe526257740 1 -- 172.21.15.39:0/3385063148 --> [v2:172.21.15.161:6836/3451544597,v1:172.21.15.161:6837/3451544597] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55e0cdaad8a0 con 0x55e0cd725da0 2024-09-21T03:54:10.551 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:10.546+0000 7fe51d27e640 1 --2- 172.21.15.39:0/3385063148 >> [v2:172.21.15.161:6836/3451544597,v1:172.21.15.161:6837/3451544597] conn(0x55e0cd725da0 0x55e0cdbc47f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:54:10.552 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:10.546+0000 7fe5017fa640 10 client.17619 ms_handle_connect on v2:172.21.15.161:6836/3451544597 2024-09-21T03:54:10.552 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:10.546+0000 7fe51d27e640 1 --2- 172.21.15.39:0/3385063148 >> [v2:172.21.15.161:6836/3451544597,v1:172.21.15.161:6837/3451544597] conn(0x55e0cd725da0 0x55e0cdbc47f0 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=763cc514c66c7899 server_cookie=c0ebd1bef08272f1 in_seq=0 out_seq=0 2024-09-21T03:54:10.552 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:10.547+0000 7fe5017fa640 1 -- 172.21.15.39:0/3385063148 <== mds.3 v2:172.21.15.161:6836/3451544597 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55e0cdaad8a0 con 0x55e0cd725da0 2024-09-21T03:54:10.552 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:10.547+0000 7fe5017fa640 10 client.17619 handle_command_reply: tid=0 2024-09-21T03:54:10.552 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:10.547+0000 7fe526257740 1 client.17619 shutdown 2024-09-21T03:54:10.552 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:10.547+0000 7fe526257740 1 -- 172.21.15.39:0/3385063148 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55e0cd771390 msgr2=0x55e0cdb49d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:54:10.552 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:10.547+0000 7fe526257740 1 --2- 172.21.15.39:0/3385063148 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55e0cd771390 0x55e0cdb49d50 secure :-1 s=READY pgs=1110 cs=0 l=1 rev1=1 crypto rx=0x55e0cdb38320 tx=0x7fe5140196a0 comp rx=0 tx=0).stop 2024-09-21T03:54:10.552 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:10.547+0000 7fe526257740 1 -- 172.21.15.39:0/3385063148 shutdown_connections 2024-09-21T03:54:10.552 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:10.547+0000 7fe526257740 1 --2- 172.21.15.39:0/3385063148 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e0cd792070 0x55e0cdbc0a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:10.552 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:10.547+0000 7fe526257740 1 --2- 172.21.15.39:0/3385063148 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55e0cd771390 0x55e0cdb49d50 unknown :-1 s=CLOSED pgs=1110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:10.552 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:10.547+0000 7fe526257740 1 --2- 172.21.15.39:0/3385063148 >> [v2:172.21.15.161:6836/3451544597,v1:172.21.15.161:6837/3451544597] conn(0x55e0cd725da0 0x55e0cdbc47f0 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:10.552 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:10.547+0000 7fe526257740 1 --2- 172.21.15.39:0/3385063148 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e0cd7691b0 0x55e0cdb49810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:10.552 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:10.547+0000 7fe5017fa640 0 client.17619 ms_handle_reset on v2:172.21.15.161:6836/3451544597 2024-09-21T03:54:10.553 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:10.547+0000 7fe526257740 1 -- 172.21.15.39:0/3385063148 >> 172.21.15.39:0/3385063148 conn(0x55e0cd786460 msgr2=0x55e0cdb3a530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:54:10.553 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:10.547+0000 7fe526257740 1 -- 172.21.15.39:0/3385063148 shutdown_connections 2024-09-21T03:54:10.553 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:10.547+0000 7fe526257740 1 -- 172.21.15.39:0/3385063148 wait complete. 2024-09-21T03:54:10.553 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:10.547+0000 7fe526257740 20 client.17619 trim_cache size 0 max 16384 2024-09-21T03:54:10.553 INFO:teuthology.orchestra.run.smithi039.stdout:[] 2024-09-21T03:54:10.554 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:10.548+0000 7fe51fce3640 1 -- 172.21.15.39:0/2278512885 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7fe5040b6680 msgr2=0x7fe5040b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:54:10.554 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:10.548+0000 7fe51fce3640 1 --2- 172.21.15.39:0/2278512885 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7fe5040b6680 0x7fe5040b8b90 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7fe5101060a0 tx=0x7fe510037e60 comp rx=0 tx=0).stop 2024-09-21T03:54:10.554 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:10.548+0000 7fe51fce3640 1 -- 172.21.15.39:0/2278512885 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe518109df0 msgr2=0x7fe5181a1ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:54:10.554 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:10.548+0000 7fe51fce3640 1 --2- 172.21.15.39:0/2278512885 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe518109df0 0x7fe5181a1ee0 secure :-1 s=READY pgs=1106 cs=0 l=1 rev1=1 crypto rx=0x7fe510005cc0 tx=0x7fe510031700 comp rx=0 tx=0).stop 2024-09-21T03:54:10.554 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:10.548+0000 7fe51fce3640 1 -- 172.21.15.39:0/2278512885 shutdown_connections 2024-09-21T03:54:10.554 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:10.548+0000 7fe51fce3640 1 --2- 172.21.15.39:0/2278512885 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7fe5040b6680 0x7fe5040b8b90 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:10.554 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:10.548+0000 7fe51fce3640 1 --2- 172.21.15.39:0/2278512885 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe518109df0 0x7fe5181a1ee0 unknown :-1 s=CLOSED pgs=1106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:10.554 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:10.548+0000 7fe51fce3640 1 --2- 172.21.15.39:0/2278512885 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe5181094a0 0x7fe51819cc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:10.554 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:10.548+0000 7fe51fce3640 1 --2- 172.21.15.39:0/2278512885 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe518103b60 0x7fe51819c6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:10.555 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:10.548+0000 7fe51fce3640 1 -- 172.21.15.39:0/2278512885 >> 172.21.15.39:0/2278512885 conn(0x7fe51807b850 msgr2=0x7fe518107350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:54:10.555 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:10.548+0000 7fe51fce3640 1 -- 172.21.15.39:0/2278512885 shutdown_connections 2024-09-21T03:54:10.555 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:10.548+0000 7fe51fce3640 1 -- 172.21.15.39:0/2278512885 wait complete. 2024-09-21T03:54:11.218 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:54:10 smithi161 ceph-mon[34533]: pgmap v571: 97 pgs: 97 active+clean; 10 MiB data, 525 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 44 KiB/s wr, 28 op/s 2024-09-21T03:54:11.218 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:54:10 smithi161 ceph-mon[34533]: osdmap e82: 12 total, 12 up, 12 in 2024-09-21T03:54:11.224 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:54:10 smithi150 ceph-mon[34289]: pgmap v571: 97 pgs: 97 active+clean; 10 MiB data, 525 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 44 KiB/s wr, 28 op/s 2024-09-21T03:54:11.225 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:54:10 smithi150 ceph-mon[34289]: osdmap e82: 12 total, 12 up, 12 in 2024-09-21T03:54:11.334 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:54:10 smithi039 ceph-mon[31211]: pgmap v571: 97 pgs: 97 active+clean; 10 MiB data, 525 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 44 KiB/s wr, 28 op/s 2024-09-21T03:54:11.334 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:54:10 smithi039 ceph-mon[31211]: osdmap e82: 12 total, 12 up, 12 in 2024-09-21T03:54:11.386 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 51116d5c-77ca-11ef-baf5-efdc52797490 -- ceph tell mds.1:2 damage ls 2024-09-21T03:54:11.653 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/51116d5c-77ca-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:54:12.169 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:12.167+0000 7ff9728a7640 1 -- 172.21.15.39:0/3514346507 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff96c102a60 msgr2=0x7ff96c102e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:54:12.169 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:12.167+0000 7ff9728a7640 1 --2- 172.21.15.39:0/3514346507 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff96c102a60 0x7ff96c102e70 secure :-1 s=READY pgs=1111 cs=0 l=1 rev1=1 crypto rx=0x7ff95c009900 tx=0x7ff95c02a410 comp rx=0 tx=0).stop 2024-09-21T03:54:12.169 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:12.167+0000 7ff9728a7640 1 -- 172.21.15.39:0/3514346507 shutdown_connections 2024-09-21T03:54:12.169 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:12.167+0000 7ff9728a7640 1 --2- 172.21.15.39:0/3514346507 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff96c108cf0 0x7ff96c10f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:12.169 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:12.167+0000 7ff9728a7640 1 --2- 172.21.15.39:0/3514346507 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff96c1083a0 0x7ff96c1087b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:12.169 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:12.167+0000 7ff9728a7640 1 --2- 172.21.15.39:0/3514346507 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff96c102a60 0x7ff96c102e70 unknown :-1 s=CLOSED pgs=1111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:12.169 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:12.167+0000 7ff9728a7640 1 -- 172.21.15.39:0/3514346507 >> 172.21.15.39:0/3514346507 conn(0x7ff96c079730 msgr2=0x7ff96c079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:54:12.169 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:12.167+0000 7ff9728a7640 1 -- 172.21.15.39:0/3514346507 shutdown_connections 2024-09-21T03:54:12.169 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:12.167+0000 7ff9728a7640 1 -- 172.21.15.39:0/3514346507 wait complete. 2024-09-21T03:54:12.171 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:12.167+0000 7ff9728a7640 1 Processor -- start 2024-09-21T03:54:12.171 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:12.168+0000 7ff9728a7640 1 -- start start 2024-09-21T03:54:12.171 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:12.168+0000 7ff9728a7640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff96c102a60 0x7ff96c19b1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:54:12.171 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:12.168+0000 7ff9728a7640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff96c1083a0 0x7ff96c19b710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:54:12.171 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:12.168+0000 7ff9728a7640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff96c108cf0 0x7ff96c19bc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:54:12.171 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:12.168+0000 7ff9728a7640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7ff96c112960 con 0x7ff96c108cf0 2024-09-21T03:54:12.171 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:12.168+0000 7ff9728a7640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7ff96c1127e0 con 0x7ff96c1083a0 2024-09-21T03:54:12.171 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:12.168+0000 7ff9728a7640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7ff96c112ae0 con 0x7ff96c102a60 2024-09-21T03:54:12.172 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:12.168+0000 7ff970e44640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff96c108cf0 0x7ff96c19bc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:54:12.172 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:12.168+0000 7ff970e44640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff96c108cf0 0x7ff96c19bc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:46128/0 (socket says 172.21.15.39:46128) 2024-09-21T03:54:12.172 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:12.168+0000 7ff970e44640 1 -- 172.21.15.39:0/3515515429 learned_addr learned my addr 172.21.15.39:0/3515515429 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:54:12.172 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:12.168+0000 7ff96bfff640 1 --2- 172.21.15.39:0/3515515429 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff96c102a60 0x7ff96c19b1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:54:12.172 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:12.168+0000 7ff963fff640 1 --2- 172.21.15.39:0/3515515429 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff96c1083a0 0x7ff96c19b710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:54:12.172 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:12.168+0000 7ff970e44640 1 -- 172.21.15.39:0/3515515429 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff96c102a60 msgr2=0x7ff96c19b1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:54:12.172 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:12.168+0000 7ff970e44640 1 --2- 172.21.15.39:0/3515515429 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff96c102a60 0x7ff96c19b1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:12.172 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:12.168+0000 7ff970e44640 1 -- 172.21.15.39:0/3515515429 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff96c1083a0 msgr2=0x7ff96c19b710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:54:12.172 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:12.168+0000 7ff970e44640 1 --2- 172.21.15.39:0/3515515429 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff96c1083a0 0x7ff96c19b710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:12.172 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:12.168+0000 7ff970e44640 1 -- 172.21.15.39:0/3515515429 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff96c1a64e0 con 0x7ff96c108cf0 2024-09-21T03:54:12.172 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:12.169+0000 7ff970e44640 1 --2- 172.21.15.39:0/3515515429 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff96c108cf0 0x7ff96c19bc50 secure :-1 s=READY pgs=1112 cs=0 l=1 rev1=1 crypto rx=0x7ff96400eaa0 tx=0x7ff96400ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:54:12.172 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:12.169+0000 7ff969ffb640 1 -- 172.21.15.39:0/3515515429 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff96400cdb0 con 0x7ff96c108cf0 2024-09-21T03:54:12.172 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:12.169+0000 7ff969ffb640 1 -- 172.21.15.39:0/3515515429 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff964005020 con 0x7ff96c108cf0 2024-09-21T03:54:12.172 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:12.169+0000 7ff9728a7640 1 -- 172.21.15.39:0/3515515429 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff96c1a67d0 con 0x7ff96c108cf0 2024-09-21T03:54:12.172 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:12.169+0000 7ff969ffb640 1 -- 172.21.15.39:0/3515515429 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9640116d0 con 0x7ff96c108cf0 2024-09-21T03:54:12.173 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:12.169+0000 7ff9728a7640 1 -- 172.21.15.39:0/3515515429 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff96c1a6d60 con 0x7ff96c108cf0 2024-09-21T03:54:12.173 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:12.171+0000 7ff978e1b740 1 Processor -- start 2024-09-21T03:54:12.173 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:12.172+0000 7ff969ffb640 1 -- 172.21.15.39:0/3515515429 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 27) ==== 154643+0+0 (secure 0 0 0) 0x7ff9640051d0 con 0x7ff96c108cf0 2024-09-21T03:54:12.173 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:12.172+0000 7ff978e1b740 1 -- start start 2024-09-21T03:54:12.175 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:12.172+0000 7ff978e1b740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c5496d7070 0x55c549a7e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:54:12.175 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:12.172+0000 7ff978e1b740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c5496ae1b0 0x55c549a7e800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:54:12.175 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:12.172+0000 7ff978e1b740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c5496b6390 0x55c549a7ed40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:54:12.175 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:12.172+0000 7ff978e1b740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x55c5499f4ea0 con 0x55c5496d7070 2024-09-21T03:54:12.175 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:12.172+0000 7ff978e1b740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x55c5499e42d0 con 0x55c5496ae1b0 2024-09-21T03:54:12.175 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:12.172+0000 7ff978e1b740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55c5499e80e0 con 0x55c5496b6390 2024-09-21T03:54:12.175 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:12.172+0000 7ff963fff640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c5496d7070 0x55c549a7e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:54:12.175 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:12.172+0000 7ff963fff640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c5496d7070 0x55c549a7e2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:46130/0 (socket says 172.21.15.39:46130) 2024-09-21T03:54:12.175 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:12.172+0000 7ff963fff640 1 -- 172.21.15.39:0/1297759207 learned_addr learned my addr 172.21.15.39:0/1297759207 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:54:12.176 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:12.172+0000 7ff96bfff640 1 --2- 172.21.15.39:0/1297759207 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c5496ae1b0 0x55c549a7e800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:54:12.176 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:12.173+0000 7ff963fff640 1 --2- 172.21.15.39:0/1297759207 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c5496b6390 0x55c549a7ed40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:54:12.176 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:12.173+0000 7ff969ffb640 1 --2- 172.21.15.39:0/3515515429 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7ff9580b6700 0x7ff9580b8c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:54:12.176 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:12.173+0000 7ff963fff640 1 -- 172.21.15.39:0/1297759207 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c5496b6390 msgr2=0x55c549a7ed40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:54:12.176 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:12.173+0000 7ff963fff640 1 --2- 172.21.15.39:0/1297759207 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c5496b6390 0x55c549a7ed40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:12.176 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:12.173+0000 7ff963fff640 1 -- 172.21.15.39:0/1297759207 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c5496ae1b0 msgr2=0x55c549a7e800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:54:12.176 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:12.173+0000 7ff963fff640 1 --2- 172.21.15.39:0/1297759207 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c5496ae1b0 0x55c549a7e800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:12.176 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:12.173+0000 7ff963fff640 1 -- 172.21.15.39:0/1297759207 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c549742650 con 0x55c5496d7070 2024-09-21T03:54:12.176 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:12.173+0000 7ff96bfff640 1 --2- 172.21.15.39:0/1297759207 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c5496ae1b0 0x55c549a7e800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:54:12.176 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:12.173+0000 7ff970e44640 1 --2- 172.21.15.39:0/3515515429 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7ff9580b6700 0x7ff9580b8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:54:12.176 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:12.173+0000 7ff963fff640 1 --2- 172.21.15.39:0/1297759207 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c5496d7070 0x55c549a7e2c0 secure :-1 s=READY pgs=1113 cs=0 l=1 rev1=1 crypto rx=0x7ff9540134b0 tx=0x7ff95400a720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc863610ac79227c server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:54:12.176 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:12.173+0000 7ff969ffb640 1 -- 172.21.15.39:0/3515515429 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8572+0+0 (secure 0 0 0) 0x7ff9640e77c0 con 0x7ff96c108cf0 2024-09-21T03:54:12.176 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:12.173+0000 7ff9627fc640 1 -- 172.21.15.39:0/1297759207 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff954019030 con 0x55c5496d7070 2024-09-21T03:54:12.176 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:12.173+0000 7ff9627fc640 1 -- 172.21.15.39:0/1297759207 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff9540110d0 con 0x55c5496d7070 2024-09-21T03:54:12.177 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:12.173+0000 7ff9627fc640 1 -- 172.21.15.39:0/1297759207 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff954004d90 con 0x55c5496d7070 2024-09-21T03:54:12.179 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:12.174+0000 7ff978e1b740 1 -- 172.21.15.39:0/1297759207 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c5496d7070 msgr2=0x55c549a7e2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:54:12.179 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:12.174+0000 7ff970e44640 1 --2- 172.21.15.39:0/3515515429 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7ff9580b6700 0x7ff9580b8c10 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7ff9640e54e0 tx=0x7ff964079240 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:54:12.180 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:12.174+0000 7ff978e1b740 1 --2- 172.21.15.39:0/1297759207 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c5496d7070 0x55c549a7e2c0 secure :-1 s=READY pgs=1113 cs=0 l=1 rev1=1 crypto rx=0x7ff9540134b0 tx=0x7ff95400a720 comp rx=0 tx=0).stop 2024-09-21T03:54:12.180 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:12.174+0000 7ff978e1b740 1 -- 172.21.15.39:0/1297759207 shutdown_connections 2024-09-21T03:54:12.180 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:12.174+0000 7ff978e1b740 1 --2- 172.21.15.39:0/1297759207 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c5496b6390 0x55c549a7ed40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:12.180 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:12.174+0000 7ff978e1b740 1 --2- 172.21.15.39:0/1297759207 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c5496ae1b0 0x55c549a7e800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:12.180 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:12.174+0000 7ff978e1b740 1 --2- 172.21.15.39:0/1297759207 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c5496d7070 0x55c549a7e2c0 unknown :-1 s=CLOSED pgs=1113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:12.180 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:12.174+0000 7ff978e1b740 1 -- 172.21.15.39:0/1297759207 >> 172.21.15.39:0/1297759207 conn(0x55c5496cb460 msgr2=0x55c549a7c1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:54:12.180 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:12.174+0000 7ff978e1b740 1 -- 172.21.15.39:0/1297759207 shutdown_connections 2024-09-21T03:54:12.180 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:12.174+0000 7ff978e1b740 1 -- 172.21.15.39:0/1297759207 wait complete. 2024-09-21T03:54:12.180 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:12.175+0000 7ff978e1b740 1 -- start start 2024-09-21T03:54:12.180 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:12.175+0000 7ff978e1b740 1 Processor -- start 2024-09-21T03:54:12.180 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:12.175+0000 7ff978e1b740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c5496ae1b0 0x55c549a899c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:54:12.180 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:12.175+0000 7ff978e1b740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c5496b6390 0x55c549a89f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:54:12.180 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:12.175+0000 7ff978e1b740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c5496d7070 0x55c549a95d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:54:12.181 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:12.175+0000 7ff978e1b740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x55c549845700 con 0x55c5496d7070 2024-09-21T03:54:12.181 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:12.175+0000 7ff978e1b740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x55c54997f160 con 0x55c5496b6390 2024-09-21T03:54:12.181 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:12.175+0000 7ff978e1b740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55c549a8caa0 con 0x55c5496ae1b0 2024-09-21T03:54:12.181 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:12.175+0000 7ff978e1b740 10 client.0 _ll_register_callbacks cb 0x55c5499989e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:54:12.181 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:12.175+0000 7ff963fff640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c5496d7070 0x55c549a95d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:54:12.181 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:12.175+0000 7ff963fff640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c5496d7070 0x55c549a95d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:46132/0 (socket says 172.21.15.39:46132) 2024-09-21T03:54:12.181 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:12.175+0000 7ff963fff640 1 -- 172.21.15.39:0/2487497556 learned_addr learned my addr 172.21.15.39:0/2487497556 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:54:12.181 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:12.176+0000 7ff96bfff640 1 --2- 172.21.15.39:0/2487497556 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c5496b6390 0x55c549a89f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:54:12.181 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:12.176+0000 7ff963fff640 1 --2- 172.21.15.39:0/2487497556 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c5496ae1b0 0x55c549a899c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:54:12.181 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:12.176+0000 7ff963fff640 1 -- 172.21.15.39:0/2487497556 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c5496ae1b0 msgr2=0x55c549a899c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:54:12.181 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:12.176+0000 7ff963fff640 1 --2- 172.21.15.39:0/2487497556 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c5496ae1b0 0x55c549a899c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:12.181 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:12.176+0000 7ff963fff640 1 -- 172.21.15.39:0/2487497556 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c5496b6390 msgr2=0x55c549a89f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:54:12.181 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:12.176+0000 7ff963fff640 1 --2- 172.21.15.39:0/2487497556 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c5496b6390 0x55c549a89f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:12.181 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:12.176+0000 7ff963fff640 1 -- 172.21.15.39:0/2487497556 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c5499acc90 con 0x55c5496d7070 2024-09-21T03:54:12.182 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:12.176+0000 7ff96bfff640 1 --2- 172.21.15.39:0/2487497556 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c5496b6390 0x55c549a89f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:54:12.182 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:12.176+0000 7ff963fff640 1 --2- 172.21.15.39:0/2487497556 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c5496d7070 0x55c549a95d90 secure :-1 s=READY pgs=1114 cs=0 l=1 rev1=1 crypto rx=0x7ff954004a70 tx=0x7ff954004970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=73d4e8fdeec7127b server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:54:12.182 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:12.176+0000 7ff94b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-09-21T03:54:12.182 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:12.176+0000 7ff94b7fe640 1 -- 172.21.15.39:0/2487497556 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff954019040 con 0x55c5496d7070 2024-09-21T03:54:12.182 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:12.176+0000 7ff94b7fe640 1 -- 172.21.15.39:0/2487497556 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8572+0+0 (secure 0 0 0) 0x7ff954005000 con 0x55c5496d7070 2024-09-21T03:54:12.182 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:12.176+0000 7ff978e1b740 1 -- 172.21.15.39:0/2487497556 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff9540110d0 con 0x55c5496d7070 2024-09-21T03:54:12.182 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:12.176+0000 7ff94b7fe640 1 -- 172.21.15.39:0/2487497556 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7ff954005200 con 0x55c5496d7070 2024-09-21T03:54:12.182 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:12.176+0000 7ff978e1b740 10 client.17631 fetch_fsmap learned FSMap version 29 2024-09-21T03:54:12.182 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:12.176+0000 7ff978e1b740 1 -- 172.21.15.39:0/2487497556 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55c549a987c0 con 0x55c5496d7070 2024-09-21T03:54:12.182 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:12.177+0000 7ff94b7fe640 1 -- 172.21.15.39:0/2487497556 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff9540110d0 con 0x55c5496d7070 2024-09-21T03:54:12.182 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:12.177+0000 7ff94b7fe640 1 -- 172.21.15.39:0/2487497556 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7ff95401e460 con 0x55c5496d7070 2024-09-21T03:54:12.182 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:12.177+0000 7ff978e1b740 10 client.17631 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T03:54:12.182 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:12.177+0000 7ff978e1b740 10 client.17631 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-21T03:54:12.182 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:12.177+0000 7ff978e1b740 20 client.17631 populate_metadata read hostname 'smithi039' 2024-09-21T03:54:12.183 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:12.177+0000 7ff978e1b740 1 --2- 172.21.15.39:0/2487497556 >> [v2:172.21.15.150:6836/4017043144,v1:172.21.15.150:6837/4017043144] conn(0x55c5496959f0 0x55c549a9bbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:54:12.183 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:12.177+0000 7ff978e1b740 4 client.17631 mds_command: new command op to 24659 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:54:12.183 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:12.177+0000 7ff978e1b740 1 -- 172.21.15.39:0/2487497556 --> [v2:172.21.15.150:6836/4017043144,v1:172.21.15.150:6837/4017043144] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c5499f6df0 con 0x55c5496959f0 2024-09-21T03:54:12.183 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:12.177+0000 7ff96bfff640 1 --2- 172.21.15.39:0/2487497556 >> [v2:172.21.15.150:6836/4017043144,v1:172.21.15.150:6837/4017043144] conn(0x55c5496959f0 0x55c549a9bbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:54:12.183 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:12.178+0000 7ff96bfff640 1 --2- 172.21.15.39:0/2487497556 >> [v2:172.21.15.150:6836/4017043144,v1:172.21.15.150:6837/4017043144] conn(0x55c5496959f0 0x55c549a9bbd0 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=2c1b9d8b619a9d5e server_cookie=3e7985c1be93b376 in_seq=0 out_seq=0 2024-09-21T03:54:12.183 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:12.178+0000 7ff94b7fe640 10 client.17631 ms_handle_connect on v2:172.21.15.150:6836/4017043144 2024-09-21T03:54:12.184 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:12.178+0000 7ff94b7fe640 1 -- 172.21.15.39:0/2487497556 <== mds.2 v2:172.21.15.150:6836/4017043144 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55c5499f6df0 con 0x55c5496959f0 2024-09-21T03:54:12.184 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:12.179+0000 7ff94b7fe640 10 client.17631 handle_command_reply: tid=0 2024-09-21T03:54:12.184 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:12.179+0000 7ff978e1b740 1 client.17631 shutdown 2024-09-21T03:54:12.184 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:12.179+0000 7ff978e1b740 1 -- 172.21.15.39:0/2487497556 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c5496d7070 msgr2=0x55c549a95d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:54:12.184 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:12.179+0000 7ff978e1b740 1 --2- 172.21.15.39:0/2487497556 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c5496d7070 0x55c549a95d90 secure :-1 s=READY pgs=1114 cs=0 l=1 rev1=1 crypto rx=0x7ff954004a70 tx=0x7ff954004970 comp rx=0 tx=0).stop 2024-09-21T03:54:12.184 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:12.179+0000 7ff978e1b740 1 -- 172.21.15.39:0/2487497556 shutdown_connections 2024-09-21T03:54:12.184 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:12.179+0000 7ff978e1b740 1 --2- 172.21.15.39:0/2487497556 >> [v2:172.21.15.150:6836/4017043144,v1:172.21.15.150:6837/4017043144] conn(0x55c5496959f0 0x55c549a9bbd0 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:12.185 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:12.179+0000 7ff978e1b740 1 --2- 172.21.15.39:0/2487497556 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c5496d7070 0x55c549a95d90 unknown :-1 s=CLOSED pgs=1114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:12.185 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:12.179+0000 7ff978e1b740 1 --2- 172.21.15.39:0/2487497556 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c5496b6390 0x55c549a89f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:12.185 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:12.179+0000 7ff978e1b740 1 --2- 172.21.15.39:0/2487497556 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c5496ae1b0 0x55c549a899c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:12.185 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:12.179+0000 7ff978e1b740 1 -- 172.21.15.39:0/2487497556 >> 172.21.15.39:0/2487497556 conn(0x55c5496cb460 msgr2=0x55c549a814a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:54:12.185 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:12.179+0000 7ff94b7fe640 0 client.17631 ms_handle_reset on v2:172.21.15.150:6836/4017043144 2024-09-21T03:54:12.185 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:12.179+0000 7ff978e1b740 1 -- 172.21.15.39:0/2487497556 shutdown_connections 2024-09-21T03:54:12.185 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:12.179+0000 7ff978e1b740 1 -- 172.21.15.39:0/2487497556 wait complete. 2024-09-21T03:54:12.185 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:12.179+0000 7ff978e1b740 20 client.17631 trim_cache size 0 max 16384 2024-09-21T03:54:12.195 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:12.192+0000 7ff978e1b740 1 Processor -- start 2024-09-21T03:54:12.195 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:12.192+0000 7ff978e1b740 1 -- start start 2024-09-21T03:54:12.195 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:12.193+0000 7ff978e1b740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c5496ae1b0 0x55c549a7fa00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:54:12.195 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:12.193+0000 7ff978e1b740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c5496b6390 0x55c549a95d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:54:12.195 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:12.193+0000 7ff978e1b740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c5496d7070 0x55c549a86bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:54:12.195 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:12.193+0000 7ff978e1b740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7ff96c1af7d0 con 0x55c5496b6390 2024-09-21T03:54:12.196 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:12.193+0000 7ff978e1b740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7ff96c1b1310 con 0x55c5496d7070 2024-09-21T03:54:12.196 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:12.193+0000 7ff978e1b740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7ff96c1afea0 con 0x55c5496ae1b0 2024-09-21T03:54:12.196 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:12.193+0000 7ff96bfff640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c5496b6390 0x55c549a95d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:54:12.196 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:12.193+0000 7ff96bfff640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c5496b6390 0x55c549a95d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:46134/0 (socket says 172.21.15.39:46134) 2024-09-21T03:54:12.196 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:12.193+0000 7ff96bfff640 1 -- 172.21.15.39:0/255986699 learned_addr learned my addr 172.21.15.39:0/255986699 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:54:12.196 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:12.193+0000 7ff963fff640 1 --2- 172.21.15.39:0/255986699 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c5496d7070 0x55c549a86bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:54:12.196 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:12.193+0000 7ff963fff640 1 --2- 172.21.15.39:0/255986699 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c5496ae1b0 0x55c549a7fa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:54:12.196 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:12.193+0000 7ff96bfff640 1 -- 172.21.15.39:0/255986699 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c5496ae1b0 msgr2=0x55c549a7fa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:54:12.196 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:12.193+0000 7ff96bfff640 1 --2- 172.21.15.39:0/255986699 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c5496ae1b0 0x55c549a7fa00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:12.196 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:12.193+0000 7ff96bfff640 1 -- 172.21.15.39:0/255986699 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c5496d7070 msgr2=0x55c549a86bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:54:12.196 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:12.193+0000 7ff96bfff640 1 --2- 172.21.15.39:0/255986699 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c5496d7070 0x55c549a86bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:12.196 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:12.193+0000 7ff96bfff640 1 -- 172.21.15.39:0/255986699 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c549a76460 con 0x55c5496b6390 2024-09-21T03:54:12.196 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:12.193+0000 7ff963fff640 1 --2- 172.21.15.39:0/255986699 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c5496d7070 0x55c549a86bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:54:12.196 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:12.193+0000 7ff96bfff640 1 --2- 172.21.15.39:0/255986699 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c5496b6390 0x55c549a95d00 secure :-1 s=READY pgs=1115 cs=0 l=1 rev1=1 crypto rx=0x7ff95c005940 tx=0x7ff95c031400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb903c9386b4d150 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:54:12.197 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:12.193+0000 7ff961ffb640 1 -- 172.21.15.39:0/255986699 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff95c035040 con 0x55c5496b6390 2024-09-21T03:54:12.197 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:12.193+0000 7ff961ffb640 1 -- 172.21.15.39:0/255986699 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff95c0054f0 con 0x55c5496b6390 2024-09-21T03:54:12.197 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:12.193+0000 7ff961ffb640 1 -- 172.21.15.39:0/255986699 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff95c0035f0 con 0x55c5496b6390 2024-09-21T03:54:12.200 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:12.194+0000 7ff978e1b740 1 -- 172.21.15.39:0/255986699 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c5496b6390 msgr2=0x55c549a95d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:54:12.200 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:12.194+0000 7ff978e1b740 1 --2- 172.21.15.39:0/255986699 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c5496b6390 0x55c549a95d00 secure :-1 s=READY pgs=1115 cs=0 l=1 rev1=1 crypto rx=0x7ff95c005940 tx=0x7ff95c031400 comp rx=0 tx=0).stop 2024-09-21T03:54:12.200 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:12.194+0000 7ff978e1b740 1 -- 172.21.15.39:0/255986699 shutdown_connections 2024-09-21T03:54:12.200 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:12.194+0000 7ff978e1b740 1 --2- 172.21.15.39:0/255986699 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c5496d7070 0x55c549a86bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:12.200 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:12.194+0000 7ff978e1b740 1 --2- 172.21.15.39:0/255986699 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c5496b6390 0x55c549a95d00 unknown :-1 s=CLOSED pgs=1115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:12.200 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:12.194+0000 7ff978e1b740 1 --2- 172.21.15.39:0/255986699 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c5496ae1b0 0x55c549a7fa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:12.201 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:12.194+0000 7ff978e1b740 1 -- 172.21.15.39:0/255986699 >> 172.21.15.39:0/255986699 conn(0x55c5496cb460 msgr2=0x55c549a7f2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:54:12.201 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:12.194+0000 7ff978e1b740 1 -- 172.21.15.39:0/255986699 shutdown_connections 2024-09-21T03:54:12.201 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:12.194+0000 7ff978e1b740 1 -- 172.21.15.39:0/255986699 wait complete. 2024-09-21T03:54:12.201 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:12.195+0000 7ff978e1b740 1 -- start start 2024-09-21T03:54:12.201 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:12.195+0000 7ff978e1b740 1 Processor -- start 2024-09-21T03:54:12.201 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:12.195+0000 7ff978e1b740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c5496ae1b0 0x55c549a9c550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:54:12.201 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:12.195+0000 7ff978e1b740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c5496b6390 0x55c549a9caf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:54:12.201 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:12.195+0000 7ff978e1b740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c5496d7070 0x55c549afe380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:54:12.201 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:12.195+0000 7ff978e1b740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7ff96c1b0c40 con 0x55c5496d7070 2024-09-21T03:54:12.201 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:12.195+0000 7ff978e1b740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7ff96c1b0570 con 0x55c5496ae1b0 2024-09-21T03:54:12.201 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:12.195+0000 7ff978e1b740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7ff96c1ad7d0 con 0x55c5496b6390 2024-09-21T03:54:12.201 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:12.195+0000 7ff978e1b740 10 client.0 _ll_register_callbacks cb 0x7ff95c009640 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:54:12.201 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:12.195+0000 7ff963fff640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c5496d7070 0x55c549afe380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:54:12.201 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:12.195+0000 7ff963fff640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c5496d7070 0x55c549afe380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:46146/0 (socket says 172.21.15.39:46146) 2024-09-21T03:54:12.202 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:12.195+0000 7ff963fff640 1 -- 172.21.15.39:0/1282184068 learned_addr learned my addr 172.21.15.39:0/1282184068 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:54:12.202 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:12.196+0000 7ff963fff640 1 --2- 172.21.15.39:0/1282184068 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c5496ae1b0 0x55c549a9c550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:54:12.202 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:12.196+0000 7ff96bfff640 1 --2- 172.21.15.39:0/1282184068 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c5496b6390 0x55c549a9caf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:54:12.202 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:12.196+0000 7ff963fff640 1 -- 172.21.15.39:0/1282184068 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c5496b6390 msgr2=0x55c549a9caf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:54:12.202 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:12.196+0000 7ff963fff640 1 --2- 172.21.15.39:0/1282184068 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c5496b6390 0x55c549a9caf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:12.202 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:12.196+0000 7ff963fff640 1 -- 172.21.15.39:0/1282184068 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c5496ae1b0 msgr2=0x55c549a9c550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:54:12.202 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:12.196+0000 7ff963fff640 1 --2- 172.21.15.39:0/1282184068 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c5496ae1b0 0x55c549a9c550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:12.202 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:12.196+0000 7ff963fff640 1 -- 172.21.15.39:0/1282184068 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c549afe9f0 con 0x55c5496d7070 2024-09-21T03:54:12.202 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:12.196+0000 7ff963fff640 1 --2- 172.21.15.39:0/1282184068 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c5496d7070 0x55c549afe380 secure :-1 s=READY pgs=1116 cs=0 l=1 rev1=1 crypto rx=0x7ff954004830 tx=0x7ff9540048d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7bb998293c84ffa7 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:54:12.202 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:12.196+0000 7ff94b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-09-21T03:54:12.202 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:12.196+0000 7ff94b7fe640 1 -- 172.21.15.39:0/1282184068 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff954019040 con 0x55c5496d7070 2024-09-21T03:54:12.202 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:12.196+0000 7ff94b7fe640 1 -- 172.21.15.39:0/1282184068 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8572+0+0 (secure 0 0 0) 0x7ff954004e20 con 0x55c5496d7070 2024-09-21T03:54:12.202 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:12.196+0000 7ff978e1b740 1 -- 172.21.15.39:0/1282184068 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff95c0054f0 con 0x55c5496d7070 2024-09-21T03:54:12.202 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:12.196+0000 7ff94b7fe640 1 -- 172.21.15.39:0/1282184068 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7ff954005020 con 0x55c5496d7070 2024-09-21T03:54:12.203 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:12.196+0000 7ff978e1b740 10 client.17637 fetch_fsmap learned FSMap version 29 2024-09-21T03:54:12.203 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:12.196+0000 7ff978e1b740 1 -- 172.21.15.39:0/1282184068 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55c549b05c80 con 0x55c5496d7070 2024-09-21T03:54:12.203 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:12.197+0000 7ff94b7fe640 1 -- 172.21.15.39:0/1282184068 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff95c0054f0 con 0x55c5496d7070 2024-09-21T03:54:12.203 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:12.197+0000 7ff94b7fe640 1 -- 172.21.15.39:0/1282184068 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7ff954011b10 con 0x55c5496d7070 2024-09-21T03:54:12.203 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:12.197+0000 7ff978e1b740 10 client.17637 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T03:54:12.203 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:12.197+0000 7ff978e1b740 10 client.17637 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-21T03:54:12.203 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:12.197+0000 7ff978e1b740 20 client.17637 populate_metadata read hostname 'smithi039' 2024-09-21T03:54:12.203 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:12.197+0000 7ff978e1b740 1 --2- 172.21.15.39:0/1282184068 >> [v2:172.21.15.150:6836/4017043144,v1:172.21.15.150:6837/4017043144] conn(0x55c54966ada0 0x55c549b082f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:54:12.203 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:12.197+0000 7ff978e1b740 4 client.17637 mds_command: new command op to 24659 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T03:54:12.203 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:12.197+0000 7ff978e1b740 1 -- 172.21.15.39:0/1282184068 --> [v2:172.21.15.150:6836/4017043144,v1:172.21.15.150:6837/4017043144] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55c5499f28a0 con 0x55c54966ada0 2024-09-21T03:54:12.203 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:12.197+0000 7ff96bfff640 1 --2- 172.21.15.39:0/1282184068 >> [v2:172.21.15.150:6836/4017043144,v1:172.21.15.150:6837/4017043144] conn(0x55c54966ada0 0x55c549b082f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:54:12.203 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:12.198+0000 7ff96bfff640 1 --2- 172.21.15.39:0/1282184068 >> [v2:172.21.15.150:6836/4017043144,v1:172.21.15.150:6837/4017043144] conn(0x55c54966ada0 0x55c549b082f0 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=baf4a781bc72be8a server_cookie=887b30b37d8a45ed in_seq=0 out_seq=0 2024-09-21T03:54:12.203 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:12.198+0000 7ff94b7fe640 10 client.17637 ms_handle_connect on v2:172.21.15.150:6836/4017043144 2024-09-21T03:54:12.203 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:12.198+0000 7ff94b7fe640 1 -- 172.21.15.39:0/1282184068 <== mds.2 v2:172.21.15.150:6836/4017043144 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55c5499f28a0 con 0x55c54966ada0 2024-09-21T03:54:12.203 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:12.198+0000 7ff94b7fe640 10 client.17637 handle_command_reply: tid=0 2024-09-21T03:54:12.204 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:12.198+0000 7ff978e1b740 1 client.17637 shutdown 2024-09-21T03:54:12.204 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:12.198+0000 7ff978e1b740 1 -- 172.21.15.39:0/1282184068 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c5496d7070 msgr2=0x55c549afe380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:54:12.204 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:12.198+0000 7ff978e1b740 1 --2- 172.21.15.39:0/1282184068 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c5496d7070 0x55c549afe380 secure :-1 s=READY pgs=1116 cs=0 l=1 rev1=1 crypto rx=0x7ff954004830 tx=0x7ff9540048d0 comp rx=0 tx=0).stop 2024-09-21T03:54:12.204 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:12.198+0000 7ff978e1b740 1 -- 172.21.15.39:0/1282184068 shutdown_connections 2024-09-21T03:54:12.204 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:12.198+0000 7ff978e1b740 1 --2- 172.21.15.39:0/1282184068 >> [v2:172.21.15.150:6836/4017043144,v1:172.21.15.150:6837/4017043144] conn(0x55c54966ada0 0x55c549b082f0 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:12.204 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:12.198+0000 7ff978e1b740 1 --2- 172.21.15.39:0/1282184068 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c5496d7070 0x55c549afe380 unknown :-1 s=CLOSED pgs=1116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:12.204 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:12.198+0000 7ff978e1b740 1 --2- 172.21.15.39:0/1282184068 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c5496b6390 0x55c549a9caf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:12.204 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:12.198+0000 7ff978e1b740 1 --2- 172.21.15.39:0/1282184068 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c5496ae1b0 0x55c549a9c550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:12.204 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:12.198+0000 7ff978e1b740 1 -- 172.21.15.39:0/1282184068 >> 172.21.15.39:0/1282184068 conn(0x55c5496cb460 msgr2=0x55c549a8ae70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:54:12.204 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:12.198+0000 7ff94b7fe640 0 client.17637 ms_handle_reset on v2:172.21.15.150:6836/4017043144 2024-09-21T03:54:12.204 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:12.198+0000 7ff978e1b740 1 -- 172.21.15.39:0/1282184068 shutdown_connections 2024-09-21T03:54:12.204 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:12.198+0000 7ff978e1b740 1 -- 172.21.15.39:0/1282184068 wait complete. 2024-09-21T03:54:12.204 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:12.198+0000 7ff978e1b740 20 client.17637 trim_cache size 0 max 16384 2024-09-21T03:54:12.205 INFO:teuthology.orchestra.run.smithi039.stdout:[] 2024-09-21T03:54:12.205 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:12.199+0000 7ff9728a7640 1 -- 172.21.15.39:0/3515515429 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7ff9580b6700 msgr2=0x7ff9580b8c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:54:12.205 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:12.199+0000 7ff9728a7640 1 --2- 172.21.15.39:0/3515515429 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7ff9580b6700 0x7ff9580b8c10 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7ff9640e54e0 tx=0x7ff964079240 comp rx=0 tx=0).stop 2024-09-21T03:54:12.205 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:12.199+0000 7ff9728a7640 1 -- 172.21.15.39:0/3515515429 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff96c108cf0 msgr2=0x7ff96c19bc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:54:12.206 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:12.199+0000 7ff9728a7640 1 --2- 172.21.15.39:0/3515515429 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff96c108cf0 0x7ff96c19bc50 secure :-1 s=READY pgs=1112 cs=0 l=1 rev1=1 crypto rx=0x7ff96400eaa0 tx=0x7ff96400ef60 comp rx=0 tx=0).stop 2024-09-21T03:54:12.206 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:12.199+0000 7ff9728a7640 1 -- 172.21.15.39:0/3515515429 shutdown_connections 2024-09-21T03:54:12.206 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:12.199+0000 7ff9728a7640 1 --2- 172.21.15.39:0/3515515429 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7ff9580b6700 0x7ff9580b8c10 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:12.206 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:12.199+0000 7ff9728a7640 1 --2- 172.21.15.39:0/3515515429 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff96c108cf0 0x7ff96c19bc50 unknown :-1 s=CLOSED pgs=1112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:12.206 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:12.199+0000 7ff9728a7640 1 --2- 172.21.15.39:0/3515515429 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff96c1083a0 0x7ff96c19b710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:12.206 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:12.199+0000 7ff9728a7640 1 --2- 172.21.15.39:0/3515515429 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff96c102a60 0x7ff96c19b1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:12.206 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:12.199+0000 7ff9728a7640 1 -- 172.21.15.39:0/3515515429 >> 172.21.15.39:0/3515515429 conn(0x7ff96c079730 msgr2=0x7ff96c101410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:54:12.206 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:12.199+0000 7ff9728a7640 1 -- 172.21.15.39:0/3515515429 shutdown_connections 2024-09-21T03:54:12.206 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:12.199+0000 7ff9728a7640 1 -- 172.21.15.39:0/3515515429 wait complete. 2024-09-21T03:54:13.030 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 51116d5c-77ca-11ef-baf5-efdc52797490 -- ceph tell mds.1:1 damage ls 2024-09-21T03:54:13.286 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/51116d5c-77ca-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:54:13.334 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:54:12 smithi039 ceph-mon[31211]: pgmap v573: 97 pgs: 97 active+clean; 10 MiB data, 525 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 53 KiB/s wr, 30 op/s 2024-09-21T03:54:13.468 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:54:12 smithi161 ceph-mon[34533]: pgmap v573: 97 pgs: 97 active+clean; 10 MiB data, 525 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 53 KiB/s wr, 30 op/s 2024-09-21T03:54:13.475 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:54:12 smithi150 ceph-mon[34289]: pgmap v573: 97 pgs: 97 active+clean; 10 MiB data, 525 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 53 KiB/s wr, 30 op/s 2024-09-21T03:54:13.785 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:13.783+0000 7fe048287640 1 -- 172.21.15.39:0/2614513182 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe040106bf0 msgr2=0x7fe040110660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:54:13.785 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:13.783+0000 7fe048287640 1 --2- 172.21.15.39:0/2614513182 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe040106bf0 0x7fe040110660 secure :-1 s=READY pgs=1117 cs=0 l=1 rev1=1 crypto rx=0x7fe038009ca0 tx=0x7fe03802a8c0 comp rx=0 tx=0).stop 2024-09-21T03:54:13.785 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:13.783+0000 7fe048287640 1 -- 172.21.15.39:0/2614513182 shutdown_connections 2024-09-21T03:54:13.785 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:13.783+0000 7fe048287640 1 --2- 172.21.15.39:0/2614513182 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe040106bf0 0x7fe040110660 unknown :-1 s=CLOSED pgs=1117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:13.785 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:13.783+0000 7fe048287640 1 --2- 172.21.15.39:0/2614513182 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe0401062a0 0x7fe0401066b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:13.786 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:13.783+0000 7fe048287640 1 --2- 172.21.15.39:0/2614513182 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe0401020b0 0x7fe0401024c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:13.786 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:13.783+0000 7fe048287640 1 -- 172.21.15.39:0/2614513182 >> 172.21.15.39:0/2614513182 conn(0x7fe040079e30 msgr2=0x7fe04007a240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:54:13.786 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:13.783+0000 7fe048287640 1 -- 172.21.15.39:0/2614513182 shutdown_connections 2024-09-21T03:54:13.786 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:13.783+0000 7fe048287640 1 -- 172.21.15.39:0/2614513182 wait complete. 2024-09-21T03:54:13.786 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:13.784+0000 7fe048287640 1 Processor -- start 2024-09-21T03:54:13.786 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:13.784+0000 7fe048287640 1 -- start start 2024-09-21T03:54:13.787 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:13.784+0000 7fe048287640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe0401020b0 0x7fe04019a4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:54:13.787 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:13.784+0000 7fe048287640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe0401062a0 0x7fe04019aa40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:54:13.788 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:13.784+0000 7fe048287640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe040106bf0 0x7fe04019fce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:54:13.788 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:13.784+0000 7fe048287640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fe040113af0 con 0x7fe0401062a0 2024-09-21T03:54:13.788 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:13.784+0000 7fe048287640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fe040113970 con 0x7fe0401020b0 2024-09-21T03:54:13.788 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:13.784+0000 7fe048287640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fe040113c70 con 0x7fe040106bf0 2024-09-21T03:54:13.788 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:13.784+0000 7fe045822640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe0401062a0 0x7fe04019aa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:54:13.788 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:13.784+0000 7fe045822640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe0401062a0 0x7fe04019aa40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:46164/0 (socket says 172.21.15.39:46164) 2024-09-21T03:54:13.788 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:13.784+0000 7fe045822640 1 -- 172.21.15.39:0/201371835 learned_addr learned my addr 172.21.15.39:0/201371835 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:54:13.788 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:13.784+0000 7fe046023640 1 --2- 172.21.15.39:0/201371835 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe0401020b0 0x7fe04019a4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:54:13.788 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:13.784+0000 7fe046824640 1 --2- 172.21.15.39:0/201371835 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe040106bf0 0x7fe04019fce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:54:13.788 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:13.785+0000 7fe045822640 1 -- 172.21.15.39:0/201371835 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe040106bf0 msgr2=0x7fe04019fce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:54:13.788 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:13.785+0000 7fe045822640 1 --2- 172.21.15.39:0/201371835 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe040106bf0 0x7fe04019fce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:13.788 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:13.785+0000 7fe045822640 1 -- 172.21.15.39:0/201371835 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe0401020b0 msgr2=0x7fe04019a4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:54:13.788 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:13.785+0000 7fe045822640 1 --2- 172.21.15.39:0/201371835 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe0401020b0 0x7fe04019a4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:13.788 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:13.785+0000 7fe045822640 1 -- 172.21.15.39:0/201371835 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe0401a0410 con 0x7fe0401062a0 2024-09-21T03:54:13.788 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:13.785+0000 7fe046824640 1 --2- 172.21.15.39:0/201371835 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe040106bf0 0x7fe04019fce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:54:13.789 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:13.785+0000 7fe045822640 1 --2- 172.21.15.39:0/201371835 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe0401062a0 0x7fe04019aa40 secure :-1 s=READY pgs=1118 cs=0 l=1 rev1=1 crypto rx=0x7fe03000c970 tx=0x7fe03000ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:54:13.789 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:13.785+0000 7fe0377fe640 1 -- 172.21.15.39:0/201371835 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe030007bb0 con 0x7fe0401062a0 2024-09-21T03:54:13.789 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:13.785+0000 7fe0377fe640 1 -- 172.21.15.39:0/201371835 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe030007d50 con 0x7fe0401062a0 2024-09-21T03:54:13.789 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:13.785+0000 7fe0377fe640 1 -- 172.21.15.39:0/201371835 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe03000f450 con 0x7fe0401062a0 2024-09-21T03:54:13.789 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:13.785+0000 7fe048287640 1 -- 172.21.15.39:0/201371835 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe0401a0700 con 0x7fe0401062a0 2024-09-21T03:54:13.789 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:13.785+0000 7fe048287640 1 -- 172.21.15.39:0/201371835 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe0401a6e60 con 0x7fe0401062a0 2024-09-21T03:54:13.789 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:13.788+0000 7fe04e7fb740 1 Processor -- start 2024-09-21T03:54:13.789 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:13.788+0000 7fe04e7fb740 1 -- start start 2024-09-21T03:54:13.791 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:13.788+0000 7fe04e7fb740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559bf9a43070 0x559bf9dea6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:54:13.791 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:13.788+0000 7fe04e7fb740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559bf9a1a1b0 0x559bf9deabe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:54:13.791 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:13.788+0000 7fe04e7fb740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x559bf9a22390 0x559bf9deb120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:54:13.791 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:13.788+0000 7fe04e7fb740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x559bf9d60ea0 con 0x559bf9a1a1b0 2024-09-21T03:54:13.791 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:13.788+0000 7fe04e7fb740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x559bf9d502d0 con 0x559bf9a22390 2024-09-21T03:54:13.792 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:13.788+0000 7fe04e7fb740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x559bf9d540e0 con 0x559bf9a43070 2024-09-21T03:54:13.792 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:13.788+0000 7fe0377fe640 1 -- 172.21.15.39:0/201371835 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 27) ==== 154643+0+0 (secure 0 0 0) 0x7fe03000f5f0 con 0x7fe0401062a0 2024-09-21T03:54:13.792 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:13.788+0000 7fe046023640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559bf9a1a1b0 0x559bf9deabe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:54:13.792 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:13.788+0000 7fe046023640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559bf9a1a1b0 0x559bf9deabe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:46178/0 (socket says 172.21.15.39:46178) 2024-09-21T03:54:13.792 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:13.788+0000 7fe046023640 1 -- 172.21.15.39:0/2871304304 learned_addr learned my addr 172.21.15.39:0/2871304304 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:54:13.792 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:13.788+0000 7fe046824640 1 --2- 172.21.15.39:0/2871304304 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559bf9a43070 0x559bf9dea6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:54:13.792 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:13.789+0000 7fe045822640 1 --2- 172.21.15.39:0/2871304304 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x559bf9a22390 0x559bf9deb120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:54:13.792 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:13.789+0000 7fe046023640 1 -- 172.21.15.39:0/2871304304 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559bf9a43070 msgr2=0x559bf9dea6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:54:13.792 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:13.789+0000 7fe046023640 1 --2- 172.21.15.39:0/2871304304 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559bf9a43070 0x559bf9dea6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:13.792 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:13.789+0000 7fe046023640 1 -- 172.21.15.39:0/2871304304 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x559bf9a22390 msgr2=0x559bf9deb120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:54:13.792 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:13.789+0000 7fe046023640 1 --2- 172.21.15.39:0/2871304304 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x559bf9a22390 0x559bf9deb120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:13.792 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:13.789+0000 7fe046023640 1 -- 172.21.15.39:0/2871304304 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559bf9aae650 con 0x559bf9a1a1b0 2024-09-21T03:54:13.792 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:13.789+0000 7fe045822640 1 --2- 172.21.15.39:0/2871304304 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x559bf9a22390 0x559bf9deb120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:54:13.792 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:13.789+0000 7fe046023640 1 --2- 172.21.15.39:0/2871304304 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559bf9a1a1b0 0x559bf9deabe0 secure :-1 s=READY pgs=1119 cs=0 l=1 rev1=1 crypto rx=0x7fe03c00ac10 tx=0x7fe03c00f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b0966e4185ea25c2 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:54:13.792 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:13.789+0000 7fe034d9a640 1 -- 172.21.15.39:0/2871304304 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe03c01d070 con 0x559bf9a1a1b0 2024-09-21T03:54:13.793 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:13.789+0000 7fe034d9a640 1 -- 172.21.15.39:0/2871304304 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe03c018af0 con 0x559bf9a1a1b0 2024-09-21T03:54:13.793 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:13.789+0000 7fe0377fe640 1 --2- 172.21.15.39:0/201371835 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7fe02c0b6680 0x7fe02c0b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:54:13.793 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:13.789+0000 7fe034d9a640 1 -- 172.21.15.39:0/2871304304 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe03c010ac0 con 0x559bf9a1a1b0 2024-09-21T03:54:13.793 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:13.789+0000 7fe046824640 1 --2- 172.21.15.39:0/201371835 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7fe02c0b6680 0x7fe02c0b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:54:13.793 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:13.789+0000 7fe0377fe640 1 -- 172.21.15.39:0/201371835 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8572+0+0 (secure 0 0 0) 0x7fe0300e7490 con 0x7fe0401062a0 2024-09-21T03:54:13.795 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:13.790+0000 7fe04e7fb740 1 -- 172.21.15.39:0/2871304304 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559bf9a1a1b0 msgr2=0x559bf9deabe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:54:13.795 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:13.790+0000 7fe046824640 1 --2- 172.21.15.39:0/201371835 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7fe02c0b6680 0x7fe02c0b8b90 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7fe038005cc0 tx=0x7fe038005c30 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:54:13.796 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:13.790+0000 7fe04e7fb740 1 --2- 172.21.15.39:0/2871304304 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559bf9a1a1b0 0x559bf9deabe0 secure :-1 s=READY pgs=1119 cs=0 l=1 rev1=1 crypto rx=0x7fe03c00ac10 tx=0x7fe03c00f790 comp rx=0 tx=0).stop 2024-09-21T03:54:13.796 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:13.790+0000 7fe04e7fb740 1 -- 172.21.15.39:0/2871304304 shutdown_connections 2024-09-21T03:54:13.796 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:13.790+0000 7fe04e7fb740 1 --2- 172.21.15.39:0/2871304304 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x559bf9a22390 0x559bf9deb120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:13.796 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:13.790+0000 7fe04e7fb740 1 --2- 172.21.15.39:0/2871304304 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559bf9a1a1b0 0x559bf9deabe0 unknown :-1 s=CLOSED pgs=1119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:13.796 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:13.790+0000 7fe04e7fb740 1 --2- 172.21.15.39:0/2871304304 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559bf9a43070 0x559bf9dea6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:13.796 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:13.790+0000 7fe04e7fb740 1 -- 172.21.15.39:0/2871304304 >> 172.21.15.39:0/2871304304 conn(0x559bf9a37460 msgr2=0x559bf9de8590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:54:13.796 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:13.790+0000 7fe04e7fb740 1 -- 172.21.15.39:0/2871304304 shutdown_connections 2024-09-21T03:54:13.796 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:13.790+0000 7fe04e7fb740 1 -- 172.21.15.39:0/2871304304 wait complete. 2024-09-21T03:54:13.796 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:13.791+0000 7fe04e7fb740 1 -- start start 2024-09-21T03:54:13.796 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:13.791+0000 7fe04e7fb740 1 Processor -- start 2024-09-21T03:54:13.796 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:13.791+0000 7fe04e7fb740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x559bf9a1a1b0 0x559bf9df5d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:54:13.796 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:13.791+0000 7fe04e7fb740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559bf9a22390 0x559bf9df62d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:54:13.796 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:13.791+0000 7fe04e7fb740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559bf9a43070 0x559bf9e021a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:54:13.797 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:13.791+0000 7fe04e7fb740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x559bf9bb1700 con 0x559bf9a43070 2024-09-21T03:54:13.797 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:13.791+0000 7fe04e7fb740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x559bf9ceb160 con 0x559bf9a1a1b0 2024-09-21T03:54:13.797 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:13.791+0000 7fe04e7fb740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x559bf9df8e80 con 0x559bf9a22390 2024-09-21T03:54:13.797 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:13.791+0000 7fe046824640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559bf9a43070 0x559bf9e021a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:54:13.797 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:13.791+0000 7fe04e7fb740 10 client.0 _ll_register_callbacks cb 0x559bf9d049e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:54:13.797 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:13.791+0000 7fe046824640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559bf9a43070 0x559bf9e021a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:46184/0 (socket says 172.21.15.39:46184) 2024-09-21T03:54:13.797 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:13.791+0000 7fe046824640 1 -- 172.21.15.39:0/758083169 learned_addr learned my addr 172.21.15.39:0/758083169 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:54:13.797 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:13.791+0000 7fe045822640 1 --2- 172.21.15.39:0/758083169 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559bf9a22390 0x559bf9df62d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:54:13.797 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:13.792+0000 7fe046023640 1 --2- 172.21.15.39:0/758083169 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x559bf9a1a1b0 0x559bf9df5d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:54:13.797 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:13.792+0000 7fe046824640 1 -- 172.21.15.39:0/758083169 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559bf9a22390 msgr2=0x559bf9df62d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:54:13.797 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:13.792+0000 7fe046824640 1 --2- 172.21.15.39:0/758083169 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559bf9a22390 0x559bf9df62d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:13.797 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:13.792+0000 7fe046824640 1 -- 172.21.15.39:0/758083169 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x559bf9a1a1b0 msgr2=0x559bf9df5d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:54:13.797 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:13.792+0000 7fe046824640 1 --2- 172.21.15.39:0/758083169 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x559bf9a1a1b0 0x559bf9df5d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:13.797 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:13.792+0000 7fe046824640 1 -- 172.21.15.39:0/758083169 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559bf9d18c90 con 0x559bf9a43070 2024-09-21T03:54:13.798 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:13.792+0000 7fe046824640 1 --2- 172.21.15.39:0/758083169 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559bf9a43070 0x559bf9e021a0 secure :-1 s=READY pgs=1120 cs=0 l=1 rev1=1 crypto rx=0x7fe038031660 tx=0x7fe03802a8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=199e49a567715645 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:54:13.798 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:13.792+0000 7fe029ffb640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-09-21T03:54:13.798 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:13.792+0000 7fe029ffb640 1 -- 172.21.15.39:0/758083169 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe038004040 con 0x559bf9a43070 2024-09-21T03:54:13.798 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:13.792+0000 7fe029ffb640 1 -- 172.21.15.39:0/758083169 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8572+0+0 (secure 0 0 0) 0x7fe0380041e0 con 0x559bf9a43070 2024-09-21T03:54:13.798 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:13.792+0000 7fe04e7fb740 1 -- 172.21.15.39:0/758083169 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe03c018af0 con 0x559bf9a43070 2024-09-21T03:54:13.798 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:13.793+0000 7fe029ffb640 1 -- 172.21.15.39:0/758083169 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fe0380043e0 con 0x559bf9a43070 2024-09-21T03:54:13.798 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:13.793+0000 7fe04e7fb740 10 client.17649 fetch_fsmap learned FSMap version 29 2024-09-21T03:54:13.798 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:13.793+0000 7fe04e7fb740 1 -- 172.21.15.39:0/758083169 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x559bf9e04b00 con 0x559bf9a43070 2024-09-21T03:54:13.798 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:13.793+0000 7fe029ffb640 1 -- 172.21.15.39:0/758083169 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe03c018af0 con 0x559bf9a43070 2024-09-21T03:54:13.798 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:13.793+0000 7fe029ffb640 1 -- 172.21.15.39:0/758083169 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fe038051070 con 0x559bf9a43070 2024-09-21T03:54:13.798 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:13.793+0000 7fe04e7fb740 10 client.17649 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T03:54:13.798 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:13.793+0000 7fe04e7fb740 10 client.17649 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-21T03:54:13.798 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:13.793+0000 7fe04e7fb740 20 client.17649 populate_metadata read hostname 'smithi039' 2024-09-21T03:54:13.798 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:13.793+0000 7fe04e7fb740 1 --2- 172.21.15.39:0/758083169 >> [v2:172.21.15.161:6838/2190107270,v1:172.21.15.161:6839/2190107270] conn(0x559bf9a019f0 0x559bf9e080a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:54:13.799 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:13.793+0000 7fe04e7fb740 4 client.17649 mds_command: new command op to 24667 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:54:13.799 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:13.793+0000 7fe04e7fb740 1 -- 172.21.15.39:0/758083169 --> [v2:172.21.15.161:6838/2190107270,v1:172.21.15.161:6839/2190107270] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559bf9d62df0 con 0x559bf9a019f0 2024-09-21T03:54:13.799 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:13.793+0000 7fe046023640 1 --2- 172.21.15.39:0/758083169 >> [v2:172.21.15.161:6838/2190107270,v1:172.21.15.161:6839/2190107270] conn(0x559bf9a019f0 0x559bf9e080a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:54:13.799 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:13.794+0000 7fe046023640 1 --2- 172.21.15.39:0/758083169 >> [v2:172.21.15.161:6838/2190107270,v1:172.21.15.161:6839/2190107270] conn(0x559bf9a019f0 0x559bf9e080a0 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=eab6e31e6f1a0536 server_cookie=65e3b628fa0b539c in_seq=0 out_seq=0 2024-09-21T03:54:13.799 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:13.794+0000 7fe029ffb640 10 client.17649 ms_handle_connect on v2:172.21.15.161:6838/2190107270 2024-09-21T03:54:13.800 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:13.794+0000 7fe029ffb640 1 -- 172.21.15.39:0/758083169 <== mds.1 v2:172.21.15.161:6838/2190107270 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x559bf9d62df0 con 0x559bf9a019f0 2024-09-21T03:54:13.800 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:13.794+0000 7fe029ffb640 10 client.17649 handle_command_reply: tid=0 2024-09-21T03:54:13.800 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:13.795+0000 7fe04e7fb740 1 client.17649 shutdown 2024-09-21T03:54:13.800 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:13.795+0000 7fe04e7fb740 1 -- 172.21.15.39:0/758083169 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559bf9a43070 msgr2=0x559bf9e021a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:54:13.800 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:13.795+0000 7fe04e7fb740 1 --2- 172.21.15.39:0/758083169 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559bf9a43070 0x559bf9e021a0 secure :-1 s=READY pgs=1120 cs=0 l=1 rev1=1 crypto rx=0x7fe038031660 tx=0x7fe03802a8c0 comp rx=0 tx=0).stop 2024-09-21T03:54:13.800 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:13.795+0000 7fe04e7fb740 1 -- 172.21.15.39:0/758083169 shutdown_connections 2024-09-21T03:54:13.800 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:13.795+0000 7fe04e7fb740 1 --2- 172.21.15.39:0/758083169 >> [v2:172.21.15.161:6838/2190107270,v1:172.21.15.161:6839/2190107270] conn(0x559bf9a019f0 0x559bf9e080a0 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:13.800 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:13.795+0000 7fe04e7fb740 1 --2- 172.21.15.39:0/758083169 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559bf9a43070 0x559bf9e021a0 unknown :-1 s=CLOSED pgs=1120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:13.800 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:13.795+0000 7fe04e7fb740 1 --2- 172.21.15.39:0/758083169 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559bf9a22390 0x559bf9df62d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:13.801 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:13.795+0000 7fe04e7fb740 1 --2- 172.21.15.39:0/758083169 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x559bf9a1a1b0 0x559bf9df5d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:13.801 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:13.795+0000 7fe04e7fb740 1 -- 172.21.15.39:0/758083169 >> 172.21.15.39:0/758083169 conn(0x559bf9a37460 msgr2=0x559bf9ded830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:54:13.801 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:13.795+0000 7fe029ffb640 0 client.17649 ms_handle_reset on v2:172.21.15.161:6838/2190107270 2024-09-21T03:54:13.801 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:13.795+0000 7fe04e7fb740 1 -- 172.21.15.39:0/758083169 shutdown_connections 2024-09-21T03:54:13.801 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:13.795+0000 7fe04e7fb740 1 -- 172.21.15.39:0/758083169 wait complete. 2024-09-21T03:54:13.801 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:13.795+0000 7fe04e7fb740 20 client.17649 trim_cache size 0 max 16384 2024-09-21T03:54:13.811 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:13.808+0000 7fe04e7fb740 1 Processor -- start 2024-09-21T03:54:13.811 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:13.808+0000 7fe04e7fb740 1 -- start start 2024-09-21T03:54:13.811 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:13.808+0000 7fe04e7fb740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x559bf9a1a1b0 0x559bf9df4610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:54:13.811 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:13.808+0000 7fe04e7fb740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559bf9a22390 0x559bf9ded7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:54:13.811 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:13.808+0000 7fe04e7fb740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559bf9a43070 0x559bf9dedce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:54:13.811 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:13.808+0000 7fe04e7fb740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fe0401af890 con 0x559bf9a22390 2024-09-21T03:54:13.811 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:13.808+0000 7fe04e7fb740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fe0401b13d0 con 0x559bf9a1a1b0 2024-09-21T03:54:13.812 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:13.808+0000 7fe04e7fb740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fe0401aff60 con 0x559bf9a43070 2024-09-21T03:54:13.812 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:13.809+0000 7fe045822640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559bf9a22390 0x559bf9ded7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:54:13.812 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:13.809+0000 7fe045822640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559bf9a22390 0x559bf9ded7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:46186/0 (socket says 172.21.15.39:46186) 2024-09-21T03:54:13.812 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:13.809+0000 7fe045822640 1 -- 172.21.15.39:0/1876126704 learned_addr learned my addr 172.21.15.39:0/1876126704 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:54:13.812 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:13.809+0000 7fe046023640 1 --2- 172.21.15.39:0/1876126704 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x559bf9a1a1b0 0x559bf9df4610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:54:13.812 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:13.809+0000 7fe046824640 1 --2- 172.21.15.39:0/1876126704 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559bf9a43070 0x559bf9dedce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:54:13.812 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:13.809+0000 7fe045822640 1 -- 172.21.15.39:0/1876126704 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559bf9a43070 msgr2=0x559bf9dedce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:54:13.812 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:13.809+0000 7fe045822640 1 --2- 172.21.15.39:0/1876126704 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559bf9a43070 0x559bf9dedce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:13.812 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:13.809+0000 7fe045822640 1 -- 172.21.15.39:0/1876126704 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x559bf9a1a1b0 msgr2=0x559bf9df4610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:54:13.812 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:13.809+0000 7fe045822640 1 --2- 172.21.15.39:0/1876126704 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x559bf9a1a1b0 0x559bf9df4610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:13.812 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:13.809+0000 7fe045822640 1 -- 172.21.15.39:0/1876126704 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559bf9a01c10 con 0x559bf9a22390 2024-09-21T03:54:13.812 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:13.809+0000 7fe045822640 1 --2- 172.21.15.39:0/1876126704 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559bf9a22390 0x559bf9ded7a0 secure :-1 s=READY pgs=1121 cs=0 l=1 rev1=1 crypto rx=0x7fe030007770 tx=0x7fe0300aca20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d34e469e22c6ace8 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:54:13.812 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:13.809+0000 7fe02bfff640 1 -- 172.21.15.39:0/1876126704 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe03001d040 con 0x559bf9a22390 2024-09-21T03:54:13.812 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:13.809+0000 7fe02bfff640 1 -- 172.21.15.39:0/1876126704 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe0300ad9b0 con 0x559bf9a22390 2024-09-21T03:54:13.812 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:13.809+0000 7fe02bfff640 1 -- 172.21.15.39:0/1876126704 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0300abf80 con 0x559bf9a22390 2024-09-21T03:54:13.816 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:13.810+0000 7fe04e7fb740 1 -- 172.21.15.39:0/1876126704 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559bf9a22390 msgr2=0x559bf9ded7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:54:13.816 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:13.810+0000 7fe04e7fb740 1 --2- 172.21.15.39:0/1876126704 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559bf9a22390 0x559bf9ded7a0 secure :-1 s=READY pgs=1121 cs=0 l=1 rev1=1 crypto rx=0x7fe030007770 tx=0x7fe0300aca20 comp rx=0 tx=0).stop 2024-09-21T03:54:13.816 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:13.810+0000 7fe04e7fb740 1 -- 172.21.15.39:0/1876126704 shutdown_connections 2024-09-21T03:54:13.816 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:13.810+0000 7fe04e7fb740 1 --2- 172.21.15.39:0/1876126704 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559bf9a43070 0x559bf9dedce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:13.816 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:13.810+0000 7fe04e7fb740 1 --2- 172.21.15.39:0/1876126704 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559bf9a22390 0x559bf9ded7a0 unknown :-1 s=CLOSED pgs=1121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:13.816 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:13.810+0000 7fe04e7fb740 1 --2- 172.21.15.39:0/1876126704 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x559bf9a1a1b0 0x559bf9df4610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:13.816 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:13.810+0000 7fe04e7fb740 1 -- 172.21.15.39:0/1876126704 >> 172.21.15.39:0/1876126704 conn(0x559bf9a37460 msgr2=0x559bf9df0f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:54:13.816 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:13.810+0000 7fe04e7fb740 1 -- 172.21.15.39:0/1876126704 shutdown_connections 2024-09-21T03:54:13.817 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:13.810+0000 7fe04e7fb740 1 -- 172.21.15.39:0/1876126704 wait complete. 2024-09-21T03:54:13.817 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:13.811+0000 7fe04e7fb740 1 -- start start 2024-09-21T03:54:13.817 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:13.811+0000 7fe04e7fb740 1 Processor -- start 2024-09-21T03:54:13.817 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:13.811+0000 7fe04e7fb740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559bf9a1a1b0 0x559bf9dfa060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:54:13.817 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:13.811+0000 7fe04e7fb740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559bf9a22390 0x559bf9dfa5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:54:13.817 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:13.811+0000 7fe04e7fb740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x559bf9a43070 0x559bf9dfaae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:54:13.817 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:13.811+0000 7fe04e7fb740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fe0401b0d00 con 0x559bf9a1a1b0 2024-09-21T03:54:13.817 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:13.811+0000 7fe04e7fb740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fe0401b0630 con 0x559bf9a43070 2024-09-21T03:54:13.817 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:13.811+0000 7fe04e7fb740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fe0401ad890 con 0x559bf9a22390 2024-09-21T03:54:13.817 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:13.811+0000 7fe04e7fb740 10 client.0 _ll_register_callbacks cb 0x7fe03802f330 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:54:13.817 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:13.811+0000 7fe046023640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559bf9a1a1b0 0x559bf9dfa060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:54:13.817 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:13.811+0000 7fe046023640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559bf9a1a1b0 0x559bf9dfa060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:46194/0 (socket says 172.21.15.39:46194) 2024-09-21T03:54:13.817 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:13.811+0000 7fe046023640 1 -- 172.21.15.39:0/2434752469 learned_addr learned my addr 172.21.15.39:0/2434752469 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:54:13.817 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:13.811+0000 7fe045822640 1 --2- 172.21.15.39:0/2434752469 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559bf9a22390 0x559bf9dfa5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:54:13.818 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:13.811+0000 7fe046023640 1 -- 172.21.15.39:0/2434752469 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559bf9a22390 msgr2=0x559bf9dfa5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:54:13.818 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:13.811+0000 7fe046023640 1 --2- 172.21.15.39:0/2434752469 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559bf9a22390 0x559bf9dfa5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:13.818 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:13.811+0000 7fe046023640 1 -- 172.21.15.39:0/2434752469 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x559bf9a43070 msgr2=0x559bf9dfaae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:54:13.818 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:13.811+0000 7fe046023640 1 --2- 172.21.15.39:0/2434752469 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x559bf9a43070 0x559bf9dfaae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:13.818 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:13.812+0000 7fe046023640 1 -- 172.21.15.39:0/2434752469 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559bf9e71aa0 con 0x559bf9a1a1b0 2024-09-21T03:54:13.818 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:13.812+0000 7fe045822640 1 --2- 172.21.15.39:0/2434752469 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559bf9a22390 0x559bf9dfa5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:54:13.818 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:13.812+0000 7fe046023640 1 --2- 172.21.15.39:0/2434752469 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559bf9a1a1b0 0x559bf9dfa060 secure :-1 s=READY pgs=1122 cs=0 l=1 rev1=1 crypto rx=0x7fe03c010670 tx=0x7fe03c010600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=768faf8fcc61317c server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:54:13.818 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:13.812+0000 7fe029ffb640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-09-21T03:54:13.818 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:13.812+0000 7fe029ffb640 1 -- 172.21.15.39:0/2434752469 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe03c01d040 con 0x559bf9a1a1b0 2024-09-21T03:54:13.818 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:13.812+0000 7fe029ffb640 1 -- 172.21.15.39:0/2434752469 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8572+0+0 (secure 0 0 0) 0x7fe03c0184c0 con 0x559bf9a1a1b0 2024-09-21T03:54:13.818 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:13.812+0000 7fe04e7fb740 1 -- 172.21.15.39:0/2434752469 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe0300ad9b0 con 0x559bf9a1a1b0 2024-09-21T03:54:13.818 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:13.812+0000 7fe029ffb640 1 -- 172.21.15.39:0/2434752469 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fe03c0186c0 con 0x559bf9a1a1b0 2024-09-21T03:54:13.818 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:13.812+0000 7fe04e7fb740 10 client.17655 fetch_fsmap learned FSMap version 29 2024-09-21T03:54:13.818 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:13.812+0000 7fe04e7fb740 1 -- 172.21.15.39:0/2434752469 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x559bf9e72cb0 con 0x559bf9a1a1b0 2024-09-21T03:54:13.818 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:13.812+0000 7fe029ffb640 1 -- 172.21.15.39:0/2434752469 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe0300ad9b0 con 0x559bf9a1a1b0 2024-09-21T03:54:13.819 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:13.813+0000 7fe029ffb640 1 -- 172.21.15.39:0/2434752469 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fe03c039070 con 0x559bf9a1a1b0 2024-09-21T03:54:13.819 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:13.813+0000 7fe04e7fb740 10 client.17655 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T03:54:13.819 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:13.813+0000 7fe04e7fb740 10 client.17655 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-21T03:54:13.819 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:13.813+0000 7fe04e7fb740 20 client.17655 populate_metadata read hostname 'smithi039' 2024-09-21T03:54:13.819 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:13.813+0000 7fe04e7fb740 1 --2- 172.21.15.39:0/2434752469 >> [v2:172.21.15.161:6838/2190107270,v1:172.21.15.161:6839/2190107270] conn(0x559bf99d6da0 0x559bf9e751f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:54:13.819 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:13.813+0000 7fe04e7fb740 4 client.17655 mds_command: new command op to 24667 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T03:54:13.819 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:13.813+0000 7fe04e7fb740 1 -- 172.21.15.39:0/2434752469 --> [v2:172.21.15.161:6838/2190107270,v1:172.21.15.161:6839/2190107270] -- command(tid 0: {"prefix": "damage ls"}) -- 0x559bf9d5e8a0 con 0x559bf99d6da0 2024-09-21T03:54:13.819 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:13.813+0000 7fe045822640 1 --2- 172.21.15.39:0/2434752469 >> [v2:172.21.15.161:6838/2190107270,v1:172.21.15.161:6839/2190107270] conn(0x559bf99d6da0 0x559bf9e751f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:54:13.819 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:13.813+0000 7fe029ffb640 10 client.17655 ms_handle_connect on v2:172.21.15.161:6838/2190107270 2024-09-21T03:54:13.819 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:13.813+0000 7fe045822640 1 --2- 172.21.15.39:0/2434752469 >> [v2:172.21.15.161:6838/2190107270,v1:172.21.15.161:6839/2190107270] conn(0x559bf99d6da0 0x559bf9e751f0 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=32fe506560a7e572 server_cookie=728022a2aac4e82a in_seq=0 out_seq=0 2024-09-21T03:54:13.819 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:13.814+0000 7fe029ffb640 1 -- 172.21.15.39:0/2434752469 <== mds.1 v2:172.21.15.161:6838/2190107270 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x559bf9d5e8a0 con 0x559bf99d6da0 2024-09-21T03:54:13.819 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:13.814+0000 7fe029ffb640 10 client.17655 handle_command_reply: tid=0 2024-09-21T03:54:13.819 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:13.814+0000 7fe04e7fb740 1 client.17655 shutdown 2024-09-21T03:54:13.819 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:13.814+0000 7fe04e7fb740 1 -- 172.21.15.39:0/2434752469 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559bf9a1a1b0 msgr2=0x559bf9dfa060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:54:13.820 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:13.814+0000 7fe04e7fb740 1 --2- 172.21.15.39:0/2434752469 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559bf9a1a1b0 0x559bf9dfa060 secure :-1 s=READY pgs=1122 cs=0 l=1 rev1=1 crypto rx=0x7fe03c010670 tx=0x7fe03c010600 comp rx=0 tx=0).stop 2024-09-21T03:54:13.820 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:13.814+0000 7fe04e7fb740 1 -- 172.21.15.39:0/2434752469 shutdown_connections 2024-09-21T03:54:13.820 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:13.814+0000 7fe04e7fb740 1 --2- 172.21.15.39:0/2434752469 >> [v2:172.21.15.161:6838/2190107270,v1:172.21.15.161:6839/2190107270] conn(0x559bf99d6da0 0x559bf9e751f0 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:13.820 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:13.814+0000 7fe04e7fb740 1 --2- 172.21.15.39:0/2434752469 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x559bf9a43070 0x559bf9dfaae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:13.820 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:13.814+0000 7fe04e7fb740 1 --2- 172.21.15.39:0/2434752469 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559bf9a22390 0x559bf9dfa5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:13.820 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:13.814+0000 7fe04e7fb740 1 --2- 172.21.15.39:0/2434752469 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559bf9a1a1b0 0x559bf9dfa060 unknown :-1 s=CLOSED pgs=1122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:13.820 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:13.814+0000 7fe04e7fb740 1 -- 172.21.15.39:0/2434752469 >> 172.21.15.39:0/2434752469 conn(0x559bf9a37460 msgr2=0x559bf9df4580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:54:13.820 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:13.814+0000 7fe029ffb640 0 client.17655 ms_handle_reset on v2:172.21.15.161:6838/2190107270 2024-09-21T03:54:13.820 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:13.814+0000 7fe04e7fb740 1 -- 172.21.15.39:0/2434752469 shutdown_connections 2024-09-21T03:54:13.820 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:13.814+0000 7fe04e7fb740 1 -- 172.21.15.39:0/2434752469 wait complete. 2024-09-21T03:54:13.820 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:13.814+0000 7fe04e7fb740 20 client.17655 trim_cache size 0 max 16384 2024-09-21T03:54:13.821 INFO:teuthology.orchestra.run.smithi039.stdout:[] 2024-09-21T03:54:13.821 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:13.815+0000 7fe048287640 1 -- 172.21.15.39:0/201371835 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7fe02c0b6680 msgr2=0x7fe02c0b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:54:13.821 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:13.815+0000 7fe048287640 1 --2- 172.21.15.39:0/201371835 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7fe02c0b6680 0x7fe02c0b8b90 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7fe038005cc0 tx=0x7fe038005c30 comp rx=0 tx=0).stop 2024-09-21T03:54:13.821 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:13.815+0000 7fe048287640 1 -- 172.21.15.39:0/201371835 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe0401062a0 msgr2=0x7fe04019aa40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:54:13.821 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:13.815+0000 7fe048287640 1 --2- 172.21.15.39:0/201371835 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe0401062a0 0x7fe04019aa40 secure :-1 s=READY pgs=1118 cs=0 l=1 rev1=1 crypto rx=0x7fe03000c970 tx=0x7fe03000ce30 comp rx=0 tx=0).stop 2024-09-21T03:54:13.822 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:13.815+0000 7fe048287640 1 -- 172.21.15.39:0/201371835 shutdown_connections 2024-09-21T03:54:13.822 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:13.815+0000 7fe048287640 1 --2- 172.21.15.39:0/201371835 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7fe02c0b6680 0x7fe02c0b8b90 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:13.822 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:13.815+0000 7fe048287640 1 --2- 172.21.15.39:0/201371835 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe040106bf0 0x7fe04019fce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:13.822 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:13.815+0000 7fe048287640 1 --2- 172.21.15.39:0/201371835 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe0401062a0 0x7fe04019aa40 unknown :-1 s=CLOSED pgs=1118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:13.822 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:13.815+0000 7fe048287640 1 --2- 172.21.15.39:0/201371835 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe0401020b0 0x7fe04019a4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:13.822 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:13.815+0000 7fe048287640 1 -- 172.21.15.39:0/201371835 >> 172.21.15.39:0/201371835 conn(0x7fe040079e30 msgr2=0x7fe040107530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:54:13.822 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:13.815+0000 7fe048287640 1 -- 172.21.15.39:0/201371835 shutdown_connections 2024-09-21T03:54:13.822 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:13.815+0000 7fe048287640 1 -- 172.21.15.39:0/201371835 wait complete. 2024-09-21T03:54:14.721 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 51116d5c-77ca-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 damage ls 2024-09-21T03:54:14.984 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/51116d5c-77ca-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:54:15.251 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:54:14 smithi039 ceph-mon[31211]: pgmap v574: 97 pgs: 97 active+clean; 10 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 53 KiB/s wr, 30 op/s 2024-09-21T03:54:15.277 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:54:14 smithi150 ceph-mon[34289]: pgmap v574: 97 pgs: 97 active+clean; 10 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 53 KiB/s wr, 30 op/s 2024-09-21T03:54:15.468 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:54:14 smithi161 ceph-mon[34533]: pgmap v574: 97 pgs: 97 active+clean; 10 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 53 KiB/s wr, 30 op/s 2024-09-21T03:54:15.469 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:15.467+0000 7fd59cae7640 1 -- 172.21.15.39:0/3064985011 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd598108d20 msgr2=0x7fd59810f550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:54:15.469 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:15.467+0000 7fd59cae7640 1 --2- 172.21.15.39:0/3064985011 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd598108d20 0x7fd59810f550 secure :-1 s=READY pgs=1123 cs=0 l=1 rev1=1 crypto rx=0x7fd590009ca0 tx=0x7fd59002a8c0 comp rx=0 tx=0).stop 2024-09-21T03:54:15.469 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:15.467+0000 7fd59cae7640 1 -- 172.21.15.39:0/3064985011 shutdown_connections 2024-09-21T03:54:15.469 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:15.467+0000 7fd59cae7640 1 --2- 172.21.15.39:0/3064985011 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd598108d20 0x7fd59810f550 unknown :-1 s=CLOSED pgs=1123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:15.469 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:15.467+0000 7fd59cae7640 1 --2- 172.21.15.39:0/3064985011 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd5981083d0 0x7fd5981087e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:15.469 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:15.467+0000 7fd59cae7640 1 --2- 172.21.15.39:0/3064985011 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd598102ae0 0x7fd598102ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:15.470 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:15.467+0000 7fd59cae7640 1 -- 172.21.15.39:0/3064985011 >> 172.21.15.39:0/3064985011 conn(0x7fd598079760 msgr2=0x7fd598079b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:54:15.470 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:15.467+0000 7fd59cae7640 1 -- 172.21.15.39:0/3064985011 shutdown_connections 2024-09-21T03:54:15.470 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:15.467+0000 7fd59cae7640 1 -- 172.21.15.39:0/3064985011 wait complete. 2024-09-21T03:54:15.472 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:15.468+0000 7fd59cae7640 1 Processor -- start 2024-09-21T03:54:15.472 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:15.468+0000 7fd59cae7640 1 -- start start 2024-09-21T03:54:15.472 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:15.468+0000 7fd59cae7640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd598102ae0 0x7fd5980ff670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:54:15.472 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:15.468+0000 7fd59cae7640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd5981083d0 0x7fd5980ffbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:54:15.472 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:15.468+0000 7fd59cae7640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd598108d20 0x7fd5981000f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:54:15.472 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:15.468+0000 7fd59cae7640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fd598112990 con 0x7fd598102ae0 2024-09-21T03:54:15.472 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:15.468+0000 7fd59cae7640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fd598112810 con 0x7fd5981083d0 2024-09-21T03:54:15.472 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:15.468+0000 7fd59cae7640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fd598112b10 con 0x7fd598108d20 2024-09-21T03:54:15.472 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:15.469+0000 7fd59659c640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd598102ae0 0x7fd5980ff670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:54:15.472 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:15.469+0000 7fd59659c640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd598102ae0 0x7fd5980ff670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:46212/0 (socket says 172.21.15.39:46212) 2024-09-21T03:54:15.473 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:15.469+0000 7fd59659c640 1 -- 172.21.15.39:0/2536452964 learned_addr learned my addr 172.21.15.39:0/2536452964 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:54:15.473 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:15.469+0000 7fd595d9b640 1 --2- 172.21.15.39:0/2536452964 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd5981083d0 0x7fd5980ffbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:54:15.473 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:15.469+0000 7fd596d9d640 1 --2- 172.21.15.39:0/2536452964 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd598108d20 0x7fd5981000f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:54:15.473 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:15.469+0000 7fd59659c640 1 -- 172.21.15.39:0/2536452964 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd598108d20 msgr2=0x7fd5981000f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:54:15.473 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:15.469+0000 7fd59659c640 1 --2- 172.21.15.39:0/2536452964 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd598108d20 0x7fd5981000f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:15.473 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:15.469+0000 7fd59659c640 1 -- 172.21.15.39:0/2536452964 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd5981083d0 msgr2=0x7fd5980ffbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:54:15.473 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:15.469+0000 7fd59659c640 1 --2- 172.21.15.39:0/2536452964 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd5981083d0 0x7fd5980ffbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:15.473 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:15.469+0000 7fd59659c640 1 -- 172.21.15.39:0/2536452964 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd5981a86b0 con 0x7fd598102ae0 2024-09-21T03:54:15.473 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:15.469+0000 7fd596d9d640 1 --2- 172.21.15.39:0/2536452964 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd598108d20 0x7fd5981000f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:54:15.473 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:15.469+0000 7fd59659c640 1 --2- 172.21.15.39:0/2536452964 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd598102ae0 0x7fd5980ff670 secure :-1 s=READY pgs=1124 cs=0 l=1 rev1=1 crypto rx=0x7fd58800d950 tx=0x7fd58800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:54:15.473 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:15.469+0000 7fd5877fe640 1 -- 172.21.15.39:0/2536452964 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd588017070 con 0x7fd598102ae0 2024-09-21T03:54:15.473 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:15.469+0000 7fd59cae7640 1 -- 172.21.15.39:0/2536452964 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd5981a89a0 con 0x7fd598102ae0 2024-09-21T03:54:15.474 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:15.469+0000 7fd5877fe640 1 -- 172.21.15.39:0/2536452964 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd588009c30 con 0x7fd598102ae0 2024-09-21T03:54:15.474 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:15.469+0000 7fd5877fe640 1 -- 172.21.15.39:0/2536452964 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd58800f620 con 0x7fd598102ae0 2024-09-21T03:54:15.474 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:15.469+0000 7fd59cae7640 1 -- 172.21.15.39:0/2536452964 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd5981a8ee0 con 0x7fd598102ae0 2024-09-21T03:54:15.476 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:15.472+0000 7fd5877fe640 1 -- 172.21.15.39:0/2536452964 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 27) ==== 154643+0+0 (secure 0 0 0) 0x7fd588010460 con 0x7fd598102ae0 2024-09-21T03:54:15.476 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:15.472+0000 7fd5a305b740 1 Processor -- start 2024-09-21T03:54:15.477 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:15.473+0000 7fd5a305b740 1 -- start start 2024-09-21T03:54:15.477 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:15.473+0000 7fd5a305b740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56106a426070 0x56106a7cd2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:54:15.477 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:15.473+0000 7fd5a305b740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56106a3fd1b0 0x56106a7cd800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:54:15.477 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:15.473+0000 7fd5a305b740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56106a405390 0x56106a7cdd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:54:15.477 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:15.473+0000 7fd5a305b740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x56106a743ea0 con 0x56106a3fd1b0 2024-09-21T03:54:15.477 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:15.473+0000 7fd5a305b740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x56106a7332d0 con 0x56106a426070 2024-09-21T03:54:15.477 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:15.473+0000 7fd5a305b740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x56106a7370e0 con 0x56106a405390 2024-09-21T03:54:15.477 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:15.473+0000 7fd59659c640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56106a3fd1b0 0x56106a7cd800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:54:15.477 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:15.473+0000 7fd59659c640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56106a3fd1b0 0x56106a7cd800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:46226/0 (socket says 172.21.15.39:46226) 2024-09-21T03:54:15.477 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:15.473+0000 7fd59659c640 1 -- 172.21.15.39:0/4274937570 learned_addr learned my addr 172.21.15.39:0/4274937570 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:54:15.477 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:15.473+0000 7fd5877fe640 1 --2- 172.21.15.39:0/2536452964 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7fd5800b6680 0x7fd5800b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:54:15.478 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:15.473+0000 7fd595d9b640 1 --2- 172.21.15.39:0/4274937570 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56106a405390 0x56106a7cdd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:54:15.478 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:15.473+0000 7fd596d9d640 1 --2- 172.21.15.39:0/4274937570 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56106a426070 0x56106a7cd2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:54:15.478 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:15.474+0000 7fd596d9d640 1 --2- 172.21.15.39:0/2536452964 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7fd5800b6680 0x7fd5800b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:54:15.478 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:15.474+0000 7fd59659c640 1 -- 172.21.15.39:0/4274937570 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56106a405390 msgr2=0x56106a7cdd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:54:15.478 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:15.474+0000 7fd59659c640 1 --2- 172.21.15.39:0/4274937570 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56106a405390 0x56106a7cdd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:15.478 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:15.474+0000 7fd59659c640 1 -- 172.21.15.39:0/4274937570 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56106a426070 msgr2=0x56106a7cd2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:54:15.478 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:15.474+0000 7fd59659c640 1 --2- 172.21.15.39:0/4274937570 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56106a426070 0x56106a7cd2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:15.478 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:15.474+0000 7fd59659c640 1 -- 172.21.15.39:0/4274937570 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56106a491650 con 0x56106a3fd1b0 2024-09-21T03:54:15.478 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:15.474+0000 7fd595d9b640 1 --2- 172.21.15.39:0/4274937570 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56106a405390 0x56106a7cdd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:54:15.478 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:15.474+0000 7fd5877fe640 1 -- 172.21.15.39:0/2536452964 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8572+0+0 (secure 0 0 0) 0x7fd5880e73c0 con 0x7fd598102ae0 2024-09-21T03:54:15.478 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:15.474+0000 7fd596d9d640 1 --2- 172.21.15.39:0/2536452964 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7fd5800b6680 0x7fd5800b8b90 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7fd590004af0 tx=0x7fd59002a8c0 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:54:15.479 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:15.474+0000 7fd59659c640 1 --2- 172.21.15.39:0/4274937570 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56106a3fd1b0 0x56106a7cd800 secure :-1 s=READY pgs=1125 cs=0 l=1 rev1=1 crypto rx=0x7fd5880acee0 tx=0x7fd588022000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=29e2ef283bfb47d3 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:54:15.479 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:15.474+0000 7fd584ff9640 1 -- 172.21.15.39:0/4274937570 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd58806f6a0 con 0x56106a3fd1b0 2024-09-21T03:54:15.479 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:15.474+0000 7fd584ff9640 1 -- 172.21.15.39:0/4274937570 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd5880accf0 con 0x56106a3fd1b0 2024-09-21T03:54:15.479 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:15.474+0000 7fd584ff9640 1 -- 172.21.15.39:0/4274937570 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd588079040 con 0x56106a3fd1b0 2024-09-21T03:54:15.483 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:15.475+0000 7fd5a305b740 1 -- 172.21.15.39:0/4274937570 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56106a3fd1b0 msgr2=0x56106a7cd800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:54:15.483 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:15.475+0000 7fd5a305b740 1 --2- 172.21.15.39:0/4274937570 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56106a3fd1b0 0x56106a7cd800 secure :-1 s=READY pgs=1125 cs=0 l=1 rev1=1 crypto rx=0x7fd5880acee0 tx=0x7fd588022000 comp rx=0 tx=0).stop 2024-09-21T03:54:15.484 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:15.475+0000 7fd5a305b740 1 -- 172.21.15.39:0/4274937570 shutdown_connections 2024-09-21T03:54:15.484 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:15.475+0000 7fd5a305b740 1 --2- 172.21.15.39:0/4274937570 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56106a405390 0x56106a7cdd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:15.484 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:15.475+0000 7fd5a305b740 1 --2- 172.21.15.39:0/4274937570 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56106a3fd1b0 0x56106a7cd800 unknown :-1 s=CLOSED pgs=1125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:15.484 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:15.475+0000 7fd5a305b740 1 --2- 172.21.15.39:0/4274937570 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56106a426070 0x56106a7cd2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:15.484 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:15.475+0000 7fd5a305b740 1 -- 172.21.15.39:0/4274937570 >> 172.21.15.39:0/4274937570 conn(0x56106a41a460 msgr2=0x56106a7cb1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:54:15.484 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:15.475+0000 7fd5a305b740 1 -- 172.21.15.39:0/4274937570 shutdown_connections 2024-09-21T03:54:15.484 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:15.475+0000 7fd5a305b740 1 -- 172.21.15.39:0/4274937570 wait complete. 2024-09-21T03:54:15.484 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:15.476+0000 7fd5a305b740 1 -- start start 2024-09-21T03:54:15.485 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:15.476+0000 7fd5a305b740 1 Processor -- start 2024-09-21T03:54:15.485 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:15.476+0000 7fd5a305b740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56106a3fd1b0 0x56106a7d89a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:54:15.485 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:15.476+0000 7fd5a305b740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56106a405390 0x56106a7d8ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:54:15.485 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:15.476+0000 7fd5a305b740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56106a426070 0x56106a7e4cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:54:15.485 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:15.476+0000 7fd5a305b740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x56106a594700 con 0x56106a405390 2024-09-21T03:54:15.485 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:15.476+0000 7fd5a305b740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x56106a6ce160 con 0x56106a3fd1b0 2024-09-21T03:54:15.485 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:15.476+0000 7fd5a305b740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x56106a7db9e0 con 0x56106a426070 2024-09-21T03:54:15.485 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:15.476+0000 7fd5a305b740 10 client.0 _ll_register_callbacks cb 0x56106a6e79e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:54:15.485 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:15.476+0000 7fd595d9b640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56106a405390 0x56106a7d8ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:54:15.485 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:15.476+0000 7fd595d9b640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56106a405390 0x56106a7d8ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:46230/0 (socket says 172.21.15.39:46230) 2024-09-21T03:54:15.485 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:15.476+0000 7fd595d9b640 1 -- 172.21.15.39:0/1833558595 learned_addr learned my addr 172.21.15.39:0/1833558595 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:54:15.485 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:15.477+0000 7fd596d9d640 1 --2- 172.21.15.39:0/1833558595 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56106a426070 0x56106a7e4cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:54:15.485 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:15.477+0000 7fd59659c640 1 --2- 172.21.15.39:0/1833558595 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56106a3fd1b0 0x56106a7d89a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:54:15.486 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:15.477+0000 7fd595d9b640 1 -- 172.21.15.39:0/1833558595 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56106a426070 msgr2=0x56106a7e4cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:54:15.486 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:15.477+0000 7fd595d9b640 1 --2- 172.21.15.39:0/1833558595 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56106a426070 0x56106a7e4cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:15.486 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:15.477+0000 7fd595d9b640 1 -- 172.21.15.39:0/1833558595 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56106a3fd1b0 msgr2=0x56106a7d89a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:54:15.486 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:15.477+0000 7fd595d9b640 1 --2- 172.21.15.39:0/1833558595 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56106a3fd1b0 0x56106a7d89a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:15.486 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:15.477+0000 7fd595d9b640 1 -- 172.21.15.39:0/1833558595 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56106a6fbc90 con 0x56106a405390 2024-09-21T03:54:15.486 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:15.477+0000 7fd596d9d640 1 --2- 172.21.15.39:0/1833558595 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56106a426070 0x56106a7e4cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:54:15.486 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:15.477+0000 7fd59659c640 1 --2- 172.21.15.39:0/1833558595 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56106a3fd1b0 0x56106a7d89a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:54:15.486 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:15.477+0000 7fd595d9b640 1 --2- 172.21.15.39:0/1833558595 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56106a405390 0x56106a7d8ee0 secure :-1 s=READY pgs=1126 cs=0 l=1 rev1=1 crypto rx=0x7fd58c00bf80 tx=0x7fd58c0118d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d6c27d009a32d06c server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:54:15.486 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:15.477+0000 7fd57dffb640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-09-21T03:54:15.486 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:15.477+0000 7fd57dffb640 1 -- 172.21.15.39:0/1833558595 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd58c004020 con 0x56106a405390 2024-09-21T03:54:15.486 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:15.477+0000 7fd57dffb640 1 -- 172.21.15.39:0/1833558595 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8572+0+0 (secure 0 0 0) 0x7fd58c0041c0 con 0x56106a405390 2024-09-21T03:54:15.486 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:15.477+0000 7fd5a305b740 1 -- 172.21.15.39:0/1833558595 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd5880accf0 con 0x56106a405390 2024-09-21T03:54:15.486 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:15.477+0000 7fd57dffb640 1 -- 172.21.15.39:0/1833558595 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fd58c0043c0 con 0x56106a405390 2024-09-21T03:54:15.486 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:15.477+0000 7fd5a305b740 10 client.17667 fetch_fsmap learned FSMap version 29 2024-09-21T03:54:15.487 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:15.477+0000 7fd5a305b740 1 -- 172.21.15.39:0/1833558595 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x56106a7e76f0 con 0x56106a405390 2024-09-21T03:54:15.487 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:15.478+0000 7fd57dffb640 1 -- 172.21.15.39:0/1833558595 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd5880accf0 con 0x56106a405390 2024-09-21T03:54:15.487 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:15.478+0000 7fd57dffb640 1 -- 172.21.15.39:0/1833558595 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fd58c020030 con 0x56106a405390 2024-09-21T03:54:15.487 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:15.478+0000 7fd5a305b740 10 client.17667 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T03:54:15.487 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:15.478+0000 7fd5a305b740 10 client.17667 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T03:54:15.487 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:15.478+0000 7fd5a305b740 20 client.17667 populate_metadata read hostname 'smithi039' 2024-09-21T03:54:15.487 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:15.478+0000 7fd5a305b740 1 --2- 172.21.15.39:0/1833558595 >> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] conn(0x56106a3e49f0 0x56106a7eac90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:54:15.487 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:15.478+0000 7fd5a305b740 4 client.17667 mds_command: new command op to 24695 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:54:15.487 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:15.478+0000 7fd5a305b740 1 -- 172.21.15.39:0/1833558595 --> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56106a745df0 con 0x56106a3e49f0 2024-09-21T03:54:15.487 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:15.478+0000 7fd59659c640 1 --2- 172.21.15.39:0/1833558595 >> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] conn(0x56106a3e49f0 0x56106a7eac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:54:15.487 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:15.479+0000 7fd59659c640 1 --2- 172.21.15.39:0/1833558595 >> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] conn(0x56106a3e49f0 0x56106a7eac90 crc :-1 s=READY pgs=135 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b001941caf422777 server_cookie=ae0bf1506bf2b27d in_seq=0 out_seq=0 2024-09-21T03:54:15.487 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:15.479+0000 7fd57dffb640 10 client.17667 ms_handle_connect on v2:172.21.15.150:6838/2949386907 2024-09-21T03:54:15.487 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:15.479+0000 7fd57dffb640 1 -- 172.21.15.39:0/1833558595 <== mds.0 v2:172.21.15.150:6838/2949386907 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x56106a745df0 con 0x56106a3e49f0 2024-09-21T03:54:15.487 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:15.479+0000 7fd57dffb640 10 client.17667 handle_command_reply: tid=0 2024-09-21T03:54:15.488 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:15.479+0000 7fd5a305b740 1 client.17667 shutdown 2024-09-21T03:54:15.488 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:15.480+0000 7fd5a305b740 1 -- 172.21.15.39:0/1833558595 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56106a405390 msgr2=0x56106a7d8ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:54:15.488 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:15.480+0000 7fd5a305b740 1 --2- 172.21.15.39:0/1833558595 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56106a405390 0x56106a7d8ee0 secure :-1 s=READY pgs=1126 cs=0 l=1 rev1=1 crypto rx=0x7fd58c00bf80 tx=0x7fd58c0118d0 comp rx=0 tx=0).stop 2024-09-21T03:54:15.488 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:15.480+0000 7fd5a305b740 1 -- 172.21.15.39:0/1833558595 shutdown_connections 2024-09-21T03:54:15.488 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:15.480+0000 7fd5a305b740 1 --2- 172.21.15.39:0/1833558595 >> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] conn(0x56106a3e49f0 0x56106a7eac90 crc :-1 s=READY pgs=135 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:15.488 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:15.480+0000 7fd5a305b740 1 --2- 172.21.15.39:0/1833558595 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56106a426070 0x56106a7e4cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:15.488 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:15.480+0000 7fd5a305b740 1 --2- 172.21.15.39:0/1833558595 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56106a405390 0x56106a7d8ee0 unknown :-1 s=CLOSED pgs=1126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:15.488 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:15.480+0000 7fd5a305b740 1 --2- 172.21.15.39:0/1833558595 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56106a3fd1b0 0x56106a7d89a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:15.488 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:15.480+0000 7fd5a305b740 1 -- 172.21.15.39:0/1833558595 >> 172.21.15.39:0/1833558595 conn(0x56106a41a460 msgr2=0x56106a7ce2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:54:15.488 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:15.480+0000 7fd57dffb640 0 client.17667 ms_handle_reset on v2:172.21.15.150:6838/2949386907 2024-09-21T03:54:15.488 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:15.480+0000 7fd5a305b740 1 -- 172.21.15.39:0/1833558595 shutdown_connections 2024-09-21T03:54:15.488 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:15.480+0000 7fd5a305b740 1 -- 172.21.15.39:0/1833558595 wait complete. 2024-09-21T03:54:15.488 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:15.480+0000 7fd5a305b740 20 client.17667 trim_cache size 0 max 16384 2024-09-21T03:54:15.496 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:15.493+0000 7fd5a305b740 1 Processor -- start 2024-09-21T03:54:15.496 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:15.493+0000 7fd5a305b740 1 -- start start 2024-09-21T03:54:15.496 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:15.493+0000 7fd5a305b740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56106a3fd1b0 0x56106a7d9ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:54:15.496 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:15.493+0000 7fd5a305b740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56106a405390 0x56106a7ca760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:54:15.496 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:15.493+0000 7fd5a305b740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56106a426070 0x56106a7caca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:54:15.496 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:15.493+0000 7fd5a305b740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fd5981b1a60 con 0x56106a426070 2024-09-21T03:54:15.496 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:15.493+0000 7fd5a305b740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fd5981b35a0 con 0x56106a3fd1b0 2024-09-21T03:54:15.496 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:15.493+0000 7fd5a305b740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fd5981b2130 con 0x56106a405390 2024-09-21T03:54:15.497 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:15.494+0000 7fd596d9d640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56106a426070 0x56106a7caca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:54:15.497 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:15.494+0000 7fd596d9d640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56106a426070 0x56106a7caca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:46246/0 (socket says 172.21.15.39:46246) 2024-09-21T03:54:15.497 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:15.494+0000 7fd596d9d640 1 -- 172.21.15.39:0/3064967686 learned_addr learned my addr 172.21.15.39:0/3064967686 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:54:15.497 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:15.494+0000 7fd59659c640 1 --2- 172.21.15.39:0/3064967686 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56106a3fd1b0 0x56106a7d9ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:54:15.497 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:15.494+0000 7fd595d9b640 1 --2- 172.21.15.39:0/3064967686 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56106a405390 0x56106a7ca760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:54:15.497 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:15.494+0000 7fd596d9d640 1 -- 172.21.15.39:0/3064967686 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56106a405390 msgr2=0x56106a7ca760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:54:15.497 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:15.494+0000 7fd596d9d640 1 --2- 172.21.15.39:0/3064967686 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56106a405390 0x56106a7ca760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:15.497 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:15.494+0000 7fd596d9d640 1 -- 172.21.15.39:0/3064967686 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56106a3fd1b0 msgr2=0x56106a7d9ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:54:15.497 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:15.494+0000 7fd596d9d640 1 --2- 172.21.15.39:0/3064967686 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56106a3fd1b0 0x56106a7d9ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:15.497 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:15.494+0000 7fd596d9d640 1 -- 172.21.15.39:0/3064967686 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56106a7dad70 con 0x56106a426070 2024-09-21T03:54:15.497 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:15.494+0000 7fd595d9b640 1 --2- 172.21.15.39:0/3064967686 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56106a405390 0x56106a7ca760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:54:15.497 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:15.494+0000 7fd59659c640 1 --2- 172.21.15.39:0/3064967686 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56106a3fd1b0 0x56106a7d9ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:54:15.497 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:15.494+0000 7fd596d9d640 1 --2- 172.21.15.39:0/3064967686 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56106a426070 0x56106a7caca0 secure :-1 s=READY pgs=1127 cs=0 l=1 rev1=1 crypto rx=0x7fd590031710 tx=0x7fd590041040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd1e0bef8aced780 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:54:15.497 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:15.494+0000 7fd57ffff640 1 -- 172.21.15.39:0/3064967686 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd590051070 con 0x56106a426070 2024-09-21T03:54:15.498 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:15.494+0000 7fd57ffff640 1 -- 172.21.15.39:0/3064967686 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd590036030 con 0x56106a426070 2024-09-21T03:54:15.498 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:15.494+0000 7fd57ffff640 1 -- 172.21.15.39:0/3064967686 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd59002f950 con 0x56106a426070 2024-09-21T03:54:15.501 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:15.495+0000 7fd5a305b740 1 -- 172.21.15.39:0/3064967686 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56106a426070 msgr2=0x56106a7caca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:54:15.501 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:15.495+0000 7fd5a305b740 1 --2- 172.21.15.39:0/3064967686 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56106a426070 0x56106a7caca0 secure :-1 s=READY pgs=1127 cs=0 l=1 rev1=1 crypto rx=0x7fd590031710 tx=0x7fd590041040 comp rx=0 tx=0).stop 2024-09-21T03:54:15.501 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:15.495+0000 7fd5a305b740 1 -- 172.21.15.39:0/3064967686 shutdown_connections 2024-09-21T03:54:15.501 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:15.495+0000 7fd5a305b740 1 --2- 172.21.15.39:0/3064967686 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56106a426070 0x56106a7caca0 unknown :-1 s=CLOSED pgs=1127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:15.501 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:15.495+0000 7fd5a305b740 1 --2- 172.21.15.39:0/3064967686 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56106a405390 0x56106a7ca760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:15.501 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:15.495+0000 7fd5a305b740 1 --2- 172.21.15.39:0/3064967686 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56106a3fd1b0 0x56106a7d9ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:15.502 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:15.495+0000 7fd5a305b740 1 -- 172.21.15.39:0/3064967686 >> 172.21.15.39:0/3064967686 conn(0x56106a41a460 msgr2=0x56106a7cecb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:54:15.502 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:15.495+0000 7fd5a305b740 1 -- 172.21.15.39:0/3064967686 shutdown_connections 2024-09-21T03:54:15.502 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:15.495+0000 7fd5a305b740 1 -- 172.21.15.39:0/3064967686 wait complete. 2024-09-21T03:54:15.502 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:15.496+0000 7fd5a305b740 1 -- start start 2024-09-21T03:54:15.502 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:15.496+0000 7fd5a305b740 1 Processor -- start 2024-09-21T03:54:15.502 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:15.496+0000 7fd5a305b740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56106a3fd1b0 0x56106a7e1580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:54:15.502 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:15.496+0000 7fd5a305b740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56106a405390 0x56106a7dca30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:54:15.502 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:15.496+0000 7fd5a305b740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56106a426070 0x56106a7dd0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:54:15.502 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:15.496+0000 7fd5a305b740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fd5981b2ed0 con 0x56106a426070 2024-09-21T03:54:15.502 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:15.496+0000 7fd5a305b740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fd5981b2800 con 0x56106a405390 2024-09-21T03:54:15.502 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:15.496+0000 7fd5a305b740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fd5981afa60 con 0x56106a3fd1b0 2024-09-21T03:54:15.502 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:15.496+0000 7fd5a305b740 10 client.0 _ll_register_callbacks cb 0x7fd58c00bb90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:54:15.502 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:15.496+0000 7fd596d9d640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56106a426070 0x56106a7dd0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:54:15.502 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:15.496+0000 7fd596d9d640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56106a426070 0x56106a7dd0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:46260/0 (socket says 172.21.15.39:46260) 2024-09-21T03:54:15.503 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:15.496+0000 7fd596d9d640 1 -- 172.21.15.39:0/1306009739 learned_addr learned my addr 172.21.15.39:0/1306009739 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:54:15.503 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:15.496+0000 7fd59659c640 1 --2- 172.21.15.39:0/1306009739 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56106a3fd1b0 0x56106a7e1580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:54:15.503 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:15.496+0000 7fd595d9b640 1 --2- 172.21.15.39:0/1306009739 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56106a405390 0x56106a7dca30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:54:15.503 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:15.496+0000 7fd596d9d640 1 -- 172.21.15.39:0/1306009739 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56106a3fd1b0 msgr2=0x56106a7e1580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:54:15.503 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:15.496+0000 7fd596d9d640 1 --2- 172.21.15.39:0/1306009739 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56106a3fd1b0 0x56106a7e1580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:15.503 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:15.496+0000 7fd596d9d640 1 -- 172.21.15.39:0/1306009739 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56106a405390 msgr2=0x56106a7dca30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:54:15.503 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:15.497+0000 7fd596d9d640 1 --2- 172.21.15.39:0/1306009739 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56106a405390 0x56106a7dca30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:15.503 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:15.497+0000 7fd596d9d640 1 -- 172.21.15.39:0/1306009739 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56106a7dd860 con 0x56106a426070 2024-09-21T03:54:15.503 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:15.497+0000 7fd595d9b640 1 --2- 172.21.15.39:0/1306009739 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56106a405390 0x56106a7dca30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:54:15.503 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:15.497+0000 7fd596d9d640 1 --2- 172.21.15.39:0/1306009739 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56106a426070 0x56106a7dd0d0 secure :-1 s=READY pgs=1128 cs=0 l=1 rev1=1 crypto rx=0x7fd590009ca0 tx=0x7fd5900047d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e0d21a0a00125da3 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:54:15.503 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:15.497+0000 7fd57dffb640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-09-21T03:54:15.503 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:15.497+0000 7fd57dffb640 1 -- 172.21.15.39:0/1306009739 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd590051030 con 0x56106a426070 2024-09-21T03:54:15.503 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:15.497+0000 7fd57dffb640 1 -- 172.21.15.39:0/1306009739 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8572+0+0 (secure 0 0 0) 0x7fd59002f860 con 0x56106a426070 2024-09-21T03:54:15.504 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:15.497+0000 7fd5a305b740 1 -- 172.21.15.39:0/1306009739 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd590036030 con 0x56106a426070 2024-09-21T03:54:15.504 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:15.497+0000 7fd57dffb640 1 -- 172.21.15.39:0/1306009739 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fd590064050 con 0x56106a426070 2024-09-21T03:54:15.504 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:15.497+0000 7fd5a305b740 10 client.17673 fetch_fsmap learned FSMap version 29 2024-09-21T03:54:15.504 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:15.497+0000 7fd5a305b740 1 -- 172.21.15.39:0/1306009739 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x56106a855700 con 0x56106a426070 2024-09-21T03:54:15.504 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:15.498+0000 7fd57dffb640 1 -- 172.21.15.39:0/1306009739 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd590036030 con 0x56106a426070 2024-09-21T03:54:15.504 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:15.498+0000 7fd57dffb640 1 -- 172.21.15.39:0/1306009739 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fd590030a70 con 0x56106a426070 2024-09-21T03:54:15.504 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:15.498+0000 7fd5a305b740 10 client.17673 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T03:54:15.504 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:15.498+0000 7fd5a305b740 10 client.17673 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T03:54:15.504 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:15.498+0000 7fd5a305b740 20 client.17673 populate_metadata read hostname 'smithi039' 2024-09-21T03:54:15.504 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:15.498+0000 7fd5a305b740 1 --2- 172.21.15.39:0/1306009739 >> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] conn(0x56106a3b9da0 0x56106a857e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:54:15.504 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:15.498+0000 7fd5a305b740 4 client.17673 mds_command: new command op to 24695 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T03:54:15.504 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:15.498+0000 7fd5a305b740 1 -- 172.21.15.39:0/1306009739 --> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56106a7418a0 con 0x56106a3b9da0 2024-09-21T03:54:15.504 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:15.498+0000 7fd59659c640 1 --2- 172.21.15.39:0/1306009739 >> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] conn(0x56106a3b9da0 0x56106a857e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:54:15.505 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:15.498+0000 7fd59659c640 1 --2- 172.21.15.39:0/1306009739 >> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] conn(0x56106a3b9da0 0x56106a857e60 crc :-1 s=READY pgs=136 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=53549cd362b6ebb6 server_cookie=c52e8bb4d0f51a6c in_seq=0 out_seq=0 2024-09-21T03:54:15.505 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:15.498+0000 7fd57dffb640 10 client.17673 ms_handle_connect on v2:172.21.15.150:6838/2949386907 2024-09-21T03:54:15.505 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:15.499+0000 7fd57dffb640 1 -- 172.21.15.39:0/1306009739 <== mds.0 v2:172.21.15.150:6838/2949386907 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56106a7418a0 con 0x56106a3b9da0 2024-09-21T03:54:15.505 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:15.499+0000 7fd57dffb640 10 client.17673 handle_command_reply: tid=0 2024-09-21T03:54:15.505 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:15.499+0000 7fd5a305b740 1 client.17673 shutdown 2024-09-21T03:54:15.505 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:15.499+0000 7fd5a305b740 1 -- 172.21.15.39:0/1306009739 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56106a426070 msgr2=0x56106a7dd0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:54:15.505 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:15.499+0000 7fd5a305b740 1 --2- 172.21.15.39:0/1306009739 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56106a426070 0x56106a7dd0d0 secure :-1 s=READY pgs=1128 cs=0 l=1 rev1=1 crypto rx=0x7fd590009ca0 tx=0x7fd5900047d0 comp rx=0 tx=0).stop 2024-09-21T03:54:15.505 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:15.499+0000 7fd5a305b740 1 -- 172.21.15.39:0/1306009739 shutdown_connections 2024-09-21T03:54:15.505 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:15.499+0000 7fd5a305b740 1 --2- 172.21.15.39:0/1306009739 >> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] conn(0x56106a3b9da0 0x56106a857e60 crc :-1 s=READY pgs=136 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:15.505 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:15.499+0000 7fd57dffb640 0 client.17673 ms_handle_reset on v2:172.21.15.150:6838/2949386907 2024-09-21T03:54:15.505 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:15.499+0000 7fd5a305b740 1 --2- 172.21.15.39:0/1306009739 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56106a426070 0x56106a7dd0d0 unknown :-1 s=CLOSED pgs=1128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:15.505 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:15.499+0000 7fd5a305b740 1 --2- 172.21.15.39:0/1306009739 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56106a405390 0x56106a7dca30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:15.506 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:15.499+0000 7fd5a305b740 1 --2- 172.21.15.39:0/1306009739 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56106a3fd1b0 0x56106a7e1580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:15.506 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:15.499+0000 7fd5a305b740 1 -- 172.21.15.39:0/1306009739 >> 172.21.15.39:0/1306009739 conn(0x56106a41a460 msgr2=0x56106a7cecb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:54:15.506 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:15.499+0000 7fd5a305b740 1 -- 172.21.15.39:0/1306009739 shutdown_connections 2024-09-21T03:54:15.506 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:15.499+0000 7fd5a305b740 1 -- 172.21.15.39:0/1306009739 wait complete. 2024-09-21T03:54:15.506 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:15.499+0000 7fd5a305b740 20 client.17673 trim_cache size 0 max 16384 2024-09-21T03:54:15.506 INFO:teuthology.orchestra.run.smithi039.stdout:[] 2024-09-21T03:54:15.507 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:15.500+0000 7fd59cae7640 1 -- 172.21.15.39:0/2536452964 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7fd5800b6680 msgr2=0x7fd5800b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:54:15.507 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:15.500+0000 7fd59cae7640 1 --2- 172.21.15.39:0/2536452964 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7fd5800b6680 0x7fd5800b8b90 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7fd590004af0 tx=0x7fd59002a8c0 comp rx=0 tx=0).stop 2024-09-21T03:54:15.507 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:15.500+0000 7fd59cae7640 1 -- 172.21.15.39:0/2536452964 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd598102ae0 msgr2=0x7fd5980ff670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:54:15.507 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:15.500+0000 7fd59cae7640 1 --2- 172.21.15.39:0/2536452964 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd598102ae0 0x7fd5980ff670 secure :-1 s=READY pgs=1124 cs=0 l=1 rev1=1 crypto rx=0x7fd58800d950 tx=0x7fd58800de10 comp rx=0 tx=0).stop 2024-09-21T03:54:15.507 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:15.500+0000 7fd59cae7640 1 -- 172.21.15.39:0/2536452964 shutdown_connections 2024-09-21T03:54:15.507 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:15.500+0000 7fd59cae7640 1 --2- 172.21.15.39:0/2536452964 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7fd5800b6680 0x7fd5800b8b90 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:15.508 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:15.500+0000 7fd59cae7640 1 --2- 172.21.15.39:0/2536452964 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd598108d20 0x7fd5981000f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:15.508 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:15.500+0000 7fd59cae7640 1 --2- 172.21.15.39:0/2536452964 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd5981083d0 0x7fd5980ffbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:15.508 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:15.500+0000 7fd59cae7640 1 --2- 172.21.15.39:0/2536452964 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd598102ae0 0x7fd5980ff670 unknown :-1 s=CLOSED pgs=1124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:15.508 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:15.500+0000 7fd59cae7640 1 -- 172.21.15.39:0/2536452964 >> 172.21.15.39:0/2536452964 conn(0x7fd598079760 msgr2=0x7fd59806be80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:54:15.508 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:15.500+0000 7fd59cae7640 1 -- 172.21.15.39:0/2536452964 shutdown_connections 2024-09-21T03:54:15.508 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:15.500+0000 7fd59cae7640 1 -- 172.21.15.39:0/2536452964 wait complete. 2024-09-21T03:54:17.334 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:54:16 smithi039 ceph-mon[31211]: pgmap v575: 97 pgs: 97 active+clean; 10 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-09-21T03:54:17.422 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-21T03:54:17.422 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 51116d5c-77ca-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-21T03:54:17.468 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:54:16 smithi161 ceph-mon[34533]: pgmap v575: 97 pgs: 97 active+clean; 10 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-09-21T03:54:17.475 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:54:16 smithi150 ceph-mon[34289]: pgmap v575: 97 pgs: 97 active+clean; 10 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-09-21T03:54:17.693 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/51116d5c-77ca-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:54:18.235 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:18.184+0000 7fbee7538640 1 -- 172.21.15.39:0/2547532495 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbee0110110 msgr2=0x7fbee0112550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:54:18.235 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:18.184+0000 7fbee7538640 1 --2- 172.21.15.39:0/2547532495 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbee0110110 0x7fbee0112550 secure :-1 s=READY pgs=1129 cs=0 l=1 rev1=1 crypto rx=0x7fbed8009ca0 tx=0x7fbed802a8c0 comp rx=0 tx=0).stop 2024-09-21T03:54:18.235 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:18.184+0000 7fbee7538640 1 -- 172.21.15.39:0/2547532495 shutdown_connections 2024-09-21T03:54:18.235 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:18.184+0000 7fbee7538640 1 --2- 172.21.15.39:0/2547532495 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbee0110110 0x7fbee0112550 unknown :-1 s=CLOSED pgs=1129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:18.235 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:18.184+0000 7fbee7538640 1 --2- 172.21.15.39:0/2547532495 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbee0076100 0x7fbee010fbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:18.235 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:18.184+0000 7fbee7538640 1 --2- 172.21.15.39:0/2547532495 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbee00757b0 0x7fbee0075bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:18.235 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:18.185+0000 7fbee7538640 1 -- 172.21.15.39:0/2547532495 >> 172.21.15.39:0/2547532495 conn(0x7fbee00ff600 msgr2=0x7fbee0101a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:54:18.235 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:18.185+0000 7fbee7538640 1 -- 172.21.15.39:0/2547532495 shutdown_connections 2024-09-21T03:54:18.236 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:18.185+0000 7fbee7538640 1 -- 172.21.15.39:0/2547532495 wait complete. 2024-09-21T03:54:18.236 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:18.185+0000 7fbee7538640 1 Processor -- start 2024-09-21T03:54:18.236 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:18.185+0000 7fbee7538640 1 -- start start 2024-09-21T03:54:18.236 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:18.186+0000 7fbee7538640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbee00757b0 0x7fbee019c6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:54:18.236 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:18.186+0000 7fbee7538640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbee0076100 0x7fbee019cc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:54:18.236 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:18.186+0000 7fbee7538640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbee0110110 0x7fbee01a1ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:54:18.236 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:18.186+0000 7fbee7538640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fbee0115cd0 con 0x7fbee00757b0 2024-09-21T03:54:18.236 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:18.186+0000 7fbee7538640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fbee0115b50 con 0x7fbee0076100 2024-09-21T03:54:18.236 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:18.186+0000 7fbee7538640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fbee0115e50 con 0x7fbee0110110 2024-09-21T03:54:18.236 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:18.186+0000 7fbee52d4640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbee00757b0 0x7fbee019c6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:54:18.236 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:18.186+0000 7fbee52d4640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbee00757b0 0x7fbee019c6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:58416/0 (socket says 172.21.15.39:58416) 2024-09-21T03:54:18.236 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:18.186+0000 7fbee52d4640 1 -- 172.21.15.39:0/853742053 learned_addr learned my addr 172.21.15.39:0/853742053 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:54:18.236 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:18.186+0000 7fbee4ad3640 1 --2- 172.21.15.39:0/853742053 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbee0076100 0x7fbee019cc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:54:18.236 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:18.186+0000 7fbee5ad5640 1 --2- 172.21.15.39:0/853742053 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbee0110110 0x7fbee01a1ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:54:18.237 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:18.186+0000 7fbee52d4640 1 -- 172.21.15.39:0/853742053 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbee0110110 msgr2=0x7fbee01a1ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:54:18.237 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:18.186+0000 7fbee52d4640 1 --2- 172.21.15.39:0/853742053 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbee0110110 0x7fbee01a1ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:18.237 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:18.186+0000 7fbee52d4640 1 -- 172.21.15.39:0/853742053 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbee0076100 msgr2=0x7fbee019cc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:54:18.237 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:18.186+0000 7fbee52d4640 1 --2- 172.21.15.39:0/853742053 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbee0076100 0x7fbee019cc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:18.237 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:18.186+0000 7fbee52d4640 1 -- 172.21.15.39:0/853742053 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbee01a2610 con 0x7fbee00757b0 2024-09-21T03:54:18.237 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:18.186+0000 7fbee52d4640 1 --2- 172.21.15.39:0/853742053 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbee00757b0 0x7fbee019c6a0 secure :-1 s=READY pgs=1130 cs=0 l=1 rev1=1 crypto rx=0x7fbedc00d950 tx=0x7fbedc00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:54:18.237 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:18.186+0000 7fbed67fc640 1 -- 172.21.15.39:0/853742053 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbedc017070 con 0x7fbee00757b0 2024-09-21T03:54:18.237 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:18.187+0000 7fbee7538640 1 -- 172.21.15.39:0/853742053 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbee01a2900 con 0x7fbee00757b0 2024-09-21T03:54:18.237 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:18.187+0000 7fbed67fc640 1 -- 172.21.15.39:0/853742053 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbedc009c30 con 0x7fbee00757b0 2024-09-21T03:54:18.237 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:18.187+0000 7fbed67fc640 1 -- 172.21.15.39:0/853742053 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbedc00f620 con 0x7fbee00757b0 2024-09-21T03:54:18.237 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:18.187+0000 7fbee7538640 1 -- 172.21.15.39:0/853742053 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbee01a9060 con 0x7fbee00757b0 2024-09-21T03:54:18.237 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:18.190+0000 7fbeedaac740 1 Processor -- start 2024-09-21T03:54:18.237 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:18.190+0000 7fbeedaac740 1 -- start start 2024-09-21T03:54:18.237 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:18.190+0000 7fbed67fc640 1 -- 172.21.15.39:0/853742053 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 27) ==== 154643+0+0 (secure 0 0 0) 0x7fbedc010460 con 0x7fbee00757b0 2024-09-21T03:54:18.238 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:18.190+0000 7fbeedaac740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563ddd189350 0x563ddd530430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:54:18.238 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:18.190+0000 7fbeedaac740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563ddd160490 0x563ddd530970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:54:18.238 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:18.190+0000 7fbeedaac740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563ddd168670 0x563ddd530eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:54:18.238 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:18.190+0000 7fbeedaac740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x563ddd4a6e60 con 0x563ddd160490 2024-09-21T03:54:18.238 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:18.190+0000 7fbeedaac740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x563ddd496290 con 0x563ddd189350 2024-09-21T03:54:18.238 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:18.190+0000 7fbeedaac740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x563ddd49a0a0 con 0x563ddd168670 2024-09-21T03:54:18.238 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:18.190+0000 7fbee52d4640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563ddd160490 0x563ddd530970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:54:18.238 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:18.190+0000 7fbee52d4640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563ddd160490 0x563ddd530970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:58424/0 (socket says 172.21.15.39:58424) 2024-09-21T03:54:18.238 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:18.190+0000 7fbee52d4640 1 -- 172.21.15.39:0/147087660 learned_addr learned my addr 172.21.15.39:0/147087660 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:54:18.238 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:18.190+0000 7fbee5ad5640 1 --2- 172.21.15.39:0/147087660 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563ddd189350 0x563ddd530430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:54:18.238 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:18.190+0000 7fbee4ad3640 1 --2- 172.21.15.39:0/147087660 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563ddd168670 0x563ddd530eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:54:18.238 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:18.190+0000 7fbee5ad5640 1 -- 172.21.15.39:0/147087660 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563ddd168670 msgr2=0x563ddd530eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:54:18.238 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:18.190+0000 7fbee5ad5640 1 --2- 172.21.15.39:0/147087660 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563ddd168670 0x563ddd530eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:18.238 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:18.190+0000 7fbee5ad5640 1 -- 172.21.15.39:0/147087660 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563ddd160490 msgr2=0x563ddd530970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:54:18.239 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:18.190+0000 7fbee5ad5640 1 --2- 172.21.15.39:0/147087660 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563ddd160490 0x563ddd530970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:18.239 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:18.190+0000 7fbee5ad5640 1 -- 172.21.15.39:0/147087660 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563ddd1f4930 con 0x563ddd189350 2024-09-21T03:54:18.239 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:18.191+0000 7fbee52d4640 1 --2- 172.21.15.39:0/147087660 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563ddd160490 0x563ddd530970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:54:18.239 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:18.191+0000 7fbee5ad5640 1 --2- 172.21.15.39:0/147087660 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563ddd189350 0x563ddd530430 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fbed8005e30 tx=0x7fbed802f3c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=66f38aa4625a764 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:54:18.239 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:18.191+0000 7fbed67fc640 1 --2- 172.21.15.39:0/853742053 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7fbecc0b6680 0x7fbecc0b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:54:18.239 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:18.191+0000 7fbecb7fe640 1 -- 172.21.15.39:0/147087660 <== mon.2 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbed8031e30 con 0x563ddd189350 2024-09-21T03:54:18.239 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:18.191+0000 7fbee52d4640 1 --2- 172.21.15.39:0/853742053 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7fbecc0b6680 0x7fbecc0b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:54:18.239 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:18.191+0000 7fbecb7fe640 1 -- 172.21.15.39:0/147087660 <== mon.2 v2:172.21.15.150:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbed8036030 con 0x563ddd189350 2024-09-21T03:54:18.239 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:18.191+0000 7fbecb7fe640 1 -- 172.21.15.39:0/147087660 <== mon.2 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbed8004330 con 0x563ddd189350 2024-09-21T03:54:18.239 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:18.191+0000 7fbed67fc640 1 -- 172.21.15.39:0/853742053 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8572+0+0 (secure 0 0 0) 0x7fbedc0e7310 con 0x7fbee00757b0 2024-09-21T03:54:18.239 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:18.192+0000 7fbeedaac740 1 -- 172.21.15.39:0/147087660 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563ddd189350 msgr2=0x563ddd530430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:54:18.239 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:18.192+0000 7fbee52d4640 1 --2- 172.21.15.39:0/853742053 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7fbecc0b6680 0x7fbecc0b8b90 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7fbedc0a97b0 tx=0x7fbedc0b4f90 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:54:18.239 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:18.192+0000 7fbeedaac740 1 --2- 172.21.15.39:0/147087660 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563ddd189350 0x563ddd530430 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fbed8005e30 tx=0x7fbed802f3c0 comp rx=0 tx=0).stop 2024-09-21T03:54:18.239 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:18.192+0000 7fbeedaac740 1 -- 172.21.15.39:0/147087660 shutdown_connections 2024-09-21T03:54:18.239 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:18.192+0000 7fbeedaac740 1 --2- 172.21.15.39:0/147087660 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563ddd168670 0x563ddd530eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:18.240 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:18.192+0000 7fbeedaac740 1 --2- 172.21.15.39:0/147087660 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563ddd160490 0x563ddd530970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:18.240 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:18.192+0000 7fbeedaac740 1 --2- 172.21.15.39:0/147087660 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563ddd189350 0x563ddd530430 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:18.240 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:18.192+0000 7fbeedaac740 1 -- 172.21.15.39:0/147087660 >> 172.21.15.39:0/147087660 conn(0x563ddd17d740 msgr2=0x563ddd52e320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:54:18.240 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:18.192+0000 7fbeedaac740 1 -- 172.21.15.39:0/147087660 shutdown_connections 2024-09-21T03:54:18.240 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:18.192+0000 7fbeedaac740 1 -- 172.21.15.39:0/147087660 wait complete. 2024-09-21T03:54:18.240 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:18.192+0000 7fbeedaac740 1 -- start start 2024-09-21T03:54:18.240 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:18.193+0000 7fbeedaac740 1 Processor -- start 2024-09-21T03:54:18.240 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:18.193+0000 7fbeedaac740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563ddd160490 0x563ddd53c0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:54:18.240 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:18.193+0000 7fbeedaac740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563ddd168670 0x563ddd53c610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:54:18.240 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:18.193+0000 7fbeedaac740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563ddd189350 0x563ddd547d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:54:18.240 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:18.193+0000 7fbeedaac740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x563ddd2f7890 con 0x563ddd168670 2024-09-21T03:54:18.240 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:18.193+0000 7fbeedaac740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x563ddd430d40 con 0x563ddd189350 2024-09-21T03:54:18.240 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:18.193+0000 7fbeedaac740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x563ddd53ea30 con 0x563ddd160490 2024-09-21T03:54:18.240 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:18.193+0000 7fbeedaac740 10 client.0 _ll_register_callbacks cb 0x563ddd494c80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:54:18.241 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:18.193+0000 7fbee4ad3640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563ddd168670 0x563ddd53c610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:54:18.241 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:18.193+0000 7fbee4ad3640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563ddd168670 0x563ddd53c610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:58430/0 (socket says 172.21.15.39:58430) 2024-09-21T03:54:18.241 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:18.193+0000 7fbee4ad3640 1 -- 172.21.15.39:0/2558901942 learned_addr learned my addr 172.21.15.39:0/2558901942 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:54:18.241 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:18.193+0000 7fbee5ad5640 1 --2- 172.21.15.39:0/2558901942 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563ddd189350 0x563ddd547d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:54:18.241 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:18.193+0000 7fbee5ad5640 1 --2- 172.21.15.39:0/2558901942 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563ddd160490 0x563ddd53c0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:54:18.241 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:18.194+0000 7fbee4ad3640 1 -- 172.21.15.39:0/2558901942 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563ddd160490 msgr2=0x563ddd53c0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:54:18.241 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:18.194+0000 7fbee4ad3640 1 --2- 172.21.15.39:0/2558901942 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563ddd160490 0x563ddd53c0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:18.241 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:18.194+0000 7fbee4ad3640 1 -- 172.21.15.39:0/2558901942 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563ddd189350 msgr2=0x563ddd547d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:54:18.241 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:18.194+0000 7fbee4ad3640 1 --2- 172.21.15.39:0/2558901942 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563ddd189350 0x563ddd547d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:18.241 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:18.194+0000 7fbee4ad3640 1 -- 172.21.15.39:0/2558901942 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563ddd548340 con 0x563ddd168670 2024-09-21T03:54:18.241 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:18.194+0000 7fbee4ad3640 1 --2- 172.21.15.39:0/2558901942 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563ddd168670 0x563ddd53c610 secure :-1 s=READY pgs=1131 cs=0 l=1 rev1=1 crypto rx=0x7fbed000b9f0 tx=0x7fbed000beb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a04b92a50205d41 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:54:18.241 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:18.194+0000 7fbec8ff9640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-09-21T03:54:18.241 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:18.194+0000 7fbec8ff9640 1 -- 172.21.15.39:0/2558901942 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbed0003e20 con 0x563ddd168670 2024-09-21T03:54:18.241 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:18.194+0000 7fbec8ff9640 1 -- 172.21.15.39:0/2558901942 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8572+0+0 (secure 0 0 0) 0x7fbed0005020 con 0x563ddd168670 2024-09-21T03:54:18.241 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:18.194+0000 7fbeedaac740 1 -- 172.21.15.39:0/2558901942 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbed8036030 con 0x563ddd168670 2024-09-21T03:54:18.242 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:18.194+0000 7fbec8ff9640 1 -- 172.21.15.39:0/2558901942 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fbed0005220 con 0x563ddd168670 2024-09-21T03:54:18.242 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:18.194+0000 7fbeedaac740 10 client.17685 fetch_fsmap learned FSMap version 29 2024-09-21T03:54:18.242 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:18.194+0000 7fbeedaac740 1 -- 172.21.15.39:0/2558901942 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x563ddd54acb0 con 0x563ddd168670 2024-09-21T03:54:18.242 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:18.194+0000 7fbec8ff9640 1 -- 172.21.15.39:0/2558901942 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbed8036030 con 0x563ddd168670 2024-09-21T03:54:18.242 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:18.195+0000 7fbec8ff9640 1 -- 172.21.15.39:0/2558901942 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fbed0022070 con 0x563ddd168670 2024-09-21T03:54:18.242 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:18.195+0000 7fbeedaac740 10 client.17685 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T03:54:18.242 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:18.195+0000 7fbeedaac740 10 client.17685 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T03:54:18.242 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:18.195+0000 7fbeedaac740 20 client.17685 populate_metadata read hostname 'smithi039' 2024-09-21T03:54:18.242 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:18.195+0000 7fbeedaac740 1 --2- 172.21.15.39:0/2558901942 >> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] conn(0x563ddd147cd0 0x563ddd54e220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:54:18.242 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:18.195+0000 7fbeedaac740 4 client.17685 mds_command: new command op to 24695 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:54:18.242 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:18.195+0000 7fbeedaac740 1 -- 172.21.15.39:0/2558901942 --> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563ddd4a8db0 con 0x563ddd147cd0 2024-09-21T03:54:18.242 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:18.195+0000 7fbee5ad5640 1 --2- 172.21.15.39:0/2558901942 >> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] conn(0x563ddd147cd0 0x563ddd54e220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:54:18.242 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:18.195+0000 7fbee5ad5640 1 --2- 172.21.15.39:0/2558901942 >> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] conn(0x563ddd147cd0 0x563ddd54e220 crc :-1 s=READY pgs=137 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2fb995b643425e82 server_cookie=f0ed6dd323d88142 in_seq=0 out_seq=0 2024-09-21T03:54:18.242 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:18.195+0000 7fbec8ff9640 10 client.17685 ms_handle_connect on v2:172.21.15.150:6838/2949386907 2024-09-21T03:54:18.243 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:18.196+0000 7fbec8ff9640 1 -- 172.21.15.39:0/2558901942 <== mds.0 v2:172.21.15.150:6838/2949386907 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x563ddd4a8db0 con 0x563ddd147cd0 2024-09-21T03:54:18.243 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:18.196+0000 7fbec8ff9640 10 client.17685 handle_command_reply: tid=0 2024-09-21T03:54:18.243 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:18.196+0000 7fbeedaac740 1 client.17685 shutdown 2024-09-21T03:54:18.243 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:18.197+0000 7fbeedaac740 1 -- 172.21.15.39:0/2558901942 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563ddd168670 msgr2=0x563ddd53c610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:54:18.243 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:18.197+0000 7fbeedaac740 1 --2- 172.21.15.39:0/2558901942 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563ddd168670 0x563ddd53c610 secure :-1 s=READY pgs=1131 cs=0 l=1 rev1=1 crypto rx=0x7fbed000b9f0 tx=0x7fbed000beb0 comp rx=0 tx=0).stop 2024-09-21T03:54:18.243 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:18.197+0000 7fbeedaac740 1 -- 172.21.15.39:0/2558901942 shutdown_connections 2024-09-21T03:54:18.243 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:18.197+0000 7fbeedaac740 1 --2- 172.21.15.39:0/2558901942 >> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] conn(0x563ddd147cd0 0x563ddd54e220 crc :-1 s=READY pgs=137 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:18.243 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:18.197+0000 7fbeedaac740 1 --2- 172.21.15.39:0/2558901942 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563ddd189350 0x563ddd547d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:18.243 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:18.197+0000 7fbec8ff9640 0 client.17685 ms_handle_reset on v2:172.21.15.150:6838/2949386907 2024-09-21T03:54:18.243 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:18.197+0000 7fbeedaac740 1 --2- 172.21.15.39:0/2558901942 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563ddd168670 0x563ddd53c610 unknown :-1 s=CLOSED pgs=1131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:18.243 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:18.197+0000 7fbeedaac740 1 --2- 172.21.15.39:0/2558901942 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563ddd160490 0x563ddd53c0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:18.243 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:18.197+0000 7fbeedaac740 1 -- 172.21.15.39:0/2558901942 >> 172.21.15.39:0/2558901942 conn(0x563ddd17d740 msgr2=0x563ddd5335c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:54:18.243 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:18.197+0000 7fbeedaac740 1 -- 172.21.15.39:0/2558901942 shutdown_connections 2024-09-21T03:54:18.243 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:18.197+0000 7fbeedaac740 1 -- 172.21.15.39:0/2558901942 wait complete. 2024-09-21T03:54:18.244 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:18.197+0000 7fbeedaac740 20 client.17685 trim_cache size 0 max 16384 2024-09-21T03:54:18.244 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:18.215+0000 7fbeedaac740 1 Processor -- start 2024-09-21T03:54:18.244 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:18.215+0000 7fbeedaac740 1 -- start start 2024-09-21T03:54:18.244 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:18.215+0000 7fbeedaac740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563ddd160490 0x563ddd537680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:54:18.244 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:18.215+0000 7fbeedaac740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563ddd168670 0x563ddd53b760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:54:18.244 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:18.215+0000 7fbeedaac740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563ddd189350 0x563ddd531b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:54:18.244 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:18.215+0000 7fbeedaac740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fbee01b1a60 con 0x563ddd168670 2024-09-21T03:54:18.244 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:18.215+0000 7fbeedaac740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fbee01b35a0 con 0x563ddd189350 2024-09-21T03:54:18.244 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:18.215+0000 7fbeedaac740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fbee01b2130 con 0x563ddd160490 2024-09-21T03:54:18.244 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:18.216+0000 7fbee4ad3640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563ddd168670 0x563ddd53b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:54:18.244 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:18.216+0000 7fbee4ad3640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563ddd168670 0x563ddd53b760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:58440/0 (socket says 172.21.15.39:58440) 2024-09-21T03:54:18.244 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:18.216+0000 7fbee4ad3640 1 -- 172.21.15.39:0/2230693671 learned_addr learned my addr 172.21.15.39:0/2230693671 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:54:18.244 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:18.216+0000 7fbee5ad5640 1 --2- 172.21.15.39:0/2230693671 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563ddd189350 0x563ddd531b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:54:18.244 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:18.216+0000 7fbee4ad3640 1 -- 172.21.15.39:0/2230693671 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563ddd160490 msgr2=0x563ddd537680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:54:18.244 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:18.216+0000 7fbee4ad3640 1 --2- 172.21.15.39:0/2230693671 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563ddd160490 0x563ddd537680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:18.245 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:18.216+0000 7fbee4ad3640 1 -- 172.21.15.39:0/2230693671 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563ddd189350 msgr2=0x563ddd531b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:54:18.245 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:18.216+0000 7fbee4ad3640 1 --2- 172.21.15.39:0/2230693671 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563ddd189350 0x563ddd531b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:18.245 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:18.216+0000 7fbee4ad3640 1 -- 172.21.15.39:0/2230693671 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563ddd532720 con 0x563ddd168670 2024-09-21T03:54:18.245 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:18.216+0000 7fbee5ad5640 1 --2- 172.21.15.39:0/2230693671 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563ddd189350 0x563ddd531b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:54:18.245 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:18.216+0000 7fbee4ad3640 1 --2- 172.21.15.39:0/2230693671 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563ddd168670 0x563ddd53b760 secure :-1 s=READY pgs=1132 cs=0 l=1 rev1=1 crypto rx=0x7fbed001b650 tx=0x7fbed0003a80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2797f848b5fb8a5e server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:54:18.245 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:18.216+0000 7fbecaffd640 1 -- 172.21.15.39:0/2230693671 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbed0028040 con 0x563ddd168670 2024-09-21T03:54:18.245 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:18.216+0000 7fbecaffd640 1 -- 172.21.15.39:0/2230693671 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbed001b840 con 0x563ddd168670 2024-09-21T03:54:18.245 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:18.216+0000 7fbecaffd640 1 -- 172.21.15.39:0/2230693671 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbed0024940 con 0x563ddd168670 2024-09-21T03:54:18.245 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:18.217+0000 7fbeedaac740 1 -- 172.21.15.39:0/2230693671 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563ddd168670 msgr2=0x563ddd53b760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:54:18.245 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:18.217+0000 7fbeedaac740 1 --2- 172.21.15.39:0/2230693671 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563ddd168670 0x563ddd53b760 secure :-1 s=READY pgs=1132 cs=0 l=1 rev1=1 crypto rx=0x7fbed001b650 tx=0x7fbed0003a80 comp rx=0 tx=0).stop 2024-09-21T03:54:18.245 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:18.217+0000 7fbeedaac740 1 -- 172.21.15.39:0/2230693671 shutdown_connections 2024-09-21T03:54:18.245 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:18.217+0000 7fbeedaac740 1 --2- 172.21.15.39:0/2230693671 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563ddd189350 0x563ddd531b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:18.245 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:18.217+0000 7fbeedaac740 1 --2- 172.21.15.39:0/2230693671 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563ddd168670 0x563ddd53b760 unknown :-1 s=CLOSED pgs=1132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:18.245 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:18.217+0000 7fbeedaac740 1 --2- 172.21.15.39:0/2230693671 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563ddd160490 0x563ddd537680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:18.245 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:18.217+0000 7fbeedaac740 1 -- 172.21.15.39:0/2230693671 >> 172.21.15.39:0/2230693671 conn(0x563ddd17d740 msgr2=0x563ddd531460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:54:18.246 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:18.217+0000 7fbeedaac740 1 -- 172.21.15.39:0/2230693671 shutdown_connections 2024-09-21T03:54:18.246 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:18.217+0000 7fbeedaac740 1 -- 172.21.15.39:0/2230693671 wait complete. 2024-09-21T03:54:18.246 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:18.218+0000 7fbeedaac740 1 -- start start 2024-09-21T03:54:18.246 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:18.218+0000 7fbeedaac740 1 Processor -- start 2024-09-21T03:54:18.246 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:18.218+0000 7fbeedaac740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563ddd160490 0x563ddd540cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:54:18.246 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:18.218+0000 7fbeedaac740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563ddd168670 0x563ddd5411f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:54:18.246 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:18.218+0000 7fbeedaac740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563ddd189350 0x563ddd541730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:54:18.246 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:18.218+0000 7fbeedaac740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fbee01b2ed0 con 0x563ddd189350 2024-09-21T03:54:18.246 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:18.218+0000 7fbeedaac740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fbee01b2800 con 0x563ddd160490 2024-09-21T03:54:18.246 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:18.218+0000 7fbeedaac740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fbee01afa60 con 0x563ddd168670 2024-09-21T03:54:18.246 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:18.218+0000 7fbeedaac740 10 client.0 _ll_register_callbacks cb 0x7fbed0004120 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:54:18.246 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:18.218+0000 7fbee5ad5640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563ddd189350 0x563ddd541730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:54:18.246 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:18.218+0000 7fbee5ad5640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563ddd189350 0x563ddd541730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:58456/0 (socket says 172.21.15.39:58456) 2024-09-21T03:54:18.246 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:18.218+0000 7fbee5ad5640 1 -- 172.21.15.39:0/1894316023 learned_addr learned my addr 172.21.15.39:0/1894316023 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:54:18.246 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:18.218+0000 7fbee5ad5640 1 --2- 172.21.15.39:0/1894316023 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563ddd160490 0x563ddd540cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:54:18.247 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:18.218+0000 7fbee4ad3640 1 --2- 172.21.15.39:0/1894316023 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563ddd168670 0x563ddd5411f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:54:18.247 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:18.219+0000 7fbee5ad5640 1 -- 172.21.15.39:0/1894316023 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563ddd168670 msgr2=0x563ddd5411f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:54:18.247 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:18.219+0000 7fbee5ad5640 1 --2- 172.21.15.39:0/1894316023 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563ddd168670 0x563ddd5411f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:18.247 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:18.219+0000 7fbee5ad5640 1 -- 172.21.15.39:0/1894316023 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563ddd160490 msgr2=0x563ddd540cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:54:18.247 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:18.219+0000 7fbee5ad5640 1 --2- 172.21.15.39:0/1894316023 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563ddd160490 0x563ddd540cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:18.247 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:18.219+0000 7fbee5ad5640 1 -- 172.21.15.39:0/1894316023 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563ddd5b8c40 con 0x563ddd189350 2024-09-21T03:54:18.247 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:18.219+0000 7fbee5ad5640 1 --2- 172.21.15.39:0/1894316023 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563ddd189350 0x563ddd541730 secure :-1 s=READY pgs=1133 cs=0 l=1 rev1=1 crypto rx=0x7fbed802ae80 tx=0x7fbed8005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=72a8bc0bfd023d53 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:54:18.247 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:18.219+0000 7fbec8ff9640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-09-21T03:54:18.247 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:18.219+0000 7fbec8ff9640 1 -- 172.21.15.39:0/1894316023 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbed805f060 con 0x563ddd189350 2024-09-21T03:54:18.247 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:18.219+0000 7fbec8ff9640 1 -- 172.21.15.39:0/1894316023 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8572+0+0 (secure 0 0 0) 0x7fbed8031450 con 0x563ddd189350 2024-09-21T03:54:18.247 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:18.219+0000 7fbeedaac740 1 -- 172.21.15.39:0/1894316023 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbed001b840 con 0x563ddd189350 2024-09-21T03:54:18.247 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:18.219+0000 7fbec8ff9640 1 -- 172.21.15.39:0/1894316023 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fbed8031650 con 0x563ddd189350 2024-09-21T03:54:18.247 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:18.219+0000 7fbeedaac740 10 client.17691 fetch_fsmap learned FSMap version 29 2024-09-21T03:54:18.247 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:18.219+0000 7fbeedaac740 1 -- 172.21.15.39:0/1894316023 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x563ddd5b9e50 con 0x563ddd189350 2024-09-21T03:54:18.247 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:18.219+0000 7fbec8ff9640 1 -- 172.21.15.39:0/1894316023 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbed001b840 con 0x563ddd189350 2024-09-21T03:54:18.247 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:18.220+0000 7fbec8ff9640 1 -- 172.21.15.39:0/1894316023 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fbed8043070 con 0x563ddd189350 2024-09-21T03:54:18.248 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:18.220+0000 7fbeedaac740 10 client.17691 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T03:54:18.248 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:18.220+0000 7fbeedaac740 10 client.17691 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T03:54:18.248 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:18.220+0000 7fbeedaac740 20 client.17691 populate_metadata read hostname 'smithi039' 2024-09-21T03:54:18.248 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:18.220+0000 7fbeedaac740 1 --2- 172.21.15.39:0/1894316023 >> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] conn(0x563ddd11d080 0x563ddd5bc250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:54:18.248 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:18.220+0000 7fbeedaac740 4 client.17691 mds_command: new command op to 24695 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-21T03:54:18.248 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:18.220+0000 7fbeedaac740 1 -- 172.21.15.39:0/1894316023 --> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x563ddd4a4860 con 0x563ddd11d080 2024-09-21T03:54:18.248 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:18.220+0000 7fbee4ad3640 1 --2- 172.21.15.39:0/1894316023 >> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] conn(0x563ddd11d080 0x563ddd5bc250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:54:18.248 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:18.220+0000 7fbee4ad3640 1 --2- 172.21.15.39:0/1894316023 >> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] conn(0x563ddd11d080 0x563ddd5bc250 crc :-1 s=READY pgs=138 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bc8ced829e497f77 server_cookie=6adeddaf6fde9919 in_seq=0 out_seq=0 2024-09-21T03:54:18.248 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:18.220+0000 7fbec8ff9640 10 client.17691 ms_handle_connect on v2:172.21.15.150:6838/2949386907 2024-09-21T03:54:18.248 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:18.222+0000 7fbec8ff9640 1 -- 172.21.15.39:0/1894316023 <== mds.0 v2:172.21.15.150:6838/2949386907 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x563ddd4a4860 con 0x563ddd11d080 2024-09-21T03:54:18.248 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:18.222+0000 7fbec8ff9640 10 client.17691 handle_command_reply: tid=0 2024-09-21T03:54:18.248 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:18.222+0000 7fbeedaac740 1 client.17691 shutdown 2024-09-21T03:54:18.248 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:18.222+0000 7fbeedaac740 1 -- 172.21.15.39:0/1894316023 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563ddd189350 msgr2=0x563ddd541730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:54:18.248 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:18.222+0000 7fbeedaac740 1 --2- 172.21.15.39:0/1894316023 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563ddd189350 0x563ddd541730 secure :-1 s=READY pgs=1133 cs=0 l=1 rev1=1 crypto rx=0x7fbed802ae80 tx=0x7fbed8005fd0 comp rx=0 tx=0).stop 2024-09-21T03:54:18.248 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:18.222+0000 7fbeedaac740 1 -- 172.21.15.39:0/1894316023 shutdown_connections 2024-09-21T03:54:18.248 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:18.222+0000 7fbeedaac740 1 --2- 172.21.15.39:0/1894316023 >> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] conn(0x563ddd11d080 0x563ddd5bc250 crc :-1 s=READY pgs=138 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:18.249 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:18.222+0000 7fbeedaac740 1 --2- 172.21.15.39:0/1894316023 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563ddd189350 0x563ddd541730 unknown :-1 s=CLOSED pgs=1133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:18.249 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:18.222+0000 7fbeedaac740 1 --2- 172.21.15.39:0/1894316023 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563ddd168670 0x563ddd5411f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:18.249 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:18.222+0000 7fbeedaac740 1 --2- 172.21.15.39:0/1894316023 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563ddd160490 0x563ddd540cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:18.249 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:18.222+0000 7fbeedaac740 1 -- 172.21.15.39:0/1894316023 >> 172.21.15.39:0/1894316023 conn(0x563ddd17d740 msgr2=0x563ddd53d380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:54:18.249 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:18.222+0000 7fbec8ff9640 0 client.17691 ms_handle_reset on v2:172.21.15.150:6838/2949386907 2024-09-21T03:54:18.249 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-09-21T03:54:18.249 INFO:teuthology.orchestra.run.smithi039.stdout: "return_code": 0, 2024-09-21T03:54:18.249 INFO:teuthology.orchestra.run.smithi039.stdout: "scrub_tag": "e28dcd13-f2ac-4cb7-a346-58917d542b15", 2024-09-21T03:54:18.249 INFO:teuthology.orchestra.run.smithi039.stdout: "mode": "asynchronous" 2024-09-21T03:54:18.249 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-09-21T03:54:18.250 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:18.223+0000 7fbeedaac740 1 -- 172.21.15.39:0/1894316023 shutdown_connections 2024-09-21T03:54:18.250 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:18.223+0000 7fbeedaac740 1 -- 172.21.15.39:0/1894316023 wait complete. 2024-09-21T03:54:18.250 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:18.223+0000 7fbeedaac740 20 client.17691 trim_cache size 0 max 16384 2024-09-21T03:54:18.250 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:18.223+0000 7fbee7538640 1 -- 172.21.15.39:0/853742053 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7fbecc0b6680 msgr2=0x7fbecc0b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:54:18.250 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:18.223+0000 7fbee7538640 1 --2- 172.21.15.39:0/853742053 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7fbecc0b6680 0x7fbecc0b8b90 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7fbedc0a97b0 tx=0x7fbedc0b4f90 comp rx=0 tx=0).stop 2024-09-21T03:54:18.250 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:18.223+0000 7fbee7538640 1 -- 172.21.15.39:0/853742053 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbee00757b0 msgr2=0x7fbee019c6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:54:18.250 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:18.223+0000 7fbee7538640 1 --2- 172.21.15.39:0/853742053 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbee00757b0 0x7fbee019c6a0 secure :-1 s=READY pgs=1130 cs=0 l=1 rev1=1 crypto rx=0x7fbedc00d950 tx=0x7fbedc00de10 comp rx=0 tx=0).stop 2024-09-21T03:54:18.250 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:18.223+0000 7fbee7538640 1 -- 172.21.15.39:0/853742053 shutdown_connections 2024-09-21T03:54:18.250 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:18.223+0000 7fbee7538640 1 --2- 172.21.15.39:0/853742053 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7fbecc0b6680 0x7fbecc0b8b90 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:18.250 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:18.223+0000 7fbee7538640 1 --2- 172.21.15.39:0/853742053 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbee0110110 0x7fbee01a1ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:18.251 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:18.223+0000 7fbee7538640 1 --2- 172.21.15.39:0/853742053 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbee0076100 0x7fbee019cc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:18.251 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:18.223+0000 7fbee7538640 1 --2- 172.21.15.39:0/853742053 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbee00757b0 0x7fbee019c6a0 unknown :-1 s=CLOSED pgs=1130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:18.251 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:18.223+0000 7fbee7538640 1 -- 172.21.15.39:0/853742053 >> 172.21.15.39:0/853742053 conn(0x7fbee00ff600 msgr2=0x7fbee0109bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:54:18.251 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:18.223+0000 7fbee7538640 1 -- 172.21.15.39:0/853742053 shutdown_connections 2024-09-21T03:54:18.251 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:18.223+0000 7fbee7538640 1 -- 172.21.15.39:0/853742053 wait complete. 2024-09-21T03:54:19.098 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 51116d5c-77ca-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub status 2024-09-21T03:54:19.334 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:54:19 smithi039 ceph-mon[31211]: pgmap v576: 97 pgs: 97 active+clean; 10 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-09-21T03:54:19.334 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:54:19 smithi039 ceph-mon[31211]: scrub queued for path: / 2024-09-21T03:54:19.334 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:54:19 smithi039 ceph-mon[31211]: scrub summary: idle+waiting paths [/] 2024-09-21T03:54:19.334 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:54:19 smithi039 ceph-mon[31211]: scrub summary: active paths [/] 2024-09-21T03:54:19.334 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:54:19 smithi039 ceph-mon[31211]: scrub summary: idle+waiting paths [/] 2024-09-21T03:54:19.361 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/51116d5c-77ca-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:54:19.468 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:54:19 smithi161 ceph-mon[34533]: pgmap v576: 97 pgs: 97 active+clean; 10 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-09-21T03:54:19.468 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:54:19 smithi161 ceph-mon[34533]: scrub queued for path: / 2024-09-21T03:54:19.468 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:54:19 smithi161 ceph-mon[34533]: scrub summary: idle+waiting paths [/] 2024-09-21T03:54:19.468 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:54:19 smithi161 ceph-mon[34533]: scrub summary: active paths [/] 2024-09-21T03:54:19.468 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:54:19 smithi161 ceph-mon[34533]: scrub summary: idle+waiting paths [/] 2024-09-21T03:54:19.475 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:54:19 smithi150 ceph-mon[34289]: pgmap v576: 97 pgs: 97 active+clean; 10 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-09-21T03:54:19.475 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:54:19 smithi150 ceph-mon[34289]: scrub queued for path: / 2024-09-21T03:54:19.475 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:54:19 smithi150 ceph-mon[34289]: scrub summary: idle+waiting paths [/] 2024-09-21T03:54:19.475 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:54:19 smithi150 ceph-mon[34289]: scrub summary: active paths [/] 2024-09-21T03:54:19.475 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:54:19 smithi150 ceph-mon[34289]: scrub summary: idle+waiting paths [/] 2024-09-21T03:54:19.851 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:19.848+0000 7f6193718640 1 -- 172.21.15.39:0/2199529706 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f618c10de70 msgr2=0x7f618c110310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:54:19.851 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:19.848+0000 7f6193718640 1 --2- 172.21.15.39:0/2199529706 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f618c10de70 0x7f618c110310 secure :-1 s=READY pgs=1134 cs=0 l=1 rev1=1 crypto rx=0x7f6184009ca0 tx=0x7f618402a8c0 comp rx=0 tx=0).stop 2024-09-21T03:54:19.852 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:19.848+0000 7f6193718640 1 -- 172.21.15.39:0/2199529706 shutdown_connections 2024-09-21T03:54:19.852 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:19.848+0000 7f6193718640 1 --2- 172.21.15.39:0/2199529706 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f618c10de70 0x7f618c110310 unknown :-1 s=CLOSED pgs=1134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:19.852 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:19.848+0000 7f6193718640 1 --2- 172.21.15.39:0/2199529706 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f618c1052f0 0x7f618c10d930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:19.852 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:19.848+0000 7f6193718640 1 --2- 172.21.15.39:0/2199529706 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f618c1048c0 0x7f618c104cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:19.852 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:19.848+0000 7f6193718640 1 -- 172.21.15.39:0/2199529706 >> 172.21.15.39:0/2199529706 conn(0x7f618c079790 msgr2=0x7f618c079ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:54:19.852 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:19.848+0000 7f6193718640 1 -- 172.21.15.39:0/2199529706 shutdown_connections 2024-09-21T03:54:19.852 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:19.848+0000 7f6193718640 1 -- 172.21.15.39:0/2199529706 wait complete. 2024-09-21T03:54:19.854 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:19.849+0000 7f6193718640 1 Processor -- start 2024-09-21T03:54:19.854 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:19.849+0000 7f6193718640 1 -- start start 2024-09-21T03:54:19.854 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:19.849+0000 7f6193718640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f618c1048c0 0x7f618c19a450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:54:19.854 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:19.849+0000 7f6193718640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f618c1052f0 0x7f618c19a9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:54:19.854 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:19.849+0000 7f6193718640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f618c10de70 0x7f618c19fc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:54:19.854 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:19.849+0000 7f6193718640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f618c113aa0 con 0x7f618c1048c0 2024-09-21T03:54:19.854 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:19.849+0000 7f6193718640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f618c113920 con 0x7f618c1052f0 2024-09-21T03:54:19.854 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:19.849+0000 7f6193718640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f618c113c20 con 0x7f618c10de70 2024-09-21T03:54:19.854 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:19.850+0000 7f61914b4640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f618c1048c0 0x7f618c19a450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:54:19.855 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:19.850+0000 7f61914b4640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f618c1048c0 0x7f618c19a450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:58480/0 (socket says 172.21.15.39:58480) 2024-09-21T03:54:19.855 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:19.850+0000 7f61914b4640 1 -- 172.21.15.39:0/1435519951 learned_addr learned my addr 172.21.15.39:0/1435519951 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:54:19.855 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:19.850+0000 7f6190cb3640 1 --2- 172.21.15.39:0/1435519951 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f618c1052f0 0x7f618c19a9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:54:19.855 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:19.850+0000 7f6191cb5640 1 --2- 172.21.15.39:0/1435519951 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f618c10de70 0x7f618c19fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:54:19.855 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:19.850+0000 7f61914b4640 1 -- 172.21.15.39:0/1435519951 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f618c10de70 msgr2=0x7f618c19fc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:54:19.855 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:19.850+0000 7f61914b4640 1 --2- 172.21.15.39:0/1435519951 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f618c10de70 0x7f618c19fc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:19.855 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:19.850+0000 7f61914b4640 1 -- 172.21.15.39:0/1435519951 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f618c1052f0 msgr2=0x7f618c19a9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:54:19.855 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:19.850+0000 7f61914b4640 1 --2- 172.21.15.39:0/1435519951 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f618c1052f0 0x7f618c19a9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:19.855 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:19.850+0000 7f61914b4640 1 -- 172.21.15.39:0/1435519951 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f618c1a03c0 con 0x7f618c1048c0 2024-09-21T03:54:19.855 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:19.850+0000 7f61914b4640 1 --2- 172.21.15.39:0/1435519951 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f618c1048c0 0x7f618c19a450 secure :-1 s=READY pgs=1135 cs=0 l=1 rev1=1 crypto rx=0x7f618800d950 tx=0x7f618800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:54:19.855 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:19.850+0000 7f61827fc640 1 -- 172.21.15.39:0/1435519951 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6188017030 con 0x7f618c1048c0 2024-09-21T03:54:19.855 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:19.851+0000 7f6193718640 1 -- 172.21.15.39:0/1435519951 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f618c1a06b0 con 0x7f618c1048c0 2024-09-21T03:54:19.856 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:19.851+0000 7f61827fc640 1 -- 172.21.15.39:0/1435519951 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6188004020 con 0x7f618c1048c0 2024-09-21T03:54:19.856 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:19.851+0000 7f61827fc640 1 -- 172.21.15.39:0/1435519951 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61880047a0 con 0x7f618c1048c0 2024-09-21T03:54:19.856 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:19.851+0000 7f6193718640 1 -- 172.21.15.39:0/1435519951 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f618c1a6e10 con 0x7f618c1048c0 2024-09-21T03:54:19.857 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:19.853+0000 7f61827fc640 1 -- 172.21.15.39:0/1435519951 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 27) ==== 154643+0+0 (secure 0 0 0) 0x7f6188004940 con 0x7f618c1048c0 2024-09-21T03:54:19.858 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:19.854+0000 7f6199c8c740 1 Processor -- start 2024-09-21T03:54:19.858 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:19.854+0000 7f6199c8c740 1 -- start start 2024-09-21T03:54:19.858 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:19.854+0000 7f6199c8c740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556020743070 0x556020aea650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:54:19.858 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:19.854+0000 7f6199c8c740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55602071a1b0 0x556020aeab90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:54:19.858 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:19.854+0000 7f6199c8c740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556020722390 0x556020aeb0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:54:19.858 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:19.854+0000 7f6199c8c740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x556020a60e50 con 0x55602071a1b0 2024-09-21T03:54:19.858 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:19.854+0000 7f6199c8c740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x556020a50280 con 0x556020743070 2024-09-21T03:54:19.858 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:19.854+0000 7f6199c8c740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x556020a54090 con 0x556020722390 2024-09-21T03:54:19.858 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:19.854+0000 7f61914b4640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55602071a1b0 0x556020aeab90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:54:19.858 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:19.854+0000 7f6190cb3640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556020722390 0x556020aeb0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:54:19.858 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:19.854+0000 7f61914b4640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55602071a1b0 0x556020aeab90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:58494/0 (socket says 172.21.15.39:58494) 2024-09-21T03:54:19.858 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:19.854+0000 7f61914b4640 1 -- 172.21.15.39:0/3422940805 learned_addr learned my addr 172.21.15.39:0/3422940805 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:54:19.858 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:19.854+0000 7f6191cb5640 1 --2- 172.21.15.39:0/3422940805 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556020743070 0x556020aea650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:54:19.858 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:19.855+0000 7f61914b4640 1 -- 172.21.15.39:0/3422940805 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556020722390 msgr2=0x556020aeb0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:54:19.859 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:19.855+0000 7f61914b4640 1 --2- 172.21.15.39:0/3422940805 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556020722390 0x556020aeb0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:19.859 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:19.855+0000 7f61914b4640 1 -- 172.21.15.39:0/3422940805 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556020743070 msgr2=0x556020aea650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:54:19.859 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:19.855+0000 7f61914b4640 1 --2- 172.21.15.39:0/3422940805 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556020743070 0x556020aea650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:19.859 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:19.855+0000 7f61914b4640 1 -- 172.21.15.39:0/3422940805 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5560207ae650 con 0x55602071a1b0 2024-09-21T03:54:19.859 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:19.855+0000 7f6190cb3640 1 --2- 172.21.15.39:0/3422940805 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556020722390 0x556020aeb0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:54:19.859 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:19.855+0000 7f6191cb5640 1 --2- 172.21.15.39:0/3422940805 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556020743070 0x556020aea650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:54:19.859 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:19.855+0000 7f61827fc640 1 --2- 172.21.15.39:0/1435519951 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f61780b6680 0x7f61780b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:54:19.859 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:19.855+0000 7f61914b4640 1 --2- 172.21.15.39:0/3422940805 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55602071a1b0 0x556020aeab90 secure :-1 s=READY pgs=1136 cs=0 l=1 rev1=1 crypto rx=0x7f61880e47f0 tx=0x7f61880ef6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d013ed2cd823fbef server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:54:19.859 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:19.855+0000 7f6191cb5640 1 --2- 172.21.15.39:0/1435519951 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f61780b6680 0x7f61780b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:54:19.862 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:19.855+0000 7f61827fc640 1 -- 172.21.15.39:0/1435519951 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8528+0+0 (secure 0 0 0) 0x7f61880e71a0 con 0x7f618c1048c0 2024-09-21T03:54:19.862 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:19.855+0000 7f61777fe640 1 -- 172.21.15.39:0/3422940805 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61880a9600 con 0x55602071a1b0 2024-09-21T03:54:19.862 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:19.855+0000 7f61777fe640 1 -- 172.21.15.39:0/3422940805 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f61880acac0 con 0x55602071a1b0 2024-09-21T03:54:19.862 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:19.855+0000 7f61777fe640 1 -- 172.21.15.39:0/3422940805 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61880ead40 con 0x55602071a1b0 2024-09-21T03:54:19.862 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:19.856+0000 7f6199c8c740 1 -- 172.21.15.39:0/3422940805 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55602071a1b0 msgr2=0x556020aeab90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:54:19.863 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:19.856+0000 7f6199c8c740 1 --2- 172.21.15.39:0/3422940805 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55602071a1b0 0x556020aeab90 secure :-1 s=READY pgs=1136 cs=0 l=1 rev1=1 crypto rx=0x7f61880e47f0 tx=0x7f61880ef6f0 comp rx=0 tx=0).stop 2024-09-21T03:54:19.863 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:19.856+0000 7f6191cb5640 1 --2- 172.21.15.39:0/1435519951 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f61780b6680 0x7f61780b8b90 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f618402f500 tx=0x7f618402f490 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:54:19.863 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:19.856+0000 7f6199c8c740 1 -- 172.21.15.39:0/3422940805 shutdown_connections 2024-09-21T03:54:19.863 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:19.856+0000 7f6199c8c740 1 --2- 172.21.15.39:0/3422940805 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556020722390 0x556020aeb0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:19.863 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:19.856+0000 7f6199c8c740 1 --2- 172.21.15.39:0/3422940805 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55602071a1b0 0x556020aeab90 unknown :-1 s=CLOSED pgs=1136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:19.863 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:19.856+0000 7f6199c8c740 1 --2- 172.21.15.39:0/3422940805 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556020743070 0x556020aea650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:19.863 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:19.856+0000 7f6199c8c740 1 -- 172.21.15.39:0/3422940805 >> 172.21.15.39:0/3422940805 conn(0x556020737460 msgr2=0x556020ae8540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:54:19.863 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:19.856+0000 7f6199c8c740 1 -- 172.21.15.39:0/3422940805 shutdown_connections 2024-09-21T03:54:19.863 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:19.856+0000 7f6199c8c740 1 -- 172.21.15.39:0/3422940805 wait complete. 2024-09-21T03:54:19.863 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:19.857+0000 7f6199c8c740 1 -- start start 2024-09-21T03:54:19.863 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:19.857+0000 7f6199c8c740 1 Processor -- start 2024-09-21T03:54:19.863 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:19.857+0000 7f6199c8c740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55602071a1b0 0x556020af5d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:54:19.863 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:19.857+0000 7f6199c8c740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556020722390 0x556020af62d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:54:19.863 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:19.857+0000 7f6199c8c740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556020743070 0x556020b020f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:54:19.864 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:19.857+0000 7f6199c8c740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x5560208b1700 con 0x556020743070 2024-09-21T03:54:19.864 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:19.857+0000 7f6199c8c740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x5560209eb110 con 0x55602071a1b0 2024-09-21T03:54:19.864 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:19.857+0000 7f6199c8c740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x556020af8dd0 con 0x556020722390 2024-09-21T03:54:19.864 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:19.857+0000 7f6199c8c740 10 client.0 _ll_register_callbacks cb 0x556020a04990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:54:19.864 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:19.858+0000 7f6191cb5640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556020743070 0x556020b020f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:54:19.864 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:19.858+0000 7f6190cb3640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556020722390 0x556020af62d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:54:19.864 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:19.858+0000 7f6191cb5640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556020743070 0x556020b020f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:58504/0 (socket says 172.21.15.39:58504) 2024-09-21T03:54:19.864 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:19.858+0000 7f6191cb5640 1 -- 172.21.15.39:0/3231732809 learned_addr learned my addr 172.21.15.39:0/3231732809 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:54:19.864 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:19.858+0000 7f61914b4640 1 --2- 172.21.15.39:0/3231732809 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55602071a1b0 0x556020af5d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:54:19.864 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:19.858+0000 7f6191cb5640 1 -- 172.21.15.39:0/3231732809 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556020722390 msgr2=0x556020af62d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:54:19.864 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:19.858+0000 7f6191cb5640 1 --2- 172.21.15.39:0/3231732809 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556020722390 0x556020af62d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:19.864 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:19.858+0000 7f6191cb5640 1 -- 172.21.15.39:0/3231732809 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55602071a1b0 msgr2=0x556020af5d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:54:19.864 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:19.858+0000 7f6191cb5640 1 --2- 172.21.15.39:0/3231732809 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55602071a1b0 0x556020af5d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:19.864 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:19.858+0000 7f6191cb5640 1 -- 172.21.15.39:0/3231732809 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556020a18c40 con 0x556020743070 2024-09-21T03:54:19.865 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:19.858+0000 7f61914b4640 1 --2- 172.21.15.39:0/3231732809 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55602071a1b0 0x556020af5d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:54:19.865 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:19.858+0000 7f6191cb5640 1 --2- 172.21.15.39:0/3231732809 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556020743070 0x556020b020f0 secure :-1 s=READY pgs=1137 cs=0 l=1 rev1=1 crypto rx=0x7f6184035000 tx=0x7f618402a8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e2797de1a29f7d server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:54:19.865 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:19.858+0000 7f6174ff9640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-09-21T03:54:19.865 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:19.858+0000 7f6174ff9640 1 -- 172.21.15.39:0/3231732809 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6184003d00 con 0x556020743070 2024-09-21T03:54:19.865 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:19.858+0000 7f6174ff9640 1 -- 172.21.15.39:0/3231732809 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8528+0+0 (secure 0 0 0) 0x7f6184004040 con 0x556020743070 2024-09-21T03:54:19.865 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:19.858+0000 7f6199c8c740 1 -- 172.21.15.39:0/3231732809 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f61880acac0 con 0x556020743070 2024-09-21T03:54:19.865 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:19.859+0000 7f6174ff9640 1 -- 172.21.15.39:0/3231732809 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f6184004240 con 0x556020743070 2024-09-21T03:54:19.865 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:19.859+0000 7f6199c8c740 10 client.17703 fetch_fsmap learned FSMap version 29 2024-09-21T03:54:19.865 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:19.859+0000 7f6199c8c740 1 -- 172.21.15.39:0/3231732809 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x556020b08f50 con 0x556020743070 2024-09-21T03:54:19.865 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:19.859+0000 7f6174ff9640 1 -- 172.21.15.39:0/3231732809 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f61880acac0 con 0x556020743070 2024-09-21T03:54:19.865 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:19.859+0000 7f6174ff9640 1 -- 172.21.15.39:0/3231732809 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f6184051070 con 0x556020743070 2024-09-21T03:54:19.865 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:19.859+0000 7f6199c8c740 10 client.17703 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T03:54:19.865 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:19.859+0000 7f6199c8c740 10 client.17703 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T03:54:19.865 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:19.859+0000 7f6199c8c740 20 client.17703 populate_metadata read hostname 'smithi039' 2024-09-21T03:54:19.865 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:19.859+0000 7f6199c8c740 1 --2- 172.21.15.39:0/3231732809 >> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] conn(0x5560207019f0 0x556020b0c400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:54:19.866 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:19.859+0000 7f6199c8c740 4 client.17703 mds_command: new command op to 24695 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:54:19.866 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:19.859+0000 7f6199c8c740 1 -- 172.21.15.39:0/3231732809 --> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556020a62da0 con 0x5560207019f0 2024-09-21T03:54:19.866 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:19.859+0000 7f61914b4640 1 --2- 172.21.15.39:0/3231732809 >> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] conn(0x5560207019f0 0x556020b0c400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:54:19.867 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:19.860+0000 7f61914b4640 1 --2- 172.21.15.39:0/3231732809 >> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] conn(0x5560207019f0 0x556020b0c400 crc :-1 s=READY pgs=139 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3c496387fb261e73 server_cookie=9edc05150235a590 in_seq=0 out_seq=0 2024-09-21T03:54:19.867 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:19.860+0000 7f6174ff9640 10 client.17703 ms_handle_connect on v2:172.21.15.150:6838/2949386907 2024-09-21T03:54:19.867 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:19.861+0000 7f6174ff9640 1 -- 172.21.15.39:0/3231732809 <== mds.0 v2:172.21.15.150:6838/2949386907 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x556020a62da0 con 0x5560207019f0 2024-09-21T03:54:19.867 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:19.861+0000 7f6174ff9640 10 client.17703 handle_command_reply: tid=0 2024-09-21T03:54:19.867 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:19.861+0000 7f6199c8c740 1 client.17703 shutdown 2024-09-21T03:54:19.867 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:19.861+0000 7f6199c8c740 1 -- 172.21.15.39:0/3231732809 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556020743070 msgr2=0x556020b020f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:54:19.867 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:19.861+0000 7f6199c8c740 1 --2- 172.21.15.39:0/3231732809 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556020743070 0x556020b020f0 secure :-1 s=READY pgs=1137 cs=0 l=1 rev1=1 crypto rx=0x7f6184035000 tx=0x7f618402a8c0 comp rx=0 tx=0).stop 2024-09-21T03:54:19.867 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:19.861+0000 7f6199c8c740 1 -- 172.21.15.39:0/3231732809 shutdown_connections 2024-09-21T03:54:19.867 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:19.861+0000 7f6199c8c740 1 --2- 172.21.15.39:0/3231732809 >> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] conn(0x5560207019f0 0x556020b0c400 crc :-1 s=READY pgs=139 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:19.867 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:19.861+0000 7f6199c8c740 1 --2- 172.21.15.39:0/3231732809 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556020743070 0x556020b020f0 unknown :-1 s=CLOSED pgs=1137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:19.867 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:19.861+0000 7f6199c8c740 1 --2- 172.21.15.39:0/3231732809 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556020722390 0x556020af62d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:19.867 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:19.861+0000 7f6174ff9640 0 client.17703 ms_handle_reset on v2:172.21.15.150:6838/2949386907 2024-09-21T03:54:19.867 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:19.861+0000 7f6199c8c740 1 --2- 172.21.15.39:0/3231732809 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55602071a1b0 0x556020af5d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:19.868 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:19.861+0000 7f6199c8c740 1 -- 172.21.15.39:0/3231732809 >> 172.21.15.39:0/3231732809 conn(0x556020737460 msgr2=0x556020aed7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:54:19.868 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:19.861+0000 7f6199c8c740 1 -- 172.21.15.39:0/3231732809 shutdown_connections 2024-09-21T03:54:19.868 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:19.861+0000 7f6199c8c740 1 -- 172.21.15.39:0/3231732809 wait complete. 2024-09-21T03:54:19.868 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:19.861+0000 7f6199c8c740 20 client.17703 trim_cache size 0 max 16384 2024-09-21T03:54:19.883 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:19.879+0000 7f6199c8c740 1 Processor -- start 2024-09-21T03:54:19.883 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:19.879+0000 7f6199c8c740 1 -- start start 2024-09-21T03:54:19.883 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:19.879+0000 7f6199c8c740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5560207019f0 0x556020af79c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:54:19.883 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:19.879+0000 7f6199c8c740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55602071a1b0 0x556020ae7b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:54:19.883 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:19.879+0000 7f6199c8c740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556020722390 0x556020ae80b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:54:19.883 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:19.879+0000 7f6199c8c740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f618c1ad550 con 0x5560207019f0 2024-09-21T03:54:19.883 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:19.879+0000 7f6199c8c740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f618c1af090 con 0x556020722390 2024-09-21T03:54:19.883 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:19.879+0000 7f6199c8c740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f618c1adc20 con 0x55602071a1b0 2024-09-21T03:54:19.883 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:19.879+0000 7f61914b4640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5560207019f0 0x556020af79c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:54:19.883 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:19.879+0000 7f61914b4640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5560207019f0 0x556020af79c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:58506/0 (socket says 172.21.15.39:58506) 2024-09-21T03:54:19.883 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:19.879+0000 7f61914b4640 1 -- 172.21.15.39:0/2369678469 learned_addr learned my addr 172.21.15.39:0/2369678469 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:54:19.884 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:19.880+0000 7f6190cb3640 1 --2- 172.21.15.39:0/2369678469 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55602071a1b0 0x556020ae7b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:54:19.884 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:19.880+0000 7f61914b4640 1 -- 172.21.15.39:0/2369678469 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55602071a1b0 msgr2=0x556020ae7b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:54:19.884 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:19.880+0000 7f6191cb5640 1 --2- 172.21.15.39:0/2369678469 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556020722390 0x556020ae80b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:54:19.884 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:19.880+0000 7f61914b4640 1 --2- 172.21.15.39:0/2369678469 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55602071a1b0 0x556020ae7b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:19.884 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:19.880+0000 7f61914b4640 1 -- 172.21.15.39:0/2369678469 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556020722390 msgr2=0x556020ae80b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:54:19.884 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:19.880+0000 7f61914b4640 1 --2- 172.21.15.39:0/2369678469 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556020722390 0x556020ae80b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:19.884 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:19.880+0000 7f61914b4640 1 -- 172.21.15.39:0/2369678469 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556020ae8eb0 con 0x5560207019f0 2024-09-21T03:54:19.884 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:19.880+0000 7f6190cb3640 1 --2- 172.21.15.39:0/2369678469 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55602071a1b0 0x556020ae7b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:54:19.884 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:19.880+0000 7f6191cb5640 1 --2- 172.21.15.39:0/2369678469 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556020722390 0x556020ae80b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:54:19.884 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:19.880+0000 7f61914b4640 1 --2- 172.21.15.39:0/2369678469 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5560207019f0 0x556020af79c0 secure :-1 s=READY pgs=1138 cs=0 l=1 rev1=1 crypto rx=0x7f61880e7490 tx=0x7f6188094040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b681b544a1433bb2 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:54:19.884 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:19.880+0000 7f6176ffd640 1 -- 172.21.15.39:0/2369678469 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61880ac720 con 0x5560207019f0 2024-09-21T03:54:19.884 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:19.880+0000 7f6176ffd640 1 -- 172.21.15.39:0/2369678469 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f61880acac0 con 0x5560207019f0 2024-09-21T03:54:19.884 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:19.880+0000 7f6176ffd640 1 -- 172.21.15.39:0/2369678469 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61880ac900 con 0x5560207019f0 2024-09-21T03:54:19.888 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:19.881+0000 7f6199c8c740 1 -- 172.21.15.39:0/2369678469 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5560207019f0 msgr2=0x556020af79c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:54:19.888 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:19.881+0000 7f6199c8c740 1 --2- 172.21.15.39:0/2369678469 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5560207019f0 0x556020af79c0 secure :-1 s=READY pgs=1138 cs=0 l=1 rev1=1 crypto rx=0x7f61880e7490 tx=0x7f6188094040 comp rx=0 tx=0).stop 2024-09-21T03:54:19.888 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:19.881+0000 7f6199c8c740 1 -- 172.21.15.39:0/2369678469 shutdown_connections 2024-09-21T03:54:19.888 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:19.881+0000 7f6199c8c740 1 --2- 172.21.15.39:0/2369678469 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556020722390 0x556020ae80b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:19.888 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:19.881+0000 7f6199c8c740 1 --2- 172.21.15.39:0/2369678469 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55602071a1b0 0x556020ae7b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:19.888 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:19.881+0000 7f6199c8c740 1 --2- 172.21.15.39:0/2369678469 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5560207019f0 0x556020af79c0 unknown :-1 s=CLOSED pgs=1138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:19.888 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:19.881+0000 7f6199c8c740 1 -- 172.21.15.39:0/2369678469 >> 172.21.15.39:0/2369678469 conn(0x556020737460 msgr2=0x556020af72b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:54:19.888 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:19.881+0000 7f6199c8c740 1 -- 172.21.15.39:0/2369678469 shutdown_connections 2024-09-21T03:54:19.889 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:19.881+0000 7f6199c8c740 1 -- 172.21.15.39:0/2369678469 wait complete. 2024-09-21T03:54:19.889 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:19.881+0000 7f6199c8c740 1 -- start start 2024-09-21T03:54:19.889 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:19.882+0000 7f6199c8c740 1 Processor -- start 2024-09-21T03:54:19.889 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:19.882+0000 7f6199c8c740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5560207019f0 0x556020afe670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:54:19.889 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:19.882+0000 7f6199c8c740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55602071a1b0 0x556020af9b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:54:19.889 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:19.882+0000 7f6199c8c740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556020722390 0x556020afa060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:54:19.889 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:19.882+0000 7f6199c8c740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f618c1ae9c0 con 0x55602071a1b0 2024-09-21T03:54:19.889 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:19.882+0000 7f6199c8c740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f618c1ae2f0 con 0x556020722390 2024-09-21T03:54:19.889 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:19.882+0000 7f6199c8c740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f618c1ab510 con 0x5560207019f0 2024-09-21T03:54:19.889 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:19.882+0000 7f6199c8c740 10 client.0 _ll_register_callbacks cb 0x7f618402f310 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:54:19.889 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:19.882+0000 7f6190cb3640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55602071a1b0 0x556020af9b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:54:19.889 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:19.882+0000 7f6190cb3640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55602071a1b0 0x556020af9b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:58508/0 (socket says 172.21.15.39:58508) 2024-09-21T03:54:19.889 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:19.882+0000 7f6190cb3640 1 -- 172.21.15.39:0/1339576835 learned_addr learned my addr 172.21.15.39:0/1339576835 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:54:19.889 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:19.882+0000 7f61914b4640 1 --2- 172.21.15.39:0/1339576835 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5560207019f0 0x556020afe670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:54:19.890 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:19.883+0000 7f6191cb5640 1 --2- 172.21.15.39:0/1339576835 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556020722390 0x556020afa060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:54:19.890 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:19.883+0000 7f6190cb3640 1 -- 172.21.15.39:0/1339576835 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5560207019f0 msgr2=0x556020afe670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:54:19.890 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:19.883+0000 7f6190cb3640 1 --2- 172.21.15.39:0/1339576835 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5560207019f0 0x556020afe670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:19.890 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:19.883+0000 7f6190cb3640 1 -- 172.21.15.39:0/1339576835 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556020722390 msgr2=0x556020afa060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:54:19.890 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:19.883+0000 7f6190cb3640 1 --2- 172.21.15.39:0/1339576835 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556020722390 0x556020afa060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:19.890 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:19.883+0000 7f6190cb3640 1 -- 172.21.15.39:0/1339576835 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556020afa620 con 0x55602071a1b0 2024-09-21T03:54:19.890 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:19.883+0000 7f6191cb5640 1 --2- 172.21.15.39:0/1339576835 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556020722390 0x556020afa060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:54:19.890 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:19.883+0000 7f6190cb3640 1 --2- 172.21.15.39:0/1339576835 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55602071a1b0 0x556020af9b20 secure :-1 s=READY pgs=1139 cs=0 l=1 rev1=1 crypto rx=0x7f617c0108f0 tx=0x7f617c010db0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=132a34a8a325c784 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:54:19.890 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:19.883+0000 7f6174ff9640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-09-21T03:54:19.890 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:19.883+0000 7f6174ff9640 1 -- 172.21.15.39:0/1339576835 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f617c023060 con 0x55602071a1b0 2024-09-21T03:54:19.890 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:19.883+0000 7f6174ff9640 1 -- 172.21.15.39:0/1339576835 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8528+0+0 (secure 0 0 0) 0x7f617c004460 con 0x55602071a1b0 2024-09-21T03:54:19.890 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:19.883+0000 7f6199c8c740 1 -- 172.21.15.39:0/1339576835 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f61880acac0 con 0x55602071a1b0 2024-09-21T03:54:19.890 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:19.883+0000 7f6174ff9640 1 -- 172.21.15.39:0/1339576835 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f617c00f8a0 con 0x55602071a1b0 2024-09-21T03:54:19.890 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:19.883+0000 7f6199c8c740 10 client.17709 fetch_fsmap learned FSMap version 29 2024-09-21T03:54:19.891 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:19.883+0000 7f6199c8c740 1 -- 172.21.15.39:0/1339576835 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x556020b76a80 con 0x55602071a1b0 2024-09-21T03:54:19.891 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:19.884+0000 7f6174ff9640 1 -- 172.21.15.39:0/1339576835 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f61880acac0 con 0x55602071a1b0 2024-09-21T03:54:19.891 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:19.884+0000 7f6174ff9640 1 -- 172.21.15.39:0/1339576835 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f617c01e470 con 0x55602071a1b0 2024-09-21T03:54:19.891 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:19.884+0000 7f6199c8c740 10 client.17709 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T03:54:19.891 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:19.884+0000 7f6199c8c740 10 client.17709 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T03:54:19.891 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:19.884+0000 7f6199c8c740 20 client.17709 populate_metadata read hostname 'smithi039' 2024-09-21T03:54:19.891 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:19.884+0000 7f6199c8c740 1 --2- 172.21.15.39:0/1339576835 >> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] conn(0x556020743070 0x556020b792a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:54:19.891 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:19.884+0000 7f6199c8c740 4 client.17709 mds_command: new command op to 24695 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-21T03:54:19.891 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:19.884+0000 7f6199c8c740 1 -- 172.21.15.39:0/1339576835 --> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] -- command(tid 0: {"prefix": "scrub status"}) -- 0x556020ae3990 con 0x556020743070 2024-09-21T03:54:19.891 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:19.884+0000 7f61914b4640 1 --2- 172.21.15.39:0/1339576835 >> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] conn(0x556020743070 0x556020b792a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:54:19.891 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:19.884+0000 7f6174ff9640 10 client.17709 ms_handle_connect on v2:172.21.15.150:6838/2949386907 2024-09-21T03:54:19.891 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:19.884+0000 7f61914b4640 1 --2- 172.21.15.39:0/1339576835 >> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] conn(0x556020743070 0x556020b792a0 crc :-1 s=READY pgs=140 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9e7834355178e60f server_cookie=87940ab7f24be6b in_seq=0 out_seq=0 2024-09-21T03:54:19.891 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:19.885+0000 7f6174ff9640 1 -- 172.21.15.39:0/1339576835 <== mds.0 v2:172.21.15.150:6838/2949386907 1 ==== command_reply(tid 0: 0 ) ==== 8+0+4634 (crc 0 0 0) 0x556020ae3990 con 0x556020743070 2024-09-21T03:54:19.891 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:19.885+0000 7f6174ff9640 10 client.17709 handle_command_reply: tid=0 2024-09-21T03:54:19.891 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:19.885+0000 7f6199c8c740 1 client.17709 shutdown 2024-09-21T03:54:19.892 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:19.885+0000 7f6199c8c740 1 -- 172.21.15.39:0/1339576835 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55602071a1b0 msgr2=0x556020af9b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:54:19.892 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:19.885+0000 7f6199c8c740 1 --2- 172.21.15.39:0/1339576835 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55602071a1b0 0x556020af9b20 secure :-1 s=READY pgs=1139 cs=0 l=1 rev1=1 crypto rx=0x7f617c0108f0 tx=0x7f617c010db0 comp rx=0 tx=0).stop 2024-09-21T03:54:19.892 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:19.885+0000 7f6199c8c740 1 -- 172.21.15.39:0/1339576835 shutdown_connections 2024-09-21T03:54:19.892 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:19.885+0000 7f6199c8c740 1 --2- 172.21.15.39:0/1339576835 >> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] conn(0x556020743070 0x556020b792a0 crc :-1 s=READY pgs=140 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:19.892 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:19.885+0000 7f6199c8c740 1 --2- 172.21.15.39:0/1339576835 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556020722390 0x556020afa060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:19.892 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:19.885+0000 7f6199c8c740 1 --2- 172.21.15.39:0/1339576835 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55602071a1b0 0x556020af9b20 unknown :-1 s=CLOSED pgs=1139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:19.892 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:19.885+0000 7f6199c8c740 1 --2- 172.21.15.39:0/1339576835 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5560207019f0 0x556020afe670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:19.892 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:19.885+0000 7f6199c8c740 1 -- 172.21.15.39:0/1339576835 >> 172.21.15.39:0/1339576835 conn(0x556020737460 msgr2=0x556020af7240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:54:19.892 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:19.885+0000 7f6174ff9640 0 client.17709 ms_handle_reset on v2:172.21.15.150:6838/2949386907 2024-09-21T03:54:19.892 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:19.885+0000 7f6199c8c740 1 -- 172.21.15.39:0/1339576835 shutdown_connections 2024-09-21T03:54:19.892 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:19.885+0000 7f6199c8c740 1 -- 172.21.15.39:0/1339576835 wait complete. 2024-09-21T03:54:19.892 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:19.885+0000 7f6199c8c740 20 client.17709 trim_cache size 0 max 16384 2024-09-21T03:54:19.893 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-09-21T03:54:19.893 INFO:teuthology.orchestra.run.smithi039.stdout: "status": "no active scrubs running", 2024-09-21T03:54:19.893 INFO:teuthology.orchestra.run.smithi039.stdout: "scrubs": {}, 2024-09-21T03:54:19.893 INFO:teuthology.orchestra.run.smithi039.stdout: "scrub_stats": { 2024-09-21T03:54:19.893 INFO:teuthology.orchestra.run.smithi039.stdout: "e28dcd13-f2ac-4cb7-a346-58917d542b15": { 2024-09-21T03:54:19.893 INFO:teuthology.orchestra.run.smithi039.stdout: "start_time": "2024-09-21T03:54:18.221680+0000", 2024-09-21T03:54:19.893 INFO:teuthology.orchestra.run.smithi039.stdout: "path": "/", 2024-09-21T03:54:19.893 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_started": 0, 2024-09-21T03:54:19.893 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_passed": 0, 2024-09-21T03:54:19.893 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_failed": 0, 2024-09-21T03:54:19.893 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_skipped": 0 2024-09-21T03:54:19.894 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:54:19.894 INFO:teuthology.orchestra.run.smithi039.stdout: "ad87da79-0248-4e18-9694-8228cb95422f": { 2024-09-21T03:54:19.894 INFO:teuthology.orchestra.run.smithi039.stdout: "start_time": "2024-09-21T03:53:18.441907+0000", 2024-09-21T03:54:19.894 INFO:teuthology.orchestra.run.smithi039.stdout: "path": "/", 2024-09-21T03:54:19.894 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_started": 0, 2024-09-21T03:54:19.894 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_passed": 0, 2024-09-21T03:54:19.894 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_failed": 0, 2024-09-21T03:54:19.894 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_skipped": 0 2024-09-21T03:54:19.894 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:54:19.894 INFO:teuthology.orchestra.run.smithi039.stdout: "02509e4f-3911-4983-b16f-df8ded0b9998": { 2024-09-21T03:54:19.894 INFO:teuthology.orchestra.run.smithi039.stdout: "start_time": "2024-09-21T03:46:17.889576+0000", 2024-09-21T03:54:19.894 INFO:teuthology.orchestra.run.smithi039.stdout: "path": "/", 2024-09-21T03:54:19.894 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_started": 0, 2024-09-21T03:54:19.894 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_passed": 0, 2024-09-21T03:54:19.895 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_failed": 0, 2024-09-21T03:54:19.895 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_skipped": 0 2024-09-21T03:54:19.895 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:54:19.895 INFO:teuthology.orchestra.run.smithi039.stdout: "10728de8-c49d-4edc-be2b-639249255e94": { 2024-09-21T03:54:19.895 INFO:teuthology.orchestra.run.smithi039.stdout: "start_time": "2024-09-21T03:47:03.965623+0000", 2024-09-21T03:54:19.895 INFO:teuthology.orchestra.run.smithi039.stdout: "path": "/", 2024-09-21T03:54:19.895 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_started": 0, 2024-09-21T03:54:19.895 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_passed": 0, 2024-09-21T03:54:19.895 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_failed": 0, 2024-09-21T03:54:19.895 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_skipped": 0 2024-09-21T03:54:19.895 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:54:19.895 INFO:teuthology.orchestra.run.smithi039.stdout: "9860c2f4-b7b1-47aa-865a-9a1c4434a5f6": { 2024-09-21T03:54:19.895 INFO:teuthology.orchestra.run.smithi039.stdout: "start_time": "2024-09-21T03:49:17.718867+0000", 2024-09-21T03:54:19.895 INFO:teuthology.orchestra.run.smithi039.stdout: "path": "/", 2024-09-21T03:54:19.895 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_started": 0, 2024-09-21T03:54:19.896 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_passed": 0, 2024-09-21T03:54:19.896 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_failed": 0, 2024-09-21T03:54:19.896 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_skipped": 0 2024-09-21T03:54:19.896 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:54:19.896 INFO:teuthology.orchestra.run.smithi039.stdout: "d9391d69-baac-453a-9d52-d046760c070d": { 2024-09-21T03:54:19.896 INFO:teuthology.orchestra.run.smithi039.stdout: "start_time": "2024-09-21T03:53:32.364622+0000", 2024-09-21T03:54:19.896 INFO:teuthology.orchestra.run.smithi039.stdout: "path": "/", 2024-09-21T03:54:19.896 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_started": 0, 2024-09-21T03:54:19.896 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_passed": 0, 2024-09-21T03:54:19.896 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_failed": 0, 2024-09-21T03:54:19.896 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_skipped": 0 2024-09-21T03:54:19.896 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:54:19.896 INFO:teuthology.orchestra.run.smithi039.stdout: "c7c700f3-2cd8-41ac-8381-b57dd1b37b02": { 2024-09-21T03:54:19.896 INFO:teuthology.orchestra.run.smithi039.stdout: "start_time": "2024-09-21T03:52:32.368858+0000", 2024-09-21T03:54:19.896 INFO:teuthology.orchestra.run.smithi039.stdout: "path": "/", 2024-09-21T03:54:19.896 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_started": 0, 2024-09-21T03:54:19.897 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_passed": 0, 2024-09-21T03:54:19.897 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_failed": 0, 2024-09-21T03:54:19.897 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_skipped": 0 2024-09-21T03:54:19.897 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:54:19.897 INFO:teuthology.orchestra.run.smithi039.stdout: "29411307-eb8d-4339-8d2b-47ebd32c0080": { 2024-09-21T03:54:19.897 INFO:teuthology.orchestra.run.smithi039.stdout: "start_time": "2024-09-21T03:47:38.092920+0000", 2024-09-21T03:54:19.897 INFO:teuthology.orchestra.run.smithi039.stdout: "path": "/", 2024-09-21T03:54:19.897 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_started": 0, 2024-09-21T03:54:19.897 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_passed": 0, 2024-09-21T03:54:19.897 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_failed": 0, 2024-09-21T03:54:19.897 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_skipped": 0 2024-09-21T03:54:19.897 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:54:19.897 INFO:teuthology.orchestra.run.smithi039.stdout: "ede73e6e-3fd0-4a7f-b5bb-f68946f57eea": { 2024-09-21T03:54:19.897 INFO:teuthology.orchestra.run.smithi039.stdout: "start_time": "2024-09-21T03:51:32.235096+0000", 2024-09-21T03:54:19.897 INFO:teuthology.orchestra.run.smithi039.stdout: "path": "/", 2024-09-21T03:54:19.897 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_started": 0, 2024-09-21T03:54:19.898 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_passed": 0, 2024-09-21T03:54:19.898 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_failed": 0, 2024-09-21T03:54:19.898 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_skipped": 0 2024-09-21T03:54:19.898 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:54:19.898 INFO:teuthology.orchestra.run.smithi039.stdout: "e28ad743-2b22-45b3-8799-a638c2883623": { 2024-09-21T03:54:19.898 INFO:teuthology.orchestra.run.smithi039.stdout: "start_time": "2024-09-21T03:48:31.448821+0000", 2024-09-21T03:54:19.898 INFO:teuthology.orchestra.run.smithi039.stdout: "path": "/", 2024-09-21T03:54:19.898 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_started": 0, 2024-09-21T03:54:19.898 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_passed": 0, 2024-09-21T03:54:19.898 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_failed": 0, 2024-09-21T03:54:19.898 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_skipped": 0 2024-09-21T03:54:19.898 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:54:19.898 INFO:teuthology.orchestra.run.smithi039.stdout: "6e760f1f-9048-4f92-bd66-d708fbef75b3": { 2024-09-21T03:54:19.898 INFO:teuthology.orchestra.run.smithi039.stdout: "start_time": "2024-09-21T03:50:03.711918+0000", 2024-09-21T03:54:19.898 INFO:teuthology.orchestra.run.smithi039.stdout: "path": "/", 2024-09-21T03:54:19.899 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_started": 0, 2024-09-21T03:54:19.899 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_passed": 0, 2024-09-21T03:54:19.899 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_failed": 0, 2024-09-21T03:54:19.899 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_skipped": 0 2024-09-21T03:54:19.899 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:54:19.899 INFO:teuthology.orchestra.run.smithi039.stdout: "c7e2b8e4-057d-435f-b2bd-b633de98035f": { 2024-09-21T03:54:19.899 INFO:teuthology.orchestra.run.smithi039.stdout: "start_time": "2024-09-21T03:47:19.400305+0000", 2024-09-21T03:54:19.899 INFO:teuthology.orchestra.run.smithi039.stdout: "path": "/", 2024-09-21T03:54:19.899 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_started": 0, 2024-09-21T03:54:19.899 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_passed": 0, 2024-09-21T03:54:19.899 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_failed": 0, 2024-09-21T03:54:19.899 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_skipped": 0 2024-09-21T03:54:19.899 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:54:19.899 INFO:teuthology.orchestra.run.smithi039.stdout: "74ab917e-ecea-4778-a1c3-c184214fb24c": { 2024-09-21T03:54:19.900 INFO:teuthology.orchestra.run.smithi039.stdout: "start_time": "2024-09-21T03:50:17.726630+0000", 2024-09-21T03:54:19.900 INFO:teuthology.orchestra.run.smithi039.stdout: "path": "/", 2024-09-21T03:54:19.900 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_started": 0, 2024-09-21T03:54:19.900 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_passed": 0, 2024-09-21T03:54:19.900 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_failed": 0, 2024-09-21T03:54:19.900 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_skipped": 0 2024-09-21T03:54:19.900 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:54:19.900 INFO:teuthology.orchestra.run.smithi039.stdout: "160750d8-401f-4810-866d-e63e7e4bc3e0": { 2024-09-21T03:54:19.900 INFO:teuthology.orchestra.run.smithi039.stdout: "start_time": "2024-09-21T03:51:04.208673+0000", 2024-09-21T03:54:19.900 INFO:teuthology.orchestra.run.smithi039.stdout: "path": "/", 2024-09-21T03:54:19.900 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_started": 0, 2024-09-21T03:54:19.900 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_passed": 0, 2024-09-21T03:54:19.900 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_failed": 0, 2024-09-21T03:54:19.900 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_skipped": 0 2024-09-21T03:54:19.900 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:54:19.900 INFO:teuthology.orchestra.run.smithi039.stdout: "ae1fc4cf-5458-47d0-afcb-282b136dba15": { 2024-09-21T03:54:19.900 INFO:teuthology.orchestra.run.smithi039.stdout: "start_time": "2024-09-21T03:51:18.560378+0000", 2024-09-21T03:54:19.901 INFO:teuthology.orchestra.run.smithi039.stdout: "path": "/", 2024-09-21T03:54:19.901 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_started": 0, 2024-09-21T03:54:19.901 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_passed": 0, 2024-09-21T03:54:19.901 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_failed": 0, 2024-09-21T03:54:19.901 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_skipped": 0 2024-09-21T03:54:19.901 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:54:19.901 INFO:teuthology.orchestra.run.smithi039.stdout: "4bda6ee3-7237-4ee4-96ce-150ad50f26dc": { 2024-09-21T03:54:19.901 INFO:teuthology.orchestra.run.smithi039.stdout: "start_time": "2024-09-21T03:52:18.622141+0000", 2024-09-21T03:54:19.901 INFO:teuthology.orchestra.run.smithi039.stdout: "path": "/", 2024-09-21T03:54:19.901 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_started": 0, 2024-09-21T03:54:19.901 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_passed": 0, 2024-09-21T03:54:19.901 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_failed": 0, 2024-09-21T03:54:19.901 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_skipped": 0 2024-09-21T03:54:19.901 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-21T03:54:19.901 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-21T03:54:19.901 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-09-21T03:54:19.902 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:19.886+0000 7f6193718640 1 -- 172.21.15.39:0/1435519951 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f61780b6680 msgr2=0x7f61780b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:54:19.902 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:19.886+0000 7f6193718640 1 --2- 172.21.15.39:0/1435519951 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f61780b6680 0x7f61780b8b90 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f618402f500 tx=0x7f618402f490 comp rx=0 tx=0).stop 2024-09-21T03:54:19.902 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:19.886+0000 7f6193718640 1 -- 172.21.15.39:0/1435519951 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f618c1048c0 msgr2=0x7f618c19a450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:54:19.902 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:19.886+0000 7f6193718640 1 --2- 172.21.15.39:0/1435519951 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f618c1048c0 0x7f618c19a450 secure :-1 s=READY pgs=1135 cs=0 l=1 rev1=1 crypto rx=0x7f618800d950 tx=0x7f618800de10 comp rx=0 tx=0).stop 2024-09-21T03:54:19.902 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:19.886+0000 7f6193718640 1 -- 172.21.15.39:0/1435519951 shutdown_connections 2024-09-21T03:54:19.903 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:19.886+0000 7f6193718640 1 --2- 172.21.15.39:0/1435519951 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f61780b6680 0x7f61780b8b90 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:19.903 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:19.886+0000 7f6193718640 1 --2- 172.21.15.39:0/1435519951 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f618c10de70 0x7f618c19fc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:19.903 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:19.886+0000 7f6193718640 1 --2- 172.21.15.39:0/1435519951 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f618c1052f0 0x7f618c19a9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:19.903 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:19.886+0000 7f6193718640 1 --2- 172.21.15.39:0/1435519951 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f618c1048c0 0x7f618c19a450 unknown :-1 s=CLOSED pgs=1135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:19.903 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:19.886+0000 7f6193718640 1 -- 172.21.15.39:0/1435519951 >> 172.21.15.39:0/1435519951 conn(0x7f618c079790 msgr2=0x7f618c1079a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:54:19.903 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:19.886+0000 7f6193718640 1 -- 172.21.15.39:0/1435519951 shutdown_connections 2024-09-21T03:54:19.903 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:19.886+0000 7f6193718640 1 -- 172.21.15.39:0/1435519951 wait complete. 2024-09-21T03:54:20.760 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-21T03:54:20.760 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 51116d5c-77ca-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T03:54:21.026 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/51116d5c-77ca-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:54:21.084 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:54:20 smithi039 ceph-mon[31211]: pgmap v577: 97 pgs: 97 active+clean; 10 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 204 B/s wr, 10 op/s 2024-09-21T03:54:21.084 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:54:20 smithi039 ceph-mon[31211]: osdmap e83: 12 total, 12 up, 12 in 2024-09-21T03:54:21.084 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:54:20 smithi039 ceph-mon[31211]: scrub summary: idle 2024-09-21T03:54:21.218 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:54:20 smithi161 ceph-mon[34533]: pgmap v577: 97 pgs: 97 active+clean; 10 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 204 B/s wr, 10 op/s 2024-09-21T03:54:21.218 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:54:20 smithi161 ceph-mon[34533]: osdmap e83: 12 total, 12 up, 12 in 2024-09-21T03:54:21.218 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:54:20 smithi161 ceph-mon[34533]: scrub summary: idle 2024-09-21T03:54:21.225 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:54:20 smithi150 ceph-mon[34289]: pgmap v577: 97 pgs: 97 active+clean; 10 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 204 B/s wr, 10 op/s 2024-09-21T03:54:21.225 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:54:20 smithi150 ceph-mon[34289]: osdmap e83: 12 total, 12 up, 12 in 2024-09-21T03:54:21.225 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:54:20 smithi150 ceph-mon[34289]: scrub summary: idle 2024-09-21T03:54:21.484 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:21.473+0000 7ff12ae50640 1 -- 172.21.15.39:0/254738548 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff124105340 msgr2=0x7ff12410d890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:54:21.485 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:21.473+0000 7ff12ae50640 1 --2- 172.21.15.39:0/254738548 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff124105340 0x7ff12410d890 secure :-1 s=READY pgs=1140 cs=0 l=1 rev1=1 crypto rx=0x7ff10c009900 tx=0x7ff10c02a410 comp rx=0 tx=0).stop 2024-09-21T03:54:21.485 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:21.473+0000 7ff12ae50640 1 -- 172.21.15.39:0/254738548 shutdown_connections 2024-09-21T03:54:21.485 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:21.473+0000 7ff12ae50640 1 --2- 172.21.15.39:0/254738548 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff12410de20 0x7ff124110310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:21.485 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:21.473+0000 7ff12ae50640 1 --2- 172.21.15.39:0/254738548 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff124105340 0x7ff12410d890 unknown :-1 s=CLOSED pgs=1140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:21.485 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:21.473+0000 7ff12ae50640 1 --2- 172.21.15.39:0/254738548 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff1241049f0 0x7ff124104e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:21.485 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:21.473+0000 7ff12ae50640 1 -- 172.21.15.39:0/254738548 >> 172.21.15.39:0/254738548 conn(0x7ff124079730 msgr2=0x7ff124079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:54:21.485 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:21.473+0000 7ff12ae50640 1 -- 172.21.15.39:0/254738548 shutdown_connections 2024-09-21T03:54:21.485 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:21.473+0000 7ff12ae50640 1 -- 172.21.15.39:0/254738548 wait complete. 2024-09-21T03:54:21.485 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:21.474+0000 7ff12ae50640 1 Processor -- start 2024-09-21T03:54:21.485 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:21.474+0000 7ff12ae50640 1 -- start start 2024-09-21T03:54:21.485 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:21.474+0000 7ff12ae50640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff1241049f0 0x7ff12419a510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:54:21.485 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:21.474+0000 7ff12ae50640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff124105340 0x7ff12419aab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:54:21.485 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:21.474+0000 7ff12ae50640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff12410de20 0x7ff12419fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:54:21.486 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:21.474+0000 7ff12ae50640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7ff124113a80 con 0x7ff124105340 2024-09-21T03:54:21.486 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:21.474+0000 7ff12ae50640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7ff124113900 con 0x7ff12410de20 2024-09-21T03:54:21.486 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:21.474+0000 7ff12ae50640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7ff124113c00 con 0x7ff1241049f0 2024-09-21T03:54:21.486 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:21.475+0000 7ff123fff640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff124105340 0x7ff12419aab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:54:21.486 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:21.475+0000 7ff123fff640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff124105340 0x7ff12419aab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:58512/0 (socket says 172.21.15.39:58512) 2024-09-21T03:54:21.486 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:21.475+0000 7ff123fff640 1 -- 172.21.15.39:0/173906922 learned_addr learned my addr 172.21.15.39:0/173906922 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:54:21.486 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:21.475+0000 7ff1293ed640 1 --2- 172.21.15.39:0/173906922 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff12410de20 0x7ff12419fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:54:21.486 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:21.475+0000 7ff128bec640 1 --2- 172.21.15.39:0/173906922 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff1241049f0 0x7ff12419a510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:54:21.486 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:21.475+0000 7ff123fff640 1 -- 172.21.15.39:0/173906922 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff1241049f0 msgr2=0x7ff12419a510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:54:21.486 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:21.475+0000 7ff123fff640 1 --2- 172.21.15.39:0/173906922 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff1241049f0 0x7ff12419a510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:21.486 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:21.475+0000 7ff123fff640 1 -- 172.21.15.39:0/173906922 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff12410de20 msgr2=0x7ff12419fd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:54:21.486 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:21.475+0000 7ff123fff640 1 --2- 172.21.15.39:0/173906922 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff12410de20 0x7ff12419fd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:21.487 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:21.475+0000 7ff123fff640 1 -- 172.21.15.39:0/173906922 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff1241a03c0 con 0x7ff124105340 2024-09-21T03:54:21.487 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:21.475+0000 7ff128bec640 1 --2- 172.21.15.39:0/173906922 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff1241049f0 0x7ff12419a510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:54:21.487 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:21.475+0000 7ff1293ed640 1 --2- 172.21.15.39:0/173906922 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff12410de20 0x7ff12419fd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:54:21.487 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:21.475+0000 7ff123fff640 1 --2- 172.21.15.39:0/173906922 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff124105340 0x7ff12419aab0 secure :-1 s=READY pgs=1141 cs=0 l=1 rev1=1 crypto rx=0x7ff10c02a930 tx=0x7ff10c004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:54:21.487 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:21.475+0000 7ff121ffb640 1 -- 172.21.15.39:0/173906922 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff10c03f030 con 0x7ff124105340 2024-09-21T03:54:21.487 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:21.475+0000 7ff121ffb640 1 -- 172.21.15.39:0/173906922 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff10c003aa0 con 0x7ff124105340 2024-09-21T03:54:21.487 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:21.475+0000 7ff12ae50640 1 -- 172.21.15.39:0/173906922 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff1241a0650 con 0x7ff124105340 2024-09-21T03:54:21.487 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:21.475+0000 7ff121ffb640 1 -- 172.21.15.39:0/173906922 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff10c035070 con 0x7ff124105340 2024-09-21T03:54:21.487 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:21.476+0000 7ff12ae50640 1 -- 172.21.15.39:0/173906922 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff1241a0b80 con 0x7ff124105340 2024-09-21T03:54:21.487 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:21.477+0000 7ff12ae50640 1 -- 172.21.15.39:0/173906922 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff124104e00 con 0x7ff124105340 2024-09-21T03:54:21.487 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:21.479+0000 7ff121ffb640 1 -- 172.21.15.39:0/173906922 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 27) ==== 154643+0+0 (secure 0 0 0) 0x7ff10c042030 con 0x7ff124105340 2024-09-21T03:54:21.487 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:21.480+0000 7ff121ffb640 1 --2- 172.21.15.39:0/173906922 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7ff1100b6730 0x7ff1100b8c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:54:21.487 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:21.480+0000 7ff128bec640 1 --2- 172.21.15.39:0/173906922 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7ff1100b6730 0x7ff1100b8c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:54:21.487 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:21.480+0000 7ff121ffb640 1 -- 172.21.15.39:0/173906922 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8528+0+0 (secure 0 0 0) 0x7ff10c108230 con 0x7ff124105340 2024-09-21T03:54:21.488 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:21.480+0000 7ff128bec640 1 --2- 172.21.15.39:0/173906922 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7ff1100b6730 0x7ff1100b8c40 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7ff12406bc20 tx=0x7ff11c005db0 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:54:21.488 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:21.484+0000 7ff121ffb640 1 -- 172.21.15.39:0/173906922 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7ff1241a0b80 con 0x7ff124105340 2024-09-21T03:54:21.712 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:21.709+0000 7ff12ae50640 1 -- 172.21.15.39:0/173906922 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff12419b640 con 0x7ff124105340 2024-09-21T03:54:21.713 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:21.710+0000 7ff121ffb640 1 -- 172.21.15.39:0/173906922 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11978 (secure 0 0 0) 0x7ff10c0ce780 con 0x7ff124105340 2024-09-21T03:54:21.713 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T03:54:21.713 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":29,"btime":"2024-09-21T03:46:08:982881+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14790,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6835/508995295","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":508995295},{"type":"v1","addr":"172.21.15.39:6835","nonce":508995295}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14802,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6837/3915912607","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":3915912607},{"type":"v1","addr":"172.21.15.39:6837","nonce":3915912607}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":29,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T03:44:15.009872+0000","modified":"2024-09-21T03:46:08.430113+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24695,"mds_1":24667,"mds_2":24659,"mds_3":24646,"mds_4":14868},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14856":{"gid":14856,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.39:6839/185509192","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6838","nonce":185509192},{"type":"v1","addr":"172.21.15.39:6839","nonce":185509192}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14868":{"gid":14868,"name":"a","rank":4,"incarnation":25,"state":"up:active","state_seq":7,"addr":"172.21.15.39:6841/4247190456","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6840","nonce":4247190456},{"type":"v1","addr":"172.21.15.39:6841","nonce":4247190456}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24599":{"gid":24599,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.150:6833/3114643217","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":3114643217},{"type":"v1","addr":"172.21.15.150:6833","nonce":3114643217}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24601":{"gid":24601,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.161:6833/2963990654","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":2963990654},{"type":"v1","addr":"172.21.15.161:6833","nonce":2963990654}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24632":{"gid":24632,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.150:6835/724594008","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6834","nonce":724594008},{"type":"v1","addr":"172.21.15.150:6835","nonce":724594008}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24640":{"gid":24640,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.161:6835/3747882110","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":3747882110},{"type":"v1","addr":"172.21.15.161:6835","nonce":3747882110}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24646":{"gid":24646,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":10,"addr":"172.21.15.161:6837/3451544597","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6836","nonce":3451544597},{"type":"v1","addr":"172.21.15.161:6837","nonce":3451544597}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24659":{"gid":24659,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.150:6837/4017043144","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6836","nonce":4017043144},{"type":"v1","addr":"172.21.15.150:6837","nonce":4017043144}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24667":{"gid":24667,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.161:6839/2190107270","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6838","nonce":2190107270},{"type":"v1","addr":"172.21.15.161:6839","nonce":2190107270}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24695":{"gid":24695,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.150:6839/2949386907","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6838","nonce":2949386907},{"type":"v1","addr":"172.21.15.150:6839","nonce":2949386907}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24695,"qdb_cluster":[24695,24667,24659,24646,14868]},"id":1}]} 2024-09-21T03:54:21.713 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 29 2024-09-21T03:54:21.715 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:21.712+0000 7ff12ae50640 1 -- 172.21.15.39:0/173906922 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7ff1100b6730 msgr2=0x7ff1100b8c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:54:21.715 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:21.712+0000 7ff12ae50640 1 --2- 172.21.15.39:0/173906922 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7ff1100b6730 0x7ff1100b8c40 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7ff12406bc20 tx=0x7ff11c005db0 comp rx=0 tx=0).stop 2024-09-21T03:54:21.715 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:21.712+0000 7ff12ae50640 1 -- 172.21.15.39:0/173906922 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff124105340 msgr2=0x7ff12419aab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:54:21.715 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:21.712+0000 7ff12ae50640 1 --2- 172.21.15.39:0/173906922 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff124105340 0x7ff12419aab0 secure :-1 s=READY pgs=1141 cs=0 l=1 rev1=1 crypto rx=0x7ff10c02a930 tx=0x7ff10c004270 comp rx=0 tx=0).stop 2024-09-21T03:54:21.716 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:21.712+0000 7ff12ae50640 1 -- 172.21.15.39:0/173906922 shutdown_connections 2024-09-21T03:54:21.716 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:21.712+0000 7ff12ae50640 1 --2- 172.21.15.39:0/173906922 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7ff1100b6730 0x7ff1100b8c40 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:21.716 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:21.713+0000 7ff12ae50640 1 --2- 172.21.15.39:0/173906922 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff12410de20 0x7ff12419fd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:21.716 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:21.713+0000 7ff12ae50640 1 --2- 172.21.15.39:0/173906922 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff124105340 0x7ff12419aab0 unknown :-1 s=CLOSED pgs=1141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:21.716 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:21.713+0000 7ff12ae50640 1 --2- 172.21.15.39:0/173906922 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff1241049f0 0x7ff12419a510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:21.716 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:21.713+0000 7ff12ae50640 1 -- 172.21.15.39:0/173906922 >> 172.21.15.39:0/173906922 conn(0x7ff124079730 msgr2=0x7ff124100840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:54:21.716 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:21.713+0000 7ff12ae50640 1 -- 172.21.15.39:0/173906922 shutdown_connections 2024-09-21T03:54:21.716 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:21.713+0000 7ff12ae50640 1 -- 172.21.15.39:0/173906922 wait complete. 2024-09-21T03:54:22.218 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:54:21 smithi161 ceph-mon[34533]: from='client.? 172.21.15.39:0/173906922' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:54:22.225 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:54:21 smithi150 ceph-mon[34289]: from='client.? 172.21.15.39:0/173906922' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:54:22.334 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:54:21 smithi039 ceph-mon[31211]: from='client.? 172.21.15.39:0/173906922' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:54:22.413 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 51116d5c-77ca-11ef-baf5-efdc52797490 -- ceph tell mds.1:4 damage ls 2024-09-21T03:54:22.672 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/51116d5c-77ca-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:54:23.034 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:54:22 smithi039 ceph-mon[31211]: pgmap v579: 97 pgs: 97 active+clean; 10 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 204 B/s wr, 10 op/s 2024-09-21T03:54:23.134 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:23.125+0000 7fea3806c640 1 -- 172.21.15.39:0/745961904 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fea301094b0 msgr2=0x7fea301098c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:54:23.135 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:23.125+0000 7fea3806c640 1 --2- 172.21.15.39:0/745961904 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fea301094b0 0x7fea301098c0 secure :-1 s=READY pgs=1142 cs=0 l=1 rev1=1 crypto rx=0x7fea24009900 tx=0x7fea2402a410 comp rx=0 tx=0).stop 2024-09-21T03:54:23.135 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:23.125+0000 7fea3806c640 1 -- 172.21.15.39:0/745961904 shutdown_connections 2024-09-21T03:54:23.135 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:23.125+0000 7fea3806c640 1 --2- 172.21.15.39:0/745961904 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fea30109e00 0x7fea30110630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:23.135 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:23.125+0000 7fea3806c640 1 --2- 172.21.15.39:0/745961904 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fea301094b0 0x7fea301098c0 unknown :-1 s=CLOSED pgs=1142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:23.135 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:23.125+0000 7fea3806c640 1 --2- 172.21.15.39:0/745961904 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fea30103bc0 0x7fea30103fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:23.135 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:23.125+0000 7fea3806c640 1 -- 172.21.15.39:0/745961904 >> 172.21.15.39:0/745961904 conn(0x7fea3007b910 msgr2=0x7fea3007bd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:54:23.135 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:23.125+0000 7fea3806c640 1 -- 172.21.15.39:0/745961904 shutdown_connections 2024-09-21T03:54:23.135 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:23.125+0000 7fea3806c640 1 -- 172.21.15.39:0/745961904 wait complete. 2024-09-21T03:54:23.135 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:23.126+0000 7fea3806c640 1 Processor -- start 2024-09-21T03:54:23.135 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:23.126+0000 7fea3806c640 1 -- start start 2024-09-21T03:54:23.136 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:23.126+0000 7fea3806c640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fea30103bc0 0x7fea3019a4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:54:23.136 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:23.126+0000 7fea3806c640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fea301094b0 0x7fea3019aa70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:54:23.136 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:23.126+0000 7fea3806c640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fea30109e00 0x7fea3019fd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:54:23.136 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:23.126+0000 7fea3806c640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fea30113ac0 con 0x7fea30103bc0 2024-09-21T03:54:23.136 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:23.126+0000 7fea3806c640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fea30113940 con 0x7fea30109e00 2024-09-21T03:54:23.136 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:23.126+0000 7fea3806c640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fea30113c40 con 0x7fea301094b0 2024-09-21T03:54:23.136 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:23.126+0000 7fea35e08640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fea30103bc0 0x7fea3019a4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:54:23.136 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:23.126+0000 7fea35e08640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fea30103bc0 0x7fea3019a4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:58534/0 (socket says 172.21.15.39:58534) 2024-09-21T03:54:23.136 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:23.126+0000 7fea35e08640 1 -- 172.21.15.39:0/2899232765 learned_addr learned my addr 172.21.15.39:0/2899232765 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:54:23.136 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:23.126+0000 7fea35607640 1 --2- 172.21.15.39:0/2899232765 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fea301094b0 0x7fea3019aa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:54:23.136 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:23.126+0000 7fea36609640 1 --2- 172.21.15.39:0/2899232765 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fea30109e00 0x7fea3019fd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:54:23.136 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:23.127+0000 7fea35e08640 1 -- 172.21.15.39:0/2899232765 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fea301094b0 msgr2=0x7fea3019aa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:54:23.136 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:23.127+0000 7fea35e08640 1 --2- 172.21.15.39:0/2899232765 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fea301094b0 0x7fea3019aa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:23.137 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:23.127+0000 7fea35e08640 1 -- 172.21.15.39:0/2899232765 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fea30109e00 msgr2=0x7fea3019fd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:54:23.137 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:23.127+0000 7fea35e08640 1 --2- 172.21.15.39:0/2899232765 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fea30109e00 0x7fea3019fd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:23.137 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:23.127+0000 7fea35e08640 1 -- 172.21.15.39:0/2899232765 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fea301a0440 con 0x7fea30103bc0 2024-09-21T03:54:23.137 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:23.127+0000 7fea36609640 1 --2- 172.21.15.39:0/2899232765 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fea30109e00 0x7fea3019fd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:54:23.137 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:23.127+0000 7fea35607640 1 --2- 172.21.15.39:0/2899232765 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fea301094b0 0x7fea3019aa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:54:23.137 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:23.127+0000 7fea35e08640 1 --2- 172.21.15.39:0/2899232765 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fea30103bc0 0x7fea3019a4d0 secure :-1 s=READY pgs=1143 cs=0 l=1 rev1=1 crypto rx=0x7fea2800d950 tx=0x7fea2800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:54:23.137 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:23.127+0000 7fea2effd640 1 -- 172.21.15.39:0/2899232765 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea28017070 con 0x7fea30103bc0 2024-09-21T03:54:23.137 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:23.127+0000 7fea2effd640 1 -- 172.21.15.39:0/2899232765 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fea28009c30 con 0x7fea30103bc0 2024-09-21T03:54:23.137 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:23.127+0000 7fea3806c640 1 -- 172.21.15.39:0/2899232765 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fea301a0730 con 0x7fea30103bc0 2024-09-21T03:54:23.137 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:23.127+0000 7fea2effd640 1 -- 172.21.15.39:0/2899232765 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea2800f620 con 0x7fea30103bc0 2024-09-21T03:54:23.137 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:23.127+0000 7fea3806c640 1 -- 172.21.15.39:0/2899232765 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fea301a6e90 con 0x7fea30103bc0 2024-09-21T03:54:23.137 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:23.130+0000 7fea3e5e0740 1 Processor -- start 2024-09-21T03:54:23.137 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:23.130+0000 7fea3e5e0740 1 -- start start 2024-09-21T03:54:23.137 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:23.130+0000 7fea2effd640 1 -- 172.21.15.39:0/2899232765 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 27) ==== 154643+0+0 (secure 0 0 0) 0x7fea28010460 con 0x7fea30103bc0 2024-09-21T03:54:23.137 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:23.130+0000 7fea3e5e0740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556ea9ed1070 0x556eaa2786a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:54:23.138 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:23.130+0000 7fea3e5e0740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556ea9ea81b0 0x556eaa278be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:54:23.138 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:23.130+0000 7fea3e5e0740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556ea9eb0390 0x556eaa279120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:54:23.138 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:23.130+0000 7fea3e5e0740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x556eaa1eeea0 con 0x556ea9eb0390 2024-09-21T03:54:23.138 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:23.130+0000 7fea3e5e0740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x556eaa1de2d0 con 0x556ea9ea81b0 2024-09-21T03:54:23.138 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:23.130+0000 7fea3e5e0740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x556eaa1e20e0 con 0x556ea9ed1070 2024-09-21T03:54:23.138 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:23.130+0000 7fea35607640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556ea9eb0390 0x556eaa279120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:54:23.138 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:23.130+0000 7fea35607640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556ea9eb0390 0x556eaa279120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:58542/0 (socket says 172.21.15.39:58542) 2024-09-21T03:54:23.138 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:23.130+0000 7fea35607640 1 -- 172.21.15.39:0/2377824599 learned_addr learned my addr 172.21.15.39:0/2377824599 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:54:23.138 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:23.130+0000 7fea36609640 1 --2- 172.21.15.39:0/2377824599 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556ea9ed1070 0x556eaa2786a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:54:23.138 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:23.130+0000 7fea35e08640 1 --2- 172.21.15.39:0/2377824599 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556ea9ea81b0 0x556eaa278be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:54:23.138 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:23.131+0000 7fea35607640 1 -- 172.21.15.39:0/2377824599 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556ea9ed1070 msgr2=0x556eaa2786a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:54:23.138 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:23.131+0000 7fea35607640 1 --2- 172.21.15.39:0/2377824599 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556ea9ed1070 0x556eaa2786a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:23.138 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:23.131+0000 7fea35607640 1 -- 172.21.15.39:0/2377824599 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556ea9ea81b0 msgr2=0x556eaa278be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:54:23.138 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:23.131+0000 7fea35607640 1 --2- 172.21.15.39:0/2377824599 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556ea9ea81b0 0x556eaa278be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:23.139 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:23.131+0000 7fea35607640 1 -- 172.21.15.39:0/2377824599 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556ea9f3c650 con 0x556ea9eb0390 2024-09-21T03:54:23.139 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:23.131+0000 7fea2effd640 1 --2- 172.21.15.39:0/2899232765 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7fea180b6700 0x7fea180b8c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:54:23.139 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:23.131+0000 7fea35607640 1 --2- 172.21.15.39:0/2377824599 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556ea9eb0390 0x556eaa279120 secure :-1 s=READY pgs=1144 cs=0 l=1 rev1=1 crypto rx=0x7fea24031440 tx=0x7fea24032d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=706b99a08eb94574 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:54:23.139 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:23.131+0000 7fea36609640 1 --2- 172.21.15.39:0/2899232765 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7fea180b6700 0x7fea180b8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:54:23.139 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:23.131+0000 7fea2effd640 1 -- 172.21.15.39:0/2899232765 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8528+0+0 (secure 0 0 0) 0x7fea280e73b0 con 0x7fea30103bc0 2024-09-21T03:54:23.139 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:23.131+0000 7fea17fff640 1 -- 172.21.15.39:0/2377824599 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea2403f070 con 0x556ea9eb0390 2024-09-21T03:54:23.139 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:23.132+0000 7fea17fff640 1 -- 172.21.15.39:0/2377824599 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fea240316a0 con 0x556ea9eb0390 2024-09-21T03:54:23.139 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:23.132+0000 7fea17fff640 1 -- 172.21.15.39:0/2377824599 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea24035070 con 0x556ea9eb0390 2024-09-21T03:54:23.142 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:23.132+0000 7fea3e5e0740 1 -- 172.21.15.39:0/2377824599 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556ea9eb0390 msgr2=0x556eaa279120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:54:23.142 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:23.132+0000 7fea36609640 1 --2- 172.21.15.39:0/2899232765 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7fea180b6700 0x7fea180b8c10 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7fea20005fd0 tx=0x7fea20009680 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:54:23.142 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:23.132+0000 7fea3e5e0740 1 --2- 172.21.15.39:0/2377824599 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556ea9eb0390 0x556eaa279120 secure :-1 s=READY pgs=1144 cs=0 l=1 rev1=1 crypto rx=0x7fea24031440 tx=0x7fea24032d00 comp rx=0 tx=0).stop 2024-09-21T03:54:23.142 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:23.132+0000 7fea3e5e0740 1 -- 172.21.15.39:0/2377824599 shutdown_connections 2024-09-21T03:54:23.142 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:23.133+0000 7fea3e5e0740 1 --2- 172.21.15.39:0/2377824599 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556ea9eb0390 0x556eaa279120 unknown :-1 s=CLOSED pgs=1144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:23.142 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:23.133+0000 7fea3e5e0740 1 --2- 172.21.15.39:0/2377824599 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556ea9ea81b0 0x556eaa278be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:23.142 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:23.133+0000 7fea3e5e0740 1 --2- 172.21.15.39:0/2377824599 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556ea9ed1070 0x556eaa2786a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:23.142 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:23.133+0000 7fea3e5e0740 1 -- 172.21.15.39:0/2377824599 >> 172.21.15.39:0/2377824599 conn(0x556ea9ec5460 msgr2=0x556eaa276590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:54:23.142 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:23.133+0000 7fea3e5e0740 1 -- 172.21.15.39:0/2377824599 shutdown_connections 2024-09-21T03:54:23.142 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:23.133+0000 7fea3e5e0740 1 -- 172.21.15.39:0/2377824599 wait complete. 2024-09-21T03:54:23.142 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:23.133+0000 7fea3e5e0740 1 -- start start 2024-09-21T03:54:23.143 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:23.133+0000 7fea3e5e0740 1 Processor -- start 2024-09-21T03:54:23.143 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:23.134+0000 7fea3e5e0740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556ea9ea81b0 0x556eaa283cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:54:23.143 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:23.134+0000 7fea3e5e0740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556ea9eb0390 0x556eaa284200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:54:23.143 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:23.134+0000 7fea3e5e0740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556ea9ed1070 0x556eaa284760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:54:23.143 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:23.134+0000 7fea3e5e0740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x556eaa03f700 con 0x556ea9eb0390 2024-09-21T03:54:23.143 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:23.134+0000 7fea3e5e0740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x556eaa179160 con 0x556ea9ea81b0 2024-09-21T03:54:23.143 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:23.134+0000 7fea3e5e0740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x556eaa286e20 con 0x556ea9ed1070 2024-09-21T03:54:23.143 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:23.134+0000 7fea3e5e0740 10 client.0 _ll_register_callbacks cb 0x556eaa1929e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:54:23.143 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:23.134+0000 7fea35607640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556ea9eb0390 0x556eaa284200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:54:23.143 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:23.134+0000 7fea35607640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556ea9eb0390 0x556eaa284200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:58556/0 (socket says 172.21.15.39:58556) 2024-09-21T03:54:23.143 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:23.134+0000 7fea35607640 1 -- 172.21.15.39:0/1587978351 learned_addr learned my addr 172.21.15.39:0/1587978351 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:54:23.143 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:23.134+0000 7fea35e08640 1 --2- 172.21.15.39:0/1587978351 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556ea9ea81b0 0x556eaa283cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:54:23.143 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:23.134+0000 7fea36609640 1 --2- 172.21.15.39:0/1587978351 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556ea9ed1070 0x556eaa284760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:54:23.143 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:23.134+0000 7fea35607640 1 -- 172.21.15.39:0/1587978351 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556ea9ed1070 msgr2=0x556eaa284760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:54:23.143 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:23.134+0000 7fea35607640 1 --2- 172.21.15.39:0/1587978351 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556ea9ed1070 0x556eaa284760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:23.144 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:23.134+0000 7fea35607640 1 -- 172.21.15.39:0/1587978351 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556ea9ea81b0 msgr2=0x556eaa283cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:54:23.144 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:23.134+0000 7fea35607640 1 --2- 172.21.15.39:0/1587978351 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556ea9ea81b0 0x556eaa283cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:23.144 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:23.134+0000 7fea35607640 1 -- 172.21.15.39:0/1587978351 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556eaa1a6c90 con 0x556ea9eb0390 2024-09-21T03:54:23.144 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:23.134+0000 7fea35e08640 1 --2- 172.21.15.39:0/1587978351 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556ea9ea81b0 0x556eaa283cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:54:23.144 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:23.134+0000 7fea35607640 1 --2- 172.21.15.39:0/1587978351 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556ea9eb0390 0x556eaa284200 secure :-1 s=READY pgs=1145 cs=0 l=1 rev1=1 crypto rx=0x7fea24004620 tx=0x7fea240314a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb54641e55a46041 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:54:23.144 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:23.134+0000 7fea157fa640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-09-21T03:54:23.144 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:23.134+0000 7fea157fa640 1 -- 172.21.15.39:0/1587978351 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea2404a020 con 0x556ea9eb0390 2024-09-21T03:54:23.144 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:23.134+0000 7fea3e5e0740 1 -- 172.21.15.39:0/1587978351 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fea240316a0 con 0x556ea9eb0390 2024-09-21T03:54:23.144 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:23.134+0000 7fea157fa640 1 -- 172.21.15.39:0/1587978351 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8528+0+0 (secure 0 0 0) 0x7fea240035f0 con 0x556ea9eb0390 2024-09-21T03:54:23.144 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:23.135+0000 7fea157fa640 1 -- 172.21.15.39:0/1587978351 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fea24055050 con 0x556ea9eb0390 2024-09-21T03:54:23.144 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:23.135+0000 7fea3e5e0740 10 client.17727 fetch_fsmap learned FSMap version 29 2024-09-21T03:54:23.144 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:23.135+0000 7fea3e5e0740 1 -- 172.21.15.39:0/1587978351 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x556eaa292aa0 con 0x556ea9eb0390 2024-09-21T03:54:23.144 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:23.135+0000 7fea157fa640 1 -- 172.21.15.39:0/1587978351 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fea240316a0 con 0x556ea9eb0390 2024-09-21T03:54:23.144 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:23.135+0000 7fea157fa640 1 -- 172.21.15.39:0/1587978351 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fea2403b7d0 con 0x556ea9eb0390 2024-09-21T03:54:23.144 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:23.135+0000 7fea3e5e0740 10 client.17727 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T03:54:23.144 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:23.135+0000 7fea3e5e0740 10 client.17727 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.a 2024-09-21T03:54:23.145 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:23.135+0000 7fea3e5e0740 20 client.17727 populate_metadata read hostname 'smithi039' 2024-09-21T03:54:23.145 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:23.135+0000 7fea3e5e0740 1 --2- 172.21.15.39:0/1587978351 >> [v2:172.21.15.39:6840/4247190456,v1:172.21.15.39:6841/4247190456] conn(0x556ea9e8f9f0 0x556eaa2960e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:54:23.145 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:23.135+0000 7fea3e5e0740 4 client.17727 mds_command: new command op to 14868 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:54:23.145 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:23.135+0000 7fea3e5e0740 1 -- 172.21.15.39:0/1587978351 --> [v2:172.21.15.39:6840/4247190456,v1:172.21.15.39:6841/4247190456] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556eaa1f0df0 con 0x556ea9e8f9f0 2024-09-21T03:54:23.145 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:23.135+0000 7fea35e08640 1 --2- 172.21.15.39:0/1587978351 >> [v2:172.21.15.39:6840/4247190456,v1:172.21.15.39:6841/4247190456] conn(0x556ea9e8f9f0 0x556eaa2960e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:54:23.145 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:23.136+0000 7fea157fa640 10 client.17727 ms_handle_connect on v2:172.21.15.39:6840/4247190456 2024-09-21T03:54:23.145 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:23.136+0000 7fea35e08640 1 --2- 172.21.15.39:0/1587978351 >> [v2:172.21.15.39:6840/4247190456,v1:172.21.15.39:6841/4247190456] conn(0x556ea9e8f9f0 0x556eaa2960e0 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=48ea6723868a4c27 server_cookie=3b163ffb772cf11b in_seq=0 out_seq=0 2024-09-21T03:54:23.145 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:23.136+0000 7fea157fa640 1 -- 172.21.15.39:0/1587978351 <== mds.4 v2:172.21.15.39:6840/4247190456 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x556eaa1f0df0 con 0x556ea9e8f9f0 2024-09-21T03:54:23.145 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:23.136+0000 7fea157fa640 10 client.17727 handle_command_reply: tid=0 2024-09-21T03:54:23.145 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:23.136+0000 7fea3e5e0740 1 client.17727 shutdown 2024-09-21T03:54:23.145 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:23.137+0000 7fea3e5e0740 1 -- 172.21.15.39:0/1587978351 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556ea9eb0390 msgr2=0x556eaa284200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:54:23.145 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:23.137+0000 7fea3e5e0740 1 --2- 172.21.15.39:0/1587978351 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556ea9eb0390 0x556eaa284200 secure :-1 s=READY pgs=1145 cs=0 l=1 rev1=1 crypto rx=0x7fea24004620 tx=0x7fea240314a0 comp rx=0 tx=0).stop 2024-09-21T03:54:23.145 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:23.137+0000 7fea3e5e0740 1 -- 172.21.15.39:0/1587978351 shutdown_connections 2024-09-21T03:54:23.146 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:23.137+0000 7fea3e5e0740 1 --2- 172.21.15.39:0/1587978351 >> [v2:172.21.15.39:6840/4247190456,v1:172.21.15.39:6841/4247190456] conn(0x556ea9e8f9f0 0x556eaa2960e0 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:23.146 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:23.137+0000 7fea3e5e0740 1 --2- 172.21.15.39:0/1587978351 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556ea9ed1070 0x556eaa284760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:23.146 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:23.137+0000 7fea3e5e0740 1 --2- 172.21.15.39:0/1587978351 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556ea9eb0390 0x556eaa284200 unknown :-1 s=CLOSED pgs=1145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:23.146 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:23.137+0000 7fea3e5e0740 1 --2- 172.21.15.39:0/1587978351 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556ea9ea81b0 0x556eaa283cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:23.146 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:23.137+0000 7fea3e5e0740 1 -- 172.21.15.39:0/1587978351 >> 172.21.15.39:0/1587978351 conn(0x556ea9ec5460 msgr2=0x556eaa27b830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:54:23.146 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:23.137+0000 7fea157fa640 0 client.17727 ms_handle_reset on v2:172.21.15.39:6840/4247190456 2024-09-21T03:54:23.146 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:23.137+0000 7fea3e5e0740 1 -- 172.21.15.39:0/1587978351 shutdown_connections 2024-09-21T03:54:23.146 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:23.137+0000 7fea3e5e0740 1 -- 172.21.15.39:0/1587978351 wait complete. 2024-09-21T03:54:23.146 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:23.137+0000 7fea3e5e0740 20 client.17727 trim_cache size 0 max 16384 2024-09-21T03:54:23.154 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:23.150+0000 7fea3e5e0740 1 Processor -- start 2024-09-21T03:54:23.154 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:23.150+0000 7fea3e5e0740 1 -- start start 2024-09-21T03:54:23.154 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:23.150+0000 7fea3e5e0740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556ea9ea81b0 0x556eaa27f470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:54:23.154 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:23.150+0000 7fea3e5e0740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556ea9eb0390 0x556eaa2858f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:54:23.154 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:23.150+0000 7fea3e5e0740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556ea9ed1070 0x556eaa275bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:54:23.154 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:23.150+0000 7fea3e5e0740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fea301af750 con 0x556ea9ea81b0 2024-09-21T03:54:23.154 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:23.150+0000 7fea3e5e0740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fea301b1290 con 0x556ea9eb0390 2024-09-21T03:54:23.154 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:23.150+0000 7fea3e5e0740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fea301afe20 con 0x556ea9ed1070 2024-09-21T03:54:23.154 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:23.150+0000 7fea35e08640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556ea9ea81b0 0x556eaa27f470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:54:23.154 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:23.150+0000 7fea35e08640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556ea9ea81b0 0x556eaa27f470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:58562/0 (socket says 172.21.15.39:58562) 2024-09-21T03:54:23.154 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:23.150+0000 7fea35e08640 1 -- 172.21.15.39:0/2126093304 learned_addr learned my addr 172.21.15.39:0/2126093304 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:54:23.155 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:23.151+0000 7fea35e08640 1 -- 172.21.15.39:0/2126093304 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556ea9ed1070 msgr2=0x556eaa275bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:54:23.155 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:23.151+0000 7fea36609640 1 --2- 172.21.15.39:0/2126093304 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556ea9ed1070 0x556eaa275bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:54:23.155 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:23.151+0000 7fea35e08640 1 --2- 172.21.15.39:0/2126093304 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556ea9ed1070 0x556eaa275bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:23.155 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:23.151+0000 7fea35607640 1 --2- 172.21.15.39:0/2126093304 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556ea9eb0390 0x556eaa2858f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:54:23.155 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:23.151+0000 7fea35e08640 1 -- 172.21.15.39:0/2126093304 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556ea9eb0390 msgr2=0x556eaa2858f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:54:23.155 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:23.151+0000 7fea35e08640 1 --2- 172.21.15.39:0/2126093304 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556ea9eb0390 0x556eaa2858f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:23.155 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:23.151+0000 7fea35e08640 1 -- 172.21.15.39:0/2126093304 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556eaa284ad0 con 0x556ea9ea81b0 2024-09-21T03:54:23.155 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:23.151+0000 7fea35e08640 1 --2- 172.21.15.39:0/2126093304 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556ea9ea81b0 0x556eaa27f470 secure :-1 s=READY pgs=1146 cs=0 l=1 rev1=1 crypto rx=0x7fea280e4aa0 tx=0x7fea280eaea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e7ceb27347142e27 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:54:23.155 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:23.151+0000 7fea177fe640 1 -- 172.21.15.39:0/2126093304 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea28020040 con 0x556ea9ea81b0 2024-09-21T03:54:23.155 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:23.151+0000 7fea177fe640 1 -- 172.21.15.39:0/2126093304 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fea280acd10 con 0x556ea9ea81b0 2024-09-21T03:54:23.155 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:23.151+0000 7fea177fe640 1 -- 172.21.15.39:0/2126093304 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea2808df90 con 0x556ea9ea81b0 2024-09-21T03:54:23.159 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:23.152+0000 7fea3e5e0740 1 -- 172.21.15.39:0/2126093304 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556ea9ea81b0 msgr2=0x556eaa27f470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:54:23.159 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:23.152+0000 7fea3e5e0740 1 --2- 172.21.15.39:0/2126093304 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556ea9ea81b0 0x556eaa27f470 secure :-1 s=READY pgs=1146 cs=0 l=1 rev1=1 crypto rx=0x7fea280e4aa0 tx=0x7fea280eaea0 comp rx=0 tx=0).stop 2024-09-21T03:54:23.159 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:23.152+0000 7fea3e5e0740 1 -- 172.21.15.39:0/2126093304 shutdown_connections 2024-09-21T03:54:23.159 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:23.152+0000 7fea3e5e0740 1 --2- 172.21.15.39:0/2126093304 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556ea9ed1070 0x556eaa275bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:23.159 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:23.152+0000 7fea3e5e0740 1 --2- 172.21.15.39:0/2126093304 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556ea9eb0390 0x556eaa2858f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:23.159 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:23.152+0000 7fea3e5e0740 1 --2- 172.21.15.39:0/2126093304 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556ea9ea81b0 0x556eaa27f470 unknown :-1 s=CLOSED pgs=1146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:23.159 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:23.152+0000 7fea3e5e0740 1 -- 172.21.15.39:0/2126093304 >> 172.21.15.39:0/2126093304 conn(0x556ea9ec5460 msgr2=0x556eaa2851e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:54:23.159 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:23.152+0000 7fea3e5e0740 1 -- 172.21.15.39:0/2126093304 shutdown_connections 2024-09-21T03:54:23.159 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:23.152+0000 7fea3e5e0740 1 -- 172.21.15.39:0/2126093304 wait complete. 2024-09-21T03:54:23.159 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:23.152+0000 7fea3e5e0740 1 -- start start 2024-09-21T03:54:23.159 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:23.153+0000 7fea3e5e0740 1 Processor -- start 2024-09-21T03:54:23.159 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:23.153+0000 7fea3e5e0740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556ea9ea81b0 0x556eaa287460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:54:23.160 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:23.153+0000 7fea3e5e0740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556ea9eb0390 0x556eaa2879a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:54:23.160 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:23.153+0000 7fea3e5e0740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556ea9ed1070 0x556eaa2fead0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:54:23.160 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:23.153+0000 7fea3e5e0740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fea301b0bc0 con 0x556ea9eb0390 2024-09-21T03:54:23.160 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:23.153+0000 7fea3e5e0740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fea301b04f0 con 0x556ea9ea81b0 2024-09-21T03:54:23.160 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:23.153+0000 7fea3e5e0740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fea301ad730 con 0x556ea9ed1070 2024-09-21T03:54:23.160 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:23.153+0000 7fea3e5e0740 10 client.0 _ll_register_callbacks cb 0x7fea280ead70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:54:23.160 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:23.153+0000 7fea35607640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556ea9eb0390 0x556eaa2879a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:54:23.160 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:23.153+0000 7fea35607640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556ea9eb0390 0x556eaa2879a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:58574/0 (socket says 172.21.15.39:58574) 2024-09-21T03:54:23.160 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:23.153+0000 7fea35607640 1 -- 172.21.15.39:0/970642850 learned_addr learned my addr 172.21.15.39:0/970642850 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:54:23.160 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:23.153+0000 7fea36609640 1 --2- 172.21.15.39:0/970642850 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556ea9ed1070 0x556eaa2fead0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:54:23.160 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:23.153+0000 7fea35e08640 1 --2- 172.21.15.39:0/970642850 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556ea9ea81b0 0x556eaa287460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:54:23.160 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:23.154+0000 7fea35607640 1 -- 172.21.15.39:0/970642850 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556ea9ed1070 msgr2=0x556eaa2fead0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:54:23.160 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:23.154+0000 7fea35607640 1 --2- 172.21.15.39:0/970642850 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556ea9ed1070 0x556eaa2fead0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:23.160 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:23.154+0000 7fea35607640 1 -- 172.21.15.39:0/970642850 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556ea9ea81b0 msgr2=0x556eaa287460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:54:23.160 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:23.154+0000 7fea35607640 1 --2- 172.21.15.39:0/970642850 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556ea9ea81b0 0x556eaa287460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:23.160 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:23.154+0000 7fea35607640 1 -- 172.21.15.39:0/970642850 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556eaa288220 con 0x556ea9eb0390 2024-09-21T03:54:23.161 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:23.154+0000 7fea35e08640 1 --2- 172.21.15.39:0/970642850 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556ea9ea81b0 0x556eaa287460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:54:23.161 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:23.154+0000 7fea35607640 1 --2- 172.21.15.39:0/970642850 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556ea9eb0390 0x556eaa2879a0 secure :-1 s=READY pgs=1147 cs=0 l=1 rev1=1 crypto rx=0x7fea2403bc70 tx=0x7fea24031720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=77ed307e0a6241ae server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:54:23.161 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:23.154+0000 7fea157fa640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-09-21T03:54:23.161 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:23.154+0000 7fea157fa640 1 -- 172.21.15.39:0/970642850 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea2404a070 con 0x556ea9eb0390 2024-09-21T03:54:23.161 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:23.154+0000 7fea157fa640 1 -- 172.21.15.39:0/970642850 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8528+0+0 (secure 0 0 0) 0x7fea2403b630 con 0x556ea9eb0390 2024-09-21T03:54:23.161 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:23.154+0000 7fea3e5e0740 1 -- 172.21.15.39:0/970642850 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fea280acd10 con 0x556ea9eb0390 2024-09-21T03:54:23.161 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:23.154+0000 7fea157fa640 1 -- 172.21.15.39:0/970642850 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fea2403be60 con 0x556ea9eb0390 2024-09-21T03:54:23.161 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:23.154+0000 7fea3e5e0740 10 client.17733 fetch_fsmap learned FSMap version 29 2024-09-21T03:54:23.161 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:23.154+0000 7fea3e5e0740 1 -- 172.21.15.39:0/970642850 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x556eaa304460 con 0x556ea9eb0390 2024-09-21T03:54:23.161 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:23.154+0000 7fea157fa640 1 -- 172.21.15.39:0/970642850 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fea280acd10 con 0x556ea9eb0390 2024-09-21T03:54:23.161 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:23.155+0000 7fea157fa640 1 -- 172.21.15.39:0/970642850 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fea24003d00 con 0x556ea9eb0390 2024-09-21T03:54:23.161 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:23.155+0000 7fea3e5e0740 10 client.17733 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T03:54:23.161 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:23.155+0000 7fea3e5e0740 10 client.17733 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.a 2024-09-21T03:54:23.161 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:23.155+0000 7fea3e5e0740 20 client.17733 populate_metadata read hostname 'smithi039' 2024-09-21T03:54:23.161 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:23.155+0000 7fea3e5e0740 1 --2- 172.21.15.39:0/970642850 >> [v2:172.21.15.39:6840/4247190456,v1:172.21.15.39:6841/4247190456] conn(0x556ea9e64da0 0x556eaa306bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:54:23.161 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:23.155+0000 7fea3e5e0740 4 client.17733 mds_command: new command op to 14868 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T03:54:23.162 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:23.155+0000 7fea3e5e0740 1 -- 172.21.15.39:0/970642850 --> [v2:172.21.15.39:6840/4247190456,v1:172.21.15.39:6841/4247190456] -- command(tid 0: {"prefix": "damage ls"}) -- 0x556eaa1ec8a0 con 0x556ea9e64da0 2024-09-21T03:54:23.162 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:23.155+0000 7fea35e08640 1 --2- 172.21.15.39:0/970642850 >> [v2:172.21.15.39:6840/4247190456,v1:172.21.15.39:6841/4247190456] conn(0x556ea9e64da0 0x556eaa306bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:54:23.162 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:23.155+0000 7fea157fa640 10 client.17733 ms_handle_connect on v2:172.21.15.39:6840/4247190456 2024-09-21T03:54:23.162 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:23.155+0000 7fea35e08640 1 --2- 172.21.15.39:0/970642850 >> [v2:172.21.15.39:6840/4247190456,v1:172.21.15.39:6841/4247190456] conn(0x556ea9e64da0 0x556eaa306bc0 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=9169668577f053ca server_cookie=19d942f0a60f08d5 in_seq=0 out_seq=0 2024-09-21T03:54:23.162 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:23.156+0000 7fea157fa640 1 -- 172.21.15.39:0/970642850 <== mds.4 v2:172.21.15.39:6840/4247190456 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x556eaa1ec8a0 con 0x556ea9e64da0 2024-09-21T03:54:23.162 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:23.156+0000 7fea157fa640 10 client.17733 handle_command_reply: tid=0 2024-09-21T03:54:23.162 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:23.156+0000 7fea3e5e0740 1 client.17733 shutdown 2024-09-21T03:54:23.162 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:23.156+0000 7fea3e5e0740 1 -- 172.21.15.39:0/970642850 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556ea9eb0390 msgr2=0x556eaa2879a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:54:23.162 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:23.156+0000 7fea3e5e0740 1 --2- 172.21.15.39:0/970642850 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556ea9eb0390 0x556eaa2879a0 secure :-1 s=READY pgs=1147 cs=0 l=1 rev1=1 crypto rx=0x7fea2403bc70 tx=0x7fea24031720 comp rx=0 tx=0).stop 2024-09-21T03:54:23.162 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:23.156+0000 7fea3e5e0740 1 -- 172.21.15.39:0/970642850 shutdown_connections 2024-09-21T03:54:23.162 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:23.156+0000 7fea3e5e0740 1 --2- 172.21.15.39:0/970642850 >> [v2:172.21.15.39:6840/4247190456,v1:172.21.15.39:6841/4247190456] conn(0x556ea9e64da0 0x556eaa306bc0 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:23.162 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:23.156+0000 7fea3e5e0740 1 --2- 172.21.15.39:0/970642850 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556ea9ed1070 0x556eaa2fead0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:23.162 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:23.156+0000 7fea3e5e0740 1 --2- 172.21.15.39:0/970642850 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556ea9eb0390 0x556eaa2879a0 unknown :-1 s=CLOSED pgs=1147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:23.162 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:23.156+0000 7fea157fa640 0 client.17733 ms_handle_reset on v2:172.21.15.39:6840/4247190456 2024-09-21T03:54:23.163 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:23.156+0000 7fea3e5e0740 1 --2- 172.21.15.39:0/970642850 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556ea9ea81b0 0x556eaa287460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:23.163 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:23.156+0000 7fea3e5e0740 1 -- 172.21.15.39:0/970642850 >> 172.21.15.39:0/970642850 conn(0x556ea9ec5460 msgr2=0x556eaa27b7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:54:23.163 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:23.156+0000 7fea3e5e0740 1 -- 172.21.15.39:0/970642850 shutdown_connections 2024-09-21T03:54:23.163 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:23.156+0000 7fea3e5e0740 1 -- 172.21.15.39:0/970642850 wait complete. 2024-09-21T03:54:23.163 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:23.156+0000 7fea3e5e0740 20 client.17733 trim_cache size 0 max 16384 2024-09-21T03:54:23.163 INFO:teuthology.orchestra.run.smithi039.stdout:[] 2024-09-21T03:54:23.164 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:23.157+0000 7fea3806c640 1 -- 172.21.15.39:0/2899232765 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7fea180b6700 msgr2=0x7fea180b8c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:54:23.164 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:23.157+0000 7fea3806c640 1 --2- 172.21.15.39:0/2899232765 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7fea180b6700 0x7fea180b8c10 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7fea20005fd0 tx=0x7fea20009680 comp rx=0 tx=0).stop 2024-09-21T03:54:23.164 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:23.157+0000 7fea3806c640 1 -- 172.21.15.39:0/2899232765 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fea30103bc0 msgr2=0x7fea3019a4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:54:23.164 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:23.157+0000 7fea3806c640 1 --2- 172.21.15.39:0/2899232765 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fea30103bc0 0x7fea3019a4d0 secure :-1 s=READY pgs=1143 cs=0 l=1 rev1=1 crypto rx=0x7fea2800d950 tx=0x7fea2800de10 comp rx=0 tx=0).stop 2024-09-21T03:54:23.164 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:23.157+0000 7fea3806c640 1 -- 172.21.15.39:0/2899232765 shutdown_connections 2024-09-21T03:54:23.164 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:23.157+0000 7fea3806c640 1 --2- 172.21.15.39:0/2899232765 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7fea180b6700 0x7fea180b8c10 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:23.164 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:23.157+0000 7fea3806c640 1 --2- 172.21.15.39:0/2899232765 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fea30109e00 0x7fea3019fd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:23.164 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:23.157+0000 7fea3806c640 1 --2- 172.21.15.39:0/2899232765 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fea301094b0 0x7fea3019aa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:23.164 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:23.157+0000 7fea3806c640 1 --2- 172.21.15.39:0/2899232765 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fea30103bc0 0x7fea3019a4d0 unknown :-1 s=CLOSED pgs=1143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:23.164 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:23.157+0000 7fea3806c640 1 -- 172.21.15.39:0/2899232765 >> 172.21.15.39:0/2899232765 conn(0x7fea3007b910 msgr2=0x7fea30107560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:54:23.164 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:23.157+0000 7fea3806c640 1 -- 172.21.15.39:0/2899232765 shutdown_connections 2024-09-21T03:54:23.164 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:23.157+0000 7fea3806c640 1 -- 172.21.15.39:0/2899232765 wait complete. 2024-09-21T03:54:23.218 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:54:22 smithi161 ceph-mon[34533]: pgmap v579: 97 pgs: 97 active+clean; 10 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 204 B/s wr, 10 op/s 2024-09-21T03:54:23.225 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:54:22 smithi150 ceph-mon[34289]: pgmap v579: 97 pgs: 97 active+clean; 10 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 204 B/s wr, 10 op/s 2024-09-21T03:54:23.897 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 51116d5c-77ca-11ef-baf5-efdc52797490 -- ceph tell mds.1:3 damage ls 2024-09-21T03:54:24.158 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/51116d5c-77ca-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:54:24.710 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:24.668+0000 7f2ad8f30640 1 -- 172.21.15.39:0/2618662911 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2ad41094a0 msgr2=0x7f2ad41098b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:54:24.710 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:24.668+0000 7f2ad8f30640 1 --2- 172.21.15.39:0/2618662911 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2ad41094a0 0x7f2ad41098b0 secure :-1 s=READY pgs=1148 cs=0 l=1 rev1=1 crypto rx=0x7f2ac8009900 tx=0x7f2ac802a410 comp rx=0 tx=0).stop 2024-09-21T03:54:24.711 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:24.668+0000 7f2ad8f30640 1 -- 172.21.15.39:0/2618662911 shutdown_connections 2024-09-21T03:54:24.711 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:24.668+0000 7f2ad8f30640 1 --2- 172.21.15.39:0/2618662911 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2ad4109df0 0x7f2ad4110620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:24.711 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:24.668+0000 7f2ad8f30640 1 --2- 172.21.15.39:0/2618662911 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2ad41094a0 0x7f2ad41098b0 unknown :-1 s=CLOSED pgs=1148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:24.711 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:24.668+0000 7f2ad8f30640 1 --2- 172.21.15.39:0/2618662911 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2ad4103b60 0x7f2ad4103f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:24.711 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:24.668+0000 7f2ad8f30640 1 -- 172.21.15.39:0/2618662911 >> 172.21.15.39:0/2618662911 conn(0x7f2ad407b850 msgr2=0x7f2ad407bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:54:24.711 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:24.668+0000 7f2ad8f30640 1 -- 172.21.15.39:0/2618662911 shutdown_connections 2024-09-21T03:54:24.711 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:24.668+0000 7f2ad8f30640 1 -- 172.21.15.39:0/2618662911 wait complete. 2024-09-21T03:54:24.711 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:24.669+0000 7f2ad8f30640 1 Processor -- start 2024-09-21T03:54:24.711 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:24.669+0000 7f2ad8f30640 1 -- start start 2024-09-21T03:54:24.711 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:24.669+0000 7f2ad8f30640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2ad4103b60 0x7f2ad419a5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:54:24.711 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:24.669+0000 7f2ad8f30640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2ad41094a0 0x7f2ad419ab80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:54:24.711 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:24.669+0000 7f2ad8f30640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2ad4109df0 0x7f2ad419fe20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:54:24.711 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:24.669+0000 7f2ad8f30640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f2ad4113ab0 con 0x7f2ad41094a0 2024-09-21T03:54:24.712 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:24.669+0000 7f2ad8f30640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f2ad4113930 con 0x7f2ad4109df0 2024-09-21T03:54:24.712 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:24.669+0000 7f2ad8f30640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f2ad4113c30 con 0x7f2ad4103b60 2024-09-21T03:54:24.712 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:24.669+0000 7f2ad1d9b640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2ad41094a0 0x7f2ad419ab80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:54:24.712 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:24.669+0000 7f2ad1d9b640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2ad41094a0 0x7f2ad419ab80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:58600/0 (socket says 172.21.15.39:58600) 2024-09-21T03:54:24.712 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:24.669+0000 7f2ad1d9b640 1 -- 172.21.15.39:0/2164371696 learned_addr learned my addr 172.21.15.39:0/2164371696 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:54:24.712 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:24.669+0000 7f2ad2d9d640 1 --2- 172.21.15.39:0/2164371696 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2ad4109df0 0x7f2ad419fe20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:54:24.712 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:24.669+0000 7f2ad259c640 1 --2- 172.21.15.39:0/2164371696 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2ad4103b60 0x7f2ad419a5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:54:24.712 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:24.669+0000 7f2ad1d9b640 1 -- 172.21.15.39:0/2164371696 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2ad4103b60 msgr2=0x7f2ad419a5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:54:24.712 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:24.669+0000 7f2ad1d9b640 1 --2- 172.21.15.39:0/2164371696 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2ad4103b60 0x7f2ad419a5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:24.712 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:24.669+0000 7f2ad1d9b640 1 -- 172.21.15.39:0/2164371696 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2ad4109df0 msgr2=0x7f2ad419fe20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:54:24.712 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:24.669+0000 7f2ad1d9b640 1 --2- 172.21.15.39:0/2164371696 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2ad4109df0 0x7f2ad419fe20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:24.712 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:24.670+0000 7f2ad1d9b640 1 -- 172.21.15.39:0/2164371696 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ad41a0550 con 0x7f2ad41094a0 2024-09-21T03:54:24.712 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:24.670+0000 7f2ad1d9b640 1 --2- 172.21.15.39:0/2164371696 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2ad41094a0 0x7f2ad419ab80 secure :-1 s=READY pgs=1149 cs=0 l=1 rev1=1 crypto rx=0x7f2ac802a930 tx=0x7f2ac8031a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:54:24.712 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:24.670+0000 7f2ac37fe640 1 -- 172.21.15.39:0/2164371696 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ac803f030 con 0x7f2ad41094a0 2024-09-21T03:54:24.712 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:24.670+0000 7f2ac37fe640 1 -- 172.21.15.39:0/2164371696 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2ac8004490 con 0x7f2ad41094a0 2024-09-21T03:54:24.713 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:24.670+0000 7f2ad8f30640 1 -- 172.21.15.39:0/2164371696 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2ad41a07e0 con 0x7f2ad41094a0 2024-09-21T03:54:24.713 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:24.670+0000 7f2ac37fe640 1 -- 172.21.15.39:0/2164371696 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ac8035070 con 0x7f2ad41094a0 2024-09-21T03:54:24.713 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:24.670+0000 7f2ad8f30640 1 -- 172.21.15.39:0/2164371696 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2ad41a6fa0 con 0x7f2ad41094a0 2024-09-21T03:54:24.713 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:24.673+0000 7f2adf4a4740 1 Processor -- start 2024-09-21T03:54:24.713 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:24.673+0000 7f2adf4a4740 1 -- start start 2024-09-21T03:54:24.713 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:24.673+0000 7f2adf4a4740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56458506d070 0x5645854146a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:54:24.713 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:24.673+0000 7f2adf4a4740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5645850441b0 0x564585414be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:54:24.713 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:24.673+0000 7f2adf4a4740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56458504c390 0x564585415120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:54:24.713 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:24.673+0000 7f2adf4a4740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x56458538aea0 con 0x5645850441b0 2024-09-21T03:54:24.713 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:24.673+0000 7f2adf4a4740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x56458537a2d0 con 0x56458506d070 2024-09-21T03:54:24.713 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:24.673+0000 7f2adf4a4740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x56458537e0e0 con 0x56458504c390 2024-09-21T03:54:24.713 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:24.673+0000 7f2ac37fe640 1 -- 172.21.15.39:0/2164371696 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 27) ==== 154643+0+0 (secure 0 0 0) 0x7f2ac8042030 con 0x7f2ad41094a0 2024-09-21T03:54:24.713 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:24.673+0000 7f2ad259c640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5645850441b0 0x564585414be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:54:24.713 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:24.674+0000 7f2ad259c640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5645850441b0 0x564585414be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:58608/0 (socket says 172.21.15.39:58608) 2024-09-21T03:54:24.713 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:24.674+0000 7f2ad259c640 1 -- 172.21.15.39:0/333913293 learned_addr learned my addr 172.21.15.39:0/333913293 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:54:24.714 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:24.674+0000 7f2ad2d9d640 1 --2- 172.21.15.39:0/333913293 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56458506d070 0x5645854146a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:54:24.714 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:24.674+0000 7f2ad259c640 1 -- 172.21.15.39:0/333913293 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56458504c390 msgr2=0x564585415120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:54:24.714 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:24.674+0000 7f2ad259c640 1 --2- 172.21.15.39:0/333913293 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56458504c390 0x564585415120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:24.714 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:24.674+0000 7f2ad259c640 1 -- 172.21.15.39:0/333913293 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56458506d070 msgr2=0x5645854146a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:54:24.714 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:24.674+0000 7f2ad259c640 1 --2- 172.21.15.39:0/333913293 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56458506d070 0x5645854146a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:24.714 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:24.674+0000 7f2ad259c640 1 -- 172.21.15.39:0/333913293 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5645850d8650 con 0x5645850441b0 2024-09-21T03:54:24.714 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:24.674+0000 7f2ad259c640 1 --2- 172.21.15.39:0/333913293 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5645850441b0 0x564585414be0 secure :-1 s=READY pgs=1150 cs=0 l=1 rev1=1 crypto rx=0x7f2ac4009c80 tx=0x7f2ac40117b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6370cfcc78f364dc server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:54:24.714 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:24.674+0000 7f2ac37fe640 1 --2- 172.21.15.39:0/2164371696 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f2abc0b6700 0x7f2abc0b8c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:54:24.714 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:24.674+0000 7f2ac0ff9640 1 -- 172.21.15.39:0/333913293 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ac4012450 con 0x5645850441b0 2024-09-21T03:54:24.714 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:24.674+0000 7f2ad2d9d640 1 --2- 172.21.15.39:0/2164371696 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f2abc0b6700 0x7f2abc0b8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:54:24.714 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:24.674+0000 7f2ac37fe640 1 -- 172.21.15.39:0/2164371696 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8528+0+0 (secure 0 0 0) 0x7f2ac8107a00 con 0x7f2ad41094a0 2024-09-21T03:54:24.714 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:24.674+0000 7f2ac0ff9640 1 -- 172.21.15.39:0/333913293 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2ac4012aa0 con 0x5645850441b0 2024-09-21T03:54:24.714 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:24.674+0000 7f2ac0ff9640 1 -- 172.21.15.39:0/333913293 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ac400fae0 con 0x5645850441b0 2024-09-21T03:54:24.714 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:24.675+0000 7f2adf4a4740 1 -- 172.21.15.39:0/333913293 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5645850441b0 msgr2=0x564585414be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:54:24.715 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:24.675+0000 7f2ad2d9d640 1 --2- 172.21.15.39:0/2164371696 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f2abc0b6700 0x7f2abc0b8c10 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f2acc005fd0 tx=0x7f2acc009500 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:54:24.715 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:24.675+0000 7f2adf4a4740 1 --2- 172.21.15.39:0/333913293 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5645850441b0 0x564585414be0 secure :-1 s=READY pgs=1150 cs=0 l=1 rev1=1 crypto rx=0x7f2ac4009c80 tx=0x7f2ac40117b0 comp rx=0 tx=0).stop 2024-09-21T03:54:24.715 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:24.675+0000 7f2adf4a4740 1 -- 172.21.15.39:0/333913293 shutdown_connections 2024-09-21T03:54:24.715 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:24.675+0000 7f2adf4a4740 1 --2- 172.21.15.39:0/333913293 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56458504c390 0x564585415120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:24.715 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:24.675+0000 7f2adf4a4740 1 --2- 172.21.15.39:0/333913293 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5645850441b0 0x564585414be0 unknown :-1 s=CLOSED pgs=1150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:24.715 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:24.675+0000 7f2adf4a4740 1 --2- 172.21.15.39:0/333913293 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56458506d070 0x5645854146a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:24.715 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:24.675+0000 7f2adf4a4740 1 -- 172.21.15.39:0/333913293 >> 172.21.15.39:0/333913293 conn(0x564585061460 msgr2=0x564585412590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:54:24.715 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:24.676+0000 7f2adf4a4740 1 -- 172.21.15.39:0/333913293 shutdown_connections 2024-09-21T03:54:24.715 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:24.676+0000 7f2adf4a4740 1 -- 172.21.15.39:0/333913293 wait complete. 2024-09-21T03:54:24.715 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:24.676+0000 7f2adf4a4740 1 -- start start 2024-09-21T03:54:24.715 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:24.676+0000 7f2adf4a4740 1 Processor -- start 2024-09-21T03:54:24.715 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:24.676+0000 7f2adf4a4740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5645850441b0 0x56458541fd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:54:24.715 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:24.676+0000 7f2adf4a4740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56458504c390 0x564585420280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:54:24.715 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:24.676+0000 7f2adf4a4740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56458506d070 0x564585420800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:54:24.716 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:24.676+0000 7f2adf4a4740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x5645851db700 con 0x56458506d070 2024-09-21T03:54:24.716 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:24.676+0000 7f2adf4a4740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x564585315160 con 0x5645850441b0 2024-09-21T03:54:24.716 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:24.676+0000 7f2adf4a4740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x564585422e80 con 0x56458504c390 2024-09-21T03:54:24.716 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:24.676+0000 7f2adf4a4740 10 client.0 _ll_register_callbacks cb 0x56458532e9e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:54:24.716 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:24.677+0000 7f2ad2d9d640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56458506d070 0x564585420800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:54:24.716 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:24.677+0000 7f2ad2d9d640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56458506d070 0x564585420800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:58622/0 (socket says 172.21.15.39:58622) 2024-09-21T03:54:24.716 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:24.677+0000 7f2ad2d9d640 1 -- 172.21.15.39:0/3976935900 learned_addr learned my addr 172.21.15.39:0/3976935900 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:54:24.716 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:24.677+0000 7f2ad259c640 1 --2- 172.21.15.39:0/3976935900 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5645850441b0 0x56458541fd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:54:24.716 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:24.677+0000 7f2ad1d9b640 1 --2- 172.21.15.39:0/3976935900 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56458504c390 0x564585420280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:54:24.716 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:24.677+0000 7f2ad2d9d640 1 -- 172.21.15.39:0/3976935900 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56458504c390 msgr2=0x564585420280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:54:24.716 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:24.677+0000 7f2ad2d9d640 1 --2- 172.21.15.39:0/3976935900 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56458504c390 0x564585420280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:24.716 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:24.677+0000 7f2ad2d9d640 1 -- 172.21.15.39:0/3976935900 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5645850441b0 msgr2=0x56458541fd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:54:24.716 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:24.677+0000 7f2ad2d9d640 1 --2- 172.21.15.39:0/3976935900 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5645850441b0 0x56458541fd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:24.716 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:24.677+0000 7f2ad2d9d640 1 -- 172.21.15.39:0/3976935900 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564585342c90 con 0x56458506d070 2024-09-21T03:54:24.716 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:24.677+0000 7f2ad2d9d640 1 --2- 172.21.15.39:0/3976935900 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56458506d070 0x564585420800 secure :-1 s=READY pgs=1151 cs=0 l=1 rev1=1 crypto rx=0x7f2acc003f70 tx=0x7f2acc004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=26e12712f414afac server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:54:24.717 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:24.677+0000 7f2ab9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-09-21T03:54:24.717 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:24.677+0000 7f2ab9ffb640 1 -- 172.21.15.39:0/3976935900 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2acc02d060 con 0x56458506d070 2024-09-21T03:54:24.717 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:24.677+0000 7f2ab9ffb640 1 -- 172.21.15.39:0/3976935900 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8528+0+0 (secure 0 0 0) 0x7f2acc018070 con 0x56458506d070 2024-09-21T03:54:24.717 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:24.677+0000 7f2adf4a4740 1 -- 172.21.15.39:0/3976935900 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2ac4012aa0 con 0x56458506d070 2024-09-21T03:54:24.717 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:24.678+0000 7f2ab9ffb640 1 -- 172.21.15.39:0/3976935900 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f2acc020820 con 0x56458506d070 2024-09-21T03:54:24.717 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:24.678+0000 7f2adf4a4740 10 client.17745 fetch_fsmap learned FSMap version 29 2024-09-21T03:54:24.717 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:24.678+0000 7f2adf4a4740 1 -- 172.21.15.39:0/3976935900 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x56458542ebb0 con 0x56458506d070 2024-09-21T03:54:24.717 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:24.678+0000 7f2ab9ffb640 1 -- 172.21.15.39:0/3976935900 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2ac4012aa0 con 0x56458506d070 2024-09-21T03:54:24.717 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:24.678+0000 7f2ab9ffb640 1 -- 172.21.15.39:0/3976935900 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f2acc01ebb0 con 0x56458506d070 2024-09-21T03:54:24.717 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:24.678+0000 7f2adf4a4740 10 client.17745 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T03:54:24.717 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:24.678+0000 7f2adf4a4740 10 client.17745 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-21T03:54:24.717 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:24.678+0000 7f2adf4a4740 20 client.17745 populate_metadata read hostname 'smithi039' 2024-09-21T03:54:24.717 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:24.678+0000 7f2adf4a4740 1 --2- 172.21.15.39:0/3976935900 >> [v2:172.21.15.161:6836/3451544597,v1:172.21.15.161:6837/3451544597] conn(0x56458502b9f0 0x5645854320c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:54:24.717 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:24.678+0000 7f2adf4a4740 4 client.17745 mds_command: new command op to 24646 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:54:24.718 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:24.678+0000 7f2adf4a4740 1 -- 172.21.15.39:0/3976935900 --> [v2:172.21.15.161:6836/3451544597,v1:172.21.15.161:6837/3451544597] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56458538cdf0 con 0x56458502b9f0 2024-09-21T03:54:24.718 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:24.678+0000 7f2ad259c640 1 --2- 172.21.15.39:0/3976935900 >> [v2:172.21.15.161:6836/3451544597,v1:172.21.15.161:6837/3451544597] conn(0x56458502b9f0 0x5645854320c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:54:24.718 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:24.679+0000 7f2ab9ffb640 10 client.17745 ms_handle_connect on v2:172.21.15.161:6836/3451544597 2024-09-21T03:54:24.718 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:24.679+0000 7f2ad259c640 1 --2- 172.21.15.39:0/3976935900 >> [v2:172.21.15.161:6836/3451544597,v1:172.21.15.161:6837/3451544597] conn(0x56458502b9f0 0x5645854320c0 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=99b8a069e97705dd server_cookie=c5e6fc0ce7c29c99 in_seq=0 out_seq=0 2024-09-21T03:54:24.718 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:24.680+0000 7f2ab9ffb640 1 -- 172.21.15.39:0/3976935900 <== mds.3 v2:172.21.15.161:6836/3451544597 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x56458538cdf0 con 0x56458502b9f0 2024-09-21T03:54:24.718 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:24.680+0000 7f2ab9ffb640 10 client.17745 handle_command_reply: tid=0 2024-09-21T03:54:24.718 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:24.680+0000 7f2adf4a4740 1 client.17745 shutdown 2024-09-21T03:54:24.718 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:24.680+0000 7f2adf4a4740 1 -- 172.21.15.39:0/3976935900 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56458506d070 msgr2=0x564585420800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:54:24.718 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:24.680+0000 7f2adf4a4740 1 --2- 172.21.15.39:0/3976935900 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56458506d070 0x564585420800 secure :-1 s=READY pgs=1151 cs=0 l=1 rev1=1 crypto rx=0x7f2acc003f70 tx=0x7f2acc004ab0 comp rx=0 tx=0).stop 2024-09-21T03:54:24.718 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:24.680+0000 7f2adf4a4740 1 -- 172.21.15.39:0/3976935900 shutdown_connections 2024-09-21T03:54:24.718 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:24.680+0000 7f2adf4a4740 1 --2- 172.21.15.39:0/3976935900 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56458506d070 0x564585420800 unknown :-1 s=CLOSED pgs=1151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:24.718 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:24.680+0000 7f2adf4a4740 1 --2- 172.21.15.39:0/3976935900 >> [v2:172.21.15.161:6836/3451544597,v1:172.21.15.161:6837/3451544597] conn(0x56458502b9f0 0x5645854320c0 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:24.718 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:24.680+0000 7f2adf4a4740 1 --2- 172.21.15.39:0/3976935900 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56458504c390 0x564585420280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:24.718 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:24.680+0000 7f2adf4a4740 1 --2- 172.21.15.39:0/3976935900 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5645850441b0 0x56458541fd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:24.719 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:24.680+0000 7f2ab9ffb640 0 client.17745 ms_handle_reset on v2:172.21.15.161:6836/3451544597 2024-09-21T03:54:24.719 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:24.680+0000 7f2adf4a4740 1 -- 172.21.15.39:0/3976935900 >> 172.21.15.39:0/3976935900 conn(0x564585061460 msgr2=0x564585417830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:54:24.719 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:24.680+0000 7f2adf4a4740 1 -- 172.21.15.39:0/3976935900 shutdown_connections 2024-09-21T03:54:24.719 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:24.680+0000 7f2adf4a4740 1 -- 172.21.15.39:0/3976935900 wait complete. 2024-09-21T03:54:24.719 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:24.680+0000 7f2adf4a4740 20 client.17745 trim_cache size 0 max 16384 2024-09-21T03:54:24.719 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:24.693+0000 7f2adf4a4740 1 Processor -- start 2024-09-21T03:54:24.719 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:24.693+0000 7f2adf4a4740 1 -- start start 2024-09-21T03:54:24.719 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:24.693+0000 7f2adf4a4740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5645850441b0 0x564585416090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:54:24.719 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:24.693+0000 7f2adf4a4740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56458504c390 0x564585421970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:54:24.719 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:24.693+0000 7f2adf4a4740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56458506d070 0x564585411bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:54:24.719 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:24.693+0000 7f2adf4a4740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f2ad41af890 con 0x56458506d070 2024-09-21T03:54:24.719 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:24.693+0000 7f2adf4a4740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f2ad41b13d0 con 0x56458504c390 2024-09-21T03:54:24.719 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:24.693+0000 7f2adf4a4740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f2ad41aff60 con 0x5645850441b0 2024-09-21T03:54:24.719 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:24.694+0000 7f2ad2d9d640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56458506d070 0x564585411bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:54:24.719 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:24.694+0000 7f2ad2d9d640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56458506d070 0x564585411bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:58638/0 (socket says 172.21.15.39:58638) 2024-09-21T03:54:24.720 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:24.694+0000 7f2ad2d9d640 1 -- 172.21.15.39:0/2150126305 learned_addr learned my addr 172.21.15.39:0/2150126305 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:54:24.720 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:24.694+0000 7f2ad1d9b640 1 --2- 172.21.15.39:0/2150126305 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56458504c390 0x564585421970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:54:24.720 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:24.694+0000 7f2ad259c640 1 --2- 172.21.15.39:0/2150126305 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5645850441b0 0x564585416090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:54:24.720 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:24.694+0000 7f2ad2d9d640 1 -- 172.21.15.39:0/2150126305 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5645850441b0 msgr2=0x564585416090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:54:24.720 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:24.694+0000 7f2ad2d9d640 1 --2- 172.21.15.39:0/2150126305 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5645850441b0 0x564585416090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:24.720 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:24.694+0000 7f2ad2d9d640 1 -- 172.21.15.39:0/2150126305 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56458504c390 msgr2=0x564585421970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:54:24.720 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:24.694+0000 7f2ad2d9d640 1 --2- 172.21.15.39:0/2150126305 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56458504c390 0x564585421970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:24.720 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:24.694+0000 7f2ad2d9d640 1 -- 172.21.15.39:0/2150126305 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56458541e240 con 0x56458506d070 2024-09-21T03:54:24.720 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:24.694+0000 7f2ad2d9d640 1 --2- 172.21.15.39:0/2150126305 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56458506d070 0x564585411bc0 secure :-1 s=READY pgs=1152 cs=0 l=1 rev1=1 crypto rx=0x7f2acc013010 tx=0x7f2acc003e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e28e52ae5190915 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:54:24.720 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:24.694+0000 7f2abbfff640 1 -- 172.21.15.39:0/2150126305 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2acc01d540 con 0x56458506d070 2024-09-21T03:54:24.720 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:24.694+0000 7f2abbfff640 1 -- 172.21.15.39:0/2150126305 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2acc01d6e0 con 0x56458506d070 2024-09-21T03:54:24.720 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:24.694+0000 7f2abbfff640 1 -- 172.21.15.39:0/2150126305 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2acc01de50 con 0x56458506d070 2024-09-21T03:54:24.720 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:24.695+0000 7f2adf4a4740 1 -- 172.21.15.39:0/2150126305 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56458506d070 msgr2=0x564585411bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:54:24.720 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:24.695+0000 7f2adf4a4740 1 --2- 172.21.15.39:0/2150126305 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56458506d070 0x564585411bc0 secure :-1 s=READY pgs=1152 cs=0 l=1 rev1=1 crypto rx=0x7f2acc013010 tx=0x7f2acc003e30 comp rx=0 tx=0).stop 2024-09-21T03:54:24.720 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:24.695+0000 7f2adf4a4740 1 -- 172.21.15.39:0/2150126305 shutdown_connections 2024-09-21T03:54:24.721 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:24.695+0000 7f2adf4a4740 1 --2- 172.21.15.39:0/2150126305 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56458506d070 0x564585411bc0 unknown :-1 s=CLOSED pgs=1152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:24.721 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:24.695+0000 7f2adf4a4740 1 --2- 172.21.15.39:0/2150126305 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56458504c390 0x564585421970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:24.721 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:24.695+0000 7f2adf4a4740 1 --2- 172.21.15.39:0/2150126305 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5645850441b0 0x564585416090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:24.721 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:24.695+0000 7f2adf4a4740 1 -- 172.21.15.39:0/2150126305 >> 172.21.15.39:0/2150126305 conn(0x564585061460 msgr2=0x564585421260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:54:24.721 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:24.695+0000 7f2adf4a4740 1 -- 172.21.15.39:0/2150126305 shutdown_connections 2024-09-21T03:54:24.721 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:24.695+0000 7f2adf4a4740 1 -- 172.21.15.39:0/2150126305 wait complete. 2024-09-21T03:54:24.721 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:24.695+0000 7f2adf4a4740 1 -- start start 2024-09-21T03:54:24.721 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:24.695+0000 7f2adf4a4740 1 Processor -- start 2024-09-21T03:54:24.721 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:24.696+0000 7f2adf4a4740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5645850441b0 0x564585424110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:54:24.721 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:24.696+0000 7f2adf4a4740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56458504c390 0x564585424650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:54:24.721 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:24.696+0000 7f2adf4a4740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56458506d070 0x564585424b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:54:24.721 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:24.696+0000 7f2adf4a4740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f2ad41b0d00 con 0x56458506d070 2024-09-21T03:54:24.721 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:24.696+0000 7f2adf4a4740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f2ad41b0630 con 0x5645850441b0 2024-09-21T03:54:24.721 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:24.696+0000 7f2adf4a4740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f2ad41ad870 con 0x56458504c390 2024-09-21T03:54:24.721 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:24.696+0000 7f2adf4a4740 10 client.0 _ll_register_callbacks cb 0x7f2acc009d30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:54:24.722 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:24.696+0000 7f2ad2d9d640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56458506d070 0x564585424b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:54:24.722 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:24.696+0000 7f2ad2d9d640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56458506d070 0x564585424b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:58654/0 (socket says 172.21.15.39:58654) 2024-09-21T03:54:24.722 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:24.696+0000 7f2ad2d9d640 1 -- 172.21.15.39:0/247951576 learned_addr learned my addr 172.21.15.39:0/247951576 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:54:24.722 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:24.696+0000 7f2ad259c640 1 --2- 172.21.15.39:0/247951576 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5645850441b0 0x564585424110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:54:24.722 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:24.696+0000 7f2ad1d9b640 1 --2- 172.21.15.39:0/247951576 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56458504c390 0x564585424650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:54:24.722 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:24.696+0000 7f2ad2d9d640 1 -- 172.21.15.39:0/247951576 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56458504c390 msgr2=0x564585424650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:54:24.722 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:24.696+0000 7f2ad2d9d640 1 --2- 172.21.15.39:0/247951576 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56458504c390 0x564585424650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:24.722 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:24.696+0000 7f2ad2d9d640 1 -- 172.21.15.39:0/247951576 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5645850441b0 msgr2=0x564585424110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:54:24.722 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:24.696+0000 7f2ad2d9d640 1 --2- 172.21.15.39:0/247951576 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5645850441b0 0x564585424110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:24.722 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:24.696+0000 7f2ad2d9d640 1 -- 172.21.15.39:0/247951576 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56458549bb00 con 0x56458506d070 2024-09-21T03:54:24.722 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:24.696+0000 7f2ad2d9d640 1 --2- 172.21.15.39:0/247951576 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56458506d070 0x564585424b90 secure :-1 s=READY pgs=1153 cs=0 l=1 rev1=1 crypto rx=0x7f2acc003fb0 tx=0x7f2acc0202f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=17cc7e3b05b5081e server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:54:24.722 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:24.696+0000 7f2ab9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-09-21T03:54:24.722 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:24.697+0000 7f2ab9ffb640 1 -- 172.21.15.39:0/247951576 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2acc01d540 con 0x56458506d070 2024-09-21T03:54:24.722 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:24.697+0000 7f2ab9ffb640 1 -- 172.21.15.39:0/247951576 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8528+0+0 (secure 0 0 0) 0x7f2acc01dde0 con 0x56458506d070 2024-09-21T03:54:24.723 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:24.697+0000 7f2adf4a4740 1 -- 172.21.15.39:0/247951576 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2acc01d6e0 con 0x56458506d070 2024-09-21T03:54:24.723 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:24.697+0000 7f2ab9ffb640 1 -- 172.21.15.39:0/247951576 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f2acc0209c0 con 0x56458506d070 2024-09-21T03:54:24.723 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:24.697+0000 7f2adf4a4740 10 client.17751 fetch_fsmap learned FSMap version 29 2024-09-21T03:54:24.723 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:24.697+0000 7f2adf4a4740 1 -- 172.21.15.39:0/247951576 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x56458549cd80 con 0x56458506d070 2024-09-21T03:54:24.723 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:24.697+0000 7f2ab9ffb640 1 -- 172.21.15.39:0/247951576 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2acc01d6e0 con 0x56458506d070 2024-09-21T03:54:24.723 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:24.697+0000 7f2ab9ffb640 1 -- 172.21.15.39:0/247951576 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f2acc04c480 con 0x56458506d070 2024-09-21T03:54:24.723 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:24.697+0000 7f2adf4a4740 10 client.17751 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T03:54:24.723 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:24.697+0000 7f2adf4a4740 10 client.17751 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-21T03:54:24.723 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:24.697+0000 7f2adf4a4740 20 client.17751 populate_metadata read hostname 'smithi039' 2024-09-21T03:54:24.723 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:24.697+0000 7f2adf4a4740 1 --2- 172.21.15.39:0/247951576 >> [v2:172.21.15.161:6836/3451544597,v1:172.21.15.161:6837/3451544597] conn(0x564585000da0 0x56458549f3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:54:24.723 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:24.697+0000 7f2adf4a4740 4 client.17751 mds_command: new command op to 24646 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T03:54:24.723 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:24.697+0000 7f2adf4a4740 1 -- 172.21.15.39:0/247951576 --> [v2:172.21.15.161:6836/3451544597,v1:172.21.15.161:6837/3451544597] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5645853888a0 con 0x564585000da0 2024-09-21T03:54:24.723 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:24.698+0000 7f2ad259c640 1 --2- 172.21.15.39:0/247951576 >> [v2:172.21.15.161:6836/3451544597,v1:172.21.15.161:6837/3451544597] conn(0x564585000da0 0x56458549f3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:54:24.723 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:24.698+0000 7f2ad259c640 1 --2- 172.21.15.39:0/247951576 >> [v2:172.21.15.161:6836/3451544597,v1:172.21.15.161:6837/3451544597] conn(0x564585000da0 0x56458549f3f0 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=ef143302f1f50f3d server_cookie=833255d8ae7ef97 in_seq=0 out_seq=0 2024-09-21T03:54:24.724 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:24.698+0000 7f2ab9ffb640 10 client.17751 ms_handle_connect on v2:172.21.15.161:6836/3451544597 2024-09-21T03:54:24.724 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:24.698+0000 7f2ab9ffb640 1 -- 172.21.15.39:0/247951576 <== mds.3 v2:172.21.15.161:6836/3451544597 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5645853888a0 con 0x564585000da0 2024-09-21T03:54:24.724 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:24.698+0000 7f2ab9ffb640 10 client.17751 handle_command_reply: tid=0 2024-09-21T03:54:24.724 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:24.698+0000 7f2adf4a4740 1 client.17751 shutdown 2024-09-21T03:54:24.724 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:24.698+0000 7f2adf4a4740 1 -- 172.21.15.39:0/247951576 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56458506d070 msgr2=0x564585424b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:54:24.724 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:24.698+0000 7f2adf4a4740 1 --2- 172.21.15.39:0/247951576 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56458506d070 0x564585424b90 secure :-1 s=READY pgs=1153 cs=0 l=1 rev1=1 crypto rx=0x7f2acc003fb0 tx=0x7f2acc0202f0 comp rx=0 tx=0).stop 2024-09-21T03:54:24.724 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:24.699+0000 7f2adf4a4740 1 -- 172.21.15.39:0/247951576 shutdown_connections 2024-09-21T03:54:24.724 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:24.699+0000 7f2adf4a4740 1 --2- 172.21.15.39:0/247951576 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56458506d070 0x564585424b90 unknown :-1 s=CLOSED pgs=1153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:24.724 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:24.699+0000 7f2adf4a4740 1 --2- 172.21.15.39:0/247951576 >> [v2:172.21.15.161:6836/3451544597,v1:172.21.15.161:6837/3451544597] conn(0x564585000da0 0x56458549f3f0 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:24.724 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:24.699+0000 7f2adf4a4740 1 --2- 172.21.15.39:0/247951576 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56458504c390 0x564585424650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:24.724 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:24.699+0000 7f2ab9ffb640 0 client.17751 ms_handle_reset on v2:172.21.15.161:6836/3451544597 2024-09-21T03:54:24.724 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:24.699+0000 7f2adf4a4740 1 --2- 172.21.15.39:0/247951576 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5645850441b0 0x564585424110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:24.724 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:24.699+0000 7f2adf4a4740 1 -- 172.21.15.39:0/247951576 >> 172.21.15.39:0/247951576 conn(0x564585061460 msgr2=0x56458542ba50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:54:24.724 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:24.699+0000 7f2adf4a4740 1 -- 172.21.15.39:0/247951576 shutdown_connections 2024-09-21T03:54:24.725 INFO:teuthology.orchestra.run.smithi039.stdout:[] 2024-09-21T03:54:24.725 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:24.699+0000 7f2adf4a4740 1 -- 172.21.15.39:0/247951576 wait complete. 2024-09-21T03:54:24.725 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:24.699+0000 7f2adf4a4740 20 client.17751 trim_cache size 0 max 16384 2024-09-21T03:54:24.725 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:24.699+0000 7f2ad8f30640 1 -- 172.21.15.39:0/2164371696 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f2abc0b6700 msgr2=0x7f2abc0b8c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:54:24.725 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:24.699+0000 7f2ad8f30640 1 --2- 172.21.15.39:0/2164371696 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f2abc0b6700 0x7f2abc0b8c10 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f2acc005fd0 tx=0x7f2acc009500 comp rx=0 tx=0).stop 2024-09-21T03:54:24.725 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:24.699+0000 7f2ad8f30640 1 -- 172.21.15.39:0/2164371696 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2ad41094a0 msgr2=0x7f2ad419ab80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:54:24.725 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:24.699+0000 7f2ad8f30640 1 --2- 172.21.15.39:0/2164371696 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2ad41094a0 0x7f2ad419ab80 secure :-1 s=READY pgs=1149 cs=0 l=1 rev1=1 crypto rx=0x7f2ac802a930 tx=0x7f2ac8031a20 comp rx=0 tx=0).stop 2024-09-21T03:54:24.726 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:24.699+0000 7f2ad8f30640 1 -- 172.21.15.39:0/2164371696 shutdown_connections 2024-09-21T03:54:24.726 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:24.699+0000 7f2ad8f30640 1 --2- 172.21.15.39:0/2164371696 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f2abc0b6700 0x7f2abc0b8c10 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:24.726 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:24.699+0000 7f2ad8f30640 1 --2- 172.21.15.39:0/2164371696 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2ad4109df0 0x7f2ad419fe20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:24.726 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:24.699+0000 7f2ad8f30640 1 --2- 172.21.15.39:0/2164371696 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2ad41094a0 0x7f2ad419ab80 unknown :-1 s=CLOSED pgs=1149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:24.726 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:24.699+0000 7f2ad8f30640 1 --2- 172.21.15.39:0/2164371696 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2ad4103b60 0x7f2ad419a5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:24.726 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:24.699+0000 7f2ad8f30640 1 -- 172.21.15.39:0/2164371696 >> 172.21.15.39:0/2164371696 conn(0x7f2ad407b850 msgr2=0x7f2ad4105610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:54:24.726 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:24.700+0000 7f2ad8f30640 1 -- 172.21.15.39:0/2164371696 shutdown_connections 2024-09-21T03:54:24.726 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:24.700+0000 7f2ad8f30640 1 -- 172.21.15.39:0/2164371696 wait complete. 2024-09-21T03:54:25.218 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:54:24 smithi161 ceph-mon[34533]: pgmap v580: 97 pgs: 97 active+clean; 10 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 102 B/s wr, 10 op/s 2024-09-21T03:54:25.224 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:54:24 smithi150 ceph-mon[34289]: pgmap v580: 97 pgs: 97 active+clean; 10 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 102 B/s wr, 10 op/s 2024-09-21T03:54:25.308 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:54:24 smithi039 ceph-mon[31211]: pgmap v580: 97 pgs: 97 active+clean; 10 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 102 B/s wr, 10 op/s 2024-09-21T03:54:25.529 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 51116d5c-77ca-11ef-baf5-efdc52797490 -- ceph tell mds.1:2 damage ls 2024-09-21T03:54:25.788 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/51116d5c-77ca-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:54:26.292 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:26.267+0000 7fd5dbfd7640 1 -- 172.21.15.39:0/1695872448 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd5d41049e0 msgr2=0x7fd5d4104df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:54:26.293 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:26.267+0000 7fd5dbfd7640 1 --2- 172.21.15.39:0/1695872448 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd5d41049e0 0x7fd5d4104df0 secure :-1 s=READY pgs=1154 cs=0 l=1 rev1=1 crypto rx=0x7fd5d0009900 tx=0x7fd5d002a410 comp rx=0 tx=0).stop 2024-09-21T03:54:26.293 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:26.267+0000 7fd5dbfd7640 1 -- 172.21.15.39:0/1695872448 shutdown_connections 2024-09-21T03:54:26.293 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:26.267+0000 7fd5dbfd7640 1 --2- 172.21.15.39:0/1695872448 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd5d410de40 0x7fd5d4110330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:26.293 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:26.267+0000 7fd5dbfd7640 1 --2- 172.21.15.39:0/1695872448 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd5d4105330 0x7fd5d410d880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:26.293 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:26.267+0000 7fd5dbfd7640 1 --2- 172.21.15.39:0/1695872448 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd5d41049e0 0x7fd5d4104df0 unknown :-1 s=CLOSED pgs=1154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:26.293 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:26.267+0000 7fd5dbfd7640 1 -- 172.21.15.39:0/1695872448 >> 172.21.15.39:0/1695872448 conn(0x7fd5d4079750 msgr2=0x7fd5d4079b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:54:26.293 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:26.267+0000 7fd5dbfd7640 1 -- 172.21.15.39:0/1695872448 shutdown_connections 2024-09-21T03:54:26.293 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:26.267+0000 7fd5dbfd7640 1 -- 172.21.15.39:0/1695872448 wait complete. 2024-09-21T03:54:26.294 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:26.268+0000 7fd5dbfd7640 1 Processor -- start 2024-09-21T03:54:26.294 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:26.268+0000 7fd5dbfd7640 1 -- start start 2024-09-21T03:54:26.294 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:26.268+0000 7fd5dbfd7640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd5d41049e0 0x7fd5d4102720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:54:26.294 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:26.268+0000 7fd5dbfd7640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd5d4105330 0x7fd5d4102c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:54:26.294 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:26.268+0000 7fd5dbfd7640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd5d410de40 0x7fd5d41031a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:54:26.294 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:26.268+0000 7fd5dbfd7640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fd5d4113ac0 con 0x7fd5d410de40 2024-09-21T03:54:26.294 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:26.268+0000 7fd5dbfd7640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fd5d4113940 con 0x7fd5d4105330 2024-09-21T03:54:26.294 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:26.268+0000 7fd5dbfd7640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fd5d4113c40 con 0x7fd5d41049e0 2024-09-21T03:54:26.294 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:26.268+0000 7fd5da574640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd5d410de40 0x7fd5d41031a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:54:26.294 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:26.268+0000 7fd5da574640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd5d410de40 0x7fd5d41031a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:58674/0 (socket says 172.21.15.39:58674) 2024-09-21T03:54:26.294 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:26.268+0000 7fd5da574640 1 -- 172.21.15.39:0/2016675698 learned_addr learned my addr 172.21.15.39:0/2016675698 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:54:26.295 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:26.269+0000 7fd5d9d73640 1 --2- 172.21.15.39:0/2016675698 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd5d41049e0 0x7fd5d4102720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:54:26.295 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:26.269+0000 7fd5d9572640 1 --2- 172.21.15.39:0/2016675698 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd5d4105330 0x7fd5d4102c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:54:26.295 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:26.269+0000 7fd5da574640 1 -- 172.21.15.39:0/2016675698 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd5d41049e0 msgr2=0x7fd5d4102720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:54:26.295 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:26.269+0000 7fd5da574640 1 --2- 172.21.15.39:0/2016675698 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd5d41049e0 0x7fd5d4102720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:26.295 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:26.269+0000 7fd5da574640 1 -- 172.21.15.39:0/2016675698 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd5d4105330 msgr2=0x7fd5d4102c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:54:26.295 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:26.269+0000 7fd5da574640 1 --2- 172.21.15.39:0/2016675698 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd5d4105330 0x7fd5d4102c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:26.295 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:26.269+0000 7fd5da574640 1 -- 172.21.15.39:0/2016675698 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd5d41a8750 con 0x7fd5d410de40 2024-09-21T03:54:26.295 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:26.269+0000 7fd5d9572640 1 --2- 172.21.15.39:0/2016675698 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd5d4105330 0x7fd5d4102c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:54:26.295 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:26.269+0000 7fd5da574640 1 --2- 172.21.15.39:0/2016675698 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd5d410de40 0x7fd5d41031a0 secure :-1 s=READY pgs=1155 cs=0 l=1 rev1=1 crypto rx=0x7fd5cc00eaa0 tx=0x7fd5cc00ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:54:26.295 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:26.269+0000 7fd5caffd640 1 -- 172.21.15.39:0/2016675698 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5cc00cdb0 con 0x7fd5d410de40 2024-09-21T03:54:26.295 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:26.269+0000 7fd5dbfd7640 1 -- 172.21.15.39:0/2016675698 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd5d41a8a40 con 0x7fd5d410de40 2024-09-21T03:54:26.295 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:26.269+0000 7fd5caffd640 1 -- 172.21.15.39:0/2016675698 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd5cc005020 con 0x7fd5d410de40 2024-09-21T03:54:26.295 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:26.269+0000 7fd5caffd640 1 -- 172.21.15.39:0/2016675698 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5cc0116d0 con 0x7fd5d410de40 2024-09-21T03:54:26.296 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:26.269+0000 7fd5dbfd7640 1 -- 172.21.15.39:0/2016675698 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd5d41a8fd0 con 0x7fd5d410de40 2024-09-21T03:54:26.296 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:26.272+0000 7fd5e254b740 1 Processor -- start 2024-09-21T03:54:26.296 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:26.272+0000 7fd5caffd640 1 -- 172.21.15.39:0/2016675698 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 27) ==== 154643+0+0 (secure 0 0 0) 0x7fd5cc0054a0 con 0x7fd5d410de40 2024-09-21T03:54:26.296 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:26.273+0000 7fd5e254b740 1 -- start start 2024-09-21T03:54:26.296 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:26.273+0000 7fd5e254b740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55807f2a0070 0x55807f6476a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:54:26.296 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:26.273+0000 7fd5e254b740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55807f2771b0 0x55807f647be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:54:26.296 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:26.273+0000 7fd5e254b740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55807f27f390 0x55807f648120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:54:26.296 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:26.273+0000 7fd5e254b740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x55807f5bdea0 con 0x55807f27f390 2024-09-21T03:54:26.296 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:26.273+0000 7fd5e254b740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x55807f5ad2d0 con 0x55807f2771b0 2024-09-21T03:54:26.296 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:26.273+0000 7fd5e254b740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55807f5b10e0 con 0x55807f2a0070 2024-09-21T03:54:26.296 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:26.273+0000 7fd5d9572640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55807f27f390 0x55807f648120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:54:26.296 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:26.273+0000 7fd5d9572640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55807f27f390 0x55807f648120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:58682/0 (socket says 172.21.15.39:58682) 2024-09-21T03:54:26.296 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:26.273+0000 7fd5d9572640 1 -- 172.21.15.39:0/1497345576 learned_addr learned my addr 172.21.15.39:0/1497345576 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:54:26.297 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:26.273+0000 7fd5d9572640 1 --2- 172.21.15.39:0/1497345576 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55807f2a0070 0x55807f6476a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:54:26.297 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:26.273+0000 7fd5d9d73640 1 --2- 172.21.15.39:0/1497345576 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55807f2771b0 0x55807f647be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:54:26.297 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:26.273+0000 7fd5caffd640 1 --2- 172.21.15.39:0/2016675698 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7fd5c00b6680 0x7fd5c00b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:54:26.297 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:26.274+0000 7fd5d9572640 1 -- 172.21.15.39:0/1497345576 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55807f2a0070 msgr2=0x55807f6476a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:54:26.297 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:26.274+0000 7fd5d9572640 1 --2- 172.21.15.39:0/1497345576 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55807f2a0070 0x55807f6476a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:26.297 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:26.274+0000 7fd5d9572640 1 -- 172.21.15.39:0/1497345576 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55807f2771b0 msgr2=0x55807f647be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:54:26.297 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:26.274+0000 7fd5d9572640 1 --2- 172.21.15.39:0/1497345576 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55807f2771b0 0x55807f647be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:26.297 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:26.274+0000 7fd5d9572640 1 -- 172.21.15.39:0/1497345576 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55807f30b650 con 0x55807f27f390 2024-09-21T03:54:26.297 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:26.274+0000 7fd5caffd640 1 -- 172.21.15.39:0/2016675698 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8528+0+0 (secure 0 0 0) 0x7fd5cc0e7940 con 0x7fd5d410de40 2024-09-21T03:54:26.297 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:26.274+0000 7fd5da574640 1 --2- 172.21.15.39:0/2016675698 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7fd5c00b6680 0x7fd5c00b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:54:26.297 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:26.274+0000 7fd5d9572640 1 --2- 172.21.15.39:0/1497345576 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55807f27f390 0x55807f648120 secure :-1 s=READY pgs=1156 cs=0 l=1 rev1=1 crypto rx=0x7fd5c40149f0 tx=0x7fd5c400b510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=33800bffe9283020 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:54:26.297 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:26.274+0000 7fd5bffff640 1 -- 172.21.15.39:0/1497345576 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5c4004030 con 0x55807f27f390 2024-09-21T03:54:26.297 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:26.274+0000 7fd5bffff640 1 -- 172.21.15.39:0/1497345576 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd5c4005020 con 0x55807f27f390 2024-09-21T03:54:26.297 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:26.274+0000 7fd5bffff640 1 -- 172.21.15.39:0/1497345576 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5c4004a00 con 0x55807f27f390 2024-09-21T03:54:26.297 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:26.275+0000 7fd5e254b740 1 -- 172.21.15.39:0/1497345576 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55807f27f390 msgr2=0x55807f648120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:54:26.298 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:26.275+0000 7fd5da574640 1 --2- 172.21.15.39:0/2016675698 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7fd5c00b6680 0x7fd5c00b8b90 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7fd5cc00cfd0 tx=0x7fd5cc000f40 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:54:26.298 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:26.275+0000 7fd5e254b740 1 --2- 172.21.15.39:0/1497345576 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55807f27f390 0x55807f648120 secure :-1 s=READY pgs=1156 cs=0 l=1 rev1=1 crypto rx=0x7fd5c40149f0 tx=0x7fd5c400b510 comp rx=0 tx=0).stop 2024-09-21T03:54:26.298 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:26.275+0000 7fd5e254b740 1 -- 172.21.15.39:0/1497345576 shutdown_connections 2024-09-21T03:54:26.298 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:26.275+0000 7fd5e254b740 1 --2- 172.21.15.39:0/1497345576 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55807f27f390 0x55807f648120 unknown :-1 s=CLOSED pgs=1156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:26.298 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:26.275+0000 7fd5e254b740 1 --2- 172.21.15.39:0/1497345576 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55807f2771b0 0x55807f647be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:26.298 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:26.275+0000 7fd5e254b740 1 --2- 172.21.15.39:0/1497345576 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55807f2a0070 0x55807f6476a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:26.298 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:26.275+0000 7fd5e254b740 1 -- 172.21.15.39:0/1497345576 >> 172.21.15.39:0/1497345576 conn(0x55807f294460 msgr2=0x55807f645590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:54:26.298 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:26.275+0000 7fd5e254b740 1 -- 172.21.15.39:0/1497345576 shutdown_connections 2024-09-21T03:54:26.298 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:26.275+0000 7fd5e254b740 1 -- 172.21.15.39:0/1497345576 wait complete. 2024-09-21T03:54:26.298 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:26.276+0000 7fd5e254b740 1 -- start start 2024-09-21T03:54:26.298 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:26.276+0000 7fd5e254b740 1 Processor -- start 2024-09-21T03:54:26.298 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:26.276+0000 7fd5e254b740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55807f2771b0 0x55807f652dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:54:26.298 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:26.276+0000 7fd5e254b740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55807f27f390 0x55807f653300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:54:26.298 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:26.276+0000 7fd5e254b740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55807f2a0070 0x55807f65f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:54:26.298 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:26.276+0000 7fd5e254b740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x55807f40e700 con 0x55807f2771b0 2024-09-21T03:54:26.299 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:26.276+0000 7fd5e254b740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x55807f548160 con 0x55807f27f390 2024-09-21T03:54:26.299 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:26.276+0000 7fd5e254b740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55807f655e80 con 0x55807f2a0070 2024-09-21T03:54:26.299 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:26.276+0000 7fd5e254b740 10 client.0 _ll_register_callbacks cb 0x55807f5619e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:54:26.299 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:26.276+0000 7fd5d9572640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55807f2771b0 0x55807f652dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:54:26.299 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:26.276+0000 7fd5d9572640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55807f2771b0 0x55807f652dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:58688/0 (socket says 172.21.15.39:58688) 2024-09-21T03:54:26.299 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:26.276+0000 7fd5d9572640 1 -- 172.21.15.39:0/3463377892 learned_addr learned my addr 172.21.15.39:0/3463377892 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:54:26.299 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:26.277+0000 7fd5d9d73640 1 --2- 172.21.15.39:0/3463377892 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55807f27f390 0x55807f653300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:54:26.299 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:26.277+0000 7fd5d9572640 1 --2- 172.21.15.39:0/3463377892 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55807f2a0070 0x55807f65f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:54:26.299 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:26.277+0000 7fd5d9572640 1 -- 172.21.15.39:0/3463377892 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55807f2a0070 msgr2=0x55807f65f1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:54:26.299 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:26.277+0000 7fd5d9572640 1 --2- 172.21.15.39:0/3463377892 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55807f2a0070 0x55807f65f1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:26.299 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:26.277+0000 7fd5d9572640 1 -- 172.21.15.39:0/3463377892 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55807f27f390 msgr2=0x55807f653300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:54:26.299 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:26.277+0000 7fd5d9572640 1 --2- 172.21.15.39:0/3463377892 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55807f27f390 0x55807f653300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:26.299 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:26.277+0000 7fd5d9572640 1 -- 172.21.15.39:0/3463377892 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55807f575c90 con 0x55807f2771b0 2024-09-21T03:54:26.299 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:26.277+0000 7fd5d9572640 1 --2- 172.21.15.39:0/3463377892 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55807f2771b0 0x55807f652dc0 secure :-1 s=READY pgs=1157 cs=0 l=1 rev1=1 crypto rx=0x7fd5c400b320 tx=0x7fd5c400bda0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b58eff6a280bda10 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:54:26.300 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:26.277+0000 7fd5bd7fa640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-09-21T03:54:26.300 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:26.277+0000 7fd5bd7fa640 1 -- 172.21.15.39:0/3463377892 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5c4027030 con 0x55807f2771b0 2024-09-21T03:54:26.300 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:26.277+0000 7fd5bd7fa640 1 -- 172.21.15.39:0/3463377892 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8528+0+0 (secure 0 0 0) 0x7fd5c40053e0 con 0x55807f2771b0 2024-09-21T03:54:26.300 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:26.277+0000 7fd5e254b740 1 -- 172.21.15.39:0/3463377892 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd5c4005020 con 0x55807f2771b0 2024-09-21T03:54:26.300 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:26.277+0000 7fd5bd7fa640 1 -- 172.21.15.39:0/3463377892 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fd5c400f760 con 0x55807f2771b0 2024-09-21T03:54:26.300 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:26.277+0000 7fd5e254b740 10 client.17763 fetch_fsmap learned FSMap version 29 2024-09-21T03:54:26.300 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:26.277+0000 7fd5e254b740 1 -- 172.21.15.39:0/3463377892 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55807f661c20 con 0x55807f2771b0 2024-09-21T03:54:26.300 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:26.277+0000 7fd5bd7fa640 1 -- 172.21.15.39:0/3463377892 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd5c4005020 con 0x55807f2771b0 2024-09-21T03:54:26.300 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:26.278+0000 7fd5bd7fa640 1 -- 172.21.15.39:0/3463377892 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fd5c4003950 con 0x55807f2771b0 2024-09-21T03:54:26.300 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:26.278+0000 7fd5e254b740 10 client.17763 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T03:54:26.300 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:26.278+0000 7fd5e254b740 10 client.17763 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-21T03:54:26.300 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:26.278+0000 7fd5e254b740 20 client.17763 populate_metadata read hostname 'smithi039' 2024-09-21T03:54:26.300 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:26.278+0000 7fd5e254b740 1 --2- 172.21.15.39:0/3463377892 >> [v2:172.21.15.150:6836/4017043144,v1:172.21.15.150:6837/4017043144] conn(0x55807f25e9f0 0x55807f664f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:54:26.301 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:26.278+0000 7fd5e254b740 4 client.17763 mds_command: new command op to 24659 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:54:26.301 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:26.278+0000 7fd5e254b740 1 -- 172.21.15.39:0/3463377892 --> [v2:172.21.15.150:6836/4017043144,v1:172.21.15.150:6837/4017043144] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55807f5bfdf0 con 0x55807f25e9f0 2024-09-21T03:54:26.301 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:26.278+0000 7fd5d9d73640 1 --2- 172.21.15.39:0/3463377892 >> [v2:172.21.15.150:6836/4017043144,v1:172.21.15.150:6837/4017043144] conn(0x55807f25e9f0 0x55807f664f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:54:26.301 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:26.279+0000 7fd5bd7fa640 10 client.17763 ms_handle_connect on v2:172.21.15.150:6836/4017043144 2024-09-21T03:54:26.301 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:26.279+0000 7fd5d9d73640 1 --2- 172.21.15.39:0/3463377892 >> [v2:172.21.15.150:6836/4017043144,v1:172.21.15.150:6837/4017043144] conn(0x55807f25e9f0 0x55807f664f80 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=1244705a4d471e0d server_cookie=8f187c10e492ee3d in_seq=0 out_seq=0 2024-09-21T03:54:26.301 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:26.279+0000 7fd5bd7fa640 1 -- 172.21.15.39:0/3463377892 <== mds.2 v2:172.21.15.150:6836/4017043144 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55807f5bfdf0 con 0x55807f25e9f0 2024-09-21T03:54:26.301 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:26.279+0000 7fd5bd7fa640 10 client.17763 handle_command_reply: tid=0 2024-09-21T03:54:26.301 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:26.279+0000 7fd5e254b740 1 client.17763 shutdown 2024-09-21T03:54:26.301 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:26.279+0000 7fd5e254b740 1 -- 172.21.15.39:0/3463377892 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55807f2771b0 msgr2=0x55807f652dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:54:26.301 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:26.279+0000 7fd5e254b740 1 --2- 172.21.15.39:0/3463377892 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55807f2771b0 0x55807f652dc0 secure :-1 s=READY pgs=1157 cs=0 l=1 rev1=1 crypto rx=0x7fd5c400b320 tx=0x7fd5c400bda0 comp rx=0 tx=0).stop 2024-09-21T03:54:26.301 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:26.280+0000 7fd5e254b740 1 -- 172.21.15.39:0/3463377892 shutdown_connections 2024-09-21T03:54:26.301 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:26.280+0000 7fd5e254b740 1 --2- 172.21.15.39:0/3463377892 >> [v2:172.21.15.150:6836/4017043144,v1:172.21.15.150:6837/4017043144] conn(0x55807f25e9f0 0x55807f664f80 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:26.301 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:26.280+0000 7fd5e254b740 1 --2- 172.21.15.39:0/3463377892 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55807f2a0070 0x55807f65f1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:26.301 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:26.280+0000 7fd5e254b740 1 --2- 172.21.15.39:0/3463377892 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55807f27f390 0x55807f653300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:26.301 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:26.280+0000 7fd5bd7fa640 0 client.17763 ms_handle_reset on v2:172.21.15.150:6836/4017043144 2024-09-21T03:54:26.302 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:26.280+0000 7fd5e254b740 1 --2- 172.21.15.39:0/3463377892 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55807f2771b0 0x55807f652dc0 unknown :-1 s=CLOSED pgs=1157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:26.302 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:26.280+0000 7fd5e254b740 1 -- 172.21.15.39:0/3463377892 >> 172.21.15.39:0/3463377892 conn(0x55807f294460 msgr2=0x55807f64a830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:54:26.302 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:26.280+0000 7fd5e254b740 1 -- 172.21.15.39:0/3463377892 shutdown_connections 2024-09-21T03:54:26.302 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:26.280+0000 7fd5e254b740 1 -- 172.21.15.39:0/3463377892 wait complete. 2024-09-21T03:54:26.302 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:26.280+0000 7fd5e254b740 20 client.17763 trim_cache size 0 max 16384 2024-09-21T03:54:26.308 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:26.293+0000 7fd5e254b740 1 Processor -- start 2024-09-21T03:54:26.308 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:26.293+0000 7fd5e254b740 1 -- start start 2024-09-21T03:54:26.308 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:26.293+0000 7fd5e254b740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55807f2771b0 0x55807f654db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:54:26.308 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:26.293+0000 7fd5e254b740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55807f27f390 0x55807f644bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:54:26.308 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:26.293+0000 7fd5e254b740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55807f2a0070 0x55807f645100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:54:26.308 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:26.293+0000 7fd5e254b740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fd5d41b1a90 con 0x55807f2a0070 2024-09-21T03:54:26.308 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:26.293+0000 7fd5e254b740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fd5d41b35d0 con 0x55807f27f390 2024-09-21T03:54:26.308 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:26.293+0000 7fd5e254b740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fd5d41b2160 con 0x55807f2771b0 2024-09-21T03:54:26.308 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:26.293+0000 7fd5d9572640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55807f2a0070 0x55807f645100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:54:26.308 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:26.293+0000 7fd5d9572640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55807f2a0070 0x55807f645100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:58704/0 (socket says 172.21.15.39:58704) 2024-09-21T03:54:26.308 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:26.293+0000 7fd5d9572640 1 -- 172.21.15.39:0/4033253126 learned_addr learned my addr 172.21.15.39:0/4033253126 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:54:26.308 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:26.294+0000 7fd5d9572640 1 --2- 172.21.15.39:0/4033253126 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55807f2771b0 0x55807f654db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:54:26.309 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:26.294+0000 7fd5d9d73640 1 --2- 172.21.15.39:0/4033253126 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55807f27f390 0x55807f644bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:54:26.309 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:26.294+0000 7fd5d9572640 1 -- 172.21.15.39:0/4033253126 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55807f2771b0 msgr2=0x55807f654db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:54:26.309 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:26.294+0000 7fd5d9572640 1 --2- 172.21.15.39:0/4033253126 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55807f2771b0 0x55807f654db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:26.309 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:26.294+0000 7fd5d9572640 1 -- 172.21.15.39:0/4033253126 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55807f27f390 msgr2=0x55807f644bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:54:26.309 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:26.294+0000 7fd5d9572640 1 --2- 172.21.15.39:0/4033253126 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55807f27f390 0x55807f644bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:26.309 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:26.294+0000 7fd5d9572640 1 -- 172.21.15.39:0/4033253126 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55807f655cd0 con 0x55807f2a0070 2024-09-21T03:54:26.309 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:26.294+0000 7fd5d9572640 1 --2- 172.21.15.39:0/4033253126 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55807f2a0070 0x55807f645100 secure :-1 s=READY pgs=1158 cs=0 l=1 rev1=1 crypto rx=0x7fd5c400b510 tx=0x7fd5c4017040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c9903ad111cd52a8 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:54:26.309 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:26.294+0000 7fd5bf7fe640 1 -- 172.21.15.39:0/4033253126 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5c4027040 con 0x55807f2a0070 2024-09-21T03:54:26.309 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:26.294+0000 7fd5bf7fe640 1 -- 172.21.15.39:0/4033253126 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd5c40055e0 con 0x55807f2a0070 2024-09-21T03:54:26.309 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:26.294+0000 7fd5bf7fe640 1 -- 172.21.15.39:0/4033253126 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5c4003b40 con 0x55807f2a0070 2024-09-21T03:54:26.309 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:26.295+0000 7fd5e254b740 1 -- 172.21.15.39:0/4033253126 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55807f2a0070 msgr2=0x55807f645100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:54:26.309 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:26.295+0000 7fd5e254b740 1 --2- 172.21.15.39:0/4033253126 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55807f2a0070 0x55807f645100 secure :-1 s=READY pgs=1158 cs=0 l=1 rev1=1 crypto rx=0x7fd5c400b510 tx=0x7fd5c4017040 comp rx=0 tx=0).stop 2024-09-21T03:54:26.309 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:26.295+0000 7fd5e254b740 1 -- 172.21.15.39:0/4033253126 shutdown_connections 2024-09-21T03:54:26.309 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:26.295+0000 7fd5e254b740 1 --2- 172.21.15.39:0/4033253126 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55807f2a0070 0x55807f645100 unknown :-1 s=CLOSED pgs=1158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:26.310 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:26.295+0000 7fd5e254b740 1 --2- 172.21.15.39:0/4033253126 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55807f27f390 0x55807f644bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:26.310 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:26.295+0000 7fd5e254b740 1 --2- 172.21.15.39:0/4033253126 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55807f2771b0 0x55807f654db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:26.310 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:26.295+0000 7fd5e254b740 1 -- 172.21.15.39:0/4033253126 >> 172.21.15.39:0/4033253126 conn(0x55807f294460 msgr2=0x55807f649040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:54:26.310 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:26.295+0000 7fd5e254b740 1 -- 172.21.15.39:0/4033253126 shutdown_connections 2024-09-21T03:54:26.310 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:26.295+0000 7fd5e254b740 1 -- 172.21.15.39:0/4033253126 wait complete. 2024-09-21T03:54:26.310 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:26.295+0000 7fd5e254b740 1 -- start start 2024-09-21T03:54:26.310 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:26.296+0000 7fd5e254b740 1 Processor -- start 2024-09-21T03:54:26.310 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:26.296+0000 7fd5e254b740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55807f2771b0 0x55807f668770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:54:26.310 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:26.296+0000 7fd5e254b740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55807f27f390 0x55807f665c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:54:26.310 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:26.296+0000 7fd5e254b740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55807f2a0070 0x55807f666180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:54:26.310 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:26.296+0000 7fd5e254b740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fd5d41b2f00 con 0x55807f2a0070 2024-09-21T03:54:26.310 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:26.296+0000 7fd5e254b740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fd5d41b2830 con 0x55807f27f390 2024-09-21T03:54:26.310 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:26.296+0000 7fd5e254b740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fd5d41afa70 con 0x55807f2771b0 2024-09-21T03:54:26.311 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:26.296+0000 7fd5e254b740 10 client.0 _ll_register_callbacks cb 0x7fd5d0009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:54:26.311 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:26.296+0000 7fd5d9572640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55807f2a0070 0x55807f666180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:54:26.311 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:26.296+0000 7fd5d9572640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55807f2a0070 0x55807f666180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:58720/0 (socket says 172.21.15.39:58720) 2024-09-21T03:54:26.311 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:26.296+0000 7fd5d9572640 1 -- 172.21.15.39:0/1450989839 learned_addr learned my addr 172.21.15.39:0/1450989839 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:54:26.311 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:26.296+0000 7fd5d9d73640 1 --2- 172.21.15.39:0/1450989839 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55807f27f390 0x55807f665c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:54:26.311 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:26.296+0000 7fd5d9572640 1 --2- 172.21.15.39:0/1450989839 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55807f2771b0 0x55807f668770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:54:26.311 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:26.297+0000 7fd5d9572640 1 -- 172.21.15.39:0/1450989839 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55807f2771b0 msgr2=0x55807f668770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:54:26.311 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:26.297+0000 7fd5d9572640 1 --2- 172.21.15.39:0/1450989839 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55807f2771b0 0x55807f668770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:26.311 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:26.297+0000 7fd5d9572640 1 -- 172.21.15.39:0/1450989839 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55807f27f390 msgr2=0x55807f665c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:54:26.311 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:26.297+0000 7fd5d9572640 1 --2- 172.21.15.39:0/1450989839 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55807f27f390 0x55807f665c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:26.311 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:26.297+0000 7fd5d9572640 1 -- 172.21.15.39:0/1450989839 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55807f6668f0 con 0x55807f2a0070 2024-09-21T03:54:26.311 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:26.297+0000 7fd5d9572640 1 --2- 172.21.15.39:0/1450989839 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55807f2a0070 0x55807f666180 secure :-1 s=READY pgs=1159 cs=0 l=1 rev1=1 crypto rx=0x7fd5c40050b0 tx=0x7fd5c4023040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d6f79a10d772d08 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:54:26.311 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:26.297+0000 7fd5bd7fa640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-09-21T03:54:26.312 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:26.297+0000 7fd5bd7fa640 1 -- 172.21.15.39:0/1450989839 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5c40035f0 con 0x55807f2a0070 2024-09-21T03:54:26.312 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:26.297+0000 7fd5bd7fa640 1 -- 172.21.15.39:0/1450989839 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8528+0+0 (secure 0 0 0) 0x7fd5c40053e0 con 0x55807f2a0070 2024-09-21T03:54:26.312 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:26.297+0000 7fd5e254b740 1 -- 172.21.15.39:0/1450989839 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd5c40055e0 con 0x55807f2a0070 2024-09-21T03:54:26.312 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:26.297+0000 7fd5bd7fa640 1 -- 172.21.15.39:0/1450989839 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fd5c4047050 con 0x55807f2a0070 2024-09-21T03:54:26.312 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:26.297+0000 7fd5e254b740 10 client.17769 fetch_fsmap learned FSMap version 29 2024-09-21T03:54:26.312 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:26.297+0000 7fd5e254b740 1 -- 172.21.15.39:0/1450989839 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55807f6cf620 con 0x55807f2a0070 2024-09-21T03:54:26.312 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:26.297+0000 7fd5bd7fa640 1 -- 172.21.15.39:0/1450989839 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd5c40055e0 con 0x55807f2a0070 2024-09-21T03:54:26.312 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:26.298+0000 7fd5bd7fa640 1 -- 172.21.15.39:0/1450989839 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fd5c402e730 con 0x55807f2a0070 2024-09-21T03:54:26.312 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:26.298+0000 7fd5e254b740 10 client.17769 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T03:54:26.312 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:26.298+0000 7fd5e254b740 10 client.17769 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-21T03:54:26.312 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:26.298+0000 7fd5e254b740 20 client.17769 populate_metadata read hostname 'smithi039' 2024-09-21T03:54:26.312 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:26.298+0000 7fd5e254b740 1 --2- 172.21.15.39:0/1450989839 >> [v2:172.21.15.150:6836/4017043144,v1:172.21.15.150:6837/4017043144] conn(0x55807f233da0 0x55807f6d1d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:54:26.312 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:26.298+0000 7fd5e254b740 4 client.17769 mds_command: new command op to 24659 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T03:54:26.313 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:26.298+0000 7fd5e254b740 1 -- 172.21.15.39:0/1450989839 --> [v2:172.21.15.150:6836/4017043144,v1:172.21.15.150:6837/4017043144] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55807f5bb8a0 con 0x55807f233da0 2024-09-21T03:54:26.313 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:26.298+0000 7fd5d9d73640 1 --2- 172.21.15.39:0/1450989839 >> [v2:172.21.15.150:6836/4017043144,v1:172.21.15.150:6837/4017043144] conn(0x55807f233da0 0x55807f6d1d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:54:26.313 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:26.299+0000 7fd5d9d73640 1 --2- 172.21.15.39:0/1450989839 >> [v2:172.21.15.150:6836/4017043144,v1:172.21.15.150:6837/4017043144] conn(0x55807f233da0 0x55807f6d1d40 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=dd53ccf35c1d4c67 server_cookie=4de626aaca5204ef in_seq=0 out_seq=0 2024-09-21T03:54:26.313 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:26.299+0000 7fd5bd7fa640 10 client.17769 ms_handle_connect on v2:172.21.15.150:6836/4017043144 2024-09-21T03:54:26.313 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:26.299+0000 7fd5bd7fa640 1 -- 172.21.15.39:0/1450989839 <== mds.2 v2:172.21.15.150:6836/4017043144 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55807f5bb8a0 con 0x55807f233da0 2024-09-21T03:54:26.313 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:26.299+0000 7fd5bd7fa640 10 client.17769 handle_command_reply: tid=0 2024-09-21T03:54:26.313 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:26.299+0000 7fd5e254b740 1 client.17769 shutdown 2024-09-21T03:54:26.313 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:26.299+0000 7fd5e254b740 1 -- 172.21.15.39:0/1450989839 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55807f2a0070 msgr2=0x55807f666180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:54:26.313 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:26.299+0000 7fd5e254b740 1 --2- 172.21.15.39:0/1450989839 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55807f2a0070 0x55807f666180 secure :-1 s=READY pgs=1159 cs=0 l=1 rev1=1 crypto rx=0x7fd5c40050b0 tx=0x7fd5c4023040 comp rx=0 tx=0).stop 2024-09-21T03:54:26.313 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:26.299+0000 7fd5e254b740 1 -- 172.21.15.39:0/1450989839 shutdown_connections 2024-09-21T03:54:26.313 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:26.299+0000 7fd5e254b740 1 --2- 172.21.15.39:0/1450989839 >> [v2:172.21.15.150:6836/4017043144,v1:172.21.15.150:6837/4017043144] conn(0x55807f233da0 0x55807f6d1d40 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:26.313 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:26.299+0000 7fd5e254b740 1 --2- 172.21.15.39:0/1450989839 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55807f2a0070 0x55807f666180 unknown :-1 s=CLOSED pgs=1159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:26.313 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:26.299+0000 7fd5e254b740 1 --2- 172.21.15.39:0/1450989839 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55807f27f390 0x55807f665c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:26.313 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:26.299+0000 7fd5e254b740 1 --2- 172.21.15.39:0/1450989839 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55807f2771b0 0x55807f668770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:26.314 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:26.299+0000 7fd5e254b740 1 -- 172.21.15.39:0/1450989839 >> 172.21.15.39:0/1450989839 conn(0x55807f294460 msgr2=0x55807f64c7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:54:26.314 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:26.299+0000 7fd5bd7fa640 0 client.17769 ms_handle_reset on v2:172.21.15.150:6836/4017043144 2024-09-21T03:54:26.314 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:26.299+0000 7fd5e254b740 1 -- 172.21.15.39:0/1450989839 shutdown_connections 2024-09-21T03:54:26.314 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:26.299+0000 7fd5e254b740 1 -- 172.21.15.39:0/1450989839 wait complete. 2024-09-21T03:54:26.314 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:26.299+0000 7fd5e254b740 20 client.17769 trim_cache size 0 max 16384 2024-09-21T03:54:26.314 INFO:teuthology.orchestra.run.smithi039.stdout:[] 2024-09-21T03:54:26.315 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:26.300+0000 7fd5dbfd7640 1 -- 172.21.15.39:0/2016675698 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7fd5c00b6680 msgr2=0x7fd5c00b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:54:26.315 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:26.300+0000 7fd5dbfd7640 1 --2- 172.21.15.39:0/2016675698 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7fd5c00b6680 0x7fd5c00b8b90 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7fd5cc00cfd0 tx=0x7fd5cc000f40 comp rx=0 tx=0).stop 2024-09-21T03:54:26.315 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:26.300+0000 7fd5dbfd7640 1 -- 172.21.15.39:0/2016675698 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd5d410de40 msgr2=0x7fd5d41031a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:54:26.315 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:26.300+0000 7fd5dbfd7640 1 --2- 172.21.15.39:0/2016675698 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd5d410de40 0x7fd5d41031a0 secure :-1 s=READY pgs=1155 cs=0 l=1 rev1=1 crypto rx=0x7fd5cc00eaa0 tx=0x7fd5cc00ef60 comp rx=0 tx=0).stop 2024-09-21T03:54:26.315 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:26.300+0000 7fd5dbfd7640 1 -- 172.21.15.39:0/2016675698 shutdown_connections 2024-09-21T03:54:26.315 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:26.300+0000 7fd5dbfd7640 1 --2- 172.21.15.39:0/2016675698 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7fd5c00b6680 0x7fd5c00b8b90 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:26.315 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:26.300+0000 7fd5dbfd7640 1 --2- 172.21.15.39:0/2016675698 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd5d410de40 0x7fd5d41031a0 unknown :-1 s=CLOSED pgs=1155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:26.315 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:26.300+0000 7fd5dbfd7640 1 --2- 172.21.15.39:0/2016675698 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd5d4105330 0x7fd5d4102c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:26.316 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:26.300+0000 7fd5dbfd7640 1 --2- 172.21.15.39:0/2016675698 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd5d41049e0 0x7fd5d4102720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:26.316 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:26.300+0000 7fd5dbfd7640 1 -- 172.21.15.39:0/2016675698 >> 172.21.15.39:0/2016675698 conn(0x7fd5d4079750 msgr2=0x7fd5d4100770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:54:26.316 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:26.300+0000 7fd5dbfd7640 1 -- 172.21.15.39:0/2016675698 shutdown_connections 2024-09-21T03:54:26.316 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:26.300+0000 7fd5dbfd7640 1 -- 172.21.15.39:0/2016675698 wait complete. 2024-09-21T03:54:27.079 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:54:26 smithi039 ceph-mon[31211]: pgmap v581: 97 pgs: 97 active+clean; 10 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 102 B/s wr, 10 op/s 2024-09-21T03:54:27.109 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 51116d5c-77ca-11ef-baf5-efdc52797490 -- ceph tell mds.1:1 damage ls 2024-09-21T03:54:27.218 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:54:26 smithi161 ceph-mon[34533]: pgmap v581: 97 pgs: 97 active+clean; 10 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 102 B/s wr, 10 op/s 2024-09-21T03:54:27.225 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:54:26 smithi150 ceph-mon[34289]: pgmap v581: 97 pgs: 97 active+clean; 10 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 102 B/s wr, 10 op/s 2024-09-21T03:54:27.365 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/51116d5c-77ca-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:54:27.867 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:27.857+0000 7f69edb17640 1 -- 172.21.15.39:0/4241222700 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f69e8105380 msgr2=0x7f69e810d8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:54:27.868 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:27.857+0000 7f69edb17640 1 --2- 172.21.15.39:0/4241222700 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f69e8105380 0x7f69e810d8d0 secure :-1 s=READY pgs=1160 cs=0 l=1 rev1=1 crypto rx=0x7f69dc009900 tx=0x7f69dc02a410 comp rx=0 tx=0).stop 2024-09-21T03:54:27.868 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:27.857+0000 7f69edb17640 1 -- 172.21.15.39:0/4241222700 shutdown_connections 2024-09-21T03:54:27.868 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:27.857+0000 7f69edb17640 1 --2- 172.21.15.39:0/4241222700 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f69e810de60 0x7f69e8110330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:27.868 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:27.857+0000 7f69edb17640 1 --2- 172.21.15.39:0/4241222700 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f69e8105380 0x7f69e810d8d0 unknown :-1 s=CLOSED pgs=1160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:27.868 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:27.857+0000 7f69edb17640 1 --2- 172.21.15.39:0/4241222700 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f69e8104a30 0x7f69e8104e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:27.868 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:27.857+0000 7f69edb17640 1 -- 172.21.15.39:0/4241222700 >> 172.21.15.39:0/4241222700 conn(0x7f69e8079760 msgr2=0x7f69e8079b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:54:27.868 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:27.858+0000 7f69edb17640 1 -- 172.21.15.39:0/4241222700 shutdown_connections 2024-09-21T03:54:27.868 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:27.858+0000 7f69edb17640 1 -- 172.21.15.39:0/4241222700 wait complete. 2024-09-21T03:54:27.868 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:27.858+0000 7f69edb17640 1 Processor -- start 2024-09-21T03:54:27.868 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:27.858+0000 7f69edb17640 1 -- start start 2024-09-21T03:54:27.868 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:27.859+0000 7f69edb17640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f69e8104a30 0x7f69e819a4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:54:27.868 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:27.859+0000 7f69edb17640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f69e8105380 0x7f69e819aa70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:54:27.869 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:27.859+0000 7f69edb17640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f69e810de60 0x7f69e819fd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:54:27.869 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:27.859+0000 7f69edb17640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f69e8113aa0 con 0x7f69e810de60 2024-09-21T03:54:27.869 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:27.859+0000 7f69edb17640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f69e8113920 con 0x7f69e8104a30 2024-09-21T03:54:27.869 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:27.859+0000 7f69edb17640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f69e8113c20 con 0x7f69e8105380 2024-09-21T03:54:27.869 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:27.859+0000 7f69e7fff640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f69e810de60 0x7f69e819fd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:54:27.869 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:27.859+0000 7f69e7fff640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f69e810de60 0x7f69e819fd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:53440/0 (socket says 172.21.15.39:53440) 2024-09-21T03:54:27.869 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:27.859+0000 7f69e7fff640 1 -- 172.21.15.39:0/3055289856 learned_addr learned my addr 172.21.15.39:0/3055289856 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:54:27.869 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:27.859+0000 7f69e6ffd640 1 --2- 172.21.15.39:0/3055289856 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f69e8105380 0x7f69e819aa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:54:27.869 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:27.859+0000 7f69e77fe640 1 --2- 172.21.15.39:0/3055289856 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f69e8104a30 0x7f69e819a4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:54:27.869 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:27.859+0000 7f69e7fff640 1 -- 172.21.15.39:0/3055289856 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f69e8105380 msgr2=0x7f69e819aa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:54:27.869 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:27.859+0000 7f69e7fff640 1 --2- 172.21.15.39:0/3055289856 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f69e8105380 0x7f69e819aa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:27.869 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:27.859+0000 7f69e7fff640 1 -- 172.21.15.39:0/3055289856 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f69e8104a30 msgr2=0x7f69e819a4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:54:27.869 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:27.859+0000 7f69e7fff640 1 --2- 172.21.15.39:0/3055289856 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f69e8104a30 0x7f69e819a4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:27.869 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:27.859+0000 7f69e7fff640 1 -- 172.21.15.39:0/3055289856 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f69e81a0440 con 0x7f69e810de60 2024-09-21T03:54:27.869 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:27.859+0000 7f69e7fff640 1 --2- 172.21.15.39:0/3055289856 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f69e810de60 0x7f69e819fd10 secure :-1 s=READY pgs=1161 cs=0 l=1 rev1=1 crypto rx=0x7f69e000eaa0 tx=0x7f69e000ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:54:27.870 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:27.860+0000 7f69e4ff9640 1 -- 172.21.15.39:0/3055289856 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69e000cdb0 con 0x7f69e810de60 2024-09-21T03:54:27.870 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:27.860+0000 7f69e4ff9640 1 -- 172.21.15.39:0/3055289856 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f69e0005020 con 0x7f69e810de60 2024-09-21T03:54:27.870 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:27.860+0000 7f69edb17640 1 -- 172.21.15.39:0/3055289856 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f69e81a0730 con 0x7f69e810de60 2024-09-21T03:54:27.870 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:27.860+0000 7f69e4ff9640 1 -- 172.21.15.39:0/3055289856 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69e00116d0 con 0x7f69e810de60 2024-09-21T03:54:27.870 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:27.860+0000 7f69edb17640 1 -- 172.21.15.39:0/3055289856 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f69e81a6e90 con 0x7f69e810de60 2024-09-21T03:54:27.870 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:27.863+0000 7f69f408b740 1 Processor -- start 2024-09-21T03:54:27.870 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:27.863+0000 7f69e4ff9640 1 -- 172.21.15.39:0/3055289856 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 27) ==== 154643+0+0 (secure 0 0 0) 0x7f69e00051d0 con 0x7f69e810de60 2024-09-21T03:54:27.870 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:27.863+0000 7f69f408b740 1 -- start start 2024-09-21T03:54:27.870 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:27.863+0000 7f69f408b740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557c10d70070 0x557c111172c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:54:27.870 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:27.863+0000 7f69f408b740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557c10d471b0 0x557c11117800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:54:27.870 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:27.864+0000 7f69f408b740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557c10d4f390 0x557c11117d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:54:27.870 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:27.864+0000 7f69f408b740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x557c1108dea0 con 0x557c10d471b0 2024-09-21T03:54:27.870 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:27.864+0000 7f69f408b740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x557c1107d2d0 con 0x557c10d70070 2024-09-21T03:54:27.870 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:27.864+0000 7f69f408b740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x557c110810e0 con 0x557c10d4f390 2024-09-21T03:54:27.871 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:27.864+0000 7f69e77fe640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557c10d471b0 0x557c11117800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:54:27.871 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:27.864+0000 7f69e77fe640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557c10d471b0 0x557c11117800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:53444/0 (socket says 172.21.15.39:53444) 2024-09-21T03:54:27.871 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:27.864+0000 7f69e77fe640 1 -- 172.21.15.39:0/1876262694 learned_addr learned my addr 172.21.15.39:0/1876262694 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:54:27.871 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:27.864+0000 7f69e6ffd640 1 --2- 172.21.15.39:0/1876262694 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557c10d70070 0x557c111172c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:54:27.871 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:27.864+0000 7f69e6ffd640 1 --2- 172.21.15.39:0/1876262694 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557c10d4f390 0x557c11117d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:54:27.871 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:27.864+0000 7f69e77fe640 1 -- 172.21.15.39:0/1876262694 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557c10d4f390 msgr2=0x557c11117d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:54:27.871 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:27.864+0000 7f69e77fe640 1 --2- 172.21.15.39:0/1876262694 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557c10d4f390 0x557c11117d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:27.871 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:27.864+0000 7f69e77fe640 1 -- 172.21.15.39:0/1876262694 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557c10d70070 msgr2=0x557c111172c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:54:27.871 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:27.864+0000 7f69e77fe640 1 --2- 172.21.15.39:0/1876262694 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557c10d70070 0x557c111172c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:27.871 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:27.864+0000 7f69e77fe640 1 -- 172.21.15.39:0/1876262694 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557c10ddb650 con 0x557c10d471b0 2024-09-21T03:54:27.871 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:27.864+0000 7f69e6ffd640 1 --2- 172.21.15.39:0/1876262694 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557c10d4f390 0x557c11117d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:54:27.871 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:27.864+0000 7f69e4ff9640 1 --2- 172.21.15.39:0/3055289856 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f69d00b6680 0x7f69d00b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:54:27.871 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:27.864+0000 7f69e77fe640 1 --2- 172.21.15.39:0/1876262694 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557c10d471b0 0x557c11117800 secure :-1 s=READY pgs=1162 cs=0 l=1 rev1=1 crypto rx=0x7f69d800bcb0 tx=0x7f69d80107a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f2b27277782c0820 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:54:27.871 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:27.865+0000 7f69e6ffd640 1 --2- 172.21.15.39:0/3055289856 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f69d00b6680 0x7f69d00b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:54:27.871 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:27.865+0000 7f69e4ff9640 1 -- 172.21.15.39:0/3055289856 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8528+0+0 (secure 0 0 0) 0x7f69e00e77f0 con 0x7f69e810de60 2024-09-21T03:54:27.873 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:27.865+0000 7f69e6ffd640 1 --2- 172.21.15.39:0/3055289856 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f69d00b6680 0x7f69d00b8b90 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f69dc009230 tx=0x7f69dc000bc0 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:54:27.874 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:27.865+0000 7f69cdffb640 1 -- 172.21.15.39:0/1876262694 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69d8004020 con 0x557c10d471b0 2024-09-21T03:54:27.874 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:27.866+0000 7f69cdffb640 1 -- 172.21.15.39:0/1876262694 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f69d80054d0 con 0x557c10d471b0 2024-09-21T03:54:27.874 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:27.866+0000 7f69cdffb640 1 -- 172.21.15.39:0/1876262694 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69d8004c20 con 0x557c10d471b0 2024-09-21T03:54:27.874 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:27.866+0000 7f69f408b740 1 -- 172.21.15.39:0/1876262694 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557c10d471b0 msgr2=0x557c11117800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:54:27.874 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:27.866+0000 7f69f408b740 1 --2- 172.21.15.39:0/1876262694 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557c10d471b0 0x557c11117800 secure :-1 s=READY pgs=1162 cs=0 l=1 rev1=1 crypto rx=0x7f69d800bcb0 tx=0x7f69d80107a0 comp rx=0 tx=0).stop 2024-09-21T03:54:27.874 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:27.866+0000 7f69f408b740 1 -- 172.21.15.39:0/1876262694 shutdown_connections 2024-09-21T03:54:27.874 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:27.867+0000 7f69f408b740 1 --2- 172.21.15.39:0/1876262694 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557c10d4f390 0x557c11117d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:27.874 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:27.867+0000 7f69f408b740 1 --2- 172.21.15.39:0/1876262694 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557c10d471b0 0x557c11117800 unknown :-1 s=CLOSED pgs=1162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:27.874 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:27.867+0000 7f69f408b740 1 --2- 172.21.15.39:0/1876262694 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557c10d70070 0x557c111172c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:27.874 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:27.867+0000 7f69f408b740 1 -- 172.21.15.39:0/1876262694 >> 172.21.15.39:0/1876262694 conn(0x557c10d64460 msgr2=0x557c111151b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:54:27.874 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:27.867+0000 7f69f408b740 1 -- 172.21.15.39:0/1876262694 shutdown_connections 2024-09-21T03:54:27.874 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:27.867+0000 7f69f408b740 1 -- 172.21.15.39:0/1876262694 wait complete. 2024-09-21T03:54:27.875 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:27.867+0000 7f69f408b740 1 -- start start 2024-09-21T03:54:27.875 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:27.867+0000 7f69f408b740 1 Processor -- start 2024-09-21T03:54:27.875 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:27.868+0000 7f69f408b740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557c10d471b0 0x557c11122980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:54:27.875 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:27.868+0000 7f69f408b740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557c10d4f390 0x557c11122ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:54:27.875 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:27.868+0000 7f69f408b740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557c10d70070 0x557c1112edc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:54:27.875 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:27.868+0000 7f69f408b740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x557c10ede700 con 0x557c10d70070 2024-09-21T03:54:27.875 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:27.868+0000 7f69f408b740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x557c11018160 con 0x557c10d471b0 2024-09-21T03:54:27.875 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:27.868+0000 7f69f408b740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x557c11125aa0 con 0x557c10d4f390 2024-09-21T03:54:27.875 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:27.868+0000 7f69f408b740 10 client.0 _ll_register_callbacks cb 0x557c110319e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:54:27.875 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:27.868+0000 7f69e6ffd640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557c10d4f390 0x557c11122ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:54:27.875 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:27.868+0000 7f69e77fe640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557c10d471b0 0x557c11122980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:54:27.875 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:27.868+0000 7f69e6ffd640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557c10d4f390 0x557c11122ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.39:52710/0 (socket says 172.21.15.39:52710) 2024-09-21T03:54:27.875 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:27.868+0000 7f69e6ffd640 1 -- 172.21.15.39:0/4104228642 learned_addr learned my addr 172.21.15.39:0/4104228642 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:54:27.875 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:27.868+0000 7f69e77fe640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557c10d471b0 0x557c11122980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.39:51854/0 (socket says 172.21.15.39:51854) 2024-09-21T03:54:27.876 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:27.868+0000 7f69e7fff640 1 --2- 172.21.15.39:0/4104228642 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557c10d70070 0x557c1112edc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:54:27.876 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:27.868+0000 7f69e77fe640 1 -- 172.21.15.39:0/4104228642 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557c10d4f390 msgr2=0x557c11122ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:54:27.876 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:27.868+0000 7f69e77fe640 1 --2- 172.21.15.39:0/4104228642 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557c10d4f390 0x557c11122ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:27.876 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:27.868+0000 7f69e77fe640 1 -- 172.21.15.39:0/4104228642 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557c10d70070 msgr2=0x557c1112edc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:54:27.876 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:27.868+0000 7f69e77fe640 1 --2- 172.21.15.39:0/4104228642 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557c10d70070 0x557c1112edc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:27.876 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:27.868+0000 7f69e77fe640 1 -- 172.21.15.39:0/4104228642 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557c11045c90 con 0x557c10d471b0 2024-09-21T03:54:27.876 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:27.868+0000 7f69e7fff640 1 --2- 172.21.15.39:0/4104228642 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557c10d70070 0x557c1112edc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:54:27.876 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:27.868+0000 7f69e6ffd640 1 --2- 172.21.15.39:0/4104228642 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557c10d4f390 0x557c11122ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:54:27.876 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:27.869+0000 7f69e77fe640 1 --2- 172.21.15.39:0/4104228642 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557c10d471b0 0x557c11122980 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f69d8011010 tx=0x7f69d800bfd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e0f9ebdd6bf3fdc9 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:54:27.876 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:27.869+0000 7f69c6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.150:3300/0 2024-09-21T03:54:27.877 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:27.874+0000 7f69c6ffd640 1 -- 172.21.15.39:0/4104228642 <== mon.2 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69d802b060 con 0x557c10d471b0 2024-09-21T03:54:27.877 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:27.874+0000 7f69c6ffd640 1 -- 172.21.15.39:0/4104228642 <== mon.2 v2:172.21.15.150:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8528+0+0 (secure 0 0 0) 0x7f69d8004b30 con 0x557c10d471b0 2024-09-21T03:54:27.877 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:27.874+0000 7f69f408b740 1 -- 172.21.15.39:0/4104228642 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f69d80054d0 con 0x557c10d471b0 2024-09-21T03:54:27.877 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:27.874+0000 7f69c6ffd640 1 -- 172.21.15.39:0/4104228642 <== mon.2 v2:172.21.15.150:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f69d8004d30 con 0x557c10d471b0 2024-09-21T03:54:27.877 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:27.874+0000 7f69f408b740 10 client.26969 fetch_fsmap learned FSMap version 29 2024-09-21T03:54:27.877 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:27.874+0000 7f69f408b740 1 -- 172.21.15.39:0/4104228642 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x557c11131780 con 0x557c10d471b0 2024-09-21T03:54:27.877 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:27.874+0000 7f69c6ffd640 1 -- 172.21.15.39:0/4104228642 <== mon.2 v2:172.21.15.150:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f69d80054d0 con 0x557c10d471b0 2024-09-21T03:54:27.878 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:27.875+0000 7f69c6ffd640 1 -- 172.21.15.39:0/4104228642 <== mon.2 v2:172.21.15.150:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f69d8011960 con 0x557c10d471b0 2024-09-21T03:54:27.878 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:27.875+0000 7f69f408b740 10 client.26969 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T03:54:27.879 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:27.875+0000 7f69f408b740 10 client.26969 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-21T03:54:27.879 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:27.875+0000 7f69f408b740 20 client.26969 populate_metadata read hostname 'smithi039' 2024-09-21T03:54:27.879 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:27.875+0000 7f69f408b740 1 --2- 172.21.15.39:0/4104228642 >> [v2:172.21.15.161:6838/2190107270,v1:172.21.15.161:6839/2190107270] conn(0x557c10d2e9f0 0x557c11134b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:54:27.879 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:27.875+0000 7f69f408b740 4 client.26969 mds_command: new command op to 24667 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:54:27.879 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:27.875+0000 7f69f408b740 1 -- 172.21.15.39:0/4104228642 --> [v2:172.21.15.161:6838/2190107270,v1:172.21.15.161:6839/2190107270] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557c1108fdf0 con 0x557c10d2e9f0 2024-09-21T03:54:27.879 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:27.875+0000 7f69e6ffd640 1 --2- 172.21.15.39:0/4104228642 >> [v2:172.21.15.161:6838/2190107270,v1:172.21.15.161:6839/2190107270] conn(0x557c10d2e9f0 0x557c11134b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:54:27.879 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:27.876+0000 7f69c6ffd640 10 client.26969 ms_handle_connect on v2:172.21.15.161:6838/2190107270 2024-09-21T03:54:27.879 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:27.876+0000 7f69e6ffd640 1 --2- 172.21.15.39:0/4104228642 >> [v2:172.21.15.161:6838/2190107270,v1:172.21.15.161:6839/2190107270] conn(0x557c10d2e9f0 0x557c11134b90 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=430f5269b99b1a61 server_cookie=e05ba1a23e365719 in_seq=0 out_seq=0 2024-09-21T03:54:27.880 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:27.876+0000 7f69c6ffd640 1 -- 172.21.15.39:0/4104228642 <== mds.1 v2:172.21.15.161:6838/2190107270 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x557c1108fdf0 con 0x557c10d2e9f0 2024-09-21T03:54:27.880 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:27.876+0000 7f69c6ffd640 10 client.26969 handle_command_reply: tid=0 2024-09-21T03:54:27.880 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:27.876+0000 7f69f408b740 1 client.26969 shutdown 2024-09-21T03:54:27.880 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:27.877+0000 7f69f408b740 1 -- 172.21.15.39:0/4104228642 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557c10d471b0 msgr2=0x557c11122980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:54:27.881 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:27.877+0000 7f69f408b740 1 --2- 172.21.15.39:0/4104228642 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557c10d471b0 0x557c11122980 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f69d8011010 tx=0x7f69d800bfd0 comp rx=0 tx=0).stop 2024-09-21T03:54:27.881 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:27.877+0000 7f69f408b740 1 -- 172.21.15.39:0/4104228642 shutdown_connections 2024-09-21T03:54:27.881 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:27.877+0000 7f69f408b740 1 --2- 172.21.15.39:0/4104228642 >> [v2:172.21.15.161:6838/2190107270,v1:172.21.15.161:6839/2190107270] conn(0x557c10d2e9f0 0x557c11134b90 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:27.881 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:27.877+0000 7f69f408b740 1 --2- 172.21.15.39:0/4104228642 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557c10d70070 0x557c1112edc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:27.881 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:27.877+0000 7f69f408b740 1 --2- 172.21.15.39:0/4104228642 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557c10d4f390 0x557c11122ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:27.881 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:27.877+0000 7f69f408b740 1 --2- 172.21.15.39:0/4104228642 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557c10d471b0 0x557c11122980 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:27.881 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:27.877+0000 7f69f408b740 1 -- 172.21.15.39:0/4104228642 >> 172.21.15.39:0/4104228642 conn(0x557c10d64460 msgr2=0x557c1111a450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:54:27.881 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:27.877+0000 7f69c6ffd640 0 client.26969 ms_handle_reset on v2:172.21.15.161:6838/2190107270 2024-09-21T03:54:27.881 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:27.877+0000 7f69f408b740 1 -- 172.21.15.39:0/4104228642 shutdown_connections 2024-09-21T03:54:27.881 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:27.877+0000 7f69f408b740 1 -- 172.21.15.39:0/4104228642 wait complete. 2024-09-21T03:54:27.881 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:27.877+0000 7f69f408b740 20 client.26969 trim_cache size 0 max 16384 2024-09-21T03:54:27.899 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:27.895+0000 7f69f408b740 1 Processor -- start 2024-09-21T03:54:27.899 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:27.895+0000 7f69f408b740 1 -- start start 2024-09-21T03:54:27.899 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:27.895+0000 7f69f408b740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557c10d471b0 0x557c11122e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:54:27.900 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:27.895+0000 7f69f408b740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557c10d4f390 0x557c1112ed30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:54:27.900 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:27.895+0000 7f69f408b740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557c10d70070 0x557c1111fb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:54:27.900 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:27.895+0000 7f69f408b740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f69e81af7f0 con 0x557c10d471b0 2024-09-21T03:54:27.900 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:27.896+0000 7f69f408b740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f69e81b1330 con 0x557c10d70070 2024-09-21T03:54:27.900 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:27.896+0000 7f69f408b740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f69e81afec0 con 0x557c10d4f390 2024-09-21T03:54:27.900 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:27.896+0000 7f69e77fe640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557c10d471b0 0x557c11122e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:54:27.900 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:27.896+0000 7f69e77fe640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557c10d471b0 0x557c11122e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:53462/0 (socket says 172.21.15.39:53462) 2024-09-21T03:54:27.900 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:27.896+0000 7f69e77fe640 1 -- 172.21.15.39:0/2959387934 learned_addr learned my addr 172.21.15.39:0/2959387934 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:54:27.900 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:27.896+0000 7f69e6ffd640 1 --2- 172.21.15.39:0/2959387934 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557c10d4f390 0x557c1112ed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:54:27.900 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:27.896+0000 7f69e77fe640 1 -- 172.21.15.39:0/2959387934 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557c10d4f390 msgr2=0x557c1112ed30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:54:27.900 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:27.896+0000 7f69e77fe640 1 --2- 172.21.15.39:0/2959387934 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557c10d4f390 0x557c1112ed30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:27.900 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:27.896+0000 7f69e77fe640 1 -- 172.21.15.39:0/2959387934 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557c10d70070 msgr2=0x557c1111fb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:54:27.900 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:27.896+0000 7f69e7fff640 1 --2- 172.21.15.39:0/2959387934 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557c10d70070 0x557c1111fb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:54:27.900 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:27.896+0000 7f69e77fe640 1 --2- 172.21.15.39:0/2959387934 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557c10d70070 0x557c1111fb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:27.900 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:27.896+0000 7f69e77fe640 1 -- 172.21.15.39:0/2959387934 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557c10d2ec20 con 0x557c10d471b0 2024-09-21T03:54:27.901 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:27.896+0000 7f69e6ffd640 1 --2- 172.21.15.39:0/2959387934 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557c10d4f390 0x557c1112ed30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:54:27.901 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:27.896+0000 7f69e77fe640 1 --2- 172.21.15.39:0/2959387934 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557c10d471b0 0x557c11122e40 secure :-1 s=READY pgs=1163 cs=0 l=1 rev1=1 crypto rx=0x7f69d8000c00 tx=0x7f69d80107a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d4b0e1c77eadc4d3 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:54:27.901 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:27.896+0000 7f69cd7fa640 1 -- 172.21.15.39:0/2959387934 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69d802b070 con 0x557c10d471b0 2024-09-21T03:54:27.901 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:27.896+0000 7f69cd7fa640 1 -- 172.21.15.39:0/2959387934 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f69d8004b30 con 0x557c10d471b0 2024-09-21T03:54:27.901 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:27.896+0000 7f69cd7fa640 1 -- 172.21.15.39:0/2959387934 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69d80116a0 con 0x557c10d471b0 2024-09-21T03:54:27.903 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:27.897+0000 7f69f408b740 1 -- 172.21.15.39:0/2959387934 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557c10d471b0 msgr2=0x557c11122e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:54:27.903 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:27.897+0000 7f69f408b740 1 --2- 172.21.15.39:0/2959387934 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557c10d471b0 0x557c11122e40 secure :-1 s=READY pgs=1163 cs=0 l=1 rev1=1 crypto rx=0x7f69d8000c00 tx=0x7f69d80107a0 comp rx=0 tx=0).stop 2024-09-21T03:54:27.903 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:27.897+0000 7f69f408b740 1 -- 172.21.15.39:0/2959387934 shutdown_connections 2024-09-21T03:54:27.904 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:27.897+0000 7f69f408b740 1 --2- 172.21.15.39:0/2959387934 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557c10d70070 0x557c1111fb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:27.904 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:27.897+0000 7f69f408b740 1 --2- 172.21.15.39:0/2959387934 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557c10d4f390 0x557c1112ed30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:27.904 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:27.897+0000 7f69f408b740 1 --2- 172.21.15.39:0/2959387934 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557c10d471b0 0x557c11122e40 unknown :-1 s=CLOSED pgs=1163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:27.904 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:27.897+0000 7f69f408b740 1 -- 172.21.15.39:0/2959387934 >> 172.21.15.39:0/2959387934 conn(0x557c10d64460 msgr2=0x557c11122730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:54:27.904 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:27.897+0000 7f69f408b740 1 -- 172.21.15.39:0/2959387934 shutdown_connections 2024-09-21T03:54:27.904 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:27.898+0000 7f69f408b740 1 -- 172.21.15.39:0/2959387934 wait complete. 2024-09-21T03:54:27.904 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:27.898+0000 7f69f408b740 1 -- start start 2024-09-21T03:54:27.904 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:27.898+0000 7f69f408b740 1 Processor -- start 2024-09-21T03:54:27.904 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:27.898+0000 7f69f408b740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557c10d471b0 0x557c111354b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:54:27.904 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:27.898+0000 7f69f408b740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557c10d4f390 0x557c11135a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:54:27.904 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:27.898+0000 7f69f408b740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557c10d70070 0x557c111972d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:54:27.904 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:27.898+0000 7f69f408b740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f69e81b0c60 con 0x557c10d471b0 2024-09-21T03:54:27.904 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:27.898+0000 7f69f408b740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f69e81b0590 con 0x557c10d4f390 2024-09-21T03:54:27.905 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:27.898+0000 7f69f408b740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f69e81ad7d0 con 0x557c10d70070 2024-09-21T03:54:27.905 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:27.898+0000 7f69f408b740 10 client.0 _ll_register_callbacks cb 0x7f69dc005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:54:27.905 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:27.899+0000 7f69e77fe640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557c10d471b0 0x557c111354b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:54:27.905 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:27.899+0000 7f69e77fe640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557c10d471b0 0x557c111354b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:53476/0 (socket says 172.21.15.39:53476) 2024-09-21T03:54:27.905 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:27.899+0000 7f69e77fe640 1 -- 172.21.15.39:0/2321271929 learned_addr learned my addr 172.21.15.39:0/2321271929 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:54:27.905 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:27.899+0000 7f69e6ffd640 1 --2- 172.21.15.39:0/2321271929 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557c10d4f390 0x557c11135a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:54:27.905 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:27.899+0000 7f69e7fff640 1 --2- 172.21.15.39:0/2321271929 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557c10d70070 0x557c111972d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:54:27.905 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:27.899+0000 7f69e77fe640 1 -- 172.21.15.39:0/2321271929 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557c10d70070 msgr2=0x557c111972d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:54:27.905 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:27.899+0000 7f69e77fe640 1 --2- 172.21.15.39:0/2321271929 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557c10d70070 0x557c111972d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:27.905 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:27.899+0000 7f69e77fe640 1 -- 172.21.15.39:0/2321271929 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557c10d4f390 msgr2=0x557c11135a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:54:27.905 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:27.899+0000 7f69e77fe640 1 --2- 172.21.15.39:0/2321271929 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557c10d4f390 0x557c11135a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:27.905 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:27.899+0000 7f69e77fe640 1 -- 172.21.15.39:0/2321271929 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557c11197940 con 0x557c10d471b0 2024-09-21T03:54:27.905 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:27.899+0000 7f69e77fe640 1 --2- 172.21.15.39:0/2321271929 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557c10d471b0 0x557c111354b0 secure :-1 s=READY pgs=1164 cs=0 l=1 rev1=1 crypto rx=0x7f69d8005f50 tx=0x7f69d800b480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e1d5a1f619a56ce5 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:54:27.905 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:27.899+0000 7f69c6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-09-21T03:54:27.905 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:27.899+0000 7f69c6ffd640 1 -- 172.21.15.39:0/2321271929 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69d802b070 con 0x557c10d471b0 2024-09-21T03:54:27.906 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:27.899+0000 7f69c6ffd640 1 -- 172.21.15.39:0/2321271929 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8528+0+0 (secure 0 0 0) 0x7f69d801b6b0 con 0x557c10d471b0 2024-09-21T03:54:27.906 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:27.899+0000 7f69f408b740 1 -- 172.21.15.39:0/2321271929 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f69d8004b30 con 0x557c10d471b0 2024-09-21T03:54:27.906 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:27.899+0000 7f69c6ffd640 1 -- 172.21.15.39:0/2321271929 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f69d801b8b0 con 0x557c10d471b0 2024-09-21T03:54:27.906 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:27.900+0000 7f69f408b740 10 client.17784 fetch_fsmap learned FSMap version 29 2024-09-21T03:54:27.906 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:27.900+0000 7f69f408b740 1 -- 172.21.15.39:0/2321271929 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x557c1119ebf0 con 0x557c10d471b0 2024-09-21T03:54:27.906 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:27.900+0000 7f69c6ffd640 1 -- 172.21.15.39:0/2321271929 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f69d8004b30 con 0x557c10d471b0 2024-09-21T03:54:27.906 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:27.900+0000 7f69c6ffd640 1 -- 172.21.15.39:0/2321271929 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f69d8004fd0 con 0x557c10d471b0 2024-09-21T03:54:27.906 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:27.900+0000 7f69f408b740 10 client.17784 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T03:54:27.906 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:27.900+0000 7f69f408b740 10 client.17784 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-21T03:54:27.906 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:27.900+0000 7f69f408b740 20 client.17784 populate_metadata read hostname 'smithi039' 2024-09-21T03:54:27.906 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:27.900+0000 7f69f408b740 1 --2- 172.21.15.39:0/2321271929 >> [v2:172.21.15.161:6838/2190107270,v1:172.21.15.161:6839/2190107270] conn(0x557c10d03da0 0x557c111a12f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:54:27.906 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:27.900+0000 7f69f408b740 4 client.17784 mds_command: new command op to 24667 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T03:54:27.906 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:27.900+0000 7f69f408b740 1 -- 172.21.15.39:0/2321271929 --> [v2:172.21.15.161:6838/2190107270,v1:172.21.15.161:6839/2190107270] -- command(tid 0: {"prefix": "damage ls"}) -- 0x557c1108b8a0 con 0x557c10d03da0 2024-09-21T03:54:27.906 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:27.900+0000 7f69e6ffd640 1 --2- 172.21.15.39:0/2321271929 >> [v2:172.21.15.161:6838/2190107270,v1:172.21.15.161:6839/2190107270] conn(0x557c10d03da0 0x557c111a12f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:54:27.907 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:27.901+0000 7f69c6ffd640 10 client.17784 ms_handle_connect on v2:172.21.15.161:6838/2190107270 2024-09-21T03:54:27.907 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:27.901+0000 7f69e6ffd640 1 --2- 172.21.15.39:0/2321271929 >> [v2:172.21.15.161:6838/2190107270,v1:172.21.15.161:6839/2190107270] conn(0x557c10d03da0 0x557c111a12f0 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=acf1a4d325345c94 server_cookie=c68bbc2ce5b28682 in_seq=0 out_seq=0 2024-09-21T03:54:27.908 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:27.901+0000 7f69c6ffd640 1 -- 172.21.15.39:0/2321271929 <== mds.1 v2:172.21.15.161:6838/2190107270 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x557c1108b8a0 con 0x557c10d03da0 2024-09-21T03:54:27.908 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:27.901+0000 7f69c6ffd640 10 client.17784 handle_command_reply: tid=0 2024-09-21T03:54:27.908 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:27.901+0000 7f69f408b740 1 client.17784 shutdown 2024-09-21T03:54:27.908 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:27.901+0000 7f69f408b740 1 -- 172.21.15.39:0/2321271929 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557c10d471b0 msgr2=0x557c111354b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:54:27.908 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:27.901+0000 7f69f408b740 1 --2- 172.21.15.39:0/2321271929 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557c10d471b0 0x557c111354b0 secure :-1 s=READY pgs=1164 cs=0 l=1 rev1=1 crypto rx=0x7f69d8005f50 tx=0x7f69d800b480 comp rx=0 tx=0).stop 2024-09-21T03:54:27.908 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:27.901+0000 7f69f408b740 1 -- 172.21.15.39:0/2321271929 shutdown_connections 2024-09-21T03:54:27.908 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:27.901+0000 7f69f408b740 1 --2- 172.21.15.39:0/2321271929 >> [v2:172.21.15.161:6838/2190107270,v1:172.21.15.161:6839/2190107270] conn(0x557c10d03da0 0x557c111a12f0 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:27.908 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:27.901+0000 7f69f408b740 1 --2- 172.21.15.39:0/2321271929 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557c10d70070 0x557c111972d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:27.908 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:27.901+0000 7f69f408b740 1 --2- 172.21.15.39:0/2321271929 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557c10d4f390 0x557c11135a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:27.908 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:27.901+0000 7f69f408b740 1 --2- 172.21.15.39:0/2321271929 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557c10d471b0 0x557c111354b0 unknown :-1 s=CLOSED pgs=1164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:27.908 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:27.901+0000 7f69f408b740 1 -- 172.21.15.39:0/2321271929 >> 172.21.15.39:0/2321271929 conn(0x557c10d64460 msgr2=0x557c11120e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:54:27.908 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:27.901+0000 7f69c6ffd640 0 client.17784 ms_handle_reset on v2:172.21.15.161:6838/2190107270 2024-09-21T03:54:27.908 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:27.901+0000 7f69f408b740 1 -- 172.21.15.39:0/2321271929 shutdown_connections 2024-09-21T03:54:27.909 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:27.901+0000 7f69f408b740 1 -- 172.21.15.39:0/2321271929 wait complete. 2024-09-21T03:54:27.909 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:27.902+0000 7f69f408b740 20 client.17784 trim_cache size 0 max 16384 2024-09-21T03:54:27.909 INFO:teuthology.orchestra.run.smithi039.stdout:[] 2024-09-21T03:54:27.910 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:27.902+0000 7f69edb17640 1 -- 172.21.15.39:0/3055289856 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f69d00b6680 msgr2=0x7f69d00b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:54:27.910 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:27.902+0000 7f69edb17640 1 --2- 172.21.15.39:0/3055289856 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f69d00b6680 0x7f69d00b8b90 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f69dc009230 tx=0x7f69dc000bc0 comp rx=0 tx=0).stop 2024-09-21T03:54:27.910 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:27.902+0000 7f69edb17640 1 -- 172.21.15.39:0/3055289856 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f69e810de60 msgr2=0x7f69e819fd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:54:27.910 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:27.902+0000 7f69edb17640 1 --2- 172.21.15.39:0/3055289856 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f69e810de60 0x7f69e819fd10 secure :-1 s=READY pgs=1161 cs=0 l=1 rev1=1 crypto rx=0x7f69e000eaa0 tx=0x7f69e000ef60 comp rx=0 tx=0).stop 2024-09-21T03:54:27.910 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:27.902+0000 7f69edb17640 1 -- 172.21.15.39:0/3055289856 shutdown_connections 2024-09-21T03:54:27.910 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:27.902+0000 7f69edb17640 1 --2- 172.21.15.39:0/3055289856 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f69d00b6680 0x7f69d00b8b90 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:27.910 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:27.902+0000 7f69edb17640 1 --2- 172.21.15.39:0/3055289856 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f69e810de60 0x7f69e819fd10 unknown :-1 s=CLOSED pgs=1161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:27.910 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:27.902+0000 7f69edb17640 1 --2- 172.21.15.39:0/3055289856 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f69e8105380 0x7f69e819aa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:27.910 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:27.902+0000 7f69edb17640 1 --2- 172.21.15.39:0/3055289856 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f69e8104a30 0x7f69e819a4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:27.910 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:27.902+0000 7f69edb17640 1 -- 172.21.15.39:0/3055289856 >> 172.21.15.39:0/3055289856 conn(0x7f69e8079760 msgr2=0x7f69e80ff830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:54:27.910 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:27.902+0000 7f69edb17640 1 -- 172.21.15.39:0/3055289856 shutdown_connections 2024-09-21T03:54:27.910 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:27.902+0000 7f69edb17640 1 -- 172.21.15.39:0/3055289856 wait complete. 2024-09-21T03:54:28.626 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 51116d5c-77ca-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 damage ls 2024-09-21T03:54:28.886 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/51116d5c-77ca-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:54:29.130 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:54:28 smithi039 ceph-mon[31211]: pgmap v582: 97 pgs: 97 active+clean; 10 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 102 B/s wr, 10 op/s 2024-09-21T03:54:29.218 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:54:28 smithi161 ceph-mon[34533]: pgmap v582: 97 pgs: 97 active+clean; 10 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 102 B/s wr, 10 op/s 2024-09-21T03:54:29.225 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:54:28 smithi150 ceph-mon[34289]: pgmap v582: 97 pgs: 97 active+clean; 10 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 102 B/s wr, 10 op/s 2024-09-21T03:54:29.334 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:29.331+0000 7f3853840640 1 -- 172.21.15.39:0/3120830933 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f384c102ac0 msgr2=0x7f384c102ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:54:29.334 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:29.331+0000 7f3853840640 1 --2- 172.21.15.39:0/3120830933 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f384c102ac0 0x7f384c102ed0 secure :-1 s=READY pgs=1165 cs=0 l=1 rev1=1 crypto rx=0x7f3844009900 tx=0x7f384402a410 comp rx=0 tx=0).stop 2024-09-21T03:54:29.334 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:29.331+0000 7f3853840640 1 -- 172.21.15.39:0/3120830933 shutdown_connections 2024-09-21T03:54:29.335 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:29.331+0000 7f3853840640 1 --2- 172.21.15.39:0/3120830933 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f384c108d00 0x7f384c10f530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:29.335 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:29.331+0000 7f3853840640 1 --2- 172.21.15.39:0/3120830933 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f384c1083b0 0x7f384c1087c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:29.335 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:29.331+0000 7f3853840640 1 --2- 172.21.15.39:0/3120830933 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f384c102ac0 0x7f384c102ed0 unknown :-1 s=CLOSED pgs=1165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:29.335 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:29.331+0000 7f3853840640 1 -- 172.21.15.39:0/3120830933 >> 172.21.15.39:0/3120830933 conn(0x7f384c079790 msgr2=0x7f384c079ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:54:29.335 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:29.331+0000 7f3853840640 1 -- 172.21.15.39:0/3120830933 shutdown_connections 2024-09-21T03:54:29.335 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:29.331+0000 7f3853840640 1 -- 172.21.15.39:0/3120830933 wait complete. 2024-09-21T03:54:29.337 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:29.332+0000 7f3853840640 1 Processor -- start 2024-09-21T03:54:29.337 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:29.332+0000 7f3853840640 1 -- start start 2024-09-21T03:54:29.337 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:29.332+0000 7f3853840640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f384c102ac0 0x7f384c0ff650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:54:29.338 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:29.332+0000 7f3853840640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f384c1083b0 0x7f384c0ffb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:54:29.338 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:29.332+0000 7f3853840640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f384c108d00 0x7f384c1000d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:54:29.338 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:29.332+0000 7f3853840640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f384c112a00 con 0x7f384c1083b0 2024-09-21T03:54:29.338 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:29.332+0000 7f3853840640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f384c112880 con 0x7f384c108d00 2024-09-21T03:54:29.338 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:29.332+0000 7f3853840640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f384c112b80 con 0x7f384c102ac0 2024-09-21T03:54:29.338 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:29.332+0000 7f3850ddb640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f384c1083b0 0x7f384c0ffb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:54:29.338 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:29.332+0000 7f3850ddb640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f384c1083b0 0x7f384c0ffb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:53490/0 (socket says 172.21.15.39:53490) 2024-09-21T03:54:29.338 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:29.332+0000 7f3850ddb640 1 -- 172.21.15.39:0/2320520961 learned_addr learned my addr 172.21.15.39:0/2320520961 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:54:29.338 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:29.332+0000 7f38515dc640 1 --2- 172.21.15.39:0/2320520961 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f384c102ac0 0x7f384c0ff650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:54:29.338 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:29.333+0000 7f3851ddd640 1 --2- 172.21.15.39:0/2320520961 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f384c108d00 0x7f384c1000d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:54:29.338 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:29.333+0000 7f3850ddb640 1 -- 172.21.15.39:0/2320520961 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f384c102ac0 msgr2=0x7f384c0ff650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:54:29.338 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:29.333+0000 7f3850ddb640 1 --2- 172.21.15.39:0/2320520961 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f384c102ac0 0x7f384c0ff650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:29.338 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:29.333+0000 7f3850ddb640 1 -- 172.21.15.39:0/2320520961 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f384c108d00 msgr2=0x7f384c1000d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:54:29.339 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:29.333+0000 7f3850ddb640 1 --2- 172.21.15.39:0/2320520961 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f384c108d00 0x7f384c1000d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:29.339 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:29.333+0000 7f3850ddb640 1 -- 172.21.15.39:0/2320520961 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f384c1a87a0 con 0x7f384c1083b0 2024-09-21T03:54:29.339 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:29.333+0000 7f3851ddd640 1 --2- 172.21.15.39:0/2320520961 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f384c108d00 0x7f384c1000d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:54:29.339 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:29.333+0000 7f3850ddb640 1 --2- 172.21.15.39:0/2320520961 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f384c1083b0 0x7f384c0ffb90 secure :-1 s=READY pgs=1166 cs=0 l=1 rev1=1 crypto rx=0x7f3840006fd0 tx=0x7f384000b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:54:29.339 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:29.333+0000 7f384a7fc640 1 -- 172.21.15.39:0/2320520961 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3840011830 con 0x7f384c1083b0 2024-09-21T03:54:29.339 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:29.333+0000 7f3853840640 1 -- 172.21.15.39:0/2320520961 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f384c1a8a90 con 0x7f384c1083b0 2024-09-21T03:54:29.339 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:29.333+0000 7f384a7fc640 1 -- 172.21.15.39:0/2320520961 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3840016070 con 0x7f384c1083b0 2024-09-21T03:54:29.339 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:29.333+0000 7f384a7fc640 1 -- 172.21.15.39:0/2320520961 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3840010e00 con 0x7f384c1083b0 2024-09-21T03:54:29.339 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:29.333+0000 7f3853840640 1 -- 172.21.15.39:0/2320520961 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f384c1a9020 con 0x7f384c1083b0 2024-09-21T03:54:29.341 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:29.336+0000 7f3859db4740 1 Processor -- start 2024-09-21T03:54:29.341 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:29.337+0000 7f3859db4740 1 -- start start 2024-09-21T03:54:29.341 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:29.337+0000 7f3859db4740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c2f4518070 0x55c2f48bf2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:54:29.341 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:29.337+0000 7f3859db4740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c2f44ef1b0 0x55c2f48bf800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:54:29.341 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:29.337+0000 7f3859db4740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c2f44f7390 0x55c2f48bfd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:54:29.341 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:29.337+0000 7f3859db4740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x55c2f4835ea0 con 0x55c2f44f7390 2024-09-21T03:54:29.341 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:29.337+0000 7f3859db4740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x55c2f48252d0 con 0x55c2f44ef1b0 2024-09-21T03:54:29.341 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:29.337+0000 7f384a7fc640 1 -- 172.21.15.39:0/2320520961 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 27) ==== 154643+0+0 (secure 0 0 0) 0x7f384001b460 con 0x7f384c1083b0 2024-09-21T03:54:29.341 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:29.337+0000 7f3859db4740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55c2f48290e0 con 0x55c2f4518070 2024-09-21T03:54:29.341 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:29.337+0000 7f38515dc640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c2f44ef1b0 0x55c2f48bf800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:54:29.342 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:29.337+0000 7f3851ddd640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c2f4518070 0x55c2f48bf2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:54:29.342 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:29.337+0000 7f38515dc640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c2f44ef1b0 0x55c2f48bf800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.39:51908/0 (socket says 172.21.15.39:51908) 2024-09-21T03:54:29.342 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:29.337+0000 7f38515dc640 1 -- 172.21.15.39:0/978205035 learned_addr learned my addr 172.21.15.39:0/978205035 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:54:29.342 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:29.337+0000 7f3851ddd640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c2f4518070 0x55c2f48bf2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.39:52764/0 (socket says 172.21.15.39:52764) 2024-09-21T03:54:29.342 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:29.337+0000 7f3850ddb640 1 --2- 172.21.15.39:0/978205035 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c2f44f7390 0x55c2f48bfd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:54:29.342 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:29.338+0000 7f3851ddd640 1 -- 172.21.15.39:0/978205035 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c2f44ef1b0 msgr2=0x55c2f48bf800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:54:29.342 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:29.338+0000 7f3851ddd640 1 --2- 172.21.15.39:0/978205035 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c2f44ef1b0 0x55c2f48bf800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:29.342 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:29.338+0000 7f3851ddd640 1 -- 172.21.15.39:0/978205035 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c2f44f7390 msgr2=0x55c2f48bfd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:54:29.342 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:29.338+0000 7f3851ddd640 1 --2- 172.21.15.39:0/978205035 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c2f44f7390 0x55c2f48bfd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:29.342 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:29.338+0000 7f3851ddd640 1 -- 172.21.15.39:0/978205035 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c2f4583650 con 0x55c2f4518070 2024-09-21T03:54:29.342 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:29.338+0000 7f38515dc640 1 --2- 172.21.15.39:0/978205035 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c2f44ef1b0 0x55c2f48bf800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:54:29.342 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:29.338+0000 7f3851ddd640 1 --2- 172.21.15.39:0/978205035 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c2f4518070 0x55c2f48bf2c0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f383c009ee0 tx=0x7f383c010740 comp rx=0 tx=0).ready entity=mon.1 client_cookie=da9c4462282d9cb4 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:54:29.342 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:29.338+0000 7f384a7fc640 1 --2- 172.21.15.39:0/2320520961 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f38340b6700 0x7f38340b8c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:54:29.343 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:29.338+0000 7f38515dc640 1 --2- 172.21.15.39:0/2320520961 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f38340b6700 0x7f38340b8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:54:29.343 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:29.338+0000 7f38337fe640 1 -- 172.21.15.39:0/978205035 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f383c016070 con 0x55c2f4518070 2024-09-21T03:54:29.343 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:29.338+0000 7f384a7fc640 1 -- 172.21.15.39:0/2320520961 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8528+0+0 (secure 0 0 0) 0x7f38400e6b90 con 0x7f384c1083b0 2024-09-21T03:54:29.343 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:29.338+0000 7f38337fe640 1 -- 172.21.15.39:0/978205035 <== mon.1 v2:172.21.15.161:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f383c01bb10 con 0x55c2f4518070 2024-09-21T03:54:29.343 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:29.338+0000 7f38337fe640 1 -- 172.21.15.39:0/978205035 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f383c012c00 con 0x55c2f4518070 2024-09-21T03:54:29.348 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:29.339+0000 7f3859db4740 1 -- 172.21.15.39:0/978205035 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c2f4518070 msgr2=0x55c2f48bf2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:54:29.348 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:29.339+0000 7f38515dc640 1 --2- 172.21.15.39:0/2320520961 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f38340b6700 0x7f38340b8c10 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x55c2f48bdf40 tx=0x7f3844031400 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:54:29.348 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:29.339+0000 7f3859db4740 1 --2- 172.21.15.39:0/978205035 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c2f4518070 0x55c2f48bf2c0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f383c009ee0 tx=0x7f383c010740 comp rx=0 tx=0).stop 2024-09-21T03:54:29.348 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:29.339+0000 7f3859db4740 1 -- 172.21.15.39:0/978205035 shutdown_connections 2024-09-21T03:54:29.348 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:29.339+0000 7f3859db4740 1 --2- 172.21.15.39:0/978205035 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c2f44f7390 0x55c2f48bfd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:29.348 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:29.339+0000 7f3859db4740 1 --2- 172.21.15.39:0/978205035 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c2f44ef1b0 0x55c2f48bf800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:29.348 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:29.339+0000 7f3859db4740 1 --2- 172.21.15.39:0/978205035 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c2f4518070 0x55c2f48bf2c0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:29.348 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:29.339+0000 7f3859db4740 1 -- 172.21.15.39:0/978205035 >> 172.21.15.39:0/978205035 conn(0x55c2f450c460 msgr2=0x55c2f48bd1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:54:29.348 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:29.339+0000 7f3859db4740 1 -- 172.21.15.39:0/978205035 shutdown_connections 2024-09-21T03:54:29.348 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:29.339+0000 7f3859db4740 1 -- 172.21.15.39:0/978205035 wait complete. 2024-09-21T03:54:29.348 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:29.340+0000 7f3859db4740 1 -- start start 2024-09-21T03:54:29.349 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:29.340+0000 7f3859db4740 1 Processor -- start 2024-09-21T03:54:29.349 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:29.340+0000 7f3859db4740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c2f44ef1b0 0x55c2f48ca9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:54:29.349 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:29.340+0000 7f3859db4740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c2f44f7390 0x55c2f48caef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:54:29.349 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:29.340+0000 7f3859db4740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c2f4518070 0x55c2f48d6dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:54:29.349 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:29.340+0000 7f3859db4740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x55c2f4686700 con 0x55c2f44f7390 2024-09-21T03:54:29.349 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:29.340+0000 7f3859db4740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x55c2f47c0160 con 0x55c2f4518070 2024-09-21T03:54:29.349 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:29.340+0000 7f3859db4740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55c2f48cdaa0 con 0x55c2f44ef1b0 2024-09-21T03:54:29.349 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:29.340+0000 7f3859db4740 10 client.0 _ll_register_callbacks cb 0x55c2f47d99e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:54:29.349 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:29.341+0000 7f3850ddb640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c2f44f7390 0x55c2f48caef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:54:29.349 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:29.341+0000 7f3850ddb640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c2f44f7390 0x55c2f48caef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:53510/0 (socket says 172.21.15.39:53510) 2024-09-21T03:54:29.349 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:29.341+0000 7f3850ddb640 1 -- 172.21.15.39:0/697329103 learned_addr learned my addr 172.21.15.39:0/697329103 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:54:29.349 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:29.341+0000 7f38515dc640 1 --2- 172.21.15.39:0/697329103 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c2f44ef1b0 0x55c2f48ca9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:54:29.349 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:29.341+0000 7f3851ddd640 1 --2- 172.21.15.39:0/697329103 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c2f4518070 0x55c2f48d6dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:54:29.350 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:29.341+0000 7f3850ddb640 1 -- 172.21.15.39:0/697329103 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c2f44ef1b0 msgr2=0x55c2f48ca9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:54:29.350 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:29.341+0000 7f3850ddb640 1 --2- 172.21.15.39:0/697329103 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c2f44ef1b0 0x55c2f48ca9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:29.350 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:29.341+0000 7f3850ddb640 1 -- 172.21.15.39:0/697329103 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c2f4518070 msgr2=0x55c2f48d6dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:54:29.350 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:29.341+0000 7f3850ddb640 1 --2- 172.21.15.39:0/697329103 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c2f4518070 0x55c2f48d6dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:29.350 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:29.341+0000 7f3850ddb640 1 -- 172.21.15.39:0/697329103 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c2f47edc90 con 0x55c2f44f7390 2024-09-21T03:54:29.350 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:29.341+0000 7f3851ddd640 1 --2- 172.21.15.39:0/697329103 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c2f4518070 0x55c2f48d6dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:54:29.350 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:29.341+0000 7f3850ddb640 1 --2- 172.21.15.39:0/697329103 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c2f44f7390 0x55c2f48caef0 secure :-1 s=READY pgs=1167 cs=0 l=1 rev1=1 crypto rx=0x7f38400ac970 tx=0x7f384000bf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b846fef418ff58be server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:54:29.350 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:29.341+0000 7f3830ff9640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-09-21T03:54:29.350 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:29.341+0000 7f3830ff9640 1 -- 172.21.15.39:0/697329103 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38400ac330 con 0x55c2f44f7390 2024-09-21T03:54:29.350 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:29.341+0000 7f3830ff9640 1 -- 172.21.15.39:0/697329103 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8528+0+0 (secure 0 0 0) 0x7f38400a8530 con 0x55c2f44f7390 2024-09-21T03:54:29.350 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:29.341+0000 7f3859db4740 1 -- 172.21.15.39:0/697329103 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f383c01bb10 con 0x55c2f44f7390 2024-09-21T03:54:29.350 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:29.342+0000 7f3830ff9640 1 -- 172.21.15.39:0/697329103 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f38400e4770 con 0x55c2f44f7390 2024-09-21T03:54:29.350 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:29.342+0000 7f3859db4740 10 client.17796 fetch_fsmap learned FSMap version 29 2024-09-21T03:54:29.351 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:29.342+0000 7f3859db4740 1 -- 172.21.15.39:0/697329103 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55c2f48d9870 con 0x55c2f44f7390 2024-09-21T03:54:29.351 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:29.342+0000 7f3830ff9640 1 -- 172.21.15.39:0/697329103 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f383c01bb10 con 0x55c2f44f7390 2024-09-21T03:54:29.351 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:29.342+0000 7f3830ff9640 1 -- 172.21.15.39:0/697329103 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f38400b2650 con 0x55c2f44f7390 2024-09-21T03:54:29.351 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:29.342+0000 7f3859db4740 10 client.17796 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T03:54:29.351 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:29.342+0000 7f3859db4740 10 client.17796 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T03:54:29.351 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:29.342+0000 7f3859db4740 20 client.17796 populate_metadata read hostname 'smithi039' 2024-09-21T03:54:29.351 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:29.342+0000 7f3859db4740 1 --2- 172.21.15.39:0/697329103 >> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] conn(0x55c2f44d69f0 0x55c2f48dcde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:54:29.351 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:29.342+0000 7f3859db4740 4 client.17796 mds_command: new command op to 24695 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:54:29.351 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:29.342+0000 7f3859db4740 1 -- 172.21.15.39:0/697329103 --> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c2f4837df0 con 0x55c2f44d69f0 2024-09-21T03:54:29.351 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:29.342+0000 7f38515dc640 1 --2- 172.21.15.39:0/697329103 >> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] conn(0x55c2f44d69f0 0x55c2f48dcde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:54:29.351 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:29.343+0000 7f38515dc640 1 --2- 172.21.15.39:0/697329103 >> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] conn(0x55c2f44d69f0 0x55c2f48dcde0 crc :-1 s=READY pgs=141 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ef1c9191bcc34ff6 server_cookie=27967b17727d4733 in_seq=0 out_seq=0 2024-09-21T03:54:29.351 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:29.343+0000 7f3830ff9640 10 client.17796 ms_handle_connect on v2:172.21.15.150:6838/2949386907 2024-09-21T03:54:29.351 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:29.344+0000 7f3830ff9640 1 -- 172.21.15.39:0/697329103 <== mds.0 v2:172.21.15.150:6838/2949386907 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55c2f4837df0 con 0x55c2f44d69f0 2024-09-21T03:54:29.351 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:29.344+0000 7f3830ff9640 10 client.17796 handle_command_reply: tid=0 2024-09-21T03:54:29.352 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:29.344+0000 7f3859db4740 1 client.17796 shutdown 2024-09-21T03:54:29.352 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:29.344+0000 7f3859db4740 1 -- 172.21.15.39:0/697329103 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c2f44f7390 msgr2=0x55c2f48caef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:54:29.352 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:29.344+0000 7f3859db4740 1 --2- 172.21.15.39:0/697329103 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c2f44f7390 0x55c2f48caef0 secure :-1 s=READY pgs=1167 cs=0 l=1 rev1=1 crypto rx=0x7f38400ac970 tx=0x7f384000bf90 comp rx=0 tx=0).stop 2024-09-21T03:54:29.352 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:29.344+0000 7f3859db4740 1 -- 172.21.15.39:0/697329103 shutdown_connections 2024-09-21T03:54:29.352 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:29.344+0000 7f3859db4740 1 --2- 172.21.15.39:0/697329103 >> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] conn(0x55c2f44d69f0 0x55c2f48dcde0 crc :-1 s=READY pgs=141 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:29.352 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:29.344+0000 7f3859db4740 1 --2- 172.21.15.39:0/697329103 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c2f4518070 0x55c2f48d6dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:29.352 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:29.344+0000 7f3859db4740 1 --2- 172.21.15.39:0/697329103 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c2f44f7390 0x55c2f48caef0 unknown :-1 s=CLOSED pgs=1167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:29.352 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:29.344+0000 7f3830ff9640 0 client.17796 ms_handle_reset on v2:172.21.15.150:6838/2949386907 2024-09-21T03:54:29.352 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:29.344+0000 7f3859db4740 1 --2- 172.21.15.39:0/697329103 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c2f44ef1b0 0x55c2f48ca9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:29.352 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:29.344+0000 7f3859db4740 1 -- 172.21.15.39:0/697329103 >> 172.21.15.39:0/697329103 conn(0x55c2f450c460 msgr2=0x55c2f48c2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:54:29.352 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:29.344+0000 7f3859db4740 1 -- 172.21.15.39:0/697329103 shutdown_connections 2024-09-21T03:54:29.352 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:29.344+0000 7f3859db4740 1 -- 172.21.15.39:0/697329103 wait complete. 2024-09-21T03:54:29.352 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:29.344+0000 7f3859db4740 20 client.17796 trim_cache size 0 max 16384 2024-09-21T03:54:29.361 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:29.357+0000 7f3859db4740 1 Processor -- start 2024-09-21T03:54:29.361 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:29.358+0000 7f3859db4740 1 -- start start 2024-09-21T03:54:29.362 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:29.358+0000 7f3859db4740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c2f44ef1b0 0x55c2f48cc890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:54:29.362 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:29.358+0000 7f3859db4740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c2f44f7390 0x55c2f48c0a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:54:29.362 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:29.358+0000 7f3859db4740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c2f4518070 0x55c2f48d6cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:54:29.362 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:29.358+0000 7f3859db4740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f384c1b1a20 con 0x55c2f44f7390 2024-09-21T03:54:29.362 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:29.358+0000 7f3859db4740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f384c1b3560 con 0x55c2f4518070 2024-09-21T03:54:29.362 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:29.358+0000 7f3859db4740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f384c1b20f0 con 0x55c2f44ef1b0 2024-09-21T03:54:29.362 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:29.358+0000 7f3850ddb640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c2f44f7390 0x55c2f48c0a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:54:29.362 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:29.358+0000 7f3850ddb640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c2f44f7390 0x55c2f48c0a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:53518/0 (socket says 172.21.15.39:53518) 2024-09-21T03:54:29.362 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:29.358+0000 7f3850ddb640 1 -- 172.21.15.39:0/2178958411 learned_addr learned my addr 172.21.15.39:0/2178958411 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:54:29.362 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:29.358+0000 7f38515dc640 1 --2- 172.21.15.39:0/2178958411 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c2f44ef1b0 0x55c2f48cc890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:54:29.362 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:29.358+0000 7f3851ddd640 1 --2- 172.21.15.39:0/2178958411 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c2f4518070 0x55c2f48d6cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:54:29.362 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:29.358+0000 7f3850ddb640 1 -- 172.21.15.39:0/2178958411 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c2f44ef1b0 msgr2=0x55c2f48cc890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:54:29.363 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:29.358+0000 7f3850ddb640 1 --2- 172.21.15.39:0/2178958411 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c2f44ef1b0 0x55c2f48cc890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:29.363 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:29.358+0000 7f3850ddb640 1 -- 172.21.15.39:0/2178958411 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c2f4518070 msgr2=0x55c2f48d6cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:54:29.363 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:29.358+0000 7f3850ddb640 1 --2- 172.21.15.39:0/2178958411 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c2f4518070 0x55c2f48d6cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:29.363 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:29.358+0000 7f3850ddb640 1 -- 172.21.15.39:0/2178958411 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c2f48b85d0 con 0x55c2f44f7390 2024-09-21T03:54:29.363 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:29.358+0000 7f3851ddd640 1 --2- 172.21.15.39:0/2178958411 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c2f4518070 0x55c2f48d6cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:54:29.363 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:29.358+0000 7f3850ddb640 1 --2- 172.21.15.39:0/2178958411 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c2f44f7390 0x55c2f48c0a00 secure :-1 s=READY pgs=1168 cs=0 l=1 rev1=1 crypto rx=0x7f38400b29f0 tx=0x7f3840010680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aebfc7ce26b0ebfc server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:54:29.363 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:29.359+0000 7f3832ffd640 1 -- 172.21.15.39:0/2178958411 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38400ac150 con 0x55c2f44f7390 2024-09-21T03:54:29.363 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:29.359+0000 7f3832ffd640 1 -- 172.21.15.39:0/2178958411 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f38400ac4f0 con 0x55c2f44f7390 2024-09-21T03:54:29.363 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:29.359+0000 7f3832ffd640 1 -- 172.21.15.39:0/2178958411 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38400a9030 con 0x55c2f44f7390 2024-09-21T03:54:29.367 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:29.359+0000 7f3859db4740 1 -- 172.21.15.39:0/2178958411 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c2f44f7390 msgr2=0x55c2f48c0a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:54:29.368 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:29.359+0000 7f3859db4740 1 --2- 172.21.15.39:0/2178958411 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c2f44f7390 0x55c2f48c0a00 secure :-1 s=READY pgs=1168 cs=0 l=1 rev1=1 crypto rx=0x7f38400b29f0 tx=0x7f3840010680 comp rx=0 tx=0).stop 2024-09-21T03:54:29.368 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:29.359+0000 7f3859db4740 1 -- 172.21.15.39:0/2178958411 shutdown_connections 2024-09-21T03:54:29.368 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:29.359+0000 7f3859db4740 1 --2- 172.21.15.39:0/2178958411 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c2f4518070 0x55c2f48d6cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:29.368 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:29.359+0000 7f3859db4740 1 --2- 172.21.15.39:0/2178958411 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c2f44f7390 0x55c2f48c0a00 unknown :-1 s=CLOSED pgs=1168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:29.368 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:29.359+0000 7f3859db4740 1 --2- 172.21.15.39:0/2178958411 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c2f44ef1b0 0x55c2f48cc890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:29.368 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:29.359+0000 7f3859db4740 1 -- 172.21.15.39:0/2178958411 >> 172.21.15.39:0/2178958411 conn(0x55c2f450c460 msgr2=0x55c2f48c02f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:54:29.368 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:29.360+0000 7f3859db4740 1 -- 172.21.15.39:0/2178958411 shutdown_connections 2024-09-21T03:54:29.368 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:29.360+0000 7f3859db4740 1 -- 172.21.15.39:0/2178958411 wait complete. 2024-09-21T03:54:29.368 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:29.360+0000 7f3859db4740 1 -- start start 2024-09-21T03:54:29.368 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:29.360+0000 7f3859db4740 1 Processor -- start 2024-09-21T03:54:29.368 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:29.360+0000 7f3859db4740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c2f44ef1b0 0x55c2f48d36b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:54:29.368 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:29.360+0000 7f3859db4740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c2f44f7390 0x55c2f48cebb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:54:29.368 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:29.360+0000 7f3859db4740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c2f4518070 0x55c2f48cf2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:54:29.369 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:29.360+0000 7f3859db4740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f384c1b2e90 con 0x55c2f44f7390 2024-09-21T03:54:29.369 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:29.360+0000 7f3859db4740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f384c1b27c0 con 0x55c2f44ef1b0 2024-09-21T03:54:29.369 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:29.360+0000 7f3859db4740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f384c1afa00 con 0x55c2f4518070 2024-09-21T03:54:29.369 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:29.361+0000 7f3859db4740 10 client.0 _ll_register_callbacks cb 0x7f3840010fc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:54:29.369 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:29.361+0000 7f3850ddb640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c2f44f7390 0x55c2f48cebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:54:29.369 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:29.361+0000 7f3850ddb640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c2f44f7390 0x55c2f48cebb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:53526/0 (socket says 172.21.15.39:53526) 2024-09-21T03:54:29.369 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:29.361+0000 7f3850ddb640 1 -- 172.21.15.39:0/3612114728 learned_addr learned my addr 172.21.15.39:0/3612114728 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:54:29.369 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:29.361+0000 7f38515dc640 1 --2- 172.21.15.39:0/3612114728 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c2f44ef1b0 0x55c2f48d36b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:54:29.369 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:29.361+0000 7f3851ddd640 1 --2- 172.21.15.39:0/3612114728 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c2f4518070 0x55c2f48cf2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:54:29.369 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:29.361+0000 7f3850ddb640 1 -- 172.21.15.39:0/3612114728 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c2f4518070 msgr2=0x55c2f48cf2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:54:29.369 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:29.361+0000 7f3850ddb640 1 --2- 172.21.15.39:0/3612114728 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c2f4518070 0x55c2f48cf2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:29.369 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:29.361+0000 7f3850ddb640 1 -- 172.21.15.39:0/3612114728 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c2f44ef1b0 msgr2=0x55c2f48d36b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:54:29.369 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:29.361+0000 7f3850ddb640 1 --2- 172.21.15.39:0/3612114728 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c2f44ef1b0 0x55c2f48d36b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:29.369 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:29.361+0000 7f3850ddb640 1 -- 172.21.15.39:0/3612114728 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c2f4946760 con 0x55c2f44f7390 2024-09-21T03:54:29.370 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:29.361+0000 7f3850ddb640 1 --2- 172.21.15.39:0/3612114728 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c2f44f7390 0x55c2f48cebb0 secure :-1 s=READY pgs=1169 cs=0 l=1 rev1=1 crypto rx=0x7f38400e4420 tx=0x7f38400b13f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e1ca2f56baaeac3e server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:54:29.370 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:29.361+0000 7f3830ff9640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-09-21T03:54:29.370 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:29.361+0000 7f3830ff9640 1 -- 172.21.15.39:0/3612114728 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38400a9030 con 0x55c2f44f7390 2024-09-21T03:54:29.370 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:29.361+0000 7f3830ff9640 1 -- 172.21.15.39:0/3612114728 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8528+0+0 (secure 0 0 0) 0x7f38400c2b90 con 0x55c2f44f7390 2024-09-21T03:54:29.370 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:29.361+0000 7f3859db4740 1 -- 172.21.15.39:0/3612114728 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f38400ac4f0 con 0x55c2f44f7390 2024-09-21T03:54:29.370 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:29.362+0000 7f3830ff9640 1 -- 172.21.15.39:0/3612114728 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f38400ac150 con 0x55c2f44f7390 2024-09-21T03:54:29.370 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:29.362+0000 7f3859db4740 10 client.17802 fetch_fsmap learned FSMap version 29 2024-09-21T03:54:29.370 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:29.362+0000 7f3859db4740 1 -- 172.21.15.39:0/3612114728 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55c2f49477c0 con 0x55c2f44f7390 2024-09-21T03:54:29.370 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:29.362+0000 7f3830ff9640 1 -- 172.21.15.39:0/3612114728 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f38400ac4f0 con 0x55c2f44f7390 2024-09-21T03:54:29.371 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:29.362+0000 7f3830ff9640 1 -- 172.21.15.39:0/3612114728 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f38400b5940 con 0x55c2f44f7390 2024-09-21T03:54:29.371 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:29.362+0000 7f3859db4740 10 client.17802 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T03:54:29.371 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:29.362+0000 7f3859db4740 10 client.17802 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T03:54:29.371 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:29.362+0000 7f3859db4740 20 client.17802 populate_metadata read hostname 'smithi039' 2024-09-21T03:54:29.371 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:29.362+0000 7f3859db4740 1 --2- 172.21.15.39:0/3612114728 >> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] conn(0x55c2f44abda0 0x55c2f4949cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:54:29.371 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:29.362+0000 7f3859db4740 4 client.17802 mds_command: new command op to 24695 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T03:54:29.371 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:29.362+0000 7f3859db4740 1 -- 172.21.15.39:0/3612114728 --> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55c2f48338a0 con 0x55c2f44abda0 2024-09-21T03:54:29.371 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:29.362+0000 7f38515dc640 1 --2- 172.21.15.39:0/3612114728 >> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] conn(0x55c2f44abda0 0x55c2f4949cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:54:29.371 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:29.363+0000 7f3830ff9640 10 client.17802 ms_handle_connect on v2:172.21.15.150:6838/2949386907 2024-09-21T03:54:29.371 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:29.363+0000 7f38515dc640 1 --2- 172.21.15.39:0/3612114728 >> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] conn(0x55c2f44abda0 0x55c2f4949cb0 crc :-1 s=READY pgs=142 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=61f74ddade54fd64 server_cookie=75aff8ad112029f8 in_seq=0 out_seq=0 2024-09-21T03:54:29.372 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:29.363+0000 7f3830ff9640 1 -- 172.21.15.39:0/3612114728 <== mds.0 v2:172.21.15.150:6838/2949386907 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55c2f48338a0 con 0x55c2f44abda0 2024-09-21T03:54:29.372 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:29.363+0000 7f3830ff9640 10 client.17802 handle_command_reply: tid=0 2024-09-21T03:54:29.372 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:29.363+0000 7f3859db4740 1 client.17802 shutdown 2024-09-21T03:54:29.372 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:29.363+0000 7f3859db4740 1 -- 172.21.15.39:0/3612114728 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c2f44f7390 msgr2=0x55c2f48cebb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:54:29.372 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:29.363+0000 7f3859db4740 1 --2- 172.21.15.39:0/3612114728 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c2f44f7390 0x55c2f48cebb0 secure :-1 s=READY pgs=1169 cs=0 l=1 rev1=1 crypto rx=0x7f38400e4420 tx=0x7f38400b13f0 comp rx=0 tx=0).stop 2024-09-21T03:54:29.372 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:29.363+0000 7f3859db4740 1 -- 172.21.15.39:0/3612114728 shutdown_connections 2024-09-21T03:54:29.372 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:29.363+0000 7f3859db4740 1 --2- 172.21.15.39:0/3612114728 >> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] conn(0x55c2f44abda0 0x55c2f4949cb0 crc :-1 s=READY pgs=142 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:29.372 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:29.363+0000 7f3859db4740 1 --2- 172.21.15.39:0/3612114728 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c2f4518070 0x55c2f48cf2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:29.372 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:29.363+0000 7f3859db4740 1 --2- 172.21.15.39:0/3612114728 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c2f44f7390 0x55c2f48cebb0 unknown :-1 s=CLOSED pgs=1169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:29.372 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:29.363+0000 7f3859db4740 1 --2- 172.21.15.39:0/3612114728 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c2f44ef1b0 0x55c2f48d36b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:29.372 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:29.363+0000 7f3859db4740 1 -- 172.21.15.39:0/3612114728 >> 172.21.15.39:0/3612114728 conn(0x55c2f450c460 msgr2=0x55c2f48cc7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:54:29.372 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:29.363+0000 7f3830ff9640 0 client.17802 ms_handle_reset on v2:172.21.15.150:6838/2949386907 2024-09-21T03:54:29.372 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:29.364+0000 7f3859db4740 1 -- 172.21.15.39:0/3612114728 shutdown_connections 2024-09-21T03:54:29.372 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:29.364+0000 7f3859db4740 1 -- 172.21.15.39:0/3612114728 wait complete. 2024-09-21T03:54:29.373 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:29.364+0000 7f3859db4740 20 client.17802 trim_cache size 0 max 16384 2024-09-21T03:54:29.373 INFO:teuthology.orchestra.run.smithi039.stdout:[] 2024-09-21T03:54:29.374 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:29.364+0000 7f3853840640 1 -- 172.21.15.39:0/2320520961 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f38340b6700 msgr2=0x7f38340b8c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:54:29.374 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:29.364+0000 7f3853840640 1 --2- 172.21.15.39:0/2320520961 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f38340b6700 0x7f38340b8c10 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x55c2f48bdf40 tx=0x7f3844031400 comp rx=0 tx=0).stop 2024-09-21T03:54:29.374 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:29.364+0000 7f3853840640 1 -- 172.21.15.39:0/2320520961 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f384c1083b0 msgr2=0x7f384c0ffb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:54:29.374 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:29.364+0000 7f3853840640 1 --2- 172.21.15.39:0/2320520961 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f384c1083b0 0x7f384c0ffb90 secure :-1 s=READY pgs=1166 cs=0 l=1 rev1=1 crypto rx=0x7f3840006fd0 tx=0x7f384000b8e0 comp rx=0 tx=0).stop 2024-09-21T03:54:29.374 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:29.364+0000 7f3853840640 1 -- 172.21.15.39:0/2320520961 shutdown_connections 2024-09-21T03:54:29.374 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:29.364+0000 7f3853840640 1 --2- 172.21.15.39:0/2320520961 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f38340b6700 0x7f38340b8c10 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:29.374 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:29.364+0000 7f3853840640 1 --2- 172.21.15.39:0/2320520961 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f384c108d00 0x7f384c1000d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:29.374 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:29.364+0000 7f3853840640 1 --2- 172.21.15.39:0/2320520961 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f384c1083b0 0x7f384c0ffb90 unknown :-1 s=CLOSED pgs=1166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:29.374 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:29.364+0000 7f3853840640 1 --2- 172.21.15.39:0/2320520961 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f384c102ac0 0x7f384c0ff650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:29.374 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:29.364+0000 7f3853840640 1 -- 172.21.15.39:0/2320520961 >> 172.21.15.39:0/2320520961 conn(0x7f384c079790 msgr2=0x7f384c101ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:54:29.374 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:29.365+0000 7f3853840640 1 -- 172.21.15.39:0/2320520961 shutdown_connections 2024-09-21T03:54:29.374 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:29.365+0000 7f3853840640 1 -- 172.21.15.39:0/2320520961 wait complete. 2024-09-21T03:54:31.093 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-21T03:54:31.093 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 51116d5c-77ca-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-21T03:54:31.218 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:54:30 smithi161 ceph-mon[34533]: pgmap v583: 97 pgs: 97 active+clean; 10 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 10 op/s 2024-09-21T03:54:31.218 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:54:30 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:54:31.218 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:54:30 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:54:31.218 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:54:30 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:54:31.218 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:54:30 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:54:31.225 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:54:30 smithi150 ceph-mon[34289]: pgmap v583: 97 pgs: 97 active+clean; 10 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 10 op/s 2024-09-21T03:54:31.225 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:54:30 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:54:31.225 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:54:30 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:54:31.225 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:54:30 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:54:31.225 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:54:30 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:54:31.334 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:54:30 smithi039 ceph-mon[31211]: pgmap v583: 97 pgs: 97 active+clean; 10 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 10 op/s 2024-09-21T03:54:31.334 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:54:30 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:54:31.334 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:54:30 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:54:31.334 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:54:30 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:54:31.334 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:54:30 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:54:31.356 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/51116d5c-77ca-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:54:31.833 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:31.817+0000 7f89da03d640 1 -- 172.21.15.39:0/1346181069 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f89d41083d0 msgr2=0x7f89d41087e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:54:31.833 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:31.817+0000 7f89da03d640 1 --2- 172.21.15.39:0/1346181069 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f89d41083d0 0x7f89d41087e0 secure :-1 s=READY pgs=1170 cs=0 l=1 rev1=1 crypto rx=0x7f89c8009900 tx=0x7f89c802a410 comp rx=0 tx=0).stop 2024-09-21T03:54:31.833 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:31.817+0000 7f89da03d640 1 -- 172.21.15.39:0/1346181069 shutdown_connections 2024-09-21T03:54:31.833 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:31.817+0000 7f89da03d640 1 --2- 172.21.15.39:0/1346181069 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f89d4108d20 0x7f89d410f550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:31.833 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:31.817+0000 7f89da03d640 1 --2- 172.21.15.39:0/1346181069 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f89d41083d0 0x7f89d41087e0 unknown :-1 s=CLOSED pgs=1170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:31.833 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:31.817+0000 7f89da03d640 1 --2- 172.21.15.39:0/1346181069 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f89d4102ae0 0x7f89d4102ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:31.833 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:31.817+0000 7f89da03d640 1 -- 172.21.15.39:0/1346181069 >> 172.21.15.39:0/1346181069 conn(0x7f89d4079760 msgr2=0x7f89d4079b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:54:31.833 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:31.818+0000 7f89da03d640 1 -- 172.21.15.39:0/1346181069 shutdown_connections 2024-09-21T03:54:31.833 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:31.818+0000 7f89da03d640 1 -- 172.21.15.39:0/1346181069 wait complete. 2024-09-21T03:54:31.834 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:31.818+0000 7f89da03d640 1 Processor -- start 2024-09-21T03:54:31.834 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:31.818+0000 7f89da03d640 1 -- start start 2024-09-21T03:54:31.834 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:31.819+0000 7f89da03d640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f89d4102ae0 0x7f89d419b2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:54:31.834 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:31.819+0000 7f89da03d640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f89d41083d0 0x7f89d419b820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:54:31.834 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:31.819+0000 7f89da03d640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f89d4108d20 0x7f89d419bd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:54:31.834 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:31.819+0000 7f89da03d640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f89d4112990 con 0x7f89d4108d20 2024-09-21T03:54:31.834 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:31.819+0000 7f89da03d640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f89d4112810 con 0x7f89d41083d0 2024-09-21T03:54:31.834 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:31.819+0000 7f89da03d640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f89d4112b10 con 0x7f89d4102ae0 2024-09-21T03:54:31.834 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:31.819+0000 7f89d3fff640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f89d4108d20 0x7f89d419bd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:54:31.834 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:31.819+0000 7f89d3fff640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f89d4108d20 0x7f89d419bd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:53540/0 (socket says 172.21.15.39:53540) 2024-09-21T03:54:31.835 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:31.819+0000 7f89d3fff640 1 -- 172.21.15.39:0/231462861 learned_addr learned my addr 172.21.15.39:0/231462861 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:54:31.835 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:31.819+0000 7f89d37fe640 1 --2- 172.21.15.39:0/231462861 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f89d4102ae0 0x7f89d419b2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:54:31.835 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:31.819+0000 7f89d2ffd640 1 --2- 172.21.15.39:0/231462861 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f89d41083d0 0x7f89d419b820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:54:31.835 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:31.819+0000 7f89d3fff640 1 -- 172.21.15.39:0/231462861 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f89d4102ae0 msgr2=0x7f89d419b2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:54:31.835 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:31.819+0000 7f89d3fff640 1 --2- 172.21.15.39:0/231462861 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f89d4102ae0 0x7f89d419b2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:31.835 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:31.819+0000 7f89d3fff640 1 -- 172.21.15.39:0/231462861 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f89d41083d0 msgr2=0x7f89d419b820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:54:31.835 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:31.819+0000 7f89d3fff640 1 --2- 172.21.15.39:0/231462861 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f89d41083d0 0x7f89d419b820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:31.835 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:31.819+0000 7f89d3fff640 1 -- 172.21.15.39:0/231462861 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f89d41a6570 con 0x7f89d4108d20 2024-09-21T03:54:31.835 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:31.819+0000 7f89d3fff640 1 --2- 172.21.15.39:0/231462861 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f89d4108d20 0x7f89d419bd60 secure :-1 s=READY pgs=1171 cs=0 l=1 rev1=1 crypto rx=0x7f89cc00eaa0 tx=0x7f89cc00ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:54:31.835 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:31.819+0000 7f89d0ff9640 1 -- 172.21.15.39:0/231462861 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89cc00cdb0 con 0x7f89d4108d20 2024-09-21T03:54:31.835 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:31.819+0000 7f89d0ff9640 1 -- 172.21.15.39:0/231462861 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f89cc005020 con 0x7f89d4108d20 2024-09-21T03:54:31.835 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:31.820+0000 7f89da03d640 1 -- 172.21.15.39:0/231462861 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f89d41a6860 con 0x7f89d4108d20 2024-09-21T03:54:31.835 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:31.820+0000 7f89d0ff9640 1 -- 172.21.15.39:0/231462861 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89cc0116d0 con 0x7f89d4108d20 2024-09-21T03:54:31.835 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:31.820+0000 7f89da03d640 1 -- 172.21.15.39:0/231462861 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f89d41a6d20 con 0x7f89d4108d20 2024-09-21T03:54:31.836 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:31.822+0000 7f89d0ff9640 1 -- 172.21.15.39:0/231462861 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 27) ==== 154643+0+0 (secure 0 0 0) 0x7f89cc011870 con 0x7f89d4108d20 2024-09-21T03:54:31.836 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:31.822+0000 7f89e05b1740 1 Processor -- start 2024-09-21T03:54:31.836 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:31.822+0000 7f89e05b1740 1 -- start start 2024-09-21T03:54:31.836 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:31.823+0000 7f89e05b1740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e397edd350 0x55e398284430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:54:31.836 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:31.823+0000 7f89e05b1740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e397eb4490 0x55e398284970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:54:31.836 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:31.823+0000 7f89e05b1740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55e397ebc670 0x55e398284eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:54:31.836 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:31.823+0000 7f89e05b1740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x55e3981fae60 con 0x55e397ebc670 2024-09-21T03:54:31.836 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:31.823+0000 7f89e05b1740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x55e3981ea290 con 0x55e397eb4490 2024-09-21T03:54:31.836 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:31.823+0000 7f89e05b1740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55e3981ee0a0 con 0x55e397edd350 2024-09-21T03:54:31.836 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:31.823+0000 7f89d2ffd640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55e397ebc670 0x55e398284eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:54:31.836 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:31.823+0000 7f89d2ffd640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55e397ebc670 0x55e398284eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:53546/0 (socket says 172.21.15.39:53546) 2024-09-21T03:54:31.836 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:31.823+0000 7f89d2ffd640 1 -- 172.21.15.39:0/761504188 learned_addr learned my addr 172.21.15.39:0/761504188 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:54:31.837 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:31.823+0000 7f89d37fe640 1 --2- 172.21.15.39:0/761504188 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e397eb4490 0x55e398284970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:54:31.837 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:31.823+0000 7f89d2ffd640 1 --2- 172.21.15.39:0/761504188 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e397edd350 0x55e398284430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:54:31.837 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:31.823+0000 7f89d0ff9640 1 --2- 172.21.15.39:0/231462861 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f89bc0b66b0 0x7f89bc0b8bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:54:31.837 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:31.823+0000 7f89d37fe640 1 -- 172.21.15.39:0/761504188 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e397edd350 msgr2=0x55e398284430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:54:31.837 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:31.823+0000 7f89d37fe640 1 --2- 172.21.15.39:0/761504188 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e397edd350 0x55e398284430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:31.837 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:31.823+0000 7f89d37fe640 1 -- 172.21.15.39:0/761504188 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55e397ebc670 msgr2=0x55e398284eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:54:31.837 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:31.823+0000 7f89d37fe640 1 --2- 172.21.15.39:0/761504188 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55e397ebc670 0x55e398284eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:31.837 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:31.823+0000 7f89d37fe640 1 -- 172.21.15.39:0/761504188 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e397f48930 con 0x55e397eb4490 2024-09-21T03:54:31.837 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:31.823+0000 7f89d3fff640 1 --2- 172.21.15.39:0/231462861 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f89bc0b66b0 0x7f89bc0b8bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:54:31.837 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:31.824+0000 7f89d2ffd640 1 --2- 172.21.15.39:0/761504188 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55e397ebc670 0x55e398284eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:54:31.837 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:31.824+0000 7f89d0ff9640 1 -- 172.21.15.39:0/231462861 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8528+0+0 (secure 0 0 0) 0x7f89cc0e7850 con 0x7f89d4108d20 2024-09-21T03:54:31.837 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:31.824+0000 7f89d37fe640 1 --2- 172.21.15.39:0/761504188 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e397eb4490 0x55e398284970 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f89c4005fd0 tx=0x7f89c400bb20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1babbf2a62785fd2 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:54:31.837 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:31.824+0000 7f89b9ffb640 1 -- 172.21.15.39:0/761504188 <== mon.2 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89c4003bd0 con 0x55e397eb4490 2024-09-21T03:54:31.838 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:31.824+0000 7f89d3fff640 1 --2- 172.21.15.39:0/231462861 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f89bc0b66b0 0x7f89bc0b8bc0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f89cc0a9c70 tx=0x7f89cc005890 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:54:31.838 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:31.824+0000 7f89b9ffb640 1 -- 172.21.15.39:0/761504188 <== mon.2 v2:172.21.15.150:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f89c4015070 con 0x55e397eb4490 2024-09-21T03:54:31.838 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:31.824+0000 7f89b9ffb640 1 -- 172.21.15.39:0/761504188 <== mon.2 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89c4004d20 con 0x55e397eb4490 2024-09-21T03:54:31.838 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:31.825+0000 7f89e05b1740 1 -- 172.21.15.39:0/761504188 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e397eb4490 msgr2=0x55e398284970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:54:31.838 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:31.825+0000 7f89e05b1740 1 --2- 172.21.15.39:0/761504188 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e397eb4490 0x55e398284970 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f89c4005fd0 tx=0x7f89c400bb20 comp rx=0 tx=0).stop 2024-09-21T03:54:31.838 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:31.825+0000 7f89e05b1740 1 -- 172.21.15.39:0/761504188 shutdown_connections 2024-09-21T03:54:31.838 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:31.825+0000 7f89e05b1740 1 --2- 172.21.15.39:0/761504188 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55e397ebc670 0x55e398284eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:31.838 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:31.825+0000 7f89e05b1740 1 --2- 172.21.15.39:0/761504188 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e397eb4490 0x55e398284970 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:31.838 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:31.825+0000 7f89e05b1740 1 --2- 172.21.15.39:0/761504188 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e397edd350 0x55e398284430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:31.838 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:31.825+0000 7f89e05b1740 1 -- 172.21.15.39:0/761504188 >> 172.21.15.39:0/761504188 conn(0x55e397ed1740 msgr2=0x55e398282320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:54:31.838 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:31.825+0000 7f89e05b1740 1 -- 172.21.15.39:0/761504188 shutdown_connections 2024-09-21T03:54:31.838 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:31.825+0000 7f89e05b1740 1 -- 172.21.15.39:0/761504188 wait complete. 2024-09-21T03:54:31.838 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:31.825+0000 7f89e05b1740 1 -- start start 2024-09-21T03:54:31.838 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:31.826+0000 7f89e05b1740 1 Processor -- start 2024-09-21T03:54:31.838 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:31.826+0000 7f89e05b1740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55e397eb4490 0x55e398294390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:54:31.839 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:31.826+0000 7f89e05b1740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e397ebc670 0x55e3982948d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:54:31.839 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:31.826+0000 7f89e05b1740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e397edd350 0x55e3982a0270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:54:31.839 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:31.826+0000 7f89e05b1740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x55e39804b890 con 0x55e397eb4490 2024-09-21T03:54:31.839 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:31.826+0000 7f89e05b1740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x55e398184d40 con 0x55e397ebc670 2024-09-21T03:54:31.839 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:31.826+0000 7f89e05b1740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55e398296f40 con 0x55e397edd350 2024-09-21T03:54:31.839 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:31.826+0000 7f89e05b1740 10 client.0 _ll_register_callbacks cb 0x55e3981e8c80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:54:31.839 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:31.826+0000 7f89d2ffd640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55e397eb4490 0x55e398294390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:54:31.839 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:31.826+0000 7f89d2ffd640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55e397eb4490 0x55e398294390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:53562/0 (socket says 172.21.15.39:53562) 2024-09-21T03:54:31.839 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:31.826+0000 7f89d2ffd640 1 -- 172.21.15.39:0/4278919853 learned_addr learned my addr 172.21.15.39:0/4278919853 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:54:31.839 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:31.826+0000 7f89d37fe640 1 --2- 172.21.15.39:0/4278919853 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e397ebc670 0x55e3982948d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:54:31.839 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:31.826+0000 7f89d2ffd640 1 --2- 172.21.15.39:0/4278919853 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e397edd350 0x55e3982a0270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:54:31.839 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:31.826+0000 7f89d2ffd640 1 -- 172.21.15.39:0/4278919853 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e397edd350 msgr2=0x55e3982a0270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:54:31.839 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:31.826+0000 7f89d2ffd640 1 --2- 172.21.15.39:0/4278919853 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e397edd350 0x55e3982a0270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:31.839 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:31.826+0000 7f89d2ffd640 1 -- 172.21.15.39:0/4278919853 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e397ebc670 msgr2=0x55e3982948d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:54:31.840 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:31.826+0000 7f89d2ffd640 1 --2- 172.21.15.39:0/4278919853 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e397ebc670 0x55e3982948d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:31.840 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:31.826+0000 7f89d2ffd640 1 -- 172.21.15.39:0/4278919853 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e3982a07e0 con 0x55e397eb4490 2024-09-21T03:54:31.840 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:31.827+0000 7f89d2ffd640 1 --2- 172.21.15.39:0/4278919853 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55e397eb4490 0x55e398294390 secure :-1 s=READY pgs=1172 cs=0 l=1 rev1=1 crypto rx=0x7f89c8000c00 tx=0x7f89c802a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=17ec14efb8f53791 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:54:31.840 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:31.827+0000 7f89b2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-09-21T03:54:31.840 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:31.827+0000 7f89b2ffd640 1 -- 172.21.15.39:0/4278919853 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89c8035040 con 0x55e397eb4490 2024-09-21T03:54:31.840 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:31.827+0000 7f89b2ffd640 1 -- 172.21.15.39:0/4278919853 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8528+0+0 (secure 0 0 0) 0x7f89c8004310 con 0x55e397eb4490 2024-09-21T03:54:31.840 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:31.827+0000 7f89e05b1740 1 -- 172.21.15.39:0/4278919853 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f89c4015070 con 0x55e397eb4490 2024-09-21T03:54:31.840 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:31.827+0000 7f89b2ffd640 1 -- 172.21.15.39:0/4278919853 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f89c8052050 con 0x55e397eb4490 2024-09-21T03:54:31.840 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:31.827+0000 7f89e05b1740 10 client.17814 fetch_fsmap learned FSMap version 29 2024-09-21T03:54:31.840 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:31.827+0000 7f89e05b1740 1 -- 172.21.15.39:0/4278919853 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55e3982a3110 con 0x55e397eb4490 2024-09-21T03:54:31.840 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:31.827+0000 7f89b2ffd640 1 -- 172.21.15.39:0/4278919853 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f89c4015070 con 0x55e397eb4490 2024-09-21T03:54:31.840 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:31.827+0000 7f89b2ffd640 1 -- 172.21.15.39:0/4278919853 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f89c803a040 con 0x55e397eb4490 2024-09-21T03:54:31.840 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:31.827+0000 7f89e05b1740 10 client.17814 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T03:54:31.840 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:31.827+0000 7f89e05b1740 10 client.17814 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T03:54:31.841 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:31.827+0000 7f89e05b1740 20 client.17814 populate_metadata read hostname 'smithi039' 2024-09-21T03:54:31.841 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:31.827+0000 7f89e05b1740 1 --2- 172.21.15.39:0/4278919853 >> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] conn(0x55e397e9bcd0 0x55e3982a65e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:54:31.841 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:31.827+0000 7f89e05b1740 4 client.17814 mds_command: new command op to 24695 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:54:31.841 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:31.827+0000 7f89e05b1740 1 -- 172.21.15.39:0/4278919853 --> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e3981fcdb0 con 0x55e397e9bcd0 2024-09-21T03:54:31.841 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:31.828+0000 7f89d37fe640 1 --2- 172.21.15.39:0/4278919853 >> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] conn(0x55e397e9bcd0 0x55e3982a65e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:54:31.841 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:31.828+0000 7f89d37fe640 1 --2- 172.21.15.39:0/4278919853 >> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] conn(0x55e397e9bcd0 0x55e3982a65e0 crc :-1 s=READY pgs=143 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=846017836fd169d7 server_cookie=6ba111c608a92be1 in_seq=0 out_seq=0 2024-09-21T03:54:31.841 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:31.828+0000 7f89b2ffd640 10 client.17814 ms_handle_connect on v2:172.21.15.150:6838/2949386907 2024-09-21T03:54:31.841 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:31.829+0000 7f89b2ffd640 1 -- 172.21.15.39:0/4278919853 <== mds.0 v2:172.21.15.150:6838/2949386907 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55e3981fcdb0 con 0x55e397e9bcd0 2024-09-21T03:54:31.841 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:31.829+0000 7f89b2ffd640 10 client.17814 handle_command_reply: tid=0 2024-09-21T03:54:31.841 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:31.829+0000 7f89e05b1740 1 client.17814 shutdown 2024-09-21T03:54:31.841 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:31.829+0000 7f89e05b1740 1 -- 172.21.15.39:0/4278919853 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55e397eb4490 msgr2=0x55e398294390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:54:31.841 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:31.829+0000 7f89e05b1740 1 --2- 172.21.15.39:0/4278919853 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55e397eb4490 0x55e398294390 secure :-1 s=READY pgs=1172 cs=0 l=1 rev1=1 crypto rx=0x7f89c8000c00 tx=0x7f89c802a410 comp rx=0 tx=0).stop 2024-09-21T03:54:31.842 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:31.829+0000 7f89e05b1740 1 -- 172.21.15.39:0/4278919853 shutdown_connections 2024-09-21T03:54:31.842 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:31.829+0000 7f89e05b1740 1 --2- 172.21.15.39:0/4278919853 >> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] conn(0x55e397e9bcd0 0x55e3982a65e0 crc :-1 s=READY pgs=143 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:31.842 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:31.829+0000 7f89e05b1740 1 --2- 172.21.15.39:0/4278919853 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e397edd350 0x55e3982a0270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:31.842 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:31.829+0000 7f89e05b1740 1 --2- 172.21.15.39:0/4278919853 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e397ebc670 0x55e3982948d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:31.842 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:31.829+0000 7f89e05b1740 1 --2- 172.21.15.39:0/4278919853 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55e397eb4490 0x55e398294390 unknown :-1 s=CLOSED pgs=1172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:31.842 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:31.829+0000 7f89e05b1740 1 -- 172.21.15.39:0/4278919853 >> 172.21.15.39:0/4278919853 conn(0x55e397ed1740 msgr2=0x55e398287580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:54:31.842 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:31.829+0000 7f89b2ffd640 0 client.17814 ms_handle_reset on v2:172.21.15.150:6838/2949386907 2024-09-21T03:54:31.842 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:31.829+0000 7f89e05b1740 1 -- 172.21.15.39:0/4278919853 shutdown_connections 2024-09-21T03:54:31.842 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:31.829+0000 7f89e05b1740 1 -- 172.21.15.39:0/4278919853 wait complete. 2024-09-21T03:54:31.842 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:31.829+0000 7f89e05b1740 20 client.17814 trim_cache size 0 max 16384 2024-09-21T03:54:31.851 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:31.847+0000 7f89e05b1740 1 Processor -- start 2024-09-21T03:54:31.851 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:31.847+0000 7f89e05b1740 1 -- start start 2024-09-21T03:54:31.851 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:31.847+0000 7f89e05b1740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e397eb4490 0x55e3982a0170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:54:31.851 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:31.847+0000 7f89e05b1740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e397ebc670 0x55e398293a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:54:31.851 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:31.847+0000 7f89e05b1740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55e397edd350 0x55e398295dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:54:31.851 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:31.847+0000 7f89e05b1740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f89d41ad560 con 0x55e397edd350 2024-09-21T03:54:31.851 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:31.847+0000 7f89e05b1740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f89d41af0a0 con 0x55e397ebc670 2024-09-21T03:54:31.851 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:31.847+0000 7f89e05b1740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f89d41adc30 con 0x55e397eb4490 2024-09-21T03:54:31.851 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:31.847+0000 7f89d2ffd640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55e397edd350 0x55e398295dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:54:31.851 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:31.847+0000 7f89d2ffd640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55e397edd350 0x55e398295dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:53564/0 (socket says 172.21.15.39:53564) 2024-09-21T03:54:31.852 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:31.847+0000 7f89d2ffd640 1 -- 172.21.15.39:0/2374097619 learned_addr learned my addr 172.21.15.39:0/2374097619 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:54:31.852 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:31.848+0000 7f89d2ffd640 1 --2- 172.21.15.39:0/2374097619 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e397eb4490 0x55e3982a0170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:54:31.852 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:31.848+0000 7f89d37fe640 1 --2- 172.21.15.39:0/2374097619 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e397ebc670 0x55e398293a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:54:31.852 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:31.848+0000 7f89d2ffd640 1 -- 172.21.15.39:0/2374097619 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e397eb4490 msgr2=0x55e3982a0170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:54:31.852 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:31.848+0000 7f89d2ffd640 1 --2- 172.21.15.39:0/2374097619 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e397eb4490 0x55e3982a0170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:31.852 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:31.848+0000 7f89d2ffd640 1 -- 172.21.15.39:0/2374097619 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e397ebc670 msgr2=0x55e398293a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:54:31.852 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:31.848+0000 7f89d2ffd640 1 --2- 172.21.15.39:0/2374097619 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e397ebc670 0x55e398293a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:31.852 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:31.848+0000 7f89d2ffd640 1 -- 172.21.15.39:0/2374097619 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e39827c520 con 0x55e397edd350 2024-09-21T03:54:31.852 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:31.848+0000 7f89d2ffd640 1 --2- 172.21.15.39:0/2374097619 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55e397edd350 0x55e398295dc0 secure :-1 s=READY pgs=1173 cs=0 l=1 rev1=1 crypto rx=0x7f89c8005860 tx=0x7f89c8031400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=962b338383db9417 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:54:31.852 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:31.848+0000 7f89b97fa640 1 -- 172.21.15.39:0/2374097619 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89c8035070 con 0x55e397edd350 2024-09-21T03:54:31.852 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:31.848+0000 7f89b97fa640 1 -- 172.21.15.39:0/2374097619 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f89c803d480 con 0x55e397edd350 2024-09-21T03:54:31.852 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:31.848+0000 7f89b97fa640 1 -- 172.21.15.39:0/2374097619 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89c8003850 con 0x55e397edd350 2024-09-21T03:54:31.855 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:31.849+0000 7f89e05b1740 1 -- 172.21.15.39:0/2374097619 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55e397edd350 msgr2=0x55e398295dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:54:31.855 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:31.849+0000 7f89e05b1740 1 --2- 172.21.15.39:0/2374097619 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55e397edd350 0x55e398295dc0 secure :-1 s=READY pgs=1173 cs=0 l=1 rev1=1 crypto rx=0x7f89c8005860 tx=0x7f89c8031400 comp rx=0 tx=0).stop 2024-09-21T03:54:31.855 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:31.849+0000 7f89e05b1740 1 -- 172.21.15.39:0/2374097619 shutdown_connections 2024-09-21T03:54:31.855 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:31.849+0000 7f89e05b1740 1 --2- 172.21.15.39:0/2374097619 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55e397edd350 0x55e398295dc0 unknown :-1 s=CLOSED pgs=1173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:31.855 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:31.849+0000 7f89e05b1740 1 --2- 172.21.15.39:0/2374097619 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e397ebc670 0x55e398293a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:31.856 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:31.849+0000 7f89e05b1740 1 --2- 172.21.15.39:0/2374097619 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e397eb4490 0x55e3982a0170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:31.856 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:31.849+0000 7f89e05b1740 1 -- 172.21.15.39:0/2374097619 >> 172.21.15.39:0/2374097619 conn(0x55e397ed1740 msgr2=0x55e3982956b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:54:31.856 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:31.849+0000 7f89e05b1740 1 -- 172.21.15.39:0/2374097619 shutdown_connections 2024-09-21T03:54:31.856 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:31.849+0000 7f89e05b1740 1 -- 172.21.15.39:0/2374097619 wait complete. 2024-09-21T03:54:31.856 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:31.849+0000 7f89e05b1740 1 -- start start 2024-09-21T03:54:31.856 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:31.850+0000 7f89e05b1740 1 Processor -- start 2024-09-21T03:54:31.856 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:31.850+0000 7f89e05b1740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e397eb4490 0x55e39829cbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:54:31.856 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:31.850+0000 7f89e05b1740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e397ebc670 0x55e3982980b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:54:31.856 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:31.850+0000 7f89e05b1740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55e397edd350 0x55e3982987c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:54:31.856 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:31.850+0000 7f89e05b1740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f89d41ae9d0 con 0x55e397edd350 2024-09-21T03:54:31.856 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:31.850+0000 7f89e05b1740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f89d41ae300 con 0x55e397eb4490 2024-09-21T03:54:31.856 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:31.850+0000 7f89e05b1740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f89d41ab540 con 0x55e397ebc670 2024-09-21T03:54:31.856 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:31.850+0000 7f89d2ffd640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55e397edd350 0x55e3982987c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:54:31.856 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:31.850+0000 7f89e05b1740 10 client.0 _ll_register_callbacks cb 0x7f89c8031600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:54:31.857 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:31.850+0000 7f89d2ffd640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55e397edd350 0x55e3982987c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:53576/0 (socket says 172.21.15.39:53576) 2024-09-21T03:54:31.857 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:31.850+0000 7f89d2ffd640 1 -- 172.21.15.39:0/3754835921 learned_addr learned my addr 172.21.15.39:0/3754835921 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:54:31.857 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:31.850+0000 7f89d2ffd640 1 --2- 172.21.15.39:0/3754835921 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e397eb4490 0x55e39829cbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:54:31.857 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:31.850+0000 7f89d37fe640 1 --2- 172.21.15.39:0/3754835921 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e397ebc670 0x55e3982980b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:54:31.857 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:31.851+0000 7f89d2ffd640 1 -- 172.21.15.39:0/3754835921 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e397ebc670 msgr2=0x55e3982980b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:54:31.857 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:31.851+0000 7f89d2ffd640 1 --2- 172.21.15.39:0/3754835921 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e397ebc670 0x55e3982980b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:31.857 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:31.851+0000 7f89d2ffd640 1 -- 172.21.15.39:0/3754835921 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e397eb4490 msgr2=0x55e39829cbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:54:31.857 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:31.851+0000 7f89d2ffd640 1 --2- 172.21.15.39:0/3754835921 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e397eb4490 0x55e39829cbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:31.857 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:31.851+0000 7f89d2ffd640 1 -- 172.21.15.39:0/3754835921 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e39830ffa0 con 0x55e397edd350 2024-09-21T03:54:31.857 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:31.851+0000 7f89d2ffd640 1 --2- 172.21.15.39:0/3754835921 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55e397edd350 0x55e3982987c0 secure :-1 s=READY pgs=1174 cs=0 l=1 rev1=1 crypto rx=0x55e398282c50 tx=0x7f89c803da80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce6fcc4de1e05cca server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:54:31.857 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:31.851+0000 7f89b2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-09-21T03:54:31.857 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:31.851+0000 7f89b2ffd640 1 -- 172.21.15.39:0/3754835921 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89c8035040 con 0x55e397edd350 2024-09-21T03:54:31.857 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:31.851+0000 7f89b2ffd640 1 -- 172.21.15.39:0/3754835921 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8528+0+0 (secure 0 0 0) 0x7f89c8031aa0 con 0x55e397edd350 2024-09-21T03:54:31.857 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:31.851+0000 7f89e05b1740 1 -- 172.21.15.39:0/3754835921 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55e3981f8860 con 0x55e397edd350 2024-09-21T03:54:31.857 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:31.851+0000 7f89b2ffd640 1 -- 172.21.15.39:0/3754835921 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f89c8004470 con 0x55e397edd350 2024-09-21T03:54:31.858 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:31.851+0000 7f89e05b1740 10 client.17820 fetch_fsmap learned FSMap version 29 2024-09-21T03:54:31.858 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:31.851+0000 7f89e05b1740 1 -- 172.21.15.39:0/3754835921 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55e398311110 con 0x55e397edd350 2024-09-21T03:54:31.858 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:31.851+0000 7f89b2ffd640 1 -- 172.21.15.39:0/3754835921 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55e3981f8860 con 0x55e397edd350 2024-09-21T03:54:31.858 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:31.852+0000 7f89b2ffd640 1 -- 172.21.15.39:0/3754835921 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f89c805e020 con 0x55e397edd350 2024-09-21T03:54:31.858 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:31.852+0000 7f89e05b1740 10 client.17820 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T03:54:31.858 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:31.852+0000 7f89e05b1740 10 client.17820 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T03:54:31.858 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:31.852+0000 7f89e05b1740 20 client.17820 populate_metadata read hostname 'smithi039' 2024-09-21T03:54:31.858 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:31.852+0000 7f89e05b1740 1 --2- 172.21.15.39:0/3754835921 >> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] conn(0x55e397e71080 0x55e398313820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:54:31.858 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:31.852+0000 7f89e05b1740 4 client.17820 mds_command: new command op to 24695 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-21T03:54:31.858 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:31.852+0000 7f89e05b1740 1 -- 172.21.15.39:0/3754835921 --> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55e3981f5c90 con 0x55e397e71080 2024-09-21T03:54:31.858 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:31.852+0000 7f89d37fe640 1 --2- 172.21.15.39:0/3754835921 >> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] conn(0x55e397e71080 0x55e398313820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:54:31.858 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:31.852+0000 7f89d37fe640 1 --2- 172.21.15.39:0/3754835921 >> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] conn(0x55e397e71080 0x55e398313820 crc :-1 s=READY pgs=144 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6d41735e12f62b7c server_cookie=2b2187e8a03df23f in_seq=0 out_seq=0 2024-09-21T03:54:31.858 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:31.852+0000 7f89b2ffd640 10 client.17820 ms_handle_connect on v2:172.21.15.150:6838/2949386907 2024-09-21T03:54:31.859 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:31.854+0000 7f89b2ffd640 1 -- 172.21.15.39:0/3754835921 <== mds.0 v2:172.21.15.150:6838/2949386907 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55e3981f5c90 con 0x55e397e71080 2024-09-21T03:54:31.859 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:31.854+0000 7f89b2ffd640 10 client.17820 handle_command_reply: tid=0 2024-09-21T03:54:31.859 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:31.854+0000 7f89e05b1740 1 client.17820 shutdown 2024-09-21T03:54:31.859 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:31.855+0000 7f89e05b1740 1 -- 172.21.15.39:0/3754835921 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55e397edd350 msgr2=0x55e3982987c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:54:31.860 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:31.855+0000 7f89e05b1740 1 --2- 172.21.15.39:0/3754835921 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55e397edd350 0x55e3982987c0 secure :-1 s=READY pgs=1174 cs=0 l=1 rev1=1 crypto rx=0x55e398282c50 tx=0x7f89c803da80 comp rx=0 tx=0).stop 2024-09-21T03:54:31.860 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:31.855+0000 7f89e05b1740 1 -- 172.21.15.39:0/3754835921 shutdown_connections 2024-09-21T03:54:31.860 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:31.855+0000 7f89e05b1740 1 --2- 172.21.15.39:0/3754835921 >> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] conn(0x55e397e71080 0x55e398313820 crc :-1 s=READY pgs=144 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:31.860 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:31.855+0000 7f89e05b1740 1 --2- 172.21.15.39:0/3754835921 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55e397edd350 0x55e3982987c0 unknown :-1 s=CLOSED pgs=1174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:31.860 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:31.855+0000 7f89e05b1740 1 --2- 172.21.15.39:0/3754835921 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e397ebc670 0x55e3982980b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:31.860 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:31.855+0000 7f89e05b1740 1 --2- 172.21.15.39:0/3754835921 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e397eb4490 0x55e39829cbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:31.860 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:31.855+0000 7f89b2ffd640 0 client.17820 ms_handle_reset on v2:172.21.15.150:6838/2949386907 2024-09-21T03:54:31.860 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:31.855+0000 7f89e05b1740 1 -- 172.21.15.39:0/3754835921 >> 172.21.15.39:0/3754835921 conn(0x55e397ed1740 msgr2=0x55e398285db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:54:31.860 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:31.855+0000 7f89e05b1740 1 -- 172.21.15.39:0/3754835921 shutdown_connections 2024-09-21T03:54:31.860 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:31.855+0000 7f89e05b1740 1 -- 172.21.15.39:0/3754835921 wait complete. 2024-09-21T03:54:31.860 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:31.855+0000 7f89e05b1740 20 client.17820 trim_cache size 0 max 16384 2024-09-21T03:54:31.861 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-09-21T03:54:31.861 INFO:teuthology.orchestra.run.smithi039.stdout: "return_code": 0, 2024-09-21T03:54:31.861 INFO:teuthology.orchestra.run.smithi039.stdout: "scrub_tag": "02e16ae1-0bac-41e9-b95e-fc6b17d207ac", 2024-09-21T03:54:31.861 INFO:teuthology.orchestra.run.smithi039.stdout: "mode": "asynchronous" 2024-09-21T03:54:31.861 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-09-21T03:54:31.861 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:31.856+0000 7f89da03d640 1 -- 172.21.15.39:0/231462861 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f89bc0b66b0 msgr2=0x7f89bc0b8bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:54:31.861 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:31.856+0000 7f89da03d640 1 --2- 172.21.15.39:0/231462861 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f89bc0b66b0 0x7f89bc0b8bc0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f89cc0a9c70 tx=0x7f89cc005890 comp rx=0 tx=0).stop 2024-09-21T03:54:31.862 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:31.856+0000 7f89da03d640 1 -- 172.21.15.39:0/231462861 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f89d4108d20 msgr2=0x7f89d419bd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:54:31.862 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:31.856+0000 7f89da03d640 1 --2- 172.21.15.39:0/231462861 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f89d4108d20 0x7f89d419bd60 secure :-1 s=READY pgs=1171 cs=0 l=1 rev1=1 crypto rx=0x7f89cc00eaa0 tx=0x7f89cc00ef60 comp rx=0 tx=0).stop 2024-09-21T03:54:31.862 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:31.856+0000 7f89da03d640 1 -- 172.21.15.39:0/231462861 shutdown_connections 2024-09-21T03:54:31.862 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:31.856+0000 7f89da03d640 1 --2- 172.21.15.39:0/231462861 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f89bc0b66b0 0x7f89bc0b8bc0 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:31.862 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:31.856+0000 7f89da03d640 1 --2- 172.21.15.39:0/231462861 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f89d4108d20 0x7f89d419bd60 unknown :-1 s=CLOSED pgs=1171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:31.862 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:31.856+0000 7f89da03d640 1 --2- 172.21.15.39:0/231462861 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f89d41083d0 0x7f89d419b820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:31.862 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:31.856+0000 7f89da03d640 1 --2- 172.21.15.39:0/231462861 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f89d4102ae0 0x7f89d419b2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:31.862 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:31.856+0000 7f89da03d640 1 -- 172.21.15.39:0/231462861 >> 172.21.15.39:0/231462861 conn(0x7f89d4079760 msgr2=0x7f89d41044a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:54:31.862 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:31.856+0000 7f89da03d640 1 -- 172.21.15.39:0/231462861 shutdown_connections 2024-09-21T03:54:31.862 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:31.856+0000 7f89da03d640 1 -- 172.21.15.39:0/231462861 wait complete. 2024-09-21T03:54:32.774 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 51116d5c-77ca-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub status 2024-09-21T03:54:33.041 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/51116d5c-77ca-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:54:33.083 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:54:32 smithi039 ceph-mon[31211]: pgmap v584: 97 pgs: 97 active+clean; 10 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 9 op/s 2024-09-21T03:54:33.084 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:54:32 smithi039 ceph-mon[31211]: scrub queued for path: / 2024-09-21T03:54:33.084 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:54:32 smithi039 ceph-mon[31211]: scrub summary: idle+waiting paths [/] 2024-09-21T03:54:33.084 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:54:32 smithi039 ceph-mon[31211]: scrub summary: active paths [/] 2024-09-21T03:54:33.084 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:54:32 smithi039 ceph-mon[31211]: scrub summary: idle+waiting paths [/] 2024-09-21T03:54:33.218 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:54:32 smithi161 ceph-mon[34533]: pgmap v584: 97 pgs: 97 active+clean; 10 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 9 op/s 2024-09-21T03:54:33.218 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:54:32 smithi161 ceph-mon[34533]: scrub queued for path: / 2024-09-21T03:54:33.218 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:54:32 smithi161 ceph-mon[34533]: scrub summary: idle+waiting paths [/] 2024-09-21T03:54:33.218 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:54:32 smithi161 ceph-mon[34533]: scrub summary: active paths [/] 2024-09-21T03:54:33.218 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:54:32 smithi161 ceph-mon[34533]: scrub summary: idle+waiting paths [/] 2024-09-21T03:54:33.225 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:54:32 smithi150 ceph-mon[34289]: pgmap v584: 97 pgs: 97 active+clean; 10 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 9 op/s 2024-09-21T03:54:33.225 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:54:32 smithi150 ceph-mon[34289]: scrub queued for path: / 2024-09-21T03:54:33.225 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:54:32 smithi150 ceph-mon[34289]: scrub summary: idle+waiting paths [/] 2024-09-21T03:54:33.225 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:54:32 smithi150 ceph-mon[34289]: scrub summary: active paths [/] 2024-09-21T03:54:33.225 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:54:32 smithi150 ceph-mon[34289]: scrub summary: idle+waiting paths [/] 2024-09-21T03:54:33.542 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:33.539+0000 7f690b1a6640 1 -- 172.21.15.39:0/1837027406 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6904105290 msgr2=0x7f690410d8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:54:33.542 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:33.539+0000 7f690b1a6640 1 --2- 172.21.15.39:0/1837027406 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6904105290 0x7f690410d8d0 secure :-1 s=READY pgs=1175 cs=0 l=1 rev1=1 crypto rx=0x7f68f8009900 tx=0x7f68f802a410 comp rx=0 tx=0).stop 2024-09-21T03:54:33.542 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:33.539+0000 7f690b1a6640 1 -- 172.21.15.39:0/1837027406 shutdown_connections 2024-09-21T03:54:33.542 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:33.539+0000 7f690b1a6640 1 --2- 172.21.15.39:0/1837027406 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f690410de10 0x7f69041102f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:33.542 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:33.539+0000 7f690b1a6640 1 --2- 172.21.15.39:0/1837027406 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6904105290 0x7f690410d8d0 unknown :-1 s=CLOSED pgs=1175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:33.542 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:33.539+0000 7f690b1a6640 1 --2- 172.21.15.39:0/1837027406 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6904104860 0x7f6904104c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:33.543 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:33.539+0000 7f690b1a6640 1 -- 172.21.15.39:0/1837027406 >> 172.21.15.39:0/1837027406 conn(0x7f6904079730 msgr2=0x7f6904079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:54:33.543 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:33.539+0000 7f690b1a6640 1 -- 172.21.15.39:0/1837027406 shutdown_connections 2024-09-21T03:54:33.543 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:33.539+0000 7f690b1a6640 1 -- 172.21.15.39:0/1837027406 wait complete. 2024-09-21T03:54:33.544 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:33.540+0000 7f690b1a6640 1 Processor -- start 2024-09-21T03:54:33.544 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:33.540+0000 7f690b1a6640 1 -- start start 2024-09-21T03:54:33.544 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:33.540+0000 7f690b1a6640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6904104860 0x7f690419a4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:54:33.545 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:33.540+0000 7f690b1a6640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6904105290 0x7f690419aa70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:54:33.545 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:33.540+0000 7f690b1a6640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f690410de10 0x7f690419fd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:54:33.545 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:33.540+0000 7f690b1a6640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f6904113a60 con 0x7f6904105290 2024-09-21T03:54:33.545 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:33.540+0000 7f690b1a6640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f69041138e0 con 0x7f6904104860 2024-09-21T03:54:33.545 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:33.540+0000 7f690b1a6640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f6904113be0 con 0x7f690410de10 2024-09-21T03:54:33.545 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:33.540+0000 7f6903fff640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6904105290 0x7f690419aa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:54:33.545 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:33.540+0000 7f6903fff640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6904105290 0x7f690419aa70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:53596/0 (socket says 172.21.15.39:53596) 2024-09-21T03:54:33.545 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:33.540+0000 7f6903fff640 1 -- 172.21.15.39:0/18243498 learned_addr learned my addr 172.21.15.39:0/18243498 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:54:33.545 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:33.540+0000 7f6908f42640 1 --2- 172.21.15.39:0/18243498 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6904104860 0x7f690419a4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:54:33.545 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:33.541+0000 7f6909743640 1 --2- 172.21.15.39:0/18243498 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f690410de10 0x7f690419fd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:54:33.545 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:33.541+0000 7f6903fff640 1 -- 172.21.15.39:0/18243498 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f690410de10 msgr2=0x7f690419fd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:54:33.545 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:33.541+0000 7f6903fff640 1 --2- 172.21.15.39:0/18243498 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f690410de10 0x7f690419fd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:33.545 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:33.541+0000 7f6903fff640 1 -- 172.21.15.39:0/18243498 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6904104860 msgr2=0x7f690419a4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:54:33.545 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:33.541+0000 7f6903fff640 1 --2- 172.21.15.39:0/18243498 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6904104860 0x7f690419a4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:33.546 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:33.541+0000 7f6903fff640 1 -- 172.21.15.39:0/18243498 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f69041a0440 con 0x7f6904105290 2024-09-21T03:54:33.546 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:33.541+0000 7f6908f42640 1 --2- 172.21.15.39:0/18243498 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6904104860 0x7f690419a4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:54:33.546 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:33.541+0000 7f6903fff640 1 --2- 172.21.15.39:0/18243498 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6904105290 0x7f690419aa70 secure :-1 s=READY pgs=1176 cs=0 l=1 rev1=1 crypto rx=0x7f68f802a930 tx=0x7f68f8004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:54:33.546 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:33.541+0000 7f6901ffb640 1 -- 172.21.15.39:0/18243498 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68f803f030 con 0x7f6904105290 2024-09-21T03:54:33.546 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:33.541+0000 7f6901ffb640 1 -- 172.21.15.39:0/18243498 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f68f8003aa0 con 0x7f6904105290 2024-09-21T03:54:33.546 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:33.541+0000 7f690b1a6640 1 -- 172.21.15.39:0/18243498 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f69041a06d0 con 0x7f6904105290 2024-09-21T03:54:33.546 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:33.541+0000 7f6901ffb640 1 -- 172.21.15.39:0/18243498 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68f8035070 con 0x7f6904105290 2024-09-21T03:54:33.546 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:33.541+0000 7f690b1a6640 1 -- 172.21.15.39:0/18243498 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f69041a6e90 con 0x7f6904105290 2024-09-21T03:54:33.547 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:33.544+0000 7f691171a740 1 Processor -- start 2024-09-21T03:54:33.547 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:33.544+0000 7f691171a740 1 -- start start 2024-09-21T03:54:33.547 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:33.544+0000 7f6901ffb640 1 -- 172.21.15.39:0/18243498 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 27) ==== 154643+0+0 (secure 0 0 0) 0x7f68f8042030 con 0x7f6904105290 2024-09-21T03:54:33.548 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:33.544+0000 7f691171a740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558f3bb5b070 0x558f3bf02650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:54:33.548 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:33.544+0000 7f691171a740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558f3bb321b0 0x558f3bf02b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:54:33.548 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:33.544+0000 7f691171a740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558f3bb3a390 0x558f3bf030d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:54:33.549 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:33.544+0000 7f691171a740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x558f3be78e50 con 0x558f3bb3a390 2024-09-21T03:54:33.549 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:33.544+0000 7f691171a740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x558f3be68280 con 0x558f3bb5b070 2024-09-21T03:54:33.549 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:33.544+0000 7f691171a740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x558f3be6c090 con 0x558f3bb321b0 2024-09-21T03:54:33.549 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:33.545+0000 7f6903fff640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558f3bb3a390 0x558f3bf030d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:54:33.549 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:33.545+0000 7f6903fff640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558f3bb3a390 0x558f3bf030d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:53598/0 (socket says 172.21.15.39:53598) 2024-09-21T03:54:33.549 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:33.545+0000 7f6903fff640 1 -- 172.21.15.39:0/947830903 learned_addr learned my addr 172.21.15.39:0/947830903 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:54:33.549 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:33.545+0000 7f6909743640 1 --2- 172.21.15.39:0/947830903 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558f3bb5b070 0x558f3bf02650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:54:33.549 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:33.545+0000 7f6908f42640 1 --2- 172.21.15.39:0/947830903 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558f3bb321b0 0x558f3bf02b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:54:33.549 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:33.545+0000 7f6903fff640 1 -- 172.21.15.39:0/947830903 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558f3bb321b0 msgr2=0x558f3bf02b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:54:33.549 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:33.545+0000 7f6903fff640 1 --2- 172.21.15.39:0/947830903 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558f3bb321b0 0x558f3bf02b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:33.549 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:33.545+0000 7f6903fff640 1 -- 172.21.15.39:0/947830903 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558f3bb5b070 msgr2=0x558f3bf02650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:54:33.549 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:33.545+0000 7f6903fff640 1 --2- 172.21.15.39:0/947830903 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558f3bb5b070 0x558f3bf02650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:33.549 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:33.545+0000 7f6903fff640 1 -- 172.21.15.39:0/947830903 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558f3bbc6650 con 0x558f3bb3a390 2024-09-21T03:54:33.549 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:33.545+0000 7f6901ffb640 1 --2- 172.21.15.39:0/18243498 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f68ec0b6680 0x7f68ec0b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:54:33.550 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:33.545+0000 7f6909743640 1 --2- 172.21.15.39:0/18243498 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f68ec0b6680 0x7f68ec0b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:54:33.550 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:33.545+0000 7f6903fff640 1 --2- 172.21.15.39:0/947830903 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558f3bb3a390 0x558f3bf030d0 secure :-1 s=READY pgs=1177 cs=0 l=1 rev1=1 crypto rx=0x7f68f8105490 tx=0x7f68f81100e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c258dcf526c495a9 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:54:33.550 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:33.546+0000 7f6901ffb640 1 -- 172.21.15.39:0/18243498 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8528+0+0 (secure 0 0 0) 0x7f68f8107e00 con 0x7f6904105290 2024-09-21T03:54:33.550 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:33.546+0000 7f68eaffd640 1 -- 172.21.15.39:0/947830903 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68f80ca2a0 con 0x558f3bb3a390 2024-09-21T03:54:33.550 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:33.546+0000 7f68eaffd640 1 -- 172.21.15.39:0/947830903 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f68f80b50c0 con 0x558f3bb3a390 2024-09-21T03:54:33.550 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:33.546+0000 7f68eaffd640 1 -- 172.21.15.39:0/947830903 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68f810b9e0 con 0x558f3bb3a390 2024-09-21T03:54:33.552 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:33.547+0000 7f691171a740 1 -- 172.21.15.39:0/947830903 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558f3bb3a390 msgr2=0x558f3bf030d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:54:33.553 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:33.547+0000 7f6909743640 1 --2- 172.21.15.39:0/18243498 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f68ec0b6680 0x7f68ec0b8b90 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f68fc00ae90 tx=0x7f68fc00a4d0 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:54:33.553 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:33.547+0000 7f691171a740 1 --2- 172.21.15.39:0/947830903 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558f3bb3a390 0x558f3bf030d0 secure :-1 s=READY pgs=1177 cs=0 l=1 rev1=1 crypto rx=0x7f68f8105490 tx=0x7f68f81100e0 comp rx=0 tx=0).stop 2024-09-21T03:54:33.553 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:33.547+0000 7f691171a740 1 -- 172.21.15.39:0/947830903 shutdown_connections 2024-09-21T03:54:33.553 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:33.547+0000 7f691171a740 1 --2- 172.21.15.39:0/947830903 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558f3bb3a390 0x558f3bf030d0 unknown :-1 s=CLOSED pgs=1177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:33.553 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:33.547+0000 7f691171a740 1 --2- 172.21.15.39:0/947830903 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558f3bb321b0 0x558f3bf02b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:33.553 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:33.547+0000 7f691171a740 1 --2- 172.21.15.39:0/947830903 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558f3bb5b070 0x558f3bf02650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:33.553 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:33.547+0000 7f691171a740 1 -- 172.21.15.39:0/947830903 >> 172.21.15.39:0/947830903 conn(0x558f3bb4f460 msgr2=0x558f3bf00540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:54:33.553 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:33.547+0000 7f691171a740 1 -- 172.21.15.39:0/947830903 shutdown_connections 2024-09-21T03:54:33.553 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:33.547+0000 7f691171a740 1 -- 172.21.15.39:0/947830903 wait complete. 2024-09-21T03:54:33.553 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:33.547+0000 7f691171a740 1 -- start start 2024-09-21T03:54:33.553 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:33.547+0000 7f691171a740 1 Processor -- start 2024-09-21T03:54:33.554 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:33.548+0000 7f691171a740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558f3bb321b0 0x558f3bf120d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:54:33.554 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:33.548+0000 7f691171a740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558f3bb3a390 0x558f3bf12610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:54:33.554 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:33.548+0000 7f691171a740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558f3bb5b070 0x558f3bf12b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:54:33.554 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:33.548+0000 7f691171a740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x558f3bcc9700 con 0x558f3bb5b070 2024-09-21T03:54:33.554 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:33.548+0000 7f691171a740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x558f3be03110 con 0x558f3bb3a390 2024-09-21T03:54:33.554 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:33.548+0000 7f691171a740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x558f3bf14f50 con 0x558f3bb321b0 2024-09-21T03:54:33.554 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:33.548+0000 7f691171a740 10 client.0 _ll_register_callbacks cb 0x558f3be1c990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:54:33.554 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:33.548+0000 7f6909743640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558f3bb5b070 0x558f3bf12b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:54:33.554 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:33.548+0000 7f6909743640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558f3bb5b070 0x558f3bf12b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:53608/0 (socket says 172.21.15.39:53608) 2024-09-21T03:54:33.554 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:33.548+0000 7f6909743640 1 -- 172.21.15.39:0/3394974771 learned_addr learned my addr 172.21.15.39:0/3394974771 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:54:33.554 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:33.548+0000 7f6903fff640 1 --2- 172.21.15.39:0/3394974771 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558f3bb3a390 0x558f3bf12610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:54:33.554 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:33.548+0000 7f6908f42640 1 --2- 172.21.15.39:0/3394974771 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558f3bb321b0 0x558f3bf120d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:54:33.554 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:33.548+0000 7f6909743640 1 -- 172.21.15.39:0/3394974771 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558f3bb321b0 msgr2=0x558f3bf120d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:54:33.554 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:33.548+0000 7f6909743640 1 --2- 172.21.15.39:0/3394974771 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558f3bb321b0 0x558f3bf120d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:33.555 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:33.548+0000 7f6909743640 1 -- 172.21.15.39:0/3394974771 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558f3bb3a390 msgr2=0x558f3bf12610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:54:33.555 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:33.548+0000 7f6909743640 1 --2- 172.21.15.39:0/3394974771 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558f3bb3a390 0x558f3bf12610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:33.555 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:33.548+0000 7f6909743640 1 -- 172.21.15.39:0/3394974771 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558f3be30c40 con 0x558f3bb5b070 2024-09-21T03:54:33.555 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:33.549+0000 7f6909743640 1 --2- 172.21.15.39:0/3394974771 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558f3bb5b070 0x558f3bf12b90 secure :-1 s=READY pgs=1178 cs=0 l=1 rev1=1 crypto rx=0x7f68fc01ec20 tx=0x7f68fc004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4484fcf7dbe512ec server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:54:33.555 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:33.549+0000 7f68e3fff640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-09-21T03:54:33.555 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:33.549+0000 7f68e3fff640 1 -- 172.21.15.39:0/3394974771 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68fc02b060 con 0x558f3bb5b070 2024-09-21T03:54:33.555 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:33.549+0000 7f68e3fff640 1 -- 172.21.15.39:0/3394974771 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8528+0+0 (secure 0 0 0) 0x7f68fc016030 con 0x558f3bb5b070 2024-09-21T03:54:33.555 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:33.549+0000 7f691171a740 1 -- 172.21.15.39:0/3394974771 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f68f80b50c0 con 0x558f3bb5b070 2024-09-21T03:54:33.555 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:33.549+0000 7f68e3fff640 1 -- 172.21.15.39:0/3394974771 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f68fc01ec50 con 0x558f3bb5b070 2024-09-21T03:54:33.555 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:33.549+0000 7f691171a740 10 client.17832 fetch_fsmap learned FSMap version 29 2024-09-21T03:54:33.555 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:33.549+0000 7f691171a740 1 -- 172.21.15.39:0/3394974771 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x558f3bf20bc0 con 0x558f3bb5b070 2024-09-21T03:54:33.555 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:33.549+0000 7f68e3fff640 1 -- 172.21.15.39:0/3394974771 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f68f80b50c0 con 0x558f3bb5b070 2024-09-21T03:54:33.555 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:33.549+0000 7f68e3fff640 1 -- 172.21.15.39:0/3394974771 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f68fc02e3b0 con 0x558f3bb5b070 2024-09-21T03:54:33.555 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:33.549+0000 7f691171a740 10 client.17832 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T03:54:33.556 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:33.549+0000 7f691171a740 10 client.17832 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T03:54:33.556 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:33.549+0000 7f691171a740 20 client.17832 populate_metadata read hostname 'smithi039' 2024-09-21T03:54:33.556 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:33.549+0000 7f691171a740 1 --2- 172.21.15.39:0/3394974771 >> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] conn(0x558f3bb199f0 0x558f3bf24160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:54:33.556 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:33.549+0000 7f691171a740 4 client.17832 mds_command: new command op to 24695 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:54:33.556 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:33.549+0000 7f691171a740 1 -- 172.21.15.39:0/3394974771 --> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558f3be7ada0 con 0x558f3bb199f0 2024-09-21T03:54:33.556 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:33.550+0000 7f6908f42640 1 --2- 172.21.15.39:0/3394974771 >> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] conn(0x558f3bb199f0 0x558f3bf24160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:54:33.557 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:33.551+0000 7f68e3fff640 10 client.17832 ms_handle_connect on v2:172.21.15.150:6838/2949386907 2024-09-21T03:54:33.557 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:33.551+0000 7f6908f42640 1 --2- 172.21.15.39:0/3394974771 >> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] conn(0x558f3bb199f0 0x558f3bf24160 crc :-1 s=READY pgs=145 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a56b9ea5e35098b server_cookie=ef3ea10434835eb2 in_seq=0 out_seq=0 2024-09-21T03:54:33.557 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:33.551+0000 7f68e3fff640 1 -- 172.21.15.39:0/3394974771 <== mds.0 v2:172.21.15.150:6838/2949386907 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x558f3be7ada0 con 0x558f3bb199f0 2024-09-21T03:54:33.557 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:33.551+0000 7f68e3fff640 10 client.17832 handle_command_reply: tid=0 2024-09-21T03:54:33.557 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:33.552+0000 7f691171a740 1 client.17832 shutdown 2024-09-21T03:54:33.557 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:33.552+0000 7f691171a740 1 -- 172.21.15.39:0/3394974771 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558f3bb5b070 msgr2=0x558f3bf12b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:54:33.557 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:33.552+0000 7f691171a740 1 --2- 172.21.15.39:0/3394974771 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558f3bb5b070 0x558f3bf12b90 secure :-1 s=READY pgs=1178 cs=0 l=1 rev1=1 crypto rx=0x7f68fc01ec20 tx=0x7f68fc004ab0 comp rx=0 tx=0).stop 2024-09-21T03:54:33.558 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:33.552+0000 7f691171a740 1 -- 172.21.15.39:0/3394974771 shutdown_connections 2024-09-21T03:54:33.558 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:33.552+0000 7f691171a740 1 --2- 172.21.15.39:0/3394974771 >> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] conn(0x558f3bb199f0 0x558f3bf24160 crc :-1 s=READY pgs=145 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:33.558 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:33.552+0000 7f691171a740 1 --2- 172.21.15.39:0/3394974771 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558f3bb5b070 0x558f3bf12b90 unknown :-1 s=CLOSED pgs=1178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:33.558 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:33.552+0000 7f691171a740 1 --2- 172.21.15.39:0/3394974771 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558f3bb3a390 0x558f3bf12610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:33.558 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:33.552+0000 7f691171a740 1 --2- 172.21.15.39:0/3394974771 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558f3bb321b0 0x558f3bf120d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:33.558 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:33.552+0000 7f691171a740 1 -- 172.21.15.39:0/3394974771 >> 172.21.15.39:0/3394974771 conn(0x558f3bb4f460 msgr2=0x558f3befeb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:54:33.558 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:33.552+0000 7f68e3fff640 0 client.17832 ms_handle_reset on v2:172.21.15.150:6838/2949386907 2024-09-21T03:54:33.558 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:33.552+0000 7f691171a740 1 -- 172.21.15.39:0/3394974771 shutdown_connections 2024-09-21T03:54:33.558 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:33.552+0000 7f691171a740 1 -- 172.21.15.39:0/3394974771 wait complete. 2024-09-21T03:54:33.558 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:33.552+0000 7f691171a740 20 client.17832 trim_cache size 0 max 16384 2024-09-21T03:54:33.573 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:33.570+0000 7f691171a740 1 Processor -- start 2024-09-21T03:54:33.574 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:33.570+0000 7f691171a740 1 -- start start 2024-09-21T03:54:33.574 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:33.570+0000 7f691171a740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558f3bb321b0 0x558f3beffe60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:54:33.574 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:33.570+0000 7f691171a740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558f3bb3a390 0x558f3bf003a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:54:33.574 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:33.570+0000 7f691171a740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558f3bb5b070 0x558f3bf0f2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:54:33.574 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:33.570+0000 7f691171a740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f69041ad5e0 con 0x558f3bb3a390 2024-09-21T03:54:33.574 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:33.570+0000 7f691171a740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f69041af120 con 0x558f3bb5b070 2024-09-21T03:54:33.574 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:33.570+0000 7f691171a740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f69041adcb0 con 0x558f3bb321b0 2024-09-21T03:54:33.574 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:33.570+0000 7f6903fff640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558f3bb3a390 0x558f3bf003a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:54:33.574 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:33.570+0000 7f6903fff640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558f3bb3a390 0x558f3bf003a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:53622/0 (socket says 172.21.15.39:53622) 2024-09-21T03:54:33.574 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:33.570+0000 7f6903fff640 1 -- 172.21.15.39:0/1375724348 learned_addr learned my addr 172.21.15.39:0/1375724348 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:54:33.574 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:33.570+0000 7f6908f42640 1 --2- 172.21.15.39:0/1375724348 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558f3bb321b0 0x558f3beffe60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:54:33.574 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:33.571+0000 7f6909743640 1 --2- 172.21.15.39:0/1375724348 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558f3bb5b070 0x558f3bf0f2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:54:33.574 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:33.571+0000 7f6903fff640 1 -- 172.21.15.39:0/1375724348 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558f3bb321b0 msgr2=0x558f3beffe60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:54:33.575 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:33.571+0000 7f6903fff640 1 --2- 172.21.15.39:0/1375724348 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558f3bb321b0 0x558f3beffe60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:33.575 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:33.571+0000 7f6903fff640 1 -- 172.21.15.39:0/1375724348 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558f3bb5b070 msgr2=0x558f3bf0f2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:54:33.575 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:33.571+0000 7f6903fff640 1 --2- 172.21.15.39:0/1375724348 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558f3bb5b070 0x558f3bf0f2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:33.575 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:33.571+0000 7f6903fff640 1 -- 172.21.15.39:0/1375724348 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558f3bf08410 con 0x558f3bb3a390 2024-09-21T03:54:33.575 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:33.571+0000 7f6909743640 1 --2- 172.21.15.39:0/1375724348 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558f3bb5b070 0x558f3bf0f2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:54:33.575 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:33.571+0000 7f6903fff640 1 --2- 172.21.15.39:0/1375724348 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558f3bb3a390 0x558f3bf003a0 secure :-1 s=READY pgs=1179 cs=0 l=1 rev1=1 crypto rx=0x7f68f8110210 tx=0x7f68f8000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af26ac576b6158a5 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:54:33.575 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:33.571+0000 7f68ea7fc640 1 -- 172.21.15.39:0/1375724348 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68f80d2070 con 0x558f3bb3a390 2024-09-21T03:54:33.575 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:33.571+0000 7f68ea7fc640 1 -- 172.21.15.39:0/1375724348 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f68f80b50c0 con 0x558f3bb3a390 2024-09-21T03:54:33.575 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:33.571+0000 7f68ea7fc640 1 -- 172.21.15.39:0/1375724348 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68f8049040 con 0x558f3bb3a390 2024-09-21T03:54:33.577 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:33.572+0000 7f691171a740 1 -- 172.21.15.39:0/1375724348 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558f3bb3a390 msgr2=0x558f3bf003a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:54:33.578 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:33.572+0000 7f691171a740 1 --2- 172.21.15.39:0/1375724348 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558f3bb3a390 0x558f3bf003a0 secure :-1 s=READY pgs=1179 cs=0 l=1 rev1=1 crypto rx=0x7f68f8110210 tx=0x7f68f8000bc0 comp rx=0 tx=0).stop 2024-09-21T03:54:33.578 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:33.572+0000 7f691171a740 1 -- 172.21.15.39:0/1375724348 shutdown_connections 2024-09-21T03:54:33.578 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:33.572+0000 7f691171a740 1 --2- 172.21.15.39:0/1375724348 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558f3bb5b070 0x558f3bf0f2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:33.578 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:33.572+0000 7f691171a740 1 --2- 172.21.15.39:0/1375724348 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558f3bb3a390 0x558f3bf003a0 unknown :-1 s=CLOSED pgs=1179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:33.578 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:33.572+0000 7f691171a740 1 --2- 172.21.15.39:0/1375724348 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558f3bb321b0 0x558f3beffe60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:33.578 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:33.572+0000 7f691171a740 1 -- 172.21.15.39:0/1375724348 >> 172.21.15.39:0/1375724348 conn(0x558f3bb4f460 msgr2=0x558f3bf10950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:54:33.578 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:33.572+0000 7f691171a740 1 -- 172.21.15.39:0/1375724348 shutdown_connections 2024-09-21T03:54:33.578 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:33.572+0000 7f691171a740 1 -- 172.21.15.39:0/1375724348 wait complete. 2024-09-21T03:54:33.578 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:33.572+0000 7f691171a740 1 -- start start 2024-09-21T03:54:33.578 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:33.572+0000 7f691171a740 1 Processor -- start 2024-09-21T03:54:33.578 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:33.573+0000 7f691171a740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558f3bb321b0 0x558f3bf1aa50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:54:33.578 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:33.573+0000 7f691171a740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558f3bb3a390 0x558f3bf15f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:54:33.579 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:33.573+0000 7f691171a740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558f3bb5b070 0x558f3bf165f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:54:33.579 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:33.573+0000 7f691171a740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f69041aea50 con 0x558f3bb5b070 2024-09-21T03:54:33.579 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:33.573+0000 7f691171a740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f69041ae380 con 0x558f3bb3a390 2024-09-21T03:54:33.579 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:33.573+0000 7f691171a740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f69041ab5c0 con 0x558f3bb321b0 2024-09-21T03:54:33.579 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:33.573+0000 7f691171a740 10 client.0 _ll_register_callbacks cb 0x7f68fc00a8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:54:33.579 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:33.573+0000 7f6908f42640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558f3bb321b0 0x558f3bf1aa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:54:33.579 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:33.573+0000 7f6909743640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558f3bb5b070 0x558f3bf165f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:54:33.579 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:33.573+0000 7f6908f42640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558f3bb321b0 0x558f3bf1aa50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.39:52874/0 (socket says 172.21.15.39:52874) 2024-09-21T03:54:33.579 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:33.573+0000 7f6908f42640 1 -- 172.21.15.39:0/1526606132 learned_addr learned my addr 172.21.15.39:0/1526606132 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:54:33.579 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:33.573+0000 7f6903fff640 1 --2- 172.21.15.39:0/1526606132 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558f3bb3a390 0x558f3bf15f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:54:33.579 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:33.573+0000 7f6909743640 1 -- 172.21.15.39:0/1526606132 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558f3bb321b0 msgr2=0x558f3bf1aa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:54:33.579 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:33.573+0000 7f6909743640 1 --2- 172.21.15.39:0/1526606132 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558f3bb321b0 0x558f3bf1aa50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:33.579 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:33.573+0000 7f6909743640 1 -- 172.21.15.39:0/1526606132 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558f3bb3a390 msgr2=0x558f3bf15f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:54:33.579 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:33.573+0000 7f6909743640 1 --2- 172.21.15.39:0/1526606132 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558f3bb3a390 0x558f3bf15f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:33.579 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:33.573+0000 7f6909743640 1 -- 172.21.15.39:0/1526606132 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558f3bf16d80 con 0x558f3bb5b070 2024-09-21T03:54:33.580 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:33.573+0000 7f6908f42640 1 --2- 172.21.15.39:0/1526606132 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558f3bb321b0 0x558f3bf1aa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:54:33.580 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:33.574+0000 7f6909743640 1 --2- 172.21.15.39:0/1526606132 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558f3bb5b070 0x558f3bf165f0 secure :-1 s=READY pgs=1180 cs=0 l=1 rev1=1 crypto rx=0x7f68fc00a990 tx=0x7f68fc00fbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=23713293ab9ac0ca server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:54:33.580 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:33.574+0000 7f68e3fff640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-09-21T03:54:33.580 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:33.574+0000 7f68e3fff640 1 -- 172.21.15.39:0/1526606132 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68fc02e5b0 con 0x558f3bb5b070 2024-09-21T03:54:33.580 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:33.574+0000 7f68e3fff640 1 -- 172.21.15.39:0/1526606132 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8528+0+0 (secure 0 0 0) 0x7f68fc026410 con 0x558f3bb5b070 2024-09-21T03:54:33.580 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:33.574+0000 7f691171a740 1 -- 172.21.15.39:0/1526606132 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f68f80b50c0 con 0x558f3bb5b070 2024-09-21T03:54:33.580 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:33.574+0000 7f68e3fff640 1 -- 172.21.15.39:0/1526606132 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f68fc02b030 con 0x558f3bb5b070 2024-09-21T03:54:33.580 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:33.574+0000 7f691171a740 10 client.17838 fetch_fsmap learned FSMap version 29 2024-09-21T03:54:33.580 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:33.574+0000 7f691171a740 1 -- 172.21.15.39:0/1526606132 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x558f3bf8eba0 con 0x558f3bb5b070 2024-09-21T03:54:33.580 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:33.574+0000 7f68e3fff640 1 -- 172.21.15.39:0/1526606132 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f68f80b50c0 con 0x558f3bb5b070 2024-09-21T03:54:33.580 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:33.574+0000 7f68e3fff640 1 -- 172.21.15.39:0/1526606132 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f68fc0386a0 con 0x558f3bb5b070 2024-09-21T03:54:33.580 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:33.575+0000 7f691171a740 10 client.17838 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T03:54:33.580 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:33.575+0000 7f691171a740 10 client.17838 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T03:54:33.580 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:33.575+0000 7f691171a740 20 client.17838 populate_metadata read hostname 'smithi039' 2024-09-21T03:54:33.580 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:33.575+0000 7f691171a740 1 --2- 172.21.15.39:0/1526606132 >> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] conn(0x558f3baeeda0 0x558f3bf913b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:54:33.581 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:33.575+0000 7f691171a740 4 client.17838 mds_command: new command op to 24695 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-21T03:54:33.581 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:33.575+0000 7f691171a740 1 -- 172.21.15.39:0/1526606132 --> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] -- command(tid 0: {"prefix": "scrub status"}) -- 0x558f3be76850 con 0x558f3baeeda0 2024-09-21T03:54:33.581 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:33.575+0000 7f6908f42640 1 --2- 172.21.15.39:0/1526606132 >> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] conn(0x558f3baeeda0 0x558f3bf913b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:54:33.582 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:33.575+0000 7f6908f42640 1 --2- 172.21.15.39:0/1526606132 >> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] conn(0x558f3baeeda0 0x558f3bf913b0 crc :-1 s=READY pgs=146 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ab918fc294a6d2e6 server_cookie=9b3343d41889c3ab in_seq=0 out_seq=0 2024-09-21T03:54:33.582 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:33.575+0000 7f68e3fff640 10 client.17838 ms_handle_connect on v2:172.21.15.150:6838/2949386907 2024-09-21T03:54:33.582 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:33.576+0000 7f68e3fff640 1 -- 172.21.15.39:0/1526606132 <== mds.0 v2:172.21.15.150:6838/2949386907 1 ==== command_reply(tid 0: 0 ) ==== 8+0+5114 (crc 0 0 0) 0x558f3be76850 con 0x558f3baeeda0 2024-09-21T03:54:33.582 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:33.576+0000 7f68e3fff640 10 client.17838 handle_command_reply: tid=0 2024-09-21T03:54:33.582 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:33.576+0000 7f691171a740 1 client.17838 shutdown 2024-09-21T03:54:33.582 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:33.576+0000 7f691171a740 1 -- 172.21.15.39:0/1526606132 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558f3bb5b070 msgr2=0x558f3bf165f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:54:33.582 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:33.576+0000 7f691171a740 1 --2- 172.21.15.39:0/1526606132 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558f3bb5b070 0x558f3bf165f0 secure :-1 s=READY pgs=1180 cs=0 l=1 rev1=1 crypto rx=0x7f68fc00a990 tx=0x7f68fc00fbc0 comp rx=0 tx=0).stop 2024-09-21T03:54:33.582 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:33.576+0000 7f691171a740 1 -- 172.21.15.39:0/1526606132 shutdown_connections 2024-09-21T03:54:33.582 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:33.576+0000 7f691171a740 1 --2- 172.21.15.39:0/1526606132 >> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] conn(0x558f3baeeda0 0x558f3bf913b0 crc :-1 s=READY pgs=146 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:33.582 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:33.576+0000 7f68e3fff640 0 client.17838 ms_handle_reset on v2:172.21.15.150:6838/2949386907 2024-09-21T03:54:33.582 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:33.576+0000 7f691171a740 1 --2- 172.21.15.39:0/1526606132 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558f3bb5b070 0x558f3bf165f0 unknown :-1 s=CLOSED pgs=1180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:33.582 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:33.576+0000 7f691171a740 1 --2- 172.21.15.39:0/1526606132 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558f3bb3a390 0x558f3bf15f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:33.583 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:33.576+0000 7f691171a740 1 --2- 172.21.15.39:0/1526606132 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558f3bb321b0 0x558f3bf1aa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:33.583 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:33.576+0000 7f691171a740 1 -- 172.21.15.39:0/1526606132 >> 172.21.15.39:0/1526606132 conn(0x558f3bb4f460 msgr2=0x558f3bf13580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:54:33.583 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:33.576+0000 7f691171a740 1 -- 172.21.15.39:0/1526606132 shutdown_connections 2024-09-21T03:54:33.583 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:33.576+0000 7f691171a740 1 -- 172.21.15.39:0/1526606132 wait complete. 2024-09-21T03:54:33.583 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:33.576+0000 7f691171a740 20 client.17838 trim_cache size 0 max 16384 2024-09-21T03:54:33.583 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-09-21T03:54:33.583 INFO:teuthology.orchestra.run.smithi039.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-21T03:54:33.583 INFO:teuthology.orchestra.run.smithi039.stdout: "scrubs": { 2024-09-21T03:54:33.583 INFO:teuthology.orchestra.run.smithi039.stdout: "02e16ae1-0bac-41e9-b95e-fc6b17d207ac": { 2024-09-21T03:54:33.583 INFO:teuthology.orchestra.run.smithi039.stdout: "path": "/", 2024-09-21T03:54:33.584 INFO:teuthology.orchestra.run.smithi039.stdout: "tag": "02e16ae1-0bac-41e9-b95e-fc6b17d207ac", 2024-09-21T03:54:33.584 INFO:teuthology.orchestra.run.smithi039.stdout: "options": "recursive,force" 2024-09-21T03:54:33.584 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-21T03:54:33.584 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:54:33.584 INFO:teuthology.orchestra.run.smithi039.stdout: "scrub_stats": { 2024-09-21T03:54:33.584 INFO:teuthology.orchestra.run.smithi039.stdout: "02e16ae1-0bac-41e9-b95e-fc6b17d207ac": { 2024-09-21T03:54:33.584 INFO:teuthology.orchestra.run.smithi039.stdout: "start_time": "2024-09-21T03:54:31.853401+0000", 2024-09-21T03:54:33.584 INFO:teuthology.orchestra.run.smithi039.stdout: "path": "/", 2024-09-21T03:54:33.584 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_started": 0, 2024-09-21T03:54:33.584 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_passed": 0, 2024-09-21T03:54:33.584 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_failed": 0, 2024-09-21T03:54:33.584 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_skipped": 0 2024-09-21T03:54:33.584 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:54:33.584 INFO:teuthology.orchestra.run.smithi039.stdout: "e28dcd13-f2ac-4cb7-a346-58917d542b15": { 2024-09-21T03:54:33.584 INFO:teuthology.orchestra.run.smithi039.stdout: "start_time": "2024-09-21T03:54:18.221680+0000", 2024-09-21T03:54:33.584 INFO:teuthology.orchestra.run.smithi039.stdout: "path": "/", 2024-09-21T03:54:33.585 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_started": 0, 2024-09-21T03:54:33.585 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_passed": 0, 2024-09-21T03:54:33.585 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_failed": 0, 2024-09-21T03:54:33.585 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_skipped": 0 2024-09-21T03:54:33.585 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:54:33.585 INFO:teuthology.orchestra.run.smithi039.stdout: "ad87da79-0248-4e18-9694-8228cb95422f": { 2024-09-21T03:54:33.585 INFO:teuthology.orchestra.run.smithi039.stdout: "start_time": "2024-09-21T03:53:18.441907+0000", 2024-09-21T03:54:33.585 INFO:teuthology.orchestra.run.smithi039.stdout: "path": "/", 2024-09-21T03:54:33.585 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_started": 0, 2024-09-21T03:54:33.585 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_passed": 0, 2024-09-21T03:54:33.585 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_failed": 0, 2024-09-21T03:54:33.585 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_skipped": 0 2024-09-21T03:54:33.585 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:54:33.585 INFO:teuthology.orchestra.run.smithi039.stdout: "02509e4f-3911-4983-b16f-df8ded0b9998": { 2024-09-21T03:54:33.585 INFO:teuthology.orchestra.run.smithi039.stdout: "start_time": "2024-09-21T03:46:17.889576+0000", 2024-09-21T03:54:33.586 INFO:teuthology.orchestra.run.smithi039.stdout: "path": "/", 2024-09-21T03:54:33.586 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_started": 0, 2024-09-21T03:54:33.586 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_passed": 0, 2024-09-21T03:54:33.586 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_failed": 0, 2024-09-21T03:54:33.586 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_skipped": 0 2024-09-21T03:54:33.586 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:54:33.586 INFO:teuthology.orchestra.run.smithi039.stdout: "10728de8-c49d-4edc-be2b-639249255e94": { 2024-09-21T03:54:33.586 INFO:teuthology.orchestra.run.smithi039.stdout: "start_time": "2024-09-21T03:47:03.965623+0000", 2024-09-21T03:54:33.586 INFO:teuthology.orchestra.run.smithi039.stdout: "path": "/", 2024-09-21T03:54:33.586 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_started": 0, 2024-09-21T03:54:33.586 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_passed": 0, 2024-09-21T03:54:33.586 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_failed": 0, 2024-09-21T03:54:33.586 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_skipped": 0 2024-09-21T03:54:33.586 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:54:33.587 INFO:teuthology.orchestra.run.smithi039.stdout: "9860c2f4-b7b1-47aa-865a-9a1c4434a5f6": { 2024-09-21T03:54:33.587 INFO:teuthology.orchestra.run.smithi039.stdout: "start_time": "2024-09-21T03:49:17.718867+0000", 2024-09-21T03:54:33.587 INFO:teuthology.orchestra.run.smithi039.stdout: "path": "/", 2024-09-21T03:54:33.587 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_started": 0, 2024-09-21T03:54:33.587 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_passed": 0, 2024-09-21T03:54:33.587 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_failed": 0, 2024-09-21T03:54:33.587 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_skipped": 0 2024-09-21T03:54:33.587 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:54:33.587 INFO:teuthology.orchestra.run.smithi039.stdout: "d9391d69-baac-453a-9d52-d046760c070d": { 2024-09-21T03:54:33.587 INFO:teuthology.orchestra.run.smithi039.stdout: "start_time": "2024-09-21T03:53:32.364622+0000", 2024-09-21T03:54:33.587 INFO:teuthology.orchestra.run.smithi039.stdout: "path": "/", 2024-09-21T03:54:33.587 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_started": 0, 2024-09-21T03:54:33.587 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_passed": 0, 2024-09-21T03:54:33.587 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_failed": 0, 2024-09-21T03:54:33.587 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_skipped": 0 2024-09-21T03:54:33.587 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:54:33.588 INFO:teuthology.orchestra.run.smithi039.stdout: "c7c700f3-2cd8-41ac-8381-b57dd1b37b02": { 2024-09-21T03:54:33.588 INFO:teuthology.orchestra.run.smithi039.stdout: "start_time": "2024-09-21T03:52:32.368858+0000", 2024-09-21T03:54:33.588 INFO:teuthology.orchestra.run.smithi039.stdout: "path": "/", 2024-09-21T03:54:33.588 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_started": 0, 2024-09-21T03:54:33.588 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_passed": 0, 2024-09-21T03:54:33.588 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_failed": 0, 2024-09-21T03:54:33.588 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_skipped": 0 2024-09-21T03:54:33.588 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:54:33.588 INFO:teuthology.orchestra.run.smithi039.stdout: "29411307-eb8d-4339-8d2b-47ebd32c0080": { 2024-09-21T03:54:33.588 INFO:teuthology.orchestra.run.smithi039.stdout: "start_time": "2024-09-21T03:47:38.092920+0000", 2024-09-21T03:54:33.588 INFO:teuthology.orchestra.run.smithi039.stdout: "path": "/", 2024-09-21T03:54:33.588 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_started": 0, 2024-09-21T03:54:33.588 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_passed": 0, 2024-09-21T03:54:33.588 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_failed": 0, 2024-09-21T03:54:33.588 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_skipped": 0 2024-09-21T03:54:33.589 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:54:33.589 INFO:teuthology.orchestra.run.smithi039.stdout: "ede73e6e-3fd0-4a7f-b5bb-f68946f57eea": { 2024-09-21T03:54:33.589 INFO:teuthology.orchestra.run.smithi039.stdout: "start_time": "2024-09-21T03:51:32.235096+0000", 2024-09-21T03:54:33.589 INFO:teuthology.orchestra.run.smithi039.stdout: "path": "/", 2024-09-21T03:54:33.589 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_started": 0, 2024-09-21T03:54:33.589 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_passed": 0, 2024-09-21T03:54:33.589 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_failed": 0, 2024-09-21T03:54:33.589 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_skipped": 0 2024-09-21T03:54:33.589 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:54:33.589 INFO:teuthology.orchestra.run.smithi039.stdout: "e28ad743-2b22-45b3-8799-a638c2883623": { 2024-09-21T03:54:33.589 INFO:teuthology.orchestra.run.smithi039.stdout: "start_time": "2024-09-21T03:48:31.448821+0000", 2024-09-21T03:54:33.589 INFO:teuthology.orchestra.run.smithi039.stdout: "path": "/", 2024-09-21T03:54:33.589 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_started": 0, 2024-09-21T03:54:33.589 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_passed": 0, 2024-09-21T03:54:33.589 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_failed": 0, 2024-09-21T03:54:33.590 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_skipped": 0 2024-09-21T03:54:33.590 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:54:33.590 INFO:teuthology.orchestra.run.smithi039.stdout: "6e760f1f-9048-4f92-bd66-d708fbef75b3": { 2024-09-21T03:54:33.590 INFO:teuthology.orchestra.run.smithi039.stdout: "start_time": "2024-09-21T03:50:03.711918+0000", 2024-09-21T03:54:33.590 INFO:teuthology.orchestra.run.smithi039.stdout: "path": "/", 2024-09-21T03:54:33.590 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_started": 0, 2024-09-21T03:54:33.590 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_passed": 0, 2024-09-21T03:54:33.590 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_failed": 0, 2024-09-21T03:54:33.590 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_skipped": 0 2024-09-21T03:54:33.590 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:54:33.590 INFO:teuthology.orchestra.run.smithi039.stdout: "c7e2b8e4-057d-435f-b2bd-b633de98035f": { 2024-09-21T03:54:33.590 INFO:teuthology.orchestra.run.smithi039.stdout: "start_time": "2024-09-21T03:47:19.400305+0000", 2024-09-21T03:54:33.591 INFO:teuthology.orchestra.run.smithi039.stdout: "path": "/", 2024-09-21T03:54:33.591 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_started": 0, 2024-09-21T03:54:33.591 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_passed": 0, 2024-09-21T03:54:33.591 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_failed": 0, 2024-09-21T03:54:33.591 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_skipped": 0 2024-09-21T03:54:33.591 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:54:33.591 INFO:teuthology.orchestra.run.smithi039.stdout: "74ab917e-ecea-4778-a1c3-c184214fb24c": { 2024-09-21T03:54:33.591 INFO:teuthology.orchestra.run.smithi039.stdout: "start_time": "2024-09-21T03:50:17.726630+0000", 2024-09-21T03:54:33.591 INFO:teuthology.orchestra.run.smithi039.stdout: "path": "/", 2024-09-21T03:54:33.591 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_started": 0, 2024-09-21T03:54:33.591 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_passed": 0, 2024-09-21T03:54:33.591 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_failed": 0, 2024-09-21T03:54:33.592 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_skipped": 0 2024-09-21T03:54:33.592 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:54:33.592 INFO:teuthology.orchestra.run.smithi039.stdout: "160750d8-401f-4810-866d-e63e7e4bc3e0": { 2024-09-21T03:54:33.592 INFO:teuthology.orchestra.run.smithi039.stdout: "start_time": "2024-09-21T03:51:04.208673+0000", 2024-09-21T03:54:33.592 INFO:teuthology.orchestra.run.smithi039.stdout: "path": "/", 2024-09-21T03:54:33.592 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_started": 0, 2024-09-21T03:54:33.592 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_passed": 0, 2024-09-21T03:54:33.592 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_failed": 0, 2024-09-21T03:54:33.592 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_skipped": 0 2024-09-21T03:54:33.592 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:54:33.592 INFO:teuthology.orchestra.run.smithi039.stdout: "ae1fc4cf-5458-47d0-afcb-282b136dba15": { 2024-09-21T03:54:33.592 INFO:teuthology.orchestra.run.smithi039.stdout: "start_time": "2024-09-21T03:51:18.560378+0000", 2024-09-21T03:54:33.592 INFO:teuthology.orchestra.run.smithi039.stdout: "path": "/", 2024-09-21T03:54:33.592 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_started": 0, 2024-09-21T03:54:33.593 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_passed": 0, 2024-09-21T03:54:33.593 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_failed": 0, 2024-09-21T03:54:33.593 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_skipped": 0 2024-09-21T03:54:33.593 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:54:33.593 INFO:teuthology.orchestra.run.smithi039.stdout: "4bda6ee3-7237-4ee4-96ce-150ad50f26dc": { 2024-09-21T03:54:33.593 INFO:teuthology.orchestra.run.smithi039.stdout: "start_time": "2024-09-21T03:52:18.622141+0000", 2024-09-21T03:54:33.593 INFO:teuthology.orchestra.run.smithi039.stdout: "path": "/", 2024-09-21T03:54:33.593 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_started": 0, 2024-09-21T03:54:33.593 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_passed": 0, 2024-09-21T03:54:33.593 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_failed": 0, 2024-09-21T03:54:33.593 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_skipped": 0 2024-09-21T03:54:33.593 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-21T03:54:33.593 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-21T03:54:33.593 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-09-21T03:54:33.594 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:33.577+0000 7f690b1a6640 1 -- 172.21.15.39:0/18243498 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f68ec0b6680 msgr2=0x7f68ec0b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:54:33.594 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:33.577+0000 7f690b1a6640 1 --2- 172.21.15.39:0/18243498 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f68ec0b6680 0x7f68ec0b8b90 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f68fc00ae90 tx=0x7f68fc00a4d0 comp rx=0 tx=0).stop 2024-09-21T03:54:33.594 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:33.577+0000 7f690b1a6640 1 -- 172.21.15.39:0/18243498 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6904105290 msgr2=0x7f690419aa70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:54:33.594 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:33.577+0000 7f690b1a6640 1 --2- 172.21.15.39:0/18243498 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6904105290 0x7f690419aa70 secure :-1 s=READY pgs=1176 cs=0 l=1 rev1=1 crypto rx=0x7f68f802a930 tx=0x7f68f8004270 comp rx=0 tx=0).stop 2024-09-21T03:54:33.594 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:33.577+0000 7f690b1a6640 1 -- 172.21.15.39:0/18243498 shutdown_connections 2024-09-21T03:54:33.594 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:33.577+0000 7f690b1a6640 1 --2- 172.21.15.39:0/18243498 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f68ec0b6680 0x7f68ec0b8b90 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:33.595 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:33.577+0000 7f690b1a6640 1 --2- 172.21.15.39:0/18243498 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f690410de10 0x7f690419fd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:33.595 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:33.577+0000 7f690b1a6640 1 --2- 172.21.15.39:0/18243498 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6904105290 0x7f690419aa70 unknown :-1 s=CLOSED pgs=1176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:33.595 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:33.577+0000 7f690b1a6640 1 --2- 172.21.15.39:0/18243498 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6904104860 0x7f690419a4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:33.595 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:33.577+0000 7f690b1a6640 1 -- 172.21.15.39:0/18243498 >> 172.21.15.39:0/18243498 conn(0x7f6904079730 msgr2=0x7f69040ff7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:54:33.595 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:33.577+0000 7f690b1a6640 1 -- 172.21.15.39:0/18243498 shutdown_connections 2024-09-21T03:54:33.595 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:54:33.577+0000 7f690b1a6640 1 -- 172.21.15.39:0/18243498 wait complete. 2024-09-21T03:54:34.316 INFO:tasks.cephfs.filesystem:scrub status for tag:02e16ae1-0bac-41e9-b95e-fc6b17d207ac - {'path': '/', 'tag': '02e16ae1-0bac-41e9-b95e-fc6b17d207ac', 'options': 'recursive,force'} 2024-09-21T03:54:35.218 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:54:34 smithi161 ceph-mon[34533]: pgmap v585: 97 pgs: 97 active+clean; 10 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 10 op/s 2024-09-21T03:54:35.224 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:54:34 smithi150 ceph-mon[34289]: pgmap v585: 97 pgs: 97 active+clean; 10 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 10 op/s 2024-09-21T03:54:35.334 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:54:34 smithi039 ceph-mon[31211]: pgmap v585: 97 pgs: 97 active+clean; 10 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 10 op/s 2024-09-21T03:54:36.218 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:54:35 smithi161 ceph-mon[34533]: scrub summary: idle 2024-09-21T03:54:36.224 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:54:35 smithi150 ceph-mon[34289]: scrub summary: idle 2024-09-21T03:54:36.334 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:54:35 smithi039 ceph-mon[31211]: scrub summary: idle 2024-09-21T03:54:37.218 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:54:36 smithi161 ceph-mon[34533]: pgmap v586: 97 pgs: 97 active+clean; 10 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 10 op/s 2024-09-21T03:54:37.225 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:54:36 smithi150 ceph-mon[34289]: pgmap v586: 97 pgs: 97 active+clean; 10 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 10 op/s 2024-09-21T03:54:37.334 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:54:36 smithi039 ceph-mon[31211]: pgmap v586: 97 pgs: 97 active+clean; 10 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 10 op/s 2024-09-21T03:54:39.218 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:54:38 smithi161 ceph-mon[34533]: pgmap v587: 97 pgs: 97 active+clean; 10 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 8 op/s 2024-09-21T03:54:39.224 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:54:38 smithi150 ceph-mon[34289]: pgmap v587: 97 pgs: 97 active+clean; 10 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 8 op/s 2024-09-21T03:54:39.334 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:54:38 smithi039 ceph-mon[31211]: pgmap v587: 97 pgs: 97 active+clean; 10 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 8 op/s 2024-09-21T03:54:41.218 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:54:40 smithi161 ceph-mon[34533]: pgmap v588: 97 pgs: 97 active+clean; 10 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 11 op/s 2024-09-21T03:54:41.225 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:54:40 smithi150 ceph-mon[34289]: pgmap v588: 97 pgs: 97 active+clean; 10 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 11 op/s 2024-09-21T03:54:41.334 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:54:40 smithi039 ceph-mon[31211]: pgmap v588: 97 pgs: 97 active+clean; 10 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 11 op/s 2024-09-21T03:54:43.218 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:54:42 smithi161 ceph-mon[34533]: pgmap v589: 97 pgs: 97 active+clean; 10 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 9 op/s 2024-09-21T03:54:43.225 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:54:42 smithi150 ceph-mon[34289]: pgmap v589: 97 pgs: 97 active+clean; 10 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 9 op/s 2024-09-21T03:54:43.334 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:54:42 smithi039 ceph-mon[31211]: pgmap v589: 97 pgs: 97 active+clean; 10 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 9 op/s 2024-09-21T03:54:45.218 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:54:44 smithi161 ceph-mon[34533]: pgmap v590: 97 pgs: 97 active+clean; 10 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 10 op/s 2024-09-21T03:54:45.224 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:54:44 smithi150 ceph-mon[34289]: pgmap v590: 97 pgs: 97 active+clean; 10 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 10 op/s 2024-09-21T03:54:45.334 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:54:44 smithi039 ceph-mon[31211]: pgmap v590: 97 pgs: 97 active+clean; 10 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 10 op/s 2024-09-21T03:54:47.218 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:54:46 smithi161 ceph-mon[34533]: pgmap v591: 97 pgs: 97 active+clean; 10 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 10 op/s 2024-09-21T03:54:47.224 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:54:46 smithi150 ceph-mon[34289]: pgmap v591: 97 pgs: 97 active+clean; 10 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 10 op/s 2024-09-21T03:54:47.334 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:54:46 smithi039 ceph-mon[31211]: pgmap v591: 97 pgs: 97 active+clean; 10 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 10 op/s 2024-09-21T03:54:49.218 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:54:48 smithi161 ceph-mon[34533]: pgmap v592: 97 pgs: 97 active+clean; 10 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-09-21T03:54:49.224 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:54:48 smithi150 ceph-mon[34289]: pgmap v592: 97 pgs: 97 active+clean; 10 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-09-21T03:54:49.334 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:54:48 smithi039 ceph-mon[31211]: pgmap v592: 97 pgs: 97 active+clean; 10 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-09-21T03:54:51.218 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:54:50 smithi161 ceph-mon[34533]: pgmap v593: 97 pgs: 97 active+clean; 10 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-09-21T03:54:51.225 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:54:50 smithi150 ceph-mon[34289]: pgmap v593: 97 pgs: 97 active+clean; 10 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-09-21T03:54:51.334 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:54:50 smithi039 ceph-mon[31211]: pgmap v593: 97 pgs: 97 active+clean; 10 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-09-21T03:54:53.218 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:54:52 smithi161 ceph-mon[34533]: pgmap v594: 97 pgs: 97 active+clean; 10 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-21T03:54:53.225 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:54:52 smithi150 ceph-mon[34289]: pgmap v594: 97 pgs: 97 active+clean; 10 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-21T03:54:53.334 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:54:52 smithi039 ceph-mon[31211]: pgmap v594: 97 pgs: 97 active+clean; 10 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-21T03:54:55.218 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:54:54 smithi161 ceph-mon[34533]: pgmap v595: 97 pgs: 97 active+clean; 10 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-09-21T03:54:55.225 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:54:54 smithi150 ceph-mon[34289]: pgmap v595: 97 pgs: 97 active+clean; 10 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-09-21T03:54:55.334 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:54:54 smithi039 ceph-mon[31211]: pgmap v595: 97 pgs: 97 active+clean; 10 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-09-21T03:54:57.218 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:54:56 smithi161 ceph-mon[34533]: pgmap v596: 97 pgs: 97 active+clean; 10 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 10 op/s 2024-09-21T03:54:57.224 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:54:56 smithi150 ceph-mon[34289]: pgmap v596: 97 pgs: 97 active+clean; 10 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 10 op/s 2024-09-21T03:54:57.334 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:54:56 smithi039 ceph-mon[31211]: pgmap v596: 97 pgs: 97 active+clean; 10 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 10 op/s 2024-09-21T03:54:59.225 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:54:58 smithi150 ceph-mon[34289]: pgmap v597: 97 pgs: 97 active+clean; 10 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 8 op/s 2024-09-21T03:54:59.334 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:54:58 smithi039 ceph-mon[31211]: pgmap v597: 97 pgs: 97 active+clean; 10 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 8 op/s 2024-09-21T03:54:59.468 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:54:58 smithi161 ceph-mon[34533]: pgmap v597: 97 pgs: 97 active+clean; 10 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 8 op/s 2024-09-21T03:55:01.225 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:55:00 smithi150 ceph-mon[34289]: pgmap v598: 97 pgs: 97 active+clean; 10 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 11 op/s 2024-09-21T03:55:01.334 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:55:00 smithi039 ceph-mon[31211]: pgmap v598: 97 pgs: 97 active+clean; 10 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 11 op/s 2024-09-21T03:55:01.468 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:55:00 smithi161 ceph-mon[34533]: pgmap v598: 97 pgs: 97 active+clean; 10 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 11 op/s 2024-09-21T03:55:03.334 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:55:02 smithi039 ceph-mon[31211]: pgmap v599: 97 pgs: 97 active+clean; 10 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 9 op/s 2024-09-21T03:55:03.471 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:55:02 smithi161 ceph-mon[34533]: pgmap v599: 97 pgs: 97 active+clean; 10 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 9 op/s 2024-09-21T03:55:03.475 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:55:02 smithi150 ceph-mon[34289]: pgmap v599: 97 pgs: 97 active+clean; 10 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 9 op/s 2024-09-21T03:55:04.317 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 51116d5c-77ca-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub status 2024-09-21T03:55:04.589 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/51116d5c-77ca-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:55:05.186 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:05.182+0000 7fdde44e7640 1 -- 172.21.15.39:0/4019580782 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdddc1083c0 msgr2=0x7fdddc1087d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:55:05.186 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:05.182+0000 7fdde44e7640 1 --2- 172.21.15.39:0/4019580782 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdddc1083c0 0x7fdddc1087d0 secure :-1 s=READY pgs=1181 cs=0 l=1 rev1=1 crypto rx=0x7fddcc009900 tx=0x7fddcc02a410 comp rx=0 tx=0).stop 2024-09-21T03:55:05.186 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:05.182+0000 7fdde44e7640 1 -- 172.21.15.39:0/4019580782 shutdown_connections 2024-09-21T03:55:05.186 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:05.182+0000 7fdde44e7640 1 --2- 172.21.15.39:0/4019580782 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdddc108d10 0x7fdddc10f540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:05.186 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:05.182+0000 7fdde44e7640 1 --2- 172.21.15.39:0/4019580782 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdddc1083c0 0x7fdddc1087d0 unknown :-1 s=CLOSED pgs=1181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:05.187 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:05.182+0000 7fdde44e7640 1 --2- 172.21.15.39:0/4019580782 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdddc102a80 0x7fdddc102e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:05.187 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:05.182+0000 7fdde44e7640 1 -- 172.21.15.39:0/4019580782 >> 172.21.15.39:0/4019580782 conn(0x7fdddc079740 msgr2=0x7fdddc079b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:55:05.187 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:05.182+0000 7fdde44e7640 1 -- 172.21.15.39:0/4019580782 shutdown_connections 2024-09-21T03:55:05.187 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:05.182+0000 7fdde44e7640 1 -- 172.21.15.39:0/4019580782 wait complete. 2024-09-21T03:55:05.187 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:05.183+0000 7fdde44e7640 1 Processor -- start 2024-09-21T03:55:05.187 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:05.183+0000 7fdde44e7640 1 -- start start 2024-09-21T03:55:05.187 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:05.183+0000 7fdde44e7640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdddc102a80 0x7fdddc199410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:55:05.187 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:05.183+0000 7fdde44e7640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdddc1083c0 0x7fdddc1999b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:55:05.187 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:05.183+0000 7fdde44e7640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdddc108d10 0x7fdddc19ec50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:55:05.187 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:05.183+0000 7fdde44e7640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fdddc112980 con 0x7fdddc1083c0 2024-09-21T03:55:05.187 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:05.183+0000 7fdde44e7640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fdddc112800 con 0x7fdddc102a80 2024-09-21T03:55:05.187 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:05.183+0000 7fdde44e7640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fdddc112b00 con 0x7fdddc108d10 2024-09-21T03:55:05.187 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:05.183+0000 7fdde1a82640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdddc1083c0 0x7fdddc1999b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:55:05.187 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:05.183+0000 7fdde1a82640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdddc1083c0 0x7fdddc1999b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:53734/0 (socket says 172.21.15.39:53734) 2024-09-21T03:55:05.188 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:05.183+0000 7fdde1a82640 1 -- 172.21.15.39:0/3447264566 learned_addr learned my addr 172.21.15.39:0/3447264566 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:55:05.188 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:05.183+0000 7fdde2283640 1 --2- 172.21.15.39:0/3447264566 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdddc102a80 0x7fdddc199410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:55:05.188 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:05.183+0000 7fdde2a84640 1 --2- 172.21.15.39:0/3447264566 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdddc108d10 0x7fdddc19ec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:55:05.188 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:05.184+0000 7fdde1a82640 1 -- 172.21.15.39:0/3447264566 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdddc108d10 msgr2=0x7fdddc19ec50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:55:05.188 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:05.184+0000 7fdde1a82640 1 --2- 172.21.15.39:0/3447264566 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdddc108d10 0x7fdddc19ec50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:05.188 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:05.184+0000 7fdde1a82640 1 -- 172.21.15.39:0/3447264566 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdddc102a80 msgr2=0x7fdddc199410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:55:05.188 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:05.184+0000 7fdde1a82640 1 --2- 172.21.15.39:0/3447264566 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdddc102a80 0x7fdddc199410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:05.188 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:05.184+0000 7fdde1a82640 1 -- 172.21.15.39:0/3447264566 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdddc19f380 con 0x7fdddc1083c0 2024-09-21T03:55:05.188 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:05.184+0000 7fdde1a82640 1 --2- 172.21.15.39:0/3447264566 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdddc1083c0 0x7fdddc1999b0 secure :-1 s=READY pgs=1182 cs=0 l=1 rev1=1 crypto rx=0x7fddcc032fd0 tx=0x7fddcc004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:55:05.188 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:05.184+0000 7fddd37fe640 1 -- 172.21.15.39:0/3447264566 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddcc03f030 con 0x7fdddc1083c0 2024-09-21T03:55:05.188 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:05.184+0000 7fddd37fe640 1 -- 172.21.15.39:0/3447264566 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fddcc004490 con 0x7fdddc1083c0 2024-09-21T03:55:05.188 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:05.184+0000 7fdde44e7640 1 -- 172.21.15.39:0/3447264566 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdddc19f610 con 0x7fdddc1083c0 2024-09-21T03:55:05.188 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:05.184+0000 7fddd37fe640 1 -- 172.21.15.39:0/3447264566 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddcc035070 con 0x7fdddc1083c0 2024-09-21T03:55:05.188 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:05.184+0000 7fdde44e7640 1 -- 172.21.15.39:0/3447264566 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdddc06bb40 con 0x7fdddc1083c0 2024-09-21T03:55:05.190 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:05.187+0000 7fddeaa5b740 1 Processor -- start 2024-09-21T03:55:05.190 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:05.187+0000 7fddeaa5b740 1 -- start start 2024-09-21T03:55:05.190 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:05.187+0000 7fddd37fe640 1 -- 172.21.15.39:0/3447264566 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 27) ==== 154643+0+0 (secure 0 0 0) 0x7fddcc042030 con 0x7fdddc1083c0 2024-09-21T03:55:05.190 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:05.187+0000 7fddeaa5b740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55806f907070 0x55806fcae270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:55:05.191 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:05.187+0000 7fddeaa5b740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55806f8de1b0 0x55806fcae7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:55:05.191 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:05.187+0000 7fddeaa5b740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55806f8e6390 0x55806fcaecf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:55:05.191 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:05.187+0000 7fddeaa5b740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x55806fc24e50 con 0x55806f8e6390 2024-09-21T03:55:05.191 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:05.187+0000 7fddeaa5b740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x55806fc14280 con 0x55806f8de1b0 2024-09-21T03:55:05.191 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:05.187+0000 7fddeaa5b740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55806fc18090 con 0x55806f907070 2024-09-21T03:55:05.191 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:05.187+0000 7fdde1a82640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55806f8e6390 0x55806fcaecf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:55:05.191 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:05.188+0000 7fdde2a84640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55806f907070 0x55806fcae270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:55:05.191 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:05.188+0000 7fdde2a84640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55806f907070 0x55806fcae270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.39:57914/0 (socket says 172.21.15.39:57914) 2024-09-21T03:55:05.191 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:05.188+0000 7fdde2a84640 1 -- 172.21.15.39:0/2095421950 learned_addr learned my addr 172.21.15.39:0/2095421950 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:55:05.191 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:05.188+0000 7fdde2283640 1 --2- 172.21.15.39:0/2095421950 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55806f8de1b0 0x55806fcae7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:55:05.191 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:05.188+0000 7fdde2283640 1 -- 172.21.15.39:0/2095421950 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55806f907070 msgr2=0x55806fcae270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:55:05.191 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:05.188+0000 7fdde2283640 1 --2- 172.21.15.39:0/2095421950 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55806f907070 0x55806fcae270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:05.191 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:05.188+0000 7fdde2283640 1 -- 172.21.15.39:0/2095421950 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55806f8e6390 msgr2=0x55806fcaecf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:55:05.191 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:05.188+0000 7fdde2283640 1 --2- 172.21.15.39:0/2095421950 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55806f8e6390 0x55806fcaecf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:05.192 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:05.188+0000 7fdde2283640 1 -- 172.21.15.39:0/2095421950 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55806f972650 con 0x55806f8de1b0 2024-09-21T03:55:05.192 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:05.188+0000 7fdde1a82640 1 --2- 172.21.15.39:0/2095421950 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55806f8e6390 0x55806fcaecf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:55:05.192 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:05.188+0000 7fddd37fe640 1 --2- 172.21.15.39:0/3447264566 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7fddc80b6680 0x7fddc80b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:55:05.192 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:05.188+0000 7fdde2a84640 1 --2- 172.21.15.39:0/3447264566 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7fddc80b6680 0x7fddc80b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:55:05.192 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:05.188+0000 7fdde2283640 1 --2- 172.21.15.39:0/2095421950 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55806f8de1b0 0x55806fcae7b0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fddd8004630 tx=0x7fddd800bb20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=63e0278a9308fc3a server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:55:05.192 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:05.188+0000 7fddd37fe640 1 -- 172.21.15.39:0/3447264566 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8528+0+0 (secure 0 0 0) 0x7fddcc107c30 con 0x7fdddc1083c0 2024-09-21T03:55:05.192 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:05.189+0000 7fddd0ff9640 1 -- 172.21.15.39:0/2095421950 <== mon.2 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddd8003bd0 con 0x55806f8de1b0 2024-09-21T03:55:05.192 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:05.189+0000 7fddd0ff9640 1 -- 172.21.15.39:0/2095421950 <== mon.2 v2:172.21.15.150:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fddd8015070 con 0x55806f8de1b0 2024-09-21T03:55:05.192 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:05.189+0000 7fddd0ff9640 1 -- 172.21.15.39:0/2095421950 <== mon.2 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddd8004cd0 con 0x55806f8de1b0 2024-09-21T03:55:05.195 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:05.189+0000 7fddeaa5b740 1 -- 172.21.15.39:0/2095421950 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55806f8de1b0 msgr2=0x55806fcae7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:55:05.195 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:05.189+0000 7fddeaa5b740 1 --2- 172.21.15.39:0/2095421950 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55806f8de1b0 0x55806fcae7b0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fddd8004630 tx=0x7fddd800bb20 comp rx=0 tx=0).stop 2024-09-21T03:55:05.195 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:05.189+0000 7fdde2a84640 1 --2- 172.21.15.39:0/3447264566 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7fddc80b6680 0x7fddc80b8b90 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7fddd4005c40 tx=0x7fddd4009760 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:55:05.195 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:05.189+0000 7fddeaa5b740 1 -- 172.21.15.39:0/2095421950 shutdown_connections 2024-09-21T03:55:05.195 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:05.189+0000 7fddeaa5b740 1 --2- 172.21.15.39:0/2095421950 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55806f8e6390 0x55806fcaecf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:05.195 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:05.189+0000 7fddeaa5b740 1 --2- 172.21.15.39:0/2095421950 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55806f8de1b0 0x55806fcae7b0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:05.195 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:05.189+0000 7fddeaa5b740 1 --2- 172.21.15.39:0/2095421950 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55806f907070 0x55806fcae270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:05.195 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:05.190+0000 7fddeaa5b740 1 -- 172.21.15.39:0/2095421950 >> 172.21.15.39:0/2095421950 conn(0x55806f8fb460 msgr2=0x55806fcac160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:55:05.195 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:05.190+0000 7fddeaa5b740 1 -- 172.21.15.39:0/2095421950 shutdown_connections 2024-09-21T03:55:05.195 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:05.190+0000 7fddeaa5b740 1 -- 172.21.15.39:0/2095421950 wait complete. 2024-09-21T03:55:05.196 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:05.190+0000 7fddeaa5b740 1 -- start start 2024-09-21T03:55:05.196 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:05.190+0000 7fddeaa5b740 1 Processor -- start 2024-09-21T03:55:05.196 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:05.190+0000 7fddeaa5b740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55806f8de1b0 0x55806fcbdd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:55:05.196 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:05.190+0000 7fddeaa5b740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55806f8e6390 0x55806fcbe2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:55:05.196 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:05.190+0000 7fddeaa5b740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55806f907070 0x55806fcca080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:55:05.196 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:05.190+0000 7fddeaa5b740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x55806fa75700 con 0x55806f8de1b0 2024-09-21T03:55:05.196 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:05.190+0000 7fddeaa5b740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x55806fbaf110 con 0x55806f907070 2024-09-21T03:55:05.196 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:05.190+0000 7fddeaa5b740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55806fcc0d50 con 0x55806f8e6390 2024-09-21T03:55:05.196 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:05.191+0000 7fddeaa5b740 10 client.0 _ll_register_callbacks cb 0x55806fbc8990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:55:05.196 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:05.191+0000 7fdde2283640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55806f8de1b0 0x55806fcbdd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:55:05.196 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:05.191+0000 7fdde2283640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55806f8de1b0 0x55806fcbdd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:53752/0 (socket says 172.21.15.39:53752) 2024-09-21T03:55:05.196 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:05.191+0000 7fdde2283640 1 -- 172.21.15.39:0/2331781330 learned_addr learned my addr 172.21.15.39:0/2331781330 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:55:05.196 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:05.191+0000 7fdde1a82640 1 --2- 172.21.15.39:0/2331781330 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55806f8e6390 0x55806fcbe2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:55:05.196 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:05.191+0000 7fdde2a84640 1 --2- 172.21.15.39:0/2331781330 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55806f907070 0x55806fcca080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:55:05.196 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:05.191+0000 7fdde2283640 1 -- 172.21.15.39:0/2331781330 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55806f8e6390 msgr2=0x55806fcbe2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:55:05.197 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:05.191+0000 7fdde2283640 1 --2- 172.21.15.39:0/2331781330 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55806f8e6390 0x55806fcbe2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:05.197 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:05.191+0000 7fdde2283640 1 -- 172.21.15.39:0/2331781330 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55806f907070 msgr2=0x55806fcca080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:55:05.197 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:05.191+0000 7fdde2283640 1 --2- 172.21.15.39:0/2331781330 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55806f907070 0x55806fcca080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:05.197 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:05.191+0000 7fdde2283640 1 -- 172.21.15.39:0/2331781330 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55806fbdcc40 con 0x55806f8de1b0 2024-09-21T03:55:05.197 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:05.191+0000 7fdde2283640 1 --2- 172.21.15.39:0/2331781330 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55806f8de1b0 0x55806fcbdd70 secure :-1 s=READY pgs=1183 cs=0 l=1 rev1=1 crypto rx=0x7fddd80049b0 tx=0x7fddd800bfa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e8deb268c5aeccb5 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:55:05.197 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:05.191+0000 7fddc5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-09-21T03:55:05.197 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:05.191+0000 7fddc5ffb640 1 -- 172.21.15.39:0/2331781330 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddd802a060 con 0x55806f8de1b0 2024-09-21T03:55:05.197 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:05.191+0000 7fddc5ffb640 1 -- 172.21.15.39:0/2331781330 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8528+0+0 (secure 0 0 0) 0x7fddd8003de0 con 0x55806f8de1b0 2024-09-21T03:55:05.197 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:05.191+0000 7fddeaa5b740 1 -- 172.21.15.39:0/2331781330 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fddd8015070 con 0x55806f8de1b0 2024-09-21T03:55:05.197 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:05.192+0000 7fddc5ffb640 1 -- 172.21.15.39:0/2331781330 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fddd8025480 con 0x55806f8de1b0 2024-09-21T03:55:05.197 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:05.192+0000 7fddeaa5b740 10 client.17850 fetch_fsmap learned FSMap version 29 2024-09-21T03:55:05.197 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:05.192+0000 7fddeaa5b740 1 -- 172.21.15.39:0/2331781330 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55806fcccaa0 con 0x55806f8de1b0 2024-09-21T03:55:05.197 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:05.192+0000 7fddc5ffb640 1 -- 172.21.15.39:0/2331781330 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fddd8015070 con 0x55806f8de1b0 2024-09-21T03:55:05.197 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:05.192+0000 7fddc5ffb640 1 -- 172.21.15.39:0/2331781330 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fddd80117b0 con 0x55806f8de1b0 2024-09-21T03:55:05.197 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:05.192+0000 7fddeaa5b740 10 client.17850 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T03:55:05.197 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:05.192+0000 7fddeaa5b740 10 client.17850 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T03:55:05.198 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:05.192+0000 7fddeaa5b740 20 client.17850 populate_metadata read hostname 'smithi039' 2024-09-21T03:55:05.198 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:05.192+0000 7fddeaa5b740 1 --2- 172.21.15.39:0/2331781330 >> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] conn(0x55806f8c59f0 0x55806fccff10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:55:05.198 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:05.192+0000 7fddeaa5b740 4 client.17850 mds_command: new command op to 24695 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:55:05.198 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:05.192+0000 7fddeaa5b740 1 -- 172.21.15.39:0/2331781330 --> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55806fc26da0 con 0x55806f8c59f0 2024-09-21T03:55:05.198 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:05.193+0000 7fdde1a82640 1 --2- 172.21.15.39:0/2331781330 >> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] conn(0x55806f8c59f0 0x55806fccff10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:55:05.198 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:05.193+0000 7fddc5ffb640 10 client.17850 ms_handle_connect on v2:172.21.15.150:6838/2949386907 2024-09-21T03:55:05.198 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:05.193+0000 7fdde1a82640 1 --2- 172.21.15.39:0/2331781330 >> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] conn(0x55806f8c59f0 0x55806fccff10 crc :-1 s=READY pgs=147 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e6898c1986774a7b server_cookie=34b6c9d84620db74 in_seq=0 out_seq=0 2024-09-21T03:55:05.199 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:05.194+0000 7fddc5ffb640 1 -- 172.21.15.39:0/2331781330 <== mds.0 v2:172.21.15.150:6838/2949386907 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55806fc26da0 con 0x55806f8c59f0 2024-09-21T03:55:05.199 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:05.194+0000 7fddc5ffb640 10 client.17850 handle_command_reply: tid=0 2024-09-21T03:55:05.199 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:05.194+0000 7fddeaa5b740 1 client.17850 shutdown 2024-09-21T03:55:05.199 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:05.194+0000 7fddeaa5b740 1 -- 172.21.15.39:0/2331781330 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55806f8de1b0 msgr2=0x55806fcbdd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:55:05.199 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:05.194+0000 7fddeaa5b740 1 --2- 172.21.15.39:0/2331781330 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55806f8de1b0 0x55806fcbdd70 secure :-1 s=READY pgs=1183 cs=0 l=1 rev1=1 crypto rx=0x7fddd80049b0 tx=0x7fddd800bfa0 comp rx=0 tx=0).stop 2024-09-21T03:55:05.199 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:05.194+0000 7fddeaa5b740 1 -- 172.21.15.39:0/2331781330 shutdown_connections 2024-09-21T03:55:05.199 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:05.194+0000 7fddeaa5b740 1 --2- 172.21.15.39:0/2331781330 >> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] conn(0x55806f8c59f0 0x55806fccff10 crc :-1 s=READY pgs=147 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:05.200 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:05.194+0000 7fddeaa5b740 1 --2- 172.21.15.39:0/2331781330 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55806f907070 0x55806fcca080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:05.200 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:05.194+0000 7fddeaa5b740 1 --2- 172.21.15.39:0/2331781330 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55806f8e6390 0x55806fcbe2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:05.200 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:05.194+0000 7fddc5ffb640 0 client.17850 ms_handle_reset on v2:172.21.15.150:6838/2949386907 2024-09-21T03:55:05.200 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:05.194+0000 7fddeaa5b740 1 --2- 172.21.15.39:0/2331781330 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55806f8de1b0 0x55806fcbdd70 unknown :-1 s=CLOSED pgs=1183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:05.200 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:05.194+0000 7fddeaa5b740 1 -- 172.21.15.39:0/2331781330 >> 172.21.15.39:0/2331781330 conn(0x55806f8fb460 msgr2=0x55806fcaa700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:55:05.200 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:05.194+0000 7fddeaa5b740 1 -- 172.21.15.39:0/2331781330 shutdown_connections 2024-09-21T03:55:05.200 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:05.194+0000 7fddeaa5b740 1 -- 172.21.15.39:0/2331781330 wait complete. 2024-09-21T03:55:05.200 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:05.194+0000 7fddeaa5b740 20 client.17850 trim_cache size 0 max 16384 2024-09-21T03:55:05.215 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:05.212+0000 7fddeaa5b740 1 Processor -- start 2024-09-21T03:55:05.215 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:05.212+0000 7fddeaa5b740 1 -- start start 2024-09-21T03:55:05.215 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:05.212+0000 7fddeaa5b740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55806f8c59f0 0x55806fcbe230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:55:05.215 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:05.212+0000 7fddeaa5b740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55806f8de1b0 0x55806fcac1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:55:05.215 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:05.212+0000 7fddeaa5b740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55806f8e6390 0x55806fcc9f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:55:05.215 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:05.212+0000 7fddeaa5b740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fdddc1ad5c0 con 0x55806f8c59f0 2024-09-21T03:55:05.215 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:05.212+0000 7fddeaa5b740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fdddc1af100 con 0x55806f8de1b0 2024-09-21T03:55:05.215 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:05.212+0000 7fddeaa5b740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fdddc1adc90 con 0x55806f8e6390 2024-09-21T03:55:05.215 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:05.212+0000 7fdde2283640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55806f8c59f0 0x55806fcbe230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:55:05.215 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:05.212+0000 7fdde2283640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55806f8c59f0 0x55806fcbe230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:53756/0 (socket says 172.21.15.39:53756) 2024-09-21T03:55:05.216 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:05.212+0000 7fdde2283640 1 -- 172.21.15.39:0/258968947 learned_addr learned my addr 172.21.15.39:0/258968947 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:55:05.216 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:05.212+0000 7fdde2a84640 1 --2- 172.21.15.39:0/258968947 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55806f8e6390 0x55806fcc9f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:55:05.216 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:05.213+0000 7fdde1a82640 1 --2- 172.21.15.39:0/258968947 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55806f8de1b0 0x55806fcac1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:55:05.216 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:05.213+0000 7fdde2283640 1 -- 172.21.15.39:0/258968947 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55806f8e6390 msgr2=0x55806fcc9f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:55:05.216 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:05.213+0000 7fdde2283640 1 --2- 172.21.15.39:0/258968947 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55806f8e6390 0x55806fcc9f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:05.216 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:05.213+0000 7fdde2283640 1 -- 172.21.15.39:0/258968947 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55806f8de1b0 msgr2=0x55806fcac1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:55:05.216 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:05.213+0000 7fdde2283640 1 --2- 172.21.15.39:0/258968947 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55806f8de1b0 0x55806fcac1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:05.216 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:05.213+0000 7fdde2283640 1 -- 172.21.15.39:0/258968947 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55806fcbcfb0 con 0x55806f8c59f0 2024-09-21T03:55:05.216 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:05.213+0000 7fdde2a84640 1 --2- 172.21.15.39:0/258968947 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55806f8e6390 0x55806fcc9f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:55:05.216 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:05.213+0000 7fdde1a82640 1 --2- 172.21.15.39:0/258968947 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55806f8de1b0 0x55806fcac1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:55:05.216 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:05.213+0000 7fdde2283640 1 --2- 172.21.15.39:0/258968947 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55806f8c59f0 0x55806fcbe230 secure :-1 s=READY pgs=1184 cs=0 l=1 rev1=1 crypto rx=0x7fddd8001eb0 tx=0x7fddd8003a70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7b7e416c4dfb35c8 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:55:05.216 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:05.213+0000 7fddc7fff640 1 -- 172.21.15.39:0/258968947 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddd802a070 con 0x55806f8c59f0 2024-09-21T03:55:05.216 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:05.213+0000 7fddc7fff640 1 -- 172.21.15.39:0/258968947 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fddd8015040 con 0x55806f8c59f0 2024-09-21T03:55:05.216 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:05.213+0000 7fddc7fff640 1 -- 172.21.15.39:0/258968947 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddd8025880 con 0x55806f8c59f0 2024-09-21T03:55:05.220 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:05.214+0000 7fddeaa5b740 1 -- 172.21.15.39:0/258968947 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55806f8c59f0 msgr2=0x55806fcbe230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:55:05.220 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:05.214+0000 7fddeaa5b740 1 --2- 172.21.15.39:0/258968947 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55806f8c59f0 0x55806fcbe230 secure :-1 s=READY pgs=1184 cs=0 l=1 rev1=1 crypto rx=0x7fddd8001eb0 tx=0x7fddd8003a70 comp rx=0 tx=0).stop 2024-09-21T03:55:05.220 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:05.214+0000 7fddeaa5b740 1 -- 172.21.15.39:0/258968947 shutdown_connections 2024-09-21T03:55:05.220 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:05.214+0000 7fddeaa5b740 1 --2- 172.21.15.39:0/258968947 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55806f8e6390 0x55806fcc9f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:05.220 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:05.214+0000 7fddeaa5b740 1 --2- 172.21.15.39:0/258968947 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55806f8de1b0 0x55806fcac1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:05.220 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:05.214+0000 7fddeaa5b740 1 --2- 172.21.15.39:0/258968947 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55806f8c59f0 0x55806fcbe230 unknown :-1 s=CLOSED pgs=1184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:05.220 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:05.214+0000 7fddeaa5b740 1 -- 172.21.15.39:0/258968947 >> 172.21.15.39:0/258968947 conn(0x55806f8fb460 msgr2=0x55806fcbdb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:55:05.220 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:05.214+0000 7fddeaa5b740 1 -- 172.21.15.39:0/258968947 shutdown_connections 2024-09-21T03:55:05.221 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:05.214+0000 7fddeaa5b740 1 -- 172.21.15.39:0/258968947 wait complete. 2024-09-21T03:55:05.221 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:05.214+0000 7fddeaa5b740 1 -- start start 2024-09-21T03:55:05.221 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:05.214+0000 7fddeaa5b740 1 Processor -- start 2024-09-21T03:55:05.221 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:05.215+0000 7fddeaa5b740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55806f8c59f0 0x55806fcc6930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:55:05.221 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:05.215+0000 7fddeaa5b740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55806f8de1b0 0x55806fcc1de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:55:05.221 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:05.215+0000 7fddeaa5b740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55806f8e6390 0x55806fcc2480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:55:05.221 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:05.215+0000 7fddeaa5b740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fdddc1aea30 con 0x55806f8e6390 2024-09-21T03:55:05.221 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:05.215+0000 7fddeaa5b740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fdddc1ae360 con 0x55806f8de1b0 2024-09-21T03:55:05.221 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:05.215+0000 7fddeaa5b740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fdddc1ab5a0 con 0x55806f8c59f0 2024-09-21T03:55:05.221 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:05.215+0000 7fddeaa5b740 10 client.0 _ll_register_callbacks cb 0x7fddcc10b5f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:55:05.221 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:05.215+0000 7fdde2a84640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55806f8e6390 0x55806fcc2480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:55:05.221 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:05.215+0000 7fdde2a84640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55806f8e6390 0x55806fcc2480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:53772/0 (socket says 172.21.15.39:53772) 2024-09-21T03:55:05.221 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:05.215+0000 7fdde2a84640 1 -- 172.21.15.39:0/1795286848 learned_addr learned my addr 172.21.15.39:0/1795286848 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:55:05.221 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:05.215+0000 7fdde1a82640 1 --2- 172.21.15.39:0/1795286848 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55806f8de1b0 0x55806fcc1de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:55:05.222 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:05.215+0000 7fdde2283640 1 --2- 172.21.15.39:0/1795286848 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55806f8c59f0 0x55806fcc6930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:55:05.222 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:05.215+0000 7fdde2a84640 1 -- 172.21.15.39:0/1795286848 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55806f8c59f0 msgr2=0x55806fcc6930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:55:05.222 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:05.215+0000 7fdde2a84640 1 --2- 172.21.15.39:0/1795286848 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55806f8c59f0 0x55806fcc6930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:05.222 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:05.215+0000 7fdde2a84640 1 -- 172.21.15.39:0/1795286848 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55806f8de1b0 msgr2=0x55806fcc1de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:55:05.222 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:05.215+0000 7fdde2a84640 1 --2- 172.21.15.39:0/1795286848 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55806f8de1b0 0x55806fcc1de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:05.222 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:05.215+0000 7fdde2a84640 1 -- 172.21.15.39:0/1795286848 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55806fcc2c10 con 0x55806f8e6390 2024-09-21T03:55:05.222 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:05.215+0000 7fdde2283640 1 --2- 172.21.15.39:0/1795286848 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55806f8c59f0 0x55806fcc6930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:55:05.222 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:05.215+0000 7fdde2a84640 1 --2- 172.21.15.39:0/1795286848 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55806f8e6390 0x55806fcc2480 secure :-1 s=READY pgs=1185 cs=0 l=1 rev1=1 crypto rx=0x7fddd40108c0 tx=0x7fddd4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2538217efb5e5530 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:55:05.222 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:05.215+0000 7fddc5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-09-21T03:55:05.222 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:05.216+0000 7fddc5ffb640 1 -- 172.21.15.39:0/1795286848 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddd402d060 con 0x55806f8e6390 2024-09-21T03:55:05.222 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:05.216+0000 7fddc5ffb640 1 -- 172.21.15.39:0/1795286848 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8528+0+0 (secure 0 0 0) 0x7fddd401e8e0 con 0x55806f8e6390 2024-09-21T03:55:05.222 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:05.216+0000 7fddeaa5b740 1 -- 172.21.15.39:0/1795286848 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fddd8015040 con 0x55806f8e6390 2024-09-21T03:55:05.223 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:05.216+0000 7fddc5ffb640 1 -- 172.21.15.39:0/1795286848 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fddd401eae0 con 0x55806f8e6390 2024-09-21T03:55:05.223 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:05.216+0000 7fddeaa5b740 10 client.17856 fetch_fsmap learned FSMap version 29 2024-09-21T03:55:05.223 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:05.216+0000 7fddeaa5b740 1 -- 172.21.15.39:0/1795286848 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55806fd3a7d0 con 0x55806f8e6390 2024-09-21T03:55:05.223 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:05.216+0000 7fddc5ffb640 1 -- 172.21.15.39:0/1795286848 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fddd8015040 con 0x55806f8e6390 2024-09-21T03:55:05.223 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:05.216+0000 7fddc5ffb640 1 -- 172.21.15.39:0/1795286848 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fddd400f040 con 0x55806f8e6390 2024-09-21T03:55:05.223 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:05.216+0000 7fddeaa5b740 10 client.17856 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T03:55:05.223 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:05.216+0000 7fddeaa5b740 10 client.17856 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T03:55:05.223 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:05.216+0000 7fddeaa5b740 20 client.17856 populate_metadata read hostname 'smithi039' 2024-09-21T03:55:05.223 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:05.216+0000 7fddeaa5b740 1 --2- 172.21.15.39:0/1795286848 >> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] conn(0x55806f907070 0x55806fd3cff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:55:05.223 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:05.216+0000 7fddeaa5b740 4 client.17856 mds_command: new command op to 24695 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-21T03:55:05.223 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:05.216+0000 7fddeaa5b740 1 -- 172.21.15.39:0/1795286848 --> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55806fc22850 con 0x55806f907070 2024-09-21T03:55:05.223 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:05.217+0000 7fdde2283640 1 --2- 172.21.15.39:0/1795286848 >> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] conn(0x55806f907070 0x55806fd3cff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:55:05.223 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:05.217+0000 7fddc5ffb640 10 client.17856 ms_handle_connect on v2:172.21.15.150:6838/2949386907 2024-09-21T03:55:05.223 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:05.217+0000 7fdde2283640 1 --2- 172.21.15.39:0/1795286848 >> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] conn(0x55806f907070 0x55806fd3cff0 crc :-1 s=READY pgs=148 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=45a4f16d0939de8b server_cookie=80de77401756883b in_seq=0 out_seq=0 2024-09-21T03:55:05.224 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:05.217+0000 7fddc5ffb640 1 -- 172.21.15.39:0/1795286848 <== mds.0 v2:172.21.15.150:6838/2949386907 1 ==== command_reply(tid 0: 0 ) ==== 8+0+4918 (crc 0 0 0) 0x55806fc22850 con 0x55806f907070 2024-09-21T03:55:05.224 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:05.217+0000 7fddc5ffb640 10 client.17856 handle_command_reply: tid=0 2024-09-21T03:55:05.224 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:05.217+0000 7fddeaa5b740 1 client.17856 shutdown 2024-09-21T03:55:05.224 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:05.218+0000 7fddeaa5b740 1 -- 172.21.15.39:0/1795286848 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55806f8e6390 msgr2=0x55806fcc2480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:55:05.224 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:05.218+0000 7fddeaa5b740 1 --2- 172.21.15.39:0/1795286848 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55806f8e6390 0x55806fcc2480 secure :-1 s=READY pgs=1185 cs=0 l=1 rev1=1 crypto rx=0x7fddd40108c0 tx=0x7fddd4004a40 comp rx=0 tx=0).stop 2024-09-21T03:55:05.224 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:05.218+0000 7fddeaa5b740 1 -- 172.21.15.39:0/1795286848 shutdown_connections 2024-09-21T03:55:05.224 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:05.218+0000 7fddeaa5b740 1 --2- 172.21.15.39:0/1795286848 >> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] conn(0x55806f907070 0x55806fd3cff0 crc :-1 s=READY pgs=148 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:05.224 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:05.218+0000 7fddeaa5b740 1 --2- 172.21.15.39:0/1795286848 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55806f8e6390 0x55806fcc2480 unknown :-1 s=CLOSED pgs=1185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:05.224 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:05.218+0000 7fddeaa5b740 1 --2- 172.21.15.39:0/1795286848 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55806f8de1b0 0x55806fcc1de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:05.224 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:05.218+0000 7fddeaa5b740 1 --2- 172.21.15.39:0/1795286848 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55806f8c59f0 0x55806fcc6930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:05.224 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:05.218+0000 7fddeaa5b740 1 -- 172.21.15.39:0/1795286848 >> 172.21.15.39:0/1795286848 conn(0x55806f8fb460 msgr2=0x55806fcaf2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:55:05.224 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:05.218+0000 7fddc5ffb640 0 client.17856 ms_handle_reset on v2:172.21.15.150:6838/2949386907 2024-09-21T03:55:05.224 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:05.218+0000 7fddeaa5b740 1 -- 172.21.15.39:0/1795286848 shutdown_connections 2024-09-21T03:55:05.224 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:05.218+0000 7fddeaa5b740 1 -- 172.21.15.39:0/1795286848 wait complete. 2024-09-21T03:55:05.225 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:05.218+0000 7fddeaa5b740 20 client.17856 trim_cache size 0 max 16384 2024-09-21T03:55:05.225 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-09-21T03:55:05.225 INFO:teuthology.orchestra.run.smithi039.stdout: "status": "no active scrubs running", 2024-09-21T03:55:05.225 INFO:teuthology.orchestra.run.smithi039.stdout: "scrubs": {}, 2024-09-21T03:55:05.225 INFO:teuthology.orchestra.run.smithi039.stdout: "scrub_stats": { 2024-09-21T03:55:05.225 INFO:teuthology.orchestra.run.smithi039.stdout: "02e16ae1-0bac-41e9-b95e-fc6b17d207ac": { 2024-09-21T03:55:05.225 INFO:teuthology.orchestra.run.smithi039.stdout: "start_time": "2024-09-21T03:54:31.853401+0000", 2024-09-21T03:55:05.225 INFO:teuthology.orchestra.run.smithi039.stdout: "path": "/", 2024-09-21T03:55:05.225 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_started": 0, 2024-09-21T03:55:05.225 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_passed": 0, 2024-09-21T03:55:05.226 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_failed": 0, 2024-09-21T03:55:05.226 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_skipped": 0 2024-09-21T03:55:05.226 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:55:05.226 INFO:teuthology.orchestra.run.smithi039.stdout: "e28dcd13-f2ac-4cb7-a346-58917d542b15": { 2024-09-21T03:55:05.226 INFO:teuthology.orchestra.run.smithi039.stdout: "start_time": "2024-09-21T03:54:18.221680+0000", 2024-09-21T03:55:05.226 INFO:teuthology.orchestra.run.smithi039.stdout: "path": "/", 2024-09-21T03:55:05.226 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_started": 0, 2024-09-21T03:55:05.226 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_passed": 0, 2024-09-21T03:55:05.226 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_failed": 0, 2024-09-21T03:55:05.226 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_skipped": 0 2024-09-21T03:55:05.226 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:55:05.227 INFO:teuthology.orchestra.run.smithi039.stdout: "ad87da79-0248-4e18-9694-8228cb95422f": { 2024-09-21T03:55:05.227 INFO:teuthology.orchestra.run.smithi039.stdout: "start_time": "2024-09-21T03:53:18.441907+0000", 2024-09-21T03:55:05.227 INFO:teuthology.orchestra.run.smithi039.stdout: "path": "/", 2024-09-21T03:55:05.227 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_started": 0, 2024-09-21T03:55:05.227 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_passed": 0, 2024-09-21T03:55:05.227 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_failed": 0, 2024-09-21T03:55:05.227 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_skipped": 0 2024-09-21T03:55:05.227 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:55:05.227 INFO:teuthology.orchestra.run.smithi039.stdout: "02509e4f-3911-4983-b16f-df8ded0b9998": { 2024-09-21T03:55:05.227 INFO:teuthology.orchestra.run.smithi039.stdout: "start_time": "2024-09-21T03:46:17.889576+0000", 2024-09-21T03:55:05.227 INFO:teuthology.orchestra.run.smithi039.stdout: "path": "/", 2024-09-21T03:55:05.227 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_started": 0, 2024-09-21T03:55:05.227 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_passed": 0, 2024-09-21T03:55:05.227 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_failed": 0, 2024-09-21T03:55:05.227 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_skipped": 0 2024-09-21T03:55:05.228 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:55:05.228 INFO:teuthology.orchestra.run.smithi039.stdout: "10728de8-c49d-4edc-be2b-639249255e94": { 2024-09-21T03:55:05.228 INFO:teuthology.orchestra.run.smithi039.stdout: "start_time": "2024-09-21T03:47:03.965623+0000", 2024-09-21T03:55:05.228 INFO:teuthology.orchestra.run.smithi039.stdout: "path": "/", 2024-09-21T03:55:05.228 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_started": 0, 2024-09-21T03:55:05.228 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_passed": 0, 2024-09-21T03:55:05.228 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_failed": 0, 2024-09-21T03:55:05.228 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_skipped": 0 2024-09-21T03:55:05.228 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:55:05.228 INFO:teuthology.orchestra.run.smithi039.stdout: "9860c2f4-b7b1-47aa-865a-9a1c4434a5f6": { 2024-09-21T03:55:05.228 INFO:teuthology.orchestra.run.smithi039.stdout: "start_time": "2024-09-21T03:49:17.718867+0000", 2024-09-21T03:55:05.228 INFO:teuthology.orchestra.run.smithi039.stdout: "path": "/", 2024-09-21T03:55:05.228 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_started": 0, 2024-09-21T03:55:05.228 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_passed": 0, 2024-09-21T03:55:05.229 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_failed": 0, 2024-09-21T03:55:05.229 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_skipped": 0 2024-09-21T03:55:05.229 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:55:05.229 INFO:teuthology.orchestra.run.smithi039.stdout: "d9391d69-baac-453a-9d52-d046760c070d": { 2024-09-21T03:55:05.229 INFO:teuthology.orchestra.run.smithi039.stdout: "start_time": "2024-09-21T03:53:32.364622+0000", 2024-09-21T03:55:05.229 INFO:teuthology.orchestra.run.smithi039.stdout: "path": "/", 2024-09-21T03:55:05.229 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_started": 0, 2024-09-21T03:55:05.229 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_passed": 0, 2024-09-21T03:55:05.229 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_failed": 0, 2024-09-21T03:55:05.229 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_skipped": 0 2024-09-21T03:55:05.229 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:55:05.229 INFO:teuthology.orchestra.run.smithi039.stdout: "c7c700f3-2cd8-41ac-8381-b57dd1b37b02": { 2024-09-21T03:55:05.229 INFO:teuthology.orchestra.run.smithi039.stdout: "start_time": "2024-09-21T03:52:32.368858+0000", 2024-09-21T03:55:05.229 INFO:teuthology.orchestra.run.smithi039.stdout: "path": "/", 2024-09-21T03:55:05.230 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_started": 0, 2024-09-21T03:55:05.230 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_passed": 0, 2024-09-21T03:55:05.230 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_failed": 0, 2024-09-21T03:55:05.230 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_skipped": 0 2024-09-21T03:55:05.230 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:55:05.230 INFO:teuthology.orchestra.run.smithi039.stdout: "29411307-eb8d-4339-8d2b-47ebd32c0080": { 2024-09-21T03:55:05.230 INFO:teuthology.orchestra.run.smithi039.stdout: "start_time": "2024-09-21T03:47:38.092920+0000", 2024-09-21T03:55:05.230 INFO:teuthology.orchestra.run.smithi039.stdout: "path": "/", 2024-09-21T03:55:05.230 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_started": 0, 2024-09-21T03:55:05.230 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_passed": 0, 2024-09-21T03:55:05.230 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_failed": 0, 2024-09-21T03:55:05.230 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_skipped": 0 2024-09-21T03:55:05.230 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:55:05.231 INFO:teuthology.orchestra.run.smithi039.stdout: "ede73e6e-3fd0-4a7f-b5bb-f68946f57eea": { 2024-09-21T03:55:05.231 INFO:teuthology.orchestra.run.smithi039.stdout: "start_time": "2024-09-21T03:51:32.235096+0000", 2024-09-21T03:55:05.231 INFO:teuthology.orchestra.run.smithi039.stdout: "path": "/", 2024-09-21T03:55:05.231 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_started": 0, 2024-09-21T03:55:05.231 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_passed": 0, 2024-09-21T03:55:05.231 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_failed": 0, 2024-09-21T03:55:05.231 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_skipped": 0 2024-09-21T03:55:05.231 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:55:05.231 INFO:teuthology.orchestra.run.smithi039.stdout: "e28ad743-2b22-45b3-8799-a638c2883623": { 2024-09-21T03:55:05.231 INFO:teuthology.orchestra.run.smithi039.stdout: "start_time": "2024-09-21T03:48:31.448821+0000", 2024-09-21T03:55:05.231 INFO:teuthology.orchestra.run.smithi039.stdout: "path": "/", 2024-09-21T03:55:05.231 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_started": 0, 2024-09-21T03:55:05.231 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_passed": 0, 2024-09-21T03:55:05.231 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_failed": 0, 2024-09-21T03:55:05.231 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_skipped": 0 2024-09-21T03:55:05.232 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:55:05.232 INFO:teuthology.orchestra.run.smithi039.stdout: "6e760f1f-9048-4f92-bd66-d708fbef75b3": { 2024-09-21T03:55:05.232 INFO:teuthology.orchestra.run.smithi039.stdout: "start_time": "2024-09-21T03:50:03.711918+0000", 2024-09-21T03:55:05.232 INFO:teuthology.orchestra.run.smithi039.stdout: "path": "/", 2024-09-21T03:55:05.232 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_started": 0, 2024-09-21T03:55:05.232 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_passed": 0, 2024-09-21T03:55:05.232 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_failed": 0, 2024-09-21T03:55:05.232 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_skipped": 0 2024-09-21T03:55:05.232 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:55:05.232 INFO:teuthology.orchestra.run.smithi039.stdout: "c7e2b8e4-057d-435f-b2bd-b633de98035f": { 2024-09-21T03:55:05.232 INFO:teuthology.orchestra.run.smithi039.stdout: "start_time": "2024-09-21T03:47:19.400305+0000", 2024-09-21T03:55:05.232 INFO:teuthology.orchestra.run.smithi039.stdout: "path": "/", 2024-09-21T03:55:05.232 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_started": 0, 2024-09-21T03:55:05.232 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_passed": 0, 2024-09-21T03:55:05.232 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_failed": 0, 2024-09-21T03:55:05.233 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_skipped": 0 2024-09-21T03:55:05.233 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:55:05.233 INFO:teuthology.orchestra.run.smithi039.stdout: "74ab917e-ecea-4778-a1c3-c184214fb24c": { 2024-09-21T03:55:05.233 INFO:teuthology.orchestra.run.smithi039.stdout: "start_time": "2024-09-21T03:50:17.726630+0000", 2024-09-21T03:55:05.233 INFO:teuthology.orchestra.run.smithi039.stdout: "path": "/", 2024-09-21T03:55:05.233 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_started": 0, 2024-09-21T03:55:05.233 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_passed": 0, 2024-09-21T03:55:05.233 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_failed": 0, 2024-09-21T03:55:05.233 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_skipped": 0 2024-09-21T03:55:05.233 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:55:05.233 INFO:teuthology.orchestra.run.smithi039.stdout: "160750d8-401f-4810-866d-e63e7e4bc3e0": { 2024-09-21T03:55:05.233 INFO:teuthology.orchestra.run.smithi039.stdout: "start_time": "2024-09-21T03:51:04.208673+0000", 2024-09-21T03:55:05.233 INFO:teuthology.orchestra.run.smithi039.stdout: "path": "/", 2024-09-21T03:55:05.233 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_started": 0, 2024-09-21T03:55:05.233 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_passed": 0, 2024-09-21T03:55:05.233 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_failed": 0, 2024-09-21T03:55:05.233 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_skipped": 0 2024-09-21T03:55:05.234 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:55:05.234 INFO:teuthology.orchestra.run.smithi039.stdout: "ae1fc4cf-5458-47d0-afcb-282b136dba15": { 2024-09-21T03:55:05.234 INFO:teuthology.orchestra.run.smithi039.stdout: "start_time": "2024-09-21T03:51:18.560378+0000", 2024-09-21T03:55:05.234 INFO:teuthology.orchestra.run.smithi039.stdout: "path": "/", 2024-09-21T03:55:05.234 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_started": 0, 2024-09-21T03:55:05.234 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_passed": 0, 2024-09-21T03:55:05.234 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_failed": 0, 2024-09-21T03:55:05.234 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_skipped": 0 2024-09-21T03:55:05.234 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:55:05.234 INFO:teuthology.orchestra.run.smithi039.stdout: "4bda6ee3-7237-4ee4-96ce-150ad50f26dc": { 2024-09-21T03:55:05.234 INFO:teuthology.orchestra.run.smithi039.stdout: "start_time": "2024-09-21T03:52:18.622141+0000", 2024-09-21T03:55:05.234 INFO:teuthology.orchestra.run.smithi039.stdout: "path": "/", 2024-09-21T03:55:05.234 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_started": 0, 2024-09-21T03:55:05.234 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_passed": 0, 2024-09-21T03:55:05.234 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_failed": 0, 2024-09-21T03:55:05.234 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_skipped": 0 2024-09-21T03:55:05.235 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-21T03:55:05.235 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-21T03:55:05.235 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-09-21T03:55:05.235 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:05.218+0000 7fdde44e7640 1 -- 172.21.15.39:0/3447264566 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7fddc80b6680 msgr2=0x7fddc80b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:55:05.235 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:05.218+0000 7fdde44e7640 1 --2- 172.21.15.39:0/3447264566 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7fddc80b6680 0x7fddc80b8b90 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7fddd4005c40 tx=0x7fddd4009760 comp rx=0 tx=0).stop 2024-09-21T03:55:05.235 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:05.218+0000 7fdde44e7640 1 -- 172.21.15.39:0/3447264566 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdddc1083c0 msgr2=0x7fdddc1999b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:55:05.235 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:05.218+0000 7fdde44e7640 1 --2- 172.21.15.39:0/3447264566 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdddc1083c0 0x7fdddc1999b0 secure :-1 s=READY pgs=1182 cs=0 l=1 rev1=1 crypto rx=0x7fddcc032fd0 tx=0x7fddcc004270 comp rx=0 tx=0).stop 2024-09-21T03:55:05.235 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:05.218+0000 7fdde44e7640 1 -- 172.21.15.39:0/3447264566 shutdown_connections 2024-09-21T03:55:05.236 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:05.218+0000 7fdde44e7640 1 --2- 172.21.15.39:0/3447264566 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7fddc80b6680 0x7fddc80b8b90 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:05.236 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:05.218+0000 7fdde44e7640 1 --2- 172.21.15.39:0/3447264566 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdddc108d10 0x7fdddc19ec50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:05.236 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:05.219+0000 7fdde44e7640 1 --2- 172.21.15.39:0/3447264566 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdddc1083c0 0x7fdddc1999b0 unknown :-1 s=CLOSED pgs=1182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:05.236 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:05.219+0000 7fdde44e7640 1 --2- 172.21.15.39:0/3447264566 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdddc102a80 0x7fdddc199410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:05.236 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:05.219+0000 7fdde44e7640 1 -- 172.21.15.39:0/3447264566 >> 172.21.15.39:0/3447264566 conn(0x7fdddc079740 msgr2=0x7fdddc104430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:55:05.236 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:05.219+0000 7fdde44e7640 1 -- 172.21.15.39:0/3447264566 shutdown_connections 2024-09-21T03:55:05.236 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:05.219+0000 7fdde44e7640 1 -- 172.21.15.39:0/3447264566 wait complete. 2024-09-21T03:55:05.334 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:55:04 smithi039 ceph-mon[31211]: pgmap v600: 97 pgs: 97 active+clean; 10 MiB data, 525 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 597 B/s wr, 10 op/s 2024-09-21T03:55:05.468 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:55:04 smithi161 ceph-mon[34533]: pgmap v600: 97 pgs: 97 active+clean; 10 MiB data, 525 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 597 B/s wr, 10 op/s 2024-09-21T03:55:05.475 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:55:04 smithi150 ceph-mon[34289]: pgmap v600: 97 pgs: 97 active+clean; 10 MiB data, 525 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 597 B/s wr, 10 op/s 2024-09-21T03:55:06.246 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-21T03:55:06.247 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 51116d5c-77ca-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T03:55:06.511 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/51116d5c-77ca-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:55:06.990 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:06.988+0000 7f9b6f444640 1 -- 172.21.15.39:0/4027038481 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9b68109e10 msgr2=0x7f9b68110640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:55:06.990 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:06.988+0000 7f9b6f444640 1 --2- 172.21.15.39:0/4027038481 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9b68109e10 0x7f9b68110640 secure :-1 s=READY pgs=1186 cs=0 l=1 rev1=1 crypto rx=0x7f9b60009900 tx=0x7f9b6002a410 comp rx=0 tx=0).stop 2024-09-21T03:55:06.991 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:06.988+0000 7f9b6f444640 1 -- 172.21.15.39:0/4027038481 shutdown_connections 2024-09-21T03:55:06.991 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:06.988+0000 7f9b6f444640 1 --2- 172.21.15.39:0/4027038481 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9b68109e10 0x7f9b68110640 unknown :-1 s=CLOSED pgs=1186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:06.991 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:06.988+0000 7f9b6f444640 1 --2- 172.21.15.39:0/4027038481 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9b681094c0 0x7f9b681098d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:06.991 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:06.988+0000 7f9b6f444640 1 --2- 172.21.15.39:0/4027038481 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9b68103b80 0x7f9b68103f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:06.991 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:06.988+0000 7f9b6f444640 1 -- 172.21.15.39:0/4027038481 >> 172.21.15.39:0/4027038481 conn(0x7f9b680ff610 msgr2=0x7f9b68101a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:55:06.991 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:06.988+0000 7f9b6f444640 1 -- 172.21.15.39:0/4027038481 shutdown_connections 2024-09-21T03:55:06.991 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:06.988+0000 7f9b6f444640 1 -- 172.21.15.39:0/4027038481 wait complete. 2024-09-21T03:55:06.993 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:06.989+0000 7f9b6f444640 1 Processor -- start 2024-09-21T03:55:06.993 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:06.989+0000 7f9b6f444640 1 -- start start 2024-09-21T03:55:06.993 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:06.989+0000 7f9b6f444640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9b68103b80 0x7f9b6819a440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:55:06.993 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:06.989+0000 7f9b6f444640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9b681094c0 0x7f9b6819a9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:55:06.993 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:06.989+0000 7f9b6f444640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9b68109e10 0x7f9b6819fc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:55:06.993 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:06.989+0000 7f9b6f444640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f9b68113ad0 con 0x7f9b681094c0 2024-09-21T03:55:06.993 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:06.989+0000 7f9b6f444640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f9b68113950 con 0x7f9b68103b80 2024-09-21T03:55:06.993 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:06.989+0000 7f9b6f444640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f9b68113c50 con 0x7f9b68109e10 2024-09-21T03:55:06.994 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:06.989+0000 7f9b6c9df640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9b681094c0 0x7f9b6819a9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:55:06.994 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:06.990+0000 7f9b6c9df640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9b681094c0 0x7f9b6819a9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:53790/0 (socket says 172.21.15.39:53790) 2024-09-21T03:55:06.994 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:06.990+0000 7f9b6c9df640 1 -- 172.21.15.39:0/1038007157 learned_addr learned my addr 172.21.15.39:0/1038007157 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:55:06.994 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:06.990+0000 7f9b6d9e1640 1 --2- 172.21.15.39:0/1038007157 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9b68109e10 0x7f9b6819fc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:55:06.994 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:06.990+0000 7f9b6d1e0640 1 --2- 172.21.15.39:0/1038007157 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9b68103b80 0x7f9b6819a440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:55:06.994 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:06.990+0000 7f9b6c9df640 1 -- 172.21.15.39:0/1038007157 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9b68109e10 msgr2=0x7f9b6819fc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:55:06.994 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:06.990+0000 7f9b6c9df640 1 --2- 172.21.15.39:0/1038007157 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9b68109e10 0x7f9b6819fc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:06.994 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:06.990+0000 7f9b6c9df640 1 -- 172.21.15.39:0/1038007157 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9b68103b80 msgr2=0x7f9b6819a440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:55:06.994 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:06.990+0000 7f9b6c9df640 1 --2- 172.21.15.39:0/1038007157 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9b68103b80 0x7f9b6819a440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:06.994 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:06.990+0000 7f9b6c9df640 1 -- 172.21.15.39:0/1038007157 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9b681a02f0 con 0x7f9b681094c0 2024-09-21T03:55:06.994 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:06.990+0000 7f9b6c9df640 1 --2- 172.21.15.39:0/1038007157 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9b681094c0 0x7f9b6819a9e0 secure :-1 s=READY pgs=1187 cs=0 l=1 rev1=1 crypto rx=0x7f9b5c00d740 tx=0x7f9b5c00dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:55:06.994 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:06.990+0000 7f9b5a7fc640 1 -- 172.21.15.39:0/1038007157 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b5c017070 con 0x7f9b681094c0 2024-09-21T03:55:06.994 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:06.990+0000 7f9b5a7fc640 1 -- 172.21.15.39:0/1038007157 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9b5c00de50 con 0x7f9b681094c0 2024-09-21T03:55:06.995 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:06.990+0000 7f9b6f444640 1 -- 172.21.15.39:0/1038007157 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9b681a05e0 con 0x7f9b681094c0 2024-09-21T03:55:06.995 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:06.990+0000 7f9b5a7fc640 1 -- 172.21.15.39:0/1038007157 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b5c00f620 con 0x7f9b681094c0 2024-09-21T03:55:06.995 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:06.990+0000 7f9b6f444640 1 -- 172.21.15.39:0/1038007157 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9b681a6e00 con 0x7f9b681094c0 2024-09-21T03:55:06.995 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:06.991+0000 7f9b6f444640 1 -- 172.21.15.39:0/1038007157 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9b68103f90 con 0x7f9b681094c0 2024-09-21T03:55:06.995 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:06.993+0000 7f9b5a7fc640 1 -- 172.21.15.39:0/1038007157 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 27) ==== 154643+0+0 (secure 0 0 0) 0x7f9b5c01a070 con 0x7f9b681094c0 2024-09-21T03:55:06.996 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:06.994+0000 7f9b5a7fc640 1 --2- 172.21.15.39:0/1038007157 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f9b540b66b0 0x7f9b540b8bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:55:06.996 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:06.994+0000 7f9b6d1e0640 1 --2- 172.21.15.39:0/1038007157 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f9b540b66b0 0x7f9b540b8bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:55:06.996 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:06.995+0000 7f9b5a7fc640 1 -- 172.21.15.39:0/1038007157 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8528+0+0 (secure 0 0 0) 0x7f9b5c0e70c0 con 0x7f9b681094c0 2024-09-21T03:55:06.997 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:06.995+0000 7f9b6d1e0640 1 --2- 172.21.15.39:0/1038007157 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f9b540b66b0 0x7f9b540b8bc0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f9b64007350 tx=0x7f9b640072e0 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:55:06.999 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:06.998+0000 7f9b5a7fc640 1 -- 172.21.15.39:0/1038007157 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f9b5c0ad610 con 0x7f9b681094c0 2024-09-21T03:55:07.228 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:07.225+0000 7f9b6f444640 1 -- 172.21.15.39:0/1038007157 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f9b6819b570 con 0x7f9b681094c0 2024-09-21T03:55:07.229 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:55:07 smithi039 ceph-mon[31211]: pgmap v601: 97 pgs: 97 active+clean; 10 MiB data, 525 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 44 KiB/s wr, 27 op/s 2024-09-21T03:55:07.229 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:07.226+0000 7f9b5a7fc640 1 -- 172.21.15.39:0/1038007157 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11978 (secure 0 0 0) 0x7f9b6819b570 con 0x7f9b681094c0 2024-09-21T03:55:07.229 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T03:55:07.230 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":29,"btime":"2024-09-21T03:46:08:982881+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14790,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6835/508995295","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":508995295},{"type":"v1","addr":"172.21.15.39:6835","nonce":508995295}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14802,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6837/3915912607","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":3915912607},{"type":"v1","addr":"172.21.15.39:6837","nonce":3915912607}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":29,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T03:44:15.009872+0000","modified":"2024-09-21T03:46:08.430113+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24695,"mds_1":24667,"mds_2":24659,"mds_3":24646,"mds_4":14868},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14856":{"gid":14856,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.39:6839/185509192","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6838","nonce":185509192},{"type":"v1","addr":"172.21.15.39:6839","nonce":185509192}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14868":{"gid":14868,"name":"a","rank":4,"incarnation":25,"state":"up:active","state_seq":7,"addr":"172.21.15.39:6841/4247190456","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6840","nonce":4247190456},{"type":"v1","addr":"172.21.15.39:6841","nonce":4247190456}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24599":{"gid":24599,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.150:6833/3114643217","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":3114643217},{"type":"v1","addr":"172.21.15.150:6833","nonce":3114643217}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24601":{"gid":24601,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.161:6833/2963990654","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":2963990654},{"type":"v1","addr":"172.21.15.161:6833","nonce":2963990654}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24632":{"gid":24632,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.150:6835/724594008","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6834","nonce":724594008},{"type":"v1","addr":"172.21.15.150:6835","nonce":724594008}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24640":{"gid":24640,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.161:6835/3747882110","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":3747882110},{"type":"v1","addr":"172.21.15.161:6835","nonce":3747882110}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24646":{"gid":24646,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":10,"addr":"172.21.15.161:6837/3451544597","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6836","nonce":3451544597},{"type":"v1","addr":"172.21.15.161:6837","nonce":3451544597}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24659":{"gid":24659,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.150:6837/4017043144","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6836","nonce":4017043144},{"type":"v1","addr":"172.21.15.150:6837","nonce":4017043144}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24667":{"gid":24667,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.161:6839/2190107270","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6838","nonce":2190107270},{"type":"v1","addr":"172.21.15.161:6839","nonce":2190107270}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24695":{"gid":24695,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.150:6839/2949386907","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6838","nonce":2949386907},{"type":"v1","addr":"172.21.15.150:6839","nonce":2949386907}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24695,"qdb_cluster":[24695,24667,24659,24646,14868]},"id":1}]} 2024-09-21T03:55:07.230 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 29 2024-09-21T03:55:07.231 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:07.229+0000 7f9b6f444640 1 -- 172.21.15.39:0/1038007157 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f9b540b66b0 msgr2=0x7f9b540b8bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:55:07.232 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:07.229+0000 7f9b6f444640 1 --2- 172.21.15.39:0/1038007157 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f9b540b66b0 0x7f9b540b8bc0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f9b64007350 tx=0x7f9b640072e0 comp rx=0 tx=0).stop 2024-09-21T03:55:07.232 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:07.229+0000 7f9b6f444640 1 -- 172.21.15.39:0/1038007157 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9b681094c0 msgr2=0x7f9b6819a9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:55:07.232 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:07.229+0000 7f9b6f444640 1 --2- 172.21.15.39:0/1038007157 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9b681094c0 0x7f9b6819a9e0 secure :-1 s=READY pgs=1187 cs=0 l=1 rev1=1 crypto rx=0x7f9b5c00d740 tx=0x7f9b5c00dc00 comp rx=0 tx=0).stop 2024-09-21T03:55:07.232 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:07.229+0000 7f9b6f444640 1 -- 172.21.15.39:0/1038007157 shutdown_connections 2024-09-21T03:55:07.232 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:07.229+0000 7f9b6f444640 1 --2- 172.21.15.39:0/1038007157 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f9b540b66b0 0x7f9b540b8bc0 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:07.232 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:07.229+0000 7f9b6f444640 1 --2- 172.21.15.39:0/1038007157 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9b68109e10 0x7f9b6819fc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:07.232 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:07.229+0000 7f9b6f444640 1 --2- 172.21.15.39:0/1038007157 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9b681094c0 0x7f9b6819a9e0 unknown :-1 s=CLOSED pgs=1187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:07.232 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:07.229+0000 7f9b6f444640 1 --2- 172.21.15.39:0/1038007157 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9b68103b80 0x7f9b6819a440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:07.232 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:07.229+0000 7f9b6f444640 1 -- 172.21.15.39:0/1038007157 >> 172.21.15.39:0/1038007157 conn(0x7f9b680ff610 msgr2=0x7f9b68101a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:55:07.232 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:07.229+0000 7f9b6f444640 1 -- 172.21.15.39:0/1038007157 shutdown_connections 2024-09-21T03:55:07.232 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:07.229+0000 7f9b6f444640 1 -- 172.21.15.39:0/1038007157 wait complete. 2024-09-21T03:55:07.468 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:55:07 smithi161 ceph-mon[34533]: pgmap v601: 97 pgs: 97 active+clean; 10 MiB data, 525 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 44 KiB/s wr, 27 op/s 2024-09-21T03:55:07.474 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:55:07 smithi150 ceph-mon[34289]: pgmap v601: 97 pgs: 97 active+clean; 10 MiB data, 525 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 44 KiB/s wr, 27 op/s 2024-09-21T03:55:07.983 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 51116d5c-77ca-11ef-baf5-efdc52797490 -- ceph tell mds.1:4 damage ls 2024-09-21T03:55:08.241 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/51116d5c-77ca-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:55:08.333 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:55:08 smithi039 ceph-mon[31211]: from='client.? 172.21.15.39:0/1038007157' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:55:08.468 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:55:08 smithi161 ceph-mon[34533]: from='client.? 172.21.15.39:0/1038007157' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:55:08.475 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:55:08 smithi150 ceph-mon[34289]: from='client.? 172.21.15.39:0/1038007157' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:55:08.684 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:08.681+0000 7f8cd0cd4640 1 -- 172.21.15.39:0/2555511047 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8ccc103b60 msgr2=0x7f8ccc103f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:55:08.684 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:08.681+0000 7f8cd0cd4640 1 --2- 172.21.15.39:0/2555511047 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8ccc103b60 0x7f8ccc103f70 secure :-1 s=READY pgs=1188 cs=0 l=1 rev1=1 crypto rx=0x7f8cbc009900 tx=0x7f8cbc02a3f0 comp rx=0 tx=0).stop 2024-09-21T03:55:08.685 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:08.681+0000 7f8cd0cd4640 1 -- 172.21.15.39:0/2555511047 shutdown_connections 2024-09-21T03:55:08.685 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:08.681+0000 7f8cd0cd4640 1 --2- 172.21.15.39:0/2555511047 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8ccc109df0 0x7f8ccc110620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:08.685 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:08.681+0000 7f8cd0cd4640 1 --2- 172.21.15.39:0/2555511047 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8ccc1094a0 0x7f8ccc1098b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:08.685 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:08.681+0000 7f8cd0cd4640 1 --2- 172.21.15.39:0/2555511047 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8ccc103b60 0x7f8ccc103f70 unknown :-1 s=CLOSED pgs=1188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:08.685 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:08.681+0000 7f8cd0cd4640 1 -- 172.21.15.39:0/2555511047 >> 172.21.15.39:0/2555511047 conn(0x7f8ccc07b850 msgr2=0x7f8ccc07bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:55:08.685 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:08.681+0000 7f8cd0cd4640 1 -- 172.21.15.39:0/2555511047 shutdown_connections 2024-09-21T03:55:08.685 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:08.681+0000 7f8cd0cd4640 1 -- 172.21.15.39:0/2555511047 wait complete. 2024-09-21T03:55:08.687 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:08.682+0000 7f8cd0cd4640 1 Processor -- start 2024-09-21T03:55:08.687 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:08.682+0000 7f8cd0cd4640 1 -- start start 2024-09-21T03:55:08.687 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:08.683+0000 7f8cd0cd4640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8ccc103b60 0x7f8ccc075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:55:08.687 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:08.683+0000 7f8cd0cd4640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8ccc1094a0 0x7f8ccc075c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:55:08.687 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:08.683+0000 7f8cd0cd4640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8ccc109df0 0x7f8ccc076190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:55:08.687 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:08.683+0000 7f8cd0cd4640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f8ccc113ab0 con 0x7f8ccc1094a0 2024-09-21T03:55:08.687 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:08.683+0000 7f8cd0cd4640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f8ccc113930 con 0x7f8ccc109df0 2024-09-21T03:55:08.687 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:08.683+0000 7f8cd0cd4640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f8ccc113c30 con 0x7f8ccc103b60 2024-09-21T03:55:08.687 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:08.683+0000 7f8cc9d9b640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8ccc1094a0 0x7f8ccc075c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:55:08.688 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:08.683+0000 7f8cc9d9b640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8ccc1094a0 0x7f8ccc075c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:39190/0 (socket says 172.21.15.39:39190) 2024-09-21T03:55:08.688 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:08.683+0000 7f8cc9d9b640 1 -- 172.21.15.39:0/3639902271 learned_addr learned my addr 172.21.15.39:0/3639902271 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:55:08.688 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:08.683+0000 7f8cca59c640 1 --2- 172.21.15.39:0/3639902271 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8ccc103b60 0x7f8ccc075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:55:08.688 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:08.683+0000 7f8ccad9d640 1 --2- 172.21.15.39:0/3639902271 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8ccc109df0 0x7f8ccc076190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:55:08.688 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:08.683+0000 7f8cc9d9b640 1 -- 172.21.15.39:0/3639902271 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8ccc103b60 msgr2=0x7f8ccc075710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:55:08.688 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:08.683+0000 7f8cc9d9b640 1 --2- 172.21.15.39:0/3639902271 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8ccc103b60 0x7f8ccc075710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:08.688 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:08.683+0000 7f8cc9d9b640 1 -- 172.21.15.39:0/3639902271 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8ccc109df0 msgr2=0x7f8ccc076190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:55:08.688 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:08.683+0000 7f8cc9d9b640 1 --2- 172.21.15.39:0/3639902271 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8ccc109df0 0x7f8ccc076190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:08.688 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:08.683+0000 7f8cc9d9b640 1 -- 172.21.15.39:0/3639902271 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ccc1a8670 con 0x7f8ccc1094a0 2024-09-21T03:55:08.688 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:08.683+0000 7f8ccad9d640 1 --2- 172.21.15.39:0/3639902271 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8ccc109df0 0x7f8ccc076190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:55:08.688 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:08.683+0000 7f8cc9d9b640 1 --2- 172.21.15.39:0/3639902271 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8ccc1094a0 0x7f8ccc075c50 secure :-1 s=READY pgs=1189 cs=0 l=1 rev1=1 crypto rx=0x7f8cc000c8c0 tx=0x7f8cc000cd80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:55:08.688 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:08.684+0000 7f8cbb7fe640 1 -- 172.21.15.39:0/3639902271 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cc0007bb0 con 0x7f8ccc1094a0 2024-09-21T03:55:08.688 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:08.684+0000 7f8cd0cd4640 1 -- 172.21.15.39:0/3639902271 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8ccc1a8960 con 0x7f8ccc1094a0 2024-09-21T03:55:08.689 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:08.684+0000 7f8cbb7fe640 1 -- 172.21.15.39:0/3639902271 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8cc0007d50 con 0x7f8ccc1094a0 2024-09-21T03:55:08.689 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:08.684+0000 7f8cbb7fe640 1 -- 172.21.15.39:0/3639902271 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cc00187c0 con 0x7f8ccc1094a0 2024-09-21T03:55:08.689 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:08.684+0000 7f8cd0cd4640 1 -- 172.21.15.39:0/3639902271 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8ccc1a8ef0 con 0x7f8ccc1094a0 2024-09-21T03:55:08.690 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:08.686+0000 7f8cd7248740 1 Processor -- start 2024-09-21T03:55:08.690 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:08.687+0000 7f8cd7248740 1 -- start start 2024-09-21T03:55:08.690 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:08.687+0000 7f8cbb7fe640 1 -- 172.21.15.39:0/3639902271 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 27) ==== 154643+0+0 (secure 0 0 0) 0x7f8cc000fa20 con 0x7f8ccc1094a0 2024-09-21T03:55:08.690 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:08.687+0000 7f8cd7248740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5565a39bf070 0x5565a3d666a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:55:08.690 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:08.687+0000 7f8cd7248740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5565a39961b0 0x5565a3d66be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:55:08.690 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:08.687+0000 7f8cd7248740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5565a399e390 0x5565a3d67120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:55:08.691 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:08.687+0000 7f8cd7248740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x5565a3cdcea0 con 0x5565a39bf070 2024-09-21T03:55:08.691 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:08.687+0000 7f8cd7248740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x5565a3ccc2d0 con 0x5565a399e390 2024-09-21T03:55:08.691 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:08.687+0000 7f8cd7248740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x5565a3cd00e0 con 0x5565a39961b0 2024-09-21T03:55:08.691 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:08.687+0000 7f8ccad9d640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5565a39bf070 0x5565a3d666a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:55:08.691 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:08.687+0000 7f8ccad9d640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5565a39bf070 0x5565a3d666a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:39200/0 (socket says 172.21.15.39:39200) 2024-09-21T03:55:08.691 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:08.687+0000 7f8ccad9d640 1 -- 172.21.15.39:0/3441473524 learned_addr learned my addr 172.21.15.39:0/3441473524 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:55:08.691 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:08.687+0000 7f8cca59c640 1 --2- 172.21.15.39:0/3441473524 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5565a39961b0 0x5565a3d66be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:55:08.691 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:08.687+0000 7f8cc9d9b640 1 --2- 172.21.15.39:0/3441473524 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5565a399e390 0x5565a3d67120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:55:08.694 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:08.687+0000 7f8ccad9d640 1 -- 172.21.15.39:0/3441473524 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5565a39961b0 msgr2=0x5565a3d66be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:55:08.694 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:08.687+0000 7f8ccad9d640 1 --2- 172.21.15.39:0/3441473524 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5565a39961b0 0x5565a3d66be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:08.694 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:08.687+0000 7f8ccad9d640 1 -- 172.21.15.39:0/3441473524 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5565a399e390 msgr2=0x5565a3d67120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:55:08.694 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:08.687+0000 7f8ccad9d640 1 --2- 172.21.15.39:0/3441473524 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5565a399e390 0x5565a3d67120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:08.694 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:08.687+0000 7f8ccad9d640 1 -- 172.21.15.39:0/3441473524 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5565a3a2a650 con 0x5565a39bf070 2024-09-21T03:55:08.694 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:08.687+0000 7f8cc9d9b640 1 --2- 172.21.15.39:0/3441473524 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5565a399e390 0x5565a3d67120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:55:08.694 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:08.688+0000 7f8ccad9d640 1 --2- 172.21.15.39:0/3441473524 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5565a39bf070 0x5565a3d666a0 secure :-1 s=READY pgs=1190 cs=0 l=1 rev1=1 crypto rx=0x7f8cc4007e60 tx=0x7f8cc400e720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d8c34c5d188b18b7 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:55:08.694 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:08.688+0000 7f8cbb7fe640 1 --2- 172.21.15.39:0/3639902271 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f8cb40b6680 0x7f8cb40b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:55:08.694 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:08.688+0000 7f8cca59c640 1 --2- 172.21.15.39:0/3639902271 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f8cb40b6680 0x7f8cb40b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:55:08.695 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:08.688+0000 7f8cb8ff9640 1 -- 172.21.15.39:0/3441473524 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cc4014070 con 0x5565a39bf070 2024-09-21T03:55:08.695 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:08.688+0000 7f8cbb7fe640 1 -- 172.21.15.39:0/3639902271 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8528+0+0 (secure 0 0 0) 0x7f8cc00e7820 con 0x7f8ccc1094a0 2024-09-21T03:55:08.695 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:08.688+0000 7f8cb8ff9640 1 -- 172.21.15.39:0/3441473524 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8cc4004d20 con 0x5565a39bf070 2024-09-21T03:55:08.695 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:08.688+0000 7f8cb8ff9640 1 -- 172.21.15.39:0/3441473524 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cc4005380 con 0x5565a39bf070 2024-09-21T03:55:08.695 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:08.689+0000 7f8cd7248740 1 -- 172.21.15.39:0/3441473524 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5565a39bf070 msgr2=0x5565a3d666a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:55:08.695 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:08.689+0000 7f8cca59c640 1 --2- 172.21.15.39:0/3639902271 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f8cb40b6680 0x7f8cb40b8b90 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f8cbc000c00 tx=0x7f8cbc031400 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:55:08.695 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:08.689+0000 7f8cd7248740 1 --2- 172.21.15.39:0/3441473524 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5565a39bf070 0x5565a3d666a0 secure :-1 s=READY pgs=1190 cs=0 l=1 rev1=1 crypto rx=0x7f8cc4007e60 tx=0x7f8cc400e720 comp rx=0 tx=0).stop 2024-09-21T03:55:08.695 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:08.689+0000 7f8cd7248740 1 -- 172.21.15.39:0/3441473524 shutdown_connections 2024-09-21T03:55:08.695 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:08.689+0000 7f8cd7248740 1 --2- 172.21.15.39:0/3441473524 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5565a399e390 0x5565a3d67120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:08.695 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:08.689+0000 7f8cd7248740 1 --2- 172.21.15.39:0/3441473524 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5565a39961b0 0x5565a3d66be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:08.695 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:08.689+0000 7f8cd7248740 1 --2- 172.21.15.39:0/3441473524 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5565a39bf070 0x5565a3d666a0 unknown :-1 s=CLOSED pgs=1190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:08.695 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:08.689+0000 7f8cd7248740 1 -- 172.21.15.39:0/3441473524 >> 172.21.15.39:0/3441473524 conn(0x5565a39b3460 msgr2=0x5565a3d64590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:55:08.695 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:08.689+0000 7f8cd7248740 1 -- 172.21.15.39:0/3441473524 shutdown_connections 2024-09-21T03:55:08.695 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:08.689+0000 7f8cd7248740 1 -- 172.21.15.39:0/3441473524 wait complete. 2024-09-21T03:55:08.696 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:08.689+0000 7f8cd7248740 1 -- start start 2024-09-21T03:55:08.696 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:08.690+0000 7f8cd7248740 1 Processor -- start 2024-09-21T03:55:08.696 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:08.690+0000 7f8cd7248740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5565a39961b0 0x5565a3d71cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:55:08.696 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:08.690+0000 7f8cd7248740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5565a399e390 0x5565a3d72230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:55:08.696 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:08.690+0000 7f8cd7248740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5565a39bf070 0x5565a3d7e1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:55:08.696 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:08.690+0000 7f8cd7248740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x5565a3b2d700 con 0x5565a39bf070 2024-09-21T03:55:08.696 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:08.690+0000 7f8cd7248740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x5565a3c67160 con 0x5565a39961b0 2024-09-21T03:55:08.696 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:08.690+0000 7f8cd7248740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x5565a3d74ed0 con 0x5565a399e390 2024-09-21T03:55:08.696 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:08.690+0000 7f8cd7248740 10 client.0 _ll_register_callbacks cb 0x5565a3c809e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:55:08.696 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:08.690+0000 7f8ccad9d640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5565a39bf070 0x5565a3d7e1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:55:08.696 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:08.690+0000 7f8ccad9d640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5565a39bf070 0x5565a3d7e1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:39208/0 (socket says 172.21.15.39:39208) 2024-09-21T03:55:08.696 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:08.690+0000 7f8ccad9d640 1 -- 172.21.15.39:0/3187567647 learned_addr learned my addr 172.21.15.39:0/3187567647 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:55:08.696 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:08.690+0000 7f8cca59c640 1 --2- 172.21.15.39:0/3187567647 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5565a39961b0 0x5565a3d71cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:55:08.696 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:08.690+0000 7f8cc9d9b640 1 --2- 172.21.15.39:0/3187567647 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5565a399e390 0x5565a3d72230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:55:08.697 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:08.690+0000 7f8ccad9d640 1 -- 172.21.15.39:0/3187567647 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5565a399e390 msgr2=0x5565a3d72230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:55:08.697 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:08.690+0000 7f8ccad9d640 1 --2- 172.21.15.39:0/3187567647 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5565a399e390 0x5565a3d72230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:08.697 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:08.690+0000 7f8ccad9d640 1 -- 172.21.15.39:0/3187567647 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5565a39961b0 msgr2=0x5565a3d71cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:55:08.697 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:08.690+0000 7f8ccad9d640 1 --2- 172.21.15.39:0/3187567647 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5565a39961b0 0x5565a3d71cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:08.697 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:08.690+0000 7f8ccad9d640 1 -- 172.21.15.39:0/3187567647 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5565a3c94c90 con 0x5565a39bf070 2024-09-21T03:55:08.697 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:08.691+0000 7f8ccad9d640 1 --2- 172.21.15.39:0/3187567647 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5565a39bf070 0x5565a3d7e1c0 secure :-1 s=READY pgs=1191 cs=0 l=1 rev1=1 crypto rx=0x7f8cc4000f80 tx=0x7f8cc4007bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=196174ebb500ebe8 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:55:08.697 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:08.691+0000 7f8cb1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-09-21T03:55:08.697 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:08.691+0000 7f8cb1ffb640 1 -- 172.21.15.39:0/3187567647 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cc4014030 con 0x5565a39bf070 2024-09-21T03:55:08.697 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:08.691+0000 7f8cb1ffb640 1 -- 172.21.15.39:0/3187567647 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8528+0+0 (secure 0 0 0) 0x7f8cc4003970 con 0x5565a39bf070 2024-09-21T03:55:08.697 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:08.691+0000 7f8cd7248740 1 -- 172.21.15.39:0/3187567647 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8cc4004d20 con 0x5565a39bf070 2024-09-21T03:55:08.697 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:08.691+0000 7f8cb1ffb640 1 -- 172.21.15.39:0/3187567647 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f8cc4004ed0 con 0x5565a39bf070 2024-09-21T03:55:08.697 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:08.691+0000 7f8cd7248740 10 client.17874 fetch_fsmap learned FSMap version 29 2024-09-21T03:55:08.697 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:08.691+0000 7f8cd7248740 1 -- 172.21.15.39:0/3187567647 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5565a3d80b80 con 0x5565a39bf070 2024-09-21T03:55:08.698 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:08.691+0000 7f8cb1ffb640 1 -- 172.21.15.39:0/3187567647 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8cc4004d20 con 0x5565a39bf070 2024-09-21T03:55:08.699 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:08.692+0000 7f8cb1ffb640 1 -- 172.21.15.39:0/3187567647 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f8cc4004620 con 0x5565a39bf070 2024-09-21T03:55:08.699 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:08.692+0000 7f8cd7248740 10 client.17874 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T03:55:08.699 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:08.692+0000 7f8cd7248740 10 client.17874 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.a 2024-09-21T03:55:08.700 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:08.692+0000 7f8cd7248740 20 client.17874 populate_metadata read hostname 'smithi039' 2024-09-21T03:55:08.700 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:08.692+0000 7f8cd7248740 1 --2- 172.21.15.39:0/3187567647 >> [v2:172.21.15.39:6840/4247190456,v1:172.21.15.39:6841/4247190456] conn(0x5565a397d9f0 0x5565a3d83f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:55:08.700 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:08.692+0000 7f8cd7248740 4 client.17874 mds_command: new command op to 14868 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:55:08.700 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:08.692+0000 7f8cd7248740 1 -- 172.21.15.39:0/3187567647 --> [v2:172.21.15.39:6840/4247190456,v1:172.21.15.39:6841/4247190456] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5565a3cdedf0 con 0x5565a397d9f0 2024-09-21T03:55:08.700 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:08.692+0000 7f8cca59c640 1 --2- 172.21.15.39:0/3187567647 >> [v2:172.21.15.39:6840/4247190456,v1:172.21.15.39:6841/4247190456] conn(0x5565a397d9f0 0x5565a3d83f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:55:08.700 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:08.692+0000 7f8cb1ffb640 10 client.17874 ms_handle_connect on v2:172.21.15.39:6840/4247190456 2024-09-21T03:55:08.700 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:08.692+0000 7f8cca59c640 1 --2- 172.21.15.39:0/3187567647 >> [v2:172.21.15.39:6840/4247190456,v1:172.21.15.39:6841/4247190456] conn(0x5565a397d9f0 0x5565a3d83f90 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=82a506ffb64b15e8 server_cookie=a04e8a6b078b758e in_seq=0 out_seq=0 2024-09-21T03:55:08.700 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:08.693+0000 7f8cb1ffb640 1 -- 172.21.15.39:0/3187567647 <== mds.4 v2:172.21.15.39:6840/4247190456 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5565a3cdedf0 con 0x5565a397d9f0 2024-09-21T03:55:08.700 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:08.693+0000 7f8cb1ffb640 10 client.17874 handle_command_reply: tid=0 2024-09-21T03:55:08.700 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:08.693+0000 7f8cd7248740 1 client.17874 shutdown 2024-09-21T03:55:08.700 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:08.693+0000 7f8cd7248740 1 -- 172.21.15.39:0/3187567647 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5565a39bf070 msgr2=0x5565a3d7e1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:55:08.700 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:08.693+0000 7f8cd7248740 1 --2- 172.21.15.39:0/3187567647 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5565a39bf070 0x5565a3d7e1c0 secure :-1 s=READY pgs=1191 cs=0 l=1 rev1=1 crypto rx=0x7f8cc4000f80 tx=0x7f8cc4007bd0 comp rx=0 tx=0).stop 2024-09-21T03:55:08.700 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:08.693+0000 7f8cd7248740 1 -- 172.21.15.39:0/3187567647 shutdown_connections 2024-09-21T03:55:08.701 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:08.693+0000 7f8cd7248740 1 --2- 172.21.15.39:0/3187567647 >> [v2:172.21.15.39:6840/4247190456,v1:172.21.15.39:6841/4247190456] conn(0x5565a397d9f0 0x5565a3d83f90 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:08.701 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:08.693+0000 7f8cd7248740 1 --2- 172.21.15.39:0/3187567647 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5565a39bf070 0x5565a3d7e1c0 unknown :-1 s=CLOSED pgs=1191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:08.701 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:08.693+0000 7f8cd7248740 1 --2- 172.21.15.39:0/3187567647 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5565a399e390 0x5565a3d72230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:08.701 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:08.693+0000 7f8cb1ffb640 0 client.17874 ms_handle_reset on v2:172.21.15.39:6840/4247190456 2024-09-21T03:55:08.701 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:08.693+0000 7f8cd7248740 1 --2- 172.21.15.39:0/3187567647 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5565a39961b0 0x5565a3d71cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:08.701 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:08.693+0000 7f8cd7248740 1 -- 172.21.15.39:0/3187567647 >> 172.21.15.39:0/3187567647 conn(0x5565a39b3460 msgr2=0x5565a3d676d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:55:08.702 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:08.693+0000 7f8cd7248740 1 -- 172.21.15.39:0/3187567647 shutdown_connections 2024-09-21T03:55:08.702 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:08.693+0000 7f8cd7248740 1 -- 172.21.15.39:0/3187567647 wait complete. 2024-09-21T03:55:08.702 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:08.693+0000 7f8cd7248740 20 client.17874 trim_cache size 0 max 16384 2024-09-21T03:55:08.709 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:08.706+0000 7f8cd7248740 1 Processor -- start 2024-09-21T03:55:08.709 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:08.706+0000 7f8cd7248740 1 -- start start 2024-09-21T03:55:08.710 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:08.707+0000 7f8cd7248740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5565a39961b0 0x5565a3d721b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:55:08.711 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:08.707+0000 7f8cd7248740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5565a399e390 0x5565a3d7e0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:55:08.711 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:08.707+0000 7f8cd7248740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5565a39bf070 0x5565a3d6eee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:55:08.711 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:08.707+0000 7f8cd7248740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f8ccc1b18e0 con 0x5565a39bf070 2024-09-21T03:55:08.711 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:08.707+0000 7f8cd7248740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f8ccc1b3420 con 0x5565a39961b0 2024-09-21T03:55:08.711 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:08.707+0000 7f8cd7248740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f8ccc1b1fb0 con 0x5565a399e390 2024-09-21T03:55:08.711 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:08.707+0000 7f8ccad9d640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5565a39bf070 0x5565a3d6eee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:55:08.711 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:08.707+0000 7f8ccad9d640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5565a39bf070 0x5565a3d6eee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:39224/0 (socket says 172.21.15.39:39224) 2024-09-21T03:55:08.711 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:08.707+0000 7f8ccad9d640 1 -- 172.21.15.39:0/1327661143 learned_addr learned my addr 172.21.15.39:0/1327661143 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:55:08.711 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:08.707+0000 7f8cca59c640 1 --2- 172.21.15.39:0/1327661143 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5565a39961b0 0x5565a3d721b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:55:08.712 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:08.707+0000 7f8cc9d9b640 1 --2- 172.21.15.39:0/1327661143 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5565a399e390 0x5565a3d7e0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:55:08.712 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:08.707+0000 7f8ccad9d640 1 -- 172.21.15.39:0/1327661143 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5565a399e390 msgr2=0x5565a3d7e0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:55:08.712 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:08.707+0000 7f8ccad9d640 1 --2- 172.21.15.39:0/1327661143 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5565a399e390 0x5565a3d7e0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:08.712 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:08.707+0000 7f8ccad9d640 1 -- 172.21.15.39:0/1327661143 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5565a39961b0 msgr2=0x5565a3d721b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:55:08.712 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:08.707+0000 7f8ccad9d640 1 --2- 172.21.15.39:0/1327661143 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5565a39961b0 0x5565a3d721b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:08.712 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:08.707+0000 7f8ccad9d640 1 -- 172.21.15.39:0/1327661143 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5565a3d674c0 con 0x5565a39bf070 2024-09-21T03:55:08.712 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:08.707+0000 7f8ccad9d640 1 --2- 172.21.15.39:0/1327661143 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5565a39bf070 0x5565a3d6eee0 secure :-1 s=READY pgs=1192 cs=0 l=1 rev1=1 crypto rx=0x7f8cc4005150 tx=0x7f8cc400e720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=72ce6b475092572d server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:55:08.712 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:08.708+0000 7f8cb3fff640 1 -- 172.21.15.39:0/1327661143 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cc4014070 con 0x5565a39bf070 2024-09-21T03:55:08.712 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:08.708+0000 7f8cb3fff640 1 -- 172.21.15.39:0/1327661143 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8cc401c020 con 0x5565a39bf070 2024-09-21T03:55:08.712 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:08.708+0000 7f8cb3fff640 1 -- 172.21.15.39:0/1327661143 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cc40055e0 con 0x5565a39bf070 2024-09-21T03:55:08.716 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:08.708+0000 7f8cd7248740 1 -- 172.21.15.39:0/1327661143 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5565a39bf070 msgr2=0x5565a3d6eee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:55:08.716 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:08.708+0000 7f8cd7248740 1 --2- 172.21.15.39:0/1327661143 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5565a39bf070 0x5565a3d6eee0 secure :-1 s=READY pgs=1192 cs=0 l=1 rev1=1 crypto rx=0x7f8cc4005150 tx=0x7f8cc400e720 comp rx=0 tx=0).stop 2024-09-21T03:55:08.716 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:08.708+0000 7f8cd7248740 1 -- 172.21.15.39:0/1327661143 shutdown_connections 2024-09-21T03:55:08.716 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:08.708+0000 7f8cd7248740 1 --2- 172.21.15.39:0/1327661143 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5565a39bf070 0x5565a3d6eee0 unknown :-1 s=CLOSED pgs=1192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:08.717 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:08.708+0000 7f8cd7248740 1 --2- 172.21.15.39:0/1327661143 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5565a399e390 0x5565a3d7e0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:08.717 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:08.708+0000 7f8cd7248740 1 --2- 172.21.15.39:0/1327661143 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5565a39961b0 0x5565a3d721b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:08.717 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:08.708+0000 7f8cd7248740 1 -- 172.21.15.39:0/1327661143 >> 172.21.15.39:0/1327661143 conn(0x5565a39b3460 msgr2=0x5565a3d71aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:55:08.717 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:08.709+0000 7f8cd7248740 1 -- 172.21.15.39:0/1327661143 shutdown_connections 2024-09-21T03:55:08.717 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:08.709+0000 7f8cd7248740 1 -- 172.21.15.39:0/1327661143 wait complete. 2024-09-21T03:55:08.717 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:08.709+0000 7f8cd7248740 1 -- start start 2024-09-21T03:55:08.717 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:08.709+0000 7f8cd7248740 1 Processor -- start 2024-09-21T03:55:08.717 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:08.709+0000 7f8cd7248740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5565a39961b0 0x5565a3d7a940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:55:08.717 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:08.709+0000 7f8cd7248740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5565a399e390 0x5565a3d75e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:55:08.717 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:08.709+0000 7f8cd7248740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5565a39bf070 0x5565a3d763c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:55:08.718 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:08.709+0000 7f8cd7248740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f8ccc1b2d50 con 0x5565a399e390 2024-09-21T03:55:08.718 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:08.709+0000 7f8cd7248740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f8ccc1b2680 con 0x5565a39961b0 2024-09-21T03:55:08.718 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:08.709+0000 7f8cd7248740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f8ccc1af8c0 con 0x5565a39bf070 2024-09-21T03:55:08.718 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:08.709+0000 7f8cd7248740 10 client.0 _ll_register_callbacks cb 0x7f8cbc009560 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:55:08.718 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:08.709+0000 7f8cc9d9b640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5565a399e390 0x5565a3d75e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:55:08.718 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:08.709+0000 7f8cc9d9b640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5565a399e390 0x5565a3d75e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:39234/0 (socket says 172.21.15.39:39234) 2024-09-21T03:55:08.718 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:08.709+0000 7f8cc9d9b640 1 -- 172.21.15.39:0/2027899890 learned_addr learned my addr 172.21.15.39:0/2027899890 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:55:08.718 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:08.710+0000 7f8ccad9d640 1 --2- 172.21.15.39:0/2027899890 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5565a39bf070 0x5565a3d763c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:55:08.718 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:08.710+0000 7f8cca59c640 1 --2- 172.21.15.39:0/2027899890 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5565a39961b0 0x5565a3d7a940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:55:08.718 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:08.710+0000 7f8cc9d9b640 1 -- 172.21.15.39:0/2027899890 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5565a39bf070 msgr2=0x5565a3d763c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:55:08.718 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:08.710+0000 7f8cc9d9b640 1 --2- 172.21.15.39:0/2027899890 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5565a39bf070 0x5565a3d763c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:08.719 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:08.710+0000 7f8cc9d9b640 1 -- 172.21.15.39:0/2027899890 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5565a39961b0 msgr2=0x5565a3d7a940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:55:08.719 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:08.710+0000 7f8cc9d9b640 1 --2- 172.21.15.39:0/2027899890 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5565a39961b0 0x5565a3d7a940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:08.719 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:08.710+0000 7f8cc9d9b640 1 -- 172.21.15.39:0/2027899890 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5565a3d76b50 con 0x5565a399e390 2024-09-21T03:55:08.719 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:08.710+0000 7f8cca59c640 1 --2- 172.21.15.39:0/2027899890 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5565a39961b0 0x5565a3d7a940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:55:08.719 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:08.710+0000 7f8cc9d9b640 1 --2- 172.21.15.39:0/2027899890 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5565a399e390 0x5565a3d75e40 secure :-1 s=READY pgs=1193 cs=0 l=1 rev1=1 crypto rx=0x7f8cc00e4fd0 tx=0x7f8cc00b6f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be067074396aef49 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:55:08.719 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:08.710+0000 7f8cb1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-09-21T03:55:08.719 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:08.710+0000 7f8cb1ffb640 1 -- 172.21.15.39:0/2027899890 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cc00acfc0 con 0x5565a399e390 2024-09-21T03:55:08.719 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:08.710+0000 7f8cb1ffb640 1 -- 172.21.15.39:0/2027899890 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8528+0+0 (secure 0 0 0) 0x7f8cc00a8fc0 con 0x5565a399e390 2024-09-21T03:55:08.719 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:08.710+0000 7f8cd7248740 1 -- 172.21.15.39:0/2027899890 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8cc401c020 con 0x5565a399e390 2024-09-21T03:55:08.719 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:08.710+0000 7f8cb1ffb640 1 -- 172.21.15.39:0/2027899890 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f8cc00ed020 con 0x5565a399e390 2024-09-21T03:55:08.719 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:08.710+0000 7f8cd7248740 10 client.17880 fetch_fsmap learned FSMap version 29 2024-09-21T03:55:08.719 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:08.710+0000 7f8cd7248740 1 -- 172.21.15.39:0/2027899890 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5565a3dee770 con 0x5565a399e390 2024-09-21T03:55:08.720 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:08.711+0000 7f8cb1ffb640 1 -- 172.21.15.39:0/2027899890 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8cc401c020 con 0x5565a399e390 2024-09-21T03:55:08.720 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:08.711+0000 7f8cb1ffb640 1 -- 172.21.15.39:0/2027899890 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f8cc00a91c0 con 0x5565a399e390 2024-09-21T03:55:08.720 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:08.711+0000 7f8cd7248740 10 client.17880 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T03:55:08.720 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:08.711+0000 7f8cd7248740 10 client.17880 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.a 2024-09-21T03:55:08.720 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:08.711+0000 7f8cd7248740 20 client.17880 populate_metadata read hostname 'smithi039' 2024-09-21T03:55:08.720 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:08.711+0000 7f8cd7248740 1 --2- 172.21.15.39:0/2027899890 >> [v2:172.21.15.39:6840/4247190456,v1:172.21.15.39:6841/4247190456] conn(0x5565a3952da0 0x5565a3df0ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:55:08.720 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:08.711+0000 7f8cd7248740 4 client.17880 mds_command: new command op to 14868 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T03:55:08.720 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:08.711+0000 7f8cd7248740 1 -- 172.21.15.39:0/2027899890 --> [v2:172.21.15.39:6840/4247190456,v1:172.21.15.39:6841/4247190456] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5565a3cda8a0 con 0x5565a3952da0 2024-09-21T03:55:08.720 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:08.711+0000 7f8cca59c640 1 --2- 172.21.15.39:0/2027899890 >> [v2:172.21.15.39:6840/4247190456,v1:172.21.15.39:6841/4247190456] conn(0x5565a3952da0 0x5565a3df0ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:55:08.720 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:08.711+0000 7f8cb1ffb640 10 client.17880 ms_handle_connect on v2:172.21.15.39:6840/4247190456 2024-09-21T03:55:08.721 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:08.711+0000 7f8cca59c640 1 --2- 172.21.15.39:0/2027899890 >> [v2:172.21.15.39:6840/4247190456,v1:172.21.15.39:6841/4247190456] conn(0x5565a3952da0 0x5565a3df0ed0 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=d5bb6214ab6198a server_cookie=f60fdca6be830bcf in_seq=0 out_seq=0 2024-09-21T03:55:08.721 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:08.712+0000 7f8cb1ffb640 1 -- 172.21.15.39:0/2027899890 <== mds.4 v2:172.21.15.39:6840/4247190456 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5565a3cda8a0 con 0x5565a3952da0 2024-09-21T03:55:08.721 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:08.712+0000 7f8cb1ffb640 10 client.17880 handle_command_reply: tid=0 2024-09-21T03:55:08.721 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:08.712+0000 7f8cd7248740 1 client.17880 shutdown 2024-09-21T03:55:08.721 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:08.712+0000 7f8cd7248740 1 -- 172.21.15.39:0/2027899890 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5565a399e390 msgr2=0x5565a3d75e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:55:08.721 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:08.712+0000 7f8cd7248740 1 --2- 172.21.15.39:0/2027899890 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5565a399e390 0x5565a3d75e40 secure :-1 s=READY pgs=1193 cs=0 l=1 rev1=1 crypto rx=0x7f8cc00e4fd0 tx=0x7f8cc00b6f40 comp rx=0 tx=0).stop 2024-09-21T03:55:08.721 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:08.712+0000 7f8cd7248740 1 -- 172.21.15.39:0/2027899890 shutdown_connections 2024-09-21T03:55:08.721 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:08.712+0000 7f8cd7248740 1 --2- 172.21.15.39:0/2027899890 >> [v2:172.21.15.39:6840/4247190456,v1:172.21.15.39:6841/4247190456] conn(0x5565a3952da0 0x5565a3df0ed0 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:08.721 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:08.712+0000 7f8cd7248740 1 --2- 172.21.15.39:0/2027899890 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5565a39bf070 0x5565a3d763c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:08.721 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:08.712+0000 7f8cd7248740 1 --2- 172.21.15.39:0/2027899890 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5565a399e390 0x5565a3d75e40 unknown :-1 s=CLOSED pgs=1193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:08.721 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:08.712+0000 7f8cd7248740 1 --2- 172.21.15.39:0/2027899890 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5565a39961b0 0x5565a3d7a940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:08.722 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:08.712+0000 7f8cd7248740 1 -- 172.21.15.39:0/2027899890 >> 172.21.15.39:0/2027899890 conn(0x5565a39b3460 msgr2=0x5565a3d731a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:55:08.722 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:08.712+0000 7f8cb1ffb640 0 client.17880 ms_handle_reset on v2:172.21.15.39:6840/4247190456 2024-09-21T03:55:08.722 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:08.712+0000 7f8cd7248740 1 -- 172.21.15.39:0/2027899890 shutdown_connections 2024-09-21T03:55:08.722 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:08.712+0000 7f8cd7248740 1 -- 172.21.15.39:0/2027899890 wait complete. 2024-09-21T03:55:08.722 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:08.712+0000 7f8cd7248740 20 client.17880 trim_cache size 0 max 16384 2024-09-21T03:55:08.722 INFO:teuthology.orchestra.run.smithi039.stdout:[] 2024-09-21T03:55:08.723 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:08.712+0000 7f8cd0cd4640 1 -- 172.21.15.39:0/3639902271 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f8cb40b6680 msgr2=0x7f8cb40b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:55:08.723 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:08.712+0000 7f8cd0cd4640 1 --2- 172.21.15.39:0/3639902271 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f8cb40b6680 0x7f8cb40b8b90 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f8cbc000c00 tx=0x7f8cbc031400 comp rx=0 tx=0).stop 2024-09-21T03:55:08.723 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:08.712+0000 7f8cd0cd4640 1 -- 172.21.15.39:0/3639902271 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8ccc1094a0 msgr2=0x7f8ccc075c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:55:08.723 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:08.713+0000 7f8cd0cd4640 1 --2- 172.21.15.39:0/3639902271 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8ccc1094a0 0x7f8ccc075c50 secure :-1 s=READY pgs=1189 cs=0 l=1 rev1=1 crypto rx=0x7f8cc000c8c0 tx=0x7f8cc000cd80 comp rx=0 tx=0).stop 2024-09-21T03:55:08.723 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:08.713+0000 7f8cd0cd4640 1 -- 172.21.15.39:0/3639902271 shutdown_connections 2024-09-21T03:55:08.724 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:08.713+0000 7f8cd0cd4640 1 --2- 172.21.15.39:0/3639902271 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f8cb40b6680 0x7f8cb40b8b90 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:08.724 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:08.713+0000 7f8cd0cd4640 1 --2- 172.21.15.39:0/3639902271 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8ccc109df0 0x7f8ccc076190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:08.724 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:08.713+0000 7f8cd0cd4640 1 --2- 172.21.15.39:0/3639902271 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8ccc1094a0 0x7f8ccc075c50 unknown :-1 s=CLOSED pgs=1189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:08.724 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:08.713+0000 7f8cd0cd4640 1 --2- 172.21.15.39:0/3639902271 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8ccc103b60 0x7f8ccc075710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:08.724 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:08.713+0000 7f8cd0cd4640 1 -- 172.21.15.39:0/3639902271 >> 172.21.15.39:0/3639902271 conn(0x7f8ccc07b850 msgr2=0x7f8ccc107590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:55:08.724 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:08.713+0000 7f8cd0cd4640 1 -- 172.21.15.39:0/3639902271 shutdown_connections 2024-09-21T03:55:08.724 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:08.713+0000 7f8cd0cd4640 1 -- 172.21.15.39:0/3639902271 wait complete. 2024-09-21T03:55:09.334 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:55:09 smithi039 ceph-mon[31211]: pgmap v602: 97 pgs: 97 active+clean; 10 MiB data, 525 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 44 KiB/s wr, 25 op/s 2024-09-21T03:55:09.468 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:55:09 smithi161 ceph-mon[34533]: pgmap v602: 97 pgs: 97 active+clean; 10 MiB data, 525 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 44 KiB/s wr, 25 op/s 2024-09-21T03:55:09.475 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:55:09 smithi150 ceph-mon[34289]: pgmap v602: 97 pgs: 97 active+clean; 10 MiB data, 525 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 44 KiB/s wr, 25 op/s 2024-09-21T03:55:09.577 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 51116d5c-77ca-11ef-baf5-efdc52797490 -- ceph tell mds.1:3 damage ls 2024-09-21T03:55:09.837 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/51116d5c-77ca-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:55:10.426 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:10.392+0000 7f88bd5b1640 1 -- 172.21.15.39:0/9812530 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f88b8103b80 msgr2=0x7f88b8103f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:55:10.426 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:10.392+0000 7f88bd5b1640 1 --2- 172.21.15.39:0/9812530 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f88b8103b80 0x7f88b8103f90 secure :-1 s=READY pgs=1194 cs=0 l=1 rev1=1 crypto rx=0x7f88a8009900 tx=0x7f88a802a3f0 comp rx=0 tx=0).stop 2024-09-21T03:55:10.426 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:10.392+0000 7f88bd5b1640 1 -- 172.21.15.39:0/9812530 shutdown_connections 2024-09-21T03:55:10.426 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:10.392+0000 7f88bd5b1640 1 --2- 172.21.15.39:0/9812530 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f88b8109e10 0x7f88b8110640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:10.426 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:10.392+0000 7f88bd5b1640 1 --2- 172.21.15.39:0/9812530 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f88b81094c0 0x7f88b81098d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:10.426 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:10.392+0000 7f88bd5b1640 1 --2- 172.21.15.39:0/9812530 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f88b8103b80 0x7f88b8103f90 unknown :-1 s=CLOSED pgs=1194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:10.426 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:10.392+0000 7f88bd5b1640 1 -- 172.21.15.39:0/9812530 >> 172.21.15.39:0/9812530 conn(0x7f88b80ff590 msgr2=0x7f88b81019e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:55:10.426 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:10.392+0000 7f88bd5b1640 1 -- 172.21.15.39:0/9812530 shutdown_connections 2024-09-21T03:55:10.427 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:10.392+0000 7f88bd5b1640 1 -- 172.21.15.39:0/9812530 wait complete. 2024-09-21T03:55:10.427 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:10.393+0000 7f88bd5b1640 1 Processor -- start 2024-09-21T03:55:10.427 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:10.393+0000 7f88bd5b1640 1 -- start start 2024-09-21T03:55:10.427 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:10.393+0000 7f88bd5b1640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f88b8103b80 0x7f88b819a560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:55:10.427 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:10.393+0000 7f88bd5b1640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f88b81094c0 0x7f88b819ab00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:55:10.427 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:10.393+0000 7f88bd5b1640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f88b8109e10 0x7f88b819fda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:55:10.427 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:10.393+0000 7f88bd5b1640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f88b8113ad0 con 0x7f88b8109e10 2024-09-21T03:55:10.427 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:10.393+0000 7f88bd5b1640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f88b8113950 con 0x7f88b81094c0 2024-09-21T03:55:10.427 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:10.393+0000 7f88bd5b1640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f88b8113c50 con 0x7f88b8103b80 2024-09-21T03:55:10.427 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:10.393+0000 7f88b77fe640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f88b8109e10 0x7f88b819fda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:55:10.427 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:10.393+0000 7f88b77fe640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f88b8109e10 0x7f88b819fda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:39264/0 (socket says 172.21.15.39:39264) 2024-09-21T03:55:10.427 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:10.393+0000 7f88b77fe640 1 -- 172.21.15.39:0/2965648696 learned_addr learned my addr 172.21.15.39:0/2965648696 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:55:10.427 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:10.393+0000 7f88b6ffd640 1 --2- 172.21.15.39:0/2965648696 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f88b8103b80 0x7f88b819a560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:55:10.427 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:10.393+0000 7f88ae7fc640 1 --2- 172.21.15.39:0/2965648696 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f88b81094c0 0x7f88b819ab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:55:10.428 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:10.394+0000 7f88b77fe640 1 -- 172.21.15.39:0/2965648696 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f88b8103b80 msgr2=0x7f88b819a560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:55:10.428 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:10.394+0000 7f88b77fe640 1 --2- 172.21.15.39:0/2965648696 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f88b8103b80 0x7f88b819a560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:10.428 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:10.394+0000 7f88b77fe640 1 -- 172.21.15.39:0/2965648696 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f88b81094c0 msgr2=0x7f88b819ab00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:55:10.428 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:10.394+0000 7f88b77fe640 1 --2- 172.21.15.39:0/2965648696 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f88b81094c0 0x7f88b819ab00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:10.428 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:10.394+0000 7f88b77fe640 1 -- 172.21.15.39:0/2965648696 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f88b81a0410 con 0x7f88b8109e10 2024-09-21T03:55:10.428 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:10.394+0000 7f88b77fe640 1 --2- 172.21.15.39:0/2965648696 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f88b8109e10 0x7f88b819fda0 secure :-1 s=READY pgs=1195 cs=0 l=1 rev1=1 crypto rx=0x7f88b000cce0 tx=0x7f88b000a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:55:10.428 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:10.394+0000 7f88b4ff9640 1 -- 172.21.15.39:0/2965648696 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88b000f450 con 0x7f88b8109e10 2024-09-21T03:55:10.428 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:10.394+0000 7f88b4ff9640 1 -- 172.21.15.39:0/2965648696 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f88b000faa0 con 0x7f88b8109e10 2024-09-21T03:55:10.428 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:10.394+0000 7f88bd5b1640 1 -- 172.21.15.39:0/2965648696 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f88b81a0700 con 0x7f88b8109e10 2024-09-21T03:55:10.428 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:10.394+0000 7f88b4ff9640 1 -- 172.21.15.39:0/2965648696 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88b0018b70 con 0x7f88b8109e10 2024-09-21T03:55:10.428 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:10.394+0000 7f88bd5b1640 1 -- 172.21.15.39:0/2965648696 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f88b81a0bf0 con 0x7f88b8109e10 2024-09-21T03:55:10.428 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:10.397+0000 7f88c3b25740 1 Processor -- start 2024-09-21T03:55:10.428 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:10.397+0000 7f88b4ff9640 1 -- 172.21.15.39:0/2965648696 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 27) ==== 154643+0+0 (secure 0 0 0) 0x7f88b0018d10 con 0x7f88b8109e10 2024-09-21T03:55:10.428 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:10.397+0000 7f88c3b25740 1 -- start start 2024-09-21T03:55:10.429 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:10.398+0000 7f88c3b25740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d3b6ee6070 0x55d3b728d6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:55:10.429 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:10.398+0000 7f88c3b25740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d3b6ebd1b0 0x55d3b728dbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:55:10.429 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:10.398+0000 7f88c3b25740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d3b6ec5390 0x55d3b728e120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:55:10.429 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:10.398+0000 7f88c3b25740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x55d3b7203ea0 con 0x55d3b6ee6070 2024-09-21T03:55:10.429 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:10.398+0000 7f88c3b25740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x55d3b71f32d0 con 0x55d3b6ebd1b0 2024-09-21T03:55:10.429 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:10.398+0000 7f88c3b25740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55d3b71f70e0 con 0x55d3b6ec5390 2024-09-21T03:55:10.429 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:10.398+0000 7f88ae7fc640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d3b6ee6070 0x55d3b728d6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:55:10.429 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:10.398+0000 7f88b6ffd640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d3b6ebd1b0 0x55d3b728dbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:55:10.429 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:10.398+0000 7f88ae7fc640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d3b6ee6070 0x55d3b728d6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:39268/0 (socket says 172.21.15.39:39268) 2024-09-21T03:55:10.429 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:10.398+0000 7f88ae7fc640 1 -- 172.21.15.39:0/3435110006 learned_addr learned my addr 172.21.15.39:0/3435110006 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:55:10.429 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:10.398+0000 7f88ae7fc640 1 --2- 172.21.15.39:0/3435110006 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d3b6ec5390 0x55d3b728e120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:55:10.429 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:10.398+0000 7f88b6ffd640 1 -- 172.21.15.39:0/3435110006 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d3b6ec5390 msgr2=0x55d3b728e120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:55:10.429 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:10.398+0000 7f88b6ffd640 1 --2- 172.21.15.39:0/3435110006 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d3b6ec5390 0x55d3b728e120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:10.429 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:10.398+0000 7f88b6ffd640 1 -- 172.21.15.39:0/3435110006 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d3b6ee6070 msgr2=0x55d3b728d6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:55:10.429 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:10.398+0000 7f88b6ffd640 1 --2- 172.21.15.39:0/3435110006 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d3b6ee6070 0x55d3b728d6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:10.430 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:10.398+0000 7f88b6ffd640 1 -- 172.21.15.39:0/3435110006 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d3b6f51650 con 0x55d3b6ebd1b0 2024-09-21T03:55:10.430 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:10.398+0000 7f88b4ff9640 1 --2- 172.21.15.39:0/2965648696 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f88a40b66b0 0x7f88a40b8bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:55:10.430 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:10.398+0000 7f88b6ffd640 1 --2- 172.21.15.39:0/3435110006 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d3b6ebd1b0 0x55d3b728dbe0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f88a8009230 tx=0x7f88a8032e60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fe2c63702fbb26e0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:55:10.430 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:10.399+0000 7f88ae7fc640 1 --2- 172.21.15.39:0/2965648696 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f88a40b66b0 0x7f88a40b8bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:55:10.430 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:10.399+0000 7f88b4ff9640 1 -- 172.21.15.39:0/2965648696 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8572+0+0 (secure 0 0 0) 0x7f88b00e7a90 con 0x7f88b8109e10 2024-09-21T03:55:10.430 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:10.399+0000 7f88ad7fa640 1 -- 172.21.15.39:0/3435110006 <== mon.2 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88a8040070 con 0x55d3b6ebd1b0 2024-09-21T03:55:10.430 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:10.399+0000 7f88ad7fa640 1 -- 172.21.15.39:0/3435110006 <== mon.2 v2:172.21.15.150:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f88a80316a0 con 0x55d3b6ebd1b0 2024-09-21T03:55:10.430 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:10.399+0000 7f88ad7fa640 1 -- 172.21.15.39:0/3435110006 <== mon.2 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88a8035040 con 0x55d3b6ebd1b0 2024-09-21T03:55:10.430 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:10.400+0000 7f88c3b25740 1 -- 172.21.15.39:0/3435110006 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d3b6ebd1b0 msgr2=0x55d3b728dbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:55:10.430 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:10.400+0000 7f88ae7fc640 1 --2- 172.21.15.39:0/2965648696 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f88a40b66b0 0x7f88a40b8bc0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f88b806aa90 tx=0x7f88a000b410 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:55:10.430 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:10.400+0000 7f88c3b25740 1 --2- 172.21.15.39:0/3435110006 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d3b6ebd1b0 0x55d3b728dbe0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f88a8009230 tx=0x7f88a8032e60 comp rx=0 tx=0).stop 2024-09-21T03:55:10.430 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:10.400+0000 7f88c3b25740 1 -- 172.21.15.39:0/3435110006 shutdown_connections 2024-09-21T03:55:10.430 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:10.400+0000 7f88c3b25740 1 --2- 172.21.15.39:0/3435110006 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d3b6ec5390 0x55d3b728e120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:10.431 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:10.400+0000 7f88c3b25740 1 --2- 172.21.15.39:0/3435110006 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d3b6ebd1b0 0x55d3b728dbe0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:10.431 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:10.400+0000 7f88c3b25740 1 --2- 172.21.15.39:0/3435110006 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d3b6ee6070 0x55d3b728d6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:10.431 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:10.400+0000 7f88c3b25740 1 -- 172.21.15.39:0/3435110006 >> 172.21.15.39:0/3435110006 conn(0x55d3b6eda460 msgr2=0x55d3b728b590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:55:10.431 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:10.400+0000 7f88c3b25740 1 -- 172.21.15.39:0/3435110006 shutdown_connections 2024-09-21T03:55:10.431 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:10.400+0000 7f88c3b25740 1 -- 172.21.15.39:0/3435110006 wait complete. 2024-09-21T03:55:10.431 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:10.400+0000 7f88c3b25740 1 -- start start 2024-09-21T03:55:10.431 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:10.400+0000 7f88c3b25740 1 Processor -- start 2024-09-21T03:55:10.431 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:10.401+0000 7f88c3b25740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d3b6ebd1b0 0x55d3b7298d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:55:10.431 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:10.401+0000 7f88c3b25740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d3b6ec5390 0x55d3b7299280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:55:10.431 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:10.401+0000 7f88c3b25740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d3b6ee6070 0x55d3b7299800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:55:10.431 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:10.401+0000 7f88c3b25740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x55d3b7054700 con 0x55d3b6ebd1b0 2024-09-21T03:55:10.431 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:10.401+0000 7f88c3b25740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x55d3b718e160 con 0x55d3b6ee6070 2024-09-21T03:55:10.431 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:10.401+0000 7f88c3b25740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55d3b729be20 con 0x55d3b6ec5390 2024-09-21T03:55:10.431 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:10.401+0000 7f88c3b25740 10 client.0 _ll_register_callbacks cb 0x55d3b71a79e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:55:10.431 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:10.401+0000 7f88b6ffd640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d3b6ebd1b0 0x55d3b7298d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:55:10.432 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:10.401+0000 7f88b6ffd640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d3b6ebd1b0 0x55d3b7298d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:39276/0 (socket says 172.21.15.39:39276) 2024-09-21T03:55:10.432 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:10.401+0000 7f88b6ffd640 1 -- 172.21.15.39:0/746898855 learned_addr learned my addr 172.21.15.39:0/746898855 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:55:10.432 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:10.401+0000 7f88ae7fc640 1 --2- 172.21.15.39:0/746898855 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d3b6ec5390 0x55d3b7299280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:55:10.432 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:10.401+0000 7f88b77fe640 1 --2- 172.21.15.39:0/746898855 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d3b6ee6070 0x55d3b7299800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:55:10.432 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:10.401+0000 7f88b6ffd640 1 -- 172.21.15.39:0/746898855 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d3b6ec5390 msgr2=0x55d3b7299280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:55:10.432 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:10.401+0000 7f88b6ffd640 1 --2- 172.21.15.39:0/746898855 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d3b6ec5390 0x55d3b7299280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:10.432 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:10.401+0000 7f88b6ffd640 1 -- 172.21.15.39:0/746898855 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d3b6ee6070 msgr2=0x55d3b7299800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:55:10.432 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:10.401+0000 7f88b6ffd640 1 --2- 172.21.15.39:0/746898855 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d3b6ee6070 0x55d3b7299800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:10.432 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:10.401+0000 7f88b6ffd640 1 -- 172.21.15.39:0/746898855 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d3b71bbc90 con 0x55d3b6ebd1b0 2024-09-21T03:55:10.432 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:10.401+0000 7f88b6ffd640 1 --2- 172.21.15.39:0/746898855 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d3b6ebd1b0 0x55d3b7298d40 secure :-1 s=READY pgs=1196 cs=0 l=1 rev1=1 crypto rx=0x7f88a8006010 tx=0x7f88a8005ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=48a47727a884f907 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:55:10.432 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:10.401+0000 7f889e7fc640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-09-21T03:55:10.432 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:10.402+0000 7f889e7fc640 1 -- 172.21.15.39:0/746898855 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88a8035070 con 0x55d3b6ebd1b0 2024-09-21T03:55:10.432 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:10.402+0000 7f88c3b25740 1 -- 172.21.15.39:0/746898855 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f88a80316a0 con 0x55d3b6ebd1b0 2024-09-21T03:55:10.433 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:10.402+0000 7f889e7fc640 1 -- 172.21.15.39:0/746898855 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8572+0+0 (secure 0 0 0) 0x7f88a8004230 con 0x55d3b6ebd1b0 2024-09-21T03:55:10.433 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:10.402+0000 7f889e7fc640 1 -- 172.21.15.39:0/746898855 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f88a8055050 con 0x55d3b6ebd1b0 2024-09-21T03:55:10.433 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:10.402+0000 7f88c3b25740 10 client.17892 fetch_fsmap learned FSMap version 29 2024-09-21T03:55:10.433 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:10.402+0000 7f88c3b25740 1 -- 172.21.15.39:0/746898855 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55d3b72a7ac0 con 0x55d3b6ebd1b0 2024-09-21T03:55:10.433 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:10.402+0000 7f889e7fc640 1 -- 172.21.15.39:0/746898855 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f88a80316a0 con 0x55d3b6ebd1b0 2024-09-21T03:55:10.433 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:10.402+0000 7f889e7fc640 1 -- 172.21.15.39:0/746898855 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f88a80046c0 con 0x55d3b6ebd1b0 2024-09-21T03:55:10.433 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:10.402+0000 7f88c3b25740 10 client.17892 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T03:55:10.433 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:10.402+0000 7f88c3b25740 10 client.17892 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-21T03:55:10.433 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:10.402+0000 7f88c3b25740 20 client.17892 populate_metadata read hostname 'smithi039' 2024-09-21T03:55:10.433 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:10.402+0000 7f88c3b25740 1 --2- 172.21.15.39:0/746898855 >> [v2:172.21.15.161:6836/3451544597,v1:172.21.15.161:6837/3451544597] conn(0x55d3b6ea49f0 0x55d3b72aae20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:55:10.433 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:10.402+0000 7f88c3b25740 4 client.17892 mds_command: new command op to 24646 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:55:10.433 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:10.402+0000 7f88c3b25740 1 -- 172.21.15.39:0/746898855 --> [v2:172.21.15.161:6836/3451544597,v1:172.21.15.161:6837/3451544597] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d3b7205df0 con 0x55d3b6ea49f0 2024-09-21T03:55:10.433 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:10.403+0000 7f88ae7fc640 1 --2- 172.21.15.39:0/746898855 >> [v2:172.21.15.161:6836/3451544597,v1:172.21.15.161:6837/3451544597] conn(0x55d3b6ea49f0 0x55d3b72aae20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:55:10.433 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:10.403+0000 7f88ae7fc640 1 --2- 172.21.15.39:0/746898855 >> [v2:172.21.15.161:6836/3451544597,v1:172.21.15.161:6837/3451544597] conn(0x55d3b6ea49f0 0x55d3b72aae20 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=b02bf5916b627ac server_cookie=ccd9f4e03f52cb5d in_seq=0 out_seq=0 2024-09-21T03:55:10.433 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:10.403+0000 7f889e7fc640 10 client.17892 ms_handle_connect on v2:172.21.15.161:6836/3451544597 2024-09-21T03:55:10.434 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:10.404+0000 7f889e7fc640 1 -- 172.21.15.39:0/746898855 <== mds.3 v2:172.21.15.161:6836/3451544597 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55d3b7205df0 con 0x55d3b6ea49f0 2024-09-21T03:55:10.434 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:10.404+0000 7f889e7fc640 10 client.17892 handle_command_reply: tid=0 2024-09-21T03:55:10.434 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:10.404+0000 7f88c3b25740 1 client.17892 shutdown 2024-09-21T03:55:10.434 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:10.404+0000 7f88c3b25740 1 -- 172.21.15.39:0/746898855 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d3b6ebd1b0 msgr2=0x55d3b7298d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:55:10.434 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:10.404+0000 7f88c3b25740 1 --2- 172.21.15.39:0/746898855 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d3b6ebd1b0 0x55d3b7298d40 secure :-1 s=READY pgs=1196 cs=0 l=1 rev1=1 crypto rx=0x7f88a8006010 tx=0x7f88a8005ef0 comp rx=0 tx=0).stop 2024-09-21T03:55:10.434 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:10.404+0000 7f88c3b25740 1 -- 172.21.15.39:0/746898855 shutdown_connections 2024-09-21T03:55:10.434 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:10.404+0000 7f88c3b25740 1 --2- 172.21.15.39:0/746898855 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d3b6ee6070 0x55d3b7299800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:10.434 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:10.404+0000 7f88c3b25740 1 --2- 172.21.15.39:0/746898855 >> [v2:172.21.15.161:6836/3451544597,v1:172.21.15.161:6837/3451544597] conn(0x55d3b6ea49f0 0x55d3b72aae20 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:10.434 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:10.404+0000 7f88c3b25740 1 --2- 172.21.15.39:0/746898855 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d3b6ec5390 0x55d3b7299280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:10.434 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:10.404+0000 7f88c3b25740 1 --2- 172.21.15.39:0/746898855 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d3b6ebd1b0 0x55d3b7298d40 unknown :-1 s=CLOSED pgs=1196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:10.434 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:10.404+0000 7f88c3b25740 1 -- 172.21.15.39:0/746898855 >> 172.21.15.39:0/746898855 conn(0x55d3b6eda460 msgr2=0x55d3b7290880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:55:10.434 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:10.404+0000 7f889e7fc640 0 client.17892 ms_handle_reset on v2:172.21.15.161:6836/3451544597 2024-09-21T03:55:10.434 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:10.404+0000 7f88c3b25740 1 -- 172.21.15.39:0/746898855 shutdown_connections 2024-09-21T03:55:10.434 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:10.404+0000 7f88c3b25740 1 -- 172.21.15.39:0/746898855 wait complete. 2024-09-21T03:55:10.435 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:10.405+0000 7f88c3b25740 20 client.17892 trim_cache size 0 max 16384 2024-09-21T03:55:10.435 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:10.418+0000 7f88c3b25740 1 Processor -- start 2024-09-21T03:55:10.435 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:10.418+0000 7f88c3b25740 1 -- start start 2024-09-21T03:55:10.435 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:10.418+0000 7f88c3b25740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d3b6ebd1b0 0x55d3b728f090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:55:10.435 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:10.418+0000 7f88c3b25740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d3b6ec5390 0x55d3b72a4780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:55:10.435 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:10.418+0000 7f88c3b25740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d3b6ee6070 0x55d3b728abc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:55:10.435 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:10.418+0000 7f88c3b25740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f88b81af7f0 con 0x55d3b6ee6070 2024-09-21T03:55:10.435 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:10.418+0000 7f88c3b25740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f88b81b1330 con 0x55d3b6ebd1b0 2024-09-21T03:55:10.435 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:10.418+0000 7f88c3b25740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f88b81afec0 con 0x55d3b6ec5390 2024-09-21T03:55:10.435 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:10.418+0000 7f88b77fe640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d3b6ee6070 0x55d3b728abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:55:10.435 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:10.418+0000 7f88b77fe640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d3b6ee6070 0x55d3b728abc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:39278/0 (socket says 172.21.15.39:39278) 2024-09-21T03:55:10.435 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:10.418+0000 7f88b77fe640 1 -- 172.21.15.39:0/909079618 learned_addr learned my addr 172.21.15.39:0/909079618 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:55:10.435 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:10.418+0000 7f88ae7fc640 1 --2- 172.21.15.39:0/909079618 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d3b6ec5390 0x55d3b72a4780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:55:10.435 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:10.418+0000 7f88b6ffd640 1 --2- 172.21.15.39:0/909079618 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d3b6ebd1b0 0x55d3b728f090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:55:10.435 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:10.418+0000 7f88b77fe640 1 -- 172.21.15.39:0/909079618 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d3b6ec5390 msgr2=0x55d3b72a4780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:55:10.436 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:10.418+0000 7f88b77fe640 1 --2- 172.21.15.39:0/909079618 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d3b6ec5390 0x55d3b72a4780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:10.436 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:10.418+0000 7f88b77fe640 1 -- 172.21.15.39:0/909079618 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d3b6ebd1b0 msgr2=0x55d3b728f090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:55:10.436 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:10.418+0000 7f88b77fe640 1 --2- 172.21.15.39:0/909079618 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d3b6ebd1b0 0x55d3b728f090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:10.436 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:10.419+0000 7f88b77fe640 1 -- 172.21.15.39:0/909079618 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d3b72b1110 con 0x55d3b6ee6070 2024-09-21T03:55:10.436 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:10.419+0000 7f88b77fe640 1 --2- 172.21.15.39:0/909079618 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d3b6ee6070 0x55d3b728abc0 secure :-1 s=READY pgs=1197 cs=0 l=1 rev1=1 crypto rx=0x7f88b001f000 tx=0x7f88b00a9810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9484344114f6c1ac server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:55:10.436 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:10.419+0000 7f88acff9640 1 -- 172.21.15.39:0/909079618 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88b00ad3c0 con 0x55d3b6ee6070 2024-09-21T03:55:10.436 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:10.419+0000 7f88acff9640 1 -- 172.21.15.39:0/909079618 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f88b00adfb0 con 0x55d3b6ee6070 2024-09-21T03:55:10.436 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:10.419+0000 7f88acff9640 1 -- 172.21.15.39:0/909079618 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88b00aee40 con 0x55d3b6ee6070 2024-09-21T03:55:10.436 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:10.420+0000 7f88c3b25740 1 -- 172.21.15.39:0/909079618 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d3b6ee6070 msgr2=0x55d3b728abc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:55:10.436 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:10.420+0000 7f88c3b25740 1 --2- 172.21.15.39:0/909079618 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d3b6ee6070 0x55d3b728abc0 secure :-1 s=READY pgs=1197 cs=0 l=1 rev1=1 crypto rx=0x7f88b001f000 tx=0x7f88b00a9810 comp rx=0 tx=0).stop 2024-09-21T03:55:10.436 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:10.420+0000 7f88c3b25740 1 -- 172.21.15.39:0/909079618 shutdown_connections 2024-09-21T03:55:10.436 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:10.420+0000 7f88c3b25740 1 --2- 172.21.15.39:0/909079618 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d3b6ee6070 0x55d3b728abc0 unknown :-1 s=CLOSED pgs=1197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:10.436 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:10.420+0000 7f88c3b25740 1 --2- 172.21.15.39:0/909079618 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d3b6ec5390 0x55d3b72a4780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:10.436 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:10.420+0000 7f88c3b25740 1 --2- 172.21.15.39:0/909079618 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d3b6ebd1b0 0x55d3b728f090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:10.436 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:10.420+0000 7f88c3b25740 1 -- 172.21.15.39:0/909079618 >> 172.21.15.39:0/909079618 conn(0x55d3b6eda460 msgr2=0x55d3b72a4070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:55:10.436 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:10.420+0000 7f88c3b25740 1 -- 172.21.15.39:0/909079618 shutdown_connections 2024-09-21T03:55:10.437 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:10.420+0000 7f88c3b25740 1 -- 172.21.15.39:0/909079618 wait complete. 2024-09-21T03:55:10.437 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:10.420+0000 7f88c3b25740 1 -- start start 2024-09-21T03:55:10.437 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:10.420+0000 7f88c3b25740 1 Processor -- start 2024-09-21T03:55:10.437 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:10.421+0000 7f88c3b25740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d3b6ebd1b0 0x55d3b72abf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:55:10.437 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:10.421+0000 7f88c3b25740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d3b6ec5390 0x55d3b72ac4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:55:10.437 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:10.421+0000 7f88c3b25740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d3b6ee6070 0x55d3b72ac9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:55:10.437 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:10.421+0000 7f88c3b25740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f88b81b0c60 con 0x55d3b6ebd1b0 2024-09-21T03:55:10.437 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:10.421+0000 7f88c3b25740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f88b81b0590 con 0x55d3b6ec5390 2024-09-21T03:55:10.437 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:10.421+0000 7f88c3b25740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f88b81ad7f0 con 0x55d3b6ee6070 2024-09-21T03:55:10.437 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:10.421+0000 7f88b6ffd640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d3b6ebd1b0 0x55d3b72abf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:55:10.437 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:10.421+0000 7f88b6ffd640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d3b6ebd1b0 0x55d3b72abf70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:39286/0 (socket says 172.21.15.39:39286) 2024-09-21T03:55:10.437 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:10.421+0000 7f88b6ffd640 1 -- 172.21.15.39:0/4166546589 learned_addr learned my addr 172.21.15.39:0/4166546589 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:55:10.437 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:10.421+0000 7f88c3b25740 10 client.0 _ll_register_callbacks cb 0x7f88a000ba90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:55:10.437 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:10.421+0000 7f88b77fe640 1 --2- 172.21.15.39:0/4166546589 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d3b6ee6070 0x55d3b72ac9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:55:10.437 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:10.421+0000 7f88ae7fc640 1 --2- 172.21.15.39:0/4166546589 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d3b6ec5390 0x55d3b72ac4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:55:10.437 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:10.421+0000 7f88b6ffd640 1 -- 172.21.15.39:0/4166546589 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d3b6ee6070 msgr2=0x55d3b72ac9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:55:10.438 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:10.421+0000 7f88b6ffd640 1 --2- 172.21.15.39:0/4166546589 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d3b6ee6070 0x55d3b72ac9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:10.438 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:10.421+0000 7f88b6ffd640 1 -- 172.21.15.39:0/4166546589 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d3b6ec5390 msgr2=0x55d3b72ac4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:55:10.438 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:10.421+0000 7f88b6ffd640 1 --2- 172.21.15.39:0/4166546589 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d3b6ec5390 0x55d3b72ac4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:10.438 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:10.421+0000 7f88b6ffd640 1 -- 172.21.15.39:0/4166546589 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d3b73146d0 con 0x55d3b6ebd1b0 2024-09-21T03:55:10.438 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:10.421+0000 7f88ae7fc640 1 --2- 172.21.15.39:0/4166546589 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d3b6ec5390 0x55d3b72ac4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:55:10.438 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:10.421+0000 7f88b6ffd640 1 --2- 172.21.15.39:0/4166546589 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d3b6ebd1b0 0x55d3b72abf70 secure :-1 s=READY pgs=1198 cs=0 l=1 rev1=1 crypto rx=0x7f88a8004c80 tx=0x7f88a8000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f2d74d54ded2a4cd server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:55:10.438 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:10.421+0000 7f889e7fc640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-09-21T03:55:10.438 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:10.422+0000 7f889e7fc640 1 -- 172.21.15.39:0/4166546589 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88a8035070 con 0x55d3b6ebd1b0 2024-09-21T03:55:10.438 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:10.422+0000 7f889e7fc640 1 -- 172.21.15.39:0/4166546589 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8572+0+0 (secure 0 0 0) 0x7f88a8031c60 con 0x55d3b6ebd1b0 2024-09-21T03:55:10.438 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:10.422+0000 7f88c3b25740 1 -- 172.21.15.39:0/4166546589 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f88b00adfb0 con 0x55d3b6ebd1b0 2024-09-21T03:55:10.438 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:10.422+0000 7f889e7fc640 1 -- 172.21.15.39:0/4166546589 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f88a8040040 con 0x55d3b6ebd1b0 2024-09-21T03:55:10.438 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:10.422+0000 7f88c3b25740 10 client.17898 fetch_fsmap learned FSMap version 29 2024-09-21T03:55:10.438 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:10.422+0000 7f88c3b25740 1 -- 172.21.15.39:0/4166546589 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55d3b7315950 con 0x55d3b6ebd1b0 2024-09-21T03:55:10.438 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:10.422+0000 7f889e7fc640 1 -- 172.21.15.39:0/4166546589 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f88b00adfb0 con 0x55d3b6ebd1b0 2024-09-21T03:55:10.438 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:10.422+0000 7f889e7fc640 1 -- 172.21.15.39:0/4166546589 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f88a8061030 con 0x55d3b6ebd1b0 2024-09-21T03:55:10.439 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:10.422+0000 7f88c3b25740 10 client.17898 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T03:55:10.439 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:10.422+0000 7f88c3b25740 10 client.17898 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-21T03:55:10.439 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:10.422+0000 7f88c3b25740 20 client.17898 populate_metadata read hostname 'smithi039' 2024-09-21T03:55:10.439 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:10.422+0000 7f88c3b25740 1 --2- 172.21.15.39:0/4166546589 >> [v2:172.21.15.161:6836/3451544597,v1:172.21.15.161:6837/3451544597] conn(0x55d3b6e79da0 0x55d3b7318190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:55:10.439 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:10.422+0000 7f88c3b25740 4 client.17898 mds_command: new command op to 24646 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T03:55:10.439 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:10.422+0000 7f88c3b25740 1 -- 172.21.15.39:0/4166546589 --> [v2:172.21.15.161:6836/3451544597,v1:172.21.15.161:6837/3451544597] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55d3b72018a0 con 0x55d3b6e79da0 2024-09-21T03:55:10.439 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:10.423+0000 7f88ae7fc640 1 --2- 172.21.15.39:0/4166546589 >> [v2:172.21.15.161:6836/3451544597,v1:172.21.15.161:6837/3451544597] conn(0x55d3b6e79da0 0x55d3b7318190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:55:10.439 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:10.423+0000 7f889e7fc640 10 client.17898 ms_handle_connect on v2:172.21.15.161:6836/3451544597 2024-09-21T03:55:10.439 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:10.423+0000 7f88ae7fc640 1 --2- 172.21.15.39:0/4166546589 >> [v2:172.21.15.161:6836/3451544597,v1:172.21.15.161:6837/3451544597] conn(0x55d3b6e79da0 0x55d3b7318190 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=f30b12bcebd0bf4 server_cookie=9fc6de017319605e in_seq=0 out_seq=0 2024-09-21T03:55:10.440 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:10.423+0000 7f889e7fc640 1 -- 172.21.15.39:0/4166546589 <== mds.3 v2:172.21.15.161:6836/3451544597 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55d3b72018a0 con 0x55d3b6e79da0 2024-09-21T03:55:10.440 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:10.423+0000 7f889e7fc640 10 client.17898 handle_command_reply: tid=0 2024-09-21T03:55:10.440 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:10.424+0000 7f88c3b25740 1 client.17898 shutdown 2024-09-21T03:55:10.440 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:10.424+0000 7f88c3b25740 1 -- 172.21.15.39:0/4166546589 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d3b6ebd1b0 msgr2=0x55d3b72abf70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:55:10.440 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:10.424+0000 7f88c3b25740 1 --2- 172.21.15.39:0/4166546589 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d3b6ebd1b0 0x55d3b72abf70 secure :-1 s=READY pgs=1198 cs=0 l=1 rev1=1 crypto rx=0x7f88a8004c80 tx=0x7f88a8000bc0 comp rx=0 tx=0).stop 2024-09-21T03:55:10.440 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:10.424+0000 7f88c3b25740 1 -- 172.21.15.39:0/4166546589 shutdown_connections 2024-09-21T03:55:10.440 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:10.424+0000 7f88c3b25740 1 --2- 172.21.15.39:0/4166546589 >> [v2:172.21.15.161:6836/3451544597,v1:172.21.15.161:6837/3451544597] conn(0x55d3b6e79da0 0x55d3b7318190 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:10.440 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:10.424+0000 7f88c3b25740 1 --2- 172.21.15.39:0/4166546589 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d3b6ee6070 0x55d3b72ac9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:10.441 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:10.424+0000 7f88c3b25740 1 --2- 172.21.15.39:0/4166546589 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d3b6ec5390 0x55d3b72ac4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:10.441 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:10.424+0000 7f889e7fc640 0 client.17898 ms_handle_reset on v2:172.21.15.161:6836/3451544597 2024-09-21T03:55:10.441 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:10.424+0000 7f88c3b25740 1 --2- 172.21.15.39:0/4166546589 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d3b6ebd1b0 0x55d3b72abf70 unknown :-1 s=CLOSED pgs=1198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:10.441 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:10.424+0000 7f88c3b25740 1 -- 172.21.15.39:0/4166546589 >> 172.21.15.39:0/4166546589 conn(0x55d3b6eda460 msgr2=0x55d3b728ee60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:55:10.441 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:10.424+0000 7f88c3b25740 1 -- 172.21.15.39:0/4166546589 shutdown_connections 2024-09-21T03:55:10.441 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:10.424+0000 7f88c3b25740 1 -- 172.21.15.39:0/4166546589 wait complete. 2024-09-21T03:55:10.441 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:10.424+0000 7f88c3b25740 20 client.17898 trim_cache size 0 max 16384 2024-09-21T03:55:10.441 INFO:teuthology.orchestra.run.smithi039.stdout:[] 2024-09-21T03:55:10.442 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:10.424+0000 7f88bd5b1640 1 -- 172.21.15.39:0/2965648696 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f88a40b66b0 msgr2=0x7f88a40b8bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:55:10.442 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:10.424+0000 7f88bd5b1640 1 --2- 172.21.15.39:0/2965648696 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f88a40b66b0 0x7f88a40b8bc0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f88b806aa90 tx=0x7f88a000b410 comp rx=0 tx=0).stop 2024-09-21T03:55:10.442 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:10.424+0000 7f88bd5b1640 1 -- 172.21.15.39:0/2965648696 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f88b8109e10 msgr2=0x7f88b819fda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:55:10.442 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:10.424+0000 7f88bd5b1640 1 --2- 172.21.15.39:0/2965648696 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f88b8109e10 0x7f88b819fda0 secure :-1 s=READY pgs=1195 cs=0 l=1 rev1=1 crypto rx=0x7f88b000cce0 tx=0x7f88b000a620 comp rx=0 tx=0).stop 2024-09-21T03:55:10.442 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:10.425+0000 7f88bd5b1640 1 -- 172.21.15.39:0/2965648696 shutdown_connections 2024-09-21T03:55:10.442 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:10.425+0000 7f88bd5b1640 1 --2- 172.21.15.39:0/2965648696 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f88a40b66b0 0x7f88a40b8bc0 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:10.442 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:10.425+0000 7f88bd5b1640 1 --2- 172.21.15.39:0/2965648696 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f88b8109e10 0x7f88b819fda0 unknown :-1 s=CLOSED pgs=1195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:10.442 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:10.425+0000 7f88bd5b1640 1 --2- 172.21.15.39:0/2965648696 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f88b81094c0 0x7f88b819ab00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:10.442 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:10.425+0000 7f88bd5b1640 1 --2- 172.21.15.39:0/2965648696 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f88b8103b80 0x7f88b819a560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:10.442 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:10.425+0000 7f88bd5b1640 1 -- 172.21.15.39:0/2965648696 >> 172.21.15.39:0/2965648696 conn(0x7f88b80ff590 msgr2=0x7f88b8107c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:55:10.442 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:10.425+0000 7f88bd5b1640 1 -- 172.21.15.39:0/2965648696 shutdown_connections 2024-09-21T03:55:10.442 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:10.425+0000 7f88bd5b1640 1 -- 172.21.15.39:0/2965648696 wait complete. 2024-09-21T03:55:11.084 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:55:11 smithi039 ceph-mon[31211]: pgmap v603: 97 pgs: 97 active+clean; 10 MiB data, 525 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 44 KiB/s wr, 28 op/s 2024-09-21T03:55:11.084 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:55:11 smithi039 ceph-mon[31211]: osdmap e84: 12 total, 12 up, 12 in 2024-09-21T03:55:11.215 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 51116d5c-77ca-11ef-baf5-efdc52797490 -- ceph tell mds.1:2 damage ls 2024-09-21T03:55:11.270 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:55:11 smithi161 ceph-mon[34533]: pgmap v603: 97 pgs: 97 active+clean; 10 MiB data, 525 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 44 KiB/s wr, 28 op/s 2024-09-21T03:55:11.271 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:55:11 smithi161 ceph-mon[34533]: osdmap e84: 12 total, 12 up, 12 in 2024-09-21T03:55:11.475 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:55:11 smithi150 ceph-mon[34289]: pgmap v603: 97 pgs: 97 active+clean; 10 MiB data, 525 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 44 KiB/s wr, 28 op/s 2024-09-21T03:55:11.475 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:55:11 smithi150 ceph-mon[34289]: osdmap e84: 12 total, 12 up, 12 in 2024-09-21T03:55:11.478 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/51116d5c-77ca-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:55:11.991 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:11.973+0000 7f3db6486640 1 -- 172.21.15.39:0/990133783 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3db0109e20 msgr2=0x7f3db0110650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:55:11.991 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:11.973+0000 7f3db6486640 1 --2- 172.21.15.39:0/990133783 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3db0109e20 0x7f3db0110650 secure :-1 s=READY pgs=1199 cs=0 l=1 rev1=1 crypto rx=0x7f3da8009ca0 tx=0x7f3da802a8c0 comp rx=0 tx=0).stop 2024-09-21T03:55:11.991 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:11.973+0000 7f3db6486640 1 -- 172.21.15.39:0/990133783 shutdown_connections 2024-09-21T03:55:11.991 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:11.973+0000 7f3db6486640 1 --2- 172.21.15.39:0/990133783 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3db0109e20 0x7f3db0110650 unknown :-1 s=CLOSED pgs=1199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:11.991 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:11.973+0000 7f3db6486640 1 --2- 172.21.15.39:0/990133783 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3db01094d0 0x7f3db01098e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:11.991 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:11.973+0000 7f3db6486640 1 --2- 172.21.15.39:0/990133783 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3db0103b90 0x7f3db0103fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:11.991 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:11.973+0000 7f3db6486640 1 -- 172.21.15.39:0/990133783 >> 172.21.15.39:0/990133783 conn(0x7f3db00ff600 msgr2=0x7f3db0101a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:55:11.991 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:11.973+0000 7f3db6486640 1 -- 172.21.15.39:0/990133783 shutdown_connections 2024-09-21T03:55:11.992 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:11.973+0000 7f3db6486640 1 -- 172.21.15.39:0/990133783 wait complete. 2024-09-21T03:55:11.992 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:11.974+0000 7f3db6486640 1 Processor -- start 2024-09-21T03:55:11.992 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:11.974+0000 7f3db6486640 1 -- start start 2024-09-21T03:55:11.992 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:11.974+0000 7f3db6486640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3db0103b90 0x7f3db019a470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:55:11.992 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:11.974+0000 7f3db6486640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3db01094d0 0x7f3db019aa10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:55:11.992 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:11.974+0000 7f3db6486640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3db0109e20 0x7f3db019fcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:55:11.992 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:11.974+0000 7f3db6486640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f3db0113ae0 con 0x7f3db0109e20 2024-09-21T03:55:11.992 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:11.974+0000 7f3db6486640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f3db0113960 con 0x7f3db0103b90 2024-09-21T03:55:11.992 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:11.974+0000 7f3db6486640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f3db0113c60 con 0x7f3db01094d0 2024-09-21T03:55:11.992 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:11.974+0000 7f3db4a23640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3db0109e20 0x7f3db019fcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:55:11.992 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:11.974+0000 7f3db4a23640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3db0109e20 0x7f3db019fcb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:39292/0 (socket says 172.21.15.39:39292) 2024-09-21T03:55:11.992 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:11.974+0000 7f3db4a23640 1 -- 172.21.15.39:0/1162706936 learned_addr learned my addr 172.21.15.39:0/1162706936 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:55:11.992 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:11.975+0000 7f3da7fff640 1 --2- 172.21.15.39:0/1162706936 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3db01094d0 0x7f3db019aa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:55:11.992 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:11.975+0000 7f3daffff640 1 --2- 172.21.15.39:0/1162706936 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3db0103b90 0x7f3db019a470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:55:11.993 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:11.975+0000 7f3db4a23640 1 -- 172.21.15.39:0/1162706936 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3db01094d0 msgr2=0x7f3db019aa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:55:11.993 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:11.975+0000 7f3db4a23640 1 --2- 172.21.15.39:0/1162706936 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3db01094d0 0x7f3db019aa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:11.993 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:11.975+0000 7f3db4a23640 1 -- 172.21.15.39:0/1162706936 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3db0103b90 msgr2=0x7f3db019a470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:55:11.993 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:11.975+0000 7f3db4a23640 1 --2- 172.21.15.39:0/1162706936 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3db0103b90 0x7f3db019a470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:11.993 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:11.975+0000 7f3db4a23640 1 -- 172.21.15.39:0/1162706936 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3db01a03e0 con 0x7f3db0109e20 2024-09-21T03:55:11.993 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:11.975+0000 7f3db4a23640 1 --2- 172.21.15.39:0/1162706936 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3db0109e20 0x7f3db019fcb0 secure :-1 s=READY pgs=1200 cs=0 l=1 rev1=1 crypto rx=0x7f3da80098d0 tx=0x7f3da80317b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:55:11.993 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:11.975+0000 7f3dadffb640 1 -- 172.21.15.39:0/1162706936 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3da8003b70 con 0x7f3db0109e20 2024-09-21T03:55:11.993 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:11.975+0000 7f3db6486640 1 -- 172.21.15.39:0/1162706936 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3db01a0670 con 0x7f3db0109e20 2024-09-21T03:55:11.993 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:11.975+0000 7f3dadffb640 1 -- 172.21.15.39:0/1162706936 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3da8003d10 con 0x7f3db0109e20 2024-09-21T03:55:11.993 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:11.975+0000 7f3dadffb640 1 -- 172.21.15.39:0/1162706936 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3da802fa20 con 0x7f3db0109e20 2024-09-21T03:55:11.993 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:11.975+0000 7f3db6486640 1 -- 172.21.15.39:0/1162706936 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3db01a6e30 con 0x7f3db0109e20 2024-09-21T03:55:11.993 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:11.978+0000 7f3dbc9fa740 1 Processor -- start 2024-09-21T03:55:11.993 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:11.978+0000 7f3dadffb640 1 -- 172.21.15.39:0/1162706936 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 27) ==== 154643+0+0 (secure 0 0 0) 0x7f3da802fc20 con 0x7f3db0109e20 2024-09-21T03:55:11.993 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:11.978+0000 7f3dbc9fa740 1 -- start start 2024-09-21T03:55:11.993 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:11.978+0000 7f3dbc9fa740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5652ee89d070 0x5652eec446a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:55:11.994 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:11.979+0000 7f3dbc9fa740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5652ee8741b0 0x5652eec44be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:55:11.994 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:11.979+0000 7f3dbc9fa740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5652ee87c390 0x5652eec45120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:55:11.994 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:11.979+0000 7f3dbc9fa740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x5652eebbaea0 con 0x5652ee87c390 2024-09-21T03:55:11.994 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:11.979+0000 7f3dbc9fa740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x5652eebaa2d0 con 0x5652ee8741b0 2024-09-21T03:55:11.994 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:11.979+0000 7f3dbc9fa740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x5652eebae0e0 con 0x5652ee89d070 2024-09-21T03:55:11.994 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:11.979+0000 7f3da7fff640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5652ee87c390 0x5652eec45120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:55:11.994 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:11.979+0000 7f3da7fff640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5652ee87c390 0x5652eec45120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:39306/0 (socket says 172.21.15.39:39306) 2024-09-21T03:55:11.994 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:11.979+0000 7f3da7fff640 1 -- 172.21.15.39:0/2393962141 learned_addr learned my addr 172.21.15.39:0/2393962141 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:55:11.994 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:11.979+0000 7f3daffff640 1 --2- 172.21.15.39:0/2393962141 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5652ee8741b0 0x5652eec44be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:55:11.994 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:11.979+0000 7f3da7fff640 1 --2- 172.21.15.39:0/2393962141 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5652ee89d070 0x5652eec446a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:55:11.994 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:11.979+0000 7f3da7fff640 1 -- 172.21.15.39:0/2393962141 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5652ee89d070 msgr2=0x5652eec446a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:55:11.994 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:11.979+0000 7f3da7fff640 1 --2- 172.21.15.39:0/2393962141 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5652ee89d070 0x5652eec446a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:11.995 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:11.979+0000 7f3da7fff640 1 -- 172.21.15.39:0/2393962141 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5652ee8741b0 msgr2=0x5652eec44be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:55:11.995 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:11.979+0000 7f3da7fff640 1 --2- 172.21.15.39:0/2393962141 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5652ee8741b0 0x5652eec44be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:11.995 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:11.979+0000 7f3da7fff640 1 -- 172.21.15.39:0/2393962141 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5652ee908650 con 0x5652ee87c390 2024-09-21T03:55:11.995 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:11.979+0000 7f3daffff640 1 --2- 172.21.15.39:0/2393962141 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5652ee8741b0 0x5652eec44be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:55:11.995 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:11.979+0000 7f3dadffb640 1 --2- 172.21.15.39:0/1162706936 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f3d9c0b6680 0x7f3d9c0b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:55:11.995 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:11.979+0000 7f3daffff640 1 --2- 172.21.15.39:0/1162706936 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f3d9c0b6680 0x7f3d9c0b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:55:11.995 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:11.979+0000 7f3da7fff640 1 --2- 172.21.15.39:0/2393962141 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5652ee87c390 0x5652eec45120 secure :-1 s=READY pgs=1201 cs=0 l=1 rev1=1 crypto rx=0x7f3d980124c0 tx=0x7f3d98005c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c97c26d6c5c65caa server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:55:11.995 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:11.979+0000 7f3dadffb640 1 -- 172.21.15.39:0/1162706936 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8572+0+0 (secure 0 0 0) 0x7f3da8108bd0 con 0x7f3db0109e20 2024-09-21T03:55:11.995 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:11.980+0000 7f3da67fc640 1 -- 172.21.15.39:0/2393962141 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d98004230 con 0x5652ee87c390 2024-09-21T03:55:11.995 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:11.980+0000 7f3da67fc640 1 -- 172.21.15.39:0/2393962141 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3d980043d0 con 0x5652ee87c390 2024-09-21T03:55:11.995 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:11.980+0000 7f3da67fc640 1 -- 172.21.15.39:0/2393962141 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d98004de0 con 0x5652ee87c390 2024-09-21T03:55:11.995 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:11.980+0000 7f3dbc9fa740 1 -- 172.21.15.39:0/2393962141 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5652ee87c390 msgr2=0x5652eec45120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:55:11.995 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:11.980+0000 7f3daffff640 1 --2- 172.21.15.39:0/1162706936 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f3d9c0b6680 0x7f3d9c0b8b90 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x5652eec43320 tx=0x7f3da0005f90 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:55:11.995 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:11.980+0000 7f3dbc9fa740 1 --2- 172.21.15.39:0/2393962141 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5652ee87c390 0x5652eec45120 secure :-1 s=READY pgs=1201 cs=0 l=1 rev1=1 crypto rx=0x7f3d980124c0 tx=0x7f3d98005c90 comp rx=0 tx=0).stop 2024-09-21T03:55:11.995 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:11.981+0000 7f3dbc9fa740 1 -- 172.21.15.39:0/2393962141 shutdown_connections 2024-09-21T03:55:11.996 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:11.981+0000 7f3dbc9fa740 1 --2- 172.21.15.39:0/2393962141 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5652ee87c390 0x5652eec45120 unknown :-1 s=CLOSED pgs=1201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:11.996 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:11.981+0000 7f3dbc9fa740 1 --2- 172.21.15.39:0/2393962141 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5652ee8741b0 0x5652eec44be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:11.996 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:11.981+0000 7f3dbc9fa740 1 --2- 172.21.15.39:0/2393962141 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5652ee89d070 0x5652eec446a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:11.996 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:11.981+0000 7f3dbc9fa740 1 -- 172.21.15.39:0/2393962141 >> 172.21.15.39:0/2393962141 conn(0x5652ee891460 msgr2=0x5652eec42590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:55:11.996 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:11.981+0000 7f3dbc9fa740 1 -- 172.21.15.39:0/2393962141 shutdown_connections 2024-09-21T03:55:11.996 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:11.981+0000 7f3dbc9fa740 1 -- 172.21.15.39:0/2393962141 wait complete. 2024-09-21T03:55:11.996 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:11.981+0000 7f3dbc9fa740 1 -- start start 2024-09-21T03:55:11.996 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:11.981+0000 7f3dbc9fa740 1 Processor -- start 2024-09-21T03:55:11.996 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:11.982+0000 7f3dbc9fa740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5652ee8741b0 0x5652eec4fd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:55:11.996 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:11.982+0000 7f3dbc9fa740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5652ee87c390 0x5652eec50260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:55:11.996 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:11.982+0000 7f3dbc9fa740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5652ee89d070 0x5652eec5c1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:55:11.996 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:11.982+0000 7f3dbc9fa740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x5652eea0b700 con 0x5652ee8741b0 2024-09-21T03:55:11.996 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:11.982+0000 7f3dbc9fa740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x5652eeb45160 con 0x5652ee87c390 2024-09-21T03:55:11.996 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:11.982+0000 7f3dbc9fa740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x5652eec52e80 con 0x5652ee89d070 2024-09-21T03:55:11.996 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:11.982+0000 7f3dbc9fa740 10 client.0 _ll_register_callbacks cb 0x5652eeb5e9e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:55:11.997 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:11.982+0000 7f3daffff640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5652ee8741b0 0x5652eec4fd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:55:11.997 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:11.982+0000 7f3daffff640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5652ee8741b0 0x5652eec4fd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:39320/0 (socket says 172.21.15.39:39320) 2024-09-21T03:55:11.997 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:11.982+0000 7f3daffff640 1 -- 172.21.15.39:0/2876306599 learned_addr learned my addr 172.21.15.39:0/2876306599 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:55:11.997 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:11.982+0000 7f3db4a23640 1 --2- 172.21.15.39:0/2876306599 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5652ee89d070 0x5652eec5c1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:55:11.997 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:11.982+0000 7f3da7fff640 1 --2- 172.21.15.39:0/2876306599 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5652ee87c390 0x5652eec50260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:55:11.997 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:11.982+0000 7f3daffff640 1 -- 172.21.15.39:0/2876306599 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5652ee89d070 msgr2=0x5652eec5c1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:55:11.997 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:11.982+0000 7f3daffff640 1 --2- 172.21.15.39:0/2876306599 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5652ee89d070 0x5652eec5c1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:11.997 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:11.982+0000 7f3daffff640 1 -- 172.21.15.39:0/2876306599 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5652ee87c390 msgr2=0x5652eec50260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:55:11.997 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:11.982+0000 7f3daffff640 1 --2- 172.21.15.39:0/2876306599 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5652ee87c390 0x5652eec50260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:11.997 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:11.982+0000 7f3daffff640 1 -- 172.21.15.39:0/2876306599 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5652eeb72c90 con 0x5652ee8741b0 2024-09-21T03:55:11.997 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:11.982+0000 7f3daffff640 1 --2- 172.21.15.39:0/2876306599 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5652ee8741b0 0x5652eec4fd20 secure :-1 s=READY pgs=1202 cs=0 l=1 rev1=1 crypto rx=0x7f3da001e9d0 tx=0x7f3da0004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=23954a86ae4b062d server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:55:11.997 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:11.982+0000 7f3d8f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-09-21T03:55:11.997 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:11.983+0000 7f3d8f7fe640 1 -- 172.21.15.39:0/2876306599 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3da002e060 con 0x5652ee8741b0 2024-09-21T03:55:11.997 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:11.983+0000 7f3d8f7fe640 1 -- 172.21.15.39:0/2876306599 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8572+0+0 (secure 0 0 0) 0x7f3da0018030 con 0x5652ee8741b0 2024-09-21T03:55:11.998 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:11.983+0000 7f3dbc9fa740 1 -- 172.21.15.39:0/2876306599 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3d980043d0 con 0x5652ee8741b0 2024-09-21T03:55:11.998 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:11.983+0000 7f3d8f7fe640 1 -- 172.21.15.39:0/2876306599 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f3da0009db0 con 0x5652ee8741b0 2024-09-21T03:55:11.998 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:11.983+0000 7f3dbc9fa740 10 client.17910 fetch_fsmap learned FSMap version 29 2024-09-21T03:55:11.998 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:11.983+0000 7f3dbc9fa740 1 -- 172.21.15.39:0/2876306599 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5652eec5ebe0 con 0x5652ee8741b0 2024-09-21T03:55:11.998 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:11.983+0000 7f3d8f7fe640 1 -- 172.21.15.39:0/2876306599 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3d980043d0 con 0x5652ee8741b0 2024-09-21T03:55:11.998 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:11.983+0000 7f3d8f7fe640 1 -- 172.21.15.39:0/2876306599 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f3da000f580 con 0x5652ee8741b0 2024-09-21T03:55:11.998 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:11.983+0000 7f3dbc9fa740 10 client.17910 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T03:55:11.998 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:11.983+0000 7f3dbc9fa740 10 client.17910 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-21T03:55:11.998 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:11.983+0000 7f3dbc9fa740 20 client.17910 populate_metadata read hostname 'smithi039' 2024-09-21T03:55:11.998 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:11.983+0000 7f3dbc9fa740 1 --2- 172.21.15.39:0/2876306599 >> [v2:172.21.15.150:6836/4017043144,v1:172.21.15.150:6837/4017043144] conn(0x5652ee85b9f0 0x5652eec62550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:55:11.998 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:11.983+0000 7f3dbc9fa740 4 client.17910 mds_command: new command op to 24659 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:55:11.998 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:11.983+0000 7f3dbc9fa740 1 -- 172.21.15.39:0/2876306599 --> [v2:172.21.15.150:6836/4017043144,v1:172.21.15.150:6837/4017043144] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5652eebbcdf0 con 0x5652ee85b9f0 2024-09-21T03:55:11.998 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:11.984+0000 7f3da7fff640 1 --2- 172.21.15.39:0/2876306599 >> [v2:172.21.15.150:6836/4017043144,v1:172.21.15.150:6837/4017043144] conn(0x5652ee85b9f0 0x5652eec62550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:55:11.998 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:11.984+0000 7f3da7fff640 1 --2- 172.21.15.39:0/2876306599 >> [v2:172.21.15.150:6836/4017043144,v1:172.21.15.150:6837/4017043144] conn(0x5652ee85b9f0 0x5652eec62550 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=4eadb73b22fb7f5b server_cookie=ea9a69dcd4d23204 in_seq=0 out_seq=0 2024-09-21T03:55:11.999 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:11.984+0000 7f3d8f7fe640 10 client.17910 ms_handle_connect on v2:172.21.15.150:6836/4017043144 2024-09-21T03:55:11.999 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:11.985+0000 7f3d8f7fe640 1 -- 172.21.15.39:0/2876306599 <== mds.2 v2:172.21.15.150:6836/4017043144 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5652eebbcdf0 con 0x5652ee85b9f0 2024-09-21T03:55:11.999 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:11.985+0000 7f3d8f7fe640 10 client.17910 handle_command_reply: tid=0 2024-09-21T03:55:11.999 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:11.985+0000 7f3dbc9fa740 1 client.17910 shutdown 2024-09-21T03:55:11.999 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:11.985+0000 7f3dbc9fa740 1 -- 172.21.15.39:0/2876306599 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5652ee8741b0 msgr2=0x5652eec4fd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:55:11.999 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:11.985+0000 7f3dbc9fa740 1 --2- 172.21.15.39:0/2876306599 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5652ee8741b0 0x5652eec4fd20 secure :-1 s=READY pgs=1202 cs=0 l=1 rev1=1 crypto rx=0x7f3da001e9d0 tx=0x7f3da0004730 comp rx=0 tx=0).stop 2024-09-21T03:55:11.999 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:11.985+0000 7f3dbc9fa740 1 -- 172.21.15.39:0/2876306599 shutdown_connections 2024-09-21T03:55:11.999 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:11.985+0000 7f3dbc9fa740 1 --2- 172.21.15.39:0/2876306599 >> [v2:172.21.15.150:6836/4017043144,v1:172.21.15.150:6837/4017043144] conn(0x5652ee85b9f0 0x5652eec62550 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:11.999 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:11.985+0000 7f3dbc9fa740 1 --2- 172.21.15.39:0/2876306599 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5652ee89d070 0x5652eec5c1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:11.999 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:11.985+0000 7f3dbc9fa740 1 --2- 172.21.15.39:0/2876306599 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5652ee87c390 0x5652eec50260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:11.999 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:11.985+0000 7f3d8f7fe640 0 client.17910 ms_handle_reset on v2:172.21.15.150:6836/4017043144 2024-09-21T03:55:11.999 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:11.985+0000 7f3dbc9fa740 1 --2- 172.21.15.39:0/2876306599 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5652ee8741b0 0x5652eec4fd20 unknown :-1 s=CLOSED pgs=1202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:11.999 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:11.985+0000 7f3dbc9fa740 1 -- 172.21.15.39:0/2876306599 >> 172.21.15.39:0/2876306599 conn(0x5652ee891460 msgr2=0x5652eec47830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:55:11.999 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:11.986+0000 7f3dbc9fa740 1 -- 172.21.15.39:0/2876306599 shutdown_connections 2024-09-21T03:55:11.999 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:11.986+0000 7f3dbc9fa740 1 -- 172.21.15.39:0/2876306599 wait complete. 2024-09-21T03:55:12.000 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:11.986+0000 7f3dbc9fa740 20 client.17910 trim_cache size 0 max 16384 2024-09-21T03:55:12.002 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:11.999+0000 7f3dbc9fa740 1 Processor -- start 2024-09-21T03:55:12.003 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:11.999+0000 7f3dbc9fa740 1 -- start start 2024-09-21T03:55:12.003 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:11.999+0000 7f3dbc9fa740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5652ee8741b0 0x5652eec51950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:55:12.003 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:11.999+0000 7f3dbc9fa740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5652ee87c390 0x5652eec41bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:55:12.003 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:11.999+0000 7f3dbc9fa740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5652ee89d070 0x5652eec42100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:55:12.003 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:11.999+0000 7f3dbc9fa740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f3db01af7f0 con 0x5652ee87c390 2024-09-21T03:55:12.003 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:11.999+0000 7f3dbc9fa740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f3db01b1330 con 0x5652ee8741b0 2024-09-21T03:55:12.003 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:11.999+0000 7f3dbc9fa740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f3db01afec0 con 0x5652ee89d070 2024-09-21T03:55:12.003 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:11.999+0000 7f3da7fff640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5652ee87c390 0x5652eec41bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:55:12.003 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:11.999+0000 7f3da7fff640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5652ee87c390 0x5652eec41bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:39334/0 (socket says 172.21.15.39:39334) 2024-09-21T03:55:12.003 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:11.999+0000 7f3da7fff640 1 -- 172.21.15.39:0/2533725727 learned_addr learned my addr 172.21.15.39:0/2533725727 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:55:12.003 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:12.000+0000 7f3db4a23640 1 --2- 172.21.15.39:0/2533725727 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5652ee89d070 0x5652eec42100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:55:12.003 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:12.000+0000 7f3daffff640 1 --2- 172.21.15.39:0/2533725727 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5652ee8741b0 0x5652eec51950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:55:12.003 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:12.000+0000 7f3da7fff640 1 -- 172.21.15.39:0/2533725727 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5652ee89d070 msgr2=0x5652eec42100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:55:12.003 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:12.000+0000 7f3da7fff640 1 --2- 172.21.15.39:0/2533725727 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5652ee89d070 0x5652eec42100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:12.004 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:12.000+0000 7f3da7fff640 1 -- 172.21.15.39:0/2533725727 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5652ee8741b0 msgr2=0x5652eec51950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:55:12.004 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:12.000+0000 7f3da7fff640 1 --2- 172.21.15.39:0/2533725727 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5652ee8741b0 0x5652eec51950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:12.004 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:12.000+0000 7f3da7fff640 1 -- 172.21.15.39:0/2533725727 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5652eec4a9d0 con 0x5652ee87c390 2024-09-21T03:55:12.004 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:12.000+0000 7f3da7fff640 1 --2- 172.21.15.39:0/2533725727 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5652ee87c390 0x5652eec41bc0 secure :-1 s=READY pgs=1203 cs=0 l=1 rev1=1 crypto rx=0x7f3d980048f0 tx=0x7f3d98005c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c7739119cae592 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:55:12.006 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:12.000+0000 7f3da5ffb640 1 -- 172.21.15.39:0/2533725727 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d98004240 con 0x5652ee87c390 2024-09-21T03:55:12.006 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:12.000+0000 7f3da5ffb640 1 -- 172.21.15.39:0/2533725727 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3d980043e0 con 0x5652ee87c390 2024-09-21T03:55:12.006 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:12.000+0000 7f3da5ffb640 1 -- 172.21.15.39:0/2533725727 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d9802a6a0 con 0x5652ee87c390 2024-09-21T03:55:12.006 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:12.001+0000 7f3dbc9fa740 1 -- 172.21.15.39:0/2533725727 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5652ee87c390 msgr2=0x5652eec41bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:55:12.006 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:12.001+0000 7f3dbc9fa740 1 --2- 172.21.15.39:0/2533725727 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5652ee87c390 0x5652eec41bc0 secure :-1 s=READY pgs=1203 cs=0 l=1 rev1=1 crypto rx=0x7f3d980048f0 tx=0x7f3d98005c90 comp rx=0 tx=0).stop 2024-09-21T03:55:12.006 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:12.001+0000 7f3dbc9fa740 1 -- 172.21.15.39:0/2533725727 shutdown_connections 2024-09-21T03:55:12.006 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:12.001+0000 7f3dbc9fa740 1 --2- 172.21.15.39:0/2533725727 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5652ee89d070 0x5652eec42100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:12.007 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:12.001+0000 7f3dbc9fa740 1 --2- 172.21.15.39:0/2533725727 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5652ee87c390 0x5652eec41bc0 unknown :-1 s=CLOSED pgs=1203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:12.007 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:12.001+0000 7f3dbc9fa740 1 --2- 172.21.15.39:0/2533725727 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5652ee8741b0 0x5652eec51950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:12.007 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:12.001+0000 7f3dbc9fa740 1 -- 172.21.15.39:0/2533725727 >> 172.21.15.39:0/2533725727 conn(0x5652ee891460 msgr2=0x5652eec51240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:55:12.007 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:12.001+0000 7f3dbc9fa740 1 -- 172.21.15.39:0/2533725727 shutdown_connections 2024-09-21T03:55:12.007 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:12.001+0000 7f3dbc9fa740 1 -- 172.21.15.39:0/2533725727 wait complete. 2024-09-21T03:55:12.007 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:12.001+0000 7f3dbc9fa740 1 -- start start 2024-09-21T03:55:12.007 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:12.002+0000 7f3dbc9fa740 1 Processor -- start 2024-09-21T03:55:12.007 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:12.002+0000 7f3dbc9fa740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5652ee8741b0 0x5652eec58920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:55:12.007 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:12.002+0000 7f3dbc9fa740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5652ee87c390 0x5652eec53e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:55:12.007 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:12.002+0000 7f3dbc9fa740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5652ee89d070 0x5652eec543a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:55:12.007 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:12.002+0000 7f3dbc9fa740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f3db01b0c60 con 0x5652ee8741b0 2024-09-21T03:55:12.007 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:12.002+0000 7f3dbc9fa740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f3db01b0590 con 0x5652ee87c390 2024-09-21T03:55:12.007 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:12.002+0000 7f3dbc9fa740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f3db01ad7f0 con 0x5652ee89d070 2024-09-21T03:55:12.007 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:12.002+0000 7f3dbc9fa740 10 client.0 _ll_register_callbacks cb 0x7f3da0005d30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:55:12.007 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:12.002+0000 7f3daffff640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5652ee8741b0 0x5652eec58920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:55:12.008 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:12.002+0000 7f3daffff640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5652ee8741b0 0x5652eec58920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:39342/0 (socket says 172.21.15.39:39342) 2024-09-21T03:55:12.008 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:12.002+0000 7f3daffff640 1 -- 172.21.15.39:0/2178498781 learned_addr learned my addr 172.21.15.39:0/2178498781 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:55:12.008 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:12.002+0000 7f3da7fff640 1 --2- 172.21.15.39:0/2178498781 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5652ee87c390 0x5652eec53e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:55:12.008 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:12.002+0000 7f3db4a23640 1 --2- 172.21.15.39:0/2178498781 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5652ee89d070 0x5652eec543a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:55:12.008 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:12.003+0000 7f3daffff640 1 -- 172.21.15.39:0/2178498781 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5652ee89d070 msgr2=0x5652eec543a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:55:12.008 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:12.003+0000 7f3daffff640 1 --2- 172.21.15.39:0/2178498781 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5652ee89d070 0x5652eec543a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:12.008 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:12.003+0000 7f3daffff640 1 -- 172.21.15.39:0/2178498781 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5652ee87c390 msgr2=0x5652eec53e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:55:12.008 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:12.003+0000 7f3daffff640 1 --2- 172.21.15.39:0/2178498781 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5652ee87c390 0x5652eec53e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:12.008 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:12.003+0000 7f3daffff640 1 -- 172.21.15.39:0/2178498781 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5652eec54b30 con 0x5652ee8741b0 2024-09-21T03:55:12.008 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:12.003+0000 7f3daffff640 1 --2- 172.21.15.39:0/2178498781 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5652ee8741b0 0x5652eec58920 secure :-1 s=READY pgs=1204 cs=0 l=1 rev1=1 crypto rx=0x7f3da0019040 tx=0x7f3da000f670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=87112cf6d4fb78d7 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:55:12.008 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:12.003+0000 7f3d8f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-09-21T03:55:12.008 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:12.003+0000 7f3d8f7fe640 1 -- 172.21.15.39:0/2178498781 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3da001e7c0 con 0x5652ee8741b0 2024-09-21T03:55:12.008 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:12.003+0000 7f3dbc9fa740 1 -- 172.21.15.39:0/2178498781 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3d980043e0 con 0x5652ee8741b0 2024-09-21T03:55:12.008 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:12.003+0000 7f3d8f7fe640 1 -- 172.21.15.39:0/2178498781 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8572+0+0 (secure 0 0 0) 0x7f3da0032650 con 0x5652ee8741b0 2024-09-21T03:55:12.008 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:12.003+0000 7f3d8f7fe640 1 -- 172.21.15.39:0/2178498781 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f3da0032850 con 0x5652ee8741b0 2024-09-21T03:55:12.009 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:12.003+0000 7f3dbc9fa740 10 client.17916 fetch_fsmap learned FSMap version 29 2024-09-21T03:55:12.009 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:12.003+0000 7f3dbc9fa740 1 -- 172.21.15.39:0/2178498781 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5652eecd12e0 con 0x5652ee8741b0 2024-09-21T03:55:12.009 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:12.003+0000 7f3d8f7fe640 1 -- 172.21.15.39:0/2178498781 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3d980043e0 con 0x5652ee8741b0 2024-09-21T03:55:12.010 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:12.004+0000 7f3d8f7fe640 1 -- 172.21.15.39:0/2178498781 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f3da003b070 con 0x5652ee8741b0 2024-09-21T03:55:12.010 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:12.004+0000 7f3dbc9fa740 10 client.17916 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T03:55:12.010 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:12.004+0000 7f3dbc9fa740 10 client.17916 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-21T03:55:12.010 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:12.004+0000 7f3dbc9fa740 20 client.17916 populate_metadata read hostname 'smithi039' 2024-09-21T03:55:12.010 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:12.004+0000 7f3dbc9fa740 1 --2- 172.21.15.39:0/2178498781 >> [v2:172.21.15.150:6836/4017043144,v1:172.21.15.150:6837/4017043144] conn(0x5652ee830da0 0x5652eecd3a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:55:12.011 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:12.004+0000 7f3dbc9fa740 4 client.17916 mds_command: new command op to 24659 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T03:55:12.011 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:12.004+0000 7f3dbc9fa740 1 -- 172.21.15.39:0/2178498781 --> [v2:172.21.15.150:6836/4017043144,v1:172.21.15.150:6837/4017043144] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5652eec3d9e0 con 0x5652ee830da0 2024-09-21T03:55:12.011 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:12.004+0000 7f3da7fff640 1 --2- 172.21.15.39:0/2178498781 >> [v2:172.21.15.150:6836/4017043144,v1:172.21.15.150:6837/4017043144] conn(0x5652ee830da0 0x5652eecd3a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:55:12.011 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:12.004+0000 7f3da7fff640 1 --2- 172.21.15.39:0/2178498781 >> [v2:172.21.15.150:6836/4017043144,v1:172.21.15.150:6837/4017043144] conn(0x5652ee830da0 0x5652eecd3a60 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=55914a4c1505bc56 server_cookie=7dda0d89f5f7513a in_seq=0 out_seq=0 2024-09-21T03:55:12.011 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:12.004+0000 7f3d8f7fe640 10 client.17916 ms_handle_connect on v2:172.21.15.150:6836/4017043144 2024-09-21T03:55:12.011 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:12.005+0000 7f3d8f7fe640 1 -- 172.21.15.39:0/2178498781 <== mds.2 v2:172.21.15.150:6836/4017043144 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5652eec3d9e0 con 0x5652ee830da0 2024-09-21T03:55:12.011 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:12.005+0000 7f3d8f7fe640 10 client.17916 handle_command_reply: tid=0 2024-09-21T03:55:12.011 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:12.005+0000 7f3dbc9fa740 1 client.17916 shutdown 2024-09-21T03:55:12.011 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:12.005+0000 7f3dbc9fa740 1 -- 172.21.15.39:0/2178498781 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5652ee8741b0 msgr2=0x5652eec58920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:55:12.011 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:12.005+0000 7f3dbc9fa740 1 --2- 172.21.15.39:0/2178498781 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5652ee8741b0 0x5652eec58920 secure :-1 s=READY pgs=1204 cs=0 l=1 rev1=1 crypto rx=0x7f3da0019040 tx=0x7f3da000f670 comp rx=0 tx=0).stop 2024-09-21T03:55:12.011 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:12.005+0000 7f3dbc9fa740 1 -- 172.21.15.39:0/2178498781 shutdown_connections 2024-09-21T03:55:12.011 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:12.005+0000 7f3dbc9fa740 1 --2- 172.21.15.39:0/2178498781 >> [v2:172.21.15.150:6836/4017043144,v1:172.21.15.150:6837/4017043144] conn(0x5652ee830da0 0x5652eecd3a60 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:12.011 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:12.005+0000 7f3dbc9fa740 1 --2- 172.21.15.39:0/2178498781 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5652ee89d070 0x5652eec543a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:12.012 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:12.005+0000 7f3dbc9fa740 1 --2- 172.21.15.39:0/2178498781 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5652ee87c390 0x5652eec53e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:12.012 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:12.005+0000 7f3d8f7fe640 0 client.17916 ms_handle_reset on v2:172.21.15.150:6836/4017043144 2024-09-21T03:55:12.012 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:12.005+0000 7f3dbc9fa740 1 --2- 172.21.15.39:0/2178498781 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5652ee8741b0 0x5652eec58920 unknown :-1 s=CLOSED pgs=1204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:12.012 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:12.005+0000 7f3dbc9fa740 1 -- 172.21.15.39:0/2178498781 >> 172.21.15.39:0/2178498781 conn(0x5652ee891460 msgr2=0x5652eec511d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:55:12.012 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:12.005+0000 7f3dbc9fa740 1 -- 172.21.15.39:0/2178498781 shutdown_connections 2024-09-21T03:55:12.012 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:12.005+0000 7f3dbc9fa740 1 -- 172.21.15.39:0/2178498781 wait complete. 2024-09-21T03:55:12.012 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:12.005+0000 7f3dbc9fa740 20 client.17916 trim_cache size 0 max 16384 2024-09-21T03:55:12.012 INFO:teuthology.orchestra.run.smithi039.stdout:[] 2024-09-21T03:55:12.013 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:12.006+0000 7f3db6486640 1 -- 172.21.15.39:0/1162706936 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f3d9c0b6680 msgr2=0x7f3d9c0b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:55:12.013 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:12.006+0000 7f3db6486640 1 --2- 172.21.15.39:0/1162706936 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f3d9c0b6680 0x7f3d9c0b8b90 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x5652eec43320 tx=0x7f3da0005f90 comp rx=0 tx=0).stop 2024-09-21T03:55:12.013 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:12.006+0000 7f3db6486640 1 -- 172.21.15.39:0/1162706936 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3db0109e20 msgr2=0x7f3db019fcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:55:12.013 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:12.006+0000 7f3db6486640 1 --2- 172.21.15.39:0/1162706936 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3db0109e20 0x7f3db019fcb0 secure :-1 s=READY pgs=1200 cs=0 l=1 rev1=1 crypto rx=0x7f3da80098d0 tx=0x7f3da80317b0 comp rx=0 tx=0).stop 2024-09-21T03:55:12.013 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:12.006+0000 7f3db6486640 1 -- 172.21.15.39:0/1162706936 shutdown_connections 2024-09-21T03:55:12.013 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:12.006+0000 7f3db6486640 1 --2- 172.21.15.39:0/1162706936 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f3d9c0b6680 0x7f3d9c0b8b90 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:12.013 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:12.006+0000 7f3db6486640 1 --2- 172.21.15.39:0/1162706936 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3db0109e20 0x7f3db019fcb0 unknown :-1 s=CLOSED pgs=1200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:12.013 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:12.006+0000 7f3db6486640 1 --2- 172.21.15.39:0/1162706936 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3db01094d0 0x7f3db019aa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:12.013 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:12.006+0000 7f3db6486640 1 --2- 172.21.15.39:0/1162706936 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3db0103b90 0x7f3db019a470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:12.013 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:12.006+0000 7f3db6486640 1 -- 172.21.15.39:0/1162706936 >> 172.21.15.39:0/1162706936 conn(0x7f3db00ff600 msgr2=0x7f3db0106aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:55:12.013 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:12.006+0000 7f3db6486640 1 -- 172.21.15.39:0/1162706936 shutdown_connections 2024-09-21T03:55:12.014 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:12.006+0000 7f3db6486640 1 -- 172.21.15.39:0/1162706936 wait complete. 2024-09-21T03:55:12.874 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 51116d5c-77ca-11ef-baf5-efdc52797490 -- ceph tell mds.1:1 damage ls 2024-09-21T03:55:13.140 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/51116d5c-77ca-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:55:13.262 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:55:13 smithi039 ceph-mon[31211]: pgmap v605: 97 pgs: 97 active+clean; 10 MiB data, 525 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 53 KiB/s wr, 30 op/s 2024-09-21T03:55:13.468 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:55:13 smithi161 ceph-mon[34533]: pgmap v605: 97 pgs: 97 active+clean; 10 MiB data, 525 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 53 KiB/s wr, 30 op/s 2024-09-21T03:55:13.474 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:55:13 smithi150 ceph-mon[34289]: pgmap v605: 97 pgs: 97 active+clean; 10 MiB data, 525 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 53 KiB/s wr, 30 op/s 2024-09-21T03:55:13.650 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:13.634+0000 7f83a65bb640 1 -- 172.21.15.39:0/4256602377 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f83a01094a0 msgr2=0x7f83a01098b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:55:13.650 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:13.634+0000 7f83a65bb640 1 --2- 172.21.15.39:0/4256602377 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f83a01094a0 0x7f83a01098b0 secure :-1 s=READY pgs=1205 cs=0 l=1 rev1=1 crypto rx=0x7f8394009900 tx=0x7f839402a3f0 comp rx=0 tx=0).stop 2024-09-21T03:55:13.651 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:13.634+0000 7f83a65bb640 1 -- 172.21.15.39:0/4256602377 shutdown_connections 2024-09-21T03:55:13.651 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:13.634+0000 7f83a65bb640 1 --2- 172.21.15.39:0/4256602377 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f83a0109df0 0x7f83a0110620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:13.651 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:13.634+0000 7f83a65bb640 1 --2- 172.21.15.39:0/4256602377 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f83a01094a0 0x7f83a01098b0 unknown :-1 s=CLOSED pgs=1205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:13.651 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:13.634+0000 7f83a65bb640 1 --2- 172.21.15.39:0/4256602377 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f83a0103b60 0x7f83a0103f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:13.651 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:13.634+0000 7f83a65bb640 1 -- 172.21.15.39:0/4256602377 >> 172.21.15.39:0/4256602377 conn(0x7f83a007b850 msgr2=0x7f83a007bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:55:13.651 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:13.634+0000 7f83a65bb640 1 -- 172.21.15.39:0/4256602377 shutdown_connections 2024-09-21T03:55:13.651 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:13.634+0000 7f83a65bb640 1 -- 172.21.15.39:0/4256602377 wait complete. 2024-09-21T03:55:13.651 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:13.635+0000 7f83a65bb640 1 Processor -- start 2024-09-21T03:55:13.651 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:13.635+0000 7f83a65bb640 1 -- start start 2024-09-21T03:55:13.651 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:13.636+0000 7f83a65bb640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f83a0103b60 0x7f83a019a580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:55:13.651 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:13.636+0000 7f83a65bb640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f83a01094a0 0x7f83a019ab20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:55:13.651 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:13.636+0000 7f83a65bb640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f83a0109df0 0x7f83a019fdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:55:13.651 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:13.636+0000 7f83a65bb640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f83a0113ab0 con 0x7f83a01094a0 2024-09-21T03:55:13.652 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:13.636+0000 7f83a65bb640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f83a0113930 con 0x7f83a0103b60 2024-09-21T03:55:13.652 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:13.636+0000 7f83a65bb640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f83a0113c30 con 0x7f83a0109df0 2024-09-21T03:55:13.652 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:13.636+0000 7f839f7fe640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f83a01094a0 0x7f83a019ab20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:55:13.652 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:13.636+0000 7f839f7fe640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f83a01094a0 0x7f83a019ab20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:39354/0 (socket says 172.21.15.39:39354) 2024-09-21T03:55:13.652 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:13.636+0000 7f839f7fe640 1 -- 172.21.15.39:0/3345910793 learned_addr learned my addr 172.21.15.39:0/3345910793 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:55:13.652 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:13.636+0000 7f839ffff640 1 --2- 172.21.15.39:0/3345910793 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f83a0103b60 0x7f83a019a580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:55:13.652 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:13.636+0000 7f83a4b58640 1 --2- 172.21.15.39:0/3345910793 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f83a0109df0 0x7f83a019fdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:55:13.652 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:13.636+0000 7f839f7fe640 1 -- 172.21.15.39:0/3345910793 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f83a0109df0 msgr2=0x7f83a019fdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:55:13.652 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:13.636+0000 7f839f7fe640 1 --2- 172.21.15.39:0/3345910793 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f83a0109df0 0x7f83a019fdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:13.652 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:13.636+0000 7f839f7fe640 1 -- 172.21.15.39:0/3345910793 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f83a0103b60 msgr2=0x7f83a019a580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:55:13.652 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:13.636+0000 7f839f7fe640 1 --2- 172.21.15.39:0/3345910793 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f83a0103b60 0x7f83a019a580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:13.652 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:13.636+0000 7f839f7fe640 1 -- 172.21.15.39:0/3345910793 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f83a01a0430 con 0x7f83a01094a0 2024-09-21T03:55:13.652 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:13.636+0000 7f839f7fe640 1 --2- 172.21.15.39:0/3345910793 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f83a01094a0 0x7f83a019ab20 secure :-1 s=READY pgs=1206 cs=0 l=1 rev1=1 crypto rx=0x7f839402a910 tx=0x7f8394004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:55:13.652 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:13.637+0000 7f839d7fa640 1 -- 172.21.15.39:0/3345910793 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f839403f030 con 0x7f83a01094a0 2024-09-21T03:55:13.653 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:13.637+0000 7f83a65bb640 1 -- 172.21.15.39:0/3345910793 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f83a01a06c0 con 0x7f83a01094a0 2024-09-21T03:55:13.653 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:13.637+0000 7f839d7fa640 1 -- 172.21.15.39:0/3345910793 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8394003cb0 con 0x7f83a01094a0 2024-09-21T03:55:13.653 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:13.637+0000 7f839d7fa640 1 -- 172.21.15.39:0/3345910793 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8394035070 con 0x7f83a01094a0 2024-09-21T03:55:13.653 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:13.637+0000 7f83a65bb640 1 -- 172.21.15.39:0/3345910793 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f83a01a0bf0 con 0x7f83a01094a0 2024-09-21T03:55:13.653 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:13.639+0000 7f839d7fa640 1 -- 172.21.15.39:0/3345910793 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 27) ==== 154643+0+0 (secure 0 0 0) 0x7f8394042030 con 0x7f83a01094a0 2024-09-21T03:55:13.653 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:13.639+0000 7f83acb2f740 1 Processor -- start 2024-09-21T03:55:13.653 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:13.640+0000 7f83acb2f740 1 -- start start 2024-09-21T03:55:13.653 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:13.640+0000 7f83acb2f740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5555c315c070 0x5555c35036a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:55:13.653 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:13.640+0000 7f83acb2f740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5555c31331b0 0x5555c3503be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:55:13.653 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:13.640+0000 7f83acb2f740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5555c313b390 0x5555c3504120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:55:13.653 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:13.640+0000 7f83acb2f740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x5555c3479ea0 con 0x5555c31331b0 2024-09-21T03:55:13.653 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:13.640+0000 7f83acb2f740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x5555c34692d0 con 0x5555c313b390 2024-09-21T03:55:13.653 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:13.640+0000 7f83acb2f740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x5555c346d0e0 con 0x5555c315c070 2024-09-21T03:55:13.653 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:13.640+0000 7f839ffff640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5555c31331b0 0x5555c3503be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:55:13.654 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:13.640+0000 7f83a4b58640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5555c315c070 0x5555c35036a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:55:13.654 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:13.640+0000 7f83a4b58640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5555c315c070 0x5555c35036a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.39:44532/0 (socket says 172.21.15.39:44532) 2024-09-21T03:55:13.654 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:13.640+0000 7f83a4b58640 1 -- 172.21.15.39:0/2822732647 learned_addr learned my addr 172.21.15.39:0/2822732647 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:55:13.654 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:13.640+0000 7f839ffff640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5555c31331b0 0x5555c3503be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:39364/0 (socket says 172.21.15.39:39364) 2024-09-21T03:55:13.654 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:13.640+0000 7f839f7fe640 1 --2- 172.21.15.39:0/2822732647 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5555c313b390 0x5555c3504120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:55:13.654 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:13.640+0000 7f839d7fa640 1 --2- 172.21.15.39:0/3345910793 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f83880b66b0 0x7f83880b8bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:55:13.654 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:13.640+0000 7f839ffff640 1 -- 172.21.15.39:0/2822732647 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5555c315c070 msgr2=0x5555c35036a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:55:13.654 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:13.640+0000 7f839ffff640 1 --2- 172.21.15.39:0/2822732647 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5555c315c070 0x5555c35036a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:13.654 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:13.640+0000 7f839ffff640 1 -- 172.21.15.39:0/2822732647 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5555c313b390 msgr2=0x5555c3504120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:55:13.654 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:13.640+0000 7f839ffff640 1 --2- 172.21.15.39:0/2822732647 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5555c313b390 0x5555c3504120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:13.654 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:13.640+0000 7f839ffff640 1 -- 172.21.15.39:0/2822732647 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5555c31c7650 con 0x5555c31331b0 2024-09-21T03:55:13.655 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:13.641+0000 7f83a4b58640 1 --2- 172.21.15.39:0/2822732647 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5555c315c070 0x5555c35036a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:55:13.655 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:13.641+0000 7f83a4b58640 1 --2- 172.21.15.39:0/3345910793 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f83880b66b0 0x7f83880b8bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:55:13.655 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:13.641+0000 7f839d7fa640 1 -- 172.21.15.39:0/3345910793 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8572+0+0 (secure 0 0 0) 0x7f8394107d60 con 0x7f83a01094a0 2024-09-21T03:55:13.655 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:13.641+0000 7f839ffff640 1 --2- 172.21.15.39:0/2822732647 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5555c31331b0 0x5555c3503be0 secure :-1 s=READY pgs=1207 cs=0 l=1 rev1=1 crypto rx=0x7f839000ab40 tx=0x7f839000f3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=96268d5ba17d1482 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:55:13.655 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:13.641+0000 7f83867fc640 1 -- 172.21.15.39:0/2822732647 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8390004020 con 0x5555c31331b0 2024-09-21T03:55:13.655 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:13.641+0000 7f83867fc640 1 -- 172.21.15.39:0/2822732647 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f83900041c0 con 0x5555c31331b0 2024-09-21T03:55:13.655 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:13.641+0000 7f83867fc640 1 -- 172.21.15.39:0/2822732647 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8390018730 con 0x5555c31331b0 2024-09-21T03:55:13.655 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:13.641+0000 7f83a4b58640 1 --2- 172.21.15.39:0/3345910793 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f83880b66b0 0x7f83880b8bc0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f8398011580 tx=0x7f8398004a40 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:55:13.655 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:13.642+0000 7f83acb2f740 1 -- 172.21.15.39:0/2822732647 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5555c31331b0 msgr2=0x5555c3503be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:55:13.655 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:13.642+0000 7f83acb2f740 1 --2- 172.21.15.39:0/2822732647 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5555c31331b0 0x5555c3503be0 secure :-1 s=READY pgs=1207 cs=0 l=1 rev1=1 crypto rx=0x7f839000ab40 tx=0x7f839000f3c0 comp rx=0 tx=0).stop 2024-09-21T03:55:13.656 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:13.642+0000 7f83acb2f740 1 -- 172.21.15.39:0/2822732647 shutdown_connections 2024-09-21T03:55:13.656 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:13.642+0000 7f83acb2f740 1 --2- 172.21.15.39:0/2822732647 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5555c313b390 0x5555c3504120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:13.656 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:13.642+0000 7f83acb2f740 1 --2- 172.21.15.39:0/2822732647 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5555c31331b0 0x5555c3503be0 unknown :-1 s=CLOSED pgs=1207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:13.656 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:13.642+0000 7f83acb2f740 1 --2- 172.21.15.39:0/2822732647 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5555c315c070 0x5555c35036a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:13.656 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:13.642+0000 7f83acb2f740 1 -- 172.21.15.39:0/2822732647 >> 172.21.15.39:0/2822732647 conn(0x5555c3150460 msgr2=0x5555c3501590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:55:13.656 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:13.642+0000 7f83acb2f740 1 -- 172.21.15.39:0/2822732647 shutdown_connections 2024-09-21T03:55:13.656 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:13.642+0000 7f83acb2f740 1 -- 172.21.15.39:0/2822732647 wait complete. 2024-09-21T03:55:13.656 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:13.642+0000 7f83acb2f740 1 -- start start 2024-09-21T03:55:13.657 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:13.643+0000 7f83acb2f740 1 Processor -- start 2024-09-21T03:55:13.657 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:13.643+0000 7f83acb2f740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5555c31331b0 0x5555c350ed40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:55:13.657 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:13.643+0000 7f83acb2f740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5555c313b390 0x5555c350f280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:55:13.657 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:13.643+0000 7f83acb2f740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5555c315c070 0x5555c351b170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:55:13.657 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:13.643+0000 7f83acb2f740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x5555c32ca700 con 0x5555c313b390 2024-09-21T03:55:13.657 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:13.643+0000 7f83acb2f740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x5555c3404160 con 0x5555c31331b0 2024-09-21T03:55:13.657 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:13.643+0000 7f83acb2f740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x5555c3511e80 con 0x5555c315c070 2024-09-21T03:55:13.657 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:13.643+0000 7f83acb2f740 10 client.0 _ll_register_callbacks cb 0x5555c341d9e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:55:13.657 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:13.643+0000 7f839f7fe640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5555c313b390 0x5555c350f280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:55:13.657 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:13.643+0000 7f839f7fe640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5555c313b390 0x5555c350f280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:39378/0 (socket says 172.21.15.39:39378) 2024-09-21T03:55:13.657 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:13.643+0000 7f839f7fe640 1 -- 172.21.15.39:0/2543592816 learned_addr learned my addr 172.21.15.39:0/2543592816 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:55:13.657 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:13.643+0000 7f83a4b58640 1 --2- 172.21.15.39:0/2543592816 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5555c315c070 0x5555c351b170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:55:13.658 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:13.643+0000 7f839ffff640 1 --2- 172.21.15.39:0/2543592816 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5555c31331b0 0x5555c350ed40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:55:13.658 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:13.643+0000 7f839f7fe640 1 -- 172.21.15.39:0/2543592816 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5555c315c070 msgr2=0x5555c351b170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:55:13.658 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:13.643+0000 7f839f7fe640 1 --2- 172.21.15.39:0/2543592816 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5555c315c070 0x5555c351b170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:13.658 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:13.643+0000 7f839f7fe640 1 -- 172.21.15.39:0/2543592816 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5555c31331b0 msgr2=0x5555c350ed40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:55:13.658 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:13.643+0000 7f839f7fe640 1 --2- 172.21.15.39:0/2543592816 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5555c31331b0 0x5555c350ed40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:13.658 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:13.644+0000 7f839f7fe640 1 -- 172.21.15.39:0/2543592816 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5555c3431c90 con 0x5555c313b390 2024-09-21T03:55:13.658 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:13.644+0000 7f839f7fe640 1 --2- 172.21.15.39:0/2543592816 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5555c313b390 0x5555c350f280 secure :-1 s=READY pgs=1208 cs=0 l=1 rev1=1 crypto rx=0x7f8394105a80 tx=0x7f83940c9be0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=890810313ca01944 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:55:13.659 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:13.644+0000 7f837f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-09-21T03:55:13.659 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:13.644+0000 7f837f7fe640 1 -- 172.21.15.39:0/2543592816 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8394049070 con 0x5555c313b390 2024-09-21T03:55:13.659 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:13.644+0000 7f837f7fe640 1 -- 172.21.15.39:0/2543592816 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8572+0+0 (secure 0 0 0) 0x7f8394105bd0 con 0x5555c313b390 2024-09-21T03:55:13.659 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:13.644+0000 7f83acb2f740 1 -- 172.21.15.39:0/2543592816 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f83900041c0 con 0x5555c313b390 2024-09-21T03:55:13.659 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:13.644+0000 7f837f7fe640 1 -- 172.21.15.39:0/2543592816 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f83940c9a20 con 0x5555c313b390 2024-09-21T03:55:13.659 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:13.644+0000 7f83acb2f740 10 client.17928 fetch_fsmap learned FSMap version 29 2024-09-21T03:55:13.659 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:13.644+0000 7f83acb2f740 1 -- 172.21.15.39:0/2543592816 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5555c351db70 con 0x5555c313b390 2024-09-21T03:55:13.659 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:13.644+0000 7f837f7fe640 1 -- 172.21.15.39:0/2543592816 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f83900041c0 con 0x5555c313b390 2024-09-21T03:55:13.661 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:13.645+0000 7f837f7fe640 1 -- 172.21.15.39:0/2543592816 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f83940d7110 con 0x5555c313b390 2024-09-21T03:55:13.661 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:13.645+0000 7f83acb2f740 10 client.17928 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T03:55:13.661 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:13.645+0000 7f83acb2f740 10 client.17928 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-21T03:55:13.662 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:13.645+0000 7f83acb2f740 20 client.17928 populate_metadata read hostname 'smithi039' 2024-09-21T03:55:13.662 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:13.645+0000 7f83acb2f740 1 --2- 172.21.15.39:0/2543592816 >> [v2:172.21.15.161:6838/2190107270,v1:172.21.15.161:6839/2190107270] conn(0x5555c311a9f0 0x5555c35210e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:55:13.662 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:13.645+0000 7f83acb2f740 4 client.17928 mds_command: new command op to 24667 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:55:13.662 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:13.645+0000 7f83acb2f740 1 -- 172.21.15.39:0/2543592816 --> [v2:172.21.15.161:6838/2190107270,v1:172.21.15.161:6839/2190107270] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5555c347bdf0 con 0x5555c311a9f0 2024-09-21T03:55:13.662 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:13.645+0000 7f839ffff640 1 --2- 172.21.15.39:0/2543592816 >> [v2:172.21.15.161:6838/2190107270,v1:172.21.15.161:6839/2190107270] conn(0x5555c311a9f0 0x5555c35210e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:55:13.662 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:13.646+0000 7f839ffff640 1 --2- 172.21.15.39:0/2543592816 >> [v2:172.21.15.161:6838/2190107270,v1:172.21.15.161:6839/2190107270] conn(0x5555c311a9f0 0x5555c35210e0 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=f79bd1904f4b5564 server_cookie=5c4bfcfc106dbbc3 in_seq=0 out_seq=0 2024-09-21T03:55:13.662 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:13.646+0000 7f837f7fe640 10 client.17928 ms_handle_connect on v2:172.21.15.161:6838/2190107270 2024-09-21T03:55:13.662 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:13.646+0000 7f837f7fe640 1 -- 172.21.15.39:0/2543592816 <== mds.1 v2:172.21.15.161:6838/2190107270 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5555c347bdf0 con 0x5555c311a9f0 2024-09-21T03:55:13.662 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:13.646+0000 7f837f7fe640 10 client.17928 handle_command_reply: tid=0 2024-09-21T03:55:13.662 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:13.646+0000 7f83acb2f740 1 client.17928 shutdown 2024-09-21T03:55:13.662 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:13.646+0000 7f83acb2f740 1 -- 172.21.15.39:0/2543592816 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5555c313b390 msgr2=0x5555c350f280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:55:13.662 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:13.646+0000 7f83acb2f740 1 --2- 172.21.15.39:0/2543592816 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5555c313b390 0x5555c350f280 secure :-1 s=READY pgs=1208 cs=0 l=1 rev1=1 crypto rx=0x7f8394105a80 tx=0x7f83940c9be0 comp rx=0 tx=0).stop 2024-09-21T03:55:13.662 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:13.647+0000 7f83acb2f740 1 -- 172.21.15.39:0/2543592816 shutdown_connections 2024-09-21T03:55:13.663 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:13.647+0000 7f83acb2f740 1 --2- 172.21.15.39:0/2543592816 >> [v2:172.21.15.161:6838/2190107270,v1:172.21.15.161:6839/2190107270] conn(0x5555c311a9f0 0x5555c35210e0 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:13.663 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:13.647+0000 7f83acb2f740 1 --2- 172.21.15.39:0/2543592816 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5555c315c070 0x5555c351b170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:13.663 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:13.647+0000 7f83acb2f740 1 --2- 172.21.15.39:0/2543592816 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5555c313b390 0x5555c350f280 unknown :-1 s=CLOSED pgs=1208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:13.663 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:13.647+0000 7f83acb2f740 1 --2- 172.21.15.39:0/2543592816 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5555c31331b0 0x5555c350ed40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:13.663 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:13.647+0000 7f83acb2f740 1 -- 172.21.15.39:0/2543592816 >> 172.21.15.39:0/2543592816 conn(0x5555c3150460 msgr2=0x5555c3506830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:55:13.663 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:13.647+0000 7f837f7fe640 0 client.17928 ms_handle_reset on v2:172.21.15.161:6838/2190107270 2024-09-21T03:55:13.663 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:13.647+0000 7f83acb2f740 1 -- 172.21.15.39:0/2543592816 shutdown_connections 2024-09-21T03:55:13.663 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:13.647+0000 7f83acb2f740 1 -- 172.21.15.39:0/2543592816 wait complete. 2024-09-21T03:55:13.663 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:13.647+0000 7f83acb2f740 20 client.17928 trim_cache size 0 max 16384 2024-09-21T03:55:13.664 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:13.660+0000 7f83acb2f740 1 Processor -- start 2024-09-21T03:55:13.664 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:13.660+0000 7f83acb2f740 1 -- start start 2024-09-21T03:55:13.665 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:13.660+0000 7f83acb2f740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5555c31331b0 0x5555c3510970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:55:13.665 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:13.660+0000 7f83acb2f740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5555c315c070 0x5555c3500bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:55:13.665 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:13.660+0000 7f83acb2f740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5555c30efda0 0x5555c3501100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:55:13.665 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:13.660+0000 7f83acb2f740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f83a01af910 con 0x5555c31331b0 2024-09-21T03:55:13.665 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:13.660+0000 7f83acb2f740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f83a01b1450 con 0x5555c30efda0 2024-09-21T03:55:13.665 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:13.660+0000 7f83acb2f740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f83a01affe0 con 0x5555c315c070 2024-09-21T03:55:13.665 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:13.661+0000 7f839ffff640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5555c31331b0 0x5555c3510970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:55:13.665 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:13.661+0000 7f839ffff640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5555c31331b0 0x5555c3510970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:39384/0 (socket says 172.21.15.39:39384) 2024-09-21T03:55:13.665 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:13.661+0000 7f839ffff640 1 -- 172.21.15.39:0/3721631984 learned_addr learned my addr 172.21.15.39:0/3721631984 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:55:13.665 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:13.661+0000 7f83a4b58640 1 --2- 172.21.15.39:0/3721631984 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5555c30efda0 0x5555c3501100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:55:13.665 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:13.661+0000 7f839f7fe640 1 --2- 172.21.15.39:0/3721631984 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5555c315c070 0x5555c3500bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:55:13.665 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:13.661+0000 7f839ffff640 1 -- 172.21.15.39:0/3721631984 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5555c315c070 msgr2=0x5555c3500bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:55:13.665 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:13.661+0000 7f839ffff640 1 --2- 172.21.15.39:0/3721631984 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5555c315c070 0x5555c3500bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:13.665 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:13.661+0000 7f839ffff640 1 -- 172.21.15.39:0/3721631984 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5555c30efda0 msgr2=0x5555c3501100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:55:13.666 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:13.661+0000 7f839ffff640 1 --2- 172.21.15.39:0/3721631984 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5555c30efda0 0x5555c3501100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:13.666 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:13.661+0000 7f839ffff640 1 -- 172.21.15.39:0/3721631984 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5555c3510000 con 0x5555c31331b0 2024-09-21T03:55:13.666 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:13.661+0000 7f83a4b58640 1 --2- 172.21.15.39:0/3721631984 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5555c30efda0 0x5555c3501100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:55:13.670 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:13.661+0000 7f839ffff640 1 --2- 172.21.15.39:0/3721631984 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5555c31331b0 0x5555c3510970 secure :-1 s=READY pgs=1209 cs=0 l=1 rev1=1 crypto rx=0x5555c3502320 tx=0x7f839000a2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=69f0d18bdad8eb62 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:55:13.670 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:13.661+0000 7f8385ffb640 1 -- 172.21.15.39:0/3721631984 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8390030070 con 0x5555c31331b0 2024-09-21T03:55:13.670 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:13.661+0000 7f8385ffb640 1 -- 172.21.15.39:0/3721631984 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f839000a700 con 0x5555c31331b0 2024-09-21T03:55:13.670 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:13.661+0000 7f8385ffb640 1 -- 172.21.15.39:0/3721631984 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83900035f0 con 0x5555c31331b0 2024-09-21T03:55:13.671 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:13.662+0000 7f83acb2f740 1 -- 172.21.15.39:0/3721631984 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5555c31331b0 msgr2=0x5555c3510970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:55:13.671 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:13.662+0000 7f83acb2f740 1 --2- 172.21.15.39:0/3721631984 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5555c31331b0 0x5555c3510970 secure :-1 s=READY pgs=1209 cs=0 l=1 rev1=1 crypto rx=0x5555c3502320 tx=0x7f839000a2d0 comp rx=0 tx=0).stop 2024-09-21T03:55:13.671 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:13.662+0000 7f83acb2f740 1 -- 172.21.15.39:0/3721631984 shutdown_connections 2024-09-21T03:55:13.671 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:13.662+0000 7f83acb2f740 1 --2- 172.21.15.39:0/3721631984 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5555c30efda0 0x5555c3501100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:13.671 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:13.662+0000 7f83acb2f740 1 --2- 172.21.15.39:0/3721631984 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5555c315c070 0x5555c3500bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:13.671 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:13.662+0000 7f83acb2f740 1 --2- 172.21.15.39:0/3721631984 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5555c31331b0 0x5555c3510970 unknown :-1 s=CLOSED pgs=1209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:13.671 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:13.662+0000 7f83acb2f740 1 -- 172.21.15.39:0/3721631984 >> 172.21.15.39:0/3721631984 conn(0x5555c3150460 msgr2=0x5555c3510260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:55:13.671 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:13.662+0000 7f83acb2f740 1 -- 172.21.15.39:0/3721631984 shutdown_connections 2024-09-21T03:55:13.671 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:13.662+0000 7f83acb2f740 1 -- 172.21.15.39:0/3721631984 wait complete. 2024-09-21T03:55:13.671 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:13.663+0000 7f83acb2f740 1 -- start start 2024-09-21T03:55:13.671 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:13.663+0000 7f83acb2f740 1 Processor -- start 2024-09-21T03:55:13.671 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:13.663+0000 7f83acb2f740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5555c30efda0 0x5555c35131b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:55:13.672 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:13.663+0000 7f83acb2f740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5555c31331b0 0x5555c35136f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:55:13.672 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:13.663+0000 7f83acb2f740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5555c315c070 0x5555c3513c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:55:13.672 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:13.663+0000 7f83acb2f740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f83a01b0d80 con 0x5555c30efda0 2024-09-21T03:55:13.672 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:13.663+0000 7f83acb2f740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f83a01b06b0 con 0x5555c31331b0 2024-09-21T03:55:13.672 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:13.663+0000 7f83acb2f740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f83a01ad910 con 0x5555c315c070 2024-09-21T03:55:13.672 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:13.663+0000 7f839ffff640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5555c30efda0 0x5555c35131b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:55:13.672 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:13.663+0000 7f83acb2f740 10 client.0 _ll_register_callbacks cb 0x7f8394005ef0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:55:13.672 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:13.663+0000 7f839ffff640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5555c30efda0 0x5555c35131b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:39398/0 (socket says 172.21.15.39:39398) 2024-09-21T03:55:13.672 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:13.663+0000 7f839ffff640 1 -- 172.21.15.39:0/59229558 learned_addr learned my addr 172.21.15.39:0/59229558 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:55:13.672 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:13.664+0000 7f839f7fe640 1 --2- 172.21.15.39:0/59229558 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5555c31331b0 0x5555c35136f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:55:13.672 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:13.664+0000 7f83a4b58640 1 --2- 172.21.15.39:0/59229558 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5555c315c070 0x5555c3513c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:55:13.673 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:13.664+0000 7f839ffff640 1 -- 172.21.15.39:0/59229558 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5555c315c070 msgr2=0x5555c3513c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:55:13.673 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:13.664+0000 7f839ffff640 1 --2- 172.21.15.39:0/59229558 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5555c315c070 0x5555c3513c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:13.673 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:13.664+0000 7f839ffff640 1 -- 172.21.15.39:0/59229558 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5555c31331b0 msgr2=0x5555c35136f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:55:13.673 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:13.664+0000 7f839ffff640 1 --2- 172.21.15.39:0/59229558 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5555c31331b0 0x5555c35136f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:13.673 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:13.664+0000 7f839ffff640 1 -- 172.21.15.39:0/59229558 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5555c358aba0 con 0x5555c30efda0 2024-09-21T03:55:13.673 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:13.664+0000 7f839ffff640 1 --2- 172.21.15.39:0/59229558 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5555c30efda0 0x5555c35131b0 secure :-1 s=READY pgs=1210 cs=0 l=1 rev1=1 crypto rx=0x7f8390005530 tx=0x7f8390005610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad206a83a1e2d349 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:55:13.673 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:13.664+0000 7f837f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-09-21T03:55:13.673 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:13.664+0000 7f837f7fe640 1 -- 172.21.15.39:0/59229558 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8390030070 con 0x5555c30efda0 2024-09-21T03:55:13.673 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:13.664+0000 7f837f7fe640 1 -- 172.21.15.39:0/59229558 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8572+0+0 (secure 0 0 0) 0x7f83900041a0 con 0x5555c30efda0 2024-09-21T03:55:13.673 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:13.664+0000 7f83acb2f740 1 -- 172.21.15.39:0/59229558 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f839000a700 con 0x5555c30efda0 2024-09-21T03:55:13.674 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:13.664+0000 7f837f7fe640 1 -- 172.21.15.39:0/59229558 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f83900035f0 con 0x5555c30efda0 2024-09-21T03:55:13.674 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:13.664+0000 7f83acb2f740 10 client.17934 fetch_fsmap learned FSMap version 29 2024-09-21T03:55:13.674 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:13.664+0000 7f83acb2f740 1 -- 172.21.15.39:0/59229558 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5555c358be20 con 0x5555c30efda0 2024-09-21T03:55:13.674 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:13.664+0000 7f837f7fe640 1 -- 172.21.15.39:0/59229558 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f839000a700 con 0x5555c30efda0 2024-09-21T03:55:13.674 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:13.665+0000 7f837f7fe640 1 -- 172.21.15.39:0/59229558 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f83900189a0 con 0x5555c30efda0 2024-09-21T03:55:13.674 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:13.665+0000 7f83acb2f740 10 client.17934 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T03:55:13.674 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:13.665+0000 7f83acb2f740 10 client.17934 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-21T03:55:13.674 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:13.665+0000 7f83acb2f740 20 client.17934 populate_metadata read hostname 'smithi039' 2024-09-21T03:55:13.674 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:13.665+0000 7f83acb2f740 1 --2- 172.21.15.39:0/59229558 >> [v2:172.21.15.161:6838/2190107270,v1:172.21.15.161:6839/2190107270] conn(0x5555c358c290 0x5555c358e6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:55:13.674 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:13.665+0000 7f83acb2f740 4 client.17934 mds_command: new command op to 24667 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T03:55:13.675 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:13.665+0000 7f83acb2f740 1 -- 172.21.15.39:0/59229558 --> [v2:172.21.15.161:6838/2190107270,v1:172.21.15.161:6839/2190107270] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5555c34778a0 con 0x5555c358c290 2024-09-21T03:55:13.675 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:13.665+0000 7f839f7fe640 1 --2- 172.21.15.39:0/59229558 >> [v2:172.21.15.161:6838/2190107270,v1:172.21.15.161:6839/2190107270] conn(0x5555c358c290 0x5555c358e6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:55:13.675 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:13.665+0000 7f837f7fe640 10 client.17934 ms_handle_connect on v2:172.21.15.161:6838/2190107270 2024-09-21T03:55:13.675 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:13.665+0000 7f839f7fe640 1 --2- 172.21.15.39:0/59229558 >> [v2:172.21.15.161:6838/2190107270,v1:172.21.15.161:6839/2190107270] conn(0x5555c358c290 0x5555c358e6b0 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=ea439f44520f9816 server_cookie=1a0f60bf66f4523d in_seq=0 out_seq=0 2024-09-21T03:55:13.675 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:13.666+0000 7f837f7fe640 1 -- 172.21.15.39:0/59229558 <== mds.1 v2:172.21.15.161:6838/2190107270 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5555c34778a0 con 0x5555c358c290 2024-09-21T03:55:13.675 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:13.666+0000 7f837f7fe640 10 client.17934 handle_command_reply: tid=0 2024-09-21T03:55:13.675 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:13.666+0000 7f83acb2f740 1 client.17934 shutdown 2024-09-21T03:55:13.675 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:13.666+0000 7f83acb2f740 1 -- 172.21.15.39:0/59229558 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5555c30efda0 msgr2=0x5555c35131b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:55:13.675 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:13.666+0000 7f83acb2f740 1 --2- 172.21.15.39:0/59229558 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5555c30efda0 0x5555c35131b0 secure :-1 s=READY pgs=1210 cs=0 l=1 rev1=1 crypto rx=0x7f8390005530 tx=0x7f8390005610 comp rx=0 tx=0).stop 2024-09-21T03:55:13.675 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:13.666+0000 7f83acb2f740 1 -- 172.21.15.39:0/59229558 shutdown_connections 2024-09-21T03:55:13.675 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:13.666+0000 7f83acb2f740 1 --2- 172.21.15.39:0/59229558 >> [v2:172.21.15.161:6838/2190107270,v1:172.21.15.161:6839/2190107270] conn(0x5555c358c290 0x5555c358e6b0 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:13.676 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:13.666+0000 7f83acb2f740 1 --2- 172.21.15.39:0/59229558 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5555c315c070 0x5555c3513c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:13.676 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:13.666+0000 7f83acb2f740 1 --2- 172.21.15.39:0/59229558 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5555c31331b0 0x5555c35136f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:13.676 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:13.666+0000 7f837f7fe640 0 client.17934 ms_handle_reset on v2:172.21.15.161:6838/2190107270 2024-09-21T03:55:13.676 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:13.666+0000 7f83acb2f740 1 --2- 172.21.15.39:0/59229558 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5555c30efda0 0x5555c35131b0 unknown :-1 s=CLOSED pgs=1210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:13.676 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:13.666+0000 7f83acb2f740 1 -- 172.21.15.39:0/59229558 >> 172.21.15.39:0/59229558 conn(0x5555c3150460 msgr2=0x5555c3510b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:55:13.676 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:13.666+0000 7f83acb2f740 1 -- 172.21.15.39:0/59229558 shutdown_connections 2024-09-21T03:55:13.676 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:13.666+0000 7f83acb2f740 1 -- 172.21.15.39:0/59229558 wait complete. 2024-09-21T03:55:13.676 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:13.666+0000 7f83acb2f740 20 client.17934 trim_cache size 0 max 16384 2024-09-21T03:55:13.676 INFO:teuthology.orchestra.run.smithi039.stdout:[] 2024-09-21T03:55:13.677 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:13.667+0000 7f83a65bb640 1 -- 172.21.15.39:0/3345910793 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f83880b66b0 msgr2=0x7f83880b8bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:55:13.677 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:13.667+0000 7f83a65bb640 1 --2- 172.21.15.39:0/3345910793 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f83880b66b0 0x7f83880b8bc0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f8398011580 tx=0x7f8398004a40 comp rx=0 tx=0).stop 2024-09-21T03:55:13.677 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:13.667+0000 7f83a65bb640 1 -- 172.21.15.39:0/3345910793 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f83a01094a0 msgr2=0x7f83a019ab20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:55:13.678 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:13.667+0000 7f83a65bb640 1 --2- 172.21.15.39:0/3345910793 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f83a01094a0 0x7f83a019ab20 secure :-1 s=READY pgs=1206 cs=0 l=1 rev1=1 crypto rx=0x7f839402a910 tx=0x7f8394004270 comp rx=0 tx=0).stop 2024-09-21T03:55:13.678 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:13.667+0000 7f83a65bb640 1 -- 172.21.15.39:0/3345910793 shutdown_connections 2024-09-21T03:55:13.678 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:13.667+0000 7f83a65bb640 1 --2- 172.21.15.39:0/3345910793 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f83880b66b0 0x7f83880b8bc0 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:13.678 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:13.667+0000 7f83a65bb640 1 --2- 172.21.15.39:0/3345910793 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f83a0109df0 0x7f83a019fdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:13.678 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:13.667+0000 7f83a65bb640 1 --2- 172.21.15.39:0/3345910793 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f83a01094a0 0x7f83a019ab20 unknown :-1 s=CLOSED pgs=1206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:13.678 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:13.667+0000 7f83a65bb640 1 --2- 172.21.15.39:0/3345910793 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f83a0103b60 0x7f83a019a580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:13.678 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:13.667+0000 7f83a65bb640 1 -- 172.21.15.39:0/3345910793 >> 172.21.15.39:0/3345910793 conn(0x7f83a007b850 msgr2=0x7f83a0105610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:55:13.678 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:13.667+0000 7f83a65bb640 1 -- 172.21.15.39:0/3345910793 shutdown_connections 2024-09-21T03:55:13.678 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:13.667+0000 7f83a65bb640 1 -- 172.21.15.39:0/3345910793 wait complete. 2024-09-21T03:55:14.566 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 51116d5c-77ca-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 damage ls 2024-09-21T03:55:14.830 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/51116d5c-77ca-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:55:15.216 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:55:15 smithi039 ceph-mon[31211]: pgmap v606: 97 pgs: 97 active+clean; 10 MiB data, 525 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 53 KiB/s wr, 30 op/s 2024-09-21T03:55:15.325 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:15.308+0000 7f2e3227e640 1 -- 172.21.15.39:0/3718379233 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2e2c1049b0 msgr2=0x7f2e2c104dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:55:15.325 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:15.308+0000 7f2e3227e640 1 --2- 172.21.15.39:0/3718379233 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2e2c1049b0 0x7f2e2c104dc0 secure :-1 s=READY pgs=1211 cs=0 l=1 rev1=1 crypto rx=0x7f2e1c009900 tx=0x7f2e1c02a410 comp rx=0 tx=0).stop 2024-09-21T03:55:15.325 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:15.308+0000 7f2e3227e640 1 -- 172.21.15.39:0/3718379233 shutdown_connections 2024-09-21T03:55:15.325 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:15.308+0000 7f2e3227e640 1 --2- 172.21.15.39:0/3718379233 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2e2c10de10 0x7f2e2c110300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:15.325 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:15.308+0000 7f2e3227e640 1 --2- 172.21.15.39:0/3718379233 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2e2c105300 0x7f2e2c10d850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:15.325 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:15.308+0000 7f2e3227e640 1 --2- 172.21.15.39:0/3718379233 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2e2c1049b0 0x7f2e2c104dc0 unknown :-1 s=CLOSED pgs=1211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:15.325 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:15.308+0000 7f2e3227e640 1 -- 172.21.15.39:0/3718379233 >> 172.21.15.39:0/3718379233 conn(0x7f2e2c079730 msgr2=0x7f2e2c079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:55:15.325 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:15.308+0000 7f2e3227e640 1 -- 172.21.15.39:0/3718379233 shutdown_connections 2024-09-21T03:55:15.325 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:15.308+0000 7f2e3227e640 1 -- 172.21.15.39:0/3718379233 wait complete. 2024-09-21T03:55:15.326 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:15.309+0000 7f2e3227e640 1 Processor -- start 2024-09-21T03:55:15.326 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:15.309+0000 7f2e3227e640 1 -- start start 2024-09-21T03:55:15.326 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:15.309+0000 7f2e3227e640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2e2c1049b0 0x7f2e2c19a4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:55:15.326 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:15.309+0000 7f2e3227e640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2e2c105300 0x7f2e2c19aa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:55:15.326 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:15.309+0000 7f2e3227e640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2e2c10de10 0x7f2e2c19fd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:55:15.326 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:15.309+0000 7f2e3227e640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f2e2c113a70 con 0x7f2e2c105300 2024-09-21T03:55:15.326 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:15.309+0000 7f2e3227e640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f2e2c1138f0 con 0x7f2e2c10de10 2024-09-21T03:55:15.326 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:15.309+0000 7f2e3227e640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f2e2c113bf0 con 0x7f2e2c1049b0 2024-09-21T03:55:15.326 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:15.309+0000 7f2e2b7fe640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2e2c105300 0x7f2e2c19aa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:55:15.326 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:15.309+0000 7f2e2b7fe640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2e2c105300 0x7f2e2c19aa80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:39430/0 (socket says 172.21.15.39:39430) 2024-09-21T03:55:15.326 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:15.309+0000 7f2e2b7fe640 1 -- 172.21.15.39:0/2324923758 learned_addr learned my addr 172.21.15.39:0/2324923758 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:55:15.326 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:15.310+0000 7f2e2bfff640 1 --2- 172.21.15.39:0/2324923758 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2e2c1049b0 0x7f2e2c19a4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:55:15.326 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:15.310+0000 7f2e3081b640 1 --2- 172.21.15.39:0/2324923758 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2e2c10de10 0x7f2e2c19fd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:55:15.327 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:15.310+0000 7f2e2b7fe640 1 -- 172.21.15.39:0/2324923758 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2e2c1049b0 msgr2=0x7f2e2c19a4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:55:15.327 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:15.310+0000 7f2e2b7fe640 1 --2- 172.21.15.39:0/2324923758 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2e2c1049b0 0x7f2e2c19a4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:15.327 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:15.310+0000 7f2e2b7fe640 1 -- 172.21.15.39:0/2324923758 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2e2c10de10 msgr2=0x7f2e2c19fd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:55:15.327 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:15.310+0000 7f2e2b7fe640 1 --2- 172.21.15.39:0/2324923758 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2e2c10de10 0x7f2e2c19fd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:15.327 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:15.310+0000 7f2e2b7fe640 1 -- 172.21.15.39:0/2324923758 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2e2c1a0450 con 0x7f2e2c105300 2024-09-21T03:55:15.327 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:15.310+0000 7f2e3081b640 1 --2- 172.21.15.39:0/2324923758 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2e2c10de10 0x7f2e2c19fd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:55:15.327 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:15.310+0000 7f2e2b7fe640 1 --2- 172.21.15.39:0/2324923758 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2e2c105300 0x7f2e2c19aa80 secure :-1 s=READY pgs=1212 cs=0 l=1 rev1=1 crypto rx=0x7f2e2000d740 tx=0x7f2e2000dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:55:15.327 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:15.310+0000 7f2e297fa640 1 -- 172.21.15.39:0/2324923758 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e20017070 con 0x7f2e2c105300 2024-09-21T03:55:15.327 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:15.310+0000 7f2e297fa640 1 -- 172.21.15.39:0/2324923758 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2e2000de50 con 0x7f2e2c105300 2024-09-21T03:55:15.327 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:15.310+0000 7f2e297fa640 1 -- 172.21.15.39:0/2324923758 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e2000f620 con 0x7f2e2c105300 2024-09-21T03:55:15.327 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:15.310+0000 7f2e3227e640 1 -- 172.21.15.39:0/2324923758 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2e2c1a0740 con 0x7f2e2c105300 2024-09-21T03:55:15.327 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:15.310+0000 7f2e3227e640 1 -- 172.21.15.39:0/2324923758 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2e2c1a6ea0 con 0x7f2e2c105300 2024-09-21T03:55:15.327 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:15.313+0000 7f2e387f2740 1 Processor -- start 2024-09-21T03:55:15.327 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:15.313+0000 7f2e387f2740 1 -- start start 2024-09-21T03:55:15.327 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:15.313+0000 7f2e387f2740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5634d8e74070 0x5634d921b6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:55:15.328 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:15.313+0000 7f2e387f2740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5634d8e4b1b0 0x5634d921bbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:55:15.328 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:15.313+0000 7f2e387f2740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5634d8e53390 0x5634d921c120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:55:15.328 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:15.313+0000 7f2e387f2740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x5634d9191ea0 con 0x5634d8e4b1b0 2024-09-21T03:55:15.328 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:15.313+0000 7f2e387f2740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x5634d91812d0 con 0x5634d8e74070 2024-09-21T03:55:15.328 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:15.313+0000 7f2e387f2740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x5634d91850e0 con 0x5634d8e53390 2024-09-21T03:55:15.328 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:15.313+0000 7f2e2bfff640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5634d8e4b1b0 0x5634d921bbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:55:15.328 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:15.313+0000 7f2e297fa640 1 -- 172.21.15.39:0/2324923758 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 27) ==== 154643+0+0 (secure 0 0 0) 0x7f2e2001a070 con 0x7f2e2c105300 2024-09-21T03:55:15.328 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:15.313+0000 7f2e2bfff640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5634d8e4b1b0 0x5634d921bbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:39436/0 (socket says 172.21.15.39:39436) 2024-09-21T03:55:15.328 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:15.313+0000 7f2e2bfff640 1 -- 172.21.15.39:0/151897623 learned_addr learned my addr 172.21.15.39:0/151897623 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:55:15.328 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:15.313+0000 7f2e3081b640 1 --2- 172.21.15.39:0/151897623 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5634d8e74070 0x5634d921b6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:55:15.328 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:15.314+0000 7f2e2b7fe640 1 --2- 172.21.15.39:0/151897623 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5634d8e53390 0x5634d921c120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:55:15.328 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:15.314+0000 7f2e2bfff640 1 -- 172.21.15.39:0/151897623 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5634d8e53390 msgr2=0x5634d921c120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:55:15.328 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:15.314+0000 7f2e2bfff640 1 --2- 172.21.15.39:0/151897623 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5634d8e53390 0x5634d921c120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:15.328 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:15.314+0000 7f2e2bfff640 1 -- 172.21.15.39:0/151897623 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5634d8e74070 msgr2=0x5634d921b6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:55:15.329 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:15.314+0000 7f2e2bfff640 1 --2- 172.21.15.39:0/151897623 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5634d8e74070 0x5634d921b6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:15.329 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:15.314+0000 7f2e2bfff640 1 -- 172.21.15.39:0/151897623 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5634d8edf650 con 0x5634d8e4b1b0 2024-09-21T03:55:15.329 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:15.314+0000 7f2e2b7fe640 1 --2- 172.21.15.39:0/151897623 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5634d8e53390 0x5634d921c120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:55:15.329 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:15.314+0000 7f2e2bfff640 1 --2- 172.21.15.39:0/151897623 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5634d8e4b1b0 0x5634d921bbe0 secure :-1 s=READY pgs=1213 cs=0 l=1 rev1=1 crypto rx=0x7f2e2c06bc60 tx=0x7f2e1c032c60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=49233c3e2c04758a server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:55:15.329 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:15.314+0000 7f2e127fc640 1 -- 172.21.15.39:0/151897623 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e1c040070 con 0x5634d8e4b1b0 2024-09-21T03:55:15.329 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:15.314+0000 7f2e127fc640 1 -- 172.21.15.39:0/151897623 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2e1c032e50 con 0x5634d8e4b1b0 2024-09-21T03:55:15.329 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:15.314+0000 7f2e127fc640 1 -- 172.21.15.39:0/151897623 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e1c035040 con 0x5634d8e4b1b0 2024-09-21T03:55:15.329 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:15.314+0000 7f2e297fa640 1 --2- 172.21.15.39:0/2324923758 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f2e140b6680 0x7f2e140b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:55:15.329 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:15.314+0000 7f2e3081b640 1 --2- 172.21.15.39:0/2324923758 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f2e140b6680 0x7f2e140b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:55:15.329 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:15.315+0000 7f2e297fa640 1 -- 172.21.15.39:0/2324923758 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8572+0+0 (secure 0 0 0) 0x7f2e200e7160 con 0x7f2e2c105300 2024-09-21T03:55:15.329 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:15.315+0000 7f2e387f2740 1 -- 172.21.15.39:0/151897623 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5634d8e4b1b0 msgr2=0x5634d921bbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:55:15.329 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:15.315+0000 7f2e3081b640 1 --2- 172.21.15.39:0/2324923758 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f2e140b6680 0x7f2e140b8b90 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f2e24005fd0 tx=0x7f2e24009790 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:55:15.329 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:15.315+0000 7f2e387f2740 1 --2- 172.21.15.39:0/151897623 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5634d8e4b1b0 0x5634d921bbe0 secure :-1 s=READY pgs=1213 cs=0 l=1 rev1=1 crypto rx=0x7f2e2c06bc60 tx=0x7f2e1c032c60 comp rx=0 tx=0).stop 2024-09-21T03:55:15.330 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:15.315+0000 7f2e387f2740 1 -- 172.21.15.39:0/151897623 shutdown_connections 2024-09-21T03:55:15.330 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:15.315+0000 7f2e387f2740 1 --2- 172.21.15.39:0/151897623 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5634d8e53390 0x5634d921c120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:15.330 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:15.315+0000 7f2e387f2740 1 --2- 172.21.15.39:0/151897623 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5634d8e4b1b0 0x5634d921bbe0 unknown :-1 s=CLOSED pgs=1213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:15.330 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:15.315+0000 7f2e387f2740 1 --2- 172.21.15.39:0/151897623 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5634d8e74070 0x5634d921b6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:15.330 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:15.315+0000 7f2e387f2740 1 -- 172.21.15.39:0/151897623 >> 172.21.15.39:0/151897623 conn(0x5634d8e68460 msgr2=0x5634d9219590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:55:15.330 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:15.315+0000 7f2e387f2740 1 -- 172.21.15.39:0/151897623 shutdown_connections 2024-09-21T03:55:15.330 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:15.315+0000 7f2e387f2740 1 -- 172.21.15.39:0/151897623 wait complete. 2024-09-21T03:55:15.330 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:15.316+0000 7f2e387f2740 1 -- start start 2024-09-21T03:55:15.330 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:15.316+0000 7f2e387f2740 1 Processor -- start 2024-09-21T03:55:15.330 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:15.316+0000 7f2e387f2740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5634d8e4b1b0 0x5634d922b140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:55:15.330 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:15.316+0000 7f2e387f2740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5634d8e53390 0x5634d922b680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:55:15.330 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:15.316+0000 7f2e387f2740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5634d8e74070 0x5634d92374b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:55:15.330 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:15.316+0000 7f2e387f2740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x5634d8fe2700 con 0x5634d8e4b1b0 2024-09-21T03:55:15.330 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:15.316+0000 7f2e387f2740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x5634d911c160 con 0x5634d8e53390 2024-09-21T03:55:15.330 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:15.316+0000 7f2e387f2740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x5634d922e180 con 0x5634d8e74070 2024-09-21T03:55:15.331 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:15.316+0000 7f2e387f2740 10 client.0 _ll_register_callbacks cb 0x5634d91359e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:55:15.331 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:15.316+0000 7f2e2bfff640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5634d8e4b1b0 0x5634d922b140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:55:15.331 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:15.316+0000 7f2e2bfff640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5634d8e4b1b0 0x5634d922b140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:39442/0 (socket says 172.21.15.39:39442) 2024-09-21T03:55:15.331 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:15.316+0000 7f2e2bfff640 1 -- 172.21.15.39:0/1314194223 learned_addr learned my addr 172.21.15.39:0/1314194223 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:55:15.331 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:15.317+0000 7f2e2b7fe640 1 --2- 172.21.15.39:0/1314194223 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5634d8e53390 0x5634d922b680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:55:15.331 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:15.317+0000 7f2e3081b640 1 --2- 172.21.15.39:0/1314194223 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5634d8e74070 0x5634d92374b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:55:15.331 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:15.317+0000 7f2e2bfff640 1 -- 172.21.15.39:0/1314194223 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5634d8e74070 msgr2=0x5634d92374b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:55:15.331 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:15.317+0000 7f2e2bfff640 1 --2- 172.21.15.39:0/1314194223 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5634d8e74070 0x5634d92374b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:15.331 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:15.317+0000 7f2e2bfff640 1 -- 172.21.15.39:0/1314194223 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5634d8e53390 msgr2=0x5634d922b680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:55:15.331 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:15.317+0000 7f2e2bfff640 1 --2- 172.21.15.39:0/1314194223 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5634d8e53390 0x5634d922b680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:15.331 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:15.317+0000 7f2e2bfff640 1 -- 172.21.15.39:0/1314194223 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5634d9149c90 con 0x5634d8e4b1b0 2024-09-21T03:55:15.331 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:15.317+0000 7f2e2b7fe640 1 --2- 172.21.15.39:0/1314194223 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5634d8e53390 0x5634d922b680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:55:15.331 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:15.317+0000 7f2e2bfff640 1 --2- 172.21.15.39:0/1314194223 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5634d8e4b1b0 0x5634d922b140 secure :-1 s=READY pgs=1214 cs=0 l=1 rev1=1 crypto rx=0x7f2e1c006010 tx=0x7f2e1c0050e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e4fa92b3e983ff65 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:55:15.331 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:15.317+0000 7f2e0b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-09-21T03:55:15.332 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:15.317+0000 7f2e0b7fe640 1 -- 172.21.15.39:0/1314194223 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e1c04b070 con 0x5634d8e4b1b0 2024-09-21T03:55:15.332 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:15.317+0000 7f2e0b7fe640 1 -- 172.21.15.39:0/1314194223 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8572+0+0 (secure 0 0 0) 0x7f2e1c004040 con 0x5634d8e4b1b0 2024-09-21T03:55:15.332 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:15.317+0000 7f2e387f2740 1 -- 172.21.15.39:0/1314194223 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2e1c032e50 con 0x5634d8e4b1b0 2024-09-21T03:55:15.332 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:15.317+0000 7f2e0b7fe640 1 -- 172.21.15.39:0/1314194223 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f2e1c057050 con 0x5634d8e4b1b0 2024-09-21T03:55:15.332 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:15.317+0000 7f2e387f2740 10 client.17946 fetch_fsmap learned FSMap version 29 2024-09-21T03:55:15.332 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:15.317+0000 7f2e387f2740 1 -- 172.21.15.39:0/1314194223 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5634d9239de0 con 0x5634d8e4b1b0 2024-09-21T03:55:15.332 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:15.318+0000 7f2e0b7fe640 1 -- 172.21.15.39:0/1314194223 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2e1c032e50 con 0x5634d8e4b1b0 2024-09-21T03:55:15.332 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:15.318+0000 7f2e0b7fe640 1 -- 172.21.15.39:0/1314194223 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f2e1c004d00 con 0x5634d8e4b1b0 2024-09-21T03:55:15.332 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:15.318+0000 7f2e387f2740 10 client.17946 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T03:55:15.332 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:15.318+0000 7f2e387f2740 10 client.17946 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T03:55:15.332 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:15.318+0000 7f2e387f2740 20 client.17946 populate_metadata read hostname 'smithi039' 2024-09-21T03:55:15.332 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:15.318+0000 7f2e387f2740 1 --2- 172.21.15.39:0/1314194223 >> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] conn(0x5634d8e329f0 0x5634d923d140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:55:15.332 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:15.318+0000 7f2e387f2740 4 client.17946 mds_command: new command op to 24695 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:55:15.332 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:15.318+0000 7f2e387f2740 1 -- 172.21.15.39:0/1314194223 --> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5634d9193df0 con 0x5634d8e329f0 2024-09-21T03:55:15.332 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:15.318+0000 7f2e2b7fe640 1 --2- 172.21.15.39:0/1314194223 >> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] conn(0x5634d8e329f0 0x5634d923d140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:55:15.333 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:15.319+0000 7f2e2b7fe640 1 --2- 172.21.15.39:0/1314194223 >> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] conn(0x5634d8e329f0 0x5634d923d140 crc :-1 s=READY pgs=149 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=599fcd1f2e29cc23 server_cookie=fecd8e06571c38e1 in_seq=0 out_seq=0 2024-09-21T03:55:15.333 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:15.319+0000 7f2e0b7fe640 10 client.17946 ms_handle_connect on v2:172.21.15.150:6838/2949386907 2024-09-21T03:55:15.333 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:15.319+0000 7f2e0b7fe640 1 -- 172.21.15.39:0/1314194223 <== mds.0 v2:172.21.15.150:6838/2949386907 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5634d9193df0 con 0x5634d8e329f0 2024-09-21T03:55:15.333 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:15.319+0000 7f2e0b7fe640 10 client.17946 handle_command_reply: tid=0 2024-09-21T03:55:15.333 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:15.319+0000 7f2e387f2740 1 client.17946 shutdown 2024-09-21T03:55:15.333 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:15.320+0000 7f2e387f2740 1 -- 172.21.15.39:0/1314194223 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5634d8e4b1b0 msgr2=0x5634d922b140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:55:15.333 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:15.320+0000 7f2e387f2740 1 --2- 172.21.15.39:0/1314194223 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5634d8e4b1b0 0x5634d922b140 secure :-1 s=READY pgs=1214 cs=0 l=1 rev1=1 crypto rx=0x7f2e1c006010 tx=0x7f2e1c0050e0 comp rx=0 tx=0).stop 2024-09-21T03:55:15.333 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:15.320+0000 7f2e387f2740 1 -- 172.21.15.39:0/1314194223 shutdown_connections 2024-09-21T03:55:15.333 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:15.320+0000 7f2e387f2740 1 --2- 172.21.15.39:0/1314194223 >> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] conn(0x5634d8e329f0 0x5634d923d140 crc :-1 s=READY pgs=149 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:15.333 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:15.320+0000 7f2e387f2740 1 --2- 172.21.15.39:0/1314194223 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5634d8e74070 0x5634d92374b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:15.333 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:15.320+0000 7f2e387f2740 1 --2- 172.21.15.39:0/1314194223 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5634d8e53390 0x5634d922b680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:15.333 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:15.320+0000 7f2e0b7fe640 0 client.17946 ms_handle_reset on v2:172.21.15.150:6838/2949386907 2024-09-21T03:55:15.333 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:15.320+0000 7f2e387f2740 1 --2- 172.21.15.39:0/1314194223 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5634d8e4b1b0 0x5634d922b140 unknown :-1 s=CLOSED pgs=1214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:15.333 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:15.320+0000 7f2e387f2740 1 -- 172.21.15.39:0/1314194223 >> 172.21.15.39:0/1314194223 conn(0x5634d8e68460 msgr2=0x5634d9217bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:55:15.334 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:15.320+0000 7f2e387f2740 1 -- 172.21.15.39:0/1314194223 shutdown_connections 2024-09-21T03:55:15.334 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:15.320+0000 7f2e387f2740 1 -- 172.21.15.39:0/1314194223 wait complete. 2024-09-21T03:55:15.334 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:15.320+0000 7f2e387f2740 20 client.17946 trim_cache size 0 max 16384 2024-09-21T03:55:15.336 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:15.333+0000 7f2e387f2740 1 Processor -- start 2024-09-21T03:55:15.336 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:15.333+0000 7f2e387f2740 1 -- start start 2024-09-21T03:55:15.336 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:15.333+0000 7f2e387f2740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5634d8e4b1b0 0x5634d922a030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:55:15.337 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:15.334+0000 7f2e387f2740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5634d8e53390 0x5634d9220f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:55:15.337 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:15.334+0000 7f2e387f2740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5634d8e74070 0x5634d9228330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:55:15.337 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:15.334+0000 7f2e387f2740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f2e2c1ad620 con 0x5634d8e4b1b0 2024-09-21T03:55:15.337 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:15.334+0000 7f2e387f2740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f2e2c1af160 con 0x5634d8e53390 2024-09-21T03:55:15.337 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:15.334+0000 7f2e387f2740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f2e2c1adcf0 con 0x5634d8e74070 2024-09-21T03:55:15.337 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:15.334+0000 7f2e2bfff640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5634d8e4b1b0 0x5634d922a030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:55:15.337 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:15.334+0000 7f2e2bfff640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5634d8e4b1b0 0x5634d922a030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:39446/0 (socket says 172.21.15.39:39446) 2024-09-21T03:55:15.337 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:15.334+0000 7f2e2bfff640 1 -- 172.21.15.39:0/3690003625 learned_addr learned my addr 172.21.15.39:0/3690003625 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:55:15.339 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:15.334+0000 7f2e2b7fe640 1 --2- 172.21.15.39:0/3690003625 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5634d8e53390 0x5634d9220f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:55:15.339 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:15.334+0000 7f2e3081b640 1 --2- 172.21.15.39:0/3690003625 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5634d8e74070 0x5634d9228330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:55:15.339 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:15.334+0000 7f2e2bfff640 1 -- 172.21.15.39:0/3690003625 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5634d8e74070 msgr2=0x5634d9228330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:55:15.339 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:15.334+0000 7f2e2bfff640 1 --2- 172.21.15.39:0/3690003625 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5634d8e74070 0x5634d9228330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:15.339 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:15.334+0000 7f2e2bfff640 1 -- 172.21.15.39:0/3690003625 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5634d8e53390 msgr2=0x5634d9220f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:55:15.339 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:15.334+0000 7f2e2bfff640 1 --2- 172.21.15.39:0/3690003625 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5634d8e53390 0x5634d9220f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:15.339 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:15.334+0000 7f2e2bfff640 1 -- 172.21.15.39:0/3690003625 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5634d922d880 con 0x5634d8e4b1b0 2024-09-21T03:55:15.339 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:15.334+0000 7f2e3081b640 1 --2- 172.21.15.39:0/3690003625 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5634d8e74070 0x5634d9228330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:55:15.339 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:15.334+0000 7f2e2b7fe640 1 --2- 172.21.15.39:0/3690003625 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5634d8e53390 0x5634d9220f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:55:15.339 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:15.334+0000 7f2e2bfff640 1 --2- 172.21.15.39:0/3690003625 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5634d8e4b1b0 0x5634d922a030 secure :-1 s=READY pgs=1215 cs=0 l=1 rev1=1 crypto rx=0x7f2e1c003640 tx=0x7f2e1c02a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=632d3dbd51ad3d9b server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:55:15.339 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:15.335+0000 7f2e11ffb640 1 -- 172.21.15.39:0/3690003625 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e1c04b030 con 0x5634d8e4b1b0 2024-09-21T03:55:15.339 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:15.335+0000 7f2e11ffb640 1 -- 172.21.15.39:0/3690003625 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2e1c0316b0 con 0x5634d8e4b1b0 2024-09-21T03:55:15.339 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:15.335+0000 7f2e11ffb640 1 -- 172.21.15.39:0/3690003625 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e1c035030 con 0x5634d8e4b1b0 2024-09-21T03:55:15.340 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:15.335+0000 7f2e387f2740 1 -- 172.21.15.39:0/3690003625 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5634d8e4b1b0 msgr2=0x5634d922a030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:55:15.340 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:15.335+0000 7f2e387f2740 1 --2- 172.21.15.39:0/3690003625 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5634d8e4b1b0 0x5634d922a030 secure :-1 s=READY pgs=1215 cs=0 l=1 rev1=1 crypto rx=0x7f2e1c003640 tx=0x7f2e1c02a410 comp rx=0 tx=0).stop 2024-09-21T03:55:15.340 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:15.335+0000 7f2e387f2740 1 -- 172.21.15.39:0/3690003625 shutdown_connections 2024-09-21T03:55:15.340 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:15.335+0000 7f2e387f2740 1 --2- 172.21.15.39:0/3690003625 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5634d8e74070 0x5634d9228330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:15.340 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:15.335+0000 7f2e387f2740 1 --2- 172.21.15.39:0/3690003625 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5634d8e53390 0x5634d9220f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:15.340 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:15.335+0000 7f2e387f2740 1 --2- 172.21.15.39:0/3690003625 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5634d8e4b1b0 0x5634d922a030 unknown :-1 s=CLOSED pgs=1215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:15.340 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:15.335+0000 7f2e387f2740 1 -- 172.21.15.39:0/3690003625 >> 172.21.15.39:0/3690003625 conn(0x5634d8e68460 msgr2=0x5634d9220840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:55:15.340 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:15.335+0000 7f2e387f2740 1 -- 172.21.15.39:0/3690003625 shutdown_connections 2024-09-21T03:55:15.340 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:15.336+0000 7f2e387f2740 1 -- 172.21.15.39:0/3690003625 wait complete. 2024-09-21T03:55:15.340 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:15.336+0000 7f2e387f2740 1 -- start start 2024-09-21T03:55:15.340 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:15.336+0000 7f2e387f2740 1 Processor -- start 2024-09-21T03:55:15.343 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:15.336+0000 7f2e387f2740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5634d8e4b1b0 0x5634d9240900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:55:15.343 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:15.336+0000 7f2e387f2740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5634d8e53390 0x5634d923de20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:55:15.343 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:15.336+0000 7f2e387f2740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5634d8e74070 0x5634d923e360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:55:15.343 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:15.336+0000 7f2e387f2740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f2e2c1aea90 con 0x5634d8e4b1b0 2024-09-21T03:55:15.343 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:15.336+0000 7f2e387f2740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f2e2c1ae3c0 con 0x5634d8e53390 2024-09-21T03:55:15.343 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:15.336+0000 7f2e387f2740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f2e2c1ab620 con 0x5634d8e74070 2024-09-21T03:55:15.343 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:15.337+0000 7f2e2bfff640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5634d8e4b1b0 0x5634d9240900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:55:15.343 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:15.337+0000 7f2e387f2740 10 client.0 _ll_register_callbacks cb 0x7f2e24009630 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:55:15.343 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:15.337+0000 7f2e2bfff640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5634d8e4b1b0 0x5634d9240900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:39454/0 (socket says 172.21.15.39:39454) 2024-09-21T03:55:15.344 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:15.337+0000 7f2e2bfff640 1 -- 172.21.15.39:0/1459705439 learned_addr learned my addr 172.21.15.39:0/1459705439 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:55:15.344 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:15.337+0000 7f2e3081b640 1 --2- 172.21.15.39:0/1459705439 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5634d8e74070 0x5634d923e360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:55:15.344 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:15.337+0000 7f2e2b7fe640 1 --2- 172.21.15.39:0/1459705439 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5634d8e53390 0x5634d923de20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:55:15.344 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:15.337+0000 7f2e2bfff640 1 -- 172.21.15.39:0/1459705439 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5634d8e74070 msgr2=0x5634d923e360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:55:15.344 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:15.337+0000 7f2e2bfff640 1 --2- 172.21.15.39:0/1459705439 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5634d8e74070 0x5634d923e360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:15.344 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:15.337+0000 7f2e2bfff640 1 -- 172.21.15.39:0/1459705439 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5634d8e53390 msgr2=0x5634d923de20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:55:15.344 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:15.337+0000 7f2e2bfff640 1 --2- 172.21.15.39:0/1459705439 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5634d8e53390 0x5634d923de20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:15.344 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:15.337+0000 7f2e2bfff640 1 -- 172.21.15.39:0/1459705439 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5634d923eaa0 con 0x5634d8e4b1b0 2024-09-21T03:55:15.344 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:15.337+0000 7f2e2b7fe640 1 --2- 172.21.15.39:0/1459705439 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5634d8e53390 0x5634d923de20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:55:15.344 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:15.337+0000 7f2e2bfff640 1 --2- 172.21.15.39:0/1459705439 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5634d8e4b1b0 0x5634d9240900 secure :-1 s=READY pgs=1216 cs=0 l=1 rev1=1 crypto rx=0x7f2e1c0036e0 tx=0x7f2e1c005740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c2185bd892268d9 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:55:15.344 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:15.337+0000 7f2e0b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-09-21T03:55:15.344 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:15.337+0000 7f2e0b7fe640 1 -- 172.21.15.39:0/1459705439 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e1c04b070 con 0x5634d8e4b1b0 2024-09-21T03:55:15.344 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:15.337+0000 7f2e0b7fe640 1 -- 172.21.15.39:0/1459705439 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8572+0+0 (secure 0 0 0) 0x7f2e1c003dc0 con 0x5634d8e4b1b0 2024-09-21T03:55:15.344 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:15.337+0000 7f2e387f2740 1 -- 172.21.15.39:0/1459705439 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2e1c0316b0 con 0x5634d8e4b1b0 2024-09-21T03:55:15.345 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:15.338+0000 7f2e0b7fe640 1 -- 172.21.15.39:0/1459705439 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f2e1c035040 con 0x5634d8e4b1b0 2024-09-21T03:55:15.345 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:15.338+0000 7f2e387f2740 10 client.17952 fetch_fsmap learned FSMap version 29 2024-09-21T03:55:15.345 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:15.338+0000 7f2e387f2740 1 -- 172.21.15.39:0/1459705439 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5634d92a7690 con 0x5634d8e4b1b0 2024-09-21T03:55:15.345 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:15.338+0000 7f2e0b7fe640 1 -- 172.21.15.39:0/1459705439 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2e1c0316b0 con 0x5634d8e4b1b0 2024-09-21T03:55:15.345 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:15.338+0000 7f2e0b7fe640 1 -- 172.21.15.39:0/1459705439 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f2e1c004b60 con 0x5634d8e4b1b0 2024-09-21T03:55:15.345 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:15.338+0000 7f2e387f2740 10 client.17952 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T03:55:15.345 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:15.338+0000 7f2e387f2740 10 client.17952 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T03:55:15.345 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:15.338+0000 7f2e387f2740 20 client.17952 populate_metadata read hostname 'smithi039' 2024-09-21T03:55:15.345 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:15.338+0000 7f2e387f2740 1 --2- 172.21.15.39:0/1459705439 >> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] conn(0x5634d8e07da0 0x5634d92a9df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:55:15.345 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:15.338+0000 7f2e387f2740 4 client.17952 mds_command: new command op to 24695 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T03:55:15.345 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:15.338+0000 7f2e387f2740 1 -- 172.21.15.39:0/1459705439 --> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5634d918f8a0 con 0x5634d8e07da0 2024-09-21T03:55:15.345 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:15.338+0000 7f2e2b7fe640 1 --2- 172.21.15.39:0/1459705439 >> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] conn(0x5634d8e07da0 0x5634d92a9df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:55:15.345 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:15.339+0000 7f2e2b7fe640 1 --2- 172.21.15.39:0/1459705439 >> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] conn(0x5634d8e07da0 0x5634d92a9df0 crc :-1 s=READY pgs=150 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ccf258b6c1766af7 server_cookie=62c944a88c51bfc8 in_seq=0 out_seq=0 2024-09-21T03:55:15.345 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:15.339+0000 7f2e0b7fe640 10 client.17952 ms_handle_connect on v2:172.21.15.150:6838/2949386907 2024-09-21T03:55:15.345 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:15.339+0000 7f2e0b7fe640 1 -- 172.21.15.39:0/1459705439 <== mds.0 v2:172.21.15.150:6838/2949386907 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5634d918f8a0 con 0x5634d8e07da0 2024-09-21T03:55:15.346 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:15.339+0000 7f2e0b7fe640 10 client.17952 handle_command_reply: tid=0 2024-09-21T03:55:15.346 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:15.339+0000 7f2e387f2740 1 client.17952 shutdown 2024-09-21T03:55:15.346 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:15.339+0000 7f2e387f2740 1 -- 172.21.15.39:0/1459705439 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5634d8e4b1b0 msgr2=0x5634d9240900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:55:15.346 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:15.339+0000 7f2e387f2740 1 --2- 172.21.15.39:0/1459705439 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5634d8e4b1b0 0x5634d9240900 secure :-1 s=READY pgs=1216 cs=0 l=1 rev1=1 crypto rx=0x7f2e1c0036e0 tx=0x7f2e1c005740 comp rx=0 tx=0).stop 2024-09-21T03:55:15.346 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:15.339+0000 7f2e387f2740 1 -- 172.21.15.39:0/1459705439 shutdown_connections 2024-09-21T03:55:15.346 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:15.339+0000 7f2e387f2740 1 --2- 172.21.15.39:0/1459705439 >> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] conn(0x5634d8e07da0 0x5634d92a9df0 crc :-1 s=READY pgs=150 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:15.346 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:15.339+0000 7f2e387f2740 1 --2- 172.21.15.39:0/1459705439 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5634d8e74070 0x5634d923e360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:15.346 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:15.339+0000 7f2e0b7fe640 0 client.17952 ms_handle_reset on v2:172.21.15.150:6838/2949386907 2024-09-21T03:55:15.346 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:15.339+0000 7f2e387f2740 1 --2- 172.21.15.39:0/1459705439 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5634d8e53390 0x5634d923de20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:15.346 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:15.339+0000 7f2e387f2740 1 --2- 172.21.15.39:0/1459705439 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5634d8e4b1b0 0x5634d9240900 unknown :-1 s=CLOSED pgs=1216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:15.346 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:15.339+0000 7f2e387f2740 1 -- 172.21.15.39:0/1459705439 >> 172.21.15.39:0/1459705439 conn(0x5634d8e68460 msgr2=0x5634d921c6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:55:15.346 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:15.339+0000 7f2e387f2740 1 -- 172.21.15.39:0/1459705439 shutdown_connections 2024-09-21T03:55:15.346 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:15.339+0000 7f2e387f2740 1 -- 172.21.15.39:0/1459705439 wait complete. 2024-09-21T03:55:15.346 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:15.340+0000 7f2e387f2740 20 client.17952 trim_cache size 0 max 16384 2024-09-21T03:55:15.347 INFO:teuthology.orchestra.run.smithi039.stdout:[] 2024-09-21T03:55:15.347 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:15.340+0000 7f2e3227e640 1 -- 172.21.15.39:0/2324923758 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f2e140b6680 msgr2=0x7f2e140b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:55:15.348 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:15.340+0000 7f2e3227e640 1 --2- 172.21.15.39:0/2324923758 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f2e140b6680 0x7f2e140b8b90 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f2e24005fd0 tx=0x7f2e24009790 comp rx=0 tx=0).stop 2024-09-21T03:55:15.348 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:15.340+0000 7f2e3227e640 1 -- 172.21.15.39:0/2324923758 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2e2c105300 msgr2=0x7f2e2c19aa80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:55:15.348 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:15.340+0000 7f2e3227e640 1 --2- 172.21.15.39:0/2324923758 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2e2c105300 0x7f2e2c19aa80 secure :-1 s=READY pgs=1212 cs=0 l=1 rev1=1 crypto rx=0x7f2e2000d740 tx=0x7f2e2000dc00 comp rx=0 tx=0).stop 2024-09-21T03:55:15.348 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:15.340+0000 7f2e3227e640 1 -- 172.21.15.39:0/2324923758 shutdown_connections 2024-09-21T03:55:15.348 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:15.340+0000 7f2e3227e640 1 --2- 172.21.15.39:0/2324923758 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f2e140b6680 0x7f2e140b8b90 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:15.348 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:15.340+0000 7f2e3227e640 1 --2- 172.21.15.39:0/2324923758 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2e2c10de10 0x7f2e2c19fd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:15.348 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:15.340+0000 7f2e3227e640 1 --2- 172.21.15.39:0/2324923758 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2e2c105300 0x7f2e2c19aa80 unknown :-1 s=CLOSED pgs=1212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:15.348 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:15.340+0000 7f2e3227e640 1 --2- 172.21.15.39:0/2324923758 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2e2c1049b0 0x7f2e2c19a4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:15.348 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:15.340+0000 7f2e3227e640 1 -- 172.21.15.39:0/2324923758 >> 172.21.15.39:0/2324923758 conn(0x7f2e2c079730 msgr2=0x7f2e2c100880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:55:15.348 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:15.340+0000 7f2e3227e640 1 -- 172.21.15.39:0/2324923758 shutdown_connections 2024-09-21T03:55:15.348 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:15.340+0000 7f2e3227e640 1 -- 172.21.15.39:0/2324923758 wait complete. 2024-09-21T03:55:15.468 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:55:15 smithi161 ceph-mon[34533]: pgmap v606: 97 pgs: 97 active+clean; 10 MiB data, 525 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 53 KiB/s wr, 30 op/s 2024-09-21T03:55:15.474 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:55:15 smithi150 ceph-mon[34289]: pgmap v606: 97 pgs: 97 active+clean; 10 MiB data, 525 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 53 KiB/s wr, 30 op/s 2024-09-21T03:55:17.147 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-21T03:55:17.148 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 51116d5c-77ca-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-21T03:55:17.334 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:55:17 smithi039 ceph-mon[31211]: pgmap v607: 97 pgs: 97 active+clean; 10 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-09-21T03:55:17.419 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/51116d5c-77ca-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:55:17.467 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:55:17 smithi161 ceph-mon[34533]: pgmap v607: 97 pgs: 97 active+clean; 10 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-09-21T03:55:17.475 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:55:17 smithi150 ceph-mon[34289]: pgmap v607: 97 pgs: 97 active+clean; 10 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-09-21T03:55:17.921 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:17.918+0000 7f5de4096640 1 -- 172.21.15.39:0/645269292 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5ddc1094c0 msgr2=0x7f5ddc1098d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:55:17.921 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:17.918+0000 7f5de4096640 1 --2- 172.21.15.39:0/645269292 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5ddc1094c0 0x7f5ddc1098d0 secure :-1 s=READY pgs=1217 cs=0 l=1 rev1=1 crypto rx=0x7f5dd0009900 tx=0x7f5dd002a410 comp rx=0 tx=0).stop 2024-09-21T03:55:17.921 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:17.918+0000 7f5de4096640 1 -- 172.21.15.39:0/645269292 shutdown_connections 2024-09-21T03:55:17.922 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:17.918+0000 7f5de4096640 1 --2- 172.21.15.39:0/645269292 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5ddc109e10 0x7f5ddc110640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:17.922 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:17.918+0000 7f5de4096640 1 --2- 172.21.15.39:0/645269292 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5ddc1094c0 0x7f5ddc1098d0 unknown :-1 s=CLOSED pgs=1217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:17.922 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:17.918+0000 7f5de4096640 1 --2- 172.21.15.39:0/645269292 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5ddc103b80 0x7f5ddc103f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:17.922 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:17.918+0000 7f5de4096640 1 -- 172.21.15.39:0/645269292 >> 172.21.15.39:0/645269292 conn(0x7f5ddc0ff610 msgr2=0x7f5ddc101a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:55:17.922 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:17.918+0000 7f5de4096640 1 -- 172.21.15.39:0/645269292 shutdown_connections 2024-09-21T03:55:17.922 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:17.918+0000 7f5de4096640 1 -- 172.21.15.39:0/645269292 wait complete. 2024-09-21T03:55:17.924 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:17.919+0000 7f5de4096640 1 Processor -- start 2024-09-21T03:55:17.924 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:17.919+0000 7f5de4096640 1 -- start start 2024-09-21T03:55:17.924 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:17.920+0000 7f5de4096640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5ddc103b80 0x7f5ddc19a520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:55:17.924 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:17.920+0000 7f5de4096640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5ddc1094c0 0x7f5ddc19aac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:55:17.924 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:17.920+0000 7f5de4096640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5ddc109e10 0x7f5ddc19fd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:55:17.924 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:17.920+0000 7f5de4096640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f5ddc113ad0 con 0x7f5ddc103b80 2024-09-21T03:55:17.924 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:17.920+0000 7f5de4096640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f5ddc113950 con 0x7f5ddc1094c0 2024-09-21T03:55:17.924 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:17.920+0000 7f5de4096640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f5ddc113c50 con 0x7f5ddc109e10 2024-09-21T03:55:17.925 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:17.920+0000 7f5de1e32640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5ddc103b80 0x7f5ddc19a520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:55:17.925 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:17.920+0000 7f5de1e32640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5ddc103b80 0x7f5ddc19a520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:47410/0 (socket says 172.21.15.39:47410) 2024-09-21T03:55:17.925 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:17.920+0000 7f5de1e32640 1 -- 172.21.15.39:0/3663062657 learned_addr learned my addr 172.21.15.39:0/3663062657 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:55:17.925 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:17.920+0000 7f5de1631640 1 --2- 172.21.15.39:0/3663062657 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5ddc1094c0 0x7f5ddc19aac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:55:17.925 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:17.920+0000 7f5de2633640 1 --2- 172.21.15.39:0/3663062657 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5ddc109e10 0x7f5ddc19fd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:55:17.925 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:17.920+0000 7f5de1e32640 1 -- 172.21.15.39:0/3663062657 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5ddc109e10 msgr2=0x7f5ddc19fd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:55:17.925 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:17.920+0000 7f5de1e32640 1 --2- 172.21.15.39:0/3663062657 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5ddc109e10 0x7f5ddc19fd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:17.925 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:17.920+0000 7f5de1e32640 1 -- 172.21.15.39:0/3663062657 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5ddc1094c0 msgr2=0x7f5ddc19aac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:55:17.925 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:17.920+0000 7f5de1e32640 1 --2- 172.21.15.39:0/3663062657 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5ddc1094c0 0x7f5ddc19aac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:17.925 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:17.920+0000 7f5de1e32640 1 -- 172.21.15.39:0/3663062657 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ddc1a0490 con 0x7f5ddc103b80 2024-09-21T03:55:17.925 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:17.920+0000 7f5de1631640 1 --2- 172.21.15.39:0/3663062657 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5ddc1094c0 0x7f5ddc19aac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:55:17.925 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:17.920+0000 7f5de2633640 1 --2- 172.21.15.39:0/3663062657 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5ddc109e10 0x7f5ddc19fd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:55:17.925 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:17.920+0000 7f5de1e32640 1 --2- 172.21.15.39:0/3663062657 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5ddc103b80 0x7f5ddc19a520 secure :-1 s=READY pgs=1218 cs=0 l=1 rev1=1 crypto rx=0x7f5dd400d950 tx=0x7f5dd400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:55:17.925 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:17.921+0000 7f5ddaffd640 1 -- 172.21.15.39:0/3663062657 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5dd4017030 con 0x7f5ddc103b80 2024-09-21T03:55:17.925 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:17.921+0000 7f5de4096640 1 -- 172.21.15.39:0/3663062657 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5ddc1a0780 con 0x7f5ddc103b80 2024-09-21T03:55:17.926 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:17.921+0000 7f5ddaffd640 1 -- 172.21.15.39:0/3663062657 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5dd4004020 con 0x7f5ddc103b80 2024-09-21T03:55:17.926 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:17.921+0000 7f5ddaffd640 1 -- 172.21.15.39:0/3663062657 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5dd40047a0 con 0x7f5ddc103b80 2024-09-21T03:55:17.926 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:17.921+0000 7f5de4096640 1 -- 172.21.15.39:0/3663062657 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5ddc1a6ee0 con 0x7f5ddc103b80 2024-09-21T03:55:17.926 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:17.923+0000 7f5dea60a740 1 Processor -- start 2024-09-21T03:55:17.926 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:17.924+0000 7f5dea60a740 1 -- start start 2024-09-21T03:55:17.927 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:17.924+0000 7f5ddaffd640 1 -- 172.21.15.39:0/3663062657 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 27) ==== 154643+0+0 (secure 0 0 0) 0x7f5dd4009a30 con 0x7f5ddc103b80 2024-09-21T03:55:17.927 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:17.924+0000 7f5dea60a740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556d2441d350 0x556d247c4430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:55:17.927 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:17.924+0000 7f5dea60a740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556d243f4490 0x556d247c4970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:55:17.928 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:17.924+0000 7f5dea60a740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556d243fc670 0x556d247c4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:55:17.928 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:17.924+0000 7f5dea60a740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x556d2473ae60 con 0x556d243fc670 2024-09-21T03:55:17.928 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:17.924+0000 7f5dea60a740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x556d2472a290 con 0x556d243f4490 2024-09-21T03:55:17.928 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:17.924+0000 7f5dea60a740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x556d2472e0a0 con 0x556d2441d350 2024-09-21T03:55:17.928 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:17.924+0000 7f5de1631640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556d243fc670 0x556d247c4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:55:17.928 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:17.924+0000 7f5de1631640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556d243fc670 0x556d247c4eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:47426/0 (socket says 172.21.15.39:47426) 2024-09-21T03:55:17.928 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:17.924+0000 7f5de1631640 1 -- 172.21.15.39:0/2916255363 learned_addr learned my addr 172.21.15.39:0/2916255363 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:55:17.929 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:17.924+0000 7f5de1e32640 1 --2- 172.21.15.39:0/2916255363 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556d243f4490 0x556d247c4970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:55:17.929 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:17.924+0000 7f5de2633640 1 --2- 172.21.15.39:0/2916255363 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556d2441d350 0x556d247c4430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:55:17.929 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:17.925+0000 7f5de1631640 1 -- 172.21.15.39:0/2916255363 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556d2441d350 msgr2=0x556d247c4430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:55:17.929 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:17.925+0000 7f5de1631640 1 --2- 172.21.15.39:0/2916255363 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556d2441d350 0x556d247c4430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:17.929 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:17.925+0000 7f5de1631640 1 -- 172.21.15.39:0/2916255363 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556d243f4490 msgr2=0x556d247c4970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:55:17.929 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:17.925+0000 7f5de1631640 1 --2- 172.21.15.39:0/2916255363 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556d243f4490 0x556d247c4970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:17.929 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:17.925+0000 7f5de1631640 1 -- 172.21.15.39:0/2916255363 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556d24488930 con 0x556d243fc670 2024-09-21T03:55:17.937 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:17.925+0000 7f5de1631640 1 --2- 172.21.15.39:0/2916255363 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556d243fc670 0x556d247c4eb0 secure :-1 s=READY pgs=1219 cs=0 l=1 rev1=1 crypto rx=0x7f5ddc06cc00 tx=0x7f5dd0032d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=44be6c6950865e76 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:55:17.937 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:17.925+0000 7f5ddaffd640 1 --2- 172.21.15.39:0/3663062657 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f5dc40b6680 0x7f5dc40b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:55:17.937 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:17.925+0000 7f5dc3fff640 1 -- 172.21.15.39:0/2916255363 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5dd0035030 con 0x556d243fc670 2024-09-21T03:55:17.937 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:17.925+0000 7f5dc3fff640 1 -- 172.21.15.39:0/2916255363 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5dd00316a0 con 0x556d243fc670 2024-09-21T03:55:17.937 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:17.925+0000 7f5dc3fff640 1 -- 172.21.15.39:0/2916255363 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5dd00046c0 con 0x556d243fc670 2024-09-21T03:55:17.938 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:17.925+0000 7f5de2633640 1 --2- 172.21.15.39:0/3663062657 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f5dc40b6680 0x7f5dc40b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:55:17.938 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:17.925+0000 7f5ddaffd640 1 -- 172.21.15.39:0/3663062657 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8572+0+0 (secure 0 0 0) 0x7f5dd40e7260 con 0x7f5ddc103b80 2024-09-21T03:55:17.938 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:17.926+0000 7f5dea60a740 1 -- 172.21.15.39:0/2916255363 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556d243fc670 msgr2=0x556d247c4eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:55:17.938 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:17.926+0000 7f5de2633640 1 --2- 172.21.15.39:0/3663062657 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f5dc40b6680 0x7f5dc40b8b90 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f5dcc005fd0 tx=0x7f5dcc009780 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:55:17.938 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:17.926+0000 7f5dea60a740 1 --2- 172.21.15.39:0/2916255363 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556d243fc670 0x556d247c4eb0 secure :-1 s=READY pgs=1219 cs=0 l=1 rev1=1 crypto rx=0x7f5ddc06cc00 tx=0x7f5dd0032d00 comp rx=0 tx=0).stop 2024-09-21T03:55:17.938 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:17.926+0000 7f5dea60a740 1 -- 172.21.15.39:0/2916255363 shutdown_connections 2024-09-21T03:55:17.939 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:17.926+0000 7f5dea60a740 1 --2- 172.21.15.39:0/2916255363 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556d243fc670 0x556d247c4eb0 unknown :-1 s=CLOSED pgs=1219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:17.939 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:17.926+0000 7f5dea60a740 1 --2- 172.21.15.39:0/2916255363 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556d243f4490 0x556d247c4970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:17.939 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:17.926+0000 7f5dea60a740 1 --2- 172.21.15.39:0/2916255363 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556d2441d350 0x556d247c4430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:17.939 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:17.926+0000 7f5dea60a740 1 -- 172.21.15.39:0/2916255363 >> 172.21.15.39:0/2916255363 conn(0x556d24411740 msgr2=0x556d247c2320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:55:17.939 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:17.926+0000 7f5dea60a740 1 -- 172.21.15.39:0/2916255363 shutdown_connections 2024-09-21T03:55:17.939 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:17.926+0000 7f5dea60a740 1 -- 172.21.15.39:0/2916255363 wait complete. 2024-09-21T03:55:17.940 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:17.927+0000 7f5dea60a740 1 -- start start 2024-09-21T03:55:17.940 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:17.927+0000 7f5dea60a740 1 Processor -- start 2024-09-21T03:55:17.940 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:17.927+0000 7f5dea60a740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556d243f4490 0x556d247d4390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:55:17.940 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:17.927+0000 7f5dea60a740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556d243fc670 0x556d247d48d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:55:17.940 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:17.927+0000 7f5dea60a740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556d2441d350 0x556d247e0240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:55:17.940 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:17.927+0000 7f5dea60a740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x556d2458b890 con 0x556d243f4490 2024-09-21T03:55:17.940 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:17.927+0000 7f5dea60a740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x556d246c4d40 con 0x556d2441d350 2024-09-21T03:55:17.941 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:17.927+0000 7f5dea60a740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x556d247d6ee0 con 0x556d243fc670 2024-09-21T03:55:17.941 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:17.927+0000 7f5dea60a740 10 client.0 _ll_register_callbacks cb 0x556d24728c80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:55:17.941 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:17.927+0000 7f5de1e32640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556d243f4490 0x556d247d4390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:55:17.941 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:17.928+0000 7f5de1e32640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556d243f4490 0x556d247d4390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:47428/0 (socket says 172.21.15.39:47428) 2024-09-21T03:55:17.941 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:17.928+0000 7f5de1e32640 1 -- 172.21.15.39:0/1827401303 learned_addr learned my addr 172.21.15.39:0/1827401303 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:55:17.941 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:17.928+0000 7f5de2633640 1 --2- 172.21.15.39:0/1827401303 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556d2441d350 0x556d247e0240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:55:17.942 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:17.928+0000 7f5de1631640 1 --2- 172.21.15.39:0/1827401303 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556d243fc670 0x556d247d48d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:55:17.942 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:17.928+0000 7f5de1e32640 1 -- 172.21.15.39:0/1827401303 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556d243fc670 msgr2=0x556d247d48d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:55:17.942 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:17.928+0000 7f5de1e32640 1 --2- 172.21.15.39:0/1827401303 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556d243fc670 0x556d247d48d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:17.942 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:17.928+0000 7f5de1e32640 1 -- 172.21.15.39:0/1827401303 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556d2441d350 msgr2=0x556d247e0240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:55:17.942 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:17.928+0000 7f5de1e32640 1 --2- 172.21.15.39:0/1827401303 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556d2441d350 0x556d247e0240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:17.942 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:17.928+0000 7f5de1e32640 1 -- 172.21.15.39:0/1827401303 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556d247e07b0 con 0x556d243f4490 2024-09-21T03:55:17.943 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:17.928+0000 7f5de1e32640 1 --2- 172.21.15.39:0/1827401303 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556d243f4490 0x556d247d4390 secure :-1 s=READY pgs=1220 cs=0 l=1 rev1=1 crypto rx=0x7f5dd40e49e0 tx=0x7f5dd4009f30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f54c611ffdcb4100 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:55:17.943 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:17.928+0000 7f5dc17fa640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-09-21T03:55:17.943 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:17.928+0000 7f5dc17fa640 1 -- 172.21.15.39:0/1827401303 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5dd4014070 con 0x556d243f4490 2024-09-21T03:55:17.943 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:17.928+0000 7f5dc17fa640 1 -- 172.21.15.39:0/1827401303 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8572+0+0 (secure 0 0 0) 0x7f5dd40a89d0 con 0x556d243f4490 2024-09-21T03:55:17.943 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:17.928+0000 7f5dea60a740 1 -- 172.21.15.39:0/1827401303 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5dd00316a0 con 0x556d243f4490 2024-09-21T03:55:17.944 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:17.928+0000 7f5dc17fa640 1 -- 172.21.15.39:0/1827401303 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f5dd40e7020 con 0x556d243f4490 2024-09-21T03:55:17.944 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:17.928+0000 7f5dea60a740 10 client.17964 fetch_fsmap learned FSMap version 29 2024-09-21T03:55:17.944 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:17.928+0000 7f5dea60a740 1 -- 172.21.15.39:0/1827401303 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x556d247e3140 con 0x556d243f4490 2024-09-21T03:55:17.944 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:17.929+0000 7f5dc17fa640 1 -- 172.21.15.39:0/1827401303 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5dd00316a0 con 0x556d243f4490 2024-09-21T03:55:17.944 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:17.929+0000 7f5dc17fa640 1 -- 172.21.15.39:0/1827401303 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f5dd40a8bd0 con 0x556d243f4490 2024-09-21T03:55:17.944 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:17.929+0000 7f5dea60a740 10 client.17964 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T03:55:17.945 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:17.929+0000 7f5dea60a740 10 client.17964 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T03:55:17.945 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:17.929+0000 7f5dea60a740 20 client.17964 populate_metadata read hostname 'smithi039' 2024-09-21T03:55:17.945 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:17.929+0000 7f5dea60a740 1 --2- 172.21.15.39:0/1827401303 >> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] conn(0x556d243dbcd0 0x556d247e6690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:55:17.945 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:17.929+0000 7f5dea60a740 4 client.17964 mds_command: new command op to 24695 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:55:17.945 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:17.929+0000 7f5dea60a740 1 -- 172.21.15.39:0/1827401303 --> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556d2473cdb0 con 0x556d243dbcd0 2024-09-21T03:55:17.945 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:17.929+0000 7f5de1631640 1 --2- 172.21.15.39:0/1827401303 >> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] conn(0x556d243dbcd0 0x556d247e6690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:55:17.946 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:17.930+0000 7f5dc17fa640 10 client.17964 ms_handle_connect on v2:172.21.15.150:6838/2949386907 2024-09-21T03:55:17.946 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:17.930+0000 7f5de1631640 1 --2- 172.21.15.39:0/1827401303 >> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] conn(0x556d243dbcd0 0x556d247e6690 crc :-1 s=READY pgs=151 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c421bd7df5bb9a18 server_cookie=8fa766f930178ddf in_seq=0 out_seq=0 2024-09-21T03:55:17.946 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:17.930+0000 7f5dc17fa640 1 -- 172.21.15.39:0/1827401303 <== mds.0 v2:172.21.15.150:6838/2949386907 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x556d2473cdb0 con 0x556d243dbcd0 2024-09-21T03:55:17.946 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:17.930+0000 7f5dc17fa640 10 client.17964 handle_command_reply: tid=0 2024-09-21T03:55:17.946 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:17.930+0000 7f5dea60a740 1 client.17964 shutdown 2024-09-21T03:55:17.946 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:17.931+0000 7f5dea60a740 1 -- 172.21.15.39:0/1827401303 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556d243f4490 msgr2=0x556d247d4390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:55:17.947 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:17.931+0000 7f5dea60a740 1 --2- 172.21.15.39:0/1827401303 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556d243f4490 0x556d247d4390 secure :-1 s=READY pgs=1220 cs=0 l=1 rev1=1 crypto rx=0x7f5dd40e49e0 tx=0x7f5dd4009f30 comp rx=0 tx=0).stop 2024-09-21T03:55:17.947 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:17.931+0000 7f5dea60a740 1 -- 172.21.15.39:0/1827401303 shutdown_connections 2024-09-21T03:55:17.947 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:17.931+0000 7f5dea60a740 1 --2- 172.21.15.39:0/1827401303 >> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] conn(0x556d243dbcd0 0x556d247e6690 crc :-1 s=READY pgs=151 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:17.947 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:17.931+0000 7f5dea60a740 1 --2- 172.21.15.39:0/1827401303 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556d2441d350 0x556d247e0240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:17.947 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:17.931+0000 7f5dea60a740 1 --2- 172.21.15.39:0/1827401303 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556d243fc670 0x556d247d48d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:17.947 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:17.931+0000 7f5dea60a740 1 --2- 172.21.15.39:0/1827401303 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556d243f4490 0x556d247d4390 unknown :-1 s=CLOSED pgs=1220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:17.947 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:17.931+0000 7f5dc17fa640 0 client.17964 ms_handle_reset on v2:172.21.15.150:6838/2949386907 2024-09-21T03:55:17.948 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:17.931+0000 7f5dea60a740 1 -- 172.21.15.39:0/1827401303 >> 172.21.15.39:0/1827401303 conn(0x556d24411740 msgr2=0x556d247c75c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:55:17.948 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:17.931+0000 7f5dea60a740 1 -- 172.21.15.39:0/1827401303 shutdown_connections 2024-09-21T03:55:17.948 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:17.931+0000 7f5dea60a740 1 -- 172.21.15.39:0/1827401303 wait complete. 2024-09-21T03:55:17.948 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:17.931+0000 7f5dea60a740 20 client.17964 trim_cache size 0 max 16384 2024-09-21T03:55:17.953 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:17.949+0000 7f5dea60a740 1 Processor -- start 2024-09-21T03:55:17.953 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:17.949+0000 7f5dea60a740 1 -- start start 2024-09-21T03:55:17.953 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:17.949+0000 7f5dea60a740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556d243f4490 0x556d247c7f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:55:17.953 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:17.949+0000 7f5dea60a740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556d243fc670 0x556d247c5b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:55:17.953 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:17.949+0000 7f5dea60a740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556d2441d350 0x556d247c1860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:55:17.953 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:17.949+0000 7f5dea60a740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f5ddc1ad5d0 con 0x556d2441d350 2024-09-21T03:55:17.953 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:17.949+0000 7f5dea60a740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f5ddc1af110 con 0x556d243f4490 2024-09-21T03:55:17.953 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:17.949+0000 7f5dea60a740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f5ddc1adca0 con 0x556d243fc670 2024-09-21T03:55:17.953 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:17.949+0000 7f5de2633640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556d2441d350 0x556d247c1860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:55:17.953 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:17.949+0000 7f5de2633640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556d2441d350 0x556d247c1860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:47436/0 (socket says 172.21.15.39:47436) 2024-09-21T03:55:17.954 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:17.949+0000 7f5de2633640 1 -- 172.21.15.39:0/2145032748 learned_addr learned my addr 172.21.15.39:0/2145032748 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:55:17.954 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:17.949+0000 7f5de1631640 1 --2- 172.21.15.39:0/2145032748 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556d243fc670 0x556d247c5b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:55:17.954 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:17.949+0000 7f5de1e32640 1 --2- 172.21.15.39:0/2145032748 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556d243f4490 0x556d247c7f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:55:17.954 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:17.949+0000 7f5de2633640 1 -- 172.21.15.39:0/2145032748 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556d243fc670 msgr2=0x556d247c5b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:55:17.954 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:17.949+0000 7f5de2633640 1 --2- 172.21.15.39:0/2145032748 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556d243fc670 0x556d247c5b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:17.954 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:17.949+0000 7f5de2633640 1 -- 172.21.15.39:0/2145032748 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556d243f4490 msgr2=0x556d247c7f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:55:17.954 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:17.949+0000 7f5de2633640 1 --2- 172.21.15.39:0/2145032748 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556d243f4490 0x556d247c7f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:17.954 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:17.949+0000 7f5de2633640 1 -- 172.21.15.39:0/2145032748 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556d247d2690 con 0x556d2441d350 2024-09-21T03:55:17.954 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:17.949+0000 7f5de1e32640 1 --2- 172.21.15.39:0/2145032748 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556d243f4490 0x556d247c7f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:55:17.954 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:17.950+0000 7f5de2633640 1 --2- 172.21.15.39:0/2145032748 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556d2441d350 0x556d247c1860 secure :-1 s=READY pgs=1221 cs=0 l=1 rev1=1 crypto rx=0x7f5dcc009db0 tx=0x7f5dcc016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e934826d47ef3bda server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:55:17.954 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:17.950+0000 7f5dc37fe640 1 -- 172.21.15.39:0/2145032748 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5dcc004d20 con 0x556d2441d350 2024-09-21T03:55:17.955 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:17.950+0000 7f5dc37fe640 1 -- 172.21.15.39:0/2145032748 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5dcc020a00 con 0x556d2441d350 2024-09-21T03:55:17.955 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:17.950+0000 7f5dc37fe640 1 -- 172.21.15.39:0/2145032748 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5dcc0055c0 con 0x556d2441d350 2024-09-21T03:55:17.957 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:17.951+0000 7f5dea60a740 1 -- 172.21.15.39:0/2145032748 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556d2441d350 msgr2=0x556d247c1860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:55:17.957 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:17.951+0000 7f5dea60a740 1 --2- 172.21.15.39:0/2145032748 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556d2441d350 0x556d247c1860 secure :-1 s=READY pgs=1221 cs=0 l=1 rev1=1 crypto rx=0x7f5dcc009db0 tx=0x7f5dcc016000 comp rx=0 tx=0).stop 2024-09-21T03:55:17.958 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:17.951+0000 7f5dea60a740 1 -- 172.21.15.39:0/2145032748 shutdown_connections 2024-09-21T03:55:17.958 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:17.951+0000 7f5dea60a740 1 --2- 172.21.15.39:0/2145032748 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556d2441d350 0x556d247c1860 unknown :-1 s=CLOSED pgs=1221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:17.958 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:17.951+0000 7f5dea60a740 1 --2- 172.21.15.39:0/2145032748 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556d243fc670 0x556d247c5b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:17.958 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:17.951+0000 7f5dea60a740 1 --2- 172.21.15.39:0/2145032748 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556d243f4490 0x556d247c7f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:17.958 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:17.951+0000 7f5dea60a740 1 -- 172.21.15.39:0/2145032748 >> 172.21.15.39:0/2145032748 conn(0x556d24411740 msgr2=0x556d247c5460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:55:17.958 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:17.951+0000 7f5dea60a740 1 -- 172.21.15.39:0/2145032748 shutdown_connections 2024-09-21T03:55:17.958 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:17.951+0000 7f5dea60a740 1 -- 172.21.15.39:0/2145032748 wait complete. 2024-09-21T03:55:17.958 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:17.951+0000 7f5dea60a740 1 -- start start 2024-09-21T03:55:17.958 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:17.951+0000 7f5dea60a740 1 Processor -- start 2024-09-21T03:55:17.958 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:17.952+0000 7f5dea60a740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556d243f4490 0x556d247dc710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:55:17.958 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:17.952+0000 7f5dea60a740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556d243fc670 0x556d247d8bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:55:17.958 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:17.952+0000 7f5dea60a740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556d2441d350 0x556d247d9110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:55:17.958 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:17.952+0000 7f5dea60a740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f5ddc1aea40 con 0x556d243fc670 2024-09-21T03:55:17.959 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:17.952+0000 7f5dea60a740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f5ddc1ae370 con 0x556d243f4490 2024-09-21T03:55:17.959 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:17.952+0000 7f5dea60a740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f5ddc1ab5d0 con 0x556d2441d350 2024-09-21T03:55:17.959 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:17.952+0000 7f5dea60a740 10 client.0 _ll_register_callbacks cb 0x7f5dd40046c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:55:17.959 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:17.952+0000 7f5de1631640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556d243fc670 0x556d247d8bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:55:17.959 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:17.952+0000 7f5de1631640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556d243fc670 0x556d247d8bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:47448/0 (socket says 172.21.15.39:47448) 2024-09-21T03:55:17.959 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:17.952+0000 7f5de1631640 1 -- 172.21.15.39:0/2110807600 learned_addr learned my addr 172.21.15.39:0/2110807600 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:55:17.959 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:17.952+0000 7f5de2633640 1 --2- 172.21.15.39:0/2110807600 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556d2441d350 0x556d247d9110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:55:17.959 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:17.952+0000 7f5de1e32640 1 --2- 172.21.15.39:0/2110807600 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556d243f4490 0x556d247dc710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:55:17.959 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:17.952+0000 7f5de1631640 1 -- 172.21.15.39:0/2110807600 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556d2441d350 msgr2=0x556d247d9110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:55:17.959 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:17.952+0000 7f5de1631640 1 --2- 172.21.15.39:0/2110807600 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556d2441d350 0x556d247d9110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:17.959 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:17.952+0000 7f5de1631640 1 -- 172.21.15.39:0/2110807600 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556d243f4490 msgr2=0x556d247dc710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:55:17.959 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:17.952+0000 7f5de1631640 1 --2- 172.21.15.39:0/2110807600 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556d243f4490 0x556d247dc710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:17.959 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:17.952+0000 7f5de1631640 1 -- 172.21.15.39:0/2110807600 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556d247d9650 con 0x556d243fc670 2024-09-21T03:55:17.959 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:17.953+0000 7f5de1631640 1 --2- 172.21.15.39:0/2110807600 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556d243fc670 0x556d247d8bd0 secure :-1 s=READY pgs=1222 cs=0 l=1 rev1=1 crypto rx=0x7f5dd0033000 tx=0x7f5dd00058d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=167841fb0886dc41 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:55:17.960 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:17.953+0000 7f5dc17fa640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-09-21T03:55:17.960 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:17.953+0000 7f5dc17fa640 1 -- 172.21.15.39:0/2110807600 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5dd005b060 con 0x556d243fc670 2024-09-21T03:55:17.960 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:17.953+0000 7f5dc17fa640 1 -- 172.21.15.39:0/2110807600 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8572+0+0 (secure 0 0 0) 0x7f5dd0056900 con 0x556d243fc670 2024-09-21T03:55:17.960 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:17.953+0000 7f5dea60a740 1 -- 172.21.15.39:0/2110807600 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5dcc020a00 con 0x556d243fc670 2024-09-21T03:55:17.960 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:17.953+0000 7f5dc17fa640 1 -- 172.21.15.39:0/2110807600 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f5dd0035040 con 0x556d243fc670 2024-09-21T03:55:17.960 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:17.953+0000 7f5dea60a740 10 client.17970 fetch_fsmap learned FSMap version 29 2024-09-21T03:55:17.960 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:17.953+0000 7f5dea60a740 1 -- 172.21.15.39:0/2110807600 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x556d247ca7c0 con 0x556d243fc670 2024-09-21T03:55:17.960 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:17.953+0000 7f5dc17fa640 1 -- 172.21.15.39:0/2110807600 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5dcc020a00 con 0x556d243fc670 2024-09-21T03:55:17.960 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:17.953+0000 7f5dc17fa640 1 -- 172.21.15.39:0/2110807600 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f5dd00047e0 con 0x556d243fc670 2024-09-21T03:55:17.960 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:17.954+0000 7f5dea60a740 10 client.17970 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T03:55:17.960 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:17.954+0000 7f5dea60a740 10 client.17970 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T03:55:17.960 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:17.954+0000 7f5dea60a740 20 client.17970 populate_metadata read hostname 'smithi039' 2024-09-21T03:55:17.961 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:17.954+0000 7f5dea60a740 1 --2- 172.21.15.39:0/2110807600 >> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] conn(0x556d243b1080 0x556d247ccb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:55:17.961 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:17.954+0000 7f5dea60a740 4 client.17970 mds_command: new command op to 24695 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-21T03:55:17.961 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:17.954+0000 7f5dea60a740 1 -- 172.21.15.39:0/2110807600 --> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x556d247c6b90 con 0x556d243b1080 2024-09-21T03:55:17.961 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:17.954+0000 7f5de1e32640 1 --2- 172.21.15.39:0/2110807600 >> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] conn(0x556d243b1080 0x556d247ccb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:55:17.961 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:17.954+0000 7f5dc17fa640 10 client.17970 ms_handle_connect on v2:172.21.15.150:6838/2949386907 2024-09-21T03:55:17.961 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:17.954+0000 7f5de1e32640 1 --2- 172.21.15.39:0/2110807600 >> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] conn(0x556d243b1080 0x556d247ccb40 crc :-1 s=READY pgs=152 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f74473c001088545 server_cookie=b9232292fedc0296 in_seq=0 out_seq=0 2024-09-21T03:55:17.962 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:17.956+0000 7f5dc17fa640 1 -- 172.21.15.39:0/2110807600 <== mds.0 v2:172.21.15.150:6838/2949386907 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x556d247c6b90 con 0x556d243b1080 2024-09-21T03:55:17.962 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:17.956+0000 7f5dc17fa640 10 client.17970 handle_command_reply: tid=0 2024-09-21T03:55:17.962 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:17.956+0000 7f5dea60a740 1 client.17970 shutdown 2024-09-21T03:55:17.962 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:17.956+0000 7f5dea60a740 1 -- 172.21.15.39:0/2110807600 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556d243fc670 msgr2=0x556d247d8bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:55:17.962 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:17.956+0000 7f5dea60a740 1 --2- 172.21.15.39:0/2110807600 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556d243fc670 0x556d247d8bd0 secure :-1 s=READY pgs=1222 cs=0 l=1 rev1=1 crypto rx=0x7f5dd0033000 tx=0x7f5dd00058d0 comp rx=0 tx=0).stop 2024-09-21T03:55:17.962 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:17.957+0000 7f5dea60a740 1 -- 172.21.15.39:0/2110807600 shutdown_connections 2024-09-21T03:55:17.962 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:17.957+0000 7f5dea60a740 1 --2- 172.21.15.39:0/2110807600 >> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] conn(0x556d243b1080 0x556d247ccb40 crc :-1 s=READY pgs=152 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:17.962 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:17.957+0000 7f5dea60a740 1 --2- 172.21.15.39:0/2110807600 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556d2441d350 0x556d247d9110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:17.962 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:17.957+0000 7f5dea60a740 1 --2- 172.21.15.39:0/2110807600 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556d243fc670 0x556d247d8bd0 unknown :-1 s=CLOSED pgs=1222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:17.963 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:17.957+0000 7f5dea60a740 1 --2- 172.21.15.39:0/2110807600 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556d243f4490 0x556d247dc710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:17.963 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:17.957+0000 7f5dea60a740 1 -- 172.21.15.39:0/2110807600 >> 172.21.15.39:0/2110807600 conn(0x556d24411740 msgr2=0x556d247c7cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:55:17.963 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:17.957+0000 7f5dc17fa640 0 client.17970 ms_handle_reset on v2:172.21.15.150:6838/2949386907 2024-09-21T03:55:17.963 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:17.957+0000 7f5dea60a740 1 -- 172.21.15.39:0/2110807600 shutdown_connections 2024-09-21T03:55:17.963 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:17.957+0000 7f5dea60a740 1 -- 172.21.15.39:0/2110807600 wait complete. 2024-09-21T03:55:17.963 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:17.957+0000 7f5dea60a740 20 client.17970 trim_cache size 0 max 16384 2024-09-21T03:55:17.963 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-09-21T03:55:17.963 INFO:teuthology.orchestra.run.smithi039.stdout: "return_code": 0, 2024-09-21T03:55:17.963 INFO:teuthology.orchestra.run.smithi039.stdout: "scrub_tag": "10e282c4-137c-4fb2-9af2-a83dc3945587", 2024-09-21T03:55:17.963 INFO:teuthology.orchestra.run.smithi039.stdout: "mode": "asynchronous" 2024-09-21T03:55:17.964 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-09-21T03:55:17.964 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:17.957+0000 7f5de4096640 1 -- 172.21.15.39:0/3663062657 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f5dc40b6680 msgr2=0x7f5dc40b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:55:17.964 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:17.957+0000 7f5de4096640 1 --2- 172.21.15.39:0/3663062657 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f5dc40b6680 0x7f5dc40b8b90 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f5dcc005fd0 tx=0x7f5dcc009780 comp rx=0 tx=0).stop 2024-09-21T03:55:17.964 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:17.957+0000 7f5de4096640 1 -- 172.21.15.39:0/3663062657 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5ddc103b80 msgr2=0x7f5ddc19a520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:55:17.964 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:17.957+0000 7f5de4096640 1 --2- 172.21.15.39:0/3663062657 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5ddc103b80 0x7f5ddc19a520 secure :-1 s=READY pgs=1218 cs=0 l=1 rev1=1 crypto rx=0x7f5dd400d950 tx=0x7f5dd400de10 comp rx=0 tx=0).stop 2024-09-21T03:55:17.964 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:17.958+0000 7f5de4096640 1 -- 172.21.15.39:0/3663062657 shutdown_connections 2024-09-21T03:55:17.964 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:17.958+0000 7f5de4096640 1 --2- 172.21.15.39:0/3663062657 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f5dc40b6680 0x7f5dc40b8b90 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:17.964 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:17.958+0000 7f5de4096640 1 --2- 172.21.15.39:0/3663062657 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5ddc109e10 0x7f5ddc19fd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:17.965 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:17.958+0000 7f5de4096640 1 --2- 172.21.15.39:0/3663062657 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5ddc1094c0 0x7f5ddc19aac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:17.965 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:17.958+0000 7f5de4096640 1 --2- 172.21.15.39:0/3663062657 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5ddc103b80 0x7f5ddc19a520 unknown :-1 s=CLOSED pgs=1218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:17.965 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:17.958+0000 7f5de4096640 1 -- 172.21.15.39:0/3663062657 >> 172.21.15.39:0/3663062657 conn(0x7f5ddc0ff610 msgr2=0x7f5ddc101a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:55:17.965 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:17.958+0000 7f5de4096640 1 -- 172.21.15.39:0/3663062657 shutdown_connections 2024-09-21T03:55:17.965 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:17.958+0000 7f5de4096640 1 -- 172.21.15.39:0/3663062657 wait complete. 2024-09-21T03:55:18.876 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 51116d5c-77ca-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub status 2024-09-21T03:55:19.139 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/51116d5c-77ca-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:55:19.260 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:55:19 smithi039 ceph-mon[31211]: pgmap v608: 97 pgs: 97 active+clean; 10 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-09-21T03:55:19.260 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:55:19 smithi039 ceph-mon[31211]: scrub queued for path: / 2024-09-21T03:55:19.260 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:55:19 smithi039 ceph-mon[31211]: scrub summary: idle+waiting paths [/] 2024-09-21T03:55:19.260 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:55:19 smithi039 ceph-mon[31211]: scrub summary: active paths [/] 2024-09-21T03:55:19.260 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:55:19 smithi039 ceph-mon[31211]: scrub summary: idle+waiting paths [/] 2024-09-21T03:55:19.468 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:55:19 smithi161 ceph-mon[34533]: pgmap v608: 97 pgs: 97 active+clean; 10 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-09-21T03:55:19.468 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:55:19 smithi161 ceph-mon[34533]: scrub queued for path: / 2024-09-21T03:55:19.468 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:55:19 smithi161 ceph-mon[34533]: scrub summary: idle+waiting paths [/] 2024-09-21T03:55:19.468 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:55:19 smithi161 ceph-mon[34533]: scrub summary: active paths [/] 2024-09-21T03:55:19.468 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:55:19 smithi161 ceph-mon[34533]: scrub summary: idle+waiting paths [/] 2024-09-21T03:55:19.474 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:55:19 smithi150 ceph-mon[34289]: pgmap v608: 97 pgs: 97 active+clean; 10 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-09-21T03:55:19.475 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:55:19 smithi150 ceph-mon[34289]: scrub queued for path: / 2024-09-21T03:55:19.475 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:55:19 smithi150 ceph-mon[34289]: scrub summary: idle+waiting paths [/] 2024-09-21T03:55:19.475 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:55:19 smithi150 ceph-mon[34289]: scrub summary: active paths [/] 2024-09-21T03:55:19.475 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:55:19 smithi150 ceph-mon[34289]: scrub summary: idle+waiting paths [/] 2024-09-21T03:55:19.653 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:19.650+0000 7f09295c4640 1 -- 172.21.15.39:0/1419811171 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f09241049b0 msgr2=0x7f0924104dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:55:19.653 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:19.650+0000 7f09295c4640 1 --2- 172.21.15.39:0/1419811171 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f09241049b0 0x7f0924104dc0 secure :-1 s=READY pgs=1223 cs=0 l=1 rev1=1 crypto rx=0x7f090c009900 tx=0x7f090c02a410 comp rx=0 tx=0).stop 2024-09-21T03:55:19.654 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:19.650+0000 7f09295c4640 1 -- 172.21.15.39:0/1419811171 shutdown_connections 2024-09-21T03:55:19.654 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:19.650+0000 7f09295c4640 1 --2- 172.21.15.39:0/1419811171 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f092410de10 0x7f0924110300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:19.654 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:19.650+0000 7f09295c4640 1 --2- 172.21.15.39:0/1419811171 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0924105300 0x7f092410d850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:19.654 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:19.650+0000 7f09295c4640 1 --2- 172.21.15.39:0/1419811171 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f09241049b0 0x7f0924104dc0 unknown :-1 s=CLOSED pgs=1223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:19.654 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:19.650+0000 7f09295c4640 1 -- 172.21.15.39:0/1419811171 >> 172.21.15.39:0/1419811171 conn(0x7f0924079730 msgr2=0x7f0924079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:55:19.654 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:19.650+0000 7f09295c4640 1 -- 172.21.15.39:0/1419811171 shutdown_connections 2024-09-21T03:55:19.654 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:19.650+0000 7f09295c4640 1 -- 172.21.15.39:0/1419811171 wait complete. 2024-09-21T03:55:19.654 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:19.650+0000 7f09295c4640 1 Processor -- start 2024-09-21T03:55:19.654 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:19.651+0000 7f09295c4640 1 -- start start 2024-09-21T03:55:19.656 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:19.651+0000 7f09295c4640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f09241049b0 0x7f09241026f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:55:19.656 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:19.651+0000 7f09295c4640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0924105300 0x7f0924102c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:55:19.656 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:19.651+0000 7f09295c4640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f092410de10 0x7f0924103170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:55:19.656 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:19.651+0000 7f09295c4640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f0924113b50 con 0x7f0924105300 2024-09-21T03:55:19.656 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:19.651+0000 7f09295c4640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f09241139d0 con 0x7f09241049b0 2024-09-21T03:55:19.656 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:19.651+0000 7f09295c4640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f0924113cd0 con 0x7f092410de10 2024-09-21T03:55:19.656 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:19.651+0000 7f09227fc640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0924105300 0x7f0924102c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:55:19.656 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:19.651+0000 7f09227fc640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0924105300 0x7f0924102c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:47458/0 (socket says 172.21.15.39:47458) 2024-09-21T03:55:19.656 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:19.651+0000 7f09227fc640 1 -- 172.21.15.39:0/1591474161 learned_addr learned my addr 172.21.15.39:0/1591474161 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:55:19.656 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:19.651+0000 7f09237fe640 1 --2- 172.21.15.39:0/1591474161 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f092410de10 0x7f0924103170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:55:19.656 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:19.651+0000 7f0922ffd640 1 --2- 172.21.15.39:0/1591474161 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f09241049b0 0x7f09241026f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:55:19.657 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:19.651+0000 7f09227fc640 1 -- 172.21.15.39:0/1591474161 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f092410de10 msgr2=0x7f0924103170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:55:19.657 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:19.651+0000 7f09227fc640 1 --2- 172.21.15.39:0/1591474161 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f092410de10 0x7f0924103170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:19.657 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:19.651+0000 7f09227fc640 1 -- 172.21.15.39:0/1591474161 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f09241049b0 msgr2=0x7f09241026f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:55:19.657 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:19.651+0000 7f09227fc640 1 --2- 172.21.15.39:0/1591474161 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f09241049b0 0x7f09241026f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:19.657 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:19.651+0000 7f09227fc640 1 -- 172.21.15.39:0/1591474161 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f09241a8920 con 0x7f0924105300 2024-09-21T03:55:19.657 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:19.652+0000 7f09227fc640 1 --2- 172.21.15.39:0/1591474161 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0924105300 0x7f0924102c30 secure :-1 s=READY pgs=1224 cs=0 l=1 rev1=1 crypto rx=0x7f0914006fd0 tx=0x7f091400b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:55:19.657 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:19.652+0000 7f091bfff640 1 -- 172.21.15.39:0/1591474161 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0914011830 con 0x7f0924105300 2024-09-21T03:55:19.657 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:19.652+0000 7f09295c4640 1 -- 172.21.15.39:0/1591474161 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f09241a8c10 con 0x7f0924105300 2024-09-21T03:55:19.657 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:19.652+0000 7f091bfff640 1 -- 172.21.15.39:0/1591474161 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0914016070 con 0x7f0924105300 2024-09-21T03:55:19.657 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:19.652+0000 7f09295c4640 1 -- 172.21.15.39:0/1591474161 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f09241a91a0 con 0x7f0924105300 2024-09-21T03:55:19.657 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:19.652+0000 7f091bfff640 1 -- 172.21.15.39:0/1591474161 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0914010380 con 0x7f0924105300 2024-09-21T03:55:19.659 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:19.655+0000 7f092fb38740 1 Processor -- start 2024-09-21T03:55:19.660 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:19.655+0000 7f091bfff640 1 -- 172.21.15.39:0/1591474161 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 27) ==== 154643+0+0 (secure 0 0 0) 0x7f0914010520 con 0x7f0924105300 2024-09-21T03:55:19.660 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:19.655+0000 7f092fb38740 1 -- start start 2024-09-21T03:55:19.660 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:19.655+0000 7f092fb38740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562cd2f5c070 0x562cd3303650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:55:19.660 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:19.655+0000 7f092fb38740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562cd2f331b0 0x562cd3303b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:55:19.660 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:19.655+0000 7f092fb38740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562cd2f3b390 0x562cd33040d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:55:19.660 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:19.655+0000 7f092fb38740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x562cd3279e50 con 0x562cd2f5c070 2024-09-21T03:55:19.660 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:19.656+0000 7f092fb38740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x562cd3269280 con 0x562cd2f3b390 2024-09-21T03:55:19.660 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:19.656+0000 7f092fb38740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x562cd326d090 con 0x562cd2f331b0 2024-09-21T03:55:19.660 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:19.656+0000 7f09237fe640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562cd2f5c070 0x562cd3303650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:55:19.660 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:19.656+0000 7f09237fe640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562cd2f5c070 0x562cd3303650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:47462/0 (socket says 172.21.15.39:47462) 2024-09-21T03:55:19.660 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:19.656+0000 7f09237fe640 1 -- 172.21.15.39:0/2387010173 learned_addr learned my addr 172.21.15.39:0/2387010173 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:55:19.661 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:19.656+0000 7f0922ffd640 1 --2- 172.21.15.39:0/2387010173 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562cd2f331b0 0x562cd3303b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:55:19.661 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:19.656+0000 7f09227fc640 1 --2- 172.21.15.39:0/2387010173 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562cd2f3b390 0x562cd33040d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:55:19.661 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:19.656+0000 7f09237fe640 1 -- 172.21.15.39:0/2387010173 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562cd2f331b0 msgr2=0x562cd3303b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:55:19.661 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:19.656+0000 7f09237fe640 1 --2- 172.21.15.39:0/2387010173 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562cd2f331b0 0x562cd3303b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:19.661 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:19.656+0000 7f09237fe640 1 -- 172.21.15.39:0/2387010173 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562cd2f3b390 msgr2=0x562cd33040d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:55:19.661 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:19.656+0000 7f09237fe640 1 --2- 172.21.15.39:0/2387010173 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562cd2f3b390 0x562cd33040d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:19.661 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:19.656+0000 7f09237fe640 1 -- 172.21.15.39:0/2387010173 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562cd2fc7650 con 0x562cd2f5c070 2024-09-21T03:55:19.661 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:19.656+0000 7f091bfff640 1 --2- 172.21.15.39:0/1591474161 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f09100b6680 0x7f09100b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:55:19.661 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:19.656+0000 7f09227fc640 1 --2- 172.21.15.39:0/2387010173 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562cd2f3b390 0x562cd33040d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:55:19.661 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:19.656+0000 7f0922ffd640 1 --2- 172.21.15.39:0/1591474161 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f09100b6680 0x7f09100b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:55:19.661 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:19.656+0000 7f091bfff640 1 -- 172.21.15.39:0/1591474161 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8572+0+0 (secure 0 0 0) 0x7f09140e6a30 con 0x7f0924105300 2024-09-21T03:55:19.661 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:19.656+0000 7f09237fe640 1 --2- 172.21.15.39:0/2387010173 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562cd2f5c070 0x562cd3303650 secure :-1 s=READY pgs=1225 cs=0 l=1 rev1=1 crypto rx=0x7f091c00a430 tx=0x7f091c009650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9db9cfecad1eced7 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:55:19.662 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:19.657+0000 7f09197fa640 1 -- 172.21.15.39:0/2387010173 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f091c013790 con 0x562cd2f5c070 2024-09-21T03:55:19.662 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:19.657+0000 7f09197fa640 1 -- 172.21.15.39:0/2387010173 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f091c019070 con 0x562cd2f5c070 2024-09-21T03:55:19.662 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:19.657+0000 7f09197fa640 1 -- 172.21.15.39:0/2387010173 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f091c014ce0 con 0x562cd2f5c070 2024-09-21T03:55:19.666 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:19.658+0000 7f092fb38740 1 -- 172.21.15.39:0/2387010173 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562cd2f5c070 msgr2=0x562cd3303650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:55:19.666 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:19.658+0000 7f092fb38740 1 --2- 172.21.15.39:0/2387010173 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562cd2f5c070 0x562cd3303650 secure :-1 s=READY pgs=1225 cs=0 l=1 rev1=1 crypto rx=0x7f091c00a430 tx=0x7f091c009650 comp rx=0 tx=0).stop 2024-09-21T03:55:19.666 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:19.658+0000 7f0922ffd640 1 --2- 172.21.15.39:0/1591474161 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f09100b6680 0x7f09100b8b90 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f090c009230 tx=0x7f090c031400 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:55:19.666 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:19.658+0000 7f092fb38740 1 -- 172.21.15.39:0/2387010173 shutdown_connections 2024-09-21T03:55:19.666 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:19.658+0000 7f092fb38740 1 --2- 172.21.15.39:0/2387010173 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562cd2f3b390 0x562cd33040d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:19.666 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:19.658+0000 7f092fb38740 1 --2- 172.21.15.39:0/2387010173 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562cd2f331b0 0x562cd3303b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:19.666 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:19.658+0000 7f092fb38740 1 --2- 172.21.15.39:0/2387010173 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562cd2f5c070 0x562cd3303650 unknown :-1 s=CLOSED pgs=1225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:19.667 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:19.658+0000 7f092fb38740 1 -- 172.21.15.39:0/2387010173 >> 172.21.15.39:0/2387010173 conn(0x562cd2f50460 msgr2=0x562cd3301540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:55:19.667 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:19.658+0000 7f092fb38740 1 -- 172.21.15.39:0/2387010173 shutdown_connections 2024-09-21T03:55:19.667 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:19.658+0000 7f092fb38740 1 -- 172.21.15.39:0/2387010173 wait complete. 2024-09-21T03:55:19.667 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:19.658+0000 7f092fb38740 1 -- start start 2024-09-21T03:55:19.667 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:19.658+0000 7f092fb38740 1 Processor -- start 2024-09-21T03:55:19.667 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:19.659+0000 7f092fb38740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562cd2f331b0 0x562cd330ec80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:55:19.667 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:19.659+0000 7f092fb38740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562cd2f3b390 0x562cd330f1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:55:19.667 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:19.659+0000 7f092fb38740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562cd2f5c070 0x562cd330f740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:55:19.667 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:19.659+0000 7f092fb38740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x562cd30ca700 con 0x562cd2f3b390 2024-09-21T03:55:19.667 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:19.659+0000 7f092fb38740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x562cd3204110 con 0x562cd2f331b0 2024-09-21T03:55:19.667 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:19.659+0000 7f092fb38740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x562cd3311e30 con 0x562cd2f5c070 2024-09-21T03:55:19.667 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:19.659+0000 7f092fb38740 10 client.0 _ll_register_callbacks cb 0x562cd321d990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:55:19.668 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:19.659+0000 7f09227fc640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562cd2f3b390 0x562cd330f1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:55:19.668 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:19.659+0000 7f09227fc640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562cd2f3b390 0x562cd330f1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:47464/0 (socket says 172.21.15.39:47464) 2024-09-21T03:55:19.668 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:19.659+0000 7f09227fc640 1 -- 172.21.15.39:0/3308903929 learned_addr learned my addr 172.21.15.39:0/3308903929 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:55:19.668 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:19.659+0000 7f09237fe640 1 --2- 172.21.15.39:0/3308903929 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562cd2f5c070 0x562cd330f740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:55:19.668 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:19.659+0000 7f0922ffd640 1 --2- 172.21.15.39:0/3308903929 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562cd2f331b0 0x562cd330ec80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:55:19.668 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:19.659+0000 7f09227fc640 1 -- 172.21.15.39:0/3308903929 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562cd2f5c070 msgr2=0x562cd330f740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:55:19.668 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:19.659+0000 7f09227fc640 1 --2- 172.21.15.39:0/3308903929 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562cd2f5c070 0x562cd330f740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:19.668 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:19.659+0000 7f09227fc640 1 -- 172.21.15.39:0/3308903929 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562cd2f331b0 msgr2=0x562cd330ec80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:55:19.668 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:19.659+0000 7f09227fc640 1 --2- 172.21.15.39:0/3308903929 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562cd2f331b0 0x562cd330ec80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:19.668 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:19.659+0000 7f09227fc640 1 -- 172.21.15.39:0/3308903929 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562cd3231c40 con 0x562cd2f3b390 2024-09-21T03:55:19.668 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:19.659+0000 7f09227fc640 1 --2- 172.21.15.39:0/3308903929 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562cd2f3b390 0x562cd330f1c0 secure :-1 s=READY pgs=1226 cs=0 l=1 rev1=1 crypto rx=0x7f0914009fd0 tx=0x7f09140e4650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b1087e850535643a server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:55:19.668 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:19.659+0000 7f090a7fc640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-09-21T03:55:19.669 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:19.660+0000 7f090a7fc640 1 -- 172.21.15.39:0/3308903929 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09140ac050 con 0x562cd2f3b390 2024-09-21T03:55:19.669 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:19.660+0000 7f090a7fc640 1 -- 172.21.15.39:0/3308903929 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8572+0+0 (secure 0 0 0) 0x7f0914013b60 con 0x562cd2f3b390 2024-09-21T03:55:19.669 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:19.660+0000 7f092fb38740 1 -- 172.21.15.39:0/3308903929 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f091c019070 con 0x562cd2f3b390 2024-09-21T03:55:19.669 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:19.660+0000 7f090a7fc640 1 -- 172.21.15.39:0/3308903929 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f09140ac3f0 con 0x562cd2f3b390 2024-09-21T03:55:19.669 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:19.660+0000 7f092fb38740 10 client.17982 fetch_fsmap learned FSMap version 29 2024-09-21T03:55:19.669 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:19.660+0000 7f092fb38740 1 -- 172.21.15.39:0/3308903929 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x562cd331daa0 con 0x562cd2f3b390 2024-09-21T03:55:19.669 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:19.660+0000 7f090a7fc640 1 -- 172.21.15.39:0/3308903929 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f091c019070 con 0x562cd2f3b390 2024-09-21T03:55:19.669 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:19.660+0000 7f090a7fc640 1 -- 172.21.15.39:0/3308903929 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f09140a8230 con 0x562cd2f3b390 2024-09-21T03:55:19.669 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:19.660+0000 7f092fb38740 10 client.17982 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T03:55:19.669 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:19.660+0000 7f092fb38740 10 client.17982 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T03:55:19.669 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:19.660+0000 7f092fb38740 20 client.17982 populate_metadata read hostname 'smithi039' 2024-09-21T03:55:19.670 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:19.660+0000 7f092fb38740 1 --2- 172.21.15.39:0/3308903929 >> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] conn(0x562cd2f1a9f0 0x562cd3321000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:55:19.670 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:19.660+0000 7f092fb38740 4 client.17982 mds_command: new command op to 24695 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:55:19.670 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:19.660+0000 7f092fb38740 1 -- 172.21.15.39:0/3308903929 --> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562cd327bda0 con 0x562cd2f1a9f0 2024-09-21T03:55:19.670 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:19.661+0000 7f0922ffd640 1 --2- 172.21.15.39:0/3308903929 >> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] conn(0x562cd2f1a9f0 0x562cd3321000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:55:19.670 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:19.661+0000 7f090a7fc640 10 client.17982 ms_handle_connect on v2:172.21.15.150:6838/2949386907 2024-09-21T03:55:19.670 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:19.661+0000 7f0922ffd640 1 --2- 172.21.15.39:0/3308903929 >> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] conn(0x562cd2f1a9f0 0x562cd3321000 crc :-1 s=READY pgs=153 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5a5b692de5405494 server_cookie=efdaca264ab3eacc in_seq=0 out_seq=0 2024-09-21T03:55:19.670 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:19.662+0000 7f090a7fc640 1 -- 172.21.15.39:0/3308903929 <== mds.0 v2:172.21.15.150:6838/2949386907 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x562cd327bda0 con 0x562cd2f1a9f0 2024-09-21T03:55:19.670 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:19.662+0000 7f090a7fc640 10 client.17982 handle_command_reply: tid=0 2024-09-21T03:55:19.670 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:19.662+0000 7f092fb38740 1 client.17982 shutdown 2024-09-21T03:55:19.670 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:19.662+0000 7f092fb38740 1 -- 172.21.15.39:0/3308903929 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562cd2f3b390 msgr2=0x562cd330f1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:55:19.670 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:19.662+0000 7f092fb38740 1 --2- 172.21.15.39:0/3308903929 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562cd2f3b390 0x562cd330f1c0 secure :-1 s=READY pgs=1226 cs=0 l=1 rev1=1 crypto rx=0x7f0914009fd0 tx=0x7f09140e4650 comp rx=0 tx=0).stop 2024-09-21T03:55:19.670 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:19.662+0000 7f092fb38740 1 -- 172.21.15.39:0/3308903929 shutdown_connections 2024-09-21T03:55:19.671 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:19.662+0000 7f092fb38740 1 --2- 172.21.15.39:0/3308903929 >> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] conn(0x562cd2f1a9f0 0x562cd3321000 crc :-1 s=READY pgs=153 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:19.671 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:19.662+0000 7f092fb38740 1 --2- 172.21.15.39:0/3308903929 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562cd2f5c070 0x562cd330f740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:19.671 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:19.662+0000 7f092fb38740 1 --2- 172.21.15.39:0/3308903929 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562cd2f3b390 0x562cd330f1c0 unknown :-1 s=CLOSED pgs=1226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:19.671 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:19.662+0000 7f092fb38740 1 --2- 172.21.15.39:0/3308903929 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562cd2f331b0 0x562cd330ec80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:19.671 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:19.662+0000 7f090a7fc640 0 client.17982 ms_handle_reset on v2:172.21.15.150:6838/2949386907 2024-09-21T03:55:19.671 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:19.662+0000 7f092fb38740 1 -- 172.21.15.39:0/3308903929 >> 172.21.15.39:0/3308903929 conn(0x562cd2f50460 msgr2=0x562cd3306800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:55:19.671 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:19.662+0000 7f092fb38740 1 -- 172.21.15.39:0/3308903929 shutdown_connections 2024-09-21T03:55:19.671 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:19.662+0000 7f092fb38740 1 -- 172.21.15.39:0/3308903929 wait complete. 2024-09-21T03:55:19.671 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:19.662+0000 7f092fb38740 20 client.17982 trim_cache size 0 max 16384 2024-09-21T03:55:19.682 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:19.680+0000 7f092fb38740 1 Processor -- start 2024-09-21T03:55:19.683 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:19.680+0000 7f092fb38740 1 -- start start 2024-09-21T03:55:19.684 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:19.680+0000 7f092fb38740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562cd2f331b0 0x562cd3305040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:55:19.684 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:19.680+0000 7f092fb38740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562cd2f3b390 0x562cd33108b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:55:19.684 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:19.680+0000 7f092fb38740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562cd2f5c070 0x562cd3300b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:55:19.684 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:19.680+0000 7f092fb38740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f09241b1b40 con 0x562cd2f331b0 2024-09-21T03:55:19.684 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:19.680+0000 7f092fb38740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f09241b3680 con 0x562cd2f5c070 2024-09-21T03:55:19.684 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:19.680+0000 7f092fb38740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f09241b2210 con 0x562cd2f3b390 2024-09-21T03:55:19.685 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:19.680+0000 7f0922ffd640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562cd2f331b0 0x562cd3305040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:55:19.685 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:19.680+0000 7f0922ffd640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562cd2f331b0 0x562cd3305040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:47476/0 (socket says 172.21.15.39:47476) 2024-09-21T03:55:19.685 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:19.680+0000 7f0922ffd640 1 -- 172.21.15.39:0/1778982173 learned_addr learned my addr 172.21.15.39:0/1778982173 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:55:19.685 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:19.681+0000 7f09227fc640 1 --2- 172.21.15.39:0/1778982173 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562cd2f3b390 0x562cd33108b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:55:19.685 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:19.681+0000 7f09237fe640 1 --2- 172.21.15.39:0/1778982173 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562cd2f5c070 0x562cd3300b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:55:19.685 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:19.681+0000 7f0922ffd640 1 -- 172.21.15.39:0/1778982173 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562cd2f3b390 msgr2=0x562cd33108b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:55:19.685 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:19.681+0000 7f0922ffd640 1 --2- 172.21.15.39:0/1778982173 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562cd2f3b390 0x562cd33108b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:19.685 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:19.681+0000 7f0922ffd640 1 -- 172.21.15.39:0/1778982173 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562cd2f5c070 msgr2=0x562cd3300b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:55:19.685 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:19.681+0000 7f0922ffd640 1 --2- 172.21.15.39:0/1778982173 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562cd2f5c070 0x562cd3300b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:19.685 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:19.681+0000 7f0922ffd640 1 -- 172.21.15.39:0/1778982173 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562cd3305c60 con 0x562cd2f331b0 2024-09-21T03:55:19.685 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:19.681+0000 7f09227fc640 1 --2- 172.21.15.39:0/1778982173 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562cd2f3b390 0x562cd33108b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:55:19.686 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:19.681+0000 7f09237fe640 1 --2- 172.21.15.39:0/1778982173 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562cd2f5c070 0x562cd3300b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:55:19.686 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:19.681+0000 7f0922ffd640 1 --2- 172.21.15.39:0/1778982173 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562cd2f331b0 0x562cd3305040 secure :-1 s=READY pgs=1227 cs=0 l=1 rev1=1 crypto rx=0x7f090c032fd0 tx=0x7f090c03e070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=92ac26dffdaa3ebe server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:55:19.686 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:19.681+0000 7f0918ff9640 1 -- 172.21.15.39:0/1778982173 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f090c035040 con 0x562cd2f331b0 2024-09-21T03:55:19.686 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:19.681+0000 7f0918ff9640 1 -- 172.21.15.39:0/1778982173 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f090c0035f0 con 0x562cd2f331b0 2024-09-21T03:55:19.686 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:19.681+0000 7f0918ff9640 1 -- 172.21.15.39:0/1778982173 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f090c03c710 con 0x562cd2f331b0 2024-09-21T03:55:19.689 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:19.682+0000 7f092fb38740 1 -- 172.21.15.39:0/1778982173 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562cd2f331b0 msgr2=0x562cd3305040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:55:19.689 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:19.682+0000 7f092fb38740 1 --2- 172.21.15.39:0/1778982173 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562cd2f331b0 0x562cd3305040 secure :-1 s=READY pgs=1227 cs=0 l=1 rev1=1 crypto rx=0x7f090c032fd0 tx=0x7f090c03e070 comp rx=0 tx=0).stop 2024-09-21T03:55:19.689 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:19.682+0000 7f092fb38740 1 -- 172.21.15.39:0/1778982173 shutdown_connections 2024-09-21T03:55:19.689 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:19.682+0000 7f092fb38740 1 --2- 172.21.15.39:0/1778982173 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562cd2f5c070 0x562cd3300b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:19.689 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:19.682+0000 7f092fb38740 1 --2- 172.21.15.39:0/1778982173 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562cd2f3b390 0x562cd33108b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:19.689 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:19.682+0000 7f092fb38740 1 --2- 172.21.15.39:0/1778982173 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562cd2f331b0 0x562cd3305040 unknown :-1 s=CLOSED pgs=1227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:19.690 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:19.682+0000 7f092fb38740 1 -- 172.21.15.39:0/1778982173 >> 172.21.15.39:0/1778982173 conn(0x562cd2f50460 msgr2=0x562cd33101a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:55:19.690 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:19.682+0000 7f092fb38740 1 -- 172.21.15.39:0/1778982173 shutdown_connections 2024-09-21T03:55:19.690 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:19.682+0000 7f092fb38740 1 -- 172.21.15.39:0/1778982173 wait complete. 2024-09-21T03:55:19.690 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:19.683+0000 7f092fb38740 1 -- start start 2024-09-21T03:55:19.690 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:19.683+0000 7f092fb38740 1 Processor -- start 2024-09-21T03:55:19.690 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:19.683+0000 7f092fb38740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562cd2f331b0 0x562cd3313340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:55:19.690 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:19.683+0000 7f092fb38740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562cd2f3b390 0x562cd3313880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:55:19.690 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:19.683+0000 7f092fb38740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562cd2f5c070 0x562cd338a990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:55:19.690 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:19.683+0000 7f092fb38740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f09241b2fb0 con 0x562cd2f3b390 2024-09-21T03:55:19.690 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:19.683+0000 7f092fb38740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f09241b28e0 con 0x562cd2f5c070 2024-09-21T03:55:19.690 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:19.683+0000 7f092fb38740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f09241afb40 con 0x562cd2f331b0 2024-09-21T03:55:19.690 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:19.683+0000 7f092fb38740 10 client.0 _ll_register_callbacks cb 0x7f091401b4f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:55:19.690 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:19.683+0000 7f09227fc640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562cd2f3b390 0x562cd3313880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:55:19.690 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:19.683+0000 7f09227fc640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562cd2f3b390 0x562cd3313880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:47488/0 (socket says 172.21.15.39:47488) 2024-09-21T03:55:19.691 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:19.683+0000 7f09227fc640 1 -- 172.21.15.39:0/1593581851 learned_addr learned my addr 172.21.15.39:0/1593581851 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:55:19.691 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:19.683+0000 7f09237fe640 1 --2- 172.21.15.39:0/1593581851 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562cd2f5c070 0x562cd338a990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:55:19.691 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:19.683+0000 7f0922ffd640 1 --2- 172.21.15.39:0/1593581851 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562cd2f331b0 0x562cd3313340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:55:19.691 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:19.683+0000 7f09227fc640 1 -- 172.21.15.39:0/1593581851 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562cd2f331b0 msgr2=0x562cd3313340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:55:19.691 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:19.683+0000 7f09227fc640 1 --2- 172.21.15.39:0/1593581851 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562cd2f331b0 0x562cd3313340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:19.691 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:19.683+0000 7f09227fc640 1 -- 172.21.15.39:0/1593581851 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562cd2f5c070 msgr2=0x562cd338a990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:55:19.691 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:19.683+0000 7f09227fc640 1 --2- 172.21.15.39:0/1593581851 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562cd2f5c070 0x562cd338a990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:19.691 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:19.683+0000 7f09227fc640 1 -- 172.21.15.39:0/1593581851 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562cd3314100 con 0x562cd2f3b390 2024-09-21T03:55:19.691 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:19.683+0000 7f09237fe640 1 --2- 172.21.15.39:0/1593581851 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562cd2f5c070 0x562cd338a990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:55:19.691 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:19.684+0000 7f09227fc640 1 --2- 172.21.15.39:0/1593581851 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562cd2f3b390 0x562cd3313880 secure :-1 s=READY pgs=1228 cs=0 l=1 rev1=1 crypto rx=0x7f09140e4750 tx=0x7f09140abdd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=219bd8f677133ee1 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:55:19.691 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:19.684+0000 7f090a7fc640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-09-21T03:55:19.691 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:19.684+0000 7f090a7fc640 1 -- 172.21.15.39:0/1593581851 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09140ac050 con 0x562cd2f3b390 2024-09-21T03:55:19.691 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:19.684+0000 7f090a7fc640 1 -- 172.21.15.39:0/1593581851 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8572+0+0 (secure 0 0 0) 0x7f09140ab7c0 con 0x562cd2f3b390 2024-09-21T03:55:19.691 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:19.684+0000 7f092fb38740 1 -- 172.21.15.39:0/1593581851 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f090c0035f0 con 0x562cd2f3b390 2024-09-21T03:55:19.692 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:19.684+0000 7f090a7fc640 1 -- 172.21.15.39:0/1593581851 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f09140a8f30 con 0x562cd2f3b390 2024-09-21T03:55:19.692 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:19.684+0000 7f092fb38740 10 client.17988 fetch_fsmap learned FSMap version 29 2024-09-21T03:55:19.692 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:19.684+0000 7f092fb38740 1 -- 172.21.15.39:0/1593581851 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x562cd338bef0 con 0x562cd2f3b390 2024-09-21T03:55:19.692 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:19.684+0000 7f090a7fc640 1 -- 172.21.15.39:0/1593581851 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f090c0035f0 con 0x562cd2f3b390 2024-09-21T03:55:19.692 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:19.684+0000 7f090a7fc640 1 -- 172.21.15.39:0/1593581851 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f09140c2a00 con 0x562cd2f3b390 2024-09-21T03:55:19.692 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:19.684+0000 7f092fb38740 10 client.17988 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T03:55:19.692 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:19.684+0000 7f092fb38740 10 client.17988 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T03:55:19.692 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:19.684+0000 7f092fb38740 20 client.17988 populate_metadata read hostname 'smithi039' 2024-09-21T03:55:19.692 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:19.685+0000 7f092fb38740 1 --2- 172.21.15.39:0/1593581851 >> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] conn(0x562cd2eefda0 0x562cd338e660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:55:19.692 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:19.685+0000 7f092fb38740 4 client.17988 mds_command: new command op to 24695 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-21T03:55:19.692 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:19.685+0000 7f092fb38740 1 -- 172.21.15.39:0/1593581851 --> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] -- command(tid 0: {"prefix": "scrub status"}) -- 0x562cd3277850 con 0x562cd2eefda0 2024-09-21T03:55:19.692 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:19.685+0000 7f0922ffd640 1 --2- 172.21.15.39:0/1593581851 >> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] conn(0x562cd2eefda0 0x562cd338e660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:55:19.692 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:19.685+0000 7f090a7fc640 10 client.17988 ms_handle_connect on v2:172.21.15.150:6838/2949386907 2024-09-21T03:55:19.692 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:19.685+0000 7f0922ffd640 1 --2- 172.21.15.39:0/1593581851 >> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] conn(0x562cd2eefda0 0x562cd338e660 crc :-1 s=READY pgs=154 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8981e62e38586be3 server_cookie=d5d5835991f23033 in_seq=0 out_seq=0 2024-09-21T03:55:19.693 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:19.686+0000 7f090a7fc640 1 -- 172.21.15.39:0/1593581851 <== mds.0 v2:172.21.15.150:6838/2949386907 1 ==== command_reply(tid 0: 0 ) ==== 8+0+5398 (crc 0 0 0) 0x562cd3277850 con 0x562cd2eefda0 2024-09-21T03:55:19.693 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:19.686+0000 7f090a7fc640 10 client.17988 handle_command_reply: tid=0 2024-09-21T03:55:19.693 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:19.686+0000 7f092fb38740 1 client.17988 shutdown 2024-09-21T03:55:19.693 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:19.686+0000 7f092fb38740 1 -- 172.21.15.39:0/1593581851 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562cd2f3b390 msgr2=0x562cd3313880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:55:19.693 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:19.686+0000 7f092fb38740 1 --2- 172.21.15.39:0/1593581851 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562cd2f3b390 0x562cd3313880 secure :-1 s=READY pgs=1228 cs=0 l=1 rev1=1 crypto rx=0x7f09140e4750 tx=0x7f09140abdd0 comp rx=0 tx=0).stop 2024-09-21T03:55:19.693 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:19.686+0000 7f092fb38740 1 -- 172.21.15.39:0/1593581851 shutdown_connections 2024-09-21T03:55:19.693 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:19.686+0000 7f092fb38740 1 --2- 172.21.15.39:0/1593581851 >> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] conn(0x562cd2eefda0 0x562cd338e660 crc :-1 s=READY pgs=154 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:19.693 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:19.686+0000 7f092fb38740 1 --2- 172.21.15.39:0/1593581851 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562cd2f5c070 0x562cd338a990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:19.693 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:19.686+0000 7f092fb38740 1 --2- 172.21.15.39:0/1593581851 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562cd2f3b390 0x562cd3313880 unknown :-1 s=CLOSED pgs=1228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:19.693 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:19.686+0000 7f092fb38740 1 --2- 172.21.15.39:0/1593581851 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562cd2f331b0 0x562cd3313340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:19.693 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:19.686+0000 7f092fb38740 1 -- 172.21.15.39:0/1593581851 >> 172.21.15.39:0/1593581851 conn(0x562cd2f50460 msgr2=0x562cd331a9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:55:19.693 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:19.686+0000 7f090a7fc640 0 client.17988 ms_handle_reset on v2:172.21.15.150:6838/2949386907 2024-09-21T03:55:19.693 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:19.686+0000 7f092fb38740 1 -- 172.21.15.39:0/1593581851 shutdown_connections 2024-09-21T03:55:19.693 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:19.686+0000 7f092fb38740 1 -- 172.21.15.39:0/1593581851 wait complete. 2024-09-21T03:55:19.693 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:19.686+0000 7f092fb38740 20 client.17988 trim_cache size 0 max 16384 2024-09-21T03:55:19.694 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-09-21T03:55:19.694 INFO:teuthology.orchestra.run.smithi039.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-21T03:55:19.694 INFO:teuthology.orchestra.run.smithi039.stdout: "scrubs": { 2024-09-21T03:55:19.694 INFO:teuthology.orchestra.run.smithi039.stdout: "10e282c4-137c-4fb2-9af2-a83dc3945587": { 2024-09-21T03:55:19.694 INFO:teuthology.orchestra.run.smithi039.stdout: "path": "/", 2024-09-21T03:55:19.694 INFO:teuthology.orchestra.run.smithi039.stdout: "tag": "10e282c4-137c-4fb2-9af2-a83dc3945587", 2024-09-21T03:55:19.694 INFO:teuthology.orchestra.run.smithi039.stdout: "options": "recursive,force" 2024-09-21T03:55:19.694 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-21T03:55:19.695 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:55:19.695 INFO:teuthology.orchestra.run.smithi039.stdout: "scrub_stats": { 2024-09-21T03:55:19.695 INFO:teuthology.orchestra.run.smithi039.stdout: "02e16ae1-0bac-41e9-b95e-fc6b17d207ac": { 2024-09-21T03:55:19.695 INFO:teuthology.orchestra.run.smithi039.stdout: "start_time": "2024-09-21T03:54:31.853401+0000", 2024-09-21T03:55:19.695 INFO:teuthology.orchestra.run.smithi039.stdout: "path": "/", 2024-09-21T03:55:19.695 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_started": 0, 2024-09-21T03:55:19.695 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_passed": 0, 2024-09-21T03:55:19.695 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_failed": 0, 2024-09-21T03:55:19.695 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_skipped": 0 2024-09-21T03:55:19.695 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:55:19.695 INFO:teuthology.orchestra.run.smithi039.stdout: "e28dcd13-f2ac-4cb7-a346-58917d542b15": { 2024-09-21T03:55:19.695 INFO:teuthology.orchestra.run.smithi039.stdout: "start_time": "2024-09-21T03:54:18.221680+0000", 2024-09-21T03:55:19.695 INFO:teuthology.orchestra.run.smithi039.stdout: "path": "/", 2024-09-21T03:55:19.695 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_started": 0, 2024-09-21T03:55:19.696 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_passed": 0, 2024-09-21T03:55:19.696 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_failed": 0, 2024-09-21T03:55:19.696 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_skipped": 0 2024-09-21T03:55:19.696 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:55:19.696 INFO:teuthology.orchestra.run.smithi039.stdout: "ad87da79-0248-4e18-9694-8228cb95422f": { 2024-09-21T03:55:19.696 INFO:teuthology.orchestra.run.smithi039.stdout: "start_time": "2024-09-21T03:53:18.441907+0000", 2024-09-21T03:55:19.696 INFO:teuthology.orchestra.run.smithi039.stdout: "path": "/", 2024-09-21T03:55:19.696 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_started": 0, 2024-09-21T03:55:19.696 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_passed": 0, 2024-09-21T03:55:19.696 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_failed": 0, 2024-09-21T03:55:19.696 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_skipped": 0 2024-09-21T03:55:19.696 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:55:19.697 INFO:teuthology.orchestra.run.smithi039.stdout: "02509e4f-3911-4983-b16f-df8ded0b9998": { 2024-09-21T03:55:19.697 INFO:teuthology.orchestra.run.smithi039.stdout: "start_time": "2024-09-21T03:46:17.889576+0000", 2024-09-21T03:55:19.697 INFO:teuthology.orchestra.run.smithi039.stdout: "path": "/", 2024-09-21T03:55:19.697 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_started": 0, 2024-09-21T03:55:19.697 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_passed": 0, 2024-09-21T03:55:19.697 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_failed": 0, 2024-09-21T03:55:19.697 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_skipped": 0 2024-09-21T03:55:19.697 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:55:19.697 INFO:teuthology.orchestra.run.smithi039.stdout: "10728de8-c49d-4edc-be2b-639249255e94": { 2024-09-21T03:55:19.697 INFO:teuthology.orchestra.run.smithi039.stdout: "start_time": "2024-09-21T03:47:03.965623+0000", 2024-09-21T03:55:19.697 INFO:teuthology.orchestra.run.smithi039.stdout: "path": "/", 2024-09-21T03:55:19.697 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_started": 0, 2024-09-21T03:55:19.698 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_passed": 0, 2024-09-21T03:55:19.698 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_failed": 0, 2024-09-21T03:55:19.698 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_skipped": 0 2024-09-21T03:55:19.698 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:55:19.698 INFO:teuthology.orchestra.run.smithi039.stdout: "10e282c4-137c-4fb2-9af2-a83dc3945587": { 2024-09-21T03:55:19.698 INFO:teuthology.orchestra.run.smithi039.stdout: "start_time": "2024-09-21T03:55:17.955458+0000", 2024-09-21T03:55:19.698 INFO:teuthology.orchestra.run.smithi039.stdout: "path": "/", 2024-09-21T03:55:19.698 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_started": 0, 2024-09-21T03:55:19.698 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_passed": 0, 2024-09-21T03:55:19.698 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_failed": 0, 2024-09-21T03:55:19.698 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_skipped": 0 2024-09-21T03:55:19.698 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:55:19.699 INFO:teuthology.orchestra.run.smithi039.stdout: "9860c2f4-b7b1-47aa-865a-9a1c4434a5f6": { 2024-09-21T03:55:19.699 INFO:teuthology.orchestra.run.smithi039.stdout: "start_time": "2024-09-21T03:49:17.718867+0000", 2024-09-21T03:55:19.699 INFO:teuthology.orchestra.run.smithi039.stdout: "path": "/", 2024-09-21T03:55:19.699 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_started": 0, 2024-09-21T03:55:19.699 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_passed": 0, 2024-09-21T03:55:19.699 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_failed": 0, 2024-09-21T03:55:19.699 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_skipped": 0 2024-09-21T03:55:19.699 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:55:19.699 INFO:teuthology.orchestra.run.smithi039.stdout: "d9391d69-baac-453a-9d52-d046760c070d": { 2024-09-21T03:55:19.699 INFO:teuthology.orchestra.run.smithi039.stdout: "start_time": "2024-09-21T03:53:32.364622+0000", 2024-09-21T03:55:19.699 INFO:teuthology.orchestra.run.smithi039.stdout: "path": "/", 2024-09-21T03:55:19.700 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_started": 0, 2024-09-21T03:55:19.700 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_passed": 0, 2024-09-21T03:55:19.700 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_failed": 0, 2024-09-21T03:55:19.700 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_skipped": 0 2024-09-21T03:55:19.700 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:55:19.700 INFO:teuthology.orchestra.run.smithi039.stdout: "c7c700f3-2cd8-41ac-8381-b57dd1b37b02": { 2024-09-21T03:55:19.700 INFO:teuthology.orchestra.run.smithi039.stdout: "start_time": "2024-09-21T03:52:32.368858+0000", 2024-09-21T03:55:19.700 INFO:teuthology.orchestra.run.smithi039.stdout: "path": "/", 2024-09-21T03:55:19.700 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_started": 0, 2024-09-21T03:55:19.700 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_passed": 0, 2024-09-21T03:55:19.700 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_failed": 0, 2024-09-21T03:55:19.700 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_skipped": 0 2024-09-21T03:55:19.701 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:55:19.701 INFO:teuthology.orchestra.run.smithi039.stdout: "29411307-eb8d-4339-8d2b-47ebd32c0080": { 2024-09-21T03:55:19.701 INFO:teuthology.orchestra.run.smithi039.stdout: "start_time": "2024-09-21T03:47:38.092920+0000", 2024-09-21T03:55:19.701 INFO:teuthology.orchestra.run.smithi039.stdout: "path": "/", 2024-09-21T03:55:19.701 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_started": 0, 2024-09-21T03:55:19.701 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_passed": 0, 2024-09-21T03:55:19.701 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_failed": 0, 2024-09-21T03:55:19.701 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_skipped": 0 2024-09-21T03:55:19.701 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:55:19.701 INFO:teuthology.orchestra.run.smithi039.stdout: "ede73e6e-3fd0-4a7f-b5bb-f68946f57eea": { 2024-09-21T03:55:19.701 INFO:teuthology.orchestra.run.smithi039.stdout: "start_time": "2024-09-21T03:51:32.235096+0000", 2024-09-21T03:55:19.702 INFO:teuthology.orchestra.run.smithi039.stdout: "path": "/", 2024-09-21T03:55:19.702 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_started": 0, 2024-09-21T03:55:19.702 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_passed": 0, 2024-09-21T03:55:19.702 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_failed": 0, 2024-09-21T03:55:19.702 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_skipped": 0 2024-09-21T03:55:19.702 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:55:19.702 INFO:teuthology.orchestra.run.smithi039.stdout: "e28ad743-2b22-45b3-8799-a638c2883623": { 2024-09-21T03:55:19.702 INFO:teuthology.orchestra.run.smithi039.stdout: "start_time": "2024-09-21T03:48:31.448821+0000", 2024-09-21T03:55:19.702 INFO:teuthology.orchestra.run.smithi039.stdout: "path": "/", 2024-09-21T03:55:19.702 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_started": 0, 2024-09-21T03:55:19.702 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_passed": 0, 2024-09-21T03:55:19.703 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_failed": 0, 2024-09-21T03:55:19.703 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_skipped": 0 2024-09-21T03:55:19.703 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:55:19.703 INFO:teuthology.orchestra.run.smithi039.stdout: "6e760f1f-9048-4f92-bd66-d708fbef75b3": { 2024-09-21T03:55:19.703 INFO:teuthology.orchestra.run.smithi039.stdout: "start_time": "2024-09-21T03:50:03.711918+0000", 2024-09-21T03:55:19.703 INFO:teuthology.orchestra.run.smithi039.stdout: "path": "/", 2024-09-21T03:55:19.703 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_started": 0, 2024-09-21T03:55:19.703 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_passed": 0, 2024-09-21T03:55:19.703 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_failed": 0, 2024-09-21T03:55:19.703 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_skipped": 0 2024-09-21T03:55:19.703 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:55:19.704 INFO:teuthology.orchestra.run.smithi039.stdout: "c7e2b8e4-057d-435f-b2bd-b633de98035f": { 2024-09-21T03:55:19.704 INFO:teuthology.orchestra.run.smithi039.stdout: "start_time": "2024-09-21T03:47:19.400305+0000", 2024-09-21T03:55:19.704 INFO:teuthology.orchestra.run.smithi039.stdout: "path": "/", 2024-09-21T03:55:19.704 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_started": 0, 2024-09-21T03:55:19.704 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_passed": 0, 2024-09-21T03:55:19.704 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_failed": 0, 2024-09-21T03:55:19.704 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_skipped": 0 2024-09-21T03:55:19.704 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:55:19.704 INFO:teuthology.orchestra.run.smithi039.stdout: "74ab917e-ecea-4778-a1c3-c184214fb24c": { 2024-09-21T03:55:19.704 INFO:teuthology.orchestra.run.smithi039.stdout: "start_time": "2024-09-21T03:50:17.726630+0000", 2024-09-21T03:55:19.704 INFO:teuthology.orchestra.run.smithi039.stdout: "path": "/", 2024-09-21T03:55:19.704 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_started": 0, 2024-09-21T03:55:19.705 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_passed": 0, 2024-09-21T03:55:19.705 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_failed": 0, 2024-09-21T03:55:19.705 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_skipped": 0 2024-09-21T03:55:19.705 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:55:19.705 INFO:teuthology.orchestra.run.smithi039.stdout: "160750d8-401f-4810-866d-e63e7e4bc3e0": { 2024-09-21T03:55:19.705 INFO:teuthology.orchestra.run.smithi039.stdout: "start_time": "2024-09-21T03:51:04.208673+0000", 2024-09-21T03:55:19.705 INFO:teuthology.orchestra.run.smithi039.stdout: "path": "/", 2024-09-21T03:55:19.705 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_started": 0, 2024-09-21T03:55:19.705 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_passed": 0, 2024-09-21T03:55:19.705 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_failed": 0, 2024-09-21T03:55:19.705 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_skipped": 0 2024-09-21T03:55:19.705 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:55:19.706 INFO:teuthology.orchestra.run.smithi039.stdout: "ae1fc4cf-5458-47d0-afcb-282b136dba15": { 2024-09-21T03:55:19.706 INFO:teuthology.orchestra.run.smithi039.stdout: "start_time": "2024-09-21T03:51:18.560378+0000", 2024-09-21T03:55:19.706 INFO:teuthology.orchestra.run.smithi039.stdout: "path": "/", 2024-09-21T03:55:19.706 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_started": 0, 2024-09-21T03:55:19.706 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_passed": 0, 2024-09-21T03:55:19.706 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_failed": 0, 2024-09-21T03:55:19.706 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_skipped": 0 2024-09-21T03:55:19.706 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:55:19.706 INFO:teuthology.orchestra.run.smithi039.stdout: "4bda6ee3-7237-4ee4-96ce-150ad50f26dc": { 2024-09-21T03:55:19.706 INFO:teuthology.orchestra.run.smithi039.stdout: "start_time": "2024-09-21T03:52:18.622141+0000", 2024-09-21T03:55:19.706 INFO:teuthology.orchestra.run.smithi039.stdout: "path": "/", 2024-09-21T03:55:19.706 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_started": 0, 2024-09-21T03:55:19.707 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_passed": 0, 2024-09-21T03:55:19.707 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_failed": 0, 2024-09-21T03:55:19.707 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_skipped": 0 2024-09-21T03:55:19.707 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-21T03:55:19.707 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-21T03:55:19.707 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-09-21T03:55:19.708 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:19.687+0000 7f09295c4640 1 -- 172.21.15.39:0/1591474161 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f09100b6680 msgr2=0x7f09100b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:55:19.708 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:19.687+0000 7f09295c4640 1 --2- 172.21.15.39:0/1591474161 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f09100b6680 0x7f09100b8b90 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f090c009230 tx=0x7f090c031400 comp rx=0 tx=0).stop 2024-09-21T03:55:19.708 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:19.687+0000 7f09295c4640 1 -- 172.21.15.39:0/1591474161 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0924105300 msgr2=0x7f0924102c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:55:19.708 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:19.687+0000 7f09295c4640 1 --2- 172.21.15.39:0/1591474161 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0924105300 0x7f0924102c30 secure :-1 s=READY pgs=1224 cs=0 l=1 rev1=1 crypto rx=0x7f0914006fd0 tx=0x7f091400b8e0 comp rx=0 tx=0).stop 2024-09-21T03:55:19.708 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:19.687+0000 7f09295c4640 1 -- 172.21.15.39:0/1591474161 shutdown_connections 2024-09-21T03:55:19.708 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:19.687+0000 7f09295c4640 1 --2- 172.21.15.39:0/1591474161 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f09100b6680 0x7f09100b8b90 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:19.708 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:19.687+0000 7f09295c4640 1 --2- 172.21.15.39:0/1591474161 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f092410de10 0x7f0924103170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:19.708 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:19.687+0000 7f09295c4640 1 --2- 172.21.15.39:0/1591474161 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0924105300 0x7f0924102c30 unknown :-1 s=CLOSED pgs=1224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:19.708 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:19.687+0000 7f09295c4640 1 --2- 172.21.15.39:0/1591474161 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f09241049b0 0x7f09241026f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:19.708 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:19.687+0000 7f09295c4640 1 -- 172.21.15.39:0/1591474161 >> 172.21.15.39:0/1591474161 conn(0x7f0924079730 msgr2=0x7f09240ff830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:55:19.709 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:19.687+0000 7f09295c4640 1 -- 172.21.15.39:0/1591474161 shutdown_connections 2024-09-21T03:55:19.709 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:19.687+0000 7f09295c4640 1 -- 172.21.15.39:0/1591474161 wait complete. 2024-09-21T03:55:20.545 INFO:tasks.cephfs.filesystem:scrub status for tag:10e282c4-137c-4fb2-9af2-a83dc3945587 - {'path': '/', 'tag': '10e282c4-137c-4fb2-9af2-a83dc3945587', 'options': 'recursive,force'} 2024-09-21T03:55:21.218 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:55:20 smithi161 ceph-mon[34533]: pgmap v609: 97 pgs: 97 active+clean; 10 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 204 B/s wr, 10 op/s 2024-09-21T03:55:21.218 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:55:20 smithi161 ceph-mon[34533]: osdmap e85: 12 total, 12 up, 12 in 2024-09-21T03:55:21.218 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:55:20 smithi161 ceph-mon[34533]: scrub summary: idle 2024-09-21T03:55:21.225 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:55:20 smithi150 ceph-mon[34289]: pgmap v609: 97 pgs: 97 active+clean; 10 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 204 B/s wr, 10 op/s 2024-09-21T03:55:21.225 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:55:20 smithi150 ceph-mon[34289]: osdmap e85: 12 total, 12 up, 12 in 2024-09-21T03:55:21.225 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:55:20 smithi150 ceph-mon[34289]: scrub summary: idle 2024-09-21T03:55:21.334 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:55:20 smithi039 ceph-mon[31211]: pgmap v609: 97 pgs: 97 active+clean; 10 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 204 B/s wr, 10 op/s 2024-09-21T03:55:21.334 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:55:20 smithi039 ceph-mon[31211]: osdmap e85: 12 total, 12 up, 12 in 2024-09-21T03:55:21.334 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:55:20 smithi039 ceph-mon[31211]: scrub summary: idle 2024-09-21T03:55:23.218 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:55:22 smithi161 ceph-mon[34533]: pgmap v611: 97 pgs: 97 active+clean; 10 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 204 B/s wr, 10 op/s 2024-09-21T03:55:23.225 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:55:22 smithi150 ceph-mon[34289]: pgmap v611: 97 pgs: 97 active+clean; 10 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 204 B/s wr, 10 op/s 2024-09-21T03:55:23.334 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:55:22 smithi039 ceph-mon[31211]: pgmap v611: 97 pgs: 97 active+clean; 10 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 204 B/s wr, 10 op/s 2024-09-21T03:55:25.218 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:55:24 smithi161 ceph-mon[34533]: pgmap v612: 97 pgs: 97 active+clean; 10 MiB data, 525 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 102 B/s wr, 10 op/s 2024-09-21T03:55:25.224 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:55:24 smithi150 ceph-mon[34289]: pgmap v612: 97 pgs: 97 active+clean; 10 MiB data, 525 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 102 B/s wr, 10 op/s 2024-09-21T03:55:25.334 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:55:24 smithi039 ceph-mon[31211]: pgmap v612: 97 pgs: 97 active+clean; 10 MiB data, 525 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 102 B/s wr, 10 op/s 2024-09-21T03:55:27.218 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:55:26 smithi161 ceph-mon[34533]: pgmap v613: 97 pgs: 97 active+clean; 10 MiB data, 525 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 102 B/s wr, 10 op/s 2024-09-21T03:55:27.224 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:55:26 smithi150 ceph-mon[34289]: pgmap v613: 97 pgs: 97 active+clean; 10 MiB data, 525 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 102 B/s wr, 10 op/s 2024-09-21T03:55:27.334 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:55:26 smithi039 ceph-mon[31211]: pgmap v613: 97 pgs: 97 active+clean; 10 MiB data, 525 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 102 B/s wr, 10 op/s 2024-09-21T03:55:29.218 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:55:28 smithi161 ceph-mon[34533]: pgmap v614: 97 pgs: 97 active+clean; 10 MiB data, 525 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 102 B/s wr, 10 op/s 2024-09-21T03:55:29.225 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:55:28 smithi150 ceph-mon[34289]: pgmap v614: 97 pgs: 97 active+clean; 10 MiB data, 525 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 102 B/s wr, 10 op/s 2024-09-21T03:55:29.336 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:55:28 smithi039 ceph-mon[31211]: pgmap v614: 97 pgs: 97 active+clean; 10 MiB data, 525 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 102 B/s wr, 10 op/s 2024-09-21T03:55:31.218 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:55:30 smithi161 ceph-mon[34533]: pgmap v615: 97 pgs: 97 active+clean; 10 MiB data, 525 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 10 op/s 2024-09-21T03:55:31.218 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:55:30 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:55:31.224 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:55:30 smithi150 ceph-mon[34289]: pgmap v615: 97 pgs: 97 active+clean; 10 MiB data, 525 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 10 op/s 2024-09-21T03:55:31.224 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:55:30 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:55:31.333 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:55:30 smithi039 ceph-mon[31211]: pgmap v615: 97 pgs: 97 active+clean; 10 MiB data, 525 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 10 op/s 2024-09-21T03:55:31.334 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:55:30 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:55:32.218 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:55:31 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:55:32.218 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:55:31 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:55:32.218 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:55:31 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:55:32.224 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:55:31 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:55:32.225 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:55:31 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:55:32.225 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:55:31 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:55:32.334 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:55:31 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:55:32.334 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:55:31 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:55:32.334 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:55:31 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:55:33.334 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:55:32 smithi039 ceph-mon[31211]: pgmap v616: 97 pgs: 97 active+clean; 10 MiB data, 525 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 9 op/s 2024-09-21T03:55:33.468 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:55:32 smithi161 ceph-mon[34533]: pgmap v616: 97 pgs: 97 active+clean; 10 MiB data, 525 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 9 op/s 2024-09-21T03:55:33.474 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:55:32 smithi150 ceph-mon[34289]: pgmap v616: 97 pgs: 97 active+clean; 10 MiB data, 525 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 9 op/s 2024-09-21T03:55:35.334 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:55:34 smithi039 ceph-mon[31211]: pgmap v617: 97 pgs: 97 active+clean; 10 MiB data, 525 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-09-21T03:55:35.468 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:55:34 smithi161 ceph-mon[34533]: pgmap v617: 97 pgs: 97 active+clean; 10 MiB data, 525 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-09-21T03:55:35.474 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:55:34 smithi150 ceph-mon[34289]: pgmap v617: 97 pgs: 97 active+clean; 10 MiB data, 525 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-09-21T03:55:37.334 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:55:36 smithi039 ceph-mon[31211]: pgmap v618: 97 pgs: 97 active+clean; 10 MiB data, 525 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 10 op/s 2024-09-21T03:55:37.468 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:55:36 smithi161 ceph-mon[34533]: pgmap v618: 97 pgs: 97 active+clean; 10 MiB data, 525 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 10 op/s 2024-09-21T03:55:37.475 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:55:36 smithi150 ceph-mon[34289]: pgmap v618: 97 pgs: 97 active+clean; 10 MiB data, 525 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 10 op/s 2024-09-21T03:55:39.334 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:55:38 smithi039 ceph-mon[31211]: pgmap v619: 97 pgs: 97 active+clean; 10 MiB data, 525 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-09-21T03:55:39.468 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:55:38 smithi161 ceph-mon[34533]: pgmap v619: 97 pgs: 97 active+clean; 10 MiB data, 525 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-09-21T03:55:39.475 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:55:38 smithi150 ceph-mon[34289]: pgmap v619: 97 pgs: 97 active+clean; 10 MiB data, 525 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-09-21T03:55:41.270 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:55:41 smithi161 ceph-mon[34533]: pgmap v620: 97 pgs: 97 active+clean; 10 MiB data, 525 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-09-21T03:55:41.334 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:55:41 smithi039 ceph-mon[31211]: pgmap v620: 97 pgs: 97 active+clean; 10 MiB data, 525 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-09-21T03:55:41.475 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:55:41 smithi150 ceph-mon[34289]: pgmap v620: 97 pgs: 97 active+clean; 10 MiB data, 525 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-09-21T03:55:43.334 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:55:43 smithi039 ceph-mon[31211]: pgmap v621: 97 pgs: 97 active+clean; 10 MiB data, 525 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-21T03:55:43.468 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:55:43 smithi161 ceph-mon[34533]: pgmap v621: 97 pgs: 97 active+clean; 10 MiB data, 525 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-21T03:55:43.475 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:55:43 smithi150 ceph-mon[34289]: pgmap v621: 97 pgs: 97 active+clean; 10 MiB data, 525 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-21T03:55:45.277 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:55:45 smithi150 ceph-mon[34289]: pgmap v622: 97 pgs: 97 active+clean; 10 MiB data, 525 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-09-21T03:55:45.334 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:55:45 smithi039 ceph-mon[31211]: pgmap v622: 97 pgs: 97 active+clean; 10 MiB data, 525 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-09-21T03:55:45.468 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:55:45 smithi161 ceph-mon[34533]: pgmap v622: 97 pgs: 97 active+clean; 10 MiB data, 525 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-09-21T03:55:47.334 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:55:47 smithi039 ceph-mon[31211]: pgmap v623: 97 pgs: 97 active+clean; 10 MiB data, 525 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 10 op/s 2024-09-21T03:55:47.468 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:55:47 smithi161 ceph-mon[34533]: pgmap v623: 97 pgs: 97 active+clean; 10 MiB data, 525 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 10 op/s 2024-09-21T03:55:47.475 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:55:47 smithi150 ceph-mon[34289]: pgmap v623: 97 pgs: 97 active+clean; 10 MiB data, 525 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 10 op/s 2024-09-21T03:55:49.334 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:55:49 smithi039 ceph-mon[31211]: pgmap v624: 97 pgs: 97 active+clean; 10 MiB data, 525 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-09-21T03:55:49.468 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:55:49 smithi161 ceph-mon[34533]: pgmap v624: 97 pgs: 97 active+clean; 10 MiB data, 525 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-09-21T03:55:49.474 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:55:49 smithi150 ceph-mon[34289]: pgmap v624: 97 pgs: 97 active+clean; 10 MiB data, 525 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-09-21T03:55:50.546 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 51116d5c-77ca-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub status 2024-09-21T03:55:50.817 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/51116d5c-77ca-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:55:51.308 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:55:51 smithi039 ceph-mon[31211]: pgmap v625: 97 pgs: 97 active+clean; 10 MiB data, 525 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-09-21T03:55:51.404 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:51.402+0000 7fcd6b93b640 1 -- 172.21.15.39:0/2267512330 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcd641094a0 msgr2=0x7fcd641098b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:55:51.404 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:51.402+0000 7fcd6b93b640 1 --2- 172.21.15.39:0/2267512330 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcd641094a0 0x7fcd641098b0 secure :-1 s=READY pgs=1229 cs=0 l=1 rev1=1 crypto rx=0x7fcd54009900 tx=0x7fcd5402a410 comp rx=0 tx=0).stop 2024-09-21T03:55:51.404 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:51.402+0000 7fcd6b93b640 1 -- 172.21.15.39:0/2267512330 shutdown_connections 2024-09-21T03:55:51.404 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:51.402+0000 7fcd6b93b640 1 --2- 172.21.15.39:0/2267512330 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcd64109df0 0x7fcd64110620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:51.404 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:51.402+0000 7fcd6b93b640 1 --2- 172.21.15.39:0/2267512330 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcd641094a0 0x7fcd641098b0 unknown :-1 s=CLOSED pgs=1229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:51.404 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:51.402+0000 7fcd6b93b640 1 --2- 172.21.15.39:0/2267512330 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcd64103b60 0x7fcd64103f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:51.404 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:51.402+0000 7fcd6b93b640 1 -- 172.21.15.39:0/2267512330 >> 172.21.15.39:0/2267512330 conn(0x7fcd6407b850 msgr2=0x7fcd6407bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:55:51.404 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:51.402+0000 7fcd6b93b640 1 -- 172.21.15.39:0/2267512330 shutdown_connections 2024-09-21T03:55:51.404 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:51.402+0000 7fcd6b93b640 1 -- 172.21.15.39:0/2267512330 wait complete. 2024-09-21T03:55:51.406 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:51.402+0000 7fcd6b93b640 1 Processor -- start 2024-09-21T03:55:51.406 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:51.403+0000 7fcd6b93b640 1 -- start start 2024-09-21T03:55:51.406 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:51.403+0000 7fcd6b93b640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcd64103b60 0x7fcd6419a560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:55:51.406 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:51.403+0000 7fcd6b93b640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcd641094a0 0x7fcd6419ab00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:55:51.406 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:51.403+0000 7fcd6b93b640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcd64109df0 0x7fcd6419fda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:55:51.406 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:51.403+0000 7fcd6b93b640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fcd64113ab0 con 0x7fcd641094a0 2024-09-21T03:55:51.406 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:51.403+0000 7fcd6b93b640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fcd64113930 con 0x7fcd64109df0 2024-09-21T03:55:51.407 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:51.403+0000 7fcd6b93b640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fcd64113c30 con 0x7fcd64103b60 2024-09-21T03:55:51.407 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:51.403+0000 7fcd68ed6640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcd641094a0 0x7fcd6419ab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:55:51.407 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:51.403+0000 7fcd68ed6640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcd641094a0 0x7fcd6419ab00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:51754/0 (socket says 172.21.15.39:51754) 2024-09-21T03:55:51.407 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:51.403+0000 7fcd68ed6640 1 -- 172.21.15.39:0/241347092 learned_addr learned my addr 172.21.15.39:0/241347092 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:55:51.407 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:51.403+0000 7fcd696d7640 1 --2- 172.21.15.39:0/241347092 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcd64103b60 0x7fcd6419a560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:55:51.407 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:51.403+0000 7fcd69ed8640 1 --2- 172.21.15.39:0/241347092 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcd64109df0 0x7fcd6419fda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:55:51.407 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:51.404+0000 7fcd68ed6640 1 -- 172.21.15.39:0/241347092 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcd64103b60 msgr2=0x7fcd6419a560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:55:51.407 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:51.404+0000 7fcd68ed6640 1 --2- 172.21.15.39:0/241347092 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcd64103b60 0x7fcd6419a560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:51.407 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:51.404+0000 7fcd68ed6640 1 -- 172.21.15.39:0/241347092 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcd64109df0 msgr2=0x7fcd6419fda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:55:51.407 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:51.404+0000 7fcd68ed6640 1 --2- 172.21.15.39:0/241347092 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcd64109df0 0x7fcd6419fda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:51.407 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:51.404+0000 7fcd68ed6640 1 -- 172.21.15.39:0/241347092 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcd641a04d0 con 0x7fcd641094a0 2024-09-21T03:55:51.407 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:51.404+0000 7fcd68ed6640 1 --2- 172.21.15.39:0/241347092 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcd641094a0 0x7fcd6419ab00 secure :-1 s=READY pgs=1230 cs=0 l=1 rev1=1 crypto rx=0x7fcd5402a930 tx=0x7fcd54004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:55:51.407 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:51.404+0000 7fcd5a7fc640 1 -- 172.21.15.39:0/241347092 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd5403f030 con 0x7fcd641094a0 2024-09-21T03:55:51.407 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:51.404+0000 7fcd6b93b640 1 -- 172.21.15.39:0/241347092 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcd641a0760 con 0x7fcd641094a0 2024-09-21T03:55:51.408 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:51.404+0000 7fcd5a7fc640 1 -- 172.21.15.39:0/241347092 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcd54003aa0 con 0x7fcd641094a0 2024-09-21T03:55:51.408 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:51.404+0000 7fcd5a7fc640 1 -- 172.21.15.39:0/241347092 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd54035070 con 0x7fcd641094a0 2024-09-21T03:55:51.408 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:51.404+0000 7fcd6b93b640 1 -- 172.21.15.39:0/241347092 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcd641a6f20 con 0x7fcd641094a0 2024-09-21T03:55:51.410 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:51.407+0000 7fcd71eaf740 1 Processor -- start 2024-09-21T03:55:51.410 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:51.407+0000 7fcd71eaf740 1 -- start start 2024-09-21T03:55:51.410 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:51.407+0000 7fcd5a7fc640 1 -- 172.21.15.39:0/241347092 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 27) ==== 154643+0+0 (secure 0 0 0) 0x7fcd54042030 con 0x7fcd641094a0 2024-09-21T03:55:51.410 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:51.407+0000 7fcd71eaf740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x560201bb1070 0x560201f58650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:55:51.410 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:51.407+0000 7fcd71eaf740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560201b881b0 0x560201f58b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:55:51.411 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:51.407+0000 7fcd71eaf740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560201b90390 0x560201f590d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:55:51.411 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:51.407+0000 7fcd71eaf740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x560201ecee50 con 0x560201bb1070 2024-09-21T03:55:51.411 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:51.407+0000 7fcd71eaf740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x560201ebe280 con 0x560201b881b0 2024-09-21T03:55:51.411 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:51.407+0000 7fcd71eaf740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x560201ec2090 con 0x560201b90390 2024-09-21T03:55:51.411 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:51.407+0000 7fcd69ed8640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x560201bb1070 0x560201f58650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:55:51.411 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:51.407+0000 7fcd69ed8640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x560201bb1070 0x560201f58650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:51762/0 (socket says 172.21.15.39:51762) 2024-09-21T03:55:51.411 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:51.407+0000 7fcd69ed8640 1 -- 172.21.15.39:0/940556458 learned_addr learned my addr 172.21.15.39:0/940556458 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:55:51.411 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:51.408+0000 7fcd696d7640 1 --2- 172.21.15.39:0/940556458 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560201b881b0 0x560201f58b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:55:51.411 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:51.408+0000 7fcd68ed6640 1 --2- 172.21.15.39:0/940556458 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560201b90390 0x560201f590d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:55:51.411 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:51.408+0000 7fcd69ed8640 1 -- 172.21.15.39:0/940556458 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560201b90390 msgr2=0x560201f590d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:55:51.411 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:51.408+0000 7fcd69ed8640 1 --2- 172.21.15.39:0/940556458 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560201b90390 0x560201f590d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:51.411 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:51.408+0000 7fcd69ed8640 1 -- 172.21.15.39:0/940556458 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560201b881b0 msgr2=0x560201f58b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:55:51.411 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:51.408+0000 7fcd69ed8640 1 --2- 172.21.15.39:0/940556458 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560201b881b0 0x560201f58b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:51.411 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:51.408+0000 7fcd69ed8640 1 -- 172.21.15.39:0/940556458 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560201c1c650 con 0x560201bb1070 2024-09-21T03:55:51.412 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:51.408+0000 7fcd696d7640 1 --2- 172.21.15.39:0/940556458 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560201b881b0 0x560201f58b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:55:51.412 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:51.408+0000 7fcd5a7fc640 1 --2- 172.21.15.39:0/241347092 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7fcd500b6680 0x7fcd500b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:55:51.412 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:51.408+0000 7fcd69ed8640 1 --2- 172.21.15.39:0/940556458 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x560201bb1070 0x560201f58650 secure :-1 s=READY pgs=1231 cs=0 l=1 rev1=1 crypto rx=0x7fcd5c009f70 tx=0x7fcd5c010800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3bfa9e3b156af086 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:55:51.412 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:51.408+0000 7fcd696d7640 1 --2- 172.21.15.39:0/241347092 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7fcd500b6680 0x7fcd500b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:55:51.412 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:51.408+0000 7fcd5a7fc640 1 -- 172.21.15.39:0/241347092 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8528+0+0 (secure 0 0 0) 0x7fcd54107d50 con 0x7fcd641094a0 2024-09-21T03:55:51.412 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:51.408+0000 7fcd4f7fe640 1 -- 172.21.15.39:0/940556458 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd5c016070 con 0x560201bb1070 2024-09-21T03:55:51.412 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:51.409+0000 7fcd4f7fe640 1 -- 172.21.15.39:0/940556458 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcd5c004d20 con 0x560201bb1070 2024-09-21T03:55:51.412 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:51.409+0000 7fcd4f7fe640 1 -- 172.21.15.39:0/940556458 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd5c012790 con 0x560201bb1070 2024-09-21T03:55:51.415 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:51.409+0000 7fcd71eaf740 1 -- 172.21.15.39:0/940556458 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x560201bb1070 msgr2=0x560201f58650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:55:51.415 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:51.409+0000 7fcd696d7640 1 --2- 172.21.15.39:0/241347092 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7fcd500b6680 0x7fcd500b8b90 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x560201f572d0 tx=0x7fcd6000b390 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:55:51.415 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:51.409+0000 7fcd71eaf740 1 --2- 172.21.15.39:0/940556458 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x560201bb1070 0x560201f58650 secure :-1 s=READY pgs=1231 cs=0 l=1 rev1=1 crypto rx=0x7fcd5c009f70 tx=0x7fcd5c010800 comp rx=0 tx=0).stop 2024-09-21T03:55:51.415 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:51.410+0000 7fcd71eaf740 1 -- 172.21.15.39:0/940556458 shutdown_connections 2024-09-21T03:55:51.415 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:51.410+0000 7fcd71eaf740 1 --2- 172.21.15.39:0/940556458 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560201b90390 0x560201f590d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:51.415 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:51.410+0000 7fcd71eaf740 1 --2- 172.21.15.39:0/940556458 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560201b881b0 0x560201f58b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:51.415 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:51.410+0000 7fcd71eaf740 1 --2- 172.21.15.39:0/940556458 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x560201bb1070 0x560201f58650 unknown :-1 s=CLOSED pgs=1231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:51.415 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:51.410+0000 7fcd71eaf740 1 -- 172.21.15.39:0/940556458 >> 172.21.15.39:0/940556458 conn(0x560201ba5460 msgr2=0x560201f56540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:55:51.416 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:51.410+0000 7fcd71eaf740 1 -- 172.21.15.39:0/940556458 shutdown_connections 2024-09-21T03:55:51.416 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:51.410+0000 7fcd71eaf740 1 -- 172.21.15.39:0/940556458 wait complete. 2024-09-21T03:55:51.416 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:51.410+0000 7fcd71eaf740 1 -- start start 2024-09-21T03:55:51.416 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:51.410+0000 7fcd71eaf740 1 Processor -- start 2024-09-21T03:55:51.416 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:51.411+0000 7fcd71eaf740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560201b881b0 0x560201f63dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:55:51.416 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:51.411+0000 7fcd71eaf740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x560201b90390 0x560201f64310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:55:51.416 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:51.411+0000 7fcd71eaf740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560201bb1070 0x560201f70120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:55:51.416 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:51.411+0000 7fcd71eaf740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x560201d1f700 con 0x560201b90390 2024-09-21T03:55:51.416 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:51.411+0000 7fcd71eaf740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x560201e59110 con 0x560201bb1070 2024-09-21T03:55:51.416 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:51.411+0000 7fcd71eaf740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x560201f66e30 con 0x560201b881b0 2024-09-21T03:55:51.416 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:51.411+0000 7fcd71eaf740 10 client.0 _ll_register_callbacks cb 0x560201e72990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:55:51.416 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:51.411+0000 7fcd68ed6640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x560201b90390 0x560201f64310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:55:51.416 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:51.411+0000 7fcd68ed6640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x560201b90390 0x560201f64310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:51774/0 (socket says 172.21.15.39:51774) 2024-09-21T03:55:51.417 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:51.411+0000 7fcd68ed6640 1 -- 172.21.15.39:0/4248793389 learned_addr learned my addr 172.21.15.39:0/4248793389 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:55:51.417 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:51.411+0000 7fcd69ed8640 1 --2- 172.21.15.39:0/4248793389 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560201bb1070 0x560201f70120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:55:51.417 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:51.411+0000 7fcd696d7640 1 --2- 172.21.15.39:0/4248793389 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560201b881b0 0x560201f63dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:55:51.417 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:51.411+0000 7fcd68ed6640 1 -- 172.21.15.39:0/4248793389 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560201b881b0 msgr2=0x560201f63dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:55:51.417 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:51.411+0000 7fcd68ed6640 1 --2- 172.21.15.39:0/4248793389 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560201b881b0 0x560201f63dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:51.417 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:51.411+0000 7fcd68ed6640 1 -- 172.21.15.39:0/4248793389 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560201bb1070 msgr2=0x560201f70120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:55:51.417 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:51.411+0000 7fcd68ed6640 1 --2- 172.21.15.39:0/4248793389 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560201bb1070 0x560201f70120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:51.417 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:51.411+0000 7fcd68ed6640 1 -- 172.21.15.39:0/4248793389 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560201e86c40 con 0x560201b90390 2024-09-21T03:55:51.417 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:51.411+0000 7fcd68ed6640 1 --2- 172.21.15.39:0/4248793389 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x560201b90390 0x560201f64310 secure :-1 s=READY pgs=1232 cs=0 l=1 rev1=1 crypto rx=0x7fcd540caec0 tx=0x7fcd5402a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7fd2c24c4c5b1af9 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:55:51.417 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:51.411+0000 7fcd4cff9640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-09-21T03:55:51.417 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:51.412+0000 7fcd4cff9640 1 -- 172.21.15.39:0/4248793389 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd54049070 con 0x560201b90390 2024-09-21T03:55:51.417 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:51.412+0000 7fcd4cff9640 1 -- 172.21.15.39:0/4248793389 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8528+0+0 (secure 0 0 0) 0x7fcd540f10b0 con 0x560201b90390 2024-09-21T03:55:51.417 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:51.412+0000 7fcd71eaf740 1 -- 172.21.15.39:0/4248793389 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcd5c004d20 con 0x560201b90390 2024-09-21T03:55:51.417 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:51.412+0000 7fcd4cff9640 1 -- 172.21.15.39:0/4248793389 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fcd540cd3c0 con 0x560201b90390 2024-09-21T03:55:51.418 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:51.412+0000 7fcd71eaf740 10 client.18000 fetch_fsmap learned FSMap version 29 2024-09-21T03:55:51.418 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:51.412+0000 7fcd71eaf740 1 -- 172.21.15.39:0/4248793389 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x560201f72ba0 con 0x560201b90390 2024-09-21T03:55:51.418 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:51.412+0000 7fcd4cff9640 1 -- 172.21.15.39:0/4248793389 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcd5c004d20 con 0x560201b90390 2024-09-21T03:55:51.418 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:51.412+0000 7fcd4cff9640 1 -- 172.21.15.39:0/4248793389 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fcd54105720 con 0x560201b90390 2024-09-21T03:55:51.418 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:51.412+0000 7fcd71eaf740 10 client.18000 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T03:55:51.418 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:51.412+0000 7fcd71eaf740 10 client.18000 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T03:55:51.418 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:51.412+0000 7fcd71eaf740 20 client.18000 populate_metadata read hostname 'smithi039' 2024-09-21T03:55:51.418 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:51.412+0000 7fcd71eaf740 1 --2- 172.21.15.39:0/4248793389 >> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] conn(0x560201b6f9f0 0x560201f76040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:55:51.418 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:51.412+0000 7fcd71eaf740 4 client.18000 mds_command: new command op to 24695 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:55:51.418 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:51.412+0000 7fcd71eaf740 1 -- 172.21.15.39:0/4248793389 --> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560201ed0da0 con 0x560201b6f9f0 2024-09-21T03:55:51.418 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:51.413+0000 7fcd696d7640 1 --2- 172.21.15.39:0/4248793389 >> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] conn(0x560201b6f9f0 0x560201f76040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:55:51.418 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:51.413+0000 7fcd696d7640 1 --2- 172.21.15.39:0/4248793389 >> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] conn(0x560201b6f9f0 0x560201f76040 crc :-1 s=READY pgs=155 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8e9499f6c1c0028c server_cookie=306053a446a60ff9 in_seq=0 out_seq=0 2024-09-21T03:55:51.418 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:51.413+0000 7fcd4cff9640 10 client.18000 ms_handle_connect on v2:172.21.15.150:6838/2949386907 2024-09-21T03:55:51.419 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:51.414+0000 7fcd4cff9640 1 -- 172.21.15.39:0/4248793389 <== mds.0 v2:172.21.15.150:6838/2949386907 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x560201ed0da0 con 0x560201b6f9f0 2024-09-21T03:55:51.419 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:51.414+0000 7fcd4cff9640 10 client.18000 handle_command_reply: tid=0 2024-09-21T03:55:51.419 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:51.414+0000 7fcd71eaf740 1 client.18000 shutdown 2024-09-21T03:55:51.420 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:51.414+0000 7fcd71eaf740 1 -- 172.21.15.39:0/4248793389 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x560201b90390 msgr2=0x560201f64310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:55:51.420 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:51.414+0000 7fcd71eaf740 1 --2- 172.21.15.39:0/4248793389 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x560201b90390 0x560201f64310 secure :-1 s=READY pgs=1232 cs=0 l=1 rev1=1 crypto rx=0x7fcd540caec0 tx=0x7fcd5402a410 comp rx=0 tx=0).stop 2024-09-21T03:55:51.420 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:51.414+0000 7fcd71eaf740 1 -- 172.21.15.39:0/4248793389 shutdown_connections 2024-09-21T03:55:51.420 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:51.414+0000 7fcd71eaf740 1 --2- 172.21.15.39:0/4248793389 >> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] conn(0x560201b6f9f0 0x560201f76040 crc :-1 s=READY pgs=155 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:51.420 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:51.414+0000 7fcd71eaf740 1 --2- 172.21.15.39:0/4248793389 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560201bb1070 0x560201f70120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:51.420 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:51.414+0000 7fcd71eaf740 1 --2- 172.21.15.39:0/4248793389 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x560201b90390 0x560201f64310 unknown :-1 s=CLOSED pgs=1232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:51.420 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:51.414+0000 7fcd71eaf740 1 --2- 172.21.15.39:0/4248793389 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560201b881b0 0x560201f63dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:51.420 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:51.414+0000 7fcd71eaf740 1 -- 172.21.15.39:0/4248793389 >> 172.21.15.39:0/4248793389 conn(0x560201ba5460 msgr2=0x560201f5b830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:55:51.420 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:51.414+0000 7fcd4cff9640 0 client.18000 ms_handle_reset on v2:172.21.15.150:6838/2949386907 2024-09-21T03:55:51.420 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:51.414+0000 7fcd71eaf740 1 -- 172.21.15.39:0/4248793389 shutdown_connections 2024-09-21T03:55:51.420 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:51.414+0000 7fcd71eaf740 1 -- 172.21.15.39:0/4248793389 wait complete. 2024-09-21T03:55:51.420 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:51.414+0000 7fcd71eaf740 20 client.18000 trim_cache size 0 max 16384 2024-09-21T03:55:51.435 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:51.432+0000 7fcd71eaf740 1 Processor -- start 2024-09-21T03:55:51.435 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:51.432+0000 7fcd71eaf740 1 -- start start 2024-09-21T03:55:51.435 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:51.432+0000 7fcd71eaf740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x560201b881b0 0x560201f56570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:55:51.435 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:51.432+0000 7fcd71eaf740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560201b90390 0x560201f70020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:55:51.435 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:51.432+0000 7fcd71eaf740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560201bb1070 0x560201f5b750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:55:51.435 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:51.432+0000 7fcd71eaf740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fcd641af7a0 con 0x560201b881b0 2024-09-21T03:55:51.435 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:51.432+0000 7fcd71eaf740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fcd641b12e0 con 0x560201b90390 2024-09-21T03:55:51.435 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:51.432+0000 7fcd71eaf740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fcd641afe70 con 0x560201bb1070 2024-09-21T03:55:51.435 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:51.433+0000 7fcd696d7640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x560201b881b0 0x560201f56570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:55:51.435 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:51.433+0000 7fcd696d7640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x560201b881b0 0x560201f56570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:51780/0 (socket says 172.21.15.39:51780) 2024-09-21T03:55:51.436 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:51.433+0000 7fcd696d7640 1 -- 172.21.15.39:0/2794718714 learned_addr learned my addr 172.21.15.39:0/2794718714 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:55:51.436 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:51.433+0000 7fcd69ed8640 1 --2- 172.21.15.39:0/2794718714 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560201bb1070 0x560201f5b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:55:51.436 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:51.433+0000 7fcd68ed6640 1 --2- 172.21.15.39:0/2794718714 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560201b90390 0x560201f70020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:55:51.436 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:51.433+0000 7fcd696d7640 1 -- 172.21.15.39:0/2794718714 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560201bb1070 msgr2=0x560201f5b750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:55:51.436 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:51.433+0000 7fcd696d7640 1 --2- 172.21.15.39:0/2794718714 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560201bb1070 0x560201f5b750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:51.436 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:51.433+0000 7fcd696d7640 1 -- 172.21.15.39:0/2794718714 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560201b90390 msgr2=0x560201f70020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:55:51.436 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:51.433+0000 7fcd696d7640 1 --2- 172.21.15.39:0/2794718714 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560201b90390 0x560201f70020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:51.436 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:51.433+0000 7fcd696d7640 1 -- 172.21.15.39:0/2794718714 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560201f66510 con 0x560201b881b0 2024-09-21T03:55:51.436 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:51.433+0000 7fcd68ed6640 1 --2- 172.21.15.39:0/2794718714 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560201b90390 0x560201f70020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:55:51.436 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:51.433+0000 7fcd696d7640 1 --2- 172.21.15.39:0/2794718714 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x560201b881b0 0x560201f56570 secure :-1 s=READY pgs=1233 cs=0 l=1 rev1=1 crypto rx=0x7fcd600044b0 tx=0x7fcd60015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=424b53a7b5962f84 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:55:51.436 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:51.433+0000 7fcd4effd640 1 -- 172.21.15.39:0/2794718714 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd60005020 con 0x560201b881b0 2024-09-21T03:55:51.436 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:51.433+0000 7fcd4effd640 1 -- 172.21.15.39:0/2794718714 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcd600047a0 con 0x560201b881b0 2024-09-21T03:55:51.436 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:51.433+0000 7fcd4effd640 1 -- 172.21.15.39:0/2794718714 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd60028630 con 0x560201b881b0 2024-09-21T03:55:51.439 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:51.434+0000 7fcd71eaf740 1 -- 172.21.15.39:0/2794718714 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x560201b881b0 msgr2=0x560201f56570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:55:51.439 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:51.434+0000 7fcd71eaf740 1 --2- 172.21.15.39:0/2794718714 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x560201b881b0 0x560201f56570 secure :-1 s=READY pgs=1233 cs=0 l=1 rev1=1 crypto rx=0x7fcd600044b0 tx=0x7fcd60015000 comp rx=0 tx=0).stop 2024-09-21T03:55:51.439 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:51.434+0000 7fcd71eaf740 1 -- 172.21.15.39:0/2794718714 shutdown_connections 2024-09-21T03:55:51.439 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:51.434+0000 7fcd71eaf740 1 --2- 172.21.15.39:0/2794718714 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560201bb1070 0x560201f5b750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:51.439 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:51.434+0000 7fcd71eaf740 1 --2- 172.21.15.39:0/2794718714 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560201b90390 0x560201f70020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:51.439 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:51.434+0000 7fcd71eaf740 1 --2- 172.21.15.39:0/2794718714 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x560201b881b0 0x560201f56570 unknown :-1 s=CLOSED pgs=1233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:51.440 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:51.434+0000 7fcd71eaf740 1 -- 172.21.15.39:0/2794718714 >> 172.21.15.39:0/2794718714 conn(0x560201ba5460 msgr2=0x560201f55e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:55:51.440 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:51.434+0000 7fcd71eaf740 1 -- 172.21.15.39:0/2794718714 shutdown_connections 2024-09-21T03:55:51.440 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:51.434+0000 7fcd71eaf740 1 -- 172.21.15.39:0/2794718714 wait complete. 2024-09-21T03:55:51.440 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:51.435+0000 7fcd71eaf740 1 -- start start 2024-09-21T03:55:51.440 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:51.435+0000 7fcd71eaf740 1 Processor -- start 2024-09-21T03:55:51.440 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:51.435+0000 7fcd71eaf740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x560201b881b0 0x560201f680c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:55:51.440 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:51.435+0000 7fcd71eaf740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560201b90390 0x560201f68600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:55:51.440 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:51.435+0000 7fcd71eaf740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560201bb1070 0x560201f68b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:55:51.440 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:51.435+0000 7fcd71eaf740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fcd641b0c10 con 0x560201b881b0 2024-09-21T03:55:51.440 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:51.435+0000 7fcd71eaf740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fcd641b0540 con 0x560201b90390 2024-09-21T03:55:51.440 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:51.435+0000 7fcd71eaf740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fcd641ad780 con 0x560201bb1070 2024-09-21T03:55:51.440 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:51.435+0000 7fcd71eaf740 10 client.0 _ll_register_callbacks cb 0x7fcd54005ef0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:55:51.440 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:51.435+0000 7fcd696d7640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x560201b881b0 0x560201f680c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:55:51.440 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:51.435+0000 7fcd696d7640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x560201b881b0 0x560201f680c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:51788/0 (socket says 172.21.15.39:51788) 2024-09-21T03:55:51.440 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:51.435+0000 7fcd696d7640 1 -- 172.21.15.39:0/1873678615 learned_addr learned my addr 172.21.15.39:0/1873678615 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:55:51.441 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:51.435+0000 7fcd69ed8640 1 --2- 172.21.15.39:0/1873678615 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560201bb1070 0x560201f68b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:55:51.441 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:51.435+0000 7fcd68ed6640 1 --2- 172.21.15.39:0/1873678615 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560201b90390 0x560201f68600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:55:51.441 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:51.436+0000 7fcd696d7640 1 -- 172.21.15.39:0/1873678615 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560201bb1070 msgr2=0x560201f68b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:55:51.441 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:51.436+0000 7fcd696d7640 1 --2- 172.21.15.39:0/1873678615 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560201bb1070 0x560201f68b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:51.441 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:51.436+0000 7fcd696d7640 1 -- 172.21.15.39:0/1873678615 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560201b90390 msgr2=0x560201f68600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:55:51.441 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:51.436+0000 7fcd696d7640 1 --2- 172.21.15.39:0/1873678615 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560201b90390 0x560201f68600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:51.441 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:51.436+0000 7fcd696d7640 1 -- 172.21.15.39:0/1873678615 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560201fdfa40 con 0x560201b881b0 2024-09-21T03:55:51.441 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:51.436+0000 7fcd696d7640 1 --2- 172.21.15.39:0/1873678615 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x560201b881b0 0x560201f680c0 secure :-1 s=READY pgs=1234 cs=0 l=1 rev1=1 crypto rx=0x7fcd6000bd90 tx=0x7fcd6000bdc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe94ba8932761f85 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:55:51.441 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:51.436+0000 7fcd4cff9640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-09-21T03:55:51.441 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:51.436+0000 7fcd4cff9640 1 -- 172.21.15.39:0/1873678615 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd6003e060 con 0x560201b881b0 2024-09-21T03:55:51.441 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:51.436+0000 7fcd4cff9640 1 -- 172.21.15.39:0/1873678615 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8528+0+0 (secure 0 0 0) 0x7fcd60005000 con 0x560201b881b0 2024-09-21T03:55:51.441 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:51.436+0000 7fcd71eaf740 1 -- 172.21.15.39:0/1873678615 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcd600047a0 con 0x560201b881b0 2024-09-21T03:55:51.441 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:51.436+0000 7fcd4cff9640 1 -- 172.21.15.39:0/1873678615 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fcd60005200 con 0x560201b881b0 2024-09-21T03:55:51.441 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:51.436+0000 7fcd71eaf740 10 client.18006 fetch_fsmap learned FSMap version 29 2024-09-21T03:55:51.441 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:51.436+0000 7fcd71eaf740 1 -- 172.21.15.39:0/1873678615 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x560201fe0cb0 con 0x560201b881b0 2024-09-21T03:55:51.442 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:51.437+0000 7fcd4cff9640 1 -- 172.21.15.39:0/1873678615 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcd600047a0 con 0x560201b881b0 2024-09-21T03:55:51.442 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:51.437+0000 7fcd4cff9640 1 -- 172.21.15.39:0/1873678615 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fcd60019040 con 0x560201b881b0 2024-09-21T03:55:51.442 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:51.437+0000 7fcd71eaf740 10 client.18006 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T03:55:51.442 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:51.437+0000 7fcd71eaf740 10 client.18006 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T03:55:51.442 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:51.437+0000 7fcd71eaf740 20 client.18006 populate_metadata read hostname 'smithi039' 2024-09-21T03:55:51.442 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:51.437+0000 7fcd71eaf740 1 --2- 172.21.15.39:0/1873678615 >> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] conn(0x560201b44da0 0x560201fe3410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:55:51.442 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:51.437+0000 7fcd71eaf740 4 client.18006 mds_command: new command op to 24695 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-21T03:55:51.442 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:51.437+0000 7fcd71eaf740 1 -- 172.21.15.39:0/1873678615 --> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] -- command(tid 0: {"prefix": "scrub status"}) -- 0x560201ecc850 con 0x560201b44da0 2024-09-21T03:55:51.442 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:51.437+0000 7fcd68ed6640 1 --2- 172.21.15.39:0/1873678615 >> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] conn(0x560201b44da0 0x560201fe3410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:55:51.442 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:51.438+0000 7fcd4cff9640 10 client.18006 ms_handle_connect on v2:172.21.15.150:6838/2949386907 2024-09-21T03:55:51.442 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:51.438+0000 7fcd68ed6640 1 --2- 172.21.15.39:0/1873678615 >> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] conn(0x560201b44da0 0x560201fe3410 crc :-1 s=READY pgs=156 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=23f606944a7c4af7 server_cookie=f9b6430973010f86 in_seq=0 out_seq=0 2024-09-21T03:55:51.443 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:51.438+0000 7fcd4cff9640 1 -- 172.21.15.39:0/1873678615 <== mds.0 v2:172.21.15.150:6838/2949386907 1 ==== command_reply(tid 0: 0 ) ==== 8+0+5202 (crc 0 0 0) 0x560201ecc850 con 0x560201b44da0 2024-09-21T03:55:51.443 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:51.438+0000 7fcd4cff9640 10 client.18006 handle_command_reply: tid=0 2024-09-21T03:55:51.443 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:51.438+0000 7fcd71eaf740 1 client.18006 shutdown 2024-09-21T03:55:51.443 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:51.438+0000 7fcd71eaf740 1 -- 172.21.15.39:0/1873678615 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x560201b881b0 msgr2=0x560201f680c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:55:51.444 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:51.438+0000 7fcd71eaf740 1 --2- 172.21.15.39:0/1873678615 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x560201b881b0 0x560201f680c0 secure :-1 s=READY pgs=1234 cs=0 l=1 rev1=1 crypto rx=0x7fcd6000bd90 tx=0x7fcd6000bdc0 comp rx=0 tx=0).stop 2024-09-21T03:55:51.444 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:51.438+0000 7fcd71eaf740 1 -- 172.21.15.39:0/1873678615 shutdown_connections 2024-09-21T03:55:51.444 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:51.438+0000 7fcd71eaf740 1 --2- 172.21.15.39:0/1873678615 >> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] conn(0x560201b44da0 0x560201fe3410 crc :-1 s=READY pgs=156 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:51.444 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:51.438+0000 7fcd71eaf740 1 --2- 172.21.15.39:0/1873678615 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560201bb1070 0x560201f68b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:51.444 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:51.438+0000 7fcd71eaf740 1 --2- 172.21.15.39:0/1873678615 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560201b90390 0x560201f68600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:51.444 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:51.438+0000 7fcd4cff9640 0 client.18006 ms_handle_reset on v2:172.21.15.150:6838/2949386907 2024-09-21T03:55:51.444 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:51.438+0000 7fcd71eaf740 1 --2- 172.21.15.39:0/1873678615 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x560201b881b0 0x560201f680c0 unknown :-1 s=CLOSED pgs=1234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:51.444 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:51.438+0000 7fcd71eaf740 1 -- 172.21.15.39:0/1873678615 >> 172.21.15.39:0/1873678615 conn(0x560201ba5460 msgr2=0x560201f5b750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:55:51.444 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:51.438+0000 7fcd71eaf740 1 -- 172.21.15.39:0/1873678615 shutdown_connections 2024-09-21T03:55:51.444 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:51.438+0000 7fcd71eaf740 1 -- 172.21.15.39:0/1873678615 wait complete. 2024-09-21T03:55:51.444 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:51.439+0000 7fcd71eaf740 20 client.18006 trim_cache size 0 max 16384 2024-09-21T03:55:51.444 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-09-21T03:55:51.445 INFO:teuthology.orchestra.run.smithi039.stdout: "status": "no active scrubs running", 2024-09-21T03:55:51.445 INFO:teuthology.orchestra.run.smithi039.stdout: "scrubs": {}, 2024-09-21T03:55:51.445 INFO:teuthology.orchestra.run.smithi039.stdout: "scrub_stats": { 2024-09-21T03:55:51.445 INFO:teuthology.orchestra.run.smithi039.stdout: "02e16ae1-0bac-41e9-b95e-fc6b17d207ac": { 2024-09-21T03:55:51.445 INFO:teuthology.orchestra.run.smithi039.stdout: "start_time": "2024-09-21T03:54:31.853401+0000", 2024-09-21T03:55:51.445 INFO:teuthology.orchestra.run.smithi039.stdout: "path": "/", 2024-09-21T03:55:51.445 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_started": 0, 2024-09-21T03:55:51.445 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_passed": 0, 2024-09-21T03:55:51.445 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_failed": 0, 2024-09-21T03:55:51.445 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_skipped": 0 2024-09-21T03:55:51.445 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:55:51.445 INFO:teuthology.orchestra.run.smithi039.stdout: "e28dcd13-f2ac-4cb7-a346-58917d542b15": { 2024-09-21T03:55:51.445 INFO:teuthology.orchestra.run.smithi039.stdout: "start_time": "2024-09-21T03:54:18.221680+0000", 2024-09-21T03:55:51.445 INFO:teuthology.orchestra.run.smithi039.stdout: "path": "/", 2024-09-21T03:55:51.445 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_started": 0, 2024-09-21T03:55:51.446 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_passed": 0, 2024-09-21T03:55:51.446 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_failed": 0, 2024-09-21T03:55:51.446 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_skipped": 0 2024-09-21T03:55:51.446 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:55:51.446 INFO:teuthology.orchestra.run.smithi039.stdout: "ad87da79-0248-4e18-9694-8228cb95422f": { 2024-09-21T03:55:51.446 INFO:teuthology.orchestra.run.smithi039.stdout: "start_time": "2024-09-21T03:53:18.441907+0000", 2024-09-21T03:55:51.446 INFO:teuthology.orchestra.run.smithi039.stdout: "path": "/", 2024-09-21T03:55:51.446 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_started": 0, 2024-09-21T03:55:51.446 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_passed": 0, 2024-09-21T03:55:51.446 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_failed": 0, 2024-09-21T03:55:51.446 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_skipped": 0 2024-09-21T03:55:51.446 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:55:51.446 INFO:teuthology.orchestra.run.smithi039.stdout: "02509e4f-3911-4983-b16f-df8ded0b9998": { 2024-09-21T03:55:51.446 INFO:teuthology.orchestra.run.smithi039.stdout: "start_time": "2024-09-21T03:46:17.889576+0000", 2024-09-21T03:55:51.447 INFO:teuthology.orchestra.run.smithi039.stdout: "path": "/", 2024-09-21T03:55:51.447 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_started": 0, 2024-09-21T03:55:51.447 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_passed": 0, 2024-09-21T03:55:51.447 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_failed": 0, 2024-09-21T03:55:51.447 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_skipped": 0 2024-09-21T03:55:51.447 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:55:51.447 INFO:teuthology.orchestra.run.smithi039.stdout: "10728de8-c49d-4edc-be2b-639249255e94": { 2024-09-21T03:55:51.447 INFO:teuthology.orchestra.run.smithi039.stdout: "start_time": "2024-09-21T03:47:03.965623+0000", 2024-09-21T03:55:51.447 INFO:teuthology.orchestra.run.smithi039.stdout: "path": "/", 2024-09-21T03:55:51.447 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_started": 0, 2024-09-21T03:55:51.447 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_passed": 0, 2024-09-21T03:55:51.447 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_failed": 0, 2024-09-21T03:55:51.447 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_skipped": 0 2024-09-21T03:55:51.447 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:55:51.447 INFO:teuthology.orchestra.run.smithi039.stdout: "10e282c4-137c-4fb2-9af2-a83dc3945587": { 2024-09-21T03:55:51.448 INFO:teuthology.orchestra.run.smithi039.stdout: "start_time": "2024-09-21T03:55:17.955458+0000", 2024-09-21T03:55:51.448 INFO:teuthology.orchestra.run.smithi039.stdout: "path": "/", 2024-09-21T03:55:51.448 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_started": 0, 2024-09-21T03:55:51.448 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_passed": 0, 2024-09-21T03:55:51.448 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_failed": 0, 2024-09-21T03:55:51.448 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_skipped": 0 2024-09-21T03:55:51.448 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:55:51.448 INFO:teuthology.orchestra.run.smithi039.stdout: "9860c2f4-b7b1-47aa-865a-9a1c4434a5f6": { 2024-09-21T03:55:51.448 INFO:teuthology.orchestra.run.smithi039.stdout: "start_time": "2024-09-21T03:49:17.718867+0000", 2024-09-21T03:55:51.448 INFO:teuthology.orchestra.run.smithi039.stdout: "path": "/", 2024-09-21T03:55:51.448 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_started": 0, 2024-09-21T03:55:51.448 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_passed": 0, 2024-09-21T03:55:51.448 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_failed": 0, 2024-09-21T03:55:51.448 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_skipped": 0 2024-09-21T03:55:51.448 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:55:51.448 INFO:teuthology.orchestra.run.smithi039.stdout: "d9391d69-baac-453a-9d52-d046760c070d": { 2024-09-21T03:55:51.449 INFO:teuthology.orchestra.run.smithi039.stdout: "start_time": "2024-09-21T03:53:32.364622+0000", 2024-09-21T03:55:51.449 INFO:teuthology.orchestra.run.smithi039.stdout: "path": "/", 2024-09-21T03:55:51.449 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_started": 0, 2024-09-21T03:55:51.449 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_passed": 0, 2024-09-21T03:55:51.449 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_failed": 0, 2024-09-21T03:55:51.449 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_skipped": 0 2024-09-21T03:55:51.449 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:55:51.449 INFO:teuthology.orchestra.run.smithi039.stdout: "c7c700f3-2cd8-41ac-8381-b57dd1b37b02": { 2024-09-21T03:55:51.449 INFO:teuthology.orchestra.run.smithi039.stdout: "start_time": "2024-09-21T03:52:32.368858+0000", 2024-09-21T03:55:51.449 INFO:teuthology.orchestra.run.smithi039.stdout: "path": "/", 2024-09-21T03:55:51.449 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_started": 0, 2024-09-21T03:55:51.449 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_passed": 0, 2024-09-21T03:55:51.449 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_failed": 0, 2024-09-21T03:55:51.449 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_skipped": 0 2024-09-21T03:55:51.449 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:55:51.450 INFO:teuthology.orchestra.run.smithi039.stdout: "29411307-eb8d-4339-8d2b-47ebd32c0080": { 2024-09-21T03:55:51.450 INFO:teuthology.orchestra.run.smithi039.stdout: "start_time": "2024-09-21T03:47:38.092920+0000", 2024-09-21T03:55:51.450 INFO:teuthology.orchestra.run.smithi039.stdout: "path": "/", 2024-09-21T03:55:51.450 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_started": 0, 2024-09-21T03:55:51.450 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_passed": 0, 2024-09-21T03:55:51.450 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_failed": 0, 2024-09-21T03:55:51.450 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_skipped": 0 2024-09-21T03:55:51.450 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:55:51.450 INFO:teuthology.orchestra.run.smithi039.stdout: "ede73e6e-3fd0-4a7f-b5bb-f68946f57eea": { 2024-09-21T03:55:51.450 INFO:teuthology.orchestra.run.smithi039.stdout: "start_time": "2024-09-21T03:51:32.235096+0000", 2024-09-21T03:55:51.450 INFO:teuthology.orchestra.run.smithi039.stdout: "path": "/", 2024-09-21T03:55:51.450 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_started": 0, 2024-09-21T03:55:51.450 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_passed": 0, 2024-09-21T03:55:51.450 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_failed": 0, 2024-09-21T03:55:51.451 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_skipped": 0 2024-09-21T03:55:51.451 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:55:51.451 INFO:teuthology.orchestra.run.smithi039.stdout: "e28ad743-2b22-45b3-8799-a638c2883623": { 2024-09-21T03:55:51.451 INFO:teuthology.orchestra.run.smithi039.stdout: "start_time": "2024-09-21T03:48:31.448821+0000", 2024-09-21T03:55:51.451 INFO:teuthology.orchestra.run.smithi039.stdout: "path": "/", 2024-09-21T03:55:51.451 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_started": 0, 2024-09-21T03:55:51.451 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_passed": 0, 2024-09-21T03:55:51.451 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_failed": 0, 2024-09-21T03:55:51.451 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_skipped": 0 2024-09-21T03:55:51.451 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:55:51.451 INFO:teuthology.orchestra.run.smithi039.stdout: "6e760f1f-9048-4f92-bd66-d708fbef75b3": { 2024-09-21T03:55:51.451 INFO:teuthology.orchestra.run.smithi039.stdout: "start_time": "2024-09-21T03:50:03.711918+0000", 2024-09-21T03:55:51.451 INFO:teuthology.orchestra.run.smithi039.stdout: "path": "/", 2024-09-21T03:55:51.451 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_started": 0, 2024-09-21T03:55:51.451 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_passed": 0, 2024-09-21T03:55:51.451 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_failed": 0, 2024-09-21T03:55:51.452 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_skipped": 0 2024-09-21T03:55:51.452 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:55:51.452 INFO:teuthology.orchestra.run.smithi039.stdout: "c7e2b8e4-057d-435f-b2bd-b633de98035f": { 2024-09-21T03:55:51.452 INFO:teuthology.orchestra.run.smithi039.stdout: "start_time": "2024-09-21T03:47:19.400305+0000", 2024-09-21T03:55:51.452 INFO:teuthology.orchestra.run.smithi039.stdout: "path": "/", 2024-09-21T03:55:51.452 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_started": 0, 2024-09-21T03:55:51.452 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_passed": 0, 2024-09-21T03:55:51.452 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_failed": 0, 2024-09-21T03:55:51.452 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_skipped": 0 2024-09-21T03:55:51.452 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:55:51.452 INFO:teuthology.orchestra.run.smithi039.stdout: "74ab917e-ecea-4778-a1c3-c184214fb24c": { 2024-09-21T03:55:51.452 INFO:teuthology.orchestra.run.smithi039.stdout: "start_time": "2024-09-21T03:50:17.726630+0000", 2024-09-21T03:55:51.452 INFO:teuthology.orchestra.run.smithi039.stdout: "path": "/", 2024-09-21T03:55:51.452 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_started": 0, 2024-09-21T03:55:51.452 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_passed": 0, 2024-09-21T03:55:51.452 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_failed": 0, 2024-09-21T03:55:51.453 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_skipped": 0 2024-09-21T03:55:51.453 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:55:51.453 INFO:teuthology.orchestra.run.smithi039.stdout: "160750d8-401f-4810-866d-e63e7e4bc3e0": { 2024-09-21T03:55:51.453 INFO:teuthology.orchestra.run.smithi039.stdout: "start_time": "2024-09-21T03:51:04.208673+0000", 2024-09-21T03:55:51.453 INFO:teuthology.orchestra.run.smithi039.stdout: "path": "/", 2024-09-21T03:55:51.453 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_started": 0, 2024-09-21T03:55:51.453 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_passed": 0, 2024-09-21T03:55:51.453 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_failed": 0, 2024-09-21T03:55:51.453 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_skipped": 0 2024-09-21T03:55:51.453 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:55:51.453 INFO:teuthology.orchestra.run.smithi039.stdout: "ae1fc4cf-5458-47d0-afcb-282b136dba15": { 2024-09-21T03:55:51.453 INFO:teuthology.orchestra.run.smithi039.stdout: "start_time": "2024-09-21T03:51:18.560378+0000", 2024-09-21T03:55:51.453 INFO:teuthology.orchestra.run.smithi039.stdout: "path": "/", 2024-09-21T03:55:51.453 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_started": 0, 2024-09-21T03:55:51.453 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_passed": 0, 2024-09-21T03:55:51.453 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_failed": 0, 2024-09-21T03:55:51.454 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_skipped": 0 2024-09-21T03:55:51.454 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:55:51.454 INFO:teuthology.orchestra.run.smithi039.stdout: "4bda6ee3-7237-4ee4-96ce-150ad50f26dc": { 2024-09-21T03:55:51.454 INFO:teuthology.orchestra.run.smithi039.stdout: "start_time": "2024-09-21T03:52:18.622141+0000", 2024-09-21T03:55:51.454 INFO:teuthology.orchestra.run.smithi039.stdout: "path": "/", 2024-09-21T03:55:51.454 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_started": 0, 2024-09-21T03:55:51.454 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_passed": 0, 2024-09-21T03:55:51.454 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_failed": 0, 2024-09-21T03:55:51.454 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_skipped": 0 2024-09-21T03:55:51.454 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-21T03:55:51.454 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-21T03:55:51.454 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-09-21T03:55:51.455 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:51.439+0000 7fcd6b93b640 1 -- 172.21.15.39:0/241347092 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7fcd500b6680 msgr2=0x7fcd500b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:55:51.455 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:51.439+0000 7fcd6b93b640 1 --2- 172.21.15.39:0/241347092 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7fcd500b6680 0x7fcd500b8b90 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x560201f572d0 tx=0x7fcd6000b390 comp rx=0 tx=0).stop 2024-09-21T03:55:51.455 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:51.439+0000 7fcd6b93b640 1 -- 172.21.15.39:0/241347092 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcd641094a0 msgr2=0x7fcd6419ab00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:55:51.455 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:51.439+0000 7fcd6b93b640 1 --2- 172.21.15.39:0/241347092 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcd641094a0 0x7fcd6419ab00 secure :-1 s=READY pgs=1230 cs=0 l=1 rev1=1 crypto rx=0x7fcd5402a930 tx=0x7fcd54004270 comp rx=0 tx=0).stop 2024-09-21T03:55:51.455 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:51.439+0000 7fcd6b93b640 1 -- 172.21.15.39:0/241347092 shutdown_connections 2024-09-21T03:55:51.455 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:51.439+0000 7fcd6b93b640 1 --2- 172.21.15.39:0/241347092 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7fcd500b6680 0x7fcd500b8b90 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:51.455 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:51.439+0000 7fcd6b93b640 1 --2- 172.21.15.39:0/241347092 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcd64109df0 0x7fcd6419fda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:51.456 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:51.439+0000 7fcd6b93b640 1 --2- 172.21.15.39:0/241347092 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcd641094a0 0x7fcd6419ab00 unknown :-1 s=CLOSED pgs=1230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:51.456 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:51.439+0000 7fcd6b93b640 1 --2- 172.21.15.39:0/241347092 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcd64103b60 0x7fcd6419a560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:51.456 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:51.439+0000 7fcd6b93b640 1 -- 172.21.15.39:0/241347092 >> 172.21.15.39:0/241347092 conn(0x7fcd6407b850 msgr2=0x7fcd64105560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:55:51.456 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:51.439+0000 7fcd6b93b640 1 -- 172.21.15.39:0/241347092 shutdown_connections 2024-09-21T03:55:51.456 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:51.439+0000 7fcd6b93b640 1 -- 172.21.15.39:0/241347092 wait complete. 2024-09-21T03:55:51.467 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:55:51 smithi161 ceph-mon[34533]: pgmap v625: 97 pgs: 97 active+clean; 10 MiB data, 525 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-09-21T03:55:51.474 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:55:51 smithi150 ceph-mon[34289]: pgmap v625: 97 pgs: 97 active+clean; 10 MiB data, 525 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-09-21T03:55:52.329 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-21T03:55:52.330 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 51116d5c-77ca-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T03:55:52.593 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/51116d5c-77ca-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:55:53.066 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:53.057+0000 7f8a9e370640 1 -- 172.21.15.39:0/2553084391 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8a98102ae0 msgr2=0x7f8a98102ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:55:53.066 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:53.057+0000 7f8a9e370640 1 --2- 172.21.15.39:0/2553084391 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8a98102ae0 0x7f8a98102ef0 secure :-1 s=READY pgs=1235 cs=0 l=1 rev1=1 crypto rx=0x7f8a88009900 tx=0x7f8a8802a410 comp rx=0 tx=0).stop 2024-09-21T03:55:53.067 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:53.057+0000 7f8a9e370640 1 -- 172.21.15.39:0/2553084391 shutdown_connections 2024-09-21T03:55:53.067 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:53.057+0000 7f8a9e370640 1 --2- 172.21.15.39:0/2553084391 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8a98108d20 0x7f8a9810f550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:53.067 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:53.057+0000 7f8a9e370640 1 --2- 172.21.15.39:0/2553084391 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8a981083d0 0x7f8a981087e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:53.067 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:53.057+0000 7f8a9e370640 1 --2- 172.21.15.39:0/2553084391 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8a98102ae0 0x7f8a98102ef0 unknown :-1 s=CLOSED pgs=1235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:53.067 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:53.057+0000 7f8a9e370640 1 -- 172.21.15.39:0/2553084391 >> 172.21.15.39:0/2553084391 conn(0x7f8a98079760 msgr2=0x7f8a98079b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:55:53.067 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:53.057+0000 7f8a9e370640 1 -- 172.21.15.39:0/2553084391 shutdown_connections 2024-09-21T03:55:53.067 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:53.057+0000 7f8a9e370640 1 -- 172.21.15.39:0/2553084391 wait complete. 2024-09-21T03:55:53.067 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:53.058+0000 7f8a9e370640 1 Processor -- start 2024-09-21T03:55:53.067 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:53.058+0000 7f8a9e370640 1 -- start start 2024-09-21T03:55:53.067 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:53.058+0000 7f8a9e370640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8a98102ae0 0x7f8a98199360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:55:53.067 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:53.058+0000 7f8a9e370640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8a981083d0 0x7f8a98199900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:55:53.067 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:53.058+0000 7f8a9e370640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8a98108d20 0x7f8a9819eba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:55:53.067 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:53.058+0000 7f8a9e370640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f8a98112990 con 0x7f8a98108d20 2024-09-21T03:55:53.068 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:53.058+0000 7f8a9e370640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f8a98112810 con 0x7f8a98102ae0 2024-09-21T03:55:53.068 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:53.058+0000 7f8a9e370640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f8a98112b10 con 0x7f8a981083d0 2024-09-21T03:55:53.068 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:53.058+0000 7f8a9c90d640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8a98108d20 0x7f8a9819eba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:55:53.068 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:53.058+0000 7f8a9c90d640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8a98108d20 0x7f8a9819eba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:51806/0 (socket says 172.21.15.39:51806) 2024-09-21T03:55:53.068 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:53.058+0000 7f8a9c90d640 1 -- 172.21.15.39:0/3405585277 learned_addr learned my addr 172.21.15.39:0/3405585277 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:55:53.068 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:53.058+0000 7f8a8ffff640 1 --2- 172.21.15.39:0/3405585277 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8a981083d0 0x7f8a98199900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:55:53.068 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:53.058+0000 7f8a97fff640 1 --2- 172.21.15.39:0/3405585277 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8a98102ae0 0x7f8a98199360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:55:53.068 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:53.058+0000 7f8a9c90d640 1 -- 172.21.15.39:0/3405585277 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8a981083d0 msgr2=0x7f8a98199900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:55:53.068 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:53.058+0000 7f8a9c90d640 1 --2- 172.21.15.39:0/3405585277 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8a981083d0 0x7f8a98199900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:53.068 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:53.058+0000 7f8a9c90d640 1 -- 172.21.15.39:0/3405585277 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8a98102ae0 msgr2=0x7f8a98199360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:55:53.068 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:53.058+0000 7f8a9c90d640 1 --2- 172.21.15.39:0/3405585277 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8a98102ae0 0x7f8a98199360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:53.068 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:53.058+0000 7f8a9c90d640 1 -- 172.21.15.39:0/3405585277 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8a9819f210 con 0x7f8a98108d20 2024-09-21T03:55:53.068 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:53.059+0000 7f8a9c90d640 1 --2- 172.21.15.39:0/3405585277 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8a98108d20 0x7f8a9819eba0 secure :-1 s=READY pgs=1236 cs=0 l=1 rev1=1 crypto rx=0x7f8a9000ebb0 tx=0x7f8a90009f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:55:53.068 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:53.059+0000 7f8a95ffb640 1 -- 172.21.15.39:0/3405585277 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a9000e2e0 con 0x7f8a98108d20 2024-09-21T03:55:53.069 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:53.059+0000 7f8a95ffb640 1 -- 172.21.15.39:0/3405585277 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8a90007950 con 0x7f8a98108d20 2024-09-21T03:55:53.069 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:53.059+0000 7f8a9e370640 1 -- 172.21.15.39:0/3405585277 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8a9819f4a0 con 0x7f8a98108d20 2024-09-21T03:55:53.069 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:53.059+0000 7f8a95ffb640 1 -- 172.21.15.39:0/3405585277 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a900056f0 con 0x7f8a98108d20 2024-09-21T03:55:53.069 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:53.059+0000 7f8a9e370640 1 -- 172.21.15.39:0/3405585277 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8a9819f990 con 0x7f8a98108d20 2024-09-21T03:55:53.069 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:53.060+0000 7f8a9e370640 1 -- 172.21.15.39:0/3405585277 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8a98103d10 con 0x7f8a98108d20 2024-09-21T03:55:53.069 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:53.062+0000 7f8a95ffb640 1 -- 172.21.15.39:0/3405585277 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 27) ==== 154643+0+0 (secure 0 0 0) 0x7f8a90005890 con 0x7f8a98108d20 2024-09-21T03:55:53.069 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:53.062+0000 7f8a95ffb640 1 --2- 172.21.15.39:0/3405585277 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f8a840b66b0 0x7f8a840b8bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:55:53.069 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:53.062+0000 7f8a97fff640 1 --2- 172.21.15.39:0/3405585277 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f8a840b66b0 0x7f8a840b8bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:55:53.069 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:53.063+0000 7f8a95ffb640 1 -- 172.21.15.39:0/3405585277 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8528+0+0 (secure 0 0 0) 0x7f8a900e90b0 con 0x7f8a98108d20 2024-09-21T03:55:53.069 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:53.063+0000 7f8a97fff640 1 --2- 172.21.15.39:0/3405585277 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f8a840b66b0 0x7f8a840b8bc0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f8a88031440 tx=0x7f8a88009560 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:55:53.070 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:53.065+0000 7f8a95ffb640 1 -- 172.21.15.39:0/3405585277 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f8a900af600 con 0x7f8a98108d20 2024-09-21T03:55:53.296 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:55:53 smithi039 ceph-mon[31211]: pgmap v626: 97 pgs: 97 active+clean; 10 MiB data, 525 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-21T03:55:53.297 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:53.294+0000 7f8a9e370640 1 -- 172.21.15.39:0/3405585277 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f8a98102ef0 con 0x7f8a98108d20 2024-09-21T03:55:53.297 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:53.295+0000 7f8a95ffb640 1 -- 172.21.15.39:0/3405585277 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11978 (secure 0 0 0) 0x7f8a900b39e0 con 0x7f8a98108d20 2024-09-21T03:55:53.298 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T03:55:53.298 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":29,"btime":"2024-09-21T03:46:08:982881+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14790,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6835/508995295","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":508995295},{"type":"v1","addr":"172.21.15.39:6835","nonce":508995295}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14802,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6837/3915912607","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":3915912607},{"type":"v1","addr":"172.21.15.39:6837","nonce":3915912607}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":29,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T03:44:15.009872+0000","modified":"2024-09-21T03:46:08.430113+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24695,"mds_1":24667,"mds_2":24659,"mds_3":24646,"mds_4":14868},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14856":{"gid":14856,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.39:6839/185509192","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6838","nonce":185509192},{"type":"v1","addr":"172.21.15.39:6839","nonce":185509192}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14868":{"gid":14868,"name":"a","rank":4,"incarnation":25,"state":"up:active","state_seq":7,"addr":"172.21.15.39:6841/4247190456","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6840","nonce":4247190456},{"type":"v1","addr":"172.21.15.39:6841","nonce":4247190456}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24599":{"gid":24599,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.150:6833/3114643217","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":3114643217},{"type":"v1","addr":"172.21.15.150:6833","nonce":3114643217}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24601":{"gid":24601,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.161:6833/2963990654","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":2963990654},{"type":"v1","addr":"172.21.15.161:6833","nonce":2963990654}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24632":{"gid":24632,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.150:6835/724594008","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6834","nonce":724594008},{"type":"v1","addr":"172.21.15.150:6835","nonce":724594008}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24640":{"gid":24640,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.161:6835/3747882110","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":3747882110},{"type":"v1","addr":"172.21.15.161:6835","nonce":3747882110}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24646":{"gid":24646,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":10,"addr":"172.21.15.161:6837/3451544597","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6836","nonce":3451544597},{"type":"v1","addr":"172.21.15.161:6837","nonce":3451544597}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24659":{"gid":24659,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.150:6837/4017043144","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6836","nonce":4017043144},{"type":"v1","addr":"172.21.15.150:6837","nonce":4017043144}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24667":{"gid":24667,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.161:6839/2190107270","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6838","nonce":2190107270},{"type":"v1","addr":"172.21.15.161:6839","nonce":2190107270}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24695":{"gid":24695,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.150:6839/2949386907","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6838","nonce":2949386907},{"type":"v1","addr":"172.21.15.150:6839","nonce":2949386907}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24695,"qdb_cluster":[24695,24667,24659,24646,14868]},"id":1}]} 2024-09-21T03:55:53.298 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 29 2024-09-21T03:55:53.300 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:53.298+0000 7f8a9e370640 1 -- 172.21.15.39:0/3405585277 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f8a840b66b0 msgr2=0x7f8a840b8bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:55:53.300 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:53.298+0000 7f8a9e370640 1 --2- 172.21.15.39:0/3405585277 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f8a840b66b0 0x7f8a840b8bc0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f8a88031440 tx=0x7f8a88009560 comp rx=0 tx=0).stop 2024-09-21T03:55:53.300 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:53.298+0000 7f8a9e370640 1 -- 172.21.15.39:0/3405585277 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8a98108d20 msgr2=0x7f8a9819eba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:55:53.300 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:53.298+0000 7f8a9e370640 1 --2- 172.21.15.39:0/3405585277 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8a98108d20 0x7f8a9819eba0 secure :-1 s=READY pgs=1236 cs=0 l=1 rev1=1 crypto rx=0x7f8a9000ebb0 tx=0x7f8a90009f60 comp rx=0 tx=0).stop 2024-09-21T03:55:53.300 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:53.298+0000 7f8a9e370640 1 -- 172.21.15.39:0/3405585277 shutdown_connections 2024-09-21T03:55:53.300 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:53.298+0000 7f8a9e370640 1 --2- 172.21.15.39:0/3405585277 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f8a840b66b0 0x7f8a840b8bc0 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:53.300 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:53.298+0000 7f8a9e370640 1 --2- 172.21.15.39:0/3405585277 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8a98108d20 0x7f8a9819eba0 unknown :-1 s=CLOSED pgs=1236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:53.300 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:53.298+0000 7f8a9e370640 1 --2- 172.21.15.39:0/3405585277 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8a981083d0 0x7f8a98199900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:53.300 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:53.298+0000 7f8a9e370640 1 --2- 172.21.15.39:0/3405585277 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8a98102ae0 0x7f8a98199360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:53.301 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:53.298+0000 7f8a9e370640 1 -- 172.21.15.39:0/3405585277 >> 172.21.15.39:0/3405585277 conn(0x7f8a98079760 msgr2=0x7f8a98101410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:55:53.301 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:53.298+0000 7f8a9e370640 1 -- 172.21.15.39:0/3405585277 shutdown_connections 2024-09-21T03:55:53.301 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:53.298+0000 7f8a9e370640 1 -- 172.21.15.39:0/3405585277 wait complete. 2024-09-21T03:55:53.468 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:55:53 smithi161 ceph-mon[34533]: pgmap v626: 97 pgs: 97 active+clean; 10 MiB data, 525 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-21T03:55:53.475 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:55:53 smithi150 ceph-mon[34289]: pgmap v626: 97 pgs: 97 active+clean; 10 MiB data, 525 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-21T03:55:54.029 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 51116d5c-77ca-11ef-baf5-efdc52797490 -- ceph tell mds.1:4 damage ls 2024-09-21T03:55:54.289 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/51116d5c-77ca-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:55:54.333 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:55:54 smithi039 ceph-mon[31211]: from='client.? 172.21.15.39:0/3405585277' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:55:54.468 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:55:54 smithi161 ceph-mon[34533]: from='client.? 172.21.15.39:0/3405585277' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:55:54.474 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:55:54 smithi150 ceph-mon[34289]: from='client.? 172.21.15.39:0/3405585277' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:55:54.854 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:54.850+0000 7fac33ebd640 1 -- 172.21.15.39:0/3245218284 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fac2c109df0 msgr2=0x7fac2c110620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:55:54.854 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:54.850+0000 7fac33ebd640 1 --2- 172.21.15.39:0/3245218284 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fac2c109df0 0x7fac2c110620 secure :-1 s=READY pgs=1237 cs=0 l=1 rev1=1 crypto rx=0x7fac24009ca0 tx=0x7fac2402a8c0 comp rx=0 tx=0).stop 2024-09-21T03:55:54.854 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:54.850+0000 7fac33ebd640 1 -- 172.21.15.39:0/3245218284 shutdown_connections 2024-09-21T03:55:54.854 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:54.850+0000 7fac33ebd640 1 --2- 172.21.15.39:0/3245218284 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fac2c109df0 0x7fac2c110620 unknown :-1 s=CLOSED pgs=1237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:54.854 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:54.850+0000 7fac33ebd640 1 --2- 172.21.15.39:0/3245218284 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fac2c1094a0 0x7fac2c1098b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:54.855 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:54.850+0000 7fac33ebd640 1 --2- 172.21.15.39:0/3245218284 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fac2c103b60 0x7fac2c103f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:54.855 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:54.850+0000 7fac33ebd640 1 -- 172.21.15.39:0/3245218284 >> 172.21.15.39:0/3245218284 conn(0x7fac2c07b850 msgr2=0x7fac2c07bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:55:54.855 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:54.850+0000 7fac33ebd640 1 -- 172.21.15.39:0/3245218284 shutdown_connections 2024-09-21T03:55:54.855 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:54.850+0000 7fac33ebd640 1 -- 172.21.15.39:0/3245218284 wait complete. 2024-09-21T03:55:54.855 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:54.851+0000 7fac33ebd640 1 Processor -- start 2024-09-21T03:55:54.855 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:54.851+0000 7fac33ebd640 1 -- start start 2024-09-21T03:55:54.855 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:54.851+0000 7fac33ebd640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fac2c103b60 0x7fac2c19a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:55:54.855 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:54.851+0000 7fac33ebd640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fac2c1094a0 0x7fac2c19aa20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:55:54.855 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:54.851+0000 7fac33ebd640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fac2c109df0 0x7fac2c19fcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:55:54.855 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:54.851+0000 7fac33ebd640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fac2c113ab0 con 0x7fac2c103b60 2024-09-21T03:55:54.855 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:54.851+0000 7fac33ebd640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fac2c113930 con 0x7fac2c1094a0 2024-09-21T03:55:54.855 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:54.851+0000 7fac33ebd640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fac2c113c30 con 0x7fac2c109df0 2024-09-21T03:55:54.855 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:54.851+0000 7fac31c59640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fac2c103b60 0x7fac2c19a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:55:54.856 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:54.851+0000 7fac31c59640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fac2c103b60 0x7fac2c19a480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:51820/0 (socket says 172.21.15.39:51820) 2024-09-21T03:55:54.856 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:54.851+0000 7fac31c59640 1 -- 172.21.15.39:0/3834966824 learned_addr learned my addr 172.21.15.39:0/3834966824 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:55:54.856 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:54.851+0000 7fac3245a640 1 --2- 172.21.15.39:0/3834966824 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fac2c109df0 0x7fac2c19fcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:55:54.856 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:54.851+0000 7fac31458640 1 --2- 172.21.15.39:0/3834966824 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fac2c1094a0 0x7fac2c19aa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:55:54.856 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:54.851+0000 7fac31c59640 1 -- 172.21.15.39:0/3834966824 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fac2c109df0 msgr2=0x7fac2c19fcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:55:54.856 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:54.851+0000 7fac31c59640 1 --2- 172.21.15.39:0/3834966824 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fac2c109df0 0x7fac2c19fcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:54.856 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:54.851+0000 7fac31c59640 1 -- 172.21.15.39:0/3834966824 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fac2c1094a0 msgr2=0x7fac2c19aa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:55:54.856 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:54.851+0000 7fac31c59640 1 --2- 172.21.15.39:0/3834966824 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fac2c1094a0 0x7fac2c19aa20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:54.856 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:54.851+0000 7fac31c59640 1 -- 172.21.15.39:0/3834966824 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fac2c1a03f0 con 0x7fac2c103b60 2024-09-21T03:55:54.856 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:54.852+0000 7fac31458640 1 --2- 172.21.15.39:0/3834966824 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fac2c1094a0 0x7fac2c19aa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:55:54.856 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:54.852+0000 7fac31c59640 1 --2- 172.21.15.39:0/3834966824 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fac2c103b60 0x7fac2c19a480 secure :-1 s=READY pgs=1238 cs=0 l=1 rev1=1 crypto rx=0x7fac2800e9f0 tx=0x7fac2800edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:55:54.856 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:54.852+0000 7fac22ffd640 1 -- 172.21.15.39:0/3834966824 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac2800cd80 con 0x7fac2c103b60 2024-09-21T03:55:54.856 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:54.852+0000 7fac33ebd640 1 -- 172.21.15.39:0/3834966824 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fac2c1a06e0 con 0x7fac2c103b60 2024-09-21T03:55:54.856 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:54.852+0000 7fac22ffd640 1 -- 172.21.15.39:0/3834966824 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fac280054d0 con 0x7fac2c103b60 2024-09-21T03:55:54.857 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:54.852+0000 7fac22ffd640 1 -- 172.21.15.39:0/3834966824 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac28010690 con 0x7fac2c103b60 2024-09-21T03:55:54.857 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:54.852+0000 7fac33ebd640 1 -- 172.21.15.39:0/3834966824 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fac2c1a6e40 con 0x7fac2c103b60 2024-09-21T03:55:54.858 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:54.855+0000 7fac22ffd640 1 -- 172.21.15.39:0/3834966824 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 27) ==== 154643+0+0 (secure 0 0 0) 0x7fac28004e90 con 0x7fac2c103b60 2024-09-21T03:55:54.858 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:54.855+0000 7fac3a431740 1 Processor -- start 2024-09-21T03:55:54.858 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:54.855+0000 7fac3a431740 1 -- start start 2024-09-21T03:55:54.858 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:54.855+0000 7fac3a431740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564624b43070 0x564624eea2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:55:54.859 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:54.855+0000 7fac3a431740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564624b1a1b0 0x564624eea800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:55:54.859 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:54.855+0000 7fac3a431740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564624b22390 0x564624eead40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:55:54.859 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:54.855+0000 7fac3a431740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x564624e60ea0 con 0x564624b22390 2024-09-21T03:55:54.859 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:54.855+0000 7fac3a431740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x564624e502d0 con 0x564624b43070 2024-09-21T03:55:54.859 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:54.855+0000 7fac3a431740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x564624e540e0 con 0x564624b1a1b0 2024-09-21T03:55:54.859 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:54.855+0000 7fac31458640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564624b22390 0x564624eead40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:55:54.859 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:54.856+0000 7fac31458640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564624b22390 0x564624eead40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:51822/0 (socket says 172.21.15.39:51822) 2024-09-21T03:55:54.859 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:54.856+0000 7fac31458640 1 -- 172.21.15.39:0/327879805 learned_addr learned my addr 172.21.15.39:0/327879805 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:55:54.859 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:54.856+0000 7fac31c59640 1 --2- 172.21.15.39:0/327879805 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564624b1a1b0 0x564624eea800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:55:54.859 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:54.856+0000 7fac3245a640 1 --2- 172.21.15.39:0/327879805 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564624b43070 0x564624eea2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:55:54.859 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:54.856+0000 7fac22ffd640 1 --2- 172.21.15.39:0/3834966824 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7fac180b6680 0x7fac180b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:55:54.859 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:54.856+0000 7fac31458640 1 -- 172.21.15.39:0/327879805 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564624b1a1b0 msgr2=0x564624eea800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:55:54.860 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:54.856+0000 7fac31458640 1 --2- 172.21.15.39:0/327879805 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564624b1a1b0 0x564624eea800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:54.860 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:54.856+0000 7fac31458640 1 -- 172.21.15.39:0/327879805 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564624b43070 msgr2=0x564624eea2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:55:54.860 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:54.856+0000 7fac31458640 1 --2- 172.21.15.39:0/327879805 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564624b43070 0x564624eea2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:54.860 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:54.856+0000 7fac31458640 1 -- 172.21.15.39:0/327879805 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564624bae650 con 0x564624b22390 2024-09-21T03:55:54.860 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:54.856+0000 7fac3245a640 1 --2- 172.21.15.39:0/3834966824 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7fac180b6680 0x7fac180b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:55:54.860 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:54.856+0000 7fac22ffd640 1 -- 172.21.15.39:0/3834966824 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8528+0+0 (secure 0 0 0) 0x7fac280e7ba0 con 0x7fac2c103b60 2024-09-21T03:55:54.860 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:54.856+0000 7fac31458640 1 --2- 172.21.15.39:0/327879805 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564624b22390 0x564624eead40 secure :-1 s=READY pgs=1239 cs=0 l=1 rev1=1 crypto rx=0x7fac1c00bc30 tx=0x7fac1c00f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e1b69eb3b22d00 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:55:54.860 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:54.856+0000 7fac17fff640 1 -- 172.21.15.39:0/327879805 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac1c004030 con 0x564624b22390 2024-09-21T03:55:54.860 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:54.856+0000 7fac17fff640 1 -- 172.21.15.39:0/327879805 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fac1c0054d0 con 0x564624b22390 2024-09-21T03:55:54.860 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:54.856+0000 7fac17fff640 1 -- 172.21.15.39:0/327879805 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac1c004b40 con 0x564624b22390 2024-09-21T03:55:54.864 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:54.857+0000 7fac3a431740 1 -- 172.21.15.39:0/327879805 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564624b22390 msgr2=0x564624eead40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:55:54.864 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:54.857+0000 7fac3245a640 1 --2- 172.21.15.39:0/3834966824 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7fac180b6680 0x7fac180b8b90 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7fac24006010 tx=0x7fac2403f040 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:55:54.864 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:54.857+0000 7fac3a431740 1 --2- 172.21.15.39:0/327879805 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564624b22390 0x564624eead40 secure :-1 s=READY pgs=1239 cs=0 l=1 rev1=1 crypto rx=0x7fac1c00bc30 tx=0x7fac1c00f730 comp rx=0 tx=0).stop 2024-09-21T03:55:54.864 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:54.857+0000 7fac3a431740 1 -- 172.21.15.39:0/327879805 shutdown_connections 2024-09-21T03:55:54.864 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:54.857+0000 7fac3a431740 1 --2- 172.21.15.39:0/327879805 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564624b22390 0x564624eead40 unknown :-1 s=CLOSED pgs=1239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:54.864 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:54.857+0000 7fac3a431740 1 --2- 172.21.15.39:0/327879805 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564624b1a1b0 0x564624eea800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:54.864 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:54.857+0000 7fac3a431740 1 --2- 172.21.15.39:0/327879805 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564624b43070 0x564624eea2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:54.864 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:54.857+0000 7fac3a431740 1 -- 172.21.15.39:0/327879805 >> 172.21.15.39:0/327879805 conn(0x564624b37460 msgr2=0x564624ee81b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:55:54.865 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:54.857+0000 7fac3a431740 1 -- 172.21.15.39:0/327879805 shutdown_connections 2024-09-21T03:55:54.865 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:54.858+0000 7fac3a431740 1 -- 172.21.15.39:0/327879805 wait complete. 2024-09-21T03:55:54.865 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:54.858+0000 7fac3a431740 1 -- start start 2024-09-21T03:55:54.865 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:54.858+0000 7fac3a431740 1 Processor -- start 2024-09-21T03:55:54.865 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:54.858+0000 7fac3a431740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564624b1a1b0 0x564624ef58e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:55:54.865 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:54.858+0000 7fac3a431740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564624b22390 0x564624ef5e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:55:54.865 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:54.858+0000 7fac3a431740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564624b43070 0x564624ef6380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:55:54.865 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:54.858+0000 7fac3a431740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x564624cb1700 con 0x564624b43070 2024-09-21T03:55:54.865 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:54.858+0000 7fac3a431740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x564624deb160 con 0x564624b22390 2024-09-21T03:55:54.865 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:54.858+0000 7fac3a431740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x564624ef8aa0 con 0x564624b1a1b0 2024-09-21T03:55:54.865 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:54.859+0000 7fac3a431740 10 client.0 _ll_register_callbacks cb 0x564624e049e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:55:54.865 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:54.859+0000 7fac3245a640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564624b43070 0x564624ef6380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:55:54.865 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:54.859+0000 7fac3245a640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564624b43070 0x564624ef6380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:51838/0 (socket says 172.21.15.39:51838) 2024-09-21T03:55:54.865 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:54.859+0000 7fac3245a640 1 -- 172.21.15.39:0/4027391869 learned_addr learned my addr 172.21.15.39:0/4027391869 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:55:54.866 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:54.859+0000 7fac31458640 1 --2- 172.21.15.39:0/4027391869 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564624b22390 0x564624ef5e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:55:54.866 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:54.859+0000 7fac31c59640 1 --2- 172.21.15.39:0/4027391869 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564624b1a1b0 0x564624ef58e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:55:54.866 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:54.859+0000 7fac3245a640 1 -- 172.21.15.39:0/4027391869 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564624b1a1b0 msgr2=0x564624ef58e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:55:54.866 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:54.859+0000 7fac3245a640 1 --2- 172.21.15.39:0/4027391869 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564624b1a1b0 0x564624ef58e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:54.866 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:54.859+0000 7fac3245a640 1 -- 172.21.15.39:0/4027391869 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564624b22390 msgr2=0x564624ef5e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:55:54.866 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:54.859+0000 7fac3245a640 1 --2- 172.21.15.39:0/4027391869 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564624b22390 0x564624ef5e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:54.866 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:54.859+0000 7fac3245a640 1 -- 172.21.15.39:0/4027391869 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564624e18c90 con 0x564624b43070 2024-09-21T03:55:54.866 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:54.859+0000 7fac3245a640 1 --2- 172.21.15.39:0/4027391869 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564624b43070 0x564624ef6380 secure :-1 s=READY pgs=1240 cs=0 l=1 rev1=1 crypto rx=0x7fac240316d0 tx=0x7fac240095c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e54b4d53d06fdfc server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:55:54.866 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:54.859+0000 7fac157fa640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-09-21T03:55:54.866 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:54.859+0000 7fac157fa640 1 -- 172.21.15.39:0/4027391869 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac24003d80 con 0x564624b43070 2024-09-21T03:55:54.866 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:54.859+0000 7fac157fa640 1 -- 172.21.15.39:0/4027391869 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8528+0+0 (secure 0 0 0) 0x7fac24004040 con 0x564624b43070 2024-09-21T03:55:54.866 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:54.859+0000 7fac3a431740 1 -- 172.21.15.39:0/4027391869 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fac1c0054d0 con 0x564624b43070 2024-09-21T03:55:54.866 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:54.860+0000 7fac157fa640 1 -- 172.21.15.39:0/4027391869 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fac24004240 con 0x564624b43070 2024-09-21T03:55:54.866 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:54.860+0000 7fac3a431740 10 client.18024 fetch_fsmap learned FSMap version 29 2024-09-21T03:55:54.867 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:54.860+0000 7fac3a431740 1 -- 172.21.15.39:0/4027391869 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x564624f04780 con 0x564624b43070 2024-09-21T03:55:54.867 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:54.860+0000 7fac157fa640 1 -- 172.21.15.39:0/4027391869 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fac1c0054d0 con 0x564624b43070 2024-09-21T03:55:54.867 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:54.860+0000 7fac157fa640 1 -- 172.21.15.39:0/4027391869 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fac24050020 con 0x564624b43070 2024-09-21T03:55:54.867 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:54.860+0000 7fac3a431740 10 client.18024 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T03:55:54.867 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:54.860+0000 7fac3a431740 10 client.18024 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.a 2024-09-21T03:55:54.867 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:54.860+0000 7fac3a431740 20 client.18024 populate_metadata read hostname 'smithi039' 2024-09-21T03:55:54.867 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:54.860+0000 7fac3a431740 1 --2- 172.21.15.39:0/4027391869 >> [v2:172.21.15.39:6840/4247190456,v1:172.21.15.39:6841/4247190456] conn(0x564624b019f0 0x564624f07c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:55:54.867 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:54.860+0000 7fac3a431740 4 client.18024 mds_command: new command op to 14868 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:55:54.867 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:54.860+0000 7fac3a431740 1 -- 172.21.15.39:0/4027391869 --> [v2:172.21.15.39:6840/4247190456,v1:172.21.15.39:6841/4247190456] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564624e62df0 con 0x564624b019f0 2024-09-21T03:55:54.867 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:54.860+0000 7fac31c59640 1 --2- 172.21.15.39:0/4027391869 >> [v2:172.21.15.39:6840/4247190456,v1:172.21.15.39:6841/4247190456] conn(0x564624b019f0 0x564624f07c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:55:54.867 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:54.861+0000 7fac157fa640 10 client.18024 ms_handle_connect on v2:172.21.15.39:6840/4247190456 2024-09-21T03:55:54.867 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:54.861+0000 7fac31c59640 1 --2- 172.21.15.39:0/4027391869 >> [v2:172.21.15.39:6840/4247190456,v1:172.21.15.39:6841/4247190456] conn(0x564624b019f0 0x564624f07c10 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=c7a39d566ed21bc6 server_cookie=a4f270aa3dbe4586 in_seq=0 out_seq=0 2024-09-21T03:55:54.867 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:54.861+0000 7fac157fa640 1 -- 172.21.15.39:0/4027391869 <== mds.4 v2:172.21.15.39:6840/4247190456 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x564624e62df0 con 0x564624b019f0 2024-09-21T03:55:54.867 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:54.861+0000 7fac157fa640 10 client.18024 handle_command_reply: tid=0 2024-09-21T03:55:54.868 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:54.861+0000 7fac3a431740 1 client.18024 shutdown 2024-09-21T03:55:54.868 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:54.862+0000 7fac3a431740 1 -- 172.21.15.39:0/4027391869 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564624b43070 msgr2=0x564624ef6380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:55:54.868 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:54.862+0000 7fac3a431740 1 --2- 172.21.15.39:0/4027391869 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564624b43070 0x564624ef6380 secure :-1 s=READY pgs=1240 cs=0 l=1 rev1=1 crypto rx=0x7fac240316d0 tx=0x7fac240095c0 comp rx=0 tx=0).stop 2024-09-21T03:55:54.868 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:54.862+0000 7fac3a431740 1 -- 172.21.15.39:0/4027391869 shutdown_connections 2024-09-21T03:55:54.868 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:54.862+0000 7fac3a431740 1 --2- 172.21.15.39:0/4027391869 >> [v2:172.21.15.39:6840/4247190456,v1:172.21.15.39:6841/4247190456] conn(0x564624b019f0 0x564624f07c10 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:54.868 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:54.862+0000 7fac3a431740 1 --2- 172.21.15.39:0/4027391869 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564624b43070 0x564624ef6380 unknown :-1 s=CLOSED pgs=1240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:54.868 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:54.862+0000 7fac3a431740 1 --2- 172.21.15.39:0/4027391869 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564624b22390 0x564624ef5e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:54.868 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:54.862+0000 7fac3a431740 1 --2- 172.21.15.39:0/4027391869 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564624b1a1b0 0x564624ef58e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:54.868 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:54.862+0000 7fac157fa640 0 client.18024 ms_handle_reset on v2:172.21.15.39:6840/4247190456 2024-09-21T03:55:54.868 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:54.862+0000 7fac3a431740 1 -- 172.21.15.39:0/4027391869 >> 172.21.15.39:0/4027391869 conn(0x564624b37460 msgr2=0x564624eed450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:55:54.868 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:54.862+0000 7fac3a431740 1 -- 172.21.15.39:0/4027391869 shutdown_connections 2024-09-21T03:55:54.868 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:54.862+0000 7fac3a431740 1 -- 172.21.15.39:0/4027391869 wait complete. 2024-09-21T03:55:54.868 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:54.862+0000 7fac3a431740 20 client.18024 trim_cache size 0 max 16384 2024-09-21T03:55:54.878 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:54.875+0000 7fac3a431740 1 Processor -- start 2024-09-21T03:55:54.878 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:54.875+0000 7fac3a431740 1 -- start start 2024-09-21T03:55:54.878 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:54.875+0000 7fac3a431740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564624b1a1b0 0x564624ef77c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:55:54.878 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:54.876+0000 7fac3a431740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564624b22390 0x564624f01cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:55:54.878 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:54.876+0000 7fac3a431740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564624b43070 0x564624eefae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:55:54.879 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:54.876+0000 7fac3a431740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fac2c1af7d0 con 0x564624b22390 2024-09-21T03:55:54.879 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:54.876+0000 7fac3a431740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fac2c1b1310 con 0x564624b43070 2024-09-21T03:55:54.879 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:54.876+0000 7fac3a431740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fac2c1afea0 con 0x564624b1a1b0 2024-09-21T03:55:54.879 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:54.876+0000 7fac31458640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564624b22390 0x564624f01cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:55:54.879 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:54.876+0000 7fac31458640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564624b22390 0x564624f01cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:51850/0 (socket says 172.21.15.39:51850) 2024-09-21T03:55:54.879 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:54.876+0000 7fac31458640 1 -- 172.21.15.39:0/3156250154 learned_addr learned my addr 172.21.15.39:0/3156250154 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:55:54.879 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:54.876+0000 7fac3245a640 1 --2- 172.21.15.39:0/3156250154 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564624b43070 0x564624eefae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:55:54.879 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:54.876+0000 7fac31c59640 1 --2- 172.21.15.39:0/3156250154 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564624b1a1b0 0x564624ef77c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:55:54.879 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:54.876+0000 7fac31458640 1 -- 172.21.15.39:0/3156250154 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564624b1a1b0 msgr2=0x564624ef77c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:55:54.879 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:54.876+0000 7fac31458640 1 --2- 172.21.15.39:0/3156250154 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564624b1a1b0 0x564624ef77c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:54.879 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:54.876+0000 7fac31458640 1 -- 172.21.15.39:0/3156250154 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564624b43070 msgr2=0x564624eefae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:55:54.879 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:54.876+0000 7fac31458640 1 --2- 172.21.15.39:0/3156250154 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564624b43070 0x564624eefae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:54.879 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:54.876+0000 7fac31458640 1 -- 172.21.15.39:0/3156250154 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564624ee35d0 con 0x564624b22390 2024-09-21T03:55:54.879 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:54.876+0000 7fac31c59640 1 --2- 172.21.15.39:0/3156250154 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564624b1a1b0 0x564624ef77c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:55:54.880 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:54.876+0000 7fac3245a640 1 --2- 172.21.15.39:0/3156250154 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564624b43070 0x564624eefae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:55:54.880 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:54.876+0000 7fac31458640 1 --2- 172.21.15.39:0/3156250154 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564624b22390 0x564624f01cf0 secure :-1 s=READY pgs=1241 cs=0 l=1 rev1=1 crypto rx=0x7fac1c0045b0 tx=0x7fac1c00b920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4185cffa6a1c5612 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:55:54.880 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:54.876+0000 7fac177fe640 1 -- 172.21.15.39:0/3156250154 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac1c02c070 con 0x564624b22390 2024-09-21T03:55:54.880 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:54.876+0000 7fac177fe640 1 -- 172.21.15.39:0/3156250154 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fac1c0054f0 con 0x564624b22390 2024-09-21T03:55:54.880 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:54.876+0000 7fac177fe640 1 -- 172.21.15.39:0/3156250154 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac1c004dd0 con 0x564624b22390 2024-09-21T03:55:54.883 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:54.877+0000 7fac3a431740 1 -- 172.21.15.39:0/3156250154 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564624b22390 msgr2=0x564624f01cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:55:54.883 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:54.877+0000 7fac3a431740 1 --2- 172.21.15.39:0/3156250154 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564624b22390 0x564624f01cf0 secure :-1 s=READY pgs=1241 cs=0 l=1 rev1=1 crypto rx=0x7fac1c0045b0 tx=0x7fac1c00b920 comp rx=0 tx=0).stop 2024-09-21T03:55:54.883 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:54.877+0000 7fac3a431740 1 -- 172.21.15.39:0/3156250154 shutdown_connections 2024-09-21T03:55:54.883 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:54.877+0000 7fac3a431740 1 --2- 172.21.15.39:0/3156250154 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564624b43070 0x564624eefae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:54.884 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:54.877+0000 7fac3a431740 1 --2- 172.21.15.39:0/3156250154 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564624b22390 0x564624f01cf0 unknown :-1 s=CLOSED pgs=1241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:54.884 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:54.877+0000 7fac3a431740 1 --2- 172.21.15.39:0/3156250154 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564624b1a1b0 0x564624ef77c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:54.884 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:54.877+0000 7fac3a431740 1 -- 172.21.15.39:0/3156250154 >> 172.21.15.39:0/3156250154 conn(0x564624b37460 msgr2=0x564624eef3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:55:54.884 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:54.877+0000 7fac3a431740 1 -- 172.21.15.39:0/3156250154 shutdown_connections 2024-09-21T03:55:54.884 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:54.877+0000 7fac3a431740 1 -- 172.21.15.39:0/3156250154 wait complete. 2024-09-21T03:55:54.884 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:54.878+0000 7fac3a431740 1 -- start start 2024-09-21T03:55:54.884 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:54.878+0000 7fac3a431740 1 Processor -- start 2024-09-21T03:55:54.884 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:54.878+0000 7fac3a431740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564624b1a1b0 0x564624ef9eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:55:54.884 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:54.878+0000 7fac3a431740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564624b22390 0x564624efa3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:55:54.884 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:54.878+0000 7fac3a431740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564624b43070 0x564624efa930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:55:54.884 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:54.878+0000 7fac3a431740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fac2c1b0c40 con 0x564624b22390 2024-09-21T03:55:54.884 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:54.878+0000 7fac3a431740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fac2c1b0570 con 0x564624b43070 2024-09-21T03:55:54.884 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:54.878+0000 7fac3a431740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fac2c1ad7b0 con 0x564624b1a1b0 2024-09-21T03:55:54.884 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:54.878+0000 7fac3a431740 10 client.0 _ll_register_callbacks cb 0x7fac24009b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:55:54.884 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:54.878+0000 7fac31458640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564624b22390 0x564624efa3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:55:54.885 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:54.878+0000 7fac31458640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564624b22390 0x564624efa3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:51854/0 (socket says 172.21.15.39:51854) 2024-09-21T03:55:54.885 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:54.879+0000 7fac31458640 1 -- 172.21.15.39:0/2688769770 learned_addr learned my addr 172.21.15.39:0/2688769770 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:55:54.885 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:54.879+0000 7fac31c59640 1 --2- 172.21.15.39:0/2688769770 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564624b1a1b0 0x564624ef9eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:55:54.885 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:54.879+0000 7fac3245a640 1 --2- 172.21.15.39:0/2688769770 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564624b43070 0x564624efa930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:55:54.885 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:54.879+0000 7fac31458640 1 -- 172.21.15.39:0/2688769770 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564624b1a1b0 msgr2=0x564624ef9eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:55:54.885 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:54.879+0000 7fac31458640 1 --2- 172.21.15.39:0/2688769770 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564624b1a1b0 0x564624ef9eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:54.885 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:54.879+0000 7fac31458640 1 -- 172.21.15.39:0/2688769770 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564624b43070 msgr2=0x564624efa930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:55:54.885 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:54.879+0000 7fac31458640 1 --2- 172.21.15.39:0/2688769770 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564624b43070 0x564624efa930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:54.885 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:54.879+0000 7fac31458640 1 -- 172.21.15.39:0/2688769770 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564624f71750 con 0x564624b22390 2024-09-21T03:55:54.885 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:54.879+0000 7fac31458640 1 --2- 172.21.15.39:0/2688769770 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564624b22390 0x564624efa3f0 secure :-1 s=READY pgs=1242 cs=0 l=1 rev1=1 crypto rx=0x7fac1c006010 tx=0x7fac1c00bf50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=61e7dca072ed52eb server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:55:54.885 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:54.879+0000 7fac157fa640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-09-21T03:55:54.885 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:54.879+0000 7fac157fa640 1 -- 172.21.15.39:0/2688769770 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac1c035060 con 0x564624b22390 2024-09-21T03:55:54.885 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:54.879+0000 7fac157fa640 1 -- 172.21.15.39:0/2688769770 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8528+0+0 (secure 0 0 0) 0x7fac1c004d10 con 0x564624b22390 2024-09-21T03:55:54.885 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:54.879+0000 7fac3a431740 1 -- 172.21.15.39:0/2688769770 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fac1c0054f0 con 0x564624b22390 2024-09-21T03:55:54.885 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:54.880+0000 7fac157fa640 1 -- 172.21.15.39:0/2688769770 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fac1c02c070 con 0x564624b22390 2024-09-21T03:55:54.885 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:54.880+0000 7fac3a431740 10 client.18030 fetch_fsmap learned FSMap version 29 2024-09-21T03:55:54.886 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:54.880+0000 7fac3a431740 1 -- 172.21.15.39:0/2688769770 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x564624f72940 con 0x564624b22390 2024-09-21T03:55:54.886 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:54.880+0000 7fac157fa640 1 -- 172.21.15.39:0/2688769770 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fac1c0054f0 con 0x564624b22390 2024-09-21T03:55:54.886 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:54.880+0000 7fac157fa640 1 -- 172.21.15.39:0/2688769770 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fac1c026030 con 0x564624b22390 2024-09-21T03:55:54.886 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:54.880+0000 7fac3a431740 10 client.18030 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T03:55:54.886 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:54.880+0000 7fac3a431740 10 client.18030 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.a 2024-09-21T03:55:54.886 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:54.880+0000 7fac3a431740 20 client.18030 populate_metadata read hostname 'smithi039' 2024-09-21T03:55:54.886 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:54.880+0000 7fac3a431740 1 --2- 172.21.15.39:0/2688769770 >> [v2:172.21.15.39:6840/4247190456,v1:172.21.15.39:6841/4247190456] conn(0x564624ad6da0 0x564624f74dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:55:54.886 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:54.880+0000 7fac3a431740 4 client.18030 mds_command: new command op to 14868 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T03:55:54.886 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:54.880+0000 7fac3a431740 1 -- 172.21.15.39:0/2688769770 --> [v2:172.21.15.39:6840/4247190456,v1:172.21.15.39:6841/4247190456] -- command(tid 0: {"prefix": "damage ls"}) -- 0x564624e5e8a0 con 0x564624ad6da0 2024-09-21T03:55:54.886 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:54.880+0000 7fac31c59640 1 --2- 172.21.15.39:0/2688769770 >> [v2:172.21.15.39:6840/4247190456,v1:172.21.15.39:6841/4247190456] conn(0x564624ad6da0 0x564624f74dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:55:54.886 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:54.881+0000 7fac157fa640 10 client.18030 ms_handle_connect on v2:172.21.15.39:6840/4247190456 2024-09-21T03:55:54.886 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:54.881+0000 7fac31c59640 1 --2- 172.21.15.39:0/2688769770 >> [v2:172.21.15.39:6840/4247190456,v1:172.21.15.39:6841/4247190456] conn(0x564624ad6da0 0x564624f74dd0 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=e9ae55c7424d625b server_cookie=be3af156268bd6d3 in_seq=0 out_seq=0 2024-09-21T03:55:54.886 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:54.881+0000 7fac157fa640 1 -- 172.21.15.39:0/2688769770 <== mds.4 v2:172.21.15.39:6840/4247190456 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x564624e5e8a0 con 0x564624ad6da0 2024-09-21T03:55:54.886 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:54.881+0000 7fac157fa640 10 client.18030 handle_command_reply: tid=0 2024-09-21T03:55:54.887 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:54.881+0000 7fac3a431740 1 client.18030 shutdown 2024-09-21T03:55:54.887 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:54.881+0000 7fac3a431740 1 -- 172.21.15.39:0/2688769770 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564624b22390 msgr2=0x564624efa3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:55:54.887 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:54.881+0000 7fac3a431740 1 --2- 172.21.15.39:0/2688769770 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564624b22390 0x564624efa3f0 secure :-1 s=READY pgs=1242 cs=0 l=1 rev1=1 crypto rx=0x7fac1c006010 tx=0x7fac1c00bf50 comp rx=0 tx=0).stop 2024-09-21T03:55:54.887 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:54.881+0000 7fac3a431740 1 -- 172.21.15.39:0/2688769770 shutdown_connections 2024-09-21T03:55:54.887 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:54.881+0000 7fac3a431740 1 --2- 172.21.15.39:0/2688769770 >> [v2:172.21.15.39:6840/4247190456,v1:172.21.15.39:6841/4247190456] conn(0x564624ad6da0 0x564624f74dd0 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:54.887 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:54.881+0000 7fac3a431740 1 --2- 172.21.15.39:0/2688769770 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564624b43070 0x564624efa930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:54.887 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:54.881+0000 7fac3a431740 1 --2- 172.21.15.39:0/2688769770 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564624b22390 0x564624efa3f0 unknown :-1 s=CLOSED pgs=1242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:54.887 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:54.881+0000 7fac3a431740 1 --2- 172.21.15.39:0/2688769770 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564624b1a1b0 0x564624ef9eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:54.887 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:54.881+0000 7fac157fa640 0 client.18030 ms_handle_reset on v2:172.21.15.39:6840/4247190456 2024-09-21T03:55:54.887 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:54.881+0000 7fac3a431740 1 -- 172.21.15.39:0/2688769770 >> 172.21.15.39:0/2688769770 conn(0x564624b37460 msgr2=0x564624eef3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:55:54.887 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:54.881+0000 7fac3a431740 1 -- 172.21.15.39:0/2688769770 shutdown_connections 2024-09-21T03:55:54.887 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:54.881+0000 7fac3a431740 1 -- 172.21.15.39:0/2688769770 wait complete. 2024-09-21T03:55:54.887 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:54.881+0000 7fac3a431740 20 client.18030 trim_cache size 0 max 16384 2024-09-21T03:55:54.888 INFO:teuthology.orchestra.run.smithi039.stdout:[] 2024-09-21T03:55:54.888 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:54.882+0000 7fac33ebd640 1 -- 172.21.15.39:0/3834966824 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7fac180b6680 msgr2=0x7fac180b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:55:54.888 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:54.882+0000 7fac33ebd640 1 --2- 172.21.15.39:0/3834966824 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7fac180b6680 0x7fac180b8b90 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7fac24006010 tx=0x7fac2403f040 comp rx=0 tx=0).stop 2024-09-21T03:55:54.889 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:54.882+0000 7fac33ebd640 1 -- 172.21.15.39:0/3834966824 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fac2c103b60 msgr2=0x7fac2c19a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:55:54.889 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:54.882+0000 7fac33ebd640 1 --2- 172.21.15.39:0/3834966824 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fac2c103b60 0x7fac2c19a480 secure :-1 s=READY pgs=1238 cs=0 l=1 rev1=1 crypto rx=0x7fac2800e9f0 tx=0x7fac2800edf0 comp rx=0 tx=0).stop 2024-09-21T03:55:54.889 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:54.882+0000 7fac33ebd640 1 -- 172.21.15.39:0/3834966824 shutdown_connections 2024-09-21T03:55:54.889 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:54.882+0000 7fac33ebd640 1 --2- 172.21.15.39:0/3834966824 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7fac180b6680 0x7fac180b8b90 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:54.889 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:54.882+0000 7fac33ebd640 1 --2- 172.21.15.39:0/3834966824 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fac2c109df0 0x7fac2c19fcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:54.889 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:54.882+0000 7fac33ebd640 1 --2- 172.21.15.39:0/3834966824 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fac2c1094a0 0x7fac2c19aa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:54.889 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:54.882+0000 7fac33ebd640 1 --2- 172.21.15.39:0/3834966824 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fac2c103b60 0x7fac2c19a480 unknown :-1 s=CLOSED pgs=1238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:54.889 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:54.882+0000 7fac33ebd640 1 -- 172.21.15.39:0/3834966824 >> 172.21.15.39:0/3834966824 conn(0x7fac2c07b850 msgr2=0x7fac2c105470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:55:54.889 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:54.882+0000 7fac33ebd640 1 -- 172.21.15.39:0/3834966824 shutdown_connections 2024-09-21T03:55:54.889 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:54.882+0000 7fac33ebd640 1 -- 172.21.15.39:0/3834966824 wait complete. 2024-09-21T03:55:55.335 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:55:55 smithi039 ceph-mon[31211]: pgmap v627: 97 pgs: 97 active+clean; 10 MiB data, 525 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-09-21T03:55:55.468 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:55:55 smithi161 ceph-mon[34533]: pgmap v627: 97 pgs: 97 active+clean; 10 MiB data, 525 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-09-21T03:55:55.474 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:55:55 smithi150 ceph-mon[34289]: pgmap v627: 97 pgs: 97 active+clean; 10 MiB data, 525 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-09-21T03:55:55.755 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 51116d5c-77ca-11ef-baf5-efdc52797490 -- ceph tell mds.1:3 damage ls 2024-09-21T03:55:56.016 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/51116d5c-77ca-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:55:56.535 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:56.526+0000 7fb2e45d0640 1 -- 172.21.15.39:0/4012746 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb2dc103b60 msgr2=0x7fb2dc103f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:55:56.535 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:56.526+0000 7fb2e45d0640 1 --2- 172.21.15.39:0/4012746 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb2dc103b60 0x7fb2dc103f70 secure :-1 s=READY pgs=1243 cs=0 l=1 rev1=1 crypto rx=0x7fb2d8009900 tx=0x7fb2d802a3f0 comp rx=0 tx=0).stop 2024-09-21T03:55:56.535 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:56.526+0000 7fb2e45d0640 1 -- 172.21.15.39:0/4012746 shutdown_connections 2024-09-21T03:55:56.535 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:56.526+0000 7fb2e45d0640 1 --2- 172.21.15.39:0/4012746 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb2dc109df0 0x7fb2dc110620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:56.535 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:56.526+0000 7fb2e45d0640 1 --2- 172.21.15.39:0/4012746 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb2dc1094a0 0x7fb2dc1098b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:56.535 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:56.526+0000 7fb2e45d0640 1 --2- 172.21.15.39:0/4012746 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb2dc103b60 0x7fb2dc103f70 unknown :-1 s=CLOSED pgs=1243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:56.535 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:56.526+0000 7fb2e45d0640 1 -- 172.21.15.39:0/4012746 >> 172.21.15.39:0/4012746 conn(0x7fb2dc07b850 msgr2=0x7fb2dc07bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:55:56.535 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:56.526+0000 7fb2e45d0640 1 -- 172.21.15.39:0/4012746 shutdown_connections 2024-09-21T03:55:56.536 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:56.526+0000 7fb2e45d0640 1 -- 172.21.15.39:0/4012746 wait complete. 2024-09-21T03:55:56.536 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:56.527+0000 7fb2e45d0640 1 Processor -- start 2024-09-21T03:55:56.536 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:56.527+0000 7fb2e45d0640 1 -- start start 2024-09-21T03:55:56.536 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:56.527+0000 7fb2e45d0640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb2dc103b60 0x7fb2dc19a500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:55:56.536 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:56.527+0000 7fb2e45d0640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb2dc1094a0 0x7fb2dc19aaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:55:56.536 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:56.527+0000 7fb2e45d0640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb2dc109df0 0x7fb2dc19fd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:55:56.536 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:56.527+0000 7fb2e45d0640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fb2dc113ab0 con 0x7fb2dc109df0 2024-09-21T03:55:56.536 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:56.527+0000 7fb2e45d0640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fb2dc113930 con 0x7fb2dc103b60 2024-09-21T03:55:56.536 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:56.527+0000 7fb2e45d0640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fb2dc113c30 con 0x7fb2dc1094a0 2024-09-21T03:55:56.536 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:56.528+0000 7fb2e2b6d640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb2dc109df0 0x7fb2dc19fd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:55:56.536 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:56.528+0000 7fb2e2b6d640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb2dc109df0 0x7fb2dc19fd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:51878/0 (socket says 172.21.15.39:51878) 2024-09-21T03:55:56.536 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:56.528+0000 7fb2e2b6d640 1 -- 172.21.15.39:0/3127144692 learned_addr learned my addr 172.21.15.39:0/3127144692 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:55:56.537 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:56.528+0000 7fb2e1b6b640 1 --2- 172.21.15.39:0/3127144692 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb2dc1094a0 0x7fb2dc19aaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:55:56.537 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:56.528+0000 7fb2e236c640 1 --2- 172.21.15.39:0/3127144692 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb2dc103b60 0x7fb2dc19a500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:55:56.537 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:56.528+0000 7fb2e2b6d640 1 -- 172.21.15.39:0/3127144692 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb2dc1094a0 msgr2=0x7fb2dc19aaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:55:56.537 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:56.528+0000 7fb2e2b6d640 1 --2- 172.21.15.39:0/3127144692 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb2dc1094a0 0x7fb2dc19aaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:56.537 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:56.528+0000 7fb2e2b6d640 1 -- 172.21.15.39:0/3127144692 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb2dc103b60 msgr2=0x7fb2dc19a500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:55:56.537 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:56.528+0000 7fb2e2b6d640 1 --2- 172.21.15.39:0/3127144692 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb2dc103b60 0x7fb2dc19a500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:56.537 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:56.528+0000 7fb2e2b6d640 1 -- 172.21.15.39:0/3127144692 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb2dc1a03b0 con 0x7fb2dc109df0 2024-09-21T03:55:56.537 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:56.528+0000 7fb2e2b6d640 1 --2- 172.21.15.39:0/3127144692 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb2dc109df0 0x7fb2dc19fd40 secure :-1 s=READY pgs=1244 cs=0 l=1 rev1=1 crypto rx=0x7fb2d400cce0 tx=0x7fb2d400a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:55:56.537 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:56.528+0000 7fb2d37fe640 1 -- 172.21.15.39:0/3127144692 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2d400f450 con 0x7fb2dc109df0 2024-09-21T03:55:56.537 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:56.528+0000 7fb2e45d0640 1 -- 172.21.15.39:0/3127144692 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb2dc1a06a0 con 0x7fb2dc109df0 2024-09-21T03:55:56.538 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:56.528+0000 7fb2d37fe640 1 -- 172.21.15.39:0/3127144692 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb2d400faa0 con 0x7fb2dc109df0 2024-09-21T03:55:56.538 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:56.528+0000 7fb2d37fe640 1 -- 172.21.15.39:0/3127144692 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2d4018b70 con 0x7fb2dc109df0 2024-09-21T03:55:56.538 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:56.528+0000 7fb2e45d0640 1 -- 172.21.15.39:0/3127144692 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb2dc1a6ec0 con 0x7fb2dc109df0 2024-09-21T03:55:56.538 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:56.531+0000 7fb2eab44740 1 Processor -- start 2024-09-21T03:55:56.538 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:56.531+0000 7fb2d37fe640 1 -- 172.21.15.39:0/3127144692 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 27) ==== 154643+0+0 (secure 0 0 0) 0x7fb2d4018d10 con 0x7fb2dc109df0 2024-09-21T03:55:56.538 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:56.531+0000 7fb2eab44740 1 -- start start 2024-09-21T03:55:56.538 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:56.532+0000 7fb2eab44740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f66aff3070 0x55f66b39a6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:55:56.538 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:56.532+0000 7fb2eab44740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55f66afca1b0 0x55f66b39abe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:55:56.538 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:56.532+0000 7fb2eab44740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f66afd2390 0x55f66b39b120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:55:56.538 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:56.532+0000 7fb2eab44740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x55f66b310ea0 con 0x55f66afca1b0 2024-09-21T03:55:56.538 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:56.532+0000 7fb2eab44740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x55f66b3002d0 con 0x55f66afd2390 2024-09-21T03:55:56.538 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:56.532+0000 7fb2eab44740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55f66b3040e0 con 0x55f66aff3070 2024-09-21T03:55:56.538 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:56.532+0000 7fb2e236c640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55f66afca1b0 0x55f66b39abe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:55:56.539 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:56.532+0000 7fb2e236c640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55f66afca1b0 0x55f66b39abe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:51880/0 (socket says 172.21.15.39:51880) 2024-09-21T03:55:56.539 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:56.532+0000 7fb2e236c640 1 -- 172.21.15.39:0/1418091987 learned_addr learned my addr 172.21.15.39:0/1418091987 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:55:56.539 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:56.532+0000 7fb2e1b6b640 1 --2- 172.21.15.39:0/1418091987 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f66aff3070 0x55f66b39a6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:55:56.539 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:56.532+0000 7fb2e1b6b640 1 --2- 172.21.15.39:0/1418091987 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f66afd2390 0x55f66b39b120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:55:56.539 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:56.532+0000 7fb2e236c640 1 -- 172.21.15.39:0/1418091987 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f66aff3070 msgr2=0x55f66b39a6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:55:56.539 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:56.532+0000 7fb2e236c640 1 --2- 172.21.15.39:0/1418091987 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f66aff3070 0x55f66b39a6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:56.539 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:56.532+0000 7fb2d37fe640 1 --2- 172.21.15.39:0/3127144692 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7fb2c80b66b0 0x7fb2c80b8bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:55:56.539 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:56.532+0000 7fb2e236c640 1 -- 172.21.15.39:0/1418091987 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f66afd2390 msgr2=0x55f66b39b120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:55:56.539 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:56.532+0000 7fb2e236c640 1 --2- 172.21.15.39:0/1418091987 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f66afd2390 0x55f66b39b120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:56.539 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:56.532+0000 7fb2e236c640 1 -- 172.21.15.39:0/1418091987 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f66b05e650 con 0x55f66afca1b0 2024-09-21T03:55:56.539 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:56.533+0000 7fb2e2b6d640 1 --2- 172.21.15.39:0/3127144692 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7fb2c80b66b0 0x7fb2c80b8bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:55:56.539 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:56.533+0000 7fb2e236c640 1 --2- 172.21.15.39:0/1418091987 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55f66afca1b0 0x55f66b39abe0 secure :-1 s=READY pgs=1245 cs=0 l=1 rev1=1 crypto rx=0x7fb2d802a910 tx=0x7fb2d8032ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e2e3ab289668ee13 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:55:56.539 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:56.533+0000 7fb2d37fe640 1 -- 172.21.15.39:0/3127144692 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8528+0+0 (secure 0 0 0) 0x7fb2d40e7ac0 con 0x7fb2dc109df0 2024-09-21T03:55:56.539 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:56.533+0000 7fb2d0ff9640 1 -- 172.21.15.39:0/1418091987 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2d803f070 con 0x55f66afca1b0 2024-09-21T03:55:56.539 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:56.533+0000 7fb2d0ff9640 1 -- 172.21.15.39:0/1418091987 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb2d8032e50 con 0x55f66afca1b0 2024-09-21T03:55:56.540 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:56.533+0000 7fb2d0ff9640 1 -- 172.21.15.39:0/1418091987 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2d8035070 con 0x55f66afca1b0 2024-09-21T03:55:56.543 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:56.534+0000 7fb2eab44740 1 -- 172.21.15.39:0/1418091987 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55f66afca1b0 msgr2=0x55f66b39abe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:55:56.543 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:56.534+0000 7fb2eab44740 1 --2- 172.21.15.39:0/1418091987 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55f66afca1b0 0x55f66b39abe0 secure :-1 s=READY pgs=1245 cs=0 l=1 rev1=1 crypto rx=0x7fb2d802a910 tx=0x7fb2d8032ce0 comp rx=0 tx=0).stop 2024-09-21T03:55:56.543 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:56.534+0000 7fb2e2b6d640 1 --2- 172.21.15.39:0/3127144692 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7fb2c80b66b0 0x7fb2c80b8bc0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7fb2d40ad610 tx=0x7fb2d400ff50 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:55:56.543 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:56.534+0000 7fb2eab44740 1 -- 172.21.15.39:0/1418091987 shutdown_connections 2024-09-21T03:55:56.543 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:56.534+0000 7fb2eab44740 1 --2- 172.21.15.39:0/1418091987 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f66afd2390 0x55f66b39b120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:56.543 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:56.534+0000 7fb2eab44740 1 --2- 172.21.15.39:0/1418091987 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55f66afca1b0 0x55f66b39abe0 unknown :-1 s=CLOSED pgs=1245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:56.544 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:56.534+0000 7fb2eab44740 1 --2- 172.21.15.39:0/1418091987 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f66aff3070 0x55f66b39a6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:56.544 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:56.534+0000 7fb2eab44740 1 -- 172.21.15.39:0/1418091987 >> 172.21.15.39:0/1418091987 conn(0x55f66afe7460 msgr2=0x55f66b398590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:55:56.544 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:56.534+0000 7fb2eab44740 1 -- 172.21.15.39:0/1418091987 shutdown_connections 2024-09-21T03:55:56.544 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:56.534+0000 7fb2eab44740 1 -- 172.21.15.39:0/1418091987 wait complete. 2024-09-21T03:55:56.544 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:56.534+0000 7fb2eab44740 1 -- start start 2024-09-21T03:55:56.544 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:56.535+0000 7fb2eab44740 1 Processor -- start 2024-09-21T03:55:56.544 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:56.535+0000 7fb2eab44740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55f66afca1b0 0x55f66b3a5d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:55:56.544 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:56.535+0000 7fb2eab44740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f66afd2390 0x55f66b3a62d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:55:56.544 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:56.535+0000 7fb2eab44740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f66aff3070 0x55f66b3b2140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:55:56.545 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:56.535+0000 7fb2eab44740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x55f66b161700 con 0x55f66afca1b0 2024-09-21T03:55:56.545 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:56.535+0000 7fb2eab44740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x55f66b29b160 con 0x55f66afd2390 2024-09-21T03:55:56.545 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:56.535+0000 7fb2eab44740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55f66b3a8e20 con 0x55f66aff3070 2024-09-21T03:55:56.545 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:56.535+0000 7fb2eab44740 10 client.0 _ll_register_callbacks cb 0x55f66b2b49e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:55:56.545 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:56.535+0000 7fb2e1b6b640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55f66afca1b0 0x55f66b3a5d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:55:56.545 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:56.535+0000 7fb2e1b6b640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55f66afca1b0 0x55f66b3a5d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:51896/0 (socket says 172.21.15.39:51896) 2024-09-21T03:55:56.545 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:56.535+0000 7fb2e1b6b640 1 -- 172.21.15.39:0/2112532391 learned_addr learned my addr 172.21.15.39:0/2112532391 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:55:56.545 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:56.535+0000 7fb2e236c640 1 --2- 172.21.15.39:0/2112532391 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f66afd2390 0x55f66b3a62d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:55:56.545 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:56.535+0000 7fb2e1b6b640 1 --2- 172.21.15.39:0/2112532391 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f66aff3070 0x55f66b3b2140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:55:56.545 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:56.535+0000 7fb2e1b6b640 1 -- 172.21.15.39:0/2112532391 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f66aff3070 msgr2=0x55f66b3b2140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:55:56.545 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:56.535+0000 7fb2e1b6b640 1 --2- 172.21.15.39:0/2112532391 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f66aff3070 0x55f66b3b2140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:56.545 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:56.535+0000 7fb2e1b6b640 1 -- 172.21.15.39:0/2112532391 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f66afd2390 msgr2=0x55f66b3a62d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:55:56.545 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:56.535+0000 7fb2e1b6b640 1 --2- 172.21.15.39:0/2112532391 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f66afd2390 0x55f66b3a62d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:56.545 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:56.535+0000 7fb2e1b6b640 1 -- 172.21.15.39:0/2112532391 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f66b2c8c90 con 0x55f66afca1b0 2024-09-21T03:55:56.546 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:56.536+0000 7fb2e1b6b640 1 --2- 172.21.15.39:0/2112532391 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55f66afca1b0 0x55f66b3a5d90 secure :-1 s=READY pgs=1246 cs=0 l=1 rev1=1 crypto rx=0x7fb2cc0035f0 tx=0x7fb2cc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f18cca535e3f008b server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:55:56.546 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:56.536+0000 7fb2c5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-09-21T03:55:56.546 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:56.536+0000 7fb2c5ffb640 1 -- 172.21.15.39:0/2112532391 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2cc005020 con 0x55f66afca1b0 2024-09-21T03:55:56.546 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:56.536+0000 7fb2c5ffb640 1 -- 172.21.15.39:0/2112532391 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8528+0+0 (secure 0 0 0) 0x7fb2cc0051c0 con 0x55f66afca1b0 2024-09-21T03:55:56.546 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:56.536+0000 7fb2eab44740 1 -- 172.21.15.39:0/2112532391 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb2d8032e50 con 0x55f66afca1b0 2024-09-21T03:55:56.546 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:56.536+0000 7fb2c5ffb640 1 -- 172.21.15.39:0/2112532391 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fb2cc0053c0 con 0x55f66afca1b0 2024-09-21T03:55:56.546 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:56.536+0000 7fb2eab44740 10 client.18042 fetch_fsmap learned FSMap version 29 2024-09-21T03:55:56.546 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:56.536+0000 7fb2eab44740 1 -- 172.21.15.39:0/2112532391 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55f66b3b4b90 con 0x55f66afca1b0 2024-09-21T03:55:56.546 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:56.536+0000 7fb2c5ffb640 1 -- 172.21.15.39:0/2112532391 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb2d8032e50 con 0x55f66afca1b0 2024-09-21T03:55:56.546 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:56.536+0000 7fb2c5ffb640 1 -- 172.21.15.39:0/2112532391 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fb2cc004830 con 0x55f66afca1b0 2024-09-21T03:55:56.546 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:56.537+0000 7fb2eab44740 10 client.18042 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T03:55:56.546 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:56.537+0000 7fb2eab44740 10 client.18042 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-21T03:55:56.546 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:56.537+0000 7fb2eab44740 20 client.18042 populate_metadata read hostname 'smithi039' 2024-09-21T03:55:56.546 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:56.537+0000 7fb2eab44740 1 --2- 172.21.15.39:0/2112532391 >> [v2:172.21.15.161:6836/3451544597,v1:172.21.15.161:6837/3451544597] conn(0x55f66afb19f0 0x55f66b3b7ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:55:56.547 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:56.537+0000 7fb2eab44740 4 client.18042 mds_command: new command op to 24646 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:55:56.547 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:56.537+0000 7fb2eab44740 1 -- 172.21.15.39:0/2112532391 --> [v2:172.21.15.161:6836/3451544597,v1:172.21.15.161:6837/3451544597] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f66b312df0 con 0x55f66afb19f0 2024-09-21T03:55:56.547 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:56.537+0000 7fb2e236c640 1 --2- 172.21.15.39:0/2112532391 >> [v2:172.21.15.161:6836/3451544597,v1:172.21.15.161:6837/3451544597] conn(0x55f66afb19f0 0x55f66b3b7ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:55:56.547 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:56.537+0000 7fb2e236c640 1 --2- 172.21.15.39:0/2112532391 >> [v2:172.21.15.161:6836/3451544597,v1:172.21.15.161:6837/3451544597] conn(0x55f66afb19f0 0x55f66b3b7ff0 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=de64dbd5e5a98278 server_cookie=f3c706a74ff36f1c in_seq=0 out_seq=0 2024-09-21T03:55:56.547 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:56.537+0000 7fb2c5ffb640 10 client.18042 ms_handle_connect on v2:172.21.15.161:6836/3451544597 2024-09-21T03:55:56.547 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:56.538+0000 7fb2c5ffb640 1 -- 172.21.15.39:0/2112532391 <== mds.3 v2:172.21.15.161:6836/3451544597 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55f66b312df0 con 0x55f66afb19f0 2024-09-21T03:55:56.547 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:56.538+0000 7fb2c5ffb640 10 client.18042 handle_command_reply: tid=0 2024-09-21T03:55:56.547 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:56.538+0000 7fb2eab44740 1 client.18042 shutdown 2024-09-21T03:55:56.547 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:56.538+0000 7fb2eab44740 1 -- 172.21.15.39:0/2112532391 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55f66afca1b0 msgr2=0x55f66b3a5d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:55:56.547 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:56.538+0000 7fb2eab44740 1 --2- 172.21.15.39:0/2112532391 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55f66afca1b0 0x55f66b3a5d90 secure :-1 s=READY pgs=1246 cs=0 l=1 rev1=1 crypto rx=0x7fb2cc0035f0 tx=0x7fb2cc0046c0 comp rx=0 tx=0).stop 2024-09-21T03:55:56.547 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:56.538+0000 7fb2eab44740 1 -- 172.21.15.39:0/2112532391 shutdown_connections 2024-09-21T03:55:56.547 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:56.538+0000 7fb2eab44740 1 --2- 172.21.15.39:0/2112532391 >> [v2:172.21.15.161:6836/3451544597,v1:172.21.15.161:6837/3451544597] conn(0x55f66afb19f0 0x55f66b3b7ff0 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:56.547 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:56.539+0000 7fb2c5ffb640 0 client.18042 ms_handle_reset on v2:172.21.15.161:6836/3451544597 2024-09-21T03:55:56.547 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:56.539+0000 7fb2eab44740 1 --2- 172.21.15.39:0/2112532391 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f66aff3070 0x55f66b3b2140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:56.547 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:56.539+0000 7fb2eab44740 1 --2- 172.21.15.39:0/2112532391 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f66afd2390 0x55f66b3a62d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:56.548 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:56.539+0000 7fb2eab44740 1 --2- 172.21.15.39:0/2112532391 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55f66afca1b0 0x55f66b3a5d90 unknown :-1 s=CLOSED pgs=1246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:56.548 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:56.539+0000 7fb2eab44740 1 -- 172.21.15.39:0/2112532391 >> 172.21.15.39:0/2112532391 conn(0x55f66afe7460 msgr2=0x55f66b39d830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:55:56.548 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:56.539+0000 7fb2eab44740 1 -- 172.21.15.39:0/2112532391 shutdown_connections 2024-09-21T03:55:56.548 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:56.539+0000 7fb2eab44740 1 -- 172.21.15.39:0/2112532391 wait complete. 2024-09-21T03:55:56.548 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:56.539+0000 7fb2eab44740 20 client.18042 trim_cache size 0 max 16384 2024-09-21T03:55:56.555 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:56.552+0000 7fb2eab44740 1 Processor -- start 2024-09-21T03:55:56.555 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:56.552+0000 7fb2eab44740 1 -- start start 2024-09-21T03:55:56.555 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:56.552+0000 7fb2eab44740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55f66afb19f0 0x55f66b3a6250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:55:56.555 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:56.552+0000 7fb2eab44740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f66afca1b0 0x55f66b39d7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:55:56.555 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:56.552+0000 7fb2eab44740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f66afd2390 0x55f66b39dce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:55:56.555 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:56.552+0000 7fb2eab44740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fb2dc1af6e0 con 0x55f66afb19f0 2024-09-21T03:55:56.555 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:56.552+0000 7fb2eab44740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fb2dc1b1220 con 0x55f66afca1b0 2024-09-21T03:55:56.555 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:56.552+0000 7fb2eab44740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fb2dc1afdb0 con 0x55f66afd2390 2024-09-21T03:55:56.556 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:56.552+0000 7fb2e1b6b640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55f66afb19f0 0x55f66b3a6250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:55:56.556 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:56.552+0000 7fb2e1b6b640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55f66afb19f0 0x55f66b3a6250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:51904/0 (socket says 172.21.15.39:51904) 2024-09-21T03:55:56.556 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:56.552+0000 7fb2e1b6b640 1 -- 172.21.15.39:0/843844456 learned_addr learned my addr 172.21.15.39:0/843844456 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:55:56.556 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:56.553+0000 7fb2e236c640 1 --2- 172.21.15.39:0/843844456 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f66afca1b0 0x55f66b39d7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:55:56.556 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:56.553+0000 7fb2e1b6b640 1 --2- 172.21.15.39:0/843844456 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f66afd2390 0x55f66b39dce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:55:56.556 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:56.553+0000 7fb2e1b6b640 1 -- 172.21.15.39:0/843844456 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f66afd2390 msgr2=0x55f66b39dce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:55:56.556 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:56.553+0000 7fb2e1b6b640 1 --2- 172.21.15.39:0/843844456 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f66afd2390 0x55f66b39dce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:56.556 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:56.553+0000 7fb2e1b6b640 1 -- 172.21.15.39:0/843844456 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f66afca1b0 msgr2=0x55f66b39d7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:55:56.556 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:56.553+0000 7fb2e1b6b640 1 --2- 172.21.15.39:0/843844456 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f66afca1b0 0x55f66b39d7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:56.556 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:56.553+0000 7fb2e1b6b640 1 -- 172.21.15.39:0/843844456 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f66b3a1620 con 0x55f66afb19f0 2024-09-21T03:55:56.556 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:56.553+0000 7fb2e1b6b640 1 --2- 172.21.15.39:0/843844456 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55f66afb19f0 0x55f66b3a6250 secure :-1 s=READY pgs=1247 cs=0 l=1 rev1=1 crypto rx=0x7fb2cc00b9f0 tx=0x7fb2cc00b940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c9479b3c5325f24b server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:55:56.556 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:56.553+0000 7fb2c7fff640 1 -- 172.21.15.39:0/843844456 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2cc02f070 con 0x55f66afb19f0 2024-09-21T03:55:56.557 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:56.553+0000 7fb2c7fff640 1 -- 172.21.15.39:0/843844456 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb2cc029480 con 0x55f66afb19f0 2024-09-21T03:55:56.557 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:56.553+0000 7fb2c7fff640 1 -- 172.21.15.39:0/843844456 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2cc0057c0 con 0x55f66afb19f0 2024-09-21T03:55:56.561 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:56.554+0000 7fb2eab44740 1 -- 172.21.15.39:0/843844456 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55f66afb19f0 msgr2=0x55f66b3a6250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:55:56.561 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:56.554+0000 7fb2eab44740 1 --2- 172.21.15.39:0/843844456 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55f66afb19f0 0x55f66b3a6250 secure :-1 s=READY pgs=1247 cs=0 l=1 rev1=1 crypto rx=0x7fb2cc00b9f0 tx=0x7fb2cc00b940 comp rx=0 tx=0).stop 2024-09-21T03:55:56.561 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:56.554+0000 7fb2eab44740 1 -- 172.21.15.39:0/843844456 shutdown_connections 2024-09-21T03:55:56.561 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:56.554+0000 7fb2eab44740 1 --2- 172.21.15.39:0/843844456 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f66afd2390 0x55f66b39dce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:56.561 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:56.554+0000 7fb2eab44740 1 --2- 172.21.15.39:0/843844456 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f66afca1b0 0x55f66b39d7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:56.561 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:56.554+0000 7fb2eab44740 1 --2- 172.21.15.39:0/843844456 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55f66afb19f0 0x55f66b3a6250 unknown :-1 s=CLOSED pgs=1247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:56.561 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:56.554+0000 7fb2eab44740 1 -- 172.21.15.39:0/843844456 >> 172.21.15.39:0/843844456 conn(0x55f66afe7460 msgr2=0x55f66b3a5b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:55:56.561 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:56.554+0000 7fb2eab44740 1 -- 172.21.15.39:0/843844456 shutdown_connections 2024-09-21T03:55:56.561 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:56.554+0000 7fb2eab44740 1 -- 172.21.15.39:0/843844456 wait complete. 2024-09-21T03:55:56.561 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:56.554+0000 7fb2eab44740 1 -- start start 2024-09-21T03:55:56.562 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:56.555+0000 7fb2eab44740 1 Processor -- start 2024-09-21T03:55:56.562 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:56.555+0000 7fb2eab44740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55f66afb19f0 0x55f66b3aa2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:55:56.562 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:56.555+0000 7fb2eab44740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f66afca1b0 0x55f66b3aa7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:55:56.562 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:56.555+0000 7fb2eab44740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f66afd2390 0x55f66b3aad30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:55:56.562 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:56.555+0000 7fb2eab44740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fb2dc1b0b50 con 0x55f66afb19f0 2024-09-21T03:55:56.562 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:56.555+0000 7fb2eab44740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fb2dc1b0480 con 0x55f66afd2390 2024-09-21T03:55:56.562 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:56.555+0000 7fb2eab44740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fb2dc1ad6c0 con 0x55f66afca1b0 2024-09-21T03:55:56.562 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:56.555+0000 7fb2eab44740 10 client.0 _ll_register_callbacks cb 0x7fb2cc0042a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:55:56.562 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:56.555+0000 7fb2e1b6b640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55f66afb19f0 0x55f66b3aa2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:55:56.562 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:56.555+0000 7fb2e1b6b640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55f66afb19f0 0x55f66b3aa2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:51906/0 (socket says 172.21.15.39:51906) 2024-09-21T03:55:56.562 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:56.555+0000 7fb2e1b6b640 1 -- 172.21.15.39:0/2653223253 learned_addr learned my addr 172.21.15.39:0/2653223253 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:55:56.562 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:56.555+0000 7fb2e236c640 1 --2- 172.21.15.39:0/2653223253 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f66afca1b0 0x55f66b3aa7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:55:56.562 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:56.555+0000 7fb2e1b6b640 1 --2- 172.21.15.39:0/2653223253 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f66afd2390 0x55f66b3aad30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:55:56.562 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:56.555+0000 7fb2e1b6b640 1 -- 172.21.15.39:0/2653223253 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f66afca1b0 msgr2=0x55f66b3aa7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:55:56.563 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:56.555+0000 7fb2e1b6b640 1 --2- 172.21.15.39:0/2653223253 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f66afca1b0 0x55f66b3aa7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:56.563 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:56.555+0000 7fb2e1b6b640 1 -- 172.21.15.39:0/2653223253 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f66afd2390 msgr2=0x55f66b3aad30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:55:56.563 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:56.555+0000 7fb2e1b6b640 1 --2- 172.21.15.39:0/2653223253 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f66afd2390 0x55f66b3aad30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:56.563 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:56.555+0000 7fb2e1b6b640 1 -- 172.21.15.39:0/2653223253 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f66b421bf0 con 0x55f66afb19f0 2024-09-21T03:55:56.563 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:56.556+0000 7fb2e1b6b640 1 --2- 172.21.15.39:0/2653223253 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55f66afb19f0 0x55f66b3aa2b0 secure :-1 s=READY pgs=1248 cs=0 l=1 rev1=1 crypto rx=0x7fb2cc000c00 tx=0x7fb2cc029670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=65bf161f318ff292 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:55:56.563 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:56.556+0000 7fb2c5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-09-21T03:55:56.563 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:56.556+0000 7fb2c5ffb640 1 -- 172.21.15.39:0/2653223253 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2cc03c020 con 0x55f66afb19f0 2024-09-21T03:55:56.563 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:56.556+0000 7fb2c5ffb640 1 -- 172.21.15.39:0/2653223253 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8528+0+0 (secure 0 0 0) 0x7fb2cc012830 con 0x55f66afb19f0 2024-09-21T03:55:56.563 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:56.556+0000 7fb2eab44740 1 -- 172.21.15.39:0/2653223253 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb2cc029480 con 0x55f66afb19f0 2024-09-21T03:55:56.563 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:56.556+0000 7fb2c5ffb640 1 -- 172.21.15.39:0/2653223253 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fb2cc02f040 con 0x55f66afb19f0 2024-09-21T03:55:56.563 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:56.556+0000 7fb2eab44740 10 client.18048 fetch_fsmap learned FSMap version 29 2024-09-21T03:55:56.563 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:56.556+0000 7fb2eab44740 1 -- 172.21.15.39:0/2653223253 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55f66b422e50 con 0x55f66afb19f0 2024-09-21T03:55:56.563 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:56.556+0000 7fb2c5ffb640 1 -- 172.21.15.39:0/2653223253 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb2cc029480 con 0x55f66afb19f0 2024-09-21T03:55:56.563 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:56.556+0000 7fb2c5ffb640 1 -- 172.21.15.39:0/2653223253 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fb2cc034070 con 0x55f66afb19f0 2024-09-21T03:55:56.564 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:56.556+0000 7fb2eab44740 10 client.18048 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T03:55:56.564 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:56.556+0000 7fb2eab44740 10 client.18048 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-21T03:55:56.564 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:56.556+0000 7fb2eab44740 20 client.18048 populate_metadata read hostname 'smithi039' 2024-09-21T03:55:56.564 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:56.556+0000 7fb2eab44740 1 --2- 172.21.15.39:0/2653223253 >> [v2:172.21.15.161:6836/3451544597,v1:172.21.15.161:6837/3451544597] conn(0x55f66aff3070 0x55f66b425450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:55:56.564 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:56.557+0000 7fb2eab44740 4 client.18048 mds_command: new command op to 24646 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T03:55:56.564 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:56.557+0000 7fb2eab44740 1 -- 172.21.15.39:0/2653223253 --> [v2:172.21.15.161:6836/3451544597,v1:172.21.15.161:6837/3451544597] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55f66b30e8a0 con 0x55f66aff3070 2024-09-21T03:55:56.564 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:56.557+0000 7fb2e236c640 1 --2- 172.21.15.39:0/2653223253 >> [v2:172.21.15.161:6836/3451544597,v1:172.21.15.161:6837/3451544597] conn(0x55f66aff3070 0x55f66b425450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:55:56.564 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:56.557+0000 7fb2e236c640 1 --2- 172.21.15.39:0/2653223253 >> [v2:172.21.15.161:6836/3451544597,v1:172.21.15.161:6837/3451544597] conn(0x55f66aff3070 0x55f66b425450 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=47ad8bcd86b5e206 server_cookie=ba0ef95ef1f27e80 in_seq=0 out_seq=0 2024-09-21T03:55:56.564 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:56.557+0000 7fb2c5ffb640 10 client.18048 ms_handle_connect on v2:172.21.15.161:6836/3451544597 2024-09-21T03:55:56.564 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:56.558+0000 7fb2c5ffb640 1 -- 172.21.15.39:0/2653223253 <== mds.3 v2:172.21.15.161:6836/3451544597 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55f66b30e8a0 con 0x55f66aff3070 2024-09-21T03:55:56.564 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:56.558+0000 7fb2c5ffb640 10 client.18048 handle_command_reply: tid=0 2024-09-21T03:55:56.564 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:56.558+0000 7fb2eab44740 1 client.18048 shutdown 2024-09-21T03:55:56.564 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:56.558+0000 7fb2eab44740 1 -- 172.21.15.39:0/2653223253 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55f66afb19f0 msgr2=0x55f66b3aa2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:55:56.564 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:56.558+0000 7fb2eab44740 1 --2- 172.21.15.39:0/2653223253 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55f66afb19f0 0x55f66b3aa2b0 secure :-1 s=READY pgs=1248 cs=0 l=1 rev1=1 crypto rx=0x7fb2cc000c00 tx=0x7fb2cc029670 comp rx=0 tx=0).stop 2024-09-21T03:55:56.565 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:56.558+0000 7fb2eab44740 1 -- 172.21.15.39:0/2653223253 shutdown_connections 2024-09-21T03:55:56.565 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:56.558+0000 7fb2eab44740 1 --2- 172.21.15.39:0/2653223253 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f66afd2390 0x55f66b3aad30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:56.565 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:56.558+0000 7fb2eab44740 1 --2- 172.21.15.39:0/2653223253 >> [v2:172.21.15.161:6836/3451544597,v1:172.21.15.161:6837/3451544597] conn(0x55f66aff3070 0x55f66b425450 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:56.565 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:56.558+0000 7fb2eab44740 1 --2- 172.21.15.39:0/2653223253 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f66afca1b0 0x55f66b3aa7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:56.565 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:56.558+0000 7fb2eab44740 1 --2- 172.21.15.39:0/2653223253 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55f66afb19f0 0x55f66b3aa2b0 unknown :-1 s=CLOSED pgs=1248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:56.565 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:56.558+0000 7fb2eab44740 1 -- 172.21.15.39:0/2653223253 >> 172.21.15.39:0/2653223253 conn(0x55f66afe7460 msgr2=0x55f66b39d7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:55:56.565 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:56.558+0000 7fb2c5ffb640 0 client.18048 ms_handle_reset on v2:172.21.15.161:6836/3451544597 2024-09-21T03:55:56.565 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:56.558+0000 7fb2eab44740 1 -- 172.21.15.39:0/2653223253 shutdown_connections 2024-09-21T03:55:56.565 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:56.558+0000 7fb2eab44740 1 -- 172.21.15.39:0/2653223253 wait complete. 2024-09-21T03:55:56.565 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:56.558+0000 7fb2eab44740 20 client.18048 trim_cache size 0 max 16384 2024-09-21T03:55:56.565 INFO:teuthology.orchestra.run.smithi039.stdout:[] 2024-09-21T03:55:56.566 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:56.558+0000 7fb2e45d0640 1 -- 172.21.15.39:0/3127144692 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7fb2c80b66b0 msgr2=0x7fb2c80b8bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:55:56.566 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:56.558+0000 7fb2e45d0640 1 --2- 172.21.15.39:0/3127144692 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7fb2c80b66b0 0x7fb2c80b8bc0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7fb2d40ad610 tx=0x7fb2d400ff50 comp rx=0 tx=0).stop 2024-09-21T03:55:56.566 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:56.558+0000 7fb2e45d0640 1 -- 172.21.15.39:0/3127144692 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb2dc109df0 msgr2=0x7fb2dc19fd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:55:56.566 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:56.558+0000 7fb2e45d0640 1 --2- 172.21.15.39:0/3127144692 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb2dc109df0 0x7fb2dc19fd40 secure :-1 s=READY pgs=1244 cs=0 l=1 rev1=1 crypto rx=0x7fb2d400cce0 tx=0x7fb2d400a620 comp rx=0 tx=0).stop 2024-09-21T03:55:56.566 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:56.559+0000 7fb2e45d0640 1 -- 172.21.15.39:0/3127144692 shutdown_connections 2024-09-21T03:55:56.567 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:56.559+0000 7fb2e45d0640 1 --2- 172.21.15.39:0/3127144692 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7fb2c80b66b0 0x7fb2c80b8bc0 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:56.567 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:56.559+0000 7fb2e45d0640 1 --2- 172.21.15.39:0/3127144692 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb2dc109df0 0x7fb2dc19fd40 unknown :-1 s=CLOSED pgs=1244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:56.567 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:56.559+0000 7fb2e45d0640 1 --2- 172.21.15.39:0/3127144692 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb2dc1094a0 0x7fb2dc19aaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:56.567 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:56.559+0000 7fb2e45d0640 1 --2- 172.21.15.39:0/3127144692 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb2dc103b60 0x7fb2dc19a500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:56.567 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:56.559+0000 7fb2e45d0640 1 -- 172.21.15.39:0/3127144692 >> 172.21.15.39:0/3127144692 conn(0x7fb2dc07b850 msgr2=0x7fb2dc1075e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:55:56.567 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:56.559+0000 7fb2e45d0640 1 -- 172.21.15.39:0/3127144692 shutdown_connections 2024-09-21T03:55:56.567 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:56.559+0000 7fb2e45d0640 1 -- 172.21.15.39:0/3127144692 wait complete. 2024-09-21T03:55:57.334 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:55:57 smithi039 ceph-mon[31211]: pgmap v628: 97 pgs: 97 active+clean; 10 MiB data, 525 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 10 op/s 2024-09-21T03:55:57.392 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 51116d5c-77ca-11ef-baf5-efdc52797490 -- ceph tell mds.1:2 damage ls 2024-09-21T03:55:57.468 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:55:57 smithi161 ceph-mon[34533]: pgmap v628: 97 pgs: 97 active+clean; 10 MiB data, 525 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 10 op/s 2024-09-21T03:55:57.474 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:55:57 smithi150 ceph-mon[34289]: pgmap v628: 97 pgs: 97 active+clean; 10 MiB data, 525 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 10 op/s 2024-09-21T03:55:57.658 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/51116d5c-77ca-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:55:58.164 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:58.161+0000 7f6d0ebd4640 1 -- 172.21.15.39:0/1026031448 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6d08108d10 msgr2=0x7f6d0810f540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:55:58.164 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:58.161+0000 7f6d0ebd4640 1 --2- 172.21.15.39:0/1026031448 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6d08108d10 0x7f6d0810f540 secure :-1 s=READY pgs=1249 cs=0 l=1 rev1=1 crypto rx=0x7f6d00009ca0 tx=0x7f6d0002a8c0 comp rx=0 tx=0).stop 2024-09-21T03:55:58.164 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:58.161+0000 7f6d0ebd4640 1 -- 172.21.15.39:0/1026031448 shutdown_connections 2024-09-21T03:55:58.164 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:58.161+0000 7f6d0ebd4640 1 --2- 172.21.15.39:0/1026031448 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6d08108d10 0x7f6d0810f540 unknown :-1 s=CLOSED pgs=1249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:58.164 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:58.161+0000 7f6d0ebd4640 1 --2- 172.21.15.39:0/1026031448 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6d081083c0 0x7f6d081087d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:58.164 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:58.161+0000 7f6d0ebd4640 1 --2- 172.21.15.39:0/1026031448 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6d08102a80 0x7f6d08102e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:58.164 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:58.161+0000 7f6d0ebd4640 1 -- 172.21.15.39:0/1026031448 >> 172.21.15.39:0/1026031448 conn(0x7f6d08079740 msgr2=0x7f6d08079b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:55:58.164 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:58.161+0000 7f6d0ebd4640 1 -- 172.21.15.39:0/1026031448 shutdown_connections 2024-09-21T03:55:58.164 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:58.161+0000 7f6d0ebd4640 1 -- 172.21.15.39:0/1026031448 wait complete. 2024-09-21T03:55:58.166 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:58.162+0000 7f6d0ebd4640 1 Processor -- start 2024-09-21T03:55:58.166 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:58.162+0000 7f6d0ebd4640 1 -- start start 2024-09-21T03:55:58.166 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:58.162+0000 7f6d0ebd4640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6d08102a80 0x7f6d0819b2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:55:58.166 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:58.162+0000 7f6d0ebd4640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6d081083c0 0x7f6d0819b830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:55:58.166 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:58.162+0000 7f6d0ebd4640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6d08108d10 0x7f6d0819bd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:55:58.166 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:58.162+0000 7f6d0ebd4640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f6d08112980 con 0x7f6d08108d10 2024-09-21T03:55:58.166 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:58.162+0000 7f6d0ebd4640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f6d08112800 con 0x7f6d081083c0 2024-09-21T03:55:58.167 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:58.162+0000 7f6d0ebd4640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f6d08112b00 con 0x7f6d08102a80 2024-09-21T03:55:58.167 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:58.162+0000 7f6d0d171640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6d08108d10 0x7f6d0819bd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:55:58.167 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:58.162+0000 7f6d0d171640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6d08108d10 0x7f6d0819bd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:33204/0 (socket says 172.21.15.39:33204) 2024-09-21T03:55:58.167 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:58.162+0000 7f6d0d171640 1 -- 172.21.15.39:0/3299861897 learned_addr learned my addr 172.21.15.39:0/3299861897 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:55:58.167 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:58.162+0000 7f6d0c970640 1 --2- 172.21.15.39:0/3299861897 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6d08102a80 0x7f6d0819b2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:55:58.167 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:58.162+0000 7f6d07fff640 1 --2- 172.21.15.39:0/3299861897 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6d081083c0 0x7f6d0819b830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:55:58.167 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:58.162+0000 7f6d0d171640 1 -- 172.21.15.39:0/3299861897 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6d08102a80 msgr2=0x7f6d0819b2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:55:58.167 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:58.162+0000 7f6d0d171640 1 --2- 172.21.15.39:0/3299861897 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6d08102a80 0x7f6d0819b2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:58.167 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:58.162+0000 7f6d0d171640 1 -- 172.21.15.39:0/3299861897 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6d081083c0 msgr2=0x7f6d0819b830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:55:58.167 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:58.162+0000 7f6d0d171640 1 --2- 172.21.15.39:0/3299861897 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6d081083c0 0x7f6d0819b830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:58.167 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:58.162+0000 7f6d0d171640 1 -- 172.21.15.39:0/3299861897 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6d081a6580 con 0x7f6d08108d10 2024-09-21T03:55:58.167 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:58.163+0000 7f6d0c970640 1 --2- 172.21.15.39:0/3299861897 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6d08102a80 0x7f6d0819b2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:55:58.167 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:58.163+0000 7f6d0d171640 1 --2- 172.21.15.39:0/3299861897 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6d08108d10 0x7f6d0819bd70 secure :-1 s=READY pgs=1250 cs=0 l=1 rev1=1 crypto rx=0x7f6d000098d0 tx=0x7f6d000317b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:55:58.167 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:58.163+0000 7f6d05ffb640 1 -- 172.21.15.39:0/3299861897 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d00003b70 con 0x7f6d08108d10 2024-09-21T03:55:58.167 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:58.163+0000 7f6d05ffb640 1 -- 172.21.15.39:0/3299861897 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6d00003d10 con 0x7f6d08108d10 2024-09-21T03:55:58.167 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:58.163+0000 7f6d05ffb640 1 -- 172.21.15.39:0/3299861897 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d0002fa20 con 0x7f6d08108d10 2024-09-21T03:55:58.168 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:58.163+0000 7f6d0ebd4640 1 -- 172.21.15.39:0/3299861897 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6d081a6810 con 0x7f6d08108d10 2024-09-21T03:55:58.168 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:58.163+0000 7f6d0ebd4640 1 -- 172.21.15.39:0/3299861897 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6d081a6da0 con 0x7f6d08108d10 2024-09-21T03:55:58.168 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:58.166+0000 7f6d15148740 1 Processor -- start 2024-09-21T03:55:58.169 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:58.166+0000 7f6d05ffb640 1 -- 172.21.15.39:0/3299861897 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 27) ==== 154643+0+0 (secure 0 0 0) 0x7f6d0002fbc0 con 0x7f6d08108d10 2024-09-21T03:55:58.169 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:58.166+0000 7f6d15148740 1 -- start start 2024-09-21T03:55:58.170 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:58.166+0000 7f6d15148740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b18cb64070 0x55b18cf0b290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:55:58.170 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:58.166+0000 7f6d15148740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b18cb3b1b0 0x55b18cf0b7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:55:58.170 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:58.167+0000 7f6d15148740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b18cb43390 0x55b18cf0bd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:55:58.171 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:58.167+0000 7f6d15148740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x55b18ce81ea0 con 0x55b18cb43390 2024-09-21T03:55:58.171 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:58.167+0000 7f6d15148740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x55b18ce712d0 con 0x55b18cb3b1b0 2024-09-21T03:55:58.171 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:58.167+0000 7f6d15148740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55b18ce750e0 con 0x55b18cb64070 2024-09-21T03:55:58.171 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:58.167+0000 7f6d07fff640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b18cb43390 0x55b18cf0bd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:55:58.171 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:58.167+0000 7f6d07fff640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b18cb43390 0x55b18cf0bd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:33208/0 (socket says 172.21.15.39:33208) 2024-09-21T03:55:58.171 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:58.167+0000 7f6d07fff640 1 -- 172.21.15.39:0/1484953562 learned_addr learned my addr 172.21.15.39:0/1484953562 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:55:58.171 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:58.167+0000 7f6d07fff640 1 --2- 172.21.15.39:0/1484953562 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b18cb64070 0x55b18cf0b290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:55:58.171 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:58.167+0000 7f6d0c970640 1 --2- 172.21.15.39:0/1484953562 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b18cb3b1b0 0x55b18cf0b7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:55:58.171 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:58.167+0000 7f6d05ffb640 1 --2- 172.21.15.39:0/3299861897 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f6cf00b6730 0x7f6cf00b8c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:55:58.171 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:58.167+0000 7f6d07fff640 1 -- 172.21.15.39:0/1484953562 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b18cb64070 msgr2=0x55b18cf0b290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:55:58.171 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:58.167+0000 7f6d07fff640 1 --2- 172.21.15.39:0/1484953562 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b18cb64070 0x55b18cf0b290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:58.171 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:58.167+0000 7f6d07fff640 1 -- 172.21.15.39:0/1484953562 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b18cb3b1b0 msgr2=0x55b18cf0b7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:55:58.171 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:58.167+0000 7f6d07fff640 1 --2- 172.21.15.39:0/1484953562 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b18cb3b1b0 0x55b18cf0b7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:58.171 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:58.167+0000 7f6d07fff640 1 -- 172.21.15.39:0/1484953562 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b18cbcf650 con 0x55b18cb43390 2024-09-21T03:55:58.172 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:58.167+0000 7f6d0d171640 1 --2- 172.21.15.39:0/3299861897 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f6cf00b6730 0x7f6cf00b8c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:55:58.172 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:58.167+0000 7f6d05ffb640 1 -- 172.21.15.39:0/3299861897 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8528+0+0 (secure 0 0 0) 0x7f6d00108a00 con 0x7f6d08108d10 2024-09-21T03:55:58.172 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:58.167+0000 7f6d07fff640 1 --2- 172.21.15.39:0/1484953562 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b18cb43390 0x55b18cf0bd10 secure :-1 s=READY pgs=1251 cs=0 l=1 rev1=1 crypto rx=0x7f6cfc0138e0 tx=0x7f6cfc00c610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ce99d67a7158070 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:55:58.172 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:58.168+0000 7f6ceeffd640 1 -- 172.21.15.39:0/1484953562 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6cfc005220 con 0x55b18cb43390 2024-09-21T03:55:58.172 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:58.168+0000 7f6ceeffd640 1 -- 172.21.15.39:0/1484953562 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6cfc004c50 con 0x55b18cb43390 2024-09-21T03:55:58.172 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:58.168+0000 7f6ceeffd640 1 -- 172.21.15.39:0/1484953562 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6cfc0119e0 con 0x55b18cb43390 2024-09-21T03:55:58.175 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:58.169+0000 7f6d15148740 1 -- 172.21.15.39:0/1484953562 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b18cb43390 msgr2=0x55b18cf0bd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:55:58.175 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:58.169+0000 7f6d0d171640 1 --2- 172.21.15.39:0/3299861897 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f6cf00b6730 0x7f6cf00b8c40 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f6d000cbac0 tx=0x7f6d000983b0 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:55:58.175 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:58.169+0000 7f6d15148740 1 --2- 172.21.15.39:0/1484953562 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b18cb43390 0x55b18cf0bd10 secure :-1 s=READY pgs=1251 cs=0 l=1 rev1=1 crypto rx=0x7f6cfc0138e0 tx=0x7f6cfc00c610 comp rx=0 tx=0).stop 2024-09-21T03:55:58.175 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:58.169+0000 7f6d15148740 1 -- 172.21.15.39:0/1484953562 shutdown_connections 2024-09-21T03:55:58.175 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:58.169+0000 7f6d15148740 1 --2- 172.21.15.39:0/1484953562 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b18cb43390 0x55b18cf0bd10 unknown :-1 s=CLOSED pgs=1251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:58.175 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:58.169+0000 7f6d15148740 1 --2- 172.21.15.39:0/1484953562 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b18cb3b1b0 0x55b18cf0b7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:58.175 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:58.169+0000 7f6d15148740 1 --2- 172.21.15.39:0/1484953562 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b18cb64070 0x55b18cf0b290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:58.175 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:58.169+0000 7f6d15148740 1 -- 172.21.15.39:0/1484953562 >> 172.21.15.39:0/1484953562 conn(0x55b18cb58460 msgr2=0x55b18cf09180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:55:58.176 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:58.169+0000 7f6d15148740 1 -- 172.21.15.39:0/1484953562 shutdown_connections 2024-09-21T03:55:58.176 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:58.169+0000 7f6d15148740 1 -- 172.21.15.39:0/1484953562 wait complete. 2024-09-21T03:55:58.176 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:58.169+0000 7f6d15148740 1 -- start start 2024-09-21T03:55:58.176 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:58.169+0000 7f6d15148740 1 Processor -- start 2024-09-21T03:55:58.176 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:58.170+0000 7f6d15148740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b18cb3b1b0 0x55b18cf16950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:55:58.176 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:58.170+0000 7f6d15148740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b18cb43390 0x55b18cf16e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:55:58.176 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:58.170+0000 7f6d15148740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b18cb64070 0x55b18cf22d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:55:58.176 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:58.170+0000 7f6d15148740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x55b18ccd2700 con 0x55b18cb64070 2024-09-21T03:55:58.176 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:58.170+0000 7f6d15148740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x55b18ce0c160 con 0x55b18cb43390 2024-09-21T03:55:58.176 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:58.170+0000 7f6d15148740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55b18cf19a70 con 0x55b18cb3b1b0 2024-09-21T03:55:58.176 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:58.170+0000 7f6d15148740 10 client.0 _ll_register_callbacks cb 0x55b18ce259e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:55:58.176 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:58.170+0000 7f6d07fff640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b18cb64070 0x55b18cf22d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:55:58.176 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:58.170+0000 7f6d07fff640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b18cb64070 0x55b18cf22d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:33212/0 (socket says 172.21.15.39:33212) 2024-09-21T03:55:58.176 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:58.170+0000 7f6d07fff640 1 -- 172.21.15.39:0/2321854473 learned_addr learned my addr 172.21.15.39:0/2321854473 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:55:58.177 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:58.170+0000 7f6d0c970640 1 --2- 172.21.15.39:0/2321854473 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b18cb43390 0x55b18cf16e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:55:58.177 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:58.170+0000 7f6d07fff640 1 --2- 172.21.15.39:0/2321854473 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b18cb3b1b0 0x55b18cf16950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:55:58.177 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:58.170+0000 7f6d07fff640 1 -- 172.21.15.39:0/2321854473 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b18cb3b1b0 msgr2=0x55b18cf16950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:55:58.177 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:58.170+0000 7f6d07fff640 1 --2- 172.21.15.39:0/2321854473 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b18cb3b1b0 0x55b18cf16950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:58.177 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:58.170+0000 7f6d07fff640 1 -- 172.21.15.39:0/2321854473 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b18cb43390 msgr2=0x55b18cf16e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:55:58.177 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:58.170+0000 7f6d07fff640 1 --2- 172.21.15.39:0/2321854473 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b18cb43390 0x55b18cf16e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:58.177 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:58.170+0000 7f6d07fff640 1 -- 172.21.15.39:0/2321854473 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b18ce39c90 con 0x55b18cb64070 2024-09-21T03:55:58.177 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:58.170+0000 7f6d07fff640 1 --2- 172.21.15.39:0/2321854473 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b18cb64070 0x55b18cf22d60 secure :-1 s=READY pgs=1252 cs=0 l=1 rev1=1 crypto rx=0x7f6cfc004630 tx=0x7f6cfc005460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b86877124323ad32 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:55:58.177 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:58.170+0000 7f6ce7fff640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-09-21T03:55:58.177 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:58.171+0000 7f6ce7fff640 1 -- 172.21.15.39:0/2321854473 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6cfc023030 con 0x55b18cb64070 2024-09-21T03:55:58.177 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:58.171+0000 7f6ce7fff640 1 -- 172.21.15.39:0/2321854473 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8528+0+0 (secure 0 0 0) 0x7f6cfc028070 con 0x55b18cb64070 2024-09-21T03:55:58.177 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:58.171+0000 7f6d15148740 1 -- 172.21.15.39:0/2321854473 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6cfc004c50 con 0x55b18cb64070 2024-09-21T03:55:58.177 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:58.171+0000 7f6ce7fff640 1 -- 172.21.15.39:0/2321854473 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f6cfc004e00 con 0x55b18cb64070 2024-09-21T03:55:58.177 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:58.171+0000 7f6d15148740 10 client.18060 fetch_fsmap learned FSMap version 29 2024-09-21T03:55:58.178 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:58.171+0000 7f6d15148740 1 -- 172.21.15.39:0/2321854473 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55b18cf25780 con 0x55b18cb64070 2024-09-21T03:55:58.178 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:58.171+0000 7f6ce7fff640 1 -- 172.21.15.39:0/2321854473 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6cfc004c50 con 0x55b18cb64070 2024-09-21T03:55:58.178 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:58.171+0000 7f6ce7fff640 1 -- 172.21.15.39:0/2321854473 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f6cfc0304c0 con 0x55b18cb64070 2024-09-21T03:55:58.178 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:58.171+0000 7f6d15148740 10 client.18060 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T03:55:58.178 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:58.171+0000 7f6d15148740 10 client.18060 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-21T03:55:58.178 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:58.171+0000 7f6d15148740 20 client.18060 populate_metadata read hostname 'smithi039' 2024-09-21T03:55:58.178 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:58.171+0000 7f6d15148740 1 --2- 172.21.15.39:0/2321854473 >> [v2:172.21.15.150:6836/4017043144,v1:172.21.15.150:6837/4017043144] conn(0x55b18cb229f0 0x55b18cf28ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:55:58.178 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:58.171+0000 7f6d15148740 4 client.18060 mds_command: new command op to 24659 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:55:58.178 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:58.171+0000 7f6d15148740 1 -- 172.21.15.39:0/2321854473 --> [v2:172.21.15.150:6836/4017043144,v1:172.21.15.150:6837/4017043144] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b18ce83df0 con 0x55b18cb229f0 2024-09-21T03:55:58.178 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:58.172+0000 7f6d0c970640 1 --2- 172.21.15.39:0/2321854473 >> [v2:172.21.15.150:6836/4017043144,v1:172.21.15.150:6837/4017043144] conn(0x55b18cb229f0 0x55b18cf28ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:55:58.178 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:58.172+0000 7f6ce7fff640 10 client.18060 ms_handle_connect on v2:172.21.15.150:6836/4017043144 2024-09-21T03:55:58.178 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:58.172+0000 7f6d0c970640 1 --2- 172.21.15.39:0/2321854473 >> [v2:172.21.15.150:6836/4017043144,v1:172.21.15.150:6837/4017043144] conn(0x55b18cb229f0 0x55b18cf28ae0 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=601688874e6236d4 server_cookie=6d7783f0cd122f32 in_seq=0 out_seq=0 2024-09-21T03:55:58.179 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:58.173+0000 7f6ce7fff640 1 -- 172.21.15.39:0/2321854473 <== mds.2 v2:172.21.15.150:6836/4017043144 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55b18ce83df0 con 0x55b18cb229f0 2024-09-21T03:55:58.181 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:58.173+0000 7f6ce7fff640 10 client.18060 handle_command_reply: tid=0 2024-09-21T03:55:58.181 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:58.173+0000 7f6d15148740 1 client.18060 shutdown 2024-09-21T03:55:58.181 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:58.173+0000 7f6d15148740 1 -- 172.21.15.39:0/2321854473 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b18cb64070 msgr2=0x55b18cf22d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:55:58.181 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:58.173+0000 7f6d15148740 1 --2- 172.21.15.39:0/2321854473 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b18cb64070 0x55b18cf22d60 secure :-1 s=READY pgs=1252 cs=0 l=1 rev1=1 crypto rx=0x7f6cfc004630 tx=0x7f6cfc005460 comp rx=0 tx=0).stop 2024-09-21T03:55:58.181 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:58.173+0000 7f6d15148740 1 -- 172.21.15.39:0/2321854473 shutdown_connections 2024-09-21T03:55:58.181 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:58.173+0000 7f6d15148740 1 --2- 172.21.15.39:0/2321854473 >> [v2:172.21.15.150:6836/4017043144,v1:172.21.15.150:6837/4017043144] conn(0x55b18cb229f0 0x55b18cf28ae0 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:58.181 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:58.173+0000 7f6d15148740 1 --2- 172.21.15.39:0/2321854473 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b18cb64070 0x55b18cf22d60 unknown :-1 s=CLOSED pgs=1252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:58.181 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:58.173+0000 7f6d15148740 1 --2- 172.21.15.39:0/2321854473 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b18cb43390 0x55b18cf16e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:58.181 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:58.173+0000 7f6d15148740 1 --2- 172.21.15.39:0/2321854473 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b18cb3b1b0 0x55b18cf16950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:58.181 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:58.173+0000 7f6d15148740 1 -- 172.21.15.39:0/2321854473 >> 172.21.15.39:0/2321854473 conn(0x55b18cb58460 msgr2=0x55b18cf0e420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:55:58.181 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:58.173+0000 7f6ce7fff640 0 client.18060 ms_handle_reset on v2:172.21.15.150:6836/4017043144 2024-09-21T03:55:58.181 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:58.173+0000 7f6d15148740 1 -- 172.21.15.39:0/2321854473 shutdown_connections 2024-09-21T03:55:58.181 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:58.173+0000 7f6d15148740 1 -- 172.21.15.39:0/2321854473 wait complete. 2024-09-21T03:55:58.181 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:58.174+0000 7f6d15148740 20 client.18060 trim_cache size 0 max 16384 2024-09-21T03:55:58.191 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:58.187+0000 7f6d15148740 1 Processor -- start 2024-09-21T03:55:58.191 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:58.187+0000 7f6d15148740 1 -- start start 2024-09-21T03:55:58.191 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:58.187+0000 7f6d15148740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b18cb3b1b0 0x55b18cf0cc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:55:58.191 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:58.187+0000 7f6d15148740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b18cb43390 0x55b18cf18580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:55:58.191 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:58.187+0000 7f6d15148740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b18cb64070 0x55b18cf22c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:55:58.191 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:58.187+0000 7f6d15148740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f6d081af800 con 0x55b18cb43390 2024-09-21T03:55:58.191 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:58.187+0000 7f6d15148740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f6d081b1340 con 0x55b18cb3b1b0 2024-09-21T03:55:58.191 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:58.187+0000 7f6d15148740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f6d081afed0 con 0x55b18cb64070 2024-09-21T03:55:58.191 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:58.187+0000 7f6d0c970640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b18cb43390 0x55b18cf18580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:55:58.191 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:58.187+0000 7f6d0c970640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b18cb43390 0x55b18cf18580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:33222/0 (socket says 172.21.15.39:33222) 2024-09-21T03:55:58.191 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:58.187+0000 7f6d0c970640 1 -- 172.21.15.39:0/3446906379 learned_addr learned my addr 172.21.15.39:0/3446906379 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:55:58.192 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:58.187+0000 7f6d07fff640 1 --2- 172.21.15.39:0/3446906379 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b18cb3b1b0 0x55b18cf0cc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:55:58.192 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:58.187+0000 7f6d07fff640 1 --2- 172.21.15.39:0/3446906379 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b18cb64070 0x55b18cf22c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:55:58.192 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:58.187+0000 7f6d0c970640 1 -- 172.21.15.39:0/3446906379 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b18cb64070 msgr2=0x55b18cf22c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:55:58.192 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:58.187+0000 7f6d0c970640 1 --2- 172.21.15.39:0/3446906379 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b18cb64070 0x55b18cf22c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:58.192 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:58.187+0000 7f6d0c970640 1 -- 172.21.15.39:0/3446906379 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b18cb3b1b0 msgr2=0x55b18cf0cc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:55:58.192 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:58.187+0000 7f6d0c970640 1 --2- 172.21.15.39:0/3446906379 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b18cb3b1b0 0x55b18cf0cc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:58.192 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:58.187+0000 7f6d0c970640 1 -- 172.21.15.39:0/3446906379 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b18cf17c10 con 0x55b18cb43390 2024-09-21T03:55:58.192 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:58.187+0000 7f6d07fff640 1 --2- 172.21.15.39:0/3446906379 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b18cb3b1b0 0x55b18cf0cc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:55:58.192 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:58.188+0000 7f6d0c970640 1 --2- 172.21.15.39:0/3446906379 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b18cb43390 0x55b18cf18580 secure :-1 s=READY pgs=1253 cs=0 l=1 rev1=1 crypto rx=0x7f6d0806c6a0 tx=0x7f6cf8014840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1422d594a55d10bb server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:55:58.192 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:58.188+0000 7f6cee7fc640 1 -- 172.21.15.39:0/3446906379 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6cf8014190 con 0x55b18cb43390 2024-09-21T03:55:58.192 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:58.188+0000 7f6cee7fc640 1 -- 172.21.15.39:0/3446906379 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6cf80054d0 con 0x55b18cb43390 2024-09-21T03:55:58.192 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:58.188+0000 7f6cee7fc640 1 -- 172.21.15.39:0/3446906379 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6cf801e690 con 0x55b18cb43390 2024-09-21T03:55:58.196 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:58.189+0000 7f6d15148740 1 -- 172.21.15.39:0/3446906379 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b18cb43390 msgr2=0x55b18cf18580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:55:58.196 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:58.189+0000 7f6d15148740 1 --2- 172.21.15.39:0/3446906379 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b18cb43390 0x55b18cf18580 secure :-1 s=READY pgs=1253 cs=0 l=1 rev1=1 crypto rx=0x7f6d0806c6a0 tx=0x7f6cf8014840 comp rx=0 tx=0).stop 2024-09-21T03:55:58.196 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:58.189+0000 7f6d15148740 1 -- 172.21.15.39:0/3446906379 shutdown_connections 2024-09-21T03:55:58.196 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:58.189+0000 7f6d15148740 1 --2- 172.21.15.39:0/3446906379 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b18cb64070 0x55b18cf22c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:58.196 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:58.189+0000 7f6d15148740 1 --2- 172.21.15.39:0/3446906379 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b18cb43390 0x55b18cf18580 unknown :-1 s=CLOSED pgs=1253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:58.196 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:58.189+0000 7f6d15148740 1 --2- 172.21.15.39:0/3446906379 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b18cb3b1b0 0x55b18cf0cc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:58.196 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:58.189+0000 7f6d15148740 1 -- 172.21.15.39:0/3446906379 >> 172.21.15.39:0/3446906379 conn(0x55b18cb58460 msgr2=0x55b18cf17e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:55:58.196 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:58.189+0000 7f6d15148740 1 -- 172.21.15.39:0/3446906379 shutdown_connections 2024-09-21T03:55:58.196 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:58.189+0000 7f6d15148740 1 -- 172.21.15.39:0/3446906379 wait complete. 2024-09-21T03:55:58.196 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:58.189+0000 7f6d15148740 1 -- start start 2024-09-21T03:55:58.197 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:58.189+0000 7f6d15148740 1 Processor -- start 2024-09-21T03:55:58.197 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:58.190+0000 7f6d15148740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b18cb3b1b0 0x55b18cf1acd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:55:58.197 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:58.190+0000 7f6d15148740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b18cb43390 0x55b18cf1b210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:55:58.197 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:58.190+0000 7f6d15148740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b18cb64070 0x55b18cf1b750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:55:58.197 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:58.190+0000 7f6d15148740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f6d081b0c70 con 0x55b18cb64070 2024-09-21T03:55:58.197 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:58.190+0000 7f6d15148740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f6d081b05a0 con 0x55b18cb3b1b0 2024-09-21T03:55:58.197 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:58.190+0000 7f6d15148740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f6d081ad7e0 con 0x55b18cb43390 2024-09-21T03:55:58.197 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:58.190+0000 7f6d15148740 10 client.0 _ll_register_callbacks cb 0x7f6cfc00cda0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:55:58.197 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:58.190+0000 7f6d07fff640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b18cb64070 0x55b18cf1b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:55:58.197 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:58.190+0000 7f6d07fff640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b18cb64070 0x55b18cf1b750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:33230/0 (socket says 172.21.15.39:33230) 2024-09-21T03:55:58.197 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:58.190+0000 7f6d07fff640 1 -- 172.21.15.39:0/4275049737 learned_addr learned my addr 172.21.15.39:0/4275049737 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:55:58.197 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:58.190+0000 7f6d0c970640 1 --2- 172.21.15.39:0/4275049737 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b18cb43390 0x55b18cf1b210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:55:58.197 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:58.190+0000 7f6d07fff640 1 --2- 172.21.15.39:0/4275049737 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b18cb3b1b0 0x55b18cf1acd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:55:58.197 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:58.190+0000 7f6d07fff640 1 -- 172.21.15.39:0/4275049737 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b18cb43390 msgr2=0x55b18cf1b210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:55:58.197 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:58.190+0000 7f6d07fff640 1 --2- 172.21.15.39:0/4275049737 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b18cb43390 0x55b18cf1b210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:58.198 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:58.190+0000 7f6d07fff640 1 -- 172.21.15.39:0/4275049737 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b18cb3b1b0 msgr2=0x55b18cf1acd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:55:58.198 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:58.190+0000 7f6d07fff640 1 --2- 172.21.15.39:0/4275049737 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b18cb3b1b0 0x55b18cf1acd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:58.198 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:58.190+0000 7f6d07fff640 1 -- 172.21.15.39:0/4275049737 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b18cf966f0 con 0x55b18cb64070 2024-09-21T03:55:58.198 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:58.190+0000 7f6d07fff640 1 --2- 172.21.15.39:0/4275049737 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b18cb64070 0x55b18cf1b750 secure :-1 s=READY pgs=1254 cs=0 l=1 rev1=1 crypto rx=0x7f6cfc00c670 tx=0x7f6cfc011af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f7269c19b30e3d2b server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:55:58.198 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:58.190+0000 7f6ce7fff640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-09-21T03:55:58.198 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:58.190+0000 7f6ce7fff640 1 -- 172.21.15.39:0/4275049737 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6cfc023040 con 0x55b18cb64070 2024-09-21T03:55:58.198 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:58.191+0000 7f6ce7fff640 1 -- 172.21.15.39:0/4275049737 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8528+0+0 (secure 0 0 0) 0x7f6cfc030ab0 con 0x55b18cb64070 2024-09-21T03:55:58.198 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:58.191+0000 7f6d15148740 1 -- 172.21.15.39:0/4275049737 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6cf80054d0 con 0x55b18cb64070 2024-09-21T03:55:58.198 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:58.191+0000 7f6ce7fff640 1 -- 172.21.15.39:0/4275049737 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f6cfc013580 con 0x55b18cb64070 2024-09-21T03:55:58.198 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:58.191+0000 7f6d15148740 10 client.18066 fetch_fsmap learned FSMap version 29 2024-09-21T03:55:58.198 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:58.191+0000 7f6d15148740 1 -- 172.21.15.39:0/4275049737 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55b18cf978e0 con 0x55b18cb64070 2024-09-21T03:55:58.198 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:58.191+0000 7f6ce7fff640 1 -- 172.21.15.39:0/4275049737 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6cf80054d0 con 0x55b18cb64070 2024-09-21T03:55:58.198 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:58.191+0000 7f6ce7fff640 1 -- 172.21.15.39:0/4275049737 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f6cfc013020 con 0x55b18cb64070 2024-09-21T03:55:58.198 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:58.191+0000 7f6d15148740 10 client.18066 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T03:55:58.199 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:58.191+0000 7f6d15148740 10 client.18066 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-21T03:55:58.199 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:58.191+0000 7f6d15148740 20 client.18066 populate_metadata read hostname 'smithi039' 2024-09-21T03:55:58.199 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:58.191+0000 7f6d15148740 1 --2- 172.21.15.39:0/4275049737 >> [v2:172.21.15.150:6836/4017043144,v1:172.21.15.150:6837/4017043144] conn(0x55b18caf7da0 0x55b18cf9a070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:55:58.199 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:58.191+0000 7f6d15148740 4 client.18066 mds_command: new command op to 24659 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T03:55:58.199 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:58.191+0000 7f6d15148740 1 -- 172.21.15.39:0/4275049737 --> [v2:172.21.15.150:6836/4017043144,v1:172.21.15.150:6837/4017043144] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55b18ce7f8a0 con 0x55b18caf7da0 2024-09-21T03:55:58.199 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:58.192+0000 7f6d0c970640 1 --2- 172.21.15.39:0/4275049737 >> [v2:172.21.15.150:6836/4017043144,v1:172.21.15.150:6837/4017043144] conn(0x55b18caf7da0 0x55b18cf9a070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:55:58.199 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:58.192+0000 7f6ce7fff640 10 client.18066 ms_handle_connect on v2:172.21.15.150:6836/4017043144 2024-09-21T03:55:58.199 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:58.192+0000 7f6d0c970640 1 --2- 172.21.15.39:0/4275049737 >> [v2:172.21.15.150:6836/4017043144,v1:172.21.15.150:6837/4017043144] conn(0x55b18caf7da0 0x55b18cf9a070 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=8270ea0b7c69db2b server_cookie=b5f0417288f34a69 in_seq=0 out_seq=0 2024-09-21T03:55:58.199 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:58.192+0000 7f6ce7fff640 1 -- 172.21.15.39:0/4275049737 <== mds.2 v2:172.21.15.150:6836/4017043144 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55b18ce7f8a0 con 0x55b18caf7da0 2024-09-21T03:55:58.199 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:58.192+0000 7f6ce7fff640 10 client.18066 handle_command_reply: tid=0 2024-09-21T03:55:58.199 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:58.192+0000 7f6d15148740 1 client.18066 shutdown 2024-09-21T03:55:58.199 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:58.192+0000 7f6d15148740 1 -- 172.21.15.39:0/4275049737 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b18cb64070 msgr2=0x55b18cf1b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:55:58.199 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:58.192+0000 7f6d15148740 1 --2- 172.21.15.39:0/4275049737 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b18cb64070 0x55b18cf1b750 secure :-1 s=READY pgs=1254 cs=0 l=1 rev1=1 crypto rx=0x7f6cfc00c670 tx=0x7f6cfc011af0 comp rx=0 tx=0).stop 2024-09-21T03:55:58.199 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:58.192+0000 7f6d15148740 1 -- 172.21.15.39:0/4275049737 shutdown_connections 2024-09-21T03:55:58.199 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:58.192+0000 7f6d15148740 1 --2- 172.21.15.39:0/4275049737 >> [v2:172.21.15.150:6836/4017043144,v1:172.21.15.150:6837/4017043144] conn(0x55b18caf7da0 0x55b18cf9a070 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:58.200 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:58.192+0000 7f6d15148740 1 --2- 172.21.15.39:0/4275049737 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b18cb64070 0x55b18cf1b750 unknown :-1 s=CLOSED pgs=1254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:58.200 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:58.192+0000 7f6d15148740 1 --2- 172.21.15.39:0/4275049737 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b18cb43390 0x55b18cf1b210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:58.200 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:58.192+0000 7f6ce7fff640 0 client.18066 ms_handle_reset on v2:172.21.15.150:6836/4017043144 2024-09-21T03:55:58.200 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:58.192+0000 7f6d15148740 1 --2- 172.21.15.39:0/4275049737 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b18cb3b1b0 0x55b18cf1acd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:58.200 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:58.192+0000 7f6d15148740 1 -- 172.21.15.39:0/4275049737 >> 172.21.15.39:0/4275049737 conn(0x55b18cb58460 msgr2=0x55b18cf0ca50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:55:58.200 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:58.193+0000 7f6d15148740 1 -- 172.21.15.39:0/4275049737 shutdown_connections 2024-09-21T03:55:58.200 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:58.193+0000 7f6d15148740 1 -- 172.21.15.39:0/4275049737 wait complete. 2024-09-21T03:55:58.200 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:58.193+0000 7f6d15148740 20 client.18066 trim_cache size 0 max 16384 2024-09-21T03:55:58.200 INFO:teuthology.orchestra.run.smithi039.stdout:[] 2024-09-21T03:55:58.201 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:58.193+0000 7f6d0ebd4640 1 -- 172.21.15.39:0/3299861897 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f6cf00b6730 msgr2=0x7f6cf00b8c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:55:58.201 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:58.193+0000 7f6d0ebd4640 1 --2- 172.21.15.39:0/3299861897 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f6cf00b6730 0x7f6cf00b8c40 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f6d000cbac0 tx=0x7f6d000983b0 comp rx=0 tx=0).stop 2024-09-21T03:55:58.201 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:58.193+0000 7f6d0ebd4640 1 -- 172.21.15.39:0/3299861897 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6d08108d10 msgr2=0x7f6d0819bd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:55:58.201 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:58.193+0000 7f6d0ebd4640 1 --2- 172.21.15.39:0/3299861897 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6d08108d10 0x7f6d0819bd70 secure :-1 s=READY pgs=1250 cs=0 l=1 rev1=1 crypto rx=0x7f6d000098d0 tx=0x7f6d000317b0 comp rx=0 tx=0).stop 2024-09-21T03:55:58.201 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:58.193+0000 7f6d0ebd4640 1 -- 172.21.15.39:0/3299861897 shutdown_connections 2024-09-21T03:55:58.201 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:58.193+0000 7f6d0ebd4640 1 --2- 172.21.15.39:0/3299861897 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f6cf00b6730 0x7f6cf00b8c40 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:58.201 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:58.193+0000 7f6d0ebd4640 1 --2- 172.21.15.39:0/3299861897 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6d08108d10 0x7f6d0819bd70 unknown :-1 s=CLOSED pgs=1250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:58.201 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:58.193+0000 7f6d0ebd4640 1 --2- 172.21.15.39:0/3299861897 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6d081083c0 0x7f6d0819b830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:58.201 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:58.193+0000 7f6d0ebd4640 1 --2- 172.21.15.39:0/3299861897 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6d08102a80 0x7f6d0819b2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:58.202 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:58.193+0000 7f6d0ebd4640 1 -- 172.21.15.39:0/3299861897 >> 172.21.15.39:0/3299861897 conn(0x7f6d08079740 msgr2=0x7f6d081043d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:55:58.202 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:58.193+0000 7f6d0ebd4640 1 -- 172.21.15.39:0/3299861897 shutdown_connections 2024-09-21T03:55:58.202 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:58.194+0000 7f6d0ebd4640 1 -- 172.21.15.39:0/3299861897 wait complete. 2024-09-21T03:55:58.979 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 51116d5c-77ca-11ef-baf5-efdc52797490 -- ceph tell mds.1:1 damage ls 2024-09-21T03:55:59.239 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/51116d5c-77ca-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:55:59.334 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:55:59 smithi039 ceph-mon[31211]: pgmap v629: 97 pgs: 97 active+clean; 10 MiB data, 525 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-09-21T03:55:59.468 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:55:59 smithi161 ceph-mon[34533]: pgmap v629: 97 pgs: 97 active+clean; 10 MiB data, 525 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-09-21T03:55:59.474 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:55:59 smithi150 ceph-mon[34289]: pgmap v629: 97 pgs: 97 active+clean; 10 MiB data, 525 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-09-21T03:55:59.754 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:59.750+0000 7f169b772640 1 -- 172.21.15.39:0/2515876931 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1694108cf0 msgr2=0x7f169410f520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:55:59.754 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:59.750+0000 7f169b772640 1 --2- 172.21.15.39:0/2515876931 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1694108cf0 0x7f169410f520 secure :-1 s=READY pgs=1255 cs=0 l=1 rev1=1 crypto rx=0x7f168c009ca0 tx=0x7f168c02a8c0 comp rx=0 tx=0).stop 2024-09-21T03:55:59.754 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:59.751+0000 7f169b772640 1 -- 172.21.15.39:0/2515876931 shutdown_connections 2024-09-21T03:55:59.754 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:59.751+0000 7f169b772640 1 --2- 172.21.15.39:0/2515876931 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1694108cf0 0x7f169410f520 unknown :-1 s=CLOSED pgs=1255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:59.754 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:59.751+0000 7f169b772640 1 --2- 172.21.15.39:0/2515876931 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f16941083a0 0x7f16941087b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:59.754 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:59.751+0000 7f169b772640 1 --2- 172.21.15.39:0/2515876931 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1694102a60 0x7f1694102e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:59.754 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:59.751+0000 7f169b772640 1 -- 172.21.15.39:0/2515876931 >> 172.21.15.39:0/2515876931 conn(0x7f1694079730 msgr2=0x7f1694079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:55:59.754 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:59.751+0000 7f169b772640 1 -- 172.21.15.39:0/2515876931 shutdown_connections 2024-09-21T03:55:59.754 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:59.751+0000 7f169b772640 1 -- 172.21.15.39:0/2515876931 wait complete. 2024-09-21T03:55:59.756 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:59.751+0000 7f169b772640 1 Processor -- start 2024-09-21T03:55:59.756 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:59.751+0000 7f169b772640 1 -- start start 2024-09-21T03:55:59.756 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:59.752+0000 7f169b772640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1694102a60 0x7f16940ff670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:55:59.756 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:59.752+0000 7f169b772640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f16941083a0 0x7f16940ffbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:55:59.756 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:59.752+0000 7f169b772640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1694108cf0 0x7f16941000f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:55:59.756 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:59.752+0000 7f169b772640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f1694112960 con 0x7f16941083a0 2024-09-21T03:55:59.756 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:59.752+0000 7f169b772640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f16941127e0 con 0x7f1694108cf0 2024-09-21T03:55:59.756 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:59.752+0000 7f169b772640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f1694112ae0 con 0x7f1694102a60 2024-09-21T03:55:59.756 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:59.752+0000 7f1698d0d640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f16941083a0 0x7f16940ffbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:55:59.757 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:59.752+0000 7f1698d0d640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f16941083a0 0x7f16940ffbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:33248/0 (socket says 172.21.15.39:33248) 2024-09-21T03:55:59.757 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:59.752+0000 7f1698d0d640 1 -- 172.21.15.39:0/3784014643 learned_addr learned my addr 172.21.15.39:0/3784014643 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:55:59.757 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:59.752+0000 7f1699d0f640 1 --2- 172.21.15.39:0/3784014643 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1694108cf0 0x7f16941000f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:55:59.757 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:59.752+0000 7f169950e640 1 --2- 172.21.15.39:0/3784014643 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1694102a60 0x7f16940ff670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:55:59.757 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:59.752+0000 7f1698d0d640 1 -- 172.21.15.39:0/3784014643 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1694102a60 msgr2=0x7f16940ff670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:55:59.757 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:59.752+0000 7f1698d0d640 1 --2- 172.21.15.39:0/3784014643 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1694102a60 0x7f16940ff670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:59.757 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:59.752+0000 7f1698d0d640 1 -- 172.21.15.39:0/3784014643 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1694108cf0 msgr2=0x7f16941000f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:55:59.757 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:59.752+0000 7f1698d0d640 1 --2- 172.21.15.39:0/3784014643 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1694108cf0 0x7f16941000f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:59.757 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:59.752+0000 7f1698d0d640 1 -- 172.21.15.39:0/3784014643 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f16941a8780 con 0x7f16941083a0 2024-09-21T03:55:59.757 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:59.752+0000 7f169950e640 1 --2- 172.21.15.39:0/3784014643 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1694102a60 0x7f16940ff670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:55:59.757 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:59.752+0000 7f1698d0d640 1 --2- 172.21.15.39:0/3784014643 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f16941083a0 0x7f16940ffbb0 secure :-1 s=READY pgs=1256 cs=0 l=1 rev1=1 crypto rx=0x7f168400d770 tx=0x7f168400dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:55:59.757 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:59.753+0000 7f168a7fc640 1 -- 172.21.15.39:0/3784014643 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1684017070 con 0x7f16941083a0 2024-09-21T03:55:59.757 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:59.753+0000 7f168a7fc640 1 -- 172.21.15.39:0/3784014643 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1684009c30 con 0x7f16941083a0 2024-09-21T03:55:59.757 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:59.753+0000 7f169b772640 1 -- 172.21.15.39:0/3784014643 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f16941a8a70 con 0x7f16941083a0 2024-09-21T03:55:59.758 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:59.753+0000 7f168a7fc640 1 -- 172.21.15.39:0/3784014643 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f168400f620 con 0x7f16941083a0 2024-09-21T03:55:59.758 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:59.753+0000 7f169b772640 1 -- 172.21.15.39:0/3784014643 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f16941a8fb0 con 0x7f16941083a0 2024-09-21T03:55:59.759 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:59.756+0000 7f16a1ce6740 1 Processor -- start 2024-09-21T03:55:59.759 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:59.756+0000 7f16a1ce6740 1 -- start start 2024-09-21T03:55:59.759 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:59.757+0000 7f168a7fc640 1 -- 172.21.15.39:0/3784014643 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 27) ==== 154643+0+0 (secure 0 0 0) 0x7f168401a070 con 0x7f16941083a0 2024-09-21T03:55:59.760 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:59.757+0000 7f16a1ce6740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b4fb9c1070 0x55b4fbd686a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:55:59.760 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:59.757+0000 7f16a1ce6740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b4fb9981b0 0x55b4fbd68be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:55:59.760 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:59.757+0000 7f16a1ce6740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b4fb9a0390 0x55b4fbd69120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:55:59.760 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:59.757+0000 7f16a1ce6740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x55b4fbcdeea0 con 0x55b4fb9a0390 2024-09-21T03:55:59.760 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:59.757+0000 7f16a1ce6740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x55b4fbcce2d0 con 0x55b4fb9981b0 2024-09-21T03:55:59.760 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:59.757+0000 7f16a1ce6740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55b4fbcd20e0 con 0x55b4fb9c1070 2024-09-21T03:55:59.761 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:59.757+0000 7f1698d0d640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b4fb9a0390 0x55b4fbd69120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:55:59.761 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:59.757+0000 7f1698d0d640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b4fb9a0390 0x55b4fbd69120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:33264/0 (socket says 172.21.15.39:33264) 2024-09-21T03:55:59.761 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:59.757+0000 7f1698d0d640 1 -- 172.21.15.39:0/1611008438 learned_addr learned my addr 172.21.15.39:0/1611008438 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:55:59.761 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:59.757+0000 7f1699d0f640 1 --2- 172.21.15.39:0/1611008438 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b4fb9c1070 0x55b4fbd686a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:55:59.761 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:59.757+0000 7f169950e640 1 --2- 172.21.15.39:0/1611008438 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b4fb9981b0 0x55b4fbd68be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:55:59.762 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:59.757+0000 7f1698d0d640 1 -- 172.21.15.39:0/1611008438 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b4fb9c1070 msgr2=0x55b4fbd686a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:55:59.762 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:59.757+0000 7f1698d0d640 1 --2- 172.21.15.39:0/1611008438 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b4fb9c1070 0x55b4fbd686a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:59.762 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:59.757+0000 7f1698d0d640 1 -- 172.21.15.39:0/1611008438 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b4fb9981b0 msgr2=0x55b4fbd68be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:55:59.762 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:59.757+0000 7f1698d0d640 1 --2- 172.21.15.39:0/1611008438 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b4fb9981b0 0x55b4fbd68be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:59.762 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:59.757+0000 7f1698d0d640 1 -- 172.21.15.39:0/1611008438 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b4fba2c650 con 0x55b4fb9a0390 2024-09-21T03:55:59.762 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:59.758+0000 7f1698d0d640 1 --2- 172.21.15.39:0/1611008438 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b4fb9a0390 0x55b4fbd69120 secure :-1 s=READY pgs=1257 cs=0 l=1 rev1=1 crypto rx=0x7f16840e4950 tx=0x7f16840ef950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6cc5496d659bf9f8 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:55:59.762 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:59.758+0000 7f168a7fc640 1 --2- 172.21.15.39:0/3784014643 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f16800b6680 0x7f16800b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:55:59.762 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:59.758+0000 7f167f7fe640 1 -- 172.21.15.39:0/1611008438 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16840fa070 con 0x55b4fb9a0390 2024-09-21T03:55:59.762 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:59.758+0000 7f167f7fe640 1 -- 172.21.15.39:0/1611008438 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f16840acc20 con 0x55b4fb9a0390 2024-09-21T03:55:59.762 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:59.758+0000 7f1699d0f640 1 --2- 172.21.15.39:0/3784014643 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f16800b6680 0x7f16800b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:55:59.762 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:59.758+0000 7f167f7fe640 1 -- 172.21.15.39:0/1611008438 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16840ac880 con 0x55b4fb9a0390 2024-09-21T03:55:59.762 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:59.758+0000 7f168a7fc640 1 -- 172.21.15.39:0/3784014643 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8528+0+0 (secure 0 0 0) 0x7f16840e72c0 con 0x7f16941083a0 2024-09-21T03:55:59.765 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:59.759+0000 7f16a1ce6740 1 -- 172.21.15.39:0/1611008438 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b4fb9a0390 msgr2=0x55b4fbd69120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:55:59.765 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:59.759+0000 7f16a1ce6740 1 --2- 172.21.15.39:0/1611008438 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b4fb9a0390 0x55b4fbd69120 secure :-1 s=READY pgs=1257 cs=0 l=1 rev1=1 crypto rx=0x7f16840e4950 tx=0x7f16840ef950 comp rx=0 tx=0).stop 2024-09-21T03:55:59.765 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:59.759+0000 7f1699d0f640 1 --2- 172.21.15.39:0/3784014643 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f16800b6680 0x7f16800b8b90 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f168c005e30 tx=0x7f168c005c30 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:55:59.765 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:59.759+0000 7f16a1ce6740 1 -- 172.21.15.39:0/1611008438 shutdown_connections 2024-09-21T03:55:59.765 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:59.759+0000 7f16a1ce6740 1 --2- 172.21.15.39:0/1611008438 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b4fb9a0390 0x55b4fbd69120 unknown :-1 s=CLOSED pgs=1257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:59.766 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:59.759+0000 7f16a1ce6740 1 --2- 172.21.15.39:0/1611008438 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b4fb9981b0 0x55b4fbd68be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:59.766 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:59.759+0000 7f16a1ce6740 1 --2- 172.21.15.39:0/1611008438 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b4fb9c1070 0x55b4fbd686a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:59.766 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:59.759+0000 7f16a1ce6740 1 -- 172.21.15.39:0/1611008438 >> 172.21.15.39:0/1611008438 conn(0x55b4fb9b5460 msgr2=0x55b4fbd66590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:55:59.766 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:59.759+0000 7f16a1ce6740 1 -- 172.21.15.39:0/1611008438 shutdown_connections 2024-09-21T03:55:59.766 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:59.759+0000 7f16a1ce6740 1 -- 172.21.15.39:0/1611008438 wait complete. 2024-09-21T03:55:59.766 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:59.760+0000 7f16a1ce6740 1 -- start start 2024-09-21T03:55:59.766 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:59.760+0000 7f16a1ce6740 1 Processor -- start 2024-09-21T03:55:59.766 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:59.760+0000 7f16a1ce6740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b4fb9981b0 0x55b4fbd73ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:55:59.766 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:59.760+0000 7f16a1ce6740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b4fb9a0390 0x55b4fbd74220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:55:59.766 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:59.760+0000 7f16a1ce6740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b4fb9c1070 0x55b4fbd747a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:55:59.766 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:59.760+0000 7f16a1ce6740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x55b4fbb2f700 con 0x55b4fb9a0390 2024-09-21T03:55:59.766 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:59.760+0000 7f16a1ce6740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x55b4fbc69160 con 0x55b4fb9981b0 2024-09-21T03:55:59.766 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:59.760+0000 7f16a1ce6740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55b4fbd76e80 con 0x55b4fb9c1070 2024-09-21T03:55:59.767 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:59.760+0000 7f16a1ce6740 10 client.0 _ll_register_callbacks cb 0x55b4fbc829e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:55:59.767 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:59.760+0000 7f1698d0d640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b4fb9a0390 0x55b4fbd74220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:55:59.767 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:59.760+0000 7f1698d0d640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b4fb9a0390 0x55b4fbd74220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:33274/0 (socket says 172.21.15.39:33274) 2024-09-21T03:55:59.767 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:59.760+0000 7f1698d0d640 1 -- 172.21.15.39:0/687785927 learned_addr learned my addr 172.21.15.39:0/687785927 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:55:59.767 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:59.761+0000 7f1699d0f640 1 --2- 172.21.15.39:0/687785927 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b4fb9c1070 0x55b4fbd747a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:55:59.767 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:59.761+0000 7f169950e640 1 --2- 172.21.15.39:0/687785927 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b4fb9981b0 0x55b4fbd73ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:55:59.767 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:59.761+0000 7f1698d0d640 1 -- 172.21.15.39:0/687785927 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b4fb9c1070 msgr2=0x55b4fbd747a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:55:59.767 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:59.761+0000 7f1698d0d640 1 --2- 172.21.15.39:0/687785927 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b4fb9c1070 0x55b4fbd747a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:59.767 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:59.761+0000 7f1698d0d640 1 -- 172.21.15.39:0/687785927 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b4fb9981b0 msgr2=0x55b4fbd73ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:55:59.767 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:59.761+0000 7f1698d0d640 1 --2- 172.21.15.39:0/687785927 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b4fb9981b0 0x55b4fbd73ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:59.767 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:59.761+0000 7f1698d0d640 1 -- 172.21.15.39:0/687785927 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b4fbc96c90 con 0x55b4fb9a0390 2024-09-21T03:55:59.767 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:59.761+0000 7f1699d0f640 1 --2- 172.21.15.39:0/687785927 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b4fb9c1070 0x55b4fbd747a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:55:59.767 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:59.761+0000 7f169950e640 1 --2- 172.21.15.39:0/687785927 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b4fb9981b0 0x55b4fbd73ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:55:59.768 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:59.761+0000 7f1698d0d640 1 --2- 172.21.15.39:0/687785927 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b4fb9a0390 0x55b4fbd74220 secure :-1 s=READY pgs=1258 cs=0 l=1 rev1=1 crypto rx=0x7f16840a96e0 tx=0x7f16840a95b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b1bbab96691332b server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:55:59.768 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:59.761+0000 7f167cff9640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-09-21T03:55:59.768 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:59.761+0000 7f167cff9640 1 -- 172.21.15.39:0/687785927 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16840905a0 con 0x55b4fb9a0390 2024-09-21T03:55:59.768 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:59.761+0000 7f167cff9640 1 -- 172.21.15.39:0/687785927 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8528+0+0 (secure 0 0 0) 0x7f16840b68a0 con 0x55b4fb9a0390 2024-09-21T03:55:59.768 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:59.761+0000 7f16a1ce6740 1 -- 172.21.15.39:0/687785927 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f16840acc20 con 0x55b4fb9a0390 2024-09-21T03:55:59.768 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:59.761+0000 7f167cff9640 1 -- 172.21.15.39:0/687785927 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f1684021020 con 0x55b4fb9a0390 2024-09-21T03:55:59.768 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:59.761+0000 7f16a1ce6740 10 client.18078 fetch_fsmap learned FSMap version 29 2024-09-21T03:55:59.768 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:59.761+0000 7f16a1ce6740 1 -- 172.21.15.39:0/687785927 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55b4fbd82b50 con 0x55b4fb9a0390 2024-09-21T03:55:59.768 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:59.762+0000 7f167cff9640 1 -- 172.21.15.39:0/687785927 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f16840acc20 con 0x55b4fb9a0390 2024-09-21T03:55:59.768 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:59.762+0000 7f167cff9640 1 -- 172.21.15.39:0/687785927 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f16840a8a60 con 0x55b4fb9a0390 2024-09-21T03:55:59.768 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:59.762+0000 7f16a1ce6740 10 client.18078 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T03:55:59.768 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:59.762+0000 7f16a1ce6740 10 client.18078 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-21T03:55:59.768 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:59.762+0000 7f16a1ce6740 20 client.18078 populate_metadata read hostname 'smithi039' 2024-09-21T03:55:59.768 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:59.762+0000 7f16a1ce6740 1 --2- 172.21.15.39:0/687785927 >> [v2:172.21.15.161:6838/2190107270,v1:172.21.15.161:6839/2190107270] conn(0x55b4fb97f9f0 0x55b4fbd85eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:55:59.769 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:59.762+0000 7f16a1ce6740 4 client.18078 mds_command: new command op to 24667 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:55:59.769 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:59.762+0000 7f16a1ce6740 1 -- 172.21.15.39:0/687785927 --> [v2:172.21.15.161:6838/2190107270,v1:172.21.15.161:6839/2190107270] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b4fbce0df0 con 0x55b4fb97f9f0 2024-09-21T03:55:59.769 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:59.762+0000 7f169950e640 1 --2- 172.21.15.39:0/687785927 >> [v2:172.21.15.161:6838/2190107270,v1:172.21.15.161:6839/2190107270] conn(0x55b4fb97f9f0 0x55b4fbd85eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:55:59.769 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:59.763+0000 7f169950e640 1 --2- 172.21.15.39:0/687785927 >> [v2:172.21.15.161:6838/2190107270,v1:172.21.15.161:6839/2190107270] conn(0x55b4fb97f9f0 0x55b4fbd85eb0 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=f39414db2f7de44a server_cookie=e8faf0764199a539 in_seq=0 out_seq=0 2024-09-21T03:55:59.769 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:59.763+0000 7f167cff9640 10 client.18078 ms_handle_connect on v2:172.21.15.161:6838/2190107270 2024-09-21T03:55:59.770 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:59.763+0000 7f167cff9640 1 -- 172.21.15.39:0/687785927 <== mds.1 v2:172.21.15.161:6838/2190107270 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55b4fbce0df0 con 0x55b4fb97f9f0 2024-09-21T03:55:59.770 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:59.763+0000 7f167cff9640 10 client.18078 handle_command_reply: tid=0 2024-09-21T03:55:59.770 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:59.763+0000 7f16a1ce6740 1 client.18078 shutdown 2024-09-21T03:55:59.770 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:59.764+0000 7f16a1ce6740 1 -- 172.21.15.39:0/687785927 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b4fb9a0390 msgr2=0x55b4fbd74220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:55:59.770 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:59.764+0000 7f16a1ce6740 1 --2- 172.21.15.39:0/687785927 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b4fb9a0390 0x55b4fbd74220 secure :-1 s=READY pgs=1258 cs=0 l=1 rev1=1 crypto rx=0x7f16840a96e0 tx=0x7f16840a95b0 comp rx=0 tx=0).stop 2024-09-21T03:55:59.771 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:59.764+0000 7f16a1ce6740 1 -- 172.21.15.39:0/687785927 shutdown_connections 2024-09-21T03:55:59.771 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:59.764+0000 7f16a1ce6740 1 --2- 172.21.15.39:0/687785927 >> [v2:172.21.15.161:6838/2190107270,v1:172.21.15.161:6839/2190107270] conn(0x55b4fb97f9f0 0x55b4fbd85eb0 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:59.771 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:59.764+0000 7f16a1ce6740 1 --2- 172.21.15.39:0/687785927 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b4fb9c1070 0x55b4fbd747a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:59.771 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:59.764+0000 7f16a1ce6740 1 --2- 172.21.15.39:0/687785927 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b4fb9a0390 0x55b4fbd74220 unknown :-1 s=CLOSED pgs=1258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:59.771 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:59.764+0000 7f16a1ce6740 1 --2- 172.21.15.39:0/687785927 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b4fb9981b0 0x55b4fbd73ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:59.771 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:59.764+0000 7f16a1ce6740 1 -- 172.21.15.39:0/687785927 >> 172.21.15.39:0/687785927 conn(0x55b4fb9b5460 msgr2=0x55b4fbd6b830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:55:59.771 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:59.764+0000 7f167cff9640 0 client.18078 ms_handle_reset on v2:172.21.15.161:6838/2190107270 2024-09-21T03:55:59.771 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:59.764+0000 7f16a1ce6740 1 -- 172.21.15.39:0/687785927 shutdown_connections 2024-09-21T03:55:59.771 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:59.764+0000 7f16a1ce6740 1 -- 172.21.15.39:0/687785927 wait complete. 2024-09-21T03:55:59.771 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:59.764+0000 7f16a1ce6740 20 client.18078 trim_cache size 0 max 16384 2024-09-21T03:55:59.780 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:59.777+0000 7f16a1ce6740 1 Processor -- start 2024-09-21T03:55:59.780 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:59.777+0000 7f16a1ce6740 1 -- start start 2024-09-21T03:55:59.781 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:59.777+0000 7f16a1ce6740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b4fb9981b0 0x55b4fbd75bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:55:59.781 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:59.777+0000 7f16a1ce6740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b4fb9a0390 0x55b4fbd696d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:55:59.781 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:59.777+0000 7f16a1ce6740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b4fb9c1070 0x55b4fbd7f810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:55:59.781 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:59.777+0000 7f16a1ce6740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f16941b1a10 con 0x55b4fb9c1070 2024-09-21T03:55:59.781 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:59.777+0000 7f16a1ce6740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f16941b3550 con 0x55b4fb9981b0 2024-09-21T03:55:59.781 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:59.777+0000 7f16a1ce6740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f16941b20e0 con 0x55b4fb9a0390 2024-09-21T03:55:59.781 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:59.778+0000 7f1699d0f640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b4fb9c1070 0x55b4fbd7f810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:55:59.781 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:59.778+0000 7f1699d0f640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b4fb9c1070 0x55b4fbd7f810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:33288/0 (socket says 172.21.15.39:33288) 2024-09-21T03:55:59.781 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:59.778+0000 7f1699d0f640 1 -- 172.21.15.39:0/3088967701 learned_addr learned my addr 172.21.15.39:0/3088967701 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:55:59.781 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:59.778+0000 7f1698d0d640 1 --2- 172.21.15.39:0/3088967701 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b4fb9a0390 0x55b4fbd696d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:55:59.781 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:59.778+0000 7f1699d0f640 1 -- 172.21.15.39:0/3088967701 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b4fb9a0390 msgr2=0x55b4fbd696d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:55:59.781 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:59.778+0000 7f1699d0f640 1 --2- 172.21.15.39:0/3088967701 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b4fb9a0390 0x55b4fbd696d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:59.781 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:59.778+0000 7f169950e640 1 --2- 172.21.15.39:0/3088967701 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b4fb9981b0 0x55b4fbd75bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:55:59.782 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:59.778+0000 7f1699d0f640 1 -- 172.21.15.39:0/3088967701 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b4fb9981b0 msgr2=0x55b4fbd75bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:55:59.782 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:59.778+0000 7f1699d0f640 1 --2- 172.21.15.39:0/3088967701 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b4fb9981b0 0x55b4fbd75bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:59.782 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:59.778+0000 7f1699d0f640 1 -- 172.21.15.39:0/3088967701 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b4fbd8c1a0 con 0x55b4fb9c1070 2024-09-21T03:55:59.784 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:59.778+0000 7f1699d0f640 1 --2- 172.21.15.39:0/3088967701 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b4fb9c1070 0x55b4fbd7f810 secure :-1 s=READY pgs=1259 cs=0 l=1 rev1=1 crypto rx=0x7f168c032e20 tx=0x7f168c041000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f1341e8064b05c32 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:55:59.785 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:59.778+0000 7f167effd640 1 -- 172.21.15.39:0/3088967701 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f168c003970 con 0x55b4fb9c1070 2024-09-21T03:55:59.785 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:59.778+0000 7f167effd640 1 -- 172.21.15.39:0/3088967701 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f168c036040 con 0x55b4fb9c1070 2024-09-21T03:55:59.785 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:59.778+0000 7f167effd640 1 -- 172.21.15.39:0/3088967701 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f168c004800 con 0x55b4fb9c1070 2024-09-21T03:55:59.785 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:59.779+0000 7f16a1ce6740 1 -- 172.21.15.39:0/3088967701 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b4fb9c1070 msgr2=0x55b4fbd7f810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:55:59.785 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:59.779+0000 7f16a1ce6740 1 --2- 172.21.15.39:0/3088967701 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b4fb9c1070 0x55b4fbd7f810 secure :-1 s=READY pgs=1259 cs=0 l=1 rev1=1 crypto rx=0x7f168c032e20 tx=0x7f168c041000 comp rx=0 tx=0).stop 2024-09-21T03:55:59.785 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:59.779+0000 7f16a1ce6740 1 -- 172.21.15.39:0/3088967701 shutdown_connections 2024-09-21T03:55:59.785 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:59.779+0000 7f16a1ce6740 1 --2- 172.21.15.39:0/3088967701 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b4fb9c1070 0x55b4fbd7f810 unknown :-1 s=CLOSED pgs=1259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:59.785 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:59.779+0000 7f16a1ce6740 1 --2- 172.21.15.39:0/3088967701 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b4fb9a0390 0x55b4fbd696d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:59.785 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:59.779+0000 7f16a1ce6740 1 --2- 172.21.15.39:0/3088967701 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b4fb9981b0 0x55b4fbd75bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:59.785 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:59.779+0000 7f16a1ce6740 1 -- 172.21.15.39:0/3088967701 >> 172.21.15.39:0/3088967701 conn(0x55b4fb9b5460 msgr2=0x55b4fbd7f100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:55:59.785 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:59.779+0000 7f16a1ce6740 1 -- 172.21.15.39:0/3088967701 shutdown_connections 2024-09-21T03:55:59.785 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:59.779+0000 7f16a1ce6740 1 -- 172.21.15.39:0/3088967701 wait complete. 2024-09-21T03:55:59.785 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:59.780+0000 7f16a1ce6740 1 -- start start 2024-09-21T03:55:59.786 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:59.780+0000 7f16a1ce6740 1 Processor -- start 2024-09-21T03:55:59.786 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:59.780+0000 7f16a1ce6740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b4fb9981b0 0x55b4fbd895b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:55:59.786 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:59.780+0000 7f16a1ce6740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b4fb9a0390 0x55b4fbd86a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:55:59.786 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:59.780+0000 7f16a1ce6740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b4fb9c1070 0x55b4fbd86fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:55:59.786 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:59.780+0000 7f16a1ce6740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f16941b2e80 con 0x55b4fb9c1070 2024-09-21T03:55:59.786 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:59.780+0000 7f16a1ce6740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f16941b27b0 con 0x55b4fb9981b0 2024-09-21T03:55:59.786 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:59.780+0000 7f16a1ce6740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f16941af9f0 con 0x55b4fb9a0390 2024-09-21T03:55:59.786 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:59.780+0000 7f16a1ce6740 10 client.0 _ll_register_callbacks cb 0x7f16840f8070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:55:59.786 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:59.780+0000 7f1699d0f640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b4fb9c1070 0x55b4fbd86fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:55:59.786 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:59.780+0000 7f1699d0f640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b4fb9c1070 0x55b4fbd86fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:33292/0 (socket says 172.21.15.39:33292) 2024-09-21T03:55:59.786 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:59.780+0000 7f1699d0f640 1 -- 172.21.15.39:0/1353078411 learned_addr learned my addr 172.21.15.39:0/1353078411 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:55:59.786 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:59.781+0000 7f169950e640 1 --2- 172.21.15.39:0/1353078411 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b4fb9981b0 0x55b4fbd895b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:55:59.786 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:59.781+0000 7f1698d0d640 1 --2- 172.21.15.39:0/1353078411 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b4fb9a0390 0x55b4fbd86a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:55:59.786 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:59.781+0000 7f1699d0f640 1 -- 172.21.15.39:0/1353078411 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b4fb9a0390 msgr2=0x55b4fbd86a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:55:59.787 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:59.781+0000 7f1699d0f640 1 --2- 172.21.15.39:0/1353078411 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b4fb9a0390 0x55b4fbd86a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:59.787 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:59.781+0000 7f1699d0f640 1 -- 172.21.15.39:0/1353078411 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b4fb9981b0 msgr2=0x55b4fbd895b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:55:59.787 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:59.781+0000 7f1699d0f640 1 --2- 172.21.15.39:0/1353078411 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b4fb9981b0 0x55b4fbd895b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:59.787 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:59.781+0000 7f1699d0f640 1 -- 172.21.15.39:0/1353078411 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b4fbd87630 con 0x55b4fb9c1070 2024-09-21T03:55:59.787 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:59.781+0000 7f1699d0f640 1 --2- 172.21.15.39:0/1353078411 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b4fb9c1070 0x55b4fbd86fc0 secure :-1 s=READY pgs=1260 cs=0 l=1 rev1=1 crypto rx=0x7f168c02a900 tx=0x7f168c009540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e6aa460dcf14e724 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:55:59.787 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:59.781+0000 7f167cff9640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-09-21T03:55:59.787 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:59.781+0000 7f167cff9640 1 -- 172.21.15.39:0/1353078411 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f168c003970 con 0x55b4fb9c1070 2024-09-21T03:55:59.787 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:59.781+0000 7f167cff9640 1 -- 172.21.15.39:0/1353078411 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8528+0+0 (secure 0 0 0) 0x7f168c003b10 con 0x55b4fb9c1070 2024-09-21T03:55:59.787 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:59.781+0000 7f16a1ce6740 1 -- 172.21.15.39:0/1353078411 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f168c036040 con 0x55b4fb9c1070 2024-09-21T03:55:59.787 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:59.781+0000 7f167cff9640 1 -- 172.21.15.39:0/1353078411 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f168c003d10 con 0x55b4fb9c1070 2024-09-21T03:55:59.787 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:59.781+0000 7f16a1ce6740 10 client.18084 fetch_fsmap learned FSMap version 29 2024-09-21T03:55:59.787 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:59.781+0000 7f16a1ce6740 1 -- 172.21.15.39:0/1353078411 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55b4fbdf0460 con 0x55b4fb9c1070 2024-09-21T03:55:59.787 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:59.782+0000 7f167cff9640 1 -- 172.21.15.39:0/1353078411 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f168c036040 con 0x55b4fb9c1070 2024-09-21T03:55:59.787 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:59.782+0000 7f167cff9640 1 -- 172.21.15.39:0/1353078411 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f168c0305c0 con 0x55b4fb9c1070 2024-09-21T03:55:59.788 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:59.782+0000 7f16a1ce6740 10 client.18084 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T03:55:59.788 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:59.782+0000 7f16a1ce6740 10 client.18084 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-21T03:55:59.788 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:59.782+0000 7f16a1ce6740 20 client.18084 populate_metadata read hostname 'smithi039' 2024-09-21T03:55:59.788 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:59.782+0000 7f16a1ce6740 1 --2- 172.21.15.39:0/1353078411 >> [v2:172.21.15.161:6838/2190107270,v1:172.21.15.161:6839/2190107270] conn(0x55b4fb954da0 0x55b4fbdf2b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:55:59.788 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:59.782+0000 7f16a1ce6740 4 client.18084 mds_command: new command op to 24667 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T03:55:59.788 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:59.782+0000 7f16a1ce6740 1 -- 172.21.15.39:0/1353078411 --> [v2:172.21.15.161:6838/2190107270,v1:172.21.15.161:6839/2190107270] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55b4fbcdc8a0 con 0x55b4fb954da0 2024-09-21T03:55:59.789 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:59.782+0000 7f169950e640 1 --2- 172.21.15.39:0/1353078411 >> [v2:172.21.15.161:6838/2190107270,v1:172.21.15.161:6839/2190107270] conn(0x55b4fb954da0 0x55b4fbdf2b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:55:59.789 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:59.783+0000 7f169950e640 1 --2- 172.21.15.39:0/1353078411 >> [v2:172.21.15.161:6838/2190107270,v1:172.21.15.161:6839/2190107270] conn(0x55b4fb954da0 0x55b4fbdf2b20 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=72db92d78f9e76da server_cookie=3e9bd86bd8cc96bd in_seq=0 out_seq=0 2024-09-21T03:55:59.789 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:59.783+0000 7f167cff9640 10 client.18084 ms_handle_connect on v2:172.21.15.161:6838/2190107270 2024-09-21T03:55:59.789 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:59.783+0000 7f167cff9640 1 -- 172.21.15.39:0/1353078411 <== mds.1 v2:172.21.15.161:6838/2190107270 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55b4fbcdc8a0 con 0x55b4fb954da0 2024-09-21T03:55:59.789 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:59.783+0000 7f167cff9640 10 client.18084 handle_command_reply: tid=0 2024-09-21T03:55:59.789 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:59.783+0000 7f16a1ce6740 1 client.18084 shutdown 2024-09-21T03:55:59.789 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:59.783+0000 7f16a1ce6740 1 -- 172.21.15.39:0/1353078411 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b4fb9c1070 msgr2=0x55b4fbd86fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:55:59.790 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:59.783+0000 7f16a1ce6740 1 --2- 172.21.15.39:0/1353078411 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b4fb9c1070 0x55b4fbd86fc0 secure :-1 s=READY pgs=1260 cs=0 l=1 rev1=1 crypto rx=0x7f168c02a900 tx=0x7f168c009540 comp rx=0 tx=0).stop 2024-09-21T03:55:59.790 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:59.783+0000 7f16a1ce6740 1 -- 172.21.15.39:0/1353078411 shutdown_connections 2024-09-21T03:55:59.790 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:59.783+0000 7f16a1ce6740 1 --2- 172.21.15.39:0/1353078411 >> [v2:172.21.15.161:6838/2190107270,v1:172.21.15.161:6839/2190107270] conn(0x55b4fb954da0 0x55b4fbdf2b20 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:59.790 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:59.783+0000 7f16a1ce6740 1 --2- 172.21.15.39:0/1353078411 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b4fb9c1070 0x55b4fbd86fc0 unknown :-1 s=CLOSED pgs=1260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:59.790 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:59.783+0000 7f16a1ce6740 1 --2- 172.21.15.39:0/1353078411 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b4fb9a0390 0x55b4fbd86a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:59.790 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:59.783+0000 7f16a1ce6740 1 --2- 172.21.15.39:0/1353078411 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b4fb9981b0 0x55b4fbd895b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:59.790 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:59.783+0000 7f16a1ce6740 1 -- 172.21.15.39:0/1353078411 >> 172.21.15.39:0/1353078411 conn(0x55b4fb9b5460 msgr2=0x55b4fbd7f890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:55:59.790 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:59.783+0000 7f167cff9640 0 client.18084 ms_handle_reset on v2:172.21.15.161:6838/2190107270 2024-09-21T03:55:59.790 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:59.783+0000 7f16a1ce6740 1 -- 172.21.15.39:0/1353078411 shutdown_connections 2024-09-21T03:55:59.790 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:59.783+0000 7f16a1ce6740 1 -- 172.21.15.39:0/1353078411 wait complete. 2024-09-21T03:55:59.790 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:59.783+0000 7f16a1ce6740 20 client.18084 trim_cache size 0 max 16384 2024-09-21T03:55:59.791 INFO:teuthology.orchestra.run.smithi039.stdout:[] 2024-09-21T03:55:59.791 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:59.784+0000 7f169b772640 1 -- 172.21.15.39:0/3784014643 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f16800b6680 msgr2=0x7f16800b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:55:59.791 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:59.784+0000 7f169b772640 1 --2- 172.21.15.39:0/3784014643 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f16800b6680 0x7f16800b8b90 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f168c005e30 tx=0x7f168c005c30 comp rx=0 tx=0).stop 2024-09-21T03:55:59.791 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:59.784+0000 7f169b772640 1 -- 172.21.15.39:0/3784014643 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f16941083a0 msgr2=0x7f16940ffbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:55:59.791 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:59.784+0000 7f169b772640 1 --2- 172.21.15.39:0/3784014643 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f16941083a0 0x7f16940ffbb0 secure :-1 s=READY pgs=1256 cs=0 l=1 rev1=1 crypto rx=0x7f168400d770 tx=0x7f168400dc30 comp rx=0 tx=0).stop 2024-09-21T03:55:59.792 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:59.784+0000 7f169b772640 1 -- 172.21.15.39:0/3784014643 shutdown_connections 2024-09-21T03:55:59.792 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:59.784+0000 7f169b772640 1 --2- 172.21.15.39:0/3784014643 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f16800b6680 0x7f16800b8b90 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:59.792 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:59.784+0000 7f169b772640 1 --2- 172.21.15.39:0/3784014643 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1694108cf0 0x7f16941000f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:59.792 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:59.784+0000 7f169b772640 1 --2- 172.21.15.39:0/3784014643 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f16941083a0 0x7f16940ffbb0 unknown :-1 s=CLOSED pgs=1256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:59.792 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:59.784+0000 7f169b772640 1 --2- 172.21.15.39:0/3784014643 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1694102a60 0x7f16940ff670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:59.792 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:59.784+0000 7f169b772640 1 -- 172.21.15.39:0/3784014643 >> 172.21.15.39:0/3784014643 conn(0x7f1694079730 msgr2=0x7f1694101a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:55:59.792 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:59.784+0000 7f169b772640 1 -- 172.21.15.39:0/3784014643 shutdown_connections 2024-09-21T03:55:59.792 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:55:59.784+0000 7f169b772640 1 -- 172.21.15.39:0/3784014643 wait complete. 2024-09-21T03:56:00.691 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 51116d5c-77ca-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 damage ls 2024-09-21T03:56:00.963 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/51116d5c-77ca-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:56:01.333 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:56:01 smithi039 ceph-mon[31211]: pgmap v630: 97 pgs: 97 active+clean; 10 MiB data, 525 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-09-21T03:56:01.447 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:01.444+0000 7f85c8a15640 1 -- 172.21.15.39:0/641727787 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f85c41051c0 msgr2=0x7f85c41055d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:56:01.447 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:01.444+0000 7f85c8a15640 1 --2- 172.21.15.39:0/641727787 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f85c41051c0 0x7f85c41055d0 secure :-1 s=READY pgs=1261 cs=0 l=1 rev1=1 crypto rx=0x7f85b8009900 tx=0x7f85b802a410 comp rx=0 tx=0).stop 2024-09-21T03:56:01.447 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:01.444+0000 7f85c8a15640 1 -- 172.21.15.39:0/641727787 shutdown_connections 2024-09-21T03:56:01.447 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:01.444+0000 7f85c8a15640 1 --2- 172.21.15.39:0/641727787 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f85c4105b10 0x7f85c410f5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:01.447 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:01.444+0000 7f85c8a15640 1 --2- 172.21.15.39:0/641727787 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f85c41051c0 0x7f85c41055d0 unknown :-1 s=CLOSED pgs=1261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:01.447 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:01.444+0000 7f85c8a15640 1 --2- 172.21.15.39:0/641727787 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f85c41031f0 0x7f85c4103600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:01.447 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:01.444+0000 7f85c8a15640 1 -- 172.21.15.39:0/641727787 >> 172.21.15.39:0/641727787 conn(0x7f85c4079e40 msgr2=0x7f85c407a250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:56:01.447 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:01.444+0000 7f85c8a15640 1 -- 172.21.15.39:0/641727787 shutdown_connections 2024-09-21T03:56:01.448 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:01.444+0000 7f85c8a15640 1 -- 172.21.15.39:0/641727787 wait complete. 2024-09-21T03:56:01.449 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:01.445+0000 7f85c8a15640 1 Processor -- start 2024-09-21T03:56:01.449 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:01.445+0000 7f85c8a15640 1 -- start start 2024-09-21T03:56:01.449 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:01.445+0000 7f85c8a15640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f85c41031f0 0x7f85c419a4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:56:01.449 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:01.445+0000 7f85c8a15640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f85c41051c0 0x7f85c419aa40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:56:01.449 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:01.445+0000 7f85c8a15640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f85c4105b10 0x7f85c419fce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:56:01.449 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:01.445+0000 7f85c8a15640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f85c41129e0 con 0x7f85c4105b10 2024-09-21T03:56:01.450 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:01.445+0000 7f85c8a15640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f85c4112860 con 0x7f85c41051c0 2024-09-21T03:56:01.450 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:01.445+0000 7f85c8a15640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f85c4112b60 con 0x7f85c41031f0 2024-09-21T03:56:01.450 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:01.445+0000 7f85c2d9d640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f85c4105b10 0x7f85c419fce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:56:01.450 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:01.445+0000 7f85c2d9d640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f85c4105b10 0x7f85c419fce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:33314/0 (socket says 172.21.15.39:33314) 2024-09-21T03:56:01.450 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:01.445+0000 7f85c2d9d640 1 -- 172.21.15.39:0/1602946257 learned_addr learned my addr 172.21.15.39:0/1602946257 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:56:01.450 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:01.445+0000 7f85c259c640 1 --2- 172.21.15.39:0/1602946257 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f85c41031f0 0x7f85c419a4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:56:01.450 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:01.446+0000 7f85c1d9b640 1 --2- 172.21.15.39:0/1602946257 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f85c41051c0 0x7f85c419aa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:56:01.450 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:01.446+0000 7f85c2d9d640 1 -- 172.21.15.39:0/1602946257 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f85c41031f0 msgr2=0x7f85c419a4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:56:01.450 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:01.446+0000 7f85c2d9d640 1 --2- 172.21.15.39:0/1602946257 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f85c41031f0 0x7f85c419a4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:01.450 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:01.446+0000 7f85c2d9d640 1 -- 172.21.15.39:0/1602946257 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f85c41051c0 msgr2=0x7f85c419aa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:56:01.450 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:01.446+0000 7f85c2d9d640 1 --2- 172.21.15.39:0/1602946257 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f85c41051c0 0x7f85c419aa40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:01.450 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:01.446+0000 7f85c2d9d640 1 -- 172.21.15.39:0/1602946257 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f85c41a0410 con 0x7f85c4105b10 2024-09-21T03:56:01.450 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:01.446+0000 7f85c259c640 1 --2- 172.21.15.39:0/1602946257 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f85c41031f0 0x7f85c419a4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:56:01.450 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:01.446+0000 7f85c2d9d640 1 --2- 172.21.15.39:0/1602946257 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f85c4105b10 0x7f85c419fce0 secure :-1 s=READY pgs=1262 cs=0 l=1 rev1=1 crypto rx=0x7f85bc00e970 tx=0x7f85bc00ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:56:01.451 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:01.446+0000 7f85b37fe640 1 -- 172.21.15.39:0/1602946257 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85bc00ccc0 con 0x7f85c4105b10 2024-09-21T03:56:01.451 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:01.446+0000 7f85b37fe640 1 -- 172.21.15.39:0/1602946257 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f85bc00ce60 con 0x7f85c4105b10 2024-09-21T03:56:01.451 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:01.446+0000 7f85c8a15640 1 -- 172.21.15.39:0/1602946257 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f85c41a0700 con 0x7f85c4105b10 2024-09-21T03:56:01.451 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:01.446+0000 7f85b37fe640 1 -- 172.21.15.39:0/1602946257 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85bc010430 con 0x7f85c4105b10 2024-09-21T03:56:01.451 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:01.446+0000 7f85c8a15640 1 -- 172.21.15.39:0/1602946257 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f85c41a6e60 con 0x7f85c4105b10 2024-09-21T03:56:01.451 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:01.449+0000 7f85cef89740 1 Processor -- start 2024-09-21T03:56:01.452 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:01.449+0000 7f85b37fe640 1 -- 172.21.15.39:0/1602946257 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 27) ==== 154643+0+0 (secure 0 0 0) 0x7f85bc0105d0 con 0x7f85c4105b10 2024-09-21T03:56:01.453 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:01.449+0000 7f85cef89740 1 -- start start 2024-09-21T03:56:01.453 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:01.449+0000 7f85cef89740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x560918580070 0x5609189276a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:56:01.454 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:01.450+0000 7f85cef89740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5609185571b0 0x560918927be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:56:01.454 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:01.450+0000 7f85cef89740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56091855f390 0x560918928120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:56:01.454 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:01.450+0000 7f85cef89740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x56091889dea0 con 0x560918580070 2024-09-21T03:56:01.454 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:01.450+0000 7f85cef89740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x56091888d2d0 con 0x5609185571b0 2024-09-21T03:56:01.454 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:01.450+0000 7f85cef89740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x5609188910e0 con 0x56091855f390 2024-09-21T03:56:01.454 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:01.450+0000 7f85c1d9b640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x560918580070 0x5609189276a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:56:01.454 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:01.450+0000 7f85c1d9b640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x560918580070 0x5609189276a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:33324/0 (socket says 172.21.15.39:33324) 2024-09-21T03:56:01.454 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:01.450+0000 7f85c1d9b640 1 -- 172.21.15.39:0/3650457241 learned_addr learned my addr 172.21.15.39:0/3650457241 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:56:01.454 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:01.450+0000 7f85c259c640 1 --2- 172.21.15.39:0/3650457241 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5609185571b0 0x560918927be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:56:01.454 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:01.450+0000 7f85c1d9b640 1 --2- 172.21.15.39:0/3650457241 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56091855f390 0x560918928120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:56:01.454 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:01.450+0000 7f85c1d9b640 1 -- 172.21.15.39:0/3650457241 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56091855f390 msgr2=0x560918928120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:56:01.454 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:01.450+0000 7f85c1d9b640 1 --2- 172.21.15.39:0/3650457241 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56091855f390 0x560918928120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:01.455 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:01.450+0000 7f85c1d9b640 1 -- 172.21.15.39:0/3650457241 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5609185571b0 msgr2=0x560918927be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:56:01.455 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:01.450+0000 7f85c1d9b640 1 --2- 172.21.15.39:0/3650457241 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5609185571b0 0x560918927be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:01.455 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:01.450+0000 7f85c1d9b640 1 -- 172.21.15.39:0/3650457241 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5609185eb650 con 0x560918580070 2024-09-21T03:56:01.455 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:01.450+0000 7f85b37fe640 1 --2- 172.21.15.39:0/1602946257 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f85ac0b6680 0x7f85ac0b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:56:01.455 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:01.450+0000 7f85c259c640 1 --2- 172.21.15.39:0/1602946257 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f85ac0b6680 0x7f85ac0b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:56:01.455 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:01.450+0000 7f85c1d9b640 1 --2- 172.21.15.39:0/3650457241 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x560918580070 0x5609189276a0 secure :-1 s=READY pgs=1263 cs=0 l=1 rev1=1 crypto rx=0x7f85b8005ab0 tx=0x7f85b8005e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=37934fd7f27158be server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:56:01.455 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:01.451+0000 7f85b37fe640 1 -- 172.21.15.39:0/1602946257 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8528+0+0 (secure 0 0 0) 0x7f85bc0e7ca0 con 0x7f85c4105b10 2024-09-21T03:56:01.455 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:01.451+0000 7f85b0ff9640 1 -- 172.21.15.39:0/3650457241 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85b8035050 con 0x560918580070 2024-09-21T03:56:01.455 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:01.451+0000 7f85b0ff9640 1 -- 172.21.15.39:0/3650457241 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f85b8004030 con 0x560918580070 2024-09-21T03:56:01.455 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:01.451+0000 7f85b0ff9640 1 -- 172.21.15.39:0/3650457241 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85b8005400 con 0x560918580070 2024-09-21T03:56:01.458 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:01.452+0000 7f85cef89740 1 -- 172.21.15.39:0/3650457241 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x560918580070 msgr2=0x5609189276a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:56:01.458 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:01.452+0000 7f85cef89740 1 --2- 172.21.15.39:0/3650457241 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x560918580070 0x5609189276a0 secure :-1 s=READY pgs=1263 cs=0 l=1 rev1=1 crypto rx=0x7f85b8005ab0 tx=0x7f85b8005e50 comp rx=0 tx=0).stop 2024-09-21T03:56:01.458 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:01.452+0000 7f85c259c640 1 --2- 172.21.15.39:0/1602946257 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f85ac0b6680 0x7f85ac0b8b90 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x560918926320 tx=0x7f85b4009880 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:56:01.458 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:01.452+0000 7f85cef89740 1 -- 172.21.15.39:0/3650457241 shutdown_connections 2024-09-21T03:56:01.458 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:01.452+0000 7f85cef89740 1 --2- 172.21.15.39:0/3650457241 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56091855f390 0x560918928120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:01.458 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:01.452+0000 7f85cef89740 1 --2- 172.21.15.39:0/3650457241 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5609185571b0 0x560918927be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:01.458 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:01.452+0000 7f85cef89740 1 --2- 172.21.15.39:0/3650457241 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x560918580070 0x5609189276a0 unknown :-1 s=CLOSED pgs=1263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:01.458 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:01.452+0000 7f85cef89740 1 -- 172.21.15.39:0/3650457241 >> 172.21.15.39:0/3650457241 conn(0x560918574460 msgr2=0x560918925590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:56:01.459 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:01.452+0000 7f85cef89740 1 -- 172.21.15.39:0/3650457241 shutdown_connections 2024-09-21T03:56:01.459 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:01.452+0000 7f85cef89740 1 -- 172.21.15.39:0/3650457241 wait complete. 2024-09-21T03:56:01.459 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:01.452+0000 7f85cef89740 1 -- start start 2024-09-21T03:56:01.459 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:01.452+0000 7f85cef89740 1 Processor -- start 2024-09-21T03:56:01.459 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:01.453+0000 7f85cef89740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5609185571b0 0x560918932d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:56:01.459 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:01.453+0000 7f85cef89740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56091855f390 0x560918933280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:56:01.459 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:01.453+0000 7f85cef89740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560918580070 0x560918933800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:56:01.459 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:01.453+0000 7f85cef89740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x5609186ee700 con 0x56091855f390 2024-09-21T03:56:01.459 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:01.453+0000 7f85cef89740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x560918828160 con 0x560918580070 2024-09-21T03:56:01.459 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:01.453+0000 7f85cef89740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x56091893a230 con 0x5609185571b0 2024-09-21T03:56:01.459 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:01.453+0000 7f85c1d9b640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56091855f390 0x560918933280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:56:01.459 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:01.453+0000 7f85cef89740 10 client.0 _ll_register_callbacks cb 0x5609188419e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:56:01.459 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:01.453+0000 7f85c1d9b640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56091855f390 0x560918933280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:33340/0 (socket says 172.21.15.39:33340) 2024-09-21T03:56:01.459 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:01.453+0000 7f85c1d9b640 1 -- 172.21.15.39:0/2601386803 learned_addr learned my addr 172.21.15.39:0/2601386803 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:56:01.460 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:01.453+0000 7f85c259c640 1 --2- 172.21.15.39:0/2601386803 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5609185571b0 0x560918932d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:56:01.460 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:01.453+0000 7f85c2d9d640 1 --2- 172.21.15.39:0/2601386803 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560918580070 0x560918933800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:56:01.460 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:01.453+0000 7f85c2d9d640 1 -- 172.21.15.39:0/2601386803 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5609185571b0 msgr2=0x560918932d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:56:01.460 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:01.453+0000 7f85c2d9d640 1 --2- 172.21.15.39:0/2601386803 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5609185571b0 0x560918932d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:01.460 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:01.453+0000 7f85c2d9d640 1 -- 172.21.15.39:0/2601386803 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56091855f390 msgr2=0x560918933280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:56:01.460 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:01.453+0000 7f85c2d9d640 1 --2- 172.21.15.39:0/2601386803 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56091855f390 0x560918933280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:01.460 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:01.453+0000 7f85c2d9d640 1 -- 172.21.15.39:0/2601386803 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560918855c90 con 0x560918580070 2024-09-21T03:56:01.460 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:01.454+0000 7f85c2d9d640 1 --2- 172.21.15.39:0/2601386803 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560918580070 0x560918933800 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f85bc0e53f0 tx=0x7f85bc0b7440 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3578c4ad418e5ccd server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:56:01.460 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:01.454+0000 7f85a9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.150:3300/0 2024-09-21T03:56:01.460 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:01.454+0000 7f85a9ffb640 1 -- 172.21.15.39:0/2601386803 <== mon.2 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85bc0b2370 con 0x560918580070 2024-09-21T03:56:01.460 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:01.454+0000 7f85a9ffb640 1 -- 172.21.15.39:0/2601386803 <== mon.2 v2:172.21.15.150:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8528+0+0 (secure 0 0 0) 0x7f85bc0e5b40 con 0x560918580070 2024-09-21T03:56:01.460 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:01.454+0000 7f85cef89740 1 -- 172.21.15.39:0/2601386803 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f85b8004030 con 0x560918580070 2024-09-21T03:56:01.460 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:01.454+0000 7f85a9ffb640 1 -- 172.21.15.39:0/2601386803 <== mon.2 v2:172.21.15.150:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f85bc0055f0 con 0x560918580070 2024-09-21T03:56:01.460 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:01.454+0000 7f85cef89740 10 client.27221 fetch_fsmap learned FSMap version 29 2024-09-21T03:56:01.460 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:01.454+0000 7f85cef89740 1 -- 172.21.15.39:0/2601386803 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x560918945f70 con 0x560918580070 2024-09-21T03:56:01.460 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:01.454+0000 7f85a9ffb640 1 -- 172.21.15.39:0/2601386803 <== mon.2 v2:172.21.15.150:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f85b8004030 con 0x560918580070 2024-09-21T03:56:01.461 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:01.454+0000 7f85a9ffb640 1 -- 172.21.15.39:0/2601386803 <== mon.2 v2:172.21.15.150:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f85bc0a9440 con 0x560918580070 2024-09-21T03:56:01.461 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:01.455+0000 7f85cef89740 10 client.27221 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T03:56:01.461 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:01.455+0000 7f85cef89740 10 client.27221 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T03:56:01.461 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:01.455+0000 7f85cef89740 20 client.27221 populate_metadata read hostname 'smithi039' 2024-09-21T03:56:01.461 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:01.455+0000 7f85cef89740 1 --2- 172.21.15.39:0/2601386803 >> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] conn(0x56091853e9f0 0x560918949490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:56:01.461 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:01.455+0000 7f85cef89740 4 client.27221 mds_command: new command op to 24695 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:56:01.461 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:01.455+0000 7f85cef89740 1 -- 172.21.15.39:0/2601386803 --> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56091889fdf0 con 0x56091853e9f0 2024-09-21T03:56:01.461 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:01.455+0000 7f85c259c640 1 --2- 172.21.15.39:0/2601386803 >> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] conn(0x56091853e9f0 0x560918949490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:56:01.461 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:01.455+0000 7f85a9ffb640 10 client.27221 ms_handle_connect on v2:172.21.15.150:6838/2949386907 2024-09-21T03:56:01.461 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:01.455+0000 7f85c259c640 1 --2- 172.21.15.39:0/2601386803 >> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] conn(0x56091853e9f0 0x560918949490 crc :-1 s=READY pgs=157 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6758385084532b05 server_cookie=2f669803b3726f6e in_seq=0 out_seq=0 2024-09-21T03:56:01.462 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:01.456+0000 7f85a9ffb640 1 -- 172.21.15.39:0/2601386803 <== mds.0 v2:172.21.15.150:6838/2949386907 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x56091889fdf0 con 0x56091853e9f0 2024-09-21T03:56:01.462 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:01.456+0000 7f85a9ffb640 10 client.27221 handle_command_reply: tid=0 2024-09-21T03:56:01.462 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:01.456+0000 7f85cef89740 1 client.27221 shutdown 2024-09-21T03:56:01.463 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:01.456+0000 7f85cef89740 1 -- 172.21.15.39:0/2601386803 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560918580070 msgr2=0x560918933800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:56:01.463 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:01.456+0000 7f85cef89740 1 --2- 172.21.15.39:0/2601386803 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560918580070 0x560918933800 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f85bc0e53f0 tx=0x7f85bc0b7440 comp rx=0 tx=0).stop 2024-09-21T03:56:01.463 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:01.456+0000 7f85cef89740 1 -- 172.21.15.39:0/2601386803 shutdown_connections 2024-09-21T03:56:01.463 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:01.456+0000 7f85cef89740 1 --2- 172.21.15.39:0/2601386803 >> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] conn(0x56091853e9f0 0x560918949490 crc :-1 s=READY pgs=157 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:01.463 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:01.456+0000 7f85cef89740 1 --2- 172.21.15.39:0/2601386803 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560918580070 0x560918933800 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:01.463 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:01.456+0000 7f85cef89740 1 --2- 172.21.15.39:0/2601386803 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56091855f390 0x560918933280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:01.463 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:01.456+0000 7f85a9ffb640 0 client.27221 ms_handle_reset on v2:172.21.15.150:6838/2949386907 2024-09-21T03:56:01.463 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:01.456+0000 7f85cef89740 1 --2- 172.21.15.39:0/2601386803 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5609185571b0 0x560918932d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:01.463 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:01.456+0000 7f85cef89740 1 -- 172.21.15.39:0/2601386803 >> 172.21.15.39:0/2601386803 conn(0x560918574460 msgr2=0x56091892a880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:56:01.463 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:01.457+0000 7f85cef89740 1 -- 172.21.15.39:0/2601386803 shutdown_connections 2024-09-21T03:56:01.463 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:01.457+0000 7f85cef89740 1 -- 172.21.15.39:0/2601386803 wait complete. 2024-09-21T03:56:01.463 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:01.457+0000 7f85cef89740 20 client.27221 trim_cache size 0 max 16384 2024-09-21T03:56:01.468 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:56:01 smithi161 ceph-mon[34533]: pgmap v630: 97 pgs: 97 active+clean; 10 MiB data, 525 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-09-21T03:56:01.473 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:01.470+0000 7f85cef89740 1 Processor -- start 2024-09-21T03:56:01.473 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:01.470+0000 7f85cef89740 1 -- start start 2024-09-21T03:56:01.474 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:01.470+0000 7f85cef89740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5609185571b0 0x56091892aeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:56:01.474 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:01.470+0000 7f85cef89740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56091855f390 0x56091892ff30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:56:01.474 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:01.470+0000 7f85cef89740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560918580070 0x560918930470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:56:01.474 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:01.470+0000 7f85cef89740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f85c41ad460 con 0x5609185571b0 2024-09-21T03:56:01.474 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:01.470+0000 7f85cef89740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f85c41aefa0 con 0x560918580070 2024-09-21T03:56:01.474 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:01.470+0000 7f85cef89740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f85c41adb30 con 0x56091855f390 2024-09-21T03:56:01.474 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:01.470+0000 7f85c259c640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5609185571b0 0x56091892aeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:56:01.474 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:01.470+0000 7f85c259c640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5609185571b0 0x56091892aeb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:33356/0 (socket says 172.21.15.39:33356) 2024-09-21T03:56:01.474 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:01.470+0000 7f85c259c640 1 -- 172.21.15.39:0/4082560760 learned_addr learned my addr 172.21.15.39:0/4082560760 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:56:01.474 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:01.470+0000 7f85c1d9b640 1 --2- 172.21.15.39:0/4082560760 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56091855f390 0x56091892ff30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:56:01.474 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:01.470+0000 7f85c2d9d640 1 --2- 172.21.15.39:0/4082560760 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560918580070 0x560918930470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:56:01.474 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:01.470+0000 7f85c259c640 1 -- 172.21.15.39:0/4082560760 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56091855f390 msgr2=0x56091892ff30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:56:01.474 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:01.470+0000 7f85c259c640 1 --2- 172.21.15.39:0/4082560760 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56091855f390 0x56091892ff30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:01.475 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:01.470+0000 7f85c259c640 1 -- 172.21.15.39:0/4082560760 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560918580070 msgr2=0x560918930470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:56:01.475 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:01.470+0000 7f85c259c640 1 --2- 172.21.15.39:0/4082560760 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560918580070 0x560918930470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:01.475 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:01.471+0000 7f85c259c640 1 -- 172.21.15.39:0/4082560760 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56091892b3f0 con 0x5609185571b0 2024-09-21T03:56:01.475 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:01.471+0000 7f85c259c640 1 --2- 172.21.15.39:0/4082560760 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5609185571b0 0x56091892aeb0 secure :-1 s=READY pgs=1264 cs=0 l=1 rev1=1 crypto rx=0x7f85c406bb50 tx=0x7f85b4016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=597583edfe7c4f28 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:56:01.475 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:01.471+0000 7f85abfff640 1 -- 172.21.15.39:0/4082560760 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85b4032430 con 0x5609185571b0 2024-09-21T03:56:01.475 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:01.471+0000 7f85abfff640 1 -- 172.21.15.39:0/4082560760 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f85b4032a80 con 0x5609185571b0 2024-09-21T03:56:01.475 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:01.471+0000 7f85abfff640 1 -- 172.21.15.39:0/4082560760 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85b4020950 con 0x5609185571b0 2024-09-21T03:56:01.476 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:56:01 smithi150 ceph-mon[34289]: pgmap v630: 97 pgs: 97 active+clean; 10 MiB data, 525 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-09-21T03:56:01.479 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:01.472+0000 7f85cef89740 1 -- 172.21.15.39:0/4082560760 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5609185571b0 msgr2=0x56091892aeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:56:01.479 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:01.472+0000 7f85cef89740 1 --2- 172.21.15.39:0/4082560760 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5609185571b0 0x56091892aeb0 secure :-1 s=READY pgs=1264 cs=0 l=1 rev1=1 crypto rx=0x7f85c406bb50 tx=0x7f85b4016000 comp rx=0 tx=0).stop 2024-09-21T03:56:01.479 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:01.472+0000 7f85cef89740 1 -- 172.21.15.39:0/4082560760 shutdown_connections 2024-09-21T03:56:01.479 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:01.472+0000 7f85cef89740 1 --2- 172.21.15.39:0/4082560760 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560918580070 0x560918930470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:01.479 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:01.472+0000 7f85cef89740 1 --2- 172.21.15.39:0/4082560760 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56091855f390 0x56091892ff30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:01.479 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:01.472+0000 7f85cef89740 1 --2- 172.21.15.39:0/4082560760 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5609185571b0 0x56091892aeb0 unknown :-1 s=CLOSED pgs=1264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:01.479 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:01.472+0000 7f85cef89740 1 -- 172.21.15.39:0/4082560760 >> 172.21.15.39:0/4082560760 conn(0x560918574460 msgr2=0x56091892a7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:56:01.479 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:01.472+0000 7f85cef89740 1 -- 172.21.15.39:0/4082560760 shutdown_connections 2024-09-21T03:56:01.479 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:01.472+0000 7f85cef89740 1 -- 172.21.15.39:0/4082560760 wait complete. 2024-09-21T03:56:01.479 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:01.472+0000 7f85cef89740 1 -- start start 2024-09-21T03:56:01.480 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:01.472+0000 7f85cef89740 1 Processor -- start 2024-09-21T03:56:01.480 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:01.473+0000 7f85cef89740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5609185571b0 0x56091893bb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:56:01.480 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:01.473+0000 7f85cef89740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56091855f390 0x56091893c0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:56:01.480 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:01.473+0000 7f85cef89740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560918580070 0x5609189b2e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:56:01.480 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:01.473+0000 7f85cef89740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f85c41ae8d0 con 0x5609185571b0 2024-09-21T03:56:01.480 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:01.473+0000 7f85cef89740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f85c41ae200 con 0x560918580070 2024-09-21T03:56:01.480 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:01.473+0000 7f85cef89740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f85c41ab440 con 0x56091855f390 2024-09-21T03:56:01.480 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:01.473+0000 7f85cef89740 10 client.0 _ll_register_callbacks cb 0x7f85bc0048b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:56:01.480 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:01.473+0000 7f85c259c640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5609185571b0 0x56091893bb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:56:01.480 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:01.473+0000 7f85c1d9b640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56091855f390 0x56091893c0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:56:01.480 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:01.473+0000 7f85c259c640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5609185571b0 0x56091893bb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:33362/0 (socket says 172.21.15.39:33362) 2024-09-21T03:56:01.480 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:01.473+0000 7f85c259c640 1 -- 172.21.15.39:0/849882006 learned_addr learned my addr 172.21.15.39:0/849882006 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:56:01.480 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:01.473+0000 7f85c2d9d640 1 --2- 172.21.15.39:0/849882006 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560918580070 0x5609189b2e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:56:01.480 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:01.473+0000 7f85c259c640 1 -- 172.21.15.39:0/849882006 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56091855f390 msgr2=0x56091893c0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:56:01.480 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:01.473+0000 7f85c259c640 1 --2- 172.21.15.39:0/849882006 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56091855f390 0x56091893c0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:01.481 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:01.473+0000 7f85c259c640 1 -- 172.21.15.39:0/849882006 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560918580070 msgr2=0x5609189b2e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:56:01.481 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:01.473+0000 7f85c259c640 1 --2- 172.21.15.39:0/849882006 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560918580070 0x5609189b2e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:01.481 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:01.473+0000 7f85c259c640 1 -- 172.21.15.39:0/849882006 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5609189b33d0 con 0x5609185571b0 2024-09-21T03:56:01.481 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:01.473+0000 7f85c2d9d640 1 --2- 172.21.15.39:0/849882006 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560918580070 0x5609189b2e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:56:01.481 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:01.473+0000 7f85c259c640 1 --2- 172.21.15.39:0/849882006 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5609185571b0 0x56091893bb60 secure :-1 s=READY pgs=1265 cs=0 l=1 rev1=1 crypto rx=0x7f85b4011810 tx=0x7f85b4011fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a15ff2c1bef7e327 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:56:01.481 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:01.473+0000 7f85a9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-09-21T03:56:01.481 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:01.474+0000 7f85a9ffb640 1 -- 172.21.15.39:0/849882006 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85b403f020 con 0x5609185571b0 2024-09-21T03:56:01.481 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:01.474+0000 7f85a9ffb640 1 -- 172.21.15.39:0/849882006 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8528+0+0 (secure 0 0 0) 0x7f85b4038040 con 0x5609185571b0 2024-09-21T03:56:01.481 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:01.474+0000 7f85cef89740 1 -- 172.21.15.39:0/849882006 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f85b4032a80 con 0x5609185571b0 2024-09-21T03:56:01.481 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:01.474+0000 7f85a9ffb640 1 -- 172.21.15.39:0/849882006 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f85b4004c50 con 0x5609185571b0 2024-09-21T03:56:01.481 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:01.474+0000 7f85cef89740 10 client.18102 fetch_fsmap learned FSMap version 29 2024-09-21T03:56:01.481 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:01.474+0000 7f85cef89740 1 -- 172.21.15.39:0/849882006 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5609189b45b0 con 0x5609185571b0 2024-09-21T03:56:01.481 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:01.474+0000 7f85a9ffb640 1 -- 172.21.15.39:0/849882006 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f85b4032a80 con 0x5609185571b0 2024-09-21T03:56:01.481 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:01.474+0000 7f85a9ffb640 1 -- 172.21.15.39:0/849882006 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f85b400f040 con 0x5609185571b0 2024-09-21T03:56:01.481 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:01.474+0000 7f85cef89740 10 client.18102 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T03:56:01.482 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:01.474+0000 7f85cef89740 10 client.18102 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T03:56:01.482 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:01.474+0000 7f85cef89740 20 client.18102 populate_metadata read hostname 'smithi039' 2024-09-21T03:56:01.482 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:01.474+0000 7f85cef89740 1 --2- 172.21.15.39:0/849882006 >> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] conn(0x560918513da0 0x5609189b6d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:56:01.482 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:01.474+0000 7f85cef89740 4 client.18102 mds_command: new command op to 24695 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T03:56:01.482 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:01.474+0000 7f85cef89740 1 -- 172.21.15.39:0/849882006 --> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56091889b8a0 con 0x560918513da0 2024-09-21T03:56:01.482 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:01.475+0000 7f85c1d9b640 1 --2- 172.21.15.39:0/849882006 >> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] conn(0x560918513da0 0x5609189b6d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:56:01.482 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:01.475+0000 7f85c1d9b640 1 --2- 172.21.15.39:0/849882006 >> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] conn(0x560918513da0 0x5609189b6d90 crc :-1 s=READY pgs=158 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2df24c9ec53ff3be server_cookie=1b225882eb38fc5a in_seq=0 out_seq=0 2024-09-21T03:56:01.482 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:01.475+0000 7f85a9ffb640 10 client.18102 ms_handle_connect on v2:172.21.15.150:6838/2949386907 2024-09-21T03:56:01.482 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:01.475+0000 7f85a9ffb640 1 -- 172.21.15.39:0/849882006 <== mds.0 v2:172.21.15.150:6838/2949386907 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56091889b8a0 con 0x560918513da0 2024-09-21T03:56:01.482 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:01.475+0000 7f85a9ffb640 10 client.18102 handle_command_reply: tid=0 2024-09-21T03:56:01.482 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:01.475+0000 7f85cef89740 1 client.18102 shutdown 2024-09-21T03:56:01.482 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:01.475+0000 7f85cef89740 1 -- 172.21.15.39:0/849882006 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5609185571b0 msgr2=0x56091893bb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:56:01.482 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:01.475+0000 7f85cef89740 1 --2- 172.21.15.39:0/849882006 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5609185571b0 0x56091893bb60 secure :-1 s=READY pgs=1265 cs=0 l=1 rev1=1 crypto rx=0x7f85b4011810 tx=0x7f85b4011fb0 comp rx=0 tx=0).stop 2024-09-21T03:56:01.482 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:01.475+0000 7f85cef89740 1 -- 172.21.15.39:0/849882006 shutdown_connections 2024-09-21T03:56:01.483 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:01.475+0000 7f85cef89740 1 --2- 172.21.15.39:0/849882006 >> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] conn(0x560918513da0 0x5609189b6d90 crc :-1 s=READY pgs=158 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:01.483 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:01.475+0000 7f85cef89740 1 --2- 172.21.15.39:0/849882006 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560918580070 0x5609189b2e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:01.483 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:01.476+0000 7f85cef89740 1 --2- 172.21.15.39:0/849882006 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56091855f390 0x56091893c0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:01.483 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:01.476+0000 7f85cef89740 1 --2- 172.21.15.39:0/849882006 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5609185571b0 0x56091893bb60 unknown :-1 s=CLOSED pgs=1265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:01.483 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:01.476+0000 7f85cef89740 1 -- 172.21.15.39:0/849882006 >> 172.21.15.39:0/849882006 conn(0x560918574460 msgr2=0x560918942470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:56:01.483 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:01.476+0000 7f85a9ffb640 0 client.18102 ms_handle_reset on v2:172.21.15.150:6838/2949386907 2024-09-21T03:56:01.483 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:01.476+0000 7f85cef89740 1 -- 172.21.15.39:0/849882006 shutdown_connections 2024-09-21T03:56:01.483 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:01.476+0000 7f85cef89740 1 -- 172.21.15.39:0/849882006 wait complete. 2024-09-21T03:56:01.483 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:01.476+0000 7f85cef89740 20 client.18102 trim_cache size 0 max 16384 2024-09-21T03:56:01.483 INFO:teuthology.orchestra.run.smithi039.stdout:[] 2024-09-21T03:56:01.484 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:01.476+0000 7f85c8a15640 1 -- 172.21.15.39:0/1602946257 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f85ac0b6680 msgr2=0x7f85ac0b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:56:01.484 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:01.476+0000 7f85c8a15640 1 --2- 172.21.15.39:0/1602946257 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f85ac0b6680 0x7f85ac0b8b90 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x560918926320 tx=0x7f85b4009880 comp rx=0 tx=0).stop 2024-09-21T03:56:01.484 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:01.476+0000 7f85c8a15640 1 -- 172.21.15.39:0/1602946257 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f85c4105b10 msgr2=0x7f85c419fce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:56:01.484 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:01.476+0000 7f85c8a15640 1 --2- 172.21.15.39:0/1602946257 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f85c4105b10 0x7f85c419fce0 secure :-1 s=READY pgs=1262 cs=0 l=1 rev1=1 crypto rx=0x7f85bc00e970 tx=0x7f85bc00ed70 comp rx=0 tx=0).stop 2024-09-21T03:56:01.484 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:01.476+0000 7f85c8a15640 1 -- 172.21.15.39:0/1602946257 shutdown_connections 2024-09-21T03:56:01.484 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:01.476+0000 7f85c8a15640 1 --2- 172.21.15.39:0/1602946257 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f85ac0b6680 0x7f85ac0b8b90 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:01.484 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:01.476+0000 7f85c8a15640 1 --2- 172.21.15.39:0/1602946257 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f85c4105b10 0x7f85c419fce0 unknown :-1 s=CLOSED pgs=1262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:01.485 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:01.476+0000 7f85c8a15640 1 --2- 172.21.15.39:0/1602946257 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f85c41051c0 0x7f85c419aa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:01.485 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:01.476+0000 7f85c8a15640 1 --2- 172.21.15.39:0/1602946257 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f85c41031f0 0x7f85c419a4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:01.485 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:01.476+0000 7f85c8a15640 1 -- 172.21.15.39:0/1602946257 >> 172.21.15.39:0/1602946257 conn(0x7f85c4079e40 msgr2=0x7f85c41064f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:56:01.485 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:01.477+0000 7f85c8a15640 1 -- 172.21.15.39:0/1602946257 shutdown_connections 2024-09-21T03:56:01.485 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:01.477+0000 7f85c8a15640 1 -- 172.21.15.39:0/1602946257 wait complete. 2024-09-21T03:56:03.334 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:56:03 smithi039 ceph-mon[31211]: pgmap v631: 97 pgs: 97 active+clean; 10 MiB data, 525 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-21T03:56:03.382 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-21T03:56:03.383 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 51116d5c-77ca-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-21T03:56:03.468 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:56:03 smithi161 ceph-mon[34533]: pgmap v631: 97 pgs: 97 active+clean; 10 MiB data, 525 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-21T03:56:03.474 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:56:03 smithi150 ceph-mon[34289]: pgmap v631: 97 pgs: 97 active+clean; 10 MiB data, 525 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-21T03:56:03.654 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/51116d5c-77ca-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:56:04.125 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:04.122+0000 7f5739da1640 1 -- 172.21.15.39:0/2300045184 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5734103b60 msgr2=0x7f5734103f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:56:04.125 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:04.122+0000 7f5739da1640 1 --2- 172.21.15.39:0/2300045184 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5734103b60 0x7f5734103f70 secure :-1 s=READY pgs=1266 cs=0 l=1 rev1=1 crypto rx=0x7f5724009900 tx=0x7f572402a3f0 comp rx=0 tx=0).stop 2024-09-21T03:56:04.125 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:04.122+0000 7f5739da1640 1 -- 172.21.15.39:0/2300045184 shutdown_connections 2024-09-21T03:56:04.125 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:04.122+0000 7f5739da1640 1 --2- 172.21.15.39:0/2300045184 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5734109df0 0x7f5734110620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:04.125 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:04.122+0000 7f5739da1640 1 --2- 172.21.15.39:0/2300045184 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f57341094a0 0x7f57341098b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:04.125 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:04.122+0000 7f5739da1640 1 --2- 172.21.15.39:0/2300045184 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5734103b60 0x7f5734103f70 unknown :-1 s=CLOSED pgs=1266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:04.125 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:04.122+0000 7f5739da1640 1 -- 172.21.15.39:0/2300045184 >> 172.21.15.39:0/2300045184 conn(0x7f573407b850 msgr2=0x7f573407bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:56:04.125 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:04.122+0000 7f5739da1640 1 -- 172.21.15.39:0/2300045184 shutdown_connections 2024-09-21T03:56:04.126 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:04.122+0000 7f5739da1640 1 -- 172.21.15.39:0/2300045184 wait complete. 2024-09-21T03:56:04.127 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:04.123+0000 7f5739da1640 1 Processor -- start 2024-09-21T03:56:04.127 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:04.123+0000 7f5739da1640 1 -- start start 2024-09-21T03:56:04.127 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:04.123+0000 7f5739da1640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5734103b60 0x7f573419a500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:56:04.127 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:04.123+0000 7f5739da1640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f57341094a0 0x7f573419aaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:56:04.127 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:04.123+0000 7f5739da1640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5734109df0 0x7f573419fd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:56:04.127 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:04.123+0000 7f5739da1640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f5734113ab0 con 0x7f57341094a0 2024-09-21T03:56:04.127 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:04.123+0000 7f5739da1640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f5734113930 con 0x7f5734103b60 2024-09-21T03:56:04.128 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:04.123+0000 7f5739da1640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f5734113c30 con 0x7f5734109df0 2024-09-21T03:56:04.128 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:04.123+0000 7f5732ffd640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f57341094a0 0x7f573419aaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:56:04.128 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:04.123+0000 7f5732ffd640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f57341094a0 0x7f573419aaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:33368/0 (socket says 172.21.15.39:33368) 2024-09-21T03:56:04.128 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:04.123+0000 7f5732ffd640 1 -- 172.21.15.39:0/3738726024 learned_addr learned my addr 172.21.15.39:0/3738726024 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:56:04.128 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:04.123+0000 7f5733fff640 1 --2- 172.21.15.39:0/3738726024 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5734109df0 0x7f573419fd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:56:04.128 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:04.123+0000 7f57337fe640 1 --2- 172.21.15.39:0/3738726024 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5734103b60 0x7f573419a500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:56:04.128 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:04.124+0000 7f5732ffd640 1 -- 172.21.15.39:0/3738726024 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5734109df0 msgr2=0x7f573419fd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:56:04.128 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:04.124+0000 7f5732ffd640 1 --2- 172.21.15.39:0/3738726024 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5734109df0 0x7f573419fd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:04.128 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:04.124+0000 7f5732ffd640 1 -- 172.21.15.39:0/3738726024 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5734103b60 msgr2=0x7f573419a500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:56:04.128 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:04.124+0000 7f5732ffd640 1 --2- 172.21.15.39:0/3738726024 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5734103b60 0x7f573419a500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:04.128 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:04.124+0000 7f5732ffd640 1 -- 172.21.15.39:0/3738726024 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f57341a03b0 con 0x7f57341094a0 2024-09-21T03:56:04.128 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:04.124+0000 7f5732ffd640 1 --2- 172.21.15.39:0/3738726024 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f57341094a0 0x7f573419aaa0 secure :-1 s=READY pgs=1267 cs=0 l=1 rev1=1 crypto rx=0x7f572800c970 tx=0x7f572800ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:56:04.128 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:04.124+0000 7f5730ff9640 1 -- 172.21.15.39:0/3738726024 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5728007bb0 con 0x7f57341094a0 2024-09-21T03:56:04.128 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:04.124+0000 7f5730ff9640 1 -- 172.21.15.39:0/3738726024 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5728007d50 con 0x7f57341094a0 2024-09-21T03:56:04.128 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:04.124+0000 7f5739da1640 1 -- 172.21.15.39:0/3738726024 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f57341a06a0 con 0x7f57341094a0 2024-09-21T03:56:04.129 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:04.124+0000 7f5730ff9640 1 -- 172.21.15.39:0/3738726024 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f572800f450 con 0x7f57341094a0 2024-09-21T03:56:04.129 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:04.124+0000 7f5739da1640 1 -- 172.21.15.39:0/3738726024 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f57340755f0 con 0x7f57341094a0 2024-09-21T03:56:04.130 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:04.127+0000 7f5730ff9640 1 -- 172.21.15.39:0/3738726024 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 27) ==== 154643+0+0 (secure 0 0 0) 0x7f572800f5f0 con 0x7f57341094a0 2024-09-21T03:56:04.130 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:04.127+0000 7f5740315740 1 Processor -- start 2024-09-21T03:56:04.130 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:04.127+0000 7f5740315740 1 -- start start 2024-09-21T03:56:04.132 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:04.127+0000 7f5740315740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56102d306350 0x56102d6ad430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:56:04.132 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:04.127+0000 7f5740315740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56102d2dd490 0x56102d6ad970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:56:04.132 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:04.127+0000 7f5740315740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56102d2e5670 0x56102d6adeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:56:04.132 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:04.127+0000 7f5740315740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x56102d623e60 con 0x56102d2dd490 2024-09-21T03:56:04.132 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:04.127+0000 7f5740315740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x56102d613290 con 0x56102d2e5670 2024-09-21T03:56:04.132 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:04.127+0000 7f5740315740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x56102d6170a0 con 0x56102d306350 2024-09-21T03:56:04.132 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:04.128+0000 7f57337fe640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56102d2dd490 0x56102d6ad970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:56:04.132 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:04.128+0000 7f57337fe640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56102d2dd490 0x56102d6ad970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:33380/0 (socket says 172.21.15.39:33380) 2024-09-21T03:56:04.132 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:04.128+0000 7f57337fe640 1 -- 172.21.15.39:0/820439976 learned_addr learned my addr 172.21.15.39:0/820439976 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:56:04.132 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:04.128+0000 7f5732ffd640 1 --2- 172.21.15.39:0/820439976 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56102d2e5670 0x56102d6adeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:56:04.132 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:04.128+0000 7f5733fff640 1 --2- 172.21.15.39:0/820439976 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56102d306350 0x56102d6ad430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:56:04.132 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:04.128+0000 7f5730ff9640 1 --2- 172.21.15.39:0/3738726024 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f571c0b6730 0x7f571c0b8c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:56:04.132 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:04.128+0000 7f57337fe640 1 -- 172.21.15.39:0/820439976 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56102d306350 msgr2=0x56102d6ad430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:56:04.133 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:04.128+0000 7f57337fe640 1 --2- 172.21.15.39:0/820439976 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56102d306350 0x56102d6ad430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:04.133 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:04.128+0000 7f57337fe640 1 -- 172.21.15.39:0/820439976 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56102d2e5670 msgr2=0x56102d6adeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:56:04.133 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:04.128+0000 7f57337fe640 1 --2- 172.21.15.39:0/820439976 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56102d2e5670 0x56102d6adeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:04.133 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:04.128+0000 7f57337fe640 1 -- 172.21.15.39:0/820439976 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56102d371930 con 0x56102d2dd490 2024-09-21T03:56:04.133 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:04.128+0000 7f5733fff640 1 --2- 172.21.15.39:0/3738726024 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f571c0b6730 0x7f571c0b8c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:56:04.133 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:04.128+0000 7f5730ff9640 1 -- 172.21.15.39:0/3738726024 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8528+0+0 (secure 0 0 0) 0x7f57280e74e0 con 0x7f57341094a0 2024-09-21T03:56:04.133 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:04.129+0000 7f57337fe640 1 --2- 172.21.15.39:0/820439976 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56102d2dd490 0x56102d6ad970 secure :-1 s=READY pgs=1268 cs=0 l=1 rev1=1 crypto rx=0x7f572402a910 tx=0x7f5724031cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=87543831371ba785 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:56:04.133 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:04.129+0000 7f5719ffb640 1 -- 172.21.15.39:0/820439976 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f572403f070 con 0x56102d2dd490 2024-09-21T03:56:04.133 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:04.129+0000 7f5719ffb640 1 -- 172.21.15.39:0/820439976 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f57240316a0 con 0x56102d2dd490 2024-09-21T03:56:04.133 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:04.129+0000 7f5719ffb640 1 -- 172.21.15.39:0/820439976 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5724035070 con 0x56102d2dd490 2024-09-21T03:56:04.136 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:04.130+0000 7f5740315740 1 -- 172.21.15.39:0/820439976 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56102d2dd490 msgr2=0x56102d6ad970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:56:04.136 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:04.130+0000 7f5740315740 1 --2- 172.21.15.39:0/820439976 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56102d2dd490 0x56102d6ad970 secure :-1 s=READY pgs=1268 cs=0 l=1 rev1=1 crypto rx=0x7f572402a910 tx=0x7f5724031cd0 comp rx=0 tx=0).stop 2024-09-21T03:56:04.136 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:04.130+0000 7f5733fff640 1 --2- 172.21.15.39:0/3738726024 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f571c0b6730 0x7f571c0b8c40 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f572c007dd0 tx=0x7f572c004a40 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:56:04.136 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:04.130+0000 7f5740315740 1 -- 172.21.15.39:0/820439976 shutdown_connections 2024-09-21T03:56:04.136 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:04.130+0000 7f5740315740 1 --2- 172.21.15.39:0/820439976 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56102d2e5670 0x56102d6adeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:04.136 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:04.130+0000 7f5740315740 1 --2- 172.21.15.39:0/820439976 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56102d2dd490 0x56102d6ad970 unknown :-1 s=CLOSED pgs=1268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:04.136 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:04.130+0000 7f5740315740 1 --2- 172.21.15.39:0/820439976 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56102d306350 0x56102d6ad430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:04.136 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:04.130+0000 7f5740315740 1 -- 172.21.15.39:0/820439976 >> 172.21.15.39:0/820439976 conn(0x56102d2fa740 msgr2=0x56102d6ab320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:56:04.136 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:04.130+0000 7f5740315740 1 -- 172.21.15.39:0/820439976 shutdown_connections 2024-09-21T03:56:04.137 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:04.130+0000 7f5740315740 1 -- 172.21.15.39:0/820439976 wait complete. 2024-09-21T03:56:04.137 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:04.130+0000 7f5740315740 1 -- start start 2024-09-21T03:56:04.137 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:04.130+0000 7f5740315740 1 Processor -- start 2024-09-21T03:56:04.137 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:04.131+0000 7f5740315740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56102d2dd490 0x56102d6b8ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:56:04.137 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:04.131+0000 7f5740315740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56102d2e5670 0x56102d6b9530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:56:04.137 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:04.131+0000 7f5740315740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56102d306350 0x56102d6c4ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:56:04.137 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:04.131+0000 7f5740315740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x56102d474890 con 0x56102d306350 2024-09-21T03:56:04.137 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:04.131+0000 7f5740315740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x56102d5add40 con 0x56102d2dd490 2024-09-21T03:56:04.137 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:04.131+0000 7f5740315740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x56102d6bbbb0 con 0x56102d2e5670 2024-09-21T03:56:04.137 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:04.131+0000 7f5740315740 10 client.0 _ll_register_callbacks cb 0x56102d611c80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:56:04.137 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:04.131+0000 7f5733fff640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56102d306350 0x56102d6c4ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:56:04.137 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:04.131+0000 7f5733fff640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56102d306350 0x56102d6c4ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:33390/0 (socket says 172.21.15.39:33390) 2024-09-21T03:56:04.137 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:04.131+0000 7f5733fff640 1 -- 172.21.15.39:0/3546698116 learned_addr learned my addr 172.21.15.39:0/3546698116 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:56:04.137 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:04.131+0000 7f57337fe640 1 --2- 172.21.15.39:0/3546698116 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56102d2dd490 0x56102d6b8ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:56:04.137 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:04.131+0000 7f5732ffd640 1 --2- 172.21.15.39:0/3546698116 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56102d2e5670 0x56102d6b9530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:56:04.138 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:04.131+0000 7f5733fff640 1 -- 172.21.15.39:0/3546698116 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56102d2e5670 msgr2=0x56102d6b9530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:56:04.138 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:04.131+0000 7f5733fff640 1 --2- 172.21.15.39:0/3546698116 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56102d2e5670 0x56102d6b9530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:04.138 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:04.131+0000 7f5733fff640 1 -- 172.21.15.39:0/3546698116 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56102d2dd490 msgr2=0x56102d6b8ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:56:04.138 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:04.131+0000 7f5733fff640 1 --2- 172.21.15.39:0/3546698116 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56102d2dd490 0x56102d6b8ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:04.138 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:04.131+0000 7f5733fff640 1 -- 172.21.15.39:0/3546698116 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56102d6c5490 con 0x56102d306350 2024-09-21T03:56:04.138 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:04.132+0000 7f5733fff640 1 --2- 172.21.15.39:0/3546698116 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56102d306350 0x56102d6c4ea0 secure :-1 s=READY pgs=1269 cs=0 l=1 rev1=1 crypto rx=0x7f572c005c00 tx=0x7f572c007470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9bd1ab87d56498b0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:56:04.138 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:04.132+0000 7f5712ffd640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-09-21T03:56:04.138 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:04.132+0000 7f5712ffd640 1 -- 172.21.15.39:0/3546698116 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f572c01dd00 con 0x56102d306350 2024-09-21T03:56:04.138 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:04.132+0000 7f5712ffd640 1 -- 172.21.15.39:0/3546698116 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8528+0+0 (secure 0 0 0) 0x7f572c01b450 con 0x56102d306350 2024-09-21T03:56:04.138 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:04.132+0000 7f5740315740 1 -- 172.21.15.39:0/3546698116 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f57240316a0 con 0x56102d306350 2024-09-21T03:56:04.138 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:04.132+0000 7f5712ffd640 1 -- 172.21.15.39:0/3546698116 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f572c01b650 con 0x56102d306350 2024-09-21T03:56:04.138 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:04.132+0000 7f5740315740 10 client.18114 fetch_fsmap learned FSMap version 29 2024-09-21T03:56:04.138 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:04.132+0000 7f5740315740 1 -- 172.21.15.39:0/3546698116 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x56102d6c7e00 con 0x56102d306350 2024-09-21T03:56:04.138 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:04.132+0000 7f5712ffd640 1 -- 172.21.15.39:0/3546698116 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f57240316a0 con 0x56102d306350 2024-09-21T03:56:04.139 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:04.132+0000 7f5712ffd640 1 -- 172.21.15.39:0/3546698116 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f572c024070 con 0x56102d306350 2024-09-21T03:56:04.139 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:04.132+0000 7f5740315740 10 client.18114 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T03:56:04.139 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:04.132+0000 7f5740315740 10 client.18114 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T03:56:04.139 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:04.133+0000 7f5740315740 20 client.18114 populate_metadata read hostname 'smithi039' 2024-09-21T03:56:04.139 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:04.133+0000 7f5740315740 1 --2- 172.21.15.39:0/3546698116 >> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] conn(0x56102d2c4cd0 0x56102d6cb400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:56:04.139 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:04.133+0000 7f5740315740 4 client.18114 mds_command: new command op to 24695 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:56:04.139 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:04.133+0000 7f5740315740 1 -- 172.21.15.39:0/3546698116 --> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56102d625db0 con 0x56102d2c4cd0 2024-09-21T03:56:04.139 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:04.133+0000 7f57337fe640 1 --2- 172.21.15.39:0/3546698116 >> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] conn(0x56102d2c4cd0 0x56102d6cb400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:56:04.139 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:04.133+0000 7f5712ffd640 10 client.18114 ms_handle_connect on v2:172.21.15.150:6838/2949386907 2024-09-21T03:56:04.139 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:04.133+0000 7f57337fe640 1 --2- 172.21.15.39:0/3546698116 >> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] conn(0x56102d2c4cd0 0x56102d6cb400 crc :-1 s=READY pgs=159 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=de85498a072c9647 server_cookie=7e169395800df34b in_seq=0 out_seq=0 2024-09-21T03:56:04.140 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:04.134+0000 7f5712ffd640 1 -- 172.21.15.39:0/3546698116 <== mds.0 v2:172.21.15.150:6838/2949386907 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x56102d625db0 con 0x56102d2c4cd0 2024-09-21T03:56:04.140 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:04.134+0000 7f5712ffd640 10 client.18114 handle_command_reply: tid=0 2024-09-21T03:56:04.140 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:04.134+0000 7f5740315740 1 client.18114 shutdown 2024-09-21T03:56:04.141 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:04.134+0000 7f5740315740 1 -- 172.21.15.39:0/3546698116 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56102d306350 msgr2=0x56102d6c4ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:56:04.141 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:04.134+0000 7f5740315740 1 --2- 172.21.15.39:0/3546698116 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56102d306350 0x56102d6c4ea0 secure :-1 s=READY pgs=1269 cs=0 l=1 rev1=1 crypto rx=0x7f572c005c00 tx=0x7f572c007470 comp rx=0 tx=0).stop 2024-09-21T03:56:04.141 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:04.134+0000 7f5740315740 1 -- 172.21.15.39:0/3546698116 shutdown_connections 2024-09-21T03:56:04.141 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:04.134+0000 7f5740315740 1 --2- 172.21.15.39:0/3546698116 >> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] conn(0x56102d2c4cd0 0x56102d6cb400 crc :-1 s=READY pgs=159 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:04.141 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:04.134+0000 7f5740315740 1 --2- 172.21.15.39:0/3546698116 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56102d306350 0x56102d6c4ea0 unknown :-1 s=CLOSED pgs=1269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:04.141 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:04.134+0000 7f5740315740 1 --2- 172.21.15.39:0/3546698116 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56102d2e5670 0x56102d6b9530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:04.141 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:04.134+0000 7f5740315740 1 --2- 172.21.15.39:0/3546698116 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56102d2dd490 0x56102d6b8ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:04.141 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:04.134+0000 7f5740315740 1 -- 172.21.15.39:0/3546698116 >> 172.21.15.39:0/3546698116 conn(0x56102d2fa740 msgr2=0x56102d6b05c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:56:04.141 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:04.134+0000 7f5712ffd640 0 client.18114 ms_handle_reset on v2:172.21.15.150:6838/2949386907 2024-09-21T03:56:04.141 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:04.135+0000 7f5740315740 1 -- 172.21.15.39:0/3546698116 shutdown_connections 2024-09-21T03:56:04.141 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:04.135+0000 7f5740315740 1 -- 172.21.15.39:0/3546698116 wait complete. 2024-09-21T03:56:04.141 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:04.135+0000 7f5740315740 20 client.18114 trim_cache size 0 max 16384 2024-09-21T03:56:04.156 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:04.152+0000 7f5740315740 1 Processor -- start 2024-09-21T03:56:04.156 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:04.152+0000 7f5740315740 1 -- start start 2024-09-21T03:56:04.157 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:04.153+0000 7f5740315740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56102d2dd490 0x56102d6aeb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:56:04.157 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:04.153+0000 7f5740315740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56102d2e5670 0x56102d6aa860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:56:04.157 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:04.153+0000 7f5740315740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56102d306350 0x56102d6aada0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:56:04.157 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:04.153+0000 7f5740315740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f57341b1990 con 0x56102d2e5670 2024-09-21T03:56:04.157 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:04.153+0000 7f5740315740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f57341b34d0 con 0x56102d306350 2024-09-21T03:56:04.157 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:04.153+0000 7f5740315740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f57341b2060 con 0x56102d2dd490 2024-09-21T03:56:04.157 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:04.153+0000 7f5732ffd640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56102d2e5670 0x56102d6aa860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:56:04.157 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:04.153+0000 7f5732ffd640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56102d2e5670 0x56102d6aa860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:33402/0 (socket says 172.21.15.39:33402) 2024-09-21T03:56:04.157 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:04.153+0000 7f5732ffd640 1 -- 172.21.15.39:0/2914218611 learned_addr learned my addr 172.21.15.39:0/2914218611 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:56:04.157 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:04.153+0000 7f5733fff640 1 --2- 172.21.15.39:0/2914218611 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56102d306350 0x56102d6aada0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:56:04.157 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:04.153+0000 7f57337fe640 1 --2- 172.21.15.39:0/2914218611 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56102d2dd490 0x56102d6aeb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:56:04.158 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:04.153+0000 7f5732ffd640 1 -- 172.21.15.39:0/2914218611 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56102d2dd490 msgr2=0x56102d6aeb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:56:04.158 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:04.153+0000 7f5732ffd640 1 --2- 172.21.15.39:0/2914218611 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56102d2dd490 0x56102d6aeb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:04.158 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:04.153+0000 7f5732ffd640 1 -- 172.21.15.39:0/2914218611 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56102d306350 msgr2=0x56102d6aada0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:56:04.158 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:04.153+0000 7f5732ffd640 1 --2- 172.21.15.39:0/2914218611 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56102d306350 0x56102d6aada0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:04.158 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:04.153+0000 7f5732ffd640 1 -- 172.21.15.39:0/2914218611 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56102d6b72f0 con 0x56102d2e5670 2024-09-21T03:56:04.158 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:04.153+0000 7f57337fe640 1 --2- 172.21.15.39:0/2914218611 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56102d2dd490 0x56102d6aeb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:56:04.158 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:04.153+0000 7f5733fff640 1 --2- 172.21.15.39:0/2914218611 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56102d306350 0x56102d6aada0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:56:04.158 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:04.153+0000 7f5732ffd640 1 --2- 172.21.15.39:0/2914218611 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56102d2e5670 0x56102d6aa860 secure :-1 s=READY pgs=1270 cs=0 l=1 rev1=1 crypto rx=0x7f57280a9970 tx=0x7f57280ed060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=987d9b605ad3b35b server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:56:04.158 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:04.153+0000 7f57197fa640 1 -- 172.21.15.39:0/2914218611 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f572801d040 con 0x56102d2e5670 2024-09-21T03:56:04.158 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:04.154+0000 7f57197fa640 1 -- 172.21.15.39:0/2914218611 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f57280aceb0 con 0x56102d2e5670 2024-09-21T03:56:04.158 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:04.154+0000 7f57197fa640 1 -- 172.21.15.39:0/2914218611 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57280b1e10 con 0x56102d2e5670 2024-09-21T03:56:04.161 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:04.154+0000 7f5740315740 1 -- 172.21.15.39:0/2914218611 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56102d2e5670 msgr2=0x56102d6aa860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:56:04.161 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:04.154+0000 7f5740315740 1 --2- 172.21.15.39:0/2914218611 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56102d2e5670 0x56102d6aa860 secure :-1 s=READY pgs=1270 cs=0 l=1 rev1=1 crypto rx=0x7f57280a9970 tx=0x7f57280ed060 comp rx=0 tx=0).stop 2024-09-21T03:56:04.161 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:04.154+0000 7f5740315740 1 -- 172.21.15.39:0/2914218611 shutdown_connections 2024-09-21T03:56:04.161 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:04.154+0000 7f5740315740 1 --2- 172.21.15.39:0/2914218611 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56102d306350 0x56102d6aada0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:04.162 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:04.154+0000 7f5740315740 1 --2- 172.21.15.39:0/2914218611 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56102d2e5670 0x56102d6aa860 unknown :-1 s=CLOSED pgs=1270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:04.162 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:04.154+0000 7f5740315740 1 --2- 172.21.15.39:0/2914218611 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56102d2dd490 0x56102d6aeb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:04.162 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:04.154+0000 7f5740315740 1 -- 172.21.15.39:0/2914218611 >> 172.21.15.39:0/2914218611 conn(0x56102d2fa740 msgr2=0x56102d6ae460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:56:04.162 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:04.154+0000 7f5740315740 1 -- 172.21.15.39:0/2914218611 shutdown_connections 2024-09-21T03:56:04.162 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:04.154+0000 7f5740315740 1 -- 172.21.15.39:0/2914218611 wait complete. 2024-09-21T03:56:04.162 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:04.155+0000 7f5740315740 1 -- start start 2024-09-21T03:56:04.162 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:04.155+0000 7f5740315740 1 Processor -- start 2024-09-21T03:56:04.162 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:04.155+0000 7f5740315740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56102d2dd490 0x56102d6bd5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:56:04.162 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:04.155+0000 7f5740315740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56102d2e5670 0x56102d6bdb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:56:04.162 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:04.155+0000 7f5740315740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56102d306350 0x56102d734dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:56:04.162 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:04.155+0000 7f5740315740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f57341b2e00 con 0x56102d306350 2024-09-21T03:56:04.162 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:04.155+0000 7f5740315740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f57341b2730 con 0x56102d2dd490 2024-09-21T03:56:04.162 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:04.155+0000 7f5740315740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f57341af970 con 0x56102d2e5670 2024-09-21T03:56:04.162 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:04.155+0000 7f5740315740 10 client.0 _ll_register_callbacks cb 0x7f572c007ab0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:56:04.163 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:04.155+0000 7f5733fff640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56102d306350 0x56102d734dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:56:04.163 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:04.156+0000 7f5733fff640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56102d306350 0x56102d734dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:33412/0 (socket says 172.21.15.39:33412) 2024-09-21T03:56:04.163 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:04.156+0000 7f5733fff640 1 -- 172.21.15.39:0/2624611460 learned_addr learned my addr 172.21.15.39:0/2624611460 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:56:04.163 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:04.156+0000 7f57337fe640 1 --2- 172.21.15.39:0/2624611460 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56102d2dd490 0x56102d6bd5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:56:04.163 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:04.156+0000 7f5732ffd640 1 --2- 172.21.15.39:0/2624611460 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56102d2e5670 0x56102d6bdb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:56:04.163 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:04.156+0000 7f5733fff640 1 -- 172.21.15.39:0/2624611460 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56102d2e5670 msgr2=0x56102d6bdb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:56:04.163 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:04.156+0000 7f5733fff640 1 --2- 172.21.15.39:0/2624611460 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56102d2e5670 0x56102d6bdb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:04.163 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:04.156+0000 7f5733fff640 1 -- 172.21.15.39:0/2624611460 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56102d2dd490 msgr2=0x56102d6bd5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:56:04.163 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:04.156+0000 7f5733fff640 1 --2- 172.21.15.39:0/2624611460 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56102d2dd490 0x56102d6bd5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:04.163 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:04.156+0000 7f5733fff640 1 -- 172.21.15.39:0/2624611460 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56102d7353b0 con 0x56102d306350 2024-09-21T03:56:04.163 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:04.156+0000 7f5732ffd640 1 --2- 172.21.15.39:0/2624611460 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56102d2e5670 0x56102d6bdb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:56:04.163 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:04.156+0000 7f5733fff640 1 --2- 172.21.15.39:0/2624611460 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56102d306350 0x56102d734dc0 secure :-1 s=READY pgs=1271 cs=0 l=1 rev1=1 crypto rx=0x7f572c005b00 tx=0x7f572c026e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f3a14c6af6bea3f8 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:56:04.163 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:04.156+0000 7f5712ffd640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-09-21T03:56:04.163 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:04.156+0000 7f5712ffd640 1 -- 172.21.15.39:0/2624611460 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f572c026650 con 0x56102d306350 2024-09-21T03:56:04.164 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:04.156+0000 7f5712ffd640 1 -- 172.21.15.39:0/2624611460 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8528+0+0 (secure 0 0 0) 0x7f572c01db00 con 0x56102d306350 2024-09-21T03:56:04.164 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:04.156+0000 7f5740315740 1 -- 172.21.15.39:0/2624611460 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f57280aceb0 con 0x56102d306350 2024-09-21T03:56:04.164 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:04.156+0000 7f5712ffd640 1 -- 172.21.15.39:0/2624611460 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f572c01dd40 con 0x56102d306350 2024-09-21T03:56:04.164 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:04.156+0000 7f5740315740 10 client.18120 fetch_fsmap learned FSMap version 29 2024-09-21T03:56:04.164 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:04.156+0000 7f5740315740 1 -- 172.21.15.39:0/2624611460 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x56102d7365c0 con 0x56102d306350 2024-09-21T03:56:04.164 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:04.157+0000 7f5712ffd640 1 -- 172.21.15.39:0/2624611460 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f57280aceb0 con 0x56102d306350 2024-09-21T03:56:04.164 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:04.157+0000 7f5712ffd640 1 -- 172.21.15.39:0/2624611460 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f572c024040 con 0x56102d306350 2024-09-21T03:56:04.164 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:04.157+0000 7f5740315740 10 client.18120 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T03:56:04.164 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:04.157+0000 7f5740315740 10 client.18120 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T03:56:04.164 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:04.157+0000 7f5740315740 20 client.18120 populate_metadata read hostname 'smithi039' 2024-09-21T03:56:04.164 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:04.157+0000 7f5740315740 1 --2- 172.21.15.39:0/2624611460 >> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] conn(0x56102d29a080 0x56102d738da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:56:04.164 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:04.157+0000 7f5740315740 4 client.18120 mds_command: new command op to 24695 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-21T03:56:04.164 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:04.157+0000 7f5740315740 1 -- 172.21.15.39:0/2624611460 --> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x56102d621860 con 0x56102d29a080 2024-09-21T03:56:04.164 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:04.157+0000 7f57337fe640 1 --2- 172.21.15.39:0/2624611460 >> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] conn(0x56102d29a080 0x56102d738da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:56:04.165 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:04.158+0000 7f5712ffd640 10 client.18120 ms_handle_connect on v2:172.21.15.150:6838/2949386907 2024-09-21T03:56:04.165 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:04.158+0000 7f57337fe640 1 --2- 172.21.15.39:0/2624611460 >> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] conn(0x56102d29a080 0x56102d738da0 crc :-1 s=READY pgs=160 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3a1f8ada712cb194 server_cookie=2d3b191450eb6f1f in_seq=0 out_seq=0 2024-09-21T03:56:04.165 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:04.160+0000 7f5712ffd640 1 -- 172.21.15.39:0/2624611460 <== mds.0 v2:172.21.15.150:6838/2949386907 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x56102d621860 con 0x56102d29a080 2024-09-21T03:56:04.166 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:04.160+0000 7f5712ffd640 10 client.18120 handle_command_reply: tid=0 2024-09-21T03:56:04.166 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:04.160+0000 7f5740315740 1 client.18120 shutdown 2024-09-21T03:56:04.166 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:04.160+0000 7f5740315740 1 -- 172.21.15.39:0/2624611460 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56102d306350 msgr2=0x56102d734dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:56:04.166 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:04.160+0000 7f5740315740 1 --2- 172.21.15.39:0/2624611460 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56102d306350 0x56102d734dc0 secure :-1 s=READY pgs=1271 cs=0 l=1 rev1=1 crypto rx=0x7f572c005b00 tx=0x7f572c026e90 comp rx=0 tx=0).stop 2024-09-21T03:56:04.166 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:04.160+0000 7f5740315740 1 -- 172.21.15.39:0/2624611460 shutdown_connections 2024-09-21T03:56:04.166 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:04.160+0000 7f5740315740 1 --2- 172.21.15.39:0/2624611460 >> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] conn(0x56102d29a080 0x56102d738da0 crc :-1 s=READY pgs=160 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:04.166 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:04.160+0000 7f5740315740 1 --2- 172.21.15.39:0/2624611460 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56102d306350 0x56102d734dc0 unknown :-1 s=CLOSED pgs=1271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:04.166 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:04.160+0000 7f5740315740 1 --2- 172.21.15.39:0/2624611460 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56102d2e5670 0x56102d6bdb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:04.166 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:04.160+0000 7f5740315740 1 --2- 172.21.15.39:0/2624611460 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56102d2dd490 0x56102d6bd5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:04.166 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:04.160+0000 7f5740315740 1 -- 172.21.15.39:0/2624611460 >> 172.21.15.39:0/2624611460 conn(0x56102d2fa740 msgr2=0x56102d6aed40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:56:04.166 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:04.160+0000 7f5712ffd640 0 client.18120 ms_handle_reset on v2:172.21.15.150:6838/2949386907 2024-09-21T03:56:04.166 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:04.160+0000 7f5740315740 1 -- 172.21.15.39:0/2624611460 shutdown_connections 2024-09-21T03:56:04.166 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:04.160+0000 7f5740315740 1 -- 172.21.15.39:0/2624611460 wait complete. 2024-09-21T03:56:04.167 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:04.161+0000 7f5740315740 20 client.18120 trim_cache size 0 max 16384 2024-09-21T03:56:04.167 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-09-21T03:56:04.167 INFO:teuthology.orchestra.run.smithi039.stdout: "return_code": 0, 2024-09-21T03:56:04.167 INFO:teuthology.orchestra.run.smithi039.stdout: "scrub_tag": "a7293877-4df2-4bbb-be12-176711d2ac8a", 2024-09-21T03:56:04.167 INFO:teuthology.orchestra.run.smithi039.stdout: "mode": "asynchronous" 2024-09-21T03:56:04.167 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-09-21T03:56:04.168 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:04.161+0000 7f5739da1640 1 -- 172.21.15.39:0/3738726024 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f571c0b6730 msgr2=0x7f571c0b8c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:56:04.168 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:04.161+0000 7f5739da1640 1 --2- 172.21.15.39:0/3738726024 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f571c0b6730 0x7f571c0b8c40 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f572c007dd0 tx=0x7f572c004a40 comp rx=0 tx=0).stop 2024-09-21T03:56:04.168 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:04.161+0000 7f5739da1640 1 -- 172.21.15.39:0/3738726024 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f57341094a0 msgr2=0x7f573419aaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:56:04.168 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:04.161+0000 7f5739da1640 1 --2- 172.21.15.39:0/3738726024 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f57341094a0 0x7f573419aaa0 secure :-1 s=READY pgs=1267 cs=0 l=1 rev1=1 crypto rx=0x7f572800c970 tx=0x7f572800ce30 comp rx=0 tx=0).stop 2024-09-21T03:56:04.168 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:04.161+0000 7f5739da1640 1 -- 172.21.15.39:0/3738726024 shutdown_connections 2024-09-21T03:56:04.168 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:04.161+0000 7f5739da1640 1 --2- 172.21.15.39:0/3738726024 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f571c0b6730 0x7f571c0b8c40 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:04.168 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:04.161+0000 7f5739da1640 1 --2- 172.21.15.39:0/3738726024 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5734109df0 0x7f573419fd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:04.168 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:04.161+0000 7f5739da1640 1 --2- 172.21.15.39:0/3738726024 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f57341094a0 0x7f573419aaa0 unknown :-1 s=CLOSED pgs=1267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:04.168 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:04.161+0000 7f5739da1640 1 --2- 172.21.15.39:0/3738726024 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5734103b60 0x7f573419a500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:04.168 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:04.161+0000 7f5739da1640 1 -- 172.21.15.39:0/3738726024 >> 172.21.15.39:0/3738726024 conn(0x7f573407b850 msgr2=0x7f57341075e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:56:04.168 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:04.161+0000 7f5739da1640 1 -- 172.21.15.39:0/3738726024 shutdown_connections 2024-09-21T03:56:04.168 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:04.161+0000 7f5739da1640 1 -- 172.21.15.39:0/3738726024 wait complete. 2024-09-21T03:56:04.992 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 51116d5c-77ca-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub status 2024-09-21T03:56:05.256 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/51116d5c-77ca-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:56:05.334 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:56:05 smithi039 ceph-mon[31211]: pgmap v632: 97 pgs: 97 active+clean; 10 MiB data, 525 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 597 B/s wr, 10 op/s 2024-09-21T03:56:05.334 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:56:05 smithi039 ceph-mon[31211]: scrub queued for path: / 2024-09-21T03:56:05.334 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:56:05 smithi039 ceph-mon[31211]: scrub summary: idle+waiting paths [/] 2024-09-21T03:56:05.334 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:56:05 smithi039 ceph-mon[31211]: scrub summary: active paths [/] 2024-09-21T03:56:05.334 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:56:05 smithi039 ceph-mon[31211]: scrub summary: idle+waiting paths [/] 2024-09-21T03:56:05.334 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:56:05 smithi039 ceph-mon[31211]: scrub summary: idle 2024-09-21T03:56:05.468 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:56:05 smithi161 ceph-mon[34533]: pgmap v632: 97 pgs: 97 active+clean; 10 MiB data, 525 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 597 B/s wr, 10 op/s 2024-09-21T03:56:05.468 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:56:05 smithi161 ceph-mon[34533]: scrub queued for path: / 2024-09-21T03:56:05.468 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:56:05 smithi161 ceph-mon[34533]: scrub summary: idle+waiting paths [/] 2024-09-21T03:56:05.468 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:56:05 smithi161 ceph-mon[34533]: scrub summary: active paths [/] 2024-09-21T03:56:05.468 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:56:05 smithi161 ceph-mon[34533]: scrub summary: idle+waiting paths [/] 2024-09-21T03:56:05.468 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:56:05 smithi161 ceph-mon[34533]: scrub summary: idle 2024-09-21T03:56:05.475 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:56:05 smithi150 ceph-mon[34289]: pgmap v632: 97 pgs: 97 active+clean; 10 MiB data, 525 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 597 B/s wr, 10 op/s 2024-09-21T03:56:05.475 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:56:05 smithi150 ceph-mon[34289]: scrub queued for path: / 2024-09-21T03:56:05.475 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:56:05 smithi150 ceph-mon[34289]: scrub summary: idle+waiting paths [/] 2024-09-21T03:56:05.475 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:56:05 smithi150 ceph-mon[34289]: scrub summary: active paths [/] 2024-09-21T03:56:05.475 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:56:05 smithi150 ceph-mon[34289]: scrub summary: idle+waiting paths [/] 2024-09-21T03:56:05.475 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:56:05 smithi150 ceph-mon[34289]: scrub summary: idle 2024-09-21T03:56:05.790 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:05.786+0000 7f192a4fc640 1 -- 172.21.15.39:0/1666235127 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f19241049e0 msgr2=0x7f1924104df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:56:05.790 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:05.786+0000 7f192a4fc640 1 --2- 172.21.15.39:0/1666235127 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f19241049e0 0x7f1924104df0 secure :-1 s=READY pgs=1272 cs=0 l=1 rev1=1 crypto rx=0x7f1914009900 tx=0x7f191402a3f0 comp rx=0 tx=0).stop 2024-09-21T03:56:05.790 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:05.787+0000 7f192a4fc640 1 -- 172.21.15.39:0/1666235127 shutdown_connections 2024-09-21T03:56:05.791 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:05.787+0000 7f192a4fc640 1 --2- 172.21.15.39:0/1666235127 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f192410de40 0x7f1924110330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:05.791 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:05.787+0000 7f192a4fc640 1 --2- 172.21.15.39:0/1666235127 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1924105330 0x7f192410d880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:05.791 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:05.787+0000 7f192a4fc640 1 --2- 172.21.15.39:0/1666235127 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f19241049e0 0x7f1924104df0 unknown :-1 s=CLOSED pgs=1272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:05.791 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:05.787+0000 7f192a4fc640 1 -- 172.21.15.39:0/1666235127 >> 172.21.15.39:0/1666235127 conn(0x7f1924079750 msgr2=0x7f1924079b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:56:05.791 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:05.787+0000 7f192a4fc640 1 -- 172.21.15.39:0/1666235127 shutdown_connections 2024-09-21T03:56:05.791 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:05.787+0000 7f192a4fc640 1 -- 172.21.15.39:0/1666235127 wait complete. 2024-09-21T03:56:05.791 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:05.787+0000 7f192a4fc640 1 Processor -- start 2024-09-21T03:56:05.791 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:05.787+0000 7f192a4fc640 1 -- start start 2024-09-21T03:56:05.792 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:05.788+0000 7f192a4fc640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f19241049e0 0x7f192419a530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:56:05.792 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:05.788+0000 7f192a4fc640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1924105330 0x7f192419aad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:56:05.792 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:05.788+0000 7f192a4fc640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f192410de40 0x7f192419fd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:56:05.792 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:05.788+0000 7f192a4fc640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f1924113aa0 con 0x7f19241049e0 2024-09-21T03:56:05.792 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:05.788+0000 7f192a4fc640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f1924113920 con 0x7f1924105330 2024-09-21T03:56:05.792 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:05.788+0000 7f192a4fc640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f1924113c20 con 0x7f192410de40 2024-09-21T03:56:05.792 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:05.788+0000 7f1923fff640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f19241049e0 0x7f192419a530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:56:05.792 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:05.788+0000 7f1923fff640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f19241049e0 0x7f192419a530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:33416/0 (socket says 172.21.15.39:33416) 2024-09-21T03:56:05.794 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:05.788+0000 7f1923fff640 1 -- 172.21.15.39:0/3651323194 learned_addr learned my addr 172.21.15.39:0/3651323194 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:56:05.794 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:05.788+0000 7f19237fe640 1 --2- 172.21.15.39:0/3651323194 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1924105330 0x7f192419aad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:56:05.794 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:05.788+0000 7f1928a99640 1 --2- 172.21.15.39:0/3651323194 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f192410de40 0x7f192419fd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:56:05.794 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:05.788+0000 7f1923fff640 1 -- 172.21.15.39:0/3651323194 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f192410de40 msgr2=0x7f192419fd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:56:05.794 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:05.788+0000 7f1923fff640 1 --2- 172.21.15.39:0/3651323194 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f192410de40 0x7f192419fd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:05.794 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:05.788+0000 7f1923fff640 1 -- 172.21.15.39:0/3651323194 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1924105330 msgr2=0x7f192419aad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:56:05.794 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:05.788+0000 7f1923fff640 1 --2- 172.21.15.39:0/3651323194 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1924105330 0x7f192419aad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:05.794 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:05.788+0000 7f1923fff640 1 -- 172.21.15.39:0/3651323194 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f19241a03e0 con 0x7f19241049e0 2024-09-21T03:56:05.795 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:05.788+0000 7f19237fe640 1 --2- 172.21.15.39:0/3651323194 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1924105330 0x7f192419aad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:56:05.795 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:05.788+0000 7f1928a99640 1 --2- 172.21.15.39:0/3651323194 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f192410de40 0x7f192419fd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:56:05.795 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:05.789+0000 7f1923fff640 1 --2- 172.21.15.39:0/3651323194 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f19241049e0 0x7f192419a530 secure :-1 s=READY pgs=1273 cs=0 l=1 rev1=1 crypto rx=0x7f1914032fd0 tx=0x7f1914004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:56:05.795 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:05.789+0000 7f19217fa640 1 -- 172.21.15.39:0/3651323194 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f191403f030 con 0x7f19241049e0 2024-09-21T03:56:05.795 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:05.789+0000 7f19217fa640 1 -- 172.21.15.39:0/3651323194 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1914004490 con 0x7f19241049e0 2024-09-21T03:56:05.795 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:05.789+0000 7f192a4fc640 1 -- 172.21.15.39:0/3651323194 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f19241a0670 con 0x7f19241049e0 2024-09-21T03:56:05.795 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:05.789+0000 7f19217fa640 1 -- 172.21.15.39:0/3651323194 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1914035070 con 0x7f19241049e0 2024-09-21T03:56:05.795 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:05.789+0000 7f192a4fc640 1 -- 172.21.15.39:0/3651323194 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f19241a0ba0 con 0x7f19241049e0 2024-09-21T03:56:05.801 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:05.791+0000 7f1930a70740 1 Processor -- start 2024-09-21T03:56:05.801 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:05.791+0000 7f1930a70740 1 -- start start 2024-09-21T03:56:05.801 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:05.792+0000 7f1930a70740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d2031ec070 0x55d203593650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:56:05.801 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:05.792+0000 7f1930a70740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d2031c31b0 0x55d203593b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:56:05.801 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:05.792+0000 7f1930a70740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d2031cb390 0x55d2035940d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:56:05.801 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:05.792+0000 7f1930a70740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x55d203509e50 con 0x55d2031c31b0 2024-09-21T03:56:05.802 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:05.792+0000 7f1930a70740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x55d2034f9280 con 0x55d2031cb390 2024-09-21T03:56:05.802 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:05.792+0000 7f1930a70740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55d2034fd090 con 0x55d2031ec070 2024-09-21T03:56:05.802 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:05.792+0000 7f19217fa640 1 -- 172.21.15.39:0/3651323194 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 27) ==== 154643+0+0 (secure 0 0 0) 0x7f1914042030 con 0x7f19241049e0 2024-09-21T03:56:05.802 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:05.792+0000 7f1928a99640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d2031ec070 0x55d203593650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:56:05.802 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:05.792+0000 7f1923fff640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d2031c31b0 0x55d203593b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:56:05.802 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:05.792+0000 7f19237fe640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d2031cb390 0x55d2035940d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:56:05.802 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:05.792+0000 7f1928a99640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d2031ec070 0x55d203593650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.39:57878/0 (socket says 172.21.15.39:57878) 2024-09-21T03:56:05.802 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:05.792+0000 7f19237fe640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d2031cb390 0x55d2035940d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.39:56968/0 (socket says 172.21.15.39:56968) 2024-09-21T03:56:05.802 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:05.792+0000 7f1928a99640 1 -- 172.21.15.39:0/3262993670 learned_addr learned my addr 172.21.15.39:0/3262993670 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:56:05.802 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:05.793+0000 7f19237fe640 1 -- 172.21.15.39:0/3262993670 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d2031ec070 msgr2=0x55d203593650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:56:05.803 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:05.793+0000 7f19237fe640 1 --2- 172.21.15.39:0/3262993670 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d2031ec070 0x55d203593650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:05.803 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:05.793+0000 7f19237fe640 1 -- 172.21.15.39:0/3262993670 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d2031c31b0 msgr2=0x55d203593b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:56:05.803 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:05.793+0000 7f19237fe640 1 --2- 172.21.15.39:0/3262993670 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d2031c31b0 0x55d203593b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:05.803 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:05.793+0000 7f19237fe640 1 -- 172.21.15.39:0/3262993670 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d203257650 con 0x55d2031cb390 2024-09-21T03:56:05.803 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:05.793+0000 7f1928a99640 1 --2- 172.21.15.39:0/3262993670 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d2031ec070 0x55d203593650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:56:05.803 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:05.793+0000 7f1923fff640 1 --2- 172.21.15.39:0/3262993670 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d2031c31b0 0x55d203593b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:56:05.803 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:05.793+0000 7f19237fe640 1 --2- 172.21.15.39:0/3262993670 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d2031cb390 0x55d2035940d0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f19241023f0 tx=0x7f191800e720 comp rx=0 tx=0).ready entity=mon.2 client_cookie=42ca2184bd95c8e8 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:56:05.803 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:05.793+0000 7f19217fa640 1 --2- 172.21.15.39:0/3651323194 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f190c0b6730 0x7f190c0b8c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:56:05.803 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:05.793+0000 7f190a7fc640 1 -- 172.21.15.39:0/3262993670 <== mon.2 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1918014070 con 0x55d2031cb390 2024-09-21T03:56:05.803 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:05.793+0000 7f1928a99640 1 --2- 172.21.15.39:0/3651323194 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f190c0b6730 0x7f190c0b8c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:56:05.803 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:05.793+0000 7f190a7fc640 1 -- 172.21.15.39:0/3262993670 <== mon.2 v2:172.21.15.150:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1918008e50 con 0x55d2031cb390 2024-09-21T03:56:05.804 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:05.793+0000 7f19217fa640 1 -- 172.21.15.39:0/3651323194 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8528+0+0 (secure 0 0 0) 0x7f1914107c20 con 0x7f19241049e0 2024-09-21T03:56:05.804 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:05.793+0000 7f190a7fc640 1 -- 172.21.15.39:0/3262993670 <== mon.2 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1918010690 con 0x55d2031cb390 2024-09-21T03:56:05.804 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:05.794+0000 7f1930a70740 1 -- 172.21.15.39:0/3262993670 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d2031cb390 msgr2=0x55d2035940d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:56:05.804 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:05.794+0000 7f1928a99640 1 --2- 172.21.15.39:0/3651323194 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f190c0b6730 0x7f190c0b8c40 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f191c0092f0 tx=0x7f191c009740 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:56:05.804 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:05.794+0000 7f1930a70740 1 --2- 172.21.15.39:0/3262993670 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d2031cb390 0x55d2035940d0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f19241023f0 tx=0x7f191800e720 comp rx=0 tx=0).stop 2024-09-21T03:56:05.804 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:05.794+0000 7f1930a70740 1 -- 172.21.15.39:0/3262993670 shutdown_connections 2024-09-21T03:56:05.804 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:05.794+0000 7f1930a70740 1 --2- 172.21.15.39:0/3262993670 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d2031cb390 0x55d2035940d0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:05.805 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:05.794+0000 7f1930a70740 1 --2- 172.21.15.39:0/3262993670 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d2031c31b0 0x55d203593b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:05.805 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:05.794+0000 7f1930a70740 1 --2- 172.21.15.39:0/3262993670 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d2031ec070 0x55d203593650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:05.805 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:05.794+0000 7f1930a70740 1 -- 172.21.15.39:0/3262993670 >> 172.21.15.39:0/3262993670 conn(0x55d2031e0460 msgr2=0x55d203591540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:56:05.805 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:05.794+0000 7f1930a70740 1 -- 172.21.15.39:0/3262993670 shutdown_connections 2024-09-21T03:56:05.805 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:05.794+0000 7f1930a70740 1 -- 172.21.15.39:0/3262993670 wait complete. 2024-09-21T03:56:05.805 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:05.795+0000 7f1930a70740 1 -- start start 2024-09-21T03:56:05.805 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:05.795+0000 7f1930a70740 1 Processor -- start 2024-09-21T03:56:05.805 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:05.795+0000 7f1930a70740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d2031c31b0 0x55d2035a3170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:56:05.805 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:05.795+0000 7f1930a70740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d2031cb390 0x55d2035a36b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:56:05.805 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:05.795+0000 7f1930a70740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d2031ec070 0x55d2035af460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:56:05.805 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:05.795+0000 7f1930a70740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x55d20335a700 con 0x55d2031ec070 2024-09-21T03:56:05.806 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:05.795+0000 7f1930a70740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x55d203494110 con 0x55d2031cb390 2024-09-21T03:56:05.806 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:05.795+0000 7f1930a70740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55d2035a6130 con 0x55d2031c31b0 2024-09-21T03:56:05.806 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:05.795+0000 7f1930a70740 10 client.0 _ll_register_callbacks cb 0x55d2034ad990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:56:05.806 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:05.796+0000 7f1928a99640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d2031ec070 0x55d2035af460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:56:05.806 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:05.796+0000 7f1928a99640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d2031ec070 0x55d2035af460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:33430/0 (socket says 172.21.15.39:33430) 2024-09-21T03:56:05.806 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:05.796+0000 7f1928a99640 1 -- 172.21.15.39:0/2563581968 learned_addr learned my addr 172.21.15.39:0/2563581968 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:56:05.806 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:05.796+0000 7f19237fe640 1 --2- 172.21.15.39:0/2563581968 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d2031cb390 0x55d2035a36b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:56:05.806 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:05.796+0000 7f1923fff640 1 --2- 172.21.15.39:0/2563581968 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d2031c31b0 0x55d2035a3170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:56:05.806 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:05.796+0000 7f1928a99640 1 -- 172.21.15.39:0/2563581968 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d2031c31b0 msgr2=0x55d2035a3170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:56:05.806 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:05.796+0000 7f1928a99640 1 --2- 172.21.15.39:0/2563581968 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d2031c31b0 0x55d2035a3170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:05.806 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:05.796+0000 7f1928a99640 1 -- 172.21.15.39:0/2563581968 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d2031cb390 msgr2=0x55d2035a36b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:56:05.806 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:05.796+0000 7f1928a99640 1 --2- 172.21.15.39:0/2563581968 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d2031cb390 0x55d2035a36b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:05.807 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:05.796+0000 7f1928a99640 1 -- 172.21.15.39:0/2563581968 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d2034c1c40 con 0x55d2031ec070 2024-09-21T03:56:05.807 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:05.796+0000 7f1928a99640 1 --2- 172.21.15.39:0/2563581968 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d2031ec070 0x55d2035af460 secure :-1 s=READY pgs=1274 cs=0 l=1 rev1=1 crypto rx=0x7f191c005fd0 tx=0x7f191c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a9be1be5d93d7dca server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:56:05.807 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:05.796+0000 7f19037fe640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-09-21T03:56:05.807 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:05.796+0000 7f19037fe640 1 -- 172.21.15.39:0/2563581968 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f191c00fd20 con 0x55d2031ec070 2024-09-21T03:56:05.807 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:05.796+0000 7f19037fe640 1 -- 172.21.15.39:0/2563581968 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8528+0+0 (secure 0 0 0) 0x7f191c01f420 con 0x55d2031ec070 2024-09-21T03:56:05.807 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:05.796+0000 7f1930a70740 1 -- 172.21.15.39:0/2563581968 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1918008e50 con 0x55d2031ec070 2024-09-21T03:56:05.807 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:05.797+0000 7f19037fe640 1 -- 172.21.15.39:0/2563581968 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f191c01f620 con 0x55d2031ec070 2024-09-21T03:56:05.807 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:05.797+0000 7f1930a70740 10 client.18132 fetch_fsmap learned FSMap version 29 2024-09-21T03:56:05.807 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:05.797+0000 7f1930a70740 1 -- 172.21.15.39:0/2563581968 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55d2035b1f10 con 0x55d2031ec070 2024-09-21T03:56:05.807 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:05.797+0000 7f19037fe640 1 -- 172.21.15.39:0/2563581968 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1918008e50 con 0x55d2031ec070 2024-09-21T03:56:05.807 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:05.797+0000 7f19037fe640 1 -- 172.21.15.39:0/2563581968 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f191c030900 con 0x55d2031ec070 2024-09-21T03:56:05.807 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:05.797+0000 7f1930a70740 10 client.18132 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T03:56:05.808 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:05.797+0000 7f1930a70740 10 client.18132 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T03:56:05.808 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:05.797+0000 7f1930a70740 20 client.18132 populate_metadata read hostname 'smithi039' 2024-09-21T03:56:05.808 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:05.797+0000 7f1930a70740 1 --2- 172.21.15.39:0/2563581968 >> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] conn(0x55d2031aa9f0 0x55d2035b5380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:56:05.808 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:05.797+0000 7f1930a70740 4 client.18132 mds_command: new command op to 24695 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:56:05.808 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:05.797+0000 7f1930a70740 1 -- 172.21.15.39:0/2563581968 --> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d20350bda0 con 0x55d2031aa9f0 2024-09-21T03:56:05.808 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:05.797+0000 7f1923fff640 1 --2- 172.21.15.39:0/2563581968 >> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] conn(0x55d2031aa9f0 0x55d2035b5380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:56:05.808 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:05.798+0000 7f19037fe640 10 client.18132 ms_handle_connect on v2:172.21.15.150:6838/2949386907 2024-09-21T03:56:05.808 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:05.798+0000 7f1923fff640 1 --2- 172.21.15.39:0/2563581968 >> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] conn(0x55d2031aa9f0 0x55d2035b5380 crc :-1 s=READY pgs=161 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6bf1f865060f332a server_cookie=6c7e6f988fa9031d in_seq=0 out_seq=0 2024-09-21T03:56:05.809 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:05.799+0000 7f19037fe640 1 -- 172.21.15.39:0/2563581968 <== mds.0 v2:172.21.15.150:6838/2949386907 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55d20350bda0 con 0x55d2031aa9f0 2024-09-21T03:56:05.809 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:05.799+0000 7f19037fe640 10 client.18132 handle_command_reply: tid=0 2024-09-21T03:56:05.810 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:05.799+0000 7f1930a70740 1 client.18132 shutdown 2024-09-21T03:56:05.810 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:05.799+0000 7f1930a70740 1 -- 172.21.15.39:0/2563581968 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d2031ec070 msgr2=0x55d2035af460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:56:05.810 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:05.799+0000 7f1930a70740 1 --2- 172.21.15.39:0/2563581968 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d2031ec070 0x55d2035af460 secure :-1 s=READY pgs=1274 cs=0 l=1 rev1=1 crypto rx=0x7f191c005fd0 tx=0x7f191c004a40 comp rx=0 tx=0).stop 2024-09-21T03:56:05.810 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:05.799+0000 7f1930a70740 1 -- 172.21.15.39:0/2563581968 shutdown_connections 2024-09-21T03:56:05.810 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:05.799+0000 7f1930a70740 1 --2- 172.21.15.39:0/2563581968 >> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] conn(0x55d2031aa9f0 0x55d2035b5380 crc :-1 s=READY pgs=161 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:05.810 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:05.799+0000 7f19037fe640 0 client.18132 ms_handle_reset on v2:172.21.15.150:6838/2949386907 2024-09-21T03:56:05.810 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:05.799+0000 7f1930a70740 1 --2- 172.21.15.39:0/2563581968 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d2031ec070 0x55d2035af460 unknown :-1 s=CLOSED pgs=1274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:05.810 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:05.799+0000 7f1930a70740 1 --2- 172.21.15.39:0/2563581968 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d2031cb390 0x55d2035a36b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:05.810 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:05.799+0000 7f1930a70740 1 --2- 172.21.15.39:0/2563581968 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d2031c31b0 0x55d2035a3170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:05.810 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:05.799+0000 7f1930a70740 1 -- 172.21.15.39:0/2563581968 >> 172.21.15.39:0/2563581968 conn(0x55d2031e0460 msgr2=0x55d203595040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:56:05.810 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:05.799+0000 7f1930a70740 1 -- 172.21.15.39:0/2563581968 shutdown_connections 2024-09-21T03:56:05.810 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:05.799+0000 7f1930a70740 1 -- 172.21.15.39:0/2563581968 wait complete. 2024-09-21T03:56:05.810 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:05.799+0000 7f1930a70740 20 client.18132 trim_cache size 0 max 16384 2024-09-21T03:56:05.822 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:05.817+0000 7f1930a70740 1 Processor -- start 2024-09-21T03:56:05.822 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:05.817+0000 7f1930a70740 1 -- start start 2024-09-21T03:56:05.822 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:05.817+0000 7f1930a70740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d2031c31b0 0x55d2035a3630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:56:05.822 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:05.817+0000 7f1930a70740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d2031cb390 0x55d2035af360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:56:05.822 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:05.817+0000 7f1930a70740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d2031ec070 0x55d20359d9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:56:05.822 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:05.817+0000 7f1930a70740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f19241ad680 con 0x55d2031c31b0 2024-09-21T03:56:05.823 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:05.817+0000 7f1930a70740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f19241af1c0 con 0x55d2031cb390 2024-09-21T03:56:05.823 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:05.817+0000 7f1930a70740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f19241add50 con 0x55d2031ec070 2024-09-21T03:56:05.823 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:05.817+0000 7f1923fff640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d2031c31b0 0x55d2035a3630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:56:05.823 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:05.817+0000 7f1923fff640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d2031c31b0 0x55d2035a3630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:33434/0 (socket says 172.21.15.39:33434) 2024-09-21T03:56:05.823 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:05.817+0000 7f1923fff640 1 -- 172.21.15.39:0/1223931531 learned_addr learned my addr 172.21.15.39:0/1223931531 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:56:05.823 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:05.817+0000 7f19237fe640 1 --2- 172.21.15.39:0/1223931531 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d2031cb390 0x55d2035af360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:56:05.823 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:05.817+0000 7f1923fff640 1 -- 172.21.15.39:0/1223931531 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d2031ec070 msgr2=0x55d20359d9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:56:05.823 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:05.817+0000 7f1923fff640 1 --2- 172.21.15.39:0/1223931531 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d2031ec070 0x55d20359d9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:05.823 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:05.817+0000 7f1923fff640 1 -- 172.21.15.39:0/1223931531 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d2031cb390 msgr2=0x55d2035af360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:56:05.823 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:05.817+0000 7f1923fff640 1 --2- 172.21.15.39:0/1223931531 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d2031cb390 0x55d2035af360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:05.823 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:05.817+0000 7f1923fff640 1 -- 172.21.15.39:0/1223931531 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d2035a20d0 con 0x55d2031c31b0 2024-09-21T03:56:05.823 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:05.818+0000 7f1923fff640 1 --2- 172.21.15.39:0/1223931531 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d2031c31b0 0x55d2035a3630 secure :-1 s=READY pgs=1275 cs=0 l=1 rev1=1 crypto rx=0x7f1914031440 tx=0x7f191410b710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ea1ea0a3591679c server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:56:05.823 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:05.818+0000 7f1909ffb640 1 -- 172.21.15.39:0/1223931531 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19140c9040 con 0x55d2031c31b0 2024-09-21T03:56:05.824 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:05.818+0000 7f1909ffb640 1 -- 172.21.15.39:0/1223931531 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f19140cd580 con 0x55d2031c31b0 2024-09-21T03:56:05.824 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:05.818+0000 7f1909ffb640 1 -- 172.21.15.39:0/1223931531 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1914049040 con 0x55d2031c31b0 2024-09-21T03:56:05.824 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:05.819+0000 7f1930a70740 1 -- 172.21.15.39:0/1223931531 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d2031c31b0 msgr2=0x55d2035a3630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:56:05.825 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:05.819+0000 7f1930a70740 1 --2- 172.21.15.39:0/1223931531 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d2031c31b0 0x55d2035a3630 secure :-1 s=READY pgs=1275 cs=0 l=1 rev1=1 crypto rx=0x7f1914031440 tx=0x7f191410b710 comp rx=0 tx=0).stop 2024-09-21T03:56:05.825 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:05.819+0000 7f1930a70740 1 -- 172.21.15.39:0/1223931531 shutdown_connections 2024-09-21T03:56:05.825 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:05.819+0000 7f1930a70740 1 --2- 172.21.15.39:0/1223931531 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d2031ec070 0x55d20359d9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:05.826 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:05.819+0000 7f1930a70740 1 --2- 172.21.15.39:0/1223931531 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d2031cb390 0x55d2035af360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:05.826 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:05.819+0000 7f1930a70740 1 --2- 172.21.15.39:0/1223931531 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d2031c31b0 0x55d2035a3630 unknown :-1 s=CLOSED pgs=1275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:05.826 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:05.819+0000 7f1930a70740 1 -- 172.21.15.39:0/1223931531 >> 172.21.15.39:0/1223931531 conn(0x55d2031e0460 msgr2=0x55d2035a2f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:56:05.826 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:05.819+0000 7f1930a70740 1 -- 172.21.15.39:0/1223931531 shutdown_connections 2024-09-21T03:56:05.827 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:05.819+0000 7f1930a70740 1 -- 172.21.15.39:0/1223931531 wait complete. 2024-09-21T03:56:05.827 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:05.819+0000 7f1930a70740 1 -- start start 2024-09-21T03:56:05.836 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:05.819+0000 7f1930a70740 1 Processor -- start 2024-09-21T03:56:05.837 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:05.820+0000 7f1930a70740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d2031c31b0 0x55d20359b790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:56:05.837 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:05.820+0000 7f1930a70740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d2031cb390 0x55d20359bd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:56:05.837 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:05.820+0000 7f1930a70740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d2031ec070 0x55d2035ab3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:56:05.837 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:05.820+0000 7f1930a70740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f19241aeaf0 con 0x55d2031cb390 2024-09-21T03:56:05.837 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:05.820+0000 7f1930a70740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f19241ae420 con 0x55d2031c31b0 2024-09-21T03:56:05.837 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:05.820+0000 7f1930a70740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f19241ab660 con 0x55d2031ec070 2024-09-21T03:56:05.837 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:05.820+0000 7f1930a70740 10 client.0 _ll_register_callbacks cb 0x7f191c009f00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:56:05.837 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:05.820+0000 7f19237fe640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d2031cb390 0x55d20359bd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:56:05.838 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:05.820+0000 7f19237fe640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d2031cb390 0x55d20359bd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:33450/0 (socket says 172.21.15.39:33450) 2024-09-21T03:56:05.838 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:05.820+0000 7f19237fe640 1 -- 172.21.15.39:0/3633707999 learned_addr learned my addr 172.21.15.39:0/3633707999 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:56:05.838 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:05.820+0000 7f1923fff640 1 --2- 172.21.15.39:0/3633707999 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d2031c31b0 0x55d20359b790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:56:05.840 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:05.820+0000 7f1928a99640 1 --2- 172.21.15.39:0/3633707999 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d2031ec070 0x55d2035ab3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:56:05.840 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:05.820+0000 7f19237fe640 1 -- 172.21.15.39:0/3633707999 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d2031ec070 msgr2=0x55d2035ab3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:56:05.840 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:05.820+0000 7f19237fe640 1 --2- 172.21.15.39:0/3633707999 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d2031ec070 0x55d2035ab3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:05.840 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:05.820+0000 7f19237fe640 1 -- 172.21.15.39:0/3633707999 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d2031c31b0 msgr2=0x55d20359b790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:56:05.840 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:05.820+0000 7f19237fe640 1 --2- 172.21.15.39:0/3633707999 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d2031c31b0 0x55d20359b790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:05.840 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:05.820+0000 7f19237fe640 1 -- 172.21.15.39:0/3633707999 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d2035aba00 con 0x55d2031cb390 2024-09-21T03:56:05.841 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:05.820+0000 7f19237fe640 1 --2- 172.21.15.39:0/3633707999 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d2031cb390 0x55d20359bd30 secure :-1 s=READY pgs=1276 cs=0 l=1 rev1=1 crypto rx=0x7f1918017000 tx=0x7f1918004f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=93423ed6f75091be server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:56:05.841 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:05.820+0000 7f19037fe640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-09-21T03:56:05.841 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:05.821+0000 7f19037fe640 1 -- 172.21.15.39:0/3633707999 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1918014040 con 0x55d2031cb390 2024-09-21T03:56:05.841 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:05.821+0000 7f19037fe640 1 -- 172.21.15.39:0/3633707999 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8528+0+0 (secure 0 0 0) 0x7f19180054f0 con 0x55d2031cb390 2024-09-21T03:56:05.841 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:05.821+0000 7f1930a70740 1 -- 172.21.15.39:0/3633707999 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f19140cd580 con 0x55d2031cb390 2024-09-21T03:56:05.841 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:05.821+0000 7f19037fe640 1 -- 172.21.15.39:0/3633707999 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f1918032050 con 0x55d2031cb390 2024-09-21T03:56:05.841 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:05.821+0000 7f1930a70740 10 client.18138 fetch_fsmap learned FSMap version 29 2024-09-21T03:56:05.841 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:05.821+0000 7f1930a70740 1 -- 172.21.15.39:0/3633707999 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55d203624ed0 con 0x55d2031cb390 2024-09-21T03:56:05.841 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:05.821+0000 7f19037fe640 1 -- 172.21.15.39:0/3633707999 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f19140cd580 con 0x55d2031cb390 2024-09-21T03:56:05.842 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:05.821+0000 7f19037fe640 1 -- 172.21.15.39:0/3633707999 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f19180108e0 con 0x55d2031cb390 2024-09-21T03:56:05.842 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:05.821+0000 7f1930a70740 10 client.18138 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T03:56:05.842 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:05.821+0000 7f1930a70740 10 client.18138 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T03:56:05.842 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:05.821+0000 7f1930a70740 20 client.18138 populate_metadata read hostname 'smithi039' 2024-09-21T03:56:05.842 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:05.821+0000 7f1930a70740 1 --2- 172.21.15.39:0/3633707999 >> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] conn(0x55d20317fda0 0x55d2036275a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:56:05.842 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:05.821+0000 7f1930a70740 4 client.18138 mds_command: new command op to 24695 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-21T03:56:05.842 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:05.821+0000 7f1930a70740 1 -- 172.21.15.39:0/3633707999 --> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55d203507850 con 0x55d20317fda0 2024-09-21T03:56:05.842 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:05.822+0000 7f1923fff640 1 --2- 172.21.15.39:0/3633707999 >> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] conn(0x55d20317fda0 0x55d2036275a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:56:05.843 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:05.822+0000 7f19037fe640 10 client.18138 ms_handle_connect on v2:172.21.15.150:6838/2949386907 2024-09-21T03:56:05.843 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:05.822+0000 7f1923fff640 1 --2- 172.21.15.39:0/3633707999 >> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] conn(0x55d20317fda0 0x55d2036275a0 crc :-1 s=READY pgs=162 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=85812bb130b488a6 server_cookie=63f0415a54e5a0c5 in_seq=0 out_seq=0 2024-09-21T03:56:05.843 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:05.822+0000 7f19037fe640 1 -- 172.21.15.39:0/3633707999 <== mds.0 v2:172.21.15.150:6838/2949386907 1 ==== command_reply(tid 0: 0 ) ==== 8+0+5486 (crc 0 0 0) 0x55d203507850 con 0x55d20317fda0 2024-09-21T03:56:05.843 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:05.822+0000 7f19037fe640 10 client.18138 handle_command_reply: tid=0 2024-09-21T03:56:05.843 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:05.822+0000 7f1930a70740 1 client.18138 shutdown 2024-09-21T03:56:05.843 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:05.822+0000 7f1930a70740 1 -- 172.21.15.39:0/3633707999 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d2031cb390 msgr2=0x55d20359bd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:56:05.843 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:05.822+0000 7f1930a70740 1 --2- 172.21.15.39:0/3633707999 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d2031cb390 0x55d20359bd30 secure :-1 s=READY pgs=1276 cs=0 l=1 rev1=1 crypto rx=0x7f1918017000 tx=0x7f1918004f40 comp rx=0 tx=0).stop 2024-09-21T03:56:05.843 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:05.823+0000 7f1930a70740 1 -- 172.21.15.39:0/3633707999 shutdown_connections 2024-09-21T03:56:05.843 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:05.823+0000 7f1930a70740 1 --2- 172.21.15.39:0/3633707999 >> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] conn(0x55d20317fda0 0x55d2036275a0 crc :-1 s=READY pgs=162 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:05.844 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:05.823+0000 7f19037fe640 0 client.18138 ms_handle_reset on v2:172.21.15.150:6838/2949386907 2024-09-21T03:56:05.844 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:05.823+0000 7f1930a70740 1 --2- 172.21.15.39:0/3633707999 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d2031ec070 0x55d2035ab3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:05.844 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:05.823+0000 7f1930a70740 1 --2- 172.21.15.39:0/3633707999 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d2031cb390 0x55d20359bd30 unknown :-1 s=CLOSED pgs=1276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:05.844 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:05.823+0000 7f1930a70740 1 --2- 172.21.15.39:0/3633707999 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d2031c31b0 0x55d20359b790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:05.844 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:05.823+0000 7f1930a70740 1 -- 172.21.15.39:0/3633707999 >> 172.21.15.39:0/3633707999 conn(0x55d2031e0460 msgr2=0x55d2035a45a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:56:05.844 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:05.823+0000 7f1930a70740 1 -- 172.21.15.39:0/3633707999 shutdown_connections 2024-09-21T03:56:05.844 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:05.823+0000 7f1930a70740 1 -- 172.21.15.39:0/3633707999 wait complete. 2024-09-21T03:56:05.844 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:05.823+0000 7f1930a70740 20 client.18138 trim_cache size 0 max 16384 2024-09-21T03:56:05.845 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-09-21T03:56:05.845 INFO:teuthology.orchestra.run.smithi039.stdout: "status": "no active scrubs running", 2024-09-21T03:56:05.845 INFO:teuthology.orchestra.run.smithi039.stdout: "scrubs": {}, 2024-09-21T03:56:05.845 INFO:teuthology.orchestra.run.smithi039.stdout: "scrub_stats": { 2024-09-21T03:56:05.846 INFO:teuthology.orchestra.run.smithi039.stdout: "a7293877-4df2-4bbb-be12-176711d2ac8a": { 2024-09-21T03:56:05.846 INFO:teuthology.orchestra.run.smithi039.stdout: "start_time": "2024-09-21T03:56:04.159511+0000", 2024-09-21T03:56:05.846 INFO:teuthology.orchestra.run.smithi039.stdout: "path": "/", 2024-09-21T03:56:05.846 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_started": 0, 2024-09-21T03:56:05.846 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_passed": 0, 2024-09-21T03:56:05.846 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_failed": 0, 2024-09-21T03:56:05.846 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_skipped": 0 2024-09-21T03:56:05.846 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:05.847 INFO:teuthology.orchestra.run.smithi039.stdout: "02e16ae1-0bac-41e9-b95e-fc6b17d207ac": { 2024-09-21T03:56:05.847 INFO:teuthology.orchestra.run.smithi039.stdout: "start_time": "2024-09-21T03:54:31.853401+0000", 2024-09-21T03:56:05.847 INFO:teuthology.orchestra.run.smithi039.stdout: "path": "/", 2024-09-21T03:56:05.847 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_started": 0, 2024-09-21T03:56:05.847 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_passed": 0, 2024-09-21T03:56:05.847 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_failed": 0, 2024-09-21T03:56:05.847 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_skipped": 0 2024-09-21T03:56:05.848 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:05.848 INFO:teuthology.orchestra.run.smithi039.stdout: "e28dcd13-f2ac-4cb7-a346-58917d542b15": { 2024-09-21T03:56:05.848 INFO:teuthology.orchestra.run.smithi039.stdout: "start_time": "2024-09-21T03:54:18.221680+0000", 2024-09-21T03:56:05.848 INFO:teuthology.orchestra.run.smithi039.stdout: "path": "/", 2024-09-21T03:56:05.848 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_started": 0, 2024-09-21T03:56:05.849 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_passed": 0, 2024-09-21T03:56:05.849 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_failed": 0, 2024-09-21T03:56:05.849 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_skipped": 0 2024-09-21T03:56:05.849 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:05.849 INFO:teuthology.orchestra.run.smithi039.stdout: "ad87da79-0248-4e18-9694-8228cb95422f": { 2024-09-21T03:56:05.849 INFO:teuthology.orchestra.run.smithi039.stdout: "start_time": "2024-09-21T03:53:18.441907+0000", 2024-09-21T03:56:05.849 INFO:teuthology.orchestra.run.smithi039.stdout: "path": "/", 2024-09-21T03:56:05.850 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_started": 0, 2024-09-21T03:56:05.850 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_passed": 0, 2024-09-21T03:56:05.850 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_failed": 0, 2024-09-21T03:56:05.850 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_skipped": 0 2024-09-21T03:56:05.850 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:05.850 INFO:teuthology.orchestra.run.smithi039.stdout: "02509e4f-3911-4983-b16f-df8ded0b9998": { 2024-09-21T03:56:05.851 INFO:teuthology.orchestra.run.smithi039.stdout: "start_time": "2024-09-21T03:46:17.889576+0000", 2024-09-21T03:56:05.851 INFO:teuthology.orchestra.run.smithi039.stdout: "path": "/", 2024-09-21T03:56:05.851 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_started": 0, 2024-09-21T03:56:05.851 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_passed": 0, 2024-09-21T03:56:05.851 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_failed": 0, 2024-09-21T03:56:05.851 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_skipped": 0 2024-09-21T03:56:05.851 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:05.851 INFO:teuthology.orchestra.run.smithi039.stdout: "10728de8-c49d-4edc-be2b-639249255e94": { 2024-09-21T03:56:05.852 INFO:teuthology.orchestra.run.smithi039.stdout: "start_time": "2024-09-21T03:47:03.965623+0000", 2024-09-21T03:56:05.852 INFO:teuthology.orchestra.run.smithi039.stdout: "path": "/", 2024-09-21T03:56:05.852 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_started": 0, 2024-09-21T03:56:05.852 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_passed": 0, 2024-09-21T03:56:05.852 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_failed": 0, 2024-09-21T03:56:05.852 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_skipped": 0 2024-09-21T03:56:05.853 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:05.853 INFO:teuthology.orchestra.run.smithi039.stdout: "10e282c4-137c-4fb2-9af2-a83dc3945587": { 2024-09-21T03:56:05.853 INFO:teuthology.orchestra.run.smithi039.stdout: "start_time": "2024-09-21T03:55:17.955458+0000", 2024-09-21T03:56:05.853 INFO:teuthology.orchestra.run.smithi039.stdout: "path": "/", 2024-09-21T03:56:05.853 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_started": 0, 2024-09-21T03:56:05.853 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_passed": 0, 2024-09-21T03:56:05.853 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_failed": 0, 2024-09-21T03:56:05.854 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_skipped": 0 2024-09-21T03:56:05.854 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:05.854 INFO:teuthology.orchestra.run.smithi039.stdout: "9860c2f4-b7b1-47aa-865a-9a1c4434a5f6": { 2024-09-21T03:56:05.854 INFO:teuthology.orchestra.run.smithi039.stdout: "start_time": "2024-09-21T03:49:17.718867+0000", 2024-09-21T03:56:05.855 INFO:teuthology.orchestra.run.smithi039.stdout: "path": "/", 2024-09-21T03:56:05.855 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_started": 0, 2024-09-21T03:56:05.855 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_passed": 0, 2024-09-21T03:56:05.855 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_failed": 0, 2024-09-21T03:56:05.855 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_skipped": 0 2024-09-21T03:56:05.855 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:05.856 INFO:teuthology.orchestra.run.smithi039.stdout: "d9391d69-baac-453a-9d52-d046760c070d": { 2024-09-21T03:56:05.856 INFO:teuthology.orchestra.run.smithi039.stdout: "start_time": "2024-09-21T03:53:32.364622+0000", 2024-09-21T03:56:05.856 INFO:teuthology.orchestra.run.smithi039.stdout: "path": "/", 2024-09-21T03:56:05.856 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_started": 0, 2024-09-21T03:56:05.856 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_passed": 0, 2024-09-21T03:56:05.857 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_failed": 0, 2024-09-21T03:56:05.857 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_skipped": 0 2024-09-21T03:56:05.857 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:05.857 INFO:teuthology.orchestra.run.smithi039.stdout: "c7c700f3-2cd8-41ac-8381-b57dd1b37b02": { 2024-09-21T03:56:05.857 INFO:teuthology.orchestra.run.smithi039.stdout: "start_time": "2024-09-21T03:52:32.368858+0000", 2024-09-21T03:56:05.858 INFO:teuthology.orchestra.run.smithi039.stdout: "path": "/", 2024-09-21T03:56:05.858 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_started": 0, 2024-09-21T03:56:05.858 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_passed": 0, 2024-09-21T03:56:05.858 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_failed": 0, 2024-09-21T03:56:05.858 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_skipped": 0 2024-09-21T03:56:05.858 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:05.858 INFO:teuthology.orchestra.run.smithi039.stdout: "29411307-eb8d-4339-8d2b-47ebd32c0080": { 2024-09-21T03:56:05.859 INFO:teuthology.orchestra.run.smithi039.stdout: "start_time": "2024-09-21T03:47:38.092920+0000", 2024-09-21T03:56:05.859 INFO:teuthology.orchestra.run.smithi039.stdout: "path": "/", 2024-09-21T03:56:05.859 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_started": 0, 2024-09-21T03:56:05.859 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_passed": 0, 2024-09-21T03:56:05.859 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_failed": 0, 2024-09-21T03:56:05.859 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_skipped": 0 2024-09-21T03:56:05.860 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:05.860 INFO:teuthology.orchestra.run.smithi039.stdout: "ede73e6e-3fd0-4a7f-b5bb-f68946f57eea": { 2024-09-21T03:56:05.860 INFO:teuthology.orchestra.run.smithi039.stdout: "start_time": "2024-09-21T03:51:32.235096+0000", 2024-09-21T03:56:05.860 INFO:teuthology.orchestra.run.smithi039.stdout: "path": "/", 2024-09-21T03:56:05.860 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_started": 0, 2024-09-21T03:56:05.860 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_passed": 0, 2024-09-21T03:56:05.861 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_failed": 0, 2024-09-21T03:56:05.861 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_skipped": 0 2024-09-21T03:56:05.861 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:05.861 INFO:teuthology.orchestra.run.smithi039.stdout: "e28ad743-2b22-45b3-8799-a638c2883623": { 2024-09-21T03:56:05.862 INFO:teuthology.orchestra.run.smithi039.stdout: "start_time": "2024-09-21T03:48:31.448821+0000", 2024-09-21T03:56:05.862 INFO:teuthology.orchestra.run.smithi039.stdout: "path": "/", 2024-09-21T03:56:05.863 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_started": 0, 2024-09-21T03:56:05.863 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_passed": 0, 2024-09-21T03:56:05.863 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_failed": 0, 2024-09-21T03:56:05.863 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_skipped": 0 2024-09-21T03:56:05.863 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:05.863 INFO:teuthology.orchestra.run.smithi039.stdout: "6e760f1f-9048-4f92-bd66-d708fbef75b3": { 2024-09-21T03:56:05.863 INFO:teuthology.orchestra.run.smithi039.stdout: "start_time": "2024-09-21T03:50:03.711918+0000", 2024-09-21T03:56:05.864 INFO:teuthology.orchestra.run.smithi039.stdout: "path": "/", 2024-09-21T03:56:05.864 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_started": 0, 2024-09-21T03:56:05.864 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_passed": 0, 2024-09-21T03:56:05.864 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_failed": 0, 2024-09-21T03:56:05.864 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_skipped": 0 2024-09-21T03:56:05.864 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:05.865 INFO:teuthology.orchestra.run.smithi039.stdout: "c7e2b8e4-057d-435f-b2bd-b633de98035f": { 2024-09-21T03:56:05.865 INFO:teuthology.orchestra.run.smithi039.stdout: "start_time": "2024-09-21T03:47:19.400305+0000", 2024-09-21T03:56:05.865 INFO:teuthology.orchestra.run.smithi039.stdout: "path": "/", 2024-09-21T03:56:05.865 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_started": 0, 2024-09-21T03:56:05.865 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_passed": 0, 2024-09-21T03:56:05.865 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_failed": 0, 2024-09-21T03:56:05.866 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_skipped": 0 2024-09-21T03:56:05.866 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:05.866 INFO:teuthology.orchestra.run.smithi039.stdout: "74ab917e-ecea-4778-a1c3-c184214fb24c": { 2024-09-21T03:56:05.866 INFO:teuthology.orchestra.run.smithi039.stdout: "start_time": "2024-09-21T03:50:17.726630+0000", 2024-09-21T03:56:05.866 INFO:teuthology.orchestra.run.smithi039.stdout: "path": "/", 2024-09-21T03:56:05.866 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_started": 0, 2024-09-21T03:56:05.866 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_passed": 0, 2024-09-21T03:56:05.867 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_failed": 0, 2024-09-21T03:56:05.867 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_skipped": 0 2024-09-21T03:56:05.867 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:05.867 INFO:teuthology.orchestra.run.smithi039.stdout: "160750d8-401f-4810-866d-e63e7e4bc3e0": { 2024-09-21T03:56:05.868 INFO:teuthology.orchestra.run.smithi039.stdout: "start_time": "2024-09-21T03:51:04.208673+0000", 2024-09-21T03:56:05.868 INFO:teuthology.orchestra.run.smithi039.stdout: "path": "/", 2024-09-21T03:56:05.868 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_started": 0, 2024-09-21T03:56:05.868 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_passed": 0, 2024-09-21T03:56:05.868 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_failed": 0, 2024-09-21T03:56:05.868 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_skipped": 0 2024-09-21T03:56:05.869 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:05.869 INFO:teuthology.orchestra.run.smithi039.stdout: "ae1fc4cf-5458-47d0-afcb-282b136dba15": { 2024-09-21T03:56:05.869 INFO:teuthology.orchestra.run.smithi039.stdout: "start_time": "2024-09-21T03:51:18.560378+0000", 2024-09-21T03:56:05.869 INFO:teuthology.orchestra.run.smithi039.stdout: "path": "/", 2024-09-21T03:56:05.869 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_started": 0, 2024-09-21T03:56:05.869 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_passed": 0, 2024-09-21T03:56:05.870 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_failed": 0, 2024-09-21T03:56:05.870 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_skipped": 0 2024-09-21T03:56:05.870 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:05.870 INFO:teuthology.orchestra.run.smithi039.stdout: "4bda6ee3-7237-4ee4-96ce-150ad50f26dc": { 2024-09-21T03:56:05.870 INFO:teuthology.orchestra.run.smithi039.stdout: "start_time": "2024-09-21T03:52:18.622141+0000", 2024-09-21T03:56:05.871 INFO:teuthology.orchestra.run.smithi039.stdout: "path": "/", 2024-09-21T03:56:05.871 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_started": 0, 2024-09-21T03:56:05.871 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_passed": 0, 2024-09-21T03:56:05.871 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_failed": 0, 2024-09-21T03:56:05.872 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_skipped": 0 2024-09-21T03:56:05.872 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-21T03:56:05.872 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-21T03:56:05.872 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-09-21T03:56:05.873 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:05.823+0000 7f192a4fc640 1 -- 172.21.15.39:0/3651323194 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f190c0b6730 msgr2=0x7f190c0b8c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:56:05.873 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:05.823+0000 7f192a4fc640 1 --2- 172.21.15.39:0/3651323194 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f190c0b6730 0x7f190c0b8c40 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f191c0092f0 tx=0x7f191c009740 comp rx=0 tx=0).stop 2024-09-21T03:56:05.873 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:05.823+0000 7f192a4fc640 1 -- 172.21.15.39:0/3651323194 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f19241049e0 msgr2=0x7f192419a530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:56:05.874 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:05.823+0000 7f192a4fc640 1 --2- 172.21.15.39:0/3651323194 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f19241049e0 0x7f192419a530 secure :-1 s=READY pgs=1273 cs=0 l=1 rev1=1 crypto rx=0x7f1914032fd0 tx=0x7f1914004270 comp rx=0 tx=0).stop 2024-09-21T03:56:05.874 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:05.823+0000 7f192a4fc640 1 -- 172.21.15.39:0/3651323194 shutdown_connections 2024-09-21T03:56:05.874 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:05.823+0000 7f192a4fc640 1 --2- 172.21.15.39:0/3651323194 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f190c0b6730 0x7f190c0b8c40 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:05.874 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:05.823+0000 7f192a4fc640 1 --2- 172.21.15.39:0/3651323194 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f192410de40 0x7f192419fd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:05.874 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:05.823+0000 7f192a4fc640 1 --2- 172.21.15.39:0/3651323194 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1924105330 0x7f192419aad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:05.874 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:05.823+0000 7f192a4fc640 1 --2- 172.21.15.39:0/3651323194 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f19241049e0 0x7f192419a530 unknown :-1 s=CLOSED pgs=1273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:05.874 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:05.823+0000 7f192a4fc640 1 -- 172.21.15.39:0/3651323194 >> 172.21.15.39:0/3651323194 conn(0x7f1924079750 msgr2=0x7f1924100870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:56:05.875 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:05.824+0000 7f192a4fc640 1 -- 172.21.15.39:0/3651323194 shutdown_connections 2024-09-21T03:56:05.875 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:05.824+0000 7f192a4fc640 1 -- 172.21.15.39:0/3651323194 wait complete. 2024-09-21T03:56:06.717 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-21T03:56:06.717 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 51116d5c-77ca-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T03:56:06.977 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/51116d5c-77ca-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:56:07.333 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:56:07 smithi039 ceph-mon[31211]: pgmap v633: 97 pgs: 97 active+clean; 10 MiB data, 525 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 44 KiB/s wr, 27 op/s 2024-09-21T03:56:07.468 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:56:07 smithi161 ceph-mon[34533]: pgmap v633: 97 pgs: 97 active+clean; 10 MiB data, 525 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 44 KiB/s wr, 27 op/s 2024-09-21T03:56:07.475 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:56:07 smithi150 ceph-mon[34289]: pgmap v633: 97 pgs: 97 active+clean; 10 MiB data, 525 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 44 KiB/s wr, 27 op/s 2024-09-21T03:56:07.504 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:07.494+0000 7f107a20a640 1 -- 172.21.15.39:0/2098505241 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1074102ae0 msgr2=0x7f1074102ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:56:07.504 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:07.494+0000 7f107a20a640 1 --2- 172.21.15.39:0/2098505241 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1074102ae0 0x7f1074102ef0 secure :-1 s=READY pgs=1277 cs=0 l=1 rev1=1 crypto rx=0x7f1064009900 tx=0x7f106402a410 comp rx=0 tx=0).stop 2024-09-21T03:56:07.504 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:07.494+0000 7f107a20a640 1 -- 172.21.15.39:0/2098505241 shutdown_connections 2024-09-21T03:56:07.504 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:07.494+0000 7f107a20a640 1 --2- 172.21.15.39:0/2098505241 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1074108d20 0x7f107410f550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:07.504 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:07.494+0000 7f107a20a640 1 --2- 172.21.15.39:0/2098505241 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f10741083d0 0x7f10741087e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:07.504 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:07.494+0000 7f107a20a640 1 --2- 172.21.15.39:0/2098505241 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1074102ae0 0x7f1074102ef0 unknown :-1 s=CLOSED pgs=1277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:07.504 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:07.494+0000 7f107a20a640 1 -- 172.21.15.39:0/2098505241 >> 172.21.15.39:0/2098505241 conn(0x7f1074079760 msgr2=0x7f1074079b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:56:07.504 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:07.494+0000 7f107a20a640 1 -- 172.21.15.39:0/2098505241 shutdown_connections 2024-09-21T03:56:07.505 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:07.494+0000 7f107a20a640 1 -- 172.21.15.39:0/2098505241 wait complete. 2024-09-21T03:56:07.505 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:07.494+0000 7f107a20a640 1 Processor -- start 2024-09-21T03:56:07.505 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:07.495+0000 7f107a20a640 1 -- start start 2024-09-21T03:56:07.505 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:07.495+0000 7f107a20a640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1074102ae0 0x7f107419a440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:56:07.505 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:07.495+0000 7f107a20a640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f10741083d0 0x7f107419a9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:56:07.505 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:07.495+0000 7f107a20a640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1074108d20 0x7f107419fc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:56:07.505 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:07.495+0000 7f107a20a640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f1074113ab0 con 0x7f10741083d0 2024-09-21T03:56:07.505 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:07.495+0000 7f107a20a640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f1074113930 con 0x7f1074102ae0 2024-09-21T03:56:07.505 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:07.495+0000 7f107a20a640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f1074113c30 con 0x7f1074108d20 2024-09-21T03:56:07.505 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:07.495+0000 7f1072ffd640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f10741083d0 0x7f107419a9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:56:07.505 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:07.495+0000 7f1072ffd640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f10741083d0 0x7f107419a9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:59624/0 (socket says 172.21.15.39:59624) 2024-09-21T03:56:07.505 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:07.495+0000 7f1072ffd640 1 -- 172.21.15.39:0/2668377733 learned_addr learned my addr 172.21.15.39:0/2668377733 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:56:07.505 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:07.495+0000 7f1073fff640 1 --2- 172.21.15.39:0/2668377733 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1074108d20 0x7f107419fc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:56:07.505 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:07.495+0000 7f10737fe640 1 --2- 172.21.15.39:0/2668377733 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1074102ae0 0x7f107419a440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:56:07.506 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:07.495+0000 7f1072ffd640 1 -- 172.21.15.39:0/2668377733 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1074108d20 msgr2=0x7f107419fc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:56:07.506 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:07.495+0000 7f1072ffd640 1 --2- 172.21.15.39:0/2668377733 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1074108d20 0x7f107419fc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:07.506 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:07.495+0000 7f1072ffd640 1 -- 172.21.15.39:0/2668377733 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1074102ae0 msgr2=0x7f107419a440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:56:07.506 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:07.495+0000 7f1072ffd640 1 --2- 172.21.15.39:0/2668377733 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1074102ae0 0x7f107419a440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:07.506 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:07.495+0000 7f1072ffd640 1 -- 172.21.15.39:0/2668377733 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f10741a03b0 con 0x7f10741083d0 2024-09-21T03:56:07.506 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:07.496+0000 7f1072ffd640 1 --2- 172.21.15.39:0/2668377733 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f10741083d0 0x7f107419a9e0 secure :-1 s=READY pgs=1278 cs=0 l=1 rev1=1 crypto rx=0x7f106800d770 tx=0x7f106800dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:56:07.506 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:07.496+0000 7f1070ff9640 1 -- 172.21.15.39:0/2668377733 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1068017070 con 0x7f10741083d0 2024-09-21T03:56:07.506 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:07.496+0000 7f1070ff9640 1 -- 172.21.15.39:0/2668377733 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1068009c30 con 0x7f10741083d0 2024-09-21T03:56:07.506 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:07.496+0000 7f1070ff9640 1 -- 172.21.15.39:0/2668377733 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f106800f620 con 0x7f10741083d0 2024-09-21T03:56:07.506 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:07.496+0000 7f107a20a640 1 -- 172.21.15.39:0/2668377733 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f10741a06a0 con 0x7f10741083d0 2024-09-21T03:56:07.506 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:07.496+0000 7f107a20a640 1 -- 172.21.15.39:0/2668377733 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f10741a6e00 con 0x7f10741083d0 2024-09-21T03:56:07.506 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:07.497+0000 7f107a20a640 1 -- 172.21.15.39:0/2668377733 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1074103d10 con 0x7f10741083d0 2024-09-21T03:56:07.506 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:07.499+0000 7f1070ff9640 1 -- 172.21.15.39:0/2668377733 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 27) ==== 154643+0+0 (secure 0 0 0) 0x7f106801a070 con 0x7f10741083d0 2024-09-21T03:56:07.506 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:07.499+0000 7f1070ff9640 1 --2- 172.21.15.39:0/2668377733 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f105c0b6700 0x7f105c0b8c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:56:07.507 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:07.499+0000 7f10737fe640 1 --2- 172.21.15.39:0/2668377733 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f105c0b6700 0x7f105c0b8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:56:07.507 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:07.499+0000 7f1070ff9640 1 -- 172.21.15.39:0/2668377733 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8528+0+0 (secure 0 0 0) 0x7f10680e70d0 con 0x7f10741083d0 2024-09-21T03:56:07.507 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:07.500+0000 7f10737fe640 1 --2- 172.21.15.39:0/2668377733 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f105c0b6700 0x7f105c0b8c10 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f1064031440 tx=0x7f1064009560 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:56:07.507 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:07.503+0000 7f1070ff9640 1 -- 172.21.15.39:0/2668377733 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f1068009de0 con 0x7f10741083d0 2024-09-21T03:56:07.731 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:07.728+0000 7f107a20a640 1 -- 172.21.15.39:0/2668377733 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1074102ef0 con 0x7f10741083d0 2024-09-21T03:56:07.732 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:07.729+0000 7f1070ff9640 1 -- 172.21.15.39:0/2668377733 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11978 (secure 0 0 0) 0x7f1074102ef0 con 0x7f10741083d0 2024-09-21T03:56:07.732 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T03:56:07.732 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":29,"btime":"2024-09-21T03:46:08:982881+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14790,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6835/508995295","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":508995295},{"type":"v1","addr":"172.21.15.39:6835","nonce":508995295}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14802,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6837/3915912607","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":3915912607},{"type":"v1","addr":"172.21.15.39:6837","nonce":3915912607}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":29,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T03:44:15.009872+0000","modified":"2024-09-21T03:46:08.430113+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24695,"mds_1":24667,"mds_2":24659,"mds_3":24646,"mds_4":14868},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14856":{"gid":14856,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.39:6839/185509192","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6838","nonce":185509192},{"type":"v1","addr":"172.21.15.39:6839","nonce":185509192}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14868":{"gid":14868,"name":"a","rank":4,"incarnation":25,"state":"up:active","state_seq":7,"addr":"172.21.15.39:6841/4247190456","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6840","nonce":4247190456},{"type":"v1","addr":"172.21.15.39:6841","nonce":4247190456}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24599":{"gid":24599,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.150:6833/3114643217","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":3114643217},{"type":"v1","addr":"172.21.15.150:6833","nonce":3114643217}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24601":{"gid":24601,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.161:6833/2963990654","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":2963990654},{"type":"v1","addr":"172.21.15.161:6833","nonce":2963990654}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24632":{"gid":24632,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.150:6835/724594008","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6834","nonce":724594008},{"type":"v1","addr":"172.21.15.150:6835","nonce":724594008}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24640":{"gid":24640,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.161:6835/3747882110","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":3747882110},{"type":"v1","addr":"172.21.15.161:6835","nonce":3747882110}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24646":{"gid":24646,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":10,"addr":"172.21.15.161:6837/3451544597","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6836","nonce":3451544597},{"type":"v1","addr":"172.21.15.161:6837","nonce":3451544597}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24659":{"gid":24659,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.150:6837/4017043144","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6836","nonce":4017043144},{"type":"v1","addr":"172.21.15.150:6837","nonce":4017043144}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24667":{"gid":24667,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.161:6839/2190107270","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6838","nonce":2190107270},{"type":"v1","addr":"172.21.15.161:6839","nonce":2190107270}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24695":{"gid":24695,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.150:6839/2949386907","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6838","nonce":2949386907},{"type":"v1","addr":"172.21.15.150:6839","nonce":2949386907}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24695,"qdb_cluster":[24695,24667,24659,24646,14868]},"id":1}]} 2024-09-21T03:56:07.732 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 29 2024-09-21T03:56:07.735 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:07.732+0000 7f107a20a640 1 -- 172.21.15.39:0/2668377733 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f105c0b6700 msgr2=0x7f105c0b8c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:56:07.736 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:07.732+0000 7f107a20a640 1 --2- 172.21.15.39:0/2668377733 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f105c0b6700 0x7f105c0b8c10 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f1064031440 tx=0x7f1064009560 comp rx=0 tx=0).stop 2024-09-21T03:56:07.736 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:07.732+0000 7f107a20a640 1 -- 172.21.15.39:0/2668377733 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f10741083d0 msgr2=0x7f107419a9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:56:07.736 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:07.732+0000 7f107a20a640 1 --2- 172.21.15.39:0/2668377733 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f10741083d0 0x7f107419a9e0 secure :-1 s=READY pgs=1278 cs=0 l=1 rev1=1 crypto rx=0x7f106800d770 tx=0x7f106800dc30 comp rx=0 tx=0).stop 2024-09-21T03:56:07.736 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:07.733+0000 7f107a20a640 1 -- 172.21.15.39:0/2668377733 shutdown_connections 2024-09-21T03:56:07.736 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:07.733+0000 7f107a20a640 1 --2- 172.21.15.39:0/2668377733 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f105c0b6700 0x7f105c0b8c10 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:07.736 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:07.733+0000 7f107a20a640 1 --2- 172.21.15.39:0/2668377733 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1074108d20 0x7f107419fc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:07.736 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:07.733+0000 7f107a20a640 1 --2- 172.21.15.39:0/2668377733 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f10741083d0 0x7f107419a9e0 unknown :-1 s=CLOSED pgs=1278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:07.736 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:07.733+0000 7f107a20a640 1 --2- 172.21.15.39:0/2668377733 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1074102ae0 0x7f107419a440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:07.736 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:07.733+0000 7f107a20a640 1 -- 172.21.15.39:0/2668377733 >> 172.21.15.39:0/2668377733 conn(0x7f1074079760 msgr2=0x7f1074101580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:56:07.736 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:07.733+0000 7f107a20a640 1 -- 172.21.15.39:0/2668377733 shutdown_connections 2024-09-21T03:56:07.736 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:07.733+0000 7f107a20a640 1 -- 172.21.15.39:0/2668377733 wait complete. 2024-09-21T03:56:08.468 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:56:08 smithi161 ceph-mon[34533]: from='client.? 172.21.15.39:0/2668377733' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:56:08.475 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:56:08 smithi150 ceph-mon[34289]: from='client.? 172.21.15.39:0/2668377733' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:56:08.475 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:56:08 smithi039 ceph-mon[31211]: from='client.? 172.21.15.39:0/2668377733' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:56:08.508 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 51116d5c-77ca-11ef-baf5-efdc52797490 -- ceph tell mds.1:4 damage ls 2024-09-21T03:56:08.768 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/51116d5c-77ca-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:56:09.187 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:56:09 smithi039 ceph-mon[31211]: pgmap v634: 97 pgs: 97 active+clean; 10 MiB data, 525 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 44 KiB/s wr, 25 op/s 2024-09-21T03:56:09.313 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:09.277+0000 7f129a921640 1 -- 172.21.15.39:0/1091855312 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1294102100 msgr2=0x7f1294102510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:56:09.313 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:09.277+0000 7f129a921640 1 --2- 172.21.15.39:0/1091855312 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1294102100 0x7f1294102510 secure :-1 s=READY pgs=1279 cs=0 l=1 rev1=1 crypto rx=0x7f1284009900 tx=0x7f128402a410 comp rx=0 tx=0).stop 2024-09-21T03:56:09.313 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:09.277+0000 7f129a921640 1 -- 172.21.15.39:0/1091855312 shutdown_connections 2024-09-21T03:56:09.313 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:09.278+0000 7f129a921640 1 --2- 172.21.15.39:0/1091855312 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1294109ea0 0x7f12941106d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:09.313 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:09.278+0000 7f129a921640 1 --2- 172.21.15.39:0/1091855312 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1294109550 0x7f1294109960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:09.313 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:09.278+0000 7f129a921640 1 --2- 172.21.15.39:0/1091855312 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1294102100 0x7f1294102510 unknown :-1 s=CLOSED pgs=1279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:09.313 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:09.278+0000 7f129a921640 1 -- 172.21.15.39:0/1091855312 >> 172.21.15.39:0/1091855312 conn(0x7f1294079ea0 msgr2=0x7f129407a2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:56:09.313 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:09.278+0000 7f129a921640 1 -- 172.21.15.39:0/1091855312 shutdown_connections 2024-09-21T03:56:09.314 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:09.278+0000 7f129a921640 1 -- 172.21.15.39:0/1091855312 wait complete. 2024-09-21T03:56:09.314 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:09.278+0000 7f129a921640 1 Processor -- start 2024-09-21T03:56:09.314 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:09.278+0000 7f129a921640 1 -- start start 2024-09-21T03:56:09.314 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:09.279+0000 7f129a921640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1294102100 0x7f129419a590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:56:09.314 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:09.279+0000 7f129a921640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1294109550 0x7f129419ab30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:56:09.314 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:09.279+0000 7f129a921640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1294109ea0 0x7f129419fdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:56:09.314 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:09.279+0000 7f129a921640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f1294113bc0 con 0x7f1294109ea0 2024-09-21T03:56:09.314 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:09.279+0000 7f129a921640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f1294113a40 con 0x7f1294109550 2024-09-21T03:56:09.314 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:09.279+0000 7f129a921640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f1294113d40 con 0x7f1294102100 2024-09-21T03:56:09.314 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:09.279+0000 7f1298ebe640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1294109ea0 0x7f129419fdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:56:09.314 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:09.279+0000 7f1298ebe640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1294109ea0 0x7f129419fdd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:59648/0 (socket says 172.21.15.39:59648) 2024-09-21T03:56:09.314 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:09.279+0000 7f1298ebe640 1 -- 172.21.15.39:0/602340387 learned_addr learned my addr 172.21.15.39:0/602340387 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:56:09.314 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:09.279+0000 7f1293fff640 1 --2- 172.21.15.39:0/602340387 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1294102100 0x7f129419a590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:56:09.315 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:09.279+0000 7f12937fe640 1 --2- 172.21.15.39:0/602340387 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1294109550 0x7f129419ab30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:56:09.315 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:09.279+0000 7f1298ebe640 1 -- 172.21.15.39:0/602340387 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1294102100 msgr2=0x7f129419a590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:56:09.315 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:09.279+0000 7f1298ebe640 1 --2- 172.21.15.39:0/602340387 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1294102100 0x7f129419a590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:09.315 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:09.279+0000 7f1298ebe640 1 -- 172.21.15.39:0/602340387 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1294109550 msgr2=0x7f129419ab30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:56:09.315 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:09.279+0000 7f1298ebe640 1 --2- 172.21.15.39:0/602340387 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1294109550 0x7f129419ab30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:09.315 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:09.279+0000 7f1298ebe640 1 -- 172.21.15.39:0/602340387 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f12941a0500 con 0x7f1294109ea0 2024-09-21T03:56:09.315 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:09.279+0000 7f1298ebe640 1 --2- 172.21.15.39:0/602340387 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1294109ea0 0x7f129419fdd0 secure :-1 s=READY pgs=1280 cs=0 l=1 rev1=1 crypto rx=0x7f128c00cad0 tx=0x7f128c00cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:56:09.315 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:09.280+0000 7f12917fa640 1 -- 172.21.15.39:0/602340387 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f128c00ae20 con 0x7f1294109ea0 2024-09-21T03:56:09.315 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:09.280+0000 7f12917fa640 1 -- 172.21.15.39:0/602340387 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f128c005020 con 0x7f1294109ea0 2024-09-21T03:56:09.315 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:09.280+0000 7f129a921640 1 -- 172.21.15.39:0/602340387 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f12941a07f0 con 0x7f1294109ea0 2024-09-21T03:56:09.315 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:09.280+0000 7f12917fa640 1 -- 172.21.15.39:0/602340387 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f128c00f6b0 con 0x7f1294109ea0 2024-09-21T03:56:09.315 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:09.280+0000 7f129a921640 1 -- 172.21.15.39:0/602340387 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f12941a6f50 con 0x7f1294109ea0 2024-09-21T03:56:09.315 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:09.282+0000 7f12a0e95740 1 Processor -- start 2024-09-21T03:56:09.315 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:09.282+0000 7f12917fa640 1 -- 172.21.15.39:0/602340387 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 27) ==== 154643+0+0 (secure 0 0 0) 0x7f128c0051d0 con 0x7f1294109ea0 2024-09-21T03:56:09.316 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:09.282+0000 7f12a0e95740 1 -- start start 2024-09-21T03:56:09.316 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:09.283+0000 7f12a0e95740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x555ffd5e7070 0x555ffd98e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:56:09.316 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:09.283+0000 7f12a0e95740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555ffd5be1b0 0x555ffd98e800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:56:09.316 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:09.283+0000 7f12a0e95740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555ffd5c6390 0x555ffd98ed40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:56:09.316 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:09.283+0000 7f12a0e95740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x555ffd904ea0 con 0x555ffd5e7070 2024-09-21T03:56:09.316 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:09.283+0000 7f12a0e95740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x555ffd8f42d0 con 0x555ffd5be1b0 2024-09-21T03:56:09.316 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:09.283+0000 7f12a0e95740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x555ffd8f80e0 con 0x555ffd5c6390 2024-09-21T03:56:09.316 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:09.283+0000 7f12937fe640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x555ffd5e7070 0x555ffd98e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:56:09.316 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:09.283+0000 7f12937fe640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x555ffd5e7070 0x555ffd98e2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:59650/0 (socket says 172.21.15.39:59650) 2024-09-21T03:56:09.316 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:09.283+0000 7f12937fe640 1 -- 172.21.15.39:0/3601909783 learned_addr learned my addr 172.21.15.39:0/3601909783 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:56:09.316 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:09.283+0000 7f12937fe640 1 --2- 172.21.15.39:0/3601909783 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555ffd5c6390 0x555ffd98ed40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:56:09.316 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:09.283+0000 7f1293fff640 1 --2- 172.21.15.39:0/3601909783 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555ffd5be1b0 0x555ffd98e800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:56:09.316 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:09.283+0000 7f12917fa640 1 --2- 172.21.15.39:0/602340387 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f127c0b6680 0x7f127c0b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:56:09.316 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:09.283+0000 7f12937fe640 1 -- 172.21.15.39:0/3601909783 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555ffd5c6390 msgr2=0x555ffd98ed40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:56:09.317 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:09.283+0000 7f12937fe640 1 --2- 172.21.15.39:0/3601909783 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555ffd5c6390 0x555ffd98ed40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:09.317 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:09.283+0000 7f12937fe640 1 -- 172.21.15.39:0/3601909783 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555ffd5be1b0 msgr2=0x555ffd98e800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:56:09.317 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:09.283+0000 7f12937fe640 1 --2- 172.21.15.39:0/3601909783 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555ffd5be1b0 0x555ffd98e800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:09.317 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:09.283+0000 7f12937fe640 1 -- 172.21.15.39:0/3601909783 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555ffd652650 con 0x555ffd5e7070 2024-09-21T03:56:09.317 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:09.283+0000 7f1298ebe640 1 --2- 172.21.15.39:0/602340387 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f127c0b6680 0x7f127c0b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:56:09.317 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:09.284+0000 7f12937fe640 1 --2- 172.21.15.39:0/3601909783 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x555ffd5e7070 0x555ffd98e2c0 secure :-1 s=READY pgs=1281 cs=0 l=1 rev1=1 crypto rx=0x7f1288006aa0 tx=0x7f128800b400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2730868828df26cf server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:56:09.317 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:09.284+0000 7f12917fa640 1 -- 172.21.15.39:0/602340387 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8528+0+0 (secure 0 0 0) 0x7f128c0e7750 con 0x7f1294109ea0 2024-09-21T03:56:09.317 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:09.284+0000 7f127a7fc640 1 -- 172.21.15.39:0/3601909783 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12880173f0 con 0x555ffd5e7070 2024-09-21T03:56:09.317 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:09.284+0000 7f127a7fc640 1 -- 172.21.15.39:0/3601909783 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1288010040 con 0x555ffd5e7070 2024-09-21T03:56:09.317 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:09.284+0000 7f127a7fc640 1 -- 172.21.15.39:0/3601909783 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12880169c0 con 0x555ffd5e7070 2024-09-21T03:56:09.317 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:09.285+0000 7f12a0e95740 1 -- 172.21.15.39:0/3601909783 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x555ffd5e7070 msgr2=0x555ffd98e2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:56:09.317 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:09.285+0000 7f1298ebe640 1 --2- 172.21.15.39:0/602340387 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f127c0b6680 0x7f127c0b8b90 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f128c0ad530 tx=0x7f128c07a880 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:56:09.317 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:09.285+0000 7f12a0e95740 1 --2- 172.21.15.39:0/3601909783 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x555ffd5e7070 0x555ffd98e2c0 secure :-1 s=READY pgs=1281 cs=0 l=1 rev1=1 crypto rx=0x7f1288006aa0 tx=0x7f128800b400 comp rx=0 tx=0).stop 2024-09-21T03:56:09.317 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:09.285+0000 7f12a0e95740 1 -- 172.21.15.39:0/3601909783 shutdown_connections 2024-09-21T03:56:09.317 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:09.285+0000 7f12a0e95740 1 --2- 172.21.15.39:0/3601909783 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555ffd5c6390 0x555ffd98ed40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:09.318 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:09.285+0000 7f12a0e95740 1 --2- 172.21.15.39:0/3601909783 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555ffd5be1b0 0x555ffd98e800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:09.318 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:09.285+0000 7f12a0e95740 1 --2- 172.21.15.39:0/3601909783 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x555ffd5e7070 0x555ffd98e2c0 unknown :-1 s=CLOSED pgs=1281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:09.318 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:09.285+0000 7f12a0e95740 1 -- 172.21.15.39:0/3601909783 >> 172.21.15.39:0/3601909783 conn(0x555ffd5db460 msgr2=0x555ffd98c1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:56:09.318 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:09.285+0000 7f12a0e95740 1 -- 172.21.15.39:0/3601909783 shutdown_connections 2024-09-21T03:56:09.318 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:09.285+0000 7f12a0e95740 1 -- 172.21.15.39:0/3601909783 wait complete. 2024-09-21T03:56:09.318 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:09.285+0000 7f12a0e95740 1 -- start start 2024-09-21T03:56:09.318 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:09.286+0000 7f12a0e95740 1 Processor -- start 2024-09-21T03:56:09.318 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:09.286+0000 7f12a0e95740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555ffd5be1b0 0x555ffd999910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:56:09.318 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:09.286+0000 7f12a0e95740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x555ffd5c6390 0x555ffd999e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:56:09.318 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:09.286+0000 7f12a0e95740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555ffd5e7070 0x555ffd99a390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:56:09.318 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:09.286+0000 7f12a0e95740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x555ffd755700 con 0x555ffd5c6390 2024-09-21T03:56:09.318 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:09.286+0000 7f12a0e95740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x555ffd88f160 con 0x555ffd5e7070 2024-09-21T03:56:09.318 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:09.286+0000 7f12a0e95740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x555ffd99ca40 con 0x555ffd5be1b0 2024-09-21T03:56:09.319 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:09.286+0000 7f12a0e95740 10 client.0 _ll_register_callbacks cb 0x555ffd8a89e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:56:09.319 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:09.286+0000 7f1293fff640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x555ffd5c6390 0x555ffd999e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:56:09.319 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:09.286+0000 7f1293fff640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x555ffd5c6390 0x555ffd999e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:59656/0 (socket says 172.21.15.39:59656) 2024-09-21T03:56:09.319 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:09.286+0000 7f1293fff640 1 -- 172.21.15.39:0/169084116 learned_addr learned my addr 172.21.15.39:0/169084116 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:56:09.319 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:09.286+0000 7f12937fe640 1 --2- 172.21.15.39:0/169084116 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555ffd5be1b0 0x555ffd999910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:56:09.319 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:09.286+0000 7f12937fe640 1 --2- 172.21.15.39:0/169084116 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555ffd5e7070 0x555ffd99a390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:56:09.319 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:09.286+0000 7f1293fff640 1 -- 172.21.15.39:0/169084116 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555ffd5be1b0 msgr2=0x555ffd999910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:56:09.319 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:09.286+0000 7f1293fff640 1 --2- 172.21.15.39:0/169084116 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555ffd5be1b0 0x555ffd999910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:09.319 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:09.286+0000 7f1293fff640 1 -- 172.21.15.39:0/169084116 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555ffd5e7070 msgr2=0x555ffd99a390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:56:09.319 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:09.286+0000 7f1293fff640 1 --2- 172.21.15.39:0/169084116 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555ffd5e7070 0x555ffd99a390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:09.319 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:09.286+0000 7f1293fff640 1 -- 172.21.15.39:0/169084116 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555ffd8bcc90 con 0x555ffd5c6390 2024-09-21T03:56:09.319 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:09.286+0000 7f12937fe640 1 --2- 172.21.15.39:0/169084116 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555ffd5e7070 0x555ffd99a390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:56:09.319 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:09.287+0000 7f1293fff640 1 --2- 172.21.15.39:0/169084116 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x555ffd5c6390 0x555ffd999e50 secure :-1 s=READY pgs=1282 cs=0 l=1 rev1=1 crypto rx=0x7f1284009900 tx=0x7f12840316d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f65d643d0156f93 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:56:09.319 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:09.287+0000 7f12737fe640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-09-21T03:56:09.320 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:09.287+0000 7f12737fe640 1 -- 172.21.15.39:0/169084116 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1284035040 con 0x555ffd5c6390 2024-09-21T03:56:09.320 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:09.287+0000 7f12737fe640 1 -- 172.21.15.39:0/169084116 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8528+0+0 (secure 0 0 0) 0x7f1284004040 con 0x555ffd5c6390 2024-09-21T03:56:09.320 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:09.287+0000 7f12a0e95740 1 -- 172.21.15.39:0/169084116 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1288010040 con 0x555ffd5c6390 2024-09-21T03:56:09.320 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:09.287+0000 7f12737fe640 1 -- 172.21.15.39:0/169084116 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f1284004240 con 0x555ffd5c6390 2024-09-21T03:56:09.320 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:09.287+0000 7f12a0e95740 10 client.18156 fetch_fsmap learned FSMap version 29 2024-09-21T03:56:09.320 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:09.287+0000 7f12a0e95740 1 -- 172.21.15.39:0/169084116 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x555ffd9a8690 con 0x555ffd5c6390 2024-09-21T03:56:09.320 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:09.287+0000 7f12737fe640 1 -- 172.21.15.39:0/169084116 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1288010040 con 0x555ffd5c6390 2024-09-21T03:56:09.320 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:09.287+0000 7f12737fe640 1 -- 172.21.15.39:0/169084116 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f1284004a10 con 0x555ffd5c6390 2024-09-21T03:56:09.320 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:09.288+0000 7f12a0e95740 10 client.18156 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T03:56:09.320 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:09.288+0000 7f12a0e95740 10 client.18156 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.a 2024-09-21T03:56:09.320 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:09.288+0000 7f12a0e95740 20 client.18156 populate_metadata read hostname 'smithi039' 2024-09-21T03:56:09.320 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:09.288+0000 7f12a0e95740 1 --2- 172.21.15.39:0/169084116 >> [v2:172.21.15.39:6840/4247190456,v1:172.21.15.39:6841/4247190456] conn(0x555ffd5a59f0 0x555ffd9abc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:56:09.320 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:09.288+0000 7f12a0e95740 4 client.18156 mds_command: new command op to 14868 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:56:09.320 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:09.288+0000 7f12a0e95740 1 -- 172.21.15.39:0/169084116 --> [v2:172.21.15.39:6840/4247190456,v1:172.21.15.39:6841/4247190456] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555ffd906df0 con 0x555ffd5a59f0 2024-09-21T03:56:09.320 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:09.288+0000 7f12937fe640 1 --2- 172.21.15.39:0/169084116 >> [v2:172.21.15.39:6840/4247190456,v1:172.21.15.39:6841/4247190456] conn(0x555ffd5a59f0 0x555ffd9abc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:56:09.321 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:09.288+0000 7f12937fe640 1 --2- 172.21.15.39:0/169084116 >> [v2:172.21.15.39:6840/4247190456,v1:172.21.15.39:6841/4247190456] conn(0x555ffd5a59f0 0x555ffd9abc20 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=fc11b74c6deef64f server_cookie=e18e11db6637f3cd in_seq=0 out_seq=0 2024-09-21T03:56:09.321 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:09.288+0000 7f12737fe640 10 client.18156 ms_handle_connect on v2:172.21.15.39:6840/4247190456 2024-09-21T03:56:09.321 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:09.289+0000 7f12737fe640 1 -- 172.21.15.39:0/169084116 <== mds.4 v2:172.21.15.39:6840/4247190456 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x555ffd906df0 con 0x555ffd5a59f0 2024-09-21T03:56:09.321 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:09.289+0000 7f12737fe640 10 client.18156 handle_command_reply: tid=0 2024-09-21T03:56:09.321 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:09.289+0000 7f12a0e95740 1 client.18156 shutdown 2024-09-21T03:56:09.321 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:09.289+0000 7f12a0e95740 1 -- 172.21.15.39:0/169084116 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x555ffd5c6390 msgr2=0x555ffd999e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:56:09.321 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:09.289+0000 7f12a0e95740 1 --2- 172.21.15.39:0/169084116 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x555ffd5c6390 0x555ffd999e50 secure :-1 s=READY pgs=1282 cs=0 l=1 rev1=1 crypto rx=0x7f1284009900 tx=0x7f12840316d0 comp rx=0 tx=0).stop 2024-09-21T03:56:09.321 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:09.289+0000 7f12a0e95740 1 -- 172.21.15.39:0/169084116 shutdown_connections 2024-09-21T03:56:09.321 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:09.289+0000 7f12a0e95740 1 --2- 172.21.15.39:0/169084116 >> [v2:172.21.15.39:6840/4247190456,v1:172.21.15.39:6841/4247190456] conn(0x555ffd5a59f0 0x555ffd9abc20 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:09.321 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:09.289+0000 7f12a0e95740 1 --2- 172.21.15.39:0/169084116 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555ffd5e7070 0x555ffd99a390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:09.321 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:09.289+0000 7f12737fe640 0 client.18156 ms_handle_reset on v2:172.21.15.39:6840/4247190456 2024-09-21T03:56:09.321 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:09.289+0000 7f12a0e95740 1 --2- 172.21.15.39:0/169084116 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x555ffd5c6390 0x555ffd999e50 unknown :-1 s=CLOSED pgs=1282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:09.321 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:09.289+0000 7f12a0e95740 1 --2- 172.21.15.39:0/169084116 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555ffd5be1b0 0x555ffd999910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:09.321 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:09.289+0000 7f12a0e95740 1 -- 172.21.15.39:0/169084116 >> 172.21.15.39:0/169084116 conn(0x555ffd5db460 msgr2=0x555ffd9914a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:56:09.321 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:09.289+0000 7f12a0e95740 1 -- 172.21.15.39:0/169084116 shutdown_connections 2024-09-21T03:56:09.321 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:09.289+0000 7f12a0e95740 1 -- 172.21.15.39:0/169084116 wait complete. 2024-09-21T03:56:09.322 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:09.289+0000 7f12a0e95740 20 client.18156 trim_cache size 0 max 16384 2024-09-21T03:56:09.322 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:09.302+0000 7f12a0e95740 1 Processor -- start 2024-09-21T03:56:09.322 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:09.303+0000 7f12a0e95740 1 -- start start 2024-09-21T03:56:09.322 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:09.303+0000 7f12a0e95740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555ffd5a59f0 0x555ffd98fcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:56:09.322 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:09.303+0000 7f12a0e95740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555ffd5be1b0 0x555ffd99b540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:56:09.322 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:09.303+0000 7f12a0e95740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x555ffd5c6390 0x555ffd996b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:56:09.322 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:09.303+0000 7f12a0e95740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f12941af900 con 0x555ffd5c6390 2024-09-21T03:56:09.322 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:09.303+0000 7f12a0e95740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f12941b1440 con 0x555ffd5a59f0 2024-09-21T03:56:09.322 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:09.303+0000 7f12a0e95740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f12941affd0 con 0x555ffd5be1b0 2024-09-21T03:56:09.322 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:09.303+0000 7f12937fe640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x555ffd5c6390 0x555ffd996b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:56:09.322 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:09.303+0000 7f12937fe640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x555ffd5c6390 0x555ffd996b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:59662/0 (socket says 172.21.15.39:59662) 2024-09-21T03:56:09.322 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:09.303+0000 7f12937fe640 1 -- 172.21.15.39:0/938518495 learned_addr learned my addr 172.21.15.39:0/938518495 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:56:09.322 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:09.303+0000 7f1293fff640 1 --2- 172.21.15.39:0/938518495 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555ffd5be1b0 0x555ffd99b540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:56:09.322 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:09.303+0000 7f12937fe640 1 --2- 172.21.15.39:0/938518495 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555ffd5a59f0 0x555ffd98fcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:56:09.322 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:09.303+0000 7f12937fe640 1 -- 172.21.15.39:0/938518495 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555ffd5be1b0 msgr2=0x555ffd99b540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:56:09.323 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:09.303+0000 7f12937fe640 1 --2- 172.21.15.39:0/938518495 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555ffd5be1b0 0x555ffd99b540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:09.323 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:09.303+0000 7f12937fe640 1 -- 172.21.15.39:0/938518495 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555ffd5a59f0 msgr2=0x555ffd98fcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:56:09.323 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:09.303+0000 7f12937fe640 1 --2- 172.21.15.39:0/938518495 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555ffd5a59f0 0x555ffd98fcb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:09.323 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:09.303+0000 7f12937fe640 1 -- 172.21.15.39:0/938518495 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555ffd99abd0 con 0x555ffd5c6390 2024-09-21T03:56:09.323 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:09.303+0000 7f12937fe640 1 --2- 172.21.15.39:0/938518495 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x555ffd5c6390 0x555ffd996b00 secure :-1 s=READY pgs=1283 cs=0 l=1 rev1=1 crypto rx=0x7f1288016590 tx=0x7f12880165c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f106bd321ee3038 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:56:09.323 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:09.304+0000 7f1279ffb640 1 -- 172.21.15.39:0/938518495 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1288016630 con 0x555ffd5c6390 2024-09-21T03:56:09.323 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:09.304+0000 7f1279ffb640 1 -- 172.21.15.39:0/938518495 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1288016c80 con 0x555ffd5c6390 2024-09-21T03:56:09.323 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:09.304+0000 7f1279ffb640 1 -- 172.21.15.39:0/938518495 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1288017a20 con 0x555ffd5c6390 2024-09-21T03:56:09.323 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:09.304+0000 7f12a0e95740 1 -- 172.21.15.39:0/938518495 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x555ffd5c6390 msgr2=0x555ffd996b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:56:09.323 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:09.304+0000 7f12a0e95740 1 --2- 172.21.15.39:0/938518495 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x555ffd5c6390 0x555ffd996b00 secure :-1 s=READY pgs=1283 cs=0 l=1 rev1=1 crypto rx=0x7f1288016590 tx=0x7f12880165c0 comp rx=0 tx=0).stop 2024-09-21T03:56:09.323 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:09.304+0000 7f12a0e95740 1 -- 172.21.15.39:0/938518495 shutdown_connections 2024-09-21T03:56:09.323 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:09.305+0000 7f12a0e95740 1 --2- 172.21.15.39:0/938518495 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x555ffd5c6390 0x555ffd996b00 unknown :-1 s=CLOSED pgs=1283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:09.323 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:09.305+0000 7f12a0e95740 1 --2- 172.21.15.39:0/938518495 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555ffd5be1b0 0x555ffd99b540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:09.323 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:09.305+0000 7f12a0e95740 1 --2- 172.21.15.39:0/938518495 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555ffd5a59f0 0x555ffd98fcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:09.324 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:09.305+0000 7f12a0e95740 1 -- 172.21.15.39:0/938518495 >> 172.21.15.39:0/938518495 conn(0x555ffd5db460 msgr2=0x555ffd99ae30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:56:09.324 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:09.305+0000 7f12a0e95740 1 -- 172.21.15.39:0/938518495 shutdown_connections 2024-09-21T03:56:09.324 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:09.305+0000 7f12a0e95740 1 -- 172.21.15.39:0/938518495 wait complete. 2024-09-21T03:56:09.324 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:09.305+0000 7f12a0e95740 1 -- start start 2024-09-21T03:56:09.324 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:09.305+0000 7f12a0e95740 1 Processor -- start 2024-09-21T03:56:09.324 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:09.306+0000 7f12a0e95740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x555ffd5a59f0 0x555ffd9a23e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:56:09.324 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:09.306+0000 7f12a0e95740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555ffd5be1b0 0x555ffd99d8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:56:09.324 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:09.306+0000 7f12a0e95740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555ffd5c6390 0x555ffd99de20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:56:09.324 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:09.306+0000 7f12a0e95740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f12941b0d70 con 0x555ffd5a59f0 2024-09-21T03:56:09.324 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:09.306+0000 7f12a0e95740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f12941b06a0 con 0x555ffd5c6390 2024-09-21T03:56:09.324 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:09.306+0000 7f12a0e95740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f12941ad8e0 con 0x555ffd5be1b0 2024-09-21T03:56:09.324 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:09.306+0000 7f12a0e95740 10 client.0 _ll_register_callbacks cb 0x7f1284009560 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:56:09.324 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:09.306+0000 7f12937fe640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x555ffd5a59f0 0x555ffd9a23e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:56:09.324 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:09.306+0000 7f12937fe640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x555ffd5a59f0 0x555ffd9a23e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:59674/0 (socket says 172.21.15.39:59674) 2024-09-21T03:56:09.324 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:09.306+0000 7f12937fe640 1 -- 172.21.15.39:0/48202577 learned_addr learned my addr 172.21.15.39:0/48202577 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:56:09.325 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:09.306+0000 7f1293fff640 1 --2- 172.21.15.39:0/48202577 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555ffd5be1b0 0x555ffd99d8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:56:09.325 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:09.306+0000 7f12937fe640 1 -- 172.21.15.39:0/48202577 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555ffd5be1b0 msgr2=0x555ffd99d8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:56:09.325 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:09.306+0000 7f12937fe640 1 --2- 172.21.15.39:0/48202577 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555ffd5be1b0 0x555ffd99d8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:09.325 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:09.306+0000 7f12937fe640 1 -- 172.21.15.39:0/48202577 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555ffd5c6390 msgr2=0x555ffd99de20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:56:09.325 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:09.306+0000 7f12937fe640 1 --2- 172.21.15.39:0/48202577 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555ffd5c6390 0x555ffd99de20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:09.325 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:09.306+0000 7f12937fe640 1 -- 172.21.15.39:0/48202577 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555ffd99e510 con 0x555ffd5a59f0 2024-09-21T03:56:09.325 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:09.306+0000 7f1293fff640 1 --2- 172.21.15.39:0/48202577 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555ffd5be1b0 0x555ffd99d8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:56:09.325 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:09.306+0000 7f12937fe640 1 --2- 172.21.15.39:0/48202577 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x555ffd5a59f0 0x555ffd9a23e0 secure :-1 s=READY pgs=1284 cs=0 l=1 rev1=1 crypto rx=0x7f1288006e10 tx=0x7f1288016e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b7ee0ba3fac7135 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:56:09.325 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:09.306+0000 7f12737fe640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-09-21T03:56:09.325 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:09.306+0000 7f12737fe640 1 -- 172.21.15.39:0/48202577 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f128803f060 con 0x555ffd5a59f0 2024-09-21T03:56:09.325 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:09.307+0000 7f12a0e95740 1 -- 172.21.15.39:0/48202577 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1288016c80 con 0x555ffd5a59f0 2024-09-21T03:56:09.325 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:09.307+0000 7f12737fe640 1 -- 172.21.15.39:0/48202577 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8528+0+0 (secure 0 0 0) 0x7f1288017600 con 0x555ffd5a59f0 2024-09-21T03:56:09.325 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:09.307+0000 7f12737fe640 1 -- 172.21.15.39:0/48202577 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f12880059c0 con 0x555ffd5a59f0 2024-09-21T03:56:09.325 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:09.307+0000 7f12a0e95740 10 client.18162 fetch_fsmap learned FSMap version 29 2024-09-21T03:56:09.325 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:09.307+0000 7f12a0e95740 1 -- 172.21.15.39:0/48202577 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x555ffda164c0 con 0x555ffd5a59f0 2024-09-21T03:56:09.326 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:09.307+0000 7f12737fe640 1 -- 172.21.15.39:0/48202577 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1288016c80 con 0x555ffd5a59f0 2024-09-21T03:56:09.326 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:09.307+0000 7f12737fe640 1 -- 172.21.15.39:0/48202577 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f1288017930 con 0x555ffd5a59f0 2024-09-21T03:56:09.326 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:09.307+0000 7f12a0e95740 10 client.18162 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T03:56:09.326 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:09.307+0000 7f12a0e95740 10 client.18162 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.a 2024-09-21T03:56:09.326 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:09.307+0000 7f12a0e95740 20 client.18162 populate_metadata read hostname 'smithi039' 2024-09-21T03:56:09.326 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:09.307+0000 7f12a0e95740 1 --2- 172.21.15.39:0/48202577 >> [v2:172.21.15.39:6840/4247190456,v1:172.21.15.39:6841/4247190456] conn(0x555ffd5e7070 0x555ffda18aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:56:09.326 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:09.307+0000 7f12a0e95740 4 client.18162 mds_command: new command op to 14868 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T03:56:09.326 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:09.307+0000 7f12a0e95740 1 -- 172.21.15.39:0/48202577 --> [v2:172.21.15.39:6840/4247190456,v1:172.21.15.39:6841/4247190456] -- command(tid 0: {"prefix": "damage ls"}) -- 0x555ffd9028a0 con 0x555ffd5e7070 2024-09-21T03:56:09.326 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:09.307+0000 7f1293fff640 1 --2- 172.21.15.39:0/48202577 >> [v2:172.21.15.39:6840/4247190456,v1:172.21.15.39:6841/4247190456] conn(0x555ffd5e7070 0x555ffda18aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:56:09.326 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:09.308+0000 7f12737fe640 10 client.18162 ms_handle_connect on v2:172.21.15.39:6840/4247190456 2024-09-21T03:56:09.326 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:09.308+0000 7f1293fff640 1 --2- 172.21.15.39:0/48202577 >> [v2:172.21.15.39:6840/4247190456,v1:172.21.15.39:6841/4247190456] conn(0x555ffd5e7070 0x555ffda18aa0 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=886c60c5e4dba2b7 server_cookie=8759b464873fe8ee in_seq=0 out_seq=0 2024-09-21T03:56:09.326 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:09.308+0000 7f12737fe640 1 -- 172.21.15.39:0/48202577 <== mds.4 v2:172.21.15.39:6840/4247190456 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x555ffd9028a0 con 0x555ffd5e7070 2024-09-21T03:56:09.326 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:09.308+0000 7f12737fe640 10 client.18162 handle_command_reply: tid=0 2024-09-21T03:56:09.326 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:09.308+0000 7f12a0e95740 1 client.18162 shutdown 2024-09-21T03:56:09.326 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:09.308+0000 7f12a0e95740 1 -- 172.21.15.39:0/48202577 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x555ffd5a59f0 msgr2=0x555ffd9a23e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:56:09.327 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:09.308+0000 7f12a0e95740 1 --2- 172.21.15.39:0/48202577 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x555ffd5a59f0 0x555ffd9a23e0 secure :-1 s=READY pgs=1284 cs=0 l=1 rev1=1 crypto rx=0x7f1288006e10 tx=0x7f1288016e30 comp rx=0 tx=0).stop 2024-09-21T03:56:09.327 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:09.308+0000 7f12a0e95740 1 -- 172.21.15.39:0/48202577 shutdown_connections 2024-09-21T03:56:09.327 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:09.308+0000 7f12a0e95740 1 --2- 172.21.15.39:0/48202577 >> [v2:172.21.15.39:6840/4247190456,v1:172.21.15.39:6841/4247190456] conn(0x555ffd5e7070 0x555ffda18aa0 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:09.327 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:09.308+0000 7f12a0e95740 1 --2- 172.21.15.39:0/48202577 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555ffd5c6390 0x555ffd99de20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:09.327 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:09.308+0000 7f12a0e95740 1 --2- 172.21.15.39:0/48202577 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555ffd5be1b0 0x555ffd99d8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:09.327 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:09.308+0000 7f12737fe640 0 client.18162 ms_handle_reset on v2:172.21.15.39:6840/4247190456 2024-09-21T03:56:09.327 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:09.308+0000 7f12a0e95740 1 --2- 172.21.15.39:0/48202577 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x555ffd5a59f0 0x555ffd9a23e0 unknown :-1 s=CLOSED pgs=1284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:09.327 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:09.308+0000 7f12a0e95740 1 -- 172.21.15.39:0/48202577 >> 172.21.15.39:0/48202577 conn(0x555ffd5db460 msgr2=0x555ffd98fa80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:56:09.327 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:09.309+0000 7f12a0e95740 1 -- 172.21.15.39:0/48202577 shutdown_connections 2024-09-21T03:56:09.327 INFO:teuthology.orchestra.run.smithi039.stdout:[] 2024-09-21T03:56:09.328 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:09.309+0000 7f12a0e95740 1 -- 172.21.15.39:0/48202577 wait complete. 2024-09-21T03:56:09.328 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:09.309+0000 7f12a0e95740 20 client.18162 trim_cache size 0 max 16384 2024-09-21T03:56:09.328 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:09.309+0000 7f129a921640 1 -- 172.21.15.39:0/602340387 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f127c0b6680 msgr2=0x7f127c0b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:56:09.328 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:09.309+0000 7f129a921640 1 --2- 172.21.15.39:0/602340387 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f127c0b6680 0x7f127c0b8b90 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f128c0ad530 tx=0x7f128c07a880 comp rx=0 tx=0).stop 2024-09-21T03:56:09.328 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:09.309+0000 7f129a921640 1 -- 172.21.15.39:0/602340387 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1294109ea0 msgr2=0x7f129419fdd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:56:09.328 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:09.309+0000 7f129a921640 1 --2- 172.21.15.39:0/602340387 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1294109ea0 0x7f129419fdd0 secure :-1 s=READY pgs=1280 cs=0 l=1 rev1=1 crypto rx=0x7f128c00cad0 tx=0x7f128c00cf90 comp rx=0 tx=0).stop 2024-09-21T03:56:09.328 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:09.309+0000 7f129a921640 1 -- 172.21.15.39:0/602340387 shutdown_connections 2024-09-21T03:56:09.328 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:09.309+0000 7f129a921640 1 --2- 172.21.15.39:0/602340387 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f127c0b6680 0x7f127c0b8b90 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:09.329 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:09.309+0000 7f129a921640 1 --2- 172.21.15.39:0/602340387 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1294109ea0 0x7f129419fdd0 unknown :-1 s=CLOSED pgs=1280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:09.329 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:09.309+0000 7f129a921640 1 --2- 172.21.15.39:0/602340387 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1294109550 0x7f129419ab30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:09.329 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:09.309+0000 7f129a921640 1 --2- 172.21.15.39:0/602340387 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1294102100 0x7f129419a590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:09.329 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:09.309+0000 7f129a921640 1 -- 172.21.15.39:0/602340387 >> 172.21.15.39:0/602340387 conn(0x7f1294079ea0 msgr2=0x7f1294107640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:56:09.329 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:09.310+0000 7f129a921640 1 -- 172.21.15.39:0/602340387 shutdown_connections 2024-09-21T03:56:09.329 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:09.310+0000 7f129a921640 1 -- 172.21.15.39:0/602340387 wait complete. 2024-09-21T03:56:09.468 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:56:09 smithi161 ceph-mon[34533]: pgmap v634: 97 pgs: 97 active+clean; 10 MiB data, 525 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 44 KiB/s wr, 25 op/s 2024-09-21T03:56:09.474 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:56:09 smithi150 ceph-mon[34289]: pgmap v634: 97 pgs: 97 active+clean; 10 MiB data, 525 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 44 KiB/s wr, 25 op/s 2024-09-21T03:56:10.242 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 51116d5c-77ca-11ef-baf5-efdc52797490 -- ceph tell mds.1:3 damage ls 2024-09-21T03:56:10.504 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/51116d5c-77ca-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:56:10.997 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:10.988+0000 7fabb7646640 1 -- 172.21.15.39:0/918015136 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fabb0108cf0 msgr2=0x7fabb010f520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:56:10.997 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:10.988+0000 7fabb7646640 1 --2- 172.21.15.39:0/918015136 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fabb0108cf0 0x7fabb010f520 secure :-1 s=READY pgs=1285 cs=0 l=1 rev1=1 crypto rx=0x7faba8009ca0 tx=0x7faba802a8c0 comp rx=0 tx=0).stop 2024-09-21T03:56:10.997 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:10.988+0000 7fabb7646640 1 -- 172.21.15.39:0/918015136 shutdown_connections 2024-09-21T03:56:10.997 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:10.988+0000 7fabb7646640 1 --2- 172.21.15.39:0/918015136 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fabb0108cf0 0x7fabb010f520 unknown :-1 s=CLOSED pgs=1285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:10.998 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:10.988+0000 7fabb7646640 1 --2- 172.21.15.39:0/918015136 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fabb01083a0 0x7fabb01087b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:10.998 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:10.988+0000 7fabb7646640 1 --2- 172.21.15.39:0/918015136 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fabb0102a60 0x7fabb0102e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:10.998 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:10.988+0000 7fabb7646640 1 -- 172.21.15.39:0/918015136 >> 172.21.15.39:0/918015136 conn(0x7fabb0079730 msgr2=0x7fabb0079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:56:10.998 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:10.988+0000 7fabb7646640 1 -- 172.21.15.39:0/918015136 shutdown_connections 2024-09-21T03:56:10.998 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:10.988+0000 7fabb7646640 1 -- 172.21.15.39:0/918015136 wait complete. 2024-09-21T03:56:10.998 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:10.989+0000 7fabb7646640 1 Processor -- start 2024-09-21T03:56:10.998 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:10.989+0000 7fabb7646640 1 -- start start 2024-09-21T03:56:10.998 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:10.989+0000 7fabb7646640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fabb0102a60 0x7fabb010ffb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:56:10.998 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:10.989+0000 7fabb7646640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fabb01083a0 0x7fabb01104f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:56:10.998 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:10.989+0000 7fabb7646640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fabb0108cf0 0x7fabb0110a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:56:10.998 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:10.989+0000 7fabb7646640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fabb0113a80 con 0x7fabb01083a0 2024-09-21T03:56:10.998 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:10.989+0000 7fabb7646640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fabb0113900 con 0x7fabb0102a60 2024-09-21T03:56:10.998 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:10.989+0000 7fabb7646640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fabb0113c00 con 0x7fabb0108cf0 2024-09-21T03:56:10.998 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:10.989+0000 7fabb4be1640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fabb01083a0 0x7fabb01104f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:56:10.999 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:10.989+0000 7fabb4be1640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fabb01083a0 0x7fabb01104f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:59696/0 (socket says 172.21.15.39:59696) 2024-09-21T03:56:10.999 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:10.989+0000 7fabb4be1640 1 -- 172.21.15.39:0/2375088157 learned_addr learned my addr 172.21.15.39:0/2375088157 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:56:10.999 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:10.990+0000 7fabb53e2640 1 --2- 172.21.15.39:0/2375088157 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fabb0102a60 0x7fabb010ffb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:56:10.999 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:10.990+0000 7fabb5be3640 1 --2- 172.21.15.39:0/2375088157 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fabb0108cf0 0x7fabb0110a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:56:10.999 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:10.990+0000 7fabb4be1640 1 -- 172.21.15.39:0/2375088157 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fabb0108cf0 msgr2=0x7fabb0110a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:56:10.999 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:10.990+0000 7fabb4be1640 1 --2- 172.21.15.39:0/2375088157 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fabb0108cf0 0x7fabb0110a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:10.999 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:10.990+0000 7fabb4be1640 1 -- 172.21.15.39:0/2375088157 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fabb0102a60 msgr2=0x7fabb010ffb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:56:10.999 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:10.990+0000 7fabb4be1640 1 --2- 172.21.15.39:0/2375088157 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fabb0102a60 0x7fabb010ffb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:10.999 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:10.990+0000 7fabb4be1640 1 -- 172.21.15.39:0/2375088157 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fabb01a86d0 con 0x7fabb01083a0 2024-09-21T03:56:10.999 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:10.990+0000 7fabb4be1640 1 --2- 172.21.15.39:0/2375088157 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fabb01083a0 0x7fabb01104f0 secure :-1 s=READY pgs=1286 cs=0 l=1 rev1=1 crypto rx=0x7faba000d740 tx=0x7faba000dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:56:10.999 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:10.990+0000 7faba67fc640 1 -- 172.21.15.39:0/2375088157 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faba0017070 con 0x7fabb01083a0 2024-09-21T03:56:10.999 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:10.990+0000 7fabb7646640 1 -- 172.21.15.39:0/2375088157 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fabb01a89c0 con 0x7fabb01083a0 2024-09-21T03:56:10.999 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:10.990+0000 7faba67fc640 1 -- 172.21.15.39:0/2375088157 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faba000de50 con 0x7fabb01083a0 2024-09-21T03:56:10.999 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:10.990+0000 7faba67fc640 1 -- 172.21.15.39:0/2375088157 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faba0004ce0 con 0x7fabb01083a0 2024-09-21T03:56:10.999 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:10.990+0000 7fabb7646640 1 -- 172.21.15.39:0/2375088157 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fabb01a8eb0 con 0x7fabb01083a0 2024-09-21T03:56:11.000 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:10.993+0000 7fabbdbba740 1 Processor -- start 2024-09-21T03:56:11.000 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:10.993+0000 7fabbdbba740 1 -- start start 2024-09-21T03:56:11.000 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:10.993+0000 7faba67fc640 1 -- 172.21.15.39:0/2375088157 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 27) ==== 154643+0+0 (secure 0 0 0) 0x7faba0004e80 con 0x7fabb01083a0 2024-09-21T03:56:11.000 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:10.994+0000 7fabbdbba740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557e4e1c5070 0x557e4e56c2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:56:11.000 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:10.994+0000 7fabbdbba740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557e4e19c1b0 0x557e4e56c800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:56:11.000 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:10.994+0000 7fabbdbba740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557e4e1a4390 0x557e4e56cd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:56:11.000 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:10.994+0000 7fabbdbba740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x557e4e4e2ea0 con 0x557e4e1c5070 2024-09-21T03:56:11.000 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:10.994+0000 7fabbdbba740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x557e4e4d22d0 con 0x557e4e19c1b0 2024-09-21T03:56:11.000 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:10.994+0000 7fabbdbba740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x557e4e4d60e0 con 0x557e4e1a4390 2024-09-21T03:56:11.000 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:10.994+0000 7fabb5be3640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557e4e1c5070 0x557e4e56c2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:56:11.000 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:10.994+0000 7fabb5be3640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557e4e1c5070 0x557e4e56c2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:59704/0 (socket says 172.21.15.39:59704) 2024-09-21T03:56:11.000 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:10.994+0000 7fabb5be3640 1 -- 172.21.15.39:0/2798185750 learned_addr learned my addr 172.21.15.39:0/2798185750 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:56:11.000 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:10.994+0000 7fabb53e2640 1 --2- 172.21.15.39:0/2798185750 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557e4e19c1b0 0x557e4e56c800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:56:11.000 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:10.994+0000 7fabb4be1640 1 --2- 172.21.15.39:0/2798185750 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557e4e1a4390 0x557e4e56cd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:56:11.000 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:10.994+0000 7fabb5be3640 1 -- 172.21.15.39:0/2798185750 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557e4e1a4390 msgr2=0x557e4e56cd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:56:11.001 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:10.994+0000 7fabb5be3640 1 --2- 172.21.15.39:0/2798185750 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557e4e1a4390 0x557e4e56cd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:11.001 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:10.994+0000 7fabb5be3640 1 -- 172.21.15.39:0/2798185750 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557e4e19c1b0 msgr2=0x557e4e56c800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:56:11.001 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:10.994+0000 7fabb5be3640 1 --2- 172.21.15.39:0/2798185750 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557e4e19c1b0 0x557e4e56c800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:11.001 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:10.994+0000 7fabb5be3640 1 -- 172.21.15.39:0/2798185750 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557e4e230650 con 0x557e4e1c5070 2024-09-21T03:56:11.001 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:10.994+0000 7fabb53e2640 1 --2- 172.21.15.39:0/2798185750 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557e4e19c1b0 0x557e4e56c800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:56:11.001 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:10.994+0000 7faba67fc640 1 --2- 172.21.15.39:0/2375088157 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7fab9c0b6680 0x7fab9c0b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:56:11.004 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:10.995+0000 7fabb5be3640 1 --2- 172.21.15.39:0/2798185750 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557e4e1c5070 0x557e4e56c2c0 secure :-1 s=READY pgs=1287 cs=0 l=1 rev1=1 crypto rx=0x7faba8005cc0 tx=0x7faba8009800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7b6e303810885f0a server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:56:11.005 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:10.995+0000 7fabb53e2640 1 --2- 172.21.15.39:0/2375088157 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7fab9c0b6680 0x7fab9c0b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:56:11.005 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:10.995+0000 7faba67fc640 1 -- 172.21.15.39:0/2375088157 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8572+0+0 (secure 0 0 0) 0x7faba00e7150 con 0x7fabb01083a0 2024-09-21T03:56:11.005 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:10.995+0000 7fab9b7fe640 1 -- 172.21.15.39:0/2798185750 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faba8031cf0 con 0x557e4e1c5070 2024-09-21T03:56:11.005 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:10.995+0000 7fab9b7fe640 1 -- 172.21.15.39:0/2798185750 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faba8036040 con 0x557e4e1c5070 2024-09-21T03:56:11.005 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:10.995+0000 7fab9b7fe640 1 -- 172.21.15.39:0/2798185750 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faba80042a0 con 0x557e4e1c5070 2024-09-21T03:56:11.005 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:10.996+0000 7fabbdbba740 1 -- 172.21.15.39:0/2798185750 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557e4e1c5070 msgr2=0x557e4e56c2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:56:11.005 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:10.996+0000 7fabbdbba740 1 --2- 172.21.15.39:0/2798185750 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557e4e1c5070 0x557e4e56c2c0 secure :-1 s=READY pgs=1287 cs=0 l=1 rev1=1 crypto rx=0x7faba8005cc0 tx=0x7faba8009800 comp rx=0 tx=0).stop 2024-09-21T03:56:11.005 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:10.996+0000 7fabb53e2640 1 --2- 172.21.15.39:0/2375088157 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7fab9c0b6680 0x7fab9c0b8b90 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x557e4e56af40 tx=0x7fabac00b390 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:56:11.005 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:10.996+0000 7fabbdbba740 1 -- 172.21.15.39:0/2798185750 shutdown_connections 2024-09-21T03:56:11.005 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:10.996+0000 7fabbdbba740 1 --2- 172.21.15.39:0/2798185750 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557e4e1a4390 0x557e4e56cd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:11.005 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:10.996+0000 7fabbdbba740 1 --2- 172.21.15.39:0/2798185750 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557e4e19c1b0 0x557e4e56c800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:11.005 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:10.996+0000 7fabbdbba740 1 --2- 172.21.15.39:0/2798185750 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557e4e1c5070 0x557e4e56c2c0 unknown :-1 s=CLOSED pgs=1287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:11.005 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:10.996+0000 7fabbdbba740 1 -- 172.21.15.39:0/2798185750 >> 172.21.15.39:0/2798185750 conn(0x557e4e1b9460 msgr2=0x557e4e56a1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:56:11.006 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:10.996+0000 7fabbdbba740 1 -- 172.21.15.39:0/2798185750 shutdown_connections 2024-09-21T03:56:11.006 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:10.996+0000 7fabbdbba740 1 -- 172.21.15.39:0/2798185750 wait complete. 2024-09-21T03:56:11.006 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:10.996+0000 7fabbdbba740 1 -- start start 2024-09-21T03:56:11.006 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:10.996+0000 7fabbdbba740 1 Processor -- start 2024-09-21T03:56:11.006 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:10.997+0000 7fabbdbba740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557e4e19c1b0 0x557e4e5779c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:56:11.006 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:10.997+0000 7fabbdbba740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557e4e1a4390 0x557e4e577f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:56:11.006 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:10.997+0000 7fabbdbba740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557e4e1c5070 0x557e4e583d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:56:11.006 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:10.997+0000 7fabbdbba740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x557e4e333700 con 0x557e4e1c5070 2024-09-21T03:56:11.006 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:10.997+0000 7fabbdbba740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x557e4e46d160 con 0x557e4e1a4390 2024-09-21T03:56:11.006 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:10.997+0000 7fabbdbba740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x557e4e57aa40 con 0x557e4e19c1b0 2024-09-21T03:56:11.006 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:10.997+0000 7fabbdbba740 10 client.0 _ll_register_callbacks cb 0x557e4e4869e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:56:11.006 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:10.997+0000 7fabb5be3640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557e4e1c5070 0x557e4e583d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:56:11.006 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:10.997+0000 7fabb5be3640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557e4e1c5070 0x557e4e583d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:59720/0 (socket says 172.21.15.39:59720) 2024-09-21T03:56:11.006 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:10.997+0000 7fabb5be3640 1 -- 172.21.15.39:0/1362036095 learned_addr learned my addr 172.21.15.39:0/1362036095 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:56:11.007 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:10.997+0000 7fabb53e2640 1 --2- 172.21.15.39:0/1362036095 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557e4e19c1b0 0x557e4e5779c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:56:11.007 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:10.997+0000 7fabb4be1640 1 --2- 172.21.15.39:0/1362036095 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557e4e1a4390 0x557e4e577f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:56:11.007 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:10.998+0000 7fabb5be3640 1 -- 172.21.15.39:0/1362036095 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557e4e19c1b0 msgr2=0x557e4e5779c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:56:11.007 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:10.998+0000 7fabb5be3640 1 --2- 172.21.15.39:0/1362036095 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557e4e19c1b0 0x557e4e5779c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:11.007 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:10.998+0000 7fabb5be3640 1 -- 172.21.15.39:0/1362036095 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557e4e1a4390 msgr2=0x557e4e577f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:56:11.007 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:10.998+0000 7fabb5be3640 1 --2- 172.21.15.39:0/1362036095 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557e4e1a4390 0x557e4e577f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:11.007 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:10.998+0000 7fabb5be3640 1 -- 172.21.15.39:0/1362036095 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557e4e49ac90 con 0x557e4e1c5070 2024-09-21T03:56:11.007 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:10.998+0000 7fabb5be3640 1 --2- 172.21.15.39:0/1362036095 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557e4e1c5070 0x557e4e583d60 secure :-1 s=READY pgs=1288 cs=0 l=1 rev1=1 crypto rx=0x7faba8009a60 tx=0x7faba8003b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b6254db23d8d9520 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:56:11.007 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:10.998+0000 7fab98ff9640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-09-21T03:56:11.007 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:10.998+0000 7fab98ff9640 1 -- 172.21.15.39:0/1362036095 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faba804a040 con 0x557e4e1c5070 2024-09-21T03:56:11.007 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:10.998+0000 7fab98ff9640 1 -- 172.21.15.39:0/1362036095 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8572+0+0 (secure 0 0 0) 0x7faba8031db0 con 0x557e4e1c5070 2024-09-21T03:56:11.007 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:10.998+0000 7fabbdbba740 1 -- 172.21.15.39:0/1362036095 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7faba8036040 con 0x557e4e1c5070 2024-09-21T03:56:11.007 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:10.998+0000 7fab98ff9640 1 -- 172.21.15.39:0/1362036095 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7faba805c050 con 0x557e4e1c5070 2024-09-21T03:56:11.007 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:10.998+0000 7fabbdbba740 10 client.18174 fetch_fsmap learned FSMap version 29 2024-09-21T03:56:11.008 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:10.998+0000 7fabbdbba740 1 -- 172.21.15.39:0/1362036095 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x557e4e586760 con 0x557e4e1c5070 2024-09-21T03:56:11.008 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:10.998+0000 7fab98ff9640 1 -- 172.21.15.39:0/1362036095 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faba8036040 con 0x557e4e1c5070 2024-09-21T03:56:11.008 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:10.999+0000 7fab98ff9640 1 -- 172.21.15.39:0/1362036095 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7faba804d070 con 0x557e4e1c5070 2024-09-21T03:56:11.008 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:10.999+0000 7fabbdbba740 10 client.18174 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T03:56:11.008 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:10.999+0000 7fabbdbba740 10 client.18174 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-21T03:56:11.008 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:10.999+0000 7fabbdbba740 20 client.18174 populate_metadata read hostname 'smithi039' 2024-09-21T03:56:11.008 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:10.999+0000 7fabbdbba740 1 --2- 172.21.15.39:0/1362036095 >> [v2:172.21.15.161:6836/3451544597,v1:172.21.15.161:6837/3451544597] conn(0x557e4e1839f0 0x557e4e589ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:56:11.008 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:10.999+0000 7fabbdbba740 4 client.18174 mds_command: new command op to 24646 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:56:11.008 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:10.999+0000 7fabbdbba740 1 -- 172.21.15.39:0/1362036095 --> [v2:172.21.15.161:6836/3451544597,v1:172.21.15.161:6837/3451544597] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557e4e4e4df0 con 0x557e4e1839f0 2024-09-21T03:56:11.008 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:10.999+0000 7fabb53e2640 1 --2- 172.21.15.39:0/1362036095 >> [v2:172.21.15.161:6836/3451544597,v1:172.21.15.161:6837/3451544597] conn(0x557e4e1839f0 0x557e4e589ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:56:11.008 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:11.000+0000 7fabb53e2640 1 --2- 172.21.15.39:0/1362036095 >> [v2:172.21.15.161:6836/3451544597,v1:172.21.15.161:6837/3451544597] conn(0x557e4e1839f0 0x557e4e589ac0 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=79e838e4123ee332 server_cookie=789b209abe8881c3 in_seq=0 out_seq=0 2024-09-21T03:56:11.008 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:11.000+0000 7fab98ff9640 10 client.18174 ms_handle_connect on v2:172.21.15.161:6836/3451544597 2024-09-21T03:56:11.008 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:11.000+0000 7fab98ff9640 1 -- 172.21.15.39:0/1362036095 <== mds.3 v2:172.21.15.161:6836/3451544597 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x557e4e4e4df0 con 0x557e4e1839f0 2024-09-21T03:56:11.008 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:11.000+0000 7fab98ff9640 10 client.18174 handle_command_reply: tid=0 2024-09-21T03:56:11.008 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:11.000+0000 7fabbdbba740 1 client.18174 shutdown 2024-09-21T03:56:11.009 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:11.001+0000 7fabbdbba740 1 -- 172.21.15.39:0/1362036095 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557e4e1c5070 msgr2=0x557e4e583d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:56:11.009 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:11.001+0000 7fabbdbba740 1 --2- 172.21.15.39:0/1362036095 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557e4e1c5070 0x557e4e583d60 secure :-1 s=READY pgs=1288 cs=0 l=1 rev1=1 crypto rx=0x7faba8009a60 tx=0x7faba8003b00 comp rx=0 tx=0).stop 2024-09-21T03:56:11.009 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:11.001+0000 7fabbdbba740 1 -- 172.21.15.39:0/1362036095 shutdown_connections 2024-09-21T03:56:11.009 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:11.001+0000 7fabbdbba740 1 --2- 172.21.15.39:0/1362036095 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557e4e1c5070 0x557e4e583d60 unknown :-1 s=CLOSED pgs=1288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:11.009 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:11.001+0000 7fabbdbba740 1 --2- 172.21.15.39:0/1362036095 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557e4e1a4390 0x557e4e577f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:11.009 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:11.001+0000 7fabbdbba740 1 --2- 172.21.15.39:0/1362036095 >> [v2:172.21.15.161:6836/3451544597,v1:172.21.15.161:6837/3451544597] conn(0x557e4e1839f0 0x557e4e589ac0 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:11.009 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:11.001+0000 7fabbdbba740 1 --2- 172.21.15.39:0/1362036095 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557e4e19c1b0 0x557e4e5779c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:11.009 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:11.001+0000 7fab98ff9640 0 client.18174 ms_handle_reset on v2:172.21.15.161:6836/3451544597 2024-09-21T03:56:11.009 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:11.001+0000 7fabbdbba740 1 -- 172.21.15.39:0/1362036095 >> 172.21.15.39:0/1362036095 conn(0x557e4e1b9460 msgr2=0x557e4e56f4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:56:11.009 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:11.001+0000 7fabbdbba740 1 -- 172.21.15.39:0/1362036095 shutdown_connections 2024-09-21T03:56:11.009 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:11.001+0000 7fabbdbba740 1 -- 172.21.15.39:0/1362036095 wait complete. 2024-09-21T03:56:11.009 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:11.001+0000 7fabbdbba740 20 client.18174 trim_cache size 0 max 16384 2024-09-21T03:56:11.018 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:11.014+0000 7fabbdbba740 1 Processor -- start 2024-09-21T03:56:11.018 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:11.014+0000 7fabbdbba740 1 -- start start 2024-09-21T03:56:11.018 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:11.015+0000 7fabbdbba740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557e4e19c1b0 0x557e4e583cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:56:11.018 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:11.015+0000 7fabbdbba740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557e4e1a4390 0x557e4e5795f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:56:11.018 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:11.015+0000 7fabbdbba740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557e4e1c5070 0x557e4e574bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:56:11.018 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:11.015+0000 7fabbdbba740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fabb01b19f0 con 0x557e4e19c1b0 2024-09-21T03:56:11.018 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:11.015+0000 7fabbdbba740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fabb01b3530 con 0x557e4e1c5070 2024-09-21T03:56:11.019 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:11.015+0000 7fabbdbba740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fabb01b20c0 con 0x557e4e1a4390 2024-09-21T03:56:11.019 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:11.015+0000 7fabb53e2640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557e4e19c1b0 0x557e4e583cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:56:11.019 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:11.015+0000 7fabb53e2640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557e4e19c1b0 0x557e4e583cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:59726/0 (socket says 172.21.15.39:59726) 2024-09-21T03:56:11.019 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:11.015+0000 7fabb53e2640 1 -- 172.21.15.39:0/3919727547 learned_addr learned my addr 172.21.15.39:0/3919727547 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:56:11.019 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:11.015+0000 7fabb5be3640 1 --2- 172.21.15.39:0/3919727547 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557e4e1c5070 0x557e4e574bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:56:11.019 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:11.015+0000 7fabb4be1640 1 --2- 172.21.15.39:0/3919727547 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557e4e1a4390 0x557e4e5795f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:56:11.019 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:11.015+0000 7fabb53e2640 1 -- 172.21.15.39:0/3919727547 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557e4e1a4390 msgr2=0x557e4e5795f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:56:11.019 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:11.015+0000 7fabb53e2640 1 --2- 172.21.15.39:0/3919727547 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557e4e1a4390 0x557e4e5795f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:11.019 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:11.015+0000 7fabb53e2640 1 -- 172.21.15.39:0/3919727547 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557e4e1c5070 msgr2=0x557e4e574bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:56:11.019 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:11.015+0000 7fabb53e2640 1 --2- 172.21.15.39:0/3919727547 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557e4e1c5070 0x557e4e574bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:11.019 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:11.015+0000 7fabb53e2640 1 -- 172.21.15.39:0/3919727547 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557e4e56e5b0 con 0x557e4e19c1b0 2024-09-21T03:56:11.019 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:11.015+0000 7fabb53e2640 1 --2- 172.21.15.39:0/3919727547 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557e4e19c1b0 0x557e4e583cd0 secure :-1 s=READY pgs=1289 cs=0 l=1 rev1=1 crypto rx=0x557e4e4e2ea0 tx=0x7fabac015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=23f910c65994d36e server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:56:11.019 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:11.015+0000 7fab9affd640 1 -- 172.21.15.39:0/3919727547 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabac010640 con 0x557e4e19c1b0 2024-09-21T03:56:11.019 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:11.016+0000 7fab9affd640 1 -- 172.21.15.39:0/3919727547 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fabac010c90 con 0x557e4e19c1b0 2024-09-21T03:56:11.019 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:11.016+0000 7fab9affd640 1 -- 172.21.15.39:0/3919727547 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabac00fb90 con 0x557e4e19c1b0 2024-09-21T03:56:11.022 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:11.016+0000 7fabbdbba740 1 -- 172.21.15.39:0/3919727547 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557e4e19c1b0 msgr2=0x557e4e583cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:56:11.022 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:11.016+0000 7fabbdbba740 1 --2- 172.21.15.39:0/3919727547 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557e4e19c1b0 0x557e4e583cd0 secure :-1 s=READY pgs=1289 cs=0 l=1 rev1=1 crypto rx=0x557e4e4e2ea0 tx=0x7fabac015000 comp rx=0 tx=0).stop 2024-09-21T03:56:11.022 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:11.016+0000 7fabbdbba740 1 -- 172.21.15.39:0/3919727547 shutdown_connections 2024-09-21T03:56:11.022 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:11.016+0000 7fabbdbba740 1 --2- 172.21.15.39:0/3919727547 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557e4e1c5070 0x557e4e574bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:11.022 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:11.016+0000 7fabbdbba740 1 --2- 172.21.15.39:0/3919727547 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557e4e1a4390 0x557e4e5795f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:11.022 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:11.016+0000 7fabbdbba740 1 --2- 172.21.15.39:0/3919727547 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557e4e19c1b0 0x557e4e583cd0 unknown :-1 s=CLOSED pgs=1289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:11.022 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:11.016+0000 7fabbdbba740 1 -- 172.21.15.39:0/3919727547 >> 172.21.15.39:0/3919727547 conn(0x557e4e1b9460 msgr2=0x557e4e578ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:56:11.022 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:11.016+0000 7fabbdbba740 1 -- 172.21.15.39:0/3919727547 shutdown_connections 2024-09-21T03:56:11.023 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:11.016+0000 7fabbdbba740 1 -- 172.21.15.39:0/3919727547 wait complete. 2024-09-21T03:56:11.023 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:11.017+0000 7fabbdbba740 1 -- start start 2024-09-21T03:56:11.023 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:11.017+0000 7fabbdbba740 1 Processor -- start 2024-09-21T03:56:11.023 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:11.017+0000 7fabbdbba740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557e4e19c1b0 0x557e4e58aa50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:56:11.023 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:11.017+0000 7fabbdbba740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557e4e1a4390 0x557e4e58af90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:56:11.023 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:11.017+0000 7fabbdbba740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557e4e1c5070 0x557e4e58b4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:56:11.023 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:11.017+0000 7fabbdbba740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fabb01b2e60 con 0x557e4e19c1b0 2024-09-21T03:56:11.023 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:11.017+0000 7fabbdbba740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fabb01b2790 con 0x557e4e1c5070 2024-09-21T03:56:11.023 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:11.017+0000 7fabbdbba740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fabb01af9f0 con 0x557e4e1a4390 2024-09-21T03:56:11.023 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:11.017+0000 7fabbdbba740 10 client.0 _ll_register_callbacks cb 0x7faba802f3a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:56:11.023 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:11.017+0000 7fabb53e2640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557e4e19c1b0 0x557e4e58aa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:56:11.023 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:11.017+0000 7fabb53e2640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557e4e19c1b0 0x557e4e58aa50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:59738/0 (socket says 172.21.15.39:59738) 2024-09-21T03:56:11.023 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:11.017+0000 7fabb53e2640 1 -- 172.21.15.39:0/3205118783 learned_addr learned my addr 172.21.15.39:0/3205118783 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:56:11.023 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:11.017+0000 7fabb5be3640 1 --2- 172.21.15.39:0/3205118783 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557e4e1c5070 0x557e4e58b4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:56:11.024 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:11.018+0000 7fabb4be1640 1 --2- 172.21.15.39:0/3205118783 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557e4e1a4390 0x557e4e58af90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:56:11.024 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:11.018+0000 7fabb53e2640 1 -- 172.21.15.39:0/3205118783 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557e4e1a4390 msgr2=0x557e4e58af90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:56:11.024 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:11.018+0000 7fabb53e2640 1 --2- 172.21.15.39:0/3205118783 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557e4e1a4390 0x557e4e58af90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:11.024 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:11.018+0000 7fabb53e2640 1 -- 172.21.15.39:0/3205118783 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557e4e1c5070 msgr2=0x557e4e58b4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:56:11.024 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:11.018+0000 7fabb53e2640 1 --2- 172.21.15.39:0/3205118783 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557e4e1c5070 0x557e4e58b4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:11.024 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:11.018+0000 7fabb53e2640 1 -- 172.21.15.39:0/3205118783 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557e4e5f3240 con 0x557e4e19c1b0 2024-09-21T03:56:11.024 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:11.018+0000 7fabb53e2640 1 --2- 172.21.15.39:0/3205118783 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557e4e19c1b0 0x557e4e58aa50 secure :-1 s=READY pgs=1290 cs=0 l=1 rev1=1 crypto rx=0x7fabac012fd0 tx=0x7fabac010ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=27245a1d6a9fd07a server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:56:11.024 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:11.018+0000 7fab98ff9640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-09-21T03:56:11.024 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:11.018+0000 7fab98ff9640 1 -- 172.21.15.39:0/3205118783 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabac03d060 con 0x557e4e19c1b0 2024-09-21T03:56:11.024 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:11.018+0000 7fab98ff9640 1 -- 172.21.15.39:0/3205118783 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8572+0+0 (secure 0 0 0) 0x7fabac00f910 con 0x557e4e19c1b0 2024-09-21T03:56:11.024 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:11.018+0000 7fabbdbba740 1 -- 172.21.15.39:0/3205118783 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fabac010c90 con 0x557e4e19c1b0 2024-09-21T03:56:11.024 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:11.018+0000 7fab98ff9640 1 -- 172.21.15.39:0/3205118783 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fabac00fb10 con 0x557e4e19c1b0 2024-09-21T03:56:11.024 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:11.018+0000 7fabbdbba740 10 client.18180 fetch_fsmap learned FSMap version 29 2024-09-21T03:56:11.024 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:11.018+0000 7fabbdbba740 1 -- 172.21.15.39:0/3205118783 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x557e4e5f4420 con 0x557e4e19c1b0 2024-09-21T03:56:11.024 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:11.019+0000 7fab98ff9640 1 -- 172.21.15.39:0/3205118783 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fabac010c90 con 0x557e4e19c1b0 2024-09-21T03:56:11.025 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:11.019+0000 7fab98ff9640 1 -- 172.21.15.39:0/3205118783 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fabac0415d0 con 0x557e4e19c1b0 2024-09-21T03:56:11.025 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:11.019+0000 7fabbdbba740 10 client.18180 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T03:56:11.025 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:11.019+0000 7fabbdbba740 10 client.18180 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-21T03:56:11.025 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:11.019+0000 7fabbdbba740 20 client.18180 populate_metadata read hostname 'smithi039' 2024-09-21T03:56:11.025 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:11.019+0000 7fabbdbba740 1 --2- 172.21.15.39:0/3205118783 >> [v2:172.21.15.161:6836/3451544597,v1:172.21.15.161:6837/3451544597] conn(0x557e4e158da0 0x557e4e5f6c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:56:11.025 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:11.019+0000 7fabbdbba740 4 client.18180 mds_command: new command op to 24646 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T03:56:11.025 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:11.019+0000 7fabbdbba740 1 -- 172.21.15.39:0/3205118783 --> [v2:172.21.15.161:6836/3451544597,v1:172.21.15.161:6837/3451544597] -- command(tid 0: {"prefix": "damage ls"}) -- 0x557e4e4e08a0 con 0x557e4e158da0 2024-09-21T03:56:11.025 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:11.019+0000 7fabb4be1640 1 --2- 172.21.15.39:0/3205118783 >> [v2:172.21.15.161:6836/3451544597,v1:172.21.15.161:6837/3451544597] conn(0x557e4e158da0 0x557e4e5f6c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:56:11.026 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:11.020+0000 7fab98ff9640 10 client.18180 ms_handle_connect on v2:172.21.15.161:6836/3451544597 2024-09-21T03:56:11.026 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:11.020+0000 7fabb4be1640 1 --2- 172.21.15.39:0/3205118783 >> [v2:172.21.15.161:6836/3451544597,v1:172.21.15.161:6837/3451544597] conn(0x557e4e158da0 0x557e4e5f6c10 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=9ac6d0a7fa40b2de server_cookie=b120609844ccb876 in_seq=0 out_seq=0 2024-09-21T03:56:11.026 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:11.020+0000 7fab98ff9640 1 -- 172.21.15.39:0/3205118783 <== mds.3 v2:172.21.15.161:6836/3451544597 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x557e4e4e08a0 con 0x557e4e158da0 2024-09-21T03:56:11.026 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:11.020+0000 7fab98ff9640 10 client.18180 handle_command_reply: tid=0 2024-09-21T03:56:11.026 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:11.020+0000 7fabbdbba740 1 client.18180 shutdown 2024-09-21T03:56:11.027 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:11.020+0000 7fabbdbba740 1 -- 172.21.15.39:0/3205118783 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557e4e19c1b0 msgr2=0x557e4e58aa50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:56:11.027 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:11.020+0000 7fabbdbba740 1 --2- 172.21.15.39:0/3205118783 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557e4e19c1b0 0x557e4e58aa50 secure :-1 s=READY pgs=1290 cs=0 l=1 rev1=1 crypto rx=0x7fabac012fd0 tx=0x7fabac010ec0 comp rx=0 tx=0).stop 2024-09-21T03:56:11.027 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:11.020+0000 7fabbdbba740 1 -- 172.21.15.39:0/3205118783 shutdown_connections 2024-09-21T03:56:11.027 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:11.020+0000 7fabbdbba740 1 --2- 172.21.15.39:0/3205118783 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557e4e1c5070 0x557e4e58b4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:11.027 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:11.020+0000 7fabbdbba740 1 --2- 172.21.15.39:0/3205118783 >> [v2:172.21.15.161:6836/3451544597,v1:172.21.15.161:6837/3451544597] conn(0x557e4e158da0 0x557e4e5f6c10 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:11.027 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:11.020+0000 7fabbdbba740 1 --2- 172.21.15.39:0/3205118783 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557e4e1a4390 0x557e4e58af90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:11.027 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:11.020+0000 7fabbdbba740 1 --2- 172.21.15.39:0/3205118783 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557e4e19c1b0 0x557e4e58aa50 unknown :-1 s=CLOSED pgs=1290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:11.027 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:11.020+0000 7fabbdbba740 1 -- 172.21.15.39:0/3205118783 >> 172.21.15.39:0/3205118783 conn(0x557e4e1b9460 msgr2=0x557e4e56dc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:56:11.027 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:11.020+0000 7fab98ff9640 0 client.18180 ms_handle_reset on v2:172.21.15.161:6836/3451544597 2024-09-21T03:56:11.027 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:11.020+0000 7fabbdbba740 1 -- 172.21.15.39:0/3205118783 shutdown_connections 2024-09-21T03:56:11.027 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:11.020+0000 7fabbdbba740 1 -- 172.21.15.39:0/3205118783 wait complete. 2024-09-21T03:56:11.027 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:11.020+0000 7fabbdbba740 20 client.18180 trim_cache size 0 max 16384 2024-09-21T03:56:11.028 INFO:teuthology.orchestra.run.smithi039.stdout:[] 2024-09-21T03:56:11.028 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:11.021+0000 7fabb7646640 1 -- 172.21.15.39:0/2375088157 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7fab9c0b6680 msgr2=0x7fab9c0b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:56:11.028 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:11.021+0000 7fabb7646640 1 --2- 172.21.15.39:0/2375088157 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7fab9c0b6680 0x7fab9c0b8b90 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x557e4e56af40 tx=0x7fabac00b390 comp rx=0 tx=0).stop 2024-09-21T03:56:11.028 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:11.021+0000 7fabb7646640 1 -- 172.21.15.39:0/2375088157 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fabb01083a0 msgr2=0x7fabb01104f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:56:11.028 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:11.021+0000 7fabb7646640 1 --2- 172.21.15.39:0/2375088157 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fabb01083a0 0x7fabb01104f0 secure :-1 s=READY pgs=1286 cs=0 l=1 rev1=1 crypto rx=0x7faba000d740 tx=0x7faba000dc00 comp rx=0 tx=0).stop 2024-09-21T03:56:11.028 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:11.021+0000 7fabb7646640 1 -- 172.21.15.39:0/2375088157 shutdown_connections 2024-09-21T03:56:11.028 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:11.021+0000 7fabb7646640 1 --2- 172.21.15.39:0/2375088157 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7fab9c0b6680 0x7fab9c0b8b90 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:11.028 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:11.021+0000 7fabb7646640 1 --2- 172.21.15.39:0/2375088157 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fabb0108cf0 0x7fabb0110a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:11.029 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:11.021+0000 7fabb7646640 1 --2- 172.21.15.39:0/2375088157 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fabb01083a0 0x7fabb01104f0 unknown :-1 s=CLOSED pgs=1286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:11.029 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:11.021+0000 7fabb7646640 1 --2- 172.21.15.39:0/2375088157 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fabb0102a60 0x7fabb010ffb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:11.029 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:11.021+0000 7fabb7646640 1 -- 172.21.15.39:0/2375088157 >> 172.21.15.39:0/2375088157 conn(0x7fabb0079730 msgr2=0x7fabb0105240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:56:11.029 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:11.021+0000 7fabb7646640 1 -- 172.21.15.39:0/2375088157 shutdown_connections 2024-09-21T03:56:11.029 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:11.021+0000 7fabb7646640 1 -- 172.21.15.39:0/2375088157 wait complete. 2024-09-21T03:56:11.334 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:56:11 smithi039 ceph-mon[31211]: pgmap v635: 97 pgs: 97 active+clean; 10 MiB data, 525 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 44 KiB/s wr, 28 op/s 2024-09-21T03:56:11.334 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:56:11 smithi039 ceph-mon[31211]: osdmap e86: 12 total, 12 up, 12 in 2024-09-21T03:56:11.468 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:56:11 smithi161 ceph-mon[34533]: pgmap v635: 97 pgs: 97 active+clean; 10 MiB data, 525 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 44 KiB/s wr, 28 op/s 2024-09-21T03:56:11.468 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:56:11 smithi161 ceph-mon[34533]: osdmap e86: 12 total, 12 up, 12 in 2024-09-21T03:56:11.474 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:56:11 smithi150 ceph-mon[34289]: pgmap v635: 97 pgs: 97 active+clean; 10 MiB data, 525 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 44 KiB/s wr, 28 op/s 2024-09-21T03:56:11.475 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:56:11 smithi150 ceph-mon[34289]: osdmap e86: 12 total, 12 up, 12 in 2024-09-21T03:56:12.022 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 51116d5c-77ca-11ef-baf5-efdc52797490 -- ceph tell mds.1:2 damage ls 2024-09-21T03:56:12.285 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/51116d5c-77ca-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:56:12.760 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:12.757+0000 7fc1f546d640 1 -- 172.21.15.39:0/2726166512 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc1f01062e0 msgr2=0x7fc1f01066f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:56:12.760 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:12.757+0000 7fc1f546d640 1 --2- 172.21.15.39:0/2726166512 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc1f01062e0 0x7fc1f01066f0 secure :-1 s=READY pgs=1291 cs=0 l=1 rev1=1 crypto rx=0x7fc1e4009ca0 tx=0x7fc1e402a8c0 comp rx=0 tx=0).stop 2024-09-21T03:56:12.760 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:12.757+0000 7fc1f546d640 1 -- 172.21.15.39:0/2726166512 shutdown_connections 2024-09-21T03:56:12.760 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:12.757+0000 7fc1f546d640 1 --2- 172.21.15.39:0/2726166512 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc1f0106c30 0x7fc1f0110680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:12.760 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:12.757+0000 7fc1f546d640 1 --2- 172.21.15.39:0/2726166512 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc1f01062e0 0x7fc1f01066f0 unknown :-1 s=CLOSED pgs=1291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:12.760 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:12.757+0000 7fc1f546d640 1 --2- 172.21.15.39:0/2726166512 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc1f0102000 0x7fc1f0102410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:12.760 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:12.757+0000 7fc1f546d640 1 -- 172.21.15.39:0/2726166512 >> 172.21.15.39:0/2726166512 conn(0x7fc1f0079e40 msgr2=0x7fc1f007a250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:56:12.760 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:12.757+0000 7fc1f546d640 1 -- 172.21.15.39:0/2726166512 shutdown_connections 2024-09-21T03:56:12.761 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:12.757+0000 7fc1f546d640 1 -- 172.21.15.39:0/2726166512 wait complete. 2024-09-21T03:56:12.762 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:12.758+0000 7fc1f546d640 1 Processor -- start 2024-09-21T03:56:12.762 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:12.758+0000 7fc1f546d640 1 -- start start 2024-09-21T03:56:12.762 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:12.758+0000 7fc1f546d640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc1f0102000 0x7fc1f019a4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:56:12.762 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:12.758+0000 7fc1f546d640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc1f01062e0 0x7fc1f019aa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:56:12.762 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:12.758+0000 7fc1f546d640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc1f0106c30 0x7fc1f019fd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:56:12.762 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:12.758+0000 7fc1f546d640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fc1f0113b10 con 0x7fc1f01062e0 2024-09-21T03:56:12.763 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:12.758+0000 7fc1f546d640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fc1f0113990 con 0x7fc1f0106c30 2024-09-21T03:56:12.763 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:12.758+0000 7fc1f546d640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fc1f0113c90 con 0x7fc1f0102000 2024-09-21T03:56:12.763 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:12.758+0000 7fc1ee7fc640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc1f01062e0 0x7fc1f019aa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:56:12.763 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:12.758+0000 7fc1ee7fc640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc1f01062e0 0x7fc1f019aa80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:59756/0 (socket says 172.21.15.39:59756) 2024-09-21T03:56:12.763 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:12.758+0000 7fc1ee7fc640 1 -- 172.21.15.39:0/1667163526 learned_addr learned my addr 172.21.15.39:0/1667163526 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:56:12.763 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:12.759+0000 7fc1ef7fe640 1 --2- 172.21.15.39:0/1667163526 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc1f0106c30 0x7fc1f019fd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:56:12.763 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:12.759+0000 7fc1ee7fc640 1 -- 172.21.15.39:0/1667163526 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc1f0102000 msgr2=0x7fc1f019a4e0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-21T03:56:12.763 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:12.759+0000 7fc1ee7fc640 1 --2- 172.21.15.39:0/1667163526 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc1f0102000 0x7fc1f019a4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:12.763 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:12.759+0000 7fc1ee7fc640 1 -- 172.21.15.39:0/1667163526 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc1f0106c30 msgr2=0x7fc1f019fd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:56:12.763 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:12.759+0000 7fc1ee7fc640 1 --2- 172.21.15.39:0/1667163526 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc1f0106c30 0x7fc1f019fd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:12.763 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:12.759+0000 7fc1ee7fc640 1 -- 172.21.15.39:0/1667163526 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc1f01a0390 con 0x7fc1f01062e0 2024-09-21T03:56:12.763 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:12.759+0000 7fc1ee7fc640 1 --2- 172.21.15.39:0/1667163526 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc1f01062e0 0x7fc1f019aa80 secure :-1 s=READY pgs=1292 cs=0 l=1 rev1=1 crypto rx=0x7fc1e402ade0 tx=0x7fc1e4003970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:56:12.763 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:12.759+0000 7fc1d7fff640 1 -- 172.21.15.39:0/1667163526 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1e4003d60 con 0x7fc1f01062e0 2024-09-21T03:56:12.763 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:12.759+0000 7fc1d7fff640 1 -- 172.21.15.39:0/1667163526 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc1e40044f0 con 0x7fc1f01062e0 2024-09-21T03:56:12.763 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:12.759+0000 7fc1f546d640 1 -- 172.21.15.39:0/1667163526 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc1f01a0620 con 0x7fc1f01062e0 2024-09-21T03:56:12.764 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:12.759+0000 7fc1d7fff640 1 -- 172.21.15.39:0/1667163526 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1e402fce0 con 0x7fc1f01062e0 2024-09-21T03:56:12.764 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:12.759+0000 7fc1f546d640 1 -- 172.21.15.39:0/1667163526 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc1f01a6ea0 con 0x7fc1f01062e0 2024-09-21T03:56:12.766 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:12.762+0000 7fc1d7fff640 1 -- 172.21.15.39:0/1667163526 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 27) ==== 154643+0+0 (secure 0 0 0) 0x7fc1e40046a0 con 0x7fc1f01062e0 2024-09-21T03:56:12.766 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:12.762+0000 7fc1fb9e1740 1 Processor -- start 2024-09-21T03:56:12.766 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:12.762+0000 7fc1fb9e1740 1 -- start start 2024-09-21T03:56:12.766 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:12.762+0000 7fc1fb9e1740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56210dacb070 0x56210de726a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:56:12.766 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:12.762+0000 7fc1fb9e1740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56210daa21b0 0x56210de72be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:56:12.766 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:12.762+0000 7fc1fb9e1740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56210daaa390 0x56210de73120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:56:12.767 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:12.763+0000 7fc1fb9e1740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x56210dde8ea0 con 0x56210daaa390 2024-09-21T03:56:12.767 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:12.763+0000 7fc1fb9e1740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x56210ddd82d0 con 0x56210dacb070 2024-09-21T03:56:12.767 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:12.763+0000 7fc1fb9e1740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x56210dddc0e0 con 0x56210daa21b0 2024-09-21T03:56:12.767 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:12.763+0000 7fc1ee7fc640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56210daaa390 0x56210de73120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:56:12.767 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:12.763+0000 7fc1ee7fc640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56210daaa390 0x56210de73120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:59760/0 (socket says 172.21.15.39:59760) 2024-09-21T03:56:12.767 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:12.763+0000 7fc1ee7fc640 1 -- 172.21.15.39:0/3616365780 learned_addr learned my addr 172.21.15.39:0/3616365780 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:56:12.767 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:12.763+0000 7fc1eeffd640 1 --2- 172.21.15.39:0/3616365780 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56210daa21b0 0x56210de72be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:56:12.767 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:12.763+0000 7fc1ef7fe640 1 --2- 172.21.15.39:0/3616365780 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56210dacb070 0x56210de726a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:56:12.767 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:12.763+0000 7fc1ee7fc640 1 -- 172.21.15.39:0/3616365780 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56210daa21b0 msgr2=0x56210de72be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:56:12.767 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:12.763+0000 7fc1ee7fc640 1 --2- 172.21.15.39:0/3616365780 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56210daa21b0 0x56210de72be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:12.767 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:12.763+0000 7fc1d7fff640 1 --2- 172.21.15.39:0/1667163526 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7fc1d80b66b0 0x7fc1d80b8bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:56:12.767 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:12.763+0000 7fc1ee7fc640 1 -- 172.21.15.39:0/3616365780 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56210dacb070 msgr2=0x56210de726a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:56:12.767 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:12.763+0000 7fc1ee7fc640 1 --2- 172.21.15.39:0/3616365780 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56210dacb070 0x56210de726a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:12.767 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:12.763+0000 7fc1ee7fc640 1 -- 172.21.15.39:0/3616365780 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56210db36650 con 0x56210daaa390 2024-09-21T03:56:12.768 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:12.763+0000 7fc1eeffd640 1 --2- 172.21.15.39:0/3616365780 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56210daa21b0 0x56210de72be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:56:12.768 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:12.763+0000 7fc1ef7fe640 1 --2- 172.21.15.39:0/1667163526 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7fc1d80b66b0 0x7fc1d80b8bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:56:12.768 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:12.763+0000 7fc1ee7fc640 1 --2- 172.21.15.39:0/3616365780 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56210daaa390 0x56210de73120 secure :-1 s=READY pgs=1293 cs=0 l=1 rev1=1 crypto rx=0x7fc1e4106310 tx=0x7fc1e40cec90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=14c3cdd7b2ef5f64 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:56:12.768 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:12.764+0000 7fc1d7fff640 1 -- 172.21.15.39:0/1667163526 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8572+0+0 (secure 0 0 0) 0x7fc1e4108c80 con 0x7fc1f01062e0 2024-09-21T03:56:12.771 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:12.764+0000 7fc1d57fa640 1 -- 172.21.15.39:0/3616365780 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1e40ce870 con 0x56210daaa390 2024-09-21T03:56:12.771 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:12.764+0000 7fc1d57fa640 1 -- 172.21.15.39:0/3616365780 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc1e4036070 con 0x56210daaa390 2024-09-21T03:56:12.771 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:12.764+0000 7fc1d57fa640 1 -- 172.21.15.39:0/3616365780 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1e40e4c50 con 0x56210daaa390 2024-09-21T03:56:12.771 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:12.765+0000 7fc1fb9e1740 1 -- 172.21.15.39:0/3616365780 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56210daaa390 msgr2=0x56210de73120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:56:12.771 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:12.765+0000 7fc1fb9e1740 1 --2- 172.21.15.39:0/3616365780 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56210daaa390 0x56210de73120 secure :-1 s=READY pgs=1293 cs=0 l=1 rev1=1 crypto rx=0x7fc1e4106310 tx=0x7fc1e40cec90 comp rx=0 tx=0).stop 2024-09-21T03:56:12.771 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:12.765+0000 7fc1ef7fe640 1 --2- 172.21.15.39:0/1667163526 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7fc1d80b66b0 0x7fc1d80b8bc0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7fc1e8006fd0 tx=0x7fc1e80047d0 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:56:12.771 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:12.765+0000 7fc1fb9e1740 1 -- 172.21.15.39:0/3616365780 shutdown_connections 2024-09-21T03:56:12.771 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:12.765+0000 7fc1fb9e1740 1 --2- 172.21.15.39:0/3616365780 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56210daaa390 0x56210de73120 unknown :-1 s=CLOSED pgs=1293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:12.772 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:12.765+0000 7fc1fb9e1740 1 --2- 172.21.15.39:0/3616365780 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56210daa21b0 0x56210de72be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:12.772 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:12.765+0000 7fc1fb9e1740 1 --2- 172.21.15.39:0/3616365780 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56210dacb070 0x56210de726a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:12.772 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:12.765+0000 7fc1fb9e1740 1 -- 172.21.15.39:0/3616365780 >> 172.21.15.39:0/3616365780 conn(0x56210dabf460 msgr2=0x56210de70590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:56:12.772 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:12.765+0000 7fc1fb9e1740 1 -- 172.21.15.39:0/3616365780 shutdown_connections 2024-09-21T03:56:12.772 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:12.765+0000 7fc1fb9e1740 1 -- 172.21.15.39:0/3616365780 wait complete. 2024-09-21T03:56:12.772 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:12.765+0000 7fc1fb9e1740 1 -- start start 2024-09-21T03:56:12.772 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:12.765+0000 7fc1fb9e1740 1 Processor -- start 2024-09-21T03:56:12.772 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:12.766+0000 7fc1fb9e1740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56210daa21b0 0x56210de7dcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:56:12.772 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:12.766+0000 7fc1fb9e1740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56210daaa390 0x56210de7e230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:56:12.772 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:12.766+0000 7fc1fb9e1740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56210dacb070 0x56210de89d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:56:12.772 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:12.766+0000 7fc1fb9e1740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x56210dc39700 con 0x56210dacb070 2024-09-21T03:56:12.772 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:12.766+0000 7fc1fb9e1740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x56210dd73160 con 0x56210daa21b0 2024-09-21T03:56:12.772 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:12.766+0000 7fc1fb9e1740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x56210dde8ea0 con 0x56210daaa390 2024-09-21T03:56:12.772 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:12.766+0000 7fc1fb9e1740 10 client.0 _ll_register_callbacks cb 0x56210dd8c9e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:56:12.772 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:12.766+0000 7fc1ef7fe640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56210dacb070 0x56210de89d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:56:12.773 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:12.766+0000 7fc1ef7fe640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56210dacb070 0x56210de89d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:59774/0 (socket says 172.21.15.39:59774) 2024-09-21T03:56:12.773 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:12.766+0000 7fc1ef7fe640 1 -- 172.21.15.39:0/225865207 learned_addr learned my addr 172.21.15.39:0/225865207 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:56:12.773 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:12.766+0000 7fc1eeffd640 1 --2- 172.21.15.39:0/225865207 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56210daa21b0 0x56210de7dcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:56:12.773 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:12.766+0000 7fc1ee7fc640 1 --2- 172.21.15.39:0/225865207 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56210daaa390 0x56210de7e230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:56:12.773 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:12.766+0000 7fc1ef7fe640 1 -- 172.21.15.39:0/225865207 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56210daaa390 msgr2=0x56210de7e230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:56:12.773 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:12.766+0000 7fc1ef7fe640 1 --2- 172.21.15.39:0/225865207 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56210daaa390 0x56210de7e230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:12.773 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:12.766+0000 7fc1ef7fe640 1 -- 172.21.15.39:0/225865207 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56210daa21b0 msgr2=0x56210de7dcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:56:12.773 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:12.766+0000 7fc1ef7fe640 1 --2- 172.21.15.39:0/225865207 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56210daa21b0 0x56210de7dcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:12.773 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:12.766+0000 7fc1ef7fe640 1 -- 172.21.15.39:0/225865207 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56210dda0c90 con 0x56210dacb070 2024-09-21T03:56:12.773 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:12.767+0000 7fc1ef7fe640 1 --2- 172.21.15.39:0/225865207 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56210dacb070 0x56210de89d70 secure :-1 s=READY pgs=1294 cs=0 l=1 rev1=1 crypto rx=0x7fc1e800b480 tx=0x7fc1e8008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=426dcf7c64ebb894 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:56:12.773 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:12.767+0000 7fc1d2ff5640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-09-21T03:56:12.773 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:12.767+0000 7fc1d2ff5640 1 -- 172.21.15.39:0/225865207 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1e8024020 con 0x56210dacb070 2024-09-21T03:56:12.773 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:12.767+0000 7fc1d2ff5640 1 -- 172.21.15.39:0/225865207 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8572+0+0 (secure 0 0 0) 0x7fc1e8013740 con 0x56210dacb070 2024-09-21T03:56:12.773 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:12.767+0000 7fc1fb9e1740 1 -- 172.21.15.39:0/225865207 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc1e4036070 con 0x56210dacb070 2024-09-21T03:56:12.773 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:12.767+0000 7fc1d2ff5640 1 -- 172.21.15.39:0/225865207 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fc1e802f050 con 0x56210dacb070 2024-09-21T03:56:12.774 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:12.767+0000 7fc1fb9e1740 10 client.18192 fetch_fsmap learned FSMap version 29 2024-09-21T03:56:12.774 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:12.767+0000 7fc1fb9e1740 1 -- 172.21.15.39:0/225865207 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x56210de8c790 con 0x56210dacb070 2024-09-21T03:56:12.774 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:12.767+0000 7fc1d2ff5640 1 -- 172.21.15.39:0/225865207 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc1e4036070 con 0x56210dacb070 2024-09-21T03:56:12.774 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:12.767+0000 7fc1d2ff5640 1 -- 172.21.15.39:0/225865207 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fc1e8020a50 con 0x56210dacb070 2024-09-21T03:56:12.774 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:12.767+0000 7fc1fb9e1740 10 client.18192 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T03:56:12.774 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:12.767+0000 7fc1fb9e1740 10 client.18192 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-21T03:56:12.774 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:12.767+0000 7fc1fb9e1740 20 client.18192 populate_metadata read hostname 'smithi039' 2024-09-21T03:56:12.774 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:12.767+0000 7fc1fb9e1740 1 --2- 172.21.15.39:0/225865207 >> [v2:172.21.15.150:6836/4017043144,v1:172.21.15.150:6837/4017043144] conn(0x56210da899f0 0x56210de8fd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:56:12.774 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:12.768+0000 7fc1fb9e1740 4 client.18192 mds_command: new command op to 24659 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:56:12.774 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:12.768+0000 7fc1fb9e1740 1 -- 172.21.15.39:0/225865207 --> [v2:172.21.15.150:6836/4017043144,v1:172.21.15.150:6837/4017043144] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56210ddeadf0 con 0x56210da899f0 2024-09-21T03:56:12.774 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:12.768+0000 7fc1eeffd640 1 --2- 172.21.15.39:0/225865207 >> [v2:172.21.15.150:6836/4017043144,v1:172.21.15.150:6837/4017043144] conn(0x56210da899f0 0x56210de8fd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:56:12.774 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:12.768+0000 7fc1d2ff5640 10 client.18192 ms_handle_connect on v2:172.21.15.150:6836/4017043144 2024-09-21T03:56:12.774 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:12.768+0000 7fc1eeffd640 1 --2- 172.21.15.39:0/225865207 >> [v2:172.21.15.150:6836/4017043144,v1:172.21.15.150:6837/4017043144] conn(0x56210da899f0 0x56210de8fd20 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=72a8a0e56f15b0c server_cookie=e9d1dfb1fc26816e in_seq=0 out_seq=0 2024-09-21T03:56:12.775 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:12.769+0000 7fc1d2ff5640 1 -- 172.21.15.39:0/225865207 <== mds.2 v2:172.21.15.150:6836/4017043144 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x56210ddeadf0 con 0x56210da899f0 2024-09-21T03:56:12.776 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:12.769+0000 7fc1d2ff5640 10 client.18192 handle_command_reply: tid=0 2024-09-21T03:56:12.776 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:12.769+0000 7fc1fb9e1740 1 client.18192 shutdown 2024-09-21T03:56:12.776 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:12.769+0000 7fc1fb9e1740 1 -- 172.21.15.39:0/225865207 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56210dacb070 msgr2=0x56210de89d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:56:12.776 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:12.769+0000 7fc1fb9e1740 1 --2- 172.21.15.39:0/225865207 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56210dacb070 0x56210de89d70 secure :-1 s=READY pgs=1294 cs=0 l=1 rev1=1 crypto rx=0x7fc1e800b480 tx=0x7fc1e8008040 comp rx=0 tx=0).stop 2024-09-21T03:56:12.776 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:12.769+0000 7fc1fb9e1740 1 -- 172.21.15.39:0/225865207 shutdown_connections 2024-09-21T03:56:12.776 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:12.769+0000 7fc1fb9e1740 1 --2- 172.21.15.39:0/225865207 >> [v2:172.21.15.150:6836/4017043144,v1:172.21.15.150:6837/4017043144] conn(0x56210da899f0 0x56210de8fd20 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:12.776 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:12.770+0000 7fc1fb9e1740 1 --2- 172.21.15.39:0/225865207 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56210dacb070 0x56210de89d70 unknown :-1 s=CLOSED pgs=1294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:12.776 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:12.770+0000 7fc1d2ff5640 0 client.18192 ms_handle_reset on v2:172.21.15.150:6836/4017043144 2024-09-21T03:56:12.776 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:12.770+0000 7fc1fb9e1740 1 --2- 172.21.15.39:0/225865207 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56210daaa390 0x56210de7e230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:12.776 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:12.770+0000 7fc1fb9e1740 1 --2- 172.21.15.39:0/225865207 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56210daa21b0 0x56210de7dcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:12.776 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:12.770+0000 7fc1fb9e1740 1 -- 172.21.15.39:0/225865207 >> 172.21.15.39:0/225865207 conn(0x56210dabf460 msgr2=0x56210de75830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:56:12.776 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:12.770+0000 7fc1fb9e1740 1 -- 172.21.15.39:0/225865207 shutdown_connections 2024-09-21T03:56:12.776 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:12.770+0000 7fc1fb9e1740 1 -- 172.21.15.39:0/225865207 wait complete. 2024-09-21T03:56:12.776 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:12.770+0000 7fc1fb9e1740 20 client.18192 trim_cache size 0 max 16384 2024-09-21T03:56:12.786 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:12.783+0000 7fc1fb9e1740 1 Processor -- start 2024-09-21T03:56:12.786 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:12.783+0000 7fc1fb9e1740 1 -- start start 2024-09-21T03:56:12.787 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:12.783+0000 7fc1fb9e1740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56210da899f0 0x56210de73de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:56:12.787 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:12.783+0000 7fc1fb9e1740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56210daa21b0 0x56210de6fbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:56:12.787 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:12.783+0000 7fc1fb9e1740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56210daaa390 0x56210de70100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:56:12.787 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:12.783+0000 7fc1fb9e1740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fc1f01af840 con 0x56210daa21b0 2024-09-21T03:56:12.787 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:12.783+0000 7fc1fb9e1740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fc1f01b1380 con 0x56210da899f0 2024-09-21T03:56:12.787 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:12.783+0000 7fc1fb9e1740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fc1f01aff10 con 0x56210daaa390 2024-09-21T03:56:12.787 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:12.783+0000 7fc1ee7fc640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56210daa21b0 0x56210de6fbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:56:12.787 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:12.783+0000 7fc1ee7fc640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56210daa21b0 0x56210de6fbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:59786/0 (socket says 172.21.15.39:59786) 2024-09-21T03:56:12.787 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:12.783+0000 7fc1ee7fc640 1 -- 172.21.15.39:0/4287820969 learned_addr learned my addr 172.21.15.39:0/4287820969 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:56:12.787 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:12.784+0000 7fc1eeffd640 1 --2- 172.21.15.39:0/4287820969 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56210da899f0 0x56210de73de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:56:12.787 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:12.784+0000 7fc1ef7fe640 1 --2- 172.21.15.39:0/4287820969 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56210daaa390 0x56210de70100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:56:12.787 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:12.784+0000 7fc1ee7fc640 1 -- 172.21.15.39:0/4287820969 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56210daaa390 msgr2=0x56210de70100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:56:12.787 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:12.784+0000 7fc1ee7fc640 1 --2- 172.21.15.39:0/4287820969 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56210daaa390 0x56210de70100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:12.787 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:12.784+0000 7fc1ee7fc640 1 -- 172.21.15.39:0/4287820969 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56210da899f0 msgr2=0x56210de73de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:56:12.788 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:12.784+0000 7fc1ee7fc640 1 --2- 172.21.15.39:0/4287820969 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56210da899f0 0x56210de73de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:12.788 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:12.784+0000 7fc1ee7fc640 1 -- 172.21.15.39:0/4287820969 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56210de74990 con 0x56210daa21b0 2024-09-21T03:56:12.788 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:12.784+0000 7fc1eeffd640 1 --2- 172.21.15.39:0/4287820969 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56210da899f0 0x56210de73de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:56:12.788 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:12.784+0000 7fc1ee7fc640 1 --2- 172.21.15.39:0/4287820969 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56210daa21b0 0x56210de6fbc0 secure :-1 s=READY pgs=1295 cs=0 l=1 rev1=1 crypto rx=0x7fc1e40cbd10 tx=0x7fc1e4005c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d758b1e2a2d3f60 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:56:12.791 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:12.784+0000 7fc1d4ff9640 1 -- 172.21.15.39:0/4287820969 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1e40ce210 con 0x56210daa21b0 2024-09-21T03:56:12.791 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:12.784+0000 7fc1d4ff9640 1 -- 172.21.15.39:0/4287820969 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc1e4036040 con 0x56210daa21b0 2024-09-21T03:56:12.791 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:12.784+0000 7fc1d4ff9640 1 -- 172.21.15.39:0/4287820969 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1e40ce3f0 con 0x56210daa21b0 2024-09-21T03:56:12.791 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:12.785+0000 7fc1fb9e1740 1 -- 172.21.15.39:0/4287820969 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56210daa21b0 msgr2=0x56210de6fbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:56:12.791 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:12.785+0000 7fc1fb9e1740 1 --2- 172.21.15.39:0/4287820969 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56210daa21b0 0x56210de6fbc0 secure :-1 s=READY pgs=1295 cs=0 l=1 rev1=1 crypto rx=0x7fc1e40cbd10 tx=0x7fc1e4005c30 comp rx=0 tx=0).stop 2024-09-21T03:56:12.791 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:12.785+0000 7fc1fb9e1740 1 -- 172.21.15.39:0/4287820969 shutdown_connections 2024-09-21T03:56:12.791 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:12.785+0000 7fc1fb9e1740 1 --2- 172.21.15.39:0/4287820969 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56210daaa390 0x56210de70100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:12.791 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:12.785+0000 7fc1fb9e1740 1 --2- 172.21.15.39:0/4287820969 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56210daa21b0 0x56210de6fbc0 unknown :-1 s=CLOSED pgs=1295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:12.791 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:12.785+0000 7fc1fb9e1740 1 --2- 172.21.15.39:0/4287820969 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56210da899f0 0x56210de73de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:12.791 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:12.785+0000 7fc1fb9e1740 1 -- 172.21.15.39:0/4287820969 >> 172.21.15.39:0/4287820969 conn(0x56210dabf460 msgr2=0x56210de736d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:56:12.791 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:12.785+0000 7fc1fb9e1740 1 -- 172.21.15.39:0/4287820969 shutdown_connections 2024-09-21T03:56:12.791 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:12.785+0000 7fc1fb9e1740 1 -- 172.21.15.39:0/4287820969 wait complete. 2024-09-21T03:56:12.791 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:12.785+0000 7fc1fb9e1740 1 -- start start 2024-09-21T03:56:12.792 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:12.785+0000 7fc1fb9e1740 1 Processor -- start 2024-09-21T03:56:12.792 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:12.786+0000 7fc1fb9e1740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56210da899f0 0x56210de81d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:56:12.792 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:12.786+0000 7fc1fb9e1740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56210daa21b0 0x56210de82290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:56:12.792 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:12.786+0000 7fc1fb9e1740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56210daaa390 0x56210de827d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:56:12.792 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:12.786+0000 7fc1fb9e1740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fc1f01b0cb0 con 0x56210da899f0 2024-09-21T03:56:12.792 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:12.786+0000 7fc1fb9e1740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fc1f01b05e0 con 0x56210daa21b0 2024-09-21T03:56:12.792 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:12.786+0000 7fc1fb9e1740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fc1f01ad840 con 0x56210daaa390 2024-09-21T03:56:12.792 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:12.786+0000 7fc1fb9e1740 10 client.0 _ll_register_callbacks cb 0x7fc1e800b500 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:56:12.792 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:12.786+0000 7fc1eeffd640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56210da899f0 0x56210de81d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:56:12.792 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:12.786+0000 7fc1eeffd640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56210da899f0 0x56210de81d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:59788/0 (socket says 172.21.15.39:59788) 2024-09-21T03:56:12.792 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:12.786+0000 7fc1eeffd640 1 -- 172.21.15.39:0/3235457543 learned_addr learned my addr 172.21.15.39:0/3235457543 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:56:12.792 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:12.786+0000 7fc1ef7fe640 1 --2- 172.21.15.39:0/3235457543 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56210daaa390 0x56210de827d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:56:12.792 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:12.786+0000 7fc1ee7fc640 1 --2- 172.21.15.39:0/3235457543 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56210daa21b0 0x56210de82290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:56:12.792 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:12.786+0000 7fc1eeffd640 1 -- 172.21.15.39:0/3235457543 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56210daaa390 msgr2=0x56210de827d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:56:12.792 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:12.786+0000 7fc1eeffd640 1 --2- 172.21.15.39:0/3235457543 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56210daaa390 0x56210de827d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:12.793 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:12.786+0000 7fc1eeffd640 1 -- 172.21.15.39:0/3235457543 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56210daa21b0 msgr2=0x56210de82290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:56:12.793 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:12.786+0000 7fc1eeffd640 1 --2- 172.21.15.39:0/3235457543 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56210daa21b0 0x56210de82290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:12.793 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:12.786+0000 7fc1eeffd640 1 -- 172.21.15.39:0/3235457543 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56210def97a0 con 0x56210da899f0 2024-09-21T03:56:12.793 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:12.786+0000 7fc1ef7fe640 1 --2- 172.21.15.39:0/3235457543 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56210daaa390 0x56210de827d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:56:12.793 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:12.786+0000 7fc1ee7fc640 1 --2- 172.21.15.39:0/3235457543 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56210daa21b0 0x56210de82290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:56:12.793 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:12.786+0000 7fc1eeffd640 1 --2- 172.21.15.39:0/3235457543 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56210da899f0 0x56210de81d50 secure :-1 s=READY pgs=1296 cs=0 l=1 rev1=1 crypto rx=0x56210de71320 tx=0x7fc1e0010800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fde5644746a369bd server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:56:12.793 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:12.786+0000 7fc1d2ff5640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-09-21T03:56:12.793 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:12.787+0000 7fc1d2ff5640 1 -- 172.21.15.39:0/3235457543 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1e0005220 con 0x56210da899f0 2024-09-21T03:56:12.793 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:12.787+0000 7fc1d2ff5640 1 -- 172.21.15.39:0/3235457543 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8572+0+0 (secure 0 0 0) 0x7fc1e00053c0 con 0x56210da899f0 2024-09-21T03:56:12.793 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:12.787+0000 7fc1fb9e1740 1 -- 172.21.15.39:0/3235457543 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc1e4036040 con 0x56210da899f0 2024-09-21T03:56:12.793 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:12.787+0000 7fc1d2ff5640 1 -- 172.21.15.39:0/3235457543 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fc1e00055c0 con 0x56210da899f0 2024-09-21T03:56:12.793 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:12.787+0000 7fc1fb9e1740 10 client.18198 fetch_fsmap learned FSMap version 29 2024-09-21T03:56:12.793 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:12.787+0000 7fc1fb9e1740 1 -- 172.21.15.39:0/3235457543 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x56210defaa90 con 0x56210da899f0 2024-09-21T03:56:12.793 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:12.787+0000 7fc1d2ff5640 1 -- 172.21.15.39:0/3235457543 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc1e4036040 con 0x56210da899f0 2024-09-21T03:56:12.794 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:12.787+0000 7fc1d2ff5640 1 -- 172.21.15.39:0/3235457543 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fc1e00218a0 con 0x56210da899f0 2024-09-21T03:56:12.794 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:12.787+0000 7fc1fb9e1740 10 client.18198 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T03:56:12.794 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:12.787+0000 7fc1fb9e1740 10 client.18198 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-21T03:56:12.794 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:12.787+0000 7fc1fb9e1740 20 client.18198 populate_metadata read hostname 'smithi039' 2024-09-21T03:56:12.794 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:12.787+0000 7fc1fb9e1740 1 --2- 172.21.15.39:0/3235457543 >> [v2:172.21.15.150:6836/4017043144,v1:172.21.15.150:6837/4017043144] conn(0x56210dacb070 0x56210defd2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:56:12.794 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:12.787+0000 7fc1fb9e1740 4 client.18198 mds_command: new command op to 24659 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T03:56:12.794 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:12.787+0000 7fc1fb9e1740 1 -- 172.21.15.39:0/3235457543 --> [v2:172.21.15.150:6836/4017043144,v1:172.21.15.150:6837/4017043144] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56210dde68a0 con 0x56210dacb070 2024-09-21T03:56:12.794 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:12.788+0000 7fc1ee7fc640 1 --2- 172.21.15.39:0/3235457543 >> [v2:172.21.15.150:6836/4017043144,v1:172.21.15.150:6837/4017043144] conn(0x56210dacb070 0x56210defd2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:56:12.795 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:12.788+0000 7fc1ee7fc640 1 --2- 172.21.15.39:0/3235457543 >> [v2:172.21.15.150:6836/4017043144,v1:172.21.15.150:6837/4017043144] conn(0x56210dacb070 0x56210defd2b0 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=9e65e2d56c89403c server_cookie=220f5a21b38fd86b in_seq=0 out_seq=0 2024-09-21T03:56:12.795 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:12.788+0000 7fc1d2ff5640 10 client.18198 ms_handle_connect on v2:172.21.15.150:6836/4017043144 2024-09-21T03:56:12.795 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:12.789+0000 7fc1d2ff5640 1 -- 172.21.15.39:0/3235457543 <== mds.2 v2:172.21.15.150:6836/4017043144 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56210dde68a0 con 0x56210dacb070 2024-09-21T03:56:12.795 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:12.789+0000 7fc1d2ff5640 10 client.18198 handle_command_reply: tid=0 2024-09-21T03:56:12.795 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:12.789+0000 7fc1fb9e1740 1 client.18198 shutdown 2024-09-21T03:56:12.795 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:12.789+0000 7fc1fb9e1740 1 -- 172.21.15.39:0/3235457543 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56210da899f0 msgr2=0x56210de81d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:56:12.795 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:12.789+0000 7fc1fb9e1740 1 --2- 172.21.15.39:0/3235457543 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56210da899f0 0x56210de81d50 secure :-1 s=READY pgs=1296 cs=0 l=1 rev1=1 crypto rx=0x56210de71320 tx=0x7fc1e0010800 comp rx=0 tx=0).stop 2024-09-21T03:56:12.795 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:12.789+0000 7fc1fb9e1740 1 -- 172.21.15.39:0/3235457543 shutdown_connections 2024-09-21T03:56:12.796 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:12.789+0000 7fc1fb9e1740 1 --2- 172.21.15.39:0/3235457543 >> [v2:172.21.15.150:6836/4017043144,v1:172.21.15.150:6837/4017043144] conn(0x56210dacb070 0x56210defd2b0 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:12.796 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:12.789+0000 7fc1fb9e1740 1 --2- 172.21.15.39:0/3235457543 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56210daaa390 0x56210de827d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:12.796 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:12.789+0000 7fc1fb9e1740 1 --2- 172.21.15.39:0/3235457543 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56210daa21b0 0x56210de82290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:12.796 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:12.789+0000 7fc1fb9e1740 1 --2- 172.21.15.39:0/3235457543 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56210da899f0 0x56210de81d50 unknown :-1 s=CLOSED pgs=1296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:12.796 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:12.789+0000 7fc1fb9e1740 1 -- 172.21.15.39:0/3235457543 >> 172.21.15.39:0/3235457543 conn(0x56210dabf460 msgr2=0x56210de74040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:56:12.796 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:12.789+0000 7fc1d2ff5640 0 client.18198 ms_handle_reset on v2:172.21.15.150:6836/4017043144 2024-09-21T03:56:12.796 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:12.789+0000 7fc1fb9e1740 1 -- 172.21.15.39:0/3235457543 shutdown_connections 2024-09-21T03:56:12.796 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:12.789+0000 7fc1fb9e1740 1 -- 172.21.15.39:0/3235457543 wait complete. 2024-09-21T03:56:12.796 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:12.789+0000 7fc1fb9e1740 20 client.18198 trim_cache size 0 max 16384 2024-09-21T03:56:12.796 INFO:teuthology.orchestra.run.smithi039.stdout:[] 2024-09-21T03:56:12.797 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:12.790+0000 7fc1f546d640 1 -- 172.21.15.39:0/1667163526 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7fc1d80b66b0 msgr2=0x7fc1d80b8bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:56:12.797 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:12.790+0000 7fc1f546d640 1 --2- 172.21.15.39:0/1667163526 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7fc1d80b66b0 0x7fc1d80b8bc0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7fc1e8006fd0 tx=0x7fc1e80047d0 comp rx=0 tx=0).stop 2024-09-21T03:56:12.797 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:12.790+0000 7fc1f546d640 1 -- 172.21.15.39:0/1667163526 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc1f01062e0 msgr2=0x7fc1f019aa80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:56:12.797 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:12.790+0000 7fc1f546d640 1 --2- 172.21.15.39:0/1667163526 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc1f01062e0 0x7fc1f019aa80 secure :-1 s=READY pgs=1292 cs=0 l=1 rev1=1 crypto rx=0x7fc1e402ade0 tx=0x7fc1e4003970 comp rx=0 tx=0).stop 2024-09-21T03:56:12.797 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:12.790+0000 7fc1f546d640 1 -- 172.21.15.39:0/1667163526 shutdown_connections 2024-09-21T03:56:12.797 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:12.790+0000 7fc1f546d640 1 --2- 172.21.15.39:0/1667163526 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7fc1d80b66b0 0x7fc1d80b8bc0 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:12.797 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:12.790+0000 7fc1f546d640 1 --2- 172.21.15.39:0/1667163526 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc1f0106c30 0x7fc1f019fd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:12.797 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:12.790+0000 7fc1f546d640 1 --2- 172.21.15.39:0/1667163526 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc1f01062e0 0x7fc1f019aa80 unknown :-1 s=CLOSED pgs=1292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:12.797 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:12.790+0000 7fc1f546d640 1 --2- 172.21.15.39:0/1667163526 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc1f0102000 0x7fc1f019a4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:12.797 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:12.790+0000 7fc1f546d640 1 -- 172.21.15.39:0/1667163526 >> 172.21.15.39:0/1667163526 conn(0x7fc1f0079e40 msgr2=0x7fc1f0107550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:56:12.798 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:12.790+0000 7fc1f546d640 1 -- 172.21.15.39:0/1667163526 shutdown_connections 2024-09-21T03:56:12.798 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:12.790+0000 7fc1f546d640 1 -- 172.21.15.39:0/1667163526 wait complete. 2024-09-21T03:56:13.440 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:56:13 smithi039 ceph-mon[31211]: pgmap v637: 97 pgs: 97 active+clean; 10 MiB data, 525 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 53 KiB/s wr, 30 op/s 2024-09-21T03:56:13.468 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:56:13 smithi161 ceph-mon[34533]: pgmap v637: 97 pgs: 97 active+clean; 10 MiB data, 525 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 53 KiB/s wr, 30 op/s 2024-09-21T03:56:13.474 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:56:13 smithi150 ceph-mon[34289]: pgmap v637: 97 pgs: 97 active+clean; 10 MiB data, 525 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 53 KiB/s wr, 30 op/s 2024-09-21T03:56:13.639 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 51116d5c-77ca-11ef-baf5-efdc52797490 -- ceph tell mds.1:1 damage ls 2024-09-21T03:56:13.906 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/51116d5c-77ca-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:56:14.457 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:14.450+0000 7f7854ec9640 1 -- 172.21.15.39:0/3004863972 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7850108d10 msgr2=0x7f785010f540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:56:14.457 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:14.450+0000 7f7854ec9640 1 --2- 172.21.15.39:0/3004863972 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7850108d10 0x7f785010f540 secure :-1 s=READY pgs=1297 cs=0 l=1 rev1=1 crypto rx=0x7f7848009ca0 tx=0x7f784802a8c0 comp rx=0 tx=0).stop 2024-09-21T03:56:14.457 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:14.450+0000 7f7854ec9640 1 -- 172.21.15.39:0/3004863972 shutdown_connections 2024-09-21T03:56:14.457 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:14.450+0000 7f7854ec9640 1 --2- 172.21.15.39:0/3004863972 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7850108d10 0x7f785010f540 unknown :-1 s=CLOSED pgs=1297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:14.457 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:14.450+0000 7f7854ec9640 1 --2- 172.21.15.39:0/3004863972 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f78501083c0 0x7f78501087d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:14.457 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:14.450+0000 7f7854ec9640 1 --2- 172.21.15.39:0/3004863972 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7850102a80 0x7f7850102e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:14.457 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:14.450+0000 7f7854ec9640 1 -- 172.21.15.39:0/3004863972 >> 172.21.15.39:0/3004863972 conn(0x7f7850079740 msgr2=0x7f7850079b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:56:14.457 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:14.450+0000 7f7854ec9640 1 -- 172.21.15.39:0/3004863972 shutdown_connections 2024-09-21T03:56:14.457 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:14.450+0000 7f7854ec9640 1 -- 172.21.15.39:0/3004863972 wait complete. 2024-09-21T03:56:14.457 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:14.451+0000 7f7854ec9640 1 Processor -- start 2024-09-21T03:56:14.457 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:14.452+0000 7f7854ec9640 1 -- start start 2024-09-21T03:56:14.458 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:14.452+0000 7f7854ec9640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7850102a80 0x7f785019a510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:56:14.458 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:14.452+0000 7f7854ec9640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f78501083c0 0x7f785019aab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:56:14.458 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:14.452+0000 7f7854ec9640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7850108d10 0x7f785019fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:56:14.458 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:14.452+0000 7f7854ec9640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f7850113ae0 con 0x7f78501083c0 2024-09-21T03:56:14.458 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:14.452+0000 7f7854ec9640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f7850113960 con 0x7f7850108d10 2024-09-21T03:56:14.458 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:14.452+0000 7f7854ec9640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f7850113c60 con 0x7f7850102a80 2024-09-21T03:56:14.458 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:14.452+0000 7f784dd9b640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f78501083c0 0x7f785019aab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:56:14.458 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:14.452+0000 7f784dd9b640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f78501083c0 0x7f785019aab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:59800/0 (socket says 172.21.15.39:59800) 2024-09-21T03:56:14.458 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:14.452+0000 7f784dd9b640 1 -- 172.21.15.39:0/3062462118 learned_addr learned my addr 172.21.15.39:0/3062462118 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:56:14.458 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:14.452+0000 7f784e59c640 1 --2- 172.21.15.39:0/3062462118 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7850102a80 0x7f785019a510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:56:14.458 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:14.452+0000 7f784ed9d640 1 --2- 172.21.15.39:0/3062462118 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7850108d10 0x7f785019fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:56:14.458 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:14.452+0000 7f784dd9b640 1 -- 172.21.15.39:0/3062462118 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7850102a80 msgr2=0x7f785019a510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:56:14.459 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:14.452+0000 7f784dd9b640 1 --2- 172.21.15.39:0/3062462118 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7850102a80 0x7f785019a510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:14.459 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:14.452+0000 7f784dd9b640 1 -- 172.21.15.39:0/3062462118 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7850108d10 msgr2=0x7f785019fd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:56:14.459 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:14.452+0000 7f784dd9b640 1 --2- 172.21.15.39:0/3062462118 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7850108d10 0x7f785019fd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:14.459 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:14.452+0000 7f784dd9b640 1 -- 172.21.15.39:0/3062462118 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f78501a0480 con 0x7f78501083c0 2024-09-21T03:56:14.459 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:14.453+0000 7f784dd9b640 1 --2- 172.21.15.39:0/3062462118 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f78501083c0 0x7f785019aab0 secure :-1 s=READY pgs=1298 cs=0 l=1 rev1=1 crypto rx=0x7f784400c970 tx=0x7f784400cd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:56:14.459 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:14.453+0000 7f783f7fe640 1 -- 172.21.15.39:0/3062462118 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7844007bb0 con 0x7f78501083c0 2024-09-21T03:56:14.459 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:14.453+0000 7f783f7fe640 1 -- 172.21.15.39:0/3062462118 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7844007d50 con 0x7f78501083c0 2024-09-21T03:56:14.459 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:14.453+0000 7f7854ec9640 1 -- 172.21.15.39:0/3062462118 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f78501a0770 con 0x7f78501083c0 2024-09-21T03:56:14.459 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:14.453+0000 7f783f7fe640 1 -- 172.21.15.39:0/3062462118 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7844004dd0 con 0x7f78501083c0 2024-09-21T03:56:14.459 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:14.453+0000 7f7854ec9640 1 -- 172.21.15.39:0/3062462118 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f78501a6ed0 con 0x7f78501083c0 2024-09-21T03:56:14.464 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:14.455+0000 7f785b43d740 1 Processor -- start 2024-09-21T03:56:14.464 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:14.455+0000 7f783f7fe640 1 -- 172.21.15.39:0/3062462118 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 27) ==== 154643+0+0 (secure 0 0 0) 0x7f7844004fb0 con 0x7f78501083c0 2024-09-21T03:56:14.464 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:14.456+0000 7f785b43d740 1 -- start start 2024-09-21T03:56:14.464 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:14.456+0000 7f785b43d740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564dd28e0070 0x564dd2c876a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:56:14.464 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:14.456+0000 7f785b43d740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564dd28b71b0 0x564dd2c87be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:56:14.464 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:14.456+0000 7f785b43d740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564dd28bf390 0x564dd2c88120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:56:14.464 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:14.456+0000 7f785b43d740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x564dd2bfdea0 con 0x564dd28b71b0 2024-09-21T03:56:14.464 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:14.456+0000 7f785b43d740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x564dd2bed2d0 con 0x564dd28e0070 2024-09-21T03:56:14.464 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:14.456+0000 7f785b43d740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x564dd2bf10e0 con 0x564dd28bf390 2024-09-21T03:56:14.464 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:14.456+0000 7f784e59c640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564dd28b71b0 0x564dd2c87be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:56:14.464 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:14.456+0000 7f784e59c640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564dd28b71b0 0x564dd2c87be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:59808/0 (socket says 172.21.15.39:59808) 2024-09-21T03:56:14.464 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:14.456+0000 7f784e59c640 1 -- 172.21.15.39:0/732976068 learned_addr learned my addr 172.21.15.39:0/732976068 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:56:14.465 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:14.456+0000 7f784ed9d640 1 --2- 172.21.15.39:0/732976068 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564dd28e0070 0x564dd2c876a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:56:14.465 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:14.456+0000 7f784dd9b640 1 --2- 172.21.15.39:0/732976068 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564dd28bf390 0x564dd2c88120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:56:14.465 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:14.457+0000 7f784e59c640 1 -- 172.21.15.39:0/732976068 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564dd28bf390 msgr2=0x564dd2c88120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:56:14.465 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:14.457+0000 7f784e59c640 1 --2- 172.21.15.39:0/732976068 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564dd28bf390 0x564dd2c88120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:14.465 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:14.457+0000 7f784e59c640 1 -- 172.21.15.39:0/732976068 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564dd28e0070 msgr2=0x564dd2c876a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:56:14.465 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:14.457+0000 7f784e59c640 1 --2- 172.21.15.39:0/732976068 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564dd28e0070 0x564dd2c876a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:14.465 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:14.457+0000 7f784e59c640 1 -- 172.21.15.39:0/732976068 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564dd294b650 con 0x564dd28b71b0 2024-09-21T03:56:14.465 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:14.457+0000 7f784dd9b640 1 --2- 172.21.15.39:0/732976068 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564dd28bf390 0x564dd2c88120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:56:14.465 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:14.457+0000 7f784ed9d640 1 --2- 172.21.15.39:0/732976068 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564dd28e0070 0x564dd2c876a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:56:14.465 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:14.457+0000 7f783f7fe640 1 --2- 172.21.15.39:0/3062462118 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f78380b6680 0x7f78380b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:56:14.465 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:14.457+0000 7f784e59c640 1 --2- 172.21.15.39:0/732976068 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564dd28b71b0 0x564dd2c87be0 secure :-1 s=READY pgs=1299 cs=0 l=1 rev1=1 crypto rx=0x7f7840009a70 tx=0x7f7840009f30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=67cff4b3ec568ac3 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:56:14.465 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:14.457+0000 7f784ed9d640 1 --2- 172.21.15.39:0/3062462118 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f78380b6680 0x7f78380b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:56:14.465 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:14.457+0000 7f783f7fe640 1 -- 172.21.15.39:0/3062462118 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8572+0+0 (secure 0 0 0) 0x7f78440e74b0 con 0x7f78501083c0 2024-09-21T03:56:14.465 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:14.457+0000 7f783cff9640 1 -- 172.21.15.39:0/732976068 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7840017070 con 0x564dd28b71b0 2024-09-21T03:56:14.466 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:14.457+0000 7f783cff9640 1 -- 172.21.15.39:0/732976068 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7840013470 con 0x564dd28b71b0 2024-09-21T03:56:14.466 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:14.457+0000 7f783cff9640 1 -- 172.21.15.39:0/732976068 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7840012630 con 0x564dd28b71b0 2024-09-21T03:56:14.466 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:14.458+0000 7f785b43d740 1 -- 172.21.15.39:0/732976068 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564dd28b71b0 msgr2=0x564dd2c87be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:56:14.466 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:14.458+0000 7f784ed9d640 1 --2- 172.21.15.39:0/3062462118 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f78380b6680 0x7f78380b8b90 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f7848006010 tx=0x7f784802f3a0 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:56:14.466 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:14.458+0000 7f785b43d740 1 --2- 172.21.15.39:0/732976068 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564dd28b71b0 0x564dd2c87be0 secure :-1 s=READY pgs=1299 cs=0 l=1 rev1=1 crypto rx=0x7f7840009a70 tx=0x7f7840009f30 comp rx=0 tx=0).stop 2024-09-21T03:56:14.466 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:14.458+0000 7f785b43d740 1 -- 172.21.15.39:0/732976068 shutdown_connections 2024-09-21T03:56:14.466 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:14.458+0000 7f785b43d740 1 --2- 172.21.15.39:0/732976068 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564dd28bf390 0x564dd2c88120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:14.466 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:14.458+0000 7f785b43d740 1 --2- 172.21.15.39:0/732976068 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564dd28b71b0 0x564dd2c87be0 unknown :-1 s=CLOSED pgs=1299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:14.466 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:14.458+0000 7f785b43d740 1 --2- 172.21.15.39:0/732976068 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564dd28e0070 0x564dd2c876a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:14.466 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:14.458+0000 7f785b43d740 1 -- 172.21.15.39:0/732976068 >> 172.21.15.39:0/732976068 conn(0x564dd28d4460 msgr2=0x564dd2c85590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:56:14.466 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:14.458+0000 7f785b43d740 1 -- 172.21.15.39:0/732976068 shutdown_connections 2024-09-21T03:56:14.466 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:14.458+0000 7f785b43d740 1 -- 172.21.15.39:0/732976068 wait complete. 2024-09-21T03:56:14.466 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:14.459+0000 7f785b43d740 1 -- start start 2024-09-21T03:56:14.466 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:14.459+0000 7f785b43d740 1 Processor -- start 2024-09-21T03:56:14.467 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:14.459+0000 7f785b43d740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564dd28b71b0 0x564dd2c92d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:56:14.467 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:14.459+0000 7f785b43d740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564dd28bf390 0x564dd2c932b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:56:14.467 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:14.459+0000 7f785b43d740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564dd28e0070 0x564dd2c9f1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:56:14.467 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:14.459+0000 7f785b43d740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x564dd2a4e700 con 0x564dd28bf390 2024-09-21T03:56:14.467 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:14.459+0000 7f785b43d740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x564dd2b88160 con 0x564dd28b71b0 2024-09-21T03:56:14.467 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:14.459+0000 7f785b43d740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x564dd2c95e80 con 0x564dd28e0070 2024-09-21T03:56:14.467 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:14.459+0000 7f785b43d740 10 client.0 _ll_register_callbacks cb 0x564dd2ba19e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:56:14.467 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:14.459+0000 7f784dd9b640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564dd28bf390 0x564dd2c932b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:56:14.467 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:14.459+0000 7f784dd9b640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564dd28bf390 0x564dd2c932b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:59824/0 (socket says 172.21.15.39:59824) 2024-09-21T03:56:14.467 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:14.459+0000 7f784dd9b640 1 -- 172.21.15.39:0/3148931217 learned_addr learned my addr 172.21.15.39:0/3148931217 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:56:14.467 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:14.460+0000 7f784ed9d640 1 --2- 172.21.15.39:0/3148931217 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564dd28e0070 0x564dd2c9f1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:56:14.467 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:14.460+0000 7f784e59c640 1 --2- 172.21.15.39:0/3148931217 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564dd28b71b0 0x564dd2c92d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:56:14.467 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:14.460+0000 7f784dd9b640 1 -- 172.21.15.39:0/3148931217 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564dd28e0070 msgr2=0x564dd2c9f1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:56:14.467 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:14.460+0000 7f784dd9b640 1 --2- 172.21.15.39:0/3148931217 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564dd28e0070 0x564dd2c9f1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:14.467 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:14.460+0000 7f784dd9b640 1 -- 172.21.15.39:0/3148931217 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564dd28b71b0 msgr2=0x564dd2c92d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:56:14.468 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:14.460+0000 7f784dd9b640 1 --2- 172.21.15.39:0/3148931217 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564dd28b71b0 0x564dd2c92d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:14.468 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:14.460+0000 7f784dd9b640 1 -- 172.21.15.39:0/3148931217 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564dd2bb5c90 con 0x564dd28bf390 2024-09-21T03:56:14.468 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:14.460+0000 7f784ed9d640 1 --2- 172.21.15.39:0/3148931217 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564dd28e0070 0x564dd2c9f1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:56:14.468 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:14.460+0000 7f784e59c640 1 --2- 172.21.15.39:0/3148931217 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564dd28b71b0 0x564dd2c92d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:56:14.468 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:14.460+0000 7f784dd9b640 1 --2- 172.21.15.39:0/3148931217 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564dd28bf390 0x564dd2c932b0 secure :-1 s=READY pgs=1300 cs=0 l=1 rev1=1 crypto rx=0x7f78440aa540 tx=0x7f78440a99e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e911a002e3fd8b13 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:56:14.468 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:14.460+0000 7f7835ffb640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-09-21T03:56:14.468 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:14.460+0000 7f7835ffb640 1 -- 172.21.15.39:0/3148931217 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f784401d040 con 0x564dd28bf390 2024-09-21T03:56:14.468 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:14.460+0000 7f7835ffb640 1 -- 172.21.15.39:0/3148931217 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8572+0+0 (secure 0 0 0) 0x7f78440052a0 con 0x564dd28bf390 2024-09-21T03:56:14.468 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:14.460+0000 7f785b43d740 1 -- 172.21.15.39:0/3148931217 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7840013470 con 0x564dd28bf390 2024-09-21T03:56:14.468 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:14.460+0000 7f7835ffb640 1 -- 172.21.15.39:0/3148931217 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f78440aca40 con 0x564dd28bf390 2024-09-21T03:56:14.468 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:14.460+0000 7f785b43d740 10 client.18210 fetch_fsmap learned FSMap version 29 2024-09-21T03:56:14.468 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:14.460+0000 7f785b43d740 1 -- 172.21.15.39:0/3148931217 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x564dd2ca1b20 con 0x564dd28bf390 2024-09-21T03:56:14.468 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:14.461+0000 7f7835ffb640 1 -- 172.21.15.39:0/3148931217 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7840013470 con 0x564dd28bf390 2024-09-21T03:56:14.469 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:14.461+0000 7f7835ffb640 1 -- 172.21.15.39:0/3148931217 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f78440abfa0 con 0x564dd28bf390 2024-09-21T03:56:14.469 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:14.461+0000 7f785b43d740 10 client.18210 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T03:56:14.469 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:14.461+0000 7f785b43d740 10 client.18210 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-21T03:56:14.469 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:14.461+0000 7f785b43d740 20 client.18210 populate_metadata read hostname 'smithi039' 2024-09-21T03:56:14.469 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:14.461+0000 7f785b43d740 1 --2- 172.21.15.39:0/3148931217 >> [v2:172.21.15.161:6838/2190107270,v1:172.21.15.161:6839/2190107270] conn(0x564dd289e9f0 0x564dd2ca50d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:56:14.469 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:14.461+0000 7f785b43d740 4 client.18210 mds_command: new command op to 24667 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:56:14.469 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:14.461+0000 7f785b43d740 1 -- 172.21.15.39:0/3148931217 --> [v2:172.21.15.161:6838/2190107270,v1:172.21.15.161:6839/2190107270] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564dd2bffdf0 con 0x564dd289e9f0 2024-09-21T03:56:14.469 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:14.461+0000 7f784e59c640 1 --2- 172.21.15.39:0/3148931217 >> [v2:172.21.15.161:6838/2190107270,v1:172.21.15.161:6839/2190107270] conn(0x564dd289e9f0 0x564dd2ca50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:56:14.470 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:14.461+0000 7f7835ffb640 10 client.18210 ms_handle_connect on v2:172.21.15.161:6838/2190107270 2024-09-21T03:56:14.470 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:14.461+0000 7f784e59c640 1 --2- 172.21.15.39:0/3148931217 >> [v2:172.21.15.161:6838/2190107270,v1:172.21.15.161:6839/2190107270] conn(0x564dd289e9f0 0x564dd2ca50d0 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=a4977bb6181a9aa1 server_cookie=cbd6a119c07ec47 in_seq=0 out_seq=0 2024-09-21T03:56:14.470 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:14.462+0000 7f7835ffb640 1 -- 172.21.15.39:0/3148931217 <== mds.1 v2:172.21.15.161:6838/2190107270 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x564dd2bffdf0 con 0x564dd289e9f0 2024-09-21T03:56:14.470 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:14.462+0000 7f7835ffb640 10 client.18210 handle_command_reply: tid=0 2024-09-21T03:56:14.471 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:14.462+0000 7f785b43d740 1 client.18210 shutdown 2024-09-21T03:56:14.471 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:14.462+0000 7f785b43d740 1 -- 172.21.15.39:0/3148931217 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564dd28bf390 msgr2=0x564dd2c932b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:56:14.471 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:14.462+0000 7f785b43d740 1 --2- 172.21.15.39:0/3148931217 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564dd28bf390 0x564dd2c932b0 secure :-1 s=READY pgs=1300 cs=0 l=1 rev1=1 crypto rx=0x7f78440aa540 tx=0x7f78440a99e0 comp rx=0 tx=0).stop 2024-09-21T03:56:14.471 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:14.463+0000 7f785b43d740 1 -- 172.21.15.39:0/3148931217 shutdown_connections 2024-09-21T03:56:14.471 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:14.463+0000 7f785b43d740 1 --2- 172.21.15.39:0/3148931217 >> [v2:172.21.15.161:6838/2190107270,v1:172.21.15.161:6839/2190107270] conn(0x564dd289e9f0 0x564dd2ca50d0 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:14.471 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:14.463+0000 7f785b43d740 1 --2- 172.21.15.39:0/3148931217 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564dd28e0070 0x564dd2c9f1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:14.471 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:14.463+0000 7f785b43d740 1 --2- 172.21.15.39:0/3148931217 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564dd28bf390 0x564dd2c932b0 unknown :-1 s=CLOSED pgs=1300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:14.471 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:14.463+0000 7f785b43d740 1 --2- 172.21.15.39:0/3148931217 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564dd28b71b0 0x564dd2c92d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:14.471 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:14.463+0000 7f7835ffb640 0 client.18210 ms_handle_reset on v2:172.21.15.161:6838/2190107270 2024-09-21T03:56:14.471 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:14.463+0000 7f785b43d740 1 -- 172.21.15.39:0/3148931217 >> 172.21.15.39:0/3148931217 conn(0x564dd28d4460 msgr2=0x564dd2c8a830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:56:14.471 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:14.463+0000 7f785b43d740 1 -- 172.21.15.39:0/3148931217 shutdown_connections 2024-09-21T03:56:14.471 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:14.463+0000 7f785b43d740 1 -- 172.21.15.39:0/3148931217 wait complete. 2024-09-21T03:56:14.471 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:14.463+0000 7f785b43d740 20 client.18210 trim_cache size 0 max 16384 2024-09-21T03:56:14.479 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:14.476+0000 7f785b43d740 1 Processor -- start 2024-09-21T03:56:14.479 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:14.476+0000 7f785b43d740 1 -- start start 2024-09-21T03:56:14.480 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:14.476+0000 7f785b43d740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564dd28b71b0 0x564dd2c94c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:56:14.480 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:14.476+0000 7f785b43d740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564dd28bf390 0x564dd2c88de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:56:14.480 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:14.476+0000 7f785b43d740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564dd28e0070 0x564dd2c84bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:56:14.480 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:14.476+0000 7f785b43d740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f78501af740 con 0x564dd28b71b0 2024-09-21T03:56:14.480 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:14.476+0000 7f785b43d740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f78501b1280 con 0x564dd28e0070 2024-09-21T03:56:14.480 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:14.476+0000 7f785b43d740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f78501afe10 con 0x564dd28bf390 2024-09-21T03:56:14.480 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:14.476+0000 7f784e59c640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564dd28b71b0 0x564dd2c94c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:56:14.480 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:14.476+0000 7f784e59c640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564dd28b71b0 0x564dd2c94c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:59836/0 (socket says 172.21.15.39:59836) 2024-09-21T03:56:14.480 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:14.476+0000 7f784e59c640 1 -- 172.21.15.39:0/2700785299 learned_addr learned my addr 172.21.15.39:0/2700785299 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:56:14.480 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:14.477+0000 7f784ed9d640 1 --2- 172.21.15.39:0/2700785299 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564dd28e0070 0x564dd2c84bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:56:14.480 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:14.477+0000 7f784dd9b640 1 --2- 172.21.15.39:0/2700785299 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564dd28bf390 0x564dd2c88de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:56:14.480 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:14.477+0000 7f784e59c640 1 -- 172.21.15.39:0/2700785299 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564dd28bf390 msgr2=0x564dd2c88de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:56:14.480 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:14.477+0000 7f784e59c640 1 --2- 172.21.15.39:0/2700785299 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564dd28bf390 0x564dd2c88de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:14.480 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:14.477+0000 7f784e59c640 1 -- 172.21.15.39:0/2700785299 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564dd28e0070 msgr2=0x564dd2c84bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:56:14.481 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:14.477+0000 7f784e59c640 1 --2- 172.21.15.39:0/2700785299 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564dd28e0070 0x564dd2c84bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:14.481 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:14.477+0000 7f784e59c640 1 -- 172.21.15.39:0/2700785299 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564dd2c94030 con 0x564dd28b71b0 2024-09-21T03:56:14.481 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:14.477+0000 7f784dd9b640 1 --2- 172.21.15.39:0/2700785299 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564dd28bf390 0x564dd2c88de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:56:14.481 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:14.477+0000 7f784ed9d640 1 --2- 172.21.15.39:0/2700785299 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564dd28e0070 0x564dd2c84bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:56:14.481 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:14.477+0000 7f784e59c640 1 --2- 172.21.15.39:0/2700785299 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564dd28b71b0 0x564dd2c94c50 secure :-1 s=READY pgs=1301 cs=0 l=1 rev1=1 crypto rx=0x7f7840013ee0 tx=0x7f7840009680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=381b00cc89379d76 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:56:14.485 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:14.477+0000 7f7837fff640 1 -- 172.21.15.39:0/2700785299 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7840017040 con 0x564dd28b71b0 2024-09-21T03:56:14.485 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:14.477+0000 7f7837fff640 1 -- 172.21.15.39:0/2700785299 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7840028030 con 0x564dd28b71b0 2024-09-21T03:56:14.485 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:14.477+0000 7f7837fff640 1 -- 172.21.15.39:0/2700785299 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7840025750 con 0x564dd28b71b0 2024-09-21T03:56:14.485 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:14.478+0000 7f785b43d740 1 -- 172.21.15.39:0/2700785299 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564dd28b71b0 msgr2=0x564dd2c94c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:56:14.485 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:14.478+0000 7f785b43d740 1 --2- 172.21.15.39:0/2700785299 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564dd28b71b0 0x564dd2c94c50 secure :-1 s=READY pgs=1301 cs=0 l=1 rev1=1 crypto rx=0x7f7840013ee0 tx=0x7f7840009680 comp rx=0 tx=0).stop 2024-09-21T03:56:14.485 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:14.478+0000 7f785b43d740 1 -- 172.21.15.39:0/2700785299 shutdown_connections 2024-09-21T03:56:14.485 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:14.478+0000 7f785b43d740 1 --2- 172.21.15.39:0/2700785299 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564dd28e0070 0x564dd2c84bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:14.485 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:14.478+0000 7f785b43d740 1 --2- 172.21.15.39:0/2700785299 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564dd28bf390 0x564dd2c88de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:14.485 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:14.478+0000 7f785b43d740 1 --2- 172.21.15.39:0/2700785299 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564dd28b71b0 0x564dd2c94c50 unknown :-1 s=CLOSED pgs=1301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:14.485 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:14.478+0000 7f785b43d740 1 -- 172.21.15.39:0/2700785299 >> 172.21.15.39:0/2700785299 conn(0x564dd28d4460 msgr2=0x564dd2c886d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:56:14.485 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:14.478+0000 7f785b43d740 1 -- 172.21.15.39:0/2700785299 shutdown_connections 2024-09-21T03:56:14.485 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:14.478+0000 7f785b43d740 1 -- 172.21.15.39:0/2700785299 wait complete. 2024-09-21T03:56:14.485 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:14.478+0000 7f785b43d740 1 -- start start 2024-09-21T03:56:14.486 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:14.478+0000 7f785b43d740 1 Processor -- start 2024-09-21T03:56:14.486 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:14.479+0000 7f785b43d740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564dd28b71b0 0x564dd2c9b730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:56:14.486 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:14.479+0000 7f785b43d740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564dd28bf390 0x564dd2c96be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:56:14.486 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:14.479+0000 7f785b43d740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564dd28e0070 0x564dd2c97120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:56:14.486 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:14.479+0000 7f785b43d740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f78501b0bb0 con 0x564dd28bf390 2024-09-21T03:56:14.486 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:14.479+0000 7f785b43d740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f78501b04e0 con 0x564dd28e0070 2024-09-21T03:56:14.486 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:14.479+0000 7f785b43d740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f78501ad740 con 0x564dd28b71b0 2024-09-21T03:56:14.486 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:14.479+0000 7f785b43d740 10 client.0 _ll_register_callbacks cb 0x7f784400aef0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:56:14.486 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:14.479+0000 7f784dd9b640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564dd28bf390 0x564dd2c96be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:56:14.486 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:14.479+0000 7f784e59c640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564dd28b71b0 0x564dd2c9b730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:56:14.486 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:14.479+0000 7f784dd9b640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564dd28bf390 0x564dd2c96be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:59848/0 (socket says 172.21.15.39:59848) 2024-09-21T03:56:14.486 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:14.479+0000 7f784dd9b640 1 -- 172.21.15.39:0/1640007889 learned_addr learned my addr 172.21.15.39:0/1640007889 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:56:14.486 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:14.479+0000 7f784ed9d640 1 --2- 172.21.15.39:0/1640007889 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564dd28e0070 0x564dd2c97120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:56:14.486 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:14.479+0000 7f784dd9b640 1 -- 172.21.15.39:0/1640007889 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564dd28b71b0 msgr2=0x564dd2c9b730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:56:14.487 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:14.479+0000 7f784dd9b640 1 --2- 172.21.15.39:0/1640007889 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564dd28b71b0 0x564dd2c9b730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:14.487 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:14.479+0000 7f784dd9b640 1 -- 172.21.15.39:0/1640007889 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564dd28e0070 msgr2=0x564dd2c97120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:56:14.487 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:14.479+0000 7f784dd9b640 1 --2- 172.21.15.39:0/1640007889 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564dd28e0070 0x564dd2c97120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:14.487 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:14.479+0000 7f784dd9b640 1 -- 172.21.15.39:0/1640007889 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564dd2c976e0 con 0x564dd28bf390 2024-09-21T03:56:14.487 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:14.479+0000 7f784dd9b640 1 --2- 172.21.15.39:0/1640007889 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564dd28bf390 0x564dd2c96be0 secure :-1 s=READY pgs=1302 cs=0 l=1 rev1=1 crypto rx=0x7f78440ac3e0 tx=0x7f78440ac970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=794799d4326bf993 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:56:14.487 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:14.479+0000 7f7835ffb640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-09-21T03:56:14.487 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:14.480+0000 7f7835ffb640 1 -- 172.21.15.39:0/1640007889 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f784401d040 con 0x564dd28bf390 2024-09-21T03:56:14.487 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:14.480+0000 7f7835ffb640 1 -- 172.21.15.39:0/1640007889 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8572+0+0 (secure 0 0 0) 0x7f78440b2fc0 con 0x564dd28bf390 2024-09-21T03:56:14.487 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:14.480+0000 7f785b43d740 1 -- 172.21.15.39:0/1640007889 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7840028030 con 0x564dd28bf390 2024-09-21T03:56:14.487 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:14.480+0000 7f7835ffb640 1 -- 172.21.15.39:0/1640007889 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f78440ad0a0 con 0x564dd28bf390 2024-09-21T03:56:14.487 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:14.480+0000 7f785b43d740 10 client.18216 fetch_fsmap learned FSMap version 29 2024-09-21T03:56:14.487 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:14.480+0000 7f785b43d740 1 -- 172.21.15.39:0/1640007889 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x564dd2d0f7e0 con 0x564dd28bf390 2024-09-21T03:56:14.487 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:14.480+0000 7f7835ffb640 1 -- 172.21.15.39:0/1640007889 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7840028030 con 0x564dd28bf390 2024-09-21T03:56:14.487 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:14.480+0000 7f7835ffb640 1 -- 172.21.15.39:0/1640007889 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f7844058a60 con 0x564dd28bf390 2024-09-21T03:56:14.487 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:14.480+0000 7f785b43d740 10 client.18216 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T03:56:14.488 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:14.480+0000 7f785b43d740 10 client.18216 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-21T03:56:14.488 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:14.480+0000 7f785b43d740 20 client.18216 populate_metadata read hostname 'smithi039' 2024-09-21T03:56:14.488 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:14.480+0000 7f785b43d740 1 --2- 172.21.15.39:0/1640007889 >> [v2:172.21.15.161:6838/2190107270,v1:172.21.15.161:6839/2190107270] conn(0x564dd2873da0 0x564dd2d11f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:56:14.488 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:14.480+0000 7f785b43d740 4 client.18216 mds_command: new command op to 24667 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T03:56:14.488 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:14.480+0000 7f785b43d740 1 -- 172.21.15.39:0/1640007889 --> [v2:172.21.15.161:6838/2190107270,v1:172.21.15.161:6839/2190107270] -- command(tid 0: {"prefix": "damage ls"}) -- 0x564dd2bfb8a0 con 0x564dd2873da0 2024-09-21T03:56:14.488 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:14.480+0000 7f784e59c640 1 --2- 172.21.15.39:0/1640007889 >> [v2:172.21.15.161:6838/2190107270,v1:172.21.15.161:6839/2190107270] conn(0x564dd2873da0 0x564dd2d11f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:56:14.488 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:14.481+0000 7f7835ffb640 10 client.18216 ms_handle_connect on v2:172.21.15.161:6838/2190107270 2024-09-21T03:56:14.488 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:14.481+0000 7f784e59c640 1 --2- 172.21.15.39:0/1640007889 >> [v2:172.21.15.161:6838/2190107270,v1:172.21.15.161:6839/2190107270] conn(0x564dd2873da0 0x564dd2d11f40 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=c4bc0e9999d81f29 server_cookie=7b3e4e40820f66ce in_seq=0 out_seq=0 2024-09-21T03:56:14.488 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:14.481+0000 7f7835ffb640 1 -- 172.21.15.39:0/1640007889 <== mds.1 v2:172.21.15.161:6838/2190107270 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x564dd2bfb8a0 con 0x564dd2873da0 2024-09-21T03:56:14.488 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:14.481+0000 7f7835ffb640 10 client.18216 handle_command_reply: tid=0 2024-09-21T03:56:14.488 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:14.481+0000 7f785b43d740 1 client.18216 shutdown 2024-09-21T03:56:14.488 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:14.481+0000 7f785b43d740 1 -- 172.21.15.39:0/1640007889 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564dd28bf390 msgr2=0x564dd2c96be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:56:14.488 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:14.481+0000 7f785b43d740 1 --2- 172.21.15.39:0/1640007889 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564dd28bf390 0x564dd2c96be0 secure :-1 s=READY pgs=1302 cs=0 l=1 rev1=1 crypto rx=0x7f78440ac3e0 tx=0x7f78440ac970 comp rx=0 tx=0).stop 2024-09-21T03:56:14.488 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:14.481+0000 7f785b43d740 1 -- 172.21.15.39:0/1640007889 shutdown_connections 2024-09-21T03:56:14.488 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:14.481+0000 7f785b43d740 1 --2- 172.21.15.39:0/1640007889 >> [v2:172.21.15.161:6838/2190107270,v1:172.21.15.161:6839/2190107270] conn(0x564dd2873da0 0x564dd2d11f40 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:14.489 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:14.481+0000 7f785b43d740 1 --2- 172.21.15.39:0/1640007889 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564dd28e0070 0x564dd2c97120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:14.489 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:14.481+0000 7f7835ffb640 0 client.18216 ms_handle_reset on v2:172.21.15.161:6838/2190107270 2024-09-21T03:56:14.489 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:14.481+0000 7f785b43d740 1 --2- 172.21.15.39:0/1640007889 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564dd28bf390 0x564dd2c96be0 unknown :-1 s=CLOSED pgs=1302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:14.489 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:14.481+0000 7f785b43d740 1 --2- 172.21.15.39:0/1640007889 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564dd28b71b0 0x564dd2c9b730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:14.489 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:14.481+0000 7f785b43d740 1 -- 172.21.15.39:0/1640007889 >> 172.21.15.39:0/1640007889 conn(0x564dd28d4460 msgr2=0x564dd2c88f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:56:14.489 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:14.481+0000 7f785b43d740 1 -- 172.21.15.39:0/1640007889 shutdown_connections 2024-09-21T03:56:14.489 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:14.481+0000 7f785b43d740 1 -- 172.21.15.39:0/1640007889 wait complete. 2024-09-21T03:56:14.489 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:14.482+0000 7f785b43d740 20 client.18216 trim_cache size 0 max 16384 2024-09-21T03:56:14.489 INFO:teuthology.orchestra.run.smithi039.stdout:[] 2024-09-21T03:56:14.490 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:14.482+0000 7f7854ec9640 1 -- 172.21.15.39:0/3062462118 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f78380b6680 msgr2=0x7f78380b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:56:14.490 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:14.482+0000 7f7854ec9640 1 --2- 172.21.15.39:0/3062462118 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f78380b6680 0x7f78380b8b90 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f7848006010 tx=0x7f784802f3a0 comp rx=0 tx=0).stop 2024-09-21T03:56:14.490 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:14.482+0000 7f7854ec9640 1 -- 172.21.15.39:0/3062462118 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f78501083c0 msgr2=0x7f785019aab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:56:14.490 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:14.482+0000 7f7854ec9640 1 --2- 172.21.15.39:0/3062462118 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f78501083c0 0x7f785019aab0 secure :-1 s=READY pgs=1298 cs=0 l=1 rev1=1 crypto rx=0x7f784400c970 tx=0x7f784400cd70 comp rx=0 tx=0).stop 2024-09-21T03:56:14.490 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:14.482+0000 7f7854ec9640 1 -- 172.21.15.39:0/3062462118 shutdown_connections 2024-09-21T03:56:14.490 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:14.482+0000 7f7854ec9640 1 --2- 172.21.15.39:0/3062462118 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f78380b6680 0x7f78380b8b90 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:14.490 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:14.482+0000 7f7854ec9640 1 --2- 172.21.15.39:0/3062462118 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7850108d10 0x7f785019fd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:14.490 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:14.482+0000 7f7854ec9640 1 --2- 172.21.15.39:0/3062462118 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f78501083c0 0x7f785019aab0 unknown :-1 s=CLOSED pgs=1298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:14.490 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:14.482+0000 7f7854ec9640 1 --2- 172.21.15.39:0/3062462118 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7850102a80 0x7f785019a510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:14.491 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:14.482+0000 7f7854ec9640 1 -- 172.21.15.39:0/3062462118 >> 172.21.15.39:0/3062462118 conn(0x7f7850079740 msgr2=0x7f78501052c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:56:14.491 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:14.482+0000 7f7854ec9640 1 -- 172.21.15.39:0/3062462118 shutdown_connections 2024-09-21T03:56:14.491 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:14.482+0000 7f7854ec9640 1 -- 172.21.15.39:0/3062462118 wait complete. 2024-09-21T03:56:15.288 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 51116d5c-77ca-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 damage ls 2024-09-21T03:56:15.468 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:56:15 smithi161 ceph-mon[34533]: pgmap v638: 97 pgs: 97 active+clean; 10 MiB data, 525 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 53 KiB/s wr, 30 op/s 2024-09-21T03:56:15.474 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:56:15 smithi150 ceph-mon[34289]: pgmap v638: 97 pgs: 97 active+clean; 10 MiB data, 525 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 53 KiB/s wr, 30 op/s 2024-09-21T03:56:15.551 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/51116d5c-77ca-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:56:15.583 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:56:15 smithi039 ceph-mon[31211]: pgmap v638: 97 pgs: 97 active+clean; 10 MiB data, 525 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 53 KiB/s wr, 30 op/s 2024-09-21T03:56:16.035 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:16.032+0000 7f2c7f2de640 1 -- 172.21.15.39:0/3150075094 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2c78103b60 msgr2=0x7f2c78103f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:56:16.035 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:16.032+0000 7f2c7f2de640 1 --2- 172.21.15.39:0/3150075094 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2c78103b60 0x7f2c78103f70 secure :-1 s=READY pgs=1303 cs=0 l=1 rev1=1 crypto rx=0x7f2c74009900 tx=0x7f2c7402a410 comp rx=0 tx=0).stop 2024-09-21T03:56:16.035 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:16.032+0000 7f2c7f2de640 1 -- 172.21.15.39:0/3150075094 shutdown_connections 2024-09-21T03:56:16.035 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:16.032+0000 7f2c7f2de640 1 --2- 172.21.15.39:0/3150075094 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2c78109df0 0x7f2c78110620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:16.035 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:16.032+0000 7f2c7f2de640 1 --2- 172.21.15.39:0/3150075094 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2c781094a0 0x7f2c781098b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:16.035 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:16.032+0000 7f2c7f2de640 1 --2- 172.21.15.39:0/3150075094 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2c78103b60 0x7f2c78103f70 unknown :-1 s=CLOSED pgs=1303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:16.036 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:16.032+0000 7f2c7f2de640 1 -- 172.21.15.39:0/3150075094 >> 172.21.15.39:0/3150075094 conn(0x7f2c7807b850 msgr2=0x7f2c7807bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:56:16.036 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:16.032+0000 7f2c7f2de640 1 -- 172.21.15.39:0/3150075094 shutdown_connections 2024-09-21T03:56:16.036 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:16.032+0000 7f2c7f2de640 1 -- 172.21.15.39:0/3150075094 wait complete. 2024-09-21T03:56:16.038 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:16.033+0000 7f2c7f2de640 1 Processor -- start 2024-09-21T03:56:16.038 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:16.033+0000 7f2c7f2de640 1 -- start start 2024-09-21T03:56:16.038 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:16.033+0000 7f2c7f2de640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2c78103b60 0x7f2c7819a580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:56:16.038 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:16.033+0000 7f2c7f2de640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2c781094a0 0x7f2c7819ab20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:56:16.038 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:16.033+0000 7f2c7f2de640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2c78109df0 0x7f2c7819fdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:56:16.038 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:16.033+0000 7f2c7f2de640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f2c78113ab0 con 0x7f2c78109df0 2024-09-21T03:56:16.038 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:16.033+0000 7f2c7f2de640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f2c78113930 con 0x7f2c781094a0 2024-09-21T03:56:16.039 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:16.033+0000 7f2c7f2de640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f2c78113c30 con 0x7f2c78103b60 2024-09-21T03:56:16.039 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:16.033+0000 7f2c7d87b640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2c78109df0 0x7f2c7819fdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:56:16.039 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:16.033+0000 7f2c7d87b640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2c78109df0 0x7f2c7819fdc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:59866/0 (socket says 172.21.15.39:59866) 2024-09-21T03:56:16.039 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:16.033+0000 7f2c7c879640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2c781094a0 0x7f2c7819ab20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:56:16.039 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:16.033+0000 7f2c7d07a640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2c78103b60 0x7f2c7819a580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:56:16.039 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:16.033+0000 7f2c7d87b640 1 -- 172.21.15.39:0/634746792 learned_addr learned my addr 172.21.15.39:0/634746792 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:56:16.039 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:16.033+0000 7f2c7d87b640 1 -- 172.21.15.39:0/634746792 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2c78103b60 msgr2=0x7f2c7819a580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:56:16.039 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:16.033+0000 7f2c7d87b640 1 --2- 172.21.15.39:0/634746792 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2c78103b60 0x7f2c7819a580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:16.039 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:16.033+0000 7f2c7d87b640 1 -- 172.21.15.39:0/634746792 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2c781094a0 msgr2=0x7f2c7819ab20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:56:16.039 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:16.033+0000 7f2c7d87b640 1 --2- 172.21.15.39:0/634746792 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2c781094a0 0x7f2c7819ab20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:16.039 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:16.033+0000 7f2c7d87b640 1 -- 172.21.15.39:0/634746792 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2c781a04f0 con 0x7f2c78109df0 2024-09-21T03:56:16.039 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:16.034+0000 7f2c7d87b640 1 --2- 172.21.15.39:0/634746792 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2c78109df0 0x7f2c7819fdc0 secure :-1 s=READY pgs=1304 cs=0 l=1 rev1=1 crypto rx=0x7f2c7000eaa0 tx=0x7f2c7000ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:56:16.040 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:16.034+0000 7f2c6e7fc640 1 -- 172.21.15.39:0/634746792 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c7000cdb0 con 0x7f2c78109df0 2024-09-21T03:56:16.040 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:16.034+0000 7f2c6e7fc640 1 -- 172.21.15.39:0/634746792 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2c70005020 con 0x7f2c78109df0 2024-09-21T03:56:16.040 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:16.034+0000 7f2c6e7fc640 1 -- 172.21.15.39:0/634746792 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c700116d0 con 0x7f2c78109df0 2024-09-21T03:56:16.040 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:16.034+0000 7f2c7f2de640 1 -- 172.21.15.39:0/634746792 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2c781a07e0 con 0x7f2c78109df0 2024-09-21T03:56:16.040 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:16.034+0000 7f2c7f2de640 1 -- 172.21.15.39:0/634746792 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2c781a6f40 con 0x7f2c78109df0 2024-09-21T03:56:16.041 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:16.037+0000 7f2c85852740 1 Processor -- start 2024-09-21T03:56:16.041 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:16.037+0000 7f2c6e7fc640 1 -- 172.21.15.39:0/634746792 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 27) ==== 154643+0+0 (secure 0 0 0) 0x7f2c700051d0 con 0x7f2c78109df0 2024-09-21T03:56:16.041 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:16.037+0000 7f2c85852740 1 -- start start 2024-09-21T03:56:16.041 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:16.037+0000 7f2c85852740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563aaf22d070 0x563aaf5d42c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:56:16.042 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:16.037+0000 7f2c85852740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563aaf2041b0 0x563aaf5d4800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:56:16.042 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:16.037+0000 7f2c85852740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563aaf20c390 0x563aaf5d4d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:56:16.042 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:16.037+0000 7f2c85852740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x563aaf54aea0 con 0x563aaf22d070 2024-09-21T03:56:16.042 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:16.037+0000 7f2c85852740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x563aaf53a2d0 con 0x563aaf2041b0 2024-09-21T03:56:16.042 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:16.037+0000 7f2c85852740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x563aaf53e0e0 con 0x563aaf20c390 2024-09-21T03:56:16.042 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:16.037+0000 7f2c7c879640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563aaf22d070 0x563aaf5d42c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:56:16.042 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:16.038+0000 7f2c7c879640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563aaf22d070 0x563aaf5d42c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:59876/0 (socket says 172.21.15.39:59876) 2024-09-21T03:56:16.042 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:16.038+0000 7f2c7c879640 1 -- 172.21.15.39:0/816805451 learned_addr learned my addr 172.21.15.39:0/816805451 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:56:16.042 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:16.038+0000 7f2c7c879640 1 --2- 172.21.15.39:0/816805451 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563aaf20c390 0x563aaf5d4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:56:16.042 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:16.038+0000 7f2c7d07a640 1 --2- 172.21.15.39:0/816805451 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563aaf2041b0 0x563aaf5d4800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:56:16.042 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:16.038+0000 7f2c6e7fc640 1 --2- 172.21.15.39:0/634746792 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f2c640b6700 0x7f2c640b8c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:56:16.042 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:16.038+0000 7f2c7c879640 1 -- 172.21.15.39:0/816805451 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563aaf20c390 msgr2=0x563aaf5d4d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:56:16.042 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:16.038+0000 7f2c7c879640 1 --2- 172.21.15.39:0/816805451 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563aaf20c390 0x563aaf5d4d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:16.043 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:16.038+0000 7f2c7c879640 1 -- 172.21.15.39:0/816805451 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563aaf2041b0 msgr2=0x563aaf5d4800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:56:16.043 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:16.038+0000 7f2c7c879640 1 --2- 172.21.15.39:0/816805451 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563aaf2041b0 0x563aaf5d4800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:16.043 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:16.038+0000 7f2c7c879640 1 -- 172.21.15.39:0/816805451 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563aaf298650 con 0x563aaf22d070 2024-09-21T03:56:16.043 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:16.038+0000 7f2c7d07a640 1 --2- 172.21.15.39:0/816805451 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563aaf2041b0 0x563aaf5d4800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:56:16.043 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:16.038+0000 7f2c7d87b640 1 --2- 172.21.15.39:0/634746792 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f2c640b6700 0x7f2c640b8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:56:16.043 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:16.038+0000 7f2c6e7fc640 1 -- 172.21.15.39:0/634746792 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8572+0+0 (secure 0 0 0) 0x7f2c700e7720 con 0x7f2c78109df0 2024-09-21T03:56:16.043 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:16.038+0000 7f2c7c879640 1 --2- 172.21.15.39:0/816805451 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563aaf22d070 0x563aaf5d42c0 secure :-1 s=READY pgs=1305 cs=0 l=1 rev1=1 crypto rx=0x7f2c680124c0 tx=0x7f2c6800b820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=88b6743b623a637a server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:56:16.043 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:16.038+0000 7f2c637fe640 1 -- 172.21.15.39:0/816805451 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c68004030 con 0x563aaf22d070 2024-09-21T03:56:16.043 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:16.039+0000 7f2c637fe640 1 -- 172.21.15.39:0/816805451 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2c68005020 con 0x563aaf22d070 2024-09-21T03:56:16.043 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:16.039+0000 7f2c637fe640 1 -- 172.21.15.39:0/816805451 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c68004b20 con 0x563aaf22d070 2024-09-21T03:56:16.046 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:16.039+0000 7f2c85852740 1 -- 172.21.15.39:0/816805451 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563aaf22d070 msgr2=0x563aaf5d42c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:56:16.046 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:16.039+0000 7f2c7d87b640 1 --2- 172.21.15.39:0/634746792 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f2c640b6700 0x7f2c640b8c10 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f2c700aa860 tx=0x7f2c70000f40 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:56:16.046 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:16.039+0000 7f2c85852740 1 --2- 172.21.15.39:0/816805451 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563aaf22d070 0x563aaf5d42c0 secure :-1 s=READY pgs=1305 cs=0 l=1 rev1=1 crypto rx=0x7f2c680124c0 tx=0x7f2c6800b820 comp rx=0 tx=0).stop 2024-09-21T03:56:16.046 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:16.040+0000 7f2c85852740 1 -- 172.21.15.39:0/816805451 shutdown_connections 2024-09-21T03:56:16.046 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:16.040+0000 7f2c85852740 1 --2- 172.21.15.39:0/816805451 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563aaf20c390 0x563aaf5d4d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:16.046 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:16.040+0000 7f2c85852740 1 --2- 172.21.15.39:0/816805451 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563aaf2041b0 0x563aaf5d4800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:16.046 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:16.040+0000 7f2c85852740 1 --2- 172.21.15.39:0/816805451 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563aaf22d070 0x563aaf5d42c0 unknown :-1 s=CLOSED pgs=1305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:16.046 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:16.040+0000 7f2c85852740 1 -- 172.21.15.39:0/816805451 >> 172.21.15.39:0/816805451 conn(0x563aaf221460 msgr2=0x563aaf5d21b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:56:16.046 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:16.040+0000 7f2c85852740 1 -- 172.21.15.39:0/816805451 shutdown_connections 2024-09-21T03:56:16.046 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:16.040+0000 7f2c85852740 1 -- 172.21.15.39:0/816805451 wait complete. 2024-09-21T03:56:16.046 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:16.040+0000 7f2c85852740 1 -- start start 2024-09-21T03:56:16.047 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:16.040+0000 7f2c85852740 1 Processor -- start 2024-09-21T03:56:16.047 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:16.041+0000 7f2c85852740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563aaf2041b0 0x563aaf5df920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:56:16.047 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:16.041+0000 7f2c85852740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563aaf20c390 0x563aaf5dfe60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:56:16.047 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:16.041+0000 7f2c85852740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563aaf22d070 0x563aaf5e03e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:56:16.047 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:16.041+0000 7f2c85852740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x563aaf39b700 con 0x563aaf2041b0 2024-09-21T03:56:16.047 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:16.041+0000 7f2c85852740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x563aaf4d5160 con 0x563aaf22d070 2024-09-21T03:56:16.047 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:16.041+0000 7f2c85852740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x563aaf5e2aa0 con 0x563aaf20c390 2024-09-21T03:56:16.047 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:16.041+0000 7f2c85852740 10 client.0 _ll_register_callbacks cb 0x563aaf4ee9e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:56:16.047 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:16.041+0000 7f2c7c879640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563aaf2041b0 0x563aaf5df920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:56:16.047 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:16.041+0000 7f2c7c879640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563aaf2041b0 0x563aaf5df920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:59882/0 (socket says 172.21.15.39:59882) 2024-09-21T03:56:16.047 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:16.041+0000 7f2c7c879640 1 -- 172.21.15.39:0/3043974756 learned_addr learned my addr 172.21.15.39:0/3043974756 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:56:16.047 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:16.041+0000 7f2c7d07a640 1 --2- 172.21.15.39:0/3043974756 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563aaf20c390 0x563aaf5dfe60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:56:16.047 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:16.041+0000 7f2c7c879640 1 --2- 172.21.15.39:0/3043974756 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563aaf22d070 0x563aaf5e03e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:56:16.047 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:16.041+0000 7f2c7c879640 1 -- 172.21.15.39:0/3043974756 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563aaf20c390 msgr2=0x563aaf5dfe60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:56:16.047 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:16.041+0000 7f2c7c879640 1 --2- 172.21.15.39:0/3043974756 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563aaf20c390 0x563aaf5dfe60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:16.048 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:16.041+0000 7f2c7c879640 1 -- 172.21.15.39:0/3043974756 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563aaf22d070 msgr2=0x563aaf5e03e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:56:16.048 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:16.041+0000 7f2c7c879640 1 --2- 172.21.15.39:0/3043974756 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563aaf22d070 0x563aaf5e03e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:16.048 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:16.041+0000 7f2c7c879640 1 -- 172.21.15.39:0/3043974756 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563aaf502c90 con 0x563aaf2041b0 2024-09-21T03:56:16.048 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:16.041+0000 7f2c7c879640 1 --2- 172.21.15.39:0/3043974756 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563aaf2041b0 0x563aaf5df920 secure :-1 s=READY pgs=1306 cs=0 l=1 rev1=1 crypto rx=0x7f2c680122e0 tx=0x7f2c6800b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5247c4b282678e18 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:56:16.048 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:16.041+0000 7f2c60ff9640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-09-21T03:56:16.048 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:16.041+0000 7f2c60ff9640 1 -- 172.21.15.39:0/3043974756 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c68004cb0 con 0x563aaf2041b0 2024-09-21T03:56:16.048 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:16.042+0000 7f2c60ff9640 1 -- 172.21.15.39:0/3043974756 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8572+0+0 (secure 0 0 0) 0x7f2c68004040 con 0x563aaf2041b0 2024-09-21T03:56:16.048 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:16.042+0000 7f2c85852740 1 -- 172.21.15.39:0/3043974756 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2c68005020 con 0x563aaf2041b0 2024-09-21T03:56:16.048 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:16.042+0000 7f2c60ff9640 1 -- 172.21.15.39:0/3043974756 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f2c68004e50 con 0x563aaf2041b0 2024-09-21T03:56:16.048 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:16.042+0000 7f2c85852740 10 client.18228 fetch_fsmap learned FSMap version 29 2024-09-21T03:56:16.048 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:16.042+0000 7f2c85852740 1 -- 172.21.15.39:0/3043974756 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x563aaf5ee740 con 0x563aaf2041b0 2024-09-21T03:56:16.048 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:16.042+0000 7f2c60ff9640 1 -- 172.21.15.39:0/3043974756 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2c68005020 con 0x563aaf2041b0 2024-09-21T03:56:16.048 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:16.042+0000 7f2c60ff9640 1 -- 172.21.15.39:0/3043974756 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f2c68033020 con 0x563aaf2041b0 2024-09-21T03:56:16.048 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:16.042+0000 7f2c85852740 10 client.18228 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T03:56:16.049 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:16.042+0000 7f2c85852740 10 client.18228 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T03:56:16.049 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:16.042+0000 7f2c85852740 20 client.18228 populate_metadata read hostname 'smithi039' 2024-09-21T03:56:16.049 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:16.042+0000 7f2c85852740 1 --2- 172.21.15.39:0/3043974756 >> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] conn(0x563aaf1eb9f0 0x563aaf5f1aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:56:16.049 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:16.042+0000 7f2c85852740 4 client.18228 mds_command: new command op to 24695 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:56:16.049 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:16.042+0000 7f2c85852740 1 -- 172.21.15.39:0/3043974756 --> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563aaf54cdf0 con 0x563aaf1eb9f0 2024-09-21T03:56:16.049 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:16.043+0000 7f2c7d07a640 1 --2- 172.21.15.39:0/3043974756 >> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] conn(0x563aaf1eb9f0 0x563aaf5f1aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:56:16.049 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:16.043+0000 7f2c7d07a640 1 --2- 172.21.15.39:0/3043974756 >> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] conn(0x563aaf1eb9f0 0x563aaf5f1aa0 crc :-1 s=READY pgs=163 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a7ae5b0330fb910e server_cookie=8ad44722c1c6981 in_seq=0 out_seq=0 2024-09-21T03:56:16.049 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:16.043+0000 7f2c60ff9640 10 client.18228 ms_handle_connect on v2:172.21.15.150:6838/2949386907 2024-09-21T03:56:16.050 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:16.044+0000 7f2c60ff9640 1 -- 172.21.15.39:0/3043974756 <== mds.0 v2:172.21.15.150:6838/2949386907 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x563aaf54cdf0 con 0x563aaf1eb9f0 2024-09-21T03:56:16.050 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:16.044+0000 7f2c60ff9640 10 client.18228 handle_command_reply: tid=0 2024-09-21T03:56:16.050 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:16.044+0000 7f2c85852740 1 client.18228 shutdown 2024-09-21T03:56:16.050 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:16.044+0000 7f2c85852740 1 -- 172.21.15.39:0/3043974756 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563aaf2041b0 msgr2=0x563aaf5df920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:56:16.050 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:16.044+0000 7f2c85852740 1 --2- 172.21.15.39:0/3043974756 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563aaf2041b0 0x563aaf5df920 secure :-1 s=READY pgs=1306 cs=0 l=1 rev1=1 crypto rx=0x7f2c680122e0 tx=0x7f2c6800b8b0 comp rx=0 tx=0).stop 2024-09-21T03:56:16.050 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:16.044+0000 7f2c85852740 1 -- 172.21.15.39:0/3043974756 shutdown_connections 2024-09-21T03:56:16.050 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:16.044+0000 7f2c85852740 1 --2- 172.21.15.39:0/3043974756 >> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] conn(0x563aaf1eb9f0 0x563aaf5f1aa0 crc :-1 s=READY pgs=163 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:16.050 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:16.044+0000 7f2c85852740 1 --2- 172.21.15.39:0/3043974756 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563aaf22d070 0x563aaf5e03e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:16.051 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:16.044+0000 7f2c85852740 1 --2- 172.21.15.39:0/3043974756 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563aaf20c390 0x563aaf5dfe60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:16.051 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:16.044+0000 7f2c60ff9640 0 client.18228 ms_handle_reset on v2:172.21.15.150:6838/2949386907 2024-09-21T03:56:16.051 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:16.044+0000 7f2c85852740 1 --2- 172.21.15.39:0/3043974756 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563aaf2041b0 0x563aaf5df920 unknown :-1 s=CLOSED pgs=1306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:16.051 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:16.044+0000 7f2c85852740 1 -- 172.21.15.39:0/3043974756 >> 172.21.15.39:0/3043974756 conn(0x563aaf221460 msgr2=0x563aaf5d7470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:56:16.051 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:16.045+0000 7f2c85852740 1 -- 172.21.15.39:0/3043974756 shutdown_connections 2024-09-21T03:56:16.051 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:16.045+0000 7f2c85852740 1 -- 172.21.15.39:0/3043974756 wait complete. 2024-09-21T03:56:16.051 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:16.045+0000 7f2c85852740 20 client.18228 trim_cache size 0 max 16384 2024-09-21T03:56:16.061 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:16.058+0000 7f2c85852740 1 Processor -- start 2024-09-21T03:56:16.062 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:16.058+0000 7f2c85852740 1 -- start start 2024-09-21T03:56:16.062 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:16.058+0000 7f2c85852740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563aaf1eb9f0 0x563aaf5d5cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:56:16.062 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:16.058+0000 7f2c85852740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563aaf2041b0 0x563aaf5e1550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:56:16.062 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:16.058+0000 7f2c85852740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563aaf20c390 0x563aaf5dcb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:56:16.062 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:16.058+0000 7f2c85852740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f2c781af9a0 con 0x563aaf20c390 2024-09-21T03:56:16.062 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:16.058+0000 7f2c85852740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f2c781b14e0 con 0x563aaf1eb9f0 2024-09-21T03:56:16.062 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:16.058+0000 7f2c85852740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f2c781b0070 con 0x563aaf2041b0 2024-09-21T03:56:16.062 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:16.058+0000 7f2c7c879640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563aaf20c390 0x563aaf5dcb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:56:16.062 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:16.058+0000 7f2c7c879640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563aaf20c390 0x563aaf5dcb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:59898/0 (socket says 172.21.15.39:59898) 2024-09-21T03:56:16.062 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:16.058+0000 7f2c7c879640 1 -- 172.21.15.39:0/3757488669 learned_addr learned my addr 172.21.15.39:0/3757488669 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:56:16.062 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:16.058+0000 7f2c7c879640 1 --2- 172.21.15.39:0/3757488669 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563aaf1eb9f0 0x563aaf5d5cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:56:16.062 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:16.059+0000 7f2c7d07a640 1 --2- 172.21.15.39:0/3757488669 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563aaf2041b0 0x563aaf5e1550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:56:16.062 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:16.059+0000 7f2c7c879640 1 -- 172.21.15.39:0/3757488669 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563aaf2041b0 msgr2=0x563aaf5e1550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:56:16.063 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:16.059+0000 7f2c7c879640 1 --2- 172.21.15.39:0/3757488669 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563aaf2041b0 0x563aaf5e1550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:16.063 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:16.059+0000 7f2c7c879640 1 -- 172.21.15.39:0/3757488669 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563aaf1eb9f0 msgr2=0x563aaf5d5cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:56:16.063 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:16.059+0000 7f2c7c879640 1 --2- 172.21.15.39:0/3757488669 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563aaf1eb9f0 0x563aaf5d5cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:16.063 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:16.059+0000 7f2c7c879640 1 -- 172.21.15.39:0/3757488669 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563aaf5d67e0 con 0x563aaf20c390 2024-09-21T03:56:16.063 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:16.059+0000 7f2c7c879640 1 --2- 172.21.15.39:0/3757488669 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563aaf20c390 0x563aaf5dcb10 secure :-1 s=READY pgs=1307 cs=0 l=1 rev1=1 crypto rx=0x7f2c68013fd0 tx=0x7f2c68004830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=820606a8ab28771f server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:56:16.065 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:16.059+0000 7f2c62ffd640 1 -- 172.21.15.39:0/3757488669 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c68005780 con 0x563aaf20c390 2024-09-21T03:56:16.066 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:16.059+0000 7f2c62ffd640 1 -- 172.21.15.39:0/3757488669 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2c68022030 con 0x563aaf20c390 2024-09-21T03:56:16.066 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:16.059+0000 7f2c62ffd640 1 -- 172.21.15.39:0/3757488669 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c68003710 con 0x563aaf20c390 2024-09-21T03:56:16.066 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:16.060+0000 7f2c85852740 1 -- 172.21.15.39:0/3757488669 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563aaf20c390 msgr2=0x563aaf5dcb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:56:16.066 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:16.060+0000 7f2c85852740 1 --2- 172.21.15.39:0/3757488669 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563aaf20c390 0x563aaf5dcb10 secure :-1 s=READY pgs=1307 cs=0 l=1 rev1=1 crypto rx=0x7f2c68013fd0 tx=0x7f2c68004830 comp rx=0 tx=0).stop 2024-09-21T03:56:16.066 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:16.060+0000 7f2c85852740 1 -- 172.21.15.39:0/3757488669 shutdown_connections 2024-09-21T03:56:16.066 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:16.060+0000 7f2c85852740 1 --2- 172.21.15.39:0/3757488669 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563aaf20c390 0x563aaf5dcb10 unknown :-1 s=CLOSED pgs=1307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:16.066 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:16.060+0000 7f2c85852740 1 --2- 172.21.15.39:0/3757488669 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563aaf2041b0 0x563aaf5e1550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:16.066 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:16.060+0000 7f2c85852740 1 --2- 172.21.15.39:0/3757488669 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563aaf1eb9f0 0x563aaf5d5cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:16.066 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:16.060+0000 7f2c85852740 1 -- 172.21.15.39:0/3757488669 >> 172.21.15.39:0/3757488669 conn(0x563aaf221460 msgr2=0x563aaf5e0e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:56:16.066 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:16.060+0000 7f2c85852740 1 -- 172.21.15.39:0/3757488669 shutdown_connections 2024-09-21T03:56:16.066 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:16.060+0000 7f2c85852740 1 -- 172.21.15.39:0/3757488669 wait complete. 2024-09-21T03:56:16.066 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:16.060+0000 7f2c85852740 1 -- start start 2024-09-21T03:56:16.067 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:16.060+0000 7f2c85852740 1 Processor -- start 2024-09-21T03:56:16.067 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:16.061+0000 7f2c85852740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563aaf1eb9f0 0x563aaf5e5850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:56:16.067 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:16.061+0000 7f2c85852740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563aaf2041b0 0x563aaf5e5e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:56:16.067 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:16.061+0000 7f2c85852740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563aaf20c390 0x563aaf653330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:56:16.067 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:16.061+0000 7f2c85852740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f2c781b0e10 con 0x563aaf2041b0 2024-09-21T03:56:16.067 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:16.061+0000 7f2c85852740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f2c781b0740 con 0x563aaf1eb9f0 2024-09-21T03:56:16.067 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:16.061+0000 7f2c85852740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f2c781ad9a0 con 0x563aaf20c390 2024-09-21T03:56:16.067 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:16.061+0000 7f2c85852740 10 client.0 _ll_register_callbacks cb 0x7f2c680123f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:56:16.067 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:16.061+0000 7f2c7d07a640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563aaf2041b0 0x563aaf5e5e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:56:16.067 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:16.061+0000 7f2c7d07a640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563aaf2041b0 0x563aaf5e5e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:59902/0 (socket says 172.21.15.39:59902) 2024-09-21T03:56:16.067 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:16.061+0000 7f2c7c879640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563aaf1eb9f0 0x563aaf5e5850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:56:16.067 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:16.061+0000 7f2c7d07a640 1 -- 172.21.15.39:0/728188045 learned_addr learned my addr 172.21.15.39:0/728188045 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:56:16.067 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:16.061+0000 7f2c7c879640 1 --2- 172.21.15.39:0/728188045 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563aaf20c390 0x563aaf653330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:56:16.067 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:16.061+0000 7f2c7d07a640 1 -- 172.21.15.39:0/728188045 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563aaf20c390 msgr2=0x563aaf653330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:56:16.067 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:16.061+0000 7f2c7d07a640 1 --2- 172.21.15.39:0/728188045 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563aaf20c390 0x563aaf653330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:16.068 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:16.061+0000 7f2c7d07a640 1 -- 172.21.15.39:0/728188045 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563aaf1eb9f0 msgr2=0x563aaf5e5850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:56:16.068 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:16.061+0000 7f2c7d07a640 1 --2- 172.21.15.39:0/728188045 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563aaf1eb9f0 0x563aaf5e5850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:16.068 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:16.061+0000 7f2c7d07a640 1 -- 172.21.15.39:0/728188045 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563aaf6539a0 con 0x563aaf2041b0 2024-09-21T03:56:16.068 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:16.061+0000 7f2c7c879640 1 --2- 172.21.15.39:0/728188045 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563aaf20c390 0x563aaf653330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:56:16.068 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:16.062+0000 7f2c7d07a640 1 --2- 172.21.15.39:0/728188045 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563aaf2041b0 0x563aaf5e5e30 secure :-1 s=READY pgs=1308 cs=0 l=1 rev1=1 crypto rx=0x7f2c7402a930 tx=0x7f2c74032f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f7a8cdb0f82d4f5 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:56:16.068 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:16.062+0000 7f2c60ff9640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-09-21T03:56:16.068 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:16.062+0000 7f2c60ff9640 1 -- 172.21.15.39:0/728188045 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c74035040 con 0x563aaf2041b0 2024-09-21T03:56:16.068 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:16.062+0000 7f2c60ff9640 1 -- 172.21.15.39:0/728188045 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8572+0+0 (secure 0 0 0) 0x7f2c74005040 con 0x563aaf2041b0 2024-09-21T03:56:16.068 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:16.062+0000 7f2c85852740 1 -- 172.21.15.39:0/728188045 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2c68022030 con 0x563aaf2041b0 2024-09-21T03:56:16.068 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:16.062+0000 7f2c60ff9640 1 -- 172.21.15.39:0/728188045 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f2c74005240 con 0x563aaf2041b0 2024-09-21T03:56:16.068 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:16.062+0000 7f2c85852740 10 client.18234 fetch_fsmap learned FSMap version 29 2024-09-21T03:56:16.068 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:16.062+0000 7f2c85852740 1 -- 172.21.15.39:0/728188045 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x563aaf65bf30 con 0x563aaf2041b0 2024-09-21T03:56:16.068 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:16.062+0000 7f2c60ff9640 1 -- 172.21.15.39:0/728188045 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2c68022030 con 0x563aaf2041b0 2024-09-21T03:56:16.068 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:16.062+0000 7f2c60ff9640 1 -- 172.21.15.39:0/728188045 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f2c74003660 con 0x563aaf2041b0 2024-09-21T03:56:16.069 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:16.063+0000 7f2c85852740 10 client.18234 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T03:56:16.069 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:16.063+0000 7f2c85852740 10 client.18234 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T03:56:16.069 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:16.063+0000 7f2c85852740 20 client.18234 populate_metadata read hostname 'smithi039' 2024-09-21T03:56:16.069 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:16.063+0000 7f2c85852740 1 --2- 172.21.15.39:0/728188045 >> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] conn(0x563aaf22d070 0x563aaf65e6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:56:16.069 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:16.063+0000 7f2c85852740 4 client.18234 mds_command: new command op to 24695 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T03:56:16.069 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:16.063+0000 7f2c85852740 1 -- 172.21.15.39:0/728188045 --> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] -- command(tid 0: {"prefix": "damage ls"}) -- 0x563aaf5488a0 con 0x563aaf22d070 2024-09-21T03:56:16.069 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:16.063+0000 7f2c7c879640 1 --2- 172.21.15.39:0/728188045 >> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] conn(0x563aaf22d070 0x563aaf65e6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:56:16.070 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:16.063+0000 7f2c7c879640 1 --2- 172.21.15.39:0/728188045 >> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] conn(0x563aaf22d070 0x563aaf65e6d0 crc :-1 s=READY pgs=164 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4ae3f6594cf61424 server_cookie=6943d9d0a6145bfd in_seq=0 out_seq=0 2024-09-21T03:56:16.070 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:16.063+0000 7f2c60ff9640 10 client.18234 ms_handle_connect on v2:172.21.15.150:6838/2949386907 2024-09-21T03:56:16.070 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:16.064+0000 7f2c60ff9640 1 -- 172.21.15.39:0/728188045 <== mds.0 v2:172.21.15.150:6838/2949386907 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x563aaf5488a0 con 0x563aaf22d070 2024-09-21T03:56:16.070 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:16.064+0000 7f2c60ff9640 10 client.18234 handle_command_reply: tid=0 2024-09-21T03:56:16.071 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:16.064+0000 7f2c85852740 1 client.18234 shutdown 2024-09-21T03:56:16.071 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:16.064+0000 7f2c85852740 1 -- 172.21.15.39:0/728188045 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563aaf2041b0 msgr2=0x563aaf5e5e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:56:16.071 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:16.064+0000 7f2c85852740 1 --2- 172.21.15.39:0/728188045 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563aaf2041b0 0x563aaf5e5e30 secure :-1 s=READY pgs=1308 cs=0 l=1 rev1=1 crypto rx=0x7f2c7402a930 tx=0x7f2c74032f40 comp rx=0 tx=0).stop 2024-09-21T03:56:16.071 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:16.064+0000 7f2c85852740 1 -- 172.21.15.39:0/728188045 shutdown_connections 2024-09-21T03:56:16.071 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:16.064+0000 7f2c85852740 1 --2- 172.21.15.39:0/728188045 >> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] conn(0x563aaf22d070 0x563aaf65e6d0 crc :-1 s=READY pgs=164 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:16.071 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:16.064+0000 7f2c85852740 1 --2- 172.21.15.39:0/728188045 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563aaf20c390 0x563aaf653330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:16.071 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:16.064+0000 7f2c85852740 1 --2- 172.21.15.39:0/728188045 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563aaf2041b0 0x563aaf5e5e30 unknown :-1 s=CLOSED pgs=1308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:16.071 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:16.064+0000 7f2c60ff9640 0 client.18234 ms_handle_reset on v2:172.21.15.150:6838/2949386907 2024-09-21T03:56:16.071 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:16.064+0000 7f2c85852740 1 --2- 172.21.15.39:0/728188045 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563aaf1eb9f0 0x563aaf5e5850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:16.071 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:16.064+0000 7f2c85852740 1 -- 172.21.15.39:0/728188045 >> 172.21.15.39:0/728188045 conn(0x563aaf221460 msgr2=0x563aaf5d7b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:56:16.071 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:16.064+0000 7f2c85852740 1 -- 172.21.15.39:0/728188045 shutdown_connections 2024-09-21T03:56:16.071 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:16.064+0000 7f2c85852740 1 -- 172.21.15.39:0/728188045 wait complete. 2024-09-21T03:56:16.071 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:16.064+0000 7f2c85852740 20 client.18234 trim_cache size 0 max 16384 2024-09-21T03:56:16.072 INFO:teuthology.orchestra.run.smithi039.stdout:[] 2024-09-21T03:56:16.072 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:16.065+0000 7f2c7f2de640 1 -- 172.21.15.39:0/634746792 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f2c640b6700 msgr2=0x7f2c640b8c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:56:16.072 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:16.065+0000 7f2c7f2de640 1 --2- 172.21.15.39:0/634746792 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f2c640b6700 0x7f2c640b8c10 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f2c700aa860 tx=0x7f2c70000f40 comp rx=0 tx=0).stop 2024-09-21T03:56:16.072 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:16.065+0000 7f2c7f2de640 1 -- 172.21.15.39:0/634746792 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2c78109df0 msgr2=0x7f2c7819fdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:56:16.073 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:16.065+0000 7f2c7f2de640 1 --2- 172.21.15.39:0/634746792 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2c78109df0 0x7f2c7819fdc0 secure :-1 s=READY pgs=1304 cs=0 l=1 rev1=1 crypto rx=0x7f2c7000eaa0 tx=0x7f2c7000ef60 comp rx=0 tx=0).stop 2024-09-21T03:56:16.073 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:16.065+0000 7f2c7f2de640 1 -- 172.21.15.39:0/634746792 shutdown_connections 2024-09-21T03:56:16.073 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:16.065+0000 7f2c7f2de640 1 --2- 172.21.15.39:0/634746792 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f2c640b6700 0x7f2c640b8c10 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:16.073 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:16.065+0000 7f2c7f2de640 1 --2- 172.21.15.39:0/634746792 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2c78109df0 0x7f2c7819fdc0 unknown :-1 s=CLOSED pgs=1304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:16.073 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:16.065+0000 7f2c7f2de640 1 --2- 172.21.15.39:0/634746792 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2c781094a0 0x7f2c7819ab20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:16.073 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:16.065+0000 7f2c7f2de640 1 --2- 172.21.15.39:0/634746792 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2c78103b60 0x7f2c7819a580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:16.073 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:16.065+0000 7f2c7f2de640 1 -- 172.21.15.39:0/634746792 >> 172.21.15.39:0/634746792 conn(0x7f2c7807b850 msgr2=0x7f2c78107640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:56:16.073 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:16.065+0000 7f2c7f2de640 1 -- 172.21.15.39:0/634746792 shutdown_connections 2024-09-21T03:56:16.073 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:16.065+0000 7f2c7f2de640 1 -- 172.21.15.39:0/634746792 wait complete. 2024-09-21T03:56:17.468 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:56:17 smithi161 ceph-mon[34533]: pgmap v639: 97 pgs: 97 active+clean; 10 MiB data, 525 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 204 B/s wr, 10 op/s 2024-09-21T03:56:17.475 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:56:17 smithi150 ceph-mon[34289]: pgmap v639: 97 pgs: 97 active+clean; 10 MiB data, 525 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 204 B/s wr, 10 op/s 2024-09-21T03:56:17.583 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:56:17 smithi039 ceph-mon[31211]: pgmap v639: 97 pgs: 97 active+clean; 10 MiB data, 525 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 204 B/s wr, 10 op/s 2024-09-21T03:56:17.887 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-21T03:56:17.887 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 51116d5c-77ca-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-21T03:56:18.158 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/51116d5c-77ca-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:56:18.492 DEBUG:teuthology.orchestra.run.smithi039:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule status --fs=cephfs --path=/' 2024-09-21T03:56:18.608 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:18.605+0000 7f6285d6e640 1 -- 172.21.15.39:0/386764142 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f62801029b0 msgr2=0x7f6280102dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:56:18.608 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:18.605+0000 7f6285d6e640 1 --2- 172.21.15.39:0/386764142 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f62801029b0 0x7f6280102dc0 secure :-1 s=READY pgs=1309 cs=0 l=1 rev1=1 crypto rx=0x7f6268009900 tx=0x7f626802a410 comp rx=0 tx=0).stop 2024-09-21T03:56:18.608 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:18.605+0000 7f6285d6e640 1 -- 172.21.15.39:0/386764142 shutdown_connections 2024-09-21T03:56:18.608 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:18.605+0000 7f6285d6e640 1 --2- 172.21.15.39:0/386764142 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6280108c30 0x7f628010f460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:18.608 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:18.605+0000 7f6285d6e640 1 --2- 172.21.15.39:0/386764142 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f62801082e0 0x7f62801086f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:18.608 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:18.605+0000 7f6285d6e640 1 --2- 172.21.15.39:0/386764142 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f62801029b0 0x7f6280102dc0 unknown :-1 s=CLOSED pgs=1309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:18.608 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:18.605+0000 7f6285d6e640 1 -- 172.21.15.39:0/386764142 >> 172.21.15.39:0/386764142 conn(0x7f628007a740 msgr2=0x7f628007ab50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:56:18.608 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:18.605+0000 7f6285d6e640 1 -- 172.21.15.39:0/386764142 shutdown_connections 2024-09-21T03:56:18.609 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:18.605+0000 7f6285d6e640 1 -- 172.21.15.39:0/386764142 wait complete. 2024-09-21T03:56:18.609 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:18.605+0000 7f6285d6e640 1 Processor -- start 2024-09-21T03:56:18.611 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:18.606+0000 7f6285d6e640 1 -- start start 2024-09-21T03:56:18.611 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:18.606+0000 7f6285d6e640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f62801029b0 0x7f62800ff550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:56:18.611 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:18.606+0000 7f6285d6e640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f62801082e0 0x7f62800ffa90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:56:18.611 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:18.606+0000 7f6285d6e640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6280108c30 0x7f62800fffd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:56:18.611 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:18.606+0000 7f6285d6e640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f62801128f0 con 0x7f62801029b0 2024-09-21T03:56:18.611 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:18.606+0000 7f6285d6e640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f6280112770 con 0x7f62801082e0 2024-09-21T03:56:18.611 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:18.606+0000 7f6285d6e640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f6280112a70 con 0x7f6280108c30 2024-09-21T03:56:18.611 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:18.606+0000 7f627f7fe640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f62801029b0 0x7f62800ff550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:56:18.611 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:18.606+0000 7f627f7fe640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f62801029b0 0x7f62800ff550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:55638/0 (socket says 172.21.15.39:55638) 2024-09-21T03:56:18.611 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:18.606+0000 7f627f7fe640 1 -- 172.21.15.39:0/3475025709 learned_addr learned my addr 172.21.15.39:0/3475025709 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:56:18.611 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:18.606+0000 7f627ffff640 1 --2- 172.21.15.39:0/3475025709 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6280108c30 0x7f62800fffd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:56:18.611 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:18.606+0000 7f627effd640 1 --2- 172.21.15.39:0/3475025709 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f62801082e0 0x7f62800ffa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:56:18.612 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:18.607+0000 7f627f7fe640 1 -- 172.21.15.39:0/3475025709 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6280108c30 msgr2=0x7f62800fffd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:56:18.612 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:18.607+0000 7f627f7fe640 1 --2- 172.21.15.39:0/3475025709 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6280108c30 0x7f62800fffd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:18.612 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:18.607+0000 7f627f7fe640 1 -- 172.21.15.39:0/3475025709 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f62801082e0 msgr2=0x7f62800ffa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:56:18.612 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:18.607+0000 7f627f7fe640 1 --2- 172.21.15.39:0/3475025709 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f62801082e0 0x7f62800ffa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:18.612 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:18.607+0000 7f627f7fe640 1 -- 172.21.15.39:0/3475025709 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f62801a86f0 con 0x7f62801029b0 2024-09-21T03:56:18.612 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:18.607+0000 7f627effd640 1 --2- 172.21.15.39:0/3475025709 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f62801082e0 0x7f62800ffa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:56:18.612 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:18.607+0000 7f627f7fe640 1 --2- 172.21.15.39:0/3475025709 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f62801029b0 0x7f62800ff550 secure :-1 s=READY pgs=1310 cs=0 l=1 rev1=1 crypto rx=0x7f6268005940 tx=0x7f6268004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:56:18.612 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:18.607+0000 7f627cff9640 1 -- 172.21.15.39:0/3475025709 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f626803f030 con 0x7f62801029b0 2024-09-21T03:56:18.612 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:18.607+0000 7f627cff9640 1 -- 172.21.15.39:0/3475025709 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6268003aa0 con 0x7f62801029b0 2024-09-21T03:56:18.612 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:18.607+0000 7f6285d6e640 1 -- 172.21.15.39:0/3475025709 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f62801a8980 con 0x7f62801029b0 2024-09-21T03:56:18.612 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:18.607+0000 7f627cff9640 1 -- 172.21.15.39:0/3475025709 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6268035070 con 0x7f62801029b0 2024-09-21T03:56:18.612 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:18.607+0000 7f6285d6e640 1 -- 172.21.15.39:0/3475025709 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f62801a8e70 con 0x7f62801029b0 2024-09-21T03:56:18.613 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:18.608+0000 7f6285d6e640 1 -- 172.21.15.39:0/3475025709 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6280102dc0 con 0x7f62801029b0 2024-09-21T03:56:18.613 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:18.610+0000 7f627cff9640 1 -- 172.21.15.39:0/3475025709 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 27) ==== 154643+0+0 (secure 0 0 0) 0x7f6268042030 con 0x7f62801029b0 2024-09-21T03:56:18.614 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:18.611+0000 7f627cff9640 1 --2- 172.21.15.39:0/3475025709 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f626c0b66b0 0x7f626c0b8bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:56:18.614 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:18.611+0000 7f627effd640 1 --2- 172.21.15.39:0/3475025709 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f626c0b66b0 0x7f626c0b8bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:56:18.614 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:18.611+0000 7f627cff9640 1 -- 172.21.15.39:0/3475025709 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8572+0+0 (secure 0 0 0) 0x7f6268108120 con 0x7f62801029b0 2024-09-21T03:56:18.615 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:18.612+0000 7f627effd640 1 --2- 172.21.15.39:0/3475025709 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f626c0b66b0 0x7f626c0b8bc0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f6270005d00 tx=0x7f6270005c70 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:56:18.617 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:18.615+0000 7f627cff9640 1 -- 172.21.15.39:0/3475025709 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f62680ce640 con 0x7f62801029b0 2024-09-21T03:56:18.683 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:18.678+0000 7f22428d0640 1 -- 172.21.15.39:0/958609999 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f223c1049c0 msgr2=0x7f223c104dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:56:18.684 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:18.678+0000 7f22428d0640 1 --2- 172.21.15.39:0/958609999 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f223c1049c0 0x7f223c104dd0 secure :-1 s=READY pgs=1311 cs=0 l=1 rev1=1 crypto rx=0x7f222c009900 tx=0x7f222c02a410 comp rx=0 tx=0).stop 2024-09-21T03:56:18.684 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:18.678+0000 7f22428d0640 1 -- 172.21.15.39:0/958609999 shutdown_connections 2024-09-21T03:56:18.684 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:18.678+0000 7f22428d0640 1 --2- 172.21.15.39:0/958609999 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f223c10de20 0x7f223c110310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:18.684 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:18.678+0000 7f22428d0640 1 --2- 172.21.15.39:0/958609999 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f223c105310 0x7f223c10d860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:18.684 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:18.678+0000 7f22428d0640 1 --2- 172.21.15.39:0/958609999 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f223c1049c0 0x7f223c104dd0 unknown :-1 s=CLOSED pgs=1311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:18.684 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:18.678+0000 7f22428d0640 1 -- 172.21.15.39:0/958609999 >> 172.21.15.39:0/958609999 conn(0x7f223c079730 msgr2=0x7f223c079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:56:18.684 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:18.678+0000 7f22428d0640 1 -- 172.21.15.39:0/958609999 shutdown_connections 2024-09-21T03:56:18.684 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:18.678+0000 7f22428d0640 1 -- 172.21.15.39:0/958609999 wait complete. 2024-09-21T03:56:18.684 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:18.679+0000 7f22428d0640 1 Processor -- start 2024-09-21T03:56:18.684 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:18.679+0000 7f22428d0640 1 -- start start 2024-09-21T03:56:18.684 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:18.679+0000 7f22428d0640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f223c1049c0 0x7f223c19a590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:56:18.684 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:18.679+0000 7f22428d0640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f223c105310 0x7f223c19ab30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:56:18.684 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:18.679+0000 7f22428d0640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f223c10de20 0x7f223c19fdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:56:18.685 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:18.680+0000 7f22428d0640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f223c113aa0 con 0x7f223c10de20 2024-09-21T03:56:18.685 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:18.680+0000 7f22428d0640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f223c113920 con 0x7f223c1049c0 2024-09-21T03:56:18.685 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:18.680+0000 7f22428d0640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f223c113c20 con 0x7f223c105310 2024-09-21T03:56:18.685 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:18.680+0000 7f2240e6d640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f223c10de20 0x7f223c19fdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:56:18.685 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:18.680+0000 7f2240e6d640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f223c10de20 0x7f223c19fdd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:55652/0 (socket says 172.21.15.39:55652) 2024-09-21T03:56:18.685 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:18.680+0000 7f2240e6d640 1 -- 172.21.15.39:0/1235449429 learned_addr learned my addr 172.21.15.39:0/1235449429 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:56:18.685 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:18.680+0000 7f223bfff640 1 --2- 172.21.15.39:0/1235449429 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f223c1049c0 0x7f223c19a590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:56:18.685 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:18.680+0000 7f2240e6d640 1 -- 172.21.15.39:0/1235449429 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f223c105310 msgr2=0x7f223c19ab30 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-21T03:56:18.685 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:18.680+0000 7f2240e6d640 1 --2- 172.21.15.39:0/1235449429 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f223c105310 0x7f223c19ab30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:18.685 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:18.680+0000 7f2240e6d640 1 -- 172.21.15.39:0/1235449429 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f223c1049c0 msgr2=0x7f223c19a590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:56:18.685 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:18.680+0000 7f2240e6d640 1 --2- 172.21.15.39:0/1235449429 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f223c1049c0 0x7f223c19a590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:18.685 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:18.680+0000 7f2240e6d640 1 -- 172.21.15.39:0/1235449429 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f223c1a0500 con 0x7f223c10de20 2024-09-21T03:56:18.685 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:18.680+0000 7f2240e6d640 1 --2- 172.21.15.39:0/1235449429 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f223c10de20 0x7f223c19fdd0 secure :-1 s=READY pgs=1312 cs=0 l=1 rev1=1 crypto rx=0x7f2234009fd0 tx=0x7f223400b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:56:18.685 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:18.681+0000 7f22397fa640 1 -- 172.21.15.39:0/1235449429 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2234017070 con 0x7f223c10de20 2024-09-21T03:56:18.686 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:18.681+0000 7f22397fa640 1 -- 172.21.15.39:0/1235449429 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2234003e20 con 0x7f223c10de20 2024-09-21T03:56:18.686 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:18.681+0000 7f22428d0640 1 -- 172.21.15.39:0/1235449429 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f223c1a07f0 con 0x7f223c10de20 2024-09-21T03:56:18.686 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:18.681+0000 7f22397fa640 1 -- 172.21.15.39:0/1235449429 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2234010460 con 0x7f223c10de20 2024-09-21T03:56:18.686 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:18.681+0000 7f22428d0640 1 -- 172.21.15.39:0/1235449429 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f223c1a6f50 con 0x7f223c10de20 2024-09-21T03:56:18.686 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:18.684+0000 7f2248e44740 1 Processor -- start 2024-09-21T03:56:18.687 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:18.684+0000 7f22397fa640 1 -- 172.21.15.39:0/1235449429 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 27) ==== 154643+0+0 (secure 0 0 0) 0x7f2234003970 con 0x7f223c10de20 2024-09-21T03:56:18.687 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:18.684+0000 7f2248e44740 1 -- start start 2024-09-21T03:56:18.688 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:18.685+0000 7f2248e44740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558af5e4c350 0x558af61f3430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:56:18.688 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:18.685+0000 7f2248e44740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558af5e23490 0x558af61f3970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:56:18.688 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:18.685+0000 7f2248e44740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558af5e2b670 0x558af61f3eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:56:18.688 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:18.685+0000 7f2248e44740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x558af6169e60 con 0x558af5e2b670 2024-09-21T03:56:18.688 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:18.685+0000 7f2248e44740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x558af6159290 con 0x558af5e23490 2024-09-21T03:56:18.688 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:18.685+0000 7f2248e44740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x558af615d0a0 con 0x558af5e4c350 2024-09-21T03:56:18.688 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:18.685+0000 7f223b7fe640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558af5e2b670 0x558af61f3eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:56:18.688 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:18.685+0000 7f223b7fe640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558af5e2b670 0x558af61f3eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:55668/0 (socket says 172.21.15.39:55668) 2024-09-21T03:56:18.688 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:18.685+0000 7f223b7fe640 1 -- 172.21.15.39:0/3011688054 learned_addr learned my addr 172.21.15.39:0/3011688054 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:56:18.688 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:18.685+0000 7f223b7fe640 1 --2- 172.21.15.39:0/3011688054 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558af5e4c350 0x558af61f3430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:56:18.688 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:18.685+0000 7f223bfff640 1 --2- 172.21.15.39:0/3011688054 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558af5e23490 0x558af61f3970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:56:18.690 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:18.685+0000 7f22397fa640 1 --2- 172.21.15.39:0/1235449429 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f22240b6680 0x7f22240b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:56:18.690 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:18.685+0000 7f223b7fe640 1 -- 172.21.15.39:0/3011688054 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558af5e4c350 msgr2=0x558af61f3430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:56:18.690 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:18.685+0000 7f223b7fe640 1 --2- 172.21.15.39:0/3011688054 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558af5e4c350 0x558af61f3430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:18.690 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:18.685+0000 7f223b7fe640 1 -- 172.21.15.39:0/3011688054 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558af5e23490 msgr2=0x558af61f3970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:56:18.690 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:18.685+0000 7f223b7fe640 1 --2- 172.21.15.39:0/3011688054 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558af5e23490 0x558af61f3970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:18.690 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:18.685+0000 7f223b7fe640 1 -- 172.21.15.39:0/3011688054 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558af5eb7930 con 0x558af5e2b670 2024-09-21T03:56:18.690 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:18.685+0000 7f2240e6d640 1 --2- 172.21.15.39:0/1235449429 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f22240b6680 0x7f22240b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:56:18.690 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:18.686+0000 7f22397fa640 1 -- 172.21.15.39:0/1235449429 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8572+0+0 (secure 0 0 0) 0x7f22340e7460 con 0x7f223c10de20 2024-09-21T03:56:18.690 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:18.686+0000 7f223b7fe640 1 --2- 172.21.15.39:0/3011688054 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558af5e2b670 0x558af61f3eb0 secure :-1 s=READY pgs=1313 cs=0 l=1 rev1=1 crypto rx=0x7f2230009fd0 tx=0x7f2230009660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fbd2d47fadb2c172 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:56:18.691 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:18.686+0000 7f22227fc640 1 -- 172.21.15.39:0/3011688054 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2230013110 con 0x558af5e2b670 2024-09-21T03:56:18.691 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:18.686+0000 7f22227fc640 1 -- 172.21.15.39:0/3011688054 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f22300132b0 con 0x558af5e2b670 2024-09-21T03:56:18.691 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:18.686+0000 7f22227fc640 1 -- 172.21.15.39:0/3011688054 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22300047a0 con 0x558af5e2b670 2024-09-21T03:56:18.691 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:18.687+0000 7f2248e44740 1 -- 172.21.15.39:0/3011688054 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558af5e2b670 msgr2=0x558af61f3eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:56:18.691 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:18.687+0000 7f2248e44740 1 --2- 172.21.15.39:0/3011688054 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558af5e2b670 0x558af61f3eb0 secure :-1 s=READY pgs=1313 cs=0 l=1 rev1=1 crypto rx=0x7f2230009fd0 tx=0x7f2230009660 comp rx=0 tx=0).stop 2024-09-21T03:56:18.691 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:18.687+0000 7f2240e6d640 1 --2- 172.21.15.39:0/1235449429 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f22240b6680 0x7f22240b8b90 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f22340e4c30 tx=0x7f2234079f30 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:56:18.691 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:18.687+0000 7f2248e44740 1 -- 172.21.15.39:0/3011688054 shutdown_connections 2024-09-21T03:56:18.691 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:18.687+0000 7f2248e44740 1 --2- 172.21.15.39:0/3011688054 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558af5e2b670 0x558af61f3eb0 unknown :-1 s=CLOSED pgs=1313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:18.691 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:18.687+0000 7f2248e44740 1 --2- 172.21.15.39:0/3011688054 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558af5e23490 0x558af61f3970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:18.691 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:18.687+0000 7f2248e44740 1 --2- 172.21.15.39:0/3011688054 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558af5e4c350 0x558af61f3430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:18.691 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:18.687+0000 7f2248e44740 1 -- 172.21.15.39:0/3011688054 >> 172.21.15.39:0/3011688054 conn(0x558af5e40740 msgr2=0x558af61f1320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:56:18.691 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:18.687+0000 7f2248e44740 1 -- 172.21.15.39:0/3011688054 shutdown_connections 2024-09-21T03:56:18.691 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:18.687+0000 7f2248e44740 1 -- 172.21.15.39:0/3011688054 wait complete. 2024-09-21T03:56:18.694 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:18.688+0000 7f2248e44740 1 -- start start 2024-09-21T03:56:18.694 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:18.688+0000 7f2248e44740 1 Processor -- start 2024-09-21T03:56:18.694 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:18.688+0000 7f2248e44740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558af5e23490 0x558af61ff030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:56:18.694 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:18.688+0000 7f2248e44740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558af5e2b670 0x558af61ff5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:56:18.694 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:18.689+0000 7f2248e44740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558af5e4c350 0x558af620af00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:56:18.694 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:18.689+0000 7f2248e44740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x558af5fba890 con 0x558af5e23490 2024-09-21T03:56:18.694 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:18.689+0000 7f2248e44740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x558af60f3d40 con 0x558af5e4c350 2024-09-21T03:56:18.694 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:18.689+0000 7f2248e44740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x558af6201c10 con 0x558af5e2b670 2024-09-21T03:56:18.694 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:18.689+0000 7f2248e44740 10 client.0 _ll_register_callbacks cb 0x558af6157c80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:56:18.694 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:18.689+0000 7f223b7fe640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558af5e23490 0x558af61ff030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:56:18.695 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:18.689+0000 7f223b7fe640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558af5e23490 0x558af61ff030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:55678/0 (socket says 172.21.15.39:55678) 2024-09-21T03:56:18.695 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:18.689+0000 7f223b7fe640 1 -- 172.21.15.39:0/2800141458 learned_addr learned my addr 172.21.15.39:0/2800141458 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:56:18.695 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:18.689+0000 7f223bfff640 1 --2- 172.21.15.39:0/2800141458 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558af5e2b670 0x558af61ff5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:56:18.695 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:18.689+0000 7f223b7fe640 1 -- 172.21.15.39:0/2800141458 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558af5e2b670 msgr2=0x558af61ff5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:56:18.695 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:18.689+0000 7f223b7fe640 1 --2- 172.21.15.39:0/2800141458 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558af5e2b670 0x558af61ff5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:18.695 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:18.689+0000 7f223b7fe640 1 -- 172.21.15.39:0/2800141458 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558af5e4c350 msgr2=0x558af620af00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:56:18.695 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:18.689+0000 7f223b7fe640 1 --2- 172.21.15.39:0/2800141458 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558af5e4c350 0x558af620af00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:18.695 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:18.689+0000 7f223b7fe640 1 -- 172.21.15.39:0/2800141458 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558af620b510 con 0x558af5e23490 2024-09-21T03:56:18.695 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:18.689+0000 7f223b7fe640 1 --2- 172.21.15.39:0/2800141458 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558af5e23490 0x558af61ff030 secure :-1 s=READY pgs=1314 cs=0 l=1 rev1=1 crypto rx=0x7f2230005890 tx=0x7f2230009280 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a922b24ea54870df server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:56:18.695 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:18.689+0000 7f221b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-09-21T03:56:18.695 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:18.689+0000 7f221b7fe640 1 -- 172.21.15.39:0/2800141458 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2230013110 con 0x558af5e23490 2024-09-21T03:56:18.695 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:18.690+0000 7f221b7fe640 1 -- 172.21.15.39:0/2800141458 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8572+0+0 (secure 0 0 0) 0x7f2230004430 con 0x558af5e23490 2024-09-21T03:56:18.695 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:18.690+0000 7f2248e44740 1 -- 172.21.15.39:0/2800141458 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f22300132b0 con 0x558af5e23490 2024-09-21T03:56:18.695 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:18.690+0000 7f221b7fe640 1 -- 172.21.15.39:0/2800141458 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f223000e750 con 0x558af5e23490 2024-09-21T03:56:18.695 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:18.690+0000 7f2248e44740 10 client.18252 fetch_fsmap learned FSMap version 29 2024-09-21T03:56:18.696 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:18.690+0000 7f2248e44740 1 -- 172.21.15.39:0/2800141458 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x558af620de40 con 0x558af5e23490 2024-09-21T03:56:18.696 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:18.690+0000 7f221b7fe640 1 -- 172.21.15.39:0/2800141458 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f22300132b0 con 0x558af5e23490 2024-09-21T03:56:18.696 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:18.690+0000 7f221b7fe640 1 -- 172.21.15.39:0/2800141458 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f2230004c10 con 0x558af5e23490 2024-09-21T03:56:18.696 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:18.690+0000 7f2248e44740 10 client.18252 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T03:56:18.696 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:18.690+0000 7f2248e44740 10 client.18252 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T03:56:18.696 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:18.690+0000 7f2248e44740 20 client.18252 populate_metadata read hostname 'smithi039' 2024-09-21T03:56:18.696 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:18.690+0000 7f2248e44740 1 --2- 172.21.15.39:0/2800141458 >> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] conn(0x558af5e0acd0 0x558af6211200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:56:18.696 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:18.690+0000 7f2248e44740 4 client.18252 mds_command: new command op to 24695 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:56:18.696 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:18.690+0000 7f2248e44740 1 -- 172.21.15.39:0/2800141458 --> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558af616bdb0 con 0x558af5e0acd0 2024-09-21T03:56:18.696 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:18.691+0000 7f223bfff640 1 --2- 172.21.15.39:0/2800141458 >> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] conn(0x558af5e0acd0 0x558af6211200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:56:18.696 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:18.691+0000 7f223bfff640 1 --2- 172.21.15.39:0/2800141458 >> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] conn(0x558af5e0acd0 0x558af6211200 crc :-1 s=READY pgs=165 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=560ec8ce71091610 server_cookie=363c585f8eab2509 in_seq=0 out_seq=0 2024-09-21T03:56:18.696 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:18.691+0000 7f221b7fe640 10 client.18252 ms_handle_connect on v2:172.21.15.150:6838/2949386907 2024-09-21T03:56:18.697 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:18.692+0000 7f221b7fe640 1 -- 172.21.15.39:0/2800141458 <== mds.0 v2:172.21.15.150:6838/2949386907 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x558af616bdb0 con 0x558af5e0acd0 2024-09-21T03:56:18.697 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:18.692+0000 7f221b7fe640 10 client.18252 handle_command_reply: tid=0 2024-09-21T03:56:18.698 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:18.692+0000 7f2248e44740 1 client.18252 shutdown 2024-09-21T03:56:18.698 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:18.692+0000 7f2248e44740 1 -- 172.21.15.39:0/2800141458 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558af5e23490 msgr2=0x558af61ff030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:56:18.698 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:18.692+0000 7f2248e44740 1 --2- 172.21.15.39:0/2800141458 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558af5e23490 0x558af61ff030 secure :-1 s=READY pgs=1314 cs=0 l=1 rev1=1 crypto rx=0x7f2230005890 tx=0x7f2230009280 comp rx=0 tx=0).stop 2024-09-21T03:56:18.698 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:18.692+0000 7f2248e44740 1 -- 172.21.15.39:0/2800141458 shutdown_connections 2024-09-21T03:56:18.698 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:18.692+0000 7f2248e44740 1 --2- 172.21.15.39:0/2800141458 >> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] conn(0x558af5e0acd0 0x558af6211200 crc :-1 s=READY pgs=165 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:18.698 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:18.692+0000 7f2248e44740 1 --2- 172.21.15.39:0/2800141458 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558af5e4c350 0x558af620af00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:18.698 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:18.692+0000 7f2248e44740 1 --2- 172.21.15.39:0/2800141458 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558af5e2b670 0x558af61ff5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:18.698 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:18.692+0000 7f2248e44740 1 --2- 172.21.15.39:0/2800141458 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558af5e23490 0x558af61ff030 unknown :-1 s=CLOSED pgs=1314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:18.698 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:18.692+0000 7f2248e44740 1 -- 172.21.15.39:0/2800141458 >> 172.21.15.39:0/2800141458 conn(0x558af5e40740 msgr2=0x558af61f65c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:56:18.698 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:18.692+0000 7f221b7fe640 0 client.18252 ms_handle_reset on v2:172.21.15.150:6838/2949386907 2024-09-21T03:56:18.698 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:18.692+0000 7f2248e44740 1 -- 172.21.15.39:0/2800141458 shutdown_connections 2024-09-21T03:56:18.698 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:18.692+0000 7f2248e44740 1 -- 172.21.15.39:0/2800141458 wait complete. 2024-09-21T03:56:18.698 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:18.692+0000 7f2248e44740 20 client.18252 trim_cache size 0 max 16384 2024-09-21T03:56:18.713 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:18.710+0000 7f2248e44740 1 Processor -- start 2024-09-21T03:56:18.713 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:18.710+0000 7f2248e44740 1 -- start start 2024-09-21T03:56:18.713 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:18.710+0000 7f2248e44740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558af5e23490 0x558af61f6ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:56:18.713 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:18.710+0000 7f2248e44740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558af5e2b670 0x558af61fe590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:56:18.714 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:18.710+0000 7f2248e44740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558af5e4c350 0x558af6200a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:56:18.714 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:18.711+0000 7f2248e44740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f223c1af900 con 0x558af5e23490 2024-09-21T03:56:18.714 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:18.711+0000 7f2248e44740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f223c1b1440 con 0x558af5e2b670 2024-09-21T03:56:18.714 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:18.711+0000 7f2248e44740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f223c1affd0 con 0x558af5e4c350 2024-09-21T03:56:18.714 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:18.711+0000 7f223b7fe640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558af5e23490 0x558af61f6ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:56:18.714 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:18.711+0000 7f223b7fe640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558af5e23490 0x558af61f6ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:55684/0 (socket says 172.21.15.39:55684) 2024-09-21T03:56:18.714 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:18.711+0000 7f223b7fe640 1 -- 172.21.15.39:0/2603980717 learned_addr learned my addr 172.21.15.39:0/2603980717 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:56:18.715 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:18.711+0000 7f223b7fe640 1 --2- 172.21.15.39:0/2603980717 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558af5e4c350 0x558af6200a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:56:18.715 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:18.711+0000 7f223bfff640 1 --2- 172.21.15.39:0/2603980717 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558af5e2b670 0x558af61fe590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:56:18.715 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:18.711+0000 7f223b7fe640 1 -- 172.21.15.39:0/2603980717 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558af5e4c350 msgr2=0x558af6200a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:56:18.715 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:18.711+0000 7f223b7fe640 1 --2- 172.21.15.39:0/2603980717 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558af5e4c350 0x558af6200a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:18.715 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:18.711+0000 7f223b7fe640 1 -- 172.21.15.39:0/2603980717 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558af5e2b670 msgr2=0x558af61fe590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:56:18.715 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:18.711+0000 7f223b7fe640 1 --2- 172.21.15.39:0/2603980717 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558af5e2b670 0x558af61fe590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:18.715 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:18.711+0000 7f223b7fe640 1 -- 172.21.15.39:0/2603980717 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558af61f5720 con 0x558af5e23490 2024-09-21T03:56:18.715 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:18.711+0000 7f223b7fe640 1 --2- 172.21.15.39:0/2603980717 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558af5e23490 0x558af61f6ba0 secure :-1 s=READY pgs=1315 cs=0 l=1 rev1=1 crypto rx=0x7f2230003a90 tx=0x7f2230005b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=47ed372badce8573 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:56:18.715 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:18.711+0000 7f2221ffb640 1 -- 172.21.15.39:0/2603980717 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2230003bb0 con 0x558af5e23490 2024-09-21T03:56:18.715 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:18.711+0000 7f2221ffb640 1 -- 172.21.15.39:0/2603980717 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2230003d50 con 0x558af5e23490 2024-09-21T03:56:18.715 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:18.711+0000 7f2221ffb640 1 -- 172.21.15.39:0/2603980717 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f223001e450 con 0x558af5e23490 2024-09-21T03:56:18.716 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:18.713+0000 7f2248e44740 1 -- 172.21.15.39:0/2603980717 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558af5e23490 msgr2=0x558af61f6ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:56:18.716 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:18.713+0000 7f2248e44740 1 --2- 172.21.15.39:0/2603980717 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558af5e23490 0x558af61f6ba0 secure :-1 s=READY pgs=1315 cs=0 l=1 rev1=1 crypto rx=0x7f2230003a90 tx=0x7f2230005b90 comp rx=0 tx=0).stop 2024-09-21T03:56:18.716 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:18.713+0000 7f2248e44740 1 -- 172.21.15.39:0/2603980717 shutdown_connections 2024-09-21T03:56:18.716 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:18.713+0000 7f2248e44740 1 --2- 172.21.15.39:0/2603980717 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558af5e4c350 0x558af6200a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:18.717 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:18.713+0000 7f2248e44740 1 --2- 172.21.15.39:0/2603980717 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558af5e2b670 0x558af61fe590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:18.717 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:18.713+0000 7f2248e44740 1 --2- 172.21.15.39:0/2603980717 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558af5e23490 0x558af61f6ba0 unknown :-1 s=CLOSED pgs=1315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:18.717 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:18.713+0000 7f2248e44740 1 -- 172.21.15.39:0/2603980717 >> 172.21.15.39:0/2603980717 conn(0x558af5e40740 msgr2=0x558af6200350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:56:18.717 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:18.713+0000 7f2248e44740 1 -- 172.21.15.39:0/2603980717 shutdown_connections 2024-09-21T03:56:18.717 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:18.713+0000 7f2248e44740 1 -- 172.21.15.39:0/2603980717 wait complete. 2024-09-21T03:56:18.717 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:18.713+0000 7f2248e44740 1 -- start start 2024-09-21T03:56:18.717 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:18.714+0000 7f2248e44740 1 Processor -- start 2024-09-21T03:56:18.718 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:18.714+0000 7f2248e44740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558af5e23490 0x558af6207580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:56:18.718 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:18.714+0000 7f2248e44740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558af5e2b670 0x558af6202ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:56:18.718 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:18.714+0000 7f2248e44740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558af5e4c350 0x558af6203010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:56:18.718 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:18.714+0000 7f2248e44740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f223c1b0d70 con 0x558af5e23490 2024-09-21T03:56:18.719 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:18.714+0000 7f2248e44740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f223c1b06a0 con 0x558af5e4c350 2024-09-21T03:56:18.719 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:18.714+0000 7f2248e44740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f223c1ad900 con 0x558af5e2b670 2024-09-21T03:56:18.719 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:18.714+0000 7f2248e44740 10 client.0 _ll_register_callbacks cb 0x7f222c009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:56:18.719 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:18.714+0000 7f223b7fe640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558af5e23490 0x558af6207580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:56:18.719 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:18.714+0000 7f223b7fe640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558af5e23490 0x558af6207580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:55694/0 (socket says 172.21.15.39:55694) 2024-09-21T03:56:18.719 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:18.714+0000 7f223b7fe640 1 -- 172.21.15.39:0/4035424726 learned_addr learned my addr 172.21.15.39:0/4035424726 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:56:18.719 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:18.714+0000 7f223bfff640 1 --2- 172.21.15.39:0/4035424726 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558af5e2b670 0x558af6202ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:56:18.719 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:18.715+0000 7f223b7fe640 1 -- 172.21.15.39:0/4035424726 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558af5e2b670 msgr2=0x558af6202ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:56:18.719 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:18.715+0000 7f223b7fe640 1 --2- 172.21.15.39:0/4035424726 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558af5e2b670 0x558af6202ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:18.719 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:18.715+0000 7f223b7fe640 1 -- 172.21.15.39:0/4035424726 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558af5e4c350 msgr2=0x558af6203010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:56:18.719 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:18.715+0000 7f223b7fe640 1 --2- 172.21.15.39:0/4035424726 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558af5e4c350 0x558af6203010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:18.719 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:18.715+0000 7f223b7fe640 1 -- 172.21.15.39:0/4035424726 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558af62036d0 con 0x558af5e23490 2024-09-21T03:56:18.719 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:18.715+0000 7f223b7fe640 1 --2- 172.21.15.39:0/4035424726 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558af5e23490 0x558af6207580 secure :-1 s=READY pgs=1316 cs=0 l=1 rev1=1 crypto rx=0x7f223000a010 tx=0x7f2230003690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c9d4edb3cd32efe7 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:56:18.719 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:18.715+0000 7f221b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-09-21T03:56:18.720 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:18.715+0000 7f221b7fe640 1 -- 172.21.15.39:0/4035424726 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f223000eaf0 con 0x558af5e23490 2024-09-21T03:56:18.720 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:18.715+0000 7f221b7fe640 1 -- 172.21.15.39:0/4035424726 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8572+0+0 (secure 0 0 0) 0x7f223000ec90 con 0x558af5e23490 2024-09-21T03:56:18.720 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:18.715+0000 7f2248e44740 1 -- 172.21.15.39:0/4035424726 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2230003d50 con 0x558af5e23490 2024-09-21T03:56:18.720 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:18.715+0000 7f221b7fe640 1 -- 172.21.15.39:0/4035424726 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f22300133e0 con 0x558af5e23490 2024-09-21T03:56:18.720 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:18.715+0000 7f2248e44740 10 client.18258 fetch_fsmap learned FSMap version 29 2024-09-21T03:56:18.720 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:18.715+0000 7f2248e44740 1 -- 172.21.15.39:0/4035424726 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x558af627b910 con 0x558af5e23490 2024-09-21T03:56:18.720 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:18.716+0000 7f221b7fe640 1 -- 172.21.15.39:0/4035424726 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2230003d50 con 0x558af5e23490 2024-09-21T03:56:18.721 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:18.716+0000 7f221b7fe640 1 -- 172.21.15.39:0/4035424726 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f2230033070 con 0x558af5e23490 2024-09-21T03:56:18.721 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:18.716+0000 7f2248e44740 10 client.18258 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T03:56:18.721 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:18.716+0000 7f2248e44740 10 client.18258 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T03:56:18.721 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:18.716+0000 7f2248e44740 20 client.18258 populate_metadata read hostname 'smithi039' 2024-09-21T03:56:18.721 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:18.716+0000 7f2248e44740 1 --2- 172.21.15.39:0/4035424726 >> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] conn(0x558af5de0080 0x558af627de00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:56:18.721 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:18.716+0000 7f2248e44740 4 client.18258 mds_command: new command op to 24695 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-21T03:56:18.721 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:18.716+0000 7f2248e44740 1 -- 172.21.15.39:0/4035424726 --> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x558af6167860 con 0x558af5de0080 2024-09-21T03:56:18.721 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:18.717+0000 7f223bfff640 1 --2- 172.21.15.39:0/4035424726 >> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] conn(0x558af5de0080 0x558af627de00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:56:18.721 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:18.718+0000 7f223bfff640 1 --2- 172.21.15.39:0/4035424726 >> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] conn(0x558af5de0080 0x558af627de00 crc :-1 s=READY pgs=166 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=28494799a4d2eb7e server_cookie=ca7e55922e51d266 in_seq=0 out_seq=0 2024-09-21T03:56:18.721 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:18.718+0000 7f221b7fe640 10 client.18258 ms_handle_connect on v2:172.21.15.150:6838/2949386907 2024-09-21T03:56:18.723 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:18.720+0000 7f221b7fe640 1 -- 172.21.15.39:0/4035424726 <== mds.0 v2:172.21.15.150:6838/2949386907 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x558af6167860 con 0x558af5de0080 2024-09-21T03:56:18.723 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:18.720+0000 7f221b7fe640 10 client.18258 handle_command_reply: tid=0 2024-09-21T03:56:18.723 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:18.720+0000 7f2248e44740 1 client.18258 shutdown 2024-09-21T03:56:18.724 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:18.720+0000 7f2248e44740 1 -- 172.21.15.39:0/4035424726 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558af5e23490 msgr2=0x558af6207580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:56:18.724 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:18.721+0000 7f2248e44740 1 --2- 172.21.15.39:0/4035424726 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558af5e23490 0x558af6207580 secure :-1 s=READY pgs=1316 cs=0 l=1 rev1=1 crypto rx=0x7f223000a010 tx=0x7f2230003690 comp rx=0 tx=0).stop 2024-09-21T03:56:18.724 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:18.721+0000 7f2248e44740 1 -- 172.21.15.39:0/4035424726 shutdown_connections 2024-09-21T03:56:18.724 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:18.721+0000 7f2248e44740 1 --2- 172.21.15.39:0/4035424726 >> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] conn(0x558af5de0080 0x558af627de00 crc :-1 s=READY pgs=166 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:18.724 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:18.721+0000 7f2248e44740 1 --2- 172.21.15.39:0/4035424726 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558af5e4c350 0x558af6203010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:18.724 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:18.721+0000 7f2248e44740 1 --2- 172.21.15.39:0/4035424726 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558af5e2b670 0x558af6202ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:18.724 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:18.721+0000 7f2248e44740 1 --2- 172.21.15.39:0/4035424726 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558af5e23490 0x558af6207580 unknown :-1 s=CLOSED pgs=1316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:18.724 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:18.721+0000 7f2248e44740 1 -- 172.21.15.39:0/4035424726 >> 172.21.15.39:0/4035424726 conn(0x558af5e40740 msgr2=0x558af6200ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:56:18.724 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:18.721+0000 7f221b7fe640 0 client.18258 ms_handle_reset on v2:172.21.15.150:6838/2949386907 2024-09-21T03:56:18.724 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:18.721+0000 7f2248e44740 1 -- 172.21.15.39:0/4035424726 shutdown_connections 2024-09-21T03:56:18.724 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:18.721+0000 7f2248e44740 1 -- 172.21.15.39:0/4035424726 wait complete. 2024-09-21T03:56:18.724 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:18.721+0000 7f2248e44740 20 client.18258 trim_cache size 0 max 16384 2024-09-21T03:56:18.725 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-09-21T03:56:18.725 INFO:teuthology.orchestra.run.smithi039.stdout: "return_code": 0, 2024-09-21T03:56:18.725 INFO:teuthology.orchestra.run.smithi039.stdout: "scrub_tag": "db151ef6-34f3-4a6d-aa7a-19d07002ce84", 2024-09-21T03:56:18.725 INFO:teuthology.orchestra.run.smithi039.stdout: "mode": "asynchronous" 2024-09-21T03:56:18.725 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-09-21T03:56:18.726 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:18.722+0000 7f22428d0640 1 -- 172.21.15.39:0/1235449429 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f22240b6680 msgr2=0x7f22240b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:56:18.726 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:18.722+0000 7f22428d0640 1 --2- 172.21.15.39:0/1235449429 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f22240b6680 0x7f22240b8b90 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f22340e4c30 tx=0x7f2234079f30 comp rx=0 tx=0).stop 2024-09-21T03:56:18.726 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:18.722+0000 7f22428d0640 1 -- 172.21.15.39:0/1235449429 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f223c10de20 msgr2=0x7f223c19fdd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:56:18.726 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:18.722+0000 7f22428d0640 1 --2- 172.21.15.39:0/1235449429 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f223c10de20 0x7f223c19fdd0 secure :-1 s=READY pgs=1312 cs=0 l=1 rev1=1 crypto rx=0x7f2234009fd0 tx=0x7f223400b260 comp rx=0 tx=0).stop 2024-09-21T03:56:18.726 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:18.722+0000 7f22428d0640 1 -- 172.21.15.39:0/1235449429 shutdown_connections 2024-09-21T03:56:18.726 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:18.722+0000 7f22428d0640 1 --2- 172.21.15.39:0/1235449429 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f22240b6680 0x7f22240b8b90 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:18.726 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:18.722+0000 7f22428d0640 1 --2- 172.21.15.39:0/1235449429 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f223c10de20 0x7f223c19fdd0 unknown :-1 s=CLOSED pgs=1312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:18.726 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:18.722+0000 7f22428d0640 1 --2- 172.21.15.39:0/1235449429 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f223c105310 0x7f223c19ab30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:18.726 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:18.722+0000 7f22428d0640 1 --2- 172.21.15.39:0/1235449429 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f223c1049c0 0x7f223c19a590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:18.726 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:18.722+0000 7f22428d0640 1 -- 172.21.15.39:0/1235449429 >> 172.21.15.39:0/1235449429 conn(0x7f223c079730 msgr2=0x7f223c100910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:56:18.726 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:18.722+0000 7f22428d0640 1 -- 172.21.15.39:0/1235449429 shutdown_connections 2024-09-21T03:56:18.726 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:18.722+0000 7f22428d0640 1 -- 172.21.15.39:0/1235449429 wait complete. 2024-09-21T03:56:18.851 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:18.848+0000 7f6285d6e640 1 -- 172.21.15.39:0/3475025709 --> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}) -- 0x7f6280063800 con 0x7f626c0b66b0 2024-09-21T03:56:18.859 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:18.856+0000 7f627cff9640 1 -- 172.21.15.39:0/3475025709 <== mgr.14223 v2:172.21.15.39:6800/1696381982 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+350 (secure 0 0 0) 0x7f6280063800 con 0x7f626c0b66b0 2024-09-21T03:56:18.860 INFO:teuthology.orchestra.run.smithi039.stdout:{"fs": "cephfs", "subvol": null, "group": null, "path": "/", "rel_path": "/", "schedule": "1m", "retention": {"m": 6, "h": 3}, "start": "2024-09-21T00:00:00", "created": "2024-09-21T03:46:17", "first": "2024-09-21T03:47:00", "last": "2024-09-21T03:56:00", "last_pruned": "2024-09-21T03:56:00", "created_count": 10, "pruned_count": 4, "active": true} 2024-09-21T03:56:18.862 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:18.859+0000 7f6285d6e640 1 -- 172.21.15.39:0/3475025709 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f626c0b66b0 msgr2=0x7f626c0b8bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:56:18.862 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:18.859+0000 7f6285d6e640 1 --2- 172.21.15.39:0/3475025709 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f626c0b66b0 0x7f626c0b8bc0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f6270005d00 tx=0x7f6270005c70 comp rx=0 tx=0).stop 2024-09-21T03:56:18.862 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:18.859+0000 7f6285d6e640 1 -- 172.21.15.39:0/3475025709 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f62801029b0 msgr2=0x7f62800ff550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:56:18.862 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:18.859+0000 7f6285d6e640 1 --2- 172.21.15.39:0/3475025709 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f62801029b0 0x7f62800ff550 secure :-1 s=READY pgs=1310 cs=0 l=1 rev1=1 crypto rx=0x7f6268005940 tx=0x7f6268004270 comp rx=0 tx=0).stop 2024-09-21T03:56:18.862 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:18.859+0000 7f6285d6e640 1 -- 172.21.15.39:0/3475025709 shutdown_connections 2024-09-21T03:56:18.862 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:18.859+0000 7f6285d6e640 1 --2- 172.21.15.39:0/3475025709 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f626c0b66b0 0x7f626c0b8bc0 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:18.862 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:18.859+0000 7f6285d6e640 1 --2- 172.21.15.39:0/3475025709 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6280108c30 0x7f62800fffd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:18.862 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:18.859+0000 7f6285d6e640 1 --2- 172.21.15.39:0/3475025709 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f62801082e0 0x7f62800ffa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:18.863 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:18.859+0000 7f6285d6e640 1 --2- 172.21.15.39:0/3475025709 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f62801029b0 0x7f62800ff550 unknown :-1 s=CLOSED pgs=1310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:18.863 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:18.859+0000 7f6285d6e640 1 -- 172.21.15.39:0/3475025709 >> 172.21.15.39:0/3475025709 conn(0x7f628007a740 msgr2=0x7f62801019d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:56:18.863 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:18.859+0000 7f6285d6e640 1 -- 172.21.15.39:0/3475025709 shutdown_connections 2024-09-21T03:56:18.863 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:18.859+0000 7f6285d6e640 1 -- 172.21.15.39:0/3475025709 wait complete. 2024-09-21T03:56:18.873 DEBUG:teuthology.orchestra.run.smithi039:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true' 2024-09-21T03:56:18.989 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:18.986+0000 7f9dc6b28640 1 -- 172.21.15.39:0/2382781530 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9dc0109d50 msgr2=0x7f9dc0110580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:56:18.989 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:18.986+0000 7f9dc6b28640 1 --2- 172.21.15.39:0/2382781530 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9dc0109d50 0x7f9dc0110580 secure :-1 s=READY pgs=1317 cs=0 l=1 rev1=1 crypto rx=0x7f9db8009ca0 tx=0x7f9db802a8c0 comp rx=0 tx=0).stop 2024-09-21T03:56:18.989 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:18.986+0000 7f9dc6b28640 1 -- 172.21.15.39:0/2382781530 shutdown_connections 2024-09-21T03:56:18.990 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:18.986+0000 7f9dc6b28640 1 --2- 172.21.15.39:0/2382781530 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9dc0109d50 0x7f9dc0110580 unknown :-1 s=CLOSED pgs=1317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:18.990 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:18.986+0000 7f9dc6b28640 1 --2- 172.21.15.39:0/2382781530 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9dc0109400 0x7f9dc0109810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:18.990 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:18.986+0000 7f9dc6b28640 1 --2- 172.21.15.39:0/2382781530 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9dc0103ad0 0x7f9dc0103ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:18.990 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:18.986+0000 7f9dc6b28640 1 -- 172.21.15.39:0/2382781530 >> 172.21.15.39:0/2382781530 conn(0x7f9dc00ff510 msgr2=0x7f9dc0101960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:56:18.990 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:18.986+0000 7f9dc6b28640 1 -- 172.21.15.39:0/2382781530 shutdown_connections 2024-09-21T03:56:18.990 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:18.986+0000 7f9dc6b28640 1 -- 172.21.15.39:0/2382781530 wait complete. 2024-09-21T03:56:18.992 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:18.987+0000 7f9dc6b28640 1 Processor -- start 2024-09-21T03:56:18.992 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:18.987+0000 7f9dc6b28640 1 -- start start 2024-09-21T03:56:18.992 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:18.987+0000 7f9dc6b28640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9dc0103ad0 0x7f9dc019a3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:56:18.992 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:18.987+0000 7f9dc6b28640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9dc0109400 0x7f9dc019a970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:56:18.992 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:18.987+0000 7f9dc6b28640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9dc0109d50 0x7f9dc019fc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:56:18.993 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:18.987+0000 7f9dc6b28640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f9dc0113a10 con 0x7f9dc0103ad0 2024-09-21T03:56:18.993 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:18.987+0000 7f9dc6b28640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f9dc0113890 con 0x7f9dc0109d50 2024-09-21T03:56:18.993 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:18.987+0000 7f9dc6b28640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f9dc0113b90 con 0x7f9dc0109400 2024-09-21T03:56:18.993 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:18.987+0000 7f9dc48c4640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9dc0103ad0 0x7f9dc019a3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:56:18.993 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:18.987+0000 7f9dc48c4640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9dc0103ad0 0x7f9dc019a3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:55718/0 (socket says 172.21.15.39:55718) 2024-09-21T03:56:18.993 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:18.987+0000 7f9dc48c4640 1 -- 172.21.15.39:0/3069201388 learned_addr learned my addr 172.21.15.39:0/3069201388 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:56:18.993 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:18.988+0000 7f9dc50c5640 1 --2- 172.21.15.39:0/3069201388 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9dc0109d50 0x7f9dc019fc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:56:18.993 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:18.988+0000 7f9dbffff640 1 --2- 172.21.15.39:0/3069201388 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9dc0109400 0x7f9dc019a970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:56:18.993 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:18.988+0000 7f9dc48c4640 1 -- 172.21.15.39:0/3069201388 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9dc0109400 msgr2=0x7f9dc019a970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:56:18.993 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:18.988+0000 7f9dc48c4640 1 --2- 172.21.15.39:0/3069201388 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9dc0109400 0x7f9dc019a970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:18.993 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:18.988+0000 7f9dc48c4640 1 -- 172.21.15.39:0/3069201388 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9dc0109d50 msgr2=0x7f9dc019fc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:56:18.993 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:18.988+0000 7f9dc48c4640 1 --2- 172.21.15.39:0/3069201388 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9dc0109d50 0x7f9dc019fc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:18.993 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:18.988+0000 7f9dc48c4640 1 -- 172.21.15.39:0/3069201388 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9dc01a0270 con 0x7f9dc0103ad0 2024-09-21T03:56:18.993 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:18.988+0000 7f9dc48c4640 1 --2- 172.21.15.39:0/3069201388 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9dc0103ad0 0x7f9dc019a3d0 secure :-1 s=READY pgs=1318 cs=0 l=1 rev1=1 crypto rx=0x7f9db000d950 tx=0x7f9db000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:56:18.994 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:18.988+0000 7f9dbdffb640 1 -- 172.21.15.39:0/3069201388 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9db0017030 con 0x7f9dc0103ad0 2024-09-21T03:56:18.994 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:18.988+0000 7f9dbdffb640 1 -- 172.21.15.39:0/3069201388 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9db0004020 con 0x7f9dc0103ad0 2024-09-21T03:56:18.994 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:18.988+0000 7f9dc6b28640 1 -- 172.21.15.39:0/3069201388 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9dc01a0560 con 0x7f9dc0103ad0 2024-09-21T03:56:18.994 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:18.988+0000 7f9dbdffb640 1 -- 172.21.15.39:0/3069201388 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9db00047a0 con 0x7f9dc0103ad0 2024-09-21T03:56:18.994 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:18.988+0000 7f9dc6b28640 1 -- 172.21.15.39:0/3069201388 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9dc01a6d80 con 0x7f9dc0103ad0 2024-09-21T03:56:18.994 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:18.989+0000 7f9dc6b28640 1 -- 172.21.15.39:0/3069201388 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9dc0103ee0 con 0x7f9dc0103ad0 2024-09-21T03:56:18.995 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:18.992+0000 7f9dbdffb640 1 -- 172.21.15.39:0/3069201388 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 27) ==== 154643+0+0 (secure 0 0 0) 0x7f9db0009a30 con 0x7f9dc0103ad0 2024-09-21T03:56:18.996 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:18.993+0000 7f9dbdffb640 1 --2- 172.21.15.39:0/3069201388 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f9da80b6700 0x7f9da80b8c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:56:18.997 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:18.993+0000 7f9dbffff640 1 --2- 172.21.15.39:0/3069201388 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f9da80b6700 0x7f9da80b8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:56:18.998 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:18.993+0000 7f9dbdffb640 1 -- 172.21.15.39:0/3069201388 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8572+0+0 (secure 0 0 0) 0x7f9db00e7140 con 0x7f9dc0103ad0 2024-09-21T03:56:18.998 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:18.994+0000 7f9dbffff640 1 --2- 172.21.15.39:0/3069201388 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f9da80b6700 0x7f9da80b8c10 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f9dc0108e50 tx=0x7f9db400c4a0 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:56:19.001 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:18.998+0000 7f9dbdffb640 1 -- 172.21.15.39:0/3069201388 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f9db00ad700 con 0x7f9dc0103ad0 2024-09-21T03:56:19.214 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:19.211+0000 7f9dc6b28640 1 -- 172.21.15.39:0/3069201388 --> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}) -- 0x7f9dc0063800 con 0x7f9da80b6700 2024-09-21T03:56:19.221 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:19.218+0000 7f9dbdffb640 1 -- 172.21.15.39:0/3069201388 <== mgr.14223 v2:172.21.15.39:6800/1696381982 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+10 (secure 0 0 0) 0x7f9dc0063800 con 0x7f9da80b6700 2024-09-21T03:56:19.221 INFO:teuthology.orchestra.run.smithi039.stdout:/ 1m 6m3h 2024-09-21T03:56:19.224 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:19.220+0000 7f9dc6b28640 1 -- 172.21.15.39:0/3069201388 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f9da80b6700 msgr2=0x7f9da80b8c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:56:19.224 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:19.220+0000 7f9dc6b28640 1 --2- 172.21.15.39:0/3069201388 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f9da80b6700 0x7f9da80b8c10 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f9dc0108e50 tx=0x7f9db400c4a0 comp rx=0 tx=0).stop 2024-09-21T03:56:19.224 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:19.220+0000 7f9dc6b28640 1 -- 172.21.15.39:0/3069201388 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9dc0103ad0 msgr2=0x7f9dc019a3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:56:19.224 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:19.220+0000 7f9dc6b28640 1 --2- 172.21.15.39:0/3069201388 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9dc0103ad0 0x7f9dc019a3d0 secure :-1 s=READY pgs=1318 cs=0 l=1 rev1=1 crypto rx=0x7f9db000d950 tx=0x7f9db000de10 comp rx=0 tx=0).stop 2024-09-21T03:56:19.224 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:19.221+0000 7f9dc6b28640 1 -- 172.21.15.39:0/3069201388 shutdown_connections 2024-09-21T03:56:19.224 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:19.221+0000 7f9dc6b28640 1 --2- 172.21.15.39:0/3069201388 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f9da80b6700 0x7f9da80b8c10 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:19.224 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:19.221+0000 7f9dc6b28640 1 --2- 172.21.15.39:0/3069201388 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9dc0109d50 0x7f9dc019fc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:19.224 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:19.221+0000 7f9dc6b28640 1 --2- 172.21.15.39:0/3069201388 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9dc0109400 0x7f9dc019a970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:19.224 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:19.221+0000 7f9dc6b28640 1 --2- 172.21.15.39:0/3069201388 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9dc0103ad0 0x7f9dc019a3d0 unknown :-1 s=CLOSED pgs=1318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:19.224 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:19.221+0000 7f9dc6b28640 1 -- 172.21.15.39:0/3069201388 >> 172.21.15.39:0/3069201388 conn(0x7f9dc00ff510 msgr2=0x7f9dc0101930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:56:19.224 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:19.221+0000 7f9dc6b28640 1 -- 172.21.15.39:0/3069201388 shutdown_connections 2024-09-21T03:56:19.224 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:19.221+0000 7f9dc6b28640 1 -- 172.21.15.39:0/3069201388 wait complete. 2024-09-21T03:56:19.234 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:56:19 smithi039 ceph-mon[31211]: pgmap v640: 97 pgs: 97 active+clean; 10 MiB data, 525 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 204 B/s wr, 10 op/s 2024-09-21T03:56:19.234 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:56:19 smithi039 ceph-mon[31211]: scrub queued for path: / 2024-09-21T03:56:19.234 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:56:19 smithi039 ceph-mon[31211]: scrub summary: idle+waiting paths [/] 2024-09-21T03:56:19.234 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:56:19 smithi039 ceph-mon[31211]: scrub summary: active paths [/] 2024-09-21T03:56:19.234 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:56:19 smithi039 ceph-mon[31211]: scrub summary: idle+waiting paths [/] 2024-09-21T03:56:19.235 DEBUG:teuthology.run_tasks:Unwinding manager fwd_scrub 2024-09-21T03:56:19.244 INFO:tasks.fwd_scrub:joining ForwardScrubbers 2024-09-21T03:56:19.468 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:56:19 smithi161 ceph-mon[34533]: pgmap v640: 97 pgs: 97 active+clean; 10 MiB data, 525 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 204 B/s wr, 10 op/s 2024-09-21T03:56:19.468 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:56:19 smithi161 ceph-mon[34533]: scrub queued for path: / 2024-09-21T03:56:19.468 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:56:19 smithi161 ceph-mon[34533]: scrub summary: idle+waiting paths [/] 2024-09-21T03:56:19.468 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:56:19 smithi161 ceph-mon[34533]: scrub summary: active paths [/] 2024-09-21T03:56:19.468 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:56:19 smithi161 ceph-mon[34533]: scrub summary: idle+waiting paths [/] 2024-09-21T03:56:19.475 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:56:19 smithi150 ceph-mon[34289]: pgmap v640: 97 pgs: 97 active+clean; 10 MiB data, 525 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 204 B/s wr, 10 op/s 2024-09-21T03:56:19.475 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:56:19 smithi150 ceph-mon[34289]: scrub queued for path: / 2024-09-21T03:56:19.475 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:56:19 smithi150 ceph-mon[34289]: scrub summary: idle+waiting paths [/] 2024-09-21T03:56:19.475 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:56:19 smithi150 ceph-mon[34289]: scrub summary: active paths [/] 2024-09-21T03:56:19.475 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:56:19 smithi150 ceph-mon[34289]: scrub summary: idle+waiting paths [/] 2024-09-21T03:56:19.567 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 51116d5c-77ca-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub status 2024-09-21T03:56:19.823 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/51116d5c-77ca-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:56:20.233 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:56:20 smithi039 ceph-mon[31211]: from='client.18240 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:56:20.233 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:56:20 smithi039 ceph-mon[31211]: from='client.18264 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:56:20.233 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:56:20 smithi039 ceph-mon[31211]: scrub summary: idle 2024-09-21T03:56:20.234 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:56:20 smithi039 ceph-mon[31211]: osdmap e87: 12 total, 12 up, 12 in 2024-09-21T03:56:20.327 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:20.324+0000 7fcf3f991640 1 -- 172.21.15.39:0/240793655 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcf38107be0 msgr2=0x7fcf381080c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:56:20.327 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:20.324+0000 7fcf3f991640 1 --2- 172.21.15.39:0/240793655 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcf38107be0 0x7fcf381080c0 secure :-1 s=READY pgs=1319 cs=0 l=1 rev1=1 crypto rx=0x7fcf30009cd0 tx=0x7fcf3002a8e0 comp rx=0 tx=0).stop 2024-09-21T03:56:20.327 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:20.324+0000 7fcf3f991640 1 -- 172.21.15.39:0/240793655 shutdown_connections 2024-09-21T03:56:20.327 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:20.324+0000 7fcf3f991640 1 --2- 172.21.15.39:0/240793655 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcf38107be0 0x7fcf381080c0 unknown :-1 s=CLOSED pgs=1319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:20.327 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:20.324+0000 7fcf3f991640 1 --2- 172.21.15.39:0/240793655 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcf381030f0 0x7fcf38107620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:20.327 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:20.324+0000 7fcf3f991640 1 --2- 172.21.15.39:0/240793655 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcf381027a0 0x7fcf38102bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:20.327 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:20.324+0000 7fcf3f991640 1 -- 172.21.15.39:0/240793655 >> 172.21.15.39:0/240793655 conn(0x7fcf38079720 msgr2=0x7fcf38079b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:56:20.328 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:20.324+0000 7fcf3f991640 1 -- 172.21.15.39:0/240793655 shutdown_connections 2024-09-21T03:56:20.328 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:20.324+0000 7fcf3f991640 1 -- 172.21.15.39:0/240793655 wait complete. 2024-09-21T03:56:20.329 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:20.325+0000 7fcf3f991640 1 Processor -- start 2024-09-21T03:56:20.329 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:20.325+0000 7fcf3f991640 1 -- start start 2024-09-21T03:56:20.329 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:20.325+0000 7fcf3f991640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcf381027a0 0x7fcf381a0a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:56:20.329 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:20.325+0000 7fcf3f991640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcf381030f0 0x7fcf381a1030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:56:20.329 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:20.325+0000 7fcf3f991640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcf38107be0 0x7fcf381a62d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:56:20.329 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:20.325+0000 7fcf3f991640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fcf3811a080 con 0x7fcf38107be0 2024-09-21T03:56:20.330 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:20.325+0000 7fcf3f991640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fcf38119f00 con 0x7fcf381027a0 2024-09-21T03:56:20.330 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:20.325+0000 7fcf3f991640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fcf3811a200 con 0x7fcf381030f0 2024-09-21T03:56:20.330 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:20.325+0000 7fcf3df2e640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcf38107be0 0x7fcf381a62d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:56:20.330 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:20.325+0000 7fcf3df2e640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcf38107be0 0x7fcf381a62d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:55738/0 (socket says 172.21.15.39:55738) 2024-09-21T03:56:20.330 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:20.325+0000 7fcf3cf2c640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcf381030f0 0x7fcf381a1030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:56:20.330 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:20.325+0000 7fcf3df2e640 1 -- 172.21.15.39:0/3925517934 learned_addr learned my addr 172.21.15.39:0/3925517934 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:56:20.330 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:20.325+0000 7fcf3d72d640 1 --2- 172.21.15.39:0/3925517934 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcf381027a0 0x7fcf381a0a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:56:20.330 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:20.325+0000 7fcf3df2e640 1 -- 172.21.15.39:0/3925517934 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcf381030f0 msgr2=0x7fcf381a1030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:56:20.330 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:20.325+0000 7fcf3df2e640 1 --2- 172.21.15.39:0/3925517934 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcf381030f0 0x7fcf381a1030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:20.330 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:20.325+0000 7fcf3df2e640 1 -- 172.21.15.39:0/3925517934 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcf381027a0 msgr2=0x7fcf381a0a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:56:20.330 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:20.325+0000 7fcf3df2e640 1 --2- 172.21.15.39:0/3925517934 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcf381027a0 0x7fcf381a0a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:20.330 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:20.325+0000 7fcf3df2e640 1 -- 172.21.15.39:0/3925517934 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcf381a6a00 con 0x7fcf38107be0 2024-09-21T03:56:20.330 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:20.326+0000 7fcf3df2e640 1 --2- 172.21.15.39:0/3925517934 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcf38107be0 0x7fcf381a62d0 secure :-1 s=READY pgs=1320 cs=0 l=1 rev1=1 crypto rx=0x7fcf30005cc0 tx=0x7fcf30003c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:56:20.330 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:20.326+0000 7fcf2e7fc640 1 -- 172.21.15.39:0/3925517934 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf30004450 con 0x7fcf38107be0 2024-09-21T03:56:20.331 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:20.326+0000 7fcf2e7fc640 1 -- 172.21.15.39:0/3925517934 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcf30031c90 con 0x7fcf38107be0 2024-09-21T03:56:20.331 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:20.326+0000 7fcf3f991640 1 -- 172.21.15.39:0/3925517934 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcf381a6c90 con 0x7fcf38107be0 2024-09-21T03:56:20.331 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:20.326+0000 7fcf2e7fc640 1 -- 172.21.15.39:0/3925517934 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf3003b400 con 0x7fcf38107be0 2024-09-21T03:56:20.331 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:20.326+0000 7fcf3f991640 1 -- 172.21.15.39:0/3925517934 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcf381ad450 con 0x7fcf38107be0 2024-09-21T03:56:20.331 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:20.329+0000 7fcf45f05740 1 Processor -- start 2024-09-21T03:56:20.332 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:20.329+0000 7fcf2e7fc640 1 -- 172.21.15.39:0/3925517934 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 27) ==== 154643+0+0 (secure 0 0 0) 0x7fcf30004680 con 0x7fcf38107be0 2024-09-21T03:56:20.332 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:20.329+0000 7fcf45f05740 1 -- start start 2024-09-21T03:56:20.333 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:20.329+0000 7fcf45f05740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55cd265c2070 0x55cd26969650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:56:20.333 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:20.329+0000 7fcf45f05740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55cd265991b0 0x55cd26969b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:56:20.333 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:20.329+0000 7fcf45f05740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55cd265a1390 0x55cd2696a0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:56:20.334 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:20.329+0000 7fcf45f05740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x55cd268dfe50 con 0x55cd265c2070 2024-09-21T03:56:20.334 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:20.329+0000 7fcf45f05740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x55cd268cf280 con 0x55cd265991b0 2024-09-21T03:56:20.334 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:20.329+0000 7fcf45f05740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55cd268d3090 con 0x55cd265a1390 2024-09-21T03:56:20.334 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:20.329+0000 7fcf3cf2c640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55cd265c2070 0x55cd26969650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:56:20.334 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:20.329+0000 7fcf3cf2c640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55cd265c2070 0x55cd26969650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:55746/0 (socket says 172.21.15.39:55746) 2024-09-21T03:56:20.334 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:20.329+0000 7fcf3cf2c640 1 -- 172.21.15.39:0/3490328210 learned_addr learned my addr 172.21.15.39:0/3490328210 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:56:20.334 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:20.329+0000 7fcf3d72d640 1 --2- 172.21.15.39:0/3490328210 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55cd265991b0 0x55cd26969b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:56:20.334 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:20.330+0000 7fcf3cf2c640 1 --2- 172.21.15.39:0/3490328210 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55cd265a1390 0x55cd2696a0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:56:20.334 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:20.330+0000 7fcf3cf2c640 1 -- 172.21.15.39:0/3490328210 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55cd265a1390 msgr2=0x55cd2696a0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:56:20.334 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:20.330+0000 7fcf3cf2c640 1 --2- 172.21.15.39:0/3490328210 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55cd265a1390 0x55cd2696a0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:20.334 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:20.330+0000 7fcf3cf2c640 1 -- 172.21.15.39:0/3490328210 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55cd265991b0 msgr2=0x55cd26969b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:56:20.334 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:20.330+0000 7fcf3cf2c640 1 --2- 172.21.15.39:0/3490328210 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55cd265991b0 0x55cd26969b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:20.334 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:20.330+0000 7fcf3cf2c640 1 -- 172.21.15.39:0/3490328210 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cd2662d650 con 0x55cd265c2070 2024-09-21T03:56:20.334 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:20.330+0000 7fcf2e7fc640 1 --2- 172.21.15.39:0/3925517934 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7fcf240b6680 0x7fcf240b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:56:20.335 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:20.330+0000 7fcf3d72d640 1 --2- 172.21.15.39:0/3925517934 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7fcf240b6680 0x7fcf240b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:56:20.335 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:20.330+0000 7fcf3cf2c640 1 --2- 172.21.15.39:0/3490328210 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55cd265c2070 0x55cd26969650 secure :-1 s=READY pgs=1321 cs=0 l=1 rev1=1 crypto rx=0x7fcf2800bfd0 tx=0x7fcf28005c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=76e5a0abdf9a37ec server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:56:20.335 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:20.330+0000 7fcf2e7fc640 1 -- 172.21.15.39:0/3925517934 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8528+0+0 (secure 0 0 0) 0x7fcf3002e030 con 0x7fcf38107be0 2024-09-21T03:56:20.335 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:20.330+0000 7fcf237fe640 1 -- 172.21.15.39:0/3490328210 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf280120d0 con 0x55cd265c2070 2024-09-21T03:56:20.335 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:20.330+0000 7fcf237fe640 1 -- 172.21.15.39:0/3490328210 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcf28012cc0 con 0x55cd265c2070 2024-09-21T03:56:20.335 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:20.330+0000 7fcf237fe640 1 -- 172.21.15.39:0/3490328210 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf28005580 con 0x55cd265c2070 2024-09-21T03:56:20.338 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:20.331+0000 7fcf45f05740 1 -- 172.21.15.39:0/3490328210 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55cd265c2070 msgr2=0x55cd26969650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:56:20.338 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:20.331+0000 7fcf3d72d640 1 --2- 172.21.15.39:0/3925517934 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7fcf240b6680 0x7fcf240b8b90 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x55cd269682d0 tx=0x7fcf3400b390 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:56:20.338 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:20.331+0000 7fcf45f05740 1 --2- 172.21.15.39:0/3490328210 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55cd265c2070 0x55cd26969650 secure :-1 s=READY pgs=1321 cs=0 l=1 rev1=1 crypto rx=0x7fcf2800bfd0 tx=0x7fcf28005c90 comp rx=0 tx=0).stop 2024-09-21T03:56:20.338 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:20.331+0000 7fcf45f05740 1 -- 172.21.15.39:0/3490328210 shutdown_connections 2024-09-21T03:56:20.338 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:20.331+0000 7fcf45f05740 1 --2- 172.21.15.39:0/3490328210 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55cd265a1390 0x55cd2696a0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:20.338 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:20.331+0000 7fcf45f05740 1 --2- 172.21.15.39:0/3490328210 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55cd265991b0 0x55cd26969b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:20.338 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:20.331+0000 7fcf45f05740 1 --2- 172.21.15.39:0/3490328210 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55cd265c2070 0x55cd26969650 unknown :-1 s=CLOSED pgs=1321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:20.338 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:20.331+0000 7fcf45f05740 1 -- 172.21.15.39:0/3490328210 >> 172.21.15.39:0/3490328210 conn(0x55cd265b6460 msgr2=0x55cd26967540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:56:20.338 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:20.331+0000 7fcf45f05740 1 -- 172.21.15.39:0/3490328210 shutdown_connections 2024-09-21T03:56:20.338 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:20.331+0000 7fcf45f05740 1 -- 172.21.15.39:0/3490328210 wait complete. 2024-09-21T03:56:20.338 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:20.332+0000 7fcf45f05740 1 -- start start 2024-09-21T03:56:20.339 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:20.332+0000 7fcf45f05740 1 Processor -- start 2024-09-21T03:56:20.339 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:20.333+0000 7fcf45f05740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55cd265991b0 0x55cd26974d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:56:20.339 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:20.333+0000 7fcf45f05740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55cd265a1390 0x55cd26975260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:56:20.339 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:20.333+0000 7fcf45f05740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55cd265c2070 0x55cd269810f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:56:20.339 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:20.333+0000 7fcf45f05740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x55cd26730700 con 0x55cd265c2070 2024-09-21T03:56:20.339 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:20.333+0000 7fcf45f05740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x55cd2686a110 con 0x55cd265991b0 2024-09-21T03:56:20.339 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:20.333+0000 7fcf45f05740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55cd26977dd0 con 0x55cd265a1390 2024-09-21T03:56:20.339 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:20.333+0000 7fcf45f05740 10 client.0 _ll_register_callbacks cb 0x55cd26883990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:56:20.339 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:20.333+0000 7fcf3df2e640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55cd265c2070 0x55cd269810f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:56:20.339 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:20.333+0000 7fcf3d72d640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55cd265991b0 0x55cd26974d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:56:20.339 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:20.333+0000 7fcf3d72d640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55cd265991b0 0x55cd26974d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.39:58700/0 (socket says 172.21.15.39:58700) 2024-09-21T03:56:20.339 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:20.333+0000 7fcf3cf2c640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55cd265a1390 0x55cd26975260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:56:20.339 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:20.333+0000 7fcf3d72d640 1 -- 172.21.15.39:0/798445275 learned_addr learned my addr 172.21.15.39:0/798445275 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:56:20.339 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:20.333+0000 7fcf3df2e640 1 -- 172.21.15.39:0/798445275 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55cd265a1390 msgr2=0x55cd26975260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:56:20.340 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:20.333+0000 7fcf3df2e640 1 --2- 172.21.15.39:0/798445275 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55cd265a1390 0x55cd26975260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:20.340 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:20.333+0000 7fcf3df2e640 1 -- 172.21.15.39:0/798445275 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55cd265991b0 msgr2=0x55cd26974d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:56:20.340 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:20.333+0000 7fcf3df2e640 1 --2- 172.21.15.39:0/798445275 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55cd265991b0 0x55cd26974d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:20.340 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:20.333+0000 7fcf3df2e640 1 -- 172.21.15.39:0/798445275 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cd26897c40 con 0x55cd265c2070 2024-09-21T03:56:20.340 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:20.334+0000 7fcf3df2e640 1 --2- 172.21.15.39:0/798445275 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55cd265c2070 0x55cd269810f0 secure :-1 s=READY pgs=1322 cs=0 l=1 rev1=1 crypto rx=0x7fcf30105e70 tx=0x7fcf300308a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a769912d69d554a server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:56:20.340 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:20.334+0000 7fcf20ff9640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-09-21T03:56:20.340 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:20.334+0000 7fcf20ff9640 1 -- 172.21.15.39:0/798445275 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf300cdce0 con 0x55cd265c2070 2024-09-21T03:56:20.340 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:20.334+0000 7fcf20ff9640 1 -- 172.21.15.39:0/798445275 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8528+0+0 (secure 0 0 0) 0x7fcf300cd240 con 0x55cd265c2070 2024-09-21T03:56:20.340 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:20.334+0000 7fcf45f05740 1 -- 172.21.15.39:0/798445275 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcf28012cc0 con 0x55cd265c2070 2024-09-21T03:56:20.340 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:20.334+0000 7fcf20ff9640 1 -- 172.21.15.39:0/798445275 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fcf300ce080 con 0x55cd265c2070 2024-09-21T03:56:20.340 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:20.334+0000 7fcf45f05740 10 client.18276 fetch_fsmap learned FSMap version 29 2024-09-21T03:56:20.340 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:20.334+0000 7fcf45f05740 1 -- 172.21.15.39:0/798445275 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55cd26983ad0 con 0x55cd265c2070 2024-09-21T03:56:20.340 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:20.335+0000 7fcf20ff9640 1 -- 172.21.15.39:0/798445275 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcf30036040 con 0x55cd265c2070 2024-09-21T03:56:20.340 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:20.335+0000 7fcf20ff9640 1 -- 172.21.15.39:0/798445275 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fcf3004d020 con 0x55cd265c2070 2024-09-21T03:56:20.340 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:20.335+0000 7fcf45f05740 10 client.18276 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T03:56:20.341 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:20.335+0000 7fcf45f05740 10 client.18276 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T03:56:20.341 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:20.335+0000 7fcf45f05740 20 client.18276 populate_metadata read hostname 'smithi039' 2024-09-21T03:56:20.341 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:20.335+0000 7fcf45f05740 1 --2- 172.21.15.39:0/798445275 >> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] conn(0x55cd265809f0 0x55cd26986fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:56:20.341 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:20.335+0000 7fcf45f05740 4 client.18276 mds_command: new command op to 24695 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:56:20.341 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:20.335+0000 7fcf45f05740 1 -- 172.21.15.39:0/798445275 --> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fcf28012cc0 con 0x55cd265809f0 2024-09-21T03:56:20.341 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:20.335+0000 7fcf3d72d640 1 --2- 172.21.15.39:0/798445275 >> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] conn(0x55cd265809f0 0x55cd26986fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:56:20.342 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:20.335+0000 7fcf20ff9640 10 client.18276 ms_handle_connect on v2:172.21.15.150:6838/2949386907 2024-09-21T03:56:20.342 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:20.335+0000 7fcf3d72d640 1 --2- 172.21.15.39:0/798445275 >> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] conn(0x55cd265809f0 0x55cd26986fa0 crc :-1 s=READY pgs=167 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e591b5cf6c2b7a19 server_cookie=6e8a533dbc9641e8 in_seq=0 out_seq=0 2024-09-21T03:56:20.342 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:20.336+0000 7fcf20ff9640 1 -- 172.21.15.39:0/798445275 <== mds.0 v2:172.21.15.150:6838/2949386907 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x7fcf28012cc0 con 0x55cd265809f0 2024-09-21T03:56:20.342 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:20.336+0000 7fcf20ff9640 10 client.18276 handle_command_reply: tid=0 2024-09-21T03:56:20.342 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:20.336+0000 7fcf45f05740 1 client.18276 shutdown 2024-09-21T03:56:20.342 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:20.336+0000 7fcf45f05740 1 -- 172.21.15.39:0/798445275 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55cd265c2070 msgr2=0x55cd269810f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:56:20.343 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:20.336+0000 7fcf45f05740 1 --2- 172.21.15.39:0/798445275 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55cd265c2070 0x55cd269810f0 secure :-1 s=READY pgs=1322 cs=0 l=1 rev1=1 crypto rx=0x7fcf30105e70 tx=0x7fcf300308a0 comp rx=0 tx=0).stop 2024-09-21T03:56:20.343 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:20.336+0000 7fcf45f05740 1 -- 172.21.15.39:0/798445275 shutdown_connections 2024-09-21T03:56:20.343 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:20.336+0000 7fcf45f05740 1 --2- 172.21.15.39:0/798445275 >> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] conn(0x55cd265809f0 0x55cd26986fa0 crc :-1 s=READY pgs=167 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:20.343 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:20.336+0000 7fcf45f05740 1 --2- 172.21.15.39:0/798445275 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55cd265c2070 0x55cd269810f0 unknown :-1 s=CLOSED pgs=1322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:20.343 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:20.336+0000 7fcf45f05740 1 --2- 172.21.15.39:0/798445275 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55cd265a1390 0x55cd26975260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:20.343 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:20.337+0000 7fcf45f05740 1 --2- 172.21.15.39:0/798445275 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55cd265991b0 0x55cd26974d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:20.343 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:20.337+0000 7fcf45f05740 1 -- 172.21.15.39:0/798445275 >> 172.21.15.39:0/798445275 conn(0x55cd265b6460 msgr2=0x55cd2696c830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:56:20.343 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:20.337+0000 7fcf20ff9640 0 client.18276 ms_handle_reset on v2:172.21.15.150:6838/2949386907 2024-09-21T03:56:20.343 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:20.337+0000 7fcf45f05740 1 -- 172.21.15.39:0/798445275 shutdown_connections 2024-09-21T03:56:20.343 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:20.337+0000 7fcf45f05740 1 -- 172.21.15.39:0/798445275 wait complete. 2024-09-21T03:56:20.343 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:20.337+0000 7fcf45f05740 20 client.18276 trim_cache size 0 max 16384 2024-09-21T03:56:20.358 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:20.354+0000 7fcf45f05740 1 Processor -- start 2024-09-21T03:56:20.358 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:20.355+0000 7fcf45f05740 1 -- start start 2024-09-21T03:56:20.358 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:20.355+0000 7fcf45f05740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55cd265991b0 0x55cd26976950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:56:20.358 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:20.355+0000 7fcf45f05740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55cd265a1390 0x55cd26966b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:56:20.358 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:20.355+0000 7fcf45f05740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55cd265c2070 0x55cd269670b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:56:20.358 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:20.355+0000 7fcf45f05740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fcf381b5cc0 con 0x55cd265c2070 2024-09-21T03:56:20.358 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:20.355+0000 7fcf45f05740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fcf381b7800 con 0x55cd265991b0 2024-09-21T03:56:20.358 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:20.355+0000 7fcf45f05740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fcf381b6390 con 0x55cd265a1390 2024-09-21T03:56:20.359 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:20.355+0000 7fcf3df2e640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55cd265c2070 0x55cd269670b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:56:20.359 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:20.355+0000 7fcf3df2e640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55cd265c2070 0x55cd269670b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:55768/0 (socket says 172.21.15.39:55768) 2024-09-21T03:56:20.359 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:20.355+0000 7fcf3df2e640 1 -- 172.21.15.39:0/3159010756 learned_addr learned my addr 172.21.15.39:0/3159010756 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:56:20.359 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:20.355+0000 7fcf3cf2c640 1 --2- 172.21.15.39:0/3159010756 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55cd265a1390 0x55cd26966b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:56:20.359 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:20.355+0000 7fcf3d72d640 1 --2- 172.21.15.39:0/3159010756 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55cd265991b0 0x55cd26976950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:56:20.359 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:20.355+0000 7fcf3df2e640 1 -- 172.21.15.39:0/3159010756 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55cd265a1390 msgr2=0x55cd26966b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:56:20.359 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:20.355+0000 7fcf3df2e640 1 --2- 172.21.15.39:0/3159010756 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55cd265a1390 0x55cd26966b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:20.359 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:20.355+0000 7fcf3df2e640 1 -- 172.21.15.39:0/3159010756 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55cd265991b0 msgr2=0x55cd26976950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:56:20.359 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:20.355+0000 7fcf3df2e640 1 --2- 172.21.15.39:0/3159010756 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55cd265991b0 0x55cd26976950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:20.359 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:20.355+0000 7fcf3df2e640 1 -- 172.21.15.39:0/3159010756 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cd2696b940 con 0x55cd265c2070 2024-09-21T03:56:20.359 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:20.355+0000 7fcf3d72d640 1 --2- 172.21.15.39:0/3159010756 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55cd265991b0 0x55cd26976950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:56:20.362 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:20.356+0000 7fcf3df2e640 1 --2- 172.21.15.39:0/3159010756 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55cd265c2070 0x55cd269670b0 secure :-1 s=READY pgs=1323 cs=0 l=1 rev1=1 crypto rx=0x7fcf300cb7e0 tx=0x7fcf300ca540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a7619ed5569cd03 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:56:20.362 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:20.356+0000 7fcf22ffd640 1 -- 172.21.15.39:0/3159010756 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf300cdce0 con 0x55cd265c2070 2024-09-21T03:56:20.362 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:20.356+0000 7fcf22ffd640 1 -- 172.21.15.39:0/3159010756 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcf30036070 con 0x55cd265c2070 2024-09-21T03:56:20.362 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:20.356+0000 7fcf22ffd640 1 -- 172.21.15.39:0/3159010756 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf300d3050 con 0x55cd265c2070 2024-09-21T03:56:20.363 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:20.357+0000 7fcf45f05740 1 -- 172.21.15.39:0/3159010756 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55cd265c2070 msgr2=0x55cd269670b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:56:20.363 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:20.357+0000 7fcf45f05740 1 --2- 172.21.15.39:0/3159010756 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55cd265c2070 0x55cd269670b0 secure :-1 s=READY pgs=1323 cs=0 l=1 rev1=1 crypto rx=0x7fcf300cb7e0 tx=0x7fcf300ca540 comp rx=0 tx=0).stop 2024-09-21T03:56:20.363 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:20.357+0000 7fcf45f05740 1 -- 172.21.15.39:0/3159010756 shutdown_connections 2024-09-21T03:56:20.363 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:20.357+0000 7fcf45f05740 1 --2- 172.21.15.39:0/3159010756 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55cd265c2070 0x55cd269670b0 unknown :-1 s=CLOSED pgs=1323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:20.363 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:20.357+0000 7fcf45f05740 1 --2- 172.21.15.39:0/3159010756 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55cd265a1390 0x55cd26966b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:20.363 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:20.357+0000 7fcf45f05740 1 --2- 172.21.15.39:0/3159010756 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55cd265991b0 0x55cd26976950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:20.363 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:20.357+0000 7fcf45f05740 1 -- 172.21.15.39:0/3159010756 >> 172.21.15.39:0/3159010756 conn(0x55cd265b6460 msgr2=0x55cd26976240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:56:20.363 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:20.357+0000 7fcf45f05740 1 -- 172.21.15.39:0/3159010756 shutdown_connections 2024-09-21T03:56:20.363 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:20.357+0000 7fcf45f05740 1 -- 172.21.15.39:0/3159010756 wait complete. 2024-09-21T03:56:20.363 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:20.357+0000 7fcf45f05740 1 -- start start 2024-09-21T03:56:20.363 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:20.357+0000 7fcf45f05740 1 Processor -- start 2024-09-21T03:56:20.363 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:20.358+0000 7fcf45f05740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55cd265991b0 0x55cd26979400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:56:20.363 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:20.358+0000 7fcf45f05740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55cd265a1390 0x55cd26979940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:56:20.363 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:20.358+0000 7fcf45f05740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55cd265c2070 0x55cd269f0930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:56:20.363 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:20.358+0000 7fcf45f05740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fcf381b7130 con 0x55cd265991b0 2024-09-21T03:56:20.364 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:20.358+0000 7fcf45f05740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fcf381b6a60 con 0x55cd265c2070 2024-09-21T03:56:20.364 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:20.358+0000 7fcf45f05740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fcf381b3ca0 con 0x55cd265a1390 2024-09-21T03:56:20.364 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:20.358+0000 7fcf3d72d640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55cd265991b0 0x55cd26979400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:56:20.364 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:20.358+0000 7fcf45f05740 10 client.0 _ll_register_callbacks cb 0x7fcf30009460 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:56:20.364 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:20.358+0000 7fcf3d72d640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55cd265991b0 0x55cd26979400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:55776/0 (socket says 172.21.15.39:55776) 2024-09-21T03:56:20.364 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:20.358+0000 7fcf3d72d640 1 -- 172.21.15.39:0/3802926538 learned_addr learned my addr 172.21.15.39:0/3802926538 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:56:20.364 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:20.358+0000 7fcf3df2e640 1 --2- 172.21.15.39:0/3802926538 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55cd265c2070 0x55cd269f0930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:56:20.364 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:20.358+0000 7fcf3cf2c640 1 --2- 172.21.15.39:0/3802926538 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55cd265a1390 0x55cd26979940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:56:20.364 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:20.358+0000 7fcf3d72d640 1 -- 172.21.15.39:0/3802926538 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55cd265a1390 msgr2=0x55cd26979940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:56:20.364 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:20.358+0000 7fcf3d72d640 1 --2- 172.21.15.39:0/3802926538 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55cd265a1390 0x55cd26979940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:20.364 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:20.358+0000 7fcf3d72d640 1 -- 172.21.15.39:0/3802926538 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55cd265c2070 msgr2=0x55cd269f0930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:56:20.364 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:20.358+0000 7fcf3d72d640 1 --2- 172.21.15.39:0/3802926538 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55cd265c2070 0x55cd269f0930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:20.364 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:20.358+0000 7fcf3d72d640 1 -- 172.21.15.39:0/3802926538 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cd269f0e70 con 0x55cd265991b0 2024-09-21T03:56:20.364 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:20.358+0000 7fcf3d72d640 1 --2- 172.21.15.39:0/3802926538 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55cd265991b0 0x55cd26979400 secure :-1 s=READY pgs=1324 cs=0 l=1 rev1=1 crypto rx=0x7fcf340043f0 tx=0x7fcf340046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6fef9b62f7c84eae server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:56:20.365 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:20.359+0000 7fcf20ff9640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-09-21T03:56:20.365 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:20.359+0000 7fcf20ff9640 1 -- 172.21.15.39:0/3802926538 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf3401eac0 con 0x55cd265991b0 2024-09-21T03:56:20.365 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:20.359+0000 7fcf20ff9640 1 -- 172.21.15.39:0/3802926538 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8528+0+0 (secure 0 0 0) 0x7fcf3401ec60 con 0x55cd265991b0 2024-09-21T03:56:20.365 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:20.359+0000 7fcf45f05740 1 -- 172.21.15.39:0/3802926538 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcf30036070 con 0x55cd265991b0 2024-09-21T03:56:20.365 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:20.359+0000 7fcf20ff9640 1 -- 172.21.15.39:0/3802926538 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fcf3401ee60 con 0x55cd265991b0 2024-09-21T03:56:20.365 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:20.359+0000 7fcf45f05740 10 client.18282 fetch_fsmap learned FSMap version 29 2024-09-21T03:56:20.365 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:20.359+0000 7fcf45f05740 1 -- 172.21.15.39:0/3802926538 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55cd269f1fb0 con 0x55cd265991b0 2024-09-21T03:56:20.365 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:20.359+0000 7fcf20ff9640 1 -- 172.21.15.39:0/3802926538 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcf30036070 con 0x55cd265991b0 2024-09-21T03:56:20.365 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:20.359+0000 7fcf20ff9640 1 -- 172.21.15.39:0/3802926538 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fcf3403a5e0 con 0x55cd265991b0 2024-09-21T03:56:20.365 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:20.359+0000 7fcf45f05740 10 client.18282 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T03:56:20.365 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:20.360+0000 7fcf45f05740 10 client.18282 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T03:56:20.365 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:20.360+0000 7fcf45f05740 20 client.18282 populate_metadata read hostname 'smithi039' 2024-09-21T03:56:20.365 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:20.360+0000 7fcf45f05740 1 --2- 172.21.15.39:0/3802926538 >> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] conn(0x55cd26555da0 0x55cd269f4730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:56:20.365 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:20.360+0000 7fcf45f05740 4 client.18282 mds_command: new command op to 24695 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-21T03:56:20.365 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:20.360+0000 7fcf45f05740 1 -- 172.21.15.39:0/3802926538 --> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55cd268e1da0 con 0x55cd26555da0 2024-09-21T03:56:20.367 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:20.360+0000 7fcf3cf2c640 1 --2- 172.21.15.39:0/3802926538 >> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] conn(0x55cd26555da0 0x55cd269f4730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:56:20.367 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:20.360+0000 7fcf3cf2c640 1 --2- 172.21.15.39:0/3802926538 >> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] conn(0x55cd26555da0 0x55cd269f4730 crc :-1 s=READY pgs=168 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=eee76c897b738370 server_cookie=7975eeb5753fa5cc in_seq=0 out_seq=0 2024-09-21T03:56:20.367 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:20.360+0000 7fcf20ff9640 10 client.18282 ms_handle_connect on v2:172.21.15.150:6838/2949386907 2024-09-21T03:56:20.367 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:20.361+0000 7fcf20ff9640 1 -- 172.21.15.39:0/3802926538 <== mds.0 v2:172.21.15.150:6838/2949386907 1 ==== command_reply(tid 0: 0 ) ==== 8+0+5770 (crc 0 0 0) 0x55cd268e1da0 con 0x55cd26555da0 2024-09-21T03:56:20.367 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:20.361+0000 7fcf20ff9640 10 client.18282 handle_command_reply: tid=0 2024-09-21T03:56:20.367 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:20.361+0000 7fcf45f05740 1 client.18282 shutdown 2024-09-21T03:56:20.367 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:20.361+0000 7fcf45f05740 1 -- 172.21.15.39:0/3802926538 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55cd265991b0 msgr2=0x55cd26979400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:56:20.367 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:20.361+0000 7fcf45f05740 1 --2- 172.21.15.39:0/3802926538 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55cd265991b0 0x55cd26979400 secure :-1 s=READY pgs=1324 cs=0 l=1 rev1=1 crypto rx=0x7fcf340043f0 tx=0x7fcf340046c0 comp rx=0 tx=0).stop 2024-09-21T03:56:20.367 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:20.361+0000 7fcf45f05740 1 -- 172.21.15.39:0/3802926538 shutdown_connections 2024-09-21T03:56:20.367 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:20.361+0000 7fcf45f05740 1 --2- 172.21.15.39:0/3802926538 >> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] conn(0x55cd26555da0 0x55cd269f4730 crc :-1 s=READY pgs=168 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:20.367 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:20.361+0000 7fcf45f05740 1 --2- 172.21.15.39:0/3802926538 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55cd265c2070 0x55cd269f0930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:20.368 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:20.361+0000 7fcf45f05740 1 --2- 172.21.15.39:0/3802926538 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55cd265a1390 0x55cd26979940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:20.368 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:20.361+0000 7fcf20ff9640 0 client.18282 ms_handle_reset on v2:172.21.15.150:6838/2949386907 2024-09-21T03:56:20.368 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:20.361+0000 7fcf45f05740 1 --2- 172.21.15.39:0/3802926538 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55cd265991b0 0x55cd26979400 unknown :-1 s=CLOSED pgs=1324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:20.368 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:20.361+0000 7fcf45f05740 1 -- 172.21.15.39:0/3802926538 >> 172.21.15.39:0/3802926538 conn(0x55cd265b6460 msgr2=0x55cd269761d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:56:20.368 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:20.361+0000 7fcf45f05740 1 -- 172.21.15.39:0/3802926538 shutdown_connections 2024-09-21T03:56:20.368 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:20.361+0000 7fcf45f05740 1 -- 172.21.15.39:0/3802926538 wait complete. 2024-09-21T03:56:20.368 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:20.361+0000 7fcf45f05740 20 client.18282 trim_cache size 0 max 16384 2024-09-21T03:56:20.368 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-09-21T03:56:20.368 INFO:teuthology.orchestra.run.smithi039.stdout: "status": "no active scrubs running", 2024-09-21T03:56:20.368 INFO:teuthology.orchestra.run.smithi039.stdout: "scrubs": {}, 2024-09-21T03:56:20.368 INFO:teuthology.orchestra.run.smithi039.stdout: "scrub_stats": { 2024-09-21T03:56:20.369 INFO:teuthology.orchestra.run.smithi039.stdout: "a7293877-4df2-4bbb-be12-176711d2ac8a": { 2024-09-21T03:56:20.369 INFO:teuthology.orchestra.run.smithi039.stdout: "start_time": "2024-09-21T03:56:04.159511+0000", 2024-09-21T03:56:20.369 INFO:teuthology.orchestra.run.smithi039.stdout: "path": "/", 2024-09-21T03:56:20.369 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_started": 0, 2024-09-21T03:56:20.369 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_passed": 0, 2024-09-21T03:56:20.369 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_failed": 0, 2024-09-21T03:56:20.369 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_skipped": 0 2024-09-21T03:56:20.369 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:20.369 INFO:teuthology.orchestra.run.smithi039.stdout: "02e16ae1-0bac-41e9-b95e-fc6b17d207ac": { 2024-09-21T03:56:20.369 INFO:teuthology.orchestra.run.smithi039.stdout: "start_time": "2024-09-21T03:54:31.853401+0000", 2024-09-21T03:56:20.369 INFO:teuthology.orchestra.run.smithi039.stdout: "path": "/", 2024-09-21T03:56:20.369 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_started": 0, 2024-09-21T03:56:20.369 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_passed": 0, 2024-09-21T03:56:20.369 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_failed": 0, 2024-09-21T03:56:20.370 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_skipped": 0 2024-09-21T03:56:20.370 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:20.370 INFO:teuthology.orchestra.run.smithi039.stdout: "e28dcd13-f2ac-4cb7-a346-58917d542b15": { 2024-09-21T03:56:20.370 INFO:teuthology.orchestra.run.smithi039.stdout: "start_time": "2024-09-21T03:54:18.221680+0000", 2024-09-21T03:56:20.370 INFO:teuthology.orchestra.run.smithi039.stdout: "path": "/", 2024-09-21T03:56:20.370 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_started": 0, 2024-09-21T03:56:20.370 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_passed": 0, 2024-09-21T03:56:20.370 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_failed": 0, 2024-09-21T03:56:20.370 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_skipped": 0 2024-09-21T03:56:20.370 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:20.370 INFO:teuthology.orchestra.run.smithi039.stdout: "ad87da79-0248-4e18-9694-8228cb95422f": { 2024-09-21T03:56:20.370 INFO:teuthology.orchestra.run.smithi039.stdout: "start_time": "2024-09-21T03:53:18.441907+0000", 2024-09-21T03:56:20.370 INFO:teuthology.orchestra.run.smithi039.stdout: "path": "/", 2024-09-21T03:56:20.370 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_started": 0, 2024-09-21T03:56:20.371 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_passed": 0, 2024-09-21T03:56:20.371 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_failed": 0, 2024-09-21T03:56:20.371 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_skipped": 0 2024-09-21T03:56:20.371 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:20.371 INFO:teuthology.orchestra.run.smithi039.stdout: "02509e4f-3911-4983-b16f-df8ded0b9998": { 2024-09-21T03:56:20.371 INFO:teuthology.orchestra.run.smithi039.stdout: "start_time": "2024-09-21T03:46:17.889576+0000", 2024-09-21T03:56:20.371 INFO:teuthology.orchestra.run.smithi039.stdout: "path": "/", 2024-09-21T03:56:20.371 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_started": 0, 2024-09-21T03:56:20.371 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_passed": 0, 2024-09-21T03:56:20.371 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_failed": 0, 2024-09-21T03:56:20.371 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_skipped": 0 2024-09-21T03:56:20.371 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:20.371 INFO:teuthology.orchestra.run.smithi039.stdout: "10728de8-c49d-4edc-be2b-639249255e94": { 2024-09-21T03:56:20.371 INFO:teuthology.orchestra.run.smithi039.stdout: "start_time": "2024-09-21T03:47:03.965623+0000", 2024-09-21T03:56:20.371 INFO:teuthology.orchestra.run.smithi039.stdout: "path": "/", 2024-09-21T03:56:20.372 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_started": 0, 2024-09-21T03:56:20.372 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_passed": 0, 2024-09-21T03:56:20.372 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_failed": 0, 2024-09-21T03:56:20.372 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_skipped": 0 2024-09-21T03:56:20.372 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:20.372 INFO:teuthology.orchestra.run.smithi039.stdout: "10e282c4-137c-4fb2-9af2-a83dc3945587": { 2024-09-21T03:56:20.372 INFO:teuthology.orchestra.run.smithi039.stdout: "start_time": "2024-09-21T03:55:17.955458+0000", 2024-09-21T03:56:20.372 INFO:teuthology.orchestra.run.smithi039.stdout: "path": "/", 2024-09-21T03:56:20.372 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_started": 0, 2024-09-21T03:56:20.372 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_passed": 0, 2024-09-21T03:56:20.372 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_failed": 0, 2024-09-21T03:56:20.372 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_skipped": 0 2024-09-21T03:56:20.372 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:20.372 INFO:teuthology.orchestra.run.smithi039.stdout: "9860c2f4-b7b1-47aa-865a-9a1c4434a5f6": { 2024-09-21T03:56:20.372 INFO:teuthology.orchestra.run.smithi039.stdout: "start_time": "2024-09-21T03:49:17.718867+0000", 2024-09-21T03:56:20.373 INFO:teuthology.orchestra.run.smithi039.stdout: "path": "/", 2024-09-21T03:56:20.373 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_started": 0, 2024-09-21T03:56:20.373 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_passed": 0, 2024-09-21T03:56:20.373 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_failed": 0, 2024-09-21T03:56:20.373 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_skipped": 0 2024-09-21T03:56:20.373 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:20.373 INFO:teuthology.orchestra.run.smithi039.stdout: "d9391d69-baac-453a-9d52-d046760c070d": { 2024-09-21T03:56:20.373 INFO:teuthology.orchestra.run.smithi039.stdout: "start_time": "2024-09-21T03:53:32.364622+0000", 2024-09-21T03:56:20.373 INFO:teuthology.orchestra.run.smithi039.stdout: "path": "/", 2024-09-21T03:56:20.373 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_started": 0, 2024-09-21T03:56:20.373 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_passed": 0, 2024-09-21T03:56:20.373 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_failed": 0, 2024-09-21T03:56:20.373 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_skipped": 0 2024-09-21T03:56:20.373 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:20.373 INFO:teuthology.orchestra.run.smithi039.stdout: "c7c700f3-2cd8-41ac-8381-b57dd1b37b02": { 2024-09-21T03:56:20.374 INFO:teuthology.orchestra.run.smithi039.stdout: "start_time": "2024-09-21T03:52:32.368858+0000", 2024-09-21T03:56:20.374 INFO:teuthology.orchestra.run.smithi039.stdout: "path": "/", 2024-09-21T03:56:20.374 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_started": 0, 2024-09-21T03:56:20.374 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_passed": 0, 2024-09-21T03:56:20.374 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_failed": 0, 2024-09-21T03:56:20.374 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_skipped": 0 2024-09-21T03:56:20.374 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:20.374 INFO:teuthology.orchestra.run.smithi039.stdout: "29411307-eb8d-4339-8d2b-47ebd32c0080": { 2024-09-21T03:56:20.374 INFO:teuthology.orchestra.run.smithi039.stdout: "start_time": "2024-09-21T03:47:38.092920+0000", 2024-09-21T03:56:20.374 INFO:teuthology.orchestra.run.smithi039.stdout: "path": "/", 2024-09-21T03:56:20.374 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_started": 0, 2024-09-21T03:56:20.374 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_passed": 0, 2024-09-21T03:56:20.374 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_failed": 0, 2024-09-21T03:56:20.374 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_skipped": 0 2024-09-21T03:56:20.374 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:20.375 INFO:teuthology.orchestra.run.smithi039.stdout: "db151ef6-34f3-4a6d-aa7a-19d07002ce84": { 2024-09-21T03:56:20.375 INFO:teuthology.orchestra.run.smithi039.stdout: "start_time": "2024-09-21T03:56:18.719213+0000", 2024-09-21T03:56:20.375 INFO:teuthology.orchestra.run.smithi039.stdout: "path": "/", 2024-09-21T03:56:20.375 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_started": 0, 2024-09-21T03:56:20.375 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_passed": 0, 2024-09-21T03:56:20.375 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_failed": 0, 2024-09-21T03:56:20.375 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_skipped": 0 2024-09-21T03:56:20.375 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:20.375 INFO:teuthology.orchestra.run.smithi039.stdout: "ede73e6e-3fd0-4a7f-b5bb-f68946f57eea": { 2024-09-21T03:56:20.375 INFO:teuthology.orchestra.run.smithi039.stdout: "start_time": "2024-09-21T03:51:32.235096+0000", 2024-09-21T03:56:20.375 INFO:teuthology.orchestra.run.smithi039.stdout: "path": "/", 2024-09-21T03:56:20.375 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_started": 0, 2024-09-21T03:56:20.375 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_passed": 0, 2024-09-21T03:56:20.375 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_failed": 0, 2024-09-21T03:56:20.375 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_skipped": 0 2024-09-21T03:56:20.376 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:20.376 INFO:teuthology.orchestra.run.smithi039.stdout: "e28ad743-2b22-45b3-8799-a638c2883623": { 2024-09-21T03:56:20.376 INFO:teuthology.orchestra.run.smithi039.stdout: "start_time": "2024-09-21T03:48:31.448821+0000", 2024-09-21T03:56:20.376 INFO:teuthology.orchestra.run.smithi039.stdout: "path": "/", 2024-09-21T03:56:20.376 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_started": 0, 2024-09-21T03:56:20.376 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_passed": 0, 2024-09-21T03:56:20.376 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_failed": 0, 2024-09-21T03:56:20.376 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_skipped": 0 2024-09-21T03:56:20.376 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:20.376 INFO:teuthology.orchestra.run.smithi039.stdout: "6e760f1f-9048-4f92-bd66-d708fbef75b3": { 2024-09-21T03:56:20.376 INFO:teuthology.orchestra.run.smithi039.stdout: "start_time": "2024-09-21T03:50:03.711918+0000", 2024-09-21T03:56:20.376 INFO:teuthology.orchestra.run.smithi039.stdout: "path": "/", 2024-09-21T03:56:20.376 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_started": 0, 2024-09-21T03:56:20.376 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_passed": 0, 2024-09-21T03:56:20.376 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_failed": 0, 2024-09-21T03:56:20.377 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_skipped": 0 2024-09-21T03:56:20.377 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:20.377 INFO:teuthology.orchestra.run.smithi039.stdout: "c7e2b8e4-057d-435f-b2bd-b633de98035f": { 2024-09-21T03:56:20.377 INFO:teuthology.orchestra.run.smithi039.stdout: "start_time": "2024-09-21T03:47:19.400305+0000", 2024-09-21T03:56:20.377 INFO:teuthology.orchestra.run.smithi039.stdout: "path": "/", 2024-09-21T03:56:20.377 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_started": 0, 2024-09-21T03:56:20.377 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_passed": 0, 2024-09-21T03:56:20.377 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_failed": 0, 2024-09-21T03:56:20.377 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_skipped": 0 2024-09-21T03:56:20.377 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:20.377 INFO:teuthology.orchestra.run.smithi039.stdout: "74ab917e-ecea-4778-a1c3-c184214fb24c": { 2024-09-21T03:56:20.377 INFO:teuthology.orchestra.run.smithi039.stdout: "start_time": "2024-09-21T03:50:17.726630+0000", 2024-09-21T03:56:20.377 INFO:teuthology.orchestra.run.smithi039.stdout: "path": "/", 2024-09-21T03:56:20.377 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_started": 0, 2024-09-21T03:56:20.377 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_passed": 0, 2024-09-21T03:56:20.378 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_failed": 0, 2024-09-21T03:56:20.378 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_skipped": 0 2024-09-21T03:56:20.378 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:20.378 INFO:teuthology.orchestra.run.smithi039.stdout: "160750d8-401f-4810-866d-e63e7e4bc3e0": { 2024-09-21T03:56:20.378 INFO:teuthology.orchestra.run.smithi039.stdout: "start_time": "2024-09-21T03:51:04.208673+0000", 2024-09-21T03:56:20.378 INFO:teuthology.orchestra.run.smithi039.stdout: "path": "/", 2024-09-21T03:56:20.378 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_started": 0, 2024-09-21T03:56:20.378 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_passed": 0, 2024-09-21T03:56:20.378 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_failed": 0, 2024-09-21T03:56:20.378 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_skipped": 0 2024-09-21T03:56:20.378 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:20.378 INFO:teuthology.orchestra.run.smithi039.stdout: "ae1fc4cf-5458-47d0-afcb-282b136dba15": { 2024-09-21T03:56:20.378 INFO:teuthology.orchestra.run.smithi039.stdout: "start_time": "2024-09-21T03:51:18.560378+0000", 2024-09-21T03:56:20.378 INFO:teuthology.orchestra.run.smithi039.stdout: "path": "/", 2024-09-21T03:56:20.378 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_started": 0, 2024-09-21T03:56:20.379 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_passed": 0, 2024-09-21T03:56:20.379 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_failed": 0, 2024-09-21T03:56:20.379 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_skipped": 0 2024-09-21T03:56:20.379 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:20.379 INFO:teuthology.orchestra.run.smithi039.stdout: "4bda6ee3-7237-4ee4-96ce-150ad50f26dc": { 2024-09-21T03:56:20.379 INFO:teuthology.orchestra.run.smithi039.stdout: "start_time": "2024-09-21T03:52:18.622141+0000", 2024-09-21T03:56:20.379 INFO:teuthology.orchestra.run.smithi039.stdout: "path": "/", 2024-09-21T03:56:20.379 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_started": 0, 2024-09-21T03:56:20.379 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_passed": 0, 2024-09-21T03:56:20.379 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_failed": 0, 2024-09-21T03:56:20.379 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_skipped": 0 2024-09-21T03:56:20.379 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-21T03:56:20.379 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-21T03:56:20.379 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-09-21T03:56:20.380 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:20.362+0000 7fcf3f991640 1 -- 172.21.15.39:0/3925517934 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7fcf240b6680 msgr2=0x7fcf240b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:56:20.380 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:20.362+0000 7fcf3f991640 1 --2- 172.21.15.39:0/3925517934 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7fcf240b6680 0x7fcf240b8b90 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x55cd269682d0 tx=0x7fcf3400b390 comp rx=0 tx=0).stop 2024-09-21T03:56:20.380 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:20.362+0000 7fcf3f991640 1 -- 172.21.15.39:0/3925517934 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcf38107be0 msgr2=0x7fcf381a62d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:56:20.380 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:20.362+0000 7fcf3f991640 1 --2- 172.21.15.39:0/3925517934 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcf38107be0 0x7fcf381a62d0 secure :-1 s=READY pgs=1320 cs=0 l=1 rev1=1 crypto rx=0x7fcf30005cc0 tx=0x7fcf30003c20 comp rx=0 tx=0).stop 2024-09-21T03:56:20.380 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:20.362+0000 7fcf3f991640 1 -- 172.21.15.39:0/3925517934 shutdown_connections 2024-09-21T03:56:20.380 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:20.362+0000 7fcf3f991640 1 --2- 172.21.15.39:0/3925517934 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7fcf240b6680 0x7fcf240b8b90 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:20.380 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:20.362+0000 7fcf3f991640 1 --2- 172.21.15.39:0/3925517934 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcf38107be0 0x7fcf381a62d0 unknown :-1 s=CLOSED pgs=1320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:20.380 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:20.362+0000 7fcf3f991640 1 --2- 172.21.15.39:0/3925517934 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcf381030f0 0x7fcf381a1030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:20.380 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:20.362+0000 7fcf3f991640 1 --2- 172.21.15.39:0/3925517934 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcf381027a0 0x7fcf381a0a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:20.380 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:20.362+0000 7fcf3f991640 1 -- 172.21.15.39:0/3925517934 >> 172.21.15.39:0/3925517934 conn(0x7fcf38079720 msgr2=0x7fcf381055f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:56:20.381 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:20.362+0000 7fcf3f991640 1 -- 172.21.15.39:0/3925517934 shutdown_connections 2024-09-21T03:56:20.381 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:20.362+0000 7fcf3f991640 1 -- 172.21.15.39:0/3925517934 wait complete. 2024-09-21T03:56:20.468 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:56:20 smithi161 ceph-mon[34533]: from='client.18240 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:56:20.468 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:56:20 smithi161 ceph-mon[34533]: from='client.18264 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:56:20.468 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:56:20 smithi161 ceph-mon[34533]: scrub summary: idle 2024-09-21T03:56:20.468 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:56:20 smithi161 ceph-mon[34533]: osdmap e87: 12 total, 12 up, 12 in 2024-09-21T03:56:20.474 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:56:20 smithi150 ceph-mon[34289]: from='client.18240 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:56:20.475 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:56:20 smithi150 ceph-mon[34289]: from='client.18264 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:56:20.475 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:56:20 smithi150 ceph-mon[34289]: scrub summary: idle 2024-09-21T03:56:20.475 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:56:20 smithi150 ceph-mon[34289]: osdmap e87: 12 total, 12 up, 12 in 2024-09-21T03:56:21.185 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-21T03:56:21.186 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 51116d5c-77ca-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T03:56:21.447 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/51116d5c-77ca-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:56:21.468 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:56:21 smithi161 ceph-mon[34533]: pgmap v641: 97 pgs: 97 active+clean; 10 MiB data, 525 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 204 B/s wr, 10 op/s 2024-09-21T03:56:21.474 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:56:21 smithi150 ceph-mon[34289]: pgmap v641: 97 pgs: 97 active+clean; 10 MiB data, 525 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 204 B/s wr, 10 op/s 2024-09-21T03:56:21.568 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:56:21 smithi039 ceph-mon[31211]: pgmap v641: 97 pgs: 97 active+clean; 10 MiB data, 525 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 204 B/s wr, 10 op/s 2024-09-21T03:56:21.950 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:21.947+0000 7f8375ba4640 1 -- 172.21.15.39:0/4216527610 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8370101ba0 msgr2=0x7f8370101fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:56:21.951 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:21.947+0000 7f8375ba4640 1 --2- 172.21.15.39:0/4216527610 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8370101ba0 0x7f8370101fb0 secure :-1 s=READY pgs=1325 cs=0 l=1 rev1=1 crypto rx=0x7f8360009900 tx=0x7f836002a410 comp rx=0 tx=0).stop 2024-09-21T03:56:21.951 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:21.947+0000 7f8375ba4640 1 -- 172.21.15.39:0/4216527610 shutdown_connections 2024-09-21T03:56:21.951 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:21.947+0000 7f8375ba4640 1 --2- 172.21.15.39:0/4216527610 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8370109ec0 0x7f8370110620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:21.951 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:21.947+0000 7f8375ba4640 1 --2- 172.21.15.39:0/4216527610 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8370109570 0x7f8370109980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:21.951 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:21.947+0000 7f8375ba4640 1 --2- 172.21.15.39:0/4216527610 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8370101ba0 0x7f8370101fb0 unknown :-1 s=CLOSED pgs=1325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:21.951 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:21.947+0000 7f8375ba4640 1 -- 172.21.15.39:0/4216527610 >> 172.21.15.39:0/4216527610 conn(0x7f8370077800 msgr2=0x7f8370077c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:56:21.951 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:21.947+0000 7f8375ba4640 1 -- 172.21.15.39:0/4216527610 shutdown_connections 2024-09-21T03:56:21.951 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:21.947+0000 7f8375ba4640 1 -- 172.21.15.39:0/4216527610 wait complete. 2024-09-21T03:56:21.953 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:21.948+0000 7f8375ba4640 1 Processor -- start 2024-09-21T03:56:21.953 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:21.949+0000 7f8375ba4640 1 -- start start 2024-09-21T03:56:21.953 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:21.949+0000 7f8375ba4640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8370101ba0 0x7f837019a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:56:21.953 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:21.949+0000 7f8375ba4640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8370109570 0x7f837019aa20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:56:21.953 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:21.949+0000 7f8375ba4640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8370109ec0 0x7f837019fcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:56:21.953 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:21.949+0000 7f8375ba4640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f8370113ab0 con 0x7f8370109ec0 2024-09-21T03:56:21.953 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:21.949+0000 7f8375ba4640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f8370113930 con 0x7f8370109570 2024-09-21T03:56:21.954 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:21.949+0000 7f8375ba4640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f8370113c30 con 0x7f8370101ba0 2024-09-21T03:56:21.954 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:21.949+0000 7f836ffff640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8370109ec0 0x7f837019fcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:56:21.954 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:21.949+0000 7f836ffff640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8370109ec0 0x7f837019fcc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:55784/0 (socket says 172.21.15.39:55784) 2024-09-21T03:56:21.954 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:21.949+0000 7f836ffff640 1 -- 172.21.15.39:0/2125361503 learned_addr learned my addr 172.21.15.39:0/2125361503 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:56:21.954 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:21.949+0000 7f836effd640 1 --2- 172.21.15.39:0/2125361503 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8370109570 0x7f837019aa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:56:21.954 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:21.949+0000 7f836f7fe640 1 --2- 172.21.15.39:0/2125361503 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8370101ba0 0x7f837019a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:56:21.954 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:21.949+0000 7f836ffff640 1 -- 172.21.15.39:0/2125361503 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8370101ba0 msgr2=0x7f837019a480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:56:21.954 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:21.949+0000 7f836ffff640 1 --2- 172.21.15.39:0/2125361503 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8370101ba0 0x7f837019a480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:21.954 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:21.949+0000 7f836ffff640 1 -- 172.21.15.39:0/2125361503 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8370109570 msgr2=0x7f837019aa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:56:21.954 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:21.949+0000 7f836ffff640 1 --2- 172.21.15.39:0/2125361503 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8370109570 0x7f837019aa20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:21.954 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:21.949+0000 7f836ffff640 1 -- 172.21.15.39:0/2125361503 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f83701a03f0 con 0x7f8370109ec0 2024-09-21T03:56:21.954 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:21.950+0000 7f836ffff640 1 --2- 172.21.15.39:0/2125361503 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8370109ec0 0x7f837019fcc0 secure :-1 s=READY pgs=1326 cs=0 l=1 rev1=1 crypto rx=0x7f836800eaa0 tx=0x7f836800ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:56:21.954 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:21.950+0000 7f836cff9640 1 -- 172.21.15.39:0/2125361503 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f836800cdb0 con 0x7f8370109ec0 2024-09-21T03:56:21.954 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:21.950+0000 7f836cff9640 1 -- 172.21.15.39:0/2125361503 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8368005020 con 0x7f8370109ec0 2024-09-21T03:56:21.955 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:21.950+0000 7f8375ba4640 1 -- 172.21.15.39:0/2125361503 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f83701a06e0 con 0x7f8370109ec0 2024-09-21T03:56:21.955 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:21.950+0000 7f836cff9640 1 -- 172.21.15.39:0/2125361503 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83680116d0 con 0x7f8370109ec0 2024-09-21T03:56:21.955 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:21.950+0000 7f8375ba4640 1 -- 172.21.15.39:0/2125361503 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f83701a6e40 con 0x7f8370109ec0 2024-09-21T03:56:21.955 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:21.951+0000 7f8375ba4640 1 -- 172.21.15.39:0/2125361503 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8370101fb0 con 0x7f8370109ec0 2024-09-21T03:56:21.956 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:21.953+0000 7f836cff9640 1 -- 172.21.15.39:0/2125361503 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 27) ==== 154643+0+0 (secure 0 0 0) 0x7f83680051d0 con 0x7f8370109ec0 2024-09-21T03:56:21.956 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:21.953+0000 7f836cff9640 1 --2- 172.21.15.39:0/2125361503 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f835c0b6700 0x7f835c0b8c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:56:21.956 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:21.954+0000 7f836f7fe640 1 --2- 172.21.15.39:0/2125361503 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f835c0b6700 0x7f835c0b8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:56:21.956 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:21.954+0000 7f836cff9640 1 -- 172.21.15.39:0/2125361503 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8528+0+0 (secure 0 0 0) 0x7f83680e7ae0 con 0x7f8370109ec0 2024-09-21T03:56:21.957 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:21.954+0000 7f836f7fe640 1 --2- 172.21.15.39:0/2125361503 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f835c0b6700 0x7f835c0b8c10 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f8360031440 tx=0x7f8360009560 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:56:21.959 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:21.957+0000 7f836cff9640 1 -- 172.21.15.39:0/2125361503 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f8368016030 con 0x7f8370109ec0 2024-09-21T03:56:22.183 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:22.180+0000 7f8375ba4640 1 -- 172.21.15.39:0/2125361503 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f837019b5b0 con 0x7f8370109ec0 2024-09-21T03:56:22.183 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:22.181+0000 7f836cff9640 1 -- 172.21.15.39:0/2125361503 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11978 (secure 0 0 0) 0x7f837019b5b0 con 0x7f8370109ec0 2024-09-21T03:56:22.184 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T03:56:22.184 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":29,"btime":"2024-09-21T03:46:08:982881+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14790,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6835/508995295","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":508995295},{"type":"v1","addr":"172.21.15.39:6835","nonce":508995295}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14802,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6837/3915912607","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":3915912607},{"type":"v1","addr":"172.21.15.39:6837","nonce":3915912607}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":29,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T03:44:15.009872+0000","modified":"2024-09-21T03:46:08.430113+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24695,"mds_1":24667,"mds_2":24659,"mds_3":24646,"mds_4":14868},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14856":{"gid":14856,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.39:6839/185509192","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6838","nonce":185509192},{"type":"v1","addr":"172.21.15.39:6839","nonce":185509192}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14868":{"gid":14868,"name":"a","rank":4,"incarnation":25,"state":"up:active","state_seq":7,"addr":"172.21.15.39:6841/4247190456","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6840","nonce":4247190456},{"type":"v1","addr":"172.21.15.39:6841","nonce":4247190456}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24599":{"gid":24599,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.150:6833/3114643217","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":3114643217},{"type":"v1","addr":"172.21.15.150:6833","nonce":3114643217}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24601":{"gid":24601,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.161:6833/2963990654","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":2963990654},{"type":"v1","addr":"172.21.15.161:6833","nonce":2963990654}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24632":{"gid":24632,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.150:6835/724594008","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6834","nonce":724594008},{"type":"v1","addr":"172.21.15.150:6835","nonce":724594008}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24640":{"gid":24640,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.161:6835/3747882110","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":3747882110},{"type":"v1","addr":"172.21.15.161:6835","nonce":3747882110}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24646":{"gid":24646,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":10,"addr":"172.21.15.161:6837/3451544597","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6836","nonce":3451544597},{"type":"v1","addr":"172.21.15.161:6837","nonce":3451544597}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24659":{"gid":24659,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.150:6837/4017043144","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6836","nonce":4017043144},{"type":"v1","addr":"172.21.15.150:6837","nonce":4017043144}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24667":{"gid":24667,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.161:6839/2190107270","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6838","nonce":2190107270},{"type":"v1","addr":"172.21.15.161:6839","nonce":2190107270}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24695":{"gid":24695,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.150:6839/2949386907","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6838","nonce":2949386907},{"type":"v1","addr":"172.21.15.150:6839","nonce":2949386907}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24695,"qdb_cluster":[24695,24667,24659,24646,14868]},"id":1}]} 2024-09-21T03:56:22.184 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 29 2024-09-21T03:56:22.186 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:22.183+0000 7f8375ba4640 1 -- 172.21.15.39:0/2125361503 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f835c0b6700 msgr2=0x7f835c0b8c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:56:22.186 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:22.183+0000 7f8375ba4640 1 --2- 172.21.15.39:0/2125361503 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f835c0b6700 0x7f835c0b8c10 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f8360031440 tx=0x7f8360009560 comp rx=0 tx=0).stop 2024-09-21T03:56:22.186 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:22.183+0000 7f8375ba4640 1 -- 172.21.15.39:0/2125361503 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8370109ec0 msgr2=0x7f837019fcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:56:22.186 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:22.183+0000 7f8375ba4640 1 --2- 172.21.15.39:0/2125361503 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8370109ec0 0x7f837019fcc0 secure :-1 s=READY pgs=1326 cs=0 l=1 rev1=1 crypto rx=0x7f836800eaa0 tx=0x7f836800ef60 comp rx=0 tx=0).stop 2024-09-21T03:56:22.186 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:22.183+0000 7f8375ba4640 1 -- 172.21.15.39:0/2125361503 shutdown_connections 2024-09-21T03:56:22.186 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:22.183+0000 7f8375ba4640 1 --2- 172.21.15.39:0/2125361503 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f835c0b6700 0x7f835c0b8c10 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:22.186 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:22.183+0000 7f8375ba4640 1 --2- 172.21.15.39:0/2125361503 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8370109ec0 0x7f837019fcc0 unknown :-1 s=CLOSED pgs=1326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:22.187 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:22.183+0000 7f8375ba4640 1 --2- 172.21.15.39:0/2125361503 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8370109570 0x7f837019aa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:22.187 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:22.183+0000 7f8375ba4640 1 --2- 172.21.15.39:0/2125361503 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8370101ba0 0x7f837019a480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:22.187 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:22.183+0000 7f8375ba4640 1 -- 172.21.15.39:0/2125361503 >> 172.21.15.39:0/2125361503 conn(0x7f8370077800 msgr2=0x7f8370100350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:56:22.187 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:22.183+0000 7f8375ba4640 1 -- 172.21.15.39:0/2125361503 shutdown_connections 2024-09-21T03:56:22.187 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:22.183+0000 7f8375ba4640 1 -- 172.21.15.39:0/2125361503 wait complete. 2024-09-21T03:56:23.013 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 51116d5c-77ca-11ef-baf5-efdc52797490 -- ceph tell mds.1:4 damage ls 2024-09-21T03:56:23.275 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/51116d5c-77ca-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:56:23.334 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:56:23 smithi039 ceph-mon[31211]: pgmap v643: 97 pgs: 97 active+clean; 10 MiB data, 525 MiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 204 B/s wr, 13 op/s 2024-09-21T03:56:23.334 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:56:23 smithi039 ceph-mon[31211]: from='client.? 172.21.15.39:0/2125361503' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:56:23.468 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:56:23 smithi161 ceph-mon[34533]: pgmap v643: 97 pgs: 97 active+clean; 10 MiB data, 525 MiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 204 B/s wr, 13 op/s 2024-09-21T03:56:23.468 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:56:23 smithi161 ceph-mon[34533]: from='client.? 172.21.15.39:0/2125361503' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:56:23.475 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:56:23 smithi150 ceph-mon[34289]: pgmap v643: 97 pgs: 97 active+clean; 10 MiB data, 525 MiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 204 B/s wr, 13 op/s 2024-09-21T03:56:23.475 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:56:23 smithi150 ceph-mon[34289]: from='client.? 172.21.15.39:0/2125361503' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:56:23.794 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:23.792+0000 7f9a22bc0640 1 -- 172.21.15.39:0/3907201953 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9a1c102ae0 msgr2=0x7f9a1c102ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:56:23.794 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:23.792+0000 7f9a22bc0640 1 --2- 172.21.15.39:0/3907201953 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9a1c102ae0 0x7f9a1c102ef0 secure :-1 s=READY pgs=1327 cs=0 l=1 rev1=1 crypto rx=0x7f9a10009900 tx=0x7f9a1002a410 comp rx=0 tx=0).stop 2024-09-21T03:56:23.794 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:23.792+0000 7f9a22bc0640 1 -- 172.21.15.39:0/3907201953 shutdown_connections 2024-09-21T03:56:23.795 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:23.792+0000 7f9a22bc0640 1 --2- 172.21.15.39:0/3907201953 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9a1c108d20 0x7f9a1c10f550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:23.795 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:23.792+0000 7f9a22bc0640 1 --2- 172.21.15.39:0/3907201953 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9a1c1083d0 0x7f9a1c1087e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:23.795 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:23.792+0000 7f9a22bc0640 1 --2- 172.21.15.39:0/3907201953 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9a1c102ae0 0x7f9a1c102ef0 unknown :-1 s=CLOSED pgs=1327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:23.795 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:23.792+0000 7f9a22bc0640 1 -- 172.21.15.39:0/3907201953 >> 172.21.15.39:0/3907201953 conn(0x7f9a1c0797b0 msgr2=0x7f9a1c079bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:56:23.795 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:23.792+0000 7f9a22bc0640 1 -- 172.21.15.39:0/3907201953 shutdown_connections 2024-09-21T03:56:23.795 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:23.792+0000 7f9a22bc0640 1 -- 172.21.15.39:0/3907201953 wait complete. 2024-09-21T03:56:23.797 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:23.793+0000 7f9a22bc0640 1 Processor -- start 2024-09-21T03:56:23.797 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:23.793+0000 7f9a22bc0640 1 -- start start 2024-09-21T03:56:23.797 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:23.793+0000 7f9a22bc0640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9a1c102ae0 0x7f9a1c19a540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:56:23.797 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:23.793+0000 7f9a22bc0640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9a1c1083d0 0x7f9a1c19aae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:56:23.797 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:23.793+0000 7f9a22bc0640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9a1c108d20 0x7f9a1c19fd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:56:23.797 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:23.793+0000 7f9a22bc0640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f9a1c112a20 con 0x7f9a1c1083d0 2024-09-21T03:56:23.797 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:23.793+0000 7f9a22bc0640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f9a1c1128a0 con 0x7f9a1c108d20 2024-09-21T03:56:23.797 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:23.793+0000 7f9a22bc0640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f9a1c112ba0 con 0x7f9a1c102ae0 2024-09-21T03:56:23.797 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:23.793+0000 7f9a1bfff640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9a1c1083d0 0x7f9a1c19aae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:56:23.797 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:23.794+0000 7f9a1bfff640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9a1c1083d0 0x7f9a1c19aae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:55812/0 (socket says 172.21.15.39:55812) 2024-09-21T03:56:23.797 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:23.794+0000 7f9a1bfff640 1 -- 172.21.15.39:0/3243482448 learned_addr learned my addr 172.21.15.39:0/3243482448 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:56:23.797 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:23.794+0000 7f9a2115d640 1 --2- 172.21.15.39:0/3243482448 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9a1c108d20 0x7f9a1c19fd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:56:23.798 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:23.794+0000 7f9a2095c640 1 --2- 172.21.15.39:0/3243482448 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9a1c102ae0 0x7f9a1c19a540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:56:23.798 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:23.794+0000 7f9a1bfff640 1 -- 172.21.15.39:0/3243482448 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9a1c102ae0 msgr2=0x7f9a1c19a540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:56:23.798 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:23.794+0000 7f9a1bfff640 1 --2- 172.21.15.39:0/3243482448 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9a1c102ae0 0x7f9a1c19a540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:23.798 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:23.794+0000 7f9a1bfff640 1 -- 172.21.15.39:0/3243482448 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9a1c108d20 msgr2=0x7f9a1c19fd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:56:23.798 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:23.794+0000 7f9a1bfff640 1 --2- 172.21.15.39:0/3243482448 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9a1c108d20 0x7f9a1c19fd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:23.798 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:23.794+0000 7f9a1bfff640 1 -- 172.21.15.39:0/3243482448 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9a1c1a04b0 con 0x7f9a1c1083d0 2024-09-21T03:56:23.798 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:23.794+0000 7f9a1bfff640 1 --2- 172.21.15.39:0/3243482448 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9a1c1083d0 0x7f9a1c19aae0 secure :-1 s=READY pgs=1328 cs=0 l=1 rev1=1 crypto rx=0x7f9a08006fd0 tx=0x7f9a0800b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:56:23.798 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:23.794+0000 7f9a19ffb640 1 -- 172.21.15.39:0/3243482448 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a08011830 con 0x7f9a1c1083d0 2024-09-21T03:56:23.798 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:23.794+0000 7f9a22bc0640 1 -- 172.21.15.39:0/3243482448 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9a1c1a07a0 con 0x7f9a1c1083d0 2024-09-21T03:56:23.798 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:23.794+0000 7f9a19ffb640 1 -- 172.21.15.39:0/3243482448 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9a08016070 con 0x7f9a1c1083d0 2024-09-21T03:56:23.799 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:23.794+0000 7f9a19ffb640 1 -- 172.21.15.39:0/3243482448 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a08010e00 con 0x7f9a1c1083d0 2024-09-21T03:56:23.799 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:23.795+0000 7f9a22bc0640 1 -- 172.21.15.39:0/3243482448 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9a1c1a6f00 con 0x7f9a1c1083d0 2024-09-21T03:56:23.801 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:23.797+0000 7f9a29134740 1 Processor -- start 2024-09-21T03:56:23.801 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:23.797+0000 7f9a29134740 1 -- start start 2024-09-21T03:56:23.801 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:23.798+0000 7f9a29134740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5648d1cab070 0x5648d20526a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:56:23.801 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:23.798+0000 7f9a29134740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5648d1c821b0 0x5648d2052be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:56:23.801 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:23.798+0000 7f9a29134740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5648d1c8a390 0x5648d2053120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:56:23.801 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:23.798+0000 7f9a29134740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x5648d1fc8ea0 con 0x5648d1c821b0 2024-09-21T03:56:23.801 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:23.798+0000 7f9a29134740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x5648d1fb82d0 con 0x5648d1cab070 2024-09-21T03:56:23.801 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:23.798+0000 7f9a29134740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x5648d1fbc0e0 con 0x5648d1c8a390 2024-09-21T03:56:23.801 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:23.798+0000 7f9a19ffb640 1 -- 172.21.15.39:0/3243482448 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 27) ==== 154643+0+0 (secure 0 0 0) 0x7f9a0801b460 con 0x7f9a1c1083d0 2024-09-21T03:56:23.801 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:23.798+0000 7f9a2095c640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5648d1c821b0 0x5648d2052be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:56:23.802 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:23.798+0000 7f9a2095c640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5648d1c821b0 0x5648d2052be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:55818/0 (socket says 172.21.15.39:55818) 2024-09-21T03:56:23.802 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:23.798+0000 7f9a2095c640 1 -- 172.21.15.39:0/1579636321 learned_addr learned my addr 172.21.15.39:0/1579636321 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:56:23.802 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:23.798+0000 7f9a1bfff640 1 --2- 172.21.15.39:0/1579636321 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5648d1c8a390 0x5648d2053120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:56:23.802 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:23.798+0000 7f9a2115d640 1 --2- 172.21.15.39:0/1579636321 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5648d1cab070 0x5648d20526a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:56:23.802 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:23.798+0000 7f9a2095c640 1 -- 172.21.15.39:0/1579636321 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5648d1c8a390 msgr2=0x5648d2053120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:56:23.802 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:23.798+0000 7f9a2095c640 1 --2- 172.21.15.39:0/1579636321 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5648d1c8a390 0x5648d2053120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:23.802 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:23.798+0000 7f9a2095c640 1 -- 172.21.15.39:0/1579636321 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5648d1cab070 msgr2=0x5648d20526a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:56:23.802 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:23.798+0000 7f9a2095c640 1 --2- 172.21.15.39:0/1579636321 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5648d1cab070 0x5648d20526a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:23.802 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:23.798+0000 7f9a2095c640 1 -- 172.21.15.39:0/1579636321 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5648d1d16650 con 0x5648d1c821b0 2024-09-21T03:56:23.802 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:23.799+0000 7f9a2095c640 1 --2- 172.21.15.39:0/1579636321 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5648d1c821b0 0x5648d2052be0 secure :-1 s=READY pgs=1329 cs=0 l=1 rev1=1 crypto rx=0x7f9a1002a930 tx=0x7f9a10032ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad33740c6544aff7 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:56:23.802 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:23.799+0000 7f9a19ffb640 1 --2- 172.21.15.39:0/3243482448 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f9a0c0b6680 0x7f9a0c0b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:56:23.803 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:23.799+0000 7f9a06ffd640 1 -- 172.21.15.39:0/1579636321 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a1003f070 con 0x5648d1c821b0 2024-09-21T03:56:23.803 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:23.799+0000 7f9a06ffd640 1 -- 172.21.15.39:0/1579636321 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9a100316a0 con 0x5648d1c821b0 2024-09-21T03:56:23.803 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:23.799+0000 7f9a06ffd640 1 -- 172.21.15.39:0/1579636321 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a10035070 con 0x5648d1c821b0 2024-09-21T03:56:23.803 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:23.799+0000 7f9a2115d640 1 --2- 172.21.15.39:0/3243482448 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f9a0c0b6680 0x7f9a0c0b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:56:23.803 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:23.799+0000 7f9a19ffb640 1 -- 172.21.15.39:0/3243482448 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8528+0+0 (secure 0 0 0) 0x7f9a080e6c40 con 0x7f9a1c1083d0 2024-09-21T03:56:23.807 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:23.800+0000 7f9a29134740 1 -- 172.21.15.39:0/1579636321 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5648d1c821b0 msgr2=0x5648d2052be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:56:23.807 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:23.800+0000 7f9a2115d640 1 --2- 172.21.15.39:0/3243482448 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f9a0c0b6680 0x7f9a0c0b8b90 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f9a14005fd0 tx=0x7f9a14009600 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:56:23.807 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:23.800+0000 7f9a29134740 1 --2- 172.21.15.39:0/1579636321 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5648d1c821b0 0x5648d2052be0 secure :-1 s=READY pgs=1329 cs=0 l=1 rev1=1 crypto rx=0x7f9a1002a930 tx=0x7f9a10032ce0 comp rx=0 tx=0).stop 2024-09-21T03:56:23.807 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:23.800+0000 7f9a29134740 1 -- 172.21.15.39:0/1579636321 shutdown_connections 2024-09-21T03:56:23.807 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:23.800+0000 7f9a29134740 1 --2- 172.21.15.39:0/1579636321 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5648d1c8a390 0x5648d2053120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:23.807 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:23.800+0000 7f9a29134740 1 --2- 172.21.15.39:0/1579636321 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5648d1c821b0 0x5648d2052be0 unknown :-1 s=CLOSED pgs=1329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:23.807 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:23.800+0000 7f9a29134740 1 --2- 172.21.15.39:0/1579636321 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5648d1cab070 0x5648d20526a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:23.807 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:23.800+0000 7f9a29134740 1 -- 172.21.15.39:0/1579636321 >> 172.21.15.39:0/1579636321 conn(0x5648d1c9f460 msgr2=0x5648d2050590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:56:23.807 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:23.800+0000 7f9a29134740 1 -- 172.21.15.39:0/1579636321 shutdown_connections 2024-09-21T03:56:23.807 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:23.800+0000 7f9a29134740 1 -- 172.21.15.39:0/1579636321 wait complete. 2024-09-21T03:56:23.807 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:23.800+0000 7f9a29134740 1 -- start start 2024-09-21T03:56:23.807 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:23.800+0000 7f9a29134740 1 Processor -- start 2024-09-21T03:56:23.807 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:23.801+0000 7f9a29134740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5648d1c821b0 0x5648d2062110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:56:23.808 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:23.801+0000 7f9a29134740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5648d1c8a390 0x5648d2062650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:56:23.808 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:23.801+0000 7f9a29134740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5648d1cab070 0x5648d2062bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:56:23.808 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:23.801+0000 7f9a29134740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x5648d1e19700 con 0x5648d1c8a390 2024-09-21T03:56:23.808 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:23.801+0000 7f9a29134740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x5648d1f53160 con 0x5648d1c821b0 2024-09-21T03:56:23.808 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:23.801+0000 7f9a29134740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x5648d2065180 con 0x5648d1cab070 2024-09-21T03:56:23.808 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:23.801+0000 7f9a29134740 10 client.0 _ll_register_callbacks cb 0x5648d1f6c9e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:56:23.808 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:23.801+0000 7f9a1bfff640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5648d1c8a390 0x5648d2062650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:56:23.808 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:23.801+0000 7f9a1bfff640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5648d1c8a390 0x5648d2062650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:55834/0 (socket says 172.21.15.39:55834) 2024-09-21T03:56:23.808 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:23.801+0000 7f9a1bfff640 1 -- 172.21.15.39:0/3843046713 learned_addr learned my addr 172.21.15.39:0/3843046713 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:56:23.808 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:23.801+0000 7f9a2095c640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5648d1c821b0 0x5648d2062110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:56:23.808 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:23.801+0000 7f9a2115d640 1 --2- 172.21.15.39:0/3843046713 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5648d1cab070 0x5648d2062bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:56:23.808 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:23.801+0000 7f9a1bfff640 1 -- 172.21.15.39:0/3843046713 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5648d1cab070 msgr2=0x5648d2062bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:56:23.808 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:23.801+0000 7f9a1bfff640 1 --2- 172.21.15.39:0/3843046713 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5648d1cab070 0x5648d2062bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:23.808 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:23.801+0000 7f9a1bfff640 1 -- 172.21.15.39:0/3843046713 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5648d1c821b0 msgr2=0x5648d2062110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:56:23.809 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:23.801+0000 7f9a1bfff640 1 --2- 172.21.15.39:0/3843046713 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5648d1c821b0 0x5648d2062110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:23.809 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:23.801+0000 7f9a1bfff640 1 -- 172.21.15.39:0/3843046713 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5648d1f80c90 con 0x5648d1c8a390 2024-09-21T03:56:23.809 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:23.801+0000 7f9a2095c640 1 --2- 172.21.15.39:0/3843046713 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5648d1c821b0 0x5648d2062110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:56:23.809 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:23.802+0000 7f9a1bfff640 1 --2- 172.21.15.39:0/3843046713 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5648d1c8a390 0x5648d2062650 secure :-1 s=READY pgs=1330 cs=0 l=1 rev1=1 crypto rx=0x7f9a080aca20 tx=0x7f9a080047d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=23c56a8bdc96d6d0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:56:23.809 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:23.802+0000 7f99fbfff640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-09-21T03:56:23.809 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:23.802+0000 7f99fbfff640 1 -- 172.21.15.39:0/3843046713 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a080ac3e0 con 0x5648d1c8a390 2024-09-21T03:56:23.809 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:23.802+0000 7f99fbfff640 1 -- 172.21.15.39:0/3843046713 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8528+0+0 (secure 0 0 0) 0x7f9a080e4ae0 con 0x5648d1c8a390 2024-09-21T03:56:23.809 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:23.802+0000 7f9a29134740 1 -- 172.21.15.39:0/3843046713 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9a100316a0 con 0x5648d1c8a390 2024-09-21T03:56:23.809 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:23.802+0000 7f99fbfff640 1 -- 172.21.15.39:0/3843046713 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f9a080ac200 con 0x5648d1c8a390 2024-09-21T03:56:23.809 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:23.802+0000 7f9a29134740 10 client.18300 fetch_fsmap learned FSMap version 29 2024-09-21T03:56:23.809 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:23.802+0000 7f9a29134740 1 -- 172.21.15.39:0/3843046713 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5648d2070ef0 con 0x5648d1c8a390 2024-09-21T03:56:23.809 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:23.802+0000 7f99fbfff640 1 -- 172.21.15.39:0/3843046713 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9a100316a0 con 0x5648d1c8a390 2024-09-21T03:56:23.809 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:23.802+0000 7f99fbfff640 1 -- 172.21.15.39:0/3843046713 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f9a080ee040 con 0x5648d1c8a390 2024-09-21T03:56:23.809 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:23.803+0000 7f9a29134740 10 client.18300 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T03:56:23.810 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:23.803+0000 7f9a29134740 10 client.18300 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.a 2024-09-21T03:56:23.810 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:23.803+0000 7f9a29134740 20 client.18300 populate_metadata read hostname 'smithi039' 2024-09-21T03:56:23.810 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:23.803+0000 7f9a29134740 1 --2- 172.21.15.39:0/3843046713 >> [v2:172.21.15.39:6840/4247190456,v1:172.21.15.39:6841/4247190456] conn(0x5648d1c699f0 0x5648d20743b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:56:23.810 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:23.803+0000 7f9a29134740 4 client.18300 mds_command: new command op to 14868 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:56:23.810 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:23.803+0000 7f9a29134740 1 -- 172.21.15.39:0/3843046713 --> [v2:172.21.15.39:6840/4247190456,v1:172.21.15.39:6841/4247190456] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5648d1fcadf0 con 0x5648d1c699f0 2024-09-21T03:56:23.810 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:23.803+0000 7f9a2095c640 1 --2- 172.21.15.39:0/3843046713 >> [v2:172.21.15.39:6840/4247190456,v1:172.21.15.39:6841/4247190456] conn(0x5648d1c699f0 0x5648d20743b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:56:23.810 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:23.803+0000 7f99fbfff640 10 client.18300 ms_handle_connect on v2:172.21.15.39:6840/4247190456 2024-09-21T03:56:23.810 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:23.803+0000 7f9a2095c640 1 --2- 172.21.15.39:0/3843046713 >> [v2:172.21.15.39:6840/4247190456,v1:172.21.15.39:6841/4247190456] conn(0x5648d1c699f0 0x5648d20743b0 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=cd5841b505260071 server_cookie=9cdd4c77961bef23 in_seq=0 out_seq=0 2024-09-21T03:56:23.810 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:23.804+0000 7f99fbfff640 1 -- 172.21.15.39:0/3843046713 <== mds.4 v2:172.21.15.39:6840/4247190456 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5648d1fcadf0 con 0x5648d1c699f0 2024-09-21T03:56:23.810 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:23.804+0000 7f99fbfff640 10 client.18300 handle_command_reply: tid=0 2024-09-21T03:56:23.810 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:23.804+0000 7f9a29134740 1 client.18300 shutdown 2024-09-21T03:56:23.810 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:23.804+0000 7f9a29134740 1 -- 172.21.15.39:0/3843046713 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5648d1c8a390 msgr2=0x5648d2062650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:56:23.810 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:23.804+0000 7f9a29134740 1 --2- 172.21.15.39:0/3843046713 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5648d1c8a390 0x5648d2062650 secure :-1 s=READY pgs=1330 cs=0 l=1 rev1=1 crypto rx=0x7f9a080aca20 tx=0x7f9a080047d0 comp rx=0 tx=0).stop 2024-09-21T03:56:23.810 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:23.804+0000 7f9a29134740 1 -- 172.21.15.39:0/3843046713 shutdown_connections 2024-09-21T03:56:23.811 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:23.804+0000 7f9a29134740 1 --2- 172.21.15.39:0/3843046713 >> [v2:172.21.15.39:6840/4247190456,v1:172.21.15.39:6841/4247190456] conn(0x5648d1c699f0 0x5648d20743b0 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:23.811 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:23.804+0000 7f99fbfff640 0 client.18300 ms_handle_reset on v2:172.21.15.39:6840/4247190456 2024-09-21T03:56:23.811 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:23.804+0000 7f9a29134740 1 --2- 172.21.15.39:0/3843046713 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5648d1cab070 0x5648d2062bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:23.811 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:23.804+0000 7f9a29134740 1 --2- 172.21.15.39:0/3843046713 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5648d1c8a390 0x5648d2062650 unknown :-1 s=CLOSED pgs=1330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:23.811 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:23.804+0000 7f9a29134740 1 --2- 172.21.15.39:0/3843046713 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5648d1c821b0 0x5648d2062110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:23.811 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:23.804+0000 7f9a29134740 1 -- 172.21.15.39:0/3843046713 >> 172.21.15.39:0/3843046713 conn(0x5648d1c9f460 msgr2=0x5648d204ebb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:56:23.811 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:23.804+0000 7f9a29134740 1 -- 172.21.15.39:0/3843046713 shutdown_connections 2024-09-21T03:56:23.811 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:23.804+0000 7f9a29134740 1 -- 172.21.15.39:0/3843046713 wait complete. 2024-09-21T03:56:23.811 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:23.804+0000 7f9a29134740 20 client.18300 trim_cache size 0 max 16384 2024-09-21T03:56:23.821 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:23.818+0000 7f9a29134740 1 Processor -- start 2024-09-21T03:56:23.821 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:23.818+0000 7f9a29134740 1 -- start start 2024-09-21T03:56:23.821 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:23.818+0000 7f9a29134740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5648d1c699f0 0x5648d2057f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:56:23.821 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:23.818+0000 7f9a29134740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5648d1c821b0 0x5648d205f300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:56:23.822 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:23.818+0000 7f9a29134740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5648d1c8a390 0x5648d205f840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:56:23.822 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:23.818+0000 7f9a29134740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f9a1c1ad660 con 0x5648d1c8a390 2024-09-21T03:56:23.822 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:23.818+0000 7f9a29134740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f9a1c1af1a0 con 0x5648d1c821b0 2024-09-21T03:56:23.822 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:23.818+0000 7f9a29134740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f9a1c1add30 con 0x5648d1c699f0 2024-09-21T03:56:23.822 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:23.819+0000 7f9a2115d640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5648d1c8a390 0x5648d205f840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:56:23.822 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:23.819+0000 7f9a2115d640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5648d1c8a390 0x5648d205f840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:55850/0 (socket says 172.21.15.39:55850) 2024-09-21T03:56:23.822 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:23.819+0000 7f9a2115d640 1 -- 172.21.15.39:0/3305877106 learned_addr learned my addr 172.21.15.39:0/3305877106 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:56:23.822 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:23.819+0000 7f9a2095c640 1 --2- 172.21.15.39:0/3305877106 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5648d1c699f0 0x5648d2057f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:56:23.822 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:23.819+0000 7f9a2115d640 1 -- 172.21.15.39:0/3305877106 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5648d1c699f0 msgr2=0x5648d2057f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:56:23.822 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:23.819+0000 7f9a2115d640 1 --2- 172.21.15.39:0/3305877106 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5648d1c699f0 0x5648d2057f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:23.822 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:23.819+0000 7f9a2115d640 1 -- 172.21.15.39:0/3305877106 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5648d1c821b0 msgr2=0x5648d205f300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:56:23.823 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:23.819+0000 7f9a1bfff640 1 --2- 172.21.15.39:0/3305877106 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5648d1c821b0 0x5648d205f300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:56:23.823 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:23.819+0000 7f9a2115d640 1 --2- 172.21.15.39:0/3305877106 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5648d1c821b0 0x5648d205f300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:23.823 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:23.819+0000 7f9a2115d640 1 -- 172.21.15.39:0/3305877106 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5648d2054fb0 con 0x5648d1c8a390 2024-09-21T03:56:23.823 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:23.819+0000 7f9a2115d640 1 --2- 172.21.15.39:0/3305877106 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5648d1c8a390 0x5648d205f840 secure :-1 s=READY pgs=1331 cs=0 l=1 rev1=1 crypto rx=0x7f9a14005c40 tx=0x7f9a14016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd78349707abec4f server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:56:23.823 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:23.819+0000 7f9a067fc640 1 -- 172.21.15.39:0/3305877106 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a140219b0 con 0x5648d1c8a390 2024-09-21T03:56:23.823 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:23.820+0000 7f9a067fc640 1 -- 172.21.15.39:0/3305877106 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9a14003970 con 0x5648d1c8a390 2024-09-21T03:56:23.823 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:23.820+0000 7f9a067fc640 1 -- 172.21.15.39:0/3305877106 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a14004640 con 0x5648d1c8a390 2024-09-21T03:56:23.827 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:23.820+0000 7f9a29134740 1 -- 172.21.15.39:0/3305877106 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5648d1c8a390 msgr2=0x5648d205f840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:56:23.827 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:23.820+0000 7f9a29134740 1 --2- 172.21.15.39:0/3305877106 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5648d1c8a390 0x5648d205f840 secure :-1 s=READY pgs=1331 cs=0 l=1 rev1=1 crypto rx=0x7f9a14005c40 tx=0x7f9a14016000 comp rx=0 tx=0).stop 2024-09-21T03:56:23.827 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:23.820+0000 7f9a29134740 1 -- 172.21.15.39:0/3305877106 shutdown_connections 2024-09-21T03:56:23.827 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:23.820+0000 7f9a29134740 1 --2- 172.21.15.39:0/3305877106 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5648d1c8a390 0x5648d205f840 unknown :-1 s=CLOSED pgs=1331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:23.827 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:23.820+0000 7f9a29134740 1 --2- 172.21.15.39:0/3305877106 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5648d1c821b0 0x5648d205f300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:23.827 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:23.820+0000 7f9a29134740 1 --2- 172.21.15.39:0/3305877106 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5648d1c699f0 0x5648d2057f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:23.827 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:23.820+0000 7f9a29134740 1 -- 172.21.15.39:0/3305877106 >> 172.21.15.39:0/3305877106 conn(0x5648d1c9f460 msgr2=0x5648d2057840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:56:23.827 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:23.820+0000 7f9a29134740 1 -- 172.21.15.39:0/3305877106 shutdown_connections 2024-09-21T03:56:23.827 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:23.820+0000 7f9a29134740 1 -- 172.21.15.39:0/3305877106 wait complete. 2024-09-21T03:56:23.827 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:23.821+0000 7f9a29134740 1 -- start start 2024-09-21T03:56:23.828 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:23.821+0000 7f9a29134740 1 Processor -- start 2024-09-21T03:56:23.828 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:23.821+0000 7f9a29134740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5648d1c699f0 0x5648d2066580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:56:23.828 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:23.821+0000 7f9a29134740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5648d1c821b0 0x5648d2066ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:56:23.828 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:23.821+0000 7f9a29134740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5648d1c8a390 0x5648d2067000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:56:23.828 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:23.821+0000 7f9a29134740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f9a1c1aead0 con 0x5648d1c699f0 2024-09-21T03:56:23.828 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:23.821+0000 7f9a29134740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f9a1c1ae400 con 0x5648d1c8a390 2024-09-21T03:56:23.828 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:23.821+0000 7f9a29134740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f9a1c1ab640 con 0x5648d1c821b0 2024-09-21T03:56:23.828 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:23.821+0000 7f9a29134740 10 client.0 _ll_register_callbacks cb 0x7f9a08010720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:56:23.828 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:23.821+0000 7f9a2095c640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5648d1c699f0 0x5648d2066580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:56:23.828 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:23.822+0000 7f9a2095c640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5648d1c699f0 0x5648d2066580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:55864/0 (socket says 172.21.15.39:55864) 2024-09-21T03:56:23.828 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:23.822+0000 7f9a2095c640 1 -- 172.21.15.39:0/3327521718 learned_addr learned my addr 172.21.15.39:0/3327521718 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:56:23.828 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:23.822+0000 7f9a1bfff640 1 --2- 172.21.15.39:0/3327521718 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5648d1c821b0 0x5648d2066ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:56:23.828 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:23.822+0000 7f9a2115d640 1 --2- 172.21.15.39:0/3327521718 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5648d1c8a390 0x5648d2067000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:56:23.828 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:23.822+0000 7f9a2095c640 1 -- 172.21.15.39:0/3327521718 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5648d1c821b0 msgr2=0x5648d2066ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:56:23.829 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:23.822+0000 7f9a2095c640 1 --2- 172.21.15.39:0/3327521718 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5648d1c821b0 0x5648d2066ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:23.829 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:23.822+0000 7f9a2095c640 1 -- 172.21.15.39:0/3327521718 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5648d1c8a390 msgr2=0x5648d2067000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:56:23.829 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:23.822+0000 7f9a2095c640 1 --2- 172.21.15.39:0/3327521718 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5648d1c8a390 0x5648d2067000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:23.829 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:23.822+0000 7f9a2095c640 1 -- 172.21.15.39:0/3327521718 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5648d2053490 con 0x5648d1c699f0 2024-09-21T03:56:23.829 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:23.822+0000 7f9a2095c640 1 --2- 172.21.15.39:0/3327521718 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5648d1c699f0 0x5648d2066580 secure :-1 s=READY pgs=1332 cs=0 l=1 rev1=1 crypto rx=0x7f9a10009900 tx=0x7f9a10032f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aaa97c9e0fc59beb server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:56:23.829 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:23.822+0000 7f99fbfff640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-09-21T03:56:23.829 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:23.822+0000 7f99fbfff640 1 -- 172.21.15.39:0/3327521718 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a1003f040 con 0x5648d1c699f0 2024-09-21T03:56:23.829 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:23.822+0000 7f99fbfff640 1 -- 172.21.15.39:0/3327521718 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8528+0+0 (secure 0 0 0) 0x7f9a10004460 con 0x5648d1c699f0 2024-09-21T03:56:23.829 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:23.822+0000 7f9a29134740 1 -- 172.21.15.39:0/3327521718 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9a14003970 con 0x5648d1c699f0 2024-09-21T03:56:23.829 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:23.823+0000 7f99fbfff640 1 -- 172.21.15.39:0/3327521718 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f9a10035040 con 0x5648d1c699f0 2024-09-21T03:56:23.829 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:23.823+0000 7f9a29134740 10 client.18306 fetch_fsmap learned FSMap version 29 2024-09-21T03:56:23.829 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:23.823+0000 7f9a29134740 1 -- 172.21.15.39:0/3327521718 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5648d20def80 con 0x5648d1c699f0 2024-09-21T03:56:23.829 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:23.823+0000 7f99fbfff640 1 -- 172.21.15.39:0/3327521718 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9a14003970 con 0x5648d1c699f0 2024-09-21T03:56:23.829 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:23.823+0000 7f99fbfff640 1 -- 172.21.15.39:0/3327521718 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f9a10003920 con 0x5648d1c699f0 2024-09-21T03:56:23.830 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:23.823+0000 7f9a29134740 10 client.18306 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T03:56:23.830 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:23.823+0000 7f9a29134740 10 client.18306 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.a 2024-09-21T03:56:23.830 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:23.823+0000 7f9a29134740 20 client.18306 populate_metadata read hostname 'smithi039' 2024-09-21T03:56:23.830 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:23.823+0000 7f9a29134740 1 --2- 172.21.15.39:0/3327521718 >> [v2:172.21.15.39:6840/4247190456,v1:172.21.15.39:6841/4247190456] conn(0x5648d1cab070 0x5648d20e12c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:56:23.830 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:23.823+0000 7f9a29134740 4 client.18306 mds_command: new command op to 14868 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T03:56:23.830 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:23.823+0000 7f9a29134740 1 -- 172.21.15.39:0/3327521718 --> [v2:172.21.15.39:6840/4247190456,v1:172.21.15.39:6841/4247190456] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5648d1fc68a0 con 0x5648d1cab070 2024-09-21T03:56:23.830 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:23.823+0000 7f9a1bfff640 1 --2- 172.21.15.39:0/3327521718 >> [v2:172.21.15.39:6840/4247190456,v1:172.21.15.39:6841/4247190456] conn(0x5648d1cab070 0x5648d20e12c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:56:23.830 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:23.824+0000 7f99fbfff640 10 client.18306 ms_handle_connect on v2:172.21.15.39:6840/4247190456 2024-09-21T03:56:23.830 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:23.824+0000 7f9a1bfff640 1 --2- 172.21.15.39:0/3327521718 >> [v2:172.21.15.39:6840/4247190456,v1:172.21.15.39:6841/4247190456] conn(0x5648d1cab070 0x5648d20e12c0 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=7416fc7451d65680 server_cookie=19c86042fd2e7877 in_seq=0 out_seq=0 2024-09-21T03:56:23.830 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:23.824+0000 7f99fbfff640 1 -- 172.21.15.39:0/3327521718 <== mds.4 v2:172.21.15.39:6840/4247190456 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5648d1fc68a0 con 0x5648d1cab070 2024-09-21T03:56:23.830 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:23.824+0000 7f99fbfff640 10 client.18306 handle_command_reply: tid=0 2024-09-21T03:56:23.830 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:23.824+0000 7f9a29134740 1 client.18306 shutdown 2024-09-21T03:56:23.830 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:23.824+0000 7f9a29134740 1 -- 172.21.15.39:0/3327521718 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5648d1c699f0 msgr2=0x5648d2066580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:56:23.830 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:23.824+0000 7f9a29134740 1 --2- 172.21.15.39:0/3327521718 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5648d1c699f0 0x5648d2066580 secure :-1 s=READY pgs=1332 cs=0 l=1 rev1=1 crypto rx=0x7f9a10009900 tx=0x7f9a10032f90 comp rx=0 tx=0).stop 2024-09-21T03:56:23.831 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:23.824+0000 7f9a29134740 1 -- 172.21.15.39:0/3327521718 shutdown_connections 2024-09-21T03:56:23.831 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:23.824+0000 7f9a29134740 1 --2- 172.21.15.39:0/3327521718 >> [v2:172.21.15.39:6840/4247190456,v1:172.21.15.39:6841/4247190456] conn(0x5648d1cab070 0x5648d20e12c0 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:23.831 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:23.824+0000 7f9a29134740 1 --2- 172.21.15.39:0/3327521718 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5648d1c8a390 0x5648d2067000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:23.831 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:23.824+0000 7f9a29134740 1 --2- 172.21.15.39:0/3327521718 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5648d1c821b0 0x5648d2066ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:23.831 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:23.824+0000 7f9a29134740 1 --2- 172.21.15.39:0/3327521718 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5648d1c699f0 0x5648d2066580 unknown :-1 s=CLOSED pgs=1332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:23.831 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:23.824+0000 7f99fbfff640 0 client.18306 ms_handle_reset on v2:172.21.15.39:6840/4247190456 2024-09-21T03:56:23.831 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:23.824+0000 7f9a29134740 1 -- 172.21.15.39:0/3327521718 >> 172.21.15.39:0/3327521718 conn(0x5648d1c9f460 msgr2=0x5648d2053660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:56:23.831 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:23.824+0000 7f9a29134740 1 -- 172.21.15.39:0/3327521718 shutdown_connections 2024-09-21T03:56:23.831 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:23.824+0000 7f9a29134740 1 -- 172.21.15.39:0/3327521718 wait complete. 2024-09-21T03:56:23.831 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:23.824+0000 7f9a29134740 20 client.18306 trim_cache size 0 max 16384 2024-09-21T03:56:23.831 INFO:teuthology.orchestra.run.smithi039.stdout:[] 2024-09-21T03:56:23.832 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:23.825+0000 7f9a22bc0640 1 -- 172.21.15.39:0/3243482448 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f9a0c0b6680 msgr2=0x7f9a0c0b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:56:23.832 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:23.825+0000 7f9a22bc0640 1 --2- 172.21.15.39:0/3243482448 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f9a0c0b6680 0x7f9a0c0b8b90 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f9a14005fd0 tx=0x7f9a14009600 comp rx=0 tx=0).stop 2024-09-21T03:56:23.832 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:23.825+0000 7f9a22bc0640 1 -- 172.21.15.39:0/3243482448 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9a1c1083d0 msgr2=0x7f9a1c19aae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:56:23.832 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:23.825+0000 7f9a22bc0640 1 --2- 172.21.15.39:0/3243482448 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9a1c1083d0 0x7f9a1c19aae0 secure :-1 s=READY pgs=1328 cs=0 l=1 rev1=1 crypto rx=0x7f9a08006fd0 tx=0x7f9a0800b8e0 comp rx=0 tx=0).stop 2024-09-21T03:56:23.832 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:23.825+0000 7f9a22bc0640 1 -- 172.21.15.39:0/3243482448 shutdown_connections 2024-09-21T03:56:23.833 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:23.825+0000 7f9a22bc0640 1 --2- 172.21.15.39:0/3243482448 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f9a0c0b6680 0x7f9a0c0b8b90 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:23.833 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:23.825+0000 7f9a22bc0640 1 --2- 172.21.15.39:0/3243482448 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9a1c108d20 0x7f9a1c19fd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:23.833 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:23.825+0000 7f9a22bc0640 1 --2- 172.21.15.39:0/3243482448 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9a1c1083d0 0x7f9a1c19aae0 unknown :-1 s=CLOSED pgs=1328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:23.833 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:23.825+0000 7f9a22bc0640 1 --2- 172.21.15.39:0/3243482448 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9a1c102ae0 0x7f9a1c19a540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:23.833 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:23.825+0000 7f9a22bc0640 1 -- 172.21.15.39:0/3243482448 >> 172.21.15.39:0/3243482448 conn(0x7f9a1c0797b0 msgr2=0x7f9a1c101b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:56:23.833 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:23.825+0000 7f9a22bc0640 1 -- 172.21.15.39:0/3243482448 shutdown_connections 2024-09-21T03:56:23.833 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:23.825+0000 7f9a22bc0640 1 -- 172.21.15.39:0/3243482448 wait complete. 2024-09-21T03:56:24.634 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 51116d5c-77ca-11ef-baf5-efdc52797490 -- ceph tell mds.1:3 damage ls 2024-09-21T03:56:24.891 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/51116d5c-77ca-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:56:25.283 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:56:25 smithi039 ceph-mon[31211]: pgmap v644: 97 pgs: 97 active+clean; 10 MiB data, 525 MiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 102 B/s wr, 13 op/s 2024-09-21T03:56:25.377 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:25.375+0000 7f4146b3e640 1 -- 172.21.15.39:0/3538698108 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4140108d20 msgr2=0x7f414010f550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:56:25.377 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:25.375+0000 7f4146b3e640 1 --2- 172.21.15.39:0/3538698108 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4140108d20 0x7f414010f550 secure :-1 s=READY pgs=1333 cs=0 l=1 rev1=1 crypto rx=0x7f4138009ca0 tx=0x7f413802a8c0 comp rx=0 tx=0).stop 2024-09-21T03:56:25.377 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:25.375+0000 7f4146b3e640 1 -- 172.21.15.39:0/3538698108 shutdown_connections 2024-09-21T03:56:25.377 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:25.375+0000 7f4146b3e640 1 --2- 172.21.15.39:0/3538698108 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4140108d20 0x7f414010f550 unknown :-1 s=CLOSED pgs=1333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:25.377 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:25.375+0000 7f4146b3e640 1 --2- 172.21.15.39:0/3538698108 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f41401083d0 0x7f41401087e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:25.377 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:25.375+0000 7f4146b3e640 1 --2- 172.21.15.39:0/3538698108 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4140102ae0 0x7f4140102ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:25.377 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:25.375+0000 7f4146b3e640 1 -- 172.21.15.39:0/3538698108 >> 172.21.15.39:0/3538698108 conn(0x7f4140079760 msgr2=0x7f4140079b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:56:25.377 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:25.375+0000 7f4146b3e640 1 -- 172.21.15.39:0/3538698108 shutdown_connections 2024-09-21T03:56:25.377 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:25.375+0000 7f4146b3e640 1 -- 172.21.15.39:0/3538698108 wait complete. 2024-09-21T03:56:25.379 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:25.375+0000 7f4146b3e640 1 Processor -- start 2024-09-21T03:56:25.379 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:25.375+0000 7f4146b3e640 1 -- start start 2024-09-21T03:56:25.379 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:25.376+0000 7f4146b3e640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4140102ae0 0x7f414010ffe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:56:25.379 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:25.376+0000 7f4146b3e640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f41401083d0 0x7f4140110520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:56:25.379 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:25.376+0000 7f4146b3e640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4140108d20 0x7f4140110a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:56:25.379 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:25.376+0000 7f4146b3e640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f4140113ae0 con 0x7f4140108d20 2024-09-21T03:56:25.379 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:25.376+0000 7f4146b3e640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f4140113960 con 0x7f41401083d0 2024-09-21T03:56:25.380 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:25.376+0000 7f4146b3e640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f4140113c60 con 0x7f4140102ae0 2024-09-21T03:56:25.380 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:25.376+0000 7f41450db640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4140108d20 0x7f4140110a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:56:25.380 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:25.376+0000 7f41448da640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4140102ae0 0x7f414010ffe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:56:25.380 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:25.376+0000 7f41450db640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4140108d20 0x7f4140110a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:55890/0 (socket says 172.21.15.39:55890) 2024-09-21T03:56:25.380 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:25.376+0000 7f41450db640 1 -- 172.21.15.39:0/3773251689 learned_addr learned my addr 172.21.15.39:0/3773251689 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:56:25.380 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:25.376+0000 7f413ffff640 1 --2- 172.21.15.39:0/3773251689 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f41401083d0 0x7f4140110520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:56:25.380 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:25.376+0000 7f41450db640 1 -- 172.21.15.39:0/3773251689 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4140102ae0 msgr2=0x7f414010ffe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:56:25.380 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:25.376+0000 7f41450db640 1 --2- 172.21.15.39:0/3773251689 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4140102ae0 0x7f414010ffe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:25.380 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:25.376+0000 7f41450db640 1 -- 172.21.15.39:0/3773251689 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f41401083d0 msgr2=0x7f4140110520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:56:25.380 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:25.376+0000 7f41450db640 1 --2- 172.21.15.39:0/3773251689 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f41401083d0 0x7f4140110520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:25.380 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:25.376+0000 7f41450db640 1 -- 172.21.15.39:0/3773251689 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f41401a86f0 con 0x7f4140108d20 2024-09-21T03:56:25.380 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:25.377+0000 7f41450db640 1 --2- 172.21.15.39:0/3773251689 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4140108d20 0x7f4140110a60 secure :-1 s=READY pgs=1334 cs=0 l=1 rev1=1 crypto rx=0x7f4138009430 tx=0x7f41380319f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:56:25.380 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:25.377+0000 7f413dffb640 1 -- 172.21.15.39:0/3773251689 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4138003c30 con 0x7f4140108d20 2024-09-21T03:56:25.380 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:25.377+0000 7f413dffb640 1 -- 172.21.15.39:0/3773251689 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4138003dd0 con 0x7f4140108d20 2024-09-21T03:56:25.381 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:25.377+0000 7f4146b3e640 1 -- 172.21.15.39:0/3773251689 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f41401a8980 con 0x7f4140108d20 2024-09-21T03:56:25.381 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:25.377+0000 7f413dffb640 1 -- 172.21.15.39:0/3773251689 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f413802faa0 con 0x7f4140108d20 2024-09-21T03:56:25.381 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:25.377+0000 7f4146b3e640 1 -- 172.21.15.39:0/3773251689 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f41401a8f10 con 0x7f4140108d20 2024-09-21T03:56:25.381 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:25.380+0000 7f414d0b2740 1 Processor -- start 2024-09-21T03:56:25.381 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:25.380+0000 7f413dffb640 1 -- 172.21.15.39:0/3773251689 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 27) ==== 154643+0+0 (secure 0 0 0) 0x7f413802fc40 con 0x7f4140108d20 2024-09-21T03:56:25.381 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:25.380+0000 7f414d0b2740 1 -- start start 2024-09-21T03:56:25.383 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:25.380+0000 7f414d0b2740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55bc11cef070 0x55bc120966a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:56:25.383 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:25.380+0000 7f414d0b2740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55bc11cc61b0 0x55bc12096be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:56:25.384 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:25.380+0000 7f414d0b2740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55bc11cce390 0x55bc12097120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:56:25.384 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:25.380+0000 7f414d0b2740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x55bc1200cea0 con 0x55bc11cef070 2024-09-21T03:56:25.384 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:25.380+0000 7f414d0b2740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x55bc11ffc2d0 con 0x55bc11cce390 2024-09-21T03:56:25.384 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:25.380+0000 7f414d0b2740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55bc120000e0 con 0x55bc11cc61b0 2024-09-21T03:56:25.384 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:25.380+0000 7f413ffff640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55bc11cef070 0x55bc120966a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:56:25.384 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:25.380+0000 7f413ffff640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55bc11cef070 0x55bc120966a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:55900/0 (socket says 172.21.15.39:55900) 2024-09-21T03:56:25.384 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:25.380+0000 7f413ffff640 1 -- 172.21.15.39:0/1048138251 learned_addr learned my addr 172.21.15.39:0/1048138251 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:56:25.384 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:25.380+0000 7f41448da640 1 --2- 172.21.15.39:0/1048138251 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55bc11cc61b0 0x55bc12096be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:56:25.384 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:25.381+0000 7f413ffff640 1 --2- 172.21.15.39:0/1048138251 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55bc11cce390 0x55bc12097120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:56:25.384 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:25.381+0000 7f413ffff640 1 -- 172.21.15.39:0/1048138251 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55bc11cc61b0 msgr2=0x55bc12096be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:56:25.384 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:25.381+0000 7f413ffff640 1 --2- 172.21.15.39:0/1048138251 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55bc11cc61b0 0x55bc12096be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:25.384 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:25.381+0000 7f413ffff640 1 -- 172.21.15.39:0/1048138251 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55bc11cce390 msgr2=0x55bc12097120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:56:25.384 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:25.381+0000 7f413ffff640 1 --2- 172.21.15.39:0/1048138251 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55bc11cce390 0x55bc12097120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:25.384 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:25.381+0000 7f413ffff640 1 -- 172.21.15.39:0/1048138251 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bc11d5a650 con 0x55bc11cef070 2024-09-21T03:56:25.385 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:25.381+0000 7f41448da640 1 --2- 172.21.15.39:0/1048138251 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55bc11cc61b0 0x55bc12096be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:56:25.385 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:25.381+0000 7f413dffb640 1 --2- 172.21.15.39:0/3773251689 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f41300b6700 0x7f41300b8c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:56:25.385 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:25.381+0000 7f413ffff640 1 --2- 172.21.15.39:0/1048138251 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55bc11cef070 0x55bc120966a0 secure :-1 s=READY pgs=1335 cs=0 l=1 rev1=1 crypto rx=0x7f412c00bbf0 tx=0x7f412c004610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cdc0c50a1e90766e server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:56:25.385 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:25.381+0000 7f41448da640 1 --2- 172.21.15.39:0/3773251689 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f41300b6700 0x7f41300b8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:56:25.385 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:25.381+0000 7f413dffb640 1 -- 172.21.15.39:0/3773251689 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8528+0+0 (secure 0 0 0) 0x7f4138108960 con 0x7f4140108d20 2024-09-21T03:56:25.385 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:25.381+0000 7f412affd640 1 -- 172.21.15.39:0/1048138251 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f412c005020 con 0x55bc11cef070 2024-09-21T03:56:25.385 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:25.381+0000 7f412affd640 1 -- 172.21.15.39:0/1048138251 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f412c005670 con 0x55bc11cef070 2024-09-21T03:56:25.385 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:25.381+0000 7f412affd640 1 -- 172.21.15.39:0/1048138251 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f412c010460 con 0x55bc11cef070 2024-09-21T03:56:25.389 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:25.382+0000 7f414d0b2740 1 -- 172.21.15.39:0/1048138251 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55bc11cef070 msgr2=0x55bc120966a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:56:25.389 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:25.382+0000 7f41448da640 1 --2- 172.21.15.39:0/3773251689 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f41300b6700 0x7f41300b8c10 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x55bc12095320 tx=0x7f413400b390 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:56:25.389 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:25.382+0000 7f414d0b2740 1 --2- 172.21.15.39:0/1048138251 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55bc11cef070 0x55bc120966a0 secure :-1 s=READY pgs=1335 cs=0 l=1 rev1=1 crypto rx=0x7f412c00bbf0 tx=0x7f412c004610 comp rx=0 tx=0).stop 2024-09-21T03:56:25.389 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:25.382+0000 7f414d0b2740 1 -- 172.21.15.39:0/1048138251 shutdown_connections 2024-09-21T03:56:25.390 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:25.382+0000 7f414d0b2740 1 --2- 172.21.15.39:0/1048138251 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55bc11cce390 0x55bc12097120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:25.390 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:25.382+0000 7f414d0b2740 1 --2- 172.21.15.39:0/1048138251 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55bc11cc61b0 0x55bc12096be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:25.390 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:25.382+0000 7f414d0b2740 1 --2- 172.21.15.39:0/1048138251 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55bc11cef070 0x55bc120966a0 unknown :-1 s=CLOSED pgs=1335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:25.390 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:25.382+0000 7f414d0b2740 1 -- 172.21.15.39:0/1048138251 >> 172.21.15.39:0/1048138251 conn(0x55bc11ce3460 msgr2=0x55bc12094590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:56:25.390 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:25.382+0000 7f414d0b2740 1 -- 172.21.15.39:0/1048138251 shutdown_connections 2024-09-21T03:56:25.390 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:25.382+0000 7f414d0b2740 1 -- 172.21.15.39:0/1048138251 wait complete. 2024-09-21T03:56:25.390 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:25.383+0000 7f414d0b2740 1 -- start start 2024-09-21T03:56:25.390 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:25.383+0000 7f414d0b2740 1 Processor -- start 2024-09-21T03:56:25.390 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:25.383+0000 7f414d0b2740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55bc11cc61b0 0x55bc120a1d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:56:25.390 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:25.383+0000 7f414d0b2740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55bc11cce390 0x55bc120a2240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:56:25.391 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:25.383+0000 7f414d0b2740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55bc11cef070 0x55bc120a2780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:56:25.391 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:25.383+0000 7f414d0b2740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x55bc11e5d700 con 0x55bc11cc61b0 2024-09-21T03:56:25.391 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:25.383+0000 7f414d0b2740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x55bc11f97160 con 0x55bc11cce390 2024-09-21T03:56:25.391 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:25.383+0000 7f414d0b2740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55bc120a4e20 con 0x55bc11cef070 2024-09-21T03:56:25.391 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:25.383+0000 7f414d0b2740 10 client.0 _ll_register_callbacks cb 0x55bc11fb09e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:56:25.391 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:25.383+0000 7f41448da640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55bc11cc61b0 0x55bc120a1d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:56:25.391 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:25.384+0000 7f41448da640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55bc11cc61b0 0x55bc120a1d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:55914/0 (socket says 172.21.15.39:55914) 2024-09-21T03:56:25.391 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:25.384+0000 7f41448da640 1 -- 172.21.15.39:0/586948293 learned_addr learned my addr 172.21.15.39:0/586948293 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:56:25.392 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:25.384+0000 7f41450db640 1 --2- 172.21.15.39:0/586948293 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55bc11cef070 0x55bc120a2780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:56:25.392 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:25.384+0000 7f413ffff640 1 --2- 172.21.15.39:0/586948293 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55bc11cce390 0x55bc120a2240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:56:25.392 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:25.384+0000 7f41448da640 1 -- 172.21.15.39:0/586948293 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55bc11cef070 msgr2=0x55bc120a2780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:56:25.392 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:25.384+0000 7f41448da640 1 --2- 172.21.15.39:0/586948293 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55bc11cef070 0x55bc120a2780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:25.392 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:25.384+0000 7f41448da640 1 -- 172.21.15.39:0/586948293 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55bc11cce390 msgr2=0x55bc120a2240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:56:25.392 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:25.384+0000 7f41448da640 1 --2- 172.21.15.39:0/586948293 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55bc11cce390 0x55bc120a2240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:25.392 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:25.384+0000 7f41448da640 1 -- 172.21.15.39:0/586948293 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bc11fc4c90 con 0x55bc11cc61b0 2024-09-21T03:56:25.392 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:25.384+0000 7f41450db640 1 --2- 172.21.15.39:0/586948293 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55bc11cef070 0x55bc120a2780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:56:25.393 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:25.384+0000 7f413ffff640 1 --2- 172.21.15.39:0/586948293 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55bc11cce390 0x55bc120a2240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:56:25.393 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:25.384+0000 7f41448da640 1 --2- 172.21.15.39:0/586948293 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55bc11cc61b0 0x55bc120a1d00 secure :-1 s=READY pgs=1336 cs=0 l=1 rev1=1 crypto rx=0x7f4134003d30 tx=0x7f4134004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9287150b7ad359ab server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:56:25.393 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:25.384+0000 7f411ffff640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-09-21T03:56:25.393 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:25.384+0000 7f411ffff640 1 -- 172.21.15.39:0/586948293 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f413402c060 con 0x55bc11cc61b0 2024-09-21T03:56:25.393 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:25.384+0000 7f411ffff640 1 -- 172.21.15.39:0/586948293 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8528+0+0 (secure 0 0 0) 0x7f4134017030 con 0x55bc11cc61b0 2024-09-21T03:56:25.393 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:25.384+0000 7f414d0b2740 1 -- 172.21.15.39:0/586948293 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f412c005670 con 0x55bc11cc61b0 2024-09-21T03:56:25.393 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:25.385+0000 7f411ffff640 1 -- 172.21.15.39:0/586948293 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f4134030410 con 0x55bc11cc61b0 2024-09-21T03:56:25.393 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:25.385+0000 7f414d0b2740 10 client.18318 fetch_fsmap learned FSMap version 29 2024-09-21T03:56:25.393 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:25.385+0000 7f414d0b2740 1 -- 172.21.15.39:0/586948293 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55bc120b0b00 con 0x55bc11cc61b0 2024-09-21T03:56:25.394 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:25.385+0000 7f411ffff640 1 -- 172.21.15.39:0/586948293 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f412c005670 con 0x55bc11cc61b0 2024-09-21T03:56:25.394 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:25.385+0000 7f411ffff640 1 -- 172.21.15.39:0/586948293 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f41340114d0 con 0x55bc11cc61b0 2024-09-21T03:56:25.394 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:25.385+0000 7f414d0b2740 10 client.18318 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T03:56:25.394 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:25.385+0000 7f414d0b2740 10 client.18318 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-21T03:56:25.394 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:25.385+0000 7f414d0b2740 20 client.18318 populate_metadata read hostname 'smithi039' 2024-09-21T03:56:25.394 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:25.385+0000 7f414d0b2740 1 --2- 172.21.15.39:0/586948293 >> [v2:172.21.15.161:6836/3451544597,v1:172.21.15.161:6837/3451544597] conn(0x55bc11cad9f0 0x55bc120b4030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:56:25.394 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:25.385+0000 7f414d0b2740 4 client.18318 mds_command: new command op to 24646 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:56:25.394 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:25.385+0000 7f414d0b2740 1 -- 172.21.15.39:0/586948293 --> [v2:172.21.15.161:6836/3451544597,v1:172.21.15.161:6837/3451544597] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bc1200edf0 con 0x55bc11cad9f0 2024-09-21T03:56:25.395 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:25.385+0000 7f413ffff640 1 --2- 172.21.15.39:0/586948293 >> [v2:172.21.15.161:6836/3451544597,v1:172.21.15.161:6837/3451544597] conn(0x55bc11cad9f0 0x55bc120b4030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:56:25.395 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:25.386+0000 7f413ffff640 1 --2- 172.21.15.39:0/586948293 >> [v2:172.21.15.161:6836/3451544597,v1:172.21.15.161:6837/3451544597] conn(0x55bc11cad9f0 0x55bc120b4030 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=85fdf0f0adb50c01 server_cookie=c5d36e9be0bae6db in_seq=0 out_seq=0 2024-09-21T03:56:25.395 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:25.386+0000 7f411ffff640 10 client.18318 ms_handle_connect on v2:172.21.15.161:6836/3451544597 2024-09-21T03:56:25.395 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:25.387+0000 7f411ffff640 1 -- 172.21.15.39:0/586948293 <== mds.3 v2:172.21.15.161:6836/3451544597 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55bc1200edf0 con 0x55bc11cad9f0 2024-09-21T03:56:25.395 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:25.387+0000 7f411ffff640 10 client.18318 handle_command_reply: tid=0 2024-09-21T03:56:25.395 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:25.387+0000 7f414d0b2740 1 client.18318 shutdown 2024-09-21T03:56:25.395 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:25.387+0000 7f414d0b2740 1 -- 172.21.15.39:0/586948293 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55bc11cc61b0 msgr2=0x55bc120a1d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:56:25.395 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:25.387+0000 7f414d0b2740 1 --2- 172.21.15.39:0/586948293 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55bc11cc61b0 0x55bc120a1d00 secure :-1 s=READY pgs=1336 cs=0 l=1 rev1=1 crypto rx=0x7f4134003d30 tx=0x7f4134004730 comp rx=0 tx=0).stop 2024-09-21T03:56:25.395 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:25.387+0000 7f414d0b2740 1 -- 172.21.15.39:0/586948293 shutdown_connections 2024-09-21T03:56:25.395 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:25.387+0000 7f414d0b2740 1 --2- 172.21.15.39:0/586948293 >> [v2:172.21.15.161:6836/3451544597,v1:172.21.15.161:6837/3451544597] conn(0x55bc11cad9f0 0x55bc120b4030 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:25.395 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:25.387+0000 7f414d0b2740 1 --2- 172.21.15.39:0/586948293 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55bc11cef070 0x55bc120a2780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:25.395 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:25.387+0000 7f414d0b2740 1 --2- 172.21.15.39:0/586948293 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55bc11cce390 0x55bc120a2240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:25.395 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:25.387+0000 7f411ffff640 0 client.18318 ms_handle_reset on v2:172.21.15.161:6836/3451544597 2024-09-21T03:56:25.395 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:25.387+0000 7f414d0b2740 1 --2- 172.21.15.39:0/586948293 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55bc11cc61b0 0x55bc120a1d00 unknown :-1 s=CLOSED pgs=1336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:25.396 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:25.387+0000 7f414d0b2740 1 -- 172.21.15.39:0/586948293 >> 172.21.15.39:0/586948293 conn(0x55bc11ce3460 msgr2=0x55bc12099880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:56:25.396 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:25.387+0000 7f414d0b2740 1 -- 172.21.15.39:0/586948293 shutdown_connections 2024-09-21T03:56:25.396 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:25.387+0000 7f414d0b2740 1 -- 172.21.15.39:0/586948293 wait complete. 2024-09-21T03:56:25.396 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:25.387+0000 7f414d0b2740 20 client.18318 trim_cache size 0 max 16384 2024-09-21T03:56:25.403 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:25.400+0000 7f414d0b2740 1 Processor -- start 2024-09-21T03:56:25.403 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:25.401+0000 7f414d0b2740 1 -- start start 2024-09-21T03:56:25.403 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:25.401+0000 7f414d0b2740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55bc11cc61b0 0x55bc12097de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:56:25.403 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:25.401+0000 7f414d0b2740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55bc11cce390 0x55bc1209eef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:56:25.404 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:25.401+0000 7f414d0b2740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55bc11cef070 0x55bc1209f430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:56:25.404 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:25.401+0000 7f414d0b2740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f41401b1910 con 0x55bc11cce390 2024-09-21T03:56:25.404 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:25.401+0000 7f414d0b2740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f41401b3450 con 0x55bc11cc61b0 2024-09-21T03:56:25.404 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:25.401+0000 7f414d0b2740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f41401b1fe0 con 0x55bc11cef070 2024-09-21T03:56:25.404 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:25.401+0000 7f413ffff640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55bc11cce390 0x55bc1209eef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:56:25.404 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:25.401+0000 7f413ffff640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55bc11cce390 0x55bc1209eef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:55920/0 (socket says 172.21.15.39:55920) 2024-09-21T03:56:25.404 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:25.401+0000 7f413ffff640 1 -- 172.21.15.39:0/188642534 learned_addr learned my addr 172.21.15.39:0/188642534 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:56:25.404 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:25.401+0000 7f41450db640 1 --2- 172.21.15.39:0/188642534 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55bc11cef070 0x55bc1209f430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:56:25.404 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:25.401+0000 7f413ffff640 1 -- 172.21.15.39:0/188642534 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55bc11cef070 msgr2=0x55bc1209f430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:56:25.404 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:25.401+0000 7f413ffff640 1 --2- 172.21.15.39:0/188642534 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55bc11cef070 0x55bc1209f430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:25.404 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:25.401+0000 7f41448da640 1 --2- 172.21.15.39:0/188642534 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55bc11cc61b0 0x55bc12097de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:56:25.404 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:25.401+0000 7f413ffff640 1 -- 172.21.15.39:0/188642534 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55bc11cc61b0 msgr2=0x55bc12097de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:56:25.404 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:25.401+0000 7f413ffff640 1 --2- 172.21.15.39:0/188642534 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55bc11cc61b0 0x55bc12097de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:25.404 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:25.401+0000 7f413ffff640 1 -- 172.21.15.39:0/188642534 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bc12094680 con 0x55bc11cce390 2024-09-21T03:56:25.405 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:25.401+0000 7f41450db640 1 --2- 172.21.15.39:0/188642534 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55bc11cef070 0x55bc1209f430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:56:25.405 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:25.401+0000 7f413ffff640 1 --2- 172.21.15.39:0/188642534 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55bc11cce390 0x55bc1209eef0 secure :-1 s=READY pgs=1337 cs=0 l=1 rev1=1 crypto rx=0x7f412c004f40 tx=0x7f412c004ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=20a2c041b76dbc35 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:56:25.405 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:25.402+0000 7f412a7fc640 1 -- 172.21.15.39:0/188642534 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f412c004610 con 0x55bc11cce390 2024-09-21T03:56:25.405 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:25.402+0000 7f412a7fc640 1 -- 172.21.15.39:0/188642534 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f412c005000 con 0x55bc11cce390 2024-09-21T03:56:25.405 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:25.402+0000 7f412a7fc640 1 -- 172.21.15.39:0/188642534 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f412c0106c0 con 0x55bc11cce390 2024-09-21T03:56:25.407 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:25.402+0000 7f414d0b2740 1 -- 172.21.15.39:0/188642534 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55bc11cce390 msgr2=0x55bc1209eef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:56:25.407 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:25.402+0000 7f414d0b2740 1 --2- 172.21.15.39:0/188642534 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55bc11cce390 0x55bc1209eef0 secure :-1 s=READY pgs=1337 cs=0 l=1 rev1=1 crypto rx=0x7f412c004f40 tx=0x7f412c004ca0 comp rx=0 tx=0).stop 2024-09-21T03:56:25.407 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:25.402+0000 7f414d0b2740 1 -- 172.21.15.39:0/188642534 shutdown_connections 2024-09-21T03:56:25.407 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:25.402+0000 7f414d0b2740 1 --2- 172.21.15.39:0/188642534 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55bc11cef070 0x55bc1209f430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:25.407 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:25.402+0000 7f414d0b2740 1 --2- 172.21.15.39:0/188642534 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55bc11cce390 0x55bc1209eef0 unknown :-1 s=CLOSED pgs=1337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:25.408 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:25.403+0000 7f414d0b2740 1 --2- 172.21.15.39:0/188642534 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55bc11cc61b0 0x55bc12097de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:25.408 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:25.403+0000 7f414d0b2740 1 -- 172.21.15.39:0/188642534 >> 172.21.15.39:0/188642534 conn(0x55bc11ce3460 msgr2=0x55bc120976d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:56:25.408 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:25.403+0000 7f414d0b2740 1 -- 172.21.15.39:0/188642534 shutdown_connections 2024-09-21T03:56:25.408 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:25.403+0000 7f414d0b2740 1 -- 172.21.15.39:0/188642534 wait complete. 2024-09-21T03:56:25.408 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:25.403+0000 7f414d0b2740 1 -- start start 2024-09-21T03:56:25.408 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:25.403+0000 7f414d0b2740 1 Processor -- start 2024-09-21T03:56:25.408 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:25.404+0000 7f414d0b2740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55bc11cc61b0 0x55bc120a60a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:56:25.408 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:25.404+0000 7f414d0b2740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55bc11cce390 0x55bc120a65e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:56:25.408 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:25.404+0000 7f414d0b2740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55bc11cef070 0x55bc120a6b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:56:25.408 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:25.404+0000 7f414d0b2740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f41401b2d80 con 0x55bc11cce390 2024-09-21T03:56:25.408 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:25.404+0000 7f414d0b2740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f41401b26b0 con 0x55bc11cc61b0 2024-09-21T03:56:25.408 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:25.404+0000 7f414d0b2740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f41401af8f0 con 0x55bc11cef070 2024-09-21T03:56:25.408 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:25.404+0000 7f414d0b2740 10 client.0 _ll_register_callbacks cb 0x7f413400b820 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:56:25.408 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:25.404+0000 7f413ffff640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55bc11cce390 0x55bc120a65e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:56:25.409 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:25.404+0000 7f413ffff640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55bc11cce390 0x55bc120a65e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:55924/0 (socket says 172.21.15.39:55924) 2024-09-21T03:56:25.409 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:25.404+0000 7f413ffff640 1 -- 172.21.15.39:0/1894745467 learned_addr learned my addr 172.21.15.39:0/1894745467 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:56:25.409 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:25.404+0000 7f41450db640 1 --2- 172.21.15.39:0/1894745467 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55bc11cef070 0x55bc120a6b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:56:25.409 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:25.404+0000 7f41448da640 1 --2- 172.21.15.39:0/1894745467 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55bc11cc61b0 0x55bc120a60a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:56:25.409 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:25.404+0000 7f413ffff640 1 -- 172.21.15.39:0/1894745467 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55bc11cef070 msgr2=0x55bc120a6b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:56:25.409 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:25.404+0000 7f413ffff640 1 --2- 172.21.15.39:0/1894745467 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55bc11cef070 0x55bc120a6b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:25.409 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:25.404+0000 7f413ffff640 1 -- 172.21.15.39:0/1894745467 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55bc11cc61b0 msgr2=0x55bc120a60a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:56:25.409 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:25.404+0000 7f413ffff640 1 --2- 172.21.15.39:0/1894745467 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55bc11cc61b0 0x55bc120a60a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:25.409 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:25.404+0000 7f413ffff640 1 -- 172.21.15.39:0/1894745467 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bc1211da20 con 0x55bc11cce390 2024-09-21T03:56:25.409 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:25.404+0000 7f41450db640 1 --2- 172.21.15.39:0/1894745467 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55bc11cef070 0x55bc120a6b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:56:25.409 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:25.404+0000 7f413ffff640 1 --2- 172.21.15.39:0/1894745467 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55bc11cce390 0x55bc120a65e0 secure :-1 s=READY pgs=1338 cs=0 l=1 rev1=1 crypto rx=0x7f412c004dd0 tx=0x7f412c0041b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=82e1575ba4d964be server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:56:25.409 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:25.404+0000 7f411ffff640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-09-21T03:56:25.409 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:25.405+0000 7f411ffff640 1 -- 172.21.15.39:0/1894745467 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f412c036070 con 0x55bc11cce390 2024-09-21T03:56:25.409 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:25.405+0000 7f411ffff640 1 -- 172.21.15.39:0/1894745467 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8528+0+0 (secure 0 0 0) 0x7f412c0055c0 con 0x55bc11cce390 2024-09-21T03:56:25.409 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:25.405+0000 7f414d0b2740 1 -- 172.21.15.39:0/1894745467 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f412c005000 con 0x55bc11cce390 2024-09-21T03:56:25.410 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:25.405+0000 7f411ffff640 1 -- 172.21.15.39:0/1894745467 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f412c0045d0 con 0x55bc11cce390 2024-09-21T03:56:25.410 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:25.405+0000 7f414d0b2740 10 client.18324 fetch_fsmap learned FSMap version 29 2024-09-21T03:56:25.410 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:25.405+0000 7f414d0b2740 1 -- 172.21.15.39:0/1894745467 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55bc1211eca0 con 0x55bc11cce390 2024-09-21T03:56:25.410 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:25.405+0000 7f411ffff640 1 -- 172.21.15.39:0/1894745467 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f412c005000 con 0x55bc11cce390 2024-09-21T03:56:25.410 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:25.405+0000 7f411ffff640 1 -- 172.21.15.39:0/1894745467 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f412c010c20 con 0x55bc11cce390 2024-09-21T03:56:25.410 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:25.405+0000 7f414d0b2740 10 client.18324 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T03:56:25.410 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:25.405+0000 7f414d0b2740 10 client.18324 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-21T03:56:25.410 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:25.405+0000 7f414d0b2740 20 client.18324 populate_metadata read hostname 'smithi039' 2024-09-21T03:56:25.410 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:25.405+0000 7f414d0b2740 1 --2- 172.21.15.39:0/1894745467 >> [v2:172.21.15.161:6836/3451544597,v1:172.21.15.161:6837/3451544597] conn(0x55bc11c82da0 0x55bc12121260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:56:25.410 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:25.405+0000 7f414d0b2740 4 client.18324 mds_command: new command op to 24646 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T03:56:25.410 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:25.405+0000 7f414d0b2740 1 -- 172.21.15.39:0/1894745467 --> [v2:172.21.15.161:6836/3451544597,v1:172.21.15.161:6837/3451544597] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55bc1200a8a0 con 0x55bc11c82da0 2024-09-21T03:56:25.410 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:25.406+0000 7f41448da640 1 --2- 172.21.15.39:0/1894745467 >> [v2:172.21.15.161:6836/3451544597,v1:172.21.15.161:6837/3451544597] conn(0x55bc11c82da0 0x55bc12121260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:56:25.411 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:25.406+0000 7f41448da640 1 --2- 172.21.15.39:0/1894745467 >> [v2:172.21.15.161:6836/3451544597,v1:172.21.15.161:6837/3451544597] conn(0x55bc11c82da0 0x55bc12121260 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=d317807a8b8d3bed server_cookie=519092520eab9bf7 in_seq=0 out_seq=0 2024-09-21T03:56:25.411 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:25.406+0000 7f411ffff640 10 client.18324 ms_handle_connect on v2:172.21.15.161:6836/3451544597 2024-09-21T03:56:25.411 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:25.406+0000 7f411ffff640 1 -- 172.21.15.39:0/1894745467 <== mds.3 v2:172.21.15.161:6836/3451544597 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55bc1200a8a0 con 0x55bc11c82da0 2024-09-21T03:56:25.411 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:25.406+0000 7f411ffff640 10 client.18324 handle_command_reply: tid=0 2024-09-21T03:56:25.412 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:25.406+0000 7f414d0b2740 1 client.18324 shutdown 2024-09-21T03:56:25.412 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:25.406+0000 7f414d0b2740 1 -- 172.21.15.39:0/1894745467 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55bc11cce390 msgr2=0x55bc120a65e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:56:25.412 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:25.406+0000 7f414d0b2740 1 --2- 172.21.15.39:0/1894745467 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55bc11cce390 0x55bc120a65e0 secure :-1 s=READY pgs=1338 cs=0 l=1 rev1=1 crypto rx=0x7f412c004dd0 tx=0x7f412c0041b0 comp rx=0 tx=0).stop 2024-09-21T03:56:25.412 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:25.406+0000 7f414d0b2740 1 -- 172.21.15.39:0/1894745467 shutdown_connections 2024-09-21T03:56:25.412 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:25.406+0000 7f414d0b2740 1 --2- 172.21.15.39:0/1894745467 >> [v2:172.21.15.161:6836/3451544597,v1:172.21.15.161:6837/3451544597] conn(0x55bc11c82da0 0x55bc12121260 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:25.412 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:25.406+0000 7f414d0b2740 1 --2- 172.21.15.39:0/1894745467 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55bc11cef070 0x55bc120a6b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:25.412 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:25.406+0000 7f414d0b2740 1 --2- 172.21.15.39:0/1894745467 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55bc11cce390 0x55bc120a65e0 unknown :-1 s=CLOSED pgs=1338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:25.412 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:25.406+0000 7f414d0b2740 1 --2- 172.21.15.39:0/1894745467 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55bc11cc61b0 0x55bc120a60a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:25.412 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:25.406+0000 7f414d0b2740 1 -- 172.21.15.39:0/1894745467 >> 172.21.15.39:0/1894745467 conn(0x55bc11ce3460 msgr2=0x55bc120a3140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:56:25.412 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:25.406+0000 7f411ffff640 0 client.18324 ms_handle_reset on v2:172.21.15.161:6836/3451544597 2024-09-21T03:56:25.412 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:25.407+0000 7f414d0b2740 1 -- 172.21.15.39:0/1894745467 shutdown_connections 2024-09-21T03:56:25.412 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:25.407+0000 7f414d0b2740 1 -- 172.21.15.39:0/1894745467 wait complete. 2024-09-21T03:56:25.412 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:25.407+0000 7f414d0b2740 20 client.18324 trim_cache size 0 max 16384 2024-09-21T03:56:25.413 INFO:teuthology.orchestra.run.smithi039.stdout:[] 2024-09-21T03:56:25.413 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:25.407+0000 7f4146b3e640 1 -- 172.21.15.39:0/3773251689 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f41300b6700 msgr2=0x7f41300b8c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:56:25.413 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:25.407+0000 7f4146b3e640 1 --2- 172.21.15.39:0/3773251689 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f41300b6700 0x7f41300b8c10 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x55bc12095320 tx=0x7f413400b390 comp rx=0 tx=0).stop 2024-09-21T03:56:25.413 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:25.407+0000 7f4146b3e640 1 -- 172.21.15.39:0/3773251689 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4140108d20 msgr2=0x7f4140110a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:56:25.413 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:25.407+0000 7f4146b3e640 1 --2- 172.21.15.39:0/3773251689 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4140108d20 0x7f4140110a60 secure :-1 s=READY pgs=1334 cs=0 l=1 rev1=1 crypto rx=0x7f4138009430 tx=0x7f41380319f0 comp rx=0 tx=0).stop 2024-09-21T03:56:25.414 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:25.407+0000 7f4146b3e640 1 -- 172.21.15.39:0/3773251689 shutdown_connections 2024-09-21T03:56:25.414 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:25.407+0000 7f4146b3e640 1 --2- 172.21.15.39:0/3773251689 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f41300b6700 0x7f41300b8c10 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:25.414 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:25.407+0000 7f4146b3e640 1 --2- 172.21.15.39:0/3773251689 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4140108d20 0x7f4140110a60 unknown :-1 s=CLOSED pgs=1334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:25.414 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:25.407+0000 7f4146b3e640 1 --2- 172.21.15.39:0/3773251689 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f41401083d0 0x7f4140110520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:25.414 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:25.407+0000 7f4146b3e640 1 --2- 172.21.15.39:0/3773251689 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4140102ae0 0x7f414010ffe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:25.414 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:25.407+0000 7f4146b3e640 1 -- 172.21.15.39:0/3773251689 >> 172.21.15.39:0/3773251689 conn(0x7f4140079760 msgr2=0x7f4140104360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:56:25.414 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:25.408+0000 7f4146b3e640 1 -- 172.21.15.39:0/3773251689 shutdown_connections 2024-09-21T03:56:25.414 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:25.408+0000 7f4146b3e640 1 -- 172.21.15.39:0/3773251689 wait complete. 2024-09-21T03:56:25.468 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:56:25 smithi161 ceph-mon[34533]: pgmap v644: 97 pgs: 97 active+clean; 10 MiB data, 525 MiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 102 B/s wr, 13 op/s 2024-09-21T03:56:25.474 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:56:25 smithi150 ceph-mon[34289]: pgmap v644: 97 pgs: 97 active+clean; 10 MiB data, 525 MiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 102 B/s wr, 13 op/s 2024-09-21T03:56:26.293 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 51116d5c-77ca-11ef-baf5-efdc52797490 -- ceph tell mds.1:2 damage ls 2024-09-21T03:56:26.552 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/51116d5c-77ca-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:56:27.077 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:27.058+0000 7fe87d248640 1 -- 172.21.15.39:0/2974152340 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe878102a60 msgr2=0x7fe878102e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:56:27.077 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:27.058+0000 7fe87d248640 1 --2- 172.21.15.39:0/2974152340 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe878102a60 0x7fe878102e70 secure :-1 s=READY pgs=1339 cs=0 l=1 rev1=1 crypto rx=0x7fe870009900 tx=0x7fe87002a410 comp rx=0 tx=0).stop 2024-09-21T03:56:27.077 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:27.058+0000 7fe87d248640 1 -- 172.21.15.39:0/2974152340 shutdown_connections 2024-09-21T03:56:27.077 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:27.058+0000 7fe87d248640 1 --2- 172.21.15.39:0/2974152340 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe878108cf0 0x7fe87810f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:27.077 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:27.058+0000 7fe87d248640 1 --2- 172.21.15.39:0/2974152340 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe8781083a0 0x7fe8781087b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:27.077 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:27.058+0000 7fe87d248640 1 --2- 172.21.15.39:0/2974152340 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe878102a60 0x7fe878102e70 unknown :-1 s=CLOSED pgs=1339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:27.077 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:27.058+0000 7fe87d248640 1 -- 172.21.15.39:0/2974152340 >> 172.21.15.39:0/2974152340 conn(0x7fe878079730 msgr2=0x7fe878079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:56:27.077 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:27.058+0000 7fe87d248640 1 -- 172.21.15.39:0/2974152340 shutdown_connections 2024-09-21T03:56:27.077 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:27.058+0000 7fe87d248640 1 -- 172.21.15.39:0/2974152340 wait complete. 2024-09-21T03:56:27.077 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:27.059+0000 7fe87d248640 1 Processor -- start 2024-09-21T03:56:27.078 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:27.059+0000 7fe87d248640 1 -- start start 2024-09-21T03:56:27.078 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:27.059+0000 7fe87d248640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe878102a60 0x7fe87819a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:56:27.078 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:27.059+0000 7fe87d248640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe8781083a0 0x7fe87819aa50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:56:27.078 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:27.059+0000 7fe87d248640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe878108cf0 0x7fe87819fcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:56:27.078 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:27.059+0000 7fe87d248640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fe878113ae0 con 0x7fe8781083a0 2024-09-21T03:56:27.078 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:27.059+0000 7fe87d248640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fe878113960 con 0x7fe878108cf0 2024-09-21T03:56:27.078 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:27.059+0000 7fe87d248640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fe878113c60 con 0x7fe878102a60 2024-09-21T03:56:27.078 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:27.059+0000 7fe87659c640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe8781083a0 0x7fe87819aa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:56:27.078 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:27.060+0000 7fe87659c640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe8781083a0 0x7fe87819aa50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:55936/0 (socket says 172.21.15.39:55936) 2024-09-21T03:56:27.078 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:27.060+0000 7fe87659c640 1 -- 172.21.15.39:0/1790088167 learned_addr learned my addr 172.21.15.39:0/1790088167 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:56:27.078 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:27.060+0000 7fe876d9d640 1 --2- 172.21.15.39:0/1790088167 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe878102a60 0x7fe87819a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:56:27.078 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:27.060+0000 7fe87759e640 1 --2- 172.21.15.39:0/1790088167 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe878108cf0 0x7fe87819fcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:56:27.078 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:27.060+0000 7fe87659c640 1 -- 172.21.15.39:0/1790088167 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe878102a60 msgr2=0x7fe87819a4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:56:27.079 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:27.060+0000 7fe87659c640 1 --2- 172.21.15.39:0/1790088167 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe878102a60 0x7fe87819a4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:27.079 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:27.060+0000 7fe87659c640 1 -- 172.21.15.39:0/1790088167 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe878108cf0 msgr2=0x7fe87819fcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:56:27.079 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:27.060+0000 7fe87659c640 1 --2- 172.21.15.39:0/1790088167 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe878108cf0 0x7fe87819fcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:27.079 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:27.060+0000 7fe87659c640 1 -- 172.21.15.39:0/1790088167 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe8781a0420 con 0x7fe8781083a0 2024-09-21T03:56:27.079 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:27.060+0000 7fe87659c640 1 --2- 172.21.15.39:0/1790088167 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe8781083a0 0x7fe87819aa50 secure :-1 s=READY pgs=1340 cs=0 l=1 rev1=1 crypto rx=0x7fe86000b450 tx=0x7fe86000b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:56:27.079 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:27.060+0000 7fe86ffff640 1 -- 172.21.15.39:0/1790088167 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe860015020 con 0x7fe8781083a0 2024-09-21T03:56:27.079 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:27.060+0000 7fe87d248640 1 -- 172.21.15.39:0/1790088167 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe8781a0710 con 0x7fe8781083a0 2024-09-21T03:56:27.079 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:27.060+0000 7fe86ffff640 1 -- 172.21.15.39:0/1790088167 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe860011d40 con 0x7fe8781083a0 2024-09-21T03:56:27.079 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:27.060+0000 7fe86ffff640 1 -- 172.21.15.39:0/1790088167 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe86000f340 con 0x7fe8781083a0 2024-09-21T03:56:27.079 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:27.060+0000 7fe87d248640 1 -- 172.21.15.39:0/1790088167 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe8781a6e70 con 0x7fe8781083a0 2024-09-21T03:56:27.079 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:27.063+0000 7fe8837bc740 1 Processor -- start 2024-09-21T03:56:27.079 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:27.064+0000 7fe8837bc740 1 -- start start 2024-09-21T03:56:27.079 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:27.064+0000 7fe8837bc740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b6c6c16070 0x55b6c6fbd6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:56:27.079 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:27.064+0000 7fe8837bc740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b6c6bed1b0 0x55b6c6fbdbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:56:27.079 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:27.064+0000 7fe8837bc740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b6c6bf5390 0x55b6c6fbe120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:56:27.080 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:27.064+0000 7fe8837bc740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x55b6c6f33ea0 con 0x55b6c6bed1b0 2024-09-21T03:56:27.080 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:27.064+0000 7fe8837bc740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x55b6c6f232d0 con 0x55b6c6c16070 2024-09-21T03:56:27.080 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:27.064+0000 7fe8837bc740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55b6c6f270e0 con 0x55b6c6bf5390 2024-09-21T03:56:27.080 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:27.064+0000 7fe86ffff640 1 -- 172.21.15.39:0/1790088167 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 27) ==== 154643+0+0 (secure 0 0 0) 0x7fe860018020 con 0x7fe8781083a0 2024-09-21T03:56:27.080 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:27.064+0000 7fe876d9d640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b6c6bed1b0 0x55b6c6fbdbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:56:27.080 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:27.064+0000 7fe876d9d640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b6c6bed1b0 0x55b6c6fbdbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:55938/0 (socket says 172.21.15.39:55938) 2024-09-21T03:56:27.080 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:27.064+0000 7fe876d9d640 1 -- 172.21.15.39:0/1775226232 learned_addr learned my addr 172.21.15.39:0/1775226232 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:56:27.080 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:27.064+0000 7fe87759e640 1 --2- 172.21.15.39:0/1775226232 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b6c6c16070 0x55b6c6fbd6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:56:27.080 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:27.065+0000 7fe876d9d640 1 -- 172.21.15.39:0/1775226232 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b6c6bf5390 msgr2=0x55b6c6fbe120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:56:27.080 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:27.065+0000 7fe876d9d640 1 --2- 172.21.15.39:0/1775226232 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b6c6bf5390 0x55b6c6fbe120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:27.080 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:27.065+0000 7fe876d9d640 1 -- 172.21.15.39:0/1775226232 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b6c6c16070 msgr2=0x55b6c6fbd6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:56:27.080 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:27.065+0000 7fe876d9d640 1 --2- 172.21.15.39:0/1775226232 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b6c6c16070 0x55b6c6fbd6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:27.080 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:27.065+0000 7fe876d9d640 1 -- 172.21.15.39:0/1775226232 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b6c6c81650 con 0x55b6c6bed1b0 2024-09-21T03:56:27.080 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:27.065+0000 7fe876d9d640 1 --2- 172.21.15.39:0/1775226232 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b6c6bed1b0 0x55b6c6fbdbe0 secure :-1 s=READY pgs=1341 cs=0 l=1 rev1=1 crypto rx=0x7fe870009900 tx=0x7fe870031e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=55a39240f520e83b server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:56:27.081 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:27.065+0000 7fe86d7fa640 1 -- 172.21.15.39:0/1775226232 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe870040070 con 0x55b6c6bed1b0 2024-09-21T03:56:27.081 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:27.065+0000 7fe86d7fa640 1 -- 172.21.15.39:0/1775226232 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe8700316a0 con 0x55b6c6bed1b0 2024-09-21T03:56:27.081 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:27.065+0000 7fe86d7fa640 1 -- 172.21.15.39:0/1775226232 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe870035040 con 0x55b6c6bed1b0 2024-09-21T03:56:27.081 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:27.065+0000 7fe86ffff640 1 --2- 172.21.15.39:0/1790088167 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7fe8640b6680 0x7fe8640b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:56:27.081 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:27.066+0000 7fe87759e640 1 --2- 172.21.15.39:0/1790088167 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7fe8640b6680 0x7fe8640b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:56:27.081 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:27.066+0000 7fe86ffff640 1 -- 172.21.15.39:0/1790088167 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8528+0+0 (secure 0 0 0) 0x7fe8600e70b0 con 0x7fe8781083a0 2024-09-21T03:56:27.081 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:27.066+0000 7fe8837bc740 1 -- 172.21.15.39:0/1775226232 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b6c6bed1b0 msgr2=0x55b6c6fbdbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:56:27.081 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:27.066+0000 7fe8837bc740 1 --2- 172.21.15.39:0/1775226232 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b6c6bed1b0 0x55b6c6fbdbe0 secure :-1 s=READY pgs=1341 cs=0 l=1 rev1=1 crypto rx=0x7fe870009900 tx=0x7fe870031e50 comp rx=0 tx=0).stop 2024-09-21T03:56:27.081 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:27.066+0000 7fe87759e640 1 --2- 172.21.15.39:0/1790088167 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7fe8640b6680 0x7fe8640b8b90 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7fe868008e40 tx=0x7fe8680085f0 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:56:27.081 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:27.066+0000 7fe8837bc740 1 -- 172.21.15.39:0/1775226232 shutdown_connections 2024-09-21T03:56:27.081 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:27.066+0000 7fe8837bc740 1 --2- 172.21.15.39:0/1775226232 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b6c6bf5390 0x55b6c6fbe120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:27.081 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:27.066+0000 7fe8837bc740 1 --2- 172.21.15.39:0/1775226232 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b6c6bed1b0 0x55b6c6fbdbe0 unknown :-1 s=CLOSED pgs=1341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:27.081 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:27.066+0000 7fe8837bc740 1 --2- 172.21.15.39:0/1775226232 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b6c6c16070 0x55b6c6fbd6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:27.081 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:27.066+0000 7fe8837bc740 1 -- 172.21.15.39:0/1775226232 >> 172.21.15.39:0/1775226232 conn(0x55b6c6c0a460 msgr2=0x55b6c6fbb590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:56:27.082 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:27.066+0000 7fe8837bc740 1 -- 172.21.15.39:0/1775226232 shutdown_connections 2024-09-21T03:56:27.082 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:27.066+0000 7fe8837bc740 1 -- 172.21.15.39:0/1775226232 wait complete. 2024-09-21T03:56:27.082 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:27.067+0000 7fe8837bc740 1 -- start start 2024-09-21T03:56:27.082 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:27.067+0000 7fe8837bc740 1 Processor -- start 2024-09-21T03:56:27.082 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:27.067+0000 7fe8837bc740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b6c6bed1b0 0x55b6c6fc8de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:56:27.082 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:27.067+0000 7fe8837bc740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b6c6bf5390 0x55b6c6fc9320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:56:27.082 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:27.067+0000 7fe8837bc740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b6c6c16070 0x55b6c6fd5170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:56:27.082 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:27.067+0000 7fe8837bc740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x55b6c6d84700 con 0x55b6c6bf5390 2024-09-21T03:56:27.082 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:27.067+0000 7fe8837bc740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x55b6c6ebe160 con 0x55b6c6bed1b0 2024-09-21T03:56:27.082 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:27.067+0000 7fe8837bc740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55b6c6fcbe20 con 0x55b6c6c16070 2024-09-21T03:56:27.082 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:27.067+0000 7fe8837bc740 10 client.0 _ll_register_callbacks cb 0x55b6c6ed79e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:56:27.082 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:27.067+0000 7fe87659c640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b6c6bf5390 0x55b6c6fc9320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:56:27.082 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:27.067+0000 7fe87659c640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b6c6bf5390 0x55b6c6fc9320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:55952/0 (socket says 172.21.15.39:55952) 2024-09-21T03:56:27.082 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:27.067+0000 7fe87659c640 1 -- 172.21.15.39:0/1476988276 learned_addr learned my addr 172.21.15.39:0/1476988276 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:56:27.083 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:27.068+0000 7fe87759e640 1 --2- 172.21.15.39:0/1476988276 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b6c6c16070 0x55b6c6fd5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:56:27.083 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:27.068+0000 7fe876d9d640 1 --2- 172.21.15.39:0/1476988276 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b6c6bed1b0 0x55b6c6fc8de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:56:27.083 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:27.068+0000 7fe87659c640 1 -- 172.21.15.39:0/1476988276 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b6c6c16070 msgr2=0x55b6c6fd5170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:56:27.083 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:27.068+0000 7fe87659c640 1 --2- 172.21.15.39:0/1476988276 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b6c6c16070 0x55b6c6fd5170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:27.083 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:27.068+0000 7fe87659c640 1 -- 172.21.15.39:0/1476988276 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b6c6bed1b0 msgr2=0x55b6c6fc8de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:56:27.083 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:27.068+0000 7fe87659c640 1 --2- 172.21.15.39:0/1476988276 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b6c6bed1b0 0x55b6c6fc8de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:27.083 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:27.068+0000 7fe87659c640 1 -- 172.21.15.39:0/1476988276 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b6c6eebc90 con 0x55b6c6bf5390 2024-09-21T03:56:27.083 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:27.068+0000 7fe876d9d640 1 --2- 172.21.15.39:0/1476988276 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b6c6bed1b0 0x55b6c6fc8de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:56:27.083 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:27.068+0000 7fe87659c640 1 --2- 172.21.15.39:0/1476988276 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b6c6bf5390 0x55b6c6fc9320 secure :-1 s=READY pgs=1342 cs=0 l=1 rev1=1 crypto rx=0x7fe8600e4860 tx=0x7fe8600e4d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=30e079817f11195e server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:56:27.083 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:27.068+0000 7fe85e7fc640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-09-21T03:56:27.083 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:27.068+0000 7fe85e7fc640 1 -- 172.21.15.39:0/1476988276 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8600aca10 con 0x55b6c6bf5390 2024-09-21T03:56:27.083 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:27.068+0000 7fe85e7fc640 1 -- 172.21.15.39:0/1476988276 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8528+0+0 (secure 0 0 0) 0x7fe8600e90d0 con 0x55b6c6bf5390 2024-09-21T03:56:27.083 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:27.068+0000 7fe8837bc740 1 -- 172.21.15.39:0/1476988276 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe8700316a0 con 0x55b6c6bf5390 2024-09-21T03:56:27.083 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:27.068+0000 7fe85e7fc640 1 -- 172.21.15.39:0/1476988276 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fe860010ea0 con 0x55b6c6bf5390 2024-09-21T03:56:27.084 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:27.068+0000 7fe8837bc740 10 client.18336 fetch_fsmap learned FSMap version 29 2024-09-21T03:56:27.084 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:27.068+0000 7fe8837bc740 1 -- 172.21.15.39:0/1476988276 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55b6c6fd7bc0 con 0x55b6c6bf5390 2024-09-21T03:56:27.084 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:27.068+0000 7fe85e7fc640 1 -- 172.21.15.39:0/1476988276 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe8700316a0 con 0x55b6c6bf5390 2024-09-21T03:56:27.084 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:27.069+0000 7fe85e7fc640 1 -- 172.21.15.39:0/1476988276 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fe8600a8850 con 0x55b6c6bf5390 2024-09-21T03:56:27.084 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:27.069+0000 7fe8837bc740 10 client.18336 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T03:56:27.084 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:27.069+0000 7fe8837bc740 10 client.18336 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-21T03:56:27.084 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:27.069+0000 7fe8837bc740 20 client.18336 populate_metadata read hostname 'smithi039' 2024-09-21T03:56:27.084 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:27.069+0000 7fe8837bc740 1 --2- 172.21.15.39:0/1476988276 >> [v2:172.21.15.150:6836/4017043144,v1:172.21.15.150:6837/4017043144] conn(0x55b6c6bd49f0 0x55b6c6fdb030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:56:27.084 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:27.069+0000 7fe8837bc740 4 client.18336 mds_command: new command op to 24659 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:56:27.084 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:27.069+0000 7fe8837bc740 1 -- 172.21.15.39:0/1476988276 --> [v2:172.21.15.150:6836/4017043144,v1:172.21.15.150:6837/4017043144] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b6c6f35df0 con 0x55b6c6bd49f0 2024-09-21T03:56:27.084 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:27.069+0000 7fe876d9d640 1 --2- 172.21.15.39:0/1476988276 >> [v2:172.21.15.150:6836/4017043144,v1:172.21.15.150:6837/4017043144] conn(0x55b6c6bd49f0 0x55b6c6fdb030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:56:27.084 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:27.070+0000 7fe85e7fc640 10 client.18336 ms_handle_connect on v2:172.21.15.150:6836/4017043144 2024-09-21T03:56:27.084 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:27.070+0000 7fe876d9d640 1 --2- 172.21.15.39:0/1476988276 >> [v2:172.21.15.150:6836/4017043144,v1:172.21.15.150:6837/4017043144] conn(0x55b6c6bd49f0 0x55b6c6fdb030 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=ec868ff6ba948581 server_cookie=67f878c6b3a22fc5 in_seq=0 out_seq=0 2024-09-21T03:56:27.084 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:27.070+0000 7fe85e7fc640 1 -- 172.21.15.39:0/1476988276 <== mds.2 v2:172.21.15.150:6836/4017043144 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55b6c6f35df0 con 0x55b6c6bd49f0 2024-09-21T03:56:27.084 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:27.070+0000 7fe85e7fc640 10 client.18336 handle_command_reply: tid=0 2024-09-21T03:56:27.085 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:27.070+0000 7fe8837bc740 1 client.18336 shutdown 2024-09-21T03:56:27.085 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:27.070+0000 7fe8837bc740 1 -- 172.21.15.39:0/1476988276 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b6c6bf5390 msgr2=0x55b6c6fc9320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:56:27.085 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:27.070+0000 7fe8837bc740 1 --2- 172.21.15.39:0/1476988276 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b6c6bf5390 0x55b6c6fc9320 secure :-1 s=READY pgs=1342 cs=0 l=1 rev1=1 crypto rx=0x7fe8600e4860 tx=0x7fe8600e4d80 comp rx=0 tx=0).stop 2024-09-21T03:56:27.085 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:27.071+0000 7fe8837bc740 1 -- 172.21.15.39:0/1476988276 shutdown_connections 2024-09-21T03:56:27.085 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:27.071+0000 7fe8837bc740 1 --2- 172.21.15.39:0/1476988276 >> [v2:172.21.15.150:6836/4017043144,v1:172.21.15.150:6837/4017043144] conn(0x55b6c6bd49f0 0x55b6c6fdb030 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:27.085 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:27.071+0000 7fe8837bc740 1 --2- 172.21.15.39:0/1476988276 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b6c6c16070 0x55b6c6fd5170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:27.085 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:27.071+0000 7fe8837bc740 1 --2- 172.21.15.39:0/1476988276 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b6c6bf5390 0x55b6c6fc9320 unknown :-1 s=CLOSED pgs=1342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:27.085 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:27.071+0000 7fe8837bc740 1 --2- 172.21.15.39:0/1476988276 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b6c6bed1b0 0x55b6c6fc8de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:27.085 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:27.071+0000 7fe8837bc740 1 -- 172.21.15.39:0/1476988276 >> 172.21.15.39:0/1476988276 conn(0x55b6c6c0a460 msgr2=0x55b6c6fc0830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:56:27.085 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:27.071+0000 7fe85e7fc640 0 client.18336 ms_handle_reset on v2:172.21.15.150:6836/4017043144 2024-09-21T03:56:27.085 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:27.071+0000 7fe8837bc740 1 -- 172.21.15.39:0/1476988276 shutdown_connections 2024-09-21T03:56:27.085 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:27.071+0000 7fe8837bc740 1 -- 172.21.15.39:0/1476988276 wait complete. 2024-09-21T03:56:27.085 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:27.071+0000 7fe8837bc740 20 client.18336 trim_cache size 0 max 16384 2024-09-21T03:56:27.086 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:27.085+0000 7fe8837bc740 1 Processor -- start 2024-09-21T03:56:27.086 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:27.085+0000 7fe8837bc740 1 -- start start 2024-09-21T03:56:27.088 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:27.085+0000 7fe8837bc740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b6c6bed1b0 0x55b6c6fbf1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:56:27.088 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:27.085+0000 7fe8837bc740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b6c6bf5390 0x55b6c6fcacc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:56:27.088 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:27.085+0000 7fe8837bc740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b6c6c16070 0x55b6c6fd5070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:56:27.088 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:27.085+0000 7fe8837bc740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fe8781af7c0 con 0x55b6c6bed1b0 2024-09-21T03:56:27.088 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:27.085+0000 7fe8837bc740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fe8781b1300 con 0x55b6c6bf5390 2024-09-21T03:56:27.088 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:27.085+0000 7fe8837bc740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fe8781afe90 con 0x55b6c6c16070 2024-09-21T03:56:27.088 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:27.085+0000 7fe876d9d640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b6c6bed1b0 0x55b6c6fbf1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:56:27.088 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:27.085+0000 7fe876d9d640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b6c6bed1b0 0x55b6c6fbf1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:55954/0 (socket says 172.21.15.39:55954) 2024-09-21T03:56:27.088 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:27.085+0000 7fe876d9d640 1 -- 172.21.15.39:0/1929408179 learned_addr learned my addr 172.21.15.39:0/1929408179 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:56:27.088 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:27.085+0000 7fe87759e640 1 --2- 172.21.15.39:0/1929408179 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b6c6c16070 0x55b6c6fd5070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:56:27.089 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:27.086+0000 7fe87659c640 1 --2- 172.21.15.39:0/1929408179 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b6c6bf5390 0x55b6c6fcacc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:56:27.089 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:27.086+0000 7fe876d9d640 1 -- 172.21.15.39:0/1929408179 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b6c6c16070 msgr2=0x55b6c6fd5070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:56:27.089 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:27.086+0000 7fe876d9d640 1 --2- 172.21.15.39:0/1929408179 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b6c6c16070 0x55b6c6fd5070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:27.089 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:27.086+0000 7fe876d9d640 1 -- 172.21.15.39:0/1929408179 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b6c6bf5390 msgr2=0x55b6c6fcacc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:56:27.089 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:27.086+0000 7fe876d9d640 1 --2- 172.21.15.39:0/1929408179 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b6c6bf5390 0x55b6c6fcacc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:27.089 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:27.086+0000 7fe876d9d640 1 -- 172.21.15.39:0/1929408179 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b6c6fbbe80 con 0x55b6c6bed1b0 2024-09-21T03:56:27.089 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:27.086+0000 7fe876d9d640 1 --2- 172.21.15.39:0/1929408179 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b6c6bed1b0 0x55b6c6fbf1a0 secure :-1 s=READY pgs=1343 cs=0 l=1 rev1=1 crypto rx=0x7fe870031c70 tx=0x7fe870005430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=36f2ef4e32af559 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:56:27.089 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:27.086+0000 7fe86cff9640 1 -- 172.21.15.39:0/1929408179 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe870040040 con 0x55b6c6bed1b0 2024-09-21T03:56:27.089 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:27.086+0000 7fe86cff9640 1 -- 172.21.15.39:0/1929408179 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe870031950 con 0x55b6c6bed1b0 2024-09-21T03:56:27.089 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:27.086+0000 7fe86cff9640 1 -- 172.21.15.39:0/1929408179 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe870035040 con 0x55b6c6bed1b0 2024-09-21T03:56:27.092 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:27.087+0000 7fe8837bc740 1 -- 172.21.15.39:0/1929408179 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b6c6bed1b0 msgr2=0x55b6c6fbf1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:56:27.092 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:27.087+0000 7fe8837bc740 1 --2- 172.21.15.39:0/1929408179 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b6c6bed1b0 0x55b6c6fbf1a0 secure :-1 s=READY pgs=1343 cs=0 l=1 rev1=1 crypto rx=0x7fe870031c70 tx=0x7fe870005430 comp rx=0 tx=0).stop 2024-09-21T03:56:27.092 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:27.087+0000 7fe8837bc740 1 -- 172.21.15.39:0/1929408179 shutdown_connections 2024-09-21T03:56:27.092 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:27.087+0000 7fe8837bc740 1 --2- 172.21.15.39:0/1929408179 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b6c6c16070 0x55b6c6fd5070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:27.092 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:27.087+0000 7fe8837bc740 1 --2- 172.21.15.39:0/1929408179 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b6c6bf5390 0x55b6c6fcacc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:27.092 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:27.087+0000 7fe8837bc740 1 --2- 172.21.15.39:0/1929408179 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b6c6bed1b0 0x55b6c6fbf1a0 unknown :-1 s=CLOSED pgs=1343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:27.092 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:27.087+0000 7fe8837bc740 1 -- 172.21.15.39:0/1929408179 >> 172.21.15.39:0/1929408179 conn(0x55b6c6c0a460 msgr2=0x55b6c6fc27b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:56:27.093 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:27.087+0000 7fe8837bc740 1 -- 172.21.15.39:0/1929408179 shutdown_connections 2024-09-21T03:56:27.093 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:27.087+0000 7fe8837bc740 1 -- 172.21.15.39:0/1929408179 wait complete. 2024-09-21T03:56:27.093 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:27.088+0000 7fe8837bc740 1 -- start start 2024-09-21T03:56:27.093 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:27.088+0000 7fe8837bc740 1 Processor -- start 2024-09-21T03:56:27.093 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:27.088+0000 7fe8837bc740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b6c6bed1b0 0x55b6c6fcd010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:56:27.093 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:27.088+0000 7fe8837bc740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b6c6bf5390 0x55b6c6fcd550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:56:27.093 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:27.088+0000 7fe8837bc740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b6c6c16070 0x55b6c6fcda90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:56:27.093 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:27.088+0000 7fe8837bc740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fe8781b0c30 con 0x55b6c6bf5390 2024-09-21T03:56:27.093 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:27.088+0000 7fe8837bc740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fe8781b0560 con 0x55b6c6bed1b0 2024-09-21T03:56:27.093 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:27.088+0000 7fe8837bc740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fe8781ad7a0 con 0x55b6c6c16070 2024-09-21T03:56:27.093 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:27.088+0000 7fe8837bc740 10 client.0 _ll_register_callbacks cb 0x7fe860011ef0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:56:27.093 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:27.088+0000 7fe87659c640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b6c6bf5390 0x55b6c6fcd550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:56:27.093 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:27.088+0000 7fe87659c640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b6c6bf5390 0x55b6c6fcd550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:55970/0 (socket says 172.21.15.39:55970) 2024-09-21T03:56:27.093 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:27.088+0000 7fe87659c640 1 -- 172.21.15.39:0/1014066209 learned_addr learned my addr 172.21.15.39:0/1014066209 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:56:27.094 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:27.088+0000 7fe876d9d640 1 --2- 172.21.15.39:0/1014066209 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b6c6bed1b0 0x55b6c6fcd010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:56:27.094 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:27.088+0000 7fe87759e640 1 --2- 172.21.15.39:0/1014066209 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b6c6c16070 0x55b6c6fcda90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:56:27.094 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:27.088+0000 7fe87659c640 1 -- 172.21.15.39:0/1014066209 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b6c6c16070 msgr2=0x55b6c6fcda90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:56:27.094 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:27.088+0000 7fe87659c640 1 --2- 172.21.15.39:0/1014066209 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b6c6c16070 0x55b6c6fcda90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:27.094 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:27.088+0000 7fe87659c640 1 -- 172.21.15.39:0/1014066209 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b6c6bed1b0 msgr2=0x55b6c6fcd010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:56:27.094 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:27.088+0000 7fe87659c640 1 --2- 172.21.15.39:0/1014066209 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b6c6bed1b0 0x55b6c6fcd010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:27.094 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:27.089+0000 7fe87659c640 1 -- 172.21.15.39:0/1014066209 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b6c70449f0 con 0x55b6c6bf5390 2024-09-21T03:56:27.094 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:27.089+0000 7fe87759e640 1 --2- 172.21.15.39:0/1014066209 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b6c6c16070 0x55b6c6fcda90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:56:27.094 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:27.089+0000 7fe87659c640 1 --2- 172.21.15.39:0/1014066209 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b6c6bf5390 0x55b6c6fcd550 secure :-1 s=READY pgs=1344 cs=0 l=1 rev1=1 crypto rx=0x7fe860009fd0 tx=0x7fe8600abca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=409ac796e0e0fdde server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:56:27.094 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:27.089+0000 7fe85e7fc640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-09-21T03:56:27.094 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:27.089+0000 7fe85e7fc640 1 -- 172.21.15.39:0/1014066209 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8600ad600 con 0x55b6c6bf5390 2024-09-21T03:56:27.094 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:27.089+0000 7fe85e7fc640 1 -- 172.21.15.39:0/1014066209 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8528+0+0 (secure 0 0 0) 0x7fe8600abdc0 con 0x55b6c6bf5390 2024-09-21T03:56:27.094 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:27.089+0000 7fe8837bc740 1 -- 172.21.15.39:0/1014066209 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe870031950 con 0x55b6c6bf5390 2024-09-21T03:56:27.095 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:27.089+0000 7fe85e7fc640 1 -- 172.21.15.39:0/1014066209 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fe8600b19e0 con 0x55b6c6bf5390 2024-09-21T03:56:27.095 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:27.089+0000 7fe8837bc740 10 client.18342 fetch_fsmap learned FSMap version 29 2024-09-21T03:56:27.095 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:27.089+0000 7fe8837bc740 1 -- 172.21.15.39:0/1014066209 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55b6c7045bd0 con 0x55b6c6bf5390 2024-09-21T03:56:27.095 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:27.089+0000 7fe85e7fc640 1 -- 172.21.15.39:0/1014066209 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe870031950 con 0x55b6c6bf5390 2024-09-21T03:56:27.095 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:27.089+0000 7fe85e7fc640 1 -- 172.21.15.39:0/1014066209 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fe8600e90d0 con 0x55b6c6bf5390 2024-09-21T03:56:27.095 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:27.090+0000 7fe8837bc740 10 client.18342 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T03:56:27.095 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:27.090+0000 7fe8837bc740 10 client.18342 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-21T03:56:27.095 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:27.090+0000 7fe8837bc740 20 client.18342 populate_metadata read hostname 'smithi039' 2024-09-21T03:56:27.095 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:27.090+0000 7fe8837bc740 1 --2- 172.21.15.39:0/1014066209 >> [v2:172.21.15.150:6836/4017043144,v1:172.21.15.150:6837/4017043144] conn(0x55b6c6ba9da0 0x55b6c7048360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:56:27.095 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:27.090+0000 7fe8837bc740 4 client.18342 mds_command: new command op to 24659 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T03:56:27.095 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:27.090+0000 7fe8837bc740 1 -- 172.21.15.39:0/1014066209 --> [v2:172.21.15.150:6836/4017043144,v1:172.21.15.150:6837/4017043144] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55b6c6f318a0 con 0x55b6c6ba9da0 2024-09-21T03:56:27.095 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:27.090+0000 7fe876d9d640 1 --2- 172.21.15.39:0/1014066209 >> [v2:172.21.15.150:6836/4017043144,v1:172.21.15.150:6837/4017043144] conn(0x55b6c6ba9da0 0x55b6c7048360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:56:27.095 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:27.090+0000 7fe85e7fc640 10 client.18342 ms_handle_connect on v2:172.21.15.150:6836/4017043144 2024-09-21T03:56:27.095 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:27.090+0000 7fe876d9d640 1 --2- 172.21.15.39:0/1014066209 >> [v2:172.21.15.150:6836/4017043144,v1:172.21.15.150:6837/4017043144] conn(0x55b6c6ba9da0 0x55b6c7048360 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=c056b7e4792e29af server_cookie=1903fa9a4463db79 in_seq=0 out_seq=0 2024-09-21T03:56:27.096 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:27.091+0000 7fe85e7fc640 1 -- 172.21.15.39:0/1014066209 <== mds.2 v2:172.21.15.150:6836/4017043144 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55b6c6f318a0 con 0x55b6c6ba9da0 2024-09-21T03:56:27.096 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:27.091+0000 7fe85e7fc640 10 client.18342 handle_command_reply: tid=0 2024-09-21T03:56:27.096 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:27.091+0000 7fe8837bc740 1 client.18342 shutdown 2024-09-21T03:56:27.097 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:27.091+0000 7fe8837bc740 1 -- 172.21.15.39:0/1014066209 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b6c6bf5390 msgr2=0x55b6c6fcd550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:56:27.097 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:27.091+0000 7fe8837bc740 1 --2- 172.21.15.39:0/1014066209 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b6c6bf5390 0x55b6c6fcd550 secure :-1 s=READY pgs=1344 cs=0 l=1 rev1=1 crypto rx=0x7fe860009fd0 tx=0x7fe8600abca0 comp rx=0 tx=0).stop 2024-09-21T03:56:27.097 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:27.091+0000 7fe8837bc740 1 -- 172.21.15.39:0/1014066209 shutdown_connections 2024-09-21T03:56:27.097 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:27.091+0000 7fe8837bc740 1 --2- 172.21.15.39:0/1014066209 >> [v2:172.21.15.150:6836/4017043144,v1:172.21.15.150:6837/4017043144] conn(0x55b6c6ba9da0 0x55b6c7048360 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:27.097 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:27.091+0000 7fe8837bc740 1 --2- 172.21.15.39:0/1014066209 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b6c6c16070 0x55b6c6fcda90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:27.097 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:27.091+0000 7fe8837bc740 1 --2- 172.21.15.39:0/1014066209 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b6c6bf5390 0x55b6c6fcd550 unknown :-1 s=CLOSED pgs=1344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:27.097 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:27.091+0000 7fe8837bc740 1 --2- 172.21.15.39:0/1014066209 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b6c6bed1b0 0x55b6c6fcd010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:27.097 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:27.091+0000 7fe8837bc740 1 -- 172.21.15.39:0/1014066209 >> 172.21.15.39:0/1014066209 conn(0x55b6c6c0a460 msgr2=0x55b6c6fc27b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:56:27.097 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:27.091+0000 7fe85e7fc640 0 client.18342 ms_handle_reset on v2:172.21.15.150:6836/4017043144 2024-09-21T03:56:27.097 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:27.091+0000 7fe8837bc740 1 -- 172.21.15.39:0/1014066209 shutdown_connections 2024-09-21T03:56:27.097 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:27.091+0000 7fe8837bc740 1 -- 172.21.15.39:0/1014066209 wait complete. 2024-09-21T03:56:27.098 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:27.091+0000 7fe8837bc740 20 client.18342 trim_cache size 0 max 16384 2024-09-21T03:56:27.098 INFO:teuthology.orchestra.run.smithi039.stdout:[] 2024-09-21T03:56:27.098 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:27.092+0000 7fe87d248640 1 -- 172.21.15.39:0/1790088167 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7fe8640b6680 msgr2=0x7fe8640b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:56:27.098 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:27.092+0000 7fe87d248640 1 --2- 172.21.15.39:0/1790088167 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7fe8640b6680 0x7fe8640b8b90 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7fe868008e40 tx=0x7fe8680085f0 comp rx=0 tx=0).stop 2024-09-21T03:56:27.098 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:27.092+0000 7fe87d248640 1 -- 172.21.15.39:0/1790088167 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe8781083a0 msgr2=0x7fe87819aa50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:56:27.099 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:27.092+0000 7fe87d248640 1 --2- 172.21.15.39:0/1790088167 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe8781083a0 0x7fe87819aa50 secure :-1 s=READY pgs=1340 cs=0 l=1 rev1=1 crypto rx=0x7fe86000b450 tx=0x7fe86000b910 comp rx=0 tx=0).stop 2024-09-21T03:56:27.099 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:27.092+0000 7fe87d248640 1 -- 172.21.15.39:0/1790088167 shutdown_connections 2024-09-21T03:56:27.099 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:27.092+0000 7fe87d248640 1 --2- 172.21.15.39:0/1790088167 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7fe8640b6680 0x7fe8640b8b90 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:27.099 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:27.092+0000 7fe87d248640 1 --2- 172.21.15.39:0/1790088167 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe878108cf0 0x7fe87819fcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:27.099 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:27.092+0000 7fe87d248640 1 --2- 172.21.15.39:0/1790088167 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe8781083a0 0x7fe87819aa50 unknown :-1 s=CLOSED pgs=1340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:27.099 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:27.092+0000 7fe87d248640 1 --2- 172.21.15.39:0/1790088167 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe878102a60 0x7fe87819a4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:27.099 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:27.092+0000 7fe87d248640 1 -- 172.21.15.39:0/1790088167 >> 172.21.15.39:0/1790088167 conn(0x7fe878079730 msgr2=0x7fe8781013d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:56:27.099 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:27.092+0000 7fe87d248640 1 -- 172.21.15.39:0/1790088167 shutdown_connections 2024-09-21T03:56:27.099 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:27.092+0000 7fe87d248640 1 -- 172.21.15.39:0/1790088167 wait complete. 2024-09-21T03:56:27.333 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:56:27 smithi039 ceph-mon[31211]: pgmap v645: 97 pgs: 97 active+clean; 10 MiB data, 525 MiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 102 B/s wr, 13 op/s 2024-09-21T03:56:27.468 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:56:27 smithi161 ceph-mon[34533]: pgmap v645: 97 pgs: 97 active+clean; 10 MiB data, 525 MiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 102 B/s wr, 13 op/s 2024-09-21T03:56:27.474 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:56:27 smithi150 ceph-mon[34289]: pgmap v645: 97 pgs: 97 active+clean; 10 MiB data, 525 MiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 102 B/s wr, 13 op/s 2024-09-21T03:56:27.938 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 51116d5c-77ca-11ef-baf5-efdc52797490 -- ceph tell mds.1:1 damage ls 2024-09-21T03:56:28.199 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/51116d5c-77ca-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:56:28.662 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:28.659+0000 7fdc8e4d8640 1 -- 172.21.15.39:0/2060399295 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdc88102a80 msgr2=0x7fdc88102e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:56:28.662 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:28.659+0000 7fdc8e4d8640 1 --2- 172.21.15.39:0/2060399295 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdc88102a80 0x7fdc88102e90 secure :-1 s=READY pgs=1345 cs=0 l=1 rev1=1 crypto rx=0x7fdc78009900 tx=0x7fdc7802a410 comp rx=0 tx=0).stop 2024-09-21T03:56:28.662 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:28.659+0000 7fdc8e4d8640 1 -- 172.21.15.39:0/2060399295 shutdown_connections 2024-09-21T03:56:28.662 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:28.659+0000 7fdc8e4d8640 1 --2- 172.21.15.39:0/2060399295 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdc88108d10 0x7fdc8810f540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:28.662 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:28.660+0000 7fdc8e4d8640 1 --2- 172.21.15.39:0/2060399295 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdc881083c0 0x7fdc881087d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:28.663 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:28.660+0000 7fdc8e4d8640 1 --2- 172.21.15.39:0/2060399295 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdc88102a80 0x7fdc88102e90 unknown :-1 s=CLOSED pgs=1345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:28.663 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:28.660+0000 7fdc8e4d8640 1 -- 172.21.15.39:0/2060399295 >> 172.21.15.39:0/2060399295 conn(0x7fdc88079740 msgr2=0x7fdc88079b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:56:28.663 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:28.660+0000 7fdc8e4d8640 1 -- 172.21.15.39:0/2060399295 shutdown_connections 2024-09-21T03:56:28.663 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:28.660+0000 7fdc8e4d8640 1 -- 172.21.15.39:0/2060399295 wait complete. 2024-09-21T03:56:28.663 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:28.660+0000 7fdc8e4d8640 1 Processor -- start 2024-09-21T03:56:28.663 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:28.660+0000 7fdc8e4d8640 1 -- start start 2024-09-21T03:56:28.664 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:28.661+0000 7fdc8e4d8640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdc88102a80 0x7fdc8819a500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:56:28.664 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:28.661+0000 7fdc8e4d8640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdc881083c0 0x7fdc8819aaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:56:28.665 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:28.661+0000 7fdc8e4d8640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdc88108d10 0x7fdc8819fd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:56:28.665 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:28.661+0000 7fdc8e4d8640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fdc8806c670 con 0x7fdc881083c0 2024-09-21T03:56:28.665 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:28.661+0000 7fdc8e4d8640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fdc8806c4f0 con 0x7fdc88108d10 2024-09-21T03:56:28.665 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:28.661+0000 7fdc8e4d8640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fdc8806c7f0 con 0x7fdc88102a80 2024-09-21T03:56:28.665 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:28.661+0000 7fdc877fe640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdc881083c0 0x7fdc8819aaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:56:28.665 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:28.661+0000 7fdc877fe640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdc881083c0 0x7fdc8819aaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:48616/0 (socket says 172.21.15.39:48616) 2024-09-21T03:56:28.665 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:28.661+0000 7fdc877fe640 1 -- 172.21.15.39:0/3420600209 learned_addr learned my addr 172.21.15.39:0/3420600209 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:56:28.665 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:28.661+0000 7fdc87fff640 1 --2- 172.21.15.39:0/3420600209 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdc88102a80 0x7fdc8819a500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:56:28.665 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:28.661+0000 7fdc8ca75640 1 --2- 172.21.15.39:0/3420600209 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdc88108d10 0x7fdc8819fd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:56:28.665 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:28.661+0000 7fdc877fe640 1 -- 172.21.15.39:0/3420600209 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdc88102a80 msgr2=0x7fdc8819a500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:56:28.665 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:28.661+0000 7fdc877fe640 1 --2- 172.21.15.39:0/3420600209 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdc88102a80 0x7fdc8819a500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:28.665 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:28.661+0000 7fdc877fe640 1 -- 172.21.15.39:0/3420600209 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdc88108d10 msgr2=0x7fdc8819fd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:56:28.666 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:28.661+0000 7fdc877fe640 1 --2- 172.21.15.39:0/3420600209 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdc88108d10 0x7fdc8819fd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:28.666 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:28.661+0000 7fdc877fe640 1 -- 172.21.15.39:0/3420600209 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdc881a0470 con 0x7fdc881083c0 2024-09-21T03:56:28.666 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:28.661+0000 7fdc8ca75640 1 --2- 172.21.15.39:0/3420600209 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdc88108d10 0x7fdc8819fd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:56:28.666 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:28.661+0000 7fdc87fff640 1 --2- 172.21.15.39:0/3420600209 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdc88102a80 0x7fdc8819a500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:56:28.666 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:28.662+0000 7fdc877fe640 1 --2- 172.21.15.39:0/3420600209 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdc881083c0 0x7fdc8819aaa0 secure :-1 s=READY pgs=1346 cs=0 l=1 rev1=1 crypto rx=0x7fdc7c00d770 tx=0x7fdc7c00dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:56:28.666 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:28.662+0000 7fdc857fa640 1 -- 172.21.15.39:0/3420600209 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc7c017070 con 0x7fdc881083c0 2024-09-21T03:56:28.666 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:28.662+0000 7fdc8e4d8640 1 -- 172.21.15.39:0/3420600209 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdc881a0760 con 0x7fdc881083c0 2024-09-21T03:56:28.666 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:28.662+0000 7fdc8e4d8640 1 -- 172.21.15.39:0/3420600209 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdc881a6ec0 con 0x7fdc881083c0 2024-09-21T03:56:28.666 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:28.662+0000 7fdc857fa640 1 -- 172.21.15.39:0/3420600209 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdc7c009c30 con 0x7fdc881083c0 2024-09-21T03:56:28.666 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:28.662+0000 7fdc857fa640 1 -- 172.21.15.39:0/3420600209 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc7c021060 con 0x7fdc881083c0 2024-09-21T03:56:28.669 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:28.665+0000 7fdc94a4c740 1 Processor -- start 2024-09-21T03:56:28.669 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:28.665+0000 7fdc94a4c740 1 -- start start 2024-09-21T03:56:28.669 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:28.665+0000 7fdc857fa640 1 -- 172.21.15.39:0/3420600209 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 27) ==== 154643+0+0 (secure 0 0 0) 0x7fdc7c01a070 con 0x7fdc881083c0 2024-09-21T03:56:28.669 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:28.665+0000 7fdc94a4c740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d85c606070 0x55d85c9ad6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:56:28.669 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:28.665+0000 7fdc94a4c740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d85c5dd1b0 0x55d85c9adbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:56:28.669 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:28.665+0000 7fdc94a4c740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d85c5e5390 0x55d85c9ae120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:56:28.669 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:28.665+0000 7fdc94a4c740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x55d85c923ea0 con 0x55d85c606070 2024-09-21T03:56:28.669 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:28.665+0000 7fdc94a4c740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x55d85c9132d0 con 0x55d85c5e5390 2024-09-21T03:56:28.669 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:28.665+0000 7fdc94a4c740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55d85c9170e0 con 0x55d85c5dd1b0 2024-09-21T03:56:28.670 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:28.665+0000 7fdc8ca75640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d85c606070 0x55d85c9ad6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:56:28.670 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:28.665+0000 7fdc8ca75640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d85c606070 0x55d85c9ad6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:48626/0 (socket says 172.21.15.39:48626) 2024-09-21T03:56:28.670 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:28.665+0000 7fdc8ca75640 1 -- 172.21.15.39:0/1942765872 learned_addr learned my addr 172.21.15.39:0/1942765872 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:56:28.670 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:28.665+0000 7fdc877fe640 1 --2- 172.21.15.39:0/1942765872 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d85c5e5390 0x55d85c9ae120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:56:28.670 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:28.665+0000 7fdc87fff640 1 --2- 172.21.15.39:0/1942765872 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d85c5dd1b0 0x55d85c9adbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:56:28.670 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:28.665+0000 7fdc8ca75640 1 -- 172.21.15.39:0/1942765872 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d85c5dd1b0 msgr2=0x55d85c9adbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:56:28.670 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:28.666+0000 7fdc8ca75640 1 --2- 172.21.15.39:0/1942765872 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d85c5dd1b0 0x55d85c9adbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:28.670 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:28.666+0000 7fdc8ca75640 1 -- 172.21.15.39:0/1942765872 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d85c5e5390 msgr2=0x55d85c9ae120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:56:28.670 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:28.666+0000 7fdc8ca75640 1 --2- 172.21.15.39:0/1942765872 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d85c5e5390 0x55d85c9ae120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:28.670 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:28.666+0000 7fdc8ca75640 1 -- 172.21.15.39:0/1942765872 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d85c671650 con 0x55d85c606070 2024-09-21T03:56:28.670 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:28.666+0000 7fdc877fe640 1 --2- 172.21.15.39:0/1942765872 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d85c5e5390 0x55d85c9ae120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:56:28.670 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:28.666+0000 7fdc857fa640 1 --2- 172.21.15.39:0/3420600209 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7fdc700b6680 0x7fdc700b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:56:28.671 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:28.666+0000 7fdc8ca75640 1 --2- 172.21.15.39:0/1942765872 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d85c606070 0x55d85c9ad6a0 secure :-1 s=READY pgs=1347 cs=0 l=1 rev1=1 crypto rx=0x7fdc8000ae80 tx=0x7fdc8000e720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b7ed8de3ba6a4cd server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:56:28.671 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:28.666+0000 7fdc6e7fc640 1 -- 172.21.15.39:0/1942765872 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc80003b70 con 0x55d85c606070 2024-09-21T03:56:28.671 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:28.666+0000 7fdc87fff640 1 --2- 172.21.15.39:0/3420600209 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7fdc700b6680 0x7fdc700b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:56:28.671 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:28.666+0000 7fdc6e7fc640 1 -- 172.21.15.39:0/1942765872 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdc80003d10 con 0x55d85c606070 2024-09-21T03:56:28.671 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:28.666+0000 7fdc6e7fc640 1 -- 172.21.15.39:0/1942765872 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc80010a00 con 0x55d85c606070 2024-09-21T03:56:28.671 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:28.666+0000 7fdc857fa640 1 -- 172.21.15.39:0/3420600209 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8528+0+0 (secure 0 0 0) 0x7fdc7c0e72d0 con 0x7fdc881083c0 2024-09-21T03:56:28.671 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:28.667+0000 7fdc87fff640 1 --2- 172.21.15.39:0/3420600209 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7fdc700b6680 0x7fdc700b8b90 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7fdc78031440 tx=0x7fdc78009640 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:56:28.671 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:28.667+0000 7fdc94a4c740 1 -- 172.21.15.39:0/1942765872 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d85c606070 msgr2=0x55d85c9ad6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:56:28.671 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:28.667+0000 7fdc94a4c740 1 --2- 172.21.15.39:0/1942765872 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d85c606070 0x55d85c9ad6a0 secure :-1 s=READY pgs=1347 cs=0 l=1 rev1=1 crypto rx=0x7fdc8000ae80 tx=0x7fdc8000e720 comp rx=0 tx=0).stop 2024-09-21T03:56:28.671 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:28.667+0000 7fdc94a4c740 1 -- 172.21.15.39:0/1942765872 shutdown_connections 2024-09-21T03:56:28.671 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:28.667+0000 7fdc94a4c740 1 --2- 172.21.15.39:0/1942765872 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d85c5e5390 0x55d85c9ae120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:28.671 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:28.667+0000 7fdc94a4c740 1 --2- 172.21.15.39:0/1942765872 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d85c5dd1b0 0x55d85c9adbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:28.671 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:28.667+0000 7fdc94a4c740 1 --2- 172.21.15.39:0/1942765872 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d85c606070 0x55d85c9ad6a0 unknown :-1 s=CLOSED pgs=1347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:28.672 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:28.667+0000 7fdc94a4c740 1 -- 172.21.15.39:0/1942765872 >> 172.21.15.39:0/1942765872 conn(0x55d85c5fa460 msgr2=0x55d85c9ab590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:56:28.672 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:28.667+0000 7fdc94a4c740 1 -- 172.21.15.39:0/1942765872 shutdown_connections 2024-09-21T03:56:28.672 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:28.667+0000 7fdc94a4c740 1 -- 172.21.15.39:0/1942765872 wait complete. 2024-09-21T03:56:28.675 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:28.667+0000 7fdc94a4c740 1 -- start start 2024-09-21T03:56:28.675 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:28.668+0000 7fdc94a4c740 1 Processor -- start 2024-09-21T03:56:28.675 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:28.668+0000 7fdc94a4c740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d85c5dd1b0 0x55d85c9b8d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:56:28.675 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:28.668+0000 7fdc94a4c740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d85c5e5390 0x55d85c9b9270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:56:28.675 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:28.668+0000 7fdc94a4c740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d85c606070 0x55d85c9c5170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:56:28.675 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:28.668+0000 7fdc94a4c740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x55d85c774700 con 0x55d85c5e5390 2024-09-21T03:56:28.675 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:28.668+0000 7fdc94a4c740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x55d85c8ae160 con 0x55d85c5dd1b0 2024-09-21T03:56:28.675 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:28.668+0000 7fdc94a4c740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55d85c9bbe80 con 0x55d85c606070 2024-09-21T03:56:28.675 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:28.668+0000 7fdc94a4c740 10 client.0 _ll_register_callbacks cb 0x55d85c8c79e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:56:28.676 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:28.668+0000 7fdc877fe640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d85c5e5390 0x55d85c9b9270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:56:28.676 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:28.668+0000 7fdc877fe640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d85c5e5390 0x55d85c9b9270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:48638/0 (socket says 172.21.15.39:48638) 2024-09-21T03:56:28.676 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:28.668+0000 7fdc877fe640 1 -- 172.21.15.39:0/179530010 learned_addr learned my addr 172.21.15.39:0/179530010 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:56:28.676 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:28.668+0000 7fdc8ca75640 1 --2- 172.21.15.39:0/179530010 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d85c606070 0x55d85c9c5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:56:28.676 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:28.669+0000 7fdc877fe640 1 -- 172.21.15.39:0/179530010 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d85c606070 msgr2=0x55d85c9c5170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:56:28.676 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:28.669+0000 7fdc877fe640 1 --2- 172.21.15.39:0/179530010 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d85c606070 0x55d85c9c5170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:28.676 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:28.669+0000 7fdc877fe640 1 -- 172.21.15.39:0/179530010 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d85c5dd1b0 msgr2=0x55d85c9b8d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:56:28.676 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:28.669+0000 7fdc877fe640 1 --2- 172.21.15.39:0/179530010 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d85c5dd1b0 0x55d85c9b8d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:28.676 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:28.669+0000 7fdc877fe640 1 -- 172.21.15.39:0/179530010 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d85c8dbc90 con 0x55d85c5e5390 2024-09-21T03:56:28.676 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:28.669+0000 7fdc877fe640 1 --2- 172.21.15.39:0/179530010 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d85c5e5390 0x55d85c9b9270 secure :-1 s=READY pgs=1348 cs=0 l=1 rev1=1 crypto rx=0x7fdc7c0ad0b0 tx=0x7fdc7c0eaba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c1963e40574c158f server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:56:28.676 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:28.669+0000 7fdc677fe640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-09-21T03:56:28.676 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:28.669+0000 7fdc677fe640 1 -- 172.21.15.39:0/179530010 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc7c0e6030 con 0x55d85c5e5390 2024-09-21T03:56:28.677 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:28.669+0000 7fdc677fe640 1 -- 172.21.15.39:0/179530010 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8528+0+0 (secure 0 0 0) 0x7fdc7c0e4ca0 con 0x55d85c5e5390 2024-09-21T03:56:28.677 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:28.669+0000 7fdc94a4c740 1 -- 172.21.15.39:0/179530010 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdc80003d10 con 0x55d85c5e5390 2024-09-21T03:56:28.677 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:28.669+0000 7fdc677fe640 1 -- 172.21.15.39:0/179530010 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fdc7c0acef0 con 0x55d85c5e5390 2024-09-21T03:56:28.677 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:28.669+0000 7fdc94a4c740 10 client.18354 fetch_fsmap learned FSMap version 29 2024-09-21T03:56:28.677 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:28.669+0000 7fdc94a4c740 1 -- 172.21.15.39:0/179530010 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55d85c9c7bc0 con 0x55d85c5e5390 2024-09-21T03:56:28.677 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:28.669+0000 7fdc677fe640 1 -- 172.21.15.39:0/179530010 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdc80003d10 con 0x55d85c5e5390 2024-09-21T03:56:28.677 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:28.670+0000 7fdc677fe640 1 -- 172.21.15.39:0/179530010 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fdc7c0a8a70 con 0x55d85c5e5390 2024-09-21T03:56:28.677 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:28.670+0000 7fdc94a4c740 10 client.18354 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T03:56:28.677 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:28.670+0000 7fdc94a4c740 10 client.18354 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-21T03:56:28.677 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:28.670+0000 7fdc94a4c740 20 client.18354 populate_metadata read hostname 'smithi039' 2024-09-21T03:56:28.677 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:28.670+0000 7fdc94a4c740 1 --2- 172.21.15.39:0/179530010 >> [v2:172.21.15.161:6838/2190107270,v1:172.21.15.161:6839/2190107270] conn(0x55d85c5c49f0 0x55d85c9cb050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:56:28.677 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:28.670+0000 7fdc94a4c740 4 client.18354 mds_command: new command op to 24667 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:56:28.678 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:28.670+0000 7fdc94a4c740 1 -- 172.21.15.39:0/179530010 --> [v2:172.21.15.161:6838/2190107270,v1:172.21.15.161:6839/2190107270] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d85c925df0 con 0x55d85c5c49f0 2024-09-21T03:56:28.678 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:28.670+0000 7fdc87fff640 1 --2- 172.21.15.39:0/179530010 >> [v2:172.21.15.161:6838/2190107270,v1:172.21.15.161:6839/2190107270] conn(0x55d85c5c49f0 0x55d85c9cb050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:56:28.678 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:28.670+0000 7fdc677fe640 10 client.18354 ms_handle_connect on v2:172.21.15.161:6838/2190107270 2024-09-21T03:56:28.678 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:28.670+0000 7fdc87fff640 1 --2- 172.21.15.39:0/179530010 >> [v2:172.21.15.161:6838/2190107270,v1:172.21.15.161:6839/2190107270] conn(0x55d85c5c49f0 0x55d85c9cb050 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=8693c8870657619c server_cookie=4369eefc31eb5497 in_seq=0 out_seq=0 2024-09-21T03:56:28.678 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:28.671+0000 7fdc677fe640 1 -- 172.21.15.39:0/179530010 <== mds.1 v2:172.21.15.161:6838/2190107270 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55d85c925df0 con 0x55d85c5c49f0 2024-09-21T03:56:28.678 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:28.671+0000 7fdc677fe640 10 client.18354 handle_command_reply: tid=0 2024-09-21T03:56:28.678 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:28.671+0000 7fdc94a4c740 1 client.18354 shutdown 2024-09-21T03:56:28.678 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:28.671+0000 7fdc94a4c740 1 -- 172.21.15.39:0/179530010 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d85c5e5390 msgr2=0x55d85c9b9270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:56:28.678 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:28.671+0000 7fdc94a4c740 1 --2- 172.21.15.39:0/179530010 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d85c5e5390 0x55d85c9b9270 secure :-1 s=READY pgs=1348 cs=0 l=1 rev1=1 crypto rx=0x7fdc7c0ad0b0 tx=0x7fdc7c0eaba0 comp rx=0 tx=0).stop 2024-09-21T03:56:28.678 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:28.672+0000 7fdc94a4c740 1 -- 172.21.15.39:0/179530010 shutdown_connections 2024-09-21T03:56:28.678 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:28.672+0000 7fdc94a4c740 1 --2- 172.21.15.39:0/179530010 >> [v2:172.21.15.161:6838/2190107270,v1:172.21.15.161:6839/2190107270] conn(0x55d85c5c49f0 0x55d85c9cb050 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:28.678 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:28.672+0000 7fdc94a4c740 1 --2- 172.21.15.39:0/179530010 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d85c606070 0x55d85c9c5170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:28.679 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:28.672+0000 7fdc94a4c740 1 --2- 172.21.15.39:0/179530010 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d85c5e5390 0x55d85c9b9270 unknown :-1 s=CLOSED pgs=1348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:28.679 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:28.672+0000 7fdc677fe640 0 client.18354 ms_handle_reset on v2:172.21.15.161:6838/2190107270 2024-09-21T03:56:28.679 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:28.672+0000 7fdc94a4c740 1 --2- 172.21.15.39:0/179530010 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d85c5dd1b0 0x55d85c9b8d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:28.679 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:28.672+0000 7fdc94a4c740 1 -- 172.21.15.39:0/179530010 >> 172.21.15.39:0/179530010 conn(0x55d85c5fa460 msgr2=0x55d85c9b0880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:56:28.679 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:28.672+0000 7fdc94a4c740 1 -- 172.21.15.39:0/179530010 shutdown_connections 2024-09-21T03:56:28.679 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:28.672+0000 7fdc94a4c740 1 -- 172.21.15.39:0/179530010 wait complete. 2024-09-21T03:56:28.679 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:28.672+0000 7fdc94a4c740 20 client.18354 trim_cache size 0 max 16384 2024-09-21T03:56:28.688 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:28.685+0000 7fdc94a4c740 1 Processor -- start 2024-09-21T03:56:28.688 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:28.685+0000 7fdc94a4c740 1 -- start start 2024-09-21T03:56:28.688 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:28.685+0000 7fdc94a4c740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d85c5dd1b0 0x55d85c9b91f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:56:28.688 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:28.685+0000 7fdc94a4c740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d85c5e5390 0x55d85c9b07a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:56:28.688 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:28.685+0000 7fdc94a4c740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d85c606070 0x55d85c9b0ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:56:28.688 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:28.685+0000 7fdc94a4c740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fdc881af830 con 0x55d85c5dd1b0 2024-09-21T03:56:28.688 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:28.685+0000 7fdc94a4c740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fdc881b1370 con 0x55d85c606070 2024-09-21T03:56:28.688 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:28.685+0000 7fdc94a4c740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fdc881aff00 con 0x55d85c5e5390 2024-09-21T03:56:28.689 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:28.685+0000 7fdc87fff640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d85c5dd1b0 0x55d85c9b91f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:56:28.689 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:28.685+0000 7fdc87fff640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d85c5dd1b0 0x55d85c9b91f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:48646/0 (socket says 172.21.15.39:48646) 2024-09-21T03:56:28.689 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:28.685+0000 7fdc87fff640 1 -- 172.21.15.39:0/1060157520 learned_addr learned my addr 172.21.15.39:0/1060157520 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:56:28.689 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:28.686+0000 7fdc877fe640 1 --2- 172.21.15.39:0/1060157520 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d85c5e5390 0x55d85c9b07a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:56:28.689 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:28.686+0000 7fdc8ca75640 1 --2- 172.21.15.39:0/1060157520 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d85c606070 0x55d85c9b0ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:56:28.689 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:28.686+0000 7fdc87fff640 1 -- 172.21.15.39:0/1060157520 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d85c5e5390 msgr2=0x55d85c9b07a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:56:28.689 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:28.686+0000 7fdc87fff640 1 --2- 172.21.15.39:0/1060157520 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d85c5e5390 0x55d85c9b07a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:28.689 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:28.686+0000 7fdc87fff640 1 -- 172.21.15.39:0/1060157520 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d85c606070 msgr2=0x55d85c9b0ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:56:28.689 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:28.686+0000 7fdc87fff640 1 --2- 172.21.15.39:0/1060157520 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d85c606070 0x55d85c9b0ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:28.689 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:28.686+0000 7fdc87fff640 1 -- 172.21.15.39:0/1060157520 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d85c9b0500 con 0x55d85c5dd1b0 2024-09-21T03:56:28.689 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:28.686+0000 7fdc8ca75640 1 --2- 172.21.15.39:0/1060157520 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d85c606070 0x55d85c9b0ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:56:28.689 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:28.686+0000 7fdc87fff640 1 --2- 172.21.15.39:0/1060157520 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d85c5dd1b0 0x55d85c9b91f0 secure :-1 s=READY pgs=1349 cs=0 l=1 rev1=1 crypto rx=0x7fdc7802a930 tx=0x7fdc7803d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f2f125fea02a0266 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:56:28.690 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:28.686+0000 7fdc6dffb640 1 -- 172.21.15.39:0/1060157520 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc78053070 con 0x55d85c5dd1b0 2024-09-21T03:56:28.690 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:28.686+0000 7fdc6dffb640 1 -- 172.21.15.39:0/1060157520 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdc78005040 con 0x55d85c5dd1b0 2024-09-21T03:56:28.690 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:28.686+0000 7fdc6dffb640 1 -- 172.21.15.39:0/1060157520 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc78035050 con 0x55d85c5dd1b0 2024-09-21T03:56:28.693 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:28.687+0000 7fdc94a4c740 1 -- 172.21.15.39:0/1060157520 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d85c5dd1b0 msgr2=0x55d85c9b91f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:56:28.693 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:28.687+0000 7fdc94a4c740 1 --2- 172.21.15.39:0/1060157520 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d85c5dd1b0 0x55d85c9b91f0 secure :-1 s=READY pgs=1349 cs=0 l=1 rev1=1 crypto rx=0x7fdc7802a930 tx=0x7fdc7803d000 comp rx=0 tx=0).stop 2024-09-21T03:56:28.693 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:28.687+0000 7fdc94a4c740 1 -- 172.21.15.39:0/1060157520 shutdown_connections 2024-09-21T03:56:28.693 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:28.687+0000 7fdc94a4c740 1 --2- 172.21.15.39:0/1060157520 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d85c606070 0x55d85c9b0ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:28.694 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:28.687+0000 7fdc94a4c740 1 --2- 172.21.15.39:0/1060157520 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d85c5e5390 0x55d85c9b07a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:28.694 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:28.687+0000 7fdc94a4c740 1 --2- 172.21.15.39:0/1060157520 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d85c5dd1b0 0x55d85c9b91f0 unknown :-1 s=CLOSED pgs=1349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:28.694 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:28.687+0000 7fdc94a4c740 1 -- 172.21.15.39:0/1060157520 >> 172.21.15.39:0/1060157520 conn(0x55d85c5fa460 msgr2=0x55d85c9b8ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:56:28.694 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:28.687+0000 7fdc94a4c740 1 -- 172.21.15.39:0/1060157520 shutdown_connections 2024-09-21T03:56:28.694 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:28.687+0000 7fdc94a4c740 1 -- 172.21.15.39:0/1060157520 wait complete. 2024-09-21T03:56:28.694 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:28.687+0000 7fdc94a4c740 1 -- start start 2024-09-21T03:56:28.694 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:28.688+0000 7fdc94a4c740 1 Processor -- start 2024-09-21T03:56:28.694 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:28.688+0000 7fdc94a4c740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d85c5dd1b0 0x55d85c9bd990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:56:28.694 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:28.688+0000 7fdc94a4c740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d85c5e5390 0x55d85c9bdf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:56:28.694 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:28.688+0000 7fdc94a4c740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d85c606070 0x55d85ca2ea20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:56:28.694 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:28.688+0000 7fdc94a4c740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fdc881b0ca0 con 0x55d85c5dd1b0 2024-09-21T03:56:28.695 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:28.688+0000 7fdc94a4c740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fdc881b05d0 con 0x55d85c606070 2024-09-21T03:56:28.695 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:28.688+0000 7fdc94a4c740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fdc881ad810 con 0x55d85c5e5390 2024-09-21T03:56:28.695 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:28.688+0000 7fdc94a4c740 10 client.0 _ll_register_callbacks cb 0x7fdc7c00d6c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:56:28.695 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:28.688+0000 7fdc87fff640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d85c5dd1b0 0x55d85c9bd990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:56:28.695 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:28.688+0000 7fdc87fff640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d85c5dd1b0 0x55d85c9bd990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:48650/0 (socket says 172.21.15.39:48650) 2024-09-21T03:56:28.695 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:28.688+0000 7fdc87fff640 1 -- 172.21.15.39:0/1549370928 learned_addr learned my addr 172.21.15.39:0/1549370928 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:56:28.695 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:28.688+0000 7fdc8ca75640 1 --2- 172.21.15.39:0/1549370928 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d85c606070 0x55d85ca2ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:56:28.695 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:28.688+0000 7fdc877fe640 1 --2- 172.21.15.39:0/1549370928 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d85c5e5390 0x55d85c9bdf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:56:28.695 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:28.689+0000 7fdc87fff640 1 -- 172.21.15.39:0/1549370928 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d85c5e5390 msgr2=0x55d85c9bdf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:56:28.695 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:28.689+0000 7fdc87fff640 1 --2- 172.21.15.39:0/1549370928 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d85c5e5390 0x55d85c9bdf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:28.695 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:28.689+0000 7fdc87fff640 1 -- 172.21.15.39:0/1549370928 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d85c606070 msgr2=0x55d85ca2ea20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:56:28.695 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:28.689+0000 7fdc87fff640 1 --2- 172.21.15.39:0/1549370928 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d85c606070 0x55d85ca2ea20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:28.695 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:28.689+0000 7fdc87fff640 1 -- 172.21.15.39:0/1549370928 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d85ca2f090 con 0x55d85c5dd1b0 2024-09-21T03:56:28.696 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:28.689+0000 7fdc877fe640 1 --2- 172.21.15.39:0/1549370928 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d85c5e5390 0x55d85c9bdf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:56:28.696 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:28.689+0000 7fdc87fff640 1 --2- 172.21.15.39:0/1549370928 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d85c5dd1b0 0x55d85c9bd990 secure :-1 s=READY pgs=1350 cs=0 l=1 rev1=1 crypto rx=0x7fdc78005270 tx=0x7fdc780052a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=841a97e4cb6c2899 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:56:28.696 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:28.689+0000 7fdc677fe640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-09-21T03:56:28.696 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:28.689+0000 7fdc677fe640 1 -- 172.21.15.39:0/1549370928 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc78035070 con 0x55d85c5dd1b0 2024-09-21T03:56:28.696 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:28.689+0000 7fdc677fe640 1 -- 172.21.15.39:0/1549370928 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8528+0+0 (secure 0 0 0) 0x7fdc780046c0 con 0x55d85c5dd1b0 2024-09-21T03:56:28.696 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:28.689+0000 7fdc94a4c740 1 -- 172.21.15.39:0/1549370928 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdc78005040 con 0x55d85c5dd1b0 2024-09-21T03:56:28.696 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:28.689+0000 7fdc677fe640 1 -- 172.21.15.39:0/1549370928 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fdc780048c0 con 0x55d85c5dd1b0 2024-09-21T03:56:28.696 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:28.689+0000 7fdc94a4c740 10 client.18360 fetch_fsmap learned FSMap version 29 2024-09-21T03:56:28.696 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:28.689+0000 7fdc94a4c740 1 -- 172.21.15.39:0/1549370928 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55d85ca36320 con 0x55d85c5dd1b0 2024-09-21T03:56:28.696 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:28.689+0000 7fdc677fe640 1 -- 172.21.15.39:0/1549370928 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdc78005040 con 0x55d85c5dd1b0 2024-09-21T03:56:28.696 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:28.690+0000 7fdc677fe640 1 -- 172.21.15.39:0/1549370928 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fdc780614e0 con 0x55d85c5dd1b0 2024-09-21T03:56:28.696 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:28.690+0000 7fdc94a4c740 10 client.18360 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T03:56:28.697 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:28.690+0000 7fdc94a4c740 10 client.18360 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-21T03:56:28.697 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:28.690+0000 7fdc94a4c740 20 client.18360 populate_metadata read hostname 'smithi039' 2024-09-21T03:56:28.697 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:28.690+0000 7fdc94a4c740 1 --2- 172.21.15.39:0/1549370928 >> [v2:172.21.15.161:6838/2190107270,v1:172.21.15.161:6839/2190107270] conn(0x55d85c599da0 0x55d85ca38ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:56:28.697 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:28.690+0000 7fdc94a4c740 4 client.18360 mds_command: new command op to 24667 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T03:56:28.697 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:28.690+0000 7fdc94a4c740 1 -- 172.21.15.39:0/1549370928 --> [v2:172.21.15.161:6838/2190107270,v1:172.21.15.161:6839/2190107270] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55d85c9218a0 con 0x55d85c599da0 2024-09-21T03:56:28.697 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:28.690+0000 7fdc877fe640 1 --2- 172.21.15.39:0/1549370928 >> [v2:172.21.15.161:6838/2190107270,v1:172.21.15.161:6839/2190107270] conn(0x55d85c599da0 0x55d85ca38ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:56:28.697 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:28.690+0000 7fdc877fe640 1 --2- 172.21.15.39:0/1549370928 >> [v2:172.21.15.161:6838/2190107270,v1:172.21.15.161:6839/2190107270] conn(0x55d85c599da0 0x55d85ca38ab0 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=668d3517e73262 server_cookie=dbe06d0266ab3c40 in_seq=0 out_seq=0 2024-09-21T03:56:28.697 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:28.690+0000 7fdc677fe640 10 client.18360 ms_handle_connect on v2:172.21.15.161:6838/2190107270 2024-09-21T03:56:28.697 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:28.691+0000 7fdc677fe640 1 -- 172.21.15.39:0/1549370928 <== mds.1 v2:172.21.15.161:6838/2190107270 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55d85c9218a0 con 0x55d85c599da0 2024-09-21T03:56:28.697 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:28.691+0000 7fdc677fe640 10 client.18360 handle_command_reply: tid=0 2024-09-21T03:56:28.697 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:28.691+0000 7fdc94a4c740 1 client.18360 shutdown 2024-09-21T03:56:28.697 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:28.691+0000 7fdc94a4c740 1 -- 172.21.15.39:0/1549370928 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d85c5dd1b0 msgr2=0x55d85c9bd990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:56:28.697 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:28.691+0000 7fdc94a4c740 1 --2- 172.21.15.39:0/1549370928 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d85c5dd1b0 0x55d85c9bd990 secure :-1 s=READY pgs=1350 cs=0 l=1 rev1=1 crypto rx=0x7fdc78005270 tx=0x7fdc780052a0 comp rx=0 tx=0).stop 2024-09-21T03:56:28.698 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:28.691+0000 7fdc94a4c740 1 -- 172.21.15.39:0/1549370928 shutdown_connections 2024-09-21T03:56:28.698 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:28.691+0000 7fdc94a4c740 1 --2- 172.21.15.39:0/1549370928 >> [v2:172.21.15.161:6838/2190107270,v1:172.21.15.161:6839/2190107270] conn(0x55d85c599da0 0x55d85ca38ab0 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:28.698 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:28.691+0000 7fdc94a4c740 1 --2- 172.21.15.39:0/1549370928 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d85c606070 0x55d85ca2ea20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:28.698 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:28.691+0000 7fdc94a4c740 1 --2- 172.21.15.39:0/1549370928 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d85c5e5390 0x55d85c9bdf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:28.698 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:28.691+0000 7fdc677fe640 0 client.18360 ms_handle_reset on v2:172.21.15.161:6838/2190107270 2024-09-21T03:56:28.698 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:28.691+0000 7fdc94a4c740 1 --2- 172.21.15.39:0/1549370928 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d85c5dd1b0 0x55d85c9bd990 unknown :-1 s=CLOSED pgs=1350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:28.698 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:28.691+0000 7fdc94a4c740 1 -- 172.21.15.39:0/1549370928 >> 172.21.15.39:0/1549370928 conn(0x55d85c5fa460 msgr2=0x55d85c9b07a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:56:28.698 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:28.691+0000 7fdc94a4c740 1 -- 172.21.15.39:0/1549370928 shutdown_connections 2024-09-21T03:56:28.698 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:28.691+0000 7fdc94a4c740 1 -- 172.21.15.39:0/1549370928 wait complete. 2024-09-21T03:56:28.698 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:28.691+0000 7fdc94a4c740 20 client.18360 trim_cache size 0 max 16384 2024-09-21T03:56:28.699 INFO:teuthology.orchestra.run.smithi039.stdout:[] 2024-09-21T03:56:28.699 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:28.692+0000 7fdc8e4d8640 1 -- 172.21.15.39:0/3420600209 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7fdc700b6680 msgr2=0x7fdc700b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:56:28.699 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:28.692+0000 7fdc8e4d8640 1 --2- 172.21.15.39:0/3420600209 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7fdc700b6680 0x7fdc700b8b90 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7fdc78031440 tx=0x7fdc78009640 comp rx=0 tx=0).stop 2024-09-21T03:56:28.699 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:28.692+0000 7fdc8e4d8640 1 -- 172.21.15.39:0/3420600209 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdc881083c0 msgr2=0x7fdc8819aaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:56:28.700 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:28.692+0000 7fdc8e4d8640 1 --2- 172.21.15.39:0/3420600209 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdc881083c0 0x7fdc8819aaa0 secure :-1 s=READY pgs=1346 cs=0 l=1 rev1=1 crypto rx=0x7fdc7c00d770 tx=0x7fdc7c00dc30 comp rx=0 tx=0).stop 2024-09-21T03:56:28.700 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:28.692+0000 7fdc8e4d8640 1 -- 172.21.15.39:0/3420600209 shutdown_connections 2024-09-21T03:56:28.700 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:28.692+0000 7fdc8e4d8640 1 --2- 172.21.15.39:0/3420600209 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7fdc700b6680 0x7fdc700b8b90 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:28.700 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:28.692+0000 7fdc8e4d8640 1 --2- 172.21.15.39:0/3420600209 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdc88108d10 0x7fdc8819fd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:28.700 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:28.692+0000 7fdc8e4d8640 1 --2- 172.21.15.39:0/3420600209 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdc881083c0 0x7fdc8819aaa0 unknown :-1 s=CLOSED pgs=1346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:28.700 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:28.692+0000 7fdc8e4d8640 1 --2- 172.21.15.39:0/3420600209 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdc88102a80 0x7fdc8819a500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:28.700 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:28.692+0000 7fdc8e4d8640 1 -- 172.21.15.39:0/3420600209 >> 172.21.15.39:0/3420600209 conn(0x7fdc88079740 msgr2=0x7fdc881014d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:56:28.700 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:28.692+0000 7fdc8e4d8640 1 -- 172.21.15.39:0/3420600209 shutdown_connections 2024-09-21T03:56:28.700 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:28.692+0000 7fdc8e4d8640 1 -- 172.21.15.39:0/3420600209 wait complete. 2024-09-21T03:56:29.446 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:56:29 smithi039 ceph-mon[31211]: pgmap v646: 97 pgs: 97 active+clean; 10 MiB data, 525 MiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 102 B/s wr, 13 op/s 2024-09-21T03:56:29.468 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:56:29 smithi161 ceph-mon[34533]: pgmap v646: 97 pgs: 97 active+clean; 10 MiB data, 525 MiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 102 B/s wr, 13 op/s 2024-09-21T03:56:29.474 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:56:29 smithi150 ceph-mon[34289]: pgmap v646: 97 pgs: 97 active+clean; 10 MiB data, 525 MiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 102 B/s wr, 13 op/s 2024-09-21T03:56:29.476 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 51116d5c-77ca-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 damage ls 2024-09-21T03:56:29.732 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/51116d5c-77ca-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:56:30.214 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:30.213+0000 7f5249fb6640 1 -- 172.21.15.39:0/1858844946 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f52441083c0 msgr2=0x7f52441087d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:56:30.215 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:30.213+0000 7f5249fb6640 1 --2- 172.21.15.39:0/1858844946 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f52441083c0 0x7f52441087d0 secure :-1 s=READY pgs=1351 cs=0 l=1 rev1=1 crypto rx=0x7f5238009900 tx=0x7f523802a410 comp rx=0 tx=0).stop 2024-09-21T03:56:30.215 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:30.213+0000 7f5249fb6640 1 -- 172.21.15.39:0/1858844946 shutdown_connections 2024-09-21T03:56:30.215 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:30.213+0000 7f5249fb6640 1 --2- 172.21.15.39:0/1858844946 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5244108d10 0x7f524410f540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:30.215 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:30.213+0000 7f5249fb6640 1 --2- 172.21.15.39:0/1858844946 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f52441083c0 0x7f52441087d0 unknown :-1 s=CLOSED pgs=1351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:30.215 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:30.213+0000 7f5249fb6640 1 --2- 172.21.15.39:0/1858844946 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5244102a80 0x7f5244102e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:30.215 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:30.213+0000 7f5249fb6640 1 -- 172.21.15.39:0/1858844946 >> 172.21.15.39:0/1858844946 conn(0x7f5244079740 msgr2=0x7f5244079b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:56:30.215 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:30.213+0000 7f5249fb6640 1 -- 172.21.15.39:0/1858844946 shutdown_connections 2024-09-21T03:56:30.215 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:30.213+0000 7f5249fb6640 1 -- 172.21.15.39:0/1858844946 wait complete. 2024-09-21T03:56:30.216 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:30.213+0000 7f5249fb6640 1 Processor -- start 2024-09-21T03:56:30.216 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:30.214+0000 7f5249fb6640 1 -- start start 2024-09-21T03:56:30.216 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:30.214+0000 7f5249fb6640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5244102a80 0x7f524419a570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:56:30.216 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:30.214+0000 7f5249fb6640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f52441083c0 0x7f524419ab10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:56:30.216 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:30.214+0000 7f5249fb6640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5244108d10 0x7f524419fdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:56:30.217 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:30.214+0000 7f5249fb6640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f5244113aa0 con 0x7f5244108d10 2024-09-21T03:56:30.217 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:30.214+0000 7f5249fb6640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f5244113920 con 0x7f5244102a80 2024-09-21T03:56:30.217 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:30.214+0000 7f5249fb6640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f5244113c20 con 0x7f52441083c0 2024-09-21T03:56:30.217 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:30.214+0000 7f5243fff640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5244108d10 0x7f524419fdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:56:30.217 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:30.214+0000 7f5242ffd640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f52441083c0 0x7f524419ab10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:56:30.217 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:30.214+0000 7f5243fff640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5244108d10 0x7f524419fdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:48666/0 (socket says 172.21.15.39:48666) 2024-09-21T03:56:30.217 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:30.214+0000 7f5243fff640 1 -- 172.21.15.39:0/1668349820 learned_addr learned my addr 172.21.15.39:0/1668349820 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:56:30.217 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:30.214+0000 7f52437fe640 1 --2- 172.21.15.39:0/1668349820 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5244102a80 0x7f524419a570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:56:30.217 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:30.214+0000 7f5243fff640 1 -- 172.21.15.39:0/1668349820 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f52441083c0 msgr2=0x7f524419ab10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:56:30.217 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:30.214+0000 7f5243fff640 1 --2- 172.21.15.39:0/1668349820 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f52441083c0 0x7f524419ab10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:30.217 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:30.214+0000 7f5243fff640 1 -- 172.21.15.39:0/1668349820 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5244102a80 msgr2=0x7f524419a570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:56:30.217 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:30.214+0000 7f5243fff640 1 --2- 172.21.15.39:0/1668349820 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5244102a80 0x7f524419a570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:30.217 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:30.214+0000 7f5243fff640 1 -- 172.21.15.39:0/1668349820 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f52441a04e0 con 0x7f5244108d10 2024-09-21T03:56:30.217 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:30.215+0000 7f5243fff640 1 --2- 172.21.15.39:0/1668349820 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5244108d10 0x7f524419fdb0 secure :-1 s=READY pgs=1352 cs=0 l=1 rev1=1 crypto rx=0x7f523c00eaa0 tx=0x7f523c00ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:56:30.218 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:30.215+0000 7f5240ff9640 1 -- 172.21.15.39:0/1668349820 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f523c00cdb0 con 0x7f5244108d10 2024-09-21T03:56:30.218 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:30.215+0000 7f5249fb6640 1 -- 172.21.15.39:0/1668349820 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f52441a07d0 con 0x7f5244108d10 2024-09-21T03:56:30.218 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:30.215+0000 7f5240ff9640 1 -- 172.21.15.39:0/1668349820 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f523c005020 con 0x7f5244108d10 2024-09-21T03:56:30.218 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:30.215+0000 7f5240ff9640 1 -- 172.21.15.39:0/1668349820 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f523c0116d0 con 0x7f5244108d10 2024-09-21T03:56:30.218 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:30.215+0000 7f5249fb6640 1 -- 172.21.15.39:0/1668349820 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f52441a6f30 con 0x7f5244108d10 2024-09-21T03:56:30.220 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:30.217+0000 7f525052a740 1 Processor -- start 2024-09-21T03:56:30.220 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:30.217+0000 7f5240ff9640 1 -- 172.21.15.39:0/1668349820 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 27) ==== 154643+0+0 (secure 0 0 0) 0x7f523c0051d0 con 0x7f5244108d10 2024-09-21T03:56:30.220 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:30.218+0000 7f525052a740 1 -- start start 2024-09-21T03:56:30.220 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:30.218+0000 7f525052a740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5622c4878070 0x5622c4c1f2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:56:30.220 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:30.218+0000 7f525052a740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5622c484f1b0 0x5622c4c1f800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:56:30.220 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:30.218+0000 7f525052a740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5622c4857390 0x5622c4c1fd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:56:30.220 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:30.218+0000 7f525052a740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x5622c4b95ea0 con 0x5622c484f1b0 2024-09-21T03:56:30.220 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:30.218+0000 7f525052a740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x5622c4b852d0 con 0x5622c4878070 2024-09-21T03:56:30.220 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:30.218+0000 7f525052a740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x5622c4b890e0 con 0x5622c4857390 2024-09-21T03:56:30.221 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:30.218+0000 7f52437fe640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5622c484f1b0 0x5622c4c1f800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:56:30.221 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:30.218+0000 7f52437fe640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5622c484f1b0 0x5622c4c1f800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:48672/0 (socket says 172.21.15.39:48672) 2024-09-21T03:56:30.221 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:30.218+0000 7f52437fe640 1 -- 172.21.15.39:0/2669670308 learned_addr learned my addr 172.21.15.39:0/2669670308 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:56:30.221 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:30.218+0000 7f5242ffd640 1 --2- 172.21.15.39:0/2669670308 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5622c4878070 0x5622c4c1f2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:56:30.221 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:30.218+0000 7f5242ffd640 1 --2- 172.21.15.39:0/2669670308 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5622c4857390 0x5622c4c1fd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:56:30.221 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:30.218+0000 7f52437fe640 1 -- 172.21.15.39:0/2669670308 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5622c4857390 msgr2=0x5622c4c1fd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:56:30.222 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:30.218+0000 7f52437fe640 1 --2- 172.21.15.39:0/2669670308 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5622c4857390 0x5622c4c1fd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:30.222 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:30.218+0000 7f52437fe640 1 -- 172.21.15.39:0/2669670308 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5622c4878070 msgr2=0x5622c4c1f2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:56:30.222 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:30.218+0000 7f52437fe640 1 --2- 172.21.15.39:0/2669670308 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5622c4878070 0x5622c4c1f2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:30.222 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:30.218+0000 7f52437fe640 1 -- 172.21.15.39:0/2669670308 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5622c48e3650 con 0x5622c484f1b0 2024-09-21T03:56:30.222 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:30.218+0000 7f5240ff9640 1 --2- 172.21.15.39:0/1668349820 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f522c0b6700 0x7f522c0b8c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:56:30.223 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:30.218+0000 7f5242ffd640 1 --2- 172.21.15.39:0/2669670308 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5622c4878070 0x5622c4c1f2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:56:30.223 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:30.219+0000 7f5242ffd640 1 --2- 172.21.15.39:0/1668349820 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f522c0b6700 0x7f522c0b8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:56:30.223 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:30.219+0000 7f5240ff9640 1 -- 172.21.15.39:0/1668349820 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8528+0+0 (secure 0 0 0) 0x7f523c0e7770 con 0x7f5244108d10 2024-09-21T03:56:30.223 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:30.219+0000 7f52437fe640 1 --2- 172.21.15.39:0/2669670308 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5622c484f1b0 0x5622c4c1f800 secure :-1 s=READY pgs=1353 cs=0 l=1 rev1=1 crypto rx=0x7f5234009cf0 tx=0x7f5234011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d8db77aefdb29484 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:56:30.223 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:30.219+0000 7f5229ffb640 1 -- 172.21.15.39:0/2669670308 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5234004230 con 0x5622c484f1b0 2024-09-21T03:56:30.223 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:30.219+0000 7f5229ffb640 1 -- 172.21.15.39:0/2669670308 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f52340043d0 con 0x5622c484f1b0 2024-09-21T03:56:30.223 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:30.219+0000 7f5229ffb640 1 -- 172.21.15.39:0/2669670308 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5234019880 con 0x5622c484f1b0 2024-09-21T03:56:30.223 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:30.220+0000 7f525052a740 1 -- 172.21.15.39:0/2669670308 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5622c484f1b0 msgr2=0x5622c4c1f800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:56:30.223 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:30.220+0000 7f525052a740 1 --2- 172.21.15.39:0/2669670308 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5622c484f1b0 0x5622c4c1f800 secure :-1 s=READY pgs=1353 cs=0 l=1 rev1=1 crypto rx=0x7f5234009cf0 tx=0x7f5234011810 comp rx=0 tx=0).stop 2024-09-21T03:56:30.223 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:30.220+0000 7f5242ffd640 1 --2- 172.21.15.39:0/1668349820 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f522c0b6700 0x7f522c0b8c10 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f5238000c00 tx=0x7f5238005780 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:56:30.223 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:30.220+0000 7f525052a740 1 -- 172.21.15.39:0/2669670308 shutdown_connections 2024-09-21T03:56:30.223 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:30.220+0000 7f525052a740 1 --2- 172.21.15.39:0/2669670308 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5622c4857390 0x5622c4c1fd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:30.223 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:30.220+0000 7f525052a740 1 --2- 172.21.15.39:0/2669670308 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5622c484f1b0 0x5622c4c1f800 unknown :-1 s=CLOSED pgs=1353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:30.223 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:30.220+0000 7f525052a740 1 --2- 172.21.15.39:0/2669670308 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5622c4878070 0x5622c4c1f2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:30.224 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:30.220+0000 7f525052a740 1 -- 172.21.15.39:0/2669670308 >> 172.21.15.39:0/2669670308 conn(0x5622c486c460 msgr2=0x5622c4c1d1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:56:30.224 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:30.220+0000 7f525052a740 1 -- 172.21.15.39:0/2669670308 shutdown_connections 2024-09-21T03:56:30.224 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:30.220+0000 7f525052a740 1 -- 172.21.15.39:0/2669670308 wait complete. 2024-09-21T03:56:30.224 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:30.220+0000 7f525052a740 1 -- start start 2024-09-21T03:56:30.224 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:30.221+0000 7f525052a740 1 Processor -- start 2024-09-21T03:56:30.226 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:30.221+0000 7f525052a740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5622c484f1b0 0x5622c4c2a8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:56:30.227 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:30.221+0000 7f525052a740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5622c4857390 0x5622c4c2ae10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:56:30.227 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:30.221+0000 7f525052a740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5622c4878070 0x5622c4c2b350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:56:30.227 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:30.221+0000 7f525052a740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x5622c49e6700 con 0x5622c4878070 2024-09-21T03:56:30.227 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:30.221+0000 7f525052a740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x5622c4b20160 con 0x5622c4857390 2024-09-21T03:56:30.227 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:30.221+0000 7f525052a740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x5622c4c2daa0 con 0x5622c484f1b0 2024-09-21T03:56:30.227 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:30.221+0000 7f525052a740 10 client.0 _ll_register_callbacks cb 0x5622c4b399e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:56:30.227 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:30.221+0000 7f5243fff640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5622c4878070 0x5622c4c2b350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:56:30.227 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:30.221+0000 7f5242ffd640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5622c4857390 0x5622c4c2ae10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:56:30.227 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:30.221+0000 7f5243fff640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5622c4878070 0x5622c4c2b350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:48676/0 (socket says 172.21.15.39:48676) 2024-09-21T03:56:30.227 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:30.221+0000 7f5243fff640 1 -- 172.21.15.39:0/3644755771 learned_addr learned my addr 172.21.15.39:0/3644755771 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:56:30.227 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:30.221+0000 7f52437fe640 1 --2- 172.21.15.39:0/3644755771 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5622c484f1b0 0x5622c4c2a8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:56:30.227 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:30.222+0000 7f5242ffd640 1 -- 172.21.15.39:0/3644755771 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5622c484f1b0 msgr2=0x5622c4c2a8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:56:30.227 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:30.222+0000 7f5242ffd640 1 --2- 172.21.15.39:0/3644755771 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5622c484f1b0 0x5622c4c2a8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:30.228 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:30.222+0000 7f5242ffd640 1 -- 172.21.15.39:0/3644755771 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5622c4878070 msgr2=0x5622c4c2b350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:56:30.228 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:30.222+0000 7f5242ffd640 1 --2- 172.21.15.39:0/3644755771 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5622c4878070 0x5622c4c2b350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:30.228 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:30.222+0000 7f5242ffd640 1 -- 172.21.15.39:0/3644755771 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5622c4b4dc90 con 0x5622c4857390 2024-09-21T03:56:30.228 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:30.222+0000 7f5243fff640 1 --2- 172.21.15.39:0/3644755771 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5622c4878070 0x5622c4c2b350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:56:30.228 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:30.222+0000 7f5242ffd640 1 --2- 172.21.15.39:0/3644755771 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5622c4857390 0x5622c4c2ae10 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f5238006010 tx=0x7f5238031400 comp rx=0 tx=0).ready entity=mon.2 client_cookie=36d194fb17d23d15 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:56:30.228 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:30.222+0000 7f5222ffd640 10 client.0 ms_handle_connect on v2:172.21.15.150:3300/0 2024-09-21T03:56:30.228 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:30.222+0000 7f5222ffd640 1 -- 172.21.15.39:0/3644755771 <== mon.2 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5238051020 con 0x5622c4857390 2024-09-21T03:56:30.228 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:30.222+0000 7f5222ffd640 1 -- 172.21.15.39:0/3644755771 <== mon.2 v2:172.21.15.150:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8528+0+0 (secure 0 0 0) 0x7f52380035f0 con 0x5622c4857390 2024-09-21T03:56:30.228 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:30.222+0000 7f525052a740 1 -- 172.21.15.39:0/3644755771 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f52340043d0 con 0x5622c4857390 2024-09-21T03:56:30.228 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:30.222+0000 7f5222ffd640 1 -- 172.21.15.39:0/3644755771 <== mon.2 v2:172.21.15.150:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f5238035040 con 0x5622c4857390 2024-09-21T03:56:30.228 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:30.222+0000 7f525052a740 10 client.27440 fetch_fsmap learned FSMap version 29 2024-09-21T03:56:30.228 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:30.222+0000 7f525052a740 1 -- 172.21.15.39:0/3644755771 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5622c4c396a0 con 0x5622c4857390 2024-09-21T03:56:30.228 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:30.223+0000 7f5222ffd640 1 -- 172.21.15.39:0/3644755771 <== mon.2 v2:172.21.15.150:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f52340043d0 con 0x5622c4857390 2024-09-21T03:56:30.228 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:30.223+0000 7f5222ffd640 1 -- 172.21.15.39:0/3644755771 <== mon.2 v2:172.21.15.150:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f523803d780 con 0x5622c4857390 2024-09-21T03:56:30.228 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:30.223+0000 7f525052a740 10 client.27440 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T03:56:30.229 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:30.223+0000 7f525052a740 10 client.27440 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T03:56:30.229 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:30.223+0000 7f525052a740 20 client.27440 populate_metadata read hostname 'smithi039' 2024-09-21T03:56:30.229 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:30.223+0000 7f525052a740 1 --2- 172.21.15.39:0/3644755771 >> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] conn(0x5622c48369f0 0x5622c4c3cb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:56:30.229 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:30.223+0000 7f525052a740 4 client.27440 mds_command: new command op to 24695 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:56:30.229 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:30.223+0000 7f525052a740 1 -- 172.21.15.39:0/3644755771 --> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5622c4b97df0 con 0x5622c48369f0 2024-09-21T03:56:30.229 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:30.223+0000 7f52437fe640 1 --2- 172.21.15.39:0/3644755771 >> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] conn(0x5622c48369f0 0x5622c4c3cb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:56:30.229 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:30.224+0000 7f52437fe640 1 --2- 172.21.15.39:0/3644755771 >> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] conn(0x5622c48369f0 0x5622c4c3cb70 crc :-1 s=READY pgs=169 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fd186bba28336929 server_cookie=f2a9b79afe072eda in_seq=0 out_seq=0 2024-09-21T03:56:30.229 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:30.224+0000 7f5222ffd640 10 client.27440 ms_handle_connect on v2:172.21.15.150:6838/2949386907 2024-09-21T03:56:30.229 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:30.224+0000 7f5222ffd640 1 -- 172.21.15.39:0/3644755771 <== mds.0 v2:172.21.15.150:6838/2949386907 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5622c4b97df0 con 0x5622c48369f0 2024-09-21T03:56:30.229 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:30.224+0000 7f5222ffd640 10 client.27440 handle_command_reply: tid=0 2024-09-21T03:56:30.229 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:30.224+0000 7f525052a740 1 client.27440 shutdown 2024-09-21T03:56:30.229 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:30.225+0000 7f525052a740 1 -- 172.21.15.39:0/3644755771 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5622c4857390 msgr2=0x5622c4c2ae10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:56:30.229 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:30.225+0000 7f525052a740 1 --2- 172.21.15.39:0/3644755771 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5622c4857390 0x5622c4c2ae10 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f5238006010 tx=0x7f5238031400 comp rx=0 tx=0).stop 2024-09-21T03:56:30.229 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:30.225+0000 7f525052a740 1 -- 172.21.15.39:0/3644755771 shutdown_connections 2024-09-21T03:56:30.230 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:30.225+0000 7f525052a740 1 --2- 172.21.15.39:0/3644755771 >> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] conn(0x5622c48369f0 0x5622c4c3cb70 crc :-1 s=READY pgs=169 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:30.230 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:30.225+0000 7f525052a740 1 --2- 172.21.15.39:0/3644755771 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5622c4878070 0x5622c4c2b350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:30.230 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:30.225+0000 7f525052a740 1 --2- 172.21.15.39:0/3644755771 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5622c4857390 0x5622c4c2ae10 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:30.230 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:30.225+0000 7f5222ffd640 0 client.27440 ms_handle_reset on v2:172.21.15.150:6838/2949386907 2024-09-21T03:56:30.230 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:30.225+0000 7f525052a740 1 --2- 172.21.15.39:0/3644755771 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5622c484f1b0 0x5622c4c2a8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:30.230 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:30.225+0000 7f525052a740 1 -- 172.21.15.39:0/3644755771 >> 172.21.15.39:0/3644755771 conn(0x5622c486c460 msgr2=0x5622c4c224a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:56:30.230 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:30.225+0000 7f525052a740 1 -- 172.21.15.39:0/3644755771 shutdown_connections 2024-09-21T03:56:30.230 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:30.225+0000 7f525052a740 1 -- 172.21.15.39:0/3644755771 wait complete. 2024-09-21T03:56:30.230 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:30.225+0000 7f525052a740 20 client.27440 trim_cache size 0 max 16384 2024-09-21T03:56:30.240 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:30.238+0000 7f525052a740 1 Processor -- start 2024-09-21T03:56:30.240 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:30.238+0000 7f525052a740 1 -- start start 2024-09-21T03:56:30.240 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:30.238+0000 7f525052a740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5622c48369f0 0x5622c4c20a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:56:30.240 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:30.238+0000 7f525052a740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5622c484f1b0 0x5622c4c27ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:56:30.240 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:30.238+0000 7f525052a740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5622c4857390 0x5622c4c28000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:56:30.240 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:30.238+0000 7f525052a740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f52441af7e0 con 0x5622c48369f0 2024-09-21T03:56:30.240 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:30.238+0000 7f525052a740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f52441b1320 con 0x5622c484f1b0 2024-09-21T03:56:30.240 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:30.238+0000 7f525052a740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f52441afeb0 con 0x5622c4857390 2024-09-21T03:56:30.241 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:30.238+0000 7f52437fe640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5622c48369f0 0x5622c4c20a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:56:30.241 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:30.238+0000 7f52437fe640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5622c48369f0 0x5622c4c20a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:48680/0 (socket says 172.21.15.39:48680) 2024-09-21T03:56:30.241 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:30.238+0000 7f52437fe640 1 -- 172.21.15.39:0/2341159255 learned_addr learned my addr 172.21.15.39:0/2341159255 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:56:30.241 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:30.239+0000 7f5242ffd640 1 --2- 172.21.15.39:0/2341159255 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5622c484f1b0 0x5622c4c27ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:56:30.241 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:30.239+0000 7f52437fe640 1 -- 172.21.15.39:0/2341159255 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5622c4857390 msgr2=0x5622c4c28000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:56:30.242 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:30.239+0000 7f52437fe640 1 --2- 172.21.15.39:0/2341159255 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5622c4857390 0x5622c4c28000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:30.242 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:30.239+0000 7f52437fe640 1 -- 172.21.15.39:0/2341159255 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5622c484f1b0 msgr2=0x5622c4c27ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:56:30.242 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:30.239+0000 7f52437fe640 1 --2- 172.21.15.39:0/2341159255 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5622c484f1b0 0x5622c4c27ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:30.242 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:30.239+0000 7f52437fe640 1 -- 172.21.15.39:0/2341159255 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5622c4c2bb90 con 0x5622c48369f0 2024-09-21T03:56:30.242 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:30.239+0000 7f5242ffd640 1 --2- 172.21.15.39:0/2341159255 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5622c484f1b0 0x5622c4c27ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:56:30.242 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:30.239+0000 7f52437fe640 1 --2- 172.21.15.39:0/2341159255 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5622c48369f0 0x5622c4c20a00 secure :-1 s=READY pgs=1354 cs=0 l=1 rev1=1 crypto rx=0x7f523401b040 tx=0x7f5234011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=84c0934c6b0ccbae server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:56:30.242 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:30.239+0000 7f52297fa640 1 -- 172.21.15.39:0/2341159255 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5234031070 con 0x5622c48369f0 2024-09-21T03:56:30.242 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:30.239+0000 7f52297fa640 1 -- 172.21.15.39:0/2341159255 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f52340054f0 con 0x5622c48369f0 2024-09-21T03:56:30.242 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:30.239+0000 7f52297fa640 1 -- 172.21.15.39:0/2341159255 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5234003b70 con 0x5622c48369f0 2024-09-21T03:56:30.245 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:30.240+0000 7f525052a740 1 -- 172.21.15.39:0/2341159255 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5622c48369f0 msgr2=0x5622c4c20a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:56:30.245 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:30.240+0000 7f525052a740 1 --2- 172.21.15.39:0/2341159255 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5622c48369f0 0x5622c4c20a00 secure :-1 s=READY pgs=1354 cs=0 l=1 rev1=1 crypto rx=0x7f523401b040 tx=0x7f5234011810 comp rx=0 tx=0).stop 2024-09-21T03:56:30.245 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:30.240+0000 7f525052a740 1 -- 172.21.15.39:0/2341159255 shutdown_connections 2024-09-21T03:56:30.245 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:30.240+0000 7f525052a740 1 --2- 172.21.15.39:0/2341159255 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5622c4857390 0x5622c4c28000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:30.245 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:30.240+0000 7f525052a740 1 --2- 172.21.15.39:0/2341159255 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5622c484f1b0 0x5622c4c27ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:30.245 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:30.240+0000 7f525052a740 1 --2- 172.21.15.39:0/2341159255 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5622c48369f0 0x5622c4c20a00 unknown :-1 s=CLOSED pgs=1354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:30.245 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:30.240+0000 7f525052a740 1 -- 172.21.15.39:0/2341159255 >> 172.21.15.39:0/2341159255 conn(0x5622c486c460 msgr2=0x5622c4c202f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:56:30.245 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:30.240+0000 7f525052a740 1 -- 172.21.15.39:0/2341159255 shutdown_connections 2024-09-21T03:56:30.245 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:30.240+0000 7f525052a740 1 -- 172.21.15.39:0/2341159255 wait complete. 2024-09-21T03:56:30.245 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:30.240+0000 7f525052a740 1 -- start start 2024-09-21T03:56:30.245 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:30.241+0000 7f525052a740 1 Processor -- start 2024-09-21T03:56:30.245 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:30.241+0000 7f525052a740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5622c48369f0 0x5622c4c336c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:56:30.246 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:30.241+0000 7f525052a740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5622c484f1b0 0x5622c4c2eb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:56:30.246 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:30.241+0000 7f525052a740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5622c4857390 0x5622c4c2f2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:56:30.246 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:30.241+0000 7f525052a740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f52441b0c50 con 0x5622c48369f0 2024-09-21T03:56:30.246 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:30.241+0000 7f525052a740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f52441b0580 con 0x5622c4857390 2024-09-21T03:56:30.246 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:30.241+0000 7f525052a740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f52441ad7c0 con 0x5622c484f1b0 2024-09-21T03:56:30.246 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:30.241+0000 7f525052a740 10 client.0 _ll_register_callbacks cb 0x7f5238009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:56:30.246 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:30.241+0000 7f52437fe640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5622c48369f0 0x5622c4c336c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:56:30.246 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:30.241+0000 7f52437fe640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5622c48369f0 0x5622c4c336c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:48690/0 (socket says 172.21.15.39:48690) 2024-09-21T03:56:30.246 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:30.241+0000 7f52437fe640 1 -- 172.21.15.39:0/1969979072 learned_addr learned my addr 172.21.15.39:0/1969979072 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:56:30.246 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:30.241+0000 7f5243fff640 1 --2- 172.21.15.39:0/1969979072 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5622c4857390 0x5622c4c2f2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:56:30.246 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:30.241+0000 7f5242ffd640 1 --2- 172.21.15.39:0/1969979072 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5622c484f1b0 0x5622c4c2eb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:56:30.246 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:30.241+0000 7f52437fe640 1 -- 172.21.15.39:0/1969979072 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5622c484f1b0 msgr2=0x5622c4c2eb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:56:30.246 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:30.241+0000 7f52437fe640 1 --2- 172.21.15.39:0/1969979072 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5622c484f1b0 0x5622c4c2eb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:30.246 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:30.241+0000 7f52437fe640 1 -- 172.21.15.39:0/1969979072 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5622c4857390 msgr2=0x5622c4c2f2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:56:30.247 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:30.241+0000 7f52437fe640 1 --2- 172.21.15.39:0/1969979072 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5622c4857390 0x5622c4c2f2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:30.247 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:30.241+0000 7f52437fe640 1 -- 172.21.15.39:0/1969979072 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5622c4ca64b0 con 0x5622c48369f0 2024-09-21T03:56:30.247 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:30.242+0000 7f52437fe640 1 --2- 172.21.15.39:0/1969979072 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5622c48369f0 0x5622c4c336c0 secure :-1 s=READY pgs=1355 cs=0 l=1 rev1=1 crypto rx=0x7f5234005fd0 tx=0x7f5234005320 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f6d3053c54b1484e server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:56:30.247 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:30.242+0000 7f5222ffd640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-09-21T03:56:30.247 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:30.242+0000 7f5222ffd640 1 -- 172.21.15.39:0/1969979072 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5234031040 con 0x5622c48369f0 2024-09-21T03:56:30.247 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:30.242+0000 7f5222ffd640 1 -- 172.21.15.39:0/1969979072 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8528+0+0 (secure 0 0 0) 0x7f52340038b0 con 0x5622c48369f0 2024-09-21T03:56:30.247 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:30.242+0000 7f525052a740 1 -- 172.21.15.39:0/1969979072 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f52340054f0 con 0x5622c48369f0 2024-09-21T03:56:30.247 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:30.242+0000 7f5222ffd640 1 -- 172.21.15.39:0/1969979072 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f5234003ab0 con 0x5622c48369f0 2024-09-21T03:56:30.247 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:30.242+0000 7f525052a740 10 client.18378 fetch_fsmap learned FSMap version 29 2024-09-21T03:56:30.247 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:30.242+0000 7f525052a740 1 -- 172.21.15.39:0/1969979072 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5622c4ca7550 con 0x5622c48369f0 2024-09-21T03:56:30.247 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:30.242+0000 7f5222ffd640 1 -- 172.21.15.39:0/1969979072 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f52340054f0 con 0x5622c48369f0 2024-09-21T03:56:30.247 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:30.242+0000 7f5222ffd640 1 -- 172.21.15.39:0/1969979072 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f523403f030 con 0x5622c48369f0 2024-09-21T03:56:30.247 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:30.242+0000 7f525052a740 10 client.18378 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T03:56:30.247 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:30.242+0000 7f525052a740 10 client.18378 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T03:56:30.247 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:30.242+0000 7f525052a740 20 client.18378 populate_metadata read hostname 'smithi039' 2024-09-21T03:56:30.248 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:30.243+0000 7f525052a740 1 --2- 172.21.15.39:0/1969979072 >> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] conn(0x5622c4878070 0x5622c4ca9a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:56:30.248 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:30.243+0000 7f525052a740 4 client.18378 mds_command: new command op to 24695 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T03:56:30.248 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:30.243+0000 7f525052a740 1 -- 172.21.15.39:0/1969979072 --> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5622c4b938a0 con 0x5622c4878070 2024-09-21T03:56:30.248 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:30.243+0000 7f5242ffd640 1 --2- 172.21.15.39:0/1969979072 >> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] conn(0x5622c4878070 0x5622c4ca9a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:56:30.248 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:30.243+0000 7f5242ffd640 1 --2- 172.21.15.39:0/1969979072 >> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] conn(0x5622c4878070 0x5622c4ca9a90 crc :-1 s=READY pgs=170 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f15072fb6c1bb8dd server_cookie=64537d1739269ccd in_seq=0 out_seq=0 2024-09-21T03:56:30.248 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:30.243+0000 7f5222ffd640 10 client.18378 ms_handle_connect on v2:172.21.15.150:6838/2949386907 2024-09-21T03:56:30.249 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:30.243+0000 7f5222ffd640 1 -- 172.21.15.39:0/1969979072 <== mds.0 v2:172.21.15.150:6838/2949386907 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5622c4b938a0 con 0x5622c4878070 2024-09-21T03:56:30.249 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:30.243+0000 7f5222ffd640 10 client.18378 handle_command_reply: tid=0 2024-09-21T03:56:30.249 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:30.243+0000 7f525052a740 1 client.18378 shutdown 2024-09-21T03:56:30.249 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:30.244+0000 7f525052a740 1 -- 172.21.15.39:0/1969979072 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5622c48369f0 msgr2=0x5622c4c336c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:56:30.249 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:30.244+0000 7f525052a740 1 --2- 172.21.15.39:0/1969979072 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5622c48369f0 0x5622c4c336c0 secure :-1 s=READY pgs=1355 cs=0 l=1 rev1=1 crypto rx=0x7f5234005fd0 tx=0x7f5234005320 comp rx=0 tx=0).stop 2024-09-21T03:56:30.249 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:30.244+0000 7f525052a740 1 -- 172.21.15.39:0/1969979072 shutdown_connections 2024-09-21T03:56:30.249 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:30.244+0000 7f525052a740 1 --2- 172.21.15.39:0/1969979072 >> [v2:172.21.15.150:6838/2949386907,v1:172.21.15.150:6839/2949386907] conn(0x5622c4878070 0x5622c4ca9a90 crc :-1 s=READY pgs=170 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:30.249 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:30.244+0000 7f525052a740 1 --2- 172.21.15.39:0/1969979072 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5622c4857390 0x5622c4c2f2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:30.250 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:30.244+0000 7f525052a740 1 --2- 172.21.15.39:0/1969979072 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5622c484f1b0 0x5622c4c2eb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:30.250 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:30.244+0000 7f5222ffd640 0 client.18378 ms_handle_reset on v2:172.21.15.150:6838/2949386907 2024-09-21T03:56:30.250 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:30.244+0000 7f525052a740 1 --2- 172.21.15.39:0/1969979072 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5622c48369f0 0x5622c4c336c0 unknown :-1 s=CLOSED pgs=1355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:30.250 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:30.244+0000 7f525052a740 1 -- 172.21.15.39:0/1969979072 >> 172.21.15.39:0/1969979072 conn(0x5622c486c460 msgr2=0x5622c4c2bd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:56:30.250 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:30.244+0000 7f525052a740 1 -- 172.21.15.39:0/1969979072 shutdown_connections 2024-09-21T03:56:30.250 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:30.244+0000 7f525052a740 1 -- 172.21.15.39:0/1969979072 wait complete. 2024-09-21T03:56:30.250 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:30.244+0000 7f525052a740 20 client.18378 trim_cache size 0 max 16384 2024-09-21T03:56:30.250 INFO:teuthology.orchestra.run.smithi039.stdout:[] 2024-09-21T03:56:30.251 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:30.244+0000 7f5249fb6640 1 -- 172.21.15.39:0/1668349820 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f522c0b6700 msgr2=0x7f522c0b8c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:56:30.251 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:30.244+0000 7f5249fb6640 1 --2- 172.21.15.39:0/1668349820 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f522c0b6700 0x7f522c0b8c10 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f5238000c00 tx=0x7f5238005780 comp rx=0 tx=0).stop 2024-09-21T03:56:30.251 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:30.244+0000 7f5249fb6640 1 -- 172.21.15.39:0/1668349820 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5244108d10 msgr2=0x7f524419fdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:56:30.251 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:30.244+0000 7f5249fb6640 1 --2- 172.21.15.39:0/1668349820 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5244108d10 0x7f524419fdb0 secure :-1 s=READY pgs=1352 cs=0 l=1 rev1=1 crypto rx=0x7f523c00eaa0 tx=0x7f523c00ef60 comp rx=0 tx=0).stop 2024-09-21T03:56:30.251 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:30.245+0000 7f5249fb6640 1 -- 172.21.15.39:0/1668349820 shutdown_connections 2024-09-21T03:56:30.251 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:30.245+0000 7f5249fb6640 1 --2- 172.21.15.39:0/1668349820 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f522c0b6700 0x7f522c0b8c10 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:30.251 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:30.245+0000 7f5249fb6640 1 --2- 172.21.15.39:0/1668349820 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5244108d10 0x7f524419fdb0 unknown :-1 s=CLOSED pgs=1352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:30.251 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:30.245+0000 7f5249fb6640 1 --2- 172.21.15.39:0/1668349820 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f52441083c0 0x7f524419ab10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:30.251 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:30.245+0000 7f5249fb6640 1 --2- 172.21.15.39:0/1668349820 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5244102a80 0x7f524419a570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:30.251 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:30.245+0000 7f5249fb6640 1 -- 172.21.15.39:0/1668349820 >> 172.21.15.39:0/1668349820 conn(0x7f5244079740 msgr2=0x7f5244104480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:56:30.251 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:30.245+0000 7f5249fb6640 1 -- 172.21.15.39:0/1668349820 shutdown_connections 2024-09-21T03:56:30.252 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:30.245+0000 7f5249fb6640 1 -- 172.21.15.39:0/1668349820 wait complete. 2024-09-21T03:56:30.989 ERROR:tasks.fwd_scrub.fs.[cephfs]:exception: Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_ceph-c_1843221895adccfb3b692060251dd6a0183bc410/qa/tasks/fwd_scrub.py", line 35, in _run self.do_scrub() File "/home/teuthworker/src/git.ceph.com_ceph-c_1843221895adccfb3b692060251dd6a0183bc410/qa/tasks/fwd_scrub.py", line 49, in do_scrub self.sleep_unless_stopped(self.sleep_between_iterations) File "/home/teuthworker/src/git.ceph.com_ceph-c_1843221895adccfb3b692060251dd6a0183bc410/qa/tasks/thrasher.py", line 50, in sleep_unless_stopped raise self.Stopped() tasks.thrasher.ThrasherGreenlet.Stopped 2024-09-21T03:56:30.989 INFO:tasks.fwd_scrub:done joining 2024-09-21T03:56:30.990 DEBUG:teuthology.run_tasks:Unwinding manager check-counter 2024-09-21T03:56:30.999 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T03:56:31.119 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:31.117+0000 7f739cde6640 1 -- 172.21.15.39:0/1781929642 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7398108c00 msgr2=0x7f739810f430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:56:31.119 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:31.117+0000 7f739cde6640 1 --2- 172.21.15.39:0/1781929642 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7398108c00 0x7f739810f430 secure :-1 s=READY pgs=1356 cs=0 l=1 rev1=1 crypto rx=0x7f7390009ca0 tx=0x7f739002a8c0 comp rx=0 tx=0).stop 2024-09-21T03:56:31.120 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:31.117+0000 7f739cde6640 1 -- 172.21.15.39:0/1781929642 shutdown_connections 2024-09-21T03:56:31.120 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:31.117+0000 7f739cde6640 1 --2- 172.21.15.39:0/1781929642 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7398108c00 0x7f739810f430 unknown :-1 s=CLOSED pgs=1356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:31.120 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:31.117+0000 7f739cde6640 1 --2- 172.21.15.39:0/1781929642 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f73981082b0 0x7f73981086c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:31.120 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:31.117+0000 7f739cde6640 1 --2- 172.21.15.39:0/1781929642 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7398102980 0x7f7398102d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:31.120 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:31.117+0000 7f739cde6640 1 -- 172.21.15.39:0/1781929642 >> 172.21.15.39:0/1781929642 conn(0x7f73980fe380 msgr2=0x7f73981007d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:56:31.120 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:31.117+0000 7f739cde6640 1 -- 172.21.15.39:0/1781929642 shutdown_connections 2024-09-21T03:56:31.120 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:31.117+0000 7f739cde6640 1 -- 172.21.15.39:0/1781929642 wait complete. 2024-09-21T03:56:31.122 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:31.118+0000 7f739cde6640 1 Processor -- start 2024-09-21T03:56:31.122 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:31.118+0000 7f739cde6640 1 -- start start 2024-09-21T03:56:31.122 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:31.119+0000 7f739cde6640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7398102980 0x7f73981992a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:56:31.122 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:31.119+0000 7f739cde6640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f73981082b0 0x7f7398199840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:56:31.122 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:31.119+0000 7f739cde6640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7398108c00 0x7f739819ead0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:56:31.122 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:31.119+0000 7f739cde6640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f73981128c0 con 0x7f7398108c00 2024-09-21T03:56:31.122 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:31.119+0000 7f739cde6640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f7398112740 con 0x7f7398102980 2024-09-21T03:56:31.122 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:31.119+0000 7f739cde6640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f7398112a40 con 0x7f73981082b0 2024-09-21T03:56:31.123 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:31.119+0000 7f7396d9d640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7398108c00 0x7f739819ead0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:56:31.123 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:31.119+0000 7f739659c640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7398102980 0x7f73981992a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:56:31.123 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:31.119+0000 7f7395d9b640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f73981082b0 0x7f7398199840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:56:31.123 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:31.119+0000 7f739659c640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7398102980 0x7f73981992a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.39:56936/0 (socket says 172.21.15.39:56936) 2024-09-21T03:56:31.123 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:31.119+0000 7f739659c640 1 -- 172.21.15.39:0/3733740233 learned_addr learned my addr 172.21.15.39:0/3733740233 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:56:31.123 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:31.119+0000 7f7396d9d640 1 -- 172.21.15.39:0/3733740233 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f73981082b0 msgr2=0x7f7398199840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:56:31.123 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:31.119+0000 7f7396d9d640 1 --2- 172.21.15.39:0/3733740233 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f73981082b0 0x7f7398199840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:31.123 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:31.119+0000 7f7396d9d640 1 -- 172.21.15.39:0/3733740233 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7398102980 msgr2=0x7f73981992a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:56:31.123 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:31.119+0000 7f7396d9d640 1 --2- 172.21.15.39:0/3733740233 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7398102980 0x7f73981992a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:31.123 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:31.119+0000 7f7396d9d640 1 -- 172.21.15.39:0/3733740233 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f739819f1d0 con 0x7f7398108c00 2024-09-21T03:56:31.123 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:31.119+0000 7f7396d9d640 1 --2- 172.21.15.39:0/3733740233 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7398108c00 0x7f739819ead0 secure :-1 s=READY pgs=1357 cs=0 l=1 rev1=1 crypto rx=0x7f739002ade0 tx=0x7f7390031660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:56:31.123 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:31.120+0000 7f73877fe640 1 -- 172.21.15.39:0/3733740233 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7390003b70 con 0x7f7398108c00 2024-09-21T03:56:31.123 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:31.120+0000 7f73877fe640 1 -- 172.21.15.39:0/3733740233 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7390031910 con 0x7f7398108c00 2024-09-21T03:56:31.124 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:31.120+0000 7f73877fe640 1 -- 172.21.15.39:0/3733740233 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f739002f950 con 0x7f7398108c00 2024-09-21T03:56:31.124 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:31.120+0000 7f739cde6640 1 -- 172.21.15.39:0/3733740233 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f739819f460 con 0x7f7398108c00 2024-09-21T03:56:31.124 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:31.120+0000 7f739cde6640 1 -- 172.21.15.39:0/3733740233 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f73981a5c50 con 0x7f7398108c00 2024-09-21T03:56:31.124 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:31.121+0000 7f739cde6640 1 -- 172.21.15.39:0/3733740233 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7398103bf0 con 0x7f7398108c00 2024-09-21T03:56:31.125 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:31.124+0000 7f73877fe640 1 -- 172.21.15.39:0/3733740233 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 27) ==== 154643+0+0 (secure 0 0 0) 0x7f739002faf0 con 0x7f7398108c00 2024-09-21T03:56:31.126 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:31.124+0000 7f73877fe640 1 --2- 172.21.15.39:0/3733740233 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f73800b6700 0x7f73800b8c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:56:31.126 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:31.125+0000 7f73877fe640 1 -- 172.21.15.39:0/3733740233 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8528+0+0 (secure 0 0 0) 0x7f73901088a0 con 0x7f7398108c00 2024-09-21T03:56:31.126 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:31.125+0000 7f739659c640 1 --2- 172.21.15.39:0/3733740233 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f73800b6700 0x7f73800b8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:56:31.127 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:31.125+0000 7f739659c640 1 --2- 172.21.15.39:0/3733740233 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f73800b6700 0x7f73800b8c10 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f7388009c90 tx=0x7f7388005f90 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:56:31.128 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:31.127+0000 7f73877fe640 1 -- 172.21.15.39:0/3733740233 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f73900cedf0 con 0x7f7398108c00 2024-09-21T03:56:31.322 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:31.321+0000 7f739cde6640 1 -- 172.21.15.39:0/3733740233 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f739819a3c0 con 0x7f7398108c00 2024-09-21T03:56:31.327 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:31.325+0000 7f73877fe640 1 -- 172.21.15.39:0/3733740233 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v87) ==== 74+0+23186 (secure 0 0 0) 0x7f7390003d10 con 0x7f7398108c00 2024-09-21T03:56:31.327 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T03:56:31.327 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":87,"fsid":"51116d5c-77ca-11ef-baf5-efdc52797490","created":"2024-09-21T03:35:33.631350+0000","modified":"2024-09-21T03:56:19.874370+0000","last_up_change":"2024-09-21T03:43:24.242275+0000","last_in_change":"2024-09-21T03:43:00.886552+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T03:39:29.859542+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-21T03:44:10.886435+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"77","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-21T03:44:13.041783+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":15,"snap_epoch":86,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1ea7148f-2ace-4c10-86fe-b8019de80c1f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6802","nonce":870188791},{"type":"v1","addr":"172.21.15.39:6803","nonce":870188791}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6804","nonce":870188791},{"type":"v1","addr":"172.21.15.39:6805","nonce":870188791}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6808","nonce":870188791},{"type":"v1","addr":"172.21.15.39:6809","nonce":870188791}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6806","nonce":870188791},{"type":"v1","addr":"172.21.15.39:6807","nonce":870188791}]},"public_addr":"172.21.15.39:6803/870188791","cluster_addr":"172.21.15.39:6805/870188791","heartbeat_back_addr":"172.21.15.39:6809/870188791","heartbeat_front_addr":"172.21.15.39:6807/870188791","state":["exists","up"]},{"osd":1,"uuid":"7cb5db90-28e8-4a30-bedb-c039687d48b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6800","nonce":172427832},{"type":"v1","addr":"172.21.15.150:6801","nonce":172427832}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6802","nonce":172427832},{"type":"v1","addr":"172.21.15.150:6803","nonce":172427832}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6806","nonce":172427832},{"type":"v1","addr":"172.21.15.150:6807","nonce":172427832}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6804","nonce":172427832},{"type":"v1","addr":"172.21.15.150:6805","nonce":172427832}]},"public_addr":"172.21.15.150:6801/172427832","cluster_addr":"172.21.15.150:6803/172427832","heartbeat_back_addr":"172.21.15.150:6807/172427832","heartbeat_front_addr":"172.21.15.150:6805/172427832","state":["exists","up"]},{"osd":2,"uuid":"b25c6ab3-fc95-4751-bd63-6e3fc23e8bd7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":19,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6800","nonce":2727876133},{"type":"v1","addr":"172.21.15.161:6801","nonce":2727876133}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6802","nonce":2727876133},{"type":"v1","addr":"172.21.15.161:6803","nonce":2727876133}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6806","nonce":2727876133},{"type":"v1","addr":"172.21.15.161:6807","nonce":2727876133}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6804","nonce":2727876133},{"type":"v1","addr":"172.21.15.161:6805","nonce":2727876133}]},"public_addr":"172.21.15.161:6801/2727876133","cluster_addr":"172.21.15.161:6803/2727876133","heartbeat_back_addr":"172.21.15.161:6807/2727876133","heartbeat_front_addr":"172.21.15.161:6805/2727876133","state":["exists","up"]},{"osd":3,"uuid":"14417ae1-4536-45ba-9a5c-c3b447d1a72d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6810","nonce":295394468},{"type":"v1","addr":"172.21.15.39:6811","nonce":295394468}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6812","nonce":295394468},{"type":"v1","addr":"172.21.15.39:6813","nonce":295394468}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6816","nonce":295394468},{"type":"v1","addr":"172.21.15.39:6817","nonce":295394468}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6814","nonce":295394468},{"type":"v1","addr":"172.21.15.39:6815","nonce":295394468}]},"public_addr":"172.21.15.39:6811/295394468","cluster_addr":"172.21.15.39:6813/295394468","heartbeat_back_addr":"172.21.15.39:6817/295394468","heartbeat_front_addr":"172.21.15.39:6815/295394468","state":["exists","up"]},{"osd":4,"uuid":"33c732ef-20c2-43c4-abe6-045b687fc521","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6808","nonce":3097721553},{"type":"v1","addr":"172.21.15.150:6809","nonce":3097721553}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6810","nonce":3097721553},{"type":"v1","addr":"172.21.15.150:6811","nonce":3097721553}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6814","nonce":3097721553},{"type":"v1","addr":"172.21.15.150:6815","nonce":3097721553}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6812","nonce":3097721553},{"type":"v1","addr":"172.21.15.150:6813","nonce":3097721553}]},"public_addr":"172.21.15.150:6809/3097721553","cluster_addr":"172.21.15.150:6811/3097721553","heartbeat_back_addr":"172.21.15.150:6815/3097721553","heartbeat_front_addr":"172.21.15.150:6813/3097721553","state":["exists","up"]},{"osd":5,"uuid":"74673722-abd6-400a-bb50-ec3dfba0c5e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6808","nonce":742751023},{"type":"v1","addr":"172.21.15.161:6809","nonce":742751023}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6810","nonce":742751023},{"type":"v1","addr":"172.21.15.161:6811","nonce":742751023}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6814","nonce":742751023},{"type":"v1","addr":"172.21.15.161:6815","nonce":742751023}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6812","nonce":742751023},{"type":"v1","addr":"172.21.15.161:6813","nonce":742751023}]},"public_addr":"172.21.15.161:6809/742751023","cluster_addr":"172.21.15.161:6811/742751023","heartbeat_back_addr":"172.21.15.161:6815/742751023","heartbeat_front_addr":"172.21.15.161:6813/742751023","state":["exists","up"]},{"osd":6,"uuid":"06548677-74c4-4efa-8054-df09b957b649","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6818","nonce":1072700290},{"type":"v1","addr":"172.21.15.39:6819","nonce":1072700290}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6820","nonce":1072700290},{"type":"v1","addr":"172.21.15.39:6821","nonce":1072700290}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6824","nonce":1072700290},{"type":"v1","addr":"172.21.15.39:6825","nonce":1072700290}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6822","nonce":1072700290},{"type":"v1","addr":"172.21.15.39:6823","nonce":1072700290}]},"public_addr":"172.21.15.39:6819/1072700290","cluster_addr":"172.21.15.39:6821/1072700290","heartbeat_back_addr":"172.21.15.39:6825/1072700290","heartbeat_front_addr":"172.21.15.39:6823/1072700290","state":["exists","up"]},{"osd":7,"uuid":"838735ca-c693-4f58-9426-b8ee857dbe3a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6816","nonce":2162619103},{"type":"v1","addr":"172.21.15.150:6817","nonce":2162619103}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6818","nonce":2162619103},{"type":"v1","addr":"172.21.15.150:6819","nonce":2162619103}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6822","nonce":2162619103},{"type":"v1","addr":"172.21.15.150:6823","nonce":2162619103}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6820","nonce":2162619103},{"type":"v1","addr":"172.21.15.150:6821","nonce":2162619103}]},"public_addr":"172.21.15.150:6817/2162619103","cluster_addr":"172.21.15.150:6819/2162619103","heartbeat_back_addr":"172.21.15.150:6823/2162619103","heartbeat_front_addr":"172.21.15.150:6821/2162619103","state":["exists","up"]},{"osd":8,"uuid":"4973b357-b4c2-4c00-b7f7-dd25012a0f72","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6816","nonce":52966633},{"type":"v1","addr":"172.21.15.161:6817","nonce":52966633}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6818","nonce":52966633},{"type":"v1","addr":"172.21.15.161:6819","nonce":52966633}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6822","nonce":52966633},{"type":"v1","addr":"172.21.15.161:6823","nonce":52966633}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6820","nonce":52966633},{"type":"v1","addr":"172.21.15.161:6821","nonce":52966633}]},"public_addr":"172.21.15.161:6817/52966633","cluster_addr":"172.21.15.161:6819/52966633","heartbeat_back_addr":"172.21.15.161:6823/52966633","heartbeat_front_addr":"172.21.15.161:6821/52966633","state":["exists","up"]},{"osd":9,"uuid":"6028613c-ebfd-42ea-9949-3f0d5f6cf025","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6826","nonce":2001303253},{"type":"v1","addr":"172.21.15.39:6827","nonce":2001303253}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6828","nonce":2001303253},{"type":"v1","addr":"172.21.15.39:6829","nonce":2001303253}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6832","nonce":2001303253},{"type":"v1","addr":"172.21.15.39:6833","nonce":2001303253}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6830","nonce":2001303253},{"type":"v1","addr":"172.21.15.39:6831","nonce":2001303253}]},"public_addr":"172.21.15.39:6827/2001303253","cluster_addr":"172.21.15.39:6829/2001303253","heartbeat_back_addr":"172.21.15.39:6833/2001303253","heartbeat_front_addr":"172.21.15.39:6831/2001303253","state":["exists","up"]},{"osd":10,"uuid":"1c71c765-2a6c-4ffe-9fe1-c8adae3d1c32","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6824","nonce":2484627938},{"type":"v1","addr":"172.21.15.150:6825","nonce":2484627938}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6826","nonce":2484627938},{"type":"v1","addr":"172.21.15.150:6827","nonce":2484627938}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6830","nonce":2484627938},{"type":"v1","addr":"172.21.15.150:6831","nonce":2484627938}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6828","nonce":2484627938},{"type":"v1","addr":"172.21.15.150:6829","nonce":2484627938}]},"public_addr":"172.21.15.150:6825/2484627938","cluster_addr":"172.21.15.150:6827/2484627938","heartbeat_back_addr":"172.21.15.150:6831/2484627938","heartbeat_front_addr":"172.21.15.150:6829/2484627938","state":["exists","up"]},{"osd":11,"uuid":"f382f522-62db-46da-997e-2b28f84d1612","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6824","nonce":2178709707},{"type":"v1","addr":"172.21.15.161:6825","nonce":2178709707}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6826","nonce":2178709707},{"type":"v1","addr":"172.21.15.161:6827","nonce":2178709707}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6830","nonce":2178709707},{"type":"v1","addr":"172.21.15.161:6831","nonce":2178709707}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6828","nonce":2178709707},{"type":"v1","addr":"172.21.15.161:6829","nonce":2178709707}]},"public_addr":"172.21.15.161:6825/2178709707","cluster_addr":"172.21.15.161:6827/2178709707","heartbeat_back_addr":"172.21.15.161:6831/2178709707","heartbeat_front_addr":"172.21.15.161:6829/2178709707","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:38:35.540904+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:39:01.455053+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:39:27.052867+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:39:55.173529+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:40:21.298396+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:40:47.131235+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:41:14.475209+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:41:39.920036+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:42:05.587065+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:42:32.624726+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:42:57.479185+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:43:22.203832+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.5","mappings":[{"from":8,"to":11}]},{"pgid":"2.8","mappings":[{"from":2,"to":11}]},{"pgid":"2.c","mappings":[{"from":2,"to":1}]},{"pgid":"2.1f","mappings":[{"from":8,"to":4}]},{"pgid":"3.2","mappings":[{"from":3,"to":9}]},{"pgid":"3.14","mappings":[{"from":4,"to":6}]},{"pgid":"3.1a","mappings":[{"from":5,"to":7}]},{"pgid":"3.1d","mappings":[{"from":4,"to":8}]},{"pgid":"3.2c","mappings":[{"from":4,"to":9}]},{"pgid":"3.38","mappings":[{"from":3,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.39:6801/2438110611":"2024-09-22T03:37:25.391457+0000","172.21.15.39:6800/2438110611":"2024-09-22T03:37:25.391457+0000","172.21.15.39:6800/173264695":"2024-09-22T03:35:57.104467+0000","172.21.15.39:6801/173264695":"2024-09-22T03:35:57.104467+0000","172.21.15.39:0/2637182284":"2024-09-22T03:35:57.104467+0000","172.21.15.39:0/1517304561":"2024-09-22T03:36:24.112592+0000","172.21.15.39:6801/1435622199":"2024-09-22T03:36:24.112592+0000","172.21.15.39:6800/1435622199":"2024-09-22T03:36:24.112592+0000","172.21.15.39:0/346832469":"2024-09-22T03:35:57.104467+0000","172.21.15.39:0/3160777371":"2024-09-22T03:36:24.112592+0000","172.21.15.39:0/1730342855":"2024-09-22T03:35:57.104467+0000","172.21.15.39:0/1955726766":"2024-09-22T03:36:24.112592+0000","172.21.15.39:0/3294132796":"2024-09-22T03:37:25.391457+0000","172.21.15.39:0/1295810162":"2024-09-22T03:37:25.391457+0000","172.21.15.39:0/3729230474":"2024-09-22T03:37:25.391457+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":3,"snaps":[{"begin":5,"length":1},{"begin":15,"length":1}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T03:56:31.330 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:31.328+0000 7f739cde6640 1 -- 172.21.15.39:0/3733740233 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f73800b6700 msgr2=0x7f73800b8c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:56:31.330 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:31.328+0000 7f739cde6640 1 --2- 172.21.15.39:0/3733740233 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f73800b6700 0x7f73800b8c10 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f7388009c90 tx=0x7f7388005f90 comp rx=0 tx=0).stop 2024-09-21T03:56:31.330 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:31.328+0000 7f739cde6640 1 -- 172.21.15.39:0/3733740233 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7398108c00 msgr2=0x7f739819ead0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:56:31.330 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:31.328+0000 7f739cde6640 1 --2- 172.21.15.39:0/3733740233 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7398108c00 0x7f739819ead0 secure :-1 s=READY pgs=1357 cs=0 l=1 rev1=1 crypto rx=0x7f739002ade0 tx=0x7f7390031660 comp rx=0 tx=0).stop 2024-09-21T03:56:31.330 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:31.328+0000 7f739cde6640 1 -- 172.21.15.39:0/3733740233 shutdown_connections 2024-09-21T03:56:31.330 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:31.328+0000 7f739cde6640 1 --2- 172.21.15.39:0/3733740233 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f73800b6700 0x7f73800b8c10 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:31.330 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:31.328+0000 7f739cde6640 1 --2- 172.21.15.39:0/3733740233 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7398108c00 0x7f739819ead0 unknown :-1 s=CLOSED pgs=1357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:31.330 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:31.328+0000 7f739cde6640 1 --2- 172.21.15.39:0/3733740233 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f73981082b0 0x7f7398199840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:31.330 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:31.328+0000 7f739cde6640 1 --2- 172.21.15.39:0/3733740233 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7398102980 0x7f73981992a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:31.330 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:31.328+0000 7f739cde6640 1 -- 172.21.15.39:0/3733740233 >> 172.21.15.39:0/3733740233 conn(0x7f73980fe380 msgr2=0x7f7398107890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:56:31.330 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:31.328+0000 7f739cde6640 1 -- 172.21.15.39:0/3733740233 shutdown_connections 2024-09-21T03:56:31.330 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:31.328+0000 7f739cde6640 1 -- 172.21.15.39:0/3733740233 wait complete. 2024-09-21T03:56:31.333 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:56:31 smithi039 ceph-mon[31211]: pgmap v647: 97 pgs: 97 active+clean; 10 MiB data, 525 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 0 B/s wr, 12 op/s 2024-09-21T03:56:31.333 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:56:31 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:56:31.342 INFO:tasks.check_counter.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-21T03:39:29.859542+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-09-21T03:44:10.886435+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '77', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-09-21T03:44:13.041783+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 15, 'snap_epoch': 86, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-09-21T03:56:31.342 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-21T03:56:31.461 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:31.459+0000 7f34a7f20640 1 -- 172.21.15.39:0/2338116119 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f34a0105210 msgr2=0x7f34a010d850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:56:31.462 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:31.459+0000 7f34a7f20640 1 --2- 172.21.15.39:0/2338116119 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f34a0105210 0x7f34a010d850 secure :-1 s=READY pgs=1358 cs=0 l=1 rev1=1 crypto rx=0x7f349c009900 tx=0x7f349c02a410 comp rx=0 tx=0).stop 2024-09-21T03:56:31.462 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:31.459+0000 7f34a7f20640 1 -- 172.21.15.39:0/2338116119 shutdown_connections 2024-09-21T03:56:31.462 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:31.459+0000 7f34a7f20640 1 --2- 172.21.15.39:0/2338116119 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f34a010dd90 0x7f34a01102a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:31.462 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:31.459+0000 7f34a7f20640 1 --2- 172.21.15.39:0/2338116119 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f34a0105210 0x7f34a010d850 unknown :-1 s=CLOSED pgs=1358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:31.462 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:31.459+0000 7f34a7f20640 1 --2- 172.21.15.39:0/2338116119 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f34a01047e0 0x7f34a0104bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:31.462 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:31.459+0000 7f34a7f20640 1 -- 172.21.15.39:0/2338116119 >> 172.21.15.39:0/2338116119 conn(0x7f34a00fe3f0 msgr2=0x7f34a0100840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:56:31.462 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:31.459+0000 7f34a7f20640 1 -- 172.21.15.39:0/2338116119 shutdown_connections 2024-09-21T03:56:31.462 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:31.459+0000 7f34a7f20640 1 -- 172.21.15.39:0/2338116119 wait complete. 2024-09-21T03:56:31.462 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:31.460+0000 7f34a7f20640 1 Processor -- start 2024-09-21T03:56:31.462 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:31.460+0000 7f34a7f20640 1 -- start start 2024-09-21T03:56:31.464 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:31.460+0000 7f34a7f20640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f34a01047e0 0x7f34a019a4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:56:31.464 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:31.460+0000 7f34a7f20640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f34a0105210 0x7f34a019aa70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:56:31.464 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:31.460+0000 7f34a7f20640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f34a010dd90 0x7f34a019fd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:56:31.464 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:31.460+0000 7f34a7f20640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f34a0113a10 con 0x7f34a0105210 2024-09-21T03:56:31.464 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:31.460+0000 7f34a7f20640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f34a0113890 con 0x7f34a010dd90 2024-09-21T03:56:31.464 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:31.460+0000 7f34a7f20640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f34a0113b90 con 0x7f34a01047e0 2024-09-21T03:56:31.465 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:31.460+0000 7f34a54bb640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f34a0105210 0x7f34a019aa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:56:31.465 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:31.460+0000 7f34a54bb640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f34a0105210 0x7f34a019aa70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:48738/0 (socket says 172.21.15.39:48738) 2024-09-21T03:56:31.465 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:31.460+0000 7f34a54bb640 1 -- 172.21.15.39:0/2449893003 learned_addr learned my addr 172.21.15.39:0/2449893003 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:56:31.465 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:31.460+0000 7f34a64bd640 1 --2- 172.21.15.39:0/2449893003 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f34a010dd90 0x7f34a019fd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:56:31.465 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:31.461+0000 7f34a5cbc640 1 --2- 172.21.15.39:0/2449893003 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f34a01047e0 0x7f34a019a4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:56:31.465 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:31.461+0000 7f34a54bb640 1 -- 172.21.15.39:0/2449893003 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f34a01047e0 msgr2=0x7f34a019a4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:56:31.465 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:31.461+0000 7f34a54bb640 1 --2- 172.21.15.39:0/2449893003 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f34a01047e0 0x7f34a019a4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:31.465 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:31.461+0000 7f34a54bb640 1 -- 172.21.15.39:0/2449893003 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f34a010dd90 msgr2=0x7f34a019fd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:56:31.465 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:31.461+0000 7f34a54bb640 1 --2- 172.21.15.39:0/2449893003 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f34a010dd90 0x7f34a019fd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:31.465 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:31.461+0000 7f34a54bb640 1 -- 172.21.15.39:0/2449893003 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f34a01a0400 con 0x7f34a0105210 2024-09-21T03:56:31.465 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:31.461+0000 7f34a64bd640 1 --2- 172.21.15.39:0/2449893003 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f34a010dd90 0x7f34a019fd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:56:31.465 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:31.461+0000 7f34a54bb640 1 --2- 172.21.15.39:0/2449893003 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f34a0105210 0x7f34a019aa70 secure :-1 s=READY pgs=1359 cs=0 l=1 rev1=1 crypto rx=0x7f349c02a930 tx=0x7f349c031a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:56:31.465 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:31.461+0000 7f3492ffd640 1 -- 172.21.15.39:0/2449893003 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f349c03f030 con 0x7f34a0105210 2024-09-21T03:56:31.465 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:31.461+0000 7f34a7f20640 1 -- 172.21.15.39:0/2449893003 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f34a01a0690 con 0x7f34a0105210 2024-09-21T03:56:31.466 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:31.461+0000 7f3492ffd640 1 -- 172.21.15.39:0/2449893003 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f349c004490 con 0x7f34a0105210 2024-09-21T03:56:31.466 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:31.461+0000 7f3492ffd640 1 -- 172.21.15.39:0/2449893003 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f349c035070 con 0x7f34a0105210 2024-09-21T03:56:31.466 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:31.461+0000 7f34a7f20640 1 -- 172.21.15.39:0/2449893003 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f34a01a6e80 con 0x7f34a0105210 2024-09-21T03:56:31.466 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:31.462+0000 7f34a7f20640 1 -- 172.21.15.39:0/2449893003 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f34a0104bf0 con 0x7f34a0105210 2024-09-21T03:56:31.467 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:31.465+0000 7f3492ffd640 1 -- 172.21.15.39:0/2449893003 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 27) ==== 154643+0+0 (secure 0 0 0) 0x7f349c042030 con 0x7f34a0105210 2024-09-21T03:56:31.467 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:31.466+0000 7f3492ffd640 1 --2- 172.21.15.39:0/2449893003 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f348c0b6680 0x7f348c0b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:56:31.468 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:31.466+0000 7f3492ffd640 1 -- 172.21.15.39:0/2449893003 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8528+0+0 (secure 0 0 0) 0x7f349c107dc0 con 0x7f34a0105210 2024-09-21T03:56:31.468 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:31.466+0000 7f34a5cbc640 1 --2- 172.21.15.39:0/2449893003 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f348c0b6680 0x7f348c0b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:56:31.468 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:56:31 smithi161 ceph-mon[34533]: pgmap v647: 97 pgs: 97 active+clean; 10 MiB data, 525 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 0 B/s wr, 12 op/s 2024-09-21T03:56:31.468 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:56:31 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:56:31.468 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:31.467+0000 7f34a5cbc640 1 --2- 172.21.15.39:0/2449893003 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f348c0b6680 0x7f348c0b8b90 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f3494009bd0 tx=0x7f3494005f90 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:56:31.471 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:31.469+0000 7f3492ffd640 1 -- 172.21.15.39:0/2449893003 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f349c0d26f0 con 0x7f34a0105210 2024-09-21T03:56:31.475 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:56:31 smithi150 ceph-mon[34289]: pgmap v647: 97 pgs: 97 active+clean; 10 MiB data, 525 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 0 B/s wr, 12 op/s 2024-09-21T03:56:31.475 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:56:31 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:56:31.670 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:31.669+0000 7f34a7f20640 1 -- 172.21.15.39:0/2449893003 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f34a01a70e0 con 0x7f34a0105210 2024-09-21T03:56:31.671 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:31.669+0000 7f3492ffd640 1 -- 172.21.15.39:0/2449893003 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v87) ==== 93+0+10 (secure 0 0 0) 0x7f349c0d7630 con 0x7f34a0105210 2024-09-21T03:56:31.671 INFO:teuthology.orchestra.run.smithi039.stdout:pg_num: 1 2024-09-21T03:56:31.673 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:31.671+0000 7f34a7f20640 1 -- 172.21.15.39:0/2449893003 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f348c0b6680 msgr2=0x7f348c0b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:56:31.673 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:31.671+0000 7f34a7f20640 1 --2- 172.21.15.39:0/2449893003 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f348c0b6680 0x7f348c0b8b90 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f3494009bd0 tx=0x7f3494005f90 comp rx=0 tx=0).stop 2024-09-21T03:56:31.673 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:31.671+0000 7f34a7f20640 1 -- 172.21.15.39:0/2449893003 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f34a0105210 msgr2=0x7f34a019aa70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:56:31.674 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:31.671+0000 7f34a7f20640 1 --2- 172.21.15.39:0/2449893003 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f34a0105210 0x7f34a019aa70 secure :-1 s=READY pgs=1359 cs=0 l=1 rev1=1 crypto rx=0x7f349c02a930 tx=0x7f349c031a20 comp rx=0 tx=0).stop 2024-09-21T03:56:31.674 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:31.671+0000 7f34a7f20640 1 -- 172.21.15.39:0/2449893003 shutdown_connections 2024-09-21T03:56:31.674 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:31.671+0000 7f34a7f20640 1 --2- 172.21.15.39:0/2449893003 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f348c0b6680 0x7f348c0b8b90 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:31.674 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:31.671+0000 7f34a7f20640 1 --2- 172.21.15.39:0/2449893003 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f34a010dd90 0x7f34a019fd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:31.674 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:31.671+0000 7f34a7f20640 1 --2- 172.21.15.39:0/2449893003 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f34a0105210 0x7f34a019aa70 unknown :-1 s=CLOSED pgs=1359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:31.674 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:31.671+0000 7f34a7f20640 1 --2- 172.21.15.39:0/2449893003 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f34a01047e0 0x7f34a019a4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:31.674 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:31.671+0000 7f34a7f20640 1 -- 172.21.15.39:0/2449893003 >> 172.21.15.39:0/2449893003 conn(0x7f34a00fe3f0 msgr2=0x7f34a0107910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:56:31.674 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:31.672+0000 7f34a7f20640 1 -- 172.21.15.39:0/2449893003 shutdown_connections 2024-09-21T03:56:31.674 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:31.672+0000 7f34a7f20640 1 -- 172.21.15.39:0/2449893003 wait complete. 2024-09-21T03:56:31.691 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-09-21T03:56:31.811 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:31.808+0000 7fa45a694640 1 -- 172.21.15.39:0/3384509482 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa4541082d0 msgr2=0x7fa4541086e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:56:31.811 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:31.808+0000 7fa45a694640 1 --2- 172.21.15.39:0/3384509482 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa4541082d0 0x7fa4541086e0 secure :-1 s=READY pgs=1360 cs=0 l=1 rev1=1 crypto rx=0x7fa448009900 tx=0x7fa44802a410 comp rx=0 tx=0).stop 2024-09-21T03:56:31.811 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:31.808+0000 7fa45a694640 1 -- 172.21.15.39:0/3384509482 shutdown_connections 2024-09-21T03:56:31.811 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:31.808+0000 7fa45a694640 1 --2- 172.21.15.39:0/3384509482 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa454108c20 0x7fa45410f450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:31.811 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:31.808+0000 7fa45a694640 1 --2- 172.21.15.39:0/3384509482 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa4541082d0 0x7fa4541086e0 unknown :-1 s=CLOSED pgs=1360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:31.811 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:31.808+0000 7fa45a694640 1 --2- 172.21.15.39:0/3384509482 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa4541029a0 0x7fa454102db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:31.811 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:31.808+0000 7fa45a694640 1 -- 172.21.15.39:0/3384509482 >> 172.21.15.39:0/3384509482 conn(0x7fa4540fe3c0 msgr2=0x7fa454100810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:56:31.811 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:31.808+0000 7fa45a694640 1 -- 172.21.15.39:0/3384509482 shutdown_connections 2024-09-21T03:56:31.811 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:31.808+0000 7fa45a694640 1 -- 172.21.15.39:0/3384509482 wait complete. 2024-09-21T03:56:31.811 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:31.809+0000 7fa45a694640 1 Processor -- start 2024-09-21T03:56:31.812 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:31.809+0000 7fa45a694640 1 -- start start 2024-09-21T03:56:31.813 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:31.809+0000 7fa45a694640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa4541029a0 0x7fa4540fe8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:56:31.813 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:31.809+0000 7fa45a694640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa4541082d0 0x7fa4540fee30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:56:31.813 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:31.809+0000 7fa45a694640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa454108c20 0x7fa4540ff370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:56:31.814 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:31.809+0000 7fa45a694640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fa4541128e0 con 0x7fa4541029a0 2024-09-21T03:56:31.814 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:31.809+0000 7fa45a694640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fa454112760 con 0x7fa454108c20 2024-09-21T03:56:31.814 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:31.809+0000 7fa45a694640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fa454112a60 con 0x7fa4541082d0 2024-09-21T03:56:31.814 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:31.810+0000 7fa453fff640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa4541029a0 0x7fa4540fe8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:56:31.814 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:31.810+0000 7fa453fff640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa4541029a0 0x7fa4540fe8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:48754/0 (socket says 172.21.15.39:48754) 2024-09-21T03:56:31.814 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:31.810+0000 7fa453fff640 1 -- 172.21.15.39:0/3428100577 learned_addr learned my addr 172.21.15.39:0/3428100577 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:56:31.814 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:31.810+0000 7fa4537fe640 1 --2- 172.21.15.39:0/3428100577 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa4541082d0 0x7fa4540fee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:56:31.814 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:31.810+0000 7fa458c31640 1 --2- 172.21.15.39:0/3428100577 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa454108c20 0x7fa4540ff370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:56:31.814 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:31.810+0000 7fa453fff640 1 -- 172.21.15.39:0/3428100577 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa4541082d0 msgr2=0x7fa4540fee30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:56:31.814 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:31.810+0000 7fa453fff640 1 --2- 172.21.15.39:0/3428100577 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa4541082d0 0x7fa4540fee30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:31.814 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:31.810+0000 7fa453fff640 1 -- 172.21.15.39:0/3428100577 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa454108c20 msgr2=0x7fa4540ff370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:56:31.814 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:31.810+0000 7fa453fff640 1 --2- 172.21.15.39:0/3428100577 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa454108c20 0x7fa4540ff370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:31.814 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:31.810+0000 7fa453fff640 1 -- 172.21.15.39:0/3428100577 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa4541a8720 con 0x7fa4541029a0 2024-09-21T03:56:31.815 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:31.810+0000 7fa453fff640 1 --2- 172.21.15.39:0/3428100577 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa4541029a0 0x7fa4540fe8f0 secure :-1 s=READY pgs=1361 cs=0 l=1 rev1=1 crypto rx=0x7fa44400e9f0 tx=0x7fa44400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:56:31.815 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:31.810+0000 7fa4517fa640 1 -- 172.21.15.39:0/3428100577 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa44400cd80 con 0x7fa4541029a0 2024-09-21T03:56:31.815 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:31.810+0000 7fa4517fa640 1 -- 172.21.15.39:0/3428100577 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa4440108e0 con 0x7fa4541029a0 2024-09-21T03:56:31.815 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:31.810+0000 7fa45a694640 1 -- 172.21.15.39:0/3428100577 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa4541a8a10 con 0x7fa4541029a0 2024-09-21T03:56:31.815 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:31.810+0000 7fa4517fa640 1 -- 172.21.15.39:0/3428100577 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa444018980 con 0x7fa4541029a0 2024-09-21T03:56:31.815 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:31.810+0000 7fa45a694640 1 -- 172.21.15.39:0/3428100577 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa4541a8f30 con 0x7fa4541029a0 2024-09-21T03:56:31.815 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:31.811+0000 7fa45a694640 1 -- 172.21.15.39:0/3428100577 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa454103c10 con 0x7fa4541029a0 2024-09-21T03:56:31.816 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:31.813+0000 7fa4517fa640 1 -- 172.21.15.39:0/3428100577 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 27) ==== 154643+0+0 (secure 0 0 0) 0x7fa444010430 con 0x7fa4541029a0 2024-09-21T03:56:31.816 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:31.814+0000 7fa4517fa640 1 --2- 172.21.15.39:0/3428100577 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7fa43c0b66b0 0x7fa43c0b8bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:56:31.816 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:31.814+0000 7fa4537fe640 1 --2- 172.21.15.39:0/3428100577 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7fa43c0b66b0 0x7fa43c0b8bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:56:31.816 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:31.814+0000 7fa4517fa640 1 -- 172.21.15.39:0/3428100577 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8528+0+0 (secure 0 0 0) 0x7fa4440e7e90 con 0x7fa4541029a0 2024-09-21T03:56:31.816 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:31.814+0000 7fa4537fe640 1 --2- 172.21.15.39:0/3428100577 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7fa43c0b66b0 0x7fa43c0b8bc0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7fa448009230 tx=0x7fa448031400 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:56:31.818 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:31.817+0000 7fa4517fa640 1 -- 172.21.15.39:0/3428100577 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fa4440b27c0 con 0x7fa4541029a0 2024-09-21T03:56:32.009 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:32.007+0000 7fa45a694640 1 -- 172.21.15.39:0/3428100577 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fa4540ffbf0 con 0x7fa4541029a0 2024-09-21T03:56:32.010 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:32.008+0000 7fa4517fa640 1 -- 172.21.15.39:0/3428100577 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v87) ==== 104+0+11 (secure 0 0 0) 0x7fa4440b7700 con 0x7fa4541029a0 2024-09-21T03:56:32.010 INFO:teuthology.orchestra.run.smithi039.stdout:pg_num: 32 2024-09-21T03:56:32.013 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:32.011+0000 7fa45a694640 1 -- 172.21.15.39:0/3428100577 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7fa43c0b66b0 msgr2=0x7fa43c0b8bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:56:32.013 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:32.011+0000 7fa45a694640 1 --2- 172.21.15.39:0/3428100577 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7fa43c0b66b0 0x7fa43c0b8bc0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7fa448009230 tx=0x7fa448031400 comp rx=0 tx=0).stop 2024-09-21T03:56:32.013 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:32.011+0000 7fa45a694640 1 -- 172.21.15.39:0/3428100577 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa4541029a0 msgr2=0x7fa4540fe8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:56:32.013 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:32.011+0000 7fa45a694640 1 --2- 172.21.15.39:0/3428100577 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa4541029a0 0x7fa4540fe8f0 secure :-1 s=READY pgs=1361 cs=0 l=1 rev1=1 crypto rx=0x7fa44400e9f0 tx=0x7fa44400eeb0 comp rx=0 tx=0).stop 2024-09-21T03:56:32.013 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:32.011+0000 7fa45a694640 1 -- 172.21.15.39:0/3428100577 shutdown_connections 2024-09-21T03:56:32.013 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:32.011+0000 7fa45a694640 1 --2- 172.21.15.39:0/3428100577 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7fa43c0b66b0 0x7fa43c0b8bc0 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:32.013 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:32.011+0000 7fa45a694640 1 --2- 172.21.15.39:0/3428100577 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa454108c20 0x7fa4540ff370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:32.013 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:32.011+0000 7fa45a694640 1 --2- 172.21.15.39:0/3428100577 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa4541082d0 0x7fa4540fee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:32.013 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:32.011+0000 7fa45a694640 1 --2- 172.21.15.39:0/3428100577 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa4541029a0 0x7fa4540fe8f0 unknown :-1 s=CLOSED pgs=1361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:32.013 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:32.011+0000 7fa45a694640 1 -- 172.21.15.39:0/3428100577 >> 172.21.15.39:0/3428100577 conn(0x7fa4540fe3c0 msgr2=0x7fa45406bce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:56:32.013 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:32.011+0000 7fa45a694640 1 -- 172.21.15.39:0/3428100577 shutdown_connections 2024-09-21T03:56:32.014 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:32.011+0000 7fa45a694640 1 -- 172.21.15.39:0/3428100577 wait complete. 2024-09-21T03:56:32.027 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-09-21T03:56:32.144 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:32.141+0000 7f1720541640 1 -- 172.21.15.39:0/3811552949 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1718101fd0 msgr2=0x7f17181023e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:56:32.144 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:32.141+0000 7f1720541640 1 --2- 172.21.15.39:0/3811552949 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1718101fd0 0x7f17181023e0 secure :-1 s=READY pgs=1362 cs=0 l=1 rev1=1 crypto rx=0x7f1714009900 tx=0x7f171402a410 comp rx=0 tx=0).stop 2024-09-21T03:56:32.144 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:32.142+0000 7f1720541640 1 -- 172.21.15.39:0/3811552949 shutdown_connections 2024-09-21T03:56:32.144 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:32.142+0000 7f1720541640 1 --2- 172.21.15.39:0/3811552949 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1718106b00 0x7f1718110590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:32.144 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:32.142+0000 7f1720541640 1 --2- 172.21.15.39:0/3811552949 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f17181061b0 0x7f17181065c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:32.144 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:32.142+0000 7f1720541640 1 --2- 172.21.15.39:0/3811552949 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1718101fd0 0x7f17181023e0 unknown :-1 s=CLOSED pgs=1362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:32.144 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:32.142+0000 7f1720541640 1 -- 172.21.15.39:0/3811552949 >> 172.21.15.39:0/3811552949 conn(0x7f1718079e00 msgr2=0x7f171807a210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:56:32.145 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:32.142+0000 7f1720541640 1 -- 172.21.15.39:0/3811552949 shutdown_connections 2024-09-21T03:56:32.145 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:32.142+0000 7f1720541640 1 -- 172.21.15.39:0/3811552949 wait complete. 2024-09-21T03:56:32.145 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:32.142+0000 7f1720541640 1 Processor -- start 2024-09-21T03:56:32.145 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:32.142+0000 7f1720541640 1 -- start start 2024-09-21T03:56:32.146 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:32.143+0000 7f1720541640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1718101fd0 0x7f171819a530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:56:32.147 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:32.143+0000 7f1720541640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f17181061b0 0x7f171819aad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:56:32.147 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:32.143+0000 7f1720541640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1718106b00 0x7f171819fd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:56:32.147 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:32.143+0000 7f1720541640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f17181139f0 con 0x7f1718101fd0 2024-09-21T03:56:32.147 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:32.143+0000 7f1720541640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f1718113870 con 0x7f17181061b0 2024-09-21T03:56:32.147 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:32.143+0000 7f1720541640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f1718113b70 con 0x7f1718106b00 2024-09-21T03:56:32.147 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:32.143+0000 7f171e2dd640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1718101fd0 0x7f171819a530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:56:32.147 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:32.143+0000 7f171e2dd640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1718101fd0 0x7f171819a530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:48786/0 (socket says 172.21.15.39:48786) 2024-09-21T03:56:32.147 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:32.143+0000 7f171e2dd640 1 -- 172.21.15.39:0/6203856 learned_addr learned my addr 172.21.15.39:0/6203856 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:56:32.147 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:32.143+0000 7f171dadc640 1 --2- 172.21.15.39:0/6203856 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f17181061b0 0x7f171819aad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:56:32.147 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:32.143+0000 7f171eade640 1 --2- 172.21.15.39:0/6203856 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1718106b00 0x7f171819fd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:56:32.147 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:32.143+0000 7f171e2dd640 1 -- 172.21.15.39:0/6203856 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1718106b00 msgr2=0x7f171819fd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:56:32.147 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:32.143+0000 7f171e2dd640 1 --2- 172.21.15.39:0/6203856 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1718106b00 0x7f171819fd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:32.147 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:32.143+0000 7f171e2dd640 1 -- 172.21.15.39:0/6203856 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f17181061b0 msgr2=0x7f171819aad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:56:32.147 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:32.143+0000 7f171e2dd640 1 --2- 172.21.15.39:0/6203856 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f17181061b0 0x7f171819aad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:32.148 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:32.143+0000 7f171e2dd640 1 -- 172.21.15.39:0/6203856 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f17181a0460 con 0x7f1718101fd0 2024-09-21T03:56:32.148 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:32.143+0000 7f171eade640 1 --2- 172.21.15.39:0/6203856 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1718106b00 0x7f171819fd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:56:32.148 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:32.143+0000 7f171dadc640 1 --2- 172.21.15.39:0/6203856 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f17181061b0 0x7f171819aad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:56:32.148 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:32.143+0000 7f171e2dd640 1 --2- 172.21.15.39:0/6203856 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1718101fd0 0x7f171819a530 secure :-1 s=READY pgs=1363 cs=0 l=1 rev1=1 crypto rx=0x7f1714032fd0 tx=0x7f1714031a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:56:32.148 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:32.144+0000 7f170f7fe640 1 -- 172.21.15.39:0/6203856 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f171403f030 con 0x7f1718101fd0 2024-09-21T03:56:32.148 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:32.144+0000 7f170f7fe640 1 -- 172.21.15.39:0/6203856 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1714004490 con 0x7f1718101fd0 2024-09-21T03:56:32.148 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:32.144+0000 7f170f7fe640 1 -- 172.21.15.39:0/6203856 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1714035070 con 0x7f1718101fd0 2024-09-21T03:56:32.148 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:32.144+0000 7f1720541640 1 -- 172.21.15.39:0/6203856 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f17181a06f0 con 0x7f1718101fd0 2024-09-21T03:56:32.148 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:32.144+0000 7f1720541640 1 -- 172.21.15.39:0/6203856 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f17181a0b90 con 0x7f1718101fd0 2024-09-21T03:56:32.148 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:32.145+0000 7f1720541640 1 -- 172.21.15.39:0/6203856 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f171806b610 con 0x7f1718101fd0 2024-09-21T03:56:32.149 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:32.147+0000 7f170f7fe640 1 -- 172.21.15.39:0/6203856 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 27) ==== 154643+0+0 (secure 0 0 0) 0x7f1714042030 con 0x7f1718101fd0 2024-09-21T03:56:32.149 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:32.148+0000 7f170f7fe640 1 --2- 172.21.15.39:0/6203856 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f17040b6700 0x7f17040b8c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:56:32.150 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:32.148+0000 7f171dadc640 1 --2- 172.21.15.39:0/6203856 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f17040b6700 0x7f17040b8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:56:32.150 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:32.148+0000 7f170f7fe640 1 -- 172.21.15.39:0/6203856 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8528+0+0 (secure 0 0 0) 0x7f1714108280 con 0x7f1718101fd0 2024-09-21T03:56:32.150 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:32.149+0000 7f171dadc640 1 --2- 172.21.15.39:0/6203856 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f17040b6700 0x7f17040b8c10 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f1718109150 tx=0x7f1708005f90 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:56:32.153 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:32.151+0000 7f170f7fe640 1 -- 172.21.15.39:0/6203856 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f17181a0b90 con 0x7f1718101fd0 2024-09-21T03:56:32.343 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:32.341+0000 7f1720541640 1 -- 172.21.15.39:0/6203856 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f171810bbf0 con 0x7f1718101fd0 2024-09-21T03:56:32.343 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:32.341+0000 7f170f7fe640 1 -- 172.21.15.39:0/6203856 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v87) ==== 100+0+11 (secure 0 0 0) 0x7f1714049030 con 0x7f1718101fd0 2024-09-21T03:56:32.343 INFO:teuthology.orchestra.run.smithi039.stdout:pg_num: 64 2024-09-21T03:56:32.345 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:32.344+0000 7f1720541640 1 -- 172.21.15.39:0/6203856 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f17040b6700 msgr2=0x7f17040b8c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:56:32.346 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:32.344+0000 7f1720541640 1 --2- 172.21.15.39:0/6203856 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f17040b6700 0x7f17040b8c10 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f1718109150 tx=0x7f1708005f90 comp rx=0 tx=0).stop 2024-09-21T03:56:32.346 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:32.344+0000 7f1720541640 1 -- 172.21.15.39:0/6203856 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1718101fd0 msgr2=0x7f171819a530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:56:32.346 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:32.344+0000 7f1720541640 1 --2- 172.21.15.39:0/6203856 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1718101fd0 0x7f171819a530 secure :-1 s=READY pgs=1363 cs=0 l=1 rev1=1 crypto rx=0x7f1714032fd0 tx=0x7f1714031a20 comp rx=0 tx=0).stop 2024-09-21T03:56:32.346 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:32.344+0000 7f1720541640 1 -- 172.21.15.39:0/6203856 shutdown_connections 2024-09-21T03:56:32.346 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:32.344+0000 7f1720541640 1 --2- 172.21.15.39:0/6203856 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f17040b6700 0x7f17040b8c10 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:32.346 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:32.344+0000 7f1720541640 1 --2- 172.21.15.39:0/6203856 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1718106b00 0x7f171819fd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:32.346 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:32.344+0000 7f1720541640 1 --2- 172.21.15.39:0/6203856 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f17181061b0 0x7f171819aad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:32.346 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:32.344+0000 7f1720541640 1 --2- 172.21.15.39:0/6203856 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1718101fd0 0x7f171819a530 unknown :-1 s=CLOSED pgs=1363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:32.346 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:32.344+0000 7f1720541640 1 -- 172.21.15.39:0/6203856 >> 172.21.15.39:0/6203856 conn(0x7f1718079e00 msgr2=0x7f1718107530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:56:32.346 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:32.344+0000 7f1720541640 1 -- 172.21.15.39:0/6203856 shutdown_connections 2024-09-21T03:56:32.346 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:32.344+0000 7f1720541640 1 -- 172.21.15.39:0/6203856 wait complete. 2024-09-21T03:56:32.357 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json-pretty 2024-09-21T03:56:32.479 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:32.475+0000 7fa1aadba640 1 -- 172.21.15.39:0/4271461352 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa1a4108c00 msgr2=0x7fa1a410f430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:56:32.479 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:32.475+0000 7fa1aadba640 1 --2- 172.21.15.39:0/4271461352 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa1a4108c00 0x7fa1a410f430 secure :-1 s=READY pgs=1364 cs=0 l=1 rev1=1 crypto rx=0x7fa19c009ca0 tx=0x7fa19c02a8c0 comp rx=0 tx=0).stop 2024-09-21T03:56:32.479 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:32.475+0000 7fa1aadba640 1 -- 172.21.15.39:0/4271461352 shutdown_connections 2024-09-21T03:56:32.479 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:32.475+0000 7fa1aadba640 1 --2- 172.21.15.39:0/4271461352 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa1a4108c00 0x7fa1a410f430 unknown :-1 s=CLOSED pgs=1364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:32.479 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:32.475+0000 7fa1aadba640 1 --2- 172.21.15.39:0/4271461352 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa1a41082b0 0x7fa1a41086c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:32.479 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:32.475+0000 7fa1aadba640 1 --2- 172.21.15.39:0/4271461352 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa1a4102980 0x7fa1a4102d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:32.479 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:32.475+0000 7fa1aadba640 1 -- 172.21.15.39:0/4271461352 >> 172.21.15.39:0/4271461352 conn(0x7fa1a40fe380 msgr2=0x7fa1a41007d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:56:32.479 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:32.475+0000 7fa1aadba640 1 -- 172.21.15.39:0/4271461352 shutdown_connections 2024-09-21T03:56:32.479 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:32.475+0000 7fa1aadba640 1 -- 172.21.15.39:0/4271461352 wait complete. 2024-09-21T03:56:32.479 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:32.475+0000 7fa1aadba640 1 Processor -- start 2024-09-21T03:56:32.480 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:32.476+0000 7fa1aadba640 1 -- start start 2024-09-21T03:56:32.480 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:32.476+0000 7fa1aadba640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa1a4102980 0x7fa1a419a370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:56:32.480 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:32.476+0000 7fa1aadba640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa1a41082b0 0x7fa1a419a910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:56:32.480 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:32.476+0000 7fa1aadba640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa1a4108c00 0x7fa1a419fba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:56:32.480 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:32.476+0000 7fa1aadba640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fa1a41128c0 con 0x7fa1a4102980 2024-09-21T03:56:32.480 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:32.476+0000 7fa1aadba640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fa1a4112740 con 0x7fa1a4108c00 2024-09-21T03:56:32.480 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:32.476+0000 7fa1aadba640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fa1a4112a40 con 0x7fa1a41082b0 2024-09-21T03:56:32.480 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:32.476+0000 7fa1a8b56640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa1a4102980 0x7fa1a419a370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:56:32.480 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:32.476+0000 7fa1a8b56640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa1a4102980 0x7fa1a419a370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:48812/0 (socket says 172.21.15.39:48812) 2024-09-21T03:56:32.480 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:32.476+0000 7fa1a8b56640 1 -- 172.21.15.39:0/2360451637 learned_addr learned my addr 172.21.15.39:0/2360451637 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:56:32.480 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:32.476+0000 7fa1a3fff640 1 --2- 172.21.15.39:0/2360451637 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa1a41082b0 0x7fa1a419a910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:56:32.480 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:32.476+0000 7fa1a9357640 1 --2- 172.21.15.39:0/2360451637 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa1a4108c00 0x7fa1a419fba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:56:32.480 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:32.476+0000 7fa1a8b56640 1 -- 172.21.15.39:0/2360451637 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa1a41082b0 msgr2=0x7fa1a419a910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:56:32.480 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:32.476+0000 7fa1a8b56640 1 --2- 172.21.15.39:0/2360451637 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa1a41082b0 0x7fa1a419a910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:32.481 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:32.476+0000 7fa1a8b56640 1 -- 172.21.15.39:0/2360451637 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa1a4108c00 msgr2=0x7fa1a419fba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:56:32.481 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:32.476+0000 7fa1a8b56640 1 --2- 172.21.15.39:0/2360451637 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa1a4108c00 0x7fa1a419fba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:32.481 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:32.476+0000 7fa1a8b56640 1 -- 172.21.15.39:0/2360451637 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa1a41a02a0 con 0x7fa1a4102980 2024-09-21T03:56:32.481 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:32.477+0000 7fa1a8b56640 1 --2- 172.21.15.39:0/2360451637 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa1a4102980 0x7fa1a419a370 secure :-1 s=READY pgs=1365 cs=0 l=1 rev1=1 crypto rx=0x7fa19800e9f0 tx=0x7fa19800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:56:32.481 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:32.477+0000 7fa1a1ffb640 1 -- 172.21.15.39:0/2360451637 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa19800cd80 con 0x7fa1a4102980 2024-09-21T03:56:32.481 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:32.477+0000 7fa1aadba640 1 -- 172.21.15.39:0/2360451637 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa1a41a0590 con 0x7fa1a4102980 2024-09-21T03:56:32.481 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:32.477+0000 7fa1a1ffb640 1 -- 172.21.15.39:0/2360451637 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa1980108e0 con 0x7fa1a4102980 2024-09-21T03:56:32.481 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:32.477+0000 7fa1a1ffb640 1 -- 172.21.15.39:0/2360451637 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa198018980 con 0x7fa1a4102980 2024-09-21T03:56:32.481 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:32.477+0000 7fa1aadba640 1 -- 172.21.15.39:0/2360451637 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa1a41a6d20 con 0x7fa1a4102980 2024-09-21T03:56:32.482 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:32.478+0000 7fa1aadba640 1 -- 172.21.15.39:0/2360451637 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa1a4103bf0 con 0x7fa1a4102980 2024-09-21T03:56:32.482 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:32.480+0000 7fa1a1ffb640 1 -- 172.21.15.39:0/2360451637 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 27) ==== 154643+0+0 (secure 0 0 0) 0x7fa198010430 con 0x7fa1a4102980 2024-09-21T03:56:32.483 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:32.481+0000 7fa1a1ffb640 1 --2- 172.21.15.39:0/2360451637 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7fa1940b6680 0x7fa1940b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:56:32.483 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:32.481+0000 7fa1a3fff640 1 --2- 172.21.15.39:0/2360451637 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7fa1940b6680 0x7fa1940b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:56:32.483 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:32.481+0000 7fa1a1ffb640 1 -- 172.21.15.39:0/2360451637 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8528+0+0 (secure 0 0 0) 0x7fa1980e7ec0 con 0x7fa1a4102980 2024-09-21T03:56:32.483 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:32.482+0000 7fa1a3fff640 1 --2- 172.21.15.39:0/2360451637 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7fa1940b6680 0x7fa1940b8b90 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7fa1a4101af0 tx=0x7fa190005f90 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:56:32.485 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:32.484+0000 7fa1a1ffb640 1 -- 172.21.15.39:0/2360451637 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fa1980b27f0 con 0x7fa1a4102980 2024-09-21T03:56:32.584 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:56:32 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:56:32.584 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:56:32 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:56:32.584 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:56:32 smithi039 ceph-mon[31211]: from='client.? 172.21.15.39:0/3733740233' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T03:56:32.584 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:56:32 smithi039 ceph-mon[31211]: from='client.? 172.21.15.39:0/2449893003' entity='client.admin' cmd={"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} : dispatch 2024-09-21T03:56:32.584 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:56:32 smithi039 ceph-mon[31211]: from='client.? 172.21.15.39:0/3428100577' entity='client.admin' cmd={"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} : dispatch 2024-09-21T03:56:32.718 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:56:32 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:56:32.718 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:56:32 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:56:32.718 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:56:32 smithi161 ceph-mon[34533]: from='client.? 172.21.15.39:0/3733740233' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T03:56:32.718 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:56:32 smithi161 ceph-mon[34533]: from='client.? 172.21.15.39:0/2449893003' entity='client.admin' cmd={"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} : dispatch 2024-09-21T03:56:32.718 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:56:32 smithi161 ceph-mon[34533]: from='client.? 172.21.15.39:0/3428100577' entity='client.admin' cmd={"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} : dispatch 2024-09-21T03:56:32.724 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:56:32 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:56:32.725 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:56:32 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:56:32.725 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:56:32 smithi150 ceph-mon[34289]: from='client.? 172.21.15.39:0/3733740233' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T03:56:32.725 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:56:32 smithi150 ceph-mon[34289]: from='client.? 172.21.15.39:0/2449893003' entity='client.admin' cmd={"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} : dispatch 2024-09-21T03:56:32.725 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:56:32 smithi150 ceph-mon[34289]: from='client.? 172.21.15.39:0/3428100577' entity='client.admin' cmd={"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} : dispatch 2024-09-21T03:56:32.743 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:32.741+0000 7fa1aadba640 1 -- 172.21.15.39:0/2360451637 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json-pretty"} v 0) -- 0x7fa1a41a7000 con 0x7fa1a4102980 2024-09-21T03:56:32.752 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:32.750+0000 7fa1a1ffb640 1 -- 172.21.15.39:0/2360451637 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json-pretty"}]=0 v27) ==== 81+0+834054 (secure 0 0 0) 0x7fa1980b7730 con 0x7fa1a4102980 2024-09-21T03:56:32.762 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T03:56:32.762 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-09-21T03:56:32.762 INFO:teuthology.orchestra.run.smithi039.stdout: "epoch": 27, 2024-09-21T03:56:32.762 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-09-21T03:56:32.762 INFO:teuthology.orchestra.run.smithi039.stdout: "active_gid": 14223, 2024-09-21T03:56:32.763 INFO:teuthology.orchestra.run.smithi039.stdout: "active_name": "x", 2024-09-21T03:56:32.763 INFO:teuthology.orchestra.run.smithi039.stdout: "active_addrs": { 2024-09-21T03:56:32.763 INFO:teuthology.orchestra.run.smithi039.stdout: "addrvec": [ 2024-09-21T03:56:32.763 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-21T03:56:32.763 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "v2", 2024-09-21T03:56:32.763 INFO:teuthology.orchestra.run.smithi039.stdout: "addr": "172.21.15.39:6800", 2024-09-21T03:56:32.763 INFO:teuthology.orchestra.run.smithi039.stdout: "nonce": 1696381982 2024-09-21T03:56:32.763 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:32.763 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-21T03:56:32.763 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "v1", 2024-09-21T03:56:32.763 INFO:teuthology.orchestra.run.smithi039.stdout: "addr": "172.21.15.39:6801", 2024-09-21T03:56:32.763 INFO:teuthology.orchestra.run.smithi039.stdout: "nonce": 1696381982 2024-09-21T03:56:32.763 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-21T03:56:32.763 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-21T03:56:32.764 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:32.764 INFO:teuthology.orchestra.run.smithi039.stdout: "active_addr": "172.21.15.39:6801/1696381982", 2024-09-21T03:56:32.764 INFO:teuthology.orchestra.run.smithi039.stdout: "active_change": "2024-09-21T03:37:25.391596+0000", 2024-09-21T03:56:32.764 INFO:teuthology.orchestra.run.smithi039.stdout: "active_mgr_features": 4540719139924082687, 2024-09-21T03:56:32.764 INFO:teuthology.orchestra.run.smithi039.stdout: "available": true, 2024-09-21T03:56:32.764 INFO:teuthology.orchestra.run.smithi039.stdout: "standbys": [ 2024-09-21T03:56:32.764 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-21T03:56:32.764 INFO:teuthology.orchestra.run.smithi039.stdout: "gid": 24142, 2024-09-21T03:56:32.764 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "z", 2024-09-21T03:56:32.764 INFO:teuthology.orchestra.run.smithi039.stdout: "mgr_features": 4540719139924082687, 2024-09-21T03:56:32.764 INFO:teuthology.orchestra.run.smithi039.stdout: "available_modules": [ 2024-09-21T03:56:32.764 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-21T03:56:32.764 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "alerts", 2024-09-21T03:56:32.765 INFO:teuthology.orchestra.run.smithi039.stdout: "can_run": true, 2024-09-21T03:56:32.765 INFO:teuthology.orchestra.run.smithi039.stdout: "error_string": "", 2024-09-21T03:56:32.765 INFO:teuthology.orchestra.run.smithi039.stdout: "module_options": { 2024-09-21T03:56:32.765 INFO:teuthology.orchestra.run.smithi039.stdout: "interval": { 2024-09-21T03:56:32.765 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "interval", 2024-09-21T03:56:32.765 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "secs", 2024-09-21T03:56:32.765 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:32.765 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:32.765 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "60", 2024-09-21T03:56:32.765 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:32.765 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:32.765 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:32.765 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "How frequently to reexamine health status", 2024-09-21T03:56:32.765 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:32.765 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:32.766 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:32.766 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:32.766 INFO:teuthology.orchestra.run.smithi039.stdout: "log_level": { 2024-09-21T03:56:32.766 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_level", 2024-09-21T03:56:32.766 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-09-21T03:56:32.766 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:32.766 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:32.766 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-09-21T03:56:32.766 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:32.766 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:32.766 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-09-21T03:56:32.766 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-09-21T03:56:32.767 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-09-21T03:56:32.767 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-09-21T03:56:32.767 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-09-21T03:56:32.767 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-09-21T03:56:32.767 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-09-21T03:56:32.767 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-09-21T03:56:32.767 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:32.767 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:32.767 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:32.767 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:32.767 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:32.767 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster": { 2024-09-21T03:56:32.767 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster", 2024-09-21T03:56:32.767 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-09-21T03:56:32.768 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:32.768 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:32.768 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-09-21T03:56:32.768 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:32.768 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:32.768 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:32.768 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:32.768 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:32.768 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:32.768 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:32.768 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:32.768 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster_level": { 2024-09-21T03:56:32.768 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster_level", 2024-09-21T03:56:32.768 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-09-21T03:56:32.768 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:32.769 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:32.769 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "info", 2024-09-21T03:56:32.769 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:32.769 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:32.769 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-09-21T03:56:32.769 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-09-21T03:56:32.769 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-09-21T03:56:32.769 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-09-21T03:56:32.769 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-09-21T03:56:32.769 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-09-21T03:56:32.769 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-09-21T03:56:32.769 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-09-21T03:56:32.769 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:32.769 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:32.770 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:32.770 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:32.770 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:32.770 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_file": { 2024-09-21T03:56:32.770 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_file", 2024-09-21T03:56:32.770 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-09-21T03:56:32.770 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:32.770 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:32.770 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-09-21T03:56:32.770 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:32.770 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:32.770 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:32.770 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:32.770 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:32.771 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:32.771 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:32.771 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:32.771 INFO:teuthology.orchestra.run.smithi039.stdout: "smtp_destination": { 2024-09-21T03:56:32.771 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "smtp_destination", 2024-09-21T03:56:32.771 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-09-21T03:56:32.771 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:32.771 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:32.771 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-09-21T03:56:32.771 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:32.771 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:32.771 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:32.771 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Email address to send alerts to, use commas to separate multiple", 2024-09-21T03:56:32.771 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:32.771 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:32.772 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:32.772 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:32.772 INFO:teuthology.orchestra.run.smithi039.stdout: "smtp_from_name": { 2024-09-21T03:56:32.772 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "smtp_from_name", 2024-09-21T03:56:32.772 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-09-21T03:56:32.772 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:32.772 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:32.772 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "Ceph", 2024-09-21T03:56:32.772 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:32.772 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:32.772 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:32.772 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Email From: name", 2024-09-21T03:56:32.772 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:32.772 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:32.772 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:32.773 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:32.773 INFO:teuthology.orchestra.run.smithi039.stdout: "smtp_host": { 2024-09-21T03:56:32.773 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "smtp_host", 2024-09-21T03:56:32.773 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-09-21T03:56:32.773 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:32.773 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:32.773 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-09-21T03:56:32.773 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:32.773 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:32.773 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:32.773 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "SMTP server", 2024-09-21T03:56:32.773 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:32.773 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:32.773 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:32.774 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:32.774 INFO:teuthology.orchestra.run.smithi039.stdout: "smtp_password": { 2024-09-21T03:56:32.774 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "smtp_password", 2024-09-21T03:56:32.774 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-09-21T03:56:32.774 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:32.774 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:32.774 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-09-21T03:56:32.774 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:32.774 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:32.774 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:32.774 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Password to authenticate with", 2024-09-21T03:56:32.774 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:32.774 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:32.774 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:32.774 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:32.775 INFO:teuthology.orchestra.run.smithi039.stdout: "smtp_port": { 2024-09-21T03:56:32.775 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "smtp_port", 2024-09-21T03:56:32.775 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-09-21T03:56:32.775 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:32.775 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:32.775 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "465", 2024-09-21T03:56:32.775 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:32.775 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:32.775 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:32.775 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "SMTP port", 2024-09-21T03:56:32.775 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:32.775 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:32.775 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:32.775 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:32.775 INFO:teuthology.orchestra.run.smithi039.stdout: "smtp_sender": { 2024-09-21T03:56:32.776 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "smtp_sender", 2024-09-21T03:56:32.776 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-09-21T03:56:32.776 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:32.776 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:32.776 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-09-21T03:56:32.776 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:32.776 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:32.776 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:32.776 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "SMTP envelope sender", 2024-09-21T03:56:32.776 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:32.776 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:32.776 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:32.776 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:32.777 INFO:teuthology.orchestra.run.smithi039.stdout: "smtp_ssl": { 2024-09-21T03:56:32.777 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "smtp_ssl", 2024-09-21T03:56:32.777 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-09-21T03:56:32.777 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:32.777 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:32.777 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2024-09-21T03:56:32.777 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:32.777 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:32.777 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:32.777 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Use SSL to connect to SMTP server", 2024-09-21T03:56:32.777 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:32.777 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:32.777 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:32.778 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:32.778 INFO:teuthology.orchestra.run.smithi039.stdout: "smtp_user": { 2024-09-21T03:56:32.778 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "smtp_user", 2024-09-21T03:56:32.778 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-09-21T03:56:32.778 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:32.778 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:32.778 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-09-21T03:56:32.778 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:32.778 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:32.778 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:32.778 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "User to authenticate as", 2024-09-21T03:56:32.778 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:32.778 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:32.778 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:32.779 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:32.779 INFO:teuthology.orchestra.run.smithi039.stdout: "sqlite3_killpoint": { 2024-09-21T03:56:32.779 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:56:32.779 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-09-21T03:56:32.779 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "dev", 2024-09-21T03:56:32.779 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:32.779 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "0", 2024-09-21T03:56:32.779 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:32.779 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:32.779 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:32.779 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:32.779 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:32.779 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:32.779 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:32.779 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-21T03:56:32.780 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-21T03:56:32.780 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:32.780 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-21T03:56:32.780 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "balancer", 2024-09-21T03:56:32.780 INFO:teuthology.orchestra.run.smithi039.stdout: "can_run": true, 2024-09-21T03:56:32.780 INFO:teuthology.orchestra.run.smithi039.stdout: "error_string": "", 2024-09-21T03:56:32.780 INFO:teuthology.orchestra.run.smithi039.stdout: "module_options": { 2024-09-21T03:56:32.780 INFO:teuthology.orchestra.run.smithi039.stdout: "active": { 2024-09-21T03:56:32.780 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "active", 2024-09-21T03:56:32.780 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-09-21T03:56:32.780 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:32.780 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:32.780 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2024-09-21T03:56:32.780 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:32.781 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:32.781 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:32.781 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "automatically balance PGs across cluster", 2024-09-21T03:56:32.781 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:32.781 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:32.781 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:32.781 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:32.781 INFO:teuthology.orchestra.run.smithi039.stdout: "begin_time": { 2024-09-21T03:56:32.781 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "begin_time", 2024-09-21T03:56:32.781 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-09-21T03:56:32.781 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:32.781 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:32.781 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "0000", 2024-09-21T03:56:32.781 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:32.781 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:32.782 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:32.782 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "beginning time of day to automatically balance", 2024-09-21T03:56:32.782 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-09-21T03:56:32.782 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:32.782 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:32.782 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:32.782 INFO:teuthology.orchestra.run.smithi039.stdout: "begin_weekday": { 2024-09-21T03:56:32.782 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "begin_weekday", 2024-09-21T03:56:32.782 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "uint", 2024-09-21T03:56:32.782 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:32.782 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:32.782 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "0", 2024-09-21T03:56:32.782 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "0", 2024-09-21T03:56:32.782 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "6", 2024-09-21T03:56:32.783 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:32.783 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-09-21T03:56:32.783 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-09-21T03:56:32.783 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:32.783 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:32.783 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:32.783 INFO:teuthology.orchestra.run.smithi039.stdout: "crush_compat_max_iterations": { 2024-09-21T03:56:32.783 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "crush_compat_max_iterations", 2024-09-21T03:56:32.783 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "uint", 2024-09-21T03:56:32.783 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:32.783 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:32.783 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "25", 2024-09-21T03:56:32.783 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "1", 2024-09-21T03:56:32.783 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "250", 2024-09-21T03:56:32.784 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:32.784 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-09-21T03:56:32.784 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:32.784 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:32.784 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:32.784 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:32.784 INFO:teuthology.orchestra.run.smithi039.stdout: "crush_compat_metrics": { 2024-09-21T03:56:32.784 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "crush_compat_metrics", 2024-09-21T03:56:32.784 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-09-21T03:56:32.784 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:32.784 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:32.784 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "pgs,objects,bytes", 2024-09-21T03:56:32.784 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:32.784 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:32.785 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:32.785 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-09-21T03:56:32.785 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-09-21T03:56:32.785 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:32.785 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:32.785 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:32.785 INFO:teuthology.orchestra.run.smithi039.stdout: "crush_compat_step": { 2024-09-21T03:56:32.785 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "crush_compat_step", 2024-09-21T03:56:32.785 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "float", 2024-09-21T03:56:32.785 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:32.785 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:32.785 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "0.5", 2024-09-21T03:56:32.785 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "0.001", 2024-09-21T03:56:32.785 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "0.999", 2024-09-21T03:56:32.785 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:32.786 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "aggressiveness of optimization", 2024-09-21T03:56:32.786 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-09-21T03:56:32.786 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:32.786 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:32.786 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:32.786 INFO:teuthology.orchestra.run.smithi039.stdout: "end_time": { 2024-09-21T03:56:32.786 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "end_time", 2024-09-21T03:56:32.786 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-09-21T03:56:32.786 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:32.786 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:32.786 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "2359", 2024-09-21T03:56:32.786 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:32.786 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:32.787 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:32.787 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "ending time of day to automatically balance", 2024-09-21T03:56:32.787 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-09-21T03:56:32.787 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:32.787 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:32.787 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:32.787 INFO:teuthology.orchestra.run.smithi039.stdout: "end_weekday": { 2024-09-21T03:56:32.787 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "end_weekday", 2024-09-21T03:56:32.787 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "uint", 2024-09-21T03:56:32.787 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:32.787 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:32.787 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "0", 2024-09-21T03:56:32.787 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "0", 2024-09-21T03:56:32.787 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "6", 2024-09-21T03:56:32.787 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:32.788 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-09-21T03:56:32.788 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-09-21T03:56:32.788 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:32.788 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:32.788 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:32.788 INFO:teuthology.orchestra.run.smithi039.stdout: "log_level": { 2024-09-21T03:56:32.788 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_level", 2024-09-21T03:56:32.788 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-09-21T03:56:32.788 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:32.788 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:32.788 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-09-21T03:56:32.788 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:32.788 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:32.788 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-09-21T03:56:32.788 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-09-21T03:56:32.789 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-09-21T03:56:32.789 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-09-21T03:56:32.789 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-09-21T03:56:32.789 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-09-21T03:56:32.789 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-09-21T03:56:32.789 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-09-21T03:56:32.789 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:32.789 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:32.789 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:32.789 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:32.789 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:32.789 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster": { 2024-09-21T03:56:32.789 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster", 2024-09-21T03:56:32.790 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-09-21T03:56:32.790 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:32.790 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:32.790 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-09-21T03:56:32.790 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:32.790 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:32.790 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:32.790 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:32.790 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:32.790 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:32.790 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:32.790 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:32.790 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster_level": { 2024-09-21T03:56:32.790 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster_level", 2024-09-21T03:56:32.791 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-09-21T03:56:32.791 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:32.791 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:32.791 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "info", 2024-09-21T03:56:32.791 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:32.791 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:32.791 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-09-21T03:56:32.791 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-09-21T03:56:32.791 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-09-21T03:56:32.791 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-09-21T03:56:32.791 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-09-21T03:56:32.791 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-09-21T03:56:32.791 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-09-21T03:56:32.791 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-09-21T03:56:32.792 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:32.792 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:32.792 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:32.792 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:32.792 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:32.792 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_file": { 2024-09-21T03:56:32.792 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_file", 2024-09-21T03:56:32.792 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-09-21T03:56:32.792 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:32.792 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:32.792 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-09-21T03:56:32.792 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:32.792 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:32.792 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:32.793 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:32.793 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:32.793 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:32.793 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:32.793 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:32.793 INFO:teuthology.orchestra.run.smithi039.stdout: "min_score": { 2024-09-21T03:56:32.793 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "min_score", 2024-09-21T03:56:32.793 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "float", 2024-09-21T03:56:32.793 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:32.793 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:32.793 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "0", 2024-09-21T03:56:32.793 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:32.793 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:32.793 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:32.793 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-09-21T03:56:32.794 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:32.794 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:32.794 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:32.794 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:32.794 INFO:teuthology.orchestra.run.smithi039.stdout: "mode": { 2024-09-21T03:56:32.794 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "mode", 2024-09-21T03:56:32.794 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-09-21T03:56:32.794 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:32.794 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:32.794 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "upmap", 2024-09-21T03:56:32.794 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:32.794 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:32.794 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-09-21T03:56:32.794 INFO:teuthology.orchestra.run.smithi039.stdout: "crush-compat", 2024-09-21T03:56:32.795 INFO:teuthology.orchestra.run.smithi039.stdout: "none", 2024-09-21T03:56:32.795 INFO:teuthology.orchestra.run.smithi039.stdout: "read", 2024-09-21T03:56:32.795 INFO:teuthology.orchestra.run.smithi039.stdout: "upmap", 2024-09-21T03:56:32.795 INFO:teuthology.orchestra.run.smithi039.stdout: "upmap-read" 2024-09-21T03:56:32.795 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-09-21T03:56:32.795 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Balancer mode", 2024-09-21T03:56:32.795 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:32.795 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:32.795 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:32.795 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:32.795 INFO:teuthology.orchestra.run.smithi039.stdout: "pool_ids": { 2024-09-21T03:56:32.795 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "pool_ids", 2024-09-21T03:56:32.795 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-09-21T03:56:32.795 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:32.796 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:32.796 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-09-21T03:56:32.796 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:32.796 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:32.796 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:32.796 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-09-21T03:56:32.796 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:32.796 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:32.796 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:32.796 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:32.796 INFO:teuthology.orchestra.run.smithi039.stdout: "sleep_interval": { 2024-09-21T03:56:32.796 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "sleep_interval", 2024-09-21T03:56:32.796 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "secs", 2024-09-21T03:56:32.796 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:32.797 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:32.797 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "60", 2024-09-21T03:56:32.797 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:32.797 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:32.797 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:32.797 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-09-21T03:56:32.797 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:32.797 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:32.797 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:32.797 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:32.797 INFO:teuthology.orchestra.run.smithi039.stdout: "sqlite3_killpoint": { 2024-09-21T03:56:32.797 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:56:32.797 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-09-21T03:56:32.797 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "dev", 2024-09-21T03:56:32.797 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:32.798 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "0", 2024-09-21T03:56:32.798 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:32.798 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:32.798 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:32.798 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:32.798 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:32.798 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:32.798 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:32.798 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:32.798 INFO:teuthology.orchestra.run.smithi039.stdout: "upmap_max_deviation": { 2024-09-21T03:56:32.798 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "upmap_max_deviation", 2024-09-21T03:56:32.798 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-09-21T03:56:32.798 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:32.798 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:32.798 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "5", 2024-09-21T03:56:32.799 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "1", 2024-09-21T03:56:32.799 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:32.799 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:32.799 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "deviation below which no optimization is attempted", 2024-09-21T03:56:32.799 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-09-21T03:56:32.799 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:32.799 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:32.799 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:32.799 INFO:teuthology.orchestra.run.smithi039.stdout: "upmap_max_optimizations": { 2024-09-21T03:56:32.799 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "upmap_max_optimizations", 2024-09-21T03:56:32.799 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "uint", 2024-09-21T03:56:32.799 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:32.799 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:32.799 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "10", 2024-09-21T03:56:32.800 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:32.800 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:32.800 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:32.800 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-09-21T03:56:32.800 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:32.800 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:32.800 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:32.800 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-21T03:56:32.800 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-21T03:56:32.800 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:32.800 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-21T03:56:32.800 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "cephadm", 2024-09-21T03:56:32.800 INFO:teuthology.orchestra.run.smithi039.stdout: "can_run": true, 2024-09-21T03:56:32.800 INFO:teuthology.orchestra.run.smithi039.stdout: "error_string": "", 2024-09-21T03:56:32.800 INFO:teuthology.orchestra.run.smithi039.stdout: "module_options": { 2024-09-21T03:56:32.801 INFO:teuthology.orchestra.run.smithi039.stdout: "agent_down_multiplier": { 2024-09-21T03:56:32.801 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "agent_down_multiplier", 2024-09-21T03:56:32.801 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "float", 2024-09-21T03:56:32.801 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:32.801 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-09-21T03:56:32.801 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "3.0", 2024-09-21T03:56:32.801 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:32.801 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:32.801 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:32.801 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-09-21T03:56:32.801 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:32.801 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:32.801 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:32.801 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:32.802 INFO:teuthology.orchestra.run.smithi039.stdout: "agent_refresh_rate": { 2024-09-21T03:56:32.802 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "agent_refresh_rate", 2024-09-21T03:56:32.802 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "secs", 2024-09-21T03:56:32.802 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:32.802 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-09-21T03:56:32.802 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "20", 2024-09-21T03:56:32.802 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:32.802 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:32.802 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:32.802 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-09-21T03:56:32.802 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:32.802 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:32.802 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:32.803 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:32.803 INFO:teuthology.orchestra.run.smithi039.stdout: "agent_starting_port": { 2024-09-21T03:56:32.803 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "agent_starting_port", 2024-09-21T03:56:32.803 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-09-21T03:56:32.803 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:32.803 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-09-21T03:56:32.803 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "4721", 2024-09-21T03:56:32.803 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:32.803 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:32.803 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:32.803 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-09-21T03:56:32.803 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:32.803 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:32.804 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:32.804 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:32.804 INFO:teuthology.orchestra.run.smithi039.stdout: "allow_ptrace": { 2024-09-21T03:56:32.804 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "allow_ptrace", 2024-09-21T03:56:32.804 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-09-21T03:56:32.804 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:32.804 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-09-21T03:56:32.804 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-09-21T03:56:32.804 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:32.804 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:32.804 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:32.804 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-09-21T03:56:32.804 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-09-21T03:56:32.804 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:32.805 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:32.805 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:32.805 INFO:teuthology.orchestra.run.smithi039.stdout: "autotune_interval": { 2024-09-21T03:56:32.805 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "autotune_interval", 2024-09-21T03:56:32.805 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "secs", 2024-09-21T03:56:32.805 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:32.805 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-09-21T03:56:32.805 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "600", 2024-09-21T03:56:32.805 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:32.805 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:32.805 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:32.805 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "how frequently to autotune daemon memory", 2024-09-21T03:56:32.805 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:32.805 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:32.805 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:32.806 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:32.806 INFO:teuthology.orchestra.run.smithi039.stdout: "autotune_memory_target_ratio": { 2024-09-21T03:56:32.806 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "autotune_memory_target_ratio", 2024-09-21T03:56:32.806 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "float", 2024-09-21T03:56:32.806 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:32.806 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-09-21T03:56:32.806 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "0.7", 2024-09-21T03:56:32.806 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:32.806 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:32.806 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:32.806 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-09-21T03:56:32.806 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:32.806 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:32.806 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:32.807 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:32.807 INFO:teuthology.orchestra.run.smithi039.stdout: "cephadm_log_destination": { 2024-09-21T03:56:32.807 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "cephadm_log_destination", 2024-09-21T03:56:32.807 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-09-21T03:56:32.807 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:32.807 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-09-21T03:56:32.807 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-09-21T03:56:32.807 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:32.807 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:32.807 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-09-21T03:56:32.807 INFO:teuthology.orchestra.run.smithi039.stdout: "file", 2024-09-21T03:56:32.807 INFO:teuthology.orchestra.run.smithi039.stdout: "file,syslog", 2024-09-21T03:56:32.807 INFO:teuthology.orchestra.run.smithi039.stdout: "syslog" 2024-09-21T03:56:32.807 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-09-21T03:56:32.807 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-09-21T03:56:32.808 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:32.808 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:32.808 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:32.808 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:32.808 INFO:teuthology.orchestra.run.smithi039.stdout: "cgroups_split": { 2024-09-21T03:56:32.808 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "cgroups_split", 2024-09-21T03:56:32.808 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-09-21T03:56:32.808 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:32.808 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-09-21T03:56:32.808 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2024-09-21T03:56:32.808 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:32.808 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:32.808 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:32.808 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-09-21T03:56:32.808 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:32.809 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:32.809 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:32.809 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:32.809 INFO:teuthology.orchestra.run.smithi039.stdout: "config_checks_enabled": { 2024-09-21T03:56:32.809 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "config_checks_enabled", 2024-09-21T03:56:32.809 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-09-21T03:56:32.809 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:32.809 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-09-21T03:56:32.809 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-09-21T03:56:32.809 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:32.809 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:32.809 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:32.809 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-09-21T03:56:32.809 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:32.809 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:32.810 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:32.810 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:32.810 INFO:teuthology.orchestra.run.smithi039.stdout: "config_dashboard": { 2024-09-21T03:56:32.810 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "config_dashboard", 2024-09-21T03:56:32.810 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-09-21T03:56:32.810 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:32.810 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-09-21T03:56:32.810 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2024-09-21T03:56:32.810 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:32.810 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:32.810 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:32.810 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-09-21T03:56:32.810 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:32.810 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:32.811 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:32.811 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:32.811 INFO:teuthology.orchestra.run.smithi039.stdout: "container_image_alertmanager": { 2024-09-21T03:56:32.811 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "container_image_alertmanager", 2024-09-21T03:56:32.811 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-09-21T03:56:32.811 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:32.811 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-09-21T03:56:32.811 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.27.0", 2024-09-21T03:56:32.811 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:32.811 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:32.811 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:32.811 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Prometheus container image", 2024-09-21T03:56:32.811 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:32.811 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:32.811 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:32.812 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:32.812 INFO:teuthology.orchestra.run.smithi039.stdout: "container_image_base": { 2024-09-21T03:56:32.812 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "container_image_base", 2024-09-21T03:56:32.812 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-09-21T03:56:32.812 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:32.812 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:32.812 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "quay.io/ceph/ceph", 2024-09-21T03:56:32.812 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:32.812 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:32.812 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:32.812 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Container image name, without the tag", 2024-09-21T03:56:32.812 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:32.812 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:32.812 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:32.812 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:32.813 INFO:teuthology.orchestra.run.smithi039.stdout: "container_image_elasticsearch": { 2024-09-21T03:56:32.813 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "container_image_elasticsearch", 2024-09-21T03:56:32.813 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-09-21T03:56:32.813 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:32.813 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-09-21T03:56:32.813 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-09-21T03:56:32.813 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:32.813 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:32.813 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:32.813 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "elasticsearch container image", 2024-09-21T03:56:32.813 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:32.813 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:32.813 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:32.814 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:32.814 INFO:teuthology.orchestra.run.smithi039.stdout: "container_image_grafana": { 2024-09-21T03:56:32.814 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "container_image_grafana", 2024-09-21T03:56:32.814 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-09-21T03:56:32.814 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:32.814 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-09-21T03:56:32.814 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "quay.io/ceph/grafana:10.4.8", 2024-09-21T03:56:32.814 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:32.814 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:32.814 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:32.814 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Prometheus container image", 2024-09-21T03:56:32.814 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:32.814 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:32.814 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:32.815 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:32.815 INFO:teuthology.orchestra.run.smithi039.stdout: "container_image_haproxy": { 2024-09-21T03:56:32.815 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "container_image_haproxy", 2024-09-21T03:56:32.815 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-09-21T03:56:32.815 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:32.815 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-09-21T03:56:32.815 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-09-21T03:56:32.815 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:32.815 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:32.815 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:32.815 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "HAproxy container image", 2024-09-21T03:56:32.815 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:32.815 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:32.815 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:32.815 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:32.816 INFO:teuthology.orchestra.run.smithi039.stdout: "container_image_jaeger_agent": { 2024-09-21T03:56:32.816 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "container_image_jaeger_agent", 2024-09-21T03:56:32.816 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-09-21T03:56:32.816 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:32.816 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-09-21T03:56:32.816 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-09-21T03:56:32.816 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:32.816 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:32.816 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:32.816 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Jaeger agent container image", 2024-09-21T03:56:32.816 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:32.816 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:32.816 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:32.816 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:32.817 INFO:teuthology.orchestra.run.smithi039.stdout: "container_image_jaeger_collector": { 2024-09-21T03:56:32.817 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "container_image_jaeger_collector", 2024-09-21T03:56:32.817 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-09-21T03:56:32.817 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:32.817 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-09-21T03:56:32.817 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-09-21T03:56:32.817 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:32.817 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:32.817 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:32.817 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Jaeger collector container image", 2024-09-21T03:56:32.817 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:32.817 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:32.817 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:32.817 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:32.817 INFO:teuthology.orchestra.run.smithi039.stdout: "container_image_jaeger_query": { 2024-09-21T03:56:32.818 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "container_image_jaeger_query", 2024-09-21T03:56:32.818 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-09-21T03:56:32.818 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:32.818 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-09-21T03:56:32.818 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-09-21T03:56:32.818 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:32.818 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:32.818 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:32.818 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Jaeger query container image", 2024-09-21T03:56:32.818 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:32.818 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:32.818 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:32.818 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:32.818 INFO:teuthology.orchestra.run.smithi039.stdout: "container_image_keepalived": { 2024-09-21T03:56:32.818 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "container_image_keepalived", 2024-09-21T03:56:32.819 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-09-21T03:56:32.819 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:32.819 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-09-21T03:56:32.819 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-09-21T03:56:32.819 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:32.819 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:32.819 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:32.819 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Keepalived container image", 2024-09-21T03:56:32.819 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:32.819 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:32.819 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:32.819 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:32.819 INFO:teuthology.orchestra.run.smithi039.stdout: "container_image_loki": { 2024-09-21T03:56:32.819 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "container_image_loki", 2024-09-21T03:56:32.819 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-09-21T03:56:32.820 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:32.820 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-09-21T03:56:32.820 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "docker.io/grafana/loki:3.0.0", 2024-09-21T03:56:32.820 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:32.820 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:32.820 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:32.820 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Loki container image", 2024-09-21T03:56:32.820 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:32.820 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:32.820 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:32.820 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:32.820 INFO:teuthology.orchestra.run.smithi039.stdout: "container_image_nginx": { 2024-09-21T03:56:32.820 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "container_image_nginx", 2024-09-21T03:56:32.820 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-09-21T03:56:32.820 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:32.821 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-09-21T03:56:32.821 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "quay.io/ceph/nginx:sclorg-nginx-126", 2024-09-21T03:56:32.821 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:32.821 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:32.821 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:32.821 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Nginx container image", 2024-09-21T03:56:32.821 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:32.821 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:32.821 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:32.821 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:32.821 INFO:teuthology.orchestra.run.smithi039.stdout: "container_image_node_exporter": { 2024-09-21T03:56:32.821 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "container_image_node_exporter", 2024-09-21T03:56:32.821 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-09-21T03:56:32.821 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:32.821 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-09-21T03:56:32.822 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.7.0", 2024-09-21T03:56:32.822 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:32.822 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:32.822 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:32.822 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Prometheus container image", 2024-09-21T03:56:32.822 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:32.822 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:32.822 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:32.822 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:32.822 INFO:teuthology.orchestra.run.smithi039.stdout: "container_image_nvmeof": { 2024-09-21T03:56:32.822 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "container_image_nvmeof", 2024-09-21T03:56:32.822 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-09-21T03:56:32.822 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:32.823 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-09-21T03:56:32.823 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.17", 2024-09-21T03:56:32.823 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:32.823 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:32.823 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:32.823 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Nvme-of container image", 2024-09-21T03:56:32.823 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:32.823 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:32.823 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:32.823 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:32.823 INFO:teuthology.orchestra.run.smithi039.stdout: "container_image_oauth2_proxy": { 2024-09-21T03:56:32.823 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "container_image_oauth2_proxy", 2024-09-21T03:56:32.823 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-09-21T03:56:32.823 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:32.823 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-09-21T03:56:32.824 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "quay.io/oauth2-proxy/oauth2-proxy:v7.6.0", 2024-09-21T03:56:32.824 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:32.824 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:32.824 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:32.824 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "oauth2-proxy container image", 2024-09-21T03:56:32.824 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:32.824 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:32.824 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:32.824 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:32.824 INFO:teuthology.orchestra.run.smithi039.stdout: "container_image_prometheus": { 2024-09-21T03:56:32.824 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "container_image_prometheus", 2024-09-21T03:56:32.824 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-09-21T03:56:32.824 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:32.824 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-09-21T03:56:32.824 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "quay.io/prometheus/prometheus:v2.51.0", 2024-09-21T03:56:32.825 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:32.825 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:32.825 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:32.825 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Prometheus container image", 2024-09-21T03:56:32.825 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:32.825 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:32.825 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:32.825 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:32.825 INFO:teuthology.orchestra.run.smithi039.stdout: "container_image_promtail": { 2024-09-21T03:56:32.825 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "container_image_promtail", 2024-09-21T03:56:32.825 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-09-21T03:56:32.825 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:32.825 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-09-21T03:56:32.826 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "docker.io/grafana/promtail:3.0.0", 2024-09-21T03:56:32.826 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:32.826 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:32.826 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:32.826 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Promtail container image", 2024-09-21T03:56:32.826 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:32.826 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:32.826 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:32.826 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:32.826 INFO:teuthology.orchestra.run.smithi039.stdout: "container_image_samba": { 2024-09-21T03:56:32.826 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "container_image_samba", 2024-09-21T03:56:32.826 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-09-21T03:56:32.826 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:32.826 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-09-21T03:56:32.827 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-09-21T03:56:32.827 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:32.827 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:32.827 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:32.827 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Samba/SMB container image", 2024-09-21T03:56:32.827 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:32.827 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:32.827 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:32.827 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:32.827 INFO:teuthology.orchestra.run.smithi039.stdout: "container_image_samba_metrics": { 2024-09-21T03:56:32.827 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "container_image_samba_metrics", 2024-09-21T03:56:32.827 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-09-21T03:56:32.827 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:32.827 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-09-21T03:56:32.827 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "quay.io/samba.org/samba-metrics:latest", 2024-09-21T03:56:32.828 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:32.828 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:32.828 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:32.828 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Samba/SMB metrics exporter container image", 2024-09-21T03:56:32.828 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:32.828 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:32.828 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:32.828 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:32.828 INFO:teuthology.orchestra.run.smithi039.stdout: "container_image_snmp_gateway": { 2024-09-21T03:56:32.828 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "container_image_snmp_gateway", 2024-09-21T03:56:32.828 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-09-21T03:56:32.828 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:32.828 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-09-21T03:56:32.828 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-09-21T03:56:32.828 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:32.829 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:32.829 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:32.829 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "SNMP Gateway container image", 2024-09-21T03:56:32.829 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:32.829 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:32.829 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:32.829 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:32.829 INFO:teuthology.orchestra.run.smithi039.stdout: "container_init": { 2024-09-21T03:56:32.829 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "container_init", 2024-09-21T03:56:32.829 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-09-21T03:56:32.829 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:32.829 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-09-21T03:56:32.829 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2024-09-21T03:56:32.829 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:32.829 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:32.830 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:32.830 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Run podman/docker with `--init`", 2024-09-21T03:56:32.830 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:32.830 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:32.830 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:32.830 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:32.830 INFO:teuthology.orchestra.run.smithi039.stdout: "daemon_cache_timeout": { 2024-09-21T03:56:32.830 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "daemon_cache_timeout", 2024-09-21T03:56:32.830 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "secs", 2024-09-21T03:56:32.830 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:32.830 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-09-21T03:56:32.830 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "600", 2024-09-21T03:56:32.830 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:32.830 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:32.830 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:32.831 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-09-21T03:56:32.831 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:32.831 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:32.831 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:32.831 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:32.831 INFO:teuthology.orchestra.run.smithi039.stdout: "default_cephadm_command_timeout": { 2024-09-21T03:56:32.831 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "default_cephadm_command_timeout", 2024-09-21T03:56:32.831 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-09-21T03:56:32.831 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:32.831 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-09-21T03:56:32.831 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "900", 2024-09-21T03:56:32.831 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:32.831 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:32.831 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:32.831 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-09-21T03:56:32.832 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:32.832 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:32.832 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:32.832 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:32.832 INFO:teuthology.orchestra.run.smithi039.stdout: "default_registry": { 2024-09-21T03:56:32.832 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "default_registry", 2024-09-21T03:56:32.832 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-09-21T03:56:32.832 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:32.832 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-09-21T03:56:32.832 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "docker.io", 2024-09-21T03:56:32.832 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:32.832 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:32.832 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:32.832 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-09-21T03:56:32.833 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:32.833 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:32.833 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:32.833 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:32.833 INFO:teuthology.orchestra.run.smithi039.stdout: "device_cache_timeout": { 2024-09-21T03:56:32.833 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "device_cache_timeout", 2024-09-21T03:56:32.833 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "secs", 2024-09-21T03:56:32.833 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:32.833 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-09-21T03:56:32.833 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "1800", 2024-09-21T03:56:32.833 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:32.833 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:32.833 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:32.833 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "seconds to cache device inventory", 2024-09-21T03:56:32.833 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:32.833 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:32.834 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:32.834 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:32.834 INFO:teuthology.orchestra.run.smithi039.stdout: "device_enhanced_scan": { 2024-09-21T03:56:32.834 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "device_enhanced_scan", 2024-09-21T03:56:32.834 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-09-21T03:56:32.834 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:32.834 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-09-21T03:56:32.834 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-09-21T03:56:32.834 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:32.834 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:32.834 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:32.834 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Use libstoragemgmt during device scans", 2024-09-21T03:56:32.834 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:32.834 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:32.834 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:32.835 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:32.835 INFO:teuthology.orchestra.run.smithi039.stdout: "facts_cache_timeout": { 2024-09-21T03:56:32.835 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "facts_cache_timeout", 2024-09-21T03:56:32.835 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "secs", 2024-09-21T03:56:32.835 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:32.835 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-09-21T03:56:32.835 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "60", 2024-09-21T03:56:32.835 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:32.835 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:32.835 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:32.835 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "seconds to cache host facts data", 2024-09-21T03:56:32.835 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:32.835 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:32.835 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:32.835 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:32.836 INFO:teuthology.orchestra.run.smithi039.stdout: "grafana_dashboards_path": { 2024-09-21T03:56:32.836 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "grafana_dashboards_path", 2024-09-21T03:56:32.836 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-09-21T03:56:32.836 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:32.836 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-09-21T03:56:32.836 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-09-21T03:56:32.836 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:32.836 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:32.836 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:32.836 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-09-21T03:56:32.836 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:32.836 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:32.836 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:32.836 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:32.837 INFO:teuthology.orchestra.run.smithi039.stdout: "host_check_interval": { 2024-09-21T03:56:32.837 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "host_check_interval", 2024-09-21T03:56:32.837 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "secs", 2024-09-21T03:56:32.837 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:32.837 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-09-21T03:56:32.837 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "600", 2024-09-21T03:56:32.837 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:32.837 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:32.837 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:32.837 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "how frequently to perform a host check", 2024-09-21T03:56:32.837 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:32.837 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:32.837 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:32.837 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:32.838 INFO:teuthology.orchestra.run.smithi039.stdout: "hw_monitoring": { 2024-09-21T03:56:32.838 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "hw_monitoring", 2024-09-21T03:56:32.838 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-09-21T03:56:32.838 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:32.838 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-09-21T03:56:32.838 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-09-21T03:56:32.838 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:32.838 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:32.838 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:32.838 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-09-21T03:56:32.838 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:32.838 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:32.838 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:32.838 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:32.838 INFO:teuthology.orchestra.run.smithi039.stdout: "inventory_list_all": { 2024-09-21T03:56:32.839 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "inventory_list_all", 2024-09-21T03:56:32.839 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-09-21T03:56:32.839 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:32.839 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-09-21T03:56:32.839 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-09-21T03:56:32.839 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:32.839 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:32.839 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:32.839 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-09-21T03:56:32.839 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:32.839 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:32.839 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:32.839 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:32.839 INFO:teuthology.orchestra.run.smithi039.stdout: "log_level": { 2024-09-21T03:56:32.840 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_level", 2024-09-21T03:56:32.840 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-09-21T03:56:32.840 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:32.840 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:32.840 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-09-21T03:56:32.840 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:32.840 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:32.840 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-09-21T03:56:32.840 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-09-21T03:56:32.840 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-09-21T03:56:32.840 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-09-21T03:56:32.840 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-09-21T03:56:32.840 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-09-21T03:56:32.840 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-09-21T03:56:32.840 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-09-21T03:56:32.841 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:32.841 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:32.841 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:32.841 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:32.841 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:32.841 INFO:teuthology.orchestra.run.smithi039.stdout: "log_refresh_metadata": { 2024-09-21T03:56:32.841 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_refresh_metadata", 2024-09-21T03:56:32.841 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-09-21T03:56:32.841 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:32.841 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-09-21T03:56:32.841 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-09-21T03:56:32.841 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:32.841 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:32.841 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:32.841 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-09-21T03:56:32.842 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:32.842 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:32.842 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:32.842 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:32.842 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster": { 2024-09-21T03:56:32.842 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster", 2024-09-21T03:56:32.842 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-09-21T03:56:32.842 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:32.842 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-09-21T03:56:32.842 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2024-09-21T03:56:32.842 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:32.842 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:32.842 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:32.842 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-09-21T03:56:32.843 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:32.843 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:32.843 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:32.843 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:32.843 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster_level": { 2024-09-21T03:56:32.843 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster_level", 2024-09-21T03:56:32.843 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-09-21T03:56:32.843 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:32.843 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:32.843 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "info", 2024-09-21T03:56:32.843 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:32.843 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:32.843 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-09-21T03:56:32.843 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-09-21T03:56:32.843 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-09-21T03:56:32.844 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-09-21T03:56:32.844 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-09-21T03:56:32.844 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-09-21T03:56:32.844 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-09-21T03:56:32.844 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-09-21T03:56:32.844 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:32.844 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:32.844 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:32.844 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:32.844 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:32.844 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_file": { 2024-09-21T03:56:32.844 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_file", 2024-09-21T03:56:32.844 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-09-21T03:56:32.844 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:32.844 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:32.844 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-09-21T03:56:32.845 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:32.845 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:32.845 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:32.845 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:32.845 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:32.845 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:32.845 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:32.845 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:32.845 INFO:teuthology.orchestra.run.smithi039.stdout: "manage_etc_ceph_ceph_conf": { 2024-09-21T03:56:32.845 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-09-21T03:56:32.845 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-09-21T03:56:32.845 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:32.845 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-09-21T03:56:32.845 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-09-21T03:56:32.845 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:32.846 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:32.846 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:32.846 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-09-21T03:56:32.846 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:32.846 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:32.846 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:32.846 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:32.846 INFO:teuthology.orchestra.run.smithi039.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-09-21T03:56:32.846 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-09-21T03:56:32.846 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-09-21T03:56:32.846 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:32.846 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-09-21T03:56:32.846 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "*", 2024-09-21T03:56:32.846 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:32.847 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:32.847 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:32.847 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-09-21T03:56:32.847 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:32.847 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:32.847 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:32.847 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:32.847 INFO:teuthology.orchestra.run.smithi039.stdout: "max_count_per_host": { 2024-09-21T03:56:32.847 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "max_count_per_host", 2024-09-21T03:56:32.847 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-09-21T03:56:32.847 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:32.847 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-09-21T03:56:32.847 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "10", 2024-09-21T03:56:32.847 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:32.847 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:32.847 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:32.848 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "max number of daemons per service per host", 2024-09-21T03:56:32.848 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:32.848 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:32.849 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:32.849 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:32.849 INFO:teuthology.orchestra.run.smithi039.stdout: "max_osd_draining_count": { 2024-09-21T03:56:32.849 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "max_osd_draining_count", 2024-09-21T03:56:32.850 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-09-21T03:56:32.850 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:32.850 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-09-21T03:56:32.850 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "10", 2024-09-21T03:56:32.850 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:32.850 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:32.850 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:32.850 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-09-21T03:56:32.850 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:32.850 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:32.851 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:32.851 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:32.851 INFO:teuthology.orchestra.run.smithi039.stdout: "migration_current": { 2024-09-21T03:56:32.851 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "migration_current", 2024-09-21T03:56:32.851 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-09-21T03:56:32.851 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:32.851 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-09-21T03:56:32.851 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-09-21T03:56:32.851 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:32.851 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:32.851 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:32.851 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "internal - do not modify", 2024-09-21T03:56:32.851 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:32.851 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:32.851 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:32.852 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:32.852 INFO:teuthology.orchestra.run.smithi039.stdout: "mode": { 2024-09-21T03:56:32.852 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "mode", 2024-09-21T03:56:32.852 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-09-21T03:56:32.852 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:32.852 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-09-21T03:56:32.852 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "root", 2024-09-21T03:56:32.852 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:32.852 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:32.852 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-09-21T03:56:32.852 INFO:teuthology.orchestra.run.smithi039.stdout: "cephadm-package", 2024-09-21T03:56:32.852 INFO:teuthology.orchestra.run.smithi039.stdout: "root" 2024-09-21T03:56:32.852 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-09-21T03:56:32.852 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "mode for remote execution of cephadm", 2024-09-21T03:56:32.853 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:32.853 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:32.853 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:32.853 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:32.853 INFO:teuthology.orchestra.run.smithi039.stdout: "oob_default_addr": { 2024-09-21T03:56:32.853 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "oob_default_addr", 2024-09-21T03:56:32.853 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-09-21T03:56:32.853 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:32.853 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-09-21T03:56:32.853 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "169.254.1.1", 2024-09-21T03:56:32.853 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:32.853 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:32.853 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:32.853 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Default address for RedFish API (oob management).", 2024-09-21T03:56:32.853 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:32.854 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:32.854 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:32.854 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:32.854 INFO:teuthology.orchestra.run.smithi039.stdout: "prometheus_alerts_path": { 2024-09-21T03:56:32.854 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "prometheus_alerts_path", 2024-09-21T03:56:32.854 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-09-21T03:56:32.854 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:32.854 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-09-21T03:56:32.854 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-09-21T03:56:32.854 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:32.854 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:32.854 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:32.854 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-09-21T03:56:32.854 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:32.855 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:32.855 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:32.855 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:32.855 INFO:teuthology.orchestra.run.smithi039.stdout: "registry_insecure": { 2024-09-21T03:56:32.855 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "registry_insecure", 2024-09-21T03:56:32.855 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-09-21T03:56:32.855 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:32.855 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-09-21T03:56:32.855 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-09-21T03:56:32.855 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:32.855 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:32.855 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:32.855 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-09-21T03:56:32.855 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:32.856 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:32.856 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:32.856 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:32.856 INFO:teuthology.orchestra.run.smithi039.stdout: "registry_password": { 2024-09-21T03:56:32.856 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "registry_password", 2024-09-21T03:56:32.856 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-09-21T03:56:32.856 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:32.856 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-09-21T03:56:32.856 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-09-21T03:56:32.856 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:32.856 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:32.856 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:32.856 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-09-21T03:56:32.856 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:32.856 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:32.856 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:32.857 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:32.857 INFO:teuthology.orchestra.run.smithi039.stdout: "registry_url": { 2024-09-21T03:56:32.857 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "registry_url", 2024-09-21T03:56:32.857 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-09-21T03:56:32.857 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:32.857 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-09-21T03:56:32.857 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-09-21T03:56:32.857 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:32.857 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:32.857 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:32.857 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-09-21T03:56:32.857 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:32.857 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:32.857 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:32.857 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:32.858 INFO:teuthology.orchestra.run.smithi039.stdout: "registry_username": { 2024-09-21T03:56:32.858 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "registry_username", 2024-09-21T03:56:32.858 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-09-21T03:56:32.858 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:32.858 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-09-21T03:56:32.858 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-09-21T03:56:32.858 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:32.858 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:32.858 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:32.858 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-09-21T03:56:32.858 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:32.858 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:32.858 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:32.858 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:32.859 INFO:teuthology.orchestra.run.smithi039.stdout: "secure_monitoring_stack": { 2024-09-21T03:56:32.859 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "secure_monitoring_stack", 2024-09-21T03:56:32.859 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-09-21T03:56:32.859 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:32.859 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-09-21T03:56:32.859 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-09-21T03:56:32.859 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:32.859 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:32.859 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:32.859 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-09-21T03:56:32.859 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:32.859 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:32.859 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:32.859 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:32.859 INFO:teuthology.orchestra.run.smithi039.stdout: "service_discovery_port": { 2024-09-21T03:56:32.860 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "service_discovery_port", 2024-09-21T03:56:32.860 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-09-21T03:56:32.860 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:32.860 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-09-21T03:56:32.860 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "8765", 2024-09-21T03:56:32.860 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:32.860 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:32.860 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:32.860 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "cephadm service discovery port", 2024-09-21T03:56:32.860 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:32.860 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:32.860 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:32.860 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:32.860 INFO:teuthology.orchestra.run.smithi039.stdout: "sqlite3_killpoint": { 2024-09-21T03:56:32.860 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:56:32.861 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-09-21T03:56:32.861 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "dev", 2024-09-21T03:56:32.861 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:32.861 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "0", 2024-09-21T03:56:32.861 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:32.861 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:32.861 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:32.861 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:32.861 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:32.861 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:32.861 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:32.861 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:32.861 INFO:teuthology.orchestra.run.smithi039.stdout: "ssh_config_file": { 2024-09-21T03:56:32.861 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "ssh_config_file", 2024-09-21T03:56:32.861 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-09-21T03:56:32.861 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:32.862 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-09-21T03:56:32.862 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-09-21T03:56:32.862 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:32.862 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:32.862 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:32.862 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-09-21T03:56:32.862 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:32.862 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:32.862 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:32.862 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:32.862 INFO:teuthology.orchestra.run.smithi039.stdout: "ssh_keepalive_count_max": { 2024-09-21T03:56:32.862 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "ssh_keepalive_count_max", 2024-09-21T03:56:32.862 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-09-21T03:56:32.863 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:32.863 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-09-21T03:56:32.863 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "3", 2024-09-21T03:56:32.863 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:32.863 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:32.863 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:32.863 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "How many times ssh connections can fail liveness checks before the host is marked offline", 2024-09-21T03:56:32.863 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:32.863 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:32.863 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:32.863 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:32.863 INFO:teuthology.orchestra.run.smithi039.stdout: "ssh_keepalive_interval": { 2024-09-21T03:56:32.863 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "ssh_keepalive_interval", 2024-09-21T03:56:32.863 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-09-21T03:56:32.863 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:32.864 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-09-21T03:56:32.864 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "7", 2024-09-21T03:56:32.864 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:32.864 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:32.864 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:32.864 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "How often ssh connections are checked for liveness", 2024-09-21T03:56:32.864 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:32.864 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:32.864 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:32.864 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:32.864 INFO:teuthology.orchestra.run.smithi039.stdout: "use_agent": { 2024-09-21T03:56:32.864 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "use_agent", 2024-09-21T03:56:32.864 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-09-21T03:56:32.864 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:32.865 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-09-21T03:56:32.865 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-09-21T03:56:32.865 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:32.865 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:32.865 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:32.865 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-09-21T03:56:32.865 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:32.865 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:32.865 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:32.865 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:32.865 INFO:teuthology.orchestra.run.smithi039.stdout: "use_repo_digest": { 2024-09-21T03:56:32.865 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "use_repo_digest", 2024-09-21T03:56:32.865 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-09-21T03:56:32.865 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:32.865 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-09-21T03:56:32.866 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2024-09-21T03:56:32.866 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:32.866 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:32.866 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:32.866 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-09-21T03:56:32.866 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:32.866 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:32.866 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:32.866 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:32.866 INFO:teuthology.orchestra.run.smithi039.stdout: "warn_on_failed_host_check": { 2024-09-21T03:56:32.866 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "warn_on_failed_host_check", 2024-09-21T03:56:32.866 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-09-21T03:56:32.866 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:32.866 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-09-21T03:56:32.867 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2024-09-21T03:56:32.867 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:32.867 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:32.867 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:32.867 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "raise a health warning if the host check fails", 2024-09-21T03:56:32.867 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:32.867 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:32.867 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:32.867 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:32.867 INFO:teuthology.orchestra.run.smithi039.stdout: "warn_on_stray_daemons": { 2024-09-21T03:56:32.867 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "warn_on_stray_daemons", 2024-09-21T03:56:32.867 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-09-21T03:56:32.867 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:32.867 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-09-21T03:56:32.867 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2024-09-21T03:56:32.868 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:32.868 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:32.868 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:32.868 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-09-21T03:56:32.868 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:32.868 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:32.868 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:32.868 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:32.868 INFO:teuthology.orchestra.run.smithi039.stdout: "warn_on_stray_hosts": { 2024-09-21T03:56:32.868 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "warn_on_stray_hosts", 2024-09-21T03:56:32.868 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-09-21T03:56:32.868 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:32.868 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-09-21T03:56:32.868 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2024-09-21T03:56:32.868 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:32.869 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:32.869 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:32.869 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-09-21T03:56:32.869 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:32.869 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:32.869 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:32.869 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-21T03:56:32.869 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-21T03:56:32.869 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:32.869 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-21T03:56:32.869 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "crash", 2024-09-21T03:56:32.869 INFO:teuthology.orchestra.run.smithi039.stdout: "can_run": true, 2024-09-21T03:56:32.869 INFO:teuthology.orchestra.run.smithi039.stdout: "error_string": "", 2024-09-21T03:56:32.869 INFO:teuthology.orchestra.run.smithi039.stdout: "module_options": { 2024-09-21T03:56:32.870 INFO:teuthology.orchestra.run.smithi039.stdout: "log_level": { 2024-09-21T03:56:32.870 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_level", 2024-09-21T03:56:32.870 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-09-21T03:56:32.870 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:32.870 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:32.870 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-09-21T03:56:32.870 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:32.870 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:32.870 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-09-21T03:56:32.870 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-09-21T03:56:32.870 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-09-21T03:56:32.870 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-09-21T03:56:32.870 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-09-21T03:56:32.870 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-09-21T03:56:32.870 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-09-21T03:56:32.871 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-09-21T03:56:32.871 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:32.871 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:32.871 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:32.871 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:32.871 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:32.871 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster": { 2024-09-21T03:56:32.871 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster", 2024-09-21T03:56:32.871 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-09-21T03:56:32.871 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:32.871 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:32.871 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-09-21T03:56:32.871 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:32.871 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:32.871 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:32.872 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:32.872 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:32.872 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:32.872 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:32.872 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:32.872 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster_level": { 2024-09-21T03:56:32.872 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster_level", 2024-09-21T03:56:32.872 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-09-21T03:56:32.872 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:32.872 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:32.872 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "info", 2024-09-21T03:56:32.872 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:32.872 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:32.872 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-09-21T03:56:32.872 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-09-21T03:56:32.873 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-09-21T03:56:32.873 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-09-21T03:56:32.873 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-09-21T03:56:32.873 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-09-21T03:56:32.873 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-09-21T03:56:32.873 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-09-21T03:56:32.873 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:32.873 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:32.873 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:32.873 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:32.873 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:32.873 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_file": { 2024-09-21T03:56:32.873 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_file", 2024-09-21T03:56:32.873 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-09-21T03:56:32.874 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:32.874 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:32.874 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-09-21T03:56:32.874 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:32.874 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:32.874 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:32.874 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:32.874 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:32.874 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:32.874 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:32.874 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:32.874 INFO:teuthology.orchestra.run.smithi039.stdout: "retain_interval": { 2024-09-21T03:56:32.874 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "retain_interval", 2024-09-21T03:56:32.875 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "secs", 2024-09-21T03:56:32.875 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:32.875 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:32.875 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "31536000", 2024-09-21T03:56:32.875 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:32.875 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:32.875 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:32.875 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "how long to retain crashes before pruning them", 2024-09-21T03:56:32.875 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:32.875 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:32.875 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:32.875 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:32.875 INFO:teuthology.orchestra.run.smithi039.stdout: "sqlite3_killpoint": { 2024-09-21T03:56:32.875 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:56:32.875 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-09-21T03:56:32.876 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "dev", 2024-09-21T03:56:32.876 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:32.876 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "0", 2024-09-21T03:56:32.876 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:32.876 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:32.876 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:32.876 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:32.876 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:32.876 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:32.876 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:32.876 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:32.876 INFO:teuthology.orchestra.run.smithi039.stdout: "warn_recent_interval": { 2024-09-21T03:56:32.876 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "warn_recent_interval", 2024-09-21T03:56:32.876 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "secs", 2024-09-21T03:56:32.876 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:32.877 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:32.877 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "1209600", 2024-09-21T03:56:32.877 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:32.877 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:32.877 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:32.877 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "time interval in which to warn about recent crashes", 2024-09-21T03:56:32.877 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:32.877 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:32.877 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:32.877 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-21T03:56:32.877 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-21T03:56:32.877 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:32.877 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-21T03:56:32.877 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "dashboard", 2024-09-21T03:56:32.877 INFO:teuthology.orchestra.run.smithi039.stdout: "can_run": true, 2024-09-21T03:56:32.878 INFO:teuthology.orchestra.run.smithi039.stdout: "error_string": "", 2024-09-21T03:56:32.878 INFO:teuthology.orchestra.run.smithi039.stdout: "module_options": { 2024-09-21T03:56:32.878 INFO:teuthology.orchestra.run.smithi039.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-09-21T03:56:32.878 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-09-21T03:56:32.878 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-09-21T03:56:32.878 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:32.878 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-09-21T03:56:32.878 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "10", 2024-09-21T03:56:32.878 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:32.878 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:32.878 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:32.878 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:32.878 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:32.879 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:32.879 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:32.879 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:32.879 INFO:teuthology.orchestra.run.smithi039.stdout: "ALERTMANAGER_API_HOST": { 2024-09-21T03:56:32.879 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "ALERTMANAGER_API_HOST", 2024-09-21T03:56:32.879 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-09-21T03:56:32.879 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:32.879 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-09-21T03:56:32.879 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-09-21T03:56:32.879 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:32.879 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:32.879 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:32.879 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:32.879 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:32.880 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:32.880 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:32.880 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:32.880 INFO:teuthology.orchestra.run.smithi039.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-09-21T03:56:32.880 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-09-21T03:56:32.880 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-09-21T03:56:32.880 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:32.880 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-09-21T03:56:32.880 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2024-09-21T03:56:32.880 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:32.880 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:32.880 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:32.880 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:32.880 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:32.880 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:32.881 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:32.881 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:32.881 INFO:teuthology.orchestra.run.smithi039.stdout: "AUDIT_API_ENABLED": { 2024-09-21T03:56:32.881 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "AUDIT_API_ENABLED", 2024-09-21T03:56:32.881 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-09-21T03:56:32.881 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:32.881 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-09-21T03:56:32.881 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-09-21T03:56:32.881 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:32.881 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:32.881 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:32.881 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:32.881 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:32.881 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:32.881 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:32.882 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:32.882 INFO:teuthology.orchestra.run.smithi039.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-09-21T03:56:32.882 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-09-21T03:56:32.882 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-09-21T03:56:32.882 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:32.882 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-09-21T03:56:32.882 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2024-09-21T03:56:32.882 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:32.882 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:32.882 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:32.882 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:32.882 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:32.882 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:32.882 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:32.883 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:32.883 INFO:teuthology.orchestra.run.smithi039.stdout: "ENABLE_BROWSABLE_API": { 2024-09-21T03:56:32.883 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "ENABLE_BROWSABLE_API", 2024-09-21T03:56:32.883 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-09-21T03:56:32.883 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:32.883 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-09-21T03:56:32.883 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2024-09-21T03:56:32.883 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:32.883 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:32.883 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:32.883 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:32.883 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:32.883 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:32.883 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:32.883 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:32.884 INFO:teuthology.orchestra.run.smithi039.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-09-21T03:56:32.884 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-09-21T03:56:32.884 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-09-21T03:56:32.884 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:32.884 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-09-21T03:56:32.884 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2024-09-21T03:56:32.884 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:32.884 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:32.884 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:32.884 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:32.884 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:32.884 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:32.884 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:32.884 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:32.885 INFO:teuthology.orchestra.run.smithi039.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-09-21T03:56:32.885 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-09-21T03:56:32.885 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-09-21T03:56:32.885 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:32.885 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-09-21T03:56:32.885 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2024-09-21T03:56:32.885 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:32.885 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:32.885 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:32.885 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:32.885 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:32.885 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:32.885 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:32.885 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:32.885 INFO:teuthology.orchestra.run.smithi039.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-09-21T03:56:32.886 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-09-21T03:56:32.886 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-09-21T03:56:32.886 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:32.886 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-09-21T03:56:32.886 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2024-09-21T03:56:32.886 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:32.886 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:32.886 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:32.886 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:32.886 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:32.886 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:32.886 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:32.886 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:32.886 INFO:teuthology.orchestra.run.smithi039.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-09-21T03:56:32.887 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-09-21T03:56:32.887 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-09-21T03:56:32.887 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:32.887 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-09-21T03:56:32.887 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2024-09-21T03:56:32.887 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:32.887 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:32.887 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:32.887 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:32.887 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:32.887 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:32.887 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:32.887 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:32.888 INFO:teuthology.orchestra.run.smithi039.stdout: "FEATURE_TOGGLE_NFS": { 2024-09-21T03:56:32.888 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-09-21T03:56:32.888 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-09-21T03:56:32.888 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:32.888 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-09-21T03:56:32.888 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2024-09-21T03:56:32.888 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:32.888 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:32.888 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:32.888 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:32.888 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:32.888 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:32.888 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:32.888 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:32.888 INFO:teuthology.orchestra.run.smithi039.stdout: "FEATURE_TOGGLE_RBD": { 2024-09-21T03:56:32.889 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-09-21T03:56:32.889 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-09-21T03:56:32.889 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:32.889 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-09-21T03:56:32.889 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2024-09-21T03:56:32.889 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:32.889 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:32.889 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:32.889 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:32.889 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:32.889 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:32.889 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:32.889 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:32.889 INFO:teuthology.orchestra.run.smithi039.stdout: "FEATURE_TOGGLE_RGW": { 2024-09-21T03:56:32.889 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-09-21T03:56:32.890 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-09-21T03:56:32.890 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:32.890 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-09-21T03:56:32.890 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2024-09-21T03:56:32.890 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:32.890 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:32.890 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:32.890 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:32.890 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:32.890 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:32.890 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:32.890 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:32.890 INFO:teuthology.orchestra.run.smithi039.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-09-21T03:56:32.890 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-09-21T03:56:32.891 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-09-21T03:56:32.891 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:32.891 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-09-21T03:56:32.891 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-09-21T03:56:32.891 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:32.891 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:32.891 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:32.891 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:32.891 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:32.891 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:32.891 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:32.891 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:32.891 INFO:teuthology.orchestra.run.smithi039.stdout: "GRAFANA_API_PASSWORD": { 2024-09-21T03:56:32.891 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "GRAFANA_API_PASSWORD", 2024-09-21T03:56:32.891 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-09-21T03:56:32.892 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:32.892 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-09-21T03:56:32.892 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "admin", 2024-09-21T03:56:32.892 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:32.892 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:32.892 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:32.892 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:32.892 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:32.892 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:32.892 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:32.892 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:32.892 INFO:teuthology.orchestra.run.smithi039.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-09-21T03:56:32.892 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-09-21T03:56:32.892 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-09-21T03:56:32.892 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:32.893 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-09-21T03:56:32.893 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2024-09-21T03:56:32.893 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:32.893 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:32.893 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:32.893 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:32.893 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:32.893 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:32.893 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:32.893 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:32.893 INFO:teuthology.orchestra.run.smithi039.stdout: "GRAFANA_API_URL": { 2024-09-21T03:56:32.893 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "GRAFANA_API_URL", 2024-09-21T03:56:32.893 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-09-21T03:56:32.893 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:32.893 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-09-21T03:56:32.894 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-09-21T03:56:32.894 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:32.894 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:32.894 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:32.894 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:32.894 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:32.894 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:32.894 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:32.894 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:32.894 INFO:teuthology.orchestra.run.smithi039.stdout: "GRAFANA_API_USERNAME": { 2024-09-21T03:56:32.894 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "GRAFANA_API_USERNAME", 2024-09-21T03:56:32.894 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-09-21T03:56:32.894 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:32.894 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-09-21T03:56:32.894 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "admin", 2024-09-21T03:56:32.895 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:32.895 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:32.895 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:32.895 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:32.895 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:32.895 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:32.895 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:32.895 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:32.895 INFO:teuthology.orchestra.run.smithi039.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-09-21T03:56:32.895 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-09-21T03:56:32.895 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-09-21T03:56:32.895 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:32.895 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-09-21T03:56:32.895 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-09-21T03:56:32.895 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:32.896 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:32.896 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:32.896 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:32.896 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:32.896 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:32.896 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:32.896 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:32.896 INFO:teuthology.orchestra.run.smithi039.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-09-21T03:56:32.896 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-09-21T03:56:32.896 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-09-21T03:56:32.896 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:32.896 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-09-21T03:56:32.896 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-09-21T03:56:32.896 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:32.896 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:32.897 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:32.897 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:32.897 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:32.897 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:32.897 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:32.897 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:32.897 INFO:teuthology.orchestra.run.smithi039.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-09-21T03:56:32.897 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-09-21T03:56:32.897 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-09-21T03:56:32.897 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:32.897 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-09-21T03:56:32.897 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2024-09-21T03:56:32.897 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:32.897 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:32.897 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:32.897 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:32.898 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:32.898 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:32.898 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:32.898 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:32.898 INFO:teuthology.orchestra.run.smithi039.stdout: "ISSUE_TRACKER_API_KEY": { 2024-09-21T03:56:32.898 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-09-21T03:56:32.898 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-09-21T03:56:32.898 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:32.898 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-09-21T03:56:32.898 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-09-21T03:56:32.898 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:32.898 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:32.898 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:32.898 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:32.899 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:32.899 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:32.899 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:32.899 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:32.899 INFO:teuthology.orchestra.run.smithi039.stdout: "MANAGED_BY_CLUSTERS": { 2024-09-21T03:56:32.899 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "MANAGED_BY_CLUSTERS", 2024-09-21T03:56:32.899 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-09-21T03:56:32.899 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:32.899 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-09-21T03:56:32.899 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "[]", 2024-09-21T03:56:32.899 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:32.899 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:32.899 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:32.899 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:32.900 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:32.900 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:32.900 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:32.900 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:32.900 INFO:teuthology.orchestra.run.smithi039.stdout: "MULTICLUSTER_CONFIG": { 2024-09-21T03:56:32.900 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "MULTICLUSTER_CONFIG", 2024-09-21T03:56:32.900 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-09-21T03:56:32.900 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:32.900 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-09-21T03:56:32.900 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "{}", 2024-09-21T03:56:32.900 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:32.900 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:32.900 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:32.900 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:32.900 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:32.901 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:32.901 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:32.901 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:32.901 INFO:teuthology.orchestra.run.smithi039.stdout: "PROMETHEUS_API_HOST": { 2024-09-21T03:56:32.901 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "PROMETHEUS_API_HOST", 2024-09-21T03:56:32.901 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-09-21T03:56:32.901 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:32.901 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-09-21T03:56:32.901 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-09-21T03:56:32.901 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:32.901 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:32.901 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:32.901 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:32.901 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:32.902 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:32.902 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:32.902 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:32.902 INFO:teuthology.orchestra.run.smithi039.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-09-21T03:56:32.902 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-09-21T03:56:32.902 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-09-21T03:56:32.902 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:32.902 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-09-21T03:56:32.902 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2024-09-21T03:56:32.902 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:32.902 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:32.902 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:32.902 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:32.902 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:32.903 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:32.903 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:32.903 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:32.903 INFO:teuthology.orchestra.run.smithi039.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-09-21T03:56:32.903 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-09-21T03:56:32.903 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-09-21T03:56:32.903 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:32.903 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-09-21T03:56:32.903 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-09-21T03:56:32.903 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:32.903 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:32.903 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:32.903 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:32.903 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:32.903 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:32.904 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:32.904 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:32.904 INFO:teuthology.orchestra.run.smithi039.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-09-21T03:56:32.904 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-09-21T03:56:32.904 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-09-21T03:56:32.904 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:32.904 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-09-21T03:56:32.904 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-09-21T03:56:32.904 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:32.904 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:32.904 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:32.904 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:32.904 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:32.904 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:32.904 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:32.905 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:32.905 INFO:teuthology.orchestra.run.smithi039.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-09-21T03:56:32.905 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-09-21T03:56:32.905 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-09-21T03:56:32.905 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:32.905 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-09-21T03:56:32.905 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2024-09-21T03:56:32.905 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:32.905 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:32.905 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:32.905 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:32.905 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:32.905 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:32.905 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:32.905 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:32.906 INFO:teuthology.orchestra.run.smithi039.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-09-21T03:56:32.906 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-09-21T03:56:32.906 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-09-21T03:56:32.906 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:32.906 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-09-21T03:56:32.906 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2024-09-21T03:56:32.906 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:32.906 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:32.906 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:32.906 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:32.906 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:32.906 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:32.906 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:32.906 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:32.907 INFO:teuthology.orchestra.run.smithi039.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-09-21T03:56:32.907 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-09-21T03:56:32.907 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-09-21T03:56:32.907 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:32.907 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-09-21T03:56:32.907 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-09-21T03:56:32.907 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:32.907 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:32.907 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:32.907 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:32.907 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:32.907 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:32.907 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:32.907 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:32.907 INFO:teuthology.orchestra.run.smithi039.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-09-21T03:56:32.908 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-09-21T03:56:32.908 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-09-21T03:56:32.908 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:32.908 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-09-21T03:56:32.908 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-09-21T03:56:32.908 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:32.908 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:32.908 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:32.908 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:32.908 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:32.908 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:32.908 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:32.908 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:32.908 INFO:teuthology.orchestra.run.smithi039.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-09-21T03:56:32.908 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-09-21T03:56:32.909 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-09-21T03:56:32.909 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:32.909 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-09-21T03:56:32.909 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-09-21T03:56:32.909 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:32.909 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:32.909 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:32.909 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:32.909 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:32.909 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:32.909 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:32.909 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:32.909 INFO:teuthology.orchestra.run.smithi039.stdout: "PWD_POLICY_ENABLED": { 2024-09-21T03:56:32.909 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "PWD_POLICY_ENABLED", 2024-09-21T03:56:32.910 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-09-21T03:56:32.910 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:32.910 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-09-21T03:56:32.910 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2024-09-21T03:56:32.910 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:32.910 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:32.910 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:32.910 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:32.910 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:32.910 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:32.910 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:32.910 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:32.910 INFO:teuthology.orchestra.run.smithi039.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-09-21T03:56:32.910 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-09-21T03:56:32.910 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-09-21T03:56:32.911 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:32.911 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-09-21T03:56:32.911 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-09-21T03:56:32.911 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:32.911 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:32.911 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:32.911 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:32.911 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:32.911 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:32.911 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:32.911 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:32.911 INFO:teuthology.orchestra.run.smithi039.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-09-21T03:56:32.911 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-09-21T03:56:32.911 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-09-21T03:56:32.911 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:32.912 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-09-21T03:56:32.912 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "10", 2024-09-21T03:56:32.912 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:32.912 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:32.912 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:32.912 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:32.912 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:32.912 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:32.912 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:32.912 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:32.912 INFO:teuthology.orchestra.run.smithi039.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-09-21T03:56:32.912 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-09-21T03:56:32.912 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-09-21T03:56:32.912 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:32.913 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-09-21T03:56:32.913 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "8", 2024-09-21T03:56:32.913 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:32.913 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:32.913 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:32.913 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:32.913 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:32.913 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:32.913 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:32.913 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:32.913 INFO:teuthology.orchestra.run.smithi039.stdout: "REST_REQUESTS_TIMEOUT": { 2024-09-21T03:56:32.913 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-09-21T03:56:32.913 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-09-21T03:56:32.913 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:32.914 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-09-21T03:56:32.914 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "45", 2024-09-21T03:56:32.914 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:32.914 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:32.914 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:32.914 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:32.914 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:32.914 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:32.914 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:32.914 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:32.914 INFO:teuthology.orchestra.run.smithi039.stdout: "RGW_API_ACCESS_KEY": { 2024-09-21T03:56:32.914 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "RGW_API_ACCESS_KEY", 2024-09-21T03:56:32.914 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-09-21T03:56:32.914 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:32.914 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-09-21T03:56:32.915 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-09-21T03:56:32.915 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:32.915 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:32.915 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:32.915 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:32.915 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:32.915 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:32.915 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:32.915 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:32.915 INFO:teuthology.orchestra.run.smithi039.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-09-21T03:56:32.915 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-09-21T03:56:32.915 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-09-21T03:56:32.915 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:32.915 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-09-21T03:56:32.916 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "admin", 2024-09-21T03:56:32.916 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:32.916 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:32.916 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:32.916 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:32.916 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:32.916 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:32.916 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:32.916 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:32.916 INFO:teuthology.orchestra.run.smithi039.stdout: "RGW_API_SECRET_KEY": { 2024-09-21T03:56:32.916 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "RGW_API_SECRET_KEY", 2024-09-21T03:56:32.916 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-09-21T03:56:32.916 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:32.916 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-09-21T03:56:32.916 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-09-21T03:56:32.917 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:32.917 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:32.917 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:32.917 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:32.917 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:32.917 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:32.917 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:32.917 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:32.917 INFO:teuthology.orchestra.run.smithi039.stdout: "RGW_API_SSL_VERIFY": { 2024-09-21T03:56:32.917 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "RGW_API_SSL_VERIFY", 2024-09-21T03:56:32.917 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-09-21T03:56:32.917 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:32.917 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-09-21T03:56:32.917 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2024-09-21T03:56:32.917 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:32.918 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:32.918 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:32.918 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:32.918 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:32.918 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:32.918 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:32.918 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:32.918 INFO:teuthology.orchestra.run.smithi039.stdout: "UNSAFE_TLS_v1_2": { 2024-09-21T03:56:32.918 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "UNSAFE_TLS_v1_2", 2024-09-21T03:56:32.918 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-09-21T03:56:32.918 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:32.918 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-09-21T03:56:32.918 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-09-21T03:56:32.918 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:32.919 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:32.919 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:32.919 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:32.919 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:32.919 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:32.919 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:32.919 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:32.919 INFO:teuthology.orchestra.run.smithi039.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-09-21T03:56:32.919 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-09-21T03:56:32.919 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-09-21T03:56:32.919 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:32.919 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-09-21T03:56:32.919 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "0", 2024-09-21T03:56:32.919 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:32.919 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:32.920 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:32.920 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:32.920 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:32.920 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:32.920 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:32.920 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:32.920 INFO:teuthology.orchestra.run.smithi039.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-09-21T03:56:32.920 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-09-21T03:56:32.920 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-09-21T03:56:32.920 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:32.920 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-09-21T03:56:32.920 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "10", 2024-09-21T03:56:32.920 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:32.920 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:32.920 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:32.921 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:32.921 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:32.921 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:32.921 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:32.921 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:32.921 INFO:teuthology.orchestra.run.smithi039.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-09-21T03:56:32.921 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-09-21T03:56:32.921 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-09-21T03:56:32.921 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:32.921 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-09-21T03:56:32.921 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "5", 2024-09-21T03:56:32.921 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:32.921 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:32.921 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:32.921 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:32.922 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:32.922 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:32.922 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:32.922 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:32.922 INFO:teuthology.orchestra.run.smithi039.stdout: "cross_origin_url": { 2024-09-21T03:56:32.922 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "cross_origin_url", 2024-09-21T03:56:32.922 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-09-21T03:56:32.922 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:32.922 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-09-21T03:56:32.922 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-09-21T03:56:32.922 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:32.922 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:32.922 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:32.922 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:32.922 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:32.923 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:32.923 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:32.923 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:32.923 INFO:teuthology.orchestra.run.smithi039.stdout: "crt_file": { 2024-09-21T03:56:32.923 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "crt_file", 2024-09-21T03:56:32.923 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-09-21T03:56:32.923 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:32.923 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-09-21T03:56:32.923 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-09-21T03:56:32.923 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:32.923 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:32.923 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:32.923 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:32.923 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:32.923 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:32.924 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:32.924 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:32.924 INFO:teuthology.orchestra.run.smithi039.stdout: "debug": { 2024-09-21T03:56:32.924 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "debug", 2024-09-21T03:56:32.924 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-09-21T03:56:32.924 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:32.924 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-09-21T03:56:32.924 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-09-21T03:56:32.924 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:32.924 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:32.924 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:32.924 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Enable/disable debug options", 2024-09-21T03:56:32.924 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:32.924 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:32.925 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:32.925 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:32.925 INFO:teuthology.orchestra.run.smithi039.stdout: "jwt_token_ttl": { 2024-09-21T03:56:32.925 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "jwt_token_ttl", 2024-09-21T03:56:32.925 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-09-21T03:56:32.925 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:32.925 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-09-21T03:56:32.925 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "28800", 2024-09-21T03:56:32.925 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:32.925 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:32.925 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:32.925 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:32.925 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:32.925 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:32.926 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:32.926 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:32.926 INFO:teuthology.orchestra.run.smithi039.stdout: "key_file": { 2024-09-21T03:56:32.926 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "key_file", 2024-09-21T03:56:32.926 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-09-21T03:56:32.926 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:32.926 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-09-21T03:56:32.926 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-09-21T03:56:32.926 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:32.926 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:32.926 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:32.926 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:32.926 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:32.926 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:32.926 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:32.927 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:32.927 INFO:teuthology.orchestra.run.smithi039.stdout: "log_level": { 2024-09-21T03:56:32.927 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_level", 2024-09-21T03:56:32.927 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-09-21T03:56:32.927 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:32.927 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:32.927 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-09-21T03:56:32.927 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:32.927 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:32.927 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-09-21T03:56:32.927 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-09-21T03:56:32.927 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-09-21T03:56:32.927 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-09-21T03:56:32.927 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-09-21T03:56:32.927 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-09-21T03:56:32.928 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-09-21T03:56:32.928 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-09-21T03:56:32.928 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:32.928 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:32.928 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:32.928 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:32.928 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:32.928 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster": { 2024-09-21T03:56:32.928 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster", 2024-09-21T03:56:32.928 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-09-21T03:56:32.928 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:32.928 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:32.928 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-09-21T03:56:32.928 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:32.928 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:32.929 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:32.929 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:32.929 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:32.929 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:32.929 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:32.929 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:32.929 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster_level": { 2024-09-21T03:56:32.929 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster_level", 2024-09-21T03:56:32.929 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-09-21T03:56:32.929 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:32.929 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:32.929 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "info", 2024-09-21T03:56:32.929 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:32.929 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:32.929 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-09-21T03:56:32.930 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-09-21T03:56:32.930 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-09-21T03:56:32.930 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-09-21T03:56:32.930 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-09-21T03:56:32.930 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-09-21T03:56:32.930 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-09-21T03:56:32.930 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-09-21T03:56:32.930 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:32.930 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:32.930 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:32.930 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:32.930 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:32.930 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_file": { 2024-09-21T03:56:32.930 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_file", 2024-09-21T03:56:32.931 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-09-21T03:56:32.931 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:32.931 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:32.931 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-09-21T03:56:32.931 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:32.931 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:32.931 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:32.931 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:32.931 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:32.931 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:32.931 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:32.931 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:32.931 INFO:teuthology.orchestra.run.smithi039.stdout: "motd": { 2024-09-21T03:56:32.931 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "motd", 2024-09-21T03:56:32.931 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-09-21T03:56:32.932 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:32.932 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-09-21T03:56:32.932 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-09-21T03:56:32.932 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:32.932 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:32.932 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:32.932 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "The message of the day", 2024-09-21T03:56:32.932 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:32.932 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:32.932 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:32.932 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:32.932 INFO:teuthology.orchestra.run.smithi039.stdout: "redirect_resolve_ip_addr": { 2024-09-21T03:56:32.932 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "redirect_resolve_ip_addr", 2024-09-21T03:56:32.932 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-09-21T03:56:32.933 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:32.933 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-09-21T03:56:32.933 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-09-21T03:56:32.933 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:32.933 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:32.933 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:32.933 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:32.933 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:32.933 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:32.933 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:32.933 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:32.933 INFO:teuthology.orchestra.run.smithi039.stdout: "server_addr": { 2024-09-21T03:56:32.933 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "server_addr", 2024-09-21T03:56:32.933 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-09-21T03:56:32.933 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:32.934 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-09-21T03:56:32.934 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "::", 2024-09-21T03:56:32.934 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:32.934 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:32.934 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:32.934 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:32.934 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:32.934 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:32.934 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:32.934 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:32.934 INFO:teuthology.orchestra.run.smithi039.stdout: "server_port": { 2024-09-21T03:56:32.934 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "server_port", 2024-09-21T03:56:32.934 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-09-21T03:56:32.934 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:32.934 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-09-21T03:56:32.935 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "8080", 2024-09-21T03:56:32.935 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:32.935 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:32.935 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:32.935 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:32.935 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:32.935 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:32.935 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:32.935 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:32.935 INFO:teuthology.orchestra.run.smithi039.stdout: "sqlite3_killpoint": { 2024-09-21T03:56:32.935 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:56:32.935 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-09-21T03:56:32.935 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "dev", 2024-09-21T03:56:32.935 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:32.935 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "0", 2024-09-21T03:56:32.936 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:32.936 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:32.936 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:32.936 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:32.936 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:32.936 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:32.936 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:32.936 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:32.936 INFO:teuthology.orchestra.run.smithi039.stdout: "ssl": { 2024-09-21T03:56:32.936 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "ssl", 2024-09-21T03:56:32.936 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-09-21T03:56:32.936 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:32.936 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-09-21T03:56:32.936 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2024-09-21T03:56:32.936 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:32.937 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:32.937 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:32.937 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:32.937 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:32.937 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:32.937 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:32.937 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:32.937 INFO:teuthology.orchestra.run.smithi039.stdout: "ssl_server_port": { 2024-09-21T03:56:32.937 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "ssl_server_port", 2024-09-21T03:56:32.937 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-09-21T03:56:32.937 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:32.937 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-09-21T03:56:32.937 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "8443", 2024-09-21T03:56:32.937 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:32.937 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:32.938 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:32.938 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:32.938 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:32.938 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:32.938 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:32.938 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:32.938 INFO:teuthology.orchestra.run.smithi039.stdout: "sso_oauth2": { 2024-09-21T03:56:32.938 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "sso_oauth2", 2024-09-21T03:56:32.938 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-09-21T03:56:32.938 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:32.938 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-09-21T03:56:32.938 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-09-21T03:56:32.939 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:32.939 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:32.939 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:32.939 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:32.939 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:32.939 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:32.939 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:32.939 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:32.939 INFO:teuthology.orchestra.run.smithi039.stdout: "standby_behaviour": { 2024-09-21T03:56:32.939 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "standby_behaviour", 2024-09-21T03:56:32.939 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-09-21T03:56:32.939 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:32.939 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-09-21T03:56:32.939 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "redirect", 2024-09-21T03:56:32.939 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:32.940 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:32.940 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-09-21T03:56:32.940 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-09-21T03:56:32.940 INFO:teuthology.orchestra.run.smithi039.stdout: "redirect" 2024-09-21T03:56:32.940 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-09-21T03:56:32.940 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:32.940 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:32.940 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:32.940 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:32.940 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:32.940 INFO:teuthology.orchestra.run.smithi039.stdout: "standby_error_status_code": { 2024-09-21T03:56:32.940 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "standby_error_status_code", 2024-09-21T03:56:32.940 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-09-21T03:56:32.940 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:32.940 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-09-21T03:56:32.941 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "500", 2024-09-21T03:56:32.941 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "400", 2024-09-21T03:56:32.941 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "599", 2024-09-21T03:56:32.941 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:32.941 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:32.941 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:32.941 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:32.941 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:32.941 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:32.941 INFO:teuthology.orchestra.run.smithi039.stdout: "url_prefix": { 2024-09-21T03:56:32.941 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "url_prefix", 2024-09-21T03:56:32.941 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-09-21T03:56:32.941 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:32.941 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-09-21T03:56:32.942 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-09-21T03:56:32.942 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:32.942 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:32.942 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:32.942 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:32.942 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:32.942 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:32.942 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:32.942 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-21T03:56:32.942 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-21T03:56:32.942 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:32.942 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-21T03:56:32.942 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "devicehealth", 2024-09-21T03:56:32.942 INFO:teuthology.orchestra.run.smithi039.stdout: "can_run": true, 2024-09-21T03:56:32.943 INFO:teuthology.orchestra.run.smithi039.stdout: "error_string": "", 2024-09-21T03:56:32.943 INFO:teuthology.orchestra.run.smithi039.stdout: "module_options": { 2024-09-21T03:56:32.943 INFO:teuthology.orchestra.run.smithi039.stdout: "enable_monitoring": { 2024-09-21T03:56:32.943 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "enable_monitoring", 2024-09-21T03:56:32.943 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-09-21T03:56:32.943 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:32.943 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:32.943 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2024-09-21T03:56:32.943 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:32.943 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:32.943 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:32.943 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "monitor device health metrics", 2024-09-21T03:56:32.943 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:32.943 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:32.943 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:32.944 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:32.944 INFO:teuthology.orchestra.run.smithi039.stdout: "log_level": { 2024-09-21T03:56:32.944 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_level", 2024-09-21T03:56:32.944 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-09-21T03:56:32.944 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:32.944 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:32.944 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-09-21T03:56:32.944 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:32.944 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:32.944 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-09-21T03:56:32.944 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-09-21T03:56:32.944 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-09-21T03:56:32.944 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-09-21T03:56:32.944 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-09-21T03:56:32.945 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-09-21T03:56:32.945 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-09-21T03:56:32.945 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-09-21T03:56:32.945 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:32.945 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:32.945 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:32.945 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:32.945 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:32.945 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster": { 2024-09-21T03:56:32.945 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster", 2024-09-21T03:56:32.945 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-09-21T03:56:32.946 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:32.946 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:32.946 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-09-21T03:56:32.946 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:32.946 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:32.946 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:32.946 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:32.946 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:32.946 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:32.946 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:32.946 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:32.946 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster_level": { 2024-09-21T03:56:32.946 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster_level", 2024-09-21T03:56:32.946 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-09-21T03:56:32.947 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:32.947 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:32.947 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "info", 2024-09-21T03:56:32.947 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:32.947 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:32.947 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-09-21T03:56:32.947 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-09-21T03:56:32.947 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-09-21T03:56:32.947 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-09-21T03:56:32.947 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-09-21T03:56:32.947 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-09-21T03:56:32.947 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-09-21T03:56:32.947 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-09-21T03:56:32.947 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:32.948 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:32.948 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:32.948 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:32.948 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:32.948 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_file": { 2024-09-21T03:56:32.948 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_file", 2024-09-21T03:56:32.948 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-09-21T03:56:32.948 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:32.948 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:32.948 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-09-21T03:56:32.948 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:32.948 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:32.948 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:32.948 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:32.948 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:32.949 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:32.949 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:32.949 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:32.949 INFO:teuthology.orchestra.run.smithi039.stdout: "mark_out_threshold": { 2024-09-21T03:56:32.949 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "mark_out_threshold", 2024-09-21T03:56:32.949 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "secs", 2024-09-21T03:56:32.949 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:32.949 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:32.949 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "2419200", 2024-09-21T03:56:32.949 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:32.949 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:32.949 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:32.949 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-09-21T03:56:32.949 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:32.950 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:32.950 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:32.950 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:32.950 INFO:teuthology.orchestra.run.smithi039.stdout: "pool_name": { 2024-09-21T03:56:32.950 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "pool_name", 2024-09-21T03:56:32.950 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-09-21T03:56:32.950 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:32.950 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:32.950 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "device_health_metrics", 2024-09-21T03:56:32.950 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:32.950 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:32.950 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:32.950 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "name of pool in which to store device health metrics", 2024-09-21T03:56:32.950 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:32.951 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:32.951 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:32.951 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:32.951 INFO:teuthology.orchestra.run.smithi039.stdout: "retention_period": { 2024-09-21T03:56:32.951 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "retention_period", 2024-09-21T03:56:32.951 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "secs", 2024-09-21T03:56:32.951 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:32.951 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:32.951 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "15552000", 2024-09-21T03:56:32.951 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:32.951 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:32.951 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:32.952 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "how long to retain device health metrics", 2024-09-21T03:56:32.952 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:32.952 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:32.952 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:32.952 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:32.952 INFO:teuthology.orchestra.run.smithi039.stdout: "scrape_frequency": { 2024-09-21T03:56:32.952 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "scrape_frequency", 2024-09-21T03:56:32.952 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "secs", 2024-09-21T03:56:32.952 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:32.952 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:32.952 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "86400", 2024-09-21T03:56:32.952 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:32.952 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:32.952 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:32.953 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "how frequently to scrape device health metrics", 2024-09-21T03:56:32.953 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:32.953 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:32.953 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:32.953 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:32.953 INFO:teuthology.orchestra.run.smithi039.stdout: "self_heal": { 2024-09-21T03:56:32.953 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "self_heal", 2024-09-21T03:56:32.953 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-09-21T03:56:32.953 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:32.953 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:32.953 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2024-09-21T03:56:32.953 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:32.953 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:32.954 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:32.954 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-09-21T03:56:32.954 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:32.954 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:32.954 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:32.954 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:32.954 INFO:teuthology.orchestra.run.smithi039.stdout: "sleep_interval": { 2024-09-21T03:56:32.954 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "sleep_interval", 2024-09-21T03:56:32.954 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "secs", 2024-09-21T03:56:32.954 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:32.954 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:32.954 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "600", 2024-09-21T03:56:32.954 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:32.955 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:32.955 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:32.955 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "how frequently to wake up and check device health", 2024-09-21T03:56:32.955 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:32.955 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:32.955 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:32.955 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:32.955 INFO:teuthology.orchestra.run.smithi039.stdout: "sqlite3_killpoint": { 2024-09-21T03:56:32.955 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:56:32.955 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-09-21T03:56:32.955 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "dev", 2024-09-21T03:56:32.955 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:32.955 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "0", 2024-09-21T03:56:32.955 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:32.956 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:32.956 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:32.956 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:32.956 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:32.956 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:32.956 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:32.956 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:32.956 INFO:teuthology.orchestra.run.smithi039.stdout: "warn_threshold": { 2024-09-21T03:56:32.956 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "warn_threshold", 2024-09-21T03:56:32.956 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "secs", 2024-09-21T03:56:32.956 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:32.956 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:32.956 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "7257600", 2024-09-21T03:56:32.956 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:32.957 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:32.957 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:32.957 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-09-21T03:56:32.957 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:32.957 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:32.957 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:32.957 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-21T03:56:32.957 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-21T03:56:32.957 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:32.957 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-21T03:56:32.957 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "diskprediction_local", 2024-09-21T03:56:32.957 INFO:teuthology.orchestra.run.smithi039.stdout: "can_run": true, 2024-09-21T03:56:32.957 INFO:teuthology.orchestra.run.smithi039.stdout: "error_string": "", 2024-09-21T03:56:32.957 INFO:teuthology.orchestra.run.smithi039.stdout: "module_options": { 2024-09-21T03:56:32.957 INFO:teuthology.orchestra.run.smithi039.stdout: "log_level": { 2024-09-21T03:56:32.958 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_level", 2024-09-21T03:56:32.958 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-09-21T03:56:32.958 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:32.958 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:32.958 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-09-21T03:56:32.958 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:32.958 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:32.958 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-09-21T03:56:32.958 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-09-21T03:56:32.958 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-09-21T03:56:32.958 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-09-21T03:56:32.958 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-09-21T03:56:32.958 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-09-21T03:56:32.958 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-09-21T03:56:32.959 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-09-21T03:56:32.959 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:32.959 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:32.959 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:32.959 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:32.959 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:32.959 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster": { 2024-09-21T03:56:32.959 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster", 2024-09-21T03:56:32.959 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-09-21T03:56:32.959 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:32.959 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:32.959 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-09-21T03:56:32.959 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:32.959 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:32.960 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:32.960 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:32.960 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:32.960 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:32.960 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:32.960 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:32.960 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster_level": { 2024-09-21T03:56:32.960 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster_level", 2024-09-21T03:56:32.960 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-09-21T03:56:32.960 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:32.960 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:32.960 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "info", 2024-09-21T03:56:32.960 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:32.960 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:32.960 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-09-21T03:56:32.961 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-09-21T03:56:32.961 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-09-21T03:56:32.961 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-09-21T03:56:32.961 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-09-21T03:56:32.961 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-09-21T03:56:32.961 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-09-21T03:56:32.961 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-09-21T03:56:32.961 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:32.961 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:32.961 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:32.961 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:32.961 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:32.961 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_file": { 2024-09-21T03:56:32.961 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_file", 2024-09-21T03:56:32.962 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-09-21T03:56:32.962 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:32.962 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:32.962 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-09-21T03:56:32.962 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:32.962 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:32.962 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:32.962 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:32.962 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:32.962 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:32.962 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:32.962 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:32.962 INFO:teuthology.orchestra.run.smithi039.stdout: "predict_interval": { 2024-09-21T03:56:32.962 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "predict_interval", 2024-09-21T03:56:32.962 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-09-21T03:56:32.963 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:32.963 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-09-21T03:56:32.963 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "86400", 2024-09-21T03:56:32.963 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:32.963 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:32.963 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:32.963 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:32.963 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:32.963 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:32.963 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:32.963 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:32.963 INFO:teuthology.orchestra.run.smithi039.stdout: "predictor_model": { 2024-09-21T03:56:32.963 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "predictor_model", 2024-09-21T03:56:32.963 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-09-21T03:56:32.964 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:32.964 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-09-21T03:56:32.964 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "prophetstor", 2024-09-21T03:56:32.964 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:32.964 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:32.964 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:32.964 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:32.964 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:32.964 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:32.964 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:32.964 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:32.964 INFO:teuthology.orchestra.run.smithi039.stdout: "sleep_interval": { 2024-09-21T03:56:32.964 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "sleep_interval", 2024-09-21T03:56:32.964 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-09-21T03:56:32.964 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:32.965 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-09-21T03:56:32.965 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "600", 2024-09-21T03:56:32.965 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:32.965 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:32.965 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:32.965 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:32.965 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:32.965 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:32.965 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:32.965 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:32.965 INFO:teuthology.orchestra.run.smithi039.stdout: "sqlite3_killpoint": { 2024-09-21T03:56:32.965 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:56:32.966 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-09-21T03:56:32.966 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "dev", 2024-09-21T03:56:32.966 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:32.966 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "0", 2024-09-21T03:56:32.966 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:32.966 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:32.966 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:32.966 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:32.966 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:32.966 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:32.966 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:32.966 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-21T03:56:32.966 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-21T03:56:32.966 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:32.967 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-21T03:56:32.967 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "influx", 2024-09-21T03:56:32.967 INFO:teuthology.orchestra.run.smithi039.stdout: "can_run": false, 2024-09-21T03:56:32.967 INFO:teuthology.orchestra.run.smithi039.stdout: "error_string": "influxdb python module not found", 2024-09-21T03:56:32.967 INFO:teuthology.orchestra.run.smithi039.stdout: "module_options": { 2024-09-21T03:56:32.967 INFO:teuthology.orchestra.run.smithi039.stdout: "batch_size": { 2024-09-21T03:56:32.967 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "batch_size", 2024-09-21T03:56:32.967 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-09-21T03:56:32.967 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:32.967 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-09-21T03:56:32.967 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "5000", 2024-09-21T03:56:32.967 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:32.967 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:32.967 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:32.968 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-09-21T03:56:32.968 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:32.968 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:32.968 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:32.968 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:32.968 INFO:teuthology.orchestra.run.smithi039.stdout: "database": { 2024-09-21T03:56:32.968 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "database", 2024-09-21T03:56:32.968 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-09-21T03:56:32.968 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:32.968 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-09-21T03:56:32.968 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "ceph", 2024-09-21T03:56:32.968 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:32.968 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:32.968 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:32.969 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-09-21T03:56:32.969 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:32.969 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:32.969 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:32.969 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:32.969 INFO:teuthology.orchestra.run.smithi039.stdout: "hostname": { 2024-09-21T03:56:32.969 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "hostname", 2024-09-21T03:56:32.969 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-09-21T03:56:32.969 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:32.969 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-09-21T03:56:32.969 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-09-21T03:56:32.969 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:32.969 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:32.969 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:32.969 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "InfluxDB server hostname", 2024-09-21T03:56:32.970 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:32.970 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:32.970 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:32.970 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:32.970 INFO:teuthology.orchestra.run.smithi039.stdout: "interval": { 2024-09-21T03:56:32.970 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "interval", 2024-09-21T03:56:32.970 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "secs", 2024-09-21T03:56:32.970 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:32.970 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-09-21T03:56:32.970 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "30", 2024-09-21T03:56:32.970 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "5", 2024-09-21T03:56:32.970 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:32.970 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:32.971 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-09-21T03:56:32.971 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:32.971 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:32.971 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:32.971 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:32.971 INFO:teuthology.orchestra.run.smithi039.stdout: "log_level": { 2024-09-21T03:56:32.971 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_level", 2024-09-21T03:56:32.971 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-09-21T03:56:32.971 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:32.971 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:32.971 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-09-21T03:56:32.971 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:32.971 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:32.971 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-09-21T03:56:32.971 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-09-21T03:56:32.972 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-09-21T03:56:32.972 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-09-21T03:56:32.972 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-09-21T03:56:32.972 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-09-21T03:56:32.972 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-09-21T03:56:32.972 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-09-21T03:56:32.972 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:32.972 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:32.972 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:32.972 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:32.972 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:32.972 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster": { 2024-09-21T03:56:32.972 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster", 2024-09-21T03:56:32.972 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-09-21T03:56:32.973 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:32.973 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:32.973 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-09-21T03:56:32.973 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:32.973 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:32.973 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:32.973 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:32.973 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:32.973 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:32.973 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:32.973 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:32.973 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster_level": { 2024-09-21T03:56:32.973 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster_level", 2024-09-21T03:56:32.973 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-09-21T03:56:32.973 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:32.974 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:32.974 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "info", 2024-09-21T03:56:32.974 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:32.974 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:32.974 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-09-21T03:56:32.974 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-09-21T03:56:32.974 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-09-21T03:56:32.974 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-09-21T03:56:32.974 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-09-21T03:56:32.974 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-09-21T03:56:32.974 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-09-21T03:56:32.974 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-09-21T03:56:32.974 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:32.974 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:32.975 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:32.975 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:32.975 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:32.975 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_file": { 2024-09-21T03:56:32.975 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_file", 2024-09-21T03:56:32.975 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-09-21T03:56:32.975 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:32.975 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:32.975 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-09-21T03:56:32.975 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:32.975 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:32.975 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:32.975 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:32.975 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:32.976 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:32.976 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:32.976 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:32.976 INFO:teuthology.orchestra.run.smithi039.stdout: "password": { 2024-09-21T03:56:32.976 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "password", 2024-09-21T03:56:32.976 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-09-21T03:56:32.976 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:32.976 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-09-21T03:56:32.976 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-09-21T03:56:32.976 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:32.976 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:32.976 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:32.976 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "password of InfluxDB server user", 2024-09-21T03:56:32.976 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:32.976 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:32.977 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:32.977 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:32.977 INFO:teuthology.orchestra.run.smithi039.stdout: "port": { 2024-09-21T03:56:32.977 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "port", 2024-09-21T03:56:32.977 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-09-21T03:56:32.977 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:32.977 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-09-21T03:56:32.977 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "8086", 2024-09-21T03:56:32.977 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:32.977 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:32.977 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:32.977 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "InfluxDB server port", 2024-09-21T03:56:32.977 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:32.977 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:32.978 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:32.978 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:32.978 INFO:teuthology.orchestra.run.smithi039.stdout: "sqlite3_killpoint": { 2024-09-21T03:56:32.978 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:56:32.978 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-09-21T03:56:32.978 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "dev", 2024-09-21T03:56:32.978 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:32.978 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "0", 2024-09-21T03:56:32.978 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:32.978 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:32.978 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:32.978 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:32.979 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:32.979 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:32.979 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:32.979 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:32.979 INFO:teuthology.orchestra.run.smithi039.stdout: "ssl": { 2024-09-21T03:56:32.979 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "ssl", 2024-09-21T03:56:32.979 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-09-21T03:56:32.979 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:32.979 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-09-21T03:56:32.979 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "false", 2024-09-21T03:56:32.979 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:32.979 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:32.979 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:32.979 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-09-21T03:56:32.979 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:32.980 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:32.980 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:32.980 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:32.980 INFO:teuthology.orchestra.run.smithi039.stdout: "threads": { 2024-09-21T03:56:32.980 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "threads", 2024-09-21T03:56:32.980 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-09-21T03:56:32.980 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:32.980 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-09-21T03:56:32.980 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "5", 2024-09-21T03:56:32.980 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "1", 2024-09-21T03:56:32.980 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "32", 2024-09-21T03:56:32.980 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:32.980 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-09-21T03:56:32.980 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:32.981 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:32.981 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:32.981 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:32.981 INFO:teuthology.orchestra.run.smithi039.stdout: "username": { 2024-09-21T03:56:32.981 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "username", 2024-09-21T03:56:32.981 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-09-21T03:56:32.981 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:32.981 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-09-21T03:56:32.981 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-09-21T03:56:32.981 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:32.981 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:32.981 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:32.981 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "username of InfluxDB server user", 2024-09-21T03:56:32.981 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:32.982 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:32.982 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:32.982 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:32.982 INFO:teuthology.orchestra.run.smithi039.stdout: "verify_ssl": { 2024-09-21T03:56:32.982 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "verify_ssl", 2024-09-21T03:56:32.982 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-09-21T03:56:32.982 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:32.982 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-09-21T03:56:32.982 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "true", 2024-09-21T03:56:32.982 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:32.982 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:32.982 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:32.982 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-09-21T03:56:32.982 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:32.983 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:32.983 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:32.983 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-21T03:56:32.983 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-21T03:56:32.983 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:32.983 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-21T03:56:32.983 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "insights", 2024-09-21T03:56:32.983 INFO:teuthology.orchestra.run.smithi039.stdout: "can_run": true, 2024-09-21T03:56:32.983 INFO:teuthology.orchestra.run.smithi039.stdout: "error_string": "", 2024-09-21T03:56:32.983 INFO:teuthology.orchestra.run.smithi039.stdout: "module_options": { 2024-09-21T03:56:32.983 INFO:teuthology.orchestra.run.smithi039.stdout: "log_level": { 2024-09-21T03:56:32.983 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_level", 2024-09-21T03:56:32.983 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-09-21T03:56:32.984 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:32.984 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:32.984 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-09-21T03:56:32.984 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:32.984 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:32.984 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-09-21T03:56:32.984 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-09-21T03:56:32.984 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-09-21T03:56:32.984 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-09-21T03:56:32.984 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-09-21T03:56:32.984 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-09-21T03:56:32.984 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-09-21T03:56:32.984 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-09-21T03:56:32.984 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:32.984 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:32.985 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:32.985 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:32.985 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:32.985 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster": { 2024-09-21T03:56:32.985 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster", 2024-09-21T03:56:32.985 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-09-21T03:56:32.985 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:32.985 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:32.985 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-09-21T03:56:32.985 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:32.985 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:32.985 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:32.985 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:32.985 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:32.986 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:32.986 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:32.986 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:32.986 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster_level": { 2024-09-21T03:56:32.986 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster_level", 2024-09-21T03:56:32.986 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-09-21T03:56:32.986 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:32.986 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:32.986 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "info", 2024-09-21T03:56:32.986 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:32.986 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:32.986 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-09-21T03:56:32.986 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-09-21T03:56:32.986 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-09-21T03:56:32.986 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-09-21T03:56:32.987 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-09-21T03:56:32.987 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-09-21T03:56:32.987 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-09-21T03:56:32.987 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-09-21T03:56:32.987 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:32.987 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:32.987 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:32.987 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:32.987 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:32.987 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_file": { 2024-09-21T03:56:32.987 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_file", 2024-09-21T03:56:32.987 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-09-21T03:56:32.987 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:32.987 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:32.988 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-09-21T03:56:32.988 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:32.988 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:32.988 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:32.988 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:32.988 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:32.988 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:32.988 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:32.988 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:32.988 INFO:teuthology.orchestra.run.smithi039.stdout: "sqlite3_killpoint": { 2024-09-21T03:56:32.988 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:56:32.988 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-09-21T03:56:32.988 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "dev", 2024-09-21T03:56:32.988 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:32.988 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "0", 2024-09-21T03:56:32.989 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:32.989 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:32.989 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:32.989 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:32.989 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:32.989 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:32.989 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:32.989 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-21T03:56:32.989 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-21T03:56:32.989 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:32.989 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-21T03:56:32.989 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "iostat", 2024-09-21T03:56:32.989 INFO:teuthology.orchestra.run.smithi039.stdout: "can_run": true, 2024-09-21T03:56:32.989 INFO:teuthology.orchestra.run.smithi039.stdout: "error_string": "", 2024-09-21T03:56:32.990 INFO:teuthology.orchestra.run.smithi039.stdout: "module_options": { 2024-09-21T03:56:32.990 INFO:teuthology.orchestra.run.smithi039.stdout: "log_level": { 2024-09-21T03:56:32.990 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_level", 2024-09-21T03:56:32.990 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-09-21T03:56:32.990 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:32.990 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:32.990 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-09-21T03:56:32.990 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:32.990 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:32.990 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-09-21T03:56:32.990 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-09-21T03:56:32.990 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-09-21T03:56:32.990 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-09-21T03:56:32.990 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-09-21T03:56:32.991 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-09-21T03:56:32.991 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-09-21T03:56:32.991 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-09-21T03:56:32.991 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:32.991 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:32.991 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:32.991 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:32.991 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:32.991 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster": { 2024-09-21T03:56:32.991 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster", 2024-09-21T03:56:32.991 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-09-21T03:56:32.991 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:32.991 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:32.992 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-09-21T03:56:32.992 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:32.992 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:32.992 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:32.992 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:32.992 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:32.992 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:32.992 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:32.992 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:32.992 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster_level": { 2024-09-21T03:56:32.992 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster_level", 2024-09-21T03:56:32.992 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-09-21T03:56:32.992 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:32.992 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:32.993 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "info", 2024-09-21T03:56:32.993 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:32.993 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:32.993 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-09-21T03:56:32.993 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-09-21T03:56:32.993 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-09-21T03:56:32.993 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-09-21T03:56:32.993 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-09-21T03:56:32.993 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-09-21T03:56:32.993 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-09-21T03:56:32.993 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-09-21T03:56:32.993 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:32.993 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:32.993 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:32.994 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:32.994 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:32.994 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_file": { 2024-09-21T03:56:32.994 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_file", 2024-09-21T03:56:32.994 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-09-21T03:56:32.994 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:32.994 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:32.994 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-09-21T03:56:32.994 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:32.994 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:32.994 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:32.994 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:32.994 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:32.994 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:32.995 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:32.995 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:32.995 INFO:teuthology.orchestra.run.smithi039.stdout: "sqlite3_killpoint": { 2024-09-21T03:56:32.995 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:56:32.995 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-09-21T03:56:32.995 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "dev", 2024-09-21T03:56:32.995 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:32.995 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "0", 2024-09-21T03:56:32.995 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:32.995 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:32.995 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:32.995 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:32.995 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:32.995 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:32.996 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:32.996 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-21T03:56:32.996 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-21T03:56:32.996 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:32.996 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-21T03:56:32.996 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "k8sevents", 2024-09-21T03:56:32.996 INFO:teuthology.orchestra.run.smithi039.stdout: "can_run": true, 2024-09-21T03:56:32.996 INFO:teuthology.orchestra.run.smithi039.stdout: "error_string": "", 2024-09-21T03:56:32.996 INFO:teuthology.orchestra.run.smithi039.stdout: "module_options": { 2024-09-21T03:56:32.996 INFO:teuthology.orchestra.run.smithi039.stdout: "ceph_event_retention_days": { 2024-09-21T03:56:32.996 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "ceph_event_retention_days", 2024-09-21T03:56:32.996 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-09-21T03:56:32.996 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:32.996 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-09-21T03:56:32.997 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "7", 2024-09-21T03:56:32.997 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:32.997 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:32.997 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:32.997 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Days to hold ceph event information within local cache", 2024-09-21T03:56:32.997 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:32.997 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:32.997 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:32.997 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:32.997 INFO:teuthology.orchestra.run.smithi039.stdout: "config_check_secs": { 2024-09-21T03:56:32.997 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "config_check_secs", 2024-09-21T03:56:32.997 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-09-21T03:56:32.997 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:32.997 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-09-21T03:56:32.997 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "10", 2024-09-21T03:56:32.998 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "10", 2024-09-21T03:56:32.998 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:32.998 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:32.998 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-09-21T03:56:32.998 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:32.998 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:32.998 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:32.998 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:32.998 INFO:teuthology.orchestra.run.smithi039.stdout: "log_level": { 2024-09-21T03:56:32.998 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_level", 2024-09-21T03:56:32.998 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-09-21T03:56:32.998 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:32.998 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:32.998 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-09-21T03:56:32.999 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:32.999 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:32.999 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-09-21T03:56:32.999 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-09-21T03:56:32.999 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-09-21T03:56:32.999 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-09-21T03:56:32.999 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-09-21T03:56:32.999 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-09-21T03:56:32.999 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-09-21T03:56:32.999 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-09-21T03:56:32.999 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:32.999 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:32.999 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:32.999 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:32.999 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.000 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster": { 2024-09-21T03:56:33.000 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster", 2024-09-21T03:56:33.000 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-09-21T03:56:33.000 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.000 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:33.000 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-09-21T03:56:33.000 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.000 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.000 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.000 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:33.000 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.000 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.000 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.000 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.000 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster_level": { 2024-09-21T03:56:33.001 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster_level", 2024-09-21T03:56:33.001 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-09-21T03:56:33.001 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.001 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:33.001 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "info", 2024-09-21T03:56:33.001 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.001 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.001 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-09-21T03:56:33.001 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-09-21T03:56:33.001 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-09-21T03:56:33.001 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-09-21T03:56:33.001 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-09-21T03:56:33.001 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-09-21T03:56:33.001 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-09-21T03:56:33.001 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-09-21T03:56:33.002 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:33.002 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.002 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.002 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.002 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.002 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_file": { 2024-09-21T03:56:33.002 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_file", 2024-09-21T03:56:33.002 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-09-21T03:56:33.002 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.002 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:33.002 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-09-21T03:56:33.002 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.002 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.002 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.003 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:33.003 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.003 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.003 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.003 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.003 INFO:teuthology.orchestra.run.smithi039.stdout: "sqlite3_killpoint": { 2024-09-21T03:56:33.003 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:56:33.003 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-09-21T03:56:33.003 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "dev", 2024-09-21T03:56:33.003 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:33.003 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "0", 2024-09-21T03:56:33.003 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.003 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.003 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.004 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:33.004 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.004 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.004 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.004 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-21T03:56:33.004 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-21T03:56:33.004 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.004 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-21T03:56:33.004 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "localpool", 2024-09-21T03:56:33.004 INFO:teuthology.orchestra.run.smithi039.stdout: "can_run": true, 2024-09-21T03:56:33.004 INFO:teuthology.orchestra.run.smithi039.stdout: "error_string": "", 2024-09-21T03:56:33.004 INFO:teuthology.orchestra.run.smithi039.stdout: "module_options": { 2024-09-21T03:56:33.004 INFO:teuthology.orchestra.run.smithi039.stdout: "failure_domain": { 2024-09-21T03:56:33.004 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "failure_domain", 2024-09-21T03:56:33.004 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-09-21T03:56:33.005 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.005 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:33.005 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "host", 2024-09-21T03:56:33.005 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.005 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.005 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.005 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "failure domain for any created local pool", 2024-09-21T03:56:33.005 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-09-21T03:56:33.005 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.005 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.005 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.005 INFO:teuthology.orchestra.run.smithi039.stdout: "log_level": { 2024-09-21T03:56:33.005 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_level", 2024-09-21T03:56:33.006 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-09-21T03:56:33.006 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.006 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:33.006 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-09-21T03:56:33.006 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.006 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.006 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-09-21T03:56:33.006 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-09-21T03:56:33.006 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-09-21T03:56:33.006 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-09-21T03:56:33.006 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-09-21T03:56:33.006 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-09-21T03:56:33.006 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-09-21T03:56:33.006 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-09-21T03:56:33.007 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:33.007 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.007 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.007 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.007 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.007 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster": { 2024-09-21T03:56:33.007 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster", 2024-09-21T03:56:33.007 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-09-21T03:56:33.007 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.007 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:33.007 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-09-21T03:56:33.007 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.007 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.007 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.007 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:33.008 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.008 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.008 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.008 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.008 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster_level": { 2024-09-21T03:56:33.008 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster_level", 2024-09-21T03:56:33.008 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-09-21T03:56:33.008 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.008 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:33.008 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "info", 2024-09-21T03:56:33.008 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.008 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.008 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-09-21T03:56:33.009 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-09-21T03:56:33.009 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-09-21T03:56:33.009 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-09-21T03:56:33.009 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-09-21T03:56:33.009 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-09-21T03:56:33.009 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-09-21T03:56:33.009 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-09-21T03:56:33.009 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:33.009 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.009 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.009 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.009 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.009 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_file": { 2024-09-21T03:56:33.009 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_file", 2024-09-21T03:56:33.010 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-09-21T03:56:33.010 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.010 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:33.010 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-09-21T03:56:33.010 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.010 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.010 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.010 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:33.010 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.010 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.010 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.010 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.010 INFO:teuthology.orchestra.run.smithi039.stdout: "min_size": { 2024-09-21T03:56:33.010 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "min_size", 2024-09-21T03:56:33.011 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-09-21T03:56:33.011 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.011 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:33.011 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-09-21T03:56:33.011 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.011 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.011 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.011 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "default min_size for any created local pool", 2024-09-21T03:56:33.011 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-09-21T03:56:33.011 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.011 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.011 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.011 INFO:teuthology.orchestra.run.smithi039.stdout: "num_rep": { 2024-09-21T03:56:33.011 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "num_rep", 2024-09-21T03:56:33.012 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-09-21T03:56:33.012 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.012 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:33.012 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "3", 2024-09-21T03:56:33.012 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.012 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.012 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.012 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "default replica count for any created local pool", 2024-09-21T03:56:33.012 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.012 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.012 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.012 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.012 INFO:teuthology.orchestra.run.smithi039.stdout: "pg_num": { 2024-09-21T03:56:33.012 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "pg_num", 2024-09-21T03:56:33.013 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-09-21T03:56:33.013 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.013 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:33.013 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "128", 2024-09-21T03:56:33.013 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.013 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.013 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.013 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "default pg_num for any created local pool", 2024-09-21T03:56:33.013 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.013 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.013 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.013 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.013 INFO:teuthology.orchestra.run.smithi039.stdout: "prefix": { 2024-09-21T03:56:33.013 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "prefix", 2024-09-21T03:56:33.014 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-09-21T03:56:33.014 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.014 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:33.014 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-09-21T03:56:33.014 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.014 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.014 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.014 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "name prefix for any created local pool", 2024-09-21T03:56:33.014 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.014 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.014 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.014 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.014 INFO:teuthology.orchestra.run.smithi039.stdout: "sqlite3_killpoint": { 2024-09-21T03:56:33.015 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:56:33.015 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-09-21T03:56:33.015 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "dev", 2024-09-21T03:56:33.015 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:33.015 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "0", 2024-09-21T03:56:33.015 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.015 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.015 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.015 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:33.015 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.015 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.015 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.015 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.015 INFO:teuthology.orchestra.run.smithi039.stdout: "subtree": { 2024-09-21T03:56:33.015 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "subtree", 2024-09-21T03:56:33.016 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-09-21T03:56:33.016 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.016 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:33.016 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "rack", 2024-09-21T03:56:33.016 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.016 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.016 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.016 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "CRUSH level for which to create a local pool", 2024-09-21T03:56:33.016 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-09-21T03:56:33.016 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.016 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.016 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-21T03:56:33.016 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-21T03:56:33.016 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.017 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-21T03:56:33.017 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "mds_autoscaler", 2024-09-21T03:56:33.017 INFO:teuthology.orchestra.run.smithi039.stdout: "can_run": true, 2024-09-21T03:56:33.017 INFO:teuthology.orchestra.run.smithi039.stdout: "error_string": "", 2024-09-21T03:56:33.017 INFO:teuthology.orchestra.run.smithi039.stdout: "module_options": { 2024-09-21T03:56:33.017 INFO:teuthology.orchestra.run.smithi039.stdout: "log_level": { 2024-09-21T03:56:33.017 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_level", 2024-09-21T03:56:33.017 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-09-21T03:56:33.017 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.017 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:33.017 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-09-21T03:56:33.017 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.017 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.017 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-09-21T03:56:33.018 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-09-21T03:56:33.018 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-09-21T03:56:33.018 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-09-21T03:56:33.018 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-09-21T03:56:33.018 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-09-21T03:56:33.018 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-09-21T03:56:33.018 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-09-21T03:56:33.018 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:33.018 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.018 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.018 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.018 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.018 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster": { 2024-09-21T03:56:33.019 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster", 2024-09-21T03:56:33.019 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-09-21T03:56:33.019 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.019 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:33.019 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-09-21T03:56:33.019 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.019 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.019 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.019 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:33.019 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.019 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.019 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.019 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.019 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster_level": { 2024-09-21T03:56:33.020 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster_level", 2024-09-21T03:56:33.020 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-09-21T03:56:33.020 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.020 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:33.020 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "info", 2024-09-21T03:56:33.020 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.020 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.020 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-09-21T03:56:33.020 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-09-21T03:56:33.020 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-09-21T03:56:33.020 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-09-21T03:56:33.020 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-09-21T03:56:33.020 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-09-21T03:56:33.020 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-09-21T03:56:33.021 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-09-21T03:56:33.021 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:33.021 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.021 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.021 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.021 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.021 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_file": { 2024-09-21T03:56:33.021 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_file", 2024-09-21T03:56:33.021 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-09-21T03:56:33.021 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.021 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:33.021 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-09-21T03:56:33.021 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.021 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.022 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.022 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:33.022 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.022 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.022 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.022 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.022 INFO:teuthology.orchestra.run.smithi039.stdout: "sqlite3_killpoint": { 2024-09-21T03:56:33.022 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:56:33.022 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-09-21T03:56:33.022 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "dev", 2024-09-21T03:56:33.022 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:33.022 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "0", 2024-09-21T03:56:33.022 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.022 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.023 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.023 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:33.023 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.023 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.023 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.023 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-21T03:56:33.023 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-21T03:56:33.023 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.023 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-21T03:56:33.023 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "mirroring", 2024-09-21T03:56:33.023 INFO:teuthology.orchestra.run.smithi039.stdout: "can_run": true, 2024-09-21T03:56:33.023 INFO:teuthology.orchestra.run.smithi039.stdout: "error_string": "", 2024-09-21T03:56:33.023 INFO:teuthology.orchestra.run.smithi039.stdout: "module_options": { 2024-09-21T03:56:33.023 INFO:teuthology.orchestra.run.smithi039.stdout: "log_level": { 2024-09-21T03:56:33.023 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_level", 2024-09-21T03:56:33.024 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-09-21T03:56:33.024 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.024 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:33.024 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-09-21T03:56:33.024 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.024 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.024 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-09-21T03:56:33.024 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-09-21T03:56:33.024 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-09-21T03:56:33.024 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-09-21T03:56:33.024 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-09-21T03:56:33.024 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-09-21T03:56:33.024 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-09-21T03:56:33.024 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-09-21T03:56:33.024 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:33.025 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.025 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.025 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.025 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.025 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster": { 2024-09-21T03:56:33.025 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster", 2024-09-21T03:56:33.025 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-09-21T03:56:33.025 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.025 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:33.025 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-09-21T03:56:33.025 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.025 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.025 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.025 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:33.026 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.026 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.026 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.026 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.026 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster_level": { 2024-09-21T03:56:33.026 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster_level", 2024-09-21T03:56:33.026 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-09-21T03:56:33.026 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.026 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:33.026 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "info", 2024-09-21T03:56:33.026 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.026 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.026 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-09-21T03:56:33.026 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-09-21T03:56:33.027 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-09-21T03:56:33.027 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-09-21T03:56:33.027 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-09-21T03:56:33.027 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-09-21T03:56:33.027 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-09-21T03:56:33.027 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-09-21T03:56:33.027 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:33.027 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.027 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.027 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.027 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.027 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_file": { 2024-09-21T03:56:33.027 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_file", 2024-09-21T03:56:33.027 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-09-21T03:56:33.028 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.028 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:33.028 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-09-21T03:56:33.028 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.028 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.028 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.028 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:33.028 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.028 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.028 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.028 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.028 INFO:teuthology.orchestra.run.smithi039.stdout: "sqlite3_killpoint": { 2024-09-21T03:56:33.028 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:56:33.028 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-09-21T03:56:33.029 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "dev", 2024-09-21T03:56:33.029 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:33.029 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "0", 2024-09-21T03:56:33.029 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.029 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.029 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.029 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:33.029 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.029 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.029 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.029 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-21T03:56:33.029 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-21T03:56:33.029 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.029 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-21T03:56:33.030 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "nfs", 2024-09-21T03:56:33.030 INFO:teuthology.orchestra.run.smithi039.stdout: "can_run": true, 2024-09-21T03:56:33.030 INFO:teuthology.orchestra.run.smithi039.stdout: "error_string": "", 2024-09-21T03:56:33.030 INFO:teuthology.orchestra.run.smithi039.stdout: "module_options": { 2024-09-21T03:56:33.030 INFO:teuthology.orchestra.run.smithi039.stdout: "log_level": { 2024-09-21T03:56:33.030 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_level", 2024-09-21T03:56:33.030 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-09-21T03:56:33.030 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.030 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:33.030 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-09-21T03:56:33.030 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.030 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.030 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-09-21T03:56:33.030 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-09-21T03:56:33.031 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-09-21T03:56:33.031 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-09-21T03:56:33.031 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-09-21T03:56:33.031 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-09-21T03:56:33.031 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-09-21T03:56:33.031 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-09-21T03:56:33.031 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:33.031 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.031 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.031 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.031 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.031 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster": { 2024-09-21T03:56:33.031 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster", 2024-09-21T03:56:33.031 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-09-21T03:56:33.031 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.032 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:33.032 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-09-21T03:56:33.032 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.032 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.032 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.032 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:33.032 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.032 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.032 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.032 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.032 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster_level": { 2024-09-21T03:56:33.032 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster_level", 2024-09-21T03:56:33.032 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-09-21T03:56:33.033 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.033 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:33.033 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "info", 2024-09-21T03:56:33.033 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.033 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.033 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-09-21T03:56:33.033 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-09-21T03:56:33.033 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-09-21T03:56:33.033 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-09-21T03:56:33.033 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-09-21T03:56:33.033 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-09-21T03:56:33.033 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-09-21T03:56:33.033 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-09-21T03:56:33.033 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:33.034 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.034 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.034 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.034 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.034 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_file": { 2024-09-21T03:56:33.034 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_file", 2024-09-21T03:56:33.034 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-09-21T03:56:33.034 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.034 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:33.034 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-09-21T03:56:33.034 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.034 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.034 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.034 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:33.035 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.035 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.035 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.035 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.035 INFO:teuthology.orchestra.run.smithi039.stdout: "sqlite3_killpoint": { 2024-09-21T03:56:33.035 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:56:33.035 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-09-21T03:56:33.035 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "dev", 2024-09-21T03:56:33.035 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:33.035 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "0", 2024-09-21T03:56:33.035 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.035 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.035 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.035 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:33.035 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.036 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.036 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.036 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-21T03:56:33.036 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-21T03:56:33.036 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.036 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-21T03:56:33.036 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "orchestrator", 2024-09-21T03:56:33.036 INFO:teuthology.orchestra.run.smithi039.stdout: "can_run": true, 2024-09-21T03:56:33.036 INFO:teuthology.orchestra.run.smithi039.stdout: "error_string": "", 2024-09-21T03:56:33.036 INFO:teuthology.orchestra.run.smithi039.stdout: "module_options": { 2024-09-21T03:56:33.036 INFO:teuthology.orchestra.run.smithi039.stdout: "fail_fs": { 2024-09-21T03:56:33.036 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "fail_fs", 2024-09-21T03:56:33.036 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-09-21T03:56:33.036 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.037 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-09-21T03:56:33.037 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-09-21T03:56:33.037 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.037 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.037 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.037 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-09-21T03:56:33.037 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.037 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.037 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.037 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.037 INFO:teuthology.orchestra.run.smithi039.stdout: "log_level": { 2024-09-21T03:56:33.037 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_level", 2024-09-21T03:56:33.037 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-09-21T03:56:33.037 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.037 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:33.038 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-09-21T03:56:33.038 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.038 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.038 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-09-21T03:56:33.038 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-09-21T03:56:33.038 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-09-21T03:56:33.038 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-09-21T03:56:33.038 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-09-21T03:56:33.038 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-09-21T03:56:33.038 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-09-21T03:56:33.038 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-09-21T03:56:33.038 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:33.038 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.038 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.039 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.039 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.039 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster": { 2024-09-21T03:56:33.039 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster", 2024-09-21T03:56:33.039 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-09-21T03:56:33.039 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.039 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:33.039 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-09-21T03:56:33.039 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.039 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.039 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.039 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:33.039 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.039 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.040 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.040 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.040 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster_level": { 2024-09-21T03:56:33.040 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster_level", 2024-09-21T03:56:33.040 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-09-21T03:56:33.040 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.040 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:33.040 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "info", 2024-09-21T03:56:33.040 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.040 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.040 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-09-21T03:56:33.040 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-09-21T03:56:33.040 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-09-21T03:56:33.040 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-09-21T03:56:33.040 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-09-21T03:56:33.041 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-09-21T03:56:33.041 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-09-21T03:56:33.041 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-09-21T03:56:33.041 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:33.041 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.041 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.041 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.041 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.041 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_file": { 2024-09-21T03:56:33.041 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_file", 2024-09-21T03:56:33.041 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-09-21T03:56:33.041 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.041 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:33.041 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-09-21T03:56:33.042 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.042 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.042 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.042 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:33.042 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.042 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.042 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.042 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.042 INFO:teuthology.orchestra.run.smithi039.stdout: "orchestrator": { 2024-09-21T03:56:33.042 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "orchestrator", 2024-09-21T03:56:33.042 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-09-21T03:56:33.042 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.042 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:33.042 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-09-21T03:56:33.042 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.043 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.043 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-09-21T03:56:33.043 INFO:teuthology.orchestra.run.smithi039.stdout: "cephadm", 2024-09-21T03:56:33.043 INFO:teuthology.orchestra.run.smithi039.stdout: "rook", 2024-09-21T03:56:33.043 INFO:teuthology.orchestra.run.smithi039.stdout: "test_orchestrator" 2024-09-21T03:56:33.043 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-09-21T03:56:33.043 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Orchestrator backend", 2024-09-21T03:56:33.043 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.043 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.043 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.043 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.043 INFO:teuthology.orchestra.run.smithi039.stdout: "sqlite3_killpoint": { 2024-09-21T03:56:33.043 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:56:33.043 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-09-21T03:56:33.044 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "dev", 2024-09-21T03:56:33.044 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:33.044 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "0", 2024-09-21T03:56:33.044 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.044 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.044 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.044 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:33.044 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.044 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.044 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.044 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-21T03:56:33.044 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-21T03:56:33.044 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.044 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-21T03:56:33.044 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "osd_perf_query", 2024-09-21T03:56:33.045 INFO:teuthology.orchestra.run.smithi039.stdout: "can_run": true, 2024-09-21T03:56:33.045 INFO:teuthology.orchestra.run.smithi039.stdout: "error_string": "", 2024-09-21T03:56:33.045 INFO:teuthology.orchestra.run.smithi039.stdout: "module_options": { 2024-09-21T03:56:33.045 INFO:teuthology.orchestra.run.smithi039.stdout: "log_level": { 2024-09-21T03:56:33.045 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_level", 2024-09-21T03:56:33.045 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-09-21T03:56:33.045 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.045 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:33.045 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-09-21T03:56:33.045 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.045 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.045 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-09-21T03:56:33.045 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-09-21T03:56:33.045 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-09-21T03:56:33.045 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-09-21T03:56:33.046 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-09-21T03:56:33.046 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-09-21T03:56:33.046 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-09-21T03:56:33.046 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-09-21T03:56:33.046 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:33.046 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.046 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.046 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.046 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.046 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster": { 2024-09-21T03:56:33.046 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster", 2024-09-21T03:56:33.046 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-09-21T03:56:33.047 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.047 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:33.047 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-09-21T03:56:33.047 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.047 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.047 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.047 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:33.047 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.047 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.047 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.047 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.047 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster_level": { 2024-09-21T03:56:33.047 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster_level", 2024-09-21T03:56:33.047 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-09-21T03:56:33.048 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.048 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:33.048 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "info", 2024-09-21T03:56:33.048 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.048 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.048 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-09-21T03:56:33.048 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-09-21T03:56:33.048 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-09-21T03:56:33.048 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-09-21T03:56:33.048 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-09-21T03:56:33.048 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-09-21T03:56:33.048 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-09-21T03:56:33.048 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-09-21T03:56:33.048 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:33.049 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.049 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.049 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.049 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.049 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_file": { 2024-09-21T03:56:33.049 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_file", 2024-09-21T03:56:33.049 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-09-21T03:56:33.049 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.049 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:33.049 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-09-21T03:56:33.049 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.049 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.049 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.049 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:33.049 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.050 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.050 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.050 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.050 INFO:teuthology.orchestra.run.smithi039.stdout: "sqlite3_killpoint": { 2024-09-21T03:56:33.050 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:56:33.050 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-09-21T03:56:33.050 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "dev", 2024-09-21T03:56:33.050 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:33.050 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "0", 2024-09-21T03:56:33.050 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.050 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.050 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.050 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:33.050 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.051 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.051 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.051 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-21T03:56:33.051 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-21T03:56:33.051 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.051 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-21T03:56:33.051 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "osd_support", 2024-09-21T03:56:33.051 INFO:teuthology.orchestra.run.smithi039.stdout: "can_run": true, 2024-09-21T03:56:33.051 INFO:teuthology.orchestra.run.smithi039.stdout: "error_string": "", 2024-09-21T03:56:33.051 INFO:teuthology.orchestra.run.smithi039.stdout: "module_options": { 2024-09-21T03:56:33.051 INFO:teuthology.orchestra.run.smithi039.stdout: "log_level": { 2024-09-21T03:56:33.051 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_level", 2024-09-21T03:56:33.051 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-09-21T03:56:33.051 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.052 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:33.052 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-09-21T03:56:33.052 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.052 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.052 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-09-21T03:56:33.052 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-09-21T03:56:33.052 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-09-21T03:56:33.052 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-09-21T03:56:33.052 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-09-21T03:56:33.052 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-09-21T03:56:33.052 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-09-21T03:56:33.052 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-09-21T03:56:33.052 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:33.052 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.052 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.053 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.053 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.053 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster": { 2024-09-21T03:56:33.053 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster", 2024-09-21T03:56:33.053 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-09-21T03:56:33.053 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.053 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:33.053 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-09-21T03:56:33.053 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.053 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.053 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.053 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:33.053 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.053 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.053 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.054 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.054 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster_level": { 2024-09-21T03:56:33.054 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster_level", 2024-09-21T03:56:33.054 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-09-21T03:56:33.054 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.054 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:33.054 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "info", 2024-09-21T03:56:33.054 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.054 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.054 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-09-21T03:56:33.054 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-09-21T03:56:33.054 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-09-21T03:56:33.054 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-09-21T03:56:33.054 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-09-21T03:56:33.055 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-09-21T03:56:33.055 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-09-21T03:56:33.055 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-09-21T03:56:33.055 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:33.055 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.055 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.055 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.055 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.055 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_file": { 2024-09-21T03:56:33.055 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_file", 2024-09-21T03:56:33.055 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-09-21T03:56:33.055 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.055 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:33.055 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-09-21T03:56:33.056 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.056 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.056 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.056 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:33.056 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.056 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.056 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.056 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.056 INFO:teuthology.orchestra.run.smithi039.stdout: "sqlite3_killpoint": { 2024-09-21T03:56:33.056 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:56:33.056 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-09-21T03:56:33.056 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "dev", 2024-09-21T03:56:33.056 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:33.056 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "0", 2024-09-21T03:56:33.056 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.057 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.057 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.057 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:33.057 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.057 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.057 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.057 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-21T03:56:33.057 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-21T03:56:33.057 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.057 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-21T03:56:33.057 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "pg_autoscaler", 2024-09-21T03:56:33.057 INFO:teuthology.orchestra.run.smithi039.stdout: "can_run": true, 2024-09-21T03:56:33.057 INFO:teuthology.orchestra.run.smithi039.stdout: "error_string": "", 2024-09-21T03:56:33.057 INFO:teuthology.orchestra.run.smithi039.stdout: "module_options": { 2024-09-21T03:56:33.058 INFO:teuthology.orchestra.run.smithi039.stdout: "log_level": { 2024-09-21T03:56:33.058 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_level", 2024-09-21T03:56:33.058 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-09-21T03:56:33.058 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.058 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:33.058 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-09-21T03:56:33.058 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.058 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.058 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-09-21T03:56:33.058 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-09-21T03:56:33.058 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-09-21T03:56:33.058 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-09-21T03:56:33.058 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-09-21T03:56:33.058 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-09-21T03:56:33.059 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-09-21T03:56:33.059 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-09-21T03:56:33.059 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:33.059 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.059 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.059 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.059 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.059 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster": { 2024-09-21T03:56:33.059 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster", 2024-09-21T03:56:33.059 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-09-21T03:56:33.059 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.059 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:33.059 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-09-21T03:56:33.059 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.059 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.060 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.060 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:33.060 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.060 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.060 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.060 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.060 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster_level": { 2024-09-21T03:56:33.060 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster_level", 2024-09-21T03:56:33.060 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-09-21T03:56:33.060 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.060 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:33.060 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "info", 2024-09-21T03:56:33.060 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.061 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.061 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-09-21T03:56:33.061 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-09-21T03:56:33.061 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-09-21T03:56:33.061 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-09-21T03:56:33.061 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-09-21T03:56:33.061 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-09-21T03:56:33.061 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-09-21T03:56:33.061 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-09-21T03:56:33.061 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:33.061 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.061 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.061 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.061 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.062 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_file": { 2024-09-21T03:56:33.062 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_file", 2024-09-21T03:56:33.062 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-09-21T03:56:33.062 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.062 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:33.062 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-09-21T03:56:33.062 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.062 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.062 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.062 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:33.062 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.062 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.062 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.062 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.063 INFO:teuthology.orchestra.run.smithi039.stdout: "sleep_interval": { 2024-09-21T03:56:33.063 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "sleep_interval", 2024-09-21T03:56:33.063 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "secs", 2024-09-21T03:56:33.063 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.063 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-09-21T03:56:33.063 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "60", 2024-09-21T03:56:33.063 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.063 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.063 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.063 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:33.063 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.063 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.063 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.063 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.064 INFO:teuthology.orchestra.run.smithi039.stdout: "sqlite3_killpoint": { 2024-09-21T03:56:33.064 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:56:33.064 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-09-21T03:56:33.064 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "dev", 2024-09-21T03:56:33.064 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:33.064 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "0", 2024-09-21T03:56:33.064 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.064 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.064 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.064 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:33.064 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.064 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.064 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.064 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.064 INFO:teuthology.orchestra.run.smithi039.stdout: "threshold": { 2024-09-21T03:56:33.065 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "threshold", 2024-09-21T03:56:33.065 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "float", 2024-09-21T03:56:33.065 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.065 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-09-21T03:56:33.065 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "3.0", 2024-09-21T03:56:33.065 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "1.0", 2024-09-21T03:56:33.065 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.065 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.065 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "scaling threshold", 2024-09-21T03:56:33.065 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-09-21T03:56:33.065 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.065 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.065 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-21T03:56:33.065 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-21T03:56:33.065 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.066 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-21T03:56:33.066 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "progress", 2024-09-21T03:56:33.066 INFO:teuthology.orchestra.run.smithi039.stdout: "can_run": true, 2024-09-21T03:56:33.066 INFO:teuthology.orchestra.run.smithi039.stdout: "error_string": "", 2024-09-21T03:56:33.066 INFO:teuthology.orchestra.run.smithi039.stdout: "module_options": { 2024-09-21T03:56:33.066 INFO:teuthology.orchestra.run.smithi039.stdout: "allow_pg_recovery_event": { 2024-09-21T03:56:33.066 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "allow_pg_recovery_event", 2024-09-21T03:56:33.066 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-09-21T03:56:33.066 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.066 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:33.066 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-09-21T03:56:33.066 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.066 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.066 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.066 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "allow the module to show pg recovery progress", 2024-09-21T03:56:33.067 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.067 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.067 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.067 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.067 INFO:teuthology.orchestra.run.smithi039.stdout: "enabled": { 2024-09-21T03:56:33.067 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "enabled", 2024-09-21T03:56:33.067 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-09-21T03:56:33.067 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.067 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-09-21T03:56:33.067 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2024-09-21T03:56:33.067 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.067 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.067 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.067 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:33.068 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.068 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.068 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.068 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.068 INFO:teuthology.orchestra.run.smithi039.stdout: "log_level": { 2024-09-21T03:56:33.068 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_level", 2024-09-21T03:56:33.068 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-09-21T03:56:33.068 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.068 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:33.068 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-09-21T03:56:33.068 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.068 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.068 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-09-21T03:56:33.068 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-09-21T03:56:33.068 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-09-21T03:56:33.069 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-09-21T03:56:33.069 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-09-21T03:56:33.069 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-09-21T03:56:33.069 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-09-21T03:56:33.069 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-09-21T03:56:33.069 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:33.069 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.069 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.069 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.069 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.069 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster": { 2024-09-21T03:56:33.069 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster", 2024-09-21T03:56:33.069 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-09-21T03:56:33.069 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.069 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:33.070 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-09-21T03:56:33.070 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.070 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.070 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.070 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:33.070 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.070 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.070 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.070 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.070 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster_level": { 2024-09-21T03:56:33.070 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster_level", 2024-09-21T03:56:33.070 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-09-21T03:56:33.070 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.071 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:33.071 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "info", 2024-09-21T03:56:33.071 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.071 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.071 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-09-21T03:56:33.071 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-09-21T03:56:33.071 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-09-21T03:56:33.071 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-09-21T03:56:33.071 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-09-21T03:56:33.071 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-09-21T03:56:33.071 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-09-21T03:56:33.071 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-09-21T03:56:33.071 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:33.071 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.072 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.072 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.072 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.072 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_file": { 2024-09-21T03:56:33.072 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_file", 2024-09-21T03:56:33.072 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-09-21T03:56:33.072 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.072 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:33.072 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-09-21T03:56:33.072 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.072 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.072 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.072 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:33.072 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.072 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.073 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.073 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.073 INFO:teuthology.orchestra.run.smithi039.stdout: "max_completed_events": { 2024-09-21T03:56:33.073 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "max_completed_events", 2024-09-21T03:56:33.073 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-09-21T03:56:33.073 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.073 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:33.073 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "50", 2024-09-21T03:56:33.073 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.073 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.074 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.074 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "number of past completed events to remember", 2024-09-21T03:56:33.074 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.074 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.074 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.074 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.074 INFO:teuthology.orchestra.run.smithi039.stdout: "sleep_interval": { 2024-09-21T03:56:33.074 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "sleep_interval", 2024-09-21T03:56:33.074 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "secs", 2024-09-21T03:56:33.074 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.074 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:33.074 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "5", 2024-09-21T03:56:33.074 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.074 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.075 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.075 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "how long the module is going to sleep", 2024-09-21T03:56:33.075 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.075 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.075 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.075 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.075 INFO:teuthology.orchestra.run.smithi039.stdout: "sqlite3_killpoint": { 2024-09-21T03:56:33.075 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:56:33.075 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-09-21T03:56:33.075 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "dev", 2024-09-21T03:56:33.075 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:33.075 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "0", 2024-09-21T03:56:33.075 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.075 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.076 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.076 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:33.076 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.076 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.076 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.076 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-21T03:56:33.076 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-21T03:56:33.076 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.076 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-21T03:56:33.076 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "prometheus", 2024-09-21T03:56:33.076 INFO:teuthology.orchestra.run.smithi039.stdout: "can_run": true, 2024-09-21T03:56:33.076 INFO:teuthology.orchestra.run.smithi039.stdout: "error_string": "", 2024-09-21T03:56:33.076 INFO:teuthology.orchestra.run.smithi039.stdout: "module_options": { 2024-09-21T03:56:33.076 INFO:teuthology.orchestra.run.smithi039.stdout: "cache": { 2024-09-21T03:56:33.077 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "cache", 2024-09-21T03:56:33.077 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-09-21T03:56:33.077 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.077 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-09-21T03:56:33.077 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2024-09-21T03:56:33.077 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.077 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.077 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.077 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:33.077 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.077 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.077 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.077 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.078 INFO:teuthology.orchestra.run.smithi039.stdout: "exclude_perf_counters": { 2024-09-21T03:56:33.078 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "exclude_perf_counters", 2024-09-21T03:56:33.078 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-09-21T03:56:33.078 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.078 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:33.078 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2024-09-21T03:56:33.078 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.078 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.078 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.078 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-09-21T03:56:33.078 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-09-21T03:56:33.078 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.078 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.078 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.079 INFO:teuthology.orchestra.run.smithi039.stdout: "log_level": { 2024-09-21T03:56:33.079 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_level", 2024-09-21T03:56:33.079 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-09-21T03:56:33.079 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.079 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:33.079 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-09-21T03:56:33.079 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.079 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.079 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-09-21T03:56:33.079 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-09-21T03:56:33.079 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-09-21T03:56:33.079 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-09-21T03:56:33.079 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-09-21T03:56:33.079 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-09-21T03:56:33.080 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-09-21T03:56:33.080 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-09-21T03:56:33.080 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:33.080 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.080 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.080 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.080 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.080 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster": { 2024-09-21T03:56:33.080 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster", 2024-09-21T03:56:33.080 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-09-21T03:56:33.080 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.080 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:33.080 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-09-21T03:56:33.080 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.081 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.081 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.081 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:33.081 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.081 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.081 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.081 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.081 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster_level": { 2024-09-21T03:56:33.081 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster_level", 2024-09-21T03:56:33.081 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-09-21T03:56:33.081 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.081 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:33.081 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "info", 2024-09-21T03:56:33.081 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.081 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.082 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-09-21T03:56:33.082 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-09-21T03:56:33.082 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-09-21T03:56:33.082 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-09-21T03:56:33.082 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-09-21T03:56:33.082 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-09-21T03:56:33.082 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-09-21T03:56:33.082 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-09-21T03:56:33.082 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:33.082 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.082 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.082 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.082 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.082 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_file": { 2024-09-21T03:56:33.083 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_file", 2024-09-21T03:56:33.083 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-09-21T03:56:33.083 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.083 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:33.083 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-09-21T03:56:33.083 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.083 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.083 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.083 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:33.083 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.083 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.083 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.083 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.083 INFO:teuthology.orchestra.run.smithi039.stdout: "rbd_stats_pools": { 2024-09-21T03:56:33.084 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "rbd_stats_pools", 2024-09-21T03:56:33.084 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-09-21T03:56:33.084 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.084 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-09-21T03:56:33.084 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-09-21T03:56:33.084 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.084 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.084 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.084 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:33.084 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.084 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.084 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.084 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.085 INFO:teuthology.orchestra.run.smithi039.stdout: "rbd_stats_pools_refresh_interval": { 2024-09-21T03:56:33.085 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-09-21T03:56:33.085 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-09-21T03:56:33.085 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.085 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-09-21T03:56:33.085 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "300", 2024-09-21T03:56:33.085 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.085 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.085 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.085 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:33.085 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.085 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.085 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.085 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.085 INFO:teuthology.orchestra.run.smithi039.stdout: "scrape_interval": { 2024-09-21T03:56:33.086 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "scrape_interval", 2024-09-21T03:56:33.086 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "float", 2024-09-21T03:56:33.086 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.086 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-09-21T03:56:33.086 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "15.0", 2024-09-21T03:56:33.086 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.086 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.086 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.086 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:33.086 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.086 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.086 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.087 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.087 INFO:teuthology.orchestra.run.smithi039.stdout: "server_addr": { 2024-09-21T03:56:33.087 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "server_addr", 2024-09-21T03:56:33.087 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-09-21T03:56:33.087 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.087 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-09-21T03:56:33.087 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "::", 2024-09-21T03:56:33.087 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.087 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.087 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.087 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-09-21T03:56:33.087 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.087 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.088 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.088 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.088 INFO:teuthology.orchestra.run.smithi039.stdout: "server_port": { 2024-09-21T03:56:33.088 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "server_port", 2024-09-21T03:56:33.088 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-09-21T03:56:33.088 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.088 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:33.088 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "9283", 2024-09-21T03:56:33.088 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.088 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.088 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.088 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-09-21T03:56:33.088 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.089 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.089 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.089 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.089 INFO:teuthology.orchestra.run.smithi039.stdout: "sqlite3_killpoint": { 2024-09-21T03:56:33.089 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:56:33.089 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-09-21T03:56:33.089 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "dev", 2024-09-21T03:56:33.089 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:33.089 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "0", 2024-09-21T03:56:33.089 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.089 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.089 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.089 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:33.089 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.089 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.090 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.090 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.090 INFO:teuthology.orchestra.run.smithi039.stdout: "stale_cache_strategy": { 2024-09-21T03:56:33.090 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "stale_cache_strategy", 2024-09-21T03:56:33.090 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-09-21T03:56:33.090 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.090 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-09-21T03:56:33.090 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "log", 2024-09-21T03:56:33.090 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.090 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.090 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.090 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:33.090 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.091 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.091 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.091 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.091 INFO:teuthology.orchestra.run.smithi039.stdout: "standby_behaviour": { 2024-09-21T03:56:33.091 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "standby_behaviour", 2024-09-21T03:56:33.091 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-09-21T03:56:33.091 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.091 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:33.091 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "default", 2024-09-21T03:56:33.091 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.091 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.091 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-09-21T03:56:33.091 INFO:teuthology.orchestra.run.smithi039.stdout: "default", 2024-09-21T03:56:33.091 INFO:teuthology.orchestra.run.smithi039.stdout: "error" 2024-09-21T03:56:33.092 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-09-21T03:56:33.092 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:33.092 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.092 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.092 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.092 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.092 INFO:teuthology.orchestra.run.smithi039.stdout: "standby_error_status_code": { 2024-09-21T03:56:33.092 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "standby_error_status_code", 2024-09-21T03:56:33.092 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-09-21T03:56:33.092 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.092 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:33.092 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "500", 2024-09-21T03:56:33.092 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "400", 2024-09-21T03:56:33.092 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "599", 2024-09-21T03:56:33.093 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.093 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:33.093 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.093 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.093 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.093 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-21T03:56:33.093 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-21T03:56:33.093 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.093 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-21T03:56:33.093 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "rbd_support", 2024-09-21T03:56:33.093 INFO:teuthology.orchestra.run.smithi039.stdout: "can_run": true, 2024-09-21T03:56:33.093 INFO:teuthology.orchestra.run.smithi039.stdout: "error_string": "", 2024-09-21T03:56:33.093 INFO:teuthology.orchestra.run.smithi039.stdout: "module_options": { 2024-09-21T03:56:33.093 INFO:teuthology.orchestra.run.smithi039.stdout: "log_level": { 2024-09-21T03:56:33.093 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_level", 2024-09-21T03:56:33.094 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-09-21T03:56:33.094 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.094 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:33.094 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-09-21T03:56:33.094 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.094 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.094 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-09-21T03:56:33.094 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-09-21T03:56:33.094 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-09-21T03:56:33.094 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-09-21T03:56:33.094 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-09-21T03:56:33.094 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-09-21T03:56:33.094 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-09-21T03:56:33.094 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-09-21T03:56:33.094 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:33.095 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.095 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.095 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.095 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.095 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster": { 2024-09-21T03:56:33.095 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster", 2024-09-21T03:56:33.095 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-09-21T03:56:33.095 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.095 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:33.095 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-09-21T03:56:33.095 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.095 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.095 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.095 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:33.096 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.096 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.096 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.096 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.096 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster_level": { 2024-09-21T03:56:33.096 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster_level", 2024-09-21T03:56:33.096 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-09-21T03:56:33.096 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.096 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:33.096 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "info", 2024-09-21T03:56:33.096 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.096 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.096 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-09-21T03:56:33.096 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-09-21T03:56:33.096 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-09-21T03:56:33.097 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-09-21T03:56:33.097 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-09-21T03:56:33.097 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-09-21T03:56:33.097 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-09-21T03:56:33.097 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-09-21T03:56:33.097 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:33.097 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.097 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.097 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.097 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.097 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_file": { 2024-09-21T03:56:33.097 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_file", 2024-09-21T03:56:33.097 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-09-21T03:56:33.097 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.098 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:33.098 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-09-21T03:56:33.098 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.098 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.098 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.098 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:33.098 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.098 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.098 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.098 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.098 INFO:teuthology.orchestra.run.smithi039.stdout: "max_concurrent_snap_create": { 2024-09-21T03:56:33.098 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "max_concurrent_snap_create", 2024-09-21T03:56:33.098 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-09-21T03:56:33.098 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.099 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-09-21T03:56:33.099 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "10", 2024-09-21T03:56:33.099 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.099 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.099 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.099 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:33.099 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.099 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.099 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.099 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.099 INFO:teuthology.orchestra.run.smithi039.stdout: "mirror_snapshot_schedule": { 2024-09-21T03:56:33.099 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "mirror_snapshot_schedule", 2024-09-21T03:56:33.099 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-09-21T03:56:33.099 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.100 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-09-21T03:56:33.100 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-09-21T03:56:33.100 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.100 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.100 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.100 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:33.100 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.100 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.100 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.100 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.100 INFO:teuthology.orchestra.run.smithi039.stdout: "sqlite3_killpoint": { 2024-09-21T03:56:33.100 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:56:33.100 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-09-21T03:56:33.100 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "dev", 2024-09-21T03:56:33.101 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:33.101 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "0", 2024-09-21T03:56:33.101 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.101 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.101 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.101 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:33.101 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.101 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.101 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.101 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.101 INFO:teuthology.orchestra.run.smithi039.stdout: "trash_purge_schedule": { 2024-09-21T03:56:33.101 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "trash_purge_schedule", 2024-09-21T03:56:33.101 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-09-21T03:56:33.101 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.101 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-09-21T03:56:33.102 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-09-21T03:56:33.102 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.102 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.102 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.102 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:33.102 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.102 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.102 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.102 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-21T03:56:33.102 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-21T03:56:33.102 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.102 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-21T03:56:33.102 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "restful", 2024-09-21T03:56:33.102 INFO:teuthology.orchestra.run.smithi039.stdout: "can_run": true, 2024-09-21T03:56:33.103 INFO:teuthology.orchestra.run.smithi039.stdout: "error_string": "", 2024-09-21T03:56:33.103 INFO:teuthology.orchestra.run.smithi039.stdout: "module_options": { 2024-09-21T03:56:33.103 INFO:teuthology.orchestra.run.smithi039.stdout: "enable_auth": { 2024-09-21T03:56:33.103 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "enable_auth", 2024-09-21T03:56:33.103 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-09-21T03:56:33.103 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.103 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-09-21T03:56:33.103 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2024-09-21T03:56:33.103 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.103 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.103 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.103 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:33.103 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.103 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.104 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.104 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.104 INFO:teuthology.orchestra.run.smithi039.stdout: "key_file": { 2024-09-21T03:56:33.104 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "key_file", 2024-09-21T03:56:33.104 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-09-21T03:56:33.104 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.104 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-09-21T03:56:33.104 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-09-21T03:56:33.104 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.104 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.104 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.104 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:33.104 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.104 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.105 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.105 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.105 INFO:teuthology.orchestra.run.smithi039.stdout: "log_level": { 2024-09-21T03:56:33.105 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_level", 2024-09-21T03:56:33.105 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-09-21T03:56:33.105 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.105 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:33.105 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-09-21T03:56:33.105 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.105 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.105 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-09-21T03:56:33.105 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-09-21T03:56:33.105 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-09-21T03:56:33.105 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-09-21T03:56:33.105 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-09-21T03:56:33.106 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-09-21T03:56:33.106 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-09-21T03:56:33.106 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-09-21T03:56:33.106 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:33.106 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.106 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.106 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.106 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.106 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster": { 2024-09-21T03:56:33.106 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster", 2024-09-21T03:56:33.106 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-09-21T03:56:33.106 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.106 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:33.106 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-09-21T03:56:33.107 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.107 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.107 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.107 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:33.107 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.107 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.107 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.107 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.107 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster_level": { 2024-09-21T03:56:33.107 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster_level", 2024-09-21T03:56:33.107 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-09-21T03:56:33.107 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.107 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:33.107 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "info", 2024-09-21T03:56:33.108 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.108 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.108 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-09-21T03:56:33.108 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-09-21T03:56:33.108 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-09-21T03:56:33.108 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-09-21T03:56:33.108 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-09-21T03:56:33.108 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-09-21T03:56:33.108 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-09-21T03:56:33.108 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-09-21T03:56:33.108 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:33.108 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.108 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.108 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.108 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.109 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_file": { 2024-09-21T03:56:33.109 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_file", 2024-09-21T03:56:33.109 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-09-21T03:56:33.109 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.109 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:33.109 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-09-21T03:56:33.109 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.109 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.109 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.109 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:33.109 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.109 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.109 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.109 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.109 INFO:teuthology.orchestra.run.smithi039.stdout: "max_requests": { 2024-09-21T03:56:33.110 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "max_requests", 2024-09-21T03:56:33.110 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-09-21T03:56:33.110 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.110 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-09-21T03:56:33.110 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "500", 2024-09-21T03:56:33.110 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.110 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.110 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.110 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Maximum number of requests to keep in memory. When new request comes in, the oldest request will be removed if the number of requests exceeds the max request number.if un-finished request is removed, error message will be logged in the ceph-mgr log.", 2024-09-21T03:56:33.110 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.110 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.110 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.110 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.110 INFO:teuthology.orchestra.run.smithi039.stdout: "server_addr": { 2024-09-21T03:56:33.111 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "server_addr", 2024-09-21T03:56:33.111 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-09-21T03:56:33.111 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.111 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-09-21T03:56:33.111 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-09-21T03:56:33.111 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.111 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.111 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.111 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:33.111 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.111 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.111 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.111 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.111 INFO:teuthology.orchestra.run.smithi039.stdout: "server_port": { 2024-09-21T03:56:33.111 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "server_port", 2024-09-21T03:56:33.112 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-09-21T03:56:33.112 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.112 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-09-21T03:56:33.112 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-09-21T03:56:33.112 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.112 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.112 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.112 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:33.112 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.112 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.112 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.112 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.112 INFO:teuthology.orchestra.run.smithi039.stdout: "sqlite3_killpoint": { 2024-09-21T03:56:33.112 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:56:33.112 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-09-21T03:56:33.113 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "dev", 2024-09-21T03:56:33.113 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:33.113 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "0", 2024-09-21T03:56:33.113 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.113 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.113 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.113 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:33.113 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.113 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.113 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.113 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-21T03:56:33.113 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-21T03:56:33.113 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.114 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-21T03:56:33.114 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "rgw", 2024-09-21T03:56:33.114 INFO:teuthology.orchestra.run.smithi039.stdout: "can_run": true, 2024-09-21T03:56:33.114 INFO:teuthology.orchestra.run.smithi039.stdout: "error_string": "", 2024-09-21T03:56:33.114 INFO:teuthology.orchestra.run.smithi039.stdout: "module_options": { 2024-09-21T03:56:33.114 INFO:teuthology.orchestra.run.smithi039.stdout: "log_level": { 2024-09-21T03:56:33.114 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_level", 2024-09-21T03:56:33.114 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-09-21T03:56:33.114 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.114 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:33.114 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-09-21T03:56:33.114 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.114 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.115 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-09-21T03:56:33.115 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-09-21T03:56:33.115 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-09-21T03:56:33.115 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-09-21T03:56:33.115 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-09-21T03:56:33.115 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-09-21T03:56:33.115 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-09-21T03:56:33.115 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-09-21T03:56:33.115 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:33.115 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.115 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.115 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.115 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.115 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster": { 2024-09-21T03:56:33.116 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster", 2024-09-21T03:56:33.116 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-09-21T03:56:33.116 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.116 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:33.116 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-09-21T03:56:33.116 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.116 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.116 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.116 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:33.116 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.116 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.116 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.116 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.116 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster_level": { 2024-09-21T03:56:33.116 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster_level", 2024-09-21T03:56:33.117 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-09-21T03:56:33.117 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.117 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:33.117 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "info", 2024-09-21T03:56:33.117 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.117 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.117 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-09-21T03:56:33.117 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-09-21T03:56:33.117 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-09-21T03:56:33.117 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-09-21T03:56:33.117 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-09-21T03:56:33.117 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-09-21T03:56:33.117 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-09-21T03:56:33.117 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-09-21T03:56:33.118 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:33.118 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.118 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.118 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.118 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.118 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_file": { 2024-09-21T03:56:33.118 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_file", 2024-09-21T03:56:33.118 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-09-21T03:56:33.118 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.118 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:33.118 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-09-21T03:56:33.118 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.118 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.118 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.119 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:33.119 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.119 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.119 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.119 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.119 INFO:teuthology.orchestra.run.smithi039.stdout: "secondary_zone_period_retry_limit": { 2024-09-21T03:56:33.119 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "secondary_zone_period_retry_limit", 2024-09-21T03:56:33.119 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-09-21T03:56:33.119 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.119 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-09-21T03:56:33.119 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "5", 2024-09-21T03:56:33.119 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.119 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.119 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.120 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "RGW module period update retry limit for secondary site", 2024-09-21T03:56:33.120 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.120 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.120 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.120 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.120 INFO:teuthology.orchestra.run.smithi039.stdout: "sqlite3_killpoint": { 2024-09-21T03:56:33.120 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:56:33.120 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-09-21T03:56:33.120 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "dev", 2024-09-21T03:56:33.120 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:33.120 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "0", 2024-09-21T03:56:33.120 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.120 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.120 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.121 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:33.121 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.121 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.121 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.121 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-21T03:56:33.121 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-21T03:56:33.121 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.121 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-21T03:56:33.121 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "rook", 2024-09-21T03:56:33.121 INFO:teuthology.orchestra.run.smithi039.stdout: "can_run": true, 2024-09-21T03:56:33.121 INFO:teuthology.orchestra.run.smithi039.stdout: "error_string": "", 2024-09-21T03:56:33.121 INFO:teuthology.orchestra.run.smithi039.stdout: "module_options": { 2024-09-21T03:56:33.121 INFO:teuthology.orchestra.run.smithi039.stdout: "log_level": { 2024-09-21T03:56:33.121 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_level", 2024-09-21T03:56:33.121 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-09-21T03:56:33.122 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.122 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:33.122 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-09-21T03:56:33.122 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.122 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.122 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-09-21T03:56:33.122 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-09-21T03:56:33.122 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-09-21T03:56:33.122 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-09-21T03:56:33.122 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-09-21T03:56:33.122 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-09-21T03:56:33.122 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-09-21T03:56:33.122 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-09-21T03:56:33.122 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:33.123 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.123 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.123 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.123 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.123 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster": { 2024-09-21T03:56:33.123 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster", 2024-09-21T03:56:33.123 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-09-21T03:56:33.123 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.123 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:33.123 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-09-21T03:56:33.123 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.123 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.123 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.123 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:33.124 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.124 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.124 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.124 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.124 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster_level": { 2024-09-21T03:56:33.124 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster_level", 2024-09-21T03:56:33.124 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-09-21T03:56:33.124 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.124 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:33.124 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "info", 2024-09-21T03:56:33.124 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.124 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.124 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-09-21T03:56:33.124 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-09-21T03:56:33.125 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-09-21T03:56:33.125 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-09-21T03:56:33.125 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-09-21T03:56:33.125 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-09-21T03:56:33.125 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-09-21T03:56:33.125 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-09-21T03:56:33.125 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:33.125 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.125 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.125 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.125 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.125 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_file": { 2024-09-21T03:56:33.125 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_file", 2024-09-21T03:56:33.125 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-09-21T03:56:33.126 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.126 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:33.126 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-09-21T03:56:33.126 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.126 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.126 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.126 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:33.126 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.126 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.126 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.126 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.126 INFO:teuthology.orchestra.run.smithi039.stdout: "sqlite3_killpoint": { 2024-09-21T03:56:33.126 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:56:33.127 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-09-21T03:56:33.127 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "dev", 2024-09-21T03:56:33.127 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:33.127 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "0", 2024-09-21T03:56:33.127 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.127 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.127 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.127 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:33.127 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.127 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.127 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.127 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.127 INFO:teuthology.orchestra.run.smithi039.stdout: "storage_class": { 2024-09-21T03:56:33.128 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "storage_class", 2024-09-21T03:56:33.128 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-09-21T03:56:33.128 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.128 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-09-21T03:56:33.128 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "local", 2024-09-21T03:56:33.128 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.128 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.128 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.128 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-09-21T03:56:33.128 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.128 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.128 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.128 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-21T03:56:33.128 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-21T03:56:33.129 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.129 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-21T03:56:33.129 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "selftest", 2024-09-21T03:56:33.129 INFO:teuthology.orchestra.run.smithi039.stdout: "can_run": true, 2024-09-21T03:56:33.129 INFO:teuthology.orchestra.run.smithi039.stdout: "error_string": "", 2024-09-21T03:56:33.129 INFO:teuthology.orchestra.run.smithi039.stdout: "module_options": { 2024-09-21T03:56:33.129 INFO:teuthology.orchestra.run.smithi039.stdout: "log_level": { 2024-09-21T03:56:33.129 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_level", 2024-09-21T03:56:33.129 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-09-21T03:56:33.129 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.129 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:33.129 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-09-21T03:56:33.129 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.129 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.129 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-09-21T03:56:33.130 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-09-21T03:56:33.130 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-09-21T03:56:33.130 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-09-21T03:56:33.130 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-09-21T03:56:33.130 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-09-21T03:56:33.130 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-09-21T03:56:33.130 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-09-21T03:56:33.130 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:33.130 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.130 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.130 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.130 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.130 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster": { 2024-09-21T03:56:33.131 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster", 2024-09-21T03:56:33.131 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-09-21T03:56:33.131 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.131 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:33.131 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-09-21T03:56:33.131 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.131 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.131 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.131 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:33.131 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.131 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.131 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.131 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.131 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster_level": { 2024-09-21T03:56:33.132 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster_level", 2024-09-21T03:56:33.132 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-09-21T03:56:33.132 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.132 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:33.132 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "info", 2024-09-21T03:56:33.132 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.132 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.132 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-09-21T03:56:33.132 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-09-21T03:56:33.132 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-09-21T03:56:33.132 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-09-21T03:56:33.132 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-09-21T03:56:33.132 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-09-21T03:56:33.132 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-09-21T03:56:33.133 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-09-21T03:56:33.133 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:33.133 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.133 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.133 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.133 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.133 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_file": { 2024-09-21T03:56:33.133 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_file", 2024-09-21T03:56:33.133 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-09-21T03:56:33.133 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.133 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:33.133 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-09-21T03:56:33.133 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.133 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.134 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.134 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:33.134 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.134 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.134 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.134 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.134 INFO:teuthology.orchestra.run.smithi039.stdout: "roption1": { 2024-09-21T03:56:33.134 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "roption1", 2024-09-21T03:56:33.134 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-09-21T03:56:33.134 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.134 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-09-21T03:56:33.134 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-09-21T03:56:33.134 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.134 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.135 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.135 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:33.135 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.135 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.135 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.135 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.135 INFO:teuthology.orchestra.run.smithi039.stdout: "roption2": { 2024-09-21T03:56:33.135 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "roption2", 2024-09-21T03:56:33.135 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-09-21T03:56:33.135 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.135 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-09-21T03:56:33.135 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "xyz", 2024-09-21T03:56:33.135 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.135 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.135 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.136 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:33.136 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.136 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.136 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.136 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.136 INFO:teuthology.orchestra.run.smithi039.stdout: "rwoption1": { 2024-09-21T03:56:33.136 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "rwoption1", 2024-09-21T03:56:33.136 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-09-21T03:56:33.136 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.136 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-09-21T03:56:33.136 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-09-21T03:56:33.136 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.136 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.136 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.137 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:33.137 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.137 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.137 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.137 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.137 INFO:teuthology.orchestra.run.smithi039.stdout: "rwoption2": { 2024-09-21T03:56:33.137 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "rwoption2", 2024-09-21T03:56:33.137 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-09-21T03:56:33.137 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.137 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-09-21T03:56:33.137 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-09-21T03:56:33.137 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.137 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.137 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.137 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:33.138 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.138 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.138 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.138 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.138 INFO:teuthology.orchestra.run.smithi039.stdout: "rwoption3": { 2024-09-21T03:56:33.138 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "rwoption3", 2024-09-21T03:56:33.138 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "float", 2024-09-21T03:56:33.138 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.138 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-09-21T03:56:33.138 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-09-21T03:56:33.138 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.138 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.138 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.138 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:33.139 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.139 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.139 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.139 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.139 INFO:teuthology.orchestra.run.smithi039.stdout: "rwoption4": { 2024-09-21T03:56:33.139 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "rwoption4", 2024-09-21T03:56:33.139 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-09-21T03:56:33.139 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.139 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-09-21T03:56:33.139 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-09-21T03:56:33.139 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.139 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.139 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.139 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:33.139 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.140 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.140 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.140 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.140 INFO:teuthology.orchestra.run.smithi039.stdout: "rwoption5": { 2024-09-21T03:56:33.140 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "rwoption5", 2024-09-21T03:56:33.140 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-09-21T03:56:33.140 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.140 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-09-21T03:56:33.140 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-09-21T03:56:33.140 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.140 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.140 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.140 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:33.140 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.141 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.141 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.141 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.141 INFO:teuthology.orchestra.run.smithi039.stdout: "rwoption6": { 2024-09-21T03:56:33.141 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "rwoption6", 2024-09-21T03:56:33.141 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-09-21T03:56:33.141 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.141 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-09-21T03:56:33.141 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2024-09-21T03:56:33.141 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.141 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.141 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.141 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:33.142 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.142 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.142 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.142 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.142 INFO:teuthology.orchestra.run.smithi039.stdout: "rwoption7": { 2024-09-21T03:56:33.142 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "rwoption7", 2024-09-21T03:56:33.142 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-09-21T03:56:33.142 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.142 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-09-21T03:56:33.142 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-09-21T03:56:33.142 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "1", 2024-09-21T03:56:33.142 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "42", 2024-09-21T03:56:33.142 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.142 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:33.143 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.143 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.143 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.143 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.143 INFO:teuthology.orchestra.run.smithi039.stdout: "sqlite3_killpoint": { 2024-09-21T03:56:33.143 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:56:33.143 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-09-21T03:56:33.143 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "dev", 2024-09-21T03:56:33.143 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:33.143 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "0", 2024-09-21T03:56:33.143 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.143 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.143 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.143 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:33.144 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.144 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.144 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.144 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.144 INFO:teuthology.orchestra.run.smithi039.stdout: "testkey": { 2024-09-21T03:56:33.144 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "testkey", 2024-09-21T03:56:33.144 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-09-21T03:56:33.144 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.144 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-09-21T03:56:33.144 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-09-21T03:56:33.144 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.144 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.144 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.144 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:33.145 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.145 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.145 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.145 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.145 INFO:teuthology.orchestra.run.smithi039.stdout: "testlkey": { 2024-09-21T03:56:33.145 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "testlkey", 2024-09-21T03:56:33.145 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-09-21T03:56:33.145 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.145 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-09-21T03:56:33.145 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-09-21T03:56:33.145 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.145 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.145 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.145 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:33.146 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.146 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.146 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.146 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.146 INFO:teuthology.orchestra.run.smithi039.stdout: "testnewline": { 2024-09-21T03:56:33.146 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "testnewline", 2024-09-21T03:56:33.146 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-09-21T03:56:33.146 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.146 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-09-21T03:56:33.146 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-09-21T03:56:33.146 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.146 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.146 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.146 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:33.147 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.147 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.147 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.147 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-21T03:56:33.147 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-21T03:56:33.147 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.147 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-21T03:56:33.147 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "smb", 2024-09-21T03:56:33.147 INFO:teuthology.orchestra.run.smithi039.stdout: "can_run": true, 2024-09-21T03:56:33.147 INFO:teuthology.orchestra.run.smithi039.stdout: "error_string": "", 2024-09-21T03:56:33.147 INFO:teuthology.orchestra.run.smithi039.stdout: "module_options": { 2024-09-21T03:56:33.147 INFO:teuthology.orchestra.run.smithi039.stdout: "internal_store_backend": { 2024-09-21T03:56:33.147 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "internal_store_backend", 2024-09-21T03:56:33.147 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-09-21T03:56:33.148 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "dev", 2024-09-21T03:56:33.148 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-09-21T03:56:33.148 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-09-21T03:56:33.148 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.148 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.148 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.148 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "set internal store backend. for develoment and testing only", 2024-09-21T03:56:33.148 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.148 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.148 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.148 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.148 INFO:teuthology.orchestra.run.smithi039.stdout: "log_level": { 2024-09-21T03:56:33.148 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_level", 2024-09-21T03:56:33.148 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-09-21T03:56:33.149 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.149 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:33.149 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-09-21T03:56:33.149 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.149 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.149 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-09-21T03:56:33.149 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-09-21T03:56:33.149 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-09-21T03:56:33.149 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-09-21T03:56:33.149 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-09-21T03:56:33.149 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-09-21T03:56:33.149 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-09-21T03:56:33.149 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-09-21T03:56:33.149 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:33.150 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.150 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.150 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.150 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.150 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster": { 2024-09-21T03:56:33.150 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster", 2024-09-21T03:56:33.150 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-09-21T03:56:33.150 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.150 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:33.150 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-09-21T03:56:33.150 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.150 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.150 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.150 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:33.150 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.151 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.151 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.151 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.151 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster_level": { 2024-09-21T03:56:33.151 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster_level", 2024-09-21T03:56:33.151 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-09-21T03:56:33.151 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.151 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:33.151 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "info", 2024-09-21T03:56:33.151 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.151 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.151 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-09-21T03:56:33.151 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-09-21T03:56:33.151 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-09-21T03:56:33.152 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-09-21T03:56:33.152 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-09-21T03:56:33.152 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-09-21T03:56:33.152 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-09-21T03:56:33.152 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-09-21T03:56:33.152 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:33.152 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.152 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.152 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.152 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.152 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_file": { 2024-09-21T03:56:33.152 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_file", 2024-09-21T03:56:33.152 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-09-21T03:56:33.152 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.152 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:33.153 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-09-21T03:56:33.153 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.153 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.153 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.153 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:33.153 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.153 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.153 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.153 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.153 INFO:teuthology.orchestra.run.smithi039.stdout: "sqlite3_killpoint": { 2024-09-21T03:56:33.153 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:56:33.153 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-09-21T03:56:33.153 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "dev", 2024-09-21T03:56:33.153 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:33.154 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "0", 2024-09-21T03:56:33.154 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.154 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.154 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.154 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:33.154 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.154 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.154 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.154 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.154 INFO:teuthology.orchestra.run.smithi039.stdout: "update_orchestration": { 2024-09-21T03:56:33.154 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "update_orchestration", 2024-09-21T03:56:33.154 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-09-21T03:56:33.155 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.155 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-09-21T03:56:33.155 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2024-09-21T03:56:33.155 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.155 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.155 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.155 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "automatically update orchestration when smb resources are changed", 2024-09-21T03:56:33.155 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.155 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.155 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.155 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-21T03:56:33.155 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-21T03:56:33.155 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.155 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-21T03:56:33.156 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "snap_schedule", 2024-09-21T03:56:33.156 INFO:teuthology.orchestra.run.smithi039.stdout: "can_run": true, 2024-09-21T03:56:33.156 INFO:teuthology.orchestra.run.smithi039.stdout: "error_string": "", 2024-09-21T03:56:33.156 INFO:teuthology.orchestra.run.smithi039.stdout: "module_options": { 2024-09-21T03:56:33.156 INFO:teuthology.orchestra.run.smithi039.stdout: "allow_m_granularity": { 2024-09-21T03:56:33.156 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "allow_m_granularity", 2024-09-21T03:56:33.156 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-09-21T03:56:33.156 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.156 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:33.156 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-09-21T03:56:33.156 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.156 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.156 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.157 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "allow minute scheduled snapshots", 2024-09-21T03:56:33.157 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.157 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.157 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.157 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.157 INFO:teuthology.orchestra.run.smithi039.stdout: "dump_on_update": { 2024-09-21T03:56:33.157 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "dump_on_update", 2024-09-21T03:56:33.157 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-09-21T03:56:33.157 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.157 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:33.157 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-09-21T03:56:33.157 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.157 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.157 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.158 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "dump database to debug log on update", 2024-09-21T03:56:33.158 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.158 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.158 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.158 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.158 INFO:teuthology.orchestra.run.smithi039.stdout: "log_level": { 2024-09-21T03:56:33.158 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_level", 2024-09-21T03:56:33.158 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-09-21T03:56:33.158 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.158 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:33.158 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-09-21T03:56:33.158 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.158 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.158 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-09-21T03:56:33.159 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-09-21T03:56:33.159 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-09-21T03:56:33.159 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-09-21T03:56:33.159 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-09-21T03:56:33.159 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-09-21T03:56:33.159 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-09-21T03:56:33.159 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-09-21T03:56:33.159 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:33.159 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.159 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.159 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.159 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.159 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster": { 2024-09-21T03:56:33.159 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster", 2024-09-21T03:56:33.160 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-09-21T03:56:33.160 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.160 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:33.160 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-09-21T03:56:33.160 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.160 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.160 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.160 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:33.160 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.160 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.160 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.160 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.160 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster_level": { 2024-09-21T03:56:33.160 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster_level", 2024-09-21T03:56:33.160 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-09-21T03:56:33.161 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.161 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:33.161 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "info", 2024-09-21T03:56:33.161 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.161 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.161 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-09-21T03:56:33.161 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-09-21T03:56:33.161 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-09-21T03:56:33.161 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-09-21T03:56:33.161 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-09-21T03:56:33.161 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-09-21T03:56:33.161 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-09-21T03:56:33.161 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-09-21T03:56:33.161 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:33.162 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.162 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.162 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.162 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.162 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_file": { 2024-09-21T03:56:33.162 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_file", 2024-09-21T03:56:33.162 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-09-21T03:56:33.162 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.162 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:33.162 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-09-21T03:56:33.162 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.162 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.162 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.162 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:33.163 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.163 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.163 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.163 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.163 INFO:teuthology.orchestra.run.smithi039.stdout: "sqlite3_killpoint": { 2024-09-21T03:56:33.163 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:56:33.163 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-09-21T03:56:33.163 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "dev", 2024-09-21T03:56:33.163 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:33.163 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "0", 2024-09-21T03:56:33.163 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.163 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.163 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.163 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:33.163 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.164 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.164 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.164 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-21T03:56:33.164 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-21T03:56:33.164 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.164 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-21T03:56:33.164 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "stats", 2024-09-21T03:56:33.164 INFO:teuthology.orchestra.run.smithi039.stdout: "can_run": true, 2024-09-21T03:56:33.164 INFO:teuthology.orchestra.run.smithi039.stdout: "error_string": "", 2024-09-21T03:56:33.164 INFO:teuthology.orchestra.run.smithi039.stdout: "module_options": { 2024-09-21T03:56:33.164 INFO:teuthology.orchestra.run.smithi039.stdout: "log_level": { 2024-09-21T03:56:33.164 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_level", 2024-09-21T03:56:33.164 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-09-21T03:56:33.164 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.165 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:33.165 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-09-21T03:56:33.165 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.165 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.165 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-09-21T03:56:33.165 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-09-21T03:56:33.165 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-09-21T03:56:33.165 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-09-21T03:56:33.165 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-09-21T03:56:33.165 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-09-21T03:56:33.165 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-09-21T03:56:33.165 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-09-21T03:56:33.165 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:33.165 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.165 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.166 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.166 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.166 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster": { 2024-09-21T03:56:33.166 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster", 2024-09-21T03:56:33.166 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-09-21T03:56:33.166 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.166 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:33.166 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-09-21T03:56:33.166 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.166 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.166 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.166 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:33.166 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.166 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.167 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.167 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.167 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster_level": { 2024-09-21T03:56:33.167 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster_level", 2024-09-21T03:56:33.167 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-09-21T03:56:33.167 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.167 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:33.167 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "info", 2024-09-21T03:56:33.167 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.167 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.167 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-09-21T03:56:33.167 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-09-21T03:56:33.167 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-09-21T03:56:33.167 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-09-21T03:56:33.167 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-09-21T03:56:33.168 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-09-21T03:56:33.168 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-09-21T03:56:33.168 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-09-21T03:56:33.168 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:33.168 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.168 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.168 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.168 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.168 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_file": { 2024-09-21T03:56:33.168 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_file", 2024-09-21T03:56:33.168 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-09-21T03:56:33.168 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.168 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:33.169 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-09-21T03:56:33.169 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.169 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.169 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.169 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:33.169 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.169 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.169 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.169 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.169 INFO:teuthology.orchestra.run.smithi039.stdout: "sqlite3_killpoint": { 2024-09-21T03:56:33.169 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:56:33.169 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-09-21T03:56:33.169 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "dev", 2024-09-21T03:56:33.170 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:33.170 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "0", 2024-09-21T03:56:33.170 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.170 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.170 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.170 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:33.170 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.170 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.170 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.170 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-21T03:56:33.170 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-21T03:56:33.170 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.170 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-21T03:56:33.170 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "status", 2024-09-21T03:56:33.170 INFO:teuthology.orchestra.run.smithi039.stdout: "can_run": true, 2024-09-21T03:56:33.171 INFO:teuthology.orchestra.run.smithi039.stdout: "error_string": "", 2024-09-21T03:56:33.171 INFO:teuthology.orchestra.run.smithi039.stdout: "module_options": { 2024-09-21T03:56:33.171 INFO:teuthology.orchestra.run.smithi039.stdout: "log_level": { 2024-09-21T03:56:33.171 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_level", 2024-09-21T03:56:33.171 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-09-21T03:56:33.171 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.171 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:33.171 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-09-21T03:56:33.171 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.171 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.171 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-09-21T03:56:33.171 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-09-21T03:56:33.171 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-09-21T03:56:33.171 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-09-21T03:56:33.172 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-09-21T03:56:33.172 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-09-21T03:56:33.172 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-09-21T03:56:33.172 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-09-21T03:56:33.172 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:33.172 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.172 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.172 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.172 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.172 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster": { 2024-09-21T03:56:33.172 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster", 2024-09-21T03:56:33.172 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-09-21T03:56:33.172 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.172 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:33.173 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-09-21T03:56:33.173 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.173 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.173 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.173 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:33.173 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.173 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.173 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.173 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.173 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster_level": { 2024-09-21T03:56:33.173 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster_level", 2024-09-21T03:56:33.173 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-09-21T03:56:33.173 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.173 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:33.173 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "info", 2024-09-21T03:56:33.174 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.174 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.174 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-09-21T03:56:33.174 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-09-21T03:56:33.174 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-09-21T03:56:33.174 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-09-21T03:56:33.174 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-09-21T03:56:33.174 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-09-21T03:56:33.174 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-09-21T03:56:33.174 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-09-21T03:56:33.174 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:33.174 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.174 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.174 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.175 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.175 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_file": { 2024-09-21T03:56:33.175 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_file", 2024-09-21T03:56:33.175 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-09-21T03:56:33.175 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.175 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:33.175 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-09-21T03:56:33.175 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.175 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.175 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.175 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:33.175 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.175 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.175 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.176 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.176 INFO:teuthology.orchestra.run.smithi039.stdout: "sqlite3_killpoint": { 2024-09-21T03:56:33.176 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:56:33.176 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-09-21T03:56:33.176 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "dev", 2024-09-21T03:56:33.176 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:33.176 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "0", 2024-09-21T03:56:33.176 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.176 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.176 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.176 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:33.176 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.176 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.176 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.176 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-21T03:56:33.177 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-21T03:56:33.177 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.177 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-21T03:56:33.177 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "telegraf", 2024-09-21T03:56:33.177 INFO:teuthology.orchestra.run.smithi039.stdout: "can_run": true, 2024-09-21T03:56:33.177 INFO:teuthology.orchestra.run.smithi039.stdout: "error_string": "", 2024-09-21T03:56:33.177 INFO:teuthology.orchestra.run.smithi039.stdout: "module_options": { 2024-09-21T03:56:33.177 INFO:teuthology.orchestra.run.smithi039.stdout: "address": { 2024-09-21T03:56:33.177 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "address", 2024-09-21T03:56:33.177 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-09-21T03:56:33.177 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.177 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-09-21T03:56:33.177 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-09-21T03:56:33.177 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.178 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.178 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.178 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:33.178 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.178 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.178 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.178 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.178 INFO:teuthology.orchestra.run.smithi039.stdout: "interval": { 2024-09-21T03:56:33.178 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "interval", 2024-09-21T03:56:33.178 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "secs", 2024-09-21T03:56:33.178 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.178 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-09-21T03:56:33.178 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "15", 2024-09-21T03:56:33.178 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.178 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.179 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.179 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:33.179 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.179 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.179 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.179 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.179 INFO:teuthology.orchestra.run.smithi039.stdout: "log_level": { 2024-09-21T03:56:33.179 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_level", 2024-09-21T03:56:33.179 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-09-21T03:56:33.179 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.179 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:33.179 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-09-21T03:56:33.179 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.179 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.180 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-09-21T03:56:33.180 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-09-21T03:56:33.180 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-09-21T03:56:33.180 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-09-21T03:56:33.180 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-09-21T03:56:33.180 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-09-21T03:56:33.180 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-09-21T03:56:33.180 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-09-21T03:56:33.180 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:33.180 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.180 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.180 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.180 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.180 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster": { 2024-09-21T03:56:33.181 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster", 2024-09-21T03:56:33.181 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-09-21T03:56:33.181 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.181 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:33.181 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-09-21T03:56:33.181 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.181 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.181 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.181 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:33.181 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.181 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.181 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.181 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.181 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster_level": { 2024-09-21T03:56:33.181 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster_level", 2024-09-21T03:56:33.182 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-09-21T03:56:33.182 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.182 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:33.182 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "info", 2024-09-21T03:56:33.182 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.182 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.182 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-09-21T03:56:33.182 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-09-21T03:56:33.182 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-09-21T03:56:33.182 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-09-21T03:56:33.182 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-09-21T03:56:33.182 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-09-21T03:56:33.182 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-09-21T03:56:33.183 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-09-21T03:56:33.183 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:33.183 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.183 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.183 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.183 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.183 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_file": { 2024-09-21T03:56:33.183 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_file", 2024-09-21T03:56:33.183 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-09-21T03:56:33.183 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.183 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:33.183 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-09-21T03:56:33.183 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.184 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.184 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.184 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:33.184 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.184 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.184 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.184 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.184 INFO:teuthology.orchestra.run.smithi039.stdout: "sqlite3_killpoint": { 2024-09-21T03:56:33.184 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:56:33.184 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-09-21T03:56:33.184 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "dev", 2024-09-21T03:56:33.184 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:33.184 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "0", 2024-09-21T03:56:33.185 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.185 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.185 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.185 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:33.185 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.185 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.185 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.185 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-21T03:56:33.185 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-21T03:56:33.185 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.185 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-21T03:56:33.185 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "telemetry", 2024-09-21T03:56:33.185 INFO:teuthology.orchestra.run.smithi039.stdout: "can_run": true, 2024-09-21T03:56:33.185 INFO:teuthology.orchestra.run.smithi039.stdout: "error_string": "", 2024-09-21T03:56:33.186 INFO:teuthology.orchestra.run.smithi039.stdout: "module_options": { 2024-09-21T03:56:33.186 INFO:teuthology.orchestra.run.smithi039.stdout: "channel_basic": { 2024-09-21T03:56:33.186 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "channel_basic", 2024-09-21T03:56:33.186 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-09-21T03:56:33.186 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.186 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-09-21T03:56:33.186 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2024-09-21T03:56:33.186 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.186 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.186 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.186 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Share basic cluster information (size, version)", 2024-09-21T03:56:33.186 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.186 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.187 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.187 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.187 INFO:teuthology.orchestra.run.smithi039.stdout: "channel_crash": { 2024-09-21T03:56:33.187 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "channel_crash", 2024-09-21T03:56:33.187 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-09-21T03:56:33.187 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.187 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-09-21T03:56:33.187 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2024-09-21T03:56:33.187 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.187 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.187 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.187 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-09-21T03:56:33.187 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.187 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.188 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.188 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.188 INFO:teuthology.orchestra.run.smithi039.stdout: "channel_device": { 2024-09-21T03:56:33.188 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "channel_device", 2024-09-21T03:56:33.188 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-09-21T03:56:33.188 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.188 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-09-21T03:56:33.188 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2024-09-21T03:56:33.188 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.188 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.188 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.188 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-09-21T03:56:33.188 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.188 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.188 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.189 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.189 INFO:teuthology.orchestra.run.smithi039.stdout: "channel_ident": { 2024-09-21T03:56:33.189 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "channel_ident", 2024-09-21T03:56:33.189 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-09-21T03:56:33.189 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.189 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-09-21T03:56:33.189 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-09-21T03:56:33.189 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.189 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.189 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.189 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-09-21T03:56:33.189 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.189 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.189 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.189 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.190 INFO:teuthology.orchestra.run.smithi039.stdout: "channel_perf": { 2024-09-21T03:56:33.190 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "channel_perf", 2024-09-21T03:56:33.190 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-09-21T03:56:33.190 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.190 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-09-21T03:56:33.190 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-09-21T03:56:33.190 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.190 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.190 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.190 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Share various performance metrics of a cluster", 2024-09-21T03:56:33.190 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.190 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.190 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.191 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.191 INFO:teuthology.orchestra.run.smithi039.stdout: "contact": { 2024-09-21T03:56:33.191 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "contact", 2024-09-21T03:56:33.191 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-09-21T03:56:33.191 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.191 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-09-21T03:56:33.191 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-09-21T03:56:33.191 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.191 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.191 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.191 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:33.191 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.191 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.191 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.192 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.192 INFO:teuthology.orchestra.run.smithi039.stdout: "description": { 2024-09-21T03:56:33.192 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "description", 2024-09-21T03:56:33.192 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-09-21T03:56:33.192 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.192 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-09-21T03:56:33.192 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-09-21T03:56:33.192 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.192 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.192 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.192 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:33.192 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.192 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.192 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.193 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.193 INFO:teuthology.orchestra.run.smithi039.stdout: "device_url": { 2024-09-21T03:56:33.193 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "device_url", 2024-09-21T03:56:33.193 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-09-21T03:56:33.193 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.193 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-09-21T03:56:33.193 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-09-21T03:56:33.193 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.193 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.193 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.193 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:33.193 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.193 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.193 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.193 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.194 INFO:teuthology.orchestra.run.smithi039.stdout: "enabled": { 2024-09-21T03:56:33.194 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "enabled", 2024-09-21T03:56:33.194 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-09-21T03:56:33.194 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.194 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-09-21T03:56:33.194 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-09-21T03:56:33.194 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.194 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.194 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.194 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:33.194 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.194 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.194 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.194 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.194 INFO:teuthology.orchestra.run.smithi039.stdout: "interval": { 2024-09-21T03:56:33.195 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "interval", 2024-09-21T03:56:33.195 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-09-21T03:56:33.195 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.195 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-09-21T03:56:33.195 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "24", 2024-09-21T03:56:33.195 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "8", 2024-09-21T03:56:33.195 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.195 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.195 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:33.195 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.195 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.195 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.195 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.196 INFO:teuthology.orchestra.run.smithi039.stdout: "last_opt_revision": { 2024-09-21T03:56:33.196 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "last_opt_revision", 2024-09-21T03:56:33.196 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-09-21T03:56:33.196 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.196 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-09-21T03:56:33.196 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "1", 2024-09-21T03:56:33.196 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.196 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.196 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.196 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:33.196 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.196 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.196 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.196 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.196 INFO:teuthology.orchestra.run.smithi039.stdout: "leaderboard": { 2024-09-21T03:56:33.197 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "leaderboard", 2024-09-21T03:56:33.197 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-09-21T03:56:33.197 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.197 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-09-21T03:56:33.197 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-09-21T03:56:33.197 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.197 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.197 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.197 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:33.197 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.197 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.197 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.197 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.197 INFO:teuthology.orchestra.run.smithi039.stdout: "leaderboard_description": { 2024-09-21T03:56:33.198 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "leaderboard_description", 2024-09-21T03:56:33.198 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-09-21T03:56:33.198 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.198 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-09-21T03:56:33.198 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-09-21T03:56:33.198 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.198 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.198 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.198 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:33.198 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.198 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.198 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.198 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.198 INFO:teuthology.orchestra.run.smithi039.stdout: "log_level": { 2024-09-21T03:56:33.199 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_level", 2024-09-21T03:56:33.199 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-09-21T03:56:33.199 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.199 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:33.199 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-09-21T03:56:33.199 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.199 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.199 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-09-21T03:56:33.199 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-09-21T03:56:33.199 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-09-21T03:56:33.199 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-09-21T03:56:33.199 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-09-21T03:56:33.199 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-09-21T03:56:33.199 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-09-21T03:56:33.200 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-09-21T03:56:33.200 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:33.200 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.200 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.200 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.200 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.200 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster": { 2024-09-21T03:56:33.200 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster", 2024-09-21T03:56:33.200 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-09-21T03:56:33.200 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.200 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:33.200 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-09-21T03:56:33.200 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.200 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.200 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.201 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:33.201 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.201 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.201 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.201 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.201 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster_level": { 2024-09-21T03:56:33.201 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster_level", 2024-09-21T03:56:33.201 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-09-21T03:56:33.201 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.201 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:33.201 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "info", 2024-09-21T03:56:33.201 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.201 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.201 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-09-21T03:56:33.202 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-09-21T03:56:33.202 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-09-21T03:56:33.202 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-09-21T03:56:33.202 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-09-21T03:56:33.202 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-09-21T03:56:33.202 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-09-21T03:56:33.202 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-09-21T03:56:33.202 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:33.202 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.202 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.202 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.202 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.202 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_file": { 2024-09-21T03:56:33.202 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_file", 2024-09-21T03:56:33.202 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-09-21T03:56:33.203 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.203 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:33.203 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-09-21T03:56:33.203 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.203 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.203 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.203 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:33.203 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.203 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.203 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.203 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.203 INFO:teuthology.orchestra.run.smithi039.stdout: "organization": { 2024-09-21T03:56:33.203 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "organization", 2024-09-21T03:56:33.203 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-09-21T03:56:33.203 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.204 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-09-21T03:56:33.204 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-09-21T03:56:33.204 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.204 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.204 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.204 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:33.204 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.204 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.204 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.204 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.204 INFO:teuthology.orchestra.run.smithi039.stdout: "proxy": { 2024-09-21T03:56:33.204 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "proxy", 2024-09-21T03:56:33.204 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-09-21T03:56:33.204 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.204 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-09-21T03:56:33.205 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-09-21T03:56:33.205 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.205 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.205 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.205 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:33.205 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.205 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.205 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.205 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.205 INFO:teuthology.orchestra.run.smithi039.stdout: "sqlite3_killpoint": { 2024-09-21T03:56:33.205 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:56:33.205 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-09-21T03:56:33.205 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "dev", 2024-09-21T03:56:33.205 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:33.206 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "0", 2024-09-21T03:56:33.206 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.206 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.206 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.206 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:33.206 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.206 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.206 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.206 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.206 INFO:teuthology.orchestra.run.smithi039.stdout: "url": { 2024-09-21T03:56:33.206 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "url", 2024-09-21T03:56:33.206 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-09-21T03:56:33.206 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.206 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-09-21T03:56:33.206 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-09-21T03:56:33.207 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.207 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.207 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.207 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:33.207 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.207 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.207 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.207 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-21T03:56:33.207 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-21T03:56:33.207 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.207 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-21T03:56:33.207 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "test_orchestrator", 2024-09-21T03:56:33.207 INFO:teuthology.orchestra.run.smithi039.stdout: "can_run": true, 2024-09-21T03:56:33.207 INFO:teuthology.orchestra.run.smithi039.stdout: "error_string": "", 2024-09-21T03:56:33.208 INFO:teuthology.orchestra.run.smithi039.stdout: "module_options": { 2024-09-21T03:56:33.208 INFO:teuthology.orchestra.run.smithi039.stdout: "log_level": { 2024-09-21T03:56:33.208 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_level", 2024-09-21T03:56:33.208 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-09-21T03:56:33.208 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.208 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:33.208 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-09-21T03:56:33.208 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.208 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.208 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-09-21T03:56:33.208 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-09-21T03:56:33.208 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-09-21T03:56:33.208 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-09-21T03:56:33.208 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-09-21T03:56:33.208 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-09-21T03:56:33.209 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-09-21T03:56:33.209 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-09-21T03:56:33.209 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:33.209 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.209 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.209 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.209 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.209 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster": { 2024-09-21T03:56:33.209 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster", 2024-09-21T03:56:33.209 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-09-21T03:56:33.209 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.209 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:33.209 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-09-21T03:56:33.209 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.210 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.210 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.210 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:33.210 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.210 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.210 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.210 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.210 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster_level": { 2024-09-21T03:56:33.210 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster_level", 2024-09-21T03:56:33.210 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-09-21T03:56:33.210 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.210 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:33.210 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "info", 2024-09-21T03:56:33.210 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.211 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.211 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-09-21T03:56:33.211 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-09-21T03:56:33.211 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-09-21T03:56:33.211 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-09-21T03:56:33.211 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-09-21T03:56:33.211 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-09-21T03:56:33.211 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-09-21T03:56:33.211 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-09-21T03:56:33.211 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:33.211 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.211 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.211 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.211 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.212 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_file": { 2024-09-21T03:56:33.212 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_file", 2024-09-21T03:56:33.212 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-09-21T03:56:33.212 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.212 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:33.212 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-09-21T03:56:33.212 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.212 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.212 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.212 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:33.212 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.212 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.212 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.212 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.212 INFO:teuthology.orchestra.run.smithi039.stdout: "sqlite3_killpoint": { 2024-09-21T03:56:33.213 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:56:33.213 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-09-21T03:56:33.213 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "dev", 2024-09-21T03:56:33.213 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:33.213 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "0", 2024-09-21T03:56:33.213 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.213 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.213 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.213 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:33.213 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.213 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.213 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.213 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-21T03:56:33.213 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-21T03:56:33.214 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.214 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-21T03:56:33.214 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "volumes", 2024-09-21T03:56:33.214 INFO:teuthology.orchestra.run.smithi039.stdout: "can_run": true, 2024-09-21T03:56:33.214 INFO:teuthology.orchestra.run.smithi039.stdout: "error_string": "", 2024-09-21T03:56:33.214 INFO:teuthology.orchestra.run.smithi039.stdout: "module_options": { 2024-09-21T03:56:33.214 INFO:teuthology.orchestra.run.smithi039.stdout: "log_level": { 2024-09-21T03:56:33.214 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_level", 2024-09-21T03:56:33.214 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-09-21T03:56:33.214 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.214 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:33.214 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-09-21T03:56:33.214 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.214 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.215 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-09-21T03:56:33.215 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-09-21T03:56:33.215 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-09-21T03:56:33.215 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-09-21T03:56:33.215 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-09-21T03:56:33.215 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-09-21T03:56:33.215 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-09-21T03:56:33.215 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-09-21T03:56:33.215 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:33.215 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.215 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.215 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.215 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.215 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster": { 2024-09-21T03:56:33.216 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster", 2024-09-21T03:56:33.216 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-09-21T03:56:33.216 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.216 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:33.216 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-09-21T03:56:33.216 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.216 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.216 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.216 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:33.216 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.216 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.216 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.216 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.216 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster_level": { 2024-09-21T03:56:33.216 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster_level", 2024-09-21T03:56:33.217 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-09-21T03:56:33.217 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.217 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:33.217 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "info", 2024-09-21T03:56:33.217 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.217 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.217 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-09-21T03:56:33.217 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-09-21T03:56:33.217 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-09-21T03:56:33.217 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-09-21T03:56:33.217 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-09-21T03:56:33.217 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-09-21T03:56:33.217 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-09-21T03:56:33.217 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-09-21T03:56:33.217 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:33.218 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.218 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.218 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.218 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.218 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_file": { 2024-09-21T03:56:33.218 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_file", 2024-09-21T03:56:33.218 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-09-21T03:56:33.218 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.218 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:33.218 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-09-21T03:56:33.218 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.218 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.218 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.218 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:33.219 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.219 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.219 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.219 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.219 INFO:teuthology.orchestra.run.smithi039.stdout: "max_concurrent_clones": { 2024-09-21T03:56:33.219 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "max_concurrent_clones", 2024-09-21T03:56:33.219 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-09-21T03:56:33.219 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.219 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-09-21T03:56:33.219 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "4", 2024-09-21T03:56:33.219 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.219 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.219 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.219 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Number of asynchronous cloner threads", 2024-09-21T03:56:33.220 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.220 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.220 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.220 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.220 INFO:teuthology.orchestra.run.smithi039.stdout: "periodic_async_work": { 2024-09-21T03:56:33.220 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "periodic_async_work", 2024-09-21T03:56:33.220 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-09-21T03:56:33.220 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.220 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-09-21T03:56:33.220 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-09-21T03:56:33.220 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.220 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.220 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.220 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Periodically check for async work", 2024-09-21T03:56:33.221 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.221 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.221 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.221 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.221 INFO:teuthology.orchestra.run.smithi039.stdout: "snapshot_clone_delay": { 2024-09-21T03:56:33.221 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "snapshot_clone_delay", 2024-09-21T03:56:33.221 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-09-21T03:56:33.221 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.221 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-09-21T03:56:33.221 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "0", 2024-09-21T03:56:33.221 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.221 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.221 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.221 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-09-21T03:56:33.222 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.222 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.222 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.222 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.222 INFO:teuthology.orchestra.run.smithi039.stdout: "snapshot_clone_no_wait": { 2024-09-21T03:56:33.222 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "snapshot_clone_no_wait", 2024-09-21T03:56:33.222 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-09-21T03:56:33.222 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.222 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-09-21T03:56:33.222 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2024-09-21T03:56:33.222 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.222 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.222 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.223 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-09-21T03:56:33.223 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.223 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.223 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.223 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.223 INFO:teuthology.orchestra.run.smithi039.stdout: "sqlite3_killpoint": { 2024-09-21T03:56:33.223 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:56:33.223 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-09-21T03:56:33.223 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "dev", 2024-09-21T03:56:33.223 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:33.223 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "0", 2024-09-21T03:56:33.223 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.223 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.224 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.224 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:33.224 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.224 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.224 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.224 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-21T03:56:33.224 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-21T03:56:33.224 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.224 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-21T03:56:33.224 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "zabbix", 2024-09-21T03:56:33.224 INFO:teuthology.orchestra.run.smithi039.stdout: "can_run": true, 2024-09-21T03:56:33.224 INFO:teuthology.orchestra.run.smithi039.stdout: "error_string": "", 2024-09-21T03:56:33.224 INFO:teuthology.orchestra.run.smithi039.stdout: "module_options": { 2024-09-21T03:56:33.224 INFO:teuthology.orchestra.run.smithi039.stdout: "discovery_interval": { 2024-09-21T03:56:33.225 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "discovery_interval", 2024-09-21T03:56:33.225 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "uint", 2024-09-21T03:56:33.225 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.225 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-09-21T03:56:33.225 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "100", 2024-09-21T03:56:33.225 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.225 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.225 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.225 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:33.225 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.225 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.225 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.225 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.225 INFO:teuthology.orchestra.run.smithi039.stdout: "identifier": { 2024-09-21T03:56:33.226 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "identifier", 2024-09-21T03:56:33.226 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-09-21T03:56:33.226 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.226 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-09-21T03:56:33.226 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-09-21T03:56:33.226 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.226 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.226 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.226 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:33.226 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.226 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.226 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.226 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.226 INFO:teuthology.orchestra.run.smithi039.stdout: "interval": { 2024-09-21T03:56:33.227 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "interval", 2024-09-21T03:56:33.227 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "secs", 2024-09-21T03:56:33.227 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.227 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-09-21T03:56:33.227 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "60", 2024-09-21T03:56:33.227 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.227 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.227 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.227 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:33.227 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.227 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.227 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.227 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.227 INFO:teuthology.orchestra.run.smithi039.stdout: "log_level": { 2024-09-21T03:56:33.228 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_level", 2024-09-21T03:56:33.228 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-09-21T03:56:33.228 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.228 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:33.228 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-09-21T03:56:33.228 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.228 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.228 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-09-21T03:56:33.228 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-09-21T03:56:33.228 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-09-21T03:56:33.228 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-09-21T03:56:33.228 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-09-21T03:56:33.228 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-09-21T03:56:33.228 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-09-21T03:56:33.229 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-09-21T03:56:33.229 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:33.229 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.229 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.229 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.229 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.229 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster": { 2024-09-21T03:56:33.229 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster", 2024-09-21T03:56:33.229 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-09-21T03:56:33.229 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.229 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:33.229 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-09-21T03:56:33.229 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.230 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.230 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.230 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:33.230 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.230 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.230 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.230 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.230 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster_level": { 2024-09-21T03:56:33.230 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster_level", 2024-09-21T03:56:33.230 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-09-21T03:56:33.230 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.230 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:33.230 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "info", 2024-09-21T03:56:33.230 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.231 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.231 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-09-21T03:56:33.231 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-09-21T03:56:33.231 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-09-21T03:56:33.231 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-09-21T03:56:33.231 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-09-21T03:56:33.231 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-09-21T03:56:33.231 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-09-21T03:56:33.231 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-09-21T03:56:33.231 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:33.231 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.231 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.231 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.231 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.231 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_file": { 2024-09-21T03:56:33.232 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_file", 2024-09-21T03:56:33.232 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-09-21T03:56:33.232 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.232 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:33.232 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-09-21T03:56:33.232 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.232 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.232 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.232 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:33.232 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.232 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.232 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.232 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.232 INFO:teuthology.orchestra.run.smithi039.stdout: "sqlite3_killpoint": { 2024-09-21T03:56:33.232 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:56:33.233 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-09-21T03:56:33.233 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "dev", 2024-09-21T03:56:33.233 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:33.233 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "0", 2024-09-21T03:56:33.233 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.233 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.233 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.233 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:33.233 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.233 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.233 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.233 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.233 INFO:teuthology.orchestra.run.smithi039.stdout: "zabbix_host": { 2024-09-21T03:56:33.233 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "zabbix_host", 2024-09-21T03:56:33.234 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-09-21T03:56:33.234 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.234 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-09-21T03:56:33.234 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-09-21T03:56:33.234 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.234 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.234 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.234 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:33.234 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.234 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.234 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.234 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.234 INFO:teuthology.orchestra.run.smithi039.stdout: "zabbix_port": { 2024-09-21T03:56:33.234 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "zabbix_port", 2024-09-21T03:56:33.234 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-09-21T03:56:33.235 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.235 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-09-21T03:56:33.235 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "10051", 2024-09-21T03:56:33.235 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.235 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.235 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.235 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:33.235 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.235 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.235 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.235 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.235 INFO:teuthology.orchestra.run.smithi039.stdout: "zabbix_sender": { 2024-09-21T03:56:33.235 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "zabbix_sender", 2024-09-21T03:56:33.235 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-09-21T03:56:33.235 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.236 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-09-21T03:56:33.236 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-09-21T03:56:33.236 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.236 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.236 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.236 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:33.236 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.236 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.236 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.236 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-21T03:56:33.236 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-21T03:56:33.236 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-21T03:56:33.236 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-21T03:56:33.237 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.237 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-21T03:56:33.237 INFO:teuthology.orchestra.run.smithi039.stdout: "gid": 24146, 2024-09-21T03:56:33.237 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "y", 2024-09-21T03:56:33.237 INFO:teuthology.orchestra.run.smithi039.stdout: "mgr_features": 4540719139924082687, 2024-09-21T03:56:33.237 INFO:teuthology.orchestra.run.smithi039.stdout: "available_modules": [ 2024-09-21T03:56:33.237 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-21T03:56:33.237 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "alerts", 2024-09-21T03:56:33.237 INFO:teuthology.orchestra.run.smithi039.stdout: "can_run": true, 2024-09-21T03:56:33.237 INFO:teuthology.orchestra.run.smithi039.stdout: "error_string": "", 2024-09-21T03:56:33.237 INFO:teuthology.orchestra.run.smithi039.stdout: "module_options": { 2024-09-21T03:56:33.237 INFO:teuthology.orchestra.run.smithi039.stdout: "interval": { 2024-09-21T03:56:33.237 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "interval", 2024-09-21T03:56:33.237 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "secs", 2024-09-21T03:56:33.238 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.238 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:33.238 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "60", 2024-09-21T03:56:33.238 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.238 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.238 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.238 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "How frequently to reexamine health status", 2024-09-21T03:56:33.238 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.238 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.238 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.238 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.238 INFO:teuthology.orchestra.run.smithi039.stdout: "log_level": { 2024-09-21T03:56:33.238 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_level", 2024-09-21T03:56:33.238 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-09-21T03:56:33.239 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.239 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:33.239 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-09-21T03:56:33.239 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.239 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.239 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-09-21T03:56:33.239 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-09-21T03:56:33.239 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-09-21T03:56:33.239 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-09-21T03:56:33.239 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-09-21T03:56:33.239 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-09-21T03:56:33.239 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-09-21T03:56:33.239 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-09-21T03:56:33.239 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:33.240 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.240 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.240 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.240 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.240 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster": { 2024-09-21T03:56:33.240 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster", 2024-09-21T03:56:33.240 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-09-21T03:56:33.240 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.240 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:33.240 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-09-21T03:56:33.240 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.240 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.240 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.240 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:33.241 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.241 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.241 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.241 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.241 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster_level": { 2024-09-21T03:56:33.241 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster_level", 2024-09-21T03:56:33.241 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-09-21T03:56:33.241 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.241 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:33.241 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "info", 2024-09-21T03:56:33.241 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.241 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.241 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-09-21T03:56:33.241 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-09-21T03:56:33.242 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-09-21T03:56:33.242 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-09-21T03:56:33.242 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-09-21T03:56:33.242 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-09-21T03:56:33.242 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-09-21T03:56:33.242 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-09-21T03:56:33.242 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:33.242 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.242 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.242 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.242 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.242 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_file": { 2024-09-21T03:56:33.242 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_file", 2024-09-21T03:56:33.242 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-09-21T03:56:33.243 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.243 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:33.243 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-09-21T03:56:33.243 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.243 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.243 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.243 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:33.243 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.243 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.243 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.243 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.243 INFO:teuthology.orchestra.run.smithi039.stdout: "smtp_destination": { 2024-09-21T03:56:33.243 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "smtp_destination", 2024-09-21T03:56:33.243 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-09-21T03:56:33.243 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.244 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:33.244 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-09-21T03:56:33.244 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.244 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.244 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.244 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Email address to send alerts to, use commas to separate multiple", 2024-09-21T03:56:33.244 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.244 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.244 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.244 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.244 INFO:teuthology.orchestra.run.smithi039.stdout: "smtp_from_name": { 2024-09-21T03:56:33.244 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "smtp_from_name", 2024-09-21T03:56:33.244 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-09-21T03:56:33.244 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.245 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:33.245 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "Ceph", 2024-09-21T03:56:33.245 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.245 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.245 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.245 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Email From: name", 2024-09-21T03:56:33.245 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.245 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.245 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.245 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.245 INFO:teuthology.orchestra.run.smithi039.stdout: "smtp_host": { 2024-09-21T03:56:33.245 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "smtp_host", 2024-09-21T03:56:33.245 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-09-21T03:56:33.245 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.245 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:33.246 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-09-21T03:56:33.246 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.246 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.246 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.246 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "SMTP server", 2024-09-21T03:56:33.246 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.246 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.246 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.246 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.246 INFO:teuthology.orchestra.run.smithi039.stdout: "smtp_password": { 2024-09-21T03:56:33.246 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "smtp_password", 2024-09-21T03:56:33.246 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-09-21T03:56:33.246 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.246 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:33.247 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-09-21T03:56:33.247 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.247 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.247 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.247 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Password to authenticate with", 2024-09-21T03:56:33.247 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.247 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.247 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.247 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.247 INFO:teuthology.orchestra.run.smithi039.stdout: "smtp_port": { 2024-09-21T03:56:33.247 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "smtp_port", 2024-09-21T03:56:33.247 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-09-21T03:56:33.247 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.247 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:33.247 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "465", 2024-09-21T03:56:33.248 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.248 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.248 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.248 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "SMTP port", 2024-09-21T03:56:33.248 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.248 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.248 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.248 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.248 INFO:teuthology.orchestra.run.smithi039.stdout: "smtp_sender": { 2024-09-21T03:56:33.248 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "smtp_sender", 2024-09-21T03:56:33.248 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-09-21T03:56:33.248 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.248 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:33.248 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-09-21T03:56:33.248 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.249 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.249 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.249 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "SMTP envelope sender", 2024-09-21T03:56:33.249 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.249 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.249 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.249 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.249 INFO:teuthology.orchestra.run.smithi039.stdout: "smtp_ssl": { 2024-09-21T03:56:33.249 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "smtp_ssl", 2024-09-21T03:56:33.249 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-09-21T03:56:33.249 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.249 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:33.249 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2024-09-21T03:56:33.250 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.250 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.250 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.250 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Use SSL to connect to SMTP server", 2024-09-21T03:56:33.250 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.250 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.250 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.250 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.250 INFO:teuthology.orchestra.run.smithi039.stdout: "smtp_user": { 2024-09-21T03:56:33.250 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "smtp_user", 2024-09-21T03:56:33.250 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-09-21T03:56:33.250 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.250 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:33.250 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-09-21T03:56:33.251 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.251 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.251 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.251 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "User to authenticate as", 2024-09-21T03:56:33.251 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.251 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.251 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.251 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.251 INFO:teuthology.orchestra.run.smithi039.stdout: "sqlite3_killpoint": { 2024-09-21T03:56:33.251 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:56:33.251 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-09-21T03:56:33.251 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "dev", 2024-09-21T03:56:33.251 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:33.251 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "0", 2024-09-21T03:56:33.252 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.252 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.252 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.252 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:33.252 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.252 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.252 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.252 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-21T03:56:33.252 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-21T03:56:33.252 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.252 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-21T03:56:33.252 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "balancer", 2024-09-21T03:56:33.252 INFO:teuthology.orchestra.run.smithi039.stdout: "can_run": true, 2024-09-21T03:56:33.252 INFO:teuthology.orchestra.run.smithi039.stdout: "error_string": "", 2024-09-21T03:56:33.253 INFO:teuthology.orchestra.run.smithi039.stdout: "module_options": { 2024-09-21T03:56:33.253 INFO:teuthology.orchestra.run.smithi039.stdout: "active": { 2024-09-21T03:56:33.253 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "active", 2024-09-21T03:56:33.253 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-09-21T03:56:33.253 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.253 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:33.253 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2024-09-21T03:56:33.253 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.253 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.253 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.253 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "automatically balance PGs across cluster", 2024-09-21T03:56:33.253 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.253 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.253 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.254 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.254 INFO:teuthology.orchestra.run.smithi039.stdout: "begin_time": { 2024-09-21T03:56:33.254 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "begin_time", 2024-09-21T03:56:33.254 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-09-21T03:56:33.254 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.254 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:33.254 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "0000", 2024-09-21T03:56:33.254 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.254 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.254 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.254 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "beginning time of day to automatically balance", 2024-09-21T03:56:33.254 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-09-21T03:56:33.254 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.254 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.255 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.255 INFO:teuthology.orchestra.run.smithi039.stdout: "begin_weekday": { 2024-09-21T03:56:33.255 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "begin_weekday", 2024-09-21T03:56:33.255 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "uint", 2024-09-21T03:56:33.255 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.255 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:33.255 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "0", 2024-09-21T03:56:33.255 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "0", 2024-09-21T03:56:33.255 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "6", 2024-09-21T03:56:33.255 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.255 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-09-21T03:56:33.255 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-09-21T03:56:33.255 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.255 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.256 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.256 INFO:teuthology.orchestra.run.smithi039.stdout: "crush_compat_max_iterations": { 2024-09-21T03:56:33.256 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "crush_compat_max_iterations", 2024-09-21T03:56:33.256 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "uint", 2024-09-21T03:56:33.256 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.256 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:33.256 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "25", 2024-09-21T03:56:33.256 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "1", 2024-09-21T03:56:33.256 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "250", 2024-09-21T03:56:33.256 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.256 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-09-21T03:56:33.256 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.256 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.256 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.257 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.257 INFO:teuthology.orchestra.run.smithi039.stdout: "crush_compat_metrics": { 2024-09-21T03:56:33.257 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "crush_compat_metrics", 2024-09-21T03:56:33.257 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-09-21T03:56:33.257 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.257 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:33.257 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "pgs,objects,bytes", 2024-09-21T03:56:33.257 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.257 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.257 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.257 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-09-21T03:56:33.257 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-09-21T03:56:33.257 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.257 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.257 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.258 INFO:teuthology.orchestra.run.smithi039.stdout: "crush_compat_step": { 2024-09-21T03:56:33.258 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "crush_compat_step", 2024-09-21T03:56:33.258 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "float", 2024-09-21T03:56:33.258 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.258 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:33.258 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "0.5", 2024-09-21T03:56:33.258 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "0.001", 2024-09-21T03:56:33.258 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "0.999", 2024-09-21T03:56:33.258 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.258 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "aggressiveness of optimization", 2024-09-21T03:56:33.258 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-09-21T03:56:33.258 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.258 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.258 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.259 INFO:teuthology.orchestra.run.smithi039.stdout: "end_time": { 2024-09-21T03:56:33.259 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "end_time", 2024-09-21T03:56:33.259 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-09-21T03:56:33.259 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.259 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:33.259 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "2359", 2024-09-21T03:56:33.259 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.259 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.259 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.259 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "ending time of day to automatically balance", 2024-09-21T03:56:33.259 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-09-21T03:56:33.259 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.259 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.259 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.260 INFO:teuthology.orchestra.run.smithi039.stdout: "end_weekday": { 2024-09-21T03:56:33.260 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "end_weekday", 2024-09-21T03:56:33.260 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "uint", 2024-09-21T03:56:33.260 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.260 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:33.260 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "0", 2024-09-21T03:56:33.260 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "0", 2024-09-21T03:56:33.260 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "6", 2024-09-21T03:56:33.260 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.260 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-09-21T03:56:33.260 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-09-21T03:56:33.260 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.260 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.260 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.260 INFO:teuthology.orchestra.run.smithi039.stdout: "log_level": { 2024-09-21T03:56:33.261 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_level", 2024-09-21T03:56:33.261 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-09-21T03:56:33.261 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.261 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:33.261 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-09-21T03:56:33.261 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.261 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.261 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-09-21T03:56:33.261 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-09-21T03:56:33.261 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-09-21T03:56:33.261 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-09-21T03:56:33.261 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-09-21T03:56:33.261 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-09-21T03:56:33.261 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-09-21T03:56:33.262 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-09-21T03:56:33.262 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:33.262 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.262 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.262 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.262 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.262 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster": { 2024-09-21T03:56:33.262 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster", 2024-09-21T03:56:33.262 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-09-21T03:56:33.262 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.262 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:33.262 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-09-21T03:56:33.262 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.263 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.263 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.263 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:33.263 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.263 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.263 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.263 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.263 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster_level": { 2024-09-21T03:56:33.263 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster_level", 2024-09-21T03:56:33.263 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-09-21T03:56:33.263 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.263 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:33.263 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "info", 2024-09-21T03:56:33.263 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.264 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.264 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-09-21T03:56:33.264 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-09-21T03:56:33.264 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-09-21T03:56:33.264 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-09-21T03:56:33.264 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-09-21T03:56:33.264 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-09-21T03:56:33.264 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-09-21T03:56:33.264 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-09-21T03:56:33.264 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:33.264 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.264 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.264 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.264 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.264 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_file": { 2024-09-21T03:56:33.265 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_file", 2024-09-21T03:56:33.265 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-09-21T03:56:33.265 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.265 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:33.265 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-09-21T03:56:33.265 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.265 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.265 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.265 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:33.265 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.265 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.265 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.265 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.265 INFO:teuthology.orchestra.run.smithi039.stdout: "min_score": { 2024-09-21T03:56:33.266 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "min_score", 2024-09-21T03:56:33.266 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "float", 2024-09-21T03:56:33.266 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.266 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:33.266 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "0", 2024-09-21T03:56:33.266 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.266 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.266 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.266 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-09-21T03:56:33.266 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.266 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.266 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.266 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.266 INFO:teuthology.orchestra.run.smithi039.stdout: "mode": { 2024-09-21T03:56:33.267 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "mode", 2024-09-21T03:56:33.267 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-09-21T03:56:33.267 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.267 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:33.267 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "upmap", 2024-09-21T03:56:33.267 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.267 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.267 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-09-21T03:56:33.267 INFO:teuthology.orchestra.run.smithi039.stdout: "crush-compat", 2024-09-21T03:56:33.267 INFO:teuthology.orchestra.run.smithi039.stdout: "none", 2024-09-21T03:56:33.267 INFO:teuthology.orchestra.run.smithi039.stdout: "read", 2024-09-21T03:56:33.267 INFO:teuthology.orchestra.run.smithi039.stdout: "upmap", 2024-09-21T03:56:33.267 INFO:teuthology.orchestra.run.smithi039.stdout: "upmap-read" 2024-09-21T03:56:33.267 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-09-21T03:56:33.268 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Balancer mode", 2024-09-21T03:56:33.268 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.268 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.268 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.268 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.268 INFO:teuthology.orchestra.run.smithi039.stdout: "pool_ids": { 2024-09-21T03:56:33.268 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "pool_ids", 2024-09-21T03:56:33.268 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-09-21T03:56:33.268 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.268 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:33.268 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-09-21T03:56:33.268 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.268 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.268 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.268 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-09-21T03:56:33.269 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.269 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.269 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.269 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.269 INFO:teuthology.orchestra.run.smithi039.stdout: "sleep_interval": { 2024-09-21T03:56:33.269 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "sleep_interval", 2024-09-21T03:56:33.269 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "secs", 2024-09-21T03:56:33.269 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.269 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:33.269 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "60", 2024-09-21T03:56:33.269 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.269 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.269 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.269 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-09-21T03:56:33.270 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.270 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.270 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.270 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.270 INFO:teuthology.orchestra.run.smithi039.stdout: "sqlite3_killpoint": { 2024-09-21T03:56:33.270 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:56:33.270 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-09-21T03:56:33.270 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "dev", 2024-09-21T03:56:33.270 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:33.270 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "0", 2024-09-21T03:56:33.270 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.270 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.270 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.270 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:33.270 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.271 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.271 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.271 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.271 INFO:teuthology.orchestra.run.smithi039.stdout: "upmap_max_deviation": { 2024-09-21T03:56:33.271 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "upmap_max_deviation", 2024-09-21T03:56:33.271 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-09-21T03:56:33.271 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.271 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:33.271 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "5", 2024-09-21T03:56:33.271 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "1", 2024-09-21T03:56:33.271 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.271 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.271 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "deviation below which no optimization is attempted", 2024-09-21T03:56:33.271 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-09-21T03:56:33.271 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.272 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.272 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.272 INFO:teuthology.orchestra.run.smithi039.stdout: "upmap_max_optimizations": { 2024-09-21T03:56:33.272 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "upmap_max_optimizations", 2024-09-21T03:56:33.272 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "uint", 2024-09-21T03:56:33.272 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.272 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:33.272 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "10", 2024-09-21T03:56:33.272 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.272 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.272 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.272 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-09-21T03:56:33.272 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.272 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.272 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.273 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-21T03:56:33.273 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-21T03:56:33.273 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.273 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-21T03:56:33.273 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "cephadm", 2024-09-21T03:56:33.273 INFO:teuthology.orchestra.run.smithi039.stdout: "can_run": true, 2024-09-21T03:56:33.273 INFO:teuthology.orchestra.run.smithi039.stdout: "error_string": "", 2024-09-21T03:56:33.273 INFO:teuthology.orchestra.run.smithi039.stdout: "module_options": { 2024-09-21T03:56:33.273 INFO:teuthology.orchestra.run.smithi039.stdout: "agent_down_multiplier": { 2024-09-21T03:56:33.273 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "agent_down_multiplier", 2024-09-21T03:56:33.273 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "float", 2024-09-21T03:56:33.273 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.273 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-09-21T03:56:33.273 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "3.0", 2024-09-21T03:56:33.274 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.274 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.274 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.274 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-09-21T03:56:33.274 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.274 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.274 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.274 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.274 INFO:teuthology.orchestra.run.smithi039.stdout: "agent_refresh_rate": { 2024-09-21T03:56:33.274 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "agent_refresh_rate", 2024-09-21T03:56:33.274 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "secs", 2024-09-21T03:56:33.274 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.274 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-09-21T03:56:33.274 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "20", 2024-09-21T03:56:33.275 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.275 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.275 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.275 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-09-21T03:56:33.275 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.275 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.275 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.275 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.275 INFO:teuthology.orchestra.run.smithi039.stdout: "agent_starting_port": { 2024-09-21T03:56:33.275 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "agent_starting_port", 2024-09-21T03:56:33.275 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-09-21T03:56:33.275 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.275 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-09-21T03:56:33.275 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "4721", 2024-09-21T03:56:33.276 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.276 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.276 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.276 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-09-21T03:56:33.276 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.276 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.276 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.276 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.276 INFO:teuthology.orchestra.run.smithi039.stdout: "allow_ptrace": { 2024-09-21T03:56:33.276 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "allow_ptrace", 2024-09-21T03:56:33.276 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-09-21T03:56:33.276 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.276 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-09-21T03:56:33.276 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-09-21T03:56:33.276 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.277 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.277 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.277 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-09-21T03:56:33.277 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-09-21T03:56:33.277 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.277 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.277 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.277 INFO:teuthology.orchestra.run.smithi039.stdout: "autotune_interval": { 2024-09-21T03:56:33.277 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "autotune_interval", 2024-09-21T03:56:33.277 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "secs", 2024-09-21T03:56:33.277 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.277 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-09-21T03:56:33.277 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "600", 2024-09-21T03:56:33.277 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.278 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.278 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.278 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "how frequently to autotune daemon memory", 2024-09-21T03:56:33.278 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.278 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.278 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.278 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.278 INFO:teuthology.orchestra.run.smithi039.stdout: "autotune_memory_target_ratio": { 2024-09-21T03:56:33.278 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "autotune_memory_target_ratio", 2024-09-21T03:56:33.278 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "float", 2024-09-21T03:56:33.278 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.278 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-09-21T03:56:33.278 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "0.7", 2024-09-21T03:56:33.278 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.279 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.279 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.279 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-09-21T03:56:33.279 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.279 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.279 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.279 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.279 INFO:teuthology.orchestra.run.smithi039.stdout: "cephadm_log_destination": { 2024-09-21T03:56:33.279 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "cephadm_log_destination", 2024-09-21T03:56:33.279 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-09-21T03:56:33.279 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.279 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-09-21T03:56:33.279 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-09-21T03:56:33.279 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.279 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.280 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-09-21T03:56:33.280 INFO:teuthology.orchestra.run.smithi039.stdout: "file", 2024-09-21T03:56:33.280 INFO:teuthology.orchestra.run.smithi039.stdout: "file,syslog", 2024-09-21T03:56:33.280 INFO:teuthology.orchestra.run.smithi039.stdout: "syslog" 2024-09-21T03:56:33.280 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-09-21T03:56:33.280 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-09-21T03:56:33.280 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.280 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.280 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.280 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.280 INFO:teuthology.orchestra.run.smithi039.stdout: "cgroups_split": { 2024-09-21T03:56:33.280 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "cgroups_split", 2024-09-21T03:56:33.280 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-09-21T03:56:33.280 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.280 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-09-21T03:56:33.281 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2024-09-21T03:56:33.281 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.281 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.281 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.281 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-09-21T03:56:33.281 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.281 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.281 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.281 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.281 INFO:teuthology.orchestra.run.smithi039.stdout: "config_checks_enabled": { 2024-09-21T03:56:33.281 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "config_checks_enabled", 2024-09-21T03:56:33.281 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-09-21T03:56:33.281 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.281 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-09-21T03:56:33.281 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-09-21T03:56:33.282 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.282 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.282 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.282 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-09-21T03:56:33.282 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.282 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.282 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.282 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.282 INFO:teuthology.orchestra.run.smithi039.stdout: "config_dashboard": { 2024-09-21T03:56:33.282 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "config_dashboard", 2024-09-21T03:56:33.282 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-09-21T03:56:33.282 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.282 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-09-21T03:56:33.282 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2024-09-21T03:56:33.283 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.283 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.283 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.283 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-09-21T03:56:33.283 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.283 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.283 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.283 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.283 INFO:teuthology.orchestra.run.smithi039.stdout: "container_image_alertmanager": { 2024-09-21T03:56:33.283 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "container_image_alertmanager", 2024-09-21T03:56:33.283 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-09-21T03:56:33.283 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.283 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-09-21T03:56:33.283 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.27.0", 2024-09-21T03:56:33.284 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.284 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.284 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.284 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Prometheus container image", 2024-09-21T03:56:33.284 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.284 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.284 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.284 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.284 INFO:teuthology.orchestra.run.smithi039.stdout: "container_image_base": { 2024-09-21T03:56:33.284 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "container_image_base", 2024-09-21T03:56:33.284 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-09-21T03:56:33.284 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.284 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:33.284 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "quay.io/ceph/ceph", 2024-09-21T03:56:33.284 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.285 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.285 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.285 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Container image name, without the tag", 2024-09-21T03:56:33.285 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.285 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.285 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.285 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.285 INFO:teuthology.orchestra.run.smithi039.stdout: "container_image_elasticsearch": { 2024-09-21T03:56:33.285 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "container_image_elasticsearch", 2024-09-21T03:56:33.285 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-09-21T03:56:33.285 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.285 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-09-21T03:56:33.285 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-09-21T03:56:33.285 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.286 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.286 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.286 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "elasticsearch container image", 2024-09-21T03:56:33.286 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.286 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.286 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.286 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.286 INFO:teuthology.orchestra.run.smithi039.stdout: "container_image_grafana": { 2024-09-21T03:56:33.286 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "container_image_grafana", 2024-09-21T03:56:33.286 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-09-21T03:56:33.286 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.286 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-09-21T03:56:33.286 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "quay.io/ceph/grafana:10.4.8", 2024-09-21T03:56:33.287 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.287 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.287 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.287 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Prometheus container image", 2024-09-21T03:56:33.287 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.287 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.287 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.287 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.287 INFO:teuthology.orchestra.run.smithi039.stdout: "container_image_haproxy": { 2024-09-21T03:56:33.287 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "container_image_haproxy", 2024-09-21T03:56:33.287 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-09-21T03:56:33.287 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.287 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-09-21T03:56:33.287 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-09-21T03:56:33.288 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.288 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.288 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.288 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "HAproxy container image", 2024-09-21T03:56:33.288 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.288 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.288 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.288 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.288 INFO:teuthology.orchestra.run.smithi039.stdout: "container_image_jaeger_agent": { 2024-09-21T03:56:33.288 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "container_image_jaeger_agent", 2024-09-21T03:56:33.288 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-09-21T03:56:33.288 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.288 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-09-21T03:56:33.288 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-09-21T03:56:33.289 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.289 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.289 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.289 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Jaeger agent container image", 2024-09-21T03:56:33.289 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.289 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.289 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.289 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.289 INFO:teuthology.orchestra.run.smithi039.stdout: "container_image_jaeger_collector": { 2024-09-21T03:56:33.289 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "container_image_jaeger_collector", 2024-09-21T03:56:33.289 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-09-21T03:56:33.289 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.289 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-09-21T03:56:33.289 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-09-21T03:56:33.289 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.290 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.290 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.290 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Jaeger collector container image", 2024-09-21T03:56:33.290 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.290 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.290 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.290 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.290 INFO:teuthology.orchestra.run.smithi039.stdout: "container_image_jaeger_query": { 2024-09-21T03:56:33.290 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "container_image_jaeger_query", 2024-09-21T03:56:33.290 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-09-21T03:56:33.290 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.290 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-09-21T03:56:33.290 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-09-21T03:56:33.290 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.291 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.291 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.291 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Jaeger query container image", 2024-09-21T03:56:33.291 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.291 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.291 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.291 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.291 INFO:teuthology.orchestra.run.smithi039.stdout: "container_image_keepalived": { 2024-09-21T03:56:33.291 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "container_image_keepalived", 2024-09-21T03:56:33.291 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-09-21T03:56:33.291 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.291 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-09-21T03:56:33.291 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-09-21T03:56:33.291 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.292 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.292 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.292 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Keepalived container image", 2024-09-21T03:56:33.292 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.292 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.292 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.292 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.292 INFO:teuthology.orchestra.run.smithi039.stdout: "container_image_loki": { 2024-09-21T03:56:33.292 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "container_image_loki", 2024-09-21T03:56:33.292 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-09-21T03:56:33.292 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.292 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-09-21T03:56:33.292 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "docker.io/grafana/loki:3.0.0", 2024-09-21T03:56:33.292 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.292 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.293 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.293 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Loki container image", 2024-09-21T03:56:33.293 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.293 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.293 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.293 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.293 INFO:teuthology.orchestra.run.smithi039.stdout: "container_image_nginx": { 2024-09-21T03:56:33.293 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "container_image_nginx", 2024-09-21T03:56:33.293 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-09-21T03:56:33.293 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.293 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-09-21T03:56:33.293 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "quay.io/ceph/nginx:sclorg-nginx-126", 2024-09-21T03:56:33.293 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.293 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.294 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.294 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Nginx container image", 2024-09-21T03:56:33.294 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.294 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.294 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.294 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.294 INFO:teuthology.orchestra.run.smithi039.stdout: "container_image_node_exporter": { 2024-09-21T03:56:33.294 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "container_image_node_exporter", 2024-09-21T03:56:33.294 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-09-21T03:56:33.294 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.294 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-09-21T03:56:33.294 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.7.0", 2024-09-21T03:56:33.294 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.295 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.295 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.295 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Prometheus container image", 2024-09-21T03:56:33.295 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.295 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.295 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.295 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.295 INFO:teuthology.orchestra.run.smithi039.stdout: "container_image_nvmeof": { 2024-09-21T03:56:33.295 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "container_image_nvmeof", 2024-09-21T03:56:33.295 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-09-21T03:56:33.295 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.295 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-09-21T03:56:33.295 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.17", 2024-09-21T03:56:33.295 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.296 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.296 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.296 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Nvme-of container image", 2024-09-21T03:56:33.296 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.296 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.296 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.296 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.296 INFO:teuthology.orchestra.run.smithi039.stdout: "container_image_oauth2_proxy": { 2024-09-21T03:56:33.296 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "container_image_oauth2_proxy", 2024-09-21T03:56:33.296 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-09-21T03:56:33.296 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.296 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-09-21T03:56:33.296 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "quay.io/oauth2-proxy/oauth2-proxy:v7.6.0", 2024-09-21T03:56:33.296 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.296 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.297 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.297 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "oauth2-proxy container image", 2024-09-21T03:56:33.297 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.297 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.297 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.297 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.297 INFO:teuthology.orchestra.run.smithi039.stdout: "container_image_prometheus": { 2024-09-21T03:56:33.297 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "container_image_prometheus", 2024-09-21T03:56:33.297 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-09-21T03:56:33.297 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.297 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-09-21T03:56:33.297 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "quay.io/prometheus/prometheus:v2.51.0", 2024-09-21T03:56:33.297 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.297 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.297 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.298 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Prometheus container image", 2024-09-21T03:56:33.298 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.298 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.298 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.298 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.298 INFO:teuthology.orchestra.run.smithi039.stdout: "container_image_promtail": { 2024-09-21T03:56:33.298 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "container_image_promtail", 2024-09-21T03:56:33.298 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-09-21T03:56:33.298 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.298 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-09-21T03:56:33.298 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "docker.io/grafana/promtail:3.0.0", 2024-09-21T03:56:33.298 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.298 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.299 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.299 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Promtail container image", 2024-09-21T03:56:33.299 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.299 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.299 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.299 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.299 INFO:teuthology.orchestra.run.smithi039.stdout: "container_image_samba": { 2024-09-21T03:56:33.299 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "container_image_samba", 2024-09-21T03:56:33.299 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-09-21T03:56:33.299 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.299 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-09-21T03:56:33.299 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-09-21T03:56:33.299 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.299 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.300 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.300 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Samba/SMB container image", 2024-09-21T03:56:33.300 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.300 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.300 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.300 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.300 INFO:teuthology.orchestra.run.smithi039.stdout: "container_image_samba_metrics": { 2024-09-21T03:56:33.300 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "container_image_samba_metrics", 2024-09-21T03:56:33.300 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-09-21T03:56:33.300 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.300 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-09-21T03:56:33.300 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "quay.io/samba.org/samba-metrics:latest", 2024-09-21T03:56:33.300 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.300 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.301 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.301 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Samba/SMB metrics exporter container image", 2024-09-21T03:56:33.301 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.301 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.301 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.301 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.301 INFO:teuthology.orchestra.run.smithi039.stdout: "container_image_snmp_gateway": { 2024-09-21T03:56:33.301 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "container_image_snmp_gateway", 2024-09-21T03:56:33.301 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-09-21T03:56:33.301 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.301 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-09-21T03:56:33.301 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-09-21T03:56:33.301 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.301 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.301 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.302 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "SNMP Gateway container image", 2024-09-21T03:56:33.302 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.302 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.302 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.302 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.302 INFO:teuthology.orchestra.run.smithi039.stdout: "container_init": { 2024-09-21T03:56:33.302 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "container_init", 2024-09-21T03:56:33.302 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-09-21T03:56:33.302 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.302 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-09-21T03:56:33.302 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2024-09-21T03:56:33.302 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.302 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.302 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.303 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Run podman/docker with `--init`", 2024-09-21T03:56:33.303 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.303 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.303 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.303 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.303 INFO:teuthology.orchestra.run.smithi039.stdout: "daemon_cache_timeout": { 2024-09-21T03:56:33.303 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "daemon_cache_timeout", 2024-09-21T03:56:33.303 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "secs", 2024-09-21T03:56:33.303 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.303 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-09-21T03:56:33.303 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "600", 2024-09-21T03:56:33.303 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.303 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.303 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.304 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-09-21T03:56:33.304 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.304 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.304 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.304 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.304 INFO:teuthology.orchestra.run.smithi039.stdout: "default_cephadm_command_timeout": { 2024-09-21T03:56:33.304 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "default_cephadm_command_timeout", 2024-09-21T03:56:33.304 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-09-21T03:56:33.304 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.304 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-09-21T03:56:33.304 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "900", 2024-09-21T03:56:33.304 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.304 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.304 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.305 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-09-21T03:56:33.305 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.305 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.305 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.305 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.305 INFO:teuthology.orchestra.run.smithi039.stdout: "default_registry": { 2024-09-21T03:56:33.305 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "default_registry", 2024-09-21T03:56:33.305 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-09-21T03:56:33.305 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.305 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-09-21T03:56:33.305 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "docker.io", 2024-09-21T03:56:33.305 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.305 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.305 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.305 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-09-21T03:56:33.306 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.306 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.306 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.306 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.306 INFO:teuthology.orchestra.run.smithi039.stdout: "device_cache_timeout": { 2024-09-21T03:56:33.306 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "device_cache_timeout", 2024-09-21T03:56:33.306 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "secs", 2024-09-21T03:56:33.306 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.306 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-09-21T03:56:33.306 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "1800", 2024-09-21T03:56:33.306 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.306 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.306 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.307 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "seconds to cache device inventory", 2024-09-21T03:56:33.307 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.307 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.307 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.307 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.307 INFO:teuthology.orchestra.run.smithi039.stdout: "device_enhanced_scan": { 2024-09-21T03:56:33.307 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "device_enhanced_scan", 2024-09-21T03:56:33.307 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-09-21T03:56:33.307 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.307 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-09-21T03:56:33.307 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-09-21T03:56:33.307 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.307 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.307 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.307 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Use libstoragemgmt during device scans", 2024-09-21T03:56:33.308 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.308 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.308 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.308 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.308 INFO:teuthology.orchestra.run.smithi039.stdout: "facts_cache_timeout": { 2024-09-21T03:56:33.308 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "facts_cache_timeout", 2024-09-21T03:56:33.308 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "secs", 2024-09-21T03:56:33.308 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.308 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-09-21T03:56:33.308 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "60", 2024-09-21T03:56:33.308 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.308 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.308 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.308 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "seconds to cache host facts data", 2024-09-21T03:56:33.309 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.309 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.309 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.309 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.309 INFO:teuthology.orchestra.run.smithi039.stdout: "grafana_dashboards_path": { 2024-09-21T03:56:33.309 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "grafana_dashboards_path", 2024-09-21T03:56:33.309 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-09-21T03:56:33.309 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.309 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-09-21T03:56:33.309 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-09-21T03:56:33.309 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.309 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.309 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.309 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-09-21T03:56:33.310 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.310 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.310 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.310 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.310 INFO:teuthology.orchestra.run.smithi039.stdout: "host_check_interval": { 2024-09-21T03:56:33.310 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "host_check_interval", 2024-09-21T03:56:33.310 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "secs", 2024-09-21T03:56:33.310 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.310 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-09-21T03:56:33.310 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "600", 2024-09-21T03:56:33.310 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.310 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.310 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.311 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "how frequently to perform a host check", 2024-09-21T03:56:33.311 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.311 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.311 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.311 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.311 INFO:teuthology.orchestra.run.smithi039.stdout: "hw_monitoring": { 2024-09-21T03:56:33.311 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "hw_monitoring", 2024-09-21T03:56:33.311 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-09-21T03:56:33.311 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.311 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-09-21T03:56:33.311 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-09-21T03:56:33.311 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.311 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.311 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.312 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-09-21T03:56:33.312 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.312 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.312 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.312 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.312 INFO:teuthology.orchestra.run.smithi039.stdout: "inventory_list_all": { 2024-09-21T03:56:33.312 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "inventory_list_all", 2024-09-21T03:56:33.312 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-09-21T03:56:33.312 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.312 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-09-21T03:56:33.312 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-09-21T03:56:33.312 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.312 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.312 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.312 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-09-21T03:56:33.313 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.313 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.313 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.313 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.313 INFO:teuthology.orchestra.run.smithi039.stdout: "log_level": { 2024-09-21T03:56:33.313 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_level", 2024-09-21T03:56:33.313 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-09-21T03:56:33.313 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.313 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:33.313 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-09-21T03:56:33.313 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.313 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.313 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-09-21T03:56:33.314 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-09-21T03:56:33.314 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-09-21T03:56:33.314 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-09-21T03:56:33.314 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-09-21T03:56:33.314 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-09-21T03:56:33.314 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-09-21T03:56:33.314 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-09-21T03:56:33.314 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:33.314 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.314 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.314 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.314 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.314 INFO:teuthology.orchestra.run.smithi039.stdout: "log_refresh_metadata": { 2024-09-21T03:56:33.315 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_refresh_metadata", 2024-09-21T03:56:33.315 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-09-21T03:56:33.315 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.315 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-09-21T03:56:33.315 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-09-21T03:56:33.315 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.315 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.315 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.315 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-09-21T03:56:33.315 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.315 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.315 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.315 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.315 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster": { 2024-09-21T03:56:33.316 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster", 2024-09-21T03:56:33.316 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-09-21T03:56:33.316 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.316 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-09-21T03:56:33.316 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2024-09-21T03:56:33.316 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.316 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.316 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.316 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-09-21T03:56:33.316 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.316 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.316 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.316 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.316 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster_level": { 2024-09-21T03:56:33.317 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster_level", 2024-09-21T03:56:33.317 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-09-21T03:56:33.317 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.317 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:33.317 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "info", 2024-09-21T03:56:33.317 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.317 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.317 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-09-21T03:56:33.317 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-09-21T03:56:33.317 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-09-21T03:56:33.317 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-09-21T03:56:33.317 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-09-21T03:56:33.317 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-09-21T03:56:33.317 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-09-21T03:56:33.317 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-09-21T03:56:33.318 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:33.318 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.318 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.318 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.318 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.318 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_file": { 2024-09-21T03:56:33.318 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_file", 2024-09-21T03:56:33.318 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-09-21T03:56:33.318 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.318 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:33.318 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-09-21T03:56:33.318 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.318 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.318 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.319 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:33.319 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.319 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.319 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.319 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.319 INFO:teuthology.orchestra.run.smithi039.stdout: "manage_etc_ceph_ceph_conf": { 2024-09-21T03:56:33.319 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-09-21T03:56:33.319 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-09-21T03:56:33.319 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.319 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-09-21T03:56:33.319 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-09-21T03:56:33.319 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.319 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.319 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.320 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-09-21T03:56:33.320 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.320 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.320 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.320 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.320 INFO:teuthology.orchestra.run.smithi039.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-09-21T03:56:33.320 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-09-21T03:56:33.320 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-09-21T03:56:33.320 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.320 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-09-21T03:56:33.320 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "*", 2024-09-21T03:56:33.320 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.320 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.320 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.320 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-09-21T03:56:33.321 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.321 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.321 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.321 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.321 INFO:teuthology.orchestra.run.smithi039.stdout: "max_count_per_host": { 2024-09-21T03:56:33.321 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "max_count_per_host", 2024-09-21T03:56:33.321 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-09-21T03:56:33.321 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.321 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-09-21T03:56:33.321 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "10", 2024-09-21T03:56:33.321 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.321 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.321 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.321 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "max number of daemons per service per host", 2024-09-21T03:56:33.322 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.322 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.322 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.322 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.322 INFO:teuthology.orchestra.run.smithi039.stdout: "max_osd_draining_count": { 2024-09-21T03:56:33.322 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "max_osd_draining_count", 2024-09-21T03:56:33.322 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-09-21T03:56:33.322 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.322 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-09-21T03:56:33.322 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "10", 2024-09-21T03:56:33.322 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.322 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.322 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.323 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-09-21T03:56:33.323 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.323 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.323 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.323 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.323 INFO:teuthology.orchestra.run.smithi039.stdout: "migration_current": { 2024-09-21T03:56:33.323 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "migration_current", 2024-09-21T03:56:33.323 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-09-21T03:56:33.323 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.323 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-09-21T03:56:33.323 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-09-21T03:56:33.323 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.323 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.324 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.324 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "internal - do not modify", 2024-09-21T03:56:33.324 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.324 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.324 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.324 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.324 INFO:teuthology.orchestra.run.smithi039.stdout: "mode": { 2024-09-21T03:56:33.324 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "mode", 2024-09-21T03:56:33.324 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-09-21T03:56:33.324 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.324 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-09-21T03:56:33.324 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "root", 2024-09-21T03:56:33.324 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.324 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.324 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-09-21T03:56:33.325 INFO:teuthology.orchestra.run.smithi039.stdout: "cephadm-package", 2024-09-21T03:56:33.325 INFO:teuthology.orchestra.run.smithi039.stdout: "root" 2024-09-21T03:56:33.325 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-09-21T03:56:33.325 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "mode for remote execution of cephadm", 2024-09-21T03:56:33.325 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.325 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.325 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.325 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.325 INFO:teuthology.orchestra.run.smithi039.stdout: "oob_default_addr": { 2024-09-21T03:56:33.325 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "oob_default_addr", 2024-09-21T03:56:33.325 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-09-21T03:56:33.325 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.325 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-09-21T03:56:33.325 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "169.254.1.1", 2024-09-21T03:56:33.326 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.326 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.326 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.326 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Default address for RedFish API (oob management).", 2024-09-21T03:56:33.326 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.326 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.326 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.326 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.326 INFO:teuthology.orchestra.run.smithi039.stdout: "prometheus_alerts_path": { 2024-09-21T03:56:33.326 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "prometheus_alerts_path", 2024-09-21T03:56:33.326 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-09-21T03:56:33.326 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.326 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-09-21T03:56:33.327 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-09-21T03:56:33.327 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.327 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.327 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.327 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-09-21T03:56:33.327 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.327 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.327 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.327 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.327 INFO:teuthology.orchestra.run.smithi039.stdout: "registry_insecure": { 2024-09-21T03:56:33.327 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "registry_insecure", 2024-09-21T03:56:33.327 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-09-21T03:56:33.327 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.327 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-09-21T03:56:33.327 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-09-21T03:56:33.328 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.328 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.328 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.328 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-09-21T03:56:33.328 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.328 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.328 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.328 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.328 INFO:teuthology.orchestra.run.smithi039.stdout: "registry_password": { 2024-09-21T03:56:33.328 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "registry_password", 2024-09-21T03:56:33.328 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-09-21T03:56:33.328 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.328 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-09-21T03:56:33.329 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-09-21T03:56:33.329 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.329 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.329 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.329 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-09-21T03:56:33.329 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.329 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.329 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.329 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.329 INFO:teuthology.orchestra.run.smithi039.stdout: "registry_url": { 2024-09-21T03:56:33.329 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "registry_url", 2024-09-21T03:56:33.329 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-09-21T03:56:33.329 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.330 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-09-21T03:56:33.330 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-09-21T03:56:33.330 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.330 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.330 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.330 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-09-21T03:56:33.330 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.330 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.330 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.330 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.330 INFO:teuthology.orchestra.run.smithi039.stdout: "registry_username": { 2024-09-21T03:56:33.330 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "registry_username", 2024-09-21T03:56:33.330 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-09-21T03:56:33.330 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.331 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-09-21T03:56:33.331 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-09-21T03:56:33.331 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.331 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.331 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.331 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-09-21T03:56:33.331 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.331 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.331 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.331 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.331 INFO:teuthology.orchestra.run.smithi039.stdout: "secure_monitoring_stack": { 2024-09-21T03:56:33.331 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "secure_monitoring_stack", 2024-09-21T03:56:33.331 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-09-21T03:56:33.331 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.332 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-09-21T03:56:33.332 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-09-21T03:56:33.332 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.332 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.332 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.332 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-09-21T03:56:33.332 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.332 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.332 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.332 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.332 INFO:teuthology.orchestra.run.smithi039.stdout: "service_discovery_port": { 2024-09-21T03:56:33.332 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "service_discovery_port", 2024-09-21T03:56:33.332 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-09-21T03:56:33.332 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.332 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-09-21T03:56:33.333 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "8765", 2024-09-21T03:56:33.333 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.333 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.333 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.333 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "cephadm service discovery port", 2024-09-21T03:56:33.333 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.333 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.333 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.333 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.333 INFO:teuthology.orchestra.run.smithi039.stdout: "sqlite3_killpoint": { 2024-09-21T03:56:33.333 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:56:33.333 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-09-21T03:56:33.333 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "dev", 2024-09-21T03:56:33.333 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:33.334 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "0", 2024-09-21T03:56:33.334 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.334 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.334 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.334 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:33.334 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.334 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.334 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.334 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.334 INFO:teuthology.orchestra.run.smithi039.stdout: "ssh_config_file": { 2024-09-21T03:56:33.334 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "ssh_config_file", 2024-09-21T03:56:33.334 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-09-21T03:56:33.334 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.334 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-09-21T03:56:33.335 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-09-21T03:56:33.335 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.335 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.335 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.335 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-09-21T03:56:33.335 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.335 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.335 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.335 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.335 INFO:teuthology.orchestra.run.smithi039.stdout: "ssh_keepalive_count_max": { 2024-09-21T03:56:33.335 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "ssh_keepalive_count_max", 2024-09-21T03:56:33.335 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-09-21T03:56:33.335 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.336 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-09-21T03:56:33.336 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "3", 2024-09-21T03:56:33.336 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.336 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.336 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.336 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "How many times ssh connections can fail liveness checks before the host is marked offline", 2024-09-21T03:56:33.336 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.336 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.336 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.336 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.336 INFO:teuthology.orchestra.run.smithi039.stdout: "ssh_keepalive_interval": { 2024-09-21T03:56:33.336 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "ssh_keepalive_interval", 2024-09-21T03:56:33.336 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-09-21T03:56:33.336 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.337 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-09-21T03:56:33.337 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "7", 2024-09-21T03:56:33.337 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.337 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.337 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.337 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "How often ssh connections are checked for liveness", 2024-09-21T03:56:33.337 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.337 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.337 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.337 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.337 INFO:teuthology.orchestra.run.smithi039.stdout: "use_agent": { 2024-09-21T03:56:33.337 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "use_agent", 2024-09-21T03:56:33.337 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-09-21T03:56:33.337 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.337 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-09-21T03:56:33.338 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-09-21T03:56:33.338 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.338 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.338 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.338 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-09-21T03:56:33.338 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.338 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.338 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.338 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.338 INFO:teuthology.orchestra.run.smithi039.stdout: "use_repo_digest": { 2024-09-21T03:56:33.338 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "use_repo_digest", 2024-09-21T03:56:33.338 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-09-21T03:56:33.338 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.338 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-09-21T03:56:33.339 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2024-09-21T03:56:33.339 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.339 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.339 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.339 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-09-21T03:56:33.339 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.339 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.339 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.339 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.339 INFO:teuthology.orchestra.run.smithi039.stdout: "warn_on_failed_host_check": { 2024-09-21T03:56:33.339 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "warn_on_failed_host_check", 2024-09-21T03:56:33.339 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-09-21T03:56:33.339 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.340 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-09-21T03:56:33.340 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2024-09-21T03:56:33.340 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.340 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.340 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.340 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "raise a health warning if the host check fails", 2024-09-21T03:56:33.340 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.340 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.340 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.340 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.340 INFO:teuthology.orchestra.run.smithi039.stdout: "warn_on_stray_daemons": { 2024-09-21T03:56:33.340 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "warn_on_stray_daemons", 2024-09-21T03:56:33.340 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-09-21T03:56:33.340 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.341 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-09-21T03:56:33.341 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2024-09-21T03:56:33.341 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.341 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.341 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.341 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-09-21T03:56:33.341 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.341 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.341 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.341 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.341 INFO:teuthology.orchestra.run.smithi039.stdout: "warn_on_stray_hosts": { 2024-09-21T03:56:33.341 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "warn_on_stray_hosts", 2024-09-21T03:56:33.341 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-09-21T03:56:33.341 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.341 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-09-21T03:56:33.342 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2024-09-21T03:56:33.342 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.342 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.342 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.342 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-09-21T03:56:33.342 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.342 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.342 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.342 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-21T03:56:33.342 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-21T03:56:33.342 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.342 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-21T03:56:33.342 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "crash", 2024-09-21T03:56:33.343 INFO:teuthology.orchestra.run.smithi039.stdout: "can_run": true, 2024-09-21T03:56:33.343 INFO:teuthology.orchestra.run.smithi039.stdout: "error_string": "", 2024-09-21T03:56:33.343 INFO:teuthology.orchestra.run.smithi039.stdout: "module_options": { 2024-09-21T03:56:33.343 INFO:teuthology.orchestra.run.smithi039.stdout: "log_level": { 2024-09-21T03:56:33.343 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_level", 2024-09-21T03:56:33.343 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-09-21T03:56:33.343 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.343 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:33.343 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-09-21T03:56:33.343 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.343 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.343 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-09-21T03:56:33.343 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-09-21T03:56:33.343 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-09-21T03:56:33.343 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-09-21T03:56:33.344 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-09-21T03:56:33.344 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-09-21T03:56:33.344 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-09-21T03:56:33.344 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-09-21T03:56:33.344 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:33.344 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.344 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.344 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.344 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.344 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster": { 2024-09-21T03:56:33.344 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster", 2024-09-21T03:56:33.344 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-09-21T03:56:33.344 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.344 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:33.344 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-09-21T03:56:33.345 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.345 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.345 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.345 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:33.345 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.345 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.345 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.345 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.345 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster_level": { 2024-09-21T03:56:33.345 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster_level", 2024-09-21T03:56:33.345 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-09-21T03:56:33.345 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.345 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:33.345 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "info", 2024-09-21T03:56:33.346 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.346 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.346 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-09-21T03:56:33.346 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-09-21T03:56:33.346 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-09-21T03:56:33.346 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-09-21T03:56:33.346 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-09-21T03:56:33.346 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-09-21T03:56:33.346 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-09-21T03:56:33.346 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-09-21T03:56:33.346 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:33.346 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.346 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.346 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.347 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.347 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_file": { 2024-09-21T03:56:33.347 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_file", 2024-09-21T03:56:33.347 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-09-21T03:56:33.347 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.347 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:33.347 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-09-21T03:56:33.347 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.347 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.347 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.347 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:33.347 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.347 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.347 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.347 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.348 INFO:teuthology.orchestra.run.smithi039.stdout: "retain_interval": { 2024-09-21T03:56:33.348 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "retain_interval", 2024-09-21T03:56:33.348 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "secs", 2024-09-21T03:56:33.348 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.348 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:33.348 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "31536000", 2024-09-21T03:56:33.348 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.348 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.348 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.348 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "how long to retain crashes before pruning them", 2024-09-21T03:56:33.348 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.348 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.348 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.349 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.349 INFO:teuthology.orchestra.run.smithi039.stdout: "sqlite3_killpoint": { 2024-09-21T03:56:33.349 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:56:33.349 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-09-21T03:56:33.349 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "dev", 2024-09-21T03:56:33.349 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:33.349 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "0", 2024-09-21T03:56:33.349 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.349 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.349 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.349 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:33.349 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.349 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.350 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.350 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.350 INFO:teuthology.orchestra.run.smithi039.stdout: "warn_recent_interval": { 2024-09-21T03:56:33.350 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "warn_recent_interval", 2024-09-21T03:56:33.350 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "secs", 2024-09-21T03:56:33.350 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.350 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:33.350 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "1209600", 2024-09-21T03:56:33.350 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.350 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.350 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.350 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "time interval in which to warn about recent crashes", 2024-09-21T03:56:33.350 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.350 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.350 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.351 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-21T03:56:33.351 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-21T03:56:33.351 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.351 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-21T03:56:33.351 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "dashboard", 2024-09-21T03:56:33.351 INFO:teuthology.orchestra.run.smithi039.stdout: "can_run": true, 2024-09-21T03:56:33.351 INFO:teuthology.orchestra.run.smithi039.stdout: "error_string": "", 2024-09-21T03:56:33.351 INFO:teuthology.orchestra.run.smithi039.stdout: "module_options": { 2024-09-21T03:56:33.351 INFO:teuthology.orchestra.run.smithi039.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-09-21T03:56:33.351 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-09-21T03:56:33.351 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-09-21T03:56:33.351 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.351 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-09-21T03:56:33.351 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "10", 2024-09-21T03:56:33.351 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.352 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.352 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.352 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:33.352 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.352 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.352 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.352 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.352 INFO:teuthology.orchestra.run.smithi039.stdout: "ALERTMANAGER_API_HOST": { 2024-09-21T03:56:33.352 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "ALERTMANAGER_API_HOST", 2024-09-21T03:56:33.352 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-09-21T03:56:33.352 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.352 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-09-21T03:56:33.352 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-09-21T03:56:33.352 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.353 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.353 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.353 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:33.353 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.353 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.353 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.353 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.353 INFO:teuthology.orchestra.run.smithi039.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-09-21T03:56:33.353 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-09-21T03:56:33.353 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-09-21T03:56:33.353 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.353 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-09-21T03:56:33.353 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2024-09-21T03:56:33.354 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.354 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.354 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.354 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:33.354 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.354 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.354 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.354 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.354 INFO:teuthology.orchestra.run.smithi039.stdout: "AUDIT_API_ENABLED": { 2024-09-21T03:56:33.354 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "AUDIT_API_ENABLED", 2024-09-21T03:56:33.354 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-09-21T03:56:33.354 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.354 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-09-21T03:56:33.354 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-09-21T03:56:33.355 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.355 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.355 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.355 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:33.355 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.355 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.355 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.355 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.355 INFO:teuthology.orchestra.run.smithi039.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-09-21T03:56:33.355 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-09-21T03:56:33.355 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-09-21T03:56:33.355 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.355 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-09-21T03:56:33.356 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2024-09-21T03:56:33.356 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.356 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.356 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.356 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:33.356 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.356 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.356 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.356 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.356 INFO:teuthology.orchestra.run.smithi039.stdout: "ENABLE_BROWSABLE_API": { 2024-09-21T03:56:33.356 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "ENABLE_BROWSABLE_API", 2024-09-21T03:56:33.356 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-09-21T03:56:33.356 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.356 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-09-21T03:56:33.357 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2024-09-21T03:56:33.357 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.357 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.357 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.357 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:33.357 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.357 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.357 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.357 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.357 INFO:teuthology.orchestra.run.smithi039.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-09-21T03:56:33.357 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-09-21T03:56:33.357 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-09-21T03:56:33.357 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.357 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-09-21T03:56:33.358 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2024-09-21T03:56:33.358 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.358 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.358 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.358 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:33.358 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.358 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.358 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.358 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.358 INFO:teuthology.orchestra.run.smithi039.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-09-21T03:56:33.358 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-09-21T03:56:33.358 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-09-21T03:56:33.358 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.359 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-09-21T03:56:33.359 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2024-09-21T03:56:33.359 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.359 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.359 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.359 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:33.359 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.359 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.359 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.359 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.359 INFO:teuthology.orchestra.run.smithi039.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-09-21T03:56:33.359 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-09-21T03:56:33.359 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-09-21T03:56:33.359 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.360 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-09-21T03:56:33.360 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2024-09-21T03:56:33.360 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.360 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.360 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.360 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:33.360 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.360 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.360 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.360 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.360 INFO:teuthology.orchestra.run.smithi039.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-09-21T03:56:33.360 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-09-21T03:56:33.360 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-09-21T03:56:33.360 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.361 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-09-21T03:56:33.361 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2024-09-21T03:56:33.361 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.361 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.361 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.361 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:33.361 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.361 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.361 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.361 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.361 INFO:teuthology.orchestra.run.smithi039.stdout: "FEATURE_TOGGLE_NFS": { 2024-09-21T03:56:33.361 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-09-21T03:56:33.361 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-09-21T03:56:33.361 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.361 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-09-21T03:56:33.362 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2024-09-21T03:56:33.362 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.362 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.362 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.362 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:33.362 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.362 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.362 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.362 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.362 INFO:teuthology.orchestra.run.smithi039.stdout: "FEATURE_TOGGLE_RBD": { 2024-09-21T03:56:33.362 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-09-21T03:56:33.362 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-09-21T03:56:33.362 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.362 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-09-21T03:56:33.363 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2024-09-21T03:56:33.363 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.363 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.363 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.363 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:33.363 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.363 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.363 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.363 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.363 INFO:teuthology.orchestra.run.smithi039.stdout: "FEATURE_TOGGLE_RGW": { 2024-09-21T03:56:33.363 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-09-21T03:56:33.363 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-09-21T03:56:33.363 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.363 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-09-21T03:56:33.363 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2024-09-21T03:56:33.364 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.364 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.364 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.364 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:33.364 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.364 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.364 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.364 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.364 INFO:teuthology.orchestra.run.smithi039.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-09-21T03:56:33.364 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-09-21T03:56:33.364 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-09-21T03:56:33.364 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.364 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-09-21T03:56:33.365 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-09-21T03:56:33.365 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.365 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.365 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.365 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:33.365 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.365 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.365 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.365 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.365 INFO:teuthology.orchestra.run.smithi039.stdout: "GRAFANA_API_PASSWORD": { 2024-09-21T03:56:33.365 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "GRAFANA_API_PASSWORD", 2024-09-21T03:56:33.365 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-09-21T03:56:33.365 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.366 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-09-21T03:56:33.366 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "admin", 2024-09-21T03:56:33.366 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.366 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.366 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.366 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:33.366 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.366 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.366 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.366 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.366 INFO:teuthology.orchestra.run.smithi039.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-09-21T03:56:33.366 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-09-21T03:56:33.366 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-09-21T03:56:33.366 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.367 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-09-21T03:56:33.367 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2024-09-21T03:56:33.367 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.367 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.367 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.367 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:33.367 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.367 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.367 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.367 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.367 INFO:teuthology.orchestra.run.smithi039.stdout: "GRAFANA_API_URL": { 2024-09-21T03:56:33.367 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "GRAFANA_API_URL", 2024-09-21T03:56:33.367 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-09-21T03:56:33.367 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.367 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-09-21T03:56:33.368 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-09-21T03:56:33.368 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.368 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.368 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.368 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:33.368 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.368 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.368 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.368 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.368 INFO:teuthology.orchestra.run.smithi039.stdout: "GRAFANA_API_USERNAME": { 2024-09-21T03:56:33.368 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "GRAFANA_API_USERNAME", 2024-09-21T03:56:33.368 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-09-21T03:56:33.368 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.369 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-09-21T03:56:33.369 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "admin", 2024-09-21T03:56:33.369 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.369 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.369 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.369 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:33.369 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.369 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.369 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.369 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.369 INFO:teuthology.orchestra.run.smithi039.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-09-21T03:56:33.369 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-09-21T03:56:33.369 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-09-21T03:56:33.369 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.369 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-09-21T03:56:33.370 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-09-21T03:56:33.370 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.370 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.370 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.370 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:33.370 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.370 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.370 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.370 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.370 INFO:teuthology.orchestra.run.smithi039.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-09-21T03:56:33.370 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-09-21T03:56:33.370 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-09-21T03:56:33.370 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.370 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-09-21T03:56:33.371 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-09-21T03:56:33.371 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.371 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.371 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.371 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:33.371 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.371 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.371 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.371 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.371 INFO:teuthology.orchestra.run.smithi039.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-09-21T03:56:33.371 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-09-21T03:56:33.371 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-09-21T03:56:33.371 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.371 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-09-21T03:56:33.371 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2024-09-21T03:56:33.371 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.372 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.372 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.372 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:33.372 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.372 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.372 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.372 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.372 INFO:teuthology.orchestra.run.smithi039.stdout: "ISSUE_TRACKER_API_KEY": { 2024-09-21T03:56:33.372 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-09-21T03:56:33.372 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-09-21T03:56:33.372 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.372 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-09-21T03:56:33.372 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-09-21T03:56:33.372 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.373 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.373 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.373 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:33.373 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.373 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.373 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.373 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.373 INFO:teuthology.orchestra.run.smithi039.stdout: "MANAGED_BY_CLUSTERS": { 2024-09-21T03:56:33.373 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "MANAGED_BY_CLUSTERS", 2024-09-21T03:56:33.373 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-09-21T03:56:33.373 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.373 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-09-21T03:56:33.373 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "[]", 2024-09-21T03:56:33.373 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.373 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.373 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.374 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:33.374 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.374 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.374 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.374 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.374 INFO:teuthology.orchestra.run.smithi039.stdout: "MULTICLUSTER_CONFIG": { 2024-09-21T03:56:33.374 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "MULTICLUSTER_CONFIG", 2024-09-21T03:56:33.374 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-09-21T03:56:33.374 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.374 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-09-21T03:56:33.374 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "{}", 2024-09-21T03:56:33.374 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.374 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.375 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.375 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:33.375 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.375 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.375 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.375 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.375 INFO:teuthology.orchestra.run.smithi039.stdout: "PROMETHEUS_API_HOST": { 2024-09-21T03:56:33.375 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "PROMETHEUS_API_HOST", 2024-09-21T03:56:33.375 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-09-21T03:56:33.375 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.375 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-09-21T03:56:33.375 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-09-21T03:56:33.375 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.375 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.375 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.376 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:33.376 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.376 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.376 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.376 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.376 INFO:teuthology.orchestra.run.smithi039.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-09-21T03:56:33.376 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-09-21T03:56:33.376 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-09-21T03:56:33.376 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.376 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-09-21T03:56:33.376 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2024-09-21T03:56:33.376 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.376 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.376 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.377 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:33.377 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.377 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.377 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.377 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.377 INFO:teuthology.orchestra.run.smithi039.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-09-21T03:56:33.377 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-09-21T03:56:33.377 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-09-21T03:56:33.377 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.377 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-09-21T03:56:33.377 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-09-21T03:56:33.377 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.377 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.377 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.378 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:33.378 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.378 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.378 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.378 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.378 INFO:teuthology.orchestra.run.smithi039.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-09-21T03:56:33.378 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-09-21T03:56:33.378 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-09-21T03:56:33.378 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.378 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-09-21T03:56:33.378 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-09-21T03:56:33.378 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.378 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.378 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.379 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:33.379 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.379 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.379 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.379 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.379 INFO:teuthology.orchestra.run.smithi039.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-09-21T03:56:33.379 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-09-21T03:56:33.379 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-09-21T03:56:33.379 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.379 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-09-21T03:56:33.379 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2024-09-21T03:56:33.379 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.379 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.379 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.380 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:33.380 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.380 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.380 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.380 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.380 INFO:teuthology.orchestra.run.smithi039.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-09-21T03:56:33.380 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-09-21T03:56:33.380 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-09-21T03:56:33.380 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.380 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-09-21T03:56:33.380 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2024-09-21T03:56:33.380 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.380 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.380 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.380 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:33.381 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.381 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.381 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.381 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.381 INFO:teuthology.orchestra.run.smithi039.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-09-21T03:56:33.381 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-09-21T03:56:33.381 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-09-21T03:56:33.381 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.381 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-09-21T03:56:33.381 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-09-21T03:56:33.381 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.381 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.381 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.381 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:33.382 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.382 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.382 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.382 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.382 INFO:teuthology.orchestra.run.smithi039.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-09-21T03:56:33.382 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-09-21T03:56:33.382 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-09-21T03:56:33.382 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.382 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-09-21T03:56:33.382 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-09-21T03:56:33.382 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.382 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.382 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.383 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:33.383 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.383 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.383 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.383 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.383 INFO:teuthology.orchestra.run.smithi039.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-09-21T03:56:33.383 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-09-21T03:56:33.383 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-09-21T03:56:33.383 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.383 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-09-21T03:56:33.383 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-09-21T03:56:33.383 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.383 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.383 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.383 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:33.384 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.384 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.384 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.384 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.384 INFO:teuthology.orchestra.run.smithi039.stdout: "PWD_POLICY_ENABLED": { 2024-09-21T03:56:33.384 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "PWD_POLICY_ENABLED", 2024-09-21T03:56:33.384 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-09-21T03:56:33.384 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.384 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-09-21T03:56:33.384 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2024-09-21T03:56:33.384 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.384 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.384 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.384 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:33.384 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.385 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.385 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.385 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.385 INFO:teuthology.orchestra.run.smithi039.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-09-21T03:56:33.385 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-09-21T03:56:33.385 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-09-21T03:56:33.385 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.385 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-09-21T03:56:33.385 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-09-21T03:56:33.385 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.385 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.385 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.385 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:33.385 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.385 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.386 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.386 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.386 INFO:teuthology.orchestra.run.smithi039.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-09-21T03:56:33.386 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-09-21T03:56:33.386 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-09-21T03:56:33.386 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.386 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-09-21T03:56:33.386 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "10", 2024-09-21T03:56:33.386 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.386 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.386 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.386 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:33.387 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.387 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.387 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.387 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.387 INFO:teuthology.orchestra.run.smithi039.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-09-21T03:56:33.387 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-09-21T03:56:33.387 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-09-21T03:56:33.387 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.387 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-09-21T03:56:33.387 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "8", 2024-09-21T03:56:33.387 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.387 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.388 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.388 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:33.388 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.388 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.388 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.388 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.388 INFO:teuthology.orchestra.run.smithi039.stdout: "REST_REQUESTS_TIMEOUT": { 2024-09-21T03:56:33.388 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-09-21T03:56:33.388 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-09-21T03:56:33.388 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.388 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-09-21T03:56:33.388 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "45", 2024-09-21T03:56:33.388 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.388 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.388 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.389 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:33.389 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.389 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.389 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.389 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.389 INFO:teuthology.orchestra.run.smithi039.stdout: "RGW_API_ACCESS_KEY": { 2024-09-21T03:56:33.389 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "RGW_API_ACCESS_KEY", 2024-09-21T03:56:33.389 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-09-21T03:56:33.389 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.389 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-09-21T03:56:33.389 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-09-21T03:56:33.389 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.389 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.389 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.390 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:33.390 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.390 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.390 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.390 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.390 INFO:teuthology.orchestra.run.smithi039.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-09-21T03:56:33.390 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-09-21T03:56:33.390 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-09-21T03:56:33.390 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.390 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-09-21T03:56:33.390 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "admin", 2024-09-21T03:56:33.390 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.390 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.390 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.391 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:33.391 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.391 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.391 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.391 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.391 INFO:teuthology.orchestra.run.smithi039.stdout: "RGW_API_SECRET_KEY": { 2024-09-21T03:56:33.391 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "RGW_API_SECRET_KEY", 2024-09-21T03:56:33.391 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-09-21T03:56:33.391 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.391 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-09-21T03:56:33.391 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-09-21T03:56:33.391 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.391 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.392 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.392 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:33.392 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.392 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.392 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.392 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.392 INFO:teuthology.orchestra.run.smithi039.stdout: "RGW_API_SSL_VERIFY": { 2024-09-21T03:56:33.392 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "RGW_API_SSL_VERIFY", 2024-09-21T03:56:33.392 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-09-21T03:56:33.392 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.392 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-09-21T03:56:33.392 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2024-09-21T03:56:33.392 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.392 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.392 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.393 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:33.393 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.393 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.393 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.393 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.393 INFO:teuthology.orchestra.run.smithi039.stdout: "UNSAFE_TLS_v1_2": { 2024-09-21T03:56:33.393 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "UNSAFE_TLS_v1_2", 2024-09-21T03:56:33.393 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-09-21T03:56:33.393 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.393 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-09-21T03:56:33.393 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-09-21T03:56:33.393 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.393 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.393 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.394 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:33.394 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.394 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.394 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.394 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.394 INFO:teuthology.orchestra.run.smithi039.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-09-21T03:56:33.394 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-09-21T03:56:33.394 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-09-21T03:56:33.394 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.394 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-09-21T03:56:33.394 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "0", 2024-09-21T03:56:33.394 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.394 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.394 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.394 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:33.395 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.395 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.395 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.395 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.395 INFO:teuthology.orchestra.run.smithi039.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-09-21T03:56:33.395 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-09-21T03:56:33.395 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-09-21T03:56:33.395 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.395 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-09-21T03:56:33.395 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "10", 2024-09-21T03:56:33.395 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.395 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.395 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.395 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:33.395 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.396 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.396 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.396 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.396 INFO:teuthology.orchestra.run.smithi039.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-09-21T03:56:33.396 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-09-21T03:56:33.396 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-09-21T03:56:33.396 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.396 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-09-21T03:56:33.396 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "5", 2024-09-21T03:56:33.396 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.396 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.396 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.396 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:33.396 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.396 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.397 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.397 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.397 INFO:teuthology.orchestra.run.smithi039.stdout: "cross_origin_url": { 2024-09-21T03:56:33.397 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "cross_origin_url", 2024-09-21T03:56:33.397 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-09-21T03:56:33.397 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.397 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-09-21T03:56:33.397 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-09-21T03:56:33.397 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.397 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.397 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.397 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:33.397 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.397 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.397 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.398 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.398 INFO:teuthology.orchestra.run.smithi039.stdout: "crt_file": { 2024-09-21T03:56:33.398 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "crt_file", 2024-09-21T03:56:33.398 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-09-21T03:56:33.398 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.398 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-09-21T03:56:33.398 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-09-21T03:56:33.398 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.398 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.398 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.398 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:33.398 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.398 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.398 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.398 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.399 INFO:teuthology.orchestra.run.smithi039.stdout: "debug": { 2024-09-21T03:56:33.399 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "debug", 2024-09-21T03:56:33.399 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-09-21T03:56:33.399 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.399 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-09-21T03:56:33.399 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-09-21T03:56:33.399 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.399 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.399 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.399 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Enable/disable debug options", 2024-09-21T03:56:33.399 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.399 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.399 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.399 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.400 INFO:teuthology.orchestra.run.smithi039.stdout: "jwt_token_ttl": { 2024-09-21T03:56:33.400 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "jwt_token_ttl", 2024-09-21T03:56:33.400 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-09-21T03:56:33.400 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.400 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-09-21T03:56:33.400 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "28800", 2024-09-21T03:56:33.400 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.400 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.400 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.400 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:33.400 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.400 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.400 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.400 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.401 INFO:teuthology.orchestra.run.smithi039.stdout: "key_file": { 2024-09-21T03:56:33.401 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "key_file", 2024-09-21T03:56:33.401 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-09-21T03:56:33.401 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.401 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-09-21T03:56:33.401 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-09-21T03:56:33.401 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.401 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.401 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.401 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:33.401 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.401 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.401 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.401 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.402 INFO:teuthology.orchestra.run.smithi039.stdout: "log_level": { 2024-09-21T03:56:33.402 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_level", 2024-09-21T03:56:33.402 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-09-21T03:56:33.402 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.402 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:33.402 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-09-21T03:56:33.402 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.402 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.402 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-09-21T03:56:33.402 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-09-21T03:56:33.402 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-09-21T03:56:33.402 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-09-21T03:56:33.402 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-09-21T03:56:33.402 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-09-21T03:56:33.403 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-09-21T03:56:33.403 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-09-21T03:56:33.403 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:33.403 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.403 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.403 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.403 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.403 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster": { 2024-09-21T03:56:33.403 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster", 2024-09-21T03:56:33.403 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-09-21T03:56:33.403 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.403 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:33.403 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-09-21T03:56:33.403 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.403 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.404 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.404 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:33.404 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.404 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.404 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.404 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.404 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster_level": { 2024-09-21T03:56:33.404 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster_level", 2024-09-21T03:56:33.404 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-09-21T03:56:33.404 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.404 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:33.404 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "info", 2024-09-21T03:56:33.404 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.405 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.405 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-09-21T03:56:33.405 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-09-21T03:56:33.405 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-09-21T03:56:33.405 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-09-21T03:56:33.405 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-09-21T03:56:33.405 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-09-21T03:56:33.405 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-09-21T03:56:33.405 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-09-21T03:56:33.405 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:33.405 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.405 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.405 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.405 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.405 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_file": { 2024-09-21T03:56:33.405 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_file", 2024-09-21T03:56:33.406 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-09-21T03:56:33.406 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.406 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:33.406 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-09-21T03:56:33.406 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.406 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.406 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.406 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:33.406 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.406 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.406 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.406 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.406 INFO:teuthology.orchestra.run.smithi039.stdout: "motd": { 2024-09-21T03:56:33.406 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "motd", 2024-09-21T03:56:33.407 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-09-21T03:56:33.407 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.407 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-09-21T03:56:33.407 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-09-21T03:56:33.407 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.407 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.407 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.407 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "The message of the day", 2024-09-21T03:56:33.407 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.407 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.407 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.407 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.407 INFO:teuthology.orchestra.run.smithi039.stdout: "redirect_resolve_ip_addr": { 2024-09-21T03:56:33.407 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "redirect_resolve_ip_addr", 2024-09-21T03:56:33.407 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-09-21T03:56:33.408 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.408 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-09-21T03:56:33.408 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-09-21T03:56:33.408 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.408 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.408 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.408 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:33.408 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.408 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.408 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.408 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.408 INFO:teuthology.orchestra.run.smithi039.stdout: "server_addr": { 2024-09-21T03:56:33.408 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "server_addr", 2024-09-21T03:56:33.408 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-09-21T03:56:33.408 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.408 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-09-21T03:56:33.409 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "::", 2024-09-21T03:56:33.409 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.409 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.409 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.409 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:33.409 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.409 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.409 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.409 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.409 INFO:teuthology.orchestra.run.smithi039.stdout: "server_port": { 2024-09-21T03:56:33.409 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "server_port", 2024-09-21T03:56:33.409 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-09-21T03:56:33.409 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.409 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-09-21T03:56:33.410 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "8080", 2024-09-21T03:56:33.410 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.410 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.410 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.410 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:33.410 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.410 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.410 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.410 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.410 INFO:teuthology.orchestra.run.smithi039.stdout: "sqlite3_killpoint": { 2024-09-21T03:56:33.410 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:56:33.410 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-09-21T03:56:33.410 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "dev", 2024-09-21T03:56:33.410 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:33.410 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "0", 2024-09-21T03:56:33.411 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.411 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.411 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.411 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:33.411 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.411 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.411 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.411 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.411 INFO:teuthology.orchestra.run.smithi039.stdout: "ssl": { 2024-09-21T03:56:33.411 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "ssl", 2024-09-21T03:56:33.411 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-09-21T03:56:33.411 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.411 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-09-21T03:56:33.411 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2024-09-21T03:56:33.411 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.411 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.412 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.412 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:33.412 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.412 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.412 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.412 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.412 INFO:teuthology.orchestra.run.smithi039.stdout: "ssl_server_port": { 2024-09-21T03:56:33.412 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "ssl_server_port", 2024-09-21T03:56:33.412 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-09-21T03:56:33.412 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.412 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-09-21T03:56:33.412 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "8443", 2024-09-21T03:56:33.412 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.412 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.412 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.413 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:33.413 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.413 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.413 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.413 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.413 INFO:teuthology.orchestra.run.smithi039.stdout: "sso_oauth2": { 2024-09-21T03:56:33.413 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "sso_oauth2", 2024-09-21T03:56:33.413 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-09-21T03:56:33.413 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.413 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-09-21T03:56:33.413 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-09-21T03:56:33.413 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.414 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.414 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.414 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:33.414 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.414 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.414 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.414 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.414 INFO:teuthology.orchestra.run.smithi039.stdout: "standby_behaviour": { 2024-09-21T03:56:33.414 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "standby_behaviour", 2024-09-21T03:56:33.414 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-09-21T03:56:33.414 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.414 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-09-21T03:56:33.415 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "redirect", 2024-09-21T03:56:33.415 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.415 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.415 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-09-21T03:56:33.415 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-09-21T03:56:33.415 INFO:teuthology.orchestra.run.smithi039.stdout: "redirect" 2024-09-21T03:56:33.415 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-09-21T03:56:33.415 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:33.415 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.415 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.415 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.415 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.415 INFO:teuthology.orchestra.run.smithi039.stdout: "standby_error_status_code": { 2024-09-21T03:56:33.415 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "standby_error_status_code", 2024-09-21T03:56:33.416 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-09-21T03:56:33.416 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.416 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-09-21T03:56:33.416 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "500", 2024-09-21T03:56:33.416 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "400", 2024-09-21T03:56:33.416 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "599", 2024-09-21T03:56:33.416 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.416 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:33.416 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.416 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.416 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.416 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.416 INFO:teuthology.orchestra.run.smithi039.stdout: "url_prefix": { 2024-09-21T03:56:33.417 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "url_prefix", 2024-09-21T03:56:33.417 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-09-21T03:56:33.417 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.417 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-09-21T03:56:33.417 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-09-21T03:56:33.417 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.417 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.417 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.417 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:33.417 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.417 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.417 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.417 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-21T03:56:33.417 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-21T03:56:33.417 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.417 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-21T03:56:33.418 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "devicehealth", 2024-09-21T03:56:33.418 INFO:teuthology.orchestra.run.smithi039.stdout: "can_run": true, 2024-09-21T03:56:33.418 INFO:teuthology.orchestra.run.smithi039.stdout: "error_string": "", 2024-09-21T03:56:33.418 INFO:teuthology.orchestra.run.smithi039.stdout: "module_options": { 2024-09-21T03:56:33.418 INFO:teuthology.orchestra.run.smithi039.stdout: "enable_monitoring": { 2024-09-21T03:56:33.418 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "enable_monitoring", 2024-09-21T03:56:33.418 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-09-21T03:56:33.418 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.418 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:33.418 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2024-09-21T03:56:33.418 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.418 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.418 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.418 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "monitor device health metrics", 2024-09-21T03:56:33.419 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.419 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.419 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.419 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.419 INFO:teuthology.orchestra.run.smithi039.stdout: "log_level": { 2024-09-21T03:56:33.419 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_level", 2024-09-21T03:56:33.419 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-09-21T03:56:33.419 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.419 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:33.419 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-09-21T03:56:33.419 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.419 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.419 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-09-21T03:56:33.419 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-09-21T03:56:33.420 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-09-21T03:56:33.420 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-09-21T03:56:33.420 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-09-21T03:56:33.420 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-09-21T03:56:33.420 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-09-21T03:56:33.420 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-09-21T03:56:33.420 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:33.420 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.420 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.420 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.420 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.420 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster": { 2024-09-21T03:56:33.420 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster", 2024-09-21T03:56:33.420 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-09-21T03:56:33.420 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.420 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:33.421 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-09-21T03:56:33.421 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.421 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.421 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.421 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:33.421 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.421 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.421 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.421 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.421 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster_level": { 2024-09-21T03:56:33.421 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster_level", 2024-09-21T03:56:33.421 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-09-21T03:56:33.421 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.422 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:33.422 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "info", 2024-09-21T03:56:33.422 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.422 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.422 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-09-21T03:56:33.422 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-09-21T03:56:33.422 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-09-21T03:56:33.422 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-09-21T03:56:33.422 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-09-21T03:56:33.422 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-09-21T03:56:33.422 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-09-21T03:56:33.422 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-09-21T03:56:33.422 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:33.423 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.423 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.423 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.423 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.423 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_file": { 2024-09-21T03:56:33.423 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_file", 2024-09-21T03:56:33.423 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-09-21T03:56:33.423 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.423 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:33.423 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-09-21T03:56:33.423 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.423 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.423 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.424 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:33.424 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.424 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.424 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.424 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.424 INFO:teuthology.orchestra.run.smithi039.stdout: "mark_out_threshold": { 2024-09-21T03:56:33.424 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "mark_out_threshold", 2024-09-21T03:56:33.424 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "secs", 2024-09-21T03:56:33.424 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.424 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:33.424 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "2419200", 2024-09-21T03:56:33.424 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.424 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.425 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.425 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-09-21T03:56:33.425 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.425 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.425 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.425 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.425 INFO:teuthology.orchestra.run.smithi039.stdout: "pool_name": { 2024-09-21T03:56:33.425 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "pool_name", 2024-09-21T03:56:33.425 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-09-21T03:56:33.425 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.425 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:33.425 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "device_health_metrics", 2024-09-21T03:56:33.425 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.425 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.426 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.426 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "name of pool in which to store device health metrics", 2024-09-21T03:56:33.426 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.426 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.426 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.426 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.426 INFO:teuthology.orchestra.run.smithi039.stdout: "retention_period": { 2024-09-21T03:56:33.426 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "retention_period", 2024-09-21T03:56:33.426 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "secs", 2024-09-21T03:56:33.426 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.426 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:33.426 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "15552000", 2024-09-21T03:56:33.426 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.426 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.426 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.427 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "how long to retain device health metrics", 2024-09-21T03:56:33.427 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.427 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.427 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.427 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.427 INFO:teuthology.orchestra.run.smithi039.stdout: "scrape_frequency": { 2024-09-21T03:56:33.427 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "scrape_frequency", 2024-09-21T03:56:33.427 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "secs", 2024-09-21T03:56:33.427 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.427 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:33.427 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "86400", 2024-09-21T03:56:33.427 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.428 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.428 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.428 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "how frequently to scrape device health metrics", 2024-09-21T03:56:33.428 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.428 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.428 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.428 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.428 INFO:teuthology.orchestra.run.smithi039.stdout: "self_heal": { 2024-09-21T03:56:33.428 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "self_heal", 2024-09-21T03:56:33.428 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-09-21T03:56:33.428 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.428 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:33.428 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2024-09-21T03:56:33.428 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.429 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.429 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.429 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-09-21T03:56:33.429 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.429 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.429 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.429 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.429 INFO:teuthology.orchestra.run.smithi039.stdout: "sleep_interval": { 2024-09-21T03:56:33.429 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "sleep_interval", 2024-09-21T03:56:33.429 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "secs", 2024-09-21T03:56:33.429 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.429 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:33.429 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "600", 2024-09-21T03:56:33.429 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.430 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.430 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.430 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "how frequently to wake up and check device health", 2024-09-21T03:56:33.430 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.430 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.430 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.430 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.430 INFO:teuthology.orchestra.run.smithi039.stdout: "sqlite3_killpoint": { 2024-09-21T03:56:33.430 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:56:33.430 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-09-21T03:56:33.430 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "dev", 2024-09-21T03:56:33.430 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:33.430 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "0", 2024-09-21T03:56:33.430 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.431 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.431 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.431 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:33.431 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.431 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.431 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.431 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.431 INFO:teuthology.orchestra.run.smithi039.stdout: "warn_threshold": { 2024-09-21T03:56:33.431 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "warn_threshold", 2024-09-21T03:56:33.431 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "secs", 2024-09-21T03:56:33.431 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.431 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:33.431 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "7257600", 2024-09-21T03:56:33.431 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.432 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.432 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.432 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-09-21T03:56:33.432 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.432 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.432 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.432 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-21T03:56:33.432 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-21T03:56:33.432 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.432 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-21T03:56:33.432 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "diskprediction_local", 2024-09-21T03:56:33.432 INFO:teuthology.orchestra.run.smithi039.stdout: "can_run": true, 2024-09-21T03:56:33.432 INFO:teuthology.orchestra.run.smithi039.stdout: "error_string": "", 2024-09-21T03:56:33.432 INFO:teuthology.orchestra.run.smithi039.stdout: "module_options": { 2024-09-21T03:56:33.433 INFO:teuthology.orchestra.run.smithi039.stdout: "log_level": { 2024-09-21T03:56:33.433 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_level", 2024-09-21T03:56:33.433 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-09-21T03:56:33.433 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.433 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:33.433 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-09-21T03:56:33.433 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.433 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.433 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-09-21T03:56:33.433 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-09-21T03:56:33.433 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-09-21T03:56:33.433 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-09-21T03:56:33.433 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-09-21T03:56:33.433 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-09-21T03:56:33.434 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-09-21T03:56:33.434 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-09-21T03:56:33.434 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:33.434 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.434 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.434 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.434 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.434 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster": { 2024-09-21T03:56:33.434 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster", 2024-09-21T03:56:33.434 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-09-21T03:56:33.434 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.434 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:33.434 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-09-21T03:56:33.435 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.435 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.435 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.435 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:33.435 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.435 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.435 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.435 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.435 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster_level": { 2024-09-21T03:56:33.435 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster_level", 2024-09-21T03:56:33.435 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-09-21T03:56:33.435 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.435 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:33.435 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "info", 2024-09-21T03:56:33.435 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.436 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.436 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-09-21T03:56:33.436 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-09-21T03:56:33.436 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-09-21T03:56:33.436 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-09-21T03:56:33.436 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-09-21T03:56:33.436 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-09-21T03:56:33.436 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-09-21T03:56:33.436 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-09-21T03:56:33.436 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:33.436 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.436 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.436 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.436 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.436 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_file": { 2024-09-21T03:56:33.437 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_file", 2024-09-21T03:56:33.437 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-09-21T03:56:33.437 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.437 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:33.437 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-09-21T03:56:33.437 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.437 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.437 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.437 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:33.437 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.437 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.437 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.437 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.437 INFO:teuthology.orchestra.run.smithi039.stdout: "predict_interval": { 2024-09-21T03:56:33.437 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "predict_interval", 2024-09-21T03:56:33.437 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-09-21T03:56:33.438 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.438 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-09-21T03:56:33.438 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "86400", 2024-09-21T03:56:33.438 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.438 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.438 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.438 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:33.438 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.438 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.438 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.438 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.438 INFO:teuthology.orchestra.run.smithi039.stdout: "predictor_model": { 2024-09-21T03:56:33.438 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "predictor_model", 2024-09-21T03:56:33.438 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-09-21T03:56:33.439 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.439 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-09-21T03:56:33.439 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "prophetstor", 2024-09-21T03:56:33.439 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.439 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.439 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.439 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:33.439 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.439 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.439 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.439 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.439 INFO:teuthology.orchestra.run.smithi039.stdout: "sleep_interval": { 2024-09-21T03:56:33.439 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "sleep_interval", 2024-09-21T03:56:33.439 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-09-21T03:56:33.439 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.440 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-09-21T03:56:33.440 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "600", 2024-09-21T03:56:33.440 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.440 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.440 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.440 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:33.440 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.440 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.440 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.440 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.440 INFO:teuthology.orchestra.run.smithi039.stdout: "sqlite3_killpoint": { 2024-09-21T03:56:33.440 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:56:33.440 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-09-21T03:56:33.440 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "dev", 2024-09-21T03:56:33.440 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:33.441 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "0", 2024-09-21T03:56:33.441 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.441 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.441 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.441 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:33.441 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.441 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.441 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.441 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-21T03:56:33.441 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-21T03:56:33.441 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.441 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-21T03:56:33.441 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "influx", 2024-09-21T03:56:33.441 INFO:teuthology.orchestra.run.smithi039.stdout: "can_run": false, 2024-09-21T03:56:33.441 INFO:teuthology.orchestra.run.smithi039.stdout: "error_string": "influxdb python module not found", 2024-09-21T03:56:33.441 INFO:teuthology.orchestra.run.smithi039.stdout: "module_options": { 2024-09-21T03:56:33.442 INFO:teuthology.orchestra.run.smithi039.stdout: "batch_size": { 2024-09-21T03:56:33.442 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "batch_size", 2024-09-21T03:56:33.442 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-09-21T03:56:33.442 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.442 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-09-21T03:56:33.442 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "5000", 2024-09-21T03:56:33.442 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.442 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.442 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.442 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-09-21T03:56:33.442 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.442 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.442 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.443 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.443 INFO:teuthology.orchestra.run.smithi039.stdout: "database": { 2024-09-21T03:56:33.443 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "database", 2024-09-21T03:56:33.443 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-09-21T03:56:33.443 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.443 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-09-21T03:56:33.443 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "ceph", 2024-09-21T03:56:33.443 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.443 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.443 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.443 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-09-21T03:56:33.443 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.443 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.444 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.444 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.444 INFO:teuthology.orchestra.run.smithi039.stdout: "hostname": { 2024-09-21T03:56:33.444 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "hostname", 2024-09-21T03:56:33.444 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-09-21T03:56:33.444 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.444 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-09-21T03:56:33.444 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-09-21T03:56:33.444 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.444 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.444 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.444 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "InfluxDB server hostname", 2024-09-21T03:56:33.444 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.444 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.444 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.444 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.445 INFO:teuthology.orchestra.run.smithi039.stdout: "interval": { 2024-09-21T03:56:33.445 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "interval", 2024-09-21T03:56:33.445 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "secs", 2024-09-21T03:56:33.445 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.445 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-09-21T03:56:33.445 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "30", 2024-09-21T03:56:33.445 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "5", 2024-09-21T03:56:33.445 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.445 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.445 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-09-21T03:56:33.445 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.445 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.445 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.445 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.445 INFO:teuthology.orchestra.run.smithi039.stdout: "log_level": { 2024-09-21T03:56:33.445 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_level", 2024-09-21T03:56:33.446 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-09-21T03:56:33.446 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.446 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:33.446 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-09-21T03:56:33.446 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.446 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.446 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-09-21T03:56:33.446 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-09-21T03:56:33.446 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-09-21T03:56:33.446 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-09-21T03:56:33.446 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-09-21T03:56:33.446 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-09-21T03:56:33.446 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-09-21T03:56:33.446 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-09-21T03:56:33.446 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:33.447 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.447 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.447 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.447 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.447 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster": { 2024-09-21T03:56:33.447 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster", 2024-09-21T03:56:33.447 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-09-21T03:56:33.447 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.447 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:33.447 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-09-21T03:56:33.447 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.447 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.447 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.447 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:33.447 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.448 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.448 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.448 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.448 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster_level": { 2024-09-21T03:56:33.448 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster_level", 2024-09-21T03:56:33.448 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-09-21T03:56:33.448 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.448 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:33.448 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "info", 2024-09-21T03:56:33.448 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.448 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.448 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-09-21T03:56:33.448 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-09-21T03:56:33.448 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-09-21T03:56:33.449 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-09-21T03:56:33.449 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-09-21T03:56:33.449 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-09-21T03:56:33.449 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-09-21T03:56:33.449 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-09-21T03:56:33.449 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:33.449 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.449 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.449 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.449 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.449 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_file": { 2024-09-21T03:56:33.449 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_file", 2024-09-21T03:56:33.449 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-09-21T03:56:33.449 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.449 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:33.450 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-09-21T03:56:33.450 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.450 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.450 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.450 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:33.450 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.450 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.450 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.450 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.450 INFO:teuthology.orchestra.run.smithi039.stdout: "password": { 2024-09-21T03:56:33.450 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "password", 2024-09-21T03:56:33.450 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-09-21T03:56:33.450 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.450 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-09-21T03:56:33.450 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-09-21T03:56:33.451 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.451 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.451 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.451 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "password of InfluxDB server user", 2024-09-21T03:56:33.451 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.451 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.451 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.451 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.451 INFO:teuthology.orchestra.run.smithi039.stdout: "port": { 2024-09-21T03:56:33.451 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "port", 2024-09-21T03:56:33.451 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-09-21T03:56:33.451 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.451 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-09-21T03:56:33.452 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "8086", 2024-09-21T03:56:33.452 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.452 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.452 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.452 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "InfluxDB server port", 2024-09-21T03:56:33.452 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.452 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.452 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.452 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.452 INFO:teuthology.orchestra.run.smithi039.stdout: "sqlite3_killpoint": { 2024-09-21T03:56:33.452 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:56:33.452 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-09-21T03:56:33.452 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "dev", 2024-09-21T03:56:33.452 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:33.453 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "0", 2024-09-21T03:56:33.453 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.453 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.453 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.453 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:33.453 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.453 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.453 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.453 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.453 INFO:teuthology.orchestra.run.smithi039.stdout: "ssl": { 2024-09-21T03:56:33.453 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "ssl", 2024-09-21T03:56:33.453 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-09-21T03:56:33.453 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.453 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-09-21T03:56:33.454 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "false", 2024-09-21T03:56:33.454 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.454 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.454 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.454 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-09-21T03:56:33.454 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.454 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.454 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.454 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.454 INFO:teuthology.orchestra.run.smithi039.stdout: "threads": { 2024-09-21T03:56:33.454 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "threads", 2024-09-21T03:56:33.454 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-09-21T03:56:33.454 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.455 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-09-21T03:56:33.455 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "5", 2024-09-21T03:56:33.455 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "1", 2024-09-21T03:56:33.455 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "32", 2024-09-21T03:56:33.455 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.455 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-09-21T03:56:33.455 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.455 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.455 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.455 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.455 INFO:teuthology.orchestra.run.smithi039.stdout: "username": { 2024-09-21T03:56:33.455 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "username", 2024-09-21T03:56:33.455 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-09-21T03:56:33.455 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.456 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-09-21T03:56:33.456 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-09-21T03:56:33.456 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.456 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.456 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.456 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "username of InfluxDB server user", 2024-09-21T03:56:33.456 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.456 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.456 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.456 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.456 INFO:teuthology.orchestra.run.smithi039.stdout: "verify_ssl": { 2024-09-21T03:56:33.456 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "verify_ssl", 2024-09-21T03:56:33.456 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-09-21T03:56:33.456 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.457 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-09-21T03:56:33.457 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "true", 2024-09-21T03:56:33.457 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.457 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.457 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.457 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-09-21T03:56:33.457 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.457 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.457 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.457 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-21T03:56:33.457 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-21T03:56:33.457 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.457 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-21T03:56:33.457 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "insights", 2024-09-21T03:56:33.458 INFO:teuthology.orchestra.run.smithi039.stdout: "can_run": true, 2024-09-21T03:56:33.458 INFO:teuthology.orchestra.run.smithi039.stdout: "error_string": "", 2024-09-21T03:56:33.458 INFO:teuthology.orchestra.run.smithi039.stdout: "module_options": { 2024-09-21T03:56:33.458 INFO:teuthology.orchestra.run.smithi039.stdout: "log_level": { 2024-09-21T03:56:33.458 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_level", 2024-09-21T03:56:33.458 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-09-21T03:56:33.458 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.458 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:33.458 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-09-21T03:56:33.458 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.458 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.458 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-09-21T03:56:33.458 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-09-21T03:56:33.458 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-09-21T03:56:33.459 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-09-21T03:56:33.459 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-09-21T03:56:33.459 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-09-21T03:56:33.459 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-09-21T03:56:33.459 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-09-21T03:56:33.459 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:33.459 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.459 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.459 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.459 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.459 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster": { 2024-09-21T03:56:33.459 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster", 2024-09-21T03:56:33.459 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-09-21T03:56:33.459 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.460 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:33.460 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-09-21T03:56:33.460 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.460 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.460 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.460 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:33.460 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.460 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.460 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.460 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.460 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster_level": { 2024-09-21T03:56:33.460 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster_level", 2024-09-21T03:56:33.460 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-09-21T03:56:33.460 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.461 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:33.461 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "info", 2024-09-21T03:56:33.461 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.461 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.461 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-09-21T03:56:33.461 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-09-21T03:56:33.461 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-09-21T03:56:33.461 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-09-21T03:56:33.461 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-09-21T03:56:33.461 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-09-21T03:56:33.461 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-09-21T03:56:33.461 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-09-21T03:56:33.461 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:33.461 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.462 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.462 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.462 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.462 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_file": { 2024-09-21T03:56:33.462 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_file", 2024-09-21T03:56:33.462 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-09-21T03:56:33.462 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.462 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:33.462 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-09-21T03:56:33.462 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.462 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.462 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.462 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:33.462 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.463 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.463 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.463 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.463 INFO:teuthology.orchestra.run.smithi039.stdout: "sqlite3_killpoint": { 2024-09-21T03:56:33.463 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:56:33.463 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-09-21T03:56:33.463 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "dev", 2024-09-21T03:56:33.463 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:33.463 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "0", 2024-09-21T03:56:33.463 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.463 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.463 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.463 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:33.463 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.464 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.464 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.464 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-21T03:56:33.464 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-21T03:56:33.464 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.464 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-21T03:56:33.464 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "iostat", 2024-09-21T03:56:33.464 INFO:teuthology.orchestra.run.smithi039.stdout: "can_run": true, 2024-09-21T03:56:33.464 INFO:teuthology.orchestra.run.smithi039.stdout: "error_string": "", 2024-09-21T03:56:33.464 INFO:teuthology.orchestra.run.smithi039.stdout: "module_options": { 2024-09-21T03:56:33.464 INFO:teuthology.orchestra.run.smithi039.stdout: "log_level": { 2024-09-21T03:56:33.464 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_level", 2024-09-21T03:56:33.464 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-09-21T03:56:33.464 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.464 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:33.465 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-09-21T03:56:33.465 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.465 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.465 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-09-21T03:56:33.465 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-09-21T03:56:33.465 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-09-21T03:56:33.465 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-09-21T03:56:33.465 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-09-21T03:56:33.465 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-09-21T03:56:33.465 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-09-21T03:56:33.465 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-09-21T03:56:33.465 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:33.465 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.465 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.465 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.466 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.466 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster": { 2024-09-21T03:56:33.466 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster", 2024-09-21T03:56:33.466 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-09-21T03:56:33.466 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.466 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:33.466 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-09-21T03:56:33.466 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.466 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.466 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.466 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:33.466 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.466 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.467 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.467 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.467 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster_level": { 2024-09-21T03:56:33.467 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster_level", 2024-09-21T03:56:33.467 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-09-21T03:56:33.467 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.467 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:33.467 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "info", 2024-09-21T03:56:33.467 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.467 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.467 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-09-21T03:56:33.467 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-09-21T03:56:33.467 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-09-21T03:56:33.467 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-09-21T03:56:33.468 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-09-21T03:56:33.468 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-09-21T03:56:33.468 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-09-21T03:56:33.468 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-09-21T03:56:33.468 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:33.468 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.468 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.468 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.468 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.468 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_file": { 2024-09-21T03:56:33.468 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_file", 2024-09-21T03:56:33.468 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-09-21T03:56:33.468 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.468 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:33.468 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-09-21T03:56:33.469 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.469 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.469 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.469 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:33.469 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.469 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.469 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.469 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.469 INFO:teuthology.orchestra.run.smithi039.stdout: "sqlite3_killpoint": { 2024-09-21T03:56:33.469 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:56:33.469 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-09-21T03:56:33.469 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "dev", 2024-09-21T03:56:33.469 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:33.469 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "0", 2024-09-21T03:56:33.470 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.470 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.470 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.470 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:33.470 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.470 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.470 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.470 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-21T03:56:33.470 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-21T03:56:33.470 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.470 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-21T03:56:33.470 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "k8sevents", 2024-09-21T03:56:33.470 INFO:teuthology.orchestra.run.smithi039.stdout: "can_run": true, 2024-09-21T03:56:33.470 INFO:teuthology.orchestra.run.smithi039.stdout: "error_string": "", 2024-09-21T03:56:33.471 INFO:teuthology.orchestra.run.smithi039.stdout: "module_options": { 2024-09-21T03:56:33.471 INFO:teuthology.orchestra.run.smithi039.stdout: "ceph_event_retention_days": { 2024-09-21T03:56:33.471 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "ceph_event_retention_days", 2024-09-21T03:56:33.471 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-09-21T03:56:33.471 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.471 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-09-21T03:56:33.471 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "7", 2024-09-21T03:56:33.471 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.471 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.471 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.471 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Days to hold ceph event information within local cache", 2024-09-21T03:56:33.471 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.471 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.471 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.472 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.472 INFO:teuthology.orchestra.run.smithi039.stdout: "config_check_secs": { 2024-09-21T03:56:33.472 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "config_check_secs", 2024-09-21T03:56:33.472 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-09-21T03:56:33.472 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.472 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-09-21T03:56:33.472 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "10", 2024-09-21T03:56:33.472 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "10", 2024-09-21T03:56:33.472 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.472 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.472 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-09-21T03:56:33.472 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.472 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.472 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.472 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.473 INFO:teuthology.orchestra.run.smithi039.stdout: "log_level": { 2024-09-21T03:56:33.473 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_level", 2024-09-21T03:56:33.473 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-09-21T03:56:33.473 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.473 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:33.473 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-09-21T03:56:33.473 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.473 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.473 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-09-21T03:56:33.473 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-09-21T03:56:33.473 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-09-21T03:56:33.473 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-09-21T03:56:33.473 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-09-21T03:56:33.473 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-09-21T03:56:33.473 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-09-21T03:56:33.474 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-09-21T03:56:33.474 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:33.474 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.474 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.474 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.474 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.474 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster": { 2024-09-21T03:56:33.474 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster", 2024-09-21T03:56:33.474 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-09-21T03:56:33.474 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.474 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:33.474 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-09-21T03:56:33.474 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.475 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.475 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.475 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:33.475 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.475 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.475 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.475 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.475 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster_level": { 2024-09-21T03:56:33.475 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster_level", 2024-09-21T03:56:33.475 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-09-21T03:56:33.475 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.475 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:33.475 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "info", 2024-09-21T03:56:33.475 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.475 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.476 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-09-21T03:56:33.476 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-09-21T03:56:33.476 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-09-21T03:56:33.476 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-09-21T03:56:33.476 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-09-21T03:56:33.476 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-09-21T03:56:33.476 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-09-21T03:56:33.476 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-09-21T03:56:33.476 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:33.476 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.476 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.476 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.476 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.476 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_file": { 2024-09-21T03:56:33.476 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_file", 2024-09-21T03:56:33.477 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-09-21T03:56:33.477 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.477 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:33.477 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-09-21T03:56:33.477 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.477 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.477 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.477 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:33.477 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.477 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.477 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.477 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.477 INFO:teuthology.orchestra.run.smithi039.stdout: "sqlite3_killpoint": { 2024-09-21T03:56:33.477 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:56:33.478 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-09-21T03:56:33.478 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "dev", 2024-09-21T03:56:33.478 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:33.478 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "0", 2024-09-21T03:56:33.478 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.478 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.478 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.478 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:33.478 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.478 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.478 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.478 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-21T03:56:33.478 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-21T03:56:33.478 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.478 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-21T03:56:33.479 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "localpool", 2024-09-21T03:56:33.479 INFO:teuthology.orchestra.run.smithi039.stdout: "can_run": true, 2024-09-21T03:56:33.479 INFO:teuthology.orchestra.run.smithi039.stdout: "error_string": "", 2024-09-21T03:56:33.479 INFO:teuthology.orchestra.run.smithi039.stdout: "module_options": { 2024-09-21T03:56:33.479 INFO:teuthology.orchestra.run.smithi039.stdout: "failure_domain": { 2024-09-21T03:56:33.479 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "failure_domain", 2024-09-21T03:56:33.479 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-09-21T03:56:33.479 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.479 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:33.479 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "host", 2024-09-21T03:56:33.479 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.479 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.479 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.479 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "failure domain for any created local pool", 2024-09-21T03:56:33.480 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-09-21T03:56:33.480 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.480 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.480 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.480 INFO:teuthology.orchestra.run.smithi039.stdout: "log_level": { 2024-09-21T03:56:33.480 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_level", 2024-09-21T03:56:33.480 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-09-21T03:56:33.480 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.480 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:33.480 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-09-21T03:56:33.480 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.480 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.480 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-09-21T03:56:33.480 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-09-21T03:56:33.481 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-09-21T03:56:33.481 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-09-21T03:56:33.481 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-09-21T03:56:33.481 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-09-21T03:56:33.481 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-09-21T03:56:33.481 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-09-21T03:56:33.481 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:33.481 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.481 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.481 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.481 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.481 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster": { 2024-09-21T03:56:33.481 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster", 2024-09-21T03:56:33.481 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-09-21T03:56:33.482 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.482 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:33.482 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-09-21T03:56:33.482 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.482 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.482 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.482 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:33.482 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.482 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.482 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.482 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.482 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster_level": { 2024-09-21T03:56:33.482 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster_level", 2024-09-21T03:56:33.482 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-09-21T03:56:33.483 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.483 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:33.483 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "info", 2024-09-21T03:56:33.483 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.483 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.483 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-09-21T03:56:33.483 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-09-21T03:56:33.483 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-09-21T03:56:33.483 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-09-21T03:56:33.483 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-09-21T03:56:33.483 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-09-21T03:56:33.483 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-09-21T03:56:33.483 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-09-21T03:56:33.483 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:33.483 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.484 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.484 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.484 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.484 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_file": { 2024-09-21T03:56:33.484 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_file", 2024-09-21T03:56:33.484 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-09-21T03:56:33.484 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.484 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:33.484 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-09-21T03:56:33.484 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.484 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.484 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.484 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:33.485 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.485 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.485 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.485 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.485 INFO:teuthology.orchestra.run.smithi039.stdout: "min_size": { 2024-09-21T03:56:33.485 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "min_size", 2024-09-21T03:56:33.485 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-09-21T03:56:33.485 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.485 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:33.485 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-09-21T03:56:33.485 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.485 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.485 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.485 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "default min_size for any created local pool", 2024-09-21T03:56:33.486 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-09-21T03:56:33.486 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.486 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.486 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.486 INFO:teuthology.orchestra.run.smithi039.stdout: "num_rep": { 2024-09-21T03:56:33.486 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "num_rep", 2024-09-21T03:56:33.486 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-09-21T03:56:33.486 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.486 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:33.486 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "3", 2024-09-21T03:56:33.486 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.486 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.486 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.486 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "default replica count for any created local pool", 2024-09-21T03:56:33.487 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.487 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.487 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.487 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.487 INFO:teuthology.orchestra.run.smithi039.stdout: "pg_num": { 2024-09-21T03:56:33.487 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "pg_num", 2024-09-21T03:56:33.487 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-09-21T03:56:33.487 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.487 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:33.487 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "128", 2024-09-21T03:56:33.487 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.487 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.487 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.487 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "default pg_num for any created local pool", 2024-09-21T03:56:33.488 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.488 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.488 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.488 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.488 INFO:teuthology.orchestra.run.smithi039.stdout: "prefix": { 2024-09-21T03:56:33.488 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "prefix", 2024-09-21T03:56:33.488 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-09-21T03:56:33.488 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.488 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:33.488 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-09-21T03:56:33.488 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.488 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.488 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.488 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "name prefix for any created local pool", 2024-09-21T03:56:33.489 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.489 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.489 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.489 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.489 INFO:teuthology.orchestra.run.smithi039.stdout: "sqlite3_killpoint": { 2024-09-21T03:56:33.489 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:56:33.489 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-09-21T03:56:33.489 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "dev", 2024-09-21T03:56:33.489 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:33.489 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "0", 2024-09-21T03:56:33.489 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.489 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.489 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.489 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:33.490 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.490 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.490 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.490 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.490 INFO:teuthology.orchestra.run.smithi039.stdout: "subtree": { 2024-09-21T03:56:33.490 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "subtree", 2024-09-21T03:56:33.490 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-09-21T03:56:33.490 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.490 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:33.490 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "rack", 2024-09-21T03:56:33.490 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.490 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.490 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.490 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "CRUSH level for which to create a local pool", 2024-09-21T03:56:33.490 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-09-21T03:56:33.491 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.491 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.491 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-21T03:56:33.491 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-21T03:56:33.491 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.491 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-21T03:56:33.491 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "mds_autoscaler", 2024-09-21T03:56:33.491 INFO:teuthology.orchestra.run.smithi039.stdout: "can_run": true, 2024-09-21T03:56:33.491 INFO:teuthology.orchestra.run.smithi039.stdout: "error_string": "", 2024-09-21T03:56:33.491 INFO:teuthology.orchestra.run.smithi039.stdout: "module_options": { 2024-09-21T03:56:33.491 INFO:teuthology.orchestra.run.smithi039.stdout: "log_level": { 2024-09-21T03:56:33.491 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_level", 2024-09-21T03:56:33.491 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-09-21T03:56:33.491 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.492 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:33.492 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-09-21T03:56:33.492 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.492 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.492 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-09-21T03:56:33.492 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-09-21T03:56:33.492 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-09-21T03:56:33.492 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-09-21T03:56:33.492 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-09-21T03:56:33.492 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-09-21T03:56:33.492 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-09-21T03:56:33.492 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-09-21T03:56:33.492 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:33.492 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.493 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.493 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.493 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.493 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster": { 2024-09-21T03:56:33.493 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster", 2024-09-21T03:56:33.493 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-09-21T03:56:33.493 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.493 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:33.493 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-09-21T03:56:33.493 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.493 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.493 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.493 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:33.494 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.494 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.494 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.494 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.494 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster_level": { 2024-09-21T03:56:33.494 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster_level", 2024-09-21T03:56:33.494 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-09-21T03:56:33.494 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.494 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:33.494 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "info", 2024-09-21T03:56:33.494 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.494 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.494 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-09-21T03:56:33.494 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-09-21T03:56:33.495 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-09-21T03:56:33.495 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-09-21T03:56:33.495 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-09-21T03:56:33.495 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-09-21T03:56:33.495 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-09-21T03:56:33.495 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-09-21T03:56:33.495 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:33.495 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.495 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.495 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.495 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.495 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_file": { 2024-09-21T03:56:33.495 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_file", 2024-09-21T03:56:33.495 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-09-21T03:56:33.496 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.496 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:33.496 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-09-21T03:56:33.496 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.496 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.496 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.496 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:33.496 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.496 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.496 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.496 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.496 INFO:teuthology.orchestra.run.smithi039.stdout: "sqlite3_killpoint": { 2024-09-21T03:56:33.496 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:56:33.496 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-09-21T03:56:33.497 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "dev", 2024-09-21T03:56:33.497 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:33.497 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "0", 2024-09-21T03:56:33.497 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.497 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.497 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.497 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:33.497 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.497 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.497 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.497 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-21T03:56:33.497 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-21T03:56:33.497 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.497 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-21T03:56:33.498 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "mirroring", 2024-09-21T03:56:33.498 INFO:teuthology.orchestra.run.smithi039.stdout: "can_run": true, 2024-09-21T03:56:33.498 INFO:teuthology.orchestra.run.smithi039.stdout: "error_string": "", 2024-09-21T03:56:33.498 INFO:teuthology.orchestra.run.smithi039.stdout: "module_options": { 2024-09-21T03:56:33.498 INFO:teuthology.orchestra.run.smithi039.stdout: "log_level": { 2024-09-21T03:56:33.498 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_level", 2024-09-21T03:56:33.498 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-09-21T03:56:33.498 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.498 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:33.498 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-09-21T03:56:33.498 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.498 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.498 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-09-21T03:56:33.498 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-09-21T03:56:33.499 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-09-21T03:56:33.499 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-09-21T03:56:33.499 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-09-21T03:56:33.499 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-09-21T03:56:33.499 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-09-21T03:56:33.499 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-09-21T03:56:33.499 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:33.499 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.499 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.499 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.499 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.499 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster": { 2024-09-21T03:56:33.499 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster", 2024-09-21T03:56:33.499 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-09-21T03:56:33.499 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.500 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:33.500 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-09-21T03:56:33.500 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.500 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.500 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.500 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:33.500 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.500 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.500 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.500 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.500 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster_level": { 2024-09-21T03:56:33.500 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster_level", 2024-09-21T03:56:33.500 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-09-21T03:56:33.500 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.501 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:33.501 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "info", 2024-09-21T03:56:33.501 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.501 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.501 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-09-21T03:56:33.501 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-09-21T03:56:33.501 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-09-21T03:56:33.501 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-09-21T03:56:33.501 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-09-21T03:56:33.501 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-09-21T03:56:33.501 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-09-21T03:56:33.501 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-09-21T03:56:33.501 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:33.501 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.502 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.502 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.502 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.502 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_file": { 2024-09-21T03:56:33.502 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_file", 2024-09-21T03:56:33.502 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-09-21T03:56:33.502 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.502 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:33.502 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-09-21T03:56:33.502 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.502 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.502 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.502 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:33.502 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.502 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.503 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.503 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.503 INFO:teuthology.orchestra.run.smithi039.stdout: "sqlite3_killpoint": { 2024-09-21T03:56:33.503 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:56:33.503 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-09-21T03:56:33.503 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "dev", 2024-09-21T03:56:33.503 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:33.503 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "0", 2024-09-21T03:56:33.503 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.503 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.503 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.503 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:33.503 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.503 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.504 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.504 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-21T03:56:33.504 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-21T03:56:33.504 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.504 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-21T03:56:33.504 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "nfs", 2024-09-21T03:56:33.504 INFO:teuthology.orchestra.run.smithi039.stdout: "can_run": true, 2024-09-21T03:56:33.504 INFO:teuthology.orchestra.run.smithi039.stdout: "error_string": "", 2024-09-21T03:56:33.504 INFO:teuthology.orchestra.run.smithi039.stdout: "module_options": { 2024-09-21T03:56:33.504 INFO:teuthology.orchestra.run.smithi039.stdout: "log_level": { 2024-09-21T03:56:33.504 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_level", 2024-09-21T03:56:33.504 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-09-21T03:56:33.504 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.504 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:33.505 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-09-21T03:56:33.505 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.505 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.505 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-09-21T03:56:33.505 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-09-21T03:56:33.505 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-09-21T03:56:33.505 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-09-21T03:56:33.505 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-09-21T03:56:33.505 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-09-21T03:56:33.505 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-09-21T03:56:33.505 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-09-21T03:56:33.505 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:33.505 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.505 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.505 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.506 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.506 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster": { 2024-09-21T03:56:33.506 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster", 2024-09-21T03:56:33.506 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-09-21T03:56:33.506 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.506 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:33.506 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-09-21T03:56:33.506 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.506 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.506 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.506 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:33.506 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.506 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.506 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.507 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.507 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster_level": { 2024-09-21T03:56:33.507 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster_level", 2024-09-21T03:56:33.507 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-09-21T03:56:33.507 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.507 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:33.507 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "info", 2024-09-21T03:56:33.507 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.507 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.507 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-09-21T03:56:33.507 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-09-21T03:56:33.507 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-09-21T03:56:33.507 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-09-21T03:56:33.507 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-09-21T03:56:33.508 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-09-21T03:56:33.508 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-09-21T03:56:33.508 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-09-21T03:56:33.508 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:33.508 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.508 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.508 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.508 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.508 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_file": { 2024-09-21T03:56:33.508 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_file", 2024-09-21T03:56:33.508 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-09-21T03:56:33.508 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.508 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:33.508 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-09-21T03:56:33.509 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.509 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.509 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.509 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:33.509 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.509 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.509 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.509 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.509 INFO:teuthology.orchestra.run.smithi039.stdout: "sqlite3_killpoint": { 2024-09-21T03:56:33.509 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:56:33.509 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-09-21T03:56:33.509 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "dev", 2024-09-21T03:56:33.509 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:33.509 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "0", 2024-09-21T03:56:33.510 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.510 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.510 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.510 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:33.510 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.510 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.510 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.510 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-21T03:56:33.510 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-21T03:56:33.510 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.510 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-21T03:56:33.510 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "orchestrator", 2024-09-21T03:56:33.510 INFO:teuthology.orchestra.run.smithi039.stdout: "can_run": true, 2024-09-21T03:56:33.511 INFO:teuthology.orchestra.run.smithi039.stdout: "error_string": "", 2024-09-21T03:56:33.511 INFO:teuthology.orchestra.run.smithi039.stdout: "module_options": { 2024-09-21T03:56:33.511 INFO:teuthology.orchestra.run.smithi039.stdout: "fail_fs": { 2024-09-21T03:56:33.511 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "fail_fs", 2024-09-21T03:56:33.511 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-09-21T03:56:33.511 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.511 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-09-21T03:56:33.511 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-09-21T03:56:33.511 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.511 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.511 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.511 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-09-21T03:56:33.511 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.511 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.512 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.512 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.512 INFO:teuthology.orchestra.run.smithi039.stdout: "log_level": { 2024-09-21T03:56:33.512 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_level", 2024-09-21T03:56:33.512 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-09-21T03:56:33.512 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.512 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:33.512 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-09-21T03:56:33.512 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.512 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.512 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-09-21T03:56:33.512 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-09-21T03:56:33.512 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-09-21T03:56:33.512 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-09-21T03:56:33.512 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-09-21T03:56:33.512 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-09-21T03:56:33.513 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-09-21T03:56:33.513 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-09-21T03:56:33.513 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:33.513 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.513 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.513 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.513 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.513 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster": { 2024-09-21T03:56:33.513 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster", 2024-09-21T03:56:33.513 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-09-21T03:56:33.513 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.513 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:33.513 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-09-21T03:56:33.513 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.513 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.514 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.514 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:33.514 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.514 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.514 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.514 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.514 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster_level": { 2024-09-21T03:56:33.514 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster_level", 2024-09-21T03:56:33.514 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-09-21T03:56:33.514 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.514 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:33.514 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "info", 2024-09-21T03:56:33.514 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.515 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.515 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-09-21T03:56:33.515 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-09-21T03:56:33.515 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-09-21T03:56:33.515 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-09-21T03:56:33.515 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-09-21T03:56:33.515 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-09-21T03:56:33.515 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-09-21T03:56:33.515 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-09-21T03:56:33.515 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:33.515 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.515 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.515 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.515 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.516 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_file": { 2024-09-21T03:56:33.516 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_file", 2024-09-21T03:56:33.516 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-09-21T03:56:33.516 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.516 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:33.516 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-09-21T03:56:33.516 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.516 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.516 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.516 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:33.516 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.516 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.516 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.516 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.517 INFO:teuthology.orchestra.run.smithi039.stdout: "orchestrator": { 2024-09-21T03:56:33.517 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "orchestrator", 2024-09-21T03:56:33.517 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-09-21T03:56:33.517 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.517 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:33.517 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-09-21T03:56:33.517 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.517 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.517 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-09-21T03:56:33.517 INFO:teuthology.orchestra.run.smithi039.stdout: "cephadm", 2024-09-21T03:56:33.517 INFO:teuthology.orchestra.run.smithi039.stdout: "rook", 2024-09-21T03:56:33.517 INFO:teuthology.orchestra.run.smithi039.stdout: "test_orchestrator" 2024-09-21T03:56:33.517 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-09-21T03:56:33.517 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Orchestrator backend", 2024-09-21T03:56:33.518 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.518 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.518 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.518 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.518 INFO:teuthology.orchestra.run.smithi039.stdout: "sqlite3_killpoint": { 2024-09-21T03:56:33.518 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:56:33.518 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-09-21T03:56:33.518 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "dev", 2024-09-21T03:56:33.518 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:33.518 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "0", 2024-09-21T03:56:33.518 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.518 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.518 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.518 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:33.519 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.519 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.519 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.519 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-21T03:56:33.519 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-21T03:56:33.519 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.519 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-21T03:56:33.519 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "osd_perf_query", 2024-09-21T03:56:33.519 INFO:teuthology.orchestra.run.smithi039.stdout: "can_run": true, 2024-09-21T03:56:33.519 INFO:teuthology.orchestra.run.smithi039.stdout: "error_string": "", 2024-09-21T03:56:33.519 INFO:teuthology.orchestra.run.smithi039.stdout: "module_options": { 2024-09-21T03:56:33.519 INFO:teuthology.orchestra.run.smithi039.stdout: "log_level": { 2024-09-21T03:56:33.519 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_level", 2024-09-21T03:56:33.519 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-09-21T03:56:33.520 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.520 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:33.520 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-09-21T03:56:33.520 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.520 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.520 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-09-21T03:56:33.520 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-09-21T03:56:33.520 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-09-21T03:56:33.520 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-09-21T03:56:33.520 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-09-21T03:56:33.520 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-09-21T03:56:33.520 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-09-21T03:56:33.520 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-09-21T03:56:33.520 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:33.521 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.521 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.521 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.521 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.521 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster": { 2024-09-21T03:56:33.521 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster", 2024-09-21T03:56:33.521 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-09-21T03:56:33.521 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.521 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:33.521 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-09-21T03:56:33.521 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.521 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.521 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.522 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:33.522 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.522 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.522 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.522 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.522 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster_level": { 2024-09-21T03:56:33.522 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster_level", 2024-09-21T03:56:33.522 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-09-21T03:56:33.522 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.522 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:33.522 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "info", 2024-09-21T03:56:33.522 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.522 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.523 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-09-21T03:56:33.523 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-09-21T03:56:33.523 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-09-21T03:56:33.523 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-09-21T03:56:33.523 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-09-21T03:56:33.523 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-09-21T03:56:33.523 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-09-21T03:56:33.523 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-09-21T03:56:33.523 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:33.523 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.523 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.523 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.523 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.523 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_file": { 2024-09-21T03:56:33.524 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_file", 2024-09-21T03:56:33.524 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-09-21T03:56:33.524 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.524 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:33.524 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-09-21T03:56:33.524 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.524 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.524 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.524 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:33.524 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.524 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.524 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.524 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.524 INFO:teuthology.orchestra.run.smithi039.stdout: "sqlite3_killpoint": { 2024-09-21T03:56:33.525 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:56:33.525 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-09-21T03:56:33.525 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "dev", 2024-09-21T03:56:33.525 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:33.525 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "0", 2024-09-21T03:56:33.525 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.525 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.525 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.525 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:33.525 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.525 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.525 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.525 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-21T03:56:33.525 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-21T03:56:33.526 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.526 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-21T03:56:33.526 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "osd_support", 2024-09-21T03:56:33.526 INFO:teuthology.orchestra.run.smithi039.stdout: "can_run": true, 2024-09-21T03:56:33.526 INFO:teuthology.orchestra.run.smithi039.stdout: "error_string": "", 2024-09-21T03:56:33.526 INFO:teuthology.orchestra.run.smithi039.stdout: "module_options": { 2024-09-21T03:56:33.526 INFO:teuthology.orchestra.run.smithi039.stdout: "log_level": { 2024-09-21T03:56:33.526 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_level", 2024-09-21T03:56:33.526 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-09-21T03:56:33.526 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.526 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:33.526 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-09-21T03:56:33.526 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.526 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.527 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-09-21T03:56:33.527 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-09-21T03:56:33.527 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-09-21T03:56:33.527 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-09-21T03:56:33.527 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-09-21T03:56:33.527 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-09-21T03:56:33.527 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-09-21T03:56:33.527 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-09-21T03:56:33.527 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:33.527 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.527 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.527 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.527 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.528 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster": { 2024-09-21T03:56:33.528 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster", 2024-09-21T03:56:33.528 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-09-21T03:56:33.528 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.528 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:33.528 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-09-21T03:56:33.528 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.528 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.528 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.528 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:33.528 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.528 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.528 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.529 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.529 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster_level": { 2024-09-21T03:56:33.529 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster_level", 2024-09-21T03:56:33.529 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-09-21T03:56:33.529 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.529 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:33.529 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "info", 2024-09-21T03:56:33.529 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.529 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.529 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-09-21T03:56:33.529 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-09-21T03:56:33.529 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-09-21T03:56:33.529 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-09-21T03:56:33.529 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-09-21T03:56:33.529 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-09-21T03:56:33.530 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-09-21T03:56:33.530 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-09-21T03:56:33.530 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:33.530 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.530 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.530 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.530 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.530 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_file": { 2024-09-21T03:56:33.530 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_file", 2024-09-21T03:56:33.530 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-09-21T03:56:33.530 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.530 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:33.530 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-09-21T03:56:33.530 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.530 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.531 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.531 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:33.531 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.531 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.531 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.531 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.531 INFO:teuthology.orchestra.run.smithi039.stdout: "sqlite3_killpoint": { 2024-09-21T03:56:33.531 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:56:33.531 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-09-21T03:56:33.531 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "dev", 2024-09-21T03:56:33.531 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:33.531 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "0", 2024-09-21T03:56:33.531 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.531 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.532 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.532 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:33.532 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.532 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.532 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.532 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-21T03:56:33.532 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-21T03:56:33.532 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.532 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-21T03:56:33.532 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "pg_autoscaler", 2024-09-21T03:56:33.532 INFO:teuthology.orchestra.run.smithi039.stdout: "can_run": true, 2024-09-21T03:56:33.532 INFO:teuthology.orchestra.run.smithi039.stdout: "error_string": "", 2024-09-21T03:56:33.532 INFO:teuthology.orchestra.run.smithi039.stdout: "module_options": { 2024-09-21T03:56:33.532 INFO:teuthology.orchestra.run.smithi039.stdout: "log_level": { 2024-09-21T03:56:33.533 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_level", 2024-09-21T03:56:33.533 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-09-21T03:56:33.533 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.533 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:33.533 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-09-21T03:56:33.533 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.533 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.533 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-09-21T03:56:33.533 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-09-21T03:56:33.533 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-09-21T03:56:33.533 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-09-21T03:56:33.533 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-09-21T03:56:33.533 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-09-21T03:56:33.533 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-09-21T03:56:33.534 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-09-21T03:56:33.534 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:33.534 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.534 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.534 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.534 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.534 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster": { 2024-09-21T03:56:33.534 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster", 2024-09-21T03:56:33.534 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-09-21T03:56:33.534 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.534 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:33.534 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-09-21T03:56:33.534 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.535 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.535 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.535 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:33.535 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.535 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.535 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.535 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.535 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster_level": { 2024-09-21T03:56:33.535 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster_level", 2024-09-21T03:56:33.535 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-09-21T03:56:33.535 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.535 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:33.535 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "info", 2024-09-21T03:56:33.536 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.536 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.536 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-09-21T03:56:33.536 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-09-21T03:56:33.536 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-09-21T03:56:33.536 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-09-21T03:56:33.536 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-09-21T03:56:33.536 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-09-21T03:56:33.536 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-09-21T03:56:33.536 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-09-21T03:56:33.536 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:33.536 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.536 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.536 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.537 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.537 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_file": { 2024-09-21T03:56:33.537 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_file", 2024-09-21T03:56:33.537 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-09-21T03:56:33.537 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.537 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:33.537 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-09-21T03:56:33.537 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.537 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.537 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.537 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:33.537 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.537 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.538 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.538 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.538 INFO:teuthology.orchestra.run.smithi039.stdout: "sleep_interval": { 2024-09-21T03:56:33.538 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "sleep_interval", 2024-09-21T03:56:33.538 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "secs", 2024-09-21T03:56:33.538 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.538 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-09-21T03:56:33.538 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "60", 2024-09-21T03:56:33.538 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.538 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.538 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.538 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:33.538 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.538 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.539 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.539 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.539 INFO:teuthology.orchestra.run.smithi039.stdout: "sqlite3_killpoint": { 2024-09-21T03:56:33.539 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:56:33.539 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-09-21T03:56:33.539 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "dev", 2024-09-21T03:56:33.539 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:33.539 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "0", 2024-09-21T03:56:33.539 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.539 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.539 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.539 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:33.539 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.539 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.540 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.540 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.540 INFO:teuthology.orchestra.run.smithi039.stdout: "threshold": { 2024-09-21T03:56:33.540 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "threshold", 2024-09-21T03:56:33.540 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "float", 2024-09-21T03:56:33.540 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.540 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-09-21T03:56:33.540 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "3.0", 2024-09-21T03:56:33.540 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "1.0", 2024-09-21T03:56:33.540 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.540 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.540 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "scaling threshold", 2024-09-21T03:56:33.540 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-09-21T03:56:33.540 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.540 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.541 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-21T03:56:33.541 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-21T03:56:33.541 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.541 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-21T03:56:33.541 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "progress", 2024-09-21T03:56:33.541 INFO:teuthology.orchestra.run.smithi039.stdout: "can_run": true, 2024-09-21T03:56:33.541 INFO:teuthology.orchestra.run.smithi039.stdout: "error_string": "", 2024-09-21T03:56:33.541 INFO:teuthology.orchestra.run.smithi039.stdout: "module_options": { 2024-09-21T03:56:33.541 INFO:teuthology.orchestra.run.smithi039.stdout: "allow_pg_recovery_event": { 2024-09-21T03:56:33.541 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "allow_pg_recovery_event", 2024-09-21T03:56:33.541 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-09-21T03:56:33.541 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.541 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:33.541 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-09-21T03:56:33.542 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.542 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.542 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.542 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "allow the module to show pg recovery progress", 2024-09-21T03:56:33.542 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.542 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.542 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.542 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.542 INFO:teuthology.orchestra.run.smithi039.stdout: "enabled": { 2024-09-21T03:56:33.542 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "enabled", 2024-09-21T03:56:33.542 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-09-21T03:56:33.542 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.542 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-09-21T03:56:33.542 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2024-09-21T03:56:33.542 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.543 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.543 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.543 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:33.543 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.543 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.543 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.543 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.543 INFO:teuthology.orchestra.run.smithi039.stdout: "log_level": { 2024-09-21T03:56:33.543 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_level", 2024-09-21T03:56:33.543 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-09-21T03:56:33.543 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.543 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:33.543 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-09-21T03:56:33.543 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.543 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.544 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-09-21T03:56:33.544 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-09-21T03:56:33.544 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-09-21T03:56:33.544 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-09-21T03:56:33.544 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-09-21T03:56:33.544 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-09-21T03:56:33.544 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-09-21T03:56:33.544 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-09-21T03:56:33.544 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:33.544 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.544 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.544 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.544 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.544 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster": { 2024-09-21T03:56:33.544 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster", 2024-09-21T03:56:33.544 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-09-21T03:56:33.545 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.545 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:33.545 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-09-21T03:56:33.545 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.545 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.545 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.545 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:33.545 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.545 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.545 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.545 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.545 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster_level": { 2024-09-21T03:56:33.545 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster_level", 2024-09-21T03:56:33.545 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-09-21T03:56:33.545 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.545 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:33.545 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "info", 2024-09-21T03:56:33.546 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.546 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.546 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-09-21T03:56:33.546 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-09-21T03:56:33.546 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-09-21T03:56:33.546 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-09-21T03:56:33.546 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-09-21T03:56:33.546 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-09-21T03:56:33.546 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-09-21T03:56:33.546 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-09-21T03:56:33.546 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:33.546 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.546 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.546 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.546 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.547 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_file": { 2024-09-21T03:56:33.547 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_file", 2024-09-21T03:56:33.547 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-09-21T03:56:33.547 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.547 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:33.547 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-09-21T03:56:33.547 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.547 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.547 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.547 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:33.547 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.547 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.547 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.547 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.547 INFO:teuthology.orchestra.run.smithi039.stdout: "max_completed_events": { 2024-09-21T03:56:33.548 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "max_completed_events", 2024-09-21T03:56:33.548 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-09-21T03:56:33.548 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.548 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:33.548 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "50", 2024-09-21T03:56:33.548 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.548 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.548 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.548 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "number of past completed events to remember", 2024-09-21T03:56:33.548 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.548 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.548 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.548 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.548 INFO:teuthology.orchestra.run.smithi039.stdout: "sleep_interval": { 2024-09-21T03:56:33.548 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "sleep_interval", 2024-09-21T03:56:33.549 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "secs", 2024-09-21T03:56:33.549 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.549 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:33.549 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "5", 2024-09-21T03:56:33.549 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.549 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.549 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.549 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "how long the module is going to sleep", 2024-09-21T03:56:33.549 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.549 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.549 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.549 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.549 INFO:teuthology.orchestra.run.smithi039.stdout: "sqlite3_killpoint": { 2024-09-21T03:56:33.549 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:56:33.550 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-09-21T03:56:33.550 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "dev", 2024-09-21T03:56:33.550 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:33.550 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "0", 2024-09-21T03:56:33.550 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.550 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.550 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.550 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:33.550 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.550 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.550 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.550 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-21T03:56:33.550 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-21T03:56:33.550 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.551 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-21T03:56:33.551 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "prometheus", 2024-09-21T03:56:33.551 INFO:teuthology.orchestra.run.smithi039.stdout: "can_run": true, 2024-09-21T03:56:33.551 INFO:teuthology.orchestra.run.smithi039.stdout: "error_string": "", 2024-09-21T03:56:33.551 INFO:teuthology.orchestra.run.smithi039.stdout: "module_options": { 2024-09-21T03:56:33.551 INFO:teuthology.orchestra.run.smithi039.stdout: "cache": { 2024-09-21T03:56:33.551 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "cache", 2024-09-21T03:56:33.551 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-09-21T03:56:33.551 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.551 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-09-21T03:56:33.551 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2024-09-21T03:56:33.551 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.551 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.552 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.552 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:33.552 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.552 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.552 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.552 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.552 INFO:teuthology.orchestra.run.smithi039.stdout: "exclude_perf_counters": { 2024-09-21T03:56:33.552 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "exclude_perf_counters", 2024-09-21T03:56:33.552 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-09-21T03:56:33.552 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.552 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:33.552 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2024-09-21T03:56:33.552 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.552 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.553 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.553 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-09-21T03:56:33.553 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-09-21T03:56:33.553 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.553 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.553 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.553 INFO:teuthology.orchestra.run.smithi039.stdout: "log_level": { 2024-09-21T03:56:33.553 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_level", 2024-09-21T03:56:33.553 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-09-21T03:56:33.553 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.553 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:33.553 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-09-21T03:56:33.553 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.553 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.554 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-09-21T03:56:33.554 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-09-21T03:56:33.554 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-09-21T03:56:33.554 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-09-21T03:56:33.554 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-09-21T03:56:33.554 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-09-21T03:56:33.554 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-09-21T03:56:33.554 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-09-21T03:56:33.554 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:33.554 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.554 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.554 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.554 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.554 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster": { 2024-09-21T03:56:33.555 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster", 2024-09-21T03:56:33.555 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-09-21T03:56:33.555 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.555 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:33.555 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-09-21T03:56:33.555 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.555 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.555 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.555 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:33.555 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.555 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.555 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.555 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.556 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster_level": { 2024-09-21T03:56:33.556 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster_level", 2024-09-21T03:56:33.556 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-09-21T03:56:33.556 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.556 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:33.556 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "info", 2024-09-21T03:56:33.556 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.556 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.556 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-09-21T03:56:33.556 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-09-21T03:56:33.556 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-09-21T03:56:33.556 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-09-21T03:56:33.556 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-09-21T03:56:33.556 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-09-21T03:56:33.557 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-09-21T03:56:33.557 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-09-21T03:56:33.557 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:33.557 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.557 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.557 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.557 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.557 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_file": { 2024-09-21T03:56:33.557 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_file", 2024-09-21T03:56:33.557 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-09-21T03:56:33.557 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.557 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:33.557 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-09-21T03:56:33.557 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.557 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.558 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.558 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:33.558 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.558 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.558 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.558 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.558 INFO:teuthology.orchestra.run.smithi039.stdout: "rbd_stats_pools": { 2024-09-21T03:56:33.558 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "rbd_stats_pools", 2024-09-21T03:56:33.558 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-09-21T03:56:33.558 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.558 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-09-21T03:56:33.558 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-09-21T03:56:33.558 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.558 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.558 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.559 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:33.559 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.559 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.559 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.559 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.559 INFO:teuthology.orchestra.run.smithi039.stdout: "rbd_stats_pools_refresh_interval": { 2024-09-21T03:56:33.559 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-09-21T03:56:33.559 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-09-21T03:56:33.559 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.559 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-09-21T03:56:33.559 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "300", 2024-09-21T03:56:33.559 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.559 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.559 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.560 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:33.560 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.560 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.560 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.560 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.560 INFO:teuthology.orchestra.run.smithi039.stdout: "scrape_interval": { 2024-09-21T03:56:33.560 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "scrape_interval", 2024-09-21T03:56:33.560 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "float", 2024-09-21T03:56:33.560 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.560 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-09-21T03:56:33.560 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "15.0", 2024-09-21T03:56:33.560 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.560 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.560 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.560 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:33.561 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.561 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.561 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.561 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.561 INFO:teuthology.orchestra.run.smithi039.stdout: "server_addr": { 2024-09-21T03:56:33.561 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "server_addr", 2024-09-21T03:56:33.561 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-09-21T03:56:33.561 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.561 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-09-21T03:56:33.561 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "::", 2024-09-21T03:56:33.561 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.561 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.561 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.562 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-09-21T03:56:33.562 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.562 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.562 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.562 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.562 INFO:teuthology.orchestra.run.smithi039.stdout: "server_port": { 2024-09-21T03:56:33.562 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "server_port", 2024-09-21T03:56:33.562 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-09-21T03:56:33.562 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.562 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:33.562 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "9283", 2024-09-21T03:56:33.562 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.562 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.562 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.563 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-09-21T03:56:33.563 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.563 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.563 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.563 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.563 INFO:teuthology.orchestra.run.smithi039.stdout: "sqlite3_killpoint": { 2024-09-21T03:56:33.563 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:56:33.563 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-09-21T03:56:33.563 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "dev", 2024-09-21T03:56:33.563 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:33.563 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "0", 2024-09-21T03:56:33.563 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.563 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.563 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.564 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:33.564 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.564 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.564 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.564 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.564 INFO:teuthology.orchestra.run.smithi039.stdout: "stale_cache_strategy": { 2024-09-21T03:56:33.564 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "stale_cache_strategy", 2024-09-21T03:56:33.564 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-09-21T03:56:33.564 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.564 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-09-21T03:56:33.564 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "log", 2024-09-21T03:56:33.564 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.564 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.564 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.564 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:33.565 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.565 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.565 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.565 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.565 INFO:teuthology.orchestra.run.smithi039.stdout: "standby_behaviour": { 2024-09-21T03:56:33.565 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "standby_behaviour", 2024-09-21T03:56:33.565 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-09-21T03:56:33.565 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.565 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:33.565 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "default", 2024-09-21T03:56:33.565 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.565 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.565 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-09-21T03:56:33.565 INFO:teuthology.orchestra.run.smithi039.stdout: "default", 2024-09-21T03:56:33.566 INFO:teuthology.orchestra.run.smithi039.stdout: "error" 2024-09-21T03:56:33.566 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-09-21T03:56:33.566 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:33.566 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.566 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.566 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.566 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.566 INFO:teuthology.orchestra.run.smithi039.stdout: "standby_error_status_code": { 2024-09-21T03:56:33.566 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "standby_error_status_code", 2024-09-21T03:56:33.566 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-09-21T03:56:33.566 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.566 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:33.566 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "500", 2024-09-21T03:56:33.566 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "400", 2024-09-21T03:56:33.567 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "599", 2024-09-21T03:56:33.567 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.567 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:33.567 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.567 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.567 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.567 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-21T03:56:33.567 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-21T03:56:33.567 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.567 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-21T03:56:33.567 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "rbd_support", 2024-09-21T03:56:33.567 INFO:teuthology.orchestra.run.smithi039.stdout: "can_run": true, 2024-09-21T03:56:33.567 INFO:teuthology.orchestra.run.smithi039.stdout: "error_string": "", 2024-09-21T03:56:33.567 INFO:teuthology.orchestra.run.smithi039.stdout: "module_options": { 2024-09-21T03:56:33.568 INFO:teuthology.orchestra.run.smithi039.stdout: "log_level": { 2024-09-21T03:56:33.568 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_level", 2024-09-21T03:56:33.568 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-09-21T03:56:33.568 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.568 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:33.568 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-09-21T03:56:33.568 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.568 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.568 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-09-21T03:56:33.568 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-09-21T03:56:33.568 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-09-21T03:56:33.568 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-09-21T03:56:33.568 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-09-21T03:56:33.568 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-09-21T03:56:33.568 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-09-21T03:56:33.569 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-09-21T03:56:33.569 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:33.569 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.569 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.569 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.569 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.569 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster": { 2024-09-21T03:56:33.569 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster", 2024-09-21T03:56:33.569 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-09-21T03:56:33.569 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.569 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:33.569 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-09-21T03:56:33.569 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.569 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.570 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.570 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:33.570 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.570 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.570 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.570 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.570 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster_level": { 2024-09-21T03:56:33.570 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster_level", 2024-09-21T03:56:33.570 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-09-21T03:56:33.570 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.570 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:33.570 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "info", 2024-09-21T03:56:33.570 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.570 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.570 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-09-21T03:56:33.571 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-09-21T03:56:33.571 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-09-21T03:56:33.571 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-09-21T03:56:33.571 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-09-21T03:56:33.571 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-09-21T03:56:33.571 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-09-21T03:56:33.571 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-09-21T03:56:33.571 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:33.571 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.571 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.571 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.571 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.571 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_file": { 2024-09-21T03:56:33.571 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_file", 2024-09-21T03:56:33.571 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-09-21T03:56:33.572 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.572 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:33.572 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-09-21T03:56:33.572 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.572 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.572 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.572 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:33.572 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.572 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.572 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.572 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.572 INFO:teuthology.orchestra.run.smithi039.stdout: "max_concurrent_snap_create": { 2024-09-21T03:56:33.572 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "max_concurrent_snap_create", 2024-09-21T03:56:33.572 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-09-21T03:56:33.572 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.573 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-09-21T03:56:33.573 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "10", 2024-09-21T03:56:33.573 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.573 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.573 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.573 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:33.573 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.573 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.573 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.573 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.573 INFO:teuthology.orchestra.run.smithi039.stdout: "mirror_snapshot_schedule": { 2024-09-21T03:56:33.573 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "mirror_snapshot_schedule", 2024-09-21T03:56:33.573 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-09-21T03:56:33.573 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.574 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-09-21T03:56:33.574 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-09-21T03:56:33.574 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.574 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.574 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.574 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:33.574 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.574 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.574 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.574 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.574 INFO:teuthology.orchestra.run.smithi039.stdout: "sqlite3_killpoint": { 2024-09-21T03:56:33.574 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:56:33.574 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-09-21T03:56:33.574 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "dev", 2024-09-21T03:56:33.575 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:33.575 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "0", 2024-09-21T03:56:33.575 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.575 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.575 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.575 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:33.575 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.575 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.575 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.575 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.575 INFO:teuthology.orchestra.run.smithi039.stdout: "trash_purge_schedule": { 2024-09-21T03:56:33.575 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "trash_purge_schedule", 2024-09-21T03:56:33.575 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-09-21T03:56:33.575 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.576 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-09-21T03:56:33.576 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-09-21T03:56:33.576 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.576 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.576 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.576 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:33.576 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.576 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.576 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.576 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-21T03:56:33.576 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-21T03:56:33.576 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.576 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-21T03:56:33.576 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "restful", 2024-09-21T03:56:33.577 INFO:teuthology.orchestra.run.smithi039.stdout: "can_run": true, 2024-09-21T03:56:33.577 INFO:teuthology.orchestra.run.smithi039.stdout: "error_string": "", 2024-09-21T03:56:33.577 INFO:teuthology.orchestra.run.smithi039.stdout: "module_options": { 2024-09-21T03:56:33.577 INFO:teuthology.orchestra.run.smithi039.stdout: "enable_auth": { 2024-09-21T03:56:33.577 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "enable_auth", 2024-09-21T03:56:33.577 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-09-21T03:56:33.577 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.577 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-09-21T03:56:33.577 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2024-09-21T03:56:33.577 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.577 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.577 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.577 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:33.577 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.577 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.578 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.578 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.578 INFO:teuthology.orchestra.run.smithi039.stdout: "key_file": { 2024-09-21T03:56:33.578 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "key_file", 2024-09-21T03:56:33.578 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-09-21T03:56:33.578 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.578 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-09-21T03:56:33.578 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-09-21T03:56:33.578 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.578 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.578 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.578 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:33.578 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.578 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.579 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.579 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.579 INFO:teuthology.orchestra.run.smithi039.stdout: "log_level": { 2024-09-21T03:56:33.579 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_level", 2024-09-21T03:56:33.579 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-09-21T03:56:33.579 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.579 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:33.579 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-09-21T03:56:33.579 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.579 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.579 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-09-21T03:56:33.579 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-09-21T03:56:33.579 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-09-21T03:56:33.579 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-09-21T03:56:33.580 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-09-21T03:56:33.580 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-09-21T03:56:33.580 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-09-21T03:56:33.580 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-09-21T03:56:33.580 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:33.580 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.580 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.580 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.580 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.580 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster": { 2024-09-21T03:56:33.580 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster", 2024-09-21T03:56:33.580 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-09-21T03:56:33.580 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.580 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:33.580 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-09-21T03:56:33.581 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.581 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.581 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.581 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:33.581 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.581 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.581 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.581 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.581 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster_level": { 2024-09-21T03:56:33.581 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster_level", 2024-09-21T03:56:33.581 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-09-21T03:56:33.581 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.581 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:33.581 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "info", 2024-09-21T03:56:33.581 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.582 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.582 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-09-21T03:56:33.582 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-09-21T03:56:33.582 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-09-21T03:56:33.582 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-09-21T03:56:33.582 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-09-21T03:56:33.582 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-09-21T03:56:33.582 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-09-21T03:56:33.582 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-09-21T03:56:33.582 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:33.582 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.582 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.582 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.582 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.583 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_file": { 2024-09-21T03:56:33.583 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_file", 2024-09-21T03:56:33.583 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-09-21T03:56:33.583 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.583 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:33.583 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-09-21T03:56:33.583 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.583 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.583 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.583 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:33.583 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.583 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.583 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.583 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.584 INFO:teuthology.orchestra.run.smithi039.stdout: "max_requests": { 2024-09-21T03:56:33.584 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "max_requests", 2024-09-21T03:56:33.584 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-09-21T03:56:33.584 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.584 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-09-21T03:56:33.584 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "500", 2024-09-21T03:56:33.584 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.584 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.584 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.584 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Maximum number of requests to keep in memory. When new request comes in, the oldest request will be removed if the number of requests exceeds the max request number.if un-finished request is removed, error message will be logged in the ceph-mgr log.", 2024-09-21T03:56:33.584 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.584 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.584 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.584 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.584 INFO:teuthology.orchestra.run.smithi039.stdout: "server_addr": { 2024-09-21T03:56:33.585 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "server_addr", 2024-09-21T03:56:33.585 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-09-21T03:56:33.585 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.585 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-09-21T03:56:33.585 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-09-21T03:56:33.585 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.585 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.585 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.585 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:33.585 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.585 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.585 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.585 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.585 INFO:teuthology.orchestra.run.smithi039.stdout: "server_port": { 2024-09-21T03:56:33.585 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "server_port", 2024-09-21T03:56:33.585 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-09-21T03:56:33.586 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.586 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-09-21T03:56:33.586 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-09-21T03:56:33.586 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.586 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.586 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.586 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:33.586 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.586 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.586 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.586 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.586 INFO:teuthology.orchestra.run.smithi039.stdout: "sqlite3_killpoint": { 2024-09-21T03:56:33.586 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:56:33.586 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-09-21T03:56:33.586 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "dev", 2024-09-21T03:56:33.587 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:33.587 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "0", 2024-09-21T03:56:33.587 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.587 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.587 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.587 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:33.587 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.587 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.587 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.587 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-21T03:56:33.587 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-21T03:56:33.587 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.587 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-21T03:56:33.587 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "rgw", 2024-09-21T03:56:33.588 INFO:teuthology.orchestra.run.smithi039.stdout: "can_run": true, 2024-09-21T03:56:33.588 INFO:teuthology.orchestra.run.smithi039.stdout: "error_string": "", 2024-09-21T03:56:33.588 INFO:teuthology.orchestra.run.smithi039.stdout: "module_options": { 2024-09-21T03:56:33.588 INFO:teuthology.orchestra.run.smithi039.stdout: "log_level": { 2024-09-21T03:56:33.588 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_level", 2024-09-21T03:56:33.588 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-09-21T03:56:33.588 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.588 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:33.588 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-09-21T03:56:33.588 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.588 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.588 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-09-21T03:56:33.588 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-09-21T03:56:33.588 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-09-21T03:56:33.589 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-09-21T03:56:33.589 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-09-21T03:56:33.589 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-09-21T03:56:33.589 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-09-21T03:56:33.589 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-09-21T03:56:33.589 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:33.589 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.589 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.589 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.589 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.589 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster": { 2024-09-21T03:56:33.589 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster", 2024-09-21T03:56:33.589 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-09-21T03:56:33.589 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.590 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:33.590 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-09-21T03:56:33.590 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.590 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.590 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.590 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:33.590 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.590 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.590 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.590 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.590 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster_level": { 2024-09-21T03:56:33.590 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster_level", 2024-09-21T03:56:33.590 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-09-21T03:56:33.591 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.591 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:33.591 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "info", 2024-09-21T03:56:33.591 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.591 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.591 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-09-21T03:56:33.591 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-09-21T03:56:33.591 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-09-21T03:56:33.591 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-09-21T03:56:33.591 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-09-21T03:56:33.591 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-09-21T03:56:33.591 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-09-21T03:56:33.591 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-09-21T03:56:33.591 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:33.592 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.592 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.592 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.592 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.592 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_file": { 2024-09-21T03:56:33.592 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_file", 2024-09-21T03:56:33.592 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-09-21T03:56:33.592 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.592 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:33.592 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-09-21T03:56:33.592 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.592 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.592 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.592 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:33.593 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.593 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.593 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.593 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.593 INFO:teuthology.orchestra.run.smithi039.stdout: "secondary_zone_period_retry_limit": { 2024-09-21T03:56:33.593 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "secondary_zone_period_retry_limit", 2024-09-21T03:56:33.593 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-09-21T03:56:33.593 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.593 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-09-21T03:56:33.593 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "5", 2024-09-21T03:56:33.593 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.593 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.593 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.593 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "RGW module period update retry limit for secondary site", 2024-09-21T03:56:33.594 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.594 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.594 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.594 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.594 INFO:teuthology.orchestra.run.smithi039.stdout: "sqlite3_killpoint": { 2024-09-21T03:56:33.594 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:56:33.594 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-09-21T03:56:33.594 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "dev", 2024-09-21T03:56:33.594 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:33.594 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "0", 2024-09-21T03:56:33.594 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.594 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.594 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.594 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:33.595 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.595 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.595 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.595 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-21T03:56:33.595 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-21T03:56:33.595 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.595 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-21T03:56:33.595 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "rook", 2024-09-21T03:56:33.595 INFO:teuthology.orchestra.run.smithi039.stdout: "can_run": true, 2024-09-21T03:56:33.595 INFO:teuthology.orchestra.run.smithi039.stdout: "error_string": "", 2024-09-21T03:56:33.595 INFO:teuthology.orchestra.run.smithi039.stdout: "module_options": { 2024-09-21T03:56:33.595 INFO:teuthology.orchestra.run.smithi039.stdout: "log_level": { 2024-09-21T03:56:33.595 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_level", 2024-09-21T03:56:33.596 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-09-21T03:56:33.596 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.596 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:33.596 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-09-21T03:56:33.596 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.596 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.596 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-09-21T03:56:33.596 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-09-21T03:56:33.596 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-09-21T03:56:33.596 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-09-21T03:56:33.596 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-09-21T03:56:33.596 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-09-21T03:56:33.596 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-09-21T03:56:33.596 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-09-21T03:56:33.596 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:33.597 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.597 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.597 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.597 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.597 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster": { 2024-09-21T03:56:33.597 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster", 2024-09-21T03:56:33.597 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-09-21T03:56:33.597 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.597 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:33.597 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-09-21T03:56:33.597 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.597 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.597 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.597 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:33.597 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.598 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.598 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.598 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.598 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster_level": { 2024-09-21T03:56:33.598 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster_level", 2024-09-21T03:56:33.598 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-09-21T03:56:33.598 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.598 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:33.598 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "info", 2024-09-21T03:56:33.598 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.598 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.598 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-09-21T03:56:33.598 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-09-21T03:56:33.599 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-09-21T03:56:33.599 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-09-21T03:56:33.599 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-09-21T03:56:33.599 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-09-21T03:56:33.599 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-09-21T03:56:33.599 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-09-21T03:56:33.599 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:33.599 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.599 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.599 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.599 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.599 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_file": { 2024-09-21T03:56:33.599 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_file", 2024-09-21T03:56:33.599 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-09-21T03:56:33.599 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.600 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:33.600 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-09-21T03:56:33.600 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.600 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.600 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.600 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:33.600 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.600 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.600 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.600 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.600 INFO:teuthology.orchestra.run.smithi039.stdout: "sqlite3_killpoint": { 2024-09-21T03:56:33.600 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:56:33.600 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-09-21T03:56:33.600 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "dev", 2024-09-21T03:56:33.601 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:33.601 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "0", 2024-09-21T03:56:33.601 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.601 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.601 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.601 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:33.601 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.601 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.601 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.601 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.601 INFO:teuthology.orchestra.run.smithi039.stdout: "storage_class": { 2024-09-21T03:56:33.601 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "storage_class", 2024-09-21T03:56:33.601 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-09-21T03:56:33.602 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.602 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-09-21T03:56:33.602 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "local", 2024-09-21T03:56:33.602 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.602 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.602 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.602 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-09-21T03:56:33.602 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.602 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.602 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.602 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-21T03:56:33.602 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-21T03:56:33.602 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.602 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-21T03:56:33.603 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "selftest", 2024-09-21T03:56:33.603 INFO:teuthology.orchestra.run.smithi039.stdout: "can_run": true, 2024-09-21T03:56:33.603 INFO:teuthology.orchestra.run.smithi039.stdout: "error_string": "", 2024-09-21T03:56:33.603 INFO:teuthology.orchestra.run.smithi039.stdout: "module_options": { 2024-09-21T03:56:33.603 INFO:teuthology.orchestra.run.smithi039.stdout: "log_level": { 2024-09-21T03:56:33.603 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_level", 2024-09-21T03:56:33.603 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-09-21T03:56:33.603 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.603 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:33.603 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-09-21T03:56:33.603 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.603 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.603 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-09-21T03:56:33.604 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-09-21T03:56:33.604 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-09-21T03:56:33.604 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-09-21T03:56:33.604 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-09-21T03:56:33.604 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-09-21T03:56:33.604 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-09-21T03:56:33.604 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-09-21T03:56:33.604 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:33.604 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.604 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.604 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.604 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.604 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster": { 2024-09-21T03:56:33.604 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster", 2024-09-21T03:56:33.605 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-09-21T03:56:33.605 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.605 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:33.605 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-09-21T03:56:33.605 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.605 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.605 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.605 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:33.605 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.605 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.605 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.605 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.605 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster_level": { 2024-09-21T03:56:33.605 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster_level", 2024-09-21T03:56:33.606 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-09-21T03:56:33.606 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.606 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:33.606 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "info", 2024-09-21T03:56:33.606 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.606 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.606 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-09-21T03:56:33.606 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-09-21T03:56:33.606 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-09-21T03:56:33.606 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-09-21T03:56:33.606 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-09-21T03:56:33.606 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-09-21T03:56:33.606 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-09-21T03:56:33.606 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-09-21T03:56:33.607 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:33.607 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.607 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.607 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.607 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.607 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_file": { 2024-09-21T03:56:33.607 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_file", 2024-09-21T03:56:33.607 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-09-21T03:56:33.607 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.607 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:33.607 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-09-21T03:56:33.607 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.607 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.607 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.608 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:33.608 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.608 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.608 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.608 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.608 INFO:teuthology.orchestra.run.smithi039.stdout: "roption1": { 2024-09-21T03:56:33.608 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "roption1", 2024-09-21T03:56:33.608 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-09-21T03:56:33.608 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.608 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-09-21T03:56:33.608 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-09-21T03:56:33.608 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.608 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.608 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.608 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:33.609 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.609 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.609 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.609 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.609 INFO:teuthology.orchestra.run.smithi039.stdout: "roption2": { 2024-09-21T03:56:33.609 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "roption2", 2024-09-21T03:56:33.609 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-09-21T03:56:33.609 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.609 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-09-21T03:56:33.609 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "xyz", 2024-09-21T03:56:33.609 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.609 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.609 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.609 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:33.609 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.610 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.610 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.610 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.610 INFO:teuthology.orchestra.run.smithi039.stdout: "rwoption1": { 2024-09-21T03:56:33.610 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "rwoption1", 2024-09-21T03:56:33.610 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-09-21T03:56:33.610 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.610 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-09-21T03:56:33.610 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-09-21T03:56:33.610 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.610 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.610 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.610 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:33.610 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.611 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.611 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.611 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.611 INFO:teuthology.orchestra.run.smithi039.stdout: "rwoption2": { 2024-09-21T03:56:33.611 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "rwoption2", 2024-09-21T03:56:33.611 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-09-21T03:56:33.611 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.611 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-09-21T03:56:33.611 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-09-21T03:56:33.611 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.611 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.611 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.611 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:33.611 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.611 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.611 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.612 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.612 INFO:teuthology.orchestra.run.smithi039.stdout: "rwoption3": { 2024-09-21T03:56:33.612 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "rwoption3", 2024-09-21T03:56:33.612 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "float", 2024-09-21T03:56:33.612 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.612 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-09-21T03:56:33.612 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-09-21T03:56:33.612 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.612 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.612 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.612 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:33.612 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.612 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.612 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.612 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.612 INFO:teuthology.orchestra.run.smithi039.stdout: "rwoption4": { 2024-09-21T03:56:33.613 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "rwoption4", 2024-09-21T03:56:33.613 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-09-21T03:56:33.613 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.613 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-09-21T03:56:33.613 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-09-21T03:56:33.613 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.613 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.613 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.613 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:33.613 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.613 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.613 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.613 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.613 INFO:teuthology.orchestra.run.smithi039.stdout: "rwoption5": { 2024-09-21T03:56:33.613 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "rwoption5", 2024-09-21T03:56:33.613 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-09-21T03:56:33.614 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.614 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-09-21T03:56:33.614 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-09-21T03:56:33.614 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.614 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.614 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.614 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:33.614 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.614 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.614 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.614 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.614 INFO:teuthology.orchestra.run.smithi039.stdout: "rwoption6": { 2024-09-21T03:56:33.614 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "rwoption6", 2024-09-21T03:56:33.614 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-09-21T03:56:33.615 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.615 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-09-21T03:56:33.615 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2024-09-21T03:56:33.615 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.615 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.615 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.615 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:33.615 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.615 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.615 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.615 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.615 INFO:teuthology.orchestra.run.smithi039.stdout: "rwoption7": { 2024-09-21T03:56:33.615 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "rwoption7", 2024-09-21T03:56:33.615 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-09-21T03:56:33.616 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.616 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-09-21T03:56:33.616 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-09-21T03:56:33.616 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "1", 2024-09-21T03:56:33.616 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "42", 2024-09-21T03:56:33.616 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.616 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:33.616 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.616 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.616 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.616 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.616 INFO:teuthology.orchestra.run.smithi039.stdout: "sqlite3_killpoint": { 2024-09-21T03:56:33.616 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:56:33.616 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-09-21T03:56:33.617 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "dev", 2024-09-21T03:56:33.617 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:33.617 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "0", 2024-09-21T03:56:33.617 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.617 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.617 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.617 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:33.617 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.617 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.617 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.617 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.617 INFO:teuthology.orchestra.run.smithi039.stdout: "testkey": { 2024-09-21T03:56:33.617 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "testkey", 2024-09-21T03:56:33.617 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-09-21T03:56:33.617 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.617 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-09-21T03:56:33.618 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-09-21T03:56:33.618 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.618 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.618 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.618 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:33.618 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.618 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.618 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.618 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.618 INFO:teuthology.orchestra.run.smithi039.stdout: "testlkey": { 2024-09-21T03:56:33.618 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "testlkey", 2024-09-21T03:56:33.618 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-09-21T03:56:33.618 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.618 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-09-21T03:56:33.618 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-09-21T03:56:33.619 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.619 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.619 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.619 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:33.619 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.619 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.619 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.619 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.619 INFO:teuthology.orchestra.run.smithi039.stdout: "testnewline": { 2024-09-21T03:56:33.619 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "testnewline", 2024-09-21T03:56:33.619 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-09-21T03:56:33.619 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.619 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-09-21T03:56:33.619 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-09-21T03:56:33.619 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.619 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.620 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.620 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:33.620 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.620 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.620 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.620 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-21T03:56:33.620 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-21T03:56:33.620 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.620 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-21T03:56:33.620 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "smb", 2024-09-21T03:56:33.620 INFO:teuthology.orchestra.run.smithi039.stdout: "can_run": true, 2024-09-21T03:56:33.620 INFO:teuthology.orchestra.run.smithi039.stdout: "error_string": "", 2024-09-21T03:56:33.620 INFO:teuthology.orchestra.run.smithi039.stdout: "module_options": { 2024-09-21T03:56:33.620 INFO:teuthology.orchestra.run.smithi039.stdout: "internal_store_backend": { 2024-09-21T03:56:33.620 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "internal_store_backend", 2024-09-21T03:56:33.621 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-09-21T03:56:33.621 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "dev", 2024-09-21T03:56:33.621 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-09-21T03:56:33.621 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-09-21T03:56:33.621 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.621 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.621 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.621 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "set internal store backend. for develoment and testing only", 2024-09-21T03:56:33.621 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.621 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.621 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.621 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.621 INFO:teuthology.orchestra.run.smithi039.stdout: "log_level": { 2024-09-21T03:56:33.621 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_level", 2024-09-21T03:56:33.621 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-09-21T03:56:33.621 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.622 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:33.622 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-09-21T03:56:33.622 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.622 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.622 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-09-21T03:56:33.622 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-09-21T03:56:33.622 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-09-21T03:56:33.622 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-09-21T03:56:33.622 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-09-21T03:56:33.622 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-09-21T03:56:33.622 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-09-21T03:56:33.622 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-09-21T03:56:33.622 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:33.622 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.623 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.623 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.623 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.623 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster": { 2024-09-21T03:56:33.623 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster", 2024-09-21T03:56:33.623 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-09-21T03:56:33.623 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.623 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:33.623 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-09-21T03:56:33.623 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.623 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.623 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.623 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:33.623 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.623 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.624 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.624 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.624 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster_level": { 2024-09-21T03:56:33.624 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster_level", 2024-09-21T03:56:33.624 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-09-21T03:56:33.624 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.624 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:33.624 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "info", 2024-09-21T03:56:33.624 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.624 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.624 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-09-21T03:56:33.624 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-09-21T03:56:33.624 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-09-21T03:56:33.624 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-09-21T03:56:33.624 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-09-21T03:56:33.625 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-09-21T03:56:33.625 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-09-21T03:56:33.625 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-09-21T03:56:33.625 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:33.625 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.625 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.625 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.625 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.625 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_file": { 2024-09-21T03:56:33.625 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_file", 2024-09-21T03:56:33.625 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-09-21T03:56:33.625 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.625 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:33.625 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-09-21T03:56:33.625 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.626 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.626 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.626 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:33.626 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.626 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.626 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.626 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.626 INFO:teuthology.orchestra.run.smithi039.stdout: "sqlite3_killpoint": { 2024-09-21T03:56:33.626 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:56:33.626 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-09-21T03:56:33.626 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "dev", 2024-09-21T03:56:33.626 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:33.626 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "0", 2024-09-21T03:56:33.626 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.627 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.627 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.627 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:33.627 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.627 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.627 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.627 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.627 INFO:teuthology.orchestra.run.smithi039.stdout: "update_orchestration": { 2024-09-21T03:56:33.627 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "update_orchestration", 2024-09-21T03:56:33.627 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-09-21T03:56:33.627 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.627 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-09-21T03:56:33.627 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2024-09-21T03:56:33.627 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.627 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.628 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.628 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "automatically update orchestration when smb resources are changed", 2024-09-21T03:56:33.628 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.628 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.628 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.628 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-21T03:56:33.628 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-21T03:56:33.628 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.628 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-21T03:56:33.628 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "snap_schedule", 2024-09-21T03:56:33.628 INFO:teuthology.orchestra.run.smithi039.stdout: "can_run": true, 2024-09-21T03:56:33.628 INFO:teuthology.orchestra.run.smithi039.stdout: "error_string": "", 2024-09-21T03:56:33.628 INFO:teuthology.orchestra.run.smithi039.stdout: "module_options": { 2024-09-21T03:56:33.629 INFO:teuthology.orchestra.run.smithi039.stdout: "allow_m_granularity": { 2024-09-21T03:56:33.629 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "allow_m_granularity", 2024-09-21T03:56:33.629 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-09-21T03:56:33.629 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.629 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:33.629 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-09-21T03:56:33.629 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.629 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.629 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.629 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "allow minute scheduled snapshots", 2024-09-21T03:56:33.629 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.629 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.629 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.629 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.629 INFO:teuthology.orchestra.run.smithi039.stdout: "dump_on_update": { 2024-09-21T03:56:33.630 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "dump_on_update", 2024-09-21T03:56:33.630 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-09-21T03:56:33.630 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.630 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:33.630 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-09-21T03:56:33.630 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.630 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.630 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.630 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "dump database to debug log on update", 2024-09-21T03:56:33.630 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.630 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.630 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.630 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.631 INFO:teuthology.orchestra.run.smithi039.stdout: "log_level": { 2024-09-21T03:56:33.631 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_level", 2024-09-21T03:56:33.631 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-09-21T03:56:33.631 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.631 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:33.631 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-09-21T03:56:33.631 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.631 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.631 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-09-21T03:56:33.631 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-09-21T03:56:33.631 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-09-21T03:56:33.631 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-09-21T03:56:33.631 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-09-21T03:56:33.631 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-09-21T03:56:33.632 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-09-21T03:56:33.632 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-09-21T03:56:33.632 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:33.632 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.632 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.632 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.632 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.632 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster": { 2024-09-21T03:56:33.632 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster", 2024-09-21T03:56:33.632 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-09-21T03:56:33.632 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.632 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:33.632 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-09-21T03:56:33.632 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.633 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.633 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.633 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:33.633 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.633 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.633 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.633 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.633 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster_level": { 2024-09-21T03:56:33.633 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster_level", 2024-09-21T03:56:33.633 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-09-21T03:56:33.633 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.633 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:33.633 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "info", 2024-09-21T03:56:33.633 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.634 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.634 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-09-21T03:56:33.634 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-09-21T03:56:33.634 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-09-21T03:56:33.634 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-09-21T03:56:33.634 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-09-21T03:56:33.634 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-09-21T03:56:33.634 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-09-21T03:56:33.634 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-09-21T03:56:33.634 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:33.634 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.634 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.634 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.634 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.634 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_file": { 2024-09-21T03:56:33.635 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_file", 2024-09-21T03:56:33.635 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-09-21T03:56:33.635 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.635 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:33.635 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-09-21T03:56:33.635 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.635 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.635 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.635 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:33.635 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.635 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.635 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.635 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.635 INFO:teuthology.orchestra.run.smithi039.stdout: "sqlite3_killpoint": { 2024-09-21T03:56:33.636 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:56:33.636 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-09-21T03:56:33.636 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "dev", 2024-09-21T03:56:33.636 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:33.636 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "0", 2024-09-21T03:56:33.636 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.636 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.636 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.636 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:33.636 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.636 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.636 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.636 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-21T03:56:33.636 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-21T03:56:33.636 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.637 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-21T03:56:33.637 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "stats", 2024-09-21T03:56:33.637 INFO:teuthology.orchestra.run.smithi039.stdout: "can_run": true, 2024-09-21T03:56:33.637 INFO:teuthology.orchestra.run.smithi039.stdout: "error_string": "", 2024-09-21T03:56:33.637 INFO:teuthology.orchestra.run.smithi039.stdout: "module_options": { 2024-09-21T03:56:33.637 INFO:teuthology.orchestra.run.smithi039.stdout: "log_level": { 2024-09-21T03:56:33.637 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_level", 2024-09-21T03:56:33.637 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-09-21T03:56:33.637 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.637 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:33.637 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-09-21T03:56:33.637 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.637 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.637 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-09-21T03:56:33.637 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-09-21T03:56:33.637 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-09-21T03:56:33.638 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-09-21T03:56:33.638 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-09-21T03:56:33.638 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-09-21T03:56:33.638 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-09-21T03:56:33.638 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-09-21T03:56:33.638 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:33.638 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.638 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.638 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.638 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.638 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster": { 2024-09-21T03:56:33.638 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster", 2024-09-21T03:56:33.638 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-09-21T03:56:33.638 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.639 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:33.639 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-09-21T03:56:33.639 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.639 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.639 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.639 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:33.639 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.639 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.639 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.639 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.639 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster_level": { 2024-09-21T03:56:33.639 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster_level", 2024-09-21T03:56:33.639 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-09-21T03:56:33.639 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.639 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:33.639 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "info", 2024-09-21T03:56:33.640 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.640 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.640 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-09-21T03:56:33.640 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-09-21T03:56:33.640 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-09-21T03:56:33.640 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-09-21T03:56:33.640 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-09-21T03:56:33.640 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-09-21T03:56:33.640 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-09-21T03:56:33.640 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-09-21T03:56:33.640 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:33.640 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.640 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.640 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.640 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.640 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_file": { 2024-09-21T03:56:33.641 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_file", 2024-09-21T03:56:33.641 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-09-21T03:56:33.641 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.641 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:33.641 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-09-21T03:56:33.641 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.641 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.641 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.641 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:33.641 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.641 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.641 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.641 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.641 INFO:teuthology.orchestra.run.smithi039.stdout: "sqlite3_killpoint": { 2024-09-21T03:56:33.642 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:56:33.642 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-09-21T03:56:33.642 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "dev", 2024-09-21T03:56:33.642 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:33.642 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "0", 2024-09-21T03:56:33.642 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.642 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.642 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.642 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:33.642 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.642 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.642 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.642 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-21T03:56:33.642 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-21T03:56:33.643 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.643 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-21T03:56:33.643 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "status", 2024-09-21T03:56:33.643 INFO:teuthology.orchestra.run.smithi039.stdout: "can_run": true, 2024-09-21T03:56:33.643 INFO:teuthology.orchestra.run.smithi039.stdout: "error_string": "", 2024-09-21T03:56:33.643 INFO:teuthology.orchestra.run.smithi039.stdout: "module_options": { 2024-09-21T03:56:33.643 INFO:teuthology.orchestra.run.smithi039.stdout: "log_level": { 2024-09-21T03:56:33.643 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_level", 2024-09-21T03:56:33.643 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-09-21T03:56:33.643 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.643 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:33.643 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-09-21T03:56:33.643 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.643 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.644 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-09-21T03:56:33.644 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-09-21T03:56:33.644 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-09-21T03:56:33.644 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-09-21T03:56:33.644 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-09-21T03:56:33.644 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-09-21T03:56:33.644 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-09-21T03:56:33.644 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-09-21T03:56:33.644 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:33.644 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.644 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.644 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.644 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.644 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster": { 2024-09-21T03:56:33.644 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster", 2024-09-21T03:56:33.644 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-09-21T03:56:33.645 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.645 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:33.645 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-09-21T03:56:33.645 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.645 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.645 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.645 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:33.645 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.645 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.645 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.645 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.645 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster_level": { 2024-09-21T03:56:33.645 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster_level", 2024-09-21T03:56:33.645 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-09-21T03:56:33.645 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.645 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:33.646 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "info", 2024-09-21T03:56:33.646 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.646 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.646 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-09-21T03:56:33.646 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-09-21T03:56:33.646 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-09-21T03:56:33.646 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-09-21T03:56:33.646 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-09-21T03:56:33.646 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-09-21T03:56:33.646 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-09-21T03:56:33.646 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-09-21T03:56:33.646 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:33.646 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.646 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.646 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.647 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.647 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_file": { 2024-09-21T03:56:33.647 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_file", 2024-09-21T03:56:33.647 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-09-21T03:56:33.647 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.647 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:33.647 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-09-21T03:56:33.647 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.647 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.647 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.647 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:33.647 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.647 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.647 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.647 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.648 INFO:teuthology.orchestra.run.smithi039.stdout: "sqlite3_killpoint": { 2024-09-21T03:56:33.648 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:56:33.648 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-09-21T03:56:33.648 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "dev", 2024-09-21T03:56:33.648 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:33.648 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "0", 2024-09-21T03:56:33.648 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.648 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.648 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.648 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:33.648 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.648 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.648 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.648 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-21T03:56:33.649 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-21T03:56:33.649 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.649 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-21T03:56:33.649 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "telegraf", 2024-09-21T03:56:33.649 INFO:teuthology.orchestra.run.smithi039.stdout: "can_run": true, 2024-09-21T03:56:33.649 INFO:teuthology.orchestra.run.smithi039.stdout: "error_string": "", 2024-09-21T03:56:33.649 INFO:teuthology.orchestra.run.smithi039.stdout: "module_options": { 2024-09-21T03:56:33.649 INFO:teuthology.orchestra.run.smithi039.stdout: "address": { 2024-09-21T03:56:33.649 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "address", 2024-09-21T03:56:33.649 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-09-21T03:56:33.649 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.649 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-09-21T03:56:33.649 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-09-21T03:56:33.649 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.649 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.650 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.650 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:33.650 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.650 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.650 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.650 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.650 INFO:teuthology.orchestra.run.smithi039.stdout: "interval": { 2024-09-21T03:56:33.650 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "interval", 2024-09-21T03:56:33.650 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "secs", 2024-09-21T03:56:33.650 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.650 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-09-21T03:56:33.650 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "15", 2024-09-21T03:56:33.650 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.650 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.650 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.651 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:33.651 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.651 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.651 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.651 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.651 INFO:teuthology.orchestra.run.smithi039.stdout: "log_level": { 2024-09-21T03:56:33.651 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_level", 2024-09-21T03:56:33.651 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-09-21T03:56:33.651 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.651 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:33.651 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-09-21T03:56:33.651 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.651 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.651 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-09-21T03:56:33.652 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-09-21T03:56:33.652 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-09-21T03:56:33.652 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-09-21T03:56:33.652 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-09-21T03:56:33.652 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-09-21T03:56:33.652 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-09-21T03:56:33.652 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-09-21T03:56:33.652 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:33.652 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.652 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.652 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.652 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.652 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster": { 2024-09-21T03:56:33.652 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster", 2024-09-21T03:56:33.652 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-09-21T03:56:33.653 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.653 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:33.653 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-09-21T03:56:33.653 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.653 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.653 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.653 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:33.653 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.653 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.653 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.653 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.653 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster_level": { 2024-09-21T03:56:33.653 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster_level", 2024-09-21T03:56:33.654 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-09-21T03:56:33.654 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.654 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:33.654 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "info", 2024-09-21T03:56:33.654 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.654 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.654 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-09-21T03:56:33.654 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-09-21T03:56:33.654 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-09-21T03:56:33.654 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-09-21T03:56:33.654 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-09-21T03:56:33.654 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-09-21T03:56:33.654 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-09-21T03:56:33.654 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-09-21T03:56:33.655 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:33.655 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.655 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.655 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.655 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.655 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_file": { 2024-09-21T03:56:33.655 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_file", 2024-09-21T03:56:33.655 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-09-21T03:56:33.655 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.655 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:33.655 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-09-21T03:56:33.655 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.655 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.655 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.656 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:33.656 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.656 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.656 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.656 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.656 INFO:teuthology.orchestra.run.smithi039.stdout: "sqlite3_killpoint": { 2024-09-21T03:56:33.656 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:56:33.656 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-09-21T03:56:33.656 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "dev", 2024-09-21T03:56:33.656 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:33.656 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "0", 2024-09-21T03:56:33.656 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.656 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.656 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.656 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:33.657 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.657 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.657 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.657 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-21T03:56:33.657 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-21T03:56:33.657 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.657 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-21T03:56:33.657 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "telemetry", 2024-09-21T03:56:33.657 INFO:teuthology.orchestra.run.smithi039.stdout: "can_run": true, 2024-09-21T03:56:33.657 INFO:teuthology.orchestra.run.smithi039.stdout: "error_string": "", 2024-09-21T03:56:33.657 INFO:teuthology.orchestra.run.smithi039.stdout: "module_options": { 2024-09-21T03:56:33.657 INFO:teuthology.orchestra.run.smithi039.stdout: "channel_basic": { 2024-09-21T03:56:33.657 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "channel_basic", 2024-09-21T03:56:33.658 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-09-21T03:56:33.658 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.658 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-09-21T03:56:33.658 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2024-09-21T03:56:33.658 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.658 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.658 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.658 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Share basic cluster information (size, version)", 2024-09-21T03:56:33.658 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.658 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.658 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.658 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.658 INFO:teuthology.orchestra.run.smithi039.stdout: "channel_crash": { 2024-09-21T03:56:33.658 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "channel_crash", 2024-09-21T03:56:33.659 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-09-21T03:56:33.659 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.659 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-09-21T03:56:33.659 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2024-09-21T03:56:33.659 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.659 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.659 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.659 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-09-21T03:56:33.659 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.659 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.659 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.659 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.659 INFO:teuthology.orchestra.run.smithi039.stdout: "channel_device": { 2024-09-21T03:56:33.659 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "channel_device", 2024-09-21T03:56:33.660 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-09-21T03:56:33.660 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.660 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-09-21T03:56:33.660 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2024-09-21T03:56:33.660 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.660 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.660 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.660 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-09-21T03:56:33.660 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.660 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.660 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.660 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.660 INFO:teuthology.orchestra.run.smithi039.stdout: "channel_ident": { 2024-09-21T03:56:33.660 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "channel_ident", 2024-09-21T03:56:33.661 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-09-21T03:56:33.661 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.661 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-09-21T03:56:33.661 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-09-21T03:56:33.661 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.661 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.661 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.661 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-09-21T03:56:33.661 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.661 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.661 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.661 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.661 INFO:teuthology.orchestra.run.smithi039.stdout: "channel_perf": { 2024-09-21T03:56:33.661 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "channel_perf", 2024-09-21T03:56:33.661 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-09-21T03:56:33.662 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.662 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-09-21T03:56:33.662 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-09-21T03:56:33.662 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.662 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.662 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.662 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Share various performance metrics of a cluster", 2024-09-21T03:56:33.662 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.662 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.662 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.662 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.662 INFO:teuthology.orchestra.run.smithi039.stdout: "contact": { 2024-09-21T03:56:33.662 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "contact", 2024-09-21T03:56:33.663 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-09-21T03:56:33.663 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.663 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-09-21T03:56:33.663 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-09-21T03:56:33.663 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.663 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.663 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.663 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:33.663 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.663 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.663 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.663 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.663 INFO:teuthology.orchestra.run.smithi039.stdout: "description": { 2024-09-21T03:56:33.663 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "description", 2024-09-21T03:56:33.664 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-09-21T03:56:33.664 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.664 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-09-21T03:56:33.664 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-09-21T03:56:33.664 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.664 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.664 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.664 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:33.664 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.664 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.664 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.664 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.664 INFO:teuthology.orchestra.run.smithi039.stdout: "device_url": { 2024-09-21T03:56:33.664 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "device_url", 2024-09-21T03:56:33.664 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-09-21T03:56:33.665 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.665 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-09-21T03:56:33.665 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-09-21T03:56:33.665 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.665 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.665 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.665 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:33.665 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.665 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.665 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.665 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.665 INFO:teuthology.orchestra.run.smithi039.stdout: "enabled": { 2024-09-21T03:56:33.665 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "enabled", 2024-09-21T03:56:33.665 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-09-21T03:56:33.665 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.666 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-09-21T03:56:33.666 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-09-21T03:56:33.666 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.666 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.666 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.666 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:33.666 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.666 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.666 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.666 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.666 INFO:teuthology.orchestra.run.smithi039.stdout: "interval": { 2024-09-21T03:56:33.666 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "interval", 2024-09-21T03:56:33.666 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-09-21T03:56:33.666 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.667 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-09-21T03:56:33.667 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "24", 2024-09-21T03:56:33.667 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "8", 2024-09-21T03:56:33.667 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.667 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.667 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:33.667 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.667 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.667 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.667 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.667 INFO:teuthology.orchestra.run.smithi039.stdout: "last_opt_revision": { 2024-09-21T03:56:33.667 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "last_opt_revision", 2024-09-21T03:56:33.667 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-09-21T03:56:33.667 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.667 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-09-21T03:56:33.668 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "1", 2024-09-21T03:56:33.668 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.668 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.668 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.668 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:33.668 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.668 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.668 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.668 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.668 INFO:teuthology.orchestra.run.smithi039.stdout: "leaderboard": { 2024-09-21T03:56:33.668 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "leaderboard", 2024-09-21T03:56:33.668 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-09-21T03:56:33.668 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.668 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-09-21T03:56:33.669 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-09-21T03:56:33.669 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.669 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.669 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.669 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:33.669 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.669 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.669 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.669 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.669 INFO:teuthology.orchestra.run.smithi039.stdout: "leaderboard_description": { 2024-09-21T03:56:33.669 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "leaderboard_description", 2024-09-21T03:56:33.669 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-09-21T03:56:33.669 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.669 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-09-21T03:56:33.670 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-09-21T03:56:33.670 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.670 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.670 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.670 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:33.670 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.670 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.670 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.670 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.670 INFO:teuthology.orchestra.run.smithi039.stdout: "log_level": { 2024-09-21T03:56:33.670 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_level", 2024-09-21T03:56:33.670 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-09-21T03:56:33.670 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.670 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:33.671 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-09-21T03:56:33.671 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.671 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.671 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-09-21T03:56:33.671 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-09-21T03:56:33.671 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-09-21T03:56:33.671 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-09-21T03:56:33.671 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-09-21T03:56:33.671 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-09-21T03:56:33.671 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-09-21T03:56:33.671 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-09-21T03:56:33.671 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:33.671 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.672 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.672 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.672 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.672 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster": { 2024-09-21T03:56:33.672 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster", 2024-09-21T03:56:33.672 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-09-21T03:56:33.672 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.672 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:33.672 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-09-21T03:56:33.672 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.672 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.672 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.672 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:33.672 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.672 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.673 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.673 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.673 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster_level": { 2024-09-21T03:56:33.673 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster_level", 2024-09-21T03:56:33.673 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-09-21T03:56:33.673 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.673 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:33.673 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "info", 2024-09-21T03:56:33.673 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.673 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.673 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-09-21T03:56:33.673 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-09-21T03:56:33.673 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-09-21T03:56:33.673 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-09-21T03:56:33.673 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-09-21T03:56:33.674 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-09-21T03:56:33.674 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-09-21T03:56:33.674 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-09-21T03:56:33.674 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:33.674 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.674 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.674 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.674 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.674 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_file": { 2024-09-21T03:56:33.674 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_file", 2024-09-21T03:56:33.674 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-09-21T03:56:33.674 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.674 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:33.674 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-09-21T03:56:33.675 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.675 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.675 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.675 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:33.675 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.675 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.675 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.675 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.675 INFO:teuthology.orchestra.run.smithi039.stdout: "organization": { 2024-09-21T03:56:33.675 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "organization", 2024-09-21T03:56:33.675 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-09-21T03:56:33.675 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.675 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-09-21T03:56:33.675 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-09-21T03:56:33.676 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.676 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.676 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.676 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:33.676 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.676 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.676 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.676 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.676 INFO:teuthology.orchestra.run.smithi039.stdout: "proxy": { 2024-09-21T03:56:33.676 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "proxy", 2024-09-21T03:56:33.676 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-09-21T03:56:33.676 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.676 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-09-21T03:56:33.676 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-09-21T03:56:33.676 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.676 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.677 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.677 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:33.677 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.677 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.677 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.677 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.677 INFO:teuthology.orchestra.run.smithi039.stdout: "sqlite3_killpoint": { 2024-09-21T03:56:33.677 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:56:33.677 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-09-21T03:56:33.677 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "dev", 2024-09-21T03:56:33.677 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:33.677 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "0", 2024-09-21T03:56:33.677 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.677 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.678 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.678 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:33.678 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.678 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.678 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.678 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.678 INFO:teuthology.orchestra.run.smithi039.stdout: "url": { 2024-09-21T03:56:33.678 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "url", 2024-09-21T03:56:33.678 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-09-21T03:56:33.678 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.678 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-09-21T03:56:33.678 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-09-21T03:56:33.678 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.678 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.679 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.679 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:33.679 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.679 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.679 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.679 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-21T03:56:33.679 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-21T03:56:33.679 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.679 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-21T03:56:33.679 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "test_orchestrator", 2024-09-21T03:56:33.679 INFO:teuthology.orchestra.run.smithi039.stdout: "can_run": true, 2024-09-21T03:56:33.679 INFO:teuthology.orchestra.run.smithi039.stdout: "error_string": "", 2024-09-21T03:56:33.679 INFO:teuthology.orchestra.run.smithi039.stdout: "module_options": { 2024-09-21T03:56:33.679 INFO:teuthology.orchestra.run.smithi039.stdout: "log_level": { 2024-09-21T03:56:33.679 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_level", 2024-09-21T03:56:33.680 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-09-21T03:56:33.680 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.680 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:33.680 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-09-21T03:56:33.680 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.680 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.680 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-09-21T03:56:33.680 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-09-21T03:56:33.680 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-09-21T03:56:33.680 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-09-21T03:56:33.680 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-09-21T03:56:33.680 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-09-21T03:56:33.680 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-09-21T03:56:33.680 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-09-21T03:56:33.680 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:33.681 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.681 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.681 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.681 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.681 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster": { 2024-09-21T03:56:33.681 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster", 2024-09-21T03:56:33.681 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-09-21T03:56:33.681 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.681 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:33.681 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-09-21T03:56:33.681 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.681 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.681 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.681 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:33.681 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.681 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.682 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.682 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.682 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster_level": { 2024-09-21T03:56:33.682 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster_level", 2024-09-21T03:56:33.682 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-09-21T03:56:33.682 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.682 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:33.682 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "info", 2024-09-21T03:56:33.682 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.682 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.682 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-09-21T03:56:33.682 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-09-21T03:56:33.682 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-09-21T03:56:33.682 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-09-21T03:56:33.683 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-09-21T03:56:33.683 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-09-21T03:56:33.683 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-09-21T03:56:33.683 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-09-21T03:56:33.683 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:33.683 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.683 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.683 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.683 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.683 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_file": { 2024-09-21T03:56:33.683 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_file", 2024-09-21T03:56:33.683 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-09-21T03:56:33.683 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.683 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:33.684 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-09-21T03:56:33.684 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.684 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.684 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.684 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:33.684 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.684 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.684 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.684 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.684 INFO:teuthology.orchestra.run.smithi039.stdout: "sqlite3_killpoint": { 2024-09-21T03:56:33.684 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:56:33.684 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-09-21T03:56:33.684 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "dev", 2024-09-21T03:56:33.684 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:33.685 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "0", 2024-09-21T03:56:33.685 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.685 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.685 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.685 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:33.685 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.685 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.685 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.685 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-21T03:56:33.685 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-21T03:56:33.685 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.685 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-21T03:56:33.685 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "volumes", 2024-09-21T03:56:33.685 INFO:teuthology.orchestra.run.smithi039.stdout: "can_run": true, 2024-09-21T03:56:33.685 INFO:teuthology.orchestra.run.smithi039.stdout: "error_string": "", 2024-09-21T03:56:33.685 INFO:teuthology.orchestra.run.smithi039.stdout: "module_options": { 2024-09-21T03:56:33.686 INFO:teuthology.orchestra.run.smithi039.stdout: "log_level": { 2024-09-21T03:56:33.686 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_level", 2024-09-21T03:56:33.686 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-09-21T03:56:33.686 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.686 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:33.686 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-09-21T03:56:33.686 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.686 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.686 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-09-21T03:56:33.686 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-09-21T03:56:33.686 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-09-21T03:56:33.686 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-09-21T03:56:33.686 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-09-21T03:56:33.686 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-09-21T03:56:33.687 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-09-21T03:56:33.687 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-09-21T03:56:33.687 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:33.687 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.687 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.687 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.687 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.687 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster": { 2024-09-21T03:56:33.687 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster", 2024-09-21T03:56:33.687 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-09-21T03:56:33.687 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.687 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:33.687 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-09-21T03:56:33.687 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.687 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.688 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.688 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:33.688 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.688 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.688 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.688 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.688 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster_level": { 2024-09-21T03:56:33.688 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster_level", 2024-09-21T03:56:33.688 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-09-21T03:56:33.688 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.688 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:33.688 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "info", 2024-09-21T03:56:33.688 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.688 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.688 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-09-21T03:56:33.688 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-09-21T03:56:33.689 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-09-21T03:56:33.689 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-09-21T03:56:33.689 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-09-21T03:56:33.689 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-09-21T03:56:33.689 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-09-21T03:56:33.689 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-09-21T03:56:33.689 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:33.689 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.689 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.689 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.689 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.689 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_file": { 2024-09-21T03:56:33.689 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_file", 2024-09-21T03:56:33.689 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-09-21T03:56:33.689 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.690 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:33.690 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-09-21T03:56:33.690 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.690 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.690 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.690 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:33.690 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.690 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.690 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.690 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.690 INFO:teuthology.orchestra.run.smithi039.stdout: "max_concurrent_clones": { 2024-09-21T03:56:33.690 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "max_concurrent_clones", 2024-09-21T03:56:33.690 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-09-21T03:56:33.690 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.691 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-09-21T03:56:33.691 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "4", 2024-09-21T03:56:33.691 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.691 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.691 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.691 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Number of asynchronous cloner threads", 2024-09-21T03:56:33.691 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.691 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.691 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.691 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.691 INFO:teuthology.orchestra.run.smithi039.stdout: "periodic_async_work": { 2024-09-21T03:56:33.691 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "periodic_async_work", 2024-09-21T03:56:33.691 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-09-21T03:56:33.691 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.691 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-09-21T03:56:33.692 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-09-21T03:56:33.692 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.692 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.692 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.692 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Periodically check for async work", 2024-09-21T03:56:33.692 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.692 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.692 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.692 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.692 INFO:teuthology.orchestra.run.smithi039.stdout: "snapshot_clone_delay": { 2024-09-21T03:56:33.692 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "snapshot_clone_delay", 2024-09-21T03:56:33.692 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-09-21T03:56:33.692 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.692 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-09-21T03:56:33.692 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "0", 2024-09-21T03:56:33.693 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.693 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.693 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.693 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-09-21T03:56:33.693 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.693 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.693 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.693 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.693 INFO:teuthology.orchestra.run.smithi039.stdout: "snapshot_clone_no_wait": { 2024-09-21T03:56:33.693 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "snapshot_clone_no_wait", 2024-09-21T03:56:33.693 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-09-21T03:56:33.693 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.693 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-09-21T03:56:33.693 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2024-09-21T03:56:33.693 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.694 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.694 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.694 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-09-21T03:56:33.694 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.694 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.694 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.694 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.694 INFO:teuthology.orchestra.run.smithi039.stdout: "sqlite3_killpoint": { 2024-09-21T03:56:33.694 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:56:33.694 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-09-21T03:56:33.694 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "dev", 2024-09-21T03:56:33.694 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:33.694 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "0", 2024-09-21T03:56:33.695 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.695 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.695 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.695 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:33.695 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.695 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.695 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.695 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-21T03:56:33.695 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-21T03:56:33.695 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.695 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-21T03:56:33.695 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "zabbix", 2024-09-21T03:56:33.695 INFO:teuthology.orchestra.run.smithi039.stdout: "can_run": true, 2024-09-21T03:56:33.696 INFO:teuthology.orchestra.run.smithi039.stdout: "error_string": "", 2024-09-21T03:56:33.696 INFO:teuthology.orchestra.run.smithi039.stdout: "module_options": { 2024-09-21T03:56:33.696 INFO:teuthology.orchestra.run.smithi039.stdout: "discovery_interval": { 2024-09-21T03:56:33.696 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "discovery_interval", 2024-09-21T03:56:33.696 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "uint", 2024-09-21T03:56:33.696 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.696 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-09-21T03:56:33.696 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "100", 2024-09-21T03:56:33.696 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.696 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.696 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.696 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:33.696 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.696 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.696 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.696 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.697 INFO:teuthology.orchestra.run.smithi039.stdout: "identifier": { 2024-09-21T03:56:33.697 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "identifier", 2024-09-21T03:56:33.697 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-09-21T03:56:33.697 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.697 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-09-21T03:56:33.697 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-09-21T03:56:33.697 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.697 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.697 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.697 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:33.697 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.697 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.697 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.697 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.697 INFO:teuthology.orchestra.run.smithi039.stdout: "interval": { 2024-09-21T03:56:33.698 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "interval", 2024-09-21T03:56:33.698 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "secs", 2024-09-21T03:56:33.698 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.698 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-09-21T03:56:33.698 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "60", 2024-09-21T03:56:33.698 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.698 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.698 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.698 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:33.698 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.698 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.698 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.698 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.698 INFO:teuthology.orchestra.run.smithi039.stdout: "log_level": { 2024-09-21T03:56:33.699 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_level", 2024-09-21T03:56:33.699 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-09-21T03:56:33.699 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.699 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:33.699 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-09-21T03:56:33.699 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.699 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.699 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-09-21T03:56:33.699 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-09-21T03:56:33.699 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-09-21T03:56:33.699 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-09-21T03:56:33.699 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-09-21T03:56:33.699 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-09-21T03:56:33.699 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-09-21T03:56:33.699 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-09-21T03:56:33.700 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:33.700 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.700 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.700 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.700 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.700 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster": { 2024-09-21T03:56:33.700 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster", 2024-09-21T03:56:33.700 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-09-21T03:56:33.700 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.700 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:33.700 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-09-21T03:56:33.700 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.700 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.700 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.700 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:33.701 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.701 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.701 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.701 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.701 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster_level": { 2024-09-21T03:56:33.701 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster_level", 2024-09-21T03:56:33.701 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-09-21T03:56:33.701 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.701 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:33.701 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "info", 2024-09-21T03:56:33.701 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.701 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.701 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-09-21T03:56:33.701 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-09-21T03:56:33.701 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-09-21T03:56:33.702 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-09-21T03:56:33.702 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-09-21T03:56:33.702 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-09-21T03:56:33.702 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-09-21T03:56:33.702 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-09-21T03:56:33.702 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:33.702 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.702 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.702 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.702 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.702 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_file": { 2024-09-21T03:56:33.702 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_file", 2024-09-21T03:56:33.702 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-09-21T03:56:33.703 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.703 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:33.703 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-09-21T03:56:33.703 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.703 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.703 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.703 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:33.703 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.703 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.703 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.703 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.703 INFO:teuthology.orchestra.run.smithi039.stdout: "sqlite3_killpoint": { 2024-09-21T03:56:33.703 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:56:33.703 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-09-21T03:56:33.703 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "dev", 2024-09-21T03:56:33.704 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:33.704 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "0", 2024-09-21T03:56:33.704 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.704 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.704 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.704 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:33.704 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.704 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.704 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.704 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.704 INFO:teuthology.orchestra.run.smithi039.stdout: "zabbix_host": { 2024-09-21T03:56:33.704 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "zabbix_host", 2024-09-21T03:56:33.704 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-09-21T03:56:33.704 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.704 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-09-21T03:56:33.705 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-09-21T03:56:33.705 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.705 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.705 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.705 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:33.705 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.705 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.705 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.705 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.705 INFO:teuthology.orchestra.run.smithi039.stdout: "zabbix_port": { 2024-09-21T03:56:33.705 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "zabbix_port", 2024-09-21T03:56:33.705 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-09-21T03:56:33.705 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.705 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-09-21T03:56:33.706 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "10051", 2024-09-21T03:56:33.706 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.706 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.706 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.706 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:33.706 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.706 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.706 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.706 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.706 INFO:teuthology.orchestra.run.smithi039.stdout: "zabbix_sender": { 2024-09-21T03:56:33.706 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "zabbix_sender", 2024-09-21T03:56:33.706 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-09-21T03:56:33.706 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.706 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-09-21T03:56:33.706 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-09-21T03:56:33.707 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.707 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.707 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.707 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:33.707 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.707 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.707 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.707 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-21T03:56:33.707 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-21T03:56:33.707 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-21T03:56:33.707 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-21T03:56:33.707 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-21T03:56:33.707 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-09-21T03:56:33.707 INFO:teuthology.orchestra.run.smithi039.stdout: "modules": [ 2024-09-21T03:56:33.708 INFO:teuthology.orchestra.run.smithi039.stdout: "cephadm", 2024-09-21T03:56:33.708 INFO:teuthology.orchestra.run.smithi039.stdout: "dashboard", 2024-09-21T03:56:33.708 INFO:teuthology.orchestra.run.smithi039.stdout: "iostat", 2024-09-21T03:56:33.708 INFO:teuthology.orchestra.run.smithi039.stdout: "nfs", 2024-09-21T03:56:33.708 INFO:teuthology.orchestra.run.smithi039.stdout: "restful", 2024-09-21T03:56:33.708 INFO:teuthology.orchestra.run.smithi039.stdout: "snap_schedule" 2024-09-21T03:56:33.708 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-09-21T03:56:33.708 INFO:teuthology.orchestra.run.smithi039.stdout: "available_modules": [ 2024-09-21T03:56:33.708 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-21T03:56:33.708 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "alerts", 2024-09-21T03:56:33.708 INFO:teuthology.orchestra.run.smithi039.stdout: "can_run": true, 2024-09-21T03:56:33.708 INFO:teuthology.orchestra.run.smithi039.stdout: "error_string": "", 2024-09-21T03:56:33.708 INFO:teuthology.orchestra.run.smithi039.stdout: "module_options": { 2024-09-21T03:56:33.708 INFO:teuthology.orchestra.run.smithi039.stdout: "interval": { 2024-09-21T03:56:33.709 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "interval", 2024-09-21T03:56:33.709 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "secs", 2024-09-21T03:56:33.709 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.709 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:33.709 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "60", 2024-09-21T03:56:33.709 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.709 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.709 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.709 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "How frequently to reexamine health status", 2024-09-21T03:56:33.709 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.709 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.709 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.709 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.709 INFO:teuthology.orchestra.run.smithi039.stdout: "log_level": { 2024-09-21T03:56:33.710 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_level", 2024-09-21T03:56:33.710 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-09-21T03:56:33.710 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.710 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:33.710 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-09-21T03:56:33.710 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.710 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.710 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-09-21T03:56:33.710 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-09-21T03:56:33.710 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-09-21T03:56:33.710 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-09-21T03:56:33.710 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-09-21T03:56:33.710 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-09-21T03:56:33.710 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-09-21T03:56:33.711 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-09-21T03:56:33.711 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:33.711 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.711 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.711 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.711 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.711 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster": { 2024-09-21T03:56:33.711 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster", 2024-09-21T03:56:33.711 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-09-21T03:56:33.711 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.711 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:33.711 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-09-21T03:56:33.711 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.711 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.712 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.712 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:33.712 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.712 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.712 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.712 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.712 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster_level": { 2024-09-21T03:56:33.712 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster_level", 2024-09-21T03:56:33.712 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-09-21T03:56:33.712 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.712 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:33.712 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "info", 2024-09-21T03:56:33.712 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.712 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.713 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-09-21T03:56:33.713 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-09-21T03:56:33.713 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-09-21T03:56:33.713 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-09-21T03:56:33.713 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-09-21T03:56:33.713 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-09-21T03:56:33.713 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-09-21T03:56:33.713 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-09-21T03:56:33.713 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:33.713 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.713 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.713 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.713 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.713 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_file": { 2024-09-21T03:56:33.713 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_file", 2024-09-21T03:56:33.714 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-09-21T03:56:33.714 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.714 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:33.714 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-09-21T03:56:33.714 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.714 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.714 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.714 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:33.714 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.714 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.714 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.714 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.714 INFO:teuthology.orchestra.run.smithi039.stdout: "smtp_destination": { 2024-09-21T03:56:33.714 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "smtp_destination", 2024-09-21T03:56:33.714 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-09-21T03:56:33.715 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.715 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:33.715 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-09-21T03:56:33.715 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.715 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.715 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.715 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Email address to send alerts to, use commas to separate multiple", 2024-09-21T03:56:33.715 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.715 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.715 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.715 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.715 INFO:teuthology.orchestra.run.smithi039.stdout: "smtp_from_name": { 2024-09-21T03:56:33.715 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "smtp_from_name", 2024-09-21T03:56:33.716 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-09-21T03:56:33.716 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.716 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:33.716 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "Ceph", 2024-09-21T03:56:33.716 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.716 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.716 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.716 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Email From: name", 2024-09-21T03:56:33.716 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.716 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.716 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.716 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.716 INFO:teuthology.orchestra.run.smithi039.stdout: "smtp_host": { 2024-09-21T03:56:33.716 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "smtp_host", 2024-09-21T03:56:33.717 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-09-21T03:56:33.717 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.717 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:33.717 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-09-21T03:56:33.717 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.717 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.717 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.717 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "SMTP server", 2024-09-21T03:56:33.717 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.717 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.717 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.717 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.717 INFO:teuthology.orchestra.run.smithi039.stdout: "smtp_password": { 2024-09-21T03:56:33.717 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "smtp_password", 2024-09-21T03:56:33.717 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-09-21T03:56:33.718 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.718 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:33.718 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-09-21T03:56:33.718 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.718 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.718 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.718 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Password to authenticate with", 2024-09-21T03:56:33.718 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.718 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.718 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.718 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.718 INFO:teuthology.orchestra.run.smithi039.stdout: "smtp_port": { 2024-09-21T03:56:33.718 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "smtp_port", 2024-09-21T03:56:33.718 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-09-21T03:56:33.718 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.718 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:33.719 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "465", 2024-09-21T03:56:33.719 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.719 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.719 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.719 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "SMTP port", 2024-09-21T03:56:33.719 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.719 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.719 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.719 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.719 INFO:teuthology.orchestra.run.smithi039.stdout: "smtp_sender": { 2024-09-21T03:56:33.719 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "smtp_sender", 2024-09-21T03:56:33.719 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-09-21T03:56:33.719 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.719 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:33.720 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-09-21T03:56:33.720 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.720 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.720 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.720 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "SMTP envelope sender", 2024-09-21T03:56:33.720 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.720 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.720 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.720 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.720 INFO:teuthology.orchestra.run.smithi039.stdout: "smtp_ssl": { 2024-09-21T03:56:33.720 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "smtp_ssl", 2024-09-21T03:56:33.720 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-09-21T03:56:33.720 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.720 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:33.720 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2024-09-21T03:56:33.721 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.721 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.721 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.721 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Use SSL to connect to SMTP server", 2024-09-21T03:56:33.721 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.721 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.721 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.721 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.721 INFO:teuthology.orchestra.run.smithi039.stdout: "smtp_user": { 2024-09-21T03:56:33.721 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "smtp_user", 2024-09-21T03:56:33.721 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-09-21T03:56:33.721 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.721 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:33.721 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-09-21T03:56:33.721 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.722 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.722 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.722 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "User to authenticate as", 2024-09-21T03:56:33.722 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.722 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.722 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.722 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.722 INFO:teuthology.orchestra.run.smithi039.stdout: "sqlite3_killpoint": { 2024-09-21T03:56:33.722 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:56:33.722 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-09-21T03:56:33.722 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "dev", 2024-09-21T03:56:33.722 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:33.722 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "0", 2024-09-21T03:56:33.723 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.723 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.723 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.723 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:33.723 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.723 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.723 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.723 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-21T03:56:33.723 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-21T03:56:33.723 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.723 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-21T03:56:33.723 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "balancer", 2024-09-21T03:56:33.723 INFO:teuthology.orchestra.run.smithi039.stdout: "can_run": true, 2024-09-21T03:56:33.723 INFO:teuthology.orchestra.run.smithi039.stdout: "error_string": "", 2024-09-21T03:56:33.723 INFO:teuthology.orchestra.run.smithi039.stdout: "module_options": { 2024-09-21T03:56:33.724 INFO:teuthology.orchestra.run.smithi039.stdout: "active": { 2024-09-21T03:56:33.724 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "active", 2024-09-21T03:56:33.724 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-09-21T03:56:33.724 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.724 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:33.724 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2024-09-21T03:56:33.724 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.724 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.724 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.724 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "automatically balance PGs across cluster", 2024-09-21T03:56:33.724 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.724 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.724 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.724 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.725 INFO:teuthology.orchestra.run.smithi039.stdout: "begin_time": { 2024-09-21T03:56:33.725 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "begin_time", 2024-09-21T03:56:33.725 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-09-21T03:56:33.725 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.725 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:33.725 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "0000", 2024-09-21T03:56:33.725 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.725 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.725 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.725 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "beginning time of day to automatically balance", 2024-09-21T03:56:33.725 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-09-21T03:56:33.725 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.725 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.725 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.726 INFO:teuthology.orchestra.run.smithi039.stdout: "begin_weekday": { 2024-09-21T03:56:33.726 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "begin_weekday", 2024-09-21T03:56:33.726 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "uint", 2024-09-21T03:56:33.726 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.726 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:33.726 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "0", 2024-09-21T03:56:33.726 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "0", 2024-09-21T03:56:33.726 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "6", 2024-09-21T03:56:33.726 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.726 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-09-21T03:56:33.726 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-09-21T03:56:33.726 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.726 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.726 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.726 INFO:teuthology.orchestra.run.smithi039.stdout: "crush_compat_max_iterations": { 2024-09-21T03:56:33.727 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "crush_compat_max_iterations", 2024-09-21T03:56:33.727 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "uint", 2024-09-21T03:56:33.727 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.727 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:33.727 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "25", 2024-09-21T03:56:33.727 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "1", 2024-09-21T03:56:33.727 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "250", 2024-09-21T03:56:33.727 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.727 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-09-21T03:56:33.727 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.727 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.727 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.727 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.728 INFO:teuthology.orchestra.run.smithi039.stdout: "crush_compat_metrics": { 2024-09-21T03:56:33.728 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "crush_compat_metrics", 2024-09-21T03:56:33.728 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-09-21T03:56:33.728 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.728 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:33.728 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "pgs,objects,bytes", 2024-09-21T03:56:33.728 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.728 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.728 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.728 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-09-21T03:56:33.728 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-09-21T03:56:33.728 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.728 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.728 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.729 INFO:teuthology.orchestra.run.smithi039.stdout: "crush_compat_step": { 2024-09-21T03:56:33.729 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "crush_compat_step", 2024-09-21T03:56:33.729 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "float", 2024-09-21T03:56:33.729 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.729 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:33.729 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "0.5", 2024-09-21T03:56:33.729 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "0.001", 2024-09-21T03:56:33.729 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "0.999", 2024-09-21T03:56:33.729 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.729 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "aggressiveness of optimization", 2024-09-21T03:56:33.729 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-09-21T03:56:33.729 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.729 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.730 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.730 INFO:teuthology.orchestra.run.smithi039.stdout: "end_time": { 2024-09-21T03:56:33.730 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "end_time", 2024-09-21T03:56:33.730 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-09-21T03:56:33.730 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.730 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:33.730 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "2359", 2024-09-21T03:56:33.730 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.730 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.730 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.730 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "ending time of day to automatically balance", 2024-09-21T03:56:33.730 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-09-21T03:56:33.730 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.730 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.731 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.731 INFO:teuthology.orchestra.run.smithi039.stdout: "end_weekday": { 2024-09-21T03:56:33.731 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "end_weekday", 2024-09-21T03:56:33.731 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "uint", 2024-09-21T03:56:33.731 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.731 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:33.731 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "0", 2024-09-21T03:56:33.731 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "0", 2024-09-21T03:56:33.731 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "6", 2024-09-21T03:56:33.731 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.731 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-09-21T03:56:33.731 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-09-21T03:56:33.731 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.731 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.731 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.732 INFO:teuthology.orchestra.run.smithi039.stdout: "log_level": { 2024-09-21T03:56:33.732 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_level", 2024-09-21T03:56:33.732 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-09-21T03:56:33.732 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.732 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:33.732 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-09-21T03:56:33.732 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.732 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.732 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-09-21T03:56:33.732 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-09-21T03:56:33.732 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-09-21T03:56:33.732 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-09-21T03:56:33.732 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-09-21T03:56:33.732 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-09-21T03:56:33.733 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-09-21T03:56:33.733 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-09-21T03:56:33.733 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:33.733 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.733 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.733 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.733 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.733 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster": { 2024-09-21T03:56:33.733 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster", 2024-09-21T03:56:33.733 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-09-21T03:56:33.733 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.733 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:33.733 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-09-21T03:56:33.733 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.734 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.734 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.734 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:33.734 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.734 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.734 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.734 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.734 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster_level": { 2024-09-21T03:56:33.734 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster_level", 2024-09-21T03:56:33.734 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-09-21T03:56:33.734 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.734 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:33.734 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "info", 2024-09-21T03:56:33.734 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.735 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.735 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-09-21T03:56:33.735 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-09-21T03:56:33.735 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-09-21T03:56:33.735 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-09-21T03:56:33.735 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-09-21T03:56:33.735 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-09-21T03:56:33.735 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-09-21T03:56:33.735 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-09-21T03:56:33.735 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:33.735 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.735 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.735 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.735 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.736 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_file": { 2024-09-21T03:56:33.736 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_file", 2024-09-21T03:56:33.736 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-09-21T03:56:33.736 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.736 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:33.736 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-09-21T03:56:33.736 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.736 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.736 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.736 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:33.736 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.736 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.736 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.736 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.736 INFO:teuthology.orchestra.run.smithi039.stdout: "min_score": { 2024-09-21T03:56:33.737 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "min_score", 2024-09-21T03:56:33.737 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "float", 2024-09-21T03:56:33.737 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.737 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:33.737 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "0", 2024-09-21T03:56:33.737 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.737 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.737 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.737 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-09-21T03:56:33.737 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.737 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.737 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.737 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.737 INFO:teuthology.orchestra.run.smithi039.stdout: "mode": { 2024-09-21T03:56:33.737 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "mode", 2024-09-21T03:56:33.738 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-09-21T03:56:33.738 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.738 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:33.738 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "upmap", 2024-09-21T03:56:33.738 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.738 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.738 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-09-21T03:56:33.738 INFO:teuthology.orchestra.run.smithi039.stdout: "crush-compat", 2024-09-21T03:56:33.738 INFO:teuthology.orchestra.run.smithi039.stdout: "none", 2024-09-21T03:56:33.738 INFO:teuthology.orchestra.run.smithi039.stdout: "read", 2024-09-21T03:56:33.738 INFO:teuthology.orchestra.run.smithi039.stdout: "upmap", 2024-09-21T03:56:33.738 INFO:teuthology.orchestra.run.smithi039.stdout: "upmap-read" 2024-09-21T03:56:33.738 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-09-21T03:56:33.738 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Balancer mode", 2024-09-21T03:56:33.738 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.739 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.739 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.739 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.739 INFO:teuthology.orchestra.run.smithi039.stdout: "pool_ids": { 2024-09-21T03:56:33.739 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "pool_ids", 2024-09-21T03:56:33.739 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-09-21T03:56:33.739 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.739 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:33.739 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-09-21T03:56:33.739 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.739 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.739 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.739 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-09-21T03:56:33.739 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.740 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.740 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.740 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.740 INFO:teuthology.orchestra.run.smithi039.stdout: "sleep_interval": { 2024-09-21T03:56:33.740 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "sleep_interval", 2024-09-21T03:56:33.740 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "secs", 2024-09-21T03:56:33.740 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.740 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:33.740 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "60", 2024-09-21T03:56:33.740 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.740 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.740 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.740 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-09-21T03:56:33.740 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.740 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.741 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.741 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.741 INFO:teuthology.orchestra.run.smithi039.stdout: "sqlite3_killpoint": { 2024-09-21T03:56:33.741 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:56:33.741 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-09-21T03:56:33.741 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "dev", 2024-09-21T03:56:33.741 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:33.741 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "0", 2024-09-21T03:56:33.741 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.741 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.741 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.741 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:33.741 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.741 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.741 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.741 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.742 INFO:teuthology.orchestra.run.smithi039.stdout: "upmap_max_deviation": { 2024-09-21T03:56:33.742 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "upmap_max_deviation", 2024-09-21T03:56:33.742 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-09-21T03:56:33.742 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.742 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:33.742 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "5", 2024-09-21T03:56:33.742 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "1", 2024-09-21T03:56:33.742 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.742 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.742 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "deviation below which no optimization is attempted", 2024-09-21T03:56:33.742 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-09-21T03:56:33.742 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.742 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.743 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.743 INFO:teuthology.orchestra.run.smithi039.stdout: "upmap_max_optimizations": { 2024-09-21T03:56:33.743 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "upmap_max_optimizations", 2024-09-21T03:56:33.743 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "uint", 2024-09-21T03:56:33.743 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.743 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:33.743 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "10", 2024-09-21T03:56:33.743 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.743 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.743 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.743 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-09-21T03:56:33.743 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.743 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.743 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.743 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-21T03:56:33.744 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-21T03:56:33.744 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.744 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-21T03:56:33.744 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "cephadm", 2024-09-21T03:56:33.744 INFO:teuthology.orchestra.run.smithi039.stdout: "can_run": true, 2024-09-21T03:56:33.744 INFO:teuthology.orchestra.run.smithi039.stdout: "error_string": "", 2024-09-21T03:56:33.744 INFO:teuthology.orchestra.run.smithi039.stdout: "module_options": { 2024-09-21T03:56:33.744 INFO:teuthology.orchestra.run.smithi039.stdout: "agent_down_multiplier": { 2024-09-21T03:56:33.744 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "agent_down_multiplier", 2024-09-21T03:56:33.744 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "float", 2024-09-21T03:56:33.744 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.744 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-09-21T03:56:33.744 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "3.0", 2024-09-21T03:56:33.744 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.744 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.744 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.745 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-09-21T03:56:33.745 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.745 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.745 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.745 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.745 INFO:teuthology.orchestra.run.smithi039.stdout: "agent_refresh_rate": { 2024-09-21T03:56:33.745 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "agent_refresh_rate", 2024-09-21T03:56:33.745 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "secs", 2024-09-21T03:56:33.745 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.745 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-09-21T03:56:33.745 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "20", 2024-09-21T03:56:33.745 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.745 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.745 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.745 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-09-21T03:56:33.746 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.746 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.746 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.746 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.746 INFO:teuthology.orchestra.run.smithi039.stdout: "agent_starting_port": { 2024-09-21T03:56:33.746 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "agent_starting_port", 2024-09-21T03:56:33.746 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-09-21T03:56:33.746 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.746 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-09-21T03:56:33.746 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "4721", 2024-09-21T03:56:33.746 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.746 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.746 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.746 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-09-21T03:56:33.747 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.747 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.747 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.747 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.747 INFO:teuthology.orchestra.run.smithi039.stdout: "allow_ptrace": { 2024-09-21T03:56:33.747 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "allow_ptrace", 2024-09-21T03:56:33.747 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-09-21T03:56:33.747 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.747 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-09-21T03:56:33.747 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-09-21T03:56:33.747 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.747 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.747 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.747 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-09-21T03:56:33.748 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-09-21T03:56:33.748 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.748 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.748 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.748 INFO:teuthology.orchestra.run.smithi039.stdout: "autotune_interval": { 2024-09-21T03:56:33.748 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "autotune_interval", 2024-09-21T03:56:33.748 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "secs", 2024-09-21T03:56:33.748 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.748 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-09-21T03:56:33.748 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "600", 2024-09-21T03:56:33.748 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.748 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.748 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.748 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "how frequently to autotune daemon memory", 2024-09-21T03:56:33.749 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.749 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.749 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.749 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.749 INFO:teuthology.orchestra.run.smithi039.stdout: "autotune_memory_target_ratio": { 2024-09-21T03:56:33.749 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "autotune_memory_target_ratio", 2024-09-21T03:56:33.749 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "float", 2024-09-21T03:56:33.749 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.749 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-09-21T03:56:33.749 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "0.7", 2024-09-21T03:56:33.749 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.749 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.749 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.749 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-09-21T03:56:33.750 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.750 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.750 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.750 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.750 INFO:teuthology.orchestra.run.smithi039.stdout: "cephadm_log_destination": { 2024-09-21T03:56:33.750 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "cephadm_log_destination", 2024-09-21T03:56:33.750 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-09-21T03:56:33.750 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.750 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-09-21T03:56:33.750 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-09-21T03:56:33.750 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.750 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.750 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-09-21T03:56:33.751 INFO:teuthology.orchestra.run.smithi039.stdout: "file", 2024-09-21T03:56:33.751 INFO:teuthology.orchestra.run.smithi039.stdout: "file,syslog", 2024-09-21T03:56:33.751 INFO:teuthology.orchestra.run.smithi039.stdout: "syslog" 2024-09-21T03:56:33.751 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-09-21T03:56:33.751 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-09-21T03:56:33.751 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.751 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.751 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.751 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.751 INFO:teuthology.orchestra.run.smithi039.stdout: "cgroups_split": { 2024-09-21T03:56:33.751 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "cgroups_split", 2024-09-21T03:56:33.751 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-09-21T03:56:33.751 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.752 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-09-21T03:56:33.752 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2024-09-21T03:56:33.752 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.752 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.752 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.752 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-09-21T03:56:33.752 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.752 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.752 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.752 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.752 INFO:teuthology.orchestra.run.smithi039.stdout: "config_checks_enabled": { 2024-09-21T03:56:33.752 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "config_checks_enabled", 2024-09-21T03:56:33.752 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-09-21T03:56:33.752 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.753 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-09-21T03:56:33.753 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-09-21T03:56:33.753 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.753 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.753 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.753 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-09-21T03:56:33.753 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.753 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.753 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.753 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.753 INFO:teuthology.orchestra.run.smithi039.stdout: "config_dashboard": { 2024-09-21T03:56:33.753 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "config_dashboard", 2024-09-21T03:56:33.754 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-09-21T03:56:33.754 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.754 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-09-21T03:56:33.754 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2024-09-21T03:56:33.754 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.754 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.754 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.754 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-09-21T03:56:33.754 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.754 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.754 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.754 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.754 INFO:teuthology.orchestra.run.smithi039.stdout: "container_image_alertmanager": { 2024-09-21T03:56:33.754 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "container_image_alertmanager", 2024-09-21T03:56:33.755 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-09-21T03:56:33.755 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.755 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-09-21T03:56:33.755 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.27.0", 2024-09-21T03:56:33.755 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.755 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.755 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.755 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Prometheus container image", 2024-09-21T03:56:33.755 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.755 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.755 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.755 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.755 INFO:teuthology.orchestra.run.smithi039.stdout: "container_image_base": { 2024-09-21T03:56:33.755 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "container_image_base", 2024-09-21T03:56:33.756 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-09-21T03:56:33.756 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.756 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:33.756 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "quay.io/ceph/ceph", 2024-09-21T03:56:33.756 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.756 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.756 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.756 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Container image name, without the tag", 2024-09-21T03:56:33.756 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.756 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.756 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.756 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.756 INFO:teuthology.orchestra.run.smithi039.stdout: "container_image_elasticsearch": { 2024-09-21T03:56:33.756 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "container_image_elasticsearch", 2024-09-21T03:56:33.757 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-09-21T03:56:33.757 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.757 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-09-21T03:56:33.757 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-09-21T03:56:33.757 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.757 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.757 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.757 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "elasticsearch container image", 2024-09-21T03:56:33.757 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.757 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.757 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.757 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.757 INFO:teuthology.orchestra.run.smithi039.stdout: "container_image_grafana": { 2024-09-21T03:56:33.757 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "container_image_grafana", 2024-09-21T03:56:33.758 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-09-21T03:56:33.758 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.758 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-09-21T03:56:33.758 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "quay.io/ceph/grafana:10.4.8", 2024-09-21T03:56:33.758 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.758 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.758 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.758 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Prometheus container image", 2024-09-21T03:56:33.758 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.758 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.758 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.758 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.758 INFO:teuthology.orchestra.run.smithi039.stdout: "container_image_haproxy": { 2024-09-21T03:56:33.759 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "container_image_haproxy", 2024-09-21T03:56:33.759 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-09-21T03:56:33.759 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.759 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-09-21T03:56:33.759 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-09-21T03:56:33.759 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.759 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.759 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.759 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "HAproxy container image", 2024-09-21T03:56:33.759 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.759 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.759 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.759 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.759 INFO:teuthology.orchestra.run.smithi039.stdout: "container_image_jaeger_agent": { 2024-09-21T03:56:33.760 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "container_image_jaeger_agent", 2024-09-21T03:56:33.760 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-09-21T03:56:33.760 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.760 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-09-21T03:56:33.760 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-09-21T03:56:33.760 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.760 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.760 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.760 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Jaeger agent container image", 2024-09-21T03:56:33.760 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.760 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.760 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.760 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.760 INFO:teuthology.orchestra.run.smithi039.stdout: "container_image_jaeger_collector": { 2024-09-21T03:56:33.760 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "container_image_jaeger_collector", 2024-09-21T03:56:33.761 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-09-21T03:56:33.761 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.761 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-09-21T03:56:33.761 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-09-21T03:56:33.761 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.761 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.761 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.761 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Jaeger collector container image", 2024-09-21T03:56:33.761 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.761 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.761 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.761 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.761 INFO:teuthology.orchestra.run.smithi039.stdout: "container_image_jaeger_query": { 2024-09-21T03:56:33.761 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "container_image_jaeger_query", 2024-09-21T03:56:33.761 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-09-21T03:56:33.762 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.762 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-09-21T03:56:33.762 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-09-21T03:56:33.762 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.762 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.762 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.762 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Jaeger query container image", 2024-09-21T03:56:33.762 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.762 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.762 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.762 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.762 INFO:teuthology.orchestra.run.smithi039.stdout: "container_image_keepalived": { 2024-09-21T03:56:33.762 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "container_image_keepalived", 2024-09-21T03:56:33.763 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-09-21T03:56:33.763 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.763 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-09-21T03:56:33.763 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-09-21T03:56:33.763 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.763 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.763 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.763 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Keepalived container image", 2024-09-21T03:56:33.763 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.763 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.763 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.763 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.763 INFO:teuthology.orchestra.run.smithi039.stdout: "container_image_loki": { 2024-09-21T03:56:33.763 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "container_image_loki", 2024-09-21T03:56:33.764 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-09-21T03:56:33.764 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.764 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-09-21T03:56:33.764 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "docker.io/grafana/loki:3.0.0", 2024-09-21T03:56:33.764 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.764 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.764 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.764 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Loki container image", 2024-09-21T03:56:33.764 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.764 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.764 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.764 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.764 INFO:teuthology.orchestra.run.smithi039.stdout: "container_image_nginx": { 2024-09-21T03:56:33.764 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "container_image_nginx", 2024-09-21T03:56:33.764 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-09-21T03:56:33.765 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.765 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-09-21T03:56:33.765 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "quay.io/ceph/nginx:sclorg-nginx-126", 2024-09-21T03:56:33.765 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.765 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.765 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.765 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Nginx container image", 2024-09-21T03:56:33.765 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.765 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.765 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.765 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.765 INFO:teuthology.orchestra.run.smithi039.stdout: "container_image_node_exporter": { 2024-09-21T03:56:33.765 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "container_image_node_exporter", 2024-09-21T03:56:33.765 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-09-21T03:56:33.766 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.766 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-09-21T03:56:33.766 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.7.0", 2024-09-21T03:56:33.766 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.766 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.766 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.766 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Prometheus container image", 2024-09-21T03:56:33.766 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.766 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.766 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.766 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.766 INFO:teuthology.orchestra.run.smithi039.stdout: "container_image_nvmeof": { 2024-09-21T03:56:33.766 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "container_image_nvmeof", 2024-09-21T03:56:33.766 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-09-21T03:56:33.767 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.767 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-09-21T03:56:33.767 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.17", 2024-09-21T03:56:33.767 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.767 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.767 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.767 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Nvme-of container image", 2024-09-21T03:56:33.767 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.767 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.767 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.767 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.767 INFO:teuthology.orchestra.run.smithi039.stdout: "container_image_oauth2_proxy": { 2024-09-21T03:56:33.767 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "container_image_oauth2_proxy", 2024-09-21T03:56:33.767 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-09-21T03:56:33.768 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.768 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-09-21T03:56:33.768 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "quay.io/oauth2-proxy/oauth2-proxy:v7.6.0", 2024-09-21T03:56:33.768 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.768 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.768 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.768 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "oauth2-proxy container image", 2024-09-21T03:56:33.768 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.768 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.768 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.768 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.769 INFO:teuthology.orchestra.run.smithi039.stdout: "container_image_prometheus": { 2024-09-21T03:56:33.769 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "container_image_prometheus", 2024-09-21T03:56:33.769 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-09-21T03:56:33.769 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.769 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-09-21T03:56:33.769 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "quay.io/prometheus/prometheus:v2.51.0", 2024-09-21T03:56:33.769 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.769 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.769 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.769 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Prometheus container image", 2024-09-21T03:56:33.769 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.769 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.769 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.770 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.770 INFO:teuthology.orchestra.run.smithi039.stdout: "container_image_promtail": { 2024-09-21T03:56:33.770 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "container_image_promtail", 2024-09-21T03:56:33.770 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-09-21T03:56:33.770 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.770 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-09-21T03:56:33.770 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "docker.io/grafana/promtail:3.0.0", 2024-09-21T03:56:33.770 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.770 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.770 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.770 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Promtail container image", 2024-09-21T03:56:33.770 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.770 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.770 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.771 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.771 INFO:teuthology.orchestra.run.smithi039.stdout: "container_image_samba": { 2024-09-21T03:56:33.771 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "container_image_samba", 2024-09-21T03:56:33.771 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-09-21T03:56:33.771 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.771 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-09-21T03:56:33.771 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-09-21T03:56:33.771 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.771 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.771 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.771 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Samba/SMB container image", 2024-09-21T03:56:33.771 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.771 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.771 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.772 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.772 INFO:teuthology.orchestra.run.smithi039.stdout: "container_image_samba_metrics": { 2024-09-21T03:56:33.772 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "container_image_samba_metrics", 2024-09-21T03:56:33.772 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-09-21T03:56:33.772 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.772 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-09-21T03:56:33.772 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "quay.io/samba.org/samba-metrics:latest", 2024-09-21T03:56:33.772 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.772 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.772 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.772 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Samba/SMB metrics exporter container image", 2024-09-21T03:56:33.772 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.772 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.772 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.772 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.773 INFO:teuthology.orchestra.run.smithi039.stdout: "container_image_snmp_gateway": { 2024-09-21T03:56:33.773 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "container_image_snmp_gateway", 2024-09-21T03:56:33.773 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-09-21T03:56:33.773 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.773 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-09-21T03:56:33.773 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-09-21T03:56:33.773 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.773 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.773 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.773 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "SNMP Gateway container image", 2024-09-21T03:56:33.773 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.773 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.773 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.773 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.774 INFO:teuthology.orchestra.run.smithi039.stdout: "container_init": { 2024-09-21T03:56:33.774 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "container_init", 2024-09-21T03:56:33.774 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-09-21T03:56:33.774 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.774 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-09-21T03:56:33.774 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2024-09-21T03:56:33.774 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.774 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.774 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.774 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Run podman/docker with `--init`", 2024-09-21T03:56:33.774 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.774 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.774 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.774 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.775 INFO:teuthology.orchestra.run.smithi039.stdout: "daemon_cache_timeout": { 2024-09-21T03:56:33.775 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "daemon_cache_timeout", 2024-09-21T03:56:33.775 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "secs", 2024-09-21T03:56:33.775 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.775 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-09-21T03:56:33.775 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "600", 2024-09-21T03:56:33.775 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.775 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.775 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.775 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-09-21T03:56:33.775 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.775 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.775 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.775 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.776 INFO:teuthology.orchestra.run.smithi039.stdout: "default_cephadm_command_timeout": { 2024-09-21T03:56:33.776 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "default_cephadm_command_timeout", 2024-09-21T03:56:33.776 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-09-21T03:56:33.776 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.776 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-09-21T03:56:33.776 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "900", 2024-09-21T03:56:33.776 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.776 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.776 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.776 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-09-21T03:56:33.776 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.776 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.776 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.777 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.777 INFO:teuthology.orchestra.run.smithi039.stdout: "default_registry": { 2024-09-21T03:56:33.777 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "default_registry", 2024-09-21T03:56:33.777 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-09-21T03:56:33.777 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.777 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-09-21T03:56:33.777 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "docker.io", 2024-09-21T03:56:33.777 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.777 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.777 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.777 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-09-21T03:56:33.777 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.777 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.777 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.778 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.778 INFO:teuthology.orchestra.run.smithi039.stdout: "device_cache_timeout": { 2024-09-21T03:56:33.778 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "device_cache_timeout", 2024-09-21T03:56:33.778 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "secs", 2024-09-21T03:56:33.778 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.778 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-09-21T03:56:33.778 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "1800", 2024-09-21T03:56:33.778 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.778 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.778 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.778 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "seconds to cache device inventory", 2024-09-21T03:56:33.778 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.778 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.779 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.779 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.779 INFO:teuthology.orchestra.run.smithi039.stdout: "device_enhanced_scan": { 2024-09-21T03:56:33.779 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "device_enhanced_scan", 2024-09-21T03:56:33.779 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-09-21T03:56:33.779 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.779 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-09-21T03:56:33.779 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-09-21T03:56:33.779 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.779 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.779 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.779 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Use libstoragemgmt during device scans", 2024-09-21T03:56:33.779 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.779 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.779 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.780 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.780 INFO:teuthology.orchestra.run.smithi039.stdout: "facts_cache_timeout": { 2024-09-21T03:56:33.780 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "facts_cache_timeout", 2024-09-21T03:56:33.780 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "secs", 2024-09-21T03:56:33.780 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.780 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-09-21T03:56:33.780 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "60", 2024-09-21T03:56:33.780 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.780 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.780 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.780 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "seconds to cache host facts data", 2024-09-21T03:56:33.780 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.780 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.781 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.781 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.781 INFO:teuthology.orchestra.run.smithi039.stdout: "grafana_dashboards_path": { 2024-09-21T03:56:33.781 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "grafana_dashboards_path", 2024-09-21T03:56:33.781 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-09-21T03:56:33.781 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.781 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-09-21T03:56:33.781 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-09-21T03:56:33.781 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.781 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.781 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.781 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-09-21T03:56:33.781 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.781 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.782 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.782 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.782 INFO:teuthology.orchestra.run.smithi039.stdout: "host_check_interval": { 2024-09-21T03:56:33.782 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "host_check_interval", 2024-09-21T03:56:33.782 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "secs", 2024-09-21T03:56:33.782 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.782 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-09-21T03:56:33.782 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "600", 2024-09-21T03:56:33.782 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.782 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.782 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.782 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "how frequently to perform a host check", 2024-09-21T03:56:33.782 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.782 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.783 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.783 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.783 INFO:teuthology.orchestra.run.smithi039.stdout: "hw_monitoring": { 2024-09-21T03:56:33.783 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "hw_monitoring", 2024-09-21T03:56:33.783 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-09-21T03:56:33.783 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.783 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-09-21T03:56:33.783 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-09-21T03:56:33.783 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.783 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.783 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.783 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-09-21T03:56:33.783 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.784 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.784 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.784 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.784 INFO:teuthology.orchestra.run.smithi039.stdout: "inventory_list_all": { 2024-09-21T03:56:33.784 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "inventory_list_all", 2024-09-21T03:56:33.784 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-09-21T03:56:33.784 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.784 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-09-21T03:56:33.784 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-09-21T03:56:33.784 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.784 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.784 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.784 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-09-21T03:56:33.784 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.785 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.785 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.785 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.785 INFO:teuthology.orchestra.run.smithi039.stdout: "log_level": { 2024-09-21T03:56:33.785 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_level", 2024-09-21T03:56:33.785 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-09-21T03:56:33.785 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.785 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:33.785 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-09-21T03:56:33.785 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.785 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.785 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-09-21T03:56:33.785 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-09-21T03:56:33.786 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-09-21T03:56:33.786 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-09-21T03:56:33.786 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-09-21T03:56:33.786 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-09-21T03:56:33.786 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-09-21T03:56:33.786 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-09-21T03:56:33.786 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:33.786 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.786 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.786 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.786 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.786 INFO:teuthology.orchestra.run.smithi039.stdout: "log_refresh_metadata": { 2024-09-21T03:56:33.786 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_refresh_metadata", 2024-09-21T03:56:33.786 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-09-21T03:56:33.787 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.787 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-09-21T03:56:33.787 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-09-21T03:56:33.787 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.787 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.787 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.787 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-09-21T03:56:33.787 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.787 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.787 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.787 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.787 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster": { 2024-09-21T03:56:33.787 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster", 2024-09-21T03:56:33.787 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-09-21T03:56:33.787 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.788 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-09-21T03:56:33.788 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2024-09-21T03:56:33.788 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.788 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.788 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.788 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-09-21T03:56:33.788 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.788 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.788 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.788 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.788 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster_level": { 2024-09-21T03:56:33.788 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster_level", 2024-09-21T03:56:33.788 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-09-21T03:56:33.788 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.788 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:33.788 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "info", 2024-09-21T03:56:33.789 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.789 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.789 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-09-21T03:56:33.789 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-09-21T03:56:33.789 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-09-21T03:56:33.789 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-09-21T03:56:33.789 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-09-21T03:56:33.789 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-09-21T03:56:33.789 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-09-21T03:56:33.789 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-09-21T03:56:33.789 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:33.789 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.789 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.789 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.789 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.790 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_file": { 2024-09-21T03:56:33.790 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_file", 2024-09-21T03:56:33.790 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-09-21T03:56:33.790 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.790 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:33.790 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-09-21T03:56:33.790 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.790 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.790 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.790 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:33.790 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.790 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.790 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.790 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.791 INFO:teuthology.orchestra.run.smithi039.stdout: "manage_etc_ceph_ceph_conf": { 2024-09-21T03:56:33.791 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-09-21T03:56:33.791 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-09-21T03:56:33.791 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.791 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-09-21T03:56:33.791 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-09-21T03:56:33.791 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.791 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.791 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.791 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-09-21T03:56:33.791 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.791 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.791 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.791 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.791 INFO:teuthology.orchestra.run.smithi039.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-09-21T03:56:33.792 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-09-21T03:56:33.792 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-09-21T03:56:33.792 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.792 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-09-21T03:56:33.792 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "*", 2024-09-21T03:56:33.792 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.792 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.792 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.792 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-09-21T03:56:33.792 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.792 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.792 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.792 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.792 INFO:teuthology.orchestra.run.smithi039.stdout: "max_count_per_host": { 2024-09-21T03:56:33.792 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "max_count_per_host", 2024-09-21T03:56:33.793 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-09-21T03:56:33.793 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.793 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-09-21T03:56:33.793 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "10", 2024-09-21T03:56:33.793 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.793 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.793 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.793 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "max number of daemons per service per host", 2024-09-21T03:56:33.793 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.793 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.793 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.793 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.793 INFO:teuthology.orchestra.run.smithi039.stdout: "max_osd_draining_count": { 2024-09-21T03:56:33.793 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "max_osd_draining_count", 2024-09-21T03:56:33.793 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-09-21T03:56:33.794 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.794 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-09-21T03:56:33.794 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "10", 2024-09-21T03:56:33.794 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.794 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.794 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.794 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-09-21T03:56:33.794 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.794 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.794 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.794 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.794 INFO:teuthology.orchestra.run.smithi039.stdout: "migration_current": { 2024-09-21T03:56:33.794 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "migration_current", 2024-09-21T03:56:33.794 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-09-21T03:56:33.794 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.795 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-09-21T03:56:33.795 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-09-21T03:56:33.795 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.795 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.795 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.795 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "internal - do not modify", 2024-09-21T03:56:33.795 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.795 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.795 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.795 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.795 INFO:teuthology.orchestra.run.smithi039.stdout: "mode": { 2024-09-21T03:56:33.795 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "mode", 2024-09-21T03:56:33.795 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-09-21T03:56:33.795 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.795 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-09-21T03:56:33.796 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "root", 2024-09-21T03:56:33.796 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.796 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.796 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-09-21T03:56:33.796 INFO:teuthology.orchestra.run.smithi039.stdout: "cephadm-package", 2024-09-21T03:56:33.796 INFO:teuthology.orchestra.run.smithi039.stdout: "root" 2024-09-21T03:56:33.796 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-09-21T03:56:33.796 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "mode for remote execution of cephadm", 2024-09-21T03:56:33.796 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.796 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.796 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.796 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.796 INFO:teuthology.orchestra.run.smithi039.stdout: "oob_default_addr": { 2024-09-21T03:56:33.796 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "oob_default_addr", 2024-09-21T03:56:33.797 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-09-21T03:56:33.797 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.797 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-09-21T03:56:33.797 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "169.254.1.1", 2024-09-21T03:56:33.797 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.797 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.797 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.797 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Default address for RedFish API (oob management).", 2024-09-21T03:56:33.797 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.797 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.797 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.797 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.797 INFO:teuthology.orchestra.run.smithi039.stdout: "prometheus_alerts_path": { 2024-09-21T03:56:33.797 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "prometheus_alerts_path", 2024-09-21T03:56:33.798 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-09-21T03:56:33.798 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.798 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-09-21T03:56:33.798 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-09-21T03:56:33.798 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.798 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.798 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.798 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-09-21T03:56:33.798 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.798 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.798 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.798 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.798 INFO:teuthology.orchestra.run.smithi039.stdout: "registry_insecure": { 2024-09-21T03:56:33.798 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "registry_insecure", 2024-09-21T03:56:33.799 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-09-21T03:56:33.799 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.799 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-09-21T03:56:33.799 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-09-21T03:56:33.799 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.799 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.799 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.799 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-09-21T03:56:33.799 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.799 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.799 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.799 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.799 INFO:teuthology.orchestra.run.smithi039.stdout: "registry_password": { 2024-09-21T03:56:33.799 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "registry_password", 2024-09-21T03:56:33.800 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-09-21T03:56:33.800 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.800 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-09-21T03:56:33.800 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-09-21T03:56:33.800 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.800 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.800 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.800 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-09-21T03:56:33.800 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.800 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.800 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.800 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.800 INFO:teuthology.orchestra.run.smithi039.stdout: "registry_url": { 2024-09-21T03:56:33.800 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "registry_url", 2024-09-21T03:56:33.801 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-09-21T03:56:33.801 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.801 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-09-21T03:56:33.801 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-09-21T03:56:33.801 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.801 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.801 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.801 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-09-21T03:56:33.801 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.801 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.801 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.801 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.801 INFO:teuthology.orchestra.run.smithi039.stdout: "registry_username": { 2024-09-21T03:56:33.802 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "registry_username", 2024-09-21T03:56:33.802 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-09-21T03:56:33.802 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.802 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-09-21T03:56:33.802 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-09-21T03:56:33.802 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.802 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.802 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.802 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-09-21T03:56:33.802 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.802 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.802 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.802 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.803 INFO:teuthology.orchestra.run.smithi039.stdout: "secure_monitoring_stack": { 2024-09-21T03:56:33.803 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "secure_monitoring_stack", 2024-09-21T03:56:33.803 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-09-21T03:56:33.803 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.803 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-09-21T03:56:33.803 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-09-21T03:56:33.803 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.803 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.803 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.803 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-09-21T03:56:33.803 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.803 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.803 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.803 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.804 INFO:teuthology.orchestra.run.smithi039.stdout: "service_discovery_port": { 2024-09-21T03:56:33.804 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "service_discovery_port", 2024-09-21T03:56:33.804 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-09-21T03:56:33.804 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.804 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-09-21T03:56:33.804 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "8765", 2024-09-21T03:56:33.804 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.804 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.804 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.804 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "cephadm service discovery port", 2024-09-21T03:56:33.804 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.804 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.804 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.804 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.805 INFO:teuthology.orchestra.run.smithi039.stdout: "sqlite3_killpoint": { 2024-09-21T03:56:33.805 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:56:33.805 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-09-21T03:56:33.805 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "dev", 2024-09-21T03:56:33.805 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:33.805 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "0", 2024-09-21T03:56:33.805 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.805 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.805 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.805 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:33.805 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.805 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.805 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.805 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.806 INFO:teuthology.orchestra.run.smithi039.stdout: "ssh_config_file": { 2024-09-21T03:56:33.806 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "ssh_config_file", 2024-09-21T03:56:33.806 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-09-21T03:56:33.806 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.806 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-09-21T03:56:33.806 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-09-21T03:56:33.806 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.806 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.806 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.806 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-09-21T03:56:33.806 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.806 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.806 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.806 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.807 INFO:teuthology.orchestra.run.smithi039.stdout: "ssh_keepalive_count_max": { 2024-09-21T03:56:33.807 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "ssh_keepalive_count_max", 2024-09-21T03:56:33.807 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-09-21T03:56:33.807 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.807 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-09-21T03:56:33.807 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "3", 2024-09-21T03:56:33.807 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.807 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.807 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.807 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "How many times ssh connections can fail liveness checks before the host is marked offline", 2024-09-21T03:56:33.807 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.807 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.807 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.807 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.807 INFO:teuthology.orchestra.run.smithi039.stdout: "ssh_keepalive_interval": { 2024-09-21T03:56:33.808 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "ssh_keepalive_interval", 2024-09-21T03:56:33.808 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-09-21T03:56:33.808 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.808 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-09-21T03:56:33.808 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "7", 2024-09-21T03:56:33.808 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.808 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.808 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.808 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "How often ssh connections are checked for liveness", 2024-09-21T03:56:33.808 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.808 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.808 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.808 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.809 INFO:teuthology.orchestra.run.smithi039.stdout: "use_agent": { 2024-09-21T03:56:33.809 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "use_agent", 2024-09-21T03:56:33.809 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-09-21T03:56:33.809 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.809 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-09-21T03:56:33.809 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-09-21T03:56:33.809 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.809 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.809 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.809 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-09-21T03:56:33.809 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.809 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.809 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.809 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.810 INFO:teuthology.orchestra.run.smithi039.stdout: "use_repo_digest": { 2024-09-21T03:56:33.810 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "use_repo_digest", 2024-09-21T03:56:33.810 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-09-21T03:56:33.810 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.810 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-09-21T03:56:33.810 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2024-09-21T03:56:33.810 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.810 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.810 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.810 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-09-21T03:56:33.810 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.810 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.810 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.810 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.811 INFO:teuthology.orchestra.run.smithi039.stdout: "warn_on_failed_host_check": { 2024-09-21T03:56:33.811 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "warn_on_failed_host_check", 2024-09-21T03:56:33.811 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-09-21T03:56:33.811 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.811 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-09-21T03:56:33.811 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2024-09-21T03:56:33.811 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.811 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.811 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.811 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "raise a health warning if the host check fails", 2024-09-21T03:56:33.811 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.811 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.811 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.811 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.812 INFO:teuthology.orchestra.run.smithi039.stdout: "warn_on_stray_daemons": { 2024-09-21T03:56:33.812 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "warn_on_stray_daemons", 2024-09-21T03:56:33.812 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-09-21T03:56:33.812 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.812 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-09-21T03:56:33.812 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2024-09-21T03:56:33.812 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.812 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.812 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.812 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-09-21T03:56:33.812 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.812 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.812 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.812 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.812 INFO:teuthology.orchestra.run.smithi039.stdout: "warn_on_stray_hosts": { 2024-09-21T03:56:33.813 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "warn_on_stray_hosts", 2024-09-21T03:56:33.813 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-09-21T03:56:33.813 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.813 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-09-21T03:56:33.813 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2024-09-21T03:56:33.813 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.813 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.813 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.813 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-09-21T03:56:33.813 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.813 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.813 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.813 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-21T03:56:33.813 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-21T03:56:33.813 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.814 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-21T03:56:33.814 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "crash", 2024-09-21T03:56:33.814 INFO:teuthology.orchestra.run.smithi039.stdout: "can_run": true, 2024-09-21T03:56:33.814 INFO:teuthology.orchestra.run.smithi039.stdout: "error_string": "", 2024-09-21T03:56:33.814 INFO:teuthology.orchestra.run.smithi039.stdout: "module_options": { 2024-09-21T03:56:33.814 INFO:teuthology.orchestra.run.smithi039.stdout: "log_level": { 2024-09-21T03:56:33.814 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_level", 2024-09-21T03:56:33.814 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-09-21T03:56:33.814 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.814 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:33.814 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-09-21T03:56:33.814 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.814 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.814 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-09-21T03:56:33.814 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-09-21T03:56:33.815 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-09-21T03:56:33.815 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-09-21T03:56:33.815 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-09-21T03:56:33.815 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-09-21T03:56:33.815 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-09-21T03:56:33.815 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-09-21T03:56:33.815 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:33.815 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.815 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.815 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.815 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.815 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster": { 2024-09-21T03:56:33.815 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster", 2024-09-21T03:56:33.816 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-09-21T03:56:33.816 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.816 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:33.816 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-09-21T03:56:33.816 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.816 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.816 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.816 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:33.816 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.816 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.816 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.816 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.816 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster_level": { 2024-09-21T03:56:33.816 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster_level", 2024-09-21T03:56:33.817 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-09-21T03:56:33.817 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.817 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:33.817 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "info", 2024-09-21T03:56:33.817 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.817 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.817 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-09-21T03:56:33.817 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-09-21T03:56:33.817 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-09-21T03:56:33.817 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-09-21T03:56:33.817 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-09-21T03:56:33.817 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-09-21T03:56:33.817 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-09-21T03:56:33.817 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-09-21T03:56:33.817 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:33.818 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.818 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.818 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.818 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.818 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_file": { 2024-09-21T03:56:33.818 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_file", 2024-09-21T03:56:33.818 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-09-21T03:56:33.818 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.818 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:33.818 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-09-21T03:56:33.818 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.818 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.818 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.818 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:33.818 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.819 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.819 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.819 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.819 INFO:teuthology.orchestra.run.smithi039.stdout: "retain_interval": { 2024-09-21T03:56:33.819 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "retain_interval", 2024-09-21T03:56:33.819 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "secs", 2024-09-21T03:56:33.819 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.819 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:33.819 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "31536000", 2024-09-21T03:56:33.819 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.819 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.819 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.819 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "how long to retain crashes before pruning them", 2024-09-21T03:56:33.819 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.819 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.820 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.820 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.820 INFO:teuthology.orchestra.run.smithi039.stdout: "sqlite3_killpoint": { 2024-09-21T03:56:33.820 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:56:33.820 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-09-21T03:56:33.820 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "dev", 2024-09-21T03:56:33.820 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:33.820 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "0", 2024-09-21T03:56:33.820 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.820 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.820 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.820 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:33.820 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.820 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.820 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.821 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.821 INFO:teuthology.orchestra.run.smithi039.stdout: "warn_recent_interval": { 2024-09-21T03:56:33.821 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "warn_recent_interval", 2024-09-21T03:56:33.821 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "secs", 2024-09-21T03:56:33.821 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.821 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:33.821 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "1209600", 2024-09-21T03:56:33.821 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.821 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.821 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.821 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "time interval in which to warn about recent crashes", 2024-09-21T03:56:33.821 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.821 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.821 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.821 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-21T03:56:33.822 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-21T03:56:33.822 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.822 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-21T03:56:33.822 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "dashboard", 2024-09-21T03:56:33.822 INFO:teuthology.orchestra.run.smithi039.stdout: "can_run": true, 2024-09-21T03:56:33.822 INFO:teuthology.orchestra.run.smithi039.stdout: "error_string": "", 2024-09-21T03:56:33.822 INFO:teuthology.orchestra.run.smithi039.stdout: "module_options": { 2024-09-21T03:56:33.822 INFO:teuthology.orchestra.run.smithi039.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-09-21T03:56:33.822 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-09-21T03:56:33.822 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-09-21T03:56:33.822 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.822 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-09-21T03:56:33.823 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "10", 2024-09-21T03:56:33.823 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.823 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.823 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.823 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:33.823 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.823 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.823 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.823 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.823 INFO:teuthology.orchestra.run.smithi039.stdout: "ALERTMANAGER_API_HOST": { 2024-09-21T03:56:33.823 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "ALERTMANAGER_API_HOST", 2024-09-21T03:56:33.823 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-09-21T03:56:33.823 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.823 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-09-21T03:56:33.824 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-09-21T03:56:33.824 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.824 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.824 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.824 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:33.824 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.824 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.824 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.824 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.824 INFO:teuthology.orchestra.run.smithi039.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-09-21T03:56:33.824 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-09-21T03:56:33.824 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-09-21T03:56:33.824 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.824 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-09-21T03:56:33.824 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2024-09-21T03:56:33.825 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.825 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.825 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.825 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:33.825 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.825 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.825 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.825 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.825 INFO:teuthology.orchestra.run.smithi039.stdout: "AUDIT_API_ENABLED": { 2024-09-21T03:56:33.825 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "AUDIT_API_ENABLED", 2024-09-21T03:56:33.825 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-09-21T03:56:33.825 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.825 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-09-21T03:56:33.825 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-09-21T03:56:33.825 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.826 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.826 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.826 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:33.826 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.826 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.826 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.826 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.826 INFO:teuthology.orchestra.run.smithi039.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-09-21T03:56:33.826 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-09-21T03:56:33.826 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-09-21T03:56:33.826 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.826 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-09-21T03:56:33.826 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2024-09-21T03:56:33.826 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.826 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.827 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.827 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:33.827 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.827 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.827 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.827 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.827 INFO:teuthology.orchestra.run.smithi039.stdout: "ENABLE_BROWSABLE_API": { 2024-09-21T03:56:33.827 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "ENABLE_BROWSABLE_API", 2024-09-21T03:56:33.827 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-09-21T03:56:33.827 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.827 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-09-21T03:56:33.827 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2024-09-21T03:56:33.827 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.827 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.827 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.828 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:33.828 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.828 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.828 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.828 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.828 INFO:teuthology.orchestra.run.smithi039.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-09-21T03:56:33.828 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-09-21T03:56:33.828 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-09-21T03:56:33.828 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.828 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-09-21T03:56:33.828 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2024-09-21T03:56:33.828 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.829 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.829 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.829 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:33.829 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.829 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.829 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.829 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.829 INFO:teuthology.orchestra.run.smithi039.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-09-21T03:56:33.829 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-09-21T03:56:33.829 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-09-21T03:56:33.829 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.829 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-09-21T03:56:33.829 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2024-09-21T03:56:33.829 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.830 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.830 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.830 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:33.830 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.830 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.830 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.830 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.830 INFO:teuthology.orchestra.run.smithi039.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-09-21T03:56:33.830 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-09-21T03:56:33.830 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-09-21T03:56:33.830 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.830 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-09-21T03:56:33.830 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2024-09-21T03:56:33.831 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.831 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.831 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.831 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:33.831 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.831 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.831 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.831 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.831 INFO:teuthology.orchestra.run.smithi039.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-09-21T03:56:33.831 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-09-21T03:56:33.831 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-09-21T03:56:33.831 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.831 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-09-21T03:56:33.831 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2024-09-21T03:56:33.832 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.832 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.832 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.832 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:33.832 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.832 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.832 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.832 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.832 INFO:teuthology.orchestra.run.smithi039.stdout: "FEATURE_TOGGLE_NFS": { 2024-09-21T03:56:33.832 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-09-21T03:56:33.832 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-09-21T03:56:33.832 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.832 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-09-21T03:56:33.832 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2024-09-21T03:56:33.833 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.833 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.833 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.833 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:33.833 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.833 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.833 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.833 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.833 INFO:teuthology.orchestra.run.smithi039.stdout: "FEATURE_TOGGLE_RBD": { 2024-09-21T03:56:33.833 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-09-21T03:56:33.833 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-09-21T03:56:33.833 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.833 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-09-21T03:56:33.833 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2024-09-21T03:56:33.834 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.834 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.834 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.834 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:33.834 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.834 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.834 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.834 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.834 INFO:teuthology.orchestra.run.smithi039.stdout: "FEATURE_TOGGLE_RGW": { 2024-09-21T03:56:33.834 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-09-21T03:56:33.834 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-09-21T03:56:33.834 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.834 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-09-21T03:56:33.834 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2024-09-21T03:56:33.835 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.835 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.835 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.835 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:33.835 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.835 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.835 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.835 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.835 INFO:teuthology.orchestra.run.smithi039.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-09-21T03:56:33.835 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-09-21T03:56:33.835 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-09-21T03:56:33.835 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.835 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-09-21T03:56:33.835 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-09-21T03:56:33.836 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.836 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.836 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.836 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:33.836 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.836 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.836 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.836 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.836 INFO:teuthology.orchestra.run.smithi039.stdout: "GRAFANA_API_PASSWORD": { 2024-09-21T03:56:33.836 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "GRAFANA_API_PASSWORD", 2024-09-21T03:56:33.836 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-09-21T03:56:33.836 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.836 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-09-21T03:56:33.836 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "admin", 2024-09-21T03:56:33.836 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.837 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.837 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.837 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:33.837 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.837 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.837 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.837 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.837 INFO:teuthology.orchestra.run.smithi039.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-09-21T03:56:33.837 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-09-21T03:56:33.837 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-09-21T03:56:33.837 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.837 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-09-21T03:56:33.837 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2024-09-21T03:56:33.837 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.837 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.838 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.838 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:33.838 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.838 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.838 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.838 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.838 INFO:teuthology.orchestra.run.smithi039.stdout: "GRAFANA_API_URL": { 2024-09-21T03:56:33.838 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "GRAFANA_API_URL", 2024-09-21T03:56:33.838 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-09-21T03:56:33.838 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.838 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-09-21T03:56:33.838 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-09-21T03:56:33.838 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.839 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.839 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.839 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:33.839 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.839 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.839 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.839 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.839 INFO:teuthology.orchestra.run.smithi039.stdout: "GRAFANA_API_USERNAME": { 2024-09-21T03:56:33.839 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "GRAFANA_API_USERNAME", 2024-09-21T03:56:33.839 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-09-21T03:56:33.839 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.839 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-09-21T03:56:33.839 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "admin", 2024-09-21T03:56:33.839 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.840 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.840 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.840 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:33.840 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.840 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.840 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.840 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.840 INFO:teuthology.orchestra.run.smithi039.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-09-21T03:56:33.840 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-09-21T03:56:33.840 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-09-21T03:56:33.840 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.840 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-09-21T03:56:33.840 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-09-21T03:56:33.840 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.840 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.841 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.841 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:33.841 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.841 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.841 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.841 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.841 INFO:teuthology.orchestra.run.smithi039.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-09-21T03:56:33.841 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-09-21T03:56:33.841 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-09-21T03:56:33.841 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.841 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-09-21T03:56:33.841 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-09-21T03:56:33.841 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.841 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.842 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.842 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:33.842 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.842 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.842 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.842 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.842 INFO:teuthology.orchestra.run.smithi039.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-09-21T03:56:33.842 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-09-21T03:56:33.842 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-09-21T03:56:33.842 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.842 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-09-21T03:56:33.842 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2024-09-21T03:56:33.842 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.842 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.843 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.843 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:33.843 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.843 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.843 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.843 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.843 INFO:teuthology.orchestra.run.smithi039.stdout: "ISSUE_TRACKER_API_KEY": { 2024-09-21T03:56:33.843 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-09-21T03:56:33.843 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-09-21T03:56:33.843 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.843 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-09-21T03:56:33.843 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-09-21T03:56:33.843 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.843 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.844 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.844 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:33.844 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.844 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.844 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.844 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.844 INFO:teuthology.orchestra.run.smithi039.stdout: "MANAGED_BY_CLUSTERS": { 2024-09-21T03:56:33.844 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "MANAGED_BY_CLUSTERS", 2024-09-21T03:56:33.844 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-09-21T03:56:33.844 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.844 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-09-21T03:56:33.844 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "[]", 2024-09-21T03:56:33.844 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.844 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.845 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.845 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:33.845 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.845 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.845 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.845 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.845 INFO:teuthology.orchestra.run.smithi039.stdout: "MULTICLUSTER_CONFIG": { 2024-09-21T03:56:33.845 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "MULTICLUSTER_CONFIG", 2024-09-21T03:56:33.845 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-09-21T03:56:33.845 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.845 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-09-21T03:56:33.845 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "{}", 2024-09-21T03:56:33.845 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.845 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.846 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.846 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:33.846 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.846 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.846 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.846 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.846 INFO:teuthology.orchestra.run.smithi039.stdout: "PROMETHEUS_API_HOST": { 2024-09-21T03:56:33.846 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "PROMETHEUS_API_HOST", 2024-09-21T03:56:33.846 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-09-21T03:56:33.846 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.846 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-09-21T03:56:33.846 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-09-21T03:56:33.846 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.847 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.847 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.847 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:33.847 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.847 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.847 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.847 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.847 INFO:teuthology.orchestra.run.smithi039.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-09-21T03:56:33.847 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-09-21T03:56:33.847 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-09-21T03:56:33.847 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.847 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-09-21T03:56:33.847 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2024-09-21T03:56:33.847 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.848 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.848 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.848 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:33.848 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.848 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.848 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.848 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.848 INFO:teuthology.orchestra.run.smithi039.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-09-21T03:56:33.848 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-09-21T03:56:33.848 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-09-21T03:56:33.848 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.848 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-09-21T03:56:33.848 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-09-21T03:56:33.848 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.848 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.849 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.849 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:33.849 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.849 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.849 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.849 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.849 INFO:teuthology.orchestra.run.smithi039.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-09-21T03:56:33.849 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-09-21T03:56:33.849 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-09-21T03:56:33.849 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.849 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-09-21T03:56:33.849 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-09-21T03:56:33.849 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.849 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.850 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.850 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:33.850 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.850 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.850 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.850 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.850 INFO:teuthology.orchestra.run.smithi039.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-09-21T03:56:33.850 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-09-21T03:56:33.850 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-09-21T03:56:33.850 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.850 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-09-21T03:56:33.850 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2024-09-21T03:56:33.850 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.850 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.851 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.851 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:33.851 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.851 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.851 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.851 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.851 INFO:teuthology.orchestra.run.smithi039.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-09-21T03:56:33.851 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-09-21T03:56:33.851 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-09-21T03:56:33.851 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.851 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-09-21T03:56:33.851 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2024-09-21T03:56:33.851 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.851 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.851 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.852 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:33.852 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.852 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.852 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.852 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.852 INFO:teuthology.orchestra.run.smithi039.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-09-21T03:56:33.852 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-09-21T03:56:33.852 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-09-21T03:56:33.852 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.852 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-09-21T03:56:33.852 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-09-21T03:56:33.852 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.852 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.852 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.852 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:33.853 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.853 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.853 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.853 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.853 INFO:teuthology.orchestra.run.smithi039.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-09-21T03:56:33.853 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-09-21T03:56:33.853 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-09-21T03:56:33.853 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.853 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-09-21T03:56:33.853 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-09-21T03:56:33.853 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.853 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.853 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.853 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:33.853 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.853 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.854 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.854 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.854 INFO:teuthology.orchestra.run.smithi039.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-09-21T03:56:33.854 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-09-21T03:56:33.854 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-09-21T03:56:33.854 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.854 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-09-21T03:56:33.854 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-09-21T03:56:33.854 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.854 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.854 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.854 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:33.854 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.854 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.854 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.855 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.855 INFO:teuthology.orchestra.run.smithi039.stdout: "PWD_POLICY_ENABLED": { 2024-09-21T03:56:33.855 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "PWD_POLICY_ENABLED", 2024-09-21T03:56:33.855 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-09-21T03:56:33.855 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.855 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-09-21T03:56:33.855 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2024-09-21T03:56:33.855 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.855 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.855 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.855 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:33.855 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.855 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.855 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.856 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.856 INFO:teuthology.orchestra.run.smithi039.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-09-21T03:56:33.856 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-09-21T03:56:33.856 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-09-21T03:56:33.856 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.856 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-09-21T03:56:33.856 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-09-21T03:56:33.856 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.856 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.856 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.856 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:33.856 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.856 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.856 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.856 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.857 INFO:teuthology.orchestra.run.smithi039.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-09-21T03:56:33.857 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-09-21T03:56:33.857 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-09-21T03:56:33.857 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.857 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-09-21T03:56:33.857 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "10", 2024-09-21T03:56:33.857 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.857 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.857 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.857 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:33.857 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.857 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.857 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.857 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.857 INFO:teuthology.orchestra.run.smithi039.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-09-21T03:56:33.858 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-09-21T03:56:33.858 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-09-21T03:56:33.858 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.858 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-09-21T03:56:33.858 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "8", 2024-09-21T03:56:33.858 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.858 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.858 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.858 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:33.858 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.858 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.858 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.858 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.859 INFO:teuthology.orchestra.run.smithi039.stdout: "REST_REQUESTS_TIMEOUT": { 2024-09-21T03:56:33.859 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-09-21T03:56:33.859 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-09-21T03:56:33.859 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.859 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-09-21T03:56:33.859 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "45", 2024-09-21T03:56:33.859 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.859 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.859 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.859 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:33.859 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.859 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.859 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.859 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.859 INFO:teuthology.orchestra.run.smithi039.stdout: "RGW_API_ACCESS_KEY": { 2024-09-21T03:56:33.860 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "RGW_API_ACCESS_KEY", 2024-09-21T03:56:33.860 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-09-21T03:56:33.860 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.860 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-09-21T03:56:33.860 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-09-21T03:56:33.860 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.860 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.860 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.860 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:33.860 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.860 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.860 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.860 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.860 INFO:teuthology.orchestra.run.smithi039.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-09-21T03:56:33.861 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-09-21T03:56:33.861 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-09-21T03:56:33.861 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.861 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-09-21T03:56:33.861 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "admin", 2024-09-21T03:56:33.861 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.861 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.861 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.861 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:33.861 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.861 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.861 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.861 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.861 INFO:teuthology.orchestra.run.smithi039.stdout: "RGW_API_SECRET_KEY": { 2024-09-21T03:56:33.862 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "RGW_API_SECRET_KEY", 2024-09-21T03:56:33.862 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-09-21T03:56:33.862 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.862 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-09-21T03:56:33.862 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-09-21T03:56:33.862 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.862 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.862 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.862 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:33.862 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.862 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.862 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.862 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.863 INFO:teuthology.orchestra.run.smithi039.stdout: "RGW_API_SSL_VERIFY": { 2024-09-21T03:56:33.863 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "RGW_API_SSL_VERIFY", 2024-09-21T03:56:33.863 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-09-21T03:56:33.863 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.863 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-09-21T03:56:33.863 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2024-09-21T03:56:33.863 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.863 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.863 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.863 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:33.863 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.863 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.863 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.863 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.864 INFO:teuthology.orchestra.run.smithi039.stdout: "UNSAFE_TLS_v1_2": { 2024-09-21T03:56:33.864 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "UNSAFE_TLS_v1_2", 2024-09-21T03:56:33.864 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-09-21T03:56:33.864 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.864 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-09-21T03:56:33.864 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-09-21T03:56:33.864 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.864 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.864 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.864 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:33.864 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.864 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.864 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.864 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.865 INFO:teuthology.orchestra.run.smithi039.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-09-21T03:56:33.865 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-09-21T03:56:33.865 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-09-21T03:56:33.865 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.865 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-09-21T03:56:33.865 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "0", 2024-09-21T03:56:33.865 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.865 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.865 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.865 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:33.865 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.865 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.865 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.865 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.865 INFO:teuthology.orchestra.run.smithi039.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-09-21T03:56:33.866 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-09-21T03:56:33.866 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-09-21T03:56:33.866 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.866 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-09-21T03:56:33.866 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "10", 2024-09-21T03:56:33.866 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.866 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.866 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.866 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:33.866 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.866 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.866 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.866 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.866 INFO:teuthology.orchestra.run.smithi039.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-09-21T03:56:33.867 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-09-21T03:56:33.867 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-09-21T03:56:33.867 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.867 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-09-21T03:56:33.867 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "5", 2024-09-21T03:56:33.867 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.867 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.867 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.867 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:33.867 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.867 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.867 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.867 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.867 INFO:teuthology.orchestra.run.smithi039.stdout: "cross_origin_url": { 2024-09-21T03:56:33.868 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "cross_origin_url", 2024-09-21T03:56:33.868 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-09-21T03:56:33.868 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.868 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-09-21T03:56:33.868 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-09-21T03:56:33.868 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.868 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.868 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.868 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:33.868 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.868 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.868 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.868 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.868 INFO:teuthology.orchestra.run.smithi039.stdout: "crt_file": { 2024-09-21T03:56:33.869 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "crt_file", 2024-09-21T03:56:33.869 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-09-21T03:56:33.869 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.869 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-09-21T03:56:33.869 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-09-21T03:56:33.869 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.869 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.869 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.869 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:33.869 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.869 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.869 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.869 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.869 INFO:teuthology.orchestra.run.smithi039.stdout: "debug": { 2024-09-21T03:56:33.870 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "debug", 2024-09-21T03:56:33.870 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-09-21T03:56:33.870 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.870 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-09-21T03:56:33.870 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-09-21T03:56:33.870 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.870 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.870 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.870 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Enable/disable debug options", 2024-09-21T03:56:33.870 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.870 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.870 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.870 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.870 INFO:teuthology.orchestra.run.smithi039.stdout: "jwt_token_ttl": { 2024-09-21T03:56:33.871 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "jwt_token_ttl", 2024-09-21T03:56:33.871 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-09-21T03:56:33.871 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.871 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-09-21T03:56:33.871 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "28800", 2024-09-21T03:56:33.871 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.871 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.871 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.871 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:33.871 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.871 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.871 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.871 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.871 INFO:teuthology.orchestra.run.smithi039.stdout: "key_file": { 2024-09-21T03:56:33.871 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "key_file", 2024-09-21T03:56:33.872 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-09-21T03:56:33.872 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.872 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-09-21T03:56:33.872 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-09-21T03:56:33.872 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.872 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.872 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.872 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:33.872 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.872 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.872 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.872 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.872 INFO:teuthology.orchestra.run.smithi039.stdout: "log_level": { 2024-09-21T03:56:33.872 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_level", 2024-09-21T03:56:33.872 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-09-21T03:56:33.873 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.873 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:33.873 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-09-21T03:56:33.873 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.873 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.873 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-09-21T03:56:33.873 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-09-21T03:56:33.873 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-09-21T03:56:33.873 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-09-21T03:56:33.873 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-09-21T03:56:33.873 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-09-21T03:56:33.873 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-09-21T03:56:33.873 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-09-21T03:56:33.873 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:33.873 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.874 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.874 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.874 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.874 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster": { 2024-09-21T03:56:33.874 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster", 2024-09-21T03:56:33.874 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-09-21T03:56:33.874 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.874 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:33.874 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-09-21T03:56:33.874 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.874 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.874 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.874 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:33.874 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.874 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.875 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.875 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.875 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster_level": { 2024-09-21T03:56:33.875 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster_level", 2024-09-21T03:56:33.875 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-09-21T03:56:33.875 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.875 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:33.875 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "info", 2024-09-21T03:56:33.875 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.875 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.875 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-09-21T03:56:33.875 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-09-21T03:56:33.875 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-09-21T03:56:33.875 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-09-21T03:56:33.875 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-09-21T03:56:33.876 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-09-21T03:56:33.876 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-09-21T03:56:33.876 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-09-21T03:56:33.876 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:33.876 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.876 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.876 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.876 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.876 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_file": { 2024-09-21T03:56:33.876 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_file", 2024-09-21T03:56:33.876 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-09-21T03:56:33.876 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.876 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:33.876 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-09-21T03:56:33.876 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.876 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.877 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.877 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:33.877 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.877 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.877 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.877 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.877 INFO:teuthology.orchestra.run.smithi039.stdout: "motd": { 2024-09-21T03:56:33.877 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "motd", 2024-09-21T03:56:33.877 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-09-21T03:56:33.877 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.877 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-09-21T03:56:33.877 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-09-21T03:56:33.877 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.877 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.877 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.878 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "The message of the day", 2024-09-21T03:56:33.878 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.878 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.878 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.878 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.878 INFO:teuthology.orchestra.run.smithi039.stdout: "redirect_resolve_ip_addr": { 2024-09-21T03:56:33.878 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "redirect_resolve_ip_addr", 2024-09-21T03:56:33.878 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-09-21T03:56:33.878 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.878 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-09-21T03:56:33.878 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-09-21T03:56:33.878 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.878 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.879 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.879 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:33.879 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.879 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.879 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.879 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.879 INFO:teuthology.orchestra.run.smithi039.stdout: "server_addr": { 2024-09-21T03:56:33.879 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "server_addr", 2024-09-21T03:56:33.879 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-09-21T03:56:33.879 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.879 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-09-21T03:56:33.879 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "::", 2024-09-21T03:56:33.879 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.879 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.879 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.880 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:33.880 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.880 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.880 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.880 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.880 INFO:teuthology.orchestra.run.smithi039.stdout: "server_port": { 2024-09-21T03:56:33.880 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "server_port", 2024-09-21T03:56:33.880 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-09-21T03:56:33.880 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.880 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-09-21T03:56:33.880 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "8080", 2024-09-21T03:56:33.880 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.880 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.880 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.881 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:33.881 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.881 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.881 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.881 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.881 INFO:teuthology.orchestra.run.smithi039.stdout: "sqlite3_killpoint": { 2024-09-21T03:56:33.881 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:56:33.881 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-09-21T03:56:33.881 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "dev", 2024-09-21T03:56:33.881 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:33.881 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "0", 2024-09-21T03:56:33.881 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.881 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.881 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.882 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:33.882 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.882 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.882 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.882 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.882 INFO:teuthology.orchestra.run.smithi039.stdout: "ssl": { 2024-09-21T03:56:33.882 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "ssl", 2024-09-21T03:56:33.882 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-09-21T03:56:33.882 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.882 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-09-21T03:56:33.882 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2024-09-21T03:56:33.882 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.882 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.882 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.883 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:33.883 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.883 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.883 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.883 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.883 INFO:teuthology.orchestra.run.smithi039.stdout: "ssl_server_port": { 2024-09-21T03:56:33.883 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "ssl_server_port", 2024-09-21T03:56:33.883 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-09-21T03:56:33.883 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.883 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-09-21T03:56:33.883 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "8443", 2024-09-21T03:56:33.883 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.883 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.883 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.884 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:33.884 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.884 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.884 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.884 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.884 INFO:teuthology.orchestra.run.smithi039.stdout: "sso_oauth2": { 2024-09-21T03:56:33.884 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "sso_oauth2", 2024-09-21T03:56:33.884 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-09-21T03:56:33.884 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.884 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-09-21T03:56:33.884 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-09-21T03:56:33.884 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.884 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.884 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.884 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:33.885 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.885 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.885 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.885 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.885 INFO:teuthology.orchestra.run.smithi039.stdout: "standby_behaviour": { 2024-09-21T03:56:33.885 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "standby_behaviour", 2024-09-21T03:56:33.885 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-09-21T03:56:33.885 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.885 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-09-21T03:56:33.885 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "redirect", 2024-09-21T03:56:33.885 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.885 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.885 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-09-21T03:56:33.885 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-09-21T03:56:33.886 INFO:teuthology.orchestra.run.smithi039.stdout: "redirect" 2024-09-21T03:56:33.886 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-09-21T03:56:33.886 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:33.886 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.886 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.886 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.886 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.886 INFO:teuthology.orchestra.run.smithi039.stdout: "standby_error_status_code": { 2024-09-21T03:56:33.886 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "standby_error_status_code", 2024-09-21T03:56:33.886 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-09-21T03:56:33.886 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.886 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-09-21T03:56:33.886 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "500", 2024-09-21T03:56:33.886 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "400", 2024-09-21T03:56:33.887 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "599", 2024-09-21T03:56:33.887 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.887 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:33.887 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.887 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.887 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.887 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.887 INFO:teuthology.orchestra.run.smithi039.stdout: "url_prefix": { 2024-09-21T03:56:33.887 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "url_prefix", 2024-09-21T03:56:33.887 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-09-21T03:56:33.887 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.887 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-09-21T03:56:33.887 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-09-21T03:56:33.887 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.887 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.888 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.888 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:33.888 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.888 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.888 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.888 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-21T03:56:33.888 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-21T03:56:33.888 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.888 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-21T03:56:33.888 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "devicehealth", 2024-09-21T03:56:33.888 INFO:teuthology.orchestra.run.smithi039.stdout: "can_run": true, 2024-09-21T03:56:33.888 INFO:teuthology.orchestra.run.smithi039.stdout: "error_string": "", 2024-09-21T03:56:33.888 INFO:teuthology.orchestra.run.smithi039.stdout: "module_options": { 2024-09-21T03:56:33.888 INFO:teuthology.orchestra.run.smithi039.stdout: "enable_monitoring": { 2024-09-21T03:56:33.888 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "enable_monitoring", 2024-09-21T03:56:33.889 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-09-21T03:56:33.889 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.889 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:33.889 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2024-09-21T03:56:33.889 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.889 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.889 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.889 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "monitor device health metrics", 2024-09-21T03:56:33.889 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.889 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.889 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.889 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.889 INFO:teuthology.orchestra.run.smithi039.stdout: "log_level": { 2024-09-21T03:56:33.889 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_level", 2024-09-21T03:56:33.890 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-09-21T03:56:33.890 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.890 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:33.890 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-09-21T03:56:33.890 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.890 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.890 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-09-21T03:56:33.890 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-09-21T03:56:33.890 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-09-21T03:56:33.890 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-09-21T03:56:33.890 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-09-21T03:56:33.890 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-09-21T03:56:33.890 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-09-21T03:56:33.890 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-09-21T03:56:33.891 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:33.891 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.891 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.891 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.891 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.891 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster": { 2024-09-21T03:56:33.891 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster", 2024-09-21T03:56:33.891 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-09-21T03:56:33.891 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.891 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:33.891 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-09-21T03:56:33.891 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.891 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.891 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.892 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:33.892 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.892 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.892 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.892 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.892 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster_level": { 2024-09-21T03:56:33.892 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster_level", 2024-09-21T03:56:33.892 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-09-21T03:56:33.892 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.892 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:33.892 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "info", 2024-09-21T03:56:33.892 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.892 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.892 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-09-21T03:56:33.892 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-09-21T03:56:33.893 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-09-21T03:56:33.893 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-09-21T03:56:33.893 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-09-21T03:56:33.893 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-09-21T03:56:33.893 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-09-21T03:56:33.893 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-09-21T03:56:33.893 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:33.893 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.893 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.893 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.893 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.893 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_file": { 2024-09-21T03:56:33.893 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_file", 2024-09-21T03:56:33.893 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-09-21T03:56:33.893 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.894 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:33.894 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-09-21T03:56:33.894 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.894 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.894 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.894 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:33.894 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.894 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.894 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.894 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.894 INFO:teuthology.orchestra.run.smithi039.stdout: "mark_out_threshold": { 2024-09-21T03:56:33.894 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "mark_out_threshold", 2024-09-21T03:56:33.894 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "secs", 2024-09-21T03:56:33.894 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.895 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:33.895 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "2419200", 2024-09-21T03:56:33.895 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.895 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.895 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.895 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-09-21T03:56:33.895 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.895 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.895 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.895 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.895 INFO:teuthology.orchestra.run.smithi039.stdout: "pool_name": { 2024-09-21T03:56:33.895 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "pool_name", 2024-09-21T03:56:33.895 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-09-21T03:56:33.895 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.896 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:33.896 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "device_health_metrics", 2024-09-21T03:56:33.896 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.896 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.896 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.896 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "name of pool in which to store device health metrics", 2024-09-21T03:56:33.896 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.896 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.896 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.896 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.896 INFO:teuthology.orchestra.run.smithi039.stdout: "retention_period": { 2024-09-21T03:56:33.896 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "retention_period", 2024-09-21T03:56:33.896 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "secs", 2024-09-21T03:56:33.896 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.896 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:33.897 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "15552000", 2024-09-21T03:56:33.897 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.897 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.897 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.897 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "how long to retain device health metrics", 2024-09-21T03:56:33.897 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.897 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.897 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.897 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.897 INFO:teuthology.orchestra.run.smithi039.stdout: "scrape_frequency": { 2024-09-21T03:56:33.897 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "scrape_frequency", 2024-09-21T03:56:33.897 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "secs", 2024-09-21T03:56:33.897 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.897 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:33.897 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "86400", 2024-09-21T03:56:33.898 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.898 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.898 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.898 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "how frequently to scrape device health metrics", 2024-09-21T03:56:33.898 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.898 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.898 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.898 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.898 INFO:teuthology.orchestra.run.smithi039.stdout: "self_heal": { 2024-09-21T03:56:33.898 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "self_heal", 2024-09-21T03:56:33.898 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-09-21T03:56:33.898 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.898 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:33.898 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2024-09-21T03:56:33.898 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.899 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.899 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.899 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-09-21T03:56:33.899 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.899 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.899 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.899 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.899 INFO:teuthology.orchestra.run.smithi039.stdout: "sleep_interval": { 2024-09-21T03:56:33.899 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "sleep_interval", 2024-09-21T03:56:33.899 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "secs", 2024-09-21T03:56:33.899 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.899 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:33.899 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "600", 2024-09-21T03:56:33.899 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.899 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.899 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.900 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "how frequently to wake up and check device health", 2024-09-21T03:56:33.900 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.900 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.900 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.900 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.900 INFO:teuthology.orchestra.run.smithi039.stdout: "sqlite3_killpoint": { 2024-09-21T03:56:33.900 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:56:33.900 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-09-21T03:56:33.900 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "dev", 2024-09-21T03:56:33.900 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:33.900 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "0", 2024-09-21T03:56:33.900 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.900 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.900 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.901 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:33.901 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.901 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.901 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.901 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.901 INFO:teuthology.orchestra.run.smithi039.stdout: "warn_threshold": { 2024-09-21T03:56:33.901 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "warn_threshold", 2024-09-21T03:56:33.901 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "secs", 2024-09-21T03:56:33.901 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.901 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:33.901 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "7257600", 2024-09-21T03:56:33.901 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.901 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.901 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.901 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-09-21T03:56:33.902 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.902 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.902 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.902 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-21T03:56:33.902 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-21T03:56:33.902 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.902 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-21T03:56:33.902 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "diskprediction_local", 2024-09-21T03:56:33.902 INFO:teuthology.orchestra.run.smithi039.stdout: "can_run": true, 2024-09-21T03:56:33.902 INFO:teuthology.orchestra.run.smithi039.stdout: "error_string": "", 2024-09-21T03:56:33.902 INFO:teuthology.orchestra.run.smithi039.stdout: "module_options": { 2024-09-21T03:56:33.902 INFO:teuthology.orchestra.run.smithi039.stdout: "log_level": { 2024-09-21T03:56:33.902 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_level", 2024-09-21T03:56:33.903 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-09-21T03:56:33.903 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.903 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:33.903 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-09-21T03:56:33.903 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.903 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.903 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-09-21T03:56:33.903 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-09-21T03:56:33.903 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-09-21T03:56:33.903 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-09-21T03:56:33.903 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-09-21T03:56:33.903 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-09-21T03:56:33.903 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-09-21T03:56:33.903 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-09-21T03:56:33.903 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:33.904 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.904 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.904 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.904 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.904 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster": { 2024-09-21T03:56:33.904 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster", 2024-09-21T03:56:33.904 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-09-21T03:56:33.904 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.904 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:33.904 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-09-21T03:56:33.904 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.904 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.904 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.904 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:33.905 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.905 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.905 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.905 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.905 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster_level": { 2024-09-21T03:56:33.905 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster_level", 2024-09-21T03:56:33.905 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-09-21T03:56:33.905 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.905 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:33.905 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "info", 2024-09-21T03:56:33.905 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.905 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.905 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-09-21T03:56:33.905 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-09-21T03:56:33.906 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-09-21T03:56:33.906 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-09-21T03:56:33.906 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-09-21T03:56:33.906 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-09-21T03:56:33.906 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-09-21T03:56:33.906 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-09-21T03:56:33.906 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:33.906 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.906 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.906 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.906 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.906 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_file": { 2024-09-21T03:56:33.906 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_file", 2024-09-21T03:56:33.906 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-09-21T03:56:33.906 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.907 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:33.907 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-09-21T03:56:33.907 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.907 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.907 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.907 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:33.907 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.907 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.907 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.907 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.907 INFO:teuthology.orchestra.run.smithi039.stdout: "predict_interval": { 2024-09-21T03:56:33.907 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "predict_interval", 2024-09-21T03:56:33.907 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-09-21T03:56:33.908 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.908 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-09-21T03:56:33.908 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "86400", 2024-09-21T03:56:33.908 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.908 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.908 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.908 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:33.908 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.908 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.908 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.908 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.908 INFO:teuthology.orchestra.run.smithi039.stdout: "predictor_model": { 2024-09-21T03:56:33.908 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "predictor_model", 2024-09-21T03:56:33.908 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-09-21T03:56:33.909 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.909 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-09-21T03:56:33.909 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "prophetstor", 2024-09-21T03:56:33.909 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.909 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.909 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.909 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:33.909 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.909 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.909 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.909 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.909 INFO:teuthology.orchestra.run.smithi039.stdout: "sleep_interval": { 2024-09-21T03:56:33.909 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "sleep_interval", 2024-09-21T03:56:33.909 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-09-21T03:56:33.910 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.910 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-09-21T03:56:33.910 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "600", 2024-09-21T03:56:33.910 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.910 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.910 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.910 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:33.910 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.910 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.910 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.910 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.910 INFO:teuthology.orchestra.run.smithi039.stdout: "sqlite3_killpoint": { 2024-09-21T03:56:33.910 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:56:33.910 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-09-21T03:56:33.911 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "dev", 2024-09-21T03:56:33.911 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:33.911 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "0", 2024-09-21T03:56:33.911 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.911 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.911 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.911 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:33.911 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.911 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.911 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.911 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-21T03:56:33.911 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-21T03:56:33.911 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.911 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-21T03:56:33.911 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "influx", 2024-09-21T03:56:33.912 INFO:teuthology.orchestra.run.smithi039.stdout: "can_run": false, 2024-09-21T03:56:33.912 INFO:teuthology.orchestra.run.smithi039.stdout: "error_string": "influxdb python module not found", 2024-09-21T03:56:33.912 INFO:teuthology.orchestra.run.smithi039.stdout: "module_options": { 2024-09-21T03:56:33.912 INFO:teuthology.orchestra.run.smithi039.stdout: "batch_size": { 2024-09-21T03:56:33.912 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "batch_size", 2024-09-21T03:56:33.912 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-09-21T03:56:33.912 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.912 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-09-21T03:56:33.912 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "5000", 2024-09-21T03:56:33.912 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.912 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.912 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.912 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-09-21T03:56:33.913 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.913 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.913 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.913 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.913 INFO:teuthology.orchestra.run.smithi039.stdout: "database": { 2024-09-21T03:56:33.913 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "database", 2024-09-21T03:56:33.913 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-09-21T03:56:33.913 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.913 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-09-21T03:56:33.913 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "ceph", 2024-09-21T03:56:33.913 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.913 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.913 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.913 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-09-21T03:56:33.913 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.914 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.914 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.914 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.914 INFO:teuthology.orchestra.run.smithi039.stdout: "hostname": { 2024-09-21T03:56:33.914 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "hostname", 2024-09-21T03:56:33.914 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-09-21T03:56:33.914 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.914 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-09-21T03:56:33.914 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-09-21T03:56:33.914 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.914 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.914 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.914 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "InfluxDB server hostname", 2024-09-21T03:56:33.914 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.915 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.915 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.915 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.915 INFO:teuthology.orchestra.run.smithi039.stdout: "interval": { 2024-09-21T03:56:33.915 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "interval", 2024-09-21T03:56:33.915 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "secs", 2024-09-21T03:56:33.915 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.915 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-09-21T03:56:33.915 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "30", 2024-09-21T03:56:33.915 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "5", 2024-09-21T03:56:33.915 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.915 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.915 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-09-21T03:56:33.915 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.916 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.916 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.916 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.916 INFO:teuthology.orchestra.run.smithi039.stdout: "log_level": { 2024-09-21T03:56:33.916 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_level", 2024-09-21T03:56:33.916 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-09-21T03:56:33.916 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.916 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:33.916 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-09-21T03:56:33.916 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.916 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.916 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-09-21T03:56:33.916 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-09-21T03:56:33.916 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-09-21T03:56:33.916 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-09-21T03:56:33.917 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-09-21T03:56:33.917 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-09-21T03:56:33.917 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-09-21T03:56:33.917 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-09-21T03:56:33.917 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:33.917 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.917 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.917 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.917 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.917 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster": { 2024-09-21T03:56:33.917 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster", 2024-09-21T03:56:33.917 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-09-21T03:56:33.917 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.917 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:33.918 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-09-21T03:56:33.918 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.918 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.918 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.918 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:33.918 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.918 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.918 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.918 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.918 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster_level": { 2024-09-21T03:56:33.918 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster_level", 2024-09-21T03:56:33.918 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-09-21T03:56:33.918 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.918 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:33.919 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "info", 2024-09-21T03:56:33.919 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.919 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.919 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-09-21T03:56:33.919 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-09-21T03:56:33.919 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-09-21T03:56:33.919 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-09-21T03:56:33.919 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-09-21T03:56:33.919 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-09-21T03:56:33.919 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-09-21T03:56:33.919 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-09-21T03:56:33.919 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:33.919 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.919 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.919 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.920 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.920 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_file": { 2024-09-21T03:56:33.920 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_file", 2024-09-21T03:56:33.920 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-09-21T03:56:33.920 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.920 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:33.920 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-09-21T03:56:33.920 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.920 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.920 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.920 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:33.920 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.920 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.921 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.921 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.921 INFO:teuthology.orchestra.run.smithi039.stdout: "password": { 2024-09-21T03:56:33.921 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "password", 2024-09-21T03:56:33.921 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-09-21T03:56:33.921 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.921 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-09-21T03:56:33.921 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-09-21T03:56:33.921 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.921 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.921 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.921 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "password of InfluxDB server user", 2024-09-21T03:56:33.921 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.921 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.921 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.922 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.922 INFO:teuthology.orchestra.run.smithi039.stdout: "port": { 2024-09-21T03:56:33.922 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "port", 2024-09-21T03:56:33.922 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-09-21T03:56:33.922 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.922 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-09-21T03:56:33.922 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "8086", 2024-09-21T03:56:33.922 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.922 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.922 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.922 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "InfluxDB server port", 2024-09-21T03:56:33.922 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.922 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.922 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.923 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.923 INFO:teuthology.orchestra.run.smithi039.stdout: "sqlite3_killpoint": { 2024-09-21T03:56:33.923 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:56:33.923 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-09-21T03:56:33.923 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "dev", 2024-09-21T03:56:33.923 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:33.923 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "0", 2024-09-21T03:56:33.923 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.923 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.923 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.923 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:33.923 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.923 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.924 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.924 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.924 INFO:teuthology.orchestra.run.smithi039.stdout: "ssl": { 2024-09-21T03:56:33.924 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "ssl", 2024-09-21T03:56:33.924 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-09-21T03:56:33.924 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.924 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-09-21T03:56:33.924 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "false", 2024-09-21T03:56:33.924 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.924 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.924 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.924 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-09-21T03:56:33.924 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.924 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.925 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.925 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.925 INFO:teuthology.orchestra.run.smithi039.stdout: "threads": { 2024-09-21T03:56:33.925 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "threads", 2024-09-21T03:56:33.925 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-09-21T03:56:33.925 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.925 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-09-21T03:56:33.925 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "5", 2024-09-21T03:56:33.925 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "1", 2024-09-21T03:56:33.925 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "32", 2024-09-21T03:56:33.925 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.925 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-09-21T03:56:33.925 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.925 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.925 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.926 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.926 INFO:teuthology.orchestra.run.smithi039.stdout: "username": { 2024-09-21T03:56:33.926 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "username", 2024-09-21T03:56:33.926 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-09-21T03:56:33.926 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.926 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-09-21T03:56:33.926 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-09-21T03:56:33.926 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.926 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.926 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.926 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "username of InfluxDB server user", 2024-09-21T03:56:33.926 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.926 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.927 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.927 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.927 INFO:teuthology.orchestra.run.smithi039.stdout: "verify_ssl": { 2024-09-21T03:56:33.927 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "verify_ssl", 2024-09-21T03:56:33.927 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-09-21T03:56:33.927 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.927 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-09-21T03:56:33.927 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "true", 2024-09-21T03:56:33.927 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.927 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.927 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.927 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-09-21T03:56:33.927 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.927 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.927 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.928 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-21T03:56:33.928 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-21T03:56:33.928 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.928 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-21T03:56:33.928 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "insights", 2024-09-21T03:56:33.928 INFO:teuthology.orchestra.run.smithi039.stdout: "can_run": true, 2024-09-21T03:56:33.928 INFO:teuthology.orchestra.run.smithi039.stdout: "error_string": "", 2024-09-21T03:56:33.928 INFO:teuthology.orchestra.run.smithi039.stdout: "module_options": { 2024-09-21T03:56:33.928 INFO:teuthology.orchestra.run.smithi039.stdout: "log_level": { 2024-09-21T03:56:33.928 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_level", 2024-09-21T03:56:33.928 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-09-21T03:56:33.928 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.928 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:33.929 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-09-21T03:56:33.929 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.929 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.929 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-09-21T03:56:33.929 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-09-21T03:56:33.929 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-09-21T03:56:33.929 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-09-21T03:56:33.929 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-09-21T03:56:33.929 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-09-21T03:56:33.929 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-09-21T03:56:33.929 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-09-21T03:56:33.929 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:33.929 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.929 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.929 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.930 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.930 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster": { 2024-09-21T03:56:33.930 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster", 2024-09-21T03:56:33.930 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-09-21T03:56:33.930 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.930 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:33.930 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-09-21T03:56:33.930 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.930 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.930 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.930 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:33.930 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.930 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.930 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.931 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.931 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster_level": { 2024-09-21T03:56:33.931 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster_level", 2024-09-21T03:56:33.931 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-09-21T03:56:33.931 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.931 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:33.931 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "info", 2024-09-21T03:56:33.931 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.931 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.931 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-09-21T03:56:33.931 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-09-21T03:56:33.931 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-09-21T03:56:33.931 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-09-21T03:56:33.932 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-09-21T03:56:33.932 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-09-21T03:56:33.932 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-09-21T03:56:33.932 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-09-21T03:56:33.932 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:33.932 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.932 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.932 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.932 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.932 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_file": { 2024-09-21T03:56:33.932 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_file", 2024-09-21T03:56:33.932 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-09-21T03:56:33.932 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.932 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:33.933 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-09-21T03:56:33.933 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.933 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.933 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.933 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:33.933 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.933 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.933 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.933 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.933 INFO:teuthology.orchestra.run.smithi039.stdout: "sqlite3_killpoint": { 2024-09-21T03:56:33.933 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:56:33.933 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-09-21T03:56:33.933 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "dev", 2024-09-21T03:56:33.933 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:33.933 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "0", 2024-09-21T03:56:33.934 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.934 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.934 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.934 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:33.934 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.934 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.934 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.934 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-21T03:56:33.934 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-21T03:56:33.934 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.934 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-21T03:56:33.934 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "iostat", 2024-09-21T03:56:33.934 INFO:teuthology.orchestra.run.smithi039.stdout: "can_run": true, 2024-09-21T03:56:33.934 INFO:teuthology.orchestra.run.smithi039.stdout: "error_string": "", 2024-09-21T03:56:33.934 INFO:teuthology.orchestra.run.smithi039.stdout: "module_options": { 2024-09-21T03:56:33.935 INFO:teuthology.orchestra.run.smithi039.stdout: "log_level": { 2024-09-21T03:56:33.935 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_level", 2024-09-21T03:56:33.935 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-09-21T03:56:33.935 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.935 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:33.935 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-09-21T03:56:33.935 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.935 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.935 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-09-21T03:56:33.935 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-09-21T03:56:33.935 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-09-21T03:56:33.935 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-09-21T03:56:33.935 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-09-21T03:56:33.935 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-09-21T03:56:33.936 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-09-21T03:56:33.936 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-09-21T03:56:33.936 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:33.936 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.936 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.936 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.936 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.936 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster": { 2024-09-21T03:56:33.936 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster", 2024-09-21T03:56:33.936 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-09-21T03:56:33.936 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.936 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:33.936 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-09-21T03:56:33.936 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.937 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.937 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.937 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:33.937 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.937 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.937 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.937 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.937 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster_level": { 2024-09-21T03:56:33.937 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster_level", 2024-09-21T03:56:33.937 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-09-21T03:56:33.937 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.937 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:33.937 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "info", 2024-09-21T03:56:33.937 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.938 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.938 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-09-21T03:56:33.938 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-09-21T03:56:33.938 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-09-21T03:56:33.938 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-09-21T03:56:33.938 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-09-21T03:56:33.938 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-09-21T03:56:33.938 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-09-21T03:56:33.938 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-09-21T03:56:33.938 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:33.938 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.938 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.938 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.938 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.939 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_file": { 2024-09-21T03:56:33.939 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_file", 2024-09-21T03:56:33.939 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-09-21T03:56:33.939 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.939 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:33.939 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-09-21T03:56:33.939 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.939 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.939 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.939 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:33.939 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.939 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.939 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.939 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.940 INFO:teuthology.orchestra.run.smithi039.stdout: "sqlite3_killpoint": { 2024-09-21T03:56:33.940 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:56:33.940 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-09-21T03:56:33.940 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "dev", 2024-09-21T03:56:33.940 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:33.940 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "0", 2024-09-21T03:56:33.940 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.940 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.940 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.940 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:33.940 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.940 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.940 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.940 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-21T03:56:33.940 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-21T03:56:33.941 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.941 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-21T03:56:33.941 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "k8sevents", 2024-09-21T03:56:33.941 INFO:teuthology.orchestra.run.smithi039.stdout: "can_run": true, 2024-09-21T03:56:33.941 INFO:teuthology.orchestra.run.smithi039.stdout: "error_string": "", 2024-09-21T03:56:33.941 INFO:teuthology.orchestra.run.smithi039.stdout: "module_options": { 2024-09-21T03:56:33.941 INFO:teuthology.orchestra.run.smithi039.stdout: "ceph_event_retention_days": { 2024-09-21T03:56:33.941 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "ceph_event_retention_days", 2024-09-21T03:56:33.941 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-09-21T03:56:33.941 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.941 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-09-21T03:56:33.941 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "7", 2024-09-21T03:56:33.941 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.942 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.942 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.942 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Days to hold ceph event information within local cache", 2024-09-21T03:56:33.942 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.942 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.942 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.942 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.942 INFO:teuthology.orchestra.run.smithi039.stdout: "config_check_secs": { 2024-09-21T03:56:33.942 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "config_check_secs", 2024-09-21T03:56:33.942 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-09-21T03:56:33.942 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.942 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-09-21T03:56:33.942 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "10", 2024-09-21T03:56:33.943 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "10", 2024-09-21T03:56:33.943 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.943 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.943 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-09-21T03:56:33.943 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.943 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.943 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.943 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.943 INFO:teuthology.orchestra.run.smithi039.stdout: "log_level": { 2024-09-21T03:56:33.943 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_level", 2024-09-21T03:56:33.943 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-09-21T03:56:33.943 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.943 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:33.943 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-09-21T03:56:33.944 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.944 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.944 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-09-21T03:56:33.944 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-09-21T03:56:33.944 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-09-21T03:56:33.944 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-09-21T03:56:33.944 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-09-21T03:56:33.944 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-09-21T03:56:33.944 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-09-21T03:56:33.944 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-09-21T03:56:33.944 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:33.944 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.944 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.944 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.945 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.945 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster": { 2024-09-21T03:56:33.945 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster", 2024-09-21T03:56:33.945 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-09-21T03:56:33.945 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.945 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:33.945 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-09-21T03:56:33.945 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.945 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.945 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.945 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:33.945 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.945 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.945 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.946 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.946 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster_level": { 2024-09-21T03:56:33.946 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster_level", 2024-09-21T03:56:33.946 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-09-21T03:56:33.946 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.946 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:33.946 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "info", 2024-09-21T03:56:33.946 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.946 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.946 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-09-21T03:56:33.946 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-09-21T03:56:33.946 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-09-21T03:56:33.946 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-09-21T03:56:33.946 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-09-21T03:56:33.946 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-09-21T03:56:33.947 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-09-21T03:56:33.947 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-09-21T03:56:33.947 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:33.947 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.947 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.947 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.947 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.947 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_file": { 2024-09-21T03:56:33.947 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_file", 2024-09-21T03:56:33.947 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-09-21T03:56:33.947 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.947 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:33.947 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-09-21T03:56:33.947 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.947 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.947 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.948 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:33.948 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.948 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.948 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.948 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.948 INFO:teuthology.orchestra.run.smithi039.stdout: "sqlite3_killpoint": { 2024-09-21T03:56:33.948 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:56:33.948 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-09-21T03:56:33.948 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "dev", 2024-09-21T03:56:33.948 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:33.948 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "0", 2024-09-21T03:56:33.948 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.948 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.948 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.948 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:33.948 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.949 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.949 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.949 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-21T03:56:33.949 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-21T03:56:33.949 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.949 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-21T03:56:33.949 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "localpool", 2024-09-21T03:56:33.949 INFO:teuthology.orchestra.run.smithi039.stdout: "can_run": true, 2024-09-21T03:56:33.949 INFO:teuthology.orchestra.run.smithi039.stdout: "error_string": "", 2024-09-21T03:56:33.949 INFO:teuthology.orchestra.run.smithi039.stdout: "module_options": { 2024-09-21T03:56:33.949 INFO:teuthology.orchestra.run.smithi039.stdout: "failure_domain": { 2024-09-21T03:56:33.949 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "failure_domain", 2024-09-21T03:56:33.949 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-09-21T03:56:33.949 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.949 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:33.949 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "host", 2024-09-21T03:56:33.950 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.950 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.950 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.950 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "failure domain for any created local pool", 2024-09-21T03:56:33.950 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-09-21T03:56:33.950 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.950 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.950 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.950 INFO:teuthology.orchestra.run.smithi039.stdout: "log_level": { 2024-09-21T03:56:33.950 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_level", 2024-09-21T03:56:33.950 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-09-21T03:56:33.950 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.950 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:33.950 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-09-21T03:56:33.950 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.951 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.951 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-09-21T03:56:33.951 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-09-21T03:56:33.951 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-09-21T03:56:33.951 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-09-21T03:56:33.951 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-09-21T03:56:33.951 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-09-21T03:56:33.951 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-09-21T03:56:33.951 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-09-21T03:56:33.951 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:33.951 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.951 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.951 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.952 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.952 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster": { 2024-09-21T03:56:33.952 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster", 2024-09-21T03:56:33.952 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-09-21T03:56:33.952 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.952 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:33.952 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-09-21T03:56:33.952 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.952 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.952 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.952 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:33.952 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.952 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.952 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.952 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.953 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster_level": { 2024-09-21T03:56:33.953 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster_level", 2024-09-21T03:56:33.953 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-09-21T03:56:33.953 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.953 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:33.953 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "info", 2024-09-21T03:56:33.953 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.953 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.953 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-09-21T03:56:33.953 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-09-21T03:56:33.953 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-09-21T03:56:33.953 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-09-21T03:56:33.953 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-09-21T03:56:33.953 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-09-21T03:56:33.954 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-09-21T03:56:33.954 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-09-21T03:56:33.954 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:33.954 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.954 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.954 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.954 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.954 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_file": { 2024-09-21T03:56:33.954 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_file", 2024-09-21T03:56:33.954 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-09-21T03:56:33.954 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.954 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:33.954 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-09-21T03:56:33.954 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.955 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.955 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.955 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:33.955 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.955 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.955 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.955 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.955 INFO:teuthology.orchestra.run.smithi039.stdout: "min_size": { 2024-09-21T03:56:33.955 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "min_size", 2024-09-21T03:56:33.955 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-09-21T03:56:33.955 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.955 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:33.955 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-09-21T03:56:33.955 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.956 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.956 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.956 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "default min_size for any created local pool", 2024-09-21T03:56:33.956 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-09-21T03:56:33.956 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.956 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.956 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.956 INFO:teuthology.orchestra.run.smithi039.stdout: "num_rep": { 2024-09-21T03:56:33.956 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "num_rep", 2024-09-21T03:56:33.956 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-09-21T03:56:33.956 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.956 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:33.956 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "3", 2024-09-21T03:56:33.956 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.957 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.957 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.957 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "default replica count for any created local pool", 2024-09-21T03:56:33.957 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.957 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.957 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.957 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.957 INFO:teuthology.orchestra.run.smithi039.stdout: "pg_num": { 2024-09-21T03:56:33.957 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "pg_num", 2024-09-21T03:56:33.957 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-09-21T03:56:33.957 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.957 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:33.957 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "128", 2024-09-21T03:56:33.957 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.957 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.958 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.958 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "default pg_num for any created local pool", 2024-09-21T03:56:33.958 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.958 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.958 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.958 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.958 INFO:teuthology.orchestra.run.smithi039.stdout: "prefix": { 2024-09-21T03:56:33.958 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "prefix", 2024-09-21T03:56:33.958 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-09-21T03:56:33.958 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.958 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:33.958 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-09-21T03:56:33.958 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.958 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.959 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.959 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "name prefix for any created local pool", 2024-09-21T03:56:33.959 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.959 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.959 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.959 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.959 INFO:teuthology.orchestra.run.smithi039.stdout: "sqlite3_killpoint": { 2024-09-21T03:56:33.959 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:56:33.959 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-09-21T03:56:33.959 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "dev", 2024-09-21T03:56:33.959 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:33.959 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "0", 2024-09-21T03:56:33.959 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.959 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.959 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.960 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:33.960 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.960 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.960 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.960 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.960 INFO:teuthology.orchestra.run.smithi039.stdout: "subtree": { 2024-09-21T03:56:33.960 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "subtree", 2024-09-21T03:56:33.960 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-09-21T03:56:33.960 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.960 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:33.960 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "rack", 2024-09-21T03:56:33.960 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.960 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.960 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.961 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "CRUSH level for which to create a local pool", 2024-09-21T03:56:33.961 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-09-21T03:56:33.961 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.961 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.961 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-21T03:56:33.961 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-21T03:56:33.961 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.961 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-21T03:56:33.961 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "mds_autoscaler", 2024-09-21T03:56:33.961 INFO:teuthology.orchestra.run.smithi039.stdout: "can_run": true, 2024-09-21T03:56:33.961 INFO:teuthology.orchestra.run.smithi039.stdout: "error_string": "", 2024-09-21T03:56:33.961 INFO:teuthology.orchestra.run.smithi039.stdout: "module_options": { 2024-09-21T03:56:33.961 INFO:teuthology.orchestra.run.smithi039.stdout: "log_level": { 2024-09-21T03:56:33.961 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_level", 2024-09-21T03:56:33.962 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-09-21T03:56:33.962 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.962 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:33.962 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-09-21T03:56:33.962 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.962 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.962 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-09-21T03:56:33.962 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-09-21T03:56:33.962 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-09-21T03:56:33.962 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-09-21T03:56:33.962 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-09-21T03:56:33.962 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-09-21T03:56:33.962 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-09-21T03:56:33.962 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-09-21T03:56:33.962 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:33.963 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.963 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.963 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.963 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.963 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster": { 2024-09-21T03:56:33.963 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster", 2024-09-21T03:56:33.963 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-09-21T03:56:33.963 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.963 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:33.963 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-09-21T03:56:33.963 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.963 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.963 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.963 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:33.964 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.964 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.964 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.964 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.964 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster_level": { 2024-09-21T03:56:33.964 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster_level", 2024-09-21T03:56:33.964 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-09-21T03:56:33.964 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.964 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:33.964 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "info", 2024-09-21T03:56:33.964 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.964 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.964 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-09-21T03:56:33.964 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-09-21T03:56:33.964 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-09-21T03:56:33.965 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-09-21T03:56:33.965 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-09-21T03:56:33.965 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-09-21T03:56:33.965 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-09-21T03:56:33.965 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-09-21T03:56:33.965 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:33.965 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.965 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.965 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.965 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.965 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_file": { 2024-09-21T03:56:33.965 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_file", 2024-09-21T03:56:33.965 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-09-21T03:56:33.965 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.965 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:33.966 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-09-21T03:56:33.966 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.966 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.966 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.966 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:33.966 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.966 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.966 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.966 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.966 INFO:teuthology.orchestra.run.smithi039.stdout: "sqlite3_killpoint": { 2024-09-21T03:56:33.966 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:56:33.966 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-09-21T03:56:33.966 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "dev", 2024-09-21T03:56:33.966 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:33.967 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "0", 2024-09-21T03:56:33.967 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.967 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.967 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.967 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:33.967 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.967 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.967 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.967 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-21T03:56:33.967 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-21T03:56:33.967 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.967 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-21T03:56:33.967 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "mirroring", 2024-09-21T03:56:33.967 INFO:teuthology.orchestra.run.smithi039.stdout: "can_run": true, 2024-09-21T03:56:33.967 INFO:teuthology.orchestra.run.smithi039.stdout: "error_string": "", 2024-09-21T03:56:33.968 INFO:teuthology.orchestra.run.smithi039.stdout: "module_options": { 2024-09-21T03:56:33.968 INFO:teuthology.orchestra.run.smithi039.stdout: "log_level": { 2024-09-21T03:56:33.968 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_level", 2024-09-21T03:56:33.968 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-09-21T03:56:33.968 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.968 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:33.968 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-09-21T03:56:33.968 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.968 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.968 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-09-21T03:56:33.968 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-09-21T03:56:33.968 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-09-21T03:56:33.968 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-09-21T03:56:33.968 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-09-21T03:56:33.969 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-09-21T03:56:33.969 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-09-21T03:56:33.969 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-09-21T03:56:33.969 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:33.969 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.969 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.969 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.969 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.969 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster": { 2024-09-21T03:56:33.969 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster", 2024-09-21T03:56:33.969 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-09-21T03:56:33.969 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.969 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:33.969 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-09-21T03:56:33.969 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.970 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.970 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.970 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:33.970 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.970 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.970 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.970 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.970 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster_level": { 2024-09-21T03:56:33.970 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster_level", 2024-09-21T03:56:33.970 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-09-21T03:56:33.970 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.970 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:33.970 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "info", 2024-09-21T03:56:33.970 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.970 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.971 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-09-21T03:56:33.971 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-09-21T03:56:33.971 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-09-21T03:56:33.971 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-09-21T03:56:33.971 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-09-21T03:56:33.971 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-09-21T03:56:33.971 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-09-21T03:56:33.971 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-09-21T03:56:33.971 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:33.971 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.971 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.971 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.971 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.971 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_file": { 2024-09-21T03:56:33.972 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_file", 2024-09-21T03:56:33.972 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-09-21T03:56:33.972 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.972 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:33.972 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-09-21T03:56:33.972 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.972 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.972 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.972 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:33.972 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.972 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.972 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.972 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.972 INFO:teuthology.orchestra.run.smithi039.stdout: "sqlite3_killpoint": { 2024-09-21T03:56:33.973 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:56:33.973 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-09-21T03:56:33.973 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "dev", 2024-09-21T03:56:33.973 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:33.973 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "0", 2024-09-21T03:56:33.973 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.973 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.973 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.973 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:33.973 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.973 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.973 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.973 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-21T03:56:33.973 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-21T03:56:33.974 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.974 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-21T03:56:33.974 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "nfs", 2024-09-21T03:56:33.974 INFO:teuthology.orchestra.run.smithi039.stdout: "can_run": true, 2024-09-21T03:56:33.974 INFO:teuthology.orchestra.run.smithi039.stdout: "error_string": "", 2024-09-21T03:56:33.974 INFO:teuthology.orchestra.run.smithi039.stdout: "module_options": { 2024-09-21T03:56:33.974 INFO:teuthology.orchestra.run.smithi039.stdout: "log_level": { 2024-09-21T03:56:33.974 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_level", 2024-09-21T03:56:33.974 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-09-21T03:56:33.974 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.974 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:33.974 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-09-21T03:56:33.974 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.975 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.975 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-09-21T03:56:33.975 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-09-21T03:56:33.975 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-09-21T03:56:33.975 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-09-21T03:56:33.975 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-09-21T03:56:33.975 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-09-21T03:56:33.975 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-09-21T03:56:33.975 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-09-21T03:56:33.975 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:33.975 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.975 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.975 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.975 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.975 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster": { 2024-09-21T03:56:33.976 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster", 2024-09-21T03:56:33.976 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-09-21T03:56:33.976 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.976 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:33.976 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-09-21T03:56:33.976 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.976 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.976 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.976 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:33.976 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.976 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.976 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.976 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.976 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster_level": { 2024-09-21T03:56:33.977 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster_level", 2024-09-21T03:56:33.977 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-09-21T03:56:33.977 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.977 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:33.977 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "info", 2024-09-21T03:56:33.977 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.977 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.977 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-09-21T03:56:33.977 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-09-21T03:56:33.977 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-09-21T03:56:33.977 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-09-21T03:56:33.977 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-09-21T03:56:33.977 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-09-21T03:56:33.978 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-09-21T03:56:33.978 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-09-21T03:56:33.978 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:33.978 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.978 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.978 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.978 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.978 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_file": { 2024-09-21T03:56:33.978 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_file", 2024-09-21T03:56:33.978 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-09-21T03:56:33.978 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.978 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:33.978 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-09-21T03:56:33.978 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.978 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.979 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.979 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:33.979 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.979 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.979 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.979 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.979 INFO:teuthology.orchestra.run.smithi039.stdout: "sqlite3_killpoint": { 2024-09-21T03:56:33.979 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:56:33.979 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-09-21T03:56:33.979 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "dev", 2024-09-21T03:56:33.979 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:33.979 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "0", 2024-09-21T03:56:33.979 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.980 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.980 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.980 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:33.980 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.980 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.980 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.980 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-21T03:56:33.980 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-21T03:56:33.980 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.980 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-21T03:56:33.980 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "orchestrator", 2024-09-21T03:56:33.980 INFO:teuthology.orchestra.run.smithi039.stdout: "can_run": true, 2024-09-21T03:56:33.980 INFO:teuthology.orchestra.run.smithi039.stdout: "error_string": "", 2024-09-21T03:56:33.980 INFO:teuthology.orchestra.run.smithi039.stdout: "module_options": { 2024-09-21T03:56:33.980 INFO:teuthology.orchestra.run.smithi039.stdout: "fail_fs": { 2024-09-21T03:56:33.981 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "fail_fs", 2024-09-21T03:56:33.981 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-09-21T03:56:33.981 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.981 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-09-21T03:56:33.981 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-09-21T03:56:33.981 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.981 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.981 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.981 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-09-21T03:56:33.981 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.981 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.981 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.981 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.981 INFO:teuthology.orchestra.run.smithi039.stdout: "log_level": { 2024-09-21T03:56:33.981 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_level", 2024-09-21T03:56:33.982 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-09-21T03:56:33.982 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.982 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:33.982 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-09-21T03:56:33.982 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.982 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.982 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-09-21T03:56:33.982 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-09-21T03:56:33.982 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-09-21T03:56:33.982 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-09-21T03:56:33.982 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-09-21T03:56:33.982 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-09-21T03:56:33.982 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-09-21T03:56:33.982 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-09-21T03:56:33.983 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:33.983 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.983 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.983 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.983 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.983 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster": { 2024-09-21T03:56:33.983 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster", 2024-09-21T03:56:33.983 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-09-21T03:56:33.983 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.983 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:33.983 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-09-21T03:56:33.983 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.983 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.983 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.983 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:33.984 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.984 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.984 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.984 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.984 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster_level": { 2024-09-21T03:56:33.984 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster_level", 2024-09-21T03:56:33.984 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-09-21T03:56:33.984 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.984 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:33.984 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "info", 2024-09-21T03:56:33.984 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.984 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.984 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-09-21T03:56:33.984 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-09-21T03:56:33.985 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-09-21T03:56:33.985 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-09-21T03:56:33.985 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-09-21T03:56:33.985 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-09-21T03:56:33.985 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-09-21T03:56:33.985 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-09-21T03:56:33.985 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:33.985 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.985 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.985 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.985 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.985 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_file": { 2024-09-21T03:56:33.985 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_file", 2024-09-21T03:56:33.985 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-09-21T03:56:33.986 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.986 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:33.986 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-09-21T03:56:33.986 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.986 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.986 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.986 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:33.986 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.986 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.986 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.986 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.986 INFO:teuthology.orchestra.run.smithi039.stdout: "orchestrator": { 2024-09-21T03:56:33.986 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "orchestrator", 2024-09-21T03:56:33.986 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-09-21T03:56:33.987 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.987 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:33.987 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-09-21T03:56:33.987 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.987 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.987 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-09-21T03:56:33.987 INFO:teuthology.orchestra.run.smithi039.stdout: "cephadm", 2024-09-21T03:56:33.987 INFO:teuthology.orchestra.run.smithi039.stdout: "rook", 2024-09-21T03:56:33.987 INFO:teuthology.orchestra.run.smithi039.stdout: "test_orchestrator" 2024-09-21T03:56:33.987 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-09-21T03:56:33.987 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Orchestrator backend", 2024-09-21T03:56:33.987 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.987 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.987 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.988 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.988 INFO:teuthology.orchestra.run.smithi039.stdout: "sqlite3_killpoint": { 2024-09-21T03:56:33.988 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:56:33.988 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-09-21T03:56:33.988 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "dev", 2024-09-21T03:56:33.988 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:33.988 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "0", 2024-09-21T03:56:33.988 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.988 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.988 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.988 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:33.988 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.988 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.988 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.988 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-21T03:56:33.988 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-21T03:56:33.989 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.989 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-21T03:56:33.989 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "osd_perf_query", 2024-09-21T03:56:33.989 INFO:teuthology.orchestra.run.smithi039.stdout: "can_run": true, 2024-09-21T03:56:33.989 INFO:teuthology.orchestra.run.smithi039.stdout: "error_string": "", 2024-09-21T03:56:33.989 INFO:teuthology.orchestra.run.smithi039.stdout: "module_options": { 2024-09-21T03:56:33.989 INFO:teuthology.orchestra.run.smithi039.stdout: "log_level": { 2024-09-21T03:56:33.989 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_level", 2024-09-21T03:56:33.989 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-09-21T03:56:33.989 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.989 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:33.989 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-09-21T03:56:33.989 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.989 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.989 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-09-21T03:56:33.990 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-09-21T03:56:33.990 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-09-21T03:56:33.990 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-09-21T03:56:33.990 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-09-21T03:56:33.990 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-09-21T03:56:33.990 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-09-21T03:56:33.990 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-09-21T03:56:33.990 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:33.990 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.990 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.990 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.990 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.990 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster": { 2024-09-21T03:56:33.991 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster", 2024-09-21T03:56:33.991 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-09-21T03:56:33.991 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.991 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:33.991 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-09-21T03:56:33.991 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.991 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.991 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.991 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:33.991 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.991 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.991 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.991 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.991 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster_level": { 2024-09-21T03:56:33.992 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster_level", 2024-09-21T03:56:33.992 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-09-21T03:56:33.992 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.992 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:33.992 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "info", 2024-09-21T03:56:33.992 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.992 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.992 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-09-21T03:56:33.992 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-09-21T03:56:33.992 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-09-21T03:56:33.992 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-09-21T03:56:33.992 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-09-21T03:56:33.992 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-09-21T03:56:33.992 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-09-21T03:56:33.993 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-09-21T03:56:33.993 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:33.993 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.993 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.993 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.993 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.993 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_file": { 2024-09-21T03:56:33.993 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_file", 2024-09-21T03:56:33.993 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-09-21T03:56:33.993 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.993 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:33.993 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-09-21T03:56:33.993 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.993 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.994 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.994 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:33.994 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.994 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.994 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.994 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.994 INFO:teuthology.orchestra.run.smithi039.stdout: "sqlite3_killpoint": { 2024-09-21T03:56:33.994 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:56:33.994 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-09-21T03:56:33.994 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "dev", 2024-09-21T03:56:33.994 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:33.994 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "0", 2024-09-21T03:56:33.994 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.995 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.995 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.995 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:33.995 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.995 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.995 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.995 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-21T03:56:33.995 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-21T03:56:33.995 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.995 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-21T03:56:33.995 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "osd_support", 2024-09-21T03:56:33.995 INFO:teuthology.orchestra.run.smithi039.stdout: "can_run": true, 2024-09-21T03:56:33.995 INFO:teuthology.orchestra.run.smithi039.stdout: "error_string": "", 2024-09-21T03:56:33.995 INFO:teuthology.orchestra.run.smithi039.stdout: "module_options": { 2024-09-21T03:56:33.996 INFO:teuthology.orchestra.run.smithi039.stdout: "log_level": { 2024-09-21T03:56:33.996 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_level", 2024-09-21T03:56:33.996 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-09-21T03:56:33.996 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.996 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:33.996 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-09-21T03:56:33.996 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.996 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.996 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-09-21T03:56:33.996 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-09-21T03:56:33.996 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-09-21T03:56:33.996 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-09-21T03:56:33.996 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-09-21T03:56:33.997 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-09-21T03:56:33.997 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-09-21T03:56:33.997 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-09-21T03:56:33.997 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:33.997 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.997 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.997 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.997 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.997 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster": { 2024-09-21T03:56:33.997 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster", 2024-09-21T03:56:33.997 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-09-21T03:56:33.997 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.997 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:33.997 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-09-21T03:56:33.997 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.998 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.998 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:33.998 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:33.998 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.998 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.998 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.998 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:33.998 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster_level": { 2024-09-21T03:56:33.998 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster_level", 2024-09-21T03:56:33.998 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-09-21T03:56:33.998 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:33.998 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:33.998 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "info", 2024-09-21T03:56:33.999 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:33.999 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:33.999 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-09-21T03:56:33.999 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-09-21T03:56:33.999 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-09-21T03:56:33.999 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-09-21T03:56:33.999 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-09-21T03:56:33.999 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-09-21T03:56:33.999 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-09-21T03:56:33.999 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-09-21T03:56:33.999 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:33.999 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:33.999 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:33.999 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:33.999 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:34.000 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_file": { 2024-09-21T03:56:34.000 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_file", 2024-09-21T03:56:34.000 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-09-21T03:56:34.000 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:34.000 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:34.000 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-09-21T03:56:34.000 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:34.000 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:34.000 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:34.000 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:34.000 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:34.000 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:34.000 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:34.000 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:34.001 INFO:teuthology.orchestra.run.smithi039.stdout: "sqlite3_killpoint": { 2024-09-21T03:56:34.001 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:56:34.001 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-09-21T03:56:34.001 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "dev", 2024-09-21T03:56:34.001 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:34.001 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "0", 2024-09-21T03:56:34.001 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:34.001 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:34.001 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:34.001 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:34.001 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:34.001 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:34.001 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:34.001 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-21T03:56:34.002 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-21T03:56:34.002 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:34.002 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-21T03:56:34.002 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "pg_autoscaler", 2024-09-21T03:56:34.002 INFO:teuthology.orchestra.run.smithi039.stdout: "can_run": true, 2024-09-21T03:56:34.002 INFO:teuthology.orchestra.run.smithi039.stdout: "error_string": "", 2024-09-21T03:56:34.002 INFO:teuthology.orchestra.run.smithi039.stdout: "module_options": { 2024-09-21T03:56:34.002 INFO:teuthology.orchestra.run.smithi039.stdout: "log_level": { 2024-09-21T03:56:34.002 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_level", 2024-09-21T03:56:34.002 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-09-21T03:56:34.002 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:34.002 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:34.002 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-09-21T03:56:34.003 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:34.003 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:34.003 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-09-21T03:56:34.003 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-09-21T03:56:34.003 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-09-21T03:56:34.003 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-09-21T03:56:34.003 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-09-21T03:56:34.003 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-09-21T03:56:34.003 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-09-21T03:56:34.003 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-09-21T03:56:34.003 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:34.003 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:34.003 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:34.003 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:34.003 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:34.004 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster": { 2024-09-21T03:56:34.004 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster", 2024-09-21T03:56:34.004 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-09-21T03:56:34.004 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:34.004 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:34.004 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-09-21T03:56:34.004 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:34.004 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:34.004 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:34.004 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:34.004 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:34.004 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:34.004 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:34.004 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:34.005 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster_level": { 2024-09-21T03:56:34.005 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster_level", 2024-09-21T03:56:34.005 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-09-21T03:56:34.005 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:34.005 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:34.005 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "info", 2024-09-21T03:56:34.005 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:34.005 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:34.005 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-09-21T03:56:34.005 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-09-21T03:56:34.005 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-09-21T03:56:34.005 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-09-21T03:56:34.005 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-09-21T03:56:34.005 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-09-21T03:56:34.006 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-09-21T03:56:34.006 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-09-21T03:56:34.006 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:34.006 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:34.006 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:34.006 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:34.006 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:34.006 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_file": { 2024-09-21T03:56:34.006 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_file", 2024-09-21T03:56:34.006 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-09-21T03:56:34.006 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:34.006 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:34.006 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-09-21T03:56:34.006 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:34.007 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:34.007 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:34.007 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:34.007 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:34.007 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:34.007 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:34.007 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:34.007 INFO:teuthology.orchestra.run.smithi039.stdout: "sleep_interval": { 2024-09-21T03:56:34.007 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "sleep_interval", 2024-09-21T03:56:34.007 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "secs", 2024-09-21T03:56:34.007 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:34.007 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-09-21T03:56:34.007 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "60", 2024-09-21T03:56:34.008 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:34.008 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:34.008 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:34.008 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:34.008 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:34.008 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:34.008 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:34.008 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:34.008 INFO:teuthology.orchestra.run.smithi039.stdout: "sqlite3_killpoint": { 2024-09-21T03:56:34.008 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:56:34.008 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-09-21T03:56:34.008 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "dev", 2024-09-21T03:56:34.008 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:34.008 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "0", 2024-09-21T03:56:34.009 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:34.009 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:34.009 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:34.009 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:34.009 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:34.009 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:34.009 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:34.009 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:34.009 INFO:teuthology.orchestra.run.smithi039.stdout: "threshold": { 2024-09-21T03:56:34.009 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "threshold", 2024-09-21T03:56:34.009 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "float", 2024-09-21T03:56:34.009 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:34.009 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-09-21T03:56:34.009 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "3.0", 2024-09-21T03:56:34.010 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "1.0", 2024-09-21T03:56:34.010 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:34.010 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:34.010 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "scaling threshold", 2024-09-21T03:56:34.010 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-09-21T03:56:34.010 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:34.010 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:34.010 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-21T03:56:34.010 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-21T03:56:34.010 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:34.010 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-21T03:56:34.010 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "progress", 2024-09-21T03:56:34.010 INFO:teuthology.orchestra.run.smithi039.stdout: "can_run": true, 2024-09-21T03:56:34.011 INFO:teuthology.orchestra.run.smithi039.stdout: "error_string": "", 2024-09-21T03:56:34.011 INFO:teuthology.orchestra.run.smithi039.stdout: "module_options": { 2024-09-21T03:56:34.011 INFO:teuthology.orchestra.run.smithi039.stdout: "allow_pg_recovery_event": { 2024-09-21T03:56:34.011 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "allow_pg_recovery_event", 2024-09-21T03:56:34.011 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-09-21T03:56:34.011 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:34.011 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:34.011 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-09-21T03:56:34.011 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:34.011 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:34.011 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:34.011 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "allow the module to show pg recovery progress", 2024-09-21T03:56:34.011 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:34.011 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:34.012 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:34.012 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:34.012 INFO:teuthology.orchestra.run.smithi039.stdout: "enabled": { 2024-09-21T03:56:34.012 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "enabled", 2024-09-21T03:56:34.012 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-09-21T03:56:34.012 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:34.012 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-09-21T03:56:34.012 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2024-09-21T03:56:34.012 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:34.012 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:34.012 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:34.012 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:34.012 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:34.012 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:34.012 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:34.013 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:34.013 INFO:teuthology.orchestra.run.smithi039.stdout: "log_level": { 2024-09-21T03:56:34.013 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_level", 2024-09-21T03:56:34.013 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-09-21T03:56:34.013 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:34.013 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:34.013 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-09-21T03:56:34.013 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:34.013 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:34.013 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-09-21T03:56:34.013 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-09-21T03:56:34.013 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-09-21T03:56:34.013 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-09-21T03:56:34.013 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-09-21T03:56:34.013 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-09-21T03:56:34.014 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-09-21T03:56:34.014 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-09-21T03:56:34.014 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:34.014 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:34.014 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:34.014 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:34.014 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:34.014 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster": { 2024-09-21T03:56:34.014 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster", 2024-09-21T03:56:34.014 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-09-21T03:56:34.014 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:34.014 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:34.014 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-09-21T03:56:34.014 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:34.014 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:34.015 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:34.015 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:34.015 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:34.015 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:34.015 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:34.015 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:34.015 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster_level": { 2024-09-21T03:56:34.015 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster_level", 2024-09-21T03:56:34.015 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-09-21T03:56:34.015 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:34.015 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:34.015 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "info", 2024-09-21T03:56:34.015 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:34.015 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:34.015 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-09-21T03:56:34.016 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-09-21T03:56:34.016 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-09-21T03:56:34.016 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-09-21T03:56:34.016 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-09-21T03:56:34.016 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-09-21T03:56:34.016 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-09-21T03:56:34.016 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-09-21T03:56:34.016 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:34.016 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:34.016 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:34.016 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:34.016 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:34.016 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_file": { 2024-09-21T03:56:34.016 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_file", 2024-09-21T03:56:34.016 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-09-21T03:56:34.017 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:34.017 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:34.017 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-09-21T03:56:34.017 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:34.017 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:34.017 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:34.017 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:34.017 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:34.017 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:34.017 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:34.017 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:34.017 INFO:teuthology.orchestra.run.smithi039.stdout: "max_completed_events": { 2024-09-21T03:56:34.017 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "max_completed_events", 2024-09-21T03:56:34.017 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-09-21T03:56:34.017 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:34.018 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:34.018 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "50", 2024-09-21T03:56:34.018 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:34.018 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:34.018 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:34.018 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "number of past completed events to remember", 2024-09-21T03:56:34.018 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:34.018 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:34.018 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:34.018 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:34.018 INFO:teuthology.orchestra.run.smithi039.stdout: "sleep_interval": { 2024-09-21T03:56:34.018 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "sleep_interval", 2024-09-21T03:56:34.018 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "secs", 2024-09-21T03:56:34.018 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:34.019 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:34.019 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "5", 2024-09-21T03:56:34.019 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:34.019 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:34.019 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:34.019 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "how long the module is going to sleep", 2024-09-21T03:56:34.019 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:34.019 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:34.019 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:34.019 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:34.019 INFO:teuthology.orchestra.run.smithi039.stdout: "sqlite3_killpoint": { 2024-09-21T03:56:34.019 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:56:34.019 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-09-21T03:56:34.019 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "dev", 2024-09-21T03:56:34.019 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:34.020 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "0", 2024-09-21T03:56:34.020 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:34.020 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:34.020 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:34.020 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:34.020 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:34.020 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:34.020 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:34.020 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-21T03:56:34.020 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-21T03:56:34.020 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:34.020 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-21T03:56:34.020 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "prometheus", 2024-09-21T03:56:34.020 INFO:teuthology.orchestra.run.smithi039.stdout: "can_run": true, 2024-09-21T03:56:34.020 INFO:teuthology.orchestra.run.smithi039.stdout: "error_string": "", 2024-09-21T03:56:34.020 INFO:teuthology.orchestra.run.smithi039.stdout: "module_options": { 2024-09-21T03:56:34.021 INFO:teuthology.orchestra.run.smithi039.stdout: "cache": { 2024-09-21T03:56:34.021 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "cache", 2024-09-21T03:56:34.021 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-09-21T03:56:34.021 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:34.021 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-09-21T03:56:34.021 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2024-09-21T03:56:34.021 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:34.021 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:34.021 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:34.021 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:34.021 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:34.021 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:34.021 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:34.021 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:34.022 INFO:teuthology.orchestra.run.smithi039.stdout: "exclude_perf_counters": { 2024-09-21T03:56:34.022 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "exclude_perf_counters", 2024-09-21T03:56:34.022 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-09-21T03:56:34.022 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:34.022 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:34.022 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2024-09-21T03:56:34.022 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:34.022 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:34.022 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:34.022 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-09-21T03:56:34.022 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-09-21T03:56:34.022 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:34.022 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:34.022 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:34.023 INFO:teuthology.orchestra.run.smithi039.stdout: "log_level": { 2024-09-21T03:56:34.023 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_level", 2024-09-21T03:56:34.023 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-09-21T03:56:34.023 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:34.023 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:34.023 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-09-21T03:56:34.023 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:34.023 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:34.023 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-09-21T03:56:34.023 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-09-21T03:56:34.023 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-09-21T03:56:34.023 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-09-21T03:56:34.023 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-09-21T03:56:34.024 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-09-21T03:56:34.024 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-09-21T03:56:34.024 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-09-21T03:56:34.024 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:34.024 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:34.024 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:34.024 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:34.024 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:34.024 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster": { 2024-09-21T03:56:34.024 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster", 2024-09-21T03:56:34.024 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-09-21T03:56:34.024 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:34.024 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:34.024 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-09-21T03:56:34.025 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:34.025 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:34.025 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:34.025 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:34.025 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:34.025 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:34.025 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:34.025 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:34.025 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster_level": { 2024-09-21T03:56:34.025 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster_level", 2024-09-21T03:56:34.025 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-09-21T03:56:34.025 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:34.025 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:34.025 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "info", 2024-09-21T03:56:34.026 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:34.026 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:34.026 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-09-21T03:56:34.026 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-09-21T03:56:34.026 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-09-21T03:56:34.026 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-09-21T03:56:34.026 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-09-21T03:56:34.026 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-09-21T03:56:34.026 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-09-21T03:56:34.026 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-09-21T03:56:34.026 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:34.026 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:34.026 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:34.026 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:34.027 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:34.027 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_file": { 2024-09-21T03:56:34.027 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_file", 2024-09-21T03:56:34.027 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-09-21T03:56:34.027 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:34.027 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:34.027 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-09-21T03:56:34.027 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:34.027 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:34.027 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:34.027 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:34.027 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:34.027 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:34.027 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:34.028 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:34.028 INFO:teuthology.orchestra.run.smithi039.stdout: "rbd_stats_pools": { 2024-09-21T03:56:34.028 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "rbd_stats_pools", 2024-09-21T03:56:34.028 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-09-21T03:56:34.028 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:34.028 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-09-21T03:56:34.028 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-09-21T03:56:34.028 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:34.028 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:34.028 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:34.028 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:34.028 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:34.028 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:34.028 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:34.028 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:34.029 INFO:teuthology.orchestra.run.smithi039.stdout: "rbd_stats_pools_refresh_interval": { 2024-09-21T03:56:34.029 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-09-21T03:56:34.029 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-09-21T03:56:34.029 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:34.029 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-09-21T03:56:34.029 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "300", 2024-09-21T03:56:34.029 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:34.029 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:34.029 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:34.029 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:34.029 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:34.029 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:34.029 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:34.029 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:34.030 INFO:teuthology.orchestra.run.smithi039.stdout: "scrape_interval": { 2024-09-21T03:56:34.030 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "scrape_interval", 2024-09-21T03:56:34.030 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "float", 2024-09-21T03:56:34.030 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:34.030 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-09-21T03:56:34.030 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "15.0", 2024-09-21T03:56:34.030 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:34.030 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:34.030 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:34.030 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:34.030 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:34.030 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:34.030 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:34.030 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:34.031 INFO:teuthology.orchestra.run.smithi039.stdout: "server_addr": { 2024-09-21T03:56:34.031 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "server_addr", 2024-09-21T03:56:34.031 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-09-21T03:56:34.031 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:34.031 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-09-21T03:56:34.031 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "::", 2024-09-21T03:56:34.031 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:34.031 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:34.031 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:34.031 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-09-21T03:56:34.031 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:34.031 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:34.031 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:34.031 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:34.032 INFO:teuthology.orchestra.run.smithi039.stdout: "server_port": { 2024-09-21T03:56:34.032 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "server_port", 2024-09-21T03:56:34.032 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-09-21T03:56:34.032 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:34.032 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:34.032 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "9283", 2024-09-21T03:56:34.032 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:34.032 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:34.032 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:34.032 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-09-21T03:56:34.032 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:34.032 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:34.032 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:34.032 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:34.033 INFO:teuthology.orchestra.run.smithi039.stdout: "sqlite3_killpoint": { 2024-09-21T03:56:34.033 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:56:34.033 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-09-21T03:56:34.033 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "dev", 2024-09-21T03:56:34.033 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:34.033 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "0", 2024-09-21T03:56:34.033 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:34.033 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:34.033 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:34.033 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:34.033 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:34.033 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:34.033 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:34.033 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:34.034 INFO:teuthology.orchestra.run.smithi039.stdout: "stale_cache_strategy": { 2024-09-21T03:56:34.034 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "stale_cache_strategy", 2024-09-21T03:56:34.034 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-09-21T03:56:34.034 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:34.034 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-09-21T03:56:34.034 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "log", 2024-09-21T03:56:34.034 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:34.034 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:34.034 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:34.034 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:34.034 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:34.034 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:34.034 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:34.034 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:34.035 INFO:teuthology.orchestra.run.smithi039.stdout: "standby_behaviour": { 2024-09-21T03:56:34.035 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "standby_behaviour", 2024-09-21T03:56:34.035 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-09-21T03:56:34.035 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:34.035 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:34.035 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "default", 2024-09-21T03:56:34.035 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:34.035 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:34.035 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-09-21T03:56:34.035 INFO:teuthology.orchestra.run.smithi039.stdout: "default", 2024-09-21T03:56:34.035 INFO:teuthology.orchestra.run.smithi039.stdout: "error" 2024-09-21T03:56:34.035 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-09-21T03:56:34.035 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:34.035 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:34.035 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:34.036 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:34.036 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:34.036 INFO:teuthology.orchestra.run.smithi039.stdout: "standby_error_status_code": { 2024-09-21T03:56:34.036 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "standby_error_status_code", 2024-09-21T03:56:34.036 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-09-21T03:56:34.036 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:34.036 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:34.036 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "500", 2024-09-21T03:56:34.036 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "400", 2024-09-21T03:56:34.036 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "599", 2024-09-21T03:56:34.036 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:34.036 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:34.036 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:34.036 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:34.036 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:34.036 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-21T03:56:34.036 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-21T03:56:34.037 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:34.037 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-21T03:56:34.037 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "rbd_support", 2024-09-21T03:56:34.037 INFO:teuthology.orchestra.run.smithi039.stdout: "can_run": true, 2024-09-21T03:56:34.037 INFO:teuthology.orchestra.run.smithi039.stdout: "error_string": "", 2024-09-21T03:56:34.037 INFO:teuthology.orchestra.run.smithi039.stdout: "module_options": { 2024-09-21T03:56:34.037 INFO:teuthology.orchestra.run.smithi039.stdout: "log_level": { 2024-09-21T03:56:34.037 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_level", 2024-09-21T03:56:34.037 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-09-21T03:56:34.037 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:34.037 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:34.037 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-09-21T03:56:34.037 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:34.037 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:34.038 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-09-21T03:56:34.038 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-09-21T03:56:34.038 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-09-21T03:56:34.038 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-09-21T03:56:34.038 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-09-21T03:56:34.038 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-09-21T03:56:34.038 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-09-21T03:56:34.038 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-09-21T03:56:34.038 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:34.038 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:34.038 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:34.038 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:34.038 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:34.038 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster": { 2024-09-21T03:56:34.039 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster", 2024-09-21T03:56:34.039 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-09-21T03:56:34.039 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:34.039 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:34.039 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-09-21T03:56:34.039 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:34.039 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:34.039 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:34.039 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:34.039 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:34.039 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:34.039 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:34.039 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:34.039 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster_level": { 2024-09-21T03:56:34.040 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster_level", 2024-09-21T03:56:34.040 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-09-21T03:56:34.040 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:34.040 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:34.040 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "info", 2024-09-21T03:56:34.040 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:34.040 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:34.040 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-09-21T03:56:34.040 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-09-21T03:56:34.040 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-09-21T03:56:34.040 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-09-21T03:56:34.040 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-09-21T03:56:34.040 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-09-21T03:56:34.040 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-09-21T03:56:34.040 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-09-21T03:56:34.041 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:34.041 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:34.041 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:34.041 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:34.041 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:34.041 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_file": { 2024-09-21T03:56:34.041 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_file", 2024-09-21T03:56:34.041 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-09-21T03:56:34.041 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:34.041 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:34.041 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-09-21T03:56:34.041 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:34.041 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:34.041 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:34.041 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:34.042 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:34.042 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:34.042 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:34.042 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:34.042 INFO:teuthology.orchestra.run.smithi039.stdout: "max_concurrent_snap_create": { 2024-09-21T03:56:34.042 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "max_concurrent_snap_create", 2024-09-21T03:56:34.042 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-09-21T03:56:34.042 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:34.042 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-09-21T03:56:34.042 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "10", 2024-09-21T03:56:34.042 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:34.042 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:34.042 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:34.043 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:34.043 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:34.043 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:34.043 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:34.043 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:34.043 INFO:teuthology.orchestra.run.smithi039.stdout: "mirror_snapshot_schedule": { 2024-09-21T03:56:34.043 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "mirror_snapshot_schedule", 2024-09-21T03:56:34.043 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-09-21T03:56:34.043 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:34.043 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-09-21T03:56:34.043 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-09-21T03:56:34.043 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:34.043 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:34.043 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:34.044 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:34.044 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:34.044 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:34.044 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:34.044 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:34.044 INFO:teuthology.orchestra.run.smithi039.stdout: "sqlite3_killpoint": { 2024-09-21T03:56:34.044 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:56:34.044 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-09-21T03:56:34.044 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "dev", 2024-09-21T03:56:34.044 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:34.044 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "0", 2024-09-21T03:56:34.044 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:34.044 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:34.044 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:34.045 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:34.045 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:34.045 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:34.045 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:34.045 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:34.045 INFO:teuthology.orchestra.run.smithi039.stdout: "trash_purge_schedule": { 2024-09-21T03:56:34.045 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "trash_purge_schedule", 2024-09-21T03:56:34.045 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-09-21T03:56:34.045 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:34.045 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-09-21T03:56:34.045 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-09-21T03:56:34.045 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:34.045 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:34.046 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:34.046 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:34.046 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:34.046 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:34.046 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:34.046 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-21T03:56:34.046 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-21T03:56:34.046 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:34.046 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-21T03:56:34.046 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "restful", 2024-09-21T03:56:34.046 INFO:teuthology.orchestra.run.smithi039.stdout: "can_run": true, 2024-09-21T03:56:34.046 INFO:teuthology.orchestra.run.smithi039.stdout: "error_string": "", 2024-09-21T03:56:34.046 INFO:teuthology.orchestra.run.smithi039.stdout: "module_options": { 2024-09-21T03:56:34.046 INFO:teuthology.orchestra.run.smithi039.stdout: "enable_auth": { 2024-09-21T03:56:34.047 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "enable_auth", 2024-09-21T03:56:34.047 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-09-21T03:56:34.047 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:34.047 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-09-21T03:56:34.047 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2024-09-21T03:56:34.047 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:34.047 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:34.047 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:34.047 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:34.047 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:34.047 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:34.047 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:34.047 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:34.047 INFO:teuthology.orchestra.run.smithi039.stdout: "key_file": { 2024-09-21T03:56:34.047 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "key_file", 2024-09-21T03:56:34.048 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-09-21T03:56:34.048 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:34.048 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-09-21T03:56:34.048 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-09-21T03:56:34.048 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:34.048 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:34.048 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:34.048 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:34.048 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:34.048 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:34.048 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:34.048 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:34.048 INFO:teuthology.orchestra.run.smithi039.stdout: "log_level": { 2024-09-21T03:56:34.048 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_level", 2024-09-21T03:56:34.048 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-09-21T03:56:34.048 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:34.049 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:34.049 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-09-21T03:56:34.049 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:34.049 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:34.049 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-09-21T03:56:34.049 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-09-21T03:56:34.049 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-09-21T03:56:34.049 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-09-21T03:56:34.049 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-09-21T03:56:34.049 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-09-21T03:56:34.049 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-09-21T03:56:34.049 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-09-21T03:56:34.049 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:34.049 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:34.049 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:34.049 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:34.050 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:34.050 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster": { 2024-09-21T03:56:34.050 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster", 2024-09-21T03:56:34.050 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-09-21T03:56:34.050 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:34.050 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:34.050 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-09-21T03:56:34.050 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:34.050 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:34.050 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:34.050 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:34.050 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:34.050 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:34.051 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:34.051 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:34.051 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster_level": { 2024-09-21T03:56:34.051 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster_level", 2024-09-21T03:56:34.051 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-09-21T03:56:34.051 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:34.051 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:34.051 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "info", 2024-09-21T03:56:34.051 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:34.051 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:34.051 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-09-21T03:56:34.051 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-09-21T03:56:34.051 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-09-21T03:56:34.051 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-09-21T03:56:34.051 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-09-21T03:56:34.051 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-09-21T03:56:34.052 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-09-21T03:56:34.052 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-09-21T03:56:34.052 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:34.052 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:34.052 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:34.052 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:34.052 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:34.052 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_file": { 2024-09-21T03:56:34.052 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_file", 2024-09-21T03:56:34.052 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-09-21T03:56:34.052 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:34.052 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:34.052 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-09-21T03:56:34.052 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:34.052 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:34.053 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:34.053 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:34.053 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:34.053 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:34.053 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:34.053 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:34.053 INFO:teuthology.orchestra.run.smithi039.stdout: "max_requests": { 2024-09-21T03:56:34.053 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "max_requests", 2024-09-21T03:56:34.053 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-09-21T03:56:34.053 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:34.053 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-09-21T03:56:34.053 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "500", 2024-09-21T03:56:34.053 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:34.053 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:34.053 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:34.053 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Maximum number of requests to keep in memory. When new request comes in, the oldest request will be removed if the number of requests exceeds the max request number.if un-finished request is removed, error message will be logged in the ceph-mgr log.", 2024-09-21T03:56:34.054 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:34.054 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:34.054 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:34.054 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:34.054 INFO:teuthology.orchestra.run.smithi039.stdout: "server_addr": { 2024-09-21T03:56:34.054 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "server_addr", 2024-09-21T03:56:34.054 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-09-21T03:56:34.054 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:34.054 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-09-21T03:56:34.054 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-09-21T03:56:34.054 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:34.054 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:34.054 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:34.054 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:34.054 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:34.055 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:34.055 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:34.055 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:34.055 INFO:teuthology.orchestra.run.smithi039.stdout: "server_port": { 2024-09-21T03:56:34.055 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "server_port", 2024-09-21T03:56:34.055 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-09-21T03:56:34.055 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:34.055 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-09-21T03:56:34.055 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-09-21T03:56:34.055 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:34.055 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:34.055 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:34.055 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:34.055 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:34.056 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:34.056 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:34.056 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:34.056 INFO:teuthology.orchestra.run.smithi039.stdout: "sqlite3_killpoint": { 2024-09-21T03:56:34.056 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:56:34.056 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-09-21T03:56:34.056 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "dev", 2024-09-21T03:56:34.056 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:34.056 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "0", 2024-09-21T03:56:34.056 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:34.056 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:34.056 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:34.056 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:34.056 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:34.057 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:34.057 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:34.057 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-21T03:56:34.057 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-21T03:56:34.057 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:34.057 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-21T03:56:34.057 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "rgw", 2024-09-21T03:56:34.057 INFO:teuthology.orchestra.run.smithi039.stdout: "can_run": true, 2024-09-21T03:56:34.057 INFO:teuthology.orchestra.run.smithi039.stdout: "error_string": "", 2024-09-21T03:56:34.057 INFO:teuthology.orchestra.run.smithi039.stdout: "module_options": { 2024-09-21T03:56:34.057 INFO:teuthology.orchestra.run.smithi039.stdout: "log_level": { 2024-09-21T03:56:34.057 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_level", 2024-09-21T03:56:34.057 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-09-21T03:56:34.057 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:34.058 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:34.058 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-09-21T03:56:34.058 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:34.058 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:34.058 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-09-21T03:56:34.058 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-09-21T03:56:34.058 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-09-21T03:56:34.058 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-09-21T03:56:34.058 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-09-21T03:56:34.058 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-09-21T03:56:34.058 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-09-21T03:56:34.058 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-09-21T03:56:34.058 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:34.058 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:34.059 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:34.059 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:34.059 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:34.059 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster": { 2024-09-21T03:56:34.059 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster", 2024-09-21T03:56:34.059 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-09-21T03:56:34.059 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:34.059 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:34.059 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-09-21T03:56:34.059 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:34.059 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:34.059 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:34.059 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:34.059 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:34.060 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:34.060 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:34.060 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:34.060 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster_level": { 2024-09-21T03:56:34.060 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster_level", 2024-09-21T03:56:34.060 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-09-21T03:56:34.060 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:34.060 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:34.060 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "info", 2024-09-21T03:56:34.060 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:34.060 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:34.060 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-09-21T03:56:34.060 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-09-21T03:56:34.061 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-09-21T03:56:34.061 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-09-21T03:56:34.061 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-09-21T03:56:34.061 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-09-21T03:56:34.061 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-09-21T03:56:34.061 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-09-21T03:56:34.061 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:34.061 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:34.061 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:34.061 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:34.061 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:34.061 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_file": { 2024-09-21T03:56:34.061 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_file", 2024-09-21T03:56:34.061 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-09-21T03:56:34.061 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:34.062 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:34.062 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-09-21T03:56:34.062 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:34.062 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:34.062 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:34.062 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:34.062 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:34.062 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:34.062 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:34.062 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:34.062 INFO:teuthology.orchestra.run.smithi039.stdout: "secondary_zone_period_retry_limit": { 2024-09-21T03:56:34.062 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "secondary_zone_period_retry_limit", 2024-09-21T03:56:34.062 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-09-21T03:56:34.062 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:34.063 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-09-21T03:56:34.063 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "5", 2024-09-21T03:56:34.063 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:34.063 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:34.063 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:34.063 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "RGW module period update retry limit for secondary site", 2024-09-21T03:56:34.063 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:34.063 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:34.063 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:34.063 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:34.063 INFO:teuthology.orchestra.run.smithi039.stdout: "sqlite3_killpoint": { 2024-09-21T03:56:34.063 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:56:34.063 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-09-21T03:56:34.063 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "dev", 2024-09-21T03:56:34.064 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:34.064 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "0", 2024-09-21T03:56:34.064 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:34.064 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:34.064 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:34.064 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:34.064 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:34.064 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:34.064 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:34.064 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-21T03:56:34.064 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-21T03:56:34.064 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:34.064 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-21T03:56:34.064 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "rook", 2024-09-21T03:56:34.065 INFO:teuthology.orchestra.run.smithi039.stdout: "can_run": true, 2024-09-21T03:56:34.065 INFO:teuthology.orchestra.run.smithi039.stdout: "error_string": "", 2024-09-21T03:56:34.065 INFO:teuthology.orchestra.run.smithi039.stdout: "module_options": { 2024-09-21T03:56:34.065 INFO:teuthology.orchestra.run.smithi039.stdout: "log_level": { 2024-09-21T03:56:34.065 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_level", 2024-09-21T03:56:34.065 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-09-21T03:56:34.065 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:34.065 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:34.065 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-09-21T03:56:34.065 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:34.065 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:34.065 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-09-21T03:56:34.065 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-09-21T03:56:34.065 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-09-21T03:56:34.065 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-09-21T03:56:34.066 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-09-21T03:56:34.066 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-09-21T03:56:34.066 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-09-21T03:56:34.066 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-09-21T03:56:34.066 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:34.066 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:34.066 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:34.066 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:34.066 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:34.066 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster": { 2024-09-21T03:56:34.066 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster", 2024-09-21T03:56:34.066 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-09-21T03:56:34.066 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:34.066 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:34.067 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-09-21T03:56:34.067 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:34.067 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:34.067 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:34.067 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:34.067 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:34.067 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:34.067 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:34.067 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:34.067 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster_level": { 2024-09-21T03:56:34.067 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster_level", 2024-09-21T03:56:34.067 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-09-21T03:56:34.067 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:34.067 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:34.067 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "info", 2024-09-21T03:56:34.068 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:34.068 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:34.068 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-09-21T03:56:34.068 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-09-21T03:56:34.068 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-09-21T03:56:34.068 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-09-21T03:56:34.068 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-09-21T03:56:34.068 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-09-21T03:56:34.068 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-09-21T03:56:34.068 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-09-21T03:56:34.068 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:34.068 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:34.068 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:34.068 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:34.069 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:34.069 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_file": { 2024-09-21T03:56:34.069 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_file", 2024-09-21T03:56:34.069 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-09-21T03:56:34.069 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:34.069 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:34.069 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-09-21T03:56:34.069 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:34.069 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:34.069 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:34.069 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:34.069 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:34.069 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:34.069 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:34.070 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:34.070 INFO:teuthology.orchestra.run.smithi039.stdout: "sqlite3_killpoint": { 2024-09-21T03:56:34.070 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:56:34.070 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-09-21T03:56:34.070 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "dev", 2024-09-21T03:56:34.070 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:34.070 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "0", 2024-09-21T03:56:34.070 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:34.070 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:34.070 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:34.070 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:34.070 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:34.070 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:34.070 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:34.071 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:34.071 INFO:teuthology.orchestra.run.smithi039.stdout: "storage_class": { 2024-09-21T03:56:34.071 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "storage_class", 2024-09-21T03:56:34.071 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-09-21T03:56:34.071 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:34.071 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-09-21T03:56:34.071 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "local", 2024-09-21T03:56:34.071 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:34.071 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:34.071 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:34.071 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-09-21T03:56:34.071 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:34.071 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:34.071 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:34.071 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-21T03:56:34.072 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-21T03:56:34.072 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:34.072 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-21T03:56:34.072 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "selftest", 2024-09-21T03:56:34.072 INFO:teuthology.orchestra.run.smithi039.stdout: "can_run": true, 2024-09-21T03:56:34.072 INFO:teuthology.orchestra.run.smithi039.stdout: "error_string": "", 2024-09-21T03:56:34.072 INFO:teuthology.orchestra.run.smithi039.stdout: "module_options": { 2024-09-21T03:56:34.072 INFO:teuthology.orchestra.run.smithi039.stdout: "log_level": { 2024-09-21T03:56:34.072 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_level", 2024-09-21T03:56:34.072 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-09-21T03:56:34.072 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:34.072 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:34.072 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-09-21T03:56:34.072 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:34.072 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:34.073 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-09-21T03:56:34.073 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-09-21T03:56:34.073 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-09-21T03:56:34.073 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-09-21T03:56:34.073 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-09-21T03:56:34.073 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-09-21T03:56:34.073 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-09-21T03:56:34.073 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-09-21T03:56:34.073 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:34.073 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:34.073 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:34.073 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:34.073 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:34.073 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster": { 2024-09-21T03:56:34.073 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster", 2024-09-21T03:56:34.074 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-09-21T03:56:34.074 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:34.074 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:34.074 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-09-21T03:56:34.074 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:34.074 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:34.074 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:34.074 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:34.074 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:34.074 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:34.074 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:34.074 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:34.074 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster_level": { 2024-09-21T03:56:34.074 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster_level", 2024-09-21T03:56:34.075 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-09-21T03:56:34.075 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:34.075 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:34.075 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "info", 2024-09-21T03:56:34.075 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:34.075 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:34.075 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-09-21T03:56:34.075 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-09-21T03:56:34.075 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-09-21T03:56:34.075 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-09-21T03:56:34.075 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-09-21T03:56:34.075 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-09-21T03:56:34.075 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-09-21T03:56:34.075 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-09-21T03:56:34.076 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:34.076 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:34.076 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:34.076 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:34.076 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:34.076 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_file": { 2024-09-21T03:56:34.076 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_file", 2024-09-21T03:56:34.076 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-09-21T03:56:34.076 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:34.076 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:34.076 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-09-21T03:56:34.076 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:34.076 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:34.076 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:34.077 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:34.077 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:34.077 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:34.077 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:34.077 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:34.077 INFO:teuthology.orchestra.run.smithi039.stdout: "roption1": { 2024-09-21T03:56:34.077 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "roption1", 2024-09-21T03:56:34.077 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-09-21T03:56:34.077 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:34.077 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-09-21T03:56:34.077 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-09-21T03:56:34.077 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:34.077 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:34.077 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:34.077 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:34.078 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:34.078 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:34.078 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:34.078 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:34.078 INFO:teuthology.orchestra.run.smithi039.stdout: "roption2": { 2024-09-21T03:56:34.078 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "roption2", 2024-09-21T03:56:34.078 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-09-21T03:56:34.078 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:34.078 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-09-21T03:56:34.078 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "xyz", 2024-09-21T03:56:34.078 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:34.078 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:34.078 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:34.078 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:34.079 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:34.079 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:34.079 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:34.079 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:34.079 INFO:teuthology.orchestra.run.smithi039.stdout: "rwoption1": { 2024-09-21T03:56:34.079 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "rwoption1", 2024-09-21T03:56:34.079 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-09-21T03:56:34.079 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:34.079 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-09-21T03:56:34.079 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-09-21T03:56:34.079 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:34.079 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:34.079 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:34.079 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:34.079 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:34.080 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:34.080 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:34.080 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:34.080 INFO:teuthology.orchestra.run.smithi039.stdout: "rwoption2": { 2024-09-21T03:56:34.080 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "rwoption2", 2024-09-21T03:56:34.080 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-09-21T03:56:34.080 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:34.080 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-09-21T03:56:34.080 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-09-21T03:56:34.080 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:34.080 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:34.080 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:34.080 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:34.080 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:34.080 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:34.081 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:34.081 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:34.081 INFO:teuthology.orchestra.run.smithi039.stdout: "rwoption3": { 2024-09-21T03:56:34.081 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "rwoption3", 2024-09-21T03:56:34.081 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "float", 2024-09-21T03:56:34.081 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:34.081 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-09-21T03:56:34.081 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-09-21T03:56:34.081 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:34.081 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:34.081 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:34.081 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:34.081 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:34.081 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:34.082 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:34.082 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:34.082 INFO:teuthology.orchestra.run.smithi039.stdout: "rwoption4": { 2024-09-21T03:56:34.082 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "rwoption4", 2024-09-21T03:56:34.082 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-09-21T03:56:34.082 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:34.082 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-09-21T03:56:34.082 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-09-21T03:56:34.082 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:34.082 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:34.082 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:34.082 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:34.082 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:34.082 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:34.082 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:34.083 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:34.083 INFO:teuthology.orchestra.run.smithi039.stdout: "rwoption5": { 2024-09-21T03:56:34.083 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "rwoption5", 2024-09-21T03:56:34.083 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-09-21T03:56:34.083 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:34.083 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-09-21T03:56:34.083 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-09-21T03:56:34.083 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:34.083 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:34.083 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:34.083 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:34.083 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:34.083 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:34.083 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:34.083 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:34.084 INFO:teuthology.orchestra.run.smithi039.stdout: "rwoption6": { 2024-09-21T03:56:34.084 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "rwoption6", 2024-09-21T03:56:34.084 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-09-21T03:56:34.084 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:34.084 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-09-21T03:56:34.084 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2024-09-21T03:56:34.084 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:34.084 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:34.084 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:34.084 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:34.084 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:34.084 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:34.084 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:34.084 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:34.084 INFO:teuthology.orchestra.run.smithi039.stdout: "rwoption7": { 2024-09-21T03:56:34.085 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "rwoption7", 2024-09-21T03:56:34.085 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-09-21T03:56:34.085 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:34.085 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-09-21T03:56:34.085 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-09-21T03:56:34.085 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "1", 2024-09-21T03:56:34.085 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "42", 2024-09-21T03:56:34.085 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:34.085 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:34.085 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:34.085 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:34.085 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:34.085 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:34.085 INFO:teuthology.orchestra.run.smithi039.stdout: "sqlite3_killpoint": { 2024-09-21T03:56:34.085 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:56:34.086 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-09-21T03:56:34.086 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "dev", 2024-09-21T03:56:34.086 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:34.086 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "0", 2024-09-21T03:56:34.086 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:34.086 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:34.086 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:34.086 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:34.086 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:34.086 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:34.086 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:34.086 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:34.086 INFO:teuthology.orchestra.run.smithi039.stdout: "testkey": { 2024-09-21T03:56:34.086 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "testkey", 2024-09-21T03:56:34.087 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-09-21T03:56:34.087 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:34.087 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-09-21T03:56:34.087 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-09-21T03:56:34.087 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:34.087 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:34.087 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:34.087 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:34.087 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:34.087 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:34.087 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:34.087 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:34.087 INFO:teuthology.orchestra.run.smithi039.stdout: "testlkey": { 2024-09-21T03:56:34.087 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "testlkey", 2024-09-21T03:56:34.087 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-09-21T03:56:34.088 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:34.088 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-09-21T03:56:34.088 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-09-21T03:56:34.088 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:34.088 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:34.088 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:34.088 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:34.088 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:34.088 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:34.088 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:34.088 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:34.088 INFO:teuthology.orchestra.run.smithi039.stdout: "testnewline": { 2024-09-21T03:56:34.088 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "testnewline", 2024-09-21T03:56:34.088 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-09-21T03:56:34.088 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:34.089 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-09-21T03:56:34.089 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-09-21T03:56:34.089 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:34.089 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:34.089 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:34.089 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:34.089 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:34.089 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:34.089 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:34.089 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-21T03:56:34.089 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-21T03:56:34.089 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:34.089 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-21T03:56:34.089 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "smb", 2024-09-21T03:56:34.089 INFO:teuthology.orchestra.run.smithi039.stdout: "can_run": true, 2024-09-21T03:56:34.090 INFO:teuthology.orchestra.run.smithi039.stdout: "error_string": "", 2024-09-21T03:56:34.090 INFO:teuthology.orchestra.run.smithi039.stdout: "module_options": { 2024-09-21T03:56:34.090 INFO:teuthology.orchestra.run.smithi039.stdout: "internal_store_backend": { 2024-09-21T03:56:34.090 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "internal_store_backend", 2024-09-21T03:56:34.090 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-09-21T03:56:34.090 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "dev", 2024-09-21T03:56:34.090 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-09-21T03:56:34.090 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-09-21T03:56:34.090 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:34.090 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:34.090 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:34.090 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "set internal store backend. for develoment and testing only", 2024-09-21T03:56:34.090 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:34.090 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:34.091 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:34.091 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:34.091 INFO:teuthology.orchestra.run.smithi039.stdout: "log_level": { 2024-09-21T03:56:34.091 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_level", 2024-09-21T03:56:34.091 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-09-21T03:56:34.091 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:34.091 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:34.091 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-09-21T03:56:34.091 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:34.091 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:34.091 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-09-21T03:56:34.091 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-09-21T03:56:34.091 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-09-21T03:56:34.091 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-09-21T03:56:34.092 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-09-21T03:56:34.092 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-09-21T03:56:34.092 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-09-21T03:56:34.092 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-09-21T03:56:34.092 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:34.092 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:34.092 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:34.092 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:34.092 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:34.092 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster": { 2024-09-21T03:56:34.092 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster", 2024-09-21T03:56:34.092 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-09-21T03:56:34.092 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:34.092 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:34.093 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-09-21T03:56:34.093 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:34.093 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:34.093 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:34.093 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:34.093 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:34.093 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:34.093 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:34.093 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:34.093 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster_level": { 2024-09-21T03:56:34.093 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster_level", 2024-09-21T03:56:34.093 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-09-21T03:56:34.093 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:34.093 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:34.093 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "info", 2024-09-21T03:56:34.094 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:34.094 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:34.094 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-09-21T03:56:34.094 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-09-21T03:56:34.094 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-09-21T03:56:34.094 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-09-21T03:56:34.094 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-09-21T03:56:34.094 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-09-21T03:56:34.094 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-09-21T03:56:34.094 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-09-21T03:56:34.094 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:34.094 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:34.094 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:34.094 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:34.095 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:34.095 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_file": { 2024-09-21T03:56:34.095 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_file", 2024-09-21T03:56:34.095 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-09-21T03:56:34.095 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:34.095 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:34.095 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-09-21T03:56:34.095 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:34.095 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:34.095 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:34.095 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:34.095 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:34.095 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:34.095 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:34.095 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:34.096 INFO:teuthology.orchestra.run.smithi039.stdout: "sqlite3_killpoint": { 2024-09-21T03:56:34.096 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:56:34.096 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-09-21T03:56:34.096 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "dev", 2024-09-21T03:56:34.096 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:34.096 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "0", 2024-09-21T03:56:34.096 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:34.096 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:34.096 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:34.096 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:34.096 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:34.096 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:34.096 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:34.096 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:34.097 INFO:teuthology.orchestra.run.smithi039.stdout: "update_orchestration": { 2024-09-21T03:56:34.097 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "update_orchestration", 2024-09-21T03:56:34.097 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-09-21T03:56:34.097 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:34.097 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-09-21T03:56:34.097 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2024-09-21T03:56:34.097 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:34.097 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:34.097 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:34.097 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "automatically update orchestration when smb resources are changed", 2024-09-21T03:56:34.097 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:34.097 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:34.097 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:34.097 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-21T03:56:34.097 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-21T03:56:34.098 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:34.098 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-21T03:56:34.098 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "snap_schedule", 2024-09-21T03:56:34.098 INFO:teuthology.orchestra.run.smithi039.stdout: "can_run": true, 2024-09-21T03:56:34.098 INFO:teuthology.orchestra.run.smithi039.stdout: "error_string": "", 2024-09-21T03:56:34.098 INFO:teuthology.orchestra.run.smithi039.stdout: "module_options": { 2024-09-21T03:56:34.098 INFO:teuthology.orchestra.run.smithi039.stdout: "allow_m_granularity": { 2024-09-21T03:56:34.098 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "allow_m_granularity", 2024-09-21T03:56:34.098 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-09-21T03:56:34.098 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:34.098 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:34.098 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-09-21T03:56:34.098 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:34.098 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:34.099 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:34.099 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "allow minute scheduled snapshots", 2024-09-21T03:56:34.099 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:34.099 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:34.099 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:34.099 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:34.099 INFO:teuthology.orchestra.run.smithi039.stdout: "dump_on_update": { 2024-09-21T03:56:34.099 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "dump_on_update", 2024-09-21T03:56:34.099 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-09-21T03:56:34.099 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:34.099 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:34.099 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-09-21T03:56:34.099 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:34.099 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:34.100 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:34.100 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "dump database to debug log on update", 2024-09-21T03:56:34.100 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:34.100 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:34.100 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:34.100 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:34.100 INFO:teuthology.orchestra.run.smithi039.stdout: "log_level": { 2024-09-21T03:56:34.100 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_level", 2024-09-21T03:56:34.100 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-09-21T03:56:34.100 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:34.100 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:34.100 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-09-21T03:56:34.100 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:34.100 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:34.100 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-09-21T03:56:34.100 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-09-21T03:56:34.101 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-09-21T03:56:34.101 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-09-21T03:56:34.101 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-09-21T03:56:34.101 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-09-21T03:56:34.101 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-09-21T03:56:34.101 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-09-21T03:56:34.101 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:34.101 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:34.101 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:34.101 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:34.101 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:34.101 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster": { 2024-09-21T03:56:34.101 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster", 2024-09-21T03:56:34.101 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-09-21T03:56:34.102 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:34.102 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:34.102 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-09-21T03:56:34.102 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:34.102 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:34.102 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:34.102 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:34.102 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:34.102 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:34.102 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:34.102 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:34.102 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster_level": { 2024-09-21T03:56:34.102 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster_level", 2024-09-21T03:56:34.102 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-09-21T03:56:34.103 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:34.103 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:34.103 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "info", 2024-09-21T03:56:34.103 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:34.103 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:34.103 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-09-21T03:56:34.103 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-09-21T03:56:34.103 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-09-21T03:56:34.103 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-09-21T03:56:34.103 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-09-21T03:56:34.103 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-09-21T03:56:34.103 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-09-21T03:56:34.103 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-09-21T03:56:34.103 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:34.103 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:34.104 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:34.104 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:34.104 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:34.104 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_file": { 2024-09-21T03:56:34.104 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_file", 2024-09-21T03:56:34.104 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-09-21T03:56:34.104 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:34.104 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:34.104 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-09-21T03:56:34.104 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:34.104 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:34.104 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:34.104 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:34.105 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:34.105 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:34.105 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:34.105 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:34.105 INFO:teuthology.orchestra.run.smithi039.stdout: "sqlite3_killpoint": { 2024-09-21T03:56:34.105 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:56:34.105 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-09-21T03:56:34.105 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "dev", 2024-09-21T03:56:34.105 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:34.105 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "0", 2024-09-21T03:56:34.105 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:34.105 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:34.105 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:34.105 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:34.106 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:34.106 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:34.106 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:34.106 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-21T03:56:34.106 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-21T03:56:34.106 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:34.106 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-21T03:56:34.106 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "stats", 2024-09-21T03:56:34.106 INFO:teuthology.orchestra.run.smithi039.stdout: "can_run": true, 2024-09-21T03:56:34.106 INFO:teuthology.orchestra.run.smithi039.stdout: "error_string": "", 2024-09-21T03:56:34.106 INFO:teuthology.orchestra.run.smithi039.stdout: "module_options": { 2024-09-21T03:56:34.106 INFO:teuthology.orchestra.run.smithi039.stdout: "log_level": { 2024-09-21T03:56:34.106 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_level", 2024-09-21T03:56:34.106 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-09-21T03:56:34.107 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:34.107 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:34.107 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-09-21T03:56:34.107 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:34.107 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:34.107 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-09-21T03:56:34.107 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-09-21T03:56:34.107 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-09-21T03:56:34.107 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-09-21T03:56:34.107 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-09-21T03:56:34.107 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-09-21T03:56:34.107 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-09-21T03:56:34.107 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-09-21T03:56:34.107 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:34.108 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:34.108 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:34.108 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:34.108 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:34.108 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster": { 2024-09-21T03:56:34.108 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster", 2024-09-21T03:56:34.108 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-09-21T03:56:34.108 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:34.108 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:34.108 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-09-21T03:56:34.108 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:34.108 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:34.108 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:34.108 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:34.108 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:34.108 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:34.109 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:34.109 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:34.109 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster_level": { 2024-09-21T03:56:34.109 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster_level", 2024-09-21T03:56:34.109 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-09-21T03:56:34.109 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:34.109 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:34.109 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "info", 2024-09-21T03:56:34.109 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:34.109 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:34.109 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-09-21T03:56:34.109 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-09-21T03:56:34.109 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-09-21T03:56:34.109 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-09-21T03:56:34.109 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-09-21T03:56:34.110 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-09-21T03:56:34.110 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-09-21T03:56:34.110 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-09-21T03:56:34.110 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:34.110 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:34.110 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:34.110 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:34.110 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:34.110 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_file": { 2024-09-21T03:56:34.110 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_file", 2024-09-21T03:56:34.110 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-09-21T03:56:34.110 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:34.110 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:34.110 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-09-21T03:56:34.111 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:34.111 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:34.111 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:34.111 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:34.111 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:34.111 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:34.111 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:34.111 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:34.111 INFO:teuthology.orchestra.run.smithi039.stdout: "sqlite3_killpoint": { 2024-09-21T03:56:34.111 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:56:34.111 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-09-21T03:56:34.111 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "dev", 2024-09-21T03:56:34.111 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:34.111 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "0", 2024-09-21T03:56:34.112 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:34.112 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:34.112 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:34.112 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:34.112 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:34.112 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:34.112 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:34.112 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-21T03:56:34.112 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-21T03:56:34.112 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:34.112 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-21T03:56:34.112 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "status", 2024-09-21T03:56:34.112 INFO:teuthology.orchestra.run.smithi039.stdout: "can_run": true, 2024-09-21T03:56:34.112 INFO:teuthology.orchestra.run.smithi039.stdout: "error_string": "", 2024-09-21T03:56:34.112 INFO:teuthology.orchestra.run.smithi039.stdout: "module_options": { 2024-09-21T03:56:34.112 INFO:teuthology.orchestra.run.smithi039.stdout: "log_level": { 2024-09-21T03:56:34.113 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_level", 2024-09-21T03:56:34.113 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-09-21T03:56:34.113 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:34.113 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:34.113 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-09-21T03:56:34.113 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:34.113 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:34.113 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-09-21T03:56:34.113 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-09-21T03:56:34.113 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-09-21T03:56:34.113 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-09-21T03:56:34.113 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-09-21T03:56:34.113 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-09-21T03:56:34.113 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-09-21T03:56:34.113 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-09-21T03:56:34.114 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:34.114 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:34.114 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:34.114 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:34.114 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:34.114 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster": { 2024-09-21T03:56:34.114 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster", 2024-09-21T03:56:34.114 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-09-21T03:56:34.114 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:34.114 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:34.114 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-09-21T03:56:34.114 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:34.114 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:34.114 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:34.115 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:34.115 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:34.115 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:34.115 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:34.115 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:34.115 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster_level": { 2024-09-21T03:56:34.115 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster_level", 2024-09-21T03:56:34.115 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-09-21T03:56:34.115 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:34.115 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:34.115 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "info", 2024-09-21T03:56:34.115 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:34.115 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:34.115 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-09-21T03:56:34.115 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-09-21T03:56:34.116 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-09-21T03:56:34.116 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-09-21T03:56:34.116 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-09-21T03:56:34.116 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-09-21T03:56:34.116 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-09-21T03:56:34.116 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-09-21T03:56:34.116 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:34.116 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:34.116 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:34.116 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:34.116 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:34.116 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_file": { 2024-09-21T03:56:34.116 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_file", 2024-09-21T03:56:34.116 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-09-21T03:56:34.117 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:34.117 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:34.117 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-09-21T03:56:34.117 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:34.117 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:34.117 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:34.117 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:34.117 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:34.117 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:34.117 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:34.117 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:34.117 INFO:teuthology.orchestra.run.smithi039.stdout: "sqlite3_killpoint": { 2024-09-21T03:56:34.117 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:56:34.117 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-09-21T03:56:34.117 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "dev", 2024-09-21T03:56:34.118 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:34.118 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "0", 2024-09-21T03:56:34.118 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:34.118 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:34.118 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:34.118 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:34.118 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:34.118 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:34.118 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:34.118 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-21T03:56:34.118 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-21T03:56:34.118 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:34.118 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-21T03:56:34.118 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "telegraf", 2024-09-21T03:56:34.119 INFO:teuthology.orchestra.run.smithi039.stdout: "can_run": true, 2024-09-21T03:56:34.119 INFO:teuthology.orchestra.run.smithi039.stdout: "error_string": "", 2024-09-21T03:56:34.119 INFO:teuthology.orchestra.run.smithi039.stdout: "module_options": { 2024-09-21T03:56:34.119 INFO:teuthology.orchestra.run.smithi039.stdout: "address": { 2024-09-21T03:56:34.119 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "address", 2024-09-21T03:56:34.119 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-09-21T03:56:34.119 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:34.119 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-09-21T03:56:34.119 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-09-21T03:56:34.119 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:34.119 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:34.119 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:34.119 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:34.119 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:34.120 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:34.120 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:34.120 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:34.120 INFO:teuthology.orchestra.run.smithi039.stdout: "interval": { 2024-09-21T03:56:34.120 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "interval", 2024-09-21T03:56:34.120 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "secs", 2024-09-21T03:56:34.120 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:34.120 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-09-21T03:56:34.120 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "15", 2024-09-21T03:56:34.120 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:34.120 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:34.120 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:34.120 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:34.120 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:34.120 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:34.121 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:34.121 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:34.121 INFO:teuthology.orchestra.run.smithi039.stdout: "log_level": { 2024-09-21T03:56:34.121 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_level", 2024-09-21T03:56:34.121 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-09-21T03:56:34.121 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:34.121 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:34.121 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-09-21T03:56:34.121 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:34.121 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:34.121 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-09-21T03:56:34.121 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-09-21T03:56:34.121 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-09-21T03:56:34.121 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-09-21T03:56:34.122 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-09-21T03:56:34.122 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-09-21T03:56:34.122 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-09-21T03:56:34.122 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-09-21T03:56:34.122 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:34.122 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:34.122 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:34.122 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:34.122 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:34.122 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster": { 2024-09-21T03:56:34.122 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster", 2024-09-21T03:56:34.122 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-09-21T03:56:34.122 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:34.122 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:34.123 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-09-21T03:56:34.123 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:34.123 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:34.123 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:34.123 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:34.123 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:34.123 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:34.123 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:34.123 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:34.123 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster_level": { 2024-09-21T03:56:34.123 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster_level", 2024-09-21T03:56:34.123 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-09-21T03:56:34.123 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:34.124 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:34.124 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "info", 2024-09-21T03:56:34.124 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:34.124 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:34.124 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-09-21T03:56:34.124 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-09-21T03:56:34.124 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-09-21T03:56:34.124 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-09-21T03:56:34.124 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-09-21T03:56:34.124 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-09-21T03:56:34.124 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-09-21T03:56:34.124 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-09-21T03:56:34.124 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:34.124 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:34.124 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:34.125 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:34.125 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:34.125 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_file": { 2024-09-21T03:56:34.125 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_file", 2024-09-21T03:56:34.125 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-09-21T03:56:34.125 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:34.125 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:34.125 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-09-21T03:56:34.125 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:34.125 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:34.125 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:34.125 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:34.125 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:34.125 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:34.126 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:34.126 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:34.126 INFO:teuthology.orchestra.run.smithi039.stdout: "sqlite3_killpoint": { 2024-09-21T03:56:34.126 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:56:34.126 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-09-21T03:56:34.126 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "dev", 2024-09-21T03:56:34.126 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:34.126 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "0", 2024-09-21T03:56:34.126 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:34.126 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:34.126 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:34.126 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:34.126 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:34.126 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:34.127 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:34.127 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-21T03:56:34.127 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-21T03:56:34.127 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:34.127 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-21T03:56:34.127 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "telemetry", 2024-09-21T03:56:34.127 INFO:teuthology.orchestra.run.smithi039.stdout: "can_run": true, 2024-09-21T03:56:34.127 INFO:teuthology.orchestra.run.smithi039.stdout: "error_string": "", 2024-09-21T03:56:34.127 INFO:teuthology.orchestra.run.smithi039.stdout: "module_options": { 2024-09-21T03:56:34.127 INFO:teuthology.orchestra.run.smithi039.stdout: "channel_basic": { 2024-09-21T03:56:34.127 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "channel_basic", 2024-09-21T03:56:34.127 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-09-21T03:56:34.127 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:34.127 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-09-21T03:56:34.127 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2024-09-21T03:56:34.128 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:34.128 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:34.128 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:34.128 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Share basic cluster information (size, version)", 2024-09-21T03:56:34.128 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:34.128 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:34.128 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:34.128 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:34.128 INFO:teuthology.orchestra.run.smithi039.stdout: "channel_crash": { 2024-09-21T03:56:34.129 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "channel_crash", 2024-09-21T03:56:34.129 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-09-21T03:56:34.130 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:34.130 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-09-21T03:56:34.130 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2024-09-21T03:56:34.130 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:34.130 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:34.130 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:34.130 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-09-21T03:56:34.130 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:34.130 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:34.130 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:34.130 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:34.130 INFO:teuthology.orchestra.run.smithi039.stdout: "channel_device": { 2024-09-21T03:56:34.130 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "channel_device", 2024-09-21T03:56:34.131 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-09-21T03:56:34.131 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:34.131 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-09-21T03:56:34.131 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2024-09-21T03:56:34.131 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:34.131 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:34.131 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:34.131 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-09-21T03:56:34.131 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:34.131 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:34.131 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:34.131 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:34.131 INFO:teuthology.orchestra.run.smithi039.stdout: "channel_ident": { 2024-09-21T03:56:34.131 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "channel_ident", 2024-09-21T03:56:34.131 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-09-21T03:56:34.132 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:34.132 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-09-21T03:56:34.132 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-09-21T03:56:34.132 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:34.132 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:34.132 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:34.132 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-09-21T03:56:34.132 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:34.132 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:34.132 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:34.132 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:34.132 INFO:teuthology.orchestra.run.smithi039.stdout: "channel_perf": { 2024-09-21T03:56:34.132 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "channel_perf", 2024-09-21T03:56:34.132 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-09-21T03:56:34.133 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:34.133 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-09-21T03:56:34.133 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-09-21T03:56:34.133 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:34.133 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:34.133 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:34.133 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Share various performance metrics of a cluster", 2024-09-21T03:56:34.133 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:34.133 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:34.133 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:34.133 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:34.133 INFO:teuthology.orchestra.run.smithi039.stdout: "contact": { 2024-09-21T03:56:34.133 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "contact", 2024-09-21T03:56:34.133 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-09-21T03:56:34.134 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:34.134 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-09-21T03:56:34.134 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-09-21T03:56:34.134 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:34.134 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:34.134 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:34.134 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:34.134 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:34.134 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:34.134 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:34.134 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:34.134 INFO:teuthology.orchestra.run.smithi039.stdout: "description": { 2024-09-21T03:56:34.134 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "description", 2024-09-21T03:56:34.135 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-09-21T03:56:34.135 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:34.135 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-09-21T03:56:34.135 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-09-21T03:56:34.135 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:34.135 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:34.135 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:34.135 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:34.135 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:34.135 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:34.135 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:34.135 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:34.135 INFO:teuthology.orchestra.run.smithi039.stdout: "device_url": { 2024-09-21T03:56:34.135 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "device_url", 2024-09-21T03:56:34.136 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-09-21T03:56:34.136 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:34.136 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-09-21T03:56:34.136 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-09-21T03:56:34.136 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:34.136 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:34.136 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:34.136 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:34.136 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:34.136 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:34.136 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:34.136 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:34.136 INFO:teuthology.orchestra.run.smithi039.stdout: "enabled": { 2024-09-21T03:56:34.136 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "enabled", 2024-09-21T03:56:34.136 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-09-21T03:56:34.137 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:34.137 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-09-21T03:56:34.137 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-09-21T03:56:34.137 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:34.137 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:34.137 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:34.137 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:34.137 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:34.137 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:34.137 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:34.137 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:34.137 INFO:teuthology.orchestra.run.smithi039.stdout: "interval": { 2024-09-21T03:56:34.137 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "interval", 2024-09-21T03:56:34.137 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-09-21T03:56:34.137 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:34.137 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-09-21T03:56:34.138 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "24", 2024-09-21T03:56:34.138 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "8", 2024-09-21T03:56:34.138 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:34.138 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:34.138 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:34.138 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:34.138 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:34.138 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:34.138 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:34.138 INFO:teuthology.orchestra.run.smithi039.stdout: "last_opt_revision": { 2024-09-21T03:56:34.138 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "last_opt_revision", 2024-09-21T03:56:34.138 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-09-21T03:56:34.138 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:34.138 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-09-21T03:56:34.138 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "1", 2024-09-21T03:56:34.139 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:34.139 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:34.139 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:34.139 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:34.139 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:34.139 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:34.139 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:34.139 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:34.139 INFO:teuthology.orchestra.run.smithi039.stdout: "leaderboard": { 2024-09-21T03:56:34.139 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "leaderboard", 2024-09-21T03:56:34.139 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-09-21T03:56:34.139 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:34.139 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-09-21T03:56:34.139 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-09-21T03:56:34.140 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:34.140 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:34.140 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:34.140 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:34.140 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:34.140 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:34.140 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:34.140 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:34.140 INFO:teuthology.orchestra.run.smithi039.stdout: "leaderboard_description": { 2024-09-21T03:56:34.140 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "leaderboard_description", 2024-09-21T03:56:34.140 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-09-21T03:56:34.140 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:34.140 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-09-21T03:56:34.140 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-09-21T03:56:34.140 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:34.141 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:34.141 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:34.141 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:34.141 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:34.141 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:34.141 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:34.141 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:34.141 INFO:teuthology.orchestra.run.smithi039.stdout: "log_level": { 2024-09-21T03:56:34.141 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_level", 2024-09-21T03:56:34.141 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-09-21T03:56:34.141 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:34.141 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:34.141 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-09-21T03:56:34.141 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:34.142 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:34.142 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-09-21T03:56:34.142 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-09-21T03:56:34.142 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-09-21T03:56:34.142 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-09-21T03:56:34.142 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-09-21T03:56:34.142 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-09-21T03:56:34.142 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-09-21T03:56:34.142 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-09-21T03:56:34.142 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:34.142 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:34.142 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:34.142 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:34.142 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:34.143 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster": { 2024-09-21T03:56:34.143 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster", 2024-09-21T03:56:34.143 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-09-21T03:56:34.143 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:34.143 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:34.143 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-09-21T03:56:34.143 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:34.143 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:34.143 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:34.143 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:34.143 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:34.143 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:34.143 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:34.143 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:34.144 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster_level": { 2024-09-21T03:56:34.144 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster_level", 2024-09-21T03:56:34.144 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-09-21T03:56:34.144 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:34.144 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:34.144 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "info", 2024-09-21T03:56:34.144 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:34.144 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:34.144 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-09-21T03:56:34.144 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-09-21T03:56:34.144 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-09-21T03:56:34.144 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-09-21T03:56:34.144 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-09-21T03:56:34.144 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-09-21T03:56:34.145 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-09-21T03:56:34.145 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-09-21T03:56:34.145 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:34.145 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:34.145 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:34.145 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:34.145 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:34.145 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_file": { 2024-09-21T03:56:34.145 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_file", 2024-09-21T03:56:34.145 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-09-21T03:56:34.145 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:34.145 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:34.145 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-09-21T03:56:34.145 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:34.145 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:34.146 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:34.146 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:34.146 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:34.146 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:34.146 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:34.146 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:34.146 INFO:teuthology.orchestra.run.smithi039.stdout: "organization": { 2024-09-21T03:56:34.146 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "organization", 2024-09-21T03:56:34.146 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-09-21T03:56:34.146 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:34.146 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-09-21T03:56:34.146 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-09-21T03:56:34.146 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:34.147 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:34.147 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:34.147 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:34.147 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:34.147 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:34.147 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:34.147 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:34.147 INFO:teuthology.orchestra.run.smithi039.stdout: "proxy": { 2024-09-21T03:56:34.147 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "proxy", 2024-09-21T03:56:34.147 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-09-21T03:56:34.147 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:34.147 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-09-21T03:56:34.147 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-09-21T03:56:34.147 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:34.148 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:34.148 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:34.148 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:34.148 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:34.148 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:34.148 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:34.148 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:34.148 INFO:teuthology.orchestra.run.smithi039.stdout: "sqlite3_killpoint": { 2024-09-21T03:56:34.148 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:56:34.148 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-09-21T03:56:34.148 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "dev", 2024-09-21T03:56:34.148 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:34.148 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "0", 2024-09-21T03:56:34.148 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:34.148 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:34.148 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:34.149 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:34.149 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:34.149 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:34.149 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:34.149 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:34.149 INFO:teuthology.orchestra.run.smithi039.stdout: "url": { 2024-09-21T03:56:34.149 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "url", 2024-09-21T03:56:34.149 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-09-21T03:56:34.149 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:34.149 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-09-21T03:56:34.149 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-09-21T03:56:34.149 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:34.149 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:34.149 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:34.149 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:34.150 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:34.150 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:34.150 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:34.150 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-21T03:56:34.150 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-21T03:56:34.150 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:34.150 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-21T03:56:34.150 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "test_orchestrator", 2024-09-21T03:56:34.150 INFO:teuthology.orchestra.run.smithi039.stdout: "can_run": true, 2024-09-21T03:56:34.150 INFO:teuthology.orchestra.run.smithi039.stdout: "error_string": "", 2024-09-21T03:56:34.150 INFO:teuthology.orchestra.run.smithi039.stdout: "module_options": { 2024-09-21T03:56:34.150 INFO:teuthology.orchestra.run.smithi039.stdout: "log_level": { 2024-09-21T03:56:34.150 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_level", 2024-09-21T03:56:34.150 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-09-21T03:56:34.151 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:34.151 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:34.151 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-09-21T03:56:34.151 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:34.151 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:34.151 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-09-21T03:56:34.151 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-09-21T03:56:34.151 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-09-21T03:56:34.151 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-09-21T03:56:34.151 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-09-21T03:56:34.151 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-09-21T03:56:34.151 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-09-21T03:56:34.151 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-09-21T03:56:34.151 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:34.152 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:34.152 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:34.152 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:34.152 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:34.152 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster": { 2024-09-21T03:56:34.152 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster", 2024-09-21T03:56:34.152 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-09-21T03:56:34.152 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:34.152 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:34.152 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-09-21T03:56:34.152 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:34.152 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:34.152 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:34.152 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:34.152 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:34.153 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:34.153 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:34.153 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:34.153 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster_level": { 2024-09-21T03:56:34.153 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster_level", 2024-09-21T03:56:34.153 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-09-21T03:56:34.153 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:34.153 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:34.153 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "info", 2024-09-21T03:56:34.153 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:34.153 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:34.153 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-09-21T03:56:34.153 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-09-21T03:56:34.153 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-09-21T03:56:34.153 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-09-21T03:56:34.153 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-09-21T03:56:34.154 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-09-21T03:56:34.154 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-09-21T03:56:34.154 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-09-21T03:56:34.154 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:34.154 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:34.154 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:34.154 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:34.154 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:34.154 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_file": { 2024-09-21T03:56:34.154 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_file", 2024-09-21T03:56:34.154 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-09-21T03:56:34.154 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:34.154 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:34.154 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-09-21T03:56:34.154 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:34.155 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:34.155 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:34.155 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:34.155 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:34.155 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:34.155 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:34.155 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:34.155 INFO:teuthology.orchestra.run.smithi039.stdout: "sqlite3_killpoint": { 2024-09-21T03:56:34.155 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:56:34.155 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-09-21T03:56:34.155 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "dev", 2024-09-21T03:56:34.155 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:34.155 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "0", 2024-09-21T03:56:34.155 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:34.156 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:34.156 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:34.156 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:34.156 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:34.156 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:34.156 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:34.156 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-21T03:56:34.156 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-21T03:56:34.156 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:34.156 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-21T03:56:34.156 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "volumes", 2024-09-21T03:56:34.156 INFO:teuthology.orchestra.run.smithi039.stdout: "can_run": true, 2024-09-21T03:56:34.156 INFO:teuthology.orchestra.run.smithi039.stdout: "error_string": "", 2024-09-21T03:56:34.156 INFO:teuthology.orchestra.run.smithi039.stdout: "module_options": { 2024-09-21T03:56:34.156 INFO:teuthology.orchestra.run.smithi039.stdout: "log_level": { 2024-09-21T03:56:34.157 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_level", 2024-09-21T03:56:34.157 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-09-21T03:56:34.157 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:34.157 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:34.157 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-09-21T03:56:34.157 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:34.157 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:34.157 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-09-21T03:56:34.157 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-09-21T03:56:34.157 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-09-21T03:56:34.157 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-09-21T03:56:34.157 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-09-21T03:56:34.157 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-09-21T03:56:34.157 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-09-21T03:56:34.157 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-09-21T03:56:34.158 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:34.158 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:34.158 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:34.158 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:34.158 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:34.158 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster": { 2024-09-21T03:56:34.158 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster", 2024-09-21T03:56:34.158 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-09-21T03:56:34.158 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:34.158 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:34.158 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-09-21T03:56:34.158 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:34.158 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:34.158 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:34.159 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:34.159 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:34.159 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:34.159 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:34.159 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:34.159 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster_level": { 2024-09-21T03:56:34.159 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster_level", 2024-09-21T03:56:34.159 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-09-21T03:56:34.159 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:34.159 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:34.159 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "info", 2024-09-21T03:56:34.159 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:34.159 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:34.160 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-09-21T03:56:34.160 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-09-21T03:56:34.160 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-09-21T03:56:34.160 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-09-21T03:56:34.160 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-09-21T03:56:34.160 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-09-21T03:56:34.160 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-09-21T03:56:34.160 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-09-21T03:56:34.160 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:34.160 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:34.160 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:34.160 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:34.160 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:34.160 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_file": { 2024-09-21T03:56:34.161 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_file", 2024-09-21T03:56:34.161 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-09-21T03:56:34.161 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:34.161 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:34.161 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-09-21T03:56:34.161 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:34.161 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:34.161 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:34.161 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:34.161 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:34.161 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:34.161 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:34.161 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:34.162 INFO:teuthology.orchestra.run.smithi039.stdout: "max_concurrent_clones": { 2024-09-21T03:56:34.162 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "max_concurrent_clones", 2024-09-21T03:56:34.162 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-09-21T03:56:34.162 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:34.162 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-09-21T03:56:34.162 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "4", 2024-09-21T03:56:34.162 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:34.162 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:34.162 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:34.162 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Number of asynchronous cloner threads", 2024-09-21T03:56:34.162 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:34.162 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:34.162 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:34.162 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:34.163 INFO:teuthology.orchestra.run.smithi039.stdout: "periodic_async_work": { 2024-09-21T03:56:34.163 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "periodic_async_work", 2024-09-21T03:56:34.163 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-09-21T03:56:34.163 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:34.163 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-09-21T03:56:34.163 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-09-21T03:56:34.163 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:34.163 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:34.163 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:34.163 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Periodically check for async work", 2024-09-21T03:56:34.163 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:34.163 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:34.163 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:34.163 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:34.164 INFO:teuthology.orchestra.run.smithi039.stdout: "snapshot_clone_delay": { 2024-09-21T03:56:34.164 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "snapshot_clone_delay", 2024-09-21T03:56:34.164 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-09-21T03:56:34.164 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:34.164 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-09-21T03:56:34.164 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "0", 2024-09-21T03:56:34.164 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:34.164 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:34.164 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:34.164 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-09-21T03:56:34.164 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:34.164 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:34.164 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:34.165 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:34.165 INFO:teuthology.orchestra.run.smithi039.stdout: "snapshot_clone_no_wait": { 2024-09-21T03:56:34.165 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "snapshot_clone_no_wait", 2024-09-21T03:56:34.165 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-09-21T03:56:34.165 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:34.165 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-09-21T03:56:34.165 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2024-09-21T03:56:34.165 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:34.165 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:34.165 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:34.165 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-09-21T03:56:34.165 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:34.165 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:34.165 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:34.166 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:34.166 INFO:teuthology.orchestra.run.smithi039.stdout: "sqlite3_killpoint": { 2024-09-21T03:56:34.166 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:56:34.166 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-09-21T03:56:34.166 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "dev", 2024-09-21T03:56:34.166 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:34.166 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "0", 2024-09-21T03:56:34.166 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:34.166 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:34.166 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:34.166 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:34.166 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:34.166 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:34.166 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:34.167 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-21T03:56:34.167 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-21T03:56:34.167 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:34.167 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-21T03:56:34.167 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "zabbix", 2024-09-21T03:56:34.167 INFO:teuthology.orchestra.run.smithi039.stdout: "can_run": true, 2024-09-21T03:56:34.167 INFO:teuthology.orchestra.run.smithi039.stdout: "error_string": "", 2024-09-21T03:56:34.167 INFO:teuthology.orchestra.run.smithi039.stdout: "module_options": { 2024-09-21T03:56:34.167 INFO:teuthology.orchestra.run.smithi039.stdout: "discovery_interval": { 2024-09-21T03:56:34.167 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "discovery_interval", 2024-09-21T03:56:34.167 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "uint", 2024-09-21T03:56:34.167 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:34.167 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-09-21T03:56:34.167 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "100", 2024-09-21T03:56:34.168 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:34.168 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:34.168 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:34.168 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:34.168 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:34.168 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:34.168 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:34.168 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:34.168 INFO:teuthology.orchestra.run.smithi039.stdout: "identifier": { 2024-09-21T03:56:34.168 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "identifier", 2024-09-21T03:56:34.168 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-09-21T03:56:34.168 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:34.168 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-09-21T03:56:34.168 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-09-21T03:56:34.169 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:34.169 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:34.169 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:34.169 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:34.169 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:34.169 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:34.169 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:34.169 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:34.169 INFO:teuthology.orchestra.run.smithi039.stdout: "interval": { 2024-09-21T03:56:34.169 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "interval", 2024-09-21T03:56:34.169 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "secs", 2024-09-21T03:56:34.169 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:34.169 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-09-21T03:56:34.170 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "60", 2024-09-21T03:56:34.170 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:34.170 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:34.170 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:34.170 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:34.170 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:34.170 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:34.170 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:34.170 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:34.170 INFO:teuthology.orchestra.run.smithi039.stdout: "log_level": { 2024-09-21T03:56:34.170 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_level", 2024-09-21T03:56:34.170 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-09-21T03:56:34.170 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:34.170 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:34.170 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-09-21T03:56:34.171 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:34.171 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:34.171 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-09-21T03:56:34.171 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-09-21T03:56:34.171 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-09-21T03:56:34.171 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-09-21T03:56:34.171 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-09-21T03:56:34.171 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-09-21T03:56:34.171 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-09-21T03:56:34.171 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-09-21T03:56:34.171 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:34.171 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:34.171 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:34.172 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:34.172 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:34.172 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster": { 2024-09-21T03:56:34.172 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster", 2024-09-21T03:56:34.172 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-09-21T03:56:34.172 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:34.172 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:34.172 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-09-21T03:56:34.172 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:34.172 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:34.172 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:34.172 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:34.172 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:34.172 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:34.173 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:34.173 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:34.173 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster_level": { 2024-09-21T03:56:34.173 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster_level", 2024-09-21T03:56:34.173 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-09-21T03:56:34.173 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:34.173 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:34.173 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "info", 2024-09-21T03:56:34.173 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:34.173 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:34.173 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-09-21T03:56:34.173 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-09-21T03:56:34.173 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-09-21T03:56:34.173 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-09-21T03:56:34.173 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-09-21T03:56:34.174 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-09-21T03:56:34.174 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-09-21T03:56:34.174 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-09-21T03:56:34.174 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:34.174 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:34.174 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:34.174 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:34.174 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:34.174 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_file": { 2024-09-21T03:56:34.174 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_file", 2024-09-21T03:56:34.174 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-09-21T03:56:34.174 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:34.174 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:34.175 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-09-21T03:56:34.175 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:34.175 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:34.175 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:34.175 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:34.175 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:34.175 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:34.175 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:34.175 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:34.175 INFO:teuthology.orchestra.run.smithi039.stdout: "sqlite3_killpoint": { 2024-09-21T03:56:34.175 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:56:34.175 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-09-21T03:56:34.175 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "dev", 2024-09-21T03:56:34.175 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-21T03:56:34.175 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "0", 2024-09-21T03:56:34.176 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:34.176 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:34.176 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:34.176 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:34.176 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:34.176 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:34.176 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:34.176 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:34.176 INFO:teuthology.orchestra.run.smithi039.stdout: "zabbix_host": { 2024-09-21T03:56:34.176 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "zabbix_host", 2024-09-21T03:56:34.176 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-09-21T03:56:34.176 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:34.176 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-09-21T03:56:34.177 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-09-21T03:56:34.177 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:34.177 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:34.177 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:34.177 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:34.177 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:34.177 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:34.177 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:34.177 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:34.177 INFO:teuthology.orchestra.run.smithi039.stdout: "zabbix_port": { 2024-09-21T03:56:34.177 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "zabbix_port", 2024-09-21T03:56:34.177 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-09-21T03:56:34.177 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:34.177 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-09-21T03:56:34.178 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "10051", 2024-09-21T03:56:34.178 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:34.178 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:34.178 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:34.178 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:34.178 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:34.178 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:34.178 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:34.178 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:34.178 INFO:teuthology.orchestra.run.smithi039.stdout: "zabbix_sender": { 2024-09-21T03:56:34.178 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "zabbix_sender", 2024-09-21T03:56:34.178 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-09-21T03:56:34.178 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-09-21T03:56:34.179 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-09-21T03:56:34.179 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-09-21T03:56:34.179 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-09-21T03:56:34.179 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-09-21T03:56:34.179 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-09-21T03:56:34.179 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-09-21T03:56:34.179 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-09-21T03:56:34.179 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-09-21T03:56:34.179 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-09-21T03:56:34.179 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-21T03:56:34.179 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-21T03:56:34.179 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-21T03:56:34.179 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-09-21T03:56:34.179 INFO:teuthology.orchestra.run.smithi039.stdout: "services": { 2024-09-21T03:56:34.179 INFO:teuthology.orchestra.run.smithi039.stdout: "dashboard": "https://172.21.15.39:8443/" 2024-09-21T03:56:34.180 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:34.180 INFO:teuthology.orchestra.run.smithi039.stdout: "always_on_modules": { 2024-09-21T03:56:34.180 INFO:teuthology.orchestra.run.smithi039.stdout: "octopus": [ 2024-09-21T03:56:34.180 INFO:teuthology.orchestra.run.smithi039.stdout: "balancer", 2024-09-21T03:56:34.180 INFO:teuthology.orchestra.run.smithi039.stdout: "crash", 2024-09-21T03:56:34.180 INFO:teuthology.orchestra.run.smithi039.stdout: "devicehealth", 2024-09-21T03:56:34.180 INFO:teuthology.orchestra.run.smithi039.stdout: "orchestrator", 2024-09-21T03:56:34.180 INFO:teuthology.orchestra.run.smithi039.stdout: "pg_autoscaler", 2024-09-21T03:56:34.180 INFO:teuthology.orchestra.run.smithi039.stdout: "progress", 2024-09-21T03:56:34.180 INFO:teuthology.orchestra.run.smithi039.stdout: "rbd_support", 2024-09-21T03:56:34.180 INFO:teuthology.orchestra.run.smithi039.stdout: "status", 2024-09-21T03:56:34.180 INFO:teuthology.orchestra.run.smithi039.stdout: "telemetry", 2024-09-21T03:56:34.180 INFO:teuthology.orchestra.run.smithi039.stdout: "volumes" 2024-09-21T03:56:34.180 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-09-21T03:56:34.181 INFO:teuthology.orchestra.run.smithi039.stdout: "pacific": [ 2024-09-21T03:56:34.181 INFO:teuthology.orchestra.run.smithi039.stdout: "balancer", 2024-09-21T03:56:34.181 INFO:teuthology.orchestra.run.smithi039.stdout: "crash", 2024-09-21T03:56:34.181 INFO:teuthology.orchestra.run.smithi039.stdout: "devicehealth", 2024-09-21T03:56:34.181 INFO:teuthology.orchestra.run.smithi039.stdout: "orchestrator", 2024-09-21T03:56:34.181 INFO:teuthology.orchestra.run.smithi039.stdout: "pg_autoscaler", 2024-09-21T03:56:34.181 INFO:teuthology.orchestra.run.smithi039.stdout: "progress", 2024-09-21T03:56:34.181 INFO:teuthology.orchestra.run.smithi039.stdout: "rbd_support", 2024-09-21T03:56:34.181 INFO:teuthology.orchestra.run.smithi039.stdout: "status", 2024-09-21T03:56:34.181 INFO:teuthology.orchestra.run.smithi039.stdout: "telemetry", 2024-09-21T03:56:34.181 INFO:teuthology.orchestra.run.smithi039.stdout: "volumes" 2024-09-21T03:56:34.181 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-09-21T03:56:34.181 INFO:teuthology.orchestra.run.smithi039.stdout: "quincy": [ 2024-09-21T03:56:34.181 INFO:teuthology.orchestra.run.smithi039.stdout: "balancer", 2024-09-21T03:56:34.181 INFO:teuthology.orchestra.run.smithi039.stdout: "crash", 2024-09-21T03:56:34.181 INFO:teuthology.orchestra.run.smithi039.stdout: "devicehealth", 2024-09-21T03:56:34.182 INFO:teuthology.orchestra.run.smithi039.stdout: "orchestrator", 2024-09-21T03:56:34.182 INFO:teuthology.orchestra.run.smithi039.stdout: "pg_autoscaler", 2024-09-21T03:56:34.182 INFO:teuthology.orchestra.run.smithi039.stdout: "progress", 2024-09-21T03:56:34.182 INFO:teuthology.orchestra.run.smithi039.stdout: "rbd_support", 2024-09-21T03:56:34.182 INFO:teuthology.orchestra.run.smithi039.stdout: "status", 2024-09-21T03:56:34.182 INFO:teuthology.orchestra.run.smithi039.stdout: "telemetry", 2024-09-21T03:56:34.182 INFO:teuthology.orchestra.run.smithi039.stdout: "volumes" 2024-09-21T03:56:34.182 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-09-21T03:56:34.182 INFO:teuthology.orchestra.run.smithi039.stdout: "reef": [ 2024-09-21T03:56:34.182 INFO:teuthology.orchestra.run.smithi039.stdout: "balancer", 2024-09-21T03:56:34.182 INFO:teuthology.orchestra.run.smithi039.stdout: "crash", 2024-09-21T03:56:34.182 INFO:teuthology.orchestra.run.smithi039.stdout: "devicehealth", 2024-09-21T03:56:34.182 INFO:teuthology.orchestra.run.smithi039.stdout: "orchestrator", 2024-09-21T03:56:34.182 INFO:teuthology.orchestra.run.smithi039.stdout: "pg_autoscaler", 2024-09-21T03:56:34.182 INFO:teuthology.orchestra.run.smithi039.stdout: "progress", 2024-09-21T03:56:34.182 INFO:teuthology.orchestra.run.smithi039.stdout: "rbd_support", 2024-09-21T03:56:34.183 INFO:teuthology.orchestra.run.smithi039.stdout: "status", 2024-09-21T03:56:34.183 INFO:teuthology.orchestra.run.smithi039.stdout: "telemetry", 2024-09-21T03:56:34.183 INFO:teuthology.orchestra.run.smithi039.stdout: "volumes" 2024-09-21T03:56:34.183 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-09-21T03:56:34.183 INFO:teuthology.orchestra.run.smithi039.stdout: "squid": [ 2024-09-21T03:56:34.183 INFO:teuthology.orchestra.run.smithi039.stdout: "balancer", 2024-09-21T03:56:34.183 INFO:teuthology.orchestra.run.smithi039.stdout: "crash", 2024-09-21T03:56:34.183 INFO:teuthology.orchestra.run.smithi039.stdout: "devicehealth", 2024-09-21T03:56:34.183 INFO:teuthology.orchestra.run.smithi039.stdout: "orchestrator", 2024-09-21T03:56:34.183 INFO:teuthology.orchestra.run.smithi039.stdout: "pg_autoscaler", 2024-09-21T03:56:34.183 INFO:teuthology.orchestra.run.smithi039.stdout: "progress", 2024-09-21T03:56:34.183 INFO:teuthology.orchestra.run.smithi039.stdout: "rbd_support", 2024-09-21T03:56:34.183 INFO:teuthology.orchestra.run.smithi039.stdout: "status", 2024-09-21T03:56:34.183 INFO:teuthology.orchestra.run.smithi039.stdout: "telemetry", 2024-09-21T03:56:34.183 INFO:teuthology.orchestra.run.smithi039.stdout: "volumes" 2024-09-21T03:56:34.184 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-21T03:56:34.184 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:34.184 INFO:teuthology.orchestra.run.smithi039.stdout: "last_failure_osd_epoch": 5, 2024-09-21T03:56:34.184 INFO:teuthology.orchestra.run.smithi039.stdout: "active_clients": [ 2024-09-21T03:56:34.184 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-21T03:56:34.184 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "devicehealth", 2024-09-21T03:56:34.184 INFO:teuthology.orchestra.run.smithi039.stdout: "addrvec": [ 2024-09-21T03:56:34.184 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-21T03:56:34.184 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "v2", 2024-09-21T03:56:34.184 INFO:teuthology.orchestra.run.smithi039.stdout: "addr": "172.21.15.39:0", 2024-09-21T03:56:34.184 INFO:teuthology.orchestra.run.smithi039.stdout: "nonce": 2409142260 2024-09-21T03:56:34.184 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-21T03:56:34.184 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-21T03:56:34.184 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:34.184 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-21T03:56:34.185 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "libcephsqlite", 2024-09-21T03:56:34.185 INFO:teuthology.orchestra.run.smithi039.stdout: "addrvec": [ 2024-09-21T03:56:34.185 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-21T03:56:34.185 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "v2", 2024-09-21T03:56:34.185 INFO:teuthology.orchestra.run.smithi039.stdout: "addr": "172.21.15.39:0", 2024-09-21T03:56:34.185 INFO:teuthology.orchestra.run.smithi039.stdout: "nonce": 3867972996 2024-09-21T03:56:34.185 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-21T03:56:34.185 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-21T03:56:34.185 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:34.185 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-21T03:56:34.185 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "rbd_support", 2024-09-21T03:56:34.185 INFO:teuthology.orchestra.run.smithi039.stdout: "addrvec": [ 2024-09-21T03:56:34.185 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-21T03:56:34.186 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "v2", 2024-09-21T03:56:34.186 INFO:teuthology.orchestra.run.smithi039.stdout: "addr": "172.21.15.39:0", 2024-09-21T03:56:34.186 INFO:teuthology.orchestra.run.smithi039.stdout: "nonce": 2292114781 2024-09-21T03:56:34.186 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-21T03:56:34.186 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-21T03:56:34.186 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:34.186 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-21T03:56:34.186 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "snap_schedule", 2024-09-21T03:56:34.186 INFO:teuthology.orchestra.run.smithi039.stdout: "addrvec": [ 2024-09-21T03:56:34.186 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-21T03:56:34.186 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "v2", 2024-09-21T03:56:34.186 INFO:teuthology.orchestra.run.smithi039.stdout: "addr": "172.21.15.39:0", 2024-09-21T03:56:34.186 INFO:teuthology.orchestra.run.smithi039.stdout: "nonce": 3442791637 2024-09-21T03:56:34.186 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-21T03:56:34.187 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-21T03:56:34.187 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:34.187 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-21T03:56:34.187 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "snap_schedule", 2024-09-21T03:56:34.187 INFO:teuthology.orchestra.run.smithi039.stdout: "addrvec": [ 2024-09-21T03:56:34.187 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-21T03:56:34.187 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "v2", 2024-09-21T03:56:34.187 INFO:teuthology.orchestra.run.smithi039.stdout: "addr": "172.21.15.39:0", 2024-09-21T03:56:34.187 INFO:teuthology.orchestra.run.smithi039.stdout: "nonce": 3367649803 2024-09-21T03:56:34.187 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-21T03:56:34.187 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-21T03:56:34.187 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:34.187 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-21T03:56:34.187 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "volumes", 2024-09-21T03:56:34.187 INFO:teuthology.orchestra.run.smithi039.stdout: "addrvec": [ 2024-09-21T03:56:34.187 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-21T03:56:34.188 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "v2", 2024-09-21T03:56:34.188 INFO:teuthology.orchestra.run.smithi039.stdout: "addr": "172.21.15.39:0", 2024-09-21T03:56:34.188 INFO:teuthology.orchestra.run.smithi039.stdout: "nonce": 3602591107 2024-09-21T03:56:34.188 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-21T03:56:34.188 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-21T03:56:34.188 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-21T03:56:34.188 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-21T03:56:34.188 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-09-21T03:56:34.189 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:32.756+0000 7fa1aadba640 1 -- 172.21.15.39:0/2360451637 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7fa1940b6680 msgr2=0x7fa1940b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:56:34.189 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:32.756+0000 7fa1aadba640 1 --2- 172.21.15.39:0/2360451637 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7fa1940b6680 0x7fa1940b8b90 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7fa1a4101af0 tx=0x7fa190005f90 comp rx=0 tx=0).stop 2024-09-21T03:56:34.189 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:32.756+0000 7fa1aadba640 1 -- 172.21.15.39:0/2360451637 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa1a4102980 msgr2=0x7fa1a419a370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:56:34.189 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:32.756+0000 7fa1aadba640 1 --2- 172.21.15.39:0/2360451637 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa1a4102980 0x7fa1a419a370 secure :-1 s=READY pgs=1365 cs=0 l=1 rev1=1 crypto rx=0x7fa19800e9f0 tx=0x7fa19800eeb0 comp rx=0 tx=0).stop 2024-09-21T03:56:34.189 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:32.756+0000 7fa1aadba640 1 -- 172.21.15.39:0/2360451637 shutdown_connections 2024-09-21T03:56:34.189 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:32.756+0000 7fa1aadba640 1 --2- 172.21.15.39:0/2360451637 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7fa1940b6680 0x7fa1940b8b90 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:34.189 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:32.756+0000 7fa1aadba640 1 --2- 172.21.15.39:0/2360451637 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa1a4108c00 0x7fa1a419fba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:34.190 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:32.756+0000 7fa1aadba640 1 --2- 172.21.15.39:0/2360451637 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa1a41082b0 0x7fa1a419a910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:34.190 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:32.756+0000 7fa1aadba640 1 --2- 172.21.15.39:0/2360451637 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa1a4102980 0x7fa1a419a370 unknown :-1 s=CLOSED pgs=1365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:34.190 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:32.756+0000 7fa1aadba640 1 -- 172.21.15.39:0/2360451637 >> 172.21.15.39:0/2360451637 conn(0x7fa1a40fe380 msgr2=0x7fa1a41078a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:56:34.190 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:32.756+0000 7fa1aadba640 1 -- 172.21.15.39:0/2360451637 shutdown_connections 2024-09-21T03:56:34.190 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:32.756+0000 7fa1aadba640 1 -- 172.21.15.39:0/2360451637 wait complete. 2024-09-21T03:56:34.232 DEBUG:tasks.check_counter:Getting stats from a 2024-09-21T03:56:34.232 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 51116d5c-77ca-11ef-baf5-efdc52797490 -- ceph daemon mds.a perf dump 2024-09-21T03:56:34.233 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:56:33 smithi039 ceph-mon[31211]: pgmap v648: 97 pgs: 97 active+clean; 10 MiB data, 525 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 0 B/s wr, 11 op/s 2024-09-21T03:56:34.233 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:56:33 smithi039 ceph-mon[31211]: from='client.? 172.21.15.39:0/6203856' entity='client.admin' cmd={"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} : dispatch 2024-09-21T03:56:34.233 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:56:33 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:56:34.233 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:56:33 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:56:34.233 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:56:33 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:56:34.233 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:56:33 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:56:34.233 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:56:33 smithi039 ceph-mon[31211]: from='client.? 172.21.15.39:0/2360451637' entity='client.admin' cmd={"prefix": "mgr dump", "format": "json-pretty"} : dispatch 2024-09-21T03:56:34.233 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:56:33 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:56:34.233 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:56:33 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:56:34.233 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:56:33 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:56:34.233 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:56:33 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:56:34.233 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:56:33 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:56:34.234 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:56:33 smithi150 ceph-mon[34289]: pgmap v648: 97 pgs: 97 active+clean; 10 MiB data, 525 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 0 B/s wr, 11 op/s 2024-09-21T03:56:34.234 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:56:33 smithi150 ceph-mon[34289]: from='client.? 172.21.15.39:0/6203856' entity='client.admin' cmd={"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} : dispatch 2024-09-21T03:56:34.234 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:56:33 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:56:34.234 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:56:33 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:56:34.235 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:56:33 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:56:34.235 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:56:33 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:56:34.235 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:56:33 smithi150 ceph-mon[34289]: from='client.? 172.21.15.39:0/2360451637' entity='client.admin' cmd={"prefix": "mgr dump", "format": "json-pretty"} : dispatch 2024-09-21T03:56:34.235 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:56:33 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:56:34.235 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:56:33 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:56:34.235 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:56:33 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:56:34.235 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:56:33 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:56:34.235 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:56:33 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:56:34.235 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:56:33 smithi161 ceph-mon[34533]: pgmap v648: 97 pgs: 97 active+clean; 10 MiB data, 525 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 0 B/s wr, 11 op/s 2024-09-21T03:56:34.235 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:56:33 smithi161 ceph-mon[34533]: from='client.? 172.21.15.39:0/6203856' entity='client.admin' cmd={"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} : dispatch 2024-09-21T03:56:34.235 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:56:33 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:56:34.235 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:56:33 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:56:34.235 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:56:33 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:56:34.236 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:56:33 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:56:34.236 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:56:33 smithi161 ceph-mon[34533]: from='client.? 172.21.15.39:0/2360451637' entity='client.admin' cmd={"prefix": "mgr dump", "format": "json-pretty"} : dispatch 2024-09-21T03:56:34.236 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:56:33 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:56:34.236 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:56:33 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:56:34.236 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:56:33 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:56:34.236 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:56:33 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:56:34.236 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:56:33 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' 2024-09-21T03:56:34.505 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/51116d5c-77ca-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:56:34.993 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-09-21T03:56:34.993 INFO:teuthology.orchestra.run.smithi039.stdout: "AsyncMessenger::Worker-0": { 2024-09-21T03:56:34.993 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_recv_messages": 299, 2024-09-21T03:56:34.993 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_send_messages": 302, 2024-09-21T03:56:34.993 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_recv_bytes": 1207297, 2024-09-21T03:56:34.993 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_send_bytes": 245075, 2024-09-21T03:56:34.993 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_created_connections": 17, 2024-09-21T03:56:34.993 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_active_connections": 4, 2024-09-21T03:56:34.993 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_running_total_time": 0.071173723, 2024-09-21T03:56:34.993 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_running_send_time": 0.027496416, 2024-09-21T03:56:34.994 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_running_recv_time": 0.054657619, 2024-09-21T03:56:34.994 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_running_fast_dispatch_time": 0.000296599, 2024-09-21T03:56:34.994 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T03:56:34.994 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 300, 2024-09-21T03:56:34.994 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.045219433, 2024-09-21T03:56:34.994 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000150731 2024-09-21T03:56:34.994 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:34.994 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_handle_ack_lat": { 2024-09-21T03:56:34.994 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 48, 2024-09-21T03:56:34.994 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000010134, 2024-09-21T03:56:34.994 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000211 2024-09-21T03:56:34.994 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:34.994 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_recv_encrypted_bytes": 1202448, 2024-09-21T03:56:34.995 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_send_encrypted_bytes": 138400 2024-09-21T03:56:34.995 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:34.995 INFO:teuthology.orchestra.run.smithi039.stdout: "AsyncMessenger::Worker-1": { 2024-09-21T03:56:34.995 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_recv_messages": 426, 2024-09-21T03:56:34.995 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_send_messages": 667, 2024-09-21T03:56:34.995 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_recv_bytes": 137241, 2024-09-21T03:56:34.995 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_send_bytes": 589460, 2024-09-21T03:56:34.995 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_created_connections": 7, 2024-09-21T03:56:34.995 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_active_connections": 4, 2024-09-21T03:56:34.995 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_running_total_time": 0.079615600, 2024-09-21T03:56:34.995 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_running_send_time": 0.042741459, 2024-09-21T03:56:34.995 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_running_recv_time": 0.030097987, 2024-09-21T03:56:34.995 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_running_fast_dispatch_time": 0.004709432, 2024-09-21T03:56:34.995 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T03:56:34.995 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 667, 2024-09-21T03:56:34.996 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.051038423, 2024-09-21T03:56:34.996 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000076519 2024-09-21T03:56:34.996 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:34.996 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_handle_ack_lat": { 2024-09-21T03:56:34.996 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 916, 2024-09-21T03:56:34.996 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000227109, 2024-09-21T03:56:34.996 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000247 2024-09-21T03:56:34.996 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:34.996 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_recv_encrypted_bytes": 160, 2024-09-21T03:56:34.996 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_send_encrypted_bytes": 357648 2024-09-21T03:56:34.996 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:34.996 INFO:teuthology.orchestra.run.smithi039.stdout: "AsyncMessenger::Worker-2": { 2024-09-21T03:56:34.996 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_recv_messages": 59, 2024-09-21T03:56:34.996 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_send_messages": 62, 2024-09-21T03:56:34.997 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_recv_bytes": 11192, 2024-09-21T03:56:34.997 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_send_bytes": 453786, 2024-09-21T03:56:34.997 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_created_connections": 43, 2024-09-21T03:56:34.997 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_active_connections": 8, 2024-09-21T03:56:34.997 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_running_total_time": 0.024219230, 2024-09-21T03:56:34.997 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_running_send_time": 0.002689252, 2024-09-21T03:56:34.997 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_running_recv_time": 0.017790503, 2024-09-21T03:56:34.997 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_running_fast_dispatch_time": 0.000450605, 2024-09-21T03:56:34.997 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T03:56:34.997 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 60, 2024-09-21T03:56:34.997 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.056825819, 2024-09-21T03:56:34.997 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000947096 2024-09-21T03:56:34.997 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:34.997 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_handle_ack_lat": { 2024-09-21T03:56:34.997 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 44, 2024-09-21T03:56:34.997 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000007160, 2024-09-21T03:56:34.998 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000162 2024-09-21T03:56:34.998 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:34.998 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_recv_encrypted_bytes": 928, 2024-09-21T03:56:34.998 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_send_encrypted_bytes": 976 2024-09-21T03:56:34.998 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:34.998 INFO:teuthology.orchestra.run.smithi039.stdout: "cct": { 2024-09-21T03:56:34.998 INFO:teuthology.orchestra.run.smithi039.stdout: "total_workers": 1, 2024-09-21T03:56:34.998 INFO:teuthology.orchestra.run.smithi039.stdout: "unhealthy_workers": 0 2024-09-21T03:56:34.998 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:34.998 INFO:teuthology.orchestra.run.smithi039.stdout: "finisher-MDSRank": { 2024-09-21T03:56:34.998 INFO:teuthology.orchestra.run.smithi039.stdout: "queue_len": 0, 2024-09-21T03:56:34.998 INFO:teuthology.orchestra.run.smithi039.stdout: "complete_latency": { 2024-09-21T03:56:34.998 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 59, 2024-09-21T03:56:34.998 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.001583369, 2024-09-21T03:56:34.999 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000026836 2024-09-21T03:56:34.999 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-21T03:56:34.999 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:34.999 INFO:teuthology.orchestra.run.smithi039.stdout: "finisher-PurgeQueue": { 2024-09-21T03:56:34.999 INFO:teuthology.orchestra.run.smithi039.stdout: "queue_len": 0, 2024-09-21T03:56:34.999 INFO:teuthology.orchestra.run.smithi039.stdout: "complete_latency": { 2024-09-21T03:56:34.999 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 2, 2024-09-21T03:56:34.999 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000018372, 2024-09-21T03:56:34.999 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000009186 2024-09-21T03:56:34.999 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-21T03:56:34.999 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:34.999 INFO:teuthology.orchestra.run.smithi039.stdout: "mds": { 2024-09-21T03:56:34.999 INFO:teuthology.orchestra.run.smithi039.stdout: "request": 1, 2024-09-21T03:56:34.999 INFO:teuthology.orchestra.run.smithi039.stdout: "reply": 0, 2024-09-21T03:56:34.999 INFO:teuthology.orchestra.run.smithi039.stdout: "reply_latency": { 2024-09-21T03:56:34.999 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2024-09-21T03:56:35.000 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2024-09-21T03:56:35.000 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2024-09-21T03:56:35.000 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:35.000 INFO:teuthology.orchestra.run.smithi039.stdout: "slow_reply": 0, 2024-09-21T03:56:35.000 INFO:teuthology.orchestra.run.smithi039.stdout: "forward": 1, 2024-09-21T03:56:35.000 INFO:teuthology.orchestra.run.smithi039.stdout: "dir_fetch_complete": 0, 2024-09-21T03:56:35.000 INFO:teuthology.orchestra.run.smithi039.stdout: "dir_fetch_keys": 0, 2024-09-21T03:56:35.000 INFO:teuthology.orchestra.run.smithi039.stdout: "dir_commit": 11, 2024-09-21T03:56:35.000 INFO:teuthology.orchestra.run.smithi039.stdout: "dir_split": 0, 2024-09-21T03:56:35.000 INFO:teuthology.orchestra.run.smithi039.stdout: "dir_merge": 0, 2024-09-21T03:56:35.000 INFO:teuthology.orchestra.run.smithi039.stdout: "inodes": 16, 2024-09-21T03:56:35.000 INFO:teuthology.orchestra.run.smithi039.stdout: "inodes_top": 0, 2024-09-21T03:56:35.000 INFO:teuthology.orchestra.run.smithi039.stdout: "inodes_bottom": 16, 2024-09-21T03:56:35.000 INFO:teuthology.orchestra.run.smithi039.stdout: "inodes_pin_tail": 0, 2024-09-21T03:56:35.000 INFO:teuthology.orchestra.run.smithi039.stdout: "inodes_pinned": 13, 2024-09-21T03:56:35.000 INFO:teuthology.orchestra.run.smithi039.stdout: "inodes_expired": 4, 2024-09-21T03:56:35.001 INFO:teuthology.orchestra.run.smithi039.stdout: "inodes_with_caps": 0, 2024-09-21T03:56:35.001 INFO:teuthology.orchestra.run.smithi039.stdout: "caps": 0, 2024-09-21T03:56:35.001 INFO:teuthology.orchestra.run.smithi039.stdout: "subtrees": 3, 2024-09-21T03:56:35.001 INFO:teuthology.orchestra.run.smithi039.stdout: "traverse": 8, 2024-09-21T03:56:35.001 INFO:teuthology.orchestra.run.smithi039.stdout: "traverse_hit": 4, 2024-09-21T03:56:35.001 INFO:teuthology.orchestra.run.smithi039.stdout: "traverse_forward": 1, 2024-09-21T03:56:35.001 INFO:teuthology.orchestra.run.smithi039.stdout: "traverse_discover": 3, 2024-09-21T03:56:35.001 INFO:teuthology.orchestra.run.smithi039.stdout: "traverse_dir_fetch": 0, 2024-09-21T03:56:35.001 INFO:teuthology.orchestra.run.smithi039.stdout: "traverse_remote_ino": 0, 2024-09-21T03:56:35.001 INFO:teuthology.orchestra.run.smithi039.stdout: "traverse_lock": 0, 2024-09-21T03:56:35.001 INFO:teuthology.orchestra.run.smithi039.stdout: "load_cent": 0, 2024-09-21T03:56:35.001 INFO:teuthology.orchestra.run.smithi039.stdout: "q": 0, 2024-09-21T03:56:35.001 INFO:teuthology.orchestra.run.smithi039.stdout: "exported": 0, 2024-09-21T03:56:35.001 INFO:teuthology.orchestra.run.smithi039.stdout: "exported_inodes": 0, 2024-09-21T03:56:35.001 INFO:teuthology.orchestra.run.smithi039.stdout: "imported": 0, 2024-09-21T03:56:35.001 INFO:teuthology.orchestra.run.smithi039.stdout: "imported_inodes": 0, 2024-09-21T03:56:35.002 INFO:teuthology.orchestra.run.smithi039.stdout: "openino_dir_fetch": 0, 2024-09-21T03:56:35.002 INFO:teuthology.orchestra.run.smithi039.stdout: "openino_backtrace_fetch": 0, 2024-09-21T03:56:35.002 INFO:teuthology.orchestra.run.smithi039.stdout: "openino_peer_discover": 0, 2024-09-21T03:56:35.002 INFO:teuthology.orchestra.run.smithi039.stdout: "root_rfiles": 4, 2024-09-21T03:56:35.002 INFO:teuthology.orchestra.run.smithi039.stdout: "root_rbytes": 234, 2024-09-21T03:56:35.002 INFO:teuthology.orchestra.run.smithi039.stdout: "root_rsnaps": 6, 2024-09-21T03:56:35.002 INFO:teuthology.orchestra.run.smithi039.stdout: "scrub_backtrace_fetch": 0, 2024-09-21T03:56:35.002 INFO:teuthology.orchestra.run.smithi039.stdout: "scrub_set_tag": 0, 2024-09-21T03:56:35.002 INFO:teuthology.orchestra.run.smithi039.stdout: "scrub_backtrace_repaired": 0, 2024-09-21T03:56:35.002 INFO:teuthology.orchestra.run.smithi039.stdout: "scrub_inotable_repaired": 0, 2024-09-21T03:56:35.002 INFO:teuthology.orchestra.run.smithi039.stdout: "scrub_dir_inodes": 0, 2024-09-21T03:56:35.002 INFO:teuthology.orchestra.run.smithi039.stdout: "scrub_dir_base_inodes": 0, 2024-09-21T03:56:35.002 INFO:teuthology.orchestra.run.smithi039.stdout: "scrub_dirfrag_rstats": 0, 2024-09-21T03:56:35.002 INFO:teuthology.orchestra.run.smithi039.stdout: "scrub_file_inodes": 0, 2024-09-21T03:56:35.003 INFO:teuthology.orchestra.run.smithi039.stdout: "handle_inode_file_caps": 0, 2024-09-21T03:56:35.003 INFO:teuthology.orchestra.run.smithi039.stdout: "ceph_cap_op_revoke": 0, 2024-09-21T03:56:35.003 INFO:teuthology.orchestra.run.smithi039.stdout: "ceph_cap_op_grant": 0, 2024-09-21T03:56:35.003 INFO:teuthology.orchestra.run.smithi039.stdout: "ceph_cap_op_trunc": 0, 2024-09-21T03:56:35.003 INFO:teuthology.orchestra.run.smithi039.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-21T03:56:35.003 INFO:teuthology.orchestra.run.smithi039.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-21T03:56:35.003 INFO:teuthology.orchestra.run.smithi039.stdout: "handle_client_caps": 0, 2024-09-21T03:56:35.003 INFO:teuthology.orchestra.run.smithi039.stdout: "handle_client_caps_dirty": 0, 2024-09-21T03:56:35.003 INFO:teuthology.orchestra.run.smithi039.stdout: "handle_client_cap_release": 0, 2024-09-21T03:56:35.003 INFO:teuthology.orchestra.run.smithi039.stdout: "process_request_cap_release": 0 2024-09-21T03:56:35.003 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:35.003 INFO:teuthology.orchestra.run.smithi039.stdout: "mds_cache": { 2024-09-21T03:56:35.003 INFO:teuthology.orchestra.run.smithi039.stdout: "dir_update": 0, 2024-09-21T03:56:35.003 INFO:teuthology.orchestra.run.smithi039.stdout: "dir_update_receipt": 11, 2024-09-21T03:56:35.003 INFO:teuthology.orchestra.run.smithi039.stdout: "dir_try_discover": 7, 2024-09-21T03:56:35.004 INFO:teuthology.orchestra.run.smithi039.stdout: "dir_send_discover": 8, 2024-09-21T03:56:35.004 INFO:teuthology.orchestra.run.smithi039.stdout: "dir_handle_discover": 0, 2024-09-21T03:56:35.004 INFO:teuthology.orchestra.run.smithi039.stdout: "num_strays": 0, 2024-09-21T03:56:35.004 INFO:teuthology.orchestra.run.smithi039.stdout: "num_strays_delayed": 0, 2024-09-21T03:56:35.004 INFO:teuthology.orchestra.run.smithi039.stdout: "num_strays_enqueuing": 0, 2024-09-21T03:56:35.004 INFO:teuthology.orchestra.run.smithi039.stdout: "strays_created": 0, 2024-09-21T03:56:35.004 INFO:teuthology.orchestra.run.smithi039.stdout: "strays_enqueued": 0, 2024-09-21T03:56:35.004 INFO:teuthology.orchestra.run.smithi039.stdout: "strays_reintegrated": 0, 2024-09-21T03:56:35.004 INFO:teuthology.orchestra.run.smithi039.stdout: "strays_migrated": 0, 2024-09-21T03:56:35.004 INFO:teuthology.orchestra.run.smithi039.stdout: "num_recovering_processing": 0, 2024-09-21T03:56:35.004 INFO:teuthology.orchestra.run.smithi039.stdout: "num_recovering_enqueued": 0, 2024-09-21T03:56:35.004 INFO:teuthology.orchestra.run.smithi039.stdout: "num_recovering_prioritized": 0, 2024-09-21T03:56:35.004 INFO:teuthology.orchestra.run.smithi039.stdout: "recovery_started": 0, 2024-09-21T03:56:35.004 INFO:teuthology.orchestra.run.smithi039.stdout: "recovery_completed": 0, 2024-09-21T03:56:35.005 INFO:teuthology.orchestra.run.smithi039.stdout: "ireq_quiesce_path": 0, 2024-09-21T03:56:35.005 INFO:teuthology.orchestra.run.smithi039.stdout: "ireq_quiesce_inode": 0, 2024-09-21T03:56:35.005 INFO:teuthology.orchestra.run.smithi039.stdout: "ireq_enqueue_scrub": 0, 2024-09-21T03:56:35.005 INFO:teuthology.orchestra.run.smithi039.stdout: "ireq_exportdir": 0, 2024-09-21T03:56:35.005 INFO:teuthology.orchestra.run.smithi039.stdout: "ireq_flush": 0, 2024-09-21T03:56:35.005 INFO:teuthology.orchestra.run.smithi039.stdout: "ireq_fragmentdir": 0, 2024-09-21T03:56:35.005 INFO:teuthology.orchestra.run.smithi039.stdout: "ireq_fragstats": 0, 2024-09-21T03:56:35.005 INFO:teuthology.orchestra.run.smithi039.stdout: "ireq_inodestats": 0, 2024-09-21T03:56:35.005 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_started": 0, 2024-09-21T03:56:35.005 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_succeeded": 0, 2024-09-21T03:56:35.005 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_write_failed": 0 2024-09-21T03:56:35.005 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:35.005 INFO:teuthology.orchestra.run.smithi039.stdout: "mds_log": { 2024-09-21T03:56:35.005 INFO:teuthology.orchestra.run.smithi039.stdout: "evlrg": 0, 2024-09-21T03:56:35.006 INFO:teuthology.orchestra.run.smithi039.stdout: "evadd": 7, 2024-09-21T03:56:35.006 INFO:teuthology.orchestra.run.smithi039.stdout: "evex": 0, 2024-09-21T03:56:35.006 INFO:teuthology.orchestra.run.smithi039.stdout: "evtrm": 0, 2024-09-21T03:56:35.006 INFO:teuthology.orchestra.run.smithi039.stdout: "ev": 7, 2024-09-21T03:56:35.006 INFO:teuthology.orchestra.run.smithi039.stdout: "evexg": 0, 2024-09-21T03:56:35.006 INFO:teuthology.orchestra.run.smithi039.stdout: "evexd": 0, 2024-09-21T03:56:35.006 INFO:teuthology.orchestra.run.smithi039.stdout: "segadd": 2, 2024-09-21T03:56:35.006 INFO:teuthology.orchestra.run.smithi039.stdout: "segex": 0, 2024-09-21T03:56:35.006 INFO:teuthology.orchestra.run.smithi039.stdout: "segtrm": 0, 2024-09-21T03:56:35.006 INFO:teuthology.orchestra.run.smithi039.stdout: "seg": 2, 2024-09-21T03:56:35.006 INFO:teuthology.orchestra.run.smithi039.stdout: "segmjr": 2, 2024-09-21T03:56:35.006 INFO:teuthology.orchestra.run.smithi039.stdout: "segexg": 0, 2024-09-21T03:56:35.006 INFO:teuthology.orchestra.run.smithi039.stdout: "segexd": 0, 2024-09-21T03:56:35.007 INFO:teuthology.orchestra.run.smithi039.stdout: "expos": 4194304, 2024-09-21T03:56:35.007 INFO:teuthology.orchestra.run.smithi039.stdout: "wrpos": 4196264, 2024-09-21T03:56:35.007 INFO:teuthology.orchestra.run.smithi039.stdout: "rdpos": 0, 2024-09-21T03:56:35.007 INFO:teuthology.orchestra.run.smithi039.stdout: "jlat": { 2024-09-21T03:56:35.007 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 6, 2024-09-21T03:56:35.007 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.013998800, 2024-09-21T03:56:35.007 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.002333133 2024-09-21T03:56:35.007 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:35.007 INFO:teuthology.orchestra.run.smithi039.stdout: "replayed": 0 2024-09-21T03:56:35.007 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:35.007 INFO:teuthology.orchestra.run.smithi039.stdout: "mds_mem": { 2024-09-21T03:56:35.007 INFO:teuthology.orchestra.run.smithi039.stdout: "ino": 20, 2024-09-21T03:56:35.007 INFO:teuthology.orchestra.run.smithi039.stdout: "ino+": 22, 2024-09-21T03:56:35.007 INFO:teuthology.orchestra.run.smithi039.stdout: "ino-": 2, 2024-09-21T03:56:35.008 INFO:teuthology.orchestra.run.smithi039.stdout: "dir": 19, 2024-09-21T03:56:35.008 INFO:teuthology.orchestra.run.smithi039.stdout: "dir+": 23, 2024-09-21T03:56:35.008 INFO:teuthology.orchestra.run.smithi039.stdout: "dir-": 4, 2024-09-21T03:56:35.008 INFO:teuthology.orchestra.run.smithi039.stdout: "dn": 16, 2024-09-21T03:56:35.008 INFO:teuthology.orchestra.run.smithi039.stdout: "dn+": 20, 2024-09-21T03:56:35.008 INFO:teuthology.orchestra.run.smithi039.stdout: "dn-": 4, 2024-09-21T03:56:35.008 INFO:teuthology.orchestra.run.smithi039.stdout: "cap": 0, 2024-09-21T03:56:35.008 INFO:teuthology.orchestra.run.smithi039.stdout: "cap+": 0, 2024-09-21T03:56:35.008 INFO:teuthology.orchestra.run.smithi039.stdout: "cap-": 0, 2024-09-21T03:56:35.008 INFO:teuthology.orchestra.run.smithi039.stdout: "rss": 118780, 2024-09-21T03:56:35.008 INFO:teuthology.orchestra.run.smithi039.stdout: "heap": 325861376 2024-09-21T03:56:35.008 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:35.008 INFO:teuthology.orchestra.run.smithi039.stdout: "mds_server": { 2024-09-21T03:56:35.009 INFO:teuthology.orchestra.run.smithi039.stdout: "dispatch_client_request": 1, 2024-09-21T03:56:35.009 INFO:teuthology.orchestra.run.smithi039.stdout: "dispatch_server_request": 0, 2024-09-21T03:56:35.009 INFO:teuthology.orchestra.run.smithi039.stdout: "handle_client_request": 1, 2024-09-21T03:56:35.009 INFO:teuthology.orchestra.run.smithi039.stdout: "handle_client_session": 17, 2024-09-21T03:56:35.009 INFO:teuthology.orchestra.run.smithi039.stdout: "handle_peer_request": 0, 2024-09-21T03:56:35.009 INFO:teuthology.orchestra.run.smithi039.stdout: "req_create_latency": { 2024-09-21T03:56:35.009 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2024-09-21T03:56:35.009 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2024-09-21T03:56:35.009 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2024-09-21T03:56:35.009 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:35.009 INFO:teuthology.orchestra.run.smithi039.stdout: "req_getattr_latency": { 2024-09-21T03:56:35.009 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2024-09-21T03:56:35.009 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2024-09-21T03:56:35.009 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2024-09-21T03:56:35.010 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:35.010 INFO:teuthology.orchestra.run.smithi039.stdout: "req_getfilelock_latency": { 2024-09-21T03:56:35.010 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2024-09-21T03:56:35.010 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2024-09-21T03:56:35.010 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2024-09-21T03:56:35.010 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:35.010 INFO:teuthology.orchestra.run.smithi039.stdout: "req_link_latency": { 2024-09-21T03:56:35.010 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2024-09-21T03:56:35.010 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2024-09-21T03:56:35.010 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2024-09-21T03:56:35.010 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:35.010 INFO:teuthology.orchestra.run.smithi039.stdout: "req_lookup_latency": { 2024-09-21T03:56:35.010 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2024-09-21T03:56:35.011 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2024-09-21T03:56:35.011 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2024-09-21T03:56:35.011 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:35.011 INFO:teuthology.orchestra.run.smithi039.stdout: "req_lookuphash_latency": { 2024-09-21T03:56:35.011 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2024-09-21T03:56:35.011 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2024-09-21T03:56:35.011 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2024-09-21T03:56:35.011 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:35.011 INFO:teuthology.orchestra.run.smithi039.stdout: "req_lookupino_latency": { 2024-09-21T03:56:35.011 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2024-09-21T03:56:35.011 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2024-09-21T03:56:35.011 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2024-09-21T03:56:35.011 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:35.011 INFO:teuthology.orchestra.run.smithi039.stdout: "req_lookupname_latency": { 2024-09-21T03:56:35.012 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2024-09-21T03:56:35.012 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2024-09-21T03:56:35.012 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2024-09-21T03:56:35.012 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:35.012 INFO:teuthology.orchestra.run.smithi039.stdout: "req_lookupparent_latency": { 2024-09-21T03:56:35.012 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2024-09-21T03:56:35.012 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2024-09-21T03:56:35.012 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2024-09-21T03:56:35.012 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:35.012 INFO:teuthology.orchestra.run.smithi039.stdout: "req_lookupsnap_latency": { 2024-09-21T03:56:35.012 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2024-09-21T03:56:35.012 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2024-09-21T03:56:35.012 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2024-09-21T03:56:35.012 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:35.013 INFO:teuthology.orchestra.run.smithi039.stdout: "req_lssnap_latency": { 2024-09-21T03:56:35.013 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2024-09-21T03:56:35.013 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2024-09-21T03:56:35.013 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2024-09-21T03:56:35.013 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:35.013 INFO:teuthology.orchestra.run.smithi039.stdout: "req_mkdir_latency": { 2024-09-21T03:56:35.013 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2024-09-21T03:56:35.013 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2024-09-21T03:56:35.013 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2024-09-21T03:56:35.014 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:35.014 INFO:teuthology.orchestra.run.smithi039.stdout: "req_mknod_latency": { 2024-09-21T03:56:35.014 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2024-09-21T03:56:35.014 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2024-09-21T03:56:35.014 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2024-09-21T03:56:35.014 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:35.014 INFO:teuthology.orchestra.run.smithi039.stdout: "req_mksnap_latency": { 2024-09-21T03:56:35.014 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2024-09-21T03:56:35.014 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2024-09-21T03:56:35.014 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2024-09-21T03:56:35.014 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:35.014 INFO:teuthology.orchestra.run.smithi039.stdout: "req_open_latency": { 2024-09-21T03:56:35.014 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2024-09-21T03:56:35.014 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2024-09-21T03:56:35.015 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2024-09-21T03:56:35.015 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:35.015 INFO:teuthology.orchestra.run.smithi039.stdout: "req_readdir_latency": { 2024-09-21T03:56:35.015 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2024-09-21T03:56:35.015 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2024-09-21T03:56:35.015 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2024-09-21T03:56:35.015 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:35.015 INFO:teuthology.orchestra.run.smithi039.stdout: "req_rename_latency": { 2024-09-21T03:56:35.015 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2024-09-21T03:56:35.015 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2024-09-21T03:56:35.015 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2024-09-21T03:56:35.015 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:35.015 INFO:teuthology.orchestra.run.smithi039.stdout: "req_renamesnap_latency": { 2024-09-21T03:56:35.015 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2024-09-21T03:56:35.016 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2024-09-21T03:56:35.016 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2024-09-21T03:56:35.016 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:35.016 INFO:teuthology.orchestra.run.smithi039.stdout: "req_snapdiff_latency": { 2024-09-21T03:56:35.016 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2024-09-21T03:56:35.016 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2024-09-21T03:56:35.016 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2024-09-21T03:56:35.016 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:35.016 INFO:teuthology.orchestra.run.smithi039.stdout: "req_rmdir_latency": { 2024-09-21T03:56:35.016 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2024-09-21T03:56:35.016 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2024-09-21T03:56:35.016 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2024-09-21T03:56:35.016 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:35.016 INFO:teuthology.orchestra.run.smithi039.stdout: "req_rmsnap_latency": { 2024-09-21T03:56:35.017 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2024-09-21T03:56:35.017 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2024-09-21T03:56:35.017 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2024-09-21T03:56:35.017 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:35.017 INFO:teuthology.orchestra.run.smithi039.stdout: "req_rmxattr_latency": { 2024-09-21T03:56:35.017 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2024-09-21T03:56:35.017 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2024-09-21T03:56:35.017 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2024-09-21T03:56:35.017 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:35.017 INFO:teuthology.orchestra.run.smithi039.stdout: "req_setattr_latency": { 2024-09-21T03:56:35.017 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2024-09-21T03:56:35.017 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2024-09-21T03:56:35.017 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2024-09-21T03:56:35.017 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:35.018 INFO:teuthology.orchestra.run.smithi039.stdout: "req_setdirlayout_latency": { 2024-09-21T03:56:35.018 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2024-09-21T03:56:35.018 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2024-09-21T03:56:35.018 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2024-09-21T03:56:35.018 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:35.018 INFO:teuthology.orchestra.run.smithi039.stdout: "req_setfilelock_latency": { 2024-09-21T03:56:35.018 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2024-09-21T03:56:35.018 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2024-09-21T03:56:35.018 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2024-09-21T03:56:35.018 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:35.018 INFO:teuthology.orchestra.run.smithi039.stdout: "req_setlayout_latency": { 2024-09-21T03:56:35.018 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2024-09-21T03:56:35.018 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2024-09-21T03:56:35.019 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2024-09-21T03:56:35.019 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:35.019 INFO:teuthology.orchestra.run.smithi039.stdout: "req_setxattr_latency": { 2024-09-21T03:56:35.019 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2024-09-21T03:56:35.019 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2024-09-21T03:56:35.019 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2024-09-21T03:56:35.019 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:35.019 INFO:teuthology.orchestra.run.smithi039.stdout: "req_symlink_latency": { 2024-09-21T03:56:35.019 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2024-09-21T03:56:35.019 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2024-09-21T03:56:35.019 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2024-09-21T03:56:35.019 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:35.019 INFO:teuthology.orchestra.run.smithi039.stdout: "req_unlink_latency": { 2024-09-21T03:56:35.019 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2024-09-21T03:56:35.020 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2024-09-21T03:56:35.020 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2024-09-21T03:56:35.020 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:35.020 INFO:teuthology.orchestra.run.smithi039.stdout: "cap_revoke_eviction": 0, 2024-09-21T03:56:35.020 INFO:teuthology.orchestra.run.smithi039.stdout: "cap_acquisition_throttle": 0, 2024-09-21T03:56:35.020 INFO:teuthology.orchestra.run.smithi039.stdout: "req_getvxattr_latency": { 2024-09-21T03:56:35.020 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2024-09-21T03:56:35.020 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2024-09-21T03:56:35.020 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2024-09-21T03:56:35.020 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-21T03:56:35.020 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:35.020 INFO:teuthology.orchestra.run.smithi039.stdout: "mds_sessions": { 2024-09-21T03:56:35.020 INFO:teuthology.orchestra.run.smithi039.stdout: "session_count": 1, 2024-09-21T03:56:35.020 INFO:teuthology.orchestra.run.smithi039.stdout: "session_add": 3, 2024-09-21T03:56:35.020 INFO:teuthology.orchestra.run.smithi039.stdout: "session_remove": 2, 2024-09-21T03:56:35.021 INFO:teuthology.orchestra.run.smithi039.stdout: "sessions_open": 1, 2024-09-21T03:56:35.021 INFO:teuthology.orchestra.run.smithi039.stdout: "sessions_stale": 0, 2024-09-21T03:56:35.021 INFO:teuthology.orchestra.run.smithi039.stdout: "total_load": 0, 2024-09-21T03:56:35.021 INFO:teuthology.orchestra.run.smithi039.stdout: "average_load": 0, 2024-09-21T03:56:35.021 INFO:teuthology.orchestra.run.smithi039.stdout: "avg_session_uptime": 624, 2024-09-21T03:56:35.021 INFO:teuthology.orchestra.run.smithi039.stdout: "mdthresh_evicted": 0 2024-09-21T03:56:35.021 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:35.021 INFO:teuthology.orchestra.run.smithi039.stdout: "mempool": { 2024-09-21T03:56:35.021 INFO:teuthology.orchestra.run.smithi039.stdout: "bloom_filter_bytes": 0, 2024-09-21T03:56:35.021 INFO:teuthology.orchestra.run.smithi039.stdout: "bloom_filter_items": 0, 2024-09-21T03:56:35.021 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_alloc_bytes": 0, 2024-09-21T03:56:35.021 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_alloc_items": 0, 2024-09-21T03:56:35.021 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_cache_data_bytes": 0, 2024-09-21T03:56:35.021 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_cache_data_items": 0, 2024-09-21T03:56:35.021 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-21T03:56:35.022 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_cache_onode_items": 0, 2024-09-21T03:56:35.022 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-21T03:56:35.022 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_cache_meta_items": 0, 2024-09-21T03:56:35.022 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_cache_other_bytes": 0, 2024-09-21T03:56:35.022 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_cache_other_items": 0, 2024-09-21T03:56:35.022 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-21T03:56:35.022 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_cache_buffer_items": 0, 2024-09-21T03:56:35.022 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_extent_bytes": 0, 2024-09-21T03:56:35.022 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_extent_items": 0, 2024-09-21T03:56:35.022 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_blob_bytes": 0, 2024-09-21T03:56:35.022 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_blob_items": 0, 2024-09-21T03:56:35.022 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-21T03:56:35.022 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_shared_blob_items": 0, 2024-09-21T03:56:35.022 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-21T03:56:35.023 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_inline_bl_items": 0, 2024-09-21T03:56:35.023 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_fsck_bytes": 0, 2024-09-21T03:56:35.023 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_fsck_items": 0, 2024-09-21T03:56:35.023 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_txc_bytes": 0, 2024-09-21T03:56:35.023 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_txc_items": 0, 2024-09-21T03:56:35.023 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-21T03:56:35.023 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_writing_deferred_items": 0, 2024-09-21T03:56:35.023 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_writing_bytes": 0, 2024-09-21T03:56:35.023 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_writing_items": 0, 2024-09-21T03:56:35.023 INFO:teuthology.orchestra.run.smithi039.stdout: "bluefs_bytes": 0, 2024-09-21T03:56:35.023 INFO:teuthology.orchestra.run.smithi039.stdout: "bluefs_items": 0, 2024-09-21T03:56:35.023 INFO:teuthology.orchestra.run.smithi039.stdout: "bluefs_file_reader_bytes": 0, 2024-09-21T03:56:35.023 INFO:teuthology.orchestra.run.smithi039.stdout: "bluefs_file_reader_items": 0, 2024-09-21T03:56:35.024 INFO:teuthology.orchestra.run.smithi039.stdout: "bluefs_file_writer_bytes": 0, 2024-09-21T03:56:35.024 INFO:teuthology.orchestra.run.smithi039.stdout: "bluefs_file_writer_items": 0, 2024-09-21T03:56:35.024 INFO:teuthology.orchestra.run.smithi039.stdout: "buffer_anon_bytes": 238711, 2024-09-21T03:56:35.024 INFO:teuthology.orchestra.run.smithi039.stdout: "buffer_anon_items": 81, 2024-09-21T03:56:35.024 INFO:teuthology.orchestra.run.smithi039.stdout: "buffer_meta_bytes": 0, 2024-09-21T03:56:35.024 INFO:teuthology.orchestra.run.smithi039.stdout: "buffer_meta_items": 0, 2024-09-21T03:56:35.024 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_bytes": 0, 2024-09-21T03:56:35.024 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_items": 0, 2024-09-21T03:56:35.024 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_mapbl_bytes": 0, 2024-09-21T03:56:35.024 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_mapbl_items": 0, 2024-09-21T03:56:35.024 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_pglog_bytes": 0, 2024-09-21T03:56:35.024 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_pglog_items": 0, 2024-09-21T03:56:35.024 INFO:teuthology.orchestra.run.smithi039.stdout: "osdmap_bytes": 8256, 2024-09-21T03:56:35.024 INFO:teuthology.orchestra.run.smithi039.stdout: "osdmap_items": 182, 2024-09-21T03:56:35.024 INFO:teuthology.orchestra.run.smithi039.stdout: "osdmap_mapping_bytes": 0, 2024-09-21T03:56:35.024 INFO:teuthology.orchestra.run.smithi039.stdout: "osdmap_mapping_items": 0, 2024-09-21T03:56:35.025 INFO:teuthology.orchestra.run.smithi039.stdout: "pgmap_bytes": 0, 2024-09-21T03:56:35.025 INFO:teuthology.orchestra.run.smithi039.stdout: "pgmap_items": 0, 2024-09-21T03:56:35.025 INFO:teuthology.orchestra.run.smithi039.stdout: "mds_co_bytes": 86065, 2024-09-21T03:56:35.025 INFO:teuthology.orchestra.run.smithi039.stdout: "mds_co_items": 419, 2024-09-21T03:56:35.025 INFO:teuthology.orchestra.run.smithi039.stdout: "unittest_1_bytes": 0, 2024-09-21T03:56:35.025 INFO:teuthology.orchestra.run.smithi039.stdout: "unittest_1_items": 0, 2024-09-21T03:56:35.025 INFO:teuthology.orchestra.run.smithi039.stdout: "unittest_2_bytes": 0, 2024-09-21T03:56:35.025 INFO:teuthology.orchestra.run.smithi039.stdout: "unittest_2_items": 0 2024-09-21T03:56:35.025 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:35.025 INFO:teuthology.orchestra.run.smithi039.stdout: "objecter": { 2024-09-21T03:56:35.025 INFO:teuthology.orchestra.run.smithi039.stdout: "op_active": 0, 2024-09-21T03:56:35.025 INFO:teuthology.orchestra.run.smithi039.stdout: "op_laggy": 0, 2024-09-21T03:56:35.025 INFO:teuthology.orchestra.run.smithi039.stdout: "op_send": 42, 2024-09-21T03:56:35.026 INFO:teuthology.orchestra.run.smithi039.stdout: "op_send_bytes": 12262, 2024-09-21T03:56:35.026 INFO:teuthology.orchestra.run.smithi039.stdout: "op_resend": 33, 2024-09-21T03:56:35.026 INFO:teuthology.orchestra.run.smithi039.stdout: "op_reply": 42, 2024-09-21T03:56:35.026 INFO:teuthology.orchestra.run.smithi039.stdout: "op_latency": { 2024-09-21T03:56:35.026 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 42, 2024-09-21T03:56:35.026 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.287994240, 2024-09-21T03:56:35.026 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.006857005 2024-09-21T03:56:35.026 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:35.026 INFO:teuthology.orchestra.run.smithi039.stdout: "op_inflight": 0, 2024-09-21T03:56:35.026 INFO:teuthology.orchestra.run.smithi039.stdout: "oplen_avg": { 2024-09-21T03:56:35.026 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 42, 2024-09-21T03:56:35.026 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 64 2024-09-21T03:56:35.026 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:35.026 INFO:teuthology.orchestra.run.smithi039.stdout: "op": 42, 2024-09-21T03:56:35.027 INFO:teuthology.orchestra.run.smithi039.stdout: "op_r": 0, 2024-09-21T03:56:35.027 INFO:teuthology.orchestra.run.smithi039.stdout: "op_w": 42, 2024-09-21T03:56:35.027 INFO:teuthology.orchestra.run.smithi039.stdout: "op_rmw": 0, 2024-09-21T03:56:35.027 INFO:teuthology.orchestra.run.smithi039.stdout: "op_pg": 0, 2024-09-21T03:56:35.027 INFO:teuthology.orchestra.run.smithi039.stdout: "osdop_stat": 0, 2024-09-21T03:56:35.027 INFO:teuthology.orchestra.run.smithi039.stdout: "osdop_create": 10, 2024-09-21T03:56:35.027 INFO:teuthology.orchestra.run.smithi039.stdout: "osdop_read": 0, 2024-09-21T03:56:35.027 INFO:teuthology.orchestra.run.smithi039.stdout: "osdop_write": 6, 2024-09-21T03:56:35.027 INFO:teuthology.orchestra.run.smithi039.stdout: "osdop_writefull": 7, 2024-09-21T03:56:35.027 INFO:teuthology.orchestra.run.smithi039.stdout: "osdop_writesame": 0, 2024-09-21T03:56:35.027 INFO:teuthology.orchestra.run.smithi039.stdout: "osdop_append": 0, 2024-09-21T03:56:35.027 INFO:teuthology.orchestra.run.smithi039.stdout: "osdop_zero": 0, 2024-09-21T03:56:35.027 INFO:teuthology.orchestra.run.smithi039.stdout: "osdop_truncate": 0, 2024-09-21T03:56:35.027 INFO:teuthology.orchestra.run.smithi039.stdout: "osdop_delete": 6, 2024-09-21T03:56:35.027 INFO:teuthology.orchestra.run.smithi039.stdout: "osdop_mapext": 0, 2024-09-21T03:56:35.028 INFO:teuthology.orchestra.run.smithi039.stdout: "osdop_sparse_read": 0, 2024-09-21T03:56:35.028 INFO:teuthology.orchestra.run.smithi039.stdout: "osdop_clonerange": 0, 2024-09-21T03:56:35.028 INFO:teuthology.orchestra.run.smithi039.stdout: "osdop_getxattr": 0, 2024-09-21T03:56:35.028 INFO:teuthology.orchestra.run.smithi039.stdout: "osdop_setxattr": 20, 2024-09-21T03:56:35.028 INFO:teuthology.orchestra.run.smithi039.stdout: "osdop_cmpxattr": 0, 2024-09-21T03:56:35.028 INFO:teuthology.orchestra.run.smithi039.stdout: "osdop_rmxattr": 0, 2024-09-21T03:56:35.028 INFO:teuthology.orchestra.run.smithi039.stdout: "osdop_resetxattrs": 0, 2024-09-21T03:56:35.028 INFO:teuthology.orchestra.run.smithi039.stdout: "osdop_call": 0, 2024-09-21T03:56:35.028 INFO:teuthology.orchestra.run.smithi039.stdout: "osdop_watch": 0, 2024-09-21T03:56:35.028 INFO:teuthology.orchestra.run.smithi039.stdout: "osdop_notify": 0, 2024-09-21T03:56:35.028 INFO:teuthology.orchestra.run.smithi039.stdout: "osdop_src_cmpxattr": 0, 2024-09-21T03:56:35.028 INFO:teuthology.orchestra.run.smithi039.stdout: "osdop_pgls": 0, 2024-09-21T03:56:35.028 INFO:teuthology.orchestra.run.smithi039.stdout: "osdop_pgls_filter": 0, 2024-09-21T03:56:35.028 INFO:teuthology.orchestra.run.smithi039.stdout: "osdop_other": 0, 2024-09-21T03:56:35.028 INFO:teuthology.orchestra.run.smithi039.stdout: "linger_active": 0, 2024-09-21T03:56:35.029 INFO:teuthology.orchestra.run.smithi039.stdout: "linger_send": 0, 2024-09-21T03:56:35.029 INFO:teuthology.orchestra.run.smithi039.stdout: "linger_resend": 0, 2024-09-21T03:56:35.029 INFO:teuthology.orchestra.run.smithi039.stdout: "linger_ping": 0, 2024-09-21T03:56:35.029 INFO:teuthology.orchestra.run.smithi039.stdout: "poolop_active": 0, 2024-09-21T03:56:35.029 INFO:teuthology.orchestra.run.smithi039.stdout: "poolop_send": 0, 2024-09-21T03:56:35.029 INFO:teuthology.orchestra.run.smithi039.stdout: "poolop_resend": 0, 2024-09-21T03:56:35.029 INFO:teuthology.orchestra.run.smithi039.stdout: "poolstat_active": 0, 2024-09-21T03:56:35.029 INFO:teuthology.orchestra.run.smithi039.stdout: "poolstat_send": 0, 2024-09-21T03:56:35.029 INFO:teuthology.orchestra.run.smithi039.stdout: "poolstat_resend": 0, 2024-09-21T03:56:35.029 INFO:teuthology.orchestra.run.smithi039.stdout: "statfs_active": 0, 2024-09-21T03:56:35.029 INFO:teuthology.orchestra.run.smithi039.stdout: "statfs_send": 0, 2024-09-21T03:56:35.029 INFO:teuthology.orchestra.run.smithi039.stdout: "statfs_resend": 0, 2024-09-21T03:56:35.029 INFO:teuthology.orchestra.run.smithi039.stdout: "command_active": 0, 2024-09-21T03:56:35.029 INFO:teuthology.orchestra.run.smithi039.stdout: "command_send": 0, 2024-09-21T03:56:35.030 INFO:teuthology.orchestra.run.smithi039.stdout: "command_resend": 0, 2024-09-21T03:56:35.030 INFO:teuthology.orchestra.run.smithi039.stdout: "map_epoch": 87, 2024-09-21T03:56:35.030 INFO:teuthology.orchestra.run.smithi039.stdout: "map_full": 0, 2024-09-21T03:56:35.030 INFO:teuthology.orchestra.run.smithi039.stdout: "map_inc": 10, 2024-09-21T03:56:35.030 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_sessions": 9, 2024-09-21T03:56:35.030 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_session_open": 9, 2024-09-21T03:56:35.030 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_session_close": 0, 2024-09-21T03:56:35.030 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_laggy": 0, 2024-09-21T03:56:35.030 INFO:teuthology.orchestra.run.smithi039.stdout: "omap_wr": 14, 2024-09-21T03:56:35.030 INFO:teuthology.orchestra.run.smithi039.stdout: "omap_rd": 0, 2024-09-21T03:56:35.030 INFO:teuthology.orchestra.run.smithi039.stdout: "omap_del": 1 2024-09-21T03:56:35.030 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:35.030 INFO:teuthology.orchestra.run.smithi039.stdout: "oft": { 2024-09-21T03:56:35.030 INFO:teuthology.orchestra.run.smithi039.stdout: "omap_total_objs": 1, 2024-09-21T03:56:35.031 INFO:teuthology.orchestra.run.smithi039.stdout: "omap_total_kv_pairs": 0, 2024-09-21T03:56:35.031 INFO:teuthology.orchestra.run.smithi039.stdout: "omap_total_updates": 0, 2024-09-21T03:56:35.031 INFO:teuthology.orchestra.run.smithi039.stdout: "omap_total_removes": 0 2024-09-21T03:56:35.031 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:35.031 INFO:teuthology.orchestra.run.smithi039.stdout: "purge_queue": { 2024-09-21T03:56:35.031 INFO:teuthology.orchestra.run.smithi039.stdout: "pq_executing_ops": 0, 2024-09-21T03:56:35.031 INFO:teuthology.orchestra.run.smithi039.stdout: "pq_executing_ops_high_water": 0, 2024-09-21T03:56:35.031 INFO:teuthology.orchestra.run.smithi039.stdout: "pq_executing": 0, 2024-09-21T03:56:35.031 INFO:teuthology.orchestra.run.smithi039.stdout: "pq_executing_high_water": 0, 2024-09-21T03:56:35.031 INFO:teuthology.orchestra.run.smithi039.stdout: "pq_executed_ops": 0, 2024-09-21T03:56:35.031 INFO:teuthology.orchestra.run.smithi039.stdout: "pq_executed": 0, 2024-09-21T03:56:35.031 INFO:teuthology.orchestra.run.smithi039.stdout: "pq_item_in_journal": 0 2024-09-21T03:56:35.031 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:35.032 INFO:teuthology.orchestra.run.smithi039.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-21T03:56:35.032 INFO:teuthology.orchestra.run.smithi039.stdout: "val": 0, 2024-09-21T03:56:35.032 INFO:teuthology.orchestra.run.smithi039.stdout: "max": 104857600, 2024-09-21T03:56:35.032 INFO:teuthology.orchestra.run.smithi039.stdout: "get_started": 0, 2024-09-21T03:56:35.032 INFO:teuthology.orchestra.run.smithi039.stdout: "get": 780, 2024-09-21T03:56:35.032 INFO:teuthology.orchestra.run.smithi039.stdout: "get_sum": 1268605, 2024-09-21T03:56:35.032 INFO:teuthology.orchestra.run.smithi039.stdout: "get_or_fail_fail": 0, 2024-09-21T03:56:35.032 INFO:teuthology.orchestra.run.smithi039.stdout: "get_or_fail_success": 780, 2024-09-21T03:56:35.032 INFO:teuthology.orchestra.run.smithi039.stdout: "take": 0, 2024-09-21T03:56:35.032 INFO:teuthology.orchestra.run.smithi039.stdout: "take_sum": 0, 2024-09-21T03:56:35.032 INFO:teuthology.orchestra.run.smithi039.stdout: "put": 780, 2024-09-21T03:56:35.032 INFO:teuthology.orchestra.run.smithi039.stdout: "put_sum": 1268605, 2024-09-21T03:56:35.032 INFO:teuthology.orchestra.run.smithi039.stdout: "wait": { 2024-09-21T03:56:35.032 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2024-09-21T03:56:35.032 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2024-09-21T03:56:35.033 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2024-09-21T03:56:35.033 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-21T03:56:35.033 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:35.033 INFO:teuthology.orchestra.run.smithi039.stdout: "throttle-objecter_bytes": { 2024-09-21T03:56:35.033 INFO:teuthology.orchestra.run.smithi039.stdout: "val": 0, 2024-09-21T03:56:35.033 INFO:teuthology.orchestra.run.smithi039.stdout: "max": 104857600, 2024-09-21T03:56:35.033 INFO:teuthology.orchestra.run.smithi039.stdout: "get_started": 0, 2024-09-21T03:56:35.033 INFO:teuthology.orchestra.run.smithi039.stdout: "get": 0, 2024-09-21T03:56:35.033 INFO:teuthology.orchestra.run.smithi039.stdout: "get_sum": 0, 2024-09-21T03:56:35.033 INFO:teuthology.orchestra.run.smithi039.stdout: "get_or_fail_fail": 0, 2024-09-21T03:56:35.033 INFO:teuthology.orchestra.run.smithi039.stdout: "get_or_fail_success": 0, 2024-09-21T03:56:35.033 INFO:teuthology.orchestra.run.smithi039.stdout: "take": 42, 2024-09-21T03:56:35.033 INFO:teuthology.orchestra.run.smithi039.stdout: "take_sum": 12262, 2024-09-21T03:56:35.033 INFO:teuthology.orchestra.run.smithi039.stdout: "put": 42, 2024-09-21T03:56:35.033 INFO:teuthology.orchestra.run.smithi039.stdout: "put_sum": 12262, 2024-09-21T03:56:35.034 INFO:teuthology.orchestra.run.smithi039.stdout: "wait": { 2024-09-21T03:56:35.034 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2024-09-21T03:56:35.034 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2024-09-21T03:56:35.034 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2024-09-21T03:56:35.034 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-21T03:56:35.034 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:35.034 INFO:teuthology.orchestra.run.smithi039.stdout: "throttle-objecter_ops": { 2024-09-21T03:56:35.034 INFO:teuthology.orchestra.run.smithi039.stdout: "val": 0, 2024-09-21T03:56:35.034 INFO:teuthology.orchestra.run.smithi039.stdout: "max": 1024, 2024-09-21T03:56:35.034 INFO:teuthology.orchestra.run.smithi039.stdout: "get_started": 0, 2024-09-21T03:56:35.034 INFO:teuthology.orchestra.run.smithi039.stdout: "get": 0, 2024-09-21T03:56:35.034 INFO:teuthology.orchestra.run.smithi039.stdout: "get_sum": 0, 2024-09-21T03:56:35.034 INFO:teuthology.orchestra.run.smithi039.stdout: "get_or_fail_fail": 0, 2024-09-21T03:56:35.034 INFO:teuthology.orchestra.run.smithi039.stdout: "get_or_fail_success": 0, 2024-09-21T03:56:35.035 INFO:teuthology.orchestra.run.smithi039.stdout: "take": 42, 2024-09-21T03:56:35.035 INFO:teuthology.orchestra.run.smithi039.stdout: "take_sum": 42, 2024-09-21T03:56:35.035 INFO:teuthology.orchestra.run.smithi039.stdout: "put": 42, 2024-09-21T03:56:35.035 INFO:teuthology.orchestra.run.smithi039.stdout: "put_sum": 42, 2024-09-21T03:56:35.035 INFO:teuthology.orchestra.run.smithi039.stdout: "wait": { 2024-09-21T03:56:35.035 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2024-09-21T03:56:35.035 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2024-09-21T03:56:35.035 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2024-09-21T03:56:35.035 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-21T03:56:35.035 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:35.035 INFO:teuthology.orchestra.run.smithi039.stdout: "throttle-write_buf_throttle": { 2024-09-21T03:56:35.035 INFO:teuthology.orchestra.run.smithi039.stdout: "val": 0, 2024-09-21T03:56:35.035 INFO:teuthology.orchestra.run.smithi039.stdout: "max": 3758096384, 2024-09-21T03:56:35.036 INFO:teuthology.orchestra.run.smithi039.stdout: "get_started": 0, 2024-09-21T03:56:35.036 INFO:teuthology.orchestra.run.smithi039.stdout: "get": 0, 2024-09-21T03:56:35.036 INFO:teuthology.orchestra.run.smithi039.stdout: "get_sum": 0, 2024-09-21T03:56:35.036 INFO:teuthology.orchestra.run.smithi039.stdout: "get_or_fail_fail": 0, 2024-09-21T03:56:35.036 INFO:teuthology.orchestra.run.smithi039.stdout: "get_or_fail_success": 0, 2024-09-21T03:56:35.036 INFO:teuthology.orchestra.run.smithi039.stdout: "take": 0, 2024-09-21T03:56:35.036 INFO:teuthology.orchestra.run.smithi039.stdout: "take_sum": 0, 2024-09-21T03:56:35.036 INFO:teuthology.orchestra.run.smithi039.stdout: "put": 0, 2024-09-21T03:56:35.036 INFO:teuthology.orchestra.run.smithi039.stdout: "put_sum": 0, 2024-09-21T03:56:35.036 INFO:teuthology.orchestra.run.smithi039.stdout: "wait": { 2024-09-21T03:56:35.036 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2024-09-21T03:56:35.036 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2024-09-21T03:56:35.036 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2024-09-21T03:56:35.036 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-21T03:56:35.036 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:35.036 INFO:teuthology.orchestra.run.smithi039.stdout: "throttle-write_buf_throttle-0x55a61fbeb450": { 2024-09-21T03:56:35.037 INFO:teuthology.orchestra.run.smithi039.stdout: "val": 0, 2024-09-21T03:56:35.037 INFO:teuthology.orchestra.run.smithi039.stdout: "max": 3758096384, 2024-09-21T03:56:35.037 INFO:teuthology.orchestra.run.smithi039.stdout: "get_started": 0, 2024-09-21T03:56:35.037 INFO:teuthology.orchestra.run.smithi039.stdout: "get": 7, 2024-09-21T03:56:35.037 INFO:teuthology.orchestra.run.smithi039.stdout: "get_sum": 1960, 2024-09-21T03:56:35.037 INFO:teuthology.orchestra.run.smithi039.stdout: "get_or_fail_fail": 0, 2024-09-21T03:56:35.037 INFO:teuthology.orchestra.run.smithi039.stdout: "get_or_fail_success": 7, 2024-09-21T03:56:35.037 INFO:teuthology.orchestra.run.smithi039.stdout: "take": 0, 2024-09-21T03:56:35.037 INFO:teuthology.orchestra.run.smithi039.stdout: "take_sum": 0, 2024-09-21T03:56:35.037 INFO:teuthology.orchestra.run.smithi039.stdout: "put": 6, 2024-09-21T03:56:35.037 INFO:teuthology.orchestra.run.smithi039.stdout: "put_sum": 1960, 2024-09-21T03:56:35.037 INFO:teuthology.orchestra.run.smithi039.stdout: "wait": { 2024-09-21T03:56:35.037 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2024-09-21T03:56:35.037 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2024-09-21T03:56:35.037 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2024-09-21T03:56:35.038 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-21T03:56:35.038 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:35.038 INFO:teuthology.orchestra.run.smithi039.stdout: "trackedop": { 2024-09-21T03:56:35.038 INFO:teuthology.orchestra.run.smithi039.stdout: "slow_ops_count": 0 2024-09-21T03:56:35.038 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-21T03:56:35.038 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-09-21T03:56:35.584 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:56:35 smithi039 ceph-mon[31211]: pgmap v649: 97 pgs: 97 active+clean; 10 MiB data, 525 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 10 op/s 2024-09-21T03:56:35.718 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:56:35 smithi161 ceph-mon[34533]: pgmap v649: 97 pgs: 97 active+clean; 10 MiB data, 525 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 10 op/s 2024-09-21T03:56:35.724 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:56:35 smithi150 ceph-mon[34289]: pgmap v649: 97 pgs: 97 active+clean; 10 MiB data, 525 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 10 op/s 2024-09-21T03:56:35.975 INFO:tasks.check_counter:Daemon mds.a mds.exported=0 2024-09-21T03:56:35.975 INFO:tasks.check_counter:Daemon mds.a mds.imported=0 2024-09-21T03:56:35.975 INFO:tasks.check_counter:Daemon mds.a mds_cache.dir_update=0 2024-09-21T03:56:35.975 INFO:tasks.check_counter:Daemon mds.a mds_cache.dir_update_receipt=11 2024-09-21T03:56:35.975 INFO:tasks.check_counter:Daemon mds.a mds.root_rsnaps=6 2024-09-21T03:56:35.976 INFO:tasks.check_counter:Daemon mds.a mds_server.req_mksnap_latency.avgcount=0 2024-09-21T03:56:35.976 INFO:tasks.check_counter:Daemon mds.a mds_server.req_rmsnap_latency.avgcount=0 2024-09-21T03:56:35.976 DEBUG:tasks.check_counter:Getting stats from d 2024-09-21T03:56:35.976 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 51116d5c-77ca-11ef-baf5-efdc52797490 -- ceph daemon mds.d perf dump 2024-09-21T03:56:36.238 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/51116d5c-77ca-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:56:36.719 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-09-21T03:56:36.720 INFO:teuthology.orchestra.run.smithi039.stdout: "AsyncMessenger::Worker-0": { 2024-09-21T03:56:36.720 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_recv_messages": 0, 2024-09-21T03:56:36.720 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_send_messages": 2, 2024-09-21T03:56:36.720 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_recv_bytes": 0, 2024-09-21T03:56:36.720 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_send_bytes": 0, 2024-09-21T03:56:36.720 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_created_connections": 4, 2024-09-21T03:56:36.720 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_active_connections": 0, 2024-09-21T03:56:36.720 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_running_total_time": 0.001229596, 2024-09-21T03:56:36.720 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_running_send_time": 0.000000000, 2024-09-21T03:56:36.720 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_running_recv_time": 0.000174920, 2024-09-21T03:56:36.720 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-21T03:56:36.720 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T03:56:36.720 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2024-09-21T03:56:36.721 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2024-09-21T03:56:36.721 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2024-09-21T03:56:36.721 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:36.721 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_handle_ack_lat": { 2024-09-21T03:56:36.721 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2024-09-21T03:56:36.721 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2024-09-21T03:56:36.721 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2024-09-21T03:56:36.721 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:36.721 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_recv_encrypted_bytes": 0, 2024-09-21T03:56:36.721 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_send_encrypted_bytes": 0 2024-09-21T03:56:36.721 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:36.721 INFO:teuthology.orchestra.run.smithi039.stdout: "AsyncMessenger::Worker-1": { 2024-09-21T03:56:36.721 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_recv_messages": 1, 2024-09-21T03:56:36.721 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_send_messages": 158, 2024-09-21T03:56:36.721 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_recv_bytes": 160, 2024-09-21T03:56:36.722 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_send_bytes": 108096, 2024-09-21T03:56:36.722 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_created_connections": 3, 2024-09-21T03:56:36.722 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_active_connections": 2, 2024-09-21T03:56:36.722 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_running_total_time": 0.016450385, 2024-09-21T03:56:36.722 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_running_send_time": 0.013147479, 2024-09-21T03:56:36.722 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_running_recv_time": 0.001175692, 2024-09-21T03:56:36.722 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-21T03:56:36.722 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T03:56:36.722 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 156, 2024-09-21T03:56:36.722 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.008790197, 2024-09-21T03:56:36.722 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000056347 2024-09-21T03:56:36.722 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:36.722 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_handle_ack_lat": { 2024-09-21T03:56:36.722 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2024-09-21T03:56:36.723 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2024-09-21T03:56:36.723 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2024-09-21T03:56:36.723 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:36.723 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_recv_encrypted_bytes": 160, 2024-09-21T03:56:36.723 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_send_encrypted_bytes": 108096 2024-09-21T03:56:36.723 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:36.723 INFO:teuthology.orchestra.run.smithi039.stdout: "AsyncMessenger::Worker-2": { 2024-09-21T03:56:36.723 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_recv_messages": 219, 2024-09-21T03:56:36.723 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_send_messages": 210, 2024-09-21T03:56:36.723 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_recv_bytes": 1148304, 2024-09-21T03:56:36.723 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_send_bytes": 128352, 2024-09-21T03:56:36.723 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_created_connections": 2, 2024-09-21T03:56:36.723 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_active_connections": 1, 2024-09-21T03:56:36.723 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_running_total_time": 0.044296748, 2024-09-21T03:56:36.724 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_running_send_time": 0.018635753, 2024-09-21T03:56:36.724 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_running_recv_time": 0.036529829, 2024-09-21T03:56:36.724 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-21T03:56:36.724 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T03:56:36.724 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 210, 2024-09-21T03:56:36.724 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.015325608, 2024-09-21T03:56:36.724 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000072979 2024-09-21T03:56:36.724 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:36.724 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_handle_ack_lat": { 2024-09-21T03:56:36.724 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2024-09-21T03:56:36.724 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2024-09-21T03:56:36.724 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2024-09-21T03:56:36.724 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:36.724 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_recv_encrypted_bytes": 1148304, 2024-09-21T03:56:36.724 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_send_encrypted_bytes": 128352 2024-09-21T03:56:36.725 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:36.725 INFO:teuthology.orchestra.run.smithi039.stdout: "cct": { 2024-09-21T03:56:36.725 INFO:teuthology.orchestra.run.smithi039.stdout: "total_workers": 0, 2024-09-21T03:56:36.725 INFO:teuthology.orchestra.run.smithi039.stdout: "unhealthy_workers": 0 2024-09-21T03:56:36.725 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:36.725 INFO:teuthology.orchestra.run.smithi039.stdout: "mempool": { 2024-09-21T03:56:36.725 INFO:teuthology.orchestra.run.smithi039.stdout: "bloom_filter_bytes": 0, 2024-09-21T03:56:36.725 INFO:teuthology.orchestra.run.smithi039.stdout: "bloom_filter_items": 0, 2024-09-21T03:56:36.725 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_alloc_bytes": 0, 2024-09-21T03:56:36.725 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_alloc_items": 0, 2024-09-21T03:56:36.725 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_cache_data_bytes": 0, 2024-09-21T03:56:36.725 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_cache_data_items": 0, 2024-09-21T03:56:36.725 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-21T03:56:36.725 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_cache_onode_items": 0, 2024-09-21T03:56:36.725 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-21T03:56:36.726 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_cache_meta_items": 0, 2024-09-21T03:56:36.726 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_cache_other_bytes": 0, 2024-09-21T03:56:36.726 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_cache_other_items": 0, 2024-09-21T03:56:36.726 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-21T03:56:36.726 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_cache_buffer_items": 0, 2024-09-21T03:56:36.726 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_extent_bytes": 0, 2024-09-21T03:56:36.726 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_extent_items": 0, 2024-09-21T03:56:36.726 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_blob_bytes": 0, 2024-09-21T03:56:36.726 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_blob_items": 0, 2024-09-21T03:56:36.726 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-21T03:56:36.726 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_shared_blob_items": 0, 2024-09-21T03:56:36.726 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-21T03:56:36.726 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_inline_bl_items": 0, 2024-09-21T03:56:36.726 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_fsck_bytes": 0, 2024-09-21T03:56:36.726 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_fsck_items": 0, 2024-09-21T03:56:36.727 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_txc_bytes": 0, 2024-09-21T03:56:36.727 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_txc_items": 0, 2024-09-21T03:56:36.727 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-21T03:56:36.727 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_writing_deferred_items": 0, 2024-09-21T03:56:36.727 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_writing_bytes": 0, 2024-09-21T03:56:36.727 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_writing_items": 0, 2024-09-21T03:56:36.727 INFO:teuthology.orchestra.run.smithi039.stdout: "bluefs_bytes": 0, 2024-09-21T03:56:36.727 INFO:teuthology.orchestra.run.smithi039.stdout: "bluefs_items": 0, 2024-09-21T03:56:36.727 INFO:teuthology.orchestra.run.smithi039.stdout: "bluefs_file_reader_bytes": 0, 2024-09-21T03:56:36.727 INFO:teuthology.orchestra.run.smithi039.stdout: "bluefs_file_reader_items": 0, 2024-09-21T03:56:36.727 INFO:teuthology.orchestra.run.smithi039.stdout: "bluefs_file_writer_bytes": 0, 2024-09-21T03:56:36.727 INFO:teuthology.orchestra.run.smithi039.stdout: "bluefs_file_writer_items": 0, 2024-09-21T03:56:36.727 INFO:teuthology.orchestra.run.smithi039.stdout: "buffer_anon_bytes": 148945, 2024-09-21T03:56:36.727 INFO:teuthology.orchestra.run.smithi039.stdout: "buffer_anon_items": 29, 2024-09-21T03:56:36.728 INFO:teuthology.orchestra.run.smithi039.stdout: "buffer_meta_bytes": 0, 2024-09-21T03:56:36.728 INFO:teuthology.orchestra.run.smithi039.stdout: "buffer_meta_items": 0, 2024-09-21T03:56:36.728 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_bytes": 0, 2024-09-21T03:56:36.728 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_items": 0, 2024-09-21T03:56:36.728 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_mapbl_bytes": 0, 2024-09-21T03:56:36.728 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_mapbl_items": 0, 2024-09-21T03:56:36.728 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_pglog_bytes": 0, 2024-09-21T03:56:36.728 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_pglog_items": 0, 2024-09-21T03:56:36.728 INFO:teuthology.orchestra.run.smithi039.stdout: "osdmap_bytes": 0, 2024-09-21T03:56:36.728 INFO:teuthology.orchestra.run.smithi039.stdout: "osdmap_items": 0, 2024-09-21T03:56:36.728 INFO:teuthology.orchestra.run.smithi039.stdout: "osdmap_mapping_bytes": 0, 2024-09-21T03:56:36.728 INFO:teuthology.orchestra.run.smithi039.stdout: "osdmap_mapping_items": 0, 2024-09-21T03:56:36.728 INFO:teuthology.orchestra.run.smithi039.stdout: "pgmap_bytes": 0, 2024-09-21T03:56:36.728 INFO:teuthology.orchestra.run.smithi039.stdout: "pgmap_items": 0, 2024-09-21T03:56:36.728 INFO:teuthology.orchestra.run.smithi039.stdout: "mds_co_bytes": 968, 2024-09-21T03:56:36.729 INFO:teuthology.orchestra.run.smithi039.stdout: "mds_co_items": 2, 2024-09-21T03:56:36.729 INFO:teuthology.orchestra.run.smithi039.stdout: "unittest_1_bytes": 0, 2024-09-21T03:56:36.729 INFO:teuthology.orchestra.run.smithi039.stdout: "unittest_1_items": 0, 2024-09-21T03:56:36.729 INFO:teuthology.orchestra.run.smithi039.stdout: "unittest_2_bytes": 0, 2024-09-21T03:56:36.729 INFO:teuthology.orchestra.run.smithi039.stdout: "unittest_2_items": 0 2024-09-21T03:56:36.729 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:36.729 INFO:teuthology.orchestra.run.smithi039.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-21T03:56:36.729 INFO:teuthology.orchestra.run.smithi039.stdout: "val": 0, 2024-09-21T03:56:36.729 INFO:teuthology.orchestra.run.smithi039.stdout: "max": 104857600, 2024-09-21T03:56:36.729 INFO:teuthology.orchestra.run.smithi039.stdout: "get_started": 0, 2024-09-21T03:56:36.729 INFO:teuthology.orchestra.run.smithi039.stdout: "get": 216, 2024-09-21T03:56:36.729 INFO:teuthology.orchestra.run.smithi039.stdout: "get_sum": 1114877, 2024-09-21T03:56:36.729 INFO:teuthology.orchestra.run.smithi039.stdout: "get_or_fail_fail": 0, 2024-09-21T03:56:36.729 INFO:teuthology.orchestra.run.smithi039.stdout: "get_or_fail_success": 216, 2024-09-21T03:56:36.729 INFO:teuthology.orchestra.run.smithi039.stdout: "take": 0, 2024-09-21T03:56:36.730 INFO:teuthology.orchestra.run.smithi039.stdout: "take_sum": 0, 2024-09-21T03:56:36.730 INFO:teuthology.orchestra.run.smithi039.stdout: "put": 216, 2024-09-21T03:56:36.730 INFO:teuthology.orchestra.run.smithi039.stdout: "put_sum": 1114877, 2024-09-21T03:56:36.730 INFO:teuthology.orchestra.run.smithi039.stdout: "wait": { 2024-09-21T03:56:36.730 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2024-09-21T03:56:36.730 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2024-09-21T03:56:36.730 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2024-09-21T03:56:36.730 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-21T03:56:36.730 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-21T03:56:36.730 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-09-21T03:56:37.492 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.d 2024-09-21T03:56:37.493 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.d 2024-09-21T03:56:37.493 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.d 2024-09-21T03:56:37.493 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.d 2024-09-21T03:56:37.493 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.d 2024-09-21T03:56:37.493 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.d 2024-09-21T03:56:37.493 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.d 2024-09-21T03:56:37.493 DEBUG:tasks.check_counter:Getting stats from g 2024-09-21T03:56:37.493 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 51116d5c-77ca-11ef-baf5-efdc52797490 -- ceph daemon mds.g perf dump 2024-09-21T03:56:37.718 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:56:37 smithi161 ceph-mon[34533]: pgmap v650: 97 pgs: 97 active+clean; 10 MiB data, 525 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 10 op/s 2024-09-21T03:56:37.725 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:56:37 smithi150 ceph-mon[34289]: pgmap v650: 97 pgs: 97 active+clean; 10 MiB data, 525 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 10 op/s 2024-09-21T03:56:37.755 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/51116d5c-77ca-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:56:37.834 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:56:37 smithi039 ceph-mon[31211]: pgmap v650: 97 pgs: 97 active+clean; 10 MiB data, 525 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 10 op/s 2024-09-21T03:56:38.244 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-09-21T03:56:38.244 INFO:teuthology.orchestra.run.smithi039.stdout: "AsyncMessenger::Worker-0": { 2024-09-21T03:56:38.244 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_recv_messages": 736, 2024-09-21T03:56:38.244 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_send_messages": 735, 2024-09-21T03:56:38.244 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_recv_bytes": 249019, 2024-09-21T03:56:38.245 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_send_bytes": 232907, 2024-09-21T03:56:38.245 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_created_connections": 6, 2024-09-21T03:56:38.245 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_active_connections": 2, 2024-09-21T03:56:38.245 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_running_total_time": 0.106366499, 2024-09-21T03:56:38.245 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_running_send_time": 0.053347458, 2024-09-21T03:56:38.245 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_running_recv_time": 0.021325793, 2024-09-21T03:56:38.245 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_running_fast_dispatch_time": 0.022568517, 2024-09-21T03:56:38.245 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T03:56:38.245 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 734, 2024-09-21T03:56:38.245 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.045900494, 2024-09-21T03:56:38.245 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000062534 2024-09-21T03:56:38.245 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:38.245 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_handle_ack_lat": { 2024-09-21T03:56:38.245 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2024-09-21T03:56:38.246 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2024-09-21T03:56:38.246 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2024-09-21T03:56:38.246 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:38.246 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_recv_encrypted_bytes": 3040, 2024-09-21T03:56:38.246 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_send_encrypted_bytes": 448 2024-09-21T03:56:38.246 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:38.246 INFO:teuthology.orchestra.run.smithi039.stdout: "AsyncMessenger::Worker-1": { 2024-09-21T03:56:38.246 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_recv_messages": 1378, 2024-09-21T03:56:38.246 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_send_messages": 1532, 2024-09-21T03:56:38.246 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_recv_bytes": 352606, 2024-09-21T03:56:38.246 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_send_bytes": 795767, 2024-09-21T03:56:38.246 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_created_connections": 5, 2024-09-21T03:56:38.246 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_active_connections": 4, 2024-09-21T03:56:38.246 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_running_total_time": 0.122112993, 2024-09-21T03:56:38.247 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_running_send_time": 0.054027204, 2024-09-21T03:56:38.247 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_running_recv_time": 0.025247282, 2024-09-21T03:56:38.247 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_running_fast_dispatch_time": 0.034499369, 2024-09-21T03:56:38.247 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T03:56:38.247 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 1530, 2024-09-21T03:56:38.247 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.060925717, 2024-09-21T03:56:38.247 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000039820 2024-09-21T03:56:38.247 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:38.247 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_handle_ack_lat": { 2024-09-21T03:56:38.247 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2024-09-21T03:56:38.247 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2024-09-21T03:56:38.247 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2024-09-21T03:56:38.247 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:38.247 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_recv_encrypted_bytes": 160, 2024-09-21T03:56:38.247 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_send_encrypted_bytes": 357824 2024-09-21T03:56:38.248 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:38.248 INFO:teuthology.orchestra.run.smithi039.stdout: "AsyncMessenger::Worker-2": { 2024-09-21T03:56:38.248 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_recv_messages": 231, 2024-09-21T03:56:38.248 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_send_messages": 222, 2024-09-21T03:56:38.248 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_recv_bytes": 1181044, 2024-09-21T03:56:38.248 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_send_bytes": 128901, 2024-09-21T03:56:38.248 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_created_connections": 4, 2024-09-21T03:56:38.248 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_active_connections": 3, 2024-09-21T03:56:38.248 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_running_total_time": 0.047680016, 2024-09-21T03:56:38.248 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_running_send_time": 0.020051795, 2024-09-21T03:56:38.248 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_running_recv_time": 0.039254194, 2024-09-21T03:56:38.248 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_running_fast_dispatch_time": 0.000065652, 2024-09-21T03:56:38.248 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T03:56:38.248 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 221, 2024-09-21T03:56:38.249 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.027220081, 2024-09-21T03:56:38.249 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000123167 2024-09-21T03:56:38.249 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:38.249 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_handle_ack_lat": { 2024-09-21T03:56:38.249 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2024-09-21T03:56:38.249 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2024-09-21T03:56:38.249 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2024-09-21T03:56:38.249 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:38.249 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_recv_encrypted_bytes": 1179696, 2024-09-21T03:56:38.249 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_send_encrypted_bytes": 127936 2024-09-21T03:56:38.249 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:38.249 INFO:teuthology.orchestra.run.smithi039.stdout: "cct": { 2024-09-21T03:56:38.249 INFO:teuthology.orchestra.run.smithi039.stdout: "total_workers": 1, 2024-09-21T03:56:38.249 INFO:teuthology.orchestra.run.smithi039.stdout: "unhealthy_workers": 0 2024-09-21T03:56:38.249 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:38.250 INFO:teuthology.orchestra.run.smithi039.stdout: "finisher-MDSRank": { 2024-09-21T03:56:38.250 INFO:teuthology.orchestra.run.smithi039.stdout: "queue_len": 0, 2024-09-21T03:56:38.250 INFO:teuthology.orchestra.run.smithi039.stdout: "complete_latency": { 2024-09-21T03:56:38.250 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 3562, 2024-09-21T03:56:38.250 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.096803668, 2024-09-21T03:56:38.250 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000027176 2024-09-21T03:56:38.250 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-21T03:56:38.250 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:38.250 INFO:teuthology.orchestra.run.smithi039.stdout: "finisher-PurgeQueue": { 2024-09-21T03:56:38.250 INFO:teuthology.orchestra.run.smithi039.stdout: "queue_len": 0, 2024-09-21T03:56:38.250 INFO:teuthology.orchestra.run.smithi039.stdout: "complete_latency": { 2024-09-21T03:56:38.250 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2024-09-21T03:56:38.250 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2024-09-21T03:56:38.250 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2024-09-21T03:56:38.250 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-21T03:56:38.251 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:38.251 INFO:teuthology.orchestra.run.smithi039.stdout: "mds": { 2024-09-21T03:56:38.251 INFO:teuthology.orchestra.run.smithi039.stdout: "request": 0, 2024-09-21T03:56:38.251 INFO:teuthology.orchestra.run.smithi039.stdout: "reply": 0, 2024-09-21T03:56:38.251 INFO:teuthology.orchestra.run.smithi039.stdout: "reply_latency": { 2024-09-21T03:56:38.251 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2024-09-21T03:56:38.251 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2024-09-21T03:56:38.251 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2024-09-21T03:56:38.251 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:38.251 INFO:teuthology.orchestra.run.smithi039.stdout: "slow_reply": 0, 2024-09-21T03:56:38.251 INFO:teuthology.orchestra.run.smithi039.stdout: "forward": 0, 2024-09-21T03:56:38.251 INFO:teuthology.orchestra.run.smithi039.stdout: "dir_fetch_complete": 0, 2024-09-21T03:56:38.251 INFO:teuthology.orchestra.run.smithi039.stdout: "dir_fetch_keys": 0, 2024-09-21T03:56:38.251 INFO:teuthology.orchestra.run.smithi039.stdout: "dir_commit": 0, 2024-09-21T03:56:38.252 INFO:teuthology.orchestra.run.smithi039.stdout: "dir_split": 0, 2024-09-21T03:56:38.252 INFO:teuthology.orchestra.run.smithi039.stdout: "dir_merge": 0, 2024-09-21T03:56:38.252 INFO:teuthology.orchestra.run.smithi039.stdout: "inodes": 0, 2024-09-21T03:56:38.252 INFO:teuthology.orchestra.run.smithi039.stdout: "inodes_top": 0, 2024-09-21T03:56:38.252 INFO:teuthology.orchestra.run.smithi039.stdout: "inodes_bottom": 0, 2024-09-21T03:56:38.252 INFO:teuthology.orchestra.run.smithi039.stdout: "inodes_pin_tail": 0, 2024-09-21T03:56:38.252 INFO:teuthology.orchestra.run.smithi039.stdout: "inodes_pinned": 0, 2024-09-21T03:56:38.252 INFO:teuthology.orchestra.run.smithi039.stdout: "inodes_expired": 0, 2024-09-21T03:56:38.252 INFO:teuthology.orchestra.run.smithi039.stdout: "inodes_with_caps": 0, 2024-09-21T03:56:38.252 INFO:teuthology.orchestra.run.smithi039.stdout: "caps": 0, 2024-09-21T03:56:38.252 INFO:teuthology.orchestra.run.smithi039.stdout: "subtrees": 1, 2024-09-21T03:56:38.252 INFO:teuthology.orchestra.run.smithi039.stdout: "traverse": 0, 2024-09-21T03:56:38.252 INFO:teuthology.orchestra.run.smithi039.stdout: "traverse_hit": 0, 2024-09-21T03:56:38.252 INFO:teuthology.orchestra.run.smithi039.stdout: "traverse_forward": 0, 2024-09-21T03:56:38.253 INFO:teuthology.orchestra.run.smithi039.stdout: "traverse_discover": 0, 2024-09-21T03:56:38.253 INFO:teuthology.orchestra.run.smithi039.stdout: "traverse_dir_fetch": 0, 2024-09-21T03:56:38.253 INFO:teuthology.orchestra.run.smithi039.stdout: "traverse_remote_ino": 0, 2024-09-21T03:56:38.253 INFO:teuthology.orchestra.run.smithi039.stdout: "traverse_lock": 0, 2024-09-21T03:56:38.253 INFO:teuthology.orchestra.run.smithi039.stdout: "load_cent": 0, 2024-09-21T03:56:38.253 INFO:teuthology.orchestra.run.smithi039.stdout: "q": 0, 2024-09-21T03:56:38.253 INFO:teuthology.orchestra.run.smithi039.stdout: "exported": 0, 2024-09-21T03:56:38.253 INFO:teuthology.orchestra.run.smithi039.stdout: "exported_inodes": 0, 2024-09-21T03:56:38.253 INFO:teuthology.orchestra.run.smithi039.stdout: "imported": 0, 2024-09-21T03:56:38.253 INFO:teuthology.orchestra.run.smithi039.stdout: "imported_inodes": 0, 2024-09-21T03:56:38.253 INFO:teuthology.orchestra.run.smithi039.stdout: "openino_dir_fetch": 0, 2024-09-21T03:56:38.253 INFO:teuthology.orchestra.run.smithi039.stdout: "openino_backtrace_fetch": 0, 2024-09-21T03:56:38.253 INFO:teuthology.orchestra.run.smithi039.stdout: "openino_peer_discover": 0, 2024-09-21T03:56:38.253 INFO:teuthology.orchestra.run.smithi039.stdout: "root_rfiles": 0, 2024-09-21T03:56:38.253 INFO:teuthology.orchestra.run.smithi039.stdout: "root_rbytes": 0, 2024-09-21T03:56:38.254 INFO:teuthology.orchestra.run.smithi039.stdout: "root_rsnaps": 0, 2024-09-21T03:56:38.254 INFO:teuthology.orchestra.run.smithi039.stdout: "scrub_backtrace_fetch": 0, 2024-09-21T03:56:38.254 INFO:teuthology.orchestra.run.smithi039.stdout: "scrub_set_tag": 0, 2024-09-21T03:56:38.254 INFO:teuthology.orchestra.run.smithi039.stdout: "scrub_backtrace_repaired": 0, 2024-09-21T03:56:38.254 INFO:teuthology.orchestra.run.smithi039.stdout: "scrub_inotable_repaired": 0, 2024-09-21T03:56:38.254 INFO:teuthology.orchestra.run.smithi039.stdout: "scrub_dir_inodes": 0, 2024-09-21T03:56:38.254 INFO:teuthology.orchestra.run.smithi039.stdout: "scrub_dir_base_inodes": 0, 2024-09-21T03:56:38.254 INFO:teuthology.orchestra.run.smithi039.stdout: "scrub_dirfrag_rstats": 0, 2024-09-21T03:56:38.254 INFO:teuthology.orchestra.run.smithi039.stdout: "scrub_file_inodes": 0, 2024-09-21T03:56:38.254 INFO:teuthology.orchestra.run.smithi039.stdout: "handle_inode_file_caps": 0, 2024-09-21T03:56:38.254 INFO:teuthology.orchestra.run.smithi039.stdout: "ceph_cap_op_revoke": 0, 2024-09-21T03:56:38.254 INFO:teuthology.orchestra.run.smithi039.stdout: "ceph_cap_op_grant": 0, 2024-09-21T03:56:38.254 INFO:teuthology.orchestra.run.smithi039.stdout: "ceph_cap_op_trunc": 0, 2024-09-21T03:56:38.255 INFO:teuthology.orchestra.run.smithi039.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-21T03:56:38.255 INFO:teuthology.orchestra.run.smithi039.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-21T03:56:38.255 INFO:teuthology.orchestra.run.smithi039.stdout: "handle_client_caps": 0, 2024-09-21T03:56:38.255 INFO:teuthology.orchestra.run.smithi039.stdout: "handle_client_caps_dirty": 0, 2024-09-21T03:56:38.255 INFO:teuthology.orchestra.run.smithi039.stdout: "handle_client_cap_release": 0, 2024-09-21T03:56:38.255 INFO:teuthology.orchestra.run.smithi039.stdout: "process_request_cap_release": 0 2024-09-21T03:56:38.255 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:38.255 INFO:teuthology.orchestra.run.smithi039.stdout: "mds_cache": { 2024-09-21T03:56:38.255 INFO:teuthology.orchestra.run.smithi039.stdout: "dir_update": 0, 2024-09-21T03:56:38.255 INFO:teuthology.orchestra.run.smithi039.stdout: "dir_update_receipt": 0, 2024-09-21T03:56:38.255 INFO:teuthology.orchestra.run.smithi039.stdout: "dir_try_discover": 0, 2024-09-21T03:56:38.255 INFO:teuthology.orchestra.run.smithi039.stdout: "dir_send_discover": 0, 2024-09-21T03:56:38.255 INFO:teuthology.orchestra.run.smithi039.stdout: "dir_handle_discover": 0, 2024-09-21T03:56:38.255 INFO:teuthology.orchestra.run.smithi039.stdout: "num_strays": 0, 2024-09-21T03:56:38.255 INFO:teuthology.orchestra.run.smithi039.stdout: "num_strays_delayed": 0, 2024-09-21T03:56:38.256 INFO:teuthology.orchestra.run.smithi039.stdout: "num_strays_enqueuing": 0, 2024-09-21T03:56:38.256 INFO:teuthology.orchestra.run.smithi039.stdout: "strays_created": 0, 2024-09-21T03:56:38.256 INFO:teuthology.orchestra.run.smithi039.stdout: "strays_enqueued": 0, 2024-09-21T03:56:38.256 INFO:teuthology.orchestra.run.smithi039.stdout: "strays_reintegrated": 0, 2024-09-21T03:56:38.256 INFO:teuthology.orchestra.run.smithi039.stdout: "strays_migrated": 0, 2024-09-21T03:56:38.256 INFO:teuthology.orchestra.run.smithi039.stdout: "num_recovering_processing": 0, 2024-09-21T03:56:38.256 INFO:teuthology.orchestra.run.smithi039.stdout: "num_recovering_enqueued": 0, 2024-09-21T03:56:38.256 INFO:teuthology.orchestra.run.smithi039.stdout: "num_recovering_prioritized": 0, 2024-09-21T03:56:38.256 INFO:teuthology.orchestra.run.smithi039.stdout: "recovery_started": 0, 2024-09-21T03:56:38.256 INFO:teuthology.orchestra.run.smithi039.stdout: "recovery_completed": 0, 2024-09-21T03:56:38.256 INFO:teuthology.orchestra.run.smithi039.stdout: "ireq_quiesce_path": 0, 2024-09-21T03:56:38.256 INFO:teuthology.orchestra.run.smithi039.stdout: "ireq_quiesce_inode": 0, 2024-09-21T03:56:38.256 INFO:teuthology.orchestra.run.smithi039.stdout: "ireq_enqueue_scrub": 0, 2024-09-21T03:56:38.256 INFO:teuthology.orchestra.run.smithi039.stdout: "ireq_exportdir": 0, 2024-09-21T03:56:38.257 INFO:teuthology.orchestra.run.smithi039.stdout: "ireq_flush": 0, 2024-09-21T03:56:38.257 INFO:teuthology.orchestra.run.smithi039.stdout: "ireq_fragmentdir": 0, 2024-09-21T03:56:38.257 INFO:teuthology.orchestra.run.smithi039.stdout: "ireq_fragstats": 0, 2024-09-21T03:56:38.257 INFO:teuthology.orchestra.run.smithi039.stdout: "ireq_inodestats": 0, 2024-09-21T03:56:38.257 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_started": 0, 2024-09-21T03:56:38.257 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_succeeded": 0, 2024-09-21T03:56:38.257 INFO:teuthology.orchestra.run.smithi039.stdout: "uninline_write_failed": 0 2024-09-21T03:56:38.257 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:38.257 INFO:teuthology.orchestra.run.smithi039.stdout: "mds_log": { 2024-09-21T03:56:38.257 INFO:teuthology.orchestra.run.smithi039.stdout: "evlrg": 0, 2024-09-21T03:56:38.257 INFO:teuthology.orchestra.run.smithi039.stdout: "evadd": 0, 2024-09-21T03:56:38.257 INFO:teuthology.orchestra.run.smithi039.stdout: "evex": 0, 2024-09-21T03:56:38.257 INFO:teuthology.orchestra.run.smithi039.stdout: "evtrm": 0, 2024-09-21T03:56:38.257 INFO:teuthology.orchestra.run.smithi039.stdout: "ev": 7, 2024-09-21T03:56:38.257 INFO:teuthology.orchestra.run.smithi039.stdout: "evexg": 0, 2024-09-21T03:56:38.258 INFO:teuthology.orchestra.run.smithi039.stdout: "evexd": 0, 2024-09-21T03:56:38.258 INFO:teuthology.orchestra.run.smithi039.stdout: "segadd": 0, 2024-09-21T03:56:38.258 INFO:teuthology.orchestra.run.smithi039.stdout: "segex": 0, 2024-09-21T03:56:38.258 INFO:teuthology.orchestra.run.smithi039.stdout: "segtrm": 0, 2024-09-21T03:56:38.258 INFO:teuthology.orchestra.run.smithi039.stdout: "seg": 2, 2024-09-21T03:56:38.258 INFO:teuthology.orchestra.run.smithi039.stdout: "segmjr": 2, 2024-09-21T03:56:38.258 INFO:teuthology.orchestra.run.smithi039.stdout: "segexg": 0, 2024-09-21T03:56:38.258 INFO:teuthology.orchestra.run.smithi039.stdout: "segexd": 0, 2024-09-21T03:56:38.258 INFO:teuthology.orchestra.run.smithi039.stdout: "expos": 4194304, 2024-09-21T03:56:38.258 INFO:teuthology.orchestra.run.smithi039.stdout: "wrpos": 4196264, 2024-09-21T03:56:38.258 INFO:teuthology.orchestra.run.smithi039.stdout: "rdpos": 4196123, 2024-09-21T03:56:38.258 INFO:teuthology.orchestra.run.smithi039.stdout: "jlat": { 2024-09-21T03:56:38.258 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2024-09-21T03:56:38.258 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2024-09-21T03:56:38.258 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2024-09-21T03:56:38.259 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:38.259 INFO:teuthology.orchestra.run.smithi039.stdout: "replayed": 7 2024-09-21T03:56:38.259 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:38.259 INFO:teuthology.orchestra.run.smithi039.stdout: "mds_mem": { 2024-09-21T03:56:38.259 INFO:teuthology.orchestra.run.smithi039.stdout: "ino": 3, 2024-09-21T03:56:38.259 INFO:teuthology.orchestra.run.smithi039.stdout: "ino+": 3, 2024-09-21T03:56:38.259 INFO:teuthology.orchestra.run.smithi039.stdout: "ino-": 0, 2024-09-21T03:56:38.259 INFO:teuthology.orchestra.run.smithi039.stdout: "dir": 1, 2024-09-21T03:56:38.259 INFO:teuthology.orchestra.run.smithi039.stdout: "dir+": 1, 2024-09-21T03:56:38.259 INFO:teuthology.orchestra.run.smithi039.stdout: "dir-": 0, 2024-09-21T03:56:38.259 INFO:teuthology.orchestra.run.smithi039.stdout: "dn": 0, 2024-09-21T03:56:38.259 INFO:teuthology.orchestra.run.smithi039.stdout: "dn+": 0, 2024-09-21T03:56:38.259 INFO:teuthology.orchestra.run.smithi039.stdout: "dn-": 0, 2024-09-21T03:56:38.259 INFO:teuthology.orchestra.run.smithi039.stdout: "cap": 0, 2024-09-21T03:56:38.259 INFO:teuthology.orchestra.run.smithi039.stdout: "cap+": 0, 2024-09-21T03:56:38.260 INFO:teuthology.orchestra.run.smithi039.stdout: "cap-": 0, 2024-09-21T03:56:38.260 INFO:teuthology.orchestra.run.smithi039.stdout: "rss": 119236, 2024-09-21T03:56:38.260 INFO:teuthology.orchestra.run.smithi039.stdout: "heap": 316428288 2024-09-21T03:56:38.260 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:38.260 INFO:teuthology.orchestra.run.smithi039.stdout: "mds_server": { 2024-09-21T03:56:38.260 INFO:teuthology.orchestra.run.smithi039.stdout: "dispatch_client_request": 0, 2024-09-21T03:56:38.260 INFO:teuthology.orchestra.run.smithi039.stdout: "dispatch_server_request": 0, 2024-09-21T03:56:38.260 INFO:teuthology.orchestra.run.smithi039.stdout: "handle_client_request": 0, 2024-09-21T03:56:38.260 INFO:teuthology.orchestra.run.smithi039.stdout: "handle_client_session": 0, 2024-09-21T03:56:38.260 INFO:teuthology.orchestra.run.smithi039.stdout: "handle_peer_request": 0, 2024-09-21T03:56:38.260 INFO:teuthology.orchestra.run.smithi039.stdout: "req_create_latency": { 2024-09-21T03:56:38.260 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2024-09-21T03:56:38.260 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2024-09-21T03:56:38.260 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2024-09-21T03:56:38.261 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:38.261 INFO:teuthology.orchestra.run.smithi039.stdout: "req_getattr_latency": { 2024-09-21T03:56:38.261 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2024-09-21T03:56:38.261 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2024-09-21T03:56:38.261 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2024-09-21T03:56:38.261 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:38.261 INFO:teuthology.orchestra.run.smithi039.stdout: "req_getfilelock_latency": { 2024-09-21T03:56:38.261 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2024-09-21T03:56:38.261 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2024-09-21T03:56:38.261 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2024-09-21T03:56:38.261 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:38.261 INFO:teuthology.orchestra.run.smithi039.stdout: "req_link_latency": { 2024-09-21T03:56:38.261 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2024-09-21T03:56:38.261 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2024-09-21T03:56:38.261 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2024-09-21T03:56:38.261 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:38.262 INFO:teuthology.orchestra.run.smithi039.stdout: "req_lookup_latency": { 2024-09-21T03:56:38.262 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2024-09-21T03:56:38.262 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2024-09-21T03:56:38.262 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2024-09-21T03:56:38.262 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:38.262 INFO:teuthology.orchestra.run.smithi039.stdout: "req_lookuphash_latency": { 2024-09-21T03:56:38.262 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2024-09-21T03:56:38.262 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2024-09-21T03:56:38.262 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2024-09-21T03:56:38.262 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:38.262 INFO:teuthology.orchestra.run.smithi039.stdout: "req_lookupino_latency": { 2024-09-21T03:56:38.262 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2024-09-21T03:56:38.262 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2024-09-21T03:56:38.262 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2024-09-21T03:56:38.262 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:38.263 INFO:teuthology.orchestra.run.smithi039.stdout: "req_lookupname_latency": { 2024-09-21T03:56:38.263 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2024-09-21T03:56:38.263 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2024-09-21T03:56:38.263 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2024-09-21T03:56:38.263 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:38.263 INFO:teuthology.orchestra.run.smithi039.stdout: "req_lookupparent_latency": { 2024-09-21T03:56:38.263 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2024-09-21T03:56:38.263 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2024-09-21T03:56:38.263 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2024-09-21T03:56:38.263 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:38.263 INFO:teuthology.orchestra.run.smithi039.stdout: "req_lookupsnap_latency": { 2024-09-21T03:56:38.263 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2024-09-21T03:56:38.263 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2024-09-21T03:56:38.263 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2024-09-21T03:56:38.264 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:38.264 INFO:teuthology.orchestra.run.smithi039.stdout: "req_lssnap_latency": { 2024-09-21T03:56:38.264 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2024-09-21T03:56:38.264 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2024-09-21T03:56:38.264 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2024-09-21T03:56:38.264 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:38.264 INFO:teuthology.orchestra.run.smithi039.stdout: "req_mkdir_latency": { 2024-09-21T03:56:38.264 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2024-09-21T03:56:38.265 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2024-09-21T03:56:38.265 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2024-09-21T03:56:38.265 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:38.265 INFO:teuthology.orchestra.run.smithi039.stdout: "req_mknod_latency": { 2024-09-21T03:56:38.265 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2024-09-21T03:56:38.265 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2024-09-21T03:56:38.265 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2024-09-21T03:56:38.265 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:38.265 INFO:teuthology.orchestra.run.smithi039.stdout: "req_mksnap_latency": { 2024-09-21T03:56:38.265 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2024-09-21T03:56:38.265 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2024-09-21T03:56:38.265 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2024-09-21T03:56:38.265 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:38.265 INFO:teuthology.orchestra.run.smithi039.stdout: "req_open_latency": { 2024-09-21T03:56:38.265 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2024-09-21T03:56:38.266 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2024-09-21T03:56:38.266 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2024-09-21T03:56:38.266 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:38.266 INFO:teuthology.orchestra.run.smithi039.stdout: "req_readdir_latency": { 2024-09-21T03:56:38.266 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2024-09-21T03:56:38.266 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2024-09-21T03:56:38.266 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2024-09-21T03:56:38.266 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:38.266 INFO:teuthology.orchestra.run.smithi039.stdout: "req_rename_latency": { 2024-09-21T03:56:38.266 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2024-09-21T03:56:38.266 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2024-09-21T03:56:38.266 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2024-09-21T03:56:38.266 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:38.267 INFO:teuthology.orchestra.run.smithi039.stdout: "req_renamesnap_latency": { 2024-09-21T03:56:38.267 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2024-09-21T03:56:38.267 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2024-09-21T03:56:38.267 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2024-09-21T03:56:38.267 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:38.267 INFO:teuthology.orchestra.run.smithi039.stdout: "req_snapdiff_latency": { 2024-09-21T03:56:38.267 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2024-09-21T03:56:38.267 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2024-09-21T03:56:38.267 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2024-09-21T03:56:38.267 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:38.267 INFO:teuthology.orchestra.run.smithi039.stdout: "req_rmdir_latency": { 2024-09-21T03:56:38.267 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2024-09-21T03:56:38.267 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2024-09-21T03:56:38.267 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2024-09-21T03:56:38.268 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:38.268 INFO:teuthology.orchestra.run.smithi039.stdout: "req_rmsnap_latency": { 2024-09-21T03:56:38.268 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2024-09-21T03:56:38.268 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2024-09-21T03:56:38.268 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2024-09-21T03:56:38.268 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:38.268 INFO:teuthology.orchestra.run.smithi039.stdout: "req_rmxattr_latency": { 2024-09-21T03:56:38.268 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2024-09-21T03:56:38.268 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2024-09-21T03:56:38.268 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2024-09-21T03:56:38.268 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:38.268 INFO:teuthology.orchestra.run.smithi039.stdout: "req_setattr_latency": { 2024-09-21T03:56:38.268 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2024-09-21T03:56:38.268 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2024-09-21T03:56:38.268 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2024-09-21T03:56:38.269 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:38.269 INFO:teuthology.orchestra.run.smithi039.stdout: "req_setdirlayout_latency": { 2024-09-21T03:56:38.269 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2024-09-21T03:56:38.269 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2024-09-21T03:56:38.269 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2024-09-21T03:56:38.269 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:38.269 INFO:teuthology.orchestra.run.smithi039.stdout: "req_setfilelock_latency": { 2024-09-21T03:56:38.269 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2024-09-21T03:56:38.269 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2024-09-21T03:56:38.269 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2024-09-21T03:56:38.269 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:38.269 INFO:teuthology.orchestra.run.smithi039.stdout: "req_setlayout_latency": { 2024-09-21T03:56:38.269 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2024-09-21T03:56:38.269 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2024-09-21T03:56:38.270 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2024-09-21T03:56:38.270 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:38.270 INFO:teuthology.orchestra.run.smithi039.stdout: "req_setxattr_latency": { 2024-09-21T03:56:38.270 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2024-09-21T03:56:38.270 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2024-09-21T03:56:38.270 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2024-09-21T03:56:38.270 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:38.270 INFO:teuthology.orchestra.run.smithi039.stdout: "req_symlink_latency": { 2024-09-21T03:56:38.270 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2024-09-21T03:56:38.270 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2024-09-21T03:56:38.270 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2024-09-21T03:56:38.270 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:38.270 INFO:teuthology.orchestra.run.smithi039.stdout: "req_unlink_latency": { 2024-09-21T03:56:38.270 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2024-09-21T03:56:38.271 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2024-09-21T03:56:38.271 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2024-09-21T03:56:38.271 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:38.271 INFO:teuthology.orchestra.run.smithi039.stdout: "cap_revoke_eviction": 0, 2024-09-21T03:56:38.271 INFO:teuthology.orchestra.run.smithi039.stdout: "cap_acquisition_throttle": 0, 2024-09-21T03:56:38.271 INFO:teuthology.orchestra.run.smithi039.stdout: "req_getvxattr_latency": { 2024-09-21T03:56:38.271 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2024-09-21T03:56:38.271 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2024-09-21T03:56:38.271 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2024-09-21T03:56:38.271 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-21T03:56:38.271 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:38.271 INFO:teuthology.orchestra.run.smithi039.stdout: "mds_sessions": { 2024-09-21T03:56:38.271 INFO:teuthology.orchestra.run.smithi039.stdout: "session_count": 1, 2024-09-21T03:56:38.271 INFO:teuthology.orchestra.run.smithi039.stdout: "session_add": 3, 2024-09-21T03:56:38.271 INFO:teuthology.orchestra.run.smithi039.stdout: "session_remove": 2, 2024-09-21T03:56:38.272 INFO:teuthology.orchestra.run.smithi039.stdout: "sessions_open": 2, 2024-09-21T03:56:38.272 INFO:teuthology.orchestra.run.smithi039.stdout: "sessions_stale": 0, 2024-09-21T03:56:38.272 INFO:teuthology.orchestra.run.smithi039.stdout: "total_load": 0, 2024-09-21T03:56:38.272 INFO:teuthology.orchestra.run.smithi039.stdout: "average_load": 0, 2024-09-21T03:56:38.272 INFO:teuthology.orchestra.run.smithi039.stdout: "avg_session_uptime": 2974, 2024-09-21T03:56:38.272 INFO:teuthology.orchestra.run.smithi039.stdout: "mdthresh_evicted": 0 2024-09-21T03:56:38.272 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:38.272 INFO:teuthology.orchestra.run.smithi039.stdout: "mempool": { 2024-09-21T03:56:38.272 INFO:teuthology.orchestra.run.smithi039.stdout: "bloom_filter_bytes": 0, 2024-09-21T03:56:38.272 INFO:teuthology.orchestra.run.smithi039.stdout: "bloom_filter_items": 0, 2024-09-21T03:56:38.272 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_alloc_bytes": 0, 2024-09-21T03:56:38.272 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_alloc_items": 0, 2024-09-21T03:56:38.272 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_cache_data_bytes": 0, 2024-09-21T03:56:38.272 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_cache_data_items": 0, 2024-09-21T03:56:38.272 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-21T03:56:38.273 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_cache_onode_items": 0, 2024-09-21T03:56:38.273 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-21T03:56:38.273 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_cache_meta_items": 0, 2024-09-21T03:56:38.273 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_cache_other_bytes": 0, 2024-09-21T03:56:38.273 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_cache_other_items": 0, 2024-09-21T03:56:38.273 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-21T03:56:38.273 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_cache_buffer_items": 0, 2024-09-21T03:56:38.273 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_extent_bytes": 0, 2024-09-21T03:56:38.273 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_extent_items": 0, 2024-09-21T03:56:38.273 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_blob_bytes": 0, 2024-09-21T03:56:38.273 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_blob_items": 0, 2024-09-21T03:56:38.273 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-21T03:56:38.273 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_shared_blob_items": 0, 2024-09-21T03:56:38.273 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-21T03:56:38.273 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_inline_bl_items": 0, 2024-09-21T03:56:38.274 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_fsck_bytes": 0, 2024-09-21T03:56:38.274 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_fsck_items": 0, 2024-09-21T03:56:38.274 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_txc_bytes": 0, 2024-09-21T03:56:38.274 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_txc_items": 0, 2024-09-21T03:56:38.274 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-21T03:56:38.274 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_writing_deferred_items": 0, 2024-09-21T03:56:38.274 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_writing_bytes": 0, 2024-09-21T03:56:38.274 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_writing_items": 0, 2024-09-21T03:56:38.274 INFO:teuthology.orchestra.run.smithi039.stdout: "bluefs_bytes": 0, 2024-09-21T03:56:38.274 INFO:teuthology.orchestra.run.smithi039.stdout: "bluefs_items": 0, 2024-09-21T03:56:38.274 INFO:teuthology.orchestra.run.smithi039.stdout: "bluefs_file_reader_bytes": 0, 2024-09-21T03:56:38.274 INFO:teuthology.orchestra.run.smithi039.stdout: "bluefs_file_reader_items": 0, 2024-09-21T03:56:38.274 INFO:teuthology.orchestra.run.smithi039.stdout: "bluefs_file_writer_bytes": 0, 2024-09-21T03:56:38.274 INFO:teuthology.orchestra.run.smithi039.stdout: "bluefs_file_writer_items": 0, 2024-09-21T03:56:38.275 INFO:teuthology.orchestra.run.smithi039.stdout: "buffer_anon_bytes": 197480, 2024-09-21T03:56:38.275 INFO:teuthology.orchestra.run.smithi039.stdout: "buffer_anon_items": 55, 2024-09-21T03:56:38.275 INFO:teuthology.orchestra.run.smithi039.stdout: "buffer_meta_bytes": 0, 2024-09-21T03:56:38.275 INFO:teuthology.orchestra.run.smithi039.stdout: "buffer_meta_items": 0, 2024-09-21T03:56:38.275 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_bytes": 0, 2024-09-21T03:56:38.275 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_items": 0, 2024-09-21T03:56:38.275 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_mapbl_bytes": 0, 2024-09-21T03:56:38.275 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_mapbl_items": 0, 2024-09-21T03:56:38.275 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_pglog_bytes": 0, 2024-09-21T03:56:38.275 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_pglog_items": 0, 2024-09-21T03:56:38.275 INFO:teuthology.orchestra.run.smithi039.stdout: "osdmap_bytes": 8256, 2024-09-21T03:56:38.275 INFO:teuthology.orchestra.run.smithi039.stdout: "osdmap_items": 182, 2024-09-21T03:56:38.275 INFO:teuthology.orchestra.run.smithi039.stdout: "osdmap_mapping_bytes": 0, 2024-09-21T03:56:38.275 INFO:teuthology.orchestra.run.smithi039.stdout: "osdmap_mapping_items": 0, 2024-09-21T03:56:38.275 INFO:teuthology.orchestra.run.smithi039.stdout: "pgmap_bytes": 0, 2024-09-21T03:56:38.276 INFO:teuthology.orchestra.run.smithi039.stdout: "pgmap_items": 0, 2024-09-21T03:56:38.276 INFO:teuthology.orchestra.run.smithi039.stdout: "mds_co_bytes": 8672, 2024-09-21T03:56:38.276 INFO:teuthology.orchestra.run.smithi039.stdout: "mds_co_items": 13, 2024-09-21T03:56:38.276 INFO:teuthology.orchestra.run.smithi039.stdout: "unittest_1_bytes": 0, 2024-09-21T03:56:38.276 INFO:teuthology.orchestra.run.smithi039.stdout: "unittest_1_items": 0, 2024-09-21T03:56:38.276 INFO:teuthology.orchestra.run.smithi039.stdout: "unittest_2_bytes": 0, 2024-09-21T03:56:38.276 INFO:teuthology.orchestra.run.smithi039.stdout: "unittest_2_items": 0 2024-09-21T03:56:38.276 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:38.276 INFO:teuthology.orchestra.run.smithi039.stdout: "objecter": { 2024-09-21T03:56:38.276 INFO:teuthology.orchestra.run.smithi039.stdout: "op_active": 0, 2024-09-21T03:56:38.276 INFO:teuthology.orchestra.run.smithi039.stdout: "op_laggy": 0, 2024-09-21T03:56:38.276 INFO:teuthology.orchestra.run.smithi039.stdout: "op_send": 2111, 2024-09-21T03:56:38.276 INFO:teuthology.orchestra.run.smithi039.stdout: "op_send_bytes": 21, 2024-09-21T03:56:38.276 INFO:teuthology.orchestra.run.smithi039.stdout: "op_resend": 3, 2024-09-21T03:56:38.276 INFO:teuthology.orchestra.run.smithi039.stdout: "op_reply": 2111, 2024-09-21T03:56:38.277 INFO:teuthology.orchestra.run.smithi039.stdout: "op_latency": { 2024-09-21T03:56:38.277 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 2111, 2024-09-21T03:56:38.277 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 2.821943768, 2024-09-21T03:56:38.277 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.001336780 2024-09-21T03:56:38.277 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:38.277 INFO:teuthology.orchestra.run.smithi039.stdout: "op_inflight": 0, 2024-09-21T03:56:38.277 INFO:teuthology.orchestra.run.smithi039.stdout: "oplen_avg": { 2024-09-21T03:56:38.277 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 2111, 2024-09-21T03:56:38.277 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 2112 2024-09-21T03:56:38.277 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:38.277 INFO:teuthology.orchestra.run.smithi039.stdout: "op": 2111, 2024-09-21T03:56:38.277 INFO:teuthology.orchestra.run.smithi039.stdout: "op_r": 2111, 2024-09-21T03:56:38.277 INFO:teuthology.orchestra.run.smithi039.stdout: "op_w": 0, 2024-09-21T03:56:38.277 INFO:teuthology.orchestra.run.smithi039.stdout: "op_rmw": 0, 2024-09-21T03:56:38.277 INFO:teuthology.orchestra.run.smithi039.stdout: "op_pg": 0, 2024-09-21T03:56:38.277 INFO:teuthology.orchestra.run.smithi039.stdout: "osdop_stat": 1370, 2024-09-21T03:56:38.278 INFO:teuthology.orchestra.run.smithi039.stdout: "osdop_create": 0, 2024-09-21T03:56:38.278 INFO:teuthology.orchestra.run.smithi039.stdout: "osdop_read": 739, 2024-09-21T03:56:38.278 INFO:teuthology.orchestra.run.smithi039.stdout: "osdop_write": 0, 2024-09-21T03:56:38.278 INFO:teuthology.orchestra.run.smithi039.stdout: "osdop_writefull": 0, 2024-09-21T03:56:38.278 INFO:teuthology.orchestra.run.smithi039.stdout: "osdop_writesame": 0, 2024-09-21T03:56:38.278 INFO:teuthology.orchestra.run.smithi039.stdout: "osdop_append": 0, 2024-09-21T03:56:38.278 INFO:teuthology.orchestra.run.smithi039.stdout: "osdop_zero": 0, 2024-09-21T03:56:38.278 INFO:teuthology.orchestra.run.smithi039.stdout: "osdop_truncate": 0, 2024-09-21T03:56:38.278 INFO:teuthology.orchestra.run.smithi039.stdout: "osdop_delete": 0, 2024-09-21T03:56:38.278 INFO:teuthology.orchestra.run.smithi039.stdout: "osdop_mapext": 0, 2024-09-21T03:56:38.278 INFO:teuthology.orchestra.run.smithi039.stdout: "osdop_sparse_read": 0, 2024-09-21T03:56:38.278 INFO:teuthology.orchestra.run.smithi039.stdout: "osdop_clonerange": 0, 2024-09-21T03:56:38.278 INFO:teuthology.orchestra.run.smithi039.stdout: "osdop_getxattr": 1, 2024-09-21T03:56:38.278 INFO:teuthology.orchestra.run.smithi039.stdout: "osdop_setxattr": 0, 2024-09-21T03:56:38.279 INFO:teuthology.orchestra.run.smithi039.stdout: "osdop_cmpxattr": 0, 2024-09-21T03:56:38.279 INFO:teuthology.orchestra.run.smithi039.stdout: "osdop_rmxattr": 0, 2024-09-21T03:56:38.279 INFO:teuthology.orchestra.run.smithi039.stdout: "osdop_resetxattrs": 0, 2024-09-21T03:56:38.279 INFO:teuthology.orchestra.run.smithi039.stdout: "osdop_call": 0, 2024-09-21T03:56:38.279 INFO:teuthology.orchestra.run.smithi039.stdout: "osdop_watch": 0, 2024-09-21T03:56:38.279 INFO:teuthology.orchestra.run.smithi039.stdout: "osdop_notify": 0, 2024-09-21T03:56:38.279 INFO:teuthology.orchestra.run.smithi039.stdout: "osdop_src_cmpxattr": 0, 2024-09-21T03:56:38.279 INFO:teuthology.orchestra.run.smithi039.stdout: "osdop_pgls": 0, 2024-09-21T03:56:38.279 INFO:teuthology.orchestra.run.smithi039.stdout: "osdop_pgls_filter": 0, 2024-09-21T03:56:38.279 INFO:teuthology.orchestra.run.smithi039.stdout: "osdop_other": 0, 2024-09-21T03:56:38.279 INFO:teuthology.orchestra.run.smithi039.stdout: "linger_active": 0, 2024-09-21T03:56:38.279 INFO:teuthology.orchestra.run.smithi039.stdout: "linger_send": 0, 2024-09-21T03:56:38.279 INFO:teuthology.orchestra.run.smithi039.stdout: "linger_resend": 0, 2024-09-21T03:56:38.279 INFO:teuthology.orchestra.run.smithi039.stdout: "linger_ping": 0, 2024-09-21T03:56:38.279 INFO:teuthology.orchestra.run.smithi039.stdout: "poolop_active": 0, 2024-09-21T03:56:38.280 INFO:teuthology.orchestra.run.smithi039.stdout: "poolop_send": 0, 2024-09-21T03:56:38.280 INFO:teuthology.orchestra.run.smithi039.stdout: "poolop_resend": 0, 2024-09-21T03:56:38.280 INFO:teuthology.orchestra.run.smithi039.stdout: "poolstat_active": 0, 2024-09-21T03:56:38.280 INFO:teuthology.orchestra.run.smithi039.stdout: "poolstat_send": 0, 2024-09-21T03:56:38.280 INFO:teuthology.orchestra.run.smithi039.stdout: "poolstat_resend": 0, 2024-09-21T03:56:38.280 INFO:teuthology.orchestra.run.smithi039.stdout: "statfs_active": 0, 2024-09-21T03:56:38.280 INFO:teuthology.orchestra.run.smithi039.stdout: "statfs_send": 0, 2024-09-21T03:56:38.280 INFO:teuthology.orchestra.run.smithi039.stdout: "statfs_resend": 0, 2024-09-21T03:56:38.280 INFO:teuthology.orchestra.run.smithi039.stdout: "command_active": 0, 2024-09-21T03:56:38.280 INFO:teuthology.orchestra.run.smithi039.stdout: "command_send": 0, 2024-09-21T03:56:38.280 INFO:teuthology.orchestra.run.smithi039.stdout: "command_resend": 0, 2024-09-21T03:56:38.280 INFO:teuthology.orchestra.run.smithi039.stdout: "map_epoch": 87, 2024-09-21T03:56:38.280 INFO:teuthology.orchestra.run.smithi039.stdout: "map_full": 0, 2024-09-21T03:56:38.280 INFO:teuthology.orchestra.run.smithi039.stdout: "map_inc": 10, 2024-09-21T03:56:38.280 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_sessions": 6, 2024-09-21T03:56:38.281 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_session_open": 6, 2024-09-21T03:56:38.281 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_session_close": 0, 2024-09-21T03:56:38.281 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_laggy": 0, 2024-09-21T03:56:38.281 INFO:teuthology.orchestra.run.smithi039.stdout: "omap_wr": 0, 2024-09-21T03:56:38.281 INFO:teuthology.orchestra.run.smithi039.stdout: "omap_rd": 2, 2024-09-21T03:56:38.281 INFO:teuthology.orchestra.run.smithi039.stdout: "omap_del": 0 2024-09-21T03:56:38.281 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:38.281 INFO:teuthology.orchestra.run.smithi039.stdout: "oft": { 2024-09-21T03:56:38.281 INFO:teuthology.orchestra.run.smithi039.stdout: "omap_total_objs": 0, 2024-09-21T03:56:38.281 INFO:teuthology.orchestra.run.smithi039.stdout: "omap_total_kv_pairs": 0, 2024-09-21T03:56:38.281 INFO:teuthology.orchestra.run.smithi039.stdout: "omap_total_updates": 0, 2024-09-21T03:56:38.281 INFO:teuthology.orchestra.run.smithi039.stdout: "omap_total_removes": 0 2024-09-21T03:56:38.281 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:38.281 INFO:teuthology.orchestra.run.smithi039.stdout: "purge_queue": { 2024-09-21T03:56:38.281 INFO:teuthology.orchestra.run.smithi039.stdout: "pq_executing_ops": 0, 2024-09-21T03:56:38.282 INFO:teuthology.orchestra.run.smithi039.stdout: "pq_executing_ops_high_water": 0, 2024-09-21T03:56:38.282 INFO:teuthology.orchestra.run.smithi039.stdout: "pq_executing": 0, 2024-09-21T03:56:38.282 INFO:teuthology.orchestra.run.smithi039.stdout: "pq_executing_high_water": 0, 2024-09-21T03:56:38.282 INFO:teuthology.orchestra.run.smithi039.stdout: "pq_executed_ops": 0, 2024-09-21T03:56:38.282 INFO:teuthology.orchestra.run.smithi039.stdout: "pq_executed": 0, 2024-09-21T03:56:38.282 INFO:teuthology.orchestra.run.smithi039.stdout: "pq_item_in_journal": 0 2024-09-21T03:56:38.282 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:38.282 INFO:teuthology.orchestra.run.smithi039.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-21T03:56:38.282 INFO:teuthology.orchestra.run.smithi039.stdout: "val": 0, 2024-09-21T03:56:38.282 INFO:teuthology.orchestra.run.smithi039.stdout: "max": 104857600, 2024-09-21T03:56:38.282 INFO:teuthology.orchestra.run.smithi039.stdout: "get_started": 0, 2024-09-21T03:56:38.282 INFO:teuthology.orchestra.run.smithi039.stdout: "get": 2341, 2024-09-21T03:56:38.282 INFO:teuthology.orchestra.run.smithi039.stdout: "get_sum": 1557233, 2024-09-21T03:56:38.282 INFO:teuthology.orchestra.run.smithi039.stdout: "get_or_fail_fail": 0, 2024-09-21T03:56:38.282 INFO:teuthology.orchestra.run.smithi039.stdout: "get_or_fail_success": 2341, 2024-09-21T03:56:38.283 INFO:teuthology.orchestra.run.smithi039.stdout: "take": 0, 2024-09-21T03:56:38.283 INFO:teuthology.orchestra.run.smithi039.stdout: "take_sum": 0, 2024-09-21T03:56:38.283 INFO:teuthology.orchestra.run.smithi039.stdout: "put": 2341, 2024-09-21T03:56:38.283 INFO:teuthology.orchestra.run.smithi039.stdout: "put_sum": 1557233, 2024-09-21T03:56:38.283 INFO:teuthology.orchestra.run.smithi039.stdout: "wait": { 2024-09-21T03:56:38.283 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2024-09-21T03:56:38.283 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2024-09-21T03:56:38.283 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2024-09-21T03:56:38.283 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-21T03:56:38.283 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:38.283 INFO:teuthology.orchestra.run.smithi039.stdout: "throttle-objecter_bytes": { 2024-09-21T03:56:38.283 INFO:teuthology.orchestra.run.smithi039.stdout: "val": 0, 2024-09-21T03:56:38.283 INFO:teuthology.orchestra.run.smithi039.stdout: "max": 104857600, 2024-09-21T03:56:38.283 INFO:teuthology.orchestra.run.smithi039.stdout: "get_started": 0, 2024-09-21T03:56:38.283 INFO:teuthology.orchestra.run.smithi039.stdout: "get": 0, 2024-09-21T03:56:38.283 INFO:teuthology.orchestra.run.smithi039.stdout: "get_sum": 0, 2024-09-21T03:56:38.284 INFO:teuthology.orchestra.run.smithi039.stdout: "get_or_fail_fail": 0, 2024-09-21T03:56:38.284 INFO:teuthology.orchestra.run.smithi039.stdout: "get_or_fail_success": 0, 2024-09-21T03:56:38.284 INFO:teuthology.orchestra.run.smithi039.stdout: "take": 2111, 2024-09-21T03:56:38.284 INFO:teuthology.orchestra.run.smithi039.stdout: "take_sum": 1965, 2024-09-21T03:56:38.284 INFO:teuthology.orchestra.run.smithi039.stdout: "put": 2111, 2024-09-21T03:56:38.284 INFO:teuthology.orchestra.run.smithi039.stdout: "put_sum": 1965, 2024-09-21T03:56:38.284 INFO:teuthology.orchestra.run.smithi039.stdout: "wait": { 2024-09-21T03:56:38.284 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2024-09-21T03:56:38.284 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2024-09-21T03:56:38.284 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2024-09-21T03:56:38.284 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-21T03:56:38.284 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:38.284 INFO:teuthology.orchestra.run.smithi039.stdout: "throttle-objecter_ops": { 2024-09-21T03:56:38.284 INFO:teuthology.orchestra.run.smithi039.stdout: "val": 0, 2024-09-21T03:56:38.284 INFO:teuthology.orchestra.run.smithi039.stdout: "max": 1024, 2024-09-21T03:56:38.285 INFO:teuthology.orchestra.run.smithi039.stdout: "get_started": 0, 2024-09-21T03:56:38.285 INFO:teuthology.orchestra.run.smithi039.stdout: "get": 0, 2024-09-21T03:56:38.285 INFO:teuthology.orchestra.run.smithi039.stdout: "get_sum": 0, 2024-09-21T03:56:38.285 INFO:teuthology.orchestra.run.smithi039.stdout: "get_or_fail_fail": 0, 2024-09-21T03:56:38.285 INFO:teuthology.orchestra.run.smithi039.stdout: "get_or_fail_success": 0, 2024-09-21T03:56:38.285 INFO:teuthology.orchestra.run.smithi039.stdout: "take": 2111, 2024-09-21T03:56:38.285 INFO:teuthology.orchestra.run.smithi039.stdout: "take_sum": 2111, 2024-09-21T03:56:38.285 INFO:teuthology.orchestra.run.smithi039.stdout: "put": 2111, 2024-09-21T03:56:38.285 INFO:teuthology.orchestra.run.smithi039.stdout: "put_sum": 2111, 2024-09-21T03:56:38.285 INFO:teuthology.orchestra.run.smithi039.stdout: "wait": { 2024-09-21T03:56:38.285 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2024-09-21T03:56:38.285 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2024-09-21T03:56:38.285 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2024-09-21T03:56:38.285 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-21T03:56:38.285 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:38.286 INFO:teuthology.orchestra.run.smithi039.stdout: "throttle-write_buf_throttle": { 2024-09-21T03:56:38.286 INFO:teuthology.orchestra.run.smithi039.stdout: "val": 0, 2024-09-21T03:56:38.286 INFO:teuthology.orchestra.run.smithi039.stdout: "max": 3758096384, 2024-09-21T03:56:38.286 INFO:teuthology.orchestra.run.smithi039.stdout: "get_started": 0, 2024-09-21T03:56:38.286 INFO:teuthology.orchestra.run.smithi039.stdout: "get": 0, 2024-09-21T03:56:38.286 INFO:teuthology.orchestra.run.smithi039.stdout: "get_sum": 0, 2024-09-21T03:56:38.286 INFO:teuthology.orchestra.run.smithi039.stdout: "get_or_fail_fail": 0, 2024-09-21T03:56:38.286 INFO:teuthology.orchestra.run.smithi039.stdout: "get_or_fail_success": 0, 2024-09-21T03:56:38.286 INFO:teuthology.orchestra.run.smithi039.stdout: "take": 0, 2024-09-21T03:56:38.286 INFO:teuthology.orchestra.run.smithi039.stdout: "take_sum": 0, 2024-09-21T03:56:38.286 INFO:teuthology.orchestra.run.smithi039.stdout: "put": 0, 2024-09-21T03:56:38.286 INFO:teuthology.orchestra.run.smithi039.stdout: "put_sum": 0, 2024-09-21T03:56:38.286 INFO:teuthology.orchestra.run.smithi039.stdout: "wait": { 2024-09-21T03:56:38.286 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2024-09-21T03:56:38.286 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2024-09-21T03:56:38.286 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2024-09-21T03:56:38.287 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-21T03:56:38.287 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:38.287 INFO:teuthology.orchestra.run.smithi039.stdout: "throttle-write_buf_throttle-0x56443bb35450": { 2024-09-21T03:56:38.287 INFO:teuthology.orchestra.run.smithi039.stdout: "val": 0, 2024-09-21T03:56:38.287 INFO:teuthology.orchestra.run.smithi039.stdout: "max": 3758096384, 2024-09-21T03:56:38.287 INFO:teuthology.orchestra.run.smithi039.stdout: "get_started": 0, 2024-09-21T03:56:38.287 INFO:teuthology.orchestra.run.smithi039.stdout: "get": 0, 2024-09-21T03:56:38.287 INFO:teuthology.orchestra.run.smithi039.stdout: "get_sum": 0, 2024-09-21T03:56:38.287 INFO:teuthology.orchestra.run.smithi039.stdout: "get_or_fail_fail": 0, 2024-09-21T03:56:38.287 INFO:teuthology.orchestra.run.smithi039.stdout: "get_or_fail_success": 0, 2024-09-21T03:56:38.287 INFO:teuthology.orchestra.run.smithi039.stdout: "take": 0, 2024-09-21T03:56:38.287 INFO:teuthology.orchestra.run.smithi039.stdout: "take_sum": 0, 2024-09-21T03:56:38.287 INFO:teuthology.orchestra.run.smithi039.stdout: "put": 0, 2024-09-21T03:56:38.287 INFO:teuthology.orchestra.run.smithi039.stdout: "put_sum": 0, 2024-09-21T03:56:38.287 INFO:teuthology.orchestra.run.smithi039.stdout: "wait": { 2024-09-21T03:56:38.288 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2024-09-21T03:56:38.288 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2024-09-21T03:56:38.288 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2024-09-21T03:56:38.288 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-21T03:56:38.288 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:38.288 INFO:teuthology.orchestra.run.smithi039.stdout: "trackedop": { 2024-09-21T03:56:38.288 INFO:teuthology.orchestra.run.smithi039.stdout: "slow_ops_count": 0 2024-09-21T03:56:38.288 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-21T03:56:38.288 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-09-21T03:56:39.172 INFO:tasks.check_counter:Daemon mds.g mds.exported=0 2024-09-21T03:56:39.172 INFO:tasks.check_counter:Daemon mds.g mds.imported=0 2024-09-21T03:56:39.172 INFO:tasks.check_counter:Daemon mds.g mds_cache.dir_update=0 2024-09-21T03:56:39.172 INFO:tasks.check_counter:Daemon mds.g mds_cache.dir_update_receipt=0 2024-09-21T03:56:39.172 INFO:tasks.check_counter:Daemon mds.g mds.root_rsnaps=0 2024-09-21T03:56:39.172 INFO:tasks.check_counter:Daemon mds.g mds_server.req_mksnap_latency.avgcount=0 2024-09-21T03:56:39.172 INFO:tasks.check_counter:Daemon mds.g mds_server.req_rmsnap_latency.avgcount=0 2024-09-21T03:56:39.172 DEBUG:tasks.check_counter:Getting stats from j 2024-09-21T03:56:39.173 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 51116d5c-77ca-11ef-baf5-efdc52797490 -- ceph daemon mds.j perf dump 2024-09-21T03:56:39.437 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/51116d5c-77ca-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:56:39.558 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:56:39 smithi039 ceph-mon[31211]: pgmap v651: 97 pgs: 97 active+clean; 10 MiB data, 525 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-09-21T03:56:39.718 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:56:39 smithi161 ceph-mon[34533]: pgmap v651: 97 pgs: 97 active+clean; 10 MiB data, 525 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-09-21T03:56:39.725 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:56:39 smithi150 ceph-mon[34289]: pgmap v651: 97 pgs: 97 active+clean; 10 MiB data, 525 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-09-21T03:56:39.964 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-09-21T03:56:39.964 INFO:teuthology.orchestra.run.smithi039.stdout: "AsyncMessenger::Worker-0": { 2024-09-21T03:56:39.964 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_recv_messages": 0, 2024-09-21T03:56:39.964 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_send_messages": 2, 2024-09-21T03:56:39.964 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_recv_bytes": 0, 2024-09-21T03:56:39.964 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_send_bytes": 0, 2024-09-21T03:56:39.964 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_created_connections": 4, 2024-09-21T03:56:39.965 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_active_connections": 0, 2024-09-21T03:56:39.965 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_running_total_time": 0.001418627, 2024-09-21T03:56:39.965 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_running_send_time": 0.000000000, 2024-09-21T03:56:39.965 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_running_recv_time": 0.000414393, 2024-09-21T03:56:39.965 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-21T03:56:39.965 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T03:56:39.965 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2024-09-21T03:56:39.965 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2024-09-21T03:56:39.965 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2024-09-21T03:56:39.965 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:39.965 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_handle_ack_lat": { 2024-09-21T03:56:39.965 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2024-09-21T03:56:39.965 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2024-09-21T03:56:39.965 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2024-09-21T03:56:39.966 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:39.966 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_recv_encrypted_bytes": 0, 2024-09-21T03:56:39.966 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_send_encrypted_bytes": 0 2024-09-21T03:56:39.966 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:39.966 INFO:teuthology.orchestra.run.smithi039.stdout: "AsyncMessenger::Worker-1": { 2024-09-21T03:56:39.966 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_recv_messages": 216, 2024-09-21T03:56:39.966 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_send_messages": 209, 2024-09-21T03:56:39.966 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_recv_bytes": 1145536, 2024-09-21T03:56:39.966 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_send_bytes": 128544, 2024-09-21T03:56:39.966 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_created_connections": 2, 2024-09-21T03:56:39.966 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_active_connections": 1, 2024-09-21T03:56:39.966 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_running_total_time": 0.043647528, 2024-09-21T03:56:39.966 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_running_send_time": 0.019693073, 2024-09-21T03:56:39.966 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_running_recv_time": 0.034954139, 2024-09-21T03:56:39.967 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-21T03:56:39.967 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T03:56:39.967 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 208, 2024-09-21T03:56:39.967 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.011792620, 2024-09-21T03:56:39.967 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000056695 2024-09-21T03:56:39.967 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:39.967 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_handle_ack_lat": { 2024-09-21T03:56:39.967 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2024-09-21T03:56:39.967 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2024-09-21T03:56:39.967 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2024-09-21T03:56:39.967 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:39.967 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_recv_encrypted_bytes": 1145536, 2024-09-21T03:56:39.967 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_send_encrypted_bytes": 128544 2024-09-21T03:56:39.967 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:39.968 INFO:teuthology.orchestra.run.smithi039.stdout: "AsyncMessenger::Worker-2": { 2024-09-21T03:56:39.968 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_recv_messages": 5, 2024-09-21T03:56:39.968 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_send_messages": 161, 2024-09-21T03:56:39.968 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_recv_bytes": 3200, 2024-09-21T03:56:39.968 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_send_bytes": 108720, 2024-09-21T03:56:39.968 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_created_connections": 3, 2024-09-21T03:56:39.968 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_active_connections": 2, 2024-09-21T03:56:39.968 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_running_total_time": 0.017525233, 2024-09-21T03:56:39.968 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_running_send_time": 0.012326372, 2024-09-21T03:56:39.968 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_running_recv_time": 0.003521149, 2024-09-21T03:56:39.968 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-21T03:56:39.968 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T03:56:39.968 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 160, 2024-09-21T03:56:39.969 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.013292333, 2024-09-21T03:56:39.969 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000083077 2024-09-21T03:56:39.969 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:39.969 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_handle_ack_lat": { 2024-09-21T03:56:39.969 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2024-09-21T03:56:39.969 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2024-09-21T03:56:39.969 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2024-09-21T03:56:39.969 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:39.969 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_recv_encrypted_bytes": 3200, 2024-09-21T03:56:39.969 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_send_encrypted_bytes": 108720 2024-09-21T03:56:39.969 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:39.969 INFO:teuthology.orchestra.run.smithi039.stdout: "cct": { 2024-09-21T03:56:39.969 INFO:teuthology.orchestra.run.smithi039.stdout: "total_workers": 0, 2024-09-21T03:56:39.969 INFO:teuthology.orchestra.run.smithi039.stdout: "unhealthy_workers": 0 2024-09-21T03:56:39.970 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:39.970 INFO:teuthology.orchestra.run.smithi039.stdout: "mempool": { 2024-09-21T03:56:39.970 INFO:teuthology.orchestra.run.smithi039.stdout: "bloom_filter_bytes": 0, 2024-09-21T03:56:39.970 INFO:teuthology.orchestra.run.smithi039.stdout: "bloom_filter_items": 0, 2024-09-21T03:56:39.970 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_alloc_bytes": 0, 2024-09-21T03:56:39.970 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_alloc_items": 0, 2024-09-21T03:56:39.970 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_cache_data_bytes": 0, 2024-09-21T03:56:39.970 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_cache_data_items": 0, 2024-09-21T03:56:39.970 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-21T03:56:39.970 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_cache_onode_items": 0, 2024-09-21T03:56:39.970 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-21T03:56:39.970 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_cache_meta_items": 0, 2024-09-21T03:56:39.970 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_cache_other_bytes": 0, 2024-09-21T03:56:39.970 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_cache_other_items": 0, 2024-09-21T03:56:39.971 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-21T03:56:39.971 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_cache_buffer_items": 0, 2024-09-21T03:56:39.971 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_extent_bytes": 0, 2024-09-21T03:56:39.971 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_extent_items": 0, 2024-09-21T03:56:39.971 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_blob_bytes": 0, 2024-09-21T03:56:39.971 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_blob_items": 0, 2024-09-21T03:56:39.971 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-21T03:56:39.971 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_shared_blob_items": 0, 2024-09-21T03:56:39.971 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-21T03:56:39.971 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_inline_bl_items": 0, 2024-09-21T03:56:39.971 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_fsck_bytes": 0, 2024-09-21T03:56:39.971 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_fsck_items": 0, 2024-09-21T03:56:39.971 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_txc_bytes": 0, 2024-09-21T03:56:39.971 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_txc_items": 0, 2024-09-21T03:56:39.971 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-21T03:56:39.972 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_writing_deferred_items": 0, 2024-09-21T03:56:39.972 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_writing_bytes": 0, 2024-09-21T03:56:39.972 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_writing_items": 0, 2024-09-21T03:56:39.972 INFO:teuthology.orchestra.run.smithi039.stdout: "bluefs_bytes": 0, 2024-09-21T03:56:39.972 INFO:teuthology.orchestra.run.smithi039.stdout: "bluefs_items": 0, 2024-09-21T03:56:39.972 INFO:teuthology.orchestra.run.smithi039.stdout: "bluefs_file_reader_bytes": 0, 2024-09-21T03:56:39.972 INFO:teuthology.orchestra.run.smithi039.stdout: "bluefs_file_reader_items": 0, 2024-09-21T03:56:39.972 INFO:teuthology.orchestra.run.smithi039.stdout: "bluefs_file_writer_bytes": 0, 2024-09-21T03:56:39.972 INFO:teuthology.orchestra.run.smithi039.stdout: "bluefs_file_writer_items": 0, 2024-09-21T03:56:39.972 INFO:teuthology.orchestra.run.smithi039.stdout: "buffer_anon_bytes": 149007, 2024-09-21T03:56:39.972 INFO:teuthology.orchestra.run.smithi039.stdout: "buffer_anon_items": 30, 2024-09-21T03:56:39.972 INFO:teuthology.orchestra.run.smithi039.stdout: "buffer_meta_bytes": 0, 2024-09-21T03:56:39.972 INFO:teuthology.orchestra.run.smithi039.stdout: "buffer_meta_items": 0, 2024-09-21T03:56:39.972 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_bytes": 0, 2024-09-21T03:56:39.972 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_items": 0, 2024-09-21T03:56:39.973 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_mapbl_bytes": 0, 2024-09-21T03:56:39.973 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_mapbl_items": 0, 2024-09-21T03:56:39.973 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_pglog_bytes": 0, 2024-09-21T03:56:39.973 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_pglog_items": 0, 2024-09-21T03:56:39.973 INFO:teuthology.orchestra.run.smithi039.stdout: "osdmap_bytes": 0, 2024-09-21T03:56:39.973 INFO:teuthology.orchestra.run.smithi039.stdout: "osdmap_items": 0, 2024-09-21T03:56:39.973 INFO:teuthology.orchestra.run.smithi039.stdout: "osdmap_mapping_bytes": 0, 2024-09-21T03:56:39.973 INFO:teuthology.orchestra.run.smithi039.stdout: "osdmap_mapping_items": 0, 2024-09-21T03:56:39.973 INFO:teuthology.orchestra.run.smithi039.stdout: "pgmap_bytes": 0, 2024-09-21T03:56:39.973 INFO:teuthology.orchestra.run.smithi039.stdout: "pgmap_items": 0, 2024-09-21T03:56:39.973 INFO:teuthology.orchestra.run.smithi039.stdout: "mds_co_bytes": 968, 2024-09-21T03:56:39.973 INFO:teuthology.orchestra.run.smithi039.stdout: "mds_co_items": 2, 2024-09-21T03:56:39.973 INFO:teuthology.orchestra.run.smithi039.stdout: "unittest_1_bytes": 0, 2024-09-21T03:56:39.973 INFO:teuthology.orchestra.run.smithi039.stdout: "unittest_1_items": 0, 2024-09-21T03:56:39.973 INFO:teuthology.orchestra.run.smithi039.stdout: "unittest_2_bytes": 0, 2024-09-21T03:56:39.973 INFO:teuthology.orchestra.run.smithi039.stdout: "unittest_2_items": 0 2024-09-21T03:56:39.974 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T03:56:39.974 INFO:teuthology.orchestra.run.smithi039.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-21T03:56:39.974 INFO:teuthology.orchestra.run.smithi039.stdout: "val": 0, 2024-09-21T03:56:39.974 INFO:teuthology.orchestra.run.smithi039.stdout: "max": 104857600, 2024-09-21T03:56:39.974 INFO:teuthology.orchestra.run.smithi039.stdout: "get_started": 0, 2024-09-21T03:56:39.974 INFO:teuthology.orchestra.run.smithi039.stdout: "get": 217, 2024-09-21T03:56:39.974 INFO:teuthology.orchestra.run.smithi039.stdout: "get_sum": 1115007, 2024-09-21T03:56:39.974 INFO:teuthology.orchestra.run.smithi039.stdout: "get_or_fail_fail": 0, 2024-09-21T03:56:39.974 INFO:teuthology.orchestra.run.smithi039.stdout: "get_or_fail_success": 217, 2024-09-21T03:56:39.974 INFO:teuthology.orchestra.run.smithi039.stdout: "take": 0, 2024-09-21T03:56:39.974 INFO:teuthology.orchestra.run.smithi039.stdout: "take_sum": 0, 2024-09-21T03:56:39.974 INFO:teuthology.orchestra.run.smithi039.stdout: "put": 217, 2024-09-21T03:56:39.974 INFO:teuthology.orchestra.run.smithi039.stdout: "put_sum": 1115007, 2024-09-21T03:56:39.974 INFO:teuthology.orchestra.run.smithi039.stdout: "wait": { 2024-09-21T03:56:39.974 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2024-09-21T03:56:39.975 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2024-09-21T03:56:39.975 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2024-09-21T03:56:39.975 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-21T03:56:39.975 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-21T03:56:39.975 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-09-21T03:56:40.752 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.j 2024-09-21T03:56:40.752 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.j 2024-09-21T03:56:40.752 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.j 2024-09-21T03:56:40.752 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.j 2024-09-21T03:56:40.752 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.j 2024-09-21T03:56:40.752 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.j 2024-09-21T03:56:40.752 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.j 2024-09-21T03:56:40.752 DEBUG:tasks.check_counter:Getting stats from b 2024-09-21T03:56:40.752 DEBUG:teuthology.orchestra.run.smithi150:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 51116d5c-77ca-11ef-baf5-efdc52797490 -- ceph daemon mds.b perf dump 2024-09-21T03:56:41.016 INFO:teuthology.orchestra.run.smithi150.stderr:Inferring config /var/lib/ceph/51116d5c-77ca-11ef-baf5-efdc52797490/mon.b/config 2024-09-21T03:56:41.475 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:56:41 smithi150 ceph-mon[34289]: pgmap v652: 97 pgs: 97 active+clean; 10 MiB data, 525 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-09-21T03:56:41.718 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:56:41 smithi161 ceph-mon[34533]: pgmap v652: 97 pgs: 97 active+clean; 10 MiB data, 525 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-09-21T03:56:41.749 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-21T03:56:41.749 INFO:teuthology.orchestra.run.smithi150.stdout: "AsyncMessenger::Worker-0": { 2024-09-21T03:56:41.749 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_recv_messages": 4768, 2024-09-21T03:56:41.750 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_send_messages": 4010, 2024-09-21T03:56:41.750 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_recv_bytes": 2753733, 2024-09-21T03:56:41.750 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_send_bytes": 5662571, 2024-09-21T03:56:41.750 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_created_connections": 16, 2024-09-21T03:56:41.750 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_active_connections": 6, 2024-09-21T03:56:41.750 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_running_total_time": 0.532862804, 2024-09-21T03:56:41.750 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_running_send_time": 0.217841769, 2024-09-21T03:56:41.750 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_running_recv_time": 0.210942571, 2024-09-21T03:56:41.750 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_running_fast_dispatch_time": 0.110446379, 2024-09-21T03:56:41.750 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T03:56:41.750 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 4010, 2024-09-21T03:56:41.751 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.573143871, 2024-09-21T03:56:41.751 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000142928 2024-09-21T03:56:41.751 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T03:56:41.751 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_handle_ack_lat": { 2024-09-21T03:56:41.751 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 2674, 2024-09-21T03:56:41.751 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000614362, 2024-09-21T03:56:41.751 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000229 2024-09-21T03:56:41.751 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T03:56:41.751 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_recv_encrypted_bytes": 1534448, 2024-09-21T03:56:41.751 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_send_encrypted_bytes": 242368 2024-09-21T03:56:41.751 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T03:56:41.752 INFO:teuthology.orchestra.run.smithi150.stdout: "AsyncMessenger::Worker-1": { 2024-09-21T03:56:41.752 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_recv_messages": 6651, 2024-09-21T03:56:41.752 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_send_messages": 7541, 2024-09-21T03:56:41.752 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_recv_bytes": 2345456, 2024-09-21T03:56:41.752 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_send_bytes": 4552609, 2024-09-21T03:56:41.752 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_created_connections": 30, 2024-09-21T03:56:41.752 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_active_connections": 8, 2024-09-21T03:56:41.752 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_running_total_time": 0.725148989, 2024-09-21T03:56:41.752 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_running_send_time": 0.332187724, 2024-09-21T03:56:41.752 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_running_recv_time": 0.432585620, 2024-09-21T03:56:41.752 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_running_fast_dispatch_time": 0.116172007, 2024-09-21T03:56:41.753 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T03:56:41.753 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 7535, 2024-09-21T03:56:41.753 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 1.191488198, 2024-09-21T03:56:41.753 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000158127 2024-09-21T03:56:41.753 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T03:56:41.753 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_handle_ack_lat": { 2024-09-21T03:56:41.753 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 7121, 2024-09-21T03:56:41.753 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.002243170, 2024-09-21T03:56:41.753 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000315 2024-09-21T03:56:41.753 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T03:56:41.753 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_recv_encrypted_bytes": 1743824, 2024-09-21T03:56:41.753 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_send_encrypted_bytes": 2450560 2024-09-21T03:56:41.754 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T03:56:41.754 INFO:teuthology.orchestra.run.smithi150.stdout: "AsyncMessenger::Worker-2": { 2024-09-21T03:56:41.754 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_recv_messages": 4138, 2024-09-21T03:56:41.754 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_send_messages": 3886, 2024-09-21T03:56:41.754 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_recv_bytes": 1089314, 2024-09-21T03:56:41.754 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_send_bytes": 7003871, 2024-09-21T03:56:41.754 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_created_connections": 131, 2024-09-21T03:56:41.754 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_active_connections": 6, 2024-09-21T03:56:41.754 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_running_total_time": 0.484686169, 2024-09-21T03:56:41.754 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_running_send_time": 0.204964718, 2024-09-21T03:56:41.754 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_running_recv_time": 0.178633617, 2024-09-21T03:56:41.755 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_running_fast_dispatch_time": 0.088333538, 2024-09-21T03:56:41.755 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T03:56:41.755 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 3887, 2024-09-21T03:56:41.755 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.442953625, 2024-09-21T03:56:41.755 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000113957 2024-09-21T03:56:41.755 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T03:56:41.755 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_handle_ack_lat": { 2024-09-21T03:56:41.755 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 2060, 2024-09-21T03:56:41.755 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000449258, 2024-09-21T03:56:41.755 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000218 2024-09-21T03:56:41.755 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T03:56:41.755 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_recv_encrypted_bytes": 36528, 2024-09-21T03:56:41.756 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_send_encrypted_bytes": 1936 2024-09-21T03:56:41.756 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T03:56:41.756 INFO:teuthology.orchestra.run.smithi150.stdout: "cct": { 2024-09-21T03:56:41.756 INFO:teuthology.orchestra.run.smithi150.stdout: "total_workers": 1, 2024-09-21T03:56:41.756 INFO:teuthology.orchestra.run.smithi150.stdout: "unhealthy_workers": 0 2024-09-21T03:56:41.756 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T03:56:41.756 INFO:teuthology.orchestra.run.smithi150.stdout: "finisher-MDSRank": { 2024-09-21T03:56:41.756 INFO:teuthology.orchestra.run.smithi150.stdout: "queue_len": 0, 2024-09-21T03:56:41.756 INFO:teuthology.orchestra.run.smithi150.stdout: "complete_latency": { 2024-09-21T03:56:41.756 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 3153, 2024-09-21T03:56:41.756 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 1.657937956, 2024-09-21T03:56:41.756 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000525828 2024-09-21T03:56:41.757 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T03:56:41.757 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T03:56:41.757 INFO:teuthology.orchestra.run.smithi150.stdout: "finisher-PurgeQueue": { 2024-09-21T03:56:41.757 INFO:teuthology.orchestra.run.smithi150.stdout: "queue_len": 0, 2024-09-21T03:56:41.757 INFO:teuthology.orchestra.run.smithi150.stdout: "complete_latency": { 2024-09-21T03:56:41.757 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 3414, 2024-09-21T03:56:41.757 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.113215644, 2024-09-21T03:56:41.757 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000033162 2024-09-21T03:56:41.757 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T03:56:41.757 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T03:56:41.757 INFO:teuthology.orchestra.run.smithi150.stdout: "mds": { 2024-09-21T03:56:41.757 INFO:teuthology.orchestra.run.smithi150.stdout: "request": 159, 2024-09-21T03:56:41.758 INFO:teuthology.orchestra.run.smithi150.stdout: "reply": 159, 2024-09-21T03:56:41.758 INFO:teuthology.orchestra.run.smithi150.stdout: "reply_latency": { 2024-09-21T03:56:41.758 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 159, 2024-09-21T03:56:41.758 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.174085898, 2024-09-21T03:56:41.758 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.001094879 2024-09-21T03:56:41.758 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T03:56:41.758 INFO:teuthology.orchestra.run.smithi150.stdout: "slow_reply": 0, 2024-09-21T03:56:41.758 INFO:teuthology.orchestra.run.smithi150.stdout: "forward": 0, 2024-09-21T03:56:41.758 INFO:teuthology.orchestra.run.smithi150.stdout: "dir_fetch_complete": 0, 2024-09-21T03:56:41.758 INFO:teuthology.orchestra.run.smithi150.stdout: "dir_fetch_keys": 0, 2024-09-21T03:56:41.758 INFO:teuthology.orchestra.run.smithi150.stdout: "dir_commit": 12, 2024-09-21T03:56:41.758 INFO:teuthology.orchestra.run.smithi150.stdout: "dir_split": 0, 2024-09-21T03:56:41.759 INFO:teuthology.orchestra.run.smithi150.stdout: "dir_merge": 0, 2024-09-21T03:56:41.759 INFO:teuthology.orchestra.run.smithi150.stdout: "inodes": 32, 2024-09-21T03:56:41.759 INFO:teuthology.orchestra.run.smithi150.stdout: "inodes_top": 0, 2024-09-21T03:56:41.759 INFO:teuthology.orchestra.run.smithi150.stdout: "inodes_bottom": 32, 2024-09-21T03:56:41.759 INFO:teuthology.orchestra.run.smithi150.stdout: "inodes_pin_tail": 0, 2024-09-21T03:56:41.759 INFO:teuthology.orchestra.run.smithi150.stdout: "inodes_pinned": 31, 2024-09-21T03:56:41.759 INFO:teuthology.orchestra.run.smithi150.stdout: "inodes_expired": 0, 2024-09-21T03:56:41.759 INFO:teuthology.orchestra.run.smithi150.stdout: "inodes_with_caps": 2, 2024-09-21T03:56:41.759 INFO:teuthology.orchestra.run.smithi150.stdout: "caps": 3, 2024-09-21T03:56:41.759 INFO:teuthology.orchestra.run.smithi150.stdout: "subtrees": 2, 2024-09-21T03:56:41.759 INFO:teuthology.orchestra.run.smithi150.stdout: "traverse": 296, 2024-09-21T03:56:41.759 INFO:teuthology.orchestra.run.smithi150.stdout: "traverse_hit": 287, 2024-09-21T03:56:41.760 INFO:teuthology.orchestra.run.smithi150.stdout: "traverse_forward": 0, 2024-09-21T03:56:41.760 INFO:teuthology.orchestra.run.smithi150.stdout: "traverse_discover": 0, 2024-09-21T03:56:41.760 INFO:teuthology.orchestra.run.smithi150.stdout: "traverse_dir_fetch": 0, 2024-09-21T03:56:41.760 INFO:teuthology.orchestra.run.smithi150.stdout: "traverse_remote_ino": 0, 2024-09-21T03:56:41.760 INFO:teuthology.orchestra.run.smithi150.stdout: "traverse_lock": 0, 2024-09-21T03:56:41.760 INFO:teuthology.orchestra.run.smithi150.stdout: "load_cent": 0, 2024-09-21T03:56:41.760 INFO:teuthology.orchestra.run.smithi150.stdout: "q": 0, 2024-09-21T03:56:41.760 INFO:teuthology.orchestra.run.smithi150.stdout: "exported": 0, 2024-09-21T03:56:41.760 INFO:teuthology.orchestra.run.smithi150.stdout: "exported_inodes": 0, 2024-09-21T03:56:41.760 INFO:teuthology.orchestra.run.smithi150.stdout: "imported": 0, 2024-09-21T03:56:41.760 INFO:teuthology.orchestra.run.smithi150.stdout: "imported_inodes": 0, 2024-09-21T03:56:41.760 INFO:teuthology.orchestra.run.smithi150.stdout: "openino_dir_fetch": 0, 2024-09-21T03:56:41.761 INFO:teuthology.orchestra.run.smithi150.stdout: "openino_backtrace_fetch": 228, 2024-09-21T03:56:41.761 INFO:teuthology.orchestra.run.smithi150.stdout: "openino_peer_discover": 0, 2024-09-21T03:56:41.761 INFO:teuthology.orchestra.run.smithi150.stdout: "root_rfiles": 4, 2024-09-21T03:56:41.761 INFO:teuthology.orchestra.run.smithi150.stdout: "root_rbytes": 234, 2024-09-21T03:56:41.761 INFO:teuthology.orchestra.run.smithi150.stdout: "root_rsnaps": 6, 2024-09-21T03:56:41.761 INFO:teuthology.orchestra.run.smithi150.stdout: "scrub_backtrace_fetch": 228, 2024-09-21T03:56:41.761 INFO:teuthology.orchestra.run.smithi150.stdout: "scrub_set_tag": 0, 2024-09-21T03:56:41.761 INFO:teuthology.orchestra.run.smithi150.stdout: "scrub_backtrace_repaired": 0, 2024-09-21T03:56:41.761 INFO:teuthology.orchestra.run.smithi150.stdout: "scrub_inotable_repaired": 0, 2024-09-21T03:56:41.761 INFO:teuthology.orchestra.run.smithi150.stdout: "scrub_dir_inodes": 146, 2024-09-21T03:56:41.761 INFO:teuthology.orchestra.run.smithi150.stdout: "scrub_dir_base_inodes": 20, 2024-09-21T03:56:41.761 INFO:teuthology.orchestra.run.smithi150.stdout: "scrub_dirfrag_rstats": 126, 2024-09-21T03:56:41.762 INFO:teuthology.orchestra.run.smithi150.stdout: "scrub_file_inodes": 82, 2024-09-21T03:56:41.762 INFO:teuthology.orchestra.run.smithi150.stdout: "handle_inode_file_caps": 0, 2024-09-21T03:56:41.762 INFO:teuthology.orchestra.run.smithi150.stdout: "ceph_cap_op_revoke": 18, 2024-09-21T03:56:41.762 INFO:teuthology.orchestra.run.smithi150.stdout: "ceph_cap_op_grant": 1413, 2024-09-21T03:56:41.762 INFO:teuthology.orchestra.run.smithi150.stdout: "ceph_cap_op_trunc": 0, 2024-09-21T03:56:41.762 INFO:teuthology.orchestra.run.smithi150.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-21T03:56:41.762 INFO:teuthology.orchestra.run.smithi150.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-21T03:56:41.762 INFO:teuthology.orchestra.run.smithi150.stdout: "handle_client_caps": 3468, 2024-09-21T03:56:41.762 INFO:teuthology.orchestra.run.smithi150.stdout: "handle_client_caps_dirty": 3445, 2024-09-21T03:56:41.762 INFO:teuthology.orchestra.run.smithi150.stdout: "handle_client_cap_release": 5, 2024-09-21T03:56:41.762 INFO:teuthology.orchestra.run.smithi150.stdout: "process_request_cap_release": 24 2024-09-21T03:56:41.762 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T03:56:41.763 INFO:teuthology.orchestra.run.smithi150.stdout: "mds_cache": { 2024-09-21T03:56:41.763 INFO:teuthology.orchestra.run.smithi150.stdout: "dir_update": 16, 2024-09-21T03:56:41.763 INFO:teuthology.orchestra.run.smithi150.stdout: "dir_update_receipt": 0, 2024-09-21T03:56:41.763 INFO:teuthology.orchestra.run.smithi150.stdout: "dir_try_discover": 0, 2024-09-21T03:56:41.763 INFO:teuthology.orchestra.run.smithi150.stdout: "dir_send_discover": 0, 2024-09-21T03:56:41.763 INFO:teuthology.orchestra.run.smithi150.stdout: "dir_handle_discover": 32, 2024-09-21T03:56:41.763 INFO:teuthology.orchestra.run.smithi150.stdout: "num_strays": 2, 2024-09-21T03:56:41.763 INFO:teuthology.orchestra.run.smithi150.stdout: "num_strays_delayed": 0, 2024-09-21T03:56:41.763 INFO:teuthology.orchestra.run.smithi150.stdout: "num_strays_enqueuing": 0, 2024-09-21T03:56:41.763 INFO:teuthology.orchestra.run.smithi150.stdout: "strays_created": 5, 2024-09-21T03:56:41.763 INFO:teuthology.orchestra.run.smithi150.stdout: "strays_enqueued": 3, 2024-09-21T03:56:41.763 INFO:teuthology.orchestra.run.smithi150.stdout: "strays_reintegrated": 0, 2024-09-21T03:56:41.764 INFO:teuthology.orchestra.run.smithi150.stdout: "strays_migrated": 0, 2024-09-21T03:56:41.764 INFO:teuthology.orchestra.run.smithi150.stdout: "num_recovering_processing": 0, 2024-09-21T03:56:41.764 INFO:teuthology.orchestra.run.smithi150.stdout: "num_recovering_enqueued": 0, 2024-09-21T03:56:41.764 INFO:teuthology.orchestra.run.smithi150.stdout: "num_recovering_prioritized": 0, 2024-09-21T03:56:41.764 INFO:teuthology.orchestra.run.smithi150.stdout: "recovery_started": 0, 2024-09-21T03:56:41.764 INFO:teuthology.orchestra.run.smithi150.stdout: "recovery_completed": 0, 2024-09-21T03:56:41.764 INFO:teuthology.orchestra.run.smithi150.stdout: "ireq_quiesce_path": 0, 2024-09-21T03:56:41.764 INFO:teuthology.orchestra.run.smithi150.stdout: "ireq_quiesce_inode": 0, 2024-09-21T03:56:41.764 INFO:teuthology.orchestra.run.smithi150.stdout: "ireq_enqueue_scrub": 20, 2024-09-21T03:56:41.764 INFO:teuthology.orchestra.run.smithi150.stdout: "ireq_exportdir": 0, 2024-09-21T03:56:41.764 INFO:teuthology.orchestra.run.smithi150.stdout: "ireq_flush": 0, 2024-09-21T03:56:41.764 INFO:teuthology.orchestra.run.smithi150.stdout: "ireq_fragmentdir": 0, 2024-09-21T03:56:41.765 INFO:teuthology.orchestra.run.smithi150.stdout: "ireq_fragstats": 0, 2024-09-21T03:56:41.765 INFO:teuthology.orchestra.run.smithi150.stdout: "ireq_inodestats": 0, 2024-09-21T03:56:41.765 INFO:teuthology.orchestra.run.smithi150.stdout: "uninline_started": 0, 2024-09-21T03:56:41.765 INFO:teuthology.orchestra.run.smithi150.stdout: "uninline_succeeded": 0, 2024-09-21T03:56:41.765 INFO:teuthology.orchestra.run.smithi150.stdout: "uninline_write_failed": 0 2024-09-21T03:56:41.765 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T03:56:41.765 INFO:teuthology.orchestra.run.smithi150.stdout: "mds_log": { 2024-09-21T03:56:41.765 INFO:teuthology.orchestra.run.smithi150.stdout: "evlrg": 0, 2024-09-21T03:56:41.765 INFO:teuthology.orchestra.run.smithi150.stdout: "evadd": 3577, 2024-09-21T03:56:41.765 INFO:teuthology.orchestra.run.smithi150.stdout: "evex": 0, 2024-09-21T03:56:41.765 INFO:teuthology.orchestra.run.smithi150.stdout: "evtrm": 0, 2024-09-21T03:56:41.765 INFO:teuthology.orchestra.run.smithi150.stdout: "ev": 3577, 2024-09-21T03:56:41.765 INFO:teuthology.orchestra.run.smithi150.stdout: "evexg": 0, 2024-09-21T03:56:41.766 INFO:teuthology.orchestra.run.smithi150.stdout: "evexd": 0, 2024-09-21T03:56:41.766 INFO:teuthology.orchestra.run.smithi150.stdout: "segadd": 6, 2024-09-21T03:56:41.766 INFO:teuthology.orchestra.run.smithi150.stdout: "segex": 0, 2024-09-21T03:56:41.766 INFO:teuthology.orchestra.run.smithi150.stdout: "segtrm": 0, 2024-09-21T03:56:41.766 INFO:teuthology.orchestra.run.smithi150.stdout: "seg": 6, 2024-09-21T03:56:41.766 INFO:teuthology.orchestra.run.smithi150.stdout: "segmjr": 2, 2024-09-21T03:56:41.766 INFO:teuthology.orchestra.run.smithi150.stdout: "segexg": 0, 2024-09-21T03:56:41.766 INFO:teuthology.orchestra.run.smithi150.stdout: "segexd": 0, 2024-09-21T03:56:41.766 INFO:teuthology.orchestra.run.smithi150.stdout: "expos": 4194304, 2024-09-21T03:56:41.766 INFO:teuthology.orchestra.run.smithi150.stdout: "wrpos": 13712721, 2024-09-21T03:56:41.766 INFO:teuthology.orchestra.run.smithi150.stdout: "rdpos": 0, 2024-09-21T03:56:41.766 INFO:teuthology.orchestra.run.smithi150.stdout: "jlat": { 2024-09-21T03:56:41.767 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 3547, 2024-09-21T03:56:41.767 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 305.629842015, 2024-09-21T03:56:41.767 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.086165729 2024-09-21T03:56:41.767 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T03:56:41.767 INFO:teuthology.orchestra.run.smithi150.stdout: "replayed": 0 2024-09-21T03:56:41.767 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T03:56:41.767 INFO:teuthology.orchestra.run.smithi150.stdout: "mds_mem": { 2024-09-21T03:56:41.767 INFO:teuthology.orchestra.run.smithi150.stdout: "ino": 25, 2024-09-21T03:56:41.767 INFO:teuthology.orchestra.run.smithi150.stdout: "ino+": 48, 2024-09-21T03:56:41.767 INFO:teuthology.orchestra.run.smithi150.stdout: "ino-": 23, 2024-09-21T03:56:41.767 INFO:teuthology.orchestra.run.smithi150.stdout: "dir": 20, 2024-09-21T03:56:41.767 INFO:teuthology.orchestra.run.smithi150.stdout: "dir+": 20, 2024-09-21T03:56:41.768 INFO:teuthology.orchestra.run.smithi150.stdout: "dir-": 0, 2024-09-21T03:56:41.768 INFO:teuthology.orchestra.run.smithi150.stdout: "dn": 32, 2024-09-21T03:56:41.768 INFO:teuthology.orchestra.run.smithi150.stdout: "dn+": 35, 2024-09-21T03:56:41.768 INFO:teuthology.orchestra.run.smithi150.stdout: "dn-": 3, 2024-09-21T03:56:41.768 INFO:teuthology.orchestra.run.smithi150.stdout: "cap": 3, 2024-09-21T03:56:41.768 INFO:teuthology.orchestra.run.smithi150.stdout: "cap+": 20, 2024-09-21T03:56:41.768 INFO:teuthology.orchestra.run.smithi150.stdout: "cap-": 17, 2024-09-21T03:56:41.768 INFO:teuthology.orchestra.run.smithi150.stdout: "rss": 127664, 2024-09-21T03:56:41.768 INFO:teuthology.orchestra.run.smithi150.stdout: "heap": 343822336 2024-09-21T03:56:41.768 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T03:56:41.768 INFO:teuthology.orchestra.run.smithi150.stdout: "mds_server": { 2024-09-21T03:56:41.768 INFO:teuthology.orchestra.run.smithi150.stdout: "dispatch_client_request": 213, 2024-09-21T03:56:41.768 INFO:teuthology.orchestra.run.smithi150.stdout: "dispatch_server_request": 1, 2024-09-21T03:56:41.769 INFO:teuthology.orchestra.run.smithi150.stdout: "handle_client_request": 159, 2024-09-21T03:56:41.769 INFO:teuthology.orchestra.run.smithi150.stdout: "handle_client_session": 29, 2024-09-21T03:56:41.769 INFO:teuthology.orchestra.run.smithi150.stdout: "handle_peer_request": 2, 2024-09-21T03:56:41.769 INFO:teuthology.orchestra.run.smithi150.stdout: "req_create_latency": { 2024-09-21T03:56:41.769 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 7, 2024-09-21T03:56:41.769 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.007894668, 2024-09-21T03:56:41.769 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.001127809 2024-09-21T03:56:41.769 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T03:56:41.769 INFO:teuthology.orchestra.run.smithi150.stdout: "req_getattr_latency": { 2024-09-21T03:56:41.769 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 98, 2024-09-21T03:56:41.769 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.019058130, 2024-09-21T03:56:41.770 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000194470 2024-09-21T03:56:41.770 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T03:56:41.770 INFO:teuthology.orchestra.run.smithi150.stdout: "req_getfilelock_latency": { 2024-09-21T03:56:41.770 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-09-21T03:56:41.770 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-09-21T03:56:41.770 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-09-21T03:56:41.770 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T03:56:41.770 INFO:teuthology.orchestra.run.smithi150.stdout: "req_link_latency": { 2024-09-21T03:56:41.770 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-09-21T03:56:41.770 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-09-21T03:56:41.770 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-09-21T03:56:41.770 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T03:56:41.771 INFO:teuthology.orchestra.run.smithi150.stdout: "req_lookup_latency": { 2024-09-21T03:56:41.771 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-09-21T03:56:41.771 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-09-21T03:56:41.771 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-09-21T03:56:41.771 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T03:56:41.771 INFO:teuthology.orchestra.run.smithi150.stdout: "req_lookuphash_latency": { 2024-09-21T03:56:41.771 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-09-21T03:56:41.771 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-09-21T03:56:41.771 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-09-21T03:56:41.771 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T03:56:41.771 INFO:teuthology.orchestra.run.smithi150.stdout: "req_lookupino_latency": { 2024-09-21T03:56:41.771 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 1, 2024-09-21T03:56:41.772 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000159538, 2024-09-21T03:56:41.772 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000159538 2024-09-21T03:56:41.772 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T03:56:41.772 INFO:teuthology.orchestra.run.smithi150.stdout: "req_lookupname_latency": { 2024-09-21T03:56:41.772 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-09-21T03:56:41.772 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-09-21T03:56:41.772 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-09-21T03:56:41.772 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T03:56:41.772 INFO:teuthology.orchestra.run.smithi150.stdout: "req_lookupparent_latency": { 2024-09-21T03:56:41.772 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-09-21T03:56:41.772 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-09-21T03:56:41.772 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-09-21T03:56:41.772 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T03:56:41.773 INFO:teuthology.orchestra.run.smithi150.stdout: "req_lookupsnap_latency": { 2024-09-21T03:56:41.773 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-09-21T03:56:41.773 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-09-21T03:56:41.773 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-09-21T03:56:41.773 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T03:56:41.774 INFO:teuthology.orchestra.run.smithi150.stdout: "req_lssnap_latency": { 2024-09-21T03:56:41.774 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 10, 2024-09-21T03:56:41.774 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.001523665, 2024-09-21T03:56:41.774 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000152366 2024-09-21T03:56:41.774 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T03:56:41.774 INFO:teuthology.orchestra.run.smithi150.stdout: "req_mkdir_latency": { 2024-09-21T03:56:41.774 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 8, 2024-09-21T03:56:41.774 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.013647040, 2024-09-21T03:56:41.774 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.001705880 2024-09-21T03:56:41.774 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T03:56:41.775 INFO:teuthology.orchestra.run.smithi150.stdout: "req_mknod_latency": { 2024-09-21T03:56:41.775 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-09-21T03:56:41.775 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-09-21T03:56:41.775 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-09-21T03:56:41.775 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T03:56:41.775 INFO:teuthology.orchestra.run.smithi150.stdout: "req_mksnap_latency": { 2024-09-21T03:56:41.775 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 10, 2024-09-21T03:56:41.775 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.060054406, 2024-09-21T03:56:41.775 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.006005440 2024-09-21T03:56:41.775 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T03:56:41.775 INFO:teuthology.orchestra.run.smithi150.stdout: "req_open_latency": { 2024-09-21T03:56:41.775 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 1, 2024-09-21T03:56:41.776 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000448793, 2024-09-21T03:56:41.776 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000448793 2024-09-21T03:56:41.776 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T03:56:41.776 INFO:teuthology.orchestra.run.smithi150.stdout: "req_readdir_latency": { 2024-09-21T03:56:41.776 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 2, 2024-09-21T03:56:41.776 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.007342136, 2024-09-21T03:56:41.776 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.003671068 2024-09-21T03:56:41.776 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T03:56:41.776 INFO:teuthology.orchestra.run.smithi150.stdout: "req_rename_latency": { 2024-09-21T03:56:41.776 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 2, 2024-09-21T03:56:41.776 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.002778799, 2024-09-21T03:56:41.777 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.001389399 2024-09-21T03:56:41.777 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T03:56:41.777 INFO:teuthology.orchestra.run.smithi150.stdout: "req_renamesnap_latency": { 2024-09-21T03:56:41.777 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-09-21T03:56:41.777 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-09-21T03:56:41.777 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-09-21T03:56:41.777 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T03:56:41.777 INFO:teuthology.orchestra.run.smithi150.stdout: "req_snapdiff_latency": { 2024-09-21T03:56:41.777 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-09-21T03:56:41.777 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-09-21T03:56:41.777 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-09-21T03:56:41.777 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T03:56:41.778 INFO:teuthology.orchestra.run.smithi150.stdout: "req_rmdir_latency": { 2024-09-21T03:56:41.778 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 2, 2024-09-21T03:56:41.778 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.009141565, 2024-09-21T03:56:41.778 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.004570782 2024-09-21T03:56:41.778 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T03:56:41.778 INFO:teuthology.orchestra.run.smithi150.stdout: "req_rmsnap_latency": { 2024-09-21T03:56:41.778 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 4, 2024-09-21T03:56:41.778 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.019775574, 2024-09-21T03:56:41.778 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.004943893 2024-09-21T03:56:41.778 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T03:56:41.778 INFO:teuthology.orchestra.run.smithi150.stdout: "req_rmxattr_latency": { 2024-09-21T03:56:41.778 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-09-21T03:56:41.779 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-09-21T03:56:41.779 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-09-21T03:56:41.779 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T03:56:41.779 INFO:teuthology.orchestra.run.smithi150.stdout: "req_setattr_latency": { 2024-09-21T03:56:41.779 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 1, 2024-09-21T03:56:41.779 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000901914, 2024-09-21T03:56:41.779 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000901914 2024-09-21T03:56:41.779 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T03:56:41.779 INFO:teuthology.orchestra.run.smithi150.stdout: "req_setdirlayout_latency": { 2024-09-21T03:56:41.779 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-09-21T03:56:41.779 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-09-21T03:56:41.779 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-09-21T03:56:41.780 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T03:56:41.780 INFO:teuthology.orchestra.run.smithi150.stdout: "req_setfilelock_latency": { 2024-09-21T03:56:41.780 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-09-21T03:56:41.780 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-09-21T03:56:41.780 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-09-21T03:56:41.780 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T03:56:41.780 INFO:teuthology.orchestra.run.smithi150.stdout: "req_setlayout_latency": { 2024-09-21T03:56:41.780 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-09-21T03:56:41.780 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-09-21T03:56:41.780 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-09-21T03:56:41.780 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T03:56:41.781 INFO:teuthology.orchestra.run.smithi150.stdout: "req_setxattr_latency": { 2024-09-21T03:56:41.781 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 10, 2024-09-21T03:56:41.781 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.024339023, 2024-09-21T03:56:41.781 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.002433902 2024-09-21T03:56:41.781 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T03:56:41.781 INFO:teuthology.orchestra.run.smithi150.stdout: "req_symlink_latency": { 2024-09-21T03:56:41.781 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-09-21T03:56:41.781 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-09-21T03:56:41.781 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-09-21T03:56:41.781 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T03:56:41.781 INFO:teuthology.orchestra.run.smithi150.stdout: "req_unlink_latency": { 2024-09-21T03:56:41.781 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 3, 2024-09-21T03:56:41.782 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.007020647, 2024-09-21T03:56:41.782 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.002340215 2024-09-21T03:56:41.782 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T03:56:41.782 INFO:teuthology.orchestra.run.smithi150.stdout: "cap_revoke_eviction": 0, 2024-09-21T03:56:41.782 INFO:teuthology.orchestra.run.smithi150.stdout: "cap_acquisition_throttle": 0, 2024-09-21T03:56:41.782 INFO:teuthology.orchestra.run.smithi150.stdout: "req_getvxattr_latency": { 2024-09-21T03:56:41.782 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-09-21T03:56:41.782 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-09-21T03:56:41.782 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-09-21T03:56:41.782 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T03:56:41.782 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T03:56:41.783 INFO:teuthology.orchestra.run.smithi150.stdout: "mds_sessions": { 2024-09-21T03:56:41.783 INFO:teuthology.orchestra.run.smithi150.stdout: "session_count": 2, 2024-09-21T03:56:41.783 INFO:teuthology.orchestra.run.smithi150.stdout: "session_add": 4, 2024-09-21T03:56:41.783 INFO:teuthology.orchestra.run.smithi150.stdout: "session_remove": 2, 2024-09-21T03:56:41.783 INFO:teuthology.orchestra.run.smithi150.stdout: "sessions_open": 2, 2024-09-21T03:56:41.783 INFO:teuthology.orchestra.run.smithi150.stdout: "sessions_stale": 0, 2024-09-21T03:56:41.783 INFO:teuthology.orchestra.run.smithi150.stdout: "total_load": 27, 2024-09-21T03:56:41.783 INFO:teuthology.orchestra.run.smithi150.stdout: "average_load": 13, 2024-09-21T03:56:41.783 INFO:teuthology.orchestra.run.smithi150.stdout: "avg_session_uptime": 605, 2024-09-21T03:56:41.783 INFO:teuthology.orchestra.run.smithi150.stdout: "mdthresh_evicted": 0 2024-09-21T03:56:41.783 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T03:56:41.783 INFO:teuthology.orchestra.run.smithi150.stdout: "mempool": { 2024-09-21T03:56:41.784 INFO:teuthology.orchestra.run.smithi150.stdout: "bloom_filter_bytes": 0, 2024-09-21T03:56:41.784 INFO:teuthology.orchestra.run.smithi150.stdout: "bloom_filter_items": 0, 2024-09-21T03:56:41.784 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_alloc_bytes": 0, 2024-09-21T03:56:41.784 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_alloc_items": 0, 2024-09-21T03:56:41.784 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_cache_data_bytes": 0, 2024-09-21T03:56:41.784 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_cache_data_items": 0, 2024-09-21T03:56:41.784 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-21T03:56:41.784 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_cache_onode_items": 0, 2024-09-21T03:56:41.784 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-21T03:56:41.784 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_cache_meta_items": 0, 2024-09-21T03:56:41.784 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_cache_other_bytes": 0, 2024-09-21T03:56:41.784 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_cache_other_items": 0, 2024-09-21T03:56:41.785 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-21T03:56:41.785 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_cache_buffer_items": 0, 2024-09-21T03:56:41.785 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_extent_bytes": 0, 2024-09-21T03:56:41.785 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_extent_items": 0, 2024-09-21T03:56:41.785 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_blob_bytes": 0, 2024-09-21T03:56:41.785 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_blob_items": 0, 2024-09-21T03:56:41.785 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-21T03:56:41.785 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_shared_blob_items": 0, 2024-09-21T03:56:41.785 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-21T03:56:41.785 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_inline_bl_items": 0, 2024-09-21T03:56:41.785 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_fsck_bytes": 0, 2024-09-21T03:56:41.785 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_fsck_items": 0, 2024-09-21T03:56:41.786 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_txc_bytes": 0, 2024-09-21T03:56:41.786 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_txc_items": 0, 2024-09-21T03:56:41.786 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-21T03:56:41.786 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_writing_deferred_items": 0, 2024-09-21T03:56:41.786 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_writing_bytes": 0, 2024-09-21T03:56:41.786 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_writing_items": 0, 2024-09-21T03:56:41.786 INFO:teuthology.orchestra.run.smithi150.stdout: "bluefs_bytes": 0, 2024-09-21T03:56:41.786 INFO:teuthology.orchestra.run.smithi150.stdout: "bluefs_items": 0, 2024-09-21T03:56:41.786 INFO:teuthology.orchestra.run.smithi150.stdout: "bluefs_file_reader_bytes": 0, 2024-09-21T03:56:41.786 INFO:teuthology.orchestra.run.smithi150.stdout: "bluefs_file_reader_items": 0, 2024-09-21T03:56:41.786 INFO:teuthology.orchestra.run.smithi150.stdout: "bluefs_file_writer_bytes": 0, 2024-09-21T03:56:41.786 INFO:teuthology.orchestra.run.smithi150.stdout: "bluefs_file_writer_items": 0, 2024-09-21T03:56:41.787 INFO:teuthology.orchestra.run.smithi150.stdout: "buffer_anon_bytes": 350553, 2024-09-21T03:56:41.787 INFO:teuthology.orchestra.run.smithi150.stdout: "buffer_anon_items": 143, 2024-09-21T03:56:41.787 INFO:teuthology.orchestra.run.smithi150.stdout: "buffer_meta_bytes": 616, 2024-09-21T03:56:41.787 INFO:teuthology.orchestra.run.smithi150.stdout: "buffer_meta_items": 7, 2024-09-21T03:56:41.787 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_bytes": 0, 2024-09-21T03:56:41.787 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_items": 0, 2024-09-21T03:56:41.787 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_mapbl_bytes": 0, 2024-09-21T03:56:41.787 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_mapbl_items": 0, 2024-09-21T03:56:41.787 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_pglog_bytes": 0, 2024-09-21T03:56:41.787 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_pglog_items": 0, 2024-09-21T03:56:41.787 INFO:teuthology.orchestra.run.smithi150.stdout: "osdmap_bytes": 8256, 2024-09-21T03:56:41.787 INFO:teuthology.orchestra.run.smithi150.stdout: "osdmap_items": 182, 2024-09-21T03:56:41.788 INFO:teuthology.orchestra.run.smithi150.stdout: "osdmap_mapping_bytes": 0, 2024-09-21T03:56:41.788 INFO:teuthology.orchestra.run.smithi150.stdout: "osdmap_mapping_items": 0, 2024-09-21T03:56:41.788 INFO:teuthology.orchestra.run.smithi150.stdout: "pgmap_bytes": 0, 2024-09-21T03:56:41.788 INFO:teuthology.orchestra.run.smithi150.stdout: "pgmap_items": 0, 2024-09-21T03:56:41.788 INFO:teuthology.orchestra.run.smithi150.stdout: "mds_co_bytes": 125681, 2024-09-21T03:56:41.788 INFO:teuthology.orchestra.run.smithi150.stdout: "mds_co_items": 1189, 2024-09-21T03:56:41.788 INFO:teuthology.orchestra.run.smithi150.stdout: "unittest_1_bytes": 0, 2024-09-21T03:56:41.788 INFO:teuthology.orchestra.run.smithi150.stdout: "unittest_1_items": 0, 2024-09-21T03:56:41.788 INFO:teuthology.orchestra.run.smithi150.stdout: "unittest_2_bytes": 0, 2024-09-21T03:56:41.788 INFO:teuthology.orchestra.run.smithi150.stdout: "unittest_2_items": 0 2024-09-21T03:56:41.788 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T03:56:41.788 INFO:teuthology.orchestra.run.smithi150.stdout: "objecter": { 2024-09-21T03:56:41.788 INFO:teuthology.orchestra.run.smithi150.stdout: "op_active": 0, 2024-09-21T03:56:41.789 INFO:teuthology.orchestra.run.smithi150.stdout: "op_laggy": 0, 2024-09-21T03:56:41.789 INFO:teuthology.orchestra.run.smithi150.stdout: "op_send": 8009, 2024-09-21T03:56:41.789 INFO:teuthology.orchestra.run.smithi150.stdout: "op_send_bytes": 9538331, 2024-09-21T03:56:41.789 INFO:teuthology.orchestra.run.smithi150.stdout: "op_resend": 32, 2024-09-21T03:56:41.789 INFO:teuthology.orchestra.run.smithi150.stdout: "op_reply": 8009, 2024-09-21T03:56:41.789 INFO:teuthology.orchestra.run.smithi150.stdout: "op_latency": { 2024-09-21T03:56:41.789 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 8009, 2024-09-21T03:56:41.789 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 361.001559917, 2024-09-21T03:56:41.789 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.045074486 2024-09-21T03:56:41.789 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T03:56:41.789 INFO:teuthology.orchestra.run.smithi150.stdout: "op_inflight": 0, 2024-09-21T03:56:41.789 INFO:teuthology.orchestra.run.smithi150.stdout: "oplen_avg": { 2024-09-21T03:56:41.790 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 8009, 2024-09-21T03:56:41.790 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 8064 2024-09-21T03:56:41.790 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T03:56:41.790 INFO:teuthology.orchestra.run.smithi150.stdout: "op": 8009, 2024-09-21T03:56:41.790 INFO:teuthology.orchestra.run.smithi150.stdout: "op_r": 271, 2024-09-21T03:56:41.790 INFO:teuthology.orchestra.run.smithi150.stdout: "op_w": 7738, 2024-09-21T03:56:41.790 INFO:teuthology.orchestra.run.smithi150.stdout: "op_rmw": 0, 2024-09-21T03:56:41.790 INFO:teuthology.orchestra.run.smithi150.stdout: "op_pg": 0, 2024-09-21T03:56:41.790 INFO:teuthology.orchestra.run.smithi150.stdout: "osdop_stat": 0, 2024-09-21T03:56:41.790 INFO:teuthology.orchestra.run.smithi150.stdout: "osdop_create": 11, 2024-09-21T03:56:41.790 INFO:teuthology.orchestra.run.smithi150.stdout: "osdop_read": 23, 2024-09-21T03:56:41.790 INFO:teuthology.orchestra.run.smithi150.stdout: "osdop_write": 3550, 2024-09-21T03:56:41.791 INFO:teuthology.orchestra.run.smithi150.stdout: "osdop_writefull": 30, 2024-09-21T03:56:41.791 INFO:teuthology.orchestra.run.smithi150.stdout: "osdop_writesame": 0, 2024-09-21T03:56:41.791 INFO:teuthology.orchestra.run.smithi150.stdout: "osdop_append": 0, 2024-09-21T03:56:41.791 INFO:teuthology.orchestra.run.smithi150.stdout: "osdop_zero": 0, 2024-09-21T03:56:41.791 INFO:teuthology.orchestra.run.smithi150.stdout: "osdop_truncate": 0, 2024-09-21T03:56:41.791 INFO:teuthology.orchestra.run.smithi150.stdout: "osdop_delete": 4100, 2024-09-21T03:56:41.791 INFO:teuthology.orchestra.run.smithi150.stdout: "osdop_mapext": 0, 2024-09-21T03:56:41.791 INFO:teuthology.orchestra.run.smithi150.stdout: "osdop_sparse_read": 0, 2024-09-21T03:56:41.791 INFO:teuthology.orchestra.run.smithi150.stdout: "osdop_clonerange": 0, 2024-09-21T03:56:41.791 INFO:teuthology.orchestra.run.smithi150.stdout: "osdop_getxattr": 248, 2024-09-21T03:56:41.791 INFO:teuthology.orchestra.run.smithi150.stdout: "osdop_setxattr": 22, 2024-09-21T03:56:41.791 INFO:teuthology.orchestra.run.smithi150.stdout: "osdop_cmpxattr": 0, 2024-09-21T03:56:41.792 INFO:teuthology.orchestra.run.smithi150.stdout: "osdop_rmxattr": 0, 2024-09-21T03:56:41.792 INFO:teuthology.orchestra.run.smithi150.stdout: "osdop_resetxattrs": 0, 2024-09-21T03:56:41.792 INFO:teuthology.orchestra.run.smithi150.stdout: "osdop_call": 0, 2024-09-21T03:56:41.792 INFO:teuthology.orchestra.run.smithi150.stdout: "osdop_watch": 0, 2024-09-21T03:56:41.792 INFO:teuthology.orchestra.run.smithi150.stdout: "osdop_notify": 0, 2024-09-21T03:56:41.792 INFO:teuthology.orchestra.run.smithi150.stdout: "osdop_src_cmpxattr": 0, 2024-09-21T03:56:41.792 INFO:teuthology.orchestra.run.smithi150.stdout: "osdop_pgls": 0, 2024-09-21T03:56:41.792 INFO:teuthology.orchestra.run.smithi150.stdout: "osdop_pgls_filter": 0, 2024-09-21T03:56:41.792 INFO:teuthology.orchestra.run.smithi150.stdout: "osdop_other": 0, 2024-09-21T03:56:41.792 INFO:teuthology.orchestra.run.smithi150.stdout: "linger_active": 0, 2024-09-21T03:56:41.792 INFO:teuthology.orchestra.run.smithi150.stdout: "linger_send": 0, 2024-09-21T03:56:41.792 INFO:teuthology.orchestra.run.smithi150.stdout: "linger_resend": 0, 2024-09-21T03:56:41.792 INFO:teuthology.orchestra.run.smithi150.stdout: "linger_ping": 0, 2024-09-21T03:56:41.793 INFO:teuthology.orchestra.run.smithi150.stdout: "poolop_active": 0, 2024-09-21T03:56:41.793 INFO:teuthology.orchestra.run.smithi150.stdout: "poolop_send": 0, 2024-09-21T03:56:41.793 INFO:teuthology.orchestra.run.smithi150.stdout: "poolop_resend": 0, 2024-09-21T03:56:41.793 INFO:teuthology.orchestra.run.smithi150.stdout: "poolstat_active": 0, 2024-09-21T03:56:41.793 INFO:teuthology.orchestra.run.smithi150.stdout: "poolstat_send": 0, 2024-09-21T03:56:41.793 INFO:teuthology.orchestra.run.smithi150.stdout: "poolstat_resend": 0, 2024-09-21T03:56:41.793 INFO:teuthology.orchestra.run.smithi150.stdout: "statfs_active": 0, 2024-09-21T03:56:41.793 INFO:teuthology.orchestra.run.smithi150.stdout: "statfs_send": 0, 2024-09-21T03:56:41.793 INFO:teuthology.orchestra.run.smithi150.stdout: "statfs_resend": 0, 2024-09-21T03:56:41.793 INFO:teuthology.orchestra.run.smithi150.stdout: "command_active": 0, 2024-09-21T03:56:41.793 INFO:teuthology.orchestra.run.smithi150.stdout: "command_send": 0, 2024-09-21T03:56:41.793 INFO:teuthology.orchestra.run.smithi150.stdout: "command_resend": 0, 2024-09-21T03:56:41.794 INFO:teuthology.orchestra.run.smithi150.stdout: "map_epoch": 87, 2024-09-21T03:56:41.794 INFO:teuthology.orchestra.run.smithi150.stdout: "map_full": 0, 2024-09-21T03:56:41.794 INFO:teuthology.orchestra.run.smithi150.stdout: "map_inc": 10, 2024-09-21T03:56:41.794 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_sessions": 12, 2024-09-21T03:56:41.794 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_session_open": 12, 2024-09-21T03:56:41.794 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_session_close": 0, 2024-09-21T03:56:41.794 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_laggy": 0, 2024-09-21T03:56:41.794 INFO:teuthology.orchestra.run.smithi150.stdout: "omap_wr": 67, 2024-09-21T03:56:41.794 INFO:teuthology.orchestra.run.smithi150.stdout: "omap_rd": 0, 2024-09-21T03:56:41.794 INFO:teuthology.orchestra.run.smithi150.stdout: "omap_del": 13 2024-09-21T03:56:41.794 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T03:56:41.794 INFO:teuthology.orchestra.run.smithi150.stdout: "oft": { 2024-09-21T03:56:41.795 INFO:teuthology.orchestra.run.smithi150.stdout: "omap_total_objs": 1, 2024-09-21T03:56:41.795 INFO:teuthology.orchestra.run.smithi150.stdout: "omap_total_kv_pairs": 0, 2024-09-21T03:56:41.795 INFO:teuthology.orchestra.run.smithi150.stdout: "omap_total_updates": 0, 2024-09-21T03:56:41.795 INFO:teuthology.orchestra.run.smithi150.stdout: "omap_total_removes": 0 2024-09-21T03:56:41.795 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T03:56:41.795 INFO:teuthology.orchestra.run.smithi150.stdout: "purge_queue": { 2024-09-21T03:56:41.795 INFO:teuthology.orchestra.run.smithi150.stdout: "pq_executing_ops": 0, 2024-09-21T03:56:41.795 INFO:teuthology.orchestra.run.smithi150.stdout: "pq_executing_ops_high_water": 2560, 2024-09-21T03:56:41.795 INFO:teuthology.orchestra.run.smithi150.stdout: "pq_executing": 0, 2024-09-21T03:56:41.795 INFO:teuthology.orchestra.run.smithi150.stdout: "pq_executing_high_water": 1, 2024-09-21T03:56:41.795 INFO:teuthology.orchestra.run.smithi150.stdout: "pq_executed_ops": 3586, 2024-09-21T03:56:41.795 INFO:teuthology.orchestra.run.smithi150.stdout: "pq_executed": 3, 2024-09-21T03:56:41.795 INFO:teuthology.orchestra.run.smithi150.stdout: "pq_item_in_journal": 0 2024-09-21T03:56:41.796 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T03:56:41.796 INFO:teuthology.orchestra.run.smithi150.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-21T03:56:41.796 INFO:teuthology.orchestra.run.smithi150.stdout: "val": 0, 2024-09-21T03:56:41.796 INFO:teuthology.orchestra.run.smithi150.stdout: "max": 104857600, 2024-09-21T03:56:41.796 INFO:teuthology.orchestra.run.smithi150.stdout: "get_started": 0, 2024-09-21T03:56:41.796 INFO:teuthology.orchestra.run.smithi150.stdout: "get": 15553, 2024-09-21T03:56:41.796 INFO:teuthology.orchestra.run.smithi150.stdout: "get_sum": 4524796, 2024-09-21T03:56:41.796 INFO:teuthology.orchestra.run.smithi150.stdout: "get_or_fail_fail": 0, 2024-09-21T03:56:41.796 INFO:teuthology.orchestra.run.smithi150.stdout: "get_or_fail_success": 15553, 2024-09-21T03:56:41.796 INFO:teuthology.orchestra.run.smithi150.stdout: "take": 0, 2024-09-21T03:56:41.796 INFO:teuthology.orchestra.run.smithi150.stdout: "take_sum": 0, 2024-09-21T03:56:41.796 INFO:teuthology.orchestra.run.smithi150.stdout: "put": 15553, 2024-09-21T03:56:41.797 INFO:teuthology.orchestra.run.smithi150.stdout: "put_sum": 4524796, 2024-09-21T03:56:41.797 INFO:teuthology.orchestra.run.smithi150.stdout: "wait": { 2024-09-21T03:56:41.797 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-09-21T03:56:41.797 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-09-21T03:56:41.797 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-09-21T03:56:41.797 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T03:56:41.797 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T03:56:41.797 INFO:teuthology.orchestra.run.smithi150.stdout: "throttle-objecter_bytes": { 2024-09-21T03:56:41.797 INFO:teuthology.orchestra.run.smithi150.stdout: "val": 0, 2024-09-21T03:56:41.797 INFO:teuthology.orchestra.run.smithi150.stdout: "max": 104857600, 2024-09-21T03:56:41.797 INFO:teuthology.orchestra.run.smithi150.stdout: "get_started": 0, 2024-09-21T03:56:41.797 INFO:teuthology.orchestra.run.smithi150.stdout: "get": 0, 2024-09-21T03:56:41.797 INFO:teuthology.orchestra.run.smithi150.stdout: "get_sum": 0, 2024-09-21T03:56:41.798 INFO:teuthology.orchestra.run.smithi150.stdout: "get_or_fail_fail": 0, 2024-09-21T03:56:41.798 INFO:teuthology.orchestra.run.smithi150.stdout: "get_or_fail_success": 0, 2024-09-21T03:56:41.798 INFO:teuthology.orchestra.run.smithi150.stdout: "take": 8009, 2024-09-21T03:56:41.798 INFO:teuthology.orchestra.run.smithi150.stdout: "take_sum": 9538634, 2024-09-21T03:56:41.798 INFO:teuthology.orchestra.run.smithi150.stdout: "put": 8009, 2024-09-21T03:56:41.798 INFO:teuthology.orchestra.run.smithi150.stdout: "put_sum": 9538634, 2024-09-21T03:56:41.798 INFO:teuthology.orchestra.run.smithi150.stdout: "wait": { 2024-09-21T03:56:41.798 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-09-21T03:56:41.798 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-09-21T03:56:41.798 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-09-21T03:56:41.798 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T03:56:41.799 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T03:56:41.799 INFO:teuthology.orchestra.run.smithi150.stdout: "throttle-objecter_ops": { 2024-09-21T03:56:41.799 INFO:teuthology.orchestra.run.smithi150.stdout: "val": 0, 2024-09-21T03:56:41.799 INFO:teuthology.orchestra.run.smithi150.stdout: "max": 1024, 2024-09-21T03:56:41.799 INFO:teuthology.orchestra.run.smithi150.stdout: "get_started": 0, 2024-09-21T03:56:41.799 INFO:teuthology.orchestra.run.smithi150.stdout: "get": 0, 2024-09-21T03:56:41.799 INFO:teuthology.orchestra.run.smithi150.stdout: "get_sum": 0, 2024-09-21T03:56:41.799 INFO:teuthology.orchestra.run.smithi150.stdout: "get_or_fail_fail": 0, 2024-09-21T03:56:41.799 INFO:teuthology.orchestra.run.smithi150.stdout: "get_or_fail_success": 0, 2024-09-21T03:56:41.799 INFO:teuthology.orchestra.run.smithi150.stdout: "take": 8009, 2024-09-21T03:56:41.799 INFO:teuthology.orchestra.run.smithi150.stdout: "take_sum": 8009, 2024-09-21T03:56:41.799 INFO:teuthology.orchestra.run.smithi150.stdout: "put": 8009, 2024-09-21T03:56:41.799 INFO:teuthology.orchestra.run.smithi150.stdout: "put_sum": 8009, 2024-09-21T03:56:41.800 INFO:teuthology.orchestra.run.smithi150.stdout: "wait": { 2024-09-21T03:56:41.800 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-09-21T03:56:41.800 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-09-21T03:56:41.800 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-09-21T03:56:41.800 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T03:56:41.800 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T03:56:41.800 INFO:teuthology.orchestra.run.smithi150.stdout: "throttle-write_buf_throttle": { 2024-09-21T03:56:41.800 INFO:teuthology.orchestra.run.smithi150.stdout: "val": 0, 2024-09-21T03:56:41.800 INFO:teuthology.orchestra.run.smithi150.stdout: "max": 3758096384, 2024-09-21T03:56:41.800 INFO:teuthology.orchestra.run.smithi150.stdout: "get_started": 0, 2024-09-21T03:56:41.800 INFO:teuthology.orchestra.run.smithi150.stdout: "get": 3, 2024-09-21T03:56:41.800 INFO:teuthology.orchestra.run.smithi150.stdout: "get_sum": 303, 2024-09-21T03:56:41.801 INFO:teuthology.orchestra.run.smithi150.stdout: "get_or_fail_fail": 0, 2024-09-21T03:56:41.801 INFO:teuthology.orchestra.run.smithi150.stdout: "get_or_fail_success": 3, 2024-09-21T03:56:41.801 INFO:teuthology.orchestra.run.smithi150.stdout: "take": 0, 2024-09-21T03:56:41.801 INFO:teuthology.orchestra.run.smithi150.stdout: "take_sum": 0, 2024-09-21T03:56:41.801 INFO:teuthology.orchestra.run.smithi150.stdout: "put": 3, 2024-09-21T03:56:41.801 INFO:teuthology.orchestra.run.smithi150.stdout: "put_sum": 303, 2024-09-21T03:56:41.801 INFO:teuthology.orchestra.run.smithi150.stdout: "wait": { 2024-09-21T03:56:41.801 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-09-21T03:56:41.801 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-09-21T03:56:41.801 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-09-21T03:56:41.801 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T03:56:41.802 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T03:56:41.802 INFO:teuthology.orchestra.run.smithi150.stdout: "throttle-write_buf_throttle-0x55ab80e5f6c0": { 2024-09-21T03:56:41.802 INFO:teuthology.orchestra.run.smithi150.stdout: "val": 0, 2024-09-21T03:56:41.802 INFO:teuthology.orchestra.run.smithi150.stdout: "max": 3758096384, 2024-09-21T03:56:41.802 INFO:teuthology.orchestra.run.smithi150.stdout: "get_started": 0, 2024-09-21T03:56:41.802 INFO:teuthology.orchestra.run.smithi150.stdout: "get": 3577, 2024-09-21T03:56:41.802 INFO:teuthology.orchestra.run.smithi150.stdout: "get_sum": 9518417, 2024-09-21T03:56:41.802 INFO:teuthology.orchestra.run.smithi150.stdout: "get_or_fail_fail": 0, 2024-09-21T03:56:41.802 INFO:teuthology.orchestra.run.smithi150.stdout: "get_or_fail_success": 3577, 2024-09-21T03:56:41.802 INFO:teuthology.orchestra.run.smithi150.stdout: "take": 0, 2024-09-21T03:56:41.802 INFO:teuthology.orchestra.run.smithi150.stdout: "take_sum": 0, 2024-09-21T03:56:41.802 INFO:teuthology.orchestra.run.smithi150.stdout: "put": 3547, 2024-09-21T03:56:41.802 INFO:teuthology.orchestra.run.smithi150.stdout: "put_sum": 9518417, 2024-09-21T03:56:41.803 INFO:teuthology.orchestra.run.smithi150.stdout: "wait": { 2024-09-21T03:56:41.803 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-09-21T03:56:41.803 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-09-21T03:56:41.803 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-09-21T03:56:41.803 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T03:56:41.803 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T03:56:41.803 INFO:teuthology.orchestra.run.smithi150.stdout: "trackedop": { 2024-09-21T03:56:41.803 INFO:teuthology.orchestra.run.smithi150.stdout: "slow_ops_count": 0 2024-09-21T03:56:41.803 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T03:56:41.803 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-21T03:56:41.834 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:56:41 smithi039 ceph-mon[31211]: pgmap v652: 97 pgs: 97 active+clean; 10 MiB data, 525 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-09-21T03:56:42.777 INFO:tasks.check_counter:Daemon mds.b mds.exported=0 2024-09-21T03:56:42.777 INFO:tasks.check_counter:Daemon mds.b mds.imported=0 2024-09-21T03:56:42.777 INFO:tasks.check_counter:Daemon mds.b mds_cache.dir_update=16 2024-09-21T03:56:42.777 INFO:tasks.check_counter:Daemon mds.b mds_cache.dir_update_receipt=0 2024-09-21T03:56:42.777 INFO:tasks.check_counter:Daemon mds.b mds.root_rsnaps=6 2024-09-21T03:56:42.777 INFO:tasks.check_counter:Daemon mds.b mds_server.req_mksnap_latency.avgcount=10 2024-09-21T03:56:42.777 INFO:tasks.check_counter:Daemon mds.b mds_server.req_rmsnap_latency.avgcount=4 2024-09-21T03:56:42.777 DEBUG:tasks.check_counter:Getting stats from e 2024-09-21T03:56:42.777 DEBUG:teuthology.orchestra.run.smithi150:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 51116d5c-77ca-11ef-baf5-efdc52797490 -- ceph daemon mds.e perf dump 2024-09-21T03:56:43.035 INFO:teuthology.orchestra.run.smithi150.stderr:Inferring config /var/lib/ceph/51116d5c-77ca-11ef-baf5-efdc52797490/mon.b/config 2024-09-21T03:56:43.525 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:56:43 smithi150 ceph-mon[34289]: pgmap v653: 97 pgs: 97 active+clean; 10 MiB data, 525 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-21T03:56:43.526 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-21T03:56:43.526 INFO:teuthology.orchestra.run.smithi150.stdout: "AsyncMessenger::Worker-0": { 2024-09-21T03:56:43.526 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_recv_messages": 978, 2024-09-21T03:56:43.526 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_send_messages": 964, 2024-09-21T03:56:43.526 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_recv_bytes": 1449888, 2024-09-21T03:56:43.526 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_send_bytes": 366268, 2024-09-21T03:56:43.526 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_created_connections": 5, 2024-09-21T03:56:43.526 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_active_connections": 1, 2024-09-21T03:56:43.526 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_running_total_time": 0.157132059, 2024-09-21T03:56:43.526 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_running_send_time": 0.077470809, 2024-09-21T03:56:43.526 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_running_recv_time": 0.061572248, 2024-09-21T03:56:43.527 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_running_fast_dispatch_time": 0.022291938, 2024-09-21T03:56:43.527 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T03:56:43.527 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 964, 2024-09-21T03:56:43.527 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.064890934, 2024-09-21T03:56:43.527 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000067314 2024-09-21T03:56:43.527 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T03:56:43.527 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_handle_ack_lat": { 2024-09-21T03:56:43.527 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-09-21T03:56:43.527 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-09-21T03:56:43.527 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-09-21T03:56:43.527 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T03:56:43.528 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_recv_encrypted_bytes": 1201920, 2024-09-21T03:56:43.528 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_send_encrypted_bytes": 131584 2024-09-21T03:56:43.528 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T03:56:43.528 INFO:teuthology.orchestra.run.smithi150.stdout: "AsyncMessenger::Worker-1": { 2024-09-21T03:56:43.528 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_recv_messages": 3, 2024-09-21T03:56:43.528 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_send_messages": 161, 2024-09-21T03:56:43.528 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_recv_bytes": 716, 2024-09-21T03:56:43.528 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_send_bytes": 362522, 2024-09-21T03:56:43.528 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_created_connections": 4, 2024-09-21T03:56:43.528 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_active_connections": 3, 2024-09-21T03:56:43.528 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_running_total_time": 0.012504975, 2024-09-21T03:56:43.528 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_running_send_time": 0.009274707, 2024-09-21T03:56:43.529 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_running_recv_time": 0.001554399, 2024-09-21T03:56:43.529 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_running_fast_dispatch_time": 0.000073008, 2024-09-21T03:56:43.529 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T03:56:43.529 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 159, 2024-09-21T03:56:43.529 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.010439804, 2024-09-21T03:56:43.529 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000065659 2024-09-21T03:56:43.529 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T03:56:43.529 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_handle_ack_lat": { 2024-09-21T03:56:43.529 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-09-21T03:56:43.529 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-09-21T03:56:43.529 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-09-21T03:56:43.529 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T03:56:43.530 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_recv_encrypted_bytes": 160, 2024-09-21T03:56:43.530 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_send_encrypted_bytes": 361824 2024-09-21T03:56:43.530 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T03:56:43.530 INFO:teuthology.orchestra.run.smithi150.stdout: "AsyncMessenger::Worker-2": { 2024-09-21T03:56:43.530 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_recv_messages": 1392, 2024-09-21T03:56:43.530 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_send_messages": 1394, 2024-09-21T03:56:43.530 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_recv_bytes": 356537, 2024-09-21T03:56:43.530 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_send_bytes": 442663, 2024-09-21T03:56:43.530 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_created_connections": 4, 2024-09-21T03:56:43.530 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_active_connections": 3, 2024-09-21T03:56:43.530 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_running_total_time": 0.114349057, 2024-09-21T03:56:43.530 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_running_send_time": 0.042491854, 2024-09-21T03:56:43.530 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_running_recv_time": 0.027257630, 2024-09-21T03:56:43.530 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_running_fast_dispatch_time": 0.038237065, 2024-09-21T03:56:43.531 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T03:56:43.531 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 1392, 2024-09-21T03:56:43.531 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.052730994, 2024-09-21T03:56:43.531 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000037881 2024-09-21T03:56:43.531 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T03:56:43.531 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_handle_ack_lat": { 2024-09-21T03:56:43.531 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-09-21T03:56:43.531 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-09-21T03:56:43.531 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-09-21T03:56:43.531 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T03:56:43.531 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_recv_encrypted_bytes": 0, 2024-09-21T03:56:43.531 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_send_encrypted_bytes": 0 2024-09-21T03:56:43.531 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T03:56:43.531 INFO:teuthology.orchestra.run.smithi150.stdout: "cct": { 2024-09-21T03:56:43.532 INFO:teuthology.orchestra.run.smithi150.stdout: "total_workers": 1, 2024-09-21T03:56:43.532 INFO:teuthology.orchestra.run.smithi150.stdout: "unhealthy_workers": 0 2024-09-21T03:56:43.532 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T03:56:43.532 INFO:teuthology.orchestra.run.smithi150.stdout: "finisher-MDSRank": { 2024-09-21T03:56:43.532 INFO:teuthology.orchestra.run.smithi150.stdout: "queue_len": 0, 2024-09-21T03:56:43.532 INFO:teuthology.orchestra.run.smithi150.stdout: "complete_latency": { 2024-09-21T03:56:43.532 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 3610, 2024-09-21T03:56:43.532 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.098108637, 2024-09-21T03:56:43.532 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000027176 2024-09-21T03:56:43.532 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T03:56:43.532 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T03:56:43.532 INFO:teuthology.orchestra.run.smithi150.stdout: "finisher-PurgeQueue": { 2024-09-21T03:56:43.532 INFO:teuthology.orchestra.run.smithi150.stdout: "queue_len": 0, 2024-09-21T03:56:43.532 INFO:teuthology.orchestra.run.smithi150.stdout: "complete_latency": { 2024-09-21T03:56:43.532 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-09-21T03:56:43.533 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-09-21T03:56:43.533 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-09-21T03:56:43.533 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T03:56:43.533 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T03:56:43.533 INFO:teuthology.orchestra.run.smithi150.stdout: "mds": { 2024-09-21T03:56:43.533 INFO:teuthology.orchestra.run.smithi150.stdout: "request": 0, 2024-09-21T03:56:43.533 INFO:teuthology.orchestra.run.smithi150.stdout: "reply": 0, 2024-09-21T03:56:43.533 INFO:teuthology.orchestra.run.smithi150.stdout: "reply_latency": { 2024-09-21T03:56:43.533 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-09-21T03:56:43.533 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-09-21T03:56:43.533 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-09-21T03:56:43.533 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T03:56:43.533 INFO:teuthology.orchestra.run.smithi150.stdout: "slow_reply": 0, 2024-09-21T03:56:43.533 INFO:teuthology.orchestra.run.smithi150.stdout: "forward": 0, 2024-09-21T03:56:43.534 INFO:teuthology.orchestra.run.smithi150.stdout: "dir_fetch_complete": 0, 2024-09-21T03:56:43.534 INFO:teuthology.orchestra.run.smithi150.stdout: "dir_fetch_keys": 0, 2024-09-21T03:56:43.534 INFO:teuthology.orchestra.run.smithi150.stdout: "dir_commit": 0, 2024-09-21T03:56:43.534 INFO:teuthology.orchestra.run.smithi150.stdout: "dir_split": 0, 2024-09-21T03:56:43.534 INFO:teuthology.orchestra.run.smithi150.stdout: "dir_merge": 0, 2024-09-21T03:56:43.534 INFO:teuthology.orchestra.run.smithi150.stdout: "inodes": 0, 2024-09-21T03:56:43.534 INFO:teuthology.orchestra.run.smithi150.stdout: "inodes_top": 0, 2024-09-21T03:56:43.534 INFO:teuthology.orchestra.run.smithi150.stdout: "inodes_bottom": 0, 2024-09-21T03:56:43.534 INFO:teuthology.orchestra.run.smithi150.stdout: "inodes_pin_tail": 0, 2024-09-21T03:56:43.534 INFO:teuthology.orchestra.run.smithi150.stdout: "inodes_pinned": 0, 2024-09-21T03:56:43.534 INFO:teuthology.orchestra.run.smithi150.stdout: "inodes_expired": 0, 2024-09-21T03:56:43.534 INFO:teuthology.orchestra.run.smithi150.stdout: "inodes_with_caps": 0, 2024-09-21T03:56:43.534 INFO:teuthology.orchestra.run.smithi150.stdout: "caps": 0, 2024-09-21T03:56:43.535 INFO:teuthology.orchestra.run.smithi150.stdout: "subtrees": 1, 2024-09-21T03:56:43.535 INFO:teuthology.orchestra.run.smithi150.stdout: "traverse": 0, 2024-09-21T03:56:43.535 INFO:teuthology.orchestra.run.smithi150.stdout: "traverse_hit": 0, 2024-09-21T03:56:43.535 INFO:teuthology.orchestra.run.smithi150.stdout: "traverse_forward": 0, 2024-09-21T03:56:43.535 INFO:teuthology.orchestra.run.smithi150.stdout: "traverse_discover": 0, 2024-09-21T03:56:43.535 INFO:teuthology.orchestra.run.smithi150.stdout: "traverse_dir_fetch": 0, 2024-09-21T03:56:43.535 INFO:teuthology.orchestra.run.smithi150.stdout: "traverse_remote_ino": 0, 2024-09-21T03:56:43.535 INFO:teuthology.orchestra.run.smithi150.stdout: "traverse_lock": 0, 2024-09-21T03:56:43.535 INFO:teuthology.orchestra.run.smithi150.stdout: "load_cent": 0, 2024-09-21T03:56:43.535 INFO:teuthology.orchestra.run.smithi150.stdout: "q": 0, 2024-09-21T03:56:43.535 INFO:teuthology.orchestra.run.smithi150.stdout: "exported": 0, 2024-09-21T03:56:43.535 INFO:teuthology.orchestra.run.smithi150.stdout: "exported_inodes": 0, 2024-09-21T03:56:43.535 INFO:teuthology.orchestra.run.smithi150.stdout: "imported": 0, 2024-09-21T03:56:43.536 INFO:teuthology.orchestra.run.smithi150.stdout: "imported_inodes": 0, 2024-09-21T03:56:43.536 INFO:teuthology.orchestra.run.smithi150.stdout: "openino_dir_fetch": 0, 2024-09-21T03:56:43.536 INFO:teuthology.orchestra.run.smithi150.stdout: "openino_backtrace_fetch": 0, 2024-09-21T03:56:43.536 INFO:teuthology.orchestra.run.smithi150.stdout: "openino_peer_discover": 0, 2024-09-21T03:56:43.536 INFO:teuthology.orchestra.run.smithi150.stdout: "root_rfiles": 0, 2024-09-21T03:56:43.536 INFO:teuthology.orchestra.run.smithi150.stdout: "root_rbytes": 0, 2024-09-21T03:56:43.536 INFO:teuthology.orchestra.run.smithi150.stdout: "root_rsnaps": 0, 2024-09-21T03:56:43.536 INFO:teuthology.orchestra.run.smithi150.stdout: "scrub_backtrace_fetch": 0, 2024-09-21T03:56:43.536 INFO:teuthology.orchestra.run.smithi150.stdout: "scrub_set_tag": 0, 2024-09-21T03:56:43.536 INFO:teuthology.orchestra.run.smithi150.stdout: "scrub_backtrace_repaired": 0, 2024-09-21T03:56:43.536 INFO:teuthology.orchestra.run.smithi150.stdout: "scrub_inotable_repaired": 0, 2024-09-21T03:56:43.536 INFO:teuthology.orchestra.run.smithi150.stdout: "scrub_dir_inodes": 0, 2024-09-21T03:56:43.536 INFO:teuthology.orchestra.run.smithi150.stdout: "scrub_dir_base_inodes": 0, 2024-09-21T03:56:43.537 INFO:teuthology.orchestra.run.smithi150.stdout: "scrub_dirfrag_rstats": 0, 2024-09-21T03:56:43.537 INFO:teuthology.orchestra.run.smithi150.stdout: "scrub_file_inodes": 0, 2024-09-21T03:56:43.537 INFO:teuthology.orchestra.run.smithi150.stdout: "handle_inode_file_caps": 0, 2024-09-21T03:56:43.537 INFO:teuthology.orchestra.run.smithi150.stdout: "ceph_cap_op_revoke": 0, 2024-09-21T03:56:43.537 INFO:teuthology.orchestra.run.smithi150.stdout: "ceph_cap_op_grant": 0, 2024-09-21T03:56:43.537 INFO:teuthology.orchestra.run.smithi150.stdout: "ceph_cap_op_trunc": 0, 2024-09-21T03:56:43.537 INFO:teuthology.orchestra.run.smithi150.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-21T03:56:43.537 INFO:teuthology.orchestra.run.smithi150.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-21T03:56:43.537 INFO:teuthology.orchestra.run.smithi150.stdout: "handle_client_caps": 0, 2024-09-21T03:56:43.537 INFO:teuthology.orchestra.run.smithi150.stdout: "handle_client_caps_dirty": 0, 2024-09-21T03:56:43.537 INFO:teuthology.orchestra.run.smithi150.stdout: "handle_client_cap_release": 0, 2024-09-21T03:56:43.537 INFO:teuthology.orchestra.run.smithi150.stdout: "process_request_cap_release": 0 2024-09-21T03:56:43.537 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T03:56:43.537 INFO:teuthology.orchestra.run.smithi150.stdout: "mds_cache": { 2024-09-21T03:56:43.538 INFO:teuthology.orchestra.run.smithi150.stdout: "dir_update": 0, 2024-09-21T03:56:43.538 INFO:teuthology.orchestra.run.smithi150.stdout: "dir_update_receipt": 0, 2024-09-21T03:56:43.538 INFO:teuthology.orchestra.run.smithi150.stdout: "dir_try_discover": 0, 2024-09-21T03:56:43.538 INFO:teuthology.orchestra.run.smithi150.stdout: "dir_send_discover": 0, 2024-09-21T03:56:43.538 INFO:teuthology.orchestra.run.smithi150.stdout: "dir_handle_discover": 0, 2024-09-21T03:56:43.538 INFO:teuthology.orchestra.run.smithi150.stdout: "num_strays": 0, 2024-09-21T03:56:43.538 INFO:teuthology.orchestra.run.smithi150.stdout: "num_strays_delayed": 0, 2024-09-21T03:56:43.538 INFO:teuthology.orchestra.run.smithi150.stdout: "num_strays_enqueuing": 0, 2024-09-21T03:56:43.538 INFO:teuthology.orchestra.run.smithi150.stdout: "strays_created": 0, 2024-09-21T03:56:43.538 INFO:teuthology.orchestra.run.smithi150.stdout: "strays_enqueued": 0, 2024-09-21T03:56:43.539 INFO:teuthology.orchestra.run.smithi150.stdout: "strays_reintegrated": 0, 2024-09-21T03:56:43.539 INFO:teuthology.orchestra.run.smithi150.stdout: "strays_migrated": 0, 2024-09-21T03:56:43.539 INFO:teuthology.orchestra.run.smithi150.stdout: "num_recovering_processing": 0, 2024-09-21T03:56:43.539 INFO:teuthology.orchestra.run.smithi150.stdout: "num_recovering_enqueued": 0, 2024-09-21T03:56:43.539 INFO:teuthology.orchestra.run.smithi150.stdout: "num_recovering_prioritized": 0, 2024-09-21T03:56:43.539 INFO:teuthology.orchestra.run.smithi150.stdout: "recovery_started": 0, 2024-09-21T03:56:43.539 INFO:teuthology.orchestra.run.smithi150.stdout: "recovery_completed": 0, 2024-09-21T03:56:43.539 INFO:teuthology.orchestra.run.smithi150.stdout: "ireq_quiesce_path": 0, 2024-09-21T03:56:43.539 INFO:teuthology.orchestra.run.smithi150.stdout: "ireq_quiesce_inode": 0, 2024-09-21T03:56:43.539 INFO:teuthology.orchestra.run.smithi150.stdout: "ireq_enqueue_scrub": 0, 2024-09-21T03:56:43.539 INFO:teuthology.orchestra.run.smithi150.stdout: "ireq_exportdir": 0, 2024-09-21T03:56:43.539 INFO:teuthology.orchestra.run.smithi150.stdout: "ireq_flush": 0, 2024-09-21T03:56:43.540 INFO:teuthology.orchestra.run.smithi150.stdout: "ireq_fragmentdir": 0, 2024-09-21T03:56:43.540 INFO:teuthology.orchestra.run.smithi150.stdout: "ireq_fragstats": 0, 2024-09-21T03:56:43.540 INFO:teuthology.orchestra.run.smithi150.stdout: "ireq_inodestats": 0, 2024-09-21T03:56:43.540 INFO:teuthology.orchestra.run.smithi150.stdout: "uninline_started": 0, 2024-09-21T03:56:43.540 INFO:teuthology.orchestra.run.smithi150.stdout: "uninline_succeeded": 0, 2024-09-21T03:56:43.540 INFO:teuthology.orchestra.run.smithi150.stdout: "uninline_write_failed": 0 2024-09-21T03:56:43.540 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T03:56:43.540 INFO:teuthology.orchestra.run.smithi150.stdout: "mds_log": { 2024-09-21T03:56:43.540 INFO:teuthology.orchestra.run.smithi150.stdout: "evlrg": 0, 2024-09-21T03:56:43.540 INFO:teuthology.orchestra.run.smithi150.stdout: "evadd": 0, 2024-09-21T03:56:43.541 INFO:teuthology.orchestra.run.smithi150.stdout: "evex": 0, 2024-09-21T03:56:43.541 INFO:teuthology.orchestra.run.smithi150.stdout: "evtrm": 0, 2024-09-21T03:56:43.541 INFO:teuthology.orchestra.run.smithi150.stdout: "ev": 6, 2024-09-21T03:56:43.541 INFO:teuthology.orchestra.run.smithi150.stdout: "evexg": 0, 2024-09-21T03:56:43.541 INFO:teuthology.orchestra.run.smithi150.stdout: "evexd": 0, 2024-09-21T03:56:43.541 INFO:teuthology.orchestra.run.smithi150.stdout: "segadd": 0, 2024-09-21T03:56:43.541 INFO:teuthology.orchestra.run.smithi150.stdout: "segex": 0, 2024-09-21T03:56:43.541 INFO:teuthology.orchestra.run.smithi150.stdout: "segtrm": 0, 2024-09-21T03:56:43.541 INFO:teuthology.orchestra.run.smithi150.stdout: "seg": 2, 2024-09-21T03:56:43.541 INFO:teuthology.orchestra.run.smithi150.stdout: "segmjr": 2, 2024-09-21T03:56:43.541 INFO:teuthology.orchestra.run.smithi150.stdout: "segexg": 0, 2024-09-21T03:56:43.541 INFO:teuthology.orchestra.run.smithi150.stdout: "segexd": 0, 2024-09-21T03:56:43.541 INFO:teuthology.orchestra.run.smithi150.stdout: "expos": 4194304, 2024-09-21T03:56:43.541 INFO:teuthology.orchestra.run.smithi150.stdout: "wrpos": 4195956, 2024-09-21T03:56:43.542 INFO:teuthology.orchestra.run.smithi150.stdout: "rdpos": 4195815, 2024-09-21T03:56:43.542 INFO:teuthology.orchestra.run.smithi150.stdout: "jlat": { 2024-09-21T03:56:43.542 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-09-21T03:56:43.542 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-09-21T03:56:43.542 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-09-21T03:56:43.542 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T03:56:43.542 INFO:teuthology.orchestra.run.smithi150.stdout: "replayed": 6 2024-09-21T03:56:43.542 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T03:56:43.542 INFO:teuthology.orchestra.run.smithi150.stdout: "mds_mem": { 2024-09-21T03:56:43.542 INFO:teuthology.orchestra.run.smithi150.stdout: "ino": 3, 2024-09-21T03:56:43.542 INFO:teuthology.orchestra.run.smithi150.stdout: "ino+": 3, 2024-09-21T03:56:43.542 INFO:teuthology.orchestra.run.smithi150.stdout: "ino-": 0, 2024-09-21T03:56:43.542 INFO:teuthology.orchestra.run.smithi150.stdout: "dir": 1, 2024-09-21T03:56:43.543 INFO:teuthology.orchestra.run.smithi150.stdout: "dir+": 1, 2024-09-21T03:56:43.543 INFO:teuthology.orchestra.run.smithi150.stdout: "dir-": 0, 2024-09-21T03:56:43.543 INFO:teuthology.orchestra.run.smithi150.stdout: "dn": 0, 2024-09-21T03:56:43.543 INFO:teuthology.orchestra.run.smithi150.stdout: "dn+": 0, 2024-09-21T03:56:43.543 INFO:teuthology.orchestra.run.smithi150.stdout: "dn-": 0, 2024-09-21T03:56:43.543 INFO:teuthology.orchestra.run.smithi150.stdout: "cap": 0, 2024-09-21T03:56:43.543 INFO:teuthology.orchestra.run.smithi150.stdout: "cap+": 0, 2024-09-21T03:56:43.543 INFO:teuthology.orchestra.run.smithi150.stdout: "cap-": 0, 2024-09-21T03:56:43.543 INFO:teuthology.orchestra.run.smithi150.stdout: "rss": 117732, 2024-09-21T03:56:43.543 INFO:teuthology.orchestra.run.smithi150.stdout: "heap": 316428288 2024-09-21T03:56:43.543 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T03:56:43.543 INFO:teuthology.orchestra.run.smithi150.stdout: "mds_server": { 2024-09-21T03:56:43.543 INFO:teuthology.orchestra.run.smithi150.stdout: "dispatch_client_request": 0, 2024-09-21T03:56:43.544 INFO:teuthology.orchestra.run.smithi150.stdout: "dispatch_server_request": 0, 2024-09-21T03:56:43.544 INFO:teuthology.orchestra.run.smithi150.stdout: "handle_client_request": 0, 2024-09-21T03:56:43.544 INFO:teuthology.orchestra.run.smithi150.stdout: "handle_client_session": 0, 2024-09-21T03:56:43.544 INFO:teuthology.orchestra.run.smithi150.stdout: "handle_peer_request": 0, 2024-09-21T03:56:43.544 INFO:teuthology.orchestra.run.smithi150.stdout: "req_create_latency": { 2024-09-21T03:56:43.544 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-09-21T03:56:43.544 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-09-21T03:56:43.544 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-09-21T03:56:43.544 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T03:56:43.545 INFO:teuthology.orchestra.run.smithi150.stdout: "req_getattr_latency": { 2024-09-21T03:56:43.545 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-09-21T03:56:43.545 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-09-21T03:56:43.545 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-09-21T03:56:43.545 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T03:56:43.545 INFO:teuthology.orchestra.run.smithi150.stdout: "req_getfilelock_latency": { 2024-09-21T03:56:43.545 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-09-21T03:56:43.545 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-09-21T03:56:43.545 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-09-21T03:56:43.545 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T03:56:43.545 INFO:teuthology.orchestra.run.smithi150.stdout: "req_link_latency": { 2024-09-21T03:56:43.545 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-09-21T03:56:43.545 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-09-21T03:56:43.546 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-09-21T03:56:43.546 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T03:56:43.546 INFO:teuthology.orchestra.run.smithi150.stdout: "req_lookup_latency": { 2024-09-21T03:56:43.546 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-09-21T03:56:43.546 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-09-21T03:56:43.546 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-09-21T03:56:43.546 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T03:56:43.546 INFO:teuthology.orchestra.run.smithi150.stdout: "req_lookuphash_latency": { 2024-09-21T03:56:43.546 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-09-21T03:56:43.546 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-09-21T03:56:43.546 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-09-21T03:56:43.546 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T03:56:43.546 INFO:teuthology.orchestra.run.smithi150.stdout: "req_lookupino_latency": { 2024-09-21T03:56:43.546 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-09-21T03:56:43.547 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-09-21T03:56:43.547 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-09-21T03:56:43.547 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T03:56:43.547 INFO:teuthology.orchestra.run.smithi150.stdout: "req_lookupname_latency": { 2024-09-21T03:56:43.547 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-09-21T03:56:43.547 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-09-21T03:56:43.547 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-09-21T03:56:43.547 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T03:56:43.547 INFO:teuthology.orchestra.run.smithi150.stdout: "req_lookupparent_latency": { 2024-09-21T03:56:43.547 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-09-21T03:56:43.547 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-09-21T03:56:43.547 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-09-21T03:56:43.547 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T03:56:43.547 INFO:teuthology.orchestra.run.smithi150.stdout: "req_lookupsnap_latency": { 2024-09-21T03:56:43.548 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-09-21T03:56:43.548 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-09-21T03:56:43.548 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-09-21T03:56:43.548 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T03:56:43.548 INFO:teuthology.orchestra.run.smithi150.stdout: "req_lssnap_latency": { 2024-09-21T03:56:43.548 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-09-21T03:56:43.548 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-09-21T03:56:43.549 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-09-21T03:56:43.549 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T03:56:43.549 INFO:teuthology.orchestra.run.smithi150.stdout: "req_mkdir_latency": { 2024-09-21T03:56:43.549 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-09-21T03:56:43.549 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-09-21T03:56:43.549 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-09-21T03:56:43.549 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T03:56:43.549 INFO:teuthology.orchestra.run.smithi150.stdout: "req_mknod_latency": { 2024-09-21T03:56:43.549 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-09-21T03:56:43.549 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-09-21T03:56:43.550 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-09-21T03:56:43.550 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T03:56:43.550 INFO:teuthology.orchestra.run.smithi150.stdout: "req_mksnap_latency": { 2024-09-21T03:56:43.550 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-09-21T03:56:43.550 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-09-21T03:56:43.550 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-09-21T03:56:43.550 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T03:56:43.550 INFO:teuthology.orchestra.run.smithi150.stdout: "req_open_latency": { 2024-09-21T03:56:43.550 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-09-21T03:56:43.550 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-09-21T03:56:43.550 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-09-21T03:56:43.550 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T03:56:43.550 INFO:teuthology.orchestra.run.smithi150.stdout: "req_readdir_latency": { 2024-09-21T03:56:43.551 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-09-21T03:56:43.551 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-09-21T03:56:43.551 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-09-21T03:56:43.551 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T03:56:43.551 INFO:teuthology.orchestra.run.smithi150.stdout: "req_rename_latency": { 2024-09-21T03:56:43.551 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-09-21T03:56:43.551 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-09-21T03:56:43.551 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-09-21T03:56:43.551 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T03:56:43.551 INFO:teuthology.orchestra.run.smithi150.stdout: "req_renamesnap_latency": { 2024-09-21T03:56:43.551 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-09-21T03:56:43.551 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-09-21T03:56:43.551 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-09-21T03:56:43.552 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T03:56:43.552 INFO:teuthology.orchestra.run.smithi150.stdout: "req_snapdiff_latency": { 2024-09-21T03:56:43.552 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-09-21T03:56:43.552 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-09-21T03:56:43.552 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-09-21T03:56:43.552 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T03:56:43.552 INFO:teuthology.orchestra.run.smithi150.stdout: "req_rmdir_latency": { 2024-09-21T03:56:43.552 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-09-21T03:56:43.552 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-09-21T03:56:43.552 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-09-21T03:56:43.552 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T03:56:43.552 INFO:teuthology.orchestra.run.smithi150.stdout: "req_rmsnap_latency": { 2024-09-21T03:56:43.552 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-09-21T03:56:43.552 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-09-21T03:56:43.553 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-09-21T03:56:43.553 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T03:56:43.553 INFO:teuthology.orchestra.run.smithi150.stdout: "req_rmxattr_latency": { 2024-09-21T03:56:43.553 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-09-21T03:56:43.553 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-09-21T03:56:43.553 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-09-21T03:56:43.553 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T03:56:43.553 INFO:teuthology.orchestra.run.smithi150.stdout: "req_setattr_latency": { 2024-09-21T03:56:43.553 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-09-21T03:56:43.553 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-09-21T03:56:43.553 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-09-21T03:56:43.553 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T03:56:43.553 INFO:teuthology.orchestra.run.smithi150.stdout: "req_setdirlayout_latency": { 2024-09-21T03:56:43.553 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-09-21T03:56:43.554 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-09-21T03:56:43.554 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-09-21T03:56:43.554 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T03:56:43.554 INFO:teuthology.orchestra.run.smithi150.stdout: "req_setfilelock_latency": { 2024-09-21T03:56:43.554 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-09-21T03:56:43.554 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-09-21T03:56:43.554 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-09-21T03:56:43.554 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T03:56:43.554 INFO:teuthology.orchestra.run.smithi150.stdout: "req_setlayout_latency": { 2024-09-21T03:56:43.554 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-09-21T03:56:43.554 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-09-21T03:56:43.554 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-09-21T03:56:43.554 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T03:56:43.554 INFO:teuthology.orchestra.run.smithi150.stdout: "req_setxattr_latency": { 2024-09-21T03:56:43.555 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-09-21T03:56:43.555 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-09-21T03:56:43.555 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-09-21T03:56:43.555 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T03:56:43.555 INFO:teuthology.orchestra.run.smithi150.stdout: "req_symlink_latency": { 2024-09-21T03:56:43.555 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-09-21T03:56:43.555 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-09-21T03:56:43.555 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-09-21T03:56:43.555 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T03:56:43.555 INFO:teuthology.orchestra.run.smithi150.stdout: "req_unlink_latency": { 2024-09-21T03:56:43.555 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-09-21T03:56:43.555 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-09-21T03:56:43.555 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-09-21T03:56:43.555 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T03:56:43.556 INFO:teuthology.orchestra.run.smithi150.stdout: "cap_revoke_eviction": 0, 2024-09-21T03:56:43.556 INFO:teuthology.orchestra.run.smithi150.stdout: "cap_acquisition_throttle": 0, 2024-09-21T03:56:43.556 INFO:teuthology.orchestra.run.smithi150.stdout: "req_getvxattr_latency": { 2024-09-21T03:56:43.556 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-09-21T03:56:43.556 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-09-21T03:56:43.556 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-09-21T03:56:43.556 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T03:56:43.556 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T03:56:43.556 INFO:teuthology.orchestra.run.smithi150.stdout: "mds_sessions": { 2024-09-21T03:56:43.556 INFO:teuthology.orchestra.run.smithi150.stdout: "session_count": 0, 2024-09-21T03:56:43.556 INFO:teuthology.orchestra.run.smithi150.stdout: "session_add": 2, 2024-09-21T03:56:43.556 INFO:teuthology.orchestra.run.smithi150.stdout: "session_remove": 2, 2024-09-21T03:56:43.556 INFO:teuthology.orchestra.run.smithi150.stdout: "sessions_open": 1, 2024-09-21T03:56:43.556 INFO:teuthology.orchestra.run.smithi150.stdout: "sessions_stale": 0, 2024-09-21T03:56:43.556 INFO:teuthology.orchestra.run.smithi150.stdout: "total_load": 0, 2024-09-21T03:56:43.557 INFO:teuthology.orchestra.run.smithi150.stdout: "average_load": 0, 2024-09-21T03:56:43.557 INFO:teuthology.orchestra.run.smithi150.stdout: "avg_session_uptime": 1551, 2024-09-21T03:56:43.557 INFO:teuthology.orchestra.run.smithi150.stdout: "mdthresh_evicted": 0 2024-09-21T03:56:43.557 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T03:56:43.557 INFO:teuthology.orchestra.run.smithi150.stdout: "mempool": { 2024-09-21T03:56:43.557 INFO:teuthology.orchestra.run.smithi150.stdout: "bloom_filter_bytes": 0, 2024-09-21T03:56:43.557 INFO:teuthology.orchestra.run.smithi150.stdout: "bloom_filter_items": 0, 2024-09-21T03:56:43.557 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_alloc_bytes": 0, 2024-09-21T03:56:43.557 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_alloc_items": 0, 2024-09-21T03:56:43.557 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_cache_data_bytes": 0, 2024-09-21T03:56:43.557 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_cache_data_items": 0, 2024-09-21T03:56:43.557 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-21T03:56:43.557 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_cache_onode_items": 0, 2024-09-21T03:56:43.557 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-21T03:56:43.557 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_cache_meta_items": 0, 2024-09-21T03:56:43.558 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_cache_other_bytes": 0, 2024-09-21T03:56:43.558 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_cache_other_items": 0, 2024-09-21T03:56:43.558 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-21T03:56:43.558 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_cache_buffer_items": 0, 2024-09-21T03:56:43.558 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_extent_bytes": 0, 2024-09-21T03:56:43.558 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_extent_items": 0, 2024-09-21T03:56:43.558 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_blob_bytes": 0, 2024-09-21T03:56:43.558 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_blob_items": 0, 2024-09-21T03:56:43.558 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-21T03:56:43.558 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_shared_blob_items": 0, 2024-09-21T03:56:43.558 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-21T03:56:43.558 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_inline_bl_items": 0, 2024-09-21T03:56:43.559 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_fsck_bytes": 0, 2024-09-21T03:56:43.559 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_fsck_items": 0, 2024-09-21T03:56:43.559 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_txc_bytes": 0, 2024-09-21T03:56:43.559 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_txc_items": 0, 2024-09-21T03:56:43.559 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-21T03:56:43.559 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_writing_deferred_items": 0, 2024-09-21T03:56:43.559 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_writing_bytes": 0, 2024-09-21T03:56:43.559 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_writing_items": 0, 2024-09-21T03:56:43.559 INFO:teuthology.orchestra.run.smithi150.stdout: "bluefs_bytes": 0, 2024-09-21T03:56:43.559 INFO:teuthology.orchestra.run.smithi150.stdout: "bluefs_items": 0, 2024-09-21T03:56:43.559 INFO:teuthology.orchestra.run.smithi150.stdout: "bluefs_file_reader_bytes": 0, 2024-09-21T03:56:43.559 INFO:teuthology.orchestra.run.smithi150.stdout: "bluefs_file_reader_items": 0, 2024-09-21T03:56:43.559 INFO:teuthology.orchestra.run.smithi150.stdout: "bluefs_file_writer_bytes": 0, 2024-09-21T03:56:43.559 INFO:teuthology.orchestra.run.smithi150.stdout: "bluefs_file_writer_items": 0, 2024-09-21T03:56:43.560 INFO:teuthology.orchestra.run.smithi150.stdout: "buffer_anon_bytes": 181337, 2024-09-21T03:56:43.560 INFO:teuthology.orchestra.run.smithi150.stdout: "buffer_anon_items": 48, 2024-09-21T03:56:43.560 INFO:teuthology.orchestra.run.smithi150.stdout: "buffer_meta_bytes": 0, 2024-09-21T03:56:43.560 INFO:teuthology.orchestra.run.smithi150.stdout: "buffer_meta_items": 0, 2024-09-21T03:56:43.560 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_bytes": 0, 2024-09-21T03:56:43.560 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_items": 0, 2024-09-21T03:56:43.560 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_mapbl_bytes": 0, 2024-09-21T03:56:43.560 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_mapbl_items": 0, 2024-09-21T03:56:43.560 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_pglog_bytes": 0, 2024-09-21T03:56:43.560 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_pglog_items": 0, 2024-09-21T03:56:43.560 INFO:teuthology.orchestra.run.smithi150.stdout: "osdmap_bytes": 8256, 2024-09-21T03:56:43.560 INFO:teuthology.orchestra.run.smithi150.stdout: "osdmap_items": 182, 2024-09-21T03:56:43.560 INFO:teuthology.orchestra.run.smithi150.stdout: "osdmap_mapping_bytes": 0, 2024-09-21T03:56:43.561 INFO:teuthology.orchestra.run.smithi150.stdout: "osdmap_mapping_items": 0, 2024-09-21T03:56:43.561 INFO:teuthology.orchestra.run.smithi150.stdout: "pgmap_bytes": 0, 2024-09-21T03:56:43.561 INFO:teuthology.orchestra.run.smithi150.stdout: "pgmap_items": 0, 2024-09-21T03:56:43.561 INFO:teuthology.orchestra.run.smithi150.stdout: "mds_co_bytes": 8672, 2024-09-21T03:56:43.561 INFO:teuthology.orchestra.run.smithi150.stdout: "mds_co_items": 13, 2024-09-21T03:56:43.561 INFO:teuthology.orchestra.run.smithi150.stdout: "unittest_1_bytes": 0, 2024-09-21T03:56:43.561 INFO:teuthology.orchestra.run.smithi150.stdout: "unittest_1_items": 0, 2024-09-21T03:56:43.561 INFO:teuthology.orchestra.run.smithi150.stdout: "unittest_2_bytes": 0, 2024-09-21T03:56:43.561 INFO:teuthology.orchestra.run.smithi150.stdout: "unittest_2_items": 0 2024-09-21T03:56:43.561 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T03:56:43.561 INFO:teuthology.orchestra.run.smithi150.stdout: "objecter": { 2024-09-21T03:56:43.561 INFO:teuthology.orchestra.run.smithi150.stdout: "op_active": 0, 2024-09-21T03:56:43.561 INFO:teuthology.orchestra.run.smithi150.stdout: "op_laggy": 0, 2024-09-21T03:56:43.561 INFO:teuthology.orchestra.run.smithi150.stdout: "op_send": 2132, 2024-09-21T03:56:43.562 INFO:teuthology.orchestra.run.smithi150.stdout: "op_send_bytes": 21, 2024-09-21T03:56:43.562 INFO:teuthology.orchestra.run.smithi150.stdout: "op_resend": 3, 2024-09-21T03:56:43.562 INFO:teuthology.orchestra.run.smithi150.stdout: "op_reply": 2132, 2024-09-21T03:56:43.562 INFO:teuthology.orchestra.run.smithi150.stdout: "op_latency": { 2024-09-21T03:56:43.562 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 2132, 2024-09-21T03:56:43.562 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 1.251974226, 2024-09-21T03:56:43.562 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000587229 2024-09-21T03:56:43.562 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T03:56:43.562 INFO:teuthology.orchestra.run.smithi150.stdout: "op_inflight": 0, 2024-09-21T03:56:43.563 INFO:teuthology.orchestra.run.smithi150.stdout: "oplen_avg": { 2024-09-21T03:56:43.563 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 2132, 2024-09-21T03:56:43.563 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 2133 2024-09-21T03:56:43.563 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T03:56:43.563 INFO:teuthology.orchestra.run.smithi150.stdout: "op": 2132, 2024-09-21T03:56:43.563 INFO:teuthology.orchestra.run.smithi150.stdout: "op_r": 2132, 2024-09-21T03:56:43.563 INFO:teuthology.orchestra.run.smithi150.stdout: "op_w": 0, 2024-09-21T03:56:43.563 INFO:teuthology.orchestra.run.smithi150.stdout: "op_rmw": 0, 2024-09-21T03:56:43.563 INFO:teuthology.orchestra.run.smithi150.stdout: "op_pg": 0, 2024-09-21T03:56:43.563 INFO:teuthology.orchestra.run.smithi150.stdout: "osdop_stat": 1384, 2024-09-21T03:56:43.563 INFO:teuthology.orchestra.run.smithi150.stdout: "osdop_create": 0, 2024-09-21T03:56:43.563 INFO:teuthology.orchestra.run.smithi150.stdout: "osdop_read": 746, 2024-09-21T03:56:43.563 INFO:teuthology.orchestra.run.smithi150.stdout: "osdop_write": 0, 2024-09-21T03:56:43.563 INFO:teuthology.orchestra.run.smithi150.stdout: "osdop_writefull": 0, 2024-09-21T03:56:43.564 INFO:teuthology.orchestra.run.smithi150.stdout: "osdop_writesame": 0, 2024-09-21T03:56:43.564 INFO:teuthology.orchestra.run.smithi150.stdout: "osdop_append": 0, 2024-09-21T03:56:43.564 INFO:teuthology.orchestra.run.smithi150.stdout: "osdop_zero": 0, 2024-09-21T03:56:43.564 INFO:teuthology.orchestra.run.smithi150.stdout: "osdop_truncate": 0, 2024-09-21T03:56:43.564 INFO:teuthology.orchestra.run.smithi150.stdout: "osdop_delete": 0, 2024-09-21T03:56:43.564 INFO:teuthology.orchestra.run.smithi150.stdout: "osdop_mapext": 0, 2024-09-21T03:56:43.564 INFO:teuthology.orchestra.run.smithi150.stdout: "osdop_sparse_read": 0, 2024-09-21T03:56:43.564 INFO:teuthology.orchestra.run.smithi150.stdout: "osdop_clonerange": 0, 2024-09-21T03:56:43.564 INFO:teuthology.orchestra.run.smithi150.stdout: "osdop_getxattr": 1, 2024-09-21T03:56:43.564 INFO:teuthology.orchestra.run.smithi150.stdout: "osdop_setxattr": 0, 2024-09-21T03:56:43.564 INFO:teuthology.orchestra.run.smithi150.stdout: "osdop_cmpxattr": 0, 2024-09-21T03:56:43.564 INFO:teuthology.orchestra.run.smithi150.stdout: "osdop_rmxattr": 0, 2024-09-21T03:56:43.564 INFO:teuthology.orchestra.run.smithi150.stdout: "osdop_resetxattrs": 0, 2024-09-21T03:56:43.565 INFO:teuthology.orchestra.run.smithi150.stdout: "osdop_call": 0, 2024-09-21T03:56:43.565 INFO:teuthology.orchestra.run.smithi150.stdout: "osdop_watch": 0, 2024-09-21T03:56:43.565 INFO:teuthology.orchestra.run.smithi150.stdout: "osdop_notify": 0, 2024-09-21T03:56:43.565 INFO:teuthology.orchestra.run.smithi150.stdout: "osdop_src_cmpxattr": 0, 2024-09-21T03:56:43.565 INFO:teuthology.orchestra.run.smithi150.stdout: "osdop_pgls": 0, 2024-09-21T03:56:43.565 INFO:teuthology.orchestra.run.smithi150.stdout: "osdop_pgls_filter": 0, 2024-09-21T03:56:43.565 INFO:teuthology.orchestra.run.smithi150.stdout: "osdop_other": 0, 2024-09-21T03:56:43.565 INFO:teuthology.orchestra.run.smithi150.stdout: "linger_active": 0, 2024-09-21T03:56:43.565 INFO:teuthology.orchestra.run.smithi150.stdout: "linger_send": 0, 2024-09-21T03:56:43.565 INFO:teuthology.orchestra.run.smithi150.stdout: "linger_resend": 0, 2024-09-21T03:56:43.565 INFO:teuthology.orchestra.run.smithi150.stdout: "linger_ping": 0, 2024-09-21T03:56:43.565 INFO:teuthology.orchestra.run.smithi150.stdout: "poolop_active": 0, 2024-09-21T03:56:43.565 INFO:teuthology.orchestra.run.smithi150.stdout: "poolop_send": 0, 2024-09-21T03:56:43.565 INFO:teuthology.orchestra.run.smithi150.stdout: "poolop_resend": 0, 2024-09-21T03:56:43.566 INFO:teuthology.orchestra.run.smithi150.stdout: "poolstat_active": 0, 2024-09-21T03:56:43.566 INFO:teuthology.orchestra.run.smithi150.stdout: "poolstat_send": 0, 2024-09-21T03:56:43.566 INFO:teuthology.orchestra.run.smithi150.stdout: "poolstat_resend": 0, 2024-09-21T03:56:43.566 INFO:teuthology.orchestra.run.smithi150.stdout: "statfs_active": 0, 2024-09-21T03:56:43.566 INFO:teuthology.orchestra.run.smithi150.stdout: "statfs_send": 0, 2024-09-21T03:56:43.566 INFO:teuthology.orchestra.run.smithi150.stdout: "statfs_resend": 0, 2024-09-21T03:56:43.566 INFO:teuthology.orchestra.run.smithi150.stdout: "command_active": 0, 2024-09-21T03:56:43.566 INFO:teuthology.orchestra.run.smithi150.stdout: "command_send": 0, 2024-09-21T03:56:43.566 INFO:teuthology.orchestra.run.smithi150.stdout: "command_resend": 0, 2024-09-21T03:56:43.566 INFO:teuthology.orchestra.run.smithi150.stdout: "map_epoch": 87, 2024-09-21T03:56:43.566 INFO:teuthology.orchestra.run.smithi150.stdout: "map_full": 0, 2024-09-21T03:56:43.566 INFO:teuthology.orchestra.run.smithi150.stdout: "map_inc": 10, 2024-09-21T03:56:43.566 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_sessions": 4, 2024-09-21T03:56:43.566 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_session_open": 4, 2024-09-21T03:56:43.567 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_session_close": 0, 2024-09-21T03:56:43.567 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_laggy": 0, 2024-09-21T03:56:43.567 INFO:teuthology.orchestra.run.smithi150.stdout: "omap_wr": 0, 2024-09-21T03:56:43.567 INFO:teuthology.orchestra.run.smithi150.stdout: "omap_rd": 2, 2024-09-21T03:56:43.567 INFO:teuthology.orchestra.run.smithi150.stdout: "omap_del": 0 2024-09-21T03:56:43.567 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T03:56:43.567 INFO:teuthology.orchestra.run.smithi150.stdout: "oft": { 2024-09-21T03:56:43.567 INFO:teuthology.orchestra.run.smithi150.stdout: "omap_total_objs": 0, 2024-09-21T03:56:43.567 INFO:teuthology.orchestra.run.smithi150.stdout: "omap_total_kv_pairs": 0, 2024-09-21T03:56:43.567 INFO:teuthology.orchestra.run.smithi150.stdout: "omap_total_updates": 0, 2024-09-21T03:56:43.567 INFO:teuthology.orchestra.run.smithi150.stdout: "omap_total_removes": 0 2024-09-21T03:56:43.568 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T03:56:43.568 INFO:teuthology.orchestra.run.smithi150.stdout: "purge_queue": { 2024-09-21T03:56:43.568 INFO:teuthology.orchestra.run.smithi150.stdout: "pq_executing_ops": 0, 2024-09-21T03:56:43.568 INFO:teuthology.orchestra.run.smithi150.stdout: "pq_executing_ops_high_water": 0, 2024-09-21T03:56:43.568 INFO:teuthology.orchestra.run.smithi150.stdout: "pq_executing": 0, 2024-09-21T03:56:43.568 INFO:teuthology.orchestra.run.smithi150.stdout: "pq_executing_high_water": 0, 2024-09-21T03:56:43.568 INFO:teuthology.orchestra.run.smithi150.stdout: "pq_executed_ops": 0, 2024-09-21T03:56:43.569 INFO:teuthology.orchestra.run.smithi150.stdout: "pq_executed": 0, 2024-09-21T03:56:43.569 INFO:teuthology.orchestra.run.smithi150.stdout: "pq_item_in_journal": 0 2024-09-21T03:56:43.569 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T03:56:43.569 INFO:teuthology.orchestra.run.smithi150.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-21T03:56:43.569 INFO:teuthology.orchestra.run.smithi150.stdout: "val": 0, 2024-09-21T03:56:43.569 INFO:teuthology.orchestra.run.smithi150.stdout: "max": 104857600, 2024-09-21T03:56:43.569 INFO:teuthology.orchestra.run.smithi150.stdout: "get_started": 0, 2024-09-21T03:56:43.569 INFO:teuthology.orchestra.run.smithi150.stdout: "get": 2369, 2024-09-21T03:56:43.569 INFO:teuthology.orchestra.run.smithi150.stdout: "get_sum": 1578798, 2024-09-21T03:56:43.569 INFO:teuthology.orchestra.run.smithi150.stdout: "get_or_fail_fail": 0, 2024-09-21T03:56:43.569 INFO:teuthology.orchestra.run.smithi150.stdout: "get_or_fail_success": 2369, 2024-09-21T03:56:43.569 INFO:teuthology.orchestra.run.smithi150.stdout: "take": 0, 2024-09-21T03:56:43.569 INFO:teuthology.orchestra.run.smithi150.stdout: "take_sum": 0, 2024-09-21T03:56:43.570 INFO:teuthology.orchestra.run.smithi150.stdout: "put": 2369, 2024-09-21T03:56:43.570 INFO:teuthology.orchestra.run.smithi150.stdout: "put_sum": 1578798, 2024-09-21T03:56:43.570 INFO:teuthology.orchestra.run.smithi150.stdout: "wait": { 2024-09-21T03:56:43.570 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-09-21T03:56:43.570 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-09-21T03:56:43.570 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-09-21T03:56:43.570 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T03:56:43.570 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T03:56:43.570 INFO:teuthology.orchestra.run.smithi150.stdout: "throttle-objecter_bytes": { 2024-09-21T03:56:43.570 INFO:teuthology.orchestra.run.smithi150.stdout: "val": 0, 2024-09-21T03:56:43.570 INFO:teuthology.orchestra.run.smithi150.stdout: "max": 104857600, 2024-09-21T03:56:43.570 INFO:teuthology.orchestra.run.smithi150.stdout: "get_started": 0, 2024-09-21T03:56:43.570 INFO:teuthology.orchestra.run.smithi150.stdout: "get": 0, 2024-09-21T03:56:43.570 INFO:teuthology.orchestra.run.smithi150.stdout: "get_sum": 0, 2024-09-21T03:56:43.571 INFO:teuthology.orchestra.run.smithi150.stdout: "get_or_fail_fail": 0, 2024-09-21T03:56:43.571 INFO:teuthology.orchestra.run.smithi150.stdout: "get_or_fail_success": 0, 2024-09-21T03:56:43.571 INFO:teuthology.orchestra.run.smithi150.stdout: "take": 2132, 2024-09-21T03:56:43.571 INFO:teuthology.orchestra.run.smithi150.stdout: "take_sum": 1657, 2024-09-21T03:56:43.571 INFO:teuthology.orchestra.run.smithi150.stdout: "put": 2132, 2024-09-21T03:56:43.571 INFO:teuthology.orchestra.run.smithi150.stdout: "put_sum": 1657, 2024-09-21T03:56:43.571 INFO:teuthology.orchestra.run.smithi150.stdout: "wait": { 2024-09-21T03:56:43.571 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-09-21T03:56:43.571 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-09-21T03:56:43.571 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-09-21T03:56:43.571 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T03:56:43.571 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T03:56:43.571 INFO:teuthology.orchestra.run.smithi150.stdout: "throttle-objecter_ops": { 2024-09-21T03:56:43.571 INFO:teuthology.orchestra.run.smithi150.stdout: "val": 0, 2024-09-21T03:56:43.572 INFO:teuthology.orchestra.run.smithi150.stdout: "max": 1024, 2024-09-21T03:56:43.572 INFO:teuthology.orchestra.run.smithi150.stdout: "get_started": 0, 2024-09-21T03:56:43.572 INFO:teuthology.orchestra.run.smithi150.stdout: "get": 0, 2024-09-21T03:56:43.572 INFO:teuthology.orchestra.run.smithi150.stdout: "get_sum": 0, 2024-09-21T03:56:43.572 INFO:teuthology.orchestra.run.smithi150.stdout: "get_or_fail_fail": 0, 2024-09-21T03:56:43.572 INFO:teuthology.orchestra.run.smithi150.stdout: "get_or_fail_success": 0, 2024-09-21T03:56:43.572 INFO:teuthology.orchestra.run.smithi150.stdout: "take": 2132, 2024-09-21T03:56:43.572 INFO:teuthology.orchestra.run.smithi150.stdout: "take_sum": 2132, 2024-09-21T03:56:43.572 INFO:teuthology.orchestra.run.smithi150.stdout: "put": 2132, 2024-09-21T03:56:43.572 INFO:teuthology.orchestra.run.smithi150.stdout: "put_sum": 2132, 2024-09-21T03:56:43.572 INFO:teuthology.orchestra.run.smithi150.stdout: "wait": { 2024-09-21T03:56:43.572 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-09-21T03:56:43.572 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-09-21T03:56:43.572 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-09-21T03:56:43.572 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T03:56:43.573 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T03:56:43.573 INFO:teuthology.orchestra.run.smithi150.stdout: "throttle-write_buf_throttle": { 2024-09-21T03:56:43.573 INFO:teuthology.orchestra.run.smithi150.stdout: "val": 0, 2024-09-21T03:56:43.573 INFO:teuthology.orchestra.run.smithi150.stdout: "max": 3758096384, 2024-09-21T03:56:43.573 INFO:teuthology.orchestra.run.smithi150.stdout: "get_started": 0, 2024-09-21T03:56:43.573 INFO:teuthology.orchestra.run.smithi150.stdout: "get": 0, 2024-09-21T03:56:43.573 INFO:teuthology.orchestra.run.smithi150.stdout: "get_sum": 0, 2024-09-21T03:56:43.573 INFO:teuthology.orchestra.run.smithi150.stdout: "get_or_fail_fail": 0, 2024-09-21T03:56:43.573 INFO:teuthology.orchestra.run.smithi150.stdout: "get_or_fail_success": 0, 2024-09-21T03:56:43.573 INFO:teuthology.orchestra.run.smithi150.stdout: "take": 0, 2024-09-21T03:56:43.573 INFO:teuthology.orchestra.run.smithi150.stdout: "take_sum": 0, 2024-09-21T03:56:43.573 INFO:teuthology.orchestra.run.smithi150.stdout: "put": 0, 2024-09-21T03:56:43.573 INFO:teuthology.orchestra.run.smithi150.stdout: "put_sum": 0, 2024-09-21T03:56:43.573 INFO:teuthology.orchestra.run.smithi150.stdout: "wait": { 2024-09-21T03:56:43.573 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-09-21T03:56:43.574 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-09-21T03:56:43.574 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-09-21T03:56:43.574 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T03:56:43.574 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T03:56:43.574 INFO:teuthology.orchestra.run.smithi150.stdout: "throttle-write_buf_throttle-0x5654a74b9450": { 2024-09-21T03:56:43.574 INFO:teuthology.orchestra.run.smithi150.stdout: "val": 0, 2024-09-21T03:56:43.574 INFO:teuthology.orchestra.run.smithi150.stdout: "max": 3758096384, 2024-09-21T03:56:43.574 INFO:teuthology.orchestra.run.smithi150.stdout: "get_started": 0, 2024-09-21T03:56:43.574 INFO:teuthology.orchestra.run.smithi150.stdout: "get": 0, 2024-09-21T03:56:43.574 INFO:teuthology.orchestra.run.smithi150.stdout: "get_sum": 0, 2024-09-21T03:56:43.574 INFO:teuthology.orchestra.run.smithi150.stdout: "get_or_fail_fail": 0, 2024-09-21T03:56:43.574 INFO:teuthology.orchestra.run.smithi150.stdout: "get_or_fail_success": 0, 2024-09-21T03:56:43.574 INFO:teuthology.orchestra.run.smithi150.stdout: "take": 0, 2024-09-21T03:56:43.575 INFO:teuthology.orchestra.run.smithi150.stdout: "take_sum": 0, 2024-09-21T03:56:43.575 INFO:teuthology.orchestra.run.smithi150.stdout: "put": 0, 2024-09-21T03:56:43.575 INFO:teuthology.orchestra.run.smithi150.stdout: "put_sum": 0, 2024-09-21T03:56:43.575 INFO:teuthology.orchestra.run.smithi150.stdout: "wait": { 2024-09-21T03:56:43.575 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-09-21T03:56:43.575 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-09-21T03:56:43.575 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-09-21T03:56:43.575 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T03:56:43.575 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T03:56:43.575 INFO:teuthology.orchestra.run.smithi150.stdout: "trackedop": { 2024-09-21T03:56:43.575 INFO:teuthology.orchestra.run.smithi150.stdout: "slow_ops_count": 0 2024-09-21T03:56:43.575 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T03:56:43.575 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-21T03:56:43.718 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:56:43 smithi161 ceph-mon[34533]: pgmap v653: 97 pgs: 97 active+clean; 10 MiB data, 525 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-21T03:56:43.834 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:56:43 smithi039 ceph-mon[31211]: pgmap v653: 97 pgs: 97 active+clean; 10 MiB data, 525 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-21T03:56:44.330 INFO:tasks.check_counter:Daemon mds.e mds.exported=0 2024-09-21T03:56:44.331 INFO:tasks.check_counter:Daemon mds.e mds.imported=0 2024-09-21T03:56:44.331 INFO:tasks.check_counter:Daemon mds.e mds_cache.dir_update=0 2024-09-21T03:56:44.331 INFO:tasks.check_counter:Daemon mds.e mds_cache.dir_update_receipt=0 2024-09-21T03:56:44.331 INFO:tasks.check_counter:Daemon mds.e mds.root_rsnaps=0 2024-09-21T03:56:44.331 INFO:tasks.check_counter:Daemon mds.e mds_server.req_mksnap_latency.avgcount=0 2024-09-21T03:56:44.331 INFO:tasks.check_counter:Daemon mds.e mds_server.req_rmsnap_latency.avgcount=0 2024-09-21T03:56:44.331 DEBUG:tasks.check_counter:Getting stats from h 2024-09-21T03:56:44.331 DEBUG:teuthology.orchestra.run.smithi150:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 51116d5c-77ca-11ef-baf5-efdc52797490 -- ceph daemon mds.h perf dump 2024-09-21T03:56:44.581 INFO:teuthology.orchestra.run.smithi150.stderr:Inferring config /var/lib/ceph/51116d5c-77ca-11ef-baf5-efdc52797490/mon.b/config 2024-09-21T03:56:45.099 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-21T03:56:45.099 INFO:teuthology.orchestra.run.smithi150.stdout: "AsyncMessenger::Worker-0": { 2024-09-21T03:56:45.099 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_recv_messages": 450, 2024-09-21T03:56:45.099 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_send_messages": 546, 2024-09-21T03:56:45.099 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_recv_bytes": 139014, 2024-09-21T03:56:45.099 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_send_bytes": 353657, 2024-09-21T03:56:45.099 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_created_connections": 10, 2024-09-21T03:56:45.099 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_active_connections": 2, 2024-09-21T03:56:45.099 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_running_total_time": 0.079533001, 2024-09-21T03:56:45.099 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_running_send_time": 0.039017796, 2024-09-21T03:56:45.099 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_running_recv_time": 0.030062797, 2024-09-21T03:56:45.100 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_running_fast_dispatch_time": 0.004947064, 2024-09-21T03:56:45.100 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T03:56:45.100 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 546, 2024-09-21T03:56:45.100 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.045637012, 2024-09-21T03:56:45.100 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000083584 2024-09-21T03:56:45.100 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T03:56:45.100 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_handle_ack_lat": { 2024-09-21T03:56:45.100 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 935, 2024-09-21T03:56:45.100 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000208896, 2024-09-21T03:56:45.100 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000223 2024-09-21T03:56:45.100 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T03:56:45.100 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_recv_encrypted_bytes": 0, 2024-09-21T03:56:45.100 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_send_encrypted_bytes": 0 2024-09-21T03:56:45.100 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T03:56:45.101 INFO:teuthology.orchestra.run.smithi150.stdout: "AsyncMessenger::Worker-1": { 2024-09-21T03:56:45.101 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_recv_messages": 301, 2024-09-21T03:56:45.101 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_send_messages": 312, 2024-09-21T03:56:45.101 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_recv_bytes": 1221720, 2024-09-21T03:56:45.101 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_send_bytes": 183324, 2024-09-21T03:56:45.101 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_created_connections": 8, 2024-09-21T03:56:45.101 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_active_connections": 5, 2024-09-21T03:56:45.101 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_running_total_time": 0.064344233, 2024-09-21T03:56:45.101 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_running_send_time": 0.025597162, 2024-09-21T03:56:45.101 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_running_recv_time": 0.051418855, 2024-09-21T03:56:45.101 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_running_fast_dispatch_time": 0.000502762, 2024-09-21T03:56:45.101 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T03:56:45.101 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 312, 2024-09-21T03:56:45.101 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.034130811, 2024-09-21T03:56:45.102 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000109393 2024-09-21T03:56:45.102 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T03:56:45.102 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_handle_ack_lat": { 2024-09-21T03:56:45.102 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 46, 2024-09-21T03:56:45.102 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000011009, 2024-09-21T03:56:45.102 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000239 2024-09-21T03:56:45.102 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T03:56:45.102 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_recv_encrypted_bytes": 1216768, 2024-09-21T03:56:45.102 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_send_encrypted_bytes": 144752 2024-09-21T03:56:45.102 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T03:56:45.102 INFO:teuthology.orchestra.run.smithi150.stdout: "AsyncMessenger::Worker-2": { 2024-09-21T03:56:45.102 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_recv_messages": 66, 2024-09-21T03:56:45.102 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_send_messages": 241, 2024-09-21T03:56:45.103 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_recv_bytes": 12007, 2024-09-21T03:56:45.103 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_send_bytes": 868663, 2024-09-21T03:56:45.103 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_created_connections": 49, 2024-09-21T03:56:45.103 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_active_connections": 9, 2024-09-21T03:56:45.103 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_running_total_time": 0.035505153, 2024-09-21T03:56:45.103 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_running_send_time": 0.012870288, 2024-09-21T03:56:45.103 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_running_recv_time": 0.019696617, 2024-09-21T03:56:45.103 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_running_fast_dispatch_time": 0.000197337, 2024-09-21T03:56:45.103 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T03:56:45.103 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 237, 2024-09-21T03:56:45.103 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.015683303, 2024-09-21T03:56:45.103 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000066174 2024-09-21T03:56:45.103 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T03:56:45.103 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_handle_ack_lat": { 2024-09-21T03:56:45.104 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 93, 2024-09-21T03:56:45.104 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000014971, 2024-09-21T03:56:45.104 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000160 2024-09-21T03:56:45.104 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T03:56:45.104 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_recv_encrypted_bytes": 4832, 2024-09-21T03:56:45.104 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_send_encrypted_bytes": 372976 2024-09-21T03:56:45.104 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T03:56:45.104 INFO:teuthology.orchestra.run.smithi150.stdout: "cct": { 2024-09-21T03:56:45.104 INFO:teuthology.orchestra.run.smithi150.stdout: "total_workers": 1, 2024-09-21T03:56:45.104 INFO:teuthology.orchestra.run.smithi150.stdout: "unhealthy_workers": 0 2024-09-21T03:56:45.104 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T03:56:45.104 INFO:teuthology.orchestra.run.smithi150.stdout: "finisher-MDSRank": { 2024-09-21T03:56:45.104 INFO:teuthology.orchestra.run.smithi150.stdout: "queue_len": 0, 2024-09-21T03:56:45.104 INFO:teuthology.orchestra.run.smithi150.stdout: "complete_latency": { 2024-09-21T03:56:45.105 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 66, 2024-09-21T03:56:45.105 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.001828255, 2024-09-21T03:56:45.105 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000027700 2024-09-21T03:56:45.105 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T03:56:45.105 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T03:56:45.105 INFO:teuthology.orchestra.run.smithi150.stdout: "finisher-PurgeQueue": { 2024-09-21T03:56:45.105 INFO:teuthology.orchestra.run.smithi150.stdout: "queue_len": 0, 2024-09-21T03:56:45.105 INFO:teuthology.orchestra.run.smithi150.stdout: "complete_latency": { 2024-09-21T03:56:45.105 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 2, 2024-09-21T03:56:45.105 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000022077, 2024-09-21T03:56:45.105 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000011038 2024-09-21T03:56:45.105 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T03:56:45.105 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T03:56:45.105 INFO:teuthology.orchestra.run.smithi150.stdout: "mds": { 2024-09-21T03:56:45.105 INFO:teuthology.orchestra.run.smithi150.stdout: "request": 5, 2024-09-21T03:56:45.106 INFO:teuthology.orchestra.run.smithi150.stdout: "reply": 4, 2024-09-21T03:56:45.106 INFO:teuthology.orchestra.run.smithi150.stdout: "reply_latency": { 2024-09-21T03:56:45.106 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 4, 2024-09-21T03:56:45.106 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.002516145, 2024-09-21T03:56:45.106 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000629036 2024-09-21T03:56:45.106 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T03:56:45.106 INFO:teuthology.orchestra.run.smithi150.stdout: "slow_reply": 0, 2024-09-21T03:56:45.106 INFO:teuthology.orchestra.run.smithi150.stdout: "forward": 1, 2024-09-21T03:56:45.106 INFO:teuthology.orchestra.run.smithi150.stdout: "dir_fetch_complete": 0, 2024-09-21T03:56:45.106 INFO:teuthology.orchestra.run.smithi150.stdout: "dir_fetch_keys": 0, 2024-09-21T03:56:45.106 INFO:teuthology.orchestra.run.smithi150.stdout: "dir_commit": 11, 2024-09-21T03:56:45.106 INFO:teuthology.orchestra.run.smithi150.stdout: "dir_split": 0, 2024-09-21T03:56:45.106 INFO:teuthology.orchestra.run.smithi150.stdout: "dir_merge": 0, 2024-09-21T03:56:45.106 INFO:teuthology.orchestra.run.smithi150.stdout: "inodes": 16, 2024-09-21T03:56:45.107 INFO:teuthology.orchestra.run.smithi150.stdout: "inodes_top": 0, 2024-09-21T03:56:45.107 INFO:teuthology.orchestra.run.smithi150.stdout: "inodes_bottom": 16, 2024-09-21T03:56:45.107 INFO:teuthology.orchestra.run.smithi150.stdout: "inodes_pin_tail": 0, 2024-09-21T03:56:45.107 INFO:teuthology.orchestra.run.smithi150.stdout: "inodes_pinned": 13, 2024-09-21T03:56:45.107 INFO:teuthology.orchestra.run.smithi150.stdout: "inodes_expired": 4, 2024-09-21T03:56:45.107 INFO:teuthology.orchestra.run.smithi150.stdout: "inodes_with_caps": 1, 2024-09-21T03:56:45.107 INFO:teuthology.orchestra.run.smithi150.stdout: "caps": 1, 2024-09-21T03:56:45.107 INFO:teuthology.orchestra.run.smithi150.stdout: "subtrees": 3, 2024-09-21T03:56:45.107 INFO:teuthology.orchestra.run.smithi150.stdout: "traverse": 13, 2024-09-21T03:56:45.107 INFO:teuthology.orchestra.run.smithi150.stdout: "traverse_hit": 10, 2024-09-21T03:56:45.107 INFO:teuthology.orchestra.run.smithi150.stdout: "traverse_forward": 0, 2024-09-21T03:56:45.107 INFO:teuthology.orchestra.run.smithi150.stdout: "traverse_discover": 3, 2024-09-21T03:56:45.107 INFO:teuthology.orchestra.run.smithi150.stdout: "traverse_dir_fetch": 0, 2024-09-21T03:56:45.108 INFO:teuthology.orchestra.run.smithi150.stdout: "traverse_remote_ino": 0, 2024-09-21T03:56:45.108 INFO:teuthology.orchestra.run.smithi150.stdout: "traverse_lock": 0, 2024-09-21T03:56:45.108 INFO:teuthology.orchestra.run.smithi150.stdout: "load_cent": 0, 2024-09-21T03:56:45.108 INFO:teuthology.orchestra.run.smithi150.stdout: "q": 0, 2024-09-21T03:56:45.108 INFO:teuthology.orchestra.run.smithi150.stdout: "exported": 0, 2024-09-21T03:56:45.108 INFO:teuthology.orchestra.run.smithi150.stdout: "exported_inodes": 0, 2024-09-21T03:56:45.108 INFO:teuthology.orchestra.run.smithi150.stdout: "imported": 0, 2024-09-21T03:56:45.108 INFO:teuthology.orchestra.run.smithi150.stdout: "imported_inodes": 0, 2024-09-21T03:56:45.108 INFO:teuthology.orchestra.run.smithi150.stdout: "openino_dir_fetch": 0, 2024-09-21T03:56:45.108 INFO:teuthology.orchestra.run.smithi150.stdout: "openino_backtrace_fetch": 0, 2024-09-21T03:56:45.108 INFO:teuthology.orchestra.run.smithi150.stdout: "openino_peer_discover": 0, 2024-09-21T03:56:45.108 INFO:teuthology.orchestra.run.smithi150.stdout: "root_rfiles": 4, 2024-09-21T03:56:45.108 INFO:teuthology.orchestra.run.smithi150.stdout: "root_rbytes": 234, 2024-09-21T03:56:45.108 INFO:teuthology.orchestra.run.smithi150.stdout: "root_rsnaps": 6, 2024-09-21T03:56:45.108 INFO:teuthology.orchestra.run.smithi150.stdout: "scrub_backtrace_fetch": 0, 2024-09-21T03:56:45.109 INFO:teuthology.orchestra.run.smithi150.stdout: "scrub_set_tag": 0, 2024-09-21T03:56:45.109 INFO:teuthology.orchestra.run.smithi150.stdout: "scrub_backtrace_repaired": 0, 2024-09-21T03:56:45.109 INFO:teuthology.orchestra.run.smithi150.stdout: "scrub_inotable_repaired": 0, 2024-09-21T03:56:45.109 INFO:teuthology.orchestra.run.smithi150.stdout: "scrub_dir_inodes": 0, 2024-09-21T03:56:45.109 INFO:teuthology.orchestra.run.smithi150.stdout: "scrub_dir_base_inodes": 0, 2024-09-21T03:56:45.109 INFO:teuthology.orchestra.run.smithi150.stdout: "scrub_dirfrag_rstats": 0, 2024-09-21T03:56:45.109 INFO:teuthology.orchestra.run.smithi150.stdout: "scrub_file_inodes": 0, 2024-09-21T03:56:45.109 INFO:teuthology.orchestra.run.smithi150.stdout: "handle_inode_file_caps": 0, 2024-09-21T03:56:45.109 INFO:teuthology.orchestra.run.smithi150.stdout: "ceph_cap_op_revoke": 0, 2024-09-21T03:56:45.109 INFO:teuthology.orchestra.run.smithi150.stdout: "ceph_cap_op_grant": 0, 2024-09-21T03:56:45.109 INFO:teuthology.orchestra.run.smithi150.stdout: "ceph_cap_op_trunc": 0, 2024-09-21T03:56:45.109 INFO:teuthology.orchestra.run.smithi150.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-21T03:56:45.109 INFO:teuthology.orchestra.run.smithi150.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-21T03:56:45.109 INFO:teuthology.orchestra.run.smithi150.stdout: "handle_client_caps": 0, 2024-09-21T03:56:45.109 INFO:teuthology.orchestra.run.smithi150.stdout: "handle_client_caps_dirty": 0, 2024-09-21T03:56:45.110 INFO:teuthology.orchestra.run.smithi150.stdout: "handle_client_cap_release": 0, 2024-09-21T03:56:45.110 INFO:teuthology.orchestra.run.smithi150.stdout: "process_request_cap_release": 0 2024-09-21T03:56:45.110 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T03:56:45.110 INFO:teuthology.orchestra.run.smithi150.stdout: "mds_cache": { 2024-09-21T03:56:45.110 INFO:teuthology.orchestra.run.smithi150.stdout: "dir_update": 0, 2024-09-21T03:56:45.110 INFO:teuthology.orchestra.run.smithi150.stdout: "dir_update_receipt": 11, 2024-09-21T03:56:45.110 INFO:teuthology.orchestra.run.smithi150.stdout: "dir_try_discover": 7, 2024-09-21T03:56:45.110 INFO:teuthology.orchestra.run.smithi150.stdout: "dir_send_discover": 8, 2024-09-21T03:56:45.110 INFO:teuthology.orchestra.run.smithi150.stdout: "dir_handle_discover": 0, 2024-09-21T03:56:45.110 INFO:teuthology.orchestra.run.smithi150.stdout: "num_strays": 0, 2024-09-21T03:56:45.110 INFO:teuthology.orchestra.run.smithi150.stdout: "num_strays_delayed": 0, 2024-09-21T03:56:45.110 INFO:teuthology.orchestra.run.smithi150.stdout: "num_strays_enqueuing": 0, 2024-09-21T03:56:45.110 INFO:teuthology.orchestra.run.smithi150.stdout: "strays_created": 0, 2024-09-21T03:56:45.111 INFO:teuthology.orchestra.run.smithi150.stdout: "strays_enqueued": 0, 2024-09-21T03:56:45.111 INFO:teuthology.orchestra.run.smithi150.stdout: "strays_reintegrated": 0, 2024-09-21T03:56:45.111 INFO:teuthology.orchestra.run.smithi150.stdout: "strays_migrated": 0, 2024-09-21T03:56:45.111 INFO:teuthology.orchestra.run.smithi150.stdout: "num_recovering_processing": 0, 2024-09-21T03:56:45.111 INFO:teuthology.orchestra.run.smithi150.stdout: "num_recovering_enqueued": 0, 2024-09-21T03:56:45.111 INFO:teuthology.orchestra.run.smithi150.stdout: "num_recovering_prioritized": 0, 2024-09-21T03:56:45.111 INFO:teuthology.orchestra.run.smithi150.stdout: "recovery_started": 0, 2024-09-21T03:56:45.111 INFO:teuthology.orchestra.run.smithi150.stdout: "recovery_completed": 0, 2024-09-21T03:56:45.111 INFO:teuthology.orchestra.run.smithi150.stdout: "ireq_quiesce_path": 0, 2024-09-21T03:56:45.111 INFO:teuthology.orchestra.run.smithi150.stdout: "ireq_quiesce_inode": 0, 2024-09-21T03:56:45.111 INFO:teuthology.orchestra.run.smithi150.stdout: "ireq_enqueue_scrub": 0, 2024-09-21T03:56:45.111 INFO:teuthology.orchestra.run.smithi150.stdout: "ireq_exportdir": 0, 2024-09-21T03:56:45.111 INFO:teuthology.orchestra.run.smithi150.stdout: "ireq_flush": 0, 2024-09-21T03:56:45.111 INFO:teuthology.orchestra.run.smithi150.stdout: "ireq_fragmentdir": 0, 2024-09-21T03:56:45.112 INFO:teuthology.orchestra.run.smithi150.stdout: "ireq_fragstats": 0, 2024-09-21T03:56:45.112 INFO:teuthology.orchestra.run.smithi150.stdout: "ireq_inodestats": 0, 2024-09-21T03:56:45.112 INFO:teuthology.orchestra.run.smithi150.stdout: "uninline_started": 0, 2024-09-21T03:56:45.112 INFO:teuthology.orchestra.run.smithi150.stdout: "uninline_succeeded": 0, 2024-09-21T03:56:45.112 INFO:teuthology.orchestra.run.smithi150.stdout: "uninline_write_failed": 0 2024-09-21T03:56:45.112 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T03:56:45.112 INFO:teuthology.orchestra.run.smithi150.stdout: "mds_log": { 2024-09-21T03:56:45.112 INFO:teuthology.orchestra.run.smithi150.stdout: "evlrg": 0, 2024-09-21T03:56:45.112 INFO:teuthology.orchestra.run.smithi150.stdout: "evadd": 8, 2024-09-21T03:56:45.112 INFO:teuthology.orchestra.run.smithi150.stdout: "evex": 0, 2024-09-21T03:56:45.112 INFO:teuthology.orchestra.run.smithi150.stdout: "evtrm": 0, 2024-09-21T03:56:45.112 INFO:teuthology.orchestra.run.smithi150.stdout: "ev": 8, 2024-09-21T03:56:45.112 INFO:teuthology.orchestra.run.smithi150.stdout: "evexg": 0, 2024-09-21T03:56:45.112 INFO:teuthology.orchestra.run.smithi150.stdout: "evexd": 0, 2024-09-21T03:56:45.112 INFO:teuthology.orchestra.run.smithi150.stdout: "segadd": 2, 2024-09-21T03:56:45.113 INFO:teuthology.orchestra.run.smithi150.stdout: "segex": 0, 2024-09-21T03:56:45.113 INFO:teuthology.orchestra.run.smithi150.stdout: "segtrm": 0, 2024-09-21T03:56:45.113 INFO:teuthology.orchestra.run.smithi150.stdout: "seg": 2, 2024-09-21T03:56:45.113 INFO:teuthology.orchestra.run.smithi150.stdout: "segmjr": 2, 2024-09-21T03:56:45.113 INFO:teuthology.orchestra.run.smithi150.stdout: "segexg": 0, 2024-09-21T03:56:45.113 INFO:teuthology.orchestra.run.smithi150.stdout: "segexd": 0, 2024-09-21T03:56:45.113 INFO:teuthology.orchestra.run.smithi150.stdout: "expos": 4194304, 2024-09-21T03:56:45.113 INFO:teuthology.orchestra.run.smithi150.stdout: "wrpos": 4196655, 2024-09-21T03:56:45.113 INFO:teuthology.orchestra.run.smithi150.stdout: "rdpos": 0, 2024-09-21T03:56:45.113 INFO:teuthology.orchestra.run.smithi150.stdout: "jlat": { 2024-09-21T03:56:45.113 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 7, 2024-09-21T03:56:45.113 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.016322180, 2024-09-21T03:56:45.113 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.002331740 2024-09-21T03:56:45.114 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T03:56:45.114 INFO:teuthology.orchestra.run.smithi150.stdout: "replayed": 0 2024-09-21T03:56:45.114 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T03:56:45.114 INFO:teuthology.orchestra.run.smithi150.stdout: "mds_mem": { 2024-09-21T03:56:45.114 INFO:teuthology.orchestra.run.smithi150.stdout: "ino": 20, 2024-09-21T03:56:45.114 INFO:teuthology.orchestra.run.smithi150.stdout: "ino+": 22, 2024-09-21T03:56:45.114 INFO:teuthology.orchestra.run.smithi150.stdout: "ino-": 2, 2024-09-21T03:56:45.114 INFO:teuthology.orchestra.run.smithi150.stdout: "dir": 19, 2024-09-21T03:56:45.114 INFO:teuthology.orchestra.run.smithi150.stdout: "dir+": 23, 2024-09-21T03:56:45.114 INFO:teuthology.orchestra.run.smithi150.stdout: "dir-": 4, 2024-09-21T03:56:45.114 INFO:teuthology.orchestra.run.smithi150.stdout: "dn": 16, 2024-09-21T03:56:45.114 INFO:teuthology.orchestra.run.smithi150.stdout: "dn+": 20, 2024-09-21T03:56:45.114 INFO:teuthology.orchestra.run.smithi150.stdout: "dn-": 4, 2024-09-21T03:56:45.114 INFO:teuthology.orchestra.run.smithi150.stdout: "cap": 1, 2024-09-21T03:56:45.114 INFO:teuthology.orchestra.run.smithi150.stdout: "cap+": 4, 2024-09-21T03:56:45.115 INFO:teuthology.orchestra.run.smithi150.stdout: "cap-": 3, 2024-09-21T03:56:45.115 INFO:teuthology.orchestra.run.smithi150.stdout: "rss": 119472, 2024-09-21T03:56:45.115 INFO:teuthology.orchestra.run.smithi150.stdout: "heap": 325857280 2024-09-21T03:56:45.115 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T03:56:45.115 INFO:teuthology.orchestra.run.smithi150.stdout: "mds_server": { 2024-09-21T03:56:45.115 INFO:teuthology.orchestra.run.smithi150.stdout: "dispatch_client_request": 8, 2024-09-21T03:56:45.115 INFO:teuthology.orchestra.run.smithi150.stdout: "dispatch_server_request": 0, 2024-09-21T03:56:45.115 INFO:teuthology.orchestra.run.smithi150.stdout: "handle_client_request": 5, 2024-09-21T03:56:45.115 INFO:teuthology.orchestra.run.smithi150.stdout: "handle_client_session": 25, 2024-09-21T03:56:45.115 INFO:teuthology.orchestra.run.smithi150.stdout: "handle_peer_request": 1, 2024-09-21T03:56:45.115 INFO:teuthology.orchestra.run.smithi150.stdout: "req_create_latency": { 2024-09-21T03:56:45.115 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-09-21T03:56:45.115 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-09-21T03:56:45.115 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-09-21T03:56:45.115 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T03:56:45.116 INFO:teuthology.orchestra.run.smithi150.stdout: "req_getattr_latency": { 2024-09-21T03:56:45.116 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 1, 2024-09-21T03:56:45.116 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.001944969, 2024-09-21T03:56:45.116 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.001944969 2024-09-21T03:56:45.116 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T03:56:45.116 INFO:teuthology.orchestra.run.smithi150.stdout: "req_getfilelock_latency": { 2024-09-21T03:56:45.116 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-09-21T03:56:45.116 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-09-21T03:56:45.116 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-09-21T03:56:45.116 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T03:56:45.116 INFO:teuthology.orchestra.run.smithi150.stdout: "req_link_latency": { 2024-09-21T03:56:45.116 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-09-21T03:56:45.116 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-09-21T03:56:45.116 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-09-21T03:56:45.117 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T03:56:45.117 INFO:teuthology.orchestra.run.smithi150.stdout: "req_lookup_latency": { 2024-09-21T03:56:45.117 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 2, 2024-09-21T03:56:45.117 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000433786, 2024-09-21T03:56:45.117 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000216893 2024-09-21T03:56:45.117 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T03:56:45.117 INFO:teuthology.orchestra.run.smithi150.stdout: "req_lookuphash_latency": { 2024-09-21T03:56:45.117 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-09-21T03:56:45.117 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-09-21T03:56:45.117 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-09-21T03:56:45.117 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T03:56:45.117 INFO:teuthology.orchestra.run.smithi150.stdout: "req_lookupino_latency": { 2024-09-21T03:56:45.117 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 1, 2024-09-21T03:56:45.117 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000137390, 2024-09-21T03:56:45.117 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000137390 2024-09-21T03:56:45.118 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T03:56:45.118 INFO:teuthology.orchestra.run.smithi150.stdout: "req_lookupname_latency": { 2024-09-21T03:56:45.118 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-09-21T03:56:45.118 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-09-21T03:56:45.118 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-09-21T03:56:45.118 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T03:56:45.118 INFO:teuthology.orchestra.run.smithi150.stdout: "req_lookupparent_latency": { 2024-09-21T03:56:45.118 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-09-21T03:56:45.118 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-09-21T03:56:45.118 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-09-21T03:56:45.118 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T03:56:45.118 INFO:teuthology.orchestra.run.smithi150.stdout: "req_lookupsnap_latency": { 2024-09-21T03:56:45.118 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-09-21T03:56:45.118 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-09-21T03:56:45.118 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-09-21T03:56:45.119 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T03:56:45.119 INFO:teuthology.orchestra.run.smithi150.stdout: "req_lssnap_latency": { 2024-09-21T03:56:45.119 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-09-21T03:56:45.119 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-09-21T03:56:45.119 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-09-21T03:56:45.119 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T03:56:45.119 INFO:teuthology.orchestra.run.smithi150.stdout: "req_mkdir_latency": { 2024-09-21T03:56:45.120 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-09-21T03:56:45.120 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-09-21T03:56:45.120 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-09-21T03:56:45.120 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T03:56:45.120 INFO:teuthology.orchestra.run.smithi150.stdout: "req_mknod_latency": { 2024-09-21T03:56:45.120 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-09-21T03:56:45.120 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-09-21T03:56:45.120 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-09-21T03:56:45.120 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T03:56:45.120 INFO:teuthology.orchestra.run.smithi150.stdout: "req_mksnap_latency": { 2024-09-21T03:56:45.120 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-09-21T03:56:45.120 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-09-21T03:56:45.120 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-09-21T03:56:45.120 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T03:56:45.121 INFO:teuthology.orchestra.run.smithi150.stdout: "req_open_latency": { 2024-09-21T03:56:45.121 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-09-21T03:56:45.121 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-09-21T03:56:45.121 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-09-21T03:56:45.121 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T03:56:45.121 INFO:teuthology.orchestra.run.smithi150.stdout: "req_readdir_latency": { 2024-09-21T03:56:45.121 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-09-21T03:56:45.121 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-09-21T03:56:45.121 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-09-21T03:56:45.121 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T03:56:45.121 INFO:teuthology.orchestra.run.smithi150.stdout: "req_rename_latency": { 2024-09-21T03:56:45.121 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-09-21T03:56:45.121 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-09-21T03:56:45.121 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-09-21T03:56:45.122 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T03:56:45.122 INFO:teuthology.orchestra.run.smithi150.stdout: "req_renamesnap_latency": { 2024-09-21T03:56:45.122 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-09-21T03:56:45.122 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-09-21T03:56:45.122 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-09-21T03:56:45.122 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T03:56:45.122 INFO:teuthology.orchestra.run.smithi150.stdout: "req_snapdiff_latency": { 2024-09-21T03:56:45.122 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-09-21T03:56:45.122 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-09-21T03:56:45.122 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-09-21T03:56:45.122 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T03:56:45.122 INFO:teuthology.orchestra.run.smithi150.stdout: "req_rmdir_latency": { 2024-09-21T03:56:45.122 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-09-21T03:56:45.122 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-09-21T03:56:45.123 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-09-21T03:56:45.123 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T03:56:45.123 INFO:teuthology.orchestra.run.smithi150.stdout: "req_rmsnap_latency": { 2024-09-21T03:56:45.123 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-09-21T03:56:45.123 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-09-21T03:56:45.123 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-09-21T03:56:45.123 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T03:56:45.123 INFO:teuthology.orchestra.run.smithi150.stdout: "req_rmxattr_latency": { 2024-09-21T03:56:45.123 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-09-21T03:56:45.123 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-09-21T03:56:45.123 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-09-21T03:56:45.123 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T03:56:45.123 INFO:teuthology.orchestra.run.smithi150.stdout: "req_setattr_latency": { 2024-09-21T03:56:45.123 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-09-21T03:56:45.123 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-09-21T03:56:45.124 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-09-21T03:56:45.124 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T03:56:45.124 INFO:teuthology.orchestra.run.smithi150.stdout: "req_setdirlayout_latency": { 2024-09-21T03:56:45.124 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-09-21T03:56:45.124 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-09-21T03:56:45.124 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-09-21T03:56:45.124 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T03:56:45.124 INFO:teuthology.orchestra.run.smithi150.stdout: "req_setfilelock_latency": { 2024-09-21T03:56:45.124 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-09-21T03:56:45.124 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-09-21T03:56:45.124 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-09-21T03:56:45.124 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T03:56:45.124 INFO:teuthology.orchestra.run.smithi150.stdout: "req_setlayout_latency": { 2024-09-21T03:56:45.124 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-09-21T03:56:45.124 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-09-21T03:56:45.125 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-09-21T03:56:45.125 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T03:56:45.125 INFO:teuthology.orchestra.run.smithi150.stdout: "req_setxattr_latency": { 2024-09-21T03:56:45.125 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-09-21T03:56:45.125 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-09-21T03:56:45.125 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-09-21T03:56:45.125 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T03:56:45.125 INFO:teuthology.orchestra.run.smithi150.stdout: "req_symlink_latency": { 2024-09-21T03:56:45.125 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-09-21T03:56:45.125 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-09-21T03:56:45.125 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-09-21T03:56:45.125 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T03:56:45.125 INFO:teuthology.orchestra.run.smithi150.stdout: "req_unlink_latency": { 2024-09-21T03:56:45.125 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-09-21T03:56:45.126 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-09-21T03:56:45.126 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-09-21T03:56:45.126 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T03:56:45.126 INFO:teuthology.orchestra.run.smithi150.stdout: "cap_revoke_eviction": 0, 2024-09-21T03:56:45.126 INFO:teuthology.orchestra.run.smithi150.stdout: "cap_acquisition_throttle": 0, 2024-09-21T03:56:45.126 INFO:teuthology.orchestra.run.smithi150.stdout: "req_getvxattr_latency": { 2024-09-21T03:56:45.126 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-09-21T03:56:45.126 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-09-21T03:56:45.126 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-09-21T03:56:45.126 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T03:56:45.126 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T03:56:45.126 INFO:teuthology.orchestra.run.smithi150.stdout: "mds_sessions": { 2024-09-21T03:56:45.126 INFO:teuthology.orchestra.run.smithi150.stdout: "session_count": 2, 2024-09-21T03:56:45.126 INFO:teuthology.orchestra.run.smithi150.stdout: "session_add": 4, 2024-09-21T03:56:45.126 INFO:teuthology.orchestra.run.smithi150.stdout: "session_remove": 2, 2024-09-21T03:56:45.127 INFO:teuthology.orchestra.run.smithi150.stdout: "sessions_open": 2, 2024-09-21T03:56:45.127 INFO:teuthology.orchestra.run.smithi150.stdout: "sessions_stale": 0, 2024-09-21T03:56:45.127 INFO:teuthology.orchestra.run.smithi150.stdout: "total_load": 3, 2024-09-21T03:56:45.127 INFO:teuthology.orchestra.run.smithi150.stdout: "average_load": 1, 2024-09-21T03:56:45.127 INFO:teuthology.orchestra.run.smithi150.stdout: "avg_session_uptime": 604, 2024-09-21T03:56:45.127 INFO:teuthology.orchestra.run.smithi150.stdout: "mdthresh_evicted": 0 2024-09-21T03:56:45.127 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T03:56:45.127 INFO:teuthology.orchestra.run.smithi150.stdout: "mempool": { 2024-09-21T03:56:45.127 INFO:teuthology.orchestra.run.smithi150.stdout: "bloom_filter_bytes": 0, 2024-09-21T03:56:45.127 INFO:teuthology.orchestra.run.smithi150.stdout: "bloom_filter_items": 0, 2024-09-21T03:56:45.127 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_alloc_bytes": 0, 2024-09-21T03:56:45.127 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_alloc_items": 0, 2024-09-21T03:56:45.127 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_cache_data_bytes": 0, 2024-09-21T03:56:45.127 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_cache_data_items": 0, 2024-09-21T03:56:45.127 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-21T03:56:45.128 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_cache_onode_items": 0, 2024-09-21T03:56:45.128 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-21T03:56:45.128 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_cache_meta_items": 0, 2024-09-21T03:56:45.128 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_cache_other_bytes": 0, 2024-09-21T03:56:45.128 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_cache_other_items": 0, 2024-09-21T03:56:45.128 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-21T03:56:45.128 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_cache_buffer_items": 0, 2024-09-21T03:56:45.128 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_extent_bytes": 0, 2024-09-21T03:56:45.128 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_extent_items": 0, 2024-09-21T03:56:45.128 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_blob_bytes": 0, 2024-09-21T03:56:45.128 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_blob_items": 0, 2024-09-21T03:56:45.128 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-21T03:56:45.128 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_shared_blob_items": 0, 2024-09-21T03:56:45.128 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-21T03:56:45.128 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_inline_bl_items": 0, 2024-09-21T03:56:45.129 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_fsck_bytes": 0, 2024-09-21T03:56:45.129 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_fsck_items": 0, 2024-09-21T03:56:45.129 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_txc_bytes": 0, 2024-09-21T03:56:45.129 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_txc_items": 0, 2024-09-21T03:56:45.129 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-21T03:56:45.129 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_writing_deferred_items": 0, 2024-09-21T03:56:45.129 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_writing_bytes": 0, 2024-09-21T03:56:45.129 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_writing_items": 0, 2024-09-21T03:56:45.129 INFO:teuthology.orchestra.run.smithi150.stdout: "bluefs_bytes": 0, 2024-09-21T03:56:45.129 INFO:teuthology.orchestra.run.smithi150.stdout: "bluefs_items": 0, 2024-09-21T03:56:45.129 INFO:teuthology.orchestra.run.smithi150.stdout: "bluefs_file_reader_bytes": 0, 2024-09-21T03:56:45.129 INFO:teuthology.orchestra.run.smithi150.stdout: "bluefs_file_reader_items": 0, 2024-09-21T03:56:45.129 INFO:teuthology.orchestra.run.smithi150.stdout: "bluefs_file_writer_bytes": 0, 2024-09-21T03:56:45.129 INFO:teuthology.orchestra.run.smithi150.stdout: "bluefs_file_writer_items": 0, 2024-09-21T03:56:45.129 INFO:teuthology.orchestra.run.smithi150.stdout: "buffer_anon_bytes": 255799, 2024-09-21T03:56:45.130 INFO:teuthology.orchestra.run.smithi150.stdout: "buffer_anon_items": 93, 2024-09-21T03:56:45.130 INFO:teuthology.orchestra.run.smithi150.stdout: "buffer_meta_bytes": 88, 2024-09-21T03:56:45.130 INFO:teuthology.orchestra.run.smithi150.stdout: "buffer_meta_items": 1, 2024-09-21T03:56:45.130 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_bytes": 0, 2024-09-21T03:56:45.130 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_items": 0, 2024-09-21T03:56:45.130 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_mapbl_bytes": 0, 2024-09-21T03:56:45.130 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_mapbl_items": 0, 2024-09-21T03:56:45.130 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_pglog_bytes": 0, 2024-09-21T03:56:45.130 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_pglog_items": 0, 2024-09-21T03:56:45.130 INFO:teuthology.orchestra.run.smithi150.stdout: "osdmap_bytes": 8256, 2024-09-21T03:56:45.130 INFO:teuthology.orchestra.run.smithi150.stdout: "osdmap_items": 182, 2024-09-21T03:56:45.130 INFO:teuthology.orchestra.run.smithi150.stdout: "osdmap_mapping_bytes": 0, 2024-09-21T03:56:45.130 INFO:teuthology.orchestra.run.smithi150.stdout: "osdmap_mapping_items": 0, 2024-09-21T03:56:45.130 INFO:teuthology.orchestra.run.smithi150.stdout: "pgmap_bytes": 0, 2024-09-21T03:56:45.130 INFO:teuthology.orchestra.run.smithi150.stdout: "pgmap_items": 0, 2024-09-21T03:56:45.131 INFO:teuthology.orchestra.run.smithi150.stdout: "mds_co_bytes": 86481, 2024-09-21T03:56:45.131 INFO:teuthology.orchestra.run.smithi150.stdout: "mds_co_items": 433, 2024-09-21T03:56:45.131 INFO:teuthology.orchestra.run.smithi150.stdout: "unittest_1_bytes": 0, 2024-09-21T03:56:45.131 INFO:teuthology.orchestra.run.smithi150.stdout: "unittest_1_items": 0, 2024-09-21T03:56:45.131 INFO:teuthology.orchestra.run.smithi150.stdout: "unittest_2_bytes": 0, 2024-09-21T03:56:45.131 INFO:teuthology.orchestra.run.smithi150.stdout: "unittest_2_items": 0 2024-09-21T03:56:45.131 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T03:56:45.131 INFO:teuthology.orchestra.run.smithi150.stdout: "objecter": { 2024-09-21T03:56:45.131 INFO:teuthology.orchestra.run.smithi150.stdout: "op_active": 0, 2024-09-21T03:56:45.131 INFO:teuthology.orchestra.run.smithi150.stdout: "op_laggy": 0, 2024-09-21T03:56:45.131 INFO:teuthology.orchestra.run.smithi150.stdout: "op_send": 48, 2024-09-21T03:56:45.131 INFO:teuthology.orchestra.run.smithi150.stdout: "op_send_bytes": 13249, 2024-09-21T03:56:45.131 INFO:teuthology.orchestra.run.smithi150.stdout: "op_resend": 33, 2024-09-21T03:56:45.131 INFO:teuthology.orchestra.run.smithi150.stdout: "op_reply": 48, 2024-09-21T03:56:45.131 INFO:teuthology.orchestra.run.smithi150.stdout: "op_latency": { 2024-09-21T03:56:45.132 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 48, 2024-09-21T03:56:45.132 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.232995208, 2024-09-21T03:56:45.132 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.004854066 2024-09-21T03:56:45.132 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T03:56:45.132 INFO:teuthology.orchestra.run.smithi150.stdout: "op_inflight": 0, 2024-09-21T03:56:45.132 INFO:teuthology.orchestra.run.smithi150.stdout: "oplen_avg": { 2024-09-21T03:56:45.132 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 48, 2024-09-21T03:56:45.132 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 73 2024-09-21T03:56:45.132 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T03:56:45.132 INFO:teuthology.orchestra.run.smithi150.stdout: "op": 48, 2024-09-21T03:56:45.132 INFO:teuthology.orchestra.run.smithi150.stdout: "op_r": 0, 2024-09-21T03:56:45.132 INFO:teuthology.orchestra.run.smithi150.stdout: "op_w": 48, 2024-09-21T03:56:45.132 INFO:teuthology.orchestra.run.smithi150.stdout: "op_rmw": 0, 2024-09-21T03:56:45.132 INFO:teuthology.orchestra.run.smithi150.stdout: "op_pg": 0, 2024-09-21T03:56:45.132 INFO:teuthology.orchestra.run.smithi150.stdout: "osdop_stat": 0, 2024-09-21T03:56:45.133 INFO:teuthology.orchestra.run.smithi150.stdout: "osdop_create": 10, 2024-09-21T03:56:45.133 INFO:teuthology.orchestra.run.smithi150.stdout: "osdop_read": 0, 2024-09-21T03:56:45.133 INFO:teuthology.orchestra.run.smithi150.stdout: "osdop_write": 7, 2024-09-21T03:56:45.133 INFO:teuthology.orchestra.run.smithi150.stdout: "osdop_writefull": 9, 2024-09-21T03:56:45.133 INFO:teuthology.orchestra.run.smithi150.stdout: "osdop_writesame": 0, 2024-09-21T03:56:45.133 INFO:teuthology.orchestra.run.smithi150.stdout: "osdop_append": 0, 2024-09-21T03:56:45.133 INFO:teuthology.orchestra.run.smithi150.stdout: "osdop_zero": 0, 2024-09-21T03:56:45.133 INFO:teuthology.orchestra.run.smithi150.stdout: "osdop_truncate": 0, 2024-09-21T03:56:45.133 INFO:teuthology.orchestra.run.smithi150.stdout: "osdop_delete": 6, 2024-09-21T03:56:45.133 INFO:teuthology.orchestra.run.smithi150.stdout: "osdop_mapext": 0, 2024-09-21T03:56:45.133 INFO:teuthology.orchestra.run.smithi150.stdout: "osdop_sparse_read": 0, 2024-09-21T03:56:45.133 INFO:teuthology.orchestra.run.smithi150.stdout: "osdop_clonerange": 0, 2024-09-21T03:56:45.133 INFO:teuthology.orchestra.run.smithi150.stdout: "osdop_getxattr": 0, 2024-09-21T03:56:45.133 INFO:teuthology.orchestra.run.smithi150.stdout: "osdop_setxattr": 20, 2024-09-21T03:56:45.133 INFO:teuthology.orchestra.run.smithi150.stdout: "osdop_cmpxattr": 0, 2024-09-21T03:56:45.134 INFO:teuthology.orchestra.run.smithi150.stdout: "osdop_rmxattr": 0, 2024-09-21T03:56:45.134 INFO:teuthology.orchestra.run.smithi150.stdout: "osdop_resetxattrs": 0, 2024-09-21T03:56:45.134 INFO:teuthology.orchestra.run.smithi150.stdout: "osdop_call": 0, 2024-09-21T03:56:45.134 INFO:teuthology.orchestra.run.smithi150.stdout: "osdop_watch": 0, 2024-09-21T03:56:45.134 INFO:teuthology.orchestra.run.smithi150.stdout: "osdop_notify": 0, 2024-09-21T03:56:45.134 INFO:teuthology.orchestra.run.smithi150.stdout: "osdop_src_cmpxattr": 0, 2024-09-21T03:56:45.134 INFO:teuthology.orchestra.run.smithi150.stdout: "osdop_pgls": 0, 2024-09-21T03:56:45.134 INFO:teuthology.orchestra.run.smithi150.stdout: "osdop_pgls_filter": 0, 2024-09-21T03:56:45.134 INFO:teuthology.orchestra.run.smithi150.stdout: "osdop_other": 0, 2024-09-21T03:56:45.134 INFO:teuthology.orchestra.run.smithi150.stdout: "linger_active": 0, 2024-09-21T03:56:45.134 INFO:teuthology.orchestra.run.smithi150.stdout: "linger_send": 0, 2024-09-21T03:56:45.134 INFO:teuthology.orchestra.run.smithi150.stdout: "linger_resend": 0, 2024-09-21T03:56:45.134 INFO:teuthology.orchestra.run.smithi150.stdout: "linger_ping": 0, 2024-09-21T03:56:45.134 INFO:teuthology.orchestra.run.smithi150.stdout: "poolop_active": 0, 2024-09-21T03:56:45.135 INFO:teuthology.orchestra.run.smithi150.stdout: "poolop_send": 0, 2024-09-21T03:56:45.135 INFO:teuthology.orchestra.run.smithi150.stdout: "poolop_resend": 0, 2024-09-21T03:56:45.135 INFO:teuthology.orchestra.run.smithi150.stdout: "poolstat_active": 0, 2024-09-21T03:56:45.135 INFO:teuthology.orchestra.run.smithi150.stdout: "poolstat_send": 0, 2024-09-21T03:56:45.135 INFO:teuthology.orchestra.run.smithi150.stdout: "poolstat_resend": 0, 2024-09-21T03:56:45.135 INFO:teuthology.orchestra.run.smithi150.stdout: "statfs_active": 0, 2024-09-21T03:56:45.135 INFO:teuthology.orchestra.run.smithi150.stdout: "statfs_send": 0, 2024-09-21T03:56:45.135 INFO:teuthology.orchestra.run.smithi150.stdout: "statfs_resend": 0, 2024-09-21T03:56:45.135 INFO:teuthology.orchestra.run.smithi150.stdout: "command_active": 0, 2024-09-21T03:56:45.135 INFO:teuthology.orchestra.run.smithi150.stdout: "command_send": 0, 2024-09-21T03:56:45.135 INFO:teuthology.orchestra.run.smithi150.stdout: "command_resend": 0, 2024-09-21T03:56:45.135 INFO:teuthology.orchestra.run.smithi150.stdout: "map_epoch": 87, 2024-09-21T03:56:45.135 INFO:teuthology.orchestra.run.smithi150.stdout: "map_full": 0, 2024-09-21T03:56:45.135 INFO:teuthology.orchestra.run.smithi150.stdout: "map_inc": 10, 2024-09-21T03:56:45.135 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_sessions": 8, 2024-09-21T03:56:45.136 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_session_open": 8, 2024-09-21T03:56:45.136 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_session_close": 0, 2024-09-21T03:56:45.136 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_laggy": 0, 2024-09-21T03:56:45.136 INFO:teuthology.orchestra.run.smithi150.stdout: "omap_wr": 20, 2024-09-21T03:56:45.136 INFO:teuthology.orchestra.run.smithi150.stdout: "omap_rd": 0, 2024-09-21T03:56:45.136 INFO:teuthology.orchestra.run.smithi150.stdout: "omap_del": 1 2024-09-21T03:56:45.136 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T03:56:45.136 INFO:teuthology.orchestra.run.smithi150.stdout: "oft": { 2024-09-21T03:56:45.136 INFO:teuthology.orchestra.run.smithi150.stdout: "omap_total_objs": 1, 2024-09-21T03:56:45.136 INFO:teuthology.orchestra.run.smithi150.stdout: "omap_total_kv_pairs": 0, 2024-09-21T03:56:45.136 INFO:teuthology.orchestra.run.smithi150.stdout: "omap_total_updates": 0, 2024-09-21T03:56:45.136 INFO:teuthology.orchestra.run.smithi150.stdout: "omap_total_removes": 0 2024-09-21T03:56:45.136 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T03:56:45.136 INFO:teuthology.orchestra.run.smithi150.stdout: "purge_queue": { 2024-09-21T03:56:45.137 INFO:teuthology.orchestra.run.smithi150.stdout: "pq_executing_ops": 0, 2024-09-21T03:56:45.137 INFO:teuthology.orchestra.run.smithi150.stdout: "pq_executing_ops_high_water": 0, 2024-09-21T03:56:45.137 INFO:teuthology.orchestra.run.smithi150.stdout: "pq_executing": 0, 2024-09-21T03:56:45.137 INFO:teuthology.orchestra.run.smithi150.stdout: "pq_executing_high_water": 0, 2024-09-21T03:56:45.137 INFO:teuthology.orchestra.run.smithi150.stdout: "pq_executed_ops": 0, 2024-09-21T03:56:45.137 INFO:teuthology.orchestra.run.smithi150.stdout: "pq_executed": 0, 2024-09-21T03:56:45.137 INFO:teuthology.orchestra.run.smithi150.stdout: "pq_item_in_journal": 0 2024-09-21T03:56:45.137 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T03:56:45.137 INFO:teuthology.orchestra.run.smithi150.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-21T03:56:45.137 INFO:teuthology.orchestra.run.smithi150.stdout: "val": 0, 2024-09-21T03:56:45.137 INFO:teuthology.orchestra.run.smithi150.stdout: "max": 104857600, 2024-09-21T03:56:45.137 INFO:teuthology.orchestra.run.smithi150.stdout: "get_started": 0, 2024-09-21T03:56:45.137 INFO:teuthology.orchestra.run.smithi150.stdout: "get": 813, 2024-09-21T03:56:45.137 INFO:teuthology.orchestra.run.smithi150.stdout: "get_sum": 1281679, 2024-09-21T03:56:45.137 INFO:teuthology.orchestra.run.smithi150.stdout: "get_or_fail_fail": 0, 2024-09-21T03:56:45.138 INFO:teuthology.orchestra.run.smithi150.stdout: "get_or_fail_success": 813, 2024-09-21T03:56:45.138 INFO:teuthology.orchestra.run.smithi150.stdout: "take": 0, 2024-09-21T03:56:45.138 INFO:teuthology.orchestra.run.smithi150.stdout: "take_sum": 0, 2024-09-21T03:56:45.138 INFO:teuthology.orchestra.run.smithi150.stdout: "put": 813, 2024-09-21T03:56:45.138 INFO:teuthology.orchestra.run.smithi150.stdout: "put_sum": 1281679, 2024-09-21T03:56:45.138 INFO:teuthology.orchestra.run.smithi150.stdout: "wait": { 2024-09-21T03:56:45.138 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-09-21T03:56:45.138 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-09-21T03:56:45.138 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-09-21T03:56:45.138 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T03:56:45.138 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T03:56:45.138 INFO:teuthology.orchestra.run.smithi150.stdout: "throttle-objecter_bytes": { 2024-09-21T03:56:45.138 INFO:teuthology.orchestra.run.smithi150.stdout: "val": 0, 2024-09-21T03:56:45.138 INFO:teuthology.orchestra.run.smithi150.stdout: "max": 104857600, 2024-09-21T03:56:45.138 INFO:teuthology.orchestra.run.smithi150.stdout: "get_started": 0, 2024-09-21T03:56:45.139 INFO:teuthology.orchestra.run.smithi150.stdout: "get": 0, 2024-09-21T03:56:45.139 INFO:teuthology.orchestra.run.smithi150.stdout: "get_sum": 0, 2024-09-21T03:56:45.139 INFO:teuthology.orchestra.run.smithi150.stdout: "get_or_fail_fail": 0, 2024-09-21T03:56:45.139 INFO:teuthology.orchestra.run.smithi150.stdout: "get_or_fail_success": 0, 2024-09-21T03:56:45.139 INFO:teuthology.orchestra.run.smithi150.stdout: "take": 48, 2024-09-21T03:56:45.139 INFO:teuthology.orchestra.run.smithi150.stdout: "take_sum": 13249, 2024-09-21T03:56:45.139 INFO:teuthology.orchestra.run.smithi150.stdout: "put": 48, 2024-09-21T03:56:45.139 INFO:teuthology.orchestra.run.smithi150.stdout: "put_sum": 13249, 2024-09-21T03:56:45.139 INFO:teuthology.orchestra.run.smithi150.stdout: "wait": { 2024-09-21T03:56:45.139 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-09-21T03:56:45.139 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-09-21T03:56:45.139 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-09-21T03:56:45.139 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T03:56:45.139 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T03:56:45.140 INFO:teuthology.orchestra.run.smithi150.stdout: "throttle-objecter_ops": { 2024-09-21T03:56:45.140 INFO:teuthology.orchestra.run.smithi150.stdout: "val": 0, 2024-09-21T03:56:45.140 INFO:teuthology.orchestra.run.smithi150.stdout: "max": 1024, 2024-09-21T03:56:45.140 INFO:teuthology.orchestra.run.smithi150.stdout: "get_started": 0, 2024-09-21T03:56:45.140 INFO:teuthology.orchestra.run.smithi150.stdout: "get": 0, 2024-09-21T03:56:45.140 INFO:teuthology.orchestra.run.smithi150.stdout: "get_sum": 0, 2024-09-21T03:56:45.140 INFO:teuthology.orchestra.run.smithi150.stdout: "get_or_fail_fail": 0, 2024-09-21T03:56:45.140 INFO:teuthology.orchestra.run.smithi150.stdout: "get_or_fail_success": 0, 2024-09-21T03:56:45.140 INFO:teuthology.orchestra.run.smithi150.stdout: "take": 48, 2024-09-21T03:56:45.140 INFO:teuthology.orchestra.run.smithi150.stdout: "take_sum": 48, 2024-09-21T03:56:45.140 INFO:teuthology.orchestra.run.smithi150.stdout: "put": 48, 2024-09-21T03:56:45.140 INFO:teuthology.orchestra.run.smithi150.stdout: "put_sum": 48, 2024-09-21T03:56:45.140 INFO:teuthology.orchestra.run.smithi150.stdout: "wait": { 2024-09-21T03:56:45.140 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-09-21T03:56:45.140 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-09-21T03:56:45.141 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-09-21T03:56:45.141 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T03:56:45.141 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T03:56:45.141 INFO:teuthology.orchestra.run.smithi150.stdout: "throttle-write_buf_throttle": { 2024-09-21T03:56:45.141 INFO:teuthology.orchestra.run.smithi150.stdout: "val": 0, 2024-09-21T03:56:45.141 INFO:teuthology.orchestra.run.smithi150.stdout: "max": 3758096384, 2024-09-21T03:56:45.141 INFO:teuthology.orchestra.run.smithi150.stdout: "get_started": 0, 2024-09-21T03:56:45.141 INFO:teuthology.orchestra.run.smithi150.stdout: "get": 0, 2024-09-21T03:56:45.141 INFO:teuthology.orchestra.run.smithi150.stdout: "get_sum": 0, 2024-09-21T03:56:45.141 INFO:teuthology.orchestra.run.smithi150.stdout: "get_or_fail_fail": 0, 2024-09-21T03:56:45.141 INFO:teuthology.orchestra.run.smithi150.stdout: "get_or_fail_success": 0, 2024-09-21T03:56:45.141 INFO:teuthology.orchestra.run.smithi150.stdout: "take": 0, 2024-09-21T03:56:45.141 INFO:teuthology.orchestra.run.smithi150.stdout: "take_sum": 0, 2024-09-21T03:56:45.141 INFO:teuthology.orchestra.run.smithi150.stdout: "put": 0, 2024-09-21T03:56:45.141 INFO:teuthology.orchestra.run.smithi150.stdout: "put_sum": 0, 2024-09-21T03:56:45.142 INFO:teuthology.orchestra.run.smithi150.stdout: "wait": { 2024-09-21T03:56:45.142 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-09-21T03:56:45.142 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-09-21T03:56:45.142 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-09-21T03:56:45.142 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T03:56:45.142 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T03:56:45.142 INFO:teuthology.orchestra.run.smithi150.stdout: "throttle-write_buf_throttle-0x55f576eb1380": { 2024-09-21T03:56:45.142 INFO:teuthology.orchestra.run.smithi150.stdout: "val": 0, 2024-09-21T03:56:45.142 INFO:teuthology.orchestra.run.smithi150.stdout: "max": 3758096384, 2024-09-21T03:56:45.142 INFO:teuthology.orchestra.run.smithi150.stdout: "get_started": 0, 2024-09-21T03:56:45.142 INFO:teuthology.orchestra.run.smithi150.stdout: "get": 8, 2024-09-21T03:56:45.142 INFO:teuthology.orchestra.run.smithi150.stdout: "get_sum": 2351, 2024-09-21T03:56:45.142 INFO:teuthology.orchestra.run.smithi150.stdout: "get_or_fail_fail": 0, 2024-09-21T03:56:45.142 INFO:teuthology.orchestra.run.smithi150.stdout: "get_or_fail_success": 8, 2024-09-21T03:56:45.143 INFO:teuthology.orchestra.run.smithi150.stdout: "take": 0, 2024-09-21T03:56:45.143 INFO:teuthology.orchestra.run.smithi150.stdout: "take_sum": 0, 2024-09-21T03:56:45.143 INFO:teuthology.orchestra.run.smithi150.stdout: "put": 7, 2024-09-21T03:56:45.143 INFO:teuthology.orchestra.run.smithi150.stdout: "put_sum": 2351, 2024-09-21T03:56:45.143 INFO:teuthology.orchestra.run.smithi150.stdout: "wait": { 2024-09-21T03:56:45.143 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-09-21T03:56:45.143 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-09-21T03:56:45.143 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-09-21T03:56:45.143 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T03:56:45.143 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T03:56:45.143 INFO:teuthology.orchestra.run.smithi150.stdout: "trackedop": { 2024-09-21T03:56:45.143 INFO:teuthology.orchestra.run.smithi150.stdout: "slow_ops_count": 0 2024-09-21T03:56:45.143 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T03:56:45.143 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-21T03:56:45.718 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:56:45 smithi161 ceph-mon[34533]: pgmap v654: 97 pgs: 97 active+clean; 10 MiB data, 525 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-09-21T03:56:45.724 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:56:45 smithi150 ceph-mon[34289]: pgmap v654: 97 pgs: 97 active+clean; 10 MiB data, 525 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-09-21T03:56:45.834 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:56:45 smithi039 ceph-mon[31211]: pgmap v654: 97 pgs: 97 active+clean; 10 MiB data, 525 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-09-21T03:56:45.980 INFO:tasks.check_counter:Daemon mds.h mds.exported=0 2024-09-21T03:56:45.980 INFO:tasks.check_counter:Daemon mds.h mds.imported=0 2024-09-21T03:56:45.980 INFO:tasks.check_counter:Daemon mds.h mds_cache.dir_update=0 2024-09-21T03:56:45.980 INFO:tasks.check_counter:Daemon mds.h mds_cache.dir_update_receipt=11 2024-09-21T03:56:45.980 INFO:tasks.check_counter:Daemon mds.h mds.root_rsnaps=6 2024-09-21T03:56:45.981 INFO:tasks.check_counter:Daemon mds.h mds_server.req_mksnap_latency.avgcount=0 2024-09-21T03:56:45.981 INFO:tasks.check_counter:Daemon mds.h mds_server.req_rmsnap_latency.avgcount=0 2024-09-21T03:56:45.981 DEBUG:tasks.check_counter:Getting stats from k 2024-09-21T03:56:45.981 DEBUG:teuthology.orchestra.run.smithi150:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 51116d5c-77ca-11ef-baf5-efdc52797490 -- ceph daemon mds.k perf dump 2024-09-21T03:56:46.227 INFO:teuthology.orchestra.run.smithi150.stderr:Inferring config /var/lib/ceph/51116d5c-77ca-11ef-baf5-efdc52797490/mon.b/config 2024-09-21T03:56:46.753 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-21T03:56:46.753 INFO:teuthology.orchestra.run.smithi150.stdout: "AsyncMessenger::Worker-0": { 2024-09-21T03:56:46.753 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_recv_messages": 980, 2024-09-21T03:56:46.753 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_send_messages": 968, 2024-09-21T03:56:46.753 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_recv_bytes": 1448672, 2024-09-21T03:56:46.753 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_send_bytes": 368694, 2024-09-21T03:56:46.753 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_created_connections": 5, 2024-09-21T03:56:46.753 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_active_connections": 1, 2024-09-21T03:56:46.753 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_running_total_time": 0.164236438, 2024-09-21T03:56:46.753 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_running_send_time": 0.080664036, 2024-09-21T03:56:46.753 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_running_recv_time": 0.063195066, 2024-09-21T03:56:46.753 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_running_fast_dispatch_time": 0.023801521, 2024-09-21T03:56:46.754 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T03:56:46.754 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 967, 2024-09-21T03:56:46.754 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.059123580, 2024-09-21T03:56:46.754 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000061141 2024-09-21T03:56:46.754 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T03:56:46.754 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_handle_ack_lat": { 2024-09-21T03:56:46.754 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-09-21T03:56:46.754 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-09-21T03:56:46.754 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-09-21T03:56:46.754 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T03:56:46.754 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_recv_encrypted_bytes": 1199696, 2024-09-21T03:56:46.754 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_send_encrypted_bytes": 133056 2024-09-21T03:56:46.754 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T03:56:46.755 INFO:teuthology.orchestra.run.smithi150.stdout: "AsyncMessenger::Worker-1": { 2024-09-21T03:56:46.755 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_recv_messages": 590, 2024-09-21T03:56:46.755 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_send_messages": 750, 2024-09-21T03:56:46.755 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_recv_bytes": 145706, 2024-09-21T03:56:46.755 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_send_bytes": 549546, 2024-09-21T03:56:46.755 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_created_connections": 5, 2024-09-21T03:56:46.755 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_active_connections": 4, 2024-09-21T03:56:46.755 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_running_total_time": 0.076852526, 2024-09-21T03:56:46.755 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_running_send_time": 0.035411100, 2024-09-21T03:56:46.755 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_running_recv_time": 0.014423667, 2024-09-21T03:56:46.755 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_running_fast_dispatch_time": 0.020659995, 2024-09-21T03:56:46.755 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T03:56:46.755 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 748, 2024-09-21T03:56:46.755 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.031419898, 2024-09-21T03:56:46.756 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000042005 2024-09-21T03:56:46.756 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T03:56:46.756 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_handle_ack_lat": { 2024-09-21T03:56:46.756 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-09-21T03:56:46.756 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-09-21T03:56:46.756 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-09-21T03:56:46.756 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T03:56:46.756 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_recv_encrypted_bytes": 160, 2024-09-21T03:56:46.756 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_send_encrypted_bytes": 362176 2024-09-21T03:56:46.756 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T03:56:46.756 INFO:teuthology.orchestra.run.smithi150.stdout: "AsyncMessenger::Worker-2": { 2024-09-21T03:56:46.756 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_recv_messages": 750, 2024-09-21T03:56:46.756 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_send_messages": 749, 2024-09-21T03:56:46.756 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_recv_bytes": 199620, 2024-09-21T03:56:46.756 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_send_bytes": 237359, 2024-09-21T03:56:46.757 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_created_connections": 5, 2024-09-21T03:56:46.757 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_active_connections": 4, 2024-09-21T03:56:46.757 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_running_total_time": 0.080875288, 2024-09-21T03:56:46.757 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_running_send_time": 0.037148410, 2024-09-21T03:56:46.757 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_running_recv_time": 0.019521260, 2024-09-21T03:56:46.757 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_running_fast_dispatch_time": 0.018261996, 2024-09-21T03:56:46.757 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T03:56:46.757 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 748, 2024-09-21T03:56:46.757 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.030599831, 2024-09-21T03:56:46.757 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000040908 2024-09-21T03:56:46.757 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T03:56:46.757 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_handle_ack_lat": { 2024-09-21T03:56:46.757 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-09-21T03:56:46.758 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-09-21T03:56:46.758 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-09-21T03:56:46.758 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T03:56:46.758 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_recv_encrypted_bytes": 3040, 2024-09-21T03:56:46.758 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_send_encrypted_bytes": 448 2024-09-21T03:56:46.758 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T03:56:46.758 INFO:teuthology.orchestra.run.smithi150.stdout: "cct": { 2024-09-21T03:56:46.758 INFO:teuthology.orchestra.run.smithi150.stdout: "total_workers": 1, 2024-09-21T03:56:46.758 INFO:teuthology.orchestra.run.smithi150.stdout: "unhealthy_workers": 0 2024-09-21T03:56:46.758 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T03:56:46.758 INFO:teuthology.orchestra.run.smithi150.stdout: "finisher-MDSRank": { 2024-09-21T03:56:46.758 INFO:teuthology.orchestra.run.smithi150.stdout: "queue_len": 0, 2024-09-21T03:56:46.758 INFO:teuthology.orchestra.run.smithi150.stdout: "complete_latency": { 2024-09-21T03:56:46.758 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 3545, 2024-09-21T03:56:46.759 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.100890164, 2024-09-21T03:56:46.759 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000028459 2024-09-21T03:56:46.759 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T03:56:46.759 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T03:56:46.759 INFO:teuthology.orchestra.run.smithi150.stdout: "finisher-PurgeQueue": { 2024-09-21T03:56:46.759 INFO:teuthology.orchestra.run.smithi150.stdout: "queue_len": 0, 2024-09-21T03:56:46.759 INFO:teuthology.orchestra.run.smithi150.stdout: "complete_latency": { 2024-09-21T03:56:46.759 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-09-21T03:56:46.759 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-09-21T03:56:46.759 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-09-21T03:56:46.759 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T03:56:46.759 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T03:56:46.759 INFO:teuthology.orchestra.run.smithi150.stdout: "mds": { 2024-09-21T03:56:46.760 INFO:teuthology.orchestra.run.smithi150.stdout: "request": 0, 2024-09-21T03:56:46.760 INFO:teuthology.orchestra.run.smithi150.stdout: "reply": 0, 2024-09-21T03:56:46.760 INFO:teuthology.orchestra.run.smithi150.stdout: "reply_latency": { 2024-09-21T03:56:46.760 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-09-21T03:56:46.760 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-09-21T03:56:46.760 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-09-21T03:56:46.760 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T03:56:46.760 INFO:teuthology.orchestra.run.smithi150.stdout: "slow_reply": 0, 2024-09-21T03:56:46.760 INFO:teuthology.orchestra.run.smithi150.stdout: "forward": 0, 2024-09-21T03:56:46.760 INFO:teuthology.orchestra.run.smithi150.stdout: "dir_fetch_complete": 0, 2024-09-21T03:56:46.760 INFO:teuthology.orchestra.run.smithi150.stdout: "dir_fetch_keys": 0, 2024-09-21T03:56:46.760 INFO:teuthology.orchestra.run.smithi150.stdout: "dir_commit": 0, 2024-09-21T03:56:46.760 INFO:teuthology.orchestra.run.smithi150.stdout: "dir_split": 0, 2024-09-21T03:56:46.760 INFO:teuthology.orchestra.run.smithi150.stdout: "dir_merge": 0, 2024-09-21T03:56:46.761 INFO:teuthology.orchestra.run.smithi150.stdout: "inodes": 0, 2024-09-21T03:56:46.761 INFO:teuthology.orchestra.run.smithi150.stdout: "inodes_top": 0, 2024-09-21T03:56:46.761 INFO:teuthology.orchestra.run.smithi150.stdout: "inodes_bottom": 0, 2024-09-21T03:56:46.761 INFO:teuthology.orchestra.run.smithi150.stdout: "inodes_pin_tail": 0, 2024-09-21T03:56:46.761 INFO:teuthology.orchestra.run.smithi150.stdout: "inodes_pinned": 0, 2024-09-21T03:56:46.761 INFO:teuthology.orchestra.run.smithi150.stdout: "inodes_expired": 0, 2024-09-21T03:56:46.761 INFO:teuthology.orchestra.run.smithi150.stdout: "inodes_with_caps": 0, 2024-09-21T03:56:46.761 INFO:teuthology.orchestra.run.smithi150.stdout: "caps": 0, 2024-09-21T03:56:46.761 INFO:teuthology.orchestra.run.smithi150.stdout: "subtrees": 1, 2024-09-21T03:56:46.761 INFO:teuthology.orchestra.run.smithi150.stdout: "traverse": 0, 2024-09-21T03:56:46.761 INFO:teuthology.orchestra.run.smithi150.stdout: "traverse_hit": 0, 2024-09-21T03:56:46.761 INFO:teuthology.orchestra.run.smithi150.stdout: "traverse_forward": 0, 2024-09-21T03:56:46.761 INFO:teuthology.orchestra.run.smithi150.stdout: "traverse_discover": 0, 2024-09-21T03:56:46.761 INFO:teuthology.orchestra.run.smithi150.stdout: "traverse_dir_fetch": 0, 2024-09-21T03:56:46.761 INFO:teuthology.orchestra.run.smithi150.stdout: "traverse_remote_ino": 0, 2024-09-21T03:56:46.762 INFO:teuthology.orchestra.run.smithi150.stdout: "traverse_lock": 0, 2024-09-21T03:56:46.762 INFO:teuthology.orchestra.run.smithi150.stdout: "load_cent": 0, 2024-09-21T03:56:46.762 INFO:teuthology.orchestra.run.smithi150.stdout: "q": 0, 2024-09-21T03:56:46.762 INFO:teuthology.orchestra.run.smithi150.stdout: "exported": 0, 2024-09-21T03:56:46.762 INFO:teuthology.orchestra.run.smithi150.stdout: "exported_inodes": 0, 2024-09-21T03:56:46.762 INFO:teuthology.orchestra.run.smithi150.stdout: "imported": 0, 2024-09-21T03:56:46.762 INFO:teuthology.orchestra.run.smithi150.stdout: "imported_inodes": 0, 2024-09-21T03:56:46.762 INFO:teuthology.orchestra.run.smithi150.stdout: "openino_dir_fetch": 0, 2024-09-21T03:56:46.762 INFO:teuthology.orchestra.run.smithi150.stdout: "openino_backtrace_fetch": 0, 2024-09-21T03:56:46.762 INFO:teuthology.orchestra.run.smithi150.stdout: "openino_peer_discover": 0, 2024-09-21T03:56:46.762 INFO:teuthology.orchestra.run.smithi150.stdout: "root_rfiles": 0, 2024-09-21T03:56:46.762 INFO:teuthology.orchestra.run.smithi150.stdout: "root_rbytes": 0, 2024-09-21T03:56:46.763 INFO:teuthology.orchestra.run.smithi150.stdout: "root_rsnaps": 0, 2024-09-21T03:56:46.763 INFO:teuthology.orchestra.run.smithi150.stdout: "scrub_backtrace_fetch": 0, 2024-09-21T03:56:46.763 INFO:teuthology.orchestra.run.smithi150.stdout: "scrub_set_tag": 0, 2024-09-21T03:56:46.763 INFO:teuthology.orchestra.run.smithi150.stdout: "scrub_backtrace_repaired": 0, 2024-09-21T03:56:46.763 INFO:teuthology.orchestra.run.smithi150.stdout: "scrub_inotable_repaired": 0, 2024-09-21T03:56:46.763 INFO:teuthology.orchestra.run.smithi150.stdout: "scrub_dir_inodes": 0, 2024-09-21T03:56:46.763 INFO:teuthology.orchestra.run.smithi150.stdout: "scrub_dir_base_inodes": 0, 2024-09-21T03:56:46.763 INFO:teuthology.orchestra.run.smithi150.stdout: "scrub_dirfrag_rstats": 0, 2024-09-21T03:56:46.763 INFO:teuthology.orchestra.run.smithi150.stdout: "scrub_file_inodes": 0, 2024-09-21T03:56:46.763 INFO:teuthology.orchestra.run.smithi150.stdout: "handle_inode_file_caps": 0, 2024-09-21T03:56:46.763 INFO:teuthology.orchestra.run.smithi150.stdout: "ceph_cap_op_revoke": 0, 2024-09-21T03:56:46.763 INFO:teuthology.orchestra.run.smithi150.stdout: "ceph_cap_op_grant": 0, 2024-09-21T03:56:46.763 INFO:teuthology.orchestra.run.smithi150.stdout: "ceph_cap_op_trunc": 0, 2024-09-21T03:56:46.763 INFO:teuthology.orchestra.run.smithi150.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-21T03:56:46.764 INFO:teuthology.orchestra.run.smithi150.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-21T03:56:46.764 INFO:teuthology.orchestra.run.smithi150.stdout: "handle_client_caps": 0, 2024-09-21T03:56:46.764 INFO:teuthology.orchestra.run.smithi150.stdout: "handle_client_caps_dirty": 0, 2024-09-21T03:56:46.764 INFO:teuthology.orchestra.run.smithi150.stdout: "handle_client_cap_release": 0, 2024-09-21T03:56:46.764 INFO:teuthology.orchestra.run.smithi150.stdout: "process_request_cap_release": 0 2024-09-21T03:56:46.764 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T03:56:46.764 INFO:teuthology.orchestra.run.smithi150.stdout: "mds_cache": { 2024-09-21T03:56:46.764 INFO:teuthology.orchestra.run.smithi150.stdout: "dir_update": 0, 2024-09-21T03:56:46.764 INFO:teuthology.orchestra.run.smithi150.stdout: "dir_update_receipt": 0, 2024-09-21T03:56:46.764 INFO:teuthology.orchestra.run.smithi150.stdout: "dir_try_discover": 0, 2024-09-21T03:56:46.764 INFO:teuthology.orchestra.run.smithi150.stdout: "dir_send_discover": 0, 2024-09-21T03:56:46.764 INFO:teuthology.orchestra.run.smithi150.stdout: "dir_handle_discover": 0, 2024-09-21T03:56:46.764 INFO:teuthology.orchestra.run.smithi150.stdout: "num_strays": 0, 2024-09-21T03:56:46.764 INFO:teuthology.orchestra.run.smithi150.stdout: "num_strays_delayed": 0, 2024-09-21T03:56:46.764 INFO:teuthology.orchestra.run.smithi150.stdout: "num_strays_enqueuing": 0, 2024-09-21T03:56:46.765 INFO:teuthology.orchestra.run.smithi150.stdout: "strays_created": 0, 2024-09-21T03:56:46.765 INFO:teuthology.orchestra.run.smithi150.stdout: "strays_enqueued": 0, 2024-09-21T03:56:46.765 INFO:teuthology.orchestra.run.smithi150.stdout: "strays_reintegrated": 0, 2024-09-21T03:56:46.765 INFO:teuthology.orchestra.run.smithi150.stdout: "strays_migrated": 0, 2024-09-21T03:56:46.765 INFO:teuthology.orchestra.run.smithi150.stdout: "num_recovering_processing": 0, 2024-09-21T03:56:46.765 INFO:teuthology.orchestra.run.smithi150.stdout: "num_recovering_enqueued": 0, 2024-09-21T03:56:46.765 INFO:teuthology.orchestra.run.smithi150.stdout: "num_recovering_prioritized": 0, 2024-09-21T03:56:46.765 INFO:teuthology.orchestra.run.smithi150.stdout: "recovery_started": 0, 2024-09-21T03:56:46.765 INFO:teuthology.orchestra.run.smithi150.stdout: "recovery_completed": 0, 2024-09-21T03:56:46.765 INFO:teuthology.orchestra.run.smithi150.stdout: "ireq_quiesce_path": 0, 2024-09-21T03:56:46.765 INFO:teuthology.orchestra.run.smithi150.stdout: "ireq_quiesce_inode": 0, 2024-09-21T03:56:46.765 INFO:teuthology.orchestra.run.smithi150.stdout: "ireq_enqueue_scrub": 0, 2024-09-21T03:56:46.765 INFO:teuthology.orchestra.run.smithi150.stdout: "ireq_exportdir": 0, 2024-09-21T03:56:46.765 INFO:teuthology.orchestra.run.smithi150.stdout: "ireq_flush": 0, 2024-09-21T03:56:46.765 INFO:teuthology.orchestra.run.smithi150.stdout: "ireq_fragmentdir": 0, 2024-09-21T03:56:46.766 INFO:teuthology.orchestra.run.smithi150.stdout: "ireq_fragstats": 0, 2024-09-21T03:56:46.766 INFO:teuthology.orchestra.run.smithi150.stdout: "ireq_inodestats": 0, 2024-09-21T03:56:46.766 INFO:teuthology.orchestra.run.smithi150.stdout: "uninline_started": 0, 2024-09-21T03:56:46.766 INFO:teuthology.orchestra.run.smithi150.stdout: "uninline_succeeded": 0, 2024-09-21T03:56:46.766 INFO:teuthology.orchestra.run.smithi150.stdout: "uninline_write_failed": 0 2024-09-21T03:56:46.766 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T03:56:46.766 INFO:teuthology.orchestra.run.smithi150.stdout: "mds_log": { 2024-09-21T03:56:46.766 INFO:teuthology.orchestra.run.smithi150.stdout: "evlrg": 0, 2024-09-21T03:56:46.766 INFO:teuthology.orchestra.run.smithi150.stdout: "evadd": 0, 2024-09-21T03:56:46.766 INFO:teuthology.orchestra.run.smithi150.stdout: "evex": 0, 2024-09-21T03:56:46.766 INFO:teuthology.orchestra.run.smithi150.stdout: "evtrm": 0, 2024-09-21T03:56:46.766 INFO:teuthology.orchestra.run.smithi150.stdout: "ev": 3, 2024-09-21T03:56:46.766 INFO:teuthology.orchestra.run.smithi150.stdout: "evexg": 0, 2024-09-21T03:56:46.767 INFO:teuthology.orchestra.run.smithi150.stdout: "evexd": 0, 2024-09-21T03:56:46.767 INFO:teuthology.orchestra.run.smithi150.stdout: "segadd": 0, 2024-09-21T03:56:46.767 INFO:teuthology.orchestra.run.smithi150.stdout: "segex": 0, 2024-09-21T03:56:46.767 INFO:teuthology.orchestra.run.smithi150.stdout: "segtrm": 0, 2024-09-21T03:56:46.767 INFO:teuthology.orchestra.run.smithi150.stdout: "seg": 2, 2024-09-21T03:56:46.767 INFO:teuthology.orchestra.run.smithi150.stdout: "segmjr": 2, 2024-09-21T03:56:46.767 INFO:teuthology.orchestra.run.smithi150.stdout: "segexg": 0, 2024-09-21T03:56:46.767 INFO:teuthology.orchestra.run.smithi150.stdout: "segexd": 0, 2024-09-21T03:56:46.767 INFO:teuthology.orchestra.run.smithi150.stdout: "expos": 4194304, 2024-09-21T03:56:46.767 INFO:teuthology.orchestra.run.smithi150.stdout: "wrpos": 4195283, 2024-09-21T03:56:46.767 INFO:teuthology.orchestra.run.smithi150.stdout: "rdpos": 4194892, 2024-09-21T03:56:46.767 INFO:teuthology.orchestra.run.smithi150.stdout: "jlat": { 2024-09-21T03:56:46.767 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-09-21T03:56:46.767 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-09-21T03:56:46.767 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-09-21T03:56:46.768 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T03:56:46.768 INFO:teuthology.orchestra.run.smithi150.stdout: "replayed": 3 2024-09-21T03:56:46.768 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T03:56:46.768 INFO:teuthology.orchestra.run.smithi150.stdout: "mds_mem": { 2024-09-21T03:56:46.768 INFO:teuthology.orchestra.run.smithi150.stdout: "ino": 3, 2024-09-21T03:56:46.768 INFO:teuthology.orchestra.run.smithi150.stdout: "ino+": 3, 2024-09-21T03:56:46.768 INFO:teuthology.orchestra.run.smithi150.stdout: "ino-": 0, 2024-09-21T03:56:46.768 INFO:teuthology.orchestra.run.smithi150.stdout: "dir": 1, 2024-09-21T03:56:46.768 INFO:teuthology.orchestra.run.smithi150.stdout: "dir+": 1, 2024-09-21T03:56:46.768 INFO:teuthology.orchestra.run.smithi150.stdout: "dir-": 0, 2024-09-21T03:56:46.768 INFO:teuthology.orchestra.run.smithi150.stdout: "dn": 0, 2024-09-21T03:56:46.768 INFO:teuthology.orchestra.run.smithi150.stdout: "dn+": 0, 2024-09-21T03:56:46.768 INFO:teuthology.orchestra.run.smithi150.stdout: "dn-": 0, 2024-09-21T03:56:46.768 INFO:teuthology.orchestra.run.smithi150.stdout: "cap": 0, 2024-09-21T03:56:46.769 INFO:teuthology.orchestra.run.smithi150.stdout: "cap+": 0, 2024-09-21T03:56:46.769 INFO:teuthology.orchestra.run.smithi150.stdout: "cap-": 0, 2024-09-21T03:56:46.769 INFO:teuthology.orchestra.run.smithi150.stdout: "rss": 117156, 2024-09-21T03:56:46.769 INFO:teuthology.orchestra.run.smithi150.stdout: "heap": 316420096 2024-09-21T03:56:46.769 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T03:56:46.769 INFO:teuthology.orchestra.run.smithi150.stdout: "mds_server": { 2024-09-21T03:56:46.769 INFO:teuthology.orchestra.run.smithi150.stdout: "dispatch_client_request": 0, 2024-09-21T03:56:46.769 INFO:teuthology.orchestra.run.smithi150.stdout: "dispatch_server_request": 0, 2024-09-21T03:56:46.769 INFO:teuthology.orchestra.run.smithi150.stdout: "handle_client_request": 0, 2024-09-21T03:56:46.769 INFO:teuthology.orchestra.run.smithi150.stdout: "handle_client_session": 0, 2024-09-21T03:56:46.769 INFO:teuthology.orchestra.run.smithi150.stdout: "handle_peer_request": 0, 2024-09-21T03:56:46.769 INFO:teuthology.orchestra.run.smithi150.stdout: "req_create_latency": { 2024-09-21T03:56:46.769 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-09-21T03:56:46.769 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-09-21T03:56:46.769 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-09-21T03:56:46.770 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T03:56:46.770 INFO:teuthology.orchestra.run.smithi150.stdout: "req_getattr_latency": { 2024-09-21T03:56:46.770 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-09-21T03:56:46.770 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-09-21T03:56:46.770 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-09-21T03:56:46.770 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T03:56:46.770 INFO:teuthology.orchestra.run.smithi150.stdout: "req_getfilelock_latency": { 2024-09-21T03:56:46.770 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-09-21T03:56:46.770 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-09-21T03:56:46.770 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-09-21T03:56:46.770 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T03:56:46.770 INFO:teuthology.orchestra.run.smithi150.stdout: "req_link_latency": { 2024-09-21T03:56:46.770 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-09-21T03:56:46.770 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-09-21T03:56:46.771 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-09-21T03:56:46.771 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T03:56:46.771 INFO:teuthology.orchestra.run.smithi150.stdout: "req_lookup_latency": { 2024-09-21T03:56:46.771 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-09-21T03:56:46.771 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-09-21T03:56:46.771 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-09-21T03:56:46.771 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T03:56:46.771 INFO:teuthology.orchestra.run.smithi150.stdout: "req_lookuphash_latency": { 2024-09-21T03:56:46.771 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-09-21T03:56:46.771 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-09-21T03:56:46.771 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-09-21T03:56:46.771 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T03:56:46.771 INFO:teuthology.orchestra.run.smithi150.stdout: "req_lookupino_latency": { 2024-09-21T03:56:46.771 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-09-21T03:56:46.771 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-09-21T03:56:46.772 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-09-21T03:56:46.772 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T03:56:46.772 INFO:teuthology.orchestra.run.smithi150.stdout: "req_lookupname_latency": { 2024-09-21T03:56:46.772 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-09-21T03:56:46.772 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-09-21T03:56:46.772 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-09-21T03:56:46.772 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T03:56:46.772 INFO:teuthology.orchestra.run.smithi150.stdout: "req_lookupparent_latency": { 2024-09-21T03:56:46.772 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-09-21T03:56:46.772 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-09-21T03:56:46.773 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-09-21T03:56:46.773 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T03:56:46.773 INFO:teuthology.orchestra.run.smithi150.stdout: "req_lookupsnap_latency": { 2024-09-21T03:56:46.773 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-09-21T03:56:46.773 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-09-21T03:56:46.773 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-09-21T03:56:46.773 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T03:56:46.773 INFO:teuthology.orchestra.run.smithi150.stdout: "req_lssnap_latency": { 2024-09-21T03:56:46.773 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-09-21T03:56:46.773 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-09-21T03:56:46.774 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-09-21T03:56:46.774 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T03:56:46.774 INFO:teuthology.orchestra.run.smithi150.stdout: "req_mkdir_latency": { 2024-09-21T03:56:46.774 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-09-21T03:56:46.774 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-09-21T03:56:46.774 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-09-21T03:56:46.774 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T03:56:46.774 INFO:teuthology.orchestra.run.smithi150.stdout: "req_mknod_latency": { 2024-09-21T03:56:46.774 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-09-21T03:56:46.774 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-09-21T03:56:46.775 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-09-21T03:56:46.775 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T03:56:46.775 INFO:teuthology.orchestra.run.smithi150.stdout: "req_mksnap_latency": { 2024-09-21T03:56:46.775 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-09-21T03:56:46.775 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-09-21T03:56:46.775 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-09-21T03:56:46.775 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T03:56:46.775 INFO:teuthology.orchestra.run.smithi150.stdout: "req_open_latency": { 2024-09-21T03:56:46.775 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-09-21T03:56:46.775 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-09-21T03:56:46.775 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-09-21T03:56:46.775 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T03:56:46.775 INFO:teuthology.orchestra.run.smithi150.stdout: "req_readdir_latency": { 2024-09-21T03:56:46.775 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-09-21T03:56:46.776 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-09-21T03:56:46.776 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-09-21T03:56:46.776 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T03:56:46.776 INFO:teuthology.orchestra.run.smithi150.stdout: "req_rename_latency": { 2024-09-21T03:56:46.776 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-09-21T03:56:46.776 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-09-21T03:56:46.776 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-09-21T03:56:46.776 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T03:56:46.776 INFO:teuthology.orchestra.run.smithi150.stdout: "req_renamesnap_latency": { 2024-09-21T03:56:46.776 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-09-21T03:56:46.776 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-09-21T03:56:46.776 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-09-21T03:56:46.776 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T03:56:46.776 INFO:teuthology.orchestra.run.smithi150.stdout: "req_snapdiff_latency": { 2024-09-21T03:56:46.777 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-09-21T03:56:46.777 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-09-21T03:56:46.777 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-09-21T03:56:46.777 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T03:56:46.777 INFO:teuthology.orchestra.run.smithi150.stdout: "req_rmdir_latency": { 2024-09-21T03:56:46.777 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-09-21T03:56:46.777 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-09-21T03:56:46.777 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-09-21T03:56:46.777 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T03:56:46.777 INFO:teuthology.orchestra.run.smithi150.stdout: "req_rmsnap_latency": { 2024-09-21T03:56:46.777 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-09-21T03:56:46.777 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-09-21T03:56:46.777 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-09-21T03:56:46.778 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T03:56:46.778 INFO:teuthology.orchestra.run.smithi150.stdout: "req_rmxattr_latency": { 2024-09-21T03:56:46.778 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-09-21T03:56:46.778 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-09-21T03:56:46.778 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-09-21T03:56:46.778 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T03:56:46.778 INFO:teuthology.orchestra.run.smithi150.stdout: "req_setattr_latency": { 2024-09-21T03:56:46.778 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-09-21T03:56:46.778 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-09-21T03:56:46.778 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-09-21T03:56:46.778 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T03:56:46.778 INFO:teuthology.orchestra.run.smithi150.stdout: "req_setdirlayout_latency": { 2024-09-21T03:56:46.778 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-09-21T03:56:46.778 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-09-21T03:56:46.779 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-09-21T03:56:46.779 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T03:56:46.779 INFO:teuthology.orchestra.run.smithi150.stdout: "req_setfilelock_latency": { 2024-09-21T03:56:46.779 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-09-21T03:56:46.779 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-09-21T03:56:46.779 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-09-21T03:56:46.779 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T03:56:46.779 INFO:teuthology.orchestra.run.smithi150.stdout: "req_setlayout_latency": { 2024-09-21T03:56:46.779 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-09-21T03:56:46.779 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-09-21T03:56:46.779 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-09-21T03:56:46.779 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T03:56:46.779 INFO:teuthology.orchestra.run.smithi150.stdout: "req_setxattr_latency": { 2024-09-21T03:56:46.779 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-09-21T03:56:46.780 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-09-21T03:56:46.780 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-09-21T03:56:46.780 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T03:56:46.780 INFO:teuthology.orchestra.run.smithi150.stdout: "req_symlink_latency": { 2024-09-21T03:56:46.780 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-09-21T03:56:46.780 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-09-21T03:56:46.780 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-09-21T03:56:46.780 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T03:56:46.781 INFO:teuthology.orchestra.run.smithi150.stdout: "req_unlink_latency": { 2024-09-21T03:56:46.782 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-09-21T03:56:46.782 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-09-21T03:56:46.782 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-09-21T03:56:46.782 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T03:56:46.782 INFO:teuthology.orchestra.run.smithi150.stdout: "cap_revoke_eviction": 0, 2024-09-21T03:56:46.782 INFO:teuthology.orchestra.run.smithi150.stdout: "cap_acquisition_throttle": 0, 2024-09-21T03:56:46.782 INFO:teuthology.orchestra.run.smithi150.stdout: "req_getvxattr_latency": { 2024-09-21T03:56:46.782 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-09-21T03:56:46.782 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-09-21T03:56:46.783 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-09-21T03:56:46.783 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T03:56:46.783 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T03:56:46.783 INFO:teuthology.orchestra.run.smithi150.stdout: "mds_sessions": { 2024-09-21T03:56:46.783 INFO:teuthology.orchestra.run.smithi150.stdout: "session_count": 1, 2024-09-21T03:56:46.783 INFO:teuthology.orchestra.run.smithi150.stdout: "session_add": 1, 2024-09-21T03:56:46.783 INFO:teuthology.orchestra.run.smithi150.stdout: "session_remove": 0, 2024-09-21T03:56:46.783 INFO:teuthology.orchestra.run.smithi150.stdout: "sessions_open": 1, 2024-09-21T03:56:46.783 INFO:teuthology.orchestra.run.smithi150.stdout: "sessions_stale": 0, 2024-09-21T03:56:46.783 INFO:teuthology.orchestra.run.smithi150.stdout: "total_load": 0, 2024-09-21T03:56:46.783 INFO:teuthology.orchestra.run.smithi150.stdout: "average_load": 0, 2024-09-21T03:56:46.783 INFO:teuthology.orchestra.run.smithi150.stdout: "avg_session_uptime": 2112, 2024-09-21T03:56:46.783 INFO:teuthology.orchestra.run.smithi150.stdout: "mdthresh_evicted": 0 2024-09-21T03:56:46.783 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T03:56:46.783 INFO:teuthology.orchestra.run.smithi150.stdout: "mempool": { 2024-09-21T03:56:46.784 INFO:teuthology.orchestra.run.smithi150.stdout: "bloom_filter_bytes": 0, 2024-09-21T03:56:46.784 INFO:teuthology.orchestra.run.smithi150.stdout: "bloom_filter_items": 0, 2024-09-21T03:56:46.784 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_alloc_bytes": 0, 2024-09-21T03:56:46.784 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_alloc_items": 0, 2024-09-21T03:56:46.784 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_cache_data_bytes": 0, 2024-09-21T03:56:46.784 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_cache_data_items": 0, 2024-09-21T03:56:46.784 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-21T03:56:46.784 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_cache_onode_items": 0, 2024-09-21T03:56:46.784 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-21T03:56:46.784 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_cache_meta_items": 0, 2024-09-21T03:56:46.784 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_cache_other_bytes": 0, 2024-09-21T03:56:46.784 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_cache_other_items": 0, 2024-09-21T03:56:46.784 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-21T03:56:46.784 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_cache_buffer_items": 0, 2024-09-21T03:56:46.785 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_extent_bytes": 0, 2024-09-21T03:56:46.785 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_extent_items": 0, 2024-09-21T03:56:46.785 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_blob_bytes": 0, 2024-09-21T03:56:46.785 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_blob_items": 0, 2024-09-21T03:56:46.785 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-21T03:56:46.785 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_shared_blob_items": 0, 2024-09-21T03:56:46.785 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-21T03:56:46.785 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_inline_bl_items": 0, 2024-09-21T03:56:46.785 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_fsck_bytes": 0, 2024-09-21T03:56:46.785 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_fsck_items": 0, 2024-09-21T03:56:46.785 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_txc_bytes": 0, 2024-09-21T03:56:46.785 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_txc_items": 0, 2024-09-21T03:56:46.785 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-21T03:56:46.785 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_writing_deferred_items": 0, 2024-09-21T03:56:46.785 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_writing_bytes": 0, 2024-09-21T03:56:46.786 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_writing_items": 0, 2024-09-21T03:56:46.786 INFO:teuthology.orchestra.run.smithi150.stdout: "bluefs_bytes": 0, 2024-09-21T03:56:46.786 INFO:teuthology.orchestra.run.smithi150.stdout: "bluefs_items": 0, 2024-09-21T03:56:46.786 INFO:teuthology.orchestra.run.smithi150.stdout: "bluefs_file_reader_bytes": 0, 2024-09-21T03:56:46.786 INFO:teuthology.orchestra.run.smithi150.stdout: "bluefs_file_reader_items": 0, 2024-09-21T03:56:46.786 INFO:teuthology.orchestra.run.smithi150.stdout: "bluefs_file_writer_bytes": 0, 2024-09-21T03:56:46.786 INFO:teuthology.orchestra.run.smithi150.stdout: "bluefs_file_writer_items": 0, 2024-09-21T03:56:46.786 INFO:teuthology.orchestra.run.smithi150.stdout: "buffer_anon_bytes": 197497, 2024-09-21T03:56:46.786 INFO:teuthology.orchestra.run.smithi150.stdout: "buffer_anon_items": 56, 2024-09-21T03:56:46.786 INFO:teuthology.orchestra.run.smithi150.stdout: "buffer_meta_bytes": 0, 2024-09-21T03:56:46.786 INFO:teuthology.orchestra.run.smithi150.stdout: "buffer_meta_items": 0, 2024-09-21T03:56:46.786 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_bytes": 0, 2024-09-21T03:56:46.786 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_items": 0, 2024-09-21T03:56:46.786 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_mapbl_bytes": 0, 2024-09-21T03:56:46.787 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_mapbl_items": 0, 2024-09-21T03:56:46.787 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_pglog_bytes": 0, 2024-09-21T03:56:46.787 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_pglog_items": 0, 2024-09-21T03:56:46.787 INFO:teuthology.orchestra.run.smithi150.stdout: "osdmap_bytes": 8256, 2024-09-21T03:56:46.787 INFO:teuthology.orchestra.run.smithi150.stdout: "osdmap_items": 182, 2024-09-21T03:56:46.787 INFO:teuthology.orchestra.run.smithi150.stdout: "osdmap_mapping_bytes": 0, 2024-09-21T03:56:46.787 INFO:teuthology.orchestra.run.smithi150.stdout: "osdmap_mapping_items": 0, 2024-09-21T03:56:46.787 INFO:teuthology.orchestra.run.smithi150.stdout: "pgmap_bytes": 0, 2024-09-21T03:56:46.787 INFO:teuthology.orchestra.run.smithi150.stdout: "pgmap_items": 0, 2024-09-21T03:56:46.787 INFO:teuthology.orchestra.run.smithi150.stdout: "mds_co_bytes": 8672, 2024-09-21T03:56:46.787 INFO:teuthology.orchestra.run.smithi150.stdout: "mds_co_items": 13, 2024-09-21T03:56:46.787 INFO:teuthology.orchestra.run.smithi150.stdout: "unittest_1_bytes": 0, 2024-09-21T03:56:46.787 INFO:teuthology.orchestra.run.smithi150.stdout: "unittest_1_items": 0, 2024-09-21T03:56:46.787 INFO:teuthology.orchestra.run.smithi150.stdout: "unittest_2_bytes": 0, 2024-09-21T03:56:46.787 INFO:teuthology.orchestra.run.smithi150.stdout: "unittest_2_items": 0 2024-09-21T03:56:46.788 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T03:56:46.788 INFO:teuthology.orchestra.run.smithi150.stdout: "objecter": { 2024-09-21T03:56:46.788 INFO:teuthology.orchestra.run.smithi150.stdout: "op_active": 0, 2024-09-21T03:56:46.788 INFO:teuthology.orchestra.run.smithi150.stdout: "op_laggy": 0, 2024-09-21T03:56:46.788 INFO:teuthology.orchestra.run.smithi150.stdout: "op_send": 2075, 2024-09-21T03:56:46.788 INFO:teuthology.orchestra.run.smithi150.stdout: "op_send_bytes": 21, 2024-09-21T03:56:46.788 INFO:teuthology.orchestra.run.smithi150.stdout: "op_resend": 3, 2024-09-21T03:56:46.788 INFO:teuthology.orchestra.run.smithi150.stdout: "op_reply": 2075, 2024-09-21T03:56:46.788 INFO:teuthology.orchestra.run.smithi150.stdout: "op_latency": { 2024-09-21T03:56:46.788 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 2075, 2024-09-21T03:56:46.788 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 1.691965235, 2024-09-21T03:56:46.788 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000815404 2024-09-21T03:56:46.788 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T03:56:46.788 INFO:teuthology.orchestra.run.smithi150.stdout: "op_inflight": 0, 2024-09-21T03:56:46.789 INFO:teuthology.orchestra.run.smithi150.stdout: "oplen_avg": { 2024-09-21T03:56:46.789 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 2075, 2024-09-21T03:56:46.789 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 2076 2024-09-21T03:56:46.789 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T03:56:46.789 INFO:teuthology.orchestra.run.smithi150.stdout: "op": 2075, 2024-09-21T03:56:46.789 INFO:teuthology.orchestra.run.smithi150.stdout: "op_r": 2075, 2024-09-21T03:56:46.789 INFO:teuthology.orchestra.run.smithi150.stdout: "op_w": 0, 2024-09-21T03:56:46.789 INFO:teuthology.orchestra.run.smithi150.stdout: "op_rmw": 0, 2024-09-21T03:56:46.789 INFO:teuthology.orchestra.run.smithi150.stdout: "op_pg": 0, 2024-09-21T03:56:46.789 INFO:teuthology.orchestra.run.smithi150.stdout: "osdop_stat": 1327, 2024-09-21T03:56:46.789 INFO:teuthology.orchestra.run.smithi150.stdout: "osdop_create": 0, 2024-09-21T03:56:46.789 INFO:teuthology.orchestra.run.smithi150.stdout: "osdop_read": 746, 2024-09-21T03:56:46.789 INFO:teuthology.orchestra.run.smithi150.stdout: "osdop_write": 0, 2024-09-21T03:56:46.789 INFO:teuthology.orchestra.run.smithi150.stdout: "osdop_writefull": 0, 2024-09-21T03:56:46.789 INFO:teuthology.orchestra.run.smithi150.stdout: "osdop_writesame": 0, 2024-09-21T03:56:46.790 INFO:teuthology.orchestra.run.smithi150.stdout: "osdop_append": 0, 2024-09-21T03:56:46.790 INFO:teuthology.orchestra.run.smithi150.stdout: "osdop_zero": 0, 2024-09-21T03:56:46.790 INFO:teuthology.orchestra.run.smithi150.stdout: "osdop_truncate": 0, 2024-09-21T03:56:46.790 INFO:teuthology.orchestra.run.smithi150.stdout: "osdop_delete": 0, 2024-09-21T03:56:46.790 INFO:teuthology.orchestra.run.smithi150.stdout: "osdop_mapext": 0, 2024-09-21T03:56:46.790 INFO:teuthology.orchestra.run.smithi150.stdout: "osdop_sparse_read": 0, 2024-09-21T03:56:46.790 INFO:teuthology.orchestra.run.smithi150.stdout: "osdop_clonerange": 0, 2024-09-21T03:56:46.790 INFO:teuthology.orchestra.run.smithi150.stdout: "osdop_getxattr": 1, 2024-09-21T03:56:46.790 INFO:teuthology.orchestra.run.smithi150.stdout: "osdop_setxattr": 0, 2024-09-21T03:56:46.790 INFO:teuthology.orchestra.run.smithi150.stdout: "osdop_cmpxattr": 0, 2024-09-21T03:56:46.790 INFO:teuthology.orchestra.run.smithi150.stdout: "osdop_rmxattr": 0, 2024-09-21T03:56:46.790 INFO:teuthology.orchestra.run.smithi150.stdout: "osdop_resetxattrs": 0, 2024-09-21T03:56:46.790 INFO:teuthology.orchestra.run.smithi150.stdout: "osdop_call": 0, 2024-09-21T03:56:46.790 INFO:teuthology.orchestra.run.smithi150.stdout: "osdop_watch": 0, 2024-09-21T03:56:46.791 INFO:teuthology.orchestra.run.smithi150.stdout: "osdop_notify": 0, 2024-09-21T03:56:46.791 INFO:teuthology.orchestra.run.smithi150.stdout: "osdop_src_cmpxattr": 0, 2024-09-21T03:56:46.791 INFO:teuthology.orchestra.run.smithi150.stdout: "osdop_pgls": 0, 2024-09-21T03:56:46.791 INFO:teuthology.orchestra.run.smithi150.stdout: "osdop_pgls_filter": 0, 2024-09-21T03:56:46.791 INFO:teuthology.orchestra.run.smithi150.stdout: "osdop_other": 0, 2024-09-21T03:56:46.791 INFO:teuthology.orchestra.run.smithi150.stdout: "linger_active": 0, 2024-09-21T03:56:46.791 INFO:teuthology.orchestra.run.smithi150.stdout: "linger_send": 0, 2024-09-21T03:56:46.791 INFO:teuthology.orchestra.run.smithi150.stdout: "linger_resend": 0, 2024-09-21T03:56:46.791 INFO:teuthology.orchestra.run.smithi150.stdout: "linger_ping": 0, 2024-09-21T03:56:46.791 INFO:teuthology.orchestra.run.smithi150.stdout: "poolop_active": 0, 2024-09-21T03:56:46.791 INFO:teuthology.orchestra.run.smithi150.stdout: "poolop_send": 0, 2024-09-21T03:56:46.791 INFO:teuthology.orchestra.run.smithi150.stdout: "poolop_resend": 0, 2024-09-21T03:56:46.791 INFO:teuthology.orchestra.run.smithi150.stdout: "poolstat_active": 0, 2024-09-21T03:56:46.791 INFO:teuthology.orchestra.run.smithi150.stdout: "poolstat_send": 0, 2024-09-21T03:56:46.792 INFO:teuthology.orchestra.run.smithi150.stdout: "poolstat_resend": 0, 2024-09-21T03:56:46.792 INFO:teuthology.orchestra.run.smithi150.stdout: "statfs_active": 0, 2024-09-21T03:56:46.792 INFO:teuthology.orchestra.run.smithi150.stdout: "statfs_send": 0, 2024-09-21T03:56:46.792 INFO:teuthology.orchestra.run.smithi150.stdout: "statfs_resend": 0, 2024-09-21T03:56:46.792 INFO:teuthology.orchestra.run.smithi150.stdout: "command_active": 0, 2024-09-21T03:56:46.792 INFO:teuthology.orchestra.run.smithi150.stdout: "command_send": 0, 2024-09-21T03:56:46.792 INFO:teuthology.orchestra.run.smithi150.stdout: "command_resend": 0, 2024-09-21T03:56:46.792 INFO:teuthology.orchestra.run.smithi150.stdout: "map_epoch": 87, 2024-09-21T03:56:46.792 INFO:teuthology.orchestra.run.smithi150.stdout: "map_full": 0, 2024-09-21T03:56:46.792 INFO:teuthology.orchestra.run.smithi150.stdout: "map_inc": 10, 2024-09-21T03:56:46.792 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_sessions": 6, 2024-09-21T03:56:46.792 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_session_open": 6, 2024-09-21T03:56:46.792 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_session_close": 0, 2024-09-21T03:56:46.792 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_laggy": 0, 2024-09-21T03:56:46.792 INFO:teuthology.orchestra.run.smithi150.stdout: "omap_wr": 0, 2024-09-21T03:56:46.792 INFO:teuthology.orchestra.run.smithi150.stdout: "omap_rd": 2, 2024-09-21T03:56:46.793 INFO:teuthology.orchestra.run.smithi150.stdout: "omap_del": 0 2024-09-21T03:56:46.793 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T03:56:46.793 INFO:teuthology.orchestra.run.smithi150.stdout: "oft": { 2024-09-21T03:56:46.793 INFO:teuthology.orchestra.run.smithi150.stdout: "omap_total_objs": 0, 2024-09-21T03:56:46.793 INFO:teuthology.orchestra.run.smithi150.stdout: "omap_total_kv_pairs": 0, 2024-09-21T03:56:46.793 INFO:teuthology.orchestra.run.smithi150.stdout: "omap_total_updates": 0, 2024-09-21T03:56:46.793 INFO:teuthology.orchestra.run.smithi150.stdout: "omap_total_removes": 0 2024-09-21T03:56:46.793 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T03:56:46.793 INFO:teuthology.orchestra.run.smithi150.stdout: "purge_queue": { 2024-09-21T03:56:46.793 INFO:teuthology.orchestra.run.smithi150.stdout: "pq_executing_ops": 0, 2024-09-21T03:56:46.793 INFO:teuthology.orchestra.run.smithi150.stdout: "pq_executing_ops_high_water": 0, 2024-09-21T03:56:46.793 INFO:teuthology.orchestra.run.smithi150.stdout: "pq_executing": 0, 2024-09-21T03:56:46.793 INFO:teuthology.orchestra.run.smithi150.stdout: "pq_executing_high_water": 0, 2024-09-21T03:56:46.793 INFO:teuthology.orchestra.run.smithi150.stdout: "pq_executed_ops": 0, 2024-09-21T03:56:46.793 INFO:teuthology.orchestra.run.smithi150.stdout: "pq_executed": 0, 2024-09-21T03:56:46.794 INFO:teuthology.orchestra.run.smithi150.stdout: "pq_item_in_journal": 0 2024-09-21T03:56:46.794 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T03:56:46.794 INFO:teuthology.orchestra.run.smithi150.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-21T03:56:46.794 INFO:teuthology.orchestra.run.smithi150.stdout: "val": 0, 2024-09-21T03:56:46.794 INFO:teuthology.orchestra.run.smithi150.stdout: "max": 104857600, 2024-09-21T03:56:46.794 INFO:teuthology.orchestra.run.smithi150.stdout: "get_started": 0, 2024-09-21T03:56:46.794 INFO:teuthology.orchestra.run.smithi150.stdout: "get": 2316, 2024-09-21T03:56:46.794 INFO:teuthology.orchestra.run.smithi150.stdout: "get_sum": 1570269, 2024-09-21T03:56:46.794 INFO:teuthology.orchestra.run.smithi150.stdout: "get_or_fail_fail": 0, 2024-09-21T03:56:46.794 INFO:teuthology.orchestra.run.smithi150.stdout: "get_or_fail_success": 2316, 2024-09-21T03:56:46.794 INFO:teuthology.orchestra.run.smithi150.stdout: "take": 0, 2024-09-21T03:56:46.794 INFO:teuthology.orchestra.run.smithi150.stdout: "take_sum": 0, 2024-09-21T03:56:46.794 INFO:teuthology.orchestra.run.smithi150.stdout: "put": 2316, 2024-09-21T03:56:46.794 INFO:teuthology.orchestra.run.smithi150.stdout: "put_sum": 1570269, 2024-09-21T03:56:46.794 INFO:teuthology.orchestra.run.smithi150.stdout: "wait": { 2024-09-21T03:56:46.795 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-09-21T03:56:46.795 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-09-21T03:56:46.795 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-09-21T03:56:46.795 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T03:56:46.795 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T03:56:46.795 INFO:teuthology.orchestra.run.smithi150.stdout: "throttle-objecter_bytes": { 2024-09-21T03:56:46.795 INFO:teuthology.orchestra.run.smithi150.stdout: "val": 0, 2024-09-21T03:56:46.795 INFO:teuthology.orchestra.run.smithi150.stdout: "max": 104857600, 2024-09-21T03:56:46.795 INFO:teuthology.orchestra.run.smithi150.stdout: "get_started": 0, 2024-09-21T03:56:46.795 INFO:teuthology.orchestra.run.smithi150.stdout: "get": 0, 2024-09-21T03:56:46.795 INFO:teuthology.orchestra.run.smithi150.stdout: "get_sum": 0, 2024-09-21T03:56:46.795 INFO:teuthology.orchestra.run.smithi150.stdout: "get_or_fail_fail": 0, 2024-09-21T03:56:46.795 INFO:teuthology.orchestra.run.smithi150.stdout: "get_or_fail_success": 0, 2024-09-21T03:56:46.795 INFO:teuthology.orchestra.run.smithi150.stdout: "take": 2075, 2024-09-21T03:56:46.795 INFO:teuthology.orchestra.run.smithi150.stdout: "take_sum": 984, 2024-09-21T03:56:46.796 INFO:teuthology.orchestra.run.smithi150.stdout: "put": 2075, 2024-09-21T03:56:46.796 INFO:teuthology.orchestra.run.smithi150.stdout: "put_sum": 984, 2024-09-21T03:56:46.796 INFO:teuthology.orchestra.run.smithi150.stdout: "wait": { 2024-09-21T03:56:46.796 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-09-21T03:56:46.796 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-09-21T03:56:46.796 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-09-21T03:56:46.796 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T03:56:46.796 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T03:56:46.796 INFO:teuthology.orchestra.run.smithi150.stdout: "throttle-objecter_ops": { 2024-09-21T03:56:46.796 INFO:teuthology.orchestra.run.smithi150.stdout: "val": 0, 2024-09-21T03:56:46.796 INFO:teuthology.orchestra.run.smithi150.stdout: "max": 1024, 2024-09-21T03:56:46.796 INFO:teuthology.orchestra.run.smithi150.stdout: "get_started": 0, 2024-09-21T03:56:46.796 INFO:teuthology.orchestra.run.smithi150.stdout: "get": 0, 2024-09-21T03:56:46.796 INFO:teuthology.orchestra.run.smithi150.stdout: "get_sum": 0, 2024-09-21T03:56:46.797 INFO:teuthology.orchestra.run.smithi150.stdout: "get_or_fail_fail": 0, 2024-09-21T03:56:46.797 INFO:teuthology.orchestra.run.smithi150.stdout: "get_or_fail_success": 0, 2024-09-21T03:56:46.797 INFO:teuthology.orchestra.run.smithi150.stdout: "take": 2075, 2024-09-21T03:56:46.797 INFO:teuthology.orchestra.run.smithi150.stdout: "take_sum": 2075, 2024-09-21T03:56:46.797 INFO:teuthology.orchestra.run.smithi150.stdout: "put": 2075, 2024-09-21T03:56:46.797 INFO:teuthology.orchestra.run.smithi150.stdout: "put_sum": 2075, 2024-09-21T03:56:46.797 INFO:teuthology.orchestra.run.smithi150.stdout: "wait": { 2024-09-21T03:56:46.797 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-09-21T03:56:46.797 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-09-21T03:56:46.797 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-09-21T03:56:46.797 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T03:56:46.797 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T03:56:46.797 INFO:teuthology.orchestra.run.smithi150.stdout: "throttle-write_buf_throttle": { 2024-09-21T03:56:46.797 INFO:teuthology.orchestra.run.smithi150.stdout: "val": 0, 2024-09-21T03:56:46.797 INFO:teuthology.orchestra.run.smithi150.stdout: "max": 3758096384, 2024-09-21T03:56:46.798 INFO:teuthology.orchestra.run.smithi150.stdout: "get_started": 0, 2024-09-21T03:56:46.798 INFO:teuthology.orchestra.run.smithi150.stdout: "get": 0, 2024-09-21T03:56:46.798 INFO:teuthology.orchestra.run.smithi150.stdout: "get_sum": 0, 2024-09-21T03:56:46.798 INFO:teuthology.orchestra.run.smithi150.stdout: "get_or_fail_fail": 0, 2024-09-21T03:56:46.798 INFO:teuthology.orchestra.run.smithi150.stdout: "get_or_fail_success": 0, 2024-09-21T03:56:46.798 INFO:teuthology.orchestra.run.smithi150.stdout: "take": 0, 2024-09-21T03:56:46.798 INFO:teuthology.orchestra.run.smithi150.stdout: "take_sum": 0, 2024-09-21T03:56:46.798 INFO:teuthology.orchestra.run.smithi150.stdout: "put": 0, 2024-09-21T03:56:46.798 INFO:teuthology.orchestra.run.smithi150.stdout: "put_sum": 0, 2024-09-21T03:56:46.798 INFO:teuthology.orchestra.run.smithi150.stdout: "wait": { 2024-09-21T03:56:46.798 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-09-21T03:56:46.798 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-09-21T03:56:46.798 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-09-21T03:56:46.799 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T03:56:46.799 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T03:56:46.799 INFO:teuthology.orchestra.run.smithi150.stdout: "throttle-write_buf_throttle-0x5624dd7cf520": { 2024-09-21T03:56:46.799 INFO:teuthology.orchestra.run.smithi150.stdout: "val": 0, 2024-09-21T03:56:46.799 INFO:teuthology.orchestra.run.smithi150.stdout: "max": 3758096384, 2024-09-21T03:56:46.799 INFO:teuthology.orchestra.run.smithi150.stdout: "get_started": 0, 2024-09-21T03:56:46.799 INFO:teuthology.orchestra.run.smithi150.stdout: "get": 0, 2024-09-21T03:56:46.799 INFO:teuthology.orchestra.run.smithi150.stdout: "get_sum": 0, 2024-09-21T03:56:46.799 INFO:teuthology.orchestra.run.smithi150.stdout: "get_or_fail_fail": 0, 2024-09-21T03:56:46.799 INFO:teuthology.orchestra.run.smithi150.stdout: "get_or_fail_success": 0, 2024-09-21T03:56:46.799 INFO:teuthology.orchestra.run.smithi150.stdout: "take": 0, 2024-09-21T03:56:46.799 INFO:teuthology.orchestra.run.smithi150.stdout: "take_sum": 0, 2024-09-21T03:56:46.799 INFO:teuthology.orchestra.run.smithi150.stdout: "put": 0, 2024-09-21T03:56:46.799 INFO:teuthology.orchestra.run.smithi150.stdout: "put_sum": 0, 2024-09-21T03:56:46.799 INFO:teuthology.orchestra.run.smithi150.stdout: "wait": { 2024-09-21T03:56:46.799 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-09-21T03:56:46.800 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-09-21T03:56:46.800 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-09-21T03:56:46.800 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T03:56:46.800 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T03:56:46.800 INFO:teuthology.orchestra.run.smithi150.stdout: "trackedop": { 2024-09-21T03:56:46.800 INFO:teuthology.orchestra.run.smithi150.stdout: "slow_ops_count": 0 2024-09-21T03:56:46.800 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T03:56:46.800 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-21T03:56:47.598 INFO:tasks.check_counter:Daemon mds.k mds.exported=0 2024-09-21T03:56:47.598 INFO:tasks.check_counter:Daemon mds.k mds.imported=0 2024-09-21T03:56:47.598 INFO:tasks.check_counter:Daemon mds.k mds_cache.dir_update=0 2024-09-21T03:56:47.598 INFO:tasks.check_counter:Daemon mds.k mds_cache.dir_update_receipt=0 2024-09-21T03:56:47.598 INFO:tasks.check_counter:Daemon mds.k mds.root_rsnaps=0 2024-09-21T03:56:47.598 INFO:tasks.check_counter:Daemon mds.k mds_server.req_mksnap_latency.avgcount=0 2024-09-21T03:56:47.598 INFO:tasks.check_counter:Daemon mds.k mds_server.req_rmsnap_latency.avgcount=0 2024-09-21T03:56:47.598 DEBUG:tasks.check_counter:Getting stats from c 2024-09-21T03:56:47.599 DEBUG:teuthology.orchestra.run.smithi161:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 51116d5c-77ca-11ef-baf5-efdc52797490 -- ceph daemon mds.c perf dump 2024-09-21T03:56:47.628 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:56:47 smithi161 ceph-mon[34533]: pgmap v655: 97 pgs: 97 active+clean; 10 MiB data, 525 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 10 op/s 2024-09-21T03:56:47.725 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:56:47 smithi150 ceph-mon[34289]: pgmap v655: 97 pgs: 97 active+clean; 10 MiB data, 525 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 10 op/s 2024-09-21T03:56:47.834 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:56:47 smithi039 ceph-mon[31211]: pgmap v655: 97 pgs: 97 active+clean; 10 MiB data, 525 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 10 op/s 2024-09-21T03:56:47.863 INFO:teuthology.orchestra.run.smithi161.stderr:Inferring config /var/lib/ceph/51116d5c-77ca-11ef-baf5-efdc52797490/mon.c/config 2024-09-21T03:56:48.553 INFO:teuthology.orchestra.run.smithi161.stdout:{ 2024-09-21T03:56:48.553 INFO:teuthology.orchestra.run.smithi161.stdout: "AsyncMessenger::Worker-0": { 2024-09-21T03:56:48.553 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_recv_messages": 37, 2024-09-21T03:56:48.553 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_messages": 67, 2024-09-21T03:56:48.554 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_recv_bytes": 8892, 2024-09-21T03:56:48.554 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_bytes": 94375, 2024-09-21T03:56:48.554 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_created_connections": 15, 2024-09-21T03:56:48.554 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_active_connections": 3, 2024-09-21T03:56:48.554 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_total_time": 0.014626242, 2024-09-21T03:56:48.554 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_send_time": 0.002460163, 2024-09-21T03:56:48.554 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_recv_time": 0.006719172, 2024-09-21T03:56:48.554 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_fast_dispatch_time": 0.000941237, 2024-09-21T03:56:48.554 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T03:56:48.554 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 65, 2024-09-21T03:56:48.554 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.021087363, 2024-09-21T03:56:48.554 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000324420 2024-09-21T03:56:48.554 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T03:56:48.555 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_handle_ack_lat": { 2024-09-21T03:56:48.555 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 50, 2024-09-21T03:56:48.555 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000010364, 2024-09-21T03:56:48.555 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000207 2024-09-21T03:56:48.555 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T03:56:48.555 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_recv_encrypted_bytes": 3136, 2024-09-21T03:56:48.555 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_encrypted_bytes": 10912 2024-09-21T03:56:48.555 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T03:56:48.555 INFO:teuthology.orchestra.run.smithi161.stdout: "AsyncMessenger::Worker-1": { 2024-09-21T03:56:48.555 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_recv_messages": 281, 2024-09-21T03:56:48.555 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_messages": 268, 2024-09-21T03:56:48.555 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_recv_bytes": 1210689, 2024-09-21T03:56:48.555 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_bytes": 142956, 2024-09-21T03:56:48.555 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_created_connections": 6, 2024-09-21T03:56:48.556 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_active_connections": 5, 2024-09-21T03:56:48.556 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_total_time": 0.051016258, 2024-09-21T03:56:48.556 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_send_time": 0.021085963, 2024-09-21T03:56:48.556 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_recv_time": 0.043965040, 2024-09-21T03:56:48.556 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_fast_dispatch_time": 0.000383415, 2024-09-21T03:56:48.556 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T03:56:48.556 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 267, 2024-09-21T03:56:48.556 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.035135702, 2024-09-21T03:56:48.556 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000131594 2024-09-21T03:56:48.556 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T03:56:48.556 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_handle_ack_lat": { 2024-09-21T03:56:48.556 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-09-21T03:56:48.556 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-09-21T03:56:48.557 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-09-21T03:56:48.557 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T03:56:48.557 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_recv_encrypted_bytes": 1207152, 2024-09-21T03:56:48.557 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_encrypted_bytes": 136576 2024-09-21T03:56:48.557 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T03:56:48.557 INFO:teuthology.orchestra.run.smithi161.stdout: "AsyncMessenger::Worker-2": { 2024-09-21T03:56:48.557 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_recv_messages": 469, 2024-09-21T03:56:48.557 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_messages": 719, 2024-09-21T03:56:48.557 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_recv_bytes": 144692, 2024-09-21T03:56:48.557 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_bytes": 1027659, 2024-09-21T03:56:48.557 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_created_connections": 44, 2024-09-21T03:56:48.557 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_active_connections": 6, 2024-09-21T03:56:48.557 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_total_time": 0.095762078, 2024-09-21T03:56:48.557 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_send_time": 0.043265230, 2024-09-21T03:56:48.558 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_recv_time": 0.044566252, 2024-09-21T03:56:48.558 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_fast_dispatch_time": 0.004873938, 2024-09-21T03:56:48.558 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T03:56:48.558 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 718, 2024-09-21T03:56:48.558 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.046165341, 2024-09-21T03:56:48.558 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000064297 2024-09-21T03:56:48.558 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T03:56:48.558 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_handle_ack_lat": { 2024-09-21T03:56:48.558 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 966, 2024-09-21T03:56:48.558 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000258605, 2024-09-21T03:56:48.558 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000267 2024-09-21T03:56:48.558 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T03:56:48.558 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_recv_encrypted_bytes": 3200, 2024-09-21T03:56:48.558 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_encrypted_bytes": 363904 2024-09-21T03:56:48.559 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T03:56:48.559 INFO:teuthology.orchestra.run.smithi161.stdout: "cct": { 2024-09-21T03:56:48.559 INFO:teuthology.orchestra.run.smithi161.stdout: "total_workers": 1, 2024-09-21T03:56:48.559 INFO:teuthology.orchestra.run.smithi161.stdout: "unhealthy_workers": 0 2024-09-21T03:56:48.559 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T03:56:48.559 INFO:teuthology.orchestra.run.smithi161.stdout: "finisher-MDSRank": { 2024-09-21T03:56:48.559 INFO:teuthology.orchestra.run.smithi161.stdout: "queue_len": 0, 2024-09-21T03:56:48.559 INFO:teuthology.orchestra.run.smithi161.stdout: "complete_latency": { 2024-09-21T03:56:48.559 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 28, 2024-09-21T03:56:48.559 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.002174434, 2024-09-21T03:56:48.559 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000077658 2024-09-21T03:56:48.559 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T03:56:48.559 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T03:56:48.559 INFO:teuthology.orchestra.run.smithi161.stdout: "finisher-PurgeQueue": { 2024-09-21T03:56:48.560 INFO:teuthology.orchestra.run.smithi161.stdout: "queue_len": 0, 2024-09-21T03:56:48.560 INFO:teuthology.orchestra.run.smithi161.stdout: "complete_latency": { 2024-09-21T03:56:48.560 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 2, 2024-09-21T03:56:48.560 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000016442, 2024-09-21T03:56:48.560 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000008221 2024-09-21T03:56:48.560 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T03:56:48.560 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T03:56:48.560 INFO:teuthology.orchestra.run.smithi161.stdout: "mds": { 2024-09-21T03:56:48.560 INFO:teuthology.orchestra.run.smithi161.stdout: "request": 5, 2024-09-21T03:56:48.560 INFO:teuthology.orchestra.run.smithi161.stdout: "reply": 3, 2024-09-21T03:56:48.560 INFO:teuthology.orchestra.run.smithi161.stdout: "reply_latency": { 2024-09-21T03:56:48.560 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 3, 2024-09-21T03:56:48.560 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000764759, 2024-09-21T03:56:48.560 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000254919 2024-09-21T03:56:48.561 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T03:56:48.561 INFO:teuthology.orchestra.run.smithi161.stdout: "slow_reply": 0, 2024-09-21T03:56:48.561 INFO:teuthology.orchestra.run.smithi161.stdout: "forward": 2, 2024-09-21T03:56:48.561 INFO:teuthology.orchestra.run.smithi161.stdout: "dir_fetch_complete": 0, 2024-09-21T03:56:48.561 INFO:teuthology.orchestra.run.smithi161.stdout: "dir_fetch_keys": 0, 2024-09-21T03:56:48.561 INFO:teuthology.orchestra.run.smithi161.stdout: "dir_commit": 11, 2024-09-21T03:56:48.561 INFO:teuthology.orchestra.run.smithi161.stdout: "dir_split": 0, 2024-09-21T03:56:48.561 INFO:teuthology.orchestra.run.smithi161.stdout: "dir_merge": 0, 2024-09-21T03:56:48.561 INFO:teuthology.orchestra.run.smithi161.stdout: "inodes": 16, 2024-09-21T03:56:48.561 INFO:teuthology.orchestra.run.smithi161.stdout: "inodes_top": 0, 2024-09-21T03:56:48.561 INFO:teuthology.orchestra.run.smithi161.stdout: "inodes_bottom": 16, 2024-09-21T03:56:48.561 INFO:teuthology.orchestra.run.smithi161.stdout: "inodes_pin_tail": 0, 2024-09-21T03:56:48.561 INFO:teuthology.orchestra.run.smithi161.stdout: "inodes_pinned": 13, 2024-09-21T03:56:48.561 INFO:teuthology.orchestra.run.smithi161.stdout: "inodes_expired": 4, 2024-09-21T03:56:48.561 INFO:teuthology.orchestra.run.smithi161.stdout: "inodes_with_caps": 1, 2024-09-21T03:56:48.562 INFO:teuthology.orchestra.run.smithi161.stdout: "caps": 1, 2024-09-21T03:56:48.562 INFO:teuthology.orchestra.run.smithi161.stdout: "subtrees": 3, 2024-09-21T03:56:48.562 INFO:teuthology.orchestra.run.smithi161.stdout: "traverse": 14, 2024-09-21T03:56:48.562 INFO:teuthology.orchestra.run.smithi161.stdout: "traverse_hit": 10, 2024-09-21T03:56:48.562 INFO:teuthology.orchestra.run.smithi161.stdout: "traverse_forward": 0, 2024-09-21T03:56:48.562 INFO:teuthology.orchestra.run.smithi161.stdout: "traverse_discover": 3, 2024-09-21T03:56:48.562 INFO:teuthology.orchestra.run.smithi161.stdout: "traverse_dir_fetch": 0, 2024-09-21T03:56:48.562 INFO:teuthology.orchestra.run.smithi161.stdout: "traverse_remote_ino": 0, 2024-09-21T03:56:48.562 INFO:teuthology.orchestra.run.smithi161.stdout: "traverse_lock": 0, 2024-09-21T03:56:48.562 INFO:teuthology.orchestra.run.smithi161.stdout: "load_cent": 0, 2024-09-21T03:56:48.562 INFO:teuthology.orchestra.run.smithi161.stdout: "q": 0, 2024-09-21T03:56:48.562 INFO:teuthology.orchestra.run.smithi161.stdout: "exported": 0, 2024-09-21T03:56:48.563 INFO:teuthology.orchestra.run.smithi161.stdout: "exported_inodes": 0, 2024-09-21T03:56:48.563 INFO:teuthology.orchestra.run.smithi161.stdout: "imported": 0, 2024-09-21T03:56:48.563 INFO:teuthology.orchestra.run.smithi161.stdout: "imported_inodes": 0, 2024-09-21T03:56:48.563 INFO:teuthology.orchestra.run.smithi161.stdout: "openino_dir_fetch": 0, 2024-09-21T03:56:48.563 INFO:teuthology.orchestra.run.smithi161.stdout: "openino_backtrace_fetch": 0, 2024-09-21T03:56:48.563 INFO:teuthology.orchestra.run.smithi161.stdout: "openino_peer_discover": 0, 2024-09-21T03:56:48.563 INFO:teuthology.orchestra.run.smithi161.stdout: "root_rfiles": 4, 2024-09-21T03:56:48.563 INFO:teuthology.orchestra.run.smithi161.stdout: "root_rbytes": 234, 2024-09-21T03:56:48.563 INFO:teuthology.orchestra.run.smithi161.stdout: "root_rsnaps": 6, 2024-09-21T03:56:48.563 INFO:teuthology.orchestra.run.smithi161.stdout: "scrub_backtrace_fetch": 0, 2024-09-21T03:56:48.563 INFO:teuthology.orchestra.run.smithi161.stdout: "scrub_set_tag": 0, 2024-09-21T03:56:48.563 INFO:teuthology.orchestra.run.smithi161.stdout: "scrub_backtrace_repaired": 0, 2024-09-21T03:56:48.563 INFO:teuthology.orchestra.run.smithi161.stdout: "scrub_inotable_repaired": 0, 2024-09-21T03:56:48.563 INFO:teuthology.orchestra.run.smithi161.stdout: "scrub_dir_inodes": 0, 2024-09-21T03:56:48.564 INFO:teuthology.orchestra.run.smithi161.stdout: "scrub_dir_base_inodes": 0, 2024-09-21T03:56:48.564 INFO:teuthology.orchestra.run.smithi161.stdout: "scrub_dirfrag_rstats": 0, 2024-09-21T03:56:48.564 INFO:teuthology.orchestra.run.smithi161.stdout: "scrub_file_inodes": 0, 2024-09-21T03:56:48.564 INFO:teuthology.orchestra.run.smithi161.stdout: "handle_inode_file_caps": 0, 2024-09-21T03:56:48.564 INFO:teuthology.orchestra.run.smithi161.stdout: "ceph_cap_op_revoke": 0, 2024-09-21T03:56:48.564 INFO:teuthology.orchestra.run.smithi161.stdout: "ceph_cap_op_grant": 0, 2024-09-21T03:56:48.564 INFO:teuthology.orchestra.run.smithi161.stdout: "ceph_cap_op_trunc": 0, 2024-09-21T03:56:48.564 INFO:teuthology.orchestra.run.smithi161.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-21T03:56:48.564 INFO:teuthology.orchestra.run.smithi161.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-21T03:56:48.564 INFO:teuthology.orchestra.run.smithi161.stdout: "handle_client_caps": 0, 2024-09-21T03:56:48.564 INFO:teuthology.orchestra.run.smithi161.stdout: "handle_client_caps_dirty": 0, 2024-09-21T03:56:48.564 INFO:teuthology.orchestra.run.smithi161.stdout: "handle_client_cap_release": 0, 2024-09-21T03:56:48.564 INFO:teuthology.orchestra.run.smithi161.stdout: "process_request_cap_release": 0 2024-09-21T03:56:48.564 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T03:56:48.564 INFO:teuthology.orchestra.run.smithi161.stdout: "mds_cache": { 2024-09-21T03:56:48.565 INFO:teuthology.orchestra.run.smithi161.stdout: "dir_update": 0, 2024-09-21T03:56:48.565 INFO:teuthology.orchestra.run.smithi161.stdout: "dir_update_receipt": 11, 2024-09-21T03:56:48.565 INFO:teuthology.orchestra.run.smithi161.stdout: "dir_try_discover": 7, 2024-09-21T03:56:48.565 INFO:teuthology.orchestra.run.smithi161.stdout: "dir_send_discover": 8, 2024-09-21T03:56:48.565 INFO:teuthology.orchestra.run.smithi161.stdout: "dir_handle_discover": 0, 2024-09-21T03:56:48.565 INFO:teuthology.orchestra.run.smithi161.stdout: "num_strays": 0, 2024-09-21T03:56:48.565 INFO:teuthology.orchestra.run.smithi161.stdout: "num_strays_delayed": 0, 2024-09-21T03:56:48.565 INFO:teuthology.orchestra.run.smithi161.stdout: "num_strays_enqueuing": 0, 2024-09-21T03:56:48.565 INFO:teuthology.orchestra.run.smithi161.stdout: "strays_created": 0, 2024-09-21T03:56:48.565 INFO:teuthology.orchestra.run.smithi161.stdout: "strays_enqueued": 0, 2024-09-21T03:56:48.565 INFO:teuthology.orchestra.run.smithi161.stdout: "strays_reintegrated": 0, 2024-09-21T03:56:48.565 INFO:teuthology.orchestra.run.smithi161.stdout: "strays_migrated": 0, 2024-09-21T03:56:48.565 INFO:teuthology.orchestra.run.smithi161.stdout: "num_recovering_processing": 0, 2024-09-21T03:56:48.565 INFO:teuthology.orchestra.run.smithi161.stdout: "num_recovering_enqueued": 0, 2024-09-21T03:56:48.565 INFO:teuthology.orchestra.run.smithi161.stdout: "num_recovering_prioritized": 0, 2024-09-21T03:56:48.566 INFO:teuthology.orchestra.run.smithi161.stdout: "recovery_started": 0, 2024-09-21T03:56:48.566 INFO:teuthology.orchestra.run.smithi161.stdout: "recovery_completed": 0, 2024-09-21T03:56:48.566 INFO:teuthology.orchestra.run.smithi161.stdout: "ireq_quiesce_path": 0, 2024-09-21T03:56:48.566 INFO:teuthology.orchestra.run.smithi161.stdout: "ireq_quiesce_inode": 0, 2024-09-21T03:56:48.566 INFO:teuthology.orchestra.run.smithi161.stdout: "ireq_enqueue_scrub": 0, 2024-09-21T03:56:48.566 INFO:teuthology.orchestra.run.smithi161.stdout: "ireq_exportdir": 0, 2024-09-21T03:56:48.566 INFO:teuthology.orchestra.run.smithi161.stdout: "ireq_flush": 0, 2024-09-21T03:56:48.566 INFO:teuthology.orchestra.run.smithi161.stdout: "ireq_fragmentdir": 0, 2024-09-21T03:56:48.566 INFO:teuthology.orchestra.run.smithi161.stdout: "ireq_fragstats": 0, 2024-09-21T03:56:48.566 INFO:teuthology.orchestra.run.smithi161.stdout: "ireq_inodestats": 0, 2024-09-21T03:56:48.566 INFO:teuthology.orchestra.run.smithi161.stdout: "uninline_started": 0, 2024-09-21T03:56:48.566 INFO:teuthology.orchestra.run.smithi161.stdout: "uninline_succeeded": 0, 2024-09-21T03:56:48.566 INFO:teuthology.orchestra.run.smithi161.stdout: "uninline_write_failed": 0 2024-09-21T03:56:48.566 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T03:56:48.567 INFO:teuthology.orchestra.run.smithi161.stdout: "mds_log": { 2024-09-21T03:56:48.567 INFO:teuthology.orchestra.run.smithi161.stdout: "evlrg": 0, 2024-09-21T03:56:48.567 INFO:teuthology.orchestra.run.smithi161.stdout: "evadd": 3, 2024-09-21T03:56:48.567 INFO:teuthology.orchestra.run.smithi161.stdout: "evex": 0, 2024-09-21T03:56:48.567 INFO:teuthology.orchestra.run.smithi161.stdout: "evtrm": 0, 2024-09-21T03:56:48.567 INFO:teuthology.orchestra.run.smithi161.stdout: "ev": 3, 2024-09-21T03:56:48.567 INFO:teuthology.orchestra.run.smithi161.stdout: "evexg": 0, 2024-09-21T03:56:48.567 INFO:teuthology.orchestra.run.smithi161.stdout: "evexd": 0, 2024-09-21T03:56:48.567 INFO:teuthology.orchestra.run.smithi161.stdout: "segadd": 2, 2024-09-21T03:56:48.567 INFO:teuthology.orchestra.run.smithi161.stdout: "segex": 0, 2024-09-21T03:56:48.567 INFO:teuthology.orchestra.run.smithi161.stdout: "segtrm": 0, 2024-09-21T03:56:48.567 INFO:teuthology.orchestra.run.smithi161.stdout: "seg": 2, 2024-09-21T03:56:48.567 INFO:teuthology.orchestra.run.smithi161.stdout: "segmjr": 2, 2024-09-21T03:56:48.567 INFO:teuthology.orchestra.run.smithi161.stdout: "segexg": 0, 2024-09-21T03:56:48.568 INFO:teuthology.orchestra.run.smithi161.stdout: "segexd": 0, 2024-09-21T03:56:48.568 INFO:teuthology.orchestra.run.smithi161.stdout: "expos": 4194304, 2024-09-21T03:56:48.568 INFO:teuthology.orchestra.run.smithi161.stdout: "wrpos": 4195283, 2024-09-21T03:56:48.568 INFO:teuthology.orchestra.run.smithi161.stdout: "rdpos": 0, 2024-09-21T03:56:48.568 INFO:teuthology.orchestra.run.smithi161.stdout: "jlat": { 2024-09-21T03:56:48.568 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 2, 2024-09-21T03:56:48.568 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.004334350, 2024-09-21T03:56:48.568 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.002167175 2024-09-21T03:56:48.568 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T03:56:48.568 INFO:teuthology.orchestra.run.smithi161.stdout: "replayed": 0 2024-09-21T03:56:48.568 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T03:56:48.568 INFO:teuthology.orchestra.run.smithi161.stdout: "mds_mem": { 2024-09-21T03:56:48.568 INFO:teuthology.orchestra.run.smithi161.stdout: "ino": 20, 2024-09-21T03:56:48.569 INFO:teuthology.orchestra.run.smithi161.stdout: "ino+": 22, 2024-09-21T03:56:48.569 INFO:teuthology.orchestra.run.smithi161.stdout: "ino-": 2, 2024-09-21T03:56:48.569 INFO:teuthology.orchestra.run.smithi161.stdout: "dir": 19, 2024-09-21T03:56:48.569 INFO:teuthology.orchestra.run.smithi161.stdout: "dir+": 23, 2024-09-21T03:56:48.569 INFO:teuthology.orchestra.run.smithi161.stdout: "dir-": 4, 2024-09-21T03:56:48.569 INFO:teuthology.orchestra.run.smithi161.stdout: "dn": 16, 2024-09-21T03:56:48.569 INFO:teuthology.orchestra.run.smithi161.stdout: "dn+": 20, 2024-09-21T03:56:48.569 INFO:teuthology.orchestra.run.smithi161.stdout: "dn-": 4, 2024-09-21T03:56:48.569 INFO:teuthology.orchestra.run.smithi161.stdout: "cap": 1, 2024-09-21T03:56:48.569 INFO:teuthology.orchestra.run.smithi161.stdout: "cap+": 1, 2024-09-21T03:56:48.569 INFO:teuthology.orchestra.run.smithi161.stdout: "cap-": 0, 2024-09-21T03:56:48.569 INFO:teuthology.orchestra.run.smithi161.stdout: "rss": 120860, 2024-09-21T03:56:48.569 INFO:teuthology.orchestra.run.smithi161.stdout: "heap": 325857280 2024-09-21T03:56:48.569 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T03:56:48.569 INFO:teuthology.orchestra.run.smithi161.stdout: "mds_server": { 2024-09-21T03:56:48.570 INFO:teuthology.orchestra.run.smithi161.stdout: "dispatch_client_request": 5, 2024-09-21T03:56:48.570 INFO:teuthology.orchestra.run.smithi161.stdout: "dispatch_server_request": 0, 2024-09-21T03:56:48.570 INFO:teuthology.orchestra.run.smithi161.stdout: "handle_client_request": 5, 2024-09-21T03:56:48.570 INFO:teuthology.orchestra.run.smithi161.stdout: "handle_client_session": 8, 2024-09-21T03:56:48.570 INFO:teuthology.orchestra.run.smithi161.stdout: "handle_peer_request": 0, 2024-09-21T03:56:48.570 INFO:teuthology.orchestra.run.smithi161.stdout: "req_create_latency": { 2024-09-21T03:56:48.570 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-09-21T03:56:48.570 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-09-21T03:56:48.570 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-09-21T03:56:48.570 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T03:56:48.570 INFO:teuthology.orchestra.run.smithi161.stdout: "req_getattr_latency": { 2024-09-21T03:56:48.570 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 3, 2024-09-21T03:56:48.570 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000764759, 2024-09-21T03:56:48.570 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000254919 2024-09-21T03:56:48.571 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T03:56:48.571 INFO:teuthology.orchestra.run.smithi161.stdout: "req_getfilelock_latency": { 2024-09-21T03:56:48.571 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-09-21T03:56:48.571 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-09-21T03:56:48.571 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-09-21T03:56:48.571 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T03:56:48.571 INFO:teuthology.orchestra.run.smithi161.stdout: "req_link_latency": { 2024-09-21T03:56:48.571 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-09-21T03:56:48.571 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-09-21T03:56:48.571 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-09-21T03:56:48.571 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T03:56:48.571 INFO:teuthology.orchestra.run.smithi161.stdout: "req_lookup_latency": { 2024-09-21T03:56:48.571 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-09-21T03:56:48.571 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-09-21T03:56:48.571 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-09-21T03:56:48.572 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T03:56:48.572 INFO:teuthology.orchestra.run.smithi161.stdout: "req_lookuphash_latency": { 2024-09-21T03:56:48.572 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-09-21T03:56:48.572 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-09-21T03:56:48.572 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-09-21T03:56:48.572 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T03:56:48.572 INFO:teuthology.orchestra.run.smithi161.stdout: "req_lookupino_latency": { 2024-09-21T03:56:48.572 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-09-21T03:56:48.572 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-09-21T03:56:48.572 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-09-21T03:56:48.572 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T03:56:48.572 INFO:teuthology.orchestra.run.smithi161.stdout: "req_lookupname_latency": { 2024-09-21T03:56:48.572 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-09-21T03:56:48.572 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-09-21T03:56:48.572 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-09-21T03:56:48.573 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T03:56:48.573 INFO:teuthology.orchestra.run.smithi161.stdout: "req_lookupparent_latency": { 2024-09-21T03:56:48.573 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-09-21T03:56:48.573 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-09-21T03:56:48.573 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-09-21T03:56:48.573 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T03:56:48.573 INFO:teuthology.orchestra.run.smithi161.stdout: "req_lookupsnap_latency": { 2024-09-21T03:56:48.573 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-09-21T03:56:48.573 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-09-21T03:56:48.573 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-09-21T03:56:48.573 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T03:56:48.573 INFO:teuthology.orchestra.run.smithi161.stdout: "req_lssnap_latency": { 2024-09-21T03:56:48.573 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-09-21T03:56:48.573 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-09-21T03:56:48.574 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-09-21T03:56:48.574 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T03:56:48.574 INFO:teuthology.orchestra.run.smithi161.stdout: "req_mkdir_latency": { 2024-09-21T03:56:48.574 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-09-21T03:56:48.574 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-09-21T03:56:48.575 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-09-21T03:56:48.575 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T03:56:48.575 INFO:teuthology.orchestra.run.smithi161.stdout: "req_mknod_latency": { 2024-09-21T03:56:48.575 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-09-21T03:56:48.575 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-09-21T03:56:48.575 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-09-21T03:56:48.575 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T03:56:48.575 INFO:teuthology.orchestra.run.smithi161.stdout: "req_mksnap_latency": { 2024-09-21T03:56:48.575 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-09-21T03:56:48.575 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-09-21T03:56:48.575 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-09-21T03:56:48.576 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T03:56:48.576 INFO:teuthology.orchestra.run.smithi161.stdout: "req_open_latency": { 2024-09-21T03:56:48.576 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-09-21T03:56:48.576 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-09-21T03:56:48.576 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-09-21T03:56:48.576 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T03:56:48.576 INFO:teuthology.orchestra.run.smithi161.stdout: "req_readdir_latency": { 2024-09-21T03:56:48.576 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-09-21T03:56:48.576 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-09-21T03:56:48.576 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-09-21T03:56:48.576 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T03:56:48.576 INFO:teuthology.orchestra.run.smithi161.stdout: "req_rename_latency": { 2024-09-21T03:56:48.576 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-09-21T03:56:48.576 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-09-21T03:56:48.577 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-09-21T03:56:48.577 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T03:56:48.577 INFO:teuthology.orchestra.run.smithi161.stdout: "req_renamesnap_latency": { 2024-09-21T03:56:48.577 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-09-21T03:56:48.577 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-09-21T03:56:48.577 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-09-21T03:56:48.577 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T03:56:48.577 INFO:teuthology.orchestra.run.smithi161.stdout: "req_snapdiff_latency": { 2024-09-21T03:56:48.577 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-09-21T03:56:48.577 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-09-21T03:56:48.577 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-09-21T03:56:48.577 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T03:56:48.577 INFO:teuthology.orchestra.run.smithi161.stdout: "req_rmdir_latency": { 2024-09-21T03:56:48.577 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-09-21T03:56:48.577 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-09-21T03:56:48.578 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-09-21T03:56:48.578 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T03:56:48.578 INFO:teuthology.orchestra.run.smithi161.stdout: "req_rmsnap_latency": { 2024-09-21T03:56:48.578 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-09-21T03:56:48.578 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-09-21T03:56:48.578 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-09-21T03:56:48.578 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T03:56:48.578 INFO:teuthology.orchestra.run.smithi161.stdout: "req_rmxattr_latency": { 2024-09-21T03:56:48.578 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-09-21T03:56:48.578 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-09-21T03:56:48.578 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-09-21T03:56:48.578 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T03:56:48.578 INFO:teuthology.orchestra.run.smithi161.stdout: "req_setattr_latency": { 2024-09-21T03:56:48.578 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-09-21T03:56:48.579 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-09-21T03:56:48.579 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-09-21T03:56:48.579 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T03:56:48.579 INFO:teuthology.orchestra.run.smithi161.stdout: "req_setdirlayout_latency": { 2024-09-21T03:56:48.579 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-09-21T03:56:48.579 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-09-21T03:56:48.579 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-09-21T03:56:48.579 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T03:56:48.579 INFO:teuthology.orchestra.run.smithi161.stdout: "req_setfilelock_latency": { 2024-09-21T03:56:48.579 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-09-21T03:56:48.579 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-09-21T03:56:48.579 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-09-21T03:56:48.579 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T03:56:48.579 INFO:teuthology.orchestra.run.smithi161.stdout: "req_setlayout_latency": { 2024-09-21T03:56:48.579 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-09-21T03:56:48.580 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-09-21T03:56:48.580 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-09-21T03:56:48.580 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T03:56:48.580 INFO:teuthology.orchestra.run.smithi161.stdout: "req_setxattr_latency": { 2024-09-21T03:56:48.580 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-09-21T03:56:48.580 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-09-21T03:56:48.580 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-09-21T03:56:48.580 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T03:56:48.580 INFO:teuthology.orchestra.run.smithi161.stdout: "req_symlink_latency": { 2024-09-21T03:56:48.580 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-09-21T03:56:48.580 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-09-21T03:56:48.580 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-09-21T03:56:48.581 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T03:56:48.581 INFO:teuthology.orchestra.run.smithi161.stdout: "req_unlink_latency": { 2024-09-21T03:56:48.581 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-09-21T03:56:48.581 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-09-21T03:56:48.581 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-09-21T03:56:48.581 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T03:56:48.581 INFO:teuthology.orchestra.run.smithi161.stdout: "cap_revoke_eviction": 0, 2024-09-21T03:56:48.581 INFO:teuthology.orchestra.run.smithi161.stdout: "cap_acquisition_throttle": 0, 2024-09-21T03:56:48.581 INFO:teuthology.orchestra.run.smithi161.stdout: "req_getvxattr_latency": { 2024-09-21T03:56:48.581 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-09-21T03:56:48.581 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-09-21T03:56:48.581 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-09-21T03:56:48.581 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T03:56:48.581 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T03:56:48.582 INFO:teuthology.orchestra.run.smithi161.stdout: "mds_sessions": { 2024-09-21T03:56:48.582 INFO:teuthology.orchestra.run.smithi161.stdout: "session_count": 1, 2024-09-21T03:56:48.582 INFO:teuthology.orchestra.run.smithi161.stdout: "session_add": 1, 2024-09-21T03:56:48.582 INFO:teuthology.orchestra.run.smithi161.stdout: "session_remove": 0, 2024-09-21T03:56:48.582 INFO:teuthology.orchestra.run.smithi161.stdout: "sessions_open": 1, 2024-09-21T03:56:48.582 INFO:teuthology.orchestra.run.smithi161.stdout: "sessions_stale": 0, 2024-09-21T03:56:48.582 INFO:teuthology.orchestra.run.smithi161.stdout: "total_load": 2, 2024-09-21T03:56:48.582 INFO:teuthology.orchestra.run.smithi161.stdout: "average_load": 2, 2024-09-21T03:56:48.582 INFO:teuthology.orchestra.run.smithi161.stdout: "avg_session_uptime": 585, 2024-09-21T03:56:48.582 INFO:teuthology.orchestra.run.smithi161.stdout: "mdthresh_evicted": 0 2024-09-21T03:56:48.582 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T03:56:48.582 INFO:teuthology.orchestra.run.smithi161.stdout: "mempool": { 2024-09-21T03:56:48.582 INFO:teuthology.orchestra.run.smithi161.stdout: "bloom_filter_bytes": 0, 2024-09-21T03:56:48.582 INFO:teuthology.orchestra.run.smithi161.stdout: "bloom_filter_items": 0, 2024-09-21T03:56:48.582 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_alloc_bytes": 0, 2024-09-21T03:56:48.583 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_alloc_items": 0, 2024-09-21T03:56:48.583 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_cache_data_bytes": 0, 2024-09-21T03:56:48.583 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_cache_data_items": 0, 2024-09-21T03:56:48.583 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-21T03:56:48.583 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_cache_onode_items": 0, 2024-09-21T03:56:48.583 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-21T03:56:48.583 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_cache_meta_items": 0, 2024-09-21T03:56:48.583 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_cache_other_bytes": 0, 2024-09-21T03:56:48.583 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_cache_other_items": 0, 2024-09-21T03:56:48.583 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-21T03:56:48.583 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_cache_buffer_items": 0, 2024-09-21T03:56:48.583 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_extent_bytes": 0, 2024-09-21T03:56:48.583 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_extent_items": 0, 2024-09-21T03:56:48.583 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_blob_bytes": 0, 2024-09-21T03:56:48.584 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_blob_items": 0, 2024-09-21T03:56:48.584 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-21T03:56:48.584 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_shared_blob_items": 0, 2024-09-21T03:56:48.584 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-21T03:56:48.584 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_inline_bl_items": 0, 2024-09-21T03:56:48.584 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_fsck_bytes": 0, 2024-09-21T03:56:48.584 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_fsck_items": 0, 2024-09-21T03:56:48.584 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_txc_bytes": 0, 2024-09-21T03:56:48.584 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_txc_items": 0, 2024-09-21T03:56:48.584 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-21T03:56:48.584 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_writing_deferred_items": 0, 2024-09-21T03:56:48.584 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_writing_bytes": 0, 2024-09-21T03:56:48.584 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_writing_items": 0, 2024-09-21T03:56:48.584 INFO:teuthology.orchestra.run.smithi161.stdout: "bluefs_bytes": 0, 2024-09-21T03:56:48.584 INFO:teuthology.orchestra.run.smithi161.stdout: "bluefs_items": 0, 2024-09-21T03:56:48.585 INFO:teuthology.orchestra.run.smithi161.stdout: "bluefs_file_reader_bytes": 0, 2024-09-21T03:56:48.585 INFO:teuthology.orchestra.run.smithi161.stdout: "bluefs_file_reader_items": 0, 2024-09-21T03:56:48.585 INFO:teuthology.orchestra.run.smithi161.stdout: "bluefs_file_writer_bytes": 0, 2024-09-21T03:56:48.585 INFO:teuthology.orchestra.run.smithi161.stdout: "bluefs_file_writer_items": 0, 2024-09-21T03:56:48.585 INFO:teuthology.orchestra.run.smithi161.stdout: "buffer_anon_bytes": 256007, 2024-09-21T03:56:48.585 INFO:teuthology.orchestra.run.smithi161.stdout: "buffer_anon_items": 90, 2024-09-21T03:56:48.585 INFO:teuthology.orchestra.run.smithi161.stdout: "buffer_meta_bytes": 88, 2024-09-21T03:56:48.585 INFO:teuthology.orchestra.run.smithi161.stdout: "buffer_meta_items": 1, 2024-09-21T03:56:48.585 INFO:teuthology.orchestra.run.smithi161.stdout: "osd_bytes": 0, 2024-09-21T03:56:48.585 INFO:teuthology.orchestra.run.smithi161.stdout: "osd_items": 0, 2024-09-21T03:56:48.586 INFO:teuthology.orchestra.run.smithi161.stdout: "osd_mapbl_bytes": 0, 2024-09-21T03:56:48.586 INFO:teuthology.orchestra.run.smithi161.stdout: "osd_mapbl_items": 0, 2024-09-21T03:56:48.586 INFO:teuthology.orchestra.run.smithi161.stdout: "osd_pglog_bytes": 0, 2024-09-21T03:56:48.586 INFO:teuthology.orchestra.run.smithi161.stdout: "osd_pglog_items": 0, 2024-09-21T03:56:48.586 INFO:teuthology.orchestra.run.smithi161.stdout: "osdmap_bytes": 8256, 2024-09-21T03:56:48.586 INFO:teuthology.orchestra.run.smithi161.stdout: "osdmap_items": 182, 2024-09-21T03:56:48.586 INFO:teuthology.orchestra.run.smithi161.stdout: "osdmap_mapping_bytes": 0, 2024-09-21T03:56:48.586 INFO:teuthology.orchestra.run.smithi161.stdout: "osdmap_mapping_items": 0, 2024-09-21T03:56:48.586 INFO:teuthology.orchestra.run.smithi161.stdout: "pgmap_bytes": 0, 2024-09-21T03:56:48.586 INFO:teuthology.orchestra.run.smithi161.stdout: "pgmap_items": 0, 2024-09-21T03:56:48.586 INFO:teuthology.orchestra.run.smithi161.stdout: "mds_co_bytes": 86369, 2024-09-21T03:56:48.587 INFO:teuthology.orchestra.run.smithi161.stdout: "mds_co_items": 419, 2024-09-21T03:56:48.587 INFO:teuthology.orchestra.run.smithi161.stdout: "unittest_1_bytes": 0, 2024-09-21T03:56:48.587 INFO:teuthology.orchestra.run.smithi161.stdout: "unittest_1_items": 0, 2024-09-21T03:56:48.587 INFO:teuthology.orchestra.run.smithi161.stdout: "unittest_2_bytes": 0, 2024-09-21T03:56:48.587 INFO:teuthology.orchestra.run.smithi161.stdout: "unittest_2_items": 0 2024-09-21T03:56:48.587 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T03:56:48.587 INFO:teuthology.orchestra.run.smithi161.stdout: "objecter": { 2024-09-21T03:56:48.587 INFO:teuthology.orchestra.run.smithi161.stdout: "op_active": 0, 2024-09-21T03:56:48.587 INFO:teuthology.orchestra.run.smithi161.stdout: "op_laggy": 0, 2024-09-21T03:56:48.587 INFO:teuthology.orchestra.run.smithi161.stdout: "op_send": 37, 2024-09-21T03:56:48.587 INFO:teuthology.orchestra.run.smithi161.stdout: "op_send_bytes": 11191, 2024-09-21T03:56:48.588 INFO:teuthology.orchestra.run.smithi161.stdout: "op_resend": 33, 2024-09-21T03:56:48.588 INFO:teuthology.orchestra.run.smithi161.stdout: "op_reply": 37, 2024-09-21T03:56:48.588 INFO:teuthology.orchestra.run.smithi161.stdout: "op_latency": { 2024-09-21T03:56:48.588 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 37, 2024-09-21T03:56:48.588 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.221995271, 2024-09-21T03:56:48.588 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.005999872 2024-09-21T03:56:48.588 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T03:56:48.588 INFO:teuthology.orchestra.run.smithi161.stdout: "op_inflight": 0, 2024-09-21T03:56:48.588 INFO:teuthology.orchestra.run.smithi161.stdout: "oplen_avg": { 2024-09-21T03:56:48.588 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 37, 2024-09-21T03:56:48.588 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 59 2024-09-21T03:56:48.588 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T03:56:48.588 INFO:teuthology.orchestra.run.smithi161.stdout: "op": 37, 2024-09-21T03:56:48.589 INFO:teuthology.orchestra.run.smithi161.stdout: "op_r": 0, 2024-09-21T03:56:48.589 INFO:teuthology.orchestra.run.smithi161.stdout: "op_w": 37, 2024-09-21T03:56:48.589 INFO:teuthology.orchestra.run.smithi161.stdout: "op_rmw": 0, 2024-09-21T03:56:48.589 INFO:teuthology.orchestra.run.smithi161.stdout: "op_pg": 0, 2024-09-21T03:56:48.589 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_stat": 0, 2024-09-21T03:56:48.589 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_create": 10, 2024-09-21T03:56:48.589 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_read": 0, 2024-09-21T03:56:48.589 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_write": 2, 2024-09-21T03:56:48.589 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_writefull": 6, 2024-09-21T03:56:48.589 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_writesame": 0, 2024-09-21T03:56:48.589 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_append": 0, 2024-09-21T03:56:48.590 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_zero": 0, 2024-09-21T03:56:48.590 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_truncate": 0, 2024-09-21T03:56:48.590 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_delete": 6, 2024-09-21T03:56:48.590 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_mapext": 0, 2024-09-21T03:56:48.590 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_sparse_read": 0, 2024-09-21T03:56:48.590 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_clonerange": 0, 2024-09-21T03:56:48.590 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_getxattr": 0, 2024-09-21T03:56:48.590 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_setxattr": 20, 2024-09-21T03:56:48.590 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_cmpxattr": 0, 2024-09-21T03:56:48.590 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_rmxattr": 0, 2024-09-21T03:56:48.590 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_resetxattrs": 0, 2024-09-21T03:56:48.590 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_call": 0, 2024-09-21T03:56:48.590 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_watch": 0, 2024-09-21T03:56:48.591 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_notify": 0, 2024-09-21T03:56:48.591 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_src_cmpxattr": 0, 2024-09-21T03:56:48.591 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_pgls": 0, 2024-09-21T03:56:48.591 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_pgls_filter": 0, 2024-09-21T03:56:48.591 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_other": 0, 2024-09-21T03:56:48.591 INFO:teuthology.orchestra.run.smithi161.stdout: "linger_active": 0, 2024-09-21T03:56:48.591 INFO:teuthology.orchestra.run.smithi161.stdout: "linger_send": 0, 2024-09-21T03:56:48.591 INFO:teuthology.orchestra.run.smithi161.stdout: "linger_resend": 0, 2024-09-21T03:56:48.591 INFO:teuthology.orchestra.run.smithi161.stdout: "linger_ping": 0, 2024-09-21T03:56:48.591 INFO:teuthology.orchestra.run.smithi161.stdout: "poolop_active": 0, 2024-09-21T03:56:48.592 INFO:teuthology.orchestra.run.smithi161.stdout: "poolop_send": 0, 2024-09-21T03:56:48.592 INFO:teuthology.orchestra.run.smithi161.stdout: "poolop_resend": 0, 2024-09-21T03:56:48.592 INFO:teuthology.orchestra.run.smithi161.stdout: "poolstat_active": 0, 2024-09-21T03:56:48.592 INFO:teuthology.orchestra.run.smithi161.stdout: "poolstat_send": 0, 2024-09-21T03:56:48.592 INFO:teuthology.orchestra.run.smithi161.stdout: "poolstat_resend": 0, 2024-09-21T03:56:48.592 INFO:teuthology.orchestra.run.smithi161.stdout: "statfs_active": 0, 2024-09-21T03:56:48.592 INFO:teuthology.orchestra.run.smithi161.stdout: "statfs_send": 0, 2024-09-21T03:56:48.592 INFO:teuthology.orchestra.run.smithi161.stdout: "statfs_resend": 0, 2024-09-21T03:56:48.592 INFO:teuthology.orchestra.run.smithi161.stdout: "command_active": 0, 2024-09-21T03:56:48.592 INFO:teuthology.orchestra.run.smithi161.stdout: "command_send": 0, 2024-09-21T03:56:48.592 INFO:teuthology.orchestra.run.smithi161.stdout: "command_resend": 0, 2024-09-21T03:56:48.593 INFO:teuthology.orchestra.run.smithi161.stdout: "map_epoch": 87, 2024-09-21T03:56:48.593 INFO:teuthology.orchestra.run.smithi161.stdout: "map_full": 0, 2024-09-21T03:56:48.593 INFO:teuthology.orchestra.run.smithi161.stdout: "map_inc": 10, 2024-09-21T03:56:48.593 INFO:teuthology.orchestra.run.smithi161.stdout: "osd_sessions": 9, 2024-09-21T03:56:48.593 INFO:teuthology.orchestra.run.smithi161.stdout: "osd_session_open": 9, 2024-09-21T03:56:48.593 INFO:teuthology.orchestra.run.smithi161.stdout: "osd_session_close": 0, 2024-09-21T03:56:48.593 INFO:teuthology.orchestra.run.smithi161.stdout: "osd_laggy": 0, 2024-09-21T03:56:48.593 INFO:teuthology.orchestra.run.smithi161.stdout: "omap_wr": 14, 2024-09-21T03:56:48.593 INFO:teuthology.orchestra.run.smithi161.stdout: "omap_rd": 0, 2024-09-21T03:56:48.593 INFO:teuthology.orchestra.run.smithi161.stdout: "omap_del": 1 2024-09-21T03:56:48.593 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T03:56:48.594 INFO:teuthology.orchestra.run.smithi161.stdout: "oft": { 2024-09-21T03:56:48.594 INFO:teuthology.orchestra.run.smithi161.stdout: "omap_total_objs": 1, 2024-09-21T03:56:48.594 INFO:teuthology.orchestra.run.smithi161.stdout: "omap_total_kv_pairs": 0, 2024-09-21T03:56:48.594 INFO:teuthology.orchestra.run.smithi161.stdout: "omap_total_updates": 0, 2024-09-21T03:56:48.594 INFO:teuthology.orchestra.run.smithi161.stdout: "omap_total_removes": 0 2024-09-21T03:56:48.594 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T03:56:48.594 INFO:teuthology.orchestra.run.smithi161.stdout: "purge_queue": { 2024-09-21T03:56:48.594 INFO:teuthology.orchestra.run.smithi161.stdout: "pq_executing_ops": 0, 2024-09-21T03:56:48.594 INFO:teuthology.orchestra.run.smithi161.stdout: "pq_executing_ops_high_water": 0, 2024-09-21T03:56:48.594 INFO:teuthology.orchestra.run.smithi161.stdout: "pq_executing": 0, 2024-09-21T03:56:48.594 INFO:teuthology.orchestra.run.smithi161.stdout: "pq_executing_high_water": 0, 2024-09-21T03:56:48.594 INFO:teuthology.orchestra.run.smithi161.stdout: "pq_executed_ops": 0, 2024-09-21T03:56:48.594 INFO:teuthology.orchestra.run.smithi161.stdout: "pq_executed": 0, 2024-09-21T03:56:48.594 INFO:teuthology.orchestra.run.smithi161.stdout: "pq_item_in_journal": 0 2024-09-21T03:56:48.595 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T03:56:48.595 INFO:teuthology.orchestra.run.smithi161.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-21T03:56:48.595 INFO:teuthology.orchestra.run.smithi161.stdout: "val": 0, 2024-09-21T03:56:48.595 INFO:teuthology.orchestra.run.smithi161.stdout: "max": 104857600, 2024-09-21T03:56:48.595 INFO:teuthology.orchestra.run.smithi161.stdout: "get_started": 0, 2024-09-21T03:56:48.595 INFO:teuthology.orchestra.run.smithi161.stdout: "get": 783, 2024-09-21T03:56:48.595 INFO:teuthology.orchestra.run.smithi161.stdout: "get_sum": 1276625, 2024-09-21T03:56:48.595 INFO:teuthology.orchestra.run.smithi161.stdout: "get_or_fail_fail": 0, 2024-09-21T03:56:48.595 INFO:teuthology.orchestra.run.smithi161.stdout: "get_or_fail_success": 783, 2024-09-21T03:56:48.595 INFO:teuthology.orchestra.run.smithi161.stdout: "take": 0, 2024-09-21T03:56:48.595 INFO:teuthology.orchestra.run.smithi161.stdout: "take_sum": 0, 2024-09-21T03:56:48.595 INFO:teuthology.orchestra.run.smithi161.stdout: "put": 783, 2024-09-21T03:56:48.595 INFO:teuthology.orchestra.run.smithi161.stdout: "put_sum": 1276625, 2024-09-21T03:56:48.595 INFO:teuthology.orchestra.run.smithi161.stdout: "wait": { 2024-09-21T03:56:48.596 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-09-21T03:56:48.596 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-09-21T03:56:48.596 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-09-21T03:56:48.596 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T03:56:48.596 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T03:56:48.596 INFO:teuthology.orchestra.run.smithi161.stdout: "throttle-objecter_bytes": { 2024-09-21T03:56:48.596 INFO:teuthology.orchestra.run.smithi161.stdout: "val": 0, 2024-09-21T03:56:48.596 INFO:teuthology.orchestra.run.smithi161.stdout: "max": 104857600, 2024-09-21T03:56:48.596 INFO:teuthology.orchestra.run.smithi161.stdout: "get_started": 0, 2024-09-21T03:56:48.596 INFO:teuthology.orchestra.run.smithi161.stdout: "get": 0, 2024-09-21T03:56:48.596 INFO:teuthology.orchestra.run.smithi161.stdout: "get_sum": 0, 2024-09-21T03:56:48.596 INFO:teuthology.orchestra.run.smithi161.stdout: "get_or_fail_fail": 0, 2024-09-21T03:56:48.596 INFO:teuthology.orchestra.run.smithi161.stdout: "get_or_fail_success": 0, 2024-09-21T03:56:48.596 INFO:teuthology.orchestra.run.smithi161.stdout: "take": 37, 2024-09-21T03:56:48.596 INFO:teuthology.orchestra.run.smithi161.stdout: "take_sum": 11191, 2024-09-21T03:56:48.596 INFO:teuthology.orchestra.run.smithi161.stdout: "put": 37, 2024-09-21T03:56:48.597 INFO:teuthology.orchestra.run.smithi161.stdout: "put_sum": 11191, 2024-09-21T03:56:48.597 INFO:teuthology.orchestra.run.smithi161.stdout: "wait": { 2024-09-21T03:56:48.597 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-09-21T03:56:48.597 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-09-21T03:56:48.597 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-09-21T03:56:48.597 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T03:56:48.597 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T03:56:48.597 INFO:teuthology.orchestra.run.smithi161.stdout: "throttle-objecter_ops": { 2024-09-21T03:56:48.597 INFO:teuthology.orchestra.run.smithi161.stdout: "val": 0, 2024-09-21T03:56:48.597 INFO:teuthology.orchestra.run.smithi161.stdout: "max": 1024, 2024-09-21T03:56:48.597 INFO:teuthology.orchestra.run.smithi161.stdout: "get_started": 0, 2024-09-21T03:56:48.597 INFO:teuthology.orchestra.run.smithi161.stdout: "get": 0, 2024-09-21T03:56:48.597 INFO:teuthology.orchestra.run.smithi161.stdout: "get_sum": 0, 2024-09-21T03:56:48.597 INFO:teuthology.orchestra.run.smithi161.stdout: "get_or_fail_fail": 0, 2024-09-21T03:56:48.598 INFO:teuthology.orchestra.run.smithi161.stdout: "get_or_fail_success": 0, 2024-09-21T03:56:48.598 INFO:teuthology.orchestra.run.smithi161.stdout: "take": 37, 2024-09-21T03:56:48.598 INFO:teuthology.orchestra.run.smithi161.stdout: "take_sum": 37, 2024-09-21T03:56:48.598 INFO:teuthology.orchestra.run.smithi161.stdout: "put": 37, 2024-09-21T03:56:48.598 INFO:teuthology.orchestra.run.smithi161.stdout: "put_sum": 37, 2024-09-21T03:56:48.598 INFO:teuthology.orchestra.run.smithi161.stdout: "wait": { 2024-09-21T03:56:48.598 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-09-21T03:56:48.598 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-09-21T03:56:48.598 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-09-21T03:56:48.598 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T03:56:48.598 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T03:56:48.598 INFO:teuthology.orchestra.run.smithi161.stdout: "throttle-write_buf_throttle": { 2024-09-21T03:56:48.599 INFO:teuthology.orchestra.run.smithi161.stdout: "val": 0, 2024-09-21T03:56:48.599 INFO:teuthology.orchestra.run.smithi161.stdout: "max": 3758096384, 2024-09-21T03:56:48.599 INFO:teuthology.orchestra.run.smithi161.stdout: "get_started": 0, 2024-09-21T03:56:48.599 INFO:teuthology.orchestra.run.smithi161.stdout: "get": 0, 2024-09-21T03:56:48.599 INFO:teuthology.orchestra.run.smithi161.stdout: "get_sum": 0, 2024-09-21T03:56:48.599 INFO:teuthology.orchestra.run.smithi161.stdout: "get_or_fail_fail": 0, 2024-09-21T03:56:48.599 INFO:teuthology.orchestra.run.smithi161.stdout: "get_or_fail_success": 0, 2024-09-21T03:56:48.599 INFO:teuthology.orchestra.run.smithi161.stdout: "take": 0, 2024-09-21T03:56:48.599 INFO:teuthology.orchestra.run.smithi161.stdout: "take_sum": 0, 2024-09-21T03:56:48.599 INFO:teuthology.orchestra.run.smithi161.stdout: "put": 0, 2024-09-21T03:56:48.599 INFO:teuthology.orchestra.run.smithi161.stdout: "put_sum": 0, 2024-09-21T03:56:48.599 INFO:teuthology.orchestra.run.smithi161.stdout: "wait": { 2024-09-21T03:56:48.599 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-09-21T03:56:48.599 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-09-21T03:56:48.599 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-09-21T03:56:48.600 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T03:56:48.600 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T03:56:48.600 INFO:teuthology.orchestra.run.smithi161.stdout: "throttle-write_buf_throttle-0x561298b436c0": { 2024-09-21T03:56:48.600 INFO:teuthology.orchestra.run.smithi161.stdout: "val": 0, 2024-09-21T03:56:48.600 INFO:teuthology.orchestra.run.smithi161.stdout: "max": 3758096384, 2024-09-21T03:56:48.600 INFO:teuthology.orchestra.run.smithi161.stdout: "get_started": 0, 2024-09-21T03:56:48.600 INFO:teuthology.orchestra.run.smithi161.stdout: "get": 3, 2024-09-21T03:56:48.600 INFO:teuthology.orchestra.run.smithi161.stdout: "get_sum": 979, 2024-09-21T03:56:48.600 INFO:teuthology.orchestra.run.smithi161.stdout: "get_or_fail_fail": 0, 2024-09-21T03:56:48.600 INFO:teuthology.orchestra.run.smithi161.stdout: "get_or_fail_success": 3, 2024-09-21T03:56:48.600 INFO:teuthology.orchestra.run.smithi161.stdout: "take": 0, 2024-09-21T03:56:48.600 INFO:teuthology.orchestra.run.smithi161.stdout: "take_sum": 0, 2024-09-21T03:56:48.600 INFO:teuthology.orchestra.run.smithi161.stdout: "put": 2, 2024-09-21T03:56:48.600 INFO:teuthology.orchestra.run.smithi161.stdout: "put_sum": 979, 2024-09-21T03:56:48.600 INFO:teuthology.orchestra.run.smithi161.stdout: "wait": { 2024-09-21T03:56:48.600 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-09-21T03:56:48.601 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-09-21T03:56:48.601 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-09-21T03:56:48.601 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T03:56:48.601 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T03:56:48.601 INFO:teuthology.orchestra.run.smithi161.stdout: "trackedop": { 2024-09-21T03:56:48.601 INFO:teuthology.orchestra.run.smithi161.stdout: "slow_ops_count": 0 2024-09-21T03:56:48.601 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T03:56:48.601 INFO:teuthology.orchestra.run.smithi161.stdout:} 2024-09-21T03:56:49.468 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:56:49 smithi161 ceph-mon[34533]: pgmap v656: 97 pgs: 97 active+clean; 10 MiB data, 525 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-09-21T03:56:49.524 INFO:tasks.check_counter:Daemon mds.c mds.exported=0 2024-09-21T03:56:49.524 INFO:tasks.check_counter:Daemon mds.c mds.imported=0 2024-09-21T03:56:49.524 INFO:tasks.check_counter:Daemon mds.c mds_cache.dir_update=0 2024-09-21T03:56:49.524 INFO:tasks.check_counter:Daemon mds.c mds_cache.dir_update_receipt=11 2024-09-21T03:56:49.524 INFO:tasks.check_counter:Daemon mds.c mds.root_rsnaps=6 2024-09-21T03:56:49.524 INFO:tasks.check_counter:Daemon mds.c mds_server.req_mksnap_latency.avgcount=0 2024-09-21T03:56:49.524 INFO:tasks.check_counter:Daemon mds.c mds_server.req_rmsnap_latency.avgcount=0 2024-09-21T03:56:49.524 DEBUG:tasks.check_counter:Getting stats from f 2024-09-21T03:56:49.525 DEBUG:teuthology.orchestra.run.smithi161:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 51116d5c-77ca-11ef-baf5-efdc52797490 -- ceph daemon mds.f perf dump 2024-09-21T03:56:49.724 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:56:49 smithi150 ceph-mon[34289]: pgmap v656: 97 pgs: 97 active+clean; 10 MiB data, 525 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-09-21T03:56:49.814 INFO:teuthology.orchestra.run.smithi161.stderr:Inferring config /var/lib/ceph/51116d5c-77ca-11ef-baf5-efdc52797490/mon.c/config 2024-09-21T03:56:49.833 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:56:49 smithi039 ceph-mon[31211]: pgmap v656: 97 pgs: 97 active+clean; 10 MiB data, 525 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-09-21T03:56:50.327 INFO:teuthology.orchestra.run.smithi161.stdout:{ 2024-09-21T03:56:50.327 INFO:teuthology.orchestra.run.smithi161.stdout: "AsyncMessenger::Worker-0": { 2024-09-21T03:56:50.327 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_recv_messages": 1011, 2024-09-21T03:56:50.327 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_messages": 1013, 2024-09-21T03:56:50.328 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_recv_bytes": 4514033, 2024-09-21T03:56:50.328 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_bytes": 321508, 2024-09-21T03:56:50.328 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_created_connections": 6, 2024-09-21T03:56:50.328 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_active_connections": 2, 2024-09-21T03:56:50.328 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_total_time": 0.144017336, 2024-09-21T03:56:50.328 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_send_time": 0.061019826, 2024-09-21T03:56:50.328 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_recv_time": 0.034466722, 2024-09-21T03:56:50.328 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_fast_dispatch_time": 0.036997302, 2024-09-21T03:56:50.328 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T03:56:50.328 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 1011, 2024-09-21T03:56:50.328 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.057010449, 2024-09-21T03:56:50.328 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000056390 2024-09-21T03:56:50.329 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T03:56:50.329 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_handle_ack_lat": { 2024-09-21T03:56:50.329 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-09-21T03:56:50.329 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-09-21T03:56:50.329 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-09-21T03:56:50.329 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T03:56:50.329 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_recv_encrypted_bytes": 0, 2024-09-21T03:56:50.329 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_encrypted_bytes": 0 2024-09-21T03:56:50.329 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T03:56:50.329 INFO:teuthology.orchestra.run.smithi161.stdout: "AsyncMessenger::Worker-1": { 2024-09-21T03:56:50.329 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_recv_messages": 1048, 2024-09-21T03:56:50.330 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_messages": 1205, 2024-09-21T03:56:50.330 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_recv_bytes": 1398323, 2024-09-21T03:56:50.330 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_bytes": 695836, 2024-09-21T03:56:50.330 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_created_connections": 5, 2024-09-21T03:56:50.330 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_active_connections": 4, 2024-09-21T03:56:50.330 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_total_time": 0.113567475, 2024-09-21T03:56:50.330 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_send_time": 0.054803753, 2024-09-21T03:56:50.330 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_recv_time": 0.023346814, 2024-09-21T03:56:50.330 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_fast_dispatch_time": 0.027616207, 2024-09-21T03:56:50.330 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T03:56:50.330 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 1204, 2024-09-21T03:56:50.330 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.071221242, 2024-09-21T03:56:50.331 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000059153 2024-09-21T03:56:50.331 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T03:56:50.331 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_handle_ack_lat": { 2024-09-21T03:56:50.331 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-09-21T03:56:50.331 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-09-21T03:56:50.331 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-09-21T03:56:50.331 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T03:56:50.331 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_recv_encrypted_bytes": 3200, 2024-09-21T03:56:50.331 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_encrypted_bytes": 364096 2024-09-21T03:56:50.331 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T03:56:50.331 INFO:teuthology.orchestra.run.smithi161.stdout: "AsyncMessenger::Worker-2": { 2024-09-21T03:56:50.331 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_recv_messages": 492, 2024-09-21T03:56:50.332 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_messages": 480, 2024-09-21T03:56:50.332 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_recv_bytes": 5456635, 2024-09-21T03:56:50.332 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_bytes": 212709, 2024-09-21T03:56:50.332 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_created_connections": 3, 2024-09-21T03:56:50.332 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_active_connections": 2, 2024-09-21T03:56:50.332 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_total_time": 0.081628870, 2024-09-21T03:56:50.332 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_send_time": 0.034930681, 2024-09-21T03:56:50.332 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_recv_time": 0.047490376, 2024-09-21T03:56:50.332 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_fast_dispatch_time": 0.009117487, 2024-09-21T03:56:50.332 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T03:56:50.332 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 479, 2024-09-21T03:56:50.332 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.026437479, 2024-09-21T03:56:50.333 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000055193 2024-09-21T03:56:50.333 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T03:56:50.333 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_handle_ack_lat": { 2024-09-21T03:56:50.333 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-09-21T03:56:50.333 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-09-21T03:56:50.333 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-09-21T03:56:50.333 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T03:56:50.333 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_recv_encrypted_bytes": 1199312, 2024-09-21T03:56:50.333 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_encrypted_bytes": 131936 2024-09-21T03:56:50.333 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T03:56:50.333 INFO:teuthology.orchestra.run.smithi161.stdout: "cct": { 2024-09-21T03:56:50.333 INFO:teuthology.orchestra.run.smithi161.stdout: "total_workers": 1, 2024-09-21T03:56:50.334 INFO:teuthology.orchestra.run.smithi161.stdout: "unhealthy_workers": 0 2024-09-21T03:56:50.334 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T03:56:50.334 INFO:teuthology.orchestra.run.smithi161.stdout: "finisher-MDSRank": { 2024-09-21T03:56:50.334 INFO:teuthology.orchestra.run.smithi161.stdout: "queue_len": 0, 2024-09-21T03:56:50.334 INFO:teuthology.orchestra.run.smithi161.stdout: "complete_latency": { 2024-09-21T03:56:50.334 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 3848, 2024-09-21T03:56:50.334 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.117870803, 2024-09-21T03:56:50.334 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000030631 2024-09-21T03:56:50.334 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T03:56:50.334 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T03:56:50.334 INFO:teuthology.orchestra.run.smithi161.stdout: "finisher-PurgeQueue": { 2024-09-21T03:56:50.334 INFO:teuthology.orchestra.run.smithi161.stdout: "queue_len": 0, 2024-09-21T03:56:50.334 INFO:teuthology.orchestra.run.smithi161.stdout: "complete_latency": { 2024-09-21T03:56:50.335 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-09-21T03:56:50.335 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-09-21T03:56:50.335 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-09-21T03:56:50.335 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T03:56:50.335 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T03:56:50.335 INFO:teuthology.orchestra.run.smithi161.stdout: "mds": { 2024-09-21T03:56:50.335 INFO:teuthology.orchestra.run.smithi161.stdout: "request": 0, 2024-09-21T03:56:50.335 INFO:teuthology.orchestra.run.smithi161.stdout: "reply": 0, 2024-09-21T03:56:50.335 INFO:teuthology.orchestra.run.smithi161.stdout: "reply_latency": { 2024-09-21T03:56:50.335 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-09-21T03:56:50.335 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-09-21T03:56:50.335 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-09-21T03:56:50.336 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T03:56:50.336 INFO:teuthology.orchestra.run.smithi161.stdout: "slow_reply": 0, 2024-09-21T03:56:50.336 INFO:teuthology.orchestra.run.smithi161.stdout: "forward": 0, 2024-09-21T03:56:50.336 INFO:teuthology.orchestra.run.smithi161.stdout: "dir_fetch_complete": 0, 2024-09-21T03:56:50.336 INFO:teuthology.orchestra.run.smithi161.stdout: "dir_fetch_keys": 0, 2024-09-21T03:56:50.336 INFO:teuthology.orchestra.run.smithi161.stdout: "dir_commit": 0, 2024-09-21T03:56:50.336 INFO:teuthology.orchestra.run.smithi161.stdout: "dir_split": 0, 2024-09-21T03:56:50.336 INFO:teuthology.orchestra.run.smithi161.stdout: "dir_merge": 0, 2024-09-21T03:56:50.336 INFO:teuthology.orchestra.run.smithi161.stdout: "inodes": 26, 2024-09-21T03:56:50.336 INFO:teuthology.orchestra.run.smithi161.stdout: "inodes_top": 0, 2024-09-21T03:56:50.336 INFO:teuthology.orchestra.run.smithi161.stdout: "inodes_bottom": 26, 2024-09-21T03:56:50.336 INFO:teuthology.orchestra.run.smithi161.stdout: "inodes_pin_tail": 0, 2024-09-21T03:56:50.337 INFO:teuthology.orchestra.run.smithi161.stdout: "inodes_pinned": 26, 2024-09-21T03:56:50.337 INFO:teuthology.orchestra.run.smithi161.stdout: "inodes_expired": 0, 2024-09-21T03:56:50.337 INFO:teuthology.orchestra.run.smithi161.stdout: "inodes_with_caps": 0, 2024-09-21T03:56:50.337 INFO:teuthology.orchestra.run.smithi161.stdout: "caps": 0, 2024-09-21T03:56:50.337 INFO:teuthology.orchestra.run.smithi161.stdout: "subtrees": 2, 2024-09-21T03:56:50.337 INFO:teuthology.orchestra.run.smithi161.stdout: "traverse": 0, 2024-09-21T03:56:50.337 INFO:teuthology.orchestra.run.smithi161.stdout: "traverse_hit": 0, 2024-09-21T03:56:50.337 INFO:teuthology.orchestra.run.smithi161.stdout: "traverse_forward": 0, 2024-09-21T03:56:50.337 INFO:teuthology.orchestra.run.smithi161.stdout: "traverse_discover": 0, 2024-09-21T03:56:50.337 INFO:teuthology.orchestra.run.smithi161.stdout: "traverse_dir_fetch": 0, 2024-09-21T03:56:50.337 INFO:teuthology.orchestra.run.smithi161.stdout: "traverse_remote_ino": 0, 2024-09-21T03:56:50.337 INFO:teuthology.orchestra.run.smithi161.stdout: "traverse_lock": 0, 2024-09-21T03:56:50.337 INFO:teuthology.orchestra.run.smithi161.stdout: "load_cent": 0, 2024-09-21T03:56:50.338 INFO:teuthology.orchestra.run.smithi161.stdout: "q": 0, 2024-09-21T03:56:50.338 INFO:teuthology.orchestra.run.smithi161.stdout: "exported": 0, 2024-09-21T03:56:50.338 INFO:teuthology.orchestra.run.smithi161.stdout: "exported_inodes": 0, 2024-09-21T03:56:50.338 INFO:teuthology.orchestra.run.smithi161.stdout: "imported": 0, 2024-09-21T03:56:50.338 INFO:teuthology.orchestra.run.smithi161.stdout: "imported_inodes": 0, 2024-09-21T03:56:50.338 INFO:teuthology.orchestra.run.smithi161.stdout: "openino_dir_fetch": 0, 2024-09-21T03:56:50.338 INFO:teuthology.orchestra.run.smithi161.stdout: "openino_backtrace_fetch": 0, 2024-09-21T03:56:50.338 INFO:teuthology.orchestra.run.smithi161.stdout: "openino_peer_discover": 0, 2024-09-21T03:56:50.338 INFO:teuthology.orchestra.run.smithi161.stdout: "root_rfiles": 4, 2024-09-21T03:56:50.338 INFO:teuthology.orchestra.run.smithi161.stdout: "root_rbytes": 234, 2024-09-21T03:56:50.338 INFO:teuthology.orchestra.run.smithi161.stdout: "root_rsnaps": 6, 2024-09-21T03:56:50.338 INFO:teuthology.orchestra.run.smithi161.stdout: "scrub_backtrace_fetch": 0, 2024-09-21T03:56:50.339 INFO:teuthology.orchestra.run.smithi161.stdout: "scrub_set_tag": 0, 2024-09-21T03:56:50.339 INFO:teuthology.orchestra.run.smithi161.stdout: "scrub_backtrace_repaired": 0, 2024-09-21T03:56:50.339 INFO:teuthology.orchestra.run.smithi161.stdout: "scrub_inotable_repaired": 0, 2024-09-21T03:56:50.339 INFO:teuthology.orchestra.run.smithi161.stdout: "scrub_dir_inodes": 0, 2024-09-21T03:56:50.339 INFO:teuthology.orchestra.run.smithi161.stdout: "scrub_dir_base_inodes": 0, 2024-09-21T03:56:50.339 INFO:teuthology.orchestra.run.smithi161.stdout: "scrub_dirfrag_rstats": 0, 2024-09-21T03:56:50.339 INFO:teuthology.orchestra.run.smithi161.stdout: "scrub_file_inodes": 0, 2024-09-21T03:56:50.339 INFO:teuthology.orchestra.run.smithi161.stdout: "handle_inode_file_caps": 0, 2024-09-21T03:56:50.339 INFO:teuthology.orchestra.run.smithi161.stdout: "ceph_cap_op_revoke": 0, 2024-09-21T03:56:50.339 INFO:teuthology.orchestra.run.smithi161.stdout: "ceph_cap_op_grant": 0, 2024-09-21T03:56:50.339 INFO:teuthology.orchestra.run.smithi161.stdout: "ceph_cap_op_trunc": 0, 2024-09-21T03:56:50.339 INFO:teuthology.orchestra.run.smithi161.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-21T03:56:50.339 INFO:teuthology.orchestra.run.smithi161.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-21T03:56:50.340 INFO:teuthology.orchestra.run.smithi161.stdout: "handle_client_caps": 0, 2024-09-21T03:56:50.340 INFO:teuthology.orchestra.run.smithi161.stdout: "handle_client_caps_dirty": 0, 2024-09-21T03:56:50.340 INFO:teuthology.orchestra.run.smithi161.stdout: "handle_client_cap_release": 0, 2024-09-21T03:56:50.340 INFO:teuthology.orchestra.run.smithi161.stdout: "process_request_cap_release": 0 2024-09-21T03:56:50.340 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T03:56:50.340 INFO:teuthology.orchestra.run.smithi161.stdout: "mds_cache": { 2024-09-21T03:56:50.340 INFO:teuthology.orchestra.run.smithi161.stdout: "dir_update": 0, 2024-09-21T03:56:50.340 INFO:teuthology.orchestra.run.smithi161.stdout: "dir_update_receipt": 0, 2024-09-21T03:56:50.340 INFO:teuthology.orchestra.run.smithi161.stdout: "dir_try_discover": 0, 2024-09-21T03:56:50.340 INFO:teuthology.orchestra.run.smithi161.stdout: "dir_send_discover": 0, 2024-09-21T03:56:50.340 INFO:teuthology.orchestra.run.smithi161.stdout: "dir_handle_discover": 0, 2024-09-21T03:56:50.340 INFO:teuthology.orchestra.run.smithi161.stdout: "num_strays": 0, 2024-09-21T03:56:50.341 INFO:teuthology.orchestra.run.smithi161.stdout: "num_strays_delayed": 0, 2024-09-21T03:56:50.341 INFO:teuthology.orchestra.run.smithi161.stdout: "num_strays_enqueuing": 0, 2024-09-21T03:56:50.341 INFO:teuthology.orchestra.run.smithi161.stdout: "strays_created": 0, 2024-09-21T03:56:50.341 INFO:teuthology.orchestra.run.smithi161.stdout: "strays_enqueued": 0, 2024-09-21T03:56:50.341 INFO:teuthology.orchestra.run.smithi161.stdout: "strays_reintegrated": 0, 2024-09-21T03:56:50.341 INFO:teuthology.orchestra.run.smithi161.stdout: "strays_migrated": 0, 2024-09-21T03:56:50.341 INFO:teuthology.orchestra.run.smithi161.stdout: "num_recovering_processing": 0, 2024-09-21T03:56:50.341 INFO:teuthology.orchestra.run.smithi161.stdout: "num_recovering_enqueued": 0, 2024-09-21T03:56:50.341 INFO:teuthology.orchestra.run.smithi161.stdout: "num_recovering_prioritized": 0, 2024-09-21T03:56:50.341 INFO:teuthology.orchestra.run.smithi161.stdout: "recovery_started": 0, 2024-09-21T03:56:50.341 INFO:teuthology.orchestra.run.smithi161.stdout: "recovery_completed": 0, 2024-09-21T03:56:50.341 INFO:teuthology.orchestra.run.smithi161.stdout: "ireq_quiesce_path": 0, 2024-09-21T03:56:50.341 INFO:teuthology.orchestra.run.smithi161.stdout: "ireq_quiesce_inode": 0, 2024-09-21T03:56:50.342 INFO:teuthology.orchestra.run.smithi161.stdout: "ireq_enqueue_scrub": 0, 2024-09-21T03:56:50.342 INFO:teuthology.orchestra.run.smithi161.stdout: "ireq_exportdir": 0, 2024-09-21T03:56:50.342 INFO:teuthology.orchestra.run.smithi161.stdout: "ireq_flush": 0, 2024-09-21T03:56:50.342 INFO:teuthology.orchestra.run.smithi161.stdout: "ireq_fragmentdir": 0, 2024-09-21T03:56:50.342 INFO:teuthology.orchestra.run.smithi161.stdout: "ireq_fragstats": 0, 2024-09-21T03:56:50.342 INFO:teuthology.orchestra.run.smithi161.stdout: "ireq_inodestats": 0, 2024-09-21T03:56:50.342 INFO:teuthology.orchestra.run.smithi161.stdout: "uninline_started": 0, 2024-09-21T03:56:50.342 INFO:teuthology.orchestra.run.smithi161.stdout: "uninline_succeeded": 0, 2024-09-21T03:56:50.342 INFO:teuthology.orchestra.run.smithi161.stdout: "uninline_write_failed": 0 2024-09-21T03:56:50.342 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T03:56:50.342 INFO:teuthology.orchestra.run.smithi161.stdout: "mds_log": { 2024-09-21T03:56:50.342 INFO:teuthology.orchestra.run.smithi161.stdout: "evlrg": 0, 2024-09-21T03:56:50.343 INFO:teuthology.orchestra.run.smithi161.stdout: "evadd": 0, 2024-09-21T03:56:50.343 INFO:teuthology.orchestra.run.smithi161.stdout: "evex": 0, 2024-09-21T03:56:50.343 INFO:teuthology.orchestra.run.smithi161.stdout: "evtrm": 0, 2024-09-21T03:56:50.343 INFO:teuthology.orchestra.run.smithi161.stdout: "ev": 3577, 2024-09-21T03:56:50.343 INFO:teuthology.orchestra.run.smithi161.stdout: "evexg": 0, 2024-09-21T03:56:50.343 INFO:teuthology.orchestra.run.smithi161.stdout: "evexd": 0, 2024-09-21T03:56:50.343 INFO:teuthology.orchestra.run.smithi161.stdout: "segadd": 0, 2024-09-21T03:56:50.343 INFO:teuthology.orchestra.run.smithi161.stdout: "segex": 0, 2024-09-21T03:56:50.343 INFO:teuthology.orchestra.run.smithi161.stdout: "segtrm": 0, 2024-09-21T03:56:50.343 INFO:teuthology.orchestra.run.smithi161.stdout: "seg": 6, 2024-09-21T03:56:50.343 INFO:teuthology.orchestra.run.smithi161.stdout: "segmjr": 2, 2024-09-21T03:56:50.343 INFO:teuthology.orchestra.run.smithi161.stdout: "segexg": 0, 2024-09-21T03:56:50.343 INFO:teuthology.orchestra.run.smithi161.stdout: "segexd": 0, 2024-09-21T03:56:50.344 INFO:teuthology.orchestra.run.smithi161.stdout: "expos": 4194304, 2024-09-21T03:56:50.344 INFO:teuthology.orchestra.run.smithi161.stdout: "wrpos": 13712721, 2024-09-21T03:56:50.344 INFO:teuthology.orchestra.run.smithi161.stdout: "rdpos": 13712609, 2024-09-21T03:56:50.344 INFO:teuthology.orchestra.run.smithi161.stdout: "jlat": { 2024-09-21T03:56:50.344 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-09-21T03:56:50.344 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-09-21T03:56:50.344 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-09-21T03:56:50.344 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T03:56:50.344 INFO:teuthology.orchestra.run.smithi161.stdout: "replayed": 3577 2024-09-21T03:56:50.344 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T03:56:50.344 INFO:teuthology.orchestra.run.smithi161.stdout: "mds_mem": { 2024-09-21T03:56:50.344 INFO:teuthology.orchestra.run.smithi161.stdout: "ino": 17, 2024-09-21T03:56:50.344 INFO:teuthology.orchestra.run.smithi161.stdout: "ino+": 20, 2024-09-21T03:56:50.345 INFO:teuthology.orchestra.run.smithi161.stdout: "ino-": 3, 2024-09-21T03:56:50.345 INFO:teuthology.orchestra.run.smithi161.stdout: "dir": 12, 2024-09-21T03:56:50.345 INFO:teuthology.orchestra.run.smithi161.stdout: "dir+": 12, 2024-09-21T03:56:50.345 INFO:teuthology.orchestra.run.smithi161.stdout: "dir-": 0, 2024-09-21T03:56:50.345 INFO:teuthology.orchestra.run.smithi161.stdout: "dn": 26, 2024-09-21T03:56:50.345 INFO:teuthology.orchestra.run.smithi161.stdout: "dn+": 26, 2024-09-21T03:56:50.345 INFO:teuthology.orchestra.run.smithi161.stdout: "dn-": 0, 2024-09-21T03:56:50.345 INFO:teuthology.orchestra.run.smithi161.stdout: "cap": 0, 2024-09-21T03:56:50.345 INFO:teuthology.orchestra.run.smithi161.stdout: "cap+": 0, 2024-09-21T03:56:50.345 INFO:teuthology.orchestra.run.smithi161.stdout: "cap-": 0, 2024-09-21T03:56:50.345 INFO:teuthology.orchestra.run.smithi161.stdout: "rss": 120704, 2024-09-21T03:56:50.345 INFO:teuthology.orchestra.run.smithi161.stdout: "heap": 318947328 2024-09-21T03:56:50.346 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T03:56:50.346 INFO:teuthology.orchestra.run.smithi161.stdout: "mds_server": { 2024-09-21T03:56:50.346 INFO:teuthology.orchestra.run.smithi161.stdout: "dispatch_client_request": 0, 2024-09-21T03:56:50.346 INFO:teuthology.orchestra.run.smithi161.stdout: "dispatch_server_request": 0, 2024-09-21T03:56:50.346 INFO:teuthology.orchestra.run.smithi161.stdout: "handle_client_request": 0, 2024-09-21T03:56:50.346 INFO:teuthology.orchestra.run.smithi161.stdout: "handle_client_session": 0, 2024-09-21T03:56:50.346 INFO:teuthology.orchestra.run.smithi161.stdout: "handle_peer_request": 0, 2024-09-21T03:56:50.346 INFO:teuthology.orchestra.run.smithi161.stdout: "req_create_latency": { 2024-09-21T03:56:50.346 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-09-21T03:56:50.346 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-09-21T03:56:50.346 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-09-21T03:56:50.346 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T03:56:50.347 INFO:teuthology.orchestra.run.smithi161.stdout: "req_getattr_latency": { 2024-09-21T03:56:50.347 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-09-21T03:56:50.347 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-09-21T03:56:50.347 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-09-21T03:56:50.347 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T03:56:50.347 INFO:teuthology.orchestra.run.smithi161.stdout: "req_getfilelock_latency": { 2024-09-21T03:56:50.347 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-09-21T03:56:50.347 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-09-21T03:56:50.347 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-09-21T03:56:50.347 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T03:56:50.347 INFO:teuthology.orchestra.run.smithi161.stdout: "req_link_latency": { 2024-09-21T03:56:50.347 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-09-21T03:56:50.347 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-09-21T03:56:50.348 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-09-21T03:56:50.348 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T03:56:50.348 INFO:teuthology.orchestra.run.smithi161.stdout: "req_lookup_latency": { 2024-09-21T03:56:50.348 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-09-21T03:56:50.348 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-09-21T03:56:50.348 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-09-21T03:56:50.348 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T03:56:50.348 INFO:teuthology.orchestra.run.smithi161.stdout: "req_lookuphash_latency": { 2024-09-21T03:56:50.348 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-09-21T03:56:50.348 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-09-21T03:56:50.348 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-09-21T03:56:50.348 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T03:56:50.348 INFO:teuthology.orchestra.run.smithi161.stdout: "req_lookupino_latency": { 2024-09-21T03:56:50.349 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-09-21T03:56:50.349 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-09-21T03:56:50.349 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-09-21T03:56:50.349 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T03:56:50.349 INFO:teuthology.orchestra.run.smithi161.stdout: "req_lookupname_latency": { 2024-09-21T03:56:50.349 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-09-21T03:56:50.349 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-09-21T03:56:50.349 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-09-21T03:56:50.349 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T03:56:50.349 INFO:teuthology.orchestra.run.smithi161.stdout: "req_lookupparent_latency": { 2024-09-21T03:56:50.349 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-09-21T03:56:50.349 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-09-21T03:56:50.349 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-09-21T03:56:50.350 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T03:56:50.350 INFO:teuthology.orchestra.run.smithi161.stdout: "req_lookupsnap_latency": { 2024-09-21T03:56:50.350 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-09-21T03:56:50.350 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-09-21T03:56:50.350 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-09-21T03:56:50.350 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T03:56:50.350 INFO:teuthology.orchestra.run.smithi161.stdout: "req_lssnap_latency": { 2024-09-21T03:56:50.350 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-09-21T03:56:50.351 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-09-21T03:56:50.351 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-09-21T03:56:50.351 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T03:56:50.351 INFO:teuthology.orchestra.run.smithi161.stdout: "req_mkdir_latency": { 2024-09-21T03:56:50.351 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-09-21T03:56:50.351 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-09-21T03:56:50.351 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-09-21T03:56:50.351 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T03:56:50.352 INFO:teuthology.orchestra.run.smithi161.stdout: "req_mknod_latency": { 2024-09-21T03:56:50.352 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-09-21T03:56:50.352 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-09-21T03:56:50.352 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-09-21T03:56:50.352 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T03:56:50.352 INFO:teuthology.orchestra.run.smithi161.stdout: "req_mksnap_latency": { 2024-09-21T03:56:50.352 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-09-21T03:56:50.352 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-09-21T03:56:50.352 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-09-21T03:56:50.352 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T03:56:50.352 INFO:teuthology.orchestra.run.smithi161.stdout: "req_open_latency": { 2024-09-21T03:56:50.352 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-09-21T03:56:50.352 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-09-21T03:56:50.353 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-09-21T03:56:50.353 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T03:56:50.353 INFO:teuthology.orchestra.run.smithi161.stdout: "req_readdir_latency": { 2024-09-21T03:56:50.353 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-09-21T03:56:50.353 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-09-21T03:56:50.353 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-09-21T03:56:50.353 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T03:56:50.353 INFO:teuthology.orchestra.run.smithi161.stdout: "req_rename_latency": { 2024-09-21T03:56:50.353 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-09-21T03:56:50.353 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-09-21T03:56:50.354 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-09-21T03:56:50.354 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T03:56:50.354 INFO:teuthology.orchestra.run.smithi161.stdout: "req_renamesnap_latency": { 2024-09-21T03:56:50.354 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-09-21T03:56:50.354 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-09-21T03:56:50.354 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-09-21T03:56:50.354 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T03:56:50.354 INFO:teuthology.orchestra.run.smithi161.stdout: "req_snapdiff_latency": { 2024-09-21T03:56:50.354 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-09-21T03:56:50.354 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-09-21T03:56:50.354 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-09-21T03:56:50.355 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T03:56:50.355 INFO:teuthology.orchestra.run.smithi161.stdout: "req_rmdir_latency": { 2024-09-21T03:56:50.355 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-09-21T03:56:50.355 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-09-21T03:56:50.355 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-09-21T03:56:50.355 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T03:56:50.355 INFO:teuthology.orchestra.run.smithi161.stdout: "req_rmsnap_latency": { 2024-09-21T03:56:50.355 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-09-21T03:56:50.355 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-09-21T03:56:50.355 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-09-21T03:56:50.355 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T03:56:50.355 INFO:teuthology.orchestra.run.smithi161.stdout: "req_rmxattr_latency": { 2024-09-21T03:56:50.355 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-09-21T03:56:50.356 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-09-21T03:56:50.356 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-09-21T03:56:50.356 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T03:56:50.356 INFO:teuthology.orchestra.run.smithi161.stdout: "req_setattr_latency": { 2024-09-21T03:56:50.356 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-09-21T03:56:50.356 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-09-21T03:56:50.356 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-09-21T03:56:50.356 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T03:56:50.356 INFO:teuthology.orchestra.run.smithi161.stdout: "req_setdirlayout_latency": { 2024-09-21T03:56:50.356 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-09-21T03:56:50.356 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-09-21T03:56:50.356 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-09-21T03:56:50.357 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T03:56:50.357 INFO:teuthology.orchestra.run.smithi161.stdout: "req_setfilelock_latency": { 2024-09-21T03:56:50.357 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-09-21T03:56:50.357 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-09-21T03:56:50.357 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-09-21T03:56:50.357 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T03:56:50.357 INFO:teuthology.orchestra.run.smithi161.stdout: "req_setlayout_latency": { 2024-09-21T03:56:50.357 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-09-21T03:56:50.357 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-09-21T03:56:50.357 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-09-21T03:56:50.357 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T03:56:50.357 INFO:teuthology.orchestra.run.smithi161.stdout: "req_setxattr_latency": { 2024-09-21T03:56:50.358 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-09-21T03:56:50.358 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-09-21T03:56:50.358 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-09-21T03:56:50.358 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T03:56:50.358 INFO:teuthology.orchestra.run.smithi161.stdout: "req_symlink_latency": { 2024-09-21T03:56:50.358 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-09-21T03:56:50.358 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-09-21T03:56:50.358 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-09-21T03:56:50.358 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T03:56:50.358 INFO:teuthology.orchestra.run.smithi161.stdout: "req_unlink_latency": { 2024-09-21T03:56:50.358 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-09-21T03:56:50.358 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-09-21T03:56:50.359 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-09-21T03:56:50.359 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T03:56:50.359 INFO:teuthology.orchestra.run.smithi161.stdout: "cap_revoke_eviction": 0, 2024-09-21T03:56:50.359 INFO:teuthology.orchestra.run.smithi161.stdout: "cap_acquisition_throttle": 0, 2024-09-21T03:56:50.359 INFO:teuthology.orchestra.run.smithi161.stdout: "req_getvxattr_latency": { 2024-09-21T03:56:50.359 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-09-21T03:56:50.359 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-09-21T03:56:50.359 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-09-21T03:56:50.359 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T03:56:50.359 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T03:56:50.359 INFO:teuthology.orchestra.run.smithi161.stdout: "mds_sessions": { 2024-09-21T03:56:50.359 INFO:teuthology.orchestra.run.smithi161.stdout: "session_count": 2, 2024-09-21T03:56:50.360 INFO:teuthology.orchestra.run.smithi161.stdout: "session_add": 4, 2024-09-21T03:56:50.360 INFO:teuthology.orchestra.run.smithi161.stdout: "session_remove": 2, 2024-09-21T03:56:50.360 INFO:teuthology.orchestra.run.smithi161.stdout: "sessions_open": 3, 2024-09-21T03:56:50.360 INFO:teuthology.orchestra.run.smithi161.stdout: "sessions_stale": 0, 2024-09-21T03:56:50.360 INFO:teuthology.orchestra.run.smithi161.stdout: "total_load": 0, 2024-09-21T03:56:50.360 INFO:teuthology.orchestra.run.smithi161.stdout: "average_load": 0, 2024-09-21T03:56:50.360 INFO:teuthology.orchestra.run.smithi161.stdout: "avg_session_uptime": 2869, 2024-09-21T03:56:50.360 INFO:teuthology.orchestra.run.smithi161.stdout: "mdthresh_evicted": 0 2024-09-21T03:56:50.360 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T03:56:50.360 INFO:teuthology.orchestra.run.smithi161.stdout: "mempool": { 2024-09-21T03:56:50.360 INFO:teuthology.orchestra.run.smithi161.stdout: "bloom_filter_bytes": 0, 2024-09-21T03:56:50.360 INFO:teuthology.orchestra.run.smithi161.stdout: "bloom_filter_items": 0, 2024-09-21T03:56:50.361 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_alloc_bytes": 0, 2024-09-21T03:56:50.361 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_alloc_items": 0, 2024-09-21T03:56:50.361 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_cache_data_bytes": 0, 2024-09-21T03:56:50.361 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_cache_data_items": 0, 2024-09-21T03:56:50.361 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-21T03:56:50.361 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_cache_onode_items": 0, 2024-09-21T03:56:50.361 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-21T03:56:50.361 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_cache_meta_items": 0, 2024-09-21T03:56:50.361 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_cache_other_bytes": 0, 2024-09-21T03:56:50.361 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_cache_other_items": 0, 2024-09-21T03:56:50.361 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-21T03:56:50.361 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_cache_buffer_items": 0, 2024-09-21T03:56:50.361 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_extent_bytes": 0, 2024-09-21T03:56:50.362 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_extent_items": 0, 2024-09-21T03:56:50.362 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_blob_bytes": 0, 2024-09-21T03:56:50.362 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_blob_items": 0, 2024-09-21T03:56:50.362 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-21T03:56:50.362 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_shared_blob_items": 0, 2024-09-21T03:56:50.362 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-21T03:56:50.362 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_inline_bl_items": 0, 2024-09-21T03:56:50.362 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_fsck_bytes": 0, 2024-09-21T03:56:50.362 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_fsck_items": 0, 2024-09-21T03:56:50.362 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_txc_bytes": 0, 2024-09-21T03:56:50.362 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_txc_items": 0, 2024-09-21T03:56:50.362 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-21T03:56:50.363 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_writing_deferred_items": 0, 2024-09-21T03:56:50.363 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_writing_bytes": 0, 2024-09-21T03:56:50.363 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_writing_items": 0, 2024-09-21T03:56:50.363 INFO:teuthology.orchestra.run.smithi161.stdout: "bluefs_bytes": 0, 2024-09-21T03:56:50.363 INFO:teuthology.orchestra.run.smithi161.stdout: "bluefs_items": 0, 2024-09-21T03:56:50.363 INFO:teuthology.orchestra.run.smithi161.stdout: "bluefs_file_reader_bytes": 0, 2024-09-21T03:56:50.363 INFO:teuthology.orchestra.run.smithi161.stdout: "bluefs_file_reader_items": 0, 2024-09-21T03:56:50.363 INFO:teuthology.orchestra.run.smithi161.stdout: "bluefs_file_writer_bytes": 0, 2024-09-21T03:56:50.363 INFO:teuthology.orchestra.run.smithi161.stdout: "bluefs_file_writer_items": 0, 2024-09-21T03:56:50.363 INFO:teuthology.orchestra.run.smithi161.stdout: "buffer_anon_bytes": 189568, 2024-09-21T03:56:50.363 INFO:teuthology.orchestra.run.smithi161.stdout: "buffer_anon_items": 55, 2024-09-21T03:56:50.363 INFO:teuthology.orchestra.run.smithi161.stdout: "buffer_meta_bytes": 0, 2024-09-21T03:56:50.364 INFO:teuthology.orchestra.run.smithi161.stdout: "buffer_meta_items": 0, 2024-09-21T03:56:50.364 INFO:teuthology.orchestra.run.smithi161.stdout: "osd_bytes": 0, 2024-09-21T03:56:50.364 INFO:teuthology.orchestra.run.smithi161.stdout: "osd_items": 0, 2024-09-21T03:56:50.364 INFO:teuthology.orchestra.run.smithi161.stdout: "osd_mapbl_bytes": 0, 2024-09-21T03:56:50.364 INFO:teuthology.orchestra.run.smithi161.stdout: "osd_mapbl_items": 0, 2024-09-21T03:56:50.364 INFO:teuthology.orchestra.run.smithi161.stdout: "osd_pglog_bytes": 0, 2024-09-21T03:56:50.364 INFO:teuthology.orchestra.run.smithi161.stdout: "osd_pglog_items": 0, 2024-09-21T03:56:50.364 INFO:teuthology.orchestra.run.smithi161.stdout: "osdmap_bytes": 8256, 2024-09-21T03:56:50.364 INFO:teuthology.orchestra.run.smithi161.stdout: "osdmap_items": 182, 2024-09-21T03:56:50.364 INFO:teuthology.orchestra.run.smithi161.stdout: "osdmap_mapping_bytes": 0, 2024-09-21T03:56:50.364 INFO:teuthology.orchestra.run.smithi161.stdout: "osdmap_mapping_items": 0, 2024-09-21T03:56:50.364 INFO:teuthology.orchestra.run.smithi161.stdout: "pgmap_bytes": 0, 2024-09-21T03:56:50.365 INFO:teuthology.orchestra.run.smithi161.stdout: "pgmap_items": 0, 2024-09-21T03:56:50.365 INFO:teuthology.orchestra.run.smithi161.stdout: "mds_co_bytes": 85945, 2024-09-21T03:56:50.365 INFO:teuthology.orchestra.run.smithi161.stdout: "mds_co_items": 688, 2024-09-21T03:56:50.365 INFO:teuthology.orchestra.run.smithi161.stdout: "unittest_1_bytes": 0, 2024-09-21T03:56:50.365 INFO:teuthology.orchestra.run.smithi161.stdout: "unittest_1_items": 0, 2024-09-21T03:56:50.365 INFO:teuthology.orchestra.run.smithi161.stdout: "unittest_2_bytes": 0, 2024-09-21T03:56:50.365 INFO:teuthology.orchestra.run.smithi161.stdout: "unittest_2_items": 0 2024-09-21T03:56:50.365 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T03:56:50.365 INFO:teuthology.orchestra.run.smithi161.stdout: "objecter": { 2024-09-21T03:56:50.365 INFO:teuthology.orchestra.run.smithi161.stdout: "op_active": 0, 2024-09-21T03:56:50.365 INFO:teuthology.orchestra.run.smithi161.stdout: "op_laggy": 0, 2024-09-21T03:56:50.365 INFO:teuthology.orchestra.run.smithi161.stdout: "op_send": 2308, 2024-09-21T03:56:50.365 INFO:teuthology.orchestra.run.smithi161.stdout: "op_send_bytes": 26, 2024-09-21T03:56:50.366 INFO:teuthology.orchestra.run.smithi161.stdout: "op_resend": 4, 2024-09-21T03:56:50.366 INFO:teuthology.orchestra.run.smithi161.stdout: "op_reply": 2308, 2024-09-21T03:56:50.366 INFO:teuthology.orchestra.run.smithi161.stdout: "op_latency": { 2024-09-21T03:56:50.366 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 2308, 2024-09-21T03:56:50.366 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 4.831897136, 2024-09-21T03:56:50.366 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.002093542 2024-09-21T03:56:50.366 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T03:56:50.366 INFO:teuthology.orchestra.run.smithi161.stdout: "op_inflight": 0, 2024-09-21T03:56:50.366 INFO:teuthology.orchestra.run.smithi161.stdout: "oplen_avg": { 2024-09-21T03:56:50.366 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 2308, 2024-09-21T03:56:50.366 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 2309 2024-09-21T03:56:50.366 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T03:56:50.367 INFO:teuthology.orchestra.run.smithi161.stdout: "op": 2308, 2024-09-21T03:56:50.367 INFO:teuthology.orchestra.run.smithi161.stdout: "op_r": 2308, 2024-09-21T03:56:50.367 INFO:teuthology.orchestra.run.smithi161.stdout: "op_w": 0, 2024-09-21T03:56:50.367 INFO:teuthology.orchestra.run.smithi161.stdout: "op_rmw": 0, 2024-09-21T03:56:50.367 INFO:teuthology.orchestra.run.smithi161.stdout: "op_pg": 0, 2024-09-21T03:56:50.367 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_stat": 1473, 2024-09-21T03:56:50.367 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_create": 0, 2024-09-21T03:56:50.367 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_read": 832, 2024-09-21T03:56:50.367 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_write": 0, 2024-09-21T03:56:50.367 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_writefull": 0, 2024-09-21T03:56:50.367 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_writesame": 0, 2024-09-21T03:56:50.367 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_append": 0, 2024-09-21T03:56:50.368 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_zero": 0, 2024-09-21T03:56:50.368 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_truncate": 0, 2024-09-21T03:56:50.368 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_delete": 0, 2024-09-21T03:56:50.368 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_mapext": 0, 2024-09-21T03:56:50.368 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_sparse_read": 0, 2024-09-21T03:56:50.368 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_clonerange": 0, 2024-09-21T03:56:50.368 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_getxattr": 2, 2024-09-21T03:56:50.368 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_setxattr": 0, 2024-09-21T03:56:50.368 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_cmpxattr": 0, 2024-09-21T03:56:50.368 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_rmxattr": 0, 2024-09-21T03:56:50.368 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_resetxattrs": 0, 2024-09-21T03:56:50.368 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_call": 0, 2024-09-21T03:56:50.368 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_watch": 0, 2024-09-21T03:56:50.369 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_notify": 0, 2024-09-21T03:56:50.369 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_src_cmpxattr": 0, 2024-09-21T03:56:50.369 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_pgls": 0, 2024-09-21T03:56:50.369 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_pgls_filter": 0, 2024-09-21T03:56:50.369 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_other": 0, 2024-09-21T03:56:50.369 INFO:teuthology.orchestra.run.smithi161.stdout: "linger_active": 0, 2024-09-21T03:56:50.369 INFO:teuthology.orchestra.run.smithi161.stdout: "linger_send": 0, 2024-09-21T03:56:50.369 INFO:teuthology.orchestra.run.smithi161.stdout: "linger_resend": 0, 2024-09-21T03:56:50.369 INFO:teuthology.orchestra.run.smithi161.stdout: "linger_ping": 0, 2024-09-21T03:56:50.369 INFO:teuthology.orchestra.run.smithi161.stdout: "poolop_active": 0, 2024-09-21T03:56:50.369 INFO:teuthology.orchestra.run.smithi161.stdout: "poolop_send": 0, 2024-09-21T03:56:50.369 INFO:teuthology.orchestra.run.smithi161.stdout: "poolop_resend": 0, 2024-09-21T03:56:50.369 INFO:teuthology.orchestra.run.smithi161.stdout: "poolstat_active": 0, 2024-09-21T03:56:50.370 INFO:teuthology.orchestra.run.smithi161.stdout: "poolstat_send": 0, 2024-09-21T03:56:50.370 INFO:teuthology.orchestra.run.smithi161.stdout: "poolstat_resend": 0, 2024-09-21T03:56:50.370 INFO:teuthology.orchestra.run.smithi161.stdout: "statfs_active": 0, 2024-09-21T03:56:50.370 INFO:teuthology.orchestra.run.smithi161.stdout: "statfs_send": 0, 2024-09-21T03:56:50.370 INFO:teuthology.orchestra.run.smithi161.stdout: "statfs_resend": 0, 2024-09-21T03:56:50.370 INFO:teuthology.orchestra.run.smithi161.stdout: "command_active": 0, 2024-09-21T03:56:50.370 INFO:teuthology.orchestra.run.smithi161.stdout: "command_send": 0, 2024-09-21T03:56:50.370 INFO:teuthology.orchestra.run.smithi161.stdout: "command_resend": 0, 2024-09-21T03:56:50.370 INFO:teuthology.orchestra.run.smithi161.stdout: "map_epoch": 87, 2024-09-21T03:56:50.370 INFO:teuthology.orchestra.run.smithi161.stdout: "map_full": 0, 2024-09-21T03:56:50.370 INFO:teuthology.orchestra.run.smithi161.stdout: "map_inc": 10, 2024-09-21T03:56:50.370 INFO:teuthology.orchestra.run.smithi161.stdout: "osd_sessions": 5, 2024-09-21T03:56:50.371 INFO:teuthology.orchestra.run.smithi161.stdout: "osd_session_open": 5, 2024-09-21T03:56:50.371 INFO:teuthology.orchestra.run.smithi161.stdout: "osd_session_close": 0, 2024-09-21T03:56:50.371 INFO:teuthology.orchestra.run.smithi161.stdout: "osd_laggy": 0, 2024-09-21T03:56:50.371 INFO:teuthology.orchestra.run.smithi161.stdout: "omap_wr": 0, 2024-09-21T03:56:50.371 INFO:teuthology.orchestra.run.smithi161.stdout: "omap_rd": 2, 2024-09-21T03:56:50.371 INFO:teuthology.orchestra.run.smithi161.stdout: "omap_del": 0 2024-09-21T03:56:50.371 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T03:56:50.371 INFO:teuthology.orchestra.run.smithi161.stdout: "oft": { 2024-09-21T03:56:50.371 INFO:teuthology.orchestra.run.smithi161.stdout: "omap_total_objs": 0, 2024-09-21T03:56:50.371 INFO:teuthology.orchestra.run.smithi161.stdout: "omap_total_kv_pairs": 0, 2024-09-21T03:56:50.372 INFO:teuthology.orchestra.run.smithi161.stdout: "omap_total_updates": 0, 2024-09-21T03:56:50.372 INFO:teuthology.orchestra.run.smithi161.stdout: "omap_total_removes": 0 2024-09-21T03:56:50.372 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T03:56:50.372 INFO:teuthology.orchestra.run.smithi161.stdout: "purge_queue": { 2024-09-21T03:56:50.372 INFO:teuthology.orchestra.run.smithi161.stdout: "pq_executing_ops": 0, 2024-09-21T03:56:50.372 INFO:teuthology.orchestra.run.smithi161.stdout: "pq_executing_ops_high_water": 0, 2024-09-21T03:56:50.372 INFO:teuthology.orchestra.run.smithi161.stdout: "pq_executing": 0, 2024-09-21T03:56:50.372 INFO:teuthology.orchestra.run.smithi161.stdout: "pq_executing_high_water": 0, 2024-09-21T03:56:50.372 INFO:teuthology.orchestra.run.smithi161.stdout: "pq_executed_ops": 0, 2024-09-21T03:56:50.372 INFO:teuthology.orchestra.run.smithi161.stdout: "pq_executed": 0, 2024-09-21T03:56:50.372 INFO:teuthology.orchestra.run.smithi161.stdout: "pq_item_in_journal": 0 2024-09-21T03:56:50.372 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T03:56:50.372 INFO:teuthology.orchestra.run.smithi161.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-21T03:56:50.373 INFO:teuthology.orchestra.run.smithi161.stdout: "val": 0, 2024-09-21T03:56:50.373 INFO:teuthology.orchestra.run.smithi161.stdout: "max": 104857600, 2024-09-21T03:56:50.373 INFO:teuthology.orchestra.run.smithi161.stdout: "get_started": 0, 2024-09-21T03:56:50.373 INFO:teuthology.orchestra.run.smithi161.stdout: "get": 2547, 2024-09-21T03:56:50.373 INFO:teuthology.orchestra.run.smithi161.stdout: "get_sum": 11124530, 2024-09-21T03:56:50.373 INFO:teuthology.orchestra.run.smithi161.stdout: "get_or_fail_fail": 0, 2024-09-21T03:56:50.373 INFO:teuthology.orchestra.run.smithi161.stdout: "get_or_fail_success": 2547, 2024-09-21T03:56:50.373 INFO:teuthology.orchestra.run.smithi161.stdout: "take": 0, 2024-09-21T03:56:50.373 INFO:teuthology.orchestra.run.smithi161.stdout: "take_sum": 0, 2024-09-21T03:56:50.373 INFO:teuthology.orchestra.run.smithi161.stdout: "put": 2547, 2024-09-21T03:56:50.373 INFO:teuthology.orchestra.run.smithi161.stdout: "put_sum": 11124530, 2024-09-21T03:56:50.373 INFO:teuthology.orchestra.run.smithi161.stdout: "wait": { 2024-09-21T03:56:50.374 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-09-21T03:56:50.374 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-09-21T03:56:50.374 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-09-21T03:56:50.374 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T03:56:50.374 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T03:56:50.374 INFO:teuthology.orchestra.run.smithi161.stdout: "throttle-objecter_bytes": { 2024-09-21T03:56:50.374 INFO:teuthology.orchestra.run.smithi161.stdout: "val": 0, 2024-09-21T03:56:50.374 INFO:teuthology.orchestra.run.smithi161.stdout: "max": 104857600, 2024-09-21T03:56:50.374 INFO:teuthology.orchestra.run.smithi161.stdout: "get_started": 0, 2024-09-21T03:56:50.374 INFO:teuthology.orchestra.run.smithi161.stdout: "get": 0, 2024-09-21T03:56:50.374 INFO:teuthology.orchestra.run.smithi161.stdout: "get_sum": 0, 2024-09-21T03:56:50.374 INFO:teuthology.orchestra.run.smithi161.stdout: "get_or_fail_fail": 0, 2024-09-21T03:56:50.375 INFO:teuthology.orchestra.run.smithi161.stdout: "get_or_fail_success": 0, 2024-09-21T03:56:50.375 INFO:teuthology.orchestra.run.smithi161.stdout: "take": 2308, 2024-09-21T03:56:50.375 INFO:teuthology.orchestra.run.smithi161.stdout: "take_sum": 9518427, 2024-09-21T03:56:50.375 INFO:teuthology.orchestra.run.smithi161.stdout: "put": 2308, 2024-09-21T03:56:50.375 INFO:teuthology.orchestra.run.smithi161.stdout: "put_sum": 9518427, 2024-09-21T03:56:50.375 INFO:teuthology.orchestra.run.smithi161.stdout: "wait": { 2024-09-21T03:56:50.375 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-09-21T03:56:50.375 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-09-21T03:56:50.375 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-09-21T03:56:50.375 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T03:56:50.375 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T03:56:50.375 INFO:teuthology.orchestra.run.smithi161.stdout: "throttle-objecter_ops": { 2024-09-21T03:56:50.375 INFO:teuthology.orchestra.run.smithi161.stdout: "val": 0, 2024-09-21T03:56:50.376 INFO:teuthology.orchestra.run.smithi161.stdout: "max": 1024, 2024-09-21T03:56:50.376 INFO:teuthology.orchestra.run.smithi161.stdout: "get_started": 0, 2024-09-21T03:56:50.376 INFO:teuthology.orchestra.run.smithi161.stdout: "get": 0, 2024-09-21T03:56:50.376 INFO:teuthology.orchestra.run.smithi161.stdout: "get_sum": 0, 2024-09-21T03:56:50.376 INFO:teuthology.orchestra.run.smithi161.stdout: "get_or_fail_fail": 0, 2024-09-21T03:56:50.376 INFO:teuthology.orchestra.run.smithi161.stdout: "get_or_fail_success": 0, 2024-09-21T03:56:50.376 INFO:teuthology.orchestra.run.smithi161.stdout: "take": 2308, 2024-09-21T03:56:50.376 INFO:teuthology.orchestra.run.smithi161.stdout: "take_sum": 2308, 2024-09-21T03:56:50.376 INFO:teuthology.orchestra.run.smithi161.stdout: "put": 2308, 2024-09-21T03:56:50.376 INFO:teuthology.orchestra.run.smithi161.stdout: "put_sum": 2308, 2024-09-21T03:56:50.376 INFO:teuthology.orchestra.run.smithi161.stdout: "wait": { 2024-09-21T03:56:50.376 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-09-21T03:56:50.377 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-09-21T03:56:50.377 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-09-21T03:56:50.377 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T03:56:50.377 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T03:56:50.377 INFO:teuthology.orchestra.run.smithi161.stdout: "throttle-write_buf_throttle": { 2024-09-21T03:56:50.377 INFO:teuthology.orchestra.run.smithi161.stdout: "val": 0, 2024-09-21T03:56:50.377 INFO:teuthology.orchestra.run.smithi161.stdout: "max": 3758096384, 2024-09-21T03:56:50.377 INFO:teuthology.orchestra.run.smithi161.stdout: "get_started": 0, 2024-09-21T03:56:50.377 INFO:teuthology.orchestra.run.smithi161.stdout: "get": 0, 2024-09-21T03:56:50.377 INFO:teuthology.orchestra.run.smithi161.stdout: "get_sum": 0, 2024-09-21T03:56:50.377 INFO:teuthology.orchestra.run.smithi161.stdout: "get_or_fail_fail": 0, 2024-09-21T03:56:50.377 INFO:teuthology.orchestra.run.smithi161.stdout: "get_or_fail_success": 0, 2024-09-21T03:56:50.377 INFO:teuthology.orchestra.run.smithi161.stdout: "take": 0, 2024-09-21T03:56:50.378 INFO:teuthology.orchestra.run.smithi161.stdout: "take_sum": 0, 2024-09-21T03:56:50.378 INFO:teuthology.orchestra.run.smithi161.stdout: "put": 0, 2024-09-21T03:56:50.378 INFO:teuthology.orchestra.run.smithi161.stdout: "put_sum": 0, 2024-09-21T03:56:50.378 INFO:teuthology.orchestra.run.smithi161.stdout: "wait": { 2024-09-21T03:56:50.378 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-09-21T03:56:50.378 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-09-21T03:56:50.378 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-09-21T03:56:50.378 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T03:56:50.378 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T03:56:50.378 INFO:teuthology.orchestra.run.smithi161.stdout: "throttle-write_buf_throttle-0x556b1ff655f0": { 2024-09-21T03:56:50.378 INFO:teuthology.orchestra.run.smithi161.stdout: "val": 0, 2024-09-21T03:56:50.378 INFO:teuthology.orchestra.run.smithi161.stdout: "max": 3758096384, 2024-09-21T03:56:50.379 INFO:teuthology.orchestra.run.smithi161.stdout: "get_started": 0, 2024-09-21T03:56:50.379 INFO:teuthology.orchestra.run.smithi161.stdout: "get": 0, 2024-09-21T03:56:50.379 INFO:teuthology.orchestra.run.smithi161.stdout: "get_sum": 0, 2024-09-21T03:56:50.379 INFO:teuthology.orchestra.run.smithi161.stdout: "get_or_fail_fail": 0, 2024-09-21T03:56:50.379 INFO:teuthology.orchestra.run.smithi161.stdout: "get_or_fail_success": 0, 2024-09-21T03:56:50.379 INFO:teuthology.orchestra.run.smithi161.stdout: "take": 0, 2024-09-21T03:56:50.379 INFO:teuthology.orchestra.run.smithi161.stdout: "take_sum": 0, 2024-09-21T03:56:50.379 INFO:teuthology.orchestra.run.smithi161.stdout: "put": 0, 2024-09-21T03:56:50.379 INFO:teuthology.orchestra.run.smithi161.stdout: "put_sum": 0, 2024-09-21T03:56:50.379 INFO:teuthology.orchestra.run.smithi161.stdout: "wait": { 2024-09-21T03:56:50.379 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-09-21T03:56:50.379 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-09-21T03:56:50.379 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-09-21T03:56:50.380 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T03:56:50.380 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T03:56:50.380 INFO:teuthology.orchestra.run.smithi161.stdout: "trackedop": { 2024-09-21T03:56:50.380 INFO:teuthology.orchestra.run.smithi161.stdout: "slow_ops_count": 0 2024-09-21T03:56:50.380 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T03:56:50.380 INFO:teuthology.orchestra.run.smithi161.stdout:} 2024-09-21T03:56:51.022 INFO:tasks.check_counter:Daemon mds.f mds.exported=0 2024-09-21T03:56:51.022 INFO:tasks.check_counter:Daemon mds.f mds.imported=0 2024-09-21T03:56:51.023 INFO:tasks.check_counter:Daemon mds.f mds_cache.dir_update=0 2024-09-21T03:56:51.023 INFO:tasks.check_counter:Daemon mds.f mds_cache.dir_update_receipt=0 2024-09-21T03:56:51.023 INFO:tasks.check_counter:Daemon mds.f mds.root_rsnaps=6 2024-09-21T03:56:51.023 INFO:tasks.check_counter:Daemon mds.f mds_server.req_mksnap_latency.avgcount=0 2024-09-21T03:56:51.023 INFO:tasks.check_counter:Daemon mds.f mds_server.req_rmsnap_latency.avgcount=0 2024-09-21T03:56:51.023 DEBUG:tasks.check_counter:Getting stats from i 2024-09-21T03:56:51.023 DEBUG:teuthology.orchestra.run.smithi161:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 51116d5c-77ca-11ef-baf5-efdc52797490 -- ceph daemon mds.i perf dump 2024-09-21T03:56:51.271 INFO:teuthology.orchestra.run.smithi161.stderr:Inferring config /var/lib/ceph/51116d5c-77ca-11ef-baf5-efdc52797490/mon.c/config 2024-09-21T03:56:51.680 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:56:51 smithi161 ceph-mon[34533]: pgmap v657: 97 pgs: 97 active+clean; 10 MiB data, 525 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-09-21T03:56:51.725 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:56:51 smithi150 ceph-mon[34289]: pgmap v657: 97 pgs: 97 active+clean; 10 MiB data, 525 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-09-21T03:56:51.802 INFO:teuthology.orchestra.run.smithi161.stdout:{ 2024-09-21T03:56:51.802 INFO:teuthology.orchestra.run.smithi161.stdout: "AsyncMessenger::Worker-0": { 2024-09-21T03:56:51.802 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_recv_messages": 713, 2024-09-21T03:56:51.802 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_messages": 785, 2024-09-21T03:56:51.802 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_recv_bytes": 1351866, 2024-09-21T03:56:51.802 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_bytes": 253190, 2024-09-21T03:56:51.802 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_created_connections": 7, 2024-09-21T03:56:51.803 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_active_connections": 3, 2024-09-21T03:56:51.803 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_total_time": 0.119457092, 2024-09-21T03:56:51.803 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_send_time": 0.052887733, 2024-09-21T03:56:51.803 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_recv_time": 0.066924192, 2024-09-21T03:56:51.803 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_fast_dispatch_time": 0.005401952, 2024-09-21T03:56:51.803 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T03:56:51.803 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 784, 2024-09-21T03:56:51.803 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.057426701, 2024-09-21T03:56:51.803 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000073248 2024-09-21T03:56:51.803 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T03:56:51.803 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_handle_ack_lat": { 2024-09-21T03:56:51.803 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 944, 2024-09-21T03:56:51.803 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000322767, 2024-09-21T03:56:51.803 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000341 2024-09-21T03:56:51.804 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T03:56:51.804 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_recv_encrypted_bytes": 1215248, 2024-09-21T03:56:51.804 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_encrypted_bytes": 135136 2024-09-21T03:56:51.804 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T03:56:51.804 INFO:teuthology.orchestra.run.smithi161.stdout: "AsyncMessenger::Worker-1": { 2024-09-21T03:56:51.804 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_recv_messages": 63, 2024-09-21T03:56:51.804 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_messages": 219, 2024-09-21T03:56:51.804 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_recv_bytes": 13996, 2024-09-21T03:56:51.804 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_bytes": 839719, 2024-09-21T03:56:51.804 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_created_connections": 46, 2024-09-21T03:56:51.804 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_active_connections": 7, 2024-09-21T03:56:51.804 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_total_time": 0.033022504, 2024-09-21T03:56:51.804 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_send_time": 0.011142007, 2024-09-21T03:56:51.804 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_recv_time": 0.019250029, 2024-09-21T03:56:51.805 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_fast_dispatch_time": 0.000278267, 2024-09-21T03:56:51.805 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T03:56:51.805 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 218, 2024-09-21T03:56:51.805 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.027885607, 2024-09-21T03:56:51.805 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000127915 2024-09-21T03:56:51.805 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T03:56:51.805 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_handle_ack_lat": { 2024-09-21T03:56:51.805 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 57, 2024-09-21T03:56:51.805 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000007881, 2024-09-21T03:56:51.805 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000138 2024-09-21T03:56:51.805 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T03:56:51.805 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_recv_encrypted_bytes": 5712, 2024-09-21T03:56:51.805 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_encrypted_bytes": 366160 2024-09-21T03:56:51.805 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T03:56:51.805 INFO:teuthology.orchestra.run.smithi161.stdout: "AsyncMessenger::Worker-2": { 2024-09-21T03:56:51.806 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_recv_messages": 21, 2024-09-21T03:56:51.806 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_messages": 23, 2024-09-21T03:56:51.806 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_recv_bytes": 4981, 2024-09-21T03:56:51.806 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_bytes": 86716, 2024-09-21T03:56:51.806 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_created_connections": 12, 2024-09-21T03:56:51.806 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_active_connections": 4, 2024-09-21T03:56:51.806 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_total_time": 0.005766372, 2024-09-21T03:56:51.806 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_send_time": 0.000752896, 2024-09-21T03:56:51.806 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_recv_time": 0.004183305, 2024-09-21T03:56:51.806 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_fast_dispatch_time": 0.000477924, 2024-09-21T03:56:51.806 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T03:56:51.806 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 21, 2024-09-21T03:56:51.806 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.015867530, 2024-09-21T03:56:51.806 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000755596 2024-09-21T03:56:51.807 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T03:56:51.807 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_handle_ack_lat": { 2024-09-21T03:56:51.807 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 6, 2024-09-21T03:56:51.807 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000554, 2024-09-21T03:56:51.807 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000092 2024-09-21T03:56:51.807 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T03:56:51.807 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_recv_encrypted_bytes": 0, 2024-09-21T03:56:51.807 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_encrypted_bytes": 0 2024-09-21T03:56:51.807 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T03:56:51.807 INFO:teuthology.orchestra.run.smithi161.stdout: "cct": { 2024-09-21T03:56:51.807 INFO:teuthology.orchestra.run.smithi161.stdout: "total_workers": 1, 2024-09-21T03:56:51.807 INFO:teuthology.orchestra.run.smithi161.stdout: "unhealthy_workers": 0 2024-09-21T03:56:51.807 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T03:56:51.807 INFO:teuthology.orchestra.run.smithi161.stdout: "finisher-MDSRank": { 2024-09-21T03:56:51.808 INFO:teuthology.orchestra.run.smithi161.stdout: "queue_len": 0, 2024-09-21T03:56:51.808 INFO:teuthology.orchestra.run.smithi161.stdout: "complete_latency": { 2024-09-21T03:56:51.808 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 46, 2024-09-21T03:56:51.808 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.001752965, 2024-09-21T03:56:51.808 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000038107 2024-09-21T03:56:51.808 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T03:56:51.808 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T03:56:51.808 INFO:teuthology.orchestra.run.smithi161.stdout: "finisher-PurgeQueue": { 2024-09-21T03:56:51.808 INFO:teuthology.orchestra.run.smithi161.stdout: "queue_len": 0, 2024-09-21T03:56:51.808 INFO:teuthology.orchestra.run.smithi161.stdout: "complete_latency": { 2024-09-21T03:56:51.808 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 2, 2024-09-21T03:56:51.808 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000027885, 2024-09-21T03:56:51.808 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000013942 2024-09-21T03:56:51.808 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T03:56:51.808 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T03:56:51.809 INFO:teuthology.orchestra.run.smithi161.stdout: "mds": { 2024-09-21T03:56:51.809 INFO:teuthology.orchestra.run.smithi161.stdout: "request": 2, 2024-09-21T03:56:51.809 INFO:teuthology.orchestra.run.smithi161.stdout: "reply": 1, 2024-09-21T03:56:51.809 INFO:teuthology.orchestra.run.smithi161.stdout: "reply_latency": { 2024-09-21T03:56:51.809 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 1, 2024-09-21T03:56:51.809 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000176243, 2024-09-21T03:56:51.809 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000176243 2024-09-21T03:56:51.809 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T03:56:51.809 INFO:teuthology.orchestra.run.smithi161.stdout: "slow_reply": 0, 2024-09-21T03:56:51.809 INFO:teuthology.orchestra.run.smithi161.stdout: "forward": 1, 2024-09-21T03:56:51.809 INFO:teuthology.orchestra.run.smithi161.stdout: "dir_fetch_complete": 0, 2024-09-21T03:56:51.809 INFO:teuthology.orchestra.run.smithi161.stdout: "dir_fetch_keys": 0, 2024-09-21T03:56:51.809 INFO:teuthology.orchestra.run.smithi161.stdout: "dir_commit": 11, 2024-09-21T03:56:51.809 INFO:teuthology.orchestra.run.smithi161.stdout: "dir_split": 0, 2024-09-21T03:56:51.810 INFO:teuthology.orchestra.run.smithi161.stdout: "dir_merge": 0, 2024-09-21T03:56:51.810 INFO:teuthology.orchestra.run.smithi161.stdout: "inodes": 16, 2024-09-21T03:56:51.810 INFO:teuthology.orchestra.run.smithi161.stdout: "inodes_top": 0, 2024-09-21T03:56:51.810 INFO:teuthology.orchestra.run.smithi161.stdout: "inodes_bottom": 16, 2024-09-21T03:56:51.810 INFO:teuthology.orchestra.run.smithi161.stdout: "inodes_pin_tail": 0, 2024-09-21T03:56:51.810 INFO:teuthology.orchestra.run.smithi161.stdout: "inodes_pinned": 13, 2024-09-21T03:56:51.810 INFO:teuthology.orchestra.run.smithi161.stdout: "inodes_expired": 4, 2024-09-21T03:56:51.810 INFO:teuthology.orchestra.run.smithi161.stdout: "inodes_with_caps": 0, 2024-09-21T03:56:51.810 INFO:teuthology.orchestra.run.smithi161.stdout: "caps": 0, 2024-09-21T03:56:51.810 INFO:teuthology.orchestra.run.smithi161.stdout: "subtrees": 3, 2024-09-21T03:56:51.810 INFO:teuthology.orchestra.run.smithi161.stdout: "traverse": 11, 2024-09-21T03:56:51.810 INFO:teuthology.orchestra.run.smithi161.stdout: "traverse_hit": 5, 2024-09-21T03:56:51.810 INFO:teuthology.orchestra.run.smithi161.stdout: "traverse_forward": 0, 2024-09-21T03:56:51.810 INFO:teuthology.orchestra.run.smithi161.stdout: "traverse_discover": 5, 2024-09-21T03:56:51.810 INFO:teuthology.orchestra.run.smithi161.stdout: "traverse_dir_fetch": 0, 2024-09-21T03:56:51.811 INFO:teuthology.orchestra.run.smithi161.stdout: "traverse_remote_ino": 0, 2024-09-21T03:56:51.811 INFO:teuthology.orchestra.run.smithi161.stdout: "traverse_lock": 0, 2024-09-21T03:56:51.811 INFO:teuthology.orchestra.run.smithi161.stdout: "load_cent": 0, 2024-09-21T03:56:51.811 INFO:teuthology.orchestra.run.smithi161.stdout: "q": 0, 2024-09-21T03:56:51.811 INFO:teuthology.orchestra.run.smithi161.stdout: "exported": 0, 2024-09-21T03:56:51.811 INFO:teuthology.orchestra.run.smithi161.stdout: "exported_inodes": 0, 2024-09-21T03:56:51.811 INFO:teuthology.orchestra.run.smithi161.stdout: "imported": 0, 2024-09-21T03:56:51.811 INFO:teuthology.orchestra.run.smithi161.stdout: "imported_inodes": 0, 2024-09-21T03:56:51.811 INFO:teuthology.orchestra.run.smithi161.stdout: "openino_dir_fetch": 0, 2024-09-21T03:56:51.811 INFO:teuthology.orchestra.run.smithi161.stdout: "openino_backtrace_fetch": 0, 2024-09-21T03:56:51.811 INFO:teuthology.orchestra.run.smithi161.stdout: "openino_peer_discover": 0, 2024-09-21T03:56:51.811 INFO:teuthology.orchestra.run.smithi161.stdout: "root_rfiles": 4, 2024-09-21T03:56:51.811 INFO:teuthology.orchestra.run.smithi161.stdout: "root_rbytes": 234, 2024-09-21T03:56:51.811 INFO:teuthology.orchestra.run.smithi161.stdout: "root_rsnaps": 6, 2024-09-21T03:56:51.811 INFO:teuthology.orchestra.run.smithi161.stdout: "scrub_backtrace_fetch": 0, 2024-09-21T03:56:51.812 INFO:teuthology.orchestra.run.smithi161.stdout: "scrub_set_tag": 0, 2024-09-21T03:56:51.812 INFO:teuthology.orchestra.run.smithi161.stdout: "scrub_backtrace_repaired": 0, 2024-09-21T03:56:51.812 INFO:teuthology.orchestra.run.smithi161.stdout: "scrub_inotable_repaired": 0, 2024-09-21T03:56:51.812 INFO:teuthology.orchestra.run.smithi161.stdout: "scrub_dir_inodes": 0, 2024-09-21T03:56:51.812 INFO:teuthology.orchestra.run.smithi161.stdout: "scrub_dir_base_inodes": 0, 2024-09-21T03:56:51.812 INFO:teuthology.orchestra.run.smithi161.stdout: "scrub_dirfrag_rstats": 0, 2024-09-21T03:56:51.812 INFO:teuthology.orchestra.run.smithi161.stdout: "scrub_file_inodes": 0, 2024-09-21T03:56:51.812 INFO:teuthology.orchestra.run.smithi161.stdout: "handle_inode_file_caps": 0, 2024-09-21T03:56:51.812 INFO:teuthology.orchestra.run.smithi161.stdout: "ceph_cap_op_revoke": 0, 2024-09-21T03:56:51.812 INFO:teuthology.orchestra.run.smithi161.stdout: "ceph_cap_op_grant": 0, 2024-09-21T03:56:51.812 INFO:teuthology.orchestra.run.smithi161.stdout: "ceph_cap_op_trunc": 0, 2024-09-21T03:56:51.812 INFO:teuthology.orchestra.run.smithi161.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-21T03:56:51.812 INFO:teuthology.orchestra.run.smithi161.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-21T03:56:51.812 INFO:teuthology.orchestra.run.smithi161.stdout: "handle_client_caps": 0, 2024-09-21T03:56:51.812 INFO:teuthology.orchestra.run.smithi161.stdout: "handle_client_caps_dirty": 0, 2024-09-21T03:56:51.813 INFO:teuthology.orchestra.run.smithi161.stdout: "handle_client_cap_release": 0, 2024-09-21T03:56:51.813 INFO:teuthology.orchestra.run.smithi161.stdout: "process_request_cap_release": 1 2024-09-21T03:56:51.813 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T03:56:51.813 INFO:teuthology.orchestra.run.smithi161.stdout: "mds_cache": { 2024-09-21T03:56:51.813 INFO:teuthology.orchestra.run.smithi161.stdout: "dir_update": 0, 2024-09-21T03:56:51.813 INFO:teuthology.orchestra.run.smithi161.stdout: "dir_update_receipt": 11, 2024-09-21T03:56:51.813 INFO:teuthology.orchestra.run.smithi161.stdout: "dir_try_discover": 7, 2024-09-21T03:56:51.813 INFO:teuthology.orchestra.run.smithi161.stdout: "dir_send_discover": 8, 2024-09-21T03:56:51.813 INFO:teuthology.orchestra.run.smithi161.stdout: "dir_handle_discover": 0, 2024-09-21T03:56:51.813 INFO:teuthology.orchestra.run.smithi161.stdout: "num_strays": 0, 2024-09-21T03:56:51.813 INFO:teuthology.orchestra.run.smithi161.stdout: "num_strays_delayed": 0, 2024-09-21T03:56:51.813 INFO:teuthology.orchestra.run.smithi161.stdout: "num_strays_enqueuing": 0, 2024-09-21T03:56:51.813 INFO:teuthology.orchestra.run.smithi161.stdout: "strays_created": 0, 2024-09-21T03:56:51.813 INFO:teuthology.orchestra.run.smithi161.stdout: "strays_enqueued": 0, 2024-09-21T03:56:51.814 INFO:teuthology.orchestra.run.smithi161.stdout: "strays_reintegrated": 0, 2024-09-21T03:56:51.814 INFO:teuthology.orchestra.run.smithi161.stdout: "strays_migrated": 0, 2024-09-21T03:56:51.814 INFO:teuthology.orchestra.run.smithi161.stdout: "num_recovering_processing": 0, 2024-09-21T03:56:51.814 INFO:teuthology.orchestra.run.smithi161.stdout: "num_recovering_enqueued": 0, 2024-09-21T03:56:51.814 INFO:teuthology.orchestra.run.smithi161.stdout: "num_recovering_prioritized": 0, 2024-09-21T03:56:51.814 INFO:teuthology.orchestra.run.smithi161.stdout: "recovery_started": 0, 2024-09-21T03:56:51.814 INFO:teuthology.orchestra.run.smithi161.stdout: "recovery_completed": 0, 2024-09-21T03:56:51.814 INFO:teuthology.orchestra.run.smithi161.stdout: "ireq_quiesce_path": 0, 2024-09-21T03:56:51.814 INFO:teuthology.orchestra.run.smithi161.stdout: "ireq_quiesce_inode": 0, 2024-09-21T03:56:51.814 INFO:teuthology.orchestra.run.smithi161.stdout: "ireq_enqueue_scrub": 0, 2024-09-21T03:56:51.814 INFO:teuthology.orchestra.run.smithi161.stdout: "ireq_exportdir": 0, 2024-09-21T03:56:51.814 INFO:teuthology.orchestra.run.smithi161.stdout: "ireq_flush": 0, 2024-09-21T03:56:51.814 INFO:teuthology.orchestra.run.smithi161.stdout: "ireq_fragmentdir": 0, 2024-09-21T03:56:51.814 INFO:teuthology.orchestra.run.smithi161.stdout: "ireq_fragstats": 0, 2024-09-21T03:56:51.814 INFO:teuthology.orchestra.run.smithi161.stdout: "ireq_inodestats": 0, 2024-09-21T03:56:51.815 INFO:teuthology.orchestra.run.smithi161.stdout: "uninline_started": 0, 2024-09-21T03:56:51.815 INFO:teuthology.orchestra.run.smithi161.stdout: "uninline_succeeded": 0, 2024-09-21T03:56:51.815 INFO:teuthology.orchestra.run.smithi161.stdout: "uninline_write_failed": 0 2024-09-21T03:56:51.815 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T03:56:51.815 INFO:teuthology.orchestra.run.smithi161.stdout: "mds_log": { 2024-09-21T03:56:51.815 INFO:teuthology.orchestra.run.smithi161.stdout: "evlrg": 0, 2024-09-21T03:56:51.815 INFO:teuthology.orchestra.run.smithi161.stdout: "evadd": 6, 2024-09-21T03:56:51.815 INFO:teuthology.orchestra.run.smithi161.stdout: "evex": 0, 2024-09-21T03:56:51.815 INFO:teuthology.orchestra.run.smithi161.stdout: "evtrm": 0, 2024-09-21T03:56:51.815 INFO:teuthology.orchestra.run.smithi161.stdout: "ev": 6, 2024-09-21T03:56:51.815 INFO:teuthology.orchestra.run.smithi161.stdout: "evexg": 0, 2024-09-21T03:56:51.815 INFO:teuthology.orchestra.run.smithi161.stdout: "evexd": 0, 2024-09-21T03:56:51.815 INFO:teuthology.orchestra.run.smithi161.stdout: "segadd": 2, 2024-09-21T03:56:51.815 INFO:teuthology.orchestra.run.smithi161.stdout: "segex": 0, 2024-09-21T03:56:51.815 INFO:teuthology.orchestra.run.smithi161.stdout: "segtrm": 0, 2024-09-21T03:56:51.816 INFO:teuthology.orchestra.run.smithi161.stdout: "seg": 2, 2024-09-21T03:56:51.816 INFO:teuthology.orchestra.run.smithi161.stdout: "segmjr": 2, 2024-09-21T03:56:51.816 INFO:teuthology.orchestra.run.smithi161.stdout: "segexg": 0, 2024-09-21T03:56:51.816 INFO:teuthology.orchestra.run.smithi161.stdout: "segexd": 0, 2024-09-21T03:56:51.816 INFO:teuthology.orchestra.run.smithi161.stdout: "expos": 4194304, 2024-09-21T03:56:51.816 INFO:teuthology.orchestra.run.smithi161.stdout: "wrpos": 4195956, 2024-09-21T03:56:51.816 INFO:teuthology.orchestra.run.smithi161.stdout: "rdpos": 0, 2024-09-21T03:56:51.816 INFO:teuthology.orchestra.run.smithi161.stdout: "jlat": { 2024-09-21T03:56:51.816 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 5, 2024-09-21T03:56:51.816 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.011481943, 2024-09-21T03:56:51.816 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.002296388 2024-09-21T03:56:51.816 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T03:56:51.816 INFO:teuthology.orchestra.run.smithi161.stdout: "replayed": 0 2024-09-21T03:56:51.816 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T03:56:51.816 INFO:teuthology.orchestra.run.smithi161.stdout: "mds_mem": { 2024-09-21T03:56:51.817 INFO:teuthology.orchestra.run.smithi161.stdout: "ino": 20, 2024-09-21T03:56:51.817 INFO:teuthology.orchestra.run.smithi161.stdout: "ino+": 22, 2024-09-21T03:56:51.817 INFO:teuthology.orchestra.run.smithi161.stdout: "ino-": 2, 2024-09-21T03:56:51.817 INFO:teuthology.orchestra.run.smithi161.stdout: "dir": 19, 2024-09-21T03:56:51.817 INFO:teuthology.orchestra.run.smithi161.stdout: "dir+": 22, 2024-09-21T03:56:51.817 INFO:teuthology.orchestra.run.smithi161.stdout: "dir-": 3, 2024-09-21T03:56:51.817 INFO:teuthology.orchestra.run.smithi161.stdout: "dn": 16, 2024-09-21T03:56:51.817 INFO:teuthology.orchestra.run.smithi161.stdout: "dn+": 20, 2024-09-21T03:56:51.817 INFO:teuthology.orchestra.run.smithi161.stdout: "dn-": 4, 2024-09-21T03:56:51.817 INFO:teuthology.orchestra.run.smithi161.stdout: "cap": 0, 2024-09-21T03:56:51.817 INFO:teuthology.orchestra.run.smithi161.stdout: "cap+": 1, 2024-09-21T03:56:51.818 INFO:teuthology.orchestra.run.smithi161.stdout: "cap-": 1, 2024-09-21T03:56:51.818 INFO:teuthology.orchestra.run.smithi161.stdout: "rss": 118680, 2024-09-21T03:56:51.818 INFO:teuthology.orchestra.run.smithi161.stdout: "heap": 325861376 2024-09-21T03:56:51.818 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T03:56:51.818 INFO:teuthology.orchestra.run.smithi161.stdout: "mds_server": { 2024-09-21T03:56:51.818 INFO:teuthology.orchestra.run.smithi161.stdout: "dispatch_client_request": 3, 2024-09-21T03:56:51.818 INFO:teuthology.orchestra.run.smithi161.stdout: "dispatch_server_request": 0, 2024-09-21T03:56:51.818 INFO:teuthology.orchestra.run.smithi161.stdout: "handle_client_request": 2, 2024-09-21T03:56:51.818 INFO:teuthology.orchestra.run.smithi161.stdout: "handle_client_session": 8, 2024-09-21T03:56:51.818 INFO:teuthology.orchestra.run.smithi161.stdout: "handle_peer_request": 0, 2024-09-21T03:56:51.818 INFO:teuthology.orchestra.run.smithi161.stdout: "req_create_latency": { 2024-09-21T03:56:51.818 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-09-21T03:56:51.819 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-09-21T03:56:51.819 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-09-21T03:56:51.819 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T03:56:51.819 INFO:teuthology.orchestra.run.smithi161.stdout: "req_getattr_latency": { 2024-09-21T03:56:51.819 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 1, 2024-09-21T03:56:51.819 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000176243, 2024-09-21T03:56:51.819 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000176243 2024-09-21T03:56:51.819 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T03:56:51.819 INFO:teuthology.orchestra.run.smithi161.stdout: "req_getfilelock_latency": { 2024-09-21T03:56:51.819 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-09-21T03:56:51.819 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-09-21T03:56:51.819 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-09-21T03:56:51.819 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T03:56:51.820 INFO:teuthology.orchestra.run.smithi161.stdout: "req_link_latency": { 2024-09-21T03:56:51.820 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-09-21T03:56:51.820 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-09-21T03:56:51.820 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-09-21T03:56:51.820 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T03:56:51.820 INFO:teuthology.orchestra.run.smithi161.stdout: "req_lookup_latency": { 2024-09-21T03:56:51.820 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-09-21T03:56:51.820 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-09-21T03:56:51.820 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-09-21T03:56:51.820 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T03:56:51.820 INFO:teuthology.orchestra.run.smithi161.stdout: "req_lookuphash_latency": { 2024-09-21T03:56:51.820 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-09-21T03:56:51.820 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-09-21T03:56:51.820 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-09-21T03:56:51.820 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T03:56:51.821 INFO:teuthology.orchestra.run.smithi161.stdout: "req_lookupino_latency": { 2024-09-21T03:56:51.821 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-09-21T03:56:51.821 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-09-21T03:56:51.821 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-09-21T03:56:51.821 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T03:56:51.821 INFO:teuthology.orchestra.run.smithi161.stdout: "req_lookupname_latency": { 2024-09-21T03:56:51.821 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-09-21T03:56:51.821 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-09-21T03:56:51.821 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-09-21T03:56:51.821 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T03:56:51.821 INFO:teuthology.orchestra.run.smithi161.stdout: "req_lookupparent_latency": { 2024-09-21T03:56:51.821 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-09-21T03:56:51.821 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-09-21T03:56:51.821 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-09-21T03:56:51.821 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T03:56:51.822 INFO:teuthology.orchestra.run.smithi161.stdout: "req_lookupsnap_latency": { 2024-09-21T03:56:51.822 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-09-21T03:56:51.822 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-09-21T03:56:51.822 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-09-21T03:56:51.822 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T03:56:51.822 INFO:teuthology.orchestra.run.smithi161.stdout: "req_lssnap_latency": { 2024-09-21T03:56:51.822 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-09-21T03:56:51.822 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-09-21T03:56:51.823 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-09-21T03:56:51.823 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T03:56:51.823 INFO:teuthology.orchestra.run.smithi161.stdout: "req_mkdir_latency": { 2024-09-21T03:56:51.823 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-09-21T03:56:51.823 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-09-21T03:56:51.823 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-09-21T03:56:51.823 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T03:56:51.823 INFO:teuthology.orchestra.run.smithi161.stdout: "req_mknod_latency": { 2024-09-21T03:56:51.823 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-09-21T03:56:51.823 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-09-21T03:56:51.823 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-09-21T03:56:51.823 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T03:56:51.823 INFO:teuthology.orchestra.run.smithi161.stdout: "req_mksnap_latency": { 2024-09-21T03:56:51.823 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-09-21T03:56:51.824 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-09-21T03:56:51.824 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-09-21T03:56:51.824 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T03:56:51.824 INFO:teuthology.orchestra.run.smithi161.stdout: "req_open_latency": { 2024-09-21T03:56:51.824 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-09-21T03:56:51.824 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-09-21T03:56:51.824 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-09-21T03:56:51.824 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T03:56:51.824 INFO:teuthology.orchestra.run.smithi161.stdout: "req_readdir_latency": { 2024-09-21T03:56:51.824 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-09-21T03:56:51.824 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-09-21T03:56:51.824 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-09-21T03:56:51.824 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T03:56:51.824 INFO:teuthology.orchestra.run.smithi161.stdout: "req_rename_latency": { 2024-09-21T03:56:51.824 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-09-21T03:56:51.825 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-09-21T03:56:51.825 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-09-21T03:56:51.825 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T03:56:51.825 INFO:teuthology.orchestra.run.smithi161.stdout: "req_renamesnap_latency": { 2024-09-21T03:56:51.825 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-09-21T03:56:51.825 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-09-21T03:56:51.825 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-09-21T03:56:51.825 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T03:56:51.825 INFO:teuthology.orchestra.run.smithi161.stdout: "req_snapdiff_latency": { 2024-09-21T03:56:51.825 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-09-21T03:56:51.825 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-09-21T03:56:51.825 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-09-21T03:56:51.825 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T03:56:51.825 INFO:teuthology.orchestra.run.smithi161.stdout: "req_rmdir_latency": { 2024-09-21T03:56:51.826 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-09-21T03:56:51.826 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-09-21T03:56:51.826 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-09-21T03:56:51.826 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T03:56:51.826 INFO:teuthology.orchestra.run.smithi161.stdout: "req_rmsnap_latency": { 2024-09-21T03:56:51.826 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-09-21T03:56:51.826 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-09-21T03:56:51.826 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-09-21T03:56:51.826 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T03:56:51.826 INFO:teuthology.orchestra.run.smithi161.stdout: "req_rmxattr_latency": { 2024-09-21T03:56:51.826 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-09-21T03:56:51.826 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-09-21T03:56:51.826 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-09-21T03:56:51.826 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T03:56:51.826 INFO:teuthology.orchestra.run.smithi161.stdout: "req_setattr_latency": { 2024-09-21T03:56:51.827 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-09-21T03:56:51.827 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-09-21T03:56:51.827 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-09-21T03:56:51.827 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T03:56:51.827 INFO:teuthology.orchestra.run.smithi161.stdout: "req_setdirlayout_latency": { 2024-09-21T03:56:51.827 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-09-21T03:56:51.827 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-09-21T03:56:51.827 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-09-21T03:56:51.827 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T03:56:51.827 INFO:teuthology.orchestra.run.smithi161.stdout: "req_setfilelock_latency": { 2024-09-21T03:56:51.827 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-09-21T03:56:51.827 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-09-21T03:56:51.827 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-09-21T03:56:51.827 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T03:56:51.827 INFO:teuthology.orchestra.run.smithi161.stdout: "req_setlayout_latency": { 2024-09-21T03:56:51.828 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-09-21T03:56:51.828 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-09-21T03:56:51.828 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-09-21T03:56:51.828 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T03:56:51.828 INFO:teuthology.orchestra.run.smithi161.stdout: "req_setxattr_latency": { 2024-09-21T03:56:51.828 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-09-21T03:56:51.828 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-09-21T03:56:51.828 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-09-21T03:56:51.828 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T03:56:51.828 INFO:teuthology.orchestra.run.smithi161.stdout: "req_symlink_latency": { 2024-09-21T03:56:51.828 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-09-21T03:56:51.828 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-09-21T03:56:51.828 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-09-21T03:56:51.828 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T03:56:51.829 INFO:teuthology.orchestra.run.smithi161.stdout: "req_unlink_latency": { 2024-09-21T03:56:51.829 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-09-21T03:56:51.829 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-09-21T03:56:51.829 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-09-21T03:56:51.829 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T03:56:51.829 INFO:teuthology.orchestra.run.smithi161.stdout: "cap_revoke_eviction": 0, 2024-09-21T03:56:51.829 INFO:teuthology.orchestra.run.smithi161.stdout: "cap_acquisition_throttle": 0, 2024-09-21T03:56:51.829 INFO:teuthology.orchestra.run.smithi161.stdout: "req_getvxattr_latency": { 2024-09-21T03:56:51.829 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-09-21T03:56:51.829 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-09-21T03:56:51.829 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-09-21T03:56:51.829 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T03:56:51.829 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T03:56:51.829 INFO:teuthology.orchestra.run.smithi161.stdout: "mds_sessions": { 2024-09-21T03:56:51.829 INFO:teuthology.orchestra.run.smithi161.stdout: "session_count": 0, 2024-09-21T03:56:51.830 INFO:teuthology.orchestra.run.smithi161.stdout: "session_add": 2, 2024-09-21T03:56:51.830 INFO:teuthology.orchestra.run.smithi161.stdout: "session_remove": 2, 2024-09-21T03:56:51.830 INFO:teuthology.orchestra.run.smithi161.stdout: "sessions_open": 0, 2024-09-21T03:56:51.830 INFO:teuthology.orchestra.run.smithi161.stdout: "sessions_stale": 0, 2024-09-21T03:56:51.830 INFO:teuthology.orchestra.run.smithi161.stdout: "total_load": 1, 2024-09-21T03:56:51.830 INFO:teuthology.orchestra.run.smithi161.stdout: "average_load": 1, 2024-09-21T03:56:51.830 INFO:teuthology.orchestra.run.smithi161.stdout: "avg_session_uptime": 71, 2024-09-21T03:56:51.830 INFO:teuthology.orchestra.run.smithi161.stdout: "mdthresh_evicted": 0 2024-09-21T03:56:51.830 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T03:56:51.830 INFO:teuthology.orchestra.run.smithi161.stdout: "mempool": { 2024-09-21T03:56:51.830 INFO:teuthology.orchestra.run.smithi161.stdout: "bloom_filter_bytes": 0, 2024-09-21T03:56:51.830 INFO:teuthology.orchestra.run.smithi161.stdout: "bloom_filter_items": 0, 2024-09-21T03:56:51.830 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_alloc_bytes": 0, 2024-09-21T03:56:51.830 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_alloc_items": 0, 2024-09-21T03:56:51.830 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_cache_data_bytes": 0, 2024-09-21T03:56:51.831 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_cache_data_items": 0, 2024-09-21T03:56:51.831 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-21T03:56:51.831 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_cache_onode_items": 0, 2024-09-21T03:56:51.831 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-21T03:56:51.831 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_cache_meta_items": 0, 2024-09-21T03:56:51.831 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_cache_other_bytes": 0, 2024-09-21T03:56:51.831 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_cache_other_items": 0, 2024-09-21T03:56:51.831 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-21T03:56:51.831 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_cache_buffer_items": 0, 2024-09-21T03:56:51.831 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_extent_bytes": 0, 2024-09-21T03:56:51.831 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_extent_items": 0, 2024-09-21T03:56:51.831 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_blob_bytes": 0, 2024-09-21T03:56:51.831 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_blob_items": 0, 2024-09-21T03:56:51.831 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-21T03:56:51.832 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_shared_blob_items": 0, 2024-09-21T03:56:51.832 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-21T03:56:51.832 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_inline_bl_items": 0, 2024-09-21T03:56:51.832 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_fsck_bytes": 0, 2024-09-21T03:56:51.832 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_fsck_items": 0, 2024-09-21T03:56:51.832 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_txc_bytes": 0, 2024-09-21T03:56:51.832 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_txc_items": 0, 2024-09-21T03:56:51.832 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-21T03:56:51.832 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_writing_deferred_items": 0, 2024-09-21T03:56:51.832 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_writing_bytes": 0, 2024-09-21T03:56:51.832 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_writing_items": 0, 2024-09-21T03:56:51.832 INFO:teuthology.orchestra.run.smithi161.stdout: "bluefs_bytes": 0, 2024-09-21T03:56:51.832 INFO:teuthology.orchestra.run.smithi161.stdout: "bluefs_items": 0, 2024-09-21T03:56:51.832 INFO:teuthology.orchestra.run.smithi161.stdout: "bluefs_file_reader_bytes": 0, 2024-09-21T03:56:51.832 INFO:teuthology.orchestra.run.smithi161.stdout: "bluefs_file_reader_items": 0, 2024-09-21T03:56:51.833 INFO:teuthology.orchestra.run.smithi161.stdout: "bluefs_file_writer_bytes": 0, 2024-09-21T03:56:51.833 INFO:teuthology.orchestra.run.smithi161.stdout: "bluefs_file_writer_items": 0, 2024-09-21T03:56:51.833 INFO:teuthology.orchestra.run.smithi161.stdout: "buffer_anon_bytes": 246887, 2024-09-21T03:56:51.833 INFO:teuthology.orchestra.run.smithi161.stdout: "buffer_anon_items": 83, 2024-09-21T03:56:51.833 INFO:teuthology.orchestra.run.smithi161.stdout: "buffer_meta_bytes": 0, 2024-09-21T03:56:51.833 INFO:teuthology.orchestra.run.smithi161.stdout: "buffer_meta_items": 0, 2024-09-21T03:56:51.833 INFO:teuthology.orchestra.run.smithi161.stdout: "osd_bytes": 0, 2024-09-21T03:56:51.833 INFO:teuthology.orchestra.run.smithi161.stdout: "osd_items": 0, 2024-09-21T03:56:51.833 INFO:teuthology.orchestra.run.smithi161.stdout: "osd_mapbl_bytes": 0, 2024-09-21T03:56:51.833 INFO:teuthology.orchestra.run.smithi161.stdout: "osd_mapbl_items": 0, 2024-09-21T03:56:51.833 INFO:teuthology.orchestra.run.smithi161.stdout: "osd_pglog_bytes": 0, 2024-09-21T03:56:51.833 INFO:teuthology.orchestra.run.smithi161.stdout: "osd_pglog_items": 0, 2024-09-21T03:56:51.833 INFO:teuthology.orchestra.run.smithi161.stdout: "osdmap_bytes": 8256, 2024-09-21T03:56:51.833 INFO:teuthology.orchestra.run.smithi161.stdout: "osdmap_items": 182, 2024-09-21T03:56:51.833 INFO:teuthology.orchestra.run.smithi161.stdout: "osdmap_mapping_bytes": 0, 2024-09-21T03:56:51.834 INFO:teuthology.orchestra.run.smithi161.stdout: "osdmap_mapping_items": 0, 2024-09-21T03:56:51.834 INFO:teuthology.orchestra.run.smithi161.stdout: "pgmap_bytes": 0, 2024-09-21T03:56:51.834 INFO:teuthology.orchestra.run.smithi161.stdout: "pgmap_items": 0, 2024-09-21T03:56:51.834 INFO:teuthology.orchestra.run.smithi161.stdout: "mds_co_bytes": 86073, 2024-09-21T03:56:51.834 INFO:teuthology.orchestra.run.smithi161.stdout: "mds_co_items": 420, 2024-09-21T03:56:51.834 INFO:teuthology.orchestra.run.smithi161.stdout: "unittest_1_bytes": 0, 2024-09-21T03:56:51.834 INFO:teuthology.orchestra.run.smithi161.stdout: "unittest_1_items": 0, 2024-09-21T03:56:51.834 INFO:teuthology.orchestra.run.smithi161.stdout: "unittest_2_bytes": 0, 2024-09-21T03:56:51.834 INFO:teuthology.orchestra.run.smithi161.stdout: "unittest_2_items": 0 2024-09-21T03:56:51.834 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T03:56:51.834 INFO:teuthology.orchestra.run.smithi161.stdout: "objecter": { 2024-09-21T03:56:51.834 INFO:teuthology.orchestra.run.smithi161.stdout: "op_active": 0, 2024-09-21T03:56:51.834 INFO:teuthology.orchestra.run.smithi161.stdout: "op_laggy": 0, 2024-09-21T03:56:51.834 INFO:teuthology.orchestra.run.smithi161.stdout: "op_send": 41, 2024-09-21T03:56:51.834 INFO:teuthology.orchestra.run.smithi161.stdout: "op_send_bytes": 11954, 2024-09-21T03:56:51.835 INFO:teuthology.orchestra.run.smithi161.stdout: "op_resend": 28, 2024-09-21T03:56:51.835 INFO:teuthology.orchestra.run.smithi161.stdout: "op_reply": 41, 2024-09-21T03:56:51.835 INFO:teuthology.orchestra.run.smithi161.stdout: "op_latency": { 2024-09-21T03:56:51.835 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 41, 2024-09-21T03:56:51.835 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.244994777, 2024-09-21T03:56:51.835 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.005975482 2024-09-21T03:56:51.835 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T03:56:51.835 INFO:teuthology.orchestra.run.smithi161.stdout: "op_inflight": 0, 2024-09-21T03:56:51.835 INFO:teuthology.orchestra.run.smithi161.stdout: "oplen_avg": { 2024-09-21T03:56:51.835 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 41, 2024-09-21T03:56:51.835 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 63 2024-09-21T03:56:51.835 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T03:56:51.835 INFO:teuthology.orchestra.run.smithi161.stdout: "op": 41, 2024-09-21T03:56:51.835 INFO:teuthology.orchestra.run.smithi161.stdout: "op_r": 0, 2024-09-21T03:56:51.835 INFO:teuthology.orchestra.run.smithi161.stdout: "op_w": 41, 2024-09-21T03:56:51.836 INFO:teuthology.orchestra.run.smithi161.stdout: "op_rmw": 0, 2024-09-21T03:56:51.836 INFO:teuthology.orchestra.run.smithi161.stdout: "op_pg": 0, 2024-09-21T03:56:51.836 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_stat": 0, 2024-09-21T03:56:51.836 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_create": 10, 2024-09-21T03:56:51.836 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_read": 0, 2024-09-21T03:56:51.836 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_write": 5, 2024-09-21T03:56:51.836 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_writefull": 7, 2024-09-21T03:56:51.836 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_writesame": 0, 2024-09-21T03:56:51.836 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_append": 0, 2024-09-21T03:56:51.836 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_zero": 0, 2024-09-21T03:56:51.836 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_truncate": 0, 2024-09-21T03:56:51.836 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_delete": 6, 2024-09-21T03:56:51.837 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_mapext": 0, 2024-09-21T03:56:51.837 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_sparse_read": 0, 2024-09-21T03:56:51.837 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_clonerange": 0, 2024-09-21T03:56:51.837 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_getxattr": 0, 2024-09-21T03:56:51.837 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_setxattr": 20, 2024-09-21T03:56:51.837 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_cmpxattr": 0, 2024-09-21T03:56:51.837 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_rmxattr": 0, 2024-09-21T03:56:51.837 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_resetxattrs": 0, 2024-09-21T03:56:51.837 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_call": 0, 2024-09-21T03:56:51.837 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_watch": 0, 2024-09-21T03:56:51.837 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_notify": 0, 2024-09-21T03:56:51.837 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_src_cmpxattr": 0, 2024-09-21T03:56:51.837 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_pgls": 0, 2024-09-21T03:56:51.837 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_pgls_filter": 0, 2024-09-21T03:56:51.838 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_other": 0, 2024-09-21T03:56:51.838 INFO:teuthology.orchestra.run.smithi161.stdout: "linger_active": 0, 2024-09-21T03:56:51.838 INFO:teuthology.orchestra.run.smithi161.stdout: "linger_send": 0, 2024-09-21T03:56:51.838 INFO:teuthology.orchestra.run.smithi161.stdout: "linger_resend": 0, 2024-09-21T03:56:51.838 INFO:teuthology.orchestra.run.smithi161.stdout: "linger_ping": 0, 2024-09-21T03:56:51.838 INFO:teuthology.orchestra.run.smithi161.stdout: "poolop_active": 0, 2024-09-21T03:56:51.838 INFO:teuthology.orchestra.run.smithi161.stdout: "poolop_send": 0, 2024-09-21T03:56:51.838 INFO:teuthology.orchestra.run.smithi161.stdout: "poolop_resend": 0, 2024-09-21T03:56:51.838 INFO:teuthology.orchestra.run.smithi161.stdout: "poolstat_active": 0, 2024-09-21T03:56:51.838 INFO:teuthology.orchestra.run.smithi161.stdout: "poolstat_send": 0, 2024-09-21T03:56:51.838 INFO:teuthology.orchestra.run.smithi161.stdout: "poolstat_resend": 0, 2024-09-21T03:56:51.838 INFO:teuthology.orchestra.run.smithi161.stdout: "statfs_active": 0, 2024-09-21T03:56:51.838 INFO:teuthology.orchestra.run.smithi161.stdout: "statfs_send": 0, 2024-09-21T03:56:51.838 INFO:teuthology.orchestra.run.smithi161.stdout: "statfs_resend": 0, 2024-09-21T03:56:51.839 INFO:teuthology.orchestra.run.smithi161.stdout: "command_active": 0, 2024-09-21T03:56:51.839 INFO:teuthology.orchestra.run.smithi161.stdout: "command_send": 0, 2024-09-21T03:56:51.839 INFO:teuthology.orchestra.run.smithi161.stdout: "command_resend": 0, 2024-09-21T03:56:51.839 INFO:teuthology.orchestra.run.smithi161.stdout: "map_epoch": 87, 2024-09-21T03:56:51.839 INFO:teuthology.orchestra.run.smithi161.stdout: "map_full": 0, 2024-09-21T03:56:51.839 INFO:teuthology.orchestra.run.smithi161.stdout: "map_inc": 10, 2024-09-21T03:56:51.839 INFO:teuthology.orchestra.run.smithi161.stdout: "osd_sessions": 9, 2024-09-21T03:56:51.839 INFO:teuthology.orchestra.run.smithi161.stdout: "osd_session_open": 9, 2024-09-21T03:56:51.839 INFO:teuthology.orchestra.run.smithi161.stdout: "osd_session_close": 0, 2024-09-21T03:56:51.839 INFO:teuthology.orchestra.run.smithi161.stdout: "osd_laggy": 0, 2024-09-21T03:56:51.839 INFO:teuthology.orchestra.run.smithi161.stdout: "omap_wr": 14, 2024-09-21T03:56:51.839 INFO:teuthology.orchestra.run.smithi161.stdout: "omap_rd": 0, 2024-09-21T03:56:51.839 INFO:teuthology.orchestra.run.smithi161.stdout: "omap_del": 1 2024-09-21T03:56:51.839 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T03:56:51.840 INFO:teuthology.orchestra.run.smithi161.stdout: "oft": { 2024-09-21T03:56:51.840 INFO:teuthology.orchestra.run.smithi161.stdout: "omap_total_objs": 1, 2024-09-21T03:56:51.840 INFO:teuthology.orchestra.run.smithi161.stdout: "omap_total_kv_pairs": 0, 2024-09-21T03:56:51.840 INFO:teuthology.orchestra.run.smithi161.stdout: "omap_total_updates": 0, 2024-09-21T03:56:51.840 INFO:teuthology.orchestra.run.smithi161.stdout: "omap_total_removes": 0 2024-09-21T03:56:51.840 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T03:56:51.840 INFO:teuthology.orchestra.run.smithi161.stdout: "purge_queue": { 2024-09-21T03:56:51.840 INFO:teuthology.orchestra.run.smithi161.stdout: "pq_executing_ops": 0, 2024-09-21T03:56:51.840 INFO:teuthology.orchestra.run.smithi161.stdout: "pq_executing_ops_high_water": 0, 2024-09-21T03:56:51.840 INFO:teuthology.orchestra.run.smithi161.stdout: "pq_executing": 0, 2024-09-21T03:56:51.840 INFO:teuthology.orchestra.run.smithi161.stdout: "pq_executing_high_water": 0, 2024-09-21T03:56:51.840 INFO:teuthology.orchestra.run.smithi161.stdout: "pq_executed_ops": 0, 2024-09-21T03:56:51.840 INFO:teuthology.orchestra.run.smithi161.stdout: "pq_executed": 0, 2024-09-21T03:56:51.840 INFO:teuthology.orchestra.run.smithi161.stdout: "pq_item_in_journal": 0 2024-09-21T03:56:51.841 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T03:56:51.841 INFO:teuthology.orchestra.run.smithi161.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-21T03:56:51.841 INFO:teuthology.orchestra.run.smithi161.stdout: "val": 0, 2024-09-21T03:56:51.841 INFO:teuthology.orchestra.run.smithi161.stdout: "max": 104857600, 2024-09-21T03:56:51.841 INFO:teuthology.orchestra.run.smithi161.stdout: "get_started": 0, 2024-09-21T03:56:51.841 INFO:teuthology.orchestra.run.smithi161.stdout: "get": 793, 2024-09-21T03:56:51.841 INFO:teuthology.orchestra.run.smithi161.stdout: "get_sum": 1282532, 2024-09-21T03:56:51.841 INFO:teuthology.orchestra.run.smithi161.stdout: "get_or_fail_fail": 0, 2024-09-21T03:56:51.841 INFO:teuthology.orchestra.run.smithi161.stdout: "get_or_fail_success": 793, 2024-09-21T03:56:51.841 INFO:teuthology.orchestra.run.smithi161.stdout: "take": 0, 2024-09-21T03:56:51.841 INFO:teuthology.orchestra.run.smithi161.stdout: "take_sum": 0, 2024-09-21T03:56:51.841 INFO:teuthology.orchestra.run.smithi161.stdout: "put": 793, 2024-09-21T03:56:51.841 INFO:teuthology.orchestra.run.smithi161.stdout: "put_sum": 1282532, 2024-09-21T03:56:51.841 INFO:teuthology.orchestra.run.smithi161.stdout: "wait": { 2024-09-21T03:56:51.841 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-09-21T03:56:51.841 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-09-21T03:56:51.841 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-09-21T03:56:51.842 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T03:56:51.842 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T03:56:51.842 INFO:teuthology.orchestra.run.smithi161.stdout: "throttle-objecter_bytes": { 2024-09-21T03:56:51.842 INFO:teuthology.orchestra.run.smithi161.stdout: "val": 0, 2024-09-21T03:56:51.842 INFO:teuthology.orchestra.run.smithi161.stdout: "max": 104857600, 2024-09-21T03:56:51.842 INFO:teuthology.orchestra.run.smithi161.stdout: "get_started": 0, 2024-09-21T03:56:51.842 INFO:teuthology.orchestra.run.smithi161.stdout: "get": 0, 2024-09-21T03:56:51.842 INFO:teuthology.orchestra.run.smithi161.stdout: "get_sum": 0, 2024-09-21T03:56:51.842 INFO:teuthology.orchestra.run.smithi161.stdout: "get_or_fail_fail": 0, 2024-09-21T03:56:51.842 INFO:teuthology.orchestra.run.smithi161.stdout: "get_or_fail_success": 0, 2024-09-21T03:56:51.842 INFO:teuthology.orchestra.run.smithi161.stdout: "take": 41, 2024-09-21T03:56:51.842 INFO:teuthology.orchestra.run.smithi161.stdout: "take_sum": 11954, 2024-09-21T03:56:51.842 INFO:teuthology.orchestra.run.smithi161.stdout: "put": 41, 2024-09-21T03:56:51.842 INFO:teuthology.orchestra.run.smithi161.stdout: "put_sum": 11954, 2024-09-21T03:56:51.842 INFO:teuthology.orchestra.run.smithi161.stdout: "wait": { 2024-09-21T03:56:51.843 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-09-21T03:56:51.843 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-09-21T03:56:51.843 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-09-21T03:56:51.843 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T03:56:51.843 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T03:56:51.843 INFO:teuthology.orchestra.run.smithi161.stdout: "throttle-objecter_ops": { 2024-09-21T03:56:51.843 INFO:teuthology.orchestra.run.smithi161.stdout: "val": 0, 2024-09-21T03:56:51.843 INFO:teuthology.orchestra.run.smithi161.stdout: "max": 1024, 2024-09-21T03:56:51.843 INFO:teuthology.orchestra.run.smithi161.stdout: "get_started": 0, 2024-09-21T03:56:51.843 INFO:teuthology.orchestra.run.smithi161.stdout: "get": 0, 2024-09-21T03:56:51.843 INFO:teuthology.orchestra.run.smithi161.stdout: "get_sum": 0, 2024-09-21T03:56:51.843 INFO:teuthology.orchestra.run.smithi161.stdout: "get_or_fail_fail": 0, 2024-09-21T03:56:51.843 INFO:teuthology.orchestra.run.smithi161.stdout: "get_or_fail_success": 0, 2024-09-21T03:56:51.843 INFO:teuthology.orchestra.run.smithi161.stdout: "take": 41, 2024-09-21T03:56:51.843 INFO:teuthology.orchestra.run.smithi161.stdout: "take_sum": 41, 2024-09-21T03:56:51.844 INFO:teuthology.orchestra.run.smithi161.stdout: "put": 41, 2024-09-21T03:56:51.844 INFO:teuthology.orchestra.run.smithi161.stdout: "put_sum": 41, 2024-09-21T03:56:51.844 INFO:teuthology.orchestra.run.smithi161.stdout: "wait": { 2024-09-21T03:56:51.844 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-09-21T03:56:51.844 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-09-21T03:56:51.844 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-09-21T03:56:51.844 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T03:56:51.844 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T03:56:51.844 INFO:teuthology.orchestra.run.smithi161.stdout: "throttle-write_buf_throttle": { 2024-09-21T03:56:51.844 INFO:teuthology.orchestra.run.smithi161.stdout: "val": 0, 2024-09-21T03:56:51.844 INFO:teuthology.orchestra.run.smithi161.stdout: "max": 3758096384, 2024-09-21T03:56:51.844 INFO:teuthology.orchestra.run.smithi161.stdout: "get_started": 0, 2024-09-21T03:56:51.844 INFO:teuthology.orchestra.run.smithi161.stdout: "get": 0, 2024-09-21T03:56:51.844 INFO:teuthology.orchestra.run.smithi161.stdout: "get_sum": 0, 2024-09-21T03:56:51.845 INFO:teuthology.orchestra.run.smithi161.stdout: "get_or_fail_fail": 0, 2024-09-21T03:56:51.845 INFO:teuthology.orchestra.run.smithi161.stdout: "get_or_fail_success": 0, 2024-09-21T03:56:51.845 INFO:teuthology.orchestra.run.smithi161.stdout: "take": 0, 2024-09-21T03:56:51.845 INFO:teuthology.orchestra.run.smithi161.stdout: "take_sum": 0, 2024-09-21T03:56:51.845 INFO:teuthology.orchestra.run.smithi161.stdout: "put": 0, 2024-09-21T03:56:51.845 INFO:teuthology.orchestra.run.smithi161.stdout: "put_sum": 0, 2024-09-21T03:56:51.845 INFO:teuthology.orchestra.run.smithi161.stdout: "wait": { 2024-09-21T03:56:51.845 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-09-21T03:56:51.845 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-09-21T03:56:51.845 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-09-21T03:56:51.845 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T03:56:51.845 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T03:56:51.845 INFO:teuthology.orchestra.run.smithi161.stdout: "throttle-write_buf_throttle-0x55f246db75f0": { 2024-09-21T03:56:51.845 INFO:teuthology.orchestra.run.smithi161.stdout: "val": 0, 2024-09-21T03:56:51.845 INFO:teuthology.orchestra.run.smithi161.stdout: "max": 3758096384, 2024-09-21T03:56:51.846 INFO:teuthology.orchestra.run.smithi161.stdout: "get_started": 0, 2024-09-21T03:56:51.846 INFO:teuthology.orchestra.run.smithi161.stdout: "get": 6, 2024-09-21T03:56:51.846 INFO:teuthology.orchestra.run.smithi161.stdout: "get_sum": 1652, 2024-09-21T03:56:51.846 INFO:teuthology.orchestra.run.smithi161.stdout: "get_or_fail_fail": 0, 2024-09-21T03:56:51.846 INFO:teuthology.orchestra.run.smithi161.stdout: "get_or_fail_success": 6, 2024-09-21T03:56:51.846 INFO:teuthology.orchestra.run.smithi161.stdout: "take": 0, 2024-09-21T03:56:51.846 INFO:teuthology.orchestra.run.smithi161.stdout: "take_sum": 0, 2024-09-21T03:56:51.846 INFO:teuthology.orchestra.run.smithi161.stdout: "put": 5, 2024-09-21T03:56:51.846 INFO:teuthology.orchestra.run.smithi161.stdout: "put_sum": 1652, 2024-09-21T03:56:51.846 INFO:teuthology.orchestra.run.smithi161.stdout: "wait": { 2024-09-21T03:56:51.847 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-09-21T03:56:51.847 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-09-21T03:56:51.847 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-09-21T03:56:51.847 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T03:56:51.847 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T03:56:51.847 INFO:teuthology.orchestra.run.smithi161.stdout: "trackedop": { 2024-09-21T03:56:51.847 INFO:teuthology.orchestra.run.smithi161.stdout: "slow_ops_count": 0 2024-09-21T03:56:51.847 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T03:56:51.847 INFO:teuthology.orchestra.run.smithi161.stdout:} 2024-09-21T03:56:51.848 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:56:51 smithi039 ceph-mon[31211]: pgmap v657: 97 pgs: 97 active+clean; 10 MiB data, 525 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-09-21T03:56:52.529 INFO:tasks.check_counter:Daemon mds.i mds.exported=0 2024-09-21T03:56:52.529 INFO:tasks.check_counter:Daemon mds.i mds.imported=0 2024-09-21T03:56:52.529 INFO:tasks.check_counter:Daemon mds.i mds_cache.dir_update=0 2024-09-21T03:56:52.529 INFO:tasks.check_counter:Daemon mds.i mds_cache.dir_update_receipt=11 2024-09-21T03:56:52.529 INFO:tasks.check_counter:Daemon mds.i mds.root_rsnaps=6 2024-09-21T03:56:52.529 INFO:tasks.check_counter:Daemon mds.i mds_server.req_mksnap_latency.avgcount=0 2024-09-21T03:56:52.529 INFO:tasks.check_counter:Daemon mds.i mds_server.req_rmsnap_latency.avgcount=0 2024-09-21T03:56:52.530 DEBUG:tasks.check_counter:Getting stats from l 2024-09-21T03:56:52.530 DEBUG:teuthology.orchestra.run.smithi161:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 51116d5c-77ca-11ef-baf5-efdc52797490 -- ceph daemon mds.l perf dump 2024-09-21T03:56:52.783 INFO:teuthology.orchestra.run.smithi161.stderr:Inferring config /var/lib/ceph/51116d5c-77ca-11ef-baf5-efdc52797490/mon.c/config 2024-09-21T03:56:53.326 INFO:teuthology.orchestra.run.smithi161.stdout:{ 2024-09-21T03:56:53.326 INFO:teuthology.orchestra.run.smithi161.stdout: "AsyncMessenger::Worker-0": { 2024-09-21T03:56:53.326 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_recv_messages": 243, 2024-09-21T03:56:53.326 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_messages": 231, 2024-09-21T03:56:53.326 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_recv_bytes": 1201410, 2024-09-21T03:56:53.326 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_bytes": 134820, 2024-09-21T03:56:53.326 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_created_connections": 5, 2024-09-21T03:56:53.326 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_active_connections": 1, 2024-09-21T03:56:53.326 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_total_time": 0.049581909, 2024-09-21T03:56:53.327 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_send_time": 0.020530688, 2024-09-21T03:56:53.327 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_recv_time": 0.042391895, 2024-09-21T03:56:53.327 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_fast_dispatch_time": 0.000023609, 2024-09-21T03:56:53.327 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T03:56:53.327 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 230, 2024-09-21T03:56:53.327 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.015223801, 2024-09-21T03:56:53.327 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000066190 2024-09-21T03:56:53.327 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T03:56:53.327 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_handle_ack_lat": { 2024-09-21T03:56:53.327 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-09-21T03:56:53.327 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-09-21T03:56:53.327 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-09-21T03:56:53.328 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T03:56:53.328 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_recv_encrypted_bytes": 1200576, 2024-09-21T03:56:53.328 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_encrypted_bytes": 134496 2024-09-21T03:56:53.328 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T03:56:53.328 INFO:teuthology.orchestra.run.smithi161.stdout: "AsyncMessenger::Worker-1": { 2024-09-21T03:56:53.328 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_recv_messages": 655, 2024-09-21T03:56:53.328 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_messages": 817, 2024-09-21T03:56:53.328 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_recv_bytes": 161066, 2024-09-21T03:56:53.328 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_bytes": 572153, 2024-09-21T03:56:53.328 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_created_connections": 5, 2024-09-21T03:56:53.328 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_active_connections": 4, 2024-09-21T03:56:53.329 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_total_time": 0.093584316, 2024-09-21T03:56:53.329 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_send_time": 0.040713712, 2024-09-21T03:56:53.329 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_recv_time": 0.018856175, 2024-09-21T03:56:53.329 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_fast_dispatch_time": 0.026235485, 2024-09-21T03:56:53.329 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T03:56:53.329 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 815, 2024-09-21T03:56:53.329 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.037113487, 2024-09-21T03:56:53.330 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000045538 2024-09-21T03:56:53.330 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T03:56:53.330 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_handle_ack_lat": { 2024-09-21T03:56:53.330 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-09-21T03:56:53.330 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-09-21T03:56:53.330 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-09-21T03:56:53.330 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T03:56:53.330 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_recv_encrypted_bytes": 160, 2024-09-21T03:56:53.330 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_encrypted_bytes": 364176 2024-09-21T03:56:53.330 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T03:56:53.330 INFO:teuthology.orchestra.run.smithi161.stdout: "AsyncMessenger::Worker-2": { 2024-09-21T03:56:53.330 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_recv_messages": 1501, 2024-09-21T03:56:53.331 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_messages": 1501, 2024-09-21T03:56:53.331 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_recv_bytes": 452616, 2024-09-21T03:56:53.331 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_bytes": 476552, 2024-09-21T03:56:53.331 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_created_connections": 4, 2024-09-21T03:56:53.331 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_active_connections": 3, 2024-09-21T03:56:53.331 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_total_time": 0.198060446, 2024-09-21T03:56:53.331 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_send_time": 0.083001763, 2024-09-21T03:56:53.331 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_recv_time": 0.046783764, 2024-09-21T03:56:53.331 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_fast_dispatch_time": 0.054130809, 2024-09-21T03:56:53.331 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T03:56:53.331 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 1500, 2024-09-21T03:56:53.331 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.073209913, 2024-09-21T03:56:53.332 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000048806 2024-09-21T03:56:53.332 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T03:56:53.332 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_handle_ack_lat": { 2024-09-21T03:56:53.332 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-09-21T03:56:53.332 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-09-21T03:56:53.332 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-09-21T03:56:53.332 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T03:56:53.332 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_recv_encrypted_bytes": 3040, 2024-09-21T03:56:53.332 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_encrypted_bytes": 448 2024-09-21T03:56:53.332 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T03:56:53.332 INFO:teuthology.orchestra.run.smithi161.stdout: "cct": { 2024-09-21T03:56:53.332 INFO:teuthology.orchestra.run.smithi161.stdout: "total_workers": 1, 2024-09-21T03:56:53.333 INFO:teuthology.orchestra.run.smithi161.stdout: "unhealthy_workers": 0 2024-09-21T03:56:53.333 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T03:56:53.333 INFO:teuthology.orchestra.run.smithi161.stdout: "finisher-MDSRank": { 2024-09-21T03:56:53.333 INFO:teuthology.orchestra.run.smithi161.stdout: "queue_len": 0, 2024-09-21T03:56:53.333 INFO:teuthology.orchestra.run.smithi161.stdout: "complete_latency": { 2024-09-21T03:56:53.333 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 3575, 2024-09-21T03:56:53.333 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.115891562, 2024-09-21T03:56:53.333 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000032417 2024-09-21T03:56:53.333 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T03:56:53.334 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T03:56:53.334 INFO:teuthology.orchestra.run.smithi161.stdout: "finisher-PurgeQueue": { 2024-09-21T03:56:53.334 INFO:teuthology.orchestra.run.smithi161.stdout: "queue_len": 0, 2024-09-21T03:56:53.334 INFO:teuthology.orchestra.run.smithi161.stdout: "complete_latency": { 2024-09-21T03:56:53.334 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-09-21T03:56:53.334 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-09-21T03:56:53.334 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-09-21T03:56:53.334 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T03:56:53.334 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T03:56:53.334 INFO:teuthology.orchestra.run.smithi161.stdout: "mds": { 2024-09-21T03:56:53.335 INFO:teuthology.orchestra.run.smithi161.stdout: "request": 0, 2024-09-21T03:56:53.335 INFO:teuthology.orchestra.run.smithi161.stdout: "reply": 0, 2024-09-21T03:56:53.335 INFO:teuthology.orchestra.run.smithi161.stdout: "reply_latency": { 2024-09-21T03:56:53.335 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-09-21T03:56:53.335 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-09-21T03:56:53.335 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-09-21T03:56:53.335 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T03:56:53.335 INFO:teuthology.orchestra.run.smithi161.stdout: "slow_reply": 0, 2024-09-21T03:56:53.335 INFO:teuthology.orchestra.run.smithi161.stdout: "forward": 0, 2024-09-21T03:56:53.335 INFO:teuthology.orchestra.run.smithi161.stdout: "dir_fetch_complete": 0, 2024-09-21T03:56:53.335 INFO:teuthology.orchestra.run.smithi161.stdout: "dir_fetch_keys": 0, 2024-09-21T03:56:53.336 INFO:teuthology.orchestra.run.smithi161.stdout: "dir_commit": 0, 2024-09-21T03:56:53.336 INFO:teuthology.orchestra.run.smithi161.stdout: "dir_split": 0, 2024-09-21T03:56:53.336 INFO:teuthology.orchestra.run.smithi161.stdout: "dir_merge": 0, 2024-09-21T03:56:53.336 INFO:teuthology.orchestra.run.smithi161.stdout: "inodes": 0, 2024-09-21T03:56:53.336 INFO:teuthology.orchestra.run.smithi161.stdout: "inodes_top": 0, 2024-09-21T03:56:53.336 INFO:teuthology.orchestra.run.smithi161.stdout: "inodes_bottom": 0, 2024-09-21T03:56:53.336 INFO:teuthology.orchestra.run.smithi161.stdout: "inodes_pin_tail": 0, 2024-09-21T03:56:53.336 INFO:teuthology.orchestra.run.smithi161.stdout: "inodes_pinned": 0, 2024-09-21T03:56:53.337 INFO:teuthology.orchestra.run.smithi161.stdout: "inodes_expired": 0, 2024-09-21T03:56:53.337 INFO:teuthology.orchestra.run.smithi161.stdout: "inodes_with_caps": 0, 2024-09-21T03:56:53.337 INFO:teuthology.orchestra.run.smithi161.stdout: "caps": 0, 2024-09-21T03:56:53.337 INFO:teuthology.orchestra.run.smithi161.stdout: "subtrees": 1, 2024-09-21T03:56:53.337 INFO:teuthology.orchestra.run.smithi161.stdout: "traverse": 0, 2024-09-21T03:56:53.337 INFO:teuthology.orchestra.run.smithi161.stdout: "traverse_hit": 0, 2024-09-21T03:56:53.337 INFO:teuthology.orchestra.run.smithi161.stdout: "traverse_forward": 0, 2024-09-21T03:56:53.337 INFO:teuthology.orchestra.run.smithi161.stdout: "traverse_discover": 0, 2024-09-21T03:56:53.337 INFO:teuthology.orchestra.run.smithi161.stdout: "traverse_dir_fetch": 0, 2024-09-21T03:56:53.337 INFO:teuthology.orchestra.run.smithi161.stdout: "traverse_remote_ino": 0, 2024-09-21T03:56:53.338 INFO:teuthology.orchestra.run.smithi161.stdout: "traverse_lock": 0, 2024-09-21T03:56:53.338 INFO:teuthology.orchestra.run.smithi161.stdout: "load_cent": 0, 2024-09-21T03:56:53.338 INFO:teuthology.orchestra.run.smithi161.stdout: "q": 0, 2024-09-21T03:56:53.338 INFO:teuthology.orchestra.run.smithi161.stdout: "exported": 0, 2024-09-21T03:56:53.338 INFO:teuthology.orchestra.run.smithi161.stdout: "exported_inodes": 0, 2024-09-21T03:56:53.338 INFO:teuthology.orchestra.run.smithi161.stdout: "imported": 0, 2024-09-21T03:56:53.338 INFO:teuthology.orchestra.run.smithi161.stdout: "imported_inodes": 0, 2024-09-21T03:56:53.338 INFO:teuthology.orchestra.run.smithi161.stdout: "openino_dir_fetch": 0, 2024-09-21T03:56:53.338 INFO:teuthology.orchestra.run.smithi161.stdout: "openino_backtrace_fetch": 0, 2024-09-21T03:56:53.338 INFO:teuthology.orchestra.run.smithi161.stdout: "openino_peer_discover": 0, 2024-09-21T03:56:53.338 INFO:teuthology.orchestra.run.smithi161.stdout: "root_rfiles": 0, 2024-09-21T03:56:53.339 INFO:teuthology.orchestra.run.smithi161.stdout: "root_rbytes": 0, 2024-09-21T03:56:53.339 INFO:teuthology.orchestra.run.smithi161.stdout: "root_rsnaps": 0, 2024-09-21T03:56:53.339 INFO:teuthology.orchestra.run.smithi161.stdout: "scrub_backtrace_fetch": 0, 2024-09-21T03:56:53.339 INFO:teuthology.orchestra.run.smithi161.stdout: "scrub_set_tag": 0, 2024-09-21T03:56:53.339 INFO:teuthology.orchestra.run.smithi161.stdout: "scrub_backtrace_repaired": 0, 2024-09-21T03:56:53.339 INFO:teuthology.orchestra.run.smithi161.stdout: "scrub_inotable_repaired": 0, 2024-09-21T03:56:53.339 INFO:teuthology.orchestra.run.smithi161.stdout: "scrub_dir_inodes": 0, 2024-09-21T03:56:53.339 INFO:teuthology.orchestra.run.smithi161.stdout: "scrub_dir_base_inodes": 0, 2024-09-21T03:56:53.340 INFO:teuthology.orchestra.run.smithi161.stdout: "scrub_dirfrag_rstats": 0, 2024-09-21T03:56:53.340 INFO:teuthology.orchestra.run.smithi161.stdout: "scrub_file_inodes": 0, 2024-09-21T03:56:53.340 INFO:teuthology.orchestra.run.smithi161.stdout: "handle_inode_file_caps": 0, 2024-09-21T03:56:53.340 INFO:teuthology.orchestra.run.smithi161.stdout: "ceph_cap_op_revoke": 0, 2024-09-21T03:56:53.340 INFO:teuthology.orchestra.run.smithi161.stdout: "ceph_cap_op_grant": 0, 2024-09-21T03:56:53.340 INFO:teuthology.orchestra.run.smithi161.stdout: "ceph_cap_op_trunc": 0, 2024-09-21T03:56:53.340 INFO:teuthology.orchestra.run.smithi161.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-21T03:56:53.340 INFO:teuthology.orchestra.run.smithi161.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-21T03:56:53.340 INFO:teuthology.orchestra.run.smithi161.stdout: "handle_client_caps": 0, 2024-09-21T03:56:53.341 INFO:teuthology.orchestra.run.smithi161.stdout: "handle_client_caps_dirty": 0, 2024-09-21T03:56:53.341 INFO:teuthology.orchestra.run.smithi161.stdout: "handle_client_cap_release": 0, 2024-09-21T03:56:53.341 INFO:teuthology.orchestra.run.smithi161.stdout: "process_request_cap_release": 0 2024-09-21T03:56:53.341 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T03:56:53.341 INFO:teuthology.orchestra.run.smithi161.stdout: "mds_cache": { 2024-09-21T03:56:53.341 INFO:teuthology.orchestra.run.smithi161.stdout: "dir_update": 0, 2024-09-21T03:56:53.341 INFO:teuthology.orchestra.run.smithi161.stdout: "dir_update_receipt": 0, 2024-09-21T03:56:53.341 INFO:teuthology.orchestra.run.smithi161.stdout: "dir_try_discover": 0, 2024-09-21T03:56:53.341 INFO:teuthology.orchestra.run.smithi161.stdout: "dir_send_discover": 0, 2024-09-21T03:56:53.341 INFO:teuthology.orchestra.run.smithi161.stdout: "dir_handle_discover": 0, 2024-09-21T03:56:53.342 INFO:teuthology.orchestra.run.smithi161.stdout: "num_strays": 0, 2024-09-21T03:56:53.342 INFO:teuthology.orchestra.run.smithi161.stdout: "num_strays_delayed": 0, 2024-09-21T03:56:53.342 INFO:teuthology.orchestra.run.smithi161.stdout: "num_strays_enqueuing": 0, 2024-09-21T03:56:53.342 INFO:teuthology.orchestra.run.smithi161.stdout: "strays_created": 0, 2024-09-21T03:56:53.342 INFO:teuthology.orchestra.run.smithi161.stdout: "strays_enqueued": 0, 2024-09-21T03:56:53.342 INFO:teuthology.orchestra.run.smithi161.stdout: "strays_reintegrated": 0, 2024-09-21T03:56:53.342 INFO:teuthology.orchestra.run.smithi161.stdout: "strays_migrated": 0, 2024-09-21T03:56:53.342 INFO:teuthology.orchestra.run.smithi161.stdout: "num_recovering_processing": 0, 2024-09-21T03:56:53.343 INFO:teuthology.orchestra.run.smithi161.stdout: "num_recovering_enqueued": 0, 2024-09-21T03:56:53.343 INFO:teuthology.orchestra.run.smithi161.stdout: "num_recovering_prioritized": 0, 2024-09-21T03:56:53.343 INFO:teuthology.orchestra.run.smithi161.stdout: "recovery_started": 0, 2024-09-21T03:56:53.343 INFO:teuthology.orchestra.run.smithi161.stdout: "recovery_completed": 0, 2024-09-21T03:56:53.343 INFO:teuthology.orchestra.run.smithi161.stdout: "ireq_quiesce_path": 0, 2024-09-21T03:56:53.343 INFO:teuthology.orchestra.run.smithi161.stdout: "ireq_quiesce_inode": 0, 2024-09-21T03:56:53.343 INFO:teuthology.orchestra.run.smithi161.stdout: "ireq_enqueue_scrub": 0, 2024-09-21T03:56:53.343 INFO:teuthology.orchestra.run.smithi161.stdout: "ireq_exportdir": 0, 2024-09-21T03:56:53.343 INFO:teuthology.orchestra.run.smithi161.stdout: "ireq_flush": 0, 2024-09-21T03:56:53.343 INFO:teuthology.orchestra.run.smithi161.stdout: "ireq_fragmentdir": 0, 2024-09-21T03:56:53.343 INFO:teuthology.orchestra.run.smithi161.stdout: "ireq_fragstats": 0, 2024-09-21T03:56:53.344 INFO:teuthology.orchestra.run.smithi161.stdout: "ireq_inodestats": 0, 2024-09-21T03:56:53.344 INFO:teuthology.orchestra.run.smithi161.stdout: "uninline_started": 0, 2024-09-21T03:56:53.344 INFO:teuthology.orchestra.run.smithi161.stdout: "uninline_succeeded": 0, 2024-09-21T03:56:53.344 INFO:teuthology.orchestra.run.smithi161.stdout: "uninline_write_failed": 0 2024-09-21T03:56:53.344 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T03:56:53.344 INFO:teuthology.orchestra.run.smithi161.stdout: "mds_log": { 2024-09-21T03:56:53.344 INFO:teuthology.orchestra.run.smithi161.stdout: "evlrg": 0, 2024-09-21T03:56:53.344 INFO:teuthology.orchestra.run.smithi161.stdout: "evadd": 0, 2024-09-21T03:56:53.344 INFO:teuthology.orchestra.run.smithi161.stdout: "evex": 0, 2024-09-21T03:56:53.344 INFO:teuthology.orchestra.run.smithi161.stdout: "evtrm": 0, 2024-09-21T03:56:53.344 INFO:teuthology.orchestra.run.smithi161.stdout: "ev": 8, 2024-09-21T03:56:53.344 INFO:teuthology.orchestra.run.smithi161.stdout: "evexg": 0, 2024-09-21T03:56:53.344 INFO:teuthology.orchestra.run.smithi161.stdout: "evexd": 0, 2024-09-21T03:56:53.345 INFO:teuthology.orchestra.run.smithi161.stdout: "segadd": 0, 2024-09-21T03:56:53.345 INFO:teuthology.orchestra.run.smithi161.stdout: "segex": 0, 2024-09-21T03:56:53.345 INFO:teuthology.orchestra.run.smithi161.stdout: "segtrm": 0, 2024-09-21T03:56:53.345 INFO:teuthology.orchestra.run.smithi161.stdout: "seg": 2, 2024-09-21T03:56:53.345 INFO:teuthology.orchestra.run.smithi161.stdout: "segmjr": 2, 2024-09-21T03:56:53.345 INFO:teuthology.orchestra.run.smithi161.stdout: "segexg": 0, 2024-09-21T03:56:53.345 INFO:teuthology.orchestra.run.smithi161.stdout: "segexd": 0, 2024-09-21T03:56:53.345 INFO:teuthology.orchestra.run.smithi161.stdout: "expos": 4194304, 2024-09-21T03:56:53.345 INFO:teuthology.orchestra.run.smithi161.stdout: "wrpos": 4196655, 2024-09-21T03:56:53.345 INFO:teuthology.orchestra.run.smithi161.stdout: "rdpos": 4196514, 2024-09-21T03:56:53.345 INFO:teuthology.orchestra.run.smithi161.stdout: "jlat": { 2024-09-21T03:56:53.346 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-09-21T03:56:53.346 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-09-21T03:56:53.346 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-09-21T03:56:53.346 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T03:56:53.346 INFO:teuthology.orchestra.run.smithi161.stdout: "replayed": 8 2024-09-21T03:56:53.346 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T03:56:53.346 INFO:teuthology.orchestra.run.smithi161.stdout: "mds_mem": { 2024-09-21T03:56:53.346 INFO:teuthology.orchestra.run.smithi161.stdout: "ino": 3, 2024-09-21T03:56:53.346 INFO:teuthology.orchestra.run.smithi161.stdout: "ino+": 3, 2024-09-21T03:56:53.346 INFO:teuthology.orchestra.run.smithi161.stdout: "ino-": 0, 2024-09-21T03:56:53.346 INFO:teuthology.orchestra.run.smithi161.stdout: "dir": 1, 2024-09-21T03:56:53.347 INFO:teuthology.orchestra.run.smithi161.stdout: "dir+": 1, 2024-09-21T03:56:53.347 INFO:teuthology.orchestra.run.smithi161.stdout: "dir-": 0, 2024-09-21T03:56:53.347 INFO:teuthology.orchestra.run.smithi161.stdout: "dn": 0, 2024-09-21T03:56:53.347 INFO:teuthology.orchestra.run.smithi161.stdout: "dn+": 0, 2024-09-21T03:56:53.347 INFO:teuthology.orchestra.run.smithi161.stdout: "dn-": 0, 2024-09-21T03:56:53.347 INFO:teuthology.orchestra.run.smithi161.stdout: "cap": 0, 2024-09-21T03:56:53.347 INFO:teuthology.orchestra.run.smithi161.stdout: "cap+": 0, 2024-09-21T03:56:53.347 INFO:teuthology.orchestra.run.smithi161.stdout: "cap-": 0, 2024-09-21T03:56:53.347 INFO:teuthology.orchestra.run.smithi161.stdout: "rss": 117600, 2024-09-21T03:56:53.347 INFO:teuthology.orchestra.run.smithi161.stdout: "heap": 316416000 2024-09-21T03:56:53.347 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T03:56:53.347 INFO:teuthology.orchestra.run.smithi161.stdout: "mds_server": { 2024-09-21T03:56:53.347 INFO:teuthology.orchestra.run.smithi161.stdout: "dispatch_client_request": 0, 2024-09-21T03:56:53.348 INFO:teuthology.orchestra.run.smithi161.stdout: "dispatch_server_request": 0, 2024-09-21T03:56:53.348 INFO:teuthology.orchestra.run.smithi161.stdout: "handle_client_request": 0, 2024-09-21T03:56:53.348 INFO:teuthology.orchestra.run.smithi161.stdout: "handle_client_session": 0, 2024-09-21T03:56:53.348 INFO:teuthology.orchestra.run.smithi161.stdout: "handle_peer_request": 0, 2024-09-21T03:56:53.348 INFO:teuthology.orchestra.run.smithi161.stdout: "req_create_latency": { 2024-09-21T03:56:53.348 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-09-21T03:56:53.348 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-09-21T03:56:53.348 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-09-21T03:56:53.348 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T03:56:53.348 INFO:teuthology.orchestra.run.smithi161.stdout: "req_getattr_latency": { 2024-09-21T03:56:53.348 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-09-21T03:56:53.348 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-09-21T03:56:53.349 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-09-21T03:56:53.349 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T03:56:53.349 INFO:teuthology.orchestra.run.smithi161.stdout: "req_getfilelock_latency": { 2024-09-21T03:56:53.349 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-09-21T03:56:53.349 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-09-21T03:56:53.349 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-09-21T03:56:53.349 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T03:56:53.349 INFO:teuthology.orchestra.run.smithi161.stdout: "req_link_latency": { 2024-09-21T03:56:53.349 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-09-21T03:56:53.349 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-09-21T03:56:53.349 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-09-21T03:56:53.349 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T03:56:53.349 INFO:teuthology.orchestra.run.smithi161.stdout: "req_lookup_latency": { 2024-09-21T03:56:53.349 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-09-21T03:56:53.350 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-09-21T03:56:53.350 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-09-21T03:56:53.350 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T03:56:53.350 INFO:teuthology.orchestra.run.smithi161.stdout: "req_lookuphash_latency": { 2024-09-21T03:56:53.350 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-09-21T03:56:53.350 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-09-21T03:56:53.350 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-09-21T03:56:53.350 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T03:56:53.350 INFO:teuthology.orchestra.run.smithi161.stdout: "req_lookupino_latency": { 2024-09-21T03:56:53.350 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-09-21T03:56:53.350 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-09-21T03:56:53.350 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-09-21T03:56:53.350 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T03:56:53.351 INFO:teuthology.orchestra.run.smithi161.stdout: "req_lookupname_latency": { 2024-09-21T03:56:53.351 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-09-21T03:56:53.351 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-09-21T03:56:53.351 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-09-21T03:56:53.351 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T03:56:53.351 INFO:teuthology.orchestra.run.smithi161.stdout: "req_lookupparent_latency": { 2024-09-21T03:56:53.351 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-09-21T03:56:53.351 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-09-21T03:56:53.351 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-09-21T03:56:53.351 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T03:56:53.351 INFO:teuthology.orchestra.run.smithi161.stdout: "req_lookupsnap_latency": { 2024-09-21T03:56:53.351 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-09-21T03:56:53.352 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-09-21T03:56:53.352 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-09-21T03:56:53.352 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T03:56:53.352 INFO:teuthology.orchestra.run.smithi161.stdout: "req_lssnap_latency": { 2024-09-21T03:56:53.352 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-09-21T03:56:53.352 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-09-21T03:56:53.353 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-09-21T03:56:53.353 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T03:56:53.353 INFO:teuthology.orchestra.run.smithi161.stdout: "req_mkdir_latency": { 2024-09-21T03:56:53.353 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-09-21T03:56:53.353 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-09-21T03:56:53.353 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-09-21T03:56:53.353 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T03:56:53.353 INFO:teuthology.orchestra.run.smithi161.stdout: "req_mknod_latency": { 2024-09-21T03:56:53.353 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-09-21T03:56:53.353 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-09-21T03:56:53.353 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-09-21T03:56:53.354 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T03:56:53.354 INFO:teuthology.orchestra.run.smithi161.stdout: "req_mksnap_latency": { 2024-09-21T03:56:53.354 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-09-21T03:56:53.354 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-09-21T03:56:53.354 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-09-21T03:56:53.354 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T03:56:53.354 INFO:teuthology.orchestra.run.smithi161.stdout: "req_open_latency": { 2024-09-21T03:56:53.354 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-09-21T03:56:53.354 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-09-21T03:56:53.354 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-09-21T03:56:53.354 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T03:56:53.354 INFO:teuthology.orchestra.run.smithi161.stdout: "req_readdir_latency": { 2024-09-21T03:56:53.355 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-09-21T03:56:53.355 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-09-21T03:56:53.355 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-09-21T03:56:53.355 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T03:56:53.355 INFO:teuthology.orchestra.run.smithi161.stdout: "req_rename_latency": { 2024-09-21T03:56:53.355 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-09-21T03:56:53.355 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-09-21T03:56:53.355 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-09-21T03:56:53.355 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T03:56:53.355 INFO:teuthology.orchestra.run.smithi161.stdout: "req_renamesnap_latency": { 2024-09-21T03:56:53.355 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-09-21T03:56:53.356 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-09-21T03:56:53.356 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-09-21T03:56:53.356 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T03:56:53.356 INFO:teuthology.orchestra.run.smithi161.stdout: "req_snapdiff_latency": { 2024-09-21T03:56:53.356 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-09-21T03:56:53.356 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-09-21T03:56:53.356 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-09-21T03:56:53.356 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T03:56:53.356 INFO:teuthology.orchestra.run.smithi161.stdout: "req_rmdir_latency": { 2024-09-21T03:56:53.356 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-09-21T03:56:53.356 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-09-21T03:56:53.356 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-09-21T03:56:53.357 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T03:56:53.357 INFO:teuthology.orchestra.run.smithi161.stdout: "req_rmsnap_latency": { 2024-09-21T03:56:53.357 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-09-21T03:56:53.357 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-09-21T03:56:53.357 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-09-21T03:56:53.357 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T03:56:53.357 INFO:teuthology.orchestra.run.smithi161.stdout: "req_rmxattr_latency": { 2024-09-21T03:56:53.357 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-09-21T03:56:53.357 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-09-21T03:56:53.357 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-09-21T03:56:53.357 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T03:56:53.357 INFO:teuthology.orchestra.run.smithi161.stdout: "req_setattr_latency": { 2024-09-21T03:56:53.358 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-09-21T03:56:53.358 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-09-21T03:56:53.358 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-09-21T03:56:53.358 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T03:56:53.358 INFO:teuthology.orchestra.run.smithi161.stdout: "req_setdirlayout_latency": { 2024-09-21T03:56:53.358 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-09-21T03:56:53.358 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-09-21T03:56:53.358 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-09-21T03:56:53.358 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T03:56:53.358 INFO:teuthology.orchestra.run.smithi161.stdout: "req_setfilelock_latency": { 2024-09-21T03:56:53.358 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-09-21T03:56:53.358 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-09-21T03:56:53.359 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-09-21T03:56:53.359 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T03:56:53.359 INFO:teuthology.orchestra.run.smithi161.stdout: "req_setlayout_latency": { 2024-09-21T03:56:53.359 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-09-21T03:56:53.359 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-09-21T03:56:53.359 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-09-21T03:56:53.359 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T03:56:53.359 INFO:teuthology.orchestra.run.smithi161.stdout: "req_setxattr_latency": { 2024-09-21T03:56:53.359 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-09-21T03:56:53.359 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-09-21T03:56:53.359 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-09-21T03:56:53.360 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T03:56:53.360 INFO:teuthology.orchestra.run.smithi161.stdout: "req_symlink_latency": { 2024-09-21T03:56:53.360 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-09-21T03:56:53.360 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-09-21T03:56:53.360 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-09-21T03:56:53.360 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T03:56:53.360 INFO:teuthology.orchestra.run.smithi161.stdout: "req_unlink_latency": { 2024-09-21T03:56:53.360 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-09-21T03:56:53.360 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-09-21T03:56:53.360 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-09-21T03:56:53.360 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T03:56:53.360 INFO:teuthology.orchestra.run.smithi161.stdout: "cap_revoke_eviction": 0, 2024-09-21T03:56:53.360 INFO:teuthology.orchestra.run.smithi161.stdout: "cap_acquisition_throttle": 0, 2024-09-21T03:56:53.361 INFO:teuthology.orchestra.run.smithi161.stdout: "req_getvxattr_latency": { 2024-09-21T03:56:53.361 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-09-21T03:56:53.361 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-09-21T03:56:53.361 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-09-21T03:56:53.361 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T03:56:53.361 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T03:56:53.361 INFO:teuthology.orchestra.run.smithi161.stdout: "mds_sessions": { 2024-09-21T03:56:53.361 INFO:teuthology.orchestra.run.smithi161.stdout: "session_count": 2, 2024-09-21T03:56:53.361 INFO:teuthology.orchestra.run.smithi161.stdout: "session_add": 4, 2024-09-21T03:56:53.361 INFO:teuthology.orchestra.run.smithi161.stdout: "session_remove": 2, 2024-09-21T03:56:53.361 INFO:teuthology.orchestra.run.smithi161.stdout: "sessions_open": 3, 2024-09-21T03:56:53.361 INFO:teuthology.orchestra.run.smithi161.stdout: "sessions_stale": 0, 2024-09-21T03:56:53.362 INFO:teuthology.orchestra.run.smithi161.stdout: "total_load": 0, 2024-09-21T03:56:53.362 INFO:teuthology.orchestra.run.smithi161.stdout: "average_load": 0, 2024-09-21T03:56:53.362 INFO:teuthology.orchestra.run.smithi161.stdout: "avg_session_uptime": 2870, 2024-09-21T03:56:53.362 INFO:teuthology.orchestra.run.smithi161.stdout: "mdthresh_evicted": 0 2024-09-21T03:56:53.362 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T03:56:53.362 INFO:teuthology.orchestra.run.smithi161.stdout: "mempool": { 2024-09-21T03:56:53.362 INFO:teuthology.orchestra.run.smithi161.stdout: "bloom_filter_bytes": 0, 2024-09-21T03:56:53.362 INFO:teuthology.orchestra.run.smithi161.stdout: "bloom_filter_items": 0, 2024-09-21T03:56:53.362 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_alloc_bytes": 0, 2024-09-21T03:56:53.362 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_alloc_items": 0, 2024-09-21T03:56:53.362 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_cache_data_bytes": 0, 2024-09-21T03:56:53.362 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_cache_data_items": 0, 2024-09-21T03:56:53.363 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-21T03:56:53.363 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_cache_onode_items": 0, 2024-09-21T03:56:53.363 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-21T03:56:53.363 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_cache_meta_items": 0, 2024-09-21T03:56:53.363 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_cache_other_bytes": 0, 2024-09-21T03:56:53.363 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_cache_other_items": 0, 2024-09-21T03:56:53.363 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-21T03:56:53.363 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_cache_buffer_items": 0, 2024-09-21T03:56:53.364 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_extent_bytes": 0, 2024-09-21T03:56:53.364 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_extent_items": 0, 2024-09-21T03:56:53.364 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_blob_bytes": 0, 2024-09-21T03:56:53.364 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_blob_items": 0, 2024-09-21T03:56:53.364 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-21T03:56:53.364 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_shared_blob_items": 0, 2024-09-21T03:56:53.364 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-21T03:56:53.364 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_inline_bl_items": 0, 2024-09-21T03:56:53.364 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_fsck_bytes": 0, 2024-09-21T03:56:53.364 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_fsck_items": 0, 2024-09-21T03:56:53.364 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_txc_bytes": 0, 2024-09-21T03:56:53.365 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_txc_items": 0, 2024-09-21T03:56:53.365 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-21T03:56:53.365 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_writing_deferred_items": 0, 2024-09-21T03:56:53.365 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_writing_bytes": 0, 2024-09-21T03:56:53.365 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_writing_items": 0, 2024-09-21T03:56:53.365 INFO:teuthology.orchestra.run.smithi161.stdout: "bluefs_bytes": 0, 2024-09-21T03:56:53.365 INFO:teuthology.orchestra.run.smithi161.stdout: "bluefs_items": 0, 2024-09-21T03:56:53.365 INFO:teuthology.orchestra.run.smithi161.stdout: "bluefs_file_reader_bytes": 0, 2024-09-21T03:56:53.365 INFO:teuthology.orchestra.run.smithi161.stdout: "bluefs_file_reader_items": 0, 2024-09-21T03:56:53.365 INFO:teuthology.orchestra.run.smithi161.stdout: "bluefs_file_writer_bytes": 0, 2024-09-21T03:56:53.365 INFO:teuthology.orchestra.run.smithi161.stdout: "bluefs_file_writer_items": 0, 2024-09-21T03:56:53.366 INFO:teuthology.orchestra.run.smithi161.stdout: "buffer_anon_bytes": 189423, 2024-09-21T03:56:53.366 INFO:teuthology.orchestra.run.smithi161.stdout: "buffer_anon_items": 51, 2024-09-21T03:56:53.366 INFO:teuthology.orchestra.run.smithi161.stdout: "buffer_meta_bytes": 0, 2024-09-21T03:56:53.366 INFO:teuthology.orchestra.run.smithi161.stdout: "buffer_meta_items": 0, 2024-09-21T03:56:53.366 INFO:teuthology.orchestra.run.smithi161.stdout: "osd_bytes": 0, 2024-09-21T03:56:53.366 INFO:teuthology.orchestra.run.smithi161.stdout: "osd_items": 0, 2024-09-21T03:56:53.366 INFO:teuthology.orchestra.run.smithi161.stdout: "osd_mapbl_bytes": 0, 2024-09-21T03:56:53.366 INFO:teuthology.orchestra.run.smithi161.stdout: "osd_mapbl_items": 0, 2024-09-21T03:56:53.366 INFO:teuthology.orchestra.run.smithi161.stdout: "osd_pglog_bytes": 0, 2024-09-21T03:56:53.366 INFO:teuthology.orchestra.run.smithi161.stdout: "osd_pglog_items": 0, 2024-09-21T03:56:53.367 INFO:teuthology.orchestra.run.smithi161.stdout: "osdmap_bytes": 8256, 2024-09-21T03:56:53.367 INFO:teuthology.orchestra.run.smithi161.stdout: "osdmap_items": 182, 2024-09-21T03:56:53.367 INFO:teuthology.orchestra.run.smithi161.stdout: "osdmap_mapping_bytes": 0, 2024-09-21T03:56:53.367 INFO:teuthology.orchestra.run.smithi161.stdout: "osdmap_mapping_items": 0, 2024-09-21T03:56:53.367 INFO:teuthology.orchestra.run.smithi161.stdout: "pgmap_bytes": 0, 2024-09-21T03:56:53.367 INFO:teuthology.orchestra.run.smithi161.stdout: "pgmap_items": 0, 2024-09-21T03:56:53.367 INFO:teuthology.orchestra.run.smithi161.stdout: "mds_co_bytes": 8672, 2024-09-21T03:56:53.367 INFO:teuthology.orchestra.run.smithi161.stdout: "mds_co_items": 13, 2024-09-21T03:56:53.367 INFO:teuthology.orchestra.run.smithi161.stdout: "unittest_1_bytes": 0, 2024-09-21T03:56:53.367 INFO:teuthology.orchestra.run.smithi161.stdout: "unittest_1_items": 0, 2024-09-21T03:56:53.367 INFO:teuthology.orchestra.run.smithi161.stdout: "unittest_2_bytes": 0, 2024-09-21T03:56:53.367 INFO:teuthology.orchestra.run.smithi161.stdout: "unittest_2_items": 0 2024-09-21T03:56:53.368 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T03:56:53.368 INFO:teuthology.orchestra.run.smithi161.stdout: "objecter": { 2024-09-21T03:56:53.368 INFO:teuthology.orchestra.run.smithi161.stdout: "op_active": 0, 2024-09-21T03:56:53.368 INFO:teuthology.orchestra.run.smithi161.stdout: "op_laggy": 0, 2024-09-21T03:56:53.368 INFO:teuthology.orchestra.run.smithi161.stdout: "op_send": 2152, 2024-09-21T03:56:53.368 INFO:teuthology.orchestra.run.smithi161.stdout: "op_send_bytes": 21, 2024-09-21T03:56:53.368 INFO:teuthology.orchestra.run.smithi161.stdout: "op_resend": 3, 2024-09-21T03:56:53.368 INFO:teuthology.orchestra.run.smithi161.stdout: "op_reply": 2152, 2024-09-21T03:56:53.369 INFO:teuthology.orchestra.run.smithi161.stdout: "op_latency": { 2024-09-21T03:56:53.369 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 2152, 2024-09-21T03:56:53.369 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 4.581902446, 2024-09-21T03:56:53.369 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.002129136 2024-09-21T03:56:53.369 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T03:56:53.369 INFO:teuthology.orchestra.run.smithi161.stdout: "op_inflight": 0, 2024-09-21T03:56:53.369 INFO:teuthology.orchestra.run.smithi161.stdout: "oplen_avg": { 2024-09-21T03:56:53.369 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 2152, 2024-09-21T03:56:53.369 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 2153 2024-09-21T03:56:53.369 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T03:56:53.369 INFO:teuthology.orchestra.run.smithi161.stdout: "op": 2152, 2024-09-21T03:56:53.370 INFO:teuthology.orchestra.run.smithi161.stdout: "op_r": 2152, 2024-09-21T03:56:53.370 INFO:teuthology.orchestra.run.smithi161.stdout: "op_w": 0, 2024-09-21T03:56:53.370 INFO:teuthology.orchestra.run.smithi161.stdout: "op_rmw": 0, 2024-09-21T03:56:53.370 INFO:teuthology.orchestra.run.smithi161.stdout: "op_pg": 0, 2024-09-21T03:56:53.370 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_stat": 1396, 2024-09-21T03:56:53.370 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_create": 0, 2024-09-21T03:56:53.370 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_read": 754, 2024-09-21T03:56:53.370 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_write": 0, 2024-09-21T03:56:53.370 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_writefull": 0, 2024-09-21T03:56:53.370 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_writesame": 0, 2024-09-21T03:56:53.371 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_append": 0, 2024-09-21T03:56:53.371 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_zero": 0, 2024-09-21T03:56:53.371 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_truncate": 0, 2024-09-21T03:56:53.371 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_delete": 0, 2024-09-21T03:56:53.371 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_mapext": 0, 2024-09-21T03:56:53.371 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_sparse_read": 0, 2024-09-21T03:56:53.371 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_clonerange": 0, 2024-09-21T03:56:53.371 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_getxattr": 1, 2024-09-21T03:56:53.371 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_setxattr": 0, 2024-09-21T03:56:53.371 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_cmpxattr": 0, 2024-09-21T03:56:53.372 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_rmxattr": 0, 2024-09-21T03:56:53.372 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_resetxattrs": 0, 2024-09-21T03:56:53.372 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_call": 0, 2024-09-21T03:56:53.372 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_watch": 0, 2024-09-21T03:56:53.372 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_notify": 0, 2024-09-21T03:56:53.372 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_src_cmpxattr": 0, 2024-09-21T03:56:53.372 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_pgls": 0, 2024-09-21T03:56:53.372 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_pgls_filter": 0, 2024-09-21T03:56:53.372 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_other": 0, 2024-09-21T03:56:53.373 INFO:teuthology.orchestra.run.smithi161.stdout: "linger_active": 0, 2024-09-21T03:56:53.373 INFO:teuthology.orchestra.run.smithi161.stdout: "linger_send": 0, 2024-09-21T03:56:53.373 INFO:teuthology.orchestra.run.smithi161.stdout: "linger_resend": 0, 2024-09-21T03:56:53.373 INFO:teuthology.orchestra.run.smithi161.stdout: "linger_ping": 0, 2024-09-21T03:56:53.373 INFO:teuthology.orchestra.run.smithi161.stdout: "poolop_active": 0, 2024-09-21T03:56:53.373 INFO:teuthology.orchestra.run.smithi161.stdout: "poolop_send": 0, 2024-09-21T03:56:53.373 INFO:teuthology.orchestra.run.smithi161.stdout: "poolop_resend": 0, 2024-09-21T03:56:53.373 INFO:teuthology.orchestra.run.smithi161.stdout: "poolstat_active": 0, 2024-09-21T03:56:53.373 INFO:teuthology.orchestra.run.smithi161.stdout: "poolstat_send": 0, 2024-09-21T03:56:53.373 INFO:teuthology.orchestra.run.smithi161.stdout: "poolstat_resend": 0, 2024-09-21T03:56:53.373 INFO:teuthology.orchestra.run.smithi161.stdout: "statfs_active": 0, 2024-09-21T03:56:53.374 INFO:teuthology.orchestra.run.smithi161.stdout: "statfs_send": 0, 2024-09-21T03:56:53.374 INFO:teuthology.orchestra.run.smithi161.stdout: "statfs_resend": 0, 2024-09-21T03:56:53.374 INFO:teuthology.orchestra.run.smithi161.stdout: "command_active": 0, 2024-09-21T03:56:53.374 INFO:teuthology.orchestra.run.smithi161.stdout: "command_send": 0, 2024-09-21T03:56:53.374 INFO:teuthology.orchestra.run.smithi161.stdout: "command_resend": 0, 2024-09-21T03:56:53.374 INFO:teuthology.orchestra.run.smithi161.stdout: "map_epoch": 87, 2024-09-21T03:56:53.374 INFO:teuthology.orchestra.run.smithi161.stdout: "map_full": 0, 2024-09-21T03:56:53.374 INFO:teuthology.orchestra.run.smithi161.stdout: "map_inc": 10, 2024-09-21T03:56:53.375 INFO:teuthology.orchestra.run.smithi161.stdout: "osd_sessions": 5, 2024-09-21T03:56:53.375 INFO:teuthology.orchestra.run.smithi161.stdout: "osd_session_open": 5, 2024-09-21T03:56:53.375 INFO:teuthology.orchestra.run.smithi161.stdout: "osd_session_close": 0, 2024-09-21T03:56:53.375 INFO:teuthology.orchestra.run.smithi161.stdout: "osd_laggy": 0, 2024-09-21T03:56:53.375 INFO:teuthology.orchestra.run.smithi161.stdout: "omap_wr": 0, 2024-09-21T03:56:53.375 INFO:teuthology.orchestra.run.smithi161.stdout: "omap_rd": 2, 2024-09-21T03:56:53.375 INFO:teuthology.orchestra.run.smithi161.stdout: "omap_del": 0 2024-09-21T03:56:53.375 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T03:56:53.375 INFO:teuthology.orchestra.run.smithi161.stdout: "oft": { 2024-09-21T03:56:53.376 INFO:teuthology.orchestra.run.smithi161.stdout: "omap_total_objs": 0, 2024-09-21T03:56:53.376 INFO:teuthology.orchestra.run.smithi161.stdout: "omap_total_kv_pairs": 0, 2024-09-21T03:56:53.376 INFO:teuthology.orchestra.run.smithi161.stdout: "omap_total_updates": 0, 2024-09-21T03:56:53.376 INFO:teuthology.orchestra.run.smithi161.stdout: "omap_total_removes": 0 2024-09-21T03:56:53.376 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T03:56:53.376 INFO:teuthology.orchestra.run.smithi161.stdout: "purge_queue": { 2024-09-21T03:56:53.376 INFO:teuthology.orchestra.run.smithi161.stdout: "pq_executing_ops": 0, 2024-09-21T03:56:53.376 INFO:teuthology.orchestra.run.smithi161.stdout: "pq_executing_ops_high_water": 0, 2024-09-21T03:56:53.376 INFO:teuthology.orchestra.run.smithi161.stdout: "pq_executing": 0, 2024-09-21T03:56:53.376 INFO:teuthology.orchestra.run.smithi161.stdout: "pq_executing_high_water": 0, 2024-09-21T03:56:53.376 INFO:teuthology.orchestra.run.smithi161.stdout: "pq_executed_ops": 0, 2024-09-21T03:56:53.377 INFO:teuthology.orchestra.run.smithi161.stdout: "pq_executed": 0, 2024-09-21T03:56:53.377 INFO:teuthology.orchestra.run.smithi161.stdout: "pq_item_in_journal": 0 2024-09-21T03:56:53.377 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T03:56:53.377 INFO:teuthology.orchestra.run.smithi161.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-21T03:56:53.377 INFO:teuthology.orchestra.run.smithi161.stdout: "val": 0, 2024-09-21T03:56:53.377 INFO:teuthology.orchestra.run.smithi161.stdout: "max": 104857600, 2024-09-21T03:56:53.377 INFO:teuthology.orchestra.run.smithi161.stdout: "get_started": 0, 2024-09-21T03:56:53.377 INFO:teuthology.orchestra.run.smithi161.stdout: "get": 2395, 2024-09-21T03:56:53.377 INFO:teuthology.orchestra.run.smithi161.stdout: "get_sum": 1584097, 2024-09-21T03:56:53.377 INFO:teuthology.orchestra.run.smithi161.stdout: "get_or_fail_fail": 0, 2024-09-21T03:56:53.377 INFO:teuthology.orchestra.run.smithi161.stdout: "get_or_fail_success": 2395, 2024-09-21T03:56:53.377 INFO:teuthology.orchestra.run.smithi161.stdout: "take": 0, 2024-09-21T03:56:53.377 INFO:teuthology.orchestra.run.smithi161.stdout: "take_sum": 0, 2024-09-21T03:56:53.378 INFO:teuthology.orchestra.run.smithi161.stdout: "put": 2395, 2024-09-21T03:56:53.378 INFO:teuthology.orchestra.run.smithi161.stdout: "put_sum": 1584097, 2024-09-21T03:56:53.378 INFO:teuthology.orchestra.run.smithi161.stdout: "wait": { 2024-09-21T03:56:53.378 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-09-21T03:56:53.378 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-09-21T03:56:53.378 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-09-21T03:56:53.378 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T03:56:53.378 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T03:56:53.378 INFO:teuthology.orchestra.run.smithi161.stdout: "throttle-objecter_bytes": { 2024-09-21T03:56:53.378 INFO:teuthology.orchestra.run.smithi161.stdout: "val": 0, 2024-09-21T03:56:53.378 INFO:teuthology.orchestra.run.smithi161.stdout: "max": 104857600, 2024-09-21T03:56:53.378 INFO:teuthology.orchestra.run.smithi161.stdout: "get_started": 0, 2024-09-21T03:56:53.379 INFO:teuthology.orchestra.run.smithi161.stdout: "get": 0, 2024-09-21T03:56:53.379 INFO:teuthology.orchestra.run.smithi161.stdout: "get_sum": 0, 2024-09-21T03:56:53.379 INFO:teuthology.orchestra.run.smithi161.stdout: "get_or_fail_fail": 0, 2024-09-21T03:56:53.379 INFO:teuthology.orchestra.run.smithi161.stdout: "get_or_fail_success": 0, 2024-09-21T03:56:53.379 INFO:teuthology.orchestra.run.smithi161.stdout: "take": 2152, 2024-09-21T03:56:53.379 INFO:teuthology.orchestra.run.smithi161.stdout: "take_sum": 2356, 2024-09-21T03:56:53.379 INFO:teuthology.orchestra.run.smithi161.stdout: "put": 2152, 2024-09-21T03:56:53.379 INFO:teuthology.orchestra.run.smithi161.stdout: "put_sum": 2356, 2024-09-21T03:56:53.379 INFO:teuthology.orchestra.run.smithi161.stdout: "wait": { 2024-09-21T03:56:53.379 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-09-21T03:56:53.379 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-09-21T03:56:53.379 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-09-21T03:56:53.380 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T03:56:53.380 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T03:56:53.380 INFO:teuthology.orchestra.run.smithi161.stdout: "throttle-objecter_ops": { 2024-09-21T03:56:53.380 INFO:teuthology.orchestra.run.smithi161.stdout: "val": 0, 2024-09-21T03:56:53.380 INFO:teuthology.orchestra.run.smithi161.stdout: "max": 1024, 2024-09-21T03:56:53.380 INFO:teuthology.orchestra.run.smithi161.stdout: "get_started": 0, 2024-09-21T03:56:53.380 INFO:teuthology.orchestra.run.smithi161.stdout: "get": 0, 2024-09-21T03:56:53.380 INFO:teuthology.orchestra.run.smithi161.stdout: "get_sum": 0, 2024-09-21T03:56:53.380 INFO:teuthology.orchestra.run.smithi161.stdout: "get_or_fail_fail": 0, 2024-09-21T03:56:53.380 INFO:teuthology.orchestra.run.smithi161.stdout: "get_or_fail_success": 0, 2024-09-21T03:56:53.380 INFO:teuthology.orchestra.run.smithi161.stdout: "take": 2152, 2024-09-21T03:56:53.380 INFO:teuthology.orchestra.run.smithi161.stdout: "take_sum": 2152, 2024-09-21T03:56:53.380 INFO:teuthology.orchestra.run.smithi161.stdout: "put": 2152, 2024-09-21T03:56:53.381 INFO:teuthology.orchestra.run.smithi161.stdout: "put_sum": 2152, 2024-09-21T03:56:53.381 INFO:teuthology.orchestra.run.smithi161.stdout: "wait": { 2024-09-21T03:56:53.381 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-09-21T03:56:53.381 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-09-21T03:56:53.381 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-09-21T03:56:53.381 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T03:56:53.381 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T03:56:53.381 INFO:teuthology.orchestra.run.smithi161.stdout: "throttle-write_buf_throttle": { 2024-09-21T03:56:53.381 INFO:teuthology.orchestra.run.smithi161.stdout: "val": 0, 2024-09-21T03:56:53.381 INFO:teuthology.orchestra.run.smithi161.stdout: "max": 3758096384, 2024-09-21T03:56:53.381 INFO:teuthology.orchestra.run.smithi161.stdout: "get_started": 0, 2024-09-21T03:56:53.381 INFO:teuthology.orchestra.run.smithi161.stdout: "get": 0, 2024-09-21T03:56:53.382 INFO:teuthology.orchestra.run.smithi161.stdout: "get_sum": 0, 2024-09-21T03:56:53.382 INFO:teuthology.orchestra.run.smithi161.stdout: "get_or_fail_fail": 0, 2024-09-21T03:56:53.382 INFO:teuthology.orchestra.run.smithi161.stdout: "get_or_fail_success": 0, 2024-09-21T03:56:53.382 INFO:teuthology.orchestra.run.smithi161.stdout: "take": 0, 2024-09-21T03:56:53.382 INFO:teuthology.orchestra.run.smithi161.stdout: "take_sum": 0, 2024-09-21T03:56:53.382 INFO:teuthology.orchestra.run.smithi161.stdout: "put": 0, 2024-09-21T03:56:53.382 INFO:teuthology.orchestra.run.smithi161.stdout: "put_sum": 0, 2024-09-21T03:56:53.382 INFO:teuthology.orchestra.run.smithi161.stdout: "wait": { 2024-09-21T03:56:53.382 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-09-21T03:56:53.382 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-09-21T03:56:53.382 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-09-21T03:56:53.382 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T03:56:53.382 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T03:56:53.383 INFO:teuthology.orchestra.run.smithi161.stdout: "throttle-write_buf_throttle-0x5626c2503450": { 2024-09-21T03:56:53.383 INFO:teuthology.orchestra.run.smithi161.stdout: "val": 0, 2024-09-21T03:56:53.383 INFO:teuthology.orchestra.run.smithi161.stdout: "max": 3758096384, 2024-09-21T03:56:53.383 INFO:teuthology.orchestra.run.smithi161.stdout: "get_started": 0, 2024-09-21T03:56:53.383 INFO:teuthology.orchestra.run.smithi161.stdout: "get": 0, 2024-09-21T03:56:53.383 INFO:teuthology.orchestra.run.smithi161.stdout: "get_sum": 0, 2024-09-21T03:56:53.383 INFO:teuthology.orchestra.run.smithi161.stdout: "get_or_fail_fail": 0, 2024-09-21T03:56:53.383 INFO:teuthology.orchestra.run.smithi161.stdout: "get_or_fail_success": 0, 2024-09-21T03:56:53.383 INFO:teuthology.orchestra.run.smithi161.stdout: "take": 0, 2024-09-21T03:56:53.383 INFO:teuthology.orchestra.run.smithi161.stdout: "take_sum": 0, 2024-09-21T03:56:53.383 INFO:teuthology.orchestra.run.smithi161.stdout: "put": 0, 2024-09-21T03:56:53.383 INFO:teuthology.orchestra.run.smithi161.stdout: "put_sum": 0, 2024-09-21T03:56:53.384 INFO:teuthology.orchestra.run.smithi161.stdout: "wait": { 2024-09-21T03:56:53.384 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-09-21T03:56:53.384 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-09-21T03:56:53.384 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-09-21T03:56:53.384 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T03:56:53.384 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-21T03:56:53.384 INFO:teuthology.orchestra.run.smithi161.stdout: "trackedop": { 2024-09-21T03:56:53.384 INFO:teuthology.orchestra.run.smithi161.stdout: "slow_ops_count": 0 2024-09-21T03:56:53.384 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-21T03:56:53.384 INFO:teuthology.orchestra.run.smithi161.stdout:} 2024-09-21T03:56:53.468 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:56:53 smithi161 ceph-mon[34533]: pgmap v658: 97 pgs: 97 active+clean; 10 MiB data, 525 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-21T03:56:53.725 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:56:53 smithi150 ceph-mon[34289]: pgmap v658: 97 pgs: 97 active+clean; 10 MiB data, 525 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-21T03:56:53.834 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:56:53 smithi039 ceph-mon[31211]: pgmap v658: 97 pgs: 97 active+clean; 10 MiB data, 525 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-21T03:56:54.163 INFO:tasks.check_counter:Daemon mds.l mds.exported=0 2024-09-21T03:56:54.163 INFO:tasks.check_counter:Daemon mds.l mds.imported=0 2024-09-21T03:56:54.163 INFO:tasks.check_counter:Daemon mds.l mds_cache.dir_update=0 2024-09-21T03:56:54.163 INFO:tasks.check_counter:Daemon mds.l mds_cache.dir_update_receipt=0 2024-09-21T03:56:54.163 INFO:tasks.check_counter:Daemon mds.l mds.root_rsnaps=0 2024-09-21T03:56:54.163 INFO:tasks.check_counter:Daemon mds.l mds_server.req_mksnap_latency.avgcount=0 2024-09-21T03:56:54.163 INFO:tasks.check_counter:Daemon mds.l mds_server.req_rmsnap_latency.avgcount=0 2024-09-21T03:56:54.163 DEBUG:teuthology.run_tasks:Unwinding manager kclient 2024-09-21T03:56:54.192 INFO:tasks.kclient:Unmounting kernel clients... 2024-09-21T03:56:54.192 DEBUG:teuthology.orchestra.run.smithi039:> set -ex 2024-09-21T03:56:54.192 DEBUG:teuthology.orchestra.run.smithi039:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T03:56:54.214 DEBUG:teuthology.orchestra.run.smithi039:> set -ex 2024-09-21T03:56:54.214 DEBUG:teuthology.orchestra.run.smithi039:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T03:56:54.275 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 51116d5c-77ca-11ef-baf5-efdc52797490 -- ceph osd blocklist ls 2024-09-21T03:56:54.584 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/51116d5c-77ca-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:56:55.090 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:55.081+0000 7f31a1f4e640 1 -- 172.21.15.39:0/1978960414 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f319c10de40 msgr2=0x7f319c110330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:56:55.091 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:55.081+0000 7f31a1f4e640 1 --2- 172.21.15.39:0/1978960414 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f319c10de40 0x7f319c110330 secure :-1 s=READY pgs=1366 cs=0 l=1 rev1=1 crypto rx=0x7f3194009ca0 tx=0x7f319402a8c0 comp rx=0 tx=0).stop 2024-09-21T03:56:55.091 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:55.081+0000 7f31a1f4e640 1 -- 172.21.15.39:0/1978960414 shutdown_connections 2024-09-21T03:56:55.091 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:55.081+0000 7f31a1f4e640 1 --2- 172.21.15.39:0/1978960414 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f319c10de40 0x7f319c110330 unknown :-1 s=CLOSED pgs=1366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:55.091 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:55.081+0000 7f31a1f4e640 1 --2- 172.21.15.39:0/1978960414 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f319c105330 0x7f319c10d880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:55.091 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:55.081+0000 7f31a1f4e640 1 --2- 172.21.15.39:0/1978960414 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f319c1049e0 0x7f319c104df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:55.091 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:55.081+0000 7f31a1f4e640 1 -- 172.21.15.39:0/1978960414 >> 172.21.15.39:0/1978960414 conn(0x7f319c079750 msgr2=0x7f319c079b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:56:55.091 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:55.081+0000 7f31a1f4e640 1 -- 172.21.15.39:0/1978960414 shutdown_connections 2024-09-21T03:56:55.091 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:55.081+0000 7f31a1f4e640 1 -- 172.21.15.39:0/1978960414 wait complete. 2024-09-21T03:56:55.091 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:55.081+0000 7f31a1f4e640 1 Processor -- start 2024-09-21T03:56:55.091 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:55.082+0000 7f31a1f4e640 1 -- start start 2024-09-21T03:56:55.091 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:55.082+0000 7f31a1f4e640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f319c1049e0 0x7f319c19a470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:56:55.091 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:55.082+0000 7f31a1f4e640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f319c105330 0x7f319c19aa10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:56:55.091 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:55.082+0000 7f31a1f4e640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f319c10de40 0x7f319c19fcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:56:55.092 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:55.082+0000 7f31a1f4e640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f319c113aa0 con 0x7f319c10de40 2024-09-21T03:56:55.092 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:55.082+0000 7f31a1f4e640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f319c113920 con 0x7f319c105330 2024-09-21T03:56:55.092 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:55.082+0000 7f31a1f4e640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f319c113c20 con 0x7f319c1049e0 2024-09-21T03:56:55.092 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:55.082+0000 7f319bfff640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f319c10de40 0x7f319c19fcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:56:55.092 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:55.082+0000 7f319bfff640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f319c10de40 0x7f319c19fcb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:42442/0 (socket says 172.21.15.39:42442) 2024-09-21T03:56:55.092 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:55.082+0000 7f319bfff640 1 -- 172.21.15.39:0/3694693446 learned_addr learned my addr 172.21.15.39:0/3694693446 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:56:55.092 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:55.082+0000 7f319b7fe640 1 --2- 172.21.15.39:0/3694693446 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f319c1049e0 0x7f319c19a470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:56:55.092 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:55.082+0000 7f319affd640 1 --2- 172.21.15.39:0/3694693446 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f319c105330 0x7f319c19aa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:56:55.092 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:55.083+0000 7f319bfff640 1 -- 172.21.15.39:0/3694693446 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f319c1049e0 msgr2=0x7f319c19a470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:56:55.092 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:55.083+0000 7f319bfff640 1 --2- 172.21.15.39:0/3694693446 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f319c1049e0 0x7f319c19a470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:55.092 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:55.083+0000 7f319bfff640 1 -- 172.21.15.39:0/3694693446 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f319c105330 msgr2=0x7f319c19aa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:56:55.092 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:55.083+0000 7f319bfff640 1 --2- 172.21.15.39:0/3694693446 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f319c105330 0x7f319c19aa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:55.092 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:55.083+0000 7f319bfff640 1 -- 172.21.15.39:0/3694693446 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f319c1a0320 con 0x7f319c10de40 2024-09-21T03:56:55.092 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:55.083+0000 7f319bfff640 1 --2- 172.21.15.39:0/3694693446 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f319c10de40 0x7f319c19fcb0 secure :-1 s=READY pgs=1367 cs=0 l=1 rev1=1 crypto rx=0x7f3194005cc0 tx=0x7f319402f470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:56:55.093 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:55.083+0000 7f3198ff9640 1 -- 172.21.15.39:0/3694693446 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3194003cf0 con 0x7f319c10de40 2024-09-21T03:56:55.093 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:55.083+0000 7f3198ff9640 1 -- 172.21.15.39:0/3694693446 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f31940044f0 con 0x7f319c10de40 2024-09-21T03:56:55.093 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:55.083+0000 7f31a1f4e640 1 -- 172.21.15.39:0/3694693446 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f319c1a05b0 con 0x7f319c10de40 2024-09-21T03:56:55.093 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:55.083+0000 7f3198ff9640 1 -- 172.21.15.39:0/3694693446 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f319402fd10 con 0x7f319c10de40 2024-09-21T03:56:55.093 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:55.083+0000 7f31a1f4e640 1 -- 172.21.15.39:0/3694693446 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f319c1a0af0 con 0x7f319c10de40 2024-09-21T03:56:55.093 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:55.084+0000 7f31a1f4e640 1 -- 172.21.15.39:0/3694693446 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f319c06c280 con 0x7f319c10de40 2024-09-21T03:56:55.093 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:55.086+0000 7f3198ff9640 1 -- 172.21.15.39:0/3694693446 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 27) ==== 154643+0+0 (secure 0 0 0) 0x7f3194004040 con 0x7f319c10de40 2024-09-21T03:56:55.093 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:55.086+0000 7f3198ff9640 1 --2- 172.21.15.39:0/3694693446 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f31840b66b0 0x7f31840b8bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:56:55.093 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:55.087+0000 7f319b7fe640 1 --2- 172.21.15.39:0/3694693446 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f31840b66b0 0x7f31840b8bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:56:55.093 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:55.087+0000 7f3198ff9640 1 -- 172.21.15.39:0/3694693446 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8528+0+0 (secure 0 0 0) 0x7f3194108b60 con 0x7f319c10de40 2024-09-21T03:56:55.093 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:55.087+0000 7f319b7fe640 1 --2- 172.21.15.39:0/3694693446 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f31840b66b0 0x7f31840b8bc0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f319c06c980 tx=0x7f318c005f90 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:56:55.094 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:55.091+0000 7f3198ff9640 1 -- 172.21.15.39:0/3694693446 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f31940cf0b0 con 0x7f319c10de40 2024-09-21T03:56:55.295 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:55.292+0000 7f31a1f4e640 1 -- 172.21.15.39:0/3694693446 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f319c19b5a0 con 0x7f319c10de40 2024-09-21T03:56:55.295 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:55.292+0000 7f3198ff9640 1 -- 172.21.15.39:0/3694693446 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 15 entries v87) ==== 81+0+885 (secure 0 0 0) 0x7f31940d3490 con 0x7f319c10de40 2024-09-21T03:56:55.296 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.39:6801/2438110611 2024-09-22T03:37:25.391457+0000 2024-09-21T03:56:55.296 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.39:6800/2438110611 2024-09-22T03:37:25.391457+0000 2024-09-21T03:56:55.296 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.39:6800/173264695 2024-09-22T03:35:57.104467+0000 2024-09-21T03:56:55.296 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.39:6801/173264695 2024-09-22T03:35:57.104467+0000 2024-09-21T03:56:55.296 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.39:0/2637182284 2024-09-22T03:35:57.104467+0000 2024-09-21T03:56:55.296 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.39:0/1517304561 2024-09-22T03:36:24.112592+0000 2024-09-21T03:56:55.296 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.39:6801/1435622199 2024-09-22T03:36:24.112592+0000 2024-09-21T03:56:55.296 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.39:6800/1435622199 2024-09-22T03:36:24.112592+0000 2024-09-21T03:56:55.296 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.39:0/346832469 2024-09-22T03:35:57.104467+0000 2024-09-21T03:56:55.296 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.39:0/3160777371 2024-09-22T03:36:24.112592+0000 2024-09-21T03:56:55.296 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.39:0/1730342855 2024-09-22T03:35:57.104467+0000 2024-09-21T03:56:55.296 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.39:0/1955726766 2024-09-22T03:36:24.112592+0000 2024-09-21T03:56:55.297 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.39:0/3294132796 2024-09-22T03:37:25.391457+0000 2024-09-21T03:56:55.297 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.39:0/1295810162 2024-09-22T03:37:25.391457+0000 2024-09-21T03:56:55.297 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.39:0/3729230474 2024-09-22T03:37:25.391457+0000 2024-09-21T03:56:55.297 INFO:teuthology.orchestra.run.smithi039.stderr:listed 15 entries 2024-09-21T03:56:55.298 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:55.295+0000 7f31a1f4e640 1 -- 172.21.15.39:0/3694693446 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f31840b66b0 msgr2=0x7f31840b8bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:56:55.298 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:55.295+0000 7f31a1f4e640 1 --2- 172.21.15.39:0/3694693446 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f31840b66b0 0x7f31840b8bc0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f319c06c980 tx=0x7f318c005f90 comp rx=0 tx=0).stop 2024-09-21T03:56:55.298 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:55.295+0000 7f31a1f4e640 1 -- 172.21.15.39:0/3694693446 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f319c10de40 msgr2=0x7f319c19fcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:56:55.298 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:55.295+0000 7f31a1f4e640 1 --2- 172.21.15.39:0/3694693446 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f319c10de40 0x7f319c19fcb0 secure :-1 s=READY pgs=1367 cs=0 l=1 rev1=1 crypto rx=0x7f3194005cc0 tx=0x7f319402f470 comp rx=0 tx=0).stop 2024-09-21T03:56:55.298 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:55.295+0000 7f31a1f4e640 1 -- 172.21.15.39:0/3694693446 shutdown_connections 2024-09-21T03:56:55.298 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:55.295+0000 7f31a1f4e640 1 --2- 172.21.15.39:0/3694693446 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f31840b66b0 0x7f31840b8bc0 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:55.298 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:55.295+0000 7f31a1f4e640 1 --2- 172.21.15.39:0/3694693446 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f319c10de40 0x7f319c19fcb0 unknown :-1 s=CLOSED pgs=1367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:55.298 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:55.295+0000 7f31a1f4e640 1 --2- 172.21.15.39:0/3694693446 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f319c105330 0x7f319c19aa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:55.299 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:55.295+0000 7f31a1f4e640 1 --2- 172.21.15.39:0/3694693446 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f319c1049e0 0x7f319c19a470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:55.299 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:55.295+0000 7f31a1f4e640 1 -- 172.21.15.39:0/3694693446 >> 172.21.15.39:0/3694693446 conn(0x7f319c079750 msgr2=0x7f319c1078d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:56:55.299 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:55.295+0000 7f31a1f4e640 1 -- 172.21.15.39:0/3694693446 shutdown_connections 2024-09-21T03:56:55.299 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:55.295+0000 7f31a1f4e640 1 -- 172.21.15.39:0/3694693446 wait complete. 2024-09-21T03:56:55.584 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:56:55 smithi039 ceph-mon[31211]: pgmap v659: 97 pgs: 97 active+clean; 10 MiB data, 525 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-09-21T03:56:55.584 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:56:55 smithi039 ceph-mon[31211]: from='client.? 172.21.15.39:0/3694693446' entity='client.admin' cmd={"prefix": "osd blocklist ls"} : dispatch 2024-09-21T03:56:55.718 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:56:55 smithi161 ceph-mon[34533]: pgmap v659: 97 pgs: 97 active+clean; 10 MiB data, 525 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-09-21T03:56:55.956 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:56:55 smithi161 ceph-mon[34533]: from='client.? 172.21.15.39:0/3694693446' entity='client.admin' cmd={"prefix": "osd blocklist ls"} : dispatch 2024-09-21T03:56:55.957 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:56:55 smithi150 ceph-mon[34289]: pgmap v659: 97 pgs: 97 active+clean; 10 MiB data, 525 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-09-21T03:56:55.957 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:56:55 smithi150 ceph-mon[34289]: from='client.? 172.21.15.39:0/3694693446' entity='client.admin' cmd={"prefix": "osd blocklist ls"} : dispatch 2024-09-21T03:56:56.098 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-09-21T03:56:56.099 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T03:56:56.099 DEBUG:teuthology.orchestra.run.smithi039:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-09-21T03:56:56.150 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi039.front.sepia.ceph.com 2024-09-21T03:56:56.151 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T03:56:56.151 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-21T03:56:56.183 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T03:56:56.183 DEBUG:teuthology.orchestra.run.smithi039:> ip netns list 2024-09-21T03:56:56.242 INFO:teuthology.orchestra.run.smithi039.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-09-21T03:56:56.242 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T03:56:56.242 DEBUG:teuthology.orchestra.run.smithi039:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-21T03:56:56.313 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T03:56:56.313 DEBUG:teuthology.orchestra.run.smithi039:> sudo ip link delete ceph-brx 2024-09-21T03:56:56.411 DEBUG:teuthology.run_tasks:Unwinding manager cephadm 2024-09-21T03:56:58.036 INFO:tasks.cephadm:Teardown begin 2024-09-21T03:56:58.037 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-09-21T03:56:58.037 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 51116d5c-77ca-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T03:56:58.038 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:56:57 smithi161 ceph-mon[34533]: pgmap v660: 97 pgs: 97 active+clean; 10 MiB data, 525 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-21T03:56:58.038 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:56:57 smithi150 ceph-mon[34289]: pgmap v660: 97 pgs: 97 active+clean; 10 MiB data, 525 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-21T03:56:58.039 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:56:57 smithi039 ceph-mon[31211]: pgmap v660: 97 pgs: 97 active+clean; 10 MiB data, 525 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-21T03:56:58.314 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/51116d5c-77ca-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:56:58.764 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:58.759+0000 7f0302c83640 1 -- 172.21.15.39:0/3924763180 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f02fc109560 msgr2=0x7f02fc109970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:56:58.764 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:58.759+0000 7f0302c83640 1 --2- 172.21.15.39:0/3924763180 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f02fc109560 0x7f02fc109970 secure :-1 s=READY pgs=1368 cs=0 l=1 rev1=1 crypto rx=0x7f02e8009900 tx=0x7f02e802a410 comp rx=0 tx=0).stop 2024-09-21T03:56:58.764 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:58.759+0000 7f0302c83640 1 -- 172.21.15.39:0/3924763180 shutdown_connections 2024-09-21T03:56:58.764 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:58.759+0000 7f0302c83640 1 --2- 172.21.15.39:0/3924763180 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f02fc109eb0 0x7f02fc1106e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:58.764 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:58.759+0000 7f0302c83640 1 --2- 172.21.15.39:0/3924763180 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f02fc109560 0x7f02fc109970 unknown :-1 s=CLOSED pgs=1368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:58.764 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:58.759+0000 7f0302c83640 1 --2- 172.21.15.39:0/3924763180 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f02fc102080 0x7f02fc102490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:58.764 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:58.759+0000 7f0302c83640 1 -- 172.21.15.39:0/3924763180 >> 172.21.15.39:0/3924763180 conn(0x7f02fc079ec0 msgr2=0x7f02fc07a2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:56:58.765 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:58.759+0000 7f0302c83640 1 -- 172.21.15.39:0/3924763180 shutdown_connections 2024-09-21T03:56:58.765 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:58.759+0000 7f0302c83640 1 -- 172.21.15.39:0/3924763180 wait complete. 2024-09-21T03:56:58.765 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:58.759+0000 7f0302c83640 1 Processor -- start 2024-09-21T03:56:58.765 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:58.760+0000 7f0302c83640 1 -- start start 2024-09-21T03:56:58.765 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:58.760+0000 7f0302c83640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f02fc102080 0x7f02fc103020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:56:58.765 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:58.760+0000 7f0302c83640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f02fc109560 0x7f02fc103560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:56:58.765 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:58.760+0000 7f0302c83640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f02fc109eb0 0x7f02fc103aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:56:58.765 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:58.760+0000 7f0302c83640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f02fc113b70 con 0x7f02fc102080 2024-09-21T03:56:58.765 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:58.760+0000 7f0302c83640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f02fc1139f0 con 0x7f02fc109eb0 2024-09-21T03:56:58.765 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:58.760+0000 7f0302c83640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f02fc113cf0 con 0x7f02fc109560 2024-09-21T03:56:58.765 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:58.760+0000 7f0300a1f640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f02fc102080 0x7f02fc103020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:56:58.765 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:58.760+0000 7f0300a1f640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f02fc102080 0x7f02fc103020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:36000/0 (socket says 172.21.15.39:36000) 2024-09-21T03:56:58.765 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:58.760+0000 7f0300a1f640 1 -- 172.21.15.39:0/157664040 learned_addr learned my addr 172.21.15.39:0/157664040 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:56:58.766 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:58.760+0000 7f02fbfff640 1 --2- 172.21.15.39:0/157664040 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f02fc109560 0x7f02fc103560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:56:58.766 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:58.760+0000 7f0301220640 1 --2- 172.21.15.39:0/157664040 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f02fc109eb0 0x7f02fc103aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:56:58.766 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:58.760+0000 7f0300a1f640 1 -- 172.21.15.39:0/157664040 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f02fc109560 msgr2=0x7f02fc103560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:56:58.766 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:58.760+0000 7f0300a1f640 1 --2- 172.21.15.39:0/157664040 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f02fc109560 0x7f02fc103560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:58.766 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:58.760+0000 7f0300a1f640 1 -- 172.21.15.39:0/157664040 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f02fc109eb0 msgr2=0x7f02fc103aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:56:58.766 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:58.760+0000 7f0300a1f640 1 --2- 172.21.15.39:0/157664040 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f02fc109eb0 0x7f02fc103aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:58.766 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:58.760+0000 7f0300a1f640 1 -- 172.21.15.39:0/157664040 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f02fc1a8790 con 0x7f02fc102080 2024-09-21T03:56:58.766 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:58.760+0000 7f0301220640 1 --2- 172.21.15.39:0/157664040 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f02fc109eb0 0x7f02fc103aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:56:58.766 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:58.760+0000 7f02fbfff640 1 --2- 172.21.15.39:0/157664040 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f02fc109560 0x7f02fc103560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:56:58.766 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:58.761+0000 7f0300a1f640 1 --2- 172.21.15.39:0/157664040 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f02fc102080 0x7f02fc103020 secure :-1 s=READY pgs=1369 cs=0 l=1 rev1=1 crypto rx=0x7f02f000c920 tx=0x7f02f000cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:56:58.766 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:58.761+0000 7f02f9ffb640 1 -- 172.21.15.39:0/157664040 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02f0007c70 con 0x7f02fc102080 2024-09-21T03:56:58.766 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:58.761+0000 7f0302c83640 1 -- 172.21.15.39:0/157664040 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f02fc1a8a80 con 0x7f02fc102080 2024-09-21T03:56:58.766 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:58.761+0000 7f02f9ffb640 1 -- 172.21.15.39:0/157664040 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f02f0007e10 con 0x7f02fc102080 2024-09-21T03:56:58.766 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:58.761+0000 7f02f9ffb640 1 -- 172.21.15.39:0/157664040 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02f0018820 con 0x7f02fc102080 2024-09-21T03:56:58.767 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:58.761+0000 7f0302c83640 1 -- 172.21.15.39:0/157664040 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f02fc1a8f90 con 0x7f02fc102080 2024-09-21T03:56:58.767 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:58.762+0000 7f0302c83640 1 -- 172.21.15.39:0/157664040 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f02fc102490 con 0x7f02fc102080 2024-09-21T03:56:58.768 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:58.763+0000 7f02f9ffb640 1 -- 172.21.15.39:0/157664040 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 27) ==== 154643+0+0 (secure 0 0 0) 0x7f02f000f450 con 0x7f02fc102080 2024-09-21T03:56:58.768 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:58.764+0000 7f02f9ffb640 1 --2- 172.21.15.39:0/157664040 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f02ec0b6680 0x7f02ec0b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:56:58.768 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:58.764+0000 7f02fbfff640 1 --2- 172.21.15.39:0/157664040 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f02ec0b6680 0x7f02ec0b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:56:58.768 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:58.764+0000 7f02f9ffb640 1 -- 172.21.15.39:0/157664040 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8528+0+0 (secure 0 0 0) 0x7f02f00e7bc0 con 0x7f02fc102080 2024-09-21T03:56:58.768 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:58.765+0000 7f02fbfff640 1 --2- 172.21.15.39:0/157664040 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f02ec0b6680 0x7f02ec0b8b90 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f02e8009230 tx=0x7f02e8000bc0 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:56:58.770 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:58.767+0000 7f02f9ffb640 1 -- 172.21.15.39:0/157664040 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f02f0014030 con 0x7f02fc102080 2024-09-21T03:56:58.994 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:58.991+0000 7f0302c83640 1 -- 172.21.15.39:0/157664040 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f02fc104320 con 0x7f02fc102080 2024-09-21T03:56:58.995 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:58.992+0000 7f02f9ffb640 1 -- 172.21.15.39:0/157664040 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11978 (secure 0 0 0) 0x7f02f00ae1a0 con 0x7f02fc102080 2024-09-21T03:56:58.995 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T03:56:58.995 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":29,"btime":"2024-09-21T03:46:08:982881+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14790,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6835/508995295","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":508995295},{"type":"v1","addr":"172.21.15.39:6835","nonce":508995295}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14802,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6837/3915912607","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":3915912607},{"type":"v1","addr":"172.21.15.39:6837","nonce":3915912607}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":29,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T03:44:15.009872+0000","modified":"2024-09-21T03:46:08.430113+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24695,"mds_1":24667,"mds_2":24659,"mds_3":24646,"mds_4":14868},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14856":{"gid":14856,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.39:6839/185509192","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6838","nonce":185509192},{"type":"v1","addr":"172.21.15.39:6839","nonce":185509192}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14868":{"gid":14868,"name":"a","rank":4,"incarnation":25,"state":"up:active","state_seq":7,"addr":"172.21.15.39:6841/4247190456","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6840","nonce":4247190456},{"type":"v1","addr":"172.21.15.39:6841","nonce":4247190456}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24599":{"gid":24599,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.150:6833/3114643217","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":3114643217},{"type":"v1","addr":"172.21.15.150:6833","nonce":3114643217}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24601":{"gid":24601,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.161:6833/2963990654","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":2963990654},{"type":"v1","addr":"172.21.15.161:6833","nonce":2963990654}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24632":{"gid":24632,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.150:6835/724594008","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6834","nonce":724594008},{"type":"v1","addr":"172.21.15.150:6835","nonce":724594008}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24640":{"gid":24640,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.161:6835/3747882110","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":3747882110},{"type":"v1","addr":"172.21.15.161:6835","nonce":3747882110}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24646":{"gid":24646,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":10,"addr":"172.21.15.161:6837/3451544597","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6836","nonce":3451544597},{"type":"v1","addr":"172.21.15.161:6837","nonce":3451544597}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24659":{"gid":24659,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.150:6837/4017043144","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6836","nonce":4017043144},{"type":"v1","addr":"172.21.15.150:6837","nonce":4017043144}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24667":{"gid":24667,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.161:6839/2190107270","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6838","nonce":2190107270},{"type":"v1","addr":"172.21.15.161:6839","nonce":2190107270}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24695":{"gid":24695,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.150:6839/2949386907","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6838","nonce":2949386907},{"type":"v1","addr":"172.21.15.150:6839","nonce":2949386907}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24695,"qdb_cluster":[24695,24667,24659,24646,14868]},"id":1}]} 2024-09-21T03:56:58.996 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 29 2024-09-21T03:56:58.998 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:58.994+0000 7f0302c83640 1 -- 172.21.15.39:0/157664040 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f02ec0b6680 msgr2=0x7f02ec0b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:56:58.998 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:58.994+0000 7f0302c83640 1 --2- 172.21.15.39:0/157664040 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f02ec0b6680 0x7f02ec0b8b90 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f02e8009230 tx=0x7f02e8000bc0 comp rx=0 tx=0).stop 2024-09-21T03:56:58.998 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:58.995+0000 7f0302c83640 1 -- 172.21.15.39:0/157664040 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f02fc102080 msgr2=0x7f02fc103020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:56:58.998 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:58.995+0000 7f0302c83640 1 --2- 172.21.15.39:0/157664040 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f02fc102080 0x7f02fc103020 secure :-1 s=READY pgs=1369 cs=0 l=1 rev1=1 crypto rx=0x7f02f000c920 tx=0x7f02f000cde0 comp rx=0 tx=0).stop 2024-09-21T03:56:58.998 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:58.995+0000 7f0302c83640 1 -- 172.21.15.39:0/157664040 shutdown_connections 2024-09-21T03:56:58.998 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:58.995+0000 7f0302c83640 1 --2- 172.21.15.39:0/157664040 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f02ec0b6680 0x7f02ec0b8b90 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:58.998 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:58.995+0000 7f0302c83640 1 --2- 172.21.15.39:0/157664040 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f02fc109eb0 0x7f02fc103aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:58.998 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:58.995+0000 7f0302c83640 1 --2- 172.21.15.39:0/157664040 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f02fc109560 0x7f02fc103560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:58.998 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:58.995+0000 7f0302c83640 1 --2- 172.21.15.39:0/157664040 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f02fc102080 0x7f02fc103020 unknown :-1 s=CLOSED pgs=1369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:58.998 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:58.995+0000 7f0302c83640 1 -- 172.21.15.39:0/157664040 >> 172.21.15.39:0/157664040 conn(0x7f02fc079ec0 msgr2=0x7f02fc107660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:56:58.998 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:58.995+0000 7f0302c83640 1 -- 172.21.15.39:0/157664040 shutdown_connections 2024-09-21T03:56:58.998 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:56:58.995+0000 7f0302c83640 1 -- 172.21.15.39:0/157664040 wait complete. 2024-09-21T03:56:59.663 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:56:59 smithi039 ceph-mon[31211]: pgmap v661: 97 pgs: 97 active+clean; 10 MiB data, 525 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 85 B/s wr, 8 op/s 2024-09-21T03:56:59.663 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:56:59 smithi039 ceph-mon[31211]: from='client.? 172.21.15.39:0/157664040' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:56:59.694 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 51116d5c-77ca-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T03:56:59.718 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:56:59 smithi161 ceph-mon[34533]: pgmap v661: 97 pgs: 97 active+clean; 10 MiB data, 525 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 85 B/s wr, 8 op/s 2024-09-21T03:56:59.718 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:56:59 smithi161 ceph-mon[34533]: from='client.? 172.21.15.39:0/157664040' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:56:59.725 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:56:59 smithi150 ceph-mon[34289]: pgmap v661: 97 pgs: 97 active+clean; 10 MiB data, 525 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 85 B/s wr, 8 op/s 2024-09-21T03:56:59.725 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:56:59 smithi150 ceph-mon[34289]: from='client.? 172.21.15.39:0/157664040' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:56:59.956 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/51116d5c-77ca-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:57:00.467 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:57:00.456+0000 7f61705ac640 1 -- 172.21.15.39:0/2901618423 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f61681083c0 msgr2=0x7f61681087d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:57:00.467 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:57:00.456+0000 7f61705ac640 1 --2- 172.21.15.39:0/2901618423 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f61681083c0 0x7f61681087d0 secure :-1 s=READY pgs=1370 cs=0 l=1 rev1=1 crypto rx=0x7f6158009900 tx=0x7f615802a410 comp rx=0 tx=0).stop 2024-09-21T03:57:00.467 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:57:00.456+0000 7f61705ac640 1 -- 172.21.15.39:0/2901618423 shutdown_connections 2024-09-21T03:57:00.467 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:57:00.456+0000 7f61705ac640 1 --2- 172.21.15.39:0/2901618423 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6168108d10 0x7f616810f540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:57:00.467 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:57:00.456+0000 7f61705ac640 1 --2- 172.21.15.39:0/2901618423 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f61681083c0 0x7f61681087d0 unknown :-1 s=CLOSED pgs=1370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:57:00.467 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:57:00.456+0000 7f61705ac640 1 --2- 172.21.15.39:0/2901618423 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6168102a80 0x7f6168102e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:57:00.468 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:57:00.456+0000 7f61705ac640 1 -- 172.21.15.39:0/2901618423 >> 172.21.15.39:0/2901618423 conn(0x7f6168079740 msgr2=0x7f6168079b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:57:00.468 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:57:00.456+0000 7f61705ac640 1 -- 172.21.15.39:0/2901618423 shutdown_connections 2024-09-21T03:57:00.468 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:57:00.456+0000 7f61705ac640 1 -- 172.21.15.39:0/2901618423 wait complete. 2024-09-21T03:57:00.468 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:57:00.457+0000 7f61705ac640 1 Processor -- start 2024-09-21T03:57:00.468 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:57:00.457+0000 7f61705ac640 1 -- start start 2024-09-21T03:57:00.468 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:57:00.457+0000 7f61705ac640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6168102a80 0x7f616819a4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:57:00.468 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:57:00.457+0000 7f61705ac640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f61681083c0 0x7f616819aa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:57:00.468 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:57:00.457+0000 7f61705ac640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6168108d10 0x7f616819fd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:57:00.468 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:57:00.457+0000 7f61705ac640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f6168112980 con 0x7f6168102a80 2024-09-21T03:57:00.468 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:57:00.457+0000 7f61705ac640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f6168112800 con 0x7f6168108d10 2024-09-21T03:57:00.468 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:57:00.457+0000 7f61705ac640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f6168112b00 con 0x7f61681083c0 2024-09-21T03:57:00.468 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:57:00.457+0000 7f616e348640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6168102a80 0x7f616819a4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:57:00.468 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:57:00.457+0000 7f616e348640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6168102a80 0x7f616819a4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:36024/0 (socket says 172.21.15.39:36024) 2024-09-21T03:57:00.468 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:57:00.457+0000 7f616e348640 1 -- 172.21.15.39:0/233478631 learned_addr learned my addr 172.21.15.39:0/233478631 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:57:00.469 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:57:00.457+0000 7f616db47640 1 --2- 172.21.15.39:0/233478631 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f61681083c0 0x7f616819aa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:57:00.469 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:57:00.457+0000 7f616eb49640 1 --2- 172.21.15.39:0/233478631 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6168108d10 0x7f616819fd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:57:00.469 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:57:00.457+0000 7f616e348640 1 -- 172.21.15.39:0/233478631 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f61681083c0 msgr2=0x7f616819aa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:57:00.469 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:57:00.457+0000 7f616e348640 1 --2- 172.21.15.39:0/233478631 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f61681083c0 0x7f616819aa60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:57:00.469 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:57:00.457+0000 7f616e348640 1 -- 172.21.15.39:0/233478631 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6168108d10 msgr2=0x7f616819fd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:57:00.469 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:57:00.457+0000 7f616e348640 1 --2- 172.21.15.39:0/233478631 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6168108d10 0x7f616819fd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:57:00.469 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:57:00.457+0000 7f616e348640 1 -- 172.21.15.39:0/233478631 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f61681a0370 con 0x7f6168102a80 2024-09-21T03:57:00.469 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:57:00.458+0000 7f616e348640 1 --2- 172.21.15.39:0/233478631 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6168102a80 0x7f616819a4c0 secure :-1 s=READY pgs=1371 cs=0 l=1 rev1=1 crypto rx=0x7f616400d770 tx=0x7f616400dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:57:00.469 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:57:00.458+0000 7f615f7fe640 1 -- 172.21.15.39:0/233478631 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6164017070 con 0x7f6168102a80 2024-09-21T03:57:00.469 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:57:00.458+0000 7f615f7fe640 1 -- 172.21.15.39:0/233478631 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f61640054d0 con 0x7f6168102a80 2024-09-21T03:57:00.469 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:57:00.458+0000 7f61705ac640 1 -- 172.21.15.39:0/233478631 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f61681a0660 con 0x7f6168102a80 2024-09-21T03:57:00.469 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:57:00.458+0000 7f615f7fe640 1 -- 172.21.15.39:0/233478631 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6164004c30 con 0x7f6168102a80 2024-09-21T03:57:00.469 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:57:00.458+0000 7f61705ac640 1 -- 172.21.15.39:0/233478631 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f61681a0b20 con 0x7f6168102a80 2024-09-21T03:57:00.469 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:57:00.459+0000 7f61705ac640 1 -- 172.21.15.39:0/233478631 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6168103d00 con 0x7f6168102a80 2024-09-21T03:57:00.469 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:57:00.461+0000 7f615f7fe640 1 -- 172.21.15.39:0/233478631 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 27) ==== 154643+0+0 (secure 0 0 0) 0x7f6164004dd0 con 0x7f6168102a80 2024-09-21T03:57:00.470 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:57:00.461+0000 7f615f7fe640 1 --2- 172.21.15.39:0/233478631 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f61540b66b0 0x7f61540b8bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:57:00.470 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:57:00.461+0000 7f616db47640 1 --2- 172.21.15.39:0/233478631 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f61540b66b0 0x7f61540b8bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:57:00.470 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:57:00.461+0000 7f615f7fe640 1 -- 172.21.15.39:0/233478631 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8528+0+0 (secure 0 0 0) 0x7f61640e7200 con 0x7f6168102a80 2024-09-21T03:57:00.470 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:57:00.462+0000 7f616db47640 1 --2- 172.21.15.39:0/233478631 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f61540b66b0 0x7f61540b8bc0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f6158000c00 tx=0x7f6158031400 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:57:00.470 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:57:00.466+0000 7f615f7fe640 1 -- 172.21.15.39:0/233478631 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f61640ad750 con 0x7f6168102a80 2024-09-21T03:57:00.694 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:57:00.691+0000 7f61705ac640 1 -- 172.21.15.39:0/233478631 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f616819b5f0 con 0x7f6168102a80 2024-09-21T03:57:00.695 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:57:00.692+0000 7f615f7fe640 1 -- 172.21.15.39:0/233478631 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11978 (secure 0 0 0) 0x7f6164092e20 con 0x7f6168102a80 2024-09-21T03:57:00.695 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T03:57:00.695 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":29,"btime":"2024-09-21T03:46:08:982881+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14790,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6835/508995295","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":508995295},{"type":"v1","addr":"172.21.15.39:6835","nonce":508995295}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14802,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6837/3915912607","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":3915912607},{"type":"v1","addr":"172.21.15.39:6837","nonce":3915912607}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":29,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T03:44:15.009872+0000","modified":"2024-09-21T03:46:08.430113+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24695,"mds_1":24667,"mds_2":24659,"mds_3":24646,"mds_4":14868},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14856":{"gid":14856,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.39:6839/185509192","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6838","nonce":185509192},{"type":"v1","addr":"172.21.15.39:6839","nonce":185509192}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14868":{"gid":14868,"name":"a","rank":4,"incarnation":25,"state":"up:active","state_seq":7,"addr":"172.21.15.39:6841/4247190456","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6840","nonce":4247190456},{"type":"v1","addr":"172.21.15.39:6841","nonce":4247190456}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24599":{"gid":24599,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.150:6833/3114643217","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":3114643217},{"type":"v1","addr":"172.21.15.150:6833","nonce":3114643217}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24601":{"gid":24601,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.161:6833/2963990654","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":2963990654},{"type":"v1","addr":"172.21.15.161:6833","nonce":2963990654}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24632":{"gid":24632,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.150:6835/724594008","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6834","nonce":724594008},{"type":"v1","addr":"172.21.15.150:6835","nonce":724594008}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24640":{"gid":24640,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.161:6835/3747882110","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":3747882110},{"type":"v1","addr":"172.21.15.161:6835","nonce":3747882110}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24646":{"gid":24646,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":10,"addr":"172.21.15.161:6837/3451544597","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6836","nonce":3451544597},{"type":"v1","addr":"172.21.15.161:6837","nonce":3451544597}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24659":{"gid":24659,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.150:6837/4017043144","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6836","nonce":4017043144},{"type":"v1","addr":"172.21.15.150:6837","nonce":4017043144}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24667":{"gid":24667,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.161:6839/2190107270","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6838","nonce":2190107270},{"type":"v1","addr":"172.21.15.161:6839","nonce":2190107270}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24695":{"gid":24695,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.150:6839/2949386907","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6838","nonce":2949386907},{"type":"v1","addr":"172.21.15.150:6839","nonce":2949386907}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24695,"qdb_cluster":[24695,24667,24659,24646,14868]},"id":1}]} 2024-09-21T03:57:00.696 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 29 2024-09-21T03:57:00.698 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:57:00.695+0000 7f61705ac640 1 -- 172.21.15.39:0/233478631 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f61540b66b0 msgr2=0x7f61540b8bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:57:00.698 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:57:00.695+0000 7f61705ac640 1 --2- 172.21.15.39:0/233478631 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f61540b66b0 0x7f61540b8bc0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f6158000c00 tx=0x7f6158031400 comp rx=0 tx=0).stop 2024-09-21T03:57:00.699 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:57:00.695+0000 7f61705ac640 1 -- 172.21.15.39:0/233478631 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6168102a80 msgr2=0x7f616819a4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:57:00.699 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:57:00.695+0000 7f61705ac640 1 --2- 172.21.15.39:0/233478631 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6168102a80 0x7f616819a4c0 secure :-1 s=READY pgs=1371 cs=0 l=1 rev1=1 crypto rx=0x7f616400d770 tx=0x7f616400dc30 comp rx=0 tx=0).stop 2024-09-21T03:57:00.699 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:57:00.696+0000 7f61705ac640 1 -- 172.21.15.39:0/233478631 shutdown_connections 2024-09-21T03:57:00.699 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:57:00.696+0000 7f61705ac640 1 --2- 172.21.15.39:0/233478631 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f61540b66b0 0x7f61540b8bc0 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:57:00.699 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:57:00.696+0000 7f61705ac640 1 --2- 172.21.15.39:0/233478631 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6168108d10 0x7f616819fd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:57:00.699 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:57:00.696+0000 7f61705ac640 1 --2- 172.21.15.39:0/233478631 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f61681083c0 0x7f616819aa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:57:00.699 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:57:00.696+0000 7f61705ac640 1 --2- 172.21.15.39:0/233478631 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6168102a80 0x7f616819a4c0 unknown :-1 s=CLOSED pgs=1371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:57:00.699 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:57:00.696+0000 7f61705ac640 1 -- 172.21.15.39:0/233478631 >> 172.21.15.39:0/233478631 conn(0x7f6168079740 msgr2=0x7f616806bf20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:57:00.699 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:57:00.696+0000 7f61705ac640 1 -- 172.21.15.39:0/233478631 shutdown_connections 2024-09-21T03:57:00.699 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:57:00.696+0000 7f61705ac640 1 -- 172.21.15.39:0/233478631 wait complete. 2024-09-21T03:57:01.418 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 51116d5c-77ca-11ef-baf5-efdc52797490 -- ceph osd dump --format=json 2024-09-21T03:57:01.676 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/51116d5c-77ca-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:57:01.718 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:57:01 smithi161 ceph-mon[34533]: pgmap v662: 97 pgs: 97 active+clean; 10 MiB data, 529 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 426 B/s wr, 11 op/s 2024-09-21T03:57:01.718 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:57:01 smithi161 ceph-mon[34533]: from='client.? 172.21.15.39:0/233478631' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:57:01.724 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:57:01 smithi150 ceph-mon[34289]: pgmap v662: 97 pgs: 97 active+clean; 10 MiB data, 529 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 426 B/s wr, 11 op/s 2024-09-21T03:57:01.725 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:57:01 smithi150 ceph-mon[34289]: from='client.? 172.21.15.39:0/233478631' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:57:01.797 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:57:01 smithi039 ceph-mon[31211]: pgmap v662: 97 pgs: 97 active+clean; 10 MiB data, 529 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 426 B/s wr, 11 op/s 2024-09-21T03:57:01.797 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:57:01 smithi039 ceph-mon[31211]: from='client.? 172.21.15.39:0/233478631' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:57:02.094 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:57:02.090+0000 7fab8a5a5640 1 -- 172.21.15.39:0/2942054174 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fab84106c80 msgr2=0x7fab84110710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:57:02.094 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:57:02.090+0000 7fab8a5a5640 1 --2- 172.21.15.39:0/2942054174 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fab84106c80 0x7fab84110710 secure :-1 s=READY pgs=1372 cs=0 l=1 rev1=1 crypto rx=0x7fab7c009ca0 tx=0x7fab7c02a8c0 comp rx=0 tx=0).stop 2024-09-21T03:57:02.094 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:57:02.091+0000 7fab8a5a5640 1 -- 172.21.15.39:0/2942054174 shutdown_connections 2024-09-21T03:57:02.094 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:57:02.091+0000 7fab8a5a5640 1 --2- 172.21.15.39:0/2942054174 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fab84106c80 0x7fab84110710 unknown :-1 s=CLOSED pgs=1372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:57:02.094 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:57:02.091+0000 7fab8a5a5640 1 --2- 172.21.15.39:0/2942054174 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fab84106330 0x7fab84106740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:57:02.094 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:57:02.091+0000 7fab8a5a5640 1 --2- 172.21.15.39:0/2942054174 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fab841020e0 0x7fab841024f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:57:02.094 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:57:02.091+0000 7fab8a5a5640 1 -- 172.21.15.39:0/2942054174 >> 172.21.15.39:0/2942054174 conn(0x7fab84079ec0 msgr2=0x7fab8407a2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:57:02.095 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:57:02.091+0000 7fab8a5a5640 1 -- 172.21.15.39:0/2942054174 shutdown_connections 2024-09-21T03:57:02.095 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:57:02.091+0000 7fab8a5a5640 1 -- 172.21.15.39:0/2942054174 wait complete. 2024-09-21T03:57:02.095 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:57:02.091+0000 7fab8a5a5640 1 Processor -- start 2024-09-21T03:57:02.095 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:57:02.091+0000 7fab8a5a5640 1 -- start start 2024-09-21T03:57:02.096 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:57:02.092+0000 7fab8a5a5640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fab841020e0 0x7fab84103080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:57:02.097 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:57:02.092+0000 7fab8a5a5640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fab84106330 0x7fab841035c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:57:02.097 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:57:02.092+0000 7fab8a5a5640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fab84106c80 0x7fab84103b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:57:02.097 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:57:02.092+0000 7fab8a5a5640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fab84113b70 con 0x7fab84106330 2024-09-21T03:57:02.097 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:57:02.092+0000 7fab8a5a5640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fab841139f0 con 0x7fab84106c80 2024-09-21T03:57:02.097 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:57:02.092+0000 7fab8a5a5640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fab84113cf0 con 0x7fab841020e0 2024-09-21T03:57:02.097 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:57:02.092+0000 7fab837fe640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fab84106330 0x7fab841035c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:57:02.097 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:57:02.092+0000 7fab837fe640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fab84106330 0x7fab841035c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:36046/0 (socket says 172.21.15.39:36046) 2024-09-21T03:57:02.097 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:57:02.092+0000 7fab837fe640 1 -- 172.21.15.39:0/1035168384 learned_addr learned my addr 172.21.15.39:0/1035168384 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:57:02.097 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:57:02.092+0000 7fab88b42640 1 --2- 172.21.15.39:0/1035168384 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fab84106c80 0x7fab84103b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:57:02.097 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:57:02.092+0000 7fab83fff640 1 --2- 172.21.15.39:0/1035168384 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fab841020e0 0x7fab84103080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:57:02.097 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:57:02.092+0000 7fab837fe640 1 -- 172.21.15.39:0/1035168384 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fab841020e0 msgr2=0x7fab84103080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:57:02.097 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:57:02.092+0000 7fab837fe640 1 --2- 172.21.15.39:0/1035168384 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fab841020e0 0x7fab84103080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:57:02.097 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:57:02.092+0000 7fab837fe640 1 -- 172.21.15.39:0/1035168384 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fab84106c80 msgr2=0x7fab84103b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:57:02.098 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:57:02.092+0000 7fab837fe640 1 --2- 172.21.15.39:0/1035168384 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fab84106c80 0x7fab84103b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:57:02.098 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:57:02.092+0000 7fab837fe640 1 -- 172.21.15.39:0/1035168384 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fab841a8760 con 0x7fab84106330 2024-09-21T03:57:02.098 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:57:02.092+0000 7fab83fff640 1 --2- 172.21.15.39:0/1035168384 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fab841020e0 0x7fab84103080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:57:02.098 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:57:02.093+0000 7fab837fe640 1 --2- 172.21.15.39:0/1035168384 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fab84106330 0x7fab841035c0 secure :-1 s=READY pgs=1373 cs=0 l=1 rev1=1 crypto rx=0x7fab7800c970 tx=0x7fab7800ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:57:02.098 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:57:02.093+0000 7fab817fa640 1 -- 172.21.15.39:0/1035168384 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab78004220 con 0x7fab84106330 2024-09-21T03:57:02.098 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:57:02.093+0000 7fab817fa640 1 -- 172.21.15.39:0/1035168384 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fab780043c0 con 0x7fab84106330 2024-09-21T03:57:02.098 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:57:02.093+0000 7fab8a5a5640 1 -- 172.21.15.39:0/1035168384 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fab841a8a50 con 0x7fab84106330 2024-09-21T03:57:02.098 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:57:02.093+0000 7fab817fa640 1 -- 172.21.15.39:0/1035168384 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab78004a00 con 0x7fab84106330 2024-09-21T03:57:02.098 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:57:02.093+0000 7fab8a5a5640 1 -- 172.21.15.39:0/1035168384 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fab841a8f90 con 0x7fab84106330 2024-09-21T03:57:02.098 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:57:02.094+0000 7fab8a5a5640 1 -- 172.21.15.39:0/1035168384 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fab841024f0 con 0x7fab84106330 2024-09-21T03:57:02.099 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:57:02.095+0000 7fab817fa640 1 -- 172.21.15.39:0/1035168384 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 27) ==== 154643+0+0 (secure 0 0 0) 0x7fab780079b0 con 0x7fab84106330 2024-09-21T03:57:02.099 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:57:02.096+0000 7fab817fa640 1 --2- 172.21.15.39:0/1035168384 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7fab6c0b6700 0x7fab6c0b8c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:57:02.099 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:57:02.096+0000 7fab83fff640 1 --2- 172.21.15.39:0/1035168384 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7fab6c0b6700 0x7fab6c0b8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:57:02.099 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:57:02.096+0000 7fab817fa640 1 -- 172.21.15.39:0/1035168384 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8528+0+0 (secure 0 0 0) 0x7fab780e7850 con 0x7fab84106330 2024-09-21T03:57:02.099 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:57:02.097+0000 7fab83fff640 1 --2- 172.21.15.39:0/1035168384 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7fab6c0b6700 0x7fab6c0b8c10 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7fab74005fd0 tx=0x7fab74005ee0 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:57:02.102 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:57:02.099+0000 7fab817fa640 1 -- 172.21.15.39:0/1035168384 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fab78014030 con 0x7fab84106330 2024-09-21T03:57:02.289 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:57:02.286+0000 7fab8a5a5640 1 -- 172.21.15.39:0/1035168384 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fab84104380 con 0x7fab84106330 2024-09-21T03:57:02.291 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:57:02.288+0000 7fab817fa640 1 -- 172.21.15.39:0/1035168384 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v87) ==== 74+0+23186 (secure 0 0 0) 0x7fab780a7e20 con 0x7fab84106330 2024-09-21T03:57:02.291 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T03:57:02.291 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":87,"fsid":"51116d5c-77ca-11ef-baf5-efdc52797490","created":"2024-09-21T03:35:33.631350+0000","modified":"2024-09-21T03:56:19.874370+0000","last_up_change":"2024-09-21T03:43:24.242275+0000","last_in_change":"2024-09-21T03:43:00.886552+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T03:39:29.859542+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-21T03:44:10.886435+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"77","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-21T03:44:13.041783+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":15,"snap_epoch":86,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1ea7148f-2ace-4c10-86fe-b8019de80c1f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6802","nonce":870188791},{"type":"v1","addr":"172.21.15.39:6803","nonce":870188791}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6804","nonce":870188791},{"type":"v1","addr":"172.21.15.39:6805","nonce":870188791}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6808","nonce":870188791},{"type":"v1","addr":"172.21.15.39:6809","nonce":870188791}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6806","nonce":870188791},{"type":"v1","addr":"172.21.15.39:6807","nonce":870188791}]},"public_addr":"172.21.15.39:6803/870188791","cluster_addr":"172.21.15.39:6805/870188791","heartbeat_back_addr":"172.21.15.39:6809/870188791","heartbeat_front_addr":"172.21.15.39:6807/870188791","state":["exists","up"]},{"osd":1,"uuid":"7cb5db90-28e8-4a30-bedb-c039687d48b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6800","nonce":172427832},{"type":"v1","addr":"172.21.15.150:6801","nonce":172427832}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6802","nonce":172427832},{"type":"v1","addr":"172.21.15.150:6803","nonce":172427832}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6806","nonce":172427832},{"type":"v1","addr":"172.21.15.150:6807","nonce":172427832}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6804","nonce":172427832},{"type":"v1","addr":"172.21.15.150:6805","nonce":172427832}]},"public_addr":"172.21.15.150:6801/172427832","cluster_addr":"172.21.15.150:6803/172427832","heartbeat_back_addr":"172.21.15.150:6807/172427832","heartbeat_front_addr":"172.21.15.150:6805/172427832","state":["exists","up"]},{"osd":2,"uuid":"b25c6ab3-fc95-4751-bd63-6e3fc23e8bd7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":19,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6800","nonce":2727876133},{"type":"v1","addr":"172.21.15.161:6801","nonce":2727876133}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6802","nonce":2727876133},{"type":"v1","addr":"172.21.15.161:6803","nonce":2727876133}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6806","nonce":2727876133},{"type":"v1","addr":"172.21.15.161:6807","nonce":2727876133}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6804","nonce":2727876133},{"type":"v1","addr":"172.21.15.161:6805","nonce":2727876133}]},"public_addr":"172.21.15.161:6801/2727876133","cluster_addr":"172.21.15.161:6803/2727876133","heartbeat_back_addr":"172.21.15.161:6807/2727876133","heartbeat_front_addr":"172.21.15.161:6805/2727876133","state":["exists","up"]},{"osd":3,"uuid":"14417ae1-4536-45ba-9a5c-c3b447d1a72d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6810","nonce":295394468},{"type":"v1","addr":"172.21.15.39:6811","nonce":295394468}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6812","nonce":295394468},{"type":"v1","addr":"172.21.15.39:6813","nonce":295394468}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6816","nonce":295394468},{"type":"v1","addr":"172.21.15.39:6817","nonce":295394468}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6814","nonce":295394468},{"type":"v1","addr":"172.21.15.39:6815","nonce":295394468}]},"public_addr":"172.21.15.39:6811/295394468","cluster_addr":"172.21.15.39:6813/295394468","heartbeat_back_addr":"172.21.15.39:6817/295394468","heartbeat_front_addr":"172.21.15.39:6815/295394468","state":["exists","up"]},{"osd":4,"uuid":"33c732ef-20c2-43c4-abe6-045b687fc521","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6808","nonce":3097721553},{"type":"v1","addr":"172.21.15.150:6809","nonce":3097721553}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6810","nonce":3097721553},{"type":"v1","addr":"172.21.15.150:6811","nonce":3097721553}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6814","nonce":3097721553},{"type":"v1","addr":"172.21.15.150:6815","nonce":3097721553}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6812","nonce":3097721553},{"type":"v1","addr":"172.21.15.150:6813","nonce":3097721553}]},"public_addr":"172.21.15.150:6809/3097721553","cluster_addr":"172.21.15.150:6811/3097721553","heartbeat_back_addr":"172.21.15.150:6815/3097721553","heartbeat_front_addr":"172.21.15.150:6813/3097721553","state":["exists","up"]},{"osd":5,"uuid":"74673722-abd6-400a-bb50-ec3dfba0c5e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6808","nonce":742751023},{"type":"v1","addr":"172.21.15.161:6809","nonce":742751023}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6810","nonce":742751023},{"type":"v1","addr":"172.21.15.161:6811","nonce":742751023}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6814","nonce":742751023},{"type":"v1","addr":"172.21.15.161:6815","nonce":742751023}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6812","nonce":742751023},{"type":"v1","addr":"172.21.15.161:6813","nonce":742751023}]},"public_addr":"172.21.15.161:6809/742751023","cluster_addr":"172.21.15.161:6811/742751023","heartbeat_back_addr":"172.21.15.161:6815/742751023","heartbeat_front_addr":"172.21.15.161:6813/742751023","state":["exists","up"]},{"osd":6,"uuid":"06548677-74c4-4efa-8054-df09b957b649","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6818","nonce":1072700290},{"type":"v1","addr":"172.21.15.39:6819","nonce":1072700290}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6820","nonce":1072700290},{"type":"v1","addr":"172.21.15.39:6821","nonce":1072700290}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6824","nonce":1072700290},{"type":"v1","addr":"172.21.15.39:6825","nonce":1072700290}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6822","nonce":1072700290},{"type":"v1","addr":"172.21.15.39:6823","nonce":1072700290}]},"public_addr":"172.21.15.39:6819/1072700290","cluster_addr":"172.21.15.39:6821/1072700290","heartbeat_back_addr":"172.21.15.39:6825/1072700290","heartbeat_front_addr":"172.21.15.39:6823/1072700290","state":["exists","up"]},{"osd":7,"uuid":"838735ca-c693-4f58-9426-b8ee857dbe3a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6816","nonce":2162619103},{"type":"v1","addr":"172.21.15.150:6817","nonce":2162619103}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6818","nonce":2162619103},{"type":"v1","addr":"172.21.15.150:6819","nonce":2162619103}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6822","nonce":2162619103},{"type":"v1","addr":"172.21.15.150:6823","nonce":2162619103}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6820","nonce":2162619103},{"type":"v1","addr":"172.21.15.150:6821","nonce":2162619103}]},"public_addr":"172.21.15.150:6817/2162619103","cluster_addr":"172.21.15.150:6819/2162619103","heartbeat_back_addr":"172.21.15.150:6823/2162619103","heartbeat_front_addr":"172.21.15.150:6821/2162619103","state":["exists","up"]},{"osd":8,"uuid":"4973b357-b4c2-4c00-b7f7-dd25012a0f72","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6816","nonce":52966633},{"type":"v1","addr":"172.21.15.161:6817","nonce":52966633}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6818","nonce":52966633},{"type":"v1","addr":"172.21.15.161:6819","nonce":52966633}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6822","nonce":52966633},{"type":"v1","addr":"172.21.15.161:6823","nonce":52966633}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6820","nonce":52966633},{"type":"v1","addr":"172.21.15.161:6821","nonce":52966633}]},"public_addr":"172.21.15.161:6817/52966633","cluster_addr":"172.21.15.161:6819/52966633","heartbeat_back_addr":"172.21.15.161:6823/52966633","heartbeat_front_addr":"172.21.15.161:6821/52966633","state":["exists","up"]},{"osd":9,"uuid":"6028613c-ebfd-42ea-9949-3f0d5f6cf025","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6826","nonce":2001303253},{"type":"v1","addr":"172.21.15.39:6827","nonce":2001303253}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6828","nonce":2001303253},{"type":"v1","addr":"172.21.15.39:6829","nonce":2001303253}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6832","nonce":2001303253},{"type":"v1","addr":"172.21.15.39:6833","nonce":2001303253}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6830","nonce":2001303253},{"type":"v1","addr":"172.21.15.39:6831","nonce":2001303253}]},"public_addr":"172.21.15.39:6827/2001303253","cluster_addr":"172.21.15.39:6829/2001303253","heartbeat_back_addr":"172.21.15.39:6833/2001303253","heartbeat_front_addr":"172.21.15.39:6831/2001303253","state":["exists","up"]},{"osd":10,"uuid":"1c71c765-2a6c-4ffe-9fe1-c8adae3d1c32","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6824","nonce":2484627938},{"type":"v1","addr":"172.21.15.150:6825","nonce":2484627938}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6826","nonce":2484627938},{"type":"v1","addr":"172.21.15.150:6827","nonce":2484627938}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6830","nonce":2484627938},{"type":"v1","addr":"172.21.15.150:6831","nonce":2484627938}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6828","nonce":2484627938},{"type":"v1","addr":"172.21.15.150:6829","nonce":2484627938}]},"public_addr":"172.21.15.150:6825/2484627938","cluster_addr":"172.21.15.150:6827/2484627938","heartbeat_back_addr":"172.21.15.150:6831/2484627938","heartbeat_front_addr":"172.21.15.150:6829/2484627938","state":["exists","up"]},{"osd":11,"uuid":"f382f522-62db-46da-997e-2b28f84d1612","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6824","nonce":2178709707},{"type":"v1","addr":"172.21.15.161:6825","nonce":2178709707}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6826","nonce":2178709707},{"type":"v1","addr":"172.21.15.161:6827","nonce":2178709707}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6830","nonce":2178709707},{"type":"v1","addr":"172.21.15.161:6831","nonce":2178709707}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6828","nonce":2178709707},{"type":"v1","addr":"172.21.15.161:6829","nonce":2178709707}]},"public_addr":"172.21.15.161:6825/2178709707","cluster_addr":"172.21.15.161:6827/2178709707","heartbeat_back_addr":"172.21.15.161:6831/2178709707","heartbeat_front_addr":"172.21.15.161:6829/2178709707","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:38:35.540904+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:39:01.455053+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:39:27.052867+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:39:55.173529+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:40:21.298396+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:40:47.131235+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:41:14.475209+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:41:39.920036+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:42:05.587065+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:42:32.624726+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:42:57.479185+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:43:22.203832+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.5","mappings":[{"from":8,"to":11}]},{"pgid":"2.8","mappings":[{"from":2,"to":11}]},{"pgid":"2.c","mappings":[{"from":2,"to":1}]},{"pgid":"2.1f","mappings":[{"from":8,"to":4}]},{"pgid":"3.2","mappings":[{"from":3,"to":9}]},{"pgid":"3.14","mappings":[{"from":4,"to":6}]},{"pgid":"3.1a","mappings":[{"from":5,"to":7}]},{"pgid":"3.1d","mappings":[{"from":4,"to":8}]},{"pgid":"3.2c","mappings":[{"from":4,"to":9}]},{"pgid":"3.38","mappings":[{"from":3,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.39:6801/2438110611":"2024-09-22T03:37:25.391457+0000","172.21.15.39:6800/2438110611":"2024-09-22T03:37:25.391457+0000","172.21.15.39:6800/173264695":"2024-09-22T03:35:57.104467+0000","172.21.15.39:6801/173264695":"2024-09-22T03:35:57.104467+0000","172.21.15.39:0/2637182284":"2024-09-22T03:35:57.104467+0000","172.21.15.39:0/1517304561":"2024-09-22T03:36:24.112592+0000","172.21.15.39:6801/1435622199":"2024-09-22T03:36:24.112592+0000","172.21.15.39:6800/1435622199":"2024-09-22T03:36:24.112592+0000","172.21.15.39:0/346832469":"2024-09-22T03:35:57.104467+0000","172.21.15.39:0/3160777371":"2024-09-22T03:36:24.112592+0000","172.21.15.39:0/1730342855":"2024-09-22T03:35:57.104467+0000","172.21.15.39:0/1955726766":"2024-09-22T03:36:24.112592+0000","172.21.15.39:0/3294132796":"2024-09-22T03:37:25.391457+0000","172.21.15.39:0/1295810162":"2024-09-22T03:37:25.391457+0000","172.21.15.39:0/3729230474":"2024-09-22T03:37:25.391457+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":3,"snaps":[{"begin":5,"length":1},{"begin":15,"length":1}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T03:57:02.293 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:57:02.290+0000 7fab8a5a5640 1 -- 172.21.15.39:0/1035168384 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7fab6c0b6700 msgr2=0x7fab6c0b8c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:57:02.293 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:57:02.290+0000 7fab8a5a5640 1 --2- 172.21.15.39:0/1035168384 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7fab6c0b6700 0x7fab6c0b8c10 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7fab74005fd0 tx=0x7fab74005ee0 comp rx=0 tx=0).stop 2024-09-21T03:57:02.293 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:57:02.290+0000 7fab8a5a5640 1 -- 172.21.15.39:0/1035168384 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fab84106330 msgr2=0x7fab841035c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:57:02.293 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:57:02.290+0000 7fab8a5a5640 1 --2- 172.21.15.39:0/1035168384 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fab84106330 0x7fab841035c0 secure :-1 s=READY pgs=1373 cs=0 l=1 rev1=1 crypto rx=0x7fab7800c970 tx=0x7fab7800ce30 comp rx=0 tx=0).stop 2024-09-21T03:57:02.294 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:57:02.290+0000 7fab8a5a5640 1 -- 172.21.15.39:0/1035168384 shutdown_connections 2024-09-21T03:57:02.294 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:57:02.290+0000 7fab8a5a5640 1 --2- 172.21.15.39:0/1035168384 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7fab6c0b6700 0x7fab6c0b8c10 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:57:02.294 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:57:02.290+0000 7fab8a5a5640 1 --2- 172.21.15.39:0/1035168384 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fab84106c80 0x7fab84103b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:57:02.294 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:57:02.290+0000 7fab8a5a5640 1 --2- 172.21.15.39:0/1035168384 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fab84106330 0x7fab841035c0 unknown :-1 s=CLOSED pgs=1373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:57:02.294 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:57:02.290+0000 7fab8a5a5640 1 --2- 172.21.15.39:0/1035168384 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fab841020e0 0x7fab84103080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:57:02.294 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:57:02.290+0000 7fab8a5a5640 1 -- 172.21.15.39:0/1035168384 >> 172.21.15.39:0/1035168384 conn(0x7fab84079ec0 msgr2=0x7fab841074c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:57:02.294 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:57:02.290+0000 7fab8a5a5640 1 -- 172.21.15.39:0/1035168384 shutdown_connections 2024-09-21T03:57:02.294 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:57:02.290+0000 7fab8a5a5640 1 -- 172.21.15.39:0/1035168384 wait complete. 2024-09-21T03:57:02.663 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:57:02 smithi039 ceph-mon[31211]: from='client.? 172.21.15.39:0/1035168384' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T03:57:02.718 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:57:02 smithi161 ceph-mon[34533]: from='client.? 172.21.15.39:0/1035168384' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T03:57:02.724 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:57:02 smithi150 ceph-mon[34289]: from='client.? 172.21.15.39:0/1035168384' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T03:57:02.994 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 51116d5c-77ca-11ef-baf5-efdc52797490 -- ceph fs fail cephfs 2024-09-21T03:57:03.251 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/51116d5c-77ca-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:57:03.693 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:57:03 smithi039 ceph-mon[31211]: pgmap v663: 97 pgs: 97 active+clean; 10 MiB data, 529 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 597 B/s wr, 10 op/s 2024-09-21T03:57:03.703 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:57:03.700+0000 7f5bcb487640 1 -- 172.21.15.39:0/816297753 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5bc4102240 msgr2=0x7f5bc4102650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:57:03.704 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:57:03.700+0000 7f5bcb487640 1 --2- 172.21.15.39:0/816297753 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5bc4102240 0x7f5bc4102650 secure :-1 s=READY pgs=1374 cs=0 l=1 rev1=1 crypto rx=0x7f5bc0009900 tx=0x7f5bc002a410 comp rx=0 tx=0).stop 2024-09-21T03:57:03.704 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:57:03.700+0000 7f5bcb487640 1 -- 172.21.15.39:0/816297753 shutdown_connections 2024-09-21T03:57:03.704 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:57:03.700+0000 7f5bcb487640 1 --2- 172.21.15.39:0/816297753 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5bc41084d0 0x7f5bc410ed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:57:03.704 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:57:03.700+0000 7f5bcb487640 1 --2- 172.21.15.39:0/816297753 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5bc4107b80 0x7f5bc4107f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:57:03.704 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:57:03.700+0000 7f5bcb487640 1 --2- 172.21.15.39:0/816297753 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5bc4102240 0x7f5bc4102650 unknown :-1 s=CLOSED pgs=1374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:57:03.704 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:57:03.701+0000 7f5bcb487640 1 -- 172.21.15.39:0/816297753 >> 172.21.15.39:0/816297753 conn(0x7f5bc4079f90 msgr2=0x7f5bc407a3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:57:03.704 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:57:03.701+0000 7f5bcb487640 1 -- 172.21.15.39:0/816297753 shutdown_connections 2024-09-21T03:57:03.704 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:57:03.701+0000 7f5bcb487640 1 -- 172.21.15.39:0/816297753 wait complete. 2024-09-21T03:57:03.706 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:57:03.701+0000 7f5bcb487640 1 Processor -- start 2024-09-21T03:57:03.707 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:57:03.701+0000 7f5bcb487640 1 -- start start 2024-09-21T03:57:03.707 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:57:03.702+0000 7f5bcb487640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5bc4102240 0x7f5bc419a590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:57:03.707 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:57:03.702+0000 7f5bcb487640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5bc4107b80 0x7f5bc419ab30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:57:03.707 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:57:03.702+0000 7f5bcb487640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5bc41084d0 0x7f5bc419fdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:57:03.707 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:57:03.702+0000 7f5bcb487640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f5bc4112a10 con 0x7f5bc41084d0 2024-09-21T03:57:03.707 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:57:03.702+0000 7f5bcb487640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f5bc4112890 con 0x7f5bc4107b80 2024-09-21T03:57:03.707 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:57:03.702+0000 7f5bcb487640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f5bc4112b90 con 0x7f5bc4102240 2024-09-21T03:57:03.707 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:57:03.702+0000 7f5bc9a24640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5bc41084d0 0x7f5bc419fdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:57:03.707 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:57:03.702+0000 7f5bc9a24640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5bc41084d0 0x7f5bc419fdd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:36076/0 (socket says 172.21.15.39:36076) 2024-09-21T03:57:03.707 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:57:03.702+0000 7f5bc9a24640 1 -- 172.21.15.39:0/481819515 learned_addr learned my addr 172.21.15.39:0/481819515 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:57:03.707 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:57:03.702+0000 7f5bc9223640 1 --2- 172.21.15.39:0/481819515 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5bc4102240 0x7f5bc419a590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:57:03.707 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:57:03.702+0000 7f5bc8a22640 1 --2- 172.21.15.39:0/481819515 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5bc4107b80 0x7f5bc419ab30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:57:03.707 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:57:03.702+0000 7f5bc9a24640 1 -- 172.21.15.39:0/481819515 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5bc4102240 msgr2=0x7f5bc419a590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:57:03.708 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:57:03.702+0000 7f5bc9a24640 1 --2- 172.21.15.39:0/481819515 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5bc4102240 0x7f5bc419a590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:57:03.708 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:57:03.702+0000 7f5bc9a24640 1 -- 172.21.15.39:0/481819515 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5bc4107b80 msgr2=0x7f5bc419ab30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:57:03.708 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:57:03.702+0000 7f5bc9a24640 1 --2- 172.21.15.39:0/481819515 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5bc4107b80 0x7f5bc419ab30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:57:03.708 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:57:03.702+0000 7f5bc9a24640 1 -- 172.21.15.39:0/481819515 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5bc41a0500 con 0x7f5bc41084d0 2024-09-21T03:57:03.708 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:57:03.702+0000 7f5bc8a22640 1 --2- 172.21.15.39:0/481819515 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5bc4107b80 0x7f5bc419ab30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:57:03.708 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:57:03.702+0000 7f5bc9a24640 1 --2- 172.21.15.39:0/481819515 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5bc41084d0 0x7f5bc419fdd0 secure :-1 s=READY pgs=1375 cs=0 l=1 rev1=1 crypto rx=0x7f5bbc009fd0 tx=0x7f5bbc00b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:57:03.708 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:57:03.703+0000 7f5bba7fc640 1 -- 172.21.15.39:0/481819515 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bbc017070 con 0x7f5bc41084d0 2024-09-21T03:57:03.708 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:57:03.703+0000 7f5bba7fc640 1 -- 172.21.15.39:0/481819515 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5bbc003e20 con 0x7f5bc41084d0 2024-09-21T03:57:03.708 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:57:03.703+0000 7f5bba7fc640 1 -- 172.21.15.39:0/481819515 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bbc010460 con 0x7f5bc41084d0 2024-09-21T03:57:03.708 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:57:03.703+0000 7f5bcb487640 1 -- 172.21.15.39:0/481819515 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5bc41a07f0 con 0x7f5bc41084d0 2024-09-21T03:57:03.708 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:57:03.703+0000 7f5bcb487640 1 -- 172.21.15.39:0/481819515 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5bc41a6f50 con 0x7f5bc41084d0 2024-09-21T03:57:03.708 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:57:03.704+0000 7f5bcb487640 1 -- 172.21.15.39:0/481819515 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5bc41034c0 con 0x7f5bc41084d0 2024-09-21T03:57:03.709 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:57:03.706+0000 7f5bba7fc640 1 -- 172.21.15.39:0/481819515 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 27) ==== 154643+0+0 (secure 0 0 0) 0x7f5bbc003970 con 0x7f5bc41084d0 2024-09-21T03:57:03.710 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:57:03.707+0000 7f5bba7fc640 1 --2- 172.21.15.39:0/481819515 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f5bb00b6680 0x7f5bb00b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:57:03.710 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:57:03.707+0000 7f5bc9223640 1 --2- 172.21.15.39:0/481819515 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f5bb00b6680 0x7f5bb00b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:57:03.710 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:57:03.707+0000 7f5bba7fc640 1 -- 172.21.15.39:0/481819515 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8528+0+0 (secure 0 0 0) 0x7f5bbc0e7420 con 0x7f5bc41084d0 2024-09-21T03:57:03.711 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:57:03.708+0000 7f5bc9223640 1 --2- 172.21.15.39:0/481819515 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f5bb00b6680 0x7f5bb00b8b90 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f5bc0031440 tx=0x7f5bc0009560 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:57:03.713 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:57:03.710+0000 7f5bba7fc640 1 -- 172.21.15.39:0/481819515 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f5bbc0ad970 con 0x7f5bc41084d0 2024-09-21T03:57:03.717 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:57:03 smithi161 ceph-mon[34533]: pgmap v663: 97 pgs: 97 active+clean; 10 MiB data, 529 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 597 B/s wr, 10 op/s 2024-09-21T03:57:03.724 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:57:03 smithi150 ceph-mon[34289]: pgmap v663: 97 pgs: 97 active+clean; 10 MiB data, 529 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 597 B/s wr, 10 op/s 2024-09-21T03:57:03.937 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:57:03.934+0000 7f5bcb487640 1 -- 172.21.15.39:0/481819515 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f5bc4102650 con 0x7f5bc41084d0 2024-09-21T03:57:03.940 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:57:03.937+0000 7f5bba7fc640 1 -- 172.21.15.39:0/481819515 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v30) ==== 161+0+0 (secure 0 0 0) 0x7f5bc4102650 con 0x7f5bc41084d0 2024-09-21T03:57:03.940 INFO:teuthology.orchestra.run.smithi039.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-21T03:57:03.942 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:57:03.939+0000 7f5bcb487640 1 -- 172.21.15.39:0/481819515 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f5bb00b6680 msgr2=0x7f5bb00b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:57:03.943 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:57:03.939+0000 7f5bcb487640 1 --2- 172.21.15.39:0/481819515 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f5bb00b6680 0x7f5bb00b8b90 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f5bc0031440 tx=0x7f5bc0009560 comp rx=0 tx=0).stop 2024-09-21T03:57:03.943 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:57:03.939+0000 7f5bcb487640 1 -- 172.21.15.39:0/481819515 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5bc41084d0 msgr2=0x7f5bc419fdd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:57:03.943 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:57:03.939+0000 7f5bcb487640 1 --2- 172.21.15.39:0/481819515 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5bc41084d0 0x7f5bc419fdd0 secure :-1 s=READY pgs=1375 cs=0 l=1 rev1=1 crypto rx=0x7f5bbc009fd0 tx=0x7f5bbc00b260 comp rx=0 tx=0).stop 2024-09-21T03:57:03.943 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:57:03.940+0000 7f5bcb487640 1 -- 172.21.15.39:0/481819515 shutdown_connections 2024-09-21T03:57:03.943 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:57:03.940+0000 7f5bcb487640 1 --2- 172.21.15.39:0/481819515 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f5bb00b6680 0x7f5bb00b8b90 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:57:03.943 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:57:03.940+0000 7f5bcb487640 1 --2- 172.21.15.39:0/481819515 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5bc41084d0 0x7f5bc419fdd0 unknown :-1 s=CLOSED pgs=1375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:57:03.943 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:57:03.940+0000 7f5bcb487640 1 --2- 172.21.15.39:0/481819515 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5bc4107b80 0x7f5bc419ab30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:57:03.943 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:57:03.940+0000 7f5bcb487640 1 --2- 172.21.15.39:0/481819515 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5bc4102240 0x7f5bc419a590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:57:03.943 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:57:03.940+0000 7f5bcb487640 1 -- 172.21.15.39:0/481819515 >> 172.21.15.39:0/481819515 conn(0x7f5bc4079f90 msgr2=0x7f5bc4101160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:57:03.944 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:57:03 smithi039 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-mon-a[31207]: 2024-09-21T03:57:03.935+0000 7f5963e99640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-21T03:57:03.944 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:57:03.940+0000 7f5bcb487640 1 -- 172.21.15.39:0/481819515 shutdown_connections 2024-09-21T03:57:03.944 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:57:03.940+0000 7f5bcb487640 1 -- 172.21.15.39:0/481819515 wait complete. 2024-09-21T03:57:04.334 INFO:journalctl@ceph.mds.a.smithi039.stdout:Sep 21 03:57:04 smithi039 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-mds-a[73576]: ignoring --setuser ceph since I am not root 2024-09-21T03:57:04.334 INFO:journalctl@ceph.mds.a.smithi039.stdout:Sep 21 03:57:04 smithi039 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-mds-a[73576]: ignoring --setgroup ceph since I am not root 2024-09-21T03:57:04.334 INFO:journalctl@ceph.mds.a.smithi039.stdout:Sep 21 03:57:04 smithi039 ceph-mds[73580]: -- 172.21.15.39:0/1776096138 <== mon.0 v2:172.21.15.39:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55a5a1f17860 con 0x55a5a60bf600 2024-09-21T03:57:04.334 INFO:journalctl@ceph.mds.a.smithi039.stdout:Sep 21 03:57:04 smithi039 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-mds-a[73576]: starting mds.a at 2024-09-21T03:57:04.334 INFO:journalctl@ceph.mds.g.smithi039.stdout:Sep 21 03:57:04 smithi039 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-mds-g[73194]: ignoring --setuser ceph since I am not root 2024-09-21T03:57:04.335 INFO:journalctl@ceph.mds.g.smithi039.stdout:Sep 21 03:57:04 smithi039 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-mds-g[73194]: ignoring --setgroup ceph since I am not root 2024-09-21T03:57:04.335 INFO:journalctl@ceph.mds.g.smithi039.stdout:Sep 21 03:57:04 smithi039 ceph-mds[73216]: -- 172.21.15.39:0/51497149 <== mon.0 v2:172.21.15.39:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55f21d8d9c20 con 0x55f221a81a80 2024-09-21T03:57:04.335 INFO:journalctl@ceph.mds.g.smithi039.stdout:Sep 21 03:57:04 smithi039 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-mds-g[73194]: starting mds.g at 2024-09-21T03:57:04.418 INFO:journalctl@ceph.mds.f.smithi161.stdout:Sep 21 03:57:04 smithi161 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-mds-f[63361]: ignoring --setuser ceph since I am not root 2024-09-21T03:57:04.418 INFO:journalctl@ceph.mds.f.smithi161.stdout:Sep 21 03:57:04 smithi161 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-mds-f[63361]: ignoring --setgroup ceph since I am not root 2024-09-21T03:57:04.418 INFO:journalctl@ceph.mds.f.smithi161.stdout:Sep 21 03:57:04 smithi161 ceph-mds[63365]: -- 172.21.15.161:0/1940832909 <== mon.0 v2:172.21.15.39:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55c58f8db860 con 0x55c593a83180 2024-09-21T03:57:04.418 INFO:journalctl@ceph.mds.f.smithi161.stdout:Sep 21 03:57:04 smithi161 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-mds-f[63361]: starting mds.f at 2024-09-21T03:57:04.419 INFO:journalctl@ceph.mds.l.smithi161.stdout:Sep 21 03:57:04 smithi161 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-mds-l[63094]: ignoring --setuser ceph since I am not root 2024-09-21T03:57:04.419 INFO:journalctl@ceph.mds.l.smithi161.stdout:Sep 21 03:57:04 smithi161 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-mds-l[63094]: ignoring --setgroup ceph since I am not root 2024-09-21T03:57:04.419 INFO:journalctl@ceph.mds.l.smithi161.stdout:Sep 21 03:57:04 smithi161 ceph-mds[63098]: -- 172.21.15.161:0/2349116464 <== mon.1 v2:172.21.15.161:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5564989c3c20 con 0x55649cb6ba80 2024-09-21T03:57:04.419 INFO:journalctl@ceph.mds.l.smithi161.stdout:Sep 21 03:57:04 smithi161 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-mds-l[63094]: starting mds.l at 2024-09-21T03:57:04.419 INFO:journalctl@ceph.mds.c.smithi161.stdout:Sep 21 03:57:04 smithi161 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-mds-c[63630]: ignoring --setuser ceph since I am not root 2024-09-21T03:57:04.419 INFO:journalctl@ceph.mds.c.smithi161.stdout:Sep 21 03:57:04 smithi161 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-mds-c[63630]: ignoring --setgroup ceph since I am not root 2024-09-21T03:57:04.419 INFO:journalctl@ceph.mds.c.smithi161.stdout:Sep 21 03:57:04 smithi161 ceph-mds[63634]: -- 172.21.15.161:0/3739451458 <== mon.2 v2:172.21.15.150:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x56546e9a9860 con 0x565472b51180 2024-09-21T03:57:04.419 INFO:journalctl@ceph.mds.c.smithi161.stdout:Sep 21 03:57:04 smithi161 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-mds-c[63630]: starting mds.c at 2024-09-21T03:57:04.420 INFO:journalctl@ceph.mds.i.smithi161.stdout:Sep 21 03:57:04 smithi161 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-mds-i[63897]: ignoring --setuser ceph since I am not root 2024-09-21T03:57:04.420 INFO:journalctl@ceph.mds.i.smithi161.stdout:Sep 21 03:57:04 smithi161 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-mds-i[63897]: ignoring --setgroup ceph since I am not root 2024-09-21T03:57:04.420 INFO:journalctl@ceph.mds.i.smithi161.stdout:Sep 21 03:57:04 smithi161 ceph-mds[63901]: -- 172.21.15.161:0/1645643418 <== mon.2 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x5566e6332ea0 con 0x5566ea4d1a80 2024-09-21T03:57:04.420 INFO:journalctl@ceph.mds.i.smithi161.stdout:Sep 21 03:57:04 smithi161 ceph-mds[63901]: -- 172.21.15.161:0/1645643418 <== mon.2 v2:172.21.15.150:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5566e6329860 con 0x5566ea4d1a80 2024-09-21T03:57:04.420 INFO:journalctl@ceph.mds.i.smithi161.stdout:Sep 21 03:57:04 smithi161 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-mds-i[63897]: starting mds.i at 2024-09-21T03:57:04.420 INFO:journalctl@ceph.mds.b.smithi150.stdout:Sep 21 03:57:04 smithi150 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-mds-b[63521]: ignoring --setuser ceph since I am not root 2024-09-21T03:57:04.420 INFO:journalctl@ceph.mds.b.smithi150.stdout:Sep 21 03:57:04 smithi150 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-mds-b[63521]: ignoring --setgroup ceph since I am not root 2024-09-21T03:57:04.420 INFO:journalctl@ceph.mds.b.smithi150.stdout:Sep 21 03:57:04 smithi150 ceph-mds[63525]: -- 172.21.15.150:0/1474928334 <== mon.1 v2:172.21.15.161:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x557adcebbc20 con 0x557ae1063a80 2024-09-21T03:57:04.420 INFO:journalctl@ceph.mds.b.smithi150.stdout:Sep 21 03:57:04 smithi150 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-mds-b[63521]: starting mds.b at 2024-09-21T03:57:04.421 INFO:journalctl@ceph.mds.b.smithi150.stdout:Sep 21 03:57:04 smithi150 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-mds-b[63521]: 2024-09-21T03:57:04.014+0000 7fab48cee640 -1 PosixStack listen unable to listen on v1:0.0.0.0:6837/0: (98) Address already in use 2024-09-21T03:57:04.421 INFO:journalctl@ceph.mds.e.smithi150.stdout:Sep 21 03:57:04 smithi150 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-mds-e[62978]: ignoring --setuser ceph since I am not root 2024-09-21T03:57:04.421 INFO:journalctl@ceph.mds.e.smithi150.stdout:Sep 21 03:57:04 smithi150 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-mds-e[62978]: ignoring --setgroup ceph since I am not root 2024-09-21T03:57:04.421 INFO:journalctl@ceph.mds.e.smithi150.stdout:Sep 21 03:57:04 smithi150 ceph-mds[62982]: -- 172.21.15.150:0/1681545397 <== mon.0 v2:172.21.15.39:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x56400fd73860 con 0x564013f1ba80 2024-09-21T03:57:04.421 INFO:journalctl@ceph.mds.e.smithi150.stdout:Sep 21 03:57:04 smithi150 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-mds-e[62978]: starting mds.e at 2024-09-21T03:57:04.421 INFO:journalctl@ceph.mds.h.smithi150.stdout:Sep 21 03:57:04 smithi150 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-mds-h[63250]: ignoring --setuser ceph since I am not root 2024-09-21T03:57:04.421 INFO:journalctl@ceph.mds.h.smithi150.stdout:Sep 21 03:57:04 smithi150 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-mds-h[63250]: ignoring --setgroup ceph since I am not root 2024-09-21T03:57:04.421 INFO:journalctl@ceph.mds.h.smithi150.stdout:Sep 21 03:57:04 smithi150 ceph-mds[63254]: --2- 172.21.15.150:0/181393983 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558b7999d180 0x558b7a594000 secure :-1 s=READY pgs=1380 cs=0 l=1 rev1=1 crypto rx=0x558b799aeba0 tx=0x558b798d4540 comp rx=0 tx=0).write_message sending message m=0x558b798628c0 seq=3 auth(proto 2 2 bytes epoch 0) 2024-09-21T03:57:04.422 INFO:journalctl@ceph.mds.h.smithi150.stdout:Sep 21 03:57:04 smithi150 ceph-mds[63254]: -- 172.21.15.150:0/181393983 <== mon.0 v2:172.21.15.39:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x558b757f5c20 con 0x558b7999d180 2024-09-21T03:57:04.422 INFO:journalctl@ceph.mds.h.smithi150.stdout:Sep 21 03:57:04 smithi150 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-mds-h[63250]: starting mds.h at 2024-09-21T03:57:04.422 INFO:journalctl@ceph.mds.k.smithi150.stdout:Sep 21 03:57:04 smithi150 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-mds-k[62698]: ignoring --setuser ceph since I am not root 2024-09-21T03:57:04.422 INFO:journalctl@ceph.mds.k.smithi150.stdout:Sep 21 03:57:04 smithi150 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-mds-k[62698]: ignoring --setgroup ceph since I am not root 2024-09-21T03:57:04.422 INFO:journalctl@ceph.mds.k.smithi150.stdout:Sep 21 03:57:04 smithi150 ceph-mds[62702]: -- 172.21.15.150:0/2094967803 <== mon.0 v2:172.21.15.39:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x564e1d285860 con 0x564e2142d180 2024-09-21T03:57:04.422 INFO:journalctl@ceph.mds.k.smithi150.stdout:Sep 21 03:57:04 smithi150 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-mds-k[62698]: starting mds.k at 2024-09-21T03:57:04.422 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:57:04 smithi150 ceph-mon[34289]: from='client.? 172.21.15.39:0/481819515' entity='client.admin' cmd={"prefix": "fs fail", "fs_name": "cephfs"} : dispatch 2024-09-21T03:57:04.422 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:57:04 smithi150 ceph-mon[34289]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-09-21T03:57:04.422 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:57:04 smithi150 ceph-mon[34289]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-21T03:57:04.423 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:57:04 smithi150 ceph-mon[34289]: from='client.? 172.21.15.39:0/481819515' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-09-21T03:57:04.423 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:57:04 smithi150 ceph-mon[34289]: osdmap e88: 12 total, 12 up, 12 in 2024-09-21T03:57:04.423 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:57:04 smithi150 ceph-mon[34289]: fsmap cephfs:0/5 2 up:standby, 5 failed 2024-09-21T03:57:04.610 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 51116d5c-77ca-11ef-baf5-efdc52797490 -- ceph fs rm cephfs --yes-i-really-mean-it 2024-09-21T03:57:04.635 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:57:04 smithi039 ceph-mon[31211]: from='client.? 172.21.15.39:0/481819515' entity='client.admin' cmd={"prefix": "fs fail", "fs_name": "cephfs"} : dispatch 2024-09-21T03:57:04.635 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:57:04 smithi039 ceph-mon[31211]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-09-21T03:57:04.635 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:57:04 smithi039 ceph-mon[31211]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-21T03:57:04.635 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:57:04 smithi039 ceph-mon[31211]: from='client.? 172.21.15.39:0/481819515' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-09-21T03:57:04.635 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:57:04 smithi039 ceph-mon[31211]: osdmap e88: 12 total, 12 up, 12 in 2024-09-21T03:57:04.635 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:57:04 smithi039 ceph-mon[31211]: fsmap cephfs:0/5 2 up:standby, 5 failed 2024-09-21T03:57:04.718 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:57:04 smithi161 ceph-mon[34533]: from='client.? 172.21.15.39:0/481819515' entity='client.admin' cmd={"prefix": "fs fail", "fs_name": "cephfs"} : dispatch 2024-09-21T03:57:04.718 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:57:04 smithi161 ceph-mon[34533]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-09-21T03:57:04.718 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:57:04 smithi161 ceph-mon[34533]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-21T03:57:04.718 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:57:04 smithi161 ceph-mon[34533]: from='client.? 172.21.15.39:0/481819515' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-09-21T03:57:04.718 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:57:04 smithi161 ceph-mon[34533]: osdmap e88: 12 total, 12 up, 12 in 2024-09-21T03:57:04.718 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:57:04 smithi161 ceph-mon[34533]: fsmap cephfs:0/5 2 up:standby, 5 failed 2024-09-21T03:57:04.867 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/51116d5c-77ca-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:57:05.360 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:57:05.351+0000 7f532c3fb640 1 -- 172.21.15.39:0/2998591274 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5324108cf0 msgr2=0x7f532410f520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:57:05.360 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:57:05.351+0000 7f532c3fb640 1 --2- 172.21.15.39:0/2998591274 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5324108cf0 0x7f532410f520 secure :-1 s=READY pgs=1384 cs=0 l=1 rev1=1 crypto rx=0x7f531c009ca0 tx=0x7f531c02a8c0 comp rx=0 tx=0).stop 2024-09-21T03:57:05.360 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:57:05.351+0000 7f532c3fb640 1 -- 172.21.15.39:0/2998591274 shutdown_connections 2024-09-21T03:57:05.360 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:57:05.351+0000 7f532c3fb640 1 --2- 172.21.15.39:0/2998591274 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5324108cf0 0x7f532410f520 unknown :-1 s=CLOSED pgs=1384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:57:05.360 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:57:05.351+0000 7f532c3fb640 1 --2- 172.21.15.39:0/2998591274 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f53241083a0 0x7f53241087b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:57:05.361 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:57:05.351+0000 7f532c3fb640 1 --2- 172.21.15.39:0/2998591274 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5324102a60 0x7f5324102e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:57:05.361 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:57:05.351+0000 7f532c3fb640 1 -- 172.21.15.39:0/2998591274 >> 172.21.15.39:0/2998591274 conn(0x7f5324079730 msgr2=0x7f5324079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:57:05.361 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:57:05.351+0000 7f532c3fb640 1 -- 172.21.15.39:0/2998591274 shutdown_connections 2024-09-21T03:57:05.361 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:57:05.351+0000 7f532c3fb640 1 -- 172.21.15.39:0/2998591274 wait complete. 2024-09-21T03:57:05.361 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:57:05.352+0000 7f532c3fb640 1 Processor -- start 2024-09-21T03:57:05.361 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:57:05.352+0000 7f532c3fb640 1 -- start start 2024-09-21T03:57:05.361 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:57:05.352+0000 7f532c3fb640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5324102a60 0x7f532419a510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:57:05.361 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:57:05.352+0000 7f532c3fb640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f53241083a0 0x7f532419aab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:57:05.361 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:57:05.352+0000 7f532c3fb640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5324108cf0 0x7f532419fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:57:05.361 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:57:05.352+0000 7f532c3fb640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f5324113ac0 con 0x7f53241083a0 2024-09-21T03:57:05.361 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:57:05.352+0000 7f532c3fb640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f5324113940 con 0x7f5324102a60 2024-09-21T03:57:05.361 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:57:05.352+0000 7f532c3fb640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f5324113c40 con 0x7f5324108cf0 2024-09-21T03:57:05.361 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:57:05.352+0000 7f5329996640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f53241083a0 0x7f532419aab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:57:05.361 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:57:05.352+0000 7f5329996640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f53241083a0 0x7f532419aab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:36128/0 (socket says 172.21.15.39:36128) 2024-09-21T03:57:05.362 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:57:05.352+0000 7f5329996640 1 -- 172.21.15.39:0/3690398350 learned_addr learned my addr 172.21.15.39:0/3690398350 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:57:05.362 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:57:05.353+0000 7f532a998640 1 --2- 172.21.15.39:0/3690398350 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5324108cf0 0x7f532419fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:57:05.362 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:57:05.353+0000 7f532a197640 1 --2- 172.21.15.39:0/3690398350 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5324102a60 0x7f532419a510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:57:05.362 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:57:05.353+0000 7f5329996640 1 -- 172.21.15.39:0/3690398350 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5324108cf0 msgr2=0x7f532419fd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:57:05.362 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:57:05.353+0000 7f5329996640 1 --2- 172.21.15.39:0/3690398350 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5324108cf0 0x7f532419fd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:57:05.362 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:57:05.353+0000 7f5329996640 1 -- 172.21.15.39:0/3690398350 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5324102a60 msgr2=0x7f532419a510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:57:05.362 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:57:05.353+0000 7f5329996640 1 --2- 172.21.15.39:0/3690398350 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5324102a60 0x7f532419a510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:57:05.362 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:57:05.353+0000 7f5329996640 1 -- 172.21.15.39:0/3690398350 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f53241a0480 con 0x7f53241083a0 2024-09-21T03:57:05.362 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:57:05.353+0000 7f532a998640 1 --2- 172.21.15.39:0/3690398350 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5324108cf0 0x7f532419fd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:57:05.362 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:57:05.353+0000 7f5329996640 1 --2- 172.21.15.39:0/3690398350 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f53241083a0 0x7f532419aab0 secure :-1 s=READY pgs=1385 cs=0 l=1 rev1=1 crypto rx=0x7f532000b450 tx=0x7f532000b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:57:05.362 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:57:05.353+0000 7f53177fe640 1 -- 172.21.15.39:0/3690398350 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5320015020 con 0x7f53241083a0 2024-09-21T03:57:05.362 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:57:05.353+0000 7f53177fe640 1 -- 172.21.15.39:0/3690398350 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5320011d40 con 0x7f53241083a0 2024-09-21T03:57:05.362 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:57:05.353+0000 7f53177fe640 1 -- 172.21.15.39:0/3690398350 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f532000f340 con 0x7f53241083a0 2024-09-21T03:57:05.362 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:57:05.353+0000 7f532c3fb640 1 -- 172.21.15.39:0/3690398350 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f53241a0770 con 0x7f53241083a0 2024-09-21T03:57:05.363 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:57:05.353+0000 7f532c3fb640 1 -- 172.21.15.39:0/3690398350 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f53241a6ed0 con 0x7f53241083a0 2024-09-21T03:57:05.363 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:57:05.354+0000 7f532c3fb640 1 -- 172.21.15.39:0/3690398350 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5324102e70 con 0x7f53241083a0 2024-09-21T03:57:05.363 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:57:05.356+0000 7f53177fe640 1 -- 172.21.15.39:0/3690398350 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 27) ==== 154643+0+0 (secure 0 0 0) 0x7f5320018020 con 0x7f53241083a0 2024-09-21T03:57:05.363 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:57:05.357+0000 7f53177fe640 1 --2- 172.21.15.39:0/3690398350 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f53100b6700 0x7f53100b8c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:57:05.363 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:57:05.357+0000 7f532a197640 1 --2- 172.21.15.39:0/3690398350 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f53100b6700 0x7f53100b8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:57:05.363 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:57:05.357+0000 7f53177fe640 1 -- 172.21.15.39:0/3690398350 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8914+0+0 (secure 0 0 0) 0x7f53200e7830 con 0x7f53241083a0 2024-09-21T03:57:05.363 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:57:05.358+0000 7f532a197640 1 --2- 172.21.15.39:0/3690398350 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f53100b6700 0x7f53100b8c10 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f53180058c0 tx=0x7f531800a4c0 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:57:05.363 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:57:05.360+0000 7f53177fe640 1 -- 172.21.15.39:0/3690398350 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f532001f020 con 0x7f53241083a0 2024-09-21T03:57:05.422 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:57:05 smithi150 ceph-mon[34289]: pgmap v664: 97 pgs: 97 active+clean; 10 MiB data, 530 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 1.1 KiB/s wr, 12 op/s 2024-09-21T03:57:05.422 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:57:05 smithi150 ceph-mon[34289]: mds.? [v2:172.21.15.39:6839/3206215624,v1:172.21.15.39:6841/3206215624] up:boot 2024-09-21T03:57:05.422 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:57:05 smithi150 ceph-mon[34289]: mds.? [v2:172.21.15.39:6838/609447534,v1:172.21.15.39:6840/609447534] up:boot 2024-09-21T03:57:05.422 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:57:05 smithi150 ceph-mon[34289]: mds.? [v2:172.21.15.150:6832/2171775868,v1:172.21.15.150:6836/2171775868] up:boot 2024-09-21T03:57:05.422 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:57:05 smithi150 ceph-mon[34289]: mds.? [v2:172.21.15.150:6833/3158090225,v1:172.21.15.150:6838/3158090225] up:boot 2024-09-21T03:57:05.422 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:57:05 smithi150 ceph-mon[34289]: mds.? [v2:172.21.15.161:6832/1593316723,v1:172.21.15.161:6834/1593316723] up:boot 2024-09-21T03:57:05.423 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:57:05 smithi150 ceph-mon[34289]: mds.? [v2:172.21.15.161:6833/999738480,v1:172.21.15.161:6835/999738480] up:boot 2024-09-21T03:57:05.423 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:57:05 smithi150 ceph-mon[34289]: mds.? [v2:172.21.15.150:6835/982999343,v1:172.21.15.150:6839/982999343] up:boot 2024-09-21T03:57:05.423 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:57:05 smithi150 ceph-mon[34289]: mds.? [v2:172.21.15.150:6834/373880172,v1:172.21.15.150:6837/373880172] up:boot 2024-09-21T03:57:05.423 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:57:05 smithi150 ceph-mon[34289]: mds.? [v2:172.21.15.161:6836/2191416795,v1:172.21.15.161:6837/2191416795] up:boot 2024-09-21T03:57:05.423 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:57:05 smithi150 ceph-mon[34289]: mds.? [v2:172.21.15.161:6838/832281368,v1:172.21.15.161:6839/832281368] up:boot 2024-09-21T03:57:05.423 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:57:05 smithi150 ceph-mon[34289]: fsmap cephfs:0/5 12 up:standby, 5 failed 2024-09-21T03:57:05.423 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:57:05 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "g"} : dispatch 2024-09-21T03:57:05.423 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:57:05 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "a"} : dispatch 2024-09-21T03:57:05.423 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:57:05 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "k"} : dispatch 2024-09-21T03:57:05.423 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:57:05 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "b"} : dispatch 2024-09-21T03:57:05.423 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:57:05 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "h"} : dispatch 2024-09-21T03:57:05.423 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:57:05 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "i"} : dispatch 2024-09-21T03:57:05.423 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:57:05 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "e"} : dispatch 2024-09-21T03:57:05.423 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:57:05 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "c"} : dispatch 2024-09-21T03:57:05.423 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:57:05 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "f"} : dispatch 2024-09-21T03:57:05.424 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:57:05 smithi150 ceph-mon[34289]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "l"} : dispatch 2024-09-21T03:57:05.584 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:57:05 smithi039 ceph-mon[31211]: pgmap v664: 97 pgs: 97 active+clean; 10 MiB data, 530 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 1.1 KiB/s wr, 12 op/s 2024-09-21T03:57:05.584 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:57:05 smithi039 ceph-mon[31211]: mds.? [v2:172.21.15.39:6839/3206215624,v1:172.21.15.39:6841/3206215624] up:boot 2024-09-21T03:57:05.584 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:57:05 smithi039 ceph-mon[31211]: mds.? [v2:172.21.15.39:6838/609447534,v1:172.21.15.39:6840/609447534] up:boot 2024-09-21T03:57:05.584 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:57:05 smithi039 ceph-mon[31211]: mds.? [v2:172.21.15.150:6832/2171775868,v1:172.21.15.150:6836/2171775868] up:boot 2024-09-21T03:57:05.584 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:57:05 smithi039 ceph-mon[31211]: mds.? [v2:172.21.15.150:6833/3158090225,v1:172.21.15.150:6838/3158090225] up:boot 2024-09-21T03:57:05.584 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:57:05 smithi039 ceph-mon[31211]: mds.? [v2:172.21.15.161:6832/1593316723,v1:172.21.15.161:6834/1593316723] up:boot 2024-09-21T03:57:05.585 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:57:05 smithi039 ceph-mon[31211]: mds.? [v2:172.21.15.161:6833/999738480,v1:172.21.15.161:6835/999738480] up:boot 2024-09-21T03:57:05.585 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:57:05 smithi039 ceph-mon[31211]: mds.? [v2:172.21.15.150:6835/982999343,v1:172.21.15.150:6839/982999343] up:boot 2024-09-21T03:57:05.585 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:57:05 smithi039 ceph-mon[31211]: mds.? [v2:172.21.15.150:6834/373880172,v1:172.21.15.150:6837/373880172] up:boot 2024-09-21T03:57:05.585 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:57:05 smithi039 ceph-mon[31211]: mds.? [v2:172.21.15.161:6836/2191416795,v1:172.21.15.161:6837/2191416795] up:boot 2024-09-21T03:57:05.585 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:57:05 smithi039 ceph-mon[31211]: mds.? [v2:172.21.15.161:6838/832281368,v1:172.21.15.161:6839/832281368] up:boot 2024-09-21T03:57:05.585 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:57:05 smithi039 ceph-mon[31211]: fsmap cephfs:0/5 12 up:standby, 5 failed 2024-09-21T03:57:05.585 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:57:05 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "g"} : dispatch 2024-09-21T03:57:05.585 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:57:05 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "a"} : dispatch 2024-09-21T03:57:05.585 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:57:05 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "k"} : dispatch 2024-09-21T03:57:05.585 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:57:05 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "b"} : dispatch 2024-09-21T03:57:05.585 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:57:05 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "h"} : dispatch 2024-09-21T03:57:05.585 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:57:05 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "i"} : dispatch 2024-09-21T03:57:05.585 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:57:05 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "e"} : dispatch 2024-09-21T03:57:05.585 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:57:05 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "c"} : dispatch 2024-09-21T03:57:05.586 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:57:05 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "f"} : dispatch 2024-09-21T03:57:05.586 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:57:05 smithi039 ceph-mon[31211]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "l"} : dispatch 2024-09-21T03:57:05.593 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:57:05.591+0000 7f532c3fb640 1 -- 172.21.15.39:0/3690398350 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f532402e640 con 0x7f53241083a0 2024-09-21T03:57:05.718 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:57:05 smithi161 ceph-mon[34533]: pgmap v664: 97 pgs: 97 active+clean; 10 MiB data, 530 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 1.1 KiB/s wr, 12 op/s 2024-09-21T03:57:05.718 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:57:05 smithi161 ceph-mon[34533]: mds.? [v2:172.21.15.39:6839/3206215624,v1:172.21.15.39:6841/3206215624] up:boot 2024-09-21T03:57:05.718 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:57:05 smithi161 ceph-mon[34533]: mds.? [v2:172.21.15.39:6838/609447534,v1:172.21.15.39:6840/609447534] up:boot 2024-09-21T03:57:05.718 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:57:05 smithi161 ceph-mon[34533]: mds.? [v2:172.21.15.150:6832/2171775868,v1:172.21.15.150:6836/2171775868] up:boot 2024-09-21T03:57:05.718 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:57:05 smithi161 ceph-mon[34533]: mds.? [v2:172.21.15.150:6833/3158090225,v1:172.21.15.150:6838/3158090225] up:boot 2024-09-21T03:57:05.718 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:57:05 smithi161 ceph-mon[34533]: mds.? [v2:172.21.15.161:6832/1593316723,v1:172.21.15.161:6834/1593316723] up:boot 2024-09-21T03:57:05.719 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:57:05 smithi161 ceph-mon[34533]: mds.? [v2:172.21.15.161:6833/999738480,v1:172.21.15.161:6835/999738480] up:boot 2024-09-21T03:57:05.719 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:57:05 smithi161 ceph-mon[34533]: mds.? [v2:172.21.15.150:6835/982999343,v1:172.21.15.150:6839/982999343] up:boot 2024-09-21T03:57:05.719 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:57:05 smithi161 ceph-mon[34533]: mds.? [v2:172.21.15.150:6834/373880172,v1:172.21.15.150:6837/373880172] up:boot 2024-09-21T03:57:05.719 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:57:05 smithi161 ceph-mon[34533]: mds.? [v2:172.21.15.161:6836/2191416795,v1:172.21.15.161:6837/2191416795] up:boot 2024-09-21T03:57:05.719 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:57:05 smithi161 ceph-mon[34533]: mds.? [v2:172.21.15.161:6838/832281368,v1:172.21.15.161:6839/832281368] up:boot 2024-09-21T03:57:05.719 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:57:05 smithi161 ceph-mon[34533]: fsmap cephfs:0/5 12 up:standby, 5 failed 2024-09-21T03:57:05.719 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:57:05 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "g"} : dispatch 2024-09-21T03:57:05.719 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:57:05 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "a"} : dispatch 2024-09-21T03:57:05.719 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:57:05 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "k"} : dispatch 2024-09-21T03:57:05.719 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:57:05 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "b"} : dispatch 2024-09-21T03:57:05.719 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:57:05 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "h"} : dispatch 2024-09-21T03:57:05.719 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:57:05 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "i"} : dispatch 2024-09-21T03:57:05.719 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:57:05 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "e"} : dispatch 2024-09-21T03:57:05.719 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:57:05 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "c"} : dispatch 2024-09-21T03:57:05.719 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:57:05 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "f"} : dispatch 2024-09-21T03:57:05.720 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:57:05 smithi161 ceph-mon[34533]: from='mgr.14223 172.21.15.39:0/1962136088' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "l"} : dispatch 2024-09-21T03:57:06.429 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:57:06.426+0000 7f53177fe640 1 -- 172.21.15.39:0/3690398350 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v32) ==== 104+0+0 (secure 0 0 0) 0x7f532000d040 con 0x7f53241083a0 2024-09-21T03:57:06.434 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:57:06.430+0000 7f532c3fb640 1 -- 172.21.15.39:0/3690398350 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f53100b6700 msgr2=0x7f53100b8c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:57:06.434 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:57:06.430+0000 7f532c3fb640 1 --2- 172.21.15.39:0/3690398350 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f53100b6700 0x7f53100b8c10 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f53180058c0 tx=0x7f531800a4c0 comp rx=0 tx=0).stop 2024-09-21T03:57:06.434 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:57:06.430+0000 7f532c3fb640 1 -- 172.21.15.39:0/3690398350 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f53241083a0 msgr2=0x7f532419aab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:57:06.434 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:57:06.430+0000 7f532c3fb640 1 --2- 172.21.15.39:0/3690398350 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f53241083a0 0x7f532419aab0 secure :-1 s=READY pgs=1385 cs=0 l=1 rev1=1 crypto rx=0x7f532000b450 tx=0x7f532000b910 comp rx=0 tx=0).stop 2024-09-21T03:57:06.434 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:57:06.431+0000 7f532c3fb640 1 -- 172.21.15.39:0/3690398350 shutdown_connections 2024-09-21T03:57:06.434 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:57:06.431+0000 7f532c3fb640 1 --2- 172.21.15.39:0/3690398350 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f53100b6700 0x7f53100b8c10 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:57:06.434 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:57:06.431+0000 7f532c3fb640 1 --2- 172.21.15.39:0/3690398350 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5324108cf0 0x7f532419fd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:57:06.434 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:57:06.431+0000 7f532c3fb640 1 --2- 172.21.15.39:0/3690398350 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f53241083a0 0x7f532419aab0 unknown :-1 s=CLOSED pgs=1385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:57:06.434 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:57:06.431+0000 7f532c3fb640 1 --2- 172.21.15.39:0/3690398350 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5324102a60 0x7f532419a510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:57:06.434 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:57:06.431+0000 7f532c3fb640 1 -- 172.21.15.39:0/3690398350 >> 172.21.15.39:0/3690398350 conn(0x7f5324079730 msgr2=0x7f53241052a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:57:06.434 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:57:06.431+0000 7f532c3fb640 1 -- 172.21.15.39:0/3690398350 shutdown_connections 2024-09-21T03:57:06.434 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:57:06.431+0000 7f532c3fb640 1 -- 172.21.15.39:0/3690398350 wait complete. 2024-09-21T03:57:06.718 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:57:06 smithi161 ceph-mon[34533]: from='client.? 172.21.15.39:0/3690398350' entity='client.admin' cmd={"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} : dispatch 2024-09-21T03:57:06.725 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:57:06 smithi150 ceph-mon[34289]: from='client.? 172.21.15.39:0/3690398350' entity='client.admin' cmd={"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} : dispatch 2024-09-21T03:57:06.789 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:57:06 smithi039 ceph-mon[31211]: from='client.? 172.21.15.39:0/3690398350' entity='client.admin' cmd={"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} : dispatch 2024-09-21T03:57:07.252 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 51116d5c-77ca-11ef-baf5-efdc52797490 -- ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-09-21T03:57:07.561 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/51116d5c-77ca-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:57:07.583 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:57:07 smithi039 ceph-mon[31211]: pgmap v666: 97 pgs: 97 active+clean; 10 MiB data, 530 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 54 KiB/s wr, 31 op/s 2024-09-21T03:57:07.584 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:57:07 smithi039 ceph-mon[31211]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-09-21T03:57:07.584 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:57:07 smithi039 ceph-mon[31211]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-09-21T03:57:07.584 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:57:07 smithi039 ceph-mon[31211]: Cluster is now healthy 2024-09-21T03:57:07.584 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:57:07 smithi039 ceph-mon[31211]: from='client.? 172.21.15.39:0/3690398350' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-09-21T03:57:07.584 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:57:07 smithi039 ceph-mon[31211]: fsmap 12 up:standby 2024-09-21T03:57:07.718 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:57:07 smithi161 ceph-mon[34533]: pgmap v666: 97 pgs: 97 active+clean; 10 MiB data, 530 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 54 KiB/s wr, 31 op/s 2024-09-21T03:57:07.718 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:57:07 smithi161 ceph-mon[34533]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-09-21T03:57:07.718 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:57:07 smithi161 ceph-mon[34533]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-09-21T03:57:07.718 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:57:07 smithi161 ceph-mon[34533]: Cluster is now healthy 2024-09-21T03:57:07.718 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:57:07 smithi161 ceph-mon[34533]: from='client.? 172.21.15.39:0/3690398350' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-09-21T03:57:07.718 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:57:07 smithi161 ceph-mon[34533]: fsmap 12 up:standby 2024-09-21T03:57:07.725 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:57:07 smithi150 ceph-mon[34289]: pgmap v666: 97 pgs: 97 active+clean; 10 MiB data, 530 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 54 KiB/s wr, 31 op/s 2024-09-21T03:57:07.725 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:57:07 smithi150 ceph-mon[34289]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-09-21T03:57:07.725 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:57:07 smithi150 ceph-mon[34289]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-09-21T03:57:07.725 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:57:07 smithi150 ceph-mon[34289]: Cluster is now healthy 2024-09-21T03:57:07.725 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:57:07 smithi150 ceph-mon[34289]: from='client.? 172.21.15.39:0/3690398350' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-09-21T03:57:07.725 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:57:07 smithi150 ceph-mon[34289]: fsmap 12 up:standby 2024-09-21T03:57:08.027 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:57:08.022+0000 7f671e78e640 1 -- 172.21.15.39:0/3912517434 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6718108d10 msgr2=0x7f671810f540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:57:08.028 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:57:08.022+0000 7f671e78e640 1 --2- 172.21.15.39:0/3912517434 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6718108d10 0x7f671810f540 secure :-1 s=READY pgs=1386 cs=0 l=1 rev1=1 crypto rx=0x7f6710009ca0 tx=0x7f671002a8c0 comp rx=0 tx=0).stop 2024-09-21T03:57:08.028 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:57:08.022+0000 7f671e78e640 1 -- 172.21.15.39:0/3912517434 shutdown_connections 2024-09-21T03:57:08.028 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:57:08.022+0000 7f671e78e640 1 --2- 172.21.15.39:0/3912517434 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6718108d10 0x7f671810f540 unknown :-1 s=CLOSED pgs=1386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:57:08.028 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:57:08.022+0000 7f671e78e640 1 --2- 172.21.15.39:0/3912517434 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f67181083c0 0x7f67181087d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:57:08.028 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:57:08.022+0000 7f671e78e640 1 --2- 172.21.15.39:0/3912517434 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6718102a80 0x7f6718102e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:57:08.028 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:57:08.022+0000 7f671e78e640 1 -- 172.21.15.39:0/3912517434 >> 172.21.15.39:0/3912517434 conn(0x7f6718079740 msgr2=0x7f6718079b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:57:08.028 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:57:08.022+0000 7f671e78e640 1 -- 172.21.15.39:0/3912517434 shutdown_connections 2024-09-21T03:57:08.028 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:57:08.022+0000 7f671e78e640 1 -- 172.21.15.39:0/3912517434 wait complete. 2024-09-21T03:57:08.028 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:57:08.023+0000 7f671e78e640 1 Processor -- start 2024-09-21T03:57:08.028 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:57:08.023+0000 7f671e78e640 1 -- start start 2024-09-21T03:57:08.028 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:57:08.023+0000 7f671e78e640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6718102a80 0x7f6718199370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:57:08.028 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:57:08.023+0000 7f671e78e640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f67181083c0 0x7f6718199910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:57:08.028 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:57:08.023+0000 7f671e78e640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6718108d10 0x7f671819ebb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:57:08.028 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:57:08.023+0000 7f671e78e640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f6718112980 con 0x7f6718102a80 2024-09-21T03:57:08.029 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:57:08.023+0000 7f671e78e640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f6718112800 con 0x7f6718108d10 2024-09-21T03:57:08.029 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:57:08.023+0000 7f671e78e640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f6718112b00 con 0x7f67181083c0 2024-09-21T03:57:08.029 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:57:08.023+0000 7f6717fff640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6718102a80 0x7f6718199370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:57:08.029 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:57:08.023+0000 7f6717fff640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6718102a80 0x7f6718199370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:53220/0 (socket says 172.21.15.39:53220) 2024-09-21T03:57:08.029 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:57:08.023+0000 7f6717fff640 1 -- 172.21.15.39:0/2194765651 learned_addr learned my addr 172.21.15.39:0/2194765651 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:57:08.029 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:57:08.023+0000 7f67177fe640 1 --2- 172.21.15.39:0/2194765651 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f67181083c0 0x7f6718199910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:57:08.029 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:57:08.024+0000 7f671cd2b640 1 --2- 172.21.15.39:0/2194765651 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6718108d10 0x7f671819ebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:57:08.029 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:57:08.024+0000 7f6717fff640 1 -- 172.21.15.39:0/2194765651 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f67181083c0 msgr2=0x7f6718199910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:57:08.029 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:57:08.024+0000 7f6717fff640 1 --2- 172.21.15.39:0/2194765651 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f67181083c0 0x7f6718199910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:57:08.029 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:57:08.024+0000 7f6717fff640 1 -- 172.21.15.39:0/2194765651 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6718108d10 msgr2=0x7f671819ebb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:57:08.029 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:57:08.024+0000 7f6717fff640 1 --2- 172.21.15.39:0/2194765651 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6718108d10 0x7f671819ebb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:57:08.029 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:57:08.024+0000 7f6717fff640 1 -- 172.21.15.39:0/2194765651 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f671819f2e0 con 0x7f6718102a80 2024-09-21T03:57:08.029 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:57:08.024+0000 7f6717fff640 1 --2- 172.21.15.39:0/2194765651 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6718102a80 0x7f6718199370 secure :-1 s=READY pgs=1387 cs=0 l=1 rev1=1 crypto rx=0x7f670800c890 tx=0x7f670800cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:57:08.029 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:57:08.024+0000 7f67157fa640 1 -- 172.21.15.39:0/2194765651 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6708004220 con 0x7f6718102a80 2024-09-21T03:57:08.029 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:57:08.024+0000 7f671e78e640 1 -- 172.21.15.39:0/2194765651 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f671819f5d0 con 0x7f6718102a80 2024-09-21T03:57:08.030 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:57:08.024+0000 7f67157fa640 1 -- 172.21.15.39:0/2194765651 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f67080043c0 con 0x7f6718102a80 2024-09-21T03:57:08.030 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:57:08.025+0000 7f67157fa640 1 -- 172.21.15.39:0/2194765651 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6708004a00 con 0x7f6718102a80 2024-09-21T03:57:08.030 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:57:08.025+0000 7f671e78e640 1 -- 172.21.15.39:0/2194765651 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f671806bb40 con 0x7f6718102a80 2024-09-21T03:57:08.030 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:57:08.026+0000 7f671e78e640 1 -- 172.21.15.39:0/2194765651 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6718103d00 con 0x7f6718102a80 2024-09-21T03:57:08.030 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:57:08.027+0000 7f67157fa640 1 -- 172.21.15.39:0/2194765651 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 27) ==== 154643+0+0 (secure 0 0 0) 0x7f67080079b0 con 0x7f6718102a80 2024-09-21T03:57:08.031 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:57:08.028+0000 7f67157fa640 1 --2- 172.21.15.39:0/2194765651 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f67000b6680 0x7f67000b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:57:08.031 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:57:08.028+0000 7f67177fe640 1 --2- 172.21.15.39:0/2194765651 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f67000b6680 0x7f67000b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:57:08.031 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:57:08.028+0000 7f67157fa640 1 -- 172.21.15.39:0/2194765651 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8914+0+0 (secure 0 0 0) 0x7f67080e7980 con 0x7f6718102a80 2024-09-21T03:57:08.031 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:57:08.029+0000 7f67177fe640 1 --2- 172.21.15.39:0/2194765651 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f67000b6680 0x7f67000b8b90 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f6718105ef0 tx=0x7f670c00d620 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:57:08.034 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:57:08.031+0000 7f67157fa640 1 -- 172.21.15.39:0/2194765651 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f6708014030 con 0x7f6718102a80 2024-09-21T03:57:08.223 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:57:08.220+0000 7f671e78e640 1 -- 172.21.15.39:0/2194765651 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f6718102e90 con 0x7f6718102a80 2024-09-21T03:57:08.434 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:57:08.431+0000 7f67157fa640 1 -- 172.21.15.39:0/2194765651 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v89) ==== 181+0+0 (secure 0 0 0) 0x7f67080b2120 con 0x7f6718102a80 2024-09-21T03:57:08.434 INFO:teuthology.orchestra.run.smithi039.stderr:pool 'cephfs_metadata' removed 2024-09-21T03:57:08.438 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:57:08.434+0000 7f671e78e640 1 -- 172.21.15.39:0/2194765651 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f67000b6680 msgr2=0x7f67000b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:57:08.438 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:57:08.434+0000 7f671e78e640 1 --2- 172.21.15.39:0/2194765651 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f67000b6680 0x7f67000b8b90 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f6718105ef0 tx=0x7f670c00d620 comp rx=0 tx=0).stop 2024-09-21T03:57:08.438 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:57:08.434+0000 7f671e78e640 1 -- 172.21.15.39:0/2194765651 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6718102a80 msgr2=0x7f6718199370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:57:08.438 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:57:08.434+0000 7f671e78e640 1 --2- 172.21.15.39:0/2194765651 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6718102a80 0x7f6718199370 secure :-1 s=READY pgs=1387 cs=0 l=1 rev1=1 crypto rx=0x7f670800c890 tx=0x7f670800cd50 comp rx=0 tx=0).stop 2024-09-21T03:57:08.438 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:57:08.435+0000 7f671e78e640 1 -- 172.21.15.39:0/2194765651 shutdown_connections 2024-09-21T03:57:08.438 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:57:08.435+0000 7f671e78e640 1 --2- 172.21.15.39:0/2194765651 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7f67000b6680 0x7f67000b8b90 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:57:08.438 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:57:08.435+0000 7f671e78e640 1 --2- 172.21.15.39:0/2194765651 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6718108d10 0x7f671819ebb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:57:08.438 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:57:08.435+0000 7f671e78e640 1 --2- 172.21.15.39:0/2194765651 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f67181083c0 0x7f6718199910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:57:08.438 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:57:08.435+0000 7f671e78e640 1 --2- 172.21.15.39:0/2194765651 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6718102a80 0x7f6718199370 unknown :-1 s=CLOSED pgs=1387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:57:08.438 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:57:08.435+0000 7f671e78e640 1 -- 172.21.15.39:0/2194765651 >> 172.21.15.39:0/2194765651 conn(0x7f6718079740 msgr2=0x7f67181043d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:57:08.439 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:57:08.435+0000 7f671e78e640 1 -- 172.21.15.39:0/2194765651 shutdown_connections 2024-09-21T03:57:08.439 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:57:08.435+0000 7f671e78e640 1 -- 172.21.15.39:0/2194765651 wait complete. 2024-09-21T03:57:08.718 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:57:08 smithi161 ceph-mon[34533]: from='client.? 172.21.15.39:0/2194765651' entity='client.admin' cmd={"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} : dispatch 2024-09-21T03:57:08.725 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:57:08 smithi150 ceph-mon[34289]: from='client.? 172.21.15.39:0/2194765651' entity='client.admin' cmd={"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} : dispatch 2024-09-21T03:57:08.797 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:57:08 smithi039 ceph-mon[31211]: from='client.? 172.21.15.39:0/2194765651' entity='client.admin' cmd={"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} : dispatch 2024-09-21T03:57:09.264 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 51116d5c-77ca-11ef-baf5-efdc52797490 -- ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-09-21T03:57:09.524 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/51116d5c-77ca-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:57:09.584 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:57:09 smithi039 ceph-mon[31211]: pgmap v667: 97 pgs: 97 active+clean; 10 MiB data, 530 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 54 KiB/s wr, 31 op/s 2024-09-21T03:57:09.584 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:57:09 smithi039 ceph-mon[31211]: from='client.? 172.21.15.39:0/2194765651' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-09-21T03:57:09.584 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:57:09 smithi039 ceph-mon[31211]: osdmap e89: 12 total, 12 up, 12 in 2024-09-21T03:57:09.718 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:57:09 smithi161 ceph-mon[34533]: pgmap v667: 97 pgs: 97 active+clean; 10 MiB data, 530 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 54 KiB/s wr, 31 op/s 2024-09-21T03:57:09.718 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:57:09 smithi161 ceph-mon[34533]: from='client.? 172.21.15.39:0/2194765651' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-09-21T03:57:09.718 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:57:09 smithi161 ceph-mon[34533]: osdmap e89: 12 total, 12 up, 12 in 2024-09-21T03:57:09.724 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:57:09 smithi150 ceph-mon[34289]: pgmap v667: 97 pgs: 97 active+clean; 10 MiB data, 530 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 54 KiB/s wr, 31 op/s 2024-09-21T03:57:09.725 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:57:09 smithi150 ceph-mon[34289]: from='client.? 172.21.15.39:0/2194765651' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-09-21T03:57:09.725 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:57:09 smithi150 ceph-mon[34289]: osdmap e89: 12 total, 12 up, 12 in 2024-09-21T03:57:10.125 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:57:10.122+0000 7fef1d8e2640 1 -- 172.21.15.39:0/250540242 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fef18102a70 msgr2=0x7fef18102e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:57:10.126 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:57:10.122+0000 7fef1d8e2640 1 --2- 172.21.15.39:0/250540242 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fef18102a70 0x7fef18102e80 secure :-1 s=READY pgs=1388 cs=0 l=1 rev1=1 crypto rx=0x7fef08009900 tx=0x7fef0802a410 comp rx=0 tx=0).stop 2024-09-21T03:57:10.126 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:57:10.123+0000 7fef1d8e2640 1 -- 172.21.15.39:0/250540242 shutdown_connections 2024-09-21T03:57:10.126 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:57:10.123+0000 7fef1d8e2640 1 --2- 172.21.15.39:0/250540242 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fef18108d00 0x7fef1810f530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:57:10.126 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:57:10.123+0000 7fef1d8e2640 1 --2- 172.21.15.39:0/250540242 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fef181083b0 0x7fef181087c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:57:10.126 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:57:10.123+0000 7fef1d8e2640 1 --2- 172.21.15.39:0/250540242 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fef18102a70 0x7fef18102e80 unknown :-1 s=CLOSED pgs=1388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:57:10.126 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:57:10.123+0000 7fef1d8e2640 1 -- 172.21.15.39:0/250540242 >> 172.21.15.39:0/250540242 conn(0x7fef18079730 msgr2=0x7fef18079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:57:10.126 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:57:10.123+0000 7fef1d8e2640 1 -- 172.21.15.39:0/250540242 shutdown_connections 2024-09-21T03:57:10.126 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:57:10.123+0000 7fef1d8e2640 1 -- 172.21.15.39:0/250540242 wait complete. 2024-09-21T03:57:10.128 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:57:10.123+0000 7fef1d8e2640 1 Processor -- start 2024-09-21T03:57:10.128 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:57:10.124+0000 7fef1d8e2640 1 -- start start 2024-09-21T03:57:10.128 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:57:10.124+0000 7fef1d8e2640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fef18102a70 0x7fef1819a450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:57:10.128 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:57:10.124+0000 7fef1d8e2640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fef181083b0 0x7fef1819a9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:57:10.129 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:57:10.124+0000 7fef1d8e2640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fef18108d00 0x7fef1819fc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:57:10.129 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:57:10.124+0000 7fef1d8e2640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fef18112970 con 0x7fef18102a70 2024-09-21T03:57:10.129 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:57:10.124+0000 7fef1d8e2640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fef181127f0 con 0x7fef181083b0 2024-09-21T03:57:10.129 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:57:10.124+0000 7fef1d8e2640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fef18112af0 con 0x7fef18108d00 2024-09-21T03:57:10.129 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:57:10.124+0000 7fef16ffd640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fef18102a70 0x7fef1819a450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:57:10.129 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:57:10.124+0000 7fef16ffd640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fef18102a70 0x7fef1819a450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:53234/0 (socket says 172.21.15.39:53234) 2024-09-21T03:57:10.129 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:57:10.124+0000 7fef16ffd640 1 -- 172.21.15.39:0/1649312448 learned_addr learned my addr 172.21.15.39:0/1649312448 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T03:57:10.129 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:57:10.125+0000 7fef0e7fc640 1 --2- 172.21.15.39:0/1649312448 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fef181083b0 0x7fef1819a9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:57:10.129 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:57:10.125+0000 7fef177fe640 1 --2- 172.21.15.39:0/1649312448 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fef18108d00 0x7fef1819fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:57:10.129 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:57:10.125+0000 7fef16ffd640 1 -- 172.21.15.39:0/1649312448 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fef18108d00 msgr2=0x7fef1819fc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:57:10.129 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:57:10.125+0000 7fef16ffd640 1 --2- 172.21.15.39:0/1649312448 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fef18108d00 0x7fef1819fc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:57:10.129 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:57:10.125+0000 7fef16ffd640 1 -- 172.21.15.39:0/1649312448 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fef181083b0 msgr2=0x7fef1819a9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:57:10.129 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:57:10.125+0000 7fef16ffd640 1 --2- 172.21.15.39:0/1649312448 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fef181083b0 0x7fef1819a9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:57:10.130 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:57:10.125+0000 7fef16ffd640 1 -- 172.21.15.39:0/1649312448 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fef181a03c0 con 0x7fef18102a70 2024-09-21T03:57:10.130 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:57:10.125+0000 7fef177fe640 1 --2- 172.21.15.39:0/1649312448 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fef18108d00 0x7fef1819fc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:57:10.130 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:57:10.125+0000 7fef16ffd640 1 --2- 172.21.15.39:0/1649312448 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fef18102a70 0x7fef1819a450 secure :-1 s=READY pgs=1389 cs=0 l=1 rev1=1 crypto rx=0x7fef08005940 tx=0x7fef080042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:57:10.130 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:57:10.125+0000 7fef1c8e0640 1 -- 172.21.15.39:0/1649312448 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef0803f030 con 0x7fef18102a70 2024-09-21T03:57:10.130 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:57:10.125+0000 7fef1c8e0640 1 -- 172.21.15.39:0/1649312448 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fef08003cb0 con 0x7fef18102a70 2024-09-21T03:57:10.130 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:57:10.125+0000 7fef1d8e2640 1 -- 172.21.15.39:0/1649312448 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fef181a0650 con 0x7fef18102a70 2024-09-21T03:57:10.130 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:57:10.125+0000 7fef1c8e0640 1 -- 172.21.15.39:0/1649312448 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef08035070 con 0x7fef18102a70 2024-09-21T03:57:10.130 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:57:10.125+0000 7fef1d8e2640 1 -- 172.21.15.39:0/1649312448 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fef181a6e10 con 0x7fef18102a70 2024-09-21T03:57:10.131 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:57:10.126+0000 7fef1d8e2640 1 -- 172.21.15.39:0/1649312448 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fef18103cf0 con 0x7fef18102a70 2024-09-21T03:57:10.131 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:57:10.128+0000 7fef1c8e0640 1 -- 172.21.15.39:0/1649312448 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 27) ==== 154643+0+0 (secure 0 0 0) 0x7fef08042030 con 0x7fef18102a70 2024-09-21T03:57:10.132 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:57:10.129+0000 7fef1c8e0640 1 --2- 172.21.15.39:0/1649312448 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7fef040b6700 0x7fef040b8c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:57:10.132 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:57:10.129+0000 7fef0e7fc640 1 --2- 172.21.15.39:0/1649312448 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7fef040b6700 0x7fef040b8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:57:10.132 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:57:10.129+0000 7fef1c8e0640 1 -- 172.21.15.39:0/1649312448 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8360+0+0 (secure 0 0 0) 0x7fef08108070 con 0x7fef18102a70 2024-09-21T03:57:10.132 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:57:10.130+0000 7fef0e7fc640 1 --2- 172.21.15.39:0/1649312448 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7fef040b6700 0x7fef040b8c10 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7fef18100ca0 tx=0x7fef0000b390 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:57:10.135 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:57:10.132+0000 7fef1c8e0640 1 -- 172.21.15.39:0/1649312448 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fef080ce700 con 0x7fef18102a70 2024-09-21T03:57:10.324 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:57:10.321+0000 7fef1d8e2640 1 -- 172.21.15.39:0/1649312448 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7fef18102e80 con 0x7fef18102a70 2024-09-21T03:57:10.441 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:57:10.438+0000 7fef1c8e0640 1 -- 172.21.15.39:0/1649312448 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v90) ==== 169+0+0 (secure 0 0 0) 0x7fef080d2ae0 con 0x7fef18102a70 2024-09-21T03:57:10.441 INFO:teuthology.orchestra.run.smithi039.stderr:pool 'cephfs_data' removed 2024-09-21T03:57:10.445 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:57:10.442+0000 7fef1d8e2640 1 -- 172.21.15.39:0/1649312448 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7fef040b6700 msgr2=0x7fef040b8c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:57:10.445 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:57:10.442+0000 7fef1d8e2640 1 --2- 172.21.15.39:0/1649312448 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7fef040b6700 0x7fef040b8c10 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7fef18100ca0 tx=0x7fef0000b390 comp rx=0 tx=0).stop 2024-09-21T03:57:10.445 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:57:10.442+0000 7fef1d8e2640 1 -- 172.21.15.39:0/1649312448 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fef18102a70 msgr2=0x7fef1819a450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:57:10.445 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:57:10.442+0000 7fef1d8e2640 1 --2- 172.21.15.39:0/1649312448 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fef18102a70 0x7fef1819a450 secure :-1 s=READY pgs=1389 cs=0 l=1 rev1=1 crypto rx=0x7fef08005940 tx=0x7fef080042f0 comp rx=0 tx=0).stop 2024-09-21T03:57:10.445 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:57:10.442+0000 7fef1d8e2640 1 -- 172.21.15.39:0/1649312448 shutdown_connections 2024-09-21T03:57:10.445 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:57:10.442+0000 7fef1d8e2640 1 --2- 172.21.15.39:0/1649312448 >> [v2:172.21.15.39:6800/1696381982,v1:172.21.15.39:6801/1696381982] conn(0x7fef040b6700 0x7fef040b8c10 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:57:10.446 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:57:10.442+0000 7fef1d8e2640 1 --2- 172.21.15.39:0/1649312448 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fef18108d00 0x7fef1819fc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:57:10.446 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:57:10.442+0000 7fef1d8e2640 1 --2- 172.21.15.39:0/1649312448 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fef181083b0 0x7fef1819a9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:57:10.446 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:57:10.442+0000 7fef1d8e2640 1 --2- 172.21.15.39:0/1649312448 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fef18102a70 0x7fef1819a450 secure :-1 s=CLOSED pgs=1389 cs=0 l=1 rev1=1 crypto rx=0x7fef08005940 tx=0x7fef080042f0 comp rx=0 tx=0).stop 2024-09-21T03:57:10.446 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:57:10.442+0000 7fef1d8e2640 1 -- 172.21.15.39:0/1649312448 >> 172.21.15.39:0/1649312448 conn(0x7fef18079730 msgr2=0x7fef181067c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:57:10.446 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:57:10.442+0000 7fef1d8e2640 1 -- 172.21.15.39:0/1649312448 shutdown_connections 2024-09-21T03:57:10.446 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T03:57:10.442+0000 7fef1d8e2640 1 -- 172.21.15.39:0/1649312448 wait complete. 2024-09-21T03:57:10.584 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:57:10 smithi039 ceph-mon[31211]: from='client.? 172.21.15.39:0/1649312448' entity='client.admin' cmd={"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} : dispatch 2024-09-21T03:57:10.718 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:57:10 smithi161 ceph-mon[34533]: from='client.? 172.21.15.39:0/1649312448' entity='client.admin' cmd={"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} : dispatch 2024-09-21T03:57:10.726 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:57:10 smithi150 ceph-mon[34289]: from='client.? 172.21.15.39:0/1649312448' entity='client.admin' cmd={"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} : dispatch 2024-09-21T03:57:11.335 DEBUG:teuthology.orchestra.run.smithi039:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-21T03:57:11.366 DEBUG:teuthology.orchestra.run.smithi150:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-21T03:57:11.398 DEBUG:teuthology.orchestra.run.smithi161:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-21T03:57:11.432 INFO:tasks.cephadm:Cleaning up testdir ceph.* files... 2024-09-21T03:57:11.433 DEBUG:teuthology.orchestra.run.smithi039:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-09-21T03:57:11.451 DEBUG:teuthology.orchestra.run.smithi150:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-09-21T03:57:11.469 DEBUG:teuthology.orchestra.run.smithi161:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-09-21T03:57:11.490 INFO:tasks.cephadm:Stopping all daemons... 2024-09-21T03:57:11.491 INFO:tasks.cephadm.mon.a:Stopping mon.a... 2024-09-21T03:57:11.491 DEBUG:teuthology.orchestra.run.smithi039:> sudo systemctl stop ceph-51116d5c-77ca-11ef-baf5-efdc52797490@mon.a 2024-09-21T03:57:11.584 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:57:11 smithi039 ceph-mon[31211]: pgmap v669: 65 pgs: 65 active+clean; 577 KiB data, 525 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T03:57:11.584 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:57:11 smithi039 ceph-mon[31211]: from='client.? 172.21.15.39:0/1649312448' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-09-21T03:57:11.584 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:57:11 smithi039 ceph-mon[31211]: osdmap e90: 12 total, 12 up, 12 in 2024-09-21T03:57:11.584 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:57:11 smithi039 systemd[1]: Stopping Ceph mon.a for 51116d5c-77ca-11ef-baf5-efdc52797490... 2024-09-21T03:57:11.718 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:57:11 smithi161 ceph-mon[34533]: pgmap v669: 65 pgs: 65 active+clean; 577 KiB data, 525 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T03:57:11.718 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:57:11 smithi161 ceph-mon[34533]: from='client.? 172.21.15.39:0/1649312448' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-09-21T03:57:11.718 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:57:11 smithi161 ceph-mon[34533]: osdmap e90: 12 total, 12 up, 12 in 2024-09-21T03:57:11.724 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:57:11 smithi150 ceph-mon[34289]: pgmap v669: 65 pgs: 65 active+clean; 577 KiB data, 525 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T03:57:11.725 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:57:11 smithi150 ceph-mon[34289]: from='client.? 172.21.15.39:0/1649312448' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-09-21T03:57:11.725 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:57:11 smithi150 ceph-mon[34289]: osdmap e90: 12 total, 12 up, 12 in 2024-09-21T03:57:12.084 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:57:11 smithi039 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-mon-a[31207]: 2024-09-21T03:57:11.808+0000 7f59696a4640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-09-21T03:57:12.084 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:57:11 smithi039 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-mon-a[31207]: 2024-09-21T03:57:11.808+0000 7f59696a4640 -1 mon.a@0(leader) e3 *** Got Signal Terminated *** 2024-09-21T03:57:12.084 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:57:12 smithi039 podman[112012]: 2024-09-21 03:57:12.024977778 +0000 UTC m=+0.323724690 container died 861e490fc665c0095d7e9950a2d6ccd4b7d704a1d4b24d58cda56cf79cda489f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410, name=ceph-51116d5c-77ca-11ef-baf5-efdc52797490-mon-a, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_CLEAN=True, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-21T03:57:12.780 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:57:12 smithi039 podman[112012]: 2024-09-21 03:57:12.568716096 +0000 UTC m=+0.867462998 container cleanup 861e490fc665c0095d7e9950a2d6ccd4b7d704a1d4b24d58cda56cf79cda489f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410, name=ceph-51116d5c-77ca-11ef-baf5-efdc52797490-mon-a, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS) 2024-09-21T03:57:12.780 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:57:12 smithi039 bash[112012]: ceph-51116d5c-77ca-11ef-baf5-efdc52797490-mon-a 2024-09-21T03:57:13.006 DEBUG:teuthology.orchestra.run.smithi039:> sudo pkill -f 'journalctl -f -n 0 -u ceph-51116d5c-77ca-11ef-baf5-efdc52797490@mon.a.service' 2024-09-21T03:57:13.031 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:57:12 smithi039 podman[112026]: 2024-09-21 03:57:12.778601898 +0000 UTC m=+0.747849676 container remove 861e490fc665c0095d7e9950a2d6ccd4b7d704a1d4b24d58cda56cf79cda489f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410, name=ceph-51116d5c-77ca-11ef-baf5-efdc52797490-mon-a, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, ceph=True, CEPH_POINT_RELEASE=, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.build-date=20240731, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, io.buildah.version=1.37.2, org.label-schema.license=GPLv2) 2024-09-21T03:57:13.031 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:57:13 smithi039 systemd[1]: ceph-51116d5c-77ca-11ef-baf5-efdc52797490@mon.a.service: Deactivated successfully. 2024-09-21T03:57:13.031 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:57:13 smithi039 systemd[1]: Stopped Ceph mon.a for 51116d5c-77ca-11ef-baf5-efdc52797490. 2024-09-21T03:57:13.031 INFO:journalctl@ceph.mon.a.smithi039.stdout:Sep 21 03:57:13 smithi039 systemd[1]: ceph-51116d5c-77ca-11ef-baf5-efdc52797490@mon.a.service: Consumed 22.251s CPU time. 2024-09-21T03:57:13.056 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-21T03:57:13.056 INFO:tasks.cephadm.mon.a:Stopped mon.a 2024-09-21T03:57:13.057 INFO:tasks.cephadm.mon.c:Stopping mon.b... 2024-09-21T03:57:13.057 DEBUG:teuthology.orchestra.run.smithi150:> sudo systemctl stop ceph-51116d5c-77ca-11ef-baf5-efdc52797490@mon.b 2024-09-21T03:57:13.373 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:57:13 smithi150 systemd[1]: Stopping Ceph mon.b for 51116d5c-77ca-11ef-baf5-efdc52797490... 2024-09-21T03:57:13.725 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:57:13 smithi150 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-mon-b[34285]: 2024-09-21T03:57:13.371+0000 7f50e1971640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-09-21T03:57:13.725 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:57:13 smithi150 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-mon-b[34285]: 2024-09-21T03:57:13.371+0000 7f50e1971640 -1 mon.b@2(peon) e3 *** Got Signal Terminated *** 2024-09-21T03:57:13.725 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:57:13 smithi150 podman[66801]: 2024-09-21 03:57:13.545359034 +0000 UTC m=+0.289281005 container died 6cc60c3531e0d4f3e3fb1862c1d4ebe3d84e01caaf4b7ef4c0a953a703ab7ffb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-51116d5c-77ca-11ef-baf5-efdc52797490-mon-b, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, io.buildah.version=1.37.2, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_CLEAN=True) 2024-09-21T03:57:14.141 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:57:13 smithi150 podman[66801]: 2024-09-21 03:57:13.946771612 +0000 UTC m=+0.690693585 container cleanup 6cc60c3531e0d4f3e3fb1862c1d4ebe3d84e01caaf4b7ef4c0a953a703ab7ffb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-51116d5c-77ca-11ef-baf5-efdc52797490-mon-b, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, io.buildah.version=1.37.2, GIT_CLEAN=True) 2024-09-21T03:57:14.141 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:57:13 smithi150 bash[66801]: ceph-51116d5c-77ca-11ef-baf5-efdc52797490-mon-b 2024-09-21T03:57:14.376 DEBUG:teuthology.orchestra.run.smithi150:> sudo pkill -f 'journalctl -f -n 0 -u ceph-51116d5c-77ca-11ef-baf5-efdc52797490@mon.b.service' 2024-09-21T03:57:14.398 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:57:14 smithi150 podman[66813]: 2024-09-21 03:57:14.13968655 +0000 UTC m=+0.593668402 container remove 6cc60c3531e0d4f3e3fb1862c1d4ebe3d84e01caaf4b7ef4c0a953a703ab7ffb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-51116d5c-77ca-11ef-baf5-efdc52797490-mon-b, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221) 2024-09-21T03:57:14.398 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:57:14 smithi150 systemd[1]: ceph-51116d5c-77ca-11ef-baf5-efdc52797490@mon.b.service: Deactivated successfully. 2024-09-21T03:57:14.398 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:57:14 smithi150 systemd[1]: Stopped Ceph mon.b for 51116d5c-77ca-11ef-baf5-efdc52797490. 2024-09-21T03:57:14.398 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 21 03:57:14 smithi150 systemd[1]: ceph-51116d5c-77ca-11ef-baf5-efdc52797490@mon.b.service: Consumed 13.900s CPU time. 2024-09-21T03:57:14.430 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-21T03:57:14.430 INFO:tasks.cephadm.mon.c:Stopped mon.b 2024-09-21T03:57:14.430 INFO:tasks.cephadm.mon.c:Stopping mon.c... 2024-09-21T03:57:14.430 DEBUG:teuthology.orchestra.run.smithi161:> sudo systemctl stop ceph-51116d5c-77ca-11ef-baf5-efdc52797490@mon.c 2024-09-21T03:57:14.718 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:57:14 smithi161 systemd[1]: Stopping Ceph mon.c for 51116d5c-77ca-11ef-baf5-efdc52797490... 2024-09-21T03:57:15.155 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:57:14 smithi161 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-mon-c[34529]: 2024-09-21T03:57:14.752+0000 7f6d7ee2d640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-09-21T03:57:15.155 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:57:14 smithi161 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-mon-c[34529]: 2024-09-21T03:57:14.752+0000 7f6d7ee2d640 -1 mon.c@1(peon) e3 *** Got Signal Terminated *** 2024-09-21T03:57:15.155 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:57:14 smithi161 podman[67263]: 2024-09-21 03:57:14.855008024 +0000 UTC m=+0.225651555 container died 4ea176de02d905eaf2709f873bc5d1e87784de9b7e4ab5dd70d60a9e4bc19b76 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-51116d5c-77ca-11ef-baf5-efdc52797490-mon-c, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_BRANCH=HEAD) 2024-09-21T03:57:15.468 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:57:15 smithi161 podman[67263]: 2024-09-21 03:57:15.252648603 +0000 UTC m=+0.623292133 container cleanup 4ea176de02d905eaf2709f873bc5d1e87784de9b7e4ab5dd70d60a9e4bc19b76 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-51116d5c-77ca-11ef-baf5-efdc52797490-mon-c, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.schema-version=1.0, ceph=True, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=) 2024-09-21T03:57:15.468 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 21 03:57:15 smithi161 bash[67263]: ceph-51116d5c-77ca-11ef-baf5-efdc52797490-mon-c 2024-09-21T03:57:15.640 DEBUG:teuthology.orchestra.run.smithi161:> sudo pkill -f 'journalctl -f -n 0 -u ceph-51116d5c-77ca-11ef-baf5-efdc52797490@mon.c.service' 2024-09-21T03:57:15.682 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-21T03:57:15.682 INFO:tasks.cephadm.mon.c:Stopped mon.c 2024-09-21T03:57:15.683 INFO:tasks.cephadm.mgr.x:Stopping mgr.x... 2024-09-21T03:57:15.683 DEBUG:teuthology.orchestra.run.smithi039:> sudo systemctl stop ceph-51116d5c-77ca-11ef-baf5-efdc52797490@mgr.x 2024-09-21T03:57:16.008 INFO:journalctl@ceph.mgr.x.smithi039.stdout:Sep 21 03:57:15 smithi039 systemd[1]: Stopping Ceph mgr.x for 51116d5c-77ca-11ef-baf5-efdc52797490... 2024-09-21T03:57:16.334 INFO:journalctl@ceph.mgr.x.smithi039.stdout:Sep 21 03:57:16 smithi039 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-mgr-x[31438]: 2024-09-21T03:57:16.006+0000 7fe4c26bd640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mgr -n mgr.x -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-21T03:57:16.334 INFO:journalctl@ceph.mgr.x.smithi039.stdout:Sep 21 03:57:16 smithi039 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-mgr-x[31438]: 2024-09-21T03:57:16.006+0000 7fe4c26bd640 -1 mgr handle_mgr_signal *** Got signal Terminated *** 2024-09-21T03:57:16.334 INFO:journalctl@ceph.mgr.x.smithi039.stdout:Sep 21 03:57:16 smithi039 podman[112134]: 2024-09-21 03:57:16.047710592 +0000 UTC m=+0.162556026 container died 622e116e4f4c4ff4ba97137a440df23968df53c17062d2444f01c9d50b66a68a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410, name=ceph-51116d5c-77ca-11ef-baf5-efdc52797490-mgr-x, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_CLEAN=True, io.buildah.version=1.37.2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, ceph=True, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-21T03:57:16.834 INFO:journalctl@ceph.mgr.x.smithi039.stdout:Sep 21 03:57:16 smithi039 podman[112134]: 2024-09-21 03:57:16.666133869 +0000 UTC m=+0.780979291 container cleanup 622e116e4f4c4ff4ba97137a440df23968df53c17062d2444f01c9d50b66a68a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410, name=ceph-51116d5c-77ca-11ef-baf5-efdc52797490-mgr-x, io.buildah.version=1.37.2, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23) 2024-09-21T03:57:16.834 INFO:journalctl@ceph.mgr.x.smithi039.stdout:Sep 21 03:57:16 smithi039 bash[112134]: ceph-51116d5c-77ca-11ef-baf5-efdc52797490-mgr-x 2024-09-21T03:57:17.152 DEBUG:teuthology.orchestra.run.smithi039:> sudo pkill -f 'journalctl -f -n 0 -u ceph-51116d5c-77ca-11ef-baf5-efdc52797490@mgr.x.service' 2024-09-21T03:57:17.174 INFO:journalctl@ceph.mgr.x.smithi039.stdout:Sep 21 03:57:16 smithi039 podman[112146]: 2024-09-21 03:57:16.918137363 +0000 UTC m=+0.872794350 container remove 622e116e4f4c4ff4ba97137a440df23968df53c17062d2444f01c9d50b66a68a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410, name=ceph-51116d5c-77ca-11ef-baf5-efdc52797490-mgr-x, ceph=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_CLEAN=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux ) 2024-09-21T03:57:17.174 INFO:journalctl@ceph.mgr.x.smithi039.stdout:Sep 21 03:57:17 smithi039 systemd[1]: ceph-51116d5c-77ca-11ef-baf5-efdc52797490@mgr.x.service: Deactivated successfully. 2024-09-21T03:57:17.174 INFO:journalctl@ceph.mgr.x.smithi039.stdout:Sep 21 03:57:17 smithi039 systemd[1]: Stopped Ceph mgr.x for 51116d5c-77ca-11ef-baf5-efdc52797490. 2024-09-21T03:57:17.174 INFO:journalctl@ceph.mgr.x.smithi039.stdout:Sep 21 03:57:17 smithi039 systemd[1]: ceph-51116d5c-77ca-11ef-baf5-efdc52797490@mgr.x.service: Consumed 54.200s CPU time. 2024-09-21T03:57:17.187 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-21T03:57:17.187 INFO:tasks.cephadm.mgr.x:Stopped mgr.x 2024-09-21T03:57:17.187 INFO:tasks.cephadm.mgr.y:Stopping mgr.y... 2024-09-21T03:57:17.188 DEBUG:teuthology.orchestra.run.smithi150:> sudo systemctl stop ceph-51116d5c-77ca-11ef-baf5-efdc52797490@mgr.y 2024-09-21T03:57:17.475 INFO:journalctl@ceph.mgr.y.smithi150.stdout:Sep 21 03:57:17 smithi150 systemd[1]: Stopping Ceph mgr.y for 51116d5c-77ca-11ef-baf5-efdc52797490... 2024-09-21T03:57:17.812 INFO:journalctl@ceph.mgr.y.smithi150.stdout:Sep 21 03:57:17 smithi150 podman[66934]: 2024-09-21 03:57:17.54973977 +0000 UTC m=+0.160338930 container died 5ebc4c7778bc6649a379adde9140ae07418d4fa84e63658822d1ed1bba8a4b9b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-51116d5c-77ca-11ef-baf5-efdc52797490-mgr-y, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, GIT_CLEAN=True, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, ceph=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0) 2024-09-21T03:57:18.204 INFO:journalctl@ceph.mgr.y.smithi150.stdout:Sep 21 03:57:17 smithi150 podman[66934]: 2024-09-21 03:57:17.926120731 +0000 UTC m=+0.536719881 container cleanup 5ebc4c7778bc6649a379adde9140ae07418d4fa84e63658822d1ed1bba8a4b9b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-51116d5c-77ca-11ef-baf5-efdc52797490-mgr-y, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, ceph=True, org.label-schema.vendor=CentOS, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, maintainer=Guillaume Abrioux ) 2024-09-21T03:57:18.204 INFO:journalctl@ceph.mgr.y.smithi150.stdout:Sep 21 03:57:17 smithi150 bash[66934]: ceph-51116d5c-77ca-11ef-baf5-efdc52797490-mgr-y 2024-09-21T03:57:18.350 DEBUG:teuthology.orchestra.run.smithi150:> sudo pkill -f 'journalctl -f -n 0 -u ceph-51116d5c-77ca-11ef-baf5-efdc52797490@mgr.y.service' 2024-09-21T03:57:18.386 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-21T03:57:18.386 INFO:tasks.cephadm.mgr.y:Stopped mgr.y 2024-09-21T03:57:18.386 INFO:tasks.cephadm.mgr.z:Stopping mgr.z... 2024-09-21T03:57:18.386 DEBUG:teuthology.orchestra.run.smithi161:> sudo systemctl stop ceph-51116d5c-77ca-11ef-baf5-efdc52797490@mgr.z 2024-09-21T03:57:18.713 INFO:journalctl@ceph.mgr.z.smithi161.stdout:Sep 21 03:57:18 smithi161 systemd[1]: Stopping Ceph mgr.z for 51116d5c-77ca-11ef-baf5-efdc52797490... 2024-09-21T03:57:18.968 INFO:journalctl@ceph.mgr.z.smithi161.stdout:Sep 21 03:57:18 smithi161 podman[67385]: 2024-09-21 03:57:18.712161814 +0000 UTC m=+0.129527584 container died 4f7e5e425925f05bf536ab9b7ebe9d13ab8466a58f8dc31b91e3966e981d9f3b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-51116d5c-77ca-11ef-baf5-efdc52797490-mgr-z, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_CLEAN=True, CEPH_POINT_RELEASE=, ceph=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, io.buildah.version=1.37.2, org.label-schema.license=GPLv2) 2024-09-21T03:57:19.351 INFO:journalctl@ceph.mgr.z.smithi161.stdout:Sep 21 03:57:19 smithi161 podman[67385]: 2024-09-21 03:57:19.082376816 +0000 UTC m=+0.499742586 container cleanup 4f7e5e425925f05bf536ab9b7ebe9d13ab8466a58f8dc31b91e3966e981d9f3b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-51116d5c-77ca-11ef-baf5-efdc52797490-mgr-z, org.label-schema.license=GPLv2, GIT_CLEAN=True, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0) 2024-09-21T03:57:19.352 INFO:journalctl@ceph.mgr.z.smithi161.stdout:Sep 21 03:57:19 smithi161 bash[67385]: ceph-51116d5c-77ca-11ef-baf5-efdc52797490-mgr-z 2024-09-21T03:57:19.482 DEBUG:teuthology.orchestra.run.smithi161:> sudo pkill -f 'journalctl -f -n 0 -u ceph-51116d5c-77ca-11ef-baf5-efdc52797490@mgr.z.service' 2024-09-21T03:57:19.517 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-21T03:57:19.517 INFO:tasks.cephadm.mgr.z:Stopped mgr.z 2024-09-21T03:57:19.517 INFO:tasks.cephadm.osd.0:Stopping osd.0... 2024-09-21T03:57:19.518 DEBUG:teuthology.orchestra.run.smithi039:> sudo systemctl stop ceph-51116d5c-77ca-11ef-baf5-efdc52797490@osd.0 2024-09-21T03:57:19.815 INFO:journalctl@ceph.osd.0.smithi039.stdout:Sep 21 03:57:19 smithi039 systemd[1]: Stopping Ceph osd.0 for 51116d5c-77ca-11ef-baf5-efdc52797490... 2024-09-21T03:57:20.084 INFO:journalctl@ceph.osd.0.smithi039.stdout:Sep 21 03:57:19 smithi039 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-0[47798]: 2024-09-21T03:57:19.812+0000 7f6c9dc2a640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.0 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-21T03:57:20.084 INFO:journalctl@ceph.osd.0.smithi039.stdout:Sep 21 03:57:19 smithi039 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-0[47798]: 2024-09-21T03:57:19.812+0000 7f6c9dc2a640 -1 osd.0 90 *** Got signal Terminated *** 2024-09-21T03:57:20.084 INFO:journalctl@ceph.osd.0.smithi039.stdout:Sep 21 03:57:19 smithi039 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-0[47798]: 2024-09-21T03:57:19.812+0000 7f6c9dc2a640 -1 osd.0 90 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-21T03:57:25.650 INFO:journalctl@ceph.osd.0.smithi039.stdout:Sep 21 03:57:25 smithi039 podman[112256]: 2024-09-21 03:57:25.358876119 +0000 UTC m=+5.644830137 container died fa4243328756310692de2de5ab557f3dc7a5eb1d7a49ac37ad2125bf283d75d6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-0, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0) 2024-09-21T03:57:26.074 INFO:journalctl@ceph.osd.0.smithi039.stdout:Sep 21 03:57:25 smithi039 podman[112256]: 2024-09-21 03:57:25.821410808 +0000 UTC m=+6.107364819 container cleanup fa4243328756310692de2de5ab557f3dc7a5eb1d7a49ac37ad2125bf283d75d6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-0, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, io.buildah.version=1.37.2, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, ceph=True) 2024-09-21T03:57:26.074 INFO:journalctl@ceph.osd.0.smithi039.stdout:Sep 21 03:57:25 smithi039 bash[112256]: ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-0 2024-09-21T03:57:26.334 INFO:journalctl@ceph.osd.0.smithi039.stdout:Sep 21 03:57:26 smithi039 podman[112272]: 2024-09-21 03:57:26.073158442 +0000 UTC m=+0.715741029 container remove fa4243328756310692de2de5ab557f3dc7a5eb1d7a49ac37ad2125bf283d75d6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-0, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_CLEAN=True, ceph=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-21T03:57:26.680 INFO:journalctl@ceph.osd.0.smithi039.stdout:Sep 21 03:57:26 smithi039 podman[112363]: 2024-09-21 03:57:26.299618875 +0000 UTC m=+0.015852311 image pull ddce307e1f3263fb077b4f3902e15660050631dad4d2823a20ee3f72bf18d977 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262 2024-09-21T03:57:26.688 INFO:journalctl@ceph.osd.0.smithi039.stdout:Sep 21 03:57:26 smithi039 podman[112363]: 2024-09-21 03:57:26.432560798 +0000 UTC m=+0.148794226 container create 46deeb3573e63f7cf000527c921a1b950226ea9e8d63e3b5e3e94ef24bdd76e1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-0-deactivate, org.label-schema.build-date=20240731, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, io.buildah.version=1.37.2, ceph=True, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0) 2024-09-21T03:57:26.696 INFO:journalctl@ceph.osd.0.smithi039.stdout:Sep 21 03:57:26 smithi039 podman[112363]: 2024-09-21 03:57:26.676767967 +0000 UTC m=+0.393001404 container init 46deeb3573e63f7cf000527c921a1b950226ea9e8d63e3b5e3e94ef24bdd76e1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-0-deactivate, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, GIT_CLEAN=True, ceph=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23) 2024-09-21T03:57:27.084 INFO:journalctl@ceph.osd.0.smithi039.stdout:Sep 21 03:57:26 smithi039 podman[112363]: 2024-09-21 03:57:26.680288648 +0000 UTC m=+0.396522077 container start 46deeb3573e63f7cf000527c921a1b950226ea9e8d63e3b5e3e94ef24bdd76e1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-0-deactivate, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, ceph=True) 2024-09-21T03:57:27.088 INFO:journalctl@ceph.osd.0.smithi039.stdout:Sep 21 03:57:26 smithi039 podman[112363]: 2024-09-21 03:57:26.751765008 +0000 UTC m=+0.467998441 container attach 46deeb3573e63f7cf000527c921a1b950226ea9e8d63e3b5e3e94ef24bdd76e1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-0-deactivate, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_BRANCH=HEAD) 2024-09-21T03:57:27.089 INFO:journalctl@ceph.osd.0.smithi039.stdout:Sep 21 03:57:26 smithi039 podman[112363]: 2024-09-21 03:57:26.877675318 +0000 UTC m=+0.593908745 container died 46deeb3573e63f7cf000527c921a1b950226ea9e8d63e3b5e3e94ef24bdd76e1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-0-deactivate, GIT_CLEAN=True, ceph=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-21T03:57:27.619 DEBUG:teuthology.orchestra.run.smithi039:> sudo pkill -f 'journalctl -f -n 0 -u ceph-51116d5c-77ca-11ef-baf5-efdc52797490@osd.0.service' 2024-09-21T03:57:27.670 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-21T03:57:27.670 INFO:tasks.cephadm.osd.0:Stopped osd.0 2024-09-21T03:57:27.670 INFO:tasks.cephadm.osd.1:Stopping osd.1... 2024-09-21T03:57:27.670 DEBUG:teuthology.orchestra.run.smithi150:> sudo systemctl stop ceph-51116d5c-77ca-11ef-baf5-efdc52797490@osd.1 2024-09-21T03:57:27.975 INFO:journalctl@ceph.osd.1.smithi150.stdout:Sep 21 03:57:27 smithi150 systemd[1]: Stopping Ceph osd.1 for 51116d5c-77ca-11ef-baf5-efdc52797490... 2024-09-21T03:57:28.475 INFO:journalctl@ceph.osd.1.smithi150.stdout:Sep 21 03:57:27 smithi150 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-1[38911]: 2024-09-21T03:57:27.988+0000 7fea193ea640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.1 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-21T03:57:28.475 INFO:journalctl@ceph.osd.1.smithi150.stdout:Sep 21 03:57:27 smithi150 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-1[38911]: 2024-09-21T03:57:27.988+0000 7fea193ea640 -1 osd.1 90 *** Got signal Terminated *** 2024-09-21T03:57:28.475 INFO:journalctl@ceph.osd.1.smithi150.stdout:Sep 21 03:57:27 smithi150 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-1[38911]: 2024-09-21T03:57:27.988+0000 7fea193ea640 -1 osd.1 90 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-21T03:57:33.699 INFO:journalctl@ceph.osd.1.smithi150.stdout:Sep 21 03:57:33 smithi150 podman[67061]: 2024-09-21 03:57:33.383029639 +0000 UTC m=+5.512996201 container died c075cba916a016b8cbda7345e2e78375177fefaf12dc7425437e6ed5cde7bb44 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-1, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, maintainer=Guillaume Abrioux , GIT_CLEAN=True, ceph=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-21T03:57:33.950 INFO:journalctl@ceph.osd.1.smithi150.stdout:Sep 21 03:57:33 smithi150 podman[67061]: 2024-09-21 03:57:33.696846886 +0000 UTC m=+5.826813458 container cleanup c075cba916a016b8cbda7345e2e78375177fefaf12dc7425437e6ed5cde7bb44 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-1, org.label-schema.schema-version=1.0, ceph=True, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS) 2024-09-21T03:57:33.950 INFO:journalctl@ceph.osd.1.smithi150.stdout:Sep 21 03:57:33 smithi150 bash[67061]: ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-1 2024-09-21T03:57:34.225 INFO:journalctl@ceph.osd.1.smithi150.stdout:Sep 21 03:57:33 smithi150 podman[67076]: 2024-09-21 03:57:33.948088883 +0000 UTC m=+0.560085378 container remove c075cba916a016b8cbda7345e2e78375177fefaf12dc7425437e6ed5cde7bb44 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-1, GIT_BRANCH=HEAD, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-21T03:57:34.720 INFO:journalctl@ceph.osd.1.smithi150.stdout:Sep 21 03:57:34 smithi150 podman[67163]: 2024-09-21 03:57:34.268698694 +0000 UTC m=+0.016158585 image pull ddce307e1f3263fb077b4f3902e15660050631dad4d2823a20ee3f72bf18d977 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262 2024-09-21T03:57:34.720 INFO:journalctl@ceph.osd.1.smithi150.stdout:Sep 21 03:57:34 smithi150 podman[67163]: 2024-09-21 03:57:34.382145259 +0000 UTC m=+0.129605136 container create e65479b0b47417086c37d27b43f94809556dd5d827a3d95086e1674c7a0e6488 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-1-deactivate, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, maintainer=Guillaume Abrioux , GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, ceph=True, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2) 2024-09-21T03:57:34.720 INFO:journalctl@ceph.osd.1.smithi150.stdout:Sep 21 03:57:34 smithi150 podman[67163]: 2024-09-21 03:57:34.609391772 +0000 UTC m=+0.356851650 container init e65479b0b47417086c37d27b43f94809556dd5d827a3d95086e1674c7a0e6488 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-1-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, GIT_BRANCH=HEAD, ceph=True, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, maintainer=Guillaume Abrioux ) 2024-09-21T03:57:34.720 INFO:journalctl@ceph.osd.1.smithi150.stdout:Sep 21 03:57:34 smithi150 podman[67163]: 2024-09-21 03:57:34.612698531 +0000 UTC m=+0.360158426 container start e65479b0b47417086c37d27b43f94809556dd5d827a3d95086e1674c7a0e6488 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-1-deactivate, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_CLEAN=True) 2024-09-21T03:57:34.975 INFO:journalctl@ceph.osd.1.smithi150.stdout:Sep 21 03:57:34 smithi150 podman[67163]: 2024-09-21 03:57:34.717794151 +0000 UTC m=+0.465254031 container attach e65479b0b47417086c37d27b43f94809556dd5d827a3d95086e1674c7a0e6488 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-1-deactivate, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, ceph=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux ) 2024-09-21T03:57:34.975 INFO:journalctl@ceph.osd.1.smithi150.stdout:Sep 21 03:57:34 smithi150 podman[67163]: 2024-09-21 03:57:34.80210476 +0000 UTC m=+0.549564652 container died e65479b0b47417086c37d27b43f94809556dd5d827a3d95086e1674c7a0e6488 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-1-deactivate, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23) 2024-09-21T03:57:35.419 DEBUG:teuthology.orchestra.run.smithi150:> sudo pkill -f 'journalctl -f -n 0 -u ceph-51116d5c-77ca-11ef-baf5-efdc52797490@osd.1.service' 2024-09-21T03:57:35.458 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-21T03:57:35.458 INFO:tasks.cephadm.osd.1:Stopped osd.1 2024-09-21T03:57:35.458 INFO:tasks.cephadm.osd.2:Stopping osd.2... 2024-09-21T03:57:35.458 DEBUG:teuthology.orchestra.run.smithi161:> sudo systemctl stop ceph-51116d5c-77ca-11ef-baf5-efdc52797490@osd.2 2024-09-21T03:57:35.912 INFO:journalctl@ceph.osd.2.smithi161.stdout:Sep 21 03:57:35 smithi161 systemd[1]: Stopping Ceph osd.2 for 51116d5c-77ca-11ef-baf5-efdc52797490... 2024-09-21T03:57:36.218 INFO:journalctl@ceph.osd.2.smithi161.stdout:Sep 21 03:57:35 smithi161 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-2[39689]: 2024-09-21T03:57:35.910+0000 7fe816c90640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.2 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-21T03:57:36.218 INFO:journalctl@ceph.osd.2.smithi161.stdout:Sep 21 03:57:35 smithi161 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-2[39689]: 2024-09-21T03:57:35.910+0000 7fe816c90640 -1 osd.2 90 *** Got signal Terminated *** 2024-09-21T03:57:36.218 INFO:journalctl@ceph.osd.2.smithi161.stdout:Sep 21 03:57:35 smithi161 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-2[39689]: 2024-09-21T03:57:35.910+0000 7fe816c90640 -1 osd.2 90 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-21T03:57:41.718 INFO:journalctl@ceph.osd.2.smithi161.stdout:Sep 21 03:57:41 smithi161 podman[67510]: 2024-09-21 03:57:41.454957078 +0000 UTC m=+5.802814535 container died f6a27d40f337e5f59b36c299169ea809f2e0ac42736d2c9c5bfba9cf5daff81a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-2, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, io.buildah.version=1.37.2, GIT_CLEAN=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221) 2024-09-21T03:57:42.105 INFO:journalctl@ceph.osd.2.smithi161.stdout:Sep 21 03:57:41 smithi161 podman[67510]: 2024-09-21 03:57:41.860582722 +0000 UTC m=+6.208440179 container cleanup f6a27d40f337e5f59b36c299169ea809f2e0ac42736d2c9c5bfba9cf5daff81a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, ceph=True, GIT_CLEAN=True, CEPH_POINT_RELEASE=) 2024-09-21T03:57:42.105 INFO:journalctl@ceph.osd.2.smithi161.stdout:Sep 21 03:57:41 smithi161 bash[67510]: ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-2 2024-09-21T03:57:42.468 INFO:journalctl@ceph.osd.2.smithi161.stdout:Sep 21 03:57:42 smithi161 podman[67525]: 2024-09-21 03:57:42.103357626 +0000 UTC m=+0.646186684 container remove f6a27d40f337e5f59b36c299169ea809f2e0ac42736d2c9c5bfba9cf5daff81a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-2, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, ceph=True) 2024-09-21T03:57:42.817 INFO:journalctl@ceph.osd.2.smithi161.stdout:Sep 21 03:57:42 smithi161 podman[67619]: 2024-09-21 03:57:42.379752001 +0000 UTC m=+0.016350808 image pull ddce307e1f3263fb077b4f3902e15660050631dad4d2823a20ee3f72bf18d977 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262 2024-09-21T03:57:42.817 INFO:journalctl@ceph.osd.2.smithi161.stdout:Sep 21 03:57:42 smithi161 podman[67619]: 2024-09-21 03:57:42.487548492 +0000 UTC m=+0.124147290 container create 693206da14a6c7778d5e1b90aa539119a240f92972262cfe81365e3e2444b1f2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-2-deactivate, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.build-date=20240731) 2024-09-21T03:57:42.817 INFO:journalctl@ceph.osd.2.smithi161.stdout:Sep 21 03:57:42 smithi161 podman[67619]: 2024-09-21 03:57:42.714714722 +0000 UTC m=+0.351313519 container init 693206da14a6c7778d5e1b90aa539119a240f92972262cfe81365e3e2444b1f2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-2-deactivate, ceph=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221) 2024-09-21T03:57:42.817 INFO:journalctl@ceph.osd.2.smithi161.stdout:Sep 21 03:57:42 smithi161 podman[67619]: 2024-09-21 03:57:42.718045637 +0000 UTC m=+0.354644443 container start 693206da14a6c7778d5e1b90aa539119a240f92972262cfe81365e3e2444b1f2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-2-deactivate, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, maintainer=Guillaume Abrioux , GIT_CLEAN=True) 2024-09-21T03:57:42.817 INFO:journalctl@ceph.osd.2.smithi161.stdout:Sep 21 03:57:42 smithi161 podman[67619]: 2024-09-21 03:57:42.814808242 +0000 UTC m=+0.451407049 container attach 693206da14a6c7778d5e1b90aa539119a240f92972262cfe81365e3e2444b1f2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-2-deactivate, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, ceph=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True) 2024-09-21T03:57:43.212 INFO:journalctl@ceph.osd.2.smithi161.stdout:Sep 21 03:57:42 smithi161 podman[67619]: 2024-09-21 03:57:42.917464414 +0000 UTC m=+0.554063220 container died 693206da14a6c7778d5e1b90aa539119a240f92972262cfe81365e3e2444b1f2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-2-deactivate, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_CLEAN=True) 2024-09-21T03:57:43.533 DEBUG:teuthology.orchestra.run.smithi161:> sudo pkill -f 'journalctl -f -n 0 -u ceph-51116d5c-77ca-11ef-baf5-efdc52797490@osd.2.service' 2024-09-21T03:57:43.574 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-21T03:57:43.574 INFO:tasks.cephadm.osd.2:Stopped osd.2 2024-09-21T03:57:43.575 INFO:tasks.cephadm.osd.3:Stopping osd.3... 2024-09-21T03:57:43.575 DEBUG:teuthology.orchestra.run.smithi039:> sudo systemctl stop ceph-51116d5c-77ca-11ef-baf5-efdc52797490@osd.3 2024-09-21T03:57:43.885 INFO:journalctl@ceph.osd.3.smithi039.stdout:Sep 21 03:57:43 smithi039 systemd[1]: Stopping Ceph osd.3 for 51116d5c-77ca-11ef-baf5-efdc52797490... 2024-09-21T03:57:44.334 INFO:journalctl@ceph.osd.3.smithi039.stdout:Sep 21 03:57:43 smithi039 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-3[53927]: 2024-09-21T03:57:43.882+0000 7f7961c17640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.3 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-21T03:57:44.334 INFO:journalctl@ceph.osd.3.smithi039.stdout:Sep 21 03:57:43 smithi039 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-3[53927]: 2024-09-21T03:57:43.882+0000 7f7961c17640 -1 osd.3 90 *** Got signal Terminated *** 2024-09-21T03:57:44.334 INFO:journalctl@ceph.osd.3.smithi039.stdout:Sep 21 03:57:43 smithi039 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-3[53927]: 2024-09-21T03:57:43.882+0000 7f7961c17640 -1 osd.3 90 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-21T03:57:46.334 INFO:journalctl@ceph.osd.9.smithi039.stdout:Sep 21 03:57:45 smithi039 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-9[67006]: 2024-09-21T03:57:45.966+0000 7f4a138c4640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-09-21T03:57:22.331574+0000 front 2024-09-21T03:57:22.332146+0000 (oldest deadline 2024-09-21T03:57:45.831720+0000) 2024-09-21T03:57:47.119 INFO:journalctl@ceph.osd.5.smithi161.stdout:Sep 21 03:57:46 smithi161 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-5[45969]: 2024-09-21T03:57:46.843+0000 7fab25e6c640 -1 osd.5 90 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-09-21T03:57:23.429632+0000 front 2024-09-21T03:57:23.429554+0000 (oldest deadline 2024-09-21T03:57:46.329612+0000) 2024-09-21T03:57:47.225 INFO:journalctl@ceph.osd.10.smithi150.stdout:Sep 21 03:57:46 smithi150 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-10[58206]: 2024-09-21T03:57:46.826+0000 7f34020e1640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-09-21T03:57:23.120033+0000 front 2024-09-21T03:57:23.120109+0000 (oldest deadline 2024-09-21T03:57:45.920434+0000) 2024-09-21T03:57:47.334 INFO:journalctl@ceph.osd.9.smithi039.stdout:Sep 21 03:57:47 smithi039 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-9[67006]: 2024-09-21T03:57:47.008+0000 7f4a138c4640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-09-21T03:57:22.331574+0000 front 2024-09-21T03:57:22.332146+0000 (oldest deadline 2024-09-21T03:57:45.831720+0000) 2024-09-21T03:57:47.334 INFO:journalctl@ceph.osd.6.smithi039.stdout:Sep 21 03:57:46 smithi039 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-6[60520]: 2024-09-21T03:57:46.899+0000 7f98a8e96640 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-09-21T03:57:22.466026+0000 front 2024-09-21T03:57:22.465966+0000 (oldest deadline 2024-09-21T03:57:46.566034+0000) 2024-09-21T03:57:47.468 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 21 03:57:47 smithi161 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-11[58829]: 2024-09-21T03:57:47.116+0000 7f538ef4a640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-09-21T03:57:21.581303+0000 front 2024-09-21T03:57:21.581333+0000 (oldest deadline 2024-09-21T03:57:46.881367+0000) 2024-09-21T03:57:48.218 INFO:journalctl@ceph.osd.5.smithi161.stdout:Sep 21 03:57:47 smithi161 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-5[45969]: 2024-09-21T03:57:47.892+0000 7fab25e6c640 -1 osd.5 90 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-09-21T03:57:23.429632+0000 front 2024-09-21T03:57:23.429554+0000 (oldest deadline 2024-09-21T03:57:46.329612+0000) 2024-09-21T03:57:48.218 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 21 03:57:48 smithi161 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-11[58829]: 2024-09-21T03:57:48.093+0000 7f538ef4a640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-09-21T03:57:21.581303+0000 front 2024-09-21T03:57:21.581333+0000 (oldest deadline 2024-09-21T03:57:46.881367+0000) 2024-09-21T03:57:48.224 INFO:journalctl@ceph.osd.10.smithi150.stdout:Sep 21 03:57:47 smithi150 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-10[58206]: 2024-09-21T03:57:47.797+0000 7f34020e1640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-09-21T03:57:23.120033+0000 front 2024-09-21T03:57:23.120109+0000 (oldest deadline 2024-09-21T03:57:45.920434+0000) 2024-09-21T03:57:48.334 INFO:journalctl@ceph.osd.6.smithi039.stdout:Sep 21 03:57:47 smithi039 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-6[60520]: 2024-09-21T03:57:47.935+0000 7f98a8e96640 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-09-21T03:57:22.466026+0000 front 2024-09-21T03:57:22.465966+0000 (oldest deadline 2024-09-21T03:57:46.566034+0000) 2024-09-21T03:57:48.335 INFO:journalctl@ceph.osd.9.smithi039.stdout:Sep 21 03:57:47 smithi039 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-9[67006]: 2024-09-21T03:57:47.991+0000 7f4a138c4640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-09-21T03:57:22.331574+0000 front 2024-09-21T03:57:22.332146+0000 (oldest deadline 2024-09-21T03:57:45.831720+0000) 2024-09-21T03:57:49.218 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 21 03:57:49 smithi161 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-11[58829]: 2024-09-21T03:57:49.078+0000 7f538ef4a640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-09-21T03:57:21.581303+0000 front 2024-09-21T03:57:21.581333+0000 (oldest deadline 2024-09-21T03:57:46.881367+0000) 2024-09-21T03:57:49.218 INFO:journalctl@ceph.osd.5.smithi161.stdout:Sep 21 03:57:48 smithi161 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-5[45969]: 2024-09-21T03:57:48.863+0000 7fab25e6c640 -1 osd.5 90 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-09-21T03:57:23.429632+0000 front 2024-09-21T03:57:23.429554+0000 (oldest deadline 2024-09-21T03:57:46.329612+0000) 2024-09-21T03:57:49.224 INFO:journalctl@ceph.osd.10.smithi150.stdout:Sep 21 03:57:48 smithi150 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-10[58206]: 2024-09-21T03:57:48.796+0000 7f34020e1640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-09-21T03:57:23.120033+0000 front 2024-09-21T03:57:23.120109+0000 (oldest deadline 2024-09-21T03:57:45.920434+0000) 2024-09-21T03:57:49.316 INFO:journalctl@ceph.osd.6.smithi039.stdout:Sep 21 03:57:48 smithi039 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-6[60520]: 2024-09-21T03:57:48.932+0000 7f98a8e96640 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-09-21T03:57:22.466026+0000 front 2024-09-21T03:57:22.465966+0000 (oldest deadline 2024-09-21T03:57:46.566034+0000) 2024-09-21T03:57:49.316 INFO:journalctl@ceph.osd.9.smithi039.stdout:Sep 21 03:57:48 smithi039 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-9[67006]: 2024-09-21T03:57:48.950+0000 7f4a138c4640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-09-21T03:57:22.331574+0000 front 2024-09-21T03:57:22.332146+0000 (oldest deadline 2024-09-21T03:57:45.831720+0000) 2024-09-21T03:57:49.584 INFO:journalctl@ceph.osd.3.smithi039.stdout:Sep 21 03:57:49 smithi039 podman[112465]: 2024-09-21 03:57:49.314730532 +0000 UTC m=+5.538419632 container died f836a786f909909957d43d135e3e7186ed95ce00bf4de830bb945428c1a2d192 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-3, maintainer=Guillaume Abrioux , GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2) 2024-09-21T03:57:49.993 INFO:journalctl@ceph.osd.6.smithi039.stdout:Sep 21 03:57:49 smithi039 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-6[60520]: 2024-09-21T03:57:49.926+0000 7f98a8e96640 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-09-21T03:57:22.466026+0000 front 2024-09-21T03:57:22.465966+0000 (oldest deadline 2024-09-21T03:57:46.566034+0000) 2024-09-21T03:57:49.993 INFO:journalctl@ceph.osd.3.smithi039.stdout:Sep 21 03:57:49 smithi039 podman[112465]: 2024-09-21 03:57:49.900382332 +0000 UTC m=+6.124071431 container cleanup f836a786f909909957d43d135e3e7186ed95ce00bf4de830bb945428c1a2d192 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-3, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, ceph=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-21T03:57:49.994 INFO:journalctl@ceph.osd.3.smithi039.stdout:Sep 21 03:57:49 smithi039 bash[112465]: ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-3 2024-09-21T03:57:50.218 INFO:journalctl@ceph.osd.5.smithi161.stdout:Sep 21 03:57:49 smithi161 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-5[45969]: 2024-09-21T03:57:49.869+0000 7fab25e6c640 -1 osd.5 90 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-09-21T03:57:23.429632+0000 front 2024-09-21T03:57:23.429554+0000 (oldest deadline 2024-09-21T03:57:46.329612+0000) 2024-09-21T03:57:50.218 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 21 03:57:50 smithi161 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-11[58829]: 2024-09-21T03:57:50.096+0000 7f538ef4a640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-09-21T03:57:21.581303+0000 front 2024-09-21T03:57:21.581333+0000 (oldest deadline 2024-09-21T03:57:46.881367+0000) 2024-09-21T03:57:50.225 INFO:journalctl@ceph.osd.10.smithi150.stdout:Sep 21 03:57:49 smithi150 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-10[58206]: 2024-09-21T03:57:49.809+0000 7f34020e1640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-09-21T03:57:23.120033+0000 front 2024-09-21T03:57:23.120109+0000 (oldest deadline 2024-09-21T03:57:45.920434+0000) 2024-09-21T03:57:50.328 INFO:journalctl@ceph.osd.9.smithi039.stdout:Sep 21 03:57:49 smithi039 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-9[67006]: 2024-09-21T03:57:49.990+0000 7f4a138c4640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-09-21T03:57:22.331574+0000 front 2024-09-21T03:57:22.332146+0000 (oldest deadline 2024-09-21T03:57:45.831720+0000) 2024-09-21T03:57:50.584 INFO:journalctl@ceph.osd.3.smithi039.stdout:Sep 21 03:57:50 smithi039 podman[112497]: 2024-09-21 03:57:50.327193368 +0000 UTC m=+1.001784813 container remove f836a786f909909957d43d135e3e7186ed95ce00bf4de830bb945428c1a2d192 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-3, org.label-schema.schema-version=1.0, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, io.buildah.version=1.37.2, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221) 2024-09-21T03:57:50.955 INFO:journalctl@ceph.osd.6.smithi039.stdout:Sep 21 03:57:50 smithi039 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-6[60520]: 2024-09-21T03:57:50.909+0000 7f98a8e96640 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-09-21T03:57:22.466026+0000 front 2024-09-21T03:57:22.465966+0000 (oldest deadline 2024-09-21T03:57:46.566034+0000) 2024-09-21T03:57:50.955 INFO:journalctl@ceph.osd.3.smithi039.stdout:Sep 21 03:57:50 smithi039 podman[112585]: 2024-09-21 03:57:50.575478466 +0000 UTC m=+0.016379653 image pull ddce307e1f3263fb077b4f3902e15660050631dad4d2823a20ee3f72bf18d977 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262 2024-09-21T03:57:50.955 INFO:journalctl@ceph.osd.3.smithi039.stdout:Sep 21 03:57:50 smithi039 podman[112585]: 2024-09-21 03:57:50.677801408 +0000 UTC m=+0.118702585 container create 00013bfcdc26bedac92c821a6f75e9032a480b2101ca992cc9d85eeff5c2385d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-3-deactivate, GIT_CLEAN=True, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, ceph=True, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.license=GPLv2, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_BRANCH=HEAD) 2024-09-21T03:57:50.955 INFO:journalctl@ceph.osd.3.smithi039.stdout:Sep 21 03:57:50 smithi039 podman[112585]: 2024-09-21 03:57:50.897227068 +0000 UTC m=+0.338128253 container init 00013bfcdc26bedac92c821a6f75e9032a480b2101ca992cc9d85eeff5c2385d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-3-deactivate, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, ceph=True, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0) 2024-09-21T03:57:50.955 INFO:journalctl@ceph.osd.3.smithi039.stdout:Sep 21 03:57:50 smithi039 podman[112585]: 2024-09-21 03:57:50.900840447 +0000 UTC m=+0.341741637 container start 00013bfcdc26bedac92c821a6f75e9032a480b2101ca992cc9d85eeff5c2385d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-3-deactivate, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_BRANCH=HEAD, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True) 2024-09-21T03:57:51.218 INFO:journalctl@ceph.osd.5.smithi161.stdout:Sep 21 03:57:50 smithi161 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-5[45969]: 2024-09-21T03:57:50.874+0000 7fab25e6c640 -1 osd.5 90 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-09-21T03:57:23.429632+0000 front 2024-09-21T03:57:23.429554+0000 (oldest deadline 2024-09-21T03:57:46.329612+0000) 2024-09-21T03:57:51.218 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 21 03:57:51 smithi161 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-11[58829]: 2024-09-21T03:57:51.101+0000 7f538ef4a640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-09-21T03:57:21.581303+0000 front 2024-09-21T03:57:21.581333+0000 (oldest deadline 2024-09-21T03:57:46.881367+0000) 2024-09-21T03:57:51.225 INFO:journalctl@ceph.osd.10.smithi150.stdout:Sep 21 03:57:50 smithi150 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-10[58206]: 2024-09-21T03:57:50.770+0000 7f34020e1640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-09-21T03:57:23.120033+0000 front 2024-09-21T03:57:23.120109+0000 (oldest deadline 2024-09-21T03:57:45.920434+0000) 2024-09-21T03:57:51.334 INFO:journalctl@ceph.osd.9.smithi039.stdout:Sep 21 03:57:50 smithi039 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-9[67006]: 2024-09-21T03:57:50.952+0000 7f4a138c4640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-09-21T03:57:22.331574+0000 front 2024-09-21T03:57:22.332146+0000 (oldest deadline 2024-09-21T03:57:45.831720+0000) 2024-09-21T03:57:51.334 INFO:journalctl@ceph.osd.3.smithi039.stdout:Sep 21 03:57:51 smithi039 podman[112585]: 2024-09-21 03:57:51.00561608 +0000 UTC m=+0.446517274 container attach 00013bfcdc26bedac92c821a6f75e9032a480b2101ca992cc9d85eeff5c2385d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-3-deactivate, ceph=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0) 2024-09-21T03:57:51.334 INFO:journalctl@ceph.osd.3.smithi039.stdout:Sep 21 03:57:51 smithi039 podman[112585]: 2024-09-21 03:57:51.09513625 +0000 UTC m=+0.536037437 container died 00013bfcdc26bedac92c821a6f75e9032a480b2101ca992cc9d85eeff5c2385d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-3-deactivate, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_BRANCH=HEAD, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221) 2024-09-21T03:57:51.747 DEBUG:teuthology.orchestra.run.smithi039:> sudo pkill -f 'journalctl -f -n 0 -u ceph-51116d5c-77ca-11ef-baf5-efdc52797490@osd.3.service' 2024-09-21T03:57:51.794 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-21T03:57:51.794 INFO:tasks.cephadm.osd.3:Stopped osd.3 2024-09-21T03:57:51.794 INFO:tasks.cephadm.osd.4:Stopping osd.4... 2024-09-21T03:57:51.794 DEBUG:teuthology.orchestra.run.smithi150:> sudo systemctl stop ceph-51116d5c-77ca-11ef-baf5-efdc52797490@osd.4 2024-09-21T03:57:52.084 INFO:journalctl@ceph.osd.6.smithi039.stdout:Sep 21 03:57:51 smithi039 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-6[60520]: 2024-09-21T03:57:51.929+0000 7f98a8e96640 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-09-21T03:57:22.466026+0000 front 2024-09-21T03:57:22.465966+0000 (oldest deadline 2024-09-21T03:57:46.566034+0000) 2024-09-21T03:57:52.084 INFO:journalctl@ceph.osd.9.smithi039.stdout:Sep 21 03:57:51 smithi039 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-9[67006]: 2024-09-21T03:57:51.955+0000 7f4a138c4640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-09-21T03:57:22.331574+0000 front 2024-09-21T03:57:22.332146+0000 (oldest deadline 2024-09-21T03:57:45.831720+0000) 2024-09-21T03:57:52.127 INFO:journalctl@ceph.osd.4.smithi150.stdout:Sep 21 03:57:51 smithi150 systemd[1]: Stopping Ceph osd.4 for 51116d5c-77ca-11ef-baf5-efdc52797490... 2024-09-21T03:57:52.127 INFO:journalctl@ceph.osd.10.smithi150.stdout:Sep 21 03:57:51 smithi150 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-10[58206]: 2024-09-21T03:57:51.789+0000 7f34020e1640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-09-21T03:57:23.120033+0000 front 2024-09-21T03:57:23.120109+0000 (oldest deadline 2024-09-21T03:57:45.920434+0000) 2024-09-21T03:57:52.218 INFO:journalctl@ceph.osd.5.smithi161.stdout:Sep 21 03:57:51 smithi161 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-5[45969]: 2024-09-21T03:57:51.855+0000 7fab25e6c640 -1 osd.5 90 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-09-21T03:57:23.429632+0000 front 2024-09-21T03:57:23.429554+0000 (oldest deadline 2024-09-21T03:57:46.329612+0000) 2024-09-21T03:57:52.218 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 21 03:57:52 smithi161 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-11[58829]: 2024-09-21T03:57:52.058+0000 7f538ef4a640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-09-21T03:57:21.581303+0000 front 2024-09-21T03:57:21.581333+0000 (oldest deadline 2024-09-21T03:57:46.881367+0000) 2024-09-21T03:57:52.475 INFO:journalctl@ceph.osd.4.smithi150.stdout:Sep 21 03:57:52 smithi150 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-4[45362]: 2024-09-21T03:57:52.124+0000 7f6b1a061640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.4 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-21T03:57:52.475 INFO:journalctl@ceph.osd.4.smithi150.stdout:Sep 21 03:57:52 smithi150 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-4[45362]: 2024-09-21T03:57:52.124+0000 7f6b1a061640 -1 osd.4 90 *** Got signal Terminated *** 2024-09-21T03:57:52.475 INFO:journalctl@ceph.osd.4.smithi150.stdout:Sep 21 03:57:52 smithi150 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-4[45362]: 2024-09-21T03:57:52.124+0000 7f6b1a061640 -1 osd.4 90 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-21T03:57:53.218 INFO:journalctl@ceph.osd.5.smithi161.stdout:Sep 21 03:57:52 smithi161 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-5[45969]: 2024-09-21T03:57:52.868+0000 7fab25e6c640 -1 osd.5 90 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-09-21T03:57:23.429632+0000 front 2024-09-21T03:57:23.429554+0000 (oldest deadline 2024-09-21T03:57:46.329612+0000) 2024-09-21T03:57:53.218 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 21 03:57:53 smithi161 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-11[58829]: 2024-09-21T03:57:53.085+0000 7f538ef4a640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-09-21T03:57:21.581303+0000 front 2024-09-21T03:57:21.581333+0000 (oldest deadline 2024-09-21T03:57:46.881367+0000) 2024-09-21T03:57:53.225 INFO:journalctl@ceph.osd.10.smithi150.stdout:Sep 21 03:57:52 smithi150 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-10[58206]: 2024-09-21T03:57:52.763+0000 7f34020e1640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-09-21T03:57:23.120033+0000 front 2024-09-21T03:57:23.120109+0000 (oldest deadline 2024-09-21T03:57:45.920434+0000) 2024-09-21T03:57:53.334 INFO:journalctl@ceph.osd.6.smithi039.stdout:Sep 21 03:57:52 smithi039 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-6[60520]: 2024-09-21T03:57:52.945+0000 7f98a8e96640 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-09-21T03:57:22.466026+0000 front 2024-09-21T03:57:22.465966+0000 (oldest deadline 2024-09-21T03:57:46.566034+0000) 2024-09-21T03:57:53.334 INFO:journalctl@ceph.osd.9.smithi039.stdout:Sep 21 03:57:52 smithi039 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-9[67006]: 2024-09-21T03:57:52.945+0000 7f4a138c4640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-09-21T03:57:22.331574+0000 front 2024-09-21T03:57:22.332146+0000 (oldest deadline 2024-09-21T03:57:45.831720+0000) 2024-09-21T03:57:54.207 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 21 03:57:54 smithi161 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-11[58829]: 2024-09-21T03:57:54.042+0000 7f538ef4a640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-09-21T03:57:21.581303+0000 front 2024-09-21T03:57:21.581333+0000 (oldest deadline 2024-09-21T03:57:46.881367+0000) 2024-09-21T03:57:54.207 INFO:journalctl@ceph.osd.5.smithi161.stdout:Sep 21 03:57:53 smithi161 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-5[45969]: 2024-09-21T03:57:53.872+0000 7fab25e6c640 -1 osd.5 90 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-09-21T03:57:23.429632+0000 front 2024-09-21T03:57:23.429554+0000 (oldest deadline 2024-09-21T03:57:46.329612+0000) 2024-09-21T03:57:54.207 INFO:journalctl@ceph.osd.5.smithi161.stdout:Sep 21 03:57:53 smithi161 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-5[45969]: 2024-09-21T03:57:53.872+0000 7fab25e6c640 -1 osd.5 90 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-09-21T03:57:32.731637+0000 front 2024-09-21T03:57:32.731745+0000 (oldest deadline 2024-09-21T03:57:53.831549+0000) 2024-09-21T03:57:54.224 INFO:journalctl@ceph.osd.10.smithi150.stdout:Sep 21 03:57:53 smithi150 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-10[58206]: 2024-09-21T03:57:53.722+0000 7f34020e1640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-09-21T03:57:23.120033+0000 front 2024-09-21T03:57:23.120109+0000 (oldest deadline 2024-09-21T03:57:45.920434+0000) 2024-09-21T03:57:54.334 INFO:journalctl@ceph.osd.9.smithi039.stdout:Sep 21 03:57:53 smithi039 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-9[67006]: 2024-09-21T03:57:53.943+0000 7f4a138c4640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-09-21T03:57:22.331574+0000 front 2024-09-21T03:57:22.332146+0000 (oldest deadline 2024-09-21T03:57:45.831720+0000) 2024-09-21T03:57:54.334 INFO:journalctl@ceph.osd.6.smithi039.stdout:Sep 21 03:57:53 smithi039 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-6[60520]: 2024-09-21T03:57:53.968+0000 7f98a8e96640 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-09-21T03:57:22.466026+0000 front 2024-09-21T03:57:22.465966+0000 (oldest deadline 2024-09-21T03:57:46.566034+0000) 2024-09-21T03:57:54.334 INFO:journalctl@ceph.osd.6.smithi039.stdout:Sep 21 03:57:53 smithi039 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-6[60520]: 2024-09-21T03:57:53.968+0000 7f98a8e96640 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-09-21T03:57:30.067078+0000 front 2024-09-21T03:57:30.067189+0000 (oldest deadline 2024-09-21T03:57:53.566934+0000) 2024-09-21T03:57:54.468 INFO:journalctl@ceph.osd.8.smithi161.stdout:Sep 21 03:57:54 smithi161 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-8[52151]: 2024-09-21T03:57:54.204+0000 7f7e9a0f6640 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-09-21T03:57:29.951582+0000 front 2024-09-21T03:57:29.951481+0000 (oldest deadline 2024-09-21T03:57:54.051508+0000) 2024-09-21T03:57:54.975 INFO:journalctl@ceph.osd.10.smithi150.stdout:Sep 21 03:57:54 smithi150 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-10[58206]: 2024-09-21T03:57:54.686+0000 7f34020e1640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-09-21T03:57:23.120033+0000 front 2024-09-21T03:57:23.120109+0000 (oldest deadline 2024-09-21T03:57:45.920434+0000) 2024-09-21T03:57:55.185 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 21 03:57:55 smithi161 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-11[58829]: 2024-09-21T03:57:55.076+0000 7f538ef4a640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-09-21T03:57:21.581303+0000 front 2024-09-21T03:57:21.581333+0000 (oldest deadline 2024-09-21T03:57:46.881367+0000) 2024-09-21T03:57:55.185 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 21 03:57:55 smithi161 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-11[58829]: 2024-09-21T03:57:55.076+0000 7f538ef4a640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-09-21T03:57:32.782524+0000 front 2024-09-21T03:57:32.782318+0000 (oldest deadline 2024-09-21T03:57:54.482443+0000) 2024-09-21T03:57:55.185 INFO:journalctl@ceph.osd.5.smithi161.stdout:Sep 21 03:57:54 smithi161 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-5[45969]: 2024-09-21T03:57:54.900+0000 7fab25e6c640 -1 osd.5 90 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-09-21T03:57:23.429632+0000 front 2024-09-21T03:57:23.429554+0000 (oldest deadline 2024-09-21T03:57:46.329612+0000) 2024-09-21T03:57:55.185 INFO:journalctl@ceph.osd.5.smithi161.stdout:Sep 21 03:57:54 smithi161 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-5[45969]: 2024-09-21T03:57:54.900+0000 7fab25e6c640 -1 osd.5 90 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-09-21T03:57:32.731637+0000 front 2024-09-21T03:57:32.731745+0000 (oldest deadline 2024-09-21T03:57:53.831549+0000) 2024-09-21T03:57:55.334 INFO:journalctl@ceph.osd.9.smithi039.stdout:Sep 21 03:57:54 smithi039 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-9[67006]: 2024-09-21T03:57:54.913+0000 7f4a138c4640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-09-21T03:57:22.331574+0000 front 2024-09-21T03:57:22.332146+0000 (oldest deadline 2024-09-21T03:57:45.831720+0000) 2024-09-21T03:57:55.334 INFO:journalctl@ceph.osd.9.smithi039.stdout:Sep 21 03:57:54 smithi039 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-9[67006]: 2024-09-21T03:57:54.913+0000 7f4a138c4640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-09-21T03:57:29.932951+0000 front 2024-09-21T03:57:29.932930+0000 (oldest deadline 2024-09-21T03:57:54.632848+0000) 2024-09-21T03:57:55.334 INFO:journalctl@ceph.osd.6.smithi039.stdout:Sep 21 03:57:54 smithi039 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-6[60520]: 2024-09-21T03:57:54.931+0000 7f98a8e96640 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-09-21T03:57:22.466026+0000 front 2024-09-21T03:57:22.465966+0000 (oldest deadline 2024-09-21T03:57:46.566034+0000) 2024-09-21T03:57:55.334 INFO:journalctl@ceph.osd.6.smithi039.stdout:Sep 21 03:57:54 smithi039 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-6[60520]: 2024-09-21T03:57:54.931+0000 7f98a8e96640 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-09-21T03:57:30.067078+0000 front 2024-09-21T03:57:30.067189+0000 (oldest deadline 2024-09-21T03:57:53.566934+0000) 2024-09-21T03:57:55.468 INFO:journalctl@ceph.osd.8.smithi161.stdout:Sep 21 03:57:55 smithi161 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-8[52151]: 2024-09-21T03:57:55.183+0000 7f7e9a0f6640 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-09-21T03:57:29.951582+0000 front 2024-09-21T03:57:29.951481+0000 (oldest deadline 2024-09-21T03:57:54.051508+0000) 2024-09-21T03:57:55.714 INFO:journalctl@ceph.osd.4.smithi150.stdout:Sep 21 03:57:55 smithi150 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-4[45362]: 2024-09-21T03:57:55.267+0000 7f6b12644640 -1 osd.4 90 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-09-21T03:57:31.630338+0000 front 2024-09-21T03:57:31.630443+0000 (oldest deadline 2024-09-21T03:57:55.130568+0000) 2024-09-21T03:57:55.948 INFO:journalctl@ceph.osd.6.smithi039.stdout:Sep 21 03:57:55 smithi039 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-6[60520]: 2024-09-21T03:57:55.903+0000 7f98a8e96640 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-09-21T03:57:22.466026+0000 front 2024-09-21T03:57:22.465966+0000 (oldest deadline 2024-09-21T03:57:46.566034+0000) 2024-09-21T03:57:55.948 INFO:journalctl@ceph.osd.6.smithi039.stdout:Sep 21 03:57:55 smithi039 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-6[60520]: 2024-09-21T03:57:55.903+0000 7f98a8e96640 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-09-21T03:57:30.067078+0000 front 2024-09-21T03:57:30.067189+0000 (oldest deadline 2024-09-21T03:57:53.566934+0000) 2024-09-21T03:57:55.974 INFO:journalctl@ceph.osd.10.smithi150.stdout:Sep 21 03:57:55 smithi150 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-10[58206]: 2024-09-21T03:57:55.712+0000 7f34020e1640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-09-21T03:57:23.120033+0000 front 2024-09-21T03:57:23.120109+0000 (oldest deadline 2024-09-21T03:57:45.920434+0000) 2024-09-21T03:57:56.218 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 21 03:57:56 smithi161 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-11[58829]: 2024-09-21T03:57:56.069+0000 7f538ef4a640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-09-21T03:57:21.581303+0000 front 2024-09-21T03:57:21.581333+0000 (oldest deadline 2024-09-21T03:57:46.881367+0000) 2024-09-21T03:57:56.218 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 21 03:57:56 smithi161 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-11[58829]: 2024-09-21T03:57:56.069+0000 7f538ef4a640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-09-21T03:57:32.782524+0000 front 2024-09-21T03:57:32.782318+0000 (oldest deadline 2024-09-21T03:57:54.482443+0000) 2024-09-21T03:57:56.218 INFO:journalctl@ceph.osd.5.smithi161.stdout:Sep 21 03:57:55 smithi161 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-5[45969]: 2024-09-21T03:57:55.877+0000 7fab25e6c640 -1 osd.5 90 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-09-21T03:57:23.429632+0000 front 2024-09-21T03:57:23.429554+0000 (oldest deadline 2024-09-21T03:57:46.329612+0000) 2024-09-21T03:57:56.218 INFO:journalctl@ceph.osd.5.smithi161.stdout:Sep 21 03:57:55 smithi161 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-5[45969]: 2024-09-21T03:57:55.877+0000 7fab25e6c640 -1 osd.5 90 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-09-21T03:57:32.731637+0000 front 2024-09-21T03:57:32.731745+0000 (oldest deadline 2024-09-21T03:57:53.831549+0000) 2024-09-21T03:57:56.263 INFO:journalctl@ceph.osd.9.smithi039.stdout:Sep 21 03:57:55 smithi039 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-9[67006]: 2024-09-21T03:57:55.946+0000 7f4a138c4640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-09-21T03:57:22.331574+0000 front 2024-09-21T03:57:22.332146+0000 (oldest deadline 2024-09-21T03:57:45.831720+0000) 2024-09-21T03:57:56.263 INFO:journalctl@ceph.osd.9.smithi039.stdout:Sep 21 03:57:55 smithi039 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-9[67006]: 2024-09-21T03:57:55.946+0000 7f4a138c4640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-09-21T03:57:29.932951+0000 front 2024-09-21T03:57:29.932930+0000 (oldest deadline 2024-09-21T03:57:54.632848+0000) 2024-09-21T03:57:56.666 INFO:journalctl@ceph.osd.7.smithi150.stdout:Sep 21 03:57:56 smithi150 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-7[51981]: 2024-09-21T03:57:56.342+0000 7f2e0cdd2640 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-09-21T03:57:31.734202+0000 front 2024-09-21T03:57:31.734234+0000 (oldest deadline 2024-09-21T03:57:55.834353+0000) 2024-09-21T03:57:56.666 INFO:journalctl@ceph.osd.4.smithi150.stdout:Sep 21 03:57:56 smithi150 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-4[45362]: 2024-09-21T03:57:56.271+0000 7f6b12644640 -1 osd.4 90 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-09-21T03:57:31.630338+0000 front 2024-09-21T03:57:31.630443+0000 (oldest deadline 2024-09-21T03:57:55.130568+0000) 2024-09-21T03:57:56.718 INFO:journalctl@ceph.osd.8.smithi161.stdout:Sep 21 03:57:56 smithi161 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-8[52151]: 2024-09-21T03:57:56.218+0000 7f7e9a0f6640 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-09-21T03:57:29.951582+0000 front 2024-09-21T03:57:29.951481+0000 (oldest deadline 2024-09-21T03:57:54.051508+0000) 2024-09-21T03:57:56.975 INFO:journalctl@ceph.osd.10.smithi150.stdout:Sep 21 03:57:56 smithi150 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-10[58206]: 2024-09-21T03:57:56.664+0000 7f34020e1640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-09-21T03:57:23.120033+0000 front 2024-09-21T03:57:23.120109+0000 (oldest deadline 2024-09-21T03:57:45.920434+0000) 2024-09-21T03:57:56.975 INFO:journalctl@ceph.osd.10.smithi150.stdout:Sep 21 03:57:56 smithi150 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-10[58206]: 2024-09-21T03:57:56.664+0000 7f34020e1640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-09-21T03:57:31.221107+0000 front 2024-09-21T03:57:31.221162+0000 (oldest deadline 2024-09-21T03:57:56.521240+0000) 2024-09-21T03:57:57.218 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 21 03:57:57 smithi161 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-11[58829]: 2024-09-21T03:57:57.074+0000 7f538ef4a640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-09-21T03:57:21.581303+0000 front 2024-09-21T03:57:21.581333+0000 (oldest deadline 2024-09-21T03:57:46.881367+0000) 2024-09-21T03:57:57.218 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 21 03:57:57 smithi161 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-11[58829]: 2024-09-21T03:57:57.074+0000 7f538ef4a640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-09-21T03:57:32.782524+0000 front 2024-09-21T03:57:32.782318+0000 (oldest deadline 2024-09-21T03:57:54.482443+0000) 2024-09-21T03:57:57.218 INFO:journalctl@ceph.osd.5.smithi161.stdout:Sep 21 03:57:56 smithi161 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-5[45969]: 2024-09-21T03:57:56.849+0000 7fab25e6c640 -1 osd.5 90 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-09-21T03:57:23.429632+0000 front 2024-09-21T03:57:23.429554+0000 (oldest deadline 2024-09-21T03:57:46.329612+0000) 2024-09-21T03:57:57.218 INFO:journalctl@ceph.osd.5.smithi161.stdout:Sep 21 03:57:56 smithi161 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-5[45969]: 2024-09-21T03:57:56.849+0000 7fab25e6c640 -1 osd.5 90 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-09-21T03:57:32.731637+0000 front 2024-09-21T03:57:32.731745+0000 (oldest deadline 2024-09-21T03:57:53.831549+0000) 2024-09-21T03:57:57.334 INFO:journalctl@ceph.osd.6.smithi039.stdout:Sep 21 03:57:56 smithi039 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-6[60520]: 2024-09-21T03:57:56.931+0000 7f98a8e96640 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-09-21T03:57:22.466026+0000 front 2024-09-21T03:57:22.465966+0000 (oldest deadline 2024-09-21T03:57:46.566034+0000) 2024-09-21T03:57:57.334 INFO:journalctl@ceph.osd.6.smithi039.stdout:Sep 21 03:57:56 smithi039 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-6[60520]: 2024-09-21T03:57:56.931+0000 7f98a8e96640 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-09-21T03:57:30.067078+0000 front 2024-09-21T03:57:30.067189+0000 (oldest deadline 2024-09-21T03:57:53.566934+0000) 2024-09-21T03:57:57.334 INFO:journalctl@ceph.osd.9.smithi039.stdout:Sep 21 03:57:56 smithi039 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-9[67006]: 2024-09-21T03:57:56.916+0000 7f4a138c4640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-09-21T03:57:22.331574+0000 front 2024-09-21T03:57:22.332146+0000 (oldest deadline 2024-09-21T03:57:45.831720+0000) 2024-09-21T03:57:57.334 INFO:journalctl@ceph.osd.9.smithi039.stdout:Sep 21 03:57:56 smithi039 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-9[67006]: 2024-09-21T03:57:56.916+0000 7f4a138c4640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-09-21T03:57:29.932951+0000 front 2024-09-21T03:57:29.932930+0000 (oldest deadline 2024-09-21T03:57:54.632848+0000) 2024-09-21T03:57:57.621 INFO:journalctl@ceph.osd.4.smithi150.stdout:Sep 21 03:57:57 smithi150 podman[67276]: 2024-09-21 03:57:57.466546416 +0000 UTC m=+5.478111234 container died 8fcc733b9e669e79b3dfe09e6ce8442988c5be1a9794b8896e68a6f896d82d64 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-4, org.label-schema.build-date=20240731, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, ceph=True, io.buildah.version=1.37.2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD) 2024-09-21T03:57:57.621 INFO:journalctl@ceph.osd.7.smithi150.stdout:Sep 21 03:57:57 smithi150 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-7[51981]: 2024-09-21T03:57:57.352+0000 7f2e0cdd2640 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-09-21T03:57:31.734202+0000 front 2024-09-21T03:57:31.734234+0000 (oldest deadline 2024-09-21T03:57:55.834353+0000) 2024-09-21T03:57:57.718 INFO:journalctl@ceph.osd.8.smithi161.stdout:Sep 21 03:57:57 smithi161 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-8[52151]: 2024-09-21T03:57:57.235+0000 7f7e9a0f6640 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-09-21T03:57:29.951582+0000 front 2024-09-21T03:57:29.951481+0000 (oldest deadline 2024-09-21T03:57:54.051508+0000) 2024-09-21T03:57:57.975 INFO:journalctl@ceph.osd.4.smithi150.stdout:Sep 21 03:57:57 smithi150 podman[67276]: 2024-09-21 03:57:57.799957376 +0000 UTC m=+5.811522184 container cleanup 8fcc733b9e669e79b3dfe09e6ce8442988c5be1a9794b8896e68a6f896d82d64 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-4, ceph=True, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_CLEAN=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, GIT_BRANCH=HEAD) 2024-09-21T03:57:57.975 INFO:journalctl@ceph.osd.4.smithi150.stdout:Sep 21 03:57:57 smithi150 bash[67276]: ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-4 2024-09-21T03:57:57.975 INFO:journalctl@ceph.osd.10.smithi150.stdout:Sep 21 03:57:57 smithi150 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-10[58206]: 2024-09-21T03:57:57.618+0000 7f34020e1640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-09-21T03:57:23.120033+0000 front 2024-09-21T03:57:23.120109+0000 (oldest deadline 2024-09-21T03:57:45.920434+0000) 2024-09-21T03:57:57.975 INFO:journalctl@ceph.osd.10.smithi150.stdout:Sep 21 03:57:57 smithi150 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-10[58206]: 2024-09-21T03:57:57.618+0000 7f34020e1640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-09-21T03:57:31.221107+0000 front 2024-09-21T03:57:31.221162+0000 (oldest deadline 2024-09-21T03:57:56.521240+0000) 2024-09-21T03:57:58.218 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 21 03:57:58 smithi161 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-11[58829]: 2024-09-21T03:57:58.047+0000 7f538ef4a640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-09-21T03:57:21.581303+0000 front 2024-09-21T03:57:21.581333+0000 (oldest deadline 2024-09-21T03:57:46.881367+0000) 2024-09-21T03:57:58.218 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 21 03:57:58 smithi161 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-11[58829]: 2024-09-21T03:57:58.047+0000 7f538ef4a640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-09-21T03:57:32.782524+0000 front 2024-09-21T03:57:32.782318+0000 (oldest deadline 2024-09-21T03:57:54.482443+0000) 2024-09-21T03:57:58.218 INFO:journalctl@ceph.osd.5.smithi161.stdout:Sep 21 03:57:57 smithi161 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-5[45969]: 2024-09-21T03:57:57.853+0000 7fab25e6c640 -1 osd.5 90 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-09-21T03:57:23.429632+0000 front 2024-09-21T03:57:23.429554+0000 (oldest deadline 2024-09-21T03:57:46.329612+0000) 2024-09-21T03:57:58.218 INFO:journalctl@ceph.osd.5.smithi161.stdout:Sep 21 03:57:57 smithi161 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-5[45969]: 2024-09-21T03:57:57.853+0000 7fab25e6c640 -1 osd.5 90 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-09-21T03:57:32.731637+0000 front 2024-09-21T03:57:32.731745+0000 (oldest deadline 2024-09-21T03:57:53.831549+0000) 2024-09-21T03:57:58.317 INFO:journalctl@ceph.osd.4.smithi150.stdout:Sep 21 03:57:57 smithi150 podman[67293]: 2024-09-21 03:57:57.984534679 +0000 UTC m=+0.513265452 container remove 8fcc733b9e669e79b3dfe09e6ce8442988c5be1a9794b8896e68a6f896d82d64 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-4, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.build-date=20240731, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2) 2024-09-21T03:57:58.475 INFO:journalctl@ceph.osd.6.smithi039.stdout:Sep 21 03:57:57 smithi039 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-6[60520]: 2024-09-21T03:57:57.970+0000 7f98a8e96640 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-09-21T03:57:22.466026+0000 front 2024-09-21T03:57:22.465966+0000 (oldest deadline 2024-09-21T03:57:46.566034+0000) 2024-09-21T03:57:58.475 INFO:journalctl@ceph.osd.6.smithi039.stdout:Sep 21 03:57:57 smithi039 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-6[60520]: 2024-09-21T03:57:57.970+0000 7f98a8e96640 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-09-21T03:57:30.067078+0000 front 2024-09-21T03:57:30.067189+0000 (oldest deadline 2024-09-21T03:57:53.566934+0000) 2024-09-21T03:57:58.476 INFO:journalctl@ceph.osd.9.smithi039.stdout:Sep 21 03:57:57 smithi039 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-9[67006]: 2024-09-21T03:57:57.896+0000 7f4a138c4640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-09-21T03:57:22.331574+0000 front 2024-09-21T03:57:22.332146+0000 (oldest deadline 2024-09-21T03:57:45.831720+0000) 2024-09-21T03:57:58.476 INFO:journalctl@ceph.osd.9.smithi039.stdout:Sep 21 03:57:57 smithi039 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-9[67006]: 2024-09-21T03:57:57.896+0000 7f4a138c4640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-09-21T03:57:29.932951+0000 front 2024-09-21T03:57:29.932930+0000 (oldest deadline 2024-09-21T03:57:54.632848+0000) 2024-09-21T03:57:58.630 INFO:journalctl@ceph.osd.7.smithi150.stdout:Sep 21 03:57:58 smithi150 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-7[51981]: 2024-09-21T03:57:58.315+0000 7f2e0cdd2640 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-09-21T03:57:31.734202+0000 front 2024-09-21T03:57:31.734234+0000 (oldest deadline 2024-09-21T03:57:55.834353+0000) 2024-09-21T03:57:58.631 INFO:journalctl@ceph.osd.4.smithi150.stdout:Sep 21 03:57:58 smithi150 podman[67382]: 2024-09-21 03:57:58.315474931 +0000 UTC m=+0.015878292 image pull ddce307e1f3263fb077b4f3902e15660050631dad4d2823a20ee3f72bf18d977 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262 2024-09-21T03:57:58.631 INFO:journalctl@ceph.osd.4.smithi150.stdout:Sep 21 03:57:58 smithi150 podman[67382]: 2024-09-21 03:57:58.418555992 +0000 UTC m=+0.118959345 container create 513807bfdf1f73c20e508cc7eafd107f1b40a72646085e114e42c290dc6225cb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-4-deactivate, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, ceph=True) 2024-09-21T03:57:58.718 INFO:journalctl@ceph.osd.8.smithi161.stdout:Sep 21 03:57:58 smithi161 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-8[52151]: 2024-09-21T03:57:58.262+0000 7f7e9a0f6640 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-09-21T03:57:29.951582+0000 front 2024-09-21T03:57:29.951481+0000 (oldest deadline 2024-09-21T03:57:54.051508+0000) 2024-09-21T03:57:58.975 INFO:journalctl@ceph.osd.10.smithi150.stdout:Sep 21 03:57:58 smithi150 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-10[58206]: 2024-09-21T03:57:58.632+0000 7f34020e1640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-09-21T03:57:23.120033+0000 front 2024-09-21T03:57:23.120109+0000 (oldest deadline 2024-09-21T03:57:45.920434+0000) 2024-09-21T03:57:58.975 INFO:journalctl@ceph.osd.10.smithi150.stdout:Sep 21 03:57:58 smithi150 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-10[58206]: 2024-09-21T03:57:58.632+0000 7f34020e1640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-09-21T03:57:31.221107+0000 front 2024-09-21T03:57:31.221162+0000 (oldest deadline 2024-09-21T03:57:56.521240+0000) 2024-09-21T03:57:58.976 INFO:journalctl@ceph.osd.4.smithi150.stdout:Sep 21 03:57:58 smithi150 podman[67382]: 2024-09-21 03:57:58.629205055 +0000 UTC m=+0.329608408 container init 513807bfdf1f73c20e508cc7eafd107f1b40a72646085e114e42c290dc6225cb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-4-deactivate, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.build-date=20240731, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, ceph=True) 2024-09-21T03:57:58.976 INFO:journalctl@ceph.osd.4.smithi150.stdout:Sep 21 03:57:58 smithi150 podman[67382]: 2024-09-21 03:57:58.632452186 +0000 UTC m=+0.332855542 container start 513807bfdf1f73c20e508cc7eafd107f1b40a72646085e114e42c290dc6225cb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-4-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, ceph=True, GIT_CLEAN=True, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, maintainer=Guillaume Abrioux , GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2) 2024-09-21T03:57:58.976 INFO:journalctl@ceph.osd.4.smithi150.stdout:Sep 21 03:57:58 smithi150 podman[67382]: 2024-09-21 03:57:58.737478114 +0000 UTC m=+0.437881468 container attach 513807bfdf1f73c20e508cc7eafd107f1b40a72646085e114e42c290dc6225cb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-4-deactivate, maintainer=Guillaume Abrioux , ceph=True, GIT_CLEAN=True, CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS) 2024-09-21T03:57:58.976 INFO:journalctl@ceph.osd.4.smithi150.stdout:Sep 21 03:57:58 smithi150 podman[67382]: 2024-09-21 03:57:58.825161627 +0000 UTC m=+0.525564980 container died 513807bfdf1f73c20e508cc7eafd107f1b40a72646085e114e42c290dc6225cb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-4-deactivate, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, ceph=True, GIT_CLEAN=True, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-21T03:57:59.218 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 21 03:57:59 smithi161 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-11[58829]: 2024-09-21T03:57:59.025+0000 7f538ef4a640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-09-21T03:57:21.581303+0000 front 2024-09-21T03:57:21.581333+0000 (oldest deadline 2024-09-21T03:57:46.881367+0000) 2024-09-21T03:57:59.218 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 21 03:57:59 smithi161 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-11[58829]: 2024-09-21T03:57:59.025+0000 7f538ef4a640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-09-21T03:57:32.782524+0000 front 2024-09-21T03:57:32.782318+0000 (oldest deadline 2024-09-21T03:57:54.482443+0000) 2024-09-21T03:57:59.218 INFO:journalctl@ceph.osd.5.smithi161.stdout:Sep 21 03:57:58 smithi161 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-5[45969]: 2024-09-21T03:57:58.902+0000 7fab25e6c640 -1 osd.5 90 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-09-21T03:57:23.429632+0000 front 2024-09-21T03:57:23.429554+0000 (oldest deadline 2024-09-21T03:57:46.329612+0000) 2024-09-21T03:57:59.218 INFO:journalctl@ceph.osd.5.smithi161.stdout:Sep 21 03:57:58 smithi161 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-5[45969]: 2024-09-21T03:57:58.902+0000 7fab25e6c640 -1 osd.5 90 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-09-21T03:57:32.731637+0000 front 2024-09-21T03:57:32.731745+0000 (oldest deadline 2024-09-21T03:57:53.831549+0000) 2024-09-21T03:57:59.334 INFO:journalctl@ceph.osd.6.smithi039.stdout:Sep 21 03:57:58 smithi039 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-6[60520]: 2024-09-21T03:57:58.970+0000 7f98a8e96640 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-09-21T03:57:22.466026+0000 front 2024-09-21T03:57:22.465966+0000 (oldest deadline 2024-09-21T03:57:46.566034+0000) 2024-09-21T03:57:59.334 INFO:journalctl@ceph.osd.6.smithi039.stdout:Sep 21 03:57:58 smithi039 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-6[60520]: 2024-09-21T03:57:58.970+0000 7f98a8e96640 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-09-21T03:57:30.067078+0000 front 2024-09-21T03:57:30.067189+0000 (oldest deadline 2024-09-21T03:57:53.566934+0000) 2024-09-21T03:57:59.334 INFO:journalctl@ceph.osd.9.smithi039.stdout:Sep 21 03:57:58 smithi039 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-9[67006]: 2024-09-21T03:57:58.890+0000 7f4a138c4640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-09-21T03:57:22.331574+0000 front 2024-09-21T03:57:22.332146+0000 (oldest deadline 2024-09-21T03:57:45.831720+0000) 2024-09-21T03:57:59.334 INFO:journalctl@ceph.osd.9.smithi039.stdout:Sep 21 03:57:58 smithi039 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-9[67006]: 2024-09-21T03:57:58.890+0000 7f4a138c4640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-09-21T03:57:29.932951+0000 front 2024-09-21T03:57:29.932930+0000 (oldest deadline 2024-09-21T03:57:54.632848+0000) 2024-09-21T03:57:59.375 INFO:journalctl@ceph.osd.7.smithi150.stdout:Sep 21 03:57:59 smithi150 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-7[51981]: 2024-09-21T03:57:59.330+0000 7f2e0cdd2640 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-09-21T03:57:31.734202+0000 front 2024-09-21T03:57:31.734234+0000 (oldest deadline 2024-09-21T03:57:55.834353+0000) 2024-09-21T03:57:59.398 DEBUG:teuthology.orchestra.run.smithi150:> sudo pkill -f 'journalctl -f -n 0 -u ceph-51116d5c-77ca-11ef-baf5-efdc52797490@osd.4.service' 2024-09-21T03:57:59.439 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-21T03:57:59.439 INFO:tasks.cephadm.osd.4:Stopped osd.4 2024-09-21T03:57:59.440 INFO:tasks.cephadm.osd.5:Stopping osd.5... 2024-09-21T03:57:59.440 DEBUG:teuthology.orchestra.run.smithi161:> sudo systemctl stop ceph-51116d5c-77ca-11ef-baf5-efdc52797490@osd.5 2024-09-21T03:57:59.718 INFO:journalctl@ceph.osd.5.smithi161.stdout:Sep 21 03:57:59 smithi161 systemd[1]: Stopping Ceph osd.5 for 51116d5c-77ca-11ef-baf5-efdc52797490... 2024-09-21T03:57:59.718 INFO:journalctl@ceph.osd.8.smithi161.stdout:Sep 21 03:57:59 smithi161 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-8[52151]: 2024-09-21T03:57:59.253+0000 7f7e9a0f6640 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-09-21T03:57:29.951582+0000 front 2024-09-21T03:57:29.951481+0000 (oldest deadline 2024-09-21T03:57:54.051508+0000) 2024-09-21T03:57:59.974 INFO:journalctl@ceph.osd.10.smithi150.stdout:Sep 21 03:57:59 smithi150 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-10[58206]: 2024-09-21T03:57:59.636+0000 7f34020e1640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-09-21T03:57:23.120033+0000 front 2024-09-21T03:57:23.120109+0000 (oldest deadline 2024-09-21T03:57:45.920434+0000) 2024-09-21T03:57:59.975 INFO:journalctl@ceph.osd.10.smithi150.stdout:Sep 21 03:57:59 smithi150 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-10[58206]: 2024-09-21T03:57:59.636+0000 7f34020e1640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-09-21T03:57:31.221107+0000 front 2024-09-21T03:57:31.221162+0000 (oldest deadline 2024-09-21T03:57:56.521240+0000) 2024-09-21T03:58:00.063 INFO:journalctl@ceph.osd.5.smithi161.stdout:Sep 21 03:57:59 smithi161 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-5[45969]: 2024-09-21T03:57:59.745+0000 7fab2d889640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.5 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-21T03:58:00.063 INFO:journalctl@ceph.osd.5.smithi161.stdout:Sep 21 03:57:59 smithi161 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-5[45969]: 2024-09-21T03:57:59.745+0000 7fab2d889640 -1 osd.5 90 *** Got signal Terminated *** 2024-09-21T03:58:00.063 INFO:journalctl@ceph.osd.5.smithi161.stdout:Sep 21 03:57:59 smithi161 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-5[45969]: 2024-09-21T03:57:59.745+0000 7fab2d889640 -1 osd.5 90 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-21T03:58:00.063 INFO:journalctl@ceph.osd.5.smithi161.stdout:Sep 21 03:57:59 smithi161 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-5[45969]: 2024-09-21T03:57:59.856+0000 7fab25e6c640 -1 osd.5 90 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-09-21T03:57:23.429632+0000 front 2024-09-21T03:57:23.429554+0000 (oldest deadline 2024-09-21T03:57:46.329612+0000) 2024-09-21T03:58:00.064 INFO:journalctl@ceph.osd.5.smithi161.stdout:Sep 21 03:57:59 smithi161 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-5[45969]: 2024-09-21T03:57:59.856+0000 7fab25e6c640 -1 osd.5 90 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-09-21T03:57:32.731637+0000 front 2024-09-21T03:57:32.731745+0000 (oldest deadline 2024-09-21T03:57:53.831549+0000) 2024-09-21T03:58:00.334 INFO:journalctl@ceph.osd.6.smithi039.stdout:Sep 21 03:57:59 smithi039 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-6[60520]: 2024-09-21T03:57:59.940+0000 7f98a8e96640 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-09-21T03:57:22.466026+0000 front 2024-09-21T03:57:22.465966+0000 (oldest deadline 2024-09-21T03:57:46.566034+0000) 2024-09-21T03:58:00.334 INFO:journalctl@ceph.osd.6.smithi039.stdout:Sep 21 03:57:59 smithi039 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-6[60520]: 2024-09-21T03:57:59.940+0000 7f98a8e96640 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-09-21T03:57:30.067078+0000 front 2024-09-21T03:57:30.067189+0000 (oldest deadline 2024-09-21T03:57:53.566934+0000) 2024-09-21T03:58:00.334 INFO:journalctl@ceph.osd.9.smithi039.stdout:Sep 21 03:57:59 smithi039 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-9[67006]: 2024-09-21T03:57:59.904+0000 7f4a138c4640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-09-21T03:57:22.331574+0000 front 2024-09-21T03:57:22.332146+0000 (oldest deadline 2024-09-21T03:57:45.831720+0000) 2024-09-21T03:58:00.334 INFO:journalctl@ceph.osd.9.smithi039.stdout:Sep 21 03:57:59 smithi039 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-9[67006]: 2024-09-21T03:57:59.904+0000 7f4a138c4640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-09-21T03:57:29.932951+0000 front 2024-09-21T03:57:29.932930+0000 (oldest deadline 2024-09-21T03:57:54.632848+0000) 2024-09-21T03:58:00.467 INFO:journalctl@ceph.osd.8.smithi161.stdout:Sep 21 03:58:00 smithi161 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-8[52151]: 2024-09-21T03:58:00.251+0000 7f7e9a0f6640 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-09-21T03:57:29.951582+0000 front 2024-09-21T03:57:29.951481+0000 (oldest deadline 2024-09-21T03:57:54.051508+0000) 2024-09-21T03:58:00.468 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 21 03:58:00 smithi161 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-11[58829]: 2024-09-21T03:58:00.061+0000 7f538ef4a640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-09-21T03:57:21.581303+0000 front 2024-09-21T03:57:21.581333+0000 (oldest deadline 2024-09-21T03:57:46.881367+0000) 2024-09-21T03:58:00.468 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 21 03:58:00 smithi161 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-11[58829]: 2024-09-21T03:58:00.061+0000 7f538ef4a640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-09-21T03:57:32.782524+0000 front 2024-09-21T03:57:32.782318+0000 (oldest deadline 2024-09-21T03:57:54.482443+0000) 2024-09-21T03:58:00.600 INFO:journalctl@ceph.osd.7.smithi150.stdout:Sep 21 03:58:00 smithi150 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-7[51981]: 2024-09-21T03:58:00.320+0000 7f2e0cdd2640 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-09-21T03:57:31.734202+0000 front 2024-09-21T03:57:31.734234+0000 (oldest deadline 2024-09-21T03:57:55.834353+0000) 2024-09-21T03:58:00.975 INFO:journalctl@ceph.osd.10.smithi150.stdout:Sep 21 03:58:00 smithi150 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-10[58206]: 2024-09-21T03:58:00.599+0000 7f34020e1640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-09-21T03:57:23.120033+0000 front 2024-09-21T03:57:23.120109+0000 (oldest deadline 2024-09-21T03:57:45.920434+0000) 2024-09-21T03:58:00.975 INFO:journalctl@ceph.osd.10.smithi150.stdout:Sep 21 03:58:00 smithi150 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-10[58206]: 2024-09-21T03:58:00.599+0000 7f34020e1640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-09-21T03:57:31.221107+0000 front 2024-09-21T03:57:31.221162+0000 (oldest deadline 2024-09-21T03:57:56.521240+0000) 2024-09-21T03:58:01.218 INFO:journalctl@ceph.osd.5.smithi161.stdout:Sep 21 03:58:00 smithi161 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-5[45969]: 2024-09-21T03:58:00.829+0000 7fab25e6c640 -1 osd.5 90 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-09-21T03:57:23.429632+0000 front 2024-09-21T03:57:23.429554+0000 (oldest deadline 2024-09-21T03:57:46.329612+0000) 2024-09-21T03:58:01.218 INFO:journalctl@ceph.osd.5.smithi161.stdout:Sep 21 03:58:00 smithi161 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-5[45969]: 2024-09-21T03:58:00.829+0000 7fab25e6c640 -1 osd.5 90 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-09-21T03:57:32.731637+0000 front 2024-09-21T03:57:32.731745+0000 (oldest deadline 2024-09-21T03:57:53.831549+0000) 2024-09-21T03:58:01.218 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 21 03:58:01 smithi161 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-11[58829]: 2024-09-21T03:58:01.029+0000 7f538ef4a640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-09-21T03:57:21.581303+0000 front 2024-09-21T03:57:21.581333+0000 (oldest deadline 2024-09-21T03:57:46.881367+0000) 2024-09-21T03:58:01.218 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 21 03:58:01 smithi161 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-11[58829]: 2024-09-21T03:58:01.029+0000 7f538ef4a640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-09-21T03:57:32.782524+0000 front 2024-09-21T03:57:32.782318+0000 (oldest deadline 2024-09-21T03:57:54.482443+0000) 2024-09-21T03:58:01.334 INFO:journalctl@ceph.osd.6.smithi039.stdout:Sep 21 03:58:00 smithi039 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-6[60520]: 2024-09-21T03:58:00.893+0000 7f98a8e96640 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-09-21T03:57:22.466026+0000 front 2024-09-21T03:57:22.465966+0000 (oldest deadline 2024-09-21T03:57:46.566034+0000) 2024-09-21T03:58:01.334 INFO:journalctl@ceph.osd.6.smithi039.stdout:Sep 21 03:58:00 smithi039 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-6[60520]: 2024-09-21T03:58:00.893+0000 7f98a8e96640 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-09-21T03:57:30.067078+0000 front 2024-09-21T03:57:30.067189+0000 (oldest deadline 2024-09-21T03:57:53.566934+0000) 2024-09-21T03:58:01.334 INFO:journalctl@ceph.osd.9.smithi039.stdout:Sep 21 03:58:00 smithi039 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-9[67006]: 2024-09-21T03:58:00.891+0000 7f4a138c4640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-09-21T03:57:22.331574+0000 front 2024-09-21T03:57:22.332146+0000 (oldest deadline 2024-09-21T03:57:45.831720+0000) 2024-09-21T03:58:01.334 INFO:journalctl@ceph.osd.9.smithi039.stdout:Sep 21 03:58:00 smithi039 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-9[67006]: 2024-09-21T03:58:00.891+0000 7f4a138c4640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-09-21T03:57:29.932951+0000 front 2024-09-21T03:57:29.932930+0000 (oldest deadline 2024-09-21T03:57:54.632848+0000) 2024-09-21T03:58:01.631 INFO:journalctl@ceph.osd.7.smithi150.stdout:Sep 21 03:58:01 smithi150 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-7[51981]: 2024-09-21T03:58:01.279+0000 7f2e0cdd2640 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-09-21T03:57:31.734202+0000 front 2024-09-21T03:57:31.734234+0000 (oldest deadline 2024-09-21T03:57:55.834353+0000) 2024-09-21T03:58:01.718 INFO:journalctl@ceph.osd.8.smithi161.stdout:Sep 21 03:58:01 smithi161 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-8[52151]: 2024-09-21T03:58:01.217+0000 7f7e9a0f6640 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-09-21T03:57:29.951582+0000 front 2024-09-21T03:57:29.951481+0000 (oldest deadline 2024-09-21T03:57:54.051508+0000) 2024-09-21T03:58:01.975 INFO:journalctl@ceph.osd.10.smithi150.stdout:Sep 21 03:58:01 smithi150 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-10[58206]: 2024-09-21T03:58:01.629+0000 7f34020e1640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-09-21T03:57:23.120033+0000 front 2024-09-21T03:57:23.120109+0000 (oldest deadline 2024-09-21T03:57:45.920434+0000) 2024-09-21T03:58:01.975 INFO:journalctl@ceph.osd.10.smithi150.stdout:Sep 21 03:58:01 smithi150 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-10[58206]: 2024-09-21T03:58:01.629+0000 7f34020e1640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-09-21T03:57:31.221107+0000 front 2024-09-21T03:57:31.221162+0000 (oldest deadline 2024-09-21T03:57:56.521240+0000) 2024-09-21T03:58:02.055 INFO:journalctl@ceph.osd.5.smithi161.stdout:Sep 21 03:58:01 smithi161 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-5[45969]: 2024-09-21T03:58:01.801+0000 7fab25e6c640 -1 osd.5 90 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-09-21T03:57:23.429632+0000 front 2024-09-21T03:57:23.429554+0000 (oldest deadline 2024-09-21T03:57:46.329612+0000) 2024-09-21T03:58:02.055 INFO:journalctl@ceph.osd.5.smithi161.stdout:Sep 21 03:58:01 smithi161 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-5[45969]: 2024-09-21T03:58:01.801+0000 7fab25e6c640 -1 osd.5 90 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-09-21T03:57:32.731637+0000 front 2024-09-21T03:57:32.731745+0000 (oldest deadline 2024-09-21T03:57:53.831549+0000) 2024-09-21T03:58:02.334 INFO:journalctl@ceph.osd.6.smithi039.stdout:Sep 21 03:58:01 smithi039 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-6[60520]: 2024-09-21T03:58:01.856+0000 7f98a8e96640 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-09-21T03:57:22.466026+0000 front 2024-09-21T03:57:22.465966+0000 (oldest deadline 2024-09-21T03:57:46.566034+0000) 2024-09-21T03:58:02.334 INFO:journalctl@ceph.osd.6.smithi039.stdout:Sep 21 03:58:01 smithi039 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-6[60520]: 2024-09-21T03:58:01.856+0000 7f98a8e96640 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-09-21T03:57:30.067078+0000 front 2024-09-21T03:57:30.067189+0000 (oldest deadline 2024-09-21T03:57:53.566934+0000) 2024-09-21T03:58:02.334 INFO:journalctl@ceph.osd.9.smithi039.stdout:Sep 21 03:58:01 smithi039 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-9[67006]: 2024-09-21T03:58:01.883+0000 7f4a138c4640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-09-21T03:57:22.331574+0000 front 2024-09-21T03:57:22.332146+0000 (oldest deadline 2024-09-21T03:57:45.831720+0000) 2024-09-21T03:58:02.334 INFO:journalctl@ceph.osd.9.smithi039.stdout:Sep 21 03:58:01 smithi039 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-9[67006]: 2024-09-21T03:58:01.883+0000 7f4a138c4640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-09-21T03:57:29.932951+0000 front 2024-09-21T03:57:29.932930+0000 (oldest deadline 2024-09-21T03:57:54.632848+0000) 2024-09-21T03:58:02.468 INFO:journalctl@ceph.osd.8.smithi161.stdout:Sep 21 03:58:02 smithi161 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-8[52151]: 2024-09-21T03:58:02.240+0000 7f7e9a0f6640 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-09-21T03:57:29.951582+0000 front 2024-09-21T03:57:29.951481+0000 (oldest deadline 2024-09-21T03:57:54.051508+0000) 2024-09-21T03:58:02.468 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 21 03:58:02 smithi161 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-11[58829]: 2024-09-21T03:58:02.053+0000 7f538ef4a640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-09-21T03:57:21.581303+0000 front 2024-09-21T03:57:21.581333+0000 (oldest deadline 2024-09-21T03:57:46.881367+0000) 2024-09-21T03:58:02.468 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 21 03:58:02 smithi161 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-11[58829]: 2024-09-21T03:58:02.053+0000 7f538ef4a640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-09-21T03:57:32.782524+0000 front 2024-09-21T03:57:32.782318+0000 (oldest deadline 2024-09-21T03:57:54.482443+0000) 2024-09-21T03:58:02.625 INFO:journalctl@ceph.osd.7.smithi150.stdout:Sep 21 03:58:02 smithi150 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-7[51981]: 2024-09-21T03:58:02.266+0000 7f2e0cdd2640 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-09-21T03:57:31.734202+0000 front 2024-09-21T03:57:31.734234+0000 (oldest deadline 2024-09-21T03:57:55.834353+0000) 2024-09-21T03:58:02.975 INFO:journalctl@ceph.osd.10.smithi150.stdout:Sep 21 03:58:02 smithi150 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-10[58206]: 2024-09-21T03:58:02.624+0000 7f34020e1640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-09-21T03:57:23.120033+0000 front 2024-09-21T03:57:23.120109+0000 (oldest deadline 2024-09-21T03:57:45.920434+0000) 2024-09-21T03:58:02.975 INFO:journalctl@ceph.osd.10.smithi150.stdout:Sep 21 03:58:02 smithi150 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-10[58206]: 2024-09-21T03:58:02.624+0000 7f34020e1640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-09-21T03:57:31.221107+0000 front 2024-09-21T03:57:31.221162+0000 (oldest deadline 2024-09-21T03:57:56.521240+0000) 2024-09-21T03:58:02.975 INFO:journalctl@ceph.osd.10.smithi150.stdout:Sep 21 03:58:02 smithi150 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-10[58206]: 2024-09-21T03:58:02.624+0000 7f34020e1640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-09-21T03:57:40.022210+0000 front 2024-09-21T03:57:40.022157+0000 (oldest deadline 2024-09-21T03:58:01.722244+0000) 2024-09-21T03:58:03.218 INFO:journalctl@ceph.osd.5.smithi161.stdout:Sep 21 03:58:02 smithi161 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-5[45969]: 2024-09-21T03:58:02.838+0000 7fab25e6c640 -1 osd.5 90 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-09-21T03:57:23.429632+0000 front 2024-09-21T03:57:23.429554+0000 (oldest deadline 2024-09-21T03:57:46.329612+0000) 2024-09-21T03:58:03.218 INFO:journalctl@ceph.osd.5.smithi161.stdout:Sep 21 03:58:02 smithi161 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-5[45969]: 2024-09-21T03:58:02.838+0000 7fab25e6c640 -1 osd.5 90 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-09-21T03:57:32.731637+0000 front 2024-09-21T03:57:32.731745+0000 (oldest deadline 2024-09-21T03:57:53.831549+0000) 2024-09-21T03:58:03.218 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 21 03:58:03 smithi161 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-11[58829]: 2024-09-21T03:58:03.087+0000 7f538ef4a640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-09-21T03:57:21.581303+0000 front 2024-09-21T03:57:21.581333+0000 (oldest deadline 2024-09-21T03:57:46.881367+0000) 2024-09-21T03:58:03.218 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 21 03:58:03 smithi161 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-11[58829]: 2024-09-21T03:58:03.087+0000 7f538ef4a640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-09-21T03:57:32.782524+0000 front 2024-09-21T03:57:32.782318+0000 (oldest deadline 2024-09-21T03:57:54.482443+0000) 2024-09-21T03:58:03.334 INFO:journalctl@ceph.osd.6.smithi039.stdout:Sep 21 03:58:02 smithi039 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-6[60520]: 2024-09-21T03:58:02.840+0000 7f98a8e96640 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-09-21T03:57:22.466026+0000 front 2024-09-21T03:57:22.465966+0000 (oldest deadline 2024-09-21T03:57:46.566034+0000) 2024-09-21T03:58:03.334 INFO:journalctl@ceph.osd.6.smithi039.stdout:Sep 21 03:58:02 smithi039 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-6[60520]: 2024-09-21T03:58:02.840+0000 7f98a8e96640 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-09-21T03:57:30.067078+0000 front 2024-09-21T03:57:30.067189+0000 (oldest deadline 2024-09-21T03:57:53.566934+0000) 2024-09-21T03:58:03.334 INFO:journalctl@ceph.osd.9.smithi039.stdout:Sep 21 03:58:02 smithi039 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-9[67006]: 2024-09-21T03:58:02.876+0000 7f4a138c4640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-09-21T03:57:22.331574+0000 front 2024-09-21T03:57:22.332146+0000 (oldest deadline 2024-09-21T03:57:45.831720+0000) 2024-09-21T03:58:03.334 INFO:journalctl@ceph.osd.9.smithi039.stdout:Sep 21 03:58:02 smithi039 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-9[67006]: 2024-09-21T03:58:02.876+0000 7f4a138c4640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-09-21T03:57:29.932951+0000 front 2024-09-21T03:57:29.932930+0000 (oldest deadline 2024-09-21T03:57:54.632848+0000) 2024-09-21T03:58:03.334 INFO:journalctl@ceph.osd.9.smithi039.stdout:Sep 21 03:58:02 smithi039 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-9[67006]: 2024-09-21T03:58:02.876+0000 7f4a138c4640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-09-21T03:57:40.434486+0000 front 2024-09-21T03:57:40.434450+0000 (oldest deadline 2024-09-21T03:58:02.734524+0000) 2024-09-21T03:58:03.621 INFO:journalctl@ceph.osd.7.smithi150.stdout:Sep 21 03:58:03 smithi150 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-7[51981]: 2024-09-21T03:58:03.260+0000 7f2e0cdd2640 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-09-21T03:57:31.734202+0000 front 2024-09-21T03:57:31.734234+0000 (oldest deadline 2024-09-21T03:57:55.834353+0000) 2024-09-21T03:58:03.621 INFO:journalctl@ceph.osd.7.smithi150.stdout:Sep 21 03:58:03 smithi150 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-7[51981]: 2024-09-21T03:58:03.260+0000 7f2e0cdd2640 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-09-21T03:57:38.735442+0000 front 2024-09-21T03:57:38.735506+0000 (oldest deadline 2024-09-21T03:58:02.835564+0000) 2024-09-21T03:58:03.720 INFO:journalctl@ceph.osd.8.smithi161.stdout:Sep 21 03:58:03 smithi161 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-8[52151]: 2024-09-21T03:58:03.244+0000 7f7e9a0f6640 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-09-21T03:57:29.951582+0000 front 2024-09-21T03:57:29.951481+0000 (oldest deadline 2024-09-21T03:57:54.051508+0000) 2024-09-21T03:58:03.976 INFO:journalctl@ceph.osd.10.smithi150.stdout:Sep 21 03:58:03 smithi150 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-10[58206]: 2024-09-21T03:58:03.617+0000 7f34020e1640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-09-21T03:57:23.120033+0000 front 2024-09-21T03:57:23.120109+0000 (oldest deadline 2024-09-21T03:57:45.920434+0000) 2024-09-21T03:58:03.976 INFO:journalctl@ceph.osd.10.smithi150.stdout:Sep 21 03:58:03 smithi150 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-10[58206]: 2024-09-21T03:58:03.617+0000 7f34020e1640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-09-21T03:57:31.221107+0000 front 2024-09-21T03:57:31.221162+0000 (oldest deadline 2024-09-21T03:57:56.521240+0000) 2024-09-21T03:58:03.976 INFO:journalctl@ceph.osd.10.smithi150.stdout:Sep 21 03:58:03 smithi150 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-10[58206]: 2024-09-21T03:58:03.617+0000 7f34020e1640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-09-21T03:57:40.022210+0000 front 2024-09-21T03:57:40.022157+0000 (oldest deadline 2024-09-21T03:58:01.722244+0000) 2024-09-21T03:58:04.099 INFO:journalctl@ceph.osd.5.smithi161.stdout:Sep 21 03:58:03 smithi161 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-5[45969]: 2024-09-21T03:58:03.789+0000 7fab25e6c640 -1 osd.5 90 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-09-21T03:57:23.429632+0000 front 2024-09-21T03:57:23.429554+0000 (oldest deadline 2024-09-21T03:57:46.329612+0000) 2024-09-21T03:58:04.099 INFO:journalctl@ceph.osd.5.smithi161.stdout:Sep 21 03:58:03 smithi161 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-5[45969]: 2024-09-21T03:58:03.789+0000 7fab25e6c640 -1 osd.5 90 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-09-21T03:57:32.731637+0000 front 2024-09-21T03:57:32.731745+0000 (oldest deadline 2024-09-21T03:57:53.831549+0000) 2024-09-21T03:58:04.334 INFO:journalctl@ceph.osd.6.smithi039.stdout:Sep 21 03:58:03 smithi039 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-6[60520]: 2024-09-21T03:58:03.881+0000 7f98a8e96640 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-09-21T03:57:22.466026+0000 front 2024-09-21T03:57:22.465966+0000 (oldest deadline 2024-09-21T03:57:46.566034+0000) 2024-09-21T03:58:04.334 INFO:journalctl@ceph.osd.6.smithi039.stdout:Sep 21 03:58:03 smithi039 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-6[60520]: 2024-09-21T03:58:03.881+0000 7f98a8e96640 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-09-21T03:57:30.067078+0000 front 2024-09-21T03:57:30.067189+0000 (oldest deadline 2024-09-21T03:57:53.566934+0000) 2024-09-21T03:58:04.334 INFO:journalctl@ceph.osd.6.smithi039.stdout:Sep 21 03:58:03 smithi039 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-6[60520]: 2024-09-21T03:58:03.881+0000 7f98a8e96640 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-09-21T03:57:38.267956+0000 front 2024-09-21T03:57:38.268018+0000 (oldest deadline 2024-09-21T03:58:02.967757+0000) 2024-09-21T03:58:04.334 INFO:journalctl@ceph.osd.9.smithi039.stdout:Sep 21 03:58:03 smithi039 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-9[67006]: 2024-09-21T03:58:03.855+0000 7f4a138c4640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-09-21T03:57:22.331574+0000 front 2024-09-21T03:57:22.332146+0000 (oldest deadline 2024-09-21T03:57:45.831720+0000) 2024-09-21T03:58:04.334 INFO:journalctl@ceph.osd.9.smithi039.stdout:Sep 21 03:58:03 smithi039 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-9[67006]: 2024-09-21T03:58:03.855+0000 7f4a138c4640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-09-21T03:57:29.932951+0000 front 2024-09-21T03:57:29.932930+0000 (oldest deadline 2024-09-21T03:57:54.632848+0000) 2024-09-21T03:58:04.334 INFO:journalctl@ceph.osd.9.smithi039.stdout:Sep 21 03:58:03 smithi039 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-9[67006]: 2024-09-21T03:58:03.855+0000 7f4a138c4640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-09-21T03:57:40.434486+0000 front 2024-09-21T03:57:40.434450+0000 (oldest deadline 2024-09-21T03:58:02.734524+0000) 2024-09-21T03:58:04.468 INFO:journalctl@ceph.osd.8.smithi161.stdout:Sep 21 03:58:04 smithi161 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-8[52151]: 2024-09-21T03:58:04.273+0000 7f7e9a0f6640 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-09-21T03:57:29.951582+0000 front 2024-09-21T03:57:29.951481+0000 (oldest deadline 2024-09-21T03:57:54.051508+0000) 2024-09-21T03:58:04.468 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 21 03:58:04 smithi161 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-11[58829]: 2024-09-21T03:58:04.097+0000 7f538ef4a640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-09-21T03:57:21.581303+0000 front 2024-09-21T03:57:21.581333+0000 (oldest deadline 2024-09-21T03:57:46.881367+0000) 2024-09-21T03:58:04.468 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 21 03:58:04 smithi161 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-11[58829]: 2024-09-21T03:58:04.097+0000 7f538ef4a640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-09-21T03:57:32.782524+0000 front 2024-09-21T03:57:32.782318+0000 (oldest deadline 2024-09-21T03:57:54.482443+0000) 2024-09-21T03:58:04.631 INFO:journalctl@ceph.osd.7.smithi150.stdout:Sep 21 03:58:04 smithi150 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-7[51981]: 2024-09-21T03:58:04.256+0000 7f2e0cdd2640 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-09-21T03:57:31.734202+0000 front 2024-09-21T03:57:31.734234+0000 (oldest deadline 2024-09-21T03:57:55.834353+0000) 2024-09-21T03:58:04.631 INFO:journalctl@ceph.osd.7.smithi150.stdout:Sep 21 03:58:04 smithi150 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-7[51981]: 2024-09-21T03:58:04.256+0000 7f2e0cdd2640 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-09-21T03:57:38.735442+0000 front 2024-09-21T03:57:38.735506+0000 (oldest deadline 2024-09-21T03:58:02.835564+0000) 2024-09-21T03:58:04.975 INFO:journalctl@ceph.osd.10.smithi150.stdout:Sep 21 03:58:04 smithi150 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-10[58206]: 2024-09-21T03:58:04.629+0000 7f34020e1640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-09-21T03:57:23.120033+0000 front 2024-09-21T03:57:23.120109+0000 (oldest deadline 2024-09-21T03:57:45.920434+0000) 2024-09-21T03:58:04.975 INFO:journalctl@ceph.osd.10.smithi150.stdout:Sep 21 03:58:04 smithi150 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-10[58206]: 2024-09-21T03:58:04.629+0000 7f34020e1640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-09-21T03:57:31.221107+0000 front 2024-09-21T03:57:31.221162+0000 (oldest deadline 2024-09-21T03:57:56.521240+0000) 2024-09-21T03:58:04.975 INFO:journalctl@ceph.osd.10.smithi150.stdout:Sep 21 03:58:04 smithi150 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-10[58206]: 2024-09-21T03:58:04.629+0000 7f34020e1640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-09-21T03:57:40.022210+0000 front 2024-09-21T03:57:40.022157+0000 (oldest deadline 2024-09-21T03:58:01.722244+0000) 2024-09-21T03:58:05.084 INFO:journalctl@ceph.osd.6.smithi039.stdout:Sep 21 03:58:04 smithi039 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-6[60520]: 2024-09-21T03:58:04.856+0000 7f98a8e96640 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-09-21T03:57:22.466026+0000 front 2024-09-21T03:57:22.465966+0000 (oldest deadline 2024-09-21T03:57:46.566034+0000) 2024-09-21T03:58:05.084 INFO:journalctl@ceph.osd.6.smithi039.stdout:Sep 21 03:58:04 smithi039 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-6[60520]: 2024-09-21T03:58:04.856+0000 7f98a8e96640 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-09-21T03:57:30.067078+0000 front 2024-09-21T03:57:30.067189+0000 (oldest deadline 2024-09-21T03:57:53.566934+0000) 2024-09-21T03:58:05.084 INFO:journalctl@ceph.osd.6.smithi039.stdout:Sep 21 03:58:04 smithi039 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-6[60520]: 2024-09-21T03:58:04.856+0000 7f98a8e96640 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-09-21T03:57:38.267956+0000 front 2024-09-21T03:57:38.268018+0000 (oldest deadline 2024-09-21T03:58:02.967757+0000) 2024-09-21T03:58:05.084 INFO:journalctl@ceph.osd.9.smithi039.stdout:Sep 21 03:58:04 smithi039 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-9[67006]: 2024-09-21T03:58:04.807+0000 7f4a138c4640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-09-21T03:57:22.331574+0000 front 2024-09-21T03:57:22.332146+0000 (oldest deadline 2024-09-21T03:57:45.831720+0000) 2024-09-21T03:58:05.084 INFO:journalctl@ceph.osd.9.smithi039.stdout:Sep 21 03:58:04 smithi039 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-9[67006]: 2024-09-21T03:58:04.807+0000 7f4a138c4640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-09-21T03:57:29.932951+0000 front 2024-09-21T03:57:29.932930+0000 (oldest deadline 2024-09-21T03:57:54.632848+0000) 2024-09-21T03:58:05.084 INFO:journalctl@ceph.osd.9.smithi039.stdout:Sep 21 03:58:04 smithi039 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-9[67006]: 2024-09-21T03:58:04.807+0000 7f4a138c4640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-09-21T03:57:40.434486+0000 front 2024-09-21T03:57:40.434450+0000 (oldest deadline 2024-09-21T03:58:02.734524+0000) 2024-09-21T03:58:05.468 INFO:journalctl@ceph.osd.5.smithi161.stdout:Sep 21 03:58:05 smithi161 podman[67727]: 2024-09-21 03:58:05.280364829 +0000 UTC m=+5.647508668 container died 23913b296872b609685a1b7d29b33f838e71bc83b84bfc113b29cbdeaa084eab (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-5, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, ceph=True, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221) 2024-09-21T03:58:05.468 INFO:journalctl@ceph.osd.8.smithi161.stdout:Sep 21 03:58:05 smithi161 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-8[52151]: 2024-09-21T03:58:05.311+0000 7f7e9a0f6640 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-09-21T03:57:29.951582+0000 front 2024-09-21T03:57:29.951481+0000 (oldest deadline 2024-09-21T03:57:54.051508+0000) 2024-09-21T03:58:05.468 INFO:journalctl@ceph.osd.8.smithi161.stdout:Sep 21 03:58:05 smithi161 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-8[52151]: 2024-09-21T03:58:05.311+0000 7f7e9a0f6640 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-09-21T03:57:39.753762+0000 front 2024-09-21T03:57:39.753690+0000 (oldest deadline 2024-09-21T03:58:04.453819+0000) 2024-09-21T03:58:05.469 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 21 03:58:05 smithi161 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-11[58829]: 2024-09-21T03:58:05.137+0000 7f538ef4a640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-09-21T03:57:21.581303+0000 front 2024-09-21T03:57:21.581333+0000 (oldest deadline 2024-09-21T03:57:46.881367+0000) 2024-09-21T03:58:05.469 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 21 03:58:05 smithi161 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-11[58829]: 2024-09-21T03:58:05.137+0000 7f538ef4a640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-09-21T03:57:32.782524+0000 front 2024-09-21T03:57:32.782318+0000 (oldest deadline 2024-09-21T03:57:54.482443+0000) 2024-09-21T03:58:05.475 INFO:journalctl@ceph.osd.7.smithi150.stdout:Sep 21 03:58:05 smithi150 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-7[51981]: 2024-09-21T03:58:05.214+0000 7f2e0cdd2640 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-09-21T03:57:31.734202+0000 front 2024-09-21T03:57:31.734234+0000 (oldest deadline 2024-09-21T03:57:55.834353+0000) 2024-09-21T03:58:05.475 INFO:journalctl@ceph.osd.7.smithi150.stdout:Sep 21 03:58:05 smithi150 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-7[51981]: 2024-09-21T03:58:05.214+0000 7f2e0cdd2640 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-09-21T03:57:38.735442+0000 front 2024-09-21T03:57:38.735506+0000 (oldest deadline 2024-09-21T03:58:02.835564+0000) 2024-09-21T03:58:05.906 INFO:journalctl@ceph.osd.5.smithi161.stdout:Sep 21 03:58:05 smithi161 podman[67727]: 2024-09-21 03:58:05.695104648 +0000 UTC m=+6.062248485 container cleanup 23913b296872b609685a1b7d29b33f838e71bc83b84bfc113b29cbdeaa084eab (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-5, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, io.buildah.version=1.37.2, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_CLEAN=True) 2024-09-21T03:58:05.906 INFO:journalctl@ceph.osd.5.smithi161.stdout:Sep 21 03:58:05 smithi161 bash[67727]: ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-5 2024-09-21T03:58:05.975 INFO:journalctl@ceph.osd.10.smithi150.stdout:Sep 21 03:58:05 smithi150 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-10[58206]: 2024-09-21T03:58:05.604+0000 7f34020e1640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-09-21T03:57:23.120033+0000 front 2024-09-21T03:57:23.120109+0000 (oldest deadline 2024-09-21T03:57:45.920434+0000) 2024-09-21T03:58:05.975 INFO:journalctl@ceph.osd.10.smithi150.stdout:Sep 21 03:58:05 smithi150 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-10[58206]: 2024-09-21T03:58:05.604+0000 7f34020e1640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-09-21T03:57:31.221107+0000 front 2024-09-21T03:57:31.221162+0000 (oldest deadline 2024-09-21T03:57:56.521240+0000) 2024-09-21T03:58:05.975 INFO:journalctl@ceph.osd.10.smithi150.stdout:Sep 21 03:58:05 smithi150 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-10[58206]: 2024-09-21T03:58:05.604+0000 7f34020e1640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-09-21T03:57:40.022210+0000 front 2024-09-21T03:57:40.022157+0000 (oldest deadline 2024-09-21T03:58:01.722244+0000) 2024-09-21T03:58:06.084 INFO:journalctl@ceph.osd.6.smithi039.stdout:Sep 21 03:58:05 smithi039 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-6[60520]: 2024-09-21T03:58:05.831+0000 7f98a8e96640 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-09-21T03:57:22.466026+0000 front 2024-09-21T03:57:22.465966+0000 (oldest deadline 2024-09-21T03:57:46.566034+0000) 2024-09-21T03:58:06.084 INFO:journalctl@ceph.osd.6.smithi039.stdout:Sep 21 03:58:05 smithi039 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-6[60520]: 2024-09-21T03:58:05.831+0000 7f98a8e96640 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-09-21T03:57:30.067078+0000 front 2024-09-21T03:57:30.067189+0000 (oldest deadline 2024-09-21T03:57:53.566934+0000) 2024-09-21T03:58:06.084 INFO:journalctl@ceph.osd.6.smithi039.stdout:Sep 21 03:58:05 smithi039 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-6[60520]: 2024-09-21T03:58:05.831+0000 7f98a8e96640 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-09-21T03:57:38.267956+0000 front 2024-09-21T03:57:38.268018+0000 (oldest deadline 2024-09-21T03:58:02.967757+0000) 2024-09-21T03:58:06.085 INFO:journalctl@ceph.osd.9.smithi039.stdout:Sep 21 03:58:05 smithi039 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-9[67006]: 2024-09-21T03:58:05.806+0000 7f4a138c4640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-09-21T03:57:22.331574+0000 front 2024-09-21T03:57:22.332146+0000 (oldest deadline 2024-09-21T03:57:45.831720+0000) 2024-09-21T03:58:06.085 INFO:journalctl@ceph.osd.9.smithi039.stdout:Sep 21 03:58:05 smithi039 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-9[67006]: 2024-09-21T03:58:05.806+0000 7f4a138c4640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-09-21T03:57:29.932951+0000 front 2024-09-21T03:57:29.932930+0000 (oldest deadline 2024-09-21T03:57:54.632848+0000) 2024-09-21T03:58:06.085 INFO:journalctl@ceph.osd.9.smithi039.stdout:Sep 21 03:58:05 smithi039 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-9[67006]: 2024-09-21T03:58:05.806+0000 7f4a138c4640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-09-21T03:57:40.434486+0000 front 2024-09-21T03:57:40.434450+0000 (oldest deadline 2024-09-21T03:58:02.734524+0000) 2024-09-21T03:58:06.218 INFO:journalctl@ceph.osd.5.smithi161.stdout:Sep 21 03:58:05 smithi161 podman[67743]: 2024-09-21 03:58:05.904693918 +0000 UTC m=+0.622929400 container remove 23913b296872b609685a1b7d29b33f838e71bc83b84bfc113b29cbdeaa084eab (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-5, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23) 2024-09-21T03:58:06.218 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 21 03:58:06 smithi161 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-11[58829]: 2024-09-21T03:58:06.116+0000 7f538ef4a640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-09-21T03:57:21.581303+0000 front 2024-09-21T03:57:21.581333+0000 (oldest deadline 2024-09-21T03:57:46.881367+0000) 2024-09-21T03:58:06.218 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 21 03:58:06 smithi161 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-11[58829]: 2024-09-21T03:58:06.116+0000 7f538ef4a640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-09-21T03:57:32.782524+0000 front 2024-09-21T03:57:32.782318+0000 (oldest deadline 2024-09-21T03:57:54.482443+0000) 2024-09-21T03:58:06.218 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 21 03:58:06 smithi161 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-11[58829]: 2024-09-21T03:58:06.116+0000 7f538ef4a640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-09-21T03:57:40.184584+0000 front 2024-09-21T03:57:40.184544+0000 (oldest deadline 2024-09-21T03:58:05.484668+0000) 2024-09-21T03:58:06.475 INFO:journalctl@ceph.osd.7.smithi150.stdout:Sep 21 03:58:06 smithi150 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-7[51981]: 2024-09-21T03:58:06.213+0000 7f2e0cdd2640 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-09-21T03:57:31.734202+0000 front 2024-09-21T03:57:31.734234+0000 (oldest deadline 2024-09-21T03:57:55.834353+0000) 2024-09-21T03:58:06.475 INFO:journalctl@ceph.osd.7.smithi150.stdout:Sep 21 03:58:06 smithi150 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-7[51981]: 2024-09-21T03:58:06.213+0000 7f2e0cdd2640 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-09-21T03:57:38.735442+0000 front 2024-09-21T03:57:38.735506+0000 (oldest deadline 2024-09-21T03:58:02.835564+0000) 2024-09-21T03:58:06.626 INFO:journalctl@ceph.osd.5.smithi161.stdout:Sep 21 03:58:06 smithi161 podman[67829]: 2024-09-21 03:58:06.213737345 +0000 UTC m=+0.015881494 image pull ddce307e1f3263fb077b4f3902e15660050631dad4d2823a20ee3f72bf18d977 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262 2024-09-21T03:58:06.626 INFO:journalctl@ceph.osd.5.smithi161.stdout:Sep 21 03:58:06 smithi161 podman[67829]: 2024-09-21 03:58:06.31365737 +0000 UTC m=+0.115801519 container create 09dcd96d79bc08895f8c5a4b1a72af39b39ac090a2b505f9525b2d31ff410ecb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-5-deactivate, org.label-schema.license=GPLv2, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, ceph=True) 2024-09-21T03:58:06.626 INFO:journalctl@ceph.osd.5.smithi161.stdout:Sep 21 03:58:06 smithi161 podman[67829]: 2024-09-21 03:58:06.541195126 +0000 UTC m=+0.343339274 container init 09dcd96d79bc08895f8c5a4b1a72af39b39ac090a2b505f9525b2d31ff410ecb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-5-deactivate, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, ceph=True) 2024-09-21T03:58:06.626 INFO:journalctl@ceph.osd.5.smithi161.stdout:Sep 21 03:58:06 smithi161 podman[67829]: 2024-09-21 03:58:06.544479867 +0000 UTC m=+0.346624016 container start 09dcd96d79bc08895f8c5a4b1a72af39b39ac090a2b505f9525b2d31ff410ecb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-5-deactivate, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, ceph=True, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=) 2024-09-21T03:58:06.626 INFO:journalctl@ceph.osd.5.smithi161.stdout:Sep 21 03:58:06 smithi161 podman[67829]: 2024-09-21 03:58:06.624496578 +0000 UTC m=+0.426640742 container attach 09dcd96d79bc08895f8c5a4b1a72af39b39ac090a2b505f9525b2d31ff410ecb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-5-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, GIT_CLEAN=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23) 2024-09-21T03:58:06.627 INFO:journalctl@ceph.osd.8.smithi161.stdout:Sep 21 03:58:06 smithi161 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-8[52151]: 2024-09-21T03:58:06.319+0000 7f7e9a0f6640 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-09-21T03:57:29.951582+0000 front 2024-09-21T03:57:29.951481+0000 (oldest deadline 2024-09-21T03:57:54.051508+0000) 2024-09-21T03:58:06.627 INFO:journalctl@ceph.osd.8.smithi161.stdout:Sep 21 03:58:06 smithi161 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-8[52151]: 2024-09-21T03:58:06.319+0000 7f7e9a0f6640 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-09-21T03:57:39.753762+0000 front 2024-09-21T03:57:39.753690+0000 (oldest deadline 2024-09-21T03:58:04.453819+0000) 2024-09-21T03:58:06.968 INFO:journalctl@ceph.osd.5.smithi161.stdout:Sep 21 03:58:06 smithi161 podman[67829]: 2024-09-21 03:58:06.740360231 +0000 UTC m=+0.542504397 container died 09dcd96d79bc08895f8c5a4b1a72af39b39ac090a2b505f9525b2d31ff410ecb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-5-deactivate, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, ceph=True, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, org.label-schema.license=GPLv2) 2024-09-21T03:58:06.975 INFO:journalctl@ceph.osd.10.smithi150.stdout:Sep 21 03:58:06 smithi150 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-10[58206]: 2024-09-21T03:58:06.568+0000 7f34020e1640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-09-21T03:57:23.120033+0000 front 2024-09-21T03:57:23.120109+0000 (oldest deadline 2024-09-21T03:57:45.920434+0000) 2024-09-21T03:58:06.975 INFO:journalctl@ceph.osd.10.smithi150.stdout:Sep 21 03:58:06 smithi150 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-10[58206]: 2024-09-21T03:58:06.568+0000 7f34020e1640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-09-21T03:57:31.221107+0000 front 2024-09-21T03:57:31.221162+0000 (oldest deadline 2024-09-21T03:57:56.521240+0000) 2024-09-21T03:58:06.975 INFO:journalctl@ceph.osd.10.smithi150.stdout:Sep 21 03:58:06 smithi150 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-10[58206]: 2024-09-21T03:58:06.568+0000 7f34020e1640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-09-21T03:57:40.022210+0000 front 2024-09-21T03:57:40.022157+0000 (oldest deadline 2024-09-21T03:58:01.722244+0000) 2024-09-21T03:58:07.084 INFO:journalctl@ceph.osd.6.smithi039.stdout:Sep 21 03:58:06 smithi039 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-6[60520]: 2024-09-21T03:58:06.795+0000 7f98a8e96640 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-09-21T03:57:22.466026+0000 front 2024-09-21T03:57:22.465966+0000 (oldest deadline 2024-09-21T03:57:46.566034+0000) 2024-09-21T03:58:07.084 INFO:journalctl@ceph.osd.6.smithi039.stdout:Sep 21 03:58:06 smithi039 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-6[60520]: 2024-09-21T03:58:06.795+0000 7f98a8e96640 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-09-21T03:57:30.067078+0000 front 2024-09-21T03:57:30.067189+0000 (oldest deadline 2024-09-21T03:57:53.566934+0000) 2024-09-21T03:58:07.084 INFO:journalctl@ceph.osd.6.smithi039.stdout:Sep 21 03:58:06 smithi039 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-6[60520]: 2024-09-21T03:58:06.795+0000 7f98a8e96640 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-09-21T03:57:38.267956+0000 front 2024-09-21T03:57:38.268018+0000 (oldest deadline 2024-09-21T03:58:02.967757+0000) 2024-09-21T03:58:07.084 INFO:journalctl@ceph.osd.9.smithi039.stdout:Sep 21 03:58:06 smithi039 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-9[67006]: 2024-09-21T03:58:06.836+0000 7f4a138c4640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-09-21T03:57:22.331574+0000 front 2024-09-21T03:57:22.332146+0000 (oldest deadline 2024-09-21T03:57:45.831720+0000) 2024-09-21T03:58:07.084 INFO:journalctl@ceph.osd.9.smithi039.stdout:Sep 21 03:58:06 smithi039 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-9[67006]: 2024-09-21T03:58:06.836+0000 7f4a138c4640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-09-21T03:57:29.932951+0000 front 2024-09-21T03:57:29.932930+0000 (oldest deadline 2024-09-21T03:57:54.632848+0000) 2024-09-21T03:58:07.084 INFO:journalctl@ceph.osd.9.smithi039.stdout:Sep 21 03:58:06 smithi039 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-9[67006]: 2024-09-21T03:58:06.836+0000 7f4a138c4640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-09-21T03:57:40.434486+0000 front 2024-09-21T03:57:40.434450+0000 (oldest deadline 2024-09-21T03:58:02.734524+0000) 2024-09-21T03:58:07.337 INFO:journalctl@ceph.osd.8.smithi161.stdout:Sep 21 03:58:07 smithi161 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-8[52151]: 2024-09-21T03:58:07.313+0000 7f7e9a0f6640 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-09-21T03:57:29.951582+0000 front 2024-09-21T03:57:29.951481+0000 (oldest deadline 2024-09-21T03:57:54.051508+0000) 2024-09-21T03:58:07.337 INFO:journalctl@ceph.osd.8.smithi161.stdout:Sep 21 03:58:07 smithi161 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-8[52151]: 2024-09-21T03:58:07.313+0000 7f7e9a0f6640 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-09-21T03:57:39.753762+0000 front 2024-09-21T03:57:39.753690+0000 (oldest deadline 2024-09-21T03:58:04.453819+0000) 2024-09-21T03:58:07.338 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 21 03:58:07 smithi161 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-11[58829]: 2024-09-21T03:58:07.109+0000 7f538ef4a640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-09-21T03:57:21.581303+0000 front 2024-09-21T03:57:21.581333+0000 (oldest deadline 2024-09-21T03:57:46.881367+0000) 2024-09-21T03:58:07.338 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 21 03:58:07 smithi161 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-11[58829]: 2024-09-21T03:58:07.109+0000 7f538ef4a640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-09-21T03:57:32.782524+0000 front 2024-09-21T03:57:32.782318+0000 (oldest deadline 2024-09-21T03:57:54.482443+0000) 2024-09-21T03:58:07.338 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 21 03:58:07 smithi161 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-11[58829]: 2024-09-21T03:58:07.109+0000 7f538ef4a640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-09-21T03:57:40.184584+0000 front 2024-09-21T03:57:40.184544+0000 (oldest deadline 2024-09-21T03:58:05.484668+0000) 2024-09-21T03:58:07.361 DEBUG:teuthology.orchestra.run.smithi161:> sudo pkill -f 'journalctl -f -n 0 -u ceph-51116d5c-77ca-11ef-baf5-efdc52797490@osd.5.service' 2024-09-21T03:58:07.402 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-21T03:58:07.402 INFO:tasks.cephadm.osd.5:Stopped osd.5 2024-09-21T03:58:07.402 INFO:tasks.cephadm.osd.6:Stopping osd.6... 2024-09-21T03:58:07.402 DEBUG:teuthology.orchestra.run.smithi039:> sudo systemctl stop ceph-51116d5c-77ca-11ef-baf5-efdc52797490@osd.6 2024-09-21T03:58:07.526 INFO:journalctl@ceph.osd.7.smithi150.stdout:Sep 21 03:58:07 smithi150 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-7[51981]: 2024-09-21T03:58:07.253+0000 7f2e0cdd2640 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-09-21T03:57:31.734202+0000 front 2024-09-21T03:57:31.734234+0000 (oldest deadline 2024-09-21T03:57:55.834353+0000) 2024-09-21T03:58:07.526 INFO:journalctl@ceph.osd.7.smithi150.stdout:Sep 21 03:58:07 smithi150 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-7[51981]: 2024-09-21T03:58:07.253+0000 7f2e0cdd2640 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-09-21T03:57:38.735442+0000 front 2024-09-21T03:57:38.735506+0000 (oldest deadline 2024-09-21T03:58:02.835564+0000) 2024-09-21T03:58:07.705 INFO:journalctl@ceph.osd.6.smithi039.stdout:Sep 21 03:58:07 smithi039 systemd[1]: Stopping Ceph osd.6 for 51116d5c-77ca-11ef-baf5-efdc52797490... 2024-09-21T03:58:07.975 INFO:journalctl@ceph.osd.10.smithi150.stdout:Sep 21 03:58:07 smithi150 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-10[58206]: 2024-09-21T03:58:07.523+0000 7f34020e1640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-09-21T03:57:23.120033+0000 front 2024-09-21T03:57:23.120109+0000 (oldest deadline 2024-09-21T03:57:45.920434+0000) 2024-09-21T03:58:07.975 INFO:journalctl@ceph.osd.10.smithi150.stdout:Sep 21 03:58:07 smithi150 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-10[58206]: 2024-09-21T03:58:07.523+0000 7f34020e1640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-09-21T03:57:31.221107+0000 front 2024-09-21T03:57:31.221162+0000 (oldest deadline 2024-09-21T03:57:56.521240+0000) 2024-09-21T03:58:07.975 INFO:journalctl@ceph.osd.10.smithi150.stdout:Sep 21 03:58:07 smithi150 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-10[58206]: 2024-09-21T03:58:07.523+0000 7f34020e1640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-09-21T03:57:40.022210+0000 front 2024-09-21T03:57:40.022157+0000 (oldest deadline 2024-09-21T03:58:01.722244+0000) 2024-09-21T03:58:08.084 INFO:journalctl@ceph.osd.6.smithi039.stdout:Sep 21 03:58:07 smithi039 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-6[60520]: 2024-09-21T03:58:07.703+0000 7f98ac8ab640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.6 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-21T03:58:08.084 INFO:journalctl@ceph.osd.6.smithi039.stdout:Sep 21 03:58:07 smithi039 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-6[60520]: 2024-09-21T03:58:07.703+0000 7f98ac8ab640 -1 osd.6 90 *** Got signal Terminated *** 2024-09-21T03:58:08.084 INFO:journalctl@ceph.osd.6.smithi039.stdout:Sep 21 03:58:07 smithi039 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-6[60520]: 2024-09-21T03:58:07.703+0000 7f98ac8ab640 -1 osd.6 90 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-21T03:58:08.084 INFO:journalctl@ceph.osd.6.smithi039.stdout:Sep 21 03:58:07 smithi039 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-6[60520]: 2024-09-21T03:58:07.790+0000 7f98a8e96640 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-09-21T03:57:22.466026+0000 front 2024-09-21T03:57:22.465966+0000 (oldest deadline 2024-09-21T03:57:46.566034+0000) 2024-09-21T03:58:08.084 INFO:journalctl@ceph.osd.6.smithi039.stdout:Sep 21 03:58:07 smithi039 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-6[60520]: 2024-09-21T03:58:07.790+0000 7f98a8e96640 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-09-21T03:57:30.067078+0000 front 2024-09-21T03:57:30.067189+0000 (oldest deadline 2024-09-21T03:57:53.566934+0000) 2024-09-21T03:58:08.084 INFO:journalctl@ceph.osd.6.smithi039.stdout:Sep 21 03:58:07 smithi039 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-6[60520]: 2024-09-21T03:58:07.790+0000 7f98a8e96640 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-09-21T03:57:38.267956+0000 front 2024-09-21T03:57:38.268018+0000 (oldest deadline 2024-09-21T03:58:02.967757+0000) 2024-09-21T03:58:08.085 INFO:journalctl@ceph.osd.9.smithi039.stdout:Sep 21 03:58:07 smithi039 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-9[67006]: 2024-09-21T03:58:07.854+0000 7f4a138c4640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-09-21T03:57:22.331574+0000 front 2024-09-21T03:57:22.332146+0000 (oldest deadline 2024-09-21T03:57:45.831720+0000) 2024-09-21T03:58:08.085 INFO:journalctl@ceph.osd.9.smithi039.stdout:Sep 21 03:58:07 smithi039 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-9[67006]: 2024-09-21T03:58:07.854+0000 7f4a138c4640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-09-21T03:57:29.932951+0000 front 2024-09-21T03:57:29.932930+0000 (oldest deadline 2024-09-21T03:57:54.632848+0000) 2024-09-21T03:58:08.085 INFO:journalctl@ceph.osd.9.smithi039.stdout:Sep 21 03:58:07 smithi039 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-9[67006]: 2024-09-21T03:58:07.854+0000 7f4a138c4640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-09-21T03:57:40.434486+0000 front 2024-09-21T03:57:40.434450+0000 (oldest deadline 2024-09-21T03:58:02.734524+0000) 2024-09-21T03:58:08.344 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 21 03:58:08 smithi161 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-11[58829]: 2024-09-21T03:58:08.077+0000 7f538ef4a640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-09-21T03:57:21.581303+0000 front 2024-09-21T03:57:21.581333+0000 (oldest deadline 2024-09-21T03:57:46.881367+0000) 2024-09-21T03:58:08.345 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 21 03:58:08 smithi161 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-11[58829]: 2024-09-21T03:58:08.077+0000 7f538ef4a640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-09-21T03:57:32.782524+0000 front 2024-09-21T03:57:32.782318+0000 (oldest deadline 2024-09-21T03:57:54.482443+0000) 2024-09-21T03:58:08.345 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 21 03:58:08 smithi161 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-11[58829]: 2024-09-21T03:58:08.077+0000 7f538ef4a640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-09-21T03:57:40.184584+0000 front 2024-09-21T03:57:40.184544+0000 (oldest deadline 2024-09-21T03:58:05.484668+0000) 2024-09-21T03:58:08.475 INFO:journalctl@ceph.osd.7.smithi150.stdout:Sep 21 03:58:08 smithi150 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-7[51981]: 2024-09-21T03:58:08.206+0000 7f2e0cdd2640 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-09-21T03:57:31.734202+0000 front 2024-09-21T03:57:31.734234+0000 (oldest deadline 2024-09-21T03:57:55.834353+0000) 2024-09-21T03:58:08.475 INFO:journalctl@ceph.osd.7.smithi150.stdout:Sep 21 03:58:08 smithi150 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-7[51981]: 2024-09-21T03:58:08.206+0000 7f2e0cdd2640 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-09-21T03:57:38.735442+0000 front 2024-09-21T03:57:38.735506+0000 (oldest deadline 2024-09-21T03:58:02.835564+0000) 2024-09-21T03:58:08.718 INFO:journalctl@ceph.osd.8.smithi161.stdout:Sep 21 03:58:08 smithi161 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-8[52151]: 2024-09-21T03:58:08.343+0000 7f7e9a0f6640 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-09-21T03:57:29.951582+0000 front 2024-09-21T03:57:29.951481+0000 (oldest deadline 2024-09-21T03:57:54.051508+0000) 2024-09-21T03:58:08.718 INFO:journalctl@ceph.osd.8.smithi161.stdout:Sep 21 03:58:08 smithi161 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-8[52151]: 2024-09-21T03:58:08.343+0000 7f7e9a0f6640 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-09-21T03:57:39.753762+0000 front 2024-09-21T03:57:39.753690+0000 (oldest deadline 2024-09-21T03:58:04.453819+0000) 2024-09-21T03:58:08.975 INFO:journalctl@ceph.osd.10.smithi150.stdout:Sep 21 03:58:08 smithi150 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-10[58206]: 2024-09-21T03:58:08.513+0000 7f34020e1640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-09-21T03:57:23.120033+0000 front 2024-09-21T03:57:23.120109+0000 (oldest deadline 2024-09-21T03:57:45.920434+0000) 2024-09-21T03:58:08.975 INFO:journalctl@ceph.osd.10.smithi150.stdout:Sep 21 03:58:08 smithi150 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-10[58206]: 2024-09-21T03:58:08.513+0000 7f34020e1640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-09-21T03:57:31.221107+0000 front 2024-09-21T03:57:31.221162+0000 (oldest deadline 2024-09-21T03:57:56.521240+0000) 2024-09-21T03:58:08.975 INFO:journalctl@ceph.osd.10.smithi150.stdout:Sep 21 03:58:08 smithi150 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-10[58206]: 2024-09-21T03:58:08.513+0000 7f34020e1640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-09-21T03:57:40.022210+0000 front 2024-09-21T03:57:40.022157+0000 (oldest deadline 2024-09-21T03:58:01.722244+0000) 2024-09-21T03:58:09.084 INFO:journalctl@ceph.osd.6.smithi039.stdout:Sep 21 03:58:08 smithi039 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-6[60520]: 2024-09-21T03:58:08.796+0000 7f98a8e96640 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-09-21T03:57:22.466026+0000 front 2024-09-21T03:57:22.465966+0000 (oldest deadline 2024-09-21T03:57:46.566034+0000) 2024-09-21T03:58:09.084 INFO:journalctl@ceph.osd.6.smithi039.stdout:Sep 21 03:58:08 smithi039 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-6[60520]: 2024-09-21T03:58:08.796+0000 7f98a8e96640 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-09-21T03:57:30.067078+0000 front 2024-09-21T03:57:30.067189+0000 (oldest deadline 2024-09-21T03:57:53.566934+0000) 2024-09-21T03:58:09.084 INFO:journalctl@ceph.osd.6.smithi039.stdout:Sep 21 03:58:08 smithi039 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-6[60520]: 2024-09-21T03:58:08.796+0000 7f98a8e96640 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-09-21T03:57:38.267956+0000 front 2024-09-21T03:57:38.268018+0000 (oldest deadline 2024-09-21T03:58:02.967757+0000) 2024-09-21T03:58:09.084 INFO:journalctl@ceph.osd.9.smithi039.stdout:Sep 21 03:58:08 smithi039 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-9[67006]: 2024-09-21T03:58:08.849+0000 7f4a138c4640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-09-21T03:57:22.331574+0000 front 2024-09-21T03:57:22.332146+0000 (oldest deadline 2024-09-21T03:57:45.831720+0000) 2024-09-21T03:58:09.084 INFO:journalctl@ceph.osd.9.smithi039.stdout:Sep 21 03:58:08 smithi039 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-9[67006]: 2024-09-21T03:58:08.849+0000 7f4a138c4640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-09-21T03:57:29.932951+0000 front 2024-09-21T03:57:29.932930+0000 (oldest deadline 2024-09-21T03:57:54.632848+0000) 2024-09-21T03:58:09.084 INFO:journalctl@ceph.osd.9.smithi039.stdout:Sep 21 03:58:08 smithi039 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-9[67006]: 2024-09-21T03:58:08.849+0000 7f4a138c4640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-09-21T03:57:40.434486+0000 front 2024-09-21T03:57:40.434450+0000 (oldest deadline 2024-09-21T03:58:02.734524+0000) 2024-09-21T03:58:09.369 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 21 03:58:09 smithi161 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-11[58829]: 2024-09-21T03:58:09.056+0000 7f538ef4a640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-09-21T03:57:21.581303+0000 front 2024-09-21T03:57:21.581333+0000 (oldest deadline 2024-09-21T03:57:46.881367+0000) 2024-09-21T03:58:09.369 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 21 03:58:09 smithi161 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-11[58829]: 2024-09-21T03:58:09.056+0000 7f538ef4a640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-09-21T03:57:32.782524+0000 front 2024-09-21T03:57:32.782318+0000 (oldest deadline 2024-09-21T03:57:54.482443+0000) 2024-09-21T03:58:09.369 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 21 03:58:09 smithi161 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-11[58829]: 2024-09-21T03:58:09.056+0000 7f538ef4a640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-09-21T03:57:40.184584+0000 front 2024-09-21T03:57:40.184544+0000 (oldest deadline 2024-09-21T03:58:05.484668+0000) 2024-09-21T03:58:09.474 INFO:journalctl@ceph.osd.7.smithi150.stdout:Sep 21 03:58:09 smithi150 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-7[51981]: 2024-09-21T03:58:09.218+0000 7f2e0cdd2640 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-09-21T03:57:31.734202+0000 front 2024-09-21T03:57:31.734234+0000 (oldest deadline 2024-09-21T03:57:55.834353+0000) 2024-09-21T03:58:09.475 INFO:journalctl@ceph.osd.7.smithi150.stdout:Sep 21 03:58:09 smithi150 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-7[51981]: 2024-09-21T03:58:09.218+0000 7f2e0cdd2640 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-09-21T03:57:38.735442+0000 front 2024-09-21T03:57:38.735506+0000 (oldest deadline 2024-09-21T03:58:02.835564+0000) 2024-09-21T03:58:09.718 INFO:journalctl@ceph.osd.8.smithi161.stdout:Sep 21 03:58:09 smithi161 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-8[52151]: 2024-09-21T03:58:09.367+0000 7f7e9a0f6640 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-09-21T03:57:29.951582+0000 front 2024-09-21T03:57:29.951481+0000 (oldest deadline 2024-09-21T03:57:54.051508+0000) 2024-09-21T03:58:09.718 INFO:journalctl@ceph.osd.8.smithi161.stdout:Sep 21 03:58:09 smithi161 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-8[52151]: 2024-09-21T03:58:09.367+0000 7f7e9a0f6640 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-09-21T03:57:39.753762+0000 front 2024-09-21T03:57:39.753690+0000 (oldest deadline 2024-09-21T03:58:04.453819+0000) 2024-09-21T03:58:09.975 INFO:journalctl@ceph.osd.10.smithi150.stdout:Sep 21 03:58:09 smithi150 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-10[58206]: 2024-09-21T03:58:09.522+0000 7f34020e1640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-09-21T03:57:23.120033+0000 front 2024-09-21T03:57:23.120109+0000 (oldest deadline 2024-09-21T03:57:45.920434+0000) 2024-09-21T03:58:09.975 INFO:journalctl@ceph.osd.10.smithi150.stdout:Sep 21 03:58:09 smithi150 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-10[58206]: 2024-09-21T03:58:09.522+0000 7f34020e1640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-09-21T03:57:31.221107+0000 front 2024-09-21T03:57:31.221162+0000 (oldest deadline 2024-09-21T03:57:56.521240+0000) 2024-09-21T03:58:09.975 INFO:journalctl@ceph.osd.10.smithi150.stdout:Sep 21 03:58:09 smithi150 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-10[58206]: 2024-09-21T03:58:09.522+0000 7f34020e1640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-09-21T03:57:40.022210+0000 front 2024-09-21T03:57:40.022157+0000 (oldest deadline 2024-09-21T03:58:01.722244+0000) 2024-09-21T03:58:10.084 INFO:journalctl@ceph.osd.9.smithi039.stdout:Sep 21 03:58:09 smithi039 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-9[67006]: 2024-09-21T03:58:09.820+0000 7f4a138c4640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-09-21T03:57:22.331574+0000 front 2024-09-21T03:57:22.332146+0000 (oldest deadline 2024-09-21T03:57:45.831720+0000) 2024-09-21T03:58:10.084 INFO:journalctl@ceph.osd.9.smithi039.stdout:Sep 21 03:58:09 smithi039 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-9[67006]: 2024-09-21T03:58:09.820+0000 7f4a138c4640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-09-21T03:57:29.932951+0000 front 2024-09-21T03:57:29.932930+0000 (oldest deadline 2024-09-21T03:57:54.632848+0000) 2024-09-21T03:58:10.084 INFO:journalctl@ceph.osd.9.smithi039.stdout:Sep 21 03:58:09 smithi039 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-9[67006]: 2024-09-21T03:58:09.820+0000 7f4a138c4640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-09-21T03:57:40.434486+0000 front 2024-09-21T03:57:40.434450+0000 (oldest deadline 2024-09-21T03:58:02.734524+0000) 2024-09-21T03:58:10.084 INFO:journalctl@ceph.osd.6.smithi039.stdout:Sep 21 03:58:09 smithi039 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-6[60520]: 2024-09-21T03:58:09.809+0000 7f98a8e96640 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-09-21T03:57:22.466026+0000 front 2024-09-21T03:57:22.465966+0000 (oldest deadline 2024-09-21T03:57:46.566034+0000) 2024-09-21T03:58:10.084 INFO:journalctl@ceph.osd.6.smithi039.stdout:Sep 21 03:58:09 smithi039 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-6[60520]: 2024-09-21T03:58:09.809+0000 7f98a8e96640 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-09-21T03:57:30.067078+0000 front 2024-09-21T03:57:30.067189+0000 (oldest deadline 2024-09-21T03:57:53.566934+0000) 2024-09-21T03:58:10.084 INFO:journalctl@ceph.osd.6.smithi039.stdout:Sep 21 03:58:09 smithi039 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-6[60520]: 2024-09-21T03:58:09.809+0000 7f98a8e96640 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-09-21T03:57:38.267956+0000 front 2024-09-21T03:57:38.268018+0000 (oldest deadline 2024-09-21T03:58:02.967757+0000) 2024-09-21T03:58:10.085 INFO:journalctl@ceph.osd.6.smithi039.stdout:Sep 21 03:58:09 smithi039 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-6[60520]: 2024-09-21T03:58:09.809+0000 7f98a8e96640 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2024-09-21T03:57:48.268378+0000 front 2024-09-21T03:57:48.268416+0000 (oldest deadline 2024-09-21T03:58:09.368661+0000) 2024-09-21T03:58:10.338 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 21 03:58:10 smithi161 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-11[58829]: 2024-09-21T03:58:10.033+0000 7f538ef4a640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-09-21T03:57:21.581303+0000 front 2024-09-21T03:57:21.581333+0000 (oldest deadline 2024-09-21T03:57:46.881367+0000) 2024-09-21T03:58:10.338 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 21 03:58:10 smithi161 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-11[58829]: 2024-09-21T03:58:10.033+0000 7f538ef4a640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-09-21T03:57:32.782524+0000 front 2024-09-21T03:57:32.782318+0000 (oldest deadline 2024-09-21T03:57:54.482443+0000) 2024-09-21T03:58:10.338 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 21 03:58:10 smithi161 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-11[58829]: 2024-09-21T03:58:10.033+0000 7f538ef4a640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-09-21T03:57:40.184584+0000 front 2024-09-21T03:57:40.184544+0000 (oldest deadline 2024-09-21T03:58:05.484668+0000) 2024-09-21T03:58:10.718 INFO:journalctl@ceph.osd.8.smithi161.stdout:Sep 21 03:58:10 smithi161 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-8[52151]: 2024-09-21T03:58:10.336+0000 7f7e9a0f6640 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-09-21T03:57:29.951582+0000 front 2024-09-21T03:57:29.951481+0000 (oldest deadline 2024-09-21T03:57:54.051508+0000) 2024-09-21T03:58:10.718 INFO:journalctl@ceph.osd.8.smithi161.stdout:Sep 21 03:58:10 smithi161 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-8[52151]: 2024-09-21T03:58:10.336+0000 7f7e9a0f6640 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-09-21T03:57:39.753762+0000 front 2024-09-21T03:57:39.753690+0000 (oldest deadline 2024-09-21T03:58:04.453819+0000) 2024-09-21T03:58:10.725 INFO:journalctl@ceph.osd.7.smithi150.stdout:Sep 21 03:58:10 smithi150 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-7[51981]: 2024-09-21T03:58:10.252+0000 7f2e0cdd2640 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-09-21T03:57:31.734202+0000 front 2024-09-21T03:57:31.734234+0000 (oldest deadline 2024-09-21T03:57:55.834353+0000) 2024-09-21T03:58:10.725 INFO:journalctl@ceph.osd.7.smithi150.stdout:Sep 21 03:58:10 smithi150 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-7[51981]: 2024-09-21T03:58:10.252+0000 7f2e0cdd2640 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-09-21T03:57:38.735442+0000 front 2024-09-21T03:57:38.735506+0000 (oldest deadline 2024-09-21T03:58:02.835564+0000) 2024-09-21T03:58:10.725 INFO:journalctl@ceph.osd.7.smithi150.stdout:Sep 21 03:58:10 smithi150 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-7[51981]: 2024-09-21T03:58:10.252+0000 7f2e0cdd2640 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2024-09-21T03:57:48.037325+0000 front 2024-09-21T03:57:48.037200+0000 (oldest deadline 2024-09-21T03:58:09.637479+0000) 2024-09-21T03:58:10.725 INFO:journalctl@ceph.osd.10.smithi150.stdout:Sep 21 03:58:10 smithi150 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-10[58206]: 2024-09-21T03:58:10.496+0000 7f34020e1640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-09-21T03:57:23.120033+0000 front 2024-09-21T03:57:23.120109+0000 (oldest deadline 2024-09-21T03:57:45.920434+0000) 2024-09-21T03:58:10.725 INFO:journalctl@ceph.osd.10.smithi150.stdout:Sep 21 03:58:10 smithi150 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-10[58206]: 2024-09-21T03:58:10.496+0000 7f34020e1640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-09-21T03:57:31.221107+0000 front 2024-09-21T03:57:31.221162+0000 (oldest deadline 2024-09-21T03:57:56.521240+0000) 2024-09-21T03:58:10.725 INFO:journalctl@ceph.osd.10.smithi150.stdout:Sep 21 03:58:10 smithi150 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-10[58206]: 2024-09-21T03:58:10.496+0000 7f34020e1640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-09-21T03:57:40.022210+0000 front 2024-09-21T03:57:40.022157+0000 (oldest deadline 2024-09-21T03:58:01.722244+0000) 2024-09-21T03:58:10.725 INFO:journalctl@ceph.osd.10.smithi150.stdout:Sep 21 03:58:10 smithi150 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-10[58206]: 2024-09-21T03:58:10.496+0000 7f34020e1640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2024-09-21T03:57:46.323919+0000 front 2024-09-21T03:57:46.323891+0000 (oldest deadline 2024-09-21T03:58:10.423788+0000) 2024-09-21T03:58:11.334 INFO:journalctl@ceph.osd.6.smithi039.stdout:Sep 21 03:58:10 smithi039 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-6[60520]: 2024-09-21T03:58:10.855+0000 7f98a8e96640 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-09-21T03:57:22.466026+0000 front 2024-09-21T03:57:22.465966+0000 (oldest deadline 2024-09-21T03:57:46.566034+0000) 2024-09-21T03:58:11.334 INFO:journalctl@ceph.osd.6.smithi039.stdout:Sep 21 03:58:10 smithi039 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-6[60520]: 2024-09-21T03:58:10.855+0000 7f98a8e96640 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-09-21T03:57:30.067078+0000 front 2024-09-21T03:57:30.067189+0000 (oldest deadline 2024-09-21T03:57:53.566934+0000) 2024-09-21T03:58:11.334 INFO:journalctl@ceph.osd.6.smithi039.stdout:Sep 21 03:58:10 smithi039 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-6[60520]: 2024-09-21T03:58:10.855+0000 7f98a8e96640 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-09-21T03:57:38.267956+0000 front 2024-09-21T03:57:38.268018+0000 (oldest deadline 2024-09-21T03:58:02.967757+0000) 2024-09-21T03:58:11.334 INFO:journalctl@ceph.osd.6.smithi039.stdout:Sep 21 03:58:10 smithi039 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-6[60520]: 2024-09-21T03:58:10.855+0000 7f98a8e96640 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2024-09-21T03:57:48.268378+0000 front 2024-09-21T03:57:48.268416+0000 (oldest deadline 2024-09-21T03:58:09.368661+0000) 2024-09-21T03:58:11.334 INFO:journalctl@ceph.osd.9.smithi039.stdout:Sep 21 03:58:10 smithi039 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-9[67006]: 2024-09-21T03:58:10.864+0000 7f4a138c4640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-09-21T03:57:22.331574+0000 front 2024-09-21T03:57:22.332146+0000 (oldest deadline 2024-09-21T03:57:45.831720+0000) 2024-09-21T03:58:11.334 INFO:journalctl@ceph.osd.9.smithi039.stdout:Sep 21 03:58:10 smithi039 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-9[67006]: 2024-09-21T03:58:10.864+0000 7f4a138c4640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-09-21T03:57:29.932951+0000 front 2024-09-21T03:57:29.932930+0000 (oldest deadline 2024-09-21T03:57:54.632848+0000) 2024-09-21T03:58:11.334 INFO:journalctl@ceph.osd.9.smithi039.stdout:Sep 21 03:58:10 smithi039 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-9[67006]: 2024-09-21T03:58:10.864+0000 7f4a138c4640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-09-21T03:57:40.434486+0000 front 2024-09-21T03:57:40.434450+0000 (oldest deadline 2024-09-21T03:58:02.734524+0000) 2024-09-21T03:58:11.335 INFO:journalctl@ceph.osd.9.smithi039.stdout:Sep 21 03:58:10 smithi039 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-9[67006]: 2024-09-21T03:58:10.864+0000 7f4a138c4640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2024-09-21T03:57:44.336000+0000 front 2024-09-21T03:57:44.336099+0000 (oldest deadline 2024-09-21T03:58:10.136509+0000) 2024-09-21T03:58:11.375 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 21 03:58:11 smithi161 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-11[58829]: 2024-09-21T03:58:11.040+0000 7f538ef4a640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-09-21T03:57:21.581303+0000 front 2024-09-21T03:57:21.581333+0000 (oldest deadline 2024-09-21T03:57:46.881367+0000) 2024-09-21T03:58:11.375 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 21 03:58:11 smithi161 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-11[58829]: 2024-09-21T03:58:11.040+0000 7f538ef4a640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-09-21T03:57:32.782524+0000 front 2024-09-21T03:57:32.782318+0000 (oldest deadline 2024-09-21T03:57:54.482443+0000) 2024-09-21T03:58:11.375 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 21 03:58:11 smithi161 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-11[58829]: 2024-09-21T03:58:11.040+0000 7f538ef4a640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-09-21T03:57:40.184584+0000 front 2024-09-21T03:57:40.184544+0000 (oldest deadline 2024-09-21T03:58:05.484668+0000) 2024-09-21T03:58:11.475 INFO:journalctl@ceph.osd.7.smithi150.stdout:Sep 21 03:58:11 smithi150 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-7[51981]: 2024-09-21T03:58:11.219+0000 7f2e0cdd2640 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-09-21T03:57:31.734202+0000 front 2024-09-21T03:57:31.734234+0000 (oldest deadline 2024-09-21T03:57:55.834353+0000) 2024-09-21T03:58:11.475 INFO:journalctl@ceph.osd.7.smithi150.stdout:Sep 21 03:58:11 smithi150 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-7[51981]: 2024-09-21T03:58:11.219+0000 7f2e0cdd2640 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-09-21T03:57:38.735442+0000 front 2024-09-21T03:57:38.735506+0000 (oldest deadline 2024-09-21T03:58:02.835564+0000) 2024-09-21T03:58:11.475 INFO:journalctl@ceph.osd.7.smithi150.stdout:Sep 21 03:58:11 smithi150 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-7[51981]: 2024-09-21T03:58:11.219+0000 7f2e0cdd2640 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2024-09-21T03:57:48.037325+0000 front 2024-09-21T03:57:48.037200+0000 (oldest deadline 2024-09-21T03:58:09.637479+0000) 2024-09-21T03:58:11.718 INFO:journalctl@ceph.osd.8.smithi161.stdout:Sep 21 03:58:11 smithi161 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-8[52151]: 2024-09-21T03:58:11.373+0000 7f7e9a0f6640 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-09-21T03:57:29.951582+0000 front 2024-09-21T03:57:29.951481+0000 (oldest deadline 2024-09-21T03:57:54.051508+0000) 2024-09-21T03:58:11.718 INFO:journalctl@ceph.osd.8.smithi161.stdout:Sep 21 03:58:11 smithi161 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-8[52151]: 2024-09-21T03:58:11.373+0000 7f7e9a0f6640 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-09-21T03:57:39.753762+0000 front 2024-09-21T03:57:39.753690+0000 (oldest deadline 2024-09-21T03:58:04.453819+0000) 2024-09-21T03:58:11.718 INFO:journalctl@ceph.osd.8.smithi161.stdout:Sep 21 03:58:11 smithi161 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-8[52151]: 2024-09-21T03:58:11.373+0000 7f7e9a0f6640 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2024-09-21T03:57:48.455067+0000 front 2024-09-21T03:57:48.455286+0000 (oldest deadline 2024-09-21T03:58:10.755147+0000) 2024-09-21T03:58:11.975 INFO:journalctl@ceph.osd.10.smithi150.stdout:Sep 21 03:58:11 smithi150 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-10[58206]: 2024-09-21T03:58:11.542+0000 7f34020e1640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-09-21T03:57:23.120033+0000 front 2024-09-21T03:57:23.120109+0000 (oldest deadline 2024-09-21T03:57:45.920434+0000) 2024-09-21T03:58:11.975 INFO:journalctl@ceph.osd.10.smithi150.stdout:Sep 21 03:58:11 smithi150 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-10[58206]: 2024-09-21T03:58:11.542+0000 7f34020e1640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-09-21T03:57:31.221107+0000 front 2024-09-21T03:57:31.221162+0000 (oldest deadline 2024-09-21T03:57:56.521240+0000) 2024-09-21T03:58:11.975 INFO:journalctl@ceph.osd.10.smithi150.stdout:Sep 21 03:58:11 smithi150 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-10[58206]: 2024-09-21T03:58:11.542+0000 7f34020e1640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-09-21T03:57:40.022210+0000 front 2024-09-21T03:57:40.022157+0000 (oldest deadline 2024-09-21T03:58:01.722244+0000) 2024-09-21T03:58:11.975 INFO:journalctl@ceph.osd.10.smithi150.stdout:Sep 21 03:58:11 smithi150 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-10[58206]: 2024-09-21T03:58:11.542+0000 7f34020e1640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2024-09-21T03:57:46.323919+0000 front 2024-09-21T03:57:46.323891+0000 (oldest deadline 2024-09-21T03:58:10.423788+0000) 2024-09-21T03:58:12.084 INFO:journalctl@ceph.osd.6.smithi039.stdout:Sep 21 03:58:11 smithi039 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-6[60520]: 2024-09-21T03:58:11.821+0000 7f98a8e96640 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-09-21T03:57:22.466026+0000 front 2024-09-21T03:57:22.465966+0000 (oldest deadline 2024-09-21T03:57:46.566034+0000) 2024-09-21T03:58:12.084 INFO:journalctl@ceph.osd.6.smithi039.stdout:Sep 21 03:58:11 smithi039 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-6[60520]: 2024-09-21T03:58:11.821+0000 7f98a8e96640 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-09-21T03:57:30.067078+0000 front 2024-09-21T03:57:30.067189+0000 (oldest deadline 2024-09-21T03:57:53.566934+0000) 2024-09-21T03:58:12.084 INFO:journalctl@ceph.osd.6.smithi039.stdout:Sep 21 03:58:11 smithi039 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-6[60520]: 2024-09-21T03:58:11.821+0000 7f98a8e96640 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-09-21T03:57:38.267956+0000 front 2024-09-21T03:57:38.268018+0000 (oldest deadline 2024-09-21T03:58:02.967757+0000) 2024-09-21T03:58:12.084 INFO:journalctl@ceph.osd.6.smithi039.stdout:Sep 21 03:58:11 smithi039 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-6[60520]: 2024-09-21T03:58:11.821+0000 7f98a8e96640 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2024-09-21T03:57:48.268378+0000 front 2024-09-21T03:57:48.268416+0000 (oldest deadline 2024-09-21T03:58:09.368661+0000) 2024-09-21T03:58:12.084 INFO:journalctl@ceph.osd.9.smithi039.stdout:Sep 21 03:58:11 smithi039 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-9[67006]: 2024-09-21T03:58:11.902+0000 7f4a138c4640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-09-21T03:57:22.331574+0000 front 2024-09-21T03:57:22.332146+0000 (oldest deadline 2024-09-21T03:57:45.831720+0000) 2024-09-21T03:58:12.084 INFO:journalctl@ceph.osd.9.smithi039.stdout:Sep 21 03:58:11 smithi039 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-9[67006]: 2024-09-21T03:58:11.902+0000 7f4a138c4640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-09-21T03:57:29.932951+0000 front 2024-09-21T03:57:29.932930+0000 (oldest deadline 2024-09-21T03:57:54.632848+0000) 2024-09-21T03:58:12.084 INFO:journalctl@ceph.osd.9.smithi039.stdout:Sep 21 03:58:11 smithi039 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-9[67006]: 2024-09-21T03:58:11.902+0000 7f4a138c4640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-09-21T03:57:40.434486+0000 front 2024-09-21T03:57:40.434450+0000 (oldest deadline 2024-09-21T03:58:02.734524+0000) 2024-09-21T03:58:12.085 INFO:journalctl@ceph.osd.9.smithi039.stdout:Sep 21 03:58:11 smithi039 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-9[67006]: 2024-09-21T03:58:11.902+0000 7f4a138c4640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2024-09-21T03:57:44.336000+0000 front 2024-09-21T03:57:44.336099+0000 (oldest deadline 2024-09-21T03:58:10.136509+0000) 2024-09-21T03:58:12.390 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 21 03:58:12 smithi161 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-11[58829]: 2024-09-21T03:58:12.028+0000 7f538ef4a640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-09-21T03:57:21.581303+0000 front 2024-09-21T03:57:21.581333+0000 (oldest deadline 2024-09-21T03:57:46.881367+0000) 2024-09-21T03:58:12.390 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 21 03:58:12 smithi161 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-11[58829]: 2024-09-21T03:58:12.028+0000 7f538ef4a640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-09-21T03:57:32.782524+0000 front 2024-09-21T03:57:32.782318+0000 (oldest deadline 2024-09-21T03:57:54.482443+0000) 2024-09-21T03:58:12.390 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 21 03:58:12 smithi161 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-11[58829]: 2024-09-21T03:58:12.028+0000 7f538ef4a640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-09-21T03:57:40.184584+0000 front 2024-09-21T03:57:40.184544+0000 (oldest deadline 2024-09-21T03:58:05.484668+0000) 2024-09-21T03:58:12.556 INFO:journalctl@ceph.osd.7.smithi150.stdout:Sep 21 03:58:12 smithi150 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-7[51981]: 2024-09-21T03:58:12.234+0000 7f2e0cdd2640 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-09-21T03:57:31.734202+0000 front 2024-09-21T03:57:31.734234+0000 (oldest deadline 2024-09-21T03:57:55.834353+0000) 2024-09-21T03:58:12.556 INFO:journalctl@ceph.osd.7.smithi150.stdout:Sep 21 03:58:12 smithi150 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-7[51981]: 2024-09-21T03:58:12.234+0000 7f2e0cdd2640 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-09-21T03:57:38.735442+0000 front 2024-09-21T03:57:38.735506+0000 (oldest deadline 2024-09-21T03:58:02.835564+0000) 2024-09-21T03:58:12.556 INFO:journalctl@ceph.osd.7.smithi150.stdout:Sep 21 03:58:12 smithi150 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-7[51981]: 2024-09-21T03:58:12.234+0000 7f2e0cdd2640 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2024-09-21T03:57:48.037325+0000 front 2024-09-21T03:57:48.037200+0000 (oldest deadline 2024-09-21T03:58:09.637479+0000) 2024-09-21T03:58:12.647 INFO:journalctl@ceph.osd.8.smithi161.stdout:Sep 21 03:58:12 smithi161 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-8[52151]: 2024-09-21T03:58:12.394+0000 7f7e9a0f6640 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-09-21T03:57:29.951582+0000 front 2024-09-21T03:57:29.951481+0000 (oldest deadline 2024-09-21T03:57:54.051508+0000) 2024-09-21T03:58:12.647 INFO:journalctl@ceph.osd.8.smithi161.stdout:Sep 21 03:58:12 smithi161 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-8[52151]: 2024-09-21T03:58:12.394+0000 7f7e9a0f6640 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-09-21T03:57:39.753762+0000 front 2024-09-21T03:57:39.753690+0000 (oldest deadline 2024-09-21T03:58:04.453819+0000) 2024-09-21T03:58:12.647 INFO:journalctl@ceph.osd.8.smithi161.stdout:Sep 21 03:58:12 smithi161 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-8[52151]: 2024-09-21T03:58:12.394+0000 7f7e9a0f6640 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2024-09-21T03:57:48.455067+0000 front 2024-09-21T03:57:48.455286+0000 (oldest deadline 2024-09-21T03:58:10.755147+0000) 2024-09-21T03:58:12.975 INFO:journalctl@ceph.osd.10.smithi150.stdout:Sep 21 03:58:12 smithi150 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-10[58206]: 2024-09-21T03:58:12.553+0000 7f34020e1640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-09-21T03:57:23.120033+0000 front 2024-09-21T03:57:23.120109+0000 (oldest deadline 2024-09-21T03:57:45.920434+0000) 2024-09-21T03:58:12.975 INFO:journalctl@ceph.osd.10.smithi150.stdout:Sep 21 03:58:12 smithi150 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-10[58206]: 2024-09-21T03:58:12.553+0000 7f34020e1640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-09-21T03:57:31.221107+0000 front 2024-09-21T03:57:31.221162+0000 (oldest deadline 2024-09-21T03:57:56.521240+0000) 2024-09-21T03:58:12.975 INFO:journalctl@ceph.osd.10.smithi150.stdout:Sep 21 03:58:12 smithi150 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-10[58206]: 2024-09-21T03:58:12.553+0000 7f34020e1640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-09-21T03:57:40.022210+0000 front 2024-09-21T03:57:40.022157+0000 (oldest deadline 2024-09-21T03:58:01.722244+0000) 2024-09-21T03:58:12.975 INFO:journalctl@ceph.osd.10.smithi150.stdout:Sep 21 03:58:12 smithi150 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-10[58206]: 2024-09-21T03:58:12.553+0000 7f34020e1640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2024-09-21T03:57:46.323919+0000 front 2024-09-21T03:57:46.323891+0000 (oldest deadline 2024-09-21T03:58:10.423788+0000) 2024-09-21T03:58:13.139 INFO:journalctl@ceph.osd.9.smithi039.stdout:Sep 21 03:58:12 smithi039 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-9[67006]: 2024-09-21T03:58:12.857+0000 7f4a138c4640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-09-21T03:57:22.331574+0000 front 2024-09-21T03:57:22.332146+0000 (oldest deadline 2024-09-21T03:57:45.831720+0000) 2024-09-21T03:58:13.140 INFO:journalctl@ceph.osd.9.smithi039.stdout:Sep 21 03:58:12 smithi039 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-9[67006]: 2024-09-21T03:58:12.857+0000 7f4a138c4640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-09-21T03:57:29.932951+0000 front 2024-09-21T03:57:29.932930+0000 (oldest deadline 2024-09-21T03:57:54.632848+0000) 2024-09-21T03:58:13.140 INFO:journalctl@ceph.osd.9.smithi039.stdout:Sep 21 03:58:12 smithi039 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-9[67006]: 2024-09-21T03:58:12.857+0000 7f4a138c4640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-09-21T03:57:40.434486+0000 front 2024-09-21T03:57:40.434450+0000 (oldest deadline 2024-09-21T03:58:02.734524+0000) 2024-09-21T03:58:13.140 INFO:journalctl@ceph.osd.9.smithi039.stdout:Sep 21 03:58:12 smithi039 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-9[67006]: 2024-09-21T03:58:12.857+0000 7f4a138c4640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2024-09-21T03:57:44.336000+0000 front 2024-09-21T03:57:44.336099+0000 (oldest deadline 2024-09-21T03:58:10.136509+0000) 2024-09-21T03:58:13.348 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 21 03:58:13 smithi161 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-11[58829]: 2024-09-21T03:58:13.003+0000 7f538ef4a640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-09-21T03:57:21.581303+0000 front 2024-09-21T03:57:21.581333+0000 (oldest deadline 2024-09-21T03:57:46.881367+0000) 2024-09-21T03:58:13.348 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 21 03:58:13 smithi161 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-11[58829]: 2024-09-21T03:58:13.003+0000 7f538ef4a640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-09-21T03:57:32.782524+0000 front 2024-09-21T03:57:32.782318+0000 (oldest deadline 2024-09-21T03:57:54.482443+0000) 2024-09-21T03:58:13.349 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 21 03:58:13 smithi161 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-11[58829]: 2024-09-21T03:58:13.003+0000 7f538ef4a640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-09-21T03:57:40.184584+0000 front 2024-09-21T03:57:40.184544+0000 (oldest deadline 2024-09-21T03:58:05.484668+0000) 2024-09-21T03:58:13.431 INFO:journalctl@ceph.osd.6.smithi039.stdout:Sep 21 03:58:13 smithi039 podman[112688]: 2024-09-21 03:58:13.138131006 +0000 UTC m=+5.535276199 container died 1cac073838778a791f879d25ab9458b93b73e7347e2320e2489fbd2db9a253bb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-6, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.schema-version=1.0, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2) 2024-09-21T03:58:13.522 INFO:journalctl@ceph.osd.7.smithi150.stdout:Sep 21 03:58:13 smithi150 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-7[51981]: 2024-09-21T03:58:13.231+0000 7f2e0cdd2640 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-09-21T03:57:31.734202+0000 front 2024-09-21T03:57:31.734234+0000 (oldest deadline 2024-09-21T03:57:55.834353+0000) 2024-09-21T03:58:13.522 INFO:journalctl@ceph.osd.7.smithi150.stdout:Sep 21 03:58:13 smithi150 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-7[51981]: 2024-09-21T03:58:13.231+0000 7f2e0cdd2640 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-09-21T03:57:38.735442+0000 front 2024-09-21T03:57:38.735506+0000 (oldest deadline 2024-09-21T03:58:02.835564+0000) 2024-09-21T03:58:13.522 INFO:journalctl@ceph.osd.7.smithi150.stdout:Sep 21 03:58:13 smithi150 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-7[51981]: 2024-09-21T03:58:13.231+0000 7f2e0cdd2640 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2024-09-21T03:57:48.037325+0000 front 2024-09-21T03:57:48.037200+0000 (oldest deadline 2024-09-21T03:58:09.637479+0000) 2024-09-21T03:58:13.602 INFO:journalctl@ceph.osd.8.smithi161.stdout:Sep 21 03:58:13 smithi161 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-8[52151]: 2024-09-21T03:58:13.345+0000 7f7e9a0f6640 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-09-21T03:57:29.951582+0000 front 2024-09-21T03:57:29.951481+0000 (oldest deadline 2024-09-21T03:57:54.051508+0000) 2024-09-21T03:58:13.602 INFO:journalctl@ceph.osd.8.smithi161.stdout:Sep 21 03:58:13 smithi161 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-8[52151]: 2024-09-21T03:58:13.345+0000 7f7e9a0f6640 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-09-21T03:57:39.753762+0000 front 2024-09-21T03:57:39.753690+0000 (oldest deadline 2024-09-21T03:58:04.453819+0000) 2024-09-21T03:58:13.602 INFO:journalctl@ceph.osd.8.smithi161.stdout:Sep 21 03:58:13 smithi161 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-8[52151]: 2024-09-21T03:58:13.345+0000 7f7e9a0f6640 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2024-09-21T03:57:48.455067+0000 front 2024-09-21T03:57:48.455286+0000 (oldest deadline 2024-09-21T03:58:10.755147+0000) 2024-09-21T03:58:13.834 INFO:journalctl@ceph.osd.6.smithi039.stdout:Sep 21 03:58:13 smithi039 podman[112688]: 2024-09-21 03:58:13.611537319 +0000 UTC m=+6.008682504 container cleanup 1cac073838778a791f879d25ab9458b93b73e7347e2320e2489fbd2db9a253bb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-6, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.schema-version=1.0) 2024-09-21T03:58:13.835 INFO:journalctl@ceph.osd.6.smithi039.stdout:Sep 21 03:58:13 smithi039 bash[112688]: ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-6 2024-09-21T03:58:13.975 INFO:journalctl@ceph.osd.10.smithi150.stdout:Sep 21 03:58:13 smithi150 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-10[58206]: 2024-09-21T03:58:13.519+0000 7f34020e1640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-09-21T03:57:23.120033+0000 front 2024-09-21T03:57:23.120109+0000 (oldest deadline 2024-09-21T03:57:45.920434+0000) 2024-09-21T03:58:13.975 INFO:journalctl@ceph.osd.10.smithi150.stdout:Sep 21 03:58:13 smithi150 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-10[58206]: 2024-09-21T03:58:13.519+0000 7f34020e1640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-09-21T03:57:31.221107+0000 front 2024-09-21T03:57:31.221162+0000 (oldest deadline 2024-09-21T03:57:56.521240+0000) 2024-09-21T03:58:13.975 INFO:journalctl@ceph.osd.10.smithi150.stdout:Sep 21 03:58:13 smithi150 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-10[58206]: 2024-09-21T03:58:13.519+0000 7f34020e1640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-09-21T03:57:40.022210+0000 front 2024-09-21T03:57:40.022157+0000 (oldest deadline 2024-09-21T03:58:01.722244+0000) 2024-09-21T03:58:13.975 INFO:journalctl@ceph.osd.10.smithi150.stdout:Sep 21 03:58:13 smithi150 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-10[58206]: 2024-09-21T03:58:13.519+0000 7f34020e1640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2024-09-21T03:57:46.323919+0000 front 2024-09-21T03:57:46.323891+0000 (oldest deadline 2024-09-21T03:58:10.423788+0000) 2024-09-21T03:58:14.240 INFO:journalctl@ceph.osd.6.smithi039.stdout:Sep 21 03:58:13 smithi039 podman[112705]: 2024-09-21 03:58:13.871618501 +0000 UTC m=+0.728368099 container remove 1cac073838778a791f879d25ab9458b93b73e7347e2320e2489fbd2db9a253bb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-6, ceph=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-21T03:58:14.240 INFO:journalctl@ceph.osd.9.smithi039.stdout:Sep 21 03:58:13 smithi039 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-9[67006]: 2024-09-21T03:58:13.882+0000 7f4a138c4640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-09-21T03:57:22.331574+0000 front 2024-09-21T03:57:22.332146+0000 (oldest deadline 2024-09-21T03:57:45.831720+0000) 2024-09-21T03:58:14.240 INFO:journalctl@ceph.osd.9.smithi039.stdout:Sep 21 03:58:13 smithi039 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-9[67006]: 2024-09-21T03:58:13.882+0000 7f4a138c4640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-09-21T03:57:29.932951+0000 front 2024-09-21T03:57:29.932930+0000 (oldest deadline 2024-09-21T03:57:54.632848+0000) 2024-09-21T03:58:14.241 INFO:journalctl@ceph.osd.9.smithi039.stdout:Sep 21 03:58:13 smithi039 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-9[67006]: 2024-09-21T03:58:13.882+0000 7f4a138c4640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-09-21T03:57:40.434486+0000 front 2024-09-21T03:57:40.434450+0000 (oldest deadline 2024-09-21T03:58:02.734524+0000) 2024-09-21T03:58:14.241 INFO:journalctl@ceph.osd.9.smithi039.stdout:Sep 21 03:58:13 smithi039 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-9[67006]: 2024-09-21T03:58:13.882+0000 7f4a138c4640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2024-09-21T03:57:44.336000+0000 front 2024-09-21T03:57:44.336099+0000 (oldest deadline 2024-09-21T03:58:10.136509+0000) 2024-09-21T03:58:14.309 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 21 03:58:13 smithi161 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-11[58829]: 2024-09-21T03:58:13.985+0000 7f538ef4a640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-09-21T03:57:21.581303+0000 front 2024-09-21T03:57:21.581333+0000 (oldest deadline 2024-09-21T03:57:46.881367+0000) 2024-09-21T03:58:14.309 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 21 03:58:13 smithi161 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-11[58829]: 2024-09-21T03:58:13.985+0000 7f538ef4a640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-09-21T03:57:32.782524+0000 front 2024-09-21T03:57:32.782318+0000 (oldest deadline 2024-09-21T03:57:54.482443+0000) 2024-09-21T03:58:14.309 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 21 03:58:13 smithi161 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-11[58829]: 2024-09-21T03:58:13.985+0000 7f538ef4a640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-09-21T03:57:40.184584+0000 front 2024-09-21T03:57:40.184544+0000 (oldest deadline 2024-09-21T03:58:05.484668+0000) 2024-09-21T03:58:14.543 INFO:journalctl@ceph.osd.7.smithi150.stdout:Sep 21 03:58:14 smithi150 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-7[51981]: 2024-09-21T03:58:14.228+0000 7f2e0cdd2640 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-09-21T03:57:31.734202+0000 front 2024-09-21T03:57:31.734234+0000 (oldest deadline 2024-09-21T03:57:55.834353+0000) 2024-09-21T03:58:14.543 INFO:journalctl@ceph.osd.7.smithi150.stdout:Sep 21 03:58:14 smithi150 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-7[51981]: 2024-09-21T03:58:14.228+0000 7f2e0cdd2640 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-09-21T03:57:38.735442+0000 front 2024-09-21T03:57:38.735506+0000 (oldest deadline 2024-09-21T03:58:02.835564+0000) 2024-09-21T03:58:14.543 INFO:journalctl@ceph.osd.7.smithi150.stdout:Sep 21 03:58:14 smithi150 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-7[51981]: 2024-09-21T03:58:14.228+0000 7f2e0cdd2640 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2024-09-21T03:57:48.037325+0000 front 2024-09-21T03:57:48.037200+0000 (oldest deadline 2024-09-21T03:58:09.637479+0000) 2024-09-21T03:58:14.543 INFO:journalctl@ceph.osd.6.smithi039.stdout:Sep 21 03:58:14 smithi039 podman[112795]: 2024-09-21 03:58:14.141178475 +0000 UTC m=+0.016416432 image pull ddce307e1f3263fb077b4f3902e15660050631dad4d2823a20ee3f72bf18d977 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262 2024-09-21T03:58:14.543 INFO:journalctl@ceph.osd.6.smithi039.stdout:Sep 21 03:58:14 smithi039 podman[112795]: 2024-09-21 03:58:14.247308497 +0000 UTC m=+0.122546441 container create cd49e59c08c2b201840960e47b55f59057871b4e507a6b605156349d9c9de3ae (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-6-deactivate, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, ceph=True, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0) 2024-09-21T03:58:14.543 INFO:journalctl@ceph.osd.6.smithi039.stdout:Sep 21 03:58:14 smithi039 podman[112795]: 2024-09-21 03:58:14.466355171 +0000 UTC m=+0.341593135 container init cd49e59c08c2b201840960e47b55f59057871b4e507a6b605156349d9c9de3ae (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-6-deactivate, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_CLEAN=True) 2024-09-21T03:58:14.543 INFO:journalctl@ceph.osd.6.smithi039.stdout:Sep 21 03:58:14 smithi039 podman[112795]: 2024-09-21 03:58:14.469803247 +0000 UTC m=+0.345041192 container start cd49e59c08c2b201840960e47b55f59057871b4e507a6b605156349d9c9de3ae (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-6-deactivate, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, ceph=True, maintainer=Guillaume Abrioux ) 2024-09-21T03:58:14.543 INFO:journalctl@ceph.osd.6.smithi039.stdout:Sep 21 03:58:14 smithi039 podman[112795]: 2024-09-21 03:58:14.541341653 +0000 UTC m=+0.416579598 container attach cd49e59c08c2b201840960e47b55f59057871b4e507a6b605156349d9c9de3ae (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-6-deactivate, GIT_CLEAN=True, maintainer=Guillaume Abrioux , RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-21T03:58:14.718 INFO:journalctl@ceph.osd.8.smithi161.stdout:Sep 21 03:58:14 smithi161 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-8[52151]: 2024-09-21T03:58:14.306+0000 7f7e9a0f6640 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-09-21T03:57:29.951582+0000 front 2024-09-21T03:57:29.951481+0000 (oldest deadline 2024-09-21T03:57:54.051508+0000) 2024-09-21T03:58:14.718 INFO:journalctl@ceph.osd.8.smithi161.stdout:Sep 21 03:58:14 smithi161 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-8[52151]: 2024-09-21T03:58:14.306+0000 7f7e9a0f6640 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-09-21T03:57:39.753762+0000 front 2024-09-21T03:57:39.753690+0000 (oldest deadline 2024-09-21T03:58:04.453819+0000) 2024-09-21T03:58:14.718 INFO:journalctl@ceph.osd.8.smithi161.stdout:Sep 21 03:58:14 smithi161 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-8[52151]: 2024-09-21T03:58:14.306+0000 7f7e9a0f6640 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2024-09-21T03:57:48.455067+0000 front 2024-09-21T03:57:48.455286+0000 (oldest deadline 2024-09-21T03:58:10.755147+0000) 2024-09-21T03:58:14.834 INFO:journalctl@ceph.osd.6.smithi039.stdout:Sep 21 03:58:14 smithi039 podman[112795]: 2024-09-21 03:58:14.667066195 +0000 UTC m=+0.542304140 container died cd49e59c08c2b201840960e47b55f59057871b4e507a6b605156349d9c9de3ae (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-6-deactivate, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_CLEAN=True, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.build-date=20240731) 2024-09-21T03:58:14.975 INFO:journalctl@ceph.osd.10.smithi150.stdout:Sep 21 03:58:14 smithi150 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-10[58206]: 2024-09-21T03:58:14.540+0000 7f34020e1640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-09-21T03:57:23.120033+0000 front 2024-09-21T03:57:23.120109+0000 (oldest deadline 2024-09-21T03:57:45.920434+0000) 2024-09-21T03:58:14.975 INFO:journalctl@ceph.osd.10.smithi150.stdout:Sep 21 03:58:14 smithi150 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-10[58206]: 2024-09-21T03:58:14.540+0000 7f34020e1640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-09-21T03:57:31.221107+0000 front 2024-09-21T03:57:31.221162+0000 (oldest deadline 2024-09-21T03:57:56.521240+0000) 2024-09-21T03:58:14.975 INFO:journalctl@ceph.osd.10.smithi150.stdout:Sep 21 03:58:14 smithi150 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-10[58206]: 2024-09-21T03:58:14.540+0000 7f34020e1640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-09-21T03:57:40.022210+0000 front 2024-09-21T03:57:40.022157+0000 (oldest deadline 2024-09-21T03:58:01.722244+0000) 2024-09-21T03:58:14.975 INFO:journalctl@ceph.osd.10.smithi150.stdout:Sep 21 03:58:14 smithi150 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-10[58206]: 2024-09-21T03:58:14.540+0000 7f34020e1640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2024-09-21T03:57:46.323919+0000 front 2024-09-21T03:57:46.323891+0000 (oldest deadline 2024-09-21T03:58:10.423788+0000) 2024-09-21T03:58:15.218 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 21 03:58:14 smithi161 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-11[58829]: 2024-09-21T03:58:14.942+0000 7f538ef4a640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-09-21T03:57:21.581303+0000 front 2024-09-21T03:57:21.581333+0000 (oldest deadline 2024-09-21T03:57:46.881367+0000) 2024-09-21T03:58:15.218 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 21 03:58:14 smithi161 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-11[58829]: 2024-09-21T03:58:14.942+0000 7f538ef4a640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-09-21T03:57:32.782524+0000 front 2024-09-21T03:57:32.782318+0000 (oldest deadline 2024-09-21T03:57:54.482443+0000) 2024-09-21T03:58:15.218 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 21 03:58:14 smithi161 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-11[58829]: 2024-09-21T03:58:14.942+0000 7f538ef4a640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-09-21T03:57:40.184584+0000 front 2024-09-21T03:57:40.184544+0000 (oldest deadline 2024-09-21T03:58:05.484668+0000) 2024-09-21T03:58:15.334 INFO:journalctl@ceph.osd.9.smithi039.stdout:Sep 21 03:58:14 smithi039 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-9[67006]: 2024-09-21T03:58:14.834+0000 7f4a138c4640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-09-21T03:57:22.331574+0000 front 2024-09-21T03:57:22.332146+0000 (oldest deadline 2024-09-21T03:57:45.831720+0000) 2024-09-21T03:58:15.334 INFO:journalctl@ceph.osd.9.smithi039.stdout:Sep 21 03:58:14 smithi039 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-9[67006]: 2024-09-21T03:58:14.834+0000 7f4a138c4640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-09-21T03:57:29.932951+0000 front 2024-09-21T03:57:29.932930+0000 (oldest deadline 2024-09-21T03:57:54.632848+0000) 2024-09-21T03:58:15.334 INFO:journalctl@ceph.osd.9.smithi039.stdout:Sep 21 03:58:14 smithi039 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-9[67006]: 2024-09-21T03:58:14.834+0000 7f4a138c4640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-09-21T03:57:40.434486+0000 front 2024-09-21T03:57:40.434450+0000 (oldest deadline 2024-09-21T03:58:02.734524+0000) 2024-09-21T03:58:15.334 INFO:journalctl@ceph.osd.9.smithi039.stdout:Sep 21 03:58:14 smithi039 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-9[67006]: 2024-09-21T03:58:14.834+0000 7f4a138c4640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2024-09-21T03:57:44.336000+0000 front 2024-09-21T03:57:44.336099+0000 (oldest deadline 2024-09-21T03:58:10.136509+0000) 2024-09-21T03:58:15.409 DEBUG:teuthology.orchestra.run.smithi039:> sudo pkill -f 'journalctl -f -n 0 -u ceph-51116d5c-77ca-11ef-baf5-efdc52797490@osd.6.service' 2024-09-21T03:58:15.450 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-21T03:58:15.451 INFO:tasks.cephadm.osd.6:Stopped osd.6 2024-09-21T03:58:15.451 INFO:tasks.cephadm.osd.7:Stopping osd.7... 2024-09-21T03:58:15.451 DEBUG:teuthology.orchestra.run.smithi150:> sudo systemctl stop ceph-51116d5c-77ca-11ef-baf5-efdc52797490@osd.7 2024-09-21T03:58:15.475 INFO:journalctl@ceph.osd.7.smithi150.stdout:Sep 21 03:58:15 smithi150 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-7[51981]: 2024-09-21T03:58:15.188+0000 7f2e0cdd2640 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-09-21T03:57:31.734202+0000 front 2024-09-21T03:57:31.734234+0000 (oldest deadline 2024-09-21T03:57:55.834353+0000) 2024-09-21T03:58:15.475 INFO:journalctl@ceph.osd.7.smithi150.stdout:Sep 21 03:58:15 smithi150 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-7[51981]: 2024-09-21T03:58:15.188+0000 7f2e0cdd2640 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-09-21T03:57:38.735442+0000 front 2024-09-21T03:57:38.735506+0000 (oldest deadline 2024-09-21T03:58:02.835564+0000) 2024-09-21T03:58:15.475 INFO:journalctl@ceph.osd.7.smithi150.stdout:Sep 21 03:58:15 smithi150 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-7[51981]: 2024-09-21T03:58:15.188+0000 7f2e0cdd2640 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2024-09-21T03:57:48.037325+0000 front 2024-09-21T03:57:48.037200+0000 (oldest deadline 2024-09-21T03:58:09.637479+0000) 2024-09-21T03:58:15.718 INFO:journalctl@ceph.osd.8.smithi161.stdout:Sep 21 03:58:15 smithi161 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-8[52151]: 2024-09-21T03:58:15.299+0000 7f7e9a0f6640 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-09-21T03:57:29.951582+0000 front 2024-09-21T03:57:29.951481+0000 (oldest deadline 2024-09-21T03:57:54.051508+0000) 2024-09-21T03:58:15.718 INFO:journalctl@ceph.osd.8.smithi161.stdout:Sep 21 03:58:15 smithi161 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-8[52151]: 2024-09-21T03:58:15.299+0000 7f7e9a0f6640 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-09-21T03:57:39.753762+0000 front 2024-09-21T03:57:39.753690+0000 (oldest deadline 2024-09-21T03:58:04.453819+0000) 2024-09-21T03:58:15.718 INFO:journalctl@ceph.osd.8.smithi161.stdout:Sep 21 03:58:15 smithi161 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-8[52151]: 2024-09-21T03:58:15.299+0000 7f7e9a0f6640 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2024-09-21T03:57:48.455067+0000 front 2024-09-21T03:57:48.455286+0000 (oldest deadline 2024-09-21T03:58:10.755147+0000) 2024-09-21T03:58:15.762 INFO:journalctl@ceph.osd.7.smithi150.stdout:Sep 21 03:58:15 smithi150 systemd[1]: Stopping Ceph osd.7 for 51116d5c-77ca-11ef-baf5-efdc52797490... 2024-09-21T03:58:15.762 INFO:journalctl@ceph.osd.10.smithi150.stdout:Sep 21 03:58:15 smithi150 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-10[58206]: 2024-09-21T03:58:15.545+0000 7f34020e1640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-09-21T03:57:23.120033+0000 front 2024-09-21T03:57:23.120109+0000 (oldest deadline 2024-09-21T03:57:45.920434+0000) 2024-09-21T03:58:15.762 INFO:journalctl@ceph.osd.10.smithi150.stdout:Sep 21 03:58:15 smithi150 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-10[58206]: 2024-09-21T03:58:15.545+0000 7f34020e1640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-09-21T03:57:31.221107+0000 front 2024-09-21T03:57:31.221162+0000 (oldest deadline 2024-09-21T03:57:56.521240+0000) 2024-09-21T03:58:15.762 INFO:journalctl@ceph.osd.10.smithi150.stdout:Sep 21 03:58:15 smithi150 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-10[58206]: 2024-09-21T03:58:15.545+0000 7f34020e1640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-09-21T03:57:40.022210+0000 front 2024-09-21T03:57:40.022157+0000 (oldest deadline 2024-09-21T03:58:01.722244+0000) 2024-09-21T03:58:15.762 INFO:journalctl@ceph.osd.10.smithi150.stdout:Sep 21 03:58:15 smithi150 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-10[58206]: 2024-09-21T03:58:15.545+0000 7f34020e1640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2024-09-21T03:57:46.323919+0000 front 2024-09-21T03:57:46.323891+0000 (oldest deadline 2024-09-21T03:58:10.423788+0000) 2024-09-21T03:58:16.084 INFO:journalctl@ceph.osd.9.smithi039.stdout:Sep 21 03:58:15 smithi039 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-9[67006]: 2024-09-21T03:58:15.814+0000 7f4a138c4640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-09-21T03:57:22.331574+0000 front 2024-09-21T03:57:22.332146+0000 (oldest deadline 2024-09-21T03:57:45.831720+0000) 2024-09-21T03:58:16.084 INFO:journalctl@ceph.osd.9.smithi039.stdout:Sep 21 03:58:15 smithi039 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-9[67006]: 2024-09-21T03:58:15.814+0000 7f4a138c4640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-09-21T03:57:29.932951+0000 front 2024-09-21T03:57:29.932930+0000 (oldest deadline 2024-09-21T03:57:54.632848+0000) 2024-09-21T03:58:16.084 INFO:journalctl@ceph.osd.9.smithi039.stdout:Sep 21 03:58:15 smithi039 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-9[67006]: 2024-09-21T03:58:15.814+0000 7f4a138c4640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-09-21T03:57:40.434486+0000 front 2024-09-21T03:57:40.434450+0000 (oldest deadline 2024-09-21T03:58:02.734524+0000) 2024-09-21T03:58:16.084 INFO:journalctl@ceph.osd.9.smithi039.stdout:Sep 21 03:58:15 smithi039 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-9[67006]: 2024-09-21T03:58:15.814+0000 7f4a138c4640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2024-09-21T03:57:44.336000+0000 front 2024-09-21T03:57:44.336099+0000 (oldest deadline 2024-09-21T03:58:10.136509+0000) 2024-09-21T03:58:16.206 INFO:journalctl@ceph.osd.7.smithi150.stdout:Sep 21 03:58:15 smithi150 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-7[51981]: 2024-09-21T03:58:15.758+0000 7f2e147ef640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.7 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-21T03:58:16.206 INFO:journalctl@ceph.osd.7.smithi150.stdout:Sep 21 03:58:15 smithi150 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-7[51981]: 2024-09-21T03:58:15.758+0000 7f2e147ef640 -1 osd.7 90 *** Got signal Terminated *** 2024-09-21T03:58:16.206 INFO:journalctl@ceph.osd.7.smithi150.stdout:Sep 21 03:58:15 smithi150 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-7[51981]: 2024-09-21T03:58:15.759+0000 7f2e147ef640 -1 osd.7 90 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-21T03:58:16.206 INFO:journalctl@ceph.osd.7.smithi150.stdout:Sep 21 03:58:16 smithi150 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-7[51981]: 2024-09-21T03:58:16.203+0000 7f2e0cdd2640 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-09-21T03:57:31.734202+0000 front 2024-09-21T03:57:31.734234+0000 (oldest deadline 2024-09-21T03:57:55.834353+0000) 2024-09-21T03:58:16.206 INFO:journalctl@ceph.osd.7.smithi150.stdout:Sep 21 03:58:16 smithi150 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-7[51981]: 2024-09-21T03:58:16.203+0000 7f2e0cdd2640 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-09-21T03:57:38.735442+0000 front 2024-09-21T03:57:38.735506+0000 (oldest deadline 2024-09-21T03:58:02.835564+0000) 2024-09-21T03:58:16.206 INFO:journalctl@ceph.osd.7.smithi150.stdout:Sep 21 03:58:16 smithi150 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-7[51981]: 2024-09-21T03:58:16.203+0000 7f2e0cdd2640 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2024-09-21T03:57:48.037325+0000 front 2024-09-21T03:57:48.037200+0000 (oldest deadline 2024-09-21T03:58:09.637479+0000) 2024-09-21T03:58:16.218 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 21 03:58:15 smithi161 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-11[58829]: 2024-09-21T03:58:15.895+0000 7f538ef4a640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-09-21T03:57:21.581303+0000 front 2024-09-21T03:57:21.581333+0000 (oldest deadline 2024-09-21T03:57:46.881367+0000) 2024-09-21T03:58:16.218 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 21 03:58:15 smithi161 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-11[58829]: 2024-09-21T03:58:15.895+0000 7f538ef4a640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-09-21T03:57:32.782524+0000 front 2024-09-21T03:57:32.782318+0000 (oldest deadline 2024-09-21T03:57:54.482443+0000) 2024-09-21T03:58:16.218 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 21 03:58:15 smithi161 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-11[58829]: 2024-09-21T03:58:15.895+0000 7f538ef4a640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-09-21T03:57:40.184584+0000 front 2024-09-21T03:57:40.184544+0000 (oldest deadline 2024-09-21T03:58:05.484668+0000) 2024-09-21T03:58:16.718 INFO:journalctl@ceph.osd.8.smithi161.stdout:Sep 21 03:58:16 smithi161 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-8[52151]: 2024-09-21T03:58:16.270+0000 7f7e9a0f6640 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-09-21T03:57:29.951582+0000 front 2024-09-21T03:57:29.951481+0000 (oldest deadline 2024-09-21T03:57:54.051508+0000) 2024-09-21T03:58:16.718 INFO:journalctl@ceph.osd.8.smithi161.stdout:Sep 21 03:58:16 smithi161 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-8[52151]: 2024-09-21T03:58:16.270+0000 7f7e9a0f6640 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-09-21T03:57:39.753762+0000 front 2024-09-21T03:57:39.753690+0000 (oldest deadline 2024-09-21T03:58:04.453819+0000) 2024-09-21T03:58:16.718 INFO:journalctl@ceph.osd.8.smithi161.stdout:Sep 21 03:58:16 smithi161 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-8[52151]: 2024-09-21T03:58:16.270+0000 7f7e9a0f6640 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2024-09-21T03:57:48.455067+0000 front 2024-09-21T03:57:48.455286+0000 (oldest deadline 2024-09-21T03:58:10.755147+0000) 2024-09-21T03:58:16.738 INFO:journalctl@ceph.osd.10.smithi150.stdout:Sep 21 03:58:16 smithi150 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-10[58206]: 2024-09-21T03:58:16.496+0000 7f34020e1640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-09-21T03:57:23.120033+0000 front 2024-09-21T03:57:23.120109+0000 (oldest deadline 2024-09-21T03:57:45.920434+0000) 2024-09-21T03:58:16.738 INFO:journalctl@ceph.osd.10.smithi150.stdout:Sep 21 03:58:16 smithi150 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-10[58206]: 2024-09-21T03:58:16.496+0000 7f34020e1640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-09-21T03:57:31.221107+0000 front 2024-09-21T03:57:31.221162+0000 (oldest deadline 2024-09-21T03:57:56.521240+0000) 2024-09-21T03:58:16.738 INFO:journalctl@ceph.osd.10.smithi150.stdout:Sep 21 03:58:16 smithi150 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-10[58206]: 2024-09-21T03:58:16.496+0000 7f34020e1640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-09-21T03:57:40.022210+0000 front 2024-09-21T03:57:40.022157+0000 (oldest deadline 2024-09-21T03:58:01.722244+0000) 2024-09-21T03:58:16.738 INFO:journalctl@ceph.osd.10.smithi150.stdout:Sep 21 03:58:16 smithi150 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-10[58206]: 2024-09-21T03:58:16.496+0000 7f34020e1640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2024-09-21T03:57:46.323919+0000 front 2024-09-21T03:57:46.323891+0000 (oldest deadline 2024-09-21T03:58:10.423788+0000) 2024-09-21T03:58:17.084 INFO:journalctl@ceph.osd.9.smithi039.stdout:Sep 21 03:58:16 smithi039 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-9[67006]: 2024-09-21T03:58:16.849+0000 7f4a138c4640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-09-21T03:57:22.331574+0000 front 2024-09-21T03:57:22.332146+0000 (oldest deadline 2024-09-21T03:57:45.831720+0000) 2024-09-21T03:58:17.084 INFO:journalctl@ceph.osd.9.smithi039.stdout:Sep 21 03:58:16 smithi039 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-9[67006]: 2024-09-21T03:58:16.849+0000 7f4a138c4640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-09-21T03:57:29.932951+0000 front 2024-09-21T03:57:29.932930+0000 (oldest deadline 2024-09-21T03:57:54.632848+0000) 2024-09-21T03:58:17.084 INFO:journalctl@ceph.osd.9.smithi039.stdout:Sep 21 03:58:16 smithi039 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-9[67006]: 2024-09-21T03:58:16.849+0000 7f4a138c4640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-09-21T03:57:40.434486+0000 front 2024-09-21T03:57:40.434450+0000 (oldest deadline 2024-09-21T03:58:02.734524+0000) 2024-09-21T03:58:17.084 INFO:journalctl@ceph.osd.9.smithi039.stdout:Sep 21 03:58:16 smithi039 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-9[67006]: 2024-09-21T03:58:16.849+0000 7f4a138c4640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2024-09-21T03:57:44.336000+0000 front 2024-09-21T03:57:44.336099+0000 (oldest deadline 2024-09-21T03:58:10.136509+0000) 2024-09-21T03:58:17.218 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 21 03:58:16 smithi161 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-11[58829]: 2024-09-21T03:58:16.938+0000 7f538ef4a640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-09-21T03:57:21.581303+0000 front 2024-09-21T03:57:21.581333+0000 (oldest deadline 2024-09-21T03:57:46.881367+0000) 2024-09-21T03:58:17.218 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 21 03:58:16 smithi161 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-11[58829]: 2024-09-21T03:58:16.938+0000 7f538ef4a640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-09-21T03:57:32.782524+0000 front 2024-09-21T03:57:32.782318+0000 (oldest deadline 2024-09-21T03:57:54.482443+0000) 2024-09-21T03:58:17.218 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 21 03:58:16 smithi161 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-11[58829]: 2024-09-21T03:58:16.938+0000 7f538ef4a640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-09-21T03:57:40.184584+0000 front 2024-09-21T03:57:40.184544+0000 (oldest deadline 2024-09-21T03:58:05.484668+0000) 2024-09-21T03:58:17.475 INFO:journalctl@ceph.osd.7.smithi150.stdout:Sep 21 03:58:17 smithi150 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-7[51981]: 2024-09-21T03:58:17.210+0000 7f2e0cdd2640 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-09-21T03:57:31.734202+0000 front 2024-09-21T03:57:31.734234+0000 (oldest deadline 2024-09-21T03:57:55.834353+0000) 2024-09-21T03:58:17.475 INFO:journalctl@ceph.osd.7.smithi150.stdout:Sep 21 03:58:17 smithi150 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-7[51981]: 2024-09-21T03:58:17.210+0000 7f2e0cdd2640 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-09-21T03:57:38.735442+0000 front 2024-09-21T03:57:38.735506+0000 (oldest deadline 2024-09-21T03:58:02.835564+0000) 2024-09-21T03:58:17.475 INFO:journalctl@ceph.osd.7.smithi150.stdout:Sep 21 03:58:17 smithi150 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-7[51981]: 2024-09-21T03:58:17.210+0000 7f2e0cdd2640 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2024-09-21T03:57:48.037325+0000 front 2024-09-21T03:57:48.037200+0000 (oldest deadline 2024-09-21T03:58:09.637479+0000) 2024-09-21T03:58:17.718 INFO:journalctl@ceph.osd.8.smithi161.stdout:Sep 21 03:58:17 smithi161 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-8[52151]: 2024-09-21T03:58:17.299+0000 7f7e9a0f6640 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-09-21T03:57:29.951582+0000 front 2024-09-21T03:57:29.951481+0000 (oldest deadline 2024-09-21T03:57:54.051508+0000) 2024-09-21T03:58:17.718 INFO:journalctl@ceph.osd.8.smithi161.stdout:Sep 21 03:58:17 smithi161 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-8[52151]: 2024-09-21T03:58:17.299+0000 7f7e9a0f6640 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-09-21T03:57:39.753762+0000 front 2024-09-21T03:57:39.753690+0000 (oldest deadline 2024-09-21T03:58:04.453819+0000) 2024-09-21T03:58:17.718 INFO:journalctl@ceph.osd.8.smithi161.stdout:Sep 21 03:58:17 smithi161 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-8[52151]: 2024-09-21T03:58:17.299+0000 7f7e9a0f6640 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2024-09-21T03:57:48.455067+0000 front 2024-09-21T03:57:48.455286+0000 (oldest deadline 2024-09-21T03:58:10.755147+0000) 2024-09-21T03:58:17.975 INFO:journalctl@ceph.osd.10.smithi150.stdout:Sep 21 03:58:17 smithi150 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-10[58206]: 2024-09-21T03:58:17.516+0000 7f34020e1640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-09-21T03:57:23.120033+0000 front 2024-09-21T03:57:23.120109+0000 (oldest deadline 2024-09-21T03:57:45.920434+0000) 2024-09-21T03:58:17.975 INFO:journalctl@ceph.osd.10.smithi150.stdout:Sep 21 03:58:17 smithi150 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-10[58206]: 2024-09-21T03:58:17.516+0000 7f34020e1640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-09-21T03:57:31.221107+0000 front 2024-09-21T03:57:31.221162+0000 (oldest deadline 2024-09-21T03:57:56.521240+0000) 2024-09-21T03:58:17.975 INFO:journalctl@ceph.osd.10.smithi150.stdout:Sep 21 03:58:17 smithi150 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-10[58206]: 2024-09-21T03:58:17.516+0000 7f34020e1640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-09-21T03:57:40.022210+0000 front 2024-09-21T03:57:40.022157+0000 (oldest deadline 2024-09-21T03:58:01.722244+0000) 2024-09-21T03:58:17.975 INFO:journalctl@ceph.osd.10.smithi150.stdout:Sep 21 03:58:17 smithi150 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-10[58206]: 2024-09-21T03:58:17.516+0000 7f34020e1640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2024-09-21T03:57:46.323919+0000 front 2024-09-21T03:57:46.323891+0000 (oldest deadline 2024-09-21T03:58:10.423788+0000) 2024-09-21T03:58:18.084 INFO:journalctl@ceph.osd.9.smithi039.stdout:Sep 21 03:58:17 smithi039 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-9[67006]: 2024-09-21T03:58:17.803+0000 7f4a138c4640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-09-21T03:57:22.331574+0000 front 2024-09-21T03:57:22.332146+0000 (oldest deadline 2024-09-21T03:57:45.831720+0000) 2024-09-21T03:58:18.084 INFO:journalctl@ceph.osd.9.smithi039.stdout:Sep 21 03:58:17 smithi039 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-9[67006]: 2024-09-21T03:58:17.803+0000 7f4a138c4640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-09-21T03:57:29.932951+0000 front 2024-09-21T03:57:29.932930+0000 (oldest deadline 2024-09-21T03:57:54.632848+0000) 2024-09-21T03:58:18.084 INFO:journalctl@ceph.osd.9.smithi039.stdout:Sep 21 03:58:17 smithi039 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-9[67006]: 2024-09-21T03:58:17.803+0000 7f4a138c4640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-09-21T03:57:40.434486+0000 front 2024-09-21T03:57:40.434450+0000 (oldest deadline 2024-09-21T03:58:02.734524+0000) 2024-09-21T03:58:18.084 INFO:journalctl@ceph.osd.9.smithi039.stdout:Sep 21 03:58:17 smithi039 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-9[67006]: 2024-09-21T03:58:17.803+0000 7f4a138c4640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2024-09-21T03:57:44.336000+0000 front 2024-09-21T03:57:44.336099+0000 (oldest deadline 2024-09-21T03:58:10.136509+0000) 2024-09-21T03:58:18.218 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 21 03:58:17 smithi161 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-11[58829]: 2024-09-21T03:58:17.947+0000 7f538ef4a640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-09-21T03:57:21.581303+0000 front 2024-09-21T03:57:21.581333+0000 (oldest deadline 2024-09-21T03:57:46.881367+0000) 2024-09-21T03:58:18.218 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 21 03:58:17 smithi161 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-11[58829]: 2024-09-21T03:58:17.947+0000 7f538ef4a640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-09-21T03:57:32.782524+0000 front 2024-09-21T03:57:32.782318+0000 (oldest deadline 2024-09-21T03:57:54.482443+0000) 2024-09-21T03:58:18.218 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 21 03:58:17 smithi161 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-11[58829]: 2024-09-21T03:58:17.947+0000 7f538ef4a640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-09-21T03:57:40.184584+0000 front 2024-09-21T03:57:40.184544+0000 (oldest deadline 2024-09-21T03:58:05.484668+0000) 2024-09-21T03:58:18.475 INFO:journalctl@ceph.osd.7.smithi150.stdout:Sep 21 03:58:18 smithi150 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-7[51981]: 2024-09-21T03:58:18.221+0000 7f2e0cdd2640 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-09-21T03:57:31.734202+0000 front 2024-09-21T03:57:31.734234+0000 (oldest deadline 2024-09-21T03:57:55.834353+0000) 2024-09-21T03:58:18.475 INFO:journalctl@ceph.osd.7.smithi150.stdout:Sep 21 03:58:18 smithi150 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-7[51981]: 2024-09-21T03:58:18.221+0000 7f2e0cdd2640 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-09-21T03:57:38.735442+0000 front 2024-09-21T03:57:38.735506+0000 (oldest deadline 2024-09-21T03:58:02.835564+0000) 2024-09-21T03:58:18.475 INFO:journalctl@ceph.osd.7.smithi150.stdout:Sep 21 03:58:18 smithi150 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-7[51981]: 2024-09-21T03:58:18.221+0000 7f2e0cdd2640 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2024-09-21T03:57:48.037325+0000 front 2024-09-21T03:57:48.037200+0000 (oldest deadline 2024-09-21T03:58:09.637479+0000) 2024-09-21T03:58:18.718 INFO:journalctl@ceph.osd.8.smithi161.stdout:Sep 21 03:58:18 smithi161 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-8[52151]: 2024-09-21T03:58:18.277+0000 7f7e9a0f6640 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-09-21T03:57:29.951582+0000 front 2024-09-21T03:57:29.951481+0000 (oldest deadline 2024-09-21T03:57:54.051508+0000) 2024-09-21T03:58:18.718 INFO:journalctl@ceph.osd.8.smithi161.stdout:Sep 21 03:58:18 smithi161 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-8[52151]: 2024-09-21T03:58:18.277+0000 7f7e9a0f6640 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-09-21T03:57:39.753762+0000 front 2024-09-21T03:57:39.753690+0000 (oldest deadline 2024-09-21T03:58:04.453819+0000) 2024-09-21T03:58:18.718 INFO:journalctl@ceph.osd.8.smithi161.stdout:Sep 21 03:58:18 smithi161 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-8[52151]: 2024-09-21T03:58:18.277+0000 7f7e9a0f6640 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2024-09-21T03:57:48.455067+0000 front 2024-09-21T03:57:48.455286+0000 (oldest deadline 2024-09-21T03:58:10.755147+0000) 2024-09-21T03:58:18.975 INFO:journalctl@ceph.osd.10.smithi150.stdout:Sep 21 03:58:18 smithi150 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-10[58206]: 2024-09-21T03:58:18.503+0000 7f34020e1640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-09-21T03:57:23.120033+0000 front 2024-09-21T03:57:23.120109+0000 (oldest deadline 2024-09-21T03:57:45.920434+0000) 2024-09-21T03:58:18.975 INFO:journalctl@ceph.osd.10.smithi150.stdout:Sep 21 03:58:18 smithi150 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-10[58206]: 2024-09-21T03:58:18.503+0000 7f34020e1640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-09-21T03:57:31.221107+0000 front 2024-09-21T03:57:31.221162+0000 (oldest deadline 2024-09-21T03:57:56.521240+0000) 2024-09-21T03:58:18.975 INFO:journalctl@ceph.osd.10.smithi150.stdout:Sep 21 03:58:18 smithi150 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-10[58206]: 2024-09-21T03:58:18.503+0000 7f34020e1640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-09-21T03:57:40.022210+0000 front 2024-09-21T03:57:40.022157+0000 (oldest deadline 2024-09-21T03:58:01.722244+0000) 2024-09-21T03:58:18.975 INFO:journalctl@ceph.osd.10.smithi150.stdout:Sep 21 03:58:18 smithi150 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-10[58206]: 2024-09-21T03:58:18.503+0000 7f34020e1640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2024-09-21T03:57:46.323919+0000 front 2024-09-21T03:57:46.323891+0000 (oldest deadline 2024-09-21T03:58:10.423788+0000) 2024-09-21T03:58:19.084 INFO:journalctl@ceph.osd.9.smithi039.stdout:Sep 21 03:58:18 smithi039 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-9[67006]: 2024-09-21T03:58:18.829+0000 7f4a138c4640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-09-21T03:57:22.331574+0000 front 2024-09-21T03:57:22.332146+0000 (oldest deadline 2024-09-21T03:57:45.831720+0000) 2024-09-21T03:58:19.084 INFO:journalctl@ceph.osd.9.smithi039.stdout:Sep 21 03:58:18 smithi039 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-9[67006]: 2024-09-21T03:58:18.829+0000 7f4a138c4640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-09-21T03:57:29.932951+0000 front 2024-09-21T03:57:29.932930+0000 (oldest deadline 2024-09-21T03:57:54.632848+0000) 2024-09-21T03:58:19.084 INFO:journalctl@ceph.osd.9.smithi039.stdout:Sep 21 03:58:18 smithi039 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-9[67006]: 2024-09-21T03:58:18.829+0000 7f4a138c4640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-09-21T03:57:40.434486+0000 front 2024-09-21T03:57:40.434450+0000 (oldest deadline 2024-09-21T03:58:02.734524+0000) 2024-09-21T03:58:19.084 INFO:journalctl@ceph.osd.9.smithi039.stdout:Sep 21 03:58:18 smithi039 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-9[67006]: 2024-09-21T03:58:18.829+0000 7f4a138c4640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2024-09-21T03:57:44.336000+0000 front 2024-09-21T03:57:44.336099+0000 (oldest deadline 2024-09-21T03:58:10.136509+0000) 2024-09-21T03:58:19.218 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 21 03:58:18 smithi161 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-11[58829]: 2024-09-21T03:58:18.903+0000 7f538ef4a640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-09-21T03:57:21.581303+0000 front 2024-09-21T03:57:21.581333+0000 (oldest deadline 2024-09-21T03:57:46.881367+0000) 2024-09-21T03:58:19.218 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 21 03:58:18 smithi161 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-11[58829]: 2024-09-21T03:58:18.903+0000 7f538ef4a640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-09-21T03:57:32.782524+0000 front 2024-09-21T03:57:32.782318+0000 (oldest deadline 2024-09-21T03:57:54.482443+0000) 2024-09-21T03:58:19.218 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 21 03:58:18 smithi161 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-11[58829]: 2024-09-21T03:58:18.903+0000 7f538ef4a640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-09-21T03:57:40.184584+0000 front 2024-09-21T03:57:40.184544+0000 (oldest deadline 2024-09-21T03:58:05.484668+0000) 2024-09-21T03:58:19.718 INFO:journalctl@ceph.osd.8.smithi161.stdout:Sep 21 03:58:19 smithi161 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-8[52151]: 2024-09-21T03:58:19.312+0000 7f7e9a0f6640 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-09-21T03:57:29.951582+0000 front 2024-09-21T03:57:29.951481+0000 (oldest deadline 2024-09-21T03:57:54.051508+0000) 2024-09-21T03:58:19.718 INFO:journalctl@ceph.osd.8.smithi161.stdout:Sep 21 03:58:19 smithi161 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-8[52151]: 2024-09-21T03:58:19.312+0000 7f7e9a0f6640 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-09-21T03:57:39.753762+0000 front 2024-09-21T03:57:39.753690+0000 (oldest deadline 2024-09-21T03:58:04.453819+0000) 2024-09-21T03:58:19.718 INFO:journalctl@ceph.osd.8.smithi161.stdout:Sep 21 03:58:19 smithi161 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-8[52151]: 2024-09-21T03:58:19.312+0000 7f7e9a0f6640 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2024-09-21T03:57:48.455067+0000 front 2024-09-21T03:57:48.455286+0000 (oldest deadline 2024-09-21T03:58:10.755147+0000) 2024-09-21T03:58:19.725 INFO:journalctl@ceph.osd.7.smithi150.stdout:Sep 21 03:58:19 smithi150 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-7[51981]: 2024-09-21T03:58:19.244+0000 7f2e0cdd2640 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-09-21T03:57:31.734202+0000 front 2024-09-21T03:57:31.734234+0000 (oldest deadline 2024-09-21T03:57:55.834353+0000) 2024-09-21T03:58:19.725 INFO:journalctl@ceph.osd.7.smithi150.stdout:Sep 21 03:58:19 smithi150 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-7[51981]: 2024-09-21T03:58:19.245+0000 7f2e0cdd2640 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-09-21T03:57:38.735442+0000 front 2024-09-21T03:57:38.735506+0000 (oldest deadline 2024-09-21T03:58:02.835564+0000) 2024-09-21T03:58:19.725 INFO:journalctl@ceph.osd.7.smithi150.stdout:Sep 21 03:58:19 smithi150 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-7[51981]: 2024-09-21T03:58:19.245+0000 7f2e0cdd2640 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2024-09-21T03:57:48.037325+0000 front 2024-09-21T03:57:48.037200+0000 (oldest deadline 2024-09-21T03:58:09.637479+0000) 2024-09-21T03:58:19.725 INFO:journalctl@ceph.osd.7.smithi150.stdout:Sep 21 03:58:19 smithi150 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-7[51981]: 2024-09-21T03:58:19.245+0000 7f2e0cdd2640 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-09-21T03:57:54.139492+0000 front 2024-09-21T03:57:54.139527+0000 (oldest deadline 2024-09-21T03:58:18.839556+0000) 2024-09-21T03:58:19.725 INFO:journalctl@ceph.osd.10.smithi150.stdout:Sep 21 03:58:19 smithi150 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-10[58206]: 2024-09-21T03:58:19.485+0000 7f34020e1640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-09-21T03:57:23.120033+0000 front 2024-09-21T03:57:23.120109+0000 (oldest deadline 2024-09-21T03:57:45.920434+0000) 2024-09-21T03:58:19.725 INFO:journalctl@ceph.osd.10.smithi150.stdout:Sep 21 03:58:19 smithi150 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-10[58206]: 2024-09-21T03:58:19.485+0000 7f34020e1640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-09-21T03:57:31.221107+0000 front 2024-09-21T03:57:31.221162+0000 (oldest deadline 2024-09-21T03:57:56.521240+0000) 2024-09-21T03:58:19.725 INFO:journalctl@ceph.osd.10.smithi150.stdout:Sep 21 03:58:19 smithi150 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-10[58206]: 2024-09-21T03:58:19.485+0000 7f34020e1640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-09-21T03:57:40.022210+0000 front 2024-09-21T03:57:40.022157+0000 (oldest deadline 2024-09-21T03:58:01.722244+0000) 2024-09-21T03:58:19.725 INFO:journalctl@ceph.osd.10.smithi150.stdout:Sep 21 03:58:19 smithi150 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-10[58206]: 2024-09-21T03:58:19.485+0000 7f34020e1640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2024-09-21T03:57:46.323919+0000 front 2024-09-21T03:57:46.323891+0000 (oldest deadline 2024-09-21T03:58:10.423788+0000) 2024-09-21T03:58:20.218 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 21 03:58:19 smithi161 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-11[58829]: 2024-09-21T03:58:19.905+0000 7f538ef4a640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-09-21T03:57:21.581303+0000 front 2024-09-21T03:57:21.581333+0000 (oldest deadline 2024-09-21T03:57:46.881367+0000) 2024-09-21T03:58:20.218 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 21 03:58:19 smithi161 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-11[58829]: 2024-09-21T03:58:19.905+0000 7f538ef4a640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-09-21T03:57:32.782524+0000 front 2024-09-21T03:57:32.782318+0000 (oldest deadline 2024-09-21T03:57:54.482443+0000) 2024-09-21T03:58:20.218 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 21 03:58:19 smithi161 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-11[58829]: 2024-09-21T03:58:19.905+0000 7f538ef4a640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-09-21T03:57:40.184584+0000 front 2024-09-21T03:57:40.184544+0000 (oldest deadline 2024-09-21T03:58:05.484668+0000) 2024-09-21T03:58:20.334 INFO:journalctl@ceph.osd.9.smithi039.stdout:Sep 21 03:58:19 smithi039 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-9[67006]: 2024-09-21T03:58:19.853+0000 7f4a138c4640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-09-21T03:57:22.331574+0000 front 2024-09-21T03:57:22.332146+0000 (oldest deadline 2024-09-21T03:57:45.831720+0000) 2024-09-21T03:58:20.334 INFO:journalctl@ceph.osd.9.smithi039.stdout:Sep 21 03:58:19 smithi039 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-9[67006]: 2024-09-21T03:58:19.853+0000 7f4a138c4640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-09-21T03:57:29.932951+0000 front 2024-09-21T03:57:29.932930+0000 (oldest deadline 2024-09-21T03:57:54.632848+0000) 2024-09-21T03:58:20.334 INFO:journalctl@ceph.osd.9.smithi039.stdout:Sep 21 03:58:19 smithi039 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-9[67006]: 2024-09-21T03:58:19.853+0000 7f4a138c4640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-09-21T03:57:40.434486+0000 front 2024-09-21T03:57:40.434450+0000 (oldest deadline 2024-09-21T03:58:02.734524+0000) 2024-09-21T03:58:20.334 INFO:journalctl@ceph.osd.9.smithi039.stdout:Sep 21 03:58:19 smithi039 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-9[67006]: 2024-09-21T03:58:19.853+0000 7f4a138c4640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2024-09-21T03:57:44.336000+0000 front 2024-09-21T03:57:44.336099+0000 (oldest deadline 2024-09-21T03:58:10.136509+0000) 2024-09-21T03:58:20.334 INFO:journalctl@ceph.osd.9.smithi039.stdout:Sep 21 03:58:19 smithi039 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-9[67006]: 2024-09-21T03:58:19.853+0000 7f4a138c4640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-09-21T03:57:56.037703+0000 front 2024-09-21T03:57:56.037728+0000 (oldest deadline 2024-09-21T03:58:19.537285+0000) 2024-09-21T03:58:20.718 INFO:journalctl@ceph.osd.8.smithi161.stdout:Sep 21 03:58:20 smithi161 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-8[52151]: 2024-09-21T03:58:20.359+0000 7f7e9a0f6640 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-09-21T03:57:29.951582+0000 front 2024-09-21T03:57:29.951481+0000 (oldest deadline 2024-09-21T03:57:54.051508+0000) 2024-09-21T03:58:20.718 INFO:journalctl@ceph.osd.8.smithi161.stdout:Sep 21 03:58:20 smithi161 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-8[52151]: 2024-09-21T03:58:20.359+0000 7f7e9a0f6640 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-09-21T03:57:39.753762+0000 front 2024-09-21T03:57:39.753690+0000 (oldest deadline 2024-09-21T03:58:04.453819+0000) 2024-09-21T03:58:20.718 INFO:journalctl@ceph.osd.8.smithi161.stdout:Sep 21 03:58:20 smithi161 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-8[52151]: 2024-09-21T03:58:20.359+0000 7f7e9a0f6640 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2024-09-21T03:57:48.455067+0000 front 2024-09-21T03:57:48.455286+0000 (oldest deadline 2024-09-21T03:58:10.755147+0000) 2024-09-21T03:58:20.718 INFO:journalctl@ceph.osd.8.smithi161.stdout:Sep 21 03:58:20 smithi161 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-8[52151]: 2024-09-21T03:58:20.359+0000 7f7e9a0f6640 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-09-21T03:57:56.556595+0000 front 2024-09-21T03:57:56.556636+0000 (oldest deadline 2024-09-21T03:58:20.056711+0000) 2024-09-21T03:58:20.725 INFO:journalctl@ceph.osd.7.smithi150.stdout:Sep 21 03:58:20 smithi150 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-7[51981]: 2024-09-21T03:58:20.229+0000 7f2e0cdd2640 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-09-21T03:57:31.734202+0000 front 2024-09-21T03:57:31.734234+0000 (oldest deadline 2024-09-21T03:57:55.834353+0000) 2024-09-21T03:58:20.725 INFO:journalctl@ceph.osd.7.smithi150.stdout:Sep 21 03:58:20 smithi150 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-7[51981]: 2024-09-21T03:58:20.229+0000 7f2e0cdd2640 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-09-21T03:57:38.735442+0000 front 2024-09-21T03:57:38.735506+0000 (oldest deadline 2024-09-21T03:58:02.835564+0000) 2024-09-21T03:58:20.725 INFO:journalctl@ceph.osd.7.smithi150.stdout:Sep 21 03:58:20 smithi150 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-7[51981]: 2024-09-21T03:58:20.229+0000 7f2e0cdd2640 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2024-09-21T03:57:48.037325+0000 front 2024-09-21T03:57:48.037200+0000 (oldest deadline 2024-09-21T03:58:09.637479+0000) 2024-09-21T03:58:20.725 INFO:journalctl@ceph.osd.7.smithi150.stdout:Sep 21 03:58:20 smithi150 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-7[51981]: 2024-09-21T03:58:20.229+0000 7f2e0cdd2640 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-09-21T03:57:54.139492+0000 front 2024-09-21T03:57:54.139527+0000 (oldest deadline 2024-09-21T03:58:18.839556+0000) 2024-09-21T03:58:20.725 INFO:journalctl@ceph.osd.10.smithi150.stdout:Sep 21 03:58:20 smithi150 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-10[58206]: 2024-09-21T03:58:20.440+0000 7f34020e1640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-09-21T03:57:23.120033+0000 front 2024-09-21T03:57:23.120109+0000 (oldest deadline 2024-09-21T03:57:45.920434+0000) 2024-09-21T03:58:20.725 INFO:journalctl@ceph.osd.10.smithi150.stdout:Sep 21 03:58:20 smithi150 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-10[58206]: 2024-09-21T03:58:20.440+0000 7f34020e1640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-09-21T03:57:31.221107+0000 front 2024-09-21T03:57:31.221162+0000 (oldest deadline 2024-09-21T03:57:56.521240+0000) 2024-09-21T03:58:20.725 INFO:journalctl@ceph.osd.10.smithi150.stdout:Sep 21 03:58:20 smithi150 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-10[58206]: 2024-09-21T03:58:20.440+0000 7f34020e1640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-09-21T03:57:40.022210+0000 front 2024-09-21T03:57:40.022157+0000 (oldest deadline 2024-09-21T03:58:01.722244+0000) 2024-09-21T03:58:20.726 INFO:journalctl@ceph.osd.10.smithi150.stdout:Sep 21 03:58:20 smithi150 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-10[58206]: 2024-09-21T03:58:20.440+0000 7f34020e1640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2024-09-21T03:57:46.323919+0000 front 2024-09-21T03:57:46.323891+0000 (oldest deadline 2024-09-21T03:58:10.423788+0000) 2024-09-21T03:58:20.726 INFO:journalctl@ceph.osd.10.smithi150.stdout:Sep 21 03:58:20 smithi150 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-10[58206]: 2024-09-21T03:58:20.440+0000 7f34020e1640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-09-21T03:57:56.225398+0000 front 2024-09-21T03:57:56.225320+0000 (oldest deadline 2024-09-21T03:58:20.325491+0000) 2024-09-21T03:58:21.218 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 21 03:58:20 smithi161 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-11[58829]: 2024-09-21T03:58:20.858+0000 7f538ef4a640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-09-21T03:57:21.581303+0000 front 2024-09-21T03:57:21.581333+0000 (oldest deadline 2024-09-21T03:57:46.881367+0000) 2024-09-21T03:58:21.218 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 21 03:58:20 smithi161 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-11[58829]: 2024-09-21T03:58:20.858+0000 7f538ef4a640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-09-21T03:57:32.782524+0000 front 2024-09-21T03:57:32.782318+0000 (oldest deadline 2024-09-21T03:57:54.482443+0000) 2024-09-21T03:58:21.218 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 21 03:58:20 smithi161 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-11[58829]: 2024-09-21T03:58:20.858+0000 7f538ef4a640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-09-21T03:57:40.184584+0000 front 2024-09-21T03:57:40.184544+0000 (oldest deadline 2024-09-21T03:58:05.484668+0000) 2024-09-21T03:58:21.218 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 21 03:58:20 smithi161 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-11[58829]: 2024-09-21T03:58:20.858+0000 7f538ef4a640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-09-21T03:57:55.387018+0000 front 2024-09-21T03:57:55.386996+0000 (oldest deadline 2024-09-21T03:58:20.086771+0000) 2024-09-21T03:58:21.334 INFO:journalctl@ceph.osd.9.smithi039.stdout:Sep 21 03:58:20 smithi039 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-9[67006]: 2024-09-21T03:58:20.877+0000 7f4a138c4640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-09-21T03:57:22.331574+0000 front 2024-09-21T03:57:22.332146+0000 (oldest deadline 2024-09-21T03:57:45.831720+0000) 2024-09-21T03:58:21.334 INFO:journalctl@ceph.osd.9.smithi039.stdout:Sep 21 03:58:20 smithi039 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-9[67006]: 2024-09-21T03:58:20.877+0000 7f4a138c4640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-09-21T03:57:29.932951+0000 front 2024-09-21T03:57:29.932930+0000 (oldest deadline 2024-09-21T03:57:54.632848+0000) 2024-09-21T03:58:21.334 INFO:journalctl@ceph.osd.9.smithi039.stdout:Sep 21 03:58:20 smithi039 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-9[67006]: 2024-09-21T03:58:20.877+0000 7f4a138c4640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-09-21T03:57:40.434486+0000 front 2024-09-21T03:57:40.434450+0000 (oldest deadline 2024-09-21T03:58:02.734524+0000) 2024-09-21T03:58:21.334 INFO:journalctl@ceph.osd.9.smithi039.stdout:Sep 21 03:58:20 smithi039 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-9[67006]: 2024-09-21T03:58:20.877+0000 7f4a138c4640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2024-09-21T03:57:44.336000+0000 front 2024-09-21T03:57:44.336099+0000 (oldest deadline 2024-09-21T03:58:10.136509+0000) 2024-09-21T03:58:21.334 INFO:journalctl@ceph.osd.9.smithi039.stdout:Sep 21 03:58:20 smithi039 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-9[67006]: 2024-09-21T03:58:20.877+0000 7f4a138c4640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-09-21T03:57:56.037703+0000 front 2024-09-21T03:57:56.037728+0000 (oldest deadline 2024-09-21T03:58:19.537285+0000) 2024-09-21T03:58:21.443 INFO:journalctl@ceph.osd.7.smithi150.stdout:Sep 21 03:58:21 smithi150 podman[67483]: 2024-09-21 03:58:21.181673079 +0000 UTC m=+5.536236022 container died 0ca8bb1614c11390568d76f1d0779c57773529739d391a2336eef17e5588041a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-7, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, ceph=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2) 2024-09-21T03:58:21.718 INFO:journalctl@ceph.osd.8.smithi161.stdout:Sep 21 03:58:21 smithi161 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-8[52151]: 2024-09-21T03:58:21.384+0000 7f7e9a0f6640 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-09-21T03:57:29.951582+0000 front 2024-09-21T03:57:29.951481+0000 (oldest deadline 2024-09-21T03:57:54.051508+0000) 2024-09-21T03:58:21.718 INFO:journalctl@ceph.osd.8.smithi161.stdout:Sep 21 03:58:21 smithi161 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-8[52151]: 2024-09-21T03:58:21.384+0000 7f7e9a0f6640 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-09-21T03:57:39.753762+0000 front 2024-09-21T03:57:39.753690+0000 (oldest deadline 2024-09-21T03:58:04.453819+0000) 2024-09-21T03:58:21.718 INFO:journalctl@ceph.osd.8.smithi161.stdout:Sep 21 03:58:21 smithi161 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-8[52151]: 2024-09-21T03:58:21.384+0000 7f7e9a0f6640 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2024-09-21T03:57:48.455067+0000 front 2024-09-21T03:57:48.455286+0000 (oldest deadline 2024-09-21T03:58:10.755147+0000) 2024-09-21T03:58:21.718 INFO:journalctl@ceph.osd.8.smithi161.stdout:Sep 21 03:58:21 smithi161 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-8[52151]: 2024-09-21T03:58:21.384+0000 7f7e9a0f6640 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-09-21T03:57:56.556595+0000 front 2024-09-21T03:57:56.556636+0000 (oldest deadline 2024-09-21T03:58:20.056711+0000) 2024-09-21T03:58:21.725 INFO:journalctl@ceph.osd.7.smithi150.stdout:Sep 21 03:58:21 smithi150 podman[67483]: 2024-09-21 03:58:21.559773154 +0000 UTC m=+5.914336093 container cleanup 0ca8bb1614c11390568d76f1d0779c57773529739d391a2336eef17e5588041a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-7, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_CLEAN=True, ceph=True) 2024-09-21T03:58:21.725 INFO:journalctl@ceph.osd.7.smithi150.stdout:Sep 21 03:58:21 smithi150 bash[67483]: ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-7 2024-09-21T03:58:21.725 INFO:journalctl@ceph.osd.10.smithi150.stdout:Sep 21 03:58:21 smithi150 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-10[58206]: 2024-09-21T03:58:21.441+0000 7f34020e1640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-09-21T03:57:23.120033+0000 front 2024-09-21T03:57:23.120109+0000 (oldest deadline 2024-09-21T03:57:45.920434+0000) 2024-09-21T03:58:21.725 INFO:journalctl@ceph.osd.10.smithi150.stdout:Sep 21 03:58:21 smithi150 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-10[58206]: 2024-09-21T03:58:21.441+0000 7f34020e1640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-09-21T03:57:31.221107+0000 front 2024-09-21T03:57:31.221162+0000 (oldest deadline 2024-09-21T03:57:56.521240+0000) 2024-09-21T03:58:21.725 INFO:journalctl@ceph.osd.10.smithi150.stdout:Sep 21 03:58:21 smithi150 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-10[58206]: 2024-09-21T03:58:21.441+0000 7f34020e1640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-09-21T03:57:40.022210+0000 front 2024-09-21T03:57:40.022157+0000 (oldest deadline 2024-09-21T03:58:01.722244+0000) 2024-09-21T03:58:21.725 INFO:journalctl@ceph.osd.10.smithi150.stdout:Sep 21 03:58:21 smithi150 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-10[58206]: 2024-09-21T03:58:21.441+0000 7f34020e1640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2024-09-21T03:57:46.323919+0000 front 2024-09-21T03:57:46.323891+0000 (oldest deadline 2024-09-21T03:58:10.423788+0000) 2024-09-21T03:58:21.725 INFO:journalctl@ceph.osd.10.smithi150.stdout:Sep 21 03:58:21 smithi150 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-10[58206]: 2024-09-21T03:58:21.441+0000 7f34020e1640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-09-21T03:57:56.225398+0000 front 2024-09-21T03:57:56.225320+0000 (oldest deadline 2024-09-21T03:58:20.325491+0000) 2024-09-21T03:58:22.176 INFO:journalctl@ceph.osd.7.smithi150.stdout:Sep 21 03:58:21 smithi150 podman[67510]: 2024-09-21 03:58:21.82782251 +0000 UTC m=+0.642946493 container remove 0ca8bb1614c11390568d76f1d0779c57773529739d391a2336eef17e5588041a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-7, maintainer=Guillaume Abrioux , GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731) 2024-09-21T03:58:22.218 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 21 03:58:21 smithi161 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-11[58829]: 2024-09-21T03:58:21.907+0000 7f538ef4a640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-09-21T03:57:21.581303+0000 front 2024-09-21T03:57:21.581333+0000 (oldest deadline 2024-09-21T03:57:46.881367+0000) 2024-09-21T03:58:22.218 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 21 03:58:21 smithi161 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-11[58829]: 2024-09-21T03:58:21.907+0000 7f538ef4a640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-09-21T03:57:32.782524+0000 front 2024-09-21T03:57:32.782318+0000 (oldest deadline 2024-09-21T03:57:54.482443+0000) 2024-09-21T03:58:22.218 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 21 03:58:21 smithi161 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-11[58829]: 2024-09-21T03:58:21.907+0000 7f538ef4a640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-09-21T03:57:40.184584+0000 front 2024-09-21T03:57:40.184544+0000 (oldest deadline 2024-09-21T03:58:05.484668+0000) 2024-09-21T03:58:22.218 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 21 03:58:21 smithi161 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-11[58829]: 2024-09-21T03:58:21.907+0000 7f538ef4a640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-09-21T03:57:55.387018+0000 front 2024-09-21T03:57:55.386996+0000 (oldest deadline 2024-09-21T03:58:20.086771+0000) 2024-09-21T03:58:22.334 INFO:journalctl@ceph.osd.9.smithi039.stdout:Sep 21 03:58:21 smithi039 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-9[67006]: 2024-09-21T03:58:21.909+0000 7f4a138c4640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-09-21T03:57:22.331574+0000 front 2024-09-21T03:57:22.332146+0000 (oldest deadline 2024-09-21T03:57:45.831720+0000) 2024-09-21T03:58:22.334 INFO:journalctl@ceph.osd.9.smithi039.stdout:Sep 21 03:58:21 smithi039 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-9[67006]: 2024-09-21T03:58:21.909+0000 7f4a138c4640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-09-21T03:57:29.932951+0000 front 2024-09-21T03:57:29.932930+0000 (oldest deadline 2024-09-21T03:57:54.632848+0000) 2024-09-21T03:58:22.334 INFO:journalctl@ceph.osd.9.smithi039.stdout:Sep 21 03:58:21 smithi039 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-9[67006]: 2024-09-21T03:58:21.909+0000 7f4a138c4640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-09-21T03:57:40.434486+0000 front 2024-09-21T03:57:40.434450+0000 (oldest deadline 2024-09-21T03:58:02.734524+0000) 2024-09-21T03:58:22.334 INFO:journalctl@ceph.osd.9.smithi039.stdout:Sep 21 03:58:21 smithi039 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-9[67006]: 2024-09-21T03:58:21.909+0000 7f4a138c4640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2024-09-21T03:57:44.336000+0000 front 2024-09-21T03:57:44.336099+0000 (oldest deadline 2024-09-21T03:58:10.136509+0000) 2024-09-21T03:58:22.334 INFO:journalctl@ceph.osd.9.smithi039.stdout:Sep 21 03:58:21 smithi039 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-9[67006]: 2024-09-21T03:58:21.909+0000 7f4a138c4640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-09-21T03:57:56.037703+0000 front 2024-09-21T03:57:56.037728+0000 (oldest deadline 2024-09-21T03:58:19.537285+0000) 2024-09-21T03:58:22.428 INFO:journalctl@ceph.osd.7.smithi150.stdout:Sep 21 03:58:22 smithi150 podman[67599]: 2024-09-21 03:58:22.077135718 +0000 UTC m=+0.015712076 image pull ddce307e1f3263fb077b4f3902e15660050631dad4d2823a20ee3f72bf18d977 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262 2024-09-21T03:58:22.428 INFO:journalctl@ceph.osd.7.smithi150.stdout:Sep 21 03:58:22 smithi150 podman[67599]: 2024-09-21 03:58:22.211884778 +0000 UTC m=+0.150461132 container create 66d9809cbc70dfa614f3f1c9ce51ad6a21c16d4d15391c0e3545c65590eba6ba (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-7-deactivate, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, ceph=True, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2) 2024-09-21T03:58:22.428 INFO:journalctl@ceph.osd.7.smithi150.stdout:Sep 21 03:58:22 smithi150 podman[67599]: 2024-09-21 03:58:22.423034276 +0000 UTC m=+0.361610642 container init 66d9809cbc70dfa614f3f1c9ce51ad6a21c16d4d15391c0e3545c65590eba6ba (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-7-deactivate, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, maintainer=Guillaume Abrioux ) 2024-09-21T03:58:22.428 INFO:journalctl@ceph.osd.7.smithi150.stdout:Sep 21 03:58:22 smithi150 podman[67599]: 2024-09-21 03:58:22.426499757 +0000 UTC m=+0.365076111 container start 66d9809cbc70dfa614f3f1c9ce51ad6a21c16d4d15391c0e3545c65590eba6ba (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-7-deactivate, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, io.buildah.version=1.37.2, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, CEPH_POINT_RELEASE=) 2024-09-21T03:58:22.718 INFO:journalctl@ceph.osd.8.smithi161.stdout:Sep 21 03:58:22 smithi161 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-8[52151]: 2024-09-21T03:58:22.424+0000 7f7e9a0f6640 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-09-21T03:57:29.951582+0000 front 2024-09-21T03:57:29.951481+0000 (oldest deadline 2024-09-21T03:57:54.051508+0000) 2024-09-21T03:58:22.718 INFO:journalctl@ceph.osd.8.smithi161.stdout:Sep 21 03:58:22 smithi161 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-8[52151]: 2024-09-21T03:58:22.424+0000 7f7e9a0f6640 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-09-21T03:57:39.753762+0000 front 2024-09-21T03:57:39.753690+0000 (oldest deadline 2024-09-21T03:58:04.453819+0000) 2024-09-21T03:58:22.718 INFO:journalctl@ceph.osd.8.smithi161.stdout:Sep 21 03:58:22 smithi161 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-8[52151]: 2024-09-21T03:58:22.424+0000 7f7e9a0f6640 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2024-09-21T03:57:48.455067+0000 front 2024-09-21T03:57:48.455286+0000 (oldest deadline 2024-09-21T03:58:10.755147+0000) 2024-09-21T03:58:22.718 INFO:journalctl@ceph.osd.8.smithi161.stdout:Sep 21 03:58:22 smithi161 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-8[52151]: 2024-09-21T03:58:22.424+0000 7f7e9a0f6640 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-09-21T03:57:56.556595+0000 front 2024-09-21T03:57:56.556636+0000 (oldest deadline 2024-09-21T03:58:20.056711+0000) 2024-09-21T03:58:22.725 INFO:journalctl@ceph.osd.7.smithi150.stdout:Sep 21 03:58:22 smithi150 podman[67599]: 2024-09-21 03:58:22.522921157 +0000 UTC m=+0.461497530 container attach 66d9809cbc70dfa614f3f1c9ce51ad6a21c16d4d15391c0e3545c65590eba6ba (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-7-deactivate, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, ceph=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731) 2024-09-21T03:58:22.725 INFO:journalctl@ceph.osd.7.smithi150.stdout:Sep 21 03:58:22 smithi150 podman[67599]: 2024-09-21 03:58:22.615125411 +0000 UTC m=+0.553701773 container died 66d9809cbc70dfa614f3f1c9ce51ad6a21c16d4d15391c0e3545c65590eba6ba (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-7-deactivate, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-21T03:58:22.725 INFO:journalctl@ceph.osd.10.smithi150.stdout:Sep 21 03:58:22 smithi150 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-10[58206]: 2024-09-21T03:58:22.451+0000 7f34020e1640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-09-21T03:57:23.120033+0000 front 2024-09-21T03:57:23.120109+0000 (oldest deadline 2024-09-21T03:57:45.920434+0000) 2024-09-21T03:58:22.725 INFO:journalctl@ceph.osd.10.smithi150.stdout:Sep 21 03:58:22 smithi150 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-10[58206]: 2024-09-21T03:58:22.451+0000 7f34020e1640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-09-21T03:57:31.221107+0000 front 2024-09-21T03:57:31.221162+0000 (oldest deadline 2024-09-21T03:57:56.521240+0000) 2024-09-21T03:58:22.725 INFO:journalctl@ceph.osd.10.smithi150.stdout:Sep 21 03:58:22 smithi150 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-10[58206]: 2024-09-21T03:58:22.451+0000 7f34020e1640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-09-21T03:57:40.022210+0000 front 2024-09-21T03:57:40.022157+0000 (oldest deadline 2024-09-21T03:58:01.722244+0000) 2024-09-21T03:58:22.725 INFO:journalctl@ceph.osd.10.smithi150.stdout:Sep 21 03:58:22 smithi150 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-10[58206]: 2024-09-21T03:58:22.451+0000 7f34020e1640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2024-09-21T03:57:46.323919+0000 front 2024-09-21T03:57:46.323891+0000 (oldest deadline 2024-09-21T03:58:10.423788+0000) 2024-09-21T03:58:22.725 INFO:journalctl@ceph.osd.10.smithi150.stdout:Sep 21 03:58:22 smithi150 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-10[58206]: 2024-09-21T03:58:22.451+0000 7f34020e1640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-09-21T03:57:56.225398+0000 front 2024-09-21T03:57:56.225320+0000 (oldest deadline 2024-09-21T03:58:20.325491+0000) 2024-09-21T03:58:23.218 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 21 03:58:22 smithi161 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-11[58829]: 2024-09-21T03:58:22.863+0000 7f538ef4a640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-09-21T03:57:21.581303+0000 front 2024-09-21T03:57:21.581333+0000 (oldest deadline 2024-09-21T03:57:46.881367+0000) 2024-09-21T03:58:23.218 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 21 03:58:22 smithi161 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-11[58829]: 2024-09-21T03:58:22.863+0000 7f538ef4a640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-09-21T03:57:32.782524+0000 front 2024-09-21T03:57:32.782318+0000 (oldest deadline 2024-09-21T03:57:54.482443+0000) 2024-09-21T03:58:23.218 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 21 03:58:22 smithi161 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-11[58829]: 2024-09-21T03:58:22.863+0000 7f538ef4a640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-09-21T03:57:40.184584+0000 front 2024-09-21T03:57:40.184544+0000 (oldest deadline 2024-09-21T03:58:05.484668+0000) 2024-09-21T03:58:23.218 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 21 03:58:22 smithi161 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-11[58829]: 2024-09-21T03:58:22.863+0000 7f538ef4a640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-09-21T03:57:55.387018+0000 front 2024-09-21T03:57:55.386996+0000 (oldest deadline 2024-09-21T03:58:20.086771+0000) 2024-09-21T03:58:23.290 DEBUG:teuthology.orchestra.run.smithi150:> sudo pkill -f 'journalctl -f -n 0 -u ceph-51116d5c-77ca-11ef-baf5-efdc52797490@osd.7.service' 2024-09-21T03:58:23.334 INFO:journalctl@ceph.osd.9.smithi039.stdout:Sep 21 03:58:22 smithi039 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-9[67006]: 2024-09-21T03:58:22.923+0000 7f4a138c4640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-09-21T03:57:22.331574+0000 front 2024-09-21T03:57:22.332146+0000 (oldest deadline 2024-09-21T03:57:45.831720+0000) 2024-09-21T03:58:23.334 INFO:journalctl@ceph.osd.9.smithi039.stdout:Sep 21 03:58:22 smithi039 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-9[67006]: 2024-09-21T03:58:22.923+0000 7f4a138c4640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-09-21T03:57:29.932951+0000 front 2024-09-21T03:57:29.932930+0000 (oldest deadline 2024-09-21T03:57:54.632848+0000) 2024-09-21T03:58:23.334 INFO:journalctl@ceph.osd.9.smithi039.stdout:Sep 21 03:58:22 smithi039 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-9[67006]: 2024-09-21T03:58:22.923+0000 7f4a138c4640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-09-21T03:57:40.434486+0000 front 2024-09-21T03:57:40.434450+0000 (oldest deadline 2024-09-21T03:58:02.734524+0000) 2024-09-21T03:58:23.334 INFO:journalctl@ceph.osd.9.smithi039.stdout:Sep 21 03:58:22 smithi039 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-9[67006]: 2024-09-21T03:58:22.923+0000 7f4a138c4640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2024-09-21T03:57:44.336000+0000 front 2024-09-21T03:57:44.336099+0000 (oldest deadline 2024-09-21T03:58:10.136509+0000) 2024-09-21T03:58:23.334 INFO:journalctl@ceph.osd.9.smithi039.stdout:Sep 21 03:58:22 smithi039 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-9[67006]: 2024-09-21T03:58:22.923+0000 7f4a138c4640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-09-21T03:57:56.037703+0000 front 2024-09-21T03:57:56.037728+0000 (oldest deadline 2024-09-21T03:58:19.537285+0000) 2024-09-21T03:58:23.369 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-21T03:58:23.369 INFO:tasks.cephadm.osd.7:Stopped osd.7 2024-09-21T03:58:23.369 INFO:tasks.cephadm.osd.8:Stopping osd.8... 2024-09-21T03:58:23.370 DEBUG:teuthology.orchestra.run.smithi161:> sudo systemctl stop ceph-51116d5c-77ca-11ef-baf5-efdc52797490@osd.8 2024-09-21T03:58:23.668 INFO:journalctl@ceph.osd.8.smithi161.stdout:Sep 21 03:58:23 smithi161 systemd[1]: Stopping Ceph osd.8 for 51116d5c-77ca-11ef-baf5-efdc52797490... 2024-09-21T03:58:23.668 INFO:journalctl@ceph.osd.8.smithi161.stdout:Sep 21 03:58:23 smithi161 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-8[52151]: 2024-09-21T03:58:23.458+0000 7f7e9a0f6640 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-09-21T03:57:29.951582+0000 front 2024-09-21T03:57:29.951481+0000 (oldest deadline 2024-09-21T03:57:54.051508+0000) 2024-09-21T03:58:23.668 INFO:journalctl@ceph.osd.8.smithi161.stdout:Sep 21 03:58:23 smithi161 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-8[52151]: 2024-09-21T03:58:23.458+0000 7f7e9a0f6640 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-09-21T03:57:39.753762+0000 front 2024-09-21T03:57:39.753690+0000 (oldest deadline 2024-09-21T03:58:04.453819+0000) 2024-09-21T03:58:23.668 INFO:journalctl@ceph.osd.8.smithi161.stdout:Sep 21 03:58:23 smithi161 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-8[52151]: 2024-09-21T03:58:23.458+0000 7f7e9a0f6640 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2024-09-21T03:57:48.455067+0000 front 2024-09-21T03:57:48.455286+0000 (oldest deadline 2024-09-21T03:58:10.755147+0000) 2024-09-21T03:58:23.668 INFO:journalctl@ceph.osd.8.smithi161.stdout:Sep 21 03:58:23 smithi161 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-8[52151]: 2024-09-21T03:58:23.458+0000 7f7e9a0f6640 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-09-21T03:57:56.556595+0000 front 2024-09-21T03:57:56.556636+0000 (oldest deadline 2024-09-21T03:58:20.056711+0000) 2024-09-21T03:58:23.725 INFO:journalctl@ceph.osd.10.smithi150.stdout:Sep 21 03:58:23 smithi150 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-10[58206]: 2024-09-21T03:58:23.457+0000 7f34020e1640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-09-21T03:57:23.120033+0000 front 2024-09-21T03:57:23.120109+0000 (oldest deadline 2024-09-21T03:57:45.920434+0000) 2024-09-21T03:58:23.725 INFO:journalctl@ceph.osd.10.smithi150.stdout:Sep 21 03:58:23 smithi150 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-10[58206]: 2024-09-21T03:58:23.457+0000 7f34020e1640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-09-21T03:57:31.221107+0000 front 2024-09-21T03:57:31.221162+0000 (oldest deadline 2024-09-21T03:57:56.521240+0000) 2024-09-21T03:58:23.725 INFO:journalctl@ceph.osd.10.smithi150.stdout:Sep 21 03:58:23 smithi150 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-10[58206]: 2024-09-21T03:58:23.457+0000 7f34020e1640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-09-21T03:57:40.022210+0000 front 2024-09-21T03:57:40.022157+0000 (oldest deadline 2024-09-21T03:58:01.722244+0000) 2024-09-21T03:58:23.725 INFO:journalctl@ceph.osd.10.smithi150.stdout:Sep 21 03:58:23 smithi150 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-10[58206]: 2024-09-21T03:58:23.457+0000 7f34020e1640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2024-09-21T03:57:46.323919+0000 front 2024-09-21T03:57:46.323891+0000 (oldest deadline 2024-09-21T03:58:10.423788+0000) 2024-09-21T03:58:23.725 INFO:journalctl@ceph.osd.10.smithi150.stdout:Sep 21 03:58:23 smithi150 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-10[58206]: 2024-09-21T03:58:23.457+0000 7f34020e1640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-09-21T03:57:56.225398+0000 front 2024-09-21T03:57:56.225320+0000 (oldest deadline 2024-09-21T03:58:20.325491+0000) 2024-09-21T03:58:23.968 INFO:journalctl@ceph.osd.8.smithi161.stdout:Sep 21 03:58:23 smithi161 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-8[52151]: 2024-09-21T03:58:23.713+0000 7f7e9db0b640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.8 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-21T03:58:23.968 INFO:journalctl@ceph.osd.8.smithi161.stdout:Sep 21 03:58:23 smithi161 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-8[52151]: 2024-09-21T03:58:23.713+0000 7f7e9db0b640 -1 osd.8 90 *** Got signal Terminated *** 2024-09-21T03:58:23.968 INFO:journalctl@ceph.osd.8.smithi161.stdout:Sep 21 03:58:23 smithi161 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-8[52151]: 2024-09-21T03:58:23.713+0000 7f7e9db0b640 -1 osd.8 90 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-21T03:58:23.968 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 21 03:58:23 smithi161 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-11[58829]: 2024-09-21T03:58:23.911+0000 7f538ef4a640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-09-21T03:57:21.581303+0000 front 2024-09-21T03:57:21.581333+0000 (oldest deadline 2024-09-21T03:57:46.881367+0000) 2024-09-21T03:58:23.968 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 21 03:58:23 smithi161 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-11[58829]: 2024-09-21T03:58:23.911+0000 7f538ef4a640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-09-21T03:57:32.782524+0000 front 2024-09-21T03:57:32.782318+0000 (oldest deadline 2024-09-21T03:57:54.482443+0000) 2024-09-21T03:58:23.968 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 21 03:58:23 smithi161 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-11[58829]: 2024-09-21T03:58:23.911+0000 7f538ef4a640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-09-21T03:57:40.184584+0000 front 2024-09-21T03:57:40.184544+0000 (oldest deadline 2024-09-21T03:58:05.484668+0000) 2024-09-21T03:58:23.968 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 21 03:58:23 smithi161 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-11[58829]: 2024-09-21T03:58:23.911+0000 7f538ef4a640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-09-21T03:57:55.387018+0000 front 2024-09-21T03:57:55.386996+0000 (oldest deadline 2024-09-21T03:58:20.086771+0000) 2024-09-21T03:58:24.334 INFO:journalctl@ceph.osd.9.smithi039.stdout:Sep 21 03:58:23 smithi039 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-9[67006]: 2024-09-21T03:58:23.949+0000 7f4a138c4640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-09-21T03:57:22.331574+0000 front 2024-09-21T03:57:22.332146+0000 (oldest deadline 2024-09-21T03:57:45.831720+0000) 2024-09-21T03:58:24.334 INFO:journalctl@ceph.osd.9.smithi039.stdout:Sep 21 03:58:23 smithi039 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-9[67006]: 2024-09-21T03:58:23.949+0000 7f4a138c4640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-09-21T03:57:29.932951+0000 front 2024-09-21T03:57:29.932930+0000 (oldest deadline 2024-09-21T03:57:54.632848+0000) 2024-09-21T03:58:24.334 INFO:journalctl@ceph.osd.9.smithi039.stdout:Sep 21 03:58:23 smithi039 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-9[67006]: 2024-09-21T03:58:23.949+0000 7f4a138c4640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-09-21T03:57:40.434486+0000 front 2024-09-21T03:57:40.434450+0000 (oldest deadline 2024-09-21T03:58:02.734524+0000) 2024-09-21T03:58:24.334 INFO:journalctl@ceph.osd.9.smithi039.stdout:Sep 21 03:58:23 smithi039 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-9[67006]: 2024-09-21T03:58:23.949+0000 7f4a138c4640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2024-09-21T03:57:44.336000+0000 front 2024-09-21T03:57:44.336099+0000 (oldest deadline 2024-09-21T03:58:10.136509+0000) 2024-09-21T03:58:24.334 INFO:journalctl@ceph.osd.9.smithi039.stdout:Sep 21 03:58:23 smithi039 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-9[67006]: 2024-09-21T03:58:23.949+0000 7f4a138c4640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-09-21T03:57:56.037703+0000 front 2024-09-21T03:57:56.037728+0000 (oldest deadline 2024-09-21T03:58:19.537285+0000) 2024-09-21T03:58:24.725 INFO:journalctl@ceph.osd.10.smithi150.stdout:Sep 21 03:58:24 smithi150 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-10[58206]: 2024-09-21T03:58:24.419+0000 7f34020e1640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-09-21T03:57:23.120033+0000 front 2024-09-21T03:57:23.120109+0000 (oldest deadline 2024-09-21T03:57:45.920434+0000) 2024-09-21T03:58:24.725 INFO:journalctl@ceph.osd.10.smithi150.stdout:Sep 21 03:58:24 smithi150 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-10[58206]: 2024-09-21T03:58:24.419+0000 7f34020e1640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-09-21T03:57:31.221107+0000 front 2024-09-21T03:57:31.221162+0000 (oldest deadline 2024-09-21T03:57:56.521240+0000) 2024-09-21T03:58:24.725 INFO:journalctl@ceph.osd.10.smithi150.stdout:Sep 21 03:58:24 smithi150 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-10[58206]: 2024-09-21T03:58:24.419+0000 7f34020e1640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-09-21T03:57:40.022210+0000 front 2024-09-21T03:57:40.022157+0000 (oldest deadline 2024-09-21T03:58:01.722244+0000) 2024-09-21T03:58:24.725 INFO:journalctl@ceph.osd.10.smithi150.stdout:Sep 21 03:58:24 smithi150 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-10[58206]: 2024-09-21T03:58:24.419+0000 7f34020e1640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2024-09-21T03:57:46.323919+0000 front 2024-09-21T03:57:46.323891+0000 (oldest deadline 2024-09-21T03:58:10.423788+0000) 2024-09-21T03:58:24.725 INFO:journalctl@ceph.osd.10.smithi150.stdout:Sep 21 03:58:24 smithi150 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-10[58206]: 2024-09-21T03:58:24.419+0000 7f34020e1640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-09-21T03:57:56.225398+0000 front 2024-09-21T03:57:56.225320+0000 (oldest deadline 2024-09-21T03:58:20.325491+0000) 2024-09-21T03:58:24.909 INFO:journalctl@ceph.osd.8.smithi161.stdout:Sep 21 03:58:24 smithi161 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-8[52151]: 2024-09-21T03:58:24.489+0000 7f7e9a0f6640 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-09-21T03:57:29.951582+0000 front 2024-09-21T03:57:29.951481+0000 (oldest deadline 2024-09-21T03:57:54.051508+0000) 2024-09-21T03:58:24.909 INFO:journalctl@ceph.osd.8.smithi161.stdout:Sep 21 03:58:24 smithi161 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-8[52151]: 2024-09-21T03:58:24.489+0000 7f7e9a0f6640 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-09-21T03:57:39.753762+0000 front 2024-09-21T03:57:39.753690+0000 (oldest deadline 2024-09-21T03:58:04.453819+0000) 2024-09-21T03:58:24.910 INFO:journalctl@ceph.osd.8.smithi161.stdout:Sep 21 03:58:24 smithi161 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-8[52151]: 2024-09-21T03:58:24.489+0000 7f7e9a0f6640 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2024-09-21T03:57:48.455067+0000 front 2024-09-21T03:57:48.455286+0000 (oldest deadline 2024-09-21T03:58:10.755147+0000) 2024-09-21T03:58:24.910 INFO:journalctl@ceph.osd.8.smithi161.stdout:Sep 21 03:58:24 smithi161 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-8[52151]: 2024-09-21T03:58:24.489+0000 7f7e9a0f6640 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-09-21T03:57:56.556595+0000 front 2024-09-21T03:57:56.556636+0000 (oldest deadline 2024-09-21T03:58:20.056711+0000) 2024-09-21T03:58:25.218 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 21 03:58:24 smithi161 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-11[58829]: 2024-09-21T03:58:24.907+0000 7f538ef4a640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-09-21T03:57:21.581303+0000 front 2024-09-21T03:57:21.581333+0000 (oldest deadline 2024-09-21T03:57:46.881367+0000) 2024-09-21T03:58:25.218 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 21 03:58:24 smithi161 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-11[58829]: 2024-09-21T03:58:24.907+0000 7f538ef4a640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-09-21T03:57:32.782524+0000 front 2024-09-21T03:57:32.782318+0000 (oldest deadline 2024-09-21T03:57:54.482443+0000) 2024-09-21T03:58:25.218 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 21 03:58:24 smithi161 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-11[58829]: 2024-09-21T03:58:24.907+0000 7f538ef4a640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-09-21T03:57:40.184584+0000 front 2024-09-21T03:57:40.184544+0000 (oldest deadline 2024-09-21T03:58:05.484668+0000) 2024-09-21T03:58:25.218 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 21 03:58:24 smithi161 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-11[58829]: 2024-09-21T03:58:24.907+0000 7f538ef4a640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-09-21T03:57:55.387018+0000 front 2024-09-21T03:57:55.386996+0000 (oldest deadline 2024-09-21T03:58:20.086771+0000) 2024-09-21T03:58:25.334 INFO:journalctl@ceph.osd.9.smithi039.stdout:Sep 21 03:58:24 smithi039 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-9[67006]: 2024-09-21T03:58:24.970+0000 7f4a138c4640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-09-21T03:57:22.331574+0000 front 2024-09-21T03:57:22.332146+0000 (oldest deadline 2024-09-21T03:57:45.831720+0000) 2024-09-21T03:58:25.334 INFO:journalctl@ceph.osd.9.smithi039.stdout:Sep 21 03:58:24 smithi039 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-9[67006]: 2024-09-21T03:58:24.970+0000 7f4a138c4640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-09-21T03:57:29.932951+0000 front 2024-09-21T03:57:29.932930+0000 (oldest deadline 2024-09-21T03:57:54.632848+0000) 2024-09-21T03:58:25.334 INFO:journalctl@ceph.osd.9.smithi039.stdout:Sep 21 03:58:24 smithi039 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-9[67006]: 2024-09-21T03:58:24.970+0000 7f4a138c4640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-09-21T03:57:40.434486+0000 front 2024-09-21T03:57:40.434450+0000 (oldest deadline 2024-09-21T03:58:02.734524+0000) 2024-09-21T03:58:25.334 INFO:journalctl@ceph.osd.9.smithi039.stdout:Sep 21 03:58:24 smithi039 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-9[67006]: 2024-09-21T03:58:24.970+0000 7f4a138c4640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2024-09-21T03:57:44.336000+0000 front 2024-09-21T03:57:44.336099+0000 (oldest deadline 2024-09-21T03:58:10.136509+0000) 2024-09-21T03:58:25.334 INFO:journalctl@ceph.osd.9.smithi039.stdout:Sep 21 03:58:24 smithi039 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-9[67006]: 2024-09-21T03:58:24.970+0000 7f4a138c4640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-09-21T03:57:56.037703+0000 front 2024-09-21T03:57:56.037728+0000 (oldest deadline 2024-09-21T03:58:19.537285+0000) 2024-09-21T03:58:25.725 INFO:journalctl@ceph.osd.10.smithi150.stdout:Sep 21 03:58:25 smithi150 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-10[58206]: 2024-09-21T03:58:25.443+0000 7f34020e1640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-09-21T03:57:23.120033+0000 front 2024-09-21T03:57:23.120109+0000 (oldest deadline 2024-09-21T03:57:45.920434+0000) 2024-09-21T03:58:25.725 INFO:journalctl@ceph.osd.10.smithi150.stdout:Sep 21 03:58:25 smithi150 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-10[58206]: 2024-09-21T03:58:25.443+0000 7f34020e1640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-09-21T03:57:31.221107+0000 front 2024-09-21T03:57:31.221162+0000 (oldest deadline 2024-09-21T03:57:56.521240+0000) 2024-09-21T03:58:25.725 INFO:journalctl@ceph.osd.10.smithi150.stdout:Sep 21 03:58:25 smithi150 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-10[58206]: 2024-09-21T03:58:25.443+0000 7f34020e1640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-09-21T03:57:40.022210+0000 front 2024-09-21T03:57:40.022157+0000 (oldest deadline 2024-09-21T03:58:01.722244+0000) 2024-09-21T03:58:25.725 INFO:journalctl@ceph.osd.10.smithi150.stdout:Sep 21 03:58:25 smithi150 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-10[58206]: 2024-09-21T03:58:25.443+0000 7f34020e1640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2024-09-21T03:57:46.323919+0000 front 2024-09-21T03:57:46.323891+0000 (oldest deadline 2024-09-21T03:58:10.423788+0000) 2024-09-21T03:58:25.725 INFO:journalctl@ceph.osd.10.smithi150.stdout:Sep 21 03:58:25 smithi150 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-10[58206]: 2024-09-21T03:58:25.443+0000 7f34020e1640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-09-21T03:57:56.225398+0000 front 2024-09-21T03:57:56.225320+0000 (oldest deadline 2024-09-21T03:58:20.325491+0000) 2024-09-21T03:58:25.951 INFO:journalctl@ceph.osd.8.smithi161.stdout:Sep 21 03:58:25 smithi161 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-8[52151]: 2024-09-21T03:58:25.529+0000 7f7e9a0f6640 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-09-21T03:57:29.951582+0000 front 2024-09-21T03:57:29.951481+0000 (oldest deadline 2024-09-21T03:57:54.051508+0000) 2024-09-21T03:58:25.951 INFO:journalctl@ceph.osd.8.smithi161.stdout:Sep 21 03:58:25 smithi161 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-8[52151]: 2024-09-21T03:58:25.529+0000 7f7e9a0f6640 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-09-21T03:57:39.753762+0000 front 2024-09-21T03:57:39.753690+0000 (oldest deadline 2024-09-21T03:58:04.453819+0000) 2024-09-21T03:58:25.951 INFO:journalctl@ceph.osd.8.smithi161.stdout:Sep 21 03:58:25 smithi161 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-8[52151]: 2024-09-21T03:58:25.529+0000 7f7e9a0f6640 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2024-09-21T03:57:48.455067+0000 front 2024-09-21T03:57:48.455286+0000 (oldest deadline 2024-09-21T03:58:10.755147+0000) 2024-09-21T03:58:25.951 INFO:journalctl@ceph.osd.8.smithi161.stdout:Sep 21 03:58:25 smithi161 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-8[52151]: 2024-09-21T03:58:25.529+0000 7f7e9a0f6640 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-09-21T03:57:56.556595+0000 front 2024-09-21T03:57:56.556636+0000 (oldest deadline 2024-09-21T03:58:20.056711+0000) 2024-09-21T03:58:26.218 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 21 03:58:25 smithi161 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-11[58829]: 2024-09-21T03:58:25.949+0000 7f538ef4a640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-09-21T03:57:21.581303+0000 front 2024-09-21T03:57:21.581333+0000 (oldest deadline 2024-09-21T03:57:46.881367+0000) 2024-09-21T03:58:26.218 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 21 03:58:25 smithi161 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-11[58829]: 2024-09-21T03:58:25.949+0000 7f538ef4a640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-09-21T03:57:32.782524+0000 front 2024-09-21T03:57:32.782318+0000 (oldest deadline 2024-09-21T03:57:54.482443+0000) 2024-09-21T03:58:26.218 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 21 03:58:25 smithi161 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-11[58829]: 2024-09-21T03:58:25.949+0000 7f538ef4a640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-09-21T03:57:40.184584+0000 front 2024-09-21T03:57:40.184544+0000 (oldest deadline 2024-09-21T03:58:05.484668+0000) 2024-09-21T03:58:26.218 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 21 03:58:25 smithi161 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-11[58829]: 2024-09-21T03:58:25.949+0000 7f538ef4a640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-09-21T03:57:55.387018+0000 front 2024-09-21T03:57:55.386996+0000 (oldest deadline 2024-09-21T03:58:20.086771+0000) 2024-09-21T03:58:26.334 INFO:journalctl@ceph.osd.9.smithi039.stdout:Sep 21 03:58:25 smithi039 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-9[67006]: 2024-09-21T03:58:25.947+0000 7f4a138c4640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-09-21T03:57:22.331574+0000 front 2024-09-21T03:57:22.332146+0000 (oldest deadline 2024-09-21T03:57:45.831720+0000) 2024-09-21T03:58:26.334 INFO:journalctl@ceph.osd.9.smithi039.stdout:Sep 21 03:58:25 smithi039 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-9[67006]: 2024-09-21T03:58:25.947+0000 7f4a138c4640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-09-21T03:57:29.932951+0000 front 2024-09-21T03:57:29.932930+0000 (oldest deadline 2024-09-21T03:57:54.632848+0000) 2024-09-21T03:58:26.334 INFO:journalctl@ceph.osd.9.smithi039.stdout:Sep 21 03:58:25 smithi039 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-9[67006]: 2024-09-21T03:58:25.947+0000 7f4a138c4640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-09-21T03:57:40.434486+0000 front 2024-09-21T03:57:40.434450+0000 (oldest deadline 2024-09-21T03:58:02.734524+0000) 2024-09-21T03:58:26.334 INFO:journalctl@ceph.osd.9.smithi039.stdout:Sep 21 03:58:25 smithi039 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-9[67006]: 2024-09-21T03:58:25.947+0000 7f4a138c4640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2024-09-21T03:57:44.336000+0000 front 2024-09-21T03:57:44.336099+0000 (oldest deadline 2024-09-21T03:58:10.136509+0000) 2024-09-21T03:58:26.334 INFO:journalctl@ceph.osd.9.smithi039.stdout:Sep 21 03:58:25 smithi039 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-9[67006]: 2024-09-21T03:58:25.947+0000 7f4a138c4640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-09-21T03:57:56.037703+0000 front 2024-09-21T03:57:56.037728+0000 (oldest deadline 2024-09-21T03:58:19.537285+0000) 2024-09-21T03:58:26.725 INFO:journalctl@ceph.osd.10.smithi150.stdout:Sep 21 03:58:26 smithi150 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-10[58206]: 2024-09-21T03:58:26.405+0000 7f34020e1640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-09-21T03:57:23.120033+0000 front 2024-09-21T03:57:23.120109+0000 (oldest deadline 2024-09-21T03:57:45.920434+0000) 2024-09-21T03:58:26.725 INFO:journalctl@ceph.osd.10.smithi150.stdout:Sep 21 03:58:26 smithi150 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-10[58206]: 2024-09-21T03:58:26.405+0000 7f34020e1640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-09-21T03:57:31.221107+0000 front 2024-09-21T03:57:31.221162+0000 (oldest deadline 2024-09-21T03:57:56.521240+0000) 2024-09-21T03:58:26.725 INFO:journalctl@ceph.osd.10.smithi150.stdout:Sep 21 03:58:26 smithi150 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-10[58206]: 2024-09-21T03:58:26.405+0000 7f34020e1640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-09-21T03:57:40.022210+0000 front 2024-09-21T03:57:40.022157+0000 (oldest deadline 2024-09-21T03:58:01.722244+0000) 2024-09-21T03:58:26.725 INFO:journalctl@ceph.osd.10.smithi150.stdout:Sep 21 03:58:26 smithi150 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-10[58206]: 2024-09-21T03:58:26.405+0000 7f34020e1640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2024-09-21T03:57:46.323919+0000 front 2024-09-21T03:57:46.323891+0000 (oldest deadline 2024-09-21T03:58:10.423788+0000) 2024-09-21T03:58:26.725 INFO:journalctl@ceph.osd.10.smithi150.stdout:Sep 21 03:58:26 smithi150 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-10[58206]: 2024-09-21T03:58:26.405+0000 7f34020e1640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-09-21T03:57:56.225398+0000 front 2024-09-21T03:57:56.225320+0000 (oldest deadline 2024-09-21T03:58:20.325491+0000) 2024-09-21T03:58:26.725 INFO:journalctl@ceph.osd.10.smithi150.stdout:Sep 21 03:58:26 smithi150 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-10[58206]: 2024-09-21T03:58:26.405+0000 7f34020e1640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.161:6812 osd.5 since back 2024-09-21T03:58:03.727214+0000 front 2024-09-21T03:58:03.727152+0000 (oldest deadline 2024-09-21T03:58:26.027019+0000) 2024-09-21T03:58:26.968 INFO:journalctl@ceph.osd.8.smithi161.stdout:Sep 21 03:58:26 smithi161 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-8[52151]: 2024-09-21T03:58:26.535+0000 7f7e9a0f6640 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-09-21T03:57:29.951582+0000 front 2024-09-21T03:57:29.951481+0000 (oldest deadline 2024-09-21T03:57:54.051508+0000) 2024-09-21T03:58:26.968 INFO:journalctl@ceph.osd.8.smithi161.stdout:Sep 21 03:58:26 smithi161 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-8[52151]: 2024-09-21T03:58:26.535+0000 7f7e9a0f6640 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-09-21T03:57:39.753762+0000 front 2024-09-21T03:57:39.753690+0000 (oldest deadline 2024-09-21T03:58:04.453819+0000) 2024-09-21T03:58:26.968 INFO:journalctl@ceph.osd.8.smithi161.stdout:Sep 21 03:58:26 smithi161 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-8[52151]: 2024-09-21T03:58:26.535+0000 7f7e9a0f6640 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2024-09-21T03:57:48.455067+0000 front 2024-09-21T03:57:48.455286+0000 (oldest deadline 2024-09-21T03:58:10.755147+0000) 2024-09-21T03:58:26.968 INFO:journalctl@ceph.osd.8.smithi161.stdout:Sep 21 03:58:26 smithi161 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-8[52151]: 2024-09-21T03:58:26.535+0000 7f7e9a0f6640 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-09-21T03:57:56.556595+0000 front 2024-09-21T03:57:56.556636+0000 (oldest deadline 2024-09-21T03:58:20.056711+0000) 2024-09-21T03:58:26.968 INFO:journalctl@ceph.osd.8.smithi161.stdout:Sep 21 03:58:26 smithi161 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-8[52151]: 2024-09-21T03:58:26.535+0000 7f7e9a0f6640 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.161:6812 osd.5 since back 2024-09-21T03:58:04.157811+0000 front 2024-09-21T03:58:04.157861+0000 (oldest deadline 2024-09-21T03:58:25.857960+0000) 2024-09-21T03:58:27.084 INFO:journalctl@ceph.osd.9.smithi039.stdout:Sep 21 03:58:26 smithi039 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-9[67006]: 2024-09-21T03:58:26.916+0000 7f4a138c4640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-09-21T03:57:22.331574+0000 front 2024-09-21T03:57:22.332146+0000 (oldest deadline 2024-09-21T03:57:45.831720+0000) 2024-09-21T03:58:27.085 INFO:journalctl@ceph.osd.9.smithi039.stdout:Sep 21 03:58:26 smithi039 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-9[67006]: 2024-09-21T03:58:26.916+0000 7f4a138c4640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-09-21T03:57:29.932951+0000 front 2024-09-21T03:57:29.932930+0000 (oldest deadline 2024-09-21T03:57:54.632848+0000) 2024-09-21T03:58:27.085 INFO:journalctl@ceph.osd.9.smithi039.stdout:Sep 21 03:58:26 smithi039 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-9[67006]: 2024-09-21T03:58:26.916+0000 7f4a138c4640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-09-21T03:57:40.434486+0000 front 2024-09-21T03:57:40.434450+0000 (oldest deadline 2024-09-21T03:58:02.734524+0000) 2024-09-21T03:58:27.086 INFO:journalctl@ceph.osd.9.smithi039.stdout:Sep 21 03:58:26 smithi039 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-9[67006]: 2024-09-21T03:58:26.916+0000 7f4a138c4640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2024-09-21T03:57:44.336000+0000 front 2024-09-21T03:57:44.336099+0000 (oldest deadline 2024-09-21T03:58:10.136509+0000) 2024-09-21T03:58:27.086 INFO:journalctl@ceph.osd.9.smithi039.stdout:Sep 21 03:58:26 smithi039 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-9[67006]: 2024-09-21T03:58:26.916+0000 7f4a138c4640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-09-21T03:57:56.037703+0000 front 2024-09-21T03:57:56.037728+0000 (oldest deadline 2024-09-21T03:58:19.537285+0000) 2024-09-21T03:58:27.468 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 21 03:58:26 smithi161 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-11[58829]: 2024-09-21T03:58:26.981+0000 7f538ef4a640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-09-21T03:57:21.581303+0000 front 2024-09-21T03:57:21.581333+0000 (oldest deadline 2024-09-21T03:57:46.881367+0000) 2024-09-21T03:58:27.468 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 21 03:58:26 smithi161 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-11[58829]: 2024-09-21T03:58:26.981+0000 7f538ef4a640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-09-21T03:57:32.782524+0000 front 2024-09-21T03:57:32.782318+0000 (oldest deadline 2024-09-21T03:57:54.482443+0000) 2024-09-21T03:58:27.468 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 21 03:58:26 smithi161 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-11[58829]: 2024-09-21T03:58:26.981+0000 7f538ef4a640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-09-21T03:57:40.184584+0000 front 2024-09-21T03:57:40.184544+0000 (oldest deadline 2024-09-21T03:58:05.484668+0000) 2024-09-21T03:58:27.468 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 21 03:58:26 smithi161 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-11[58829]: 2024-09-21T03:58:26.981+0000 7f538ef4a640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-09-21T03:57:55.387018+0000 front 2024-09-21T03:57:55.386996+0000 (oldest deadline 2024-09-21T03:58:20.086771+0000) 2024-09-21T03:58:27.475 INFO:journalctl@ceph.osd.10.smithi150.stdout:Sep 21 03:58:27 smithi150 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-10[58206]: 2024-09-21T03:58:27.392+0000 7f34020e1640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-09-21T03:57:23.120033+0000 front 2024-09-21T03:57:23.120109+0000 (oldest deadline 2024-09-21T03:57:45.920434+0000) 2024-09-21T03:58:27.475 INFO:journalctl@ceph.osd.10.smithi150.stdout:Sep 21 03:58:27 smithi150 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-10[58206]: 2024-09-21T03:58:27.392+0000 7f34020e1640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-09-21T03:57:31.221107+0000 front 2024-09-21T03:57:31.221162+0000 (oldest deadline 2024-09-21T03:57:56.521240+0000) 2024-09-21T03:58:27.475 INFO:journalctl@ceph.osd.10.smithi150.stdout:Sep 21 03:58:27 smithi150 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-10[58206]: 2024-09-21T03:58:27.392+0000 7f34020e1640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-09-21T03:57:40.022210+0000 front 2024-09-21T03:57:40.022157+0000 (oldest deadline 2024-09-21T03:58:01.722244+0000) 2024-09-21T03:58:27.475 INFO:journalctl@ceph.osd.10.smithi150.stdout:Sep 21 03:58:27 smithi150 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-10[58206]: 2024-09-21T03:58:27.392+0000 7f34020e1640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2024-09-21T03:57:46.323919+0000 front 2024-09-21T03:57:46.323891+0000 (oldest deadline 2024-09-21T03:58:10.423788+0000) 2024-09-21T03:58:27.475 INFO:journalctl@ceph.osd.10.smithi150.stdout:Sep 21 03:58:27 smithi150 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-10[58206]: 2024-09-21T03:58:27.392+0000 7f34020e1640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-09-21T03:57:56.225398+0000 front 2024-09-21T03:57:56.225320+0000 (oldest deadline 2024-09-21T03:58:20.325491+0000) 2024-09-21T03:58:27.475 INFO:journalctl@ceph.osd.10.smithi150.stdout:Sep 21 03:58:27 smithi150 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-10[58206]: 2024-09-21T03:58:27.392+0000 7f34020e1640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.161:6812 osd.5 since back 2024-09-21T03:58:03.727214+0000 front 2024-09-21T03:58:03.727152+0000 (oldest deadline 2024-09-21T03:58:26.027019+0000) 2024-09-21T03:58:27.968 INFO:journalctl@ceph.osd.8.smithi161.stdout:Sep 21 03:58:27 smithi161 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-8[52151]: 2024-09-21T03:58:27.536+0000 7f7e9a0f6640 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-09-21T03:57:29.951582+0000 front 2024-09-21T03:57:29.951481+0000 (oldest deadline 2024-09-21T03:57:54.051508+0000) 2024-09-21T03:58:27.968 INFO:journalctl@ceph.osd.8.smithi161.stdout:Sep 21 03:58:27 smithi161 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-8[52151]: 2024-09-21T03:58:27.536+0000 7f7e9a0f6640 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-09-21T03:57:39.753762+0000 front 2024-09-21T03:57:39.753690+0000 (oldest deadline 2024-09-21T03:58:04.453819+0000) 2024-09-21T03:58:27.968 INFO:journalctl@ceph.osd.8.smithi161.stdout:Sep 21 03:58:27 smithi161 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-8[52151]: 2024-09-21T03:58:27.536+0000 7f7e9a0f6640 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2024-09-21T03:57:48.455067+0000 front 2024-09-21T03:57:48.455286+0000 (oldest deadline 2024-09-21T03:58:10.755147+0000) 2024-09-21T03:58:27.968 INFO:journalctl@ceph.osd.8.smithi161.stdout:Sep 21 03:58:27 smithi161 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-8[52151]: 2024-09-21T03:58:27.536+0000 7f7e9a0f6640 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-09-21T03:57:56.556595+0000 front 2024-09-21T03:57:56.556636+0000 (oldest deadline 2024-09-21T03:58:20.056711+0000) 2024-09-21T03:58:27.968 INFO:journalctl@ceph.osd.8.smithi161.stdout:Sep 21 03:58:27 smithi161 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-8[52151]: 2024-09-21T03:58:27.536+0000 7f7e9a0f6640 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.161:6812 osd.5 since back 2024-09-21T03:58:04.157811+0000 front 2024-09-21T03:58:04.157861+0000 (oldest deadline 2024-09-21T03:58:25.857960+0000) 2024-09-21T03:58:28.334 INFO:journalctl@ceph.osd.9.smithi039.stdout:Sep 21 03:58:27 smithi039 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-9[67006]: 2024-09-21T03:58:27.964+0000 7f4a138c4640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-09-21T03:57:22.331574+0000 front 2024-09-21T03:57:22.332146+0000 (oldest deadline 2024-09-21T03:57:45.831720+0000) 2024-09-21T03:58:28.334 INFO:journalctl@ceph.osd.9.smithi039.stdout:Sep 21 03:58:27 smithi039 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-9[67006]: 2024-09-21T03:58:27.964+0000 7f4a138c4640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-09-21T03:57:29.932951+0000 front 2024-09-21T03:57:29.932930+0000 (oldest deadline 2024-09-21T03:57:54.632848+0000) 2024-09-21T03:58:28.334 INFO:journalctl@ceph.osd.9.smithi039.stdout:Sep 21 03:58:27 smithi039 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-9[67006]: 2024-09-21T03:58:27.964+0000 7f4a138c4640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-09-21T03:57:40.434486+0000 front 2024-09-21T03:57:40.434450+0000 (oldest deadline 2024-09-21T03:58:02.734524+0000) 2024-09-21T03:58:28.334 INFO:journalctl@ceph.osd.9.smithi039.stdout:Sep 21 03:58:27 smithi039 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-9[67006]: 2024-09-21T03:58:27.964+0000 7f4a138c4640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2024-09-21T03:57:44.336000+0000 front 2024-09-21T03:57:44.336099+0000 (oldest deadline 2024-09-21T03:58:10.136509+0000) 2024-09-21T03:58:28.334 INFO:journalctl@ceph.osd.9.smithi039.stdout:Sep 21 03:58:27 smithi039 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-9[67006]: 2024-09-21T03:58:27.964+0000 7f4a138c4640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-09-21T03:57:56.037703+0000 front 2024-09-21T03:57:56.037728+0000 (oldest deadline 2024-09-21T03:58:19.537285+0000) 2024-09-21T03:58:28.334 INFO:journalctl@ceph.osd.9.smithi039.stdout:Sep 21 03:58:27 smithi039 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-9[67006]: 2024-09-21T03:58:27.964+0000 7f4a138c4640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.161:6812 osd.5 since back 2024-09-21T03:58:03.438999+0000 front 2024-09-21T03:58:03.439023+0000 (oldest deadline 2024-09-21T03:58:26.938893+0000) 2024-09-21T03:58:28.468 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 21 03:58:28 smithi161 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-11[58829]: 2024-09-21T03:58:28.026+0000 7f538ef4a640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-09-21T03:57:21.581303+0000 front 2024-09-21T03:57:21.581333+0000 (oldest deadline 2024-09-21T03:57:46.881367+0000) 2024-09-21T03:58:28.468 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 21 03:58:28 smithi161 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-11[58829]: 2024-09-21T03:58:28.026+0000 7f538ef4a640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-09-21T03:57:32.782524+0000 front 2024-09-21T03:57:32.782318+0000 (oldest deadline 2024-09-21T03:57:54.482443+0000) 2024-09-21T03:58:28.468 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 21 03:58:28 smithi161 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-11[58829]: 2024-09-21T03:58:28.026+0000 7f538ef4a640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-09-21T03:57:40.184584+0000 front 2024-09-21T03:57:40.184544+0000 (oldest deadline 2024-09-21T03:58:05.484668+0000) 2024-09-21T03:58:28.468 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 21 03:58:28 smithi161 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-11[58829]: 2024-09-21T03:58:28.026+0000 7f538ef4a640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-09-21T03:57:55.387018+0000 front 2024-09-21T03:57:55.386996+0000 (oldest deadline 2024-09-21T03:58:20.086771+0000) 2024-09-21T03:58:28.468 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 21 03:58:28 smithi161 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-11[58829]: 2024-09-21T03:58:28.026+0000 7f538ef4a640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.161:6812 osd.5 since back 2024-09-21T03:58:02.887862+0000 front 2024-09-21T03:58:02.887979+0000 (oldest deadline 2024-09-21T03:58:27.587929+0000) 2024-09-21T03:58:28.725 INFO:journalctl@ceph.osd.10.smithi150.stdout:Sep 21 03:58:28 smithi150 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-10[58206]: 2024-09-21T03:58:28.412+0000 7f34020e1640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-09-21T03:57:23.120033+0000 front 2024-09-21T03:57:23.120109+0000 (oldest deadline 2024-09-21T03:57:45.920434+0000) 2024-09-21T03:58:28.725 INFO:journalctl@ceph.osd.10.smithi150.stdout:Sep 21 03:58:28 smithi150 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-10[58206]: 2024-09-21T03:58:28.412+0000 7f34020e1640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-09-21T03:57:31.221107+0000 front 2024-09-21T03:57:31.221162+0000 (oldest deadline 2024-09-21T03:57:56.521240+0000) 2024-09-21T03:58:28.725 INFO:journalctl@ceph.osd.10.smithi150.stdout:Sep 21 03:58:28 smithi150 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-10[58206]: 2024-09-21T03:58:28.412+0000 7f34020e1640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-09-21T03:57:40.022210+0000 front 2024-09-21T03:57:40.022157+0000 (oldest deadline 2024-09-21T03:58:01.722244+0000) 2024-09-21T03:58:28.725 INFO:journalctl@ceph.osd.10.smithi150.stdout:Sep 21 03:58:28 smithi150 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-10[58206]: 2024-09-21T03:58:28.412+0000 7f34020e1640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2024-09-21T03:57:46.323919+0000 front 2024-09-21T03:57:46.323891+0000 (oldest deadline 2024-09-21T03:58:10.423788+0000) 2024-09-21T03:58:28.725 INFO:journalctl@ceph.osd.10.smithi150.stdout:Sep 21 03:58:28 smithi150 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-10[58206]: 2024-09-21T03:58:28.412+0000 7f34020e1640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-09-21T03:57:56.225398+0000 front 2024-09-21T03:57:56.225320+0000 (oldest deadline 2024-09-21T03:58:20.325491+0000) 2024-09-21T03:58:28.725 INFO:journalctl@ceph.osd.10.smithi150.stdout:Sep 21 03:58:28 smithi150 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-10[58206]: 2024-09-21T03:58:28.412+0000 7f34020e1640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.161:6812 osd.5 since back 2024-09-21T03:58:03.727214+0000 front 2024-09-21T03:58:03.727152+0000 (oldest deadline 2024-09-21T03:58:26.027019+0000) 2024-09-21T03:58:28.968 INFO:journalctl@ceph.osd.8.smithi161.stdout:Sep 21 03:58:28 smithi161 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-8[52151]: 2024-09-21T03:58:28.578+0000 7f7e9a0f6640 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-09-21T03:57:29.951582+0000 front 2024-09-21T03:57:29.951481+0000 (oldest deadline 2024-09-21T03:57:54.051508+0000) 2024-09-21T03:58:28.968 INFO:journalctl@ceph.osd.8.smithi161.stdout:Sep 21 03:58:28 smithi161 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-8[52151]: 2024-09-21T03:58:28.578+0000 7f7e9a0f6640 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-09-21T03:57:39.753762+0000 front 2024-09-21T03:57:39.753690+0000 (oldest deadline 2024-09-21T03:58:04.453819+0000) 2024-09-21T03:58:28.968 INFO:journalctl@ceph.osd.8.smithi161.stdout:Sep 21 03:58:28 smithi161 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-8[52151]: 2024-09-21T03:58:28.578+0000 7f7e9a0f6640 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2024-09-21T03:57:48.455067+0000 front 2024-09-21T03:57:48.455286+0000 (oldest deadline 2024-09-21T03:58:10.755147+0000) 2024-09-21T03:58:28.968 INFO:journalctl@ceph.osd.8.smithi161.stdout:Sep 21 03:58:28 smithi161 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-8[52151]: 2024-09-21T03:58:28.578+0000 7f7e9a0f6640 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-09-21T03:57:56.556595+0000 front 2024-09-21T03:57:56.556636+0000 (oldest deadline 2024-09-21T03:58:20.056711+0000) 2024-09-21T03:58:28.968 INFO:journalctl@ceph.osd.8.smithi161.stdout:Sep 21 03:58:28 smithi161 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-8[52151]: 2024-09-21T03:58:28.578+0000 7f7e9a0f6640 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.161:6812 osd.5 since back 2024-09-21T03:58:04.157811+0000 front 2024-09-21T03:58:04.157861+0000 (oldest deadline 2024-09-21T03:58:25.857960+0000) 2024-09-21T03:58:29.334 INFO:journalctl@ceph.osd.9.smithi039.stdout:Sep 21 03:58:29 smithi039 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-9[67006]: 2024-09-21T03:58:29.012+0000 7f4a138c4640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-09-21T03:57:22.331574+0000 front 2024-09-21T03:57:22.332146+0000 (oldest deadline 2024-09-21T03:57:45.831720+0000) 2024-09-21T03:58:29.334 INFO:journalctl@ceph.osd.9.smithi039.stdout:Sep 21 03:58:29 smithi039 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-9[67006]: 2024-09-21T03:58:29.012+0000 7f4a138c4640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-09-21T03:57:29.932951+0000 front 2024-09-21T03:57:29.932930+0000 (oldest deadline 2024-09-21T03:57:54.632848+0000) 2024-09-21T03:58:29.334 INFO:journalctl@ceph.osd.9.smithi039.stdout:Sep 21 03:58:29 smithi039 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-9[67006]: 2024-09-21T03:58:29.012+0000 7f4a138c4640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-09-21T03:57:40.434486+0000 front 2024-09-21T03:57:40.434450+0000 (oldest deadline 2024-09-21T03:58:02.734524+0000) 2024-09-21T03:58:29.334 INFO:journalctl@ceph.osd.9.smithi039.stdout:Sep 21 03:58:29 smithi039 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-9[67006]: 2024-09-21T03:58:29.012+0000 7f4a138c4640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2024-09-21T03:57:44.336000+0000 front 2024-09-21T03:57:44.336099+0000 (oldest deadline 2024-09-21T03:58:10.136509+0000) 2024-09-21T03:58:29.334 INFO:journalctl@ceph.osd.9.smithi039.stdout:Sep 21 03:58:29 smithi039 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-9[67006]: 2024-09-21T03:58:29.012+0000 7f4a138c4640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-09-21T03:57:56.037703+0000 front 2024-09-21T03:57:56.037728+0000 (oldest deadline 2024-09-21T03:58:19.537285+0000) 2024-09-21T03:58:29.334 INFO:journalctl@ceph.osd.9.smithi039.stdout:Sep 21 03:58:29 smithi039 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-9[67006]: 2024-09-21T03:58:29.012+0000 7f4a138c4640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.161:6812 osd.5 since back 2024-09-21T03:58:03.438999+0000 front 2024-09-21T03:58:03.439023+0000 (oldest deadline 2024-09-21T03:58:26.938893+0000) 2024-09-21T03:58:29.468 INFO:journalctl@ceph.osd.8.smithi161.stdout:Sep 21 03:58:29 smithi161 podman[67945]: 2024-09-21 03:58:29.186431251 +0000 UTC m=+5.618681317 container died 6f8b1dd488571f102f6dd97c68d3f32cbc96a0bd4ff884321a6c0f16f3281e07 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-8, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, ceph=True, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.vendor=CentOS, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0) 2024-09-21T03:58:29.657 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 21 03:58:28 smithi161 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-11[58829]: 2024-09-21T03:58:28.977+0000 7f538ef4a640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-09-21T03:57:21.581303+0000 front 2024-09-21T03:57:21.581333+0000 (oldest deadline 2024-09-21T03:57:46.881367+0000) 2024-09-21T03:58:29.657 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 21 03:58:28 smithi161 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-11[58829]: 2024-09-21T03:58:28.977+0000 7f538ef4a640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-09-21T03:57:32.782524+0000 front 2024-09-21T03:57:32.782318+0000 (oldest deadline 2024-09-21T03:57:54.482443+0000) 2024-09-21T03:58:29.657 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 21 03:58:28 smithi161 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-11[58829]: 2024-09-21T03:58:28.977+0000 7f538ef4a640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-09-21T03:57:40.184584+0000 front 2024-09-21T03:57:40.184544+0000 (oldest deadline 2024-09-21T03:58:05.484668+0000) 2024-09-21T03:58:29.657 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 21 03:58:28 smithi161 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-11[58829]: 2024-09-21T03:58:28.977+0000 7f538ef4a640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-09-21T03:57:55.387018+0000 front 2024-09-21T03:57:55.386996+0000 (oldest deadline 2024-09-21T03:58:20.086771+0000) 2024-09-21T03:58:29.657 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 21 03:58:28 smithi161 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-11[58829]: 2024-09-21T03:58:28.977+0000 7f538ef4a640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.161:6812 osd.5 since back 2024-09-21T03:58:02.887862+0000 front 2024-09-21T03:58:02.887979+0000 (oldest deadline 2024-09-21T03:58:27.587929+0000) 2024-09-21T03:58:29.725 INFO:journalctl@ceph.osd.10.smithi150.stdout:Sep 21 03:58:29 smithi150 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-10[58206]: 2024-09-21T03:58:29.378+0000 7f34020e1640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-09-21T03:57:23.120033+0000 front 2024-09-21T03:57:23.120109+0000 (oldest deadline 2024-09-21T03:57:45.920434+0000) 2024-09-21T03:58:29.725 INFO:journalctl@ceph.osd.10.smithi150.stdout:Sep 21 03:58:29 smithi150 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-10[58206]: 2024-09-21T03:58:29.378+0000 7f34020e1640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-09-21T03:57:31.221107+0000 front 2024-09-21T03:57:31.221162+0000 (oldest deadline 2024-09-21T03:57:56.521240+0000) 2024-09-21T03:58:29.725 INFO:journalctl@ceph.osd.10.smithi150.stdout:Sep 21 03:58:29 smithi150 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-10[58206]: 2024-09-21T03:58:29.378+0000 7f34020e1640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-09-21T03:57:40.022210+0000 front 2024-09-21T03:57:40.022157+0000 (oldest deadline 2024-09-21T03:58:01.722244+0000) 2024-09-21T03:58:29.725 INFO:journalctl@ceph.osd.10.smithi150.stdout:Sep 21 03:58:29 smithi150 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-10[58206]: 2024-09-21T03:58:29.378+0000 7f34020e1640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2024-09-21T03:57:46.323919+0000 front 2024-09-21T03:57:46.323891+0000 (oldest deadline 2024-09-21T03:58:10.423788+0000) 2024-09-21T03:58:29.725 INFO:journalctl@ceph.osd.10.smithi150.stdout:Sep 21 03:58:29 smithi150 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-10[58206]: 2024-09-21T03:58:29.378+0000 7f34020e1640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-09-21T03:57:56.225398+0000 front 2024-09-21T03:57:56.225320+0000 (oldest deadline 2024-09-21T03:58:20.325491+0000) 2024-09-21T03:58:29.725 INFO:journalctl@ceph.osd.10.smithi150.stdout:Sep 21 03:58:29 smithi150 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-10[58206]: 2024-09-21T03:58:29.378+0000 7f34020e1640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.161:6812 osd.5 since back 2024-09-21T03:58:03.727214+0000 front 2024-09-21T03:58:03.727152+0000 (oldest deadline 2024-09-21T03:58:26.027019+0000) 2024-09-21T03:58:29.823 INFO:journalctl@ceph.osd.8.smithi161.stdout:Sep 21 03:58:29 smithi161 podman[67945]: 2024-09-21 03:58:29.554705711 +0000 UTC m=+5.986955778 container cleanup 6f8b1dd488571f102f6dd97c68d3f32cbc96a0bd4ff884321a6c0f16f3281e07 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-8, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2) 2024-09-21T03:58:29.824 INFO:journalctl@ceph.osd.8.smithi161.stdout:Sep 21 03:58:29 smithi161 bash[67945]: ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-8 2024-09-21T03:58:30.177 INFO:journalctl@ceph.osd.8.smithi161.stdout:Sep 21 03:58:29 smithi161 podman[67960]: 2024-09-21 03:58:29.822267589 +0000 UTC m=+0.633537408 container remove 6f8b1dd488571f102f6dd97c68d3f32cbc96a0bd4ff884321a6c0f16f3281e07 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-8, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, ceph=True, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux ) 2024-09-21T03:58:30.178 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 21 03:58:29 smithi161 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-11[58829]: 2024-09-21T03:58:29.931+0000 7f538ef4a640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-09-21T03:57:21.581303+0000 front 2024-09-21T03:57:21.581333+0000 (oldest deadline 2024-09-21T03:57:46.881367+0000) 2024-09-21T03:58:30.179 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 21 03:58:29 smithi161 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-11[58829]: 2024-09-21T03:58:29.931+0000 7f538ef4a640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-09-21T03:57:32.782524+0000 front 2024-09-21T03:57:32.782318+0000 (oldest deadline 2024-09-21T03:57:54.482443+0000) 2024-09-21T03:58:30.179 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 21 03:58:29 smithi161 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-11[58829]: 2024-09-21T03:58:29.931+0000 7f538ef4a640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-09-21T03:57:40.184584+0000 front 2024-09-21T03:57:40.184544+0000 (oldest deadline 2024-09-21T03:58:05.484668+0000) 2024-09-21T03:58:30.179 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 21 03:58:29 smithi161 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-11[58829]: 2024-09-21T03:58:29.931+0000 7f538ef4a640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-09-21T03:57:55.387018+0000 front 2024-09-21T03:57:55.386996+0000 (oldest deadline 2024-09-21T03:58:20.086771+0000) 2024-09-21T03:58:30.180 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 21 03:58:29 smithi161 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-11[58829]: 2024-09-21T03:58:29.931+0000 7f538ef4a640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.161:6812 osd.5 since back 2024-09-21T03:58:02.887862+0000 front 2024-09-21T03:58:02.887979+0000 (oldest deadline 2024-09-21T03:58:27.587929+0000) 2024-09-21T03:58:30.334 INFO:journalctl@ceph.osd.9.smithi039.stdout:Sep 21 03:58:29 smithi039 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-9[67006]: 2024-09-21T03:58:29.973+0000 7f4a138c4640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-09-21T03:57:22.331574+0000 front 2024-09-21T03:57:22.332146+0000 (oldest deadline 2024-09-21T03:57:45.831720+0000) 2024-09-21T03:58:30.334 INFO:journalctl@ceph.osd.9.smithi039.stdout:Sep 21 03:58:29 smithi039 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-9[67006]: 2024-09-21T03:58:29.973+0000 7f4a138c4640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-09-21T03:57:29.932951+0000 front 2024-09-21T03:57:29.932930+0000 (oldest deadline 2024-09-21T03:57:54.632848+0000) 2024-09-21T03:58:30.334 INFO:journalctl@ceph.osd.9.smithi039.stdout:Sep 21 03:58:29 smithi039 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-9[67006]: 2024-09-21T03:58:29.973+0000 7f4a138c4640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-09-21T03:57:40.434486+0000 front 2024-09-21T03:57:40.434450+0000 (oldest deadline 2024-09-21T03:58:02.734524+0000) 2024-09-21T03:58:30.335 INFO:journalctl@ceph.osd.9.smithi039.stdout:Sep 21 03:58:29 smithi039 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-9[67006]: 2024-09-21T03:58:29.973+0000 7f4a138c4640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2024-09-21T03:57:44.336000+0000 front 2024-09-21T03:57:44.336099+0000 (oldest deadline 2024-09-21T03:58:10.136509+0000) 2024-09-21T03:58:30.335 INFO:journalctl@ceph.osd.9.smithi039.stdout:Sep 21 03:58:29 smithi039 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-9[67006]: 2024-09-21T03:58:29.973+0000 7f4a138c4640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-09-21T03:57:56.037703+0000 front 2024-09-21T03:57:56.037728+0000 (oldest deadline 2024-09-21T03:58:19.537285+0000) 2024-09-21T03:58:30.335 INFO:journalctl@ceph.osd.9.smithi039.stdout:Sep 21 03:58:29 smithi039 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-9[67006]: 2024-09-21T03:58:29.973+0000 7f4a138c4640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.161:6812 osd.5 since back 2024-09-21T03:58:03.438999+0000 front 2024-09-21T03:58:03.439023+0000 (oldest deadline 2024-09-21T03:58:26.938893+0000) 2024-09-21T03:58:30.443 INFO:journalctl@ceph.osd.8.smithi161.stdout:Sep 21 03:58:30 smithi161 podman[68051]: 2024-09-21 03:58:30.078229767 +0000 UTC m=+0.016139066 image pull ddce307e1f3263fb077b4f3902e15660050631dad4d2823a20ee3f72bf18d977 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262 2024-09-21T03:58:30.443 INFO:journalctl@ceph.osd.8.smithi161.stdout:Sep 21 03:58:30 smithi161 podman[68051]: 2024-09-21 03:58:30.21477186 +0000 UTC m=+0.152681147 container create ca4e2a17ce84361efd0e4e6a37dc682a168b44ddc214ccbd4ad1695301853f5f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-8-deactivate, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23) 2024-09-21T03:58:30.718 INFO:journalctl@ceph.osd.8.smithi161.stdout:Sep 21 03:58:30 smithi161 podman[68051]: 2024-09-21 03:58:30.442236315 +0000 UTC m=+0.380145613 container init ca4e2a17ce84361efd0e4e6a37dc682a168b44ddc214ccbd4ad1695301853f5f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-8-deactivate, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, ceph=True, GIT_CLEAN=True, org.label-schema.build-date=20240731, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD) 2024-09-21T03:58:30.718 INFO:journalctl@ceph.osd.8.smithi161.stdout:Sep 21 03:58:30 smithi161 podman[68051]: 2024-09-21 03:58:30.445665929 +0000 UTC m=+0.383575220 container start ca4e2a17ce84361efd0e4e6a37dc682a168b44ddc214ccbd4ad1695301853f5f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-8-deactivate, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, org.label-schema.build-date=20240731, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD) 2024-09-21T03:58:30.718 INFO:journalctl@ceph.osd.8.smithi161.stdout:Sep 21 03:58:30 smithi161 podman[68051]: 2024-09-21 03:58:30.533645386 +0000 UTC m=+0.471554689 container attach ca4e2a17ce84361efd0e4e6a37dc682a168b44ddc214ccbd4ad1695301853f5f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-8-deactivate, org.label-schema.vendor=CentOS, ceph=True, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, io.buildah.version=1.37.2, GIT_CLEAN=True, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2) 2024-09-21T03:58:30.719 INFO:journalctl@ceph.osd.8.smithi161.stdout:Sep 21 03:58:30 smithi161 podman[68051]: 2024-09-21 03:58:30.63028391 +0000 UTC m=+0.568193207 container died ca4e2a17ce84361efd0e4e6a37dc682a168b44ddc214ccbd4ad1695301853f5f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-8-deactivate, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, io.buildah.version=1.37.2, ceph=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23) 2024-09-21T03:58:30.725 INFO:journalctl@ceph.osd.10.smithi150.stdout:Sep 21 03:58:30 smithi150 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-10[58206]: 2024-09-21T03:58:30.404+0000 7f34020e1640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-09-21T03:57:23.120033+0000 front 2024-09-21T03:57:23.120109+0000 (oldest deadline 2024-09-21T03:57:45.920434+0000) 2024-09-21T03:58:30.725 INFO:journalctl@ceph.osd.10.smithi150.stdout:Sep 21 03:58:30 smithi150 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-10[58206]: 2024-09-21T03:58:30.404+0000 7f34020e1640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-09-21T03:57:31.221107+0000 front 2024-09-21T03:57:31.221162+0000 (oldest deadline 2024-09-21T03:57:56.521240+0000) 2024-09-21T03:58:30.725 INFO:journalctl@ceph.osd.10.smithi150.stdout:Sep 21 03:58:30 smithi150 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-10[58206]: 2024-09-21T03:58:30.404+0000 7f34020e1640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-09-21T03:57:40.022210+0000 front 2024-09-21T03:57:40.022157+0000 (oldest deadline 2024-09-21T03:58:01.722244+0000) 2024-09-21T03:58:30.725 INFO:journalctl@ceph.osd.10.smithi150.stdout:Sep 21 03:58:30 smithi150 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-10[58206]: 2024-09-21T03:58:30.404+0000 7f34020e1640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2024-09-21T03:57:46.323919+0000 front 2024-09-21T03:57:46.323891+0000 (oldest deadline 2024-09-21T03:58:10.423788+0000) 2024-09-21T03:58:30.725 INFO:journalctl@ceph.osd.10.smithi150.stdout:Sep 21 03:58:30 smithi150 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-10[58206]: 2024-09-21T03:58:30.404+0000 7f34020e1640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-09-21T03:57:56.225398+0000 front 2024-09-21T03:57:56.225320+0000 (oldest deadline 2024-09-21T03:58:20.325491+0000) 2024-09-21T03:58:30.725 INFO:journalctl@ceph.osd.10.smithi150.stdout:Sep 21 03:58:30 smithi150 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-10[58206]: 2024-09-21T03:58:30.404+0000 7f34020e1640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.161:6812 osd.5 since back 2024-09-21T03:58:03.727214+0000 front 2024-09-21T03:58:03.727152+0000 (oldest deadline 2024-09-21T03:58:26.027019+0000) 2024-09-21T03:58:31.197 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 21 03:58:30 smithi161 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-11[58829]: 2024-09-21T03:58:30.926+0000 7f538ef4a640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-09-21T03:57:21.581303+0000 front 2024-09-21T03:57:21.581333+0000 (oldest deadline 2024-09-21T03:57:46.881367+0000) 2024-09-21T03:58:31.197 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 21 03:58:30 smithi161 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-11[58829]: 2024-09-21T03:58:30.926+0000 7f538ef4a640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-09-21T03:57:32.782524+0000 front 2024-09-21T03:57:32.782318+0000 (oldest deadline 2024-09-21T03:57:54.482443+0000) 2024-09-21T03:58:31.197 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 21 03:58:30 smithi161 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-11[58829]: 2024-09-21T03:58:30.926+0000 7f538ef4a640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-09-21T03:57:40.184584+0000 front 2024-09-21T03:57:40.184544+0000 (oldest deadline 2024-09-21T03:58:05.484668+0000) 2024-09-21T03:58:31.197 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 21 03:58:30 smithi161 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-11[58829]: 2024-09-21T03:58:30.926+0000 7f538ef4a640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-09-21T03:57:55.387018+0000 front 2024-09-21T03:57:55.386996+0000 (oldest deadline 2024-09-21T03:58:20.086771+0000) 2024-09-21T03:58:31.197 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 21 03:58:30 smithi161 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-11[58829]: 2024-09-21T03:58:30.926+0000 7f538ef4a640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.161:6812 osd.5 since back 2024-09-21T03:58:02.887862+0000 front 2024-09-21T03:58:02.887979+0000 (oldest deadline 2024-09-21T03:58:27.587929+0000) 2024-09-21T03:58:31.221 DEBUG:teuthology.orchestra.run.smithi161:> sudo pkill -f 'journalctl -f -n 0 -u ceph-51116d5c-77ca-11ef-baf5-efdc52797490@osd.8.service' 2024-09-21T03:58:31.262 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-21T03:58:31.262 INFO:tasks.cephadm.osd.8:Stopped osd.8 2024-09-21T03:58:31.263 INFO:tasks.cephadm.osd.9:Stopping osd.9... 2024-09-21T03:58:31.263 DEBUG:teuthology.orchestra.run.smithi039:> sudo systemctl stop ceph-51116d5c-77ca-11ef-baf5-efdc52797490@osd.9 2024-09-21T03:58:31.292 INFO:journalctl@ceph.osd.9.smithi039.stdout:Sep 21 03:58:30 smithi039 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-9[67006]: 2024-09-21T03:58:30.932+0000 7f4a138c4640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-09-21T03:57:22.331574+0000 front 2024-09-21T03:57:22.332146+0000 (oldest deadline 2024-09-21T03:57:45.831720+0000) 2024-09-21T03:58:31.293 INFO:journalctl@ceph.osd.9.smithi039.stdout:Sep 21 03:58:30 smithi039 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-9[67006]: 2024-09-21T03:58:30.932+0000 7f4a138c4640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-09-21T03:57:29.932951+0000 front 2024-09-21T03:57:29.932930+0000 (oldest deadline 2024-09-21T03:57:54.632848+0000) 2024-09-21T03:58:31.293 INFO:journalctl@ceph.osd.9.smithi039.stdout:Sep 21 03:58:30 smithi039 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-9[67006]: 2024-09-21T03:58:30.932+0000 7f4a138c4640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-09-21T03:57:40.434486+0000 front 2024-09-21T03:57:40.434450+0000 (oldest deadline 2024-09-21T03:58:02.734524+0000) 2024-09-21T03:58:31.293 INFO:journalctl@ceph.osd.9.smithi039.stdout:Sep 21 03:58:30 smithi039 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-9[67006]: 2024-09-21T03:58:30.932+0000 7f4a138c4640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2024-09-21T03:57:44.336000+0000 front 2024-09-21T03:57:44.336099+0000 (oldest deadline 2024-09-21T03:58:10.136509+0000) 2024-09-21T03:58:31.293 INFO:journalctl@ceph.osd.9.smithi039.stdout:Sep 21 03:58:30 smithi039 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-9[67006]: 2024-09-21T03:58:30.932+0000 7f4a138c4640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-09-21T03:57:56.037703+0000 front 2024-09-21T03:57:56.037728+0000 (oldest deadline 2024-09-21T03:58:19.537285+0000) 2024-09-21T03:58:31.293 INFO:journalctl@ceph.osd.9.smithi039.stdout:Sep 21 03:58:30 smithi039 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-9[67006]: 2024-09-21T03:58:30.932+0000 7f4a138c4640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.161:6812 osd.5 since back 2024-09-21T03:58:03.438999+0000 front 2024-09-21T03:58:03.439023+0000 (oldest deadline 2024-09-21T03:58:26.938893+0000) 2024-09-21T03:58:31.584 INFO:journalctl@ceph.osd.9.smithi039.stdout:Sep 21 03:58:31 smithi039 systemd[1]: Stopping Ceph osd.9 for 51116d5c-77ca-11ef-baf5-efdc52797490... 2024-09-21T03:58:31.725 INFO:journalctl@ceph.osd.10.smithi150.stdout:Sep 21 03:58:31 smithi150 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-10[58206]: 2024-09-21T03:58:31.401+0000 7f34020e1640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-09-21T03:57:23.120033+0000 front 2024-09-21T03:57:23.120109+0000 (oldest deadline 2024-09-21T03:57:45.920434+0000) 2024-09-21T03:58:31.725 INFO:journalctl@ceph.osd.10.smithi150.stdout:Sep 21 03:58:31 smithi150 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-10[58206]: 2024-09-21T03:58:31.401+0000 7f34020e1640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-09-21T03:57:31.221107+0000 front 2024-09-21T03:57:31.221162+0000 (oldest deadline 2024-09-21T03:57:56.521240+0000) 2024-09-21T03:58:31.725 INFO:journalctl@ceph.osd.10.smithi150.stdout:Sep 21 03:58:31 smithi150 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-10[58206]: 2024-09-21T03:58:31.401+0000 7f34020e1640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-09-21T03:57:40.022210+0000 front 2024-09-21T03:57:40.022157+0000 (oldest deadline 2024-09-21T03:58:01.722244+0000) 2024-09-21T03:58:31.725 INFO:journalctl@ceph.osd.10.smithi150.stdout:Sep 21 03:58:31 smithi150 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-10[58206]: 2024-09-21T03:58:31.401+0000 7f34020e1640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2024-09-21T03:57:46.323919+0000 front 2024-09-21T03:57:46.323891+0000 (oldest deadline 2024-09-21T03:58:10.423788+0000) 2024-09-21T03:58:31.725 INFO:journalctl@ceph.osd.10.smithi150.stdout:Sep 21 03:58:31 smithi150 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-10[58206]: 2024-09-21T03:58:31.401+0000 7f34020e1640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-09-21T03:57:56.225398+0000 front 2024-09-21T03:57:56.225320+0000 (oldest deadline 2024-09-21T03:58:20.325491+0000) 2024-09-21T03:58:31.725 INFO:journalctl@ceph.osd.10.smithi150.stdout:Sep 21 03:58:31 smithi150 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-10[58206]: 2024-09-21T03:58:31.401+0000 7f34020e1640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.161:6812 osd.5 since back 2024-09-21T03:58:03.727214+0000 front 2024-09-21T03:58:03.727152+0000 (oldest deadline 2024-09-21T03:58:26.027019+0000) 2024-09-21T03:58:31.958 INFO:journalctl@ceph.osd.9.smithi039.stdout:Sep 21 03:58:31 smithi039 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-9[67006]: 2024-09-21T03:58:31.589+0000 7f4a172d9640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.9 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-21T03:58:31.958 INFO:journalctl@ceph.osd.9.smithi039.stdout:Sep 21 03:58:31 smithi039 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-9[67006]: 2024-09-21T03:58:31.589+0000 7f4a172d9640 -1 osd.9 90 *** Got signal Terminated *** 2024-09-21T03:58:31.958 INFO:journalctl@ceph.osd.9.smithi039.stdout:Sep 21 03:58:31 smithi039 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-9[67006]: 2024-09-21T03:58:31.589+0000 7f4a172d9640 -1 osd.9 90 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-21T03:58:32.218 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 21 03:58:31 smithi161 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-11[58829]: 2024-09-21T03:58:31.944+0000 7f538ef4a640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-09-21T03:57:21.581303+0000 front 2024-09-21T03:57:21.581333+0000 (oldest deadline 2024-09-21T03:57:46.881367+0000) 2024-09-21T03:58:32.218 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 21 03:58:31 smithi161 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-11[58829]: 2024-09-21T03:58:31.944+0000 7f538ef4a640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-09-21T03:57:32.782524+0000 front 2024-09-21T03:57:32.782318+0000 (oldest deadline 2024-09-21T03:57:54.482443+0000) 2024-09-21T03:58:32.218 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 21 03:58:31 smithi161 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-11[58829]: 2024-09-21T03:58:31.944+0000 7f538ef4a640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-09-21T03:57:40.184584+0000 front 2024-09-21T03:57:40.184544+0000 (oldest deadline 2024-09-21T03:58:05.484668+0000) 2024-09-21T03:58:32.218 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 21 03:58:31 smithi161 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-11[58829]: 2024-09-21T03:58:31.944+0000 7f538ef4a640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-09-21T03:57:55.387018+0000 front 2024-09-21T03:57:55.386996+0000 (oldest deadline 2024-09-21T03:58:20.086771+0000) 2024-09-21T03:58:32.218 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 21 03:58:31 smithi161 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-11[58829]: 2024-09-21T03:58:31.944+0000 7f538ef4a640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.161:6812 osd.5 since back 2024-09-21T03:58:02.887862+0000 front 2024-09-21T03:58:02.887979+0000 (oldest deadline 2024-09-21T03:58:27.587929+0000) 2024-09-21T03:58:32.334 INFO:journalctl@ceph.osd.9.smithi039.stdout:Sep 21 03:58:31 smithi039 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-9[67006]: 2024-09-21T03:58:31.955+0000 7f4a138c4640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-09-21T03:57:22.331574+0000 front 2024-09-21T03:57:22.332146+0000 (oldest deadline 2024-09-21T03:57:45.831720+0000) 2024-09-21T03:58:32.334 INFO:journalctl@ceph.osd.9.smithi039.stdout:Sep 21 03:58:31 smithi039 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-9[67006]: 2024-09-21T03:58:31.955+0000 7f4a138c4640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-09-21T03:57:29.932951+0000 front 2024-09-21T03:57:29.932930+0000 (oldest deadline 2024-09-21T03:57:54.632848+0000) 2024-09-21T03:58:32.334 INFO:journalctl@ceph.osd.9.smithi039.stdout:Sep 21 03:58:31 smithi039 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-9[67006]: 2024-09-21T03:58:31.955+0000 7f4a138c4640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-09-21T03:57:40.434486+0000 front 2024-09-21T03:57:40.434450+0000 (oldest deadline 2024-09-21T03:58:02.734524+0000) 2024-09-21T03:58:32.334 INFO:journalctl@ceph.osd.9.smithi039.stdout:Sep 21 03:58:31 smithi039 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-9[67006]: 2024-09-21T03:58:31.955+0000 7f4a138c4640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2024-09-21T03:57:44.336000+0000 front 2024-09-21T03:57:44.336099+0000 (oldest deadline 2024-09-21T03:58:10.136509+0000) 2024-09-21T03:58:32.334 INFO:journalctl@ceph.osd.9.smithi039.stdout:Sep 21 03:58:31 smithi039 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-9[67006]: 2024-09-21T03:58:31.955+0000 7f4a138c4640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-09-21T03:57:56.037703+0000 front 2024-09-21T03:57:56.037728+0000 (oldest deadline 2024-09-21T03:58:19.537285+0000) 2024-09-21T03:58:32.334 INFO:journalctl@ceph.osd.9.smithi039.stdout:Sep 21 03:58:31 smithi039 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-9[67006]: 2024-09-21T03:58:31.955+0000 7f4a138c4640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.161:6812 osd.5 since back 2024-09-21T03:58:03.438999+0000 front 2024-09-21T03:58:03.439023+0000 (oldest deadline 2024-09-21T03:58:26.938893+0000) 2024-09-21T03:58:32.725 INFO:journalctl@ceph.osd.10.smithi150.stdout:Sep 21 03:58:32 smithi150 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-10[58206]: 2024-09-21T03:58:32.398+0000 7f34020e1640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-09-21T03:57:23.120033+0000 front 2024-09-21T03:57:23.120109+0000 (oldest deadline 2024-09-21T03:57:45.920434+0000) 2024-09-21T03:58:32.725 INFO:journalctl@ceph.osd.10.smithi150.stdout:Sep 21 03:58:32 smithi150 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-10[58206]: 2024-09-21T03:58:32.398+0000 7f34020e1640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-09-21T03:57:31.221107+0000 front 2024-09-21T03:57:31.221162+0000 (oldest deadline 2024-09-21T03:57:56.521240+0000) 2024-09-21T03:58:32.725 INFO:journalctl@ceph.osd.10.smithi150.stdout:Sep 21 03:58:32 smithi150 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-10[58206]: 2024-09-21T03:58:32.398+0000 7f34020e1640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-09-21T03:57:40.022210+0000 front 2024-09-21T03:57:40.022157+0000 (oldest deadline 2024-09-21T03:58:01.722244+0000) 2024-09-21T03:58:32.725 INFO:journalctl@ceph.osd.10.smithi150.stdout:Sep 21 03:58:32 smithi150 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-10[58206]: 2024-09-21T03:58:32.398+0000 7f34020e1640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2024-09-21T03:57:46.323919+0000 front 2024-09-21T03:57:46.323891+0000 (oldest deadline 2024-09-21T03:58:10.423788+0000) 2024-09-21T03:58:32.725 INFO:journalctl@ceph.osd.10.smithi150.stdout:Sep 21 03:58:32 smithi150 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-10[58206]: 2024-09-21T03:58:32.398+0000 7f34020e1640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-09-21T03:57:56.225398+0000 front 2024-09-21T03:57:56.225320+0000 (oldest deadline 2024-09-21T03:58:20.325491+0000) 2024-09-21T03:58:32.725 INFO:journalctl@ceph.osd.10.smithi150.stdout:Sep 21 03:58:32 smithi150 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-10[58206]: 2024-09-21T03:58:32.398+0000 7f34020e1640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.161:6812 osd.5 since back 2024-09-21T03:58:03.727214+0000 front 2024-09-21T03:58:03.727152+0000 (oldest deadline 2024-09-21T03:58:26.027019+0000) 2024-09-21T03:58:33.218 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 21 03:58:32 smithi161 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-11[58829]: 2024-09-21T03:58:32.943+0000 7f538ef4a640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-09-21T03:57:21.581303+0000 front 2024-09-21T03:57:21.581333+0000 (oldest deadline 2024-09-21T03:57:46.881367+0000) 2024-09-21T03:58:33.218 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 21 03:58:32 smithi161 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-11[58829]: 2024-09-21T03:58:32.943+0000 7f538ef4a640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-09-21T03:57:32.782524+0000 front 2024-09-21T03:57:32.782318+0000 (oldest deadline 2024-09-21T03:57:54.482443+0000) 2024-09-21T03:58:33.218 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 21 03:58:32 smithi161 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-11[58829]: 2024-09-21T03:58:32.943+0000 7f538ef4a640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-09-21T03:57:40.184584+0000 front 2024-09-21T03:57:40.184544+0000 (oldest deadline 2024-09-21T03:58:05.484668+0000) 2024-09-21T03:58:33.218 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 21 03:58:32 smithi161 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-11[58829]: 2024-09-21T03:58:32.943+0000 7f538ef4a640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-09-21T03:57:55.387018+0000 front 2024-09-21T03:57:55.386996+0000 (oldest deadline 2024-09-21T03:58:20.086771+0000) 2024-09-21T03:58:33.218 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 21 03:58:32 smithi161 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-11[58829]: 2024-09-21T03:58:32.943+0000 7f538ef4a640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.161:6812 osd.5 since back 2024-09-21T03:58:02.887862+0000 front 2024-09-21T03:58:02.887979+0000 (oldest deadline 2024-09-21T03:58:27.587929+0000) 2024-09-21T03:58:33.334 INFO:journalctl@ceph.osd.9.smithi039.stdout:Sep 21 03:58:32 smithi039 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-9[67006]: 2024-09-21T03:58:32.965+0000 7f4a138c4640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-09-21T03:57:22.331574+0000 front 2024-09-21T03:57:22.332146+0000 (oldest deadline 2024-09-21T03:57:45.831720+0000) 2024-09-21T03:58:33.334 INFO:journalctl@ceph.osd.9.smithi039.stdout:Sep 21 03:58:32 smithi039 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-9[67006]: 2024-09-21T03:58:32.965+0000 7f4a138c4640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-09-21T03:57:29.932951+0000 front 2024-09-21T03:57:29.932930+0000 (oldest deadline 2024-09-21T03:57:54.632848+0000) 2024-09-21T03:58:33.334 INFO:journalctl@ceph.osd.9.smithi039.stdout:Sep 21 03:58:32 smithi039 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-9[67006]: 2024-09-21T03:58:32.965+0000 7f4a138c4640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-09-21T03:57:40.434486+0000 front 2024-09-21T03:57:40.434450+0000 (oldest deadline 2024-09-21T03:58:02.734524+0000) 2024-09-21T03:58:33.334 INFO:journalctl@ceph.osd.9.smithi039.stdout:Sep 21 03:58:32 smithi039 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-9[67006]: 2024-09-21T03:58:32.965+0000 7f4a138c4640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2024-09-21T03:57:44.336000+0000 front 2024-09-21T03:57:44.336099+0000 (oldest deadline 2024-09-21T03:58:10.136509+0000) 2024-09-21T03:58:33.334 INFO:journalctl@ceph.osd.9.smithi039.stdout:Sep 21 03:58:32 smithi039 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-9[67006]: 2024-09-21T03:58:32.965+0000 7f4a138c4640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-09-21T03:57:56.037703+0000 front 2024-09-21T03:57:56.037728+0000 (oldest deadline 2024-09-21T03:58:19.537285+0000) 2024-09-21T03:58:33.334 INFO:journalctl@ceph.osd.9.smithi039.stdout:Sep 21 03:58:32 smithi039 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-9[67006]: 2024-09-21T03:58:32.965+0000 7f4a138c4640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.161:6812 osd.5 since back 2024-09-21T03:58:03.438999+0000 front 2024-09-21T03:58:03.439023+0000 (oldest deadline 2024-09-21T03:58:26.938893+0000) 2024-09-21T03:58:33.725 INFO:journalctl@ceph.osd.10.smithi150.stdout:Sep 21 03:58:33 smithi150 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-10[58206]: 2024-09-21T03:58:33.433+0000 7f34020e1640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-09-21T03:57:23.120033+0000 front 2024-09-21T03:57:23.120109+0000 (oldest deadline 2024-09-21T03:57:45.920434+0000) 2024-09-21T03:58:33.725 INFO:journalctl@ceph.osd.10.smithi150.stdout:Sep 21 03:58:33 smithi150 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-10[58206]: 2024-09-21T03:58:33.433+0000 7f34020e1640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-09-21T03:57:31.221107+0000 front 2024-09-21T03:57:31.221162+0000 (oldest deadline 2024-09-21T03:57:56.521240+0000) 2024-09-21T03:58:33.725 INFO:journalctl@ceph.osd.10.smithi150.stdout:Sep 21 03:58:33 smithi150 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-10[58206]: 2024-09-21T03:58:33.433+0000 7f34020e1640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-09-21T03:57:40.022210+0000 front 2024-09-21T03:57:40.022157+0000 (oldest deadline 2024-09-21T03:58:01.722244+0000) 2024-09-21T03:58:33.725 INFO:journalctl@ceph.osd.10.smithi150.stdout:Sep 21 03:58:33 smithi150 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-10[58206]: 2024-09-21T03:58:33.433+0000 7f34020e1640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2024-09-21T03:57:46.323919+0000 front 2024-09-21T03:57:46.323891+0000 (oldest deadline 2024-09-21T03:58:10.423788+0000) 2024-09-21T03:58:33.725 INFO:journalctl@ceph.osd.10.smithi150.stdout:Sep 21 03:58:33 smithi150 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-10[58206]: 2024-09-21T03:58:33.433+0000 7f34020e1640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-09-21T03:57:56.225398+0000 front 2024-09-21T03:57:56.225320+0000 (oldest deadline 2024-09-21T03:58:20.325491+0000) 2024-09-21T03:58:33.725 INFO:journalctl@ceph.osd.10.smithi150.stdout:Sep 21 03:58:33 smithi150 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-10[58206]: 2024-09-21T03:58:33.433+0000 7f34020e1640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.161:6812 osd.5 since back 2024-09-21T03:58:03.727214+0000 front 2024-09-21T03:58:03.727152+0000 (oldest deadline 2024-09-21T03:58:26.027019+0000) 2024-09-21T03:58:34.218 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 21 03:58:33 smithi161 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-11[58829]: 2024-09-21T03:58:33.927+0000 7f538ef4a640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-09-21T03:57:21.581303+0000 front 2024-09-21T03:57:21.581333+0000 (oldest deadline 2024-09-21T03:57:46.881367+0000) 2024-09-21T03:58:34.218 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 21 03:58:33 smithi161 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-11[58829]: 2024-09-21T03:58:33.927+0000 7f538ef4a640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-09-21T03:57:32.782524+0000 front 2024-09-21T03:57:32.782318+0000 (oldest deadline 2024-09-21T03:57:54.482443+0000) 2024-09-21T03:58:34.218 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 21 03:58:33 smithi161 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-11[58829]: 2024-09-21T03:58:33.927+0000 7f538ef4a640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-09-21T03:57:40.184584+0000 front 2024-09-21T03:57:40.184544+0000 (oldest deadline 2024-09-21T03:58:05.484668+0000) 2024-09-21T03:58:34.218 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 21 03:58:33 smithi161 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-11[58829]: 2024-09-21T03:58:33.927+0000 7f538ef4a640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-09-21T03:57:55.387018+0000 front 2024-09-21T03:57:55.386996+0000 (oldest deadline 2024-09-21T03:58:20.086771+0000) 2024-09-21T03:58:34.218 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 21 03:58:33 smithi161 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-11[58829]: 2024-09-21T03:58:33.927+0000 7f538ef4a640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.161:6812 osd.5 since back 2024-09-21T03:58:02.887862+0000 front 2024-09-21T03:58:02.887979+0000 (oldest deadline 2024-09-21T03:58:27.587929+0000) 2024-09-21T03:58:34.334 INFO:journalctl@ceph.osd.9.smithi039.stdout:Sep 21 03:58:33 smithi039 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-9[67006]: 2024-09-21T03:58:33.933+0000 7f4a138c4640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-09-21T03:57:22.331574+0000 front 2024-09-21T03:57:22.332146+0000 (oldest deadline 2024-09-21T03:57:45.831720+0000) 2024-09-21T03:58:34.334 INFO:journalctl@ceph.osd.9.smithi039.stdout:Sep 21 03:58:33 smithi039 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-9[67006]: 2024-09-21T03:58:33.933+0000 7f4a138c4640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-09-21T03:57:29.932951+0000 front 2024-09-21T03:57:29.932930+0000 (oldest deadline 2024-09-21T03:57:54.632848+0000) 2024-09-21T03:58:34.334 INFO:journalctl@ceph.osd.9.smithi039.stdout:Sep 21 03:58:33 smithi039 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-9[67006]: 2024-09-21T03:58:33.933+0000 7f4a138c4640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-09-21T03:57:40.434486+0000 front 2024-09-21T03:57:40.434450+0000 (oldest deadline 2024-09-21T03:58:02.734524+0000) 2024-09-21T03:58:34.334 INFO:journalctl@ceph.osd.9.smithi039.stdout:Sep 21 03:58:33 smithi039 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-9[67006]: 2024-09-21T03:58:33.933+0000 7f4a138c4640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2024-09-21T03:57:44.336000+0000 front 2024-09-21T03:57:44.336099+0000 (oldest deadline 2024-09-21T03:58:10.136509+0000) 2024-09-21T03:58:34.334 INFO:journalctl@ceph.osd.9.smithi039.stdout:Sep 21 03:58:33 smithi039 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-9[67006]: 2024-09-21T03:58:33.933+0000 7f4a138c4640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-09-21T03:57:56.037703+0000 front 2024-09-21T03:57:56.037728+0000 (oldest deadline 2024-09-21T03:58:19.537285+0000) 2024-09-21T03:58:34.334 INFO:journalctl@ceph.osd.9.smithi039.stdout:Sep 21 03:58:33 smithi039 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-9[67006]: 2024-09-21T03:58:33.933+0000 7f4a138c4640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.161:6812 osd.5 since back 2024-09-21T03:58:03.438999+0000 front 2024-09-21T03:58:03.439023+0000 (oldest deadline 2024-09-21T03:58:26.938893+0000) 2024-09-21T03:58:34.725 INFO:journalctl@ceph.osd.10.smithi150.stdout:Sep 21 03:58:34 smithi150 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-10[58206]: 2024-09-21T03:58:34.395+0000 7f34020e1640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-09-21T03:57:23.120033+0000 front 2024-09-21T03:57:23.120109+0000 (oldest deadline 2024-09-21T03:57:45.920434+0000) 2024-09-21T03:58:34.725 INFO:journalctl@ceph.osd.10.smithi150.stdout:Sep 21 03:58:34 smithi150 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-10[58206]: 2024-09-21T03:58:34.395+0000 7f34020e1640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-09-21T03:57:31.221107+0000 front 2024-09-21T03:57:31.221162+0000 (oldest deadline 2024-09-21T03:57:56.521240+0000) 2024-09-21T03:58:34.725 INFO:journalctl@ceph.osd.10.smithi150.stdout:Sep 21 03:58:34 smithi150 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-10[58206]: 2024-09-21T03:58:34.395+0000 7f34020e1640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-09-21T03:57:40.022210+0000 front 2024-09-21T03:57:40.022157+0000 (oldest deadline 2024-09-21T03:58:01.722244+0000) 2024-09-21T03:58:34.725 INFO:journalctl@ceph.osd.10.smithi150.stdout:Sep 21 03:58:34 smithi150 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-10[58206]: 2024-09-21T03:58:34.395+0000 7f34020e1640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2024-09-21T03:57:46.323919+0000 front 2024-09-21T03:57:46.323891+0000 (oldest deadline 2024-09-21T03:58:10.423788+0000) 2024-09-21T03:58:34.725 INFO:journalctl@ceph.osd.10.smithi150.stdout:Sep 21 03:58:34 smithi150 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-10[58206]: 2024-09-21T03:58:34.395+0000 7f34020e1640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-09-21T03:57:56.225398+0000 front 2024-09-21T03:57:56.225320+0000 (oldest deadline 2024-09-21T03:58:20.325491+0000) 2024-09-21T03:58:34.725 INFO:journalctl@ceph.osd.10.smithi150.stdout:Sep 21 03:58:34 smithi150 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-10[58206]: 2024-09-21T03:58:34.395+0000 7f34020e1640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.161:6812 osd.5 since back 2024-09-21T03:58:03.727214+0000 front 2024-09-21T03:58:03.727152+0000 (oldest deadline 2024-09-21T03:58:26.027019+0000) 2024-09-21T03:58:35.334 INFO:journalctl@ceph.osd.9.smithi039.stdout:Sep 21 03:58:34 smithi039 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-9[67006]: 2024-09-21T03:58:34.978+0000 7f4a138c4640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-09-21T03:57:22.331574+0000 front 2024-09-21T03:57:22.332146+0000 (oldest deadline 2024-09-21T03:57:45.831720+0000) 2024-09-21T03:58:35.334 INFO:journalctl@ceph.osd.9.smithi039.stdout:Sep 21 03:58:34 smithi039 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-9[67006]: 2024-09-21T03:58:34.978+0000 7f4a138c4640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-09-21T03:57:29.932951+0000 front 2024-09-21T03:57:29.932930+0000 (oldest deadline 2024-09-21T03:57:54.632848+0000) 2024-09-21T03:58:35.334 INFO:journalctl@ceph.osd.9.smithi039.stdout:Sep 21 03:58:34 smithi039 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-9[67006]: 2024-09-21T03:58:34.978+0000 7f4a138c4640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-09-21T03:57:40.434486+0000 front 2024-09-21T03:57:40.434450+0000 (oldest deadline 2024-09-21T03:58:02.734524+0000) 2024-09-21T03:58:35.334 INFO:journalctl@ceph.osd.9.smithi039.stdout:Sep 21 03:58:34 smithi039 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-9[67006]: 2024-09-21T03:58:34.978+0000 7f4a138c4640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2024-09-21T03:57:44.336000+0000 front 2024-09-21T03:57:44.336099+0000 (oldest deadline 2024-09-21T03:58:10.136509+0000) 2024-09-21T03:58:35.335 INFO:journalctl@ceph.osd.9.smithi039.stdout:Sep 21 03:58:34 smithi039 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-9[67006]: 2024-09-21T03:58:34.978+0000 7f4a138c4640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-09-21T03:57:56.037703+0000 front 2024-09-21T03:57:56.037728+0000 (oldest deadline 2024-09-21T03:58:19.537285+0000) 2024-09-21T03:58:35.335 INFO:journalctl@ceph.osd.9.smithi039.stdout:Sep 21 03:58:34 smithi039 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-9[67006]: 2024-09-21T03:58:34.978+0000 7f4a138c4640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.161:6812 osd.5 since back 2024-09-21T03:58:03.438999+0000 front 2024-09-21T03:58:03.439023+0000 (oldest deadline 2024-09-21T03:58:26.938893+0000) 2024-09-21T03:58:35.335 INFO:journalctl@ceph.osd.9.smithi039.stdout:Sep 21 03:58:34 smithi039 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-9[67006]: 2024-09-21T03:58:34.978+0000 7f4a138c4640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.39:6822 osd.6 since back 2024-09-21T03:58:10.439927+0000 front 2024-09-21T03:58:10.439959+0000 (oldest deadline 2024-09-21T03:58:34.539861+0000) 2024-09-21T03:58:35.468 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 21 03:58:34 smithi161 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-11[58829]: 2024-09-21T03:58:34.976+0000 7f538ef4a640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-09-21T03:57:21.581303+0000 front 2024-09-21T03:57:21.581333+0000 (oldest deadline 2024-09-21T03:57:46.881367+0000) 2024-09-21T03:58:35.468 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 21 03:58:34 smithi161 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-11[58829]: 2024-09-21T03:58:34.976+0000 7f538ef4a640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-09-21T03:57:32.782524+0000 front 2024-09-21T03:57:32.782318+0000 (oldest deadline 2024-09-21T03:57:54.482443+0000) 2024-09-21T03:58:35.468 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 21 03:58:34 smithi161 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-11[58829]: 2024-09-21T03:58:34.976+0000 7f538ef4a640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-09-21T03:57:40.184584+0000 front 2024-09-21T03:57:40.184544+0000 (oldest deadline 2024-09-21T03:58:05.484668+0000) 2024-09-21T03:58:35.468 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 21 03:58:34 smithi161 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-11[58829]: 2024-09-21T03:58:34.976+0000 7f538ef4a640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-09-21T03:57:55.387018+0000 front 2024-09-21T03:57:55.386996+0000 (oldest deadline 2024-09-21T03:58:20.086771+0000) 2024-09-21T03:58:35.469 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 21 03:58:34 smithi161 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-11[58829]: 2024-09-21T03:58:34.976+0000 7f538ef4a640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.161:6812 osd.5 since back 2024-09-21T03:58:02.887862+0000 front 2024-09-21T03:58:02.887979+0000 (oldest deadline 2024-09-21T03:58:27.587929+0000) 2024-09-21T03:58:35.725 INFO:journalctl@ceph.osd.10.smithi150.stdout:Sep 21 03:58:35 smithi150 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-10[58206]: 2024-09-21T03:58:35.435+0000 7f34020e1640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-09-21T03:57:23.120033+0000 front 2024-09-21T03:57:23.120109+0000 (oldest deadline 2024-09-21T03:57:45.920434+0000) 2024-09-21T03:58:35.725 INFO:journalctl@ceph.osd.10.smithi150.stdout:Sep 21 03:58:35 smithi150 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-10[58206]: 2024-09-21T03:58:35.435+0000 7f34020e1640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-09-21T03:57:31.221107+0000 front 2024-09-21T03:57:31.221162+0000 (oldest deadline 2024-09-21T03:57:56.521240+0000) 2024-09-21T03:58:35.725 INFO:journalctl@ceph.osd.10.smithi150.stdout:Sep 21 03:58:35 smithi150 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-10[58206]: 2024-09-21T03:58:35.435+0000 7f34020e1640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-09-21T03:57:40.022210+0000 front 2024-09-21T03:57:40.022157+0000 (oldest deadline 2024-09-21T03:58:01.722244+0000) 2024-09-21T03:58:35.725 INFO:journalctl@ceph.osd.10.smithi150.stdout:Sep 21 03:58:35 smithi150 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-10[58206]: 2024-09-21T03:58:35.435+0000 7f34020e1640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2024-09-21T03:57:46.323919+0000 front 2024-09-21T03:57:46.323891+0000 (oldest deadline 2024-09-21T03:58:10.423788+0000) 2024-09-21T03:58:35.725 INFO:journalctl@ceph.osd.10.smithi150.stdout:Sep 21 03:58:35 smithi150 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-10[58206]: 2024-09-21T03:58:35.435+0000 7f34020e1640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-09-21T03:57:56.225398+0000 front 2024-09-21T03:57:56.225320+0000 (oldest deadline 2024-09-21T03:58:20.325491+0000) 2024-09-21T03:58:35.725 INFO:journalctl@ceph.osd.10.smithi150.stdout:Sep 21 03:58:35 smithi150 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-10[58206]: 2024-09-21T03:58:35.435+0000 7f34020e1640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.161:6812 osd.5 since back 2024-09-21T03:58:03.727214+0000 front 2024-09-21T03:58:03.727152+0000 (oldest deadline 2024-09-21T03:58:26.027019+0000) 2024-09-21T03:58:36.334 INFO:journalctl@ceph.osd.9.smithi039.stdout:Sep 21 03:58:35 smithi039 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-9[67006]: 2024-09-21T03:58:35.988+0000 7f4a138c4640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-09-21T03:57:22.331574+0000 front 2024-09-21T03:57:22.332146+0000 (oldest deadline 2024-09-21T03:57:45.831720+0000) 2024-09-21T03:58:36.334 INFO:journalctl@ceph.osd.9.smithi039.stdout:Sep 21 03:58:35 smithi039 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-9[67006]: 2024-09-21T03:58:35.988+0000 7f4a138c4640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-09-21T03:57:29.932951+0000 front 2024-09-21T03:57:29.932930+0000 (oldest deadline 2024-09-21T03:57:54.632848+0000) 2024-09-21T03:58:36.334 INFO:journalctl@ceph.osd.9.smithi039.stdout:Sep 21 03:58:35 smithi039 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-9[67006]: 2024-09-21T03:58:35.988+0000 7f4a138c4640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-09-21T03:57:40.434486+0000 front 2024-09-21T03:57:40.434450+0000 (oldest deadline 2024-09-21T03:58:02.734524+0000) 2024-09-21T03:58:36.334 INFO:journalctl@ceph.osd.9.smithi039.stdout:Sep 21 03:58:35 smithi039 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-9[67006]: 2024-09-21T03:58:35.988+0000 7f4a138c4640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2024-09-21T03:57:44.336000+0000 front 2024-09-21T03:57:44.336099+0000 (oldest deadline 2024-09-21T03:58:10.136509+0000) 2024-09-21T03:58:36.334 INFO:journalctl@ceph.osd.9.smithi039.stdout:Sep 21 03:58:35 smithi039 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-9[67006]: 2024-09-21T03:58:35.988+0000 7f4a138c4640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-09-21T03:57:56.037703+0000 front 2024-09-21T03:57:56.037728+0000 (oldest deadline 2024-09-21T03:58:19.537285+0000) 2024-09-21T03:58:36.334 INFO:journalctl@ceph.osd.9.smithi039.stdout:Sep 21 03:58:35 smithi039 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-9[67006]: 2024-09-21T03:58:35.988+0000 7f4a138c4640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.161:6812 osd.5 since back 2024-09-21T03:58:03.438999+0000 front 2024-09-21T03:58:03.439023+0000 (oldest deadline 2024-09-21T03:58:26.938893+0000) 2024-09-21T03:58:36.334 INFO:journalctl@ceph.osd.9.smithi039.stdout:Sep 21 03:58:35 smithi039 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-9[67006]: 2024-09-21T03:58:35.988+0000 7f4a138c4640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.39:6822 osd.6 since back 2024-09-21T03:58:10.439927+0000 front 2024-09-21T03:58:10.439959+0000 (oldest deadline 2024-09-21T03:58:34.539861+0000) 2024-09-21T03:58:36.468 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 21 03:58:35 smithi161 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-11[58829]: 2024-09-21T03:58:35.967+0000 7f538ef4a640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-09-21T03:57:21.581303+0000 front 2024-09-21T03:57:21.581333+0000 (oldest deadline 2024-09-21T03:57:46.881367+0000) 2024-09-21T03:58:36.468 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 21 03:58:35 smithi161 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-11[58829]: 2024-09-21T03:58:35.967+0000 7f538ef4a640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-09-21T03:57:32.782524+0000 front 2024-09-21T03:57:32.782318+0000 (oldest deadline 2024-09-21T03:57:54.482443+0000) 2024-09-21T03:58:36.468 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 21 03:58:35 smithi161 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-11[58829]: 2024-09-21T03:58:35.967+0000 7f538ef4a640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-09-21T03:57:40.184584+0000 front 2024-09-21T03:57:40.184544+0000 (oldest deadline 2024-09-21T03:58:05.484668+0000) 2024-09-21T03:58:36.468 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 21 03:58:35 smithi161 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-11[58829]: 2024-09-21T03:58:35.967+0000 7f538ef4a640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-09-21T03:57:55.387018+0000 front 2024-09-21T03:57:55.386996+0000 (oldest deadline 2024-09-21T03:58:20.086771+0000) 2024-09-21T03:58:36.468 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 21 03:58:35 smithi161 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-11[58829]: 2024-09-21T03:58:35.967+0000 7f538ef4a640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.161:6812 osd.5 since back 2024-09-21T03:58:02.887862+0000 front 2024-09-21T03:58:02.887979+0000 (oldest deadline 2024-09-21T03:58:27.587929+0000) 2024-09-21T03:58:36.468 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 21 03:58:35 smithi161 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-11[58829]: 2024-09-21T03:58:35.967+0000 7f538ef4a640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.39:6822 osd.6 since back 2024-09-21T03:58:09.888903+0000 front 2024-09-21T03:58:09.888888+0000 (oldest deadline 2024-09-21T03:58:35.188844+0000) 2024-09-21T03:58:36.725 INFO:journalctl@ceph.osd.10.smithi150.stdout:Sep 21 03:58:36 smithi150 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-10[58206]: 2024-09-21T03:58:36.415+0000 7f34020e1640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-09-21T03:57:23.120033+0000 front 2024-09-21T03:57:23.120109+0000 (oldest deadline 2024-09-21T03:57:45.920434+0000) 2024-09-21T03:58:36.725 INFO:journalctl@ceph.osd.10.smithi150.stdout:Sep 21 03:58:36 smithi150 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-10[58206]: 2024-09-21T03:58:36.415+0000 7f34020e1640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-09-21T03:57:31.221107+0000 front 2024-09-21T03:57:31.221162+0000 (oldest deadline 2024-09-21T03:57:56.521240+0000) 2024-09-21T03:58:36.725 INFO:journalctl@ceph.osd.10.smithi150.stdout:Sep 21 03:58:36 smithi150 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-10[58206]: 2024-09-21T03:58:36.415+0000 7f34020e1640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-09-21T03:57:40.022210+0000 front 2024-09-21T03:57:40.022157+0000 (oldest deadline 2024-09-21T03:58:01.722244+0000) 2024-09-21T03:58:36.725 INFO:journalctl@ceph.osd.10.smithi150.stdout:Sep 21 03:58:36 smithi150 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-10[58206]: 2024-09-21T03:58:36.415+0000 7f34020e1640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2024-09-21T03:57:46.323919+0000 front 2024-09-21T03:57:46.323891+0000 (oldest deadline 2024-09-21T03:58:10.423788+0000) 2024-09-21T03:58:36.725 INFO:journalctl@ceph.osd.10.smithi150.stdout:Sep 21 03:58:36 smithi150 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-10[58206]: 2024-09-21T03:58:36.415+0000 7f34020e1640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-09-21T03:57:56.225398+0000 front 2024-09-21T03:57:56.225320+0000 (oldest deadline 2024-09-21T03:58:20.325491+0000) 2024-09-21T03:58:36.725 INFO:journalctl@ceph.osd.10.smithi150.stdout:Sep 21 03:58:36 smithi150 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-10[58206]: 2024-09-21T03:58:36.415+0000 7f34020e1640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.161:6812 osd.5 since back 2024-09-21T03:58:03.727214+0000 front 2024-09-21T03:58:03.727152+0000 (oldest deadline 2024-09-21T03:58:26.027019+0000) 2024-09-21T03:58:37.334 INFO:journalctl@ceph.osd.9.smithi039.stdout:Sep 21 03:58:37 smithi039 podman[112910]: 2024-09-21 03:58:37.075195796 +0000 UTC m=+5.606890459 container died 71eed5410760a6f2bd59f6080622532635ad1eb4ab415bf2727a8eb7ff92b70d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-9, org.label-schema.schema-version=1.0, GIT_CLEAN=True, ceph=True, GIT_BRANCH=HEAD, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux ) 2024-09-21T03:58:37.468 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 21 03:58:37 smithi161 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-11[58829]: 2024-09-21T03:58:37.002+0000 7f538ef4a640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-09-21T03:57:21.581303+0000 front 2024-09-21T03:57:21.581333+0000 (oldest deadline 2024-09-21T03:57:46.881367+0000) 2024-09-21T03:58:37.468 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 21 03:58:37 smithi161 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-11[58829]: 2024-09-21T03:58:37.002+0000 7f538ef4a640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-09-21T03:57:32.782524+0000 front 2024-09-21T03:57:32.782318+0000 (oldest deadline 2024-09-21T03:57:54.482443+0000) 2024-09-21T03:58:37.468 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 21 03:58:37 smithi161 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-11[58829]: 2024-09-21T03:58:37.002+0000 7f538ef4a640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-09-21T03:57:40.184584+0000 front 2024-09-21T03:57:40.184544+0000 (oldest deadline 2024-09-21T03:58:05.484668+0000) 2024-09-21T03:58:37.468 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 21 03:58:37 smithi161 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-11[58829]: 2024-09-21T03:58:37.002+0000 7f538ef4a640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-09-21T03:57:55.387018+0000 front 2024-09-21T03:57:55.386996+0000 (oldest deadline 2024-09-21T03:58:20.086771+0000) 2024-09-21T03:58:37.468 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 21 03:58:37 smithi161 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-11[58829]: 2024-09-21T03:58:37.002+0000 7f538ef4a640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.161:6812 osd.5 since back 2024-09-21T03:58:02.887862+0000 front 2024-09-21T03:58:02.887979+0000 (oldest deadline 2024-09-21T03:58:27.587929+0000) 2024-09-21T03:58:37.469 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 21 03:58:37 smithi161 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-11[58829]: 2024-09-21T03:58:37.002+0000 7f538ef4a640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.39:6822 osd.6 since back 2024-09-21T03:58:09.888903+0000 front 2024-09-21T03:58:09.888888+0000 (oldest deadline 2024-09-21T03:58:35.188844+0000) 2024-09-21T03:58:37.725 INFO:journalctl@ceph.osd.10.smithi150.stdout:Sep 21 03:58:37 smithi150 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-10[58206]: 2024-09-21T03:58:37.439+0000 7f34020e1640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-09-21T03:57:23.120033+0000 front 2024-09-21T03:57:23.120109+0000 (oldest deadline 2024-09-21T03:57:45.920434+0000) 2024-09-21T03:58:37.725 INFO:journalctl@ceph.osd.10.smithi150.stdout:Sep 21 03:58:37 smithi150 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-10[58206]: 2024-09-21T03:58:37.439+0000 7f34020e1640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-09-21T03:57:31.221107+0000 front 2024-09-21T03:57:31.221162+0000 (oldest deadline 2024-09-21T03:57:56.521240+0000) 2024-09-21T03:58:37.725 INFO:journalctl@ceph.osd.10.smithi150.stdout:Sep 21 03:58:37 smithi150 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-10[58206]: 2024-09-21T03:58:37.439+0000 7f34020e1640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-09-21T03:57:40.022210+0000 front 2024-09-21T03:57:40.022157+0000 (oldest deadline 2024-09-21T03:58:01.722244+0000) 2024-09-21T03:58:37.725 INFO:journalctl@ceph.osd.10.smithi150.stdout:Sep 21 03:58:37 smithi150 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-10[58206]: 2024-09-21T03:58:37.439+0000 7f34020e1640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2024-09-21T03:57:46.323919+0000 front 2024-09-21T03:57:46.323891+0000 (oldest deadline 2024-09-21T03:58:10.423788+0000) 2024-09-21T03:58:37.725 INFO:journalctl@ceph.osd.10.smithi150.stdout:Sep 21 03:58:37 smithi150 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-10[58206]: 2024-09-21T03:58:37.439+0000 7f34020e1640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-09-21T03:57:56.225398+0000 front 2024-09-21T03:57:56.225320+0000 (oldest deadline 2024-09-21T03:58:20.325491+0000) 2024-09-21T03:58:37.725 INFO:journalctl@ceph.osd.10.smithi150.stdout:Sep 21 03:58:37 smithi150 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-10[58206]: 2024-09-21T03:58:37.439+0000 7f34020e1640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.161:6812 osd.5 since back 2024-09-21T03:58:03.727214+0000 front 2024-09-21T03:58:03.727152+0000 (oldest deadline 2024-09-21T03:58:26.027019+0000) 2024-09-21T03:58:37.726 INFO:journalctl@ceph.osd.10.smithi150.stdout:Sep 21 03:58:37 smithi150 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-10[58206]: 2024-09-21T03:58:37.439+0000 7f34020e1640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.39:6822 osd.6 since back 2024-09-21T03:58:11.129278+0000 front 2024-09-21T03:58:11.129260+0000 (oldest deadline 2024-09-21T03:58:36.429196+0000) 2024-09-21T03:58:37.834 INFO:journalctl@ceph.osd.9.smithi039.stdout:Sep 21 03:58:37 smithi039 podman[112910]: 2024-09-21 03:58:37.531243055 +0000 UTC m=+6.062937717 container cleanup 71eed5410760a6f2bd59f6080622532635ad1eb4ab415bf2727a8eb7ff92b70d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-9, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.build-date=20240731, io.buildah.version=1.37.2) 2024-09-21T03:58:37.834 INFO:journalctl@ceph.osd.9.smithi039.stdout:Sep 21 03:58:37 smithi039 bash[112910]: ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-9 2024-09-21T03:58:38.162 INFO:journalctl@ceph.osd.9.smithi039.stdout:Sep 21 03:58:37 smithi039 podman[112925]: 2024-09-21 03:58:37.832573892 +0000 UTC m=+0.759263516 container remove 71eed5410760a6f2bd59f6080622532635ad1eb4ab415bf2727a8eb7ff92b70d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-9, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_CLEAN=True, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0) 2024-09-21T03:58:38.468 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 21 03:58:38 smithi161 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-11[58829]: 2024-09-21T03:58:38.018+0000 7f538ef4a640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-09-21T03:57:21.581303+0000 front 2024-09-21T03:57:21.581333+0000 (oldest deadline 2024-09-21T03:57:46.881367+0000) 2024-09-21T03:58:38.468 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 21 03:58:38 smithi161 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-11[58829]: 2024-09-21T03:58:38.018+0000 7f538ef4a640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-09-21T03:57:32.782524+0000 front 2024-09-21T03:57:32.782318+0000 (oldest deadline 2024-09-21T03:57:54.482443+0000) 2024-09-21T03:58:38.468 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 21 03:58:38 smithi161 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-11[58829]: 2024-09-21T03:58:38.018+0000 7f538ef4a640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-09-21T03:57:40.184584+0000 front 2024-09-21T03:57:40.184544+0000 (oldest deadline 2024-09-21T03:58:05.484668+0000) 2024-09-21T03:58:38.468 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 21 03:58:38 smithi161 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-11[58829]: 2024-09-21T03:58:38.018+0000 7f538ef4a640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-09-21T03:57:55.387018+0000 front 2024-09-21T03:57:55.386996+0000 (oldest deadline 2024-09-21T03:58:20.086771+0000) 2024-09-21T03:58:38.468 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 21 03:58:38 smithi161 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-11[58829]: 2024-09-21T03:58:38.018+0000 7f538ef4a640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.161:6812 osd.5 since back 2024-09-21T03:58:02.887862+0000 front 2024-09-21T03:58:02.887979+0000 (oldest deadline 2024-09-21T03:58:27.587929+0000) 2024-09-21T03:58:38.468 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 21 03:58:38 smithi161 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-11[58829]: 2024-09-21T03:58:38.018+0000 7f538ef4a640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.39:6822 osd.6 since back 2024-09-21T03:58:09.888903+0000 front 2024-09-21T03:58:09.888888+0000 (oldest deadline 2024-09-21T03:58:35.188844+0000) 2024-09-21T03:58:38.479 INFO:journalctl@ceph.osd.9.smithi039.stdout:Sep 21 03:58:38 smithi039 podman[113013]: 2024-09-21 03:58:38.06334886 +0000 UTC m=+0.016244090 image pull ddce307e1f3263fb077b4f3902e15660050631dad4d2823a20ee3f72bf18d977 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262 2024-09-21T03:58:38.479 INFO:journalctl@ceph.osd.9.smithi039.stdout:Sep 21 03:58:38 smithi039 podman[113013]: 2024-09-21 03:58:38.191836382 +0000 UTC m=+0.144731665 container create d4b4336e2e1fef6a3e6e42edf8319c7bbad9efbcf62db8f34679d79ae7111433 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-9-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.build-date=20240731, ceph=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2) 2024-09-21T03:58:38.479 INFO:journalctl@ceph.osd.9.smithi039.stdout:Sep 21 03:58:38 smithi039 podman[113013]: 2024-09-21 03:58:38.394604411 +0000 UTC m=+0.347499635 container init d4b4336e2e1fef6a3e6e42edf8319c7bbad9efbcf62db8f34679d79ae7111433 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-9-deactivate, ceph=True, org.label-schema.license=GPLv2, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_CLEAN=True) 2024-09-21T03:58:38.479 INFO:journalctl@ceph.osd.9.smithi039.stdout:Sep 21 03:58:38 smithi039 podman[113013]: 2024-09-21 03:58:38.398060943 +0000 UTC m=+0.350956163 container start d4b4336e2e1fef6a3e6e42edf8319c7bbad9efbcf62db8f34679d79ae7111433 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-9-deactivate, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, ceph=True, io.buildah.version=1.37.2, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-21T03:58:38.479 INFO:journalctl@ceph.osd.9.smithi039.stdout:Sep 21 03:58:38 smithi039 podman[113013]: 2024-09-21 03:58:38.477695253 +0000 UTC m=+0.430590491 container attach d4b4336e2e1fef6a3e6e42edf8319c7bbad9efbcf62db8f34679d79ae7111433 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-9-deactivate, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , ceph=True, io.buildah.version=1.37.2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.schema-version=1.0) 2024-09-21T03:58:38.725 INFO:journalctl@ceph.osd.10.smithi150.stdout:Sep 21 03:58:38 smithi150 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-10[58206]: 2024-09-21T03:58:38.410+0000 7f34020e1640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-09-21T03:57:23.120033+0000 front 2024-09-21T03:57:23.120109+0000 (oldest deadline 2024-09-21T03:57:45.920434+0000) 2024-09-21T03:58:38.725 INFO:journalctl@ceph.osd.10.smithi150.stdout:Sep 21 03:58:38 smithi150 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-10[58206]: 2024-09-21T03:58:38.410+0000 7f34020e1640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-09-21T03:57:31.221107+0000 front 2024-09-21T03:57:31.221162+0000 (oldest deadline 2024-09-21T03:57:56.521240+0000) 2024-09-21T03:58:38.725 INFO:journalctl@ceph.osd.10.smithi150.stdout:Sep 21 03:58:38 smithi150 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-10[58206]: 2024-09-21T03:58:38.410+0000 7f34020e1640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-09-21T03:57:40.022210+0000 front 2024-09-21T03:57:40.022157+0000 (oldest deadline 2024-09-21T03:58:01.722244+0000) 2024-09-21T03:58:38.725 INFO:journalctl@ceph.osd.10.smithi150.stdout:Sep 21 03:58:38 smithi150 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-10[58206]: 2024-09-21T03:58:38.410+0000 7f34020e1640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2024-09-21T03:57:46.323919+0000 front 2024-09-21T03:57:46.323891+0000 (oldest deadline 2024-09-21T03:58:10.423788+0000) 2024-09-21T03:58:38.725 INFO:journalctl@ceph.osd.10.smithi150.stdout:Sep 21 03:58:38 smithi150 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-10[58206]: 2024-09-21T03:58:38.410+0000 7f34020e1640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-09-21T03:57:56.225398+0000 front 2024-09-21T03:57:56.225320+0000 (oldest deadline 2024-09-21T03:58:20.325491+0000) 2024-09-21T03:58:38.725 INFO:journalctl@ceph.osd.10.smithi150.stdout:Sep 21 03:58:38 smithi150 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-10[58206]: 2024-09-21T03:58:38.410+0000 7f34020e1640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.161:6812 osd.5 since back 2024-09-21T03:58:03.727214+0000 front 2024-09-21T03:58:03.727152+0000 (oldest deadline 2024-09-21T03:58:26.027019+0000) 2024-09-21T03:58:38.725 INFO:journalctl@ceph.osd.10.smithi150.stdout:Sep 21 03:58:38 smithi150 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-10[58206]: 2024-09-21T03:58:38.410+0000 7f34020e1640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.39:6822 osd.6 since back 2024-09-21T03:58:11.129278+0000 front 2024-09-21T03:58:11.129260+0000 (oldest deadline 2024-09-21T03:58:36.429196+0000) 2024-09-21T03:58:38.834 INFO:journalctl@ceph.osd.9.smithi039.stdout:Sep 21 03:58:38 smithi039 podman[113013]: 2024-09-21 03:58:38.600004331 +0000 UTC m=+0.552899569 container died d4b4336e2e1fef6a3e6e42edf8319c7bbad9efbcf62db8f34679d79ae7111433 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-9-deactivate, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.build-date=20240731, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-21T03:58:39.242 DEBUG:teuthology.orchestra.run.smithi039:> sudo pkill -f 'journalctl -f -n 0 -u ceph-51116d5c-77ca-11ef-baf5-efdc52797490@osd.9.service' 2024-09-21T03:58:39.325 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-21T03:58:39.325 INFO:tasks.cephadm.osd.9:Stopped osd.9 2024-09-21T03:58:39.326 INFO:tasks.cephadm.osd.10:Stopping osd.10... 2024-09-21T03:58:39.326 DEBUG:teuthology.orchestra.run.smithi150:> sudo systemctl stop ceph-51116d5c-77ca-11ef-baf5-efdc52797490@osd.10 2024-09-21T03:58:39.468 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 21 03:58:39 smithi161 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-11[58829]: 2024-09-21T03:58:39.022+0000 7f538ef4a640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-09-21T03:57:21.581303+0000 front 2024-09-21T03:57:21.581333+0000 (oldest deadline 2024-09-21T03:57:46.881367+0000) 2024-09-21T03:58:39.468 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 21 03:58:39 smithi161 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-11[58829]: 2024-09-21T03:58:39.022+0000 7f538ef4a640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-09-21T03:57:32.782524+0000 front 2024-09-21T03:57:32.782318+0000 (oldest deadline 2024-09-21T03:57:54.482443+0000) 2024-09-21T03:58:39.468 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 21 03:58:39 smithi161 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-11[58829]: 2024-09-21T03:58:39.022+0000 7f538ef4a640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-09-21T03:57:40.184584+0000 front 2024-09-21T03:57:40.184544+0000 (oldest deadline 2024-09-21T03:58:05.484668+0000) 2024-09-21T03:58:39.468 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 21 03:58:39 smithi161 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-11[58829]: 2024-09-21T03:58:39.022+0000 7f538ef4a640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-09-21T03:57:55.387018+0000 front 2024-09-21T03:57:55.386996+0000 (oldest deadline 2024-09-21T03:58:20.086771+0000) 2024-09-21T03:58:39.469 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 21 03:58:39 smithi161 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-11[58829]: 2024-09-21T03:58:39.022+0000 7f538ef4a640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.161:6812 osd.5 since back 2024-09-21T03:58:02.887862+0000 front 2024-09-21T03:58:02.887979+0000 (oldest deadline 2024-09-21T03:58:27.587929+0000) 2024-09-21T03:58:39.469 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 21 03:58:39 smithi161 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-11[58829]: 2024-09-21T03:58:39.022+0000 7f538ef4a640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.39:6822 osd.6 since back 2024-09-21T03:58:09.888903+0000 front 2024-09-21T03:58:09.888888+0000 (oldest deadline 2024-09-21T03:58:35.188844+0000) 2024-09-21T03:58:39.622 INFO:journalctl@ceph.osd.10.smithi150.stdout:Sep 21 03:58:39 smithi150 systemd[1]: Stopping Ceph osd.10 for 51116d5c-77ca-11ef-baf5-efdc52797490... 2024-09-21T03:58:39.623 INFO:journalctl@ceph.osd.10.smithi150.stdout:Sep 21 03:58:39 smithi150 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-10[58206]: 2024-09-21T03:58:39.440+0000 7f34020e1640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-09-21T03:57:23.120033+0000 front 2024-09-21T03:57:23.120109+0000 (oldest deadline 2024-09-21T03:57:45.920434+0000) 2024-09-21T03:58:39.623 INFO:journalctl@ceph.osd.10.smithi150.stdout:Sep 21 03:58:39 smithi150 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-10[58206]: 2024-09-21T03:58:39.440+0000 7f34020e1640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-09-21T03:57:31.221107+0000 front 2024-09-21T03:57:31.221162+0000 (oldest deadline 2024-09-21T03:57:56.521240+0000) 2024-09-21T03:58:39.623 INFO:journalctl@ceph.osd.10.smithi150.stdout:Sep 21 03:58:39 smithi150 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-10[58206]: 2024-09-21T03:58:39.440+0000 7f34020e1640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-09-21T03:57:40.022210+0000 front 2024-09-21T03:57:40.022157+0000 (oldest deadline 2024-09-21T03:58:01.722244+0000) 2024-09-21T03:58:39.623 INFO:journalctl@ceph.osd.10.smithi150.stdout:Sep 21 03:58:39 smithi150 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-10[58206]: 2024-09-21T03:58:39.440+0000 7f34020e1640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2024-09-21T03:57:46.323919+0000 front 2024-09-21T03:57:46.323891+0000 (oldest deadline 2024-09-21T03:58:10.423788+0000) 2024-09-21T03:58:39.623 INFO:journalctl@ceph.osd.10.smithi150.stdout:Sep 21 03:58:39 smithi150 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-10[58206]: 2024-09-21T03:58:39.440+0000 7f34020e1640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-09-21T03:57:56.225398+0000 front 2024-09-21T03:57:56.225320+0000 (oldest deadline 2024-09-21T03:58:20.325491+0000) 2024-09-21T03:58:39.623 INFO:journalctl@ceph.osd.10.smithi150.stdout:Sep 21 03:58:39 smithi150 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-10[58206]: 2024-09-21T03:58:39.440+0000 7f34020e1640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.161:6812 osd.5 since back 2024-09-21T03:58:03.727214+0000 front 2024-09-21T03:58:03.727152+0000 (oldest deadline 2024-09-21T03:58:26.027019+0000) 2024-09-21T03:58:39.623 INFO:journalctl@ceph.osd.10.smithi150.stdout:Sep 21 03:58:39 smithi150 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-10[58206]: 2024-09-21T03:58:39.440+0000 7f34020e1640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.39:6822 osd.6 since back 2024-09-21T03:58:11.129278+0000 front 2024-09-21T03:58:11.129260+0000 (oldest deadline 2024-09-21T03:58:36.429196+0000) 2024-09-21T03:58:39.623 INFO:journalctl@ceph.osd.10.smithi150.stdout:Sep 21 03:58:39 smithi150 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-10[58206]: 2024-09-21T03:58:39.620+0000 7f3405af6640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.10 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-21T03:58:39.623 INFO:journalctl@ceph.osd.10.smithi150.stdout:Sep 21 03:58:39 smithi150 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-10[58206]: 2024-09-21T03:58:39.620+0000 7f3405af6640 -1 osd.10 90 *** Got signal Terminated *** 2024-09-21T03:58:39.623 INFO:journalctl@ceph.osd.10.smithi150.stdout:Sep 21 03:58:39 smithi150 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-10[58206]: 2024-09-21T03:58:39.620+0000 7f3405af6640 -1 osd.10 90 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-21T03:58:40.468 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 21 03:58:40 smithi161 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-11[58829]: 2024-09-21T03:58:40.055+0000 7f538ef4a640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-09-21T03:57:21.581303+0000 front 2024-09-21T03:57:21.581333+0000 (oldest deadline 2024-09-21T03:57:46.881367+0000) 2024-09-21T03:58:40.468 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 21 03:58:40 smithi161 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-11[58829]: 2024-09-21T03:58:40.055+0000 7f538ef4a640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-09-21T03:57:32.782524+0000 front 2024-09-21T03:57:32.782318+0000 (oldest deadline 2024-09-21T03:57:54.482443+0000) 2024-09-21T03:58:40.468 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 21 03:58:40 smithi161 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-11[58829]: 2024-09-21T03:58:40.055+0000 7f538ef4a640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-09-21T03:57:40.184584+0000 front 2024-09-21T03:57:40.184544+0000 (oldest deadline 2024-09-21T03:58:05.484668+0000) 2024-09-21T03:58:40.468 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 21 03:58:40 smithi161 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-11[58829]: 2024-09-21T03:58:40.055+0000 7f538ef4a640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-09-21T03:57:55.387018+0000 front 2024-09-21T03:57:55.386996+0000 (oldest deadline 2024-09-21T03:58:20.086771+0000) 2024-09-21T03:58:40.468 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 21 03:58:40 smithi161 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-11[58829]: 2024-09-21T03:58:40.055+0000 7f538ef4a640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.161:6812 osd.5 since back 2024-09-21T03:58:02.887862+0000 front 2024-09-21T03:58:02.887979+0000 (oldest deadline 2024-09-21T03:58:27.587929+0000) 2024-09-21T03:58:40.468 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 21 03:58:40 smithi161 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-11[58829]: 2024-09-21T03:58:40.055+0000 7f538ef4a640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.39:6822 osd.6 since back 2024-09-21T03:58:09.888903+0000 front 2024-09-21T03:58:09.888888+0000 (oldest deadline 2024-09-21T03:58:35.188844+0000) 2024-09-21T03:58:40.725 INFO:journalctl@ceph.osd.10.smithi150.stdout:Sep 21 03:58:40 smithi150 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-10[58206]: 2024-09-21T03:58:40.452+0000 7f34020e1640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-09-21T03:57:23.120033+0000 front 2024-09-21T03:57:23.120109+0000 (oldest deadline 2024-09-21T03:57:45.920434+0000) 2024-09-21T03:58:40.725 INFO:journalctl@ceph.osd.10.smithi150.stdout:Sep 21 03:58:40 smithi150 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-10[58206]: 2024-09-21T03:58:40.452+0000 7f34020e1640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-09-21T03:57:31.221107+0000 front 2024-09-21T03:57:31.221162+0000 (oldest deadline 2024-09-21T03:57:56.521240+0000) 2024-09-21T03:58:40.725 INFO:journalctl@ceph.osd.10.smithi150.stdout:Sep 21 03:58:40 smithi150 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-10[58206]: 2024-09-21T03:58:40.452+0000 7f34020e1640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-09-21T03:57:40.022210+0000 front 2024-09-21T03:57:40.022157+0000 (oldest deadline 2024-09-21T03:58:01.722244+0000) 2024-09-21T03:58:40.725 INFO:journalctl@ceph.osd.10.smithi150.stdout:Sep 21 03:58:40 smithi150 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-10[58206]: 2024-09-21T03:58:40.452+0000 7f34020e1640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2024-09-21T03:57:46.323919+0000 front 2024-09-21T03:57:46.323891+0000 (oldest deadline 2024-09-21T03:58:10.423788+0000) 2024-09-21T03:58:40.725 INFO:journalctl@ceph.osd.10.smithi150.stdout:Sep 21 03:58:40 smithi150 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-10[58206]: 2024-09-21T03:58:40.452+0000 7f34020e1640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-09-21T03:57:56.225398+0000 front 2024-09-21T03:57:56.225320+0000 (oldest deadline 2024-09-21T03:58:20.325491+0000) 2024-09-21T03:58:40.725 INFO:journalctl@ceph.osd.10.smithi150.stdout:Sep 21 03:58:40 smithi150 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-10[58206]: 2024-09-21T03:58:40.452+0000 7f34020e1640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.161:6812 osd.5 since back 2024-09-21T03:58:03.727214+0000 front 2024-09-21T03:58:03.727152+0000 (oldest deadline 2024-09-21T03:58:26.027019+0000) 2024-09-21T03:58:40.725 INFO:journalctl@ceph.osd.10.smithi150.stdout:Sep 21 03:58:40 smithi150 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-10[58206]: 2024-09-21T03:58:40.452+0000 7f34020e1640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.39:6822 osd.6 since back 2024-09-21T03:58:11.129278+0000 front 2024-09-21T03:58:11.129260+0000 (oldest deadline 2024-09-21T03:58:36.429196+0000) 2024-09-21T03:58:41.369 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 21 03:58:41 smithi161 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-11[58829]: 2024-09-21T03:58:41.079+0000 7f538ef4a640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-09-21T03:57:21.581303+0000 front 2024-09-21T03:57:21.581333+0000 (oldest deadline 2024-09-21T03:57:46.881367+0000) 2024-09-21T03:58:41.369 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 21 03:58:41 smithi161 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-11[58829]: 2024-09-21T03:58:41.079+0000 7f538ef4a640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-09-21T03:57:32.782524+0000 front 2024-09-21T03:57:32.782318+0000 (oldest deadline 2024-09-21T03:57:54.482443+0000) 2024-09-21T03:58:41.369 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 21 03:58:41 smithi161 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-11[58829]: 2024-09-21T03:58:41.079+0000 7f538ef4a640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-09-21T03:57:40.184584+0000 front 2024-09-21T03:57:40.184544+0000 (oldest deadline 2024-09-21T03:58:05.484668+0000) 2024-09-21T03:58:41.369 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 21 03:58:41 smithi161 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-11[58829]: 2024-09-21T03:58:41.079+0000 7f538ef4a640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-09-21T03:57:55.387018+0000 front 2024-09-21T03:57:55.386996+0000 (oldest deadline 2024-09-21T03:58:20.086771+0000) 2024-09-21T03:58:41.370 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 21 03:58:41 smithi161 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-11[58829]: 2024-09-21T03:58:41.079+0000 7f538ef4a640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.161:6812 osd.5 since back 2024-09-21T03:58:02.887862+0000 front 2024-09-21T03:58:02.887979+0000 (oldest deadline 2024-09-21T03:58:27.587929+0000) 2024-09-21T03:58:41.370 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 21 03:58:41 smithi161 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-11[58829]: 2024-09-21T03:58:41.079+0000 7f538ef4a640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.39:6822 osd.6 since back 2024-09-21T03:58:09.888903+0000 front 2024-09-21T03:58:09.888888+0000 (oldest deadline 2024-09-21T03:58:35.188844+0000) 2024-09-21T03:58:41.725 INFO:journalctl@ceph.osd.10.smithi150.stdout:Sep 21 03:58:41 smithi150 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-10[58206]: 2024-09-21T03:58:41.432+0000 7f34020e1640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-09-21T03:57:23.120033+0000 front 2024-09-21T03:57:23.120109+0000 (oldest deadline 2024-09-21T03:57:45.920434+0000) 2024-09-21T03:58:41.725 INFO:journalctl@ceph.osd.10.smithi150.stdout:Sep 21 03:58:41 smithi150 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-10[58206]: 2024-09-21T03:58:41.432+0000 7f34020e1640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-09-21T03:57:31.221107+0000 front 2024-09-21T03:57:31.221162+0000 (oldest deadline 2024-09-21T03:57:56.521240+0000) 2024-09-21T03:58:41.725 INFO:journalctl@ceph.osd.10.smithi150.stdout:Sep 21 03:58:41 smithi150 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-10[58206]: 2024-09-21T03:58:41.432+0000 7f34020e1640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-09-21T03:57:40.022210+0000 front 2024-09-21T03:57:40.022157+0000 (oldest deadline 2024-09-21T03:58:01.722244+0000) 2024-09-21T03:58:41.725 INFO:journalctl@ceph.osd.10.smithi150.stdout:Sep 21 03:58:41 smithi150 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-10[58206]: 2024-09-21T03:58:41.432+0000 7f34020e1640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2024-09-21T03:57:46.323919+0000 front 2024-09-21T03:57:46.323891+0000 (oldest deadline 2024-09-21T03:58:10.423788+0000) 2024-09-21T03:58:41.725 INFO:journalctl@ceph.osd.10.smithi150.stdout:Sep 21 03:58:41 smithi150 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-10[58206]: 2024-09-21T03:58:41.432+0000 7f34020e1640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-09-21T03:57:56.225398+0000 front 2024-09-21T03:57:56.225320+0000 (oldest deadline 2024-09-21T03:58:20.325491+0000) 2024-09-21T03:58:41.725 INFO:journalctl@ceph.osd.10.smithi150.stdout:Sep 21 03:58:41 smithi150 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-10[58206]: 2024-09-21T03:58:41.432+0000 7f34020e1640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.161:6812 osd.5 since back 2024-09-21T03:58:03.727214+0000 front 2024-09-21T03:58:03.727152+0000 (oldest deadline 2024-09-21T03:58:26.027019+0000) 2024-09-21T03:58:41.725 INFO:journalctl@ceph.osd.10.smithi150.stdout:Sep 21 03:58:41 smithi150 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-10[58206]: 2024-09-21T03:58:41.432+0000 7f34020e1640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.39:6822 osd.6 since back 2024-09-21T03:58:11.129278+0000 front 2024-09-21T03:58:11.129260+0000 (oldest deadline 2024-09-21T03:58:36.429196+0000) 2024-09-21T03:58:42.468 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 21 03:58:42 smithi161 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-11[58829]: 2024-09-21T03:58:42.128+0000 7f538ef4a640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-09-21T03:57:21.581303+0000 front 2024-09-21T03:57:21.581333+0000 (oldest deadline 2024-09-21T03:57:46.881367+0000) 2024-09-21T03:58:42.468 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 21 03:58:42 smithi161 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-11[58829]: 2024-09-21T03:58:42.128+0000 7f538ef4a640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-09-21T03:57:32.782524+0000 front 2024-09-21T03:57:32.782318+0000 (oldest deadline 2024-09-21T03:57:54.482443+0000) 2024-09-21T03:58:42.468 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 21 03:58:42 smithi161 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-11[58829]: 2024-09-21T03:58:42.128+0000 7f538ef4a640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-09-21T03:57:40.184584+0000 front 2024-09-21T03:57:40.184544+0000 (oldest deadline 2024-09-21T03:58:05.484668+0000) 2024-09-21T03:58:42.468 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 21 03:58:42 smithi161 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-11[58829]: 2024-09-21T03:58:42.128+0000 7f538ef4a640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-09-21T03:57:55.387018+0000 front 2024-09-21T03:57:55.386996+0000 (oldest deadline 2024-09-21T03:58:20.086771+0000) 2024-09-21T03:58:42.468 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 21 03:58:42 smithi161 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-11[58829]: 2024-09-21T03:58:42.128+0000 7f538ef4a640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.161:6812 osd.5 since back 2024-09-21T03:58:02.887862+0000 front 2024-09-21T03:58:02.887979+0000 (oldest deadline 2024-09-21T03:58:27.587929+0000) 2024-09-21T03:58:42.468 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 21 03:58:42 smithi161 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-11[58829]: 2024-09-21T03:58:42.128+0000 7f538ef4a640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.39:6822 osd.6 since back 2024-09-21T03:58:09.888903+0000 front 2024-09-21T03:58:09.888888+0000 (oldest deadline 2024-09-21T03:58:35.188844+0000) 2024-09-21T03:58:42.725 INFO:journalctl@ceph.osd.10.smithi150.stdout:Sep 21 03:58:42 smithi150 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-10[58206]: 2024-09-21T03:58:42.425+0000 7f34020e1640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-09-21T03:57:23.120033+0000 front 2024-09-21T03:57:23.120109+0000 (oldest deadline 2024-09-21T03:57:45.920434+0000) 2024-09-21T03:58:42.725 INFO:journalctl@ceph.osd.10.smithi150.stdout:Sep 21 03:58:42 smithi150 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-10[58206]: 2024-09-21T03:58:42.425+0000 7f34020e1640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-09-21T03:57:31.221107+0000 front 2024-09-21T03:57:31.221162+0000 (oldest deadline 2024-09-21T03:57:56.521240+0000) 2024-09-21T03:58:42.725 INFO:journalctl@ceph.osd.10.smithi150.stdout:Sep 21 03:58:42 smithi150 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-10[58206]: 2024-09-21T03:58:42.425+0000 7f34020e1640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-09-21T03:57:40.022210+0000 front 2024-09-21T03:57:40.022157+0000 (oldest deadline 2024-09-21T03:58:01.722244+0000) 2024-09-21T03:58:42.725 INFO:journalctl@ceph.osd.10.smithi150.stdout:Sep 21 03:58:42 smithi150 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-10[58206]: 2024-09-21T03:58:42.425+0000 7f34020e1640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2024-09-21T03:57:46.323919+0000 front 2024-09-21T03:57:46.323891+0000 (oldest deadline 2024-09-21T03:58:10.423788+0000) 2024-09-21T03:58:42.725 INFO:journalctl@ceph.osd.10.smithi150.stdout:Sep 21 03:58:42 smithi150 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-10[58206]: 2024-09-21T03:58:42.425+0000 7f34020e1640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-09-21T03:57:56.225398+0000 front 2024-09-21T03:57:56.225320+0000 (oldest deadline 2024-09-21T03:58:20.325491+0000) 2024-09-21T03:58:42.725 INFO:journalctl@ceph.osd.10.smithi150.stdout:Sep 21 03:58:42 smithi150 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-10[58206]: 2024-09-21T03:58:42.425+0000 7f34020e1640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.161:6812 osd.5 since back 2024-09-21T03:58:03.727214+0000 front 2024-09-21T03:58:03.727152+0000 (oldest deadline 2024-09-21T03:58:26.027019+0000) 2024-09-21T03:58:42.725 INFO:journalctl@ceph.osd.10.smithi150.stdout:Sep 21 03:58:42 smithi150 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-10[58206]: 2024-09-21T03:58:42.425+0000 7f34020e1640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.39:6822 osd.6 since back 2024-09-21T03:58:11.129278+0000 front 2024-09-21T03:58:11.129260+0000 (oldest deadline 2024-09-21T03:58:36.429196+0000) 2024-09-21T03:58:43.468 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 21 03:58:43 smithi161 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-11[58829]: 2024-09-21T03:58:43.108+0000 7f538ef4a640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-09-21T03:57:21.581303+0000 front 2024-09-21T03:57:21.581333+0000 (oldest deadline 2024-09-21T03:57:46.881367+0000) 2024-09-21T03:58:43.468 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 21 03:58:43 smithi161 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-11[58829]: 2024-09-21T03:58:43.108+0000 7f538ef4a640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-09-21T03:57:32.782524+0000 front 2024-09-21T03:57:32.782318+0000 (oldest deadline 2024-09-21T03:57:54.482443+0000) 2024-09-21T03:58:43.468 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 21 03:58:43 smithi161 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-11[58829]: 2024-09-21T03:58:43.108+0000 7f538ef4a640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-09-21T03:57:40.184584+0000 front 2024-09-21T03:57:40.184544+0000 (oldest deadline 2024-09-21T03:58:05.484668+0000) 2024-09-21T03:58:43.469 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 21 03:58:43 smithi161 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-11[58829]: 2024-09-21T03:58:43.108+0000 7f538ef4a640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-09-21T03:57:55.387018+0000 front 2024-09-21T03:57:55.386996+0000 (oldest deadline 2024-09-21T03:58:20.086771+0000) 2024-09-21T03:58:43.469 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 21 03:58:43 smithi161 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-11[58829]: 2024-09-21T03:58:43.108+0000 7f538ef4a640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.161:6812 osd.5 since back 2024-09-21T03:58:02.887862+0000 front 2024-09-21T03:58:02.887979+0000 (oldest deadline 2024-09-21T03:58:27.587929+0000) 2024-09-21T03:58:43.469 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 21 03:58:43 smithi161 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-11[58829]: 2024-09-21T03:58:43.108+0000 7f538ef4a640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.39:6822 osd.6 since back 2024-09-21T03:58:09.888903+0000 front 2024-09-21T03:58:09.888888+0000 (oldest deadline 2024-09-21T03:58:35.188844+0000) 2024-09-21T03:58:43.469 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 21 03:58:43 smithi161 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-11[58829]: 2024-09-21T03:58:43.108+0000 7f538ef4a640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.150:6820 osd.7 since back 2024-09-21T03:58:17.489916+0000 front 2024-09-21T03:58:17.489821+0000 (oldest deadline 2024-09-21T03:58:42.189775+0000) 2024-09-21T03:58:43.725 INFO:journalctl@ceph.osd.10.smithi150.stdout:Sep 21 03:58:43 smithi150 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-10[58206]: 2024-09-21T03:58:43.413+0000 7f34020e1640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-09-21T03:57:23.120033+0000 front 2024-09-21T03:57:23.120109+0000 (oldest deadline 2024-09-21T03:57:45.920434+0000) 2024-09-21T03:58:43.725 INFO:journalctl@ceph.osd.10.smithi150.stdout:Sep 21 03:58:43 smithi150 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-10[58206]: 2024-09-21T03:58:43.413+0000 7f34020e1640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-09-21T03:57:31.221107+0000 front 2024-09-21T03:57:31.221162+0000 (oldest deadline 2024-09-21T03:57:56.521240+0000) 2024-09-21T03:58:43.725 INFO:journalctl@ceph.osd.10.smithi150.stdout:Sep 21 03:58:43 smithi150 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-10[58206]: 2024-09-21T03:58:43.413+0000 7f34020e1640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-09-21T03:57:40.022210+0000 front 2024-09-21T03:57:40.022157+0000 (oldest deadline 2024-09-21T03:58:01.722244+0000) 2024-09-21T03:58:43.725 INFO:journalctl@ceph.osd.10.smithi150.stdout:Sep 21 03:58:43 smithi150 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-10[58206]: 2024-09-21T03:58:43.413+0000 7f34020e1640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2024-09-21T03:57:46.323919+0000 front 2024-09-21T03:57:46.323891+0000 (oldest deadline 2024-09-21T03:58:10.423788+0000) 2024-09-21T03:58:43.725 INFO:journalctl@ceph.osd.10.smithi150.stdout:Sep 21 03:58:43 smithi150 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-10[58206]: 2024-09-21T03:58:43.413+0000 7f34020e1640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-09-21T03:57:56.225398+0000 front 2024-09-21T03:57:56.225320+0000 (oldest deadline 2024-09-21T03:58:20.325491+0000) 2024-09-21T03:58:43.725 INFO:journalctl@ceph.osd.10.smithi150.stdout:Sep 21 03:58:43 smithi150 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-10[58206]: 2024-09-21T03:58:43.413+0000 7f34020e1640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.161:6812 osd.5 since back 2024-09-21T03:58:03.727214+0000 front 2024-09-21T03:58:03.727152+0000 (oldest deadline 2024-09-21T03:58:26.027019+0000) 2024-09-21T03:58:43.725 INFO:journalctl@ceph.osd.10.smithi150.stdout:Sep 21 03:58:43 smithi150 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-10[58206]: 2024-09-21T03:58:43.413+0000 7f34020e1640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.39:6822 osd.6 since back 2024-09-21T03:58:11.129278+0000 front 2024-09-21T03:58:11.129260+0000 (oldest deadline 2024-09-21T03:58:36.429196+0000) 2024-09-21T03:58:44.468 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 21 03:58:44 smithi161 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-11[58829]: 2024-09-21T03:58:44.128+0000 7f538ef4a640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-09-21T03:57:21.581303+0000 front 2024-09-21T03:57:21.581333+0000 (oldest deadline 2024-09-21T03:57:46.881367+0000) 2024-09-21T03:58:44.468 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 21 03:58:44 smithi161 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-11[58829]: 2024-09-21T03:58:44.128+0000 7f538ef4a640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-09-21T03:57:32.782524+0000 front 2024-09-21T03:57:32.782318+0000 (oldest deadline 2024-09-21T03:57:54.482443+0000) 2024-09-21T03:58:44.468 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 21 03:58:44 smithi161 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-11[58829]: 2024-09-21T03:58:44.128+0000 7f538ef4a640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-09-21T03:57:40.184584+0000 front 2024-09-21T03:57:40.184544+0000 (oldest deadline 2024-09-21T03:58:05.484668+0000) 2024-09-21T03:58:44.468 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 21 03:58:44 smithi161 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-11[58829]: 2024-09-21T03:58:44.128+0000 7f538ef4a640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-09-21T03:57:55.387018+0000 front 2024-09-21T03:57:55.386996+0000 (oldest deadline 2024-09-21T03:58:20.086771+0000) 2024-09-21T03:58:44.468 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 21 03:58:44 smithi161 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-11[58829]: 2024-09-21T03:58:44.128+0000 7f538ef4a640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.161:6812 osd.5 since back 2024-09-21T03:58:02.887862+0000 front 2024-09-21T03:58:02.887979+0000 (oldest deadline 2024-09-21T03:58:27.587929+0000) 2024-09-21T03:58:44.468 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 21 03:58:44 smithi161 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-11[58829]: 2024-09-21T03:58:44.128+0000 7f538ef4a640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.39:6822 osd.6 since back 2024-09-21T03:58:09.888903+0000 front 2024-09-21T03:58:09.888888+0000 (oldest deadline 2024-09-21T03:58:35.188844+0000) 2024-09-21T03:58:44.469 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 21 03:58:44 smithi161 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-11[58829]: 2024-09-21T03:58:44.128+0000 7f538ef4a640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.150:6820 osd.7 since back 2024-09-21T03:58:17.489916+0000 front 2024-09-21T03:58:17.489821+0000 (oldest deadline 2024-09-21T03:58:42.189775+0000) 2024-09-21T03:58:44.725 INFO:journalctl@ceph.osd.10.smithi150.stdout:Sep 21 03:58:44 smithi150 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-10[58206]: 2024-09-21T03:58:44.417+0000 7f34020e1640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-09-21T03:57:23.120033+0000 front 2024-09-21T03:57:23.120109+0000 (oldest deadline 2024-09-21T03:57:45.920434+0000) 2024-09-21T03:58:44.725 INFO:journalctl@ceph.osd.10.smithi150.stdout:Sep 21 03:58:44 smithi150 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-10[58206]: 2024-09-21T03:58:44.417+0000 7f34020e1640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-09-21T03:57:31.221107+0000 front 2024-09-21T03:57:31.221162+0000 (oldest deadline 2024-09-21T03:57:56.521240+0000) 2024-09-21T03:58:44.725 INFO:journalctl@ceph.osd.10.smithi150.stdout:Sep 21 03:58:44 smithi150 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-10[58206]: 2024-09-21T03:58:44.417+0000 7f34020e1640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-09-21T03:57:40.022210+0000 front 2024-09-21T03:57:40.022157+0000 (oldest deadline 2024-09-21T03:58:01.722244+0000) 2024-09-21T03:58:44.725 INFO:journalctl@ceph.osd.10.smithi150.stdout:Sep 21 03:58:44 smithi150 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-10[58206]: 2024-09-21T03:58:44.417+0000 7f34020e1640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2024-09-21T03:57:46.323919+0000 front 2024-09-21T03:57:46.323891+0000 (oldest deadline 2024-09-21T03:58:10.423788+0000) 2024-09-21T03:58:44.725 INFO:journalctl@ceph.osd.10.smithi150.stdout:Sep 21 03:58:44 smithi150 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-10[58206]: 2024-09-21T03:58:44.417+0000 7f34020e1640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-09-21T03:57:56.225398+0000 front 2024-09-21T03:57:56.225320+0000 (oldest deadline 2024-09-21T03:58:20.325491+0000) 2024-09-21T03:58:44.725 INFO:journalctl@ceph.osd.10.smithi150.stdout:Sep 21 03:58:44 smithi150 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-10[58206]: 2024-09-21T03:58:44.417+0000 7f34020e1640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.161:6812 osd.5 since back 2024-09-21T03:58:03.727214+0000 front 2024-09-21T03:58:03.727152+0000 (oldest deadline 2024-09-21T03:58:26.027019+0000) 2024-09-21T03:58:44.725 INFO:journalctl@ceph.osd.10.smithi150.stdout:Sep 21 03:58:44 smithi150 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-10[58206]: 2024-09-21T03:58:44.417+0000 7f34020e1640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.39:6822 osd.6 since back 2024-09-21T03:58:11.129278+0000 front 2024-09-21T03:58:11.129260+0000 (oldest deadline 2024-09-21T03:58:36.429196+0000) 2024-09-21T03:58:45.225 INFO:journalctl@ceph.osd.10.smithi150.stdout:Sep 21 03:58:44 smithi150 podman[67701]: 2024-09-21 03:58:44.950554108 +0000 UTC m=+5.433840431 container died 5e25d378fe9b7851a11cf5d149572c7b753419f42baedeace7538671d6255b0a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-10, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, ceph=True, io.buildah.version=1.37.2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.schema-version=1.0) 2024-09-21T03:58:45.468 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 21 03:58:45 smithi161 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-11[58829]: 2024-09-21T03:58:45.129+0000 7f538ef4a640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-09-21T03:57:21.581303+0000 front 2024-09-21T03:57:21.581333+0000 (oldest deadline 2024-09-21T03:57:46.881367+0000) 2024-09-21T03:58:45.468 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 21 03:58:45 smithi161 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-11[58829]: 2024-09-21T03:58:45.129+0000 7f538ef4a640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-09-21T03:57:32.782524+0000 front 2024-09-21T03:57:32.782318+0000 (oldest deadline 2024-09-21T03:57:54.482443+0000) 2024-09-21T03:58:45.468 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 21 03:58:45 smithi161 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-11[58829]: 2024-09-21T03:58:45.129+0000 7f538ef4a640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-09-21T03:57:40.184584+0000 front 2024-09-21T03:57:40.184544+0000 (oldest deadline 2024-09-21T03:58:05.484668+0000) 2024-09-21T03:58:45.468 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 21 03:58:45 smithi161 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-11[58829]: 2024-09-21T03:58:45.129+0000 7f538ef4a640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-09-21T03:57:55.387018+0000 front 2024-09-21T03:57:55.386996+0000 (oldest deadline 2024-09-21T03:58:20.086771+0000) 2024-09-21T03:58:45.468 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 21 03:58:45 smithi161 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-11[58829]: 2024-09-21T03:58:45.129+0000 7f538ef4a640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.161:6812 osd.5 since back 2024-09-21T03:58:02.887862+0000 front 2024-09-21T03:58:02.887979+0000 (oldest deadline 2024-09-21T03:58:27.587929+0000) 2024-09-21T03:58:45.469 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 21 03:58:45 smithi161 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-11[58829]: 2024-09-21T03:58:45.129+0000 7f538ef4a640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.39:6822 osd.6 since back 2024-09-21T03:58:09.888903+0000 front 2024-09-21T03:58:09.888888+0000 (oldest deadline 2024-09-21T03:58:35.188844+0000) 2024-09-21T03:58:45.469 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 21 03:58:45 smithi161 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-11[58829]: 2024-09-21T03:58:45.129+0000 7f538ef4a640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.150:6820 osd.7 since back 2024-09-21T03:58:17.489916+0000 front 2024-09-21T03:58:17.489821+0000 (oldest deadline 2024-09-21T03:58:42.189775+0000) 2024-09-21T03:58:45.725 INFO:journalctl@ceph.osd.10.smithi150.stdout:Sep 21 03:58:45 smithi150 podman[67701]: 2024-09-21 03:58:45.25378924 +0000 UTC m=+5.737075562 container cleanup 5e25d378fe9b7851a11cf5d149572c7b753419f42baedeace7538671d6255b0a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-10, org.label-schema.build-date=20240731, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS) 2024-09-21T03:58:45.725 INFO:journalctl@ceph.osd.10.smithi150.stdout:Sep 21 03:58:45 smithi150 bash[67701]: ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-10 2024-09-21T03:58:45.725 INFO:journalctl@ceph.osd.10.smithi150.stdout:Sep 21 03:58:45 smithi150 podman[67720]: 2024-09-21 03:58:45.479922703 +0000 UTC m=+0.521500056 container remove 5e25d378fe9b7851a11cf5d149572c7b753419f42baedeace7538671d6255b0a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-10, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221) 2024-09-21T03:58:46.109 INFO:journalctl@ceph.osd.10.smithi150.stdout:Sep 21 03:58:45 smithi150 podman[67805]: 2024-09-21 03:58:45.694580603 +0000 UTC m=+0.019634287 image pull ddce307e1f3263fb077b4f3902e15660050631dad4d2823a20ee3f72bf18d977 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262 2024-09-21T03:58:46.109 INFO:journalctl@ceph.osd.10.smithi150.stdout:Sep 21 03:58:45 smithi150 podman[67805]: 2024-09-21 03:58:45.814125464 +0000 UTC m=+0.139179177 container create 18252a5fee3fd7d593a8998a80f4d27d8e60ac64e0b30e77f74f7218148de93f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-10-deactivate, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, ceph=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-21T03:58:46.110 INFO:journalctl@ceph.osd.10.smithi150.stdout:Sep 21 03:58:46 smithi150 podman[67805]: 2024-09-21 03:58:46.033622752 +0000 UTC m=+0.358676432 container init 18252a5fee3fd7d593a8998a80f4d27d8e60ac64e0b30e77f74f7218148de93f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-10-deactivate, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_BRANCH=HEAD, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23) 2024-09-21T03:58:46.110 INFO:journalctl@ceph.osd.10.smithi150.stdout:Sep 21 03:58:46 smithi150 podman[67805]: 2024-09-21 03:58:46.03723557 +0000 UTC m=+0.362289258 container start 18252a5fee3fd7d593a8998a80f4d27d8e60ac64e0b30e77f74f7218148de93f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-10-deactivate, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, ceph=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23) 2024-09-21T03:58:46.110 INFO:journalctl@ceph.osd.10.smithi150.stdout:Sep 21 03:58:46 smithi150 podman[67805]: 2024-09-21 03:58:46.107956219 +0000 UTC m=+0.433009891 container attach 18252a5fee3fd7d593a8998a80f4d27d8e60ac64e0b30e77f74f7218148de93f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-10-deactivate, io.buildah.version=1.37.2, GIT_CLEAN=True, org.label-schema.license=GPLv2, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.build-date=20240731, ceph=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-21T03:58:46.468 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 21 03:58:46 smithi161 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-11[58829]: 2024-09-21T03:58:46.085+0000 7f538ef4a640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-09-21T03:57:21.581303+0000 front 2024-09-21T03:57:21.581333+0000 (oldest deadline 2024-09-21T03:57:46.881367+0000) 2024-09-21T03:58:46.468 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 21 03:58:46 smithi161 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-11[58829]: 2024-09-21T03:58:46.085+0000 7f538ef4a640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-09-21T03:57:32.782524+0000 front 2024-09-21T03:57:32.782318+0000 (oldest deadline 2024-09-21T03:57:54.482443+0000) 2024-09-21T03:58:46.468 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 21 03:58:46 smithi161 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-11[58829]: 2024-09-21T03:58:46.085+0000 7f538ef4a640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-09-21T03:57:40.184584+0000 front 2024-09-21T03:57:40.184544+0000 (oldest deadline 2024-09-21T03:58:05.484668+0000) 2024-09-21T03:58:46.468 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 21 03:58:46 smithi161 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-11[58829]: 2024-09-21T03:58:46.085+0000 7f538ef4a640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-09-21T03:57:55.387018+0000 front 2024-09-21T03:57:55.386996+0000 (oldest deadline 2024-09-21T03:58:20.086771+0000) 2024-09-21T03:58:46.468 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 21 03:58:46 smithi161 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-11[58829]: 2024-09-21T03:58:46.085+0000 7f538ef4a640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.161:6812 osd.5 since back 2024-09-21T03:58:02.887862+0000 front 2024-09-21T03:58:02.887979+0000 (oldest deadline 2024-09-21T03:58:27.587929+0000) 2024-09-21T03:58:46.469 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 21 03:58:46 smithi161 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-11[58829]: 2024-09-21T03:58:46.085+0000 7f538ef4a640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.39:6822 osd.6 since back 2024-09-21T03:58:09.888903+0000 front 2024-09-21T03:58:09.888888+0000 (oldest deadline 2024-09-21T03:58:35.188844+0000) 2024-09-21T03:58:46.469 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 21 03:58:46 smithi161 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-11[58829]: 2024-09-21T03:58:46.085+0000 7f538ef4a640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.150:6820 osd.7 since back 2024-09-21T03:58:17.489916+0000 front 2024-09-21T03:58:17.489821+0000 (oldest deadline 2024-09-21T03:58:42.189775+0000) 2024-09-21T03:58:46.474 INFO:journalctl@ceph.osd.10.smithi150.stdout:Sep 21 03:58:46 smithi150 podman[67805]: 2024-09-21 03:58:46.22615268 +0000 UTC m=+0.551206362 container died 18252a5fee3fd7d593a8998a80f4d27d8e60ac64e0b30e77f74f7218148de93f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-10-deactivate, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS) 2024-09-21T03:58:46.843 DEBUG:teuthology.orchestra.run.smithi150:> sudo pkill -f 'journalctl -f -n 0 -u ceph-51116d5c-77ca-11ef-baf5-efdc52797490@osd.10.service' 2024-09-21T03:58:46.880 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-21T03:58:46.881 INFO:tasks.cephadm.osd.10:Stopped osd.10 2024-09-21T03:58:46.881 INFO:tasks.cephadm.osd.11:Stopping osd.11... 2024-09-21T03:58:46.881 DEBUG:teuthology.orchestra.run.smithi161:> sudo systemctl stop ceph-51116d5c-77ca-11ef-baf5-efdc52797490@osd.11 2024-09-21T03:58:47.191 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 21 03:58:46 smithi161 systemd[1]: Stopping Ceph osd.11 for 51116d5c-77ca-11ef-baf5-efdc52797490... 2024-09-21T03:58:47.191 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 21 03:58:47 smithi161 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-11[58829]: 2024-09-21T03:58:47.067+0000 7f538ef4a640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-09-21T03:57:21.581303+0000 front 2024-09-21T03:57:21.581333+0000 (oldest deadline 2024-09-21T03:57:46.881367+0000) 2024-09-21T03:58:47.191 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 21 03:58:47 smithi161 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-11[58829]: 2024-09-21T03:58:47.067+0000 7f538ef4a640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-09-21T03:57:32.782524+0000 front 2024-09-21T03:57:32.782318+0000 (oldest deadline 2024-09-21T03:57:54.482443+0000) 2024-09-21T03:58:47.191 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 21 03:58:47 smithi161 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-11[58829]: 2024-09-21T03:58:47.067+0000 7f538ef4a640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-09-21T03:57:40.184584+0000 front 2024-09-21T03:57:40.184544+0000 (oldest deadline 2024-09-21T03:58:05.484668+0000) 2024-09-21T03:58:47.191 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 21 03:58:47 smithi161 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-11[58829]: 2024-09-21T03:58:47.067+0000 7f538ef4a640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-09-21T03:57:55.387018+0000 front 2024-09-21T03:57:55.386996+0000 (oldest deadline 2024-09-21T03:58:20.086771+0000) 2024-09-21T03:58:47.192 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 21 03:58:47 smithi161 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-11[58829]: 2024-09-21T03:58:47.067+0000 7f538ef4a640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.161:6812 osd.5 since back 2024-09-21T03:58:02.887862+0000 front 2024-09-21T03:58:02.887979+0000 (oldest deadline 2024-09-21T03:58:27.587929+0000) 2024-09-21T03:58:47.192 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 21 03:58:47 smithi161 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-11[58829]: 2024-09-21T03:58:47.067+0000 7f538ef4a640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.39:6822 osd.6 since back 2024-09-21T03:58:09.888903+0000 front 2024-09-21T03:58:09.888888+0000 (oldest deadline 2024-09-21T03:58:35.188844+0000) 2024-09-21T03:58:47.192 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 21 03:58:47 smithi161 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-11[58829]: 2024-09-21T03:58:47.067+0000 7f538ef4a640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.150:6820 osd.7 since back 2024-09-21T03:58:17.489916+0000 front 2024-09-21T03:58:17.489821+0000 (oldest deadline 2024-09-21T03:58:42.189775+0000) 2024-09-21T03:58:47.192 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 21 03:58:47 smithi161 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-11[58829]: 2024-09-21T03:58:47.188+0000 7f539295f640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.11 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-21T03:58:47.192 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 21 03:58:47 smithi161 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-11[58829]: 2024-09-21T03:58:47.188+0000 7f539295f640 -1 osd.11 90 *** Got signal Terminated *** 2024-09-21T03:58:47.192 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 21 03:58:47 smithi161 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-11[58829]: 2024-09-21T03:58:47.188+0000 7f539295f640 -1 osd.11 90 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-21T03:58:48.468 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 21 03:58:48 smithi161 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-11[58829]: 2024-09-21T03:58:48.071+0000 7f538ef4a640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-09-21T03:57:21.581303+0000 front 2024-09-21T03:57:21.581333+0000 (oldest deadline 2024-09-21T03:57:46.881367+0000) 2024-09-21T03:58:48.468 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 21 03:58:48 smithi161 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-11[58829]: 2024-09-21T03:58:48.071+0000 7f538ef4a640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-09-21T03:57:32.782524+0000 front 2024-09-21T03:57:32.782318+0000 (oldest deadline 2024-09-21T03:57:54.482443+0000) 2024-09-21T03:58:48.468 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 21 03:58:48 smithi161 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-11[58829]: 2024-09-21T03:58:48.071+0000 7f538ef4a640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-09-21T03:57:40.184584+0000 front 2024-09-21T03:57:40.184544+0000 (oldest deadline 2024-09-21T03:58:05.484668+0000) 2024-09-21T03:58:48.468 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 21 03:58:48 smithi161 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-11[58829]: 2024-09-21T03:58:48.071+0000 7f538ef4a640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-09-21T03:57:55.387018+0000 front 2024-09-21T03:57:55.386996+0000 (oldest deadline 2024-09-21T03:58:20.086771+0000) 2024-09-21T03:58:48.468 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 21 03:58:48 smithi161 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-11[58829]: 2024-09-21T03:58:48.071+0000 7f538ef4a640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.161:6812 osd.5 since back 2024-09-21T03:58:02.887862+0000 front 2024-09-21T03:58:02.887979+0000 (oldest deadline 2024-09-21T03:58:27.587929+0000) 2024-09-21T03:58:48.468 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 21 03:58:48 smithi161 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-11[58829]: 2024-09-21T03:58:48.071+0000 7f538ef4a640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.39:6822 osd.6 since back 2024-09-21T03:58:09.888903+0000 front 2024-09-21T03:58:09.888888+0000 (oldest deadline 2024-09-21T03:58:35.188844+0000) 2024-09-21T03:58:48.468 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 21 03:58:48 smithi161 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-11[58829]: 2024-09-21T03:58:48.071+0000 7f538ef4a640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.150:6820 osd.7 since back 2024-09-21T03:58:17.489916+0000 front 2024-09-21T03:58:17.489821+0000 (oldest deadline 2024-09-21T03:58:42.189775+0000) 2024-09-21T03:58:49.468 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 21 03:58:49 smithi161 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-11[58829]: 2024-09-21T03:58:49.067+0000 7f538ef4a640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-09-21T03:57:21.581303+0000 front 2024-09-21T03:57:21.581333+0000 (oldest deadline 2024-09-21T03:57:46.881367+0000) 2024-09-21T03:58:49.468 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 21 03:58:49 smithi161 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-11[58829]: 2024-09-21T03:58:49.067+0000 7f538ef4a640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-09-21T03:57:32.782524+0000 front 2024-09-21T03:57:32.782318+0000 (oldest deadline 2024-09-21T03:57:54.482443+0000) 2024-09-21T03:58:49.468 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 21 03:58:49 smithi161 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-11[58829]: 2024-09-21T03:58:49.067+0000 7f538ef4a640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-09-21T03:57:40.184584+0000 front 2024-09-21T03:57:40.184544+0000 (oldest deadline 2024-09-21T03:58:05.484668+0000) 2024-09-21T03:58:49.468 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 21 03:58:49 smithi161 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-11[58829]: 2024-09-21T03:58:49.067+0000 7f538ef4a640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-09-21T03:57:55.387018+0000 front 2024-09-21T03:57:55.386996+0000 (oldest deadline 2024-09-21T03:58:20.086771+0000) 2024-09-21T03:58:49.468 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 21 03:58:49 smithi161 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-11[58829]: 2024-09-21T03:58:49.067+0000 7f538ef4a640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.161:6812 osd.5 since back 2024-09-21T03:58:02.887862+0000 front 2024-09-21T03:58:02.887979+0000 (oldest deadline 2024-09-21T03:58:27.587929+0000) 2024-09-21T03:58:49.468 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 21 03:58:49 smithi161 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-11[58829]: 2024-09-21T03:58:49.067+0000 7f538ef4a640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.39:6822 osd.6 since back 2024-09-21T03:58:09.888903+0000 front 2024-09-21T03:58:09.888888+0000 (oldest deadline 2024-09-21T03:58:35.188844+0000) 2024-09-21T03:58:49.468 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 21 03:58:49 smithi161 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-11[58829]: 2024-09-21T03:58:49.067+0000 7f538ef4a640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.150:6820 osd.7 since back 2024-09-21T03:58:17.489916+0000 front 2024-09-21T03:58:17.489821+0000 (oldest deadline 2024-09-21T03:58:42.189775+0000) 2024-09-21T03:58:50.468 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 21 03:58:50 smithi161 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-11[58829]: 2024-09-21T03:58:50.020+0000 7f538ef4a640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-09-21T03:57:21.581303+0000 front 2024-09-21T03:57:21.581333+0000 (oldest deadline 2024-09-21T03:57:46.881367+0000) 2024-09-21T03:58:50.468 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 21 03:58:50 smithi161 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-11[58829]: 2024-09-21T03:58:50.020+0000 7f538ef4a640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-09-21T03:57:32.782524+0000 front 2024-09-21T03:57:32.782318+0000 (oldest deadline 2024-09-21T03:57:54.482443+0000) 2024-09-21T03:58:50.468 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 21 03:58:50 smithi161 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-11[58829]: 2024-09-21T03:58:50.020+0000 7f538ef4a640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-09-21T03:57:40.184584+0000 front 2024-09-21T03:57:40.184544+0000 (oldest deadline 2024-09-21T03:58:05.484668+0000) 2024-09-21T03:58:50.468 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 21 03:58:50 smithi161 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-11[58829]: 2024-09-21T03:58:50.020+0000 7f538ef4a640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-09-21T03:57:55.387018+0000 front 2024-09-21T03:57:55.386996+0000 (oldest deadline 2024-09-21T03:58:20.086771+0000) 2024-09-21T03:58:50.468 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 21 03:58:50 smithi161 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-11[58829]: 2024-09-21T03:58:50.020+0000 7f538ef4a640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.161:6812 osd.5 since back 2024-09-21T03:58:02.887862+0000 front 2024-09-21T03:58:02.887979+0000 (oldest deadline 2024-09-21T03:58:27.587929+0000) 2024-09-21T03:58:50.468 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 21 03:58:50 smithi161 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-11[58829]: 2024-09-21T03:58:50.020+0000 7f538ef4a640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.39:6822 osd.6 since back 2024-09-21T03:58:09.888903+0000 front 2024-09-21T03:58:09.888888+0000 (oldest deadline 2024-09-21T03:58:35.188844+0000) 2024-09-21T03:58:50.468 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 21 03:58:50 smithi161 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-11[58829]: 2024-09-21T03:58:50.020+0000 7f538ef4a640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.150:6820 osd.7 since back 2024-09-21T03:58:17.489916+0000 front 2024-09-21T03:58:17.489821+0000 (oldest deadline 2024-09-21T03:58:42.189775+0000) 2024-09-21T03:58:50.469 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 21 03:58:50 smithi161 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-11[58829]: 2024-09-21T03:58:50.020+0000 7f538ef4a640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.161:6820 osd.8 since back 2024-09-21T03:58:25.090697+0000 front 2024-09-21T03:58:25.090656+0000 (oldest deadline 2024-09-21T03:58:49.190728+0000) 2024-09-21T03:58:51.421 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 21 03:58:51 smithi161 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-11[58829]: 2024-09-21T03:58:51.042+0000 7f538ef4a640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-09-21T03:57:21.581303+0000 front 2024-09-21T03:57:21.581333+0000 (oldest deadline 2024-09-21T03:57:46.881367+0000) 2024-09-21T03:58:51.421 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 21 03:58:51 smithi161 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-11[58829]: 2024-09-21T03:58:51.042+0000 7f538ef4a640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-09-21T03:57:32.782524+0000 front 2024-09-21T03:57:32.782318+0000 (oldest deadline 2024-09-21T03:57:54.482443+0000) 2024-09-21T03:58:51.421 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 21 03:58:51 smithi161 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-11[58829]: 2024-09-21T03:58:51.042+0000 7f538ef4a640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-09-21T03:57:40.184584+0000 front 2024-09-21T03:57:40.184544+0000 (oldest deadline 2024-09-21T03:58:05.484668+0000) 2024-09-21T03:58:51.421 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 21 03:58:51 smithi161 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-11[58829]: 2024-09-21T03:58:51.042+0000 7f538ef4a640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-09-21T03:57:55.387018+0000 front 2024-09-21T03:57:55.386996+0000 (oldest deadline 2024-09-21T03:58:20.086771+0000) 2024-09-21T03:58:51.421 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 21 03:58:51 smithi161 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-11[58829]: 2024-09-21T03:58:51.042+0000 7f538ef4a640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.161:6812 osd.5 since back 2024-09-21T03:58:02.887862+0000 front 2024-09-21T03:58:02.887979+0000 (oldest deadline 2024-09-21T03:58:27.587929+0000) 2024-09-21T03:58:51.421 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 21 03:58:51 smithi161 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-11[58829]: 2024-09-21T03:58:51.042+0000 7f538ef4a640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.39:6822 osd.6 since back 2024-09-21T03:58:09.888903+0000 front 2024-09-21T03:58:09.888888+0000 (oldest deadline 2024-09-21T03:58:35.188844+0000) 2024-09-21T03:58:51.421 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 21 03:58:51 smithi161 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-11[58829]: 2024-09-21T03:58:51.042+0000 7f538ef4a640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.150:6820 osd.7 since back 2024-09-21T03:58:17.489916+0000 front 2024-09-21T03:58:17.489821+0000 (oldest deadline 2024-09-21T03:58:42.189775+0000) 2024-09-21T03:58:51.421 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 21 03:58:51 smithi161 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-11[58829]: 2024-09-21T03:58:51.042+0000 7f538ef4a640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.161:6820 osd.8 since back 2024-09-21T03:58:25.090697+0000 front 2024-09-21T03:58:25.090656+0000 (oldest deadline 2024-09-21T03:58:49.190728+0000) 2024-09-21T03:58:52.468 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 21 03:58:52 smithi161 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-11[58829]: 2024-09-21T03:58:52.017+0000 7f538ef4a640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-09-21T03:57:21.581303+0000 front 2024-09-21T03:57:21.581333+0000 (oldest deadline 2024-09-21T03:57:46.881367+0000) 2024-09-21T03:58:52.468 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 21 03:58:52 smithi161 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-11[58829]: 2024-09-21T03:58:52.017+0000 7f538ef4a640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-09-21T03:57:32.782524+0000 front 2024-09-21T03:57:32.782318+0000 (oldest deadline 2024-09-21T03:57:54.482443+0000) 2024-09-21T03:58:52.468 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 21 03:58:52 smithi161 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-11[58829]: 2024-09-21T03:58:52.017+0000 7f538ef4a640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-09-21T03:57:40.184584+0000 front 2024-09-21T03:57:40.184544+0000 (oldest deadline 2024-09-21T03:58:05.484668+0000) 2024-09-21T03:58:52.469 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 21 03:58:52 smithi161 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-11[58829]: 2024-09-21T03:58:52.017+0000 7f538ef4a640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-09-21T03:57:55.387018+0000 front 2024-09-21T03:57:55.386996+0000 (oldest deadline 2024-09-21T03:58:20.086771+0000) 2024-09-21T03:58:52.469 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 21 03:58:52 smithi161 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-11[58829]: 2024-09-21T03:58:52.017+0000 7f538ef4a640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.161:6812 osd.5 since back 2024-09-21T03:58:02.887862+0000 front 2024-09-21T03:58:02.887979+0000 (oldest deadline 2024-09-21T03:58:27.587929+0000) 2024-09-21T03:58:52.469 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 21 03:58:52 smithi161 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-11[58829]: 2024-09-21T03:58:52.017+0000 7f538ef4a640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.39:6822 osd.6 since back 2024-09-21T03:58:09.888903+0000 front 2024-09-21T03:58:09.888888+0000 (oldest deadline 2024-09-21T03:58:35.188844+0000) 2024-09-21T03:58:52.469 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 21 03:58:52 smithi161 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-11[58829]: 2024-09-21T03:58:52.017+0000 7f538ef4a640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.150:6820 osd.7 since back 2024-09-21T03:58:17.489916+0000 front 2024-09-21T03:58:17.489821+0000 (oldest deadline 2024-09-21T03:58:42.189775+0000) 2024-09-21T03:58:52.469 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 21 03:58:52 smithi161 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-11[58829]: 2024-09-21T03:58:52.017+0000 7f538ef4a640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.161:6820 osd.8 since back 2024-09-21T03:58:25.090697+0000 front 2024-09-21T03:58:25.090656+0000 (oldest deadline 2024-09-21T03:58:49.190728+0000) 2024-09-21T03:58:52.916 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 21 03:58:52 smithi161 podman[68166]: 2024-09-21 03:58:52.651957293 +0000 UTC m=+5.574411206 container died 3b22bc180075782a74d55c073e6d54726dcab2ee68647a9b2e97eacb10f096b9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-11, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, io.buildah.version=1.37.2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0) 2024-09-21T03:58:53.218 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 21 03:58:53 smithi161 podman[68166]: 2024-09-21 03:58:53.013519156 +0000 UTC m=+5.935973072 container cleanup 3b22bc180075782a74d55c073e6d54726dcab2ee68647a9b2e97eacb10f096b9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-11, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-21T03:58:53.218 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 21 03:58:53 smithi161 bash[68166]: ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-11 2024-09-21T03:58:53.625 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 21 03:58:53 smithi161 podman[68181]: 2024-09-21 03:58:53.306031869 +0000 UTC m=+0.648174135 container remove 3b22bc180075782a74d55c073e6d54726dcab2ee68647a9b2e97eacb10f096b9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-11, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, ceph=True, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=) 2024-09-21T03:58:53.952 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 21 03:58:53 smithi161 podman[68273]: 2024-09-21 03:58:53.623629366 +0000 UTC m=+0.104408439 container create 728f33d63baf02c0dfedd96c34efb2ee8fa8899eab6b99ae38a8f877587b14b9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-11-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, GIT_BRANCH=HEAD, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.build-date=20240731, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.vendor=CentOS) 2024-09-21T03:58:53.952 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 21 03:58:53 smithi161 podman[68273]: 2024-09-21 03:58:53.53512669 +0000 UTC m=+0.015905739 image pull ddce307e1f3263fb077b4f3902e15660050631dad4d2823a20ee3f72bf18d977 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262 2024-09-21T03:58:53.952 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 21 03:58:53 smithi161 podman[68273]: 2024-09-21 03:58:53.867622414 +0000 UTC m=+0.348401445 container init 728f33d63baf02c0dfedd96c34efb2ee8fa8899eab6b99ae38a8f877587b14b9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-11-deactivate, org.label-schema.license=GPLv2, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , ceph=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-21T03:58:53.952 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 21 03:58:53 smithi161 podman[68273]: 2024-09-21 03:58:53.870947224 +0000 UTC m=+0.351726260 container start 728f33d63baf02c0dfedd96c34efb2ee8fa8899eab6b99ae38a8f877587b14b9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-11-deactivate, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True) 2024-09-21T03:58:53.952 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 21 03:58:53 smithi161 podman[68273]: 2024-09-21 03:58:53.950760445 +0000 UTC m=+0.431539480 container attach 728f33d63baf02c0dfedd96c34efb2ee8fa8899eab6b99ae38a8f877587b14b9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-11-deactivate, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_CLEAN=True, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2) 2024-09-21T03:58:54.218 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 21 03:58:54 smithi161 podman[68273]: 2024-09-21 03:58:54.055357182 +0000 UTC m=+0.536136224 container died 728f33d63baf02c0dfedd96c34efb2ee8fa8899eab6b99ae38a8f877587b14b9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-51116d5c-77ca-11ef-baf5-efdc52797490-osd-11-deactivate, ceph=True, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_BRANCH=HEAD) 2024-09-21T03:58:54.640 DEBUG:teuthology.orchestra.run.smithi161:> sudo pkill -f 'journalctl -f -n 0 -u ceph-51116d5c-77ca-11ef-baf5-efdc52797490@osd.11.service' 2024-09-21T03:58:54.679 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-21T03:58:54.679 INFO:tasks.cephadm.osd.11:Stopped osd.11 2024-09-21T03:58:54.679 INFO:tasks.cephadm.mds.a:Stopping mds.a... 2024-09-21T03:58:54.679 DEBUG:teuthology.orchestra.run.smithi039:> sudo systemctl stop ceph-51116d5c-77ca-11ef-baf5-efdc52797490@mds.a 2024-09-21T03:58:54.977 INFO:journalctl@ceph.mds.a.smithi039.stdout:Sep 21 03:58:54 smithi039 systemd[1]: Stopping Ceph mds.a for 51116d5c-77ca-11ef-baf5-efdc52797490... 2024-09-21T03:58:55.334 INFO:journalctl@ceph.mds.a.smithi039.stdout:Sep 21 03:58:54 smithi039 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-mds-a[73576]: 2024-09-21T03:58:54.975+0000 7f1de5356640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-21T03:58:55.334 INFO:journalctl@ceph.mds.a.smithi039.stdout:Sep 21 03:58:54 smithi039 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-mds-a[73576]: 2024-09-21T03:58:54.975+0000 7f1de5356640 -1 mds.a *** got signal Terminated *** 2024-09-21T03:58:56.334 INFO:journalctl@ceph.mds.a.smithi039.stdout:Sep 21 03:58:56 smithi039 podman[113128]: 2024-09-21 03:58:56.031050465 +0000 UTC m=+1.153833009 container died 4d6edea429af090cfd57687082a277ab3490d8152c67db4f10ef4aac87969fb9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-51116d5c-77ca-11ef-baf5-efdc52797490-mds-a, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-21T03:58:56.736 INFO:journalctl@ceph.mds.a.smithi039.stdout:Sep 21 03:58:56 smithi039 podman[113128]: 2024-09-21 03:58:56.433559755 +0000 UTC m=+1.556342306 container cleanup 4d6edea429af090cfd57687082a277ab3490d8152c67db4f10ef4aac87969fb9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-51116d5c-77ca-11ef-baf5-efdc52797490-mds-a, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, io.buildah.version=1.37.2) 2024-09-21T03:58:56.736 INFO:journalctl@ceph.mds.a.smithi039.stdout:Sep 21 03:58:56 smithi039 bash[113128]: ceph-51116d5c-77ca-11ef-baf5-efdc52797490-mds-a 2024-09-21T03:58:56.877 DEBUG:teuthology.orchestra.run.smithi039:> sudo pkill -f 'journalctl -f -n 0 -u ceph-51116d5c-77ca-11ef-baf5-efdc52797490@mds.a.service' 2024-09-21T03:58:56.912 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-21T03:58:56.913 INFO:tasks.cephadm.mds.a:Stopped mds.a 2024-09-21T03:58:56.913 INFO:tasks.cephadm.mds.d:Stopping mds.d... 2024-09-21T03:58:56.913 DEBUG:teuthology.orchestra.run.smithi039:> sudo systemctl stop ceph-51116d5c-77ca-11ef-baf5-efdc52797490@mds.d 2024-09-21T03:58:57.334 INFO:journalctl@ceph.mds.d.smithi039.stdout:Sep 21 03:58:57 smithi039 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-mds-d[72460]: 2024-09-21T03:58:57.220+0000 7f32ad248640 -1 mds.d *** got signal Terminated *** 2024-09-21T03:58:59.948 INFO:journalctl@ceph.mds.d.smithi039.stdout:Sep 21 03:58:59 smithi039 podman[113250]: 2024-09-21 03:58:59.660635158 +0000 UTC m=+2.529045016 container died d295f08191b8d7dd4df379f216095354cf22c404398a83cd7f1ac861d3d5aa17 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-51116d5c-77ca-11ef-baf5-efdc52797490-mds-d, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.schema-version=1.0, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, ceph=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=) 2024-09-21T03:59:00.281 INFO:journalctl@ceph.mds.d.smithi039.stdout:Sep 21 03:59:00 smithi039 podman[113250]: 2024-09-21 03:59:00.028858522 +0000 UTC m=+2.897268372 container cleanup d295f08191b8d7dd4df379f216095354cf22c404398a83cd7f1ac861d3d5aa17 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-51116d5c-77ca-11ef-baf5-efdc52797490-mds-d, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, ceph=True, org.label-schema.license=GPLv2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_CLEAN=True) 2024-09-21T03:59:00.282 INFO:journalctl@ceph.mds.d.smithi039.stdout:Sep 21 03:59:00 smithi039 bash[113250]: ceph-51116d5c-77ca-11ef-baf5-efdc52797490-mds-d 2024-09-21T03:59:00.427 DEBUG:teuthology.orchestra.run.smithi039:> sudo pkill -f 'journalctl -f -n 0 -u ceph-51116d5c-77ca-11ef-baf5-efdc52797490@mds.d.service' 2024-09-21T03:59:00.461 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-21T03:59:00.461 INFO:tasks.cephadm.mds.d:Stopped mds.d 2024-09-21T03:59:00.461 INFO:tasks.cephadm.mds.g:Stopping mds.g... 2024-09-21T03:59:00.461 DEBUG:teuthology.orchestra.run.smithi039:> sudo systemctl stop ceph-51116d5c-77ca-11ef-baf5-efdc52797490@mds.g 2024-09-21T03:59:00.834 INFO:journalctl@ceph.mds.g.smithi039.stdout:Sep 21 03:59:00 smithi039 systemd[1]: Stopping Ceph mds.g for 51116d5c-77ca-11ef-baf5-efdc52797490... 2024-09-21T03:59:00.834 INFO:journalctl@ceph.mds.g.smithi039.stdout:Sep 21 03:59:00 smithi039 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-mds-g[73194]: 2024-09-21T03:59:00.782+0000 7fb545c0d640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.g -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-21T03:59:00.834 INFO:journalctl@ceph.mds.g.smithi039.stdout:Sep 21 03:59:00 smithi039 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-mds-g[73194]: 2024-09-21T03:59:00.782+0000 7fb545c0d640 -1 mds.g *** got signal Terminated *** 2024-09-21T03:59:04.334 INFO:journalctl@ceph.mds.g.smithi039.stdout:Sep 21 03:59:04 smithi039 podman[113373]: 2024-09-21 03:59:04.032613718 +0000 UTC m=+3.350045211 container died c75c7c7aee9065a806a56bf27d116525d838d15b97a3cc62edffaefaa579ddb9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-51116d5c-77ca-11ef-baf5-efdc52797490-mds-g, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0) 2024-09-21T03:59:04.685 INFO:journalctl@ceph.mds.g.smithi039.stdout:Sep 21 03:59:04 smithi039 podman[113373]: 2024-09-21 03:59:04.424534274 +0000 UTC m=+3.741965768 container cleanup c75c7c7aee9065a806a56bf27d116525d838d15b97a3cc62edffaefaa579ddb9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-51116d5c-77ca-11ef-baf5-efdc52797490-mds-g, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , ceph=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23) 2024-09-21T03:59:04.685 INFO:journalctl@ceph.mds.g.smithi039.stdout:Sep 21 03:59:04 smithi039 bash[113373]: ceph-51116d5c-77ca-11ef-baf5-efdc52797490-mds-g 2024-09-21T03:59:04.834 DEBUG:teuthology.orchestra.run.smithi039:> sudo pkill -f 'journalctl -f -n 0 -u ceph-51116d5c-77ca-11ef-baf5-efdc52797490@mds.g.service' 2024-09-21T03:59:04.868 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-21T03:59:04.868 INFO:tasks.cephadm.mds.g:Stopped mds.g 2024-09-21T03:59:04.868 INFO:tasks.cephadm.mds.j:Stopping mds.j... 2024-09-21T03:59:04.868 DEBUG:teuthology.orchestra.run.smithi039:> sudo systemctl stop ceph-51116d5c-77ca-11ef-baf5-efdc52797490@mds.j 2024-09-21T03:59:05.334 INFO:journalctl@ceph.mds.j.smithi039.stdout:Sep 21 03:59:04 smithi039 systemd[1]: Stopping Ceph mds.j for 51116d5c-77ca-11ef-baf5-efdc52797490... 2024-09-21T03:59:05.334 INFO:journalctl@ceph.mds.j.smithi039.stdout:Sep 21 03:59:05 smithi039 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-mds-j[72066]: 2024-09-21T03:59:05.186+0000 7ff192fa6640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.j -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-21T03:59:05.334 INFO:journalctl@ceph.mds.j.smithi039.stdout:Sep 21 03:59:05 smithi039 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-mds-j[72066]: 2024-09-21T03:59:05.186+0000 7ff192fa6640 -1 mds.j *** got signal Terminated *** 2024-09-21T03:59:08.430 INFO:journalctl@ceph.mds.j.smithi039.stdout:Sep 21 03:59:08 smithi039 podman[113493]: 2024-09-21 03:59:08.162435697 +0000 UTC m=+3.070721156 container died ae337640ab42027fb7f045b75f267ec7469d17b989cf80ed21a54070ca3153e9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-51116d5c-77ca-11ef-baf5-efdc52797490-mds-j, maintainer=Guillaume Abrioux , RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, ceph=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2) 2024-09-21T03:59:08.834 INFO:journalctl@ceph.mds.j.smithi039.stdout:Sep 21 03:59:08 smithi039 podman[113493]: 2024-09-21 03:59:08.61169364 +0000 UTC m=+3.519979130 container cleanup ae337640ab42027fb7f045b75f267ec7469d17b989cf80ed21a54070ca3153e9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-51116d5c-77ca-11ef-baf5-efdc52797490-mds-j, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, CEPH_POINT_RELEASE=) 2024-09-21T03:59:08.834 INFO:journalctl@ceph.mds.j.smithi039.stdout:Sep 21 03:59:08 smithi039 bash[113493]: ceph-51116d5c-77ca-11ef-baf5-efdc52797490-mds-j 2024-09-21T03:59:09.009 DEBUG:teuthology.orchestra.run.smithi039:> sudo pkill -f 'journalctl -f -n 0 -u ceph-51116d5c-77ca-11ef-baf5-efdc52797490@mds.j.service' 2024-09-21T03:59:09.042 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-21T03:59:09.042 INFO:tasks.cephadm.mds.j:Stopped mds.j 2024-09-21T03:59:09.042 INFO:tasks.cephadm.mds.b:Stopping mds.b... 2024-09-21T03:59:09.043 DEBUG:teuthology.orchestra.run.smithi150:> sudo systemctl stop ceph-51116d5c-77ca-11ef-baf5-efdc52797490@mds.b 2024-09-21T03:59:09.340 INFO:journalctl@ceph.mds.b.smithi150.stdout:Sep 21 03:59:09 smithi150 systemd[1]: Stopping Ceph mds.b for 51116d5c-77ca-11ef-baf5-efdc52797490... 2024-09-21T03:59:09.725 INFO:journalctl@ceph.mds.b.smithi150.stdout:Sep 21 03:59:09 smithi150 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-mds-b[63521]: 2024-09-21T03:59:09.337+0000 7fab46cea640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-21T03:59:09.725 INFO:journalctl@ceph.mds.b.smithi150.stdout:Sep 21 03:59:09 smithi150 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-mds-b[63521]: 2024-09-21T03:59:09.337+0000 7fab46cea640 -1 mds.b *** got signal Terminated *** 2024-09-21T03:59:12.315 INFO:journalctl@ceph.mds.b.smithi150.stdout:Sep 21 03:59:12 smithi150 podman[67922]: 2024-09-21 03:59:12.038079121 +0000 UTC m=+2.803705932 container died 5600e78780f50d80188ae5ca31c15591bef32c07ce9c7f217de363a23bbad4ed (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-51116d5c-77ca-11ef-baf5-efdc52797490-mds-b, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-21T03:59:12.665 INFO:journalctl@ceph.mds.b.smithi150.stdout:Sep 21 03:59:12 smithi150 podman[67922]: 2024-09-21 03:59:12.420891855 +0000 UTC m=+3.186518662 container cleanup 5600e78780f50d80188ae5ca31c15591bef32c07ce9c7f217de363a23bbad4ed (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-51116d5c-77ca-11ef-baf5-efdc52797490-mds-b, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS) 2024-09-21T03:59:12.665 INFO:journalctl@ceph.mds.b.smithi150.stdout:Sep 21 03:59:12 smithi150 bash[67922]: ceph-51116d5c-77ca-11ef-baf5-efdc52797490-mds-b 2024-09-21T03:59:12.798 DEBUG:teuthology.orchestra.run.smithi150:> sudo pkill -f 'journalctl -f -n 0 -u ceph-51116d5c-77ca-11ef-baf5-efdc52797490@mds.b.service' 2024-09-21T03:59:12.835 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-21T03:59:12.835 INFO:tasks.cephadm.mds.b:Stopped mds.b 2024-09-21T03:59:12.835 INFO:tasks.cephadm.mds.e:Stopping mds.e... 2024-09-21T03:59:12.835 DEBUG:teuthology.orchestra.run.smithi150:> sudo systemctl stop ceph-51116d5c-77ca-11ef-baf5-efdc52797490@mds.e 2024-09-21T03:59:13.225 INFO:journalctl@ceph.mds.e.smithi150.stdout:Sep 21 03:59:12 smithi150 systemd[1]: Stopping Ceph mds.e for 51116d5c-77ca-11ef-baf5-efdc52797490... 2024-09-21T03:59:13.225 INFO:journalctl@ceph.mds.e.smithi150.stdout:Sep 21 03:59:13 smithi150 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-mds-e[62978]: 2024-09-21T03:59:13.157+0000 7f1418c41640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.e -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-21T03:59:13.225 INFO:journalctl@ceph.mds.e.smithi150.stdout:Sep 21 03:59:13 smithi150 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-mds-e[62978]: 2024-09-21T03:59:13.157+0000 7f1418c41640 -1 mds.e *** got signal Terminated *** 2024-09-21T03:59:16.368 INFO:journalctl@ceph.mds.e.smithi150.stdout:Sep 21 03:59:16 smithi150 podman[68042]: 2024-09-21 03:59:16.038466853 +0000 UTC m=+2.982790346 container died fd7950219c69717ebeae78a39a2b761ade3ab5de72a7bfbe58e4d8146ec1689c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-51116d5c-77ca-11ef-baf5-efdc52797490-mds-e, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23) 2024-09-21T03:59:16.661 INFO:journalctl@ceph.mds.e.smithi150.stdout:Sep 21 03:59:16 smithi150 podman[68042]: 2024-09-21 03:59:16.46627516 +0000 UTC m=+3.410598659 container cleanup fd7950219c69717ebeae78a39a2b761ade3ab5de72a7bfbe58e4d8146ec1689c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-51116d5c-77ca-11ef-baf5-efdc52797490-mds-e, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, ceph=True) 2024-09-21T03:59:16.661 INFO:journalctl@ceph.mds.e.smithi150.stdout:Sep 21 03:59:16 smithi150 bash[68042]: ceph-51116d5c-77ca-11ef-baf5-efdc52797490-mds-e 2024-09-21T03:59:16.903 DEBUG:teuthology.orchestra.run.smithi150:> sudo pkill -f 'journalctl -f -n 0 -u ceph-51116d5c-77ca-11ef-baf5-efdc52797490@mds.e.service' 2024-09-21T03:59:16.924 INFO:journalctl@ceph.mds.e.smithi150.stdout:Sep 21 03:59:16 smithi150 podman[68055]: 2024-09-21 03:59:16.659144074 +0000 UTC m=+0.616902869 container remove fd7950219c69717ebeae78a39a2b761ade3ab5de72a7bfbe58e4d8146ec1689c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-51116d5c-77ca-11ef-baf5-efdc52797490-mds-e, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.license=GPLv2, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221) 2024-09-21T03:59:16.924 INFO:journalctl@ceph.mds.e.smithi150.stdout:Sep 21 03:59:16 smithi150 systemd[1]: ceph-51116d5c-77ca-11ef-baf5-efdc52797490@mds.e.service: Deactivated successfully. 2024-09-21T03:59:16.925 INFO:journalctl@ceph.mds.e.smithi150.stdout:Sep 21 03:59:16 smithi150 systemd[1]: Stopped Ceph mds.e for 51116d5c-77ca-11ef-baf5-efdc52797490. 2024-09-21T03:59:16.925 INFO:journalctl@ceph.mds.e.smithi150.stdout:Sep 21 03:59:16 smithi150 systemd[1]: ceph-51116d5c-77ca-11ef-baf5-efdc52797490@mds.e.service: Consumed 2.211s CPU time. 2024-09-21T03:59:16.936 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-21T03:59:16.936 INFO:tasks.cephadm.mds.e:Stopped mds.e 2024-09-21T03:59:16.936 INFO:tasks.cephadm.mds.h:Stopping mds.h... 2024-09-21T03:59:16.936 DEBUG:teuthology.orchestra.run.smithi150:> sudo systemctl stop ceph-51116d5c-77ca-11ef-baf5-efdc52797490@mds.h 2024-09-21T03:59:17.225 INFO:journalctl@ceph.mds.h.smithi150.stdout:Sep 21 03:59:17 smithi150 systemd[1]: Stopping Ceph mds.h for 51116d5c-77ca-11ef-baf5-efdc52797490... 2024-09-21T03:59:17.725 INFO:journalctl@ceph.mds.h.smithi150.stdout:Sep 21 03:59:17 smithi150 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-mds-h[63250]: 2024-09-21T03:59:17.261+0000 7f7025282640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.h -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-21T03:59:17.725 INFO:journalctl@ceph.mds.h.smithi150.stdout:Sep 21 03:59:17 smithi150 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-mds-h[63250]: 2024-09-21T03:59:17.261+0000 7f7025282640 -1 mds.h *** got signal Terminated *** 2024-09-21T03:59:20.414 INFO:journalctl@ceph.mds.h.smithi150.stdout:Sep 21 03:59:20 smithi150 podman[68161]: 2024-09-21 03:59:20.035090113 +0000 UTC m=+2.885593002 container died 28ae457df36beee65b54e204dea8e1d5e011aafa663b12e7ef4342f4caefbefb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-51116d5c-77ca-11ef-baf5-efdc52797490-mds-h, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, ceph=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux ) 2024-09-21T03:59:20.726 INFO:journalctl@ceph.mds.h.smithi150.stdout:Sep 21 03:59:20 smithi150 podman[68161]: 2024-09-21 03:59:20.537123422 +0000 UTC m=+3.387626310 container cleanup 28ae457df36beee65b54e204dea8e1d5e011aafa663b12e7ef4342f4caefbefb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-51116d5c-77ca-11ef-baf5-efdc52797490-mds-h, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_CLEAN=True, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, io.buildah.version=1.37.2, GIT_BRANCH=HEAD) 2024-09-21T03:59:20.726 INFO:journalctl@ceph.mds.h.smithi150.stdout:Sep 21 03:59:20 smithi150 bash[68161]: ceph-51116d5c-77ca-11ef-baf5-efdc52797490-mds-h 2024-09-21T03:59:20.961 DEBUG:teuthology.orchestra.run.smithi150:> sudo pkill -f 'journalctl -f -n 0 -u ceph-51116d5c-77ca-11ef-baf5-efdc52797490@mds.h.service' 2024-09-21T03:59:20.995 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-21T03:59:20.996 INFO:tasks.cephadm.mds.h:Stopped mds.h 2024-09-21T03:59:20.996 INFO:tasks.cephadm.mds.k:Stopping mds.k... 2024-09-21T03:59:20.996 DEBUG:teuthology.orchestra.run.smithi150:> sudo systemctl stop ceph-51116d5c-77ca-11ef-baf5-efdc52797490@mds.k 2024-09-21T03:59:21.410 INFO:journalctl@ceph.mds.k.smithi150.stdout:Sep 21 03:59:21 smithi150 systemd[1]: Stopping Ceph mds.k for 51116d5c-77ca-11ef-baf5-efdc52797490... 2024-09-21T03:59:21.725 INFO:journalctl@ceph.mds.k.smithi150.stdout:Sep 21 03:59:21 smithi150 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-mds-k[62698]: 2024-09-21T03:59:21.407+0000 7fe0c1257640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.k -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-21T03:59:21.725 INFO:journalctl@ceph.mds.k.smithi150.stdout:Sep 21 03:59:21 smithi150 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-mds-k[62698]: 2024-09-21T03:59:21.407+0000 7fe0c1257640 -1 mds.k *** got signal Terminated *** 2024-09-21T03:59:24.446 INFO:journalctl@ceph.mds.k.smithi150.stdout:Sep 21 03:59:24 smithi150 podman[68295]: 2024-09-21 03:59:24.039926554 +0000 UTC m=+2.833484914 container died f4cfa3dbe0d41529e139e9e981c737f90ad26426d3bd1b97d7133136c0966574 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-51116d5c-77ca-11ef-baf5-efdc52797490-mds-k, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, ceph=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux ) 2024-09-21T03:59:24.725 INFO:journalctl@ceph.mds.k.smithi150.stdout:Sep 21 03:59:24 smithi150 podman[68295]: 2024-09-21 03:59:24.532683922 +0000 UTC m=+3.326242282 container cleanup f4cfa3dbe0d41529e139e9e981c737f90ad26426d3bd1b97d7133136c0966574 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-51116d5c-77ca-11ef-baf5-efdc52797490-mds-k, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.build-date=20240731, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, maintainer=Guillaume Abrioux , GIT_CLEAN=True) 2024-09-21T03:59:24.725 INFO:journalctl@ceph.mds.k.smithi150.stdout:Sep 21 03:59:24 smithi150 bash[68295]: ceph-51116d5c-77ca-11ef-baf5-efdc52797490-mds-k 2024-09-21T03:59:24.884 DEBUG:teuthology.orchestra.run.smithi150:> sudo pkill -f 'journalctl -f -n 0 -u ceph-51116d5c-77ca-11ef-baf5-efdc52797490@mds.k.service' 2024-09-21T03:59:24.917 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-21T03:59:24.917 INFO:tasks.cephadm.mds.k:Stopped mds.k 2024-09-21T03:59:24.917 INFO:tasks.cephadm.mds.c:Stopping mds.c... 2024-09-21T03:59:24.918 DEBUG:teuthology.orchestra.run.smithi161:> sudo systemctl stop ceph-51116d5c-77ca-11ef-baf5-efdc52797490@mds.c 2024-09-21T03:59:25.218 INFO:journalctl@ceph.mds.c.smithi161.stdout:Sep 21 03:59:24 smithi161 systemd[1]: Stopping Ceph mds.c for 51116d5c-77ca-11ef-baf5-efdc52797490... 2024-09-21T03:59:25.480 INFO:journalctl@ceph.mds.c.smithi161.stdout:Sep 21 03:59:25 smithi161 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-mds-c[63630]: 2024-09-21T03:59:25.227+0000 7f0a0d09c640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-21T03:59:25.480 INFO:journalctl@ceph.mds.c.smithi161.stdout:Sep 21 03:59:25 smithi161 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-mds-c[63630]: 2024-09-21T03:59:25.227+0000 7f0a0d09c640 -1 mds.c *** got signal Terminated *** 2024-09-21T03:59:28.338 INFO:journalctl@ceph.mds.c.smithi161.stdout:Sep 21 03:59:28 smithi161 podman[68376]: 2024-09-21 03:59:28.040414877 +0000 UTC m=+2.918846686 container died 21811c0fe34e1544f194ab49cf7af9c7e2f57588432a4ebf98e4cf9e6a19cfb7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-51116d5c-77ca-11ef-baf5-efdc52797490-mds-c, CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, org.label-schema.vendor=CentOS) 2024-09-21T03:59:28.680 INFO:journalctl@ceph.mds.c.smithi161.stdout:Sep 21 03:59:28 smithi161 podman[68376]: 2024-09-21 03:59:28.427815205 +0000 UTC m=+3.306247023 container cleanup 21811c0fe34e1544f194ab49cf7af9c7e2f57588432a4ebf98e4cf9e6a19cfb7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-51116d5c-77ca-11ef-baf5-efdc52797490-mds-c, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, GIT_CLEAN=True, ceph=True, GIT_BRANCH=HEAD) 2024-09-21T03:59:28.680 INFO:journalctl@ceph.mds.c.smithi161.stdout:Sep 21 03:59:28 smithi161 bash[68376]: ceph-51116d5c-77ca-11ef-baf5-efdc52797490-mds-c 2024-09-21T03:59:28.860 DEBUG:teuthology.orchestra.run.smithi161:> sudo pkill -f 'journalctl -f -n 0 -u ceph-51116d5c-77ca-11ef-baf5-efdc52797490@mds.c.service' 2024-09-21T03:59:28.896 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-21T03:59:28.897 INFO:tasks.cephadm.mds.c:Stopped mds.c 2024-09-21T03:59:28.897 INFO:tasks.cephadm.mds.f:Stopping mds.f... 2024-09-21T03:59:28.897 DEBUG:teuthology.orchestra.run.smithi161:> sudo systemctl stop ceph-51116d5c-77ca-11ef-baf5-efdc52797490@mds.f 2024-09-21T03:59:29.218 INFO:journalctl@ceph.mds.f.smithi161.stdout:Sep 21 03:59:28 smithi161 systemd[1]: Stopping Ceph mds.f for 51116d5c-77ca-11ef-baf5-efdc52797490... 2024-09-21T03:59:29.718 INFO:journalctl@ceph.mds.f.smithi161.stdout:Sep 21 03:59:29 smithi161 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-mds-f[63361]: 2024-09-21T03:59:29.223+0000 7f75df219640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.f -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-21T03:59:29.718 INFO:journalctl@ceph.mds.f.smithi161.stdout:Sep 21 03:59:29 smithi161 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-mds-f[63361]: 2024-09-21T03:59:29.223+0000 7f75df219640 -1 mds.f *** got signal Terminated *** 2024-09-21T03:59:32.325 INFO:journalctl@ceph.mds.f.smithi161.stdout:Sep 21 03:59:32 smithi161 podman[68508]: 2024-09-21 03:59:32.047894582 +0000 UTC m=+2.931731314 container died eb6e976e5fd4c4e61531a6b24327e749ec80e2314449b00c7baf4e7e2cd94b27 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-51116d5c-77ca-11ef-baf5-efdc52797490-mds-f, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, ceph=True, org.label-schema.schema-version=1.0, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_CLEAN=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, io.buildah.version=1.37.2) 2024-09-21T03:59:32.667 INFO:journalctl@ceph.mds.f.smithi161.stdout:Sep 21 03:59:32 smithi161 podman[68508]: 2024-09-21 03:59:32.422984178 +0000 UTC m=+3.306820909 container cleanup eb6e976e5fd4c4e61531a6b24327e749ec80e2314449b00c7baf4e7e2cd94b27 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-51116d5c-77ca-11ef-baf5-efdc52797490-mds-f, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.build-date=20240731, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0) 2024-09-21T03:59:32.667 INFO:journalctl@ceph.mds.f.smithi161.stdout:Sep 21 03:59:32 smithi161 bash[68508]: ceph-51116d5c-77ca-11ef-baf5-efdc52797490-mds-f 2024-09-21T03:59:32.809 DEBUG:teuthology.orchestra.run.smithi161:> sudo pkill -f 'journalctl -f -n 0 -u ceph-51116d5c-77ca-11ef-baf5-efdc52797490@mds.f.service' 2024-09-21T03:59:32.842 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-21T03:59:32.842 INFO:tasks.cephadm.mds.f:Stopped mds.f 2024-09-21T03:59:32.842 INFO:tasks.cephadm.mds.i:Stopping mds.i... 2024-09-21T03:59:32.842 DEBUG:teuthology.orchestra.run.smithi161:> sudo systemctl stop ceph-51116d5c-77ca-11ef-baf5-efdc52797490@mds.i 2024-09-21T03:59:33.217 INFO:journalctl@ceph.mds.i.smithi161.stdout:Sep 21 03:59:32 smithi161 systemd[1]: Stopping Ceph mds.i for 51116d5c-77ca-11ef-baf5-efdc52797490... 2024-09-21T03:59:33.718 INFO:journalctl@ceph.mds.i.smithi161.stdout:Sep 21 03:59:33 smithi161 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-mds-i[63897]: 2024-09-21T03:59:33.284+0000 7f6a42dbf640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.i -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-21T03:59:33.718 INFO:journalctl@ceph.mds.i.smithi161.stdout:Sep 21 03:59:33 smithi161 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-mds-i[63897]: 2024-09-21T03:59:33.284+0000 7f6a42dbf640 -1 mds.i *** got signal Terminated *** 2024-09-21T03:59:36.218 INFO:journalctl@ceph.mds.i.smithi161.stdout:Sep 21 03:59:36 smithi161 podman[68630]: 2024-09-21 03:59:36.041164319 +0000 UTC m=+2.987378178 container died 97ce66e526b24938fa3e9c132e2f84bc247cb2711e65c9e16e0dbed43fc833a8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-51116d5c-77ca-11ef-baf5-efdc52797490-mds-i, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, ceph=True, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-21T03:59:36.687 INFO:journalctl@ceph.mds.i.smithi161.stdout:Sep 21 03:59:36 smithi161 podman[68630]: 2024-09-21 03:59:36.401258104 +0000 UTC m=+3.347471974 container cleanup 97ce66e526b24938fa3e9c132e2f84bc247cb2711e65c9e16e0dbed43fc833a8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-51116d5c-77ca-11ef-baf5-efdc52797490-mds-i, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, ceph=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2) 2024-09-21T03:59:36.687 INFO:journalctl@ceph.mds.i.smithi161.stdout:Sep 21 03:59:36 smithi161 bash[68630]: ceph-51116d5c-77ca-11ef-baf5-efdc52797490-mds-i 2024-09-21T03:59:36.838 DEBUG:teuthology.orchestra.run.smithi161:> sudo pkill -f 'journalctl -f -n 0 -u ceph-51116d5c-77ca-11ef-baf5-efdc52797490@mds.i.service' 2024-09-21T03:59:36.870 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-21T03:59:36.871 INFO:tasks.cephadm.mds.i:Stopped mds.i 2024-09-21T03:59:36.871 INFO:tasks.cephadm.mds.l:Stopping mds.l... 2024-09-21T03:59:36.871 DEBUG:teuthology.orchestra.run.smithi161:> sudo systemctl stop ceph-51116d5c-77ca-11ef-baf5-efdc52797490@mds.l 2024-09-21T03:59:37.219 INFO:journalctl@ceph.mds.l.smithi161.stdout:Sep 21 03:59:36 smithi161 systemd[1]: Stopping Ceph mds.l for 51116d5c-77ca-11ef-baf5-efdc52797490... 2024-09-21T03:59:37.219 INFO:journalctl@ceph.mds.l.smithi161.stdout:Sep 21 03:59:37 smithi161 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-mds-l[63094]: 2024-09-21T03:59:37.179+0000 7f375a1c7640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.l -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-21T03:59:37.219 INFO:journalctl@ceph.mds.l.smithi161.stdout:Sep 21 03:59:37 smithi161 ceph-51116d5c-77ca-11ef-baf5-efdc52797490-mds-l[63094]: 2024-09-21T03:59:37.179+0000 7f375a1c7640 -1 mds.l *** got signal Terminated *** 2024-09-21T03:59:40.365 INFO:journalctl@ceph.mds.l.smithi161.stdout:Sep 21 03:59:40 smithi161 podman[68752]: 2024-09-21 03:59:40.042819241 +0000 UTC m=+2.958934943 container died 988b75038bf2f4bad8a0510d1676e5ba920c14b4f5483640bdd32b9c274ff8c5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-51116d5c-77ca-11ef-baf5-efdc52797490-mds-l, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, io.buildah.version=1.37.2, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD) 2024-09-21T03:59:40.365 INFO:journalctl@ceph.mds.l.smithi161.stdout:Sep 21 03:59:40 smithi161 podman[68752]: 2024-09-21 03:59:40.363119795 +0000 UTC m=+3.279235543 container cleanup 988b75038bf2f4bad8a0510d1676e5ba920c14b4f5483640bdd32b9c274ff8c5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-51116d5c-77ca-11ef-baf5-efdc52797490-mds-l, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.license=GPLv2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, ceph=True, GIT_CLEAN=True, io.buildah.version=1.37.2) 2024-09-21T03:59:40.668 INFO:journalctl@ceph.mds.l.smithi161.stdout:Sep 21 03:59:40 smithi161 bash[68752]: ceph-51116d5c-77ca-11ef-baf5-efdc52797490-mds-l 2024-09-21T03:59:40.668 INFO:journalctl@ceph.mds.l.smithi161.stdout:Sep 21 03:59:40 smithi161 podman[68767]: 2024-09-21 03:59:40.564115565 +0000 UTC m=+0.515763472 container remove 988b75038bf2f4bad8a0510d1676e5ba920c14b4f5483640bdd32b9c274ff8c5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-51116d5c-77ca-11ef-baf5-efdc52797490-mds-l, org.label-schema.build-date=20240731, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, ceph=True, maintainer=Guillaume Abrioux , RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_CLEAN=True, io.buildah.version=1.37.2) 2024-09-21T03:59:40.680 DEBUG:teuthology.orchestra.run.smithi161:> sudo pkill -f 'journalctl -f -n 0 -u ceph-51116d5c-77ca-11ef-baf5-efdc52797490@mds.l.service' 2024-09-21T03:59:40.711 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-21T03:59:40.711 INFO:tasks.cephadm.mds.l:Stopped mds.l 2024-09-21T03:59:40.712 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 51116d5c-77ca-11ef-baf5-efdc52797490 --force --keep-logs 2024-09-21T03:59:40.925 INFO:teuthology.orchestra.run.smithi039.stdout:Deleting cluster with fsid: 51116d5c-77ca-11ef-baf5-efdc52797490 2024-09-21T03:59:43.510 DEBUG:teuthology.orchestra.run.smithi150:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 51116d5c-77ca-11ef-baf5-efdc52797490 --force --keep-logs 2024-09-21T03:59:43.714 INFO:teuthology.orchestra.run.smithi150.stdout:Deleting cluster with fsid: 51116d5c-77ca-11ef-baf5-efdc52797490 2024-09-21T03:59:46.116 DEBUG:teuthology.orchestra.run.smithi161:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 51116d5c-77ca-11ef-baf5-efdc52797490 --force --keep-logs 2024-09-21T03:59:46.317 INFO:teuthology.orchestra.run.smithi161.stdout:Deleting cluster with fsid: 51116d5c-77ca-11ef-baf5-efdc52797490 2024-09-21T03:59:48.766 DEBUG:teuthology.orchestra.run.smithi039:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-21T03:59:48.801 DEBUG:teuthology.orchestra.run.smithi150:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-21T03:59:48.828 DEBUG:teuthology.orchestra.run.smithi161:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-21T03:59:48.854 INFO:tasks.cephadm:Archiving crash dumps... 2024-09-21T03:59:48.855 DEBUG:teuthology.misc:Transferring archived files from smithi039:/var/lib/ceph/51116d5c-77ca-11ef-baf5-efdc52797490/crash to /home/teuthworker/archive/pdonnell-2024-09-20_23:48:22-fs-wip-pdonnell-testing-20240920.212106-debug-distro-default-smithi/7913471/remote/smithi039/crash 2024-09-21T03:59:48.856 DEBUG:teuthology.orchestra.run.smithi039:> sudo tar c -f - -C /var/lib/ceph/51116d5c-77ca-11ef-baf5-efdc52797490/crash -- . 2024-09-21T03:59:48.925 INFO:teuthology.orchestra.run.smithi039.stderr:tar: /var/lib/ceph/51116d5c-77ca-11ef-baf5-efdc52797490/crash: Cannot open: No such file or directory 2024-09-21T03:59:48.925 INFO:teuthology.orchestra.run.smithi039.stderr:tar: Error is not recoverable: exiting now 2024-09-21T03:59:48.928 DEBUG:teuthology.misc:Transferring archived files from smithi150:/var/lib/ceph/51116d5c-77ca-11ef-baf5-efdc52797490/crash to /home/teuthworker/archive/pdonnell-2024-09-20_23:48:22-fs-wip-pdonnell-testing-20240920.212106-debug-distro-default-smithi/7913471/remote/smithi150/crash 2024-09-21T03:59:48.928 DEBUG:teuthology.orchestra.run.smithi150:> sudo tar c -f - -C /var/lib/ceph/51116d5c-77ca-11ef-baf5-efdc52797490/crash -- . 2024-09-21T03:59:48.962 INFO:teuthology.orchestra.run.smithi150.stderr:tar: /var/lib/ceph/51116d5c-77ca-11ef-baf5-efdc52797490/crash: Cannot open: No such file or directory 2024-09-21T03:59:48.962 INFO:teuthology.orchestra.run.smithi150.stderr:tar: Error is not recoverable: exiting now 2024-09-21T03:59:48.963 DEBUG:teuthology.misc:Transferring archived files from smithi161:/var/lib/ceph/51116d5c-77ca-11ef-baf5-efdc52797490/crash to /home/teuthworker/archive/pdonnell-2024-09-20_23:48:22-fs-wip-pdonnell-testing-20240920.212106-debug-distro-default-smithi/7913471/remote/smithi161/crash 2024-09-21T03:59:48.964 DEBUG:teuthology.orchestra.run.smithi161:> sudo tar c -f - -C /var/lib/ceph/51116d5c-77ca-11ef-baf5-efdc52797490/crash -- . 2024-09-21T03:59:49.003 INFO:teuthology.orchestra.run.smithi161.stderr:tar: /var/lib/ceph/51116d5c-77ca-11ef-baf5-efdc52797490/crash: Cannot open: No such file or directory 2024-09-21T03:59:49.003 INFO:teuthology.orchestra.run.smithi161.stderr:tar: Error is not recoverable: exiting now 2024-09-21T03:59:49.004 INFO:tasks.cephadm:Checking cluster log for badness... 2024-09-21T03:59:49.004 DEBUG:teuthology.orchestra.run.smithi039:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/51116d5c-77ca-11ef-baf5-efdc52797490/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v 'fs.*is degraded' | egrep -v 'filesystem is degraded' | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v 'filesystem is offline' | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v 'insufficient standby MDS daemons available' | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v 'online, but wants' | egrep -v 'filesystem is online with fewer MDS than max_mds' | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'do not have an application enabled' | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v PG_AVAILABILITY | egrep -v PG_DEGRADED | egrep -v 'Reduced data availability' | egrep -v 'Degraded data redundancy' | egrep -v 'slow metadata IO' | egrep -v SLOW_OPS | egrep -v 'slow request' | head -n 1 2024-09-21T03:59:49.047 INFO:tasks.cephadm:Compressing logs... 2024-09-21T03:59:49.048 DEBUG:teuthology.orchestra.run.smithi039:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-09-21T03:59:49.090 DEBUG:teuthology.orchestra.run.smithi150:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-09-21T03:59:49.092 DEBUG:teuthology.orchestra.run.smithi161:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-09-21T03:59:49.115 INFO:teuthology.orchestra.run.smithi150.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/51116d5c-77ca-11ef-baf5-efdc52797490/ceph-osd.10.log 2024-09-21T03:59:49.115 INFO:teuthology.orchestra.run.smithi150.stderr:: No such file or directory 2024-09-21T03:59:49.115 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/51116d5c-77ca-11ef-baf5-efdc52797490/ceph-osd.1.log 2024-09-21T03:59:49.120 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/51116d5c-77ca-11ef-baf5-efdc52797490/ceph-osd.7.log 2024-09-21T03:59:49.120 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/51116d5c-77ca-11ef-baf5-efdc52797490/ceph-osd.10.log: /var/log/ceph/51116d5c-77ca-11ef-baf5-efdc52797490/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/51116d5c-77ca-11ef-baf5-efdc52797490/ceph-volume.log 2024-09-21T03:59:49.120 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/51116d5c-77ca-11ef-baf5-efdc52797490/ceph-osd.7.log: gzip -5 --verbose -- /var/log/ceph/51116d5c-77ca-11ef-baf5-efdc52797490/ceph-mds.b.log 2024-09-21T03:59:49.120 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/51116d5c-77ca-11ef-baf5-efdc52797490/ceph-mds.k.log 2024-09-21T03:59:49.120 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/51116d5c-77ca-11ef-baf5-efdc52797490/ceph-volume.log: gzip/var/log/ceph/51116d5c-77ca-11ef-baf5-efdc52797490/ceph-mds.b.log: -5 --verbose -- /var/log/ceph/51116d5c-77ca-11ef-baf5-efdc52797490/ceph-osd.4.log 2024-09-21T03:59:49.120 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/51116d5c-77ca-11ef-baf5-efdc52797490/ceph-mds.k.log: gzip -5 --verbose -- /var/log/ceph/51116d5c-77ca-11ef-baf5-efdc52797490/ceph.audit.log 2024-09-21T03:59:49.120 INFO:teuthology.orchestra.run.smithi039.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/51116d5c-77ca-11ef-baf5-efdc52797490/ceph-osd.0.log 2024-09-21T03:59:49.120 INFO:teuthology.orchestra.run.smithi039.stderr:: No such file or directory 2024-09-21T03:59:49.120 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/51116d5c-77ca-11ef-baf5-efdc52797490/ceph-osd.9.log 2024-09-21T03:59:49.120 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/51116d5c-77ca-11ef-baf5-efdc52797490/ceph-volume.log 2024-09-21T03:59:49.120 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/51116d5c-77ca-11ef-baf5-efdc52797490/ceph-osd.0.log: /var/log/ceph/51116d5c-77ca-11ef-baf5-efdc52797490/ceph-osd.9.log: gzip -5 --verbose -- /var/log/ceph/51116d5c-77ca-11ef-baf5-efdc52797490/ceph.audit.log 2024-09-21T03:59:49.121 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/51116d5c-77ca-11ef-baf5-efdc52797490/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/51116d5c-77ca-11ef-baf5-efdc52797490/ceph-mds.a.log 2024-09-21T03:59:49.121 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/51116d5c-77ca-11ef-baf5-efdc52797490/ceph.cephadm.log 2024-09-21T03:59:49.121 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/51116d5c-77ca-11ef-baf5-efdc52797490/ceph.audit.log: /var/log/ceph/51116d5c-77ca-11ef-baf5-efdc52797490/ceph-mds.a.log: gzip -5 --verbose -- /var/log/ceph/51116d5c-77ca-11ef-baf5-efdc52797490/ceph-mon.a.log 2024-09-21T03:59:49.121 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/51116d5c-77ca-11ef-baf5-efdc52797490/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/51116d5c-77ca-11ef-baf5-efdc52797490/ceph-mds.g.log 2024-09-21T03:59:49.121 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/51116d5c-77ca-11ef-baf5-efdc52797490/ceph-osd.6.log 2024-09-21T03:59:49.121 INFO:teuthology.orchestra.run.smithi039.stderr: 84.0% -- replaced with /var/log/ceph/51116d5c-77ca-11ef-baf5-efdc52797490/ceph.cephadm.log.gz 2024-09-21T03:59:49.124 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/51116d5c-77ca-11ef-baf5-efdc52797490/ceph-mon.a.log: /var/log/ceph/51116d5c-77ca-11ef-baf5-efdc52797490/ceph-mds.g.log: 90.1% -- replaced with /var/log/ceph/51116d5c-77ca-11ef-baf5-efdc52797490/ceph.audit.log.gz 2024-09-21T03:59:49.124 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/51116d5c-77ca-11ef-baf5-efdc52797490/ceph.log 2024-09-21T03:59:49.124 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/51116d5c-77ca-11ef-baf5-efdc52797490/ceph-osd.6.log: gzip -5 --verbose -- /var/log/ceph/51116d5c-77ca-11ef-baf5-efdc52797490/ceph-mds.d.log 2024-09-21T03:59:49.127 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/51116d5c-77ca-11ef-baf5-efdc52797490/ceph.log: 87.4% -- replaced with /var/log/ceph/51116d5c-77ca-11ef-baf5-efdc52797490/ceph.log.gz 2024-09-21T03:59:49.127 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/51116d5c-77ca-11ef-baf5-efdc52797490/ceph-mgr.x.log 2024-09-21T03:59:49.128 INFO:teuthology.orchestra.run.smithi161.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/51116d5c-77ca-11ef-baf5-efdc52797490/ceph-volume.log 2024-09-21T03:59:49.128 INFO:teuthology.orchestra.run.smithi161.stderr:: No such file or directory 2024-09-21T03:59:49.128 INFO:teuthology.orchestra.run.smithi161.stderr:gzip -5 --verbose -- /var/log/ceph/51116d5c-77ca-11ef-baf5-efdc52797490/ceph.audit.log 2024-09-21T03:59:49.128 INFO:teuthology.orchestra.run.smithi161.stderr:/var/log/ceph/51116d5c-77ca-11ef-baf5-efdc52797490/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/51116d5c-77ca-11ef-baf5-efdc52797490/ceph-mds.i.log 2024-09-21T03:59:49.129 INFO:teuthology.orchestra.run.smithi161.stderr:/var/log/ceph/51116d5c-77ca-11ef-baf5-efdc52797490/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/51116d5c-77ca-11ef-baf5-efdc52797490/ceph-mon.c.log 2024-09-21T03:59:49.129 INFO:teuthology.orchestra.run.smithi161.stderr:/var/log/ceph/51116d5c-77ca-11ef-baf5-efdc52797490/ceph-mds.i.log: gzip -5 --verbose -- /var/log/ceph/51116d5c-77ca-11ef-baf5-efdc52797490/ceph.cephadm.log 2024-09-21T03:59:49.129 INFO:teuthology.orchestra.run.smithi161.stderr:gzip -5 --verbose -- /var/log/ceph/51116d5c-77ca-11ef-baf5-efdc52797490/ceph-osd.11.log 2024-09-21T03:59:49.129 INFO:teuthology.orchestra.run.smithi161.stderr:/var/log/ceph/51116d5c-77ca-11ef-baf5-efdc52797490/ceph-mon.c.log: /var/log/ceph/51116d5c-77ca-11ef-baf5-efdc52797490/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/51116d5c-77ca-11ef-baf5-efdc52797490/ceph-osd.2.log 2024-09-21T03:59:49.129 INFO:teuthology.orchestra.run.smithi161.stderr: 81.2% -- replaced with /var/log/ceph/51116d5c-77ca-11ef-baf5-efdc52797490/ceph.cephadm.log.gz 2024-09-21T03:59:49.129 INFO:teuthology.orchestra.run.smithi161.stderr:/var/log/ceph/51116d5c-77ca-11ef-baf5-efdc52797490/ceph-osd.11.log: gzip -5 --verbose -- /var/log/ceph/51116d5c-77ca-11ef-baf5-efdc52797490/ceph-osd.5.log 2024-09-21T03:59:49.129 INFO:teuthology.orchestra.run.smithi161.stderr:/var/log/ceph/51116d5c-77ca-11ef-baf5-efdc52797490/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/51116d5c-77ca-11ef-baf5-efdc52797490/ceph.log 2024-09-21T03:59:49.129 INFO:teuthology.orchestra.run.smithi161.stderr:/var/log/ceph/51116d5c-77ca-11ef-baf5-efdc52797490/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/51116d5c-77ca-11ef-baf5-efdc52797490/ceph-mds.l.log 2024-09-21T03:59:49.129 INFO:teuthology.orchestra.run.smithi161.stderr: 90.3% -- replaced with /var/log/ceph/51116d5c-77ca-11ef-baf5-efdc52797490/ceph.audit.log.gz 2024-09-21T03:59:49.129 INFO:teuthology.orchestra.run.smithi161.stderr:/var/log/ceph/51116d5c-77ca-11ef-baf5-efdc52797490/ceph.log: 87.4% -- replaced with /var/log/ceph/51116d5c-77ca-11ef-baf5-efdc52797490/ceph.log.gz 2024-09-21T03:59:49.129 INFO:teuthology.orchestra.run.smithi161.stderr:gzip -5 --verbose -- /var/log/ceph/51116d5c-77ca-11ef-baf5-efdc52797490/ceph-osd.8.log 2024-09-21T03:59:49.129 INFO:teuthology.orchestra.run.smithi161.stderr:/var/log/ceph/51116d5c-77ca-11ef-baf5-efdc52797490/ceph-mds.l.log: gzip -5 --verbose -- /var/log/ceph/51116d5c-77ca-11ef-baf5-efdc52797490/ceph-mgr.z.log 2024-09-21T03:59:49.130 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/51116d5c-77ca-11ef-baf5-efdc52797490/ceph-osd.4.log: 90.9% -- replaced with /var/log/ceph/51116d5c-77ca-11ef-baf5-efdc52797490/ceph-volume.log.gz 2024-09-21T03:59:49.131 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/51116d5c-77ca-11ef-baf5-efdc52797490/ceph.cephadm.log 2024-09-21T03:59:49.132 INFO:teuthology.orchestra.run.smithi161.stderr:/var/log/ceph/51116d5c-77ca-11ef-baf5-efdc52797490/ceph-osd.8.log: 90.9% -- replaced with /var/log/ceph/51116d5c-77ca-11ef-baf5-efdc52797490/ceph-volume.log.gz 2024-09-21T03:59:49.132 INFO:teuthology.orchestra.run.smithi161.stderr:gzip -5 --verbose -- /var/log/ceph/51116d5c-77ca-11ef-baf5-efdc52797490/ceph-mds.c.log 2024-09-21T03:59:49.133 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/51116d5c-77ca-11ef-baf5-efdc52797490/ceph-mds.d.log: 90.9% 95.1% -- replaced with /var/log/ceph/51116d5c-77ca-11ef-baf5-efdc52797490/ceph-volume.log.gz -- replaced with /var/log/ceph/51116d5c-77ca-11ef-baf5-efdc52797490/ceph-mds.d.log.gz 2024-09-21T03:59:49.133 INFO:teuthology.orchestra.run.smithi039.stderr: 2024-09-21T03:59:49.133 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/51116d5c-77ca-11ef-baf5-efdc52797490/ceph-mds.j.log 2024-09-21T03:59:49.134 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose --/var/log/ceph/51116d5c-77ca-11ef-baf5-efdc52797490/ceph-mgr.x.log: /var/log/ceph/51116d5c-77ca-11ef-baf5-efdc52797490/ceph-osd.3.log 2024-09-21T03:59:49.135 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/51116d5c-77ca-11ef-baf5-efdc52797490/ceph.audit.log: 90.3% -- replaced with /var/log/ceph/51116d5c-77ca-11ef-baf5-efdc52797490/ceph.audit.log.gz 2024-09-21T03:59:49.136 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/51116d5c-77ca-11ef-baf5-efdc52797490/ceph-mgr.y.log 2024-09-21T03:59:49.136 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/51116d5c-77ca-11ef-baf5-efdc52797490/ceph.cephadm.log: 81.3% -- replaced with /var/log/ceph/51116d5c-77ca-11ef-baf5-efdc52797490/ceph.cephadm.log.gz 2024-09-21T03:59:49.136 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/51116d5c-77ca-11ef-baf5-efdc52797490/ceph.log 2024-09-21T03:59:49.139 INFO:teuthology.orchestra.run.smithi161.stderr:/var/log/ceph/51116d5c-77ca-11ef-baf5-efdc52797490/ceph-mgr.z.log: 92.6% -- replaced with /var/log/ceph/51116d5c-77ca-11ef-baf5-efdc52797490/ceph-mgr.z.log.gz 2024-09-21T03:59:49.139 INFO:teuthology.orchestra.run.smithi161.stderr:gzip -5 --verbose -- /var/log/ceph/51116d5c-77ca-11ef-baf5-efdc52797490/ceph-mds.f.log 2024-09-21T03:59:49.141 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/51116d5c-77ca-11ef-baf5-efdc52797490/ceph-mds.j.log: 95.0% -- replaced with /var/log/ceph/51116d5c-77ca-11ef-baf5-efdc52797490/ceph-mds.j.log.gz 2024-09-21T03:59:49.141 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-09-21T03:59:49.143 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/51116d5c-77ca-11ef-baf5-efdc52797490/ceph-mgr.y.log: 92.5% -- replaced with /var/log/ceph/51116d5c-77ca-11ef-baf5-efdc52797490/ceph-mgr.y.log.gz 2024-09-21T03:59:49.144 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/51116d5c-77ca-11ef-baf5-efdc52797490/ceph-mds.h.log 2024-09-21T03:59:49.146 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/51116d5c-77ca-11ef-baf5-efdc52797490/ceph.log: 87.4% -- replaced with /var/log/ceph/51116d5c-77ca-11ef-baf5-efdc52797490/ceph.log.gz 2024-09-21T03:59:49.148 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/51116d5c-77ca-11ef-baf5-efdc52797490/ceph-mon.b.log 2024-09-21T03:59:49.149 INFO:teuthology.orchestra.run.smithi161.stderr:/var/log/ceph/51116d5c-77ca-11ef-baf5-efdc52797490/ceph-mds.c.log: gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-09-21T03:59:49.156 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/51116d5c-77ca-11ef-baf5-efdc52797490/ceph-mds.h.log: gzip -5 --verbose -- /var/log/ceph/51116d5c-77ca-11ef-baf5-efdc52797490/ceph-mds.e.log 2024-09-21T03:59:49.168 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/51116d5c-77ca-11ef-baf5-efdc52797490/ceph-mon.b.log: gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-09-21T03:59:49.173 INFO:teuthology.orchestra.run.smithi161.stderr:/var/log/ceph/51116d5c-77ca-11ef-baf5-efdc52797490/ceph-mds.f.log: /var/log/ceph/cephadm.log: 92.2% -- replaced with /var/log/ceph/cephadm.log.gz 2024-09-21T03:59:49.174 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/51116d5c-77ca-11ef-baf5-efdc52797490/ceph-osd.3.log: /var/log/ceph/cephadm.log: 94.5% -- replaced with /var/log/ceph/cephadm.log.gz 2024-09-21T03:59:49.189 INFO:teuthology.orchestra.run.smithi161.stderr: 93.2% -- replaced with /var/log/ceph/51116d5c-77ca-11ef-baf5-efdc52797490/ceph-mds.i.log.gz 2024-09-21T03:59:49.191 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/51116d5c-77ca-11ef-baf5-efdc52797490/ceph-mds.e.log: /var/log/ceph/cephadm.log: 91.8% -- replaced with /var/log/ceph/cephadm.log.gz 2024-09-21T03:59:49.193 INFO:teuthology.orchestra.run.smithi039.stderr: 93.1% -- replaced with /var/log/ceph/51116d5c-77ca-11ef-baf5-efdc52797490/ceph-mds.a.log.gz 2024-09-21T03:59:49.201 INFO:teuthology.orchestra.run.smithi039.stderr: 94.3% -- replaced with /var/log/ceph/51116d5c-77ca-11ef-baf5-efdc52797490/ceph-mds.g.log.gz 2024-09-21T03:59:49.202 INFO:teuthology.orchestra.run.smithi150.stderr: 94.4% -- replaced with /var/log/ceph/51116d5c-77ca-11ef-baf5-efdc52797490/ceph-mds.k.log.gz 2024-09-21T03:59:49.207 INFO:teuthology.orchestra.run.smithi161.stderr: 94.3% -- replaced with /var/log/ceph/51116d5c-77ca-11ef-baf5-efdc52797490/ceph-mds.l.log.gz 2024-09-21T03:59:49.215 INFO:teuthology.orchestra.run.smithi161.stderr: 93.1% -- replaced with /var/log/ceph/51116d5c-77ca-11ef-baf5-efdc52797490/ceph-mds.c.log.gz 2024-09-21T03:59:49.246 INFO:teuthology.orchestra.run.smithi150.stderr: 93.0% -- replaced with /var/log/ceph/51116d5c-77ca-11ef-baf5-efdc52797490/ceph-mds.h.log.gz 2024-09-21T03:59:49.267 INFO:teuthology.orchestra.run.smithi150.stderr: 94.5% -- replaced with /var/log/ceph/51116d5c-77ca-11ef-baf5-efdc52797490/ceph-mds.e.log.gz 2024-09-21T03:59:49.349 INFO:teuthology.orchestra.run.smithi161.stderr: 96.0% -- replaced with /var/log/ceph/51116d5c-77ca-11ef-baf5-efdc52797490/ceph-mds.f.log.gz 2024-09-21T03:59:49.480 INFO:teuthology.orchestra.run.smithi161.stderr: 91.6% -- replaced with /var/log/ceph/51116d5c-77ca-11ef-baf5-efdc52797490/ceph-mon.c.log.gz 2024-09-21T03:59:49.502 INFO:teuthology.orchestra.run.smithi039.stderr: 90.4% -- replaced with /var/log/ceph/51116d5c-77ca-11ef-baf5-efdc52797490/ceph-mgr.x.log.gz 2024-09-21T03:59:49.674 INFO:teuthology.orchestra.run.smithi150.stderr: 91.7% -- replaced with /var/log/ceph/51116d5c-77ca-11ef-baf5-efdc52797490/ceph-mon.b.log.gz 2024-09-21T03:59:49.872 INFO:teuthology.orchestra.run.smithi161.stderr: 93.3% -- replaced with /var/log/ceph/51116d5c-77ca-11ef-baf5-efdc52797490/ceph-osd.11.log.gz 2024-09-21T03:59:49.972 INFO:teuthology.orchestra.run.smithi039.stderr: 93.0% -- replaced with /var/log/ceph/51116d5c-77ca-11ef-baf5-efdc52797490/ceph-osd.6.log.gz 2024-09-21T03:59:49.995 INFO:teuthology.orchestra.run.smithi039.stderr: 93.0% -- replaced with /var/log/ceph/51116d5c-77ca-11ef-baf5-efdc52797490/ceph-osd.9.log.gz 2024-09-21T03:59:50.056 INFO:teuthology.orchestra.run.smithi039.stderr: 93.3% -- replaced with /var/log/ceph/51116d5c-77ca-11ef-baf5-efdc52797490/ceph-osd.0.log.gz 2024-09-21T03:59:50.060 INFO:teuthology.orchestra.run.smithi161.stderr: 93.1% -- replaced with /var/log/ceph/51116d5c-77ca-11ef-baf5-efdc52797490/ceph-osd.8.log.gz 2024-09-21T03:59:50.107 INFO:teuthology.orchestra.run.smithi150.stderr: 93.0% -- replaced with /var/log/ceph/51116d5c-77ca-11ef-baf5-efdc52797490/ceph-osd.4.log.gz 2024-09-21T03:59:50.135 INFO:teuthology.orchestra.run.smithi150.stderr: 93.1% -- replaced with /var/log/ceph/51116d5c-77ca-11ef-baf5-efdc52797490/ceph-osd.1.log.gz 2024-09-21T03:59:50.144 INFO:teuthology.orchestra.run.smithi161.stderr: 93.5% -- replaced with /var/log/ceph/51116d5c-77ca-11ef-baf5-efdc52797490/ceph-osd.2.log.gz 2024-09-21T03:59:50.270 INFO:teuthology.orchestra.run.smithi150.stderr: 93.3% -- replaced with /var/log/ceph/51116d5c-77ca-11ef-baf5-efdc52797490/ceph-osd.7.log.gz 2024-09-21T03:59:50.275 INFO:teuthology.orchestra.run.smithi039.stderr: 93.1% -- replaced with /var/log/ceph/51116d5c-77ca-11ef-baf5-efdc52797490/ceph-osd.3.log.gz 2024-09-21T03:59:50.287 INFO:teuthology.orchestra.run.smithi039.stderr: 90.5% -- replaced with /var/log/ceph/51116d5c-77ca-11ef-baf5-efdc52797490/ceph-mon.a.log.gz 2024-09-21T03:59:50.289 INFO:teuthology.orchestra.run.smithi039.stderr: 2024-09-21T03:59:50.289 INFO:teuthology.orchestra.run.smithi039.stderr:real 0m1.185s 2024-09-21T03:59:50.289 INFO:teuthology.orchestra.run.smithi039.stderr:user 0m5.342s 2024-09-21T03:59:50.289 INFO:teuthology.orchestra.run.smithi039.stderr:sys 0m0.215s 2024-09-21T03:59:50.533 INFO:teuthology.orchestra.run.smithi161.stderr: 93.7% -- replaced with /var/log/ceph/51116d5c-77ca-11ef-baf5-efdc52797490/ceph-osd.5.log.gz 2024-09-21T03:59:50.535 INFO:teuthology.orchestra.run.smithi161.stderr: 2024-09-21T03:59:50.536 INFO:teuthology.orchestra.run.smithi161.stderr:real 0m1.429s 2024-09-21T03:59:50.536 INFO:teuthology.orchestra.run.smithi161.stderr:user 0m4.736s 2024-09-21T03:59:50.536 INFO:teuthology.orchestra.run.smithi161.stderr:sys 0m0.191s 2024-09-21T03:59:50.583 INFO:teuthology.orchestra.run.smithi150.stderr: 93.7% -- replaced with /var/log/ceph/51116d5c-77ca-11ef-baf5-efdc52797490/ceph-osd.10.log.gz 2024-09-21T03:59:50.991 INFO:teuthology.orchestra.run.smithi150.stderr: 94.3% -- replaced with /var/log/ceph/51116d5c-77ca-11ef-baf5-efdc52797490/ceph-mds.b.log.gz 2024-09-21T03:59:50.993 INFO:teuthology.orchestra.run.smithi150.stderr: 2024-09-21T03:59:50.994 INFO:teuthology.orchestra.run.smithi150.stderr:real 0m1.889s 2024-09-21T03:59:50.994 INFO:teuthology.orchestra.run.smithi150.stderr:user 0m6.934s 2024-09-21T03:59:50.994 INFO:teuthology.orchestra.run.smithi150.stderr:sys 0m0.283s 2024-09-21T03:59:50.994 INFO:tasks.cephadm:Archiving logs... 2024-09-21T03:59:50.994 DEBUG:teuthology.misc:Transferring archived files from smithi039:/var/log/ceph to /home/teuthworker/archive/pdonnell-2024-09-20_23:48:22-fs-wip-pdonnell-testing-20240920.212106-debug-distro-default-smithi/7913471/remote/smithi039/log 2024-09-21T03:59:50.995 DEBUG:teuthology.orchestra.run.smithi039:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-21T03:59:51.691 DEBUG:teuthology.misc:Transferring archived files from smithi150:/var/log/ceph to /home/teuthworker/archive/pdonnell-2024-09-20_23:48:22-fs-wip-pdonnell-testing-20240920.212106-debug-distro-default-smithi/7913471/remote/smithi150/log 2024-09-21T03:59:51.692 DEBUG:teuthology.orchestra.run.smithi150:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-21T03:59:52.528 DEBUG:teuthology.misc:Transferring archived files from smithi161:/var/log/ceph to /home/teuthworker/archive/pdonnell-2024-09-20_23:48:22-fs-wip-pdonnell-testing-20240920.212106-debug-distro-default-smithi/7913471/remote/smithi161/log 2024-09-21T03:59:52.529 DEBUG:teuthology.orchestra.run.smithi161:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-21T03:59:53.250 INFO:tasks.cephadm:Removing cluster... 2024-09-21T03:59:53.251 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 51116d5c-77ca-11ef-baf5-efdc52797490 --force 2024-09-21T03:59:53.465 INFO:teuthology.orchestra.run.smithi039.stdout:Deleting cluster with fsid: 51116d5c-77ca-11ef-baf5-efdc52797490 2024-09-21T03:59:53.888 DEBUG:teuthology.orchestra.run.smithi150:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 51116d5c-77ca-11ef-baf5-efdc52797490 --force 2024-09-21T03:59:54.091 INFO:teuthology.orchestra.run.smithi150.stdout:Deleting cluster with fsid: 51116d5c-77ca-11ef-baf5-efdc52797490 2024-09-21T03:59:54.348 DEBUG:teuthology.orchestra.run.smithi161:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 51116d5c-77ca-11ef-baf5-efdc52797490 --force 2024-09-21T03:59:54.550 INFO:teuthology.orchestra.run.smithi161.stdout:Deleting cluster with fsid: 51116d5c-77ca-11ef-baf5-efdc52797490 2024-09-21T03:59:54.812 INFO:tasks.cephadm:Removing cephadm ... 2024-09-21T03:59:54.812 DEBUG:teuthology.orchestra.run.smithi039:> rm -rf /home/ubuntu/cephtest/cephadm 2024-09-21T03:59:54.832 DEBUG:teuthology.orchestra.run.smithi150:> rm -rf /home/ubuntu/cephtest/cephadm 2024-09-21T03:59:54.849 DEBUG:teuthology.orchestra.run.smithi161:> rm -rf /home/ubuntu/cephtest/cephadm 2024-09-21T03:59:54.865 INFO:tasks.cephadm:Teardown complete 2024-09-21T03:59:54.865 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-09-21T03:59:54.880 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-09-21T03:59:54.881 DEBUG:teuthology.orchestra.run.smithi039:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-21T03:59:54.883 DEBUG:teuthology.orchestra.run.smithi150:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-21T03:59:54.891 DEBUG:teuthology.orchestra.run.smithi161:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-21T03:59:55.001 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-09-21T03:59:55.034 INFO:teuthology.task.clock:Checking final clock skew... 2024-09-21T03:59:55.035 DEBUG:teuthology.orchestra.run.smithi039:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-21T03:59:55.037 DEBUG:teuthology.orchestra.run.smithi150:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-21T03:59:55.040 DEBUG:teuthology.orchestra.run.smithi161:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-21T03:59:55.052 INFO:teuthology.orchestra.run.smithi150.stderr:bash: line 1: ntpq: command not found 2024-09-21T03:59:55.054 INFO:teuthology.orchestra.run.smithi161.stderr:bash: line 1: ntpq: command not found 2024-09-21T03:59:55.055 INFO:teuthology.orchestra.run.smithi039.stderr:bash: line 1: ntpq: command not found 2024-09-21T03:59:55.056 INFO:teuthology.orchestra.run.smithi150.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-21T03:59:55.057 INFO:teuthology.orchestra.run.smithi150.stdout:=============================================================================== 2024-09-21T03:59:55.057 INFO:teuthology.orchestra.run.smithi150.stdout:^* hv01.front.sepia.ceph.com 3 6 377 16 -490us[ -450us] +/- 41ms 2024-09-21T03:59:55.057 INFO:teuthology.orchestra.run.smithi150.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 12 +969us[ +969us] +/- 50ms 2024-09-21T03:59:55.057 INFO:teuthology.orchestra.run.smithi150.stdout:^+ hv03.front.sepia.ceph.com 3 7 377 17 -340us[ -301us] +/- 71ms 2024-09-21T03:59:55.057 INFO:teuthology.orchestra.run.smithi150.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-09-21T03:59:55.059 INFO:teuthology.orchestra.run.smithi161.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-21T03:59:55.059 INFO:teuthology.orchestra.run.smithi161.stdout:=============================================================================== 2024-09-21T03:59:55.059 INFO:teuthology.orchestra.run.smithi161.stdout:^* hv01.front.sepia.ceph.com 3 6 377 14 -495us[ -481us] +/- 41ms 2024-09-21T03:59:55.059 INFO:teuthology.orchestra.run.smithi161.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 16 +956us[ +970us] +/- 50ms 2024-09-21T03:59:55.059 INFO:teuthology.orchestra.run.smithi161.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 13 -365us[ -365us] +/- 71ms 2024-09-21T03:59:55.059 INFO:teuthology.orchestra.run.smithi161.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-09-21T03:59:55.059 INFO:teuthology.orchestra.run.smithi039.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-21T03:59:55.059 INFO:teuthology.orchestra.run.smithi039.stdout:=============================================================================== 2024-09-21T03:59:55.059 INFO:teuthology.orchestra.run.smithi039.stdout:^* hv01.front.sepia.ceph.com 3 6 377 14 -479us[ -482us] +/- 41ms 2024-09-21T03:59:55.060 INFO:teuthology.orchestra.run.smithi039.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 15 +961us[ +958us] +/- 50ms 2024-09-21T03:59:55.060 INFO:teuthology.orchestra.run.smithi039.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 14 -351us[ -354us] +/- 71ms 2024-09-21T03:59:55.060 INFO:teuthology.orchestra.run.smithi039.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-09-21T03:59:55.060 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-09-21T03:59:55.084 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-09-21T03:59:55.085 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-09-21T03:59:55.134 DEBUG:teuthology.orchestra.run.smithi039:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-09-21T03:59:55.225 DEBUG:teuthology.orchestra.run.smithi150:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-09-21T03:59:55.291 DEBUG:teuthology.orchestra.run.smithi161:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-09-21T03:59:55.358 DEBUG:teuthology.orchestra.run.smithi039:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 2024-09-21T03:59:55.389 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T03:59:55.389 DEBUG:teuthology.orchestra.run.smithi150:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 2024-09-21T03:59:55.417 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T03:59:55.417 DEBUG:teuthology.orchestra.run.smithi161:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 2024-09-21T03:59:55.445 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T03:59:55.445 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-09-21T03:59:55.457 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-09-21T03:59:55.493 INFO:teuthology.task.internal:Duration was 1944.864041 seconds 2024-09-21T03:59:55.493 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-09-21T03:59:55.535 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-09-21T03:59:55.535 DEBUG:teuthology.orchestra.run.smithi039:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-21T03:59:55.538 DEBUG:teuthology.orchestra.run.smithi150:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-21T03:59:55.540 DEBUG:teuthology.orchestra.run.smithi161:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-21T03:59:55.577 INFO:teuthology.orchestra.run.smithi150.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-21T03:59:55.578 INFO:teuthology.orchestra.run.smithi161.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-21T03:59:55.578 INFO:teuthology.orchestra.run.smithi039.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-21T03:59:56.017 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-09-21T03:59:56.017 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi039.front.sepia.ceph.com 2024-09-21T03:59:56.017 DEBUG:teuthology.orchestra.run.smithi039:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-21T03:59:56.040 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi150.front.sepia.ceph.com 2024-09-21T03:59:56.041 DEBUG:teuthology.orchestra.run.smithi150:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-21T03:59:56.081 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi161.front.sepia.ceph.com 2024-09-21T03:59:56.082 DEBUG:teuthology.orchestra.run.smithi161:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-21T03:59:56.107 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-09-21T03:59:56.107 DEBUG:teuthology.orchestra.run.smithi039:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-21T03:59:56.109 DEBUG:teuthology.orchestra.run.smithi150:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-21T03:59:56.124 DEBUG:teuthology.orchestra.run.smithi161:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-21T03:59:56.174 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-09-21T03:59:56.174 DEBUG:teuthology.orchestra.run.smithi039:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-21T03:59:56.178 DEBUG:teuthology.orchestra.run.smithi150:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-21T03:59:56.191 DEBUG:teuthology.orchestra.run.smithi161:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-21T03:59:57.567 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-09-21T03:59:57.579 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-09-21T03:59:57.579 DEBUG:teuthology.orchestra.run.smithi039:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-21T03:59:57.606 DEBUG:teuthology.orchestra.run.smithi150:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-21T03:59:57.636 DEBUG:teuthology.orchestra.run.smithi161:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-21T03:59:57.665 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-09-21T03:59:57.678 DEBUG:teuthology.orchestra.run.smithi039:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-21T03:59:57.680 DEBUG:teuthology.orchestra.run.smithi150:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-21T03:59:57.683 DEBUG:teuthology.orchestra.run.smithi161:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-21T03:59:57.702 INFO:teuthology.orchestra.run.smithi039.stdout:kernel.core_pattern = core 2024-09-21T03:59:57.708 INFO:teuthology.orchestra.run.smithi150.stdout:kernel.core_pattern = core 2024-09-21T03:59:57.728 INFO:teuthology.orchestra.run.smithi161.stdout:kernel.core_pattern = core 2024-09-21T03:59:57.761 DEBUG:teuthology.orchestra.run.smithi039:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-21T03:59:57.776 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T03:59:57.776 DEBUG:teuthology.orchestra.run.smithi150:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-21T03:59:57.806 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T03:59:57.807 DEBUG:teuthology.orchestra.run.smithi161:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-21T03:59:57.821 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T03:59:57.822 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-09-21T03:59:57.833 INFO:teuthology.task.internal:Transferring archived files... 2024-09-21T03:59:57.834 DEBUG:teuthology.misc:Transferring archived files from smithi039:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/pdonnell-2024-09-20_23:48:22-fs-wip-pdonnell-testing-20240920.212106-debug-distro-default-smithi/7913471/remote/smithi039 2024-09-21T03:59:57.834 DEBUG:teuthology.orchestra.run.smithi039:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-21T03:59:57.899 DEBUG:teuthology.misc:Transferring archived files from smithi150:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/pdonnell-2024-09-20_23:48:22-fs-wip-pdonnell-testing-20240920.212106-debug-distro-default-smithi/7913471/remote/smithi150 2024-09-21T03:59:57.899 DEBUG:teuthology.orchestra.run.smithi150:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-21T03:59:57.938 DEBUG:teuthology.misc:Transferring archived files from smithi161:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/pdonnell-2024-09-20_23:48:22-fs-wip-pdonnell-testing-20240920.212106-debug-distro-default-smithi/7913471/remote/smithi161 2024-09-21T03:59:57.938 DEBUG:teuthology.orchestra.run.smithi161:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-21T03:59:57.975 INFO:teuthology.task.internal:Removing archive directory... 2024-09-21T03:59:57.975 DEBUG:teuthology.orchestra.run.smithi039:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-21T03:59:57.978 DEBUG:teuthology.orchestra.run.smithi150:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-21T03:59:57.980 DEBUG:teuthology.orchestra.run.smithi161:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-21T03:59:58.024 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-09-21T03:59:58.043 INFO:teuthology.task.internal:Not uploading archives. 2024-09-21T03:59:58.044 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-09-21T03:59:58.102 INFO:teuthology.task.internal:Tidying up after the test... 2024-09-21T03:59:58.103 DEBUG:teuthology.orchestra.run.smithi039:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-21T03:59:58.105 DEBUG:teuthology.orchestra.run.smithi150:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-21T03:59:58.107 DEBUG:teuthology.orchestra.run.smithi161:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-21T03:59:58.118 INFO:teuthology.orchestra.run.smithi039.stdout: 262169 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 21 03:59 /home/ubuntu/cephtest 2024-09-21T03:59:58.121 INFO:teuthology.orchestra.run.smithi150.stdout: 262165 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 21 03:59 /home/ubuntu/cephtest 2024-09-21T03:59:58.122 INFO:teuthology.orchestra.run.smithi161.stdout: 262165 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 21 03:59 /home/ubuntu/cephtest 2024-09-21T03:59:58.123 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-09-21T03:59:58.154 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-09-21T03:59:58.255 INFO:teuthology.run:Summary data: description: fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v1} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} ms_mode/secure wsync/yes} objectstore-ec/bluestore-bitmap omap_limit/10 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/random export-check n/5 replication/always} standby-replay tasks/{0-subvolume/{with-namespace-isolated-and-quota} 1-check-counter 2-scrub/yes 3-snaps/yes 4-flush/no 5-quiesce/no 6-workunit/suites/iozone}} duration: 1944.8640410900116 flavor: default owner: scheduled_pdonnell@teuthology success: true 2024-09-21T03:59:58.255 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-09-21T03:59:58.319 INFO:teuthology.run:pass